├── Chapter_02.ipynb ├── Chapter_03.ipynb ├── Chapter_04.ipynb ├── Chapter_05.ipynb ├── Chapter_06.ipynb ├── Chapter_07.ipynb ├── Chapter_08.ipynb ├── LICENSE └── README.md /Chapter_02.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Predicting categories with K-Nearest Neighbors\n", 8 | "\n", 9 | "**Aim**: The aim of this notebook is to predict if a mobile transaction is fraudulent or not by using the K-NN algorithm with scikit-learn.\n", 10 | "\n", 11 | "## Table of contents\n", 12 | "\n", 13 | "1. Data preparation\n", 14 | "2. Implementing the k-NN algorithm\n", 15 | "3. Fine-tuning parameters using GridsearchCV\n", 16 | "4. Scaling" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "## Package Requirements" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 62, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "import pandas as pd\n", 33 | "import numpy as np\n", 34 | "import matplotlib.pyplot as plt\n", 35 | "from sklearn.preprocessing import LabelEncoder\n", 36 | "from sklearn.preprocessing import OneHotEncoder\n", 37 | "from sklearn.model_selection import train_test_split\n", 38 | "from sklearn.neighbors import KNeighborsClassifier\n", 39 | "from sklearn.model_selection import GridSearchCV\n", 40 | "from sklearn.preprocessing import StandardScaler\n", 41 | "from sklearn.pipeline import Pipeline\n", 42 | "from sklearn.metrics import confusion_matrix\n", 43 | "from sklearn.metrics import classification_report\n", 44 | "from sklearn.metrics import roc_curve\n", 45 | "from sklearn.metrics import roc_auc_score" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "## Data preparation" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 3, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "#Reading in the dataset\n", 62 | "\n", 63 | "df = pd.read_csv('PS_20174392719_1491204439457_log.csv')" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 4, 69 | "metadata": {}, 70 | "outputs": [ 71 | { 72 | "data": { 73 | "text/html": [ 74 | "
\n", 75 | "\n", 88 | "\n", 89 | " \n", 90 | " \n", 91 | " \n", 92 | " \n", 93 | " \n", 94 | " \n", 95 | " \n", 96 | " \n", 97 | " \n", 98 | " \n", 99 | " \n", 100 | " \n", 101 | " \n", 102 | " \n", 103 | " \n", 104 | " \n", 105 | " \n", 106 | " \n", 107 | " \n", 108 | " \n", 109 | " \n", 110 | " \n", 111 | " \n", 112 | " \n", 113 | " \n", 114 | " \n", 115 | " \n", 116 | " \n", 117 | " \n", 118 | " \n", 119 | " \n", 120 | " \n", 121 | " \n", 122 | " \n", 123 | " \n", 124 | " \n", 125 | " \n", 126 | " \n", 127 | " \n", 128 | " \n", 129 | " \n", 130 | " \n", 131 | " \n", 132 | " \n", 133 | " \n", 134 | " \n", 135 | " \n", 136 | " \n", 137 | " \n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | "
steptypeamountnameOrigoldbalanceOrgnewbalanceOrignameDestoldbalanceDestnewbalanceDestisFraudisFlaggedFraud
01PAYMENT9839.64C1231006815170136.0160296.36M19797871550.00.000
11PAYMENT1864.28C166654429521249.019384.72M20442822250.00.000
21TRANSFER181.00C1305486145181.00.00C5532640650.00.010
31CASH_OUT181.00C840083671181.00.00C3899701021182.00.010
41PAYMENT11668.14C204853772041554.029885.86M12307017030.00.000
\n", 178 | "
" 179 | ], 180 | "text/plain": [ 181 | " step type amount nameOrig oldbalanceOrg newbalanceOrig \\\n", 182 | "0 1 PAYMENT 9839.64 C1231006815 170136.0 160296.36 \n", 183 | "1 1 PAYMENT 1864.28 C1666544295 21249.0 19384.72 \n", 184 | "2 1 TRANSFER 181.00 C1305486145 181.0 0.00 \n", 185 | "3 1 CASH_OUT 181.00 C840083671 181.0 0.00 \n", 186 | "4 1 PAYMENT 11668.14 C2048537720 41554.0 29885.86 \n", 187 | "\n", 188 | " nameDest oldbalanceDest newbalanceDest isFraud isFlaggedFraud \n", 189 | "0 M1979787155 0.0 0.0 0 0 \n", 190 | "1 M2044282225 0.0 0.0 0 0 \n", 191 | "2 C553264065 0.0 0.0 1 0 \n", 192 | "3 C38997010 21182.0 0.0 1 0 \n", 193 | "4 M1230701703 0.0 0.0 0 0 " 194 | ] 195 | }, 196 | "execution_count": 4, 197 | "metadata": {}, 198 | "output_type": "execute_result" 199 | } 200 | ], 201 | "source": [ 202 | "#Viewing the data\n", 203 | "\n", 204 | "df.head()" 205 | ] 206 | }, 207 | { 208 | "cell_type": "markdown", 209 | "metadata": {}, 210 | "source": [ 211 | "**Dropping the redundant features**" 212 | ] 213 | }, 214 | { 215 | "cell_type": "code", 216 | "execution_count": 7, 217 | "metadata": {}, 218 | "outputs": [], 219 | "source": [ 220 | "#Dropping the redundant features\n", 221 | "\n", 222 | "df = df.drop(['nameOrig', 'nameDest', 'isFlaggedFraud'], axis = 1)" 223 | ] 224 | }, 225 | { 226 | "cell_type": "code", 227 | "execution_count": 8, 228 | "metadata": {}, 229 | "outputs": [ 230 | { 231 | "name": "stdout", 232 | "output_type": "stream", 233 | "text": [ 234 | "\n", 235 | "RangeIndex: 6362620 entries, 0 to 6362619\n", 236 | "Data columns (total 8 columns):\n", 237 | "step int64\n", 238 | "type object\n", 239 | "amount float64\n", 240 | "oldbalanceOrg float64\n", 241 | "newbalanceOrig float64\n", 242 | "oldbalanceDest float64\n", 243 | "newbalanceDest float64\n", 244 | "isFraud int64\n", 245 | "dtypes: float64(5), int64(2), object(1)\n", 246 | "memory usage: 388.3+ MB\n" 247 | ] 248 | } 249 | ], 250 | "source": [ 251 | "#Inspecting the data\n", 252 | "\n", 253 | "df.info()" 254 | ] 255 | }, 256 | { 257 | "cell_type": "markdown", 258 | "metadata": {}, 259 | "source": [ 260 | "**Reducing the size of the data**" 261 | ] 262 | }, 263 | { 264 | "cell_type": "code", 265 | "execution_count": 10, 266 | "metadata": {}, 267 | "outputs": [], 268 | "source": [ 269 | "#Storing the fraudulent data into a dataframe\n", 270 | "\n", 271 | "df_fraud = df[df['isFraud'] == 1]" 272 | ] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "execution_count": 12, 277 | "metadata": {}, 278 | "outputs": [], 279 | "source": [ 280 | "#Storing the non-fraudulent data into a dataframe \n", 281 | "\n", 282 | "df_nofraud = df[df['isFraud'] == 0]" 283 | ] 284 | }, 285 | { 286 | "cell_type": "code", 287 | "execution_count": 14, 288 | "metadata": {}, 289 | "outputs": [], 290 | "source": [ 291 | "#Storing 12,000 rows of non-fraudulent data\n", 292 | "\n", 293 | "df_nofraud = df_nofraud.head(12000)" 294 | ] 295 | }, 296 | { 297 | "cell_type": "code", 298 | "execution_count": 15, 299 | "metadata": {}, 300 | "outputs": [], 301 | "source": [ 302 | "#Joining both datasets together \n", 303 | "\n", 304 | "df = pd.concat([df_fraud, df_nofraud], axis = 0)" 305 | ] 306 | }, 307 | { 308 | "cell_type": "code", 309 | "execution_count": 16, 310 | "metadata": {}, 311 | "outputs": [ 312 | { 313 | "name": "stdout", 314 | "output_type": "stream", 315 | "text": [ 316 | "\n", 317 | "Int64Index: 20213 entries, 2 to 12071\n", 318 | "Data columns (total 8 columns):\n", 319 | "step 20213 non-null int64\n", 320 | "type 20213 non-null object\n", 321 | "amount 20213 non-null float64\n", 322 | "oldbalanceOrg 20213 non-null float64\n", 323 | "newbalanceOrig 20213 non-null float64\n", 324 | "oldbalanceDest 20213 non-null float64\n", 325 | "newbalanceDest 20213 non-null float64\n", 326 | "isFraud 20213 non-null int64\n", 327 | "dtypes: float64(5), int64(2), object(1)\n", 328 | "memory usage: 1.4+ MB\n" 329 | ] 330 | } 331 | ], 332 | "source": [ 333 | "df.info()" 334 | ] 335 | }, 336 | { 337 | "cell_type": "markdown", 338 | "metadata": {}, 339 | "source": [ 340 | "**Encoding the categorical feature**" 341 | ] 342 | }, 343 | { 344 | "cell_type": "code", 345 | "execution_count": 19, 346 | "metadata": {}, 347 | "outputs": [], 348 | "source": [ 349 | "#Converting the type column to categorical\n", 350 | "\n", 351 | "df['type'] = df['type'].astype('category')" 352 | ] 353 | }, 354 | { 355 | "cell_type": "code", 356 | "execution_count": 21, 357 | "metadata": {}, 358 | "outputs": [], 359 | "source": [ 360 | "#Integer Encoding the 'type' column\n", 361 | "\n", 362 | "type_encode = LabelEncoder()" 363 | ] 364 | }, 365 | { 366 | "cell_type": "code", 367 | "execution_count": 22, 368 | "metadata": {}, 369 | "outputs": [], 370 | "source": [ 371 | "#Integer encoding the 'type' column\n", 372 | "\n", 373 | "df['type'] = type_encode.fit_transform(df.type)" 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "execution_count": 23, 379 | "metadata": {}, 380 | "outputs": [ 381 | { 382 | "data": { 383 | "text/plain": [ 384 | "3 6732\n", 385 | "1 5694\n", 386 | "4 5231\n", 387 | "0 2186\n", 388 | "2 370\n", 389 | "Name: type, dtype: int64" 390 | ] 391 | }, 392 | "execution_count": 23, 393 | "metadata": {}, 394 | "output_type": "execute_result" 395 | } 396 | ], 397 | "source": [ 398 | "df['type'].value_counts()" 399 | ] 400 | }, 401 | { 402 | "cell_type": "code", 403 | "execution_count": 25, 404 | "metadata": {}, 405 | "outputs": [], 406 | "source": [ 407 | "#One hot encoding the 'type' column\n", 408 | "\n", 409 | "type_one_hot = OneHotEncoder()\n", 410 | "type_one_hot_encode = type_one_hot.fit_transform(df.type.values.reshape(-1,1)).toarray()" 411 | ] 412 | }, 413 | { 414 | "cell_type": "code", 415 | "execution_count": 26, 416 | "metadata": {}, 417 | "outputs": [], 418 | "source": [ 419 | "#Adding the one hot encoded variables to the dataset \n", 420 | "\n", 421 | "ohe_variable = pd.DataFrame(type_one_hot_encode, columns = [\"type_\"+str(int(i)) for i in range(type_one_hot_encode.shape[1])])\n", 422 | "df = pd.concat([df, ohe_variable], axis=1)" 423 | ] 424 | }, 425 | { 426 | "cell_type": "code", 427 | "execution_count": null, 428 | "metadata": {}, 429 | "outputs": [], 430 | "source": [ 431 | "#Dropping the original type variable \n", 432 | "\n", 433 | "df = df.drop('type', axis = 1)" 434 | ] 435 | }, 436 | { 437 | "cell_type": "markdown", 438 | "metadata": {}, 439 | "source": [ 440 | "**Checking for missing values**" 441 | ] 442 | }, 443 | { 444 | "cell_type": "code", 445 | "execution_count": 32, 446 | "metadata": {}, 447 | "outputs": [ 448 | { 449 | "data": { 450 | "text/plain": [ 451 | "step True\n", 452 | "type True\n", 453 | "amount True\n", 454 | "oldbalanceOrg True\n", 455 | "newbalanceOrig True\n", 456 | "oldbalanceDest True\n", 457 | "newbalanceDest True\n", 458 | "isFraud True\n", 459 | "type_0 True\n", 460 | "type_1 True\n", 461 | "type_2 True\n", 462 | "type_3 True\n", 463 | "type_4 True\n", 464 | "dtype: bool" 465 | ] 466 | }, 467 | "execution_count": 32, 468 | "metadata": {}, 469 | "output_type": "execute_result" 470 | } 471 | ], 472 | "source": [ 473 | "#Checking every column for missing values\n", 474 | "\n", 475 | "df.isnull().any()" 476 | ] 477 | }, 478 | { 479 | "cell_type": "code", 480 | "execution_count": 33, 481 | "metadata": {}, 482 | "outputs": [], 483 | "source": [ 484 | "#Imputing the missing values with a 0\n", 485 | "\n", 486 | "df = df.fillna(0)" 487 | ] 488 | }, 489 | { 490 | "cell_type": "code", 491 | "execution_count": 34, 492 | "metadata": {}, 493 | "outputs": [ 494 | { 495 | "data": { 496 | "text/plain": [ 497 | "step False\n", 498 | "type False\n", 499 | "amount False\n", 500 | "oldbalanceOrg False\n", 501 | "newbalanceOrig False\n", 502 | "oldbalanceDest False\n", 503 | "newbalanceDest False\n", 504 | "isFraud False\n", 505 | "type_0 False\n", 506 | "type_1 False\n", 507 | "type_2 False\n", 508 | "type_3 False\n", 509 | "type_4 False\n", 510 | "dtype: bool" 511 | ] 512 | }, 513 | "execution_count": 34, 514 | "metadata": {}, 515 | "output_type": "execute_result" 516 | } 517 | ], 518 | "source": [ 519 | "#Checking if there are missing values left\n", 520 | "\n", 521 | "df.isnull().any()" 522 | ] 523 | }, 524 | { 525 | "cell_type": "markdown", 526 | "metadata": {}, 527 | "source": [ 528 | "**Exporting the dataset**" 529 | ] 530 | }, 531 | { 532 | "cell_type": "code", 533 | "execution_count": 35, 534 | "metadata": {}, 535 | "outputs": [], 536 | "source": [ 537 | "df.to_csv('fraud_prediction.csv')" 538 | ] 539 | }, 540 | { 541 | "cell_type": "markdown", 542 | "metadata": {}, 543 | "source": [ 544 | "## Implementing the k-NN Algorithm" 545 | ] 546 | }, 547 | { 548 | "cell_type": "code", 549 | "execution_count": 38, 550 | "metadata": {}, 551 | "outputs": [], 552 | "source": [ 553 | "#Creating the features \n", 554 | "\n", 555 | "features = df.drop('isFraud', axis = 1).values\n", 556 | "target = df['isFraud'].values" 557 | ] 558 | }, 559 | { 560 | "cell_type": "markdown", 561 | "metadata": {}, 562 | "source": [ 563 | "**Splitting the data into training and test sets**" 564 | ] 565 | }, 566 | { 567 | "cell_type": "code", 568 | "execution_count": 39, 569 | "metadata": {}, 570 | "outputs": [], 571 | "source": [ 572 | "X_train, X_test, y_train, y_test = train_test_split(features, target, test_size = 0.3, random_state = 42, stratify = target)" 573 | ] 574 | }, 575 | { 576 | "cell_type": "markdown", 577 | "metadata": {}, 578 | "source": [ 579 | "**Building the knn classifier**" 580 | ] 581 | }, 582 | { 583 | "cell_type": "code", 584 | "execution_count": 41, 585 | "metadata": {}, 586 | "outputs": [], 587 | "source": [ 588 | "knn_classifier = KNeighborsClassifier(n_neighbors=3)" 589 | ] 590 | }, 591 | { 592 | "cell_type": "code", 593 | "execution_count": 42, 594 | "metadata": {}, 595 | "outputs": [ 596 | { 597 | "data": { 598 | "text/plain": [ 599 | "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n", 600 | " metric_params=None, n_jobs=1, n_neighbors=3, p=2,\n", 601 | " weights='uniform')" 602 | ] 603 | }, 604 | "execution_count": 42, 605 | "metadata": {}, 606 | "output_type": "execute_result" 607 | } 608 | ], 609 | "source": [ 610 | "knn_classifier.fit(X_train, y_train)" 611 | ] 612 | }, 613 | { 614 | "cell_type": "code", 615 | "execution_count": 44, 616 | "metadata": {}, 617 | "outputs": [ 618 | { 619 | "data": { 620 | "text/plain": [ 621 | "0.98306679209783632" 622 | ] 623 | }, 624 | "execution_count": 44, 625 | "metadata": {}, 626 | "output_type": "execute_result" 627 | } 628 | ], 629 | "source": [ 630 | "knn_classifier.score(X_test, y_test)" 631 | ] 632 | }, 633 | { 634 | "cell_type": "markdown", 635 | "metadata": {}, 636 | "source": [ 637 | "## Fine Tuning Parameters using GridSearchCV" 638 | ] 639 | }, 640 | { 641 | "cell_type": "code", 642 | "execution_count": 48, 643 | "metadata": {}, 644 | "outputs": [ 645 | { 646 | "data": { 647 | "text/plain": [ 648 | "GridSearchCV(cv=10, error_score='raise',\n", 649 | " estimator=KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n", 650 | " metric_params=None, n_jobs=1, n_neighbors=5, p=2,\n", 651 | " weights='uniform'),\n", 652 | " fit_params=None, iid=True, n_jobs=1,\n", 653 | " param_grid={'n_neighbors': array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,\n", 654 | " 18, 19, 20, 21, 22, 23, 24])},\n", 655 | " pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n", 656 | " scoring=None, verbose=0)" 657 | ] 658 | }, 659 | "execution_count": 48, 660 | "metadata": {}, 661 | "output_type": "execute_result" 662 | } 663 | ], 664 | "source": [ 665 | "#Initializing a grid with possible number of neighbors from 1 to 24\n", 666 | "\n", 667 | "grid = {'n_neighbors' : np.arange(1, 25)}\n", 668 | "\n", 669 | "#Initializing a k-NN classifier \n", 670 | "\n", 671 | "knn_classifier = KNeighborsClassifier()\n", 672 | "\n", 673 | "#Using cross validation to find optimal number of neighbors \n", 674 | "\n", 675 | "knn = GridSearchCV(knn_classifier, grid, cv = 10)\n", 676 | "\n", 677 | "knn.fit(X_train, y_train)" 678 | ] 679 | }, 680 | { 681 | "cell_type": "code", 682 | "execution_count": 49, 683 | "metadata": {}, 684 | "outputs": [ 685 | { 686 | "data": { 687 | "text/plain": [ 688 | "{'n_neighbors': 1}" 689 | ] 690 | }, 691 | "execution_count": 49, 692 | "metadata": {}, 693 | "output_type": "execute_result" 694 | } 695 | ], 696 | "source": [ 697 | "#Extracting the optimal number of neighbors \n", 698 | "\n", 699 | "knn.best_params_" 700 | ] 701 | }, 702 | { 703 | "cell_type": "code", 704 | "execution_count": 50, 705 | "metadata": {}, 706 | "outputs": [ 707 | { 708 | "data": { 709 | "text/plain": [ 710 | "0.9850813971070006" 711 | ] 712 | }, 713 | "execution_count": 50, 714 | "metadata": {}, 715 | "output_type": "execute_result" 716 | } 717 | ], 718 | "source": [ 719 | "#Extracting the accuracy score for optimal number of neighbors\n", 720 | "\n", 721 | "knn.best_score_" 722 | ] 723 | }, 724 | { 725 | "cell_type": "markdown", 726 | "metadata": {}, 727 | "source": [ 728 | "## Scaling" 729 | ] 730 | }, 731 | { 732 | "cell_type": "code", 733 | "execution_count": 55, 734 | "metadata": {}, 735 | "outputs": [ 736 | { 737 | "data": { 738 | "text/plain": [ 739 | "0.99753057384760113" 740 | ] 741 | }, 742 | "execution_count": 55, 743 | "metadata": {}, 744 | "output_type": "execute_result" 745 | } 746 | ], 747 | "source": [ 748 | "#Setting up the scaling pipeline \n", 749 | "\n", 750 | "pipeline_order = [('scaler', StandardScaler()), ('knn', KNeighborsClassifier(n_neighbors = 1))]\n", 751 | "\n", 752 | "pipeline = Pipeline(pipeline_order)\n", 753 | "\n", 754 | "#Fitting the classfier to the scaled dataset \n", 755 | "\n", 756 | "knn_classifier_scaled = pipeline.fit(X_train, y_train)\n", 757 | "\n", 758 | "#Extracting the score \n", 759 | "\n", 760 | "knn_classifier_scaled.score(X_test, y_test)" 761 | ] 762 | } 763 | ], 764 | "metadata": { 765 | "kernelspec": { 766 | "display_name": "Python 3", 767 | "language": "python", 768 | "name": "python3" 769 | }, 770 | "language_info": { 771 | "codemirror_mode": { 772 | "name": "ipython", 773 | "version": 3 774 | }, 775 | "file_extension": ".py", 776 | "mimetype": "text/x-python", 777 | "name": "python", 778 | "nbconvert_exporter": "python", 779 | "pygments_lexer": "ipython3", 780 | "version": "3.6.1" 781 | } 782 | }, 783 | "nbformat": 4, 784 | "nbformat_minor": 2 785 | } 786 | -------------------------------------------------------------------------------- /Chapter_03.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Predicting categories with Logistic Regression\n", 8 | "\n", 9 | "**Aim**: The aim of this notebook is to predict if a mobile transaction is fraudulent or not by using the logistic regression algorithm with scikit-learn.\n", 10 | "\n", 11 | "## Table of contents\n", 12 | "\n", 13 | "2. Implementing the logistic regression algorithm\n", 14 | "3. Fine-tuning parameters using GridsearchCV\n", 15 | "4. Scaling\n", 16 | "5. Interpreting the results" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "## Package Requirements" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 15, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "import pandas as pd\n", 33 | "from sklearn.model_selection import train_test_split\n", 34 | "from sklearn import linear_model\n", 35 | "from sklearn.model_selection import GridSearchCV\n", 36 | "import matplotlib.pyplot as plt\n", 37 | "from sklearn.preprocessing import StandardScaler\n", 38 | "from sklearn.pipeline import Pipeline" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": 3, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "# Reading in the dataset \n", 48 | "\n", 49 | "df = pd.read_csv('fraud_prediction.csv')" 50 | ] 51 | }, 52 | { 53 | "cell_type": "markdown", 54 | "metadata": {}, 55 | "source": [ 56 | "## Implementing the logistic regression algorithm" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "**Splitting the data into training and test sets**" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 4, 69 | "metadata": {}, 70 | "outputs": [], 71 | "source": [ 72 | "#Creating the features \n", 73 | "\n", 74 | "features = df.drop('isFraud', axis = 1).values\n", 75 | "target = df['isFraud'].values" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": 5, 81 | "metadata": {}, 82 | "outputs": [], 83 | "source": [ 84 | "X_train, X_test, y_train, y_test = train_test_split(features, target, test_size = 0.3, random_state = 42, stratify = target)" 85 | ] 86 | }, 87 | { 88 | "cell_type": "markdown", 89 | "metadata": {}, 90 | "source": [ 91 | "**Creating and evaluating the base classifier**" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": 6, 97 | "metadata": {}, 98 | "outputs": [ 99 | { 100 | "data": { 101 | "text/plain": [ 102 | "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n", 103 | " intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n", 104 | " penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n", 105 | " verbose=0, warm_start=False)" 106 | ] 107 | }, 108 | "execution_count": 6, 109 | "metadata": {}, 110 | "output_type": "execute_result" 111 | } 112 | ], 113 | "source": [ 114 | "#Initializing an logistic regression object\n", 115 | "\n", 116 | "logistic_regression = linear_model.LogisticRegression()\n", 117 | "\n", 118 | "#Fitting the model to the training and test sets\n", 119 | "\n", 120 | "logistic_regression.fit(X_train, y_train)" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": 7, 126 | "metadata": {}, 127 | "outputs": [ 128 | { 129 | "data": { 130 | "text/plain": [ 131 | "0.58936970837253055" 132 | ] 133 | }, 134 | "execution_count": 7, 135 | "metadata": {}, 136 | "output_type": "execute_result" 137 | } 138 | ], 139 | "source": [ 140 | "#Accuracy score of the logistic regression model\n", 141 | "\n", 142 | "logistic_regression.score(X_test, y_test)" 143 | ] 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "metadata": {}, 148 | "source": [ 149 | "## Fine tuning parameters using GridSearchCV" 150 | ] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": 8, 155 | "metadata": {}, 156 | "outputs": [ 157 | { 158 | "name": "stdout", 159 | "output_type": "stream", 160 | "text": [ 161 | "The most optimal inverse regularization strength is: {'C': 10}\n" 162 | ] 163 | } 164 | ], 165 | "source": [ 166 | "#Building the model with L1 penality \n", 167 | "\n", 168 | "logistic_regression = linear_model.LogisticRegression(penalty='l1')\n", 169 | "\n", 170 | "#Using GridSearchCV to search for the best parameter\n", 171 | "\n", 172 | "grid = GridSearchCV(logistic_regression, {'C':[0.0001, 0.001, 0.01, 0.1, 10]})\n", 173 | "grid.fit(X_train, y_train)\n", 174 | "\n", 175 | "# Print out the best parameter\n", 176 | "\n", 177 | "print(\"The most optimal inverse regularization strength is:\", grid.best_params_)" 178 | ] 179 | }, 180 | { 181 | "cell_type": "code", 182 | "execution_count": 9, 183 | "metadata": {}, 184 | "outputs": [ 185 | { 186 | "data": { 187 | "text/plain": [ 188 | "LogisticRegression(C=10, class_weight=None, dual=False, fit_intercept=True,\n", 189 | " intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n", 190 | " penalty='l1', random_state=None, solver='liblinear', tol=0.0001,\n", 191 | " verbose=0, warm_start=False)" 192 | ] 193 | }, 194 | "execution_count": 9, 195 | "metadata": {}, 196 | "output_type": "execute_result" 197 | } 198 | ], 199 | "source": [ 200 | "#Initializing an logistic regression object\n", 201 | "\n", 202 | "logistic_regression = linear_model.LogisticRegression(C = 10, penalty = 'l1')\n", 203 | "\n", 204 | "#Fitting the model to the training and test sets\n", 205 | "\n", 206 | "logistic_regression.fit(X_train, y_train)" 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "execution_count": 10, 212 | "metadata": {}, 213 | "outputs": [ 214 | { 215 | "data": { 216 | "text/plain": [ 217 | "0.99670743179680155" 218 | ] 219 | }, 220 | "execution_count": 10, 221 | "metadata": {}, 222 | "output_type": "execute_result" 223 | } 224 | ], 225 | "source": [ 226 | "#Accuracy score of the logistic regression model\n", 227 | "\n", 228 | "logistic_regression.score(X_test, y_test)" 229 | ] 230 | }, 231 | { 232 | "cell_type": "code", 233 | "execution_count": 11, 234 | "metadata": {}, 235 | "outputs": [ 236 | { 237 | "data": { 238 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEOCAYAAAB4nTvgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xt8XWWd7/HPN5c2Lb2Ftlza9AaW\nSxWhEOoFVDgqFI+Cenw5oM6oM8eOcwAvRzzCa7goHpXjbZRz8FIdRh0HEfHGaBVQ6egoaJMW6L1N\nS9ImQCltWlralCb5nT/WSru7m2TvlOzsvZPv+/Xar6z1rGet9dur6fplrWc961FEYGZm1p+KYgdg\nZmalz8nCzMxycrIwM7OcnCzMzCwnJwszM8vJycLMzHJysjAzs5ycLMzMLCcnCzMzy8nJwszMcqoq\ndgCDZcqUKTF79uxih2FmVlYaGxufjYipueoNm2Qxe/ZsGhoaih2GmVlZkdSSTz3fhjIzs5ycLMzM\nLKeCJQtJd0p6RtKqPpZL0u2SmiQ9LuncjGXvlbQx/by3UDGamVl+Cnll8R1gYT/LLwPmpp9FwNcB\nJB0P3AK8AlgA3CKptoBxmplZDgVLFhHxe2BnP1WuAL4XiUeASZJOBi4FHoyInRHRDjxI/0nHzMwK\nrJhtFtOBrRnzrWlZX+VmZlYkZf3orKRFJLewmDlzZpGjsVIUEURAZM5DWpYsI52Hw2WR1k3K0uV5\n1InDlY4qy1wnczTjPutkLCcz1j7iz5zvrSxzu4frZR+fo+tE+mWil+1mH1eyv0tGneFEuZb3U6G/\nZcm2+1u59+IJNdWcN6uwd+uLmSzagBkZ83VpWRtwUVb50t42EBGLgcUA9fX1w/F38iir2nbzsxVt\ndKUnnO6I9JP8h+3uhu7ubiK6UXRBdxeR/lR0QzpNdKXL07K0fmaZutNtRLKuMn5CNxXReURZRXQn\n06Tr0oUiiAiUnH7S3/U4dCaKnrJDZ6/0rBwk62SU96wb6TL1nJYObb9nfQ6tL0XG/6/D9QQZ04fr\nKGM/R5fHofWUcQpUjrpk7TN7G9Lher3Flb19jto+vdTrfxtH7L+X4yJIj13v37O3GI/+nkfWLSeZ\n36m375N97I6qoxzLc2z7yOne/g2PLN9YfTrc+EhBj0kxk8V9wDWS7iZpzN4dEU9Juh/4bEaj9iXA\nDcUKstQs/8mXWfTsd6hSF5V0U0E3lUT6s/vQz7KnrJ+D5KhT5qHt69DOQiLzdNHzp2CQUa6sIKWM\n5cmPQ/PK2Pah+Z79H7mNQ6ff7Lh6lmXWP2JbZK2fGUtPXYCKjG1XZGzzyLjQ0bGTXU/J38CRVV+9\nbSPXn9MlKfOY9xw/Hf3vetS/m474Heopy/63iUPrHf6dO7yvisO/p0f9ux69j9qJdQU/GgVLFpJ+\nQHKFMEVSK8kTTtUAEfENYAnwJqAJ2Ae8P122U9KngWXppm6NiP4aykeMiOCsHb+metQoas9+C1RU\ngirTnxVZ85VQUZE1PwjlFVUD24Yqkg8cfbI5VEZWea7prJNXrumM/WUsMbMBKFiyiIirciwP4Oo+\nlt0J3FmIuMpZ8zO7mBebeKLuKmrf/OVih2NmI4h7cJeRzaseYbQOMmHuq4sdipmNME4WZWTfpocB\nOGnea4ociZmNNE4WZWTc9uXsqJxKxaTCN2aZmWVysigTu/cfZO4La9lZ+/Jih2JmI5CTRZlYvX49\ndXqWqpmvKHYoZjYCOVmUiWfX/icAJ730tUWOxMxGIieLMlHR1sBBqhgz69zclc3MBpmTRRno7Orm\npD0reXrsaVA1utjhmNkI5GRRBtY9uZOXspkXTq4vdihmNkI5WZSBltV/ZoxeYNJpFxQ7FDMboZws\nysD+zcnbJI8/3T23zaw4nCzKwIQdj7KrcjKaOCN3ZTOzAnCyKHHbnuvgjINraT/+nDJ9zbOZDQdO\nFiVu5fqNzKzYzqjZryx2KGY2gjlZlLgd6/4IwAnzLixyJGY2kjlZlLiqJxvopIrqOnfGM7PiKWiy\nkLRQ0npJTZKu72X5LEm/lfS4pKWS6jKWdUl6NP3cV8g4S1XHwS5mPL+SZ447Haprih2OmY1gBUsW\nkiqBO4DLgHnAVZLmZVX7IvC9iHg5cCvwuYxl+yPinPRzeaHiLGWrtj7LWdpM57Tzih2KmY1whbyy\nWAA0RcTmiHgBuBu4IqvOPOB36fRDvSwf0ZrXLGOMXqD2dHfGM7PiKmSymA5szZhvTcsyPQa8PZ1+\nGzBe0uR0vkZSg6RHJL21gHGWrANPJJ3xxr/EnfHMrLiK3cB9HfA6SSuA1wFtQFe6bFZE1APvAr4i\n6dTslSUtShNKw/bt24cs6KEQEdTufJTdVZPBnfHMrMgKmSzagMyzXF1adkhEPBkRb4+I+cA/pmW7\n0p9t6c/NwFJgfvYOImJxRNRHRP3UqVML8iWKpXnHPuZ1beC5ye6MZ2bFV8hksQyYK2mOpFHAlcAR\nTzVJmiKpJ4YbgDvT8lpJo3vqABcAawoYa8lZtaGJ2RXbqJnjznhmVnwFSxYR0QlcA9wPrAXuiYjV\nkm6V1PN000XAekkbgBOBz6TlZwINkh4jafi+LSJGVLJoX/8nACaf4c54ZlZ8VYXceEQsAZZkld2c\nMX0vcG8v6/0JOKuQsZW6UU830EklVdOPuvtmZjbkit3Abb3Yvf8gs/avYce406F6TLHDMTNzsihF\njzZv52xtonu6R8Yzs9LgZFGCtqxtYKwOUHu62yvMrDQ4WZSgg1v+DEDNnFcUORIzs4STRYnp7Opm\ncvtj7Kk6HibNKnY4ZmaAk0XJWff0Hl4eG9g7db4745lZyXCyKDGrmzYxp2IbY09xZzwzKx1OFiVm\n98aHAZgw1y8PNLPS4WRRYmqebqSLSjTNI+OZWelwsigh257r4CUH1rJz/GkwamyxwzEzO8TJooQs\nb36Wsys2Qd35xQ7FzOwIBX03lA3M1nWNHKcDjD7NI+OZWWnxlUUJ6Uo741XNcmc8MystThYlouNg\nFyc+9zjPV9VC7exih2NmdgQnixKxsm03Z9PE8yec6854ZlZyCposJC2UtF5Sk6Tre1k+S9JvJT0u\naamkuoxl75W0Mf28t5BxloJVTU9wasVTHHfqq4odipnZUQqWLCRVAncAlwHzgKskzcuq9kXgexHx\ncuBW4HPpuscDtwCvABYAt0iqLVSspeC5tDOek4WZlaJCXlksAJoiYnNEvADcDVyRVWce8Lt0+qGM\n5ZcCD0bEzohoBx4EFhYw1qKKCMY+s5wuKmGaR8Yzs9JTyGQxHdiaMd+almV6DHh7Ov02YLykyXmu\nO2w079jHmZ3r2D3hNBh1XLHDMTM7SrEbuK8DXidpBfA6oA3oyndlSYskNUhq2L59e6FiLLjG5mc5\np2ITFTMWFDsUM7NeFTJZtAEzMubr0rJDIuLJiHh7RMwH/jEt25XPumndxRFRHxH1U6dOHez4h0zb\n+uWMUwcT5rq9wsxKUyGTxTJgrqQ5kkYBVwL3ZVaQNEVSTww3AHem0/cDl0iqTRu2L0nLhqVoXQbg\nKwszK1kFSxYR0QlcQ3KSXwvcExGrJd0q6fK02kXAekkbgBOBz6Tr7gQ+TZJwlgG3pmXDzu79B5m+\ndyX7qybB8acUOxwzs14V9N1QEbEEWJJVdnPG9L3AvX2seyeHrzSGrRVb2jlXG9l/4nmMcWc8MytR\nxW7gHvHWNDVzasVTjHuJR8Yzs9LlZFFkezcnLw8cNdvJwsxKl5NFEXV2dTP+2RV0UwEeGc/MSpiT\nRRGte3oPL+tez54Jc2H0uGKHY2bWJyeLIlrRknTGq/T4FWZW4jxSXhG1bXyU8dpP+OWBZlbifGVR\nREo742mGryzMrLQ5WRTJtuc6mL1/LR3V7oxnZqXPyaJIlre0c27FRg6c5JHxzKz05ZUsJF0o6f3p\n9FRJcwob1vC3alMLcyvaPNiRmZWFnMlC0i3AJ0he9AdQDXy/kEGNBPueSDrjVc30ywPNrPTlc2Xx\nNuBy4HlIXisOjC9kUMNdx8Euanc+lnTGm35escMxM8spn2TxQkQEEACSPJTbi7SybTfnsIHnJ86F\n0c67Zlb68kkW90j6JjBJ0geA3wDfKmxYw1tj8w7OqdhE9Ww/Mmtm5SFnp7yI+KKkNwLPAacDN0fE\ngwWPbBhr2/gYE7QP/PJAMysT/SYLSZXAbyLiYmDACULSQuCrQCXw7Yi4LWv5TOC7wKS0zvURsUTS\nbJIBk9anVR+JiA8OdP+lKCKoerIhmfHIeGZWJvpNFhHRJalb0sSI2D2QDaeJ5g7gjUArsEzSfRGx\nJqPajSQj6H1d0jySgZJmp8s2RcQ5A9lnOWjesY/TD67lwJgJjJ78kmKHY2aWl3zeDbUXWCnpQdIn\nogAi4kM51lsANEXEZgBJdwNXAJnJIoAJ6fRE4Mk84y5bjWlnvM6Tz2O0O+OZWZnIJ1n8JP0M1HRg\na8Z8K5DdovtJ4AFJ1wLHAW/IWDZH0gqStpIbI+IPxxBDyVm9eQvvqGij+5T3FjsUM7O85dPA/V1J\no4DT0qL1EXFwkPZ/FfCdiPiSpFcB/yrpZcBTwMyI2CHpPOBnkl4aEc9lrixpEbAIYObMmYMUUmF1\nPPEXACrcXmFmZSSfHtwXARtJ2h++BmyQ9No8tt0GzMiYr0vLMv0dcA9ARDwM1ABTIuJAROxIyxuB\nTRxOVodExOKIqI+I+qlTp+YRUnHt3n+QE557nEDujGdmZSWffhZfAi6JiNdFxGuBS4F/ymO9ZcBc\nSXPSK5Mrgfuy6mwBXg8g6UySZLE9ff9UZVp+CjAX2JzPFyplK7a0M19N7Js0F2om5F7BzKxE5JMs\nqiOi5xFWImIDyfuh+hURncA1wP0kj8HeExGrJd0q6fK02seAD0h6DPgB8L60t/hrgcclPQrcC3ww\nInYO5IuVouXNO5hfsZFR7l9hZmUmnwbuBknf5vDLA98NNOSz8YhYQvI4bGbZzRnTa4ALelnvx8CP\n89lHOXlq80omah94GFUzKzP5JIt/AK4Geh6V/QNJ24UNQGdXN6Ofakyu5ercuG1m5SWfZFEFfDUi\nvgyHOtuNLmhUw9C6p/cwr3s9L4yewCh3xjOzMpNPm8VvgTEZ82NIXiZoA7B8SzvnVjTRNe08qPAA\nhWZWXvI5a9VExN6emXR6bOFCGp5Wb97KaRWt1Mxx47aZlZ98ksXzks7tmUk7ye0vXEjD04GWBioI\nVHd+sUMxMxuwfNosPgL8SNKTgICTgL8qaFTDzLbnOpjx/CqiWqiuvtjhmJkNWD6v+1gm6QySsSxg\ncF/3MSIsT18e2DFpLmNqJhY7HDOzAevzNpSk8yWdBJAmh3OBzwBfknT8EMU3LDQ072B+RZM745lZ\n2eqvzeKbwAsA6bugbgO+B+wGFhc+tOHjqc2rmaTnqZzp9gozK0/93YaqzHjFxl8Bi3t6Vqev4bA8\ndBzsYvz25cmRdmc8MytT/V1ZVErqSSavB36XsSyfhnEDVrbt5mw2cLB6PEw56sW5ZmZlob+T/g+A\n/5D0LMmjsn8AkPQSkltRlofGlnZeV9FETHdnPDMrX30mi4j4jKTfAicDD6Rvg4XkauTaoQhuOFj1\nRBsfqGilcta7ih2Kmdkx6/d2UkQ80kvZhsKFM7xEBJ1bllFJN8xw47aZlS/fFymg5h37eMmBtcnM\ndHfGM7PyVdBkIWmhpPWSmiRd38vymZIekrRC0uOS3pSx7IZ0vfWSLi1knIXS2NLO/IomDtTOhTGT\nih2Omdkxy2cM7msl1Q50w+mrzO8ALgPmAVdJmpdV7UaSEfTmkwy7+rV03Xnp/EuBhcDXeoZZLSeN\nzTs5t7KJUR7syMzKXD5XFicCyyTdk14pKM9tLwCaImJzRLwA3A1ckVUngJ7BqCcCT6bTVwB3R8SB\niHgCaEq3V1a2PbGaWvagGWUXupnZEXImi4i4EZgL/DPwPmCjpM9KOjXHqtOBrRnzrWlZpk8C75HU\nSjL8as9TVvmsW9J27z9IbXvad9FvmjWzMpdXm0X62OzT6acTqAXulfT5F7n/q4DvREQd8CbgXyXl\n3Y4iaZGkBkkN27dvf5GhDK4VW9qZr410Vo+DqWcUOxwzsxclnzaLD0tqBD4P/BE4KyL+ATgP+G/9\nrNoGzMiYr0vLMv0dcA9ARDwM1ABT8lyXiFgcEfURUT916tRcX2VIJW+abULT690Zz8zKXj5nseOB\nt0fEpRHxo57Xk0dEN/DmftZbBsyVNEfSKJIG6/uy6mwheZUIks4kSRbb03pXShotaQ7JbbC/DOB7\nFd3q5jZOr9hKpRu3zWwYyOcdT78Cel4oiKQJwJkR8eeIWNvXShHRKeka4H6gErgzIlZLuhVoiIj7\ngI8B35L0UZLG7velt7xWS7oHWENy2+vqiOg6xu845Dq7uuluXU5lRbfbK8xsWMgnWXydZCyLHnt7\nKetVRCwhabjOLLs5Y3oNcEEf636GZPyMsrN+2x7O7FqfXLd5ZDwzGwbyuQ2ljPdC9dx+8ltn+9GY\njox3sPYlMGbAXVTMzEpOPslis6QPSapOPx8GNhc6sHLW2LyT8yo3UTXL/SvMbHjIJ1l8EHg1ydNI\nrcArgEWFDKrcPd28llqeQx7syMyGiZy3kyLiGZInmSwP257rYNqelTAKcM9tMxsmciYLSTUk/SFe\nSvJoKwAR8bcFjKtsLU/bK7qqx1HpznhmNkzkcxvqX4GTgEuB/yDpILenkEGVs4aWds6tbELTz4OK\nsnv3oZlZr/JJFi+JiJuA5yPiu8B/JWm3sF6san6KM7SFipm+BWVmw0c+yeJg+nOXpJeRvB32hMKF\nVL46DnZR9fSKZGQ8N26b2TCST3+Jxel4FjeSvIZjHHBTQaMqUyvbdnN2z6iz7oxnZsNIv8kifQPs\ncxHRDvweOGVIoipTPSPjddWeSuXY44sdjpnZoOn3NlTaW/t/DVEsZa+xeSf1VU1UznSTjpkNL/m0\nWfxG0nWSZkg6vudT8MjKTETwTMs6amM3zPDLA81seMmnzeKv0p9XZ5QFviV1hOYd+5jdsSbpjOc3\nzZrZMJNPD+45QxFIuet5eWB39XFUnDCv2OGYmQ2qfHpw/01v5RHxvcEPp3w1trTz11XujGdmw1M+\nt6Ey76nUkIxstxxwssiwuvkpTqcFzXh7sUMxMxt0+dyGujZzXtIk4O58Ni5pIfBVkpHyvh0Rt2Ut\n/yfg4nR2LHBCRExKl3UBK9NlWyLi8nz2WQy79x9kzLOPUznKnfHMbHg6lkGMngdytmNIqgTuAN5I\n8mrzZZLuS0fHAyAiPppR/1pgfsYm9kfEOccQ35BbsaWd+dqYzLhx28yGoXzaLP6d5OknSB61nQfc\nk8e2FwBNEbE53c7dwBUk42r35irgljy2W3J63jTbXXsqFcdNLnY4ZmaDLp8riy9mTHcCLRHRmsd6\n04GtGfM9AycdRdIskquV32UU10hqSPd5W0T8rJf1FpEOxDRz5sw8QiqMxpadvL9qExUzFhYtBjOz\nQsonWWwBnoqIDgBJYyTNjojmQYzjSuDeiOjKKJsVEW2STgF+J2llRGzKXCkiFgOLAerr64Mi6Ozq\nZvvWjdRW7HJnPDMbtvLpwf0joDtjvisty6UNmJExX5eW9eZK4AeZBRHRlv7cDCzlyPaMkrF+2x7O\n6FyXzLhx28yGqXySRVVEvNAzk06PymO9ZcBcSXMkjSJJCPdlV5J0BlALPJxRVitpdDo9BbiAvts6\niupwZ7yx4M54ZjZM5ZMstks69NiqpCuAZ3OtFBGdwDXA/cBa4J6IWC3p1sztkSSRuyMi8zbSmUCD\npMeAh0jaLEo2WSyo3oSmnwuVx/JwmZlZ6cvn7PZB4N8k/b90vhXotVd3tohYAizJKrs5a/6Tvaz3\nJ+CsfPZRbCubn+b0aEZ1VxQ7FDOzgsmnU94m4JWSxqXzewseVZnY9lwHk3evoXJ0F8xwe4WZDV85\nb0NJ+qykSRGxNyL2pu0J/3sogit1y1vamV+Rdsab7pHxzGz4yqfN4rKI2NUzk46a96bChVQ+Glra\nqa9sImrnwLipxQ7HzKxg8kkWlT1PJkHSzwIY3U/9EaOxeSfnVzUh34Iys2EunwbufwN+K+lf0vn3\n4zfO0nGwi11PbaK2ut3vgzKzYS+fBu7/kz7C+oa06NMRcX9hwyp9K9t2c1ZsSGacLMxsmMurY0BE\n/Br4NYCkCyXdERFX51htWOvpjBdVY9GJLyt2OGZmBZVPmwWS5kv6vKRm4NPAuoJGVQYaW9p55ajN\n7oxnZiNCn2c5SaeRvDb8KpIe2z8EFBEX97XOSBERrG5+mtO6N0Pdfy12OGZmBdffn8TrgD8Ab46I\nJgBJH+2n/ojRvGMfJ+/f4M54ZjZi9Hcb6u3AU8BDkr4l6fWAhias0tbTXgG4cdvMRoQ+k0VE/Cwi\nrgTOIHmZ30eAEyR9XdIlQxVgKUpeHthETJoF404odjhmZgWXs4E7Ip6PiLsi4i0kY1KsAD5R8MhK\n2PLmndRXujOemY0ceT0N1SMi2iNicUS8vlABlbrd+w+yd3sztV07PdiRmY0YA0oWBiu2tDNfaXuF\nh1E1sxGioMlC0kJJ6yU1Sbq+l+X/JOnR9LNB0q6MZe+VtDH9vLeQcQ7E8pZ2zq1oIqrGgDvjmdkI\nUbDeZJIqgTuAN5IMmLRM0n2ZI95FxEcz6l9LOs62pOOBW4B6IIDGdN32QsWbr8Yt7dw0ejOaNh8q\nq4sdjpnZkCjklcUCoCkiNqfjdt8N9Dec3FXAD9LpS4EHI2JnmiAeBBYWMNa8dHZ1s2bLM7yke5Nv\nQZnZiFLIZDEd2Jox35qWHUXSLGAO8LuBrjuU1m/bw5yDm6iKTvevMLMRpVQauK8E7o2IroGsJGmR\npAZJDdu3by9QaIc1Zo6M5yehzGwEKWSyaANmZMzXpWW9uZLDt6DyXjd9jLc+IuqnTi38SHWNLe28\netRmYtJMGH9iwfdnZlYqCpkslgFzJc2RNIokIdyXXUnSGUAt8HBG8f3AJel437XAJWlZUTW2tHNu\n5UbkqwozG2EK9jRURHRKuobkJF8J3BkRqyXdCjRERE/iuBK4OyIiY92dkj5NknAAbo2InYWKNR/b\nnuugs72V2ppn3V5hZiNOQQdiiIglwJKsspuz5j/Zx7p3AncWLLgBWp758kA/CWVmI0ypNHCXvMaW\nds6vaiKqauDEs4odjpnZkHKyyFNDSzsXjN6MTj4HqkYVOxwzsyHlZJGHjoNdbHzyWU7pdGc8MxuZ\nnCzysLJtN6d1P0FVHHT/CjMbkZws8uCR8cxspHOyyENjSzsXjnkCJs6ACScXOxwzsyHnZJFDRLC8\npZ1z2OirCjMbsZwscmjesY+q55+mtvMZ8DCqZjZCOVnk4PYKMzMni5waW9p5xahNROVoOOnlxQ7H\nzKwonCxyWN7SzgWjNqNp7oxnZiOXk0U/du8/yBPPtDPnYJNvQZnZiOZk0Y8VW9o5k2aq4gUnCzMb\n0Zws+rG8pZ3zKnveNOsnocxs5HKy6EfjlnZeN7YZJtTBhGnFDsfMrGgKmiwkLZS0XlKTpOv7qPNO\nSWskrZZ0V0Z5l6RH089RI+wVWmdXN49u2cXZsQHq6od692ZmJaVggx9JqgTuAN4ItALLJN0XEWsy\n6swFbgAuiIh2SSdkbGJ/RJxTqPhyWb9tD8e98CyTKrb5FpSZjXiFvLJYADRFxOaIeAG4G7giq84H\ngDsioh0gIp4pYDwD0tjSzvxDnfGcLMxsZCtkspgObM2Yb03LMp0GnCbpj5IekbQwY1mNpIa0/K0F\njLNXjS3tXFizmagcBSe7M56ZjWwFHYM7z/3PBS4C6oDfSzorInYBsyKiTdIpwO8krYyITZkrS1oE\nLAKYOXPmoAbW2NLOtaOeQFPOhqrRg7ptM7NyU8hk0QbMyJivS8sytQJ/joiDwBOSNpAkj2UR0QYQ\nEZslLQXmA0cki4hYDCwGqK+vj8EKfNtzHWxr38PssRug7r8P1mbNrAQdPHiQ1tZWOjo6ih1KQdXU\n1FBXV0d1dfUxrV/IZLEMmCtpDkmSuBJ4V1adnwFXAf8iaQrJbanNkmqBfRFxIC2/APh8AWM9wvKW\nds5UC1XdBzyMqtkw19rayvjx45k9ezaSih1OQUQEO3bsoLW1lTlz5hzTNgrWZhERncA1wP3AWuCe\niFgt6VZJl6fV7gd2SFoDPAR8PCJ2AGcCDZIeS8tvy3yKqtAaW9o5v6opmXHjttmw1tHRweTJk4dt\nogCQxOTJk1/U1VNB2ywiYgmwJKvs5ozpAP5n+sms8yfgrELG1p+GlnauO64FqqfBxOw2eTMbboZz\noujxYr+je3Bn6TjYxeond/OyWO9bUGZWcLt27eJrX/vagNd705vexK5duwoQUe+cLLKsbNvNpK52\nJh14yregzKzg+koWnZ2d/a63ZMkSJk2aVKiwjlLsR2dLzpGd8XxlYWaFdf3117Np0ybOOeccqqur\nqampoba2lnXr1rFhwwbe+ta3snXrVjo6Ovjwhz/MokWLAJg9ezYNDQ3s3buXyy67jAsvvJA//elP\nTJ8+nZ///OeMGTNmUOP0lUWWxpZ2Lj6uBSqq4eSzix2OmQ1zt912G6eeeiqPPvooX/jCF1i+fDlf\n/epX2bBhAwB33nknjY2NNDQ0cPvtt7Njx46jtrFx40auvvpqVq9ezaRJk/jxj3886HH6yiJDRLC8\npZ3rRzfBCWdDdU2xQzKzIfSpf1/NmiefG9Rtzps2gVve8tK86y9YsOCIx1tvv/12fvrTnwKwdetW\nNm7cyOTJk49YZ86cOZxzTvIqvfPOO4/m5uYXH3gWJ4sMzTv2sfv5fcyKDTDjb4sdjpmNQMcdd9yh\n6aVLl/Kb3/yGhx9+mLFjx3LRRRf1+vjr6NGH3zJRWVnJ/v37Bz0uJ4sMjS3tnKEtVHV3+LXkZiPQ\nQK4ABsv48ePZs2dPr8t2795NbW0tY8eOZd26dTzyyCNDHN1hThYZGlvaefXozcmMn4QysyEwefJk\nLrjgAl72spcxZswYTjzxxEPLFi5cyDe+8Q3OPPNMTj/9dF75ylcWLU4niwzLW9q5ZWwzVJ4ME+uK\nHY6ZjRB33XVXr+WjR4/mV78k6oSZAAAOKElEQVT6Va/LetolpkyZwqpVqw6VX3fddYMeH/hpqEN2\n7z/Ihmf28NLu9cktqBHQo9PMLF9OFqkVW9o5PnYzsaPNt6DMzLI4WaSWt7RzXmXaGc/DqJqZHcHJ\nItW4pZ03jNviznhmZr1wsgA6u7p5dMsuzq/eBCedBdWD203ezKzcOVkA67ftoeOFF5ixf51vQZmZ\n9aKgyULSQknrJTVJur6POu+UtEbSakl3ZZS/V9LG9PPeQsZ5qDNe136/PNDMhtSxvqIc4Ctf+Qr7\n9u0b5Ih6V7BkIakSuAO4DJgHXCVpXladucANwAUR8VLgI2n58cAtwCuABcAt6VCrBdHY0s5rxzQn\nM04WZjaEyiVZFLJT3gKgKSI2A0i6G7gCyBwe9QPAHRHRDhARz6TllwIPRsTOdN0HgYXADwoRaGNL\nO+867gmIE2HSzELswsysV5mvKH/jG9/ICSecwD333MOBAwd429vexqc+9Smef/553vnOd9La2kpX\nVxc33XQT27Zt48knn+Tiiy9mypQpPPTQQwWNs5DJYjqwNWO+leRKIdNpAJL+CFQCn4yIX/exbkHG\nN932XAet7fs5o3Y9zDrfnfHMbEjddtttrFq1ikcffZQHHniAe++9l7/85S9EBJdffjm///3v2b59\nO9OmTeOXv/wlkLwzauLEiXz5y1/moYceYsqUKQWPs9iv+6gC5gIXAXXA7yXlPfa2pEXAIoCZM4/t\nimDS2Gp+9J5TmXjvVpjxgWPahpkNE7+6Hp5eObjbPOksuOy2vKo+8MADPPDAA8yfPx+AvXv3snHj\nRl7zmtfwsY99jE984hO8+c1v5jWvec3gxpiHQiaLNmBGxnxdWpapFfhzRBwEnpC0gSR5tJEkkMx1\nl2bvICIWA4sB6uvr41iCHF1VyfnVT6R7cXuFmRVPRHDDDTfw93//90ctW758OUuWLOHGG2/k9a9/\nPTfffPOQxlbIZLEMmCtpDsnJ/0rgXVl1fgZcBfyLpCkkt6U2A5uAz2Y0al9C0hBeGFv/AhVVMG1+\nwXZhZmUgzyuAwZT5ivJLL72Um266iXe/+92MGzeOtrY2qqur6ezs5Pjjj+c973kPkyZN4tvf/vYR\n65b1baiI6JR0DXA/SXvEnRGxWtKtQENE3Jcuu0TSGqAL+HhE7ACQ9GmShANwa09jd0G0LnNnPDMr\nisxXlF922WW8613v4lWvehUA48aN4/vf/z5NTU18/OMfp6Kigurqar7+9a8DsGjRIhYuXMi0adMK\n3sCtiGO6e1Ny6uvro6GhYeArdnXCbTNg/nvgTV8Y/MDMrKStXbuWM888s9hhDInevqukxojIOdqb\ne3DvfRomTIMZ2Q9qmZlZj2I/DVV8E+vg2kYYJldYZmaF4CuLHu5fYWbWJycLMxvxhkvbbX9e7Hd0\nsjCzEa2mpoYdO3YM64QREezYsYOamppj3obbLMxsRKurq6O1tZXt27cXO5SCqqmpoa6u7pjXd7Iw\nsxGturqaOXPmFDuMkufbUGZmlpOThZmZ5eRkYWZmOQ2b131I2g60pLMTgd39TPdWNgV4doC7zdxO\nvsuyy/ua7y/uwY61r+W5ysrp2OYbt4/t8Du2+cQ+ko/trIiYmrN2RAy7D7C4v+k+yhpezH7yXZZd\n3td8f3EPdqx9Lc9VVk7HNt+4fWyH37HNJ3Yf29yf4Xob6t9zTPe1/MXsJ99l2eV9zeeKe6Byrdvb\n8lxl5XRsBxL3QPnY9j9d7GObT+w+tjkMm9tQL5akhsjjzYuloJxihfKKt5xihfKKt5xihfKKdyhi\nHa5XFsdicbEDGIByihXKK95yihXKK95yihXKK96Cx+orCzMzy8lXFmZmlpOThZmZ5eRkYWZmOTlZ\n5EHScZIaJL252LHkIulMSd+QdK+kfyh2PP2R9FZJ35L0Q0mXFDueXCSdIumfJd1b7Fh6k/6efjc9\npu8udjy5lPrxzFSGv6uDfx4YaEeOcvoAdwLPAKuyyhcC64Em4Po8tnMr8L+AN5dDvOk6FcD3yyTW\nWuCfy+jY3lvIWI81buCvgbek0z8cqhhf7HEeyuM5CLEW/Hd1kOMdtPPAkH/hIT64rwXOzTy4QCWw\nCTgFGAU8BswDzgJ+kfU5AXgjcCXwviFIFi863nSdy4FfAe8q9VjT9b4EnFsOxzZdbyiTxUDivgE4\nJ61z11DFeKzxFuN4DkKsBf9dHax4B/s8MKzHs4iI30uanVW8AGiKiM0Aku4GroiIzwFH3WaSdBFw\nHMl/xv2SlkREd6nGm27nPuA+Sb8E7irVWCUJuA34VUQsL0ScgxlvMQwkbqAVqAMepUi3mAcY75qh\nje5IA4lV0lqG6He1LwM9toN9HhiJbRbTga0Z861pWa8i4h8j4iMkB/tbhUoU/RhQvJIuknS7pG8C\nSwodXJYBxQpcC7wBeIekDxYysD4M9NhOlvQNYL6kGwodXD/6ivsnwH+T9HVe3GsgBluv8ZbQ8czU\n17Et9u9qX/o6toN+HhjWVxaDKSK+U+wY8hERS4GlRQ4jLxFxO3B7sePIV0TsAErpRHGEiHgeeH+x\n48hXqR/PTGX4u7qUQT4PjMQrizZgRsZ8XVpWqsop3nKKFcov3h7lFnc5xVtOscIQxjsSk8UyYK6k\nOZJGkTRe31fkmPpTTvGWU6xQfvH2KLe4yynecooVhjLeYrTqD+HTAz8AngIOktzL+7u0/E3ABpKn\nCP6x2HGWY7zlFGs5xluucZdTvOUUaynE6xcJmplZTiPxNpSZmQ2Qk4WZmeXkZGFmZjk5WZiZWU5O\nFmZmlpOThZmZ5eRkYUeRdJKkuyVtktQoaYmk03qpN0bSf0iqlDRb0qpixFts6Xt4fjHAdaYdyzgO\nkiZJ+h8vdjsD2N9bJc0r1Paz9vU+SdMy5pslTeml3psl3ToUMdlhThZ2hPRNsD8FlkbEqRFxHsmr\nr0/spfrfAj+JiK4hiq1ykLZT1HeiSaqKiCcj4h3HsPok4FCyeBHbyddbSd64fJQCHMf3AdNyVQJ+\nCbxF0thB3r/1w8nCsl0MHIyIb/QURMRjEfGHXuq+G/h5dmH6F+JPJP1a0kZJn0/LPyjpC1n1/l86\n/R5Jf5H0qKRv9iQGSXslfUnSY8CrJN0maY2kxyV9Ma0zVdKPJS1LPxf0EdN9kn4H/DYt+3ha/3FJ\nn8qoe5Ok9ZL+U9IPJF2Xli+VVJ9OT5HU3Mt+Fkh6WNIKSX+SdHpv+8+8EpP07fR7Pyppu6RbJI2T\n9FtJyyWtlHRFuovbgFPTul/I2k6NpH9J66+QdHF//x69xH7EsZX0apIxEb6Q7u/U9Bh8RVID8OG+\njr2kT0q6M62/WdKH+ju+kt4B1AP/lu5rTFr92oxjcAZAJD2Jl1Iir5EfMYrdhd2f0voAHwL+KY96\no4CnM+Znkw7KQvIX4mZgIlADtJC87Gwqybv3e9b5FXAhcCbJK7Wr0/KvAX+TTgfwznR6MsmIYD1v\nHpiU/rwLuDCdngms7SXe95G8IuH4dP4SYDEgkj+afkEyuMz5JONB1ADjgY3Adek6S4H6dHoK0JxO\nXwT8Ip2eAFSl028AftzH/g8dr4wYZwFr059VwISMfTWlsR6xXtZx/xhwZzp9BrAl/R69/ntk7buv\nY/sd4B0Z9ZYCX8uY7/XYA58E/gSMTuPfAVTne3zT+Wbg2nT6fwDfzlj2buD/Fvv/y0j6+BXldqym\nALv6Wf7biNgNIGkNMCsi/jP9K/OVJCeJM4A/AlcD5wHLkrtgjCEZPhKgC/hxOr0b6AD+WUkbQU87\nwRuAeem6ABMkjYuIvVkxPRgRO9PpS9LPinR+HDCX5AT284joADokDXRciInAdyXNJUl01X3s/wiS\naoAfkZwcWyRVA5+V9Fqgm2Tcgt5uBWa6EPi/ABGxTlIL0NPWdNS/B0eOg9DXse3NDzOmez326fQv\nI+IAcEDSM2n8FzCw4/uT9Gcj8PaM8mfI75aVDRInC8u2GsjnHvh+kr8O+3IgY7qLw79rdwPvBNYB\nP42IUHKm+W5E9DYATkekbSIR0SlpAfD6NMZrgP9CcmXwyvQE1J/nM6YFfC4ivplZQdJH+lm/k8O3\nbvv67p8GHoqItykZ1WxpH/vP9g2S9p/fpPPvJrkSOy8iDqa3vPo73rn09e8B9Htse5P5PXo99mny\n6HefA4w7e/0akt9BGyJus7BsvwNGS1rUUyDp5ZJek1kpItqByvQv4oH4Kcmwj1eRJA5I2hDeIemE\ndH/HS5qVvWL6F+vEiFgCfBQ4O130AMlIZj31zskjjvuBv+35K1jS9HT/fyRpPK1Jl2XeF28muQKC\nvhPqRA6PJ/C+POJA0tXA+Ii4LWs7z6SJ4mKSKwGAPSRXP735A0mSQcnTazNJbi3lE0Nfx7a//cHA\nj31/xzfXvjKdBozIp++KxcnCjhDJDeG3AW9Q8ujsauBzwNO9VH+A5NbHQLbfTnpfPiL+kpatAW4E\nHpD0OPAgcHIvq48HfpHW+U/gf6blHwLq04bZNeQx+lpEPEByv/1hSSuBe0lO2MtIxgN4nKRNZSXJ\nLRqALwL/IGkFyW243nwe+FxaJ9+/pK8Dzspo5P4g8G/pd1oJ/A3JlRiRjC73R0mrlPGwQOprQEW6\nzg+B96W3gfLR17G9G/h42mB+ai/rDejY5zi+3wG+kdXA3ZeLSZ6KsiHiV5TbMZN0LvDRiPjrYscy\nmHraO5Q8mvl7YFFELC92XMPFiz2+kk4E7oqI1xcsSDuK2yzsmEXEckkPSaqMIeprMUQWK+mIVkPS\nluJEMbhe7PGdSfLklw0hX1mYmVlObrMwM7OcnCzMzCwnJwszM8vJycLMzHJysjAzs5ycLMzMLKf/\nD+ly5UDvw8L6AAAAAElFTkSuQmCC\n", 239 | "text/plain": [ 240 | "" 241 | ] 242 | }, 243 | "metadata": {}, 244 | "output_type": "display_data" 245 | } 246 | ], 247 | "source": [ 248 | "train_errors = []\n", 249 | "test_errors = []\n", 250 | "\n", 251 | "C_list = [0.0001, 0.001, 0.01, 0.1, 10, 100, 1000]\n", 252 | "\n", 253 | "# Evaluate the training and test classification errors for each value of C\n", 254 | "\n", 255 | "for value in C_list:\n", 256 | " \n", 257 | " # Create LogisticRegression object and fit\n", 258 | " logistic_regression = linear_model.LogisticRegression(C= value, penalty = 'l1')\n", 259 | " logistic_regression.fit(X_train, y_train)\n", 260 | " \n", 261 | " # Evaluate error rates and append to lists\n", 262 | " train_errors.append(logistic_regression.score(X_train, y_train) )\n", 263 | " test_errors.append(logistic_regression.score(X_test, y_test))\n", 264 | " \n", 265 | "# Plot results\n", 266 | "plt.semilogx(C_list, train_errors, C_list, test_errors)\n", 267 | "plt.legend((\"train\", \"test\"))\n", 268 | "plt.ylabel('Accuracy Score')\n", 269 | "plt.xlabel('C (Inverse regularization strength)')\n", 270 | "plt.show()" 271 | ] 272 | }, 273 | { 274 | "cell_type": "markdown", 275 | "metadata": {}, 276 | "source": [ 277 | "## Scaling your data" 278 | ] 279 | }, 280 | { 281 | "cell_type": "code", 282 | "execution_count": 16, 283 | "metadata": {}, 284 | "outputs": [ 285 | { 286 | "data": { 287 | "text/plain": [ 288 | "0.99729539040451554" 289 | ] 290 | }, 291 | "execution_count": 16, 292 | "metadata": {}, 293 | "output_type": "execute_result" 294 | } 295 | ], 296 | "source": [ 297 | "#Setting up the scaling pipeline \n", 298 | "\n", 299 | "pipeline_order = [('scaler', StandardScaler()), ('logistic_reg', linear_model.LogisticRegression(C = 10, penalty = 'l1'))]\n", 300 | "\n", 301 | "pipeline = Pipeline(pipeline_order)\n", 302 | "\n", 303 | "#Fitting the classfier to the scaled dataset \n", 304 | "\n", 305 | "logistic_regression_scaled = pipeline.fit(X_train, y_train)\n", 306 | "\n", 307 | "#Extracting the score \n", 308 | "\n", 309 | "logistic_regression_scaled.score(X_test, y_test)" 310 | ] 311 | }, 312 | { 313 | "cell_type": "markdown", 314 | "metadata": {}, 315 | "source": [ 316 | "## Interpreting the results of the model " 317 | ] 318 | }, 319 | { 320 | "cell_type": "code", 321 | "execution_count": 18, 322 | "metadata": {}, 323 | "outputs": [ 324 | { 325 | "name": "stdout", 326 | "output_type": "stream", 327 | "text": [ 328 | "[[ 4.80188666e-05 1.59768979e-01 2.51418163e-01 -4.70355274e-06\n", 329 | " 2.36326041e-05 -3.43658187e-05 -1.55507920e-06 -8.30036365e-08\n", 330 | " -1.13670693e+01 -9.12306047e+00 -1.67613709e+01 -1.17033896e+01\n", 331 | " -9.11703172e+00]]\n" 332 | ] 333 | } 334 | ], 335 | "source": [ 336 | "#Printing out the coefficients of each variable \n", 337 | "\n", 338 | "print(logistic_regression.coef_)" 339 | ] 340 | }, 341 | { 342 | "cell_type": "code", 343 | "execution_count": 19, 344 | "metadata": {}, 345 | "outputs": [ 346 | { 347 | "name": "stdout", 348 | "output_type": "stream", 349 | "text": [ 350 | "[ 2.68799332]\n" 351 | ] 352 | } 353 | ], 354 | "source": [ 355 | "#Printing out the intercept of the model\n", 356 | "\n", 357 | "print(logistic_regression.intercept_)" 358 | ] 359 | } 360 | ], 361 | "metadata": { 362 | "kernelspec": { 363 | "display_name": "Python 3", 364 | "language": "python", 365 | "name": "python3" 366 | }, 367 | "language_info": { 368 | "codemirror_mode": { 369 | "name": "ipython", 370 | "version": 3 371 | }, 372 | "file_extension": ".py", 373 | "mimetype": "text/x-python", 374 | "name": "python", 375 | "nbconvert_exporter": "python", 376 | "pygments_lexer": "ipython3", 377 | "version": "3.6.1" 378 | } 379 | }, 380 | "nbformat": 4, 381 | "nbformat_minor": 2 382 | } 383 | -------------------------------------------------------------------------------- /Chapter_04.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Predicting categories with Naive Bayes & SVMS\n", 8 | "\n", 9 | "**Aim**: The aim of this notebook is to provide code-based examples for the implementation of the Naive Bayes & Linear Support Vector Machines using scikit-learn. \n", 10 | "\n", 11 | "## Table of contents \n", 12 | "\n", 13 | "1. Naive Bayes Classifier\n", 14 | "2. Linear Support Vector Machines" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "## Package Requirements " 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": 31, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "import pandas as pd\n", 31 | "import matplotlib.pyplot as plt\n", 32 | "from sklearn.model_selection import train_test_split\n", 33 | "from sklearn.model_selection import GridSearchCV\n", 34 | "from sklearn.naive_bayes import GaussianNB\n", 35 | "from sklearn.svm import LinearSVC\n", 36 | "from sklearn.preprocessing import StandardScaler\n", 37 | "from sklearn.pipeline import Pipeline" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "## Naive Bayes Classifier" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": 3, 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "df = pd.read_csv('fraud_prediction.csv')\n", 54 | "\n", 55 | "df = df.drop(['Unnamed: 0'], axis = 1)\n", 56 | "\n", 57 | "#Creating the features \n", 58 | "\n", 59 | "features = df.drop('isFraud', axis = 1).values\n", 60 | "target = df['isFraud'].values" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "**Splitting the data into training & test sets**" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": 6, 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "X_train, X_test, y_train, y_test = train_test_split(features, target, test_size = 0.3, random_state = 42, \n", 77 | " stratify = target)" 78 | ] 79 | }, 80 | { 81 | "cell_type": "markdown", 82 | "metadata": {}, 83 | "source": [ 84 | "**Building the Naive Bayes Clssifier**" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": 11, 90 | "metadata": {}, 91 | "outputs": [], 92 | "source": [ 93 | "#Initializing an NB classifier\n", 94 | "\n", 95 | "nb_classifier = GaussianNB()" 96 | ] 97 | }, 98 | { 99 | "cell_type": "code", 100 | "execution_count": 12, 101 | "metadata": {}, 102 | "outputs": [ 103 | { 104 | "data": { 105 | "text/plain": [ 106 | "GaussianNB(priors=None)" 107 | ] 108 | }, 109 | "execution_count": 12, 110 | "metadata": {}, 111 | "output_type": "execute_result" 112 | } 113 | ], 114 | "source": [ 115 | "#Fitting the classifier into the training data\n", 116 | "\n", 117 | "nb_classifier.fit(X_train, y_train)" 118 | ] 119 | }, 120 | { 121 | "cell_type": "code", 122 | "execution_count": 13, 123 | "metadata": {}, 124 | "outputs": [ 125 | { 126 | "data": { 127 | "text/plain": [ 128 | "0.87735183443085607" 129 | ] 130 | }, 131 | "execution_count": 13, 132 | "metadata": {}, 133 | "output_type": "execute_result" 134 | } 135 | ], 136 | "source": [ 137 | "#Extracting the accuracy score from the base classifier\n", 138 | "\n", 139 | "nb_classifier.score(X_test, y_test)" 140 | ] 141 | }, 142 | { 143 | "cell_type": "markdown", 144 | "metadata": {}, 145 | "source": [ 146 | "## Linear Support Vector Machines" 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": 14, 152 | "metadata": {}, 153 | "outputs": [], 154 | "source": [ 155 | "df = pd.read_csv('fraud_prediction.csv')\n", 156 | "\n", 157 | "df = df.drop(['Unnamed: 0'], axis = 1)\n", 158 | "\n", 159 | "#Creating the features \n", 160 | "\n", 161 | "features = df.drop('isFraud', axis = 1).values\n", 162 | "target = df['isFraud'].values" 163 | ] 164 | }, 165 | { 166 | "cell_type": "code", 167 | "execution_count": 15, 168 | "metadata": {}, 169 | "outputs": [], 170 | "source": [ 171 | "X_train, X_test, y_train, y_test = train_test_split(features, target, test_size = 0.3, random_state = 42, \n", 172 | " stratify = target)" 173 | ] 174 | }, 175 | { 176 | "cell_type": "markdown", 177 | "metadata": {}, 178 | "source": [ 179 | "**Building the linear SVM**" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": 19, 185 | "metadata": {}, 186 | "outputs": [], 187 | "source": [ 188 | "#Initializing a SVM model \n", 189 | "\n", 190 | "svm = LinearSVC(random_state = 42)" 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": 20, 196 | "metadata": {}, 197 | "outputs": [ 198 | { 199 | "data": { 200 | "text/plain": [ 201 | "LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n", 202 | " intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n", 203 | " multi_class='ovr', penalty='l2', random_state=42, tol=0.0001,\n", 204 | " verbose=0)" 205 | ] 206 | }, 207 | "execution_count": 20, 208 | "metadata": {}, 209 | "output_type": "execute_result" 210 | } 211 | ], 212 | "source": [ 213 | "#Fitting the model to the training data\n", 214 | "\n", 215 | "svm.fit(X_train, y_train)" 216 | ] 217 | }, 218 | { 219 | "cell_type": "code", 220 | "execution_count": 21, 221 | "metadata": {}, 222 | "outputs": [ 223 | { 224 | "data": { 225 | "text/plain": [ 226 | "0.98541862652869239" 227 | ] 228 | }, 229 | "execution_count": 21, 230 | "metadata": {}, 231 | "output_type": "execute_result" 232 | } 233 | ], 234 | "source": [ 235 | "#Extracting the accuracy score from the training data\n", 236 | "\n", 237 | "svm.score(X_test, y_test)" 238 | ] 239 | }, 240 | { 241 | "cell_type": "markdown", 242 | "metadata": {}, 243 | "source": [ 244 | "**Graphical hyper-parameter optimization**" 245 | ] 246 | }, 247 | { 248 | "cell_type": "code", 249 | "execution_count": 26, 250 | "metadata": {}, 251 | "outputs": [ 252 | { 253 | "data": { 254 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEOCAYAAACjJpHCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd8VuXZwPHflZ2QQCYzjCAbVEYE\nByqIA9CK2ta6am19X9q6uuStfupo6dBaW62te1St4qh1UGU5QNGiEDZh74QwwgiE7HG9f5wTfAgZ\nT8jz5ORJru/n83xyzn3WdQ7huXLu+5z7FlXFGGOMOVlhXgdgjDEmtFkiMcYY0yyWSIwxxjSLJRJj\njDHNYonEGGNMs1giMcYY0yyWSIwxxjSLJRJjjDHNYonEGGNMs1giMcYY0ywRXgfQElJTU7VPnz5e\nh2GMMSFl6dKl+1U1rbH12kUi6dOnD1lZWV6HYYwxIUVEdviznlVtGWOMaRZLJMYYY5rFEokxxphm\naRdtJHWpqKggNzeX0tJSr0MJqpiYGNLT04mMjPQ6FGNMG9VuE0lubi4JCQn06dMHEfE6nKBQVQ4c\nOEBubi4ZGRleh2OMaaPabdVWaWkpKSkpbTaJAIgIKSkpbf6uyxjjrXabSIA2nURqtNZzzD1UzMa9\nhV6HYYwJgHadSLxUUFDAE0880eTtJk+eTEFBQRAiajnLdh5i8l8XcvEjnzHx0c94csEW8gpKvA7L\nGHOSLJF4pL5EUllZ2eB2s2bNIjExMVhhBd0Xm/dzw3NfkdwhinsuHUxcVDh/nLOesx/8hO88vYjX\nFu/kcHGF12EaY5qg3Ta2e+2uu+5iy5YtDB8+nMjISGJiYkhKSmL9+vVs3LiRK664gpycHEpLS/nJ\nT37C1KlTga/f0j969CiTJk1i7Nix/Pe//6VHjx689957xMbGenxm9ZuXvYfbZiynb2ocb47eSMfi\n+fzP5IvZ0WEs763ax7srdnH326u5/71sxg9K44rhPRg/qDMxkeFeh26MaYCoqtcxBF1mZqbW7iJl\n3bp1DB48GIDf/CebtXlHAnrMId07cv83hta7fPv27Vx22WWsWbOGBQsWcOmll7JmzZpjT1cdPHiQ\n5ORkSkpKOOOMM/j0009JSUk5LpH069ePrKwshg8fztVXX83ll1/ODTfccMKxfM/VK+8sz+XOf61i\nRPc4Xu32BtGrZ4CEgVZDTCL0vxgdOInsuEzezj7KzJV57D9aRkJMBJOHdeOKET0Yk5FMWFjrbPMx\npi0SkaWqmtnYenZH0kqMHj36uEd0H3vsMd555x0AcnJy2LRpEykpKcdtk5GRwfDhwwEYNWoU27dv\nb7F4m+Kfi7Zz73vZXJQRxZNRfyJi9edw/i/hrNtg63zYMBs2zkVWv8mwsAiG9RnLryZMIit6DG9s\nFt5flccbWTl06xTD5ad354oRPRjcraPXp2WaqKyyiryCUnIPFZNzsITcQ8XkHiqhqKySm8/N4OxT\nUr0O0ZwkSyTQ4J1DS+nQocOx6QULFvDRRx+xaNEi4uLiGDduXJ2P8EZHRx+bDg8Pp6Sk9TVYP7Fg\nMw/N2cB1/Sr4XfF9hO3bCVc+A6d/x1lhyBTnU10FOYth42zYMJvwub9kDDCm81Aqxl7CosjRvLw9\nnuc/38bTn21lYJcEpozozpThPeiR2Hqr89qTiqpq8gpKyD30dZLIOVjszpewt7AU3wqQiDChe2Is\nZZVVXPfsV1x+enfuuXQwnTvGeHcS5qQENZGIyETgr0A48JyqPlhreW/gBSANOAjcoKq5IjIeeMRn\n1UHANar6roi8CJwPHHaX3aSqK4J5HsGQkJBAYWHdj78ePnyYpKQk4uLiWL9+PV9++WULR9d8qsof\n52zgqU+38LMB+7kj/9cIwI3vQe+zT9wgLBx6n+V8LpoOB7a4dypziFz0V87TKs7r0JnSURexKOIM\nnt0Vy0NzNvDQnA2MzkjmyhE9mDysG53i7A3+YKmsqmb34VInQRwq/jphuHcXe46UUu2TKMLDhG6d\nYkhPimVs/1R6JsWRnhRLelIsPZPj6NIxhvAwobSiiicWbOGpBVv4ZP0+fnbRAL53Vm8iwu1ZoFAR\ntEQiIuHA48BFQC6wRERmqupan9UeBl5W1ZdE5ALgAeC7qjofGO7uJxnYDMzz2W6aqr4VrNhbQkpK\nCueccw7Dhg0jNjaWLl26HFs2ceJEnnrqKQYPHszAgQM588wzPYy06aqrlftmruGVL3fy0IB1fHvX\nH5HEXnDdm5Byin87STkFzr7N+RQfhM0fwYbZxGz8D+PLXmV8RAwlg8ayKGI0T+7ux91vH+T+97IZ\nNzCNK0dYI/3JqKpW9hwpJde9i/BNFjkHS9hzpJQqn0wRJtC1YwzpyXGceUoK6Ulx9EyKJd1NGN06\nxfiVDGIiw/n5RQO4akQP7p+ZzW/fX8u/snL43RXDyOyTHMxTNgEStMZ2ETkL+LWqXuLO3w2gqg/4\nrJMNTFTVHHHenDusqh1r7WcqcL6qXu/Ovwi835RE0lhje1vXkudaUVXNtH+t5N0Vu3jllE8Yu+t5\n6HMuXP0yxAXgS6GyHHb+17lb2TAbCpzhEopTT2Nx1Gie3TeIL452IyEmkknDunLFiB6cmZFijfQ4\nCX5vYelxdxJfJ4sS8gpKqPRJFCLQJSHm2B3EsbuJpDjSk+Lo2imGqAg/7xqqKqHkEBTvh6L9Pj8P\nQlU5nH4NpPZHVZmbvYfp/1lL3uFSvj0qnbsmDSIlPrrxY5iAaw2N7T2AHJ/5XGBMrXVWAlfhVH9d\nCSSISIqqHvBZ5xrgL7W2+72I3Ad8DNylqmUBjdyclNKKKm5/bTmfrc1hXq/XGLBrLgy/AS57BCKi\nAnOQiCjoO875THwQ9q2DjbOJ2zCbcbnPMw6lNLkbS6PH8PKqwdyUNYjkjglcPrw7VwzvweBuCa32\nbf/mqq5W9h8tI8enjcK3YTuvoJTyqurjtumcEE16UizDeybyjdO7Hbub6JkUR7fEGKIj6rmrqyiF\nojwoPuAmhAO1EsSB48tKCoB6/miVMFj4ZxgyBTn3F0wcdhrnDUjjsY8389zCrcxbu5dplwzk2tG9\nCLc/CFqlYN6RfAvnbuN/3PnvAmNU9TafdboDfwcygM+AbwLDVLXAXd4NWAV0V9UKn7I9QBTwDLBF\nVafXcfypwFSAXr16jdqx4/iBvuyOJLCKyiqZ+s8s1m3expyuT9G5YAVMuB/G/sz507YlHN0HG+fC\nxjmw5ROoKKYyIo4VUaN448hQPq4cTmoX56mvUGykV1X2Hy3/uiHb524i92AxuQUllFcenyhS46OO\nJYf0pDh6Jn9d9dQjMdap/lOFskI3CRxwfh6XGGqXHYDyo3UHKeEQl+J8OqT6/KyZTnGmj5UlO0nm\nyydgyXNQdgT6Xwzn3gm9xrB5XyH3vpvNoq0HOC29E7+dMozTe4buC7mhxt87Ek+rtmqtHw+sV9V0\nn7KfAENVdWo924wD7lTVyxqKxaq2gnuuh4sruOnFxRTlruWdxEfpUL4frnwahl4RtGM2qqIEti2E\nDbOcxFK4m2rC2BA5iHeLT+ej6pGk9BrGFSPTmXxqVxLjAnTH1AyqyqHiCp8nnYqPTxaHiimtOD5R\nJHeI8qlucqqe0hOj6R1XTvfIImLKDx1fjXRc1ZJ711C836leqktEjPvF7yaHY0nAJ1H4lsUkQthJ\nNpKXFMCSZ2HRE1ByEHqPhfN+gWaMY+aq3fzug3XsP1rGdaN7Me2Sga3i36ytaw2JJALYCEwAdgFL\ngOtUNdtnnVTgoKpWi8jvgSpVvc9n+ZfA3W7je01ZN1Xd7bapPAKUqupdDcViiSR457qvsJQbn19M\nl/1f8lzsY0RGxcK1r0P6qKAc76Sowu4VsGGOk1j2rAIgV7oxp2I488kkof9YLh/ZmwuC2Eivqhwu\nqaj1WOzxdxfF5VXHbZMYF0mfxAgGJZRzSnwZvWNK6B5ZRFp4IUl6hKjyQydWLZUccl70rEt0x+Pv\nEk5IELXKojq03B1ljfIiWPoi/PdvULgbuo+E8+6ksPeFPPLRFl5atJ1OsZHcNWkQ3xqZbu1fQeR5\nInGDmAw8ivP47wuq+nsRmQ5kqepMt/rrAZzK08+AW2vaO0SkD/AF0FP16/8VIvIJzuPCAqwAfqSq\n9dxnOyyRBOdccw8Vc8NzX3Fu4Sx+E/4CYWkD4Lo3ILFXwI8VUIdzYeMcdMMcdOunhFWXc4QOfFJ1\nOl+EnUHM4IuZlDmIMX1Tmlwn7ySKr+8iaieMo2UVxFFGshwhhSP0iC7mlLgyesUUH5cgOlQdJqb8\nEGElB5zqnrpIGMQm+9wZ+CaIOsriUgLXVtUSKstgxQz44lE4tB06D4GxP2dt8gTu+896snYcYlTv\nJH47ZRhDutsLqsHQKhJJa2GJJPDnuiX/KDc+u4j/KX+Z7zMT+l0I3/oHxITYf+iyo7B1PtXrZ1G5\nfg5RZQep0HAWVw/iq6jRxAy9jPPPPIMh3ToiIhwtqzz+zeyDRRzYv4+iQ3spP7KPmPJDJMsRkigk\nRQrpHF5I98giUsMKSdRCOlQVEFFdz7Mh4VE+dwSpddw51CqLTXTev2nrqioh+22nQT5/PSRlUH3O\nT3mn6lx+P28rBcXlfO/sPvz8ogEkxNh7RIFkicRHa0wkBQUFzJgxg1tuuaXJ2z766KNMnTqVuLg4\nv9YP9Llm5x1m6nML+a3+lQt0MWTeDJMegvAQ7yihugpys6hcP4uS1f8hoXALABuq01kZORytqqBD\nVQEpFJIsR9yEcZQIqbsaSaPiIS4FaeguwbcsOqHlq5FCSXU1bPgAPnvYqapM6E7xGbfwUP6ZvJS1\nj9T4aO65dDCXn969zT6Z19IskfhojYnEt9PGpqrpuDE11b++iQJ5rkt3HOTOf8zjCXmIQboVmfgA\njPlR2/wCPLiV4tXvc3jlf0g9tJzysDjKo5Oojk0lPD6FmE6die7U5etEEZd8fDVSpHX1ERSqzlN5\nC/8MO76AuBR2D/4BP9t2Bl/mVXJW3xSmTxlK/y4JXkca8lrDeySmAb7dyF900UV07tyZN998k7Ky\nMq688kp+85vfUFRUxNVXX01ubi5VVVXce++97N27l7y8PMaPH09qairz589v/GAB8vmm/Tz88lu8\nHvEn0iKKkW+9BgMntdjxW1xyX+LOv4O48+8AIBLo0PAWpiWIQL8JzmfHIlj4MN2W/onXojuSPfQ7\n3LrlTCb99SA3n5vBHRf0p0O0fc0Fm11hgNl3wZ7Vgd1n11Nh0oP1Ln7wwQdZs2YNK1asYN68ebz1\n1lssXrwYVeXyyy/ns88+Iz8/n+7du/PBBx8ATh9cnTp14i9/+Qvz58/3+44kEOZm7+Gt115gRsRj\nRHdIJOz6OdDt9BY7vjF16n0W9P435K1AFv6ZYeueY0HEqyxMvpRpn57PzBV53HfZECYO62rVXUFk\nvaK1AvPmzWPevHmMGDGCkSNHsn79ejZt2sSpp57Khx9+yC9/+UsWLlxIp06dPInv7WW5fPnaH3gq\n4k9Ed+5H+NRPLImY1qX7cPjOP+HWr5ChUzjv0Nssivs59+rTPDBjNt/7xxK27S/yOso2y+5IoME7\nh5agqtx999388Ic/PGHZsmXLmDVrFvfccw8TJkzgvvvuq2MPwfPKF5upnH0390fMo7L/RCK+/YLz\nboExrVHaQLjyKRh3F2Ff/JVJy19hYvSHzNpxNrc+MoULzz+fW8b3sw49A8zuSDzi2438JZdcwgsv\nvMDRo87rMLt27WLfvn3k5eURFxfHDTfcwLRp01i2bNkJ2waLqvLMhyvpPudmboqYR+WYW4m4doYl\nERMakvrAZY8gP1lF2Fm3MDlqGbMipzFk4a3c9vDzfLxur9cRtil2R+IR327kJ02axHXXXcdZZ50F\nQHx8PK+88gqbN29m2rRphIWFERkZyZNPPgnA1KlTmThxIt27dw9KY7uq8sR7C7hg2R0MCN9F1eS/\nEDH65oAfx5ig69gNLvk9Yef+Ar56igsXPcnEsml8NuNl/tjjB1x39TX0TPbvMXpTP3v8tx1oyrlW\nVStPvfYW3954Jx3DK4i69p+E9Z8Q5AiNaSGlR6hc/BzlC/9GXMVBsnQgeafewiVTbiA6MvT/ri6r\nrGJ7fhE7dm7jQO4mSvZt5ZvX/YhOnU7uRWF7/Nc0WUVVNS+/8Hd+kPtbymJSiP7BbKTLEK/DMiZw\nYjoScd7PiTjrxxR88Tx9P/8rmWt+wqa1f6XynJ8xePwNJ9/pZAsqO3qQvO3r2b9zI8V7t6KHthNT\nlEtK5R56kc9A+boTzp154+nU6YygxmOJxABQWl7Jf568i+8ffI59nYbRderbEN/Z67CMCY7IWBLH\n3QZjp7Lhw+eIW/w3ei68nT1f/YmY8dNIHH0thHvY3UpFCRTspHz/Ng7kbqRo7xaqD+4g5mgOSeW7\nSaCIDJzxNwCO0oGDUd0o7dSfXUkX0qFLP1LS+xOVmkGv5L5BD9cSieFocQmL/34T3y6ew/Zul9Dn\n5pcgMrTG6jDmpEREMXDSLZRe8D/MfudZMtY9xaC5t1P46QPEjf854SO/G5weCqoq4UguHNoBBTuo\nPLCdo3u3UH1wO1GFOcRXOGP7RQHdgFKNJJc0dkd0Y2vHoUhSb+K6nEJqen+69RlEfMcU4gMfpd/a\ndSJR1Tb/klJjbWCHD+5j+5Pf5oKKFawf8EMGXfNgSNzaGxNIMdFRTLrmVnbuv4nH3nyec/e8xIjZ\nd1K+4CGizr0DRn0fopvwVa0KR/ceSxTOz+1UHdxB1YHtRBTlEaY+QwZoGIWaQo6mkcupFMX1QBN7\nE5vWl5Se/enVK4OMtHj61TdipcfabWP7tm3bSEhIICUlpc0mE1XlwIEDFBYWkpGRccLy/TnrKf7H\nN+latZuNo3/PsEt/7EGUxrQ+H2bvYea7r3NN6ZucE55NdUwSYWf+GMZMhdgkZ6WSQ7USxdc/tWAn\nUll63D4PSBLbq1LJ0TRytDN5dKayY09iOvclpXtf+nVNZECXBPqkdCAqonX8MWedNvqoK5FUVFSQ\nm5tLaWlpPVu1DTExMaSnpxMZeXx9777sT4n61w2g1eRc/AynnnOpRxEa0zqVlFfx+PzNfLVwLreE\nv8t4WYpGxSPJGXBoJ5QdPn798AT2hXdhe1UqG8tS2Klp5Ggae6QLESm96d01lf6d4xnQJYH+nePp\n3YoSRn0skfioK5G0Z3u+eIWkD3/KblI5etUMhp020uuQjGm1tuYf5f6Z2eRvXsadCfPoE1vKjuo0\n1pUksbo4kVztTI6mURqeQN+0DvTrHE//zgkM6BJP/y5OwogMb90Joz72+K85kSp7359O16V/YSlD\niL/xdYb17e11VMa0an3T4nn5B6OZtboX934wgAP7y+mb1oH+pyQwrHM8V3aJp3+XBHonxxERogmj\nuYKaSERkIvBXnKF2n1PVB2st7w28gDN07kHgBlXNdZdVATVd8u5U1cvd8gzgdSAFWAp8V1XLMQ2r\nLOPAjKl02fous8LGM/iHL5DRJdnrqIwJCSLCpad1Y9KwrlSrttuEUZ+gXQ0RCQceByYBQ4BrRaT2\n220PAy+r6mnAdJzx22uUqOpw93O5T/kfgUdUtR9wCLC+OxpTdIDDT08mZeu7PB91PaffPsOSiDEn\nISxMLInUIZhXZDSwWVW3uncMrwNTaq0zBPjEnZ5fx/LjiPN41QXAW27RS8AVAYu4Ldq/iaInxhGz\nbyUPdPg/Lr/9EXokWd9CxpjACWYi6QHk+MznumW+VgJXudNXAgkikuLOx4hIloh8KSI1ySIFKFDV\nygb2aWps+4zypy+g5GgB9yc/yC23/h9pCdFeR2WMaWO8bmy/E/i7iNwEfAbsAmre0umtqrtEpC/w\niYisBg7XvZsTichUYCpAr169Ahp0SFj+CtUzf8L2qi482f0P/O77l9mQo8aYoAjmHckuoKfPfLpb\ndoyq5qnqVao6AviVW1bg/tzl/twKLABGAAeARBGJqG+fPvt+RlUzVTUzLS0tYCfV6lVXox/+Gt67\nlc8rB/N43yd44OZvWBIxxgRNMBPJEqC/iGSISBRwDTDTdwURSRWRmhjuxnmCCxFJEpHomnWAc4C1\n6rz0Mh/4lrvN94D3gngOoaW8GP3X95AvHuHVygnMHPoof/7ueTYanDEmqIKWSNx2jNuAucA64E1V\nzRaR6SJS8xTWOGCDiGwEugC/d8sHA1kishIncTyoqmvdZb8Efi4im3HaTJ4P1jmElPIi9MXL0HX/\n4bcV17N+1G946OpR9oSJMSbo7M32NqJ65RuEvTOVO8pvJf28G5l2ycA224eYMaZl2Jvt7cze5bOI\n1nj6X3Ajt184yOtwjDHtiNV7tAWqxOcuZJGextRx/b2OxhjTzlgiaQv2ZpNQeYC9aWcT3UrHKzDG\ntF2WSNqAQ6tnAxA/5CKPIzHGtEeWSNqA0vUfsaE6nczThnkdijGmHbJEEurKi0k9uJTlkSPISO3g\ndTTGmHbIEkmIq9j2OZFaQUmvcfa4rzHGE5ZIQlz+8lmUaSTpwyd4HYoxpp2yRBLionYsYIkO4syB\n6V6HYoxppyyRhLLDu0gt2ca2TmNIiIn0OhpjTDtliSSEFWbPAyBiwIUeR2KMac+si5QQVrBmLsWa\nyLDhZ3kdijGmHbM7klBVXUXyni9YHDacoT06eR2NMaYds0QSoqp3raBD9REOdR1LWJg99muM8Y4l\nkhCVv2IWAMmnXeJxJMaY9s4SSYiq2vwxq6v7MGbYQK9DMca0c5ZIQlHpETofXsXauDNIS4j2Ohpj\nTDsX1EQiIhNFZIOIbBaRu+pY3ltEPhaRVSKyQETS3fLhIrJIRLLdZd/x2eZFEdkmIivcz/BgnkNr\nVLJxPhFUUZUx3utQjDEmeIlERMKBx4FJwBDgWhEZUmu1h4GXVfU0YDrwgFteDNyoqkOBicCjIpLo\ns900VR3uflYE6xxaq/0rZ1Ok0fQdcYHXoRhjTOOJRES+LSIJ7vQ9IvK2iIz0Y9+jgc2qulVVy4HX\ngSm11hkCfOJOz69ZrqobVXWTO50H7APS/Dmh9iAu5zOWMJSRfbt4HYoxxvh1R3KvqhaKyFjgQuB5\n4Ek/tusB5PjM57plvlYCV7nTVwIJIpLiu4KIjAaigC0+xb93q7weEZF21UigB7aSUr6LvJSziIqw\nJi5jjPf8+Saqcn9eCjyjqh/gfLEHwp3A+SKyHDgf2OVzPESkG/BP4PuqWu0W3w0MAs4AkoFf1rVj\nEZkqIlkikpWfnx+gcL13YNUcAGKHXOxxJMYY4/AnkewSkaeB7wCz3DsAv7YDevrMp7tlx6hqnqpe\npaojgF+5ZQUAItIR+AD4lap+6bPNbnWUAf/AqUI7gao+o6qZqpqZltZ2asWK180jV1MZOTzT61CM\nMQbwLyFcDcwFLnG/5JOBaX5stwToLyIZIhIFXAPM9F1BRFJFpCaGu4EX3PIo4B2chvi3am3Tzf0p\nwBXAGj9iaRuqKkjL/4plkSPpnRrvdTTGGAP4kUhUtRinsXusW1QJbPJju0rgNpwktA54U1WzRWS6\niFzurjYO2CAiG4EuwO/d8quB84Cb6njM91URWQ2sBlKB3zV+mm1D+Y7FxGoxRenneR2KMcYc02jv\nvyJyP5AJDMSpSooEXgHOaWxbVZ0FzKpVdp/P9FvAW3Vs94p7jLr22W6fed277AO6q9B1+ESvQzHG\nmGP8qdq6ErgcKIJjj+MmBDMoU7fwbfNZqf04Y3CG16EYY8wx/iSSclVVQAFEpENwQzJ1Kj5I16J1\nbOk0hvhoG0bGGNN6+JNI3nSf2koUkf8FPgKeDW5YprbD2R8ShhLWb4LXoRhjzHEa/dNWVR8WkYuA\nIzjtJPep6odBj8wc5+CqOaBxDBx5vtehGGPMcRpMJG5/WR+p6njAkodXVEncvZCssNMY3yPZ62iM\nMeY4DVZtqWoVUC0iNparh6r2rSepMp8DXWw0RGNM6+NPq+1RYLWIfIj75BaAqt4RtKjMcfYsm0UP\nIPFUGw3RGNP6+JNI3nY/xiOVGz9iS3U3Rp5+utehGGPMCfxpbH/J7bJkgFu0QVUrghuWOaailK6H\nljIv7mK+Ed+uOjo2xoQIf95sHwe8BGwHBOgpIt9T1c+CG5oBKNryBR0oo7zPOK9DMcaYOvlTtfVn\n4GJV3QAgIgOA14BRwQzMOPYu+4B0DafXCOs23hjTOvnzQmJkTRIBZ/RCnP62TAuI2fkpKxjE8H7p\nXodijDF18ieRZInIcyIyzv08C2QFOzADWriH7qWbyU05i8hwGw3RGNM6+VO19WPgVqDmcd+FwBNB\ni8gcs2/FHLoAMQMv9DoUY4yplz+JJAL4q6r+BY697W6PD7WAwuy5hGtHTs081+tQjDGmXv7Ul3wM\nxPrMx+J03GiCqbqatH3/ZUXkcHqm2GiIxpjWy59EEqOqR2tm3Om44IVkAMp2raRTdQGFPayTRmNM\n6+ZPIikSkZE1MyIyCijxZ+ciMlFENojIZhG5q47lvUXkYxFZJSILRCTdZ9n3RGST+/me7/FFZLW7\nz8fcsdvbnLylHwCQNty6RTHGtG7+JJKfAv8SkYUi8jnwBs5Y7A1y21IeByYBQ4BrRWRIrdUeBl5W\n1dOA6cAD7rbJwP3AGGA0cL+IJLnbPAn8L9Df/bTJcWdlyyes116MHDrY61CMMaZBjSYSVV0CDMJ5\neutHwGBVXerHvkcDm1V1q6qWA68DU2qtMwT4xJ2e77P8EuBDVT2oqodwurCfKCLdgI6q+qU7auPL\nwBV+xBJayovoUbiSzQmjiYuy0RCNMa1bo4lERL6N006yBudL+w3fqq4G9AByfOZz3TJfK4Gr3Okr\ngQQRSWlg2x7udEP7rIl7qohkiUhWfn6+H+G2HgezPyaSSjjFRkM0xrR+/lRt3auqhSIyFpgAPI9T\nvRQIdwLni8hy4HxgF1AViB2r6jOqmqmqmWlpaYHYZYvZv3IOpRpJv0x7f8QY0/r5k0hqvtgvBZ5V\n1Q+AKD+22wX09JlPd8uOUdU8Vb1KVUcAv3LLChrYdpc7Xe8+24KOeQtZHjaUgemhlQCNMe2TP4lk\nl4g8DXwHmCUi0X5utwToLyJ/bn1sAAAbo0lEQVQZbjf01wAzfVcQkVQRqdnX3cAL7vRc4GIRSXIb\n2S8G5qrqbuCIiJzpPq11I/CeH7GEjKpDO+lavpN9ncfSRh9IM8a0Mf4khKtxvtgvce8WkoFpjW2k\nqpU4T3fNBdYBb6pqtohMF5HL3dXGARtEZCPQBfi9u+1B4Lc4yWgJMN0tA7gFeA7YDGwBZvtxDiEj\nN8t57LfjMOvt1xgTGsR5+Klty8zM1Kys0OhncuPfriJh/3Jipq0nyQayMsZ4SESWqmpmY+tZl7Kt\nSXUV3Q9+RXbMKEsixpiQYYmkFTm6dTHxepTS3uO8DsUYY/zmz3skt/u8VW6CaNfSD6hWocfISV6H\nYowxfvPnjqQLsERE3nT7zrJHiYIkesd8sqUvp/bP8DoUY4zxmz9dpNyD06fV88BNwCYR+YOInBLk\n2NoVLTlEevFachLPJMJGQzTGhBC/vrHcfq32uJ9KIAl4S0QeCmJs7cruFfOIoJqoQRd5HYoxxjRJ\noz0CishPcF7824/z/sY0Va1wXyTcBPxfcENsHw6vmUuCxjL4DOtfyxgTWvzpWjYZuEpVd/gWqmq1\niFwWnLDaGVVS93zOqsjTOCelo9fRGGNMk/hTtTUbqHmrHBHpKCJjAFR1XbACa0/K9m0irWovBd3G\neh2KMcY0mT+J5EngqM/8UQLX+68Bdi55H4CU0+2xX2NM6PEnkYj69KOiqtX4VyVm/FS96WN2amdO\nP82fYV6MMaZ18SeRbBWRO0Qk0v38BNga7MDajcpyeh5eyqb40cRGhXsdjTHGNJk/ieRHwNk4437k\n4oyjPjWYQbUn+9cvJI4Sqvpe4HUoxhhzUhqtolLVfThjiZgg2Lt8NokaRt8zJnodijHGnBR/3iOJ\nAW4GhgIxNeWq+oMgxtVuxOd+ypqwgZzes7vXoRhjzEnxp2rrn0BX4BLgU5zhbQuDGVR7UXlkHz3L\nNrE37WwbDdEYE7L8SST9VPVeoEhVX8IZu31McMNqH3YunUUYSochNhqiMSZ0+ZNIKtyfBSIyDOgE\ndPZn525vwRtEZLOI3FXH8l4iMl9ElovIKhGZ7JZfLyIrfD7VIjLcXbbA3WfNMr9iaY2K133IIY1n\nWOb5XodijDEnzZ/3QZ5xxyO5B5gJxAP3NraRiIQDjwMX4TzttUREZqrqWp/V7sEZy/1JERkCzAL6\nqOqrwKvufk4F3lXVFT7bXa+qoTF2bn1U6bZ/EdkxIxgbH+t1NMYYc9IaTCRux4xHVPUQ8BnQtwn7\nHg1sVtWt7r5eB6YAvolEgZrOpToBeXXs51rg9SYcNyQc3rGKlOoDFPe0uxFjTGhrsGrLfYv9ZHv3\n7QHk+MznumW+fg3cICK5OHcjt9exn+8Ar9Uq+4dbrXVvfQNtichUEckSkaz8/PyTOoFgys1yukXp\nOmKyx5EYY0zz+NNG8pGI3CkiPUUkueYToONfC7yoqunAZOCf7l0QAG7nkMWqusZnm+tV9VTgXPfz\n3bp2rKrPqGqmqmampaUFKNzAidg2ny30YMigwV6HYowxzeJPIvkOcCtO1dZS9+NP+8QuoKfPfLpb\n5utm4E0AVV2E855Kqs/ya6h1N6Kqu9yfhcAMnCq0kKLlxfQuWsmOxLNsNERjTMjz5832kx1AfAnQ\nX0QycBLINcB1tdbZCUwAXhSRwTiJJB+Otc9cjXPXgVsWASSq6n4RiQQuAz46yfg8k7PyY3pRTnh/\n6xbFGBP6/Hmz/ca6ylX15Ya2U9VKEbkNmAuEAy+oaraITAeyVHUm8AvgWRH5GU7D+00+PQ2fB+TU\nNNa7ooG5bhIJx0kizzZ2Dq3NoVVz6aIRDBhj3aIYY0KfP4//nuEzHYNzB7EMaDCRAKjqLJxGdN+y\n+3ym1wLn1LPtAuDMWmVFwCg/Ym7VkncvJDtiKCNTU7wOxRhjms2fqq3jnqQSkUTa4OO4LaXkQA49\nK7ezscetXodijDEBcTItvUXAybabtHvbv3Ie+008zaq1jDFtgz9tJP/Bab8AJ/EMwX3SyjRd5aaP\nyddODB1xltehGGNMQPjTRvKwz3QlsENVc4MUT9tWXU3Pgq/I7jCac6IivY7GGGMCwp9EshPYraql\nACISKyJ9VHV7UCNrg/Zs/IqueoSKjPFeh2KMMQHjTxvJv4Bqn/kqt8w00Z5lHwDQ+wzrFsUY03b4\nk0giVLW8ZsadjgpeSG1X7M7P2CgZ9OltzyoYY9oOfxJJvohcXjMjIlOA/cELqW2qKD5M35I15KXa\naIjGmLbFnzaSHwGvisjf3flcoM633U39tmXNZYBUETvoIq9DMcaYgPLnhcQtwJkiEu/OHw16VG3Q\n0bXzKNZoBo22RGKMaVsardoSkT+ISKKqHlXVoyKSJCK/a4ng2pIu+75gXfRpdEqI9zoUY4wJKH/a\nSCapakHNjDtaoj121ASHcjfSozqPo+k2GqIxpu3xJ5GEi0h0zYyIxOL0wmv8tGOJ0y1K5xGTPI7E\nGGMCz5/G9leBj0XkH+789/Gj51/ztbCt89lNCgOGjPQ6FGOMCTh/Gtv/KCIrgQvdot+q6tzghtV2\naFUFGYVZrO40nm42GqIxpg3y65tNVeeo6p2qeidQJCKPBzmuNmP7yoUkUIycYqMhGmPaJr8SiYiM\nEJGHRGQ78FtgvZ/bTRSRDSKyWUTuqmN5LxGZLyLLRWSViEx2y/uISImIrHA/T/lsM0pEVrv7fExa\n+dt9B1bOokqFU8Zc6nUoxhgTFPVWbYnIAOBa97MfeAMQVfWrx0ERCQceBy7CeYlxiYjMdEdFrHEP\n8KaqPikiQ3BGU+zjLtuiqsPr2PWTwP8CX7nrTwRm+xOTFzrmfc7GiAEM7tLN61CMMSYoGrojWQ9c\nAFymqmNV9W84HTb6azSwWVW3uv1zvQ5MqbWOAh3d6U5AXkM7FJFuQEdV/dId2/1l4IomxNSiigr2\nc0r5eg50Get1KMYYEzQNJZKrgN3AfBF5VkQmAE2pRuoB5PjM57plvn4N3CAiuTh3F77D+ma4VV6f\nisi5Pvv0HQulrn22GlsXf0C4KB2HXex1KMYYEzT1JhJVfVdVrwEGAfOBnwKdReRJEQnUN+O1wIuq\nmo7zkuM/RSQMJ4H1UtURwM+BGSLSsYH9nEBEpopIlohk5efnByjcpinb8CGFGsvATBt/xBjTdjXa\n2K6qRao6Q1W/AaQDy4Ff+rHvXUBPn/l0t8zXzbjD9qrqIiAGSFXVMlU94JYvBbYAA9zt0xvZZ03c\nz6hqpqpmpqWl+RFugKmSfvBLNsaNJDrK3t80xrRdTXqxQVUPuV/QE/xYfQnQX0QyRCQKuAaYWWud\nncAEABEZjJNI8kUkzW2sR0T6Av2Braq6GzgiIme6T2vdCLzXlHNoKXlbVtNV8ynrY3cjxpi2zZ83\n20+KqlaKyG3AXCAceEFVs0VkOpClqjOBXwDPisjPcBreb1JVFZHzgOkiUoEzOuOPVPWgu+tbgBeB\nWJyntVrlE1u7st6nO5CeaY/9GmPatqAlEgBVnYXTiO5bdp/P9FrgnDq2+zfw73r2mQUMC2ykgRe1\nYwE50o2efQd7HYoxxgSV9dkRBOWlJfQvXkFu8lk2GqIxps2zRBIEm5d9TJyUEW2jIRpj2gFLJEFw\nZM1cKjSc/qMneh2KMcYEnSWSIEjd+wWbooaQ0CnZ61CMMSboLJEE2P49OfSr2sLhHuc2vrIxxrQB\nlkgCbNviDwBIO91GQzTGtA+WSAJt88cUkEDf0054qtkYY9okSyQBVF1VTcaRxWxJOIOw8HCvwzHG\nmBZhiSSAtmZ/RSoFqI2GaIxpRyyRBNC+FU5vLX1GX+ZxJMYY03IskQRQfO5nbA/rTWr3DK9DMcaY\nFmOJJEAKCw8zsGwN+zpbI7sxpn2xRBIgGxfPJVoq6DDURkM0xrQvlkgCpHT9h5RqJP0zLZEYY9oX\nSyQBoKp037+IzXGnERXbwetwjDGmRVkiCYCc7ZvI0BxKe53vdSjGGNPiLJEEQE6W0y1K91H22K8x\npv0JaiIRkYkiskFENovIXXUs7yUi80VkuYisEpHJbvlFIrJURFa7Py/w2WaBu88V7qdzMM/BH5Hb\n5rNfkujef6TXoRhjTIsL2lC7IhIOPA5cBOQCS0Rkpju8bo17gDdV9UkRGYIzLG8fYD/wDVXNE5Fh\nOOO+9/DZ7np3yF3PlZWXM6BoKVuTzyXVRkM0xrRDwbwjGQ1sVtWtqloOvA5MqbWOAh3d6U5AHoCq\nLlfVPLc8G4gVkeggxnrSNixbSKIcJXKgjYZojGmfgplIegA5PvO5HH9XAfBr4AYRycW5G7m9jv18\nE1imqmU+Zf9wq7XuFY8HRT+0Zg7VKmSMudTLMIwxxjNeN7ZfC7yoqunAZOCfInIsJhEZCvwR+KHP\nNter6qnAue7nu3XtWESmikiWiGTl5+cH7QRS9nzOtqh+xCd1DdoxjDGmNQtmItkF9PSZT3fLfN0M\nvAmgqouAGCAVQETSgXeAG1V1S80GqrrL/VkIzMCpQjuBqj6jqpmqmpmWlhaQE6otPz+fQRXrKeg2\nNij7N8aYUBDMRLIE6C8iGSISBVwDzKy1zk5gAoCIDMZJJPkikgh8ANylql/UrCwiESJSk2gigcuA\nNUE8hwZtXDybCKkm6TQbDdEY034FLZGoaiVwG84TV+twns7KFpHpInK5u9ovgP8VkZXAa8BNqqru\ndv2A+2o95hsNzBWRVcAKnDucZ4N1Do2p3vQRRcTQ5/RxXoVgjDGeE+d7u23LzMzUrKzAPi1cVa3s\nmj6QI/GnMOzO2QHdtzHGtAYislRVMxtbz+vG9pC1cd0qerGXqozxXodijDGeskRykvYsmwVAz9Hf\n8DgSY4zxliWSk9Qh91P2hnUhOX2Q16EYY4ynLJGchCNFxQwuXcHutLPBukUxxrRzlkhOwrrFH5Mg\nJcQNsm5RjDHGEslJKFr3IZUaRsboyV6HYowxnrNE0kSqStf8L9geM5jIDkleh2OMMZ6zRNJE23Nz\nGFS9haKe53kdijHGtAqWSJpo+1ezCBOl6wjr7dcYY8ASSZOFb5tPIR3oMugsr0MxxphWwRJJE5SW\nVzLg6GJ2Jo6G8KANLmmMMSHFEkkTZK9aQlc5SHj/Cxpf2Rhj2glLJE1wYKXTOWNv6xbFGGOOsUTS\nBEm7F5IXkU5sWobXoRhjTKthicRPew4UMKxiDQe6nut1KMYY06pYIvHT+sXziJVyEodd4nUoxhjT\nqlgi8VPFxo+oIIL0Eda/ljHG+ApqIhGRiSKyQUQ2i8hddSzvJSLzRWS5iKwSkck+y+52t9sgIpf4\nu89gqKpWeh36ku1xpyLR8S1xSGOMCRlBSyQiEg48DkwChgDXisiQWqvdgzOW+wjgGuAJd9sh7vxQ\nYCLwhIiE+7nPgFu7cSMD2UGFjYZojDEnCOYdyWhgs6puVdVy4HVgSq11FOjoTncC8tzpKcDrqlqm\nqtuAze7+/NlnwOUtdUZDTB9l3aIYY0xtwUwkPYAcn/lct8zXr4EbRCQXmAXc3si2/uwz4GJ2fkqB\nJNKxz8hgH8oYY0KO143t1wIvqmo6MBn4p4gEJCYRmSoiWSKSlZ+ff9L7OVxUxtDSpeSlnAlhXl8u\nY4xpfYL5zbgL6Okzn+6W+boZeBNAVRcBMUBqA9v6s0/c/T2jqpmqmpmWlnbSJ7F62eekyhFiB114\n0vswxpi2LJiJZAnQX0QyRCQKp/F8Zq11dgITAERkME4iyXfXu0ZEokUkA+gPLPZznwFVuGYuAD0z\nrX3EGGPqErQubFW1UkRuA+YC4cALqpotItOBLFWdCfwCeFZEfobT8H6TqiqQLSJvAmuBSuBWVa0C\nqGufQTwH0vZ9QW5UX9ITuwfrMMYYE9LE+d5u2zIzMzUrK6vJ223O3UuvZwezue+NDPneo0GIzBhj\nWi8RWaqqmY2tZ63HDdiyZA5RUkXa8Ileh2KMMa2WJZKGbPmEUqJIGzLO60iMMabVsmH+GjCmRzRF\nRecRExnjdSjGGNNqWSJpQOK1z0A7aEMyxpjmsKqtxoh4HYExxrRqlkiMMcY0iyUSY4wxzWKJxBhj\nTLNYIjHGGNMslkiMMcY0iyUSY4wxzWKJxBhjTLO0i04bRSQf2IEznO9ht7ix6ZqfqcD+Jh7Sd3/+\nLq9d1tB87Rh9ywIdb33L/ImvsbhD4do2J9bG4rVra9e2oeVeXVvf/fZW1cYHdFLVdvMBnvF32udn\nVnOO4+/y2mUNzdeOMZjx1rfMn/jawrVtTqx2be3ahuK1bSzWuj7trWrrP02Y9i1rznH8XV67rKH5\numIMVrz1LfMnvvqmQ+naNifWxra3a9s8dm1PnG6JWE/QLqq2mkNEstSP/vhbi1CK12INnlCKN5Ri\nhdCKt6VibW93JCfjGa8DaKJQitdiDZ5QijeUYoXQirdFYrU7EmOMMc1idyTGGGOaxRKJMcaYZrFE\nYowxplkskTSTiHQQkSwRuczrWBoiIoNF5CkReUtEfux1PI0RkStE5FkReUNELvY6noaISF8ReV5E\n3vI6lvq4v6cvudf0eq/jaUgoXE9fIfa7GpzvgZN5EagtfIAXgH3AmlrlE4ENwGbgLj/2Mx34P+Cy\n1h6ru00Y8EooXFt3myTg+RCJ9a1gXtfmxA58F/iGO/1GS8Z5ste5pa9nAOIN6u9qgGMN6PeAJ/9I\nreEDnAeM9L34QDiwBegLRAErgSHAqcD7tT6dgYuAa4CbgpxImh2ru83lwGzgutZ+bX22+zMwMkRi\nbelE0pTY7waGu+vMaMk4mxqrV9czAPEG9Xc1ULEG43sggnZKVT8TkT61ikcDm1V1K4CIvA5MUdUH\ngBOqrkRkHNAB5z9qiYjMUtXq1hiru5+ZwEwR+QCYEeg4AxmviAjwIDBbVZe15li90pTYgVwgHViB\nB1XaTYx1bctGd6KmxCsi62iB39X6NPXaBuN7wNpIjtcDyPGZz3XL6qSqv1LVn+L8YzwbjCTSgCbF\nKiLjROQxEXkamBXs4OrQpHiB24ELgW+JyI+CGVgdmnptU0TkKWCEiNwd7OAaUV/sbwPfFJEnaX7X\nJIFSZ6yt7Hr6qu/aevm7Wp/6rm1Qvgfa7R1JIKnqi17H0BhVXQAs8DgMv6nqY8BjXsfhD1U9ALSW\nL5A6qWoR8H2v4/BHKFxPXyH2u7qAIHwP2B3J8XYBPX3m092y1iiUYoXQijeUYq0tlGIPpVghtOJt\n0VgtkRxvCdBfRDJEJAqnIX2mxzHVJ5RihdCKN5RirS2UYg+lWCG04m3ZWL16KsLrD/AasBuowKk/\nvNktnwxsxHni4VdexxlqsYZavKEUayjHHkqxhlq8rSFW67TRGGNMs1jVljHGmGaxRGKMMaZZLJEY\nY4xpFkskxhhjmsUSiTHGmGaxRGKMMaZZLJEYv4lIVxF5XUS2iMhSEZklIgPqWC9WRD4VkXAR6SMi\na7yI12tuv0bvN3Gb7iczDoeIJIrILc3dTxOOd4WIDAnW/msd6yYR6e4zv11EUutY7zIRmd4SMZnj\nWSIxfnF7430HWKCqp6jqKJyuybvUsfoPgLdVtaqFYgsP0H487XtORCJUNU9Vv3USmycCxxJJM/bj\nrytwer0+QRCu401A98ZWAj4AviEicQE+vmmEJRLjr/FAhao+VVOgqitVdWEd614PvFe70P3L8m0R\nmSMim0TkIbf8RyLyp1rr/d2dvkFEFovIChF5uiZpiMhREfmziKwEzhKRB0VkrYisEpGH3XXSROTf\nIrLE/ZxTT0wzReQT4GO3bJq7/ioR+Y3PuveKyAYR+VxEXhORO93yBSKS6U6nisj2Oo4zWkQWichy\nEfmviAys6/i+d3Ai8px73itEJF9E7heReBH5WESWichqEZniHuJB4BR33T/V2k+MiPzDXX+5iIxv\n6N+jjtiPu7YicjbOmBZ/co93insNHhWRLOAn9V17Efm1iLzgrr9VRO5o6PqKyLeATOBV91ix7uq3\n+1yDQQDqvF29gFbU1X+74fXr/fYJjQ9wB/CIH+tFAXt85vvgDriD85flVqATEAPswOlYLg1n7ISa\nbWYDY4HBOF2eR7rlTwA3utMKXO1Op+CMBFfTU0Oi+3MGMNad7gWsqyPem3C6lUh25y8GngEE5w+t\n93EGDjoDZyyPGCAB2ATc6W6zAMh0p1OB7e70OOB9d7ojEOFOXwj8u57jH7tePjH2Bta5PyOAjj7H\n2uzGetx2ta77L4AX3OlBwE73POr896h17Pqu7YvAt3zWWwA84TNf57UHfg38F4h24z8ARPp7fd35\n7cDt7vQtwHM+y64H/ub1/5f29rFu5E2gpQIFDSz/WFUPA4jIWqC3qn7u/nV6Js4XyCDgC+BWYBSw\nxKlZIxZnSFGAKuDf7vRhoBR4Xpw2iZp2iQuBIe62AB1FJF5Vj9aK6UNVPehOX+x+lrvz8UB/nC+3\n91S1FCgVkaaO6dEJeElE+uMkwch6jn8cEYkB/oXzxblDRCKBP4jIeUA1zrgTdVUv+hoL/A1AVdeL\nyA6gpm3rhH8Pjh/Hor5rW5c3fKbrvPbu9AeqWgaUicg+N/5zaNr1fdv9uRS4yqd8H/5Vg5kAskRi\n/JUN+FPnXoLzV2V9ynymq/j6d/B14GpgPfCOqqo430IvqWpdgxuVqtsGo6qVIjIamODGeBtwAc4d\nxZnul1NDinymBXhAVZ/2XUFEftrA9pV8XU1c37n/FpivqleKM5rdgnqOX9tTOO1NH7nz1+PcwY1S\n1Qq3Gq2h692Y+v49gAavbV18z6POa+8mlgaP2cS4a28fg/M7aFqQtZEYf30CRIvI1JoCETlNRM71\nXUlVDwHh7l/STfEOzlCg1+IkFXDaLL4lIp3d4yWLSO/aG7p/6XZS1VnAz4DT3UXzcEavq1lvuB9x\nzAV+UPPXs4j0cI//BU5Dboy7zLcefjvOnRPUn2w78fV4EDf5EQciciuQoKoP1trPPjeJjMe5gwAo\nxLlrqstCnASEOE/Z9cKprvInhvqubUPHg6Zf+4aub2PH8jUAaJdPCXrJEonxizoV0FcCF4rz+G82\n8ACwp47V5+FUpzRl/4dw2wFUdbFbtha4B5gnIquAD4FudWyeALzvrvM58HO3/A4g020kXosfo+6p\n6jyc+v1FIrIaeAvny3wJzngOq3DacFbjVPsAPAz8WESW41Tt1eUh4AF3HX//Ar8TONWnwf1HwKvu\nOa0GbsS5g0OdUQW/EJE14vPggusJIMzd5g3gJrdqyR/1XdvXgWlu4/0pdWzXpGvfyPV9EXiqVmN7\nfcbjPL1lWpB1I28CTkRGAj9T1e96HUsg1bSviPN46WfAVFVd5nVcbUVzr6+IdAFmqOqEoAVp6mRt\nJCbgVHWZiMwXkXBtoXdJWsgz4ryEF4PTdmNJJLCae3174TyhZlqY3ZEYY4xpFmsjMcYY0yyWSIwx\nxjSLJRJjjDHNYonEGGNMs1giMcYY0yyWSIwxxjTL/wO5pbH6+LI0ogAAAABJRU5ErkJggg==\n", 255 | "text/plain": [ 256 | "" 257 | ] 258 | }, 259 | "metadata": {}, 260 | "output_type": "display_data" 261 | } 262 | ], 263 | "source": [ 264 | "training_scores = []\n", 265 | "testing_scores = []\n", 266 | "\n", 267 | "param_list = [0.0001, 0.001, 0.01, 0.1, 10, 100, 1000]\n", 268 | "\n", 269 | "# Evaluate the training and test classification errors for each value of the parameter\n", 270 | "\n", 271 | "for param in param_list:\n", 272 | " \n", 273 | " # Create SVM object and fit\n", 274 | " \n", 275 | " svm = LinearSVC(C = param, random_state = 42)\n", 276 | " svm.fit(X_train, y_train)\n", 277 | " \n", 278 | " # Evaluate the accuracy scores and append to lists\n", 279 | " \n", 280 | " training_scores.append(svm.score(X_train, y_train) )\n", 281 | " testing_scores.append(svm.score(X_test, y_test) )\n", 282 | " \n", 283 | "# Plot results\n", 284 | "\n", 285 | "plt.semilogx(param_list, training_scores, param_list, testing_scores)\n", 286 | "plt.legend((\"train\", \"test\"))\n", 287 | "plt.ylabel('Accuracy scores')\n", 288 | "plt.xlabel('C (Inverse regularization strength)')\n", 289 | "plt.show()" 290 | ] 291 | }, 292 | { 293 | "cell_type": "markdown", 294 | "metadata": {}, 295 | "source": [ 296 | "**Hyper-parameter optimization using GridSearchCV**" 297 | ] 298 | }, 299 | { 300 | "cell_type": "code", 301 | "execution_count": 29, 302 | "metadata": {}, 303 | "outputs": [ 304 | { 305 | "name": "stdout", 306 | "output_type": "stream", 307 | "text": [ 308 | "The best value of the inverse regularization strength is: {'C': 0.1}\n" 309 | ] 310 | } 311 | ], 312 | "source": [ 313 | "#Building the model \n", 314 | "\n", 315 | "svm = LinearSVC(random_state = 50)\n", 316 | "\n", 317 | "#Using GridSearchCV to search for the best parameter\n", 318 | "\n", 319 | "grid = GridSearchCV(svm, {'C':[0.00001, 0.0001, 0.001, 0.01, 0.1, 10]})\n", 320 | "grid.fit(X_train, y_train)\n", 321 | "\n", 322 | "# Print out the best parameter\n", 323 | "\n", 324 | "print(\"The best value of the inverse regularization strength is:\", grid.best_params_)" 325 | ] 326 | }, 327 | { 328 | "cell_type": "markdown", 329 | "metadata": {}, 330 | "source": [ 331 | "**Scaling the data**" 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "execution_count": 32, 337 | "metadata": {}, 338 | "outputs": [ 339 | { 340 | "data": { 341 | "text/plain": [ 342 | "0.99717779868297274" 343 | ] 344 | }, 345 | "execution_count": 32, 346 | "metadata": {}, 347 | "output_type": "execute_result" 348 | } 349 | ], 350 | "source": [ 351 | "#Setting up the scaling pipeline \n", 352 | "\n", 353 | "order = [('scaler', StandardScaler()), ('SVM', LinearSVC(C = 0.1, random_state = 50))]\n", 354 | "\n", 355 | "pipeline = Pipeline(order)\n", 356 | "\n", 357 | "#Fitting the classfier to the scaled dataset \n", 358 | "\n", 359 | "svm_scaled = pipeline.fit(X_train, y_train)\n", 360 | "\n", 361 | "#Extracting the score \n", 362 | "\n", 363 | "svm_scaled.score(X_test, y_test)" 364 | ] 365 | } 366 | ], 367 | "metadata": { 368 | "kernelspec": { 369 | "display_name": "Python 3", 370 | "language": "python", 371 | "name": "python3" 372 | }, 373 | "language_info": { 374 | "codemirror_mode": { 375 | "name": "ipython", 376 | "version": 3 377 | }, 378 | "file_extension": ".py", 379 | "mimetype": "text/x-python", 380 | "name": "python", 381 | "nbconvert_exporter": "python", 382 | "pygments_lexer": "ipython3", 383 | "version": "3.6.1" 384 | } 385 | }, 386 | "nbformat": 4, 387 | "nbformat_minor": 2 388 | } 389 | -------------------------------------------------------------------------------- /Chapter_05.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Predicting Numeric Outcomes with Linear Regression\n", 8 | "\n", 9 | "**Aim**: The aim of this notebook is to predict the amount of a mobile transaction (numeric outcome) given all the other features in the dataset." 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "metadata": {}, 15 | "source": [ 16 | "## Table of contents\n", 17 | "\n", 18 | "1. Linear Regression in 2-Dimensions\n", 19 | "2. Linear Regression to predict transaction amount\n", 20 | "3. Model Optimization" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "## Package requirements" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": 40, 33 | "metadata": {}, 34 | "outputs": [], 35 | "source": [ 36 | "import pandas as pd\n", 37 | "from sklearn import linear_model\n", 38 | "import matplotlib.pyplot as plt\n", 39 | "import numpy as np\n", 40 | "from sklearn.model_selection import train_test_split\n", 41 | "from sklearn.preprocessing import StandardScaler\n", 42 | "from sklearn.pipeline import Pipeline\n", 43 | "from sklearn.linear_model import Ridge\n", 44 | "from sklearn.model_selection import GridSearchCV\n", 45 | "from sklearn.linear_model import Lasso\n", 46 | "import warnings" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": 2, 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "#Reading in the dataset\n", 56 | "\n", 57 | "df = pd.read_csv('fraud_prediction.csv')" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": 5, 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "#Define the feature and target arrays\n", 67 | "\n", 68 | "feature = df['oldbalanceOrg'].values\n", 69 | "target = df['amount'].values" 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "## Linear Regression in 2-Dimensions" 77 | ] 78 | }, 79 | { 80 | "cell_type": "code", 81 | "execution_count": 8, 82 | "metadata": {}, 83 | "outputs": [ 84 | { 85 | "data": { 86 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYHVWd//H3J00DzRIaJsExCwQl\nRnHBaAQEF1wwoAIZFCUsioPD6AgjMEZBGVlm/InmGR1UUBlZBVmFGJExIJvokJhAgiFANLKmAxKB\nZm2g0/n+/qi6zU3nLtVL9d0+r+e5T25Vnar6Vnfnfm+dc+ocRQRmZmYAY2odgJmZ1Q8nBTMz6+ek\nYGZm/ZwUzMysn5OCmZn1c1IwM7N+TgpmNSBpiqSQtEmZ7adKujinc+d2bGt8Tgo2JJJukfSUpM1q\nHUspko6U9LsK238k6aIS63eV9JKk7YZ5/l0kzZf0tKRnJd0sac/hHLPo2HtLWi/pufTVJem0kTi2\nmZOCDZqkKcC7gQAOqGkwQ3chcJCkLQesPwK4NiKeHOqBJb0W+D2wHNgJmABcA1wv6Z1DPe4AayJi\nq4jYCngXcJSkWSN0bGthTgo2FJ8CFgIXAJ8u3iDpAklnS/rf9Fvs7yX9vaT/Tu8s7pM0vaj8G9K7\njm5JKyQdULTtFkmfLVre4Nt/Wv3yOUl/Tvc/S4k3AD8C3pnG0D3wAiLidqAL+FjR8dqAQ4GL0uXd\nJC2R9Iykv0r6Tsafz6nA7RHxtYh4MiKejYjvAT8FvlVqB0k7Sbo1vau4ARiX8VxExAPA/wG7FB3v\nTEmPpLHfIend5faXdKWkx9K7mt9KemPRtgvSn+uv0tgWpUmvsP2Nkm6Q9GT6M/pqun6MpBMl/UXS\nE5KuGO7dl40OJwUbik8Bl6SvmZJeNWD7J4CTST7YXgJuB+5Ml68CvgMgqR34JXA9sD1wLHCJpGmD\niOWjwDuAt6TnnRkR9wKfI/lg3ioiOsvse1F6LQUfBNqB69LlM4EzI2Is8Frgiowx7QNcWWL9FcBe\nkjpKbPsZcAfJz+g/GJBsK5E0FdiLJFEXLAbeCmyXHvtKSZuXOcT/AlNJfgd3kvxeix0CnAZsC6wC\nvpGed2vgN8CvSe6GdgZuTPc5FpgFvDfd9hRwVtZrstppyKQg6TxJj0u6O0PZ70palr7+VOpbo2Un\n6V3AjsAVEXEH8BeSb9fFromIOyLiRZJqkxcj4qKI6AMuBwp3CnsAWwFnRMTLEXETcC0wexAhnRER\n3RHxMHAzyQdhVj8F3itpUrr8KeBnEdGbLvcCO0saFxHPRcTCkkfZ2Djg0RLrHyX5P7fBN2ZJO5Ak\ntn+PiJci4rckybKSCend0TPAn4BFQP9dVERcHBFPRMS6iPgvYDOgZLKNiPPSu5mXSO5ydpW0TVGR\nayLiDxGxjiRhFH7GHwUei4j/iogX02MsSrd9DvhaRKwuOu7HyzWsW/1oyKRAUm2xb5aCEXF8RLw1\nIt4KfB+4Os/AWsCngesj4m/p8s/Y+FvtX4ve95RY3ip9PwF4JCLWF21/CJg4iHgeK3r/QtGxq0oT\nyW+BwyVtRfLNtrjx+SjgdcB9khZL+mjGQ/8NeHWJ9a8G1pN8ay42AXgqIp4vWvdQlXOsiYjO9C6m\nk+TnemFho6QvSbo3rRLqBrahRJWUpDZJZ6TVPM8AD6abisuW+xlPJvlSUMqOwDVp4uoG7gX6gIF3\nlVZnGjIppN+kNmgIlPRaSb9O609vk/T6ErvOBi4dlSCbUFrt8QmSb9ePSXoMOJ7km+WuQzjkGmCy\npOK/wx1I6voBnge2KNr294M4dtbhfy8kaVz+GPBAeveTHCDizxExm6Ra5VvAVSUapkv5DXBwifWf\nIKnSemHA+keBbQcce4eM8RMRT5Mk5/0B0vaDL6fn2zatPnsaUIndDwUOJKk62waYkq4vVXagR4DX\nVNi2X5q4Cq/NI6KrTHmrEw2ZFMo4Bzg2It4OfAk4u3ijpB1JeoLcVIPYmsUskm97u5BUIbwVeANw\nGxvWzWe1iOSb55cltUvam+SD7bJ0+zKSHkJbSNqZ5Jt7Vn8FJknatEq5n5N8AJ9G0TdtAEmHSxqf\n3skUqh3XU91pwJ6SviFpO0lbSzqW5Gf0lYGFI+IhYAlwmqRN0yq6/TOcpxDnViT1/ivSVVsD64C1\nwCaSvg6MLbP71iTtPk+QJOD/l/W8JFV9r5Z0nKTN0uvcPd32I+Ab6f87JI2XdOAgjm010hRJIf1P\nsSdJY9oy4MdsfPt+CHBVWq9tQ/Np4PyIeDgiHiu8gB8Ahw22vjgiXib58NuPpMrlbOBTEXFfWuS7\nwMskH/AXsnEDaCU3kXxIPibpb+UKpVU2PwcmlTj+vsAKSc+RNDofEhE9AGmvppI9eiLizyTdRHcl\nqY55lOROZGZE/L5MKIcCu5PcAZ/ChtVYpUxIY3iOpKppO+CwdNsCksbfP6XbXiT55l7KRWmZLuAe\nNmysriginiVpVN+fpIrpz8D70s1nAvNJuuE+mx5391LHsfqiRp1kR0lf+Wsj4k2SxgIrI6JUPW6h\n/FLgCxHxf6MUoplZw2mKO4WIeAZ4QNLBAEr013Gn7QvbknSNNDOzMhoyKUi6lOQDfpqk1ZKOIrl1\nPkrSXSTVBsX1l4cAl0Wj3haZmY2Shq0+MjOzkdeQdwpmZpaPhnu6cNy4cTFlypRah2Fm1lDuuOOO\nv0XE+GrlGi4pTJkyhSVLltQ6DDOzhiKp2lPygKuPzMysiJOCmZn1c1IwM7N+TgpmZtbPScHMzPrl\n1vtI0nkkk3A8HhFvKrFdJINmfZhkpMwjI+LOvOJpFlNO/FWtQ8hdR/sYPvb2Sdx831rWdPcwobOD\n971+PDfft5au7h4kKDxz2dE+hjESz7+88TiHW7SP4YXe9YwRrC/zjObU7bfkhhP23mDdyfOWc+mi\nR+grerBzYmcHc2ZOY9b0ZKqHeUu7mLtgJV3dPbRJ9EUwMY3z6jtW80JvMpiqBHu+ZjsefKKn/1pK\nHWdNdw/bdLTT27d+g2vp7Gjn1APe2F++OLY2iT1es23ZY2e9loLiWArHAjZaN3C/SvuXK1uqfOF3\nXG7/vI8/moZ7LXnGntsTzZLeAzwHXFQmKXyYZMq+D5OMnnhmRFQdRXHGjBnRql1SWyEh1EJxYjh5\n3nIuXvhwyXId7W1886A3A3DS1cvp6R3agLuDPU77GDH34F1Z8tCTZWMbeOxZ0ydmupbi5DQwlvYx\nAkFvX5Tdr6DU/uXKlitfKca8jz+aRuJahhK7pDsiYka1crlVH5WaCGeAA0kSRqTTHHZKKjvKqVle\n/vz4KxOeXbqo3AjT0NPbx9wFK5m7YOWQE8JQjtO7Ppi7YGXF2AYeG7JdS0GpWHrXxwYJodR+lfYv\nV7Zc+Uox5n380TQS15Jn7LV8eG0iG47xvjpdt9HctpKOBo4G2GGHzBNSmQ1aX5U75zXdPSNynsEe\nZ013T+ap5ArHHsy1DCaeUmXL7T/Y9eXK5X380TRS15JX7A3R0BwR50TEjIiYMX581ae0zYasTZVn\noZzQ2cGEzo5hn2ewx5nQ2VE1tuKykO1aSr3Pevxq64ayvly5vI8/mkbqWvKKvZZJoYtk4u+CSbwy\nN68NcPK85bUOoWlN3f6VqZFn7z65bLmO9jbmzJzGnJnT6GhvG/L5Bnuc9jFizsxpFWMbeGzIdi0F\npWJpHyPa21Rxv0r7lytbrnylGPM+/mgaiWvJM/ZaVh/NB46RdBlJQ/PTEbFR1ZElqjUwNpNa9j76\nz1lJA3CWHjsj0fuocJwsvY8K+2TtfTSYaym8H2rvo3L7l2sILVW+Uu+gvI8/mkbiWhq199GlwN7A\nOJI5dk8B2gEi4kdpl9QfkMyD+wLwmYio2q2oVXsfVep5dPgeO/R/AJiZlZK191FudwoRMbvK9gC+\nkNf5W0mjJYTR7HNtZoPTcENnt6Jmak8Y2Oe6q7uHk65Ors+Jwaz2GqL3UavL0j+9UYx2n2szGxwn\nhQZQrb95IxntPtdmNjhOCg2uuDtlIxjtPtdmNjhOCnWuWnvCwMHc6t1o97k2s8FxQ3Odu6TJnk8Y\n7T7XZjY4Tgp1rnlaE15R/CCWmdUXVx/VsWpVR2M3G/pQC2ZmpTgp1LFqQ1v88bR9RykSM2sVTgpm\nZtbPSaFBuerIzPLgpFCn9vnOLRW3u+rIzPLgpFCniqeINDMbLU4KZmbWz0nBzMz6OSk0oMP32KHW\nIZhZk3JSqEPVGpkbbVIdM2scTgp1qFIj8yYqu8nMbNicFBrMqm9+pNYhmFkTc1IwM7N+Tgp1ZueT\nflXrEMyshTkp1Jl1zThWtpk1DCeFBuKuqGaWNyeFOlJt/gR3RTWzvDkp1JFLFz1S6xDMrMU5KdSR\nvnCDgpnVlpNCg3B7gpmNBieFBuH2BDMbDU4KdaLaeEdmZqPBSaFOeFIdM6sHTgoNwIPgmdloyTUp\nSNpX0kpJqySdWGL7DpJulrRU0h8lfTjPeOpVtecTPAiemY2W3JKCpDbgLGA/YBdgtqRdBhQ7Gbgi\nIqYDhwBn5xVPPbt44cO1DsHMDMj3TmE3YFVE3B8RLwOXAQcOKBPA2PT9NsCaHOMxM7Mq8kwKE4Hi\nR3RXp+uKnQocLmk1cB1wbKkDSTpa0hJJS9auXZtHrHVr6vZb1joEM2shtW5ong1cEBGTgA8DP5W0\nUUwRcU5EzIiIGePHjx/1IGvphhP2rnUIZtZC8kwKXcDkouVJ6bpiRwFXAETE7cDmwLgcY6o7nj/B\nzOpJnklhMTBV0k6SNiVpSJ4/oMzDwAcAJL2BJCm0VP1QpfkTPLSFmY223JJCRKwDjgEWAPeS9DJa\nIel0SQekxf4N+CdJdwGXAkdGtM6ocB4q28zqzSbVCkjaGTgBmFJcPiI+VG3fiLiOpAG5eN3Xi97f\nA+yVPdzm4q6oZlZvqiYF4CrgXOBioC/fcMzMrJayJIX1EfH93CMxM7Oay9Km8Iv0OYHxksYWXrlH\n1uSqjYo6drO20QnEzKxIljuFz6b//nvRugDcNWYYqo2K+sfT9h2lSMzMXlE1KUTE5GplzOYt7WLu\ngpWs6e5hQmcHc2ZOY9b0gQ+wm1m9y9L7aBPgaOA96apbgJ+kXU7NmLe0i5OuXk5Pb9IPoau7h5Ou\nTrrbOjGYNZYsbQpnAXsC56WvPWnR0UxHS6ONdzR3wcr+hFDQ09vH3AUraxSRmQ1VljaFPSJi16Ll\n69OHzWyIqg1t0WjjHa3p7hnUejOrX1nuFNZLmlJYSN+vzyec1lBpaItGNKGzY1Drzax+ZUkKXwFu\nk/QbSTcCtwJz8g2rdTXieEdzZk6jo33DLrQd7W3MmTmtRhGZ2VBl6X10vaTXAW9IV90bEa4XGKK3\nnPLritsbcbyjQmOyex+ZNb6ySUHSeyPi1qLB6womSSIiBo54ahk881JzjhQya/pEJwGzJlDpTmEf\nkqqig0tsCzYeBtvMzBpc2aQQESenb78WERsM5ymp8Sq+G4CHtjCzWsvS0Dwv4zqrotp4Rx7awsxq\nrVKbQqFxeZsB7QpjSWZIs0GqNt6RmVmtVWpTeCNwENDJhu0KzwL/nGdQZmZWG5XaFK4BrpH0roj4\n3SjG1JIa8fkEM2s+WdoUPiOps7AgaVtJ/5NjTE2pGZ9PMLPmkyUpvC0iugsLEfEU8Pb8QmpOzfp8\ngpk1lyxJYYykbQoLkrYF2vMLyczMaiXLKKn/Ddwu6XJAwCeAb+caVZM5ed7yitsbbahsM2teWcY+\nOl/SncD70lWHRMQf8w2ruVy88OGK2xttqGwza15Z7hSIiLskPUL6fIKkCRGxJtfIzMxs1FVtU5D0\nEUl/AlYDC4FHgJvyDszMzEZflobmbwB7ASsjYgdgX+C2XKNqItWGtjAzqydZksK6iFhL0gtJEXED\nsFvOcTWNakNbPHjGR0YpEjOz6rK0KTwtaSvgd8BFkh4HPMmOmVkTynKnMIskCRwH3AJ0AfvnGJOZ\nmdVIpuojYH1E9JIkhfuAp/IMqllUez7B8yeYWb3JkhRuAzokvZqk19E/AedlObikfSWtlLRK0oll\nynxC0j2SVkj6WebIG8AlVZ5P8PwJZlZvsrQpjImIFyT9I/DDiDhD0rJqO0lqA84imdZzNbBY0vyI\nuKeozFTgJGCviHhK0vZDu4z6FLUOwMxskLKOffQO4DDg2nRdlnqP3YBVEXF/RLwMXAYcOKDMPwFn\npYPsERGPZwvbzMzykCUpnACcBlwbEXdLeg3ZnlOYSPKgW8HqdF2x1wGvk/R7SQsllaxPkXS0pCWS\nlqxduzbDqWuv2lDZ7opqZvUoy9hHN1H0BHNE3A/8ywiefyqwNzAJ+K2kNxcP1Z2e8xzgHIAZM2Y0\nRK2Mh8o2s0ZUNSlI2pnkbmFKcfmI+FCVXbuAyUXLk9J1xVYDi9KeTQ+kw2lMBRZXjdzMzEZclobm\nq4BzgYuBwXz9XQxMlbQTSTI4BDh0QJl5wGzgfEnjSKqT7h/EOczMbARlSQrrI+L7gz1wRKyTdAyw\ngKRh+ryIWCHpdGBJRMxPt31I0j0kCWdORDwx2HPVm2rjHXk+ZjOrV1mSwi8kHQ1cA7xUWBkRz1Tb\nMSKuA64bsO7rRe+DpGrqhKwBN4Jq4x15PmYzq1dZksJn03//vWhdAP66a2bWZLL0PppcrYxlp1oH\nYGZWQaaZ1yS9HtiFdOY1gIhoqiEpRsrOJ/2q4vbD3J5gZnUsS5fUk4EPAa8naRieSTKMtpNCCeuq\nPEXh9gQzq2dZnmj+JPA+4NGIOALYFdgy16jMzKwmsiSFnojoA9ZJ2hp4DNgx37Cak7uimlm9y9Km\nsFRSJ8lw2UuAZ4A/5BpVk3LVkZnVu4pJQZKAU9OxiM6StAAYGxF3jkp0DabaIHhmZvWuYlKIiJB0\nA/CmdHnVqETVoDwInpk1uixtCsskTc89kia3iR9QMLMGkKVNYTrJrGl/AZ4nef4qIuJtuUbWYKpV\nHa36pudPMLP6VzYpSNokItYBB4xiPA3LVUdm1gwq3Sn8AXhbRPxltIIxM7PaqtSm4FpwM7MWU+lO\nYbykskNaR8R3coinIU05sfJ4R56P2cwaRaWk0AZshe8YzMxaRqWk8GhEnD5qkZiZWc25TWGYqnVF\nfdXWm45SJGZmw1cpKXxg1KJoYNW6oi762j6jFImZ2fCVTQoR8eRoBmJmZrVXNilI2mw0A2lGrn8z\ns0ZTqfrodgBJPx2lWBpOtfYET71pZo2mUu+jTSUdCuwp6aCBGyPi6vzCagzV2hM8f4KZNZpKSeFz\nwGFAJ7D/gG0BtHxSMDNrNmWTQkT8DvidpCURce4oxmRmZjWSZejsn0r6V+A96fKtwI8ioje/sOrf\nPt+5peL2qdtvOTqBmJmNoCxJ4WygPf0X4Ajgh8Bn8wqqEfz58ecrbr/hhL1HJxAzsxGUJSm8IyJ2\nLVq+SdJdeQVkZma1k2U6zj5Jry0sSHoN4BllzMyaUJY7hTnAzZLuJ3kea0fgM7lGVeeqDZXt9gQz\na1RVk0JE3ChpKjAtXbUyIl7KN6zG5vYEM2tUWaqPiIiXIuKP6StzQpC0r6SVklZJOrFCuY9JCkkz\nsh7bzMxGXqakMBSS2oCzgP2AXYDZknYpUW5r4IvAorxiMTOzbHJLCsBuwKqIuD8iXgYuAw4sUe4/\ngG8BL+YYy4ipNt7RJh4Fz8waWNWkIOnGLOtKmAg8UrS8Ol1XfJy3AZMjomLLraSjJS2RtGTt2rUZ\nTp2fauMdrfqm52M2s8ZVtqFZ0ubAFsA4SdvyykjQYxnw4T4UksYA3wGOrFY2Is4BzgGYMWNGDPZc\n85Z2MXfBStZ09zChs4M5M6cxa/qwL8HMrOlU6n30z8BxwATgDl5JCs8AP8hw7C5gctHypHRdwdbA\nm4BbJAH8PTBf0gERsSRT9BnMW9rFSVcvp6c3+Ybf1d3DSVcvB3BiMDMboNLMa2dGxE7AlyLiNRGx\nU/raNSKyJIXFwFRJO0naFDgEmF90/KcjYlxETImIKcBCYEQTAsDcBSv7E0JBT28fcxesHPSx/HyC\nmTW7LM8pfF/SnsCU4vIRcVGV/dZJOgZYALQB50XECkmnA0siYn6l/UfKmu6eQa0fDj+fYGaNrmpS\nSGdeey2wjFeGtwigYlIAiIjrgOsGrPt6mbJ7VzveUEzo7KCrRAKY0NmRx+nMzBpalmEuZgC7RMSg\nG3jrwZyZ0zZoUwDoaG9jzsxpFfYaPHdFNbNmkCUp3E3SCPxozrHkotCYPNzeR9XaE9wV1cyaQZak\nMA64R9IfgP4hLiLigNyiGmGzpk90TyMzswyyJIVT8w7CzMzqQ5beR7eORiBmZlZ7WXofPUvS2whg\nU5KpOZ+PiLF5BlZPqrUnPHiG2xPMrDlkuVPYuvBeyaPHBwJ75BmUmZnVxqBGSY3EPGBmTvGYmVkN\nZak+OqhocQzJcwsNMcz1SJi3tKvidj+fYGbNJEvvo/2L3q8DHqT0vAhN6bjLl1Xc7ucTzKyZZGlT\n+MxoBGJmZrWXZZKdSZKukfR4+vq5pEmjEZyZmY2uLNVH5wM/Aw5Olw9P1+2TV1Aj7eR5y7l00SP0\nRdAmMXv3yfznrDdn2s/MrJVk6X00PiLOj4h16esCYHzOcY2Yk+ct5+KFD9OXjufXF8HFCx/O9IF/\n8cKHK2738wlm1myyJIUnJB0uqS19HQ48kXdgI+WSMh/s5dabmbWyLEnhH4FPAI+RjJT6caBhGp/L\njffdkOOAm5nlLEvvo4eAhhkRdaRUG9rCzKwZZXl4bSfgWDaejrPlEkUxtyeYWTPK0vtoHnAu8Etg\nfb7hjDwJys0Zt9cZNw1pwh0zs2aVJSm8GBHfyz2SnBy2+w5lexF1dfdw0tVJLyQnBjOzbA3NZ0o6\nRdI7Jb2t8Mo9slHS09vH3AUrN1hXrT3h8D12yDMkM7OayXKn8GbgCOD9vFJ9FOly3cvS9XRNd8+g\njpnlwTczs0aUJSkcDLwmIl7OO5g8ZOl6OqGzI/c4zMwaQZbqo7uBzrwDqZWO9jbmzJxW6zDMzOpC\nljuFTuA+SYuBlworm6FLamdHOxIcf/ky5i5YyZyZ06oOlT11+y37389b2sXcBStZ093DhM4O92Qy\ns4aXJSmcknsUNfLSuvX09PYBG/ZEquSGE/YGkoRw0tXLS+7vxGBmjapq9VFE3Fr8AvpIhr1oeIUP\n9HLLlcxdsLLk/gN7MpmZNZIsdwpImg4cStLo/ADw8zyDagTleiwNtieTmVk9KZsUJL0OmJ2+/gZc\nDigi3jdKsdWd4vmYJ3R20FUiAbgnk5k1skrVR/eRPIvw0Yh4V0R8n6TqqGUVz8c8Z+Y0OtrbNtju\nnkxm1ugqJYWDSIbKvlnS/0j6AKAK5TciaV9JKyWtknRiie0nSLpH0h8l3Shpx8GFP3LaNKhLY9b0\niXzzoDczsbMDARM7O/jmQW92I7OZNbSy1UcRMQ+YJ2lL4EDgOGB7ST8EromI6ysdWFIbcBbJtJ2r\ngcWS5kfEPUXFlgIzIuIFSZ8Hvg18clhXNAQd7W2M3byNvz47uOfzZk2f6CRgZk0lS++j5yPiZxGx\nPzCJ5IP8KxmOvRuwKiLuT5+GvowkuRQf++aIeCFdXJgef9QUf8OvlhA83pGZtYJMvY8KIuIp4Jz0\nVc1E4JGi5dXA7hXKHwX8b6kNko4GjgbYYYeR+3B+oGhOhGoPrXm8IzNrBVmGuchdOu/zDGBuqe0R\ncU5EzIiIGePHjx/d4MzMWkieSaELmFy0PCldtwFJHwS+BhwQES8N3J6neUuTcHY+yVNvmplBvklh\nMTBV0k6SNgUOAeYXF0gfivsxSUJ4PMdYSio8fbyuylCqnnrTzFpFbkkhItYBxwALgHuBKyJihaTT\nJRUG05sLbAVcKWmZpPllDpeLru6e/rsFMzMbZEPzYEXEdcB1A9Z9vej9B/M8fxZZBsEzM2sVddHQ\nXEuDGQTPzKzZtXxSqMbtCWbWSpwUzMysn5OCmZn1c1IwM7N+TgoVuD3BzFqNk4KZmfVzUjAzs35O\nCmZm1s9JoQy3J5hZK3JSMDOzfk4KZmbWz0mhDI+eamatyEmhjMJcC2ZmrSTXobMbWWGuhVnTJ/av\nm7e0i7kLVrKmu4cJnR3MmTmtf3ulbWZmjcJJoYLiuRZO++UKnnqht3+5q7tng+0nXb28fxju4m1O\nDGbWSJwUKujp7ePU+St4ad36kvMu9PT28W9X3MXYjk022t7T28fcBSuHlBR812FmteKkUEV3T2/F\n7X0RG9xBFFvT3TPo881b2uW7DjOrGSeFHAWw1xk3lf2mX+qOYO6ClSXvOo6/YhngxGBm+XJSyFlX\ndw9zrroLYKNG61J3BOWmB41gg+O4isnM8uCkMAp6+4KvXv3HDT7En39pXck7gmrHKXSVnXPlXfSu\nDyBNPFc6YZjZ8DkpjJIXetfzQtrG0DWEtoaCru4ejrt82Ubre9cHx1++jCUPPcnlix+ht68oYZS4\nUzEzK8UPrzWRAC5e+HB/Qijo7QtO++WK2gRlZg3FSaFFlOshZWZWzEnBzMz6OSm0kOmnX++B/sys\nIieFFvLUC70cd/kyTp63vHphM2tJTgot6OKFDzsxmFlJTgotyonBzEpxUmhhFy982G0MZrYBJ4UW\nd+p8P79gZq/INSlI2lfSSkmrJJ1YYvtmki5Pty+SNCXPeGxj1UaBNbPWkltSkNQGnAXsB+wCzJa0\ny4BiRwFPRcTOwHeBb+UVj5mZVZfnncJuwKqIuD8iXgYuAw4cUOZA4ML0/VXAByQpx5hsgG23aK91\nCGZWR/JMChOBR4qWV6frSpaJiHXA08DfDTyQpKMlLZG0ZO3atTmF25pO2f+NtQ7BzOpIQzQ0R8Q5\nETEjImaMHz++1uE0jcP32MEjp5rZBvIcOrsLmFy0PCldV6rMakmbANsAT+QYU8toHwO960tv23aL\ndk7Z/41OCGa2kTyTwmJgqqSdSD78DwEOHVBmPvBp4Hbg48BNERGMoAfP+AhTTvzVSB6yrk30pDpm\nNgy5JYWIWCfpGGAB0AacFxErJJ0OLImI+cC5wE8lrQKeJEkcI+7BMz6Sx2HNzJpOrjOvRcR1wHUD\n1n296P2LwMF5xmBmZtk1REPE1tjuAAAJl0lEQVSzmZmNDicFMzPr56RgZmb9nBTMzKyfRrgHaO4k\nrQUeGuLu44C/jWA4teRrqV/NdD2+lvo0lGvZMSKqPv3bcElhOCQtiYgZtY5jJPha6lczXY+vpT7l\neS2uPjIzs35OCmZm1q/VksI5tQ5gBPla6lczXY+vpT7ldi0t1aZgZmaVtdqdgpmZVeCkYGZm/Vom\nKUjaV9JKSasknVjreIZK0nmSHpd0d61jGS5JkyXdLOkeSSskfbHWMQ2VpM0l/UHSXem1nFbrmIZL\nUpukpZKurXUswyHpQUnLJS2TtKTW8QyHpE5JV0m6T9K9kt454udohTYFSW3An4B9SKYFXQzMjoh7\nahrYEEh6D/AccFFEvKnW8QyHpFcDr46IOyVtDdwBzGrQ34uALSPiOUntwO+AL0bEwhqHNmSSTgBm\nAGMj4qO1jmeoJD0IzIiIhn9wTdKFwG0R8RNJmwJbRET3SJ6jVe4UdgNWRcT9EfEycBlwYI1jGpKI\n+C3J3BMNLyIejYg70/fPAvey8TzeDSESz6WL7emrYb9xSZoEfAT4Sa1jsYSkbYD3kMxDQ0S8PNIJ\nAVonKUwEHilaXk2Dfvg0K0lTgOnAotpGMnRpdcsy4HHghoho2GsB/hv4MlBmUteGEsD1ku6QdHSt\ngxmGnYC1wPlptd5PJG050idplaRgdUzSVsDPgeMi4plaxzNUEdEXEW8lmY98N0kNWb0n6aPA4xFx\nR61jGSHvioi3AfsBX0irYBvRJsDbgB9GxHTgeWDE20dbJSl0AZOLliel66zG0vr3nwOXRMTVtY5n\nJKS39DcD+9Y6liHaCzggrYu/DHi/pItrG9LQRURX+u/jwDUk1cmNaDWwuugO9CqSJDGiWiUpLAam\nStopbZw5BJhf45haXto4ey5wb0R8p9bxDIek8ZI60/cdJJ0a7qttVEMTESdFxKSImELyf+WmiDi8\nxmENiaQt004MpFUtHwIasudeRDwGPCJpWrrqA8CId8rIdY7mehER6yQdAywA2oDzImJFjcMaEkmX\nAnsD4yStBk6JiHNrG9WQ7QUcASxP6+IBvprO7d1oXg1cmPZ0GwNcEREN3ZWzSbwKuCb5/sEmwM8i\n4te1DWlYjgUuSb/c3g98ZqRP0BJdUs3MLJtWqT4yM7MMnBTMzKyfk4KZmfVzUjAzs35OCmZmdWww\ng2BK+m468N8ySX+SNOhhMJwUrCJJkyT9QtKfJf1F0plpdzgk7V1uBM10ZMpxZdYXRqxcLqnqGFTl\njlUrkt6djoS6LH0mYeD2WZJC0utrEd+AWPaWtGeZbUdK+sGAdbdIqjghfIXf7amSvjS8iK2EC8j4\nIGREHB8Rb02frP8+MOgHQp0UrKz04bKrgXkRMRV4HbAV8I1hHvp96R/tx4HvDfNYtXAY8M30P19P\nie2zSUZJnT26YZW0N1AyKdSapJZ4Tmq4Sg2CKem1kn6djud0W5kvILOBSwd7PicFq+T9wIsRcT4k\nY/sAxwP/KGmL4oKS/k7S9ek36J8AynD8scBTRceYl/6Rryg3cFm5MpKek/SNdD6DhZJela5/laRr\n0vV3Fb41SzpcyfwHyyT9OH3obOC5PpAOPLY8vYXfTNJngU8A/yHpkhL7bAW8CziK5Gng4m1fSY91\nl6Qz0nU7S/pNuu7O9D+7JM2VdHda/pNp2Q3uzCT9QNKR6fsHJZ2WHmO5pNcrGWTwc8Dx6XW+O8Pv\npDje2emx7pb0rTJlvpZWU/wOmFa0vuSHlqQLJP1I0iLg24OJxzZwDnBsRLwd+BJwdvFGSTuSDKB3\n06CPHBF++VXyBfwr8N0S65cCbyH5Fnptuu57wNfT9x8hGZlyXIl9HwSWkww18ALw0aJt26X/dqTb\n/65on3FVygSwf/r+28DJ6fvLSQbag+Rp9m2ANwC/BNrT9WcDnxoQ5+YkI+u+Ll2+qOg4FwAfL/Mz\nOww4N33/f8Db0/f7pctbDLiORcA/FJ1zC+BjwA1pvK8CHiZ5Yrr/552W/wFwZNHP6Nj0/b8AP0nf\nnwp8qUysR5KMurms6PUcyRwKE9Lzjid5Evgmkrku+n8fwNvT3+UWJAl+VeFcwI3A1PT97iRDZRR+\ndtcCbbX++26kFzAFuDt9vxXQM+D3du+A8l8Bvj+Uc/n2zUbKe4CDACLiV5KeqlD2fRHxN0mvBW6U\ndEskcxH8q6R/SMtMBqYCTwzYt1yZl0k+bCCZrGef9P37gU+lcfUBT0s6guQDbXFSQ0YHyXDXxaYB\nD0TEn9LlC4EvkAwpXcls4Mz0/WXp8h3AB4HzI+KFNJYnlYzJMzEirknXvQgg6V3ApWm8f5V0K/AO\noNoIsoX64ztIfxcZXB4RxxQWJN2Svn0HcEtErE3XX0LyO55XtO+7gWsK1yRpfvrvViRVVlemP1+A\nzYr2uzK9NhuaMUB3JFWw5RxC8vc6aE4KVsk9JPX+/SSNBXYg+VY4rNEmI+Ivkv4K7JJWR30QeGdE\nvJB+OG0+4Nx7VyjTG+lXJKCPyn/bAi6MiJOGE/9GB5W2I0lCb5YUJN/0Q9KcETrFOjas8t18wPaX\n0n+rXX/eqn1oPT+awTSbiHhG0gOSDo6IK9O2v7dExF0AaVXdtsDtQzm+2xSskhuBLSR9CvqnNf0v\n4ILCt8MivwUOTcvtR/JHWZGk7UnqPR8iqdZ5Kv2wfz2wR4ldspQpdQ2fL8SvZPaqG4GPp+dH0nZp\nHWyxlcAUSTuny0cAt1Y518eBn0bEjhExJSImAw+QfKO+AfhMoS1G0naRzDa3WtKsdN1m6fbbgE+m\n8Y4n+Yb+h/TntEtarpNklMxqngW2zlBuoD8A75U0Lv29z2bj6/8tMEtSR3rXsz8kH1rAA5IOTq9L\nknYdQgxG/yCYtwPTJK2WdBRJNeVRku4CVrDhTJKHAJcVfUkaFN8pWFkREWlVzdmS/p3kS8R1wFdL\nFD8NuFTSCpK684crHPpmSX0kU1aeGBF/lfRr4HOS7iX5QC41t3GWMgN9ETgn/Y/UB3w+Im6XdDLJ\nbFxjgF6SW+2Hiq79RUmfIakC2YRk+PUfVTnXbGBgg+zPSeYD/7yktwJLJL3MKz/HI4AfSzo9jeNg\nkjH/3wncRdJW8uVIhk1G0hUkbSkPkLTtVPNL4ColXX+PjYjbMuxDRDwq6USSeSEE/CoifjGgzJ2S\nLk/jfJzkZ1RwGPDD9OfcTlKVdleWc9uGIqJcL7aS3VQj4tThnM+jpJqZWT9XH5mZWT8nBTMz6+ek\nYGZm/ZwUzMysn5OCmZn1c1IwM7N+TgpmZtbv/wPVUtYmRlSA2wAAAABJRU5ErkJggg==\n", 87 | "text/plain": [ 88 | "" 89 | ] 90 | }, 91 | "metadata": {}, 92 | "output_type": "display_data" 93 | } 94 | ], 95 | "source": [ 96 | "#Creating a scatter plot\n", 97 | "\n", 98 | "plt.scatter(feature, target)\n", 99 | "plt.xlabel('Old Balance of Account Holder')\n", 100 | "plt.ylabel('Amount of Transaction')\n", 101 | "plt.title('Amount Vs. Old Balance')\n", 102 | "plt.show()" 103 | ] 104 | }, 105 | { 106 | "cell_type": "markdown", 107 | "metadata": {}, 108 | "source": [ 109 | "**Building the linear regression model**" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": 12, 115 | "metadata": {}, 116 | "outputs": [ 117 | { 118 | "data": { 119 | "text/plain": [ 120 | "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)" 121 | ] 122 | }, 123 | "execution_count": 12, 124 | "metadata": {}, 125 | "output_type": "execute_result" 126 | } 127 | ], 128 | "source": [ 129 | "#Initializing a linear regression model \n", 130 | "\n", 131 | "linear_reg = linear_model.LinearRegression()\n", 132 | "\n", 133 | "#Reshaping the array since we only have a single feature\n", 134 | "\n", 135 | "feature = feature.reshape(-1, 1)\n", 136 | "target = target.reshape(-1, 1)\n", 137 | "\n", 138 | "#Fitting the model on the data\n", 139 | "\n", 140 | "linear_reg.fit(feature, target)" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": 17, 146 | "metadata": {}, 147 | "outputs": [ 148 | { 149 | "data": { 150 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xm8VfP+x/HXu1QiFbe4hpIxQsQx\nZYoQ19TlusrFjyvJNcUVmWWmi5uIS6FIhiah20BJpqhEAxGiEkWTBqrT5/fHd51zd8c+56wz7LPO\nPufzfDz2o73X+u61Puuczv7s77C+X5kZzjnnHECNpANwzjlXeXhScM45l8+TgnPOuXyeFJxzzuXz\npOCccy6fJwXnnHP5PCk4lwBJzSSZpE0K2X+bpOcydO6MHdtlP08KrlQkvSVpqaQ6SceSjqTzJb1T\nxP7HJQ1Is31fSb9J2qqM528haYSk5ZJ+kTReUuuyHDPl2G0kbZC0MnoskNSjPI7tnCcFV2KSmgFH\nAAacmmgwpdcfOF3S5gW2nwu8ZmZLSntgSbsA7wLTgZ2A7YBhwBhJh5b2uAV8b2b1zKwecDhwoaT2\n5XRsV415UnClcR7wAfAM8H+pOyQ9I6mPpP9G32LflfRHSf+OahafS2qVUn7PqNaxTNJMSaem7HtL\nUqeU1xt9+4+aX7pI+jJ6/6MK9gQeBw6NYlhW8ALM7H1gAXBGyvFqAmcDA6LXB0maLGmFpB8lPRjz\n53Mb8L6Z3WhmS8zsFzN7GHgWuC/dGyTtJGlCVKsYCzSKeS7M7BvgPaBFyvF6SZoXxT5F0hGFvV/S\ny5J+iGo1b0vaK2XfM9HP9fUotklR0svbv5eksZKWRD+jG6LtNSR1l/SVpJ8lvVTW2perGJ4UXGmc\nBwyMHu0kbVNg/1+BmwgfbL8B7wNTo9eDgQcBJNUCXgXGAFsDlwMDJTUvQSwnAwcCLaPztjOzz4Au\nhA/membWsJD3DoiuJc+xQC1gZPS6F9DLzOoDuwAvxYzpOODlNNtfAg6TVDfNvueBKYSf0R0USLZF\nkbQbcBghUef5CNgP2Co69suSNi3kEP8FdiP8DqYSfq+pOgA9gC2BOcBd0Xm3AN4ARhFqQ7sCb0bv\nuRxoDxwV7VsKPBr3mlxysjIpSHpK0iJJM2KUfUjStOjxRbpvjS4+SYcDOwIvmdkU4CvCt+tUw8xs\nipn9Smg2+dXMBphZLvAikFdTOASoB9xrZmvNbBzwGtCxBCHda2bLzOw7YDzhgzCuZ4GjJO0QvT4P\neN7M1kWv1wG7SmpkZivN7IO0R/m9RsDCNNsXEv7mNvrGLKkpIbHdbGa/mdnbhGRZlO2i2tEK4Atg\nEpBfizKz58zsZzNbb2YPAHWAtMnWzJ6KajO/EWo5+0pqkFJkmJl9aGbrCQkj72d8MvCDmT1gZr9G\nx5gU7esC3Ghm81OO+5fCOtZd5ZGVSYHQbHFCnIJmdpWZ7Wdm+wG9gaGZDKwa+D9gjJn9FL1+nt9/\nq/0x5fmaNK/rRc+3A+aZ2YaU/d8C25cgnh9Snq9OOXaxokTyNnCOpHqEb7apnc8XArsDn0v6SNLJ\nMQ/9E7Btmu3bAhsI35pTbQcsNbNVKdu+LeYc35tZw6gW05Dwc+2ft1PSNZI+i5qElgENSNMkJamm\npHujZp4VwNxoV2rZwn7GTQhfCtLZERgWJa5lwGdALlCwVukqmaxMCtE3qY06AiXtImlU1H46UdIe\nad7aERhUIUFWQVGzx18J365/kPQDcBXhm+W+pTjk90ATSan/D5sS2voBVgGbpez7YwmOHXf63/6E\nzuUzgG+i2k84gNmXZtaR0KxyHzA4Tcd0Om8AZ6bZ/ldCk9bqAtsXAlsWOHbTmPFjZssJyfkUgKj/\n4NrofFtGzWfLAaV5+9nAaYSmswZAs2h7urIFzQN2LmLfiVHiyntsamYLCinvKomsTAqFeAK43MwO\nAK4B+qTulLQjYSTIuARiqyraE77ttSA0IewH7AlMZOO2+bgmEb55XiuplqQ2hA+2F6L90wgjhDaT\ntCvhm3tcPwI7SKpdTLkhhA/gHqR80waQdI6kxlFNJq/ZcQPF6wG0lnSXpK0kbSHpcsLP6LqChc3s\nW2Ay0ENS7aiJ7pQY58mLsx6h3X9mtGkLYD2wGNhE0i1A/ULevgWh3+dnQgK+O+55CU1920rqKqlO\ndJ0HR/seB+6K/u6Q1FjSaSU4tktIlUgK0R9Fa0Jn2jTgP/y++t4BGBy1a7vS+T/gaTP7zsx+yHsA\njwB/K2l7sZmtJXz4nUhocukDnGdmn0dFHgLWEj7g+/P7DtCijCN8SP4g6afCCkVNNkOAHdIc/wRg\npqSVhE7nDma2BiAa1ZR2RI+ZfUkYJrovoTlmIaEm0s7M3i0klLOBgwk14FvZuBkrne2iGFYSmpq2\nAv4W7RtN6Pz9Itr3K+GbezoDojILgFls3FldJDP7hdCpfgqhielL4Ohody9gBGEY7i/RcQ9OdxxX\nuShbF9lRGCv/mpntLak+MNvM0rXj5pX/GLjUzN6roBCdcy7rVImagpmtAL6RdCaAgvw27qh/YUvC\n0EjnnHOFyMqkIGkQ4QO+uaT5ki4kVJ0vlPQJodkgtf2yA/CCZWu1yDnnKkjWNh8555wrf1lZU3DO\nOZcZWXd3YaNGjaxZs2ZJh+Gcc1llypQpP5lZ4+LKZV1SaNasGZMnT046DOecyyqSirtLHvDmI+ec\ncyk8KTjnnMvnScE551w+TwrOOefyeVJwzjmXz5OCc865fJ4UnHPO5fOk4Jxzld26dXDPPfDRRxk/\nVdbdvOacc9XK1Klw4YUwbRqsWAEHHpjR02WspiCpiaTxkmZJminpyjRl2kRryE6LHrdkKh7nnMsq\na9bAddfBQQfBDz/AkCGhtpBhmawprAf+aWZTJW0BTJE01sxmFSg30cziLojunHNV34QJcNFF8OWX\noZbQsydsuWWFnDpjNQUzW2hmU6PnvwCfAdtn6nzOOZf1li+HLl2gTRvIzYU33oC+fSssIUAFdTRH\nS2e2IizUXtChkj6R9F9JexXy/s6SJkuavHjx4gxG6pxzCXn1VdhrL3jySbj6avj0U2jbtsLDyHhS\nkFSPsDB612jZzFRTgR3NbF+gNzA83THM7AkzyzGznMaNi5351TnnsseiRdChA5x6aqgRvP8+PPAA\nbL55IuFkNClIqkVICAPNbGjB/Wa2wsxWRs9HArUkNcpkTM45VymYwXPPwZ57wtCh0KMHTJkSOpYT\nlMnRRwL6AZ+Z2YOFlPljVA5JB0Xx/JypmJxzrlL49lv405/g3HNh993h44/hllugdu2kI8vo6KPD\ngHOB6ZKmRdtuAJoCmNnjwF+ASyStB9YAHcwXjXbOVVUbNkCfPtC9e6gp9OoFl14KNWsmHVm+jCUF\nM3sHUDFlHgEeyVQMzjlXaXz2GXTqBO+9B8cdB088AZVwaWGf5sI55zJp3Tq4807Yb7+QGJ55BkaP\nrpQJAXyaC+ecy5zJk8PNZ59+CmeeCb17wzbbJB1Vkbym4Jxz5W31aujWDQ4+GBYvhmHD4KWXKn1C\nAK8pOOdc+Ro/PkxR8dVX4d/774eGDZOOKjavKTjnXHlYtgw6d4Zjjgmvx40LnclZlBDAk4JzzpXd\nK6+EKSr69YNrrgl9CEcfnXRUpeJJwTnnSuvHH+Gss6B9e2jUCCZNCjOabrZZ0pGVmicF55wrKTMY\nMCBMUTF8ONxxRxhplJOTdGRl5h3NzjlXEnPnwsUXw5gxcNhhYWrrPfZIOqpy4zUF55yLIzc3TEux\n997hruRHHoG3365SCQG8puCcc8WbNSvchPbBB3DiifD449C0adJRZYTXFJxzrjBr18Ltt4cpKr78\nEp59Fl5/vcomBPCagnPOpffhh6F2MGNGWASnVy/Yeuuko8o4ryk451yqVavgn/+EQw+FpUthxAgY\nNKhaJATwmoJzzv3Pm2+GqSm++SaMMLrvPmjQIOmoKpTXFJxzbunS0FR07LFhwZu33gqdydUsIYAn\nBedcdTd0KLRoAf37w3XXhSkqjjoq6agS481Hzrnq6Ycf4LLLYMiQMLro9ddh//2TjipxXlNwzlUv\nZvD002GKitdeg7vvDiONPCEAXlNwzlUnX38dOpDfeAMOPzxMUdG8edJRVSpeU3DOVX25ufDvf8M+\n+4S7kvv0gQkTPCGk4TUF51zVNnNmGFk0aRKcdBI89hg0aZJ0VJWW1xScc1XT2rXQowe0ahWWxhw4\nEF591RNCMbym4Jyrej74ADp1CrWEs88OTUeNGycdVVbwmoJzrupYuRK6doXWrWH58jC6aOBATwgl\n4DUF51zVMHYsdO4cFsH5xz/gnnugfv2ko8o6XlNwzmW3JUvgggvg+OOhdu2w8M2jj3pCKCVPCs65\n7GQGgweHKSqefRauvx4++QSOOCLpyLKaNx8557LP99/DpZfC8OHhTuRRo8JUFa7MvKbgnMseZuEu\n5BYtQiK4775w/4EnhHKTsaQgqYmk8ZJmSZop6co0ZSTpYUlzJH0qyScfcc6lN2cOtG0b1jvYb78w\nm+m118Im3uBRnjJZU1gP/NPMWgCHAJdKalGgzInAbtGjM/BYBuNxzmWj9evhX/+Cli1hypSwzsG4\ncbDbbklHViVlLMWa2UJgYfT8F0mfAdsDs1KKnQYMMDMDPpDUUNK20Xudc9Xdp5+GKSomT4ZTTglz\nFu2wQ9JRVWkV0qcgqRnQCphUYNf2wLyU1/OjbQXf31nSZEmTFy9enKkwnXOVxW+/wc03wwEHwLff\nwgsvwCuveEKoABlPCpLqAUOArma2ojTHMLMnzCzHzHIa+52JzlVt774b+gzuvBM6doTPPoOzzgIp\n6ciqhWKbjyTtClwNNEstb2bHx3hvLUJCGGhmQ9MUWQCkzk61Q7TNOVfdrFwJN9wAjzwSJq0bORJO\nPDHpqKqdOH0Kg4F+wHNAbtwDS1L0vs/M7MFCio0ALpP0AnAwsNz7E5yrhkaNCovfzJsX7j+4+27Y\nYouko6qW4iSFDWbWuxTHPgw4F5guaVq07QagKYCZPQ6MBP4EzAFWAxeU4jzOuWz1889w1VXhjuQ9\n9oB33gmT2bnExEkKr0jqDAwDfsvbWFz/gJm9AxTZCBiNOro0RgzOuarEDF56CS6/HJYuhZtughtv\nhE03TTqyai9OUugU/XtzyjYj+sbvnHMlsmBBmMV0xAjIyQmzm+67b9JRuUixScHMfJki51zZbdgQ\npqjo1g3WrQs3pF15pd+RXMnEGX20CeFu4yOjTW8Bfc1sfQbjcs5VJV9+GaanmDABjj4annwSdtkl\n6ahcGnHuU3gUaA08FT1aA30yGZRzropYvx7uvz9MUTFtWkgGb77pCaESi1NvO8TMUhv8xkj6JFMB\nOeeqiGnTwhQVU6dC+/Zh4Zvttks6KleMODWFDdE0FUD+lBUbMhOOcy7r/fprGEmUkwPz58PLL8PQ\noZ4QskScmsJ1wERJswlDTHcFLsxoVM657PTOO9CpE8yeDeefDw88AFttlXRUrgTijD4aI2l3YM9o\n02dmtiazYTnnssqKFWE5zD59oFkzGD06rJnssk6hSUHSUWY2QdKpBXbtIAkzG5Hh2Jxz2WDkSOjS\nJTQVXXllmMiuXr2ko3KlVFRN4ThgAnBmmn1GmLfIOVdd/fQTdO0KAweG5THffRcOPTTpqFwZFZoU\nzOym6OmNZvZd6j5Jfjezc9WVWVjf4IorYPlyuPXW0HRUp07SkblyEGf00fCY25xzVd28eWEFtLPP\nhp13DsNNb7vNE0IVUlSfQl7ncoMC/Qr1AZ+1yrnqZMMG+M9/4LrrIDcXHnww1BRq1kw6MlfOiupT\n2As4HWjIxv0KvwAXZzIo51wl8sUXYZjpxInQti088USoJbgqqag+hWHAMEmHR9NgO+eqk3Xrwn0G\nt90GdevCU0+Few98WcwqLU6fwgWSGua9kLSlpCczGJNzLmlTp8JBB4UO5JNPhlmz4IILPCFUA3GS\nwv5mtizvhZktBQ7IXEjOucSsWQPdu4eE8MMPMGQIDB4M226bdGSugsSZ5qKGpAZmthxCTQGoldmw\nnHMV7u23Q9/Bl1/C3/8e1jvYcsuko3IVLE5S+DfwvqQXCXMf/RW4P6NROecqzooVYVTR44/DTjuF\nldCOPTbpqFxC4sx99LSkqcDR0aYOZvZpZsNyzlWIV1+FSy6BhQvh6qvh9tth882TjsolKNY6eGb2\niaR5RPcnSNrOzL7PaGTOucxZtCjMU/TCC7D33qHv4OCDk47KVQLFdjRLOknSF8B84ANgHjAu04E5\n5zLADJ57LsxVNGQI9OgBU6Z4QnD54ow+ugs4DJhtZk2BE4CJGY3KOVf+vvsOTjoJzj0XdtsNPv4Y\nbrkFatdOOjJXicRJCuvNbDFhFJLMbCxwUIbjcs6Vlw0bwlKYe+0FEyZAr15hMZy99ko6MlcJxelT\nWC6pHvAOMEDSIsAX2XEuG3z+eRhm+u67cNxxYf6inXZKOipXicWpKbQnJIGuwFvAAuCUDMbknCur\ndevgrrtg333D3cjPPBNWQ/OE4IoRp6awHthgZrmS3gKaA0szGpVzrvSmTAk3n336KZx5JvTuDdts\nk3RULkvEqSlMBOpK2pYw6ugi4KmMRuWcK7nVq+Haa8MUFYsXw7Bh8NJLnhBcicRJCjXMbDVwBvCY\nmf0ZaJnZsJxzJTJ+PLRsCT17hlrCrFnQvn3SUbksFCspSDoQ+BvwWrSt2JU1JD0laZGkGYXsbyNp\nuaRp0eOW+GE75wBYtgw6d4Zjjgmvx42DJ5+Ehg2Lfp9zhYiTFK4GegCvmdkMSTsT7z6FZwj3NBRl\nopntFz1uj3FM51yeV14Jw0r79YNrrgl9CEcfXfz7nCtCnLmPxpFyB7OZfQ38I8b73pbUrCzBOefS\n+PHHsBTmSy+FJqNXXoGcnKSjclVEsUlB0q6E2kKz1PJmdnw5nP9QSZ8A3wPXmNnMcjimc1WTGTz7\nLHTtCqtWwR13hNlNa/lM9q78xBmSOhjoBzwH5JbjuacCO5rZSkl/AoYDu6UrKKkz0BmgadOm5RiC\nc1li7ly4+GIYMwZat4a+fWHPPZOOylVBcfoUNphZbzN7z8wm5T3KemIzW2FmK6PnI4FakhoVUvYJ\nM8sxs5zGjRuX9dTOZY/cXHj44TCT6XvvhXsOJk70hOAyJk5SeEVSZ0mNJdXPe5T1xJL+KIUFXyUd\nFMXyc1mP61yVMWsWHHFEmOL6iCNgxgy47DKoEefP1rnSidN81Cn69+aUbQYU2Y4jaRDQBmgkaT5w\nK9Eynmb2OPAX4BJJ6wnTaHQwMytR9M5VRWvXwn33wZ13Qr16MGAAnHMOhO9QzmVUnNFHTUpzYDPr\nWMz+R4BHSnNs56qsjz6CCy+E6dOhQ4cwo+nWWycdlatGYq28JmkPoAXRymsAZvZ8poJyrtpZvRpu\nvRUefBC23RZGjIBTfN5JV/HiDEm9CTge2AMYDbQjTKPtScG58jBuHFx0EXz9dRhhdN990KBB0lG5\naipOj9VZwNHAQjM7F9gX8JW9nSurZctCMmjbNnQev/UWPP64JwSXqDhJYY2Z5QLrJW0B/ADsmNmw\nnKvihg8P6yQ//XS4Ae3TT+Goo5KOyrlYfQofS2pImC57MrAC+DCjUTlXVf34I1x+Obz8Muy3H7z2\nGuy/f9JROZevyKQQ3Udwm5ktAx6VNBqob2ZTKyQ656oKM+jfH66+OnQq3313mMTOp6hwlUyRScHM\nTNJYYO/o9ZwKicq5qmTu3DC99dixcPjhYYqK5s2Tjsq5tOL0KUyT1CrjkThX1eTmhvsM9toL3n8f\n+vSBCRM8IbhKLU6fQivgI0lfAasAESoR3hDqXGFmzgw3oU2aBCedBI89Bk1KdR+ocxWq0KQgaRMz\nWw+cWoHxOJfd1q6Fe+6Bu+4KQ0sHDoSOHX2KCpc1iqopfAjsb2ZfVVQwzmW1SZNC7WDmTDj7bPj3\nv8Fn9XVZpqg+Bf9q41wcq1aFUUWHHgrLl4dhpgMHekJwWamomkJjSVcXttPMHsxAPM5llzfeCCOL\nvvkGLrkE7r0X6pd5ZnnnElNUUqgJ1MNrDM793tKl8M9/hjuSd989jCo68siko3KuzIpKCgvN7PYK\ni8S5bDFkSFjsZvFi6N49zG666abFv8+5LFBUUvAagnOpFi4MyWDoUGjVCkaODP86V4UU1dHctsKi\ncK4yM4OnngoT2I0cGfoNPvzQE4KrkgqtKZjZkooMxLlK6euvQ0fym2+GPoMnnwx9CM5VUYXWFCTV\nqchAnKtUcnPDKmh77x1qBY89BuPHe0JwVV5RzUfvA0h6toJica5ymD4dWrcOo4vatoVZs6BLl7AQ\njnNVXFEdzbUlnQ20lnR6wZ1mNjRzYTmXgN9+C1Na3303bLklDBoEZ53lU1S4aqWopNAF+BvQECi4\ngrgBnhRc1fH++9CpU6gVnHsuPPQQ/OEPSUflXIUrqqP5HeAdSZPNrF8FxuRcxVm5Em68EXr3DrOY\njhwJJ56YdFTOJSbO1NnPSroCyLtdcwLwuJmty1xYzlWAMWPCyKLvvoNLLw3NRltskXRUziUqTs9Z\nH+CA6N8+wP7AY5kMyrmMWrIEzj8f2rWDunXhnXdCTcETgnOxagoHmtm+Ka/HSfokUwE5lzFmMHhw\nuCt5yRK46abQdORTVDiXL05SyJW0S966CpJ2BnIzG5Zz5ez77+Ef/4BXXoGcnNB0tO++xb/PuWom\nTlLoBoyX9DVhPqQdgQsyGpVz5cUM+vaFbt3Cqmj/+hdceSVsEue/vnPVT7F/GWb2pqTdgLzVxmeb\n2W+ZDcu5cjBnTuhIHj8ejj46TFGxyy5JR+VcpRbr61KUBD7NcCzOlY/168NSmLfcArVrh2Rw4YV+\nE5pzMWSsDi3pKeBkYJGZ7Z1mv4BewJ+A1cD5ZjY1U/FUFc26v550CBknQeudt2Luz2v4ftkatmtY\nl6P3aMz4zxfz/bI1bFqrBr+t38AGC+2ZtTcJrwH2XPQ19/33YVr+MIcxux3Czcd1YeV327D6+pFY\nmnPV2aQG953Rkvatts/fdtPw6QyaNI9c+987tm9Yl27tmueXG/7xAnqOns33y9bQoG4tJFi2eh0N\n6tZi9dr1rM21/Gv528FNydlxq/zy26Ucq+Bx1uVuYNXa/3XZNaxbi9tO3Sv/vKmx1ZQ4ZOctN/o5\npcaYF2ePV2eydPW6Qo+ZKjWegj/3dMcv6r1FlS3NuTJ9/IpU1mvJZOwyS/enUg4Hlo4EVgIDCkkK\nfwIuJySFg4FeZnZwccfNycmxyZMnl3e4WaE6JITSqrN+LZe99yJdJg1m2aZbcMtxXfhv88Ni1Q5q\nCB786360b7U9Nw2fznMffJe2XN1aNbnn9H0AuH7odNasiz/eogawocCxzjhge4ZMWVDscWrVED3P\n3JfJ3y4pNLaCMeYlnG6DP2Fd7u//xvOOWfBDt7jrSj1+qnTvLaxsac6V6eNXpPK4ltLELmmKmeUU\nV67Y+xQkvRlnW0Fm9jZQ1PTbpxEShpnZB0BDSdsWd1znCjpg/ixef/oKLn//RV5p0YbjOvXhv3sc\nHru5aINBz9GzARg0aV6h5dasy6Xn6Nn0HD27RAkBNk4IeccaNGlerOOs22D0HD27yNgKxgjhmtIl\nhNRjpopzXanHL+69hZUtzbkyffyKVB7XksnYC20+krQpsBnQSNKW/G8ltvpAeaTW7YHU/+Xzo20L\n08TSGegM0LRp03I4dfbZ9XqvJRS0+W+r6fb2AM6b+jrf12/MeWf24O2dDyjVsb5ftgZgoyajosqV\nh+LOVfC8cUvnxVhcrAX3x722dOUKe29JtxdWLtPHr0jldS2Zir2omsLFwBRgj+jfvMcrwCMZiaYQ\nZvaEmeWYWU7jxo0r8tSVwk3Dp7M+M618WavNV5MZ0+9Szpv6Ov0POJnjL3y01AkBYLuGdQGoWUzt\nYruGdfPLllVx5yp43rjl8+IrLs6C++NeV7pyhb23pNsLK5fp41ek8rqWTMVeaFIws15mthNwjZnt\nbGY7RY99zaw8ksICoEnK6x2iba6A4tqRq5MtVy/nwdce4JnBt7G69qb85Zz76XHsxayuXfo/kBqC\nbu3CiOuOBzcptFzdWjXp1q453do1p26tmiU7R5pjdTy4Sazj1KohurVrXmRsBWOEcE21aqZPJHnH\nTBXnulKPX9x7CytbmnNl+vgVqTyuJZOxx7lPobek1kCz1PJmNqCM5x4BXCbpBUJH83Iz+13TUXV3\n3INvJR1ChUs7+qh5I2q89BJXjHiEBr+u5OHDOvLIIX9l3Sa1qJMy+mij4xDmeN+8dk1Wr82NNfro\nzvahI7m40UdAuYw+St1e3OijvHPHHX2U92/c0Ud5r0szYifde4saIVPSc2X6+BWpPK4l0dFH0cpr\nuwDT+N/0FmZmVxTzvkFAG6AR8CNwK1ArevPj0ZDUR4ATCENSLzCzYocVVafRR0WNhEl1ziFN8z/M\nKrudur+e9sNZwDf3npT+TfPnhykqXn0VDjwQ+vWDfbLjep2rLOKOPopzn0IO0MJKOHbVzDoWs9+A\nS0tyzOomTkKoX6dm1iQECO2gC9J0kKVtH92wAZ54Aq69NtyQ9sADYYqKmiVrunHOxRdn6uwZwB8z\nHYjbWNzRRp/2OCHDkZSv2O2jX3wRpqa45JJQO5gxA66+2hOCcxkWp6bQCJgl6UMgf84jMzs1Y1G5\nWKONdtt688wHUs6KbR/NqxHcemuY0rpfP7jgAp+iwrkKEicp3JbpINzG4t65PPbqNpkNJENSO003\nMm1amKNo6lQ4/XR45BHY1u9ndK4ixRl9NKEiAnFB3GajuYV1ymajNWvg9tuhZ09o1CgshHPGGUlH\n5Vy1VGxSkPQL5A8YqU0YQbTKzOpnMrDqKk6zUf06VahdfeJE6NQp9CH8/e9hvYMtt0w6KueqrTg1\nhfyFa6NhpKcBh2QyqOoqbrNRtnUup7ViBXTvDo89BjvtBGPHwrHHJh2Vc9VenNFH+aLJ64YD7TIU\nT7UV9ya1KtFs9PrrsNde8J//wFVXwfTpnhCcqyTiNB+dnvKyBuG+hV8zFlE19eWiVcWWyfqEsHhx\nuM9g0KCQFAYPhoOLnS3dOVeYOeskAAAWkUlEQVSB4ow+OiXl+XpgLqEJyZWTOM1Gm2TziEwzeP75\nkBBWrIAePULTUe3aSUfmnCsgTp/CBRURSHXV8tZRscrNuSdLawnz5kGXLjByJBxyCPTtG2oJzrlK\nKc4iOztIGiZpUfQYImmHigiuOljxW/GLrGRlLWHDBujTJySAt94Kaya/844nBOcquTgdzU8TZjTd\nLnq8Gm1zZRR3tFHW1RJmz4ajjoJLLw21g5kzfc4i57JEnKTQ2MyeNrP10eMZoPqtdFPO4iaEbOlc\nHv7xAo68aww9j/o/ftt7H9Z+Oh2eeQZGj4ZmzZIOzzkXU5yk8LOkcyTVjB7nAD9nOjCXXQnh2UeH\n8vjDXej29gDG7nIwx/z9MYa3PNbnLHIuy8QZffR3oDfwEOHO5vcA73wug7i1hKywZg0ru/6TFye+\nzM+bN6Tzn29kzO6HAmHSuyQWMXHOlV6c0UffAj4jajmJO9ooK2oJEybARRdxzpdfMqjl8dxz9N9Z\nsWm9/N1JLIrunCubODev7QRczu+X4/REUQpxRhtVesuXw3XXhTuSd96Zyzv9i1f/sMfviiWxKLpz\nrmziNB8NB/oRRh39fiFcF1uV6Fx+9dWw8M3ChXDNNdCjB21nL+WNodNZs+5/CS+pRdGdc2UTJyn8\namYPZzySKi7rp8RetCgMK33hhbA+8rBhYUU0oH2rzYCKW1jcOZc5cZJCL0m3AmPYeOW1qRmLqgqK\nMyV2pWQGzz0HXbvCypVwxx1hzeQCU1QUunCOcy6rxEkK+wDnAsfwv+Yji167GLK22ejbb8MUFaNG\nQevWYYqKPfdMOirnXAbFSQpnAjub2dpMB1MVxU0I5xzSNMORlEDeFBXdu4fXvXvDP/4BNUo007pz\nLgvFSQozgIbAogzHUm1tIriz/T5JhxF8/nlYCe3dd+GEE+Dxx2HHHZOOyjlXQeIkhYbA55I+YuM+\nBR+SWoy4ncuVYm6jtWvh/vtDn0G9etC/P5x7rt+R7Fw1Eycp3JrxKKqoOJ3LlaLZ6KOP4MILwwpo\nZ50FDz8MW2+ddFTOuQQU20hsZhNSH0Au8NfMh5bd4vYlJNpstHp1uNfgkEPg55/hlVfCkFNPCM5V\nW3FqCkhqBZxN6HT+BhiSyaCyXVaMNho3Djp3hq++gosvhvvugwYNkovHOVcpFJoUJO0OdIwePwEv\nAjKzoysotqxU6UcbLVsG3bqF4aW77grjx0ObNsnE4pyrdIpqPvqccC/CyWZ2uJn1JjQduXKQSLPR\n8OHQogU8/XS4Ae3TTz0hOOc2UlRSOB1YCIyX9KSktoAPRSlCpW02+uEHOPNM+POfQ3/BpEmhuaiu\nT1jnnNtYoUnBzIabWQdgD2A80BXYWtJjko6Pc3BJJ0iaLWmOpO5p9p8vabGkadGjU2kvJGlxp8Te\nbevNMxxJCrOw+lmLFmEiu7vvDiONDjig4mJwzmWVOKOPVpnZ82Z2CrAD8DFwXXHvk1QTeBQ4EWgB\ndJTUIk3RF81sv+jRt2ThVx5xp8Qee3WbzAaSZ+5caNcOLrggJIVp0+D666FWrYo5v3MuK5Vo3gIz\nW2pmT5hZ2xjFDwLmmNnX0RQZLwCnlSbIyq5SNRvl5kKvXrDXXvD++/Doo/D227DH79c7cM65gjI5\nmc32wLyU1/OjbQWdIelTSYMlNUl3IEmdJU2WNHnx4sWZiLXUbho+PVa5CkkIM2fCYYeFGU3btAmv\nfc4i51wJJP1p8SrQzMxaAmOB/ukKRbWTHDPLady4cYUGWJznPvgu6RDCFBU9ekCrVjBnDgwcCK+9\nBk0rwd3SzrmsEuvmtVJaAKR+898h2pbPzH5OedkXuD+D8ZS7StFsNGlSmMBuxgzo2DE0HVWyxOmc\nyx6ZrCl8BOwmaSdJtYEOwIjUApK2TXl5KvBZBuMpV3GbjTJ2k9qqVXD11XDooeGGtNdeg+ef94Tg\nnCuTjNUUzGy9pMuA0UBN4CkzmynpdmCymY0ArpB0KrAeWAKcn6l4ylvcZqOM3KT25ptw0UXwzTdh\nveR774X69cv/PM65akdm2bVOZE5Ojk2ePDnRGBJrNlq6NExg99RTsPvu8OSTcOSR5XsO51yVJGmK\nmeUUVy7pjuasc9yDbyVz4qFDw/0G/fuHFdE++cQTgnOu3GWyo7nSGP7xAnqOns33y9awXcO6dGvX\nvNSLzH+5aFWscuVWS1i4EC67LCSFVq1g5Mjwr3POZUCVrykM/3gB1w+dzoJlazBgwbI1XD90OsM/\nXlDsewuq0GYjs9BM1KJFSAT33gsffugJwTmXUVU+KfQcPZs16zaegmLNulx6jp6dUEQxfP01HH98\nWA2tZcvQVHTddbBJtajYOecSVOWTwvfL1pRoe2EqpJaQmwsPPQT77BPuP3jssbDewe67l/6YzjlX\nAlU+KWzXMP300IVtT6dCEsKMGdC6dbj34JhjYNYs6NLFp6hwzlWoKv+J061dc+rWqrnRtrq1atKt\nXfOEIirgt9/g1lth//3DfQeDBsGIEbDDDklH5pyrhqp8Umjfanv2b7rx2sP7N20Qe/RRRmsJH3wQ\nksHtt8NZZ4XaQYcOIF/LyDmXjCqfFG4aPp13v1qy0bZ3v1oSa5qKjCWElSvDTKatW8Mvv4TRRc8+\nC40alew4zjlXzqp8UihsOorEZjcdMwb23hsefhguvTRMb33iicnE4pxzBVT5pFCUw+4dV+j9CuVe\nS1iyBM4/P6yGVrcuTJwIvXvDFlvEjNY55zKvWieFwm5kK9eEYAYvvxxuQhs4EG68ET7+OCyG45xz\nlUy1vxtqzbpcur44jZ6jZ5d4+otip8/4/vvQRDR8OBxwAIweDfvum4GrcM658lGtawqp8moNcWsJ\n/z5rv8KnzzCDvn1D7WDUKOjZM4w08oTgnKvkPCmkKDgdRmF223rzQqfPeH7gOGjbNqx30KoVTJ8e\nprv2KSqcc1nAk0IpjL26ze+myai5IZeLJg1lQK9OMGUKPPEEjBsHu+6aUJTOOVdy/vW1hPI6l7dr\nWJcFUWLYc9HX3Pffh2n5wxwm7tmaI8a+BNuXbmpu55xLktcUSiB1veVu7ZrTQLn88+1nGdH/KrZd\n8RNdT7+Bn5970ROCcy5reU2hBFLXW26/ei5tX/gnW8ydw5C9j6Fv+8u4+PSDSr14j3POVQaeFGLK\nvyfhl1/ghhvg0UfZokkTGDWKM9q144xkw3POuXLhzUcxbJI3P92oUWGKikcfDUtkzpwZ7lB2zrkq\nwmsKMcy55hA477wwad2ee8K778KhhyYdlnPOlTtPCkUxY+7+q0IiWLoUbr45TFNRp07SkTnnXEZ4\nUijEH1f8xB1j+8D9H8KBB8IbbzA89w/0fOjdtNNaFDvlhXPOZQFPCgXINtDxk9F0H/80tTbkMuOq\nm5lzdid6DP+cpavn5ZdbsGwNXV+cRtcXp9Gwbi1WrV3PulzL33f90LBeQ2kSgycY51xSZGZJx1Ai\nOTk5Nnny5Njl485lBNBsyQLuHdWbQ+bN4N0dW3J9u8v5bsttSxNmPgF/O6TpRsNZ86T78Ae4fuj0\n302hsXntmtz15328ZuKcKxVJU8wsp9hynhTCFBWdPhrGVe88z9qatbjz6At5qeVxFb4sZq0aYv0G\no7S/kYZ1a3HbqXsx+dslDJo0j1wzakp0PLhJ2qTknKs+4iaFat981OLHr7nvv73Y58ev+O/urbnl\nuC4srrdVIrGs21C2BL1szTq6vjhto225Zjz3wXd8s3glAy/yEVPOuaJV26RQZ/1arnh3EBdPGsLS\nzerTpf31jGpedRe+eferJTTr/jp1NqnBfWe09KYm51xa1TIpHDhvBveO6s0uSxbw0j7HcucxnVix\nab2kw6oQv63fkN9Bfk4hfR3Oueoro0lB0glAL6Am0NfM7i2wvw4wADgA+Bk4y8zmZiqeer+t5toJ\n/Tnv49eZ12AbzvnrHbyzU6tMna7Se+6D7wA8MTjn8mVsmgtJNYFHgROBFkBHSS0KFLsQWGpmuwIP\nAfdlKp6D5s1gTL9/cM7HI+mXcxrH//3Rap0Q8gyaNK/4Qs65aiOTNYWDgDlm9jWApBeA04BZKWVO\nA26Lng8GHpEky8CQqF/qbMbSuvW59LTufLz9HuV9+KyVm2Wjz5xzmZXJpLA9kPo1dD5wcGFlzGy9\npOXAH4CfUgtJ6gx0BmjatCml8dnWO3PS+b0qfJhpZVfTfx7OuRRZMUuqmT1hZjlmltO4cePSH8g/\nAH+n48FNkg7BOVeJZDIpLABSP3F2iLalLSNpE6ABocPZVYDDdtnKO5mdcxvJZFL4CNhN0k6SagMd\ngBEFyowA/i96/hdgXHn3J+QvjlNNNKxbi3MOaUrDurV+t2/z2jURsH3Duvz7rP38Zjbn3O9kdJoL\nSX8C/k0YkvqUmd0l6XZgspmNkLQp8CzQClgCdMjrmC5MSae5cM45V0mmuTCzkcDIAttuSXn+K3Bm\nJmNwzjkXX1Z0NDvnnKsYnhScc87l86TgnHMunycF55xz+bJukR1Ji4FvS/n2RhS4WzqLVaVrgap1\nPX4tlVN1v5YdzazYu3+zLimUhaTJcYZkZYOqdC1Qta7Hr6Vy8muJx5uPnHPO5fOk4JxzLl91SwpP\nJB1AOapK1wJV63r8Wionv5YYqlWfgnPOuaJVt5qCc865InhScM45l6/aJAVJJ0iaLWmOpO5Jx1Na\nkp6StEjSjKRjKStJTSSNlzRL0kxJVyYdU2lJ2lTSh5I+ia6lR9IxlZWkmpI+lvRa0rGUhaS5kqZL\nmiYpq6dYltRQ0mBJn0v6TFK5z39fLfoUJNUEvgCOIywL+hHQ0cxmFfnGSkjSkcBKYICZ7Z10PGUh\naVtgWzObKmkLYArQPkt/LwI2N7OVkmoB7wBXmtkHCYdWapKuBnKA+mZ2ctLxlJakuUCOmWX9jWuS\n+gMTzaxvtE7NZma2rDzPUV1qCgcBc8zsazNbC7wAnJZwTKViZm8T1p7Iema20MymRs9/AT4jrNud\ndSxYGb2sFT2y9huXpB2Ak4C+ScfiAkkNgCOBfgBmtra8EwJUn6SwPTAv5fV8svTDp6qS1Iyw2NKk\nZCMpvai5ZRqwCBhrZll7LYTFsa4FNiQdSDkwYIykKZI6Jx1MGewELAaejpr1+kravLxPUl2SgqvE\nJNUDhgBdzWxF0vGUlpnlmtl+hPXID5KUlc17kk4GFpnZlKRjKSeHm9n+wInApVETbDbaBNgfeMzM\nWgGrgHLvH60uSWEB0CTl9Q7RNpewqP19CDDQzIYmHU95iKr044ETko6llA4DTo3a4l8AjpH0XLIh\nlZ6ZLYj+XQQMIzQnZ6P5wPyUGuhgQpIoV9UlKXwE7CZpp6hzpgMwIuGYqr2oc7Yf8JmZPZh0PGUh\nqbGkhtHzuoRBDZ8nG1XpmNn1ZraDmTUj/K2MM7NzEg6rVCRtHg1iIGpqOR7IypF7ZvYDME9S82hT\nW6DcB2VkdI3mysLM1ku6DBgN1ASeMrOZCYdVKpIGAW2ARpLmA7eaWb9koyq1w4BzgelRWzzADdHa\n3tlmW6B/NNKtBvCSmWX1UM4qYhtgWPj+wSbA82Y2KtmQyuRyYGD05fZr4ILyPkG1GJLqnHMunurS\nfOSccy4GTwrOOefyeVJwzjmXz5OCc865fJ4UnHOuEivJJJiSHoom/psm6QtJJZ4Gw5OCK5KkHSS9\nIulLSV9J6hUNh0NSm8Jm0IxmpmxUyPa8GSunSyp2DqrCjpUUSUdEM6FOi+5JKLi/vSSTtEcS8RWI\npY2k1oXsO1/SIwW2vSWpyAXhi/jd3ibpmrJF7NJ4hpg3QprZVWa2X3RnfW+gxDeEelJwhYpuLhsK\nDDez3YDdgXrAXWU89NHRf9q/AA+X8VhJ+BtwT/THtybN/o6EWVI7VmxYabUB0iaFpEmqFvdJlVW6\nSTAl7SJpVDSf08RCvoB0BAaV9HyeFFxRjgF+NbOnIcztA1wF/F3SZqkFJf1B0pjoG3RfQDGOXx9Y\nmnKM4dF/8pmFTVxWWBlJKyXdFa1n8IGkbaLt20gaFm3/JO9bs6RzFNY/mCbpP9FNZwXP1TaaeGx6\nVIWvI6kT8FfgDkkD07ynHnA4cCHhbuDUfddFx/pE0r3Rtl0lvRFtmxr9sUtST0kzovJnRWU3qplJ\nekTS+dHzuZJ6RMeYLmkPhUkGuwBXRdd5RIzfSWq8HaNjzZB0XyFlboyaKd4BmqdsT/uhJekZSY9L\nmgTcX5J43EaeAC43swOAa4A+qTsl7UiYQG9ciY9sZv7wR9oHcAXwUJrtHwMtCd9CX4u2PQzcEj0/\niTAzZaM0750LTCdMNbAaODll31bRv3Wj/X9IeU+jYsoYcEr0/H7gpuj5i4SJ9iDczd4A2BN4FagV\nbe8DnFcgzk0JM+vuHr0ekHKcZ4C/FPIz+xvQL3r+HnBA9PzE6PVmBa5jEvDnlHNuBpwBjI3i3Qb4\njnDHdP7POyr/CHB+ys/o8uj5P4C+0fPbgGsKifV8wqyb01IeKwlrKGwXnbcx4U7gcYS1LvJ/H8AB\n0e9yM0KCn5N3LuBNYLfo+cGEqTLyfnavATWT/v+dTQ+gGTAjel4PWFPg9/ZZgfLXAb1Lcy6vvrny\nciRwOoCZvS5paRFljzaznyTtArwp6S0LaxFcIenPUZkmwG7AzwXeW1iZtYQPGwiL9RwXPT8GOC+K\nKxdYLulcwgfaR6GFjLqE6a5TNQe+MbMvotf9gUsJU0oXpSPQK3r+QvR6CnAs8LSZrY5iWaIwJ8/2\nZjYs2vYrgKTDgUFRvD9KmgAcCBQ3g2xe+/EUot9FDC+a2WV5LyS9FT09EHjLzBZH2wcSfsfDU957\nBDAs75okjYj+rUdosno5+vkC1El538vRtbnSqQEss9AEW5gOhP+vJeZJwRVlFqHdP5+k+kBTwrfC\nMs02aWZfSfoRaBE1Rx0LHGpmq6MPp00LnLtNEWXWWfQVCcil6P/bAvqb2fVlif93B5W2IiShfSQZ\n4Zu+SepWTqdYz8ZNvpsW2P9b9G9x159pxX1orarIYKoaM1sh6RtJZ5rZy1HfX0sz+wQgaqrbEni/\nNMf3PgVXlDeBzSSdB/nLmj4APJP37TDF28DZUbkTCf8piyRpa0K757eEZp2l0Yf9HsAhad4Sp0y6\na7gkL36F1aveBP4SnR9JW0VtsKlmA80k7Rq9PheYUMy5/gI8a2Y7mlkzM2sCfEP4Rj0WuCCvL0bS\nVhZWm5svqX20rU60fyJwVhRvY8I39A+jn1OLqFxDwiyZxfkF2CJGuYI+BI6S1Cj6vXfk99f/NtBe\nUt2o1nMKhA8t4BtJZ0bXJUn7liIGR/4kmO8DzSXNl3QhoZnyQkmfADPZeCXJDsALKV+SSsRrCq5Q\nZmZRU00fSTcTvkSMBG5IU7wHMEjSTELb+XdFHHq8pFzCkpXdzexHSaOALpI+I3wgp1vbOE6Zgq4E\nnoj+kHKBS8zsfUk3EVbjqgGsI1S1v0259l8lXUBoAtmEMP3648WcqyNQsEN2CGE98Esk7QdMlrSW\n//0czwX+I+n2KI4zCXP+Hwp8QugrudbCtMlIeonQl/INoW+nOK8CgxWG/l5uZhNjvAczWyipO2Fd\nCAGvm9krBcpMlfRiFOciws8oz9+Ax6Kfcy1CU9oncc7tNmZmhY1iSztM1cxuK8v5fJZU55xz+bz5\nyDnnXD5PCs455/J5UnDOOZfPk4Jzzrl8nhScc87l86TgnHMunycF55xz+f4fh7r+Ot7WiZQAAAAA\nSUVORK5CYII=\n", 151 | "text/plain": [ 152 | "" 153 | ] 154 | }, 155 | "metadata": {}, 156 | "output_type": "display_data" 157 | } 158 | ], 159 | "source": [ 160 | "#Define the limits of the x-axis \n", 161 | "\n", 162 | "x_lim = np.linspace(min(feature), max(feature)).reshape(-1, 1)\n", 163 | "\n", 164 | "#Creating the scatter plot\n", 165 | "\n", 166 | "plt.scatter(feature, target)\n", 167 | "plt.xlabel('Old Balance of Account Holder')\n", 168 | "plt.ylabel('Amount of Transaction')\n", 169 | "plt.title('Amount Vs. Old Balance')\n", 170 | "\n", 171 | "#Creating the prediction line \n", 172 | "\n", 173 | "plt.plot(x_lim, linear_reg.predict(x_lim), color = 'red')\n", 174 | "\n", 175 | "#Show the plot\n", 176 | "\n", 177 | "plt.show()" 178 | ] 179 | }, 180 | { 181 | "cell_type": "markdown", 182 | "metadata": {}, 183 | "source": [ 184 | "## Linear Regression to predict transaction amount" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": 3, 190 | "metadata": {}, 191 | "outputs": [], 192 | "source": [ 193 | "# Reading in the dataset \n", 194 | "\n", 195 | "df = pd.read_csv('fraud_prediction.csv')" 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": 4, 201 | "metadata": {}, 202 | "outputs": [], 203 | "source": [ 204 | "#Creating the features \n", 205 | "\n", 206 | "features = df.drop('isFraud', axis = 1).values\n", 207 | "target = df['isFraud'].values" 208 | ] 209 | }, 210 | { 211 | "cell_type": "code", 212 | "execution_count": 7, 213 | "metadata": {}, 214 | "outputs": [], 215 | "source": [ 216 | "X_train, X_test, y_train, y_test = train_test_split(features, target, test_size = 0.3, random_state = 42, stratify = target)" 217 | ] 218 | }, 219 | { 220 | "cell_type": "markdown", 221 | "metadata": {}, 222 | "source": [ 223 | "**Fitting and evaluating the accuracy of the linear regression model**" 224 | ] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "execution_count": 8, 229 | "metadata": {}, 230 | "outputs": [], 231 | "source": [ 232 | "#Initializing a linear regression model \n", 233 | "\n", 234 | "linear_reg = linear_model.LinearRegression()" 235 | ] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": 10, 240 | "metadata": {}, 241 | "outputs": [ 242 | { 243 | "data": { 244 | "text/plain": [ 245 | "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)" 246 | ] 247 | }, 248 | "execution_count": 10, 249 | "metadata": {}, 250 | "output_type": "execute_result" 251 | } 252 | ], 253 | "source": [ 254 | "#Fitting the model on the data\n", 255 | "\n", 256 | "linear_reg.fit(X_train, y_train)" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": 11, 262 | "metadata": {}, 263 | "outputs": [ 264 | { 265 | "data": { 266 | "text/plain": [ 267 | "0.98579848177097251" 268 | ] 269 | }, 270 | "execution_count": 11, 271 | "metadata": {}, 272 | "output_type": "execute_result" 273 | } 274 | ], 275 | "source": [ 276 | "#Accuracy of the model\n", 277 | "\n", 278 | "linear_reg.score(X_test, y_test)" 279 | ] 280 | }, 281 | { 282 | "cell_type": "markdown", 283 | "metadata": {}, 284 | "source": [ 285 | "**Scaling your data**" 286 | ] 287 | }, 288 | { 289 | "cell_type": "code", 290 | "execution_count": 16, 291 | "metadata": {}, 292 | "outputs": [ 293 | { 294 | "data": { 295 | "text/plain": [ 296 | "0.98579848176994012" 297 | ] 298 | }, 299 | "execution_count": 16, 300 | "metadata": {}, 301 | "output_type": "execute_result" 302 | } 303 | ], 304 | "source": [ 305 | "#Setting up the scaling pipeline \n", 306 | "\n", 307 | "pipeline_order = [('scaler', StandardScaler()), ('linear_reg', linear_model.LinearRegression())]\n", 308 | "\n", 309 | "pipeline = Pipeline(pipeline_order)\n", 310 | "\n", 311 | "#Fitting the classfier to the scaled dataset \n", 312 | "\n", 313 | "linear_reg_scaled = pipeline.fit(X_train, y_train)\n", 314 | "\n", 315 | "#Extracting the score \n", 316 | "\n", 317 | "linear_reg_scaled.score(X_test, y_test)" 318 | ] 319 | }, 320 | { 321 | "cell_type": "markdown", 322 | "metadata": {}, 323 | "source": [ 324 | "## Model Optimization" 325 | ] 326 | }, 327 | { 328 | "cell_type": "markdown", 329 | "metadata": {}, 330 | "source": [ 331 | "**Ridge Regression**" 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "execution_count": 18, 337 | "metadata": {}, 338 | "outputs": [], 339 | "source": [ 340 | "# Reading in the dataset \n", 341 | "\n", 342 | "df = pd.read_csv('fraud_prediction.csv')" 343 | ] 344 | }, 345 | { 346 | "cell_type": "code", 347 | "execution_count": 19, 348 | "metadata": {}, 349 | "outputs": [], 350 | "source": [ 351 | "#Creating the features \n", 352 | "\n", 353 | "features = df.drop('isFraud', axis = 1).values\n", 354 | "target = df['isFraud'].values" 355 | ] 356 | }, 357 | { 358 | "cell_type": "code", 359 | "execution_count": 20, 360 | "metadata": {}, 361 | "outputs": [], 362 | "source": [ 363 | "X_train, X_test, y_train, y_test = train_test_split(features, target, test_size = 0.3, random_state = 42, stratify = target)" 364 | ] 365 | }, 366 | { 367 | "cell_type": "code", 368 | "execution_count": 21, 369 | "metadata": {}, 370 | "outputs": [], 371 | "source": [ 372 | "#Initialize a ridge regression model\n", 373 | "\n", 374 | "ridge_reg = Ridge(alpha = 0, normalize = True)" 375 | ] 376 | }, 377 | { 378 | "cell_type": "code", 379 | "execution_count": 22, 380 | "metadata": {}, 381 | "outputs": [ 382 | { 383 | "data": { 384 | "text/plain": [ 385 | "Ridge(alpha=0, copy_X=True, fit_intercept=True, max_iter=None, normalize=True,\n", 386 | " random_state=None, solver='auto', tol=0.001)" 387 | ] 388 | }, 389 | "execution_count": 22, 390 | "metadata": {}, 391 | "output_type": "execute_result" 392 | } 393 | ], 394 | "source": [ 395 | "#Fit the model to the training data \n", 396 | "\n", 397 | "ridge_reg.fit(X_train, y_train)" 398 | ] 399 | }, 400 | { 401 | "cell_type": "code", 402 | "execution_count": 23, 403 | "metadata": {}, 404 | "outputs": [ 405 | { 406 | "data": { 407 | "text/plain": [ 408 | "0.98579848176994034" 409 | ] 410 | }, 411 | "execution_count": 23, 412 | "metadata": {}, 413 | "output_type": "execute_result" 414 | } 415 | ], 416 | "source": [ 417 | "#Extract the score from the test data\n", 418 | "\n", 419 | "ridge_reg.score(X_test, y_test)" 420 | ] 421 | }, 422 | { 423 | "cell_type": "markdown", 424 | "metadata": {}, 425 | "source": [ 426 | "**Optimizing alpha using GridSearchCV**" 427 | ] 428 | }, 429 | { 430 | "cell_type": "code", 431 | "execution_count": 26, 432 | "metadata": {}, 433 | "outputs": [ 434 | { 435 | "name": "stdout", 436 | "output_type": "stream", 437 | "text": [ 438 | "The most optimal value of alpha is: {'alpha': 0.01}\n" 439 | ] 440 | } 441 | ], 442 | "source": [ 443 | "#Building the model \n", 444 | "\n", 445 | "ridge_regression = Ridge()\n", 446 | "\n", 447 | "#Using GridSearchCV to search for the best parameter\n", 448 | "\n", 449 | "grid = GridSearchCV(ridge_regression, {'alpha':[0.0001, 0.001, 0.01, 0.1, 10]})\n", 450 | "grid.fit(X_train, y_train)\n", 451 | "\n", 452 | "# Print out the best parameter\n", 453 | "\n", 454 | "print(\"The most optimal value of alpha is:\", grid.best_params_)" 455 | ] 456 | }, 457 | { 458 | "cell_type": "code", 459 | "execution_count": 29, 460 | "metadata": {}, 461 | "outputs": [ 462 | { 463 | "data": { 464 | "text/plain": [ 465 | "Ridge(alpha=0.01, copy_X=True, fit_intercept=True, max_iter=None,\n", 466 | " normalize=False, random_state=None, solver='auto', tol=0.001)" 467 | ] 468 | }, 469 | "execution_count": 29, 470 | "metadata": {}, 471 | "output_type": "execute_result" 472 | } 473 | ], 474 | "source": [ 475 | "#Initializing an ridge regression object\n", 476 | "\n", 477 | "ridge_regression = Ridge(alpha = 0.01)\n", 478 | "\n", 479 | "#Fitting the model to the training and test sets\n", 480 | "\n", 481 | "ridge_regression.fit(X_train, y_train)" 482 | ] 483 | }, 484 | { 485 | "cell_type": "code", 486 | "execution_count": 30, 487 | "metadata": {}, 488 | "outputs": [ 489 | { 490 | "data": { 491 | "text/plain": [ 492 | "0.98579844599883282" 493 | ] 494 | }, 495 | "execution_count": 30, 496 | "metadata": {}, 497 | "output_type": "execute_result" 498 | } 499 | ], 500 | "source": [ 501 | "#Accuracy score of the ridge regression model\n", 502 | "\n", 503 | "ridge_regression.score(X_test, y_test)" 504 | ] 505 | }, 506 | { 507 | "cell_type": "code", 508 | "execution_count": 31, 509 | "metadata": {}, 510 | "outputs": [ 511 | { 512 | "data": { 513 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEOCAYAAAC0BAELAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XeYFUX2//H3Z4aBARlyWhgEFAOI\niDAgSUExgIFoQkARFBO77s/wVdewrq6r66q7umJCEVERERQwIioYQJRByVkMBCUpSBAlnN8f3bjj\nCMwFbk9POK/nuQ8dqrpPOciZ6upbJTPDOeecS7aUuANwzjlXNHmCcc45FwlPMM455yLhCcY551wk\nPME455yLhCcY55xzkfAE45xzLhKeYJxzzkXCE4xzzrlIeIJxzjkXiRJxBxCnKlWqWN26deMOwznn\nCpXp06evNbOqeZUr1gmmbt26ZGdnxx2Gc84VKpK+TqScPyJzzjkXCU8wzjnnIuEJxjnnXCSK9RiM\nc87tj23btrF8+XK2bt0adyiRSk9PJzMzk7S0tP2qH2mCkdQReBBIBZ40s3tyna8DDAGqAt8Dvc1s\neXjuXuAMgl7WBOBqMzNJJYGHgfbATuBmMxud45o9gFFAczPzEXznXNItX76cjIwM6tati6S4w4mE\nmbFu3TqWL19OvXr19usakT0ik5QKDAI6AQ2BnpIa5ip2HzDMzBoDdwB3h3VbA22AxkAjoDnQLqxz\nM7DazA4Pr/t+jntmAFcDn0TULOecY+vWrVSuXLnIJhcASVSuXPmAemlR9mBaAEvMbCmApBFAF2Be\njjINgWvC7YnAmHDbgHSgJCAgDVgVnusHHAlgZjuBtTmudyfwT+D6JLflN75au5nFqzdFeQvniOuf\nriYHV6BK2VIx3b3wKMrJZZcDbWOUCaYWsCzH/nLguFxlZgLdCR6jdQMyJFU2s48lTQS+Jfj/7GEz\nmy+pQljvTkntgS+AgWa2SlJToLaZvS5pjwlG0gBgAMDBBx+8Xw0bP/c77n5zwX7Vda6gq3xQSR7v\n04ysupXiDsXtwfr16xk+fDhXXnnlPtU7/fTTGT58OBUqVMi7cBLEPch/HfCwpL7AB8AKYIek+kAD\nIDMsN0HS8cD88NgUM7tG0jXAfZIuAh4A+uZ1QzN7AngCICsry/Yn6O5NM2lTv8r+VHUuIbZffzMP\n3Mat27h5zBx6Dp7KP7odzTlZteMJxO3V+vXreeSRR36XYLZv306JEnv+Z/2NN96IOrTfiDLBrABy\n/u3MDI/9ysxWEvRgkFQW6GFm6yVdCkw1s03huTeBVsBHwBbg5fASLwH9gQyCsZpJYZeuBjBOUuco\nBvqrZpSiaoY/QnBF05gr23DV8M+4ftQsFq3ayI2dGpCaUvQfBxUmN954I1988QVNmjQhLS2N9PR0\nKlasyIIFC1i0aBFdu3Zl2bJlbN26lauvvpoBAwYA/5u9ZNOmTXTq1Im2bdsyZcoUatWqxdixYyld\nunRS44wywUwDDpNUjyCxnA9ckLOApCrA9+FYyk0Eb5QBfANcKulugkdk7YD/hG+RvUrwBtl7QAdg\nnpltAKrkuO4k4Dp/i8y5fVe+TBpDL27O31+fz+APv2Tx6k081PNYyqXv36uqRd3fXp3LvJU/JvWa\nDWuW469nHbXH8/fccw9z5sxhxowZTJo0iTPOOIM5c+b8+rbXkCFDqFSpEj/99BPNmzenR48eVK5c\n+TfXWLx4MS+88AKDBw/m3HPPZfTo0fTu3Tup7YjsLTIz2w4MBMYTPNoaaWZzJd0hqXNYrD2wUNIi\noDpwV3h8FMH4ymyCcZqZZvZqeO4G4HZJs4A+wLVRtcG54qpEagq3dz6Ku7o14qPFa+n+yBS+Wrs5\n7rDcHrRo0eI3rxI/9NBDHHPMMbRs2ZJly5axePHi39WpV68eTZo0AaBZs2Z89dVXSY8r0jEYM3sD\neCPXsdtybI8iSCa56+0ALtvDNb8GTsjjvu33I1znXC69jqvDIVXKcsXz0+n6yGQe6dWU1of6+GNO\ne+tp5JeDDjro1+1Jkybxzjvv8PHHH1OmTBnat2+/21eNS5X632P+1NRUfvrpp6TH5VPFOOf2qtWh\nlRl3VVuqli3FhU99ynNTE5pI10UoIyODjRs37vbchg0bqFixImXKlGHBggVMnTo1n6P7n7jfInPO\nFQIHVy7Dy1e25uoRM7hlzBwWrdrIrWc2JC3Vf0eNQ+XKlWnTpg2NGjWidOnSVK9e/ddzHTt25LHH\nHqNBgwYcccQRtGzZMrY4ZXG9D1kAZGVlma8H41ziduw07n1rAY9/sJQ29Ssz6IKmVChTMu6w8t38\n+fNp0KBB3GHki921VdJ0M8vKq67/+uGcS1hqirjp9Abcd84xTPvyB7oOmsyS1bt/VOOcJxjn3D47\nu1kmLww4jk0/b6fboClMWrg67pBcAeQJxjm3X5rVqcTYgW2pXakM/YZO46mPvqQ4P3J3v+cJxjm3\n32pVKM2oK1pxasMa3PnaPG4cPZtftu+MOyxXQHiCcc4dkDIlS/BIr6b86aT6vJi9jF5PTmXtpp/j\nDssVAJ5gnHMHLCVFXHPqEfy357HMWr6BLg9PZv63yZ0+xRU+nmCcc0lz1jE1eenyVmzfuZMej07h\n7bnfxR1SkbRrNuX98Z///IctW7YkOaLd8wTjnEuqxpkVGDewLYdVK8tlz01n0MQlPvifZIUlwfg3\n+Z1zSVe9XDovXtaKG0bP4l/jF7J41Ubu6dGY9LTUuEMrEnJO13/KKadQrVo1Ro4cyc8//0y3bt34\n29/+xubNmzn33HNZvnw5O3bs4NZbb2XVqlWsXLmSE088kSpVqjBx4sRI4/QE45yLRHpaKv85rwmH\nV8/gX+MX8uW6LQzu04xq5dLjDi253rwRvpud3GvWOBo63bPH0zmn63/77bcZNWoUn376KWZG586d\n+eCDD1izZg01a9bk9ddfB4I5ysqXL88DDzzAxIkTqVIl+klL/RGZcy4ykrjqxPo83qcZi1dtpPPD\nk5m9fEPcYRUpb7/9Nm+//TbHHnssTZs2ZcGCBSxevJijjz6aCRMmcMMNN/Dhhx9Svnz5fI/NezDO\nuciddlQNRl3emkuHZXPO41O475xjOLNxzbjDSo699DTyg5lx0003cdllv1/h5LPPPuONN97glltu\noUOHDtx22227uUJ0vAfjnMsXDWuWY+zANhxdqzwDh3/OAxMWsXOnD/7vj5zT9Z922mkMGTKETZs2\nAbBixQpWr17NypUrKVOmDL179+b666/ns88++13dqHkPxjmXb6qULcVzlxzHLa/M4aF3F7N41Ubu\nP/cYypT0f4r2Rc7p+jt16sQFF1xAq1atAChbtizPPfccS5Ys4frrryclJYW0tDQeffRRAAYMGEDH\njh2pWbNm5IP8Pl2/T9fvXL4zM5766Ev+8cZ8jqxRjsEXZVGrQum4w0qYT9fv0/U75wooSVxy/CE8\n1bc5y77fQpeHJzP96x/iDsslmScY51xsTjyiGq9c1ZqDSqXS84mpjJ6+PO6QXBJ5gnHOxap+tQzG\nXNmGrLoVufalmdz95nx2+OB/kRBpgpHUUdJCSUsk3bib83UkvStplqRJkjJznLtX0lxJ8yU9JEnh\n8ZKSnpC0SNICST3C49dImhde611JdaJsm3MueSoeVJJn+rWgT8s6PP7+UgYMy2bj1m1xh7VXxWH8\n+kDbGFmCkZQKDAI6AQ2BnpIa5ip2HzDMzBoDdwB3h3VbA22AxkAjoDnQLqxzM7DazA4Pr/t+ePxz\nICu81ijg3oia5pyLQFpqCnd2bcSdXY5i0qI19Hh0Ct+sy585s/ZVeno669atK9JJxsxYt24d6en7\nP/NClO8GtgCWmNlSAEkjgC7AvBxlGgLXhNsTgTHhtgHpQElAQBqwKjzXDzgSwMx2AmvD7Zzv200F\neie3Oc65/NCnVV0OrVqWK57/jC6DPuLR3s1oeUjluMP6jczMTJYvX86aNWviDiVS6enpZGZm5l1w\nD6JMMLWAZTn2lwPH5SozE+gOPAh0AzIkVTazjyVNBL4lSDAPm9l8SRXCendKag98AQw0s1W5rtsf\neDOprXHO5ZvW9asw9qo29H9mGr2f/IQ7ujTiguMOjjusX6WlpVGvXr24wyjw4h7kvw5oJ+lzgkdg\nK4AdkuoDDYBMgkR1kqTjCRJiJjDFzJoCHxM8ZvuVpN5AFvCv3d1Q0gBJ2ZKyi/pvH84VZnWrHMQr\nV7WhTf0q/OWV2dw+bi7bd/hyzIVJlAlmBVA7x35meOxXZrbSzLqb2bEEYyuY2XqC3sxUM9tkZpsI\neiOtgHXAFuDl8BIvAU13XU/SyeF1OpvZbtdsNbMnzCzLzLKqVq2ahGY656JSLj2NIX2bc+nx9Rg6\n5SsuHjqNDVsK9uC/+58oE8w04DBJ9SSVBM4HxuUsIKmKpF0x3AQMCbe/IejZlJCURtC7mW/BiNqr\nQPuwXAfCMR1JxwKPEySX1dE1yzmXn1JTxM1nNOTesxszdek6uj0ymaVrNsUdlktAZAnGzLYDA4Hx\nwHxgpJnNlXSHpM5hsfbAQkmLgOrAXeHxUQTjK7MJxmlmmtmr4bkbgNslzQL6ANeGx/8FlAVekjRD\n0m+SmXOucDs3qzbDL23Jhp+20XXQZD5Y5I+4Czqfi8znInOuUFn2/RYuHZbNolUbufXMhvRtXZfw\na3Iun/hcZM65Iql2pTKMvqI1HRpU52+vzuMvr8zml+0++F8QeYJxzhU6B5UqweO9mzHwxPq88Oky\nej/1Cd9v/iXusFwunmCcc4VSSoq47rQjePD8JsxYtp4ugz5i4Xf5s5CWS4wnGOdcodalSS1GXtaK\nn7ftpPsjk3lnXu7vXbu4eIJxzhV6TWpXYNzAthxStSyXPpvNY+9/UaTnCSssPME454qEGuXTGXlZ\nK844+g/c8+YCrh05k63bdsQdVrHmC2E754qM0iVT+W/PYzmiegb3T1jEl+s283ifZlTL2P8Zgd3+\n8x6Mc65IkcQfOxzGY72bsuDbjXR5eDJzVmyIO6xiyROMc65I6tjoD4y6ohUCznnsY96Y/W3cIRU7\nnmCcc0XWUTXLM3ZgWxr8IYMrn/+MB99Z7IP/+cgTjHOuSKuaUYoXBrSke9Na/PudRQx84XN++sUH\n//ODD/I754q8UiVSuf+cYziyRgZ3v7mAr9dtZvCFWfyhfOm4QyvSvAfjnCsWJDHghEN56qIsvlq7\nhc4PT+bzb36IO6wizROMc65YOenI6rx8ZWtKp6Vy3hNTGfP5irwruf3iCcY5V+wcXj2DMVe14dja\nFfjzizP451sL2LnTB/+TzROMc65YqnRQSZ7tfxw9WxzMo5O+YMCz09n08/a4wypSPME454qtkiVS\n+Ee3Rvyt81FMXLiasx+dwrLvt8QdVpHhCcY5V6xJ4qLWdXnm4hasXP8TXQZN5tMvv487rCLBE4xz\nzgFtD6vCmKvaUKFMGr2enMqL076JO6RCzxOMc86FDqlalleubEPLQypzw+jZ3PHqPLbv8OWY95cn\nGOecy6F86TSe7tuci9vUZcjkL+n3TDYbftoWd1iFkicY55zLpURqCn896yju6X40H3+xlm6PTObL\ntZvjDqvQiTTBSOooaaGkJZJu3M35OpLelTRL0iRJmTnO3StprqT5kh6SpPB4SUlPSFokaYGkHuHx\nUpJeDO/1iaS6UbbNOVf0nd/iYJ7rfxzrt2yj66DJfLR4bdwhFSqRJRhJqcAgoBPQEOgpqWGuYvcB\nw8ysMXAHcHdYtzXQBmgMNAKaA+3COjcDq83s8PC674fH+wM/mFl94N/APyNqmnOuGDnukMqMvaoN\nNcqlc9HTnzLs46/iDqnQiLIH0wJYYmZLzewXYATQJVeZhsB74fbEHOcNSAdKAqWANGBVeK4fYSIy\ns51mtutXii7AM+H2KKDDrl6Pc84diNqVyjD6ytaceERVbhs7l5tfmc02H/zPU5QJphawLMf+8vBY\nTjOB7uF2NyBDUmUz+5gg4Xwbfsab2XxJFcKyd0r6TNJLkqrnvp+ZbQc2AJWT3SjnXPFUtlQJHu+T\nxRXtD+X5T76hz1Of8MPmX+IOq0BLKMFIaivp4nC7qqR6Sbr/dUA7SZ8TPAJbAeyQVB9oAGQSJI6T\nJB1PsLxAJjDFzJoCHxM8ZkuYpAGSsiVlr1mzJknNcM4VB6kp4oaOR/LAucfw2dfr6TJoMotXbYw7\nrAIrzwQj6a/ADcBN4aE04LkErr0CqJ1jPzM89iszW2lm3c3sWIKxFcxsPUFvZqqZbTKzTcCbQCtg\nHbAFeDm8xEtA09z3k1QCKB+W/w0ze8LMsswsq2rVqgk0wznnfqt700xGXNaSLb/soNsjU5i4YHXc\nIRVIifRgugGdgc0QJAUgI4F604DDJNWTVBI4HxiXs4CkKpJ2xXATMCTc/oagZ1NCUhpB72a+BWud\nvgq0D8t1AOaF2+OAi8Lts4H3zNdGdc5FpOnBFRk3sA11Kpeh3zPTGPzBUl+OOZdEEswv4T/UBiDp\noEQuHI6DDATGA/OBkWY2V9IdkjqHxdoDCyUtAqoDd4XHRwFfALMJxmlmmtmr4bkbgNslzQL6ANeG\nx58CKktaAlwD/O61aOecS6aaFUrz0uWt6NSoBne9MZ/rR83i5+2+HPMuyivjSroOOAw4heDtrX7A\ncDP7b/ThRSsrK8uys7PjDsM5V8jt3Gk89N5i/vPOYprVqchjvZtRNaNU3GFFRtJ0M8vKq1yePRgz\nu4+gRzEaOAK4rSgkF+ecS5aUFPHnkw9n0AVNmbtyA10HTWbeyh/jDit2e00wklIlTTSzCWZ2vZld\nZ2YT8is455wrTM5o/AdGXd6anWb0eHQKb835Lu6QYrXXBGNmO4CdksrnUzzOOVeoNapVnrFXteGI\nGhlc/tx0Hn5vcbEd/C+RQJlNwGxJEwjfJAMwsz9FFpVzzhVi1cqlM2JAS256eTb3vb2Ihas28a+z\nG5Oelhp3aPkqkQTzMv/73olzzrkEpKel8sC5x3B49QzuHb+Ar9dt5ok+WdQonx53aPkmz7fIIJjB\nGDg83F1oZkVicQR/i8w5lx8mzFvFn0d8zkGlSjD4wiyOqV0h70oFWNLeIpPUHlhMMDPyI8AiSScc\ncITOOVdMnNKwOqOvbE3JEimc+/jHjJ2xIu9KRUAiX7S8HzjVzNqZ2QnAaQTT4TvnnEvQkTXKMfaq\nNhxTuwJXj5jB/W8vZOfOoj34n0iCSTOzhbt2zGwRwXxkzjnn9kHlsqV4rv9xnN+8Nv99bwlXPD+d\nzT9vjzusyCSSYLIlPSmpffgZDPjAhXPO7YeSJVK4u/vR3HZmQybMW0WPR6ew/IctcYcViUQSzBUE\nE0r+KfzMC48555zbD5Lo17YeT1/cghXrf6LLw5PJ/ur7uMNKukQSTAngwXBa/e7AQ0DxepnbOeci\n0O7wqoy5qg3lSqfRc/BUXspelnelQiSRBPMuUDrHfmngnWjCcc654uXQqmUZc2UbjqtXmetHzeKu\n1+exo4gM/ieSYNLDRb8ACLfLRBeSc84VL+XLpDH04ub0bV2XwR9+ySXPTOPHrYX/64aJJJjNknat\nGomkZsBP0YXknHPFT4nUFG7vfBR3dWvEh4vX0v2RKXy1dnPeFQuwRBLMn4GXJH0o6SPgRYKFxJxz\nziVZr+Pq8Gz/41i76We6PjKZKV+sjTuk/ZbIejDTgCMJ3hy7HGhgZtOjDsw554qrVodWZuxVbaha\nthQXPvUpz039Ou6Q9sseE4yk5pJqAIRzjzUlWNL4fkmV8ik+55wrlupUPoiXr2zNCYdX5ZYxc7ht\n7By27dgZd1j7ZG89mMeBXwDCucfuAYYBG4Anog/NOeeKt4z0NAZfmMVlJxzCsI+/pu/Tn7J+yy9x\nh5WwvSWYVDPb9c2f84AnzGy0md0K1I8+NOecc6kp4qbTG3DfOccw7csf6DpoMktWb8q7YgGw1wQj\nadd6MR2A93KcS2QdGeecc0lydrNMXhhwHJt+3k63QZOZtHB13CHlaW8J5gXgfUljCV5L/hBAUn2C\nx2TOOefyUbM6lRg7sC2ZlcrQb+g0nvroywK9HPMeE4yZ3QVcCwwF2tr/WpEC/DGRi0vqKGmhpCWS\nbtzN+TqS3pU0S9IkSZk5zt0raa6k+ZIekqTw+KTwmjPCT7Xw+MGSJkr6PLze6Yn+R3DOucKiVoXS\njLq8Fac2rMGdr83jxtGz+WV7wRz83+trymY21cxeMbPNOY4tMrPP8rqwpFSCRco6AQ2BnpIa5ip2\nHzDMzBoDdwB3h3VbA22AxkAjoDnQLke9XmbWJPzs6ifeAow0s2OB8wkWR3POuSLnoFIleKRXU/50\nUn1ezF5G7yc/Yd2mn+MO63cS+aLl/moBLDGzpWb2CzAC6JKrTEP+N7YzMcd5A9KBkkApgvVnVuVx\nPwPKhdvlgZUHFL1zzhVgKSnimlOP4L89j2Xm8vV0fngyC777Me6wfiPKwfpaQM6pQZcDx+UqMxPo\nDjwIdAMyJFU2s48lTQS+BQQ8bGbzc9R7WtIOYDTw9/Dx3e3A25L+CBwEnBxBmwKfDob3/xnZ5V1B\no1y7ivl8xPEdeTp0uB1S/V2ewuCsY2pSp3IZLh2WTY9HpvDv85pw6lE14g4LSCDBhP9gP2dmP0Rw\n/+uAhyX1BT4AVgA7whcJGgC7xmQmSDrezD4keDy2QlIGQYLpQ/D9nJ7AUDO7X1Ir4FlJjczsNw8n\nJQ0ABgAcfPDB+xd1pUOgwVn7V9cVLr8bQLUkn//dDfP5/rn2t26AKf+FNYvg7CFQqmzuAF0B1Diz\nAuMGtmXAsGwue2461516BFe2PxT97peJ/KW83kCQ9HeCMY3PgCHAeEvgtYXwH/nbzey0cP8mADO7\new/lywILzCxT0vUEszjfGZ67DdhqZvfmqtMXyDKzgZLmAh3NbFl4binQMscYze9kZWVZdrYvzunc\nb0x7Ct64HqofBReMhHJ/iDsil6Ct23bwf6NmMW7mSro2qck9PRqTnpb85bskTTezrLzKJTIX2S3A\nYcBTQF9gsaR/SDo0j6rTgMMk1ZNUkiBJjcsVZBVJu2K4iSCBAXwDtJNUQlIawQD//HC/Slg3DTgT\nmJOjTofwXAOCMZw1ebXPOZdL8/5wwYvw/VJ4sgN8NyfvOq5ASE9L5cHzm3D9aUcwZsZKzntiKqt/\n3BpbPAkN8oc9lu/Cz3agIjBK0r17qbOdYNbl8cB8gje85kq6Q1LnsFh7YKGkRUB1grnOAEYBXwCz\nCcZpZprZqwQD/uMlzQJmEDxSGxzWuRa4VNJMgu/w9E2kp+Wc243DToF+bwWP0IZ0hCW+xmBhIYmr\nTqzP432asXjVRjo/PJnZy+P56mIij8iuBi4E1gJPAmPMbFvY81hsZnn1ZAosf0TmXB42rIDh58Hq\neXDG/ZB1cdwRuX0wb+WPXDosm3Wbf+a+c47hzMY1k3LdpD0iAyoB3c3sNDN7KZxZmXDw/MwDjNM5\nV5CVrwX93oRDT4LX/gwT/go7C+aX+tzvNaxZjrED29CoZnkGDv+cByYsYmc+LsecSIJ5E9g16SWS\nykk6DiDXq8POuaKoVAb0HAFZ/WDyf2DUxbDNF7UtLKqULcXzlx7HOc0yeejdxVw1/DO2/LI9X+6d\nSIJ5FMg5deem8JhzrrhILQFnPACn3AnzxsAznWFz4V1psbgpVSKVe89uzC1nNGD83O84+9GPWbk+\n+l8SEkkwyjlYHj4a829gOVfcSNDmT3DOM/DdLHjyZFi7JO6oXIIkccnxh/BU3+Ys+34LE/NhNuZE\nEsxSSX+SlBZ+rgaWRh2Yc66AOqorXPQa/LwRnjoZvp4Sd0RuH5x4RDXeubYdF7TYzy+a74NEEszl\nQGuCV4J3TfcyIMqgnHMFXO3mcMk7UKYKDOsCs16KOyK3D6qXS8+Xb/nn+agr/Cb8+ZFH4pwrXCrV\ng/5vw4t94OVL4Iev4ITrdjPXmSuuEpmLLB3oDxxF8O14AMysX4RxOecKgzKVoM/LMO6PMPHvQZI5\n6z+QmhZ3ZK4ASOQR2bNADeA04H2CCSg3RhmUc64QKVEKuj0O7W6AGc/Bcz3gp/VxR+UKgEQSTH0z\nuxXYbGbPAGfw+2n3nXPFmQQn/gW6PhoM+g85DdZ/E3dULmaJJJht4Z/rJTUiWMyrWnQhOecKrSYX\nBI/MfvwWBneAFXkufuuKsEQSzBOSKhIsSTwOmAf4alvOud2rdwJcMgHS0mHoGbDg9bgjcjHZa4IJ\nJ7T80cx+MLMPzOwQM6tmZo/nU3zOucKo6hFwybtQ9UgY0Qum+uQfxdFeE0z4rf3/y6dYnHNFSdlq\n0Pd1OPIMeOtGeOP/YOeOuKNy+SiRR2TvSLpOUm1JlXZ9Io/MOVf4lSwD5w6DVgPh08eD3swvm+OO\nyuWTROYUOy/886ocxww4JPnhOOeKnJRUOO0uqFgX3vw/ePr0YMXMjBpxR+Yilsg3+evlRyDOuSKu\nxaVQvjaM6hdMlHnBSKjeMO6oXIQS+Sb/hbs7bmbDkh+Oc65IO6IjXPxGsErmkNOCx2eHnhh3VC4i\niYzBNM/xOR64HegcYUzOuaKsZhO49N2gN/P82fDZs3FH5CKSyCOyP+bcl1QBGBFZRM65oq98JvR7\nC17qC+MGwg9fwom3QEoiv/O6wmJ/fpqbAR+Xcc4dmPRywWB/04vgw/uDGZm3bY07KpdEeSYYSa9K\nGhd+XgMWAq8kcnFJHSUtlLRE0o27OV9H0ruSZkmaJCkzx7l7Jc2VNF/SQwoXLwjLLZQ0I/xUy1Hn\nXEnzwnrDE4nRORej1DQ460E4+XaYMxqe7Qpbvo87KpckibymfF+O7e3A12a2PK9KklKBQcApBAuV\nTZM0zszm5br2MDN7RtJJwN1AH0mtgTZA47DcR0A7YFK438vMsnPd7zDgJqCNmf2QM/E45wowCdr+\nP6hwMLxyRfCGWa+XoPKhcUfmDlAij8i+AT4xs/fNbDKwTlLdBOq1AJaY2VIz+4Vg3KZLrjINgffC\n7Yk5zhvB2jMlgVJAGrAqj/tdCgwysx/g14XSnHOFRaMecNE4+OmHIMl8MzXuiNwBSiTBvATszLG/\nIzyWl1rAshz7y8NjOc0EuodecIqHAAAS8ElEQVTb3YAMSZXN7GOChPNt+BlvZvNz1Hs6fDx2665H\nZ8DhwOGSJkuaKqljAjE65wqSg1sGSzGXrgjPdA4em7lCK5EEUyLsgQAQbpdM0v2vA9pJ+pzgEdgK\nYIek+kADgsXNagEnSTo+rNPLzI4meGX6eKDPrjiBw4D2QE9gcPjG229IGiApW1L2mjVrktQM51zS\nVD40SDK1mgZfyvzwATCLOyq3HxJJMGsk/fq9F0ldgLUJ1FsB1M6xnxke+5WZrTSz7mZ2LHBzeGw9\nQW9mqpltMrNNwJtAq/D8ivDPjcBwgkdxEPSQxpnZNjP7ElhEkHB+w8yeMLMsM8uqWrVqAs1wzuW7\nMpWgzxhodDa8+zd49U+wY1ve9VyBkkiCuRz4i6RvJH0D3ABclkC9acBhkupJKgmcT7CezK8kVQmX\nBIBggH5IuP0NQc+mhKQ0gt7N/HC/Slg3DTgTmBPWGUPQeyEscziwNIE4nXMFUVo6dB8Mx18Hnw2D\n4efC1g1xR+X2QZ4Jxsy+MLOWBAPyDc2stZktSaDedmAgMB6YD4w0s7mS7sjRI2oPLJS0CKgO3BUe\nHwV8AcwmGKeZaWavEgz4j5c0C5hB0CMaHNYZT/ACwjyC8ZvrzWxdnv8FnHMFV0oKdLgVOj8MX34A\nQzrC+mV513MFgiyPZ5uS/gHcGz66Ilzd8lozuyUf4otUVlaWZWdn513QORe/LybCyAshrUzwBc2a\nTeKOqNiSNN3MsvIql8gjsk67kgtA+Brw6QcSnHPO7bNDT4R+44MvZz59Oix8K+6IXB4SSTCpkkrt\n2pFUmuBRlXPO5a/qDYM3zKocBiN6widPxB2R24tEEszzwLuS+kvqD0wAfKp+51w8MmoEU/4f3hHe\nvB7eusmXYi6gEplN+Z+SZgInh4fuNLPx0YblnHN7UfIgOO85GH8zTH0E1n8TvHFWskzckbkcEppN\n2czeMrPrzOw6YLOkQRHH5Zxze5eSCp3ugY7/hAWvw9AzYJPPEFWQJJRgJB0bzm78FXAnsCDSqJxz\nLlEtL4fzh8OaBfBkB1jt/zwVFHtMMJIOl/RXSQuA/xLMKyYzO9HM/ptvETrnXF6OPB36vh6sJ/PU\nqbD0/bgjcuy9B7MAOAk408zahknFR9KccwVTrabBUszlasJz3WGGLwkVt70lmO4EMxlPlDRYUgdA\neynvnHPxqnAw9B8PddvCmCvgvbt8oswY7THBmNkYMzsfOJJg6pU/A9UkPSrp1PwK0Dnn9kl6eeg1\nCo7tDR/cC69cBtt/jjuqYimRucg2m9lwMzuLYEbkzwkmvHTOuYIpNS2Yv+ykW2DWi/BsN1+KOQYJ\nvUW2i5n9EE533yGqgJxzLikkOOF66PEULJ8WDP5/7xOs56d9SjDOOVfoHH02XDgWtqwNlmJe9mnc\nERUbnmCcc0VfndbQ/x0oVQ6eOQvmjok7omLBE4xzrnioUj+YKPMPx8BLF8HkB/0Ns4h5gnHOFR8H\nVYELx8FR3WDCbfDa/4Md2+OOqsjKc7JL55wrUtLSoccQqFgXPvo3bFgG5wyFUhlxR1bkeA/GOVf8\npKTAybfDWQ8GK2UO6QQbVsQdVZHjCcY5V3w16wu9RsIPXwUTZX47K+6IihRPMM654q3+ydDvLVAK\nPN0JFr0dd0RFhicY55yr0QgueRcqHQIvnAfTnow7oiLBE4xzzgGU+wNc/CbUPwVevxbevgV27ow7\nqkIt0gQjqaOkhZKWSLpxN+frSHpX0ixJkyRl5jh3r6S5kuZLekiSwuOTwmvOCD/Vcl2zhySTlBVl\n25xzRVCpssHiZc0vhSn/Db4vs+2nuKMqtCJLMJJSgUFAJ6Ah0FNSw1zF7gOGmVlj4A7g7rBua6AN\n0BhoBDQH2uWo18vMmoSfX9dIlZQBXA18Ek2rnHNFXmoJOP1fcNo/YP6rMPRM2LQm7qgKpSh7MC2A\nJWa21Mx+AUYAXXKVaQi8F25PzHHegHSgJFAKSANWJXDPO4F/AlsPLHTnXLEmQaur4LxnYdXc4A2z\nNYvijqrQiTLB1CJYZnmX5eGxnGYSLGwG0A3IkFTZzD4mSDjfhp/xZjY/R72nw8djt+Z4dNYUqG1m\nr0fQFudccdTgrHAp5i3w1Mnw1UdxR1SoxD3Ifx3QTtLnBI/AVgA7JNUHGhCsP1MLOEnS8WGdXmZ2\nNHB8+OkjKQV4ALg2rxtKGiApW1L2mjXe7XXO5SGzWTCHWdkaMKwrzHwx7ogKjSgTzAqgdo79zPDY\nr8xspZl1N7NjgZvDY+sJejNTzWyTmW0C3gRahedXhH9uBIYTPIrLIBirmSTpK6AlMG53A/3hejZZ\nZpZVtWrVZLbXOVdUVawbLMV8cEt4ZQBM+qdPlJmAKBPMNOAwSfUklQTOB8blLCCpStj7ALgJGBJu\nf0PQsykhKY2gdzM/3K8S1k0DzgTmmNkGM6tiZnXNrC4wFehsZtkRts85V5yUrgi9X4ZjesKkf8CY\nK2H7L3FHVaBFlmDMbDswEBgPzAdGmtlcSXdI6hwWaw8slLQIqA7cFR4fBXwBzCYYp5lpZq8SDPiP\nlzQLmEHQIxocVRucc+43SpSEro9C+7/AzOHwXHf46Ye4oyqwZMW4m5eVlWXZ2d7Jcc7th5kvwtir\noFI96PVS8BitmJA03czy/K5h3IP8zjlXOB1zHlw4BjatDpZiXj497ogKHE8wzjm3v+q2hf4TIK0M\nDD0j+GKm+5UnGOecOxBVDw8myqx+FLzYB6Y87G+YhTzBOOfcgSpbFfq+Fnwx8+2b4Y3rfClmPME4\n51xypJWGc56B1n8KpvsfcQH8vCnuqGLlCcY555IlJQVOvRPOeACWTAgWMPvx27ijio0nGOecS7bm\n/eGCkfD90mCizO/mxB1RLDzBOOdcFA47JVjAzHbCkI6w5J24I8p3nmCccy4qf2gcvGFWsS48fy5k\nPx13RPnKE4xzzkWpfC3o9yYceiK89meY8NdisxSzJxjnnItaqQzo+SJk9YPJ/4HR/YrFUswl4g7A\nOeeKhdQSwdtlFevChNtgwwro+QIcVCXuyCLjPRjnnMsvErS5Ovi+zHezgjnM1i6JO6rIeIJxzrn8\ndlRXuOg1+HljsBTz11PijigSnmCccy4OtZsHSzGXqQLDusCsl+KOKOk8wTjnXFwq1YP+b0Nmc3j5\nEnj/X0VqokxPMM45F6cylaDPK3D0uTDx7zB2IOzYFndUSeFvkTnnXNxKlILuTwQ9mvf/CRuWwbnD\noHSFuCM7IN6Dcc65gkCCE/8CXR+FrycH08us/ybuqA6IJxjnnCtImlwAvV+GH1cGrzGv+CzuiPab\nJxjnnCtoDmkXDP6nlgqWYl7wetwR7RdPMM45VxBVOzJ4jbnqkTCiF0x9NO6I9lmkCUZSR0kLJS2R\ndONuzteR9K6kWZImScrMce5eSXMlzZf0kCSFxyeF15wRfqqFx6+RNC+81ruS6kTZNueci1xGdej7\nOhx5Brx1I7x5A+zcEXdUCYsswUhKBQYBnYCGQE9JDXMVuw8YZmaNgTuAu8O6rYE2QGOgEdAcaJej\nXi8zaxJ+VofHPgeywmuNAu6NpmXOOZePSpYJ3ihreRV88hi82Bt+2Rx3VAmJsgfTAlhiZkvN7Bdg\nBNAlV5mGwHvh9sQc5w1IB0oCpYA0YNXebmZmE81sS7g7FcjcW3nnnCs0UlKh4z/g9Ptg0Vvw9Omw\n8bu4o8pTlAmmFrAsx/7y8FhOM4Hu4XY3IENSZTP7mCDhfBt+xpvZ/Bz1ng4fj92669FZLv2BN5PR\nCOecKzBaXArnvwBrFwVvmK2aF3dEexX3IP91QDtJnxM8AlsB7JBUH2hA0AupBZwk6fiwTi8zOxo4\nPvz0yXlBSb2BLOBfu7uhpAGSsiVlr1mzJoo2OedcdI7oGCzFvGMbDDkNvpgYd0R7FGWCWQHUzrGf\nGR77lZmtNLPuZnYscHN4bD1Bb2aqmW0ys00EvZFW4fkV4Z8bgeEEj+IAkHRyeJ3OZvbz7oIysyfM\nLMvMsqpWrZqcljrnXH6q2QQufRfK14bnz4bPno07ot2KMsFMAw6TVE9SSeB8YFzOApKqSNoVw03A\nkHD7G4KeTQlJaQS9m/nhfpWwbhpwJjAn3D8WeJwguazGOeeKsvKZ0O8tqHcCjBsI795R4JZijizB\nmNl2YCAwHpgPjDSzuZLukNQ5LNYeWChpEVAduCs8Pgr4AphNME4z08xeJRjwHy9pFjCDoEc0OKzz\nL6As8FI4PvObZOacc0VOejm4YCQ0vRA+vD+YkXnb1rij+pWsCE0Nva+ysrIsOzs77jCcc+7AmMFH\n/4Z3/wYHt4LzhwezNEdE0nQzy8qrXNyD/M455w6UBMdfA2cPCeYue/JkWPdF3FF5gnHOuSKjUQ+4\naBz89EOQZL6ZGms4nmCcc64oObhlMIdZ6YrwTGeYMzq2UDzBOOdcUVP50CDJ1GoKo/rBhw/EshSz\nJxjnnCuKylSCPmOg0dnB4P+rf8r3pZh9yWTnnCuq0tKh+2CoWBc+vA82LIdzngleb84H3oNxzrmi\nLCUFOtwKnR+GLz8IlmLesDx/bp0vd3HOORevpn2g1yjYsAwGd4CVMyK/pScY55wrLg49EfqNh9SS\nsP7ryG/nYzDOOVecVG8IA6cF4zMR8x6Mc84VN/mQXMATjHPOuYh4gnHOORcJTzDOOeci4QnGOedc\nJDzBOOeci4QnGOecc5HwBOOccy4SxXrJZElrgK+B8sCGHKdy7u/pXBVgbZJCyX2P/S23p/O7O55o\nm3NuJ6vNibY3kbLe5j0f35f9wtjmff0Z594vyG1O1t/r3PvJanMdM6uaZykzK/Yf4Ik97e/pHJAd\n1f33t9yezu/ueKJtzrWdlDYn2l5v84G1eV/2C2Ob9/VnXJjanKy/1/nR5r19/BFZ4NW97O/tXFT3\n399yezq/u+OJtjnO9iZS1tu85+P7sl8Y27yvP+Pc+wW5zcn6e517P4o271GxfkR2ICRlm1lW3HHk\nJ29z8eBtLh7yo83eg9l/T8QdQAy8zcWDt7l4iLzN3oNxzjkXCe/BOOeci4QnGOecc5HwBOOccy4S\nnmAiIukgSdmSzow7lvwgqYGkxySNknRF3PHkB0ldJQ2W9KKkU+OOJz9IOkTSU5JGxR1LVML/d58J\nf7a94o4nP0T1c/UEk4ukIZJWS5qT63hHSQslLZF0YwKXugEYGU2UyZWMNpvZfDO7HDgXaBNlvMmQ\npDaPMbNLgcuB86KMNxmS1OalZtY/2kiTbx/b3h0YFf5sO+d7sEmyL22O6ufqCeb3hgIdcx6QlAoM\nAjoBDYGekhpKOlrSa7k+1SSdAswDVud38PtpKAfY5rBOZ+B14I38DX+/DCUJbQ7dEtYr6IaSvDYX\nNkNJsO1AJrAsLLYjH2NMtqEk3uZIlIjqwoWVmX0gqW6uwy2AJWa2FEDSCKCLmd0N/O4RmKT2wEEE\nP8CfJL1hZjujjPtAJKPN4XXGAeMkvQ4Mjy7iA5ekn7OAe4A3zeyzaCM+cMn6ORdG+9J2YDlBkplB\nIf4lfB/bPC+KGArtf7x8Vov//UYDwV/AWnsqbGY3m9mfCf6RHVyQk8te7FObJbWX9JCkxykcPZjd\n2ac2A38ETgbOlnR5lIFFaF9/zpUlPQYcK+mmqIOL2J7a/jLQQ9Kj5PPUKvlgt22O6ufqPZgImdnQ\nuGPIL2Y2CZgUcxj5ysweAh6KO478ZGbrCMaciiwz2wxcHHcc+Smqn6v3YBKzAqidYz8zPFaUeZu9\nzUVdcWx7vrbZE0xipgGHSaonqSRwPjAu5pii5m32Nhd1xbHt+dpmTzC5SHoB+Bg4QtJySf3NbDsw\nEBgPzAdGmtncOONMJm+zt5ki2uZdimPbC0KbfbJL55xzkfAejHPOuUh4gnHOORcJTzDOOeci4QnG\nOedcJDzBOOeci4QnGOecc5HwBONcPlCwdoxJOjLcr5t7GvXd1MmzjHMFmScY5/JHT+Cj8E/nigVP\nMM5FTFJZoC3Qn2Bqjtzn+0oaK2mSpMWS/prjdKqClRXnSnpbUumwzqWSpkmaKWm0pDL50xrnEucJ\nxrnodQHeMrNFwDpJzXZTpgXQA2gMnCMpKzx+GDDIzI4C1odlAF42s+ZmdgzBlB+FbpVJV/R5gnEu\nej2BEeH2CHb/mGyCma0zs58I1iNpGx7/0sxmhNvTgbrhdiNJH0qaDfQCjookcucOgK8H41yEJFUC\nTgKOlmRAKmD8fonl3JMC7tr/OcexHUDpcHso0NXMZkrqC7RPXtTOJYf3YJyL1tnAs2ZWx8zqmllt\n4Et+uyYHwCmSKoVjLF2ByXlcNwP4VlIaQQ/GuQLHE4xz0eoJvJLr2Ggg97K0n4bHZwGjzSw7j+ve\nCnxCkIgWJCFO55LOp+t3LmbhI64sMxsYdyzOJZP3YJxzzkXCezDOOeci4T0Y55xzkfAE45xzLhKe\nYJxzzkXCE4xzzrlIeIJxzjkXCU8wzjnnIvH/AZJCP7eu9g+eAAAAAElFTkSuQmCC\n", 514 | "text/plain": [ 515 | "" 516 | ] 517 | }, 518 | "metadata": {}, 519 | "output_type": "display_data" 520 | } 521 | ], 522 | "source": [ 523 | "train_errors = []\n", 524 | "test_errors = []\n", 525 | "\n", 526 | "alpha_list = [0.0001, 0.001, 0.01, 0.1, 10]\n", 527 | "\n", 528 | "# Evaluate the training and test classification errors for each value of alpha\n", 529 | "\n", 530 | "for value in alpha_list:\n", 531 | " \n", 532 | " # Create Ridge object and fit\n", 533 | " ridge_regression = Ridge(alpha= value)\n", 534 | " ridge_regression.fit(X_train, y_train)\n", 535 | " \n", 536 | " # Evaluate error rates and append to lists\n", 537 | " train_errors.append(ridge_regression.score(X_train, y_train) )\n", 538 | " test_errors.append(ridge_regression.score(X_test, y_test))\n", 539 | " \n", 540 | "# Plot results\n", 541 | "plt.semilogx(alpha_list, train_errors, alpha_list, test_errors)\n", 542 | "plt.legend((\"train\", \"test\"))\n", 543 | "plt.ylabel('Accuracy Score')\n", 544 | "plt.xlabel('Alpha')\n", 545 | "plt.show()" 546 | ] 547 | }, 548 | { 549 | "cell_type": "markdown", 550 | "metadata": {}, 551 | "source": [ 552 | "**Lasso Regression**" 553 | ] 554 | }, 555 | { 556 | "cell_type": "code", 557 | "execution_count": 33, 558 | "metadata": {}, 559 | "outputs": [], 560 | "source": [ 561 | "# Reading in the dataset \n", 562 | "\n", 563 | "df = pd.read_csv('fraud_prediction.csv')" 564 | ] 565 | }, 566 | { 567 | "cell_type": "code", 568 | "execution_count": 34, 569 | "metadata": {}, 570 | "outputs": [], 571 | "source": [ 572 | "#Creating the features \n", 573 | "\n", 574 | "features = df.drop('isFraud', axis = 1).values\n", 575 | "target = df['isFraud'].values" 576 | ] 577 | }, 578 | { 579 | "cell_type": "code", 580 | "execution_count": 35, 581 | "metadata": {}, 582 | "outputs": [], 583 | "source": [ 584 | "X_train, X_test, y_train, y_test = train_test_split(features, target, test_size = 0.3, random_state = 42, stratify = target)" 585 | ] 586 | }, 587 | { 588 | "cell_type": "code", 589 | "execution_count": 36, 590 | "metadata": {}, 591 | "outputs": [], 592 | "source": [ 593 | "#Initialize a lasso regression model\n", 594 | "\n", 595 | "lasso_reg = Lasso(alpha = 0, normalize = True)" 596 | ] 597 | }, 598 | { 599 | "cell_type": "code", 600 | "execution_count": 43, 601 | "metadata": {}, 602 | "outputs": [], 603 | "source": [ 604 | "#Fit the model to the training data \n", 605 | "\n", 606 | "lasso_reg.fit(X_train, y_train)\n", 607 | "\n", 608 | "warnings.filterwarnings('ignore')" 609 | ] 610 | }, 611 | { 612 | "cell_type": "code", 613 | "execution_count": 44, 614 | "metadata": {}, 615 | "outputs": [ 616 | { 617 | "data": { 618 | "text/plain": [ 619 | "0.98579848163563688" 620 | ] 621 | }, 622 | "execution_count": 44, 623 | "metadata": {}, 624 | "output_type": "execute_result" 625 | } 626 | ], 627 | "source": [ 628 | "#Extract the score from the test data\n", 629 | "\n", 630 | "lasso_reg.score(X_test, y_test)" 631 | ] 632 | }, 633 | { 634 | "cell_type": "markdown", 635 | "metadata": {}, 636 | "source": [ 637 | "**Optimizing alpha using GridSearchCV**" 638 | ] 639 | }, 640 | { 641 | "cell_type": "code", 642 | "execution_count": 45, 643 | "metadata": {}, 644 | "outputs": [ 645 | { 646 | "name": "stdout", 647 | "output_type": "stream", 648 | "text": [ 649 | "The most optimal value of alpha is: {'alpha': 0.0001}\n" 650 | ] 651 | } 652 | ], 653 | "source": [ 654 | "#Building the model \n", 655 | "\n", 656 | "lasso_regression = Lasso()\n", 657 | "\n", 658 | "#Using GridSearchCV to search for the best parameter\n", 659 | "\n", 660 | "grid = GridSearchCV(lasso_regression, {'alpha':[0.0001, 0.001, 0.01, 0.1, 10]})\n", 661 | "grid.fit(X_train, y_train)\n", 662 | "\n", 663 | "# Print out the best parameter\n", 664 | "\n", 665 | "print(\"The most optimal value of alpha is:\", grid.best_params_)" 666 | ] 667 | }, 668 | { 669 | "cell_type": "code", 670 | "execution_count": 46, 671 | "metadata": {}, 672 | "outputs": [ 673 | { 674 | "data": { 675 | "text/plain": [ 676 | "Lasso(alpha=0.0001, copy_X=True, fit_intercept=True, max_iter=1000,\n", 677 | " normalize=False, positive=False, precompute=False, random_state=None,\n", 678 | " selection='cyclic', tol=0.0001, warm_start=False)" 679 | ] 680 | }, 681 | "execution_count": 46, 682 | "metadata": {}, 683 | "output_type": "execute_result" 684 | } 685 | ], 686 | "source": [ 687 | "#Initializing an lasso regression object\n", 688 | "\n", 689 | "lasso_regression = Lasso(alpha = 0.0001)\n", 690 | "\n", 691 | "#Fitting the model to the training and test sets\n", 692 | "\n", 693 | "lasso_regression.fit(X_train, y_train)" 694 | ] 695 | }, 696 | { 697 | "cell_type": "code", 698 | "execution_count": 47, 699 | "metadata": {}, 700 | "outputs": [ 701 | { 702 | "data": { 703 | "text/plain": [ 704 | "0.98576284355162636" 705 | ] 706 | }, 707 | "execution_count": 47, 708 | "metadata": {}, 709 | "output_type": "execute_result" 710 | } 711 | ], 712 | "source": [ 713 | "#Accuracy score of the lasso regression model\n", 714 | "\n", 715 | "lasso_regression.score(X_test, y_test)" 716 | ] 717 | }, 718 | { 719 | "cell_type": "code", 720 | "execution_count": 48, 721 | "metadata": {}, 722 | "outputs": [ 723 | { 724 | "data": { 725 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEOCAYAAAC0BAELAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuUVtWd5vHvIxQWKvHCxbSUS+hR\nE4g4Eks6XmgIxgQTg4qJ0SA9yWQk9orG9IyOurxkYrejbTuZNCskBtJ46bQSgkmE8YI3qtUEO5QK\nCJRcNIlUYbRCghHjDfzNH2cXvhZ1eSneXWUVz2ets+rsffY5Z28KedznvO85igjMzMwqba+e7oCZ\nmfVNDhgzM8vCAWNmZlk4YMzMLAsHjJmZZeGAMTOzLBwwZmaWhQPGzMyycMCYmVkWDhgzM8uif093\noCcNGTIkRowY0dPdMDPrVZ588snfR8TQztrt0QEzYsQI6uvre7obZma9iqTfltPOl8jMzCwLB4yZ\nmWXhgDEzsywcMGZmloUDxszMsnDAmJlZFnv0x5S76luLVrNm0596uhtmZl02+pAP8M3PfiTrOTyD\nMTOzLDyD6YLcqW9m1hd4BmNmZlk4YMzMLAsHjJmZZeGAMTOzLBwwZmaWhQPGzMyycMCYmVkWDhgz\nM8sia8BImixpraQNki5vY/thkh6WtFJSnaSakm03SlotqUHSTElK9XXpmMvTMizV/9+SunWStuQc\nm5mZdSzbN/kl9QNmAacAjcAySQsjYk1Js5uA2yPiNkmTgOuB6ZJOAE4Ejk7tHgcmAHWpPC0i3vOu\n44j4u5JzXwSMrfyozMysXDlnMOOADRHxfES8BcwDTm/VZjTwSFpfUrI9gGpgALA3UAW8tAvnPhe4\ns4v9NjOzCsgZMMOBjSXlxlRXagUwNa2fCQySNDgillIEzotpWRwRDSX73ZIuhV3dcumshaTDgJG8\nG1xmZtYDevom/yXABElPU1wCawK2SzocGAXUUITSJEnj0z7TImIMMD4t01sd8xxgQURsb+uEkmZI\nqpdU39zcXPkRmZkZkDdgmoBDS8o1qW6HiNgUEVMjYixwZarbQjGbeSIitkbEVuA+4Pi0vSn9fBW4\ng+JSXKlz6ODyWETMjojaiKgdOnTo7ozPzMw6kDNglgFHSBopaQDFP/wLSxtIGiKppQ9XAHPT+gsU\nM5v+kqooZjcNqTwk7VsFnAasKjneh4EDgaUZx2VmZmXIFjARsQ24EFgMNADzI2K1pGslTUnNJgJr\nJa0DDgauS/ULgOeAZyju06yIiEUUN/wXS1oJLKeYEc0pOe05wLyIiFzjMjOz8mhP/re4trY26uvr\nO29oZmY7SHoyImo7a9fTN/nNzKyPcsCYmVkWDhgzM8vCAWNmZlk4YMzMLAsHjJmZZeGAMTOzLBww\nZmaWhQPGzMyycMCYmVkWDhgzM8vCAWNmZlk4YMzMLAsHjJmZZeGAMTOzLBwwZmaWhQPGzMyycMCY\nmVkWDhgzM8vCAWNmZlk4YMzMLAsHjJmZZeGAMTOzLLIGjKTJktZK2iDp8ja2HybpYUkrJdVJqinZ\ndqOk1ZIaJM2UpFRfl465PC3DSvY5W9KatN8dOcdmZmYd65/rwJL6AbOAU4BGYJmkhRGxpqTZTcDt\nEXGbpEnA9cB0SScAJwJHp3aPAxOAulSeFhH1rc53BHAFcGJE/LE0eMzMrPvlnMGMAzZExPMR8RYw\nDzi9VZvRwCNpfUnJ9gCqgQHA3kAV8FIn5zsfmBURfwSIiJd3ewRmZtZlOQNmOLCxpNyY6kqtAKam\n9TOBQZIGR8RSisB5MS2LI6KhZL9b0uWxq1sunQFHAkdK+oWkJyRNbqtTkmZIqpdU39zcvHsjNDOz\ndvX0Tf5LgAmSnqa4BNYEbJd0ODAKqKEIpUmSxqd9pkXEGGB8Wqan+v7AEcBE4FxgjqQDWp8wImZH\nRG1E1A4dOjTfyMzM9nA5A6YJOLSkXJPqdoiITRExNSLGAlemui0Us5knImJrRGwF7gOOT9ub0s9X\ngTsoLsVBMUNaGBFvR8SvgXUUgWNmZj0gZ8AsA46QNFLSAOAcYGFpA0lDJLX04Qpgblp/gWJm019S\nFcXspiGVh6R9q4DTgFVpn59TzF5IbY4Ens81ODMz61i2gImIbcCFwGKgAZgfEaslXStpSmo2EVgr\naR1wMHBdql8APAc8Q3GfZkVELKK44b9Y0kpgOcWMaE7aZzGwWdIaivs3l0bE5lzjMzOzjikieroP\nPaa2tjbq6+s7b2hmZjtIejIiajtr19M3+c3MrI9ywJiZWRYOGDMzy8IBY2ZmWThgzMwsCweMmZll\n4YAxM7MsHDBmZpaFA8bMzLJwwJiZWRYOGDMzy6KsgJF0kqQvp/Whkkbm7ZaZmfV2nQaMpG8Cl1E8\nTh+K1xf/KGenzMys9ytnBnMmMAV4DYqXhAGDcnbKzMx6v3IC5q0onukfAJL2zdslMzPrC8oJmPmS\nfgAcIOl84CHefcmXmZlZm/p31iAibpJ0CvAn4EPANRHxYPaemZlZr9ZhwEjqBzwUER8HHCpmZla2\nDi+RRcR24B1J+3dTf8zMrI/o9BIZsBV4RtKDpE+SAUTE17P1yszMer1yAuanaTEzMytbOTf5b5M0\nADgyVa2NiLfzdsvMzHq7cr7JPxFYD8wCvgesk/TX5Rxc0mRJayVtkHR5G9sPk/SwpJWS6iTVlGy7\nUdJqSQ2SZkpSqq9Lx1yelmGp/kuSmkvq/1tZfwJmZpZFOZfI/g/wyYhYCyDpSOBO4NiOdkqfQJsF\nnAI0AsskLYyINSXNbgJuT7OkScD1wHRJJwAnAkendo8DE4C6VJ4WEfVtnPbHEXFhGWMyM7PMyvmi\nZVVLuABExDqK55F1ZhywISKej4i3gHnA6a3ajAYeSetLSrYHUA0MAPZO53upjHOamdn7RDkBUy/p\nh5ImpmUO0NbsobXhwMaScmOqK7UCmJrWzwQGSRocEUspAufFtCyOiIaS/W5Jl8Gubrl0lpyVLrct\nkHRoGX00M7NMygmYvwXWAF9Py5pUVwmXABMkPU1xCawJ2C7pcGAUUEMRSpMkjU/7TIuIMcD4tExP\n9YuAERFxNMWXQm9r64SSZkiql1Tf3NxcoWGYmVlr5QRMf+CfI2JqREwFZgL9ytivCSidRdSkuh0i\nYlM67ljgylS3hWI280REbI2IrcB9wPFpe1P6+SpwB8WlOCJic0S8mQ79Q9q5RxQRsyOiNiJqhw4d\nWsYwzMysK8oJmIeBgSXlgRQPvOzMMuAISSPTx5zPARaWNpA0RFJLH64A5qb1FyhmNv0lVVHMbhpS\neUjatwo4DViVyn9RcugpQOklNTMz62blfIqsOs0iAIiIrZL26WyniNgm6UJgMcWMZ25ErJZ0LVAf\nEQuBicD1kgJ4FPha2n0BMAl4huKG//0RsSi9KmBxCpd+vPfJzl+XNAXYBvwB+FIZYzMzs0xUvOql\ngwbSL4CLIuKpVD4W+G5EHN8N/cuqtrY26uvL+byCmZm1kPRkRNR21q6cGcw3gJ9I2gQI+CDwhd3s\nn5mZ9XHlPCpmmaQPU7wLBvyoGDPbw7399ts0Njbyxhtv9HRXsqqurqampoaqqnK++rizdgNG0nHA\nxoj4XUS8LemjwFnAbyX9r4j4Q9e6bGbWuzU2NjJo0CBGjBjBe7+K13dEBJs3b6axsZGRI0d26Rgd\nfYrsB8BbAOnZYzcAtwOvALO7dDYzsz7gjTfeYPDgwX02XAAkMXjw4N2apXV0iaxfySzlC8DsiLgL\nuEvS8i6f0cysD+jL4dJid8fY0Qymn6SWADqZd58ZBuV9OMDMzDLYsmUL3/ve93Z5v09/+tNs2bIl\nQ4/a1lHA3An8u6S7gdeBxwDSY1xe6Ya+mZlZG9oLmG3btnW437333ssBBxyQq1s7aXcmEhHXSXoY\n+AvggXj3CzN7ARd1R+fMzGxnl19+Oc899xzHHHMMVVVVVFdXc+CBB/Lss8+ybt06zjjjDDZu3Mgb\nb7zBxRdfzIwZMwAYMWIE9fX1bN26lVNPPZWTTjqJX/7ylwwfPpy7776bgQMHdnLmXdPhpa6IeKKN\nunUV7YGZWS/2rUWrWbPpTxU95uhDPsA3P/uRdrffcMMNrFq1iuXLl1NXV8dnPvMZVq1atePTXnPn\nzuWggw7i9ddf57jjjuOss85i8ODB7znG+vXrufPOO5kzZw5nn302d911F+edd15Fx+F7KWZmvdy4\ncePe81HimTNn8rOf/QyAjRs3sn79+p0CZuTIkRxzzDEAHHvssfzmN7+peL8cMGZmu6GjmUZ32Xff\nfXes19XV8dBDD7F06VL22WcfJk6c2OZHjffee+8d6/369eP111+veL86fZqypIskHVjxM5uZWZcM\nGjSIV199tc1tr7zyCgceeCD77LMPzz77LE88sdOdjm5TzgzmYGCZpKcoHqe/ODp7QqaZmWUzePBg\nTjzxRI466igGDhzIwQcfvGPb5MmTufnmmxk1ahQf+tCH+NjHPtZj/ez0acoA6bXEnwS+DNQC84F/\niYjn8nYvLz9N2cy6oqGhgVGjRvV0N7pFW2Mt92nK5bxwjDRj+V1atgEHAgsk3bjr3TUzsz1Bp5fI\nJF0M/A3we4pXEV+aHn65F7Ae+J95u2hmZr1ROfdgDgKmRsRvSysj4h1Jp+XplpmZ9XblXCK7j+IV\nxABI+oCkvwKICL/33szM2lROwHwf2FpS3prqzMzM2lVOwKj0Y8kR8Q7+gqaZmXWinIB5XtLXJVWl\n5WLg+dwdMzOztnX1cf0A3/nOd/jzn/9c4R61rZyAuQA4AWgCGoG/Ambk7JSZmbWvtwRMp5e6IuJl\n4JyuHFzSZOCfgX7ADyPihlbbD6N4OsBQig8SnBcRjWnbjcBnKELwQeDiiAhJdRSvEGh5cM4nUx9b\njnkWsAA4LiL8LUoz63NKH9d/yimnMGzYMObPn8+bb77JmWeeybe+9S1ee+01zj77bBobG9m+fTtX\nX301L730Eps2beLjH/84Q4YMYcmSJVn7Wc73YKqBrwAfAapb6iPiv3ayXz9gFnAKxcxnmaSFEbGm\npNlNwO0RcZukScD1wHRJJwAnAkendo8DE4C6VJ7WVnhIGgRcDPxHZ+MyM6uI+y6H3z1T2WN+cAyc\nekO7m0sf1//AAw+wYMECfvWrXxERTJkyhUcffZTm5mYOOeQQ7rnnHqB4Rtn+++/Pt7/9bZYsWcKQ\nIUMq2+c2lHOJ7F+BDwKfAv4dqAHafsrae40DNkTE8xHxFjAPOL1Vm9G8+yrmJSXbgyLMBgB7A1XA\nS2Wc8++BfwR2fnSomVkf9MADD/DAAw8wduxYPvrRj/Lss8+yfv16xowZw4MPPshll13GY489xv77\n79/tfSvn02CHR8TnJZ2eZhp3kF6f3InhwMaScsv9m1IrgKkUl9HOBAZJGhwRSyUtAV4EBHy31Xdu\nbpG0HbgL+Id06eyjwKERcY+kS8von5nZ7utgptEdIoIrrriCr371qztte+qpp7j33nu56qqrOPnk\nk7nmmmu6tW/lzGDeTj+3SDoK2B8YVqHzXwJMkPQ0xSWwJmC7pMOBURSzpeHAJEnj0z7TImIMMD4t\n09Nja74N/I/OTihphqR6SfXNzc0VGoaZWfcpfVz/pz71KebOncvWrcXXFZuamnj55ZfZtGkT++yz\nD+eddx6XXnopTz311E775lbODGZ2eh/MVcBCYD/g6jL2awIOLSnXpLodImITxQwGSfsBZ0XEFknn\nA09ExNa07T7geOCxiGhK+76aZlPjgLuBo4C64sHPfBBYKGlK63s1ETEbmA3F05TLGIeZ2ftK6eP6\nTz31VL74xS9y/PHHA7Dffvvxox/9iA0bNnDppZey1157UVVVxfe/X3w/fsaMGUyePJlDDjkk+03+\nDh/Xn2YGn4uI+bt8YKk/sA44mSJYlgFfjIjVJW2GAH9IzzW7DtgeEddI+gJwPjCZ4hLZ/cB3KB5b\nc0BE/F5SFXAn8FBE3Nzq3HXAJZ19isyP6zezrvDj+ivwuP70rf0uPS05IrYBFwKLgQZgfkSslnSt\npCmp2URgraR1FC82uy7VLwCeA56huE+zIiIWUdzwXyxpJbCcIrjmdKV/ZmaWVzmXyB6SdAnwY+C1\nlsqI+EP7u+xocy9wb6u6a0rWF1CESev9tgM73bGKiNeAY8s478TO2piZWV7lBMwX0s+vldQF8JeV\n746ZmfUV5XyTf2R3dMTMrDeJCNKHivqsju7Rl6Ocb/L/TTsnvn23zmxm1ktVV1ezefNmBg8e3GdD\nJiLYvHkz1dXVnTduRzmXyI4rWa+m+FTYU4ADxsz2SDU1NTQ2NtLXv0tXXV1NTU1Nl/cv5xLZRaVl\nSQdQPPbFzGyPVFVVxciRvnvQmXK+yd/aa4D/ZM3MrEPl3INZRPGpMSgCaTSwy1+8NDOzPUs592Bu\nKlnfBvy25Z0tZmZm7SknYF4AXoyINwAkDZQ0IiJ+k7VnZmbWq5VzD+YnwDsl5e2pzszMrF3lBEz/\n9MIwANL6gHxdMjOzvqCcgGkueTglkk4Hfp+vS2Zm1heUcw/mAuDfJH03lRuBNr/db2Zm1qKcL1o+\nB3wsvRCMlpeAmZmZdaTTS2SS/rekAyJia0RslXSgpH/ojs6ZmVnvVc49mFMjYktLISL+CHw6X5fM\nzKwvKCdg+knau6UgaSDFmyXNzMzaVc5N/n8DHpZ0Syp/GT9J2czMOlHOTf5/lLQC+ESq+vuIWJy3\nW2Zm1tuVM4MhIu4H7geQdJKkWRHxtU52MzOzPVhZASNpLHAucDbwa+CnOTtlZma9X7sBI+lIilA5\nl+Kb+z8GFBEf76a+mZlZL9bRDOZZ4DHgtIjYACDp77qlV2Zm1ut19DHlqcCLwBJJcySdDGhXDi5p\nsqS1kjZIuryN7YdJeljSSkl1kmpKtt0oabWkBkkzJSnV16VjLk/LsFR/gaRnUt3jkkbvSl/NzKyy\n2g2YiPh5RJwDfBhYAnwDGCbp+5I+2dmBJfUDZgGnUrwF89w2/tG/Cbg9Io4GrgWuT/ueAJwIHA0c\nBRwHTCjZb1pEHJOWl1PdHRExJiKOAW4Evt1ZH83MLJ9Ov2gZEa9FxB0R8VmgBngauKyMY48DNkTE\n8+kR//OA01u1GQ08ktaXlGwPoJritQB7A1XAS530808lxX159zXPZmbWA8r5Jv8OEfHHiJgdESeX\n0Xw4sLGk3JjqSq2guBQHcCYwSNLgiFhKETgvpmVxRDSU7HdLuhR2dculMwBJX5P0HMUM5uttdUrS\nDEn1kuqbm5vLGIaZmXXFLgVMBpcAEyQ9TXEJrAnYLulwYBTFjGk4MEnS+LTPtIgYA4xPy/SWg0XE\nrIj4TxQzrKvaOmEKyNqIqB06dGiucZmZ7fFyBkwTcGhJuSbV7RARmyJiakSMBa5MdVsoZjNPtDzB\nGbgPOD5tb0o/XwXuoLgU19o84IzKDsfMzHZFzoBZBhwhaaSkAcA5wMLSBpKGSGrpwxXA3LT+AsXM\npr+kKorZTUMqD0n7VgGnAatS+YiSQ38GWJ9pXGZmVoayvsnfFRGxTdKFwGKgHzA3IlZLuhaoj4iF\nwETgekkBPAq0PH5mATAJeIbiZv39EbFI0r7A4hQu/YCHgDlpnwslfQJ4G/gj8F9yjc3MzDqniD33\nw1a1tbVRX1/f090wM+tVJD0ZEbWdtevpm/xmZtZHOWDMzCwLB4yZmWXhgDEzsywcMGZmloUDxszM\nsnDAmJlZFg4YMzPLwgFjZmZZOGDMzCwLB4yZmWXhgDEzsywcMGZmloUDxszMsnDAmJlZFg4YMzPL\nwgFjZmZZOGDMzCwLB4yZmWXhgDEzsywcMGZmloUDxszMssgaMJImS1oraYOky9vYfpikhyWtlFQn\nqaZk242SVktqkDRTklJ9XTrm8rQMS/X/XdKadKyHJR2Wc2xmZtaxbAEjqR8wCzgVGA2cK2l0q2Y3\nAbdHxNHAtcD1ad8TgBOBo4GjgOOACSX7TYuIY9Lycqp7GqhNx1oA3JhnZGZmVo6cM5hxwIaIeD4i\n3gLmAae3ajMaeCStLynZHkA1MADYG6gCXuroZBGxJCL+nIpPADUdtTczs7xyBsxwYGNJuTHVlVoB\nTE3rZwKDJA2OiKUUgfNiWhZHREPJfreky2NXt1w6a+UrwH2VGISZmXVNT9/kvwSYIOlpiktgTcB2\nSYcDoyhmIcOBSZLGp32mRcQYYHxappceUNJ5QC3wT22dUNIMSfWS6pubm3OMyczMyBswTcChJeWa\nVLdDRGyKiKkRMRa4MtVtoZjNPBERWyNiK8Vs5Pi0vSn9fBW4g+JSHACSPpGOMyUi3myrUxExOyJq\nI6J26NChlRmpmZntJGfALAOOkDRS0gDgHGBhaQNJQyS19OEKYG5af4FiZtNfUhXF7KYhlYekfauA\n04BVqTwW+AFFuLyMmZn1qGwBExHbgAuBxUADMD8iVku6VtKU1GwisFbSOuBg4LpUvwB4DniG4j7N\niohYRHHDf7GklcByihnRnLTPPwH7AT9J92feE2ZmZta9FBE93YceU1tbG/X19T3dDTOzXkXSkxFR\n21m7nr7Jb2ZmfZQDxszMsnDAmJlZFg4YMzPLwgFjZmZZOGDMzCwLB4yZmWXhgDEzsywcMGZmloUD\nxszMsnDAmJlZFg4YMzPLwgFjZmZZOGDMzCwLB4yZmWXhgDEzsywcMGZmloUDxszMsnDAmJlZFg4Y\nMzPLwgFjZmZZ9O/pDvRK910Ov3ump3thZtZ1HxwDp96Q9RRZZzCSJktaK2mDpMvb2H6YpIclrZRU\nJ6mmZNuNklZLapA0U5JSfV065vK0DEv1fy3pKUnbJH0u57jMzKxz2WYwkvoBs4BTgEZgmaSFEbGm\npNlNwO0RcZukScD1wHRJJwAnAkendo8DE4C6VJ4WEfWtTvkC8CXgkgzDea/MqW9m1hfknMGMAzZE\nxPMR8RYwDzi9VZvRwCNpfUnJ9gCqgQHA3kAV8FJHJ4uI30TESuCdynTfzMx2R86AGQ5sLCk3prpS\nK4Cpaf1MYJCkwRGxlCJwXkzL4ohoKNnvlnR57OqWS2dmZvb+0tOfIrsEmCDpaYpLYE3AdkmHA6OA\nGopQmiRpfNpnWkSMAcanZfqunFDSDEn1kuqbm5srNQ4zM2slZ8A0AYeWlGtS3Q4RsSkipkbEWODK\nVLeFYjbzRERsjYitwH3A8Wl7U/r5KnAHxaW4skXE7IiojYjaoUOHdm1kZmbWqZwBsww4QtJISQOA\nc4CFpQ0kDZHU0ocrgLlp/QWKmU1/SVUUs5uGVB6S9q0CTgNWZRyDmZl1UbaAiYhtwIXAYqABmB8R\nqyVdK2lKajYRWCtpHXAwcF2qXwA8BzxDcZ9mRUQsorjhv1jSSmA5xYxoDoCk4yQ1Ap8HfiBpda6x\nmZlZ5xQRPd2HHlNbWxv19a0/7WxmZh2R9GRE1HbWrqdv8puZWR+1R89gJDUDvwX2B14p2VRabm/b\nEOD3FepK63N0tV1729uqL3fMpeuVGnO54y2nrcfcfv2ulHvjmHf1d9y6/H4ec6X+XrcuV2rMh0VE\n55+Siog9fgFmt1dubxtQn+v8XW3X3va26ssdc6v1ioy53PF6zLs35l0p98Yx7+rvuDeNuVJ/r7tj\nzB0tvkRWWNRBuaNtuc7f1XbtbW+rvtwx9+R4y2nrMbdfvyvl3jjmXf0dty6/n8dcqb/Xrcs5xtyu\nPfoS2e6QVB9l3OTqSzzmPYPHvGfojjF7BtN1s3u6Az3AY94zeMx7huxj9gzGzMyy8AzGzMyycMCY\nmVkWDhgzM8vCAZOJpH3TawFO6+m+dAdJoyTdLGmBpL/t6f50B0lnSJoj6ceSPtnT/ekOkv5S0r9I\nWtDTfckl/bd7W/rdTuvp/nSHXL9XB0wrkuZKelnSqlb1kyWtlbRB0uVlHOoyYH6eXlZWJcYcEQ0R\ncQFwNsXrrt/XKjTmn0fE+cAFwBdy9rcSKjTm5yPiK3l7Wnm7OPapwIL0u52y08F6iV0Zc67fqwNm\nZ7cCk0srJPUDZgGnUrzm+VxJoyWNkfT/Wi3DJJ0CrAFe7u7Od9Gt7OaY0z5TgHuAe7u3+11yKxUY\nc3JV2u/97lYqN+be5lbKHDvFu6ta3sa7vRv7WGm3Uv6Ys+if68C9VUQ8KmlEq+pxwIaIeB5A0jzg\n9Ii4nuKdNO8haSKwL8Uv8HVJ90bEOzn7vTsqMeZ0nIXAQkn3ULwM7n2rQr9nATcA90XEU3l7vPsq\n9XvujXZl7BSvd6+heCVIr/2f8F0c85ocfei1f3jdbDjv/h8NFH8Bh7fXOCKujIhvUPwjO+f9HC4d\n2KUxS5ooaaakH9A7ZjBt2aUxAxcBnwA+J+mCnB3LaFd/z4Ml3QyMlXRF7s5l1t7YfwqcJen7dPOj\nVbpBm2PO9Xv1DCajiLi1p/vQXSKiDqjr4W50q4iYCczs6X50p4jYTHHPqc+KiNeAL/d0P7pTrt+r\nZzDlaQIOLSnXpLq+zGP2mPu6PXHs3TpmB0x5lgFHSBopaQBwDrCwh/uUm8fsMfd1e+LYu3XMDphW\nJN0JLAU+JKlR0lciYhtwIbAYaADmR8TqnuxnJXnMHjN9dMwt9sSxvx/G7IddmplZFp7BmJlZFg4Y\nMzPLwgFjZmZZOGDMzCwLB4yZmWXhgDEzsywcMGbdQMW7Y0LSh1N5ROvHqLexT6dtzN7PHDBm3eNc\n4PH002yP4IAxy0zSfsBJwFcoHs3RevuXJN0tqU7SeknfLNncT8WbFVdLekDSwLTP+ZKWSVoh6S5J\n+3TPaMzK54Axy+904P6IWAdslnRsG23GAWcBRwOfl1Sb6o8AZkXER4AtqQ3ATyPiuIj4zxSP/Oh1\nb5m0vs8BY5bfucC8tD6Pti+TPRgRmyPidYr3kZyU6n8dEcvT+pPAiLR+lKTHJD0DTAM+kqXnZrvB\n74Mxy0jSQcAkYIykAPoBwc6vWG79UMCW8psldduBgWn9VuCMiFgh6UvAxMr12qwyPIMxy+tzwL9G\nxGERMSIiDgV+zXvfyQFwiqSD0j2WM4BfdHLcQcCLkqooZjBm7zsOGLO8zgV+1qruLqD1a2l/lepX\nAndFRH0nx70a+A+KIHq2Av2u8YpOAAAASElEQVQ0qzg/rt+sh6VLXLURcWFP98WskjyDMTOzLDyD\nMTOzLDyDMTOzLBwwZmaWhQPGzMyycMCYmVkWDhgzM8vCAWNmZln8f+AxNIuju68FAAAAAElFTkSu\nQmCC\n", 726 | "text/plain": [ 727 | "" 728 | ] 729 | }, 730 | "metadata": {}, 731 | "output_type": "display_data" 732 | } 733 | ], 734 | "source": [ 735 | "train_errors = []\n", 736 | "test_errors = []\n", 737 | "\n", 738 | "alpha_list = [0.0001, 0.001, 0.01, 0.1, 10]\n", 739 | "\n", 740 | "# Evaluate the training and test classification errors for each value of alpha\n", 741 | "\n", 742 | "for value in alpha_list:\n", 743 | " \n", 744 | " # Create Lasso object and fit\n", 745 | " lasso_regression = Lasso(alpha= value)\n", 746 | " lasso_regression.fit(X_train, y_train)\n", 747 | " \n", 748 | " # Evaluate error rates and append to lists\n", 749 | " train_errors.append(ridge_regression.score(X_train, y_train) )\n", 750 | " test_errors.append(ridge_regression.score(X_test, y_test))\n", 751 | " \n", 752 | "# Plot results\n", 753 | "plt.semilogx(alpha_list, train_errors, alpha_list, test_errors)\n", 754 | "plt.legend((\"train\", \"test\"))\n", 755 | "plt.ylabel('Accuracy Score')\n", 756 | "plt.xlabel('Alpha')\n", 757 | "plt.show()" 758 | ] 759 | } 760 | ], 761 | "metadata": { 762 | "kernelspec": { 763 | "display_name": "Python 3", 764 | "language": "python", 765 | "name": "python3" 766 | }, 767 | "language_info": { 768 | "codemirror_mode": { 769 | "name": "ipython", 770 | "version": 3 771 | }, 772 | "file_extension": ".py", 773 | "mimetype": "text/x-python", 774 | "name": "python", 775 | "nbconvert_exporter": "python", 776 | "pygments_lexer": "ipython3", 777 | "version": "3.6.1" 778 | } 779 | }, 780 | "nbformat": 4, 781 | "nbformat_minor": 2 782 | } 783 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Packt 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | # Machine Learning with scikit-learn Quick Start Guide 5 | 6 | Machine Learning with scikit-learn Quick Start Guide 7 | 8 | This is the code repository for [Machine Learning with scikit-learn Quick Start Guide](https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-scikit-learn-quick-start-guide?utm_source=github&utm_medium=repository&utm_campaign=9781789343700 ), published by Packt. 9 | 10 | **Classification, regression, and clustering techniques in Python** 11 | 12 | ## What is this book about? 13 | Scikit-learn is a robust machine learning library for the Python programming language. It provides a set of supervised and unsupervised learning algorithms. This book is the easiest way to learn how to deploy, optimize, and evaluate all of the important machine learning algorithms that scikit-learn provides. 14 | 15 | This book covers the following exciting features: 16 | * Learn how to work with all scikit-learn's machine learning algorithms 17 | * Install and set up scikit-learn to build your first machine learning model 18 | * Employ Unsupervised Machine Learning Algorithms to cluster unlabelled data into groups 19 | * Perform classification and regression machine learning 20 | * Use an effective pipeline to build a machine learning project from scratch 21 | 22 | If you feel this book is for you, get your [copy](https://www.amazon.com/dp/1789343704) today! 23 | 24 | https://www.packtpub.com/ 26 | 27 | ## Instructions and Navigations 28 | All of the code is organized into folders. For example, Chapter02. 29 | 30 | The code will look like the following: 31 | ``` 32 | from sklearn.naive_bayes import GaussianNB 33 | 34 | #Initializing an NB classifier 35 | 36 | nb_classifier = GaussianNB() 37 | 38 | #Fitting the classifier into the training data 39 | 40 | nb_classifier.fit(X_train, y_train) 41 | ``` 42 | 43 | **Following is what you need for this book:** 44 | This book is for aspiring machine learning developers who want to get started with scikit-learn. Intermediate knowledge of Python programming and some fundamental knowledge of linear algebra and probability will help. 45 | 46 | With the following software and hardware list you can run all code files present in the book (Chapter 1-8). 47 | ### Software and Hardware List 48 | | Chapter | Software required | OS required | 49 | | -------- | ------------------------------------ | ----------------------------------- | 50 | | 1 | Pandas (= 0.23.4) Matplotlib (= 3.0.0) Scikit-learn (= 0.20.0) Tree (Latest) NumPy (= 1.15.1 Pydotplus (= 2.0.2) Image (= 5.3.0) | Windows, Mac OS X | 51 | | 2 | Pandas (= 0.23.4) Scikit-learn (= 0.20.0) | Windows, Mac OS X | 52 | | 3 | Pandas (= 0.23.4) Scikit-learn (= 0.20.0) | Windows, Mac OS X | 53 | | 4 | Pandas (= 0.23.4) Scikit-learn (= 0.20.0) | Windows, Mac OS X | 54 | | 5 | Pandas (= 0.23.4) Scikit-learn (= 0.20.0) | Windows, Mac OS X | 55 | | 6 | Pandas (0.23.4) Scikit-learn (= 0.20.0) Tree (Latest) NumPy (= 1.15.1) ydotplus (= 2.0.2) IL (= 5.3.0) | Windows, Mac OS X | 56 | | 7 | Pandas (= 0.23.4) Scikit-learn (= 0.20.0) | Windows, Mac OS X | 57 | | 8 | Pandas (= 0.23.4) Scikit-learn (= 0.20.0) Scikit-plot (= 0.3.7) | Windows, Mac OS X | 58 | 59 | ## Code in Action 60 | 61 | Click on the following link to see the Code in Action: 62 | 63 | [http://bit.ly/2OcWIGH](http://bit.ly/2OcWIGH) 64 | 65 | 66 | ### Related products 67 | * Python Machine Learning Blueprints - Second Edition [[Packt]](https://www.packtpub.com/big-data-and-business-intelligence/python-machine-learning-blueprints-second-edition?utm_source=github&utm_medium=repository&utm_campaign=9781788994170 ) [[Amazon]](https://www.amazon.com/dp/B07JLMHWRG) 68 | 69 | * scikit-learn Cookbook [[Packt]](https://www.packtpub.com/big-data-and-business-intelligence/scikit-learn-cookbook?utm_source=github&utm_medium=repository&utm_campaign=9781783989485 ) [[Amazon]](https://www.amazon.com/dp/1783989483) 70 | 71 | 72 | ## Get to Know the Author 73 | **Kevin Jolly** 74 | is a formally educated data scientist with a master's degree in data science from the prestigious King's College London. Kevin works as a statistical analyst with a digital healthcare start-up, Connido Limited, in London, where he is primarily involved in leading the data science projects that the company undertakes. He has built machine learning pipelines for small and big data, with a focus on scaling such pipelines into production for the products that the company has built. 75 | 76 | Kevin is also the author of a book titled Hands-On Data Visualization with Bokeh, published by Packt. He is the editor-in-chief of Linear, a weekly online publication on data science software and products. 77 | 78 | ## Other books by the authors 79 | [Hands-On Data Visualization with Bokeh](https://www.packtpub.com/big-data-and-business-intelligence/hands-data-visualization-bokeh?utm_source=github&utm_medium=repository&utm_campaign=9781789135404 ) 80 | 81 | ### Suggestions and Feedback 82 | [Click here](https://docs.google.com/forms/d/e/1FAIpQLSdy7dATC6QmEL81FIUuymZ0Wy9vH1jHkvpY57OiMeKGqib_Ow/viewform) if you have any feedback or suggestions. 83 | 84 | 85 | ### Download a free PDF 86 | 87 | If you have already purchased a print or Kindle version of this book, you can get a DRM-free PDF version at no cost.
Simply click on the link to claim your free PDF. 88 |

https://packt.link/free-ebook/9781789343700

--------------------------------------------------------------------------------