├── Anomaly Detection ├── Anomaly Detection.ipynb ├── ex8_movieParams.mat ├── ex8_movies.mat ├── ex8data1.mat ├── ex8data2.mat └── movie_ids.txt ├── Bias_Vs_Variance ├── ML_BiasVsVariance.ipynb └── ex5data1.mat ├── KmeansClustering_PCA ├── K-means Clustering & PCA.ipynb ├── bird_small.mat ├── ex7data1.mat ├── ex7data2.mat └── ex7faces.mat ├── LinearRegression ├── Kaggle │ ├── House Price Prediction (Andrew Ng).ipynb │ ├── data_description.txt │ ├── test.csv │ └── train.csv ├── ML_LinearRegression.ipynb ├── Multi_linear.txt └── Uni_linear.txt ├── LogisticRegression ├── ML_LogisticRegression.ipynb ├── ML_RegularizedLogisticRegression.ipynb ├── ex2data1.txt └── ex2data2.txt ├── NeuralNetworks ├── ML_NeuralNetworks.ipynb ├── ex3data1.mat ├── ex3weights.mat ├── ex4data1.mat └── ex4weights.mat ├── README.md └── SupportVectorMachines ├── Support Vector Machines.ipynb ├── emailSample1.txt ├── emailSample2.txt ├── ex6data1.mat ├── ex6data2.mat ├── ex6data3.mat ├── spamSample1.txt ├── spamSample2.txt ├── spamTest.mat ├── spamTrain.mat └── vocab.txt /Anomaly Detection/ex8_movieParams.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/Anomaly Detection/ex8_movieParams.mat -------------------------------------------------------------------------------- /Anomaly Detection/ex8_movies.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/Anomaly Detection/ex8_movies.mat -------------------------------------------------------------------------------- /Anomaly Detection/ex8data1.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/Anomaly Detection/ex8data1.mat -------------------------------------------------------------------------------- /Anomaly Detection/ex8data2.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/Anomaly Detection/ex8data2.mat -------------------------------------------------------------------------------- /Anomaly Detection/movie_ids.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/Anomaly Detection/movie_ids.txt -------------------------------------------------------------------------------- /Bias_Vs_Variance/ex5data1.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/Bias_Vs_Variance/ex5data1.mat -------------------------------------------------------------------------------- /KmeansClustering_PCA/bird_small.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/KmeansClustering_PCA/bird_small.mat -------------------------------------------------------------------------------- /KmeansClustering_PCA/ex7data1.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/KmeansClustering_PCA/ex7data1.mat -------------------------------------------------------------------------------- /KmeansClustering_PCA/ex7data2.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/KmeansClustering_PCA/ex7data2.mat -------------------------------------------------------------------------------- /KmeansClustering_PCA/ex7faces.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/KmeansClustering_PCA/ex7faces.mat -------------------------------------------------------------------------------- /LinearRegression/Kaggle/data_description.txt: -------------------------------------------------------------------------------- 1 | MSSubClass: Identifies the type of dwelling involved in the sale. 2 | 3 | 20 1-STORY 1946 & NEWER ALL STYLES 4 | 30 1-STORY 1945 & OLDER 5 | 40 1-STORY W/FINISHED ATTIC ALL AGES 6 | 45 1-1/2 STORY - UNFINISHED ALL AGES 7 | 50 1-1/2 STORY FINISHED ALL AGES 8 | 60 2-STORY 1946 & NEWER 9 | 70 2-STORY 1945 & OLDER 10 | 75 2-1/2 STORY ALL AGES 11 | 80 SPLIT OR MULTI-LEVEL 12 | 85 SPLIT FOYER 13 | 90 DUPLEX - ALL STYLES AND AGES 14 | 120 1-STORY PUD (Planned Unit Development) - 1946 & NEWER 15 | 150 1-1/2 STORY PUD - ALL AGES 16 | 160 2-STORY PUD - 1946 & NEWER 17 | 180 PUD - MULTILEVEL - INCL SPLIT LEV/FOYER 18 | 190 2 FAMILY CONVERSION - ALL STYLES AND AGES 19 | 20 | MSZoning: Identifies the general zoning classification of the sale. 21 | 22 | A Agriculture 23 | C Commercial 24 | FV Floating Village Residential 25 | I Industrial 26 | RH Residential High Density 27 | RL Residential Low Density 28 | RP Residential Low Density Park 29 | RM Residential Medium Density 30 | 31 | LotFrontage: Linear feet of street connected to property 32 | 33 | LotArea: Lot size in square feet 34 | 35 | Street: Type of road access to property 36 | 37 | Grvl Gravel 38 | Pave Paved 39 | 40 | Alley: Type of alley access to property 41 | 42 | Grvl Gravel 43 | Pave Paved 44 | NA No alley access 45 | 46 | LotShape: General shape of property 47 | 48 | Reg Regular 49 | IR1 Slightly irregular 50 | IR2 Moderately Irregular 51 | IR3 Irregular 52 | 53 | LandContour: Flatness of the property 54 | 55 | Lvl Near Flat/Level 56 | Bnk Banked - Quick and significant rise from street grade to building 57 | HLS Hillside - Significant slope from side to side 58 | Low Depression 59 | 60 | Utilities: Type of utilities available 61 | 62 | AllPub All public Utilities (E,G,W,& S) 63 | NoSewr Electricity, Gas, and Water (Septic Tank) 64 | NoSeWa Electricity and Gas Only 65 | ELO Electricity only 66 | 67 | LotConfig: Lot configuration 68 | 69 | Inside Inside lot 70 | Corner Corner lot 71 | CulDSac Cul-de-sac 72 | FR2 Frontage on 2 sides of property 73 | FR3 Frontage on 3 sides of property 74 | 75 | LandSlope: Slope of property 76 | 77 | Gtl Gentle slope 78 | Mod Moderate Slope 79 | Sev Severe Slope 80 | 81 | Neighborhood: Physical locations within Ames city limits 82 | 83 | Blmngtn Bloomington Heights 84 | Blueste Bluestem 85 | BrDale Briardale 86 | BrkSide Brookside 87 | ClearCr Clear Creek 88 | CollgCr College Creek 89 | Crawfor Crawford 90 | Edwards Edwards 91 | Gilbert Gilbert 92 | IDOTRR Iowa DOT and Rail Road 93 | MeadowV Meadow Village 94 | Mitchel Mitchell 95 | Names North Ames 96 | NoRidge Northridge 97 | NPkVill Northpark Villa 98 | NridgHt Northridge Heights 99 | NWAmes Northwest Ames 100 | OldTown Old Town 101 | SWISU South & West of Iowa State University 102 | Sawyer Sawyer 103 | SawyerW Sawyer West 104 | Somerst Somerset 105 | StoneBr Stone Brook 106 | Timber Timberland 107 | Veenker Veenker 108 | 109 | Condition1: Proximity to various conditions 110 | 111 | Artery Adjacent to arterial street 112 | Feedr Adjacent to feeder street 113 | Norm Normal 114 | RRNn Within 200' of North-South Railroad 115 | RRAn Adjacent to North-South Railroad 116 | PosN Near positive off-site feature--park, greenbelt, etc. 117 | PosA Adjacent to postive off-site feature 118 | RRNe Within 200' of East-West Railroad 119 | RRAe Adjacent to East-West Railroad 120 | 121 | Condition2: Proximity to various conditions (if more than one is present) 122 | 123 | Artery Adjacent to arterial street 124 | Feedr Adjacent to feeder street 125 | Norm Normal 126 | RRNn Within 200' of North-South Railroad 127 | RRAn Adjacent to North-South Railroad 128 | PosN Near positive off-site feature--park, greenbelt, etc. 129 | PosA Adjacent to postive off-site feature 130 | RRNe Within 200' of East-West Railroad 131 | RRAe Adjacent to East-West Railroad 132 | 133 | BldgType: Type of dwelling 134 | 135 | 1Fam Single-family Detached 136 | 2FmCon Two-family Conversion; originally built as one-family dwelling 137 | Duplx Duplex 138 | TwnhsE Townhouse End Unit 139 | TwnhsI Townhouse Inside Unit 140 | 141 | HouseStyle: Style of dwelling 142 | 143 | 1Story One story 144 | 1.5Fin One and one-half story: 2nd level finished 145 | 1.5Unf One and one-half story: 2nd level unfinished 146 | 2Story Two story 147 | 2.5Fin Two and one-half story: 2nd level finished 148 | 2.5Unf Two and one-half story: 2nd level unfinished 149 | SFoyer Split Foyer 150 | SLvl Split Level 151 | 152 | OverallQual: Rates the overall material and finish of the house 153 | 154 | 10 Very Excellent 155 | 9 Excellent 156 | 8 Very Good 157 | 7 Good 158 | 6 Above Average 159 | 5 Average 160 | 4 Below Average 161 | 3 Fair 162 | 2 Poor 163 | 1 Very Poor 164 | 165 | OverallCond: Rates the overall condition of the house 166 | 167 | 10 Very Excellent 168 | 9 Excellent 169 | 8 Very Good 170 | 7 Good 171 | 6 Above Average 172 | 5 Average 173 | 4 Below Average 174 | 3 Fair 175 | 2 Poor 176 | 1 Very Poor 177 | 178 | YearBuilt: Original construction date 179 | 180 | YearRemodAdd: Remodel date (same as construction date if no remodeling or additions) 181 | 182 | RoofStyle: Type of roof 183 | 184 | Flat Flat 185 | Gable Gable 186 | Gambrel Gabrel (Barn) 187 | Hip Hip 188 | Mansard Mansard 189 | Shed Shed 190 | 191 | RoofMatl: Roof material 192 | 193 | ClyTile Clay or Tile 194 | CompShg Standard (Composite) Shingle 195 | Membran Membrane 196 | Metal Metal 197 | Roll Roll 198 | Tar&Grv Gravel & Tar 199 | WdShake Wood Shakes 200 | WdShngl Wood Shingles 201 | 202 | Exterior1st: Exterior covering on house 203 | 204 | AsbShng Asbestos Shingles 205 | AsphShn Asphalt Shingles 206 | BrkComm Brick Common 207 | BrkFace Brick Face 208 | CBlock Cinder Block 209 | CemntBd Cement Board 210 | HdBoard Hard Board 211 | ImStucc Imitation Stucco 212 | MetalSd Metal Siding 213 | Other Other 214 | Plywood Plywood 215 | PreCast PreCast 216 | Stone Stone 217 | Stucco Stucco 218 | VinylSd Vinyl Siding 219 | Wd Sdng Wood Siding 220 | WdShing Wood Shingles 221 | 222 | Exterior2nd: Exterior covering on house (if more than one material) 223 | 224 | AsbShng Asbestos Shingles 225 | AsphShn Asphalt Shingles 226 | BrkComm Brick Common 227 | BrkFace Brick Face 228 | CBlock Cinder Block 229 | CemntBd Cement Board 230 | HdBoard Hard Board 231 | ImStucc Imitation Stucco 232 | MetalSd Metal Siding 233 | Other Other 234 | Plywood Plywood 235 | PreCast PreCast 236 | Stone Stone 237 | Stucco Stucco 238 | VinylSd Vinyl Siding 239 | Wd Sdng Wood Siding 240 | WdShing Wood Shingles 241 | 242 | MasVnrType: Masonry veneer type 243 | 244 | BrkCmn Brick Common 245 | BrkFace Brick Face 246 | CBlock Cinder Block 247 | None None 248 | Stone Stone 249 | 250 | MasVnrArea: Masonry veneer area in square feet 251 | 252 | ExterQual: Evaluates the quality of the material on the exterior 253 | 254 | Ex Excellent 255 | Gd Good 256 | TA Average/Typical 257 | Fa Fair 258 | Po Poor 259 | 260 | ExterCond: Evaluates the present condition of the material on the exterior 261 | 262 | Ex Excellent 263 | Gd Good 264 | TA Average/Typical 265 | Fa Fair 266 | Po Poor 267 | 268 | Foundation: Type of foundation 269 | 270 | BrkTil Brick & Tile 271 | CBlock Cinder Block 272 | PConc Poured Contrete 273 | Slab Slab 274 | Stone Stone 275 | Wood Wood 276 | 277 | BsmtQual: Evaluates the height of the basement 278 | 279 | Ex Excellent (100+ inches) 280 | Gd Good (90-99 inches) 281 | TA Typical (80-89 inches) 282 | Fa Fair (70-79 inches) 283 | Po Poor (<70 inches 284 | NA No Basement 285 | 286 | BsmtCond: Evaluates the general condition of the basement 287 | 288 | Ex Excellent 289 | Gd Good 290 | TA Typical - slight dampness allowed 291 | Fa Fair - dampness or some cracking or settling 292 | Po Poor - Severe cracking, settling, or wetness 293 | NA No Basement 294 | 295 | BsmtExposure: Refers to walkout or garden level walls 296 | 297 | Gd Good Exposure 298 | Av Average Exposure (split levels or foyers typically score average or above) 299 | Mn Mimimum Exposure 300 | No No Exposure 301 | NA No Basement 302 | 303 | BsmtFinType1: Rating of basement finished area 304 | 305 | GLQ Good Living Quarters 306 | ALQ Average Living Quarters 307 | BLQ Below Average Living Quarters 308 | Rec Average Rec Room 309 | LwQ Low Quality 310 | Unf Unfinshed 311 | NA No Basement 312 | 313 | BsmtFinSF1: Type 1 finished square feet 314 | 315 | BsmtFinType2: Rating of basement finished area (if multiple types) 316 | 317 | GLQ Good Living Quarters 318 | ALQ Average Living Quarters 319 | BLQ Below Average Living Quarters 320 | Rec Average Rec Room 321 | LwQ Low Quality 322 | Unf Unfinshed 323 | NA No Basement 324 | 325 | BsmtFinSF2: Type 2 finished square feet 326 | 327 | BsmtUnfSF: Unfinished square feet of basement area 328 | 329 | TotalBsmtSF: Total square feet of basement area 330 | 331 | Heating: Type of heating 332 | 333 | Floor Floor Furnace 334 | GasA Gas forced warm air furnace 335 | GasW Gas hot water or steam heat 336 | Grav Gravity furnace 337 | OthW Hot water or steam heat other than gas 338 | Wall Wall furnace 339 | 340 | HeatingQC: Heating quality and condition 341 | 342 | Ex Excellent 343 | Gd Good 344 | TA Average/Typical 345 | Fa Fair 346 | Po Poor 347 | 348 | CentralAir: Central air conditioning 349 | 350 | N No 351 | Y Yes 352 | 353 | Electrical: Electrical system 354 | 355 | SBrkr Standard Circuit Breakers & Romex 356 | FuseA Fuse Box over 60 AMP and all Romex wiring (Average) 357 | FuseF 60 AMP Fuse Box and mostly Romex wiring (Fair) 358 | FuseP 60 AMP Fuse Box and mostly knob & tube wiring (poor) 359 | Mix Mixed 360 | 361 | 1stFlrSF: First Floor square feet 362 | 363 | 2ndFlrSF: Second floor square feet 364 | 365 | LowQualFinSF: Low quality finished square feet (all floors) 366 | 367 | GrLivArea: Above grade (ground) living area square feet 368 | 369 | BsmtFullBath: Basement full bathrooms 370 | 371 | BsmtHalfBath: Basement half bathrooms 372 | 373 | FullBath: Full bathrooms above grade 374 | 375 | HalfBath: Half baths above grade 376 | 377 | Bedroom: Bedrooms above grade (does NOT include basement bedrooms) 378 | 379 | Kitchen: Kitchens above grade 380 | 381 | KitchenQual: Kitchen quality 382 | 383 | Ex Excellent 384 | Gd Good 385 | TA Typical/Average 386 | Fa Fair 387 | Po Poor 388 | 389 | TotRmsAbvGrd: Total rooms above grade (does not include bathrooms) 390 | 391 | Functional: Home functionality (Assume typical unless deductions are warranted) 392 | 393 | Typ Typical Functionality 394 | Min1 Minor Deductions 1 395 | Min2 Minor Deductions 2 396 | Mod Moderate Deductions 397 | Maj1 Major Deductions 1 398 | Maj2 Major Deductions 2 399 | Sev Severely Damaged 400 | Sal Salvage only 401 | 402 | Fireplaces: Number of fireplaces 403 | 404 | FireplaceQu: Fireplace quality 405 | 406 | Ex Excellent - Exceptional Masonry Fireplace 407 | Gd Good - Masonry Fireplace in main level 408 | TA Average - Prefabricated Fireplace in main living area or Masonry Fireplace in basement 409 | Fa Fair - Prefabricated Fireplace in basement 410 | Po Poor - Ben Franklin Stove 411 | NA No Fireplace 412 | 413 | GarageType: Garage location 414 | 415 | 2Types More than one type of garage 416 | Attchd Attached to home 417 | Basment Basement Garage 418 | BuiltIn Built-In (Garage part of house - typically has room above garage) 419 | CarPort Car Port 420 | Detchd Detached from home 421 | NA No Garage 422 | 423 | GarageYrBlt: Year garage was built 424 | 425 | GarageFinish: Interior finish of the garage 426 | 427 | Fin Finished 428 | RFn Rough Finished 429 | Unf Unfinished 430 | NA No Garage 431 | 432 | GarageCars: Size of garage in car capacity 433 | 434 | GarageArea: Size of garage in square feet 435 | 436 | GarageQual: Garage quality 437 | 438 | Ex Excellent 439 | Gd Good 440 | TA Typical/Average 441 | Fa Fair 442 | Po Poor 443 | NA No Garage 444 | 445 | GarageCond: Garage condition 446 | 447 | Ex Excellent 448 | Gd Good 449 | TA Typical/Average 450 | Fa Fair 451 | Po Poor 452 | NA No Garage 453 | 454 | PavedDrive: Paved driveway 455 | 456 | Y Paved 457 | P Partial Pavement 458 | N Dirt/Gravel 459 | 460 | WoodDeckSF: Wood deck area in square feet 461 | 462 | OpenPorchSF: Open porch area in square feet 463 | 464 | EnclosedPorch: Enclosed porch area in square feet 465 | 466 | 3SsnPorch: Three season porch area in square feet 467 | 468 | ScreenPorch: Screen porch area in square feet 469 | 470 | PoolArea: Pool area in square feet 471 | 472 | PoolQC: Pool quality 473 | 474 | Ex Excellent 475 | Gd Good 476 | TA Average/Typical 477 | Fa Fair 478 | NA No Pool 479 | 480 | Fence: Fence quality 481 | 482 | GdPrv Good Privacy 483 | MnPrv Minimum Privacy 484 | GdWo Good Wood 485 | MnWw Minimum Wood/Wire 486 | NA No Fence 487 | 488 | MiscFeature: Miscellaneous feature not covered in other categories 489 | 490 | Elev Elevator 491 | Gar2 2nd Garage (if not described in garage section) 492 | Othr Other 493 | Shed Shed (over 100 SF) 494 | TenC Tennis Court 495 | NA None 496 | 497 | MiscVal: $Value of miscellaneous feature 498 | 499 | MoSold: Month Sold (MM) 500 | 501 | YrSold: Year Sold (YYYY) 502 | 503 | SaleType: Type of sale 504 | 505 | WD Warranty Deed - Conventional 506 | CWD Warranty Deed - Cash 507 | VWD Warranty Deed - VA Loan 508 | New Home just constructed and sold 509 | COD Court Officer Deed/Estate 510 | Con Contract 15% Down payment regular terms 511 | ConLw Contract Low Down payment and low interest 512 | ConLI Contract Low Interest 513 | ConLD Contract Low Down 514 | Oth Other 515 | 516 | SaleCondition: Condition of sale 517 | 518 | Normal Normal Sale 519 | Abnorml Abnormal Sale - trade, foreclosure, short sale 520 | AdjLand Adjoining Land Purchase 521 | Alloca Allocation - two linked properties with separate deeds, typically condo with a garage unit 522 | Family Sale between family members 523 | Partial Home was not completed when last assessed (associated with New Homes) 524 | -------------------------------------------------------------------------------- /LinearRegression/Multi_linear.txt: -------------------------------------------------------------------------------- 1 | 2104,3,399900 2 | 1600,3,329900 3 | 2400,3,369000 4 | 1416,2,232000 5 | 3000,4,539900 6 | 1985,4,299900 7 | 1534,3,314900 8 | 1427,3,198999 9 | 1380,3,212000 10 | 1494,3,242500 11 | 1940,4,239999 12 | 2000,3,347000 13 | 1890,3,329999 14 | 4478,5,699900 15 | 1268,3,259900 16 | 2300,4,449900 17 | 1320,2,299900 18 | 1236,3,199900 19 | 2609,4,499998 20 | 3031,4,599000 21 | 1767,3,252900 22 | 1888,2,255000 23 | 1604,3,242900 24 | 1962,4,259900 25 | 3890,3,573900 26 | 1100,3,249900 27 | 1458,3,464500 28 | 2526,3,469000 29 | 2200,3,475000 30 | 2637,3,299900 31 | 1839,2,349900 32 | 1000,1,169900 33 | 2040,4,314900 34 | 3137,3,579900 35 | 1811,4,285900 36 | 1437,3,249900 37 | 1239,3,229900 38 | 2132,4,345000 39 | 4215,4,549000 40 | 2162,4,287000 41 | 1664,2,368500 42 | 2238,3,329900 43 | 2567,4,314000 44 | 1200,3,299000 45 | 852,2,179900 46 | 1852,4,299900 47 | 1203,3,239500 48 | -------------------------------------------------------------------------------- /LinearRegression/Uni_linear.txt: -------------------------------------------------------------------------------- 1 | 6.1101,17.592 2 | 5.5277,9.1302 3 | 8.5186,13.662 4 | 7.0032,11.854 5 | 5.8598,6.8233 6 | 8.3829,11.886 7 | 7.4764,4.3483 8 | 8.5781,12 9 | 6.4862,6.5987 10 | 5.0546,3.8166 11 | 5.7107,3.2522 12 | 14.164,15.505 13 | 5.734,3.1551 14 | 8.4084,7.2258 15 | 5.6407,0.71618 16 | 5.3794,3.5129 17 | 6.3654,5.3048 18 | 5.1301,0.56077 19 | 6.4296,3.6518 20 | 7.0708,5.3893 21 | 6.1891,3.1386 22 | 20.27,21.767 23 | 5.4901,4.263 24 | 6.3261,5.1875 25 | 5.5649,3.0825 26 | 18.945,22.638 27 | 12.828,13.501 28 | 10.957,7.0467 29 | 13.176,14.692 30 | 22.203,24.147 31 | 5.2524,-1.22 32 | 6.5894,5.9966 33 | 9.2482,12.134 34 | 5.8918,1.8495 35 | 8.2111,6.5426 36 | 7.9334,4.5623 37 | 8.0959,4.1164 38 | 5.6063,3.3928 39 | 12.836,10.117 40 | 6.3534,5.4974 41 | 5.4069,0.55657 42 | 6.8825,3.9115 43 | 11.708,5.3854 44 | 5.7737,2.4406 45 | 7.8247,6.7318 46 | 7.0931,1.0463 47 | 5.0702,5.1337 48 | 5.8014,1.844 49 | 11.7,8.0043 50 | 5.5416,1.0179 51 | 7.5402,6.7504 52 | 5.3077,1.8396 53 | 7.4239,4.2885 54 | 7.6031,4.9981 55 | 6.3328,1.4233 56 | 6.3589,-1.4211 57 | 6.2742,2.4756 58 | 5.6397,4.6042 59 | 9.3102,3.9624 60 | 9.4536,5.4141 61 | 8.8254,5.1694 62 | 5.1793,-0.74279 63 | 21.279,17.929 64 | 14.908,12.054 65 | 18.959,17.054 66 | 7.2182,4.8852 67 | 8.2951,5.7442 68 | 10.236,7.7754 69 | 5.4994,1.0173 70 | 20.341,20.992 71 | 10.136,6.6799 72 | 7.3345,4.0259 73 | 6.0062,1.2784 74 | 7.2259,3.3411 75 | 5.0269,-2.6807 76 | 6.5479,0.29678 77 | 7.5386,3.8845 78 | 5.0365,5.7014 79 | 10.274,6.7526 80 | 5.1077,2.0576 81 | 5.7292,0.47953 82 | 5.1884,0.20421 83 | 6.3557,0.67861 84 | 9.7687,7.5435 85 | 6.5159,5.3436 86 | 8.5172,4.2415 87 | 9.1802,6.7981 88 | 6.002,0.92695 89 | 5.5204,0.152 90 | 5.0594,2.8214 91 | 5.7077,1.8451 92 | 7.6366,4.2959 93 | 5.8707,7.2029 94 | 5.3054,1.9869 95 | 8.2934,0.14454 96 | 13.394,9.0551 97 | 5.4369,0.61705 98 | -------------------------------------------------------------------------------- /LogisticRegression/ML_LogisticRegression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Machine Learning - Andrew Ng ( Python Implementation)\n", 8 | "\n", 9 | "## Logistic Regression" 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "metadata": {}, 15 | "source": [ 16 | "### Loading of Data" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": 1, 22 | "metadata": {}, 23 | "outputs": [], 24 | "source": [ 25 | "import numpy as np\n", 26 | "import matplotlib.pyplot as plt\n", 27 | "import pandas as pd" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": 2, 33 | "metadata": {}, 34 | "outputs": [], 35 | "source": [ 36 | "df=pd.read_csv(\"ex2data1.txt\",header=None)\n", 37 | "X=df.iloc[:,:-1].values\n", 38 | "y=df.iloc[:,-1].values" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "df.head()" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": null, 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "df.describe()" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "### Plotting of Data" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 3, 69 | "metadata": {}, 70 | "outputs": [ 71 | { 72 | "data": { 73 | "text/plain": [ 74 | "" 75 | ] 76 | }, 77 | "execution_count": 3, 78 | "metadata": {}, 79 | "output_type": "execute_result" 80 | }, 81 | { 82 | "data": { 83 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuUFPWZ//H3I6CgoggiiyKC7kTDRQYdTYgoRETM6sq6USN6BI2G6KLGW6LZC17O/rL6098xcnJRDFHMccHIqhiTGAmKGiTqoAg4yqJxwIk4jAgICgGG5/dHVY/N0DP0zHR1VXV/XufM6e6a7q5narrrqe+lnjJ3R0REpLm94g5ARESSSQlCRERyUoIQEZGclCBERCQnJQgREclJCUJERHJSghARkZyUIEREJCclCBERyalz3AF0xMEHH+wDBgyIOwwRkVRZvHjxx+7ee0/PS3WCGDBgANXV1XGHISKSKma2Kp/nqYtJRERyUoIQEZGclCBERCSnyBKEmf3SzNaa2fKsZT3NbJ6ZrQxvDwqXm5lNM7N3zWypmR0XVVwiIpKfKFsQDwFnNFt2MzDf3SuA+eFjgG8AFeHPZODnEcYlIiJ5iCxBuPuLwCfNFo8HZob3ZwL/lLX8YQ/8GehhZn2jik1ERPas2GMQfdx9DUB4e0i4/DDgg6zn1YXLdmNmk82s2syqGxoaIg1WRKScJWWQ2nIsy3ktVHef7u5V7l7Vu/cez/OI1LyaeqbOXc68mvpY4xARiUKxE0R9pusovF0bLq8DDs96Xj/gwyLH1ibzauq5ZtYbPLxoFdfMekNJQkRKTrETxFPApPD+JGBu1vKJ4WymrwIbM11RSfXSyga2bG8EYMv2Rl5aqe4uESktUU5znQUsAo42szozuwy4AxhrZiuBseFjgN8BfwHeBR4A/iWquArl5IredOvSCYBuXTpxckWRurtGjw5+pHzofy4xiawWk7tPaOFXY3I814EpUcUShbGD+jBtwnBeWtnAyRW9GTuoT9whiYgUVKqL9cVt7KA+xUsMmSPIF17Y9fGCBcVZf7mLY3vrfy4xS8osJpHkUJdO22mblSS1INIic9Soo8jiivMoviP/c31OpACUIEQyciWDJUugsjKuiJJP3WAlTQkibfTFK67KymCbx7nja0/LQTtsKQAliDzNq6nXjKVSp268ttM2K2lKEHnInDW9ZXsjj1XXMW3CcCWJcpOWHZ922FJAShB5yHXWtBJECdNOte20zUqSprnmIbazpkXaa8EC7bSlw9SCyIPOmhaRcqQEkaeinjUtIpIA6mISSSudvSwRU4IQEZGc1MUkkjY6GU6KRC0IEYmWusJSSy0IkbTRyXBSJEoQIhINdYWlnhKESFppRysRU4IQkWhkEliPHrs+ltSIZZDazL5nZsvN7C0zuzZc1tPM5pnZyvD2oDhiEylrhRxQzrzXxo3BjwarU6foCcLMhgDfAU4EhgFnmVkFcDMw390rgPnhYxERiUkcXUxfBv7s7p8DmNkLwDnAeGB0+JyZwALgphjiEykv2VfOK+SAsmZbpV4cXUzLgVPMrJeZ7Qv8A3A40Mfd1wCEt4fEEJskhbojRGJX9BaEu79tZncC84DNwJvAjnxfb2aTgckA/fv3jyRGkbLQfBoqwIEHfnGZ1UJRyyG1YpnF5O4zgBkAZvYjoA6oN7O+7r7GzPoCa1t47XRgOkBVVZUXKeTyVswuAs2dlyQq089hLAnCzA5x97Vm1h/4Z2AEMBCYBNwR3s6NIzaRsqExAtmDuM6D+B8z6wVsB6a4+3ozuwP4tZldBqwGzospNsmI42heOy1JkjJv0cbVxXRyjmXrgDExhCNS3lra2ZXZzrBJuf7dOehM6lJTyA93nEfz+nJKEpR5i1YJQkR2Va7dKkn5uxO0vZUgSkWUH+4EfFAjlaAvpCRUmX42lCBEZFfl2q0S99+dlBZMFiWIUhH3hzuNEviFBOKNIynbQBJBCUJEcivXJBHX353AgzwliFKTgA9VaiTtCxlniyaprSmJlRKESDlTQcTkSVBSVoIQScoXMgnnnajlIFmUIETKUUtdSiJZlCBEkibOo3e1HCSLEoRIOVKXkuQhjivKiYhICqgFIVLO1HKQVqgFISIiOSlBiIhITkoQIiKSkxKEiIjkpAQhUiijR+uEMykpShAiIpJTLNNczew64HLAgWXApUBfYDbQE3gduNjdt8URn0ibqBKqlKiityDM7DDgGqDK3YcAnYALgDuBe9y9AlgPXFbs2ERE5AtxnSjXGehmZtuBfYE1wKnAheHvZwK3Aj+PJTqRtlDZCilRRW9BuPtfgbuB1QSJYSOwGNjg7jvCp9UBh+V6vZlNNrNqM6tuaGgoRsgiImUpji6mg4DxwEDgUGA/4Bs5nuq5Xu/u0929yt2revfuHV2gIm21YIFaD1JS4uhiOg14390bAMzsceBrQA8z6xy2IvoBH8YQW6rNq6nnpZUNnFzRm7GD+sQdjsRB3VxSQHFMc10NfNXM9jUzA8YANcDzwLnhcyYBc2OILbXm1dRzzaw3eHjRKq6Z9QbzaurjDqm86BwIKUFFb0G4+ytmNodgKusO4A1gOvBbYLaZ/We4bEaxY+uIuI/eX1rZwJbtjQBs2d7ISysb1IooJ5pqKxGIZRaTu98C3NJs8V+AE2MIp8MyR+9btjfyWHUd0yYML/rO+eSK3jxWXceW7Y1069KJkys0PlMU2jFLCdP1IAogCUfvYwf1YdqE4RqDKFeaapscJfQ/UIIogKQcvY8d1EeJodi0Yy4Mbb9EUoIoAB29S2JoBxufEuxuVIIoEB2955DEL0hUMSXpb0yTEtyplhIliISLe3aUiOSpBLsblSASLAmzo9oliUeFSYyp1LRnm5bgTrWU7PFEOTP7kpnNN7Pl4eNjzezfow8tPebV1DN17vKCn5yWa3aUiCRcCZVcyacF8QDwfeB+AHdfamb/DfxnlIGlRZRH+UmZHdVmSTwqTGJMpaIQrTP9PxIpnwSxr7u/GlTFaLKjpSeXmyjPgdDsKEk1JePUyydBfGxmRxFWVzWzcwnKdAvRH+WnenZUEncMSYwp7Vpqnak2VerlkyCmENRKOsbM/gq8D1wUaVQpoqN8kWY0IaBktJogzGwvgkuDnmZm+wF7ufum4oSWHqk+ypcvaEfWMWo5lJxWE4S77zSzq4Bfu/tnRYpJJBmUMNpHEwJKRj5dTPPM7EbgUaApSbj7J5FFVUJ0olsKtNQlIpJURUq++SSIb4e3U7KWOXBk4cMpLak90a3cLVkS3G7cGNzqSLh9tL1Sb48Jwt0HFiOQUpSEMuCSh5b6zjMtCpGkKPIEgD0mCDPrAlwJnBIuWgDc7+7bI4mohKT2RLdypz50ESC/LqafA12An4WPLw6XXR5VUKVCU2BTRolAkq7IBy/5JIgT3H1Y1uPnzOzNqAIqNYWcAqsB7yJTwpAyl0+CaDSzo9z9PQAzOxJobO8KzexoghlRGUcCU4GHw+UDgFrgfHdf3971lBoNeEvs1OWWHEX6H+yxmitBob7nzWyBmb0APAfc0N4VuvsKd69090rgeOBz4AngZmC+u1cA88PHElJl1wQYPVpTYKWs5DOLab6ZVQBHAwa84+5/K9D6xwDvufsqMxsPjA6XzyQYDL+pQOtJPQ14S8Hl2yIoxdIZpfA3FEE+s5imAI+4+9Lw8UFmdpm7/2wPL83HBcCs8H4fd18D4O5rzOyQArx/ySi5Ae80fUFLcQcpkod8xiC+4+4/zTxw9/Vm9h2+mNXULma2N3A28MM2vm4yMBmgf//+HQkhdVTzSQqirQkv18yZTHdb2pJk0pN9wuLJJ0HsZWbm7ply352AvQuw7m8Ar7t75jJs9WbWN2w99AXW5nqRu08nqC5LVVWVFyAOKaakf0Fz0XkRkgQxfP7ySRB/AH5tZvcRlNi4AnimAOuewBfdSwBPAZOAO8LbuQVYh4g0196El91ySFOCz5bUZJ/QA6d8EsRNBF06VxIMUj8L/KIjKzWzfYGxwHezFt9BkIguA1YD53VkHZJQSf2C5iNNsUrpiDF55DOLaSdwH3CfmfUE+rl7u8+DCN/zc6BXs2XrCGY1iUgxtGcHk+YEny1pcSd0u+Yzi2kBwWByZ2AJ0GBmL7j79RHHJqUsIV8AkcSLMXnk08V0oLt/amaXAw+6+y1mtjTqwEQkwZTgo5Gw7ZpPgugczio6H/i3iOMRKS8J61KQBIvhM5JPqY3bCWYyvevur4W1mFZGG5ZEbV5NPVPnLmdeTf2enywiZcnC0xtSqaqqyqurq+MOI3WyC/9169JJhf/i0HxmyqhRwa1aEvEos5acmS1296o9PS+fFoSUGBX+E4lYiRR2zGcMQkqMCv8lQEKnNZadhJ6glhRKEAkU9YWBUl34T19gSbISSzitJggzOwY4DHjF3TdnLT/D3QtRbkOaKdaFgVT4LyFSuuMoGWrJtarFBGFm1wBTgLeBGWb2PXfP1Ef6EYWpxyTN5Bof0I6ckjsykxJVYgmntRbEd4Dj3X2zmQ0A5pjZAHe/l6Amk0RA4wMiMUj5jjwqrSWITpluJXevNbPRBEniCJQgIpPq8YEoldiRmZS4Evl8tpYgPjKzSndfAhC2JM4CfgkMLUp0ZUrjAyKSBK0liInAjuwF7r4DmGhm90calZS8ds/UKpEjM5E0aDFBuHtdK79bGE04Ug6KNVNLRDpGZ1JL0elMbpF0UIKQoju5ojfdunQC0EwtkQTL+0xqMzsg+/nu/kkkEUnJ00wtkXTI54py3yUo+b0FyJR+deDICOOSElf2M7U0XVfaKqFXlLsRGOzuH0cdjJSXqGtOiUjH5JMg3gM+L+RKzawH8AtgCEFr5NvACuBRYABQC5zv7usLud4oaWfXNmU7k0klQ6StYvzM5DNI/UPgZTO738ymZX46uN57gWfc/RhgGEG9p5uB+e5eAcwPH6dCZmf38KJVXDPrDV2lLQ+aySSSfPm0IO4HngOWATs7usJwsPsU4BIAd98GbDOz8cDo8GkzgQXATR1dXzGowF6gLa2oSGtOJfmoXCVDpK1i/MzkkyB2uPv1BVznkUAD8KCZDQMWA98D+rj7GgB3X2NmhxRwnZFSgb22dxlpJpNI8u3xmtRm9n+AVcBvgL9llrd3mquZVQF/Bk5y91fM7F7gU+Bqd++R9bz17n5QjtdPBiYD9O/f//hVq1a1J4yCK/cxiKlzl/Pwoi/+FxNHHMHt44cUNwhd51kkL/lekzqfFsSF4e0Ps5Z1ZJprHVDn7q+Ej+cQjDfUm1nfsPXQF1ib68XuPh2YDlBVVdV6diuicp+2qVaUSOnZY4Jw94GFXKG7f2RmH5jZ0e6+AhgD1IQ/k4A7wtu5rbyNJEwiuozUvy9SUHmdSW1mQ4BBQNfMMnd/uAPrvRp4xMz2Bv4CXEowo+rXZnYZsBo4rwPvLzEo91aUSKnJ50zqWwhmFw0Cfgd8A/gT0O4EEV5jIlf/15j2vqdIE7UcRAoin/MgziXYcX/k7pcSnLewT6RRiYjsyejRX3QnSiTySRBb3H0nsCM8h2EtqsMkIlLy8hmDqA5LYzxAcM7CZuDVSKMSEYHcEw5UrqRo8pnF9C/h3fvM7BngAHdfGm1YUizlfv6GiLQsn0Hqy9x9BoC715pZJzO7xd1viz48iVLZFsyT5GutlaDpzEWTzxjEGDP7nZn1Dae7/hnoHnFcUgQqmCcircmni+lCM/sWQbG+z4EJ7r4w8sgkcjr7WRIrn1aCWg6Ry6eLqYKgmN7/AF8GLjazN9y9oNeIkOJLxNnPIpJY+cxi+g0wxd3nm5kB1wOvAYMjjUyKQmc/S6KplRCrfBLEie7+KYAHpV//n5k9FW1YIiIStxYHqc3sBwDu/qmZNa+LdGmkUYmISOxam8V0Qdb9Hzb73RkRxCIiIgnSWoKwFu7neiwiUVC9IYlRawnCW7if67GIiJSY1gaph5nZpwSthW7hfcLHXVt+mYh0mOoNSQK0mCDcvVMxAxFpSRLqRSUhBpFiy+uKciKtiXLnmYR6UbHEoHpDkgD51GISaVFm5/nwolVcM+sN5tXUF/T9k1AvKgkxiMRBCUI6JOqd58kVvenWJejtjKteVKwxZFcvFSkydTFJh0Rd8C8J9aKSEINIHCyonlHklZrVApuARmCHu1eZWU/gUWAAUAuc7+7rW3ufqqoqr66ujjZY2aNSH8At9b9Pyo+ZLXb3qj0+L8YEUeXuH2ct+7/AJ+5+h5ndDBzk7je19j5KEBK17AHqbl066aJKUhLyTRBJGoMYD8wM788E/inqFc6rqWfq3OUFH1iV0tm2GqCWchZXgnDgWTNbbGaTw2V93H0NQHh7SK4XmtlkM6s2s+qGhvZ/WaOefVPOSmnbJmGQXCQucQ1Sn+TuH5rZIcA8M3sn3xe6+3RgOgRdTO0NINeRoboOCqP5tr37D8G/N43bVwPUUs5iaUG4+4fh7VrgCeBEoN7M+gKEt2ujjEFHhtHJ3rYAK+o3p7olMXZQH24fP0TJodhUqDB2RU8QZrafmXXP3AdOB5YDTwGTwqdNAuZGGUfmyHDiiCM08FhgmW17dJ/9m5ap/14kfeLoYuoDPBFcvZTOwH+7+zNm9hrwazO7DFgNNL9IUcHpcpvRyWzX7BlAaqWlWDFLfqhQYWIUPUG4+1+AYTmWrwPGFDseiY7670XSTWdSS6TUSku5OI7mVagwMZQgIqAzb1umbZNQ2hlLDkoQBZaE8tRJpW3TukQmzziP5pWsYpekM6lLgs68bZm2TctiO7kwM5X0hReCH00tlSxKEAWm8ytym1dTzweffM7enYKPnLbNrhKfPEu97HhUiTHlCVddTAWmmTu7y+5a2rvTXnz96N5c+JUjtG2yRF02vUUaEJZWKEFEQDN3dpV9dLytcSeH99xX26eZkj2wSHriiWqWVomcy6EEIZGL7eg4ZWI9sEjZjkuKI5brQRSKrgeRHomcoSPRaX4EPWpUcJvURBTVEX5CWw75Xg9CLYgC0M5vz9TtJpI+Zd+C6OjOXVccE2lFIY+gi3U0ntCj/kJK4xXliq4tc89bukJa4qcnioi0U1l3MeV70aDWzgDWAKxIKwrZcoh6RlCJzDwqpLJuQeR7UltrrQRdV0I6qlSu391e5f73J5nGIPIYg9A4Q3mIY7JBuX+22vT3awyiYDSLKU/5zK4p2ZOYpElchQTL/dro5f73J13ZJ4h8aZpmaYtrR5WEMaw4p2m36e8v1hF9Cbcc2koJQoT4dtRxt07jLsEe998vrVOCEKG4O6rmR+xxtk6T0MWj1nlyxZYgzKwTUA381d3PMrOBwGygJ/A6cLG7b4srPik/xdhRxX3E3lwSurhA1QiSKs5prt8D3s56fCdwj7tXAOuBy2KJSiRCSTuxMgnTtGO7WJLsUSwJwsz6AWcCvwgfG3AqMCd8ykzgn+KITSRKSbyg1NhBfbh9/JBEdXNJMsTVxfRj4AdA9/BxL2CDu+8IH9cBh8URmEiUNCi7u6R0c8nuip4gzOwsYK27Lzaz0ZnFOZ6a8ww+M5sMTAbo37//br/fvn07dXV1bN26tTABS4d17dqVfv360aVLl4K/dxr7rjUouyslzeQq+pnUZvZfwMXADqArcADwBDAO+Dt332FmI4Bb3X1ca++V60zq999/n+7du9OrVy+CniuJk7uzbt06Nm3axMCBAwv63uV+FrLEL40HKJDgaq7u/kN37+fuA4ALgOfc/SLgeeDc8GmTgLntef+tW7cqOSSImdGrV69IWnTqu5Y4lcPgepKK9d0EXG9m7xKMScxo7xspOSRLVP+PJA74SvlozwFK2goTxpog3H2Bu58V3v+Lu5/o7n/v7ue5+9/ijK2jnnjiCcyMd955J+fvL7nkEubMmZPzd7l8+OGHnHtu0MBasmQJv/vd75p+t2DBAl5++eU2xzhgwAA+/vjjNr8uKZIwRVPKV1sPUNLY4khSC6KkzJo1i5EjRzJ79uyCvN+hhx7alFAKlSBKQdxTNKV8tfUAJY1dokoQEJT3zZT4LYDNmzezcOFCZsyY0ZQg3J2rrrqKQYMGceaZZ7J27dqm5w8YMIB//dd/ZcSIEVRVVfH6668zbtw4jjrqKO677z4AamtrGTJkCNu2bWPq1Kk8+uijVFZWcuedd3Lfffdxzz33UFlZyUsvvURDQwPf/OY3OeGEEzjhhBNYuHAhAOvWreP0009n+PDhfPe73yXNpd5FkqAtByhp7BJVLaYIPPnkk5xxxhl86UtfomfPnrz++uvU1tayYsUKli1bRn19PYMGDeLb3/5202sOP/xwFi1axHXXXccll1zCwoUL2bp1K4MHD+aKK65oet7ee+/N7bffTnV1NT/5yU8A2LJlC/vvvz833ngjABdeeCHXXXcdI0eOZPXq1YwbN463336b2267jZEjRzJ16lR++9vfMn369OJuGJEylsbpvOWdICK6xOCsWbO49tprAbjggguYNWsW27dvZ8KECXTq1IlDDz2UU089dZfXnH322QAMHTqUzZs30717d7p3707Xrl3ZsGFDm9b/xz/+kZqamqbHn376KZs2beLFF1/k8ccfB+DMM8/koIMO6sifKSJtlLZzYMo7QURg3bp1PPfccyxfvhwzo7GxETPjnHPOaXU2zz777APAXnvt1XQ/83jHjh0tvSynnTt3smjRIrp167bb7zTDS0TyVd5jEAsWBD+jRgU/mccdMGfOHCZOnMiqVauora3lgw8+YODAgfTs2ZPZs2fT2NjImjVreP7559u9ju7du7Np06YWH59++ulN3U8QDGoDnHLKKTzyyCMA/P73v2f9+vXtjiFOaZsqKJJW5Z0gIjBr1izOOeecXZZ985vf5KOPPqKiooKhQ4dy5ZVXMmrUqHav4+tf/zo1NTVUVlby6KOP8o//+I888cQTTYPU06ZNo7q6mmOPPZZBgwY1DXTfcsstvPjiixx33HE8++yzOUuVJF0apwqKpFXRS20UUq5SG2+//TZf/vKXY4pIWlKo/8vUuct5eNGqpscTRxzB7eOHdPh9RcpJYkttiHREGqcKiqSVBqklVdI4VVAkrZQgJHXSNlVQJK3UxSQiIjkpQYiISE5KECIikpMSRATMjBtuuKHp8d13382tt97a6muefPLJXcpjtEdby3c/9dRT3HHHHTnX/9BDD/Hhhx+2af2ZgoIiUhqUICKwzz778Pjjj7dpZ12IBNFWZ599NjfffHPO9bcnQYhIaVGCiEDnzp2ZPHky99xzz26/W7VqFWPGjOHYY49lzJgxrF69mpdffpmnnnqK73//+1RWVvLee+/t8prf/OY3fOUrX2H48OGcdtpp1NcHZw+3VL67traWY445hssvv5whQ4Zw0UUX8cc//pGTTjqJiooKXn31VSBIAlddddVu67/zzjuprq7moosuorKyki1btrB48WJGjRrF8ccfz7hx41izZg0AixcvZtiwYYwYMYKf/vSnUW5WESk2d0/tz/HHH+/N1dTU7LZsT5596yP/jyeX+bNvfdTm1+ay3377+caNG/2II47wDRs2+F133eW33HKLu7ufddZZ/tBDD7m7+4wZM3z8+PHu7j5p0iR/7LHHcr7fJ5984jt37nR39wceeMCvv/56d3e/+uqr/bbbbnN396efftoBb2ho8Pfff987derkS5cu9cbGRj/uuOP80ksv9Z07d/qTTz7ZtM4HH3zQp0yZknP9o0aN8tdee83d3bdt2+YjRozwtWvXurv77Nmz/dJLL3V396FDh/qCBQvc3f3GG2/0wYMH5/wb2vN/EZFoANWexz627M+DyNT22bK9kceq6wp26coDDjiAiRMnMm3atF2qqi5atKip5PbFF1/MD37wgz2+V11dHd/61rdYs2YN27ZtY+DAgQCtlu8eOHAgQ4cOBWDw4MGMGTMGM2Po0KHU1ta26W9ZsWIFy5cvZ+zYsQA0NjbSt29fNm7cyIYNG5rqSl188cX8/ve/b9N7i0hylX0XU5SXAbz22muZMWMGn332WYvPyaf89tVXX81VV13FsmXLuP/++9m6deseX9+8ZHh2OfG2lg93dwYPHsySJUtYsmQJy5Yt49lnn8XdVT5cpIQVPUGYWVcze9XM3jSzt8zstnD5QDN7xcxWmtmjZrZ3MeKJsrZPz549Of/885kxY0bTsq997WtNlyF95JFHGDlyJLB7ye5sGzdu5LDDDgNg5syZTcsLWb67tRLiRx99NA0NDSxatAiA7du389Zbb9GjRw8OPPBA/vSnPzX9PZJOKqEuucTRgvgbcKq7DwMqgTPM7KvAncA97l4BrAcuK0Ywbb3weFvdcMMNu8xmmjZtGg8++CDHHnssv/rVr7j33nuB4Mpzd911F8OHD99tkPrWW2/lvPPO4+STT+bggw9uWl7I8t3N13/JJZdwxRVXUFlZSWNjI3PmzOGmm25i2LBhVFZW8vLLLwPw4IMPMmXKFEaMGJHzAkWSfCqhLi2Jtdy3me0L/Am4Evgt8HfuvsPMRgC3uvu41l6vct/pof9LcqmEevlJdLlvM+tkZkuAtcA84D1gg7tnOsfrgMNaeO1kM6s2s+qGhsKNF4iUK5VQl5bEMovJ3RuBSjPrATwB5Dq0zNm0cffpwHQIWhCRBSlSJlRCXVoS6zRXd99gZguArwI9zKxz2IroB+g0XpEiUQl1ySWOWUy9w5YDZtYNOA14G3geODd82iRgbnvXEee4iuxO/w+RdIpjDKIv8LyZLQVeA+a5+9PATcD1ZvYu0AuY0cp7tKhr166sW7dOO6WEcHfWrVtH165d4w5FRNqo6F1M7r4UGJ5j+V+AEzv6/v369aOurg4NYCdH165d6devX9xhiEgblVypjS5dujSVohARkfYr+1IbIiKSmxKEiIjkpAQhIiI5xVpqo6PMrAFYtccntuxgIP/LvsUvTfGmKVZIV7xpihXSFW+5xHqEu+/xlPlUJ4iOMrPqfOqRJEWa4k1TrJCueNMUK6QrXsW6K3UxiYhITkoQIiKSU7kniOlxB9BGaYo3TbFCuuJNU6yQrngVa5ayHoMQEZGWlXsLQkREWlA2CSJp18LOR3hpKwS4AAAGnklEQVRhpTfM7OnwcZJjrTWzZWa2xMyqw2U9zWxeGO88Mzso7jgBzKyHmc0xs3fM7G0zG5HgWI8Ot2nm51MzuzbB8V4Xfr+Wm9ms8HuXyM+tmX0vjPMtM7s2XJaY7WpmvzSztWa2PGtZzvgsMM3M3jWzpWZ2XCFiKJsEQcKuhZ2n7xGUQs9IcqwAX3f3yqypdzcD88N454ePk+Be4Bl3PwYYRrCNExmru68It2klcDzwOcFFthIXr5kdBlwDVLn7EKATcAEJ/Nya2RDgOwQFQocBZ5lZBcnarg8BZzRb1lJ83wAqwp/JwM8LEoG7l90PsC/wOvAVghNNOofLRwB/iDu+MJZ+4QfgVOBpwJIaaxhPLXBws2UrgL7h/b7AigTEeQDwPuH4W5JjzRH76cDCpMZLcJngD4CeBIVAnwbGJfFzC5wH/CLr8X8AP0jadgUGAMuzHueMD7gfmJDreR35KacWRIeuhR2DHxN8YHeGj3uR3FghuETss2a22Mwmh8v6uPsagPD2kNii+8KRQAPwYNh99wsz249kxtrcBcCs8H7i4nX3vwJ3A6uBNcBGYDHJ/NwuB04xs15mti/wD8DhJHC7NtNSfJnknFGQ7VxWCcLdGz1oqvcjaFrmfS3sYjKzs4C17r44e3GOp8Yea5aT3P04gqbuFDM7Je6AWtAZOA74ubsPBz4jAd0zexL2258NPBZ3LC0J+8PHAwOBQ4H9CD4PzcX+uXX3twm6vuYBzwBvAjtafVGyRbJ/KKsEkeHuG4AFZF0LO/xVUq6FfRJwtpnVArMJupl+TDJjBcDdPwxv1xL0kZ8I1JtZX4Dwdm18ETapA+rc/ZXw8RyChJHEWLN9A3jd3evDx0mM9zTgfXdvcPftwOPA10jo59bdZ7j7ce5+CvAJsJJkbtdsLcVXR9ACyijIdi6bBGFFuBZ2obj7D929n7sPIOhWeM7dLyKBsQKY2X5m1j1zn6CvfDnwFEGckJB43f0j4AMzOzpcNAaoIYGxNjOBL7qXIJnxrga+amb7mpnxxbZN6uf2kPC2P/DPBNs3ids1W0vxPQVMDGczfRXYmOmK6pC4B4uKONhzLPAGsJRg5zU1XH4k8CrwLkHzfZ+4Y20W92jg6STHGsb1ZvjzFvBv4fJeBAPtK8PbnnHHGsZVCVSHn4UngYOSGmsY777AOuDArGWJjBe4DXgn/I79CtgnwZ/blwgS2JvAmKRtV4KEtQbYTtBCuKyl+Ai6mH5KMK66jGAmWYdj0JnUIiKSU9l0MYmISNsoQYiISE5KECIikpMShIiI5KQEISIiOSlBSEkys8ZmVVCLdrZ0riqcImmkaa5Sksxss7vvH9O6TwE2Aw97UNW0GOvs5O6NxViXlA+1IKRsmNmBZrYicxZ1eL2C74T3f25m1ZZ1rZBwea2Z/cjMFoW/P87M/mBm75nZFbnW4+4vEpRuaC2W88JrEbxpZi+GyzqZ2d0WXFdjqZldHS4fExYWXBa2TvbJim2qmf0JOM/MjjKzZ8KCiS+Z2TGF2G5Svjrv+SkiqdQtrNyb8V/u/qiZXQU8ZGb3Age5+wPh7//N3T8xs07AfDM71t2Xhr/7wN1HmNk9BDX6TwK6Epw1fl8745sKjHP3v2ZKwBDU8R8IDHf3HeHFYbqG6xzj7v9rZg8DVxLU5gLY6u4jAcxsPnCFu680s68APyOo4yXSLkoQUqq2eFC5dxfuPs/MziMoSzAs61fnh2XKOxPU2R9EUIoDgjo3EJQw2N/dNwGbzGyrmfXwoPhjWy0kSFS/JihqB0F9sPs8LI0dJqxhBAXw/jd8zkxgCl8kiEcBzGx/gsJ4jwVlkICgzIVIuylBSFkxs70IyrxvIbiwTZ2ZDQRuBE5w9/Vm9hBBCyHjb+Htzqz7mcft+g65+xXhUf6ZwBIzqySop9N8UDBXGedsn4W3exFcd2G3pCjSXhqDkHJzHUEV3wnAL82sC8FV5j4DNppZH3Jfw6CgzOwod3/F3acSXHHtcOBZ4IpMaWwz60lQ+G6Amf19+NKLgReav5+7fwq8H7aOMtcoHtb8eSJtoQQhpapbs2mud5jZl4DLgRvc/SXgReDf3f1Ngkq/bwG/JOj+aTczmwUsAo42szozy3UN5rvCQeflYRxvAr8gKJm91MzeBC50963ApQRdR8sIWi0tjXtcBFwWvvYtgov3iLSbprmKiEhOakGIiEhOShAiIpKTEoSIiOSkBCEiIjkpQYiISE5KECIikpMShIiI5KQEISIiOf1/GVxhSC+htKEAAAAASUVORK5CYII=\n", 84 | "text/plain": [ 85 | "
" 86 | ] 87 | }, 88 | "metadata": {}, 89 | "output_type": "display_data" 90 | } 91 | ], 92 | "source": [ 93 | "pos , neg = (y==1).reshape(100,1) , (y==0).reshape(100,1)\n", 94 | "plt.scatter(X[pos[:,0],0],X[pos[:,0],1],c=\"r\",marker=\"+\")\n", 95 | "plt.scatter(X[neg[:,0],0],X[neg[:,0],1],marker=\"o\",s=10)\n", 96 | "plt.xlabel(\"Exam 1 score\")\n", 97 | "plt.ylabel(\"Exam 2 score\")\n", 98 | "plt.legend([\"Admitted\",\"Not admitted\"],loc=0)" 99 | ] 100 | }, 101 | { 102 | "cell_type": "markdown", 103 | "metadata": {}, 104 | "source": [ 105 | "### Sigmoid function\n", 106 | "\n", 107 | "$ g(z) = \\frac{1}{(1+e^{-z})}$" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": 4, 113 | "metadata": {}, 114 | "outputs": [], 115 | "source": [ 116 | "def sigmoid(z):\n", 117 | " \"\"\"\n", 118 | " return the sigmoid of z\n", 119 | " \"\"\"\n", 120 | " \n", 121 | " return 1/ (1 + np.exp(-z))" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": 5, 127 | "metadata": {}, 128 | "outputs": [ 129 | { 130 | "data": { 131 | "text/plain": [ 132 | "0.5" 133 | ] 134 | }, 135 | "execution_count": 5, 136 | "metadata": {}, 137 | "output_type": "execute_result" 138 | } 139 | ], 140 | "source": [ 141 | "# testing the sigmoid function\n", 142 | "sigmoid(0)" 143 | ] 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "metadata": {}, 148 | "source": [ 149 | "### Compute the Cost Function and Gradient\n", 150 | "\n", 151 | "$J(\\Theta) = \\frac{1}{m} \\sum_{i=1}^{m} [ -y^{(i)}log(h_{\\Theta}(x^{(i)})) - (1 - y^{(i)})log(1 - (h_{\\Theta}(x^{(i)}))]$\n", 152 | "\n", 153 | "$ \\frac{\\partial J(\\Theta)}{\\partial \\Theta_j} = \\frac{1}{m} \\sum_{i=1}^{m} (h_{\\Theta}(x^{(i)}) - y^{(i)})x_j^{(i)}$" 154 | ] 155 | }, 156 | { 157 | "cell_type": "code", 158 | "execution_count": 6, 159 | "metadata": {}, 160 | "outputs": [], 161 | "source": [ 162 | "def costFunction(theta, X, y):\n", 163 | " \"\"\"\n", 164 | " Takes in numpy array theta, x and y and return the logistic regression cost function and gradient\n", 165 | " \"\"\"\n", 166 | " \n", 167 | " m=len(y)\n", 168 | " \n", 169 | " predictions = sigmoid(np.dot(X,theta))\n", 170 | " error = (-y * np.log(predictions)) - ((1-y)*np.log(1-predictions))\n", 171 | "\n", 172 | " cost = 1/m * sum(error)\n", 173 | " \n", 174 | " grad = 1/m * np.dot(X.transpose(),(predictions - y))\n", 175 | " \n", 176 | " return cost[0] , grad" 177 | ] 178 | }, 179 | { 180 | "cell_type": "markdown", 181 | "metadata": {}, 182 | "source": [ 183 | "### Feature scaling" 184 | ] 185 | }, 186 | { 187 | "cell_type": "code", 188 | "execution_count": 7, 189 | "metadata": {}, 190 | "outputs": [], 191 | "source": [ 192 | "def featureNormalization(X):\n", 193 | " \"\"\"\n", 194 | " Take in numpy array of X values and return normalize X values,\n", 195 | " the mean and standard deviation of each feature\n", 196 | " \"\"\"\n", 197 | " mean=np.mean(X,axis=0)\n", 198 | " std=np.std(X,axis=0)\n", 199 | " \n", 200 | " X_norm = (X - mean)/std\n", 201 | " \n", 202 | " return X_norm , mean , std" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": 8, 208 | "metadata": {}, 209 | "outputs": [ 210 | { 211 | "name": "stdout", 212 | "output_type": "stream", 213 | "text": [ 214 | "Cost of initial theta is 0.693147180559946\n", 215 | "Gradient at initial theta (zeros): [[-0.1 ]\n", 216 | " [-0.28122914]\n", 217 | " [-0.25098615]]\n" 218 | ] 219 | } 220 | ], 221 | "source": [ 222 | "m , n = X.shape[0], X.shape[1]\n", 223 | "X, X_mean, X_std = featureNormalization(X)\n", 224 | "X= np.append(np.ones((m,1)),X,axis=1)\n", 225 | "y=y.reshape(m,1)\n", 226 | "initial_theta = np.zeros((n+1,1))\n", 227 | "cost, grad= costFunction(initial_theta,X,y)\n", 228 | "print(\"Cost of initial theta is\",cost)\n", 229 | "print(\"Gradient at initial theta (zeros):\",grad)" 230 | ] 231 | }, 232 | { 233 | "cell_type": "markdown", 234 | "metadata": {}, 235 | "source": [ 236 | "### Gradient Descent" 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "execution_count": 9, 242 | "metadata": {}, 243 | "outputs": [], 244 | "source": [ 245 | "def gradientDescent(X,y,theta,alpha,num_iters):\n", 246 | " \"\"\"\n", 247 | " Take in numpy array X, y and theta and update theta by taking num_iters gradient steps\n", 248 | " with learning rate of alpha\n", 249 | " \n", 250 | " return theta and the list of the cost of theta during each iteration\n", 251 | " \"\"\"\n", 252 | " \n", 253 | " m=len(y)\n", 254 | " J_history =[]\n", 255 | " \n", 256 | " for i in range(num_iters):\n", 257 | " cost, grad = costFunction(theta,X,y)\n", 258 | " theta = theta - (alpha * grad)\n", 259 | " J_history.append(cost)\n", 260 | " \n", 261 | " return theta , J_history" 262 | ] 263 | }, 264 | { 265 | "cell_type": "code", 266 | "execution_count": 10, 267 | "metadata": {}, 268 | "outputs": [], 269 | "source": [ 270 | "theta , J_history = gradientDescent(X,y,initial_theta,1,400)" 271 | ] 272 | }, 273 | { 274 | "cell_type": "code", 275 | "execution_count": 11, 276 | "metadata": {}, 277 | "outputs": [ 278 | { 279 | "name": "stdout", 280 | "output_type": "stream", 281 | "text": [ 282 | "Theta optimized by gradient descent: [[1.65947664]\n", 283 | " [3.8670477 ]\n", 284 | " [3.60347302]]\n", 285 | "The cost of the optimized theta: 0.20360044248226664\n" 286 | ] 287 | } 288 | ], 289 | "source": [ 290 | "print(\"Theta optimized by gradient descent:\",theta)\n", 291 | "print(\"The cost of the optimized theta:\",J_history[-1])" 292 | ] 293 | }, 294 | { 295 | "cell_type": "markdown", 296 | "metadata": {}, 297 | "source": [ 298 | "### Plotting of Cost Function" 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "execution_count": 10, 304 | "metadata": {}, 305 | "outputs": [ 306 | { 307 | "data": { 308 | "text/plain": [ 309 | "Text(0.5,1,'Cost function using Gradient Descent')" 310 | ] 311 | }, 312 | "execution_count": 10, 313 | "metadata": {}, 314 | "output_type": "execute_result" 315 | }, 316 | { 317 | "data": { 318 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEWCAYAAACT7WsrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm8XHV9//HXe+bu2ZcLZAESJCq4AQbUulEBCyigv1rFrfpzQVGqVaviUn8WW0tdUNtCFa1K3ZBiVYqpYBVEQSABIRJCMIQgkUAukJ3k5i6f3x/nO8lkMjP3Jtw5c8O8n4/HPGbOOd855zNn5s77nmW+RxGBmZlZNYVmF2BmZuOXQ8LMzGpySJiZWU0OCTMzq8khYWZmNTkkzMysJoeE1STpbEkPSdoiaUaOy/2opK/ltbyRSHq9pKubXcfjIWm1pBPT43G1fm18c0jsByS9TtKS9GW9VtL/SHrB45znzi+NGtPbgQuAl0bExIh45PEsr85yjpe0pnxcRHw6It7WiOXti4j4TkS8tFHzl3SmpJskbZW0Lj1+lyQ1YnljtX4lzZMUktrqtPmkpAFJm9Ptbkn/KmnW411+o6TXdHiz6xgvHBLjnKT3A18EPg0cCBwCXASc0eBFHwh0AcsavJyWJukDwJeAzwIHka33dwLPBzpqPKeYW4Fj4/sRMQmYDryS7HXeMp6DwspEhG/j9AZMAbYAf1GnTSdZiDyQbl8EOtO0mcCVwAbgUeBXZP8YfAsYBral+X+oYp5PBrYCkab/ApiXhtvK2l0LvC09fjPwa+BzwHrgXuCUsrbTgW+kGtcDPwImpBqG03K2ALOBTwLfLnvu6WRhtSEt84iyaauBvwGWAhuB7wNdNdZV5Xx3e03pNawCNqf6X1/+2sqeF2Rf5L9Pr+VCQGlaEfg88HCaxzmV663i/d0K/PkIn4NvAv8GLErtTwReBvwW2ATcD3yy4jlvBO4DHgE+ltbTiTXWw3OBG9L6vR04vuI9/hRwfVovVwMz07Q/lH1GtgDPG2mdl62j24HPlY17OXBbquEG4Jll0z4M/DEtfwVwQtl8Pgrck6bdAhycpj0V+BnZ534F8OqK9Xkh8JP0vJuAJ6Vp16XXtDW9ptc0+3ug2bemF+BbnTcHTgYGq33BlLU5D7gROADoTX9gn0rT/hH4MtCebi8s+zLb+aVRY77z2P0LdLfhNO5adg+JAeDt6Y/3bLJAKC3vJ2Rf4NNSLS9O448H1lQse+cXC7sC66T0vA8BK4GOstdxM1m4TAeWA++s8Zp2+8Iqf01kgbUJeEqaNgt4WtlrqwyJK4GpZFt2fcDJado7gTuBuem1/m/letub9ze1+yZZAD6fLOS70np7Rhp+JvAQ8IrU/kiyL7gXkf0TcUFazh4hAcwhC5JT07xOSsO9Ze/xPel96E7D59f6TIy0zis+tzelx8cA64DnkH123pTe107gKWQhOLtsmaUv9A8Cv0ttBDwLmJHey/uB/5ve22PIQvtpZevzUeC4NP07wKUV7+/hzf77Hy83724a32YAD0fEYJ02rwfOi4h1EdEH/B3Zf5GQfWnPAg6NiIGI+FWkv4IGuS8ivhoRQ8AladkHpt0Kp5B9ea9PtfxylPN8DfCTiPhZRAyQbal0A39S1uafI+KBiHgU+G/gqH2sfxh4uqTuiFgbEfV2tZ0fERsi4g/ANWXLfDXwpYhYExHrgfPrzGMmFe+vpBskbZC0TdKLytr+OCKuj4jhiNgeEddGxO/S8FLge8CLU9tXAVdGxHUR0Q/8bXpt1bwBWBQRi9K8fgYsIQuNkm9ExN0RsQ24jH1fv+UeIAt1yP6x+EpE3BQRQxFxCdBPtoUzRBYWR0pqj4jVEXFPet7bgI9HxIrI3B7ZsbOXA6sj4hsRMRgRtwI/SOul5L8i4ua07r8zRq/pCckhMb49Asysd2CQ7D/o+8qG70vjINvPvRK4WtIqSec2psydHiw9iIjH0sOJwMHAo+lLc2/t9voiYpjsv8Q51ZYLPJaWuVciYitZIL0TWCvpJ5KeWucptZY5O9VXUv640h7vb0T8SURMTdPK/z53m4+k50i6RlKfpI2p7pnVakivrdaJB4cCf5GCaYOkDcALyAJ+pNf6eMwh+2++VMMHKmo4mGzrYSXw12RbJOskXSqp9Pk+mGwrp9prek7F/F5Pdiykka/pCckhMb79BtgOvKJOmwfI/ihKDknjiIjNEfGBiDgMOA14v6QTUru93aLYmu57ysYdVK1hFfcD0yVNrTJtpDp2e33pjJ+DyfZR762t1Kk/Iq6KiJPIviDvAr66D8tYS7arqeTgOm1/Q/Yf82hOQqhcT98FriDbBz+FbLdi6WyoteXLldRDtlVazf3AtyJiatltQkTU2wKqVdOoSCqQfR5/VVbDP1TU0BMR3wOIiO9GxAvIPgcB/FPZ855U4zX9smJ+EyPi7H2pt9U5JMaxiNgIfAK4UNIrJPVIapd0iqTPpGbfAz4uqVfSzNT+2wCSXi7p8PTFuols030oPe8h4LC9qKWP7Iv5DZKKkt5C9T/Qas9dC/wPcJGkaek1lHalPATMkDSlxtMvA14m6YR0Wu4HyL5Ybxht7WVuA14k6ZC0vI+UJkg6UNLpkiak+W9h17raG5cB75U0J4Xih2s1jIgNZLsHL5L0KkkTJRUkHUW2X72eSWRbZ9slHQe8rmza5cDLJb1AUgfZ/v9af+vfBk6T9Gfpfe1KpyXPrdG+XB/ZbqxRfY7S+34E2Wf2ILJjJZCF8TvT1pEkTZD0MkmTJD1F0kskdZL9w7SNXe/L14BPSVqQnvdMZb/nuRJ4sqQ3pmW2Szo2LXs09upv44nOITHORcQFwPuBj5P9Ud5PdsbMj1KTvyfbh7yU7CDerWkcwAKyA6dbyP5rvSgirk3T/pEsXDZI+ptRlvN2soOFjwBPY+++qN9IdozkLrKDlH+dXt9dZF8aq1Its8ufFBEryPab/wvZwcfTgNMiYsdeLLs0r5+RHTxfSnYmzJVlkwtkAfQA2W6QFwPv2ttlkH3hXZ2W8VuyM5IGqRE4EfEZsvf3Q2Tr5SHgK2ThUm/9vgs4T9Jmsn8MLiub5zLg3WRbG2vJzsBaU20mEXE/2ZbMR9n1+fogo/huSLsU/wG4Pr13z63R9DWStpCduXQF2efn2RFR2uJdQvbZ+tdU60qykwUgOx5xPtl7/yDZCRofTdMuSK/7arJ/gv4d6I6IzcBLgTPJ3s8HybY+Okd6TckngUvSa3r1KJ/zhFU688TMGkDSKcCXI+LQERubjUPekjAbQ5K6JZ0qqU3SHOD/AT9sdl1m+8pbEmZjKB0k/iXZj7m2kf0+5L0RsamphZntI4eEmZnV5N1NZmZWU70fae0XZs6cGfPmzWt2GWZm+5Vbbrnl4YjoHandfh8S8+bNY8mSJc0uw8xsvyLpvpFbeXeTmZnV4ZAwM7Oacg0JSSdLWiFpZbXO5iR9QdJt6XZ36pjLzMyaJLdjEulqWheS9Ve/Blgs6YqIuLPUJiLeV9b+r4Cj86rPzMz2lOeWxHHAyohYlfrduZT6vV++lqxPHzMza5I8Q2IOu/eJv4bdrwmwk6RDgflkl82sNv0sSUskLenr6xvzQs3MLJNnSKjKuFo/9z4TuDxd4WzPJ0VcHBELI2Jhb++Ip/mamdk+yjMk1rD7BVjmki6OU8WZNHhX0+LVj/L5q1cwMFTrqo5mZpZnSCwGFkiany6EciZZ3/K7kfQUsgvI/6aRxdx633r+5RcrHRJmZnXkFhLpguPnAFcBy4HLImKZpPMknV7W9LXApdHgngeLhWzv1+CwOzg0M6sl1245ImIR2ZW6ysd9omL4k3nUUgqJoSGHhJlZLS37i+u2Uki4q3Qzs5paNiQKpZDw7iYzs5paNiTaHBJmZiNq2ZAoFrKX7pAwM6uthUMiu/fZTWZmtbVwSHhLwsxsJK0bEvIxCTOzkbRuSOz8MZ1/cW1mVkvLhkTp7CZnhJlZbS0bEt6SMDMbWcuHxLB/cW1mVlPLhkRpd9Og+24yM6upZUPC3XKYmY2sZUPCHfyZmY2sZUOi4OtJmJmNqGVDYtcpsA4JM7NaWjYkfGU6M7ORtXxI+MC1mVltLRsSvp6EmdnIWjYkCu7gz8xsRC0bEm2pq3AfkzAzq61lQ6JY9NlNZmYjad2QkM9uMjMbSeuGhH9xbWY2opYNiZ1nNw25q3Azs1paNiTcLYeZ2chaNiTafD0JM7MRtWxIuFsOM7ORtXxI+BRYM7PaWjckfAqsmdmIWjYkCgUhuVsOM7N6WjYkIDt47ZAwM6st15CQdLKkFZJWSjq3RptXS7pT0jJJ321kPQU5JMzM6mnLa0GSisCFwEnAGmCxpCsi4s6yNguAjwDPj4j1kg5oZE1tBfmYhJlZHXluSRwHrIyIVRGxA7gUOKOizduBCyNiPUBErGtkQUXvbjIzqyvPkJgD3F82vCaNK/dk4MmSrpd0o6STq81I0lmSlkha0tfXt88FOSTMzOrLMyRUZVzlN3QbsAA4Hngt8DVJU/d4UsTFEbEwIhb29vbuc0HFQsEd/JmZ1ZFnSKwBDi4bngs8UKXNjyNiICLuBVaQhUZDtBXE0JBDwsysljxDYjGwQNJ8SR3AmcAVFW1+BPwpgKSZZLufVjWqoKIPXJuZ1ZVbSETEIHAOcBWwHLgsIpZJOk/S6anZVcAjku4ErgE+GBGPNKqmYkHu4M/MrI7cToEFiIhFwKKKcZ8oexzA+9Ot4bwlYWZWX0v/4rpYkDv4MzOro6VDIvsxna9MZ2ZWS0uHhLvlMDOrr6VDoq3okDAzq6elQ6IgH7g2M6unpUPCXYWbmdXX0iHhvpvMzOpzSDgkzMxqckj4F9dmZjW1dEj4mISZWX0tHRLFQoEB9wJrZlZTS4dER5sYHPIvrs3MamnpkGgvFtjhkDAzq6nlQ2Jg0CFhZlZLS4dER1uBHT4mYWZWU2uHRLHAgHc3mZnV1NIh0V6UQ8LMrI4WDwlvSZiZ1eOQGArCv7o2M6uqpUOioy17+T4N1sysupYOifaiAPyrazOzGlo6JDqK2cv3byXMzKpr6ZBoT7ubfPDazKy61g6Joo9JmJnV09IhsXN3k49JmJlV1dIhsXNLwsckzMyqavGQKJ3d5JAwM6umpUPCv5MwM6uvtUPCp8CamdXV0iGx6xRYH7g2M6umtUOi6N9JmJnVk2tISDpZ0gpJKyWdW2X6myX1Sbot3d7WyHpKB677vbvJzKyqtrwWJKkIXAicBKwBFku6IiLurGj6/Yg4J4+aOrwlYWZWV55bEscBKyNiVUTsAC4Fzshx+XvocLccZmZ15RkSc4D7y4bXpHGV/lzSUkmXSzq42owknSVpiaQlfX19+1yQj0mYmdWXZ0ioyrjK04r+G5gXEc8E/he4pNqMIuLiiFgYEQt7e3v3uaBdfTf57CYzs2ryDIk1QPmWwVzggfIGEfFIRPSnwa8Cz25kQf6dhJlZfXmGxGJggaT5kjqAM4EryhtImlU2eDqwvJEFtbdlGzf+xbWZWXW5nd0UEYOSzgGuAorA1yNimaTzgCURcQXwHkmnA4PAo8CbG1lTu7ckzMzqyi0kACJiEbCoYtwnyh5/BPhIXvW0FdzBn5lZPS39i2tJdLQVfODazKyGlg4JyA5ee0vCzKw6h0Rbge0DQ80uw8xsXGr5kOhuL7J9wFsSZmbVtHxIdLYX2D7oLQkzs2paPiS624ts3+GQMDOrxiHRXmSbj0mYmVW11yEhaULq9vsJoau96APXZmY1jBgSkgqSXifpJ5LWAXcBayUtk/RZSQsaX2bjdLUX2eYD12ZmVY1mS+Ia4Elkv4Q+KCIOjogDgBcCNwLnS3pDA2tsqO6OIv3ekjAzq2o03XKcGBEDlSMj4lHgB8APJLWPeWU56Wor+JiEmVkNI4ZEKSAkdQGHk10D4p6I2F7ZZn/U3eED12ZmtYzmmESbpM+QXQ/iEuDbwP2SzpeUaweBjdDtA9dmZjWN5pjEZ4FpwHzgyog4muwYxUzgcw2sLRed6RfXw8Pu5M/MrNJoQuLlwFkRsRk4DSAiNgHvSNP2a93t2dm8/b6mhJnZHkYTEhERpX+zVTZyCNjvv1m72rNV4F1OZmZ7Gk1ILJf0l+nxzmtSp9NeG3p50TyUtiR88NrMbE+jOfD8buCHkt4C3CLpc8CxQBfwykYWl4fuDoeEmVktozkFdg1wrKQTgCPJdjktiohfNLq4PHS2ZSHh3U1mZnsaMSQkKTI/B35er82YV5eD0paEQ8LMbE+j6pZD0l9JOqR8pKQOSS+RdAnwpsaU13g7j0ns2O+PwZuZjbnRHJM4GXgL8D1J84ENQDdZwFwNfCEibmtciY3ls5vMzGobzTGJ7cBFwEWpj6aZwLaI2NDo4vLQk3Y3bd0x2ORKzMzGn9Eck7gAWJpuyyJibcOrytHEzqxvwq393pIwM6s0mt1NK4HnAm8HjpD0ILtCYzFwXUT0N67ExprYla2CLf37bR+FZmYNM5rdTReVD6fjEs8AngmcDXxF0tkRcVVjSmysnvYiEmzZ7t1NZmaV9roX14i4F7gXuAJA0izgSmC/DIlCQUzsaGNzv0PCzKzSXl/julI6RvHdMailaSZ2tXlLwsysiscdEgAR8fmxmE+zTOxsY4u3JMzM9jAmIbG/m9jlkDAzq8YhQbYlsdm7m8zM9pBrSEg6WdIKSSslnVun3askhaSFedQ1yVsSZmZV5RYSkorAhcApZL3JvlbSkVXaTQLeA9yUV22TOtt94NrMrIo8tySOA1ZGxKqI2AFcCpxRpd2ngM8A2/MqzMckzMyqyzMk5gD3lw2vSeN2knQ0cHBEXJljXTvPbhoe3i97Ozcza5g8Q0JVxu38VpZUAL4AfGDEGUlnSVoiaUlfX9/jLmxS6prDnfyZme0uz5BYAxxcNjyXsmtmA5OApwPXSlpN1l/UFdUOXkfExRGxMCIW9vb2Pu7CSiGxcZv7bzIzK5dnSCwGFkiaL6kDOJPUtQdARGyMiJkRMS8i5gE3AqdHxJJGFza1pwOADY85JMzMyuUWEhExCJxD1sfTcuCyiFgm6TxJp+dVRzXTUkisf2xHM8swMxt39rqDv8cjIhYBiyrGfaJG2+PzqAlgWk92TYn13pIwM9uNf3ENTJuQtiS2ekvCzKycQwKY2l3aknBImJmVc0gAbcUCk7rafODazKyCQyKZ1tPhLQkzswoOiWTahA4e9TEJM7PdOCSSaT3t3t1kZlbBIZFM7/GWhJlZJYdE0ju5k77N/US4kz8zsxKHRHLApC52DA17l5OZWRmHRHLApE4A1m3ub3IlZmbjh0MiOXByFwAPbcrtWkdmZuOeQyLxloSZ2Z4cEskBk0sh4S0JM7MSh0TS09HGpM421m3yloSZWYlDosxBU7pYu3Fbs8swMxs3HBJl5k7rZs16h4SZWYlDoswch4SZ2W4cEmXmTuth47YBNm/3D+rMzMAhsZu507oB+OMGb02YmYFDYjdzp/UAsOZRh4SZGTgkdnPI9CwkVj+ytcmVmJmNDw6JMtN62pna086qhx0SZmbgkNiNJA6bOYFVfVuaXYqZ2bjgkKhwWO9EVvV5S8LMDBwSezisdwLrNvf7NFgzMxwSe1hwwCQA7n7Iu5zMzBwSFY6YlYXE8rWbmlyJmVnzOSQqzJnazeSuNoeEmRkOiT1I4qmzJnOnQ8LMzCFRzdNnT2H52k0MDA03uxQzs6ZySFSxcN40tg8Ms+wBb02YWWtzSFSx8NBpACxZ/WiTKzEza65cQ0LSyZJWSFop6dwq098p6XeSbpP0a0lH5llfyQGTuzh0Rg+LHRJm1uJyCwlJReBC4BTgSOC1VULguxHxjIg4CvgMcEFe9VVaeOh0lqxeT0Q0qwQzs6bLc0viOGBlRKyKiB3ApcAZ5Q0iovwgwASgad/Qx86bxiNbd3CvO/szsxaWZ0jMAe4vG16Txu1G0rsl3UO2JfGeajOSdJakJZKW9PX1NaTYY+dPB+Cme73LycxaV54hoSrj9thSiIgLI+JJwIeBj1ebUURcHBELI2Jhb2/vGJeZOWzmBOZM7ebny9c1ZP5mZvuDPENiDXBw2fBc4IE67S8FXtHQiuqQxElHHsivV/axbcdQs8owM2uqPENiMbBA0nxJHcCZwBXlDSQtKBt8GfD7HOvbw0uPPJDtA8Nc9/vG7NIyMxvvcguJiBgEzgGuApYDl0XEMknnSTo9NTtH0jJJtwHvB96UV33VHDt/OlO627l62UPNLMPMrGna8lxYRCwCFlWM+0TZ4/fmWc9I2osFTnjqAfzv8ofoHxyis63Y7JLMzHLlX1yP4BVHz2HjtgGu8taEmbUgh8QIXnD4TOZO6+bSm//Q7FLMzHLnkBhBoSBee9wh3HDPI/5hnZm1HIfEKPzFs+fSVhCX3LC62aWYmeXKITEKB0zu4pVHz+F7N/+BdZu3N7scM7PcOCRG6d1/ejgDQ8N89bpVzS7FzCw3DolRmjdzAq84ag7fuvE+HtiwrdnlmJnlwiGxF9530pOJgE8vWt7sUszMcuGQ2AsHT+/hXccfzpVL13LDyoebXY6ZWcM5JPbSO158GPNm9PDBy5eyaftAs8sxM2soh8Re6movcsFrjuLBTdv52x/d4SvXmdkTmkNiHxxzyDTee8ICfnzbA/7thJk9oTkk9tE5f3o4Jx5xIOddeSfXrvCFiczsickhsY8KBfHFM4/iKQdN5h3fuoXrfSDbzJ6AHBKPw8TONr791uOYN2MCb71kMTfc46AwsycWh8TjNGNiJ995+3M4ZHoPb/7GYn742zXNLsnMbMw4JMbAzImdfP+s53HMIVN53/dv559+ehdDwz7rycz2fw6JMTJtQgf/8Zbn8NrjDuHfrr2HV3/lN6x21+Jmtp9zSIyhjrYCn37l0/nSmUfx+4c2c+o//4qv/WoVA0PDzS7NzGyfOCTGmCTOOGoOV73vRRw3fzp//5PlnPqlX3HtinX+4Z2Z7XccEg0ya0o333jzsXz1LxfSPzjMm7+xmFdedAPX3OWwMLP9h/b3L6yFCxfGkiVLml1GXTsGh7n8ljVceM1K/rhhG0fMmswbn3soZxw1mwmdbc0uz8xakKRbImLhiO0cEvkZGBrmh7f+ka9ffy93PbiZSZ1tnHbUbE575myOmz+dYkHNLtHMWoRDYhyLCG79w3q+feMf+OkdD7JtYIgDJnVy6jNm8dKnHcjCQ6fT0eY9gWbWOA6J/cRjOwb5+fJ1/GTpWq5ZsY7+wWF6Ooo877AZvOjJvTz3sBksOGAiBW9lmNkYGm1IeId4k/V0tHHas2Zz2rNms6V/kN/c8wjX3d3HL+/u4+d3ZR0HTupq45hDprHw0Gkcfcg0jpw9mekTOppcuZm1AofEODKxs42TjjyQk448EID7HtnK4tXrueW+9dxy36N8/md9O9seNLmLI2ZN4ohZkzli1mSe1DuReTN76OnwW2pmY8ffKOPYoTMmcOiMCbzq2XMB2PjYAEv/uIHlazexfO1mlq/dxK9+/zCDZV2AHDS5i3kze5g/cyLzZ/Ywd1oPs6Z0MXtqN70TO73bysz2ikNiPzKlp50XLujlhQt6d47rHxzinnVbWfXwFlY/vJVVD29l9cNb+ekda1n/2O6XV20riAMndzFnajezpnZx0JQueid2MmNiBzMmZPczJ3YyfUIH7UUfODczh8R+r7OtyJGzJ3Pk7Ml7TNvw2A7+uGEbazdsZ+3GbTywcTtrN2T3t/5hPQ9t7GdHjS5DpnS3M2NiB9N6Opjc1caU7nYmd7dn913pvruNyWXDPR1FJnS20dlWQPIWi9kTgUPiCWxqTwdTezp42uwpVadHBJv7B3lkyw4e2dLPw1v6eXjLjmx4az+PbNnBhm076NvSzz19W9m4bYBN2wcY6YS4gmBCRxs9nUUmdLTR3VHcbbino0hPR5Hujja62gt0thXpbCvQWf64rUBne5GudF8+rvS4o61Ae6HgXWhmDZRrSEg6GfgSUAS+FhHnV0x/P/A2YBDoA94SEfflWWMrkcTkrmxLYP7MCaN6zvBwsGXHIJu2DWShsW1wZ3g81j/IYwNDPNY/xNYdgzvvt+3I7h/duoP7H30sDQ/x2I5BBoYe/ynYxYJoK4j2YoG2YnbfXhBtpeFCgfY20VYo0F7M7tuKoiNNbytvXxDFdCuo8jEUJQoF7bpPyy61LU0rFqjy/Oy+tIxSW4nshigoe18KpXESIpuXtOteiEKh/DnV20J2FcXy8Tvns9s41RifLaO0Yaj0uSk9zj5Huz5P9sSTW0hIKgIXAicBa4DFkq6IiDvLmv0WWBgRj0k6G/gM8Jq8arSRFQq7gmXutMc/v6HhYMfgMP2DQ/QPDtM/UPZ4cCgNV5++fWCIgaFgcHiYwaHY+XhgaDh7PDTMwHC6HwoGhrJ2g8PDbBtIbQeDgfT8Uvvh4WAogqGyx8PD7BxnIxtVqKCdI8unKQ1pj2na+XjP56lue1U8UapeT7V57PGaKrKwrKoa0yuGKxrsEa178fz3nrCA0541u3IOYyrPLYnjgJURsQpA0qXAGcDOkIiIa8ra3wi8Icf6rAmKBdHdUaS7o9jsUkZttxCJYLAUJhWBsvu4Xc8p3YYjGBrOdvsFMBwBAcMBQWT3EURpeJid7UrjR9M2m2eUPTdrO1x+D2Xz3LMtkObNzseQLatyWmli1GhXaxq7zX/k9uW7PaNsmbvq2XMeu4Yrpo3QvrzGcpX/MlT+OHnP6Y/v+ZUjpnS3V7YYc3mGxBzg/rLhNcBz6rR/K/A/1SZIOgs4C+CQQw4Zq/rMRqVQEAVE+/6Ta2b7LM/zHKvtsKy67S7pDcBC4LPVpkfExRGxMCIW9vb2VmtiZmZjIM8tiTXAwWXDc4EHKhtJOhH4GPDiiOjPqTYzM6sizy2JxcACSfMldQBnAleUN5B0NPAV4PSIWJdjbWZmVkVuIRERg8A5wFXAcuCyiFgm6TxJp6dmnwUmAv8p6TZJV9SYnZmZ5SDX30lExCJgUcW4T5Q9PjHPeszMrD530GNmZjU5JMzMrCaHhJmZ1bTfX75UUh+wr/07zQQeHsNyxsp4rQvGb22ndd+dAAAGx0lEQVSua++4rr3zRKzr0IgY8Ydm+31IPB6SlozmGq95G691wfitzXXtHde1d1q5Lu9uMjOzmhwSZmZWU6uHxMXNLqCG8VoXjN/aXNfecV17p2XrauljEmZmVl+rb0mYmVkdDgkzM6upZUNC0smSVkhaKencJteyWtLvUqeGS9K46ZJ+Jun36X4MLhY6Yh1fl7RO0h1l46rWocw/p/W3VNIxOdf1SUl/TOvsNkmnlk37SKprhaQ/a2BdB0u6RtJyScskvTeNb+o6q1NXU9eZpC5JN0u6PdX1d2n8fEk3pfX1/dRLNJI60/DKNH1eI+oaobZvSrq3bJ0dlcbn+fkvSvqtpCvTcL7rK3ZeCrF1bkARuAc4DOgAbgeObGI9q4GZFeM+A5ybHp8L/FMOdbwIOAa4Y6Q6gFPJrhwo4LnATTnX9Ungb6q0PTK9n53A/PQ+FxtU1yzgmPR4EnB3Wn5T11mdupq6ztLrnpgetwM3pfVwGXBmGv9l4Oz0+F3Al9PjM4HvN/AzVqu2bwKvqtI+z8//+4HvAlem4VzXV6tuSey83nZE7ABK19seT84ALkmPLwFe0egFRsR1wKOjrOMM4D8icyMwVdKsHOuq5Qzg0ojoj4h7gZVk73cj6lobEbemx5vJusCfQ5PXWZ26asllnaXXvSUNtqdbAC8BLk/jK9dXaT1eDpwgqdoVLhtZWy25vJeS5gIvA76WhkXO66tVQ6La9bbr/RE1WgBXS7pF2fW7AQ6MiLWQ/dEDBzSptlp1jId1eE7a1P962e64ptSVNu2PJvsPdNyss4q6oMnrLO06uQ1YB/yMbKtlQ2TXm6lc9s660vSNwIxG1FWttogorbN/SOvsC5I6K2urUvdY+iLwIWA4Dc8g5/XVqiEx6utt5+T5EXEMcArwbkkvamIto9XsdfhvwJOAo4C1wOfT+NzrkjQR+AHw1xGxqV7TKuMaVluVupq+ziJiKCKOIrt88XHAEXWWnev6qqxN0tOBjwBPBY4FpgMfzqs2SS8H1kXELeWj6yy3ITW1akiM6nrbeYmIB9L9OuCHZH88D5U2X9N9sy7nWquOpq7DiHgo/VEPA19l1+6RXOuS1E72RfydiPivNLrp66xaXeNlnaVaNgDXku3PnyqpdAG08mXvrCtNn8LodzuORW0np113ERH9wDfId509Hzhd0mqyXeIvIduyyHV9tWpIjHi97bxImiBpUukx8FLgjlTPm1KzNwE/bkZ9deq4AvjLdJbHc4GNpV0seajY//tKsnVWquvMdKbHfGABcHODahDw78DyiLigbFJT11mtupq9ziT1SpqaHncDJ5IdL7kGeFVqVrm+SuvxVcAvIh2Vzam2u8rCXmT7/svXWUPfy4j4SETMjYh5ZN9Rv4iI15P3+hqrI/D7243s7IS7yfaJfqyJdRxGdmbJ7cCyUi1k+xJ/Dvw+3U/PoZbvke2GGCD7r+Stteog27S9MK2/3wELc67rW2m5S9Mfx6yy9h9Lda0ATmlgXS8g25xfCtyWbqc2e53Vqaup6wx4JvDbtPw7gE+U/Q3cTHbA/D+BzjS+Kw2vTNMPa+B7Wau2X6R1dgfwbXadAZXb5z8t73h2nd2U6/pytxxmZlZTq+5uMjOzUXBImJlZTQ4JMzOrySFhZmY1OSTMzKwmh4RZImlLup8n6XVjPO+PVgzfMJbzN2sUh4TZnuYBexUSkoojNNktJCLiT/ayJrOmcEiY7el84IXp+gHvSx2/fVbS4tTR2zsAJB2v7LoN3yX7QRWSfpQ6alxW6qxR0vlAd5rfd9K40laL0rzvUHZNkdeUzftaSZdLukvSdxrVA6pZPW0jNzFrOeeSXXfh5QDpy35jRBybegG9XtLVqe1xwNMj62Ib4C0R8Wjq2mGxpB9ExLmSzoms87hK/4esw71nATPTc65L044GnkbWN8/1ZH35/HrsX65Zbd6SMBvZS8n66bmNrMvtGWT9GwHcXBYQAO+RdDtwI1lnawuo7wXA9yLreO8h4JdkPY6W5r0msg75biPbDWaWK29JmI1MwF9FxFW7jZSOB7ZWDJ8IPC8iHpN0LVl/OiPNu5b+ssdD+O/VmsBbEmZ72kx22c+Sq4CzU/fbSHpy6rG30hRgfQqIp5J1g10yUHp+heuA16TjHr1kl2ptSK+1ZvvC/5mY7WkpMJh2G30T+BLZrp5b08HjPqpfTvanwDslLSXrTfXGsmkXA0sl3RpZd88lPwSeR9YLcAAfiogHU8iYNZ17gTUzs5q8u8nMzGpySJiZWU0OCTMzq8khYWZmNTkkzMysJoeEmZnV5JAwM7Oa/j+F7dcNj3WPEwAAAABJRU5ErkJggg==\n", 319 | "text/plain": [ 320 | "
" 321 | ] 322 | }, 323 | "metadata": {}, 324 | "output_type": "display_data" 325 | } 326 | ], 327 | "source": [ 328 | "plt.plot(J_history)\n", 329 | "plt.xlabel(\"Iteration\")\n", 330 | "plt.ylabel(\"$J(\\Theta)$\")\n", 331 | "plt.title(\"Cost function using Gradient Descent\")" 332 | ] 333 | }, 334 | { 335 | "cell_type": "markdown", 336 | "metadata": {}, 337 | "source": [ 338 | "### Plotting the decision boundary\n", 339 | "\n", 340 | "From Machine Learning Resources:\n", 341 | " \n", 342 | "$h_\\Theta(x) = g(z)$, where g is the sigmoid function and $z = \\Theta^Tx$\n", 343 | "\n", 344 | "Since $h_\\Theta(x) \\geq 0.5$ is interpreted as predicting class \"1\", $g(\\Theta^Tx) \\geq 0.5$ or $\\Theta^Tx \\geq 0$ predict class \"1\" \n", 345 | "\n", 346 | "$\\Theta_1 + \\Theta_2x_2 + \\Theta_3x_3 = 0$ is the decision boundary \n", 347 | "\n", 348 | "Since, we plot $x_2$ against $x_3$, the boundary line will be the equation $ x_3 = \\frac{-(\\Theta_1+\\Theta_2x_2)}{\\Theta_3}$" 349 | ] 350 | }, 351 | { 352 | "cell_type": "code", 353 | "execution_count": 13, 354 | "metadata": {}, 355 | "outputs": [ 356 | { 357 | "data": { 358 | "text/plain": [ 359 | "" 360 | ] 361 | }, 362 | "execution_count": 13, 363 | "metadata": {}, 364 | "output_type": "execute_result" 365 | }, 366 | { 367 | "data": { 368 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VGX2wPHvSWgqCIioCCLIKh1CE8ESqgUUftgVAUFFSqTZsEGwrSysFAlBXUBlEbHBosJSpYoIKB0iFlAEBVEQXBBIzu+PmYFJSJkkM3PvzJzP89wn0zL3zM3knvu+733PFVXFGGOMiXM6AGOMMe5gCcEYYwxgCcEYY4yXJQRjjDGAJQRjjDFelhCMMcYAlhCMMcZ4OZYQROQiEflURLaKyGYR6e9ULMYYY0CcmpgmIhWACqr6pYiUAtYC/6eqWxwJyBhjYlwRp1asqnuAPd7bh0RkK1ARyDEhnHvuuVqlSpXwBGiMMVFi7dq1v6pq+bxe51hC8CciVYAGwKpsnusJ9ASoXLkya9asCWtsxhgT6URkZyCvc3xQWURKAh8AA1T1j6zPq+prqtpYVRuXL59ngjPGGFNAjiYEESmKJxlMVdUPnYzFGGNinZNnGQkwEdiqqi87FYcxxhgPJ1sIVwJdgFYiss67tHMwHmOMiWlOnmW0HBCn1m+MMSYzxweVjTHGuIMlhHzIOofPLjZnjIkmlhAClJwMAweeSgKqnvvJyU5GZYwxwWMJIQCqcOAAjBlzKikMHOi5f+CAtRSMMdHBFTOV3U4ERo3y3B4zxrMA9O/veVxsaNwYEwUcK25XEI0bN1YnS1eoQpxfmyojw5KBCYEWLTw/Fy92MgoTRURkrao2zut11mUUIF83kT//MYWQatHi1E7CGGNCxLqMAuA/ZuDrJvLdB+s2MkHiS/pLlmS+by0FEyaWEAIgAmXKZB4z8I0plCkTwmRgO4jwybptbVvnzLZN1LKEEKDkZE9Lwbfz9yUFaxmYoClMMrKdtAkCSwg58N/5Z3cfwpAM7Gg19LK2wsqU8fw8eDDz87btrcUaAywhZCM52TO/wNcC8I0hlCljE9FMGBSkZWA7aRMElhCy8J+EBpkHkPv3z76lEHL2zx06ObXCbMd6Ots2Uc8SQhY2Cc1EFNtJmyCyiWk5sEloJqJYQjC5sIlpheDoJDRjCmLxYksGptBiLiFM3TCV34/8nuPzWSehZWR4fvoXtjPGmGgUU2MIOw/spNvMbpQ7sxwj247knnr3IFn6gRybhGaMMQ6LuTGEr/Z8Re9PerPqp1Vcc/E1jG83ntrn1T7tdYHMQzAmrGycwBSQjSHkoEGFBnx232e8duNrbPxlIwmvJvDY/Mc4fOxwpteFfRKaMcY4LOZaCP72/bmPwQsGM2ndJCqdXYkx14+hU41Op3UjGeOorJPPEhM9P93YUrBWjCtZCyEA5c8qz8SOE1nefTnnnHEOt7x7C+3fbs+3v33rdGjGGBN2Md1C8Hci4wTjvhjHM58+w4mMEzx51ZM8euWjlChSIiTrMybf3Hz0HUmtmBhkLYR8KhJXhAFXDGBb3210rN6RIYuHUC+1HvO+ned0aMZEnnXrnI7AFIC1EHIw/9v59J3dl+2/bef22rfz8rUvU/HsimFZtzFhE+xWR4sWnmSQkGCtAxexFkIhta3Wlo29N/Jcy+eYlTaLGik1GLVyFCcyTjgdmjHu5EsGBw96uo7s0q8RxxJCLooXKc7T1zzN5j6buebiaxg0bxCNXmvEih9WOB2aMQXXooVnlmWLFp4ddzB33gkJhX8P4xhLCAG4pOwlfHzXx3x4+4f8fuR3rpp8Fff95z5+/d+vTocWXeyIMrL56iklJnoWq68UcSwhBEhE6FSzE1v6buGx5o/x1oa3qD6uOq+vfZ0MzXA6PGPy5ku4S5acuiJc6dK2884qhg9MLCHkU8liJRnedjjrHlxHnfPq0PPjnjSf2Jyv9nzldGjBFc5/Cv8dlfU9Rz5LLhErporbBVPt82qzuNti/r3h3zwy/xEav96YpCZJPNvyWUqXKO10eMaczi6mkzu7HKklhMIQEbrU78KNl93I04ue5pUvXuHdLe/y8rUvc2edOyOzBIYT/xS2ozLGFazLKAjKnlGWlPYpfPHAF1Q6uxJ3f3g3baa0Yduv25wOzZjTZdelE6vddP6f2wbFrYUQTI0vbMzn933Oa2tf44mFT1AvtR6PNn+Up655ijOLnhmalQb7qNrJo/UY++czxm0sIQRZfFw8vZv05uaaN/PYgsd4cfmLTN04lVdueIWbqt/kdHjGZBar/ea5fe5wfnaXbW/rMgqR80uez5v/9yaLuy3mrGJn0eGdDnR8pyM7DuwIzgpCfWZOtDeXY7WLxJhcWAshxBKrJLLuwXWM/nw0yUuSqZVSi2eueYaHmz9MsfhiTodn3MCpo0T/9brsSDXknD6RwaUtM0sIYVA0viiPXvkod9S5g4FzB/Lkoid5a8NbpLRLoVXVVgV7U6e/0JHKpf+IxriBowlBRCYBNwJ7VbWOk7GEQ+XSlfng9g+YvX02D815iNZvtaZz3c6MvHYkF5S8wOnwTLiFOzll7SKzpOjcZ3bpAZ3TLYQ3gHHAWw7HEVbtLm1HyyoteWn5S7y04iU++vojXmj1Ar0b9yY+Lj5/b+aSL1LEcOk/ojFu4Pj1EESkCvBxIC2EcF4PIVy2799O39l9mf/dfBpc0IDU9qk0rdTU6bCin5sSQrhaBlmvZubjhm1gQsquhxAhLi13KXPvmcu7t77LL3/+QrOJzej1cS9+O/Kb06FFt2g/i8qYAnB9C0FEegI9ASpXrtxo586d4QsuzA79dYjkxcmMWTWGsmeUZUTbEXSt35U4sbxtgsBNrSITVlHTQlDV11S1sao2Ll++vNPhhFSp4qX453X/5MsHv+SycpfR/T/dSXwjkY2/bHQ6NGNMDHB9QohF9c6vx7Luy5jUYRJb922lwasNeGTeIxz665DToZlIZt1kJg+OJgQRmQasBKqLyC4Ruc/JeNwkTuLo3qA7aUlp3NfgPv658p/UTKnJ+1vex+luPmNMdHI0IajqXapaQVWLqmolVZ3oZDxuVO7Mcrx606usvG8l5c8qz23v3cYNU29g+/7tTodmjIky1mUUIa6odAWrH1jN2OvHsnLXSuqk1mHop0M5cvyI06EZY6KEJYQIUiSuCA81fYhtfbdxa61beXbps9RNrcuc7XOcDi02WYE8E2UsIUSgCqUqMPXmqSzsupAicUVo93Y7bn33Vn48+KPToRljIpglhAjWqmor1vdaz4utXmT29tnUTKnJyM9Gcjz9uNOhRbdQlx43xiGWECJc8SLFeeLqJ9jSdwutqrbi0fmP0uDVBizbuczp0IwxEcbxmcr5EY21jIJtVtos+s3px86DO+lWvxv/aPsPzjvrPKfDik4289dEiKiZqWxyljWXq0KH6h3Y3GczT1z1BG9vfJvq46ozYc0E0jPSnQnSGBMxLCFEqORkGDjwVFJQ9dxPToazip3Fi61fZH2v9TS4oAG9P+lNs4nNWLt7rZMhRx+b+WuijCWEAsru6Dyc6z5wAMaMOZUUBg703D9w4FQsNcvXZGHXhUy9eSo/HPyBJq83IWl2EgeOHghfsCa0bEDbBJElhALI7eg8HERg1Cjo39+TBOLiPD/79/c8LuL/WuHuuneTlpRG0uVJpK5Jpfq46vx7w7+tBEZh2I7YRCFLCPkU6NF5qPmSgr+sycBf6RKlGXvDWFY/sJoqZarQZUYXWr7Zki37toQ+WBN8duqrCQFLCPmUn6PzUPIlIn/+rZacNKzQkJX3reTVG19lwy8bqD+hPoMXDObPY3+GLthoYjti4y/K/v6WEAogv0fnwebfKunfHzIyTiWoQJJCnMTRs1FP0pLS6FKvC8NXDKfW+FrM3DbTupEihW9AOzHRs9gAtwmCIk4HEIlyOjoPRlJQzfweWe+D536ZMplbJb4EVaZM4DGUP6s8kzpOokeDHvT+pDedpnei/aXtGXvDWC4pe0nhPki08u10bQ5CwUXDtst6nepo+ExYCyHfCnt0npv8DFYnJ2dOQL6kUJCB7asqX8WXPb/kn9f+kyU7l1B7fG2eX/o8f534q2AfxE2irEl/GmsZmCCyFkI+iUDp0pmPzl9+2fNcfo7Os/IfrAbPe/snnpxaCrndz4+i8UUZ1GwQt9e+nUFzB/HMp88wZcMUUtql0OaSNoG/UZQcKeUp2j9fKETTUXWUthSthZBPyclw8KAnCYh4dtSDBnmSRGFOO3XLYHWlsyvx7m3v8t/O/yVDM2g7pS13vn8nuw/tDk8AwWKDv6Fn2zTqWELIB/+j+EGDMncfHTxY+FNOnR6s9nfd365jY++NDGsxjJnbZlJjXA3GfD6GExknsv8F2wGbvETjQHg0fAY/1mWUD/477DFjTnXvBOsoPpSD1QVRokgJhiQOoXPdziTNSWLA3AFMXjeZ1PapNLuoWfgDyo8obdK7QjR1/ZhMrIWQT6E6ig/lYHVhVTunGrPvns0Ht3/A/iP7aT6pOQ/MeoD9/9t/6kXRePRn8ifQVqF9N1zLWgj5FKqj+GCdShoqIsLNNW/m2mrX8uySZxn1+ShmbJvB8DbD6d6gO3Hi0mML2/EEX06tL+sijHh2PYR8yHoUn/VMIN8OPK95BHmtozC/Hy6b9m6izyd9WPbDMppVakZq+1TqX1Df6bBMOGVNAL4upMREz09Lxq5h10MIgZyO4vv39zw+bFjhi94F81TSUKpzXh2W3LuEN//vTb757RsavtaQgf8dyB9//eF0aIHL2sVhA+H5Y10/USfPhCAil4nIQhHZ5L1fT0SeDn1o7pTThLChQ91R9C6cRISu9buSlpTGg40eZMyqMdQYV4Ppm6ZbCYxYYuNHoRXGA5U8u4xEZAnwKPCqqjbwPrZJVeuEIb5MnO4yyot/EvAJ9zwCJ63+aTW9P+nN2j1raXNJG1LapXBZucucDut0Wc+SKV3a8/PgQc9P6/IoGDvbKDSCsF2D2WV0pqp+keWxHE5Gj23BPgPJyYvwFESTik1Ydf8qUtqlsPqn1dRNrcszi57hyPEjTodmwsFaBsHlwNyeQBLCryJSDVAAEbkV2BPSqCJUQUtSZ8fpi/AUVHxcPH2a9CEtKY07at/B88uep/b42nzy9SdOh3ZK1i6OAwc8i3V5mBgXSELoC7wK1BCRn4ABQK+QRhWBgjmPwC0X4SmM80uez1ud3uLTbp9SokgJbpx2I52md2LngZ1Oh2ZMZHBgbCbXeQgiEgc0VtU2InIWEKeqh0IaUYQK5jyCUM+IDqcWVVqwrtc6Rn8+mmFLhlFrfC2GXDOEgc0GUiy+mLPBZf3nslZBZjYmEHMCGVReqqrXhCmeXLl9UBmCO49A1VPkzicjI7KSQVY/HPyBAf8dwIxtM6h5bk3Gtx9PiyotnA7L5MQSQtQI5qDyfBF5REQuEpFzfEsQYoxKwZpHEMzxCLeoXLoyH97xIR/f9TFHTxyl5Zst6TKjCz8f/tnp0LIXTfMS8vNZorFQYTR8hjAIJCH0wDOOsBRY613cfZge4dxc16hQvP+U7S9rz6Y+m3j66qd5d/O71BhXg5QvUkjPSHc6QmNimpWucKnkZM8Asm/MwJckypRx/5lGOcqmCyLt1zSS5iSx4LsFNKzQkNT2qVxe8XJHwjsp6zyFSJ6XUJjP4v/3itTuIzf/LcO4TQPtMsqzuJ2IFAV6A75xhMV4JqkdL1SEJlfJyZnHH3wDzRE5hpBLueTq51Zn3j3zeHfzuwycO5Ar/nUFDzZ6kBdbv0jZM8o6Ea0x7uBAEg5kUPlfQFHgTe9DXYB0Vb0/xLGdJpZaCFElwKO0P/76g6GfDuWVL17hnDPOYUTbEXSt3xVxKgtG6lFxdgr6Wdx8hJ0fbvpbBrpNgxhz0FoIQBNV9S9juUhE1hc8NBNzArxYzdnFz2bU9aO4N+Feen/Sm3v/cy8Tv5rI+PbjqXNe2CulGOMMBy9AFEhCSBeRaqr6LYCIXALY6J8JmfoX1Gd5j+W8se4NHpv/GAkTEhh4xUCGthhKyWIlwxeIG44mg6WgnyVarjznprhdvE0DSQiPAp+KyHeAABcD3UMalYlO+fjix0kcPRr0oGP1jjyx8AlGrhzJO5vfYfR1o7m55s3OdSMZE2oOJoyAzjISkeJAdTwJYZuq/hXqwLJjYwgFFykX3snJyh9X0vuT3qz/ZT3X/+16XrnhFf52zt+cDiv/XHhUaFzKgTGEQK6H0Bc4Q1U3qOp64EwR6VPoCD3vfb2IpInINyIyOBjvaU4XqYXy/DW7qBlreq5hzPVjWPHDCuqMr8OwxcM4euKo06EZExoOFFkMZGLaA6p6wHdHVX8HHijsikUkHkgBbgBqAXeJSK3Cvq/JLBoK5fkUiStCv6b92Ja0jU41O5G8JJk64+sw95u5ToeWt2ic/RupbNvnKJCEECd+HbbeHXkwqpJdDnyjqt+p6jHgHaBjEN43ooT6mgf+l/kcM8ZTG8n/GtCR1G3kc2GpC5l2yzTmd5lPfFw810+9ntveu41df+xyOjQTq6IkyQQyD2EEUAWYgOeaCL2AH1X14UKt2HNdhet98xlEpAvQVFWTsryuJ9AToHLlyo127oye8snhnI0c0YXyculL/evEX4z8bCTPL3ueInFFGNZiGA9d/hBF44uGNcSA2RiCc0I5p8Llf9dgFrd7HFiIZ7ZyX+/txwoXHuAZoM7qtOykqq+pamNVbVy+fPkgrNYdwtmVE42F8nyKFynOU9c8xZY+W0i8OJGH5z1Mo9casfyH5U6HZmJBtHUFqmrAC3AOUC8/v5PLezUD5vrdfwJ4IrffadSokUaTjAzV/v1VPbtmz9K/v+fxUKzD995Z77tWYqJn8W0c3/0cZGRk6MytM7XyqMpKMtp9Znfde3hvmII1ESOP71GB3ivA76hTgDUawH45kLOMFovI2d6S1+uAySLychBy0WrgUhGpKiLFgDuBWUF434gR7Gsw57SO7C7c079//i/c43YiQscaHdnSZwuDrxzMlA1TqD6uOq+tfY0MzXA6PBONHLiqWSgFMobwlao2EJH7gYtUdaiIbFDVeoVeuUg7YDQQD0xS1Rdye320zUPw7ybyCdVgr9PzEAq1/gL2z27Zt4W+s/uyeMdiLq94OantU2lYoWG+3sOYgMTQGEIREakA3A58XOjI/KjqbFW9TFWr5ZUMok24r3kQrAv3FIRT8yBqla/Foq6L+Henf7PzwE6avN6EfnP6cfDowdCu2MSeCG8Z+ASSEJ4F5uI5RXS1t5bR9tCGFf1ipSsnKIPnhfhnExE61+vMtqRt9Gnch5TVKVQfV523N75NXq1jY2KNXSDHYU535YRDOLvG8rJ291p6f9Kb1btX07JKS1LapVCzfM3wBmFMmAXaZWQJwYSFm+ZBpGek868v/8XghYP589ifPNL8EZ6+5mnOLHqmMwEZE2LBHEMwplDcNg8iPi6eBxs/SFpSGp3rdebvy/9OrZRazEqLqZPcjDmNJQQTUv7dRf36ZR48HzDA2clx5511HpM7TmbpvUspVbwUHd/pSIdpHdhxYIdzQRnjoFwTgojUEJHWIlIyy+PXhzYs9wt1DaJo4Rs8b9r01GOjRnmSw6pVMGyYc7H5XH3x1XzZ80tGth3Jou8XUSulFi8ue5G/TgSxynukz2A14eXQ9yXHhCAi/YD/AA8Bm0TEv/Dci6EOzM2ioZx0OA0d6kkIY8dm7jpatco9FVeLxhfl4eYPsy1pG+0va89Ti56i/oT6LPxuodOhGRM+OU1hBjYCJb23qwBrgP7e+18FMg062IsbSldEdCmIIMv6WXP77OEo0xFMc7bP0WpjqinJ6F3v36W7/9hdsDeKkNIGxiVC9H0hCKUr4lX1sDdp7ABaADd4y1ZE2YmRgYvGctIFkd9WUkjLdISgeX39365nU59NJCcm8+HWD6mRUoOxq8ZyIuNEUNdjjKvklCmARUBClseKAG8B6YFkm2Avbmgh+GRkZD7adeuRbigUpJUU0hZCiI+6t+/frtdNuU5JRhMmJOjKH1fm/02sZWDyI8jfFwJsIeSWECoBF+Tw3JWBvHmwF7ckhEjr/giF/GyDkHWzhbE7JiMjQ9/f/L5W/GdFJRl9YNYD+uufv+Y/VmMC4VBCyLHLSFV3qerPOTy3IpitlEji6xoJVw0it8pPF1A0lOkQEW6pdQtb+27lkWaPMOmrSVQfV51JX00KrJJqlNS6MWHi0PfFZioXQDivdOZW/onRJ69xFA1VmQ4HKk1u/GUjfWb3YfkPy2l+UXNS26dS7/xCFwA2JiRspnIIJSdn3vH5jnhjMRnkp5XkZMXVYKt7fl2W3ruUNzq+wdf7v6bhqw0ZNHcQh/465HRoxhRYwAnBd5Ec3xLKoCJBNO3c8st1XUBONa9F6JbQjbSkNB5o+ACjPx9NjZQavLv5XSKp5W2MTyAXyHkQTwnsI5y65rGq6iUhju00bukyMh4h6wKKUF/89AW9P+nNl3u+pO0lbRnXbhyXlbvM6bCih8svQuNmwewyegSorapVVLWqdwl7MjDBkTX/F+ZANpZbSdm5vOLlfHH/F4y7YRyrflpF3dS6DPl0CEeOH3E6NGMCEkhC+Bb4X6gDMaFnJTdCLz4unr6X9yUtKY3ba9/Oc0ufo/b42szePtvp0CJDdpMMfY8tWeJZrC5UyASSEJ4APhORV0VkrG8JdWAmuDQYVy4zAbug5AVM6TSFRV0XUbxIcdq/3Z6bp9/MDwd/cDo0Y3IUyBjCF8ByPLWNTp5wrapvhja009kYQuEU5FRRU3jH0o/x8sqXeXbJs4gIQxOHMvCKgRSNL+p0aO7hO+JfssTzMzHR89N/vMDGEAosmGMIJ1R1kKpOVtU3fUsQYjRhFtJ6QiZHxeKLMfiqwWztu5W2l7Tl8QWPk/BqAkt3LnU6NGMyCSQhfCoiPUWkgp12Gtl8LQR/sTS72mkXl7mYmXfOZNads/jf8f+R+EYi3WZ245fDvzgdmvN8pw4nJnqW7E4lttneIRdIQrgb7zgCsNa7WL9NhLGSG+5xU/Wb2NxnM09d/RTTNk6jRkoNUlenkp6R7nRoJsZZ6YoYYiU33Cft1zT6zu7Lwu8X0vjCxqS2T6XxhXl29RqTL4GOIQSUEESkDlALKOF7TFXfKlSEBWAJofBsMpn7qCrTN09n4NyB/HL4F3o37s3zrZ6n7BllnQ7NRImgDSqLyFDgFe/SEvgH0KHQERpH2GQy9xER7qxzJ9v6bqNf035MWDuBGik1mLJ+ipXAMGEVyBjCrUBr4GdV7Q7UB4qHNCoT04I5mzqSlC5RmtEvrWPtFw24pOwldJ3ZlRZvtmDz3s1Oh2ZiRCAJ4YiqZgAnRORsYC9gpStiWCh32G6ZTe1kUko4XJIVPVbw+k2vs2nvJhJeTeDx+Y9z+Njh8AVhYlIgCWGNiJQBXsdzhtGXwBchjcq4Vih32G6ZTe1IUspSniGuZSvuH/Rv0pLS6Fa/G//47B/USqnFjK0zrBvJhEyeCUFV+6jqAVWdALQFunm7jkyMCfUO27+M9pgxEBd36jTZcE2gc0tS8jn3zHP5V4d/saLHCsqeUZab372ZG6fdyHe/fxfeQExsyOsam8B9We7HA0MDuT5nsBe3XFM5loXjetIZGZnfP9zXqnb0mtm5XEv3ePpxHbVylJZ8saSWeL6EPrv4WT16/GgYgjKRjsJeU9lPaxGZ7Z2pXAf4HCgVsgxlXC3U5S+cnk3tOw3XjSU+isQVYcAVA9jWdxsdq3dkyOIh1E2ty/xv5zsbmIkagXQZ3Q28iae43WxggKo+EurAjDvltMPOyDj9dQV9b6dmU/vGDjIyHExKAZRnqHh2Rd659R3m3TMPgGv/fS13vH8HP/3xU+jjM1EtkHkIlwL9gQ+AHUAXETkzxHGFRKyezhgsue2wGzU6lRQKOgjr5KU5/ccOGjXy/ExI8DyXkODOEh9tq7VlY++NPNfyOWalzaJGSg1GrRzFiYwTTodmIlVefUrANqC197YADwObA+mPCvZSmDGEoUMz9wP7+omHDi3wW8akrNsxPV01ISFzP7uv/72g/e5ZfydcYwg5jR2kp7v/u/Ltb99qu6ntlGS0Xmo9Xb5zudMhGRchwDGEQBLC2dk8dmkgbx7spaAJIbudVGF3WrEs6/by7TCz25FGmpwGtCPhO5KRkaEzts7Qi16+SElGe8zsofv+3Od0WMYFAk0IOXYZichj3hbEHyJyW5anI+q0UzeczhhNsm6vuDgoXTrzYy+/DIMGRVbRvNwGtCPhOyIi/F+N/2Nr3608fuXjvLXhLaqPq87ra18nQzPyfgMT83IbQ7jT7/YTWZ67PgSxhJRbzxyJBhkZMGtW5sd8/fCRcnlOpwe0g+msYmfxUpuXWN9rPXXPq0vPj3vSfGJzvtrzldOh5cyuk+wKuSUEyeF2dvfzRURuE5HNIpIhImGp9Zvb0Z8pOFVPS2DdulODsHDq/ssvR0bSdXJAO1Rqla/Fp90+ZUqnKXx/4Hsav96Y/nP6c/DoQadDM26VU18S8GV2t7O7n98FqAlUBxYDjQP9PbeOITg1COoWvoHm9PTM/e9DhjgdWf5F69/y9yO/a99P+qoki14w8gJ9e8PbmpHbh8tlglxQ+dbj+9KEa70xhiBMTKsvIn+IyCGgnve2737dQiahraqaVpj3yI9QHv25pRibk5KTT40Z+MvaXRQJrbFwlQfPui1CvW3KlCjDuHbj+OKBL6h0diXu/vBu2kxpw7Zft4V2xSayBJI1QrUQphaCT7CP/uzsJY/sPnfTpp77/fp57ttpvqc4fQr0ifQTmro6Vcu8VEaLPltUn1zwpP557E/Pk04dsVvLIKQIYumKAhGRBSKyKZulYz7fp6eIrBGRNfv27StkTLnfL8j72dlLp7fAAJo29fxctcrz08kCcW6iLiieFx8XT69H3iFtQU3urns3Ly5/kVoptfgo7aPQr9y4WyBZI1QLYW4hhIrTxdjcwv+INyPD0zpwpECcyzlaPM/H74h8yY4lWjultpKMdpjWQb///Xs7Yo82HDorAAAbC0lEQVQyBNhCCOiayqEiIouBR1Q1oAslu/Gayv5HeD6x1kLwl5zsOdL1tRTi/NqgGRmxuU2yo+rQtvGd2rlkiednYiIAxxfOZ/Tno0lekoyq8kzaBTz8YyWKfbo0DEFFCd+2zaMWlROCdk3lUBCRTiKyC2gGfCIic52Io7D8k0Gkn7seDP7dIQMGeBZ/AwbE3jbJju9748/p70vR+KI8euWjbOu7jRsuvYEnq31P/Vv3sej7Rc4FFUo27yFbRZxYqarOAGY4se5gyunsJYjcc9cLw/f5VWHs2FOP9+vn+Tl27KnXxNq28cl6EDFqVOYWZsi3je/oNYej2YtKX8QHt3/A7O2zeWjOQ7R+qzWd63Zm5LUjuaDkBXm/v4uPkkMma6srgreBIwnBSZqlDEHW+/l9j+TkzM39WN/hicDo0ZkTwujRp56LxUTpL1IOItpd2o6WVVry0vKXeGnFS3z09Ue80OoFejfuTXxcvNPhFVwU7bxDwdExhPwq7BiCf/+2yKmjtTJlAp83EIz3iGa5jamAe3Z4TgvGgUm4bN+/naQ5Scz7dh4NLmhAavtUmlZqmvlFOYxNuG5HG8o4XZxcXD2G4IT8nu6X3X03nDLoZnmNqZhTwjUBLhguLXcp/+38X9677T32/rmXZhOb0evjXvx25LfQrTRUffy+CxAlJnqWAC5IFEtipsvIv2k+ZsypI9jszgjKrRUQ6HvEokjpDnFSJLUM/IkIt9a6leuqXcewJcMY/floPtj6ASPajqBr/a7E5TE24ePKzx+sI/toSCyBnJvqliUY8xDymjMQyOxjm3eQu0ioB+REjE7PUA6mDT9v0CsnXqkko1dNuko3/LzB80Qu8xcC+vxOzJSOgTkXOD1T2Y0COd0vr9nHvt/J7T1indu7Q5yoPxVt3Y11z6/L0u5LmdxxMtt+3UaDVxvwyLxHODT3o2yPlF35+X3dUkuWeBY7FTV2Wgj5rTuUXSvAahdFPif/hm6YoRyKltH+/+3XnrN6qiSLVvxnRX1v83vZVlLN1+cPZ8sgBiqtEqxLaLppKWyXUaBN9ty+uNHU7I9VTu6YnexuDPV3d+WPK7XBhAZKMnrdlOt0+/7tp70m4M8fzp1zFCcCH0sIOcjrCCnQMYTc3sO4X7h2zP7v62R9p3C1jI6nH9exn4/Vs/9+thZ/rrgO/XSoHjl+5LQYXFXfyhJC7CaEQFgrILqFa8fk/z3yTwZNmzrT3RjOHfLuP3br3R/crSSj1cZU09lfz7HuVgdZQigkawVEp3AdKbv1GhHh7rJa+N1Crf5KdSUZPbfPLVqr2Q+anu55Lj1dNSEh6g/OXSHQhBBTZxnlh9vPlDEFE65rJ2d3ttqqVZ66TqNHe573vSZcM9zVgaJ6raq2YkPvDbzQ6kUOlJ/NlpY1SXxiJMdOHM90Le5QxmACF1OlK0zk0yBNbArW+wSyHjeUAPclg+yK6oVrYuX3v++gzcj+fFdsFuytDR+n0r/T1TapMwysdIWJOsGcPxCOFqATR+Q5CVfLKDdVy1bhm+f/A9P+A8UOQ49rONDiXvb9b2/oV24CE0i/klsWt14xzYRepM0BcWu8To6NZRrULvqn0upJjRtaVMu8VEZTV6fqifQT4QumACJ5XBEbQzDRJNKuX+2GI/Kc4srtfqhk7bLK+OtM+td9gYyUDZQ81JDen/Sm+aTmrN29NjwB5VNBWqdZW4KR0DtvYwgmorilTz5Q4RqriAQ5FY0sXUapccs7DJo3iL1/7qVP4z481+o5ypQo43TIQMHGX9xWJj/QMQTHu4Hys1iXUWxz7cQmE7Dcul0OHDmg/Wb307hhcXr+iPN1yvop2ZbAcEJ+vntu7C7E5iGYaOLGfzITGl/u/lKbvt5USUYTJyfq5r2bnQ5JVfM3h8NtBy+BJgQbQzARwa198ib4GlRowGf3fcZrN77Ghl82UH9CfQYvGMyfx/50LCZfl4+/3M4Y878WiI8bx7pOE0jWcMtiLQQTyWd6mPzbe3iv9pjZQ0lGK4+qrDO2zgh7N1JBWqfWQjAmDGwGeWwpf1Z5JnacyPLuyyldvDSdpnfipmk38d3v34Uthvy2TlVzv5RsTq0KN7CzjIyrqZ2lY7xOZJzglVWvMGTxEE5knOCpq5/i0eaPUrxI8bCsPz/fxUg9y8gSgnEtt/1TGXf46Y+fGDRvEO9ufpfLyl1GSrsU2lzSxumwTuOmgxkrXWEimqoLL7loXKHi2RWZfut05t4zlwzNoO2Uttz5/p3sPrTb6dAyicTuTWshGNfyTwI+bp2ZbJxx9MRRRqwYwQvLXqBYfDGea/kcfS/vS5G4Ik6H5irWZWSigkbYzGTjjG9/+5aH5jzEnG/mUP/8+qS2T6XZRc2cDss1rMvIRLz8nvttYle1c6rxyd2f8OHtH7L/yH6aT2rOA7MeYP//9jsdWkSxhGBcKZJP3TPOEBE61ezE1r5beaz5Y7yx/g2qj6vOxC8nkqEZTocXESwhGFeymcmmoEoWK8nwtsNZ9+A6ap9Xm/s/up+rJl3F+p/XOx2a69kYgnE1N526ZyKPqvLvDf/m4XkPs//Ifvpd3o9hLYdxdvGznQ4trGJmUPn48ePs2rWLo0ePOhSVyapEiRJUqlSJokWLOh2KMQD8fuR3nl70NKlrUrmg5AWMum4Ut9e+HYmRo4uYSQjff/89pUqVoly5cjHzx3UzVWX//v0cOnSIqlWrOh2OMZms/mk1vT/pzdo9a2lzSRtS2qVwWbnLnA4r5GLmLKOjR49aMnAREaFcuXLWYjOu1KRiE1bdv4qUdims/mk1dVPr8syiZzhy/IjToblCxCcEwJKBy9jfw7hZfFw8fZr0IS0pjTtq38Hzy56n9vjafPL1J06H5rioSAhuMGPGDESEbdu2Zfv8vffey/vvvx/w++3evZtbb70VgHXr1jF79uyTzy1evJjPPvss3zFWqVKFX3/9Nd+/Z6JL1l7iCOo1DqrzS57PW53eYnG3xZxR9AxunHYjnaZ34oeDPzgdmmMsIQTJtGnTuOqqq3jnnXeC8n4XXnjhyQQSrIRgTEEuFh/tEqsk8tWDXzG8zXDmfTuPmik1Gb58OMfSjzkdWtjFZkJo0cKzBMnhw4dZsWIFEydOPJkQVJWkpCRq1apF+/bt2bt378nXV6lShSeffJJmzZrRuHFjvvzyS6677jqqVavGhAkTANixYwd16tTh2LFjDBkyhOnTp5OQkMDw4cOZMGECo0aNIiEhgWXLlrFv3z5uueUWmjRpQpMmTVixYgUA+/fv59prr6VBgwY8+OCDRNIJBCb4rGBgzorFF+OxKx9ja9+tXFftOgYvHEzChAQW71jsdGjhFchVdIK9ACOAbcAGYAZQJpDfy+6KaVu2bMn/5YMSEz1LkEyZMkV79OihqqrNmjXTtWvX6gcffKBt2rTREydO6E8//aSlS5fW9957T1VVL774Yh0/fryqqg4YMEDr1q2rf/zxh+7du1fLly+vqqrff/+91q5dW1VVJ0+erH379j25vqFDh+qIESNO3r/rrrt02bJlqqq6c+dOrVGjhqqqPvTQQzps2DBVVf34448V0H379gXtc+emQH8XE3Juu5KXW32c9rFWHV1VSUbv+fAe/fnQz06HVCgEeMU0p0oCzgeeUNUTIjIceAJ4PORr9bUKlizJfH/x4kK97bRp0xgwYAAAd955J9OmTeP48ePcddddxMfHc+GFF9KqVatMv9OhQwcA6taty+HDhylVqhSlSpWiRIkSHDhwIF/rX7BgAVu2bDl5/48//uDQoUMsXbqUDz/8EID27dtTtmzZwnxMEwV8M779K8ha9djTtb+sPa2qtuLvy//O8BXD+SjtI15o9QK9GvciPi7e6fBCxpGEoKrz/O5+DtzqRBzBsH//fhYtWsSmTZsQEdLT0z01VTp1yvVsm+LFPVd5iouLO3nbd//EiRP5iiEjI4OVK1dyxhlnnPacnfFj/OVUMNCSwunOKHoGz7Z8lnvq3UPS7CSS5iQxed1kUtun0qRiE6fDCwk3jCH0AOaEZU2LF3uWxETP4rtfCO+//z5du3Zl586d7Nixgx9//JGqVatyzjnn8M4775Cens6ePXv49NNPC7yOUqVKcejQoRzvX3vttYwbN+7k/XXr1gFwzTXXMHXqVADmzJnD77//XuAYTOSzgoEFc1m5y5h7z1zevfVd9hzeQ9N/NaXPJ334/Uj0/T+FLCGIyAIR2ZTN0tHvNU8BJ4CpubxPTxFZIyJr9u3bF6pwC2zatGl06tQp02O33HILP//8M5deeil169ald+/eJCYmFngdLVu2ZMuWLSQkJDB9+nRuuukmZsyYcXJQeezYsaxZs4Z69epRq1atkwPTQ4cOZenSpTRs2JB58+ZRuXLlQn1WE9msYGDBiQi31b6NrX23MuCKAby29jWqj6vOW+vfiqqTNRwrXSEi3YBeQGtV/V8gv5Nd6YqtW7dSs2bNEERoCsP+Lu5lBQMLb/3P6+kzuw+f/fgZV1e+mvHtx1PnvDpOh5UjV5euEJHr8Qwidwg0GRhjgiMSr/XrNvUvqM+y7suY2GEiW/ZtocGrDXhs/mMcPnbY6dAKxakxhHFAKWC+iKwTkQkOxWGMMQUSJ3H0aNCDtKQ0uid0Z8RnI6iZUpMPtnwQsd1IjiQEVf2bql6kqgnepZcTcRhjTGGVO7Mcr930GivvW8m5Z57Lre/dSru32/HNb984HVq+ueEsI2OMiXhXVLqC1Q+sZsz1Y1jxwwrqjK/DsMXDOHoicir/WkIwxpggKRJXhH5N+5GWlMbNNW8meUkydcbXYe43c50OLSCWEIwxJsgqlKrA27e8zYIuCygSV4Trp17Pbe/dxq4/djkdWq4sIQSBiPDwww+fvD9y5EiS8ygfOXPmzEzlJgoiv+WsZ82axUsvvZTt+t944w12796dr/X7CvAZY7LX+pLWrO+1nhdavcDHX39MzZSavLzyZY6nH3c6tGzFXEIIRS344sWL8+GHH+Zr5xyMhJBfHTp0YPDgwdmuvyAJwRiTt+JFivPk1U+ypc8WWlRpwcPzHqbRa41Y/sNyp0M7TUwlhFDVgi9SpAg9e/Zk1KhRpz23c+dOWrduTb169WjdujU//PADn332GbNmzeLRRx8lISGBb7/9NtPvfPTRRzRt2pQGDRrQpk0bfvnlFyDnctY7duygRo0a3H///dSpU4fOnTuzYMECrrzySi699FK++OILwLPTT0pKOm39w4cPZ82aNXTu3JmEhASOHDnC2rVrSUxMpFGjRlx33XXs2bMHgLVr11K/fn2aNWtGSkpK4TacMTGkatmqfHTXR/znzv9w8K+DXD35anr8pwf7/nRRBYZASqK6ZSlM+Wv/sr++cr9Z7xfUWWedpQcPHtSLL75YDxw4oCNGjNChQ4eqquqNN96ob7zxhqqqTpw4UTt27Kiqqt26dTtZDjur3377TTO8Ab3++us6aNAgVc25nPX333+v8fHxumHDBk1PT9eGDRtq9+7dNSMjQ2fOnHlynf5ltLOuPzExUVevXq2qqseOHdNmzZrp3r17VVX1nXfe0e7du6uqat26dXXx4sWqqvrII4+cLNGdlZW/NiZnh/86rE8seEKLPltUy75UVl9d86qmZ6SHbH24vPx12PnqtoCnmJev/K9/XZfCOPvss+natStjx47NVHV05cqVJ0tQd+nShcceeyzP99q1axd33HEHe/bs4dixY1StWhUg13LWVatWpW7dugDUrl2b1q1bIyLUrVuXHTt25OuzpKWlsWnTJtq2bQtAeno6FSpU4ODBgxw4cOBkXaYuXbowZ0546hIaE03OKnYWL7Z+kS71utB3dl8e/PhBJn41kdT2qTSs0NCxuGKqy8g/KfgEs+zvgAEDmDhxIn/++WcuMeS9soceeoikpCQ2btzIq6++ytGjp85jzun3s5bQ9i+vnd9y2qpK7dq1WbduHevWrWPjxo3MmzcPVbVy2sYEUc3yNVnYdSFTb57KzgM7afJ6E/rN6cfBowcdiSemEkJOteCDNcv8nHPO4fbbb2fixIknH2vevPnJy2pOnTqVq666Cji9hLW/gwcPUrFiRQDefPPNk48Hs5x1biW1q1evzr59+1i5ciUAx48fZ/PmzZQpU4bSpUuzfPnyk5/HGFM4IsLdde9mW9I2+jbpS8rqFKqPq87bG98OewmMmEkI4aoF//DDD2c622js2LFMnjyZevXqMWXKFMZ4+6ruvPNORowYQYMGDU4bVE5OTua2227j6quv5txzzz35eDDLWWdd/7333kuvXr1ISEggPT2d999/n8cff5z69euTkJDAZ599BsDkyZPp27cvzZo1y/aCPMaYgilTogxjbxjL6gdWc3GZi+n8YWdav9Warfu2hi0Gx8pfF0Rhy18nJ3suJu7rJvIliTJlCn+mkcnMyl8bU3AZmsHra1/niYVPcPjYYYa3Gc7AZgPz/sUcBFr+OmYGlcGz0/ev/e4bU7BucWOMm8RJHA82fpCba97M4wsep9o51cKy3phKCGC14I0xkaP8WeWZ1HFS2NYXM2MIxhhjchcVCSGSxkFigf09jIlMEZ8QSpQowf79+20n5BKqyv79+ylRooTToRhj8inixxAqVarErl272LfPRfVAYlyJEiWoVKmS02EYY/Ip4hNC0aJFT5Z2MMYYU3AR32VkjDEmOCwhGGOMASwhGGOM8Yqo0hUisg/Y6ffQuUDglylzjsUZPJEQI1icwWZxFs7Fqlo+rxdFVELISkTWBFKfw2kWZ/BEQoxgcQabxRke1mVkjDEGsIRgjDHGK9ITwmtOBxAgizN4IiFGsDiDzeIMg4geQzDGGBM8kd5CMMYYEyQRlRBEZISIbBORDSIyQ0TK5PC660UkTUS+EZHBDsR5m4hsFpEMEcnxjAMR2SEiG0VknYisyel1oZKPOB3bniJyjojMF5Ht3p9lc3hdunc7rhORWWGML9dtIyLFRWS69/lVIlIlXLFliSOvOO8VkX1+2/B+B2KcJCJ7RWRTDs+LiIz1foYNItIw3DF648grzhYictBvWw4Jd4wFpqoRswDXAkW8t4cDw7N5TTzwLXAJUAxYD9QKc5w1gerAYqBxLq/bAZzr4PbMM06ntyfwD2Cw9/bg7P7m3ucOO7D98tw2QB9ggvf2ncB0l8Z5LzAu3LFlieEaoCGwKYfn2wFzAAGuAFa5NM4WwMdObsuCLhHVQlDVeap6wnv3cyC7kpqXA9+o6neqegx4B+gYrhgBVHWrqqaFc50FEWCcTm/PjsCb3ttvAv8XxnXnJZBt4x//+0BrkbBfp8/pv2FAVHUp8FsuL+kIvKUenwNlRKRCeKI7JYA4I1ZEJYQseuA5WsiqIvCj3/1d3sfcSIF5IrJWRHo6HUwOnN6e56vqHgDvz/NyeF0JEVkjIp+LSLiSRiDb5uRrvAczB4FyYYkumxi8cvob3uLtinlfRC4KT2j54vR3MT+aich6EZkjIrWdDiZQrit/LSILgAuyeeopVf2P9zVPASeAqdm9RTaPBf1UqkDiDMCVqrpbRM4D5ovINu/RR9AEIc6Qb8/cYszH21T2bstLgEUislFVvw1OhDkKZNuE5fuYh0Bi+AiYpqp/iUgvPK2aViGPLH/csC0D8SWeUhGHRaQdMBO41OGYAuK6hKCqbXJ7XkS6ATcCrdXbYZfFLsD/6KYSsDt4EXrkFWeA77Hb+3OviMzA07QPakIIQpwh3565xSgiv4hIBVXd4+0e2JvDe/i25XcishhogKffPJQC2Ta+1+wSkSJAacLf3ZBnnKq63+/u63jG6NwmLP/bhaWqf/jdni0i40XkXFV1Y42jTCKqy0hErgceBzqo6v9yeNlq4FIRqSoixfAM5IXtrJNAichZIlLKdxvPgHm2Zy04zOntOQvo5r3dDTitVSMiZUWkuPf2ucCVwJYwxBbItvGP/1ZgUQ4HMqGUZ5xZ+uI7AFvDGF+gZgFdvWcbXQEc9HUnuomIXOAbJxKRy/HsZ/fn/lsu4fSodn4W4Bs8fYjrvIvv7I0Lgdl+r2sHfI3nCPEpB+LshOdo5i/gF2Bu1jjxnPGx3rtsdmucTm9PPP3tC4Ht3p/neB9vDPzLe7s5sNG7LTcC94UxvtO2DfAsnoMWgBLAe97v7hfAJeH+OwcY59+938P1wKdADQdinAbsAY57v5f3Ab2AXt7nBUjxfoaN5HIGn8NxJvlty8+B5k7EWZDFZiobY4wBIqzLyBhjTOhYQjDGGANYQjDGGONlCcEYYwxgCcEYY4yXJQQTFbJUO10XzqqseVW/NCZS2GmnJiqIyGFVLenQuq8BDuMpvFYnTOuMV9X0cKzLxA5rIZioJSKlvdcAqO69P01EHvDeTvUWw9ssIsP8fmeHiLwoIiu9zzcUkbki8q23xs9pNIDql+K59sQmb8Gzpd7H4kVkpHiuibFBRB7yPt5aRL7yPj7Jbxb2DhEZIiLLgdtEpJqI/NdbHHGZiNQIxnYzsct1tYyMKaAzRGSd3/2/q+p0EUkC3hCRMUBZVX3d+/xTqvqbiMQDC0Wknqpu8D73o6o2E5FRwBt4SmGUwDP7dEIB4xsCXKeqP8mpCzv1BKoCDVT1hHguBlTCu87Wqvq1iLwF9AZGe3/nqKpeBSAiC/HMjt0uIk2B8bivIJ2JIJYQTLQ4oqoJWR9U1fkichuekgf1/Z663VtyvAhQAagF+BKCr87PRqCkqh4CDonIUREpo6oHChDfCjyJ6V3gQ+9jbfCUXznhjfU3EakPfK+qX3tf8ybQl1MJYTqAiJTEU7LjPTl1eYXiBYjLmJMsIZioJiJxeK4MdwQ4B0/V0arAI0ATVf1dRN7A0wLw+cv7M8Pvtu9+gf5nVLWX9yi+PbBORBLw1OYJpFy2vz+9P+OAA9klQWMKysYQTLQbiKdy513AJBEpCpyNZ8d6UETOB24IdRAiUk1VV6nqEOBXPGWc5wG9vGWxEZFzgG1AFRH5m/dXuwBLsr6fekosf+9t/fiuN1w/6+uMyQ9LCCZanJHltNOXROQy4H7gYVVdhudaE0+r6nrgKzxjApPwdOcUmIhMA1YC1UVkl4jcl83LRngHiTd541gP/Av4AdggIuuBu1X1KNAdT1fQRjytkpzGLToD93l/dzMuvCymiSx22qkxxhjAWgjGGGO8LCEYY4wBLCEYY4zxsoRgjDEGsIRgjDHGyxKCMcYYwBKCMcYYL0sIxhhjAPh/R7D7JonW+5AAAAAASUVORK5CYII=\n", 369 | "text/plain": [ 370 | "
" 371 | ] 372 | }, 373 | "metadata": {}, 374 | "output_type": "display_data" 375 | } 376 | ], 377 | "source": [ 378 | "plt.scatter(X[pos[:,0],1],X[pos[:,0],2],c=\"r\",marker=\"+\",label=\"Admitted\")\n", 379 | "plt.scatter(X[neg[:,0],1],X[neg[:,0],2],c=\"b\",marker=\"x\",label=\"Not admitted\")\n", 380 | "x_value= np.array([np.min(X[:,1]),np.max(X[:,1])])\n", 381 | "y_value=-(theta[0] +theta[1]*x_value)/theta[2]\n", 382 | "plt.plot(x_value,y_value, \"g\")\n", 383 | "plt.xlabel(\"Exam 1 score\")\n", 384 | "plt.ylabel(\"Exam 2 score\")\n", 385 | "plt.legend(loc=0)" 386 | ] 387 | }, 388 | { 389 | "cell_type": "markdown", 390 | "metadata": {}, 391 | "source": [ 392 | "### Prediction" 393 | ] 394 | }, 395 | { 396 | "cell_type": "code", 397 | "execution_count": 16, 398 | "metadata": {}, 399 | "outputs": [], 400 | "source": [ 401 | "def classifierPredict(theta,X):\n", 402 | " \"\"\"\n", 403 | " take in numpy array of theta and X and predict the class \n", 404 | " \"\"\"\n", 405 | " predictions = X.dot(theta)\n", 406 | " \n", 407 | " return predictions>0" 408 | ] 409 | }, 410 | { 411 | "cell_type": "code", 412 | "execution_count": 14, 413 | "metadata": {}, 414 | "outputs": [ 415 | { 416 | "name": "stdout", 417 | "output_type": "stream", 418 | "text": [ 419 | "For a student with scores 45 and 85, we predict an admission probability of 0.7677628875792492\n" 420 | ] 421 | } 422 | ], 423 | "source": [ 424 | "x_test = np.array([45,85])\n", 425 | "x_test = (x_test - X_mean)/X_std\n", 426 | "x_test = np.append(np.ones(1),x_test)\n", 427 | "prob = sigmoid(x_test.dot(theta))\n", 428 | "print(\"For a student with scores 45 and 85, we predict an admission probability of\",prob[0])" 429 | ] 430 | }, 431 | { 432 | "cell_type": "markdown", 433 | "metadata": {}, 434 | "source": [ 435 | "### Accuracy on training set " 436 | ] 437 | }, 438 | { 439 | "cell_type": "code", 440 | "execution_count": 17, 441 | "metadata": {}, 442 | "outputs": [ 443 | { 444 | "name": "stdout", 445 | "output_type": "stream", 446 | "text": [ 447 | "Train Accuracy: 89 %\n" 448 | ] 449 | } 450 | ], 451 | "source": [ 452 | "p=classifierPredict(theta,X)\n", 453 | "print(\"Train Accuracy:\", sum(p==y)[0],\"%\")" 454 | ] 455 | }, 456 | { 457 | "cell_type": "code", 458 | "execution_count": null, 459 | "metadata": {}, 460 | "outputs": [], 461 | "source": [] 462 | } 463 | ], 464 | "metadata": { 465 | "kernelspec": { 466 | "display_name": "Python 3", 467 | "language": "python", 468 | "name": "python3" 469 | }, 470 | "language_info": { 471 | "codemirror_mode": { 472 | "name": "ipython", 473 | "version": 3 474 | }, 475 | "file_extension": ".py", 476 | "mimetype": "text/x-python", 477 | "name": "python", 478 | "nbconvert_exporter": "python", 479 | "pygments_lexer": "ipython3", 480 | "version": "3.6.5" 481 | } 482 | }, 483 | "nbformat": 4, 484 | "nbformat_minor": 2 485 | } 486 | -------------------------------------------------------------------------------- /LogisticRegression/ML_RegularizedLogisticRegression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Machine Learning - Andrew Ng ( Python Implementation)\n", 8 | "\n", 9 | "## Regularized Logistic regression" 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "metadata": {}, 15 | "source": [ 16 | "### Loading the data" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": 1, 22 | "metadata": {}, 23 | "outputs": [], 24 | "source": [ 25 | "import numpy as np\n", 26 | "import pandas as pd\n", 27 | "import matplotlib.pyplot as plt" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": 2, 33 | "metadata": {}, 34 | "outputs": [ 35 | { 36 | "data": { 37 | "text/html": [ 38 | "
\n", 39 | "\n", 52 | "\n", 53 | " \n", 54 | " \n", 55 | " \n", 56 | " \n", 57 | " \n", 58 | " \n", 59 | " \n", 60 | " \n", 61 | " \n", 62 | " \n", 63 | " \n", 64 | " \n", 65 | " \n", 66 | " \n", 67 | " \n", 68 | " \n", 69 | " \n", 70 | " \n", 71 | " \n", 72 | " \n", 73 | " \n", 74 | " \n", 75 | " \n", 76 | " \n", 77 | " \n", 78 | " \n", 79 | " \n", 80 | " \n", 81 | " \n", 82 | " \n", 83 | " \n", 84 | " \n", 85 | " \n", 86 | " \n", 87 | " \n", 88 | " \n", 89 | " \n", 90 | " \n", 91 | " \n", 92 | " \n", 93 | "
012
00.0512670.699561
1-0.0927420.684941
2-0.2137100.692251
3-0.3750000.502191
4-0.5132500.465641
\n", 94 | "
" 95 | ], 96 | "text/plain": [ 97 | " 0 1 2\n", 98 | "0 0.051267 0.69956 1\n", 99 | "1 -0.092742 0.68494 1\n", 100 | "2 -0.213710 0.69225 1\n", 101 | "3 -0.375000 0.50219 1\n", 102 | "4 -0.513250 0.46564 1" 103 | ] 104 | }, 105 | "execution_count": 2, 106 | "metadata": {}, 107 | "output_type": "execute_result" 108 | } 109 | ], 110 | "source": [ 111 | "df=pd.read_csv(\"ex2data2.txt\", header=None)\n", 112 | "df.head()" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": 3, 118 | "metadata": {}, 119 | "outputs": [ 120 | { 121 | "data": { 122 | "text/html": [ 123 | "
\n", 124 | "\n", 137 | "\n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | " \n", 180 | " \n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " \n", 188 | " \n", 189 | " \n", 190 | " \n", 191 | " \n", 192 | " \n", 193 | " \n", 194 | " \n", 195 | " \n", 196 | "
012
count118.000000118.000000118.000000
mean0.0547790.1831020.491525
std0.4966540.5197430.502060
min-0.830070-0.7697400.000000
25%-0.372120-0.2543850.000000
50%-0.0063360.2134550.000000
75%0.4789700.6465621.000000
max1.0709001.1089001.000000
\n", 197 | "
" 198 | ], 199 | "text/plain": [ 200 | " 0 1 2\n", 201 | "count 118.000000 118.000000 118.000000\n", 202 | "mean 0.054779 0.183102 0.491525\n", 203 | "std 0.496654 0.519743 0.502060\n", 204 | "min -0.830070 -0.769740 0.000000\n", 205 | "25% -0.372120 -0.254385 0.000000\n", 206 | "50% -0.006336 0.213455 0.000000\n", 207 | "75% 0.478970 0.646562 1.000000\n", 208 | "max 1.070900 1.108900 1.000000" 209 | ] 210 | }, 211 | "execution_count": 3, 212 | "metadata": {}, 213 | "output_type": "execute_result" 214 | } 215 | ], 216 | "source": [ 217 | "df.describe()" 218 | ] 219 | }, 220 | { 221 | "cell_type": "code", 222 | "execution_count": 4, 223 | "metadata": {}, 224 | "outputs": [], 225 | "source": [ 226 | "X=df.iloc[:,:-1].values\n", 227 | "y=df.iloc[:,-1].values" 228 | ] 229 | }, 230 | { 231 | "cell_type": "markdown", 232 | "metadata": {}, 233 | "source": [ 234 | "### Plotting of the data" 235 | ] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": 5, 240 | "metadata": {}, 241 | "outputs": [ 242 | { 243 | "data": { 244 | "text/plain": [ 245 | "" 246 | ] 247 | }, 248 | "execution_count": 5, 249 | "metadata": {}, 250 | "output_type": "execute_result" 251 | }, 252 | { 253 | "data": { 254 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEKCAYAAAA1qaOTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuUVOWZ7/HvE4JBR42i6KCA4Do4YhBR2wveABXF0UiyxAQdFRMNMaPxzOQkEY+JRk6cEOOKCUczDuOVTA4qJiqZaBCjGCdIYmsaRbxwEWMDAmlixgsgl+f8sXc3RVNVXd297/X7rFWrq/al9tO7q+vZ72W/r7k7IiIi3fWxtAMQEZFiUEIREZFIKKGIiEgklFBERCQSSigiIhIJJRQREYmEEoqIiERCCUVERCKhhCIiIpH4eNoBJGnffff1gQMHph2GiEiuvPDCC3929z4dbVdXCWXgwIE0NjamHYaISK6Y2Vu1bKcqLxERiYQSioiIREIJRUREIqGEIiIikVBCERGRSCihiIhIJJRQREQkEkooIiISibq6sVGknLmL1/DsknWcPLgPYw7bP+1wRHJLJRSpa3MXr+HqmX9kxnNvcfXMPzJ38Zq0QxLJLSUUqWvPLlnHhs1bAdiweSvPLlmXckTxmrt4Ddc/ukiJU2KhhCJ17eTBfdi1Zw8Adu3Zg5MHdzj+XW6pNCZxUxuK1LUxh+3PtAuOrIs2lHKlsSL/vpI8JRSpe2MO278uvlhPHtyHWY3NbNi8tfClMUmHEopInain0pikQwlFpI7US2lM0qFGeRERiUSqCcXM7jaztWa2qMJ6M7NpZrbUzF4ys6NK1k00syXhY2JyUYuISDlpl1DuBcZWWX8WMDh8TAL+FcDMegM3AMcBxwI3mNnesUYqIiJVpZpQ3P23wPoqm4wDZnhgAbCXmfUFzgTmuvt6d/8LMJfqiUmq0M1unaPzJVJe1hvlDwTeLnndHC6rtFw6qfVmtw2btzKrsZlpFxxZF422XR2/q17Pl0gt0q7y6oiVWeZVlu/8BmaTzKzRzBrXrSv2sBpdUW9Dj0D37hivx/MlUqusJ5RmoH/J637AqirLd+Lu0929wd0b+vTRjVztZW7okVGjgkeMupMUMne+RDIk61Ves4GrzOx+ggb4v7r7ajObA/xLSUP8GcC1aQWZZ/V4s1t37hivx/MlUitzL1tTlMzBzWYCo4B9gTUEPbd6Arj7HWZmwG0EDe4fAl9w98Zw3y8C/zt8q5vc/Z6OjtfQ0OCNjY1R/xoShdZSyTPPBD9Hjgx+zpsXy+E0B4pI7czsBXdv6Gi7VEso7n5BB+sduLLCuruBu+OIS4pPd4yLRC/rVV5SL1pLIq0llZhKJnmmUpVknRKKREJfdjuL8pyou7LkQdZ7eUkORDpx07x5hSidRD2ZlborSx4oodShqO/01pfdzqI+J+quLHmghFJn4pgGVl92O4v6nLR2V75kxEGq7qpCw+KkK9Vuw0lTt2G4/tFFzHjurbbXl4w4iCnjhnb7fdWGsjOdk2SVtjPt2rOHEm+EctFtWJIX1zSw6oa7M52TZJWrZtT5T5YSSp3Rnd5SVHFdLEntVOUlIoWhasZ4qMpLROqOqhnTpV5eIiISCZVQRCrRMDBlqVpJKlEJRYolgflU6lkc9zFJcaiEItJe+6H0VVJpo665Uo0SihSDkkAi1DVXqlFCkfRk9UtfQ+lXpPuYpBolFCmGnCWBqBu2k2woV9dcqSTVhGJmY4EfAz2AO919arv1twKjw5e7Afu5+17huq3Ay+G6P7n7uclELd2Wl+qpGKcfjnJuE82VIlmRWkIxsx7A7cAYoBl43sxmu/vi1m3c/Z9Ltv8qcGTJW2xw9+FJxSs5kbWkVEbUDdt5bShX9+PiSbPb8LHAUndf7u4fAfcD46psfwEwM5HIJF6tk2iNHBk8CjKpVq2iHto+j9MHqPtxMaVZ5XUg8HbJ62bguHIbmtlBwCDgqZLFvcysEdgCTHX3R+IKVCRKUTds57GhPK+lKqkuzYRiZZZVGqlyAvCQu28tWTbA3VeZ2cHAU2b2srsv2+kgZpOASQADBgzobswSpToqlbQXdcN23hrKo+p+rGqzbEkzoTQD/Ute9wNWVdh2AnBl6QJ3XxX+XG5m8wjaV3ZKKO4+HZgOwWjD3Y5aRLotilKVOiNkT5oJ5XlgsJkNAlYSJI0L229kZn8H7A08V7Jsb+BDd99kZvsCJwI3JxJ1jsR59ZbJK8Ms9RbLUiwZ1d1SlarNsie1Rnl33wJcBcwBXgUedPdXzGyKmZV2Ab4AuN93nLhlCNBoZguBpwnaUBYjbeJs9Ey1QVVjdUUvp+c0j50Rii7V+1Dc/THgsXbLrm/3+jtl9psPHB5rcDkX59Vb5q4Mu3pfSxyliLzcY1MAeeyMUHS6Uz5FcVYbxTnmUirjOemLOnpxnNOE/y5564xQdEooKYm7QTHOq7fMXRl2dtiVOJNTzoaAEYmSEkpKkqg2ivPqLfErw1q+qJuaEgqmIKJMfnVcgsxkB5WUKKGkRMOAx2B4jSPxJFGKqIMvUlHX5faUUFKSuWqjvFxRlouvjq+OI1Hwqr44SxCZ66CSMiWUFKlBMWUZ+tKTeMRdglBNw46UUOpdEa7uM3x1nFWxXbVn7NzHXYLIXE1DypRQROpMPdX7J1GCUE3DdkooeRXV1XiRru7zHHuC6qneXyWIZKU5H4qIpGCnIUvuvTV3Q6/MXbyG6x9dVNOwP2MO258p44YqmSTAdhwiq9gaGhq8sbEx7TC6p32bx8iRwU9dnRdDQiXFHdpQ/vHziRwzKqVVdrv27FHoKrusMLMX3L2ho+1U5SVSh8Yctv/2RJKzDhn1VGWXN0ooeVOkNo+0ZekcFqG3XULUVTe7lFCk2PTFXFlOL07U0J5dSih5lZN//kzKYmmg2pd7FuLLGHXVzSYllIRpILmEZDFpxKk7v19Rz4kkTgklQfV0Q1mmZbmqp1zJpF6SouReqgnFzMYCPwZ6AHe6+9R26y8FfkAw5zzAbe5+Z7huIvCtcPl33f2+RILuBvVOSVCWk0aUlHQkQ1JLKGbWA7gdGAM0A8+b2ewyc8M/4O5Xtdu3N3AD0AA48EK4718SCL3LCt07JY9fZFmPtV6SohRGmiWUY4Gl7r4cwMzuB8YB7RNKOWcCc919fbjvXGAsMDOmWCMRR+8Utcl0oOhfwkVMOhn6XfT/1TlpJpQDgbdLXjcDx5XZ7jwzOwV4A/hnd3+7wr4HxhVolKLsnZKJNhlVucRP5zIVmfj/ypk0x/KyMsvajwPzS2Cguw8DngRa20lq2TfY0GySmTWaWeO6deu6HGwWlWuTqcmoUdkauylr8eTRvHn5Tzytn4NnngkeKX8uuvz/VcfSTCjNQP+S1/2AVaUbuHuLu28KX/47cHSt+5a8x3R3b3D3hj59CtRmQZlB/tJok2n9Ihs5MngU4YtNhIz8f+VMmlVezwODzWwQQS+uCcCFpRuYWV93Xx2+PBd4NXw+B/gXM9s7fH0GcG38IWdLp9tkslY9lbV4JF0Zaw/SHfmdl1pCcfctZnYVQXLoAdzt7q+Y2RSg0d1nA1eb2bnAFmA9cGm473oz+z8ESQlgSmsDfb3JzB3DSgKSV1USWGb+v3JCw9fXo4xcAbbJWjxSX/T565CGrxcRqUZVrpFTQqlHWfuHyVo8ItIlSiginaUr2WLIWCeAItCc8iIiEgmVUERqpTr3YtLfLzIqoYiISCRUQhGplercRapSQpH80Re6FEiRRjRWQhHpLCUyiUjRRjRWQskLXZWrUVwKp2izuKpRXkQkJUUb0VgllKzL21V5nPGpUVwKpmgjGiuh1KhIDWcikh1FGtFYCaUGqTac5eWqPMmSVFbPgUidUxtKDTQVqIhIx1RCqcHJg/swq7GZDZu31txwFnkVWdavyvNSkhKR2FRMKGa2B/BNgvnaH3f3B0vW/V93/2oC8WVCZxvOita3XCR1Gb5QUfvqdtWqvO4GdgV+BVxqZg+YWc9w3YlRHNzMxprZ62a21Mwml1n/NTNbbGYvmdlvzOygknVbzawpfMyOIp5qxhy2P1PGDa3pA1PXVWTz5mXyn14kDq0XjzOee4urZ/6RuYvXpB1SqqollMHu/nV3f8jd/x5YDDxlZntHcWAz6wHcDpwFHAZcYGaHtdvsj0CDuw8DHgJuLlm3wd2Hh49zo4gpKkXrWy6SmlGjgsczzwSP1tcZUdcXj2VUa0PpZWYfc/dtAO5+o5k1A88Cu0dw7GOBpe6+HMDM7gfGESQuwmM+XbL9AuCiCI4bu6L1LReR8rrSvlpk1RLKr4DTgLmtC9z9LjN7B7gtgmMfCLxd8roZOK7K9pcBj5e87mVmjcAWYKq7PxJBTJEpUt9ykdRkvLOHLh53VDGhuPv/qrD8VwTJprus3NuX3dDsIqABGFmyeIC7rzKzgwmq4l5292Vl9p0ETAIYMGBA96OuNxn9RxbJCl08bpdmt+FmoH/J637AqvYbmdnpwHXASHff1Lrc3VeFP5eb2TzgSGCnhOLu04HpAA0NDWUTVt1QcpC80mc2F9JMKM8Dg81sELASmABcWLqBmR0J/Bsw1t3XlizfG/jQ3TeZ2b4Evc5KG+ylu/I2hpiIpK7DhGJmH3f3LR0t6yx332JmVwFzgB7A3e7+iplNARrdfTbwA4IOALPMDOBPYY+uIcC/mdk2gp5qU919cdkDiZKDiCSilhLKH4CjaljWae7+GPBYu2XXlzw/vcJ+84HDu3t8qSLjjaEikj3V7pTfD+gL7Gpmh7O9EX1PYLcEYpOoFC05FOX3ECmYaiWUs4EvEjSW3872hPIe8O2Y45Ks0Je25IEuMjKhWrfhe4B7zOxzpeN4SY7l/Z9NbUEimVZLG8p+Zranu/+3md1B0HZyrbv/JubYRESq00VGptSSUCa5+21mdgZB9ddXCO7rODrWyETaK1pbkEjB1JJQWm8GPAu4x91fMDNNzCUi6dNFRqbUklAWmtljwCHAdWa2OxWGSBFJhL40RDKploTyBYLqraXu/mF4Z/pl8YYlItIJusjIhA6rrtx9K3AwQdsJBJNuqcpLRCTj5i5ew/WPLkps4q8OE4OZ3QaMZvtcJB8Ad8QZlIiIdE8as0nWUtI4wd2/DGwEcPf1wC6xRiUiUue6W7pIYzbJWhLK5rBXlwOY2T7AtlijEhGpY1GULtKYirzaWF6tIwrfDvwc6GNmNwKfA26MPTIRkTpVrnTR2Um80phNslovrz8AR7n7DDN7ATidYDyv8919UeyRiYjUqajmqk96NslqCaVtil53fwV4Jf5w6svcxWvyPRe1biYTiUVe56qvllD6mNnXKq109x/GEE/daK0j3bB5K7Mam5l2wZG5+dBEQslIpKo8zlVfLaH0IJgt0apsI10URR1pajQgn9RCn4u6Uy2hrHb3KXEe3MzGAj8mSF53uvvUdus/AcwguFO/Bfi8u68I111LcMf+VuBqd58TZ6xRi6qONHf/tEpGImXlvgqcGttQ4mBmPQh6kI0BmoHnzWx2u7nhLwP+4u7/w8wmAN8HPm9mhwETgE8BBwBPmtkh4V39uZDXOlJAA/JJdbpo6LSiVIFXSyinxXzsYwnGB1sOYGb3A+OA0oQyDvhO+Pwh4DYzs3D5/e6+CXjTzJaG7/dczDFHqlt1pHn9p+1OMsrL7yjSSbmuAi9RbcbG9TEf+0Dg7ZLXzcBxlbZx9y1m9ldgn3D5gnb7HhhfqFKWvtilHJVgOy2yKvCU1TLacFzKVam1Hxa/0ja17Bu8gdkkYBLAgAEDOhNftuX9n7YrJZO8lcZEapTrKvASaSaUZqB/yet+wKoK2zSb2ceBTwLra9wXAHefTjDDJA0NDZrHRSQpCSf8vDdq57GbcHtpJpTngcFmNghYSdDIfmG7bWYDEwnaRsYDT7m7m9ls4P+Z2Q8JGuUHE9zZX3/q4So976UxiV1RGrXzLrV5TcJxwq4C5gCvAg+6+ytmNsXMzg03uwvYJ2x0/xowOdz3FeBBggb8XwNX5qmHl4hEK42RdWVnaZZQcPfHgMfaLbu+5PlG4PwK+94E3BRrgBK7TlVTqGQiFRSlUTvvUk0o0jV5rytupWoKiUpRGrXzTgklQVEkgiJ9CRel771kQ9YatYty4dcZmhs+IVFNx1mkuuI0JgASSUIa0+9mgRJKQqJKBEX6Em6tprhkxEG5LmmJtFekC7/OUJVXQqKcMKdIdcVZq6YQiUK9dhIw9/q516+hocEbGxtTO3491qmK1Ksi/b+b2Qvu3tDhdkooIlIougE2crUmFFV5yQ6KdFUlIslSQpE2ReqSLHVIg4imTr28pE299kyJ1ahR27/YRApOJRRpU689U6QgNIho6pRQpE3RuiR3SVRfRqp+2VnWz0HW48sBJRTZQSL3hegfV+Kkz1VqlFBEIPoShapftst6aS3r8eWIEookpyv/uPrnloIpctd8JZQMKvIHLrPiKlFkMREmnaSzXlpLML6id81XQsmYQn/gOvOPq2oIKaCiT9mQSkIxs97AA8BAYAXwOXf/S7tthgP/CuwJbAVucvcHwnX3AiOBv4abX+ruTUnEHreif+Ayr8gJK+0kncRxuvM7JRBf0bvmp1VCmQz8xt2nmtnk8PU17bb5ELjE3ZeY2QHAC2Y2x93fDdd/w90fSjDmRBT9AwfU9o+b9WoSkS4oetf8tBLKOGBU+Pw+YB7tEoq7v1HyfJWZrQX6AO+SI51tD8nbB07tPTlS5CSddumrE4o8ZUNaCWV/d18N4O6rzWy/ahub2bHALsCyksU3mdn1wG+Aye6+KbZou6ir7SGZ/cC1+ydNpL0nqhsMM/jFIlI0sSUUM3sS+Nsyq67r5Pv0BX4KTHT3beHia4F3CJLMdILSzZQK+08CJgEMGDCgM4futqK3hxT99yusIibXIpe+ciS2hOLup1daZ2ZrzKxvWDrpC6ytsN2ewK+Ab7n7gpL3Xh0+3WRm9wBfrxLHdIKkQ0NDQ6KTvxSmPaRCdcLJP3kgu79fjqpARIoirSqv2cBEYGr489H2G5jZLsDDwAx3n9VuXWsyMuAzwKL4Q+68vLWHdFbRfz/JIV0wpCqVGRvNbB/gQWAA8CfgfHdfb2YNwBXufrmZXQTcA7xSsuul7t5kZk8RNNAb0BTu835Hx9WMjTvrVKN6Hq/y8xizSMZkesZGd28BTiuzvBG4PHz+H8B/VNj/1FgDrBOFvolSRBKnO+XrWKcb1fN4lZ/HmEVySjM21rGTB/dh1549ALLXqC4iuaMSSh1To7qIREkJpc5l9iZKEckdVXmJiEgkVEKRQtIYY5KWev7sqYQihdPaHXrGc29x9cw/MnfxmrRDkjpR7589JRQpnHLdoduMGrX9ZkeRiFX97NUBJRQpnMS7Q2clSWUljjpW713x1YYinZb1OuKy3aE1WKQkoN674iuh5FgaX+x5Ga4lke7QWUlSWYmjjKxffMShnrviK6HkVFpf7LmdA0XzZSQuLxcfEh0llJxK64u9MHO8RCErSSorcbST24sP6TIllJxK64s9j3XEO1S7ZOTLth7o4qP+pDIfSlqKNh9KPdZPd1ZptcuuPXuo2iVh+owWQ6bnQ5Fo1HPjX61U7ZIufUbri+5DkUKr9/sCRJKUSgnFzHoDDwADgRXA59z9L2W22wq8HL78k7ufGy4fBNwP9AZeBC5294/ij1zyJo9tPiJ5ldac8jcD6919qplNBvZ292vKbPe+u+9eZvmDwC/c/X4zuwNY6O7/2tFxi9aGIiKShFrbUNKq8hoH3Bc+vw/4TK07mpkBpwIPdWV/kczS0CmSc2kllP3dfTVA+HO/Ctv1MrNGM1tgZq1JYx/gXXffEr5uBg6MN1wRiZWSaSHE1oZiZk8Cf1tm1XWdeJsB7r7KzA4GnjKzl4H/LrNdxXo7M5sETAIYMGBAJw4tkpAMD53SbVn4XRKIQd2jA7ElFHc/vdI6M1tjZn3dfbWZ9QXWVniPVeHP5WY2DzgS+Dmwl5l9PCyl9ANWVYljOjAdgjaU9us3b95Mc3MzGzdurP2Xkx306tWLfv360bNnz7RDkbwpQDLVEDPbpXUfymxgIjA1/Plo+w3MbG/gQ3ffZGb7AicCN7u7m9nTwHiCnl5l969Vc3Mze+yxBwMHDiRonpHOcHdaWlpobm5m0KBBaYeTTxkdOqVbspAoEopB9zptl1YbylRgjJktAcaErzGzBjO7M9xmCNBoZguBp4Gp7r44XHcN8DUzW0rQpnJXVwPZuHEj++yzj5JJF5kZ++yzj0p49SCOdo5584LHyJHBo/V1juhep+1SKaG4ewtwWpnljcDl4fP5wOEV9l8OHBtVPEom3aPzF5GcfZFWlYVSVwwxlGsr0b1O2+lO+Yx4+OGHMTNee+21RI73ox/9iA8//LBT+8ybN49zzjknpogks1pLJs88EzziLKlkWLX54scctj9Txg2t62QCSiiZMXPmTE466STuv//+RI7XlYRSb+YuXsP1jy7a4YujHo4d6fGzkCgiiqHe54uvhRJKV0R8hfb+++/zu9/9jrvuumuHhHLzzTdz+OGHc8QRRzB58mQAli5dyumnn84RRxzBUUcdxbJlywD4wQ9+wDHHHMOwYcO44YYbAFixYgWHHnooEydOZNiwYYwfP54PP/yQadOmsWrVKkaPHs3o0aMBeOKJJxgxYgRHHXUU559/Pu+//z4Av/71rzn00EM56aST+MUvfhHZ75x11a5GM3HsGO/b2On4P3kgtnaOtBNnZ6itpGNKKBnwyCOPMHbsWA455BB69+7Niy++yOOPP84jjzzC73//exYuXMg3v/lNAP7hH/6BK6+8koULFzJ//nz69u3LE088wZIlS/jDH/5AU1MTL7zwAr/97W8BeP3115k0aRIvvfQSe+65Jz/5yU+4+uqrOeCAA3j66ad5+umn+fOf/8x3v/tdnnzySV588UUaGhr44Q9/yMaNG/nSl77EL3/5S5599lneeeedNE9TotK8Gk37Sjip46eZtLuita3kkhEH1XXX4Go0fH1nxNQNcebMmfzTP/0TABMmTGDmzJls27aNL3zhC+y2224A9O7dm/fee4+VK1fy2c9+Fgju/4CgdPHEE09w5JFHAkGJZ8mSJQwYMID+/ftz4oknAnDRRRcxbdo0vv71r+9w/AULFrB48eK27T766CNGjBjBa6+9xqBBgxg8eHDb/tOnT+/W75oXaU4OVfXYCXSFrXh8dbfVcPwdUEJJWUtLC0899RSLFi3CzNi6dStmxnnnnbdT76lKA3m6O9deey1f/vKXd1i+YsWKnd6jXI8sd2fMmDHMnDlzh+VNTU1124MrzZ47afcaSur4mtGxgNy9bh5HH320t7d48eKdlnVo5MjgEYE77rjDJ02atMOyU045xadMmeIjRozwDz74wN3dW1pa3N39uOOO84cfftjd3Tdu3OgffPCBz5kzx4899lh/77333N29ubnZ16xZ42+++aYDPn/+fHd3v/zyy/2WW25xd/ehQ4f68uXL3d197dq13r9/f1+yZIm7u3/wwQf++uuv+4YNG7x///6+dOlSd3efMGGCn3322WV/jy6dR+m6CD+DaXrilXf824+87E+88k7aoUgVQKPX8B2rNpSUzZw5s60Kq9V5553HqlWrOPfcc2loaGD48OHccsstAPz0pz9l2rRpDBs2jBNOOIF33nmHM844gwsvvJARI0Zw+OGHM378eN577z0AhgwZwn333cewYcNYv349X/nKVwCYNGkSZ511FqNHj6ZPnz7ce++9XHDBBQwbNozjjz+e1157jV69ejF9+nTOPvtsTjrpJA466KBkTw75arSVzlN322Kp+znlX331VYYMGZJSRPFasWIF55xzDosWLYr9WHGcR80HL5INWZ8PRaRDafd2EpHOUUIpsIEDByZSOomL+v2L5It6eUlmpd3bSUQ6RwlFMk39/kXyQ1VeIiISCSUUERGJhBJKBvTo0YPhw4czdOhQPv3pT/Puu+92uM8JJ5zQpWM98sgjLF68uOMN29l99927dDwRqR9KKBmw66670tTUxKJFi+jduze33357h/vMnz+/S8fqakIREelIKgnFzHqb2VwzWxL+3LvMNqPNrKnksdHMPhOuu9fM3ixZNzz53yIeI0aMYOXKlW2vyw1LDzuWGCptM2PGDIYNG8YRRxzBxRdfzPz585k9ezbf+MY3GD58OMuWLWPZsmWMHTuWo48+mpNPPrltgq8333yTESNGcMwxx/Dtb387gd9cRHKvlvFZon4ANwOTw+eTge93sH1vYD2wW/j6XmB8Z48b1VheUY8/9Dd/8zfu7r5lyxYfP368P/744+7uPmfOHP/Sl77k27Zt861bt/rZZ5/tzzzzzA77VNpm0aJFfsghh/i6devcfftYYBMnTvRZs2a1HfvUU0/1N954w93dFyxY4KNHj3Z3909/+tN+3333ubv7bbfd1na8SjSWlyRBY3+lgxrH8kqr2/A4YFT4/D5gHnBNle3HA4+7e+pTDJYOBzKrsTmS4UA2bNjA8OHDWbFiBUcffTRjxowBKg9Lf8opp7TtW2mbhQsXMn78ePbdd18gGP6+vffff5/58+dz/vnnty3btGkTAL/73e/4+c9/DsDFF1/MNddU+/NIrcrNSS61ieN/T6KVVhvK/u6+GiD8uV8H208AZrZbdpOZvWRmt5rZJ+IIspw4hgNpbUN56623+Oijj9raUNyDYembmppoampi6dKlXHbZZTvsW2kbd+9w6Plt27ax1157te3b1NTEq6++2ra+Xoeuj0veJpTKGg3Fk32xJRQze9LMFpV5jOvk+/QFDgfmlCy+FjgUOIagOqzi5bOZTTKzRjNrXLeu+x/AOIcD+eQnP8m0adO45ZZb2Lx5M2eeeSZ3331323S8K1euZO3atTvsU2mb0047jQcffJCWlhYA1q9fD8Aee+zRNhLxnnvuyaBBg5g1axYQJKeFCxcCcOKJJ7ZNR/yzn/0sst+xnmXhCzHPozdrKJ4cqKVeLOoH8DrQN3zeF3i9yrb/E5heZf0o4D9rOW7W21BanXPOOT5jxgxUI0gdAAAJAUlEQVR3d//Rj37kQ4cO9aFDh/rxxx/fNjfJ7rvv3rZ9pW3uvfde/9SnPuXDhg3ziRMnurv7f/3Xf/mQIUN8+PDhvnTpUl++fLmfeeaZPmzYMB8yZIjfeOON7u6+fPlyP/74472hocG/973vqQ0lAk+88o4f+q3H/aBr/tMP/dbjibcDpH38KKgNJR3U2IaSyvD1ZvYDoMXdp5rZZKC3u3+zwrYLgGvd/emSZX3dfbUFdTK3AhvdfXJHxy3K8PUtLS0cddRRvPXWW2mH0iaP5zENabahXP/oImY8t/0zc8mIg5gybmiiMUg+1Tp8fVqN8lOBB83sMuBPwPkAZtYAXOHul4evBwL9gWfa7f8zM+sDGNAEXJFM2OlbtWoVo0aN2mleeMmHNMcm05S7ErdUEoq7twCnlVneCFxe8noFcGCZ7U6NM74sO+CAA3jjjTfSDkNySKM3S9w02rBIHdHozRInDb0CpNGOVCQ6fyICSij06tWLlpYWfSl2kbvT0tJCr1690g5FRFJW91Ve/fr1o7m5mSjuUalXvXr1ol+/fmmHISIpq/uE0rNnTwYNGpR2GCIiuVf3VV4iIhINJRQREYmEEoqIiEQilaFX0mJm64CkxyvZF/hzwsfsjCzHl+XYQPF1V5bjy3JskHx8B7l7h0Mr1FVCSYOZNdYyBk5ashxflmMDxdddWY4vy7FBduNTlZeIiERCCUVERCKhhBK/6WkH0IEsx5fl2EDxdVeW48tybJDR+NSGIiIikVAJRUREIqGEEgEz621mc81sSfhz7zLbjDazppLHRjP7TLjuXjN7s2Td8CRjC7fbWnL82SXLB5nZ78P9HzCzXaKKrdb4zGy4mT1nZq+Y2Utm9vmSdbGcOzMba2avm9nScFbR9us/EZ6PpeH5GViy7tpw+etmdmYU8XQytq+Z2eLwXP3GzA4qWVf275xwfJea2bqSOC4vWTcx/CwsMbOJKcV3a0lsb5jZuyXrYj1/Zna3ma01s0UV1puZTQtjf8nMjipZF/u561At8wTrUf0B3AxMDp9PBr7fwfa9gfXAbuHre4HxacYGvF9h+YPAhPD5HcBXko4POAQYHD4/AFgN7BXXuQN6AMuAg4FdgIXAYe22+UfgjvD5BOCB8Plh4fafAAaF79Mj4dhGl3y2vtIaW7W/c8LxXQrcVmbf3sDy8Ofe4fO9k46v3fZfBe5O8PydAhwFLKqw/u+Bxwlmqz0e+H1S566Wh0oo0RgH3Bc+vw/4TAfbjwced/cPY40q0NnY2piZAacCD3Vl/xp1GJ+7v+HuS8Lnq4C1QJzz1x4LLHX35e7+EXB/GGep0rgfAk4Lz9c44H533+TubwJLw/dLLDZ3f7rks7UASHIo6FrOXSVnAnPdfb27/wWYC4xNOb4LgJkRx1CRu/+W4GKzknHADA8sAPYys74kc+46pIQSjf3dfTVA+HO/DrafwM4f0pvCIuytZvaJFGLrZWaNZragtSoO2Ad41923hK+bKTMlc0LxAWBmxxJcWS4rWRz1uTsQeLvkdbnfu22b8Pz8leB81bJv3LGVuozgirZVub9zlGqN77zwb/aQmfXv5L5JxEdYVTgIeKpkcdznryOV4k/i3HWo7oevr5WZPQn8bZlV13XyffoChwNzShZfC7xD8EU5HbgGmJJwbAPcfZWZHQw8ZWYvA/9dZrtOdwuM+Nz9FJjo7tvCxd06d5UOVWZZ+9+70ja17NsdNb+/mV0ENAAjSxbv9Hd292Xl9o8xvl8CM919k5ldQVDSO7XGfZOIr9UE4CF331qyLO7z15G0Pnc1UUKpkbufXmmdma0xs77uvjr80ltb5a0+Bzzs7ptL3nt1+HSTmd0DfD3p2MKqJNx9uZnNA44Efk5QpP54eBXeD1jVmdiiis/M9gR+BXwrLOq3vne3zl0FzUD/ktflfu/WbZrN7OPAJwmqKmrZN+7YMLPTCRL2SHff1Lq8wt85yi/EDuNz95aSl/8OfL9k31Ht9p0XYWw1xVdiAnBl6YIEzl9HKsWfxLnrkKq8ojEbaO1VMRF4tMq2O9XJhl+krW0WnwHK9vCIKzYz27u1qsjM9gVOBBZ70Nr3NEGbT8X9E4hvF+BhgrrjWe3WxXHungcGW9DDbReCL5b2PXpK4x4PPBWer9nABAt6gQ0CBgN/iCCmmmMzsyOBfwPOdfe1JcvL/p0jjK3W+PqWvDwXeDV8Pgc4I4xzb+AMdizJJxJfGOPfETRuP1eyLInz15HZwCVhb6/jgb+GF1VJnLuOJd0LoIgPgrrz3wBLwp+9w+UNwJ0l2w0EVgIfa7f/U8DLBF+G/wHsnmRswAnh8ReGPy8r2f9ggi/EpcAs4BNJnzvgImAz0FTyGB7nuSPoTfMGwdXndeGyKQRf0gC9wvOxNDw/B5fse1243+vAWTF83jqK7UlgTcm5mt3R3znh+L4HvBLG8TRwaMm+XwzP6VLgC2nEF77+DjC13X6xnz+Ci83V4ee9maAN7ArginC9AbeHsb8MNCR57jp66E55ERGJhKq8REQkEkooIiISCSUUERGJhBKKiIhEQglFREQioYQiEhEz26dkJNp3zGxlyeuaR2k2sy+aWbmRBTCzz1swkvA2i3BUapEo6E55kYh4cAf4cAAz+w7ByLS3dOGtvgi8SDCkTHsvE9zAeXcXwxSJjRKKSALC+SmuJBhzbD5wFUENwT0EScgIxiJbE75+wMw2AMd6MCouAO6+OHy/ROMXqYUSikjMzGwo8FngBHffYmbTCYb8WAbs6+6Hh9vt5e7vmtlXgavcvSm9qEU6TwlFJH6nA8cAjWHJYleCocbnAH9nZj8GHgOeSC1CkQgooYjEzwhm/fv2TivMhgFnAVcD5wGTEo5NJDLq5SUSvyeBz4Uj1Lb2BhtgZn0A82AE5RsIpn4FeA/YI51QRbpOJRSRmLn7y2Z2I/CkmX2MYCTZK4CtwF3h0PtOMDkYBA31d5ZrlDez84FbCaZAnmNmje5+doK/jkhFGm1YREQioSovERGJhBKKiIhEQglFREQioYQiIiKRUEIREZFIKKGIiEgklFBERCQSSigiIhKJ/w9PzehUh3bnMQAAAABJRU5ErkJggg==\n", 255 | "text/plain": [ 256 | "
" 257 | ] 258 | }, 259 | "metadata": {}, 260 | "output_type": "display_data" 261 | } 262 | ], 263 | "source": [ 264 | "pos , neg = (y==1).reshape(118,1) , (y==0).reshape(118,1)\n", 265 | "plt.scatter(X[pos[:,0],0],X[pos[:,0],1],c=\"r\",marker=\"+\")\n", 266 | "plt.scatter(X[neg[:,0],0],X[neg[:,0],1],marker=\"o\",s=10)\n", 267 | "plt.xlabel(\"Test 1\")\n", 268 | "plt.ylabel(\"Test 2\")\n", 269 | "plt.legend([\"Accepted\",\"Rejected\"],loc=0)" 270 | ] 271 | }, 272 | { 273 | "cell_type": "markdown", 274 | "metadata": {}, 275 | "source": [ 276 | "### Feature Mapping\n", 277 | "\n", 278 | "with k variables and degrees of n, the number of polynomial terms:\n", 279 | "\n", 280 | "$ {k+ n \\choose n}$" 281 | ] 282 | }, 283 | { 284 | "cell_type": "code", 285 | "execution_count": 6, 286 | "metadata": {}, 287 | "outputs": [], 288 | "source": [ 289 | "def mapFeature(x1,x2,degree):\n", 290 | " \"\"\"\n", 291 | " take in numpy array of x1 and x2, return all polynomial terms up to the given degree\n", 292 | " \"\"\"\n", 293 | " out = np.ones(len(x1)).reshape(len(x1),1)\n", 294 | " for i in range(1,degree+1):\n", 295 | " for j in range(i+1):\n", 296 | " terms= (x1**(i-j) * x2**j).reshape(len(x1),1)\n", 297 | " out= np.hstack((out,terms))\n", 298 | " return out" 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "execution_count": 7, 304 | "metadata": {}, 305 | "outputs": [], 306 | "source": [ 307 | "X = mapFeature(X[:,0], X[:,1],6)" 308 | ] 309 | }, 310 | { 311 | "cell_type": "markdown", 312 | "metadata": {}, 313 | "source": [ 314 | "### Computing Regularize Cost Function and Gradient\n", 315 | "\n", 316 | "$J(\\Theta) = \\frac{1}{m} \\sum_{i=1}^{m} [ -y^{(i)}log(h_{\\Theta}(x^{(i)})) - (1 - y^{(i)})log(1 - (h_{\\Theta}(x^{(i)}))] + \\frac{\\lambda}{2m} \\sum_{j=1}^n \\Theta_j^2$\n", 317 | "\n", 318 | "$ \\frac{\\partial J(\\Theta)}{\\partial \\Theta_j} = \\frac{1}{m} \\sum_{i=1}^{m} (h_{\\Theta}(x^{(i)}) - y^{(i)})x_j^{(i)}$ for $j=0$\n", 319 | "\n", 320 | "$ \\frac{\\partial J(\\Theta)}{\\partial \\Theta_j} = \\frac{1}{m} \\sum_{i=1}^{m} (h_{\\Theta}(x^{(i)}) - y^{(i)})x_j^{(i)} + \\frac{\\lambda}{m}\\Theta_j$ for $j\\geq 1$" 321 | ] 322 | }, 323 | { 324 | "cell_type": "code", 325 | "execution_count": 8, 326 | "metadata": {}, 327 | "outputs": [], 328 | "source": [ 329 | "def sigmoid(z):\n", 330 | " \"\"\"\n", 331 | " return the sigmoid of z\n", 332 | " \"\"\"\n", 333 | " \n", 334 | " return 1/ (1 + np.exp(-z))" 335 | ] 336 | }, 337 | { 338 | "cell_type": "code", 339 | "execution_count": 9, 340 | "metadata": {}, 341 | "outputs": [], 342 | "source": [ 343 | "def costFunctionReg(theta, X, y ,Lambda):\n", 344 | " \"\"\"\n", 345 | " Take in numpy array of theta, X, and y to return the regularize cost function and gradient\n", 346 | " of a logistic regression\n", 347 | " \"\"\"\n", 348 | " \n", 349 | " m=len(y)\n", 350 | " y=y[:,np.newaxis]\n", 351 | " predictions = sigmoid(X @ theta)\n", 352 | " error = (-y * np.log(predictions)) - ((1-y)*np.log(1-predictions))\n", 353 | " cost = 1/m * sum(error)\n", 354 | " regCost= cost + Lambda/(2*m) * sum(theta**2)\n", 355 | " \n", 356 | " # compute gradient\n", 357 | " j_0= 1/m * (X.transpose() @ (predictions - y))[0]\n", 358 | " j_1 = 1/m * (X.transpose() @ (predictions - y))[1:] + (Lambda/m)* theta[1:]\n", 359 | " grad= np.vstack((j_0[:,np.newaxis],j_1))\n", 360 | " return cost[0], grad" 361 | ] 362 | }, 363 | { 364 | "cell_type": "code", 365 | "execution_count": 10, 366 | "metadata": {}, 367 | "outputs": [], 368 | "source": [ 369 | "# Initialize fitting parameters\n", 370 | "initial_theta = np.zeros((X.shape[1], 1))\n", 371 | "\n", 372 | "# Set regularization parameter lambda to 1\n", 373 | "Lambda = 1\n", 374 | "\n", 375 | "#Compute and display initial cost and gradient for regularized logistic regression\n", 376 | "cost, grad=costFunctionReg(initial_theta, X, y, Lambda)" 377 | ] 378 | }, 379 | { 380 | "cell_type": "code", 381 | "execution_count": 11, 382 | "metadata": {}, 383 | "outputs": [ 384 | { 385 | "name": "stdout", 386 | "output_type": "stream", 387 | "text": [ 388 | "Cost at initial theta (zeros): 0.6931471805599461\n" 389 | ] 390 | } 391 | ], 392 | "source": [ 393 | "print(\"Cost at initial theta (zeros):\",cost)" 394 | ] 395 | }, 396 | { 397 | "cell_type": "markdown", 398 | "metadata": {}, 399 | "source": [ 400 | "### Gradient Descent" 401 | ] 402 | }, 403 | { 404 | "cell_type": "code", 405 | "execution_count": 12, 406 | "metadata": {}, 407 | "outputs": [], 408 | "source": [ 409 | "def gradientDescent(X,y,theta,alpha,num_iters,Lambda):\n", 410 | " \"\"\"\n", 411 | " Take in numpy array X, y and theta and update theta by taking num_iters gradient steps\n", 412 | " with learning rate of alpha\n", 413 | " \n", 414 | " return theta and the list of the cost of theta during each iteration\n", 415 | " \"\"\"\n", 416 | " \n", 417 | " m=len(y)\n", 418 | " J_history =[]\n", 419 | " \n", 420 | " for i in range(num_iters):\n", 421 | " cost, grad = costFunctionReg(theta,X,y,Lambda)\n", 422 | " theta = theta - (alpha * grad)\n", 423 | " J_history.append(cost)\n", 424 | " \n", 425 | " return theta , J_history" 426 | ] 427 | }, 428 | { 429 | "cell_type": "code", 430 | "execution_count": 13, 431 | "metadata": {}, 432 | "outputs": [], 433 | "source": [ 434 | "theta , J_history = gradientDescent(X,y,initial_theta,1,800,0.2)" 435 | ] 436 | }, 437 | { 438 | "cell_type": "code", 439 | "execution_count": 14, 440 | "metadata": {}, 441 | "outputs": [ 442 | { 443 | "name": "stdout", 444 | "output_type": "stream", 445 | "text": [ 446 | "The regularized theta using ridge regression:\n", 447 | " [[ 2.25319932]\n", 448 | " [ 1.44552306]\n", 449 | " [ 2.33283911]\n", 450 | " [-3.57008233]\n", 451 | " [-2.35002884]\n", 452 | " [-3.07090829]\n", 453 | " [ 0.50119267]\n", 454 | " [-0.88407626]\n", 455 | " [-0.69606495]\n", 456 | " [-0.22204243]\n", 457 | " [-2.69560917]\n", 458 | " [ 0.04161168]\n", 459 | " [-1.31573798]\n", 460 | " [-0.80286232]\n", 461 | " [-2.23101395]\n", 462 | " [-0.39638083]\n", 463 | " [-0.43758167]\n", 464 | " [ 0.09476652]\n", 465 | " [-0.69300577]\n", 466 | " [-0.78996079]\n", 467 | " [-0.45416815]\n", 468 | " [-2.01051679]\n", 469 | " [ 0.16446857]\n", 470 | " [-0.59855483]\n", 471 | " [ 0.11471778]\n", 472 | " [-0.73960173]\n", 473 | " [-0.52044034]\n", 474 | " [-1.38532427]]\n" 475 | ] 476 | } 477 | ], 478 | "source": [ 479 | "print(\"The regularized theta using ridge regression:\\n\",theta)" 480 | ] 481 | }, 482 | { 483 | "cell_type": "markdown", 484 | "metadata": {}, 485 | "source": [ 486 | "### Plotting of Cost function" 487 | ] 488 | }, 489 | { 490 | "cell_type": "code", 491 | "execution_count": 15, 492 | "metadata": {}, 493 | "outputs": [ 494 | { 495 | "data": { 496 | "text/plain": [ 497 | "Text(0.5,1,'Cost function using Gradient Descent')" 498 | ] 499 | }, 500 | "execution_count": 15, 501 | "metadata": {}, 502 | "output_type": "execute_result" 503 | }, 504 | { 505 | "data": { 506 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4XOWZ/vHvoxl1ybJsyV2u2AZsio1CCSUsLSYhkE3Y0DaB325CIAtJNm1JdlMWtrDZ1M3CEpKQsAklBFIMIQGWAAk1lgGDK7hiucq2ZMtN9fn9cV7ZIzEqY0szI+v+XNdcM+c97znnmaK5dcqcY+6OiIhIKnIyXYCIiAw+Cg8REUmZwkNERFKm8BARkZQpPEREJGUKDxERSZnCQ1JmZteb2RYz221mI9O43C+Z2Q/TtbzemNlVZvZ4pus4HGa21szOC4+z6vWV7KbwGMTM7Eozqwlf4pvM7HdmdsZhzvPAl0k343OBbwEXuHuJu28/nOX1sJyzzaw2sc3d/83dPzoQyzsU7n6Pu18wUPM3s8vN7CUz22NmW8PjT5iZDcTy+uv1NbPJZuZmFu+hz9fMrMXMGsPtDTP7bzMbe7jLHyjhOR2V6TqyhcJjkDKzzwDfAf4NGA1MBG4HLhngRY8GCoAlA7ycIc3MPgt8F/hPYAzR634dcDqQ1800sbQV2D9+7u6lwAjgL4me58JsDhBJ4O66DbIbUAbsBv6qhz75ROGyMdy+A+SHcRXAI0ADsAP4E9E/Ej8F2oF9Yf5f6DLPGcAewMP4PwCTw3A8od/TwEfD42uAZ4FvAPXAGuDChL4jgB+HGuuBXwPFoYb2sJzdwDjga8DPEqa9mCjEGsIyj0kYtxb4HPAasBP4OVDQzWvVdb6dnlN4DquBxlD/VYnPLWE6J/qCfzM8l9sAC+NiwDeBbWEeN3R93bq8v3uAD/byOfgJ8D/Ao6H/ecB7gVeAXcB64GtdpvkwsA7YDvxjeJ3O6+Z1OBV4Pry+i4Czu7zHtwDPhdflcaAijHsr4TOyGzitt9c84TVaBHwjoe0i4NVQw/PA8Qnj/gHYEJa/Ajg3YT5fAlaFcQuBqjDuaOAJos/9CuBDXV7P24DfhuleAqaFcX8Mz2lPeE6XZfp7INO3jBeg2yG8aTAPaE32xZPQ52bgRWAUUBn+8G4J4/4duAPIDbczE77kDnyZdDPfyXT+Yu00HNqepnN4tAAfC3/U1xMFRcfyfkv0xV4eanlXaD8bqO2y7ANfOBwMsvPDdF8AVgJ5Cc/jz0ShMwJYBlzXzXPq9EWW+JyIgmwXMDOMGwvMSnhuXcPjEWA40ZpgHTAvjLsOWApMCM/1/7q+bqm8v6HfT4iC8XSi8C8Ir9txYfh4YAvw/tD/WKIvvrOI/rn4VljO28IDGE8UMO8J8zo/DFcmvMerwvtQGIZv7e4z0dtr3uVz+1J4PBfYCpxC9Nm5Oryv+cBMonAcl7DMji/6zwOvhz4GnACMDO/leuD/hfd2LlGYz0p4PXcAJ4fx9wD3d3l/j8r033+23LTZanAaCWxz99Ye+lwF3OzuW929Dvhnov86IfoyHwtMcvcWd/+Th7+OAbLO3X/g7m3A3WHZo8PmiQuJvtTrQy3P9HGelwG/dfcn3L2FaM2mEHhnQp//cveN7r4DeBg48RDrbwdmm1mhu29y95422d3q7g3u/hbwVMIyPwR8191r3b0euLWHeVTQ5f01s+fNrMHM9pnZWQl9f+Puz7l7u7vvd/en3f31MPwacB/wrtD3UuARd/+juzcBXw7PLZm/Bh5190fDvJ4AaojCpMOP3f0Nd98HPMChv76JNhKFPUT/cHzf3V9y9zZ3vxtoIlojaiMKkWPNLNfd17r7qjDdR4F/cvcVHlnk0b65i4C17v5jd29195eBh8Lr0uGX7v7n8Nrf00/P6Yik8BictgMVPe2QJPqPe13C8LrQBtF29JXA42a22sxuGpgyD9jc8cDd94aHJUAVsCN8maaq0/Nz93ai/yrHJ1susDcsMyXuvocoqK4DNpnZb83s6B4m6W6Z40J9HRIfd/W299fd3+nuw8O4xL/bTvMxs1PM7CkzqzOznaHuimQ1hOfW3QEPk4C/CoHVYGYNwBlEwd/bcz0c44n++++o4bNdaqgiWttYCXyaaA1mq5ndb2Ydn+8qorWiZM/plC7zu4poX8tAPqcjksJjcHoB2A+8v4c+G4n+WDpMDG24e6O7f9bdpwLvAz5jZueGfqmugewJ90UJbWOSdUxiPTDCzIYnGddbHZ2eXzgCqYpoG3iq9tBD/e7+mLufT/TFuRz4wSEsYxPRJqsOVT30fYHoP+y+HPzQ9XW6F5hPtI2/jGjzZMfRWZsSl2tmRURrscmsB37q7sMTbsXu3tMaU3c19YmZ5RB9Hv+UUMO/dqmhyN3vA3D3e939DKLPgQP/kTDdtG6e0zNd5lfi7tcfSr1DncJjEHL3ncBXgNvM7P1mVmRmuWZ2oZl9PXS7D/gnM6s0s4rQ/2cAZnaRmR0VvnB3EW0CaAvTbQGmplBLHdEX9l+bWczM/obkf7jJpt0E/A643czKw3Po2CSzBRhpZmXdTP4A8F4zOzccPvxZoi/c5/tae4JXgbPMbGJY3hc7RpjZaDO72MyKw/x3c/C1SsUDwKfMbHwIy3/orqO7NxBtZrzdzC41sxIzyzGzE4m22/eklGhtbr+ZnQxcmTDuQeAiMzvDzPKI9i909x3wM+B9Zvbu8L4WhMOnJ3TTP1Ed0eawPn2Owvt+DNFndgzRvhiIQvq6sDZlZlZsZu81s1Izm2lm55hZPtE/Uvs4+L78ELjFzKaH6Y636PdIjwAzzOzDYZm5ZvaOsOy+SOlv40in8Bik3P1bwGeAfyL6Y11PdATPr0OXfyHaRv0a0c7Dl0MbwHSiHba7if7Lvd3dnw7j/p0odBrM7HN9LOdjRDsptwOzSO0L/MNE+2CWE+0c/XR4fsuJvkxWh1rGJU7k7iuItst/j2in5/uA97l7cwrL7pjXE0Q77V8jOjLnkYTROUTBtJFoc8q7gE+kugyiL8LHwzJeITpCqpVugsjdv070/n6B6HXZAnyfKHR6en0/AdxsZo1E/zA8kDDPJcDfEa2dbCI6Iqw22UzcfT3Rms+XOPj5+jx9+M4Imyb/FXguvHendtP1MjPbTXQk1Xyiz89J7t6xhlxD9Nn671DrSqKDFCDa33Er0Xu/mejAkC+Fcd8Kz/txon+OfgQUunsjcAFwOdH7uZlobSW/t+cUfA24OzynD/VxmiNWxxEvIpJGZnYhcIe7T+q1s0gW0pqHSBqYWaGZvcfM4mY2Hvgq8KtM1yVyqLTmIZIGYef0M0Q/UttH9PuWT7n7rowWJnKIFB4iIpIybbYSEZGU9fQjs0GtoqLCJ0+enOkyREQGlYULF25z98re+h2x4TF58mRqamoyXYaIyKBiZut676XNViIicgjSGh5mNs/MVpjZymTnUzKzb5vZq+H2Rjj3TMe4q83szXC7Op11i4hIZ2nbbGXRhWpuIzq1cy2wwMzmu/vSjj7u/vcJ/W8E5oTHI4iOi68mOofNwjDtoZxQT0REDlM61zxOBla6++pwCon76fnEb1cQnZ4C4N3AE+7ecQbWJ4iueSAiIhmQzvAYT+fTR9fS+fTZB5jZJGAK0ZXq+jytmV1r0TW9a+rq6vqlaBERebt0hoclaevuF4qXAw+Giwf1eVp3v9Pdq929urKy1yPNRETkEKUzPGrpfA2DCYTrSyRxOQc3WaU6rYiIDLB0hscCYLqZTQnXEric6DTMnZjZTKJrPL+Q0PwYcEG45kM50WmVHxuIIhv3t/DtJ97g1fUNvXcWERmi0hYe4ZrANxB96S8DHnD3JWZ2s5ldnND1CqKLznvCtDuAW4gCaAHRtbl3MABa25zvPvkmL6/TgVwiIt1J6y/M3f1RoovgJLZ9pcvw17qZ9i7grgErLigpiF6Sxv2tA70oEZFBS78w7yI3lkNhbozG/S2ZLkVEJGspPJIoLYizu0lrHiIi3VF4JFFSENdmKxGRHig8kigtyGWXNluJiHRL4ZHEMK15iIj0SOGRhPZ5iIj0TOGRREl+XEdbiYj0QOGRRGlBrjZbiYj0QOGRRGlBnL3NbbS1d3feRhGRoU3hkURpQS4Au7X2ISKSlMIjidL86BQlOlxXRCQ5hUcSwwoVHiIiPVF4JDGiOB+AHXuaM1yJiEh2UngkMbIkD1B4iIh0R+GRREVY89i2W+EhIpKMwiOJYYVx4jnG9t1NmS5FRCQrKTySMDNGFOexXWseIiJJKTy6MbIkn+17tOYhIpKMwqMbFSV5bNcOcxGRpBQe3RipzVYiIt1SeHRjRHG+dpiLiHRD4dGNkSV57GluY19zW6ZLERHJOmkNDzObZ2YrzGylmd3UTZ8PmdlSM1tiZvcmtLeZ2avhNn+ga60IPxTUTnMRkbeLp2tBZhYDbgPOB2qBBWY2392XJvSZDnwRON3d681sVMIs9rn7iemqd2TCKUomlBela7EiIoNCOtc8TgZWuvtqd28G7gcu6dLnY8Bt7l4P4O5b01hfJx2nKNmm/R4iIm+TzvAYD6xPGK4NbYlmADPM7Dkze9HM5iWMKzCzmtD+/mQLMLNrQ5+aurq6wyq2sjRa89i6S+EhItJV2jZbAZakreul+uLAdOBsYALwJzOb7e4NwER332hmU4E/mNnr7r6q08zc7wTuBKiurj6sywCOKi3ADDbv2n84sxEROSKlc82jFqhKGJ4AbEzS5zfu3uLua4AVRGGCu28M96uBp4E5A1lsXjyHkcX5bN6p8BAR6Sqd4bEAmG5mU8wsD7gc6HrU1K+BvwAwswqizVirzazczPIT2k8HljLAxpYVaM1DRCSJtG22cvdWM7sBeAyIAXe5+xIzuxmocff5YdwFZrYUaAM+7+7bzeydwPfNrJ0o8G5NPEproIweVkBt/d6BXoyIyKCTzn0euPujwKNd2r6S8NiBz4RbYp/ngePSUWOiMWX51Kzbke7FiohkPf3CvAdjywpp2NvC/hb9ylxEJJHCowejhxUAaKe5iEgXCo8ejC0L4aGd5iIinSg8eqA1DxGR5BQePehY89ik8BAR6UTh0YPi/DjlRbk6XFdEpAuFRy8mlBexvn5fpssQEckqCo9eVI0opHaH1jxERBIpPHpRVV5Ebf0+2tsP6zyLIiJHFIVHLyaMKKK5rZ2tjTo1u4hIB4VHL6rKCwFYr53mIiIHKDx6UTUiugStjrgSETlI4dGL8cPDmscOHXElItJB4dGLgtwYo0rzWa8jrkREDlB49EHViCLt8xARSaDw6IOq8kJtthIRSaDw6IOJI4rYtHMfza3tmS5FRCQrKDz6YEplMe0Ob+3Yk+lSRESygsKjD6ZWlACwqk7hISICCo8+mVJZDMBqhYeICKDw6JNhBblUlOSzZtvuTJciIpIVFB59NLWyWGseIiJBWsPDzOaZ2QozW2lmN3XT50NmttTMlpjZvQntV5vZm+F2dfqqjkyrLGb1NoWHiAhAPF0LMrMYcBtwPlALLDCz+e6+NKHPdOCLwOnuXm9mo0L7COCrQDXgwMIwbX266p9aUcKOPetp2NvM8KK8dC1WRCQrpXPN42Rgpbuvdvdm4H7gki59Pgbc1hEK7r41tL8beMLdd4RxTwDz0lQ3EG22Ah1xJSIC6Q2P8cD6hOHa0JZoBjDDzJ4zsxfNbF4K02Jm15pZjZnV1NXV9WPpMLUyOlx3dZ12mouIpDM8LElb18vzxYHpwNnAFcAPzWx4H6fF3e9092p3r66srDzMcjurKi8kN2asVHiIiKQ1PGqBqoThCcDGJH1+4+4t7r4GWEEUJn2ZdkDFYzlMqyzhzS0KDxGRdIbHAmC6mU0xszzgcmB+lz6/Bv4CwMwqiDZjrQYeAy4ws3IzKwcuCG1pNWN0KSs2N6Z7sSIiWSdt4eHurcANRF/6y4AH3H2Jmd1sZheHbo8B281sKfAU8Hl33+7uO4BbiAJoAXBzaEurmWNK2dCwj8b9LeletIhIVknboboA7v4o8GiXtq8kPHbgM+HWddq7gLsGusaeHD2mFIA3tjRy0qQRmSxFRCSj9AvzFMwYHYXHis3a7yEiQ5vCIwUTygspyY+zYvOuTJciIpJRCo8UmBkzRpewXDvNRWSIU3ikaOaYUt7Y0ki0e0ZEZGhSeKTo6DHDqN/bwuZd+zNdiohIxig8UjR7fBkAr9fuzHAlIiKZo/BI0bFjh5FjsHiDwkNEhi6FR4oK82IcNaqE1xUeIjKEKTwOwezxZby+YZd2movIkKXwOATHjS9j2+4mtuxqynQpIiIZofA4BMd17DTXpisRGaIUHofg2HHRTnOFh4gMVQqPQ1CUF2daZYmOuBKRIUvhcYiOG1/Ga7U7tdNcRIYkhcchOnHicLbtbqK2fl+mSxERSTuFxyE6aVI5AC+/VZ/hSkRE0k/hcYhmji6lOC/GwnUKDxEZehQehygey+HEicMVHiIyJCk8DsNJE8tZtmkXe5paM12KiEhaKTwOw9xJ5bQ7LFrfkOlSRETSSuFxGOZMLMcMbboSkSFH4XEYygpzmTGqlBqFh4gMMQqPw1Q9uZyF6+ppbWvPdCkiImmT1vAws3lmtsLMVprZTUnGX2NmdWb2arh9NGFcW0L7/HTW3ZPTpo1kd1MrizfuynQpIiJpE0/XgswsBtwGnA/UAgvMbL67L+3S9efufkOSWexz9xMHus5UnTp1JADPr9rGiVXDM1yNiEh6pHPN42Rgpbuvdvdm4H7gkjQuf0BUlOQzc3QpL6zanulSRETSJp3hMR5YnzBcG9q6+qCZvWZmD5pZVUJ7gZnVmNmLZvb+ZAsws2tDn5q6urp+LL1np00byYK1O2hu1X4PERkaUg4PMysOm6BSnjRJW9dT0j4MTHb344H/A+5OGDfR3auBK4HvmNm0t83M/U53r3b36srKykMo8dC8c9pI9re086p+7yEiQ0Sv4WFmOWZ2pZn91sy2AsuBTWa2xMz+08ym93FZtUDimsQEYGNiB3ff7u4d13b9AXBSwriN4X418DQwp4/LHXCnTBmJWbTfQ0RkKOjLmsdTwDTgi8AYd69y91HAmcCLwK1m9td9mM8CYLqZTTGzPOByoNNRU2Y2NmHwYmBZaC83s/zwuAI4Hei6oz1jyopymT2ujOdXar+HiAwNfTna6jx3b+na6O47gIeAh8wst7eZuHurmd0APAbEgLvcfYmZ3QzUuPt84JNmdjHQCuwArgmTHwN838zaiQLv1iRHaWXUWTMquOOZ1ezc10JZYa8vh4jIoGZ9vRKemRUARxHtp1jl7vsHsrDDVV1d7TU1NWlbXs3aHVx6xwvcftVc3nPc2N4nEBHJQma2MOxf7lFf9nnEzezrRPss7gZ+Bqw3s1vNLG2/E8l2J1YNZ1hBnKeWb810KSIiA64v+zz+EygHpgCPuPscon0gFcA3BrC2QSUey+GsGZU8/UadrmsuIke8voTHRcC17t4IvA/A3XcBHw/jJDh75ijqGptYolOViMgRri/h4X7wX2lLaGwD9Ku4BO+aEf225OkV2nQlIke2voTHMjP7SHh84HcZ4fDcZQNS1SBVWZrPCRPKeGKZwkNEjmx9CY+/A240s6eB5Wb2DTN7BrgRuH4gixuMLpg1hkXrG9jYsC/TpYiIDJhew8Pda939HcAtwFrgLeCf3f2Ujl99y0EXzh4DwGNLNme4EhGRgdProbZmZh55Eniypz79Xt0gNLWyhBmjS/jd4s38v9OnZLocEZEB0afTk5jZjWY2MbHRzPLM7Bwzuxu4emDKG5zmzR7LgrU7qGts6r2ziMgg1JfwmAe0AfeZ2UYzW2pma4A3gSuAb7v7TwawxkFn3qwxuMMTS7dkuhQRkQHR62arcBqS24HbwzmsKoiu6qfzj3fjmLGlTBpZxO8Wb+LKUyb2PoGIyCDTl9OTfCtcW3wukOPumxQcPTMz5s0ew/OrtrN9tzZdiciRpy+brVYCpwLfI7qOx1Izu9/MvmRm53ecKl06+8s542lrdx5epAPSROTI05dDdW939+vc/XR3HwG8F7g3THs90Y8I3z3AdQ46R48ZxjFjh/GrVzZkuhQRkX6X8mVo3X2Nu893939x9w8QXZjp3/q/tMHvA3PGs6h2J6vqdme6FBGRfpVyeHTl7puI1kSki0tOHEeOwa+19iEiR5jDDg8Ad/9mf8znSDNqWAGnH1XBr17ZQHu7fkMpIkeOfgkP6d4H5o6ntn4fL63ZkelSRET6jcJjgF04eyzDCuLc++e3Ml2KiEi/UXgMsILcGJeeVMXvF29im37zISJHCIVHGlx5ykRa2pwHatZnuhQRkX6h8EiDo0aVcOrUEdz70lvacS4iR4S0hoeZzTOzFWa20sxuSjL+GjOrM7NXw+2jCeOuNrM3w23QncX3qlMmUVu/j2ferMt0KSIihy1t4WFmMeA24ELgWOAKMzs2Sdefu/uJ4fbDMO0I4KvAKcDJwFfNrDxNpfeLd88aQ0VJPj95bm2mSxEROWzpXPM4GVjp7qvdvRm4H7ikj9O+G3jC3Xe4ez3wBNGp4geNvHgO17xzEs+8UceKzY2ZLkdE5LCkMzzGA4l7jGtDW1cfNLPXzOxBM6tKZVozu9bMasyspq4u+zYPXXXKJApzY/zgT6szXYqIyGFJZ3hYkraue48fBia7+/HA/wF3pzAt7n6nu1e7e3VlZeVhFTsQyovz+FD1BH7z6ga27Nqf6XJERA5ZOsOjFqhKGJ4AdDpfubtvd/eOH0P8ADipr9MOFn9zxhTa2p2fPL8206WIiByydIbHAmC6mU0xszzgcmB+YgczG5sweDGwLDx+DLjAzMrDjvILQtugM2lkMRfOHsvPXljHzr0tmS5HROSQpC083L0VuIHoS38Z8IC7LzGzm83s4tDtk2a2xMwWAZ8ErgnT7gBuIQqgBcDNoW1QuuGco2hsauVHz63JdCkiIofE3I/MH61VV1d7TU1Npsvo1vU/W8izb27j2X84h7Ki3EyXIyICgJktdPfq3vrpF+YZ8slzp2vtQ0QGLYVHhhwzdhjvOW4MP352DfV7mjNdjohIShQeGfT3581gT3Mr333yzUyXIiKSEoVHBk0fXcrlJ0/kZy+uY822PZkuR0SkzxQeGfbp86aTH8/h1t8t672ziEiWUHhk2KjSAq571zQeW7KFl1Zvz3Q5IiJ9ovDIAh89cypjhhVwy2+X0qbrfYjIIKDwyAKFeTH+8b3HsHjDLn76wtpMlyMi0iuFR5a46PixnDm9gm88/gabd+qkiSKS3RQeWcLM+Jf3z6alrZ1bHlma6XJERHqk8Mgik0YWc+M5R/Hb1zfxh+VbMl2OiEi3FB5Z5mNnTWXm6FJueuh1/fJcRLKWwiPL5MdjfOuyE6jf28yXf7M40+WIiCSl8MhCs8aV8alzp/PIa5uYv2hQXvNKRI5wCo8sdd27pnFi1XC+/OvFbGjYl+lyREQ6UXhkqXgsh29fdiJt7c4N975Mc2t7pksSETlA4ZHFplQUc+sHj+OVtxr4+u+XZ7ocEZEDFB5Z7qLjx3H1aZP44bNr+P3izZkuR0QEUHgMCl967zGcMKGMz/1iESs2N2a6HBERhcdgkB+PcceHT6IoL8bf3r2A7bubMl2SiAxxCo9BYmxZIT/4SDV1jU18/KcLaWpty3RJIjKEKTwGkROqhvPND51Azbp6bnroddp1+nYRyZC0hoeZzTOzFWa20sxu6qHfpWbmZlYdhieb2T4zezXc7khf1dnlouPH8dnzZ/CrVzbwr48uw10BIiLpF0/XgswsBtwGnA/UAgvMbL67L+3SrxT4JPBSl1mscvcT01JslrvhnKPYvqeZHz27hvKiXG44Z3qmSxKRISadax4nAyvdfbW7NwP3A5ck6XcL8HVAF7XohpnxlYuO5S/njOcbj7+hC0iJSNqlMzzGA+sThmtD2wFmNgeocvdHkkw/xcxeMbNnzOzMZAsws2vNrMbMaurq6vqt8GyUk2N8/dLjOe+YUXz5N0sUICKSVukMD0vSdmCDvZnlAN8GPpuk3yZgorvPAT4D3Gtmw942M/c73b3a3asrKyv7qezslRvL4bar5nLeMaP58m+WcNezazJdkogMEekMj1qgKmF4ApB4ythSYDbwtJmtBU4F5ptZtbs3uft2AHdfCKwCZqSl6iyXH49x+1VzmTdrDDc/spQ7nlmV6ZJEZAhIZ3gsAKab2RQzywMuB+Z3jHT3ne5e4e6T3X0y8CJwsbvXmFll2OGOmU0FpgOr01h7VsuL5/C9K+fwvhPGcevvlnPzw0t1GK+IDKi0HW3l7q1mdgPwGBAD7nL3JWZ2M1Dj7vN7mPws4GYzawXagOvcfcfAVz145MZy+M5lJzKyOI+7nlvDll37+eaHTqAgN5bp0kTkCGRH6u8EqqurvaamJtNlpJ2784M/rebfHl3OyZNH8P0Pn0R5cV6myxKRQcLMFrp7dW/99AvzI4yZce1Z0/ivK+bw6voGLr7tWZZt2pXpskTkCKPwOEJdfMI47v/4qTS3tvOB25/nYV3OVkT6kcLjCDZ3YjkP33gGs8YN48b7XuFfHlmqKxKKSL9QeBzhRpUWcO/HTuUj4YJSH/yf51mzbU+myxKRQU7hMQTkxXO4+ZLZ3PHXJ/HWjr2897/+xC9q1uukiiJyyBQeQ8i82WP4/afP5LjxZXz+wdf4xD0vs7VRpxATkdQpPIaYsWWF3PuxU/mHeUfz5PKtnP+tP/LQwlqthYhIShQeQ1Asx7j+7Gk8+skzmT6qhM/+YhHX/HgBb23fm+nSRGSQUHgMYUeNKuGBj5/GzZfMombtDs779jN88/EV7G1uzXRpIpLlFB5DXE6O8ZHTJvPkZ8/mPbPH8L0/rOTcbz7Dw4s2alOWiHRL4SEAjCkr4DuXz+EX151GeVEeN973Ch/8n+d5YdX2TJcmIllI4SGdvGPyCB6+8Qz+/QPHsbFhP1f84EU+/KOXeK22IdOliUgW0YkRpVv7W9r46QvruP3pldTvbWHerDHccM5RzB5flunSRGSA9PXEiAoP6VXj/hZ++Kc1/OjZNexuauXKef5QAAAPXUlEQVSsGZVc/65pnDp1BGbJLhApIoOVwkPh0e927mvhnpfWcdeza9i2u5k5E4dz7ZlTOf/Y0cRj2gIqciRQeCg8Bsz+ljZ+sbCWO/+4ivU79jGurICrTp3EZe+ooqIkP9PlichhUHgoPAZcW7vz5LIt/O8L63h25TbyYjlcdPxYrjp1InMnlmuTlsgg1NfwSNtlaOXIE8sxLpg1hgtmjWHl1kZ++sI6HlxYyy9f2cDUimI+eNIEPjB3PGPLCjNdqoj0M615SL/a3dTKo69v4sGFtfx5zQ5yDM6YXskH547nvGNGU5yv/1dEspk2Wyk8Mm7d9j08tLCWh17ewIaGfeTHc/iLmaN4z/FjOffoUQoSkSyk8FB4ZI32dqdmXT2Pvr6JR1/fxNbGpgNBcuFxY3jXjEqGF+VlukwRQeGh8MhSbe1OzdodUZAs3kxdYxM5BtWTRnDOMaM45+hRTB9Vop3tIhmSleFhZvOA7wIx4Ifufms3/S4FfgG8w91rQtsXgb8F2oBPuvtjPS1L4ZH92tudRbUNPLV8K08u38qSjbsAmFBeyNkzKzl9WgWnTRuptRKRNMq68DCzGPAGcD5QCywArnD3pV36lQK/BfKAG9y9xsyOBe4DTgbGAf8HzHD3tu6Wp/AYfDbt3MdTy+v4w/ItvLBqO3ua2zCDWeOGcfq0Ct55VAXvmFxOUZ72lYgMlGw8VPdkYKW7rwYws/uBS4ClXfrdAnwd+FxC2yXA/e7eBKwxs5Vhfi8MeNWSNmPLCrnylIlcecpEWtraWbS+gedWbuf5Vdu467k1fP+Pq8mNGcdPGM5Jk8oP3PTDRJH0S2d4jAfWJwzXAqckdjCzOUCVuz9iZp/rMu2LXaYdP1CFSublxnKonjyC6skj+NR509nX3MaCtTt4btU2atbW85Pn1nLnH1cDMHlkEXMnlVM9aQRzJw3nqMoSnS5FZIClMzyS7QE9sM3MzHKAbwPXpDptwjyuBa4FmDhx4iEVKdmpMC/GWTMqOWtGJRCdImXxhp0sXFdPzbp6nllRxy9f3gBAQW4Ox4wdxnHjy5g9vozjxpcxfZQCRaQ/pTM8aoGqhOEJwMaE4VJgNvB0ONJmDDDfzC7uw7QAuPudwJ0Q7fPoz+IluxTkxg6smXwccHfWbt/LovUNvL5hJ69v2MlDC2v53xfWAZAfjwJl1rhhHD2mlBmjo1t5sXbGixyKdO4wjxPtMD8X2EC0w/xKd1/STf+ngc+FHeazgHs5uMP8SWC6dphLT9rbnTXb97B4w05er40CZdmmXezaf/Aa7aNK85kZwmTm6FJmjCllWmUxpQW5GaxcJHOyboe5u7ea2Q3AY0SH6t7l7kvM7Gagxt3n9zDtEjN7gGjneivwdz0FhwhE12efVlnCtMoSLjkx2kXm7mzZ1cSKLY28sbkxut/SyD0vrWN/S/uBaStL85lSUcyUkcVMqSxmSkUxUyuKmTiyiPx4LFNPSSRr6EeCIkQ/Xqyt38vyzY2srtvDmm27WbNtD2u27WHb7uYD/XIMxpcXMnlkMVUjiphQXkhVeXQ/obyIipI8/cBRBrWsW/MQyWaxHGPSyGImjSx+27id+1pYG4Ik8bZ4wybq97Z06luQm8OEA2ESBcu44YWMKStgzLACRg3L15qLHBEUHiK9KCvM5YSq4ZxQNfxt43Y3tbKhfh+19XtZv2MvtfX7olvDXl5d30BDl3ABGFGcx5hhBYwpK2D0sILwOD96XFbAqNIChhfmkpOjNRjJXgoPkcNQkh9n5phSZo4pTTp+1/4WNjXsZ/Ou/WzZGd0nPl60voHte5rfNl0sxxhZnMfIknwqSvKoSLjv3JbPyJI8cnUYsqSZwkNkAA0ryGXYmNxuwwWgqbWNrbua2BKCpa6xiW27m9jW2Mz2PU3U7W5mdd0etu1uoqm1Pek8SgvilBflUV6US1m4Ly/KY3hRLsMLcykvzmN4aB9emMfw4lxK8+PaPyOHTOEhkmH58RhVI4qoGlHUYz93Z09zG9sam6JQaWyOQmZ3Ew17W6jf20z93hZ27m1m7bY91O9tpjHhsOSuYjnG8MJcSgvilBbkMqwwTmn+weHSgjjDwvhhHX0KOsZHw3lxrfEMVQoPkUHCzCjJj1OSH2dyxdt37CfT2tbOzn0t1O9toWFv84GQadjbQsO+KGwa97fSuD+6r2vcTeP+Vnbta2FPc+9Hw+fHcygtiFOUF6coL0ZxfrjPi1OUH6MkPxpXnBejKL/LfV70XIryD/YvzI1pE9wgofAQOYLFYzmMDPtJUtXW7uze38qu/QcDZldC0HQM72lqZW9z24H73U2tbN3VxJ7maNye5jaau9nclkwsxyjMjVGQm0NBbizcckJbwi2eQ2FerFOfgngstEX983NjFMRj5MVzyA+3vI5bLCe0x8iNmTbhpUjhISJJxXKMsqJcyooO/9f2LW3t7G1uY29zK3uaDt5H4XIwfPa3tLGvpY39Le0HHje1tIe2KJi27W5mfxhO7H+48uI55McOhkuyoMmLxw60J/btGB+P5ZCbY9F9zMiN5RCPGbk54T60x8NwXiyaJmmfMK8D84jlEM8xYjnZEXQKDxEZcLmxHMoKcygrHJjTvrg7Ta3tScNnf3MbTW3tNLcm3MJwU2vbgbbEPk1J+ja3trNzb3M0rq3z+KaWdlra2mltT8+PrvNCoMQTwiWek0Msx4jHjFnjyvjeFXMGtAaFh4gMemZ2YPPV23+Nkz7uTmu709rmNLe10xoCpaWtnda26L6lzWltD/dhuKU9Gt/a1h6m662Pd5p3W7vT0ua0tUdtVeWFA/5cFR4iIv3EzMLmKijkyD6TgA5rEBGRlCk8REQkZQoPERFJmcJDRERSpvAQEZGUKTxERCRlCg8REUmZwkNERFJ2xF7D3MzqgHWHMYsKYFs/ldOfVFdqVFdqVFdqjsS6Jrl7ZW+djtjwOFxmVtOXi8Cnm+pKjepKjepKzVCuS5utREQkZQoPERFJmcKje3dmuoBuqK7UqK7UqK7UDNm6tM9DRERSpjUPERFJmcJDRERSpvDowszmmdkKM1tpZjeledl3mdlWM1uc0DbCzJ4wszfDfXloNzP7r1Dna2Y2dwDrqjKzp8xsmZktMbNPZUNtZlZgZn82s0Whrn8O7VPM7KVQ18/NLC+054fhlWH85IGoK6G+mJm9YmaPZFlda83sdTN71cxqQls2fM6Gm9mDZrY8fNZOy3RdZjYzvE4dt11m9ulM1xWW9ffhc7/YzO4Lfw/p+4y5u27hBsSAVcBUIA9YBBybxuWfBcwFFie0fR24KTy+CfiP8Pg9wO8AA04FXhrAusYCc8PjUuAN4NhM1xbmXxIe5wIvheU9AFwe2u8Arg+PPwHcER5fDvx8gN/PzwD3Ao+E4Wypay1Q0aUtGz5ndwMfDY/zgOHZUFdCfTFgMzAp03UB44E1QGHCZ+uadH7GBvTFHmw34DTgsYThLwJfTHMNk+kcHiuAseHxWGBFePx94Ipk/dJQ42+A87OpNqAIeBk4heiXtfGu7ynwGHBaeBwP/WyA6pkAPAmcAzwSvkwyXldYxlreHh4ZfS+BYeHL0LKpri61XAA8lw11EYXHemBE+Mw8Arw7nZ8xbbbqrOMN6VAb2jJptLtvAgj3o0J7RmoNq7tziP7Lz3htYdPQq8BW4AmiNccGd29NsuwDdYXxO4GRA1EX8B3gC0B7GB6ZJXUBOPC4mS00s2tDW6bfy6lAHfDjsKnvh2ZWnAV1JbocuC88zmhd7r4B+AbwFrCJ6DOzkDR+xhQenVmStmw9ljnttZpZCfAQ8Gl339VT1yRtA1Kbu7e5+4lE/+mfDBzTw7LTUpeZXQRsdfeFic2ZrivB6e4+F7gQ+DszO6uHvumqLU60yfZ/3H0OsIdoc1Cm64oWFu07uBj4RW9dk7QNxGesHLgEmAKMA4qJ3s/ult3vdSk8OqsFqhKGJwAbM1RLhy1mNhYg3G8N7Wmt1cxyiYLjHnf/ZTbVBuDuDcDTRNuZh5tZPMmyD9QVxpcBOwagnNOBi81sLXA/0aar72RBXQC4+8ZwvxX4FVHoZvq9rAVq3f2lMPwgUZhkuq4OFwIvu/uWMJzpus4D1rh7nbu3AL8E3kkaP2MKj84WANPDEQt5RKup8zNc03zg6vD4aqL9DR3tHwlHd5wK7OxYje5vZmbAj4Bl7v6tbKnNzCrNbHh4XEj0B7UMeAq4tJu6Ouq9FPiDh43A/cndv+juE9x9MtFn6A/uflWm6wIws2IzK+14TLQdfzEZfi/dfTOw3sxmhqZzgaWZrivBFRzcZNWx/EzW9RZwqpkVhb/PjtcrfZ+xgdzBNBhvREdLvEG07fwf07zs+4i2X7YQ/afwt0TbJZ8E3gz3I0JfA24Ldb4OVA9gXWcQreK+Brwabu/JdG3A8cAroa7FwFdC+1Tgz8BKos0M+aG9IAyvDOOnpuE9PZuDR1tlvK5Qw6JwW9LxGc/0exmWdSJQE97PXwPlWVJXEbAdKEtoy4a6/hlYHj77PwXy0/kZ0+lJREQkZdpsJSIiKVN4iIhIyhQeIiKSMoWHiIikTOEhIiIpU3iI9IGZ7Q73k83syn6e95e6DD/fn/MXGQgKD5HUTAZSCg8zi/XSpVN4uPs7U6xJJO0UHiKpuRU4M1zb4e/DiRn/08wWhOs3fBzAzM626Boo9xL9WAwz+3U4GeGSjhMSmtmtQGGY3z2hrWMtx8K8F1t0/Y3LEub9tB289sU94VfGImkT772LiCS4Cficu18EEEJgp7u/w8zygefM7PHQ92RgtruvCcN/4+47wqlUFpjZQ+5+k5nd4NHJHbv6ANGvrk8AKsI0fwzj5gCziM5d9BzR+bSe7f+nK5Kc1jxEDs8FROcyepXoNPUjgelh3J8TggPgk2a2CHiR6CR10+nZGcB9Hp05eAvwDPCOhHnXuns70eliJvfLsxHpI615iBweA25098c6NZqdTXRa8cTh84guyLPXzJ4mOt9Qb/PuTlPC4zb0tyxppjUPkdQ0El2Kt8NjwPXhlPWY2YxwttquyoD6EBxHE506vkNLx/Rd/BG4LOxXqSS6TPGf++VZiBwm/bcikprXgNaw+eknwHeJNhm9HHZa1wHvTzLd74HrzOw1okuTvpgw7k7gNTN72aNTt3f4FdGlRBcRndX4C+6+OYSPSEbprLoiIpIybbYSEZGUKTxERCRlCg8REUmZwkNERFKm8BARkZQpPEREJGUKDxERSdn/B2T+nv9a9ZoOAAAAAElFTkSuQmCC\n", 507 | "text/plain": [ 508 | "
" 509 | ] 510 | }, 511 | "metadata": {}, 512 | "output_type": "display_data" 513 | } 514 | ], 515 | "source": [ 516 | "plt.plot(J_history)\n", 517 | "plt.xlabel(\"Iteration\")\n", 518 | "plt.ylabel(\"$J(\\Theta)$\")\n", 519 | "plt.title(\"Cost function using Gradient Descent\")" 520 | ] 521 | }, 522 | { 523 | "cell_type": "markdown", 524 | "metadata": {}, 525 | "source": [ 526 | "### Plot Decision Boundary" 527 | ] 528 | }, 529 | { 530 | "cell_type": "code", 531 | "execution_count": 16, 532 | "metadata": {}, 533 | "outputs": [], 534 | "source": [ 535 | "def mapFeaturePlot(x1,x2,degree):\n", 536 | " \"\"\"\n", 537 | " take in numpy array of x1 and x2, return all polynomial terms up to the given degree\n", 538 | " \"\"\"\n", 539 | " out = np.ones(1)\n", 540 | " for i in range(1,degree+1):\n", 541 | " for j in range(i+1):\n", 542 | " terms= (x1**(i-j) * x2**j)\n", 543 | " out= np.hstack((out,terms))\n", 544 | " return out" 545 | ] 546 | }, 547 | { 548 | "cell_type": "code", 549 | "execution_count": 17, 550 | "metadata": {}, 551 | "outputs": [ 552 | { 553 | "data": { 554 | "text/plain": [ 555 | "" 556 | ] 557 | }, 558 | "execution_count": 17, 559 | "metadata": {}, 560 | "output_type": "execute_result" 561 | }, 562 | { 563 | "data": { 564 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJztnXecFFXywL+1S84sOSdRJEc9DOwSBEwoCiZMGDARxHCnpz9AzzvzIZz5RNE7D1RUDhXPvKCI4oJkRNICS85xl2V36vfHzCzDsmF2pift1Pfz6c9M93S/V93T3fWq6r16oqoYhmEYRqAkRFoAwzAMI7YxRWIYhmEEhSkSwzAMIyhMkRiGYRhBYYrEMAzDCApTJIZhGEZQRFSRiMibIrJTRJYX8nuKiBwQkcWeZVy4ZTQMwzCKpkyE658KvAi8U8Q+36vqJeERxzAMwygpEbVIVHUusDeSMhiGYRjBEWmLxB96isgSYCvwgKquyL+DiIwARgBUrly5W5s2bcIsomEYRmyzcOHC3apaJ5Bjo12RLAKaqephEbkImAm0zr+Tqr4OvA7QvXt3TUtLC6+UhmEYMY6IbAz02KjutaWqB1X1sOf7bKCsiNSOsFiGYRiGD1GtSESkvoiI5/tZuOXdE1mpDMMwDF8i6toSkWlAClBbRDKA8UBZAFV9FRgC3CUiOUAmcI1aumLDMIyoIqKKRFWvLeb3F3F3DzYMI0Y5fvw4GRkZZGVlRVoUA6hQoQKNGzembNmyjpUZ7cF2wzBinIyMDKpWrUrz5s3xeKqNCKGq7Nmzh4yMDFq0aOFYuVEdIzEMI/bJysqiVq1apkSiABGhVq1ajluHpkgMwwg5pkSih1D8F6ZIDMMwjKAwRWIYRlzw8ccfIyL89ttvBf5+8803M2PGDL/L27p1K0OGDAFg8eLFzJ49O++31NRUfvzxxxLL2Lx5c3bv3l3i4yKNKRLDMOKCadOmcd555zF9+nRHymvYsGGe4nFKkcQqpkgMw4g+UlLci0McPnyYefPmMWXKlDxFoqqMHDmStm3bcvHFF7Nz5868/Zs3b86f//xnevbsSffu3Vm0aBEDBgygVatWvPrqqwCkp6fTvn17srOzGTduHO+99x6dO3fm6aef5tVXX2XixIl07tyZ77//nl27dnHllVfSo0cPevTowbx58wDYs2cP/fv3p0uXLtxxxx3E6jA56/5rGEapZ+bMmQwcOJDTTz+dpKQkFi1aRHp6OqtXr2bZsmXs2LGDtm3bcsstt+Qd06RJE+bPn8/YsWO5+eabmTdvHllZWbRr144777wzb79y5crx+OOPk5aWxosvuoe9ZWZmUqVKFR544AEArrvuOsaOHct5553Hpk2bGDBgAKtWreKxxx7jvPPOY9y4cXz22We8/vrr4b0wDmGKxDCM6MFrhcyZc/J6ampQxU6bNo17770XgGuuuYZp06Zx/Phxrr32WhITE2nYsCF9+vQ56ZhBgwYB0KFDBw4fPkzVqlWpWrUqFSpUYP/+/SWq/+uvv2blypV56wcPHuTQoUPMnTuXjz76CICLL76YmjVrBnOaEcMUiWEYpZo9e/bw7bffsnz5ckSE3NxcRITBgwcX2RW2fPnyACQkJOR9967n5OSUSAaXy8X8+fOpWLHiKb+Vhq7RFiMxDCN6SE11L8nJ7sW7HgQzZszgxhtvZOPGjaSnp7N582ZatGhBUlIS06dPJzc3l23btvHdd98FXEfVqlU5dOhQoev9+/fPc3uBOzgP0KtXL959910APv/8c/bt2xewDJHEFIlhGKWaadOmMXjw4JO2XXnllWzfvp3WrVvToUMH7rrrLpKTkwOuo3fv3qxcuZLOnTvz3nvvcemll/Lxxx/nBdsnT55MWloaHTt2pG3btnkB+/HjxzN37ly6du3Kl19+SdOmTYM610ghsdpLoDBsYivDiC5WrVrFmWeeGWkxDB8K+k9EZKGqdg+kPLNIDMMwjKAwRWIYhmEEhSkSwzAMIyhMkRiGYRhBYYrEMAzDCApTJIZhGEZQmCIxDKNUIyLcf//9eevPPfccEyZMKPKYmTNnnpTSJBBKmhJ+1qxZPPXUUwXWP3XqVLZu3Vqi+r1JJcOBKRLDMKKK/EPbgh3qVr58eT766KMSvdSdUCQlZdCgQTz00EMF1h+IIgknpkgMw4gaJkyAsWNPKA9V93oxBkSRlClThhEjRjBx4sRTftu4cSN9+/alY8eO9O3bl02bNvHjjz8ya9YsHnzwQTp37sy6detOOuaTTz7h7LPPpkuXLvTr148dO3YAhaeET09Pp02bNtx22220b9+eYcOG8fXXX3PuuefSunVrFixYALiVxciRI0+p/+mnnyYtLY1hw4bRuXNnMjMzWbhwIcnJyXTr1o0BAwawbds2ABYuXEinTp3o2bMnL730UuAXraSoaqlaunXrpoZhRA8rV670az+XS3XMGFVwfxa0HgiVK1fWAwcOaLNmzXT//v367LPP6vjx41VV9ZJLLtGpU6eqquqUKVP0sssuU1XVm266ST/44IMCy9u7d6+6PML885//1Pvuu09VVUeNGqWPPfaYqqp++umnCuiuXbt0w4YNmpiYqEuXLtXc3Fzt2rWrDh8+XF0ul86cOTOvzrfeekvvueeeAutPTk7WX375RVVVs7OztWfPnrpz505VVZ0+fboOHz5cVVU7dOigqampqqr6wAMPaLt27Qo8h4L+EyBNA3zvWvZfwzCiAhHwGg2TJrkXgDFj3NuDSZJbrVo1brzxRiZPnnxSBt758+fnpXG/4YYb+OMf/1hsWRkZGVx99dVs27aN7OxsWrRoAVBkSvgWLVrQoUMHANq1a0ffvn0RETp06EB6enqJzmX16tUsX76cCy64AIDc3FwaNGjAgQMH2L9/f17OsBtuuIHPP/+8RGUHirm2jJjDaR+6ET34KhMvwSoRL/feey9TpkzhyJEjRdRffEWjRo1i5MiRLFu2jNdee42srKxij8+fht43RX1JU9KrKu3atWPx4sUsXryYZcuW8eWXX6KqEUtJb4rEiClC4UM3ogfv/+mL7/8dDElJSVx11VVMmTIlb9s555yTN/Xuu+++y3nnnQecmgbelwMHDtCoUSMA3n777bztTqaELyot/RlnnMGuXbuYP38+AMePH2fFihXUqFGD6tWr88MPP+SdT7gwRWLEDKqwf7/b5eF9uYwd617fvz/2LBOzrE7G9/8cMwZcLven7/8dLPfff/9JvbcmT57MW2+9RceOHfnXv/7FJI8/7ZprruHZZ5+lS5cupwTbJ0yYwNChQzn//POpXbt23nYnU8Lnr//mm2/mzjvvpHPnzuTm5jJjxgz+9Kc/0alTJzp37syPP/4IwFtvvcU999xDz549C5xEK1RYGnkjpvB92XhxwocebiZMcCs/r9ze86pRo/RZVyVJIx9P1yWSOJ1G3oLtRkzh9aH7KpJYUyK+lhW45fdtiavG1vk4yYQJJ5+/9/+O1+sRK5hry4gpQulDDxfel6PXbZOQcEKJ2Evz1POP9+sRC5giMWKGcPjQw0UoeydFI6XNhR7LhOK/MEVixAwibl+5b8vd27KvUSO2XsKlwbLylwoVKrBnzx5TJlGAqrJnzx4qVKjgaLkWIzFiitLgQ89vWfnGSCD2zqc4GjduTEZGBrt27Yq0KAZuxd64cWNHyzRFYsQcse5DL8yygtizrPyhbNmyeaO/jdKJdf8txeTv/ROPvYGi+RpEs2xG/BFM99+IxkhE5E0R2Skiywv5XURksoisFZGlItI13DLGKqVtBHggg/ei/RrEumVlGF4iHWyfCgws4vcLgdaeZQTwShhkinlK2wjwQBRCabsGhhHNRDRGoqpzRaR5EbtcBrzjSXH8k4jUEJEGqrotLALGKKHMohpuAh28V5qugWFEOxGPkXgUyaeqesqckCLyKfCUqv7gWf8G+JOqpuXbbwRui4WmTZt227hxY6jFjglU3YPdvLhcYXyBpqS4P1NTgy4qmLQoEb0GhhFDxGyMxA8KeuRP0Xyq+rqqdlfV7nXq1AmDWNFPaRqnEOjgvdJ0DQwjmol2RZIBNPFZbwxE78TFUUJER4CnpLiXOXPci3c9CAJRCOG8BpbF14h3on0cySxgpIhMB84GDlh8pHiCHacQTd1SAx285+81CPZcLVutYURYkYjINCAFqC0iGcB4oCyAqr4KzAYuAtYCR4HhkZE0dvC+CL0jwL34OwI86BejNybiUIwkGKVY3Cj4YM/VsvgahodAJ3uP1qVbt24FTnYfD4wfrzpmjKrL5V53udzr48f7d7x3fzhRTv51v0lOdi8Okb/uEslSSHlOnKvvcd6lxNcqBnH6/zAiD5CmAb53I/7id3qJV0ViL8aS49S5ulwnl1Ear5UvwTZYjOjEFIkpElW1F2MgBHuu8aR4VR22Wo2owhSJKZI87MXoP8Gea7y+VOPpHoknglEk0d791ygBGuS4Ce/xpWHiqOJw4lxL0/woJSHeJuUyiifau/8afpL/xRjIHBfxlN7cqXMtDfOjlJTCGiyl/byNwol4ihSniec08k6NafB9MRa0np/MI1ns2ryHnZt2s3fbPg7vO8LBvYc4vO8Ih/Yd5tC+Ixzae5isI1nkZOdw/FgOx7NzOH7suGf9OAAJiQl5S2KZRBITE0gok0j5iuWoWKUCFSqXp0Ll8p7v7qVqUhWq1arq+axCtaQqVK1Vleq1q1KtVlUSEoo2ukt6rtFCpOQuqsFiecxim2BSpJhFUopwqnVcUHrzg3sOsWH5JtKXbyZj9VZ2bt7Nzk3u5eCeQwWUIVSpUYmqSVWoUrMKVWpWpnajJMqUK0PZ8mUoW7YMZcuXda+Xc9+GrlwXubku92dOrufTRXZWNpmHs8g6kkXWkWPs33mQzMNZZB7K5NC+I+Tm5BZ4HgmJCdSsV52kBjVJql+DpPruz1qNkqjbtHbeUrlapULPPRqJ5CDIeLJaDf8xRVLKcGKOi6OHMlk6ZyWrfvqd335ZS/ryzezdti/v90pVK1K3mfsl3Oas06jXrE7eSzmpQU2qJlWhcvVKxVoDTqCqHD2UyaG9hzl45XUcyhEOPjyBA7sOsnf7PvZt38+e7fvZvWUvaxauZ//OA7hcJ1vhlatXypO/0WkNaNa2MU3bNqZZ28ZUrVkl5OdQEjQKBkHGozvPKBpTJAYAW9Zu4+fPFvHzZwtZOmclOcdzSUhMoEWHpnTr35EW7ZrSvH0TWnRoSq2GSUio3holHBEvIlSuVonK1SpRv2qOe+M15xa6f25OLnu372eXj0W1c9NudmXsYXv6ThZ/u5xjmdl5+yfVr+FWKmc2pnn7prTs1IwW7ZtQsUrFAE8wOKIlPb5NymX4YjGSOMXlcrFs7irmf5LGgtmL2LzanQuz6ZmNOPuirpx1UVfanN2aCpXKh1ewQFKreI+ZM8f9mZxc8jI8uFwudmzcxcYVGWz+bQsbV2awcVUGm1ZmcPRQJuBWXg1a1aNlx2Z5S+uuLajTpHaegg11DEMtPb7hMBYjMfxm2/odfDH1O756Zw47N+2mbLkydOrdjkF3D+Tsi7vSoGW9wAoONrdWfmXg4HwmJSEhIYHX3q7H/v31mDixW14M4t57lYqyi0G9N7J+6UbWL01n/dJNzPt4Ad7GWPXaVTmta0t2HGmJVG/JE5Nb0KBFXUAcjWFYrykj2jBFEgdkZ2Uz54P5fPHWdyxJXYGI0K1/R25/+nrOvrhrxNw0juFgosjCYhCTJwtjxtSl56C6nHNZj7z9Mw9nsmHZJtb+ms6ahev4fdF6MpbNAlcuN82GqklV0KqnsST9dPpc3pojB8+gcrXAr3ew3bxjtZeaEd2Ya6sUk3M8h09f/YrpT3/Mnq37aNiqHv1v7s0FNyZTt0ltZypx0K10UnmBHO+QFeP7svZSkhjEscxsRt+yic+mb6Aaa6nOGqpIBqAkJCZwRo9WdEppT5c+7Wl/XhvKVShXIvkC7bVlKe+NogjGtWWKpJSyZM4KXhw5hfQVm+mY3Jbr/nwFXft1dD5IHk2KxEGCjUHkP/7Q/qOsXrCGJakrWJK6gtW/rCM3J5dyFcrSMbkt3S7oRLf+nWjerolf/1FJLQsnxn+YNVO6MUXiQ7wrkj3b9vHPP/6Lb979nnrN6nD3pOH0vLR76HpZeYkSBeAEwVok/hyfeTiTpXNXsfDLJSz8agmbVm0BIKlBTbr170j3/p3pPqAT1ZKqRsV5mTVT+glGkUQ8yaLTSzwlbfRNkpdzPEc/fOFTHVT9Br2w/DX61qPTNPNIVkBlFbReLA7PPxIpgk3EGOjxOzbt0s+nfKNPXPN3HVzrZu0nQ7R/4lAdc94j+p+/faRrF29QlwNZEQNJ6hmvySnjDYJI2mgWSYzi20LcsmYrT1wzkXWL06nRqhMTZ99K49YNAirLWpvBX49gj8/NzeX3tPX8/NlCFsxexJpFGwCo07gW515+Fr2vO48zz25dYiszGIskWCvNiH7MIokRi8SpWeV8W4Qjrlmjg2vdrAMqDte6zNfRo10lKjdqWpuRsmgKqTfY/8rJGQR3bdmjn0/5Rsdd/rReWOFa7SdD9KbTR+m//zJDt6fv9FueYP/neJqnJh4hCIvEuv+GCSdb/d7RzUcyVvD79KfIpjq/8n+MGFOvxC3EiIyUdmrMSQjjMcGO3A74+ALOrXbDJAbe0oeBt/ThyMGjfD/jJ7761xymjpvO1HHT6ZTSjn43JNNryB+oVLXgrsXB5sjy3q++2NgVI49ANVC0LtFokYSi1f/TZwv1oorX6h+4V8uxJ+gWYlhbm75WgPe7t2J/LBMnrJdA6g0HJZBj24Yd+q/HP9AbW4/UfjJEL650nT590z902fcrC42nBGIpRcpqtXnhwwtmkUQ3Trf653wwnyeHTaJsUjMWZj7CcaoBgbcQw9baLGj0+uLF0Llz4MdDqegpFsi51W9el+v/bwjDHr2SVT/9zpdvz+G7aT/w1TtzaFoph4v+cisDhvemSo3KeccEYilFIuOvxe1ijEA1ULQu0WiReHGi1b/s+5U6sNzVOqjRo5rI4aBbiGFtbRZlBVSv7l4CPT5YmSKNQ+d29HCmfn5Gfx1d/TLtJ0N0ULUb9JX7puqOjf7FUoqiOAshFDFA6yUWPjCLJPpxotW/K2MPjw99nnrN69J48EO0yKocdAsxrK3NYFOZOJgKJepw4txSUqgIDFw9h4HA2m59mZGRxceTZ/Px5Nn0u6EX1z58RYl69PlSlDUTihggRDbDsVECAtVA0bpEo0XiRAvrWOYxveesP+mlVa/X9BWb8srNX0+B+NG6Das/OhpiJFHCKde9V3Lg51bItdyxcae+NOZNvajitdo/cag+deNk3fRbRlBynyRziCwI6yUWXjCLJLpxotX/8r1TWf3LOiZ89CDN2jbJKzd/PcHI6FRZxRKsJVFKLJECW/FdUt2t+EAKLMSqqQvc/cJwrnnocmY8/wmfvPIl3/z7e1KuOYdhj1yZdz8FSigsCCcseCOMBKqBonWJRovES6Ct/sXfLdd+MkRff/CdklUYrT2TCiKaZQsBIY0DFHMt9+3cr2889G+9tOr1ekHCUH3yhkm6e+veImUtat13uxMWhMVIIgNmkcQGgbT6NTmFt5dUp3aj+tz0+NWhEcwIOyGNAxRjsdWoU51bnxzG0AcG8cFzs/hw4qfMn5XGrX8bxsV39CMxMTFvX39jH05aEDYvfAwSqAaK1iWaLZJAWNypn/aTIfrxP2YHXkictfZjiVNa8cHESAJk8+9b9cF+j2k/GaIjz35I1/y6Pk82fyyDUMZIilo3nIUgLJKE4hSNiJwuIt+IyHLPekcReTTkGi7eSUmBlBT+vSSbJM3kovefPuH7NiKH539xggJb8etGomFOf9e4dQOe/vL/eOhfo9mevot7ejzEq/e/TdaRTCZOdFsGkya50+IXlHa+MAtizJjgLAibFz52KDZpo4jMAR4EXlPVLp5ty1W1fRjkKzGlJmljSgrLDpTlviU1uEsXc0VyI/f2UhJojllCMHnWmDEw8dcUxq4byaQtQxjDC0zsNdP94gzz/31o32GmPPQun/3za+o0qcWYV0Zw1oVd/ZqbRW2+kpgmmKSNxVokQCVVXZBvW04glcUj+fW0363N1FQ+aXsxVcu4uOi8Ru4XSrwpkcJa/w5aBSWWZc4c9xKkDAW24lu9yBheoAb7I/YCrlqzCve+dgcv/PAElatX4tFLnuS6s/8D5ObtM3ZswfexWRDxiz/B9t0i0gpQABEZAmwLqVSlhGAGaWUdPcb8WWn0qX2MColF72tEkCAslAkTfFrtqakIMDE5JSKWSH7anXMGL/78FNef9Sa70z7m8uYbeHvhGMY9XsXv+eGN+MEfRXIP8DrQRkS2ABuAYSGVqhSg6lYivg+dryujOLN/wexFZB05RvLMJ6Fvh/AIHS0UlnfKSyRybRU28rwoq8QP+aK5FV++YjnaXXkn6+u1Ytf3Uxh51kOM//BBoJn1njJOokhFIiIJQHdV7ScilYEEVT0UHtFim2C7d6a+/yM161WnU0rb0ApqBEaoEkhGmfvSbTVdwKqfmvLYkOe599xHGPvPu+hz7blBl20xldKDP8H2uaraKySViwwEJgGJwBuq+lS+328GngW2eDa9qKpvFFVmtAXbVfErUOlL9rHjXJF0M/1v7s3ol26L3AMX6ZxWhdUfabl8ZfAqkuRk92dqatG/xTB7tu3jL1c9z4p5q7lx/FVcP25IiWdp9GLZfaOPUAfbvxKRB0SkiYgkeZdAKvNFRBKBl4ALgbbAtSJSUPP7PVXt7FmKVCLRRmGDtIoLuK9fks6xzGy69GnPhAknH+Mt0x62COPt/JCc7F5KQ2eIYjoQ1GpQk2e/Gc8FNyXzzmPv84973sDlcpW4Gl+3r/fe9rp99+8vQYcUI2rwJ0Zyi+fzHp9tCrQMsu6zgLWquh5ARKYDlwErgyw3Kjile+fEk+e8Lsq9tfqXdQCc3r0VH34feJwlYKJl3o/C6ov2F3YpzlJctlxZHnzzHmrWqc77z81CEoSR/7i1RJaJZfctfRSrSFS1RYjqbgRs9lnPAM4uYL8rRaQX8DswVlU3599BREYAIwCaNm0aAlFLTjBpHlanraVG3erUbVo7sAcuUi+wUvjiLJbScK4lbDiICLc9fT2qygfPf0LZcmW44/mbAlIm3nsaTInEMsUqEhEpC9wFeOMkqbgHJx4Psu6Cbpn8Ru0nwDRVPSYidwJvA31OOUj1ddw9y+jevXvUGMYnde/kxMNT3MPy+y/rOKNHq7wHM+wPXCluUYeVUnzdRITbn7mB49k5fPjCZ5SrWI5b/nqd38c7mZvLiDz+uLZeAcoCL3vWb/Bsuy3IujMA3/zVjYGtvjuo6h6f1X8CTwdZZ9gpafdOVWXr2u2cfXE3z3oJHrhIuaSixRVmBEaADQcR4e4XhnM86zjTnvyYes3qcPGICwre2afsYNy+RnTijyLpoaqdfNa/FZElDtT9C9BaRFrg7pV1DXBSk0ZEGqiqd/DjIGCVA/VGNYf3H+F4dg5J9WtE/oEzRWAUg4gw6qXb2LVlD5PveYO6zerQY0DnYo6x7L6lDX8USa6ItFLVdQAi0hLffAkBoqo5IjIS+AJ39983VXWFiDyOOwvlLGC0iAzCnZJlL3BzsPVGO/t2HACgZr3qJX/gIuWSMldY6SDA/y2xTCKPTBvL2F7/xxNX/Z0XfvgLLTo0c/9YiLU6wWOZlNTta0Qn/iiSB4HvRGQ97rhGM2C4E5Wr6mxgdr5t43y+Pww87ERdscK+HfsBqFGvBhB4nMUwwkmlqhV54pOHGfWHh3nkkif5x09PUqtBzSKPieZR/UbJ8KfX1jci0ho4A7ci+U1Vj4Vcsjjl4G534oAadarlbSvxAxcpiyBaLRGzlMJCnca1eOKThxnb6/94ctgknv1mPGLWalzgz3wk9wAVVXWpqi4BKonI3aEXLT7xZhqw1pkRi5zWpQV3TRzOktQVfPHWd5EWxwgT/oxsv11V93tXVHUfcHvoRIpvEhLdf0lubslHDBv5cDj1u+EfA2/pTYdeZ/L6g+/kuWrDNfI/4GkbjKDwR5EkiM9II09qk3KhEym+8SoSlykSI0ZJSEjg3lfvIOvIMV65b2rY6rV0QpHDH0XyBfC+iPQVkT7ANOB/oRUrfkmMF0USDuugNObDihGatmnEtQ9fwXfT5vHLF4tDXp/l74os/vTa+hPu9CN34Q62fwnEVPLEWKJStUoAHDlwNMKSGEZwXP3Q5Xz97lxef+AduvbrQGJi6GZos/xdkaXYNPIn7ezO+ttYVZeGTqTgiLY08iVl67rt3NR6FA+efpD+v30RaXGcp5SmWDcKZu6M+fzlqr9z/xt3MfCWU7IbOU4g0zYYbkKaRl5EUkWkmkeJLAbeEpG/B1JZrBLOAF6Sp+/9nmx/vI6GEd2cf+UfaHN2a6aOm07W0dCOGgh02gYjePx5W1VX1YPAFcBbqtoN6BdasaKHsAbwUlKocNEAqmg2ezdsi55eRk7KYXGLuEJEGPHMDezZuo+PXvgsZPXkTyfkcrk/fWMmRujwR5GUEZEGwFXApyGWJ6ooaQDPKculNplsp3JgBxtGlNHh/DPpOag77z0zkyMHQxP7Kyyd0Jgxlr8rLKhqkQswFFgKvOxZbwl8WNxxkVq6deumTuJyqY4Zo+pWC+5lzBj3dl/Gjz95u/e48eNLXucTdS/R6ysODlb04ElOdi/eE/euG0YJ+W3BGu0nQ/T952b5fUz+Zyz/ulPHGG5w5zgM6L1brEWiqh+oakdVvduzvl5VrwydaosufHuDeMnfC8TproctK+ewPSsxZK03wygWh92qZ/Q4jc692/HRC59yPLv4qYwCdSlb/q7IYBHdYvAngOdrRk+a5O414pv6vaQ3c8t/PAHAhmWbgpQ+SCyeYTjIVX+8nN1b9vLtf34ocj8bExJ7mCIpgpIE8PyxXPylRUd3Cu51i9MDF94wAiGEaWW69+9E0zMb8dnrXxW5n9MNMyP0mCIpgpIE8JzselincS1qNazJih9/C+4EnMIsEcMBRIR+1yez6qc17Mp2xI/LAAAgAElEQVTYU8y+zjXMjNBTpCIRkTae1ChV8m0fGFqxoocJE06+gb03uK+v1umuhyJCp5R2LPluRV42YMMICyF2Z547+CwA5s1cUOR+NiYktihUkYjIaOC/wChguYhc5vPz30ItWDRRXAAvFF0POyW3Y+/2/WxevbX4nQMlWsapGHFD0zaNaHpmI+Z9/HOh+9iYkNijqFxbtwPdVPWwiDQHZohIc1WdhDvnluGD0zMZdurdDoAlqSto2qaRfwfZ5EGGU4TwHjr38rN475n/cnDPIarVqnrK7zane+xRlCJJVNXDAKqaLiIpuJVJM0yRFIiTXQ8btqpPnSa1SPtiMZfe2T84wfJTyDzapoCMcHDOZT2Y9uTHpH25hD7XnlfgPjbFdGxRVIxku4h09q54lMolQG2gQ6gFi3dEhHMG9SDti8VkHskqemebwMmIIVp3a0mlahVZmrqiyP1sTEjsUJQiuRHY7rtBVXNU9UagV0ilMgA45/KzyM46TtoXS5wtOBrGh5iyi1sSExPp2Ksti4tRJEbsUKhrS1UzivhtXmjEMXzp2OtMqtaszLyZP3P+FWcXvqNXCZiLyggHDtxnnVLa8dOnC9m9ZQ+1G9VyRCwjcvgzsZURIcqULcMfLu3O/FlpZB87TrnyZZ2tIBIKx+IzBm5FArB0zkr6XHd+hKUxgsUGJEY5va85l8P7j/DzpwuL39kGDhqhxMFYXMuOzShXoSy/L1zvoIBGpPDbIhGRar77q+rekEhknETXCzpSq2FNvnw7lfOv/EOkxQkec8MZQGKZRFp0aMq6JemRFsVwgGIViYjcATwOZALeoUCKO528EWISExPpd30vPnj+E/Zu30dS/ZqRFsmIVxxuBLTq1JzvP/oZVUWitEuWbxfkgtYNN/64th4A2qlqc1Vt4VlMiYSRC25KwZXr4pt/fx9pUZzD3HBxz2ldWnBo72F2bd4d0noCnXAurLOjxjj+KJJ1gE2MEUGandmYtj1P59PXviQ3JzfS4hjxjkONgObtmwKwcWWhHUSDJlBlYKnsS4Y/iuRh4EcReU1EJnuXUAtmnMxVD17G1nU7ip3LwTBihQYt6wKwPX1XSMoPRhlYKvuSIcVllxWRBcAPwDLA5d2uqm+HVrTA6N69u6alpUVaDMdRVe7s+iDHjmYzZcVEEsskRlokwwgKl8vFJZWGMXjMxdz+9PUhqcNXeXgpiTJQdSsRLy5X6VUiIrJQVbsHcqw/FkmOqt6nqm+p6tveJZDK4p1AfbXgTplyw7ihbFmzjW+nmVVixD4JCQnUbVaH7ek7Q1ZHMPOaWCp7//FHkXwnIiNEpIGIJHmXkEtWynAicHfOZT1o2akZ//7LDL/mvTaMaKdes9rs3Bga1xYErgwslX3J8EeRXIcnTgIs9Cylz3cUQpwK3CUkJHDLX69j69rtfPba16ERNpBBZpY3ywiQ6nWqcXDPoZCUHYwyCMUcQ6WZYseRqGqLcAhSmvE1rydNOuGvDSRwd9aFXejStwP/evwD+t3Qiyo1KjsvsGGEiSo1qnBo3xHHy/WO9/Aqg7//veTzmlgqe/8pNtgOICLtgbZABe82VX0nhHIFTDQH24MK3PkMAlv76wbu7v4nhj4wyLkgZf4cWMnJefU5eowRPmIge8Bbj05j+lMf83n2dBISnMnYNGGC29L3vvRdLrjvPrfyyK8cjBOENNguIuOBf3iW3sAzwKBAKiug7IEislpE1orIQwX8Xl5E3vP8/rNnpsaYxMnA3WldWtDvhl58PHk22zbscEZAw4gAVWpWweVSMg9lOlJeQW7k++472Y1sSiQEqGqRC+5uvwnAEs96PeCT4o7zo9xE3IMdWwLlgCVA23z73A286vl+DfBeceV269ZNow2XS3XMGFVwfxa0XijJye7F/Qzkre/cvFsvqTxM/++yp5wV1ltfqI4JpHyjZBRyzzhF/vu1yPu3GGa98oX2kyG6e+ve4ITKJ4/3+fIuxT5nhgJpGuD73B9bMlNVXUCOJ3HjTpzJs3UWsFZV16tqNjAduCzfPpcB3q7GM4C+Eq1JeYogFIG7Oo1rcf24ocyflcb8T6LTlWeUPpxOG+J9nNXBblDBdPk1AsOf7L9pIlID+CfuHluHgQUO1N0I2OyzngHkn70pbx9VzRGRA0At4KTkPCIyAhgB0LRpUwdEc56AA3dFJMq74t6L+OqdVF4e8yZd+nagQqXywQsaiD/dn2NsHpLwEaIMy75uI3Dfv769ogJxG3n3V5dziqQwN7Ipk9BRrEWiqner6n5VfRW4ALhJVYc7UHdBf2n+u8mffVDV11W1u6p2r1OnjgOihQan56AuW64so168je3pu3j3iQ+DKyxIghlsacQGoUgbIp4Au1MWiY3/iAz+pJG/VVWnAKhquogkish4VX0syLozgCY+642BrYXskyEiZYDqQNzMg3JSCy811b2eb59OKe3of3MK7z8zk+4DOtEpuV24xTyll4z3Yfb2kgFsHpJIEIJr7FUmvilHgmrpO/xmL8yNDDb+I5T4EyPpKyKzPSPb2wM/AVUdqPsXoLWItBCRcriD6bPy7TMLuMnzfQjwrTrpTA0xwbTSS+KLvmfSLTQ8rT5/u/YF9u3YH6i4AWFZUuMLp9OGZB7OAqBS1Yp+11/UOrifEV/l5lUmlv49hPgTkQeuxh2X2AScG2hkv4ByLwJ+x9176xHPtseBQZ7vFYAPgLW44zItiyszWnptjR9/ck8Rb0+S8eOLPzaQXl7rlqTrRRWv1Qf6TtCcnBwnT6VE8lovmdJLUL0PC+Ht8e9pPxni1z0bzDNlFA9B9Nry52XfGnd6lNeAucCrQKVAKwz1Eg2KxIkHLpCX8+dvfqv9ZIhOHTfduZPxE5frZFlNiZROnH6ZvzL2Lb206vXF7hcKJWacTKgVyW9AX893Ae4HVgRaYaiXaFAkqs600gN5OT8z/EW9IGGoLvh8UeDClxCzSOILJ8eRPHfLS3p1o9v9rtfus9ARjCLxJ0Zylqp+43GDqao+D1weuDMtPgi2L7sG6Ise9eJttOjYlL9e+wIZv5/ou5D/uOLK8RevnNZLJn5wsvfhnu37qVmvht/12viQ6KRQRSIifwRQ1YMiMjTfz050/y3VBKoIfI8N5OVcoVJ5Hvv4jySWSWTc5c9w5ODRkM49bVlSjWDYtWk39ZrV9mvfYJ4pI8QUZqoAiwr6XtB6NC3R4Npywp8brC968XfLtX+Zq/SRS57U0aNzQ+5bdtLdYcQQQaRfcblcemnV6/XF0VP82NdiJKGGIFxbRY0jkUK+F7Ru+OBEX/ZgU1h3SmnH3S8M58VRU7hy7L8YPfpGJk2SoFLYF4XTgy2N0s+hfYfJPJxFvWbFDyK28SHRTVGKRAv5XtC6kQ8n5jII9uU86O4BbP5tCx9O/JRhj5bHPVTHjfmW82EDJUuGAylvtq7dDkD9FnX92t/mB4leilIknUTkIG7ro6LnO571CoUfZniJdCtdRLh70nCOZWbz7hMf0oJENuAOd1nuISPSrF+yEYBWnZr7fUyknymjYApVJKqaGE5BjNAgksCmynewVV20kvd57DFh4Z4hJyXei/mHMRhrIl6SSTp9XoGmvPHZf92SdCpVrUi95tGbH8/wD3+y/xohxNdUL2g9WESgZs0E+o26k4b7Xbwz/j2GP5HAmDFXOONbLq0vXiPkrFuSTstOzYKeGTHUz5BRPKZIIohfyQ4dqkc1EZfrblSVtx6dxk2PK8MeuYKw9ZsIhcJxwpoo7ckkQ21xldQS8ciRm5zC+p9qccEdFwZVfbieIaNoTJFECA3B3A5FIQKJiYk8+NY9SILw9rjpZB3O5NYnhxHQXGHx4hIyQsLaw2XIzE2g3bltAi4j3M+QUTimSCKEb/fFSZMIWbfc/CSWcSuTCpXK894z/2Xvjv2Mfe0OypYrG5oKQ6lwnLQmIqkAQ6mEo8XiyifH4otGwa//pnPvwKc9iNQzZJyKKZIQ4K/P1vG5HfwkISGB0S/fTlL9mrzz2Pts37CT8TMeoHrtav77m6PlBWVEjiD++8XfLaNZ28Yk1a8ZlAiReoaMkzFF4jAl8dkWlvIhHA+CiHDD+KE0PqMhzw5/iVF/+DP1BzxEdtnGzvqbw6FwYlWJhdM9GC3XKDWV49nHWZ40nAHDewddXCSfIeMEwXWXME7C12db3CRP0ZLssPc15/LcdxPIPJzFkjce5j+Tfi7ZBFWpqdHzkjLCQ0qKe5kzx7141/1kSepKso4eo2u/jkGJES3PkGEWiaOUxGcbTSkf2j40gpdbJPBY87NxLXiOTycNZtKkq4FE5/zNwYzxKK2KKk7dg3M/mE/FKhXo1j84RRJNz1C8I1rK1Hb37t01LS0tojKogm/XeJer8Js6lH3g/S7b8yLL/uIrXhw5hc+nfMNu7cwKRnLMVT1yD2S8vGBj9TwDkDvneA5XNxxBt/4d+fO79zoiho0jcQYRWaiq3QM51lxbDlPSVNehSvngV+r4fC6Ksv0vYNPntVmlI0hiOX/gAW6/ckn4XQRBuk5ijjhyDy5JXcHBPYfoNaSnY2Va2pTIY4rEQaLFZ1uSWE3eMcDYdSOZtGUIQ8ZcwOuLn6JarSpsnPkE13R5g6OHMsMjvBE7BKAAU6fPo2KVCvQY2DkkIhmRwWIkDhItPlu/YzU+PnoBaqQMYUxej7NmzNj0FDed9x/2Lv6cOzr/ygNv3k2n5MD7/ftNnMYOSjt7t+/ju+nz6HPd+ZSvWD7S4hgOYjGSEBAtPlu/YzU+L+yCZF/+wyqeu+Ultq7bwWX3DOTWp4ZRsXIYEkCbIilVvDT6TWa98gVTVr5A49YNIi2OkQ+LkUQZ0eCzLVGsxsdFUZDsHc4/k1cXP8floy7kvy/9jxEd7+fn2YtCIXahchmxzY6Nu/j0tS8ZOLy3KZFSiCmSUkgoYjUVK1fgnkm38HzqY5QtX5ZHL3mS8YOfYeu67Y7LXtS6EZu889j7SEIC148bGtDxdl9EN6ZISiGFxWrGjAk+VtOxV1teW/wstz45jEVfL+XWtvfy2gPvcHj/kaDl9qunmRFzLPt+FV9OTWXwqAup07hWiY+3+yIGCHSy92hdunXrFtDE96URl6vo9WDZvXWvPnfLS3pBwlC9ovZwnfni55pzPCegslwu1TFjVMH9WdC6EXscy8rW4WeO0etb3KVHD2eW+Hi7L8IHkKYBvncj/uJ3ejFFEn7WLFqv9/cer/1kiA5vM1q/m/6D5ubmlrgc35eEd8l7WSQnuxcjppg6brr2kyG64H+/BlxGkfeF4RjBKBLrtRXlaJh6gAVbj6oyf1Yabz06jfQVm2nRoSk3PXY151zWo0TznWhhPc2c6sEVRz3BwnXvFEb6is3c1fVBkq8+h4feGR1UWYXeF4ZjWK+tUkq4fMNO1CMinHNZD15d/CwPvzuG7KzjTLjiWe7s8iCp780jNze32DK89foytskMNDkldke5R0jeSMcVsrOyeeamf1CpWiXufP6moMoq8L6wpIxRhSmSKEUDGJ0eDfUkJibS59rzmLJiIn+cOpLj2Tn89doXuOXMe/ns9a84lnmsUDkK7Gm2ZQhj140k6NONo7Qr4bp3iuKVsVNZs2gD90+5ixp1qgdcTqH3hWX4jS4C9YlF61KaYiTh8g2Hsp7c3Fyd++FPelf3P2o/GaJX1B6uU/78ru7K2H3KvuPHn1xvbq57ffx4VU1OVlev5MAF8cZYvCcY6phLuOvLRyTjCl//e672kyH6+oPvOFJe/vvCe27jxztSvOEBi5GcoDTGSMLhGw51ParK8h9+48MXPuXHmb+QkJhA8lU9GTz6Itqc1fokOUROTBD297+75dLkFMauG0mN24YE554JV4wk/6RVycnhqdeHSMQVFnz+KxMGP0Obs1vzzNfjKFPWmSxMkY73xAMWIymlhMs3HI56RIQO55/JhA8f5O01/+DykQP56ZOFjPrDnxlz7iP8781vOXLwaN7MjF7XzH33eeTrksqkLUPC5poJGu+o/ORk9xLmUfqRiCvM/ySNCVc8S7N2TZjw8YOOKRGIjmwRRhEEaspE61JaXFvh6j8fyX76hw8c0Y8mfaY3nzFK+8kQvbjSdfrX6ybqgs8X6fHsnNLR5TMC3ZbD/Z+6XC798IVP9YKEoXp3jz/pgd0Hna3ACAsE4dqy7L9RSrgyCUcyY3HlapUYPPoiLh91Ib8tWMtX78whdfoPfDdtHkn1a9DnuvOpQjKHaQbE6DzcEehmHM7/NDcnl5fvfYtZL3/BuYPP4qF/jaZCJcvsG29EJEYiIknAe0BzIB24SlX3FbBfLrDMs7pJVQcVV3ZpjJHEwjgSp8g+dpwFsxfx1Ttz+PGTReDK5bA2ZgfnMOCGnrz8duPYUyYRItT/6ZGDR/nrtS/wy+e/MvT+S7nt6etJSDBveawSTIwkUorkGWCvqj4lIg8BNVX1TwXsd1hVq5Sk7NKmSOIRr3//5UkHuab3fOrojyydswpQytdqwtWjz6HXlWfT9MzGJRrsaDjH9vSdjLvsaTauzGD0S7dx8YgLHK8jWho38UIsKpLVQIqqbhORBkCqqp5RwH6mSOIUb68tr2tm99Z9jB32E0c3zOfQ5t9QVRqeVp+zBnah+4BOdExpF/o5UkrjqPgSntORg0eZ/tRMPnrhU8qUK8O4D+6n2wWdHBcr///vbVzUqGHJGkNFMIokUjGSeqq6DcCjTOoWsl8FEUkDcoCnVHVmQTuJyAhgBEDTpk1DIa8RZiZMOLkFWrthTd759kJELmT31r3Mn5XG/E/S+HzKN8x88XPKlitD+/PPpHv/TvQY2Jnm7ZuatQKOKb+c4zl8+tpX/PvxDziw+xB9rjuP4U9cS/3m+R5dB+rz7bUHbmXiOyjRLJPoI2QWiYh8DdQv4KdHgLdVtYbPvvtUtWYBZTRU1a0i0hL4FuirquuKqtcskvgiOyub5T/8xi//W0zal4tJX74ZgKQGNemU0pbOKe3pmNKORqfVD1yxRMGYkIAp7MXu5zmpKj98vIApD7/LljXb6JTSjtufuYEzurcqWX1+4KsgVOHee2Hy5BO/nzJVtOEoUWmRqGq/wn4TkR0i0sDHtbWzkDK2ej7Xi0gq0AUoUpEY8UW5CuXo2q8jXft15A5uZFfGHhZ+uYRF3yxlyXcr+G7aPABqN0qiU+92dEpuR8fktjRsFYRiiQXyK4oSvuBzc3NJ+2IJ//nbR6z8cTXN2jbmiU8e4qyLuhZ83YKsL78rqyBMiUQvkXJtzQJuAp7yfP43/w4iUhM4qqrHRKQ2cC7wTFilNGIGb2u2TuNaDLylDwOG9wGUzau3siR1BUtSl7Pwy6V88+/vAahRpxqtu7fi9G4tOb17K07v3oraDZMKLtz7MkxJcdfj83IMpZul0GCzE+4qn3PyrqsqaxauI3X6PFLf/5Fdm/eQ1KAmY1+7gwHDe5NYJjHw+oqgIFdWfmsE3O4tUybRSaQUyVPA+yJyK7AJGAogIt2BO1X1NuBM4DURceEegf+Uqq6MkLxGFFN4YFaYMKERTds04tI7+6OqbFyZwYp5v7HqpzX8vnAdC79YjMvldu8mNahJq87NadG+KS06uJcmbRpRrnxZdz3pN7M/pwoTNfQB4CKDzf4UUICiKAhVSD+aSOqj00h9/0e2rt1OmbKJdOvfiTufv4meg7pTtlxZx+orCN9xLpMmnVAoAKNHwwsvnIiRgCmTaCQiikRV9wB9C9ieBtzm+f4j0CHMohkxRkkCsyJC83ZNaN6uSV531ayjx1i/JJ3f09azOm0t65ds5Nevl5Jz3J32PrFMIo1Pb0Dz9k35vUFjvv2pPpnD1vH8Kw14dHylkASAizynRjPQLXMQCOilnXM8h3WL01n2w2+smPcby1efxv6dB0j49WM692nPNX+6nHMHn0W1pKrOnIyfeJVJQUoknINkjcCwpI1GzONtrfu+hIIJzOYczyHj921sWLaJ9OWb2LB8ExuWbWJH+i58n5djWp2ajRpy/oD61G9el7pNa1N34l+pWz6X2nO/yLNkHDunRjOY2OpFZK5/Qf+Dew+R8fs2tqzZxubftrDqp9/57ee1ZB11p/Jv0LIe7c9rQ/tzz2T2L92o3bBGxLrWFnS+vorEu48pkdARc+NIQokpkvhEw5DpNjsrmy1rd3B2h61UYhuV2MqAXtvZ8vtW9u04cMr+SfVrUKdJLWrUrU612lWpUbsa1WpXo1qtqlSrVYVK1SpRoXJ5ylcsR7kKZd2fFctRvmI5EsskkpurVK3iAlwIyu5disvl4tigwRzOEY48N5nD+49weP8Rjuw/ysG9h9ievpMta7azZc02Du09nCdLQoLQqnNz2p3bhrS1Z/Kf2Wdw15ikUyy4SLiNfJWIV4ZIyxSPRGWvLcPwEuoRyoVlunX6BVS2fDkmvdGEXTTJ29a/C7yXCseTU9h1LIGdC5azk0rsbN6Wnccy2VWzGXu37WP90o0c2HWQ7KzjJaqzt4/8Q/KGbNRyf/SZcNK+IkLtxkk0at2AXkN60qh1Axq1rk+j1g1o0LJenoWkCtXHnhyPiOQLO5L53gxnMIvECCmhHqEcrtZssfX8muKup4hxGapK1tFjHNpziIN7DnPk4FGOHc3mWGY2x7Pcn95l1kwX835M4LzzhcsuF2bNSmDOnASSU4RhN5SnSs1KVKlRmSo1KlO5RiUqV3cviYn+9awKhwVXUiwlSmQxi8SISsIxQjlqsiTPSXWvFBEAFxEqVq5AxcoVqNu0TpH1rToCdXqcqGvI2BMKeOAtwZ1LuCy4kmJzjsQwgeafj9altMxHUloI53TBRa2HrR4H5x8JxTlFcv4ZI7rBpto9gbm2oo9odKPEM5YQ0SgIc20ZUUu0ulHimfwJMb1uOvs/jECxWWiMkJE/QO1yuT8nTQr9/OFG0Vg8wnASs0iMkGHdOg0jPrAYiRFyrFunYUQ/wcRIzLVlhBxzo8Qu+duZpazdaTiEKRLDMApkwoSTY1nemJf17DLyY4rEMHywFrgb38GkXmXi7Tixf3/8XhejYCzYbhgewj2+IppjR4XNEWJJFI2CMIvEMAh/CzwW3Ea+ysSLKRGjIEyRGAYnXprecS4JCaFLYx4rbqPCBpNGi3xGFBFobpVoXSzXlhEMLtfJecFCmbMrHDnIAsVycsUfBJFryywSw/AQzhZ4tLuNChtMOmaMDSY1TsWC7YZB0fONQGjcW9Geg8xychn+YhaJYRDeFngs5SCzwaSGP5hFYhgewtUCtxxkRmnDcm0ZRoSI5nEkRvxhubYMIwYxt5FRWjBFYhiGYQSFKRLDMAwjKEyRGIZhGEFhisQwDMMIClMkhmEYRlCYIjEMwzCCwhSJYRiGERSmSAzDMIygMEViGIZhBIUpEsMwDCMoTJEYhmEYQRERRSIiQ0VkhYi4RKTQJGEiMlBEVovIWhF5KJwyGoZhGP4RKYtkOXAFMLewHUQkEXgJuBBoC1wrIm3DI55hGIbhLxGZj0RVVwFI0elOzwLWqup6z77TgcuAlSEX0DAMw/CbaJ7YqhGw2Wc9Azi7oB1FZAQwwrN6TESWh1i2WKE2sDvSQkQJdi1OYNfiBHYtTnBGoAeGTJGIyNdA/QJ+ekRV/+tPEQVsK3AWLlV9HXjdU29aoJOzlDbsWpzArsUJ7FqcwK7FCUQk4BkBQ6ZIVLVfkEVkAE181hsDW4Ms0zAMw3CYaO7++wvQWkRaiEg54BpgVoRlMgzDMPIRqe6/g0UkA+gJfCYiX3i2NxSR2QCqmgOMBL4AVgHvq+oKP4p/PURixyJ2LU5g1+IEdi1OYNfiBAFfC1EtMOxgGIZhGH4Rza4twzAMIwYwRWIYhmEERcwrEku3cgIRSRKRr0RkjeezZiH75YrIYs9SqjowFPc/i0h5EXnP8/vPItI8/FKGBz+uxc0issvnXrgtEnKGGhF5U0R2Fja+TNxM9lynpSLSNdwyhgs/rkWKiBzwuSfG+VNuzCsSLN2KLw8B36hqa+Abz3pBZKpqZ88yKHzihRY//+dbgX2qehowEXg6vFKGhxLc8+/53AtvhFXI8DEVGFjE7xcCrT3LCOCVMMgUKaZS9LUA+N7nnnjcn0JjXpGo6ipVXV3MbnnpVlQ1G/CmWyltXAa87fn+NnB5BGWJBP78z77XaAbQV4rJ1ROjxMs9XyyqOhfYW8QulwHvqJufgBoi0iA80oUXP65FQMS8IvGTgtKtNIqQLKGknqpuA/B81i1kvwoikiYiP4lIaVI2/vzPeft4upgfAGqFRbrw4u89f6XHnTNDRJoU8Hs8EC/vB3/pKSJLRORzEWnnzwHRnGsrj3CmW4l2iroWJSimqapuFZGWwLciskxV1zkjYUTx538uNfdCMfhznp8A01T1mIjcidtS6xNyyaKPeLkn/GER0ExVD4vIRcBM3C6/IokJRWLpVk5Q1LUQkR0i0kBVt3lM852FlLHV87leRFKBLkBpUCT+/M/efTJEpAxQnRCY+lFAsddCVff4rP6TUhov8oNS834IFlU96PN9toi8LCK1VbXIxJbx4tqKl3Qrs4CbPN9vAk6x1kSkpoiU93yvDZxL6UnN78//7HuNhgDfaukclVvstcgXBxiEO4NEPDILuNHTe+sPwAGvizjeEJH63pihiJyFW0fsKfooQFVjegEG425RHAN2AF94tjcEZvvsdxHwO+6W9yORljtE16IW7t5aazyfSZ7t3YE3PN/PAZYBSzyft0ZaboevwSn/M/A4MMjzvQLwAbAWWAC0jLTMEbwWTwIrPPfCd0CbSMscouswDdgGHPe8K24F7gTu9PwuuHu4rfM8E90jLXMEr8VIn3viJ+Acf8q1FCmGYRhGUMSLa8swDMMIEaZIDMMwjKAwRWIYhmEEhSkSwzAMIyhMkRiGYRhBYYrEiBvyZT1eHM4s0MVlXTWMWMa6/8MzHsoAAAK3SURBVBpxg4gcVtUqEaq7F3AYd3LA9mGqM1FVc8NRlxHfmEVixDUiUt0zZ8cZnvVpInK75/srnuSWK0TkMZ9j0kXkbyIy3/N7VxH5QkTWeXJWnYL6kXVV3HPrLPckzJvr2ZYoIs+JyDJPcsVRnu19ReRXz/Y3fbIVpIvIOBH5ARgqIq1E5H8islBEvheRNk5cN8PwJSZybRmGQ1QUkcU+60+q6nsiMhKYKiKTgJqq+k/P74+o6l7P3B7fiEhHVV3q+W2zqvYUkYm453g4F/eo+RXAqwHKNw4YoKpbRKSGZ9sIoAXQRVVzxD15WQVPnX1V9XcReQe4C3jBc0yWqp4HICLf4B61vEZEzgZeJj4TMxohxBSJEU9kqmrn/BtV9SsRGYo7TUYnn5+uEpERuJ+TBrgniPIqEm/eqmVAFVU9BBwSkSwRqaGq+wOQbx5uhfY+8JFnWz/gVXWnvMej2DoBG1T1d88+bwP3cEKRvAcgIlVwp8T5wGfKlfIByGUYRWKKxIh7RCQBOBPIBJJwZwZuATwA9FDVfSIyFbfF4eWY59Pl8927HtBzpap3eqyGi4HFItIZdx4of1Lh+3LE85kA7C9IeRqGk1iMxDBgLO7Mt9cCb4pIWaAa7hfyARGph3s61pAiIq1U9WdVHQfsxp3a/EvgTk/Ke0QkCfgNaC4ip3kOvQGYk788dacE3+Cxtrxzk3fKv59hBIspEiOeqJiv++9TInI6cBtwv6p+D8wFHlXVJcCvuGMeb+J2OwWMiEwD5gNniEiGiNxawG7PeoLnyz1yLAHeADYBS0VkCXCdqmYBw3G7rJbhtoIKi8sMA271HLuCOJ1u1wgt1v3XMAzDCAqzSAzDMIygMEViGIZhBIUpEsMwDCMoTJEYhmEYQWGKxDAMwwgKUySGYRhGUJgiMQzDMILi/wGd4IN8DzQ2QQAAAABJRU5ErkJggg==\n", 565 | "text/plain": [ 566 | "
" 567 | ] 568 | }, 569 | "metadata": {}, 570 | "output_type": "display_data" 571 | } 572 | ], 573 | "source": [ 574 | "plt.scatter(X[pos[:,0],1],X[pos[:,0],2],c=\"r\",marker=\"+\",label=\"Admitted\")\n", 575 | "plt.scatter(X[neg[:,0],1],X[neg[:,0],2],c=\"b\",marker=\"x\",label=\"Not admitted\")\n", 576 | "\n", 577 | "# Plotting decision boundary\n", 578 | "\n", 579 | "u_vals = np.linspace(-1,1.5,50)\n", 580 | "v_vals= np.linspace(-1,1.5,50)\n", 581 | "z=np.zeros((len(u_vals),len(v_vals)))\n", 582 | "for i in range(len(u_vals)):\n", 583 | " for j in range(len(v_vals)):\n", 584 | " z[i,j] =mapFeaturePlot(u_vals[i],v_vals[j],6) @ theta \n", 585 | "\n", 586 | "plt.contour(u_vals,v_vals,z.T,0)\n", 587 | "plt.xlabel(\"Exam 1 score\")\n", 588 | "plt.ylabel(\"Exam 2 score\")\n", 589 | "plt.legend(loc=0)" 590 | ] 591 | }, 592 | { 593 | "cell_type": "markdown", 594 | "metadata": {}, 595 | "source": [ 596 | "### Accuracy of model" 597 | ] 598 | }, 599 | { 600 | "cell_type": "code", 601 | "execution_count": 18, 602 | "metadata": {}, 603 | "outputs": [], 604 | "source": [ 605 | "def classifierPredict(theta,X):\n", 606 | " \"\"\"\n", 607 | " take in numpy array of theta and X and predict the class \n", 608 | " \"\"\"\n", 609 | " predictions = X.dot(theta)\n", 610 | " \n", 611 | " return predictions>0" 612 | ] 613 | }, 614 | { 615 | "cell_type": "code", 616 | "execution_count": 19, 617 | "metadata": {}, 618 | "outputs": [ 619 | { 620 | "name": "stdout", 621 | "output_type": "stream", 622 | "text": [ 623 | "Train Accuracy: 83.05084745762711 %\n" 624 | ] 625 | } 626 | ], 627 | "source": [ 628 | "p=classifierPredict(theta,X)\n", 629 | "print(\"Train Accuracy:\", (sum(p==y[:,np.newaxis])/len(y) *100)[0],\"%\")" 630 | ] 631 | }, 632 | { 633 | "cell_type": "markdown", 634 | "metadata": {}, 635 | "source": [ 636 | "### Lasso Regression\n", 637 | "\n", 638 | "$J(\\Theta) = \\frac{1}{m} \\sum_{i=1}^{m} [ -y^{(i)}log(h_{\\Theta}(x^{(i)})) - (1 - y^{(i)})log(1 - (h_{\\Theta}(x^{(i)}))] + \\frac{\\lambda}{2m} \\sum_{j=1}^n |\\Theta_j|$" 639 | ] 640 | }, 641 | { 642 | "cell_type": "code", 643 | "execution_count": 20, 644 | "metadata": {}, 645 | "outputs": [ 646 | { 647 | "data": { 648 | "text/plain": [ 649 | "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n", 650 | " intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n", 651 | " penalty='l1', random_state=None, solver='liblinear', tol=0.0001,\n", 652 | " verbose=0, warm_start=False)" 653 | ] 654 | }, 655 | "execution_count": 20, 656 | "metadata": {}, 657 | "output_type": "execute_result" 658 | } 659 | ], 660 | "source": [ 661 | "from sklearn.linear_model import LogisticRegression\n", 662 | "\n", 663 | "clf = LogisticRegression(penalty=\"l1\")\n", 664 | "clf.fit(X,y)" 665 | ] 666 | }, 667 | { 668 | "cell_type": "code", 669 | "execution_count": 21, 670 | "metadata": {}, 671 | "outputs": [], 672 | "source": [ 673 | "thetaLasso=clf.coef_" 674 | ] 675 | }, 676 | { 677 | "cell_type": "code", 678 | "execution_count": 23, 679 | "metadata": {}, 680 | "outputs": [ 681 | { 682 | "name": "stdout", 683 | "output_type": "stream", 684 | "text": [ 685 | "The regularized theta using lasso regression:\n", 686 | " [[ 0.74485806]\n", 687 | " [ 0.68667195]\n", 688 | " [ 1.280452 ]\n", 689 | " [-4.86272543]\n", 690 | " [-1.62185281]\n", 691 | " [-2.34325986]\n", 692 | " [ 0. ]\n", 693 | " [ 0. ]\n", 694 | " [ 0. ]\n", 695 | " [ 0. ]\n", 696 | " [ 0. ]\n", 697 | " [ 0. ]\n", 698 | " [ 0. ]\n", 699 | " [ 0. ]\n", 700 | " [-2.36651879]\n", 701 | " [ 0. ]\n", 702 | " [ 0. ]\n", 703 | " [ 0. ]\n", 704 | " [ 0. ]\n", 705 | " [ 0. ]\n", 706 | " [ 0. ]\n", 707 | " [ 0. ]\n", 708 | " [ 0. ]\n", 709 | " [ 0. ]\n", 710 | " [ 0. ]\n", 711 | " [ 0. ]\n", 712 | " [ 0. ]\n", 713 | " [ 0. ]]\n" 714 | ] 715 | } 716 | ], 717 | "source": [ 718 | "print(\"The regularized theta using lasso regression:\\n\",thetaLasso.reshape(28,1))" 719 | ] 720 | }, 721 | { 722 | "cell_type": "code", 723 | "execution_count": null, 724 | "metadata": {}, 725 | "outputs": [], 726 | "source": [] 727 | } 728 | ], 729 | "metadata": { 730 | "kernelspec": { 731 | "display_name": "Python 3", 732 | "language": "python", 733 | "name": "python3" 734 | }, 735 | "language_info": { 736 | "codemirror_mode": { 737 | "name": "ipython", 738 | "version": 3 739 | }, 740 | "file_extension": ".py", 741 | "mimetype": "text/x-python", 742 | "name": "python", 743 | "nbconvert_exporter": "python", 744 | "pygments_lexer": "ipython3", 745 | "version": "3.6.5" 746 | } 747 | }, 748 | "nbformat": 4, 749 | "nbformat_minor": 2 750 | } 751 | -------------------------------------------------------------------------------- /LogisticRegression/ex2data1.txt: -------------------------------------------------------------------------------- 1 | 34.62365962451697,78.0246928153624,0 2 | 30.28671076822607,43.89499752400101,0 3 | 35.84740876993872,72.90219802708364,0 4 | 60.18259938620976,86.30855209546826,1 5 | 79.0327360507101,75.3443764369103,1 6 | 45.08327747668339,56.3163717815305,0 7 | 61.10666453684766,96.51142588489624,1 8 | 75.02474556738889,46.55401354116538,1 9 | 76.09878670226257,87.42056971926803,1 10 | 84.43281996120035,43.53339331072109,1 11 | 95.86155507093572,38.22527805795094,0 12 | 75.01365838958247,30.60326323428011,0 13 | 82.30705337399482,76.48196330235604,1 14 | 69.36458875970939,97.71869196188608,1 15 | 39.53833914367223,76.03681085115882,0 16 | 53.9710521485623,89.20735013750205,1 17 | 69.07014406283025,52.74046973016765,1 18 | 67.94685547711617,46.67857410673128,0 19 | 70.66150955499435,92.92713789364831,1 20 | 76.97878372747498,47.57596364975532,1 21 | 67.37202754570876,42.83843832029179,0 22 | 89.67677575072079,65.79936592745237,1 23 | 50.534788289883,48.85581152764205,0 24 | 34.21206097786789,44.20952859866288,0 25 | 77.9240914545704,68.9723599933059,1 26 | 62.27101367004632,69.95445795447587,1 27 | 80.1901807509566,44.82162893218353,1 28 | 93.114388797442,38.80067033713209,0 29 | 61.83020602312595,50.25610789244621,0 30 | 38.78580379679423,64.99568095539578,0 31 | 61.379289447425,72.80788731317097,1 32 | 85.40451939411645,57.05198397627122,1 33 | 52.10797973193984,63.12762376881715,0 34 | 52.04540476831827,69.43286012045222,1 35 | 40.23689373545111,71.16774802184875,0 36 | 54.63510555424817,52.21388588061123,0 37 | 33.91550010906887,98.86943574220611,0 38 | 64.17698887494485,80.90806058670817,1 39 | 74.78925295941542,41.57341522824434,0 40 | 34.1836400264419,75.2377203360134,0 41 | 83.90239366249155,56.30804621605327,1 42 | 51.54772026906181,46.85629026349976,0 43 | 94.44336776917852,65.56892160559052,1 44 | 82.36875375713919,40.61825515970618,0 45 | 51.04775177128865,45.82270145776001,0 46 | 62.22267576120188,52.06099194836679,0 47 | 77.19303492601364,70.45820000180959,1 48 | 97.77159928000232,86.7278223300282,1 49 | 62.07306379667647,96.76882412413983,1 50 | 91.56497449807442,88.69629254546599,1 51 | 79.94481794066932,74.16311935043758,1 52 | 99.2725269292572,60.99903099844988,1 53 | 90.54671411399852,43.39060180650027,1 54 | 34.52451385320009,60.39634245837173,0 55 | 50.2864961189907,49.80453881323059,0 56 | 49.58667721632031,59.80895099453265,0 57 | 97.64563396007767,68.86157272420604,1 58 | 32.57720016809309,95.59854761387875,0 59 | 74.24869136721598,69.82457122657193,1 60 | 71.79646205863379,78.45356224515052,1 61 | 75.3956114656803,85.75993667331619,1 62 | 35.28611281526193,47.02051394723416,0 63 | 56.25381749711624,39.26147251058019,0 64 | 30.05882244669796,49.59297386723685,0 65 | 44.66826172480893,66.45008614558913,0 66 | 66.56089447242954,41.09209807936973,0 67 | 40.45755098375164,97.53518548909936,1 68 | 49.07256321908844,51.88321182073966,0 69 | 80.27957401466998,92.11606081344084,1 70 | 66.74671856944039,60.99139402740988,1 71 | 32.72283304060323,43.30717306430063,0 72 | 64.0393204150601,78.03168802018232,1 73 | 72.34649422579923,96.22759296761404,1 74 | 60.45788573918959,73.09499809758037,1 75 | 58.84095621726802,75.85844831279042,1 76 | 99.82785779692128,72.36925193383885,1 77 | 47.26426910848174,88.47586499559782,1 78 | 50.45815980285988,75.80985952982456,1 79 | 60.45555629271532,42.50840943572217,0 80 | 82.22666157785568,42.71987853716458,0 81 | 88.9138964166533,69.80378889835472,1 82 | 94.83450672430196,45.69430680250754,1 83 | 67.31925746917527,66.58935317747915,1 84 | 57.23870631569862,59.51428198012956,1 85 | 80.36675600171273,90.96014789746954,1 86 | 68.46852178591112,85.59430710452014,1 87 | 42.0754545384731,78.84478600148043,0 88 | 75.47770200533905,90.42453899753964,1 89 | 78.63542434898018,96.64742716885644,1 90 | 52.34800398794107,60.76950525602592,0 91 | 94.09433112516793,77.15910509073893,1 92 | 90.44855097096364,87.50879176484702,1 93 | 55.48216114069585,35.57070347228866,0 94 | 74.49269241843041,84.84513684930135,1 95 | 89.84580670720979,45.35828361091658,1 96 | 83.48916274498238,48.38028579728175,1 97 | 42.2617008099817,87.10385094025457,1 98 | 99.31500880510394,68.77540947206617,1 99 | 55.34001756003703,64.9319380069486,1 100 | 74.77589300092767,89.52981289513276,1 101 | -------------------------------------------------------------------------------- /LogisticRegression/ex2data2.txt: -------------------------------------------------------------------------------- 1 | 0.051267,0.69956,1 2 | -0.092742,0.68494,1 3 | -0.21371,0.69225,1 4 | -0.375,0.50219,1 5 | -0.51325,0.46564,1 6 | -0.52477,0.2098,1 7 | -0.39804,0.034357,1 8 | -0.30588,-0.19225,1 9 | 0.016705,-0.40424,1 10 | 0.13191,-0.51389,1 11 | 0.38537,-0.56506,1 12 | 0.52938,-0.5212,1 13 | 0.63882,-0.24342,1 14 | 0.73675,-0.18494,1 15 | 0.54666,0.48757,1 16 | 0.322,0.5826,1 17 | 0.16647,0.53874,1 18 | -0.046659,0.81652,1 19 | -0.17339,0.69956,1 20 | -0.47869,0.63377,1 21 | -0.60541,0.59722,1 22 | -0.62846,0.33406,1 23 | -0.59389,0.005117,1 24 | -0.42108,-0.27266,1 25 | -0.11578,-0.39693,1 26 | 0.20104,-0.60161,1 27 | 0.46601,-0.53582,1 28 | 0.67339,-0.53582,1 29 | -0.13882,0.54605,1 30 | -0.29435,0.77997,1 31 | -0.26555,0.96272,1 32 | -0.16187,0.8019,1 33 | -0.17339,0.64839,1 34 | -0.28283,0.47295,1 35 | -0.36348,0.31213,1 36 | -0.30012,0.027047,1 37 | -0.23675,-0.21418,1 38 | -0.06394,-0.18494,1 39 | 0.062788,-0.16301,1 40 | 0.22984,-0.41155,1 41 | 0.2932,-0.2288,1 42 | 0.48329,-0.18494,1 43 | 0.64459,-0.14108,1 44 | 0.46025,0.012427,1 45 | 0.6273,0.15863,1 46 | 0.57546,0.26827,1 47 | 0.72523,0.44371,1 48 | 0.22408,0.52412,1 49 | 0.44297,0.67032,1 50 | 0.322,0.69225,1 51 | 0.13767,0.57529,1 52 | -0.0063364,0.39985,1 53 | -0.092742,0.55336,1 54 | -0.20795,0.35599,1 55 | -0.20795,0.17325,1 56 | -0.43836,0.21711,1 57 | -0.21947,-0.016813,1 58 | -0.13882,-0.27266,1 59 | 0.18376,0.93348,0 60 | 0.22408,0.77997,0 61 | 0.29896,0.61915,0 62 | 0.50634,0.75804,0 63 | 0.61578,0.7288,0 64 | 0.60426,0.59722,0 65 | 0.76555,0.50219,0 66 | 0.92684,0.3633,0 67 | 0.82316,0.27558,0 68 | 0.96141,0.085526,0 69 | 0.93836,0.012427,0 70 | 0.86348,-0.082602,0 71 | 0.89804,-0.20687,0 72 | 0.85196,-0.36769,0 73 | 0.82892,-0.5212,0 74 | 0.79435,-0.55775,0 75 | 0.59274,-0.7405,0 76 | 0.51786,-0.5943,0 77 | 0.46601,-0.41886,0 78 | 0.35081,-0.57968,0 79 | 0.28744,-0.76974,0 80 | 0.085829,-0.75512,0 81 | 0.14919,-0.57968,0 82 | -0.13306,-0.4481,0 83 | -0.40956,-0.41155,0 84 | -0.39228,-0.25804,0 85 | -0.74366,-0.25804,0 86 | -0.69758,0.041667,0 87 | -0.75518,0.2902,0 88 | -0.69758,0.68494,0 89 | -0.4038,0.70687,0 90 | -0.38076,0.91886,0 91 | -0.50749,0.90424,0 92 | -0.54781,0.70687,0 93 | 0.10311,0.77997,0 94 | 0.057028,0.91886,0 95 | -0.10426,0.99196,0 96 | -0.081221,1.1089,0 97 | 0.28744,1.087,0 98 | 0.39689,0.82383,0 99 | 0.63882,0.88962,0 100 | 0.82316,0.66301,0 101 | 0.67339,0.64108,0 102 | 1.0709,0.10015,0 103 | -0.046659,-0.57968,0 104 | -0.23675,-0.63816,0 105 | -0.15035,-0.36769,0 106 | -0.49021,-0.3019,0 107 | -0.46717,-0.13377,0 108 | -0.28859,-0.060673,0 109 | -0.61118,-0.067982,0 110 | -0.66302,-0.21418,0 111 | -0.59965,-0.41886,0 112 | -0.72638,-0.082602,0 113 | -0.83007,0.31213,0 114 | -0.72062,0.53874,0 115 | -0.59389,0.49488,0 116 | -0.48445,0.99927,0 117 | -0.0063364,0.99927,0 118 | 0.63265,-0.030612,0 119 | -------------------------------------------------------------------------------- /NeuralNetworks/ex3data1.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/NeuralNetworks/ex3data1.mat -------------------------------------------------------------------------------- /NeuralNetworks/ex3weights.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/NeuralNetworks/ex3weights.mat -------------------------------------------------------------------------------- /NeuralNetworks/ex4data1.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/NeuralNetworks/ex4data1.mat -------------------------------------------------------------------------------- /NeuralNetworks/ex4weights.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/NeuralNetworks/ex4weights.mat -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Machine-Learning-by-Andrew-Ng-in-Python 2 | ### Documenting my python implementation of Andrew Ng's Machine Learning Course 3 | 4 | [Linear Regression](https://github.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/tree/master/LinearRegression)
5 | [Logistic Regression](https://github.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/tree/master/LogisticRegression)
6 | [Neural Networks](https://github.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/tree/master/NeuralNetworks)
7 | [Bias Vs Variance](https://github.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/tree/master/Bias_Vs_Variance)
8 | [Support Vector Machines](https://github.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/tree/master/SupportVectorMachines)
9 | [Unsupervised Learning](https://github.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/tree/master/KmeansClustering_PCA)
10 | [Anomaly Detection](https://github.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/tree/master/Anomaly%20Detection)
11 | -------------------------------------------------------------------------------- /SupportVectorMachines/emailSample1.txt: -------------------------------------------------------------------------------- 1 | > Anyone knows how much it costs to host a web portal ? 2 | > 3 | Well, it depends on how many visitors you're expecting. 4 | This can be anywhere from less than 10 bucks a month to a couple of $100. 5 | You should checkout http://www.rackspace.com/ or perhaps Amazon EC2 6 | if youre running something big.. 7 | 8 | To unsubscribe yourself from this mailing list, send an email to: 9 | groupname-unsubscribe@egroups.com 10 | 11 | -------------------------------------------------------------------------------- /SupportVectorMachines/emailSample2.txt: -------------------------------------------------------------------------------- 1 | Folks, 2 | 3 | my first time posting - have a bit of Unix experience, but am new to Linux. 4 | 5 | 6 | Just got a new PC at home - Dell box with Windows XP. Added a second hard disk 7 | for Linux. Partitioned the disk and have installed Suse 7.2 from CD, which went 8 | fine except it didn't pick up my monitor. 9 | 10 | I have a Dell branded E151FPp 15" LCD flat panel monitor and a nVidia GeForce4 11 | Ti4200 video card, both of which are probably too new to feature in Suse's default 12 | set. I downloaded a driver from the nVidia website and installed it using RPM. 13 | Then I ran Sax2 (as was recommended in some postings I found on the net), but 14 | it still doesn't feature my video card in the available list. What next? 15 | 16 | Another problem. I have a Dell branded keyboard and if I hit Caps-Lock twice, 17 | the whole machine crashes (in Linux, not Windows) - even the on/off switch is 18 | inactive, leaving me to reach for the power cable instead. 19 | 20 | If anyone can help me in any way with these probs., I'd be really grateful - 21 | I've searched the 'net but have run out of ideas. 22 | 23 | Or should I be going for a different version of Linux such as RedHat? Opinions 24 | welcome. 25 | 26 | Thanks a lot, 27 | Peter 28 | 29 | -- 30 | Irish Linux Users' Group: ilug@linux.ie 31 | http://www.linux.ie/mailman/listinfo/ilug for (un)subscription information. 32 | List maintainer: listmaster@linux.ie 33 | 34 | 35 | -------------------------------------------------------------------------------- /SupportVectorMachines/ex6data1.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/SupportVectorMachines/ex6data1.mat -------------------------------------------------------------------------------- /SupportVectorMachines/ex6data2.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/SupportVectorMachines/ex6data2.mat -------------------------------------------------------------------------------- /SupportVectorMachines/ex6data3.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/SupportVectorMachines/ex6data3.mat -------------------------------------------------------------------------------- /SupportVectorMachines/spamSample1.txt: -------------------------------------------------------------------------------- 1 | Do You Want To Make $1000 Or More Per Week? 2 | 3 | 4 | 5 | If you are a motivated and qualified individual - I 6 | will personally demonstrate to you a system that will 7 | make you $1,000 per week or more! This is NOT mlm. 8 | 9 | 10 | 11 | Call our 24 hour pre-recorded number to get the 12 | details. 13 | 14 | 15 | 16 | 000-456-789 17 | 18 | 19 | 20 | I need people who want to make serious money. Make 21 | the call and get the facts. 22 | 23 | Invest 2 minutes in yourself now! 24 | 25 | 26 | 27 | 000-456-789 28 | 29 | 30 | 31 | Looking forward to your call and I will introduce you 32 | to people like yourself who 33 | are currently making $10,000 plus per week! 34 | 35 | 36 | 37 | 000-456-789 38 | 39 | 40 | 41 | 3484lJGv6-241lEaN9080lRmS6-271WxHo7524qiyT5-438rjUv5615hQcf0-662eiDB9057dMtVl72 42 | 43 | -------------------------------------------------------------------------------- /SupportVectorMachines/spamSample2.txt: -------------------------------------------------------------------------------- 1 | Best Buy Viagra Generic Online 2 | 3 | Viagra 100mg x 60 Pills $125, Free Pills & Reorder Discount, Top Selling 100% Quality & Satisfaction guaranteed! 4 | 5 | We accept VISA, Master & E-Check Payments, 90000+ Satisfied Customers! 6 | http://medphysitcstech.ru 7 | 8 | 9 | -------------------------------------------------------------------------------- /SupportVectorMachines/spamTest.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/SupportVectorMachines/spamTest.mat -------------------------------------------------------------------------------- /SupportVectorMachines/spamTrain.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Benlau93/Machine-Learning-by-Andrew-Ng-in-Python/a066fa6792d5ced753f25ceef064d64cf0563257/SupportVectorMachines/spamTrain.mat -------------------------------------------------------------------------------- /SupportVectorMachines/vocab.txt: -------------------------------------------------------------------------------- 1 | 1 aa 2 | 2 ab 3 | 3 abil 4 | 4 abl 5 | 5 about 6 | 6 abov 7 | 7 absolut 8 | 8 abus 9 | 9 ac 10 | 10 accept 11 | 11 access 12 | 12 accord 13 | 13 account 14 | 14 achiev 15 | 15 acquir 16 | 16 across 17 | 17 act 18 | 18 action 19 | 19 activ 20 | 20 actual 21 | 21 ad 22 | 22 adam 23 | 23 add 24 | 24 addit 25 | 25 address 26 | 26 administr 27 | 27 adult 28 | 28 advanc 29 | 29 advantag 30 | 30 advertis 31 | 31 advic 32 | 32 advis 33 | 33 ae 34 | 34 af 35 | 35 affect 36 | 36 affili 37 | 37 afford 38 | 38 africa 39 | 39 after 40 | 40 ag 41 | 41 again 42 | 42 against 43 | 43 agenc 44 | 44 agent 45 | 45 ago 46 | 46 agre 47 | 47 agreement 48 | 48 aid 49 | 49 air 50 | 50 al 51 | 51 alb 52 | 52 align 53 | 53 all 54 | 54 allow 55 | 55 almost 56 | 56 alon 57 | 57 along 58 | 58 alreadi 59 | 59 alsa 60 | 60 also 61 | 61 altern 62 | 62 although 63 | 63 alwai 64 | 64 am 65 | 65 amaz 66 | 66 america 67 | 67 american 68 | 68 among 69 | 69 amount 70 | 70 amp 71 | 71 an 72 | 72 analysi 73 | 73 analyst 74 | 74 and 75 | 75 ani 76 | 76 anim 77 | 77 announc 78 | 78 annual 79 | 79 annuiti 80 | 80 anoth 81 | 81 answer 82 | 82 anti 83 | 83 anumb 84 | 84 anybodi 85 | 85 anymor 86 | 86 anyon 87 | 87 anyth 88 | 88 anywai 89 | 89 anywher 90 | 90 aol 91 | 91 ap 92 | 92 apolog 93 | 93 app 94 | 94 appar 95 | 95 appear 96 | 96 appl 97 | 97 appli 98 | 98 applic 99 | 99 appreci 100 | 100 approach 101 | 101 approv 102 | 102 apt 103 | 103 ar 104 | 104 archiv 105 | 105 area 106 | 106 aren 107 | 107 argument 108 | 108 arial 109 | 109 arm 110 | 110 around 111 | 111 arrai 112 | 112 arriv 113 | 113 art 114 | 114 articl 115 | 115 artist 116 | 116 as 117 | 117 ascii 118 | 118 ask 119 | 119 asset 120 | 120 assist 121 | 121 associ 122 | 122 assum 123 | 123 assur 124 | 124 at 125 | 125 atol 126 | 126 attach 127 | 127 attack 128 | 128 attempt 129 | 129 attent 130 | 130 attornei 131 | 131 attract 132 | 132 audio 133 | 133 aug 134 | 134 august 135 | 135 author 136 | 136 auto 137 | 137 autom 138 | 138 automat 139 | 139 avail 140 | 140 averag 141 | 141 avoid 142 | 142 awai 143 | 143 awar 144 | 144 award 145 | 145 ba 146 | 146 babi 147 | 147 back 148 | 148 background 149 | 149 backup 150 | 150 bad 151 | 151 balanc 152 | 152 ban 153 | 153 bank 154 | 154 bar 155 | 155 base 156 | 156 basenumb 157 | 157 basi 158 | 158 basic 159 | 159 bb 160 | 160 bc 161 | 161 bd 162 | 162 be 163 | 163 beat 164 | 164 beberg 165 | 165 becaus 166 | 166 becom 167 | 167 been 168 | 168 befor 169 | 169 begin 170 | 170 behalf 171 | 171 behavior 172 | 172 behind 173 | 173 believ 174 | 174 below 175 | 175 benefit 176 | 176 best 177 | 177 beta 178 | 178 better 179 | 179 between 180 | 180 bf 181 | 181 big 182 | 182 bill 183 | 183 billion 184 | 184 bin 185 | 185 binari 186 | 186 bit 187 | 187 black 188 | 188 blank 189 | 189 block 190 | 190 blog 191 | 191 blood 192 | 192 blue 193 | 193 bnumber 194 | 194 board 195 | 195 bodi 196 | 196 boi 197 | 197 bonu 198 | 198 book 199 | 199 boot 200 | 200 border 201 | 201 boss 202 | 202 boston 203 | 203 botan 204 | 204 both 205 | 205 bottl 206 | 206 bottom 207 | 207 boundari 208 | 208 box 209 | 209 brain 210 | 210 brand 211 | 211 break 212 | 212 brian 213 | 213 bring 214 | 214 broadcast 215 | 215 broker 216 | 216 browser 217 | 217 bug 218 | 218 bui 219 | 219 build 220 | 220 built 221 | 221 bulk 222 | 222 burn 223 | 223 bush 224 | 224 busi 225 | 225 but 226 | 226 button 227 | 227 by 228 | 228 byte 229 | 229 ca 230 | 230 cabl 231 | 231 cach 232 | 232 calcul 233 | 233 california 234 | 234 call 235 | 235 came 236 | 236 camera 237 | 237 campaign 238 | 238 can 239 | 239 canada 240 | 240 cannot 241 | 241 canon 242 | 242 capabl 243 | 243 capillari 244 | 244 capit 245 | 245 car 246 | 246 card 247 | 247 care 248 | 248 career 249 | 249 carri 250 | 250 cartridg 251 | 251 case 252 | 252 cash 253 | 253 cat 254 | 254 catch 255 | 255 categori 256 | 256 caus 257 | 257 cb 258 | 258 cc 259 | 259 cd 260 | 260 ce 261 | 261 cell 262 | 262 cent 263 | 263 center 264 | 264 central 265 | 265 centuri 266 | 266 ceo 267 | 267 certain 268 | 268 certainli 269 | 269 cf 270 | 270 challeng 271 | 271 chanc 272 | 272 chang 273 | 273 channel 274 | 274 char 275 | 275 charact 276 | 276 charg 277 | 277 charset 278 | 278 chat 279 | 279 cheap 280 | 280 check 281 | 281 cheer 282 | 282 chief 283 | 283 children 284 | 284 china 285 | 285 chip 286 | 286 choic 287 | 287 choos 288 | 288 chri 289 | 289 citi 290 | 290 citizen 291 | 291 civil 292 | 292 claim 293 | 293 class 294 | 294 classifi 295 | 295 clean 296 | 296 clear 297 | 297 clearli 298 | 298 click 299 | 299 client 300 | 300 close 301 | 301 clue 302 | 302 cnet 303 | 303 cnumber 304 | 304 co 305 | 305 code 306 | 306 collect 307 | 307 colleg 308 | 308 color 309 | 309 com 310 | 310 combin 311 | 311 come 312 | 312 comfort 313 | 313 command 314 | 314 comment 315 | 315 commentari 316 | 316 commerci 317 | 317 commiss 318 | 318 commit 319 | 319 common 320 | 320 commun 321 | 321 compani 322 | 322 compar 323 | 323 comparison 324 | 324 compat 325 | 325 compet 326 | 326 competit 327 | 327 compil 328 | 328 complet 329 | 329 comprehens 330 | 330 comput 331 | 331 concentr 332 | 332 concept 333 | 333 concern 334 | 334 condit 335 | 335 conf 336 | 336 confer 337 | 337 confid 338 | 338 confidenti 339 | 339 config 340 | 340 configur 341 | 341 confirm 342 | 342 conflict 343 | 343 confus 344 | 344 congress 345 | 345 connect 346 | 346 consid 347 | 347 consolid 348 | 348 constitut 349 | 349 construct 350 | 350 consult 351 | 351 consum 352 | 352 contact 353 | 353 contain 354 | 354 content 355 | 355 continu 356 | 356 contract 357 | 357 contribut 358 | 358 control 359 | 359 conveni 360 | 360 convers 361 | 361 convert 362 | 362 cool 363 | 363 cooper 364 | 364 copi 365 | 365 copyright 366 | 366 core 367 | 367 corpor 368 | 368 correct 369 | 369 correspond 370 | 370 cost 371 | 371 could 372 | 372 couldn 373 | 373 count 374 | 374 countri 375 | 375 coupl 376 | 376 cours 377 | 377 court 378 | 378 cover 379 | 379 coverag 380 | 380 crash 381 | 381 creat 382 | 382 creativ 383 | 383 credit 384 | 384 critic 385 | 385 cross 386 | 386 cultur 387 | 387 current 388 | 388 custom 389 | 389 cut 390 | 390 cv 391 | 391 da 392 | 392 dagga 393 | 393 dai 394 | 394 daili 395 | 395 dan 396 | 396 danger 397 | 397 dark 398 | 398 data 399 | 399 databas 400 | 400 datapow 401 | 401 date 402 | 402 dave 403 | 403 david 404 | 404 dc 405 | 405 de 406 | 406 dead 407 | 407 deal 408 | 408 dear 409 | 409 death 410 | 410 debt 411 | 411 decad 412 | 412 decid 413 | 413 decis 414 | 414 declar 415 | 415 declin 416 | 416 decor 417 | 417 default 418 | 418 defend 419 | 419 defens 420 | 420 defin 421 | 421 definit 422 | 422 degre 423 | 423 delai 424 | 424 delet 425 | 425 deliv 426 | 426 deliveri 427 | 427 dell 428 | 428 demand 429 | 429 democrat 430 | 430 depart 431 | 431 depend 432 | 432 deposit 433 | 433 describ 434 | 434 descript 435 | 435 deserv 436 | 436 design 437 | 437 desir 438 | 438 desktop 439 | 439 despit 440 | 440 detail 441 | 441 detect 442 | 442 determin 443 | 443 dev 444 | 444 devel 445 | 445 develop 446 | 446 devic 447 | 447 di 448 | 448 dial 449 | 449 did 450 | 450 didn 451 | 451 diet 452 | 452 differ 453 | 453 difficult 454 | 454 digit 455 | 455 direct 456 | 456 directli 457 | 457 director 458 | 458 directori 459 | 459 disabl 460 | 460 discount 461 | 461 discov 462 | 462 discoveri 463 | 463 discuss 464 | 464 disk 465 | 465 displai 466 | 466 disposit 467 | 467 distanc 468 | 468 distribut 469 | 469 dn 470 | 470 dnumber 471 | 471 do 472 | 472 doc 473 | 473 document 474 | 474 doe 475 | 475 doer 476 | 476 doesn 477 | 477 dollar 478 | 478 dollarac 479 | 479 dollarnumb 480 | 480 domain 481 | 481 don 482 | 482 done 483 | 483 dont 484 | 484 doubl 485 | 485 doubt 486 | 486 down 487 | 487 download 488 | 488 dr 489 | 489 draw 490 | 490 dream 491 | 491 drive 492 | 492 driver 493 | 493 drop 494 | 494 drug 495 | 495 due 496 | 496 dure 497 | 497 dvd 498 | 498 dw 499 | 499 dynam 500 | 500 ea 501 | 501 each 502 | 502 earli 503 | 503 earlier 504 | 504 earn 505 | 505 earth 506 | 506 easi 507 | 507 easier 508 | 508 easili 509 | 509 eat 510 | 510 eb 511 | 511 ebai 512 | 512 ec 513 | 513 echo 514 | 514 econom 515 | 515 economi 516 | 516 ed 517 | 517 edg 518 | 518 edit 519 | 519 editor 520 | 520 educ 521 | 521 eff 522 | 522 effect 523 | 523 effici 524 | 524 effort 525 | 525 either 526 | 526 el 527 | 527 electron 528 | 528 elimin 529 | 529 els 530 | 530 email 531 | 531 emailaddr 532 | 532 emerg 533 | 533 empir 534 | 534 employ 535 | 535 employe 536 | 536 en 537 | 537 enabl 538 | 538 encod 539 | 539 encourag 540 | 540 end 541 | 541 enemi 542 | 542 enenkio 543 | 543 energi 544 | 544 engin 545 | 545 english 546 | 546 enhanc 547 | 547 enjoi 548 | 548 enough 549 | 549 ensur 550 | 550 enter 551 | 551 enterpris 552 | 552 entertain 553 | 553 entir 554 | 554 entri 555 | 555 enumb 556 | 556 environ 557 | 557 equal 558 | 558 equip 559 | 559 equival 560 | 560 error 561 | 561 especi 562 | 562 essenti 563 | 563 establish 564 | 564 estat 565 | 565 estim 566 | 566 et 567 | 567 etc 568 | 568 euro 569 | 569 europ 570 | 570 european 571 | 571 even 572 | 572 event 573 | 573 eventu 574 | 574 ever 575 | 575 everi 576 | 576 everyon 577 | 577 everyth 578 | 578 evid 579 | 579 evil 580 | 580 exactli 581 | 581 exampl 582 | 582 excel 583 | 583 except 584 | 584 exchang 585 | 585 excit 586 | 586 exclus 587 | 587 execut 588 | 588 exercis 589 | 589 exist 590 | 590 exmh 591 | 591 expand 592 | 592 expect 593 | 593 expens 594 | 594 experi 595 | 595 expert 596 | 596 expir 597 | 597 explain 598 | 598 explor 599 | 599 express 600 | 600 extend 601 | 601 extens 602 | 602 extra 603 | 603 extract 604 | 604 extrem 605 | 605 ey 606 | 606 fa 607 | 607 face 608 | 608 fact 609 | 609 factor 610 | 610 fail 611 | 611 fair 612 | 612 fall 613 | 613 fals 614 | 614 famili 615 | 615 faq 616 | 616 far 617 | 617 fast 618 | 618 faster 619 | 619 fastest 620 | 620 fat 621 | 621 father 622 | 622 favorit 623 | 623 fax 624 | 624 fb 625 | 625 fd 626 | 626 featur 627 | 627 feder 628 | 628 fee 629 | 629 feed 630 | 630 feedback 631 | 631 feel 632 | 632 femal 633 | 633 few 634 | 634 ffffff 635 | 635 ffnumber 636 | 636 field 637 | 637 fight 638 | 638 figur 639 | 639 file 640 | 640 fill 641 | 641 film 642 | 642 filter 643 | 643 final 644 | 644 financ 645 | 645 financi 646 | 646 find 647 | 647 fine 648 | 648 finish 649 | 649 fire 650 | 650 firewal 651 | 651 firm 652 | 652 first 653 | 653 fit 654 | 654 five 655 | 655 fix 656 | 656 flag 657 | 657 flash 658 | 658 flow 659 | 659 fnumber 660 | 660 focu 661 | 661 folder 662 | 662 folk 663 | 663 follow 664 | 664 font 665 | 665 food 666 | 666 for 667 | 667 forc 668 | 668 foreign 669 | 669 forev 670 | 670 forget 671 | 671 fork 672 | 672 form 673 | 673 format 674 | 674 former 675 | 675 fortun 676 | 676 forward 677 | 677 found 678 | 678 foundat 679 | 679 four 680 | 680 franc 681 | 681 free 682 | 682 freedom 683 | 683 french 684 | 684 freshrpm 685 | 685 fri 686 | 686 fridai 687 | 687 friend 688 | 688 from 689 | 689 front 690 | 690 ftoc 691 | 691 ftp 692 | 692 full 693 | 693 fulli 694 | 694 fun 695 | 695 function 696 | 696 fund 697 | 697 further 698 | 698 futur 699 | 699 ga 700 | 700 gain 701 | 701 game 702 | 702 gari 703 | 703 garrigu 704 | 704 gave 705 | 705 gcc 706 | 706 geek 707 | 707 gener 708 | 708 get 709 | 709 gif 710 | 710 gift 711 | 711 girl 712 | 712 give 713 | 713 given 714 | 714 global 715 | 715 gnome 716 | 716 gnu 717 | 717 gnupg 718 | 718 go 719 | 719 goal 720 | 720 god 721 | 721 goe 722 | 722 gold 723 | 723 gone 724 | 724 good 725 | 725 googl 726 | 726 got 727 | 727 govern 728 | 728 gpl 729 | 729 grand 730 | 730 grant 731 | 731 graphic 732 | 732 great 733 | 733 greater 734 | 734 ground 735 | 735 group 736 | 736 grow 737 | 737 growth 738 | 738 gt 739 | 739 guarante 740 | 740 guess 741 | 741 gui 742 | 742 guid 743 | 743 ha 744 | 744 hack 745 | 745 had 746 | 746 half 747 | 747 ham 748 | 748 hand 749 | 749 handl 750 | 750 happen 751 | 751 happi 752 | 752 hard 753 | 753 hardwar 754 | 754 hat 755 | 755 hate 756 | 756 have 757 | 757 haven 758 | 758 he 759 | 759 head 760 | 760 header 761 | 761 headlin 762 | 762 health 763 | 763 hear 764 | 764 heard 765 | 765 heart 766 | 766 heaven 767 | 767 hei 768 | 768 height 769 | 769 held 770 | 770 hello 771 | 771 help 772 | 772 helvetica 773 | 773 her 774 | 774 herba 775 | 775 here 776 | 776 hermio 777 | 777 hettinga 778 | 778 hi 779 | 779 high 780 | 780 higher 781 | 781 highli 782 | 782 highlight 783 | 783 him 784 | 784 histori 785 | 785 hit 786 | 786 hold 787 | 787 home 788 | 788 honor 789 | 789 hope 790 | 790 host 791 | 791 hot 792 | 792 hour 793 | 793 hous 794 | 794 how 795 | 795 howev 796 | 796 hp 797 | 797 html 798 | 798 http 799 | 799 httpaddr 800 | 800 huge 801 | 801 human 802 | 802 hundr 803 | 803 ibm 804 | 804 id 805 | 805 idea 806 | 806 ident 807 | 807 identifi 808 | 808 idnumb 809 | 809 ie 810 | 810 if 811 | 811 ignor 812 | 812 ii 813 | 813 iii 814 | 814 iiiiiiihnumberjnumberhnumberjnumberhnumb 815 | 815 illeg 816 | 816 im 817 | 817 imag 818 | 818 imagin 819 | 819 immedi 820 | 820 impact 821 | 821 implement 822 | 822 import 823 | 823 impress 824 | 824 improv 825 | 825 in 826 | 826 inc 827 | 827 includ 828 | 828 incom 829 | 829 increas 830 | 830 incred 831 | 831 inde 832 | 832 independ 833 | 833 index 834 | 834 india 835 | 835 indian 836 | 836 indic 837 | 837 individu 838 | 838 industri 839 | 839 info 840 | 840 inform 841 | 841 initi 842 | 842 inlin 843 | 843 innov 844 | 844 input 845 | 845 insert 846 | 846 insid 847 | 847 instal 848 | 848 instanc 849 | 849 instant 850 | 850 instead 851 | 851 institut 852 | 852 instruct 853 | 853 insur 854 | 854 int 855 | 855 integr 856 | 856 intel 857 | 857 intellig 858 | 858 intend 859 | 859 interact 860 | 860 interest 861 | 861 interfac 862 | 862 intern 863 | 863 internet 864 | 864 interview 865 | 865 into 866 | 866 intro 867 | 867 introduc 868 | 868 inumb 869 | 869 invest 870 | 870 investig 871 | 871 investor 872 | 872 invok 873 | 873 involv 874 | 874 ip 875 | 875 ireland 876 | 876 irish 877 | 877 is 878 | 878 island 879 | 879 isn 880 | 880 iso 881 | 881 isp 882 | 882 issu 883 | 883 it 884 | 884 item 885 | 885 itself 886 | 886 jabber 887 | 887 jame 888 | 888 java 889 | 889 jim 890 | 890 jnumberiiiiiiihepihepihf 891 | 891 job 892 | 892 joe 893 | 893 john 894 | 894 join 895 | 895 journal 896 | 896 judg 897 | 897 judgment 898 | 898 jul 899 | 899 juli 900 | 900 jump 901 | 901 june 902 | 902 just 903 | 903 justin 904 | 904 keep 905 | 905 kei 906 | 906 kept 907 | 907 kernel 908 | 908 kevin 909 | 909 keyboard 910 | 910 kid 911 | 911 kill 912 | 912 kind 913 | 913 king 914 | 914 kingdom 915 | 915 knew 916 | 916 know 917 | 917 knowledg 918 | 918 known 919 | 919 la 920 | 920 lack 921 | 921 land 922 | 922 languag 923 | 923 laptop 924 | 924 larg 925 | 925 larger 926 | 926 largest 927 | 927 laser 928 | 928 last 929 | 929 late 930 | 930 later 931 | 931 latest 932 | 932 launch 933 | 933 law 934 | 934 lawrenc 935 | 935 le 936 | 936 lead 937 | 937 leader 938 | 938 learn 939 | 939 least 940 | 940 leav 941 | 941 left 942 | 942 legal 943 | 943 lender 944 | 944 length 945 | 945 less 946 | 946 lesson 947 | 947 let 948 | 948 letter 949 | 949 level 950 | 950 lib 951 | 951 librari 952 | 952 licens 953 | 953 life 954 | 954 lifetim 955 | 955 light 956 | 956 like 957 | 957 limit 958 | 958 line 959 | 959 link 960 | 960 linux 961 | 961 list 962 | 962 listen 963 | 963 littl 964 | 964 live 965 | 965 ll 966 | 966 lo 967 | 967 load 968 | 968 loan 969 | 969 local 970 | 970 locat 971 | 971 lock 972 | 972 lockergnom 973 | 973 log 974 | 974 long 975 | 975 longer 976 | 976 look 977 | 977 lose 978 | 978 loss 979 | 979 lost 980 | 980 lot 981 | 981 love 982 | 982 low 983 | 983 lower 984 | 984 lowest 985 | 985 lt 986 | 986 ma 987 | 987 mac 988 | 988 machin 989 | 989 made 990 | 990 magazin 991 | 991 mai 992 | 992 mail 993 | 993 mailer 994 | 994 main 995 | 995 maintain 996 | 996 major 997 | 997 make 998 | 998 maker 999 | 999 male 1000 | 1000 man 1001 | 1001 manag 1002 | 1002 mani 1003 | 1003 manual 1004 | 1004 manufactur 1005 | 1005 map 1006 | 1006 march 1007 | 1007 margin 1008 | 1008 mark 1009 | 1009 market 1010 | 1010 marshal 1011 | 1011 mass 1012 | 1012 master 1013 | 1013 match 1014 | 1014 materi 1015 | 1015 matter 1016 | 1016 matthia 1017 | 1017 mayb 1018 | 1018 me 1019 | 1019 mean 1020 | 1020 measur 1021 | 1021 mechan 1022 | 1022 media 1023 | 1023 medic 1024 | 1024 meet 1025 | 1025 member 1026 | 1026 membership 1027 | 1027 memori 1028 | 1028 men 1029 | 1029 mention 1030 | 1030 menu 1031 | 1031 merchant 1032 | 1032 messag 1033 | 1033 method 1034 | 1034 mh 1035 | 1035 michael 1036 | 1036 microsoft 1037 | 1037 middl 1038 | 1038 might 1039 | 1039 mike 1040 | 1040 mile 1041 | 1041 militari 1042 | 1042 million 1043 | 1043 mime 1044 | 1044 mind 1045 | 1045 mine 1046 | 1046 mini 1047 | 1047 minimum 1048 | 1048 minut 1049 | 1049 miss 1050 | 1050 mistak 1051 | 1051 mobil 1052 | 1052 mode 1053 | 1053 model 1054 | 1054 modem 1055 | 1055 modifi 1056 | 1056 modul 1057 | 1057 moment 1058 | 1058 mon 1059 | 1059 mondai 1060 | 1060 monei 1061 | 1061 monitor 1062 | 1062 month 1063 | 1063 monthli 1064 | 1064 more 1065 | 1065 morn 1066 | 1066 mortgag 1067 | 1067 most 1068 | 1068 mostli 1069 | 1069 mother 1070 | 1070 motiv 1071 | 1071 move 1072 | 1072 movi 1073 | 1073 mpnumber 1074 | 1074 mr 1075 | 1075 ms 1076 | 1076 msg 1077 | 1077 much 1078 | 1078 multi 1079 | 1079 multipart 1080 | 1080 multipl 1081 | 1081 murphi 1082 | 1082 music 1083 | 1083 must 1084 | 1084 my 1085 | 1085 myself 1086 | 1086 name 1087 | 1087 nation 1088 | 1088 natur 1089 | 1089 nbsp 1090 | 1090 near 1091 | 1091 nearli 1092 | 1092 necessari 1093 | 1093 need 1094 | 1094 neg 1095 | 1095 net 1096 | 1096 netscap 1097 | 1097 network 1098 | 1098 never 1099 | 1099 new 1100 | 1100 newslett 1101 | 1101 next 1102 | 1102 nextpart 1103 | 1103 nice 1104 | 1104 nigeria 1105 | 1105 night 1106 | 1106 no 1107 | 1107 nobodi 1108 | 1108 non 1109 | 1109 none 1110 | 1110 nor 1111 | 1111 normal 1112 | 1112 north 1113 | 1113 not 1114 | 1114 note 1115 | 1115 noth 1116 | 1116 notic 1117 | 1117 now 1118 | 1118 nt 1119 | 1119 null 1120 | 1120 number 1121 | 1121 numbera 1122 | 1122 numberam 1123 | 1123 numberanumb 1124 | 1124 numberb 1125 | 1125 numberbit 1126 | 1126 numberc 1127 | 1127 numbercb 1128 | 1128 numbercbr 1129 | 1129 numbercfont 1130 | 1130 numbercli 1131 | 1131 numbercnumb 1132 | 1132 numbercp 1133 | 1133 numberctd 1134 | 1134 numberd 1135 | 1135 numberdari 1136 | 1136 numberdnumb 1137 | 1137 numberenumb 1138 | 1138 numberf 1139 | 1139 numberfb 1140 | 1140 numberff 1141 | 1141 numberffont 1142 | 1142 numberfp 1143 | 1143 numberftd 1144 | 1144 numberk 1145 | 1145 numberm 1146 | 1146 numbermb 1147 | 1147 numberp 1148 | 1148 numberpd 1149 | 1149 numberpm 1150 | 1150 numberpx 1151 | 1151 numberst 1152 | 1152 numberth 1153 | 1153 numbertnumb 1154 | 1154 numberx 1155 | 1155 object 1156 | 1156 oblig 1157 | 1157 obtain 1158 | 1158 obvious 1159 | 1159 occur 1160 | 1160 oct 1161 | 1161 octob 1162 | 1162 of 1163 | 1163 off 1164 | 1164 offer 1165 | 1165 offic 1166 | 1166 offici 1167 | 1167 often 1168 | 1168 oh 1169 | 1169 ok 1170 | 1170 old 1171 | 1171 on 1172 | 1172 onc 1173 | 1173 onli 1174 | 1174 onlin 1175 | 1175 open 1176 | 1176 oper 1177 | 1177 opinion 1178 | 1178 opportun 1179 | 1179 opt 1180 | 1180 optim 1181 | 1181 option 1182 | 1182 or 1183 | 1183 order 1184 | 1184 org 1185 | 1185 organ 1186 | 1186 origin 1187 | 1187 os 1188 | 1188 osdn 1189 | 1189 other 1190 | 1190 otherwis 1191 | 1191 our 1192 | 1192 out 1193 | 1193 outlook 1194 | 1194 output 1195 | 1195 outsid 1196 | 1196 over 1197 | 1197 own 1198 | 1198 owner 1199 | 1199 oz 1200 | 1200 pacif 1201 | 1201 pack 1202 | 1202 packag 1203 | 1203 page 1204 | 1204 pai 1205 | 1205 paid 1206 | 1206 pain 1207 | 1207 palm 1208 | 1208 panel 1209 | 1209 paper 1210 | 1210 paragraph 1211 | 1211 parent 1212 | 1212 part 1213 | 1213 parti 1214 | 1214 particip 1215 | 1215 particular 1216 | 1216 particularli 1217 | 1217 partit 1218 | 1218 partner 1219 | 1219 pass 1220 | 1220 password 1221 | 1221 past 1222 | 1222 patch 1223 | 1223 patent 1224 | 1224 path 1225 | 1225 pattern 1226 | 1226 paul 1227 | 1227 payment 1228 | 1228 pc 1229 | 1229 peac 1230 | 1230 peopl 1231 | 1231 per 1232 | 1232 percent 1233 | 1233 percentag 1234 | 1234 perfect 1235 | 1235 perfectli 1236 | 1236 perform 1237 | 1237 perhap 1238 | 1238 period 1239 | 1239 perl 1240 | 1240 perman 1241 | 1241 permiss 1242 | 1242 person 1243 | 1243 pgp 1244 | 1244 phone 1245 | 1245 photo 1246 | 1246 php 1247 | 1247 phrase 1248 | 1248 physic 1249 | 1249 pick 1250 | 1250 pictur 1251 | 1251 piec 1252 | 1252 piiiiiiii 1253 | 1253 pipe 1254 | 1254 pjnumber 1255 | 1255 place 1256 | 1256 plai 1257 | 1257 plain 1258 | 1258 plan 1259 | 1259 planet 1260 | 1260 plant 1261 | 1261 planta 1262 | 1262 platform 1263 | 1263 player 1264 | 1264 pleas 1265 | 1265 plu 1266 | 1266 plug 1267 | 1267 pm 1268 | 1268 pocket 1269 | 1269 point 1270 | 1270 polic 1271 | 1271 polici 1272 | 1272 polit 1273 | 1273 poor 1274 | 1274 pop 1275 | 1275 popul 1276 | 1276 popular 1277 | 1277 port 1278 | 1278 posit 1279 | 1279 possibl 1280 | 1280 post 1281 | 1281 potenti 1282 | 1282 pound 1283 | 1283 powel 1284 | 1284 power 1285 | 1285 powershot 1286 | 1286 practic 1287 | 1287 pre 1288 | 1288 predict 1289 | 1289 prefer 1290 | 1290 premium 1291 | 1291 prepar 1292 | 1292 present 1293 | 1293 presid 1294 | 1294 press 1295 | 1295 pretti 1296 | 1296 prevent 1297 | 1297 previou 1298 | 1298 previous 1299 | 1299 price 1300 | 1300 principl 1301 | 1301 print 1302 | 1302 printabl 1303 | 1303 printer 1304 | 1304 privaci 1305 | 1305 privat 1306 | 1306 prize 1307 | 1307 pro 1308 | 1308 probabl 1309 | 1309 problem 1310 | 1310 procedur 1311 | 1311 process 1312 | 1312 processor 1313 | 1313 procmail 1314 | 1314 produc 1315 | 1315 product 1316 | 1316 profession 1317 | 1317 profil 1318 | 1318 profit 1319 | 1319 program 1320 | 1320 programm 1321 | 1321 progress 1322 | 1322 project 1323 | 1323 promis 1324 | 1324 promot 1325 | 1325 prompt 1326 | 1326 properti 1327 | 1327 propos 1328 | 1328 proprietari 1329 | 1329 prospect 1330 | 1330 protect 1331 | 1331 protocol 1332 | 1332 prove 1333 | 1333 proven 1334 | 1334 provid 1335 | 1335 proxi 1336 | 1336 pub 1337 | 1337 public 1338 | 1338 publish 1339 | 1339 pudg 1340 | 1340 pull 1341 | 1341 purchas 1342 | 1342 purpos 1343 | 1343 put 1344 | 1344 python 1345 | 1345 qnumber 1346 | 1346 qualifi 1347 | 1347 qualiti 1348 | 1348 quarter 1349 | 1349 question 1350 | 1350 quick 1351 | 1351 quickli 1352 | 1352 quit 1353 | 1353 quot 1354 | 1354 radio 1355 | 1355 ragga 1356 | 1356 rais 1357 | 1357 random 1358 | 1358 rang 1359 | 1359 rate 1360 | 1360 rather 1361 | 1361 ratio 1362 | 1362 razor 1363 | 1363 razornumb 1364 | 1364 re 1365 | 1365 reach 1366 | 1366 read 1367 | 1367 reader 1368 | 1368 readi 1369 | 1369 real 1370 | 1370 realiz 1371 | 1371 realli 1372 | 1372 reason 1373 | 1373 receiv 1374 | 1374 recent 1375 | 1375 recipi 1376 | 1376 recommend 1377 | 1377 record 1378 | 1378 red 1379 | 1379 redhat 1380 | 1380 reduc 1381 | 1381 refer 1382 | 1382 refin 1383 | 1383 reg 1384 | 1384 regard 1385 | 1385 region 1386 | 1386 regist 1387 | 1387 regul 1388 | 1388 regular 1389 | 1389 rel 1390 | 1390 relat 1391 | 1391 relationship 1392 | 1392 releas 1393 | 1393 relev 1394 | 1394 reliabl 1395 | 1395 remain 1396 | 1396 rememb 1397 | 1397 remot 1398 | 1398 remov 1399 | 1399 replac 1400 | 1400 repli 1401 | 1401 report 1402 | 1402 repositori 1403 | 1403 repres 1404 | 1404 republ 1405 | 1405 request 1406 | 1406 requir 1407 | 1407 research 1408 | 1408 reserv 1409 | 1409 resid 1410 | 1410 resourc 1411 | 1411 respect 1412 | 1412 respond 1413 | 1413 respons 1414 | 1414 rest 1415 | 1415 result 1416 | 1416 retail 1417 | 1417 return 1418 | 1418 reveal 1419 | 1419 revenu 1420 | 1420 revers 1421 | 1421 review 1422 | 1422 revok 1423 | 1423 rh 1424 | 1424 rich 1425 | 1425 right 1426 | 1426 risk 1427 | 1427 road 1428 | 1428 robert 1429 | 1429 rock 1430 | 1430 role 1431 | 1431 roll 1432 | 1432 rom 1433 | 1433 roman 1434 | 1434 room 1435 | 1435 root 1436 | 1436 round 1437 | 1437 rpm 1438 | 1438 rss 1439 | 1439 rule 1440 | 1440 run 1441 | 1441 sa 1442 | 1442 safe 1443 | 1443 sai 1444 | 1444 said 1445 | 1445 sale 1446 | 1446 same 1447 | 1447 sampl 1448 | 1448 san 1449 | 1449 saou 1450 | 1450 sat 1451 | 1451 satellit 1452 | 1452 save 1453 | 1453 saw 1454 | 1454 scan 1455 | 1455 schedul 1456 | 1456 school 1457 | 1457 scienc 1458 | 1458 score 1459 | 1459 screen 1460 | 1460 script 1461 | 1461 se 1462 | 1462 search 1463 | 1463 season 1464 | 1464 second 1465 | 1465 secret 1466 | 1466 section 1467 | 1467 secur 1468 | 1468 see 1469 | 1469 seed 1470 | 1470 seek 1471 | 1471 seem 1472 | 1472 seen 1473 | 1473 select 1474 | 1474 self 1475 | 1475 sell 1476 | 1476 seminar 1477 | 1477 send 1478 | 1478 sender 1479 | 1479 sendmail 1480 | 1480 senior 1481 | 1481 sens 1482 | 1482 sensit 1483 | 1483 sent 1484 | 1484 sep 1485 | 1485 separ 1486 | 1486 septemb 1487 | 1487 sequenc 1488 | 1488 seri 1489 | 1489 serif 1490 | 1490 seriou 1491 | 1491 serv 1492 | 1492 server 1493 | 1493 servic 1494 | 1494 set 1495 | 1495 setup 1496 | 1496 seven 1497 | 1497 seventh 1498 | 1498 sever 1499 | 1499 sex 1500 | 1500 sexual 1501 | 1501 sf 1502 | 1502 shape 1503 | 1503 share 1504 | 1504 she 1505 | 1505 shell 1506 | 1506 ship 1507 | 1507 shop 1508 | 1508 short 1509 | 1509 shot 1510 | 1510 should 1511 | 1511 show 1512 | 1512 side 1513 | 1513 sign 1514 | 1514 signatur 1515 | 1515 signific 1516 | 1516 similar 1517 | 1517 simpl 1518 | 1518 simpli 1519 | 1519 sinc 1520 | 1520 sincer 1521 | 1521 singl 1522 | 1522 sit 1523 | 1523 site 1524 | 1524 situat 1525 | 1525 six 1526 | 1526 size 1527 | 1527 skeptic 1528 | 1528 skill 1529 | 1529 skin 1530 | 1530 skip 1531 | 1531 sleep 1532 | 1532 slow 1533 | 1533 small 1534 | 1534 smart 1535 | 1535 smoke 1536 | 1536 smtp 1537 | 1537 snumber 1538 | 1538 so 1539 | 1539 social 1540 | 1540 societi 1541 | 1541 softwar 1542 | 1542 sold 1543 | 1543 solut 1544 | 1544 solv 1545 | 1545 some 1546 | 1546 someon 1547 | 1547 someth 1548 | 1548 sometim 1549 | 1549 son 1550 | 1550 song 1551 | 1551 soni 1552 | 1552 soon 1553 | 1553 sorri 1554 | 1554 sort 1555 | 1555 sound 1556 | 1556 sourc 1557 | 1557 south 1558 | 1558 space 1559 | 1559 spain 1560 | 1560 spam 1561 | 1561 spamassassin 1562 | 1562 spamd 1563 | 1563 spammer 1564 | 1564 speak 1565 | 1565 spec 1566 | 1566 special 1567 | 1567 specif 1568 | 1568 specifi 1569 | 1569 speech 1570 | 1570 speed 1571 | 1571 spend 1572 | 1572 sponsor 1573 | 1573 sport 1574 | 1574 spot 1575 | 1575 src 1576 | 1576 ssh 1577 | 1577 st 1578 | 1578 stabl 1579 | 1579 staff 1580 | 1580 stai 1581 | 1581 stand 1582 | 1582 standard 1583 | 1583 star 1584 | 1584 start 1585 | 1585 state 1586 | 1586 statement 1587 | 1587 statu 1588 | 1588 step 1589 | 1589 steve 1590 | 1590 still 1591 | 1591 stock 1592 | 1592 stop 1593 | 1593 storag 1594 | 1594 store 1595 | 1595 stori 1596 | 1596 strategi 1597 | 1597 stream 1598 | 1598 street 1599 | 1599 string 1600 | 1600 strip 1601 | 1601 strong 1602 | 1602 structur 1603 | 1603 studi 1604 | 1604 stuff 1605 | 1605 stupid 1606 | 1606 style 1607 | 1607 subject 1608 | 1608 submit 1609 | 1609 subscrib 1610 | 1610 subscript 1611 | 1611 substanti 1612 | 1612 success 1613 | 1613 such 1614 | 1614 suffer 1615 | 1615 suggest 1616 | 1616 suit 1617 | 1617 sum 1618 | 1618 summari 1619 | 1619 summer 1620 | 1620 sun 1621 | 1621 super 1622 | 1622 suppli 1623 | 1623 support 1624 | 1624 suppos 1625 | 1625 sure 1626 | 1626 surpris 1627 | 1627 suse 1628 | 1628 suspect 1629 | 1629 sweet 1630 | 1630 switch 1631 | 1631 system 1632 | 1632 tab 1633 | 1633 tabl 1634 | 1634 tablet 1635 | 1635 tag 1636 | 1636 take 1637 | 1637 taken 1638 | 1638 talk 1639 | 1639 tape 1640 | 1640 target 1641 | 1641 task 1642 | 1642 tax 1643 | 1643 teach 1644 | 1644 team 1645 | 1645 tech 1646 | 1646 technic 1647 | 1647 techniqu 1648 | 1648 technolog 1649 | 1649 tel 1650 | 1650 telecom 1651 | 1651 telephon 1652 | 1652 tell 1653 | 1653 temperatur 1654 | 1654 templ 1655 | 1655 ten 1656 | 1656 term 1657 | 1657 termin 1658 | 1658 terror 1659 | 1659 terrorist 1660 | 1660 test 1661 | 1661 texa 1662 | 1662 text 1663 | 1663 than 1664 | 1664 thank 1665 | 1665 that 1666 | 1666 the 1667 | 1667 thei 1668 | 1668 their 1669 | 1669 them 1670 | 1670 themselv 1671 | 1671 then 1672 | 1672 theori 1673 | 1673 there 1674 | 1674 therefor 1675 | 1675 these 1676 | 1676 thi 1677 | 1677 thing 1678 | 1678 think 1679 | 1679 thinkgeek 1680 | 1680 third 1681 | 1681 those 1682 | 1682 though 1683 | 1683 thought 1684 | 1684 thousand 1685 | 1685 thread 1686 | 1686 threat 1687 | 1687 three 1688 | 1688 through 1689 | 1689 thu 1690 | 1690 thursdai 1691 | 1691 ti 1692 | 1692 ticket 1693 | 1693 tim 1694 | 1694 time 1695 | 1695 tip 1696 | 1696 tire 1697 | 1697 titl 1698 | 1698 tm 1699 | 1699 to 1700 | 1700 todai 1701 | 1701 togeth 1702 | 1702 token 1703 | 1703 told 1704 | 1704 toll 1705 | 1705 tom 1706 | 1706 toner 1707 | 1707 toni 1708 | 1708 too 1709 | 1709 took 1710 | 1710 tool 1711 | 1711 top 1712 | 1712 topic 1713 | 1713 total 1714 | 1714 touch 1715 | 1715 toward 1716 | 1716 track 1717 | 1717 trade 1718 | 1718 tradit 1719 | 1719 traffic 1720 | 1720 train 1721 | 1721 transact 1722 | 1722 transfer 1723 | 1723 travel 1724 | 1724 treat 1725 | 1725 tree 1726 | 1726 tri 1727 | 1727 trial 1728 | 1728 trick 1729 | 1729 trip 1730 | 1730 troubl 1731 | 1731 true 1732 | 1732 truli 1733 | 1733 trust 1734 | 1734 truth 1735 | 1735 try 1736 | 1736 tue 1737 | 1737 tuesdai 1738 | 1738 turn 1739 | 1739 tv 1740 | 1740 two 1741 | 1741 type 1742 | 1742 uk 1743 | 1743 ultim 1744 | 1744 un 1745 | 1745 under 1746 | 1746 understand 1747 | 1747 unfortun 1748 | 1748 uniqu 1749 | 1749 unison 1750 | 1750 unit 1751 | 1751 univers 1752 | 1752 unix 1753 | 1753 unless 1754 | 1754 unlik 1755 | 1755 unlimit 1756 | 1756 unseen 1757 | 1757 unsolicit 1758 | 1758 unsubscrib 1759 | 1759 until 1760 | 1760 up 1761 | 1761 updat 1762 | 1762 upgrad 1763 | 1763 upon 1764 | 1764 urgent 1765 | 1765 url 1766 | 1766 us 1767 | 1767 usa 1768 | 1768 usag 1769 | 1769 usb 1770 | 1770 usd 1771 | 1771 usdollarnumb 1772 | 1772 useless 1773 | 1773 user 1774 | 1774 usr 1775 | 1775 usual 1776 | 1776 util 1777 | 1777 vacat 1778 | 1778 valid 1779 | 1779 valu 1780 | 1780 valuabl 1781 | 1781 var 1782 | 1782 variabl 1783 | 1783 varieti 1784 | 1784 variou 1785 | 1785 ve 1786 | 1786 vendor 1787 | 1787 ventur 1788 | 1788 veri 1789 | 1789 verifi 1790 | 1790 version 1791 | 1791 via 1792 | 1792 video 1793 | 1793 view 1794 | 1794 virtual 1795 | 1795 visa 1796 | 1796 visit 1797 | 1797 visual 1798 | 1798 vnumber 1799 | 1799 voic 1800 | 1800 vote 1801 | 1801 vs 1802 | 1802 vulner 1803 | 1803 wa 1804 | 1804 wai 1805 | 1805 wait 1806 | 1806 wake 1807 | 1807 walk 1808 | 1808 wall 1809 | 1809 want 1810 | 1810 war 1811 | 1811 warm 1812 | 1812 warn 1813 | 1813 warranti 1814 | 1814 washington 1815 | 1815 wasn 1816 | 1816 wast 1817 | 1817 watch 1818 | 1818 water 1819 | 1819 we 1820 | 1820 wealth 1821 | 1821 weapon 1822 | 1822 web 1823 | 1823 weblog 1824 | 1824 websit 1825 | 1825 wed 1826 | 1826 wednesdai 1827 | 1827 week 1828 | 1828 weekli 1829 | 1829 weight 1830 | 1830 welcom 1831 | 1831 well 1832 | 1832 went 1833 | 1833 were 1834 | 1834 west 1835 | 1835 what 1836 | 1836 whatev 1837 | 1837 when 1838 | 1838 where 1839 | 1839 whether 1840 | 1840 which 1841 | 1841 while 1842 | 1842 white 1843 | 1843 whitelist 1844 | 1844 who 1845 | 1845 whole 1846 | 1846 whose 1847 | 1847 why 1848 | 1848 wi 1849 | 1849 wide 1850 | 1850 width 1851 | 1851 wife 1852 | 1852 will 1853 | 1853 william 1854 | 1854 win 1855 | 1855 window 1856 | 1856 wing 1857 | 1857 winner 1858 | 1858 wireless 1859 | 1859 wish 1860 | 1860 with 1861 | 1861 within 1862 | 1862 without 1863 | 1863 wnumberp 1864 | 1864 woman 1865 | 1865 women 1866 | 1866 won 1867 | 1867 wonder 1868 | 1868 word 1869 | 1869 work 1870 | 1870 worker 1871 | 1871 world 1872 | 1872 worldwid 1873 | 1873 worri 1874 | 1874 worst 1875 | 1875 worth 1876 | 1876 would 1877 | 1877 wouldn 1878 | 1878 write 1879 | 1879 written 1880 | 1880 wrong 1881 | 1881 wrote 1882 | 1882 www 1883 | 1883 ximian 1884 | 1884 xml 1885 | 1885 xp 1886 | 1886 yahoo 1887 | 1887 ye 1888 | 1888 yeah 1889 | 1889 year 1890 | 1890 yesterdai 1891 | 1891 yet 1892 | 1892 york 1893 | 1893 you 1894 | 1894 young 1895 | 1895 your 1896 | 1896 yourself 1897 | 1897 zdnet 1898 | 1898 zero 1899 | 1899 zip 1900 | --------------------------------------------------------------------------------