├── Housing Price Prediction.md
├── README.md
├── output_29_1.png
└── output_31_1.png


/Housing Price Prediction.md:
--------------------------------------------------------------------------------
   1 | 
   2 |  <a href="https://www.bigdatauniversity.com"><img src = "https://ibm.box.com/shared/static/ugcqz6ohbvff804xp84y4kqnvvk3bq1g.png" width = 300, align = "center"></a>
   3 | 
   4 | <h1 align=center><font size = 5>Data Analysis with Python</font></h1>
   5 | 
   6 | # House Sales in King County, USA
   7 | 
   8 | This dataset contains house sale prices for King County, which includes Seattle. It includes homes sold between May 2014 and May 2015.
   9 | 
  10 | <b>id</b> :a notation for a house
  11 | 
  12 | <b> date</b>: Date house was sold
  13 | 
  14 | 
  15 | <b>price</b>: Price is prediction target
  16 | 
  17 | 
  18 | <b>bedrooms</b>: Number of Bedrooms/House
  19 | 
  20 | 
  21 | <b>bathrooms</b>: Number of bathrooms/bedrooms
  22 | 
  23 | <b>sqft_living</b>: square footage of the home
  24 | 
  25 | <b>sqft_lot</b>: square footage of the lot
  26 | 
  27 | 
  28 | <b>floors</b> :Total floors (levels) in house
  29 | 
  30 | 
  31 | <b>waterfront</b> :House which has a view to a waterfront
  32 | 
  33 | 
  34 | <b>view</b>: Has been viewed
  35 | 
  36 | 
  37 | <b>condition</b> :How good the condition is  Overall
  38 | 
  39 | <b>grade</b>: overall grade given to the housing unit, based on King County grading system
  40 | 
  41 | 
  42 | <b>sqft_above</b> :square footage of house apart from basement
  43 | 
  44 | 
  45 | <b>sqft_basement</b>: square footage of the basement
  46 | 
  47 | <b>yr_built</b> :Built Year
  48 | 
  49 | 
  50 | <b>yr_renovated</b> :Year when house was renovated
  51 | 
  52 | <b>zipcode</b>:zip code
  53 | 
  54 | 
  55 | <b>lat</b>: Latitude coordinate
  56 | 
  57 | <b>long</b>: Longitude coordinate
  58 | 
  59 | <b>sqft_living15</b> :Living room area in 2015(implies-- some renovations) This might or might not have affected the lotsize area
  60 | 
  61 | 
  62 | <b>sqft_lot15</b> :lotSize area in 2015(implies-- some renovations)
  63 | 
  64 | You will require the following libraries 
  65 | 
  66 | 
  67 | ```python
  68 | import pandas as pd
  69 | import matplotlib.pyplot as plt
  70 | import numpy as np
  71 | import seaborn as sns
  72 | from sklearn.pipeline import Pipeline
  73 | from sklearn.preprocessing import StandardScaler,PolynomialFeatures
  74 | %matplotlib inline
  75 | ```
  76 | 
  77 | # 1.0 Importing the Data 
  78 | 
  79 |  Load the csv:  
  80 | 
  81 | 
  82 | ```python
  83 | file_name='https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DA0101EN/coursera/project/kc_house_data_NaN.csv'
  84 | df=pd.read_csv(file_name)
  85 | ```
  86 | 
  87 | 
  88 | we use the method <code>head</code> to display the first 5 columns of the dataframe.
  89 | 
  90 | 
  91 | ```python
  92 | df.head(5)
  93 | ```
  94 | 
  95 | 
  96 | 
  97 | 
  98 | <div>
  99 | <style scoped>
 100 |     .dataframe tbody tr th:only-of-type {
 101 |         vertical-align: middle;
 102 |     }
 103 | 
 104 |     .dataframe tbody tr th {
 105 |         vertical-align: top;
 106 |     }
 107 | 
 108 |     .dataframe thead th {
 109 |         text-align: right;
 110 |     }
 111 | </style>
 112 | <table border="1" class="dataframe">
 113 |   <thead>
 114 |     <tr style="text-align: right;">
 115 |       <th></th>
 116 |       <th>Unnamed: 0</th>
 117 |       <th>id</th>
 118 |       <th>date</th>
 119 |       <th>price</th>
 120 |       <th>bedrooms</th>
 121 |       <th>bathrooms</th>
 122 |       <th>sqft_living</th>
 123 |       <th>sqft_lot</th>
 124 |       <th>floors</th>
 125 |       <th>waterfront</th>
 126 |       <th>...</th>
 127 |       <th>grade</th>
 128 |       <th>sqft_above</th>
 129 |       <th>sqft_basement</th>
 130 |       <th>yr_built</th>
 131 |       <th>yr_renovated</th>
 132 |       <th>zipcode</th>
 133 |       <th>lat</th>
 134 |       <th>long</th>
 135 |       <th>sqft_living15</th>
 136 |       <th>sqft_lot15</th>
 137 |     </tr>
 138 |   </thead>
 139 |   <tbody>
 140 |     <tr>
 141 |       <th>0</th>
 142 |       <td>0</td>
 143 |       <td>7129300520</td>
 144 |       <td>20141013T000000</td>
 145 |       <td>221900.0</td>
 146 |       <td>3.0</td>
 147 |       <td>1.00</td>
 148 |       <td>1180</td>
 149 |       <td>5650</td>
 150 |       <td>1.0</td>
 151 |       <td>0</td>
 152 |       <td>...</td>
 153 |       <td>7</td>
 154 |       <td>1180</td>
 155 |       <td>0</td>
 156 |       <td>1955</td>
 157 |       <td>0</td>
 158 |       <td>98178</td>
 159 |       <td>47.5112</td>
 160 |       <td>-122.257</td>
 161 |       <td>1340</td>
 162 |       <td>5650</td>
 163 |     </tr>
 164 |     <tr>
 165 |       <th>1</th>
 166 |       <td>1</td>
 167 |       <td>6414100192</td>
 168 |       <td>20141209T000000</td>
 169 |       <td>538000.0</td>
 170 |       <td>3.0</td>
 171 |       <td>2.25</td>
 172 |       <td>2570</td>
 173 |       <td>7242</td>
 174 |       <td>2.0</td>
 175 |       <td>0</td>
 176 |       <td>...</td>
 177 |       <td>7</td>
 178 |       <td>2170</td>
 179 |       <td>400</td>
 180 |       <td>1951</td>
 181 |       <td>1991</td>
 182 |       <td>98125</td>
 183 |       <td>47.7210</td>
 184 |       <td>-122.319</td>
 185 |       <td>1690</td>
 186 |       <td>7639</td>
 187 |     </tr>
 188 |     <tr>
 189 |       <th>2</th>
 190 |       <td>2</td>
 191 |       <td>5631500400</td>
 192 |       <td>20150225T000000</td>
 193 |       <td>180000.0</td>
 194 |       <td>2.0</td>
 195 |       <td>1.00</td>
 196 |       <td>770</td>
 197 |       <td>10000</td>
 198 |       <td>1.0</td>
 199 |       <td>0</td>
 200 |       <td>...</td>
 201 |       <td>6</td>
 202 |       <td>770</td>
 203 |       <td>0</td>
 204 |       <td>1933</td>
 205 |       <td>0</td>
 206 |       <td>98028</td>
 207 |       <td>47.7379</td>
 208 |       <td>-122.233</td>
 209 |       <td>2720</td>
 210 |       <td>8062</td>
 211 |     </tr>
 212 |     <tr>
 213 |       <th>3</th>
 214 |       <td>3</td>
 215 |       <td>2487200875</td>
 216 |       <td>20141209T000000</td>
 217 |       <td>604000.0</td>
 218 |       <td>4.0</td>
 219 |       <td>3.00</td>
 220 |       <td>1960</td>
 221 |       <td>5000</td>
 222 |       <td>1.0</td>
 223 |       <td>0</td>
 224 |       <td>...</td>
 225 |       <td>7</td>
 226 |       <td>1050</td>
 227 |       <td>910</td>
 228 |       <td>1965</td>
 229 |       <td>0</td>
 230 |       <td>98136</td>
 231 |       <td>47.5208</td>
 232 |       <td>-122.393</td>
 233 |       <td>1360</td>
 234 |       <td>5000</td>
 235 |     </tr>
 236 |     <tr>
 237 |       <th>4</th>
 238 |       <td>4</td>
 239 |       <td>1954400510</td>
 240 |       <td>20150218T000000</td>
 241 |       <td>510000.0</td>
 242 |       <td>3.0</td>
 243 |       <td>2.00</td>
 244 |       <td>1680</td>
 245 |       <td>8080</td>
 246 |       <td>1.0</td>
 247 |       <td>0</td>
 248 |       <td>...</td>
 249 |       <td>8</td>
 250 |       <td>1680</td>
 251 |       <td>0</td>
 252 |       <td>1987</td>
 253 |       <td>0</td>
 254 |       <td>98074</td>
 255 |       <td>47.6168</td>
 256 |       <td>-122.045</td>
 257 |       <td>1800</td>
 258 |       <td>7503</td>
 259 |     </tr>
 260 |   </tbody>
 261 | </table>
 262 | <p>5 rows × 22 columns</p>
 263 | </div>
 264 | 
 265 | 
 266 | 
 267 | #### Question 1 
 268 | Display the data types of each column using the attribute dtype, then take a screenshot and submit it, include your code in the image. 
 269 | 
 270 | 
 271 | ```python
 272 | df.dtypes
 273 | ```
 274 | 
 275 | 
 276 | 
 277 | 
 278 |     Unnamed: 0         int64
 279 |     id                 int64
 280 |     date              object
 281 |     price            float64
 282 |     bedrooms         float64
 283 |     bathrooms        float64
 284 |     sqft_living        int64
 285 |     sqft_lot           int64
 286 |     floors           float64
 287 |     waterfront         int64
 288 |     view               int64
 289 |     condition          int64
 290 |     grade              int64
 291 |     sqft_above         int64
 292 |     sqft_basement      int64
 293 |     yr_built           int64
 294 |     yr_renovated       int64
 295 |     zipcode            int64
 296 |     lat              float64
 297 |     long             float64
 298 |     sqft_living15      int64
 299 |     sqft_lot15         int64
 300 |     dtype: object
 301 | 
 302 | 
 303 | 
 304 | We use the method describe to obtain a statistical summary of the dataframe.
 305 | 
 306 | 
 307 | ```python
 308 | df.describe()
 309 | ```
 310 | 
 311 | 
 312 | 
 313 | 
 314 | <div>
 315 | <style scoped>
 316 |     .dataframe tbody tr th:only-of-type {
 317 |         vertical-align: middle;
 318 |     }
 319 | 
 320 |     .dataframe tbody tr th {
 321 |         vertical-align: top;
 322 |     }
 323 | 
 324 |     .dataframe thead th {
 325 |         text-align: right;
 326 |     }
 327 | </style>
 328 | <table border="1" class="dataframe">
 329 |   <thead>
 330 |     <tr style="text-align: right;">
 331 |       <th></th>
 332 |       <th>Unnamed: 0</th>
 333 |       <th>id</th>
 334 |       <th>price</th>
 335 |       <th>bedrooms</th>
 336 |       <th>bathrooms</th>
 337 |       <th>sqft_living</th>
 338 |       <th>sqft_lot</th>
 339 |       <th>floors</th>
 340 |       <th>waterfront</th>
 341 |       <th>view</th>
 342 |       <th>...</th>
 343 |       <th>grade</th>
 344 |       <th>sqft_above</th>
 345 |       <th>sqft_basement</th>
 346 |       <th>yr_built</th>
 347 |       <th>yr_renovated</th>
 348 |       <th>zipcode</th>
 349 |       <th>lat</th>
 350 |       <th>long</th>
 351 |       <th>sqft_living15</th>
 352 |       <th>sqft_lot15</th>
 353 |     </tr>
 354 |   </thead>
 355 |   <tbody>
 356 |     <tr>
 357 |       <th>count</th>
 358 |       <td>21613.00000</td>
 359 |       <td>2.161300e+04</td>
 360 |       <td>2.161300e+04</td>
 361 |       <td>21600.000000</td>
 362 |       <td>21603.000000</td>
 363 |       <td>21613.000000</td>
 364 |       <td>2.161300e+04</td>
 365 |       <td>21613.000000</td>
 366 |       <td>21613.000000</td>
 367 |       <td>21613.000000</td>
 368 |       <td>...</td>
 369 |       <td>21613.000000</td>
 370 |       <td>21613.000000</td>
 371 |       <td>21613.000000</td>
 372 |       <td>21613.000000</td>
 373 |       <td>21613.000000</td>
 374 |       <td>21613.000000</td>
 375 |       <td>21613.000000</td>
 376 |       <td>21613.000000</td>
 377 |       <td>21613.000000</td>
 378 |       <td>21613.000000</td>
 379 |     </tr>
 380 |     <tr>
 381 |       <th>mean</th>
 382 |       <td>10806.00000</td>
 383 |       <td>4.580302e+09</td>
 384 |       <td>5.400881e+05</td>
 385 |       <td>3.372870</td>
 386 |       <td>2.115736</td>
 387 |       <td>2079.899736</td>
 388 |       <td>1.510697e+04</td>
 389 |       <td>1.494309</td>
 390 |       <td>0.007542</td>
 391 |       <td>0.234303</td>
 392 |       <td>...</td>
 393 |       <td>7.656873</td>
 394 |       <td>1788.390691</td>
 395 |       <td>291.509045</td>
 396 |       <td>1971.005136</td>
 397 |       <td>84.402258</td>
 398 |       <td>98077.939805</td>
 399 |       <td>47.560053</td>
 400 |       <td>-122.213896</td>
 401 |       <td>1986.552492</td>
 402 |       <td>12768.455652</td>
 403 |     </tr>
 404 |     <tr>
 405 |       <th>std</th>
 406 |       <td>6239.28002</td>
 407 |       <td>2.876566e+09</td>
 408 |       <td>3.671272e+05</td>
 409 |       <td>0.926657</td>
 410 |       <td>0.768996</td>
 411 |       <td>918.440897</td>
 412 |       <td>4.142051e+04</td>
 413 |       <td>0.539989</td>
 414 |       <td>0.086517</td>
 415 |       <td>0.766318</td>
 416 |       <td>...</td>
 417 |       <td>1.175459</td>
 418 |       <td>828.090978</td>
 419 |       <td>442.575043</td>
 420 |       <td>29.373411</td>
 421 |       <td>401.679240</td>
 422 |       <td>53.505026</td>
 423 |       <td>0.138564</td>
 424 |       <td>0.140828</td>
 425 |       <td>685.391304</td>
 426 |       <td>27304.179631</td>
 427 |     </tr>
 428 |     <tr>
 429 |       <th>min</th>
 430 |       <td>0.00000</td>
 431 |       <td>1.000102e+06</td>
 432 |       <td>7.500000e+04</td>
 433 |       <td>1.000000</td>
 434 |       <td>0.500000</td>
 435 |       <td>290.000000</td>
 436 |       <td>5.200000e+02</td>
 437 |       <td>1.000000</td>
 438 |       <td>0.000000</td>
 439 |       <td>0.000000</td>
 440 |       <td>...</td>
 441 |       <td>1.000000</td>
 442 |       <td>290.000000</td>
 443 |       <td>0.000000</td>
 444 |       <td>1900.000000</td>
 445 |       <td>0.000000</td>
 446 |       <td>98001.000000</td>
 447 |       <td>47.155900</td>
 448 |       <td>-122.519000</td>
 449 |       <td>399.000000</td>
 450 |       <td>651.000000</td>
 451 |     </tr>
 452 |     <tr>
 453 |       <th>25%</th>
 454 |       <td>5403.00000</td>
 455 |       <td>2.123049e+09</td>
 456 |       <td>3.219500e+05</td>
 457 |       <td>3.000000</td>
 458 |       <td>1.750000</td>
 459 |       <td>1427.000000</td>
 460 |       <td>5.040000e+03</td>
 461 |       <td>1.000000</td>
 462 |       <td>0.000000</td>
 463 |       <td>0.000000</td>
 464 |       <td>...</td>
 465 |       <td>7.000000</td>
 466 |       <td>1190.000000</td>
 467 |       <td>0.000000</td>
 468 |       <td>1951.000000</td>
 469 |       <td>0.000000</td>
 470 |       <td>98033.000000</td>
 471 |       <td>47.471000</td>
 472 |       <td>-122.328000</td>
 473 |       <td>1490.000000</td>
 474 |       <td>5100.000000</td>
 475 |     </tr>
 476 |     <tr>
 477 |       <th>50%</th>
 478 |       <td>10806.00000</td>
 479 |       <td>3.904930e+09</td>
 480 |       <td>4.500000e+05</td>
 481 |       <td>3.000000</td>
 482 |       <td>2.250000</td>
 483 |       <td>1910.000000</td>
 484 |       <td>7.618000e+03</td>
 485 |       <td>1.500000</td>
 486 |       <td>0.000000</td>
 487 |       <td>0.000000</td>
 488 |       <td>...</td>
 489 |       <td>7.000000</td>
 490 |       <td>1560.000000</td>
 491 |       <td>0.000000</td>
 492 |       <td>1975.000000</td>
 493 |       <td>0.000000</td>
 494 |       <td>98065.000000</td>
 495 |       <td>47.571800</td>
 496 |       <td>-122.230000</td>
 497 |       <td>1840.000000</td>
 498 |       <td>7620.000000</td>
 499 |     </tr>
 500 |     <tr>
 501 |       <th>75%</th>
 502 |       <td>16209.00000</td>
 503 |       <td>7.308900e+09</td>
 504 |       <td>6.450000e+05</td>
 505 |       <td>4.000000</td>
 506 |       <td>2.500000</td>
 507 |       <td>2550.000000</td>
 508 |       <td>1.068800e+04</td>
 509 |       <td>2.000000</td>
 510 |       <td>0.000000</td>
 511 |       <td>0.000000</td>
 512 |       <td>...</td>
 513 |       <td>8.000000</td>
 514 |       <td>2210.000000</td>
 515 |       <td>560.000000</td>
 516 |       <td>1997.000000</td>
 517 |       <td>0.000000</td>
 518 |       <td>98118.000000</td>
 519 |       <td>47.678000</td>
 520 |       <td>-122.125000</td>
 521 |       <td>2360.000000</td>
 522 |       <td>10083.000000</td>
 523 |     </tr>
 524 |     <tr>
 525 |       <th>max</th>
 526 |       <td>21612.00000</td>
 527 |       <td>9.900000e+09</td>
 528 |       <td>7.700000e+06</td>
 529 |       <td>33.000000</td>
 530 |       <td>8.000000</td>
 531 |       <td>13540.000000</td>
 532 |       <td>1.651359e+06</td>
 533 |       <td>3.500000</td>
 534 |       <td>1.000000</td>
 535 |       <td>4.000000</td>
 536 |       <td>...</td>
 537 |       <td>13.000000</td>
 538 |       <td>9410.000000</td>
 539 |       <td>4820.000000</td>
 540 |       <td>2015.000000</td>
 541 |       <td>2015.000000</td>
 542 |       <td>98199.000000</td>
 543 |       <td>47.777600</td>
 544 |       <td>-121.315000</td>
 545 |       <td>6210.000000</td>
 546 |       <td>871200.000000</td>
 547 |     </tr>
 548 |   </tbody>
 549 | </table>
 550 | <p>8 rows × 21 columns</p>
 551 | </div>
 552 | 
 553 | 
 554 | 
 555 | # 2.0 Data Wrangling
 556 | 
 557 | #### Question 2 
 558 | Drop the columns <code>"id"</code>  and <code>"Unnamed: 0"</code> from axis 1 using the method <code>drop()</code>, then use the method <code>describe()</code> to obtain a statistical summary of the data. Take a screenshot and submit it, make sure the inplace parameter is set to <code>True</code>
 559 | 
 560 | 
 561 | ```python
 562 | df.drop('id', axis=1, inplace=True)
 563 | df.drop('Unnamed: 0', axis=1, inplace=True)
 564 | df.describe()
 565 | ```
 566 | 
 567 | 
 568 | 
 569 | 
 570 | <div>
 571 | <style scoped>
 572 |     .dataframe tbody tr th:only-of-type {
 573 |         vertical-align: middle;
 574 |     }
 575 | 
 576 |     .dataframe tbody tr th {
 577 |         vertical-align: top;
 578 |     }
 579 | 
 580 |     .dataframe thead th {
 581 |         text-align: right;
 582 |     }
 583 | </style>
 584 | <table border="1" class="dataframe">
 585 |   <thead>
 586 |     <tr style="text-align: right;">
 587 |       <th></th>
 588 |       <th>price</th>
 589 |       <th>bedrooms</th>
 590 |       <th>bathrooms</th>
 591 |       <th>sqft_living</th>
 592 |       <th>sqft_lot</th>
 593 |       <th>floors</th>
 594 |       <th>waterfront</th>
 595 |       <th>view</th>
 596 |       <th>condition</th>
 597 |       <th>grade</th>
 598 |       <th>sqft_above</th>
 599 |       <th>sqft_basement</th>
 600 |       <th>yr_built</th>
 601 |       <th>yr_renovated</th>
 602 |       <th>zipcode</th>
 603 |       <th>lat</th>
 604 |       <th>long</th>
 605 |       <th>sqft_living15</th>
 606 |       <th>sqft_lot15</th>
 607 |     </tr>
 608 |   </thead>
 609 |   <tbody>
 610 |     <tr>
 611 |       <th>count</th>
 612 |       <td>2.161300e+04</td>
 613 |       <td>21600.000000</td>
 614 |       <td>21603.000000</td>
 615 |       <td>21613.000000</td>
 616 |       <td>2.161300e+04</td>
 617 |       <td>21613.000000</td>
 618 |       <td>21613.000000</td>
 619 |       <td>21613.000000</td>
 620 |       <td>21613.000000</td>
 621 |       <td>21613.000000</td>
 622 |       <td>21613.000000</td>
 623 |       <td>21613.000000</td>
 624 |       <td>21613.000000</td>
 625 |       <td>21613.000000</td>
 626 |       <td>21613.000000</td>
 627 |       <td>21613.000000</td>
 628 |       <td>21613.000000</td>
 629 |       <td>21613.000000</td>
 630 |       <td>21613.000000</td>
 631 |     </tr>
 632 |     <tr>
 633 |       <th>mean</th>
 634 |       <td>5.400881e+05</td>
 635 |       <td>3.372870</td>
 636 |       <td>2.115736</td>
 637 |       <td>2079.899736</td>
 638 |       <td>1.510697e+04</td>
 639 |       <td>1.494309</td>
 640 |       <td>0.007542</td>
 641 |       <td>0.234303</td>
 642 |       <td>3.409430</td>
 643 |       <td>7.656873</td>
 644 |       <td>1788.390691</td>
 645 |       <td>291.509045</td>
 646 |       <td>1971.005136</td>
 647 |       <td>84.402258</td>
 648 |       <td>98077.939805</td>
 649 |       <td>47.560053</td>
 650 |       <td>-122.213896</td>
 651 |       <td>1986.552492</td>
 652 |       <td>12768.455652</td>
 653 |     </tr>
 654 |     <tr>
 655 |       <th>std</th>
 656 |       <td>3.671272e+05</td>
 657 |       <td>0.926657</td>
 658 |       <td>0.768996</td>
 659 |       <td>918.440897</td>
 660 |       <td>4.142051e+04</td>
 661 |       <td>0.539989</td>
 662 |       <td>0.086517</td>
 663 |       <td>0.766318</td>
 664 |       <td>0.650743</td>
 665 |       <td>1.175459</td>
 666 |       <td>828.090978</td>
 667 |       <td>442.575043</td>
 668 |       <td>29.373411</td>
 669 |       <td>401.679240</td>
 670 |       <td>53.505026</td>
 671 |       <td>0.138564</td>
 672 |       <td>0.140828</td>
 673 |       <td>685.391304</td>
 674 |       <td>27304.179631</td>
 675 |     </tr>
 676 |     <tr>
 677 |       <th>min</th>
 678 |       <td>7.500000e+04</td>
 679 |       <td>1.000000</td>
 680 |       <td>0.500000</td>
 681 |       <td>290.000000</td>
 682 |       <td>5.200000e+02</td>
 683 |       <td>1.000000</td>
 684 |       <td>0.000000</td>
 685 |       <td>0.000000</td>
 686 |       <td>1.000000</td>
 687 |       <td>1.000000</td>
 688 |       <td>290.000000</td>
 689 |       <td>0.000000</td>
 690 |       <td>1900.000000</td>
 691 |       <td>0.000000</td>
 692 |       <td>98001.000000</td>
 693 |       <td>47.155900</td>
 694 |       <td>-122.519000</td>
 695 |       <td>399.000000</td>
 696 |       <td>651.000000</td>
 697 |     </tr>
 698 |     <tr>
 699 |       <th>25%</th>
 700 |       <td>3.219500e+05</td>
 701 |       <td>3.000000</td>
 702 |       <td>1.750000</td>
 703 |       <td>1427.000000</td>
 704 |       <td>5.040000e+03</td>
 705 |       <td>1.000000</td>
 706 |       <td>0.000000</td>
 707 |       <td>0.000000</td>
 708 |       <td>3.000000</td>
 709 |       <td>7.000000</td>
 710 |       <td>1190.000000</td>
 711 |       <td>0.000000</td>
 712 |       <td>1951.000000</td>
 713 |       <td>0.000000</td>
 714 |       <td>98033.000000</td>
 715 |       <td>47.471000</td>
 716 |       <td>-122.328000</td>
 717 |       <td>1490.000000</td>
 718 |       <td>5100.000000</td>
 719 |     </tr>
 720 |     <tr>
 721 |       <th>50%</th>
 722 |       <td>4.500000e+05</td>
 723 |       <td>3.000000</td>
 724 |       <td>2.250000</td>
 725 |       <td>1910.000000</td>
 726 |       <td>7.618000e+03</td>
 727 |       <td>1.500000</td>
 728 |       <td>0.000000</td>
 729 |       <td>0.000000</td>
 730 |       <td>3.000000</td>
 731 |       <td>7.000000</td>
 732 |       <td>1560.000000</td>
 733 |       <td>0.000000</td>
 734 |       <td>1975.000000</td>
 735 |       <td>0.000000</td>
 736 |       <td>98065.000000</td>
 737 |       <td>47.571800</td>
 738 |       <td>-122.230000</td>
 739 |       <td>1840.000000</td>
 740 |       <td>7620.000000</td>
 741 |     </tr>
 742 |     <tr>
 743 |       <th>75%</th>
 744 |       <td>6.450000e+05</td>
 745 |       <td>4.000000</td>
 746 |       <td>2.500000</td>
 747 |       <td>2550.000000</td>
 748 |       <td>1.068800e+04</td>
 749 |       <td>2.000000</td>
 750 |       <td>0.000000</td>
 751 |       <td>0.000000</td>
 752 |       <td>4.000000</td>
 753 |       <td>8.000000</td>
 754 |       <td>2210.000000</td>
 755 |       <td>560.000000</td>
 756 |       <td>1997.000000</td>
 757 |       <td>0.000000</td>
 758 |       <td>98118.000000</td>
 759 |       <td>47.678000</td>
 760 |       <td>-122.125000</td>
 761 |       <td>2360.000000</td>
 762 |       <td>10083.000000</td>
 763 |     </tr>
 764 |     <tr>
 765 |       <th>max</th>
 766 |       <td>7.700000e+06</td>
 767 |       <td>33.000000</td>
 768 |       <td>8.000000</td>
 769 |       <td>13540.000000</td>
 770 |       <td>1.651359e+06</td>
 771 |       <td>3.500000</td>
 772 |       <td>1.000000</td>
 773 |       <td>4.000000</td>
 774 |       <td>5.000000</td>
 775 |       <td>13.000000</td>
 776 |       <td>9410.000000</td>
 777 |       <td>4820.000000</td>
 778 |       <td>2015.000000</td>
 779 |       <td>2015.000000</td>
 780 |       <td>98199.000000</td>
 781 |       <td>47.777600</td>
 782 |       <td>-121.315000</td>
 783 |       <td>6210.000000</td>
 784 |       <td>871200.000000</td>
 785 |     </tr>
 786 |   </tbody>
 787 | </table>
 788 | </div>
 789 | 
 790 | 
 791 | 
 792 | we can see we have missing values for the columns <code> bedrooms</code>  and <code> bathrooms </code>
 793 | 
 794 | 
 795 | ```python
 796 | print("number of NaN values for the column bedrooms :", df['bedrooms'].isnull().sum())
 797 | print("number of NaN values for the column bathrooms :", df['bathrooms'].isnull().sum())
 798 | 
 799 | ```
 800 | 
 801 |     number of NaN values for the column bedrooms : 13
 802 |     number of NaN values for the column bathrooms : 10
 803 | 
 804 | 
 805 | 
 806 | We can replace the missing values of the column <code>'bedrooms'</code> with the mean of the column  <code>'bedrooms' </code> using the method replace. Don't forget to set the <code>inplace</code> parameter top <code>True</code>
 807 | 
 808 | 
 809 | ```python
 810 | mean=df['bedrooms'].mean()
 811 | df['bedrooms'].replace(np.nan,mean, inplace=True)
 812 | ```
 813 | 
 814 | 
 815 | We also replace the missing values of the column <code>'bathrooms'</code> with the mean of the column  <code>'bedrooms' </codse> using the method replace.Don't forget to set the <code> inplace </code>  parameter top <code> Ture </code>
 816 | 
 817 | 
 818 | ```python
 819 | mean=df['bathrooms'].mean()
 820 | df['bathrooms'].replace(np.nan,mean, inplace=True)
 821 | ```
 822 | 
 823 | 
 824 | ```python
 825 | print("number of NaN values for the column bedrooms :", df['bedrooms'].isnull().sum())
 826 | print("number of NaN values for the column bathrooms :", df['bathrooms'].isnull().sum())
 827 | ```
 828 | 
 829 |     number of NaN values for the column bedrooms : 0
 830 |     number of NaN values for the column bathrooms : 0
 831 | 
 832 | 
 833 | # 3.0 Exploratory data analysis
 834 | 
 835 | #### Question 3
 836 | Use the method value_counts to count the number of houses with unique floor values, use the method .to_frame() to convert it to a dataframe.
 837 | 
 838 | 
 839 | 
 840 | ```python
 841 | df['floors'].value_counts().to_frame()
 842 | ```
 843 | 
 844 | 
 845 | 
 846 | 
 847 | <div>
 848 | <style scoped>
 849 |     .dataframe tbody tr th:only-of-type {
 850 |         vertical-align: middle;
 851 |     }
 852 | 
 853 |     .dataframe tbody tr th {
 854 |         vertical-align: top;
 855 |     }
 856 | 
 857 |     .dataframe thead th {
 858 |         text-align: right;
 859 |     }
 860 | </style>
 861 | <table border="1" class="dataframe">
 862 |   <thead>
 863 |     <tr style="text-align: right;">
 864 |       <th></th>
 865 |       <th>floors</th>
 866 |     </tr>
 867 |   </thead>
 868 |   <tbody>
 869 |     <tr>
 870 |       <th>1.0</th>
 871 |       <td>10680</td>
 872 |     </tr>
 873 |     <tr>
 874 |       <th>2.0</th>
 875 |       <td>8241</td>
 876 |     </tr>
 877 |     <tr>
 878 |       <th>1.5</th>
 879 |       <td>1910</td>
 880 |     </tr>
 881 |     <tr>
 882 |       <th>3.0</th>
 883 |       <td>613</td>
 884 |     </tr>
 885 |     <tr>
 886 |       <th>2.5</th>
 887 |       <td>161</td>
 888 |     </tr>
 889 |     <tr>
 890 |       <th>3.5</th>
 891 |       <td>8</td>
 892 |     </tr>
 893 |   </tbody>
 894 | </table>
 895 | </div>
 896 | 
 897 | 
 898 | 
 899 | ### Question 4
 900 | Use the function <code>boxplot</code> in the seaborn library  to  determine whether houses with a waterfront view or without a waterfront view have more price outliers .
 901 | 
 902 | 
 903 | ```python
 904 | sns.boxplot(x='waterfront', y='price', data=df)
 905 | ```
 906 | 
 907 | 
 908 | 
 909 | 
 910 |     <matplotlib.axes._subplots.AxesSubplot at 0x7f9c22dc9b70>
 911 | 
 912 | 
 913 | 
 914 | 
 915 | ![png](output_29_1.png)
 916 | 
 917 | 
 918 | ### Question 5
 919 | Use the function <code> regplot</code>  in the seaborn library  to  determine if the feature <code>sqft_above</code> is negatively or positively correlated with price.
 920 | 
 921 | 
 922 | ```python
 923 | sns.regplot(x='sqft_above', y='price', data=df)
 924 | ```
 925 | 
 926 | 
 927 | 
 928 | 
 929 |     <matplotlib.axes._subplots.AxesSubplot at 0x7f9bc0307a90>
 930 | 
 931 | 
 932 | 
 933 | 
 934 | ![png](output_31_1.png)
 935 | 
 936 | 
 937 | 
 938 | We can use the Pandas method <code>corr()</code>  to find the feature other than price that is most correlated with price.
 939 | 
 940 | 
 941 | ```python
 942 | df.corr()['price'].sort_values()
 943 | ```
 944 | 
 945 | # Module 4: Model Development
 946 | 
 947 | Import libraries 
 948 | 
 949 | 
 950 | ```python
 951 | import matplotlib.pyplot as plt
 952 | from sklearn.linear_model import LinearRegression
 953 | 
 954 | ```
 955 | 
 956 | 
 957 | We can Fit a linear regression model using the  longitude feature <code> 'long'</code> and  caculate the R^2.
 958 | 
 959 | 
 960 | ```python
 961 | X = df[['long']]
 962 | Y = df['price']
 963 | lm = LinearRegression()
 964 | lm
 965 | lm.fit(X,Y)
 966 | lm.score(X, Y)
 967 | ```
 968 | 
 969 | ### Question  6
 970 | Fit a linear regression model to predict the <code>'price'</code> using the feature 'sqft_living' then calculate the R^2. Take a screenshot of your code and the value of the R^2.
 971 | 
 972 | 
 973 | ```python
 974 | U = df[['sqft_living']]
 975 | V = df['price']
 976 | lm.fit(U,V)
 977 | lm.score(U,V)
 978 | ```
 979 | 
 980 | 
 981 | 
 982 | 
 983 |     0.49285321790379316
 984 | 
 985 | 
 986 | 
 987 | ### Question 7
 988 | Fit a linear regression model to predict the 'price' using the list of features:
 989 | 
 990 | 
 991 | ```python
 992 | features =["floors", "waterfront","lat" ,"bedrooms" ,"sqft_basement" ,"view" ,"bathrooms","sqft_living15","sqft_above","grade","sqft_living"]     
 993 | X = df[features]
 994 | Y = df['price']
 995 | lm.fit(X,Y)
 996 | ```
 997 | 
 998 | 
 999 | 
1000 | 
1001 |     LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,
1002 |              normalize=False)
1003 | 
1004 | 
1005 | 
1006 | the calculate the R^2. Take a screenshot of your code
1007 | 
1008 | 
1009 | ```python
1010 | lm.score(X,Y)
1011 | ```
1012 | 
1013 | 
1014 | 
1015 | 
1016 |     0.6576951666037504
1017 | 
1018 | 
1019 | 
1020 | #### this will help with Question 8
1021 | 
1022 | Create a list of tuples, the first element in the tuple contains the name of the estimator:
1023 | 
1024 | <code>'scale'</code>
1025 | 
1026 | <code>'polynomial'</code>
1027 | 
1028 | <code>'model'</code>
1029 | 
1030 | The second element in the tuple  contains the model constructor 
1031 | 
1032 | <code>StandardScaler()</code>
1033 | 
1034 | <code>PolynomialFeatures(include_bias=False)</code>
1035 | 
1036 | <code>LinearRegression()</code>
1037 | 
1038 | 
1039 | 
1040 | ```python
1041 | Input=[('scale',StandardScaler()),('polynomial', PolynomialFeatures(include_bias=False)),('model',LinearRegression())]
1042 | ```
1043 | 
1044 | ### Question 8
1045 | Use the list to create a pipeline object,  predict the 'price', fit the object using the features in the list <code> features </code>, then fit the model and calculate the R^2
1046 | 
1047 | 
1048 | ```python
1049 | pipe=Pipeline(Input)
1050 | pipe
1051 | ```
1052 | 
1053 | 
1054 | 
1055 | 
1056 |     Pipeline(memory=None,
1057 |          steps=[('scale', StandardScaler(copy=True, with_mean=True, with_std=True)), ('polynomial', PolynomialFeatures(degree=2, include_bias=False, interaction_only=False)), ('model', LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,
1058 |              normalize=False))])
1059 | 
1060 | 
1061 | 
1062 | 
1063 | ```python
1064 | pipe.fit(X,Y)
1065 | ```
1066 | 
1067 |     /opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by StandardScaler.
1068 |       return self.partial_fit(X, y)
1069 |     /opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/base.py:467: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by StandardScaler.
1070 |       return self.fit(X, y, **fit_params).transform(X)
1071 | 
1072 | 
1073 | 
1074 | 
1075 | 
1076 |     Pipeline(memory=None,
1077 |          steps=[('scale', StandardScaler(copy=True, with_mean=True, with_std=True)), ('polynomial', PolynomialFeatures(degree=2, include_bias=False, interaction_only=False)), ('model', LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,
1078 |              normalize=False))])
1079 | 
1080 | 
1081 | 
1082 | 
1083 | ```python
1084 | pipe.score(X,Y)
1085 | ```
1086 | 
1087 |     /opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/pipeline.py:511: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by StandardScaler.
1088 |       Xt = transform.transform(Xt)
1089 | 
1090 | 
1091 | 
1092 | 
1093 | 
1094 |     0.7513427797293394
1095 | 
1096 | 
1097 | 
1098 | # Module 5: MODEL EVALUATION AND REFINEMENT
1099 | 
1100 | import the necessary modules  
1101 | 
1102 | 
1103 | ```python
1104 | from sklearn.model_selection import cross_val_score
1105 | from sklearn.model_selection import train_test_split
1106 | print("done")
1107 | ```
1108 | 
1109 |     done
1110 | 
1111 | 
1112 | we will split the data into training and testing set
1113 | 
1114 | 
1115 | ```python
1116 | features =["floors", "waterfront","lat" ,"bedrooms" ,"sqft_basement" ,"view" ,"bathrooms","sqft_living15","sqft_above","grade","sqft_living"]    
1117 | X = df[features ]
1118 | Y = df['price']
1119 | 
1120 | x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.15, random_state=1)
1121 | 
1122 | 
1123 | print("number of test samples :", x_test.shape[0])
1124 | print("number of training samples:",x_train.shape[0])
1125 | ```
1126 | 
1127 |     number of test samples : 3242
1128 |     number of training samples: 18371
1129 | 
1130 | 
1131 | ### Question 9
1132 | Create and fit a Ridge regression object using the training data, setting the regularization parameter to 0.1 and calculate the R^2 using the test data. 
1133 | 
1134 | 
1135 | 
1136 | ```python
1137 | from sklearn.linear_model import Ridge
1138 | ```
1139 | 
1140 | 
1141 | ```python
1142 | RigeModel=Ridge(alpha=0.1)
1143 | RigeModel.fit(x_train, y_train)
1144 | RigeModel.score(x_test, y_test)
1145 | ```
1146 | 
1147 | 
1148 | 
1149 | 
1150 |     0.6478759163939111
1151 | 
1152 | 
1153 | 
1154 | ### Question 10
1155 | Perform a second order polynomial transform on both the training data and testing data. Create and fit a Ridge regression object using the training data, setting the regularisation parameter to 0.1.  Calculate the R^2 utilising the test data provided. Take a screenshot of your code and the R^2.
1156 | 
1157 | 
1158 | ```python
1159 | pr = PolynomialFeatures(degree=2)
1160 | x_train_pr = pr.fit_transform(x_train)
1161 | x_test_pr = pr.fit_transform(x_test)
1162 | 
1163 | RigeModel=Ridge(alpha=0.1)
1164 | RigeModel.fit(x_train_pr, y_train)
1165 | RigeModel.score(x_test_pr, y_test)
1166 | ```
1167 | 
1168 | 
1169 | 
1170 | 
1171 |     0.7002744268659787
1172 | 
1173 | 
1174 | 
1175 | <h2>About this Project:</h2> 
1176 | 
1177 | This project is part of a graded excercise in "Data Analysis Using Python" course on Coursera offered by IBM
1178 | 
1179 | 
1180 | 
1181 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # House-sale-price-prediction-using-python
2 | This project analyzes and predicts housing sale price based on features such as square footage, number of bedrooms, views, locations, etc. It uses the dataset of house sale prices for King County, USA, including home sales between May 2014 and May 2015.
3 | 
4 | It uses python codes to do data-cleaning, analyse data and create models for price prediction, evaluate and refine models. Major activities covered include:
5 | - Numerical representation of data using correlation, linear and polynomial regression, R-Squared values, etc
6 | - Graphical representation of data using boxplot, and seaborn's regplot.
7 | - Model refinement suing ridge regression object.
8 | - Polynomial transform of training and test data, etc.
9 | 


--------------------------------------------------------------------------------
/output_29_1.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/calistus-igwilo/House-sale-price-prediction-using-python/23d5b4735e772ff99a759b5548f76355ed63346d/output_29_1.png


--------------------------------------------------------------------------------
/output_31_1.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/calistus-igwilo/House-sale-price-prediction-using-python/23d5b4735e772ff99a759b5548f76355ed63346d/output_31_1.png


--------------------------------------------------------------------------------