├── .gitignore ├── LICENSE ├── README.md ├── composer.json ├── data_description.txt ├── dataset.csv ├── docs └── images │ ├── training-loss.png │ └── validation-score.png ├── predict.php ├── sample_submission.csv ├── train.php └── unknown.csv /.gitignore: -------------------------------------------------------------------------------- 1 | /vendor 2 | composer.lock 3 | predictions.csv 4 | progress.csv 5 | report.json 6 | *.rbx 7 | *.old 8 | .vscode 9 | .vs 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Andrew DalPino 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Rubix ML - Housing Price Predictor 2 | An example Rubix ML project that predicts house prices using a Gradient Boosted Machine (GBM) and a popular dataset from a [Kaggle competition](https://www.kaggle.com/c/house-prices-advanced-regression-techniques). In this tutorial, you'll learn about regression and the stage-wise additive boosting ensemble called [Gradient Boost](https://docs.rubixml.com/latest/regressors/gradient-boost.html). By the end of the tutorial, you'll be able to submit your own predictions to the Kaggle competition. 3 | 4 | - **Difficulty:** Medium 5 | - **Training time:** Minutes 6 | 7 | From Kaggle: 8 | 9 | > Ask a home buyer to describe their dream house, and they probably won't begin with the height of the basement ceiling or the proximity to an east-west railroad. But this playground competition's dataset proves that much more influences price negotiations than the number of bedrooms or a white-picket fence. 10 | > 11 | > With 79 explanatory variables describing (almost) every aspect of residential homes in Ames, Iowa, this competition challenges you to predict the final price of each home. 12 | 13 | ## Installation 14 | Clone the project locally using [Composer](https://getcomposer.org/): 15 | ```sh 16 | $ composer create-project rubix/housing 17 | ``` 18 | 19 | ## Requirements 20 | - [PHP](https://php.net) 7.4 or above 21 | 22 | #### Recommended 23 | - [Tensor extension](https://github.com/RubixML/Tensor) for faster training and inference 24 | - 1G of system memory or more 25 | 26 | ## Tutorial 27 | 28 | ### Introduction 29 | [Kaggle](https://www.kaggle.com) is a platform that allows you to test your data science skills by engaging with contests. This tutorial is designed to walk you through a regression problem in Rubix ML using the Kaggle housing prices challenge as an example. We are given a training set consisting of 1,460 labeled samples that we'll use to train the learner and 1,459 unlabeled samples for making predictions. Each sample contains a heterogeneous mix of categorical and continuous data types. Our goal is to build an estimator that correctly predicts the sale price of a house. We'll choose [Gradient Boost](https://docs.rubixml.com/latest/regressors/gradient-boost.html) as our learner since it offers good performance and is capable of handling both categorical and continuous features. 30 | 31 | > **Note:** The source code for this example can be found in the [train.php](https://github.com/RubixML/Housing/blob/master/train.php) file in project root. 32 | 33 | ### Extracting the Data 34 | The data are given to us in two separate CSV files - `dataset.csv` which has labels for training and `unknown.csv` without labels for predicting. Each feature column is denoted by a title in the CSV header which we'll use to identify the column with our [Column Picker](https://docs.rubixml.com/latest/extractors/column-picker.html). Column Picker allows us to select and rearrange the columns of a data table while the data is in flight. It wraps another iterator object such as the built-in [CSV](https://docs.rubixml.com/latest/extractors/csv.html) extractor. In this case, we don't need the `Id` column of the dataset because it is uncorrelated with the outcome so we'll only specify the columns we need. When instantiating a new [Labeled](https://docs.rubixml.com/latest/datasets/labeled.html) dataset object via the `fromIterator()` method, the last column of the data table is taken to be the labels. 35 | 36 | ```php 37 | use Rubix\ML\Datasets\Labeled; 38 | use Rubix\ML\Extractors\CSV; 39 | use Rubix\ML\Extractors\ColumnPicker; 40 | 41 | $extractor = new ColumnPicker(new CSV('dataset.csv', true), [ 42 | 'MSSubClass', 'MSZoning', 'LotFrontage', 'LotArea', 'Street', 'Alley', 43 | 'LotShape', 'LandContour', 'Utilities', 'LotConfig', 'LandSlope', 44 | 'Neighborhood', 'Condition1', 'Condition2', 'BldgType', 'HouseStyle', 45 | 'OverallQual', 'OverallCond', 'YearBuilt', 'YearRemodAdd', 'RoofStyle', 46 | 'RoofMatl', 'Exterior1st', 'Exterior2nd', 'MasVnrType', 'MasVnrArea', 47 | 'ExterQual', 'ExterCond', 'Foundation', 'BsmtQual', 'BsmtCond', 48 | 'BsmtExposure', 'BsmtFinType1', 'BsmtFinSF1', 'BsmtFinType2', 'BsmtFinSF2', 49 | 'BsmtUnfSF', 'TotalBsmtSF', 'Heating', 'HeatingQC', 'CentralAir', 50 | 'Electrical', '1stFlrSF', '2ndFlrSF', 'LowQualFinSF', 'GrLivArea', 51 | 'BsmtFullBath', 'BsmtHalfBath', 'FullBath', 'HalfBath', 'BedroomAbvGr', 52 | 'KitchenAbvGr', 'KitchenQual', 'TotRmsAbvGrd', 'Functional', 'Fireplaces', 53 | 'FireplaceQu', 'GarageType', 'GarageYrBlt', 'GarageFinish', 'GarageCars', 54 | 'GarageArea', 'GarageQual', 'GarageCond', 'PavedDrive', 'WoodDeckSF', 55 | 'OpenPorchSF', 'EnclosedPorch', '3SsnPorch', 'ScreenPorch', 'PoolArea', 56 | 'PoolQC', 'Fence', 'MiscFeature', 'MiscVal', 'MoSold', 'YrSold', 57 | 'SaleType', 'SaleCondition', 'SalePrice', 58 | ]); 59 | 60 | $dataset = Labeled::fromIterator($extractor); 61 | ``` 62 | 63 | ### Dataset Preparation 64 | Next we'll apply a series of transformations to the training set to prepare it for the learner. By default, the CSV Reader imports everything as a string - therefore, we must convert the numerical values to integers and floating point numbers beforehand so they can be recognized by the learner as continuous features. The [Numeric String Converter](https://docs.rubixml.com/latest/transformers/numeric-string-converter.html) will handle this for us. Since some feature columns contain missing data, we'll also apply the [Missing Data Imputer](https://docs.rubixml.com/latest/transformers/missing-data-imputer.html) which replaces missing values with a pretty good guess. Lastly, since the labels should also be continuous, we'll apply a separate transformation to the labels using a standard PHP function `intval()` callback which converts values to integers. 65 | 66 | ```php 67 | use Rubix\ML\Transformers\NumericStringConverter; 68 | use Rubix\ML\Transformers\MissingDataImputer; 69 | 70 | $dataset->apply(new NumericStringConverter()) 71 | ->apply(new MissingDataImputer()) 72 | ->transformLabels('intval'); 73 | ``` 74 | 75 | ### Instantiating the Learner 76 | A Gradient Boosted Machine (GBM) is a type of ensemble estimator that uses [Regression Trees](https://docs.rubixml.com/latest/regressors/regression-tree.html) to fix up the errors of a *weak* base learner. It does so in an iterative process that involves training a new Regression Tree (called a *booster*) on the error residuals of the predictions given by the previous estimator. Thus, GBM produces an additive model whose predictions become more refined as the number of boosters are added. The coordination of multiple estimators to act as a single estimator is called *ensemble* learning. 77 | 78 | Next, we'll create the estimator instance by instantiating [Gradient Boost](https://docs.rubixml.com/latest/regressors/gradient-boost.html) and wrapping it in a [Persistent Model](https://docs.rubixml.com/latest/persistent-model.html) meta-estimator so we can save it to make predictions later in another process. 79 | 80 | ```php 81 | use Rubix\ML\PersistentModel; 82 | use Rubix\ML\Regressors\GradientBoost; 83 | use Rubix\ML\Regressors\RegressionTree; 84 | use Rubix\ML\Persisters\Filesystem; 85 | 86 | $estimator = new PersistentModel( 87 | new GradientBoost(new RegressionTree(4), 0.1), 88 | new Filesystem('housing.rbx', true) 89 | ); 90 | ``` 91 | 92 | The first two hyper-parameters of Gradient Boost are the booster's settings and the learning rate, respectively. For this example, we'll use a standard Regression Tree with a maximum depth of 4 as the booster and a learning rate of 0.1 but feel free to play with these settings on your own. 93 | 94 | The Persistent Model meta-estimator constructor takes the GBM instance as its first argument and a Persister object as the second. The [Filesystem](https://docs.rubixml.com/latest/persisters/filesystem.html) persister is responsible for storing and loading the model on disk and takes the path of the model file as an argument. In addition, we'll tell the persister to keep a copy of every saved model by setting history mode to true. 95 | 96 | ### Setting a Logger 97 | Since Gradient Boost implements the [Verbose](https://docs.rubixml.com/latest/verbose.html) interface, we can monitor its progress during training by setting a logger instance on the learner. The built-in [Screen](https://docs.rubixml.com/latest/other/loggers/screen.html) logger will work well for most cases, but you can use any PSR-3 compatible logger including [Monolog](https://github.com/Seldaek/monolog). 98 | 99 | ```php 100 | use Rubix\ML\Other\Loggers\Screen; 101 | 102 | $estimator->setLogger(new Screen()); 103 | ``` 104 | 105 | ### Training 106 | Now, we're ready to train the learner by calling the `train()` method with the training dataset as an argument. 107 | 108 | ```php 109 | $estimator->train($dataset); 110 | ``` 111 | 112 | ### Validation Score and Loss 113 | During training, the learner will record the validation score and the training loss at each iteration or *epoch*. The validation score is calculated using the default [RMSE](https://docs.rubixml.com/latest/cross-validation/metrics/rmse.html) metric on a hold out portion of the training set. Contrariwise, the training loss is the value of the cost function (in this case the L2 or *quadratic* loss) computed over the training data. We can visualize the training progress by plotting these metrics. To output the scores and losses you can call the additional `steps()` method on the learner instance. Then we can export the data to a CSV file by exporting the iterator returned by `steps()` to a CSV file. 114 | 115 | ```php 116 | use Rubix\ML\Extractors\CSV; 117 | 118 | $extractor = new CSV('progress.csv', true); 119 | 120 | $extractor->export($estimator->steps()); 121 | ``` 122 | 123 | Here is an example of what the validation score and training loss look like when plotted. You can plot the values yourself by importing the `progress.csv` file into your favorite plotting software. 124 | 125 | ![R Squared Score](https://raw.githubusercontent.com/RubixML/Housing/master/docs/images/validation-score.png) 126 | 127 | ![L2 Loss](https://raw.githubusercontent.com/RubixML/Housing/master/docs/images/training-loss.png) 128 | 129 | 130 | ### Saving 131 | Lastly, save the model so it can be used later to predict the house prices of the unknown samples. 132 | 133 | ```php 134 | $estimator->save(); 135 | ``` 136 | 137 | Now we're ready to execute the training script by calling it from the command line. 138 | ```sh 139 | $ php train.php 140 | ``` 141 | 142 | ### Inference 143 | The goal of the Kaggle contest is to predict the correct sale prices of each house given a list of unknown samples. If all went well during training, we should be able to achieve good results with just this basic example. We'll start by importing the unlabeled samples from the `unknown.csv` file. 144 | 145 | > **Note:** The source code for this example can be found in the [predict.php](https://github.com/RubixML/Housing/blob/master/predict.php) file in the project root. 146 | 147 | ```php 148 | use Rubix\ML\Datasets\Unlabeled; 149 | use Rubix\ML\Extractors\CSV; 150 | use Rubix\ML\Transformers\NumericStringConverter; 151 | 152 | $dataset = Unlabeled::fromIterator(new CSV('unknown.csv', true)) 153 | ->apply(new NumericStringConverter()); 154 | ``` 155 | 156 | ### Load Model from Storage 157 | Now, let's load the persisted Gradient Boost estimator into our script using the static `load()` method on the Persistent Model class by passing it a [Persister](https://docs.rubixml.com/latest/persisters/api.html) instance pointing to the model in storage. 158 | 159 | ```php 160 | use Rubix\ML\PersistentModel; 161 | use Rubix\ML\Persisters\Filesystem; 162 | 163 | $estimator = PersistentModel::load(new Filesystem('housing.model')); 164 | ``` 165 | 166 | ### Make Predictions 167 | To obtain the predictions from the model, call the `predict()` method with the dataset containing the unknown samples. 168 | 169 | ```php 170 | $predictions = $estimator->predict($dataset); 171 | ``` 172 | 173 | Then we'll use the CSV extractor to export the IDs and predictions to a file that we'll submit to the competition. 174 | 175 | ```php 176 | use Rubix\ML\Extractors\ColumnPicker; 177 | use Rubix\ML\Extractors\CSV; 178 | 179 | $extractor = new ColumnPicker(new CSV('dataset.csv', true), ['Id']); 180 | 181 | $ids = array_column(iterator_to_array($extractor), 'Id'); 182 | 183 | array_unshift($ids, 'Id'); 184 | array_unshift($predictions, 'SalePrice'); 185 | 186 | $extractor = new CSV('predictions.csv'); 187 | 188 | $extractor->export(array_transpose([$ids, $predictions])); 189 | ``` 190 | 191 | Now run the prediction script by calling it from the command line. 192 | ```sh 193 | $ php predict.php 194 | ``` 195 | 196 | Nice work! Now you can submit your predictions with their IDs to the [contest page](https://www.kaggle.com/c/house-prices-advanced-regression-techniques) to see how well you did. 197 | 198 | ### Wrap Up 199 | 200 | - Regressors are a type of estimator that predict continuous valued outcomes such as house prices. 201 | - Ensemble learning combines the predictions of multiple estimators into one. 202 | - [Gradient Boost](https://docs.rubixml.com/latest/regressors/gradient-boost.html) is an ensemble learner that uses Regression Trees to fix up the errors of a *weak* base estimator. 203 | - Gradient Boost can handle both categorical and continuous data types at the same time by default. 204 | - Data science competitions are a great way to practice your machine learning skills. 205 | 206 | ### Next Steps 207 | Have a look at the [Gradient Boost](https://docs.rubixml.com/latest/regressors/gradient-boost.html) documentation page to get a better sense of what the learner can do. Try tuning the hyper-parameters for better results. Consider filtering out noise samples from the dataset by using methods on the dataset object. For example, you may want to remove extremely large and expensive houses from the training set. 208 | 209 | ### References 210 | >- D. De Cock. (2011). Ames, Iowa: Alternative to the Boston Housing Data as an End of Semester Regression Project. Journal of Statistics Education, Volume 19, Number 3. 211 | 212 | ## License 213 | The code is licensed [MIT](LICENSE.md) and the tutorial is licensed [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). 214 | -------------------------------------------------------------------------------- /composer.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "rubix/housing", 3 | "type": "project", 4 | "description": "An example project that predicts house prices for a Kaggle competition using a Gradient Boosted Machine.", 5 | "homepage": "https://github.com/RubixML/Housing", 6 | "license": "MIT", 7 | "keywords": [ 8 | "dataset", "data science", "ensemble", "example project", "gbm", "gradient boost", 9 | "gradient boosted machine", "imputation", "machine learning", "ml", "php", "php ml", 10 | "price prediction", "regression", "regressor", "regression tree", "rubix ml", "rubixml", 11 | "tutorial" 12 | ], 13 | "authors": [ 14 | { 15 | "name": "Andrew DalPino", 16 | "homepage": "https://github.com/andrewdalpino", 17 | "role": "Lead Engineer" 18 | } 19 | ], 20 | "require": { 21 | "php": ">=7.4", 22 | "rubix/ml": "^2.0" 23 | }, 24 | "scripts": { 25 | "predict": "@php predict.php", 26 | "train": "@php train.php" 27 | }, 28 | "config": { 29 | "preferred-install": "dist", 30 | "sort-packages": true 31 | }, 32 | "minimum-stability": "dev", 33 | "prefer-stable": true 34 | } 35 | -------------------------------------------------------------------------------- /data_description.txt: -------------------------------------------------------------------------------- 1 | MSSubClass: Identifies the type of dwelling involved in the sale. 2 | 3 | 20 1-STORY 1946 & NEWER ALL STYLES 4 | 30 1-STORY 1945 & OLDER 5 | 40 1-STORY W/FINISHED ATTIC ALL AGES 6 | 45 1-1/2 STORY - UNFINISHED ALL AGES 7 | 50 1-1/2 STORY FINISHED ALL AGES 8 | 60 2-STORY 1946 & NEWER 9 | 70 2-STORY 1945 & OLDER 10 | 75 2-1/2 STORY ALL AGES 11 | 80 SPLIT OR MULTI-LEVEL 12 | 85 SPLIT FOYER 13 | 90 DUPLEX - ALL STYLES AND AGES 14 | 120 1-STORY PUD (Planned Unit Development) - 1946 & NEWER 15 | 150 1-1/2 STORY PUD - ALL AGES 16 | 160 2-STORY PUD - 1946 & NEWER 17 | 180 PUD - MULTILEVEL - INCL SPLIT LEV/FOYER 18 | 190 2 FAMILY CONVERSION - ALL STYLES AND AGES 19 | 20 | MSZoning: Identifies the general zoning classification of the sale. 21 | 22 | A Agriculture 23 | C Commercial 24 | FV Floating Village Residential 25 | I Industrial 26 | RH Residential High Density 27 | RL Residential Low Density 28 | RP Residential Low Density Park 29 | RM Residential Medium Density 30 | 31 | LotFrontage: Linear feet of street connected to property 32 | 33 | LotArea: Lot size in square feet 34 | 35 | Street: Type of road access to property 36 | 37 | Grvl Gravel 38 | Pave Paved 39 | 40 | Alley: Type of alley access to property 41 | 42 | Grvl Gravel 43 | Pave Paved 44 | NA No alley access 45 | 46 | LotShape: General shape of property 47 | 48 | Reg Regular 49 | IR1 Slightly irregular 50 | IR2 Moderately Irregular 51 | IR3 Irregular 52 | 53 | LandContour: Flatness of the property 54 | 55 | Lvl Near Flat/Level 56 | Bnk Banked - Quick and significant rise from street grade to building 57 | HLS Hillside - Significant slope from side to side 58 | Low Depression 59 | 60 | Utilities: Type of utilities available 61 | 62 | AllPub All public Utilities (E,G,W,& S) 63 | NoSewr Electricity, Gas, and Water (Septic Tank) 64 | NoSeWa Electricity and Gas Only 65 | ELO Electricity only 66 | 67 | LotConfig: Lot configuration 68 | 69 | Inside Inside lot 70 | Corner Corner lot 71 | CulDSac Cul-de-sac 72 | FR2 Frontage on 2 sides of property 73 | FR3 Frontage on 3 sides of property 74 | 75 | LandSlope: Slope of property 76 | 77 | Gtl Gentle slope 78 | Mod Moderate Slope 79 | Sev Severe Slope 80 | 81 | Neighborhood: Physical locations within Ames city limits 82 | 83 | Blmngtn Bloomington Heights 84 | Blueste Bluestem 85 | BrDale Briardale 86 | BrkSide Brookside 87 | ClearCr Clear Creek 88 | CollgCr College Creek 89 | Crawfor Crawford 90 | Edwards Edwards 91 | Gilbert Gilbert 92 | IDOTRR Iowa DOT and Rail Road 93 | MeadowV Meadow Village 94 | Mitchel Mitchell 95 | Names North Ames 96 | NoRidge Northridge 97 | NPkVill Northpark Villa 98 | NridgHt Northridge Heights 99 | NWAmes Northwest Ames 100 | OldTown Old Town 101 | SWISU South & West of Iowa State University 102 | Sawyer Sawyer 103 | SawyerW Sawyer West 104 | Somerst Somerset 105 | StoneBr Stone Brook 106 | Timber Timberland 107 | Veenker Veenker 108 | 109 | Condition1: Proximity to various conditions 110 | 111 | Artery Adjacent to arterial street 112 | Feedr Adjacent to feeder street 113 | Norm Normal 114 | RRNn Within 200' of North-South Railroad 115 | RRAn Adjacent to North-South Railroad 116 | PosN Near positive off-site feature--park, greenbelt, etc. 117 | PosA Adjacent to postive off-site feature 118 | RRNe Within 200' of East-West Railroad 119 | RRAe Adjacent to East-West Railroad 120 | 121 | Condition2: Proximity to various conditions (if more than one is present) 122 | 123 | Artery Adjacent to arterial street 124 | Feedr Adjacent to feeder street 125 | Norm Normal 126 | RRNn Within 200' of North-South Railroad 127 | RRAn Adjacent to North-South Railroad 128 | PosN Near positive off-site feature--park, greenbelt, etc. 129 | PosA Adjacent to postive off-site feature 130 | RRNe Within 200' of East-West Railroad 131 | RRAe Adjacent to East-West Railroad 132 | 133 | BldgType: Type of dwelling 134 | 135 | 1Fam Single-family Detached 136 | 2FmCon Two-family Conversion; originally built as one-family dwelling 137 | Duplx Duplex 138 | TwnhsE Townhouse End Unit 139 | TwnhsI Townhouse Inside Unit 140 | 141 | HouseStyle: Style of dwelling 142 | 143 | 1Story One story 144 | 1.5Fin One and one-half story: 2nd level finished 145 | 1.5Unf One and one-half story: 2nd level unfinished 146 | 2Story Two story 147 | 2.5Fin Two and one-half story: 2nd level finished 148 | 2.5Unf Two and one-half story: 2nd level unfinished 149 | SFoyer Split Foyer 150 | SLvl Split Level 151 | 152 | OverallQual: Rates the overall material and finish of the house 153 | 154 | 10 Very Excellent 155 | 9 Excellent 156 | 8 Very Good 157 | 7 Good 158 | 6 Above Average 159 | 5 Average 160 | 4 Below Average 161 | 3 Fair 162 | 2 Poor 163 | 1 Very Poor 164 | 165 | OverallCond: Rates the overall condition of the house 166 | 167 | 10 Very Excellent 168 | 9 Excellent 169 | 8 Very Good 170 | 7 Good 171 | 6 Above Average 172 | 5 Average 173 | 4 Below Average 174 | 3 Fair 175 | 2 Poor 176 | 1 Very Poor 177 | 178 | YearBuilt: Original construction date 179 | 180 | YearRemodAdd: Remodel date (same as construction date if no remodeling or additions) 181 | 182 | RoofStyle: Type of roof 183 | 184 | Flat Flat 185 | Gable Gable 186 | Gambrel Gabrel (Barn) 187 | Hip Hip 188 | Mansard Mansard 189 | Shed Shed 190 | 191 | RoofMatl: Roof material 192 | 193 | ClyTile Clay or Tile 194 | CompShg Standard (Composite) Shingle 195 | Membran Membrane 196 | Metal Metal 197 | Roll Roll 198 | Tar&Grv Gravel & Tar 199 | WdShake Wood Shakes 200 | WdShngl Wood Shingles 201 | 202 | Exterior1st: Exterior covering on house 203 | 204 | AsbShng Asbestos Shingles 205 | AsphShn Asphalt Shingles 206 | BrkComm Brick Common 207 | BrkFace Brick Face 208 | CBlock Cinder Block 209 | CemntBd Cement Board 210 | HdBoard Hard Board 211 | ImStucc Imitation Stucco 212 | MetalSd Metal Siding 213 | Other Other 214 | Plywood Plywood 215 | PreCast PreCast 216 | Stone Stone 217 | Stucco Stucco 218 | VinylSd Vinyl Siding 219 | Wd Sdng Wood Siding 220 | WdShing Wood Shingles 221 | 222 | Exterior2nd: Exterior covering on house (if more than one material) 223 | 224 | AsbShng Asbestos Shingles 225 | AsphShn Asphalt Shingles 226 | BrkComm Brick Common 227 | BrkFace Brick Face 228 | CBlock Cinder Block 229 | CemntBd Cement Board 230 | HdBoard Hard Board 231 | ImStucc Imitation Stucco 232 | MetalSd Metal Siding 233 | Other Other 234 | Plywood Plywood 235 | PreCast PreCast 236 | Stone Stone 237 | Stucco Stucco 238 | VinylSd Vinyl Siding 239 | Wd Sdng Wood Siding 240 | WdShing Wood Shingles 241 | 242 | MasVnrType: Masonry veneer type 243 | 244 | BrkCmn Brick Common 245 | BrkFace Brick Face 246 | CBlock Cinder Block 247 | None None 248 | Stone Stone 249 | 250 | MasVnrArea: Masonry veneer area in square feet 251 | 252 | ExterQual: Evaluates the quality of the material on the exterior 253 | 254 | Ex Excellent 255 | Gd Good 256 | TA Average/Typical 257 | Fa Fair 258 | Po Poor 259 | 260 | ExterCond: Evaluates the present condition of the material on the exterior 261 | 262 | Ex Excellent 263 | Gd Good 264 | TA Average/Typical 265 | Fa Fair 266 | Po Poor 267 | 268 | Foundation: Type of foundation 269 | 270 | BrkTil Brick & Tile 271 | CBlock Cinder Block 272 | PConc Poured Contrete 273 | Slab Slab 274 | Stone Stone 275 | Wood Wood 276 | 277 | BsmtQual: Evaluates the height of the basement 278 | 279 | Ex Excellent (100+ inches) 280 | Gd Good (90-99 inches) 281 | TA Typical (80-89 inches) 282 | Fa Fair (70-79 inches) 283 | Po Poor (<70 inches 284 | NA No Basement 285 | 286 | BsmtCond: Evaluates the general condition of the basement 287 | 288 | Ex Excellent 289 | Gd Good 290 | TA Typical - slight dampness allowed 291 | Fa Fair - dampness or some cracking or settling 292 | Po Poor - Severe cracking, settling, or wetness 293 | NA No Basement 294 | 295 | BsmtExposure: Refers to walkout or garden level walls 296 | 297 | Gd Good Exposure 298 | Av Average Exposure (split levels or foyers typically score average or above) 299 | Mn Mimimum Exposure 300 | No No Exposure 301 | NA No Basement 302 | 303 | BsmtFinType1: Rating of basement finished area 304 | 305 | GLQ Good Living Quarters 306 | ALQ Average Living Quarters 307 | BLQ Below Average Living Quarters 308 | Rec Average Rec Room 309 | LwQ Low Quality 310 | Unf Unfinshed 311 | NA No Basement 312 | 313 | BsmtFinSF1: Type 1 finished square feet 314 | 315 | BsmtFinType2: Rating of basement finished area (if multiple types) 316 | 317 | GLQ Good Living Quarters 318 | ALQ Average Living Quarters 319 | BLQ Below Average Living Quarters 320 | Rec Average Rec Room 321 | LwQ Low Quality 322 | Unf Unfinshed 323 | NA No Basement 324 | 325 | BsmtFinSF2: Type 2 finished square feet 326 | 327 | BsmtUnfSF: Unfinished square feet of basement area 328 | 329 | TotalBsmtSF: Total square feet of basement area 330 | 331 | Heating: Type of heating 332 | 333 | Floor Floor Furnace 334 | GasA Gas forced warm air furnace 335 | GasW Gas hot water or steam heat 336 | Grav Gravity furnace 337 | OthW Hot water or steam heat other than gas 338 | Wall Wall furnace 339 | 340 | HeatingQC: Heating quality and condition 341 | 342 | Ex Excellent 343 | Gd Good 344 | TA Average/Typical 345 | Fa Fair 346 | Po Poor 347 | 348 | CentralAir: Central air conditioning 349 | 350 | N No 351 | Y Yes 352 | 353 | Electrical: Electrical system 354 | 355 | SBrkr Standard Circuit Breakers & Romex 356 | FuseA Fuse Box over 60 AMP and all Romex wiring (Average) 357 | FuseF 60 AMP Fuse Box and mostly Romex wiring (Fair) 358 | FuseP 60 AMP Fuse Box and mostly knob & tube wiring (poor) 359 | Mix Mixed 360 | 361 | 1stFlrSF: First Floor square feet 362 | 363 | 2ndFlrSF: Second floor square feet 364 | 365 | LowQualFinSF: Low quality finished square feet (all floors) 366 | 367 | GrLivArea: Above grade (ground) living area square feet 368 | 369 | BsmtFullBath: Basement full bathrooms 370 | 371 | BsmtHalfBath: Basement half bathrooms 372 | 373 | FullBath: Full bathrooms above grade 374 | 375 | HalfBath: Half baths above grade 376 | 377 | Bedroom: Bedrooms above grade (does NOT include basement bedrooms) 378 | 379 | Kitchen: Kitchens above grade 380 | 381 | KitchenQual: Kitchen quality 382 | 383 | Ex Excellent 384 | Gd Good 385 | TA Typical/Average 386 | Fa Fair 387 | Po Poor 388 | 389 | TotRmsAbvGrd: Total rooms above grade (does not include bathrooms) 390 | 391 | Functional: Home functionality (Assume typical unless deductions are warranted) 392 | 393 | Typ Typical Functionality 394 | Min1 Minor Deductions 1 395 | Min2 Minor Deductions 2 396 | Mod Moderate Deductions 397 | Maj1 Major Deductions 1 398 | Maj2 Major Deductions 2 399 | Sev Severely Damaged 400 | Sal Salvage only 401 | 402 | Fireplaces: Number of fireplaces 403 | 404 | FireplaceQu: Fireplace quality 405 | 406 | Ex Excellent - Exceptional Masonry Fireplace 407 | Gd Good - Masonry Fireplace in main level 408 | TA Average - Prefabricated Fireplace in main living area or Masonry Fireplace in basement 409 | Fa Fair - Prefabricated Fireplace in basement 410 | Po Poor - Ben Franklin Stove 411 | NA No Fireplace 412 | 413 | GarageType: Garage location 414 | 415 | 2Types More than one type of garage 416 | Attchd Attached to home 417 | Basment Basement Garage 418 | BuiltIn Built-In (Garage part of house - typically has room above garage) 419 | CarPort Car Port 420 | Detchd Detached from home 421 | NA No Garage 422 | 423 | GarageYrBlt: Year garage was built 424 | 425 | GarageFinish: Interior finish of the garage 426 | 427 | Fin Finished 428 | RFn Rough Finished 429 | Unf Unfinished 430 | NA No Garage 431 | 432 | GarageCars: Size of garage in car capacity 433 | 434 | GarageArea: Size of garage in square feet 435 | 436 | GarageQual: Garage quality 437 | 438 | Ex Excellent 439 | Gd Good 440 | TA Typical/Average 441 | Fa Fair 442 | Po Poor 443 | NA No Garage 444 | 445 | GarageCond: Garage condition 446 | 447 | Ex Excellent 448 | Gd Good 449 | TA Typical/Average 450 | Fa Fair 451 | Po Poor 452 | NA No Garage 453 | 454 | PavedDrive: Paved driveway 455 | 456 | Y Paved 457 | P Partial Pavement 458 | N Dirt/Gravel 459 | 460 | WoodDeckSF: Wood deck area in square feet 461 | 462 | OpenPorchSF: Open porch area in square feet 463 | 464 | EnclosedPorch: Enclosed porch area in square feet 465 | 466 | 3SsnPorch: Three season porch area in square feet 467 | 468 | ScreenPorch: Screen porch area in square feet 469 | 470 | PoolArea: Pool area in square feet 471 | 472 | PoolQC: Pool quality 473 | 474 | Ex Excellent 475 | Gd Good 476 | TA Average/Typical 477 | Fa Fair 478 | NA No Pool 479 | 480 | Fence: Fence quality 481 | 482 | GdPrv Good Privacy 483 | MnPrv Minimum Privacy 484 | GdWo Good Wood 485 | MnWw Minimum Wood/Wire 486 | NA No Fence 487 | 488 | MiscFeature: Miscellaneous feature not covered in other categories 489 | 490 | Elev Elevator 491 | Gar2 2nd Garage (if not described in garage section) 492 | Othr Other 493 | Shed Shed (over 100 SF) 494 | TenC Tennis Court 495 | NA None 496 | 497 | MiscVal: $Value of miscellaneous feature 498 | 499 | MoSold: Month Sold (MM) 500 | 501 | YrSold: Year Sold (YYYY) 502 | 503 | SaleType: Type of sale 504 | 505 | WD Warranty Deed - Conventional 506 | CWD Warranty Deed - Cash 507 | VWD Warranty Deed - VA Loan 508 | New Home just constructed and sold 509 | COD Court Officer Deed/Estate 510 | Con Contract 15% Down payment regular terms 511 | ConLw Contract Low Down payment and low interest 512 | ConLI Contract Low Interest 513 | ConLD Contract Low Down 514 | Oth Other 515 | 516 | SaleCondition: Condition of sale 517 | 518 | Normal Normal Sale 519 | Abnorml Abnormal Sale - trade, foreclosure, short sale 520 | AdjLand Adjoining Land Purchase 521 | Alloca Allocation - two linked properties with separate deeds, typically condo with a garage unit 522 | Family Sale between family members 523 | Partial Home was not completed when last assessed (associated with New Homes) 524 | -------------------------------------------------------------------------------- /docs/images/training-loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RubixML/Housing/e5c8cf636b7dd103e5291f7701be39ee731d6a7d/docs/images/training-loss.png -------------------------------------------------------------------------------- /docs/images/validation-score.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RubixML/Housing/e5c8cf636b7dd103e5291f7701be39ee731d6a7d/docs/images/validation-score.png -------------------------------------------------------------------------------- /predict.php: -------------------------------------------------------------------------------- 1 | info('Loading data into memory'); 20 | 21 | $extractor = new ColumnPicker(new CSV('dataset.csv', true), [ 22 | 'MSSubClass', 'MSZoning', 'LotFrontage', 'LotArea', 'Street', 'Alley', 23 | 'LotShape', 'LandContour', 'Utilities', 'LotConfig', 'LandSlope', 24 | 'Neighborhood', 'Condition1', 'Condition2', 'BldgType', 'HouseStyle', 25 | 'OverallQual', 'OverallCond', 'YearBuilt', 'YearRemodAdd', 'RoofStyle', 26 | 'RoofMatl', 'Exterior1st', 'Exterior2nd', 'MasVnrType', 'MasVnrArea', 27 | 'ExterQual', 'ExterCond', 'Foundation', 'BsmtQual', 'BsmtCond', 28 | 'BsmtExposure', 'BsmtFinType1', 'BsmtFinSF1', 'BsmtFinType2', 'BsmtFinSF2', 29 | 'BsmtUnfSF', 'TotalBsmtSF', 'Heating', 'HeatingQC', 'CentralAir', 30 | 'Electrical', '1stFlrSF', '2ndFlrSF', 'LowQualFinSF', 'GrLivArea', 31 | 'BsmtFullBath', 'BsmtHalfBath', 'FullBath', 'HalfBath', 'BedroomAbvGr', 32 | 'KitchenAbvGr', 'KitchenQual', 'TotRmsAbvGrd', 'Functional', 'Fireplaces', 33 | 'FireplaceQu', 'GarageType', 'GarageYrBlt', 'GarageFinish', 'GarageCars', 34 | 'GarageArea', 'GarageQual', 'GarageCond', 'PavedDrive', 'WoodDeckSF', 35 | 'OpenPorchSF', 'EnclosedPorch', '3SsnPorch', 'ScreenPorch', 'PoolArea', 36 | 'PoolQC', 'Fence', 'MiscFeature', 'MiscVal', 'MoSold', 'YrSold', 37 | 'SaleType', 'SaleCondition', 38 | ]); 39 | 40 | $dataset = Unlabeled::fromIterator($extractor) 41 | ->apply(new NumericStringConverter()); 42 | 43 | $estimator = PersistentModel::load(new Filesystem('housing.rbx')); 44 | 45 | $logger->info('Making predictions'); 46 | 47 | $predictions = $estimator->predict($dataset); 48 | 49 | $extractor = new ColumnPicker(new CSV('dataset.csv', true), ['Id']); 50 | 51 | $ids = array_column(iterator_to_array($extractor), 'Id'); 52 | 53 | array_unshift($ids, 'Id'); 54 | array_unshift($predictions, 'SalePrice'); 55 | 56 | $extractor = new CSV('predictions.csv'); 57 | 58 | $extractor->export(array_transpose([$ids, $predictions])); 59 | 60 | $logger->info('Predictions saved to predictions.csv'); 61 | -------------------------------------------------------------------------------- /sample_submission.csv: -------------------------------------------------------------------------------- 1 | Id,SalePrice 2 | 1461,169277.0524984 3 | 1462,187758.393988768 4 | 1463,183583.683569555 5 | 1464,179317.47751083 6 | 1465,150730.079976501 7 | 1466,177150.989247307 8 | 1467,172070.659229164 9 | 1468,175110.956519547 10 | 1469,162011.698831665 11 | 1470,160726.247831419 12 | 1471,157933.279456005 13 | 1472,145291.245020389 14 | 1473,159672.017631819 15 | 1474,164167.518301885 16 | 1475,150891.638244053 17 | 1476,179460.96518734 18 | 1477,185034.62891405 19 | 1478,182352.192644656 20 | 1479,183053.458213802 21 | 1480,187823.339254278 22 | 1481,186544.114327568 23 | 1482,158230.77520516 24 | 1483,190552.829321091 25 | 1484,147183.67487199 26 | 1485,185855.300905493 27 | 1486,174350.470676986 28 | 1487,201740.620690863 29 | 1488,162986.378895754 30 | 1489,162330.199085679 31 | 1490,165845.938616539 32 | 1491,180929.622876974 33 | 1492,163481.501519718 34 | 1493,187798.076714233 35 | 1494,198822.198942566 36 | 1495,194868.409899858 37 | 1496,152605.298564403 38 | 1497,147797.702836811 39 | 1498,150521.96899297 40 | 1499,146991.630153739 41 | 1500,150306.307814534 42 | 1501,151164.372534604 43 | 1502,151133.706960953 44 | 1503,156214.042540726 45 | 1504,171992.760735142 46 | 1505,173214.912549738 47 | 1506,192429.187345783 48 | 1507,190878.69508543 49 | 1508,194542.544135519 50 | 1509,191849.439072822 51 | 1510,176363.773907793 52 | 1511,176954.185412429 53 | 1512,176521.216975696 54 | 1513,179436.704810176 55 | 1514,220079.756777048 56 | 1515,175502.918109444 57 | 1516,188321.073833569 58 | 1517,163276.324450004 59 | 1518,185911.366293097 60 | 1519,171392.830997252 61 | 1520,174418.207020775 62 | 1521,179682.709603774 63 | 1522,179423.751581665 64 | 1523,171756.918091777 65 | 1524,166849.638174419 66 | 1525,181122.168676666 67 | 1526,170934.462746566 68 | 1527,159738.292580329 69 | 1528,174445.759557658 70 | 1529,174706.363659627 71 | 1530,164507.672539365 72 | 1531,163602.512172832 73 | 1532,154126.270249525 74 | 1533,171104.853481351 75 | 1534,167735.39270528 76 | 1535,183003.613338104 77 | 1536,172580.381161499 78 | 1537,165407.889104689 79 | 1538,176363.773907793 80 | 1539,175182.950898522 81 | 1540,190757.177789246 82 | 1541,167186.995771991 83 | 1542,167839.376779276 84 | 1543,173912.421165137 85 | 1544,154034.917445551 86 | 1545,156002.955794336 87 | 1546,168173.94329857 88 | 1547,168882.437104132 89 | 1548,168173.94329857 90 | 1549,157580.177551642 91 | 1550,181922.15256011 92 | 1551,155134.227842592 93 | 1552,188885.573319552 94 | 1553,183963.193012381 95 | 1554,161298.762306335 96 | 1555,188613.66763056 97 | 1556,175080.111822945 98 | 1557,174744.400305232 99 | 1558,168175.911336919 100 | 1559,182333.472575006 101 | 1560,158307.206742274 102 | 1561,193053.055502348 103 | 1562,175031.089987177 104 | 1563,160713.294602908 105 | 1564,173186.215014436 106 | 1565,191736.7598055 107 | 1566,170401.630997116 108 | 1567,164626.577880222 109 | 1568,205469.409444832 110 | 1569,209561.784211885 111 | 1570,182271.503072356 112 | 1571,178081.549427793 113 | 1572,178425.956138831 114 | 1573,162015.318511503 115 | 1574,181722.420373045 116 | 1575,156705.730169433 117 | 1576,182902.420342386 118 | 1577,157574.595395085 119 | 1578,184380.739100813 120 | 1579,169364.469225677 121 | 1580,175846.179822063 122 | 1581,189673.295302136 123 | 1582,174401.317715566 124 | 1583,179021.448718583 125 | 1584,189196.845337149 126 | 1585,139647.095720655 127 | 1586,161468.198288911 128 | 1587,171557.32317862 129 | 1588,179447.36804185 130 | 1589,169611.619017694 131 | 1590,172088.872655744 132 | 1591,171190.624128768 133 | 1592,154850.508361878 134 | 1593,158617.655719941 135 | 1594,209258.33693701 136 | 1595,177939.027626751 137 | 1596,194631.100299584 138 | 1597,213618.871562568 139 | 1598,198342.504228533 140 | 1599,138607.971472497 141 | 1600,150778.958976731 142 | 1601,146966.230339786 143 | 1602,162182.59620952 144 | 1603,176825.940961269 145 | 1604,152799.812402444 146 | 1605,180322.322067129 147 | 1606,177508.027228367 148 | 1607,208029.642652019 149 | 1608,181987.282510201 150 | 1609,160172.72797397 151 | 1610,176761.317654248 152 | 1611,176515.497545231 153 | 1612,176270.453065471 154 | 1613,183050.846258475 155 | 1614,150011.102062216 156 | 1615,159270.537808667 157 | 1616,163419.663729346 158 | 1617,163399.983345859 159 | 1618,173364.161505756 160 | 1619,169556.835902417 161 | 1620,183690.595995738 162 | 1621,176980.914909382 163 | 1622,204773.36222471 164 | 1623,174728.655998442 165 | 1624,181873.458244461 166 | 1625,177322.000823979 167 | 1626,193927.939041863 168 | 1627,181715.622732304 169 | 1628,199270.841200324 170 | 1629,177109.589956218 171 | 1630,153909.578271486 172 | 1631,162931.203336223 173 | 1632,166386.7567182 174 | 1633,173719.30379824 175 | 1634,179757.925656704 176 | 1635,179007.601964376 177 | 1636,180370.808623106 178 | 1637,185102.616730563 179 | 1638,198825.563452058 180 | 1639,184294.576009142 181 | 1640,200443.7920562 182 | 1641,181294.784484153 183 | 1642,174354.336267919 184 | 1643,172023.677781517 185 | 1644,181666.922855025 186 | 1645,179024.491269586 187 | 1646,178324.191575907 188 | 1647,184534.676687694 189 | 1648,159397.250378784 190 | 1649,178430.966728182 191 | 1650,177743.799385967 192 | 1651,179395.305519087 193 | 1652,151713.38474815 194 | 1653,151713.38474815 195 | 1654,168434.977996215 196 | 1655,153999.100311019 197 | 1656,164096.097354123 198 | 1657,166335.403036551 199 | 1658,163020.725375757 200 | 1659,155862.510668829 201 | 1660,182760.651095509 202 | 1661,201912.270622883 203 | 1662,185988.233987516 204 | 1663,183778.44888032 205 | 1664,170935.85921771 206 | 1665,184468.908382254 207 | 1666,191569.089663229 208 | 1667,232991.025583822 209 | 1668,180980.721388278 210 | 1669,164279.13048219 211 | 1670,183859.460411109 212 | 1671,185922.465682076 213 | 1672,191742.778119363 214 | 1673,199954.072465842 215 | 1674,180690.274752587 216 | 1675,163099.3096358 217 | 1676,140791.922472443 218 | 1677,166481.86647592 219 | 1678,172080.434496773 220 | 1679,191719.161659178 221 | 1680,160741.098612515 222 | 1681,157829.546854733 223 | 1682,196896.748596341 224 | 1683,159675.423990355 225 | 1684,182084.790901946 226 | 1685,179233.926374487 227 | 1686,155774.270901623 228 | 1687,181354.326716058 229 | 1688,179605.563663918 230 | 1689,181609.34866147 231 | 1690,178221.531623281 232 | 1691,175559.920735795 233 | 1692,200328.822792041 234 | 1693,178630.060559899 235 | 1694,177174.535221728 236 | 1695,172515.687368714 237 | 1696,204032.992922943 238 | 1697,176023.232787689 239 | 1698,202202.073341595 240 | 1699,181734.480075862 241 | 1700,183982.158993126 242 | 1701,188007.94241481 243 | 1702,185922.966763517 244 | 1703,183978.544874918 245 | 1704,177199.618638821 246 | 1705,181878.647956764 247 | 1706,173622.088728263 248 | 1707,180728.168562655 249 | 1708,176477.026606328 250 | 1709,184282.266697609 251 | 1710,162062.47538448 252 | 1711,182550.070992189 253 | 1712,180987.949624695 254 | 1713,178173.79762147 255 | 1714,179980.635948606 256 | 1715,173257.637826205 257 | 1716,177271.291059307 258 | 1717,175338.355442312 259 | 1718,177548.140549508 260 | 1719,175969.91662932 261 | 1720,175011.481953462 262 | 1721,185199.372568143 263 | 1722,188514.050228937 264 | 1723,185080.145268797 265 | 1724,157304.402574096 266 | 1725,194260.859481297 267 | 1726,181262.329995106 268 | 1727,157003.292706732 269 | 1728,182924.499359899 270 | 1729,181902.586375439 271 | 1730,188985.371708134 272 | 1731,185290.904495068 273 | 1732,177304.425752748 274 | 1733,166274.900490809 275 | 1734,177807.420530107 276 | 1735,180330.624816201 277 | 1736,179069.112234629 278 | 1737,175943.371816948 279 | 1738,185199.050609653 280 | 1739,167350.910824524 281 | 1740,149315.311876449 282 | 1741,139010.847766793 283 | 1742,155412.151845447 284 | 1743,171308.313985441 285 | 1744,176220.543265638 286 | 1745,177643.434991809 287 | 1746,187222.653264601 288 | 1747,185635.132083154 289 | 1748,206492.534215854 290 | 1749,181681.021081956 291 | 1750,180500.198072685 292 | 1751,206486.17086841 293 | 1752,161334.301195429 294 | 1753,176156.558313965 295 | 1754,191642.223478994 296 | 1755,191945.808027777 297 | 1756,164146.306037354 298 | 1757,179883.057071096 299 | 1758,178071.137668844 300 | 1759,188241.637896875 301 | 1760,174559.656173171 302 | 1761,182347.363042264 303 | 1762,191507.251872857 304 | 1763,199751.865597358 305 | 1764,162106.416145131 306 | 1765,164575.982314367 307 | 1766,179176.352180931 308 | 1767,177327.403857584 309 | 1768,177818.083761781 310 | 1769,186965.204048443 311 | 1770,178762.742169197 312 | 1771,183322.866146283 313 | 1772,178903.295931891 314 | 1773,186570.129421778 315 | 1774,199144.242829024 316 | 1775,172154.713310956 317 | 1776,177444.019201603 318 | 1777,166200.938073485 319 | 1778,158995.770555632 320 | 1779,168273.282454755 321 | 1780,189680.453052788 322 | 1781,181681.021081956 323 | 1782,160277.142643643 324 | 1783,197318.54715833 325 | 1784,162228.935604196 326 | 1785,187340.455456083 327 | 1786,181065.347037275 328 | 1787,190233.609102705 329 | 1788,157929.594852031 330 | 1789,168557.001935469 331 | 1790,160805.584645628 332 | 1791,221648.391978216 333 | 1792,180539.88079815 334 | 1793,182105.616283853 335 | 1794,166380.852603154 336 | 1795,178942.155617426 337 | 1796,162804.747800461 338 | 1797,183077.684392615 339 | 1798,171728.4720292 340 | 1799,164786.741540638 341 | 1800,177427.267170302 342 | 1801,197318.54715833 343 | 1802,178658.114178223 344 | 1803,185437.320523764 345 | 1804,169759.652489529 346 | 1805,173986.635055186 347 | 1806,168607.664289468 348 | 1807,194138.519145183 349 | 1808,192502.440921994 350 | 1809,176746.969818601 351 | 1810,177604.891703134 352 | 1811,193283.746584832 353 | 1812,181627.061006609 354 | 1813,169071.62025834 355 | 1814,167398.006470987 356 | 1815,150106.505141704 357 | 1816,159650.304285848 358 | 1817,179471.23597476 359 | 1818,177109.589956218 360 | 1819,166558.113328453 361 | 1820,153796.714319583 362 | 1821,174520.152570658 363 | 1822,196297.95829524 364 | 1823,169100.681601175 365 | 1824,176911.319164431 366 | 1825,169234.6454828 367 | 1826,172386.297919134 368 | 1827,156031.904802362 369 | 1828,168202.892306596 370 | 1829,166505.984017547 371 | 1830,176507.37022149 372 | 1831,180116.752553161 373 | 1832,183072.740591406 374 | 1833,189595.964677698 375 | 1834,167523.919076265 376 | 1835,210817.775863413 377 | 1836,172942.930813351 378 | 1837,145286.278144089 379 | 1838,176468.653371492 380 | 1839,159040.069562187 381 | 1840,178518.204332507 382 | 1841,169163.980786825 383 | 1842,189786.685274579 384 | 1843,181246.728523853 385 | 1844,176349.927153587 386 | 1845,205266.631009142 387 | 1846,187397.993362224 388 | 1847,208943.427726113 389 | 1848,165014.532907657 390 | 1849,182492.037566236 391 | 1850,161718.71259042 392 | 1851,180084.118941162 393 | 1852,178534.950802179 394 | 1853,151217.259961305 395 | 1854,156342.717587562 396 | 1855,188511.443835239 397 | 1856,183570.337896789 398 | 1857,225810.160292177 399 | 1858,214217.401131694 400 | 1859,187665.64101603 401 | 1860,161157.177744039 402 | 1861,187643.992594193 403 | 1862,228156.372839158 404 | 1863,220449.534665317 405 | 1864,220522.352084222 406 | 1865,156647.763531624 407 | 1866,187388.833374873 408 | 1867,178640.723791573 409 | 1868,180847.216739049 410 | 1869,159505.170529478 411 | 1870,164305.538020654 412 | 1871,180181.19673723 413 | 1872,184602.734989972 414 | 1873,193440.372174434 415 | 1874,184199.788209911 416 | 1875,196241.892907637 417 | 1876,175588.618271096 418 | 1877,179503.046546829 419 | 1878,183658.076582555 420 | 1879,193700.976276404 421 | 1880,165399.62450704 422 | 1881,186847.944787446 423 | 1882,198127.73287817 424 | 1883,183320.898107934 425 | 1884,181613.606696657 426 | 1885,178298.791761954 427 | 1886,185733.534000593 428 | 1887,180008.188485489 429 | 1888,175127.59621604 430 | 1889,183467.176862723 431 | 1890,182705.546021743 432 | 1891,152324.943593181 433 | 1892,169878.515981342 434 | 1893,183735.975076576 435 | 1894,224118.280105941 436 | 1895,169355.202465146 437 | 1896,180054.276407441 438 | 1897,174081.601977368 439 | 1898,168494.985022146 440 | 1899,181871.598843299 441 | 1900,173554.489658383 442 | 1901,169805.382165577 443 | 1902,176192.990728755 444 | 1903,204264.39284654 445 | 1904,169630.906956928 446 | 1905,185724.838807268 447 | 1906,195699.036281861 448 | 1907,189494.276162169 449 | 1908,149607.905673439 450 | 1909,154650.199045978 451 | 1910,151579.558140433 452 | 1911,185147.380531144 453 | 1912,196314.53120359 454 | 1913,210802.395364155 455 | 1914,166271.2863726 456 | 1915,154865.359142973 457 | 1916,173575.5052865 458 | 1917,179399.563554274 459 | 1918,164280.776562049 460 | 1919,171247.48948121 461 | 1920,166878.587182445 462 | 1921,188129.459710994 463 | 1922,183517.34369691 464 | 1923,175522.026925727 465 | 1924,190060.105331152 466 | 1925,174179.824771856 467 | 1926,171059.523675194 468 | 1927,183004.186769318 469 | 1928,183601.647387418 470 | 1929,163539.327185998 471 | 1930,164677.676391525 472 | 1931,162395.073865424 473 | 1932,182207.6323195 474 | 1933,192223.939790304 475 | 1934,176391.829390125 476 | 1935,181913.179121348 477 | 1936,179136.097888261 478 | 1937,196595.568243212 479 | 1938,194822.365690957 480 | 1939,148356.669440918 481 | 1940,160387.604263899 482 | 1941,181276.500571809 483 | 1942,192474.817899346 484 | 1943,157699.907796437 485 | 1944,215785.540813051 486 | 1945,181824.300998793 487 | 1946,221813.00948166 488 | 1947,165281.292597397 489 | 1948,255629.49047034 490 | 1949,173154.590990955 491 | 1950,183884.65246539 492 | 1951,200210.353608489 493 | 1952,186599.221265342 494 | 1953,192718.532696106 495 | 1954,178628.665952764 496 | 1955,180650.342418406 497 | 1956,206003.107947263 498 | 1957,166457.67844853 499 | 1958,202916.221653487 500 | 1959,192463.969983091 501 | 1960,171775.497189898 502 | 1961,175249.222149411 503 | 1962,147086.59893993 504 | 1963,149709.672100371 505 | 1964,171411.404533743 506 | 1965,178188.964799425 507 | 1966,156491.711373235 508 | 1967,180953.241201168 509 | 1968,203909.759061135 510 | 1969,175470.149087545 511 | 1970,205578.333622415 512 | 1971,199428.857699441 513 | 1972,187599.163869476 514 | 1973,192265.198109864 515 | 1974,196666.554897677 516 | 1975,155537.862252682 517 | 1976,169543.240620935 518 | 1977,202487.010170501 519 | 1978,208232.716273485 520 | 1979,173621.195202569 521 | 1980,172414.608571812 522 | 1981,164400.75641556 523 | 1982,160480.424024781 524 | 1983,156060.853810389 525 | 1984,157437.192820581 526 | 1985,158163.720929772 527 | 1986,154849.043268978 528 | 1987,152186.609341561 529 | 1988,180340.215399228 530 | 1989,178344.62451356 531 | 1990,190170.382266827 532 | 1991,168092.975480832 533 | 1992,178757.912566805 534 | 1993,174518.256882082 535 | 1994,198168.490116289 536 | 1995,176882.693978902 537 | 1996,183801.672896251 538 | 1997,196400.046680661 539 | 1998,172281.605004025 540 | 1999,196380.366297173 541 | 2000,198228.354306682 542 | 2001,195556.581268962 543 | 2002,186453.264469043 544 | 2003,181869.381196234 545 | 2004,175610.840124147 546 | 2005,183438.730800145 547 | 2006,179584.488673295 548 | 2007,182386.152242034 549 | 2008,160750.367237054 550 | 2009,182477.505046008 551 | 2010,187720.359207171 552 | 2011,187201.942081511 553 | 2012,176385.102235149 554 | 2013,175901.787841278 555 | 2014,182584.280198283 556 | 2015,195664.686104237 557 | 2016,181420.346494222 558 | 2017,176676.04995228 559 | 2018,181594.678867334 560 | 2019,178521.747964951 561 | 2020,175895.883726231 562 | 2021,168468.005916477 563 | 2022,200973.129447888 564 | 2023,197030.641992202 565 | 2024,192867.417844592 566 | 2025,196449.247639381 567 | 2026,141684.196398607 568 | 2027,153353.334123901 569 | 2028,151143.549016705 570 | 2029,163753.087114229 571 | 2030,158682.460013921 572 | 2031,144959.835250915 573 | 2032,160144.390548579 574 | 2033,156286.534303521 575 | 2034,165726.707619571 576 | 2035,182427.481047359 577 | 2036,173310.56154032 578 | 2037,173310.56154032 579 | 2038,151556.01403002 580 | 2039,158908.146068683 581 | 2040,209834.383092536 582 | 2041,192410.516550815 583 | 2042,174026.247294886 584 | 2043,195499.830115336 585 | 2044,200918.018812493 586 | 2045,207243.616023976 587 | 2046,196149.783851876 588 | 2047,192097.914850217 589 | 2048,178570.948923671 590 | 2049,228617.968325428 591 | 2050,199929.884438451 592 | 2051,160206.365612859 593 | 2052,179854.431885567 594 | 2053,185987.340461822 595 | 2054,161122.505607926 596 | 2055,175949.342720138 597 | 2056,183683.590595324 598 | 2057,176401.34762338 599 | 2058,205832.532527897 600 | 2059,177799.799849436 601 | 2060,167565.362080406 602 | 2061,186348.958436557 603 | 2062,179782.759465081 604 | 2063,169837.623333323 605 | 2064,178817.275675758 606 | 2065,174444.479149339 607 | 2066,192834.968917174 608 | 2067,196564.717984981 609 | 2068,206977.567039357 610 | 2069,157054.253944128 611 | 2070,175142.948078577 612 | 2071,159932.1643654 613 | 2072,182801.408333628 614 | 2073,181510.375176825 615 | 2074,181613.035129451 616 | 2075,186920.512597635 617 | 2076,157950.170625222 618 | 2077,176115.159022876 619 | 2078,182744.514344465 620 | 2079,180660.683691591 621 | 2080,160775.629777099 622 | 2081,186711.715848082 623 | 2082,223581.758190888 624 | 2083,172330.943236652 625 | 2084,163474.633393212 626 | 2085,175308.263299874 627 | 2086,187462.725306432 628 | 2087,180655.101535034 629 | 2088,152121.98603454 630 | 2089,159856.233909727 631 | 2090,186559.854936737 632 | 2091,183962.550959411 633 | 2092,162107.168699296 634 | 2093,162582.288981283 635 | 2094,154407.701597409 636 | 2095,181625.666399474 637 | 2096,164810.609473548 638 | 2097,176429.401241704 639 | 2098,179188.089925259 640 | 2099,145997.635377703 641 | 2100,218676.768270367 642 | 2101,188323.861214226 643 | 2102,168690.0722914 644 | 2103,165088.746797705 645 | 2104,191435.007885166 646 | 2105,168864.404664512 647 | 2106,176041.882371574 648 | 2107,215911.674390325 649 | 2108,167388.238629016 650 | 2109,163854.786753017 651 | 2110,163299.477980171 652 | 2111,178298.214633119 653 | 2112,176376.586164775 654 | 2113,170211.043976522 655 | 2114,170818.344786366 656 | 2115,174388.867432503 657 | 2116,161112.987374671 658 | 2117,172179.082325307 659 | 2118,157798.309713876 660 | 2119,169106.151422924 661 | 2120,170129.531364292 662 | 2121,157680.227412949 663 | 2122,162690.209131977 664 | 2123,146968.379365095 665 | 2124,181507.721372455 666 | 2125,191215.589752983 667 | 2126,189432.689844522 668 | 2127,207271.484957719 669 | 2128,170030.807488363 670 | 2129,148409.806476335 671 | 2130,193850.613979055 672 | 2131,193808.319298263 673 | 2132,166300.235380627 674 | 2133,163474.633393212 675 | 2134,177473.606564978 676 | 2135,157443.925537187 677 | 2136,180681.007992057 678 | 2137,183463.17030026 679 | 2138,182481.763081195 680 | 2139,193717.15117887 681 | 2140,182782.55099007 682 | 2141,175530.651633287 683 | 2142,177804.057884623 684 | 2143,159448.670848577 685 | 2144,181338.976717529 686 | 2145,178553.558537021 687 | 2146,162820.928264556 688 | 2147,188832.479997186 689 | 2148,164682.185899437 690 | 2149,181549.735943801 691 | 2150,199158.097008868 692 | 2151,152889.520990566 693 | 2152,181150.551679116 694 | 2153,181416.732376013 695 | 2154,164391.238182305 696 | 2155,185421.046498812 697 | 2156,193981.327550004 698 | 2157,178824.324789223 699 | 2158,209270.051606246 700 | 2159,177801.266806344 701 | 2160,179053.762236101 702 | 2161,178762.170601992 703 | 2162,184655.300458183 704 | 2163,191284.655779772 705 | 2164,179598.085818785 706 | 2165,167517.628078595 707 | 2166,182873.903794044 708 | 2167,177484.91371363 709 | 2168,188444.597319524 710 | 2169,179184.153848562 711 | 2170,184365.175780982 712 | 2171,184479.322005212 713 | 2172,182927.863869391 714 | 2173,178611.639373646 715 | 2174,181943.343613558 716 | 2175,175080.614768394 717 | 2176,190720.794649138 718 | 2177,198422.868144723 719 | 2178,184482.11308349 720 | 2179,139214.952187861 721 | 2180,169233.113601757 722 | 2181,180664.118686848 723 | 2182,178818.742632666 724 | 2183,180422.049969947 725 | 2184,178601.93645581 726 | 2185,183083.159775993 727 | 2186,173163.101499699 728 | 2187,185968.161159774 729 | 2188,171226.050683054 730 | 2189,281643.976116786 731 | 2190,160031.711281258 732 | 2191,162775.979779394 733 | 2192,160735.445970193 734 | 2193,166646.109048572 735 | 2194,188384.548444549 736 | 2195,165830.697255197 737 | 2196,182138.358533039 738 | 2197,171595.397975647 739 | 2198,160337.079183809 740 | 2199,191215.088671543 741 | 2200,166956.093232213 742 | 2201,186581.830878692 743 | 2202,176450.548582099 744 | 2203,193743.194909801 745 | 2204,198882.566078408 746 | 2205,176385.102235149 747 | 2206,162447.639333636 748 | 2207,193782.555676777 749 | 2208,183653.890897141 750 | 2209,210578.623546866 751 | 2210,158527.164107319 752 | 2211,163081.025723456 753 | 2212,174388.867432503 754 | 2213,191905.870131966 755 | 2214,174388.867432503 756 | 2215,161642.711648983 757 | 2216,186939.507215101 758 | 2217,172482.165792649 759 | 2218,159695.999763546 760 | 2219,157230.369671007 761 | 2220,179188.089925259 762 | 2221,157972.82120994 763 | 2222,156804.951429181 764 | 2223,211491.972463654 765 | 2224,186537.246201062 766 | 2225,200468.161070551 767 | 2226,182241.340444154 768 | 2227,157342.225898399 769 | 2228,182022.387105998 770 | 2229,181244.510876788 771 | 2230,178556.671573788 772 | 2231,189547.199876284 773 | 2232,187948.65165563 774 | 2233,194107.287565956 775 | 2234,183521.710369283 776 | 2235,183682.123638416 777 | 2236,178483.353073443 778 | 2237,184003.879764736 779 | 2238,171318.59033449 780 | 2239,162039.754313997 781 | 2240,154846.252190699 782 | 2241,194822.365690957 783 | 2242,169788.738771463 784 | 2243,178891.554489941 785 | 2244,152084.772428865 786 | 2245,139169.86642879 787 | 2246,192439.536044606 788 | 2247,161067.859766557 789 | 2248,158762.648504781 790 | 2249,175569.690441774 791 | 2250,183659.795012187 792 | 2251,280618.132617258 793 | 2252,180051.809151659 794 | 2253,176519.18031559 795 | 2254,179028.429210291 796 | 2255,177161.583857224 797 | 2256,180081.508849842 798 | 2257,205895.254584712 799 | 2258,183389.78131415 800 | 2259,178543.647859512 801 | 2260,194798.320499104 802 | 2261,162845.613675766 803 | 2262,148103.867006579 804 | 2263,201016.171121215 805 | 2264,277936.12694354 806 | 2265,249768.279823405 807 | 2266,161596.052159825 808 | 2267,158011.114889899 809 | 2268,194089.683858004 810 | 2269,181733.336941451 811 | 2270,182852.32772198 812 | 2271,189893.003058465 813 | 2272,194650.210979875 814 | 2273,187904.461286262 815 | 2274,171774.925622692 816 | 2275,177998.685921479 817 | 2276,175648.484325498 818 | 2277,196918.071362067 819 | 2278,184299.838071218 820 | 2279,182379.855682734 821 | 2280,184050.725802482 822 | 2281,158296.975970284 823 | 2282,175053.355553278 824 | 2283,162293.376090644 825 | 2284,186328.880047186 826 | 2285,151422.116936538 827 | 2286,181969.358707768 828 | 2287,189122.67702416 829 | 2288,185645.475220346 830 | 2289,182829.898109257 831 | 2290,195848.788183328 832 | 2291,198785.059550672 833 | 2292,181676.126555428 834 | 2293,194131.012663328 835 | 2294,201416.004864508 836 | 2295,185096.577205616 837 | 2296,195158.972598372 838 | 2297,184795.783735112 839 | 2298,189168.263864671 840 | 2299,216855.260149095 841 | 2300,184946.642483576 842 | 2301,189317.51282069 843 | 2302,180803.277842406 844 | 2303,175061.18585763 845 | 2304,179074.839090732 846 | 2305,145708.764336107 847 | 2306,142398.022752011 848 | 2307,161474.534863641 849 | 2308,157025.945155458 850 | 2309,163424.037827357 851 | 2310,164692.778645345 852 | 2311,152163.2443541 853 | 2312,192383.215486656 854 | 2313,182520.230322476 855 | 2314,187254.507549722 856 | 2315,176489.659740359 857 | 2316,181520.466841293 858 | 2317,186414.978214721 859 | 2318,185197.764639705 860 | 2319,178657.794083741 861 | 2320,179731.198023759 862 | 2321,161748.271317074 863 | 2322,158608.749069322 864 | 2323,178807.370559878 865 | 2324,184187.158803897 866 | 2325,181686.10402108 867 | 2326,190311.050228337 868 | 2327,192252.496354076 869 | 2328,193954.849525775 870 | 2329,181044.201560887 871 | 2330,180258.131219792 872 | 2331,199641.657313834 873 | 2332,197530.775205517 874 | 2333,191777.196949138 875 | 2334,195779.543033588 876 | 2335,202112.046522999 877 | 2336,192343.34807661 878 | 2337,185191.359443218 879 | 2338,186760.207965688 880 | 2339,177733.78193528 881 | 2340,164430.391189608 882 | 2341,185299.601552401 883 | 2342,186414.012339254 884 | 2343,176401.921054593 885 | 2344,182381.322639642 886 | 2345,176334.184710805 887 | 2346,184901.735847457 888 | 2347,180085.766885029 889 | 2348,184901.735847457 890 | 2349,183967.561548763 891 | 2350,193046.301574659 892 | 2351,168538.969495849 893 | 2352,170157.842016969 894 | 2353,196559.709259637 895 | 2354,177133.709361852 896 | 2355,181553.279576244 897 | 2356,185770.606634739 898 | 2357,177017.595099274 899 | 2358,184123.358536806 900 | 2359,165970.357492196 901 | 2360,158151.985049452 902 | 2361,177086.476441481 903 | 2362,196373.896176551 904 | 2363,172465.707083115 905 | 2364,168590.782409896 906 | 2365,158820.474171061 907 | 2366,151611.37057651 908 | 2367,152125.028585543 909 | 2368,158404.073081048 910 | 2369,160692.078640755 911 | 2370,170175.22684199 912 | 2371,169854.436591138 913 | 2372,183410.785819008 914 | 2373,180347.194026928 915 | 2374,178930.528374292 916 | 2375,153346.220086301 917 | 2376,182675.204270589 918 | 2377,180770.649792036 919 | 2378,188714.148087543 920 | 2379,191393.608594076 921 | 2380,174016.157494425 922 | 2381,183189.685319552 923 | 2382,183621.508757866 924 | 2383,168991.29635758 925 | 2384,185306.650665866 926 | 2385,189030.680303208 927 | 2386,179208.665698449 928 | 2387,174901.452792889 929 | 2388,168337.406544343 930 | 2389,158234.96461859 931 | 2390,179562.453368834 932 | 2391,174176.391640607 933 | 2392,173931.531845427 934 | 2393,184111.729429665 935 | 2394,179374.482001188 936 | 2395,207348.811884535 937 | 2396,186983.419339031 938 | 2397,206779.094049527 939 | 2398,177472.074683935 940 | 2399,156727.948324862 941 | 2400,157090.568462479 942 | 2401,160387.032696693 943 | 2402,172410.28005086 944 | 2403,191603.365657467 945 | 2404,182152.207151253 946 | 2405,180161.697340702 947 | 2406,169652.235284283 948 | 2407,182503.520140218 949 | 2408,179714.630677039 950 | 2409,180282.570719908 951 | 2410,192600.338060371 952 | 2411,166115.491248565 953 | 2412,186379.553524443 954 | 2413,184361.992258449 955 | 2414,186220.965458121 956 | 2415,198176.47090687 957 | 2416,168437.776500131 958 | 2417,178003.582312015 959 | 2418,179180.469244588 960 | 2419,191930.561104806 961 | 2420,175590.266214964 962 | 2421,176713.19307219 963 | 2422,180159.090947005 964 | 2423,188090.100808026 965 | 2424,186184.717727913 966 | 2425,223055.588672278 967 | 2426,158270.753116401 968 | 2427,184733.12846644 969 | 2428,199926.378957429 970 | 2429,175075.785166001 971 | 2430,180917.925148076 972 | 2431,182067.760625207 973 | 2432,178238.60191545 974 | 2433,173454.944606532 975 | 2434,176821.936262814 976 | 2435,183642.191304235 977 | 2436,177254.582741058 978 | 2437,168715.950111702 979 | 2438,180096.931198144 980 | 2439,160620.728178758 981 | 2440,175286.544392273 982 | 2441,153494.783276297 983 | 2442,156407.65915545 984 | 2443,162162.525245786 985 | 2444,166809.886827197 986 | 2445,172929.156408918 987 | 2446,193514.330894137 988 | 2447,181612.141603756 989 | 2448,191745.386377068 990 | 2449,171369.325038261 991 | 2450,184425.470567051 992 | 2451,170563.252355189 993 | 2452,184522.369240168 994 | 2453,164968.947931153 995 | 2454,157939.621592364 996 | 2455,151520.381580069 997 | 2456,176129.508722531 998 | 2457,171112.978971478 999 | 2458,169762.081624282 1000 | 2459,162246.828936295 1001 | 2460,171339.303381589 1002 | 2461,189034.753653813 1003 | 2462,175758.873595981 1004 | 2463,163351.721489893 1005 | 2464,189806.546645026 1006 | 2465,175370.990918319 1007 | 2466,196895.599900301 1008 | 2467,176905.917994834 1009 | 2468,176866.557227858 1010 | 2469,163590.677170026 1011 | 2470,212693.502958393 1012 | 2471,192686.931747717 1013 | 2472,181578.684951827 1014 | 2473,166475.457581812 1015 | 2474,185998.255166219 1016 | 2475,185527.714877908 1017 | 2476,159027.118197683 1018 | 2477,181169.654933769 1019 | 2478,176732.915304722 1020 | 2479,191619.294648838 1021 | 2480,189114.303789324 1022 | 2481,180934.635330334 1023 | 2482,164573.372223048 1024 | 2483,173902.011270196 1025 | 2484,165625.127741229 1026 | 2485,179555.219570787 1027 | 2486,196899.720661579 1028 | 2487,207566.12470446 1029 | 2488,163899.981149274 1030 | 2489,189179.428177786 1031 | 2490,193892.880023125 1032 | 2491,178980.874331431 1033 | 2492,179749.876244365 1034 | 2493,197999.674975598 1035 | 2494,203717.470295797 1036 | 2495,185249.261156892 1037 | 2496,201691.208274848 1038 | 2497,181956.548314794 1039 | 2498,171895.936275806 1040 | 2499,187245.168439419 1041 | 2500,157816.77461318 1042 | 2501,191702.912573325 1043 | 2502,198599.420028908 1044 | 2503,187193.313676329 1045 | 2504,220514.993999535 1046 | 2505,181814.527595192 1047 | 2506,183750.755371907 1048 | 2507,183000.431679579 1049 | 2508,185830.971906573 1050 | 2509,185497.872344187 1051 | 2510,179613.437681321 1052 | 2511,164454.967963631 1053 | 2512,185127.237217638 1054 | 2513,178750.613844623 1055 | 2514,160927.61044889 1056 | 2515,192562.808057836 1057 | 2516,180990.24148554 1058 | 2517,180064.941503122 1059 | 2518,196070.997393789 1060 | 2519,180352.919019023 1061 | 2520,183367.953769362 1062 | 2521,176734.841494027 1063 | 2522,180848.220765939 1064 | 2523,187806.059368823 1065 | 2524,180521.52640004 1066 | 2525,181502.754496154 1067 | 2526,174525.87942676 1068 | 2527,188927.984063168 1069 | 2528,184728.870431253 1070 | 2529,179857.975518011 1071 | 2530,180962.868071609 1072 | 2531,179194.066390078 1073 | 2532,179591.789259484 1074 | 2533,180638.463702549 1075 | 2534,185846.215131922 1076 | 2535,195174.031139141 1077 | 2536,192474.56829063 1078 | 2537,164200.595496827 1079 | 2538,178403.094096818 1080 | 2539,170774.84018302 1081 | 2540,179879.945898337 1082 | 2541,177668.192752792 1083 | 2542,180174.328610725 1084 | 2543,170643.303572141 1085 | 2544,165448.004289838 1086 | 2545,195531.754886222 1087 | 2546,165314.177682121 1088 | 2547,172532.757660882 1089 | 2548,203310.218069877 1090 | 2549,175090.062515883 1091 | 2550,230841.338626282 1092 | 2551,155225.19006632 1093 | 2552,168322.342441945 1094 | 2553,165956.259265265 1095 | 2554,193956.817564124 1096 | 2555,171070.367893827 1097 | 2556,166285.243628001 1098 | 2557,182875.801346628 1099 | 2558,218108.536769738 1100 | 2559,174378.777632042 1101 | 2560,164731.316372391 1102 | 2561,156969.695083273 1103 | 2562,173388.854342604 1104 | 2563,177559.628685119 1105 | 2564,194297.789279905 1106 | 2565,174894.588364005 1107 | 2566,196544.144075798 1108 | 2567,179036.158528149 1109 | 2568,211423.986511149 1110 | 2569,208156.398935188 1111 | 2570,159233.941347257 1112 | 2571,210820.115134931 1113 | 2572,140196.10979821 1114 | 2573,198678.469082978 1115 | 2574,186818.610760803 1116 | 2575,175044.797633861 1117 | 2576,180031.162892704 1118 | 2577,176889.171525162 1119 | 2578,159638.856165666 1120 | 2579,154287.264375509 1121 | 2580,191885.618181273 1122 | 2581,177503.378612934 1123 | 2582,166548.31684976 1124 | 2583,164475.14942856 1125 | 2584,167484.744857879 1126 | 2585,188683.160555403 1127 | 2586,162243.399502668 1128 | 2587,180807.213919103 1129 | 2588,176279.079637039 1130 | 2589,163438.959094218 1131 | 2590,161495.5393685 1132 | 2591,216032.303722443 1133 | 2592,176632.181541401 1134 | 2593,168743.001567144 1135 | 2594,183810.11848086 1136 | 2595,156794.36054728 1137 | 2596,169136.43011395 1138 | 2597,183203.318752456 1139 | 2598,213252.926930889 1140 | 2599,190550.327866959 1141 | 2600,234707.209860273 1142 | 2601,135751.318892816 1143 | 2602,164228.45886894 1144 | 2603,153219.437030419 1145 | 2604,164210.746523801 1146 | 2605,163883.229117973 1147 | 2606,154892.776269956 1148 | 2607,197092.08733832 1149 | 2608,228148.376399122 1150 | 2609,178680.587503997 1151 | 2610,165643.341167808 1152 | 2611,222406.642660249 1153 | 2612,184021.843582599 1154 | 2613,170871.094939159 1155 | 2614,189562.873697309 1156 | 2615,170591.884966356 1157 | 2616,172934.351682851 1158 | 2617,186425.069879189 1159 | 2618,218648.131133006 1160 | 2619,183035.606761141 1161 | 2620,178378.906069427 1162 | 2621,184516.716597846 1163 | 2622,181419.5253183 1164 | 2623,196858.923438425 1165 | 2624,189228.701486278 1166 | 2625,208973.380761028 1167 | 2626,180269.86896412 1168 | 2627,159488.713683953 1169 | 2628,191490.299507521 1170 | 2629,228684.245137946 1171 | 2630,201842.998700429 1172 | 2631,209242.82289186 1173 | 2632,202357.62258493 1174 | 2633,168238.61218265 1175 | 2634,202524.12465369 1176 | 2635,170588.771929588 1177 | 2636,198375.31512987 1178 | 2637,170636.827889889 1179 | 2638,181991.079479377 1180 | 2639,183994.54251844 1181 | 2640,182951.482193584 1182 | 2641,174126.297156192 1183 | 2642,170575.496742588 1184 | 2643,175332.239869971 1185 | 2644,167522.061539111 1186 | 2645,168095.583738538 1187 | 2646,154406.415627461 1188 | 2647,170996.973346087 1189 | 2648,159056.890245639 1190 | 2649,181373.6165193 1191 | 2650,152272.560975937 1192 | 2651,168664.346821336 1193 | 2652,211007.008292301 1194 | 2653,182909.515032911 1195 | 2654,203926.829353303 1196 | 2655,179082.825442944 1197 | 2656,206260.099795032 1198 | 2657,181732.443415757 1199 | 2658,189698.740693148 1200 | 2659,203074.34678979 1201 | 2660,201670.634365666 1202 | 2661,173756.812589691 1203 | 2662,181387.076390881 1204 | 2663,184859.155270535 1205 | 2664,158313.615666777 1206 | 2665,151951.955409666 1207 | 2666,162537.52704471 1208 | 2667,178998.337067854 1209 | 2668,186732.584943041 1210 | 2669,187323.318406165 1211 | 2670,199437.232798284 1212 | 2671,185546.680858653 1213 | 2672,161595.015798593 1214 | 2673,154672.422763036 1215 | 2674,159355.710116165 1216 | 2675,155919.014077746 1217 | 2676,182424.87095604 1218 | 2677,178100.589622319 1219 | 2678,202577.900044456 1220 | 2679,177862.778940605 1221 | 2680,182056.024744887 1222 | 2681,191403.199177104 1223 | 2682,196264.754980043 1224 | 2683,209375.003419718 1225 | 2684,196691.81930173 1226 | 2685,192458.431539585 1227 | 2686,182242.80926507 1228 | 2687,183259.503900506 1229 | 2688,188108.243748841 1230 | 2689,171418.640195797 1231 | 2690,194698.882220432 1232 | 2691,174841.84007522 1233 | 2692,172965.476488899 1234 | 2693,189386.323677132 1235 | 2694,185682.618340257 1236 | 2695,176412.012719061 1237 | 2696,174976.489722867 1238 | 2697,180718.581707643 1239 | 2698,186131.188248242 1240 | 2699,165220.786354033 1241 | 2700,164115.893800435 1242 | 2701,182125.729127024 1243 | 2702,182285.140233276 1244 | 2703,196325.442210366 1245 | 2704,164865.215329881 1246 | 2705,182694.492209823 1247 | 2706,185425.485520958 1248 | 2707,171414.7041191 1249 | 2708,183433.472466085 1250 | 2709,176844.981155794 1251 | 2710,180568.187753206 1252 | 2711,185948.625475832 1253 | 2712,189388.291715481 1254 | 2713,142754.489165865 1255 | 2714,156106.800760811 1256 | 2715,155895.397617561 1257 | 2716,159851.977738548 1258 | 2717,185157.832305524 1259 | 2718,180716.291710805 1260 | 2719,176901.093954071 1261 | 2720,181017.222455218 1262 | 2721,183269.159407668 1263 | 2722,193550.830097069 1264 | 2723,170625.842699726 1265 | 2724,182012.405942725 1266 | 2725,179162.507290733 1267 | 2726,183269.159407668 1268 | 2727,180589.836175042 1269 | 2728,181465.935198741 1270 | 2729,196053.029878304 1271 | 2730,183421.020319014 1272 | 2731,167926.839083612 1273 | 2732,168027.530997889 1274 | 2733,182164.26685407 1275 | 2734,172469.071592608 1276 | 2735,181059.374300472 1277 | 2736,182997.570115536 1278 | 2737,166140.504179894 1279 | 2738,198515.546934075 1280 | 2739,193789.648503294 1281 | 2740,173550.025727531 1282 | 2741,176487.943174734 1283 | 2742,188813.302559147 1284 | 2743,178531.911979192 1285 | 2744,182145.731469001 1286 | 2745,179196.465024103 1287 | 2746,169618.349900686 1288 | 2747,170010.168655046 1289 | 2748,181739.671652174 1290 | 2749,172846.934955574 1291 | 2750,195560.8830172 1292 | 2751,180358.114292956 1293 | 2752,211817.702818093 1294 | 2753,176170.128686742 1295 | 2754,234492.248263699 1296 | 2755,182450.956536015 1297 | 2756,174902.068073146 1298 | 2757,173684.174293738 1299 | 2758,147196.673677562 1300 | 2759,175231.189709791 1301 | 2760,193417.64740633 1302 | 2761,183313.601249761 1303 | 2762,180882.250849082 1304 | 2763,186735.697979808 1305 | 2764,172922.865411247 1306 | 2765,202551.677190573 1307 | 2766,190485.634074173 1308 | 2767,173439.49362151 1309 | 2768,196613.598849219 1310 | 2769,178152.259700828 1311 | 2770,174519.904825949 1312 | 2771,172627.796932837 1313 | 2772,173732.689486435 1314 | 2773,209219.844787023 1315 | 2774,181059.374300472 1316 | 2775,188515.443002459 1317 | 2776,182164.26685407 1318 | 2777,188137.901597981 1319 | 2778,158893.54306269 1320 | 2779,189579.65066771 1321 | 2780,165229.803505847 1322 | 2781,162186.071220207 1323 | 2782,166374.879866351 1324 | 2783,161665.184974757 1325 | 2784,175079.328798445 1326 | 2785,203840.874021305 1327 | 2786,152129.078861057 1328 | 2787,181012.141380101 1329 | 2788,161305.53503837 1330 | 2789,203326.392972343 1331 | 2790,168385.571141831 1332 | 2791,183564.365159986 1333 | 2792,163784.619440861 1334 | 2793,171989.192193993 1335 | 2794,180839.95616829 1336 | 2795,170895.923185907 1337 | 2796,174071.054808518 1338 | 2797,259423.859147546 1339 | 2798,188000.824679588 1340 | 2799,179171.703565498 1341 | 2800,171022.241447762 1342 | 2801,174126.297156192 1343 | 2802,187625.573271948 1344 | 2803,199567.946369234 1345 | 2804,205328.078219268 1346 | 2805,166231.535025379 1347 | 2806,154743.91606057 1348 | 2807,159714.537012622 1349 | 2808,185563.069082422 1350 | 2809,171500.796725006 1351 | 2810,180983.443844799 1352 | 2811,183141.236914997 1353 | 2812,178498.634450214 1354 | 2813,224323.710512388 1355 | 2814,218200.642127877 1356 | 2815,182283.177756557 1357 | 2816,190054.639237419 1358 | 2817,160192.453934518 1359 | 2818,171289.393581756 1360 | 2819,151131.098733642 1361 | 2820,181721.458225594 1362 | 2821,172725.053851858 1363 | 2822,222438.699143414 1364 | 2823,235419.373448928 1365 | 2824,185150.926027596 1366 | 2825,184772.239624699 1367 | 2826,180658.216435809 1368 | 2827,209673.316647174 1369 | 2828,205939.810625621 1370 | 2829,165633.573325837 1371 | 2830,186030.317211014 1372 | 2831,160312.319589212 1373 | 2832,190702.440251029 1374 | 2833,175122.810326699 1375 | 2834,183783.13937519 1376 | 2835,178290.666302221 1377 | 2836,181605.343963015 1378 | 2837,187992.451444752 1379 | 2838,188885.11781517 1380 | 2839,189959.344795118 1381 | 2840,179258.619211334 1382 | 2841,181518.750275669 1383 | 2842,193008.659237315 1384 | 2843,186313.89385619 1385 | 2844,181499.39185067 1386 | 2845,174126.297156192 1387 | 2846,183918.612062767 1388 | 2847,184114.270899227 1389 | 2848,158540.947801398 1390 | 2849,197034.759055859 1391 | 2850,185170.284452595 1392 | 2851,221134.533635148 1393 | 2852,184306.637575967 1394 | 2853,199792.302740996 1395 | 2854,143237.803559736 1396 | 2855,177294.838897736 1397 | 2856,182368.620883855 1398 | 2857,176487.943174734 1399 | 2858,183849.408762071 1400 | 2859,184964.141507413 1401 | 2860,196395.969632434 1402 | 2861,188374.936650438 1403 | 2862,176261.296806135 1404 | 2863,163628.142248426 1405 | 2864,180618.032628904 1406 | 2865,161647.329794081 1407 | 2866,167129.598867773 1408 | 2867,174750.988352687 1409 | 2868,177560.202116333 1410 | 2869,192577.796112839 1411 | 2870,199202.898960871 1412 | 2871,182818.156667308 1413 | 2872,148217.262540651 1414 | 2873,188997.797082492 1415 | 2874,185807.928877601 1416 | 2875,177030.477842021 1417 | 2876,175942.474593632 1418 | 2877,172912.518576433 1419 | 2878,198359.248864591 1420 | 2879,184379.133036383 1421 | 2880,194255.566948886 1422 | 2881,209449.651603064 1423 | 2882,169979.323958443 1424 | 2883,188206.281858748 1425 | 2884,186412.438609167 1426 | 2885,196761.386409959 1427 | 2886,208353.269558209 1428 | 2887,166548.067241044 1429 | 2888,175942.474593632 1430 | 2889,166790.457916434 1431 | 2890,160515.850579067 1432 | 2891,192167.621096362 1433 | 2892,178751.551083369 1434 | 2893,198678.894117024 1435 | 2894,164553.120272354 1436 | 2895,156887.932862327 1437 | 2896,164185.777305524 1438 | 2897,212992.120630876 1439 | 2898,197468.550532521 1440 | 2899,180106.84373966 1441 | 2900,183972.071056674 1442 | 2901,245283.198337927 1443 | 2902,170351.963410756 1444 | 2903,195596.307707478 1445 | 2904,189369.756330412 1446 | 2905,223667.404551664 1447 | 2906,169335.310624364 1448 | 2907,167411.02835165 1449 | 2908,187709.555003968 1450 | 2909,196526.002998991 1451 | 2910,137402.569855589 1452 | 2911,165086.775061735 1453 | 2912,188506.431412274 1454 | 2913,172917.456816012 1455 | 2914,166274.325225982 1456 | 2915,167081.220948984 1457 | 2916,164788.778231138 1458 | 2917,219222.423400059 1459 | 2918,184924.279658997 1460 | 2919,187741.866657478 1461 | -------------------------------------------------------------------------------- /train.php: -------------------------------------------------------------------------------- 1 | info('Loading data into memory'); 21 | 22 | $extractor = new ColumnPicker(new CSV('dataset.csv', true), [ 23 | 'MSSubClass', 'MSZoning', 'LotFrontage', 'LotArea', 'Street', 'Alley', 24 | 'LotShape', 'LandContour', 'Utilities', 'LotConfig', 'LandSlope', 25 | 'Neighborhood', 'Condition1', 'Condition2', 'BldgType', 'HouseStyle', 26 | 'OverallQual', 'OverallCond', 'YearBuilt', 'YearRemodAdd', 'RoofStyle', 27 | 'RoofMatl', 'Exterior1st', 'Exterior2nd', 'MasVnrType', 'MasVnrArea', 28 | 'ExterQual', 'ExterCond', 'Foundation', 'BsmtQual', 'BsmtCond', 29 | 'BsmtExposure', 'BsmtFinType1', 'BsmtFinSF1', 'BsmtFinType2', 'BsmtFinSF2', 30 | 'BsmtUnfSF', 'TotalBsmtSF', 'Heating', 'HeatingQC', 'CentralAir', 31 | 'Electrical', '1stFlrSF', '2ndFlrSF', 'LowQualFinSF', 'GrLivArea', 32 | 'BsmtFullBath', 'BsmtHalfBath', 'FullBath', 'HalfBath', 'BedroomAbvGr', 33 | 'KitchenAbvGr', 'KitchenQual', 'TotRmsAbvGrd', 'Functional', 'Fireplaces', 34 | 'FireplaceQu', 'GarageType', 'GarageYrBlt', 'GarageFinish', 'GarageCars', 35 | 'GarageArea', 'GarageQual', 'GarageCond', 'PavedDrive', 'WoodDeckSF', 36 | 'OpenPorchSF', 'EnclosedPorch', '3SsnPorch', 'ScreenPorch', 'PoolArea', 37 | 'PoolQC', 'Fence', 'MiscFeature', 'MiscVal', 'MoSold', 'YrSold', 38 | 'SaleType', 'SaleCondition', 'SalePrice', 39 | ]); 40 | 41 | $dataset = Labeled::fromIterator($extractor); 42 | 43 | $dataset->apply(new NumericStringConverter()) 44 | ->apply(new MissingDataImputer()) 45 | ->transformLabels('intval'); 46 | 47 | $estimator = new PersistentModel( 48 | new GradientBoost(new RegressionTree(4), 0.1), 49 | new Filesystem('housing.rbx', true) 50 | ); 51 | 52 | $estimator->setLogger($logger); 53 | 54 | $estimator->train($dataset); 55 | 56 | $extractor = new CSV('progress.csv', true); 57 | 58 | $extractor->export($estimator->steps()); 59 | 60 | $logger->info('Progress saved to progress.csv'); 61 | 62 | if (strtolower(readline('Save this model? (y|[n]): ')) === 'y') { 63 | $estimator->save(); 64 | } 65 | --------------------------------------------------------------------------------