├── Linear regression with one variable ├── ex1.m ├── plotData.m ├── computeCost.m ├── warmUpExercise.m ├── gradientDescent.m └── ex1data1.txt ├── Linear regression with multiple variables ├── ex1_multi.m ├── normalEqn.m ├── computeCostMulti.m ├── featureNormalize.m ├── gradientDescentMulti.m └── ex1data2.txt ├── .gitignore └── README.md /Linear regression with one variable/ex1.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with one variable/ex1.m -------------------------------------------------------------------------------- /Linear regression with one variable/plotData.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with one variable/plotData.m -------------------------------------------------------------------------------- /Linear regression with one variable/computeCost.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with one variable/computeCost.m -------------------------------------------------------------------------------- /Linear regression with one variable/warmUpExercise.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with one variable/warmUpExercise.m -------------------------------------------------------------------------------- /Linear regression with multiple variables/ex1_multi.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with multiple variables/ex1_multi.m -------------------------------------------------------------------------------- /Linear regression with multiple variables/normalEqn.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with multiple variables/normalEqn.m -------------------------------------------------------------------------------- /Linear regression with one variable/gradientDescent.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with one variable/gradientDescent.m -------------------------------------------------------------------------------- /Linear regression with multiple variables/computeCostMulti.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with multiple variables/computeCostMulti.m -------------------------------------------------------------------------------- /Linear regression with multiple variables/featureNormalize.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with multiple variables/featureNormalize.m -------------------------------------------------------------------------------- /Linear regression with multiple variables/gradientDescentMulti.m: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Shuai-Xie/Linear-Regression/HEAD/Linear regression with multiple variables/gradientDescentMulti.m -------------------------------------------------------------------------------- /Linear regression with multiple variables/ex1data2.txt: -------------------------------------------------------------------------------- 1 | 2104,3,399900 2 | 1600,3,329900 3 | 2400,3,369000 4 | 1416,2,232000 5 | 3000,4,539900 6 | 1985,4,299900 7 | 1534,3,314900 8 | 1427,3,198999 9 | 1380,3,212000 10 | 1494,3,242500 11 | 1940,4,239999 12 | 2000,3,347000 13 | 1890,3,329999 14 | 4478,5,699900 15 | 1268,3,259900 16 | 2300,4,449900 17 | 1320,2,299900 18 | 1236,3,199900 19 | 2609,4,499998 20 | 3031,4,599000 21 | 1767,3,252900 22 | 1888,2,255000 23 | 1604,3,242900 24 | 1962,4,259900 25 | 3890,3,573900 26 | 1100,3,249900 27 | 1458,3,464500 28 | 2526,3,469000 29 | 2200,3,475000 30 | 2637,3,299900 31 | 1839,2,349900 32 | 1000,1,169900 33 | 2040,4,314900 34 | 3137,3,579900 35 | 1811,4,285900 36 | 1437,3,249900 37 | 1239,3,229900 38 | 2132,4,345000 39 | 4215,4,549000 40 | 2162,4,287000 41 | 1664,2,368500 42 | 2238,3,329900 43 | 2567,4,314000 44 | 1200,3,299000 45 | 852,2,179900 46 | 1852,4,299900 47 | 1203,3,239500 48 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | # Created by https://www.gitignore.io/api/macos,matlab 3 | 4 | ### macOS ### 5 | *.DS_Store 6 | .AppleDouble 7 | .LSOverride 8 | 9 | # Icon must end with two \r 10 | Icon 11 | 12 | # Thumbnails 13 | ._* 14 | 15 | # Files that might appear in the root of a volume 16 | .DocumentRevisions-V100 17 | .fseventsd 18 | .Spotlight-V100 19 | .TemporaryItems 20 | .Trashes 21 | .VolumeIcon.icns 22 | .com.apple.timemachine.donotpresent 23 | 24 | # Directories potentially created on remote AFP share 25 | .AppleDB 26 | .AppleDesktop 27 | Network Trash Folder 28 | Temporary Items 29 | .apdisk 30 | 31 | ### Matlab ### 32 | ##--------------------------------------------------- 33 | ## Remove autosaves generated by the Matlab editor 34 | ## We have git for backups! 35 | ##--------------------------------------------------- 36 | 37 | # Windows default autosave extension 38 | *.asv 39 | 40 | # OSX / *nix default autosave extension 41 | *.m~ 42 | 43 | # Compiled MEX binaries (all platforms) 44 | *.mex* 45 | 46 | # Simulink Code Generation 47 | slprj/ 48 | 49 | # Session info 50 | octave-workspace 51 | 52 | # Simulink autosave extension 53 | *.autosave 54 | 55 | # End of https://www.gitignore.io/api/macos,matlab -------------------------------------------------------------------------------- /Linear regression with one variable/ex1data1.txt: -------------------------------------------------------------------------------- 1 | 6.1101,17.592 2 | 5.5277,9.1302 3 | 8.5186,13.662 4 | 7.0032,11.854 5 | 5.8598,6.8233 6 | 8.3829,11.886 7 | 7.4764,4.3483 8 | 8.5781,12 9 | 6.4862,6.5987 10 | 5.0546,3.8166 11 | 5.7107,3.2522 12 | 14.164,15.505 13 | 5.734,3.1551 14 | 8.4084,7.2258 15 | 5.6407,0.71618 16 | 5.3794,3.5129 17 | 6.3654,5.3048 18 | 5.1301,0.56077 19 | 6.4296,3.6518 20 | 7.0708,5.3893 21 | 6.1891,3.1386 22 | 20.27,21.767 23 | 5.4901,4.263 24 | 6.3261,5.1875 25 | 5.5649,3.0825 26 | 18.945,22.638 27 | 12.828,13.501 28 | 10.957,7.0467 29 | 13.176,14.692 30 | 22.203,24.147 31 | 5.2524,-1.22 32 | 6.5894,5.9966 33 | 9.2482,12.134 34 | 5.8918,1.8495 35 | 8.2111,6.5426 36 | 7.9334,4.5623 37 | 8.0959,4.1164 38 | 5.6063,3.3928 39 | 12.836,10.117 40 | 6.3534,5.4974 41 | 5.4069,0.55657 42 | 6.8825,3.9115 43 | 11.708,5.3854 44 | 5.7737,2.4406 45 | 7.8247,6.7318 46 | 7.0931,1.0463 47 | 5.0702,5.1337 48 | 5.8014,1.844 49 | 11.7,8.0043 50 | 5.5416,1.0179 51 | 7.5402,6.7504 52 | 5.3077,1.8396 53 | 7.4239,4.2885 54 | 7.6031,4.9981 55 | 6.3328,1.4233 56 | 6.3589,-1.4211 57 | 6.2742,2.4756 58 | 5.6397,4.6042 59 | 9.3102,3.9624 60 | 9.4536,5.4141 61 | 8.8254,5.1694 62 | 5.1793,-0.74279 63 | 21.279,17.929 64 | 14.908,12.054 65 | 18.959,17.054 66 | 7.2182,4.8852 67 | 8.2951,5.7442 68 | 10.236,7.7754 69 | 5.4994,1.0173 70 | 20.341,20.992 71 | 10.136,6.6799 72 | 7.3345,4.0259 73 | 6.0062,1.2784 74 | 7.2259,3.3411 75 | 5.0269,-2.6807 76 | 6.5479,0.29678 77 | 7.5386,3.8845 78 | 5.0365,5.7014 79 | 10.274,6.7526 80 | 5.1077,2.0576 81 | 5.7292,0.47953 82 | 5.1884,0.20421 83 | 6.3557,0.67861 84 | 9.7687,7.5435 85 | 6.5159,5.3436 86 | 8.5172,4.2415 87 | 9.1802,6.7981 88 | 6.002,0.92695 89 | 5.5204,0.152 90 | 5.0594,2.8214 91 | 5.7077,1.8451 92 | 7.6366,4.2959 93 | 5.8707,7.2029 94 | 5.3054,1.9869 95 | 8.2934,0.14454 96 | 13.394,9.0551 97 | 5.4369,0.61705 98 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [Linear regression with one variable - 简书](http://www.jianshu.com/p/31f9f6aa4b16) 2 | 3 | [Linear regression with multiple variables - 简书](http://www.jianshu.com/p/d6b0d0f044e7) 4 | 5 | 多变量线性回归 预测房价 6 | 7 | ex1data2.txt（房屋尺寸，卧室数量，房屋价格） 8 | 9 | ``` 10 | 2104,3,399900 11 | 1600,3,329900 12 | 2400,3,369000 13 | 1416,2,232000 14 | 3000,4,539900 15 | 1985,4,299900 16 | …… 17 | ``` 18 | 19 | ### Part 1: Feature Normalization 20 | 对于多维特征，要使用特征缩放保证其具有相近尺度，这样梯度下降算法会更快收敛。这个例子中，房屋尺寸是卧室数量接近1000倍。 21 | 22 | ```matlab 23 | %% ================ Part 1: Feature Normalization ================ 24 | fprintf('Loading data ...\n'); 25 | 26 | data = load('ex1data2.txt'); 27 | X = data(:, 1:2); % 房屋特征2个 (m*2) 28 | y = data(:, 3); % 价格 (m*1) 29 | 30 | m = length(y); % 样本数量 31 | 32 | % 输出10个样本点 33 | fprintf('First 10 examples from the dataset: \n'); 34 | 35 | % X 2个特征，y 1个价格 36 | % 因为fprintf按列输出矩阵，所以要将矩阵先转置，使得每一列对应一个实例 37 | fprintf(' x = [%.0f %.0f], y = %.0f \n', [X(1:10,:) y(1:10,:)]'); 38 | % fprintf(' x = [%.0f %.0f], y = %.0f \n', data(1:10, :)'); % 也可以 39 | 40 | fprintf('Program paused. Press enter to continue.\n'); 41 | pause; 42 | 43 | % 特征缩放 44 | fprintf('Normalizing Features ...\n'); 45 | % X是特征缩放后的特征矩阵，mu是平均值矩阵，sigma是方差矩阵 46 | [X, mu, sigma] = featureNormalize(X); 47 | X = [ones(m, 1), X]; % X 补充 theta0 48 | 49 | % 格式化输出特征缩放后的矩阵 50 | fprintf(' x = [%.6f %.6f %.6f], y = %.0f \n', [X(1:10,:) y(1:10,:)]'); 51 | fprintf('Program paused. Press enter to continue.\n'); 52 | pause; 53 | ``` 54 | ``` 55 | Loading data ... 56 | First 10 examples from the dataset: 57 | x = [2104 3], y = 399900 58 | x = [1600 3], y = 329900 59 | x = [2400 3], y = 369000 60 | x = [1416 2], y = 232000 61 | x = [3000 4], y = 539900 62 | x = [1985 4], y = 299900 63 | x = [1534 3], y = 314900 64 | x = [1427 3], y = 198999 65 | x = [1380 3], y = 212000 66 | x = [1494 3], y = 242500 67 | Program paused. Press enter to continue. 68 | Normalizing Features ... 69 | x = [1.000000 0.130010 -0.223675], y = 399900 70 | x = [1.000000 -0.504190 -0.223675], y = 329900 71 | x = [1.000000 0.502476 -0.223675], y = 369000 72 | x = [1.000000 -0.735723 -1.537767], y = 232000 73 | x = [1.000000 1.257476 1.090417], y = 539900 74 | x = [1.000000 -0.019732 1.090417], y = 299900 75 | x = [1.000000 -0.587240 -0.223675], y = 314900 76 | x = [1.000000 -0.721881 -0.223675], y = 198999 77 | x = [1.000000 -0.781023 -0.223675], y = 212000 78 | x = [1.000000 -0.637573 -0.223675], y = 242500 79 | Program paused. Press enter to continue. 80 | ``` 81 | featureNormalize 函数 82 | ```matlab 83 | function [X_norm, mu, sigma] = featureNormalize(X) 84 | % 标准化 X 的特征到[0,1]之间 85 | 86 | % 初始化 87 | X_norm = X; % X(47*2) 88 | 89 | % 利用matlab自带函数mean，std求平均值和标准差 90 | mu = mean(X); % mu(1*2) a row vector containing the mean value of each column. 91 | sigma = std(X); % sigma(1*2) 92 | 93 | % size(X, 1) 获得 X 矩阵的行数，即样本数 = 47 94 | % size(X, 2) 获得 X 矩阵的列数，即特征数 = 2 95 | m = size(X, 1); 96 | 97 | % 缩放 X 的 2 个特征 98 | for i = 1:m 99 | X_norm(i, 1) = ( X_norm(i, 1) - mu(1) ) / sigma(1); 100 | X_norm(i, 2) = ( X_norm(i, 2) - mu(2) ) / sigma(2); 101 | end 102 | 103 | end 104 | ``` 105 | 106 | ![特征缩放](http://upload-images.jianshu.io/upload_images/1877813-6e885a87f831981d.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240) 107 | 108 | ### Part 2: Gradient Descent 109 | ```matlab 110 | %% ================ Part 2: Gradient Descent ================ 111 | fprintf('Running gradient descent ...\n'); 112 | 113 | % Choose some alpha value 3个学习率 114 | alpha = 0.01; % 学习率 115 | alpha1 = 0.03; 116 | alpha2 = 0.1; 117 | num_iters = 400; % 迭代次数 118 | 119 | % Init Theta and Run Gradient Descent 120 | theta = zeros(3, 1); % 3个特征 121 | theta1 = zeros(3, 1); 122 | theta2 = zeros(3, 1); 123 | [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters); % 多变量梯度下降 124 | [theta1, J_history1] = gradientDescentMulti(X, y, theta1, alpha1, num_iters); 125 | [theta2, J_history2] = gradientDescentMulti(X, y, theta2, alpha2, num_iters); 126 | 127 | % Plot the convergence graph 128 | figure; 129 | plot(1:numel(J_history), J_history, '-b', 'LineWidth', 2); 130 | hold on; % 继续画图 131 | plot(1:numel(J_history1), J_history1, '-r', 'LineWidth', 2); 132 | plot(1:numel(J_history2), J_history2, '-k', 'LineWidth', 2); 133 | % numel: number of elements 返回元素个数，作为x轴迭代次数 134 | xlabel('Number of iterations'); 135 | ylabel('Cost J'); 136 | legend('alpha = 0.01', 'alpha = 0.03', 'alpha = 0.1'); % 图例 137 | 138 | % 显示theta值 139 | fprintf('Theta computed from gradient descent: \n'); 140 | fprintf(' %f \n', theta); % 3*1 141 | 142 | % 估计房屋价格 1650 sq-ft, 3 br house 143 | % x1不需要特征缩放为1，将后面的x2，x3特征缩放 144 | % price = 0; % You should change this 145 | price = [1, (1650 - mu(1,1)) / sigma(1,1), (3 - mu(1,2)) / sigma(1,2)] * theta; 146 | 147 | fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ... 148 | '(using gradient descent): $%f\n'], price); 149 | 150 | fprintf('Program paused. Press enter to continue.\n'); 151 | pause; 152 | ``` 153 | ``` 154 | Running gradient descent ... 155 | Theta computed from gradient descent: 156 | 334302.063993 157 | 100087.116006 158 | 3673.548451 159 | Predicted price of a 1650 sq-ft, 3 br house (using gradient descent): $289314.620338 160 | Program paused. Press enter to continue. 161 | ``` 162 | 163 | 164 | ![不同学习率下的收敛曲线](http://upload-images.jianshu.io/upload_images/1877813-9dff72d194b0d358.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240) 165 | 166 | gradientDescentMulti 函数 167 | ```matlab 168 | function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters) 169 | 170 | % Initialize some useful values 171 | J_history = zeros(num_iters, 1); 172 | m = length(y); 173 | 174 | % 迭代 175 | for iter = 1:num_iters 176 | 177 | tmp = (X' * ( X * theta - y )); % 向量运算 求和项 (3*m)*(m*1)=(3*1) 178 | theta = theta - alpha / m * tmp; 179 | 180 | % Save the cost J in every iteration 181 | J_history(iter) = computeCostMulti(X, y, theta); % 根据theta求J 182 | 183 | end 184 | ``` 185 | computeCostMulti 函数 186 | ```matlab 187 | function J = computeCostMulti(X, y, theta) 188 | 189 | % Initialize some useful values 190 | m = length(y); % number of training examples 191 | 192 | E = X * theta - y; % 偏差矩阵(m*1) 193 | J = (E' * E) / (2 * m); % (E'*E) 方差和 194 | 195 | end 196 | ``` 197 | ### Part 3: Normal Equations 198 | 正规方程求 ***θ*** 199 | ```matlab 200 | %% ================ Part 3: Normal Equations ================ 201 | fprintf('Solving with normal equations...\n'); 202 | 203 | % Load Data 204 | data = csvread('ex1data2.txt'); % load也可以 205 | X = data(:, 1:2); 206 | y = data(:, 3); 207 | m = length(y); 208 | 209 | % Add intercept term to X 210 | X = [ones(m, 1) X]; 211 | 212 | % Calculate the parameters from the normal equation 213 | theta = normalEqn(X, y); 214 | 215 | % Display normal equation result 216 | fprintf('Theta computed from the normal equations: \n'); 217 | fprintf(' %f \n', theta); 218 | 219 | % Estimate the price of a 1650 sq-ft, 3 br house 220 | % 不需要特征缩放 221 | price = [1, 1650, 3] * theta; 222 | 223 | fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ... 224 | '(using normal equations): $%f\n'], price); 225 | ``` 226 | 预测结果与梯度下降法有出入，正规预测的是准确值，梯度下降是递进值。 227 | ``` 228 | Solving with normal equations... 229 | Theta computed from the normal equations: 230 | 89597.909544 231 | 139.210674 232 | -8738.019113 233 | Predicted price of a 1650 sq-ft, 3 br house (using normal equations): $293081.464335 234 | ``` 235 | normalEqn 函数 236 | ```matlab 237 | function [theta] = normalEqn(X, y) 238 | 239 | theta = pinv(X' * X) * X' * y; % 调用方程 240 | 241 | end 242 | ``` 243 | --------------------------------------------------------------------------------