├── README.md ├── lecture-notes ├── cs229-evaluation-metrics-slides.pptx ├── cs229-intro-slides.pptx ├── cs229-linalg.pdf ├── cs229-notes-deep_learning.pdf ├── cs229-notes1.pdf ├── cs229-notes10.pdf ├── cs229-notes11.pdf ├── cs229-notes12.pdf ├── cs229-notes2.pdf ├── cs229-notes3.pdf ├── cs229-notes4.pdf ├── cs229-notes5.pdf ├── cs229-notes7a.pdf ├── cs229-notes7b.pdf ├── cs229-notes8.pdf ├── cs229-notes9.pdf ├── cs229-prob-slide.pdf └── cs229-prob.pdf ├── materials ├── BiasVarianceAnalysis.pdf ├── MaxEnt.pdf ├── gaussian_processes.pdf ├── gaussians.pdf └── more_on_gaussians.pdf ├── problem-sets-solutions ├── README.md ├── ps1-sol.pdf ├── ps1-sol.zip ├── ps2-sol.pdf ├── ps2-sol.zip ├── ps3-sol.pdf └── ps3-sol.zip ├── problem-sets ├── ps1.pdf ├── ps1.zip ├── ps2.pdf ├── ps2.zip ├── ps3.pdf └── ps3.zip └── syllabus.html /README.md: -------------------------------------------------------------------------------- 1 | # CS229 Summer 2019 2 | 3 | All lecture notes, slides and assignments for [CS229: Machine Learning](http://cs229.stanford.edu/) course by Stanford University. 4 | 5 | The videos of all lectures are available [on YouTube](https://www.youtube.com/playlist?list=PLoROMvodv4rNH7qL6-efu_q2_bPuy0adh). 6 | 7 | Useful links: 8 | - [CS229 Autumn 2018 edition](https://github.com/maxim5/cs229-2018-autumn) 9 | -------------------------------------------------------------------------------- /lecture-notes/cs229-evaluation-metrics-slides.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-evaluation-metrics-slides.pptx -------------------------------------------------------------------------------- /lecture-notes/cs229-intro-slides.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-intro-slides.pptx -------------------------------------------------------------------------------- /lecture-notes/cs229-linalg.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-linalg.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes-deep_learning.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes-deep_learning.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes1.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes10.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes10.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes11.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes11.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes12.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes12.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes2.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes3.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes4.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes4.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes5.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes7a.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes7a.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes7b.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes7b.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes8.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes8.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-notes9.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-notes9.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-prob-slide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-prob-slide.pdf -------------------------------------------------------------------------------- /lecture-notes/cs229-prob.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/lecture-notes/cs229-prob.pdf -------------------------------------------------------------------------------- /materials/BiasVarianceAnalysis.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/materials/BiasVarianceAnalysis.pdf -------------------------------------------------------------------------------- /materials/MaxEnt.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/materials/MaxEnt.pdf -------------------------------------------------------------------------------- /materials/gaussian_processes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/materials/gaussian_processes.pdf -------------------------------------------------------------------------------- /materials/gaussians.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/materials/gaussians.pdf -------------------------------------------------------------------------------- /materials/more_on_gaussians.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/materials/more_on_gaussians.pdf -------------------------------------------------------------------------------- /problem-sets-solutions/README.md: -------------------------------------------------------------------------------- 1 | # Problem sets solutions 2 | 3 | Solutions taken from [this repo](https://github.com/kumi123/CS229). 4 | -------------------------------------------------------------------------------- /problem-sets-solutions/ps1-sol.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets-solutions/ps1-sol.pdf -------------------------------------------------------------------------------- /problem-sets-solutions/ps1-sol.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets-solutions/ps1-sol.zip -------------------------------------------------------------------------------- /problem-sets-solutions/ps2-sol.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets-solutions/ps2-sol.pdf -------------------------------------------------------------------------------- /problem-sets-solutions/ps2-sol.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets-solutions/ps2-sol.zip -------------------------------------------------------------------------------- /problem-sets-solutions/ps3-sol.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets-solutions/ps3-sol.pdf -------------------------------------------------------------------------------- /problem-sets-solutions/ps3-sol.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets-solutions/ps3-sol.zip -------------------------------------------------------------------------------- /problem-sets/ps1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets/ps1.pdf -------------------------------------------------------------------------------- /problem-sets/ps1.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets/ps1.zip -------------------------------------------------------------------------------- /problem-sets/ps2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets/ps2.pdf -------------------------------------------------------------------------------- /problem-sets/ps2.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets/ps2.zip -------------------------------------------------------------------------------- /problem-sets/ps3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets/ps3.pdf -------------------------------------------------------------------------------- /problem-sets/ps3.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maxim5/cs229-2019-summer/798be99270e4c4e18249d6a70d70588ff263cf49/problem-sets/ps3.zip -------------------------------------------------------------------------------- /syllabus.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | CS229: Machine Learning 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 33 | 34 | 35 |
36 |
37 |

Syllabus and Course Schedule

38 |

39 | [Previous offerings: Autumn 2018, Spring 2019]

40 |
41 |
42 |
43 | 44 | 45 |
46 | * Below is a collection of topics, of which we plan to cover a large subset this quarter. The specific topics and the order is subject to change. 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 59 | 60 | 67 | 68 | 69 | 70 | 71 | 82 | 83 | 84 | 85 | 86 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 112 | 113 | 114 | 115 | 116 | 125 | 126 | 127 | 128 | 129 | 135 |
CategoryTopic
Review 61 |
    62 |
  • Linear Algebra 63 |
  • Matrix Calculus 64 |
  • Probability and Statistics 65 |
66 |
Supervised Learning
    72 |
  • Linear Regression (Gradient Descent, Normal Equations) 73 |
  • Weighted Linear Regression (LWR) 74 |
  • Logistic Regression, Perceptron 75 |
  • Newton's Method, KL-divergence, (cross-)Entropy, Natural Gradient 76 |
  • Exponential Family and Generalized Linear Models 77 |
  • Generative Models (Gaussian Discriminant Analysis, Naive Bayes) 78 |
  • Kernel Method (SVM, Gaussian Processes) 79 |
  • Tree Ensembles (Decision trees, Random Forests, Boosting and Gradient Boosting) 80 |
81 |
Learning Theory
    87 |
  • Regularization 88 |
  • Bias-Variance Decomposition and Tradeoff 89 |
  • Concentration Inequalities 90 |
  • Generalization and Uniform Convergence 91 |
  • VC-dimension 92 |
93 |
Deep Learning
  • Neural Networks
  • Backpropagation
  • Deep Architectures
Unsupervised Learning
    104 |
  • K-means 105 |
  • Gaussian Mixture Model (GMM) 106 |
  • Expectation Maximization (EM) 107 |
  • Variational Auto-encoder (VAE) 108 |
  • Factor Analysis 109 |
  • Principal Components Analysis (PCA) 110 |
  • Independent Components Analysis (ICA) 111 |
Reinforcement Learning (RL) 117 |
    118 |
  • Markov Decision Processes (MDP) 119 |
  • Bellmans Equations 120 |
  • Value Iteration and Policy Iteration 121 |
  • Value Function Approximation 122 |
  • Q-Learning 123 |
124 |
Application 130 |
    131 |
  • Advice on structuring an ML project 132 |
  • Evaluation Metrics 133 |
134 |
136 | 137 |
138 | 139 |
140 | 141 | This table will be updated regularly through the quarter to reflect what was actually covered, along with corresponding readings and notes. 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 154 | 155 | 156 | 157 | 159 | 168 | 169 | 170 | 171 | 172 | 173 | 179 | 187 | 188 | 189 | 190 | 191 | 192 | 198 | 205 | 206 | 207 | 208 | 209 | 210 | 219 | 225 | 226 | 227 | 228 | 229 | 230 | 237 | 243 | 244 | 245 | 246 | 247 | 248 | 254 | 260 | 261 | 262 | 263 | 264 | 265 | 272 | 278 | 279 | 280 | 281 | 282 | 283 | 289 | 295 | 296 | 297 | 298 | 299 | 300 | 303 | 314 | 315 | 316 | 317 | 318 | 319 | 322 | 332 | 333 | 334 | 335 | 336 | 337 | 342 | 345 | 346 | 347 | 348 | 349 | 350 | 357 | 363 | 364 | 365 | 366 | 367 | 368 | 374 | 381 | 382 | 383 | 384 | 385 | 386 | 393 | 399 | 400 | 401 | 402 | 403 | 404 | 411 | 417 | 418 | 419 | 420 | 421 | 422 | 430 | 438 | 439 | 440 | 441 | 442 | 443 | 449 | 456 | 457 | 458 | 459 | 460 | 461 | 468 | 475 | 476 | 477 | 478 | 479 | 480 | 487 | 493 | 494 | 495 | 496 | 497 | 498 | 505 | 511 | 512 | 513 | 514 | 515 | 516 | 521 | 527 | 528 | 529 | 530 | 531 | 532 | 538 | 541 | 542 | 543 | 544 | 545 | 546 | 551 | 554 | 555 | 556 | 557 | 558 | 559 | 560 | 561 | 562 | 563 | 564 | 565 | 566 | 573 | 574 | 575 | 582 | 583 | 584 | 585 | 595 | 596 | 597 | 598 |
EventDateDescriptionMaterials and Assignments
Lecture 1 6/24
  • Introduction and Logistics
  • Review of Linear Algebra
    • 158 |
160 | 161 | Class Notes 162 |
    163 |
  • Introduction [pptx]
    164 |
  • Linear Algebra (section 1-3) [pdf] 165 |
166 | 167 |
Lecture 26/26 174 |
    175 |
  • Review of Matrix Calculus 176 |
  • Review of Probability 177 |
178 | 		180 | Class Notes 181 |
    182 |
  • Linear Algebra (section 4) [pdf] 183 |
  • Probability Theory [pdf] 184 |
  • Probability Theory Slides [pdf] 185 |
186 |
Lecture 36/28 193 |
    194 |
  • Review of Probability and Statistics 195 |
  • Setting of Supervised Learning 196 |
197 | 		199 | Class Notes 200 |
    201 |
  • Supervised Learning [pdf] 202 |
  • Probability Theory [pdf] 203 |
204 |
Lecture 47/1 211 |
    212 |
  • Linear Regression 213 |
  • Gradient Descent (GD), Stochastic Gradient Descent (SGD) 214 |
  • Normal Equations 215 |
  • Probabilistic Interpretation 216 |
  • Maximum Likelihood Estimation (MLE) 217 |
218 | 		220 | Class Notes 221 |
    222 |
  • Supervised Learning (section 1-3) [pdf] 223 |
224 |
Lecture 57/3 231 |
    232 |
  • Perceptron 233 |
  • Logistic Regression 234 |
  • Newton's Method 235 |
236 | 		238 | Class Notes 239 |
    240 |
  • Supervised Learning (section 5-7) [pdf] 241 |
242 |
Lecture 67/5 249 |
    250 |
  • Exponential Family 251 |
  • Generalized Linear Models (GLM) 252 |
253 | 		255 | Class Notes 256 |
    257 |
  • Supervised Learning (section 8-9) [pdf] 258 |
259 |
Lecture 77/8 266 |
    267 |
  • Gaussian Discriminant Analysis (GDA) 268 |
  • Naive Bayes 269 |
  • Laplace Smoothing 270 |
271 | 		273 | Class Notes 274 |
    275 |
  • Generative Algorithms [pdf] 276 |
277 |
Lecture 87/10 284 |
    285 |
  • Kernel Methods 286 |
  • Support Vector Machine 287 |
288 | 		290 | Class Notes 291 |
    292 |
  • Kernel Methods and SVM [pdf] 293 |
294 |
Lecture 97/12 301 |
  • Gaussian Processes
302 | 		304 | Class Notes 305 |
    306 |
  • Gaussian Processes [pdf]
    • 307 |
308 | Optional 309 |
    310 |
  • The Multivariate Gaussian Distribution [pdf]
    • 311 |
  • More on Gaussian Distribution [pdf]
    • 312 |
313 |
Lecture 107/15 320 |
  • Neural Networks and Deep Learning
321 | 		323 | Class Notes 324 |
    325 |
  • Deep Learning (skip Sec 3.3) [pdf] 326 |
327 | Optional 328 |
    329 |
  • Backpropagation [pdf] 330 |
331 |
Lecture 117/17 338 |
    339 |
  • Deep Learning (contd) 340 |
341 | 		343 | 344 |
Lecture 127/19 351 |
    352 |
  • Bias and Variance 353 |
  • Regularization, Bayesian Interpretation 354 |
  • Model Selection 355 |
356 | 		358 | Class Notes 359 |
    360 |
  • Regularization and Model Selection [pdf] 361 |
362 |
Lecture 137/22 369 |
    370 |
  • Bias-Variance tradeoff (wrap-up) 371 |
  • Uniform Convergence 372 |
373 | 		375 | Class Notes 376 |
    377 |
  • Bias Variance Analysis [pdf] 378 |
  • Statistical Learning Theory [pdf] 379 |
380 |
Lecture 147/24 387 |
    388 |
  • Reinforcement Learning (RL) 389 |
  • Markov Decision Processes (MDP) 390 |
  • Value and Policy Iterations 391 |
392 | 		394 | Class Notes 395 |
    396 |
  • Reinforcement Learning and Control (Sec 1-2) [pdf] 397 |
398 |
Lecture 157/26 405 |
    406 |
  • RL (wrap-up) 407 |
  • Learning MDP model 408 |
  • Continuous States 409 |
410 | 		412 | Class Notes 413 |
    414 |
  • Reinforcement Learning and Control (Sec 3-4) [pdf] 415 |
416 |
Lecture 167/29 423 | Unsupervised Learning 424 |
    425 |
  • K-means clustering 426 |
  • Mixture of Gaussians (GMM) 427 |
  • Expectation Maximization (EM) 428 |
429 | 		431 | Class Notes 432 |
    433 |
  • K-means [pdf] 434 |
  • Mixture of Gaussians [pdf] 435 |
  • Expectation Maximization (Sec 1-2, skip 2.1) [pdf] 436 |
437 |
Lecture 177/31 444 |
    445 |
  • EM (wrap-up) 446 |
  • Factor Analysis 447 |
448 | 		450 | Class Notes 451 |
    452 |
  • Expectation Maximization (Sec 3) [pdf] 453 |
  • Factor Analysis [pdf] 454 |
455 |
Lecture 188/2 462 |
    463 |
  • Factor Analysis (wrap-up) 464 |
  • Principal Components Analysis (PCA) 465 |
  • Independent Components Analysis (ICA) 466 |
467 | 		469 | Class Notes 470 |
    471 |
  • Principal Components Analysis [pdf] 472 |
  • Independent Components Analysis [pdf] 473 |
474 |
Lecture 198/5 481 |
    482 |
  • Maximum Entropy and Exponential Family 483 |
  • KL-Divergence 484 |
  • Calibration and Proper Scoring Rules 485 |
486 | 		488 | Class Notes 489 |
    490 |
  • Maximum Entropy [pdf] 491 |
492 |
Lecture 208/7 499 |
    500 |
  • Variational Inference 501 |
  • EM Variants 502 |
  • Variational Autoencoder 503 |
504 | 		506 | Class Notes 507 |
    508 |
  • VAE (Sec 4) [pdf] 509 |
510 |
Lecture 218/9 517 |
    518 |
  • Evaluation Metrics
    • 519 |
520 | 		522 | Class Notes 523 |
    524 |
  • Evaluation Metrics [pptx] 525 |
526 |
Lecture 228/12 533 |
    534 |
  • Practical advice and tips 535 |
  • Review for Finals 536 |
537 | 		539 | Class Notes 540 |
Lecture 238/14 547 |
    548 |
  • Review for Finals 549 |
550 | 		552 | Class Notes 553 |
Final 8/16
586 | Other Resources 587 |
    588 |
  1. Advice on applying machine learning: Slides from Andrew's lecture on getting machine learning algorithms to work in practice can be found here.
    2. 589 |
  2. Previous projects: A list of last year's final projects can be found here.
    3. 590 |
  3. Data: Here is the UCI Machine learning repository, which contains a large collection of standard datasets for testing learning algorithms. If you want to see examples of recent work in machine learning, start by taking a look at the conferences NeurIPS (all old NeurIPS papers are online) and ICML. Some other related conferences include UAI, AAAI, IJCAI.
    4. 591 |
  4. Viewing PostScript and PDF files: Depending on the computer you are using, you may be able to download a PostScript viewer or PDF viewer for it if you don't already have one.
    5. 592 |
  5. Machine learning study guides tailored to CS 229 by Afshine Amidi and Shervine Amidi.
    6. 593 |
594 |
599 |
600 | 601 | 602 | 603 | 604 | --------------------------------------------------------------------------------