└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Deep Learning Mathematics Roadmap 2 | 3 | This roadmap outlines the mathematical concepts and topics covered in various deep learning resources. It provides a structured path to understand the necessary mathematical foundations for deep learning. 4 | 5 | ## Table of Contents 6 | 1. [Linear Algebra](#1-linear-algebra) 7 | 1. [Probability and Statistics](#2-probability-and-statistics) 8 | 1. [Calculus](#3-calculus) 9 | 1. [Numerical Computation](#4-numerical-computation) 10 | 1. [Machine Learning Fundamentals](#5-machine-learning-fundamentals) 11 | 1. [Deep Learning Basics](#6-deep-learning-basics) 12 | 1. [Advanced Deep Learning Topics](#7-advanced-deep-learning-topics) 13 | # 1. Linear Algebra 14 | ## - Vectors and Matrices: Vector operations, matrix operations, dot product, matrix-vector multiplication. 15 | 16 | 17 | 1. Vectors and Matrices by Math Is Fun 18 | - [Website Link](https://www.mathsisfun.com/algebra/matrix-introduction.html) 19 | 20 | 2. Vectors and Matrices Crash Course by BetterExplained 21 | - [Article Link](https://betterexplained.com/articles/linear-algebra-guide/) 22 | 23 | 3. Vectors and Matrices in MATLAB 24 | - [Tutorial Link](https://www.tutorialspoint.com/matlab/matlab_vectors.htm) 25 | 26 | 4. Introduction to Vectors and Matrices by Purplemath 27 | - [Tutorial Link](https://www.purplemath.com/modules/mtrxrows.htm) 28 | 29 | 5. Vector and Matrix Basics by MathBootCamps (YouTube video) 30 | - [Video Link](https://www.youtube.com/watch?v=QVk7TZFOY7s) 31 | 32 | 6. Linear Algebra: Vectors and Matrices by MIT OpenCourseWare 33 | - [Course Link](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/) 34 | 35 | 7. Linear Algebra: Foundations to Frontiers by edX 36 | - [Course Link](https://www.edx.org/professional-certificate/pennx-linear-algebra-foundations-to-frontiers) 37 | 38 | 8. Linear Algebra Refresher Course by Khan Academy 39 | - [Course Link](https://www.khanacademy.org/math/linear-algebra) 40 | 41 | 9. Introduction to Linear Algebra by Gilbert Strang (Textbook) 42 | - [Book Link](https://math.mit.edu/~gs/linearalgebra/) 43 | 44 | 10. Linear Algebra Done Right by Sheldon Axler (Textbook) 45 | - [Book Link](https://linear.axler.net/) 46 | 47 | 11. 3Blue1Brown: Essence of Linear Algebra (YouTube series) 48 | - [YouTube Playlist Link](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab) 49 | 50 | ## - Matrix Operations: Transpose, trace, determinant, matrix inverse, matrix rank. 51 | 52 | 53 | 1. Matrix Operations: Addition, Subtraction, Scalar Multiplication by Math Easy Solutions 54 | - [Website Link](https://matheasysolutions.com/2021/02/09/matrix-operations-addition-subtraction-scalar-multiplication/) 55 | 56 | 2. Matrix Operations: Multiplication, Transpose, and Determinant by Math CUE math 57 | - [Website Link](https://www.cuemath.com/algebra/matrix-operations/) 58 | 59 | 3. Matrix Operations by Purplemath 60 | - [Website Link](https://www.purplemath.com/modules/mtrxoprs.htm) 61 | 62 | 4. Linear Algebra Toolkit: Matrix Operations by MathsIsPower4U 63 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLZ6kagz8q0bvxaUKCe2RRvU_h7wtNNxx9) 64 | 65 | 5. Matrix Operations by MathOnlineSchool 66 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLqEfivt9ITkP3NssLCdPN1WmSS7kkjnwn) 67 | 68 | 6. Matrix Operations by Khan Academy 69 | - [Tutorial Link](https://www.khanacademy.org/math/linear-algebra/matrix-transformations/matrix-multiplication/v/matrix-multiplication-intro) 70 | 71 | 7. Matrix Operations by MathIsPower4U 72 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLC15FE3F64C4C5E13) 73 | 74 | 8. Linear Algebra: Matrix Operations by MathDoctorBob 75 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhFw4x8DEjl9IflfqF4GwGcH) 76 | 77 | 9. Matrix Operations by Krista King Math 78 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BVKErFko9je9IBZ0hXWXVtV) 79 | 80 | 10. Matrix Operations by MathPortal 81 | - [Website Link](https://www.mathportal.org/algebra/matrix/operations.php) 82 | 83 | 84 | ## - Linear Independence and Rank: Linearly independent vectors, rank of a matrix. 85 | 86 | 1. Linear Independence and Span by Khan Academy 87 | - [Tutorial Link](https://www.khanacademy.org/math/linear-algebra/vectors-and-spaces/linear-independence/v/linear-independence-example) 88 | 89 | 2. Rank of a Matrix by Math Is Fun 90 | - [Website Link](https://www.mathsisfun.com/algebra/matrix-rank.html) 91 | 92 | 3. Linear Independence and Rank by MIT OpenCourseWare 93 | - [Course Link](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/lecture-7-linear-independence-and-rank/) 94 | 95 | 4. Linear Independence and Rank by MathDoctorBob 96 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhFpDNUT2A7HjryYVpNW6i9E) 97 | 98 | 5. Linear Algebra: Introduction to Linear Independence and Span by MathBootCamps 99 | - [YouTube Video Link](https://www.youtube.com/watch?v=2nIxoR-xKbA) 100 | 101 | 6. Linear Independence and Rank by MathOnlineSchool 102 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLqEfivt9ITkOaPhXIFvR82Ng1H8N5kQlX) 103 | 104 | 7. Linear Independence and Rank by Krista King Math 105 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BVKErFko9je9IBZ0hXWXVtV) 106 | 107 | 8. Linear Algebra: Basis, Linear Independence, and Rank by MathTheBeautiful 108 | - [YouTube Video Link](https://www.youtube.com/watch?v=chFCCllGYGw) 109 | 110 | 9. Linear Independence and Rank by MathPortal 111 | - [Website Link](https://www.mathportal.org/algebra/matrices/linear-independence-and-rank.php) 112 | 113 | ## - Matrix Inverse and Pseudoinverse: Inverse matrix, pseudoinverse. 114 | 115 | 116 | 1. Matrix Inverse by Khan Academy 117 | - [Tutorial Link](https://www.khanacademy.org/math/precalculus/precalc-matrices/inverting_matrices/v/linear-algebra-inverse-of-a-matrix) 118 | 119 | 2. Pseudoinverse by Math Is Fun 120 | - [Website Link](https://www.mathsisfun.com/algebra/matrix-inverse-pseudoinverse.html) 121 | 122 | 3. Matrix Inverse and Pseudoinverse by MIT OpenCourseWare 123 | - [Course Link](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/lecture-9-matrix-inverses-and-determinants/) 124 | 125 | 4. Matrix Inverse and Pseudoinverse by MathDoctorBob 126 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhFg5AdqGR9ILm0G9f9S0UGx) 127 | 128 | 5. Matrix Inverse and Pseudoinverse by MathTheBeautiful 129 | - [YouTube Video Link](https://www.youtube.com/watch?v=8BX2_zAXMT4) 130 | 131 | 6. Matrix Inversion and Pseudoinverse by MathOnlineSchool 132 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLqEfivt9ITkOCEXKviqRG1fHEgxf_9v5V) 133 | 134 | 7. Pseudoinverse by Krista King Math 135 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BVSvDJBXo-uOgGwcE8fTi-x) 136 | 137 | 8. Matrix Inverse and Pseudoinverse by MathPortal 138 | - [Website Link](https://www.mathportal.org/algebra/matrices/matrix-inverse-and-pseudoinverse.php) 139 | 140 | ## - Eigendecomposition and Diagonalization: Eigenvalues, eigenvectors, eigendecomposition, diagonalization. 141 | 142 | 143 | 1. Eigenvectors and Eigenvalues by Khan Academy 144 | - [Tutorial Link](https://www.khanacademy.org/math/linear-algebra/alternate-bases/eigen-everything/v/linear-algebra-introduction-to-eigenvalues-and-eigenvectors) 145 | 146 | 2. Eigendecomposition by Math Is Fun 147 | - [Website Link](https://www.mathsisfun.com/algebra/diagonalization.html) 148 | 149 | 3. Eigendecomposition and Diagonalization by MIT OpenCourseWare 150 | - [Course Link](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/lecture-22-eigenvalues-and-eigenvectors/) 151 | 152 | 4. Eigenvectors, Eigenvalues, and Diagonalization by MathDoctorBob 153 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhE8oGRC6lHK5VX4GuizpZ_o) 154 | 155 | 5. Eigenvalues and Eigenvectors by 3Blue1Brown 156 | - [YouTube Video Link](https://www.youtube.com/watch?v=PFDu9oVAE-g) 157 | 158 | 6. Eigendecomposition and Diagonalization by MathTheBeautiful 159 | - [YouTube Video Link](https://www.youtube.com/watch?v=slO5YVK5Gz8) 160 | 161 | 7. Eigenvectors and Eigenvalues by MathOnlineSchool 162 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLqEfivt9ITkO_3j9_V7ko84yEAcIof0wW) 163 | 164 | 8. Eigendecomposition and Diagonalization by Krista King Math 165 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BV6UMedecvzfrcR9V9X9L7r) 166 | 167 | 9. Eigendecomposition and Diagonalization by MathPortal 168 | - [Website Link](https://www.mathportal.org/algebra/matrices/eigenvalue-decomposition-diagonalization.php) 169 | 170 | ## - Singular Value Decomposition (SVD): SVD theorem, SVD computation, low-rank approximation. 171 | 172 | 173 | 1. Singular Value Decomposition (SVD) by Khan Academy 174 | - [Tutorial Link](https://www.khanacademy.org/math/linear-algebra/matrix-transformations/singular-value-decomposition/v/singular-value-decomposition-svd-introduction-part-1) 175 | 176 | 2. Singular Value Decomposition (SVD) by Math Is Fun 177 | - [Website Link](https://www.mathsisfun.com/algebra/matrix-decomposition-singular-value.html) 178 | 179 | 3. Singular Value Decomposition (SVD) by MIT OpenCourseWare 180 | - [Course Link](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/lecture-24-singular-value-decomposition/) 181 | 182 | 4. Singular Value Decomposition (SVD) by MathDoctorBob 183 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhFsC5jnzX_JzKj0ko4WxDs9) 184 | 185 | 5. Singular Value Decomposition (SVD) by MathTheBeautiful 186 | - [YouTube Video Link](https://www.youtube.com/watch?v=H1sdrDtqOXQ) 187 | 188 | 6. Singular Value Decomposition (SVD) by MathOnlineSchool 189 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLqEfivt9ITkONzDH3wjHqcOVjKJApLIQo) 190 | 191 | 7. Singular Value Decomposition (SVD) by Krista King Math 192 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BUlQeriB9F1b5aVJnW4vs_y) 193 | 194 | 8. Singular Value Decomposition (SVD) by MathPortal 195 | - [Website Link](https://www.mathportal.org/algebra/matrices/singular-value-decomposition-svd.php) 196 | 197 | 198 | # 2. Probability and Statistics 199 | ## - Probability Basics: Sample space, events, probability axioms, conditional probability, Bayes' rule. 200 | s 201 | 202 | 1. Probability Basics by Khan Academy 203 | - [Tutorial Link](https://www.khanacademy.org/math/statistics-probability/probability-library) 204 | 205 | 2. Introduction to Probability by MIT OpenCourseWare 206 | - [Course Link](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/) 207 | 208 | 3. Probability Basics by Math Is Fun 209 | - [Website Link](https://www.mathsisfun.com/data/probability.html) 210 | 211 | 4. Probability Basics by Stat Trek 212 | - [Website Link](https://stattrek.com/probability/probability-basics.aspx) 213 | 214 | 5. Probability Basics by CrashCourse 215 | - [YouTube Video Link](https://www.youtube.com/watch?v=uzkc-qNVoOk) 216 | 217 | 6. Introduction to Probability by Khan Academy 218 | - [Course Link](https://www.khanacademy.org/math/statistics-probability/probability-library) 219 | 220 | 7. Probability Fundamentals by Udacity 221 | - [Course Link](https://www.udacity.com/course/probability-the-science-of-uncertainty-and-data--ud501) 222 | 223 | 8. Introduction to Probability and Data by Duke University (Coursera) 224 | - [Course Link](https://www.coursera.org/learn/probability-intro) 225 | 226 | 9. Probability Basics by Statistics How To 227 | - [Website Link](https://www.statisticshowto.com/probability-basics/) 228 | 229 | 10. Probability Basics by Math Goodies 230 | - [Website Link](https://www.mathgoodies.com/lessons/vol6/intro_probability) 231 | 232 | ## - Random Variables and Probability Distributions: Discrete and continuous random variables, probability mass function (PMF), probability density function (PDF). 233 | 234 | 235 | 1. Random Variables and Probability Distributions by Khan Academy 236 | - [Tutorial Link](https://www.khanacademy.org/math/ap-statistics/random-variables-ap) 237 | 238 | 2. Random Variables and Probability Distributions by MIT OpenCourseWare 239 | - [Course Link](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/) 240 | 241 | 3. Random Variables and Probability Distributions by Stat Trek 242 | - [Website Link](https://stattrek.com/probability/random-variable.aspx) 243 | 244 | 4. Random Variables and Probability Distributions by Math Is Fun 245 | - [Website Link](https://www.mathsisfun.com/data/random-variables.html) 246 | 247 | 5. Random Variables and Probability Distributions by CrashCourse 248 | - [YouTube Video Link](https://www.youtube.com/watch?v=8tYd-zruE78) 249 | 250 | 6. Random Variables and Probability Distributions by Khan Academy 251 | - [Course Link](https://www.khanacademy.org/math/statistics-probability/random-variables-stats-library) 252 | 253 | 7. Introduction to Random Variables and Probability Distributions by Rice University (Coursera) 254 | - [Course Link](https://www.coursera.org/learn/probability-intro-random-variables-distributions) 255 | 256 | 8. Random Variables and Probability Distributions by CliffsNotes 257 | - [Website Link](https://www.cliffsnotes.com/study-guides/statistics/random-variables-and-probability-distributions) 258 | 259 | 9. Introduction to Random Variables and Probability Distributions by Study.com 260 | - [Website Link](https://study.com/academy/topic/random-variables-and-probability-distributions.html) 261 | 262 | 10. Random Variables and Probability Distributions by Math Goodies 263 | - [Website Link](https://www.mathgoodies.com/lessons/vol6/random_variables) 264 | 265 | ## - Expectation, Variance, and Covariance: Expected value, variance, covariance, correlation coefficient. 266 | ## Learning Resources: Expectation, Variance, and Covariance 267 | 268 | 1. Expectation, Variance, and Covariance by Khan Academy 269 | - [Tutorial Link](https://www.khanacademy.org/math/statistics-probability/random-variables-stats-library/expected-value-library/v/random-variables-and-probability-distributions) 270 | 271 | 2. Expectation, Variance, and Covariance by Math Is Fun 272 | - [Website Link](https://www.mathsisfun.com/data/expectation.html) 273 | 274 | 3. Expectation, Variance, and Covariance by Stat Trek 275 | - [Website Link](https://stattrek.com/probability-distributions/expectation.aspx) 276 | 277 | 4. Expectation, Variance, and Covariance by MIT OpenCourseWare 278 | - [Course Link](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/) 279 | 280 | 5. Expectation, Variance, and Covariance by CrashCourse 281 | - [YouTube Video Link](https://www.youtube.com/watch?v=MRqtXL2WX2M) 282 | 283 | 6. Expectation, Variance, and Covariance by MathDoctorBob 284 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhFGggdcoHHjB0jPJ7SmcROQ) 285 | 286 | 7. Expectation, Variance, and Covariance by Krista King Math 287 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BUdfqNZk3eZ-2C-v-WP7c3N) 288 | 289 | 8. Expectation, Variance, and Covariance by Math Goodies 290 | - [Website Link](https://www.mathgoodies.com/lessons/vol6/expectations) 291 | 292 | ## - Common Probability Distributions: Uniform, Bernoulli, Binomial, Gaussian (Normal), Exponential, Poisson. 293 | ## Learning Resources: Common Probability Distributions 294 | 295 | 1. Common Probability Distributions by Khan Academy 296 | - [Tutorial Link](https://www.khanacademy.org/math/ap-statistics/random-variables-ap) 297 | 298 | 2. Common Probability Distributions by Stat Trek 299 | - [Website Link](https://stattrek.com/probability-distributions/common.aspx) 300 | 301 | 3. Common Probability Distributions by Math Is Fun 302 | - [Website Link](https://www.mathsisfun.com/data/probability-distributions.html) 303 | 304 | 4. Common Probability Distributions by CrashCourse 305 | - [YouTube Video Link](https://www.youtube.com/watch?v=JwJ8HUF2UCk) 306 | 307 | 5. Common Probability Distributions by Rice University (Coursera) 308 | - [Course Link](https://www.coursera.org/learn/probability-distributions) 309 | 310 | 6. Common Probability Distributions by MIT OpenCourseWare 311 | - [Course Link](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/) 312 | 313 | 7. Probability Distributions by Krista King Math 314 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BUZCWUOk9Zm5pBCLYO6_9Xl) 315 | 316 | 8. Introduction to Probability Distributions by Study.com 317 | - [Website Link](https://study.com/academy/topic/introduction-to-probability-distributions.html) 318 | 319 | 9. Common Probability Distributions by Math Goodies 320 | - [Website Link](https://www.mathgoodies.com/lessons/vol6/probability_distributions) 321 | 322 | ## - Bayes' Rule and Conditional Probability: Bayes' theorem, prior probability, posterior probability. 323 | ## Learning Resources: Bayes' Rule and Conditional Probability 324 | 325 | 1. Bayes' Rule and Conditional Probability by Khan Academy 326 | - [Tutorial Link](https://www.khanacademy.org/math/ap-statistics/probability-ap/stats-conditional-probability/a/bayes-theorem) 327 | 328 | 2. Bayes' Rule and Conditional Probability by Math Is Fun 329 | - [Website Link](https://www.mathsisfun.com/data/bayes-theorem.html) 330 | 331 | 3. Bayes' Rule and Conditional Probability by CrashCourse 332 | - [YouTube Video Link](https://www.youtube.com/watch?v=HZGCoVF3YvM) 333 | 334 | 4. Bayes' Rule and Conditional Probability by Stat Trek 335 | - [Website Link](https://stattrek.com/probability/bayes-theorem.aspx) 336 | 337 | 5. Bayes' Rule and Conditional Probability by MIT OpenCourseWare 338 | - [Course Link](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/) 339 | 340 | 6. Bayes' Rule and Conditional Probability by MathDoctorBob 341 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhFv5guFqph_9Lhvg3fX3Hmr) 342 | 343 | 7. Conditional Probability and Bayes' Rule by Krista King Math 344 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BUeF2o-MbXhLnmzqcVbVYas) 345 | 346 | 8. Bayes' Rule and Conditional Probability by Math Goodies 347 | - [Website Link](https://www.mathgoodies.com/lessons/vol6/conditional) 348 | 349 | 9. Bayes' Theorem by Better Explained 350 | - [Article Link](https://betterexplained.com/articles/an-intuitive-and-short-explanation-of-bayes-theorem/) 351 | 352 | 10. Bayes' Rule and Conditional Probability by Study.com 353 | - [Website Link](https://study.com/academy/topic/bayes-theorem-conditional-probability.html) 354 | 355 | ## - Information Theory: Entropy, cross-entropy, Kullback-Leibler (KL) divergence. 356 | ## Learning Resources: Information Theory 357 | 358 | 1. Information Theory by Khan Academy 359 | - [Tutorial Link](https://www.khanacademy.org/computing/computer-science/informationtheory) 360 | 361 | 2. Information Theory by MIT OpenCourseWare 362 | - [Course Link](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-441-information-theory-spring-2016/) 363 | 364 | 3. Information Theory by Math Is Fun 365 | - [Website Link](https://www.mathsisfun.com/data/information-theory.html) 366 | 367 | 4. Information Theory by CrashCourse 368 | - [YouTube Video Link](https://www.youtube.com/watch?v=9w4g5V63DkA) 369 | 370 | 5. Information Theory by Stanford University (Coursera) 371 | - [Course Link](https://www.coursera.org/learn/information-theory) 372 | 373 | 6. Information Theory by Krista King Math 374 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BV7ZJ8HY4CH6kY3d7O2fZ3i) 375 | 376 | 7. An Introduction to Information Theory by John Watrous 377 | - [Lecture Notes Link](https://cs.uwaterloo.ca/~watrous/CPSC519/LectureNotes/12.pdf) 378 | 379 | 8. Information Theory by All About Circuits 380 | - [Article Link](https://www.allaboutcircuits.com/technical-articles/information-theory-what-is-information-theory/) 381 | 382 | 9. Information Theory by Math Goodies 383 | - [Website Link](https://www.mathgoodies.com/lessons/vol10/information_theory) 384 | 385 | 386 | # 3. Calculus 387 | ## - Differential Calculus: Derivatives, chain rule, partial derivatives. 388 | ## Learning Resources: Differential Calculus 389 | 390 | 1. Differential Calculus by Khan Academy 391 | - [Tutorial Link](https://www.khanacademy.org/math/differential-calculus) 392 | 393 | 2. Differential Calculus by MIT OpenCourseWare 394 | - [Course Link](https://ocw.mit.edu/courses/mathematics/18-01-single-variable-calculus-fall-2006/) 395 | 396 | 3. Differential Calculus by Math Is Fun 397 | - [Website Link](https://www.mathsisfun.com/calculus/) 398 | 399 | 4. Differential Calculus by CrashCourse 400 | - [YouTube Video Link](https://www.youtube.com/watch?v=EKvHQc3QEow) 401 | 402 | 5. Calculus 1: Differentiation by The Essence of Mathematics 403 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr) 404 | 405 | 6. Differential Calculus by Krista King Math 406 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BVLnSPlVmvbDk4y9xnz7jpj) 407 | 408 | 7. Differential Calculus by MathDoctorBob 409 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhGB6ZdW0jLjRFTUj5N6Lq3F) 410 | 411 | 8. Calculus I: Differentiation by UCI Open 412 | - [Course Link](https://ocw.uci.edu/courses/math_1a_calculus_i.html) 413 | 414 | 9. Differential Calculus by Math Goodies 415 | - [Website Link](https://www.mathgoodies.com/lessons/vol1/differential) 416 | 417 | ## - Integral Calculus: Integrals, definite and indefinite integrals, multivariable calculus, gradients. 418 | ## Learning Resources: Integral Calculus 419 | 420 | 1. Integral Calculus by Khan Academy 421 | - [Tutorial Link](https://www.khanacademy.org/math/integral-calculus) 422 | 423 | 2. Integral Calculus by MIT OpenCourseWare 424 | - [Course Link](https://ocw.mit.edu/courses/mathematics/18-02sc-multivariable-calculus-fall-2010/) 425 | 426 | 3. Integral Calculus by Math Is Fun 427 | - [Website Link](https://www.mathsisfun.com/calculus/integration-introduction.html) 428 | 429 | 4. Integral Calculus by CrashCourse 430 | - [YouTube Video Link](https://www.youtube.com/watch?v=rfG8ce4nNh0) 431 | 432 | 5. Calculus 2: Integration by The Essence of Mathematics 433 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLZHQObOWTQDNPOjrT6KVlfJuKtYTftqH6K) 434 | 435 | 6. Integral Calculus by Krista King Math 436 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BVoTlaXWFcFZ7fU3b9DgnF5) 437 | 438 | 7. Integral Calculus by MathDoctorBob 439 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhGQznkg8v9ugxvOoKNEIazf) 440 | 441 | 8. Calculus II: Integration by UCI Open 442 | - [Course Link](https://ocw.uci.edu/courses/math_1b_calculus_ii.html) 443 | 444 | 9. Integral Calculus by Math Goodies 445 | - [Website Link](https://www.mathgoodies.com/lessons/vol2/integration) 446 | 447 | ## - Optimization Techniques: Gradient descent, stochastic gradient descent (SGD), learning rate, convex optimization. 448 | ## Learning Resources: Optimization Techniques 449 | 450 | 1. Optimization Techniques by Khan Academy 451 | - [Tutorial Link](https://www.khanacademy.org/math/ap-calculus-ab/ab-derivatives-advanced/ab-optimization) 452 | 453 | 2. Optimization Techniques by MIT OpenCourseWare 454 | - [Course Link](https://ocw.mit.edu/courses/sloan-school-of-management/15-053-optimization-methods-in-management-science-spring-2013/) 455 | 456 | 3. Optimization Techniques by Math Is Fun 457 | - [Website Link](https://www.mathsisfun.com/algebra/optimization.html) 458 | 459 | 4. Optimization Techniques by CrashCourse 460 | - [YouTube Video Link](https://www.youtube.com/watch?v=qnYhE2f2PjA) 461 | 462 | 5. Optimization Techniques by Stanford University (Coursera) 463 | - [Course Link](https://www.coursera.org/learn/optimization) 464 | 465 | 6. Optimization Techniques by Krista King Math 466 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PL0o_zxa4K1BXTBxhWGZkVJsdl-zeojvho) 467 | 468 | 7. Optimization Techniques by MathDoctorBob 469 | - [Video Playlist Link](https://www.youtube.com/playlist?list=PLkTxgafCgJhGtKoBHhwIat0Ku7s8bThqh) 470 | 471 | 8. Optimization Techniques by University of Washington (Coursera) 472 | - [Course Link](https://www.coursera.org/learn/introduction-to-optimization) 473 | 474 | 9. Optimization Techniques by Math Goodies 475 | - [Website Link](https://www.mathgoodies.com/lessons/vol10/optimization) 476 | 477 | 478 | # 4. Numerical Computation 479 | ## - Floating Point Representation: Floating point format, precision, machine epsilon. 480 | ## Learning Resources: Floating Point Representation 481 | 482 | 1. Floating Point Representation by Khan Academy 483 | - [Tutorial Link](https://www.khanacademy.org/computing/computer-science/cryptography/modarithmetic/a/what-is-modular-arithmetic) 484 | 485 | 2. Floating Point Representation by Wikipedia 486 | - [Article Link](https://en.wikipedia.org/wiki/Floating-point_arithmetic) 487 | 488 | 3. Floating Point Representation by Exploring Binary 489 | - [Website Link](https://www.exploringbinary.com/) 490 | 491 | 4. IEEE 754 Floating Point Standard by Explained Visually 492 | - [Interactive Visualization](https://www.h-schmidt.net/FloatConverter/IEEE754.html) 493 | 494 | 5. Floating Point Representation by Computerphile 495 | - [YouTube Video Link](https://www.youtube.com/watch?v=PZRI1IfStY0) 496 | 497 | 6. Floating Point Representation by MathWorks 498 | - [Documentation Link](https://www.mathworks.com/help/matlab/matlab_prog/floating-point-numbers.html) 499 | 500 | 7. Floating Point Representation by GeeksforGeeks 501 | - [Article Link](https://www.geeksforgeeks.org/floating-point-representation-basics-examples/) 502 | 503 | 8. Floating Point Representation by Math Goodies 504 | - [Website Link](https://www.mathgoodies.com/lessons/vol2/floating_point) 505 | 506 | ## - Numerical Stability: Stability issues in numerical computations, conditioning and ill-conditioning. 507 | ## Learning Resources: Numerical Stability 508 | 509 | 1. Numerical Stability by Wikipedia 510 | - [Article Link](https://en.wikipedia.org/wiki/Numerical_stability) 511 | 512 | 2. Numerical Stability and Conditioning by Khan Academy 513 | - [Tutorial Link](https://www.khanacademy.org/computing/computer-science/comp-numeric-calc/comp-precision-arithmetic/a/numerical-stability) 514 | 515 | 3. Numerical Stability by Numerical Tours 516 | - [Website Link](https://www.numerical-tours.com/nutshell/) 517 | 518 | 4. Numerical Stability by MathWorks 519 | - [Documentation Link](https://www.mathworks.com/help/matlab/math/numerical-stability.html) 520 | 521 | 5. Numerical Stability by MIT OpenCourseWare 522 | - [Course Link](https://ocw.mit.edu/courses/mathematics/18-335j-introduction-to-numerical-methods-fall-2010/) 523 | 524 | 6. Numerical Stability and Conditioning by Numerical Methods for Engineers 525 | - [Video Lecture Link](https://www.youtube.com/watch?v=UvmEy6n6el0) 526 | 527 | 7. Floating Point Arithmetic and Numerical Stability by Computational Physics with Python 528 | - [Book Chapter Link](https://www.fizika.unios.hr/rf/wp-content/uploads/sites/96/2020/10/Computational-Physics-with-Python-M-Lecture-Notes.pdf#page=69) 529 | 530 | 8. Numerical Stability in Machine Learning by Towards Data Science 531 | - [Article Link](https://towardsdatascience.com/numerical-stability-in-machine-learning-501c09e8b0de) 532 | 533 | 9. Numerical Stability in Deep Learning by Machine Learning Mastery 534 | - [Article Link](https://machinelearningmastery.com/understanding-numerical-stability-in-deep-learning-models/) 535 | 536 | ## - Gradient-Based Optimization: Calculating gradients, optimization algorithms, learning rate tuning. 537 | ## Learning Resources: Gradient-Based Optimization 538 | 539 | 1. Gradient Descent by Khan Academy 540 | - [Tutorial Link](https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/gradient-and-directional-derivatives/v/gradient) 541 | 542 | 2. Gradient-Based Optimization by Stanford University (Coursera) 543 | - [Course Link](https://www.coursera.org/learn/machine-learning) 544 | 545 | 3. Gradient-Based Optimization by Machine Learning Mastery 546 | - [Article Link](https://machinelearningmastery.com/gradient-descent-for-machine-learning/) 547 | 548 | 4. Gradient-Based Optimization by Andrew Ng 549 | - [Video Lecture Link](https://www.youtube.com/watch?v=4qJaSmvhxi8) 550 | 551 | 5. Gradient-Based Optimization by DeepLearning.AI 552 | - [Course Link](https://www.deeplearning.ai/) 553 | 554 | 6. Gradient-Based Optimization by MathWorks 555 | - [Documentation Link](https://www.mathworks.com/help/optim/ug/gradient-based-optimization.html) 556 | 557 | 7. Gradient Descent Optimization Algorithms by Sebastian Ruder 558 | - [Article Link](https://ruder.io/optimizing-gradient-descent/) 559 | 560 | 8. Gradient-Based Optimization by Christopher Bishop 561 | - [Book: "Pattern Recognition and Machine Learning" Link](https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf) 562 | 563 | 9. Gradient-Based Optimization by OpenAI Spinning Up 564 | - [Guide Link](https://spinningup.openai.com/en/latest/algorithms/gradient_descent.html) 565 | 566 | ## - Autodiff and Symbolic Differentiation: Automatic differentiation, symbolic differentiation. 567 | ## Learning Resources: Autodiff and Symbolic Differentiation 568 | 569 | 1. Automatic Differentiation by Khan Academy 570 | - [Tutorial Link](https://www.khanacademy.org/math/ap-calculus-ab/ab-differentiation-2-new/ab-2-5/v/definition-of-derivative-as-limit-of-difference-quotient) 571 | 572 | 2. Automatic Differentiation by Stanford University 573 | - [Course Link](https://www.coursera.org/learn/algorithmic-differentiation) 574 | 575 | 3. Automatic Differentiation by DiffSharp 576 | - [Website Link](https://diffsharp.github.io/DiffSharp/) 577 | 578 | 4. Symbolic Differentiation by Math Is Fun 579 | - [Website Link](https://www.mathsisfun.com/calculus/differentiation-rules.html) 580 | 581 | 5. Symbolic Differentiation by MIT OpenCourseWare 582 | - [Course Link](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/) 583 | 584 | 6. Automatic Differentiation by TensorFlow 585 | - [Documentation Link](https://www.tensorflow.org/guide/autodiff) 586 | 587 | 7. Automatic Differentiation by PyTorch 588 | - [Documentation Link](https://pytorch.org/tutorials/beginner/basics/autogradqs_tutorial.html) 589 | 590 | 8. Automatic Differentiation and Symbolic Differentiation by MathWorks 591 | - [Documentation Link](https://www.mathworks.com/help/symbolic/automatic-differentiation-and-symbolic-differentiation.html) 592 | 593 | 9. Automatic Differentiation and Symbolic Differentiation by UC Berkeley 594 | - [Lecture Notes Link](https://people.eecs.berkeley.edu/~jrs/189/fa05/lectures/lecture23.pdf) 595 | 596 | 597 | # 5. Machine Learning Fundamentals 598 | ## - Linear Regression: Model representation, cost function, normal equation, gradient descent for linear regression. 599 | ## Learning Resources: Linear Regression 600 | 601 | 1. Linear Regression by Khan Academy 602 | - [Tutorial Link](https://www.khanacademy.org/math/statistics-probability/describing-relationships-quantitative-data) 603 | 604 | 2. Linear Regression by Stanford University (Coursera) 605 | - [Course Link](https://www.coursera.org/learn/machine-learning) 606 | 607 | 3. Linear Regression by Andrew Ng 608 | - [Video Lecture Link](https://www.youtube.com/watch?v=kHwlB_j7Hkc) 609 | 610 | 4. Linear Regression by Towards Data Science 611 | - [Article Link](https://towardsdatascience.com/linear-regression-basics-from-the-mathematics-behind-to-real-life-d5a3b4fbd085) 612 | 613 | 5. Linear Regression by StatQuest with Josh Starmer 614 | - [YouTube Video Link](https://www.youtube.com/watch?v=ewnc1qCZVo4) 615 | 616 | 6. Linear Regression by MathWorks 617 | - [Documentation Link](https://www.mathworks.com/help/stats/linear-regression.html) 618 | 619 | 7. Linear Regression by Machine Learning Mastery 620 | - [Article Link](https://machinelearningmastery.com/linear-regression-for-machine-learning/) 621 | 622 | 8. Linear Regression by Python Data Science Handbook 623 | - [Tutorial Link](https://jakevdp.github.io/PythonDataScienceHandbook/05.06-linear-regression.html) 624 | 625 | 9. Linear Regression by OpenAI Spinning Up 626 | - [Guide Link](https://spinningup.openai.com/en/latest/algorithms/linear_regression.html) 627 | 628 | ## - Logistic Regression: Sigmoid function, logistic regression model, binary and multiclass logistic regression. 629 | ## Learning Resources: Logistic Regression 630 | 631 | 1. Logistic Regression by Khan Academy 632 | - [Tutorial Link](https://www.khanacademy.org/math/statistics-probability/describing-relationships-quantitative-data) 633 | 634 | 2. Logistic Regression by Stanford University (Coursera) 635 | - [Course Link](https://www.coursera.org/learn/machine-learning) 636 | 637 | 3. Logistic Regression by Andrew Ng 638 | - [Video Lecture Link](https://www.youtube.com/watch?v=-la3q9d7AKQ) 639 | 640 | 4. Logistic Regression by Towards Data Science 641 | - [Article Link](https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc) 642 | 643 | 5. Logistic Regression by StatQuest with Josh Starmer 644 | - [YouTube Video Link](https://www.youtube.com/watch?v=yIYKR4sgzI8) 645 | 646 | 6. Logistic Regression by MathWorks 647 | - [Documentation Link](https://www.mathworks.com/help/stats/logistic-regression.html) 648 | 649 | 7. Logistic Regression by Machine Learning Mastery 650 | - [Article Link](https://machinelearningmastery.com/logistic-regression-for-machine-learning/) 651 | 652 | 8. Logistic Regression by Python Data Science Handbook 653 | - [Tutorial Link](https://jakevdp.github.io/PythonDataScienceHandbook/05.06-linear-regression.html) 654 | 655 | 9. Logistic Regression by OpenAI Spinning Up 656 | - [Guide Link](https://spinningup.openai.com/en/latest/algorithms/logistic_regression.html) 657 | 658 | ## - Support Vector Machines (SVM): Linear SVM, kernel trick, soft margin SVM. 659 | ## Learning Resources: Support Vector Machines (SVM) 660 | 661 | 1. Support Vector Machines (SVM) by Khan Academy 662 | - [Tutorial Link](https://www.khanacademy.org/computing/computer-science/machine-learning) 663 | 664 | 2. Support Vector Machines (SVM) by Stanford University (Coursera) 665 | - [Course Link](https://www.coursera.org/learn/machine-learning) 666 | 667 | 3. Support Vector Machines (SVM) by Andrew Ng 668 | - [Video Lecture Link](https://www.youtube.com/watch?v=_PwhiWxHK8o) 669 | 670 | 4. Support Vector Machines (SVM) by StatQuest with Josh Starmer 671 | - [YouTube Video Link](https://www.youtube.com/watch?v=efR1C6CvhmE) 672 | 673 | 5. Support Vector Machines (SVM) by Scikit-learn Documentation 674 | - [Tutorial Link](https://scikit-learn.org/stable/modules/svm.html) 675 | 676 | 6. Support Vector Machines (SVM) by Machine Learning Mastery 677 | - [Article Link](https://machinelearningmastery.com/support-vector-machines-for-machine-learning/) 678 | 679 | 7. Support Vector Machines (SVM) by Python Data Science Handbook 680 | - [Tutorial Link](https://jakevdp.github.io/PythonDataScienceHandbook/05.07-support-vector-machines.html) 681 | 682 | 8. Support Vector Machines (SVM) by OpenAI Spinning Up 683 | - [Guide Link](https://spinningup.openai.com/en/latest/algorithms/svm.html) 684 | 685 | 9. Support Vector Machines (SVM) by LIBSVM 686 | - [Website Link](https://www.csie.ntu.edu.tw/~cjlin/libsvm/) 687 | 688 | ## - Decision Trees and Random Forests: Decision tree construction, random forests. 689 | ## Learning Resources: Decision Trees and Random Forests 690 | 691 | 1. Decision Trees by Khan Academy 692 | - [Tutorial Link](https://www.khanacademy.org/math/statistics-probability/random-variables-stats-library) 693 | 694 | 2. Decision Trees and Random Forests by Stanford University (Coursera) 695 | - [Course Link](https://www.coursera.org/learn/machine-learning) 696 | 697 | 3. Decision Trees by Andrew Ng 698 | - [Video Lecture Link](https://www.youtube.com/watch?v=ZDR418e8Kpw) 699 | 700 | 4. Decision Trees and Random Forests by StatQuest with Josh Starmer 701 | - [YouTube Video Link](https://www.youtube.com/watch?v=7VeUPuFGJHk) 702 | 703 | 5. Decision Trees and Random Forests by Scikit-learn Documentation 704 | - [Tutorial Link](https://scikit-learn.org/stable/modules/tree.html) 705 | 706 | 6. Decision Trees and Random Forests by Machine Learning Mastery 707 | - [Article Link](https://machinelearningmastery.com/implement-decision-tree-algorithm-scratch-python/) 708 | 709 | 7. Decision Trees and Random Forests by Python Data Science Handbook 710 | - [Tutorial Link](https://jakevdp.github.io/PythonDataScienceHandbook/05.08-random-forests.html) 711 | 712 | 8. Decision Trees and Random Forests by OpenAI Spinning Up 713 | - [Guide Link](https://spinningup.openai.com/en/latest/algorithms/random_forest.html) 714 | 715 | 9. Decision Trees and Random Forests by Scikit-learn 716 | - [Documentation Link](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html) 717 | 718 | ## - Evaluation Metrics: Accuracy, precision, recall, F1 score, ROC curve, AUC-ROC. 719 | ## Learning Resources: Evaluation Metrics 720 | 721 | 1. Evaluation Metrics for Machine Learning by Towards Data Science 722 | - [Article Link](https://towardsdatascience.com/understanding-evaluation-metrics-in-machine-learning-3aa524904912) 723 | 724 | 2. Evaluation Metrics for Classification by Machine Learning Mastery 725 | - [Article Link](https://machinelearningmastery.com/classification-accuracy-is-not-enough-more-performance-measures-you-can-use/) 726 | 727 | 3. Evaluation Metrics for Regression by Machine Learning Mastery 728 | - [Article Link](https://machinelearningmastery.com/regression-metrics-for-machine-learning/) 729 | 730 | 4. Evaluation Metrics for Binary Classification by Scikit-learn Documentation 731 | - [Tutorial Link](https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics) 732 | 733 | 5. Evaluation Metrics for Multiclass Classification by Scikit-learn Documentation 734 | - [Tutorial Link](https://scikit-learn.org/stable/modules/model_evaluation.html#multiclass-and-multilabel-classification) 735 | 736 | 6. Evaluation Metrics for Imbalanced Classification by Machine Learning Mastery 737 | - [Article Link](https://machinelearningmastery.com/roc-curves-and-precision-recall-curves-for-classification-in-python/) 738 | 739 | 7. Evaluation Metrics for Clustering by Scikit-learn Documentation 740 | - [Tutorial Link](https://scikit-learn.org/stable/modules/clustering.html#clustering-performance-evaluation) 741 | 742 | 8. Evaluation Metrics for Recommender Systems by Machine Learning Mastery 743 | - [Article Link](https://machinelearningmastery.com/evaluate-recommender-systems-machine-learning-python/) 744 | 745 | 9. Evaluation Metrics for Natural Language Processing (NLP) by Machine Learning Mastery 746 | - [Article Link](https://machinelearningmastery.com/evaluate-machine-learning-models-natural-language-processing/) 747 | 748 | 10. Evaluation Metrics for Time Series Forecasting by Machine Learning Mastery 749 | - [Article Link](https://machinelearningmastery.com/time-series-forecasting-performance-measures-with-python/) 750 | 751 | 752 | 753 | # 6. Deep Learning Basics 754 | ## - Feedforward Neural Networks: Architecture, activation functions, forward propagation, backward propagation. 755 | ## Learning Resources: Feedforward Neural Networks 756 | 757 | 1. Neural Networks by Khan Academy 758 | - [Tutorial Link](https://www.khanacademy.org/computing/computer-science/machine-learning) 759 | 760 | 2. Feedforward Neural Networks by Stanford University (Coursera) 761 | - [Course Link](https://www.coursera.org/learn/neural-networks-deep-learning) 762 | 763 | 3. Feedforward Neural Networks by Andrew Ng 764 | - [Video Lecture Link](https://www.youtube.com/watch?v=CS4cs9xVecg) 765 | 766 | 4. Feedforward Neural Networks by DeepLearning.AI 767 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 768 | 769 | 5. Feedforward Neural Networks by PyTorch 770 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 771 | 772 | 6. Feedforward Neural Networks by TensorFlow 773 | - [Tutorial Link](https://www.tensorflow.org/guide/keras/sequential_model) 774 | 775 | 7. Feedforward Neural Networks by Machine Learning Mastery 776 | - [Article Link](https://machinelearningmastery.com/neural-networks-crash-course/) 777 | 778 | 8. Feedforward Neural Networks by Python Data Science Handbook 779 | - [Tutorial Link](https://jakevdp.github.io/PythonDataScienceHandbook/05.00-machine-learning.html) 780 | 781 | 9. Feedforward Neural Networks by OpenAI Spinning Up 782 | - [Guide Link](https://spinningup.openai.com/en/latest/algorithms/ppo.html) 783 | 784 | ## - Backpropagation Algorithm: Calculating gradients using backpropagation, weight updates. 785 | ## Learning Resources: Backpropagation Algorithm 786 | 787 | 1. Backpropagation Algorithm by Khan Academy 788 | - [Tutorial Link](https://www.khanacademy.org/computing/computer-science/machine-learning) 789 | 790 | 2. Backpropagation Algorithm by Stanford University (Coursera) 791 | - [Course Link](https://www.coursera.org/learn/neural-networks-deep-learning) 792 | 793 | 3. Backpropagation Algorithm by Andrew Ng 794 | - [Video Lecture Link](https://www.youtube.com/watch?v=Ilg3gGewQ5U) 795 | 796 | 4. Backpropagation Algorithm by DeepLearning.AI 797 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 798 | 799 | 5. Backpropagation Algorithm by Machine Learning Mastery 800 | - [Article Link](https://machinelearningmastery.com/gentle-introduction-backpropagation-time/) 801 | 802 | 6. Backpropagation Algorithm by Towards Data Science 803 | - [Article Link](https://towardsdatascience.com/backpropagation-in-neural-networks-intuition-forward-and-backward-passes-2677a99da874) 804 | 805 | 7. Backpropagation Algorithm by Python Data Science Handbook 806 | - [Tutorial Link](https://jakevdp.github.io/PythonDataScienceHandbook/05.03-hyperparameters-and-model-validation.html) 807 | 808 | 8. Backpropagation Algorithm by OpenAI Spinning Up 809 | - [Guide Link](https://spinningup.openai.com/en/latest/algorithms/vpg.html) 810 | 811 | 9. Backpropagation Algorithm by Deep Learning with PyTorch 812 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 813 | 814 | ## - Weight Initialization: Xavier/Glorot initialization, He initialization. 815 | ## Learning Resources: Weight Initialization 816 | 817 | 1. Weight Initialization in Neural Networks by Machine Learning Mastery 818 | - [Article Link](https://machinelearningmastery.com/weight-initialization-for-deep-learning-neural-networks/) 819 | 820 | 2. Weight Initialization in Deep Learning by Deeplearning.AI 821 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 822 | 823 | 3. Weight Initialization in Neural Networks by TensorFlow 824 | - [Documentation Link](https://www.tensorflow.org/guide/keras/initializers) 825 | 826 | 4. Weight Initialization in Neural Networks by PyTorch 827 | - [Documentation Link](https://pytorch.org/docs/stable/generated/torch.nn.init.html) 828 | 829 | 5. Weight Initialization in Neural Networks by Deep Learning with Python book 830 | - [Chapter 6: Deep Learning for Computer Vision](https://www.manning.com/books/deep-learning-with-python) 831 | 832 | 6. Weight Initialization in Neural Networks by Towards Data Science 833 | - [Article Link](https://towardsdatascience.com/weight-initialization-techniques-in-neural-networks-26c649eb3b78) 834 | 835 | 7. Weight Initialization in Neural Networks by Neural Designer 836 | - [Article Link](https://www.neuraldesigner.com/blog/weight-initialization) 837 | 838 | 8. Weight Initialization in Neural Networks by Machine Learning Wiki 839 | - [Article Link](https://machinelearning.wiki/topics/deep-learning/weight-initialization) 840 | 841 | 9. Weight Initialization in Neural Networks by OpenAI Spinning Up 842 | - [Discussion Link](https://github.com/openai/spinningup/discussions/33) 843 | 844 | ## - Gradient-Based Optimization Algorithms: Gradient descent, mini-batch gradient descent, stochastic gradient descent. 845 | ## Learning Resources: Gradient-Based Optimization Algorithms 846 | 847 | 1. Gradient-Based Optimization Algorithms by Machine Learning Mastery 848 | - [Article Link](https://machinelearningmastery.com/gradient-descent-optimization-with-nadam-from-scratch/) 849 | 850 | 2. Gradient-Based Optimization Algorithms by Stanford University (Coursera) 851 | - [Course Link](https://www.coursera.org/learn/machine-learning) 852 | 853 | 3. Gradient-Based Optimization Algorithms by Andrew Ng 854 | - [Video Lecture Link](https://www.youtube.com/watch?v=P2YmAIwKp-M) 855 | 856 | 4. Gradient-Based Optimization Algorithms by DeepLearning.AI 857 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 858 | 859 | 5. Gradient-Based Optimization Algorithms by Sebastian Ruder 860 | - [Article Link](https://ruder.io/optimizing-gradient-descent/) 861 | 862 | 6. Gradient-Based Optimization Algorithms by PyTorch 863 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 864 | 865 | 7. Gradient-Based Optimization Algorithms by TensorFlow 866 | - [Tutorial Link](https://www.tensorflow.org/guide/keras/train_and_evaluate) 867 | 868 | 8. Gradient-Based Optimization Algorithms by OpenAI Spinning Up 869 | - [Guide Link](https://spinningup.openai.com/en/latest/algorithms/vpg.html) 870 | 871 | 9. Gradient-Based Optimization Algorithms by Machine Learning Wiki 872 | - [Article Link](https://machinelearning.wiki/topics/optimization/gradient-based-optimization) 873 | 874 | 10. Gradient-Based Optimization Algorithms by Deep Learning with Python book 875 | - [Chapter 8: Deep Learning for Text and Sequences](https://www.manning.com/books/deep-learning-with-python) 876 | 877 | ## - Regularization Techniques: L1 and L2 regularization, dropout. 878 | ## Learning Resources: Regularization Techniques 879 | 880 | 1. Regularization Techniques in Machine Learning by Machine Learning Mastery 881 | - [Article Link](https://machinelearningmastery.com/regularization-for-machine-learning/) 882 | 883 | 2. Regularization Techniques in Deep Learning by Deeplearning.AI 884 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 885 | 886 | 3. Regularization Techniques in Machine Learning by Towards Data Science 887 | - [Article Link](https://towardsdatascience.com/regularization-in-machine-learning-76441ddcf99a) 888 | 889 | 4. Regularization Techniques in Neural Networks by DeepLearning.AI 890 | - [Course Link](https://www.coursera.org/learn/deep-neural-network) 891 | 892 | 5. Regularization Techniques in Machine Learning by Scikit-learn Documentation 893 | - [Tutorial Link](https://scikit-learn.org/stable/modules/linear_model.html#ridge-regression-and-classification) 894 | 895 | 6. Regularization Techniques in Deep Learning by TensorFlow 896 | - [Tutorial Link](https://www.tensorflow.org/guide/keras/regularization) 897 | 898 | 7. Regularization Techniques in Neural Networks by PyTorch 899 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 900 | 901 | 8. Regularization Techniques in Machine Learning by Sebastian Raschka 902 | - [Article Link](https://sebastianraschka.com/faq/docs/regression-concepts-history.html) 903 | 904 | 9. Regularization Techniques in Machine Learning by OpenAI Spinning Up 905 | - [Discussion Link](https://github.com/openai/spinningup/discussions/18) 906 | 907 | ## - Convolutional Neural Networks (CNNs): Convolutional layers, pooling layers, convolution arithmetic. 908 | ## Learning Resources: Convolutional Neural Networks (CNNs) 909 | 910 | 1. Convolutional Neural Networks by Stanford University (Coursera) 911 | - [Course Link](https://www.coursera.org/learn/convolutional-neural-networks) 912 | 913 | 2. Convolutional Neural Networks (CNNs) by DeepLearning.AI 914 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 915 | 916 | 3. Convolutional Neural Networks (CNNs) by Andrew Ng 917 | - [Video Lecture Link](https://www.youtube.com/watch?v=bNb2fEVKeEo) 918 | 919 | 4. Convolutional Neural Networks by Machine Learning Mastery 920 | - [Article Link](https://machinelearningmastery.com/convolutional-neural-network-architectures-an-overview/) 921 | 922 | 5. Convolutional Neural Networks by TensorFlow 923 | - [Tutorial Link](https://www.tensorflow.org/tutorials/images/cnn) 924 | 925 | 6. Convolutional Neural Networks by PyTorch 926 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 927 | 928 | 7. Convolutional Neural Networks by Deep Learning with Python book 929 | - [Chapter 5: Deep Learning for Computer Vision](https://www.manning.com/books/deep-learning-with-python) 930 | 931 | 8. Convolutional Neural Networks by Machine Learning Wiki 932 | - [Article Link](https://machinelearning.wiki/topics/deep-learning/convolutional-neural-networks) 933 | 934 | 9. Convolutional Neural Networks by OpenAI Spinning Up 935 | - [Discussion Link](https://github.com/openai/spinningup/discussions/21) 936 | 937 | ## - Recurrent Neural Networks (RNNs): RNN cells, LSTM, bidirectional RNNs. 938 | ## Learning Resources: Recurrent Neural Networks (RNNs) 939 | 940 | 1. Recurrent Neural Networks by Stanford University (Coursera) 941 | - [Course Link](https://www.coursera.org/learn/nlp-sequence-models) 942 | 943 | 2. Recurrent Neural Networks (RNNs) by DeepLearning.AI 944 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 945 | 946 | 3. Recurrent Neural Networks (RNNs) by Andrew Ng 947 | - [Video Lecture Link](https://www.youtube.com/watch?v=SEnXr6v2ifU) 948 | 949 | 4. Recurrent Neural Networks by Machine Learning Mastery 950 | - [Article Link](https://machinelearningmastery.com/gentle-introduction-recurrent-neural-networks/) 951 | 952 | 5. Recurrent Neural Networks by TensorFlow 953 | - [Tutorial Link](https://www.tensorflow.org/guide/keras/rnn) 954 | 955 | 6. Recurrent Neural Networks by PyTorch 956 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 957 | 958 | 7. Recurrent Neural Networks by Deep Learning with Python book 959 | - [Chapter 6: Deep Learning for Text and Sequences](https://www.manning.com/books/deep-learning-with-python) 960 | 961 | 8. Recurrent Neural Networks by Machine Learning Wiki 962 | - [Article Link](https://machinelearning.wiki/topics/deep-learning/recurrent-neural-networks) 963 | 964 | 9. Recurrent Neural Networks by OpenAI Spinning Up 965 | - [Discussion Link](https://github.com/openai/spinningup/discussions/28) 966 | 967 | 968 | ## - Generative Adversarial Networks (GANs): Generator and discriminator networks, GAN training. 969 | ## Learning Resources: Generative Adversarial Networks (GANs) 970 | 971 | 1. Generative Adversarial Networks (GANs) by Stanford University (Coursera) 972 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 973 | 974 | 2. Generative Adversarial Networks (GANs) by DeepLearning.AI 975 | - [Course Link](https://www.coursera.org/specializations/generative-adversarial-networks-gans) 976 | 977 | 3. Generative Adversarial Networks (GANs) by Ian Goodfellow, et al. (Original GAN Paper) 978 | - [Paper Link](https://arxiv.org/abs/1406.2661) 979 | 980 | 4. Generative Adversarial Networks (GANs) by Machine Learning Mastery 981 | - [Article Link](https://machinelearningmastery.com/what-are-generative-adversarial-networks-gans/) 982 | 983 | 5. Generative Adversarial Networks (GANs) by TensorFlow 984 | - [Tutorial Link](https://www.tensorflow.org/tutorials/generative/dcgan) 985 | 986 | 6. Generative Adversarial Networks (GANs) by PyTorch 987 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html) 988 | 989 | 7. Generative Adversarial Networks (GANs) by Deep Learning with Python book 990 | - [Chapter 8: Deep Learning for Text and Sequences](https://www.manning.com/books/deep-learning-with-python) 991 | 992 | 8. Generative Adversarial Networks (GANs) by Machine Learning Wiki 993 | - [Article Link](https://machinelearning.wiki/topics/generative-adversarial-networks) 994 | 995 | 9. Generative Adversarial Networks (GANs) by OpenAI Spinning Up 996 | - [Discussion Link](https://github.com/openai/spinningup/discussions/14) 997 | 998 | 999 | # 7. Advanced Deep Learning Topics 1000 | 1001 | ## - Batch Normalization: Normalizing activations in deep neural networks. 1002 | ## Learning Resources: Batch Normalization 1003 | 1004 | 1. Batch Normalization by Machine Learning Mastery 1005 | - [Article Link](https://machinelearningmastery.com/batch-normalization-for-training-of-deep-neural-networks/) 1006 | 1007 | 2. Batch Normalization by Stanford University (Coursera) 1008 | - [Course Link](https://www.coursera.org/learn/deep-neural-network) 1009 | 1010 | 3. Batch Normalization by DeepLearning.AI 1011 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 1012 | 1013 | 4. Batch Normalization by Andrew Ng 1014 | - [Video Lecture Link](https://www.youtube.com/watch?v=dXB-KQYkzNU) 1015 | 1016 | 5. Batch Normalization by TensorFlow 1017 | - [Tutorial Link](https://www.tensorflow.org/guide/keras/normalization) 1018 | 1019 | 6. Batch Normalization by PyTorch 1020 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 1021 | 1022 | 7. Batch Normalization by Deep Learning with Python book 1023 | - [Chapter 5: Deep Learning for Computer Vision](https://www.manning.com/books/deep-learning-with-python) 1024 | 1025 | 8. Batch Normalization by Machine Learning Wiki 1026 | - [Article Link](https://machinelearning.wiki/topics/deep-learning/batch-normalization) 1027 | 1028 | 9. Batch Normalization by OpenAI Spinning Up 1029 | - [Discussion Link](https://github.com/openai/spinningup/discussions/24) 1030 | 1031 | ## - Transfer Learning: Leveraging pre-trained models for new tasks. 1032 | ## Learning Resources: Batch Normalization 1033 | 1034 | 1. Batch Normalization by Machine Learning Mastery 1035 | - [Article Link](https://machinelearningmastery.com/batch-normalization-for-training-of-deep-neural-networks/) 1036 | 1037 | 2. Batch Normalization by Stanford University (Coursera) 1038 | - [Course Link](https://www.coursera.org/learn/deep-neural-network) 1039 | 1040 | 3. Batch Normalization by DeepLearning.AI 1041 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 1042 | 1043 | 4. Batch Normalization by Andrew Ng 1044 | - [Video Lecture Link](https://www.youtube.com/watch?v=dXB-KQYkzNU) 1045 | 1046 | 5. Batch Normalization by TensorFlow 1047 | - [Tutorial Link](https://www.tensorflow.org/guide/keras/normalization) 1048 | 1049 | 6. Batch Normalization by PyTorch 1050 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 1051 | 1052 | 7. Batch Normalization by Deep Learning with Python book 1053 | - [Chapter 5: Deep Learning for Computer Vision](https://www.manning.com/books/deep-learning-with-python) 1054 | 1055 | 8. Batch Normalization by Machine Learning Wiki 1056 | - [Article Link](https://machinelearning.wiki/topics/deep-learning/batch-normalization) 1057 | 1058 | 9. Batch Normalization by OpenAI Spinning Up 1059 | - [Discussion Link](https://github.com/openai/spinningup/discussions/24) 1060 | 1061 | ## - Reinforcement Learning: Markov decision processes, Q-learning, policy gradients. 1062 | ## Learning Resources: Reinforcement Learning 1063 | 1064 | 1. Reinforcement Learning by David Silver (DeepMind) 1065 | - [Video Lecture Series](https://www.youtube.com/playlist?list=PL7-jPKtc4r78-wCZcO5vUxpadlWMp-1PX) 1066 | 1067 | 2. Reinforcement Learning by Stanford University (Coursera) 1068 | - [Course Link](https://www.coursera.org/specializations/reinforcement-learning) 1069 | 1070 | 3. Reinforcement Learning by DeepLearning.AI 1071 | - [Course Link](https://www.coursera.org/specializations/reinforcement-learning) 1072 | 1073 | 4. Reinforcement Learning by OpenAI 1074 | - [Gym Documentation](https://gym.openai.com/docs/) 1075 | 1076 | 5. Reinforcement Learning by Sutton and Barto (Book) 1077 | - [Book Link](http://incompleteideas.net/book/the-book-2nd.html) 1078 | 1079 | 6. Reinforcement Learning by TensorFlow 1080 | - [Tutorial Link](https://www.tensorflow.org/agents) 1081 | 1082 | 7. Reinforcement Learning by PyTorch 1083 | - [Tutorial Link](https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html) 1084 | 1085 | 8. Reinforcement Learning by Machine Learning Wiki 1086 | - [Article Link](https://machinelearning.wiki/topics/reinforcement-learning) 1087 | 1088 | 9. Reinforcement Learning by OpenAI Spinning Up 1089 | - [Discussion Link](https://github.com/openai/spinningup/discussions/7) 1090 | 1091 | ## - Natural Language Processing (NLP): Word embeddings, recurrent neural networks for sequence modeling. 1092 | ## Learning Resources: Natural Language Processing (NLP) 1093 | 1094 | 1. Natural Language Processing by Stanford University (Coursera) 1095 | - [Course Link](https://www.coursera.org/specializations/natural-language-processing) 1096 | 1097 | 2. Natural Language Processing with Deep Learning by DeepLearning.AI 1098 | - [Course Link](https://www.coursera.org/specializations/natural-language-processing) 1099 | 1100 | 3. Natural Language Processing (NLP) by Fast.ai 1101 | - [Course Link](https://www.fast.ai/course/nlp) 1102 | 1103 | 4. Natural Language Processing (NLP) by TensorFlow 1104 | - [Tutorial Link](https://www.tensorflow.org/tutorials/text/word_embeddings) 1105 | 1106 | 5. Natural Language Processing (NLP) by PyTorch 1107 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 1108 | 1109 | 6. Natural Language Processing (NLP) by NLTK (Natural Language Toolkit) 1110 | - [Tutorial Link](https://www.nltk.org/book/) 1111 | 1112 | 7. Natural Language Processing (NLP) by Machine Learning Mastery 1113 | - [Article Link](https://machinelearningmastery.com/start-here/#nlp) 1114 | 1115 | 8. Natural Language Processing (NLP) by Machine Learning Wiki 1116 | - [Article Link](https://machinelearning.wiki/topics/natural-language-processing) 1117 | 1118 | 9. Natural Language Processing (NLP) by OpenAI Spinning Up 1119 | - [Discussion Link](https://github.com/openai/spinningup/discussions/3) 1120 | 1121 | ## - Time Series Analysis: Modeling and forecasting time series data with deep learning. 1122 | ## Learning Resources: Time Series Analysis 1123 | 1124 | 1. Time Series Analysis and Its Applications by Shumway and Stoffer (Book) 1125 | - [Book Link](https://www.stat.pitt.edu/stoffer/tsa4/) 1126 | 1127 | 2. Practical Time Series Analysis by Aileen Nielsen 1128 | - [Book Link](https://www.oreilly.com/library/view/practical-time-series/9781492041641/) 1129 | 1130 | 3. Time Series Analysis by Stanford University (Coursera) 1131 | - [Course Link](https://www.coursera.org/learn/practical-time-series-analysis) 1132 | 1133 | 4. Time Series Analysis by Kaggle 1134 | - [Tutorial Link](https://www.kaggle.com/learn/time-series-analysis) 1135 | 1136 | 5. Time Series Analysis by Machine Learning Mastery 1137 | - [Article Link](https://machinelearningmastery.com/start-here/#time-series) 1138 | 1139 | 6. Time Series Analysis by TensorFlow 1140 | - [Tutorial Link](https://www.tensorflow.org/tutorials/structured_data/time_series) 1141 | 1142 | 7. Time Series Analysis by PyTorch 1143 | - [Tutorial Link](https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html) 1144 | 1145 | 8. Time Series Analysis by Machine Learning Wiki 1146 | - [Article Link](https://machinelearning.wiki/topics/time-series-analysis) 1147 | 1148 | 9. Time Series Analysis by OpenAI Spinning Up 1149 | - [Discussion Link](https://github.com/openai/spinningup/discussions/18) 1150 | 1151 | 1152 | ## - Autoencoders and Variational Autoencoders (VAEs): Unsupervised learning, dimensionality reduction, generative models. 1153 | ## Learning Resources: Autoencoders and Variational Autoencoders (VAEs) 1154 | 1155 | 1. Autoencoders by Stanford University (Coursera) 1156 | - [Course Link](https://www.coursera.org/learn/deep-learning) 1157 | 1158 | 2. Autoencoders and Variational Autoencoders (VAEs) by DeepLearning.AI 1159 | - [Course Link](https://www.coursera.org/specializations/deep-learning) 1160 | 1161 | 3. Autoencoders by TensorFlow 1162 | - [Tutorial Link](https://www.tensorflow.org/tutorials/generative/autoencoder) 1163 | 1164 | 4. Variational Autoencoders (VAEs) by TensorFlow 1165 | - [Tutorial Link](https://www.tensorflow.org/tutorials/generative/cvae) 1166 | 1167 | 5. Autoencoders and Variational Autoencoders (VAEs) by PyTorch 1168 | - [Tutorial Link](https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html) 1169 | 1170 | 6. Autoencoders and Variational Autoencoders (VAEs) by Machine Learning Mastery 1171 | - [Article Link](https://machinelearningmastery.com/start-here/#autoencoders) 1172 | 1173 | 7. Autoencoders and Variational Autoencoders (VAEs) by Machine Learning Wiki 1174 | - [Article Link](https://machinelearning.wiki/topics/autoencoders) 1175 | 1176 | 8. Autoencoders and Variational Autoencoders (VAEs) by OpenAI Spinning Up 1177 | - [Discussion Link](https://github.com/openai/spinningup/discussions/13) 1178 | 1179 | 9. Autoencoders and Variational Autoencoders (VAEs) by Christopher Olah 1180 | - [Blog Post](https://colah.github.io/archive.html?tag=autoencoders) 1181 | 1182 | 10. Variational Autoencoders (VAEs) by Carl Doersch 1183 | - [Tutorial Link](https://arxiv.org/abs/1606.05908) 1184 | 1185 | 11. Autoencoders and Variational Autoencoders (VAEs) by OpenAI 1186 | - [Blog Post](https://openai.com/blog/introduction-to-variational-autoencoders/) 1187 | 1188 | 12. Building Autoencoders in Keras by François Chollet (Keras Blog) 1189 | - [Blog Post](https://blog.keras.io/building-autoencoders-in-keras.html) 1190 | 1191 | 13. Autoencoders and Variational Autoencoders (VAEs) by Google Developers 1192 | - [Tutorial Link](https://developers.google.com/machine-learning/clustering/autoencoders) 1193 | 1194 | 14. Deep Learning Book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville 1195 | - [Chapter 14: Autoencoders](https://www.deeplearningbook.org/contents/autoencoders.html) 1196 | 1197 | 15. Autoencoders and Variational Autoencoders (VAEs) by Distill.pub 1198 | - [Article Link](https://distill.pub/2016/deconv-checkerboard/) 1199 | 1200 | 16. Variational Autoencoders (VAEs) by OpenAI Spinning Up 1201 | - [Discussion Link](https://github.com/openai/spinningup/discussions/9) 1202 | 1203 | 1204 | ## - Model Interpretability and Explainability: Techniques to interpret and explain deep learning models. 1205 | ## Learning Resources: Model Interpretability and Explainability 1206 | 1207 | 1. Interpretable Machine Learning by Christoph Molnar (Book) 1208 | - [Book Link](https://christophm.github.io/interpretable-ml-book/) 1209 | 1210 | 2. Interpretable Machine Learning by Microsoft Research 1211 | - [Website Link](https://www.interpretable-ml.org/) 1212 | 1213 | 3. Model Interpretability and Explainability by Google AI 1214 | - [Article Link](https://ai.googleblog.com/2017/06/interpreting-deep-learning-models-with.html) 1215 | 1216 | 4. Model Interpretability and Explainability by OpenAI 1217 | - [Blog Post](https://openai.com/blog/interpretability/) 1218 | 1219 | 5. Model Interpretability and Explainability by scikit-learn (Python Library) 1220 | - [Tutorial Link](https://scikit-learn.org/stable/modules/inspection.html) 1221 | 1222 | 6. Explainable AI (XAI) by DARPA 1223 | - [Website Link](https://www.darpa.mil/program/explainable-artificial-intelligence) 1224 | 1225 | 7. Model Interpretability and Explainability by Machine Learning Wiki 1226 | - [Article Link](https://machinelearning.wiki/topics/interpretable-machine-learning) 1227 | 1228 | 8. Model Interpretability and Explainability by Towards Data Science 1229 | - [Article Link](https://towardsdatascience.com/interpretable-machine-learning-part-1-xai-models-for-global-interpretability-2a5637d80f3d) 1230 | 1231 | 9. Model Interpretability and Explainability by OpenAI Spinning Up 1232 | - [Discussion Link](https://github.com/openai/spinningup/discussions/19) 1233 | 1234 | 10. A Unified Approach to Interpreting Model Predictions by Marco Tulio Ribeiro, et al. 1235 | - [Research Paper](https://arxiv.org/abs/1705.07874) 1236 | 1237 | 11. SHAP (SHapley Additive exPlanations) by Lundberg and Lee 1238 | - [Python Library](https://shap.readthedocs.io/en/latest/) 1239 | 1240 | 12. LIME (Local Interpretable Model-Agnostic Explanations) by Ribeiro, Singh, and Guestrin 1241 | - [GitHub Repository](https://github.com/marcotcr/lime) 1242 | 1243 | 13. Anchors: High-Precision Model-Agnostic Explanations by Ribeiro, Singh, and Guestrin 1244 | - [Research Paper](https://homes.cs.washington.edu/~marcotcr/aaai18.pdf) 1245 | 1246 | 14. Interpretable Deep Learning with Python by Yuriy Guts 1247 | - [Book Link](https://www.apress.com/gp/book/9781484230964) 1248 | 1249 | 15. Explainable AI and Machine Learning Interpretability by IBM Developer 1250 | - [Tutorial Link](https://developer.ibm.com/technologies/explainable-ai) 1251 | 1252 | 16. Interpretable Machine Learning in Python by Christoph Molnar 1253 | - [GitHub Repository](https://github.com/christophM/interpretable-ml-book) 1254 | 1255 | 17. InterpretML: A Python Library for Model Interpretability by Microsoft 1256 | - [GitHub Repository](https://github.com/interpretml/interpret) 1257 | 1258 | 18. Model Interpretability and Explainability by OpenAI Spinning Up 1259 | - [Discussion Link](https://github.com/openai/spinningup/discussions/20) 1260 | 1261 | ## Contributing 1262 | This roadmap is a compilation of mathematical concepts covered in various deep learning resources, including the following: 1263 | 1264 | 1. "Deep Learning" book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 1265 | 1. "Deep Learning with Python, Second Edition" by François Chollet. 1266 | 1. "Grokking Deep Learning" by Andrew Trask. 1267 | 1. "Deep Learning: A Practitioner's Approach" by Josh Patterson and Adam Gibson. 1268 | 1. "Deep Learning for Coders with fastai and PyTorch" by Jeremy Howard and Sylvain Gugger. 1269 | --------------------------------------------------------------------------------