โ””โ”€โ”€ README.md /README.md: -------------------------------------------------------------------------------- 1 | # Mathematics for Machine Learning 2 | 3 | A collection of resources to learn and review mathematics for machine learning. 4 | 5 | # :book: Books 6 | 7 | ### Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Machine Learning 8 | *by Jean Gallier and Jocelyn Quaintance* 9 | 10 | Includes mathematical concepts for machine learning and computer science. 11 | 12 | Book: https://www.cis.upenn.edu/~jean/math-deep.pdf 13 | 14 | ### Applied Math and Machine Learning Basics 15 | *by Ian Goodfellow and Yoshua Bengio and Aaron Courville* 16 | 17 | This includes the math basics for deep learning from the [Deep Learning](https://www.deeplearningbook.org/) book. 18 | 19 | Chapter: https://www.deeplearningbook.org/contents/part_basics.html 20 | 21 | ### Mathematics for Machine Learning 22 | *by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong* 23 | 24 | This is probably the place you want to start. Start slowly and work on some examples. Pay close attention to the notation and get comfortable with it. 25 | 26 | Book: https://mml-book.github.io 27 | 28 | ### Probabilistic Machine Learning: An Introduction 29 | *by Kevin Patrick Murphy* 30 | 31 | This book contains a comprehensive overview of classical machine learning methods and the principles explaining them. 32 | 33 | Book: https://probml.github.io/pml-book/book1.html 34 | 35 | ### Mathematics for Deep Learning 36 | *by Brent Werness, Rachel Hu et al.* 37 | 38 | This reference contains some mathematical concepts to help build a better understanding of deep learning. 39 | 40 | Chapter: https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/index.html 41 | 42 | ### The Mathematical Engineering of Deep Learning 43 | *by Benoit Liquet, Sarat Moka and Yoni Nazarathy* 44 | 45 | This book provides a complete and concise overview of the mathematical engineering of deep learning. In addition to overviewing deep learning foundations, the treatment includes convolutional neural networks, recurrent neural networks, transformers, generative adversarial networks, reinforcement learning, and multiple tricks of the trade. The focus is on the basic mathematical description of deep learning models, algorithms and methods. 46 | 47 | Book: https://deeplearningmath.org 48 | 49 | ### Bayes Rules! An Introduction to Applied Bayesian Modeling 50 | *by Alicia A. Johnson, Miles Q. Ott, Mine Dogucu* 51 | 52 | Great online book covering Bayesian approaches. 53 | 54 | Book: https://www.bayesrulesbook.com/index.html 55 | 56 | # ๐Ÿ“„ Papers 57 | 58 | ### The Matrix Calculus You Need For Deep Learning 59 | *by Terence Parr & Jeremy Howard* 60 | 61 | In deep learning, you need to understand a bunch of fundamental matrix operations. If you want to dive deep into the math of matrix calculus this is your guide. 62 | 63 | Paper: https://arxiv.org/abs/1802.01528 64 | 65 | ### The Mathematics of AI 66 | *by Gitta Kutyniok* 67 | 68 | An article summarising the importance of mathematics in deep learning research and how itโ€™s helping to advance the field. 69 | 70 | Paper: https://arxiv.org/pdf/2203.08890.pdf 71 | 72 | # ๐ŸŽฅ Video Lectures 73 | 74 | ### Multivariate Calculus by Imperial College London 75 | *by Dr. Sam Cooper & Dr. David Dye* 76 | 77 | Backpropagation is a key algorithm for training deep neural nets that rely on Calculus. Get familiar with concepts like chain rule, Jacobian, gradient descent. 78 | 79 | Video Playlist: https://www.youtube.com/playlist?list=PLiiljHvN6z193BBzS0Ln8NnqQmzimTW23 80 | 81 | ### Mathematics for Machine Learning - Linear Algebra 82 | *by Dr. Sam Cooper & Dr. David Dye* 83 | 84 | A great companion to the previous video lectures. Neural networks perform transformations on data and you need linear algebra to get better intuitions of how that is done. 85 | 86 | Video Playlist: https://www.youtube.com/playlist?list=PLiiljHvN6z1_o1ztXTKWPrShrMrBLo5P3 87 | 88 | ### CS229: Machine Learning 89 | *by Anand Avati* 90 | 91 | Lectures containing mathematical explanations to many concepts in machine learning. 92 | 93 | Course: https://www.youtube.com/playlist?list=PLoROMvodv4rNH7qL6-efu_q2_bPuy0adh 94 | 95 | # ๐Ÿงฎ Math Basics 96 | 97 | ### The Elements of Statistical Learning 98 | *by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie* 99 | 100 | Machine learning deals with data and in turn uncertainty which is what statistics aims to teach. Get comfortable with topics like estimators, statistical significance, etc. 101 | 102 | Book: https://hastie.su.domains/ElemStatLearn/ 103 | 104 | If you are interested in an introduction to statistical learning, then you might want to check out ["An Introduction to Statistical Learning"](https://www.statlearning.com/). 105 | 106 | 107 | ### Probability Theory: The Logic of Science 108 | *by E. T. Jaynes* 109 | 110 | In machine learning, we are interested in building probabilistic models and thus you will come across concepts from probability theory like conditional probability and different probability distributions. 111 | 112 | Source: https://bayes.wustl.edu/etj/prob/book.pdf 113 | 114 | ### Information Theory, Inference and Learning Algorithms 115 | *by David J. C. MacKay* 116 | 117 | When you are applying machine learning you are dealing with information processing which in essence relies on ideas from information theory such as entropy and KL Divergence,... 118 | 119 | Book: https://www.inference.org.uk/itprnn/book.html 120 | 121 | ### Statistics and probability 122 | *by Khan Academy* 123 | 124 | A complete overview of statistics and probability required for machine learning. 125 | 126 | Course: https://www.khanacademy.org/math/statistics-probability 127 | 128 | ### Linear Algebra Done Right 129 | *by Sheldon Axler* 130 | 131 | Slides and video lectures on the popular linear algebra book Linear Algebra Done Right. 132 | 133 | Lecture and Slides: https://linear.axler.net/LADRvideos.html 134 | 135 | ### Linear Algebra 136 | *by Khan Academy* 137 | 138 | Vectors, matrices, operations on them, dot & cross product, matrix multiplication etc. is essential for the most basic understanding of ML maths. 139 | 140 | Course: https://www.khanacademy.org/math/linear-algebra 141 | 142 | ## Calculus 143 | *by Khan Academy* 144 | 145 | Precalculus, Differential Calculus, Integral Calculus, Multivariate Calculus 146 | 147 | Course: https://www.khanacademy.org/math/calculus-home 148 | 149 | --- 150 | This collection is far from exhaustive but it should provide a good foundation to start learning some of the mathematical concepts used in machine learning. Reach out on [Twitter](https://twitter.com/omarsar0) if you have any questions. 151 | --------------------------------------------------------------------------------