├── LICENSE ├── README.md ├── exercises ├── backprop.png ├── section-01.md ├── section-03.md ├── section-04.md └── section-07.md ├── readings ├── section-01.md ├── section-02.md └── section-03.md └── slides ├── README.md ├── Section 4 - Multilayer perceptrons.pdf ├── Session 1 - Introduction to Deep Learning - Dive into Deep Learning Study Group.pdf ├── Session 3 - Linear Neural Networks.pdf ├── session-6-intro-cnn.pdf └── session-7-modern-CNN.pdf /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 DAIR.AI 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Deep Learning Study Group 2 | 3 | ### About 4 | In this free online study program, we will be studying the ["Dive into Deep Learning"](https://d2l.ai/index.html) book by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola. 5 | 6 | All study sessions will be free and fully-remote. All sessions will be recorded and uploaded to [YouTube](https://www.youtube.com/channel/UCyna_OxOWL7IEuOwb7WhmxQ?view_as=subscriber). 7 | 8 | Unlike other study groups, the format (tentative) will be as follows: 9 | - Chapters will be presented with slides, which will be followed by code walkthroughs 10 | - For every session, there will be a segment for discussion and Q&A 11 | - After every session, we will assign take-home exercises corresponding to the material covered in that session. We will ask you to upload programming solutions either through GitHub or Google Colab. Find the instructions for each assignment in the full schedule below. 12 | - Before every session, we will provide a list of extra reading material, ahead of time, that will be helpful for the upcoming session. 13 | - We plan to have guest lectures and presentations to provide additional practical value to students (TBA) 14 | - There will be **one final project** which will be done in groups. The groups have to present their work towards the end of the program. (More information coming soon!) 15 | 16 | ### Instructions 17 | - Make sure to join our Slack group(channel: **#d2l-study-group**) for the latest schedule, updates and announcements. Feel free to reach out to me on [Twitter](https://twitter.com/omarsar0) for an invite to our Slack group. 18 | - You will be eligible to receive a **Certificate of Completion** if you complete more than 80% of the exercises. In addition, you must complete and present your final project to be eligible for the certificate. We won't use any kind of grading system and we will just mark the assignments as either complete or incomplete. We hope that all submissions of the exercises are true attempts to complete them. We will provide feedback on all submissions to help with completion when needed. 19 | - You are free to audit the sessions as well. 20 | 21 | ### Schedule and Registration 22 | 23 | To fully register for this program: 24 | - Ensure that you have joined our Slack group for more updates on the program. You will also find a spreadsheet there to officially enroll in the program and signup to be considered for the Certificate of Completion. 25 | 26 | **All dates below are tentative and subject to change.** 27 | 28 | | Chapter | Suggested Readings | Exercises | Live Session | Date/Time | Slides/Notebook | Recording | 29 | |------|-------|-------|------|-----|------|----| 30 | | Session 1 - Introduction to Deep Learning| Find readings [here](https://github.com/dair-ai/d2l-study-group/blob/master/readings/section-01.md) | Complete the list [here](https://github.com/dair-ai/d2l-study-group/blob/master/exercises/section-01.md) | [Zoom (requires registration)](https://us02web.zoom.us/meeting/register/tZwtduyuqjsiHNdD0NxIB2A-rSRhaoEjL9Nn), [YouTube Live](https://www.youtube.com/watch?v=xS3_b0BsSes&feature=youtu.be) | August 1, 2020, 15:00 - 17:00 CEST | [PDF](https://github.com/dair-ai/d2l-study-group/blob/master/slides/Session%201%20-%20Introduction%20to%20Deep%20Learning%20-%20Dive%20into%20Deep%20Learning%20Study%20Group.pdf) | [YouTube](https://www.youtube.com/watch?v=xS3_b0BsSes) | 31 | | Session 2 - Preliminaries| Find readings [here](https://github.com/dair-ai/d2l-study-group/blob/master/readings/section-02.md) | NA| [Zoom](https://us02web.zoom.us/j/84658372189?pwd=am9lY3pHVHhCRGVuSnJDZmVDUHVvQT09) (Check Slack group for password), [YouTube Live](https://youtu.be/RyNM1PdgFUQ) | August 15, 2020, 15:00 - 17:00 CEST | [Preliminaries](https://colab.research.google.com/drive/1a_1pTRPToTXMuLDxzEbdsGTws_AXOY4U?usp=sharing), [Hacking Guide to Neural Networks - Draft](https://colab.research.google.com/drive/1m0lNJ9n8LUXHHU4pOLrZZjSMjoeRKCrN?usp=sharing) | [YouTube](https://youtu.be/RyNM1PdgFUQ) | 32 | | Session 3 - Linear Neural Networks| Find suggested readings [here](https://github.com/dair-ai/d2l-study-group/blob/master/readings/section-03.md) | [Assignment 2](https://github.com/dair-ai/d2l-study-group/blob/master/exercises/section-03.md) | [Zoom](https://us02web.zoom.us/j/81313374436?pwd=c3A5VkFQQVZ6UEFaQnNlTFgwWGoxZz09) (Check Slack group for password) | September 05, 2020, 15:00 - 17:00 CEST | [Slides](https://github.com/dair-ai/d2l-study-group/blob/master/slides/Session%203%20-%20Linear%20Neural%20Networks.pdf), [Notebook](https://colab.research.google.com/drive/1tqdWN073CUxk-Fikcg42ATnjYr3kxfCR?usp=sharing) | [YouTube](https://youtu.be/OFo85Zq3taU) | 33 | | Session 4 - Multilayer Perceptrons| TBA| [Assignment 3](https://github.com/dair-ai/d2l-study-group/blob/master/exercises/section-04.md) | [Zoom (Check Slack group for password)](https://us02web.zoom.us/j/81516716541?pwd=aUpRaXdwM0IrTG1SYWlhTlFwa3JRQT09), [YouTube Live](https://youtu.be/ABWUlfMpDt8) | September 12, 2020, 15:00 - 17:00 CEST | [Notebook](https://colab.research.google.com/drive/1ybr2gkjePOIm4rNQDUL-jpGq4bplHr1N?usp=sharing), [Slides](https://github.com/dair-ai/d2l-study-group/blob/master/slides/Section%204%20-%20Multilayer%20perceptrons.pdf) | [YouTube](https://youtu.be/ABWUlfMpDt8) | 34 | | Session 5 - Deep Learning Computation| TBA | - | - | September 24, 2020, 15:00 - 17:00 CEST | [Notebook](https://colab.research.google.com/drive/15KjalW-DIHPyyqPx4YFuEkmXJkwFINjH?usp=sharing) | [YouTube](https://youtu.be/Sbo30zbquYs) | 35 | | Session 6 - Convolutional Neural Networks| -| TBA | [Zoom - check Slack channel for password](https://us02web.zoom.us/j/82395300378?pwd=QnpKcWJ0VFBpWFlPTTdpN25JSGpKZz09), [YouTube Live](https://youtu.be/9VU4QpHAD5U) | September 26, 2020, 15:00 - 17:00 CEST | [Slides](https://github.com/dair-ai/d2l-study-group/blob/master/slides/session-6-intro-cnn.pdf) | [YouTube](https://youtu.be/9VU4QpHAD5U) | 36 | | Session 7 - Modern Convolutional Neural Networks| - | [Assignment 4](https://github.com/dair-ai/d2l-study-group/blob/master/exercises/section-07.md) | [Zoom (check Slack channel for password)](https://us02web.zoom.us/j/82201914115?pwd=SGJibjRUVEtUVlFyb2xTRU5qR3N5UT09), [YouTube Live](https://youtu.be/57PBRdG99aA) | October 3, 2020, 15:00 - 17:00 CEST | [Slides](https://github.com/dair-ai/d2l-study-group/blob/master/slides/session-7-modern-CNN.pdf) | [YouTube](https://youtu.be/57PBRdG99aA) | 37 | | Session 8 - Recurrent Neural Networks| TBA| TBA | TBA | TBA| TBA | TBA | 38 | | Session 9 - Modern Recurrent Neural Networks| TBA| TBA | TBA | TBA | TBA | TBA | 39 | | Session 10 - Attention Mechanism| TBA| TBA | TBA | TBA| TBA | TBA | 40 | | Session 11 - Optimization Algorithms| TBA| TBA | TBA | TBA | TBA | TBA | 41 | | Session 12 - Computational Performance| TBA| TBA | TBA | TBA | TBA | TBA | 42 | | Session 13 - Computer Vision| TBA| TBA | TBA | TBA| TBA | TBA | 43 | | Session 14 (Part 1) - Natural Language Processing: Pretraining| TBA| TBA | TBA | TBA| TBA | TBA | 44 | | Session 14 (Part 2) - Project Announcement| TBA | TBA | TBA | TBA | TBA | TBA | 45 | | Session 15 - Natural Language Processing: Applications| TBA| TBA | TBA | TBA | TBA | TBA | 46 | | Session 16 - Generative Adversarial Networks| TBA| TBA | TBA| TBA | TBA | TBA | 47 | | Session 17 - Recommender Systems| TBA| TBA | TBA| TBA | TBA | TBA | 48 | | Session 18 - Final Projects Presentation | TBA| TBA | TBA| TBA | TBA | TBA | 49 | 50 | 51 | 52 | ### How to Contribute 53 | If you are interested to deliver chapters from the book, help as a TA, or deliver a special lecture, please reach out to me directly at ellfae@gmail.com. 54 | 55 | --- 56 | 57 | ### Frequently Asked Questions 58 | **Q: How do I register to be considered for the certificate of completion?** 59 | 60 | **A:** You will need to join our Slack group and then enroll officially to be consided for the certificate of completion by adding your name to the spreadsheet shared in the #d2l-study-group channel. Look for the pinned message in the channel. 61 | 62 | --- 63 | **Q: How do I qualify for the certificate of completion?** 64 | 65 | **A:** The first step is to enroll in the program as stated above. Then you will need to complete at least 80% of the exercises assigned throughout the program. You will also need to complete the final project which will be done as a group work and presented towards the end of the program. Failure to complete at least 80% of the assignments or engaging in plagiarism will automatically disqualify from being awarded a certificate of completion. 66 | 67 | --- 68 | **Q: How are the assignments graded?** 69 | 70 | **A:** We don't pass or fail assignments. You will be given either a complete or incomplete status for each assignment. If your assignment is incomplete, we will provide you feedback and will allow you to resubmit but it has to be resubmitted in the period of 48 hours after the original deadline. If you submit the assignments after the deadline they will be labeled as incomplete and we won't provide you feedback for these cases. 71 | 72 | --- 73 | 74 | **Q: Can I audit the sessions of the program?** 75 | 76 | **A:** You are free to audit the sessions without the need to complete the exercises. All sessions will be streamed publicly on both Zoom and YouTube. Schedule information will be provided here and on our [Meetup page](https://www.meetup.com/dair-ai/). 77 | 78 | --- 79 | 80 | **Q: Where do I go for the latest information regarding the program?** 81 | 82 | **A:** All the latest information regarding the program such as schedule, upcoming sessions, and video recodings will be maintained in this repository. If you have any other questions, you can open an issue here or submit your questions in the Slack channel. 83 | 84 | --- 85 | -------------------------------------------------------------------------------- /exercises/backprop.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dair-ai/d2l-study-group/a52a0171d73c049734a5f2ef4c4cc6eb64b2012a/exercises/backprop.png -------------------------------------------------------------------------------- /exercises/section-01.md: -------------------------------------------------------------------------------- 1 | # Exercises 2 | 3 | - [ ] Complete d2l.ai [exercises](https://d2l.ai/chapter_introduction/index.html#exercises) 4 | - [ ] Write one page summary of [Chapter 1](https://d2l.ai/chapter_introduction/index.html), including what you learned and your main takeaways. Feel free to add any additional teachings learned from the [suggested additional readings](https://github.com/dair-ai/d2l-study-group/blob/master/readings/section-01.md). 5 | 6 | **Submission Instructions:** 7 | - Your submission should include a 2-page PDF report including answers to the first set of exercises and the one page summary. 8 | - Make sure to include your full name, email, and date of submission in your report. 9 | - Please include all your references where applicable. You can use either MLA or APA citation style for including references. 10 | - Submit a link to your report (**PDF format**) using this [form](https://forms.gle/uNiauair61YCkBFHA). Make sure to include **1** for "Assignment #". 11 | - Please make sure that the link to the submission is *publicly accessible*. 12 | - You won't receive an actual grade but you we will label your submission as either **completed** or **incomplete**, depending on whether the submission is accepatble or not. Where necessary we will provide feedback and this could help guide completion of the submissions. 13 | - Submission deadline: **August 8, 2020 - (23:59 [Anywhere on Earth](https://www.timeanddate.com/time/zones/aoe))**. Late submissions will not be checked and won't count towards the percentage needed to obtain a certificate of completion. 14 | - Plagiarism is strictly prohibited. If you plagiarize at any point in the program you will be disqualified from obtaining a certificate of completion. 15 | -------------------------------------------------------------------------------- /exercises/section-03.md: -------------------------------------------------------------------------------- 1 | # Assignment 2 2 | 3 | Please implement the backward function from scratch. This is an important part of the bakpropagation algorithm which is used to learn the parameters. Feel free to implement this however you want and make it workable with the linear regression model implemented in this [notebook](https://colab.research.google.com/drive/1tqdWN073CUxk-Fikcg42ATnjYr3kxfCR?usp=sharing). If you have any questions, please ask in the Slack group. 4 | 5 | **Submission Instructions:** 6 | - Clone the following Colab [notebook](https://colab.research.google.com/drive/1tqdWN073CUxk-Fikcg42ATnjYr3kxfCR?usp=sharing) 7 | - Implement the `backward()` fuction from scratch (refer to the [session recording](https://youtu.be/OFo85Zq3taU) and this [notebook](https://colab.research.google.com/drive/1tqdWN073CUxk-Fikcg42ATnjYr3kxfCR?usp=sharing) for more information). Essentially, you will be computing the gradients manually. Feel free to adjust any of the code in the notebook to make your code work with your backward function. 8 | - Hint: First, figure out what this `backward()` function does by checking the source code in the PyTorch documentation. Then, try to incorporate your implementation of it. Make sure to reference any material you use to help you implement this function. 9 | - Submit a link to your Google Colab using this [form](https://forms.gle/uNiauair61YCkBFHA). Make sure to include **2** for "Assignment #". 10 | - The deadline for this assignment is **September 19, 2020**. 11 | -------------------------------------------------------------------------------- /exercises/section-04.md: -------------------------------------------------------------------------------- 1 | # Assignment 3 2 | 3 | The assignment this week is divided into two parts: 4 | - Provide the equations for the backward propagation step in the figure below. In addition, add the path and direction (i.e., how the information flows) in the computation graph. The equations are provided in the d2l.ai book. Make sure to follow the same notations. This [resource](https://colah.github.io/posts/2015-08-Backprop/) by Chris Olah could provide some hints on how you can do this part of the assignment. We are providing the figure as a [Google slide presentation](https://docs.google.com/presentation/d/1igNMML0_zNL2qBi-ausBLRDj6lSpAaLzmPSUV-EVChE/edit?usp=sharing). Create your own copy from this and complete the assignment. We are using the [Math Equations UI](https://gsuite.google.com/marketplace/app/math_equations/825973477142) plugin to add equations to the slide. 5 | ![](https://github.com/dair-ai/d2l-study-group/blob/master/exercises/backprop.png) 6 | 7 | - Create a new Google Colab notebook and show how to implement Xavier initialization from scratch. Compare it with the PyTorch builtin [Xavier implementation](https://pytorch.org/docs/stable/nn.init.html#torch.nn.init.xavier_uniform_). Then show how you would use it effectively to initialize your parameters and train a simple model as used in the notebook. You can use it on any dataset you desire, we just want to see how you apply what you have learned so far. 8 | 9 | If you have any further questions about the assignment please reach out via Slack. 10 | 11 | **Submission Instructions:** 12 | - Submit a link to a PDF report that contains the solution to the first part of the assignment. In that same report, provide a Google Colab link to the second part of the assignment. Use this [form](https://forms.gle/uNiauair61YCkBFHA) to submit the link to the PDF report. Make sure to include **3** for "Assignment #". 13 | - The deadline for this assignment is **September 26, 2020**. 14 | -------------------------------------------------------------------------------- /exercises/section-07.md: -------------------------------------------------------------------------------- 1 | # Assignment 4 2 | 3 | The assignment this week is as follows: 4 | - Select one of the Kaggle datasets suggested below and train a classifier using CNNs. Feel free to use any of the techniques we have learned so far: 5 | 6 | 1. [Animal Image Dataset(DOG, CAT and PANDA)](https://www.kaggle.com/ashishsaxena2209/animal-image-datasetdog-cat-and-panda) 7 | 2. [Natural Images](https://www.kaggle.com/prasunroy/natural-images) 8 | 3. [225 Bird Species](https://www.kaggle.com/gpiosenka/100-bird-species) 9 | 4. [Fruits 360](https://www.kaggle.com/moltean/fruits) 10 | 11 | If you have any further questions about the assignment please reach out via Slack. 12 | 13 | **Submission Instructions:** 14 | - Write a short one page report of your model architecture, findings, and the techniques you used. In the report, provide a link to a publicly accessible Colab notebook, Kaggle notebook, or GitHub repo. Use this [form](https://forms.gle/uNiauair61YCkBFHA) to submit the link to the PDF report. Make sure to include **4** for the assignment number. 15 | - The deadline for this assignment is **October 31, 2020**. 16 | -------------------------------------------------------------------------------- /readings/section-01.md: -------------------------------------------------------------------------------- 1 | # Chapter 1: Introduction to Deep Learning 2 | 3 | In this first chapter, you will be focusing on obtaining a high-level overview of the deep learning field. At this point, we don't recommend you to jump into coding yet. The recommendations below will help you to understand a bit of where the field is and help you understand the motivations behind deep learning methods and their applications. All resources shared below are open and should be accesible to everyone. If you have any issues accessing any of the resources please open an issue. 4 | 5 | ### Papers 📄 6 | - [Deep learning](https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf) (by Yann LeCun, Yoshua Bengio, Geoffrey Hinton) 7 | - [Deep Learning in Neural Networks: An Overview](https://arxiv.org/abs/1404.7828) (by Jürgen Schmidhuber) 8 | 9 | 10 | ### Articles/Chapters/Slides 📝 11 | - [Introduction to Deep Learning](https://d2l.ai/chapter_introduction/index.html) (by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola) 12 | - [Introduction to Deep Learning](http://www.deeplearningbook.org/contents/intro.html) (by Ian Goodfellow and Yoshua Bengio and Aaron Courville) 13 | - [Deep Learning: Our Miraculous Year 1990-1991](http://people.idsia.ch/~juergen/deep-learning-miraculous-year-1990-1991.html) (by Jürgen Schmidhuber) 14 | - [Deep Learning Papers Reading Roadmap](https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap) (by Flood Sung) 15 | 16 | ### Videos 📺 17 | - [Neural Networks and Deep Learning - Complete 4 weeks](https://www.coursera.org/learn/neural-networks-deep-learning?specialization=deep-learning) (by deeplearning.ai) 18 | - [MIT Deep Learning Basic: Introducton and Overview](https://www.youtube.com/watch?v=O5xeyoRL95U) (by Lex fridman) 19 | - [Neural Networks Foundations](https://youtu.be/FBggC-XVF4M) (by DeepMind) 20 | - [History, motivation, and evolution of Deep Learning](https://www.youtube.com/watch?v=0bMe_vCZo30) (by Yann LeCun) 21 | - [But what is a Neural Network?](https://www.youtube.com/watch?v=aircAruvnKk) (by 3Blue1Brown) 22 | 23 | ### Code ⚙️ 24 | - We don't recommend any code implementations for this particular section. 25 | 26 | ### Recommended Books 📚 27 | - [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/index.html)(by Michael Nielsen) 28 | - [Deep Learning](http://www.deeplearningbook.org/) (by Ian Goodfellow and Yoshua Bengio and Aaron Courville) 29 | - [Dive into Deep Learning](https://d2l.ai/index.html) (by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola) 30 | -------------------------------------------------------------------------------- /readings/section-02.md: -------------------------------------------------------------------------------- 1 | # Chapter 2: Deep Learning Preliminaries 2 | In this chapter, you will be covering a few prelinaries that will help you develop basic skills needed to get started with deep learning. Some of these important preliminaries include data manipulation, data processing, linear algebra, calculus, automatic differentiation, and probability. 3 | 4 | ## Linear Algebra and Calculus 5 | 6 | 7 | ### Papers 📄 8 | - [The Matrix Calculus You Need for Deep Learning](https://arxiv.org/abs/1802.01528) (by Terence Parr and Jeremy Howard) 9 | 10 | ### Articles/Chapters/Slides 📝 11 | - [Deep Learning - Linear Algebra](https://www.deeplearningbook.org/contents/linear_algebra.html) (by Ian Goodfellow and Yoshua Bengio and Aaron Courville) 12 | - [Deep Learning - Numerical Computation](https://www.deeplearningbook.org/contents/numerical.html) (by Ian Goodfellow and Yoshua Bengio and Aaron Courville) 13 | 14 | ### Videos 📺 15 | - [MIT 18.065 Matrix Methods in Data Analysis, Signal Processing, and Machine Learning, Spring 2018](https://www.youtube.com/playlist?list=PLUl4u3cNGP63oMNUHXqIUcrkS2PivhN3k) (by MIT, Gilbert Strang) 16 | - [Basic Calculus](https://www.calc1.org/videos/fall-20-online-w-butler) (by Steve Butler) 17 | - [Mathematics for Machine Learning - Multivariate Calculus](https://www.youtube.com/playlist?list=PLiiljHvN6z193BBzS0Ln8NnqQmzimTW23) (by Imperial College London) 18 | - [Mathematics for Machine Learning - Linear Algebra](https://www.youtube.com/playlist?list=PLiiljHvN6z1_o1ztXTKWPrShrMrBLo5P3) (by Imperial College London) 19 | - [Gilbert Strang lectures on Linear Algebra (MIT)](https://www.youtube.com/playlist?list=PL49CF3715CB9EF31D) (by Gilbert Strang) 20 | 21 | ### Code ⚙️ 22 | - [Dive into Deep Learning - Preliminaries](https://d2l.ai/chapter_preliminaries/index.html) (by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola) 23 | - [Dive into Deep Learning - Mathematics for Deep Learning](https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/index.html) (by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola) 24 | 25 | ### Recommended Books 📚 26 | - [Mathematics for Machine Learning](https://mml-book.github.io/) (by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong) 27 | - [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/index.html) (by Michael Nielsen) 28 | - [Deep Learning](http://www.deeplearningbook.org/) (by Ian Goodfellow and Yoshua Bengio and Aaron Courville) 29 | - [Dive into Deep Learning](https://d2l.ai/index.html) (by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola) 30 | 31 | 32 | --- 33 | ## Probability and Statistics 34 | 35 | ### Articles/Chapters/Slides 📝 36 | - [Deep Learning - Probability and Information Theory](https://www.deeplearningbook.org/contents/prob.html) (by Ian Goodfellow and Yoshua Bengio and Aaron Courville) 37 | 38 | ### Videos 📺 39 | - [Statistics and Probability](https://www.khanacademy.org/math/statistics-probability) (by Khan Academy) 40 | - [Discrete Math (Full Course: Sets, Logic, Proofs, Probability, Graph Theory, etc)](https://www.youtube.com/playlist?list=PLHXZ9OQGMqxersk8fUxiUMSIx0DBqsKZS) (by Trefor Bazett) 41 | - [Statistics and Probability](https://www.youtube.com/watch?v=Vfo5le26IhY) (by Great Learning) 42 | 43 | ### Code ⚙️ 44 | - [Dive into Deep Learning - Mathematics for Deep Learning](https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/index.html) (by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola) 45 | 46 | ### Recommended Books 📚 47 | - [Elements of Statistics](https://web.stanford.edu/~hastie/Papers/ESLII.pdf) (Trevor Hastie, Robert Tibshirani, Jerome Friedman) 48 | 49 | --- 50 | ## Data Manipulation and Data Processing 51 | - TBA 52 | 53 | -------------------------------------------------------------------------------- /readings/section-03.md: -------------------------------------------------------------------------------- 1 | # Chapter 3: Linear Neural Networks 2 | In this chapter, we will learn how to cast linear and softmax regression as linear neural networks. You will beging to understand some of the fundamental components that make up a neural network and what it takes to train one. 3 | 4 | ### Papers 📄 5 | - [The Matrix Calculus You Need for Deep Learning](https://arxiv.org/abs/1802.01528) (by Terence Parr and Jeremy Howard) 6 | 7 | ### Articles/Chapters/Slides 📝 8 | - [Linear Neural Networks](https://d2l.ai/chapter_linear-networks/index.html) (by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola) 9 | - [Linear Predictors - Chapter 9 - Understanding Machine Learning: From Theory to Algorithms](https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/copy.html) (by Shai Shalev-Shwartz and Shai Ben-David) 10 | - [Deep Learning - Numerical Computation](https://www.deeplearningbook.org/contents/numerical.html) (by Ian Goodfellow and Yoshua Bengio and Aaron Courville) 11 | 12 | ### Videos 📺 13 | - [Neural Networks](https://www.3blue1brown.com/neural-networks) (by 3blue1brown) 14 | 15 | ### Code ⚙️ 16 | - [Dive into Deep Learning - Linear Neural Networks](https://d2l.ai/chapter_linear-networks/index.html) (by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola) 17 | - [Linear regression explanation and implementation in numpy](https://www.simonwardjones.co.uk/posts/linear_regression/) (by Simon Ward-Jones) 18 | 19 | ### Recommended Books 📚 20 | - [Understanding Machine Learning: From Theory to Algorithms](https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/copy.html) (by Shai Shalev-Shwartz and Shai Ben-David) 21 | - [Mathematics for Machine Learning](https://mml-book.github.io/) (by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong) 22 | - [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/index.html) (by Michael Nielsen) 23 | - [Deep Learning](http://www.deeplearningbook.org/) (by Ian Goodfellow and Yoshua Bengio and Aaron Courville) 24 | - [Dive into Deep Learning](https://d2l.ai/index.html) (by Aston Zhang, Zack C. Lipton, Mu Li, and Alex J. Smola) 25 | 26 | 27 | -------------------------------------------------------------------------------- /slides/README.md: -------------------------------------------------------------------------------- 1 | Slides for our study group. 2 | -------------------------------------------------------------------------------- /slides/Section 4 - Multilayer perceptrons.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dair-ai/d2l-study-group/a52a0171d73c049734a5f2ef4c4cc6eb64b2012a/slides/Section 4 - Multilayer perceptrons.pdf -------------------------------------------------------------------------------- /slides/Session 1 - Introduction to Deep Learning - Dive into Deep Learning Study Group.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dair-ai/d2l-study-group/a52a0171d73c049734a5f2ef4c4cc6eb64b2012a/slides/Session 1 - Introduction to Deep Learning - Dive into Deep Learning Study Group.pdf -------------------------------------------------------------------------------- /slides/Session 3 - Linear Neural Networks.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dair-ai/d2l-study-group/a52a0171d73c049734a5f2ef4c4cc6eb64b2012a/slides/Session 3 - Linear Neural Networks.pdf -------------------------------------------------------------------------------- /slides/session-6-intro-cnn.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dair-ai/d2l-study-group/a52a0171d73c049734a5f2ef4c4cc6eb64b2012a/slides/session-6-intro-cnn.pdf -------------------------------------------------------------------------------- /slides/session-7-modern-CNN.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dair-ai/d2l-study-group/a52a0171d73c049734a5f2ef4c4cc6eb64b2012a/slides/session-7-modern-CNN.pdf --------------------------------------------------------------------------------