├── Part 1 ├── 01. Chap 5-1_Machine Learning Basics_Jinwook Kim.pdf └── 02. Chap 5-2 Machine Learning Basics_Hyun-Lim Yang.pdf ├── Part 2 ├── 03. Chap 6_Deep FeedForward Networks_Eunjeong Yi.pdf ├── 04. Chap 7-1 Regularization for Deep Learning-Keonwoo Noh.pdf ├── 05. Chap 7-2 Regularization for Deep Learning-Hyun-Lim Yang.pdf ├── 06. Chap 8_Optimization for Training Deep Models_Jinwook Kim.pdf ├── 07. Chap 9-1 Convolutional Neural Network_Keonwoo Noh.pdf ├── 08. Chap 9-2_Convolutional Neural Network_Heechul Lim.pdf ├── 09. Chap 10-1_Sequence Modeling Recurrent and Recursive nets_Eunjeong Yi.pdf ├── 10. Chap 10-2 Sequence Modeling Recurrent and Recursive Net-Hyun-Lim Yang.pdf └── 11. Chap 11 12_Practical Methodology and Applications_Heechul Lim.pdf ├── Part 3 ├── 12. Chap 13_Linear Factor Model_Eunjeong Yi.pdf ├── 13. Chap 14 Autoencoders_Keonwoo Noh.pdf ├── 14. Chap 15_Representation learning_Heechul Lim.pdf ├── 15. Chap 16 17_Structured Probabilistic Models and Monte Carlo Methods_Jinwook Kim.pdf ├── 16. Chap 18_Confronting the Partition Function_Eunjeong Yi.pdf ├── 17. Chap 19_Approximate Inference_Heechul Lim.pdf └── 18. Chap 20 Deep Generative Models_Keonwoo Noh.pdf └── README.md /Part 1/01. Chap 5-1_Machine Learning Basics_Jinwook Kim.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 1/01. Chap 5-1_Machine Learning Basics_Jinwook Kim.pdf -------------------------------------------------------------------------------- /Part 1/02. Chap 5-2 Machine Learning Basics_Hyun-Lim Yang.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 1/02. Chap 5-2 Machine Learning Basics_Hyun-Lim Yang.pdf -------------------------------------------------------------------------------- /Part 2/03. Chap 6_Deep FeedForward Networks_Eunjeong Yi.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 2/03. Chap 6_Deep FeedForward Networks_Eunjeong Yi.pdf -------------------------------------------------------------------------------- /Part 2/04. Chap 7-1 Regularization for Deep Learning-Keonwoo Noh.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 2/04. Chap 7-1 Regularization for Deep Learning-Keonwoo Noh.pdf -------------------------------------------------------------------------------- /Part 2/05. Chap 7-2 Regularization for Deep Learning-Hyun-Lim Yang.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 2/05. Chap 7-2 Regularization for Deep Learning-Hyun-Lim Yang.pdf -------------------------------------------------------------------------------- /Part 2/06. Chap 8_Optimization for Training Deep Models_Jinwook Kim.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 2/06. Chap 8_Optimization for Training Deep Models_Jinwook Kim.pdf -------------------------------------------------------------------------------- /Part 2/07. Chap 9-1 Convolutional Neural Network_Keonwoo Noh.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 2/07. Chap 9-1 Convolutional Neural Network_Keonwoo Noh.pdf -------------------------------------------------------------------------------- /Part 2/08. Chap 9-2_Convolutional Neural Network_Heechul Lim.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 2/08. Chap 9-2_Convolutional Neural Network_Heechul Lim.pdf -------------------------------------------------------------------------------- /Part 2/09. Chap 10-1_Sequence Modeling Recurrent and Recursive nets_Eunjeong Yi.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 2/09. Chap 10-1_Sequence Modeling Recurrent and Recursive nets_Eunjeong Yi.pdf -------------------------------------------------------------------------------- /Part 2/10. Chap 10-2 Sequence Modeling Recurrent and Recursive Net-Hyun-Lim Yang.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 2/10. Chap 10-2 Sequence Modeling Recurrent and Recursive Net-Hyun-Lim Yang.pdf -------------------------------------------------------------------------------- /Part 2/11. Chap 11 12_Practical Methodology and Applications_Heechul Lim.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 2/11. Chap 11 12_Practical Methodology and Applications_Heechul Lim.pdf -------------------------------------------------------------------------------- /Part 3/12. Chap 13_Linear Factor Model_Eunjeong Yi.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 3/12. Chap 13_Linear Factor Model_Eunjeong Yi.pdf -------------------------------------------------------------------------------- /Part 3/13. Chap 14 Autoencoders_Keonwoo Noh.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 3/13. Chap 14 Autoencoders_Keonwoo Noh.pdf -------------------------------------------------------------------------------- /Part 3/14. Chap 15_Representation learning_Heechul Lim.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 3/14. Chap 15_Representation learning_Heechul Lim.pdf -------------------------------------------------------------------------------- /Part 3/15. Chap 16 17_Structured Probabilistic Models and Monte Carlo Methods_Jinwook Kim.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 3/15. Chap 16 17_Structured Probabilistic Models and Monte Carlo Methods_Jinwook Kim.pdf -------------------------------------------------------------------------------- /Part 3/16. Chap 18_Confronting the Partition Function_Eunjeong Yi.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 3/16. Chap 18_Confronting the Partition Function_Eunjeong Yi.pdf -------------------------------------------------------------------------------- /Part 3/17. Chap 19_Approximate Inference_Heechul Lim.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 3/17. Chap 19_Approximate Inference_Heechul Lim.pdf -------------------------------------------------------------------------------- /Part 3/18. Chap 20 Deep Generative Models_Keonwoo Noh.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/InfolabAI/DeepLearning/23b7fd95584e3d5393547d285f12b7a0d7dc81c5/Part 3/18. Chap 20 Deep Generative Models_Keonwoo Noh.pdf -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Lecture Slides for Deeplearning book 2 | =================================== 3 | This repo contains lecture slides for [Deeplearning book](http://www.deeplearningbook.org/). This project is maintained by [_InfoLab_ @ DGIST](https://infolab.dgist.ac.kr/) (_Large-scale Deep Learning Team_), and have been made for _InfoSeminar_. It is freely available only if the source is marked. 4 | 5 | > The slides contain additional materials which have not detailed in [the book](http://www.deeplearningbook.org/).
6 | > Also, some materials in the book have been omitted. 7 |
8 | 9 | 10 | 11 | ## Maintained by 12 | **[_InfoLab_](https://infolab.dgist.ac.kr/)** @ DGIST(Daegu Gyeongbuk Institute of Science & Technology)
13 | 14 | ## Coverage 15 | This repo covers Chapter 5 to 20 in the book. 16 | 17 | 18 | ## Credits 19 | Name | Chapters 20 | ------------ | ------------- 21 | [Jinwook Kim](https://infolab.dgist.ac.kr/~bm010515/) | 5-1, 8, 16, 17 22 | [Heechul Lim](https://infolab.dgist.ac.kr/~hclim/) | 9-2, 11, 12, 15, 19 23 | [Hyun-Lim Yang](https://infolab.dgist.ac.kr/~hlyang/) | 5-2, 7-2, 10-2 24 | [Keonwoo Noh](https://infolab.dgist.ac.kr/~kwnoh/) | 7-1, 9-1, 14, 20 25 | [Eunjeong Yi](https://infolab.dgist.ac.kr/~ejyi/) | 6, 10-1, 13, 18 26 | 27 | ## The contents of lecture slides 28 | 29 | ### Part 1. Applied Math and Machine Learning Basics 30 | > ### 5-1. Machine Learning Basics 31 | > - Learning algorithms 32 | > - Capacity, overfitting and underfitting 33 | > - Hyperparameters and validation sets 34 | > - Estimators, bias and variance 35 | > - Maximum likelihood estimation 36 | 37 | > ### 5-2. Machine Learning Basics 38 | > - Bayesian statistics 39 | > - Supervised learning algorithms 40 | > - Unsupervised learning algorithms 41 | > - Stochastic gradient descent 42 | 43 | ### Part 2. Deep Networks: Modern Practices 44 | > ### 6. Deep Feedforward Networks 45 | > - Example: Learning XOR 46 | > - Gradient-Based Learning 47 | > - Hidden Units 48 | > - Architecture Design 49 | > - Back-Propagation and Other Differentiation 50 | 51 | > ### 7-1 Regularization for Deep Learning 52 | > - Parameter Norm Penalties 53 | > - Norm Penalties as Constrained Optimization 54 | > - Regularization and Under-Constrained Problems 55 | > - Dataset Augmentation 56 | > - Noise Robustness 57 | > - Semi-Supervised Learning 58 | > - Multitask Learning 59 | > - Early stopping 60 | 61 | > ### 7-2 Regularization for Deep Learning 62 | > - Parameter Tying and Parameter Sharing 63 | > - Bagging and Other Ensemble Methods 64 | > - Dropout 65 | > - Adversarial Training 66 | 67 | > ### 8 Optimization for Training Deep Models 68 | > - How Learning Differs from Pure Optimization 69 | > - Challenges in Neural Networks 70 | > - Basic Algorithms 71 | > - Algorithms with Adaptive Learning Rates 72 | > - Parameter Initialization Strategies 73 | > - Approximate Second-order Methods 74 | > - Optimization Strategies and Meta-algorithms 75 | 76 | > ### 9-1 Convolutional Networks 77 | > - The Convolution Operation 78 | > - Motivation 79 | > - Pooling 80 | > - Convolution and Pooling as an Infinitely Strong Prior 81 | > - Variants of the Basic Convolution Function 82 | 83 | > ### 9-2 Convolutional Networks 84 | > - Structured Outputs 85 | > - Data Types 86 | > - Efficient Convolution Algorithms 87 | > - Random or Unsupervised Features 88 | > - The Neuroscientific Basis for Convolutional Networks 89 | 90 | > ### 10-1 Sequence modeling: Recurrent and Recursive Nets 91 | > - Unfolding Computational Graphs 92 | > - Recurrent Neural Networks 93 | > - Bidirectional RNNs 94 | > - Encoder-Decoder Sequence-to-Sequence Architectures 95 | > - Deep Recurrent Networks 96 | > - Recursive Neural Networks 97 | 98 | > ### 10-2 Sequence modeling: Recurrent and Recursive Nets 99 | > - The challenge of Long-term 100 | > - Echo State Networks 101 | > - Leaky Units and Other strategies for Multiple Time Scales 102 | > - The Long Short-Term Memory and Other Gated RNNs 103 | > - Optimization for Long-Term Dependencies 104 | > - Explicit Memory 105 | 106 | > ### 11, 12 Practical Methodology and Applications 107 | > - Performance Metrics 108 | > - Default Baseline Models 109 | > - Determining Whether to Gather More Data 110 | > - Selecting Hyperparameters 111 | > - Debugging Strategies 112 | > - Computer Vision 113 | 114 | ### Part 3. Deep Learning Research 115 | > ### 13 Linear Factor Models 116 | > - Probabilistic PCA and Factor Analysis 117 | > - Independent Component Analysis 118 | > - Sparse Coding 119 | 120 | > ### 14 Autoencoders 121 | > - Introduction 122 | > - Stochastic Encoders and Decoders 123 | > - Regularized autoencoders 124 | > - Representational Power, Layer Size and Depth 125 | > 126 | > ### 15 Representation Learning 127 | > - Unsupervised pre-training 128 | > - Introduction of supervised(SL) and unsupervised learning(UL) 129 | > - Representation 130 | > - Clustering 131 | > - K-means 132 | > - Gaussian Mixture Model 133 | > - EM algorithm 134 | > - Practical example 135 | > 136 | > ### 16, 17 Structured Probabilistic Models for Deep Learning and Monte Carlo Methods 137 | > - The Challenge of Unstructured Modeling 138 | > - Using Graphs to Describe Model Structure 139 | > - Sampling from Graphical Models 140 | > - The Deep Learning Approach to Structured Probabilistic Models 141 | > - Sampling and Monte Carlo Methods 142 | > - Markov Chain Monte Carlo Methods 143 | > - Gibbs Sampling 144 | > 145 | > ### 18 Confronting the Partition Function 146 | > - The Log-Likelihood Gradient 147 | > - Stochastic Maximum Likelihood and Contrastive Divergence 148 | > - Estimating the Partition Function 149 | > 150 | > ### 19 Approximate Inference 151 | > - Approximation 152 | > - Maximum Likelihood(MLE) and Maximum A Posteriori(MAP) 153 | > - Inference 154 | > - Taxonomy of deep generative models 155 | > - KL-Divergence 156 | > - Variational Inference 157 | > 158 | > ### 20 Deep Generative Models 159 | > - Generative models 160 | > - Boltzmann Machines 161 | > - Restricted Boltzmann Machines 162 | > - Deep Belief Networks 163 | 164 | 165 | *** 166 | License: [CC-BY](https://creativecommons.org/licenses/by/3.0/) 167 | --------------------------------------------------------------------------------