├── 01. Introduction ├── Introduction.ipynb ├── README.md └── Requirements.txt ├── 02. Model Production ├── BearDetector.ipynb ├── Model │ └── export.pkl ├── README.md └── Requirements.txt ├── 03. Training a Classifier ├── DigitClassifier.ipynb ├── README.md └── Requirements.txt ├── 04. Image Classification ├── ImageClassification.ipynb ├── README.md └── Requirements.txt ├── 05. MultilabelClassification Regression ├── MultilabelClassification.ipynb ├── README.md ├── Regression.ipynb └── Requirements.txt ├── 06. Advanced Classification ├── ImagenetteClassification.ipynb ├── README.md └── Requirements.txt ├── 07. Collaborative Filtering ├── CollaborativeFiltering.ipynb ├── README.md └── Requirements.txt ├── 08. Tabular Modeling ├── README.md ├── Requirements.txt └── TabularModel.ipynb ├── 09. Natural Language Processing ├── NLP.ipynb ├── README.md └── Requirements.txt ├── 10. Data Munging ├── DataMunging.ipynb ├── README.md └── Requirements.txt ├── 11. Language Model ├── LanguageModel.ipynb ├── README.md └── Requirements.txt ├── 12. Convolutional Neural Networks ├── CNN.ipynb ├── README.md └── Requirements.txt ├── 13. ResNets ├── README.md ├── Requirements.txt └── ResNets.ipynb ├── 14. Architecture Details ├── Architectures.ipynb ├── README.md └── Requirements.txt ├── 15. Training Process ├── README.md ├── Requirements.txt └── Training.ipynb ├── 16. Neural Network Foundations ├── NeuralFoundations.ipynb ├── README.md └── Requirements.txt ├── 17. CNN Interpretation ├── CNN Interpretation.ipynb ├── README.md └── Requirements.txt ├── 18. Fastai Learner ├── Fastai Learner.ipynb ├── README.md └── Requirements.txt ├── 19. Chest XRays Classification ├── README.md ├── Requirements.txt └── XRays Classification.ipynb ├── 20. Transformers ├── README.md ├── Requirements.txt └── Transformers.ipynb ├── 21. Stable Diffusion ├── .gitignore ├── README.md ├── app.py ├── stable_diffusion_demo.ipynb └── stable_diffusion_pipe.ipynb ├── LICENSE └── README.md /01. Introduction/README.md: -------------------------------------------------------------------------------- 1 | ## **Fastai : Introduction Notebook** 2 | 3 | The [**Introduction**](https://github.com/ThinamXx/Fastai/blob/main/1.%20Introduction/Introduction.ipynb) notebook is a comprehensive notebook as it contains a list of projects such as **Cat and Dog Classification**, **Semantic Segmentation**, **Sentiment Classification**, **Tabular Classification** and **Recommendation System**. 4 | 5 | **Note:** 6 | - 📑[**Introduction Notebook**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/1.%20Introduction/Introduction.ipynb) 7 | 8 | **1. Cat and Dog Classification** 9 | - I will download and extract a dataset called the Oxford-IIIT Pet Dataset that contains 7349 images of cats and dogs from 37 breeds from the fast.ai datasets collection to the GPU server. I will use pretrained model that has already been trained on 1.3 million images. The pretrained model will be fine tuned using the latest advances in transfer learning to create a model that is specially customized for recognizing dogs and cats. I have presented the implementation of Cat and Dog Classification using Fastai here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20191.PNG) 12 | 13 | **2. Semantic Segmentation** 14 | - Creating a model that can recognize the content of every individual pixel in an image is called Segmentation. 15 | 16 | **3. Sentiment Classification** 17 | - I will create a model that can classify the sentiment of a movie review. Computers can now generate text, translate automatically from one language to another, analyze comments, label words in sentences with the advancement of deep learning in Natural Language Processing. I have presented the implementation of Semantic Segmentation and Sentiment Classification using Fastai here in the snapshot. 18 | 19 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20192.PNG) 20 | 21 | **4. Tabular Classification** 22 | - The model will be using [**Adult**](https://archive.ics.uci.edu/ml/datasets/adult) dataset which contains some demographic data about individuals such as their education, marital status, race, sex and whether they have an annual income greater than $50k. 23 | 24 | **5. Recommendation System** 25 | - I will train a model that will predict movies people might like based on their previous viewing habits using [**MovieLens Dataset**](https://dl.acm.org/doi/10.1145/2827872). 26 | 27 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20193.PNG) 28 | -------------------------------------------------------------------------------- /01. Introduction/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai.vision.all 2 | fastai.text.all 3 | fastai.tabular.all 4 | fastai.collab 5 | -------------------------------------------------------------------------------- /02. Model Production/Model/export.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ThinamXx/Fastai/ee0962bc3277420d6bc13b12cf5c2f4f338b7396/02. Model Production/Model/export.pkl -------------------------------------------------------------------------------- /02. Model Production/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Model to Production** 2 | 3 | The [**BearDetector**](https://github.com/ThinamXx/Fastai/edit/main/2.%20Model%20Production/README.md) notebook contains all the dependencies for a complete **Image Classification** project. 4 | 5 | **Note:** 6 | - 📑[**Image Classification: Bear Detector**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/2.%20Model%20Production/BearDetector.ipynb) 7 | 8 | **Initializing Dataset:** 9 | - I will use [DuckDuckGo](https://duckduckgo.com/) to download images. It is a privacy first search service with many useful features. I have presented the implementation of Gathering Data for Object Detection using Duck Duck Go and Fastai here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20194.PNG) 12 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20195.PNG) 13 | 14 | **Data to DataLoaders:** 15 | - DataLoaders is defined as a Fastai class that stores multiple DataLoader objects you pass to it. Resize crops the images to fit a square shape of the size requested using the full width or height. This can result in losing some important details. Therefore I will ask Fastai to pad the images or stretch or squish the images. When the images are squished or stretched, they end up as unrealistic shapes and leading to a model that learns that things look different from how they actually are and results in lower accuracy. Therefore I will randomly select part of the image and then crop to just that part. On each epoch I will randomly select a different part of each image which means that the model can learn to focus on and recognize different features in images. It also reflects how images work in the real world: different photos of the same thing may be framed in slightly different ways. 16 | 17 | **Data Augmentation:** 18 | - Data Augmentation refers to creating random variations of the input data such that they appear different but do not change the meaning of the data. RandomResizedCrop is a specific example of Data Augmentation. 19 | 20 | **Training the Model:** 21 | - I have presented the implementation of Data Loaders, Data Augmentation and Training the Model using Fastai here in the snapshot. 22 | 23 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20196.PNG) 24 | 25 | **Model Inference:** 26 | - When the model is used for getting predictions instead of training, it is called Inference. To create a Inference Learner, I will use exported file. I have presented the implementation of Classification Interpretation, Cleaning Dataset, Inference Model and Parameters using Fastai here in the snapshot. 27 | 28 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20197.PNG) 29 | -------------------------------------------------------------------------------- /02. Model Production/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook 2 | fastai -------------------------------------------------------------------------------- /03. Training a Classifier/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Training a Classifier** 2 | 3 | The [**DigitClassifier**](https://github.com/ThinamXx/Fastai/blob/main/3.%20Training%20a%20Classifier/DigitClassifier.ipynb) notebook contains all the dependencies required for **Image Classification** project from scratch. 4 | 5 | **Note:** 6 | - 📑[**Digit Classifier**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/3.%20Training%20a%20Classifier/DigitClassifier.ipynb) 7 | 8 | **Getting the Data** 9 | - I will download a sample of MNIST. 10 | 11 | **Pixel Similarity** 12 | - I will get the average of pixel values for each groups of 3 and 7. I will create a tensor containing all the 3s stacked together. I have presented the simple implementation of Pixels and Computer Vision using Fastai here in the snapshot. 13 | 14 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20198.PNG) 15 | 16 | **L1 and L2 Norm** 17 | - Taking the mean of absolute value of differences is called Mean Absolute Difference or L1 Norm. Taking the mean of square of differences and then taking the square root is called Root Mean Squared Error or L2 Norm. I have presented the simple implementation of Arrays and Tensors, L1 and L2 Norm using Fastai here in the snapshot. 18 | 19 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20199.PNG) 20 | 21 | **Stochastic Gradient Descent** 22 | - The steps of **Gradient Descent** process are: 23 | - Step 1: Initialize the Parameters 24 | - Step 2: Calculate the Predictions 25 | - Step 3: Calculate the Loss 26 | - Step 4: Calculate the Gradients 27 | - Step 5: Step the Weights 28 | - Step 6: Repeat the Process 29 | - Step 7: Stop 30 | 31 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20201.PNG) 32 | 33 | **Accuracy and Loss Function** 34 | - The key difference between metric such as accuracy and loss function is that the loss is to drive automated learning and the metric is to drive human understanding. The loss must be a function with meaningful derivative and metrics focuses on performance of the model. The function weights\*pixels is not flexible. I will initialize a random number to intercept as well. In Neural Networks the equation y=w\*x+b, w is called the weights and the b is called the bias. Together the weights and bias make up the parameters. The Sigmoid function always outputs a number between 0 and 1. I have presented the implementation of Loss Function and Sigmoid using Fastai and PyTorch here in the snapshot. 35 | 36 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20203.PNG) 37 | 38 | **SGD and Minibatches** 39 | - The process to change or update the weights based on the gradients in order to consider some of the details involved in the next phase of the learning process is called an Optimization Step. The calculation of average loss for a few data items at a time is called a Minibatch. The number of data items in the Minibatch is called Batchsize. A larger Batchsize means more accurate and stable estimate of the dataset gradients from the loss function whereas a single Batchsize result in an imprecise and unstable gradient. A collection that contains the tuples of independent and dependent variables is known in PyTorch as a Dataset. I have presented the implementation of DataLoader and Gradients using Fastai and PyTorch here in the snapshot. 40 | 41 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20204.PNG) 42 | 43 | **Implementation using Fastai** 44 | - I have presented the implementation of Creating Simple Neural Networks using Fastai and PyTorch here in the snapshot. 45 | 46 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20207.PNG) 47 | -------------------------------------------------------------------------------- /03. Training a Classifier/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai.vision.all 2 | fastbook -------------------------------------------------------------------------------- /04. Image Classification/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Image Classification** 2 | 3 | The [**Image Classification**](https://github.com/ThinamXx/Fastai/blob/main/4.%20Image%20Classification/ImageClassification.ipynb) notebook contains all the dependencies for **Image Classification** such as getting image data ready for modeling i.e presizing and data block summary and for fitting the model i.e learning rate finder, unfreezing, discriminative learning rates, setting the number of epochs and using deeper architectures. It has explanations of cross entropy loss function as well. 4 | 5 | **Note:** 6 | - 📑[**Image Classification**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/4.%20Image%20Classification/ImageClassification.ipynb) 7 | 8 | **Getting the Dataset:** 9 | - I will download the [Pets](https://www.robots.ox.ac.uk/~vgg/data/pets/) dataset. The dataset provides images and annotations directories. The Pets dataset website tells that the annotations directory contains information about where the pets are rather than what they are. Since it is a Classification rather than Localization, I will ignore the annotations directory for now. 10 | 11 | **Initializing Datablock and Dataloaders:** 12 | - I have used Resize as an item transform with a large size and RandomResizedCrop as a batch transform with a smaller size. RandomResizedCrop will be added if min scale parameter is passed in aug transforms function as was done in DataBlock call above. I have presented the implementation of Creating and Debugging Data Block and Data Loaders using Fastai and PyTorch here in the snapshot. 13 | 14 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20208.PNG) 15 | 16 | **Cross Entropy Loss:** 17 | - Cross Entropy Loss is a loss function which works even when the dependent variable has more than two categories. It results faster and more reliable training. When we first take the Softmax and then the Log Likelihood of that, that combination is called Cross Entropy Loss. 18 | 19 | **Softmax Activation Function:** 20 | - The Softmax Activation Function is used in the final layer to ensure that the activations are all between 0 and 1 and that they sum to 1. Softmax is similar to the Sigmoid function. Softmax is the multicategory equivalent of Sigmoid. 21 | 22 | **Exponential Function:** 23 | - Exponential Function is defined as e\*\*x where e is a special number approximately equal to 2.718. It is the inverse of natural logarithm function. Exponential Function is always positive and increases very rapidly. I have presented the implementation of Softmax Function and Negative Log Likelihood using Fastai and PyTorch here in the snapshot. 24 | 25 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20209.PNG) 26 | 27 | **Model Interpretation:** 28 | - I have presented the implementation of Cross Entropy Loss, Confusion Matrix and Learning Rate Finder using Fastai and PyTorch here in the snapshot. 29 | 30 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20210.PNG) 31 | 32 | **Unfreezing and Transfer Learning:** 33 | - I have presented the implementation of Unfreezing and Transfer Learning and Discriminative Learning Rates using Fastai and PyTorch here in the snapshot. 34 | 35 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20211.PNG) 36 | -------------------------------------------------------------------------------- /04. Image Classification/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai.vision.all 2 | fastbook -------------------------------------------------------------------------------- /05. MultilabelClassification Regression/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Multilabel Classification and Regression** 2 | 3 | The [**Multilabel Classification**](https://github.com/ThinamXx/Fastai/blob/main/5.%20MultilabelClassification%20Regression/MultilabelClassification.ipynb) notebook contains all the dependencies required to understand **Multilabel Classification**. It contains the explanations of initializing **DataBlock** and **DataLoaders**. 4 | 5 | The [**Regression**](https://github.com/ThinamXx/Fastai/blob/main/5.%20MultilabelClassification%20Regression/Regression.ipynb) notebook contains all the dependencies required to understand **Image Regression**. 6 | 7 | **Note:** 8 | - 📑[**Multilabel Classification**](https://github.com/ThinamXx/Fastai/blob/main/5.%20MultilabelClassification%20Regression/MultilabelClassification.ipynb) 9 | - 📑[**Image Regression**](https://github.com/ThinamXx/Fastai/blob/main/5.%20MultilabelClassification%20Regression/Regression.ipynb) 10 | 11 | **DataBlock and DataLoaders:** 12 | - Dataset: A collection that returns a tuple of independent and dependent variable for a single item. 13 | - DataLoader: An iterator that provides a stream of minibatches where each minibatch is a couple of a batch of independent variables and a batch of dependent variables. 14 | - Datasets: An iterator that contains a training Dataset and a validation Dataset. 15 | - DataLoaders: An object that contains a training DataLoader and a validation DataLoader. 16 | 17 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20213.PNG) 18 | 19 | **Binary CrossEntropy** 20 | - I will create a **Learner** which contains four things such as the model, a **DataLoaders** object, an **Optimizer** and the loss function to use. **F.binary_cross_entropy** and its module equivalent **nn.BCELoss** calculate cross entropy on a one hot encoded target but don't include the initial sigmoid. Normally, **F.binary_cross_entropy_with_logits** or **nn.BCEWithLogitsLoss** do both sigmoid and binary cross entropy in a single function. Similarly for single label dataset, **F.nll_loss** or **nn.NLLoss** for the version without initial softmax and **F.cross_entropy** or **nn.CrossEntropyLoss** for the version with initial softmax. 21 | 22 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20214.PNG) 23 | 24 | **Image Regression** 25 | - I have presented the implementation of Initializing DataBlock and DataLoaders and Training Image Regression using Fastai and PyTorch here in the snapshot. 26 | 27 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20216a.PNG) 28 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20216b.PNG) 29 | -------------------------------------------------------------------------------- /05. MultilabelClassification Regression/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai 2 | fastai.vision.all 3 | fastbook 4 | -------------------------------------------------------------------------------- /06. Advanced Classification/ImagenetteClassification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "ImagenetteClassification.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [] 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "language_info": { 15 | "name": "python" 16 | }, 17 | "accelerator": "GPU" 18 | }, 19 | "cells": [ 20 | { 21 | "cell_type": "markdown", 22 | "metadata": { 23 | "id": "8jdU0tjMOq8O" 24 | }, 25 | "source": [ 26 | "**INITIALIZATION**\n", 27 | "- I use these three lines of code on top of my each notebooks because it will help to prevent any problems while reloading the same project. And the third line of code helps to make visualization within the notebook." 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "metadata": { 33 | "id": "yVdVcLM6MYjy" 34 | }, 35 | "source": [ 36 | "#@ INITIALIZATION: \n", 37 | "%reload_ext autoreload\n", 38 | "%autoreload 2\n", 39 | "%matplotlib inline" 40 | ], 41 | "execution_count": 1, 42 | "outputs": [] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": { 47 | "id": "4F3cfT9zO2wd" 48 | }, 49 | "source": [ 50 | "**LIBRARIES AND DEPENDENCIES**\n", 51 | "- I have downloaded all the libraries and dependencies required for the project in one particular cell." 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "metadata": { 57 | "id": "vgiKpiETO0C1" 58 | }, 59 | "source": [ 60 | "#@ INSTALLING DEPENDENCIES: UNCOMMENT BELOW: \n", 61 | "# !pip install -Uqq fastbook\n", 62 | "# import fastbook\n", 63 | "# fastbook.setup_book()" 64 | ], 65 | "execution_count": 3, 66 | "outputs": [] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "metadata": { 71 | "id": "RXaHyA1mPBdN" 72 | }, 73 | "source": [ 74 | "#@ DOWNLOADING LIBRARIES AND DEPENDENCIES: \n", 75 | "from fastbook import * # Getting all the Libraries. \n", 76 | "from fastai.callback.fp16 import *\n", 77 | "from fastai.vision.all import * # Getting all the Libraries." 78 | ], 79 | "execution_count": 4, 80 | "outputs": [] 81 | }, 82 | { 83 | "cell_type": "markdown", 84 | "metadata": { 85 | "id": "mu5l5FkGPWfJ" 86 | }, 87 | "source": [ 88 | "**GETTING THE DATASET**\n", 89 | "- I will use **Imagenette** dataset here. " 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "metadata": { 95 | "colab": { 96 | "base_uri": "https://localhost:8080/", 97 | "height": 34 98 | }, 99 | "id": "jbw6TEu-PSKm", 100 | "outputId": "f5d9de9c-e59a-4abf-cd90-b61e00cddb9f" 101 | }, 102 | "source": [ 103 | "#@ GETTING THE DATASET: \n", 104 | "path = untar_data(URLs.IMAGENETTE) # Path to the Dataset. \n", 105 | "path.ls() # Inspecting the Dataset. " 106 | ], 107 | "execution_count": 5, 108 | "outputs": [ 109 | { 110 | "output_type": "display_data", 111 | "data": { 112 | "text/html": [ 113 | "" 114 | ], 115 | "text/plain": [ 116 | "" 117 | ] 118 | }, 119 | "metadata": { 120 | "tags": [] 121 | } 122 | }, 123 | { 124 | "output_type": "execute_result", 125 | "data": { 126 | "text/plain": [ 127 | "(#3) [Path('/root/.fastai/data/imagenette2/train'),Path('/root/.fastai/data/imagenette2/val'),Path('/root/.fastai/data/imagenette2/noisy_imagenette.csv')]" 128 | ] 129 | }, 130 | "metadata": { 131 | "tags": [] 132 | }, 133 | "execution_count": 5 134 | } 135 | ] 136 | }, 137 | { 138 | "cell_type": "markdown", 139 | "metadata": { 140 | "id": "DMVMdfYeQ0ui" 141 | }, 142 | "source": [ 143 | "**DATABLOCK AND DATALOADERS**\n", 144 | "- I will get the dataset into **DataLoaders** object using the **Presizing**. " 145 | ] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "metadata": { 150 | "id": "QqA7IdoOP2Yt" 151 | }, 152 | "source": [ 153 | "#@ INITIALIZING DATABLOCK AND DATALOADERS:\n", 154 | "dblock = DataBlock(blocks=(ImageBlock(), CategoryBlock()), # Initializing DataBlock and Category Block. \n", 155 | " get_items=get_image_files, # Getting Images. \n", 156 | " get_y=parent_label, # Getting Labels. \n", 157 | " item_tfms=Resize(460), # Resizing Images. \n", 158 | " batch_tfms=aug_transforms(size=224, \n", 159 | " min_scale=0.75)) # Initializing DataBlock and Augmentation. \n", 160 | "dls = dblock.dataloaders(path, bs=64) # Initializing DataLoaders and Batchsize. " 161 | ], 162 | "execution_count": 7, 163 | "outputs": [] 164 | }, 165 | { 166 | "cell_type": "markdown", 167 | "metadata": { 168 | "id": "d3oHgAIhTTnA" 169 | }, 170 | "source": [ 171 | "**TRAINING THE MODEL**" 172 | ] 173 | }, 174 | { 175 | "cell_type": "code", 176 | "metadata": { 177 | "colab": { 178 | "base_uri": "https://localhost:8080/", 179 | "height": 204 180 | }, 181 | "id": "zA4w46m2S0gu", 182 | "outputId": "e0470bae-7f06-4ac4-a5a1-bbff3338fa43" 183 | }, 184 | "source": [ 185 | "#@ TRAINING THE MODEL: BASELINE:\n", 186 | "model = xresnet50() # Initializing the Model. \n", 187 | "learn = Learner(dls, model, loss_func=CrossEntropyLossFlat(), \n", 188 | " metrics=accuracy).to_fp16() # Initializing the Learner. \n", 189 | "learn.fit_one_cycle(5, 3e-3) # Training the Model. " 190 | ], 191 | "execution_count": 9, 192 | "outputs": [ 193 | { 194 | "output_type": "display_data", 195 | "data": { 196 | "text/html": [ 197 | "\n", 198 | " \n", 199 | " \n", 200 | " \n", 201 | " \n", 202 | " \n", 203 | " \n", 204 | " \n", 205 | " \n", 206 | " \n", 207 | " \n", 208 | " \n", 209 | " \n", 210 | " \n", 211 | " \n", 212 | " \n", 213 | " \n", 214 | " \n", 215 | " \n", 216 | " \n", 217 | " \n", 218 | " \n", 219 | " \n", 220 | " \n", 221 | " \n", 222 | " \n", 223 | " \n", 224 | " \n", 225 | " \n", 226 | " \n", 227 | " \n", 228 | " \n", 229 | " \n", 230 | " \n", 231 | " \n", 232 | " \n", 233 | " \n", 234 | " \n", 235 | " \n", 236 | " \n", 237 | " \n", 238 | " \n", 239 | " \n", 240 | " \n", 241 | " \n", 242 | " \n", 243 | " \n", 244 | "
epochtrain_lossvalid_lossaccuracytime
01.6307682.1063630.42905202:12
11.2488321.5147850.53248702:13
20.9749680.9991280.70014902:13
30.7389640.7896450.74869302:15
40.5838990.5835390.82076202:15
" 245 | ], 246 | "text/plain": [ 247 | "" 248 | ] 249 | }, 250 | "metadata": { 251 | "tags": [] 252 | } 253 | } 254 | ] 255 | }, 256 | { 257 | "cell_type": "markdown", 258 | "metadata": { 259 | "id": "2_I-8hatLvLa" 260 | }, 261 | "source": [ 262 | "**NORMALIZATION**\n", 263 | "- When the model is training, it helps if the input data is normalized i.e has a mean of 0 and standard deviation of 1. " 264 | ] 265 | }, 266 | { 267 | "cell_type": "code", 268 | "metadata": { 269 | "id": "Hnr8tRL3d_DH", 270 | "colab": { 271 | "base_uri": "https://localhost:8080/" 272 | }, 273 | "outputId": "50a26296-7de4-41fb-d7dc-b316510d1e82" 274 | }, 275 | "source": [ 276 | "#@ GETTING MEAN AND STANDARD DEVIATION: \n", 277 | "x, y = dls.one_batch() # Getting a Batch of Data. \n", 278 | "x.mean(dim=[0, 2, 3]), x.std(dim=[0, 2, 3]) # Getting Mean and SD. " 279 | ], 280 | "execution_count": 10, 281 | "outputs": [ 282 | { 283 | "output_type": "execute_result", 284 | "data": { 285 | "text/plain": [ 286 | "(TensorImage([0.4465, 0.4698, 0.4701], device='cuda:0'),\n", 287 | " TensorImage([0.2860, 0.2840, 0.3174], device='cuda:0'))" 288 | ] 289 | }, 290 | "metadata": { 291 | "tags": [] 292 | }, 293 | "execution_count": 10 294 | } 295 | ] 296 | }, 297 | { 298 | "cell_type": "code", 299 | "metadata": { 300 | "colab": { 301 | "base_uri": "https://localhost:8080/" 302 | }, 303 | "id": "RUIc7mu5OSlH", 304 | "outputId": "7537bc76-30be-4039-d0f8-de6dd1a2618b" 305 | }, 306 | "source": [ 307 | "#@ INITIALIZING NORMALIZATION: \n", 308 | "def get_dls(bs, size):\n", 309 | " dblock = DataBlock(blocks=(ImageBlock, CategoryBlock), # Initializing DataBlock and Category Block.\n", 310 | " get_items=get_image_files, # Getting Images. \n", 311 | " get_y=parent_label, # Getting Labels.\n", 312 | " item_tfms=Resize(460), # Resizing Images.\n", 313 | " batch_tfms=[*aug_transforms(size=size, min_scale=0.75), # Initializing DataBlock.\n", 314 | " Normalize.from_stats(*imagenet_stats)]) # Normalization. \n", 315 | " return dblock.dataloaders(path, bs=bs) # Getting DataLoaders. \n", 316 | "\n", 317 | "#@ GETTING DATALOADERS: \n", 318 | "dls = get_dls(64, 224) # Getting DataLoaders. \n", 319 | "#@ GETTING MEAN AND STANDARD DEVIATION: \n", 320 | "x, y = dls.one_batch() # Getting a Batch of Data. \n", 321 | "x.mean(dim=[0, 2, 3]), x.std(dim=[0, 2, 3]) # Getting Mean and SD. " 322 | ], 323 | "execution_count": 11, 324 | "outputs": [ 325 | { 326 | "output_type": "execute_result", 327 | "data": { 328 | "text/plain": [ 329 | "(TensorImage([-0.0286, 0.0227, 0.1717], device='cuda:0'),\n", 330 | " TensorImage([1.2934, 1.3130, 1.3981], device='cuda:0'))" 331 | ] 332 | }, 333 | "metadata": { 334 | "tags": [] 335 | }, 336 | "execution_count": 11 337 | } 338 | ] 339 | }, 340 | { 341 | "cell_type": "code", 342 | "metadata": { 343 | "colab": { 344 | "base_uri": "https://localhost:8080/", 345 | "height": 204 346 | }, 347 | "id": "njVJk_lRQKZy", 348 | "outputId": "f92dd8e1-77cf-46b2-ac86-4e1b2ec881f8" 349 | }, 350 | "source": [ 351 | "#@ TRAINING THE MODEL: AFTER NORMALIZATION: \n", 352 | "model = xresnet50() # Initializing the Model. \n", 353 | "learn = Learner(dls, model, loss_func=CrossEntropyLossFlat(), \n", 354 | " metrics=accuracy).to_fp16() # Initializing the Learner. \n", 355 | "learn.fit_one_cycle(5, 3e-3) # Training the Model. " 356 | ], 357 | "execution_count": 12, 358 | "outputs": [ 359 | { 360 | "output_type": "display_data", 361 | "data": { 362 | "text/html": [ 363 | "\n", 364 | " \n", 365 | " \n", 366 | " \n", 367 | " \n", 368 | " \n", 369 | " \n", 370 | " \n", 371 | " \n", 372 | " \n", 373 | " \n", 374 | " \n", 375 | " \n", 376 | " \n", 377 | " \n", 378 | " \n", 379 | " \n", 380 | " \n", 381 | " \n", 382 | " \n", 383 | " \n", 384 | " \n", 385 | " \n", 386 | " \n", 387 | " \n", 388 | " \n", 389 | " \n", 390 | " \n", 391 | " \n", 392 | " \n", 393 | " \n", 394 | " \n", 395 | " \n", 396 | " \n", 397 | " \n", 398 | " \n", 399 | " \n", 400 | " \n", 401 | " \n", 402 | " \n", 403 | " \n", 404 | " \n", 405 | " \n", 406 | " \n", 407 | " \n", 408 | " \n", 409 | " \n", 410 | "
epochtrain_lossvalid_lossaccuracytime
01.6542042.8480610.40141902:14
11.2471071.2412980.61389102:14
20.9513461.3966910.63480202:13
30.7471250.7441100.77147102:13
40.5820270.5941420.82038802:14
" 411 | ], 412 | "text/plain": [ 413 | "" 414 | ] 415 | }, 416 | "metadata": { 417 | "tags": [] 418 | } 419 | } 420 | ] 421 | }, 422 | { 423 | "cell_type": "markdown", 424 | "metadata": { 425 | "id": "IHwXZ8U3TLo5" 426 | }, 427 | "source": [ 428 | "**PROGRESSIVE RESIZING**\n", 429 | "- Progressive Resizing is the process of gradually using larger and larger images as training progresses. " 430 | ] 431 | }, 432 | { 433 | "cell_type": "code", 434 | "metadata": { 435 | "colab": { 436 | "base_uri": "https://localhost:8080/", 437 | "height": 173 438 | }, 439 | "id": "QIXZPlXRQezG", 440 | "outputId": "fdbcf852-36f9-48b1-b4e6-7efc6c8c7ccd" 441 | }, 442 | "source": [ 443 | "#@ TRAINING THE MODEL: SMALL IMAGES: \n", 444 | "dls = get_dls(128, 128) # Getting DataLoaders. \n", 445 | "learn = Learner(dls,xresnet50(),loss_func=CrossEntropyLossFlat(), \n", 446 | " metrics=accuracy).to_fp16() # Initializing Learner. \n", 447 | "learn.fit_one_cycle(4, 3e-3) # Training the Model. " 448 | ], 449 | "execution_count": 13, 450 | "outputs": [ 451 | { 452 | "output_type": "display_data", 453 | "data": { 454 | "text/html": [ 455 | "\n", 456 | " \n", 457 | " \n", 458 | " \n", 459 | " \n", 460 | " \n", 461 | " \n", 462 | " \n", 463 | " \n", 464 | " \n", 465 | " \n", 466 | " \n", 467 | " \n", 468 | " \n", 469 | " \n", 470 | " \n", 471 | " \n", 472 | " \n", 473 | " \n", 474 | " \n", 475 | " \n", 476 | " \n", 477 | " \n", 478 | " \n", 479 | " \n", 480 | " \n", 481 | " \n", 482 | " \n", 483 | " \n", 484 | " \n", 485 | " \n", 486 | " \n", 487 | " \n", 488 | " \n", 489 | " \n", 490 | " \n", 491 | " \n", 492 | " \n", 493 | " \n", 494 | " \n", 495 | "
epochtrain_lossvalid_lossaccuracytime
01.8879022.4395640.41523502:00
11.3113011.1909880.62360001:59
20.9704310.9129420.71135201:59
30.7574820.6861850.78603402:00
" 496 | ], 497 | "text/plain": [ 498 | "" 499 | ] 500 | }, 501 | "metadata": { 502 | "tags": [] 503 | } 504 | } 505 | ] 506 | }, 507 | { 508 | "cell_type": "code", 509 | "metadata": { 510 | "id": "_SvcSTD_VesD", 511 | "colab": { 512 | "base_uri": "https://localhost:8080/", 513 | "height": 267 514 | }, 515 | "outputId": "71a30d11-a9ef-463a-f495-61d6959dfa1b" 516 | }, 517 | "source": [ 518 | "#@ TRAINING THE MODEL: LARGE IMAGES: \n", 519 | "learn.dls = get_dls(64, 224) # Initializing DataLoaders. \n", 520 | "learn.fine_tune(5, 1e-3) # Training the Model. " 521 | ], 522 | "execution_count": 14, 523 | "outputs": [ 524 | { 525 | "output_type": "display_data", 526 | "data": { 527 | "text/html": [ 528 | "\n", 529 | " \n", 530 | " \n", 531 | " \n", 532 | " \n", 533 | " \n", 534 | " \n", 535 | " \n", 536 | " \n", 537 | " \n", 538 | " \n", 539 | " \n", 540 | " \n", 541 | " \n", 542 | " \n", 543 | " \n", 544 | " \n", 545 | " \n", 546 | " \n", 547 | "
epochtrain_lossvalid_lossaccuracytime
00.8542250.9004010.71284502:14
" 548 | ], 549 | "text/plain": [ 550 | "" 551 | ] 552 | }, 553 | "metadata": { 554 | "tags": [] 555 | } 556 | }, 557 | { 558 | "output_type": "display_data", 559 | "data": { 560 | "text/html": [ 561 | "\n", 562 | " \n", 563 | " \n", 564 | " \n", 565 | " \n", 566 | " \n", 567 | " \n", 568 | " \n", 569 | " \n", 570 | " \n", 571 | " \n", 572 | " \n", 573 | " \n", 574 | " \n", 575 | " \n", 576 | " \n", 577 | " \n", 578 | " \n", 579 | " \n", 580 | " \n", 581 | " \n", 582 | " \n", 583 | " \n", 584 | " \n", 585 | " \n", 586 | " \n", 587 | " \n", 588 | " \n", 589 | " \n", 590 | " \n", 591 | " \n", 592 | " \n", 593 | " \n", 594 | " \n", 595 | " \n", 596 | " \n", 597 | " \n", 598 | " \n", 599 | " \n", 600 | " \n", 601 | " \n", 602 | " \n", 603 | " \n", 604 | " \n", 605 | " \n", 606 | " \n", 607 | " \n", 608 | "
epochtrain_lossvalid_lossaccuracytime
00.6730410.7461890.76699002:15
10.6451870.7937680.75989502:16
20.5973420.7722770.76475002:15
30.4813230.4804500.85175502:14
40.4411540.4457710.86519802:14
" 609 | ], 610 | "text/plain": [ 611 | "" 612 | ] 613 | }, 614 | "metadata": { 615 | "tags": [] 616 | } 617 | } 618 | ] 619 | }, 620 | { 621 | "cell_type": "markdown", 622 | "metadata": { 623 | "id": "7KXi34wKYL5R" 624 | }, 625 | "source": [ 626 | "**TEST TIME AUGMENTATION**\n", 627 | "- During inference or validation, creating multiple versions of each image using data augmentation and then taking the average or maximum of the predictions for each augmented version of the image is called **Test Time Augmentation**. " 628 | ] 629 | }, 630 | { 631 | "cell_type": "code", 632 | "metadata": { 633 | "id": "tIWNAzwWZMvb", 634 | "colab": { 635 | "base_uri": "https://localhost:8080/", 636 | "height": 54 637 | }, 638 | "outputId": "d8f99245-efb8-4fd0-fb10-8a71cf59b0d3" 639 | }, 640 | "source": [ 641 | "#@ INSPECTING ACCURACY: TEST TIME AUGMENTATION: \n", 642 | "preds, targs = learn.tta() # Implementation of TTA. \n", 643 | "accuracy(preds, targs).item() # Getting Accuracy. " 644 | ], 645 | "execution_count": 15, 646 | "outputs": [ 647 | { 648 | "output_type": "display_data", 649 | "data": { 650 | "text/html": [ 651 | "\n", 652 | "
\n", 653 | " \n", 665 | " \n", 666 | " \n", 667 | "
\n", 668 | " \n" 669 | ], 670 | "text/plain": [ 671 | "" 672 | ] 673 | }, 674 | "metadata": { 675 | "tags": [] 676 | } 677 | }, 678 | { 679 | "output_type": "display_data", 680 | "data": { 681 | "text/html": [ 682 | "" 683 | ], 684 | "text/plain": [ 685 | "" 686 | ] 687 | }, 688 | "metadata": { 689 | "tags": [] 690 | } 691 | }, 692 | { 693 | "output_type": "execute_result", 694 | "data": { 695 | "text/plain": [ 696 | "0.8711724877357483" 697 | ] 698 | }, 699 | "metadata": { 700 | "tags": [] 701 | }, 702 | "execution_count": 15 703 | } 704 | ] 705 | }, 706 | { 707 | "cell_type": "markdown", 708 | "metadata": { 709 | "id": "4xA96UcNaga_" 710 | }, 711 | "source": [ 712 | "**MIXUP AUGMENTATION**\n", 713 | "- Mixup Augmentation works as follows: \n", 714 | " - Select another image from the dataset at random. \n", 715 | " - Pick a weight at random. \n", 716 | " - Take a weighted average using the weight from above of the selected image with your image and it will be independent variable. \n", 717 | " - Take a weighted average using the same weight of this image label with your image label and it will be dependent variable. " 718 | ] 719 | }, 720 | { 721 | "cell_type": "code", 722 | "metadata": { 723 | "colab": { 724 | "base_uri": "https://localhost:8080/", 725 | "height": 204 726 | }, 727 | "id": "KLgONhcDbsqQ", 728 | "outputId": "954deab7-1f69-430b-dd44-5ea5fb243131" 729 | }, 730 | "source": [ 731 | "#@ TRAINING THE MODEL WITH MIXUP AUGMENTATION: REQUIRES MORE EPOCHS: \n", 732 | "model = xresnet50() # Initializing the Model. \n", 733 | "learn = Learner(dls, model, loss_func=CrossEntropyLossFlat(), \n", 734 | " metrics=accuracy, cbs=MixUp).to_fp16() # Initializing the Learner. \n", 735 | "learn.fit_one_cycle(5, 3e-3) # Training the Model. " 736 | ], 737 | "execution_count": 17, 738 | "outputs": [ 739 | { 740 | "output_type": "display_data", 741 | "data": { 742 | "text/html": [ 743 | "\n", 744 | " \n", 745 | " \n", 746 | " \n", 747 | " \n", 748 | " \n", 749 | " \n", 750 | " \n", 751 | " \n", 752 | " \n", 753 | " \n", 754 | " \n", 755 | " \n", 756 | " \n", 757 | " \n", 758 | " \n", 759 | " \n", 760 | " \n", 761 | " \n", 762 | " \n", 763 | " \n", 764 | " \n", 765 | " \n", 766 | " \n", 767 | " \n", 768 | " \n", 769 | " \n", 770 | " \n", 771 | " \n", 772 | " \n", 773 | " \n", 774 | " \n", 775 | " \n", 776 | " \n", 777 | " \n", 778 | " \n", 779 | " \n", 780 | " \n", 781 | " \n", 782 | " \n", 783 | " \n", 784 | " \n", 785 | " \n", 786 | " \n", 787 | " \n", 788 | " \n", 789 | " \n", 790 | "
epochtrain_lossvalid_lossaccuracytime
02.1619762.6070040.36519801:59
11.6998051.5328580.56236002:00
21.4714201.0415810.66579501:59
31.2845370.9510210.70313701:58
41.1825440.7080460.79014202:00
" 791 | ], 792 | "text/plain": [ 793 | "" 794 | ] 795 | }, 796 | "metadata": { 797 | "tags": [] 798 | } 799 | } 800 | ] 801 | }, 802 | { 803 | "cell_type": "markdown", 804 | "metadata": { 805 | "id": "cIfu8h9GflmK" 806 | }, 807 | "source": [ 808 | "**LABEL SMOOTHING**\n", 809 | "- **Label Smoothing** is a process which replaces all the labels i.e 1s with a number a bit less than 1 and 0s with a number a bit more than 0 for training. It will make training more robust even if there is mislabeled data which results to be a model that generalizes better at inference. " 810 | ] 811 | }, 812 | { 813 | "cell_type": "code", 814 | "metadata": { 815 | "colab": { 816 | "base_uri": "https://localhost:8080/", 817 | "height": 204 818 | }, 819 | "id": "pv_H7kvGgvR-", 820 | "outputId": "645b7b47-5c00-4e90-df9b-bb4b5917eb68" 821 | }, 822 | "source": [ 823 | "#@ TRAINING THE MODEL WITH LABEL SMOOTHING: REQUIRES MORE EPOCHS: \n", 824 | "model = xresnet50() # Initializing the Model. \n", 825 | "learn = Learner(dls,model,loss_func=LabelSmoothingCrossEntropy(), \n", 826 | " metrics=accuracy).to_fp16() # Initializing the Learner. \n", 827 | "learn.fit_one_cycle(5, 3e-3) # Training the Model. " 828 | ], 829 | "execution_count": 18, 830 | "outputs": [ 831 | { 832 | "output_type": "display_data", 833 | "data": { 834 | "text/html": [ 835 | "\n", 836 | " \n", 837 | " \n", 838 | " \n", 839 | " \n", 840 | " \n", 841 | " \n", 842 | " \n", 843 | " \n", 844 | " \n", 845 | " \n", 846 | " \n", 847 | " \n", 848 | " \n", 849 | " \n", 850 | " \n", 851 | " \n", 852 | " \n", 853 | " \n", 854 | " \n", 855 | " \n", 856 | " \n", 857 | " \n", 858 | " \n", 859 | " \n", 860 | " \n", 861 | " \n", 862 | " \n", 863 | " \n", 864 | " \n", 865 | " \n", 866 | " \n", 867 | " \n", 868 | " \n", 869 | " \n", 870 | " \n", 871 | " \n", 872 | " \n", 873 | " \n", 874 | " \n", 875 | " \n", 876 | " \n", 877 | " \n", 878 | " \n", 879 | " \n", 880 | " \n", 881 | " \n", 882 | "
epochtrain_lossvalid_lossaccuracytime
02.7464743.7160410.34055301:59
12.2353282.0531930.64339102:00
21.9549262.0163630.65160602:00
31.7569291.6411830.78976801:59
41.6150351.5902360.80918602:00
" 883 | ], 884 | "text/plain": [ 885 | "" 886 | ] 887 | }, 888 | "metadata": { 889 | "tags": [] 890 | } 891 | } 892 | ] 893 | } 894 | ] 895 | } -------------------------------------------------------------------------------- /06. Advanced Classification/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Advanced Classification** 2 | 3 | The [**Imagenette Classification**](https://github.com/ThinamXx/Fastai/blob/main/6.%20Advanced%20Classification/ImagenetteClassification.ipynb) notebook contains all the dependencies required to train a state of art machine learning model in computer vision whether from scratch or using transfer learning. It contains explanations and implementation of **Normalization**, **Progressive Resizing**, **Test Time Augmentation**, **Mixup Augmentation** and **Label Smoothing**. 4 | 5 | **Note:** 6 | - 📑[**Imagenette Classification**](https://github.com/ThinamXx/Fastai/blob/main/6.%20Advanced%20Classification/ImagenetteClassification.ipynb) 7 | 8 | **Progressive Resizing** 9 | - Progressive Resizing is the process of gradually using larger and larger images as training progresses. When the model is training, it helps if the input data is normalized i.e has a mean of 0 and standard deviation of 1. I have presented the implementation of Initializing DataBlock and DataLoaders, Normalization and Progressive Resizing using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20217a.PNG) 12 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20217b.PNG) 13 | 14 | **Test Time Augmentation** 15 | - During inference or validation, creating multiple versions of each image using data augmentation and then taking the average or maximum of the predictions for each augmented version of the image is called Test Time Augmentation. 16 | 17 | **Mixup Augmentation** 18 | - Mixup Augmentation works as follows: 19 | - Select another image from the dataset at random. 20 | - Pick a weight at random. 21 | - Take a weighted average using the weight from above of the selected image with your image and it will be independent variable. 22 | - Take a weighted average using the same weight of this image label with your image label and it will be dependent variable. 23 | 24 | **Label Smoothing** 25 | - Label Smoothing is a process which replaces all the labels i.e 1s with a number a bit less than 1 and 0s with a number a bit more than 0 for training. It will make training more robust even if there is mislabeled data which results to be a model that generalizes better at inference. I have presented the implementation of Progressive Resizing, Test Time Augmentation, Mixup Augmentation and Label Smoothing using Fastai and PyTorch here in the snapshot. 26 | 27 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20218.PNG) 28 | -------------------------------------------------------------------------------- /06. Advanced Classification/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai.vision.all 2 | fastbook 3 | fastai.callback.fp16 4 | -------------------------------------------------------------------------------- /07. Collaborative Filtering/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Collaborative Filtering** 2 | 3 | The [**Collaborative Filtering**](https://github.com/ThinamXx/Fastai/blob/main/7.%20Collaborative%20Filtering/CollaborativeFiltering.ipynb) notebook contains all the dependencies required to build a **Recommendation System**. It presents how gradient descent can learn intrinsic factors or biases about items from a history of ratings which then gives information about the data. 4 | 5 | **Note:** 6 | - 📑[**Collaborative Filtering**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/7.%20Collaborative%20Filtering/CollaborativeFiltering.ipynb) 7 | 8 | **Loading the Data** 9 | - I will use the subset of [**MovieLens**](https://grouplens.org/datasets/movielens/) dataset. The entire dataset contains tens of millions of movie rankings i.e a combination of a movie ID, a user ID and a numeric rating. The mathematical operation of multiplying the elements of two vectors together and then summing up the result is called **Dot Product**. I have presented the implementation of Initializing Dataset and Creating DataLoaders using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20219.PNG) 12 | 13 | **Embedding** 14 | - The special layer that indexes into a vector using an integer but has its derivative calculated in such a way that it is identical to what it would have been if it had done a matrix multiplication with a one hot encoded vector is called **Embedding**. Multiplying by a one hot encoded matrix using the computational shortcut that it can be implemented by simply indexing directly. The thing that multiply the one hot encoded matrix is called the **Embedding Matrix**. 15 | 16 | **Weight Decay Regularization** 17 | - **Weight Decay** consists of adding sum of the squared weights to the loss function. The idea is that the larger the coefficients are, the sharper the canyons will be in the loss function. 18 | 19 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20221a.PNG) 20 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20221b.PNG) 21 | 22 | -------------------------------------------------------------------------------- /07. Collaborative Filtering/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai.tabular.all 2 | fastai.collab 3 | -------------------------------------------------------------------------------- /08. Tabular Modeling/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Tabular Modeling** 2 | 3 | The [**Tabular Model**](https://github.com/ThinamXx/Fastai/blob/main/8.%20Tabular%20Modeling/TabularModel.ipynb) notebook contains all the dependencies required for **Tabular Modeling**. It presents the detailed explanations of two approaches to **Tabular Modeling: Decision Tree Ensembles and Neural Networks**. 4 | 5 | **Note:** 6 | - 📑[**Tabular Modeling**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/8.%20Tabular%20Modeling/TabularModel.ipynb) 7 | 8 | **Loading the Data** 9 | - I will use the dataset from [**Blue Book for Bulldozers Kaggle Competition**](https://www.kaggle.com/c/bluebook-for-bulldozers/overview) here. I have presented the implementation of Handling Dates, Tabular Pandas and Tabular Proc using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20224.PNG) 12 | 13 | **Random Forest** 14 | - A Random Forest is a model that averages the predictions of a large number of decision trees which are generated by randomly varying various parameters that specify what data is used to train the tree and other tree parameters. Bagging is a particular approach to ensembling or combining the results of multiple models together. Random Forest Model just averages the predictions of a number of trees and therefore it can never predict values outside the range of the training data. Random Forests are not able to extrapolate outside the types of data i.e out of domain data. 15 | 16 | **Out of Bag Error** 17 | - The Out of Bag Error or OOB error is a way of measuring prediction error in the training dataset by including in the calculation of a rows error trees only where that row was not included in the training. I have presented the implementation of Creating Random Forest and Model Interpretation using Fastai and PyTorch here in the snapshot. 18 | 19 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20226.PNG) 20 | 21 | **Neural Networks** 22 | 23 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20231a.PNG) 24 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20231b.PNG) 25 | -------------------------------------------------------------------------------- /08. Tabular Modeling/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook 2 | kaggle 3 | pandas 4 | sklearn 5 | fastai.tabular.all 6 | fastai.collab 7 | waterfallcharts 8 | treeinterpreter 9 | dtreeviz 10 | warnings 11 | -------------------------------------------------------------------------------- /09. Natural Language Processing/NLP.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "NLP.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [] 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "language_info": { 15 | "name": "python" 16 | }, 17 | "accelerator": "GPU" 18 | }, 19 | "cells": [ 20 | { 21 | "cell_type": "markdown", 22 | "metadata": { 23 | "id": "jk5SHfqIc6Ey" 24 | }, 25 | "source": [ 26 | "### **INITIALIZATION:**\n", 27 | "- I use these three lines of code on top of my each notebooks because it will help to prevent any problems while reloading the same project. And the third line of code helps to make visualization within the notebook." 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "metadata": { 33 | "id": "z9K2einKadKr" 34 | }, 35 | "source": [ 36 | "#@ INITIALIZATION: \n", 37 | "%reload_ext autoreload\n", 38 | "%autoreload 2\n", 39 | "%matplotlib inline" 40 | ], 41 | "execution_count": 1, 42 | "outputs": [] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": { 47 | "id": "RTFZaPENdHTs" 48 | }, 49 | "source": [ 50 | "**LIBRARIES AND DEPENDENCIES:**\n", 51 | "- I have downloaded all the libraries and dependencies required for the project in one particular cell." 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "metadata": { 57 | "id": "GZDkvIrydDtY" 58 | }, 59 | "source": [ 60 | "#@ INSTALLING DEPENDENCIES: UNCOMMENT BELOW: \n", 61 | "# !pip install -Uqq fastbook\n", 62 | "# import fastbook\n", 63 | "# fastbook.setup_book()" 64 | ], 65 | "execution_count": 3, 66 | "outputs": [] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "metadata": { 71 | "id": "qTIdUUtEdOah" 72 | }, 73 | "source": [ 74 | "#@ DOWNLOADING LIBRARIES AND DEPENDENCIES: \n", 75 | "from fastbook import * # Getting all the Libraries. \n", 76 | "from fastai.callback.fp16 import *\n", 77 | "from fastai.text.all import * # Getting all the Libraries.\n", 78 | "from IPython.display import display, HTML" 79 | ], 80 | "execution_count": 4, 81 | "outputs": [] 82 | }, 83 | { 84 | "cell_type": "markdown", 85 | "metadata": { 86 | "id": "PAY-kcjCd-Tz" 87 | }, 88 | "source": [ 89 | "### **GETTING THE DATASET:**\n", 90 | "- I will get the **IMDB Dataset** here. " 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "metadata": { 96 | "colab": { 97 | "base_uri": "https://localhost:8080/", 98 | "height": 54 99 | }, 100 | "id": "iv-s-9D1dsia", 101 | "outputId": "e45ff349-4494-4c2d-a21a-6184d08452b7" 102 | }, 103 | "source": [ 104 | "#@ GETTING THE DATASET: \n", 105 | "path = untar_data(URLs.IMDB) # Getting Path to the Dataset. \n", 106 | "path.ls() # Inspecting the Path. " 107 | ], 108 | "execution_count": 5, 109 | "outputs": [ 110 | { 111 | "output_type": "display_data", 112 | "data": { 113 | "text/html": [ 114 | "" 115 | ], 116 | "text/plain": [ 117 | "" 118 | ] 119 | }, 120 | "metadata": { 121 | "tags": [] 122 | } 123 | }, 124 | { 125 | "output_type": "execute_result", 126 | "data": { 127 | "text/plain": [ 128 | "(#7) [Path('/root/.fastai/data/imdb/unsup'),Path('/root/.fastai/data/imdb/tmp_clas'),Path('/root/.fastai/data/imdb/train'),Path('/root/.fastai/data/imdb/imdb.vocab'),Path('/root/.fastai/data/imdb/test'),Path('/root/.fastai/data/imdb/README'),Path('/root/.fastai/data/imdb/tmp_lm')]" 129 | ] 130 | }, 131 | "metadata": { 132 | "tags": [] 133 | }, 134 | "execution_count": 5 135 | } 136 | ] 137 | }, 138 | { 139 | "cell_type": "code", 140 | "metadata": { 141 | "colab": { 142 | "base_uri": "https://localhost:8080/", 143 | "height": 35 144 | }, 145 | "id": "dBavrB_6e_Of", 146 | "outputId": "87fe6ba0-d4e2-4921-8f5c-09d3aa9d3877" 147 | }, 148 | "source": [ 149 | "#@ GETTING TEXT FILES: \n", 150 | "files = get_text_files(path, folders=[\"train\", \"test\", \"unsup\"]) # Getting Text Files. \n", 151 | "txt = files[0].open().read() # Getting a Text. \n", 152 | "txt[:75] # Inspecting Text. " 153 | ], 154 | "execution_count": 6, 155 | "outputs": [ 156 | { 157 | "output_type": "execute_result", 158 | "data": { 159 | "application/vnd.google.colaboratory.intrinsic+json": { 160 | "type": "string" 161 | }, 162 | "text/plain": [ 163 | "\"Well it wasn't horrible but it wasn't great. No where near as good as the o\"" 164 | ] 165 | }, 166 | "metadata": { 167 | "tags": [] 168 | }, 169 | "execution_count": 6 170 | } 171 | ] 172 | }, 173 | { 174 | "cell_type": "markdown", 175 | "metadata": { 176 | "id": "kG832kuRgXeo" 177 | }, 178 | "source": [ 179 | "### **WORD TOKENIZATION:**\n", 180 | "- **Word Tokenization** splits a sentence on spaces as well as applying language specific rules to try to separate parts of meaning even when there are no spaces. Generally punctuation marks are also split into separate tokens. **Token** is a element of a list created by the **Tokenization** process which could be a word, a part of a word or subword or a single character. " 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "metadata": { 186 | "colab": { 187 | "base_uri": "https://localhost:8080/" 188 | }, 189 | "id": "yx1lsBcYgQ9K", 190 | "outputId": "a97b8cdd-70b7-473b-e322-aefacf89acfc" 191 | }, 192 | "source": [ 193 | "#@ INITIALIZING WORD TOKENIZATION: \n", 194 | "spacy = WordTokenizer() # Initializing Tokenizer. \n", 195 | "toks = first(spacy([txt])) # Getting Tokens of Words. \n", 196 | "print(coll_repr(toks, 30)) # Inspecting Tokens. " 197 | ], 198 | "execution_count": 7, 199 | "outputs": [ 200 | { 201 | "output_type": "stream", 202 | "text": [ 203 | "(#298) ['Well','it','was',\"n't\",'horrible','but','it','was',\"n't\",'great','.','No','where','near','as','good','as','the','original','.','It','kinda','tried','to','hard','to','be','the','first','movie'...]\n" 204 | ], 205 | "name": "stdout" 206 | } 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "metadata": { 212 | "colab": { 213 | "base_uri": "https://localhost:8080/" 214 | }, 215 | "id": "ti9Hi_uriKxe", 216 | "outputId": "3ab387c8-7c32-4966-bed7-fd8a701d8f3e" 217 | }, 218 | "source": [ 219 | "#@ INSPECTING TOKENIZATION: EXAMPLE:\n", 220 | "first(spacy(['The U.S. dollar $1 is $1.00.'])) # Inspecting Tokens. " 221 | ], 222 | "execution_count": 8, 223 | "outputs": [ 224 | { 225 | "output_type": "execute_result", 226 | "data": { 227 | "text/plain": [ 228 | "(#9) ['The','U.S.','dollar','$','1','is','$','1.00','.']" 229 | ] 230 | }, 231 | "metadata": { 232 | "tags": [] 233 | }, 234 | "execution_count": 8 235 | } 236 | ] 237 | }, 238 | { 239 | "cell_type": "code", 240 | "metadata": { 241 | "colab": { 242 | "base_uri": "https://localhost:8080/" 243 | }, 244 | "id": "liWRboFJi84p", 245 | "outputId": "dee03077-aa0c-4f98-9f81-44bf1904ebee" 246 | }, 247 | "source": [ 248 | "#@ INITIALIZING WORD TOKENIZATION WITH FASTAI: \n", 249 | "tkn = Tokenizer(spacy) # Initializing Tokenizer. \n", 250 | "print(coll_repr(tkn(txt), 31)) # Inspecting Tokens. " 251 | ], 252 | "execution_count": 9, 253 | "outputs": [ 254 | { 255 | "output_type": "stream", 256 | "text": [ 257 | "(#331) ['xxbos','xxmaj','well','it','was',\"n't\",'horrible','but','it','was',\"n't\",'great','.','xxmaj','no','where','near','as','good','as','the','original','.','xxmaj','it','kinda','tried','to','hard','to','be'...]\n" 258 | ], 259 | "name": "stdout" 260 | } 261 | ] 262 | }, 263 | { 264 | "cell_type": "markdown", 265 | "metadata": { 266 | "id": "lsVt5tA8P1hU" 267 | }, 268 | "source": [ 269 | "**Note:**\n", 270 | "- **xxbos** : Indicates the beginning of a text. \n", 271 | "- **xxmaj** : Indicates the next word begins with a capital. \n", 272 | "- **xxunk** : Indicates the next word is unknown. " 273 | ] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "metadata": { 278 | "colab": { 279 | "base_uri": "https://localhost:8080/", 280 | "height": 35 281 | }, 282 | "id": "X-GgI67Ajr-K", 283 | "outputId": "d74da75e-bd5e-4308-c48f-4e2353f7c647" 284 | }, 285 | "source": [ 286 | "#@ INSPECTING TOKENIZATION: EXAMPLE:\n", 287 | "coll_repr(tkn('© Fast.ai www.fast.ai/INDEX'), 30) # Inspecting Tokens. " 288 | ], 289 | "execution_count": 10, 290 | "outputs": [ 291 | { 292 | "output_type": "execute_result", 293 | "data": { 294 | "application/vnd.google.colaboratory.intrinsic+json": { 295 | "type": "string" 296 | }, 297 | "text/plain": [ 298 | "\"(#11) ['xxbos','©','xxmaj','fast.ai','xxrep','3','w','.fast.ai','/','xxup','index']\"" 299 | ] 300 | }, 301 | "metadata": { 302 | "tags": [] 303 | }, 304 | "execution_count": 10 305 | } 306 | ] 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "metadata": { 311 | "id": "Z8clYuvALmBg" 312 | }, 313 | "source": [ 314 | "### **SUBWORD TOKENIZATION:**\n", 315 | "- **Word Tokenization** relies on an assumption that spaces provide a useful separation of components of meaning in a sentence which is not always appropriate. Languages such as Chinese and Japanese don't use spaces and in such cases **Subword Tokenization** generally plays the best role. **Subword Tokenization** splits words into smaller parts based on the most commonly occurring sub strings. " 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "metadata": { 321 | "id": "AO-mr3aLm2QB", 322 | "colab": { 323 | "base_uri": "https://localhost:8080/", 324 | "height": 52 325 | }, 326 | "outputId": "3bc61f99-1e67-4d46-a21e-cbc7c46f0933" 327 | }, 328 | "source": [ 329 | "#@ INITIALIZING SUBWORD TOKENIZATION: EXAMPLE:\n", 330 | "txts = L(o.open().read() for o in files[:2000]) # Getting List of Reviews. \n", 331 | "\n", 332 | "#@ INITIALIZING SUBWORD TOKENIZER: \n", 333 | "def subword(sz): # Defining Function. \n", 334 | " sp = SubwordTokenizer(vocab_sz=sz) # Initializing Subword Tokenizer. \n", 335 | " sp.setup(txts) # Getting Sequence of Characters. \n", 336 | " return \" \".join(first(sp([txt]))[:40]) # Inspecting the Vocab. \n", 337 | "\n", 338 | "#@ IMPLEMENTATION: \n", 339 | "subword(1000) # Inspecting Subword Tokenization. " 340 | ], 341 | "execution_count": 11, 342 | "outputs": [ 343 | { 344 | "output_type": "display_data", 345 | "data": { 346 | "text/html": [ 347 | "" 348 | ], 349 | "text/plain": [ 350 | "" 351 | ] 352 | }, 353 | "metadata": { 354 | "tags": [] 355 | } 356 | }, 357 | { 358 | "output_type": "execute_result", 359 | "data": { 360 | "application/vnd.google.colaboratory.intrinsic+json": { 361 | "type": "string" 362 | }, 363 | "text/plain": [ 364 | "\"▁We ll ▁it ▁was n ' t ▁horrible ▁but ▁it ▁was n ' t ▁great . ▁No ▁where ▁near ▁as ▁good ▁as ▁the ▁original . ▁It ▁kind a ▁tri ed ▁to ▁hard ▁to ▁be ▁the ▁first ▁movie . ▁I ▁think\"" 365 | ] 366 | }, 367 | "metadata": { 368 | "tags": [] 369 | }, 370 | "execution_count": 11 371 | } 372 | ] 373 | }, 374 | { 375 | "cell_type": "markdown", 376 | "metadata": { 377 | "id": "KWSi_eQzQcxL" 378 | }, 379 | "source": [ 380 | "**Notes:**\n", 381 | "- Here **setup** is a special fastai method that is called automatically in usual data processing pipelines which reads the documents and find the common sequences of characters to create the vocab. Similarly [**L**](https://fastcore.fast.ai/#L) is also referred as superpowered list. The special character '_' represents a space character in the original text. " 382 | ] 383 | }, 384 | { 385 | "cell_type": "code", 386 | "metadata": { 387 | "colab": { 388 | "base_uri": "https://localhost:8080/", 389 | "height": 52 390 | }, 391 | "id": "MpSPYDjIQbVP", 392 | "outputId": "75fbfe62-3b0e-48a7-d503-67e2a52b6c82" 393 | }, 394 | "source": [ 395 | "#@ IMPLEMENTATION OF SUBWORD TOKENIZATION: \n", 396 | "subword(200) # Inspecting Vocab. \n", 397 | "subword(10000) # Inspecting Vocab. " 398 | ], 399 | "execution_count": 12, 400 | "outputs": [ 401 | { 402 | "output_type": "display_data", 403 | "data": { 404 | "text/html": [ 405 | "" 406 | ], 407 | "text/plain": [ 408 | "" 409 | ] 410 | }, 411 | "metadata": { 412 | "tags": [] 413 | } 414 | }, 415 | { 416 | "output_type": "display_data", 417 | "data": { 418 | "text/html": [ 419 | "" 420 | ], 421 | "text/plain": [ 422 | "" 423 | ] 424 | }, 425 | "metadata": { 426 | "tags": [] 427 | } 428 | }, 429 | { 430 | "output_type": "execute_result", 431 | "data": { 432 | "application/vnd.google.colaboratory.intrinsic+json": { 433 | "type": "string" 434 | }, 435 | "text/plain": [ 436 | "\"▁Well ▁it ▁wasn ' t ▁horrible ▁but ▁it ▁wasn ' t ▁great . ▁No ▁where ▁near ▁as ▁good ▁as ▁the ▁original . ▁It ▁kind a ▁tried ▁to ▁hard ▁to ▁be ▁the ▁first ▁movie . ▁I ▁think ▁it ▁needed ▁a ▁better\"" 437 | ] 438 | }, 439 | "metadata": { 440 | "tags": [] 441 | }, 442 | "execution_count": 12 443 | } 444 | ] 445 | }, 446 | { 447 | "cell_type": "markdown", 448 | "metadata": { 449 | "id": "xTxPsa_RT-oU" 450 | }, 451 | "source": [ 452 | "**Note:**\n", 453 | "- A larger vocab means fewer tokens per sentence which means faster training, less memory, and less state for the model to remember but it means larger embedding matrices and require more data to learn. **Subword Tokenization** provides a way to easily scale between character tokenization i.e. using a small subword vocab and word tokenization i.e using a large subword vocab and handles every human language without needing language specific algorithms to be developed. " 454 | ] 455 | }, 456 | { 457 | "cell_type": "markdown", 458 | "metadata": { 459 | "id": "9scF0s-DVQjf" 460 | }, 461 | "source": [ 462 | "### **NUMERICALIZATION:**\n", 463 | "- **Numericalization** is the process of mapping tokens to integers. It involves making a list of all possible levels of that categorical variable or the vocab and replacing each level with its index in the vocab." 464 | ] 465 | }, 466 | { 467 | "cell_type": "code", 468 | "metadata": { 469 | "colab": { 470 | "base_uri": "https://localhost:8080/" 471 | }, 472 | "id": "vV8wXsyFTb42", 473 | "outputId": "256b3df0-a2aa-4826-ba1a-74454dfd901d" 474 | }, 475 | "source": [ 476 | "#@ INITIALIZING TOKENS: \n", 477 | "toks = tkn(txt) # Getting Tokens. \n", 478 | "print(coll_repr(tkn(txt), 31)) # Inspecting Tokens. " 479 | ], 480 | "execution_count": 13, 481 | "outputs": [ 482 | { 483 | "output_type": "stream", 484 | "text": [ 485 | "(#331) ['xxbos','xxmaj','well','it','was',\"n't\",'horrible','but','it','was',\"n't\",'great','.','xxmaj','no','where','near','as','good','as','the','original','.','xxmaj','it','kinda','tried','to','hard','to','be'...]\n" 486 | ], 487 | "name": "stdout" 488 | } 489 | ] 490 | }, 491 | { 492 | "cell_type": "code", 493 | "metadata": { 494 | "colab": { 495 | "base_uri": "https://localhost:8080/" 496 | }, 497 | "id": "5GORafHQXscv", 498 | "outputId": "47f5ece8-fad8-4402-8181-a6a8e763a528" 499 | }, 500 | "source": [ 501 | "#@ INITIALIZING TOKENS: \n", 502 | "toks200 = txts[:200].map(tkn) # Getting Tokens. \n", 503 | "toks200[0] # Inspecting Tokens. " 504 | ], 505 | "execution_count": 14, 506 | "outputs": [ 507 | { 508 | "output_type": "execute_result", 509 | "data": { 510 | "text/plain": [ 511 | "(#331) ['xxbos','xxmaj','well','it','was',\"n't\",'horrible','but','it','was'...]" 512 | ] 513 | }, 514 | "metadata": { 515 | "tags": [] 516 | }, 517 | "execution_count": 14 518 | } 519 | ] 520 | }, 521 | { 522 | "cell_type": "code", 523 | "metadata": { 524 | "colab": { 525 | "base_uri": "https://localhost:8080/", 526 | "height": 35 527 | }, 528 | "id": "ta9px3-xZfNs", 529 | "outputId": "b1dc6542-a1a8-4591-a774-54743f09096b" 530 | }, 531 | "source": [ 532 | "#@ NUMERICALIZATION USING FASTAI: \n", 533 | "num = Numericalize() # Initializing Numericalization. \n", 534 | "num.setup(toks200) # Getting Integers. \n", 535 | "coll_repr(num.vocab, 20) # Inspecting Vocabulary. " 536 | ], 537 | "execution_count": 15, 538 | "outputs": [ 539 | { 540 | "output_type": "execute_result", 541 | "data": { 542 | "application/vnd.google.colaboratory.intrinsic+json": { 543 | "type": "string" 544 | }, 545 | "text/plain": [ 546 | "\"(#2120) ['xxunk','xxpad','xxbos','xxeos','xxfld','xxrep','xxwrep','xxup','xxmaj','the',',','.','and','a','of','to','is','in','it','i'...]\"" 547 | ] 548 | }, 549 | "metadata": { 550 | "tags": [] 551 | }, 552 | "execution_count": 15 553 | } 554 | ] 555 | }, 556 | { 557 | "cell_type": "code", 558 | "metadata": { 559 | "colab": { 560 | "base_uri": "https://localhost:8080/", 561 | "height": 35 562 | }, 563 | "id": "tt8fU-YNfROJ", 564 | "outputId": "ac4ee7a6-e7aa-4c61-f910-7c939dd7bcdb" 565 | }, 566 | "source": [ 567 | "#@ INITIALIZING NUMERICALIZATION: \n", 568 | "nums = num(toks)[:20]; nums # Inspection. \n", 569 | "\" \".join(num.vocab[o] for o in nums) # Getting Original Text. " 570 | ], 571 | "execution_count": 16, 572 | "outputs": [ 573 | { 574 | "output_type": "execute_result", 575 | "data": { 576 | "application/vnd.google.colaboratory.intrinsic+json": { 577 | "type": "string" 578 | }, 579 | "text/plain": [ 580 | "\"xxbos xxmaj well it was n't horrible but it was n't great . xxmaj no where near as good as\"" 581 | ] 582 | }, 583 | "metadata": { 584 | "tags": [] 585 | }, 586 | "execution_count": 16 587 | } 588 | ] 589 | }, 590 | { 591 | "cell_type": "markdown", 592 | "metadata": { 593 | "id": "F_ZJYk8jVowc" 594 | }, 595 | "source": [ 596 | "### **CREATING BATCHES FOR LANGUAGE MODEL:**\n", 597 | "- At every epoch I will shuffle the collection of documents and concatenate them into a stream of tokens and cut that stream into a batch of fixedsize consecutive ministreams. The model will then read the ministreams in order. " 598 | ] 599 | }, 600 | { 601 | "cell_type": "code", 602 | "metadata": { 603 | "colab": { 604 | "base_uri": "https://localhost:8080/" 605 | }, 606 | "id": "thsG0lr0Xq1w", 607 | "outputId": "56362e99-1d77-4877-eee9-043d2ae0c806" 608 | }, 609 | "source": [ 610 | "#@ CREATING BATCHES FOR LANGUAGE MODEL: \n", 611 | "nums200 = toks200.map(num) # Initializing Numericalization. \n", 612 | "dl = LMDataLoader(nums200) # Creating Language Model Data Loaders. \n", 613 | "\n", 614 | "#@ INSPECTING FIRST BATCH: \n", 615 | "x, y = first(dl) # Getting First Batch of Data. \n", 616 | "x.shape, y.shape # Inspecting Shape of Data. " 617 | ], 618 | "execution_count": 17, 619 | "outputs": [ 620 | { 621 | "output_type": "execute_result", 622 | "data": { 623 | "text/plain": [ 624 | "(torch.Size([64, 72]), torch.Size([64, 72]))" 625 | ] 626 | }, 627 | "metadata": { 628 | "tags": [] 629 | }, 630 | "execution_count": 17 631 | } 632 | ] 633 | }, 634 | { 635 | "cell_type": "code", 636 | "metadata": { 637 | "colab": { 638 | "base_uri": "https://localhost:8080/", 639 | "height": 35 640 | }, 641 | "id": "nYkjwoIMZEIQ", 642 | "outputId": "73ab7655-5530-4077-c6ee-8f8e1d4be81b" 643 | }, 644 | "source": [ 645 | "#@ INSPECTING THE DATA: \n", 646 | "\" \".join(num.vocab[o] for o in x[0][:20]) # Inspecting Independent Variable. " 647 | ], 648 | "execution_count": 18, 649 | "outputs": [ 650 | { 651 | "output_type": "execute_result", 652 | "data": { 653 | "application/vnd.google.colaboratory.intrinsic+json": { 654 | "type": "string" 655 | }, 656 | "text/plain": [ 657 | "\"xxbos xxmaj well it was n't horrible but it was n't great . xxmaj no where near as good as\"" 658 | ] 659 | }, 660 | "metadata": { 661 | "tags": [] 662 | }, 663 | "execution_count": 18 664 | } 665 | ] 666 | }, 667 | { 668 | "cell_type": "code", 669 | "metadata": { 670 | "colab": { 671 | "base_uri": "https://localhost:8080/", 672 | "height": 35 673 | }, 674 | "id": "jnoIOJ2oZsNJ", 675 | "outputId": "179fe7c0-6cda-491d-8d4f-0d9c8c1cc08d" 676 | }, 677 | "source": [ 678 | "#@ INSPECTING THE DATA: \n", 679 | "\" \".join(num.vocab[o] for o in y[0][:20]) # Inspecting Dependent Variable. " 680 | ], 681 | "execution_count": 19, 682 | "outputs": [ 683 | { 684 | "output_type": "execute_result", 685 | "data": { 686 | "application/vnd.google.colaboratory.intrinsic+json": { 687 | "type": "string" 688 | }, 689 | "text/plain": [ 690 | "\"xxmaj well it was n't horrible but it was n't great . xxmaj no where near as good as the\"" 691 | ] 692 | }, 693 | "metadata": { 694 | "tags": [] 695 | }, 696 | "execution_count": 19 697 | } 698 | ] 699 | }, 700 | { 701 | "cell_type": "markdown", 702 | "metadata": { 703 | "id": "KYAR0Mx9a9DY" 704 | }, 705 | "source": [ 706 | "### **TRAINING A TEXT CLASSIFIER:**" 707 | ] 708 | }, 709 | { 710 | "cell_type": "markdown", 711 | "metadata": { 712 | "id": "uDYi_GrzfvqP" 713 | }, 714 | "source": [ 715 | "**LANGUAGE MODEL USING DATABLOCK:**" 716 | ] 717 | }, 718 | { 719 | "cell_type": "code", 720 | "metadata": { 721 | "colab": { 722 | "base_uri": "https://localhost:8080/", 723 | "height": 17 724 | }, 725 | "id": "NjRlvq8KbbXF", 726 | "outputId": "5b9453c8-24e0-4fa8-f199-4d9ed3deb96a" 727 | }, 728 | "source": [ 729 | "#@ CREATING LANGUAGE MODEL USING DATABLOCK: \n", 730 | "get_imdb = partial(get_text_files, folders=[\"train\", \"test\", \"unsup\"]) # Getting Text Files. \n", 731 | "db = DataBlock(blocks=TextBlock.from_folder(path, is_lm=True), # Initializing TextBlock. \n", 732 | " get_items=get_imdb, splitter=RandomSplitter(0.1)) # Initializing DataBlock. \n", 733 | "\n", 734 | "#@ CREATING LANGUAGE MODEL DATALOADERS: \n", 735 | "dls_lm = db.dataloaders(path, path=path, bs=128, seq_len=80) # Initializing Data Loaders. " 736 | ], 737 | "execution_count": 20, 738 | "outputs": [ 739 | { 740 | "output_type": "display_data", 741 | "data": { 742 | "text/html": [ 743 | "" 744 | ], 745 | "text/plain": [ 746 | "" 747 | ] 748 | }, 749 | "metadata": { 750 | "tags": [] 751 | } 752 | } 753 | ] 754 | }, 755 | { 756 | "cell_type": "code", 757 | "metadata": { 758 | "colab": { 759 | "base_uri": "https://localhost:8080/", 760 | "height": 247 761 | }, 762 | "id": "t8zab_Z5fBgA", 763 | "outputId": "a75a06d5-5dde-4af7-ace8-c33647941da3" 764 | }, 765 | "source": [ 766 | "#@ INSPECTING THE BATCHES OF DATA: \n", 767 | "dls_lm.show_batch(max_n=2)" 768 | ], 769 | "execution_count": 21, 770 | "outputs": [ 771 | { 772 | "output_type": "display_data", 773 | "data": { 774 | "text/html": [ 775 | "\n", 776 | " \n", 777 | " \n", 778 | " \n", 779 | " \n", 780 | " \n", 781 | " \n", 782 | " \n", 783 | " \n", 784 | " \n", 785 | " \n", 786 | " \n", 787 | " \n", 788 | " \n", 789 | " \n", 790 | " \n", 791 | " \n", 792 | " \n", 793 | " \n", 794 | " \n", 795 | "
texttext_
0xxbos xxmaj this is basically a goofball comedy , with somewhat odd pacing due to some dramatic elements . xxmaj for xxmaj michael xxup j. xxmaj fox and xxmaj paul xxmaj reubens , it was their first xxunk had previously been in a short lived xxup tv series and a xxup tv movie ) . \\n\\n xxmaj since the movie is basically a race / scavenger hunt type movie , like \" cannonball xxmaj run \" , \" it 'sxxmaj this is basically a goofball comedy , with somewhat odd pacing due to some dramatic elements . xxmaj for xxmaj michael xxup j. xxmaj fox and xxmaj paul xxmaj reubens , it was their first xxunk had previously been in a short lived xxup tv series and a xxup tv movie ) . \\n\\n xxmaj since the movie is basically a race / scavenger hunt type movie , like \" cannonball xxmaj run \" , \" it 's a
1concerned only with the behavior of its characters , it 's original and challenging . xxmaj then it turns into a story filled with familiar elements , and by the end everything is happening by the numbers . xxbos xxmaj what goes through the mind of the office drone that snaps and shoots up the place ? xxmaj what drives a person like that ? xxmaj come on , we 've all seen the reports \" he xxmaj was aonly with the behavior of its characters , it 's original and challenging . xxmaj then it turns into a story filled with familiar elements , and by the end everything is happening by the numbers . xxbos xxmaj what goes through the mind of the office drone that snaps and shoots up the place ? xxmaj what drives a person like that ? xxmaj come on , we 've all seen the reports \" he xxmaj was a xxmaj
" 796 | ], 797 | "text/plain": [ 798 | "" 799 | ] 800 | }, 801 | "metadata": { 802 | "tags": [] 803 | } 804 | } 805 | ] 806 | }, 807 | { 808 | "cell_type": "markdown", 809 | "metadata": { 810 | "id": "Oa7FZgFegs1r" 811 | }, 812 | "source": [ 813 | "**FINETUNING THE LANGAUGE MODEL:**\n", 814 | "- I will use **Embeddings** to convert the integer word indices into activations that can be used for the neural networks. These embeddings are feed into **Recurrent Neural Network** using and architecture called **AWD-LSTM**. " 815 | ] 816 | }, 817 | { 818 | "cell_type": "code", 819 | "metadata": { 820 | "colab": { 821 | "base_uri": "https://localhost:8080/", 822 | "height": 17 823 | }, 824 | "id": "K3Qda2V4f9O_", 825 | "outputId": "a47f5c8e-cdbc-4e47-c295-a0186abad3e4" 826 | }, 827 | "source": [ 828 | "#@ INITIALIZING LANGUAGE MODEL LEARNER: \n", 829 | "learn = language_model_learner(dls_lm, AWD_LSTM, drop_mult=0.3, # Using AWD LSTM Architecture. \n", 830 | " metrics=[accuracy, Perplexity()]).to_fp16() # Initializing LM Learner. " 831 | ], 832 | "execution_count": 22, 833 | "outputs": [ 834 | { 835 | "output_type": "display_data", 836 | "data": { 837 | "text/html": [ 838 | "" 839 | ], 840 | "text/plain": [ 841 | "" 842 | ] 843 | }, 844 | "metadata": { 845 | "tags": [] 846 | } 847 | } 848 | ] 849 | }, 850 | { 851 | "cell_type": "code", 852 | "metadata": { 853 | "colab": { 854 | "base_uri": "https://localhost:8080/", 855 | "height": 80 856 | }, 857 | "id": "DwMl0Jl5mZuA", 858 | "outputId": "859de7e5-7d5f-4dff-cf34-dcc1890894bb" 859 | }, 860 | "source": [ 861 | "#@ TRAINING EMBEDDINGS WITH RANDOM INITIALIZATION: \n", 862 | "learn.fit_one_cycle(1, 2e-2)" 863 | ], 864 | "execution_count": null, 865 | "outputs": [ 866 | { 867 | "output_type": "display_data", 868 | "data": { 869 | "text/html": [ 870 | "\n", 871 | " \n", 872 | " \n", 873 | " \n", 874 | " \n", 875 | " \n", 876 | " \n", 877 | " \n", 878 | " \n", 879 | " \n", 880 | " \n", 881 | " \n", 882 | " \n", 883 | " \n", 884 | " \n", 885 | " \n", 886 | " \n", 887 | " \n", 888 | " \n", 889 | " \n", 890 | " \n", 891 | "
epochtrain_lossvalid_lossaccuracyperplexitytime
04.0066343.9039910.29904949.60001821:03
" 892 | ], 893 | "text/plain": [ 894 | "" 895 | ] 896 | }, 897 | "metadata": { 898 | "tags": [] 899 | } 900 | } 901 | ] 902 | }, 903 | { 904 | "cell_type": "markdown", 905 | "metadata": { 906 | "id": "PlBstN_DoYVY" 907 | }, 908 | "source": [ 909 | "**SAVING AND LOADING MODELS:**" 910 | ] 911 | }, 912 | { 913 | "cell_type": "code", 914 | "metadata": { 915 | "colab": { 916 | "base_uri": "https://localhost:8080/" 917 | }, 918 | "id": "KJP-rFdEnDH_", 919 | "outputId": "e85809a6-da32-49e1-82e8-2b4eec8aaa7e" 920 | }, 921 | "source": [ 922 | "#@ SAVING MODELS: \n", 923 | "learn.save(\"/content/gdrive/MyDrive/1Epoch\") # Saving the Model. " 924 | ], 925 | "execution_count": null, 926 | "outputs": [ 927 | { 928 | "output_type": "execute_result", 929 | "data": { 930 | "text/plain": [ 931 | "Path('/content/gdrive/MyDrive/1Epoch.pth')" 932 | ] 933 | }, 934 | "metadata": { 935 | "tags": [] 936 | }, 937 | "execution_count": 34 938 | } 939 | ] 940 | }, 941 | { 942 | "cell_type": "code", 943 | "metadata": { 944 | "id": "UweY10WKotCV" 945 | }, 946 | "source": [ 947 | "#@ LOADING MODELS: \n", 948 | "learn = learn.load(\"/content/gdrive/MyDrive/1Epoch\") # Loading the Model. " 949 | ], 950 | "execution_count": 23, 951 | "outputs": [] 952 | }, 953 | { 954 | "cell_type": "code", 955 | "metadata": { 956 | "colab": { 957 | "base_uri": "https://localhost:8080/", 958 | "height": 359 959 | }, 960 | "id": "cqgqbPgQxn7o", 961 | "outputId": "707075b6-6ad5-40fb-e3cf-a4c1d1ef1887" 962 | }, 963 | "source": [ 964 | "#@ FINETUNING THE LANGUAGE MODEL: \n", 965 | "learn.unfreeze() # Unfreezing the Layers. \n", 966 | "learn.fit_one_cycle(10, 2e-3) # Training the Model. " 967 | ], 968 | "execution_count": 24, 969 | "outputs": [ 970 | { 971 | "output_type": "display_data", 972 | "data": { 973 | "text/html": [ 974 | "\n", 975 | " \n", 976 | " \n", 977 | " \n", 978 | " \n", 979 | " \n", 980 | " \n", 981 | " \n", 982 | " \n", 983 | " \n", 984 | " \n", 985 | " \n", 986 | " \n", 987 | " \n", 988 | " \n", 989 | " \n", 990 | " \n", 991 | " \n", 992 | " \n", 993 | " \n", 994 | " \n", 995 | " \n", 996 | " \n", 997 | " \n", 998 | " \n", 999 | " \n", 1000 | " \n", 1001 | " \n", 1002 | " \n", 1003 | " \n", 1004 | " \n", 1005 | " \n", 1006 | " \n", 1007 | " \n", 1008 | " \n", 1009 | " \n", 1010 | " \n", 1011 | " \n", 1012 | " \n", 1013 | " \n", 1014 | " \n", 1015 | " \n", 1016 | " \n", 1017 | " \n", 1018 | " \n", 1019 | " \n", 1020 | " \n", 1021 | " \n", 1022 | " \n", 1023 | " \n", 1024 | " \n", 1025 | " \n", 1026 | " \n", 1027 | " \n", 1028 | " \n", 1029 | " \n", 1030 | " \n", 1031 | " \n", 1032 | " \n", 1033 | " \n", 1034 | " \n", 1035 | " \n", 1036 | " \n", 1037 | " \n", 1038 | " \n", 1039 | " \n", 1040 | " \n", 1041 | " \n", 1042 | " \n", 1043 | " \n", 1044 | " \n", 1045 | " \n", 1046 | " \n", 1047 | " \n", 1048 | " \n", 1049 | " \n", 1050 | " \n", 1051 | " \n", 1052 | " \n", 1053 | " \n", 1054 | " \n", 1055 | " \n", 1056 | " \n", 1057 | " \n", 1058 | " \n", 1059 | " \n", 1060 | " \n", 1061 | " \n", 1062 | " \n", 1063 | " \n", 1064 | " \n", 1065 | " \n", 1066 | " \n", 1067 | "
epochtrain_lossvalid_lossaccuracyperplexitytime
04.3383804.2534960.29175470.35094522:53
13.9757563.9088510.31335049.84167123:06
23.7986443.7622200.32388243.04386122:59
33.6908223.6824230.33061739.74258822:59
43.6061293.6406160.33428338.11531422:53
53.5296133.6101610.33743236.97200823:02
63.4554613.5897170.33972836.22383923:07
73.3843313.5825240.34102835.96420323:23
83.3401923.5809340.34152635.90704723:15
93.3129693.5837580.34137736.00862123:11
" 1068 | ], 1069 | "text/plain": [ 1070 | "" 1071 | ] 1072 | }, 1073 | "metadata": { 1074 | "tags": [] 1075 | } 1076 | } 1077 | ] 1078 | }, 1079 | { 1080 | "cell_type": "markdown", 1081 | "metadata": { 1082 | "id": "bKYtCD3ozJ8f" 1083 | }, 1084 | "source": [ 1085 | "**ENCODER**\n", 1086 | "- **Encoder** is defined as the model which doesn't contain task specific final layers. The term **Encoder** means much the same thing as body when applied to vision **CNN** but **Encoder** tends to be more used for NLP and generative models." 1087 | ] 1088 | }, 1089 | { 1090 | "cell_type": "code", 1091 | "metadata": { 1092 | "id": "YmDWEDIfy3Pw" 1093 | }, 1094 | "source": [ 1095 | "#@ SAVING MODELS: \n", 1096 | "learn.save_encoder(\"/content/gdrive/MyDrive/FineTuned\") # Saving the Model. " 1097 | ], 1098 | "execution_count": 25, 1099 | "outputs": [] 1100 | }, 1101 | { 1102 | "cell_type": "markdown", 1103 | "metadata": { 1104 | "id": "SmrreawCleIT" 1105 | }, 1106 | "source": [ 1107 | "### **TEXT GENERATION:**\n", 1108 | "- I will use the model to write new reviews. " 1109 | ] 1110 | }, 1111 | { 1112 | "cell_type": "code", 1113 | "metadata": { 1114 | "id": "r0qs7j0P1Hio", 1115 | "colab": { 1116 | "base_uri": "https://localhost:8080/", 1117 | "height": 88 1118 | }, 1119 | "outputId": "c17e773c-5e03-4745-81b6-148b7cbb66ae" 1120 | }, 1121 | "source": [ 1122 | "#@ INITIALIZING TEXT GENERATION: \n", 1123 | "TEXT = \"I hate the movie because\" # Text Example. \n", 1124 | "N_WORDS = 40\n", 1125 | "N_SENTENCES = 2\n", 1126 | "preds = [learn.predict(TEXT, N_WORDS, temperature=0.75)\n", 1127 | " for _ in range(N_SENTENCES)] # Initializing Text Generation. \n", 1128 | "print(\" \".join(preds)) # Inspecting Generated Reviews. " 1129 | ], 1130 | "execution_count": 26, 1131 | "outputs": [ 1132 | { 1133 | "output_type": "display_data", 1134 | "data": { 1135 | "text/html": [ 1136 | "" 1137 | ], 1138 | "text/plain": [ 1139 | "" 1140 | ] 1141 | }, 1142 | "metadata": { 1143 | "tags": [] 1144 | } 1145 | }, 1146 | { 1147 | "output_type": "display_data", 1148 | "data": { 1149 | "text/html": [ 1150 | "" 1151 | ], 1152 | "text/plain": [ 1153 | "" 1154 | ] 1155 | }, 1156 | "metadata": { 1157 | "tags": [] 1158 | } 1159 | }, 1160 | { 1161 | "output_type": "stream", 1162 | "text": [ 1163 | "i hate the movie because of the awful acting and the Canadian scenery . \n", 1164 | "\n", 1165 | " This movie is not worth a penny . There is nothing funny or funny about this movie . Here 's another one of those b movies i hate the movie because it is so predictable and ridiculous . No , this one is not for people who are not . This movie is a \" thriller \" , with a lot of nudity and problem in the plot .\n" 1166 | ], 1167 | "name": "stdout" 1168 | } 1169 | ] 1170 | }, 1171 | { 1172 | "cell_type": "markdown", 1173 | "metadata": { 1174 | "id": "4wtpZHuJn966" 1175 | }, 1176 | "source": [ 1177 | "### **TEXT CLASSIFICATION:**\n", 1178 | "- Here, I am moving towards **Classifier** fine tuning rather than **Language Model** fine tuning as mentioned above. A **Language Model** predicts the next word of a document so it doesn't require any external labels. A **Classifier** predicts an external labels. " 1179 | ] 1180 | }, 1181 | { 1182 | "cell_type": "code", 1183 | "metadata": { 1184 | "colab": { 1185 | "base_uri": "https://localhost:8080/", 1186 | "height": 247 1187 | }, 1188 | "id": "xpjJuTaUo65Z", 1189 | "outputId": "87c20cf4-06fb-4585-bd7b-31ef88eeefea" 1190 | }, 1191 | "source": [ 1192 | "#@ INITIALIZING DATALOADERS:\n", 1193 | "db_clas = DataBlock(blocks=(TextBlock.from_folder(path, vocab=dls_lm.vocab), # Initializing Text Blocks. \n", 1194 | " CategoryBlock), # Initializing Category Block.\n", 1195 | " get_y=parent_label, # Getting Target. \n", 1196 | " get_items=partial(get_text_files,folders=[\"train\",\"test\"]), # Getting Text Files. \n", 1197 | " splitter=GrandparentSplitter(valid_name=\"test\")) # Splitting the Data. \n", 1198 | "dls_clas = db_clas.dataloaders(path, path=path, bs=128, seq_len=72) # Initializing DataLoaders. \n", 1199 | "\n", 1200 | "#@ INSPECTING THE BATCHES: \n", 1201 | "dls_clas.show_batch(max_n=2) # Inspection. " 1202 | ], 1203 | "execution_count": 27, 1204 | "outputs": [ 1205 | { 1206 | "output_type": "display_data", 1207 | "data": { 1208 | "text/html": [ 1209 | "\n", 1210 | " \n", 1211 | " \n", 1212 | " \n", 1213 | " \n", 1214 | " \n", 1215 | " \n", 1216 | " \n", 1217 | " \n", 1218 | " \n", 1219 | " \n", 1220 | " \n", 1221 | " \n", 1222 | " \n", 1223 | " \n", 1224 | " \n", 1225 | " \n", 1226 | " \n", 1227 | " \n", 1228 | " \n", 1229 | "
textcategory
0xxbos xxmaj match 1 : xxmaj tag xxmaj team xxmaj table xxmaj match xxmaj bubba xxmaj ray and xxmaj spike xxmaj dudley vs xxmaj eddie xxmaj guerrero and xxmaj chris xxmaj benoit xxmaj bubba xxmaj ray and xxmaj spike xxmaj dudley started things off with a xxmaj tag xxmaj team xxmaj table xxmaj match against xxmaj eddie xxmaj guerrero and xxmaj chris xxmaj benoit . xxmaj according to the rules of the match , both opponents have to go through tables in order to get the win . xxmaj benoit and xxmaj guerrero heated up early on by taking turns hammering first xxmaj spike and then xxmaj bubba xxmaj ray . a xxmaj german xxunk by xxmaj benoit to xxmaj bubba took the wind out of the xxmaj dudley brother . xxmaj spike tried to help his brother , but the referee restrained him while xxmaj benoit and xxmaj guerreropos
1xxbos * * attention xxmaj spoilers * * \\n\\n xxmaj first of all , let me say that xxmaj rob xxmaj roy is one of the best films of the 90 's . xxmaj it was an amazing achievement for all those involved , especially the acting of xxmaj liam xxmaj neeson , xxmaj jessica xxmaj lange , xxmaj john xxmaj hurt , xxmaj brian xxmaj cox , and xxmaj tim xxmaj roth . xxmaj michael xxmaj canton xxmaj jones painted a wonderful portrait of the honor and dishonor that men can represent in themselves . xxmaj but alas … \\n\\n it constantly , and unfairly gets compared to \" braveheart \" . xxmaj these are two entirely different films , probably only similar in the fact that they are both about xxmaj scots in historical xxmaj scotland . xxmaj yet , this comparison frequently bothers me because it seemspos
" 1230 | ], 1231 | "text/plain": [ 1232 | "" 1233 | ] 1234 | }, 1235 | "metadata": { 1236 | "tags": [] 1237 | } 1238 | } 1239 | ] 1240 | }, 1241 | { 1242 | "cell_type": "code", 1243 | "metadata": { 1244 | "id": "m3zGfwJ119qn" 1245 | }, 1246 | "source": [ 1247 | "#@ CREATING MODEL FOR TEXT CLASSIFICATION: \n", 1248 | "learn = text_classifier_learner(dls_clas, AWD_LSTM, drop_mult=0.5, \n", 1249 | " metrics=accuracy).to_fp16() # Initializing Text Classifier Learner. \n", 1250 | "learn = learn.load_encoder(\"/content/gdrive/MyDrive/FineTuned\") # Loading the Encoder. " 1251 | ], 1252 | "execution_count": 30, 1253 | "outputs": [] 1254 | }, 1255 | { 1256 | "cell_type": "markdown", 1257 | "metadata": { 1258 | "id": "Fj-j8-CZ4yK0" 1259 | }, 1260 | "source": [ 1261 | "**FINETUNING THE CLASSIFIER:**\n", 1262 | "- I will train the **Classifier** with discriminative learning rates and gradaul unfreezing. In computer vision unfreezing the model at once is common approach but for **NLP Classifier** unfreezing a few layers at a time will make a real difference. " 1263 | ] 1264 | }, 1265 | { 1266 | "cell_type": "code", 1267 | "metadata": { 1268 | "colab": { 1269 | "base_uri": "https://localhost:8080/", 1270 | "height": 80 1271 | }, 1272 | "id": "I998L26w2riy", 1273 | "outputId": "a625671a-e120-433f-9b98-645b4ce01644" 1274 | }, 1275 | "source": [ 1276 | "#@ TRAINING THE CLASSIFIERS: \n", 1277 | "learn.fit_one_cycle(1, 2e-2) # Freezing. " 1278 | ], 1279 | "execution_count": 31, 1280 | "outputs": [ 1281 | { 1282 | "output_type": "display_data", 1283 | "data": { 1284 | "text/html": [ 1285 | "\n", 1286 | " \n", 1287 | " \n", 1288 | " \n", 1289 | " \n", 1290 | " \n", 1291 | " \n", 1292 | " \n", 1293 | " \n", 1294 | " \n", 1295 | " \n", 1296 | " \n", 1297 | " \n", 1298 | " \n", 1299 | " \n", 1300 | " \n", 1301 | " \n", 1302 | " \n", 1303 | " \n", 1304 | "
epochtrain_lossvalid_lossaccuracytime
00.2382000.1802420.93256001:09
" 1305 | ], 1306 | "text/plain": [ 1307 | "" 1308 | ] 1309 | }, 1310 | "metadata": { 1311 | "tags": [] 1312 | } 1313 | } 1314 | ] 1315 | }, 1316 | { 1317 | "cell_type": "code", 1318 | "metadata": { 1319 | "colab": { 1320 | "base_uri": "https://localhost:8080/", 1321 | "height": 80 1322 | }, 1323 | "id": "eYa6uUIA2tyR", 1324 | "outputId": "ee40e601-2b10-4787-8c3d-ed37812c4175" 1325 | }, 1326 | "source": [ 1327 | "#@ TRAINING THE CLASSIFIERS: UNFREEZING LAYERS: \n", 1328 | "learn.freeze_to(-2) # Unfreezing. \n", 1329 | "learn.fit_one_cycle(1, slice(1e-2/(2.6**4), 1e-2)) # Training the Classifier. " 1330 | ], 1331 | "execution_count": 32, 1332 | "outputs": [ 1333 | { 1334 | "output_type": "display_data", 1335 | "data": { 1336 | "text/html": [ 1337 | "\n", 1338 | " \n", 1339 | " \n", 1340 | " \n", 1341 | " \n", 1342 | " \n", 1343 | " \n", 1344 | " \n", 1345 | " \n", 1346 | " \n", 1347 | " \n", 1348 | " \n", 1349 | " \n", 1350 | " \n", 1351 | " \n", 1352 | " \n", 1353 | " \n", 1354 | " \n", 1355 | " \n", 1356 | "
epochtrain_lossvalid_lossaccuracytime
00.2196550.1631690.93788001:14
" 1357 | ], 1358 | "text/plain": [ 1359 | "" 1360 | ] 1361 | }, 1362 | "metadata": { 1363 | "tags": [] 1364 | } 1365 | } 1366 | ] 1367 | }, 1368 | { 1369 | "cell_type": "code", 1370 | "metadata": { 1371 | "colab": { 1372 | "base_uri": "https://localhost:8080/", 1373 | "height": 80 1374 | }, 1375 | "id": "IDYKJTwf7X5B", 1376 | "outputId": "c8e5159c-4663-4ed7-d752-49fce5473d07" 1377 | }, 1378 | "source": [ 1379 | "#@ TRAINING THE CLASSIFIERS: UNFREEZING LAYERS: \n", 1380 | "learn.freeze_to(-3) # Unfreezing. \n", 1381 | "learn.fit_one_cycle(1, slice(5e-3/(2.6**4), 5e-3)) # Training the Classifier. " 1382 | ], 1383 | "execution_count": 33, 1384 | "outputs": [ 1385 | { 1386 | "output_type": "display_data", 1387 | "data": { 1388 | "text/html": [ 1389 | "\n", 1390 | " \n", 1391 | " \n", 1392 | " \n", 1393 | " \n", 1394 | " \n", 1395 | " \n", 1396 | " \n", 1397 | " \n", 1398 | " \n", 1399 | " \n", 1400 | " \n", 1401 | " \n", 1402 | " \n", 1403 | " \n", 1404 | " \n", 1405 | " \n", 1406 | " \n", 1407 | " \n", 1408 | "
epochtrain_lossvalid_lossaccuracytime
00.1878170.1477820.94460001:37
" 1409 | ], 1410 | "text/plain": [ 1411 | "" 1412 | ] 1413 | }, 1414 | "metadata": { 1415 | "tags": [] 1416 | } 1417 | } 1418 | ] 1419 | }, 1420 | { 1421 | "cell_type": "code", 1422 | "metadata": { 1423 | "colab": { 1424 | "base_uri": "https://localhost:8080/", 1425 | "height": 111 1426 | }, 1427 | "id": "yN2WgnOR7r1P", 1428 | "outputId": "e427cd5b-82e8-4f90-ac4b-55fecc19a505" 1429 | }, 1430 | "source": [ 1431 | "#@ TRAINING THE CLASSIFIERS: UNFREEZING LAYERS: \n", 1432 | "learn.unfreeze() # Unfreezing. \n", 1433 | "learn.fit_one_cycle(2, slice(1e-3/(2.6**4), 1e-3)) # Training the Classifier. " 1434 | ], 1435 | "execution_count": 34, 1436 | "outputs": [ 1437 | { 1438 | "output_type": "display_data", 1439 | "data": { 1440 | "text/html": [ 1441 | "\n", 1442 | " \n", 1443 | " \n", 1444 | " \n", 1445 | " \n", 1446 | " \n", 1447 | " \n", 1448 | " \n", 1449 | " \n", 1450 | " \n", 1451 | " \n", 1452 | " \n", 1453 | " \n", 1454 | " \n", 1455 | " \n", 1456 | " \n", 1457 | " \n", 1458 | " \n", 1459 | " \n", 1460 | " \n", 1461 | " \n", 1462 | " \n", 1463 | " \n", 1464 | " \n", 1465 | " \n", 1466 | " \n", 1467 | "
epochtrain_lossvalid_lossaccuracytime
00.1588340.1491360.94448001:55
10.1435650.1486900.94772001:55
" 1468 | ], 1469 | "text/plain": [ 1470 | "" 1471 | ] 1472 | }, 1473 | "metadata": { 1474 | "tags": [] 1475 | } 1476 | } 1477 | ] 1478 | } 1479 | ] 1480 | } -------------------------------------------------------------------------------- /09. Natural Language Processing/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Natural Language Processing** 2 | 3 | The [**NLP**](https://github.com/ThinamXx/Fastai/blob/main/9.%20Natural%20Language%20Processing/NLP.ipynb) notebook contains all the dependencies required build **Language Model** that can generate texts and a **Classifier Model** that determines whether a review is positive or negative. It presents the state of art **Classifier Model** which is build using a pretrained language model and fine tuned it to the corpus of task. Then the **Encoder** model is used for classification. 4 | 5 | **Note:** 6 | - 📑[**Natural Language Processing**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/9.%20Natural%20Language%20Processing/NLP.ipynb) 7 | 8 | **Loading the Data** 9 | - I will get the IMDB Dataset here. I have presented the implementation of Loading the Data and Word Tokenization using Fastai and PyTorch here in the snapshot. **Token** is a element of a list created by the Tokenization process which could be a word, a part of a word or subword or a single character. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20232.PNG) 12 | 13 | **Word Tokenization** 14 | - Word Tokenization splits a sentence on spaces as well as applying language specific rules to try to separate parts of meaning even when there are no spaces. Generally punctuation marks are also split into separate tokens. Token is a element of a list created by the Tokenization process which could be a word, a part of a word or subword or a single character. 15 | 16 | **Subword Tokenization** 17 | - Word Tokenization relies on an assumption that spaces provide a useful separation of components of meaning in a sentence which is not always appropriate. Languages such as Chinese and Japanese don't use spaces and in such cases Subword Tokenization generally plays the best role. Subword Tokenization splits words into smaller parts based on the most commonly occurring sub strings. I have presented the implementation of Subword Tokenization and Numericalization using Fastai and PyTorch here in the snapshot. 18 | 19 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20233.PNG) 20 | 21 | **Numericalization** 22 | - Numericalization is the process of mapping tokens to integers. It involves making a list of all possible levels of that categorical variable or the vocab and replacing each level with its index in the vocab. I have presented the implementation of Creating Data Loaders and Data Block for Language Model using Fastai and PyTorch here in the snapshot. 23 | 24 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20234.PNG) 25 | 26 | **Training the Classifier Model** 27 | - I will use Embeddings to convert the integer word indices into activations that can be used for the neural networks. These embeddings are feed into Recurrent Neural Network using and architecture called AWD-LSTM. Encoder is defined as the model which doesn't contain task specific final layers. The term Encoder means much the same thing as body when applied to vision CNN but Encoder tends to be more used for NLP and generative models. I will train the Classifier with discriminative learning rates and gradaul unfreezing. In computer vision unfreezing the model at once is common approach but for NLP Classifier unfreezing a few layers at a time will make a real difference. I have presented the implementation of Training Text Classifier Model using Discriminative Learning Rates and Gradual Unfreezing using Fastai and PyTorch here in the snapshot. 28 | 29 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20235a.PNG) 30 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20235b.PNG) 31 | -------------------------------------------------------------------------------- /09. Natural Language Processing/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook 2 | fastai 3 | fastai.text 4 | ipython 5 | -------------------------------------------------------------------------------- /10. Data Munging/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Data Munging** 2 | 3 | The [**DataMunging**](https://github.com/ThinamXx/Fastai/blob/main/10.%20Data%20Munging/DataMunging.ipynb) notebook contains all the dependencies required to implement mid level API of **Fast.ai** in **Natural Language Processing** and **Computer Vision** which provides greater flexibility to apply transformations on data items. 4 | 5 | **Note:** 6 | - 📑[**Data Munging**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/10.%20Data%20Munging/DataMunging.ipynb) 7 | 8 | **IMDB Dataset:** 9 | - I have presented the implementation of Creating Data Loaders, Tokenization and Numericalization using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20236.PNG) 12 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20237.PNG) 13 | 14 | **Decorator:** 15 | - Python has a special syntax for passing a function to another function or something that behaves like a function which is also known as callable in Python is called a Decorator. A Decorator is used by prepending a callable with @ and placing it before the function definition. 16 | 17 | **Datasets:** 18 | - Datasets will apply two or more pipelines in parallel to the same raw object and build a tuple with the result. It will automatically do the setup and index into a Datasets. I have presented the implementation of Datasets Class, Transformed Collections and Data Loaders using Fastai and PyTorch here in the snapshot. 19 | 20 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20238.PNG) 21 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20239.PNG) 22 | -------------------------------------------------------------------------------- /10. Data Munging/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook 2 | fastai 3 | fastai.text 4 | fastai.vision 5 | ipython 6 | -------------------------------------------------------------------------------- /11. Language Model/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Language Model from Scratch** 2 | 3 | The [**LanguageModel**](https://github.com/ThinamXx/Fastai/blob/main/11.%20Language%20Model/LanguageModel.ipynb) notebook contains all the dependencies that is inside **AWD-LSTM** architecture for **Text Classification**. It presents the implementation of **Language Model** using simple **Linear Model**, **Recurrent Neural Network**, **Long Short Term Memory**, **Dropout Regularization** and **Activation Regularization**. 4 | 5 | **Note:** 6 | - 📑[**Language Model**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/11.%20Language%20Model/LanguageModel.ipynb) 7 | 8 | **Loading the Data** 9 | - I will use **Human Numbers** dataset here. It contains the first 10000 numbers written out in English. I will tokenize the dataset by splitting on spaces. Then I will create a list of unique tokens called vocab for Numericalization. Then I will convert the tokens into numbers by looking up in the index of each in the vocab. I have presented the implementation of Preparing Sequence of Tensors for Language Model using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20241.PNG) 12 | 13 | **Language Model** 14 | - I will create neural network architecture that takes three words as input and returns the predictions of the probability of each possible next word in the vocab. I will use three standard linear layers. The first linear layer will use only the first words embedding as activations. The second layer will use the second words embedding plus the first layers output activations and the third layer will use the third words embedding plus the second layers output activations. The key effect is that every word is interpreted in the information context of any words preceding it. Each of these three layers will use the same weight matrix. I have presented the implementation of Creating Data Loaders, Language Model from Scratch and Training using Fastai and PyTorch here in the snapshot. 15 | 16 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20242.PNG) 17 | 18 | **Recurrent Neural Networks** 19 | - I will simplify the neural networks model or module defined above by replacing the duplicated code that calls the layers with a for loop. The module can work equally to token sequences of different lengths. Recurrent Neural Network is a network which is a refactoring of a multilayer neural network using for loop. I will define a stateful Recurrent Neural Network as it remembers its activations between different calls to forward which represents its use for different samples in the batch. I have presented the implementation of Recurrent Neural Networks and Language Model using Fastai and PyTorch here in the snapshot. 20 | 21 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20243.PNG) 22 | 23 | **Backpropagation Through Time** 24 | - Backpropagation through Time is a process of treating a neural network with effectively one layer per time step as one big model and calculating gradients on it in the usual way. The BPTT technique is used to avoid running out of memory and time which detaches the history of computation steps in the hidden state every few time steps. 25 | 26 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20244.PNG) 27 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20245b.PNG) 28 | 29 | **Long Short Term Memory** 30 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/LSTM.png) 31 | - The green circles are elementwise operations and the four orange boxes represent four neural network layers which are also called gates with the activations. The first gate from left is called the forget gate. The second gate is called the input gate and the third gate is sometimes called the cell gate. The last gate is the output gate. Tanh is just a Sigmoid function rescaled to the range of -1 and 1. 32 | 33 | **Dropout Regularization** 34 | - Dropout is a regularization technique which randomly changes some activations to zero at a training time. 35 | 36 | **Activation Regularization** 37 | - Activation Regularization is a process of adding the small penalty to the final activations produced by the LSTM to make it as small as possible. It is a regularization method very similar to weight decay. 38 | 39 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20248.PNG) 40 | -------------------------------------------------------------------------------- /11. Language Model/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook 2 | fastai 3 | fastai.text 4 | ipython 5 | -------------------------------------------------------------------------------- /12. Convolutional Neural Networks/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Convolutional Neural Networks** 2 | 3 | The [**CNN**](https://github.com/ThinamXx/Fastai/blob/main/12.%20Convolutional%20Neural%20Networks/CNN.ipynb) notebook contains all the dependencies required to understand **Convolutional Neural Networks**. **Convolutions** are just a type of matrix multiplication with two constraints on the weight matrix: some elements are always zero and some elements are tied or forced to always have the same value. 4 | 5 | **Note:** 6 | - 📑[**Convolutional Neural Networks**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/12.%20Convolutional%20Neural%20Networks/CNN.ipynb) 7 | 8 | **Feature Engineering** 9 | - Feature Engineering is the process of creating a new transformations of the input data in order to make it easier to model. I have presented the implementation of Feature Engineering and Mapping a Convolutional Kernel using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20249.PNG) 12 | 13 | **Channels and Features** 14 | - Channels and Features are largely used interchangeably and refer to the size of the second axis of a weight matrix which is the number of activations per grid cell after a convolution. Channels refer to the input data i.e colors or activations inside the network. Using a stride 2 convolution often increases the number of Features at the same time because the number of activations in the activation map decrease by the factor of 4. A channel is a single basic color in an image. For a regular full color images, there are three channels : red, green and blue. Kernels passed to convolutions need to be rank 4 tensors. 15 | 16 | **Strides and Padding** 17 | - If we add a kernel of size ks by ks with ks an odd number then the necessary padding on each side to keep the same shape is ks//2. The general formula for each dimension is (n + 2\*padding - kernel size) // stride + 1. 18 | 19 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20251.PNG) 20 | 21 | **Receptive Fields** 22 | - The Receptive Field is the area of an image that is involved in the calculation of a layer. In the deeper layers of network, we have semantically rich features, corresponding to larger Receptive Fields. 23 | 24 | **1 Cycle Training** 25 | - 1 Cycle Training is a combination of warmup and annealing. Warmup is the one where learning rate grows from the minimum value to the maximum value and Annealing is the one where it decreases back to the minimum value. 26 | 27 | **Convolutional Neural Networks** 28 | - Convolutions are just a type of matrix multiplication with two constraints on the weight matrix: some elements are always zero and some elements are tied or forced to always have the same value. Batch Normalization adds some extra randomness to the training process. Larger batches have gradients that are more accurate since they are calculated from more data. But larger batch size means fewer batches per epoch which means fewer opportunities for the model to update weights. 29 | 30 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20252.PNG) 31 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20253.PNG) 32 | -------------------------------------------------------------------------------- /12. Convolutional Neural Networks/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook 2 | fastai 3 | fastai.vision 4 | ipython 5 | fastai.callback 6 | -------------------------------------------------------------------------------- /13. ResNets/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Residual Networks** 2 | 3 | - The [**ResNets**](https://github.com/ThinamXx/Fastai/blob/main/13.%20ResNets/ResNets.ipynb) notebook contains all the dependencies required to understand the implementation of skip connections which allow deeper models to be trained. **ResNet** is the pretrained model when using **Transfer Learning**. 4 | 5 | **Note:** 6 | - 📑[**Residual Networks**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/13.%20ResNets/ResNets.ipynb) 7 | 8 | **Fully Convolutional Neural Networks** 9 | - The idea in Fully Convolutional Networks is to take the average of activations across a convolutional grid. A Fully Convolutional Networks has a number of convolutional layers, some of which will be stride 2 convolutions at the end of which is an adaptive average pooling layer, a flatten layer to remove the unit axis and finally a linear layer. Larger batches have gradients that are more accurate since they are calculated from more data. But larger batch size means fewer batches per epoch which means fewer opportunities for the model to update weights. I have presented the implementation of Preparing Data and Fully Convolutional Networks using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20254.PNG) 12 | 13 | **Identity Mapping** 14 | - Identity Mapping is the process of returning the input without changing it at all which is performed by the identity function. I have presented the implementation of ResNet Architecture and Skip Connections using Fastai and PyTorch here in the snapshot. 15 | 16 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20255.PNG) 17 | 18 | **Bottleneck Layers** 19 | - Bottleneck Layers use three convolutions: Two 1X1 at the begining and the end and one 3X3. The 1X1 convolutions are much faster which facilitates to use higher number of filters in and out. The 1X1 convolutions diminish and then restore the number of channels so called Bottleneck. The overall impact is to facilitate the use of more filters in the same amount of time. I have presented the implementation of Training Deeper Networks and Bottleneck Layers using Fastai and PyTorch here in the snapshot. 20 | 21 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20256.PNG) 22 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20257.PNG) 23 | -------------------------------------------------------------------------------- /13. ResNets/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai.vision.all 2 | fastbook 3 | fastai.callback.fp16 4 | -------------------------------------------------------------------------------- /14. Architecture Details/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Architecture Details** 2 | 3 | - The [**Architecture Details**](https://github.com/ThinamXx/Fastai/blob/main/14.%20Architecture%20Details/Architectures.ipynb) notebook contains all the dependencies required to create a complete state of art computer vision models. It presents some aspects of natural language processing as well. 4 | 5 | **Note:** 6 | - 📑[**Architecture Details**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/14.%20Architecture%20Details/Architectures.ipynb) 7 | 8 | **Body and Head of Network** 9 | - The head of a neural net is the part that is specialized for a particular task. For a Convolutional Neural Network, it's generally the part after the adaptive average pooling layer. The body of the neural net contains everything except the head of a network and also includes the stem of the network. I will use pretrained model to create the encoder. The last layer of the encoder has 512 features so the head of the network will receive 512X4 features. Here 4 is used because there are 2 images and 2 for concatenation and pooling. 10 | 11 | **Splitter Function** 12 | - A splitter is a function that tells the fastai library how to split the model into parameter groups which are used to train only the head of the model during transfer learning. The params is just a function that returns all parameters of a given module. I have presented the implementation of Siamese Network Model, Loss Function and Splitter Function using Fastai and PyTorch here in the snapshot. 13 | 14 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20258.PNG) 15 | -------------------------------------------------------------------------------- /14. Architecture Details/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook 2 | fastai.vision 3 | fastai.callback.fp16 4 | -------------------------------------------------------------------------------- /15. Training Process/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Training Process** 2 | 3 | The [**Training**](https://github.com/ThinamXx/Fastai/blob/main/15.%20Training%20Process/Training.ipynb) notebook contains all the dependencies required to create a training loop and explored variants of Stochastic Gradient Descent. 4 | 5 | **Note:** 6 | - 📑[**Training Process**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/15.%20Training%20Process/Training.ipynb) 7 | 8 | **Preparing the Dataset** 9 | - I have presented the implementation of Preparing Dataset and Baseline Model using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20259.PNG) 12 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20260.PNG) 13 | 14 | **Adam Optimizer** 15 | - Adam mixes the idea of SGD with momentum and RMSProp together where it uses the moving average of the gradients as a direction and divides by the square root of the moving average of the gradients squared to give an adaptive learning rate to each parameter. It takes the unbiased moving average. I have presented the implementation of RMS Prop and Adam Optimizer using Fastai and PyTorch here in the snapshot. 16 | 17 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20262.PNG) 18 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20263.PNG) 19 | -------------------------------------------------------------------------------- /15. Training Process/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai.vision.all 2 | fastbook -------------------------------------------------------------------------------- /16. Neural Network Foundations/NeuralFoundations.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "NeuralFoundations.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [] 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "language_info": { 15 | "name": "python" 16 | }, 17 | "accelerator": "GPU" 18 | }, 19 | "cells": [ 20 | { 21 | "cell_type": "markdown", 22 | "metadata": { 23 | "id": "HlHj7lUPa187" 24 | }, 25 | "source": [ 26 | "**INITIALIZATION**\n", 27 | "- I use these three lines of code on top of my each notebooks because it will help to prevent any problems while reloading the same project. And the third line of code helps to make visualization within the notebook." 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "metadata": { 33 | "id": "taFeJgiOafpU" 34 | }, 35 | "source": [ 36 | "#@ INITIALIZATION: \n", 37 | "%reload_ext autoreload\n", 38 | "%autoreload 2\n", 39 | "%matplotlib inline" 40 | ], 41 | "execution_count": 1, 42 | "outputs": [] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": { 47 | "id": "DxH0FCEga_zV" 48 | }, 49 | "source": [ 50 | "**LIBRARIES AND DEPENDENCIES**\n", 51 | "- I have downloaded all the libraries and dependencies required for the project in one particular cell." 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "metadata": { 57 | "id": "t4v7GAR2a-IJ" 58 | }, 59 | "source": [ 60 | "#@ INSTALLING DEPENDENCIES: UNCOMMENT BELOW: \n", 61 | "# !pip install -Uqq fastbook\n", 62 | "# import fastbook\n", 63 | "# fastbook.setup_book()" 64 | ], 65 | "execution_count": 3, 66 | "outputs": [] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "metadata": { 71 | "id": "xk9RqKhcbJor" 72 | }, 73 | "source": [ 74 | "#@ DOWNLOADING LIBRARIES AND DEPENDENCIES: \n", 75 | "from fastbook import * # Getting all the Libraries. \n", 76 | "from fastai.callback.fp16 import *\n", 77 | "from torch.autograd import Function" 78 | ], 79 | "execution_count": 32, 80 | "outputs": [] 81 | }, 82 | { 83 | "cell_type": "markdown", 84 | "metadata": { 85 | "id": "zi-60nNpbv1b" 86 | }, 87 | "source": [ 88 | "**MODELING NEURON**" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "metadata": { 94 | "id": "4urwjTeBbeHB" 95 | }, 96 | "source": [ 97 | "#@ MODELING NEURON FROM SCRATCH: UNCOMMENT BELOW: \n", 98 | "# output = sum([x*w for x,w in zip(inputs, weights)]) + bias # Adding Weighted Inputs and Bias. \n", 99 | "# def relu(x): return x if x >= 0 else 0 # Defining RELU Activation Function. \n", 100 | "# y[i, j] = sum([a*b for a,b in zip(x[i,:], w[j,:])]) + b[j] # Initializing Matrix Multiplication. " 101 | ], 102 | "execution_count": null, 103 | "outputs": [] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "metadata": { 108 | "colab": { 109 | "base_uri": "https://localhost:8080/" 110 | }, 111 | "id": "YYCUMtyJgXza", 112 | "outputId": "c27f0024-89af-4113-9f65-66b06f250a79" 113 | }, 114 | "source": [ 115 | "#@ MATRIX MULTIPLICATION FROM SCRATCH: \n", 116 | "def matmul(a, b): # Defining Matrix Multiplication Function. \n", 117 | " ar, ac = a.shape # Inspecting Shape. \n", 118 | " br, bc = b.shape # Inspecting Shape. \n", 119 | " assert ac == br # Asserting Rows and Columns. \n", 120 | " c = torch.zeros(ar, bc) # Initializing Tensor. \n", 121 | " for i in range(ar): # Getting Row Indices. \n", 122 | " for j in range(bc): # Getting Column Indices. \n", 123 | " for k in range(ac): # Getting Inner Sum. \n", 124 | " c[i,j] += a[i,k] * b[k,j] # Getting Matrix Multiplication. \n", 125 | " return c\n", 126 | "\n", 127 | "#@ IMPLEMENTATION OF MATRIX MULTIPLICATION: \n", 128 | "m1 = torch.randn(5, 28*28) # Initializing Matrix. \n", 129 | "m2 = torch.randn(784, 10) # Initializing Matrix. \n", 130 | "%time t1 = matmul(m1, m2) # Implementation of Matrix Multiplication. \n", 131 | "%timeit -n 20 t2 = m1@m2 # Implementation of Matrix Multiplication. " 132 | ], 133 | "execution_count": 5, 134 | "outputs": [ 135 | { 136 | "output_type": "stream", 137 | "name": "stdout", 138 | "text": [ 139 | "CPU times: user 659 ms, sys: 0 ns, total: 659 ms\n", 140 | "Wall time: 731 ms\n", 141 | "The slowest run took 142.02 times longer than the fastest. This could mean that an intermediate result is being cached.\n", 142 | "20 loops, best of 5: 7.82 µs per loop\n" 143 | ] 144 | } 145 | ] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "metadata": { 150 | "colab": { 151 | "base_uri": "https://localhost:8080/" 152 | }, 153 | "id": "CY-ny3EzjFXX", 154 | "outputId": "4e390293-78b6-4077-c2eb-b3d307060f19" 155 | }, 156 | "source": [ 157 | "#@ ELEMENTWISE ARITHMETIC: \n", 158 | "a = tensor([10., 6, -4]) # Initializing a Tensor. \n", 159 | "b = tensor([2., 8, 7]) # Initializing a Tensor. \n", 160 | "a + b # Initializing Elementwise Addition. \n", 161 | "(aij\",a,b) # Implementation of Einstein Summation. \n", 339 | "%timeit -n 20 t5 = m1@m2 # Implementation of Matrix Multiplication. " 340 | ], 341 | "execution_count": 11, 342 | "outputs": [ 343 | { 344 | "output_type": "stream", 345 | "name": "stdout", 346 | "text": [ 347 | "20 loops, best of 5: 7.12 µs per loop\n" 348 | ] 349 | } 350 | ] 351 | }, 352 | { 353 | "cell_type": "markdown", 354 | "metadata": { 355 | "id": "fKRcwfi0D_72" 356 | }, 357 | "source": [ 358 | "**FORWARD AND BACKWARD PASSES**\n", 359 | "- Computing all the gradients of a given loss with respect to its parameters is known as **Backward Pass**. Similarly computing the output of the model on a given input based on the matrix products is known as **Forward Pass**. " 360 | ] 361 | }, 362 | { 363 | "cell_type": "code", 364 | "metadata": { 365 | "colab": { 366 | "base_uri": "https://localhost:8080/" 367 | }, 368 | "id": "tdwHrYUQDqyz", 369 | "outputId": "724f09dc-8096-446b-b61e-6eb850226b9f" 370 | }, 371 | "source": [ 372 | "#@ DEFINING AND INITIALIZING LAYER: \n", 373 | "def lin(x, w, b): return x @ w + b # Defining Linear Layer. \n", 374 | "x = torch.randn(200, 100) # Initializing Random Input Tensors. \n", 375 | "y = torch.randn(200) # Initializing Random Output Tensors. \n", 376 | "w1 = torch.randn(100, 50) # Initializing Random Weights. \n", 377 | "b1 = torch.zeros(50) # Initializing Random Bias. \n", 378 | "w2 = torch.randn(50, 1) # Initializing Random Weights. \n", 379 | "b2 = torch.zeros(1) # Initializing Random Bias. \n", 380 | "\n", 381 | "#@ IMPLEMENTATION OF LINEAR FUNCTION: \n", 382 | "l1 = lin(x, w1, b1) # Implementation of Function. \n", 383 | "l1.shape # Inspecting the Shape. " 384 | ], 385 | "execution_count": 12, 386 | "outputs": [ 387 | { 388 | "output_type": "execute_result", 389 | "data": { 390 | "text/plain": [ 391 | "torch.Size([200, 50])" 392 | ] 393 | }, 394 | "metadata": {}, 395 | "execution_count": 12 396 | } 397 | ] 398 | }, 399 | { 400 | "cell_type": "code", 401 | "metadata": { 402 | "colab": { 403 | "base_uri": "https://localhost:8080/" 404 | }, 405 | "id": "qFJC1NyH_IFe", 406 | "outputId": "8e5b52a2-c60a-4fdc-8367-35d3181c2d5a" 407 | }, 408 | "source": [ 409 | "#@ INSPECTING MEAN AND STANDARD DEVIATION: \n", 410 | "l1.mean(), l1.std()" 411 | ], 412 | "execution_count": 13, 413 | "outputs": [ 414 | { 415 | "output_type": "execute_result", 416 | "data": { 417 | "text/plain": [ 418 | "(tensor(-0.0569), tensor(10.1162))" 419 | ] 420 | }, 421 | "metadata": {}, 422 | "execution_count": 13 423 | } 424 | ] 425 | }, 426 | { 427 | "cell_type": "code", 428 | "metadata": { 429 | "colab": { 430 | "base_uri": "https://localhost:8080/" 431 | }, 432 | "id": "myV0eK0r_Vma", 433 | "outputId": "01845b7e-a237-490a-e6ed-fca8615a8aeb" 434 | }, 435 | "source": [ 436 | "#@ UNDERSTANDING MATRIX MULTIPLICATIONS: \n", 437 | "x = torch.randn(200, 100) # Initializing Random Numbers. \n", 438 | "for i in range(50):\n", 439 | " x = x @ torch.randn(100, 100) # Initializing Matrix Multiplications. \n", 440 | "x[0:5, 0:5]" 441 | ], 442 | "execution_count": 14, 443 | "outputs": [ 444 | { 445 | "output_type": "execute_result", 446 | "data": { 447 | "text/plain": [ 448 | "tensor([[nan, nan, nan, nan, nan],\n", 449 | " [nan, nan, nan, nan, nan],\n", 450 | " [nan, nan, nan, nan, nan],\n", 451 | " [nan, nan, nan, nan, nan],\n", 452 | " [nan, nan, nan, nan, nan]])" 453 | ] 454 | }, 455 | "metadata": {}, 456 | "execution_count": 14 457 | } 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "metadata": { 463 | "colab": { 464 | "base_uri": "https://localhost:8080/" 465 | }, 466 | "id": "wNrC5a1rAU5n", 467 | "outputId": "dfcd3b4d-a4a2-48b0-9ea3-b06a756be00d" 468 | }, 469 | "source": [ 470 | "#@ UNDERSTANDING MATRIX MULTIPLICATIONS: \n", 471 | "x = torch.randn(200, 100) # Initializing Random Numbers. \n", 472 | "for i in range(50):\n", 473 | " x = x @ (torch.randn(100, 100) * 0.01) # Initializing Matrix Multiplications. \n", 474 | "x[0:5, 0:5] # Inspection. " 475 | ], 476 | "execution_count": 15, 477 | "outputs": [ 478 | { 479 | "output_type": "execute_result", 480 | "data": { 481 | "text/plain": [ 482 | "tensor([[0., 0., 0., 0., 0.],\n", 483 | " [0., 0., 0., 0., 0.],\n", 484 | " [0., 0., 0., 0., 0.],\n", 485 | " [0., 0., 0., 0., 0.],\n", 486 | " [0., 0., 0., 0., 0.]])" 487 | ] 488 | }, 489 | "metadata": {}, 490 | "execution_count": 15 491 | } 492 | ] 493 | }, 494 | { 495 | "cell_type": "code", 496 | "metadata": { 497 | "colab": { 498 | "base_uri": "https://localhost:8080/" 499 | }, 500 | "id": "zPsXfYvLAw7f", 501 | "outputId": "481f3091-cef8-4dd7-a18e-ed722be26ffd" 502 | }, 503 | "source": [ 504 | "#@ UNDERSTANDING MATRIX MULTIPLICATIONS: \n", 505 | "x = torch.randn(200, 100) # Initializing Random Numbers. \n", 506 | "for i in range(50):\n", 507 | " x = x @ (torch.randn(100, 100) * 0.1) # Initializing Matrix Multiplications. \n", 508 | "x[0:5, 0:5] # Inspection. " 509 | ], 510 | "execution_count": 16, 511 | "outputs": [ 512 | { 513 | "output_type": "execute_result", 514 | "data": { 515 | "text/plain": [ 516 | "tensor([[ 2.5783e-02, 5.1167e-01, 1.0287e-03, 1.4076e-01, -1.0012e+00],\n", 517 | " [ 5.4434e-01, 1.2963e+00, 1.0347e+00, 1.5280e+00, -9.2810e-01],\n", 518 | " [-6.0311e-01, -1.0227e+00, -3.7155e-01, -6.7006e-01, 5.1546e-01],\n", 519 | " [-1.4638e-01, -7.7870e-01, -4.5685e-01, -9.4609e-01, 4.8461e-01],\n", 520 | " [-3.0557e-01, -5.9048e-02, -8.6489e-02, -4.2007e-01, -3.3071e-01]])" 521 | ] 522 | }, 523 | "metadata": {}, 524 | "execution_count": 16 525 | } 526 | ] 527 | }, 528 | { 529 | "cell_type": "code", 530 | "metadata": { 531 | "colab": { 532 | "base_uri": "https://localhost:8080/" 533 | }, 534 | "id": "1YsTAz_MBSpX", 535 | "outputId": "dea6b7a2-10bb-4e36-b352-6164b844808a" 536 | }, 537 | "source": [ 538 | "#@ IMPLEMENTATION OF XAVIER INITIALIZATION: \n", 539 | "x = torch.randn(200, 100) # Initializing Random Input Tensors. \n", 540 | "y = torch.randn(200) # Initializing Random Output Tensors. \n", 541 | "w1 = torch.randn(100, 50) / math.sqrt(100) # Initializing Random Weights. \n", 542 | "b1 = torch.zeros(50) # Initializing Random Bias. \n", 543 | "w2 = torch.randn(50, 1) / math.sqrt(50) # Initializing Random Weights. \n", 544 | "b2 = torch.zeros(1) # Initializing Random Bias. \n", 545 | "\n", 546 | "#@ IMPLEMENTATION OF LINEAR FUNCTION: \n", 547 | "l1 = lin(x, w1, b1) # Implementation of Function. \n", 548 | "l1.mean(), l1.std() # Inspection. " 549 | ], 550 | "execution_count": 17, 551 | "outputs": [ 552 | { 553 | "output_type": "execute_result", 554 | "data": { 555 | "text/plain": [ 556 | "(tensor(0.0013), tensor(1.0043))" 557 | ] 558 | }, 559 | "metadata": {}, 560 | "execution_count": 17 561 | } 562 | ] 563 | }, 564 | { 565 | "cell_type": "code", 566 | "metadata": { 567 | "colab": { 568 | "base_uri": "https://localhost:8080/" 569 | }, 570 | "id": "R3kcpDdgDVAh", 571 | "outputId": "3ff290ce-944f-40bc-b4de-62e1d828c975" 572 | }, 573 | "source": [ 574 | "#@ IMPLEMENTATION OF RELU ACTIVATION FUNCTION: \n", 575 | "def relu(x): # Defining Relu Function. \n", 576 | " return x.clamp_min(0.) # Replacing Negatives with Zeros. \n", 577 | "l2 = relu(l1) # Implementation of RELU. \n", 578 | "l2.mean(), l2.std() # Inspection. " 579 | ], 580 | "execution_count": 18, 581 | "outputs": [ 582 | { 583 | "output_type": "execute_result", 584 | "data": { 585 | "text/plain": [ 586 | "(tensor(0.3996), tensor(0.5879))" 587 | ] 588 | }, 589 | "metadata": {}, 590 | "execution_count": 18 591 | } 592 | ] 593 | }, 594 | { 595 | "cell_type": "code", 596 | "metadata": { 597 | "colab": { 598 | "base_uri": "https://localhost:8080/" 599 | }, 600 | "id": "0dS_SrN-Ew1B", 601 | "outputId": "0abfb9da-63e5-4e0d-bc5e-2b6e847a3f9a" 602 | }, 603 | "source": [ 604 | "#@ MATRIX MULTIPLICATIONS AND RELU: \n", 605 | "x = torch.randn(200, 100) # Initializing Random Numbers. \n", 606 | "for i in range(50):\n", 607 | " x = relu(x @ (torch.randn(100, 100) * 0.1)) # Initializing Matrix Multiplications. \n", 608 | "x[0:5, 0:5] # Inspection. " 609 | ], 610 | "execution_count": 19, 611 | "outputs": [ 612 | { 613 | "output_type": "execute_result", 614 | "data": { 615 | "text/plain": [ 616 | "tensor([[1.3917e-08, 1.9313e-09, 0.0000e+00, 0.0000e+00, 0.0000e+00],\n", 617 | " [1.1168e-08, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00],\n", 618 | " [2.0183e-08, 1.0267e-09, 0.0000e+00, 0.0000e+00, 0.0000e+00],\n", 619 | " [2.0421e-08, 9.1188e-10, 0.0000e+00, 0.0000e+00, 0.0000e+00],\n", 620 | " [1.1986e-08, 1.4588e-09, 0.0000e+00, 0.0000e+00, 0.0000e+00]])" 621 | ] 622 | }, 623 | "metadata": {}, 624 | "execution_count": 19 625 | } 626 | ] 627 | }, 628 | { 629 | "cell_type": "code", 630 | "metadata": { 631 | "colab": { 632 | "base_uri": "https://localhost:8080/" 633 | }, 634 | "id": "_5jC9JL9FHEM", 635 | "outputId": "8dc80f39-e990-4c86-9957-5dc8168c0708" 636 | }, 637 | "source": [ 638 | "#@ MATRIX MULTIPLICATIONS AND RELU: \n", 639 | "x = torch.randn(200, 100) # Initializing Random Numbers. \n", 640 | "for i in range(50):\n", 641 | " x = relu(x @ (torch.randn(100, 100) * \n", 642 | " math.sqrt(2/100))) # Initializing Matrix Multiplications. \n", 643 | "x[0:5, 0:5] # Inspection. " 644 | ], 645 | "execution_count": 20, 646 | "outputs": [ 647 | { 648 | "output_type": "execute_result", 649 | "data": { 650 | "text/plain": [ 651 | "tensor([[0.4409, 0.0000, 0.0000, 0.0000, 0.0000],\n", 652 | " [1.7408, 0.0000, 0.0000, 0.4863, 0.1915],\n", 653 | " [1.0620, 0.0000, 0.0000, 0.0000, 0.0000],\n", 654 | " [0.7964, 0.0000, 0.0000, 0.0000, 0.0000],\n", 655 | " [0.3552, 0.0000, 0.0000, 0.0000, 0.0000]])" 656 | ] 657 | }, 658 | "metadata": {}, 659 | "execution_count": 20 660 | } 661 | ] 662 | }, 663 | { 664 | "cell_type": "code", 665 | "metadata": { 666 | "colab": { 667 | "base_uri": "https://localhost:8080/" 668 | }, 669 | "id": "4bdP5IjeDF_y", 670 | "outputId": "fd1ea971-5bf9-4b4a-d222-6a9a8183848c" 671 | }, 672 | "source": [ 673 | "#@ IMPLEMENTATION OF KAIMING INITILIZATION: \n", 674 | "x = torch.randn(200, 100) # Initializing Random Input Tensors.\n", 675 | "y = torch.randn(200) # Initializing Random Output Tensors.\n", 676 | "w1 = torch.randn(100, 50) * math.sqrt(2/100) # Initializing Weight Tensors. \n", 677 | "b1 = torch.zeros(50) # Initializing Bias. \n", 678 | "w2 = torch.rand(50, 1) * math.sqrt(2/50) # Initializing Weight Tensors. \n", 679 | "b2 = torch.zeros(1) # Initializing Bias. \n", 680 | "\n", 681 | "#@ IMPLEMENTATION OF LINEAR LAYER AND RELU: \n", 682 | "l1 = lin(x, w1, b1) # Implementation of Linear Function. \n", 683 | "l2 = relu(l1) # Implementation of RELU. \n", 684 | "l2.mean(), l2.std() # Inspection. " 685 | ], 686 | "execution_count": 21, 687 | "outputs": [ 688 | { 689 | "output_type": "execute_result", 690 | "data": { 691 | "text/plain": [ 692 | "(tensor(0.5623), tensor(0.8257))" 693 | ] 694 | }, 695 | "metadata": {}, 696 | "execution_count": 21 697 | } 698 | ] 699 | }, 700 | { 701 | "cell_type": "code", 702 | "metadata": { 703 | "colab": { 704 | "base_uri": "https://localhost:8080/" 705 | }, 706 | "id": "zqsAoiwfFBbx", 707 | "outputId": "9b374677-767d-4025-aa22-7bf76cbd7970" 708 | }, 709 | "source": [ 710 | "#@ DEFINING MODEL: \n", 711 | "def model(x): # Defining Model Function. \n", 712 | " l1 = lin(x, w1, b1) # Implementation of Linear Function. \n", 713 | " l2 = relu(l1) # Implementation of RELU Function. \n", 714 | " l3 = lin(l2, w2, b2) # Implementation of Linear Function. \n", 715 | " return l3\n", 716 | "\n", 717 | "#@ IMPLEMENTATION OF MODEL: \n", 718 | "out = model(x) # Initializing the Model. \n", 719 | "out.shape # Inspection. " 720 | ], 721 | "execution_count": 22, 722 | "outputs": [ 723 | { 724 | "output_type": "execute_result", 725 | "data": { 726 | "text/plain": [ 727 | "torch.Size([200, 1])" 728 | ] 729 | }, 730 | "metadata": {}, 731 | "execution_count": 22 732 | } 733 | ] 734 | }, 735 | { 736 | "cell_type": "code", 737 | "metadata": { 738 | "id": "CkbtFm71GJ9C" 739 | }, 740 | "source": [ 741 | "#@ DEFINING LOSS FUNCTION: \n", 742 | "def mse(output, targ): # Defining Loss Function. \n", 743 | " return (output.squeeze(-1)-targ).pow(2).mean() # Initializing Mean Squared Error. \n", 744 | "loss = mse(out, y) # Inspecting Loss. " 745 | ], 746 | "execution_count": 23, 747 | "outputs": [] 748 | }, 749 | { 750 | "cell_type": "code", 751 | "metadata": { 752 | "id": "QRNluxaAG31R" 753 | }, 754 | "source": [ 755 | "#@ GRADIENT OF LOSS FUNCTION: \n", 756 | "def mse_grad(inp, targ): # Defining Gradient Function. \n", 757 | " inp.g = 2. * (inp.squeeze() - targ\n", 758 | " ).unsqueeze(-1) / inp.shape[0] # Calculating Gradients of Loss. \n", 759 | "\n", 760 | "#@ GRADIENT OF RELU ACTIVATION FUNCTION: \n", 761 | "def relu_grad(inp, out): # Defining Gradient Function. \n", 762 | " inp.g = (inp>0).float() * out.g # Calculating Gradients. " 763 | ], 764 | "execution_count": 24, 765 | "outputs": [] 766 | }, 767 | { 768 | "cell_type": "code", 769 | "metadata": { 770 | "id": "mPBckm1ZUJFS" 771 | }, 772 | "source": [ 773 | "#@ GRADIENT OF MATRIX MULTIPLICATION: \n", 774 | "def lin_grad(inp, out, w, b): # Defining the Function. \n", 775 | " inp.g = out.g @ w.t() # Getting the Gradients of Input. \n", 776 | " w.g = inp.t() @ out.g # Getting the Gradients of Weight. \n", 777 | " b.g = out.g.sum(0) # Getting the Gradients of Bias. \n", 778 | "\n", 779 | "#@ FORWARD AND BACKWARD PROPAGATION FUNCTION: \n", 780 | "def forward_and_backward(inp, targ): # Defining the Function. \n", 781 | " l1 = inp @ w1 + b1 # Initializing Linear Layer. \n", 782 | " l2 = relu(l1) # Implementation of RELU. \n", 783 | " out = l2 @ w2 + b2 # Initializing Linear Layer. \n", 784 | " loss = mse(out, targ) # Initializing Loss Function. \n", 785 | "\n", 786 | " mse_grad(out, targ) # Calculating Gradients of Loss. \n", 787 | " lin_grad(l2, out, w2, b2) # Getting Gradients. \n", 788 | " relu_grad(l1, l2) # Getting Gradients. \n", 789 | " lin_grad(inp, l1, w1, b1) # Getting Gradients. " 790 | ], 791 | "execution_count": 25, 792 | "outputs": [] 793 | }, 794 | { 795 | "cell_type": "markdown", 796 | "metadata": { 797 | "id": "K6Yw7y7sfJLJ" 798 | }, 799 | "source": [ 800 | "**REFACTORING THE MODEL**" 801 | ] 802 | }, 803 | { 804 | "cell_type": "code", 805 | "metadata": { 806 | "id": "U1rGohYYfEbd" 807 | }, 808 | "source": [ 809 | "#@ INITIALIZING RELU CLASS: \n", 810 | "class Relu(): # Defining RELU Class. \n", 811 | " def __call__(self, inp): # Initializing Callable Function. \n", 812 | " self.inp = inp # Initialization. \n", 813 | " self.out = inp.clamp_min(0.) # Initialization. \n", 814 | " return self.out\n", 815 | " \n", 816 | " def backward(self): # Backward Propagation Function. \n", 817 | " self.inp.g = (self.inp>0).float()*self.out.g # Getting Gradients of Inputs. " 818 | ], 819 | "execution_count": 26, 820 | "outputs": [] 821 | }, 822 | { 823 | "cell_type": "code", 824 | "metadata": { 825 | "id": "XoDv4LE6ibk9" 826 | }, 827 | "source": [ 828 | "#@ INITIALIZING LINEAR CLASS: \n", 829 | "class Lin(): # Defining Linear Class. \n", 830 | " def __init__(self, w, b): # Initializing Constructor Function. \n", 831 | " self.w, self.b = w, b # Initialization. \n", 832 | " \n", 833 | " def __call__(self, inp): # Initializing Callable Function. \n", 834 | " self.inp = inp # Initializing Inputs. \n", 835 | " self.out = inp @ self.w + self.b # Initializing Outputs. \n", 836 | " return self.out\n", 837 | " \n", 838 | " def backward(self): # Backward Propagation Function. \n", 839 | " self.inp.g = self.out.g @ self.w.t() # Calculating Gradients of Inputs. \n", 840 | " self.w.g = self.inp.t() @ self.out.g # Calculating Gradients of Weights.\n", 841 | " self.b.g = self.out.g.sum(0) # Calculating Gradients of Bias. " 842 | ], 843 | "execution_count": 27, 844 | "outputs": [] 845 | }, 846 | { 847 | "cell_type": "code", 848 | "metadata": { 849 | "id": "vM7lqXB9koyc" 850 | }, 851 | "source": [ 852 | "#@ INITIALIZING MEAN SQUARED ERROR: \n", 853 | "class Mse(): # Defining MSE Class. \n", 854 | " def __call__(self, inp, targ): # Initializing Callable Function. \n", 855 | " self.inp = inp # Initializing Inputs. \n", 856 | " self.targ = targ # Initializing Targets. \n", 857 | " self.out = (inp.squeeze()-targ).pow(2).mean() # Initializing Outputs. \n", 858 | " return self.out \n", 859 | " \n", 860 | " def backward(self): # Backward Propagation Function. \n", 861 | " x = (self.inp.squeeze()-self.targ\n", 862 | " ).unsqueeze(-1) # Calculating Loss. \n", 863 | " self.inp.g = 2.*x / self.targ.shape[0] # Calculating Gradients of Inputs. " 864 | ], 865 | "execution_count": 28, 866 | "outputs": [] 867 | }, 868 | { 869 | "cell_type": "code", 870 | "metadata": { 871 | "id": "AyqfjivVS4TR" 872 | }, 873 | "source": [ 874 | "#@ DEFINING THE MODEL ARCHITECTURE: \n", 875 | "class Model(): # Defining Model Class. \n", 876 | " def __init__(self, w1, b1, w2, b2): # Initializing Constructor Function. \n", 877 | " self.layers = [Lin(w1,b1), Relu(), Lin(w2,b2)] # Initializing Layers of Network. \n", 878 | " self.loss = Mse() # Initializing Mean Squared Error Loss. \n", 879 | " \n", 880 | " def __call__(self, x, targ): # Initializing Callable Function. \n", 881 | " for l in self.layers: x = l(x) # Implementation of Layers of Network. \n", 882 | " return self.loss(x, targ) # Implementation of Loss Function. \n", 883 | "\n", 884 | " def backward(self): # Initializing Back Propagation Function. \n", 885 | " self.loss.backward() # Calculating Gradients of Loss. \n", 886 | " for l in reversed(self.layers): l.backward() # Calculating Gradients. \n", 887 | "\n", 888 | "#@ IMPLEMENTATION OF MODEL: \n", 889 | "model = Model(w1, b1, w2, b2) # Initializing the Model. \n", 890 | "loss = model(x, y) # Forward Propagation Function.\n", 891 | "model.backward() # Back Propagation Function. " 892 | ], 893 | "execution_count": 29, 894 | "outputs": [] 895 | }, 896 | { 897 | "cell_type": "markdown", 898 | "metadata": { 899 | "id": "OpUByh32mqvu" 900 | }, 901 | "source": [ 902 | "**PYTORCH IMPLEMENTATION**" 903 | ] 904 | }, 905 | { 906 | "cell_type": "code", 907 | "metadata": { 908 | "id": "freCczi7lkB6" 909 | }, 910 | "source": [ 911 | "#@ DEFINING BASE CLASS FUNCTION: \n", 912 | "class LayerFunction(): # Defining Layer Function Class. \n", 913 | " def __call__(self, *args): # Initializing Callable Function. \n", 914 | " self.args = args # Initialization. \n", 915 | " self.out = self.forward(*args) # Initialization. \n", 916 | " return self.out\n", 917 | " \n", 918 | " def forward(self): raise Exception(\"not implemented\") # Forward Propagation Function. \n", 919 | " def bwd(self): raise Exception(\"not implemented\") \n", 920 | " def backward(self): self.bwd(self.out, *self.args) # Backward Propagation Function. \n", 921 | "\n", 922 | "#@ DEFINING SUBCLASS: RELU FUNCTION: \n", 923 | "class Relu(LayerFunction): # Defining RELU. \n", 924 | " def forward(self, inp): return inp.clamp_min(0.) # Forward Propagation Function. \n", 925 | " def bwd(self, out, inp): inp.g = (inp>0).float()*out.g # Back Propagation Function. " 926 | ], 927 | "execution_count": 30, 928 | "outputs": [] 929 | }, 930 | { 931 | "cell_type": "code", 932 | "metadata": { 933 | "id": "1gS5Ig9ZqFg2" 934 | }, 935 | "source": [ 936 | "#@ DEFINING SUBCLASS: LINEAR LAYER FUNCTION:\n", 937 | "class Lin(LayerFunction): # Defining Linear Layer Function. \n", 938 | " def __init__(self, w, b): # Initializing Constructor Function. \n", 939 | " self.w, self.b = w, b # Initialization. \n", 940 | " \n", 941 | " def forward(self, inp): # Forward Propagation Function. \n", 942 | " return inp @ self.w + self.b # Initializing Linear Function. \n", 943 | "\n", 944 | " def bwd(self, out, inp): # Initializing Back Propagation. \n", 945 | " inp.g = out.g @ self.w.t() # Calculating the Gradients. \n", 946 | " self.w.g = self.inp.t() @ self.out.g # Calculating the Gradients. \n", 947 | " self.b.g = out.g.sum(0) # Calculating the Gradients. \n", 948 | "\n", 949 | "#@ DEFINING SUBCLASS: MEAN SQUARED ERROR FUNCTION: \n", 950 | "class Mse(LayerFunction): # Defining MSE Function. \n", 951 | " def forward(self, inp, targ): # Forward Propagation Function. \n", 952 | " return (inp.squeeze() - targ).pow(2).mean() # Calculating Mean Squared Error. \n", 953 | " \n", 954 | " def bwd(self, out, inp, targ):\n", 955 | " inp.g = 2*(inp.squeeze() - targ\n", 956 | " ).unsqueeze(-1) / targ.shape[0] # Initializing Back Propagation. " 957 | ], 958 | "execution_count": 31, 959 | "outputs": [] 960 | }, 961 | { 962 | "cell_type": "code", 963 | "metadata": { 964 | "id": "1WlzhcvqpGr9" 965 | }, 966 | "source": [ 967 | "#@ DEFINING CUSTOM FUNCTION: \n", 968 | "class MyRelu(Function): # Defining RELU Function. \n", 969 | " @staticmethod\n", 970 | " def forward(ctx, i): # Forward Propagation Function. \n", 971 | " result = i.clamp_min(0.)\n", 972 | " ctx.save_for_backward(i)\n", 973 | " return result\n", 974 | " \n", 975 | " @staticmethod\n", 976 | " def backward(ctx, grad_output): # Back Propagation Function. \n", 977 | " i, = ctx.saved_tensors\n", 978 | " return grad_output * (i>0).float()" 979 | ], 980 | "execution_count": 33, 981 | "outputs": [] 982 | }, 983 | { 984 | "cell_type": "markdown", 985 | "metadata": { 986 | "id": "c_QYBUWerKHe" 987 | }, 988 | "source": [ 989 | "**NEURAL NETWORK MODULE**" 990 | ] 991 | }, 992 | { 993 | "cell_type": "code", 994 | "metadata": { 995 | "colab": { 996 | "base_uri": "https://localhost:8080/" 997 | }, 998 | "id": "EgTNTm_IqvoF", 999 | "outputId": "b0b29230-38e1-469a-f30d-eedeef95f6c8" 1000 | }, 1001 | "source": [ 1002 | "#@ DEFINING LINEAR LAYER FROM SCRATCH: \n", 1003 | "class LinearLayer(nn.Module): # Defining Linear Layer. \n", 1004 | " def __init__(self, n_in, n_out): # Initializing Constructor Function. \n", 1005 | " super().__init__()\n", 1006 | " self.weight = nn.Parameter(\n", 1007 | " torch.randn(n_out, n_in) * math.sqrt(2/n_in)) # Initializing Weight Parameters. \n", 1008 | " self.bias = nn.Parameter(torch.zeros(n_out)) # Initializing Bias Parameters.\n", 1009 | "\n", 1010 | " def forward(self, x): # Forward Propagation Function. \n", 1011 | " return x @ self.weight.t() + self.bias # Initializing Linear Function. \n", 1012 | "\n", 1013 | "#@ IMPLEMENTATION OF LINEAR LAYER FUNCTION: \n", 1014 | "lin = LinearLayer(10, 2) # Initializing Linear Layer. \n", 1015 | "p1, p2 = lin.parameters() # Initializing Parameters. \n", 1016 | "p1.shape, p2.shape # Insepcting Parameters. " 1017 | ], 1018 | "execution_count": 34, 1019 | "outputs": [ 1020 | { 1021 | "output_type": "execute_result", 1022 | "data": { 1023 | "text/plain": [ 1024 | "(torch.Size([2, 10]), torch.Size([2]))" 1025 | ] 1026 | }, 1027 | "metadata": {}, 1028 | "execution_count": 34 1029 | } 1030 | ] 1031 | }, 1032 | { 1033 | "cell_type": "code", 1034 | "metadata": { 1035 | "id": "SZevmnMFt-w9" 1036 | }, 1037 | "source": [ 1038 | "#@ DEFINING LINEAR MODEL: PYTORCH:\n", 1039 | "class Model(nn.Module): # Defining Linear Model. \n", 1040 | " def __init__(self, n_in, nh, n_out): # Initializing Constructor Function. \n", 1041 | " super().__init__()\n", 1042 | " self.layers = nn.Sequential(nn.Linear(n_in, nh), # Initializing Linear Layer. \n", 1043 | " nn.ReLU(), # Initializing RELU Function. \n", 1044 | " nn.Linear(nh, n_out)) # Initializing Linear Layer. \n", 1045 | " self.loss = mse # Initializing MSE Loss Function. \n", 1046 | " \n", 1047 | " def forward(self, x, targ): # Forward Propagation Function. \n", 1048 | " return self.loss(self.layers(x).squeeze(), targ) # Calculating Loss. \n", 1049 | "\n", 1050 | "#@ DEFINING LINEAR MODEL: FASTAI:\n", 1051 | "class Model(Module): # Defining Linear Model. \n", 1052 | " def __init__(self, n_in, nh, n_out): # Initializing Constructor Function. \n", 1053 | " self.layers = nn.Sequential(nn.Linear(n_in, nh), # Initializing Linear Layer. \n", 1054 | " nn.ReLU(), # Initializing RELU Function. \n", 1055 | " nn.Linear(nh, n_out)) # Initializing Linear Layer. \n", 1056 | " \n", 1057 | " def forward(self, x, targ): # Forward Propagation Function. \n", 1058 | " return self.loss(self.layers(x).squeeze(), targ) # Calculating Loss. " 1059 | ], 1060 | "execution_count": 36, 1061 | "outputs": [] 1062 | } 1063 | ] 1064 | } -------------------------------------------------------------------------------- /16. Neural Network Foundations/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Neural Network Foundations** 2 | 3 | The [**Neural Foundations**](https://github.com/ThinamXx/Fastai/blob/main/16.%20Neural%20Network%20Foundations/NeuralFoundations.ipynb) notebook contains all the dependencies required to understand the foundations of deep learning, begining with matrix multiplication and moving on to implementing the forward and backward passes of a neural net from scratch. 4 | 5 | **Note** 6 | - 📑[**Neural Network Foundations**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/16.%20Neural%20Network%20Foundations/NeuralFoundations.ipynb) 7 | 8 | **Matrix Multiplications** 9 | - I have presented the implementation of Matrix Multiplication from Scratch, Elementwise Arithmetic, Einstein Summation and Defining and Initializing Linear Layer using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20264.PNG) 12 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20265.PNG) 13 | 14 | **Forward and Backward Passes** 15 | - Computing all the gradients of a given loss with respect to its parameters is known as Backward Pass. Similarly computing the output of the model on a given input based on the matrix products is known as Forward Pass. I have presented the implementation of Kaiming Initialization, MSE Loss Function and Gradients using Fastai and PyTorch here in the snapshot. 16 | 17 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20268.PNG) 18 | 19 | **Linear Model** 20 | - I have presented the implementation of Defining Model Architecture, Layer Function and RELU, Defining Linear Layer and Linear Model using Fastai and PyTorch here in the snapshot. 21 | 22 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20269.PNG) 23 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20270.PNG) 24 | -------------------------------------------------------------------------------- /16. Neural Network Foundations/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook -------------------------------------------------------------------------------- /17. CNN Interpretation/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : CNN Interpretation with CAM** 2 | 3 | The [**CNN Interpretation**](https://github.com/ThinamXx/Fastai/blob/main/17.%20CNN%20Interpretation/CNN%20Interpretation.ipynb) notebook presents the implementation of **Class Activation Maps** in model interpretation. Class activation maps give insights into why a model predicted a certain result by showing the areas of images that were most responsible for a given prediction. 4 | 5 | **Note:** 6 | - 📑[**CNN Interpretation with CAM**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/17.%20CNN%20Interpretation/CNN%20Interpretation.ipynb) 7 | 8 | **Class Activation Map** 9 | - The Class Activation Map uses the output of the last convolutional layer which is just before the average pooling layer together with predictions to give a heatmap visualization of model decision. I have presented the implementation of Defining Hook Function and Decoding Images using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20271.PNG) 12 | 13 | **Hook Class** 14 | - I have presented the implementation of Defining Hook Function, Activations, Gradients and Heatmap Visualization using Fastai and PyTorch here in the snapshot. 15 | 16 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20272a.PNG) 17 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20272b.PNG) 18 | -------------------------------------------------------------------------------- /17. CNN Interpretation/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook 2 | fastai.vision.all -------------------------------------------------------------------------------- /18. Fastai Learner/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Fastai Learner from Scratch** 2 | 3 | The [**Fastai Learner**](https://github.com/ThinamXx/Fastai/blob/main/18.%20Fastai%20Learner/Fastai%20Learner.ipynb) notebook contains all the dependencies to understand the key concepts of **Fastai**. 4 | 5 | **Note:** 6 | - 📑[**Fastai Learner from Scratch**](https://nbviewer.jupyter.org/github/ThinamXx/Fastai/blob/main/18.%20Fastai%20Learner/Fastai%20Learner.ipynb) 7 | 8 | **Preparing Dataset and DataLoaders** 9 | - I have presented the implementation of Preparing Training and Validation Dataset, Initializing Data Loader and Normalization using Fastai and PyTorch here in the snapshot. 10 | 11 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20273.PNG) 12 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20274.PNG) 13 | 14 | **Linear Model** 15 | - I have presented the implementation of Defining Module : Convolutional Layer and Linear Model using Fastai and PyTorch here in the snapshot. 16 | 17 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20275.PNG) 18 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20276.PNG) 19 | 20 | **Learner Module** 21 | - I have presented the implementation of Negative Log Likelihood Function, Cross Entropy Loss Function, SGD Optimizer, Data Loaders, Learner and Callbacks using Fastai and PyTorch here in the snapshot. 22 | 23 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20277.PNG) 24 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20278.PNG) 25 | -------------------------------------------------------------------------------- /18. Fastai Learner/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastbook 2 | fastai.vision.all -------------------------------------------------------------------------------- /19. Chest XRays Classification/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Chest XRays Classification** 2 | 3 | **Note:** 4 | - 📑[**Chest XRays Classification**](https://github.com/ThinamXx/Fastai/blob/main/19.%20Chest%20XRays%20Classification/XRays%20Classification.ipynb) 5 | 6 | **DICOM Files** 7 | - DICOM stands for Digital Imaging and Communications in Medicine. It is the de-facto standard that establishes rules that allow the medical images and associated information to be exchanged between imaging equipment from different vendors, computers and hospitals. I have presented the implementation of Initializing Data Block and Data Loaders, Training Pretrained Model and Interpretation using Fastai and PyTorch here in the snapshot. 8 | 9 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20279.PNG) 10 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20280.PNG) 11 | 12 | **Sensitivity & Specificity** 13 | - Sensitivity = True Positive / (True Positive + False Negative). It is also known as a Type II Error. Specificity = True Negative / (False Positive + True Negative). It is also known as a Type I Error. I have presented the implementation of Confusion Matrix, Sensitivity and Specificity, Accuracy using Fastai and PyTorch here in the snapshot. 14 | 15 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20281.PNG) 16 | -------------------------------------------------------------------------------- /19. Chest XRays Classification/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai.basics 2 | fastai.callback 3 | fastai.vision 4 | fastai.medical.imaging 5 | pydicom 6 | pandas -------------------------------------------------------------------------------- /20. Transformers/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Transformers Model** 2 | 3 | **Note:** 4 | - 📑[**Transformers**](https://github.com/ThinamXx/Fastai/blob/main/20.%20Transformers/Transformers.ipynb) 5 | 6 | **GPT2 Model** 7 | - There are several versions of GPT2 Model : [**Transformers Documentation**](https://huggingface.co/transformers/pretrained_models.html). I will inspect the Tokenizer and Model. The Tokenizers in HuggingFace usually do the tokenization and numericalization in one step. The Model can generate predictions. I have presented the implementation of Pretrained GPT2 Model and Tokenizer and Transformed DataLoaders using Fastai and PyTorch here in the snapshot. 8 | 9 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20299a.PNG) 10 | 11 | **Transformers Tokenizer** 12 | - Fastai Transform is defined as: 13 | - an encodes method that is applied when transform is called. 14 | - a decodes method that is applied when decode method of transform is called. 15 | - a setups method that sets inner state of Transform. 16 | 17 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20299b.PNG) 18 | 19 | **Fine Tuning Model** 20 | - Since it is a Language Model, I will pass perplexity as a metric. Perplexity is the exponential of the cross entropy loss function. I have presented the implementation of Initializing DataLoaders, Fine Tuning GPT2Model and LR Finder using Fastai and PyTorch here in the snapshot. 21 | 22 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20300a.PNG) 23 | ![Image](https://github.com/ThinamXx/300Days__MachineLearningDeepLearning/blob/main/Images/Day%20300b.PNG) 24 | -------------------------------------------------------------------------------- /20. Transformers/Requirements.txt: -------------------------------------------------------------------------------- 1 | fastai.basics 2 | fastai.callback 3 | fastai.text 4 | transformers 5 | torch -------------------------------------------------------------------------------- /21. Stable Diffusion/.gitignore: -------------------------------------------------------------------------------- 1 | sam_* 2 | .vscode/ -------------------------------------------------------------------------------- /21. Stable Diffusion/README.md: -------------------------------------------------------------------------------- 1 | # **Fastai : Stable Diffusion** 2 | -------------------------------------------------------------------------------- /21. Stable Diffusion/app.py: -------------------------------------------------------------------------------- 1 | import gradio as gr 2 | import numpy as np 3 | import torch 4 | from diffusers import StableDiffusionInpaintPipeline 5 | from PIL import Image 6 | from segment_anything import SamPredictor, sam_model_registry 7 | 8 | device = "cuda" 9 | checkpoint = "./sam_vit_b_01ec64.pth" 10 | model_type = "vit_b" 11 | sam = sam_model_registry[model_type](checkpoint=checkpoint) 12 | sam = sam.to(device) 13 | predictor = SamPredictor(sam) 14 | 15 | pipe = StableDiffusionInpaintPipeline.from_pretrained( 16 | "stabilityai/stable-diffusion-2-inpainting", 17 | torch_dtype=torch.float16, 18 | ) 19 | pipe = pipe.to(device) 20 | 21 | # gradio 22 | selected_pixels = [] 23 | with gr.Blocks() as demo: 24 | with gr.Row(): 25 | input_img = gr.Image(label="Input Image") 26 | mask_img = gr.Image(label="Mask") 27 | output_img = gr.Image(label="Output Image") 28 | 29 | with gr.Row(): 30 | prompt_text = gr.Textbox(lines=1, label="Prompt Text") 31 | 32 | with gr.Row(): 33 | submit = gr.Button(label="Submit") 34 | 35 | def generate_mask(image, evt: gr.SelectData): 36 | selected_pixels.append(evt.index) 37 | 38 | predictor.set_image(image) 39 | input_points = np.array(selected_pixels) 40 | input_labels = np.ones(input_points.shape[0]) 41 | mask, _, _ = predictor.predict( 42 | point_coords=input_points, 43 | point_labels=input_labels, 44 | multimask_output=False 45 | ) 46 | mask = Image.fromarray(mask[0, :, :]) 47 | return mask 48 | 49 | def inpaint(image, mask, prompt): 50 | image = Image.fromarray(image) 51 | mask = Image.fromarray(mask) 52 | 53 | image = image.resize((512, 512)) 54 | mask = mask.resize((512, 512)) 55 | 56 | output = pipe(prompt=prompt, image=image, mask_image=mask).images[0] 57 | return output 58 | 59 | 60 | input_img.select(generate_mask, [input_img], [mask_img]) 61 | submit.click( 62 | inpaint, 63 | inputs=[input_img, mask_img, prompt_text], 64 | outputs=[output_img] 65 | ) 66 | 67 | 68 | if __name__ == "__main__": 69 | demo.launch() 70 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Thinam Tamang 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # **DEEP LEARNING FOR CODERS WITH FASTAI AND PYTORCH** 2 | 3 | The repository contains a list of the projects which I have worked on while reading the book [**Deep Learning For Coders with Fastai and PyTorch**](https://course.fast.ai/#). 4 | 5 | ### **📚NOTEBOOKS:** 6 | 7 | [**1. INTRODUCTION**](https://github.com/ThinamXx/Fastai/tree/main/1.%20Introduction) 8 | - The **Introduction** notebook is a comprehensive notebook as it contains a list of projects such as **Cat and Dog Classification, Semantic Segmentation, Sentiment Classification, Tabular Classification and Recommendation System**. 9 | 10 | [**2. MODEL PRODUCTION**](https://github.com/ThinamXx/Fastai/tree/main/2.%20Model%20Production) 11 | - The **BearDetector notebook** contains all the dependencies for a complete Image Classification project. 12 | 13 | [**3. TRAINING A CLASSIFIER**](https://github.com/ThinamXx/Fastai/tree/main/3.%20Training%20a%20Classifier) 14 | - The **DigitClassifier** notebook contains all the dependencies required for Image Classification project from scratch. 15 | 16 | [**4. IMAGE CLASSIFICATION**](https://github.com/ThinamXx/Fastai/tree/main/4.%20Image%20Classification) 17 | - The **Image Classification** notebook contains all the dependencies for Image Classification such as getting image data ready for modeling i.e presizing and data block summary and for fitting the model i.e learning rate finder, unfreezing, discriminative learning rates, setting the number of epochs and using deeper architectures. It has explanations of cross entropy loss function as well. 18 | 19 | [**5. MULTILABEL CLASSIFICATION AND REGRESSION**](https://github.com/ThinamXx/Fastai/tree/main/5.%20MultilabelClassification%20Regression) 20 | - The **Multilabel Classification** notebook contains all the dependencies required to understand **Multilabel Classification**. It contains the explanations of initializing **DataBlock** and **DataLoaders**. The **Regression** notebook contains all the dependencies required to understand **Image Regression**. 21 | 22 | [**6. ADVANCED CLASSIFICATION**](https://github.com/ThinamXx/Fastai/tree/main/6.%20Advanced%20Classification) 23 | - The **Imagenette Classification** notebook contains all the dependencies required to train a state of art machine learning model in computer vision whether from scratch or using transfer learning. It contains explanations and implementation of **Normalization, Progressive Resizing, Test Time Augmentation, Mixup Augmentation and Label Smoothing**. 24 | 25 | [**7. COLLABORATIVE FILTERING**](https://github.com/ThinamXx/Fastai/tree/main/7.%20Collaborative%20Filtering) 26 | - The **Collaborative Filtering** notebook contains all the dependencies required to build a **Recommendation System**. It presents how gradient descent can learn intrinsic factors or biases about items from a history of ratings which then gives information about the data. 27 | 28 | [**8. TABULAR MODELING**](https://github.com/ThinamXx/Fastai/tree/main/8.%20Tabular%20Modeling) 29 | - The **Tabular Model** notebook contains all the dependencies required for Tabular Modeling. It presents the detailed explanations of two approaches to **Tabular Modeling: Decision Tree Ensembles and Neural Networks**. 30 | 31 | [**9. NATURAL LANGUAGE PROCESSING**](https://github.com/ThinamXx/Fastai/tree/main/9.%20Natural%20Language%20Processing) 32 | - The **NLP** notebook contains all the dependencies required build Language Model that can generate texts and a Classifier Model that determines whether a review is positive or negative. It presents the state of art Classifier Model which is build using a pretrained language model and fine tuned it to the corpus of task. Then the Encoder model is used for classification. 33 | 34 | [**10. DATA MUNGING**](https://github.com/ThinamXx/Fastai/tree/main/10.%20Data%20Munging) 35 | - The **DataMunging** notebook contains all the dependencies required to implement mid level API of Fast.ai in Natural Language Processing and Computer Vision which provides greater flexibility to apply transformations on data items. 36 | 37 | [**11. LANGUAGE MODEL FROM SCRATCH**](https://github.com/ThinamXx/Fastai/tree/main/11.%20Language%20Model) 38 | - The **LanguageModel** notebook contains all the dependencies that is inside AWD-LSTM architecture for Text Classification. It presents the implementation of Language Model using simple Linear Model, Recurrent Neural Network, Long Short Term Memory, Dropout Regularization and Activation Regularization. 39 | 40 | [**12. CONVOLUTIONAL NEURAL NETWORK**](https://github.com/ThinamXx/Fastai/tree/main/12.%20Convolutional%20Neural%20Networks) 41 | - The **CNN** notebook contains all the dependencies required to understand Convolutional Neural Networks. Convolutions are just a type of matrix multiplication with two constraints on the weight matrix: some elements are always zero and some elements are tied or forced to always have the same value. 42 | 43 | [**13. RESIDUAL NETWORKS**](https://github.com/ThinamXx/Fastai/tree/main/13.%20ResNets) 44 | - The **ResNets** notebook contains all the dependencies required to understand the implementation of skip connections which allow deeper models to be trained. ResNet is the pretrained model when using **Transfer Learning**. 45 | 46 | [**14. ARCHITECTURE DETAILS**](https://github.com/ThinamXx/Fastai/tree/main/14.%20Architecture%20Details) 47 | - The **Architecture Details** notebook contains all the dependencies required to create a complete state of art computer vision models. It presents some aspects of natural language processing as well. 48 | 49 | [**15. TRAINING PROCESS**](https://github.com/ThinamXx/Fastai/tree/main/15.%20Training%20Process) 50 | - The **Training** notebook contains all the dependencies required to create a training loop and explored variants of Stochastic Gradient Descent. 51 | 52 | [**16. NEURAL NETWORK FOUNDATIONS**](https://github.com/ThinamXx/Fastai/tree/main/16.%20Neural%20Network%20Foundations) 53 | - The **Neural Foundations** notebook contains all the dependencies required to understand the foundations of deep learning, begining with matrix multiplication and moving on to implementing the forward and backward passes of a neural net from scratch. 54 | 55 | [**17. CNN INTERPRETATION WITH CAM**](https://github.com/ThinamXx/Fastai/tree/main/17.%20CNN%20Interpretation) 56 | - The **CNN Interpretation** notebook presents the implementation of Class Activation Maps in model interpretation. Class activation maps give insights into why a model predicted a certain result by showing the areas of images that were most responsible for a given prediction. 57 | 58 | [**18. FASTAI LEARNER FROM SCRATCH**](https://github.com/ThinamXx/Fastai/tree/main/18.%20Fastai%20Learner) 59 | - The **Fastai Learner** notebook contains all the dependencies to understand the key concepts of Fastai. 60 | 61 | [**19. CHEST X-RAYS CLASSIFICATION**](https://github.com/ThinamXx/Fastai/tree/main/19.%20Chest%20XRays%20Classification) 62 | 63 | [**20. TRANSFORMERS MODEL**](https://github.com/ThinamXx/Fastai/tree/main/20.%20Transformers) 64 | --------------------------------------------------------------------------------