├── .gitignore ├── .pre-commit-config.yaml ├── README.md ├── assets ├── celeb-a.gif ├── cifar-10.gif ├── cifar-100.gif ├── cnn_01.jpg ├── colab_01.jpg ├── colab_02.jpg ├── colab_03.jpg ├── fashion-mnist.gif ├── generator_architecture.jpg ├── lin_reg_01.jpg ├── mlp_01.jpg ├── mnist.gif ├── perceptron_01.jpg ├── samples_celeb-a.png ├── samples_cifar-10.png ├── samples_cifar-100.png ├── samples_fashion_mnist.png ├── samples_mnist.png ├── vae_01.jpg ├── vae_02.jpg ├── vae_03.jpg └── vae_04.jpg ├── autograd ├── mlx │ ├── autograd_mlx.ipynb │ └── requirements.txt └── torch │ ├── autograd_torch.ipynb │ └── requirements.txt ├── cnn ├── cifar │ ├── mlx │ │ └── cnn_cifar_mlx.ipynb │ └── torch │ │ ├── cnn_cifar_torch.ipynb │ │ └── cnn_cifar_torch_analysis.ipynb └── mnist │ └── torch │ ├── cnn_mnist_torch.ipynb │ └── requirements.txt ├── gan └── torch │ ├── gan_torch_celeba.ipynb │ ├── gan_torch_fashion_mnist.ipynb │ ├── gan_torch_mnist.ipynb │ └── requirements.txt ├── linear_regression ├── mlx │ ├── linear_regression_mlx.ipynb │ └── requirements.txt └── torch │ ├── linear_regression_torch.ipynb │ ├── linear_regression_torch_manual_grad_desc.ipynb │ ├── linear_regression_torch_optim.ipynb │ └── requirements.txt ├── multi_layer_perceptron ├── mlx │ ├── mlp_2class_mlx.ipynb │ ├── mlp_XOR_mlx.ipynb │ └── requirements.txt └── torch │ ├── mlp_2class_torch.ipynb │ ├── mlp_XOR_torch.ipynb │ ├── mlp_mnist_torch.ipynb │ └── requirements.txt ├── nbutils ├── __init__.py ├── colab.py ├── data.py ├── exec.py ├── git.py └── requirements.py ├── perceptron ├── mlx │ ├── perceptron_OR_mlx.ipynb │ └── requirements.txt ├── numpy │ ├── perceptron_OR_spelled_out_numpy.ipynb │ └── requirements.txt └── torch │ ├── perceptron_OR_spelled_out_torch.ipynb │ ├── perceptron_OR_torch.ipynb │ ├── perceptron_XOR_torch.ipynb │ └── requirements.txt ├── requirements_mlx.txt ├── requirements_torch.txt ├── setupOnColab.ipynb └── vae └── torch ├── requirements.txt ├── vae_torch_celeba.ipynb ├── vae_torch_fashion_mnist.ipynb └── vae_torch_mnist.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | *.code-workspace 3 | cnn/cifar/mlx/mlx-cifar-10-cnn 4 | cnn/cifar/mlx/*.npz 5 | cnn/cifar/torch/torch-cifar-10-cnn 6 | cnn/cifar/torch/*.pth 7 | cnn/cifar/torch/data/ 8 | cnn/cifar/mlx/models 9 | cnn/cifar/torch/models 10 | vae/torch/torch-vae/ 11 | vae/torch/data/ 12 | vae/torch/data 13 | gan/torch/data 14 | gan/torch/torch-gan 15 | */__pycache__/ 16 | multi_layer_perceptron/torch/data 17 | cnn/mnist/torch/data/ 18 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | repos: 2 | - repo: https://github.com/psf/black-pre-commit-mirror 3 | rev: 23.12.1 4 | hooks: 5 | - id: black 6 | - id: black-jupyter 7 | - repo: https://github.com/pycqa/isort 8 | rev: 5.12.0 9 | hooks: 10 | - id: isort 11 | args: 12 | - --profile=black 13 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Computer Vision and Machine Learning Jupyter Notebooks 2 | 3 | ## About 4 | 5 | Author: [Markus Enzweiler](https://markus-enzweiler-de), markus.enzweiler@hs-esslingen.de 6 | 7 | This repository contains Jupyter notebooks used in my lectures at Esslingen University of Applied Sciences. The notebooks mainly use PyTorch (https://pytorch.org/) and are written for educational purposes with explanations and visualizations. There are additional notebooks available for some topics that use MLX for Apple M(x) SoCs (https://github.com/ml-explore/mlx) or pure NumPy (https://numpy.org/). 8 | 9 | ## Contributing 10 | 11 | **Feel free to use and contribute!** 12 | 13 | I welcome contributions from the community. If you find bugs or have suggestions for additional features, please feel free to submit pull requests. 14 | 15 | ### Suggested Contributions: 16 | - **Additional Topics**: Contributions that introduce new topics such as CNNs for object detection / semantic segmentation, diffusion models, RNNs (Recurrent Neural Networks), LSTMs (Long Short-Term Memory networks), Transformers, and other advanced neural network architectures are highly appreciated. 17 | - **JAX Examples**: I'm particularly interested in expanding the examples to include JAX implementations. 18 | - **Theoretical Explanations**: In-depth theoretical explanations or visualizations that enhance understanding of the presented concepts in machine learning and neural network design are welcome. Contributions that bridge the gap between theory and practical implementation would be especially valuable. 19 | 20 | 21 | ### Use of the Material: 22 | Please cite or acknowledge this repository when using its contents for your teaching or projects. 23 | 24 | 25 | ## Contents 26 | 27 | To work through all of the material, the following order is recommended. 28 | 29 | ### 1) Automatic Differentiation 30 | * `autograd`: Demo of automatic differentiation in PyTorch and MLX. 31 | 32 | 33 | ### 2) Linear Regression 34 | * `linear_regression`: Demo of linear regression in PyTorch and MLX. The PyTorch examples use three variants: "manual" gradient descent, gradient descent via `loss.backward()`, and gradient descent using PyTorch optimizers (`torch.optim`) 35 | 36 | 37 | 38 | 39 | ### 3) Perceptrons 40 | * `perceptron`: Multiple demos of perceptrons in NumPy, PyTorch and MLX. Some of the demos are very "spelled-out" to provide some insights into the inner working of gradient descent, e.g. comparison of manually derived gradients vs. automatic differentiation. There are several different variants of (the same) gradient descent implemented: "manual" gradient descent, gradient descent via autograd, gradient descent via backprop (`loss.backward()`), and gradient descent using PyTorch optimizers (`torch.optim`). 41 | 42 | 43 | 44 | 45 | ### 4) Multi-Layer Perceptrons (MLPs) 46 | * `multi_layer_perceptron`: Multiple demos of multi-layer perceptrons in PyTorch and MLX including visualizations of decision boundaries. 47 | 48 | 49 | 50 | 51 | ### 5) Convolutional Neural Networks (CNNs) 52 | * `cnn`: Demos of CNNs in PyTorch and MLX for image classification. This example uses code from the repositories https://github.com/menzHSE/torch-cifar-10-cnn and https://github.com/menzHSE/mlx-cifar-10-cnn. It demonstrates defining, training and testing a small custom ResNet-like CNN from scratch as well as finetuning a larger ResNet that has been pre-trained on ImageNet to CIFAR-10 / 100. 53 | 54 | 55 | 56 | 57 | ### 6) Convolutional Variational Autoencoders (VAEs) 58 | * `vae`: Demos and analyses of convolutional Variational Autoencoders (VAEs) using the MNIST, Fashion MNIST and Celeb-A datasets in PyTorch including: Training, reconstruction of training and test data, generation of random samples, visualization of latent spaces. This example comes with pretrained VAE models and uses code from the repository https://github.com/menzHSE/torch-vae. 59 | 60 | 61 | 62 |
63 | 64 |
65 | 66 |
67 | 68 | 69 | 70 | ### 7) Convolutional Generative Adversarial Networks (GANs) 71 | * `gan`: Demos and analyses of convolutional Deep Generative Adversarial Networks (DCGANs) using the MNIST, Fashion MNIST and Celeb-A datasets in PyTorch including: Training, generation of random samples, visualization of latent spaces. This example comes with pretrained GAN models and uses code from the repository https://github.com/menzHSE/torch-gan. 72 | 73 | **MNIST** 74 | 75 | 76 |
77 | 78 | 79 | 80 | 81 | **Fashion MNIST** 82 | 83 | 84 |
85 | 86 | 87 | 88 | 89 | **CelebA** 90 | 91 | 92 |
93 | 94 | 95 | 96 | 97 | 98 | ## Usage 99 | You can run the Jupyter notebooks locally or in a cloud environment, e.g. Google Colab. There is a `requirements.txt` file in every folder that is automatically used by the notebooks to install dependencies within the active environments. 100 | 101 | ### Google Colab 102 | On Google Colab this repository works by interfacing with Google Drive to host the repository itself. To set this up, a notebook `setupOnColab.ipynb` is provided in the repository. This notebook mounts and clones the repository to Google Drive. **It has to be executed once in Colab.** After it has been executed, the repository is available on your Google Drive. In each Jupyter notebook, the Google Drive is then mounted to the local Colab virtual machine. 103 | 104 | There a two different way to open and execute `setupOnColab.ipynb` in Colab. 105 | 106 | #### 1) Open Colab and select File -> Open Notebook. From the menu, choose GitHub and open `setupOnColab.ipynb` 107 | 108 | 109 | 110 | #### 2) Clone this repo locally, copy `setupOnColab.ipynb` to your Google Drive and open `setupOnColab.ipynb` from there in Colab 111 | 112 | 113 | 114 | #### Once the repository has been cloned to your Google Drive with either of the two methods, open the notebooks directly from there in Colab. Make sure to select a GPU runtime for computation-heavy notebooks! 115 | 116 | 117 | 118 | ### Local Environment 119 | 120 | For local setups, a `conda` environment with Jupyter kernels is recommended, e.g. 121 | 122 | ``` 123 | conda create --name cv-ml-torch python=3.10 124 | conda activate cv-ml-torch 125 | conda install ipykernel 126 | git clone https://github.com/menzHSE/cv-ml-lecture-notebooks 127 | pip install -r cv-ml-lecture-notebooks/requirements_torch.txt 128 | python -m ipykernel install --user --name=cv-ml-torch 129 | ``` 130 | 131 | There are two different requirements files, `requirements_torch.txt` for PyTorch environments and `requirements_mlx.txt` for Apple MLX environments, that can be installed via `pip`. On an Apple silicon machine, both can be installed in the same environment. 132 | 133 | After installation of all dependencies and the custom Jupyter kernel you can open notebooks in your favorite tool (e.g. VSCode) and select the installed Jupyter kernel `cv-ml-torch` to execute the notebooks. 134 | -------------------------------------------------------------------------------- /assets/celeb-a.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/celeb-a.gif -------------------------------------------------------------------------------- /assets/cifar-10.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/cifar-10.gif -------------------------------------------------------------------------------- /assets/cifar-100.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/cifar-100.gif -------------------------------------------------------------------------------- /assets/cnn_01.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/cnn_01.jpg -------------------------------------------------------------------------------- /assets/colab_01.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/colab_01.jpg -------------------------------------------------------------------------------- /assets/colab_02.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/colab_02.jpg -------------------------------------------------------------------------------- /assets/colab_03.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/colab_03.jpg -------------------------------------------------------------------------------- /assets/fashion-mnist.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/fashion-mnist.gif -------------------------------------------------------------------------------- /assets/generator_architecture.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/generator_architecture.jpg -------------------------------------------------------------------------------- /assets/lin_reg_01.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/lin_reg_01.jpg -------------------------------------------------------------------------------- /assets/mlp_01.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/mlp_01.jpg -------------------------------------------------------------------------------- /assets/mnist.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/mnist.gif -------------------------------------------------------------------------------- /assets/perceptron_01.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/perceptron_01.jpg -------------------------------------------------------------------------------- /assets/samples_celeb-a.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/samples_celeb-a.png -------------------------------------------------------------------------------- /assets/samples_cifar-10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/samples_cifar-10.png -------------------------------------------------------------------------------- /assets/samples_cifar-100.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/samples_cifar-100.png -------------------------------------------------------------------------------- /assets/samples_fashion_mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/samples_fashion_mnist.png -------------------------------------------------------------------------------- /assets/samples_mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/samples_mnist.png -------------------------------------------------------------------------------- /assets/vae_01.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/vae_01.jpg -------------------------------------------------------------------------------- /assets/vae_02.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/vae_02.jpg -------------------------------------------------------------------------------- /assets/vae_03.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/vae_03.jpg -------------------------------------------------------------------------------- /assets/vae_04.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/assets/vae_04.jpg -------------------------------------------------------------------------------- /autograd/mlx/autograd_mlx.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "epxcwtWj5yJs" 7 | }, 8 | "source": [ 9 | "# Automatic differentiation in MLX\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute.\n", 14 | "\n", 15 | "**Note: This requires a machine with an Apple SoC, e.g. M1/M2/M3 etc.**\n", 16 | "\n", 17 | "See: https://github.com/ml-explore/mlx\n" 18 | ] 19 | }, 20 | { 21 | "cell_type": "markdown", 22 | "metadata": { 23 | "id": "IJsjl_l47q28" 24 | }, 25 | "source": [ 26 | "## Setup\n", 27 | "\n", 28 | "Adapt `packagePath` to point to the directory containing this notebeook." 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": { 35 | "executionInfo": { 36 | "elapsed": 236, 37 | "status": "ok", 38 | "timestamp": 1703326573912, 39 | "user": { 40 | "displayName": "Markus Enzweiler", 41 | "userId": "04524044579212347608" 42 | }, 43 | "user_tz": -60 44 | }, 45 | "id": "1iHkPBml98YG" 46 | }, 47 | "outputs": [], 48 | "source": [ 49 | "# Notebook id\n", 50 | "nb_id = \"autograd/mlx\"\n", 51 | "\n", 52 | "# Imports\n", 53 | "import sys\n", 54 | "import os" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": null, 60 | "metadata": { 61 | "colab": { 62 | "base_uri": "https://localhost:8080/" 63 | }, 64 | "executionInfo": { 65 | "elapsed": 2602, 66 | "status": "ok", 67 | "timestamp": 1703326576880, 68 | "user": { 69 | "displayName": "Markus Enzweiler", 70 | "userId": "04524044579212347608" 71 | }, 72 | "user_tz": -60 73 | }, 74 | "id": "DY4880S378_F", 75 | "outputId": "a8ce2d26-fe24-4aa2-c232-acc31b71d394" 76 | }, 77 | "outputs": [], 78 | "source": [ 79 | "# Package Path (folder of this notebook)\n", 80 | "\n", 81 | "#####################\n", 82 | "# Local environment #\n", 83 | "#####################\n", 84 | "\n", 85 | "package_path = \"./\"\n", 86 | "\n", 87 | "\n", 88 | "#########\n", 89 | "# Colab #\n", 90 | "#########\n", 91 | "\n", 92 | "\n", 93 | "def check_for_colab():\n", 94 | " try:\n", 95 | " import google.colab\n", 96 | "\n", 97 | " return True\n", 98 | " except ImportError:\n", 99 | " return False\n", 100 | "\n", 101 | "\n", 102 | "# running on Colab?\n", 103 | "on_colab = check_for_colab()\n", 104 | "\n", 105 | "if on_colab:\n", 106 | " # assume this notebook is run from Google Drive and the whole\n", 107 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 108 | "\n", 109 | " # Google Drive mount point\n", 110 | " gdrive_mnt = \"/content/drive\"\n", 111 | "\n", 112 | " ##########################################################################\n", 113 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 114 | " ##########################################################################\n", 115 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 116 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 117 | "\n", 118 | " # mount drive\n", 119 | " from google.colab import drive\n", 120 | "\n", 121 | " drive.mount(gdrive_mnt, force_remount=True)\n", 122 | "\n", 123 | " # set package path\n", 124 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 125 | "\n", 126 | "# check whether package path exists\n", 127 | "if not os.path.isdir(package_path):\n", 128 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 129 | "\n", 130 | "print(f\"Package path: {package_path}\")" 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": null, 136 | "metadata": {}, 137 | "outputs": [], 138 | "source": [ 139 | "# Additional imports\n", 140 | "\n", 141 | "# Repository Root\n", 142 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 143 | "# Add the repository root to the system path\n", 144 | "if repo_root not in sys.path:\n", 145 | " sys.path.append(repo_root)\n", 146 | "\n", 147 | "# Package Imports\n", 148 | "from nbutils import requirements as nb_reqs\n", 149 | "from nbutils import colab as nb_colab\n", 150 | "from nbutils import git as nb_git\n", 151 | "from nbutils import exec as nb_exec\n", 152 | "from nbutils import data as nb_data" 153 | ] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "execution_count": null, 158 | "metadata": { 159 | "colab": { 160 | "base_uri": "https://localhost:8080/" 161 | }, 162 | "executionInfo": { 163 | "elapsed": 6888, 164 | "status": "ok", 165 | "timestamp": 1703326583765, 166 | "user": { 167 | "displayName": "Markus Enzweiler", 168 | "userId": "04524044579212347608" 169 | }, 170 | "user_tz": -60 171 | }, 172 | "id": "iaURjw5n6pLq", 173 | "outputId": "5c40ee52-1fa3-44df-a94e-b81bec144100" 174 | }, 175 | "outputs": [], 176 | "source": [ 177 | "# Additional requirements for this notebook\n", 178 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 179 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": null, 185 | "metadata": { 186 | "executionInfo": { 187 | "elapsed": 12, 188 | "status": "ok", 189 | "timestamp": 1703326583765, 190 | "user": { 191 | "displayName": "Markus Enzweiler", 192 | "userId": "04524044579212347608" 193 | }, 194 | "user_tz": -60 195 | }, 196 | "id": "iRERDI8aAnzr" 197 | }, 198 | "outputs": [], 199 | "source": [ 200 | "# Now we should be able to import the additional packages\n", 201 | "import mlx\n", 202 | "import mlx.core as mx" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": { 208 | "id": "_aI-pchqASeB" 209 | }, 210 | "source": [ 211 | "## MLX\n", 212 | "\n", 213 | "MLX provides composable function transformations, supporting automatic differentiation, automatic vectorization, and optimization of computation graphs. Computation graphs within MLX are dynamically constructed. A key feature of MLX is the use of (and optimization for) unified memory present in the Apple SoCs. \n", 214 | "\n", 215 | "See:\n", 216 | "- https://github.com/ml-explore/mlx\n", 217 | "- https://ml-explore.github.io/mlx/build/html/quick_start.html" 218 | ] 219 | }, 220 | { 221 | "cell_type": "markdown", 222 | "metadata": { 223 | "id": "aUACL9QoFTXg" 224 | }, 225 | "source": [ 226 | "### Automatic differentiation with scalar functions in MLX" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "metadata": {}, 233 | "outputs": [], 234 | "source": [ 235 | "def f(x):\n", 236 | " return x**2 + 3 * x + 2\n", 237 | "\n", 238 | "\n", 239 | "x_tensor = mx.arange(-5, 5, 1, dtype=mx.float32)\n", 240 | "print(mx.grad(f)(x_tensor[0]))" 241 | ] 242 | }, 243 | { 244 | "cell_type": "code", 245 | "execution_count": null, 246 | "metadata": { 247 | "colab": { 248 | "base_uri": "https://localhost:8080/" 249 | }, 250 | "executionInfo": { 251 | "elapsed": 11, 252 | "status": "ok", 253 | "timestamp": 1703326583765, 254 | "user": { 255 | "displayName": "Markus Enzweiler", 256 | "userId": "04524044579212347608" 257 | }, 258 | "user_tz": -60 259 | }, 260 | "id": "lgoO6NHFAlcW", 261 | "outputId": "acd5d89c-aab0-4392-decd-3740da08d524" 262 | }, 263 | "outputs": [], 264 | "source": [ 265 | "# MLX has mlx.core.grad to automatically compute gradients of a function.\n", 266 | "# See: https://ml-explore.github.io/mlx/build/html/python/_autosummary/mlx.core.grad.html#mlx.core.grad\n", 267 | "\n", 268 | "# Let's try it out with simple functions first.\n", 269 | "\n", 270 | "\n", 271 | "# Define the function, x^2+3x+2\n", 272 | "def f(x):\n", 273 | " return x**2 + 3 * x + 2\n", 274 | "\n", 275 | "\n", 276 | "# Manual gradient w.r.t x\n", 277 | "def f_grad(x):\n", 278 | " return 2 * x + 3\n", 279 | "\n", 280 | "\n", 281 | "def mlxgrad(func, x):\n", 282 | " # Initialize an empty list for gradients\n", 283 | " gradients = []\n", 284 | "\n", 285 | " # Compute the gradient for each element in the tensor\n", 286 | " for xi in x:\n", 287 | " # Compute the function on the i-th element\n", 288 | " y = func(xi)\n", 289 | "\n", 290 | " # Compute the gradient for the i-th element\n", 291 | " gradients.append(mlx.core.grad(func)(xi))\n", 292 | "\n", 293 | " # Similar to PyTorch autograd, the mlx.core.grad function is designed to compute gradients of scalar\n", 294 | " # outputs with respect to inputs.\n", 295 | "\n", 296 | " # In our case, the function f(x) applied to x_tensor results in a vector (a tensor with multiple elements),\n", 297 | " # not a single scalar. Hence, mlx.core.grad cannot directly compute the gradient for each element\n", 298 | " # of this vector. To resolve this, we loop over each element of x_tensor, treating each function evaluation\n", 299 | " # f(x[i]) as a scalar output, and compute its gradient individually. This way, we are effectively computing\n", 300 | " # the gradient of multiple scalar functions, each dependent on a single element of x_tensor.\n", 301 | "\n", 302 | " return gradients\n", 303 | "\n", 304 | "\n", 305 | "# Compute some function values and gradients\n", 306 | "# make sure to set requires_grad=True to enable gradient tracking on the computational graph\n", 307 | "x_tensor = mx.arange(-5, 5, 1, dtype=mx.float32)\n", 308 | "\n", 309 | "f_value = f(x_tensor)\n", 310 | "f_grad = f_grad(x_tensor)\n", 311 | "f_autograd = mlxgrad(f, x_tensor)\n", 312 | "\n", 313 | "for i in range(len(x_tensor)):\n", 314 | " print(\n", 315 | " f\"x = {x_tensor[i].item():5.2f}: \"\n", 316 | " f\"f(x) = {f_value[i].item():5.2f}, \"\n", 317 | " f\"f_grad(x) = {f_grad[i].item():5.2f}, \"\n", 318 | " f\"autograd(x) = {f_autograd[i].item():5.2f}\"\n", 319 | " )" 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "metadata": { 325 | "id": "Cv03sygVFXEQ" 326 | }, 327 | "source": [ 328 | "### Automatic differentiation with tensors in MLX" 329 | ] 330 | }, 331 | { 332 | "cell_type": "code", 333 | "execution_count": null, 334 | "metadata": { 335 | "colab": { 336 | "base_uri": "https://localhost:8080/" 337 | }, 338 | "executionInfo": { 339 | "elapsed": 9, 340 | "status": "ok", 341 | "timestamp": 1703326583765, 342 | "user": { 343 | "displayName": "Markus Enzweiler", 344 | "userId": "04524044579212347608" 345 | }, 346 | "user_tz": -60 347 | }, 348 | "id": "E4PBVyEiFYjV", 349 | "outputId": "75cf9c1f-9a7d-4b3d-e481-2aecf6dcebf5" 350 | }, 351 | "outputs": [], 352 | "source": [ 353 | "# Define two tensors and track computations\n", 354 | "t1 = mx.array([[1, 2, 3], [4, 5, 6]], dtype=mx.float32)\n", 355 | "\n", 356 | "t2 = mx.array([[7, 8, 9], [10, 11, 12]], dtype=mx.float32)" 357 | ] 358 | }, 359 | { 360 | "cell_type": "code", 361 | "execution_count": null, 362 | "metadata": { 363 | "colab": { 364 | "base_uri": "https://localhost:8080/" 365 | }, 366 | "executionInfo": { 367 | "elapsed": 7, 368 | "status": "ok", 369 | "timestamp": 1703326583765, 370 | "user": { 371 | "displayName": "Markus Enzweiler", 372 | "userId": "04524044579212347608" 373 | }, 374 | "user_tz": -60 375 | }, 376 | "id": "K6XvVawsGuz7", 377 | "outputId": "2bc21f21-24de-488e-cbaa-02efe744fa3c" 378 | }, 379 | "outputs": [], 380 | "source": [ 381 | "# Perform element-wise multiplication of t1 and t2\n", 382 | "t1_mul_t2 = t1 * t2\n", 383 | "\n", 384 | "\n", 385 | "def mul_and_reduce_sum(t1, t2):\n", 386 | " return mx.sum(t1 * t2)" 387 | ] 388 | }, 389 | { 390 | "cell_type": "markdown", 391 | "metadata": { 392 | "id": "ZZ56e4tALbuL" 393 | }, 394 | "source": [ 395 | "After `backward()`, `t1.grad` and `t2.grad` are populated.\n", 396 | "\n", 397 | "The gradient of each element of `t1` is equal to the corresponding element in `t2`, and vice versa. This is because the derivative of `t1[i] * t2[i]` w.r.t. `t1[i]` is `t2[i]`, and w.r.t. `t2[i]` is `t1[i]`." 398 | ] 399 | }, 400 | { 401 | "cell_type": "code", 402 | "execution_count": null, 403 | "metadata": { 404 | "colab": { 405 | "base_uri": "https://localhost:8080/" 406 | }, 407 | "executionInfo": { 408 | "elapsed": 7, 409 | "status": "ok", 410 | "timestamp": 1703326583766, 411 | "user": { 412 | "displayName": "Markus Enzweiler", 413 | "userId": "04524044579212347608" 414 | }, 415 | "user_tz": -60 416 | }, 417 | "id": "tFqFLFpHHO8l", 418 | "outputId": "14c57946-85fb-450f-d07d-e9874ccbeab4" 419 | }, 420 | "outputs": [], 421 | "source": [ 422 | "# Compute gradients of the sum of all elements in t1_mul_t2 with respect to t1 and t2\n", 423 | "\n", 424 | "lvalue, t1_grad = mx.value_and_grad(mul_and_reduce_sum)(t1, t2)\n", 425 | "lvalue, t2_grad = mx.value_and_grad(mul_and_reduce_sum)(t2, t1)\n", 426 | "\n", 427 | "\n", 428 | "# The gradient at each element in t1 and t2 indicates the rate of change of the sum with respect to that element.\n", 429 | "# For element-wise multiplication, the gradient at each element of t1 is equal to the corresponding element\n", 430 | "# in t2 and vice versa. This is because the derivative of t1[i] * t2[i] w.r.t. t1[i] is t2[i],\n", 431 | "# and w.r.t. t2[i] is t1[i].\n", 432 | "\n", 433 | "print(f\"t1_grad = {t1_grad}\")\n", 434 | "print(f\"t2_grad = {t2_grad}\")" 435 | ] 436 | }, 437 | { 438 | "cell_type": "code", 439 | "execution_count": null, 440 | "metadata": { 441 | "colab": { 442 | "base_uri": "https://localhost:8080/" 443 | }, 444 | "executionInfo": { 445 | "elapsed": 244, 446 | "status": "ok", 447 | "timestamp": 1703327263018, 448 | "user": { 449 | "displayName": "Markus Enzweiler", 450 | "userId": "04524044579212347608" 451 | }, 452 | "user_tz": -60 453 | }, 454 | "id": "mceG39AiMCuM", 455 | "outputId": "d40802e9-5ee5-4fe8-e694-4f9ad3e6b045" 456 | }, 457 | "outputs": [], 458 | "source": [ 459 | "# Analyzing the gradient at t2[0,1]. If t2_grad[0,1] is 2, it means that a unit change in t2[0,1] results in a\n", 460 | "# change of 2 in the sum. Therefore, increasing t2[0,1] by 3 should increase the sum by 3 * t2_grad[0,1], under\n", 461 | "# linear approximation.\n", 462 | "\n", 463 | "# Create a new tensor and add 3 to t2[0,1]\n", 464 | "t2_modified = t2\n", 465 | "t2_modified[0, 1] = t2[0, 1] + 3\n", 466 | "\n", 467 | "# Perform the computation again with the modified t2\n", 468 | "t1_mul_t2_updated = t1 * t2_modified\n", 469 | "updated_sum = t1_mul_t2_updated.sum()\n", 470 | "\n", 471 | "# Compare the change in sum\n", 472 | "change_in_sum = updated_sum - t1_mul_t2.sum()\n", 473 | "print(f\"Change in sum: {change_in_sum}\")" 474 | ] 475 | } 476 | ], 477 | "metadata": { 478 | "colab": { 479 | "collapsed_sections": [ 480 | "JcF1mpJo-taz" 481 | ], 482 | "provenance": [] 483 | }, 484 | "kernelspec": { 485 | "display_name": "mlx-m1-2023-12", 486 | "language": "python", 487 | "name": "python3" 488 | }, 489 | "language_info": { 490 | "codemirror_mode": { 491 | "name": "ipython", 492 | "version": 3 493 | }, 494 | "file_extension": ".py", 495 | "mimetype": "text/x-python", 496 | "name": "python", 497 | "nbconvert_exporter": "python", 498 | "pygments_lexer": "ipython3", 499 | "version": "3.11.6" 500 | } 501 | }, 502 | "nbformat": 4, 503 | "nbformat_minor": 0 504 | } 505 | -------------------------------------------------------------------------------- /autograd/mlx/requirements.txt: -------------------------------------------------------------------------------- 1 | mlx -------------------------------------------------------------------------------- /autograd/torch/requirements.txt: -------------------------------------------------------------------------------- 1 | torch -------------------------------------------------------------------------------- /cnn/cifar/mlx/cnn_cifar_mlx.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Convolutional neural networks (CNN) for CIFAR-10/100 using MLX\n", 8 | "\n", 9 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 10 | "\n", 11 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "We build and train a CNN for CIFAR-10 / CIFAR-100 image classification, see https://www.cs.toronto.edu/~kriz/cifar.html. We use the Python code from https://github.com/menzHSE/mlx-cifar-10-cnn.git and execute it via this notebook. " 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "\n", 26 | "**Note: This requires a machine with an Apple SoC, e.g. M1/M2/M3 etc.**\n", 27 | "\n", 28 | "See: https://github.com/ml-explore/mlx" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "## Setup\n", 36 | "\n", 37 | "Adapt `packagePath` to point to the directory containing this notebeook." 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "# Notebook id\n", 47 | "nb_id = \"cnn/cifar/mlx\"\n", 48 | "\n", 49 | "# Imports\n", 50 | "import sys\n", 51 | "import os" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": null, 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "# Package Path (folder of this notebook)\n", 61 | "\n", 62 | "#####################\n", 63 | "# Local environment #\n", 64 | "#####################\n", 65 | "\n", 66 | "package_path = \"./\"\n", 67 | "\n", 68 | "\n", 69 | "#########\n", 70 | "# Colab #\n", 71 | "#########\n", 72 | "\n", 73 | "\n", 74 | "def check_for_colab():\n", 75 | " try:\n", 76 | " import google.colab\n", 77 | "\n", 78 | " return True\n", 79 | " except ImportError:\n", 80 | " return False\n", 81 | "\n", 82 | "\n", 83 | "# running on Colab?\n", 84 | "on_colab = check_for_colab()\n", 85 | "\n", 86 | "if on_colab:\n", 87 | " # assume this notebook is run from Google Drive and the whole\n", 88 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 89 | "\n", 90 | " # Google Drive mount point\n", 91 | " gdrive_mnt = \"/content/drive\"\n", 92 | "\n", 93 | " ##########################################################################\n", 94 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 95 | " ##########################################################################\n", 96 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 97 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 98 | "\n", 99 | " # mount drive\n", 100 | " from google.colab import drive\n", 101 | "\n", 102 | " drive.mount(gdrive_mnt, force_remount=True)\n", 103 | "\n", 104 | " # set package path\n", 105 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 106 | "\n", 107 | "# check whether package path exists\n", 108 | "if not os.path.isdir(package_path):\n", 109 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 110 | "\n", 111 | "print(f\"Package path: {package_path}\")" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": null, 117 | "metadata": {}, 118 | "outputs": [], 119 | "source": [ 120 | "# Additional imports\n", 121 | "\n", 122 | "# Repository Root\n", 123 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\", \"..\"))\n", 124 | "# Add the repository root to the system path\n", 125 | "if repo_root not in sys.path:\n", 126 | " sys.path.append(repo_root)\n", 127 | "\n", 128 | "# Package Imports\n", 129 | "from nbutils import requirements as nb_reqs\n", 130 | "from nbutils import colab as nb_colab\n", 131 | "from nbutils import git as nb_git\n", 132 | "from nbutils import exec as nb_exec\n", 133 | "from nbutils import data as nb_data" 134 | ] 135 | }, 136 | { 137 | "cell_type": "code", 138 | "execution_count": null, 139 | "metadata": {}, 140 | "outputs": [], 141 | "source": [ 142 | "# Clone git repository\n", 143 | "\n", 144 | "# Absolute path of the repository directory\n", 145 | "repo_dir = os.path.join(package_path, \"mlx-cifar-10-cnn\")\n", 146 | "repo_url = \"https://github.com/menzHSE/mlx-cifar-10-cnn.git\"\n", 147 | "\n", 148 | "nb_git.clone(repo_url, repo_dir, on_colab)" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": null, 154 | "metadata": {}, 155 | "outputs": [], 156 | "source": [ 157 | "# Install requirements in the current Jupyter kernel\n", 158 | "req_file = os.path.join(repo_dir, \"requirements.txt\")\n", 159 | "nb_reqs.pip_install_reqs(req_file, on_colab)\n", 160 | "\n", 161 | "# Additional requirements for this notebook\n", 162 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 163 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 164 | ] 165 | }, 166 | { 167 | "cell_type": "markdown", 168 | "metadata": {}, 169 | "source": [ 170 | "# Train and test CNN on CIFAR-10\n", 171 | "\n" 172 | ] 173 | }, 174 | { 175 | "cell_type": "code", 176 | "execution_count": null, 177 | "metadata": {}, 178 | "outputs": [], 179 | "source": [ 180 | "# Let's see what we can do with train.py\n", 181 | "nb_exec.executePythonScript(\n", 182 | " os.path.join(repo_dir, \"train.py\"), {\"help\": None}, on_colab\n", 183 | ")" 184 | ] 185 | }, 186 | { 187 | "cell_type": "markdown", 188 | "metadata": {}, 189 | "source": [ 190 | "## Parameters" 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": null, 196 | "metadata": {}, 197 | "outputs": [], 198 | "source": [ 199 | "# parameters\n", 200 | "batchsize = 32\n", 201 | "seed = 42\n", 202 | "lr = 3e-4\n", 203 | "epochs = 30\n", 204 | "dataset = \"CIFAR-10\"" 205 | ] 206 | }, 207 | { 208 | "cell_type": "markdown", 209 | "metadata": {}, 210 | "source": [ 211 | "## Train" 212 | ] 213 | }, 214 | { 215 | "cell_type": "code", 216 | "execution_count": null, 217 | "metadata": {}, 218 | "outputs": [], 219 | "source": [ 220 | "params = {\n", 221 | " \"dataset\": dataset, # dataset name\n", 222 | " \"batchsize\": batchsize, # batch size\n", 223 | " \"seed\": seed, # random seed\n", 224 | " \"lr\": lr, # learning rate\n", 225 | " \"epochs\": epochs, # number of epochs\n", 226 | "}\n", 227 | "\n", 228 | "# Execute 'train.py' with parameters\n", 229 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"train.py\"), params, on_colab)" 230 | ] 231 | }, 232 | { 233 | "cell_type": "markdown", 234 | "metadata": {}, 235 | "source": [ 236 | "## Test " 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "execution_count": null, 242 | "metadata": {}, 243 | "outputs": [], 244 | "source": [ 245 | "# parameters\n", 246 | "params = {\"model\": f\"models/model_{dataset}_{epochs-1:03d}.npz\"} # model name\n", 247 | "\n", 248 | "# Execute 'test.py' with parameters\n", 249 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"test.py\"), params, on_colab)" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": {}, 255 | "source": [ 256 | "# Train and test CNN on CIFAR-100" 257 | ] 258 | }, 259 | { 260 | "cell_type": "markdown", 261 | "metadata": {}, 262 | "source": [ 263 | "## Parameters" 264 | ] 265 | }, 266 | { 267 | "cell_type": "code", 268 | "execution_count": null, 269 | "metadata": {}, 270 | "outputs": [], 271 | "source": [ 272 | "# parameters\n", 273 | "batchsize = 32\n", 274 | "seed = 42\n", 275 | "lr = 3e-4\n", 276 | "epochs = 30\n", 277 | "dataset = \"CIFAR-100\"" 278 | ] 279 | }, 280 | { 281 | "cell_type": "markdown", 282 | "metadata": {}, 283 | "source": [ 284 | "## Train" 285 | ] 286 | }, 287 | { 288 | "cell_type": "code", 289 | "execution_count": null, 290 | "metadata": {}, 291 | "outputs": [], 292 | "source": [ 293 | "params = {\n", 294 | " \"dataset\": dataset, # dataset name\n", 295 | " \"batchsize\": batchsize, # batch size\n", 296 | " \"seed\": seed, # random seed\n", 297 | " \"lr\": lr, # learning rate\n", 298 | " \"epochs\": epochs, # number of epochs\n", 299 | "}\n", 300 | "\n", 301 | "# Execute 'train.py' with parameters\n", 302 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"train.py\"), params, on_colab)" 303 | ] 304 | }, 305 | { 306 | "cell_type": "markdown", 307 | "metadata": {}, 308 | "source": [ 309 | "## Test" 310 | ] 311 | }, 312 | { 313 | "cell_type": "code", 314 | "execution_count": null, 315 | "metadata": {}, 316 | "outputs": [], 317 | "source": [ 318 | "# parameters\n", 319 | "params = {\n", 320 | " \"dataset\": dataset, # dataset name\n", 321 | " \"model\": f\"models/model_{dataset}_{epochs-1:03d}.npz\", # model name\n", 322 | "}\n", 323 | "\n", 324 | "# Execute 'test.py' with parameters\n", 325 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"test.py\"), params, on_colab)" 326 | ] 327 | } 328 | ], 329 | "metadata": { 330 | "kernelspec": { 331 | "display_name": "mlx-m1-2023-12", 332 | "language": "python", 333 | "name": "python3" 334 | }, 335 | "language_info": { 336 | "codemirror_mode": { 337 | "name": "ipython", 338 | "version": 3 339 | }, 340 | "file_extension": ".py", 341 | "mimetype": "text/x-python", 342 | "name": "python", 343 | "nbconvert_exporter": "python", 344 | "pygments_lexer": "ipython3", 345 | "version": "3.11.6" 346 | } 347 | }, 348 | "nbformat": 4, 349 | "nbformat_minor": 2 350 | } 351 | -------------------------------------------------------------------------------- /cnn/cifar/torch/cnn_cifar_torch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Convolutional neural networks (CNN) for CIFAR-10/100 using PyTorch\n", 8 | "\n", 9 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 10 | "\n", 11 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "We build and train a CNN for CIFAR-10 / CIFAR-100 image classification, see https://www.cs.toronto.edu/~kriz/cifar.html. We use the Python code from https://github.com/menzHSE/torch-cifar-10-cnn.git and execute it via this notebook. " 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "## Setup\n", 26 | "\n", 27 | "Adapt `packagePath` to point to the directory containing this notebeook." 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": null, 33 | "metadata": {}, 34 | "outputs": [], 35 | "source": [ 36 | "# Notebook id\n", 37 | "nb_id = \"cnn/cifar/torch\"\n", 38 | "\n", 39 | "# Imports\n", 40 | "import sys\n", 41 | "import os" 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": null, 47 | "metadata": {}, 48 | "outputs": [], 49 | "source": [ 50 | "# Package Path (folder of this notebook)\n", 51 | "\n", 52 | "#####################\n", 53 | "# Local environment #\n", 54 | "#####################\n", 55 | "\n", 56 | "package_path = \"./\"\n", 57 | "\n", 58 | "\n", 59 | "#########\n", 60 | "# Colab #\n", 61 | "#########\n", 62 | "\n", 63 | "\n", 64 | "def check_for_colab():\n", 65 | " try:\n", 66 | " import google.colab\n", 67 | "\n", 68 | " return True\n", 69 | " except ImportError:\n", 70 | " return False\n", 71 | "\n", 72 | "\n", 73 | "# running on Colab?\n", 74 | "on_colab = check_for_colab()\n", 75 | "\n", 76 | "if on_colab:\n", 77 | " # assume this notebook is run from Google Drive and the whole\n", 78 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 79 | "\n", 80 | " # Google Drive mount point\n", 81 | " gdrive_mnt = \"/content/drive\"\n", 82 | "\n", 83 | " ##########################################################################\n", 84 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 85 | " ##########################################################################\n", 86 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 87 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 88 | "\n", 89 | " # mount drive\n", 90 | " from google.colab import drive\n", 91 | "\n", 92 | " drive.mount(gdrive_mnt, force_remount=True)\n", 93 | "\n", 94 | " # set package path\n", 95 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 96 | "\n", 97 | "# check whether package path exists\n", 98 | "if not os.path.isdir(package_path):\n", 99 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 100 | "\n", 101 | "print(f\"Package path: {package_path}\")" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": {}, 108 | "outputs": [], 109 | "source": [ 110 | "# Additional imports\n", 111 | "\n", 112 | "# Repository Root\n", 113 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\", \"..\"))\n", 114 | "# Add the repository root to the system path\n", 115 | "if repo_root not in sys.path:\n", 116 | " sys.path.append(repo_root)\n", 117 | "\n", 118 | "# Package Imports\n", 119 | "from nbutils import requirements as nb_reqs\n", 120 | "from nbutils import colab as nb_colab\n", 121 | "from nbutils import git as nb_git\n", 122 | "from nbutils import exec as nb_exec\n", 123 | "from nbutils import data as nb_data" 124 | ] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "execution_count": null, 129 | "metadata": {}, 130 | "outputs": [], 131 | "source": [ 132 | "# Clone git repository\n", 133 | "\n", 134 | "# Absolute path of the repository directory\n", 135 | "repo_dir = os.path.join(package_path, \"torch-cifar-10-cnn\")\n", 136 | "repo_url = \"https://github.com/menzHSE/torch-cifar-10-cnn.git\"\n", 137 | "\n", 138 | "nb_git.clone(repo_url, repo_dir, on_colab)" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": null, 144 | "metadata": {}, 145 | "outputs": [], 146 | "source": [ 147 | "# Install requirements in the current Jupyter kernel\n", 148 | "req_file = os.path.join(repo_dir, \"requirements.txt\")\n", 149 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "metadata": {}, 155 | "source": [ 156 | "# Train and test CNN on CIFAR-10\n", 157 | "\n" 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": null, 163 | "metadata": {}, 164 | "outputs": [], 165 | "source": [ 166 | "# Let's see what we can do with train.py\n", 167 | "nb_exec.executePythonScript(\n", 168 | " os.path.join(repo_dir, \"train.py\"), {\"help\": None}, on_colab\n", 169 | ")" 170 | ] 171 | }, 172 | { 173 | "cell_type": "markdown", 174 | "metadata": {}, 175 | "source": [ 176 | "## Parameters" 177 | ] 178 | }, 179 | { 180 | "cell_type": "code", 181 | "execution_count": null, 182 | "metadata": {}, 183 | "outputs": [], 184 | "source": [ 185 | "# parameters\n", 186 | "batchsize = 32\n", 187 | "seed = 42\n", 188 | "lr = 3e-4\n", 189 | "epochs = 30\n", 190 | "dataset = \"CIFAR-10\"" 191 | ] 192 | }, 193 | { 194 | "cell_type": "markdown", 195 | "metadata": {}, 196 | "source": [ 197 | "## Train" 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": null, 203 | "metadata": {}, 204 | "outputs": [], 205 | "source": [ 206 | "params = {\n", 207 | " \"dataset\": dataset, # dataset name\n", 208 | " \"batchsize\": batchsize, # batch size\n", 209 | " \"seed\": seed, # random seed\n", 210 | " \"lr\": lr, # learning rate\n", 211 | " \"epochs\": epochs, # number of epochs\n", 212 | "}\n", 213 | "\n", 214 | "# Execute 'train.py' with parameters\n", 215 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"train.py\"), params, on_colab)" 216 | ] 217 | }, 218 | { 219 | "cell_type": "markdown", 220 | "metadata": {}, 221 | "source": [ 222 | "## Test " 223 | ] 224 | }, 225 | { 226 | "cell_type": "code", 227 | "execution_count": null, 228 | "metadata": {}, 229 | "outputs": [], 230 | "source": [ 231 | "# parameters\n", 232 | "params = {\"model\": f\"models/model_{dataset}_{epochs-1:03d}.pth\"} # model name\n", 233 | "\n", 234 | "# Execute 'test.py' with parameters\n", 235 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"test.py\"), params, on_colab)" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "# Train and test CNN on CIFAR-100" 243 | ] 244 | }, 245 | { 246 | "cell_type": "markdown", 247 | "metadata": {}, 248 | "source": [ 249 | "## Parameters" 250 | ] 251 | }, 252 | { 253 | "cell_type": "code", 254 | "execution_count": null, 255 | "metadata": {}, 256 | "outputs": [], 257 | "source": [ 258 | "# parameters\n", 259 | "batchsize = 32\n", 260 | "seed = 42\n", 261 | "lr = 3e-4\n", 262 | "epochs = 30\n", 263 | "dataset = \"CIFAR-100\"" 264 | ] 265 | }, 266 | { 267 | "cell_type": "markdown", 268 | "metadata": {}, 269 | "source": [ 270 | "## Train" 271 | ] 272 | }, 273 | { 274 | "cell_type": "code", 275 | "execution_count": null, 276 | "metadata": {}, 277 | "outputs": [], 278 | "source": [ 279 | "params = {\n", 280 | " \"dataset\": dataset, # dataset name\n", 281 | " \"batchsize\": batchsize, # batch size\n", 282 | " \"seed\": seed, # random seed\n", 283 | " \"lr\": lr, # learning rate\n", 284 | " \"epochs\": epochs, # number of epochs\n", 285 | "}\n", 286 | "\n", 287 | "# Execute 'train.py' with parameters\n", 288 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"train.py\"), params, on_colab)" 289 | ] 290 | }, 291 | { 292 | "cell_type": "markdown", 293 | "metadata": {}, 294 | "source": [ 295 | "## Test" 296 | ] 297 | }, 298 | { 299 | "cell_type": "code", 300 | "execution_count": null, 301 | "metadata": {}, 302 | "outputs": [], 303 | "source": [ 304 | "# parameters\n", 305 | "params = {\n", 306 | " \"dataset\": dataset, # dataset name\n", 307 | " \"model\": f\"models/model_{dataset}_{epochs-1:03d}.pth\", # model name\n", 308 | "}\n", 309 | "\n", 310 | "# Execute 'test.py' with parameters\n", 311 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"test.py\"), params, on_colab)" 312 | ] 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "metadata": {}, 317 | "source": [ 318 | "# Finetune a ResNet on CIFAR-100" 319 | ] 320 | }, 321 | { 322 | "cell_type": "markdown", 323 | "metadata": {}, 324 | "source": [ 325 | "## Parameters" 326 | ] 327 | }, 328 | { 329 | "cell_type": "code", 330 | "execution_count": null, 331 | "metadata": {}, 332 | "outputs": [], 333 | "source": [ 334 | "# parameters\n", 335 | "batchsize = 32\n", 336 | "seed = 42\n", 337 | "lr = 3e-4\n", 338 | "epochs = 5\n", 339 | "dataset = \"CIFAR-100\"\n", 340 | "finetune = \"resnet18\"" 341 | ] 342 | }, 343 | { 344 | "cell_type": "markdown", 345 | "metadata": {}, 346 | "source": [ 347 | "## Train" 348 | ] 349 | }, 350 | { 351 | "cell_type": "code", 352 | "execution_count": null, 353 | "metadata": {}, 354 | "outputs": [], 355 | "source": [ 356 | "params = {\n", 357 | " \"dataset\": dataset, # dataset name\n", 358 | " \"batchsize\": batchsize, # batch size\n", 359 | " \"seed\": seed, # random seed\n", 360 | " \"lr\": lr, # learning rate\n", 361 | " \"epochs\": epochs, # number of epochs\n", 362 | " \"finetune\": finetune, # finetune model\n", 363 | "}\n", 364 | "\n", 365 | "# Execute 'train.py' with parameters\n", 366 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"train.py\"), params, on_colab)" 367 | ] 368 | }, 369 | { 370 | "cell_type": "markdown", 371 | "metadata": {}, 372 | "source": [ 373 | "## Test" 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "execution_count": null, 379 | "metadata": {}, 380 | "outputs": [], 381 | "source": [ 382 | "# parameters\n", 383 | "params = {\n", 384 | " \"dataset\": dataset, # dataset name\n", 385 | " \"model\": f\"models/model_{finetune}_{dataset}_{epochs-1:03d}.pth\", # model name\n", 386 | " \"finetune\": finetune, # finetune model\n", 387 | "}\n", 388 | "\n", 389 | "# Execute 'test.py' with parameters\n", 390 | "nb_exec.executePythonScript(os.path.join(repo_dir, \"test.py\"), params, on_colab)" 391 | ] 392 | } 393 | ], 394 | "metadata": { 395 | "kernelspec": { 396 | "display_name": "pytorch-m1-2023-10", 397 | "language": "python", 398 | "name": "pytorch-m1-2023-10" 399 | }, 400 | "language_info": { 401 | "codemirror_mode": { 402 | "name": "ipython", 403 | "version": 3 404 | }, 405 | "file_extension": ".py", 406 | "mimetype": "text/x-python", 407 | "name": "python", 408 | "nbconvert_exporter": "python", 409 | "pygments_lexer": "ipython3", 410 | "version": "3.9.18" 411 | } 412 | }, 413 | "nbformat": 4, 414 | "nbformat_minor": 2 415 | } 416 | -------------------------------------------------------------------------------- /cnn/cifar/torch/cnn_cifar_torch_analysis.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Convolutional neural networks (CNN) for CIFAR-10/100 using PyTorch\n", 8 | "\n", 9 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 10 | "\n", 11 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "We test a pretrained CNN for CIFAR-10 / CIFAR-100 image classification, see https://www.cs.toronto.edu/~kriz/cifar.html. We use the Python code from https://github.com/menzHSE/torch-cifar-10-cnn.git." 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "## Setup\n", 26 | "\n", 27 | "Adapt `packagePath` to point to the directory containing this notebeook." 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": null, 33 | "metadata": {}, 34 | "outputs": [], 35 | "source": [ 36 | "# Notebook id\n", 37 | "nb_id = \"cnn/cifar/torch\"\n", 38 | "\n", 39 | "# Imports\n", 40 | "import sys\n", 41 | "import os" 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": null, 47 | "metadata": {}, 48 | "outputs": [], 49 | "source": [ 50 | "# Package Path (folder of this notebook)\n", 51 | "\n", 52 | "#####################\n", 53 | "# Local environment #\n", 54 | "#####################\n", 55 | "\n", 56 | "package_path = \"./\"\n", 57 | "\n", 58 | "\n", 59 | "#########\n", 60 | "# Colab #\n", 61 | "#########\n", 62 | "\n", 63 | "\n", 64 | "def check_for_colab():\n", 65 | " try:\n", 66 | " import google.colab\n", 67 | "\n", 68 | " return True\n", 69 | " except ImportError:\n", 70 | " return False\n", 71 | "\n", 72 | "\n", 73 | "# running on Colab?\n", 74 | "on_colab = check_for_colab()\n", 75 | "\n", 76 | "if on_colab:\n", 77 | " # assume this notebook is run from Google Drive and the whole\n", 78 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 79 | "\n", 80 | " # Google Drive mount point\n", 81 | " gdrive_mnt = \"/content/drive\"\n", 82 | "\n", 83 | " ##########################################################################\n", 84 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 85 | " ##########################################################################\n", 86 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 87 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 88 | "\n", 89 | " # mount drive\n", 90 | " from google.colab import drive\n", 91 | "\n", 92 | " drive.mount(gdrive_mnt, force_remount=True)\n", 93 | "\n", 94 | " # set package path\n", 95 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 96 | "\n", 97 | "# check whether package path exists\n", 98 | "if not os.path.isdir(package_path):\n", 99 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 100 | "\n", 101 | "print(f\"Package path: {package_path}\")" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": {}, 108 | "outputs": [], 109 | "source": [ 110 | "# Additional imports\n", 111 | "\n", 112 | "# Repository Root\n", 113 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\", \"..\"))\n", 114 | "# Add the repository root to the system path\n", 115 | "if repo_root not in sys.path:\n", 116 | " sys.path.append(repo_root)\n", 117 | "\n", 118 | "# Package Imports\n", 119 | "from nbutils import requirements as nb_reqs\n", 120 | "from nbutils import colab as nb_colab\n", 121 | "from nbutils import git as nb_git\n", 122 | "from nbutils import exec as nb_exec\n", 123 | "from nbutils import data as nb_data" 124 | ] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "execution_count": null, 129 | "metadata": {}, 130 | "outputs": [], 131 | "source": [ 132 | "# Clone git repository\n", 133 | "\n", 134 | "# Absolute path of the repository directory\n", 135 | "repo_dir = os.path.join(package_path, \"torch-cifar-10-cnn\")\n", 136 | "repo_url = \"https://github.com/menzHSE/torch-cifar-10-cnn.git\"\n", 137 | "\n", 138 | "nb_git.clone(repo_url, repo_dir, on_colab)" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": null, 144 | "metadata": {}, 145 | "outputs": [], 146 | "source": [ 147 | "# Install requirements in the current Jupyter kernel\n", 148 | "req_file = os.path.join(repo_dir, \"requirements.txt\")\n", 149 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 150 | ] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": null, 155 | "metadata": {}, 156 | "outputs": [], 157 | "source": [ 158 | "# Add the directory containing models.py to the system path\n", 159 | "sys.path.append(os.path.join(package_path, \"torch-cifar-10-cnn\"))\n", 160 | "\n", 161 | "# Now we can import torch\n", 162 | "import torch\n", 163 | "\n", 164 | "# Now we can import the model and dataset\n", 165 | "import model\n", 166 | "import dataset\n", 167 | "import device" 168 | ] 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "metadata": {}, 173 | "source": [ 174 | "# Inference on CIFAR-10\n", 175 | "\n" 176 | ] 177 | }, 178 | { 179 | "cell_type": "markdown", 180 | "metadata": {}, 181 | "source": [ 182 | "## Load the pretrained model and the CIFAR-10 test data" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": null, 188 | "metadata": {}, 189 | "outputs": [], 190 | "source": [ 191 | "# device\n", 192 | "dev = device.autoselectDevice()\n", 193 | "print(f\"Device: {dev}\")\n", 194 | "\n", 195 | "# load the dataset\n", 196 | "cifar_version = \"CIFAR-10\"\n", 197 | "batch_size = 64\n", 198 | "img_size = 32 # we assume 32x32 pixel images\n", 199 | "_, test_loader, classes = dataset.cifar(\n", 200 | " batch_size=batch_size, custom_transforms=None, cifar_version=cifar_version\n", 201 | ")\n", 202 | "num_classes = len(classes)\n", 203 | "\n", 204 | "# load the model\n", 205 | "model_fname = os.path.join(\n", 206 | " package_path, \"torch-cifar-10-cnn\", \"pretrained_models\", \"model_CIFAR-10.pth\"\n", 207 | ")\n", 208 | "\n", 209 | "cnn = model.CNN(num_classes)\n", 210 | "cnn.load(model_fname, dev)\n", 211 | "\n", 212 | "print(f\"Loaded model for {cifar_version} from {model_fname}\")" 213 | ] 214 | }, 215 | { 216 | "cell_type": "markdown", 217 | "metadata": {}, 218 | "source": [ 219 | "## Run inference on a batch of test data" 220 | ] 221 | }, 222 | { 223 | "cell_type": "code", 224 | "execution_count": null, 225 | "metadata": {}, 226 | "outputs": [], 227 | "source": [ 228 | "import matplotlib.pyplot as plt\n", 229 | "import torch.nn.functional as F\n", 230 | "import torchvision\n", 231 | "\n", 232 | "# put the model in evaluation mode and on the device we are using\n", 233 | "cnn.eval()\n", 234 | "cnn.to(dev)\n", 235 | "\n", 236 | "# get a batch of testing images\n", 237 | "data = next(iter(test_loader))\n", 238 | "\n", 239 | "with torch.no_grad():\n", 240 | " # Get the testing data and push the data to the device we are using\n", 241 | " images, labels = data[0].to(dev), data[1].to(dev)\n", 242 | "\n", 243 | " # Get the model predictions\n", 244 | " predictions = cnn(images)\n", 245 | " _, predicted_labels = torch.max(predictions, 1)\n", 246 | " # CNN output posterior probability estimate via softmax\n", 247 | " probabilities = F.softmax(predictions, dim=1) # softmax along the rows\n", 248 | "\n", 249 | " # plot the images in the batch, along with the corresponding labels and predictions\n", 250 | "\n", 251 | " # Make a grid from the batch\n", 252 | " grid = torchvision.utils.make_grid(\n", 253 | " images.cpu(), nrow=8, normalize=True, scale_each=True\n", 254 | " )\n", 255 | "\n", 256 | " # Convert grid to a numpy image\n", 257 | " grid = grid.numpy().transpose((1, 2, 0))\n", 258 | "\n", 259 | " plt.figure(figsize=(15, 15))\n", 260 | " plt.imshow(grid)\n", 261 | " plt.axis(\"off\")\n", 262 | "\n", 263 | " # Add labels\n", 264 | " for i in range(batch_size):\n", 265 | " row = i // 8\n", 266 | " col = i % 8\n", 267 | "\n", 268 | " # Ground truth and predicted class\n", 269 | " gt_label = classes[labels[i]]\n", 270 | " pr_label = classes[predicted_labels[i]]\n", 271 | " output_prob = probabilities[i][predicted_labels[i]]\n", 272 | "\n", 273 | " if gt_label == pr_label:\n", 274 | " # Green label for correct predictions\n", 275 | " label_color = \"lightgreen\"\n", 276 | " else:\n", 277 | " # Red label for incorrect predictions\n", 278 | " label_color = \"red\"\n", 279 | "\n", 280 | " plt.text(\n", 281 | " col * (img_size + 2) + 2,\n", 282 | " row * (img_size + 2) + 4,\n", 283 | " f\"T: {gt_label}\",\n", 284 | " color=label_color,\n", 285 | " backgroundcolor=\"black\",\n", 286 | " )\n", 287 | "\n", 288 | " plt.text(\n", 289 | " col * (img_size + 2) + 2,\n", 290 | " row * (img_size + 2) + 10,\n", 291 | " f\"P: {pr_label} ({output_prob:.2f})\",\n", 292 | " color=label_color,\n", 293 | " backgroundcolor=\"black\",\n", 294 | " )\n", 295 | "\n", 296 | " plt.title(\"True (T) and Predicted Classes (P) with Confidence Value for CIFAR-10\")\n", 297 | " plt.show()" 298 | ] 299 | } 300 | ], 301 | "metadata": { 302 | "kernelspec": { 303 | "display_name": "pytorch-m1-2023-10", 304 | "language": "python", 305 | "name": "pytorch-m1-2023-10" 306 | }, 307 | "language_info": { 308 | "codemirror_mode": { 309 | "name": "ipython", 310 | "version": 3 311 | }, 312 | "file_extension": ".py", 313 | "mimetype": "text/x-python", 314 | "name": "python", 315 | "nbconvert_exporter": "python", 316 | "pygments_lexer": "ipython3", 317 | "version": "3.9.18" 318 | } 319 | }, 320 | "nbformat": 4, 321 | "nbformat_minor": 2 322 | } 323 | -------------------------------------------------------------------------------- /cnn/mnist/torch/requirements.txt: -------------------------------------------------------------------------------- 1 | torch 2 | torchvision 3 | torchinfo 4 | numpy 5 | Pillow 6 | 7 | -------------------------------------------------------------------------------- /gan/torch/gan_torch_celeba.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Convolutional Deep Generative Adversarial Networks (DCGAN) using PyTorch\n", 8 | "\n", 9 | "Author: [Markus Enzweiler](https://markus-enzweiler-de), markus.enzweiler@hs-esslingen.de\n", 10 | "\n", 11 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "**See `gan_torch_mnist.ipynb` for a more in-depth notebook on GANs.**" 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "## Setup\n", 26 | "\n", 27 | "Adapt `packagePath` to point to the directory containing this notebeook." 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": null, 33 | "metadata": {}, 34 | "outputs": [], 35 | "source": [ 36 | "# Notebook id\n", 37 | "nb_id = \"gan/torch\"\n", 38 | "\n", 39 | "# Imports\n", 40 | "import sys\n", 41 | "import os" 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": null, 47 | "metadata": {}, 48 | "outputs": [], 49 | "source": [ 50 | "# Package Path (folder of this notebook)\n", 51 | "\n", 52 | "#####################\n", 53 | "# Local environment #\n", 54 | "#####################\n", 55 | "\n", 56 | "package_path = \"./\"\n", 57 | "\n", 58 | "\n", 59 | "#########\n", 60 | "# Colab #\n", 61 | "#########\n", 62 | "\n", 63 | "\n", 64 | "def check_for_colab():\n", 65 | " try:\n", 66 | " import google.colab\n", 67 | "\n", 68 | " return True\n", 69 | " except ImportError:\n", 70 | " return False\n", 71 | "\n", 72 | "\n", 73 | "# running on Colab?\n", 74 | "on_colab = check_for_colab()\n", 75 | "\n", 76 | "if on_colab:\n", 77 | " # assume this notebook is run from Google Drive and the whole\n", 78 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 79 | "\n", 80 | " # Google Drive mount point\n", 81 | " gdrive_mnt = \"/content/drive\"\n", 82 | "\n", 83 | " ##########################################################################\n", 84 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 85 | " ##########################################################################\n", 86 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 87 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 88 | "\n", 89 | " # mount drive\n", 90 | " from google.colab import drive\n", 91 | "\n", 92 | " drive.mount(gdrive_mnt, force_remount=True)\n", 93 | "\n", 94 | " # set package path\n", 95 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 96 | "\n", 97 | "# check whether package path exists\n", 98 | "if not os.path.isdir(package_path):\n", 99 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 100 | "\n", 101 | "print(f\"Package path: {package_path}\")" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": {}, 108 | "outputs": [], 109 | "source": [ 110 | "# Additional imports\n", 111 | "\n", 112 | "# Repository Root\n", 113 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 114 | "# Add the repository root to the system path\n", 115 | "if repo_root not in sys.path:\n", 116 | " sys.path.append(repo_root)\n", 117 | "\n", 118 | "# Package Imports\n", 119 | "from nbutils import requirements as nb_reqs\n", 120 | "from nbutils import colab as nb_colab\n", 121 | "from nbutils import git as nb_git\n", 122 | "from nbutils import exec as nb_exec\n", 123 | "from nbutils import data as nb_data" 124 | ] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "execution_count": null, 129 | "metadata": {}, 130 | "outputs": [], 131 | "source": [ 132 | "# Clone git repository\n", 133 | "\n", 134 | "# Absolute path of the repository directory\n", 135 | "repo_dir = os.path.join(package_path, \"torch-gan\")\n", 136 | "repo_url = \"https://github.com/menzHSE/torch-gan.git\"\n", 137 | "\n", 138 | "nb_git.clone(repo_url, repo_dir, on_colab)" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": null, 144 | "metadata": {}, 145 | "outputs": [], 146 | "source": [ 147 | "# Install requirements in the current Jupyter kernel\n", 148 | "req_file = os.path.join(repo_dir, \"requirements.txt\")\n", 149 | "nb_reqs.pip_install_reqs(req_file, on_colab)\n", 150 | "\n", 151 | "# Additional requirements for this notebook\n", 152 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 153 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 154 | ] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": {}, 159 | "source": [ 160 | "# Sample CelebA faces\n", 161 | "\n", 162 | "If the dataset cannot be automatically downloaded by PyTorch due to **daily quota exceeded** you can manually download it and put it in the ```data/celaba``` folder, see code cell below.\n", 163 | "\n", 164 | "The following files are necessary:\n", 165 | "- img_align_celeba.zip\n", 166 | "- list_attr_celeba.txt\n", 167 | "- list_bbox_celeba.txt\n", 168 | "- list_eval_partition.txt\n", 169 | "- list_landmarks_align_celeba.txt\n", 170 | "- list_landmarks_celeba.txt" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": null, 176 | "metadata": {}, 177 | "outputs": [], 178 | "source": [ 179 | "# To get around the problem of \"Quota Exceeded\" on the \"official\" CelebA\n", 180 | "# download via torchvision (https://github.com/pytorch/vision/issues/1920),\n", 181 | "# we use an alternative data source.\n", 182 | "\n", 183 | "# Set this to True to download the dataset from an alternative source\n", 184 | "use_alternative_data_source = True\n", 185 | "\n", 186 | "if use_alternative_data_source:\n", 187 | " print(\"Dowloading CelebA ... this will take a while\")\n", 188 | " nb_data.download_celeba(\"./data/celeba\", on_colab, verbose=False)" 189 | ] 190 | }, 191 | { 192 | "cell_type": "markdown", 193 | "metadata": {}, 194 | "source": [ 195 | "## Load Model and Data" 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": null, 201 | "metadata": {}, 202 | "outputs": [], 203 | "source": [ 204 | "import torch\n", 205 | "import torchvision\n", 206 | "import numpy as np\n", 207 | "import matplotlib.pyplot as plt\n", 208 | "\n", 209 | "# random seed\n", 210 | "seed = 42\n", 211 | "torch.manual_seed(seed)\n", 212 | "np.random.seed(seed)\n", 213 | "\n", 214 | "# Add the directory containing models.py to the system path\n", 215 | "sys.path.append(os.path.join(package_path, \"torch-gan\"))\n", 216 | "\n", 217 | "\n", 218 | "# Now we can import the model and dataset\n", 219 | "import model\n", 220 | "import dataset\n", 221 | "import device\n", 222 | "\n", 223 | "# parameters\n", 224 | "dataset_id = \"celeb-a\"\n", 225 | "num_latent_dims = 100\n", 226 | "max_num_filters = 512\n", 227 | "img_size = (64, 64)\n", 228 | "batch_size = 32\n", 229 | "model_id = f\"G_filters_{max_num_filters:04d}_dims_{num_latent_dims:04d}.pth\"\n", 230 | "gen_fname = os.path.join(package_path, \"torch-gan\", \"pretrained\", dataset_id, model_id)\n", 231 | "dev = device.autoselectDevice()\n", 232 | "\n", 233 | "# load dataset\n", 234 | "(\n", 235 | " celeba_train_loader,\n", 236 | " celeba_test_loader,\n", 237 | " _,\n", 238 | " celeba_num_img_channels,\n", 239 | ") = dataset.get_loaders(dataset_id, img_size=img_size, batch_size=batch_size)\n", 240 | "\n", 241 | "# load the generator\n", 242 | "G = model.Generator(num_latent_dims, celeba_num_img_channels, max_num_filters, dev)\n", 243 | "G.load(gen_fname, dev)\n", 244 | "\n", 245 | "if G:\n", 246 | " print(f\"Model {gen_fname} loaded successfully!\")\n", 247 | " print(f\"Device used: {dev}\")\n", 248 | " G.to(dev)\n", 249 | " G.eval()" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": {}, 255 | "source": [ 256 | "## Show some Training Images" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": null, 262 | "metadata": {}, 263 | "outputs": [], 264 | "source": [ 265 | "def normalizeForDisplay(images):\n", 266 | " # normalize from [-1, 1] to [0, 1]\n", 267 | " return (images + 1.0) / 2.0" 268 | ] 269 | }, 270 | { 271 | "cell_type": "code", 272 | "execution_count": null, 273 | "metadata": {}, 274 | "outputs": [], 275 | "source": [ 276 | "import matplotlib.pyplot as plt\n", 277 | "\n", 278 | "# get a batch of images from the training set and display them\n", 279 | "# we use the torchvision.utils.make_grid function to create a grid of images\n", 280 | "images, labels = next(iter(celeba_train_loader))\n", 281 | "grid_img = torchvision.utils.make_grid(\n", 282 | " normalizeForDisplay(images), nrow=batch_size // 4\n", 283 | ")\n", 284 | "plt.figure(figsize=(10, 10))\n", 285 | "plt.imshow(np.transpose(grid_img, (1, 2, 0)))\n", 286 | "plt.axis(\"off\")\n", 287 | "plt.title(\"Batch from the Training Set\")\n", 288 | "plt.show()" 289 | ] 290 | }, 291 | { 292 | "cell_type": "markdown", 293 | "metadata": {}, 294 | "source": [ 295 | "## Generate CelebA-like Samples" 296 | ] 297 | }, 298 | { 299 | "cell_type": "code", 300 | "execution_count": null, 301 | "metadata": {}, 302 | "outputs": [], 303 | "source": [ 304 | "import utils\n", 305 | "\n", 306 | "\n", 307 | "def sampleAndPlot(G, num_latent_dims, num_samples=batch_size):\n", 308 | " with torch.no_grad():\n", 309 | " for i in range(num_samples):\n", 310 | " # generate a random latent vector\n", 311 | " z = utils.sample_latent_vectors(1, num_latent_dims, dev)\n", 312 | "\n", 313 | " # generate an image from the latent vector\n", 314 | " img = G(z)\n", 315 | "\n", 316 | " if i == 0:\n", 317 | " pics = img\n", 318 | " else:\n", 319 | " pics = torch.cat((pics, img), dim=0)\n", 320 | "\n", 321 | " # Create a grid of images\n", 322 | " grid_img = torchvision.utils.make_grid(pics, nrow=batch_size // 4)\n", 323 | "\n", 324 | " # Convert grid to numpy and transpose axes for plotting\n", 325 | " grid_np = grid_img.cpu().numpy()\n", 326 | " grid_np = np.transpose(grid_np, (1, 2, 0))\n", 327 | "\n", 328 | " # Plotting\n", 329 | " plt.figure(figsize=(10, 10))\n", 330 | " plt.imshow(normalizeForDisplay(grid_np))\n", 331 | " plt.axis(\"off\")\n", 332 | " plt.title(\n", 333 | " f\"Randomly Generated Images from the Generator with {num_latent_dims} Latent Dimensions\"\n", 334 | " )\n", 335 | " plt.show()" 336 | ] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "execution_count": null, 341 | "metadata": {}, 342 | "outputs": [], 343 | "source": [ 344 | "sampleAndPlot(G, num_latent_dims)" 345 | ] 346 | } 347 | ], 348 | "metadata": { 349 | "kernelspec": { 350 | "display_name": "pytorch-m1-2023-10", 351 | "language": "python", 352 | "name": "pytorch-m1-2023-10" 353 | }, 354 | "language_info": { 355 | "codemirror_mode": { 356 | "name": "ipython", 357 | "version": 3 358 | }, 359 | "file_extension": ".py", 360 | "mimetype": "text/x-python", 361 | "name": "python", 362 | "nbconvert_exporter": "python", 363 | "pygments_lexer": "ipython3", 364 | "version": "3.9.18" 365 | } 366 | }, 367 | "nbformat": 4, 368 | "nbformat_minor": 2 369 | } 370 | -------------------------------------------------------------------------------- /gan/torch/requirements.txt: -------------------------------------------------------------------------------- 1 | matplotlib 2 | 3 | -------------------------------------------------------------------------------- /linear_regression/mlx/linear_regression_mlx.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "epxcwtWj5yJs" 7 | }, 8 | "source": [ 9 | "# Linear regression in MLX\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute.\n", 14 | "\n", 15 | "**Note: This requires a machine with an Apple SoC, e.g. M1/M2/M3 etc.**\n", 16 | "\n", 17 | "See: https://github.com/ml-explore/mlx\n" 18 | ] 19 | }, 20 | { 21 | "cell_type": "markdown", 22 | "metadata": { 23 | "id": "IJsjl_l47q28" 24 | }, 25 | "source": [ 26 | "## Setup\n", 27 | "\n", 28 | "Adapt `packagePath` to point to the directory containing this notebeook." 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": { 35 | "executionInfo": { 36 | "elapsed": 236, 37 | "status": "ok", 38 | "timestamp": 1703326573912, 39 | "user": { 40 | "displayName": "Markus Enzweiler", 41 | "userId": "04524044579212347608" 42 | }, 43 | "user_tz": -60 44 | }, 45 | "id": "1iHkPBml98YG" 46 | }, 47 | "outputs": [], 48 | "source": [ 49 | "# Notebook id\n", 50 | "nb_id = \"linear_regression/mlx\"\n", 51 | "\n", 52 | "# Imports\n", 53 | "import sys\n", 54 | "import os" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": null, 60 | "metadata": {}, 61 | "outputs": [], 62 | "source": [ 63 | "# Package Path (folder of this notebook)\n", 64 | "\n", 65 | "#####################\n", 66 | "# Local environment #\n", 67 | "#####################\n", 68 | "\n", 69 | "package_path = \"./\"\n", 70 | "\n", 71 | "\n", 72 | "#########\n", 73 | "# Colab #\n", 74 | "#########\n", 75 | "\n", 76 | "\n", 77 | "def check_for_colab():\n", 78 | " try:\n", 79 | " import google.colab\n", 80 | "\n", 81 | " return True\n", 82 | " except ImportError:\n", 83 | " return False\n", 84 | "\n", 85 | "\n", 86 | "# running on Colab?\n", 87 | "on_colab = check_for_colab()\n", 88 | "\n", 89 | "if on_colab:\n", 90 | " # assume this notebook is run from Google Drive and the whole\n", 91 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 92 | "\n", 93 | " # Google Drive mount point\n", 94 | " gdrive_mnt = \"/content/drive\"\n", 95 | "\n", 96 | " ##########################################################################\n", 97 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 98 | " ##########################################################################\n", 99 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 100 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 101 | "\n", 102 | " # mount drive\n", 103 | " from google.colab import drive\n", 104 | "\n", 105 | " drive.mount(gdrive_mnt, force_remount=True)\n", 106 | "\n", 107 | " # set package path\n", 108 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 109 | "\n", 110 | "# check whether package path exists\n", 111 | "if not os.path.isdir(package_path):\n", 112 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 113 | "\n", 114 | "print(f\"Package path: {package_path}\")" 115 | ] 116 | }, 117 | { 118 | "cell_type": "code", 119 | "execution_count": null, 120 | "metadata": { 121 | "colab": { 122 | "base_uri": "https://localhost:8080/" 123 | }, 124 | "executionInfo": { 125 | "elapsed": 2602, 126 | "status": "ok", 127 | "timestamp": 1703326576880, 128 | "user": { 129 | "displayName": "Markus Enzweiler", 130 | "userId": "04524044579212347608" 131 | }, 132 | "user_tz": -60 133 | }, 134 | "id": "DY4880S378_F", 135 | "outputId": "a8ce2d26-fe24-4aa2-c232-acc31b71d394" 136 | }, 137 | "outputs": [], 138 | "source": [ 139 | "# Additional imports\n", 140 | "\n", 141 | "# Repository Root\n", 142 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 143 | "# Add the repository root to the system path\n", 144 | "if repo_root not in sys.path:\n", 145 | " sys.path.append(repo_root)\n", 146 | "\n", 147 | "# Package Imports\n", 148 | "from nbutils import requirements as nb_reqs\n", 149 | "from nbutils import colab as nb_colab\n", 150 | "from nbutils import git as nb_git\n", 151 | "from nbutils import exec as nb_exec\n", 152 | "from nbutils import data as nb_data" 153 | ] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "execution_count": null, 158 | "metadata": { 159 | "colab": { 160 | "base_uri": "https://localhost:8080/" 161 | }, 162 | "executionInfo": { 163 | "elapsed": 6888, 164 | "status": "ok", 165 | "timestamp": 1703326583765, 166 | "user": { 167 | "displayName": "Markus Enzweiler", 168 | "userId": "04524044579212347608" 169 | }, 170 | "user_tz": -60 171 | }, 172 | "id": "iaURjw5n6pLq", 173 | "outputId": "5c40ee52-1fa3-44df-a94e-b81bec144100" 174 | }, 175 | "outputs": [], 176 | "source": [ 177 | "# Install requirements in the current Jupyter kernel\n", 178 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 179 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": null, 185 | "metadata": { 186 | "executionInfo": { 187 | "elapsed": 12, 188 | "status": "ok", 189 | "timestamp": 1703326583765, 190 | "user": { 191 | "displayName": "Markus Enzweiler", 192 | "userId": "04524044579212347608" 193 | }, 194 | "user_tz": -60 195 | }, 196 | "id": "iRERDI8aAnzr" 197 | }, 198 | "outputs": [], 199 | "source": [ 200 | "# Now we should be able to import the additional packages\n", 201 | "import mlx\n", 202 | "import mlx.core as mx\n", 203 | "\n", 204 | "import numpy as np\n", 205 | "import matplotlib.pyplot as plt\n", 206 | "\n", 207 | "# Set the random seed for reproducibility\n", 208 | "mx.random.seed(42)" 209 | ] 210 | }, 211 | { 212 | "cell_type": "markdown", 213 | "metadata": { 214 | "id": "_aI-pchqASeB" 215 | }, 216 | "source": [ 217 | "## MLX\n", 218 | "\n", 219 | "MLX provides composable function transformations, supporting automatic differentiation, automatic vectorization, and optimization of computation graphs. Computation graphs within MLX are dynamically constructed. A key feature of MLX is the use of (and optimization for) unified memory present in the Apple SoCs. \n", 220 | "\n", 221 | "See:\n", 222 | "- https://github.com/ml-explore/mlx\n", 223 | "- https://ml-explore.github.io/mlx/build/html/quick_start.html" 224 | ] 225 | }, 226 | { 227 | "cell_type": "markdown", 228 | "metadata": { 229 | "id": "aUACL9QoFTXg" 230 | }, 231 | "source": [ 232 | "## Linear regression" 233 | ] 234 | }, 235 | { 236 | "cell_type": "markdown", 237 | "metadata": {}, 238 | "source": [ 239 | "### Create some data based on adding noise to a known linear function" 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": null, 245 | "metadata": {}, 246 | "outputs": [], 247 | "source": [ 248 | "# Creating a function f(x) with a slope of 2 and bias of 1, e.g. f(x) = 2x + 1\n", 249 | "# and added Gaussian noise\n", 250 | "\n", 251 | "# True parameters\n", 252 | "w_true = 2\n", 253 | "b_true = 1\n", 254 | "params_true = mx.array([w_true, b_true])\n", 255 | "\n", 256 | "X = mx.arange(-5, 5, 0.1)\n", 257 | "Y = w_true * X + b_true + 2 * mx.random.normal(X.shape)\n", 258 | "\n", 259 | "# Visualize\n", 260 | "plt.scatter(X, Y, alpha=0.5)\n", 261 | "plt.title(\"Scatter plot of f(x) = 2x + 1 + Gaussian noise\")\n", 262 | "plt.show()" 263 | ] 264 | }, 265 | { 266 | "cell_type": "markdown", 267 | "metadata": {}, 268 | "source": [ 269 | "### Linear model and loss\n", 270 | "\n", 271 | "Our linear regression model is $ y = f(x) = w \\cdot x + b$. We solve for $w$ and $b$ using gradient descent. " 272 | ] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "execution_count": null, 277 | "metadata": {}, 278 | "outputs": [], 279 | "source": [ 280 | "def lin_model(params, x):\n", 281 | " w, b = params\n", 282 | " return w * x + b" 283 | ] 284 | }, 285 | { 286 | "cell_type": "markdown", 287 | "metadata": {}, 288 | "source": [ 289 | "We uses mean squared error loss between the predictions of our model and true values. " 290 | ] 291 | }, 292 | { 293 | "cell_type": "code", 294 | "execution_count": null, 295 | "metadata": {}, 296 | "outputs": [], 297 | "source": [ 298 | "def loss_fn(params, x, y):\n", 299 | " y_pred = lin_model(params, x)\n", 300 | " return mx.mean(mx.square(y - y_pred))" 301 | ] 302 | }, 303 | { 304 | "cell_type": "markdown", 305 | "metadata": {}, 306 | "source": [ 307 | "### Optimization via gradient descent" 308 | ] 309 | }, 310 | { 311 | "cell_type": "markdown", 312 | "metadata": {}, 313 | "source": [ 314 | "Initialize parameters $w$ and $b$ randomly." 315 | ] 316 | }, 317 | { 318 | "cell_type": "code", 319 | "execution_count": null, 320 | "metadata": {}, 321 | "outputs": [], 322 | "source": [ 323 | "params = 1e-2 * mx.random.normal((2,))\n", 324 | "w, b = params\n", 325 | "print(f\"Initial weight : {w}\")\n", 326 | "print(f\"Initial bias : {b}\")" 327 | ] 328 | }, 329 | { 330 | "cell_type": "markdown", 331 | "metadata": {}, 332 | "source": [ 333 | "Optimize via gradient descent" 334 | ] 335 | }, 336 | { 337 | "cell_type": "code", 338 | "execution_count": null, 339 | "metadata": {}, 340 | "outputs": [], 341 | "source": [ 342 | "# Hyperparameters\n", 343 | "num_iters = 10000\n", 344 | "learning_rate = 3e-4\n", 345 | "\n", 346 | "# Function that computes loss and its gradient\n", 347 | "loss_and_grad_fn = mx.value_and_grad(loss_fn)\n", 348 | "\n", 349 | "# Loop over the number of iterations\n", 350 | "for it in range(num_iters):\n", 351 | " # gradient of loss function (vectorized)\n", 352 | " loss, loss_grad = loss_and_grad_fn(params, X, Y)\n", 353 | "\n", 354 | " # update parameters via gradient descent update rules\n", 355 | " params -= learning_rate * loss_grad\n", 356 | "\n", 357 | " # Evaluate the parameters explicitly, because MLX uses lazy evaluation\n", 358 | " mx.eval(params)\n", 359 | "\n", 360 | " # Give some status output once in a while\n", 361 | " if it % 500 == 0 or it == num_iters - 1:\n", 362 | " w, b = params\n", 363 | " error_norm = mx.sum(mx.square(params - params_true)) ** 0.5\n", 364 | " print(\n", 365 | " f\"Iteration {it:5d} | Loss {loss.item():>10.5f} | \"\n", 366 | " f\"w {w.item():> 8.5f} | b {b.item():> 8.5f} | Error norm {error_norm.item():>.5f}\"\n", 367 | " )\n", 368 | "\n", 369 | "\n", 370 | "w, b = params\n", 371 | "print(f\"Final weight after optimization : {w.item():.5f} (true: {w_true})\")\n", 372 | "print(f\"Final bias after optimization : {b.item():.5f} (true: {b_true})\")" 373 | ] 374 | }, 375 | { 376 | "cell_type": "markdown", 377 | "metadata": {}, 378 | "source": [ 379 | "Visualize linear fit" 380 | ] 381 | }, 382 | { 383 | "cell_type": "code", 384 | "execution_count": null, 385 | "metadata": {}, 386 | "outputs": [], 387 | "source": [ 388 | "# Visualize\n", 389 | "plt.scatter(X, Y, alpha=0.5)\n", 390 | "plt.title(\"Scatter plot of f(x) = 2x + 1 + Gaussian noise\")\n", 391 | "\n", 392 | "# Plot the recovered line\n", 393 | "Y_model = lin_model(params, X)\n", 394 | "plt.plot(X.tolist(), Y_model.tolist(), color=\"red\")\n", 395 | "\n", 396 | "plt.legend([\"data\", f\"f(x) = {w.item():.3f}x + {b.item():.3f}\"])\n", 397 | "plt.show()" 398 | ] 399 | } 400 | ], 401 | "metadata": { 402 | "colab": { 403 | "collapsed_sections": [ 404 | "JcF1mpJo-taz" 405 | ], 406 | "provenance": [] 407 | }, 408 | "kernelspec": { 409 | "display_name": "mlx-m1-2023-12", 410 | "language": "python", 411 | "name": "python3" 412 | }, 413 | "language_info": { 414 | "codemirror_mode": { 415 | "name": "ipython", 416 | "version": 3 417 | }, 418 | "file_extension": ".py", 419 | "mimetype": "text/x-python", 420 | "name": "python", 421 | "nbconvert_exporter": "python", 422 | "pygments_lexer": "ipython3", 423 | "version": "3.11.6" 424 | } 425 | }, 426 | "nbformat": 4, 427 | "nbformat_minor": 0 428 | } 429 | -------------------------------------------------------------------------------- /linear_regression/mlx/requirements.txt: -------------------------------------------------------------------------------- 1 | mlx 2 | numpy 3 | matplotlib -------------------------------------------------------------------------------- /linear_regression/torch/linear_regression_torch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "epxcwtWj5yJs" 7 | }, 8 | "source": [ 9 | "# Linear regression in PyTorch\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": { 19 | "id": "IJsjl_l47q28" 20 | }, 21 | "source": [ 22 | "## Setup\n", 23 | "\n", 24 | "Adapt `packagePath` to point to the directory containing this notebeook." 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": null, 30 | "metadata": { 31 | "executionInfo": { 32 | "elapsed": 236, 33 | "status": "ok", 34 | "timestamp": 1703326573912, 35 | "user": { 36 | "displayName": "Markus Enzweiler", 37 | "userId": "04524044579212347608" 38 | }, 39 | "user_tz": -60 40 | }, 41 | "id": "1iHkPBml98YG" 42 | }, 43 | "outputs": [], 44 | "source": [ 45 | "# Notebook id\n", 46 | "nb_id = \"linear_regression/torch\"\n", 47 | "\n", 48 | "# Imports\n", 49 | "import sys\n", 50 | "import os" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [ 59 | "# Package Path (folder of this notebook)\n", 60 | "\n", 61 | "#####################\n", 62 | "# Local environment #\n", 63 | "#####################\n", 64 | "\n", 65 | "package_path = \"./\"\n", 66 | "\n", 67 | "\n", 68 | "#########\n", 69 | "# Colab #\n", 70 | "#########\n", 71 | "\n", 72 | "\n", 73 | "def check_for_colab():\n", 74 | " try:\n", 75 | " import google.colab\n", 76 | "\n", 77 | " return True\n", 78 | " except ImportError:\n", 79 | " return False\n", 80 | "\n", 81 | "\n", 82 | "# running on Colab?\n", 83 | "on_colab = check_for_colab()\n", 84 | "\n", 85 | "if on_colab:\n", 86 | " # assume this notebook is run from Google Drive and the whole\n", 87 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 88 | "\n", 89 | " # Google Drive mount point\n", 90 | " gdrive_mnt = \"/content/drive\"\n", 91 | "\n", 92 | " ##########################################################################\n", 93 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 94 | " ##########################################################################\n", 95 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 96 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 97 | "\n", 98 | " # mount drive\n", 99 | " from google.colab import drive\n", 100 | "\n", 101 | " drive.mount(gdrive_mnt, force_remount=True)\n", 102 | "\n", 103 | " # set package path\n", 104 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 105 | "\n", 106 | "# check whether package path exists\n", 107 | "if not os.path.isdir(package_path):\n", 108 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 109 | "\n", 110 | "print(f\"Package path: {package_path}\")" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": null, 116 | "metadata": { 117 | "colab": { 118 | "base_uri": "https://localhost:8080/" 119 | }, 120 | "executionInfo": { 121 | "elapsed": 2602, 122 | "status": "ok", 123 | "timestamp": 1703326576880, 124 | "user": { 125 | "displayName": "Markus Enzweiler", 126 | "userId": "04524044579212347608" 127 | }, 128 | "user_tz": -60 129 | }, 130 | "id": "DY4880S378_F", 131 | "outputId": "a8ce2d26-fe24-4aa2-c232-acc31b71d394" 132 | }, 133 | "outputs": [], 134 | "source": [ 135 | "# Additional imports\n", 136 | "\n", 137 | "# Repository Root\n", 138 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 139 | "# Add the repository root to the system path\n", 140 | "if repo_root not in sys.path:\n", 141 | " sys.path.append(repo_root)\n", 142 | "\n", 143 | "# Package Imports\n", 144 | "from nbutils import requirements as nb_reqs\n", 145 | "from nbutils import colab as nb_colab\n", 146 | "from nbutils import git as nb_git\n", 147 | "from nbutils import exec as nb_exec\n", 148 | "from nbutils import data as nb_data" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": null, 154 | "metadata": { 155 | "colab": { 156 | "base_uri": "https://localhost:8080/" 157 | }, 158 | "executionInfo": { 159 | "elapsed": 6888, 160 | "status": "ok", 161 | "timestamp": 1703326583765, 162 | "user": { 163 | "displayName": "Markus Enzweiler", 164 | "userId": "04524044579212347608" 165 | }, 166 | "user_tz": -60 167 | }, 168 | "id": "iaURjw5n6pLq", 169 | "outputId": "5c40ee52-1fa3-44df-a94e-b81bec144100" 170 | }, 171 | "outputs": [], 172 | "source": [ 173 | "# Install requirements in the current Jupyter kernel\n", 174 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 175 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 176 | ] 177 | }, 178 | { 179 | "cell_type": "code", 180 | "execution_count": null, 181 | "metadata": { 182 | "executionInfo": { 183 | "elapsed": 12, 184 | "status": "ok", 185 | "timestamp": 1703326583765, 186 | "user": { 187 | "displayName": "Markus Enzweiler", 188 | "userId": "04524044579212347608" 189 | }, 190 | "user_tz": -60 191 | }, 192 | "id": "iRERDI8aAnzr" 193 | }, 194 | "outputs": [], 195 | "source": [ 196 | "# Now we should be able to import the additional packages\n", 197 | "import torch\n", 198 | "import numpy as np\n", 199 | "import matplotlib.pyplot as plt\n", 200 | "\n", 201 | "# Set the random seed for reproducibility\n", 202 | "torch.manual_seed(0);" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": { 208 | "id": "aUACL9QoFTXg" 209 | }, 210 | "source": [ 211 | "## Linear regression" 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "metadata": {}, 217 | "source": [ 218 | "### Create some data based on adding noise to a known linear function" 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "execution_count": null, 224 | "metadata": {}, 225 | "outputs": [], 226 | "source": [ 227 | "# Creating a function f(x) with a slope of 2 and bias of 1, e.g. f(x) = 2x + 1\n", 228 | "# and added Gaussian noise\n", 229 | "\n", 230 | "# True parameters\n", 231 | "w_true = 2\n", 232 | "b_true = 1\n", 233 | "params_true = torch.tensor([w_true, b_true])\n", 234 | "\n", 235 | "X = torch.arange(-5, 5, 0.1)\n", 236 | "Y = w_true * X + b_true + 2 * torch.randn(X.shape)\n", 237 | "\n", 238 | "# Visualize\n", 239 | "plt.scatter(X, Y, alpha=0.5)\n", 240 | "plt.title(\"Scatter plot of f(x) = 2x + 1 + Gaussian noise\")\n", 241 | "plt.show()" 242 | ] 243 | }, 244 | { 245 | "cell_type": "markdown", 246 | "metadata": {}, 247 | "source": [ 248 | "### Linear model and loss\n", 249 | "\n", 250 | "Our linear regression model is $ y = f(x) = w \\cdot x + b$. We solve for $w$ and $b$ using gradient descent. " 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": null, 256 | "metadata": {}, 257 | "outputs": [], 258 | "source": [ 259 | "def lin_model(params, x):\n", 260 | " w, b = params\n", 261 | " return w * x + b" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "metadata": {}, 267 | "source": [ 268 | "We uses mean squared error loss between the predictions of our model and true values. " 269 | ] 270 | }, 271 | { 272 | "cell_type": "code", 273 | "execution_count": null, 274 | "metadata": {}, 275 | "outputs": [], 276 | "source": [ 277 | "def loss_fn(y_pred, y):\n", 278 | " return torch.mean(torch.square(y - y_pred))" 279 | ] 280 | }, 281 | { 282 | "cell_type": "markdown", 283 | "metadata": {}, 284 | "source": [ 285 | "### Optimization via gradient descent" 286 | ] 287 | }, 288 | { 289 | "cell_type": "markdown", 290 | "metadata": {}, 291 | "source": [ 292 | "Initialize parameters $w$ and $b$ randomly." 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": null, 298 | "metadata": {}, 299 | "outputs": [], 300 | "source": [ 301 | "params = 1e-2 * torch.randn(2, dtype=torch.float32)\n", 302 | "\n", 303 | "# we track gradients for the parameters w and b\n", 304 | "params.requires_grad_()\n", 305 | "\n", 306 | "w, b = params\n", 307 | "print(f\"Initial weight : {w}\")\n", 308 | "print(f\"Initial bias : {b}\")" 309 | ] 310 | }, 311 | { 312 | "cell_type": "markdown", 313 | "metadata": {}, 314 | "source": [ 315 | "Optimize via gradient descent" 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": null, 321 | "metadata": {}, 322 | "outputs": [], 323 | "source": [ 324 | "# Hyperparameters\n", 325 | "num_iters = 10000\n", 326 | "learning_rate = 3e-4\n", 327 | "\n", 328 | "# Loop over the number of iterations\n", 329 | "for it in range(num_iters):\n", 330 | " # predict y from x\n", 331 | " Y_pred = lin_model(params, X)\n", 332 | "\n", 333 | " # Compute the loss\n", 334 | " loss = loss_fn(Y_pred, Y)\n", 335 | "\n", 336 | " # Gradient of loss function w.r.t parameters\n", 337 | " loss.backward()\n", 338 | "\n", 339 | " # update parameters via gradient descent update rules\n", 340 | " with torch.no_grad():\n", 341 | " params -= learning_rate * params.grad\n", 342 | " params.grad.zero_()\n", 343 | "\n", 344 | " # Give some status output once in a while\n", 345 | " if it % 500 == 0 or it == num_iters - 1:\n", 346 | " w, b = params\n", 347 | " error_norm = torch.sum(torch.square(params - params_true)) ** 0.5\n", 348 | " print(\n", 349 | " f\"Iteration {it:5d} | Loss {loss.item():>10.5f} | \"\n", 350 | " f\"w {w.item():> 8.5f} | b {b.item():> 8.5f} | Error norm {error_norm.item():>.5f}\"\n", 351 | " )\n", 352 | "\n", 353 | "w, b = params\n", 354 | "print(f\"Final weight after optimization : {w.item():.5f} (true: {w_true})\")\n", 355 | "print(f\"Final bias after optimization : {b.item():.5f} (true: {b_true})\")" 356 | ] 357 | }, 358 | { 359 | "cell_type": "markdown", 360 | "metadata": {}, 361 | "source": [ 362 | "Visualize linear fit" 363 | ] 364 | }, 365 | { 366 | "cell_type": "code", 367 | "execution_count": null, 368 | "metadata": {}, 369 | "outputs": [], 370 | "source": [ 371 | "# Visualize\n", 372 | "plt.scatter(X, Y, alpha=0.5)\n", 373 | "plt.title(\"Scatter plot of f(x) = 2x + 1 + Gaussian noise\")\n", 374 | "\n", 375 | "# Plot the recovered line\n", 376 | "Y_model = lin_model(params, X)\n", 377 | "plt.plot(X.tolist(), Y_model.tolist(), color=\"red\")\n", 378 | "\n", 379 | "plt.legend([\"data\", f\"f(x) = {w.item():.3f}x + {b.item():.3f}\"])\n", 380 | "plt.show()" 381 | ] 382 | } 383 | ], 384 | "metadata": { 385 | "colab": { 386 | "collapsed_sections": [ 387 | "JcF1mpJo-taz" 388 | ], 389 | "provenance": [] 390 | }, 391 | "kernelspec": { 392 | "display_name": "mlx-pytorch-2024-01", 393 | "language": "python", 394 | "name": "mlx-pytorch-2024-01" 395 | }, 396 | "language_info": { 397 | "codemirror_mode": { 398 | "name": "ipython", 399 | "version": 3 400 | }, 401 | "file_extension": ".py", 402 | "mimetype": "text/x-python", 403 | "name": "python", 404 | "nbconvert_exporter": "python", 405 | "pygments_lexer": "ipython3", 406 | "version": "3.11.6" 407 | } 408 | }, 409 | "nbformat": 4, 410 | "nbformat_minor": 0 411 | } 412 | -------------------------------------------------------------------------------- /linear_regression/torch/linear_regression_torch_manual_grad_desc.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "epxcwtWj5yJs" 7 | }, 8 | "source": [ 9 | "# Linear regression in PyTorch with manual gradient descent\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": { 19 | "id": "IJsjl_l47q28" 20 | }, 21 | "source": [ 22 | "## Setup\n", 23 | "\n", 24 | "Adapt `packagePath` to point to the directory containing this notebeook." 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": null, 30 | "metadata": { 31 | "executionInfo": { 32 | "elapsed": 236, 33 | "status": "ok", 34 | "timestamp": 1703326573912, 35 | "user": { 36 | "displayName": "Markus Enzweiler", 37 | "userId": "04524044579212347608" 38 | }, 39 | "user_tz": -60 40 | }, 41 | "id": "1iHkPBml98YG" 42 | }, 43 | "outputs": [], 44 | "source": [ 45 | "# Notebook id\n", 46 | "nb_id = \"linear_regression/torch\"\n", 47 | "\n", 48 | "# Imports\n", 49 | "import sys\n", 50 | "import os" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [ 59 | "# Package Path (folder of this notebook)\n", 60 | "\n", 61 | "#####################\n", 62 | "# Local environment #\n", 63 | "#####################\n", 64 | "\n", 65 | "package_path = \"./\"\n", 66 | "\n", 67 | "\n", 68 | "#########\n", 69 | "# Colab #\n", 70 | "#########\n", 71 | "\n", 72 | "\n", 73 | "def check_for_colab():\n", 74 | " try:\n", 75 | " import google.colab\n", 76 | "\n", 77 | " return True\n", 78 | " except ImportError:\n", 79 | " return False\n", 80 | "\n", 81 | "\n", 82 | "# running on Colab?\n", 83 | "on_colab = check_for_colab()\n", 84 | "\n", 85 | "if on_colab:\n", 86 | " # assume this notebook is run from Google Drive and the whole\n", 87 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 88 | "\n", 89 | " # Google Drive mount point\n", 90 | " gdrive_mnt = \"/content/drive\"\n", 91 | "\n", 92 | " ##########################################################################\n", 93 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 94 | " ##########################################################################\n", 95 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 96 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 97 | "\n", 98 | " # mount drive\n", 99 | " from google.colab import drive\n", 100 | "\n", 101 | " drive.mount(gdrive_mnt, force_remount=True)\n", 102 | "\n", 103 | " # set package path\n", 104 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 105 | "\n", 106 | "# check whether package path exists\n", 107 | "if not os.path.isdir(package_path):\n", 108 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 109 | "\n", 110 | "print(f\"Package path: {package_path}\")" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": null, 116 | "metadata": { 117 | "colab": { 118 | "base_uri": "https://localhost:8080/" 119 | }, 120 | "executionInfo": { 121 | "elapsed": 2602, 122 | "status": "ok", 123 | "timestamp": 1703326576880, 124 | "user": { 125 | "displayName": "Markus Enzweiler", 126 | "userId": "04524044579212347608" 127 | }, 128 | "user_tz": -60 129 | }, 130 | "id": "DY4880S378_F", 131 | "outputId": "a8ce2d26-fe24-4aa2-c232-acc31b71d394" 132 | }, 133 | "outputs": [], 134 | "source": [ 135 | "# Additional imports\n", 136 | "\n", 137 | "# Repository Root\n", 138 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 139 | "# Add the repository root to the system path\n", 140 | "if repo_root not in sys.path:\n", 141 | " sys.path.append(repo_root)\n", 142 | "\n", 143 | "# Package Imports\n", 144 | "from nbutils import requirements as nb_reqs\n", 145 | "from nbutils import colab as nb_colab\n", 146 | "from nbutils import git as nb_git\n", 147 | "from nbutils import exec as nb_exec\n", 148 | "from nbutils import data as nb_data" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": null, 154 | "metadata": { 155 | "colab": { 156 | "base_uri": "https://localhost:8080/" 157 | }, 158 | "executionInfo": { 159 | "elapsed": 6888, 160 | "status": "ok", 161 | "timestamp": 1703326583765, 162 | "user": { 163 | "displayName": "Markus Enzweiler", 164 | "userId": "04524044579212347608" 165 | }, 166 | "user_tz": -60 167 | }, 168 | "id": "iaURjw5n6pLq", 169 | "outputId": "5c40ee52-1fa3-44df-a94e-b81bec144100" 170 | }, 171 | "outputs": [], 172 | "source": [ 173 | "# Install requirements in the current Jupyter kernel\n", 174 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 175 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 176 | ] 177 | }, 178 | { 179 | "cell_type": "code", 180 | "execution_count": null, 181 | "metadata": { 182 | "executionInfo": { 183 | "elapsed": 12, 184 | "status": "ok", 185 | "timestamp": 1703326583765, 186 | "user": { 187 | "displayName": "Markus Enzweiler", 188 | "userId": "04524044579212347608" 189 | }, 190 | "user_tz": -60 191 | }, 192 | "id": "iRERDI8aAnzr" 193 | }, 194 | "outputs": [], 195 | "source": [ 196 | "# Now we should be able to import the additional packages\n", 197 | "import torch\n", 198 | "import numpy as np\n", 199 | "import matplotlib.pyplot as plt\n", 200 | "\n", 201 | "# Set the random seed for reproducibility\n", 202 | "torch.manual_seed(0);" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": { 208 | "id": "aUACL9QoFTXg" 209 | }, 210 | "source": [ 211 | "## Linear regression" 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "metadata": {}, 217 | "source": [ 218 | "### Create some data based on adding noise to a known linear function" 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "execution_count": null, 224 | "metadata": {}, 225 | "outputs": [], 226 | "source": [ 227 | "# Creating a function f(x) with a slope of 2 and bias of 1, e.g. f(x) = 2x + 1\n", 228 | "# and added Gaussian noise\n", 229 | "\n", 230 | "# True parameters\n", 231 | "w_true = 2\n", 232 | "b_true = 1\n", 233 | "\n", 234 | "X = torch.arange(-5, 5, 0.1)\n", 235 | "Y = w_true * X + b_true + 2 * torch.randn(X.shape)\n", 236 | "\n", 237 | "# Visualize\n", 238 | "plt.scatter(X, Y, alpha=0.5)\n", 239 | "plt.title(\"Scatter plot of f(x) = 2x + 1 + Gaussian noise\")\n", 240 | "plt.show()" 241 | ] 242 | }, 243 | { 244 | "cell_type": "markdown", 245 | "metadata": {}, 246 | "source": [ 247 | "### Linear model and loss\n", 248 | "\n", 249 | "Our linear regression model is $ y = f(x) = w \\cdot x + b$. We solve for $w$ and $b$ using gradient descent. " 250 | ] 251 | }, 252 | { 253 | "cell_type": "code", 254 | "execution_count": null, 255 | "metadata": {}, 256 | "outputs": [], 257 | "source": [ 258 | "def lin_model(w, b, x):\n", 259 | " return w * x + b" 260 | ] 261 | }, 262 | { 263 | "cell_type": "markdown", 264 | "metadata": {}, 265 | "source": [ 266 | "We uses mean squared error loss between the predictions $\\hat{y_i}$ of our model and true values $y_i$:\n", 267 | "\n", 268 | "$$L(w,b) = \\frac{1}{N} \\sum_{i=1}^{N} (y_i - \\hat{y_i})^2= \\frac{1}{N} \\sum_{i=1}^{N} (y_i - (w \\cdot x_i + b))^2$$\n" 269 | ] 270 | }, 271 | { 272 | "cell_type": "code", 273 | "execution_count": null, 274 | "metadata": {}, 275 | "outputs": [], 276 | "source": [ 277 | "def loss_fn(y_pred, y):\n", 278 | " return torch.mean(torch.square(y - y_pred))" 279 | ] 280 | }, 281 | { 282 | "cell_type": "markdown", 283 | "metadata": {}, 284 | "source": [ 285 | "### Optimization via gradient descent" 286 | ] 287 | }, 288 | { 289 | "cell_type": "markdown", 290 | "metadata": {}, 291 | "source": [ 292 | "The gradients of the loss $L(w,b)$ with respect to both $w$ and $b$ are (application of chain rule):\n", 293 | "\n", 294 | "\n", 295 | "\n", 296 | "$$\\frac{\\partial L}{\\partial w} = \\frac{1}{N} \\sum_{i=1}^{N} 2(y_i - (w \\cdot x_i + b)) \\cdot \\frac{\\partial}{\\partial w} (y_i - (w \\cdot x_i + b))$$\n", 297 | "\n", 298 | "$$ = \\frac{1}{N} \\sum_{i=1}^{N} 2(y_i - (w \\cdot x_i + b)) \\cdot (-x_i)$$ \n", 299 | "\n", 300 | "$$ = \\frac{1}{N} \\sum_{i=1}^{N} -2(y_i - w \\cdot x_i - b) \\cdot x_i$$\n", 301 | "\n", 302 | "---\n", 303 | "\n", 304 | "$$\\frac{\\partial L}{\\partial b} = \\frac{1}{N} \\sum_{i=1}^{N} 2(y_i - (w \\cdot x_i + b)) \\cdot \\frac{\\partial}{\\partial b} (y_i - (w \\cdot x_i + b))$$\n", 305 | "\n", 306 | "$$ = \\frac{1}{N} \\sum_{i=1}^{N} 2(y_i - (w \\cdot x_i + b)) \\cdot (-b)$$\n", 307 | "\n", 308 | "$$ = \\frac{1}{N} \\sum_{i=1}^{N} -2(y_i - w \\cdot x_i - b)$$\n", 309 | "\n", 310 | "\n", 311 | " \n", 312 | "\n", 313 | "\n", 314 | "\n", 315 | "\n" 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": null, 321 | "metadata": {}, 322 | "outputs": [], 323 | "source": [ 324 | "def grad_loss_w(x, y, w, b):\n", 325 | " return torch.mean(-2 * (y - w * x - b) * x)\n", 326 | "\n", 327 | "\n", 328 | "def grad_loss_b(x, y, w, b):\n", 329 | " return torch.mean(-2 * (y - w * x - b))" 330 | ] 331 | }, 332 | { 333 | "cell_type": "markdown", 334 | "metadata": {}, 335 | "source": [ 336 | "Initialize parameters $w$ and $b$ randomly." 337 | ] 338 | }, 339 | { 340 | "cell_type": "code", 341 | "execution_count": null, 342 | "metadata": {}, 343 | "outputs": [], 344 | "source": [ 345 | "w = 1e-2 * torch.randn(1, dtype=torch.float32)\n", 346 | "b = 1e-2 * torch.randn(1, dtype=torch.float32)\n", 347 | "\n", 348 | "print(f\"Initial weight : {w}\")\n", 349 | "print(f\"Initial bias : {b}\")" 350 | ] 351 | }, 352 | { 353 | "cell_type": "markdown", 354 | "metadata": {}, 355 | "source": [ 356 | "Optimize via gradient descent" 357 | ] 358 | }, 359 | { 360 | "cell_type": "code", 361 | "execution_count": null, 362 | "metadata": {}, 363 | "outputs": [], 364 | "source": [ 365 | "# Hyperparameters\n", 366 | "num_iters = 10000\n", 367 | "learning_rate = 3e-4\n", 368 | "\n", 369 | "# Loop over the number of iterations\n", 370 | "for it in range(num_iters):\n", 371 | " # predict y from x\n", 372 | " Y_pred = lin_model(w, b, X)\n", 373 | "\n", 374 | " # Compute the loss\n", 375 | " loss = loss_fn(Y_pred, Y)\n", 376 | "\n", 377 | " # Gradient of loss function w.r.t parameters\n", 378 | " grad_w = grad_loss_w(X, Y, w, b)\n", 379 | " grad_b = grad_loss_b(X, Y, w, b)\n", 380 | "\n", 381 | " # update parameters via gradient descent update rules\n", 382 | " with torch.no_grad():\n", 383 | " w -= learning_rate * grad_w\n", 384 | " b -= learning_rate * grad_b\n", 385 | "\n", 386 | " # Give some status output once in a while\n", 387 | " if it % 500 == 0 or it == num_iters - 1:\n", 388 | " print(\n", 389 | " f\"Iteration {it:5d} | Loss {loss.item():>10.5f} | \"\n", 390 | " f\"w {w.item():> 8.5f} | b {b.item():> 8.5f}\"\n", 391 | " )\n", 392 | "\n", 393 | "print(f\"Final weight after optimization : {w.item():.5f} (true: {w_true})\")\n", 394 | "print(f\"Final bias after optimization : {b.item():.5f} (true: {b_true})\")" 395 | ] 396 | }, 397 | { 398 | "cell_type": "markdown", 399 | "metadata": {}, 400 | "source": [ 401 | "Visualize linear fit" 402 | ] 403 | }, 404 | { 405 | "cell_type": "code", 406 | "execution_count": null, 407 | "metadata": {}, 408 | "outputs": [], 409 | "source": [ 410 | "# Visualize\n", 411 | "plt.scatter(X, Y, alpha=0.5)\n", 412 | "plt.title(\"Scatter plot of f(x) = 2x + 1 + Gaussian noise\")\n", 413 | "\n", 414 | "# Plot the recovered line\n", 415 | "Y_model = lin_model(w, b, X)\n", 416 | "plt.plot(X.tolist(), Y_model.tolist(), color=\"red\")\n", 417 | "\n", 418 | "plt.legend([\"data\", f\"f(x) = {w.item():.3f}x + {b.item():.3f}\"])\n", 419 | "plt.show()" 420 | ] 421 | }, 422 | { 423 | "cell_type": "markdown", 424 | "metadata": {}, 425 | "source": [ 426 | "Visualize the loss surface" 427 | ] 428 | }, 429 | { 430 | "cell_type": "code", 431 | "execution_count": null, 432 | "metadata": {}, 433 | "outputs": [], 434 | "source": [ 435 | "import torch\n", 436 | "import matplotlib.pyplot as plt\n", 437 | "from mpl_toolkits.mplot3d import Axes3D # Import the 3D plotting module\n", 438 | "\n", 439 | "# Assume loss_fn, lin_model, X, and Y are defined as in your existing code\n", 440 | "\n", 441 | "# Grid of w and b values\n", 442 | "w_grid = torch.arange(-5, 8, 0.1)\n", 443 | "b_grid = torch.arange(-5, 8, 0.1)\n", 444 | "\n", 445 | "# Create meshgrid for 3D plotting\n", 446 | "W, B = torch.meshgrid(w_grid, b_grid)\n", 447 | "\n", 448 | "# Loss values for each w,b combination\n", 449 | "loss_grid = torch.zeros((len(w_grid), len(b_grid)))\n", 450 | "\n", 451 | "# Loop over all w and b values\n", 452 | "for i, w_val in enumerate(w_grid):\n", 453 | " for j, b_val in enumerate(b_grid):\n", 454 | " loss_grid[i, j] = loss_fn(lin_model(w_val, b_val, X), Y)\n", 455 | "\n", 456 | "# Create a 3D plot\n", 457 | "fig = plt.figure(figsize=(15, 10))\n", 458 | "ax = fig.add_subplot(111, projection=\"3d\")\n", 459 | "\n", 460 | "# Plot the loss surface\n", 461 | "surf = ax.plot_surface(W.numpy(), B.numpy(), loss_grid.numpy(), cmap=\"coolwarm\")\n", 462 | "\n", 463 | "# Adjust camera position\n", 464 | "ax.view_init(elev=30, azim=60)\n", 465 | "\n", 466 | "# Adjust and label the axes\n", 467 | "ax.set_xlim(8, -5)\n", 468 | "ax.set_ylim(8, -5)\n", 469 | "ax.set_xlabel(\"w\")\n", 470 | "ax.set_ylabel(\"b\")\n", 471 | "ax.set_zlabel(\"Loss\")\n", 472 | "\n", 473 | "plt.title(\"Loss Surface\")\n", 474 | "plt.show()" 475 | ] 476 | } 477 | ], 478 | "metadata": { 479 | "colab": { 480 | "collapsed_sections": [ 481 | "JcF1mpJo-taz" 482 | ], 483 | "provenance": [] 484 | }, 485 | "kernelspec": { 486 | "display_name": "mlx-pytorch-2024-01", 487 | "language": "python", 488 | "name": "mlx-pytorch-2024-01" 489 | }, 490 | "language_info": { 491 | "codemirror_mode": { 492 | "name": "ipython", 493 | "version": 3 494 | }, 495 | "file_extension": ".py", 496 | "mimetype": "text/x-python", 497 | "name": "python", 498 | "nbconvert_exporter": "python", 499 | "pygments_lexer": "ipython3", 500 | "version": "3.11.6" 501 | } 502 | }, 503 | "nbformat": 4, 504 | "nbformat_minor": 0 505 | } 506 | -------------------------------------------------------------------------------- /linear_regression/torch/linear_regression_torch_optim.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "epxcwtWj5yJs" 7 | }, 8 | "source": [ 9 | "# Linear regression in PyTorch using torch.optim\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": { 19 | "id": "IJsjl_l47q28" 20 | }, 21 | "source": [ 22 | "## Setup\n", 23 | "\n", 24 | "Adapt `packagePath` to point to the directory containing this notebeook." 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": null, 30 | "metadata": { 31 | "executionInfo": { 32 | "elapsed": 236, 33 | "status": "ok", 34 | "timestamp": 1703326573912, 35 | "user": { 36 | "displayName": "Markus Enzweiler", 37 | "userId": "04524044579212347608" 38 | }, 39 | "user_tz": -60 40 | }, 41 | "id": "1iHkPBml98YG" 42 | }, 43 | "outputs": [], 44 | "source": [ 45 | "# Notebook id\n", 46 | "nb_id = \"linear_regression/torch\"\n", 47 | "\n", 48 | "# Imports\n", 49 | "import sys\n", 50 | "import os" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [ 59 | "# Package Path (folder of this notebook)\n", 60 | "\n", 61 | "#####################\n", 62 | "# Local environment #\n", 63 | "#####################\n", 64 | "\n", 65 | "package_path = \"./\"\n", 66 | "\n", 67 | "\n", 68 | "#########\n", 69 | "# Colab #\n", 70 | "#########\n", 71 | "\n", 72 | "\n", 73 | "def check_for_colab():\n", 74 | " try:\n", 75 | " import google.colab\n", 76 | "\n", 77 | " return True\n", 78 | " except ImportError:\n", 79 | " return False\n", 80 | "\n", 81 | "\n", 82 | "# running on Colab?\n", 83 | "on_colab = check_for_colab()\n", 84 | "\n", 85 | "if on_colab:\n", 86 | " # assume this notebook is run from Google Drive and the whole\n", 87 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 88 | "\n", 89 | " # Google Drive mount point\n", 90 | " gdrive_mnt = \"/content/drive\"\n", 91 | "\n", 92 | " ##########################################################################\n", 93 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 94 | " ##########################################################################\n", 95 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 96 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 97 | "\n", 98 | " # mount drive\n", 99 | " from google.colab import drive\n", 100 | "\n", 101 | " drive.mount(gdrive_mnt, force_remount=True)\n", 102 | "\n", 103 | " # set package path\n", 104 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 105 | "\n", 106 | "# check whether package path exists\n", 107 | "if not os.path.isdir(package_path):\n", 108 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 109 | "\n", 110 | "print(f\"Package path: {package_path}\")" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": null, 116 | "metadata": { 117 | "colab": { 118 | "base_uri": "https://localhost:8080/" 119 | }, 120 | "executionInfo": { 121 | "elapsed": 2602, 122 | "status": "ok", 123 | "timestamp": 1703326576880, 124 | "user": { 125 | "displayName": "Markus Enzweiler", 126 | "userId": "04524044579212347608" 127 | }, 128 | "user_tz": -60 129 | }, 130 | "id": "DY4880S378_F", 131 | "outputId": "a8ce2d26-fe24-4aa2-c232-acc31b71d394" 132 | }, 133 | "outputs": [], 134 | "source": [ 135 | "# Additional imports\n", 136 | "\n", 137 | "# Repository Root\n", 138 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 139 | "# Add the repository root to the system path\n", 140 | "if repo_root not in sys.path:\n", 141 | " sys.path.append(repo_root)\n", 142 | "\n", 143 | "# Package Imports\n", 144 | "from nbutils import requirements as nb_reqs\n", 145 | "from nbutils import colab as nb_colab\n", 146 | "from nbutils import git as nb_git\n", 147 | "from nbutils import exec as nb_exec\n", 148 | "from nbutils import data as nb_data" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": null, 154 | "metadata": { 155 | "colab": { 156 | "base_uri": "https://localhost:8080/" 157 | }, 158 | "executionInfo": { 159 | "elapsed": 6888, 160 | "status": "ok", 161 | "timestamp": 1703326583765, 162 | "user": { 163 | "displayName": "Markus Enzweiler", 164 | "userId": "04524044579212347608" 165 | }, 166 | "user_tz": -60 167 | }, 168 | "id": "iaURjw5n6pLq", 169 | "outputId": "5c40ee52-1fa3-44df-a94e-b81bec144100" 170 | }, 171 | "outputs": [], 172 | "source": [ 173 | "# Install requirements in the current Jupyter kernel\n", 174 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 175 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 176 | ] 177 | }, 178 | { 179 | "cell_type": "code", 180 | "execution_count": null, 181 | "metadata": { 182 | "executionInfo": { 183 | "elapsed": 12, 184 | "status": "ok", 185 | "timestamp": 1703326583765, 186 | "user": { 187 | "displayName": "Markus Enzweiler", 188 | "userId": "04524044579212347608" 189 | }, 190 | "user_tz": -60 191 | }, 192 | "id": "iRERDI8aAnzr" 193 | }, 194 | "outputs": [], 195 | "source": [ 196 | "# Now we should be able to import the additional packages\n", 197 | "import torch\n", 198 | "import numpy as np\n", 199 | "import matplotlib.pyplot as plt\n", 200 | "\n", 201 | "# Set the random seed for reproducibility\n", 202 | "torch.manual_seed(0);" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": { 208 | "id": "aUACL9QoFTXg" 209 | }, 210 | "source": [ 211 | "## Linear regression" 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "metadata": {}, 217 | "source": [ 218 | "### Create some data based on adding noise to a known linear function" 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "execution_count": null, 224 | "metadata": {}, 225 | "outputs": [], 226 | "source": [ 227 | "# Creating a function f(x) with a slope of 2 and bias of 1, e.g. f(x) = 2x + 1\n", 228 | "# and added Gaussian noise\n", 229 | "\n", 230 | "# True parameters\n", 231 | "w_true = 2\n", 232 | "b_true = 1\n", 233 | "params_true = torch.tensor([w_true, b_true])\n", 234 | "\n", 235 | "X = torch.arange(-5, 5, 0.1)\n", 236 | "Y = w_true * X + b_true + 2 * torch.randn(X.shape)\n", 237 | "\n", 238 | "# Visualize\n", 239 | "plt.scatter(X, Y, alpha=0.5)\n", 240 | "plt.title(\"Scatter plot of f(x) = 2x + 1 + Gaussian noise\")\n", 241 | "plt.show()" 242 | ] 243 | }, 244 | { 245 | "cell_type": "markdown", 246 | "metadata": {}, 247 | "source": [ 248 | "### Linear model and loss\n", 249 | "\n", 250 | "Our linear regression model is $ y = f(x) = w \\cdot x + b$. We solve for $w$ and $b$ using gradient descent. " 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": null, 256 | "metadata": {}, 257 | "outputs": [], 258 | "source": [ 259 | "def lin_model(params, x):\n", 260 | " w, b = params\n", 261 | " return w * x + b" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "metadata": {}, 267 | "source": [ 268 | "We uses mean squared error loss between the predictions of our model and true values. " 269 | ] 270 | }, 271 | { 272 | "cell_type": "code", 273 | "execution_count": null, 274 | "metadata": {}, 275 | "outputs": [], 276 | "source": [ 277 | "def loss_fn(y_pred, y):\n", 278 | " return torch.mean(torch.square(y - y_pred))" 279 | ] 280 | }, 281 | { 282 | "cell_type": "markdown", 283 | "metadata": {}, 284 | "source": [ 285 | "### Optimization via gradient descent" 286 | ] 287 | }, 288 | { 289 | "cell_type": "markdown", 290 | "metadata": {}, 291 | "source": [ 292 | "Initialize parameters $w$ and $b$ randomly." 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": null, 298 | "metadata": {}, 299 | "outputs": [], 300 | "source": [ 301 | "params = 1e-2 * torch.randn(2, dtype=torch.float32)\n", 302 | "\n", 303 | "# we track gradients for the parameters w and b\n", 304 | "params.requires_grad_()\n", 305 | "\n", 306 | "w, b = params\n", 307 | "print(f\"Initial weight : {w}\")\n", 308 | "print(f\"Initial bias : {b}\")" 309 | ] 310 | }, 311 | { 312 | "cell_type": "markdown", 313 | "metadata": {}, 314 | "source": [ 315 | "Optimize via gradient descent" 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": null, 321 | "metadata": {}, 322 | "outputs": [], 323 | "source": [ 324 | "# Hyperparameters\n", 325 | "num_iters = 10000\n", 326 | "learning_rate = 3e-4\n", 327 | "\n", 328 | "# Optimizer\n", 329 | "optimizer = torch.optim.Adam([params], lr=learning_rate)\n", 330 | "\n", 331 | "# Loop over the number of iterations\n", 332 | "for it in range(num_iters):\n", 333 | " # predict y from x\n", 334 | " Y_pred = lin_model(params, X)\n", 335 | "\n", 336 | " # Compute the loss\n", 337 | " loss = loss_fn(Y_pred, Y)\n", 338 | "\n", 339 | " # Gradient of loss function w.r.t parameters\n", 340 | " loss.backward()\n", 341 | "\n", 342 | " # update parameters via gradient descent update rules of selected optimizer\n", 343 | " optimizer.step()\n", 344 | " # zero the gradients for the next iteration\n", 345 | " optimizer.zero_grad()\n", 346 | "\n", 347 | " # Give some status output once in a while\n", 348 | " if it % 500 == 0 or it == num_iters - 1:\n", 349 | " w, b = params\n", 350 | " error_norm = torch.sum(torch.square(params - params_true)) ** 0.5\n", 351 | " print(\n", 352 | " f\"Iteration {it:5d} | Loss {loss.item():>10.5f} | \"\n", 353 | " f\"w {w.item():> 8.5f} | b {b.item():> 8.5f} | Error norm {error_norm.item():>.5f}\"\n", 354 | " )\n", 355 | "\n", 356 | "w, b = params\n", 357 | "print(f\"Final weight after optimization : {w.item():.5f} (true: {w_true})\")\n", 358 | "print(f\"Final bias after optimization : {b.item():.5f} (true: {b_true})\")" 359 | ] 360 | }, 361 | { 362 | "cell_type": "markdown", 363 | "metadata": {}, 364 | "source": [ 365 | "Visualize linear fit" 366 | ] 367 | }, 368 | { 369 | "cell_type": "code", 370 | "execution_count": null, 371 | "metadata": {}, 372 | "outputs": [], 373 | "source": [ 374 | "# Visualize\n", 375 | "plt.scatter(X, Y, alpha=0.5)\n", 376 | "plt.title(\"Scatter plot of f(x) = 2x + 1 + Gaussian noise\")\n", 377 | "\n", 378 | "# Plot the recovered line\n", 379 | "Y_model = lin_model(params, X)\n", 380 | "plt.plot(X.tolist(), Y_model.tolist(), color=\"red\")\n", 381 | "\n", 382 | "plt.legend([\"data\", f\"f(x) = {w.item():.3f}x + {b.item():.3f}\"])\n", 383 | "plt.show()" 384 | ] 385 | } 386 | ], 387 | "metadata": { 388 | "colab": { 389 | "collapsed_sections": [ 390 | "JcF1mpJo-taz" 391 | ], 392 | "provenance": [] 393 | }, 394 | "kernelspec": { 395 | "display_name": "mlx-pytorch-2024-01", 396 | "language": "python", 397 | "name": "mlx-pytorch-2024-01" 398 | }, 399 | "language_info": { 400 | "codemirror_mode": { 401 | "name": "ipython", 402 | "version": 3 403 | }, 404 | "file_extension": ".py", 405 | "mimetype": "text/x-python", 406 | "name": "python", 407 | "nbconvert_exporter": "python", 408 | "pygments_lexer": "ipython3", 409 | "version": "3.11.6" 410 | } 411 | }, 412 | "nbformat": 4, 413 | "nbformat_minor": 0 414 | } 415 | -------------------------------------------------------------------------------- /linear_regression/torch/requirements.txt: -------------------------------------------------------------------------------- 1 | torch 2 | numpy 3 | matplotlib -------------------------------------------------------------------------------- /multi_layer_perceptron/mlx/mlp_XOR_mlx.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "epxcwtWj5yJs" 7 | }, 8 | "source": [ 9 | "# Multi-Layer Perceptron for the XOR problem in MLX\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": {}, 19 | "source": [ 20 | "We build and train a multi-layer perceptron to act as a simple XOR gate with two inputs and one output. \n", 21 | "XOR gates have the following behavior:\n", 22 | "\n", 23 | "If both inputs are identical, the output is 0 (off)\n", 24 | "If both inputs are different, the output is 1 (on)\n", 25 | "\n", 26 | "| observation # | input 1 | input 2 | output |\n", 27 | "|---------------|---------|---------|--------|\n", 28 | "| 0 | 0 | 0 | 0 |\n", 29 | "| 1 | 0 | 1 | 1 |\n", 30 | "| 2 | 1 | 0 | 1 |\n", 31 | "| 3 | 1 | 1 | 0 |" 32 | ] 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "metadata": {}, 37 | "source": [ 38 | "\n", 39 | "**Note: This requires a machine with an Apple SoC, e.g. M1/M2/M3 etc.**\n", 40 | "\n", 41 | "See: https://github.com/ml-explore/mlx" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": { 47 | "id": "IJsjl_l47q28" 48 | }, 49 | "source": [ 50 | "## Setup\n", 51 | "\n", 52 | "Adapt `packagePath` to point to the directory containing this notebeook." 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "metadata": { 59 | "executionInfo": { 60 | "elapsed": 236, 61 | "status": "ok", 62 | "timestamp": 1703326573912, 63 | "user": { 64 | "displayName": "Markus Enzweiler", 65 | "userId": "04524044579212347608" 66 | }, 67 | "user_tz": -60 68 | }, 69 | "id": "1iHkPBml98YG" 70 | }, 71 | "outputs": [], 72 | "source": [ 73 | "# Notebook id\n", 74 | "nb_id = \"multi_layer_perceptron/mlx\"\n", 75 | "\n", 76 | "# Imports\n", 77 | "import sys\n", 78 | "import os" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "metadata": {}, 85 | "outputs": [], 86 | "source": [ 87 | "# Package Path (folder of this notebook)\n", 88 | "\n", 89 | "#####################\n", 90 | "# Local environment #\n", 91 | "#####################\n", 92 | "\n", 93 | "package_path = \"./\"\n", 94 | "\n", 95 | "\n", 96 | "#########\n", 97 | "# Colab #\n", 98 | "#########\n", 99 | "\n", 100 | "\n", 101 | "def check_for_colab():\n", 102 | " try:\n", 103 | " import google.colab\n", 104 | "\n", 105 | " return True\n", 106 | " except ImportError:\n", 107 | " return False\n", 108 | "\n", 109 | "\n", 110 | "# running on Colab?\n", 111 | "on_colab = check_for_colab()\n", 112 | "\n", 113 | "if on_colab:\n", 114 | " # assume this notebook is run from Google Drive and the whole\n", 115 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 116 | "\n", 117 | " # Google Drive mount point\n", 118 | " gdrive_mnt = \"/content/drive\"\n", 119 | "\n", 120 | " ##########################################################################\n", 121 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 122 | " ##########################################################################\n", 123 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 124 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 125 | "\n", 126 | " # mount drive\n", 127 | " from google.colab import drive\n", 128 | "\n", 129 | " drive.mount(gdrive_mnt, force_remount=True)\n", 130 | "\n", 131 | " # set package path\n", 132 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 133 | "\n", 134 | "# check whether package path exists\n", 135 | "if not os.path.isdir(package_path):\n", 136 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 137 | "\n", 138 | "print(f\"Package path: {package_path}\")" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": null, 144 | "metadata": { 145 | "colab": { 146 | "base_uri": "https://localhost:8080/" 147 | }, 148 | "executionInfo": { 149 | "elapsed": 2602, 150 | "status": "ok", 151 | "timestamp": 1703326576880, 152 | "user": { 153 | "displayName": "Markus Enzweiler", 154 | "userId": "04524044579212347608" 155 | }, 156 | "user_tz": -60 157 | }, 158 | "id": "DY4880S378_F", 159 | "outputId": "a8ce2d26-fe24-4aa2-c232-acc31b71d394" 160 | }, 161 | "outputs": [], 162 | "source": [ 163 | "# Additional imports\n", 164 | "\n", 165 | "# Repository Root\n", 166 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 167 | "# Add the repository root to the system path\n", 168 | "if repo_root not in sys.path:\n", 169 | " sys.path.append(repo_root)\n", 170 | "\n", 171 | "# Package Imports\n", 172 | "from nbutils import requirements as nb_reqs\n", 173 | "from nbutils import colab as nb_colab\n", 174 | "from nbutils import git as nb_git\n", 175 | "from nbutils import exec as nb_exec\n", 176 | "from nbutils import data as nb_data" 177 | ] 178 | }, 179 | { 180 | "cell_type": "code", 181 | "execution_count": null, 182 | "metadata": { 183 | "colab": { 184 | "base_uri": "https://localhost:8080/" 185 | }, 186 | "executionInfo": { 187 | "elapsed": 6888, 188 | "status": "ok", 189 | "timestamp": 1703326583765, 190 | "user": { 191 | "displayName": "Markus Enzweiler", 192 | "userId": "04524044579212347608" 193 | }, 194 | "user_tz": -60 195 | }, 196 | "id": "iaURjw5n6pLq", 197 | "outputId": "5c40ee52-1fa3-44df-a94e-b81bec144100" 198 | }, 199 | "outputs": [], 200 | "source": [ 201 | "# Install requirements in the current Jupyter kernel\n", 202 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 203 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": null, 209 | "metadata": { 210 | "executionInfo": { 211 | "elapsed": 12, 212 | "status": "ok", 213 | "timestamp": 1703326583765, 214 | "user": { 215 | "displayName": "Markus Enzweiler", 216 | "userId": "04524044579212347608" 217 | }, 218 | "user_tz": -60 219 | }, 220 | "id": "iRERDI8aAnzr" 221 | }, 222 | "outputs": [], 223 | "source": [ 224 | "# Now we should be able to import the additional packages\n", 225 | "import numpy as np\n", 226 | "import matplotlib.pyplot as plt\n", 227 | "import mlx\n", 228 | "import mlx.core as mx\n", 229 | "import mlx.nn as nn\n", 230 | "import mlx.optimizers as optim\n", 231 | "\n", 232 | "# Set the random seed for reproducibility\n", 233 | "np.random.seed(42)\n", 234 | "mx.random.seed(42)" 235 | ] 236 | }, 237 | { 238 | "cell_type": "markdown", 239 | "metadata": { 240 | "id": "aUACL9QoFTXg" 241 | }, 242 | "source": [ 243 | "## Create the training data" 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": null, 249 | "metadata": {}, 250 | "outputs": [], 251 | "source": [ 252 | "# Define the training data for the XOR problem in numpy\n", 253 | "\n", 254 | "\n", 255 | "# Define the training data for the XOR problem\n", 256 | "X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])\n", 257 | "Y = np.array([0, 1, 1, 0])\n", 258 | "\n", 259 | "# Convert numpy arrays to mlx tensors\n", 260 | "X = mx.array(X, dtype=mx.float32)\n", 261 | "Y = mx.array(Y, dtype=mx.float32)\n", 262 | "\n", 263 | "print(\"Training data X with labels y:\")\n", 264 | "for i in range(len(X)):\n", 265 | " print(f\"{X[i]} -> {Y[i]}\")" 266 | ] 267 | }, 268 | { 269 | "cell_type": "markdown", 270 | "metadata": {}, 271 | "source": [ 272 | "# Define the Multi-Layer Perceptron (MLP)" 273 | ] 274 | }, 275 | { 276 | "cell_type": "markdown", 277 | "metadata": {}, 278 | "source": [ 279 | "## MLP class" 280 | ] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": null, 285 | "metadata": {}, 286 | "outputs": [], 287 | "source": [ 288 | "class MultiLayerPerceptron(nn.Module):\n", 289 | " # override constructor from nn.Module\n", 290 | " def __init__(self, num_inputs, num_hidden_layer_neurons=2):\n", 291 | " super().__init__() ## call constructor of nn.Module\n", 292 | "\n", 293 | " # layer 1 defines the transformation from input to hidden layer\n", 294 | " self.layer1 = nn.Linear(\n", 295 | " input_dims=num_inputs, output_dims=num_hidden_layer_neurons\n", 296 | " )\n", 297 | " # layer 2 defines the transformation from hidden layer to output\n", 298 | " self.layer2 = nn.Linear(input_dims=num_hidden_layer_neurons, output_dims=1)\n", 299 | " self.sigmoid = mx.sigmoid\n", 300 | "\n", 301 | " def __call__(self, x):\n", 302 | " return self.forward(x)\n", 303 | "\n", 304 | " def forward(self, x):\n", 305 | " # x (input) -> hidden layer -> sigmoid -> output layer -> sigmoid\n", 306 | " x = self.sigmoid(self.layer1(x))\n", 307 | " x = self.sigmoid(self.layer2(x))\n", 308 | " return x" 309 | ] 310 | }, 311 | { 312 | "cell_type": "markdown", 313 | "metadata": {}, 314 | "source": [ 315 | "# MLP training with gradient descent" 316 | ] 317 | }, 318 | { 319 | "cell_type": "markdown", 320 | "metadata": {}, 321 | "source": [ 322 | "## Training and testing functions" 323 | ] 324 | }, 325 | { 326 | "cell_type": "code", 327 | "execution_count": null, 328 | "metadata": {}, 329 | "outputs": [], 330 | "source": [ 331 | "# Training function\n", 332 | "def train(model, X, Y, optimizer, loss_and_grad_fn, num_epochs):\n", 333 | " # Loop over epochs\n", 334 | " for epoch in range(num_epochs):\n", 335 | " # Reset accumulated loss per epoch\n", 336 | " acc_loss = 0\n", 337 | "\n", 338 | " # Loop over all training data\n", 339 | " for i in range(len(X)):\n", 340 | " # forward and backward pass\n", 341 | " loss, gradients = loss_and_grad_fn(\n", 342 | " model,\n", 343 | " X[i],\n", 344 | " Y[i].reshape(\n", 345 | " 1,\n", 346 | " ),\n", 347 | " )\n", 348 | " acc_loss += loss\n", 349 | "\n", 350 | " # Update the model with the gradients. So far no computation has happened.\n", 351 | " optimizer.update(model, gradients)\n", 352 | "\n", 353 | " # Compute the new parameters and also the new optimizer state.\n", 354 | " mx.eval(model.parameters(), optimizer.state)\n", 355 | "\n", 356 | " # Print accumulated average loss per epoch once in a while\n", 357 | " if (epoch % (num_epochs // 10)) == 0 or epoch == num_epochs - 1:\n", 358 | " print(f\"Epoch {epoch:5d}: loss = {mx.mean(acc_loss).item():.5f}\")" 359 | ] 360 | }, 361 | { 362 | "cell_type": "code", 363 | "execution_count": null, 364 | "metadata": {}, 365 | "outputs": [], 366 | "source": [ 367 | "# Testing function\n", 368 | "def test(model, X, Y):\n", 369 | " # test the model on all data points\n", 370 | " print(\"Testing ...\")\n", 371 | " for i in range(len(X)):\n", 372 | " prediction = model(X[i])\n", 373 | " print(f\"{X[i]} -> {prediction} (label: {Y[i]})\")" 374 | ] 375 | }, 376 | { 377 | "cell_type": "markdown", 378 | "metadata": {}, 379 | "source": [ 380 | "## Train and test" 381 | ] 382 | }, 383 | { 384 | "cell_type": "code", 385 | "execution_count": null, 386 | "metadata": {}, 387 | "outputs": [], 388 | "source": [ 389 | "# Perceptron for our OR problem\n", 390 | "model = MultiLayerPerceptron(num_inputs=2)\n", 391 | "# Evaluate because mlx uses lazy evaluation\n", 392 | "mx.eval(model.parameters())\n", 393 | "\n", 394 | "# Hyperparameters\n", 395 | "num_epochs = 10000\n", 396 | "eta = 0.25\n", 397 | "\n", 398 | "\n", 399 | "# Loss function\n", 400 | "def loss_fn(model, X, y):\n", 401 | " return nn.losses.mse_loss((model(X)), y)\n", 402 | "\n", 403 | "\n", 404 | "# Create the gradient function\n", 405 | "loss_and_grad_fn = nn.value_and_grad(model, loss_fn)\n", 406 | "\n", 407 | "# Stochastic gradient descent (SGD) optimizer\n", 408 | "optimizer = optim.SGD(learning_rate=eta)\n", 409 | "\n", 410 | "# Train the model\n", 411 | "train(model, X, Y, optimizer, loss_and_grad_fn, num_epochs)\n", 412 | "\n", 413 | "# Test the model\n", 414 | "test(model, X, Y)" 415 | ] 416 | }, 417 | { 418 | "cell_type": "markdown", 419 | "metadata": {}, 420 | "source": [ 421 | "# Visualize decision boundary" 422 | ] 423 | }, 424 | { 425 | "cell_type": "code", 426 | "execution_count": null, 427 | "metadata": {}, 428 | "outputs": [], 429 | "source": [ 430 | "import matplotlib.cm as cm\n", 431 | "import matplotlib.gridspec as gridspec\n", 432 | "\n", 433 | "\n", 434 | "def show_decision_boundary(model, data, labels, subplot_spec=None):\n", 435 | " data = np.array(data)\n", 436 | " labels = np.array(labels)\n", 437 | "\n", 438 | " wratio = (15, 1)\n", 439 | " if subplot_spec is None:\n", 440 | " gs = gridspec.GridSpec(1, 2, width_ratios=wratio)\n", 441 | " else:\n", 442 | " gs = gridspec.GridSpecFromSubplotSpec(\n", 443 | " 1, 2, subplot_spec=subplot_spec, width_ratios=wratio\n", 444 | " )\n", 445 | "\n", 446 | " ax = plt.subplot(gs[0])\n", 447 | " ax.set_title(\"Dataset and decision function\")\n", 448 | "\n", 449 | " x_min, x_max = data[:, 0].min() - 1, data[:, 0].max() + 1\n", 450 | " y_min, y_max = data[:, 1].min() - 1, data[:, 1].max() + 1\n", 451 | " h = 0.01 # Reduced step size for higher resolution\n", 452 | " xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))\n", 453 | "\n", 454 | " Z = model(mx.array(np.c_[xx.ravel(), yy.ravel()], dtype=mx.float32))\n", 455 | " Z = Z.reshape(xx.shape)\n", 456 | "\n", 457 | " # Increase the number of levels for smoother color transitions\n", 458 | " levels = np.linspace(0, 1, 100)\n", 459 | " ctr = ax.contourf(xx, yy, np.array(Z), levels, cmap=cm.gray, vmin=0, vmax=1)\n", 460 | "\n", 461 | " unique_labels = np.unique(labels)\n", 462 | "\n", 463 | " # Define colors for each class\n", 464 | " colors = [\"red\", \"blue\"]\n", 465 | " for i, yi in enumerate(unique_labels):\n", 466 | " color = colors[i]\n", 467 | " ax.scatter(\n", 468 | " data[np.where(labels.flatten() == yi), 0],\n", 469 | " data[np.where(labels.flatten() == yi), 1],\n", 470 | " color=color,\n", 471 | " linewidth=0,\n", 472 | " label=\"Class %d (y=%d)\" % (yi, yi),\n", 473 | " )\n", 474 | " ax.legend()\n", 475 | " ax.set_xlim((x_min, x_max))\n", 476 | " ax.set_ylim((y_min, y_max))\n", 477 | "\n", 478 | " # Create colorbar\n", 479 | " cbar = plt.colorbar(ctr, cax=plt.subplot(gs[1]))\n", 480 | " cbar.set_ticks(np.arange(0, 1.1, 0.1)) # Set ticks from 0 to 1 with 0.1 increments\n", 481 | " cbar.set_label(\"Decision value\")" 482 | ] 483 | }, 484 | { 485 | "cell_type": "code", 486 | "execution_count": null, 487 | "metadata": {}, 488 | "outputs": [], 489 | "source": [ 490 | "# Plot decision boundary\n", 491 | "show_decision_boundary(model, X, Y)" 492 | ] 493 | } 494 | ], 495 | "metadata": { 496 | "colab": { 497 | "collapsed_sections": [ 498 | "JcF1mpJo-taz" 499 | ], 500 | "provenance": [] 501 | }, 502 | "kernelspec": { 503 | "display_name": "mlx-m1-2023-12", 504 | "language": "python", 505 | "name": "python3" 506 | }, 507 | "language_info": { 508 | "codemirror_mode": { 509 | "name": "ipython", 510 | "version": 3 511 | }, 512 | "file_extension": ".py", 513 | "mimetype": "text/x-python", 514 | "name": "python", 515 | "nbconvert_exporter": "python", 516 | "pygments_lexer": "ipython3", 517 | "version": "3.11.6" 518 | } 519 | }, 520 | "nbformat": 4, 521 | "nbformat_minor": 0 522 | } 523 | -------------------------------------------------------------------------------- /multi_layer_perceptron/mlx/requirements.txt: -------------------------------------------------------------------------------- 1 | mlx 2 | numpy 3 | matplotlib 4 | -------------------------------------------------------------------------------- /multi_layer_perceptron/torch/mlp_XOR_torch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "epxcwtWj5yJs" 7 | }, 8 | "source": [ 9 | "# Multi-Layer Perceptron for the XOR problem in PyTorch\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": {}, 19 | "source": [ 20 | "We build and train a multi-layer perceptron to act as a simple XOR gate with two inputs and one output. \n", 21 | "XOR gates have the following behavior:\n", 22 | "\n", 23 | "If both inputs are identical, the output is 0 (off)\n", 24 | "If both inputs are different, the output is 1 (on)\n", 25 | "\n", 26 | "| observation # | input 1 | input 2 | output |\n", 27 | "|---------------|---------|---------|--------|\n", 28 | "| 0 | 0 | 0 | 0 |\n", 29 | "| 1 | 0 | 1 | 1 |\n", 30 | "| 2 | 1 | 0 | 1 |\n", 31 | "| 3 | 1 | 1 | 0 |\n", 32 | "\n" 33 | ] 34 | }, 35 | { 36 | "cell_type": "markdown", 37 | "metadata": {}, 38 | "source": [ 39 | "In this demonstration, we replace our custom Perceptron class from the previous notebooks and use torch.nn\n", 40 | "\n", 41 | "See https://pytorch.org/docs/stable/nn.html and in particular:\n", 42 | "- https://pytorch.org/docs/stable/generated/torch.nn.Module.html \n", 43 | "- https://pytorch.org/docs/stable/generated/torch.nn.Linear.html " 44 | ] 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "metadata": { 49 | "id": "IJsjl_l47q28" 50 | }, 51 | "source": [ 52 | "## Setup\n", 53 | "\n", 54 | "Adapt `packagePath` to point to the directory containing this notebeook." 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": null, 60 | "metadata": { 61 | "executionInfo": { 62 | "elapsed": 236, 63 | "status": "ok", 64 | "timestamp": 1703326573912, 65 | "user": { 66 | "displayName": "Markus Enzweiler", 67 | "userId": "04524044579212347608" 68 | }, 69 | "user_tz": -60 70 | }, 71 | "id": "1iHkPBml98YG" 72 | }, 73 | "outputs": [], 74 | "source": [ 75 | "# Notebook id\n", 76 | "nb_id = \"multi_layer_perceptron/torch\"\n", 77 | "\n", 78 | "# Imports\n", 79 | "import sys\n", 80 | "import os" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": null, 86 | "metadata": {}, 87 | "outputs": [], 88 | "source": [ 89 | "# Package Path (folder of this notebook)\n", 90 | "\n", 91 | "#####################\n", 92 | "# Local environment #\n", 93 | "#####################\n", 94 | "\n", 95 | "package_path = \"./\"\n", 96 | "\n", 97 | "\n", 98 | "#########\n", 99 | "# Colab #\n", 100 | "#########\n", 101 | "\n", 102 | "\n", 103 | "def check_for_colab():\n", 104 | " try:\n", 105 | " import google.colab\n", 106 | "\n", 107 | " return True\n", 108 | " except ImportError:\n", 109 | " return False\n", 110 | "\n", 111 | "\n", 112 | "# running on Colab?\n", 113 | "on_colab = check_for_colab()\n", 114 | "\n", 115 | "if on_colab:\n", 116 | " # assume this notebook is run from Google Drive and the whole\n", 117 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 118 | "\n", 119 | " # Google Drive mount point\n", 120 | " gdrive_mnt = \"/content/drive\"\n", 121 | "\n", 122 | " ##########################################################################\n", 123 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 124 | " ##########################################################################\n", 125 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 126 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 127 | "\n", 128 | " # mount drive\n", 129 | " from google.colab import drive\n", 130 | "\n", 131 | " drive.mount(gdrive_mnt, force_remount=True)\n", 132 | "\n", 133 | " # set package path\n", 134 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 135 | "\n", 136 | "# check whether package path exists\n", 137 | "if not os.path.isdir(package_path):\n", 138 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 139 | "\n", 140 | "print(f\"Package path: {package_path}\")" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": { 147 | "colab": { 148 | "base_uri": "https://localhost:8080/" 149 | }, 150 | "executionInfo": { 151 | "elapsed": 2602, 152 | "status": "ok", 153 | "timestamp": 1703326576880, 154 | "user": { 155 | "displayName": "Markus Enzweiler", 156 | "userId": "04524044579212347608" 157 | }, 158 | "user_tz": -60 159 | }, 160 | "id": "DY4880S378_F", 161 | "outputId": "a8ce2d26-fe24-4aa2-c232-acc31b71d394" 162 | }, 163 | "outputs": [], 164 | "source": [ 165 | "# Additional imports\n", 166 | "\n", 167 | "# Repository Root\n", 168 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 169 | "# Add the repository root to the system path\n", 170 | "if repo_root not in sys.path:\n", 171 | " sys.path.append(repo_root)\n", 172 | "\n", 173 | "# Package Imports\n", 174 | "from nbutils import requirements as nb_reqs\n", 175 | "from nbutils import colab as nb_colab\n", 176 | "from nbutils import git as nb_git\n", 177 | "from nbutils import exec as nb_exec\n", 178 | "from nbutils import data as nb_data" 179 | ] 180 | }, 181 | { 182 | "cell_type": "code", 183 | "execution_count": null, 184 | "metadata": { 185 | "colab": { 186 | "base_uri": "https://localhost:8080/" 187 | }, 188 | "executionInfo": { 189 | "elapsed": 6888, 190 | "status": "ok", 191 | "timestamp": 1703326583765, 192 | "user": { 193 | "displayName": "Markus Enzweiler", 194 | "userId": "04524044579212347608" 195 | }, 196 | "user_tz": -60 197 | }, 198 | "id": "iaURjw5n6pLq", 199 | "outputId": "5c40ee52-1fa3-44df-a94e-b81bec144100" 200 | }, 201 | "outputs": [], 202 | "source": [ 203 | "# Install requirements in the current Jupyter kernel\n", 204 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 205 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "execution_count": null, 211 | "metadata": { 212 | "executionInfo": { 213 | "elapsed": 12, 214 | "status": "ok", 215 | "timestamp": 1703326583765, 216 | "user": { 217 | "displayName": "Markus Enzweiler", 218 | "userId": "04524044579212347608" 219 | }, 220 | "user_tz": -60 221 | }, 222 | "id": "iRERDI8aAnzr" 223 | }, 224 | "outputs": [], 225 | "source": [ 226 | "# Now we should be able to import the additional packages\n", 227 | "import numpy as np\n", 228 | "import matplotlib.pyplot as plt\n", 229 | "import torch\n", 230 | "import torch.nn as nn\n", 231 | "\n", 232 | "# Set the random seed for reproducibility\n", 233 | "np.random.seed(42)\n", 234 | "torch.manual_seed(42);" 235 | ] 236 | }, 237 | { 238 | "cell_type": "markdown", 239 | "metadata": { 240 | "id": "aUACL9QoFTXg" 241 | }, 242 | "source": [ 243 | "## Create the training data" 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": null, 249 | "metadata": {}, 250 | "outputs": [], 251 | "source": [ 252 | "# Define the training data for the XOR problem in numpy\n", 253 | "\n", 254 | "\n", 255 | "# Define the training data for the XOR problem\n", 256 | "X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])\n", 257 | "Y = np.array([0, 1, 1, 0])\n", 258 | "\n", 259 | "# Convert numpy arrays to PyTorch tensors\n", 260 | "X = torch.tensor(X, dtype=torch.float32)\n", 261 | "Y = torch.tensor(Y, dtype=torch.float32)\n", 262 | "\n", 263 | "print(\"Training data X with labels y:\")\n", 264 | "for i in range(len(X)):\n", 265 | " print(f\"{X[i]} -> {Y[i]}\")" 266 | ] 267 | }, 268 | { 269 | "cell_type": "markdown", 270 | "metadata": {}, 271 | "source": [ 272 | "# Define the Multi-Layer Perceptron (MLP)" 273 | ] 274 | }, 275 | { 276 | "cell_type": "markdown", 277 | "metadata": {}, 278 | "source": [ 279 | "## MLP class" 280 | ] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": null, 285 | "metadata": {}, 286 | "outputs": [], 287 | "source": [ 288 | "# Multi-layer perceptron model\n", 289 | "class MultiLayerPerceptron(nn.Module):\n", 290 | " def __init__(self, num_inputs, num_hidden_layer_neurons=2):\n", 291 | " super().__init__()\n", 292 | "\n", 293 | " # layer 1 defines the transformation from input to hidden layer\n", 294 | " self.layer1 = nn.Linear(num_inputs, num_hidden_layer_neurons)\n", 295 | " # layer 2 defines the transformation from hidden layer to output\n", 296 | " self.layer2 = nn.Linear(num_hidden_layer_neurons, 1)\n", 297 | " # sigmoid activation function\n", 298 | " self.sigmoid = nn.Sigmoid()\n", 299 | "\n", 300 | " def __call__(self, x):\n", 301 | " return self.forward(x)\n", 302 | "\n", 303 | " def forward(self, x):\n", 304 | " # x (input) -> hidden layer -> sigmoid -> output layer -> sigmoid\n", 305 | " x = self.sigmoid(self.layer1(x))\n", 306 | " x = self.sigmoid(self.layer2(x))\n", 307 | " return x" 308 | ] 309 | }, 310 | { 311 | "cell_type": "markdown", 312 | "metadata": {}, 313 | "source": [ 314 | "# MLP training with gradient descent" 315 | ] 316 | }, 317 | { 318 | "cell_type": "markdown", 319 | "metadata": {}, 320 | "source": [ 321 | "## Training and testing functions" 322 | ] 323 | }, 324 | { 325 | "cell_type": "code", 326 | "execution_count": null, 327 | "metadata": {}, 328 | "outputs": [], 329 | "source": [ 330 | "# Training function\n", 331 | "def train(model, X, Y, optimizer, loss_fn, num_epochs):\n", 332 | " # Loop over epochs\n", 333 | " for epoch in range(num_epochs):\n", 334 | " # Reset accumulated loss per epoch\n", 335 | " acc_loss = 0\n", 336 | "\n", 337 | " # Loop over all training data\n", 338 | " for i in range(len(X)):\n", 339 | " # training sample and label\n", 340 | " x, y = X[i], Y[i].unsqueeze(0)\n", 341 | "\n", 342 | " # forward pass\n", 343 | " y_hat = model(x)\n", 344 | "\n", 345 | " # backward pass\n", 346 | " loss = loss_fn(y_hat, y)\n", 347 | " loss.backward()\n", 348 | "\n", 349 | " # accumulate loss\n", 350 | " acc_loss += loss\n", 351 | "\n", 352 | " # Update weights and bias\n", 353 | " optimizer.step()\n", 354 | " optimizer.zero_grad()\n", 355 | "\n", 356 | " # Print accumulated average loss per epoch once in a while\n", 357 | " if (epoch % (num_epochs // 10)) == 0 or epoch == num_epochs - 1:\n", 358 | " print(f\"Epoch {epoch:5d}: loss = {torch.mean(acc_loss):.5f}\")" 359 | ] 360 | }, 361 | { 362 | "cell_type": "code", 363 | "execution_count": null, 364 | "metadata": {}, 365 | "outputs": [], 366 | "source": [ 367 | "# Testing function\n", 368 | "def test(model, X, Y):\n", 369 | " # test the perceptron on all data points\n", 370 | " print(\"Testing ...\")\n", 371 | " for i in range(len(X)):\n", 372 | " prediction = model(X[i])\n", 373 | " print(f\"{X[i]} -> {prediction} (label: {Y[i]})\")" 374 | ] 375 | }, 376 | { 377 | "cell_type": "markdown", 378 | "metadata": {}, 379 | "source": [ 380 | "## Train and test" 381 | ] 382 | }, 383 | { 384 | "cell_type": "code", 385 | "execution_count": null, 386 | "metadata": {}, 387 | "outputs": [], 388 | "source": [ 389 | "# Train our multi-layer perceptron model\n", 390 | "\n", 391 | "# The model to train\n", 392 | "model = MultiLayerPerceptron(num_inputs=2)\n", 393 | "\n", 394 | "# Hyperparameters\n", 395 | "num_epochs = 10000\n", 396 | "eta = 0.25\n", 397 | "# Stochastic gradient descent (SGD) optimizer\n", 398 | "optimizer = torch.optim.SGD(model.parameters(), lr=eta)\n", 399 | "\n", 400 | "# We can use L2 (mean squared error) loss from PyTorch\n", 401 | "loss_fn = nn.MSELoss()\n", 402 | "\n", 403 | "# Train the model\n", 404 | "train(model, X, Y, optimizer, loss_fn, num_epochs)\n", 405 | "\n", 406 | "# Test the model\n", 407 | "test(model, X, Y)" 408 | ] 409 | }, 410 | { 411 | "cell_type": "markdown", 412 | "metadata": {}, 413 | "source": [ 414 | "# Visualize decision boundary" 415 | ] 416 | }, 417 | { 418 | "cell_type": "code", 419 | "execution_count": null, 420 | "metadata": {}, 421 | "outputs": [], 422 | "source": [ 423 | "import matplotlib.cm as cm\n", 424 | "import matplotlib.gridspec as gridspec\n", 425 | "\n", 426 | "\n", 427 | "def show_decision_boundary(model, data, labels, subplot_spec=None):\n", 428 | " data = data.numpy()\n", 429 | " labels = labels.numpy()\n", 430 | "\n", 431 | " wratio = (15, 1)\n", 432 | " if subplot_spec is None:\n", 433 | " gs = gridspec.GridSpec(1, 2, width_ratios=wratio)\n", 434 | " else:\n", 435 | " gs = gridspec.GridSpecFromSubplotSpec(\n", 436 | " 1, 2, subplot_spec=subplot_spec, width_ratios=wratio\n", 437 | " )\n", 438 | "\n", 439 | " ax = plt.subplot(gs[0])\n", 440 | " ax.set_title(\"Dataset and Decision Function\")\n", 441 | "\n", 442 | " x_min, x_max = data[:, 0].min() - 1, data[:, 0].max() + 1\n", 443 | " y_min, y_max = data[:, 1].min() - 1, data[:, 1].max() + 1\n", 444 | " h = 0.01 # Reduced step size for higher resolution\n", 445 | " xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))\n", 446 | "\n", 447 | " Z = model(torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32))\n", 448 | " Z = Z.reshape(xx.shape)\n", 449 | "\n", 450 | " # Increase the number of levels for smoother color transitions\n", 451 | " levels = np.linspace(0, 1, 100)\n", 452 | " ctr = ax.contourf(xx, yy, Z.detach().numpy(), levels, cmap=cm.gray, vmin=0, vmax=1)\n", 453 | "\n", 454 | " unique_labels = np.unique(labels)\n", 455 | "\n", 456 | " # Define colors for each class\n", 457 | " colors = [\"red\", \"blue\"]\n", 458 | " for i, yi in enumerate(unique_labels):\n", 459 | " color = colors[i]\n", 460 | " ax.scatter(\n", 461 | " data[np.where(labels.flatten() == yi), 0],\n", 462 | " data[np.where(labels.flatten() == yi), 1],\n", 463 | " color=color,\n", 464 | " linewidth=0,\n", 465 | " label=\"Class %d (y=%d)\" % (yi, yi),\n", 466 | " )\n", 467 | " ax.legend()\n", 468 | " ax.set_xlim((x_min, x_max))\n", 469 | " ax.set_ylim((y_min, y_max))\n", 470 | "\n", 471 | " # Create colorbar\n", 472 | " cbar = plt.colorbar(ctr, cax=plt.subplot(gs[1]))\n", 473 | " cbar.set_ticks(np.arange(0, 1.1, 0.1)) # Set ticks from 0 to 1 with 0.1 increments\n", 474 | " cbar.set_label(\"Decision value\")" 475 | ] 476 | }, 477 | { 478 | "cell_type": "code", 479 | "execution_count": null, 480 | "metadata": {}, 481 | "outputs": [], 482 | "source": [ 483 | "# Plot decision boundary\n", 484 | "show_decision_boundary(model, X, Y)" 485 | ] 486 | } 487 | ], 488 | "metadata": { 489 | "colab": { 490 | "collapsed_sections": [ 491 | "JcF1mpJo-taz" 492 | ], 493 | "provenance": [] 494 | }, 495 | "kernelspec": { 496 | "display_name": "pytorch-m1-2023-10", 497 | "language": "python", 498 | "name": "pytorch-m1-2023-10" 499 | }, 500 | "language_info": { 501 | "codemirror_mode": { 502 | "name": "ipython", 503 | "version": 3 504 | }, 505 | "file_extension": ".py", 506 | "mimetype": "text/x-python", 507 | "name": "python", 508 | "nbconvert_exporter": "python", 509 | "pygments_lexer": "ipython3", 510 | "version": "3.9.18" 511 | } 512 | }, 513 | "nbformat": 4, 514 | "nbformat_minor": 0 515 | } 516 | -------------------------------------------------------------------------------- /multi_layer_perceptron/torch/requirements.txt: -------------------------------------------------------------------------------- 1 | torch 2 | numpy 3 | matplotlib 4 | -------------------------------------------------------------------------------- /nbutils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/menzHSE/cv-ml-lecture-notebooks/0b78e601f65774e0f276533c4b2cb9f24a7ca2c5/nbutils/__init__.py -------------------------------------------------------------------------------- /nbutils/colab.py: -------------------------------------------------------------------------------- 1 | # Markus Enzweiler, markus.enzweiler@hs-esslingen.de 2 | 3 | 4 | def check_for_colab(): 5 | try: 6 | import google.colab 7 | 8 | return True 9 | except ImportError: 10 | return False 11 | 12 | 13 | def mount_gdrive(gdrive_mnt="/content/drive"): 14 | if not check_for_colab(): 15 | return 16 | 17 | from google.colab import drive 18 | 19 | drive.mount(gdrive_mnt, force_remount=True) 20 | 21 | 22 | def unmount_gdrive(): 23 | if not check_for_colab(): 24 | return 25 | 26 | from google.colab import drive 27 | 28 | drive.flush_and_unmount() 29 | -------------------------------------------------------------------------------- /nbutils/data.py: -------------------------------------------------------------------------------- 1 | # Markus Enzweiler, markus.enzweiler@hs-esslingen.de 2 | 3 | import os 4 | 5 | from nbutils import exec as nb_exec 6 | 7 | 8 | # To get around the problem of "Quota Exceeded" on the "official" CelebA 9 | # download via torchvision (https://github.com/pytorch/vision/issues/1920), 10 | # we use an alternative data source. 11 | def download_celeba(dest_dir, on_colab, verbose): 12 | # Base URL for data 13 | data_url = "https://graal.ift.ulaval.ca/public/celeba/" 14 | 15 | # Ensure the destination directory exists 16 | os.makedirs(dest_dir, exist_ok=True) 17 | 18 | # List of files to download 19 | celeba_files = [ 20 | "img_align_celeba.zip", 21 | "list_attr_celeba.txt", 22 | "list_bbox_celeba.txt", 23 | "list_eval_partition.txt", 24 | "list_landmarks_align_celeba.txt", 25 | "list_landmarks_celeba.txt", 26 | "identity_CelebA.txt", 27 | ] 28 | 29 | # Download each file using wget with -nc option 30 | for file in celeba_files: 31 | file_url = data_url + file 32 | print(f"Downloading {file_url} ... ") 33 | # Run wget command 34 | if verbose: 35 | command = ["wget", "-nc", file_url, "-P", dest_dir] 36 | else: 37 | command = ["wget", "-nc", "-nv", file_url, "-P", dest_dir] 38 | nb_exec.executeCommand(command, on_colab) 39 | 40 | # Seems that we need to wait a bit here to make sure 41 | # that the files actually exist in Colab ... weird :( 42 | if on_colab: 43 | import time 44 | 45 | all_files_exist = False 46 | while not all_files_exist: 47 | # Assume all files exist until proven otherwise 48 | all_files_exist = True 49 | 50 | # Check each file 51 | for file in celeba_files: 52 | file_path = os.path.join(dest_dir, file) 53 | if not os.path.exists(file_path): 54 | all_files_exist = False 55 | break 56 | 57 | # Wait for a bit before checking again if not all files are found 58 | if not all_files_exist: 59 | time.sleep(5) # wait for 5 seconds before checking again 60 | 61 | print("Download complete.") 62 | -------------------------------------------------------------------------------- /nbutils/exec.py: -------------------------------------------------------------------------------- 1 | # Markus Enzweiler, markus.enzweiler@hs-esslingen.de 2 | 3 | import errno 4 | import os 5 | import subprocess 6 | import sys 7 | import threading 8 | 9 | from nbutils import colab as nb_colab 10 | 11 | if nb_colab.check_for_colab(): 12 | import fcntl 13 | 14 | 15 | def executePythonScript(script_path, params, on_colab): 16 | if os.path.exists(script_path): 17 | print(f"Executing script: {script_path}") 18 | # Create the command list starting with Python and the script path 19 | command = [sys.executable, script_path] 20 | # Add additional arguments from the params dictionary 21 | if params: 22 | for key, value in params.items(): 23 | command.append(f"--{key}") 24 | command.append(str(value)) 25 | executeCommand(command, on_colab) 26 | else: 27 | print(f"Script not found: {script_path}") 28 | 29 | 30 | def executeCommand(command, on_colab): 31 | if on_colab: 32 | executeCaptureColabCommand(command) 33 | else: 34 | executeCaptureCommand(command) 35 | 36 | 37 | def executeCaptureCommand(command): 38 | print(command) 39 | subprocess.run(command) 40 | 41 | 42 | # This is very hacky ... but it's hard to capture the output of a subprocess in Colab 43 | def executeCaptureColabCommand(command): 44 | print(command) 45 | 46 | # Start the subprocess 47 | process = subprocess.Popen( 48 | command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True 49 | ) 50 | 51 | # Set the stdout to non-blocking 52 | fd = process.stdout.fileno() 53 | fl = fcntl.fcntl(fd, fcntl.F_GETFL) 54 | fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK) 55 | 56 | # Function to continuously output lines from a stream 57 | def stream_output(stream): 58 | while True: 59 | try: 60 | line = stream.readline() 61 | if line: 62 | print(line, end="") 63 | elif process.poll() is not None: 64 | break 65 | except IOError as e: 66 | # Ignore the error if no data is available yet 67 | if e.errno != errno.EAGAIN and e.errno != errno.EWOULDBLOCK: 68 | raise 69 | 70 | # Use a thread to capture the output stream 71 | output_thread = threading.Thread(target=stream_output, args=(process.stdout,)) 72 | output_thread.start() 73 | 74 | # Wait for the subprocess to complete and the output thread to end 75 | process.wait() 76 | output_thread.join() 77 | -------------------------------------------------------------------------------- /nbutils/git.py: -------------------------------------------------------------------------------- 1 | # Markus Enzweiler, markus.enzweiler@hs-esslingen.de 2 | 3 | import os 4 | 5 | from nbutils import exec as nb_exec 6 | 7 | 8 | def clone(repo_url, repo_target_dir, on_colab, verbose=True): 9 | # Check if the directory already exists using the absolute path 10 | if os.path.exists(repo_target_dir): 11 | if verbose: 12 | print(f"Repository {repo_target_dir} exists. Resetting to HEAD...") 13 | # reset to head 14 | reset_to_head_cmd(repo_target_dir, on_colab) 15 | else: 16 | if verbose: 17 | print(f"Cloning repository {repo_url} ...") 18 | # Clone the repository if it doesn't exist 19 | clone_cmd(repo_url, repo_target_dir, on_colab) 20 | 21 | if verbose: 22 | print(f"Repository {repo_target_dir} is ready.") 23 | 24 | 25 | def clone_cmd(repo_url, repo_target_dir, on_colab): 26 | command = ["git", "clone", repo_url, repo_target_dir] 27 | nb_exec.executeCommand(command, on_colab) 28 | 29 | 30 | def reset_to_head_cmd(repo_target_dir, on_colab): 31 | # Fetch the latest changes from the remote 32 | command = ["git", "-C", repo_target_dir, "fetch", "origin"] 33 | nb_exec.executeCommand(command, on_colab) 34 | 35 | # Reset the local branch to the latest commit from the remote 36 | command = ["git", "-C", repo_target_dir, "reset", "--hard", "origin/HEAD"] 37 | nb_exec.executeCommand(command, on_colab) 38 | -------------------------------------------------------------------------------- /nbutils/requirements.py: -------------------------------------------------------------------------------- 1 | # Markus Enzweiler, markus.enzweiler@hs-esslingen.de 2 | 3 | import os 4 | import sys 5 | 6 | from nbutils import exec as nb_exec 7 | 8 | 9 | def pip_install_reqs(req_file, on_colab): 10 | # Install requirements in the current Jupyter kernel 11 | if os.path.exists(req_file): 12 | command = [sys.executable, "-m", "pip", "install", "-r", req_file] 13 | nb_exec.executeCommand(command, on_colab) 14 | else: 15 | print(f"Requirements file not found: {req_file}") 16 | -------------------------------------------------------------------------------- /perceptron/mlx/requirements.txt: -------------------------------------------------------------------------------- 1 | mlx 2 | numpy 3 | matplotlib 4 | -------------------------------------------------------------------------------- /perceptron/numpy/requirements.txt: -------------------------------------------------------------------------------- 1 | numpy 2 | matplotlib 3 | -------------------------------------------------------------------------------- /perceptron/torch/perceptron_OR_torch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "epxcwtWj5yJs" 7 | }, 8 | "source": [ 9 | "# Perceptron for the OR problem in PyTorch\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": {}, 19 | "source": [ 20 | "We build and train a perceptron to act as a simple OR gate with two inputs and one output. \n", 21 | "OR gates have the following behavior:\n", 22 | "\n", 23 | "If both inputs are 0 (off), the output is 0 (off)\n", 24 | "If both inputs are 1 (on), the output is 1 (on)\n", 25 | "If either input is 1 (on), the output is still 1 (on)\n", 26 | "\n", 27 | "| observation # | input 1 | input 2 | output |\n", 28 | "|---------------|---------|---------|--------|\n", 29 | "| 0 | 0 | 0 | 0 |\n", 30 | "| 1 | 0 | 1 | 1 |\n", 31 | "| 2 | 1 | 0 | 1 |\n", 32 | "| 3 | 1 | 1 | 1 |\n", 33 | "\n" 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "In this demonstration, we replace our custom Perceptron class from the previous notebooks and use torch.nn\n", 41 | "\n", 42 | "See https://pytorch.org/docs/stable/nn.html and in particular:\n", 43 | "- https://pytorch.org/docs/stable/generated/torch.nn.Module.html \n", 44 | "- https://pytorch.org/docs/stable/generated/torch.nn.Linear.html " 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": { 50 | "id": "IJsjl_l47q28" 51 | }, 52 | "source": [ 53 | "## Setup\n", 54 | "\n", 55 | "Adapt `packagePath` to point to the directory containing this notebeook." 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": null, 61 | "metadata": { 62 | "executionInfo": { 63 | "elapsed": 236, 64 | "status": "ok", 65 | "timestamp": 1703326573912, 66 | "user": { 67 | "displayName": "Markus Enzweiler", 68 | "userId": "04524044579212347608" 69 | }, 70 | "user_tz": -60 71 | }, 72 | "id": "1iHkPBml98YG" 73 | }, 74 | "outputs": [], 75 | "source": [ 76 | "# Notebook id\n", 77 | "nb_id = \"perceptron/torch\"\n", 78 | "\n", 79 | "# Imports\n", 80 | "import sys\n", 81 | "import os" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": null, 87 | "metadata": { 88 | "colab": { 89 | "base_uri": "https://localhost:8080/" 90 | }, 91 | "executionInfo": { 92 | "elapsed": 2602, 93 | "status": "ok", 94 | "timestamp": 1703326576880, 95 | "user": { 96 | "displayName": "Markus Enzweiler", 97 | "userId": "04524044579212347608" 98 | }, 99 | "user_tz": -60 100 | }, 101 | "id": "DY4880S378_F", 102 | "outputId": "a8ce2d26-fe24-4aa2-c232-acc31b71d394" 103 | }, 104 | "outputs": [], 105 | "source": [ 106 | "# Package Path (folder of this notebook)\n", 107 | "\n", 108 | "#####################\n", 109 | "# Local environment #\n", 110 | "#####################\n", 111 | "\n", 112 | "package_path = \"./\"\n", 113 | "\n", 114 | "\n", 115 | "#########\n", 116 | "# Colab #\n", 117 | "#########\n", 118 | "\n", 119 | "\n", 120 | "def check_for_colab():\n", 121 | " try:\n", 122 | " import google.colab\n", 123 | "\n", 124 | " return True\n", 125 | " except ImportError:\n", 126 | " return False\n", 127 | "\n", 128 | "\n", 129 | "# running on Colab?\n", 130 | "on_colab = check_for_colab()\n", 131 | "\n", 132 | "if on_colab:\n", 133 | " # assume this notebook is run from Google Drive and the whole\n", 134 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 135 | "\n", 136 | " # Google Drive mount point\n", 137 | " gdrive_mnt = \"/content/drive\"\n", 138 | "\n", 139 | " ##########################################################################\n", 140 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 141 | " ##########################################################################\n", 142 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 143 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 144 | "\n", 145 | " # mount drive\n", 146 | " from google.colab import drive\n", 147 | "\n", 148 | " drive.mount(gdrive_mnt, force_remount=True)\n", 149 | "\n", 150 | " # set package path\n", 151 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 152 | "\n", 153 | "# check whether package path exists\n", 154 | "if not os.path.isdir(package_path):\n", 155 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 156 | "\n", 157 | "print(f\"Package path: {package_path}\")" 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": null, 163 | "metadata": {}, 164 | "outputs": [], 165 | "source": [ 166 | "# Additional imports\n", 167 | "\n", 168 | "# Repository Root\n", 169 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 170 | "# Add the repository root to the system path\n", 171 | "if repo_root not in sys.path:\n", 172 | " sys.path.append(repo_root)\n", 173 | "\n", 174 | "# Package Imports\n", 175 | "from nbutils import requirements as nb_reqs\n", 176 | "from nbutils import colab as nb_colab\n", 177 | "from nbutils import git as nb_git\n", 178 | "from nbutils import exec as nb_exec\n", 179 | "from nbutils import data as nb_data" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": null, 185 | "metadata": { 186 | "colab": { 187 | "base_uri": "https://localhost:8080/" 188 | }, 189 | "executionInfo": { 190 | "elapsed": 6888, 191 | "status": "ok", 192 | "timestamp": 1703326583765, 193 | "user": { 194 | "displayName": "Markus Enzweiler", 195 | "userId": "04524044579212347608" 196 | }, 197 | "user_tz": -60 198 | }, 199 | "id": "iaURjw5n6pLq", 200 | "outputId": "5c40ee52-1fa3-44df-a94e-b81bec144100" 201 | }, 202 | "outputs": [], 203 | "source": [ 204 | "# Install requirements in the current Jupyter kernel\n", 205 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 206 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "execution_count": null, 212 | "metadata": { 213 | "executionInfo": { 214 | "elapsed": 12, 215 | "status": "ok", 216 | "timestamp": 1703326583765, 217 | "user": { 218 | "displayName": "Markus Enzweiler", 219 | "userId": "04524044579212347608" 220 | }, 221 | "user_tz": -60 222 | }, 223 | "id": "iRERDI8aAnzr" 224 | }, 225 | "outputs": [], 226 | "source": [ 227 | "# Now we should be able to import the additional packages\n", 228 | "import numpy as np\n", 229 | "import matplotlib.pyplot as plt\n", 230 | "import torch\n", 231 | "import torch.nn as nn\n", 232 | "\n", 233 | "# Set the random seed for reproducibility\n", 234 | "np.random.seed(42)\n", 235 | "torch.manual_seed(42);" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": { 241 | "id": "aUACL9QoFTXg" 242 | }, 243 | "source": [ 244 | "## Create the training data" 245 | ] 246 | }, 247 | { 248 | "cell_type": "code", 249 | "execution_count": null, 250 | "metadata": {}, 251 | "outputs": [], 252 | "source": [ 253 | "# Define the training data for the OR problem in numpy\n", 254 | "\n", 255 | "\n", 256 | "# Define the training data for the OR problem\n", 257 | "X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])\n", 258 | "Y = np.array([0, 1, 1, 1])\n", 259 | "\n", 260 | "# Convert numpy arrays to PyTorch tensors\n", 261 | "X = torch.tensor(X, dtype=torch.float32)\n", 262 | "Y = torch.tensor(Y, dtype=torch.float32)\n", 263 | "\n", 264 | "print(\"Training data X with labels y:\")\n", 265 | "for i in range(len(X)):\n", 266 | " print(f\"{X[i]} -> {Y[i]}\")" 267 | ] 268 | }, 269 | { 270 | "cell_type": "markdown", 271 | "metadata": {}, 272 | "source": [ 273 | "# Define the Perceptron" 274 | ] 275 | }, 276 | { 277 | "cell_type": "markdown", 278 | "metadata": {}, 279 | "source": [ 280 | "## Perceptron class" 281 | ] 282 | }, 283 | { 284 | "cell_type": "code", 285 | "execution_count": null, 286 | "metadata": {}, 287 | "outputs": [], 288 | "source": [ 289 | "## Replace our custom Perceptron class with PyTorch nn.Module and nn.Linear\n", 290 | "\n", 291 | "\n", 292 | "class Perceptron(nn.Module):\n", 293 | " # override constructor from nn.Module\n", 294 | " def __init__(self, num_inputs):\n", 295 | " super().__init__() ## call constructor of nn.Module\n", 296 | "\n", 297 | " # we define the components of our perceptron\n", 298 | " # we want to have a linear layer with num_inputs inputs and one output\n", 299 | " self.linear = nn.Linear(in_features=num_inputs, out_features=1)\n", 300 | " self.sigmoid = nn.Sigmoid()\n", 301 | "\n", 302 | " def __call__(self, x):\n", 303 | " return self.forward(x)\n", 304 | "\n", 305 | " # override forward method from nn.Module\n", 306 | " def forward(self, x):\n", 307 | " # and the computation of the forward pass\n", 308 | " return self.sigmoid(self.linear(x))\n", 309 | "\n", 310 | " def print(self):\n", 311 | " print(f\"Weights: {self.linear.weight}\")\n", 312 | " print(f\"Bias: {self.linear.bias}\")" 313 | ] 314 | }, 315 | { 316 | "cell_type": "markdown", 317 | "metadata": {}, 318 | "source": [ 319 | "## Create the Perceptron" 320 | ] 321 | }, 322 | { 323 | "cell_type": "code", 324 | "execution_count": null, 325 | "metadata": {}, 326 | "outputs": [], 327 | "source": [ 328 | "# Perceptron for our OR problem\n", 329 | "perceptron = Perceptron(num_inputs=2)\n", 330 | "perceptron.print()\n", 331 | "\n", 332 | "# All parameters of the perceptron are initialized randomly.\n", 333 | "# Let's see the initial predictions for the training data X.\n", 334 | "\n", 335 | "\n", 336 | "def test(perceptron, X, Y):\n", 337 | " # test the perceptron on all data points\n", 338 | " print(\"Testing ...\")\n", 339 | " for i in range(len(X)):\n", 340 | " prediction = perceptron(X[i])\n", 341 | " print(f\"{X[i]} -> {prediction} (label: {Y[i]})\")\n", 342 | "\n", 343 | "\n", 344 | "print(\"Initial predictions before training:\")\n", 345 | "test(perceptron, X, Y)" 346 | ] 347 | }, 348 | { 349 | "cell_type": "markdown", 350 | "metadata": {}, 351 | "source": [ 352 | "# Perceptron training with gradient descent" 353 | ] 354 | }, 355 | { 356 | "cell_type": "markdown", 357 | "metadata": {}, 358 | "source": [ 359 | "## Training" 360 | ] 361 | }, 362 | { 363 | "cell_type": "code", 364 | "execution_count": null, 365 | "metadata": {}, 366 | "outputs": [], 367 | "source": [ 368 | "# Training\n", 369 | "\n", 370 | "# Train for 10000 training epochs with SGD\n", 371 | "num_epochs = 10000\n", 372 | "eta = 0.25\n", 373 | "\n", 374 | "# We can use L2 (mean squared error) loss from PyTorch\n", 375 | "loss_fn = nn.MSELoss()\n", 376 | "# Stochastic gradient descent (SGD) optimizer\n", 377 | "optimizer = torch.optim.SGD(perceptron.parameters(), lr=eta)\n", 378 | "\n", 379 | "# Loop over epochs\n", 380 | "for epoch in range(num_epochs):\n", 381 | " # Reset accumulated loss per epoch\n", 382 | " acc_loss = 0\n", 383 | "\n", 384 | " # Loop over all training data\n", 385 | " for i in range(len(X)):\n", 386 | " # forward pass\n", 387 | " y_hat = perceptron(X[i])\n", 388 | "\n", 389 | " # backward pass\n", 390 | " loss = loss_fn(y_hat, Y[i].view(1))\n", 391 | " loss.backward()\n", 392 | " acc_loss += loss\n", 393 | "\n", 394 | " # Update weights and bias\n", 395 | " optimizer.step()\n", 396 | " optimizer.zero_grad()\n", 397 | "\n", 398 | " # Print accumulated average loss per epoch once in a while\n", 399 | " if (epoch % (num_epochs // 10)) == 0 or epoch == num_epochs - 1:\n", 400 | " print(f\"Epoch {epoch:5d}: loss = {torch.mean(acc_loss):.5f}\")" 401 | ] 402 | }, 403 | { 404 | "cell_type": "code", 405 | "execution_count": null, 406 | "metadata": {}, 407 | "outputs": [], 408 | "source": [ 409 | "print(\"-------------------------------------------\")\n", 410 | "print(f\"Perceptron after training for {num_epochs} epochs:\")\n", 411 | "perceptron.print()\n", 412 | "test(perceptron, X, Y)" 413 | ] 414 | }, 415 | { 416 | "cell_type": "markdown", 417 | "metadata": {}, 418 | "source": [ 419 | "# Visualize decision boundary" 420 | ] 421 | }, 422 | { 423 | "cell_type": "code", 424 | "execution_count": null, 425 | "metadata": {}, 426 | "outputs": [], 427 | "source": [ 428 | "import matplotlib.cm as cm\n", 429 | "import matplotlib.gridspec as gridspec\n", 430 | "\n", 431 | "\n", 432 | "def show_decision_boundary(model, data, labels, subplot_spec=None):\n", 433 | " data = data.numpy()\n", 434 | " labels = labels.numpy()\n", 435 | "\n", 436 | " wratio = (15, 1)\n", 437 | " if subplot_spec is None:\n", 438 | " gs = gridspec.GridSpec(1, 2, width_ratios=wratio)\n", 439 | " else:\n", 440 | " gs = gridspec.GridSpecFromSubplotSpec(\n", 441 | " 1, 2, subplot_spec=subplot_spec, width_ratios=wratio\n", 442 | " )\n", 443 | "\n", 444 | " ax = plt.subplot(gs[0])\n", 445 | " ax.set_title(\"Dataset and decision function\")\n", 446 | "\n", 447 | " x_min, x_max = data[:, 0].min() - 1, data[:, 0].max() + 1\n", 448 | " y_min, y_max = data[:, 1].min() - 1, data[:, 1].max() + 1\n", 449 | " h = 0.01 # Reduced step size for higher resolution\n", 450 | " xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))\n", 451 | "\n", 452 | " Z = model(torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32))\n", 453 | " Z = Z.reshape(xx.shape)\n", 454 | "\n", 455 | " # Increase the number of levels for smoother color transitions\n", 456 | " levels = np.linspace(0, 1, 100)\n", 457 | " ctr = ax.contourf(xx, yy, Z.detach().numpy(), levels, cmap=cm.gray, vmin=0, vmax=1)\n", 458 | "\n", 459 | " unique_labels = np.unique(labels)\n", 460 | "\n", 461 | " # Define colors for each class\n", 462 | " colors = [\"red\", \"blue\"]\n", 463 | " for i, yi in enumerate(unique_labels):\n", 464 | " color = colors[i]\n", 465 | " ax.scatter(\n", 466 | " data[np.where(labels.flatten() == yi), 0],\n", 467 | " data[np.where(labels.flatten() == yi), 1],\n", 468 | " color=color,\n", 469 | " linewidth=0,\n", 470 | " label=\"Class %d (y=%d)\" % (yi, yi),\n", 471 | " )\n", 472 | " ax.legend()\n", 473 | " ax.set_xlim((x_min, x_max))\n", 474 | " ax.set_ylim((y_min, y_max))\n", 475 | "\n", 476 | " # Create colorbar\n", 477 | " cbar = plt.colorbar(ctr, cax=plt.subplot(gs[1]))\n", 478 | " cbar.set_ticks(np.arange(0, 1.1, 0.1)) # Set ticks from 0 to 1 with 0.1 increments\n", 479 | " cbar.set_label(\"Decision value\")" 480 | ] 481 | }, 482 | { 483 | "cell_type": "code", 484 | "execution_count": null, 485 | "metadata": {}, 486 | "outputs": [], 487 | "source": [ 488 | "# Plot decision boundary\n", 489 | "show_decision_boundary(perceptron, X, Y)" 490 | ] 491 | } 492 | ], 493 | "metadata": { 494 | "colab": { 495 | "collapsed_sections": [ 496 | "JcF1mpJo-taz" 497 | ], 498 | "provenance": [] 499 | }, 500 | "kernelspec": { 501 | "display_name": "pytorch-m1-2023-10", 502 | "language": "python", 503 | "name": "pytorch-m1-2023-10" 504 | }, 505 | "language_info": { 506 | "codemirror_mode": { 507 | "name": "ipython", 508 | "version": 3 509 | }, 510 | "file_extension": ".py", 511 | "mimetype": "text/x-python", 512 | "name": "python", 513 | "nbconvert_exporter": "python", 514 | "pygments_lexer": "ipython3", 515 | "version": "3.9.18" 516 | } 517 | }, 518 | "nbformat": 4, 519 | "nbformat_minor": 0 520 | } 521 | -------------------------------------------------------------------------------- /perceptron/torch/perceptron_XOR_torch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "epxcwtWj5yJs" 7 | }, 8 | "source": [ 9 | "# Perceptron for the XOR problem in PyTorch\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": {}, 19 | "source": [ 20 | "We build and train a perceptron to act as a simple XOR gate with two inputs and one output. \n", 21 | "XOR gates have the following behavior:\n", 22 | "\n", 23 | "If both inputs are identical, the output is 0 (off)\n", 24 | "If both inputs are different, the output is 1 (on)\n", 25 | "\n", 26 | "| observation # | input 1 | input 2 | output |\n", 27 | "|---------------|---------|---------|--------|\n", 28 | "| 0 | 0 | 0 | 0 |\n", 29 | "| 1 | 0 | 1 | 1 |\n", 30 | "| 2 | 1 | 0 | 1 |\n", 31 | "| 3 | 1 | 1 | 0 |\n", 32 | "\n" 33 | ] 34 | }, 35 | { 36 | "cell_type": "markdown", 37 | "metadata": {}, 38 | "source": [ 39 | "In this demonstration, we replace our custom Perceptron class from the previous notebooks and use torch.nn\n", 40 | "\n", 41 | "See https://pytorch.org/docs/stable/nn.html and in particular:\n", 42 | "- https://pytorch.org/docs/stable/generated/torch.nn.Module.html \n", 43 | "- https://pytorch.org/docs/stable/generated/torch.nn.Linear.html " 44 | ] 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "metadata": { 49 | "id": "IJsjl_l47q28" 50 | }, 51 | "source": [ 52 | "## Setup\n", 53 | "\n", 54 | "Adapt `packagePath` to point to the directory containing this notebeook." 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": null, 60 | "metadata": { 61 | "executionInfo": { 62 | "elapsed": 236, 63 | "status": "ok", 64 | "timestamp": 1703326573912, 65 | "user": { 66 | "displayName": "Markus Enzweiler", 67 | "userId": "04524044579212347608" 68 | }, 69 | "user_tz": -60 70 | }, 71 | "id": "1iHkPBml98YG" 72 | }, 73 | "outputs": [], 74 | "source": [ 75 | "# Notebook id\n", 76 | "nb_id = \"perceptron/torch\"\n", 77 | "\n", 78 | "# Imports\n", 79 | "import sys\n", 80 | "import os" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": null, 86 | "metadata": { 87 | "colab": { 88 | "base_uri": "https://localhost:8080/" 89 | }, 90 | "executionInfo": { 91 | "elapsed": 2602, 92 | "status": "ok", 93 | "timestamp": 1703326576880, 94 | "user": { 95 | "displayName": "Markus Enzweiler", 96 | "userId": "04524044579212347608" 97 | }, 98 | "user_tz": -60 99 | }, 100 | "id": "DY4880S378_F", 101 | "outputId": "a8ce2d26-fe24-4aa2-c232-acc31b71d394" 102 | }, 103 | "outputs": [], 104 | "source": [ 105 | "# Package Path (folder of this notebook)\n", 106 | "\n", 107 | "#####################\n", 108 | "# Local environment #\n", 109 | "#####################\n", 110 | "\n", 111 | "package_path = \"./\"\n", 112 | "\n", 113 | "\n", 114 | "#########\n", 115 | "# Colab #\n", 116 | "#########\n", 117 | "\n", 118 | "\n", 119 | "def check_for_colab():\n", 120 | " try:\n", 121 | " import google.colab\n", 122 | "\n", 123 | " return True\n", 124 | " except ImportError:\n", 125 | " return False\n", 126 | "\n", 127 | "\n", 128 | "# running on Colab?\n", 129 | "on_colab = check_for_colab()\n", 130 | "\n", 131 | "if on_colab:\n", 132 | " # assume this notebook is run from Google Drive and the whole\n", 133 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 134 | "\n", 135 | " # Google Drive mount point\n", 136 | " gdrive_mnt = \"/content/drive\"\n", 137 | "\n", 138 | " ##########################################################################\n", 139 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 140 | " ##########################################################################\n", 141 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 142 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 143 | "\n", 144 | " # mount drive\n", 145 | " from google.colab import drive\n", 146 | "\n", 147 | " drive.mount(gdrive_mnt, force_remount=True)\n", 148 | "\n", 149 | " # set package path\n", 150 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 151 | "\n", 152 | "# check whether package path exists\n", 153 | "if not os.path.isdir(package_path):\n", 154 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 155 | "\n", 156 | "print(f\"Package path: {package_path}\")" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "# Additional imports\n", 166 | "\n", 167 | "# Repository Root\n", 168 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 169 | "# Add the repository root to the system path\n", 170 | "if repo_root not in sys.path:\n", 171 | " sys.path.append(repo_root)\n", 172 | "\n", 173 | "# Package Imports\n", 174 | "from nbutils import requirements as nb_reqs\n", 175 | "from nbutils import colab as nb_colab\n", 176 | "from nbutils import git as nb_git\n", 177 | "from nbutils import exec as nb_exec\n", 178 | "from nbutils import data as nb_data" 179 | ] 180 | }, 181 | { 182 | "cell_type": "code", 183 | "execution_count": null, 184 | "metadata": { 185 | "colab": { 186 | "base_uri": "https://localhost:8080/" 187 | }, 188 | "executionInfo": { 189 | "elapsed": 6888, 190 | "status": "ok", 191 | "timestamp": 1703326583765, 192 | "user": { 193 | "displayName": "Markus Enzweiler", 194 | "userId": "04524044579212347608" 195 | }, 196 | "user_tz": -60 197 | }, 198 | "id": "iaURjw5n6pLq", 199 | "outputId": "5c40ee52-1fa3-44df-a94e-b81bec144100" 200 | }, 201 | "outputs": [], 202 | "source": [ 203 | "# Install requirements in the current Jupyter kernel\n", 204 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 205 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "execution_count": null, 211 | "metadata": { 212 | "executionInfo": { 213 | "elapsed": 12, 214 | "status": "ok", 215 | "timestamp": 1703326583765, 216 | "user": { 217 | "displayName": "Markus Enzweiler", 218 | "userId": "04524044579212347608" 219 | }, 220 | "user_tz": -60 221 | }, 222 | "id": "iRERDI8aAnzr" 223 | }, 224 | "outputs": [], 225 | "source": [ 226 | "# Now we should be able to import the additional packages\n", 227 | "import numpy as np\n", 228 | "import matplotlib.pyplot as plt\n", 229 | "import torch\n", 230 | "import torch.nn as nn\n", 231 | "\n", 232 | "# Set the random seed for reproducibility\n", 233 | "np.random.seed(42)\n", 234 | "torch.manual_seed(42);" 235 | ] 236 | }, 237 | { 238 | "cell_type": "markdown", 239 | "metadata": { 240 | "id": "aUACL9QoFTXg" 241 | }, 242 | "source": [ 243 | "## Create the training data" 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": null, 249 | "metadata": {}, 250 | "outputs": [], 251 | "source": [ 252 | "# Define the training data for the OR problem in numpy\n", 253 | "\n", 254 | "\n", 255 | "# Define the training data for the OR problem\n", 256 | "X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])\n", 257 | "Y = np.array([0, 1, 1, 0])\n", 258 | "\n", 259 | "# Convert numpy arrays to PyTorch tensors\n", 260 | "X = torch.tensor(X, dtype=torch.float32)\n", 261 | "Y = torch.tensor(Y, dtype=torch.float32)\n", 262 | "\n", 263 | "print(\"Training data X with labels y:\")\n", 264 | "for i in range(len(X)):\n", 265 | " print(f\"{X[i]} -> {Y[i]}\")" 266 | ] 267 | }, 268 | { 269 | "cell_type": "markdown", 270 | "metadata": {}, 271 | "source": [ 272 | "# Define the Perceptron" 273 | ] 274 | }, 275 | { 276 | "cell_type": "markdown", 277 | "metadata": {}, 278 | "source": [ 279 | "## Perceptron class" 280 | ] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": null, 285 | "metadata": {}, 286 | "outputs": [], 287 | "source": [ 288 | "## Replace our custom Perceptron class with PyTorch nn.Module and nn.Linear\n", 289 | "\n", 290 | "\n", 291 | "class Perceptron(nn.Module):\n", 292 | " # override constructor from nn.Module\n", 293 | " def __init__(self, num_inputs):\n", 294 | " super().__init__() ## call constructor of nn.Module\n", 295 | "\n", 296 | " # we define the components of our perceptron\n", 297 | " # we want to have a linear layer with num_inputs inputs and one output\n", 298 | " self.linear = nn.Linear(in_features=num_inputs, out_features=1)\n", 299 | " self.sigmoid = nn.Sigmoid()\n", 300 | "\n", 301 | " def __call__(self, x):\n", 302 | " return self.forward(x)\n", 303 | "\n", 304 | " # override forward method from nn.Module\n", 305 | " def forward(self, x):\n", 306 | " # and the computation of the forward pass\n", 307 | " return self.sigmoid(self.linear(x))\n", 308 | "\n", 309 | " def print(self):\n", 310 | " print(f\"Weights: {self.linear.weight}\")\n", 311 | " print(f\"Bias: {self.linear.bias}\")" 312 | ] 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "metadata": {}, 317 | "source": [ 318 | "## Create the Perceptron" 319 | ] 320 | }, 321 | { 322 | "cell_type": "code", 323 | "execution_count": null, 324 | "metadata": {}, 325 | "outputs": [], 326 | "source": [ 327 | "# Perceptron for our OR problem\n", 328 | "perceptron = Perceptron(num_inputs=2)\n", 329 | "perceptron.print()\n", 330 | "\n", 331 | "# All parameters of the perceptron are initialized randomly.\n", 332 | "# Let's see the initial predictions for the training data X.\n", 333 | "\n", 334 | "\n", 335 | "def test(perceptron, X, Y):\n", 336 | " # test the perceptron on all data points\n", 337 | " print(\"Testing ...\")\n", 338 | " for i in range(len(X)):\n", 339 | " prediction = perceptron(X[i])\n", 340 | " print(f\"{X[i]} -> {prediction} (label: {Y[i]})\")\n", 341 | "\n", 342 | "\n", 343 | "print(\"Initial predictions before training:\")\n", 344 | "test(perceptron, X, Y)" 345 | ] 346 | }, 347 | { 348 | "cell_type": "markdown", 349 | "metadata": {}, 350 | "source": [ 351 | "# Perceptron training with gradient descent" 352 | ] 353 | }, 354 | { 355 | "cell_type": "markdown", 356 | "metadata": {}, 357 | "source": [ 358 | "## Training" 359 | ] 360 | }, 361 | { 362 | "cell_type": "code", 363 | "execution_count": null, 364 | "metadata": {}, 365 | "outputs": [], 366 | "source": [ 367 | "# Training\n", 368 | "\n", 369 | "# Train for 10000 training epochs with SGD\n", 370 | "num_epochs = 10000\n", 371 | "eta = 0.25\n", 372 | "\n", 373 | "# We can use L2 (mean squared error) loss from PyTorch\n", 374 | "loss_fn = nn.MSELoss()\n", 375 | "# Stochastic gradient descent (SGD) optimizer\n", 376 | "optimizer = torch.optim.SGD(perceptron.parameters(), lr=eta)\n", 377 | "\n", 378 | "# Loop over epochs\n", 379 | "for epoch in range(num_epochs):\n", 380 | " # Reset accumulated loss per epoch\n", 381 | " acc_loss = 0\n", 382 | "\n", 383 | " # Loop over all training data\n", 384 | " for i in range(len(X)):\n", 385 | " # forward pass\n", 386 | " y_hat = perceptron(X[i])\n", 387 | "\n", 388 | " # backward pass\n", 389 | " loss = loss_fn(y_hat, Y[i].view(1))\n", 390 | " loss.backward()\n", 391 | " acc_loss += loss\n", 392 | "\n", 393 | " # Update weights and bias\n", 394 | " optimizer.step()\n", 395 | " optimizer.zero_grad()\n", 396 | "\n", 397 | " # Print accumulated average loss per epoch once in a while\n", 398 | " if (epoch % (num_epochs // 10)) == 0 or epoch == num_epochs - 1:\n", 399 | " print(f\"Epoch {epoch:5d}: loss = {torch.mean(acc_loss):.5f}\")" 400 | ] 401 | }, 402 | { 403 | "cell_type": "code", 404 | "execution_count": null, 405 | "metadata": {}, 406 | "outputs": [], 407 | "source": [ 408 | "print(\"-------------------------------------------\")\n", 409 | "print(f\"Perceptron after training for {num_epochs} epochs:\")\n", 410 | "perceptron.print()\n", 411 | "test(perceptron, X, Y)" 412 | ] 413 | }, 414 | { 415 | "cell_type": "markdown", 416 | "metadata": {}, 417 | "source": [ 418 | "# Visualize decision boundary" 419 | ] 420 | }, 421 | { 422 | "cell_type": "code", 423 | "execution_count": null, 424 | "metadata": {}, 425 | "outputs": [], 426 | "source": [ 427 | "import matplotlib.cm as cm\n", 428 | "import matplotlib.gridspec as gridspec\n", 429 | "\n", 430 | "\n", 431 | "def show_decision_boundary(model, data, labels, subplot_spec=None):\n", 432 | " data = data.numpy()\n", 433 | " labels = labels.numpy()\n", 434 | "\n", 435 | " wratio = (15, 1)\n", 436 | " if subplot_spec is None:\n", 437 | " gs = gridspec.GridSpec(1, 2, width_ratios=wratio)\n", 438 | " else:\n", 439 | " gs = gridspec.GridSpecFromSubplotSpec(\n", 440 | " 1, 2, subplot_spec=subplot_spec, width_ratios=wratio\n", 441 | " )\n", 442 | "\n", 443 | " ax = plt.subplot(gs[0])\n", 444 | " ax.set_title(\"Dataset and Decision Function\")\n", 445 | "\n", 446 | " x_min, x_max = data[:, 0].min() - 1, data[:, 0].max() + 1\n", 447 | " y_min, y_max = data[:, 1].min() - 1, data[:, 1].max() + 1\n", 448 | " h = 0.01 # Reduced step size for higher resolution\n", 449 | " xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))\n", 450 | "\n", 451 | " Z = model(torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32))\n", 452 | " Z = Z.reshape(xx.shape)\n", 453 | "\n", 454 | " # Increase the number of levels for smoother color transitions\n", 455 | " levels = np.linspace(0, 1, 100)\n", 456 | " ctr = ax.contourf(xx, yy, Z.detach().numpy(), levels, cmap=cm.gray, vmin=0, vmax=1)\n", 457 | "\n", 458 | " unique_labels = np.unique(labels)\n", 459 | "\n", 460 | " # Define colors for each class\n", 461 | " colors = [\"red\", \"blue\"]\n", 462 | " for i, yi in enumerate(unique_labels):\n", 463 | " color = colors[i]\n", 464 | " ax.scatter(\n", 465 | " data[np.where(labels.flatten() == yi), 0],\n", 466 | " data[np.where(labels.flatten() == yi), 1],\n", 467 | " color=color,\n", 468 | " linewidth=0,\n", 469 | " label=\"Class %d (y=%d)\" % (yi, yi),\n", 470 | " )\n", 471 | " ax.legend()\n", 472 | " ax.set_xlim((x_min, x_max))\n", 473 | " ax.set_ylim((y_min, y_max))\n", 474 | "\n", 475 | " # Create colorbar\n", 476 | " cbar = plt.colorbar(ctr, cax=plt.subplot(gs[1]))\n", 477 | " cbar.set_ticks(np.arange(0, 1.1, 0.1)) # Set ticks from 0 to 1 with 0.1 increments\n", 478 | " cbar.set_label(\"Decision value\")" 479 | ] 480 | }, 481 | { 482 | "cell_type": "code", 483 | "execution_count": null, 484 | "metadata": {}, 485 | "outputs": [], 486 | "source": [ 487 | "# Plot decision boundary\n", 488 | "show_decision_boundary(perceptron, X, Y)" 489 | ] 490 | }, 491 | { 492 | "cell_type": "markdown", 493 | "metadata": {}, 494 | "source": [ 495 | "**A single perceptron cannot solve the XOR problem, because it is not linearly separable. We need multi-layer perceptrons for that.**" 496 | ] 497 | } 498 | ], 499 | "metadata": { 500 | "colab": { 501 | "collapsed_sections": [ 502 | "JcF1mpJo-taz" 503 | ], 504 | "provenance": [] 505 | }, 506 | "kernelspec": { 507 | "display_name": "cv-ml-torch", 508 | "language": "python", 509 | "name": "cv-ml-torch" 510 | }, 511 | "language_info": { 512 | "codemirror_mode": { 513 | "name": "ipython", 514 | "version": 3 515 | }, 516 | "file_extension": ".py", 517 | "mimetype": "text/x-python", 518 | "name": "python", 519 | "nbconvert_exporter": "python", 520 | "pygments_lexer": "ipython3", 521 | "version": "3.10.13" 522 | } 523 | }, 524 | "nbformat": 4, 525 | "nbformat_minor": 0 526 | } 527 | -------------------------------------------------------------------------------- /perceptron/torch/requirements.txt: -------------------------------------------------------------------------------- 1 | torch 2 | numpy 3 | matplotlib 4 | -------------------------------------------------------------------------------- /requirements_mlx.txt: -------------------------------------------------------------------------------- 1 | numpy 2 | matplotlib 3 | Pillow 4 | mlx 5 | mlx-data -------------------------------------------------------------------------------- /requirements_torch.txt: -------------------------------------------------------------------------------- 1 | numpy 2 | matplotlib 3 | Pillow 4 | torch 5 | torchvision 6 | torchinfo 7 | 8 | -------------------------------------------------------------------------------- /setupOnColab.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Setup\n", 8 | "\n", 9 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 10 | "\n", 11 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute.\n", 12 | "\n", 13 | "Execute this notebook on Google Colab to mount your Google drive and checkout the cv-ml-lecture-notebooks repository. \n", 14 | "\n", 15 | "**Change `gdrive_repo_root` to your desired location on Google drive.**" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": null, 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "import os\n", 25 | "import subprocess\n", 26 | "\n", 27 | "# Google drive mount point\n", 28 | "gdrive_mnt = \"/content/drive\"\n", 29 | "\n", 30 | "####################\n", 31 | "### CHANGE THIS ####\n", 32 | "####################\n", 33 | "\n", 34 | "# Target dir for cv-ml-lecture-notebooks repo on Google Drive\n", 35 | "gdrive_repo_root = gdrive_mnt + \"/MyDrive/cv-ml-lecture-notebooks\"\n", 36 | "\n", 37 | "# Branch of cv-ml-lecture-notebooks to checkout\n", 38 | "branch_name = \"main\"\n", 39 | "###################" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": null, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "def clone(repo_url, repo_target_dir, branch_name, verbose=True):\n", 49 | " # Check if the directory already exists using the absolute path\n", 50 | " if os.path.exists(repo_target_dir):\n", 51 | " if verbose:\n", 52 | " print(\n", 53 | " f\"Repository {repo_target_dir} exists. Resetting to {branch_name} branch...\"\n", 54 | " )\n", 55 | " # reset to the specified branch\n", 56 | " reset_to_head_cmd(repo_target_dir, branch_name)\n", 57 | " else:\n", 58 | " if verbose:\n", 59 | " print(f\"Cloning repository {repo_url} on branch {branch_name}...\")\n", 60 | " # Clone the repository if it doesn't exist, and checkout the specified branch\n", 61 | " clone_cmd(repo_url, repo_target_dir, branch_name)\n", 62 | "\n", 63 | " if verbose:\n", 64 | " print(f\"Repository {repo_target_dir} is ready.\")\n", 65 | "\n", 66 | "\n", 67 | "def clone_cmd(repo_url, repo_target_dir, branch_name):\n", 68 | " subprocess.run([\"git\", \"clone\", \"-b\", branch_name, repo_url, repo_target_dir])\n", 69 | "\n", 70 | "\n", 71 | "def reset_to_head_cmd(repo_target_dir, branch_name):\n", 72 | " # Fetch the latest changes from the remote\n", 73 | " subprocess.run([\"git\", \"-C\", repo_target_dir, \"fetch\", \"origin\"])\n", 74 | " # Checkout the specified branch\n", 75 | " subprocess.run([\"git\", \"-C\", repo_target_dir, \"checkout\", branch_name])\n", 76 | " # Reset the local branch to the latest commit of the specified branch\n", 77 | " subprocess.run(\n", 78 | " [\"git\", \"-C\", repo_target_dir, \"reset\", \"--hard\", f\"origin/{branch_name}\"]\n", 79 | " )" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "# Mount Google Drive\n", 89 | "from google.colab import drive\n", 90 | "\n", 91 | "drive.mount(gdrive_mnt, force_remount=True)\n", 92 | "\n", 93 | "# Clone the repository\n", 94 | "repo_name = \"cv-ml-lecture-notebooks\"\n", 95 | "repo_url = f\"https://github.com/menzHSE/{repo_name}\"\n", 96 | "clone(repo_url, gdrive_repo_root, branch_name, verbose=True)" 97 | ] 98 | } 99 | ], 100 | "metadata": { 101 | "language_info": { 102 | "name": "python" 103 | } 104 | }, 105 | "nbformat": 4, 106 | "nbformat_minor": 2 107 | } 108 | -------------------------------------------------------------------------------- /vae/torch/requirements.txt: -------------------------------------------------------------------------------- 1 | matplotlib 2 | 3 | -------------------------------------------------------------------------------- /vae/torch/vae_torch_celeba.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "djI9z564ZZgD" 7 | }, 8 | "source": [ 9 | "# Convolutional Variational Autoencoders using PyTorch\n", 10 | "\n", 11 | "Markus Enzweiler, markus.enzweiler@hs-esslingen.de\n", 12 | "\n", 13 | "This is a demo used in a Computer Vision & Machine Learning lecture. Feel free to use and contribute." 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": { 19 | "id": "MWhhr5ouZZgG" 20 | }, 21 | "source": [ 22 | "**See `vae_torch_mnist.ipynb` for a more in-depth notebook on variational autoencoders.**" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": { 28 | "id": "E70btZtYZZgG" 29 | }, 30 | "source": [ 31 | "## Setup\n", 32 | "\n", 33 | "Adapt `packagePath` to point to the directory containing this notebeook." 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "metadata": { 40 | "id": "pvArrIPJZZgH" 41 | }, 42 | "outputs": [], 43 | "source": [ 44 | "# Notebook id\n", 45 | "nb_id = \"vae/torch\"\n", 46 | "\n", 47 | "# Imports\n", 48 | "import sys\n", 49 | "import os" 50 | ] 51 | }, 52 | { 53 | "cell_type": "code", 54 | "execution_count": null, 55 | "metadata": { 56 | "id": "bg-7y3C3ZZgI" 57 | }, 58 | "outputs": [], 59 | "source": [ 60 | "# Package Path (folder of this notebook)\n", 61 | "\n", 62 | "#####################\n", 63 | "# Local environment #\n", 64 | "#####################\n", 65 | "\n", 66 | "package_path = os.path.abspath(\"./\")\n", 67 | "\n", 68 | "\n", 69 | "#########\n", 70 | "# Colab #\n", 71 | "#########\n", 72 | "\n", 73 | "\n", 74 | "def check_for_colab():\n", 75 | " try:\n", 76 | " import google.colab\n", 77 | "\n", 78 | " return True\n", 79 | " except ImportError:\n", 80 | " return False\n", 81 | "\n", 82 | "\n", 83 | "# running on Colab?\n", 84 | "on_colab = check_for_colab()\n", 85 | "\n", 86 | "if on_colab:\n", 87 | " # assume this notebook is run from Google Drive and the whole\n", 88 | " # cv-ml-lecture-notebooks repo has been setup via setupOnColab.ipynb\n", 89 | "\n", 90 | " # Google Drive mount point\n", 91 | " gdrive_mnt = \"/content/drive\"\n", 92 | "\n", 93 | " ##########################################################################\n", 94 | " # Ensure that this is the same as gdrive_repo_root in setupOnColab.ipynb #\n", 95 | " ##########################################################################\n", 96 | " # Path on Google Drive to the cv-ml-lecture-notebooks repo\n", 97 | " gdrive_repo_root = f\"{gdrive_mnt}/MyDrive/cv-ml-lecture-notebooks\"\n", 98 | "\n", 99 | " # mount drive\n", 100 | " from google.colab import drive\n", 101 | "\n", 102 | " drive.mount(gdrive_mnt, force_remount=True)\n", 103 | "\n", 104 | " # set package path\n", 105 | " package_path = f\"{gdrive_repo_root}/{nb_id}\"\n", 106 | "\n", 107 | "# check whether package path exists\n", 108 | "if not os.path.isdir(package_path):\n", 109 | " raise FileNotFoundError(f\"Package path does not exist: {package_path}\")\n", 110 | "\n", 111 | "print(f\"Package path: {package_path}\")" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": null, 117 | "metadata": { 118 | "id": "Y2kXwImaZZgI" 119 | }, 120 | "outputs": [], 121 | "source": [ 122 | "# Additional imports\n", 123 | "\n", 124 | "# Repository Root\n", 125 | "repo_root = os.path.abspath(os.path.join(package_path, \"..\", \"..\"))\n", 126 | "# Add the repository root to the system path\n", 127 | "if repo_root not in sys.path:\n", 128 | " sys.path.append(repo_root)\n", 129 | "\n", 130 | "# Package Imports\n", 131 | "from nbutils import requirements as nb_reqs\n", 132 | "from nbutils import colab as nb_colab\n", 133 | "from nbutils import git as nb_git\n", 134 | "from nbutils import exec as nb_exec\n", 135 | "from nbutils import data as nb_data" 136 | ] 137 | }, 138 | { 139 | "cell_type": "code", 140 | "execution_count": null, 141 | "metadata": { 142 | "id": "uHjHq68vZZgI" 143 | }, 144 | "outputs": [], 145 | "source": [ 146 | "# Clone git repository\n", 147 | "\n", 148 | "# Absolute path of the repository directory\n", 149 | "repo_dir = os.path.join(package_path, \"torch-vae\")\n", 150 | "repo_url = \"https://github.com/menzHSE/torch-vae.git\"\n", 151 | "\n", 152 | "nb_git.clone(repo_url, repo_dir, on_colab)" 153 | ] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "execution_count": null, 158 | "metadata": { 159 | "id": "_LhO25t8ZZgI" 160 | }, 161 | "outputs": [], 162 | "source": [ 163 | "# Install requirements in the current Jupyter kernel\n", 164 | "req_file = os.path.join(repo_dir, \"requirements.txt\")\n", 165 | "nb_reqs.pip_install_reqs(req_file, on_colab)\n", 166 | "\n", 167 | "# Additional requirements for this notebook\n", 168 | "req_file = os.path.join(package_path, \"requirements.txt\")\n", 169 | "nb_reqs.pip_install_reqs(req_file, on_colab)" 170 | ] 171 | }, 172 | { 173 | "cell_type": "markdown", 174 | "metadata": { 175 | "id": "WCF7igNgZZgI" 176 | }, 177 | "source": [ 178 | "# Reconstruct and sample CelebA faces\n", 179 | "\n", 180 | "If the dataset cannot be automatically downloaded by PyTorch due to **daily quota exceeded** you can manually download it and put it in the ```data/celaba``` folder, see code cell below.\n", 181 | "\n", 182 | "The following files are necessary:\n", 183 | "- img_align_celeba.zip\n", 184 | "- list_attr_celeba.txt\n", 185 | "- list_bbox_celeba.txt\n", 186 | "- list_eval_partition.txt\n", 187 | "- list_landmarks_align_celeba.txt\n", 188 | "- list_landmarks_celeba.txt" 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "execution_count": null, 194 | "metadata": { 195 | "id": "M_Dk-jekZZgJ" 196 | }, 197 | "outputs": [], 198 | "source": [ 199 | "# To get around the problem of \"Quota Exceeded\" on the \"official\" CelebA\n", 200 | "# download via torchvision (https://github.com/pytorch/vision/issues/1920),\n", 201 | "# we use an alternative data source.\n", 202 | "\n", 203 | "# Set this to True to download the dataset from an alternative source\n", 204 | "use_alternative_data_source = True\n", 205 | "\n", 206 | "if use_alternative_data_source:\n", 207 | " print(\"Dowloading CelebA ... this will take a while\")\n", 208 | " nb_data.download_celeba(\"./data/celeba\", on_colab, verbose=False)" 209 | ] 210 | }, 211 | { 212 | "cell_type": "markdown", 213 | "metadata": { 214 | "id": "-8XU3omFZZgJ" 215 | }, 216 | "source": [ 217 | "## Load model and data" 218 | ] 219 | }, 220 | { 221 | "cell_type": "code", 222 | "execution_count": null, 223 | "metadata": { 224 | "id": "LG2xmE_UZZgJ" 225 | }, 226 | "outputs": [], 227 | "source": [ 228 | "import torch\n", 229 | "import torchvision\n", 230 | "import numpy as np\n", 231 | "import matplotlib.pyplot as plt\n", 232 | "\n", 233 | "# random seed\n", 234 | "seed = 0\n", 235 | "torch.manual_seed(seed)\n", 236 | "np.random.seed(seed)\n", 237 | "\n", 238 | "# Add the directory containing models.py to the system path\n", 239 | "sys.path.append(os.path.join(package_path, \"torch-vae\"))\n", 240 | "\n", 241 | "\n", 242 | "# Now we can import the model and dataset\n", 243 | "import model\n", 244 | "import dataset\n", 245 | "import device\n", 246 | "\n", 247 | "# parameters\n", 248 | "dataset_id = \"celeb-a\"\n", 249 | "num_latent_dims = 64\n", 250 | "max_num_filters = 128\n", 251 | "img_size = (64, 64)\n", 252 | "batch_size = 32\n", 253 | "model_id = f\"vae_filters_{max_num_filters:04d}_dims_{num_latent_dims:04d}.pth\"\n", 254 | "vae_fname = os.path.join(package_path, \"torch-vae\", \"models\", dataset_id, model_id)\n", 255 | "device = device.autoselectDevice()\n", 256 | "\n", 257 | "# load dataset\n", 258 | "(\n", 259 | " celeba_train_loader,\n", 260 | " celeba_test_loader,\n", 261 | " _,\n", 262 | " celeba_num_img_channels,\n", 263 | ") = dataset.get_loaders(dataset_id, img_size=img_size, batch_size=batch_size)\n", 264 | "\n", 265 | "# load the VAE model\n", 266 | "vae_celeb = model.VAE(num_latent_dims, celeba_num_img_channels, max_num_filters, device)\n", 267 | "vae_celeb.load_state_dict(torch.load(vae_fname, map_location=device))\n", 268 | "\n", 269 | "if vae_celeb:\n", 270 | " print(f\"Model {vae_fname} loaded successfully!\")\n", 271 | " print(f\"Device used: {device}\")\n", 272 | " vae_celeb.to(device)\n", 273 | " vae_celeb.eval()" 274 | ] 275 | }, 276 | { 277 | "cell_type": "markdown", 278 | "metadata": { 279 | "id": "rUML2NyoZZgJ" 280 | }, 281 | "source": [ 282 | "## Reconstruction of the CelebA test samples\n" 283 | ] 284 | }, 285 | { 286 | "cell_type": "code", 287 | "execution_count": null, 288 | "metadata": { 289 | "id": "i2irE1dDZZgJ" 290 | }, 291 | "outputs": [], 292 | "source": [ 293 | "def reconstructAndPlot(vae, num_latent_dims, data_loader):\n", 294 | " # Take the first batch from the data_loader\n", 295 | " data = next(iter(data_loader))\n", 296 | " with torch.no_grad():\n", 297 | " # Get the testing data and push the data to the device we are using\n", 298 | " images = data[0].to(device)\n", 299 | "\n", 300 | " # Reconstruct (encode and decode) the images\n", 301 | " images_recon = vae(images)\n", 302 | "\n", 303 | " # Interleave original and reconstructed images\n", 304 | " images_comparison = torch.stack([images, images_recon], dim=1).view(\n", 305 | " -1, *images.size()[1:]\n", 306 | " )\n", 307 | "\n", 308 | " # Display the images in a grid\n", 309 | " # nrow is set to 2 since we want each pair (original and reconstructed) to be side by side\n", 310 | " grid_img = torchvision.utils.make_grid(\n", 311 | " images_comparison.cpu(), nrow=batch_size // 4\n", 312 | " )\n", 313 | "\n", 314 | " # Convert grid to numpy and transpose axes for plotting\n", 315 | " grid_np = grid_img.numpy()\n", 316 | " grid_np = np.transpose(grid_np, (1, 2, 0))\n", 317 | "\n", 318 | " # Plotting\n", 319 | " plt.figure(figsize=(15, 15))\n", 320 | " plt.imshow(grid_np)\n", 321 | " plt.axis(\"off\")\n", 322 | " plt.title(\n", 323 | " f\"Original and Reconstructed Images with {num_latent_dims} Latent Dimensions\"\n", 324 | " )\n", 325 | " plt.show()" 326 | ] 327 | }, 328 | { 329 | "cell_type": "code", 330 | "execution_count": null, 331 | "metadata": { 332 | "id": "sRrIcXoQZZgK" 333 | }, 334 | "outputs": [], 335 | "source": [ 336 | "reconstructAndPlot(vae_celeb, num_latent_dims, celeba_test_loader)" 337 | ] 338 | }, 339 | { 340 | "cell_type": "markdown", 341 | "metadata": { 342 | "id": "-6iH5a13ZZgK" 343 | }, 344 | "source": [ 345 | "## Generate random CelebA-like samples from the VAE" 346 | ] 347 | }, 348 | { 349 | "cell_type": "code", 350 | "execution_count": null, 351 | "metadata": { 352 | "id": "mJpuOISEZZgK" 353 | }, 354 | "outputs": [], 355 | "source": [ 356 | "def sampleAndPlot(vae, num_latent_dims, num_samples=batch_size):\n", 357 | " with torch.no_grad():\n", 358 | " for i in range(num_samples):\n", 359 | " # generate a random latent vector\n", 360 | "\n", 361 | " # during training we have made sure that the distribution in latent\n", 362 | " # space remains close to a normal distribution\n", 363 | "\n", 364 | " z = torch.randn(num_latent_dims).to(device)\n", 365 | "\n", 366 | " # generate an image from the latent vector\n", 367 | " img = vae.decode(z)\n", 368 | "\n", 369 | " if i == 0:\n", 370 | " pics = img\n", 371 | " else:\n", 372 | " pics = torch.cat((pics, img), dim=0)\n", 373 | "\n", 374 | " # Create a grid of images\n", 375 | " grid_img = torchvision.utils.make_grid(pics, nrow=batch_size // 4)\n", 376 | "\n", 377 | " # Convert grid to numpy and transpose axes for plotting\n", 378 | " grid_np = grid_img.cpu().numpy()\n", 379 | " grid_np = np.transpose(grid_np, (1, 2, 0))\n", 380 | "\n", 381 | " # Plotting\n", 382 | " plt.figure(figsize=(10, 10))\n", 383 | " plt.imshow(grid_np)\n", 384 | " plt.axis(\"off\")\n", 385 | " plt.title(\n", 386 | " f\"Randomly Generated Images from the VAE with {num_latent_dims} Latent Dimensions\"\n", 387 | " )\n", 388 | " plt.show()" 389 | ] 390 | }, 391 | { 392 | "cell_type": "code", 393 | "execution_count": null, 394 | "metadata": { 395 | "id": "LmUcE_fyZZgK" 396 | }, 397 | "outputs": [], 398 | "source": [ 399 | "sampleAndPlot(vae_celeb, num_latent_dims)" 400 | ] 401 | } 402 | ], 403 | "metadata": { 404 | "accelerator": "GPU", 405 | "colab": { 406 | "gpuType": "T4", 407 | "provenance": [] 408 | }, 409 | "kernelspec": { 410 | "display_name": "cv-ml-torch", 411 | "language": "python", 412 | "name": "cv-ml-torch" 413 | }, 414 | "language_info": { 415 | "codemirror_mode": { 416 | "name": "ipython", 417 | "version": 3 418 | }, 419 | "file_extension": ".py", 420 | "mimetype": "text/x-python", 421 | "name": "python", 422 | "nbconvert_exporter": "python", 423 | "pygments_lexer": "ipython3", 424 | "version": "3.10.13" 425 | } 426 | }, 427 | "nbformat": 4, 428 | "nbformat_minor": 0 429 | } 430 | --------------------------------------------------------------------------------