├── .gitignore ├── environment.yaml ├── LICENSE ├── README.md ├── 1c_convolution.ipynb ├── 1b_softmax_function.ipynb ├── 1a_logistic_regression.ipynb └── 2a_convolutional_neural_network.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | .ipynb_checkpoints/ 2 | 3 | -------------------------------------------------------------------------------- /environment.yaml: -------------------------------------------------------------------------------- 1 | channels: 2 | - conda-forge 3 | - defaults 4 | - nvidia 5 | - pytorch 6 | dependencies: 7 | - cudatoolkit=11.1 8 | - imageio 9 | - ipython 10 | - matplotlib 11 | - pytorch 12 | - scikit-image 13 | - torchvision 14 | name: convnet_tutorials 15 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Laurens van der Maaten 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Convolutional Network Tutorials (AMMI 2025) 2 | 3 | These are the tutorials for the Computer Vision class of the African Master's in Machine Intelligence given by Laurens van der Maaten. 4 | 5 | The tutorials are iPython Notebooks in which you have to implements missing parts of the code yourself, run small experiments, and answer questions about the results of your experiments. The notebooks should be self-contained. 6 | 7 | ## Run notebooks on Google Cloud via colab 8 | 9 | You can run the notebooks on Google Cloud via colab. The advantage of this approach is that your experiments will run on Google's servers, which are likely faster than your own laptop. You will need a stable wifi connection to use colab. The following links take you to the colab notebooks: 10 | 11 | * [Tutorial 1a: Logistic Regression](https://colab.research.google.com/github/lvdmaaten/convnet_tutorials/blob/master/1a_logistic_regression.ipynb) 12 | * [Tutorial 1b: Softmax Function](https://colab.research.google.com/github/lvdmaaten/convnet_tutorials/blob/master/1b_softmax_function.ipynb) 13 | * [Tutorial 1c: Convolution](https://colab.research.google.com/github/lvdmaaten/convnet_tutorials/blob/master/1c_convolution.ipynb) 14 | * [Tutorial 2a: Convolutional Neural Network (CNN)](https://colab.research.google.com/github/lvdmaaten/convnet_tutorials/blob/master/2a_convolutional_neural_network.ipynb) 15 | * [Tutorial 2b: Batch Normalization](https://colab.research.google.com/github/lvdmaaten/convnet_tutorials/blob/master/2b_batch_normalization.ipynb) 16 | * [Tutorial 3: Residual Neural Network (ResNet)](https://colab.research.google.com/github/lvdmaaten/convnet_tutorials/blob/master/3_residual_neural_network.ipynb) 17 | 18 | ## Run notebooks locally using Jupyter 19 | 20 | You can also run the tutorial notebooks locally on your laptop. This way, you do not need an internet connection while you are working on most parts of the tutorial. 21 | 22 | Running notebooks locally is done via [Jupyter](https://jupyter.org/). If you do not have Jupyter installed yet, you first we need to create a Conda environment (see [here](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) for details). To do so, `git clone` this repository and run the following commands in your Terminal from the `convnet_tutorials` directory: 23 | 24 | You can install all packages from conda environment as follows: 25 | 26 | ``` 27 | conda env create -f environment.yaml 28 | 29 | conda activate convnet_tutorials 30 | ``` 31 | 32 | You can run `conda env list` to confirm the `convnet_tutorials` environment is activated. 33 | 34 | Next, start Jupyter Notebook by going into the `convnet_tutorials` directory in your Terminal and running `jupyter notebook`. This will open a browser window with the Jupyter notebook interface. In this interface, you can navigate to the correct folder and open the tutorial notebooks (with the `.ipynb` extension) to run them. 35 | -------------------------------------------------------------------------------- /1c_convolution.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "fbd4748d-9796-4331-b665-93e6763bd4e1", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from torch.nn import Conv2d\n", 11 | "import imageio\n", 12 | "import numpy\n", 13 | "import skimage.transform\n", 14 | "import torch\n", 15 | "import torch.nn.functional as F\n", 16 | "import matplotlib.pyplot" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "id": "21fe4293-e0f6-446f-8865-6fcb759b363b", 22 | "metadata": {}, 23 | "source": [ 24 | "# Tutorial 1c. Convolution\n", 25 | "\n", 26 | "The spatial dimensions of the ouput image (width and height) depend on the spatial dimensions of the input image, kernel_size, padding, and striding. In order to build efficient convolutional networks, it's important to understand what the sizes are after after each convolutional layer.\n", 27 | "\n", 28 | "In this exersise you will derive the dependency between input and output image sizes. For the sake of simplicity we assume that the input tensor is _square_, i.e., width = height = image_size.\n", 29 | "\n", 30 | "We will use the nn.Conv2d layer here. We have not yet discussed what a convolutional layer is yet, but if you set the first two parameters (input channels and output channels) to 1, then this defines a basic convolution.\n", 31 | "\n", 32 | "If your code is correct, you should see 'OK'." 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": null, 38 | "id": "46c2cca2-182a-4386-8f28-98569ccec982", 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "def compute_conv_output_size(image_size, kernel_size, padding, stride):\n", 43 | " ###########################################################################\n", 44 | " # Add code that computes the size of the image after a conv layer. #\n", 45 | " ###########################################################################\n", 46 | "\n", 47 | " return output_size\n", 48 | "\n", 49 | "\n", 50 | "# Compare the size of the output of nn.Conv2d with compute_convnet_output_size.\n", 51 | "for image_size in range(5, 21, 1):\n", 52 | " # Shape: batch x channels x height x width.\n", 53 | " input_tensor = torch.zeros((1, 1, image_size, image_size))\n", 54 | " \n", 55 | " for kernel_size in 2, 3, 5, 7:\n", 56 | " for padding in 0, 5:\n", 57 | " for stride in 1, 2, 3, 4:\n", 58 | " if kernel_size >= image_size:\n", 59 | " continue\n", 60 | " \n", 61 | " output_tensor = Conv2d(1, 1, kernel_size, stride, padding)(input_tensor)\n", 62 | " \n", 63 | " output_size = output_tensor.size(2)\n", 64 | " \n", 65 | " predicted_output_size = compute_conv_output_size(image_size, kernel_size, padding, stride)\n", 66 | " \n", 67 | " assert output_size == predicted_output_size, (\n", 68 | " f\"ERROR: the real size is {output_size},\"\n", 69 | " f\" but got {predicted_output_size}.\"\n", 70 | " f\"\\nimage_size={image_size}\"\n", 71 | " f\" kernel_size={kernel_size}\"\n", 72 | " f\" padding={padding}\"\n", 73 | " f\" stride={stride}\"\n", 74 | " )\n", 75 | "\n", 76 | "print(\"OK\")" 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "id": "af786813-5f81-4154-a5bb-f531f11db00e", 82 | "metadata": {}, 83 | "source": [ 84 | "You can now use the function you just implemented to compute the size of the output of a convolution." 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": null, 90 | "id": "cb4cb2fb-560e-4bae-9e1f-bc4b805ba3c4", 91 | "metadata": {}, 92 | "outputs": [], 93 | "source": [ 94 | "compute_conv_output_size(1, 1, 1, 1)" 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "id": "b6bb21a7-fc42-48f8-9ddf-3e47fdb499d6", 100 | "metadata": {}, 101 | "source": [ 102 | "**Question [optional]:** Implement your own convolution operator **without** using any of PyTorch's (or numpy's) pre-defined convolutional functions." 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": null, 108 | "id": "c6c11526-2778-48a5-a7cf-23a3d84f28e6", 109 | "metadata": {}, 110 | "outputs": [], 111 | "source": [ 112 | "def conv_naive(x, w, b, conv_param):\n", 113 | " \"\"\"\n", 114 | " A naive Python implementation of a convolution.\n", 115 | " The input consists of an image tensor with height H and\n", 116 | " width W. We convolve each input with a filter F, where the filter\n", 117 | " has height HH and width WW.\n", 118 | " Input:\n", 119 | " - x: Input data of shape (H, W)\n", 120 | " - w: Filter weights of shape (HH, WW)\n", 121 | " - b: Bias for filter\n", 122 | " - conv_param: A dictionary with the following keys:\n", 123 | " - 'stride': The number of pixels between adjacent receptive fields in the\n", 124 | " horizontal and vertical directions.\n", 125 | " - 'pad': The number of pixels that will be used to zero-pad the input.\n", 126 | "\n", 127 | " During padding, 'pad' zeros should be placed symmetrically (i.e equally on both sides)\n", 128 | " along the height and width axes of the input. Be careful not to modfiy the original\n", 129 | " input x directly.\n", 130 | " Returns an array.\n", 131 | " - out: Output data, of shape (H', W') where H' and W' are given by\n", 132 | " H' = 1 + (H + 2 * pad - HH) / stride\n", 133 | " W' = 1 + (W + 2 * pad - WW) / stride\n", 134 | " \"\"\"\n", 135 | " out = None\n", 136 | "\n", 137 | " H, W = x.shape\n", 138 | " filter_height, filter_width = w.shape\n", 139 | " stride, pad = conv_param[\"stride\"], conv_param[\"pad\"]\n", 140 | "\n", 141 | " # Check dimensions.\n", 142 | " assert (W + 2 * pad - filter_width) % stride == 0, \"width does not work\"\n", 143 | " assert (H + 2 * pad - filter_height) % stride == 0, \"height does not work\"\n", 144 | "\n", 145 | " ###########################################################################\n", 146 | " # TODO: Implement the convolutional forward pass. #\n", 147 | " # Hint: you can use the function torch.nn.functional.pad for padding. #\n", 148 | " ###########################################################################" 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "id": "e29c1544-2d7d-4cca-9244-7241b424de82", 154 | "metadata": {}, 155 | "source": [ 156 | "You can test your implementation by running the following:" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "id": "61dc9ae3-deec-4316-aafd-7fe67415dd9a", 163 | "metadata": {}, 164 | "outputs": [], 165 | "source": [ 166 | "# Make convolution module.\n", 167 | "w_shape = (4, 4)\n", 168 | "w = torch.linspace(-0.2, 0.3, steps=torch.prod(torch.tensor(w_shape))).reshape(w_shape)\n", 169 | "b = torch.linspace(-0.1, 0.2, steps=1)\n", 170 | "\n", 171 | "# Compute output of module and compare against reference values.\n", 172 | "x_shape = (4, 4)\n", 173 | "x = torch.linspace(-0.1, 0.5, steps=torch.prod(torch.tensor(x_shape))).reshape(x_shape)\n", 174 | "out = conv_naive(x, w, b, {\"stride\": 2, \"pad\": 1})\n", 175 | "\n", 176 | "correct_out = torch.tensor([[0.156, 0.162], [0.036, -0.054]])\n", 177 | "\n", 178 | "# Compare your output to ours; difference should be around e-8\n", 179 | "print(\"Testing conv_forward_naive\")\n", 180 | "rel_error = ((out - correct_out) / (out + correct_out + 1e-6)).mean()\n", 181 | "print(\"difference: \", rel_error)\n", 182 | "if abs(rel_error) < 1e-6:\n", 183 | " print(\"Nice work! Your implementation of a convolution layer works correctly.\")\n", 184 | "else:\n", 185 | " print(\n", 186 | " \"Something is wrong. The output was expected to be {} but it was {}\".format(\n", 187 | " correct_out, out\n", 188 | " )\n", 189 | " )" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "id": "7570b9ff-9030-4682-90cd-5ccdb8343a6d", 195 | "metadata": {}, 196 | "source": [ 197 | "**Aside: Image processing via convolutions:**\n", 198 | "\n", 199 | "As fun way to gain a better understanding of the type of operation that convolutional layers can perform, we will set up an input containing two images and manually set up filters that perform common image processing operations (grayscale conversion and edge detection). The convolution forward pass will apply these operations to each of the input images. We can then visualize the results as a sanity check." 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": null, 205 | "id": "300a0e4f-a87d-4f98-973d-45f37094123a", 206 | "metadata": {}, 207 | "outputs": [], 208 | "source": [ 209 | "# Load image of a kitten and a puppy.\n", 210 | "kitten_uri = \"https://upload.wikimedia.org/wikipedia/commons/thumb/1/1b/Persian_Cat_%28kitten%29.jpg/256px-Persian_Cat_%28kitten%29.jpg\"\n", 211 | "puppy_uri = \"https://upload.wikimedia.org/wikipedia/commons/thumb/6/6e/Golde33443.jpg/256px-Golde33443.jpg\"\n", 212 | "kitten, puppy = imageio.imread(kitten_uri), imageio.imread(puppy_uri)\n", 213 | "\n", 214 | "img_size = 200 # Make this smaller if it runs too slow\n", 215 | "x = numpy.zeros((2, 3, img_size, img_size))\n", 216 | "x[0, :, :, :] = skimage.transform.resize(puppy, (img_size, img_size)).transpose(\n", 217 | " (2, 0, 1)\n", 218 | ")\n", 219 | "x[1, :, :, :] = skimage.transform.resize(kitten, (img_size, img_size)).transpose(\n", 220 | " (2, 0, 1)\n", 221 | ")\n", 222 | "x = torch.FloatTensor(x)\n", 223 | "\n", 224 | "# Set up a convolutional weights holding 2 filters, each 3x3\n", 225 | "w = torch.zeros((2, 3, 3, 3), dtype=torch.float)\n", 226 | "\n", 227 | "# The first filter converts the image to grayscale.\n", 228 | "# Set up the red, green, and blue channels of the filter.\n", 229 | "w[0, 0, :, :] = torch.tensor([[0, 0, 0], [0, 0.3, 0], [0, 0, 0]])\n", 230 | "w[0, 1, :, :] = torch.tensor([[0, 0, 0], [0, 0.6, 0], [0, 0, 0]])\n", 231 | "w[0, 2, :, :] = torch.tensor([[0, 0, 0], [0, 0.1, 0], [0, 0, 0]])\n", 232 | "\n", 233 | "# Second filter detects horizontal edges in the blue channel.\n", 234 | "w[1, 2, :, :] = torch.tensor([[1, 2, 1], [0, 0, 0], [-1, -2, -1]])\n", 235 | "\n", 236 | "# Vector of biases. We don't need any bias for the grayscale\n", 237 | "# filter, but for the edge detection filter we want to add 128\n", 238 | "# to each output so that nothing is negative.\n", 239 | "b = torch.tensor([0, 128], dtype=torch.float)\n", 240 | "\n", 241 | "# Compute the result of convolving each input in x with each filter in w,\n", 242 | "# offsetting by b, and storing the results in out.\n", 243 | "out = F.conv2d(x, w, b, stride=1, padding=1).numpy()\n", 244 | "\n", 245 | "\n", 246 | "def imshow_noax(img, normalize=True):\n", 247 | " \"\"\"Tiny helper to show images as uint8 and remove axis labels.\"\"\"\n", 248 | " if normalize:\n", 249 | " img_max, img_min = numpy.max(img), numpy.min(img)\n", 250 | " img = 255.0 * (img - img_min) / (img_max - img_min)\n", 251 | " matplotlib.pyplot.imshow(img.astype(\"uint8\"))\n", 252 | " matplotlib.pyplot.gca().axis(\"off\")\n", 253 | "\n", 254 | "\n", 255 | "# Show the original images and the results of the conv operation\n", 256 | "matplotlib.pyplot.subplot(2, 3, 1)\n", 257 | "imshow_noax(puppy, normalize=False)\n", 258 | "matplotlib.pyplot.title(\"Original image\")\n", 259 | "matplotlib.pyplot.subplot(2, 3, 2)\n", 260 | "imshow_noax(out[0, 0])\n", 261 | "matplotlib.pyplot.title(\"Grayscale\")\n", 262 | "matplotlib.pyplot.subplot(2, 3, 3)\n", 263 | "imshow_noax(out[0, 1])\n", 264 | "matplotlib.pyplot.title(\"Edges\")\n", 265 | "matplotlib.pyplot.subplot(2, 3, 4)\n", 266 | "imshow_noax(kitten, normalize=False)\n", 267 | "matplotlib.pyplot.subplot(2, 3, 5)\n", 268 | "imshow_noax(out[1, 0])\n", 269 | "matplotlib.pyplot.subplot(2, 3, 6)\n", 270 | "imshow_noax(out[1, 1])\n", 271 | "matplotlib.pyplot.show()" 272 | ] 273 | } 274 | ], 275 | "metadata": { 276 | "kernelspec": { 277 | "display_name": "convnet_tutorials", 278 | "language": "python", 279 | "name": "convnet_tutorials" 280 | }, 281 | "language_info": { 282 | "codemirror_mode": { 283 | "name": "ipython", 284 | "version": 3 285 | }, 286 | "file_extension": ".py", 287 | "mimetype": "text/x-python", 288 | "name": "python", 289 | "nbconvert_exporter": "python", 290 | "pygments_lexer": "ipython3", 291 | "version": "3.9.10" 292 | } 293 | }, 294 | "nbformat": 4, 295 | "nbformat_minor": 5 296 | } 297 | -------------------------------------------------------------------------------- /1b_softmax_function.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": { 7 | "colab": {}, 8 | "colab_type": "code", 9 | "id": "hqnl0AKVXIA4" 10 | }, 11 | "outputs": [], 12 | "source": [ 13 | "import torch\n", 14 | "from torch import Tensor" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "# Tutorial 1b: Softmax Function" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": { 27 | "colab_type": "text", 28 | "id": "g1DV-MS2bxYq" 29 | }, 30 | "source": [ 31 | "**Question:** To have the logistic regressor output probabilities, they need to be processed through a softmax layer. Implement a softmax layer yourself. What numerical issues may arise in this layer? How can you solve them? Use the testing code to confirm you implemented it correctly." 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": null, 37 | "metadata": {}, 38 | "outputs": [], 39 | "source": [ 40 | "logits = torch.rand((1, 20)) + 100" 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": null, 46 | "metadata": { 47 | "colab": { 48 | "base_uri": "https://localhost:8080/", 49 | "height": 34 50 | }, 51 | "colab_type": "code", 52 | "executionInfo": { 53 | "elapsed": 38247, 54 | "status": "ok", 55 | "timestamp": 1586535123635, 56 | "user": { 57 | "displayName": "Anton Bakhtin", 58 | "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gjq-QU8evTujx-PnMoIketNeW4yB_kgdtnRJ03IxnY=s64", 59 | "userId": "12331399045734156147" 60 | }, 61 | "user_tz": 240 62 | }, 63 | "id": "Dj4X2PnOfK9W", 64 | "outputId": "f3b3f1e7-682d-4fbc-cc57-367badf11b7b" 65 | }, 66 | "outputs": [], 67 | "source": [ 68 | "def bad_softmax(x: Tensor) -> Tensor:\n", 69 | " return torch.exp(x) / torch.sum(torch.exp(x), axis=1)" 70 | ] 71 | }, 72 | { 73 | "cell_type": "code", 74 | "execution_count": null, 75 | "metadata": {}, 76 | "outputs": [], 77 | "source": [ 78 | "torch.sum(bad_softmax(logits))" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "metadata": {}, 85 | "outputs": [], 86 | "source": [ 87 | "def good_softmax(x: Tensor) -> Tensor:\n", 88 | " ###########################################################################\n", 89 | " # TODO: Implement a more stable way to compute softmax #\n", 90 | " ###########################################################################\n", 91 | " pass" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "torch.sum(good_softmax(logits))" 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": { 106 | "colab_type": "text", 107 | "id": "4C_J5S0RScXJ" 108 | }, 109 | "source": [ 110 | "Because of numerical issues like the one you just experiences, PyTorch code typically uses a `LogSoftmax` layer." 111 | ] 112 | }, 113 | { 114 | "cell_type": "markdown", 115 | "metadata": { 116 | "colab_type": "text", 117 | "id": "lgStX-ctjIms" 118 | }, 119 | "source": [ 120 | "**Question [optional]:** PyTorch automatically computes the backpropagation gradient of a module for you. However, it can be instructive to derive and implement your own backward function. Try and implement the backward function for your softmax module and confirm that it is correct." 121 | ] 122 | } 123 | ], 124 | "metadata": { 125 | "accelerator": "GPU", 126 | "colab": { 127 | "collapsed_sections": [], 128 | "name": "convnet_tutorial1_filled.ipynb", 129 | "provenance": [ 130 | { 131 | "file_id": "1Eq_v8eTk4XAxFXhjL6rk26gwfwvpZYBn", 132 | "timestamp": 1553454632964 133 | } 134 | ] 135 | }, 136 | "kernelspec": { 137 | "display_name": "convnet_tutorials", 138 | "language": "python", 139 | "name": "convnet_tutorials" 140 | }, 141 | "language_info": { 142 | "codemirror_mode": { 143 | "name": "ipython", 144 | "version": 3 145 | }, 146 | "file_extension": ".py", 147 | "mimetype": "text/x-python", 148 | "name": "python", 149 | "nbconvert_exporter": "python", 150 | "pygments_lexer": "ipython3", 151 | "version": "3.9.10" 152 | }, 153 | "widgets": { 154 | "application/vnd.jupyter.widget-state+json": { 155 | "04f7f68d4f784cf19f942ee29c24ecc7": { 156 | "model_module": "@jupyter-widgets/base", 157 | "model_name": "LayoutModel", 158 | "state": { 159 | "_model_module": "@jupyter-widgets/base", 160 | "_model_module_version": "1.2.0", 161 | "_model_name": "LayoutModel", 162 | "_view_count": null, 163 | "_view_module": "@jupyter-widgets/base", 164 | "_view_module_version": "1.2.0", 165 | "_view_name": "LayoutView", 166 | "align_content": null, 167 | "align_items": null, 168 | "align_self": null, 169 | "border": null, 170 | "bottom": null, 171 | "display": null, 172 | "flex": null, 173 | "flex_flow": null, 174 | "grid_area": null, 175 | "grid_auto_columns": null, 176 | "grid_auto_flow": null, 177 | "grid_auto_rows": null, 178 | "grid_column": null, 179 | "grid_gap": null, 180 | "grid_row": null, 181 | "grid_template_areas": null, 182 | "grid_template_columns": null, 183 | "grid_template_rows": null, 184 | "height": null, 185 | "justify_content": null, 186 | "justify_items": null, 187 | "left": null, 188 | "margin": null, 189 | "max_height": null, 190 | "max_width": null, 191 | "min_height": null, 192 | "min_width": null, 193 | "object_fit": null, 194 | "object_position": null, 195 | "order": null, 196 | "overflow": null, 197 | "overflow_x": null, 198 | "overflow_y": null, 199 | "padding": null, 200 | "right": null, 201 | "top": null, 202 | "visibility": null, 203 | "width": null 204 | } 205 | }, 206 | "10c91bce2e8b42c587557b0a649a79fe": { 207 | "model_module": "@jupyter-widgets/base", 208 | "model_name": "LayoutModel", 209 | "state": { 210 | "_model_module": "@jupyter-widgets/base", 211 | "_model_module_version": "1.2.0", 212 | "_model_name": "LayoutModel", 213 | "_view_count": null, 214 | "_view_module": "@jupyter-widgets/base", 215 | "_view_module_version": "1.2.0", 216 | "_view_name": "LayoutView", 217 | "align_content": null, 218 | "align_items": null, 219 | "align_self": null, 220 | "border": null, 221 | "bottom": null, 222 | "display": null, 223 | "flex": null, 224 | "flex_flow": null, 225 | "grid_area": null, 226 | "grid_auto_columns": null, 227 | "grid_auto_flow": null, 228 | "grid_auto_rows": null, 229 | "grid_column": null, 230 | "grid_gap": null, 231 | "grid_row": null, 232 | "grid_template_areas": null, 233 | "grid_template_columns": null, 234 | "grid_template_rows": null, 235 | "height": null, 236 | "justify_content": null, 237 | "justify_items": null, 238 | "left": null, 239 | "margin": null, 240 | "max_height": null, 241 | "max_width": null, 242 | "min_height": null, 243 | "min_width": null, 244 | "object_fit": null, 245 | "object_position": null, 246 | "order": null, 247 | "overflow": null, 248 | "overflow_x": null, 249 | "overflow_y": null, 250 | "padding": null, 251 | "right": null, 252 | "top": null, 253 | "visibility": null, 254 | "width": null 255 | } 256 | }, 257 | "1a38c10f1d0e4f20a4958d453757f9e7": { 258 | "model_module": "@jupyter-widgets/controls", 259 | "model_name": "DescriptionStyleModel", 260 | "state": { 261 | "_model_module": "@jupyter-widgets/controls", 262 | "_model_module_version": "1.5.0", 263 | "_model_name": "DescriptionStyleModel", 264 | "_view_count": null, 265 | "_view_module": "@jupyter-widgets/base", 266 | "_view_module_version": "1.2.0", 267 | "_view_name": "StyleView", 268 | "description_width": "" 269 | } 270 | }, 271 | "200b60fab8514008a571603c6a2519f8": { 272 | "model_module": "@jupyter-widgets/controls", 273 | "model_name": "IntProgressModel", 274 | "state": { 275 | "_dom_classes": [], 276 | "_model_module": "@jupyter-widgets/controls", 277 | "_model_module_version": "1.5.0", 278 | "_model_name": "IntProgressModel", 279 | "_view_count": null, 280 | "_view_module": "@jupyter-widgets/controls", 281 | "_view_module_version": "1.5.0", 282 | "_view_name": "ProgressView", 283 | "bar_style": "success", 284 | "description": "", 285 | "description_tooltip": null, 286 | "layout": "IPY_MODEL_04f7f68d4f784cf19f942ee29c24ecc7", 287 | "max": 1, 288 | "min": 0, 289 | "orientation": "horizontal", 290 | "style": "IPY_MODEL_8f98aca9cfae490dab5dccf54b939ce0", 291 | "value": 1 292 | } 293 | }, 294 | "229d85ab3245444c9ad5b57c6ec58688": { 295 | "model_module": "@jupyter-widgets/base", 296 | "model_name": "LayoutModel", 297 | "state": { 298 | "_model_module": "@jupyter-widgets/base", 299 | "_model_module_version": "1.2.0", 300 | "_model_name": "LayoutModel", 301 | "_view_count": null, 302 | "_view_module": "@jupyter-widgets/base", 303 | "_view_module_version": "1.2.0", 304 | "_view_name": "LayoutView", 305 | "align_content": null, 306 | "align_items": null, 307 | "align_self": null, 308 | "border": null, 309 | "bottom": null, 310 | "display": null, 311 | "flex": null, 312 | "flex_flow": null, 313 | "grid_area": null, 314 | "grid_auto_columns": null, 315 | "grid_auto_flow": null, 316 | "grid_auto_rows": null, 317 | "grid_column": null, 318 | "grid_gap": null, 319 | "grid_row": null, 320 | "grid_template_areas": null, 321 | "grid_template_columns": null, 322 | "grid_template_rows": null, 323 | "height": null, 324 | "justify_content": null, 325 | "justify_items": null, 326 | "left": null, 327 | "margin": null, 328 | "max_height": null, 329 | "max_width": null, 330 | "min_height": null, 331 | "min_width": null, 332 | "object_fit": null, 333 | "object_position": null, 334 | "order": null, 335 | "overflow": null, 336 | "overflow_x": null, 337 | "overflow_y": null, 338 | "padding": null, 339 | "right": null, 340 | "top": null, 341 | "visibility": null, 342 | "width": null 343 | } 344 | }, 345 | "2de5166b2a40479fb69c08e7fda49594": { 346 | "model_module": "@jupyter-widgets/controls", 347 | "model_name": "IntProgressModel", 348 | "state": { 349 | "_dom_classes": [], 350 | "_model_module": "@jupyter-widgets/controls", 351 | "_model_module_version": "1.5.0", 352 | "_model_name": "IntProgressModel", 353 | "_view_count": null, 354 | "_view_module": "@jupyter-widgets/controls", 355 | "_view_module_version": "1.5.0", 356 | "_view_name": "ProgressView", 357 | "bar_style": "info", 358 | "description": " 0%", 359 | "description_tooltip": null, 360 | "layout": "IPY_MODEL_d57756abec8742db9d650c814b474137", 361 | "max": 1, 362 | "min": 0, 363 | "orientation": "horizontal", 364 | "style": "IPY_MODEL_bcbba5e1db1d498eb4b9ce545ff26d8a", 365 | "value": 0 366 | } 367 | }, 368 | "34a38f1355e54a24ad0757e91e016ff4": { 369 | "model_module": "@jupyter-widgets/controls", 370 | "model_name": "HTMLModel", 371 | "state": { 372 | "_dom_classes": [], 373 | "_model_module": "@jupyter-widgets/controls", 374 | "_model_module_version": "1.5.0", 375 | "_model_name": "HTMLModel", 376 | "_view_count": null, 377 | "_view_module": "@jupyter-widgets/controls", 378 | "_view_module_version": "1.5.0", 379 | "_view_name": "HTMLView", 380 | "description": "", 381 | "description_tooltip": null, 382 | "layout": "IPY_MODEL_466dc3fe5f2c447ebc9f7259e886e606", 383 | "placeholder": "​", 384 | "style": "IPY_MODEL_99bb01a7fb144b149460669c65b22654", 385 | "value": " 1654784/? [00:00<00:00, 1924711.94it/s]" 386 | } 387 | }, 388 | "3a488dd9262d42ac9b119cd37e2e71c9": { 389 | "model_module": "@jupyter-widgets/controls", 390 | "model_name": "ProgressStyleModel", 391 | "state": { 392 | "_model_module": "@jupyter-widgets/controls", 393 | "_model_module_version": "1.5.0", 394 | "_model_name": "ProgressStyleModel", 395 | "_view_count": null, 396 | "_view_module": "@jupyter-widgets/base", 397 | "_view_module_version": "1.2.0", 398 | "_view_name": "StyleView", 399 | "bar_color": null, 400 | "description_width": "initial" 401 | } 402 | }, 403 | "3c129350f337408f920706b3836528b9": { 404 | "model_module": "@jupyter-widgets/controls", 405 | "model_name": "DescriptionStyleModel", 406 | "state": { 407 | "_model_module": "@jupyter-widgets/controls", 408 | "_model_module_version": "1.5.0", 409 | "_model_name": "DescriptionStyleModel", 410 | "_view_count": null, 411 | "_view_module": "@jupyter-widgets/base", 412 | "_view_module_version": "1.2.0", 413 | "_view_name": "StyleView", 414 | "description_width": "" 415 | } 416 | }, 417 | "41f12163cbd84febb7fade8126ed7d5e": { 418 | "model_module": "@jupyter-widgets/controls", 419 | "model_name": "HBoxModel", 420 | "state": { 421 | "_dom_classes": [], 422 | "_model_module": "@jupyter-widgets/controls", 423 | "_model_module_version": "1.5.0", 424 | "_model_name": "HBoxModel", 425 | "_view_count": null, 426 | "_view_module": "@jupyter-widgets/controls", 427 | "_view_module_version": "1.5.0", 428 | "_view_name": "HBoxView", 429 | "box_style": "", 430 | "children": [ 431 | "IPY_MODEL_8f7373518a7840b38d00de7608a9e218", 432 | "IPY_MODEL_e92daa0c7043415b8463dffd654e93e5" 433 | ], 434 | "layout": "IPY_MODEL_48dc62d211994fb1b3b323da3c8c8a26" 435 | } 436 | }, 437 | "466dc3fe5f2c447ebc9f7259e886e606": { 438 | "model_module": "@jupyter-widgets/base", 439 | "model_name": "LayoutModel", 440 | "state": { 441 | "_model_module": "@jupyter-widgets/base", 442 | "_model_module_version": "1.2.0", 443 | "_model_name": "LayoutModel", 444 | "_view_count": null, 445 | "_view_module": "@jupyter-widgets/base", 446 | "_view_module_version": "1.2.0", 447 | "_view_name": "LayoutView", 448 | "align_content": null, 449 | "align_items": null, 450 | "align_self": null, 451 | "border": null, 452 | "bottom": null, 453 | "display": null, 454 | "flex": null, 455 | "flex_flow": null, 456 | "grid_area": null, 457 | "grid_auto_columns": null, 458 | "grid_auto_flow": null, 459 | "grid_auto_rows": null, 460 | "grid_column": null, 461 | "grid_gap": null, 462 | "grid_row": null, 463 | "grid_template_areas": null, 464 | "grid_template_columns": null, 465 | "grid_template_rows": null, 466 | "height": null, 467 | "justify_content": null, 468 | "justify_items": null, 469 | "left": null, 470 | "margin": null, 471 | "max_height": null, 472 | "max_width": null, 473 | "min_height": null, 474 | "min_width": null, 475 | "object_fit": null, 476 | "object_position": null, 477 | "order": null, 478 | "overflow": null, 479 | "overflow_x": null, 480 | "overflow_y": null, 481 | "padding": null, 482 | "right": null, 483 | "top": null, 484 | "visibility": null, 485 | "width": null 486 | } 487 | }, 488 | "48dc62d211994fb1b3b323da3c8c8a26": { 489 | "model_module": "@jupyter-widgets/base", 490 | "model_name": "LayoutModel", 491 | "state": { 492 | "_model_module": "@jupyter-widgets/base", 493 | "_model_module_version": "1.2.0", 494 | "_model_name": "LayoutModel", 495 | "_view_count": null, 496 | "_view_module": "@jupyter-widgets/base", 497 | "_view_module_version": "1.2.0", 498 | "_view_name": "LayoutView", 499 | "align_content": null, 500 | "align_items": null, 501 | "align_self": null, 502 | "border": null, 503 | "bottom": null, 504 | "display": null, 505 | "flex": null, 506 | "flex_flow": null, 507 | "grid_area": null, 508 | "grid_auto_columns": null, 509 | "grid_auto_flow": null, 510 | "grid_auto_rows": null, 511 | "grid_column": null, 512 | "grid_gap": null, 513 | "grid_row": null, 514 | "grid_template_areas": null, 515 | "grid_template_columns": null, 516 | "grid_template_rows": null, 517 | "height": null, 518 | "justify_content": null, 519 | "justify_items": null, 520 | "left": null, 521 | "margin": null, 522 | "max_height": null, 523 | "max_width": null, 524 | "min_height": null, 525 | "min_width": null, 526 | "object_fit": null, 527 | "object_position": null, 528 | "order": null, 529 | "overflow": null, 530 | "overflow_x": null, 531 | "overflow_y": null, 532 | "padding": null, 533 | "right": null, 534 | "top": null, 535 | "visibility": null, 536 | "width": null 537 | } 538 | }, 539 | "590b36901152442088a4c5e05e781b3d": { 540 | "model_module": "@jupyter-widgets/base", 541 | "model_name": "LayoutModel", 542 | "state": { 543 | "_model_module": "@jupyter-widgets/base", 544 | "_model_module_version": "1.2.0", 545 | "_model_name": "LayoutModel", 546 | "_view_count": null, 547 | "_view_module": "@jupyter-widgets/base", 548 | "_view_module_version": "1.2.0", 549 | "_view_name": "LayoutView", 550 | "align_content": null, 551 | "align_items": null, 552 | "align_self": null, 553 | "border": null, 554 | "bottom": null, 555 | "display": null, 556 | "flex": null, 557 | "flex_flow": null, 558 | "grid_area": null, 559 | "grid_auto_columns": null, 560 | "grid_auto_flow": null, 561 | "grid_auto_rows": null, 562 | "grid_column": null, 563 | "grid_gap": null, 564 | "grid_row": null, 565 | "grid_template_areas": null, 566 | "grid_template_columns": null, 567 | "grid_template_rows": null, 568 | "height": null, 569 | "justify_content": null, 570 | "justify_items": null, 571 | "left": null, 572 | "margin": null, 573 | "max_height": null, 574 | "max_width": null, 575 | "min_height": null, 576 | "min_width": null, 577 | "object_fit": null, 578 | "object_position": null, 579 | "order": null, 580 | "overflow": null, 581 | "overflow_x": null, 582 | "overflow_y": null, 583 | "padding": null, 584 | "right": null, 585 | "top": null, 586 | "visibility": null, 587 | "width": null 588 | } 589 | }, 590 | "59a8e0ee48e14c9691224d013d6c46fe": { 591 | "model_module": "@jupyter-widgets/controls", 592 | "model_name": "HBoxModel", 593 | "state": { 594 | "_dom_classes": [], 595 | "_model_module": "@jupyter-widgets/controls", 596 | "_model_module_version": "1.5.0", 597 | "_model_name": "HBoxModel", 598 | "_view_count": null, 599 | "_view_module": "@jupyter-widgets/controls", 600 | "_view_module_version": "1.5.0", 601 | "_view_name": "HBoxView", 602 | "box_style": "", 603 | "children": [ 604 | "IPY_MODEL_bcc108058dc4490986ec987c568b2034", 605 | "IPY_MODEL_ab7e6d830cc74c14876a052480eb185a" 606 | ], 607 | "layout": "IPY_MODEL_ee8b4317d860412fa60bca6d1d0c935a" 608 | } 609 | }, 610 | "655321e930014d03a986ab862d9c01a1": { 611 | "model_module": "@jupyter-widgets/controls", 612 | "model_name": "HTMLModel", 613 | "state": { 614 | "_dom_classes": [], 615 | "_model_module": "@jupyter-widgets/controls", 616 | "_model_module_version": "1.5.0", 617 | "_model_name": "HTMLModel", 618 | "_view_count": null, 619 | "_view_module": "@jupyter-widgets/controls", 620 | "_view_module_version": "1.5.0", 621 | "_view_name": "HTMLView", 622 | "description": "", 623 | "description_tooltip": null, 624 | "layout": "IPY_MODEL_78f9f8dd4275405b96194f2507dae175", 625 | "placeholder": "​", 626 | "style": "IPY_MODEL_3c129350f337408f920706b3836528b9", 627 | "value": " 0/4542 [00:00<?, ?it/s]" 628 | } 629 | }, 630 | "70658583c62a40c28345d43ceb70810b": { 631 | "model_module": "@jupyter-widgets/base", 632 | "model_name": "LayoutModel", 633 | "state": { 634 | "_model_module": "@jupyter-widgets/base", 635 | "_model_module_version": "1.2.0", 636 | "_model_name": "LayoutModel", 637 | "_view_count": null, 638 | "_view_module": "@jupyter-widgets/base", 639 | "_view_module_version": "1.2.0", 640 | "_view_name": "LayoutView", 641 | "align_content": null, 642 | "align_items": null, 643 | "align_self": null, 644 | "border": null, 645 | "bottom": null, 646 | "display": null, 647 | "flex": null, 648 | "flex_flow": null, 649 | "grid_area": null, 650 | "grid_auto_columns": null, 651 | "grid_auto_flow": null, 652 | "grid_auto_rows": null, 653 | "grid_column": null, 654 | "grid_gap": null, 655 | "grid_row": null, 656 | "grid_template_areas": null, 657 | "grid_template_columns": null, 658 | "grid_template_rows": null, 659 | "height": null, 660 | "justify_content": null, 661 | "justify_items": null, 662 | "left": null, 663 | "margin": null, 664 | "max_height": null, 665 | "max_width": null, 666 | "min_height": null, 667 | "min_width": null, 668 | "object_fit": null, 669 | "object_position": null, 670 | "order": null, 671 | "overflow": null, 672 | "overflow_x": null, 673 | "overflow_y": null, 674 | "padding": null, 675 | "right": null, 676 | "top": null, 677 | "visibility": null, 678 | "width": null 679 | } 680 | }, 681 | "78f9f8dd4275405b96194f2507dae175": { 682 | "model_module": "@jupyter-widgets/base", 683 | "model_name": "LayoutModel", 684 | "state": { 685 | "_model_module": "@jupyter-widgets/base", 686 | "_model_module_version": "1.2.0", 687 | "_model_name": "LayoutModel", 688 | "_view_count": null, 689 | "_view_module": "@jupyter-widgets/base", 690 | "_view_module_version": "1.2.0", 691 | "_view_name": "LayoutView", 692 | "align_content": null, 693 | "align_items": null, 694 | "align_self": null, 695 | "border": null, 696 | "bottom": null, 697 | "display": null, 698 | "flex": null, 699 | "flex_flow": null, 700 | "grid_area": null, 701 | "grid_auto_columns": null, 702 | "grid_auto_flow": null, 703 | "grid_auto_rows": null, 704 | "grid_column": null, 705 | "grid_gap": null, 706 | "grid_row": null, 707 | "grid_template_areas": null, 708 | "grid_template_columns": null, 709 | "grid_template_rows": null, 710 | "height": null, 711 | "justify_content": null, 712 | "justify_items": null, 713 | "left": null, 714 | "margin": null, 715 | "max_height": null, 716 | "max_width": null, 717 | "min_height": null, 718 | "min_width": null, 719 | "object_fit": null, 720 | "object_position": null, 721 | "order": null, 722 | "overflow": null, 723 | "overflow_x": null, 724 | "overflow_y": null, 725 | "padding": null, 726 | "right": null, 727 | "top": null, 728 | "visibility": null, 729 | "width": null 730 | } 731 | }, 732 | "8f7373518a7840b38d00de7608a9e218": { 733 | "model_module": "@jupyter-widgets/controls", 734 | "model_name": "IntProgressModel", 735 | "state": { 736 | "_dom_classes": [], 737 | "_model_module": "@jupyter-widgets/controls", 738 | "_model_module_version": "1.5.0", 739 | "_model_name": "IntProgressModel", 740 | "_view_count": null, 741 | "_view_module": "@jupyter-widgets/controls", 742 | "_view_module_version": "1.5.0", 743 | "_view_name": "ProgressView", 744 | "bar_style": "success", 745 | "description": "", 746 | "description_tooltip": null, 747 | "layout": "IPY_MODEL_10c91bce2e8b42c587557b0a649a79fe", 748 | "max": 1, 749 | "min": 0, 750 | "orientation": "horizontal", 751 | "style": "IPY_MODEL_df0848e4bca84d95b178cbe955417bc4", 752 | "value": 1 753 | } 754 | }, 755 | "8f98aca9cfae490dab5dccf54b939ce0": { 756 | "model_module": "@jupyter-widgets/controls", 757 | "model_name": "ProgressStyleModel", 758 | "state": { 759 | "_model_module": "@jupyter-widgets/controls", 760 | "_model_module_version": "1.5.0", 761 | "_model_name": "ProgressStyleModel", 762 | "_view_count": null, 763 | "_view_module": "@jupyter-widgets/base", 764 | "_view_module_version": "1.2.0", 765 | "_view_name": "StyleView", 766 | "bar_color": null, 767 | "description_width": "initial" 768 | } 769 | }, 770 | "99bb01a7fb144b149460669c65b22654": { 771 | "model_module": "@jupyter-widgets/controls", 772 | "model_name": "DescriptionStyleModel", 773 | "state": { 774 | "_model_module": "@jupyter-widgets/controls", 775 | "_model_module_version": "1.5.0", 776 | "_model_name": "DescriptionStyleModel", 777 | "_view_count": null, 778 | "_view_module": "@jupyter-widgets/base", 779 | "_view_module_version": "1.2.0", 780 | "_view_name": "StyleView", 781 | "description_width": "" 782 | } 783 | }, 784 | "a739b97fb09848dbae2fb6d0fef09ef4": { 785 | "model_module": "@jupyter-widgets/base", 786 | "model_name": "LayoutModel", 787 | "state": { 788 | "_model_module": "@jupyter-widgets/base", 789 | "_model_module_version": "1.2.0", 790 | "_model_name": "LayoutModel", 791 | "_view_count": null, 792 | "_view_module": "@jupyter-widgets/base", 793 | "_view_module_version": "1.2.0", 794 | "_view_name": "LayoutView", 795 | "align_content": null, 796 | "align_items": null, 797 | "align_self": null, 798 | "border": null, 799 | "bottom": null, 800 | "display": null, 801 | "flex": null, 802 | "flex_flow": null, 803 | "grid_area": null, 804 | "grid_auto_columns": null, 805 | "grid_auto_flow": null, 806 | "grid_auto_rows": null, 807 | "grid_column": null, 808 | "grid_gap": null, 809 | "grid_row": null, 810 | "grid_template_areas": null, 811 | "grid_template_columns": null, 812 | "grid_template_rows": null, 813 | "height": null, 814 | "justify_content": null, 815 | "justify_items": null, 816 | "left": null, 817 | "margin": null, 818 | "max_height": null, 819 | "max_width": null, 820 | "min_height": null, 821 | "min_width": null, 822 | "object_fit": null, 823 | "object_position": null, 824 | "order": null, 825 | "overflow": null, 826 | "overflow_x": null, 827 | "overflow_y": null, 828 | "padding": null, 829 | "right": null, 830 | "top": null, 831 | "visibility": null, 832 | "width": null 833 | } 834 | }, 835 | "a9ccabebd2c6428594120631518c505c": { 836 | "model_module": "@jupyter-widgets/controls", 837 | "model_name": "DescriptionStyleModel", 838 | "state": { 839 | "_model_module": "@jupyter-widgets/controls", 840 | "_model_module_version": "1.5.0", 841 | "_model_name": "DescriptionStyleModel", 842 | "_view_count": null, 843 | "_view_module": "@jupyter-widgets/base", 844 | "_view_module_version": "1.2.0", 845 | "_view_name": "StyleView", 846 | "description_width": "" 847 | } 848 | }, 849 | "ab7e6d830cc74c14876a052480eb185a": { 850 | "model_module": "@jupyter-widgets/controls", 851 | "model_name": "HTMLModel", 852 | "state": { 853 | "_dom_classes": [], 854 | "_model_module": "@jupyter-widgets/controls", 855 | "_model_module_version": "1.5.0", 856 | "_model_name": "HTMLModel", 857 | "_view_count": null, 858 | "_view_module": "@jupyter-widgets/controls", 859 | "_view_module_version": "1.5.0", 860 | "_view_name": "HTMLView", 861 | "description": "", 862 | "description_tooltip": null, 863 | "layout": "IPY_MODEL_a739b97fb09848dbae2fb6d0fef09ef4", 864 | "placeholder": "​", 865 | "style": "IPY_MODEL_a9ccabebd2c6428594120631518c505c", 866 | "value": " 9920512/? [00:20<00:00, 1061366.86it/s]" 867 | } 868 | }, 869 | "bcbba5e1db1d498eb4b9ce545ff26d8a": { 870 | "model_module": "@jupyter-widgets/controls", 871 | "model_name": "ProgressStyleModel", 872 | "state": { 873 | "_model_module": "@jupyter-widgets/controls", 874 | "_model_module_version": "1.5.0", 875 | "_model_name": "ProgressStyleModel", 876 | "_view_count": null, 877 | "_view_module": "@jupyter-widgets/base", 878 | "_view_module_version": "1.2.0", 879 | "_view_name": "StyleView", 880 | "bar_color": null, 881 | "description_width": "initial" 882 | } 883 | }, 884 | "bcc108058dc4490986ec987c568b2034": { 885 | "model_module": "@jupyter-widgets/controls", 886 | "model_name": "IntProgressModel", 887 | "state": { 888 | "_dom_classes": [], 889 | "_model_module": "@jupyter-widgets/controls", 890 | "_model_module_version": "1.5.0", 891 | "_model_name": "IntProgressModel", 892 | "_view_count": null, 893 | "_view_module": "@jupyter-widgets/controls", 894 | "_view_module_version": "1.5.0", 895 | "_view_name": "ProgressView", 896 | "bar_style": "info", 897 | "description": "", 898 | "description_tooltip": null, 899 | "layout": "IPY_MODEL_229d85ab3245444c9ad5b57c6ec58688", 900 | "max": 1, 901 | "min": 0, 902 | "orientation": "horizontal", 903 | "style": "IPY_MODEL_3a488dd9262d42ac9b119cd37e2e71c9", 904 | "value": 1 905 | } 906 | }, 907 | "d57756abec8742db9d650c814b474137": { 908 | "model_module": "@jupyter-widgets/base", 909 | "model_name": "LayoutModel", 910 | "state": { 911 | "_model_module": "@jupyter-widgets/base", 912 | "_model_module_version": "1.2.0", 913 | "_model_name": "LayoutModel", 914 | "_view_count": null, 915 | "_view_module": "@jupyter-widgets/base", 916 | "_view_module_version": "1.2.0", 917 | "_view_name": "LayoutView", 918 | "align_content": null, 919 | "align_items": null, 920 | "align_self": null, 921 | "border": null, 922 | "bottom": null, 923 | "display": null, 924 | "flex": null, 925 | "flex_flow": null, 926 | "grid_area": null, 927 | "grid_auto_columns": null, 928 | "grid_auto_flow": null, 929 | "grid_auto_rows": null, 930 | "grid_column": null, 931 | "grid_gap": null, 932 | "grid_row": null, 933 | "grid_template_areas": null, 934 | "grid_template_columns": null, 935 | "grid_template_rows": null, 936 | "height": null, 937 | "justify_content": null, 938 | "justify_items": null, 939 | "left": null, 940 | "margin": null, 941 | "max_height": null, 942 | "max_width": null, 943 | "min_height": null, 944 | "min_width": null, 945 | "object_fit": null, 946 | "object_position": null, 947 | "order": null, 948 | "overflow": null, 949 | "overflow_x": null, 950 | "overflow_y": null, 951 | "padding": null, 952 | "right": null, 953 | "top": null, 954 | "visibility": null, 955 | "width": null 956 | } 957 | }, 958 | "d677eff7d5a4494087a60ff1db4dab92": { 959 | "model_module": "@jupyter-widgets/base", 960 | "model_name": "LayoutModel", 961 | "state": { 962 | "_model_module": "@jupyter-widgets/base", 963 | "_model_module_version": "1.2.0", 964 | "_model_name": "LayoutModel", 965 | "_view_count": null, 966 | "_view_module": "@jupyter-widgets/base", 967 | "_view_module_version": "1.2.0", 968 | "_view_name": "LayoutView", 969 | "align_content": null, 970 | "align_items": null, 971 | "align_self": null, 972 | "border": null, 973 | "bottom": null, 974 | "display": null, 975 | "flex": null, 976 | "flex_flow": null, 977 | "grid_area": null, 978 | "grid_auto_columns": null, 979 | "grid_auto_flow": null, 980 | "grid_auto_rows": null, 981 | "grid_column": null, 982 | "grid_gap": null, 983 | "grid_row": null, 984 | "grid_template_areas": null, 985 | "grid_template_columns": null, 986 | "grid_template_rows": null, 987 | "height": null, 988 | "justify_content": null, 989 | "justify_items": null, 990 | "left": null, 991 | "margin": null, 992 | "max_height": null, 993 | "max_width": null, 994 | "min_height": null, 995 | "min_width": null, 996 | "object_fit": null, 997 | "object_position": null, 998 | "order": null, 999 | "overflow": null, 1000 | "overflow_x": null, 1001 | "overflow_y": null, 1002 | "padding": null, 1003 | "right": null, 1004 | "top": null, 1005 | "visibility": null, 1006 | "width": null 1007 | } 1008 | }, 1009 | "df0848e4bca84d95b178cbe955417bc4": { 1010 | "model_module": "@jupyter-widgets/controls", 1011 | "model_name": "ProgressStyleModel", 1012 | "state": { 1013 | "_model_module": "@jupyter-widgets/controls", 1014 | "_model_module_version": "1.5.0", 1015 | "_model_name": "ProgressStyleModel", 1016 | "_view_count": null, 1017 | "_view_module": "@jupyter-widgets/base", 1018 | "_view_module_version": "1.2.0", 1019 | "_view_name": "StyleView", 1020 | "bar_color": null, 1021 | "description_width": "initial" 1022 | } 1023 | }, 1024 | "e11d5108f5a74dd6b7143172c693dc6a": { 1025 | "model_module": "@jupyter-widgets/controls", 1026 | "model_name": "HBoxModel", 1027 | "state": { 1028 | "_dom_classes": [], 1029 | "_model_module": "@jupyter-widgets/controls", 1030 | "_model_module_version": "1.5.0", 1031 | "_model_name": "HBoxModel", 1032 | "_view_count": null, 1033 | "_view_module": "@jupyter-widgets/controls", 1034 | "_view_module_version": "1.5.0", 1035 | "_view_name": "HBoxView", 1036 | "box_style": "", 1037 | "children": [ 1038 | "IPY_MODEL_2de5166b2a40479fb69c08e7fda49594", 1039 | "IPY_MODEL_655321e930014d03a986ab862d9c01a1" 1040 | ], 1041 | "layout": "IPY_MODEL_590b36901152442088a4c5e05e781b3d" 1042 | } 1043 | }, 1044 | "e488365503444d9c8313fd2017d05f9e": { 1045 | "model_module": "@jupyter-widgets/controls", 1046 | "model_name": "HBoxModel", 1047 | "state": { 1048 | "_dom_classes": [], 1049 | "_model_module": "@jupyter-widgets/controls", 1050 | "_model_module_version": "1.5.0", 1051 | "_model_name": "HBoxModel", 1052 | "_view_count": null, 1053 | "_view_module": "@jupyter-widgets/controls", 1054 | "_view_module_version": "1.5.0", 1055 | "_view_name": "HBoxView", 1056 | "box_style": "", 1057 | "children": [ 1058 | "IPY_MODEL_200b60fab8514008a571603c6a2519f8", 1059 | "IPY_MODEL_34a38f1355e54a24ad0757e91e016ff4" 1060 | ], 1061 | "layout": "IPY_MODEL_d677eff7d5a4494087a60ff1db4dab92" 1062 | } 1063 | }, 1064 | "e92daa0c7043415b8463dffd654e93e5": { 1065 | "model_module": "@jupyter-widgets/controls", 1066 | "model_name": "HTMLModel", 1067 | "state": { 1068 | "_dom_classes": [], 1069 | "_model_module": "@jupyter-widgets/controls", 1070 | "_model_module_version": "1.5.0", 1071 | "_model_name": "HTMLModel", 1072 | "_view_count": null, 1073 | "_view_module": "@jupyter-widgets/controls", 1074 | "_view_module_version": "1.5.0", 1075 | "_view_name": "HTMLView", 1076 | "description": "", 1077 | "description_tooltip": null, 1078 | "layout": "IPY_MODEL_70658583c62a40c28345d43ceb70810b", 1079 | "placeholder": "​", 1080 | "style": "IPY_MODEL_1a38c10f1d0e4f20a4958d453757f9e7", 1081 | "value": " 32768/? [00:01<00:00, 28639.26it/s]" 1082 | } 1083 | }, 1084 | "ee8b4317d860412fa60bca6d1d0c935a": { 1085 | "model_module": "@jupyter-widgets/base", 1086 | "model_name": "LayoutModel", 1087 | "state": { 1088 | "_model_module": "@jupyter-widgets/base", 1089 | "_model_module_version": "1.2.0", 1090 | "_model_name": "LayoutModel", 1091 | "_view_count": null, 1092 | "_view_module": "@jupyter-widgets/base", 1093 | "_view_module_version": "1.2.0", 1094 | "_view_name": "LayoutView", 1095 | "align_content": null, 1096 | "align_items": null, 1097 | "align_self": null, 1098 | "border": null, 1099 | "bottom": null, 1100 | "display": null, 1101 | "flex": null, 1102 | "flex_flow": null, 1103 | "grid_area": null, 1104 | "grid_auto_columns": null, 1105 | "grid_auto_flow": null, 1106 | "grid_auto_rows": null, 1107 | "grid_column": null, 1108 | "grid_gap": null, 1109 | "grid_row": null, 1110 | "grid_template_areas": null, 1111 | "grid_template_columns": null, 1112 | "grid_template_rows": null, 1113 | "height": null, 1114 | "justify_content": null, 1115 | "justify_items": null, 1116 | "left": null, 1117 | "margin": null, 1118 | "max_height": null, 1119 | "max_width": null, 1120 | "min_height": null, 1121 | "min_width": null, 1122 | "object_fit": null, 1123 | "object_position": null, 1124 | "order": null, 1125 | "overflow": null, 1126 | "overflow_x": null, 1127 | "overflow_y": null, 1128 | "padding": null, 1129 | "right": null, 1130 | "top": null, 1131 | "visibility": null, 1132 | "width": null 1133 | } 1134 | } 1135 | } 1136 | } 1137 | }, 1138 | "nbformat": 4, 1139 | "nbformat_minor": 4 1140 | } 1141 | -------------------------------------------------------------------------------- /1a_logistic_regression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": { 7 | "colab": {}, 8 | "colab_type": "code", 9 | "id": "hqnl0AKVXIA4" 10 | }, 11 | "outputs": [], 12 | "source": [ 13 | "import matplotlib.pyplot\n", 14 | "import numpy.random\n", 15 | "import torch.utils.data\n", 16 | "import torchvision\n", 17 | "from torch import Tensor\n", 18 | "from torch.nn import Module\n", 19 | "from torch.utils.data import DataLoader\n", 20 | "from torchvision.datasets import MNIST\n", 21 | "from torchvision.transforms import ToTensor" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": { 27 | "colab_type": "text", 28 | "id": "mCt-frpFH8oN" 29 | }, 30 | "source": [ 31 | "# Tutorial 1a. Logistic Regression\n", 32 | "\n", 33 | "In the first tutorial, we are going to train a logistic regressor on the MNIST dataset of handwritten digits. Next, we will turn this logistic regressor into a non-linear convolutional network." 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "metadata": { 40 | "colab": {}, 41 | "colab_type": "code", 42 | "id": "dzfEE578uSNp" 43 | }, 44 | "outputs": [], 45 | "source": [ 46 | "if torch.cuda.is_available():\n", 47 | " DEVICE = \"cuda:0\"\n", 48 | "else:\n", 49 | " DEVICE = \"cpu\"" 50 | ] 51 | }, 52 | { 53 | "cell_type": "markdown", 54 | "metadata": {}, 55 | "source": [ 56 | "## Loading Datasets\n", 57 | "\n", 58 | "The following code will load the MNIST dataset. Run it and inspect some of the images and their labels to confirm they are correct." 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": null, 64 | "metadata": { 65 | "colab": { 66 | "base_uri": "https://localhost:8080/", 67 | "height": 349, 68 | "referenced_widgets": [ 69 | "59a8e0ee48e14c9691224d013d6c46fe", 70 | "ee8b4317d860412fa60bca6d1d0c935a", 71 | "bcc108058dc4490986ec987c568b2034", 72 | "ab7e6d830cc74c14876a052480eb185a", 73 | "3a488dd9262d42ac9b119cd37e2e71c9", 74 | "229d85ab3245444c9ad5b57c6ec58688", 75 | "a9ccabebd2c6428594120631518c505c", 76 | "a739b97fb09848dbae2fb6d0fef09ef4", 77 | "41f12163cbd84febb7fade8126ed7d5e", 78 | "48dc62d211994fb1b3b323da3c8c8a26", 79 | "8f7373518a7840b38d00de7608a9e218", 80 | "e92daa0c7043415b8463dffd654e93e5", 81 | "df0848e4bca84d95b178cbe955417bc4", 82 | "10c91bce2e8b42c587557b0a649a79fe", 83 | "1a38c10f1d0e4f20a4958d453757f9e7", 84 | "70658583c62a40c28345d43ceb70810b", 85 | "e488365503444d9c8313fd2017d05f9e", 86 | "d677eff7d5a4494087a60ff1db4dab92", 87 | "200b60fab8514008a571603c6a2519f8", 88 | "34a38f1355e54a24ad0757e91e016ff4", 89 | "8f98aca9cfae490dab5dccf54b939ce0", 90 | "04f7f68d4f784cf19f942ee29c24ecc7", 91 | "99bb01a7fb144b149460669c65b22654", 92 | "466dc3fe5f2c447ebc9f7259e886e606", 93 | "e11d5108f5a74dd6b7143172c693dc6a", 94 | "590b36901152442088a4c5e05e781b3d", 95 | "2de5166b2a40479fb69c08e7fda49594", 96 | "655321e930014d03a986ab862d9c01a1", 97 | "bcbba5e1db1d498eb4b9ce545ff26d8a", 98 | "d57756abec8742db9d650c814b474137", 99 | "3c129350f337408f920706b3836528b9", 100 | "78f9f8dd4275405b96194f2507dae175" 101 | ] 102 | }, 103 | "colab_type": "code", 104 | "executionInfo": { 105 | "elapsed": 7621, 106 | "status": "ok", 107 | "timestamp": 1586535092902, 108 | "user": { 109 | "displayName": "Anton Bakhtin", 110 | "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gjq-QU8evTujx-PnMoIketNeW4yB_kgdtnRJ03IxnY=s64", 111 | "userId": "12331399045734156147" 112 | }, 113 | "user_tz": 240 114 | }, 115 | "id": "GX_ky5qC--uv", 116 | "outputId": "97a811cf-59f5-4bb5-be96-03e19190cd4b" 117 | }, 118 | "outputs": [], 119 | "source": [ 120 | "train_dataloader = DataLoader(\n", 121 | " dataset=MNIST(\n", 122 | " root=\"/tmp/mnist\",\n", 123 | " train=True,\n", 124 | " transform=ToTensor(),\n", 125 | " download=True,\n", 126 | " ),\n", 127 | " batch_size=64,\n", 128 | " shuffle=True,\n", 129 | ")\n", 130 | "\n", 131 | "test_dataloader = DataLoader(\n", 132 | " dataset=MNIST(\n", 133 | " root=\"/tmp/mnist\",\n", 134 | " train=False,\n", 135 | " transform=ToTensor(),\n", 136 | " download=True,\n", 137 | " ),\n", 138 | " batch_size=64,\n", 139 | " shuffle=False,\n", 140 | ")" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": { 147 | "colab": { 148 | "base_uri": "https://localhost:8080/", 149 | "height": 62 150 | }, 151 | "colab_type": "code", 152 | "executionInfo": { 153 | "elapsed": 7559, 154 | "status": "ok", 155 | "timestamp": 1586535092902, 156 | "user": { 157 | "displayName": "Anton Bakhtin", 158 | "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gjq-QU8evTujx-PnMoIketNeW4yB_kgdtnRJ03IxnY=s64", 159 | "userId": "12331399045734156147" 160 | }, 161 | "user_tz": 240 162 | }, 163 | "id": "E8dy7-kBHwco", 164 | "outputId": "66a535b5-83bc-48e0-f94a-1a79de8c3e66" 165 | }, 166 | "outputs": [], 167 | "source": [ 168 | "image, target = [*test_dataloader][0]\n", 169 | "\n", 170 | "random_index = numpy.random.randint(0, 64)\n", 171 | "\n", 172 | "image, target = image[random_index, 0], target[random_index]\n", 173 | "\n", 174 | "matplotlib.pyplot.imshow(\n", 175 | " image, \n", 176 | " cmap=\"gray\", \n", 177 | " interpolation=\"nearest\",\n", 178 | ")\n", 179 | "\n", 180 | "matplotlib.pyplot.title(f\"target = {target}\")\n", 181 | "matplotlib.pyplot.axis(\"off\")" 182 | ] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": { 187 | "colab_type": "text", 188 | "id": "aKbXgidhWr6L" 189 | }, 190 | "source": [ 191 | "Next, we create a PyTorch dataloader for the MNIST dataset." 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "metadata": { 197 | "colab_type": "text", 198 | "id": "liekFZzvYX9E" 199 | }, 200 | "source": [ 201 | "Next, implement a logistic regression model in PyTorch. Note that a logistic regressor uses a linear transformation of the input.\n" 202 | ] 203 | }, 204 | { 205 | "cell_type": "markdown", 206 | "metadata": {}, 207 | "source": [ 208 | "## Exercise 1. Logistic Regression Module" 209 | ] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "execution_count": null, 214 | "metadata": { 215 | "colab": {}, 216 | "colab_type": "code", 217 | "id": "zEw5YorSYkWF" 218 | }, 219 | "outputs": [], 220 | "source": [ 221 | "# class LogisticRegression(Module):\n", 222 | "# def __init__(self, input_size: int, num_classes: int):\n", 223 | "# super().__init__()\n", 224 | "#y = ... \n", 225 | "# ###########################################################################\n", 226 | "# # TODO: Instantiate the layer here. #\n", 227 | "# ###########################################################################\n", 228 | "# \n", 229 | "# def forward(self, x: Tensor) -> Tensor:\n", 230 | "# ###########################################################################\n", 231 | "# # TODO: Apply the layer to the input. #\n", 232 | "# ###########################################################################\n", 233 | "# pass" 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": null, 239 | "metadata": {}, 240 | "outputs": [], 241 | "source": [ 242 | "module = LogisticRegression(28 * 28, 10)\n", 243 | "\n", 244 | "module = module.to(device=DEVICE)" 245 | ] 246 | }, 247 | { 248 | "cell_type": "code", 249 | "execution_count": null, 250 | "metadata": {}, 251 | "outputs": [], 252 | "source": [ 253 | "###########################################################################\n", 254 | "# TODO: Create criterion and optimizer here. #\n", 255 | "###########################################################################\n", 256 | "\n", 257 | "criterion = \n", 258 | "\n", 259 | "optimizer = " 260 | ] 261 | }, 262 | { 263 | "cell_type": "markdown", 264 | "metadata": {}, 265 | "source": [ 266 | "## Training" 267 | ] 268 | }, 269 | { 270 | "cell_type": "markdown", 271 | "metadata": {}, 272 | "source": [ 273 | "We will use the following generic training loop for a PyTorch model." 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": null, 279 | "metadata": { 280 | "colab": {}, 281 | "colab_type": "code", 282 | "id": "qVyEKl3OZLJw" 283 | }, 284 | "outputs": [], 285 | "source": [ 286 | "# Train the model. If everything is correct, the loss should go below 0.45.\n", 287 | "\n", 288 | "EPOCHS = 5\n", 289 | "\n", 290 | "# Exponential moving average of the loss:\n", 291 | "ema = None\n", 292 | "\n", 293 | "for epoch in range(EPOCHS):\n", 294 | " for batch_index, (train_images, train_targets) in enumerate(train_dataloader):\n", 295 | " train_images = train_images.view(-1, 28 * 28).requires_grad_().to(device=DEVICE)\n", 296 | "\n", 297 | " train_targets = train_targets.to(device=DEVICE)\n", 298 | "\n", 299 | " # Clear gradients w.r.t. parameters\n", 300 | " optimizer.zero_grad()\n", 301 | "\n", 302 | " # Forward pass to get output/logits\n", 303 | " outputs = module(train_images)\n", 304 | "\n", 305 | " # Calculate Loss: softmax --> cross entropy loss\n", 306 | " loss = criterion(outputs, train_targets)\n", 307 | "\n", 308 | " # Getting gradients w.r.t. parameters\n", 309 | " loss.backward()\n", 310 | "\n", 311 | " # Updates parameters:\n", 312 | " optimizer.step()\n", 313 | "\n", 314 | " # NOTE: It is important to call .item() on the loss before summing.\n", 315 | " if ema is None:\n", 316 | " ema = loss.item()\n", 317 | " else:\n", 318 | " ema += (loss.item() - ema) * 0.01\n", 319 | "\n", 320 | " if batch_index % 500 == 0:\n", 321 | " print(\n", 322 | " \"Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}\".format(\n", 323 | " epoch,\n", 324 | " batch_index * len(train_images),\n", 325 | " len(train_dataloader.dataset),\n", 326 | " 100.0 * batch_index / len(train_dataloader),\n", 327 | " ema,\n", 328 | " ),\n", 329 | " )" 330 | ] 331 | }, 332 | { 333 | "cell_type": "markdown", 334 | "metadata": { 335 | "colab_type": "text", 336 | "id": "oN8qTuGkZ-3X" 337 | }, 338 | "source": [ 339 | "**Question:** For the model you are currently using, is there any difference between using the model in `train` mode or using it in `eval` mode? \n", 340 | "\n", 341 | "Create an SGD optimizer and us it to train the logistic regressor on the MNIST training data for a few epochs. What loss function do you need to use?" 342 | ] 343 | }, 344 | { 345 | "cell_type": "markdown", 346 | "metadata": { 347 | "colab_type": "text", 348 | "id": "x0QN7Mhiar7d" 349 | }, 350 | "source": [ 351 | "### Embeddings \n", 352 | "\n", 353 | "Visualize the weights of the trained model. What do you see? Why?" 354 | ] 355 | }, 356 | { 357 | "cell_type": "code", 358 | "execution_count": null, 359 | "metadata": {}, 360 | "outputs": [], 361 | "source": [ 362 | "assert module.y.weight.shape == (10, 28 * 28)" 363 | ] 364 | }, 365 | { 366 | "cell_type": "code", 367 | "execution_count": null, 368 | "metadata": { 369 | "colab": { 370 | "base_uri": "https://localhost:8080/", 371 | "height": 167 372 | }, 373 | "colab_type": "code", 374 | "executionInfo": { 375 | "elapsed": 37796, 376 | "status": "ok", 377 | "timestamp": 1586535123165, 378 | "user": { 379 | "displayName": "Anton Bakhtin", 380 | "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gjq-QU8evTujx-PnMoIketNeW4yB_kgdtnRJ03IxnY=s64", 381 | "userId": "12331399045734156147" 382 | }, 383 | "user_tz": 240 384 | }, 385 | "id": "gxletH44a4MX", 386 | "outputId": "1ba6c589-bcbb-4b86-a9ab-7fd4bc5bdfa1" 387 | }, 388 | "outputs": [], 389 | "source": [ 390 | "matplotlib.pyplot.imshow(\n", 391 | " numpy.transpose(\n", 392 | " torchvision.utils.make_grid(\n", 393 | " module.y.weight.view(10, 1, 28, 28).cpu(),\n", 394 | " normalize=True,\n", 395 | " nrow=5,\n", 396 | " ), \n", 397 | " (1, 2, 0),\n", 398 | " ), \n", 399 | " interpolation=\"nearest\",\n", 400 | ")\n", 401 | "\n", 402 | "matplotlib.pyplot.grid(False)\n", 403 | "matplotlib.pyplot.gca().axis(\"off\")" 404 | ] 405 | }, 406 | { 407 | "cell_type": "markdown", 408 | "metadata": {}, 409 | "source": [ 410 | "## Evaluation" 411 | ] 412 | }, 413 | { 414 | "cell_type": "markdown", 415 | "metadata": { 416 | "colab_type": "text", 417 | "id": "Lxf5NxQ6a5cT" 418 | }, 419 | "source": [ 420 | "Use the following function to measure the test accuracy of your trained model." 421 | ] 422 | }, 423 | { 424 | "cell_type": "code", 425 | "execution_count": null, 426 | "metadata": { 427 | "colab": {}, 428 | "colab_type": "code", 429 | "id": "8UYWbqZYa9Qr" 430 | }, 431 | "outputs": [], 432 | "source": [ 433 | "correct_predictions = 0\n", 434 | "predictions = 0\n", 435 | "\n", 436 | "# Iterate through test dataset\n", 437 | "for test_images, test_targets in test_dataloader:\n", 438 | " test_images = test_images.view(-1, 28 * 28).to(DEVICE)\n", 439 | "\n", 440 | " # Forward pass only to get logits/output\n", 441 | " outputs = module(test_images)\n", 442 | "\n", 443 | " # Get predictions from the maximum value\n", 444 | " _, predicted = torch.max(outputs.data, 1)\n", 445 | "\n", 446 | " predictions += test_targets.size(0)\n", 447 | "\n", 448 | " if torch.cuda.is_available():\n", 449 | " correct_predictions += (predicted.cpu() == test_targets.cpu()).sum()\n", 450 | " else:\n", 451 | " correct_predictions += (predicted == test_targets).sum()\n", 452 | "\n", 453 | "correct_predictions.item() / predictions" 454 | ] 455 | } 456 | ], 457 | "metadata": { 458 | "accelerator": "GPU", 459 | "colab": { 460 | "collapsed_sections": [], 461 | "name": "convnet_tutorial1_filled.ipynb", 462 | "provenance": [ 463 | { 464 | "file_id": "1Eq_v8eTk4XAxFXhjL6rk26gwfwvpZYBn", 465 | "timestamp": 1553454632964 466 | } 467 | ] 468 | }, 469 | "kernelspec": { 470 | "display_name": "convnet_tutorials", 471 | "language": "python", 472 | "name": "convnet_tutorials" 473 | }, 474 | "language_info": { 475 | "codemirror_mode": { 476 | "name": "ipython", 477 | "version": 3 478 | }, 479 | "file_extension": ".py", 480 | "mimetype": "text/x-python", 481 | "name": "python", 482 | "nbconvert_exporter": "python", 483 | "pygments_lexer": "ipython3", 484 | "version": "3.9.10" 485 | }, 486 | "widgets": { 487 | "application/vnd.jupyter.widget-state+json": { 488 | "04f7f68d4f784cf19f942ee29c24ecc7": { 489 | "model_module": "@jupyter-widgets/base", 490 | "model_name": "LayoutModel", 491 | "state": { 492 | "_model_module": "@jupyter-widgets/base", 493 | "_model_module_version": "1.2.0", 494 | "_model_name": "LayoutModel", 495 | "_view_count": null, 496 | "_view_module": "@jupyter-widgets/base", 497 | "_view_module_version": "1.2.0", 498 | "_view_name": "LayoutView", 499 | "align_content": null, 500 | "align_items": null, 501 | "align_self": null, 502 | "border": null, 503 | "bottom": null, 504 | "display": null, 505 | "flex": null, 506 | "flex_flow": null, 507 | "grid_area": null, 508 | "grid_auto_columns": null, 509 | "grid_auto_flow": null, 510 | "grid_auto_rows": null, 511 | "grid_column": null, 512 | "grid_gap": null, 513 | "grid_row": null, 514 | "grid_template_areas": null, 515 | "grid_template_columns": null, 516 | "grid_template_rows": null, 517 | "height": null, 518 | "justify_content": null, 519 | "justify_items": null, 520 | "left": null, 521 | "margin": null, 522 | "max_height": null, 523 | "max_width": null, 524 | "min_height": null, 525 | "min_width": null, 526 | "object_fit": null, 527 | "object_position": null, 528 | "order": null, 529 | "overflow": null, 530 | "overflow_x": null, 531 | "overflow_y": null, 532 | "padding": null, 533 | "right": null, 534 | "top": null, 535 | "visibility": null, 536 | "width": null 537 | } 538 | }, 539 | "10c91bce2e8b42c587557b0a649a79fe": { 540 | "model_module": "@jupyter-widgets/base", 541 | "model_name": "LayoutModel", 542 | "state": { 543 | "_model_module": "@jupyter-widgets/base", 544 | "_model_module_version": "1.2.0", 545 | "_model_name": "LayoutModel", 546 | "_view_count": null, 547 | "_view_module": "@jupyter-widgets/base", 548 | "_view_module_version": "1.2.0", 549 | "_view_name": "LayoutView", 550 | "align_content": null, 551 | "align_items": null, 552 | "align_self": null, 553 | "border": null, 554 | "bottom": null, 555 | "display": null, 556 | "flex": null, 557 | "flex_flow": null, 558 | "grid_area": null, 559 | "grid_auto_columns": null, 560 | "grid_auto_flow": null, 561 | "grid_auto_rows": null, 562 | "grid_column": null, 563 | "grid_gap": null, 564 | "grid_row": null, 565 | "grid_template_areas": null, 566 | "grid_template_columns": null, 567 | "grid_template_rows": null, 568 | "height": null, 569 | "justify_content": null, 570 | "justify_items": null, 571 | "left": null, 572 | "margin": null, 573 | "max_height": null, 574 | "max_width": null, 575 | "min_height": null, 576 | "min_width": null, 577 | "object_fit": null, 578 | "object_position": null, 579 | "order": null, 580 | "overflow": null, 581 | "overflow_x": null, 582 | "overflow_y": null, 583 | "padding": null, 584 | "right": null, 585 | "top": null, 586 | "visibility": null, 587 | "width": null 588 | } 589 | }, 590 | "1a38c10f1d0e4f20a4958d453757f9e7": { 591 | "model_module": "@jupyter-widgets/controls", 592 | "model_name": "DescriptionStyleModel", 593 | "state": { 594 | "_model_module": "@jupyter-widgets/controls", 595 | "_model_module_version": "1.5.0", 596 | "_model_name": "DescriptionStyleModel", 597 | "_view_count": null, 598 | "_view_module": "@jupyter-widgets/base", 599 | "_view_module_version": "1.2.0", 600 | "_view_name": "StyleView", 601 | "description_width": "" 602 | } 603 | }, 604 | "200b60fab8514008a571603c6a2519f8": { 605 | "model_module": "@jupyter-widgets/controls", 606 | "model_name": "IntProgressModel", 607 | "state": { 608 | "_dom_classes": [], 609 | "_model_module": "@jupyter-widgets/controls", 610 | "_model_module_version": "1.5.0", 611 | "_model_name": "IntProgressModel", 612 | "_view_count": null, 613 | "_view_module": "@jupyter-widgets/controls", 614 | "_view_module_version": "1.5.0", 615 | "_view_name": "ProgressView", 616 | "bar_style": "success", 617 | "description": "", 618 | "description_tooltip": null, 619 | "layout": "IPY_MODEL_04f7f68d4f784cf19f942ee29c24ecc7", 620 | "max": 1, 621 | "min": 0, 622 | "orientation": "horizontal", 623 | "style": "IPY_MODEL_8f98aca9cfae490dab5dccf54b939ce0", 624 | "value": 1 625 | } 626 | }, 627 | "229d85ab3245444c9ad5b57c6ec58688": { 628 | "model_module": "@jupyter-widgets/base", 629 | "model_name": "LayoutModel", 630 | "state": { 631 | "_model_module": "@jupyter-widgets/base", 632 | "_model_module_version": "1.2.0", 633 | "_model_name": "LayoutModel", 634 | "_view_count": null, 635 | "_view_module": "@jupyter-widgets/base", 636 | "_view_module_version": "1.2.0", 637 | "_view_name": "LayoutView", 638 | "align_content": null, 639 | "align_items": null, 640 | "align_self": null, 641 | "border": null, 642 | "bottom": null, 643 | "display": null, 644 | "flex": null, 645 | "flex_flow": null, 646 | "grid_area": null, 647 | "grid_auto_columns": null, 648 | "grid_auto_flow": null, 649 | "grid_auto_rows": null, 650 | "grid_column": null, 651 | "grid_gap": null, 652 | "grid_row": null, 653 | "grid_template_areas": null, 654 | "grid_template_columns": null, 655 | "grid_template_rows": null, 656 | "height": null, 657 | "justify_content": null, 658 | "justify_items": null, 659 | "left": null, 660 | "margin": null, 661 | "max_height": null, 662 | "max_width": null, 663 | "min_height": null, 664 | "min_width": null, 665 | "object_fit": null, 666 | "object_position": null, 667 | "order": null, 668 | "overflow": null, 669 | "overflow_x": null, 670 | "overflow_y": null, 671 | "padding": null, 672 | "right": null, 673 | "top": null, 674 | "visibility": null, 675 | "width": null 676 | } 677 | }, 678 | "2de5166b2a40479fb69c08e7fda49594": { 679 | "model_module": "@jupyter-widgets/controls", 680 | "model_name": "IntProgressModel", 681 | "state": { 682 | "_dom_classes": [], 683 | "_model_module": "@jupyter-widgets/controls", 684 | "_model_module_version": "1.5.0", 685 | "_model_name": "IntProgressModel", 686 | "_view_count": null, 687 | "_view_module": "@jupyter-widgets/controls", 688 | "_view_module_version": "1.5.0", 689 | "_view_name": "ProgressView", 690 | "bar_style": "info", 691 | "description": " 0%", 692 | "description_tooltip": null, 693 | "layout": "IPY_MODEL_d57756abec8742db9d650c814b474137", 694 | "max": 1, 695 | "min": 0, 696 | "orientation": "horizontal", 697 | "style": "IPY_MODEL_bcbba5e1db1d498eb4b9ce545ff26d8a", 698 | "value": 0 699 | } 700 | }, 701 | "34a38f1355e54a24ad0757e91e016ff4": { 702 | "model_module": "@jupyter-widgets/controls", 703 | "model_name": "HTMLModel", 704 | "state": { 705 | "_dom_classes": [], 706 | "_model_module": "@jupyter-widgets/controls", 707 | "_model_module_version": "1.5.0", 708 | "_model_name": "HTMLModel", 709 | "_view_count": null, 710 | "_view_module": "@jupyter-widgets/controls", 711 | "_view_module_version": "1.5.0", 712 | "_view_name": "HTMLView", 713 | "description": "", 714 | "description_tooltip": null, 715 | "layout": "IPY_MODEL_466dc3fe5f2c447ebc9f7259e886e606", 716 | "placeholder": "​", 717 | "style": "IPY_MODEL_99bb01a7fb144b149460669c65b22654", 718 | "value": " 1654784/? [00:00<00:00, 1924711.94it/s]" 719 | } 720 | }, 721 | "3a488dd9262d42ac9b119cd37e2e71c9": { 722 | "model_module": "@jupyter-widgets/controls", 723 | "model_name": "ProgressStyleModel", 724 | "state": { 725 | "_model_module": "@jupyter-widgets/controls", 726 | "_model_module_version": "1.5.0", 727 | "_model_name": "ProgressStyleModel", 728 | "_view_count": null, 729 | "_view_module": "@jupyter-widgets/base", 730 | "_view_module_version": "1.2.0", 731 | "_view_name": "StyleView", 732 | "bar_color": null, 733 | "description_width": "initial" 734 | } 735 | }, 736 | "3c129350f337408f920706b3836528b9": { 737 | "model_module": "@jupyter-widgets/controls", 738 | "model_name": "DescriptionStyleModel", 739 | "state": { 740 | "_model_module": "@jupyter-widgets/controls", 741 | "_model_module_version": "1.5.0", 742 | "_model_name": "DescriptionStyleModel", 743 | "_view_count": null, 744 | "_view_module": "@jupyter-widgets/base", 745 | "_view_module_version": "1.2.0", 746 | "_view_name": "StyleView", 747 | "description_width": "" 748 | } 749 | }, 750 | "41f12163cbd84febb7fade8126ed7d5e": { 751 | "model_module": "@jupyter-widgets/controls", 752 | "model_name": "HBoxModel", 753 | "state": { 754 | "_dom_classes": [], 755 | "_model_module": "@jupyter-widgets/controls", 756 | "_model_module_version": "1.5.0", 757 | "_model_name": "HBoxModel", 758 | "_view_count": null, 759 | "_view_module": "@jupyter-widgets/controls", 760 | "_view_module_version": "1.5.0", 761 | "_view_name": "HBoxView", 762 | "box_style": "", 763 | "children": [ 764 | "IPY_MODEL_8f7373518a7840b38d00de7608a9e218", 765 | "IPY_MODEL_e92daa0c7043415b8463dffd654e93e5" 766 | ], 767 | "layout": "IPY_MODEL_48dc62d211994fb1b3b323da3c8c8a26" 768 | } 769 | }, 770 | "466dc3fe5f2c447ebc9f7259e886e606": { 771 | "model_module": "@jupyter-widgets/base", 772 | "model_name": "LayoutModel", 773 | "state": { 774 | "_model_module": "@jupyter-widgets/base", 775 | "_model_module_version": "1.2.0", 776 | "_model_name": "LayoutModel", 777 | "_view_count": null, 778 | "_view_module": "@jupyter-widgets/base", 779 | "_view_module_version": "1.2.0", 780 | "_view_name": "LayoutView", 781 | "align_content": null, 782 | "align_items": null, 783 | "align_self": null, 784 | "border": null, 785 | "bottom": null, 786 | "display": null, 787 | "flex": null, 788 | "flex_flow": null, 789 | "grid_area": null, 790 | "grid_auto_columns": null, 791 | "grid_auto_flow": null, 792 | "grid_auto_rows": null, 793 | "grid_column": null, 794 | "grid_gap": null, 795 | "grid_row": null, 796 | "grid_template_areas": null, 797 | "grid_template_columns": null, 798 | "grid_template_rows": null, 799 | "height": null, 800 | "justify_content": null, 801 | "justify_items": null, 802 | "left": null, 803 | "margin": null, 804 | "max_height": null, 805 | "max_width": null, 806 | "min_height": null, 807 | "min_width": null, 808 | "object_fit": null, 809 | "object_position": null, 810 | "order": null, 811 | "overflow": null, 812 | "overflow_x": null, 813 | "overflow_y": null, 814 | "padding": null, 815 | "right": null, 816 | "top": null, 817 | "visibility": null, 818 | "width": null 819 | } 820 | }, 821 | "48dc62d211994fb1b3b323da3c8c8a26": { 822 | "model_module": "@jupyter-widgets/base", 823 | "model_name": "LayoutModel", 824 | "state": { 825 | "_model_module": "@jupyter-widgets/base", 826 | "_model_module_version": "1.2.0", 827 | "_model_name": "LayoutModel", 828 | "_view_count": null, 829 | "_view_module": "@jupyter-widgets/base", 830 | "_view_module_version": "1.2.0", 831 | "_view_name": "LayoutView", 832 | "align_content": null, 833 | "align_items": null, 834 | "align_self": null, 835 | "border": null, 836 | "bottom": null, 837 | "display": null, 838 | "flex": null, 839 | "flex_flow": null, 840 | "grid_area": null, 841 | "grid_auto_columns": null, 842 | "grid_auto_flow": null, 843 | "grid_auto_rows": null, 844 | "grid_column": null, 845 | "grid_gap": null, 846 | "grid_row": null, 847 | "grid_template_areas": null, 848 | "grid_template_columns": null, 849 | "grid_template_rows": null, 850 | "height": null, 851 | "justify_content": null, 852 | "justify_items": null, 853 | "left": null, 854 | "margin": null, 855 | "max_height": null, 856 | "max_width": null, 857 | "min_height": null, 858 | "min_width": null, 859 | "object_fit": null, 860 | "object_position": null, 861 | "order": null, 862 | "overflow": null, 863 | "overflow_x": null, 864 | "overflow_y": null, 865 | "padding": null, 866 | "right": null, 867 | "top": null, 868 | "visibility": null, 869 | "width": null 870 | } 871 | }, 872 | "590b36901152442088a4c5e05e781b3d": { 873 | "model_module": "@jupyter-widgets/base", 874 | "model_name": "LayoutModel", 875 | "state": { 876 | "_model_module": "@jupyter-widgets/base", 877 | "_model_module_version": "1.2.0", 878 | "_model_name": "LayoutModel", 879 | "_view_count": null, 880 | "_view_module": "@jupyter-widgets/base", 881 | "_view_module_version": "1.2.0", 882 | "_view_name": "LayoutView", 883 | "align_content": null, 884 | "align_items": null, 885 | "align_self": null, 886 | "border": null, 887 | "bottom": null, 888 | "display": null, 889 | "flex": null, 890 | "flex_flow": null, 891 | "grid_area": null, 892 | "grid_auto_columns": null, 893 | "grid_auto_flow": null, 894 | "grid_auto_rows": null, 895 | "grid_column": null, 896 | "grid_gap": null, 897 | "grid_row": null, 898 | "grid_template_areas": null, 899 | "grid_template_columns": null, 900 | "grid_template_rows": null, 901 | "height": null, 902 | "justify_content": null, 903 | "justify_items": null, 904 | "left": null, 905 | "margin": null, 906 | "max_height": null, 907 | "max_width": null, 908 | "min_height": null, 909 | "min_width": null, 910 | "object_fit": null, 911 | "object_position": null, 912 | "order": null, 913 | "overflow": null, 914 | "overflow_x": null, 915 | "overflow_y": null, 916 | "padding": null, 917 | "right": null, 918 | "top": null, 919 | "visibility": null, 920 | "width": null 921 | } 922 | }, 923 | "59a8e0ee48e14c9691224d013d6c46fe": { 924 | "model_module": "@jupyter-widgets/controls", 925 | "model_name": "HBoxModel", 926 | "state": { 927 | "_dom_classes": [], 928 | "_model_module": "@jupyter-widgets/controls", 929 | "_model_module_version": "1.5.0", 930 | "_model_name": "HBoxModel", 931 | "_view_count": null, 932 | "_view_module": "@jupyter-widgets/controls", 933 | "_view_module_version": "1.5.0", 934 | "_view_name": "HBoxView", 935 | "box_style": "", 936 | "children": [ 937 | "IPY_MODEL_bcc108058dc4490986ec987c568b2034", 938 | "IPY_MODEL_ab7e6d830cc74c14876a052480eb185a" 939 | ], 940 | "layout": "IPY_MODEL_ee8b4317d860412fa60bca6d1d0c935a" 941 | } 942 | }, 943 | "655321e930014d03a986ab862d9c01a1": { 944 | "model_module": "@jupyter-widgets/controls", 945 | "model_name": "HTMLModel", 946 | "state": { 947 | "_dom_classes": [], 948 | "_model_module": "@jupyter-widgets/controls", 949 | "_model_module_version": "1.5.0", 950 | "_model_name": "HTMLModel", 951 | "_view_count": null, 952 | "_view_module": "@jupyter-widgets/controls", 953 | "_view_module_version": "1.5.0", 954 | "_view_name": "HTMLView", 955 | "description": "", 956 | "description_tooltip": null, 957 | "layout": "IPY_MODEL_78f9f8dd4275405b96194f2507dae175", 958 | "placeholder": "​", 959 | "style": "IPY_MODEL_3c129350f337408f920706b3836528b9", 960 | "value": " 0/4542 [00:00<?, ?it/s]" 961 | } 962 | }, 963 | "70658583c62a40c28345d43ceb70810b": { 964 | "model_module": "@jupyter-widgets/base", 965 | "model_name": "LayoutModel", 966 | "state": { 967 | "_model_module": "@jupyter-widgets/base", 968 | "_model_module_version": "1.2.0", 969 | "_model_name": "LayoutModel", 970 | "_view_count": null, 971 | "_view_module": "@jupyter-widgets/base", 972 | "_view_module_version": "1.2.0", 973 | "_view_name": "LayoutView", 974 | "align_content": null, 975 | "align_items": null, 976 | "align_self": null, 977 | "border": null, 978 | "bottom": null, 979 | "display": null, 980 | "flex": null, 981 | "flex_flow": null, 982 | "grid_area": null, 983 | "grid_auto_columns": null, 984 | "grid_auto_flow": null, 985 | "grid_auto_rows": null, 986 | "grid_column": null, 987 | "grid_gap": null, 988 | "grid_row": null, 989 | "grid_template_areas": null, 990 | "grid_template_columns": null, 991 | "grid_template_rows": null, 992 | "height": null, 993 | "justify_content": null, 994 | "justify_items": null, 995 | "left": null, 996 | "margin": null, 997 | "max_height": null, 998 | "max_width": null, 999 | "min_height": null, 1000 | "min_width": null, 1001 | "object_fit": null, 1002 | "object_position": null, 1003 | "order": null, 1004 | "overflow": null, 1005 | "overflow_x": null, 1006 | "overflow_y": null, 1007 | "padding": null, 1008 | "right": null, 1009 | "top": null, 1010 | "visibility": null, 1011 | "width": null 1012 | } 1013 | }, 1014 | "78f9f8dd4275405b96194f2507dae175": { 1015 | "model_module": "@jupyter-widgets/base", 1016 | "model_name": "LayoutModel", 1017 | "state": { 1018 | "_model_module": "@jupyter-widgets/base", 1019 | "_model_module_version": "1.2.0", 1020 | "_model_name": "LayoutModel", 1021 | "_view_count": null, 1022 | "_view_module": "@jupyter-widgets/base", 1023 | "_view_module_version": "1.2.0", 1024 | "_view_name": "LayoutView", 1025 | "align_content": null, 1026 | "align_items": null, 1027 | "align_self": null, 1028 | "border": null, 1029 | "bottom": null, 1030 | "display": null, 1031 | "flex": null, 1032 | "flex_flow": null, 1033 | "grid_area": null, 1034 | "grid_auto_columns": null, 1035 | "grid_auto_flow": null, 1036 | "grid_auto_rows": null, 1037 | "grid_column": null, 1038 | "grid_gap": null, 1039 | "grid_row": null, 1040 | "grid_template_areas": null, 1041 | "grid_template_columns": null, 1042 | "grid_template_rows": null, 1043 | "height": null, 1044 | "justify_content": null, 1045 | "justify_items": null, 1046 | "left": null, 1047 | "margin": null, 1048 | "max_height": null, 1049 | "max_width": null, 1050 | "min_height": null, 1051 | "min_width": null, 1052 | "object_fit": null, 1053 | "object_position": null, 1054 | "order": null, 1055 | "overflow": null, 1056 | "overflow_x": null, 1057 | "overflow_y": null, 1058 | "padding": null, 1059 | "right": null, 1060 | "top": null, 1061 | "visibility": null, 1062 | "width": null 1063 | } 1064 | }, 1065 | "8f7373518a7840b38d00de7608a9e218": { 1066 | "model_module": "@jupyter-widgets/controls", 1067 | "model_name": "IntProgressModel", 1068 | "state": { 1069 | "_dom_classes": [], 1070 | "_model_module": "@jupyter-widgets/controls", 1071 | "_model_module_version": "1.5.0", 1072 | "_model_name": "IntProgressModel", 1073 | "_view_count": null, 1074 | "_view_module": "@jupyter-widgets/controls", 1075 | "_view_module_version": "1.5.0", 1076 | "_view_name": "ProgressView", 1077 | "bar_style": "success", 1078 | "description": "", 1079 | "description_tooltip": null, 1080 | "layout": "IPY_MODEL_10c91bce2e8b42c587557b0a649a79fe", 1081 | "max": 1, 1082 | "min": 0, 1083 | "orientation": "horizontal", 1084 | "style": "IPY_MODEL_df0848e4bca84d95b178cbe955417bc4", 1085 | "value": 1 1086 | } 1087 | }, 1088 | "8f98aca9cfae490dab5dccf54b939ce0": { 1089 | "model_module": "@jupyter-widgets/controls", 1090 | "model_name": "ProgressStyleModel", 1091 | "state": { 1092 | "_model_module": "@jupyter-widgets/controls", 1093 | "_model_module_version": "1.5.0", 1094 | "_model_name": "ProgressStyleModel", 1095 | "_view_count": null, 1096 | "_view_module": "@jupyter-widgets/base", 1097 | "_view_module_version": "1.2.0", 1098 | "_view_name": "StyleView", 1099 | "bar_color": null, 1100 | "description_width": "initial" 1101 | } 1102 | }, 1103 | "99bb01a7fb144b149460669c65b22654": { 1104 | "model_module": "@jupyter-widgets/controls", 1105 | "model_name": "DescriptionStyleModel", 1106 | "state": { 1107 | "_model_module": "@jupyter-widgets/controls", 1108 | "_model_module_version": "1.5.0", 1109 | "_model_name": "DescriptionStyleModel", 1110 | "_view_count": null, 1111 | "_view_module": "@jupyter-widgets/base", 1112 | "_view_module_version": "1.2.0", 1113 | "_view_name": "StyleView", 1114 | "description_width": "" 1115 | } 1116 | }, 1117 | "a739b97fb09848dbae2fb6d0fef09ef4": { 1118 | "model_module": "@jupyter-widgets/base", 1119 | "model_name": "LayoutModel", 1120 | "state": { 1121 | "_model_module": "@jupyter-widgets/base", 1122 | "_model_module_version": "1.2.0", 1123 | "_model_name": "LayoutModel", 1124 | "_view_count": null, 1125 | "_view_module": "@jupyter-widgets/base", 1126 | "_view_module_version": "1.2.0", 1127 | "_view_name": "LayoutView", 1128 | "align_content": null, 1129 | "align_items": null, 1130 | "align_self": null, 1131 | "border": null, 1132 | "bottom": null, 1133 | "display": null, 1134 | "flex": null, 1135 | "flex_flow": null, 1136 | "grid_area": null, 1137 | "grid_auto_columns": null, 1138 | "grid_auto_flow": null, 1139 | "grid_auto_rows": null, 1140 | "grid_column": null, 1141 | "grid_gap": null, 1142 | "grid_row": null, 1143 | "grid_template_areas": null, 1144 | "grid_template_columns": null, 1145 | "grid_template_rows": null, 1146 | "height": null, 1147 | "justify_content": null, 1148 | "justify_items": null, 1149 | "left": null, 1150 | "margin": null, 1151 | "max_height": null, 1152 | "max_width": null, 1153 | "min_height": null, 1154 | "min_width": null, 1155 | "object_fit": null, 1156 | "object_position": null, 1157 | "order": null, 1158 | "overflow": null, 1159 | "overflow_x": null, 1160 | "overflow_y": null, 1161 | "padding": null, 1162 | "right": null, 1163 | "top": null, 1164 | "visibility": null, 1165 | "width": null 1166 | } 1167 | }, 1168 | "a9ccabebd2c6428594120631518c505c": { 1169 | "model_module": "@jupyter-widgets/controls", 1170 | "model_name": "DescriptionStyleModel", 1171 | "state": { 1172 | "_model_module": "@jupyter-widgets/controls", 1173 | "_model_module_version": "1.5.0", 1174 | "_model_name": "DescriptionStyleModel", 1175 | "_view_count": null, 1176 | "_view_module": "@jupyter-widgets/base", 1177 | "_view_module_version": "1.2.0", 1178 | "_view_name": "StyleView", 1179 | "description_width": "" 1180 | } 1181 | }, 1182 | "ab7e6d830cc74c14876a052480eb185a": { 1183 | "model_module": "@jupyter-widgets/controls", 1184 | "model_name": "HTMLModel", 1185 | "state": { 1186 | "_dom_classes": [], 1187 | "_model_module": "@jupyter-widgets/controls", 1188 | "_model_module_version": "1.5.0", 1189 | "_model_name": "HTMLModel", 1190 | "_view_count": null, 1191 | "_view_module": "@jupyter-widgets/controls", 1192 | "_view_module_version": "1.5.0", 1193 | "_view_name": "HTMLView", 1194 | "description": "", 1195 | "description_tooltip": null, 1196 | "layout": "IPY_MODEL_a739b97fb09848dbae2fb6d0fef09ef4", 1197 | "placeholder": "​", 1198 | "style": "IPY_MODEL_a9ccabebd2c6428594120631518c505c", 1199 | "value": " 9920512/? [00:20<00:00, 1061366.86it/s]" 1200 | } 1201 | }, 1202 | "bcbba5e1db1d498eb4b9ce545ff26d8a": { 1203 | "model_module": "@jupyter-widgets/controls", 1204 | "model_name": "ProgressStyleModel", 1205 | "state": { 1206 | "_model_module": "@jupyter-widgets/controls", 1207 | "_model_module_version": "1.5.0", 1208 | "_model_name": "ProgressStyleModel", 1209 | "_view_count": null, 1210 | "_view_module": "@jupyter-widgets/base", 1211 | "_view_module_version": "1.2.0", 1212 | "_view_name": "StyleView", 1213 | "bar_color": null, 1214 | "description_width": "initial" 1215 | } 1216 | }, 1217 | "bcc108058dc4490986ec987c568b2034": { 1218 | "model_module": "@jupyter-widgets/controls", 1219 | "model_name": "IntProgressModel", 1220 | "state": { 1221 | "_dom_classes": [], 1222 | "_model_module": "@jupyter-widgets/controls", 1223 | "_model_module_version": "1.5.0", 1224 | "_model_name": "IntProgressModel", 1225 | "_view_count": null, 1226 | "_view_module": "@jupyter-widgets/controls", 1227 | "_view_module_version": "1.5.0", 1228 | "_view_name": "ProgressView", 1229 | "bar_style": "info", 1230 | "description": "", 1231 | "description_tooltip": null, 1232 | "layout": "IPY_MODEL_229d85ab3245444c9ad5b57c6ec58688", 1233 | "max": 1, 1234 | "min": 0, 1235 | "orientation": "horizontal", 1236 | "style": "IPY_MODEL_3a488dd9262d42ac9b119cd37e2e71c9", 1237 | "value": 1 1238 | } 1239 | }, 1240 | "d57756abec8742db9d650c814b474137": { 1241 | "model_module": "@jupyter-widgets/base", 1242 | "model_name": "LayoutModel", 1243 | "state": { 1244 | "_model_module": "@jupyter-widgets/base", 1245 | "_model_module_version": "1.2.0", 1246 | "_model_name": "LayoutModel", 1247 | "_view_count": null, 1248 | "_view_module": "@jupyter-widgets/base", 1249 | "_view_module_version": "1.2.0", 1250 | "_view_name": "LayoutView", 1251 | "align_content": null, 1252 | "align_items": null, 1253 | "align_self": null, 1254 | "border": null, 1255 | "bottom": null, 1256 | "display": null, 1257 | "flex": null, 1258 | "flex_flow": null, 1259 | "grid_area": null, 1260 | "grid_auto_columns": null, 1261 | "grid_auto_flow": null, 1262 | "grid_auto_rows": null, 1263 | "grid_column": null, 1264 | "grid_gap": null, 1265 | "grid_row": null, 1266 | "grid_template_areas": null, 1267 | "grid_template_columns": null, 1268 | "grid_template_rows": null, 1269 | "height": null, 1270 | "justify_content": null, 1271 | "justify_items": null, 1272 | "left": null, 1273 | "margin": null, 1274 | "max_height": null, 1275 | "max_width": null, 1276 | "min_height": null, 1277 | "min_width": null, 1278 | "object_fit": null, 1279 | "object_position": null, 1280 | "order": null, 1281 | "overflow": null, 1282 | "overflow_x": null, 1283 | "overflow_y": null, 1284 | "padding": null, 1285 | "right": null, 1286 | "top": null, 1287 | "visibility": null, 1288 | "width": null 1289 | } 1290 | }, 1291 | "d677eff7d5a4494087a60ff1db4dab92": { 1292 | "model_module": "@jupyter-widgets/base", 1293 | "model_name": "LayoutModel", 1294 | "state": { 1295 | "_model_module": "@jupyter-widgets/base", 1296 | "_model_module_version": "1.2.0", 1297 | "_model_name": "LayoutModel", 1298 | "_view_count": null, 1299 | "_view_module": "@jupyter-widgets/base", 1300 | "_view_module_version": "1.2.0", 1301 | "_view_name": "LayoutView", 1302 | "align_content": null, 1303 | "align_items": null, 1304 | "align_self": null, 1305 | "border": null, 1306 | "bottom": null, 1307 | "display": null, 1308 | "flex": null, 1309 | "flex_flow": null, 1310 | "grid_area": null, 1311 | "grid_auto_columns": null, 1312 | "grid_auto_flow": null, 1313 | "grid_auto_rows": null, 1314 | "grid_column": null, 1315 | "grid_gap": null, 1316 | "grid_row": null, 1317 | "grid_template_areas": null, 1318 | "grid_template_columns": null, 1319 | "grid_template_rows": null, 1320 | "height": null, 1321 | "justify_content": null, 1322 | "justify_items": null, 1323 | "left": null, 1324 | "margin": null, 1325 | "max_height": null, 1326 | "max_width": null, 1327 | "min_height": null, 1328 | "min_width": null, 1329 | "object_fit": null, 1330 | "object_position": null, 1331 | "order": null, 1332 | "overflow": null, 1333 | "overflow_x": null, 1334 | "overflow_y": null, 1335 | "padding": null, 1336 | "right": null, 1337 | "top": null, 1338 | "visibility": null, 1339 | "width": null 1340 | } 1341 | }, 1342 | "df0848e4bca84d95b178cbe955417bc4": { 1343 | "model_module": "@jupyter-widgets/controls", 1344 | "model_name": "ProgressStyleModel", 1345 | "state": { 1346 | "_model_module": "@jupyter-widgets/controls", 1347 | "_model_module_version": "1.5.0", 1348 | "_model_name": "ProgressStyleModel", 1349 | "_view_count": null, 1350 | "_view_module": "@jupyter-widgets/base", 1351 | "_view_module_version": "1.2.0", 1352 | "_view_name": "StyleView", 1353 | "bar_color": null, 1354 | "description_width": "initial" 1355 | } 1356 | }, 1357 | "e11d5108f5a74dd6b7143172c693dc6a": { 1358 | "model_module": "@jupyter-widgets/controls", 1359 | "model_name": "HBoxModel", 1360 | "state": { 1361 | "_dom_classes": [], 1362 | "_model_module": "@jupyter-widgets/controls", 1363 | "_model_module_version": "1.5.0", 1364 | "_model_name": "HBoxModel", 1365 | "_view_count": null, 1366 | "_view_module": "@jupyter-widgets/controls", 1367 | "_view_module_version": "1.5.0", 1368 | "_view_name": "HBoxView", 1369 | "box_style": "", 1370 | "children": [ 1371 | "IPY_MODEL_2de5166b2a40479fb69c08e7fda49594", 1372 | "IPY_MODEL_655321e930014d03a986ab862d9c01a1" 1373 | ], 1374 | "layout": "IPY_MODEL_590b36901152442088a4c5e05e781b3d" 1375 | } 1376 | }, 1377 | "e488365503444d9c8313fd2017d05f9e": { 1378 | "model_module": "@jupyter-widgets/controls", 1379 | "model_name": "HBoxModel", 1380 | "state": { 1381 | "_dom_classes": [], 1382 | "_model_module": "@jupyter-widgets/controls", 1383 | "_model_module_version": "1.5.0", 1384 | "_model_name": "HBoxModel", 1385 | "_view_count": null, 1386 | "_view_module": "@jupyter-widgets/controls", 1387 | "_view_module_version": "1.5.0", 1388 | "_view_name": "HBoxView", 1389 | "box_style": "", 1390 | "children": [ 1391 | "IPY_MODEL_200b60fab8514008a571603c6a2519f8", 1392 | "IPY_MODEL_34a38f1355e54a24ad0757e91e016ff4" 1393 | ], 1394 | "layout": "IPY_MODEL_d677eff7d5a4494087a60ff1db4dab92" 1395 | } 1396 | }, 1397 | "e92daa0c7043415b8463dffd654e93e5": { 1398 | "model_module": "@jupyter-widgets/controls", 1399 | "model_name": "HTMLModel", 1400 | "state": { 1401 | "_dom_classes": [], 1402 | "_model_module": "@jupyter-widgets/controls", 1403 | "_model_module_version": "1.5.0", 1404 | "_model_name": "HTMLModel", 1405 | "_view_count": null, 1406 | "_view_module": "@jupyter-widgets/controls", 1407 | "_view_module_version": "1.5.0", 1408 | "_view_name": "HTMLView", 1409 | "description": "", 1410 | "description_tooltip": null, 1411 | "layout": "IPY_MODEL_70658583c62a40c28345d43ceb70810b", 1412 | "placeholder": "​", 1413 | "style": "IPY_MODEL_1a38c10f1d0e4f20a4958d453757f9e7", 1414 | "value": " 32768/? [00:01<00:00, 28639.26it/s]" 1415 | } 1416 | }, 1417 | "ee8b4317d860412fa60bca6d1d0c935a": { 1418 | "model_module": "@jupyter-widgets/base", 1419 | "model_name": "LayoutModel", 1420 | "state": { 1421 | "_model_module": "@jupyter-widgets/base", 1422 | "_model_module_version": "1.2.0", 1423 | "_model_name": "LayoutModel", 1424 | "_view_count": null, 1425 | "_view_module": "@jupyter-widgets/base", 1426 | "_view_module_version": "1.2.0", 1427 | "_view_name": "LayoutView", 1428 | "align_content": null, 1429 | "align_items": null, 1430 | "align_self": null, 1431 | "border": null, 1432 | "bottom": null, 1433 | "display": null, 1434 | "flex": null, 1435 | "flex_flow": null, 1436 | "grid_area": null, 1437 | "grid_auto_columns": null, 1438 | "grid_auto_flow": null, 1439 | "grid_auto_rows": null, 1440 | "grid_column": null, 1441 | "grid_gap": null, 1442 | "grid_row": null, 1443 | "grid_template_areas": null, 1444 | "grid_template_columns": null, 1445 | "grid_template_rows": null, 1446 | "height": null, 1447 | "justify_content": null, 1448 | "justify_items": null, 1449 | "left": null, 1450 | "margin": null, 1451 | "max_height": null, 1452 | "max_width": null, 1453 | "min_height": null, 1454 | "min_width": null, 1455 | "object_fit": null, 1456 | "object_position": null, 1457 | "order": null, 1458 | "overflow": null, 1459 | "overflow_x": null, 1460 | "overflow_y": null, 1461 | "padding": null, 1462 | "right": null, 1463 | "top": null, 1464 | "visibility": null, 1465 | "width": null 1466 | } 1467 | } 1468 | } 1469 | } 1470 | }, 1471 | "nbformat": 4, 1472 | "nbformat_minor": 4 1473 | } 1474 | -------------------------------------------------------------------------------- /2a_convolutional_neural_network.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "colab_type": "text", 7 | "id": "mCt-frpFH8oN" 8 | }, 9 | "source": [ 10 | "The tutorials use PyTorch. You will need to load the following dependencies." 11 | ] 12 | }, 13 | { 14 | "cell_type": "code", 15 | "execution_count": null, 16 | "metadata": { 17 | "colab": {}, 18 | "colab_type": "code", 19 | "id": "hqnl0AKVXIA4" 20 | }, 21 | "outputs": [], 22 | "source": [ 23 | "import random\n", 24 | "\n", 25 | "import imageio\n", 26 | "import matplotlib.pyplot as plt\n", 27 | "import numpy as np\n", 28 | "import PIL\n", 29 | "import skimage.transform\n", 30 | "import torch\n", 31 | "import torch.nn as nn\n", 32 | "import torch.utils.data\n", 33 | "import torchvision\n", 34 | "from IPython import display\n", 35 | "from torchvision import datasets, transforms" 36 | ] 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "metadata": {}, 41 | "source": [ 42 | "# Tutorial 2a: Convolutional Neural Network (CNN)" 43 | ] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "metadata": { 48 | "colab_type": "text", 49 | "id": "CKeYuM-cIXxs" 50 | }, 51 | "source": [ 52 | "The code below may be helpful in visualizing PyTorch tensors as images." 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "metadata": { 59 | "colab": {}, 60 | "colab_type": "code", 61 | "id": "ZZd_rI8edYIB" 62 | }, 63 | "outputs": [], 64 | "source": [ 65 | "%matplotlib inline\n", 66 | "\n", 67 | "\n", 68 | "def show(img):\n", 69 | " \"\"\"Show PyTorch tensor img as an image in matplotlib.\"\"\"\n", 70 | " npimg = img.cpu().detach().numpy()\n", 71 | " plt.imshow(np.transpose(npimg, (1, 2, 0)), interpolation=\"nearest\")\n", 72 | " plt.grid(False)\n", 73 | " plt.gca().axis(\"off\")\n", 74 | "\n", 75 | "\n", 76 | "def display_thumb(img):\n", 77 | " display.display(transforms.Resize(128)(img))" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": null, 83 | "metadata": { 84 | "colab": {}, 85 | "colab_type": "code", 86 | "id": "dzfEE578uSNp" 87 | }, 88 | "outputs": [], 89 | "source": [ 90 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": { 96 | "colab_type": "text", 97 | "id": "zgE0byUgKwM6" 98 | }, 99 | "source": [ 100 | "Load MNIST and define train/test functions as before. Please make sure you read the code carefully and understand what it is doing." 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": null, 106 | "metadata": { 107 | "colab": { 108 | "base_uri": "https://localhost:8080/", 109 | "height": 332, 110 | "referenced_widgets": [ 111 | "a42490f444564fd7ac946d852614792f", 112 | "3373c12efcad42459f1d367d75ea3d46", 113 | "07dc9f747d104c11a43f01a95ed57902", 114 | "6e22ffc1689840a080a7a0306b6e40aa", 115 | "5ba9ed21d81a4642a60bf8a25043b231", 116 | "da77a43ce62847e095c45f116b46d575", 117 | "3b9341e0dd1942a590a8a75b47c4c02a", 118 | "af5aba6f88f94c03b40f93feaf5513b4", 119 | "efa04fca6187462393be792a87847ead", 120 | "34c560af3a2c4560af7d0ff51e3ba192", 121 | "927c1878303a4601a439af11796c860d", 122 | "ba3f6e39cfab49c2bdbf95eb8dcea867", 123 | "c6d63cfbab0d43ef97fd112250844b4b", 124 | "71de708bde354564bb46cdbd2254557c", 125 | "d83b581aeb674ca293e3b67ecdc9059f", 126 | "5cc2b2260d394c79abdac992228b0e31", 127 | "ef04916378e54e12806b08b0bcfbb8fd", 128 | "76bca89552e94a14a770b01d2ed5a632", 129 | "63cefcf9944d48cd8896bff26f85d8d0", 130 | "1b7ef3e5f8014502929183c185bde554", 131 | "053499a12c9a4c4ab3c2effba82aba70", 132 | "4dc4b3a5bc7d4d1896a459836fc7610f", 133 | "e95daa37066d4e369e3541e1816e07c1", 134 | "0542a250bd1a443c89cba5d525ee5e7b", 135 | "694a02b8ec6340b29c7bd53fd4a91e29", 136 | "8047845f5cf24c9889c98d227bdbab79", 137 | "8a64b78684d54b6bbc4949ac797d2ebe", 138 | "2e4528b2db2a497c89907cd6366ad893", 139 | "3bd8a63651ee4c538747d328c7b1e192", 140 | "f723b21b2c314b599f65d06b686693bf", 141 | "9954acb0661f4e438c7baf7e4a43f493", 142 | "f768eb9193774ea28185768e913147b9" 143 | ] 144 | }, 145 | "colab_type": "code", 146 | "executionInfo": { 147 | "elapsed": 10900, 148 | "status": "ok", 149 | "timestamp": 1586534977022, 150 | "user": { 151 | "displayName": "Anton Bakhtin", 152 | "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gjq-QU8evTujx-PnMoIketNeW4yB_kgdtnRJ03IxnY=s64", 153 | "userId": "12331399045734156147" 154 | }, 155 | "user_tz": 240 156 | }, 157 | "id": "-NMUce6PKu-o", 158 | "outputId": "de31434a-01c4-42c3-cdbe-df59992d0355" 159 | }, 160 | "outputs": [], 161 | "source": [ 162 | "# Load the training and test dataset.\n", 163 | "mnist_train = datasets.MNIST(\n", 164 | " \"/tmp/mnist\", train=True, download=True, transform=transforms.ToTensor()\n", 165 | ")\n", 166 | "mnist_test = datasets.MNIST(\n", 167 | " \"/tmp/mnist\", train=False, download=True, transform=transforms.ToTensor()\n", 168 | ")\n", 169 | "\n", 170 | "# Size of the batches the data loader will produce.\n", 171 | "batch_size = 64\n", 172 | "\n", 173 | "# This creates the dataloaders.\n", 174 | "train_loader = torch.utils.data.DataLoader(\n", 175 | " mnist_train, batch_size=batch_size, shuffle=True\n", 176 | ")\n", 177 | "test_loader = torch.utils.data.DataLoader(\n", 178 | " mnist_test, batch_size=batch_size, shuffle=False\n", 179 | ")" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": null, 185 | "metadata": { 186 | "colab": { 187 | "base_uri": "https://localhost:8080/", 188 | "height": 34 189 | }, 190 | "colab_type": "code", 191 | "executionInfo": { 192 | "elapsed": 10899, 193 | "status": "ok", 194 | "timestamp": 1586534977023, 195 | "user": { 196 | "displayName": "Anton Bakhtin", 197 | "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gjq-QU8evTujx-PnMoIketNeW4yB_kgdtnRJ03IxnY=s64", 198 | "userId": "12331399045734156147" 199 | }, 200 | "user_tz": 240 201 | }, 202 | "id": "TgAJ94UgK1VU", 203 | "outputId": "7aad0856-eaf6-4cb0-a70f-da77c43c4e7e" 204 | }, 205 | "outputs": [], 206 | "source": [ 207 | "def train(model, criterion, data_loader, optimizer, num_epochs):\n", 208 | " \"\"\"Simple training loop for a PyTorch model.\"\"\"\n", 209 | "\n", 210 | " # Make sure model is in training mode.\n", 211 | " model.train()\n", 212 | "\n", 213 | " # Move model to the device (CPU or GPU).\n", 214 | " model.to(device)\n", 215 | "\n", 216 | " # Exponential moving average of the loss.\n", 217 | " ema_loss = None\n", 218 | "\n", 219 | " # Loop over epochs.\n", 220 | " for epoch in range(num_epochs):\n", 221 | "\n", 222 | " # Loop over data.\n", 223 | " for batch_idx, (data, target) in enumerate(data_loader):\n", 224 | "\n", 225 | " # Forward pass.\n", 226 | " output = model(data.to(device))\n", 227 | " loss = criterion(output.to(device), target.to(device))\n", 228 | "\n", 229 | " # Backward pass.\n", 230 | " optimizer.zero_grad()\n", 231 | " loss.backward()\n", 232 | " optimizer.step()\n", 233 | "\n", 234 | " # NOTE: It is important to call .item() on the loss before summing.\n", 235 | " if ema_loss is None:\n", 236 | " ema_loss = loss.item()\n", 237 | " else:\n", 238 | " ema_loss += (loss.item() - ema_loss) * 0.01\n", 239 | "\n", 240 | " # Print out progress the end of epoch.\n", 241 | " print(\n", 242 | " \"Train Epoch: {} \\tLoss: {:.6f}\".format(epoch, ema_loss),\n", 243 | " )\n", 244 | "\n", 245 | "\n", 246 | "def test(model, data_loader):\n", 247 | " \"\"\"Measures the accuracy of a model on a data set.\"\"\"\n", 248 | " # Make sure the model is in evaluation mode.\n", 249 | " model.eval()\n", 250 | " correct = 0\n", 251 | "\n", 252 | " # We do not need to maintain intermediate activations while testing.\n", 253 | " with torch.no_grad():\n", 254 | "\n", 255 | " # Loop over test data.\n", 256 | " for data, target in data_loader:\n", 257 | "\n", 258 | " # Forward pass.\n", 259 | " output = model(data.to(device))\n", 260 | "\n", 261 | " # Get the label corresponding to the highest predicted probability.\n", 262 | " pred = output.argmax(dim=1, keepdim=True)\n", 263 | "\n", 264 | " # Count number of correct predictions.\n", 265 | " correct += pred.cpu().eq(target.view_as(pred)).sum().item()\n", 266 | "\n", 267 | " # Print test accuracy.\n", 268 | " percent = 100.0 * correct / len(data_loader.dataset)\n", 269 | " print(f\"Accuracy: {correct} / {len(data_loader.dataset)} ({percent:.0f}%)\")\n", 270 | " return percent" 271 | ] 272 | }, 273 | { 274 | "cell_type": "markdown", 275 | "metadata": {}, 276 | "source": [ 277 | "In the last tutorial, you implemented a naive convolution. In this section you will implement your own version of forward pass of nn.Conv2d without using any of PyTorch's (or numpy's) pre-defined convolutional functions." 278 | ] 279 | }, 280 | { 281 | "cell_type": "code", 282 | "execution_count": null, 283 | "metadata": {}, 284 | "outputs": [], 285 | "source": [ 286 | "def conv_forward_naive(x, w, b, conv_param):\n", 287 | " \"\"\"\n", 288 | " A naive Python implementation of a convolutional layer.\n", 289 | " The input consists of N data points, each with C channels, height H and\n", 290 | " width W. We convolve each input with F different filters, where each filter\n", 291 | " spans all C channels and has height HH and width WW.\n", 292 | " Input:\n", 293 | " - x: Input data of shape (N, C, H, W)\n", 294 | " - w: Filter weights of shape (F, C, HH, WW)\n", 295 | " - b: Biases, of shape (F,)\n", 296 | " - conv_param: A dictionary with the following keys:\n", 297 | " - 'stride': The number of pixels between adjacent receptive fields in the\n", 298 | " horizontal and vertical directions.\n", 299 | " - 'pad': The number of pixels that will be used to zero-pad the input.\n", 300 | "\n", 301 | " During padding, 'pad' zeros should be placed symmetrically (i.e., equally on both sides)\n", 302 | " along the height and width axes of the input. Be careful not to modfiy the original\n", 303 | " input x directly.\n", 304 | " Returns an array.\n", 305 | " - out: Output data, of shape (N, F, H', W') where H' and W' are given by\n", 306 | " H' = 1 + (H + 2 * pad - HH) / stride\n", 307 | " W' = 1 + (W + 2 * pad - WW) / stride\n", 308 | " \"\"\"\n", 309 | " out = None\n", 310 | "\n", 311 | " N, C, H, W = x.shape\n", 312 | " num_filters, _, filter_height, filter_width = w.shape\n", 313 | " stride, pad = conv_param[\"stride\"], conv_param[\"pad\"]\n", 314 | "\n", 315 | " # Check dimensions.\n", 316 | " assert (W + 2 * pad - filter_width) % stride == 0, \"width does not work\"\n", 317 | " assert (H + 2 * pad - filter_height) % stride == 0, \"height does not work\"\n", 318 | "\n", 319 | " ###########################################################################\n", 320 | " # TODO: Implement the forward pass of a convolutional layer without using #\n", 321 | " # nn.Conv2D or other implementations of convolutions. Instead, use #\n", 322 | " # standard for- and while-loops to iterate over the tensors. #\n", 323 | " # #\n", 324 | " # Hint: you can use the function torch.nn.functional.pad for padding. #\n", 325 | " ###########################################################################" 326 | ] 327 | }, 328 | { 329 | "cell_type": "markdown", 330 | "metadata": {}, 331 | "source": [ 332 | "You can test your implementation by running the following testing code:" 333 | ] 334 | }, 335 | { 336 | "cell_type": "code", 337 | "execution_count": null, 338 | "metadata": {}, 339 | "outputs": [], 340 | "source": [ 341 | "# Make convolution module.\n", 342 | "w_shape = (3, 3, 4, 4)\n", 343 | "w = torch.linspace(-0.2, 0.3, steps=torch.prod(torch.tensor(w_shape))).reshape(w_shape)\n", 344 | "b = torch.linspace(-0.1, 0.2, steps=3)\n", 345 | "\n", 346 | "# Compute output of module and compare against reference values.\n", 347 | "x_shape = (2, 3, 4, 4)\n", 348 | "x = torch.linspace(-0.1, 0.5, steps=torch.prod(torch.tensor(x_shape))).reshape(x_shape)\n", 349 | "out = conv_forward_naive(x, w, b, {\"stride\": 2, \"pad\": 1})\n", 350 | "\n", 351 | "correct_out = torch.tensor(\n", 352 | " [\n", 353 | " [\n", 354 | " [[-0.08759809, -0.10987781], [-0.18387192, -0.2109216]],\n", 355 | " [[0.21027089, 0.21661097], [0.22847626, 0.23004637]],\n", 356 | " [[0.50813986, 0.54309974], [0.64082444, 0.67101435]],\n", 357 | " ],\n", 358 | " [\n", 359 | " [[-0.98053589, -1.03143541], [-1.19128892, -1.24695841]],\n", 360 | " [[0.69108355, 0.66880383], [0.59480972, 0.56776003]],\n", 361 | " [[2.36270298, 2.36904306], [2.38090835, 2.38247847]],\n", 362 | " ],\n", 363 | " ]\n", 364 | ")\n", 365 | "\n", 366 | "# Compare your output to ours; difference should be around e-8\n", 367 | "print(\"Testing conv_forward_naive\")\n", 368 | "rel_error = ((out - correct_out) / (out + correct_out + 1e-6)).mean()\n", 369 | "print(\"difference: \", rel_error)\n", 370 | "if abs(rel_error) < 1e-6:\n", 371 | " print(\"Nice work! Your implementation of a convolution layer works correctly.\")\n", 372 | "else:\n", 373 | " print(\n", 374 | " \"Something is wrong. The output was expected to be {} but it was {}\".format(\n", 375 | " correct_out, out\n", 376 | " )\n", 377 | " )" 378 | ] 379 | }, 380 | { 381 | "cell_type": "markdown", 382 | "metadata": { 383 | "colab_type": "text", 384 | "id": "GjTq5-k_q8s_" 385 | }, 386 | "source": [ 387 | "\n", 388 | "We will now replace the logistic regressor from the last tutorial by a small convolutional network with two convolutional layers and a linear layer, and ReLU activations in between the layers. Implement the model and use the same functions as before to train and test the convolutional network." 389 | ] 390 | }, 391 | { 392 | "cell_type": "code", 393 | "execution_count": null, 394 | "metadata": { 395 | "colab": { 396 | "base_uri": "https://localhost:8080/", 397 | "height": 34 398 | }, 399 | "colab_type": "code", 400 | "executionInfo": { 401 | "elapsed": 11424, 402 | "status": "ok", 403 | "timestamp": 1586534977557, 404 | "user": { 405 | "displayName": "Anton Bakhtin", 406 | "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gjq-QU8evTujx-PnMoIketNeW4yB_kgdtnRJ03IxnY=s64", 407 | "userId": "12331399045734156147" 408 | }, 409 | "user_tz": 240 410 | }, 411 | "id": "5t4hWoUYpp05", 412 | "outputId": "958f173f-ff66-466e-93d2-d8961a88b8a7" 413 | }, 414 | "outputs": [], 415 | "source": [ 416 | "class ConvolutionalNetwork(nn.Module):\n", 417 | " \"\"\"Simple convolutional network.\"\"\"\n", 418 | "\n", 419 | " def __init__(self, image_side_size, num_classes, in_channels=1):\n", 420 | " super(ConvolutionalNetwork, self).__init__()\n", 421 | "\n", 422 | " # Fill these in:\n", 423 | " ##########################################################################\n", 424 | " # TODO: Implement a convulutional and a linear part. #\n", 425 | " # Hint: see forward() to understand how they should work together. #\n", 426 | " ##########################################################################\n", 427 | " self.conv_network = None\n", 428 | " self.linear = None\n", 429 | "\n", 430 | " def forward(self, x):\n", 431 | " x = self.conv_network(x)\n", 432 | " x = self.linear(x.view(x.size(0), -1))\n", 433 | " return x\n", 434 | "\n", 435 | "\n", 436 | "# Create and train convolutional network.\n", 437 | "# The accuracy should be around 96%.\n", 438 | "conv_model = ConvolutionalNetwork(28, 10)\n", 439 | "###########################################################################\n", 440 | "# TODO: Create criterion and optimize here. #\n", 441 | "###########################################################################\n", 442 | "criterion = None\n", 443 | "optimizer = None\n", 444 | "\n", 445 | "train(conv_model, criterion, train_loader, optimizer, num_epochs=5)\n", 446 | "test(conv_model, test_loader)" 447 | ] 448 | }, 449 | { 450 | "cell_type": "markdown", 451 | "metadata": {}, 452 | "source": [ 453 | "Inspect the filters in the first layer of the trained convolutional network. What do they look like? Why?" 454 | ] 455 | }, 456 | { 457 | "cell_type": "code", 458 | "execution_count": null, 459 | "metadata": { 460 | "colab": { 461 | "base_uri": "https://localhost:8080/", 462 | "height": 34 463 | }, 464 | "colab_type": "code", 465 | "executionInfo": { 466 | "elapsed": 11418, 467 | "status": "ok", 468 | "timestamp": 1586534977560, 469 | "user": { 470 | "displayName": "Anton Bakhtin", 471 | "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gjq-QU8evTujx-PnMoIketNeW4yB_kgdtnRJ03IxnY=s64", 472 | "userId": "12331399045734156147" 473 | }, 474 | "user_tz": 240 475 | }, 476 | "id": "LuSuQHT6bNdU", 477 | "outputId": "39179847-8353-4a8e-b5e3-e372c1bafcc7" 478 | }, 479 | "outputs": [], 480 | "source": [ 481 | "first_conv = list(conv_model.conv_network.children())[0]\n", 482 | "show(\n", 483 | " torchvision.utils.make_grid(\n", 484 | " first_conv.weight,\n", 485 | " normalize=True,\n", 486 | " nrow=8,\n", 487 | " )\n", 488 | ")" 489 | ] 490 | } 491 | ], 492 | "metadata": { 493 | "accelerator": "GPU", 494 | "colab": { 495 | "collapsed_sections": [], 496 | "name": "convnet_tutorial2_filled.ipynb", 497 | "provenance": [ 498 | { 499 | "file_id": "1Eq_v8eTk4XAxFXhjL6rk26gwfwvpZYBn", 500 | "timestamp": 1553633151446 501 | } 502 | ] 503 | }, 504 | "kernelspec": { 505 | "display_name": "Python 3 (ipykernel)", 506 | "language": "python", 507 | "name": "python3" 508 | }, 509 | "language_info": { 510 | "codemirror_mode": { 511 | "name": "ipython", 512 | "version": 3 513 | }, 514 | "file_extension": ".py", 515 | "mimetype": "text/x-python", 516 | "name": "python", 517 | "nbconvert_exporter": "python", 518 | "pygments_lexer": "ipython3", 519 | "version": "3.10.2" 520 | }, 521 | "widgets": { 522 | "application/vnd.jupyter.widget-state+json": { 523 | "053499a12c9a4c4ab3c2effba82aba70": { 524 | "model_module": "@jupyter-widgets/controls", 525 | "model_name": "ProgressStyleModel", 526 | "state": { 527 | "_model_module": "@jupyter-widgets/controls", 528 | "_model_module_version": "1.5.0", 529 | "_model_name": "ProgressStyleModel", 530 | "_view_count": null, 531 | "_view_module": "@jupyter-widgets/base", 532 | "_view_module_version": "1.2.0", 533 | "_view_name": "StyleView", 534 | "bar_color": null, 535 | "description_width": "initial" 536 | } 537 | }, 538 | "0542a250bd1a443c89cba5d525ee5e7b": { 539 | "model_module": "@jupyter-widgets/base", 540 | "model_name": "LayoutModel", 541 | "state": { 542 | "_model_module": "@jupyter-widgets/base", 543 | "_model_module_version": "1.2.0", 544 | "_model_name": "LayoutModel", 545 | "_view_count": null, 546 | "_view_module": "@jupyter-widgets/base", 547 | "_view_module_version": "1.2.0", 548 | "_view_name": "LayoutView", 549 | "align_content": null, 550 | "align_items": null, 551 | "align_self": null, 552 | "border": null, 553 | "bottom": null, 554 | "display": null, 555 | "flex": null, 556 | "flex_flow": null, 557 | "grid_area": null, 558 | "grid_auto_columns": null, 559 | "grid_auto_flow": null, 560 | "grid_auto_rows": null, 561 | "grid_column": null, 562 | "grid_gap": null, 563 | "grid_row": null, 564 | "grid_template_areas": null, 565 | "grid_template_columns": null, 566 | "grid_template_rows": null, 567 | "height": null, 568 | "justify_content": null, 569 | "justify_items": null, 570 | "left": null, 571 | "margin": null, 572 | "max_height": null, 573 | "max_width": null, 574 | "min_height": null, 575 | "min_width": null, 576 | "object_fit": null, 577 | "object_position": null, 578 | "order": null, 579 | "overflow": null, 580 | "overflow_x": null, 581 | "overflow_y": null, 582 | "padding": null, 583 | "right": null, 584 | "top": null, 585 | "visibility": null, 586 | "width": null 587 | } 588 | }, 589 | "07dc9f747d104c11a43f01a95ed57902": { 590 | "model_module": "@jupyter-widgets/controls", 591 | "model_name": "IntProgressModel", 592 | "state": { 593 | "_dom_classes": [], 594 | "_model_module": "@jupyter-widgets/controls", 595 | "_model_module_version": "1.5.0", 596 | "_model_name": "IntProgressModel", 597 | "_view_count": null, 598 | "_view_module": "@jupyter-widgets/controls", 599 | "_view_module_version": "1.5.0", 600 | "_view_name": "ProgressView", 601 | "bar_style": "info", 602 | "description": "", 603 | "description_tooltip": null, 604 | "layout": "IPY_MODEL_da77a43ce62847e095c45f116b46d575", 605 | "max": 1, 606 | "min": 0, 607 | "orientation": "horizontal", 608 | "style": "IPY_MODEL_5ba9ed21d81a4642a60bf8a25043b231", 609 | "value": 1 610 | } 611 | }, 612 | "1b7ef3e5f8014502929183c185bde554": { 613 | "model_module": "@jupyter-widgets/controls", 614 | "model_name": "HTMLModel", 615 | "state": { 616 | "_dom_classes": [], 617 | "_model_module": "@jupyter-widgets/controls", 618 | "_model_module_version": "1.5.0", 619 | "_model_name": "HTMLModel", 620 | "_view_count": null, 621 | "_view_module": "@jupyter-widgets/controls", 622 | "_view_module_version": "1.5.0", 623 | "_view_name": "HTMLView", 624 | "description": "", 625 | "description_tooltip": null, 626 | "layout": "IPY_MODEL_0542a250bd1a443c89cba5d525ee5e7b", 627 | "placeholder": "​", 628 | "style": "IPY_MODEL_e95daa37066d4e369e3541e1816e07c1", 629 | "value": " 1654784/? [00:16<00:00, 1174712.52it/s]" 630 | } 631 | }, 632 | "2e4528b2db2a497c89907cd6366ad893": { 633 | "model_module": "@jupyter-widgets/controls", 634 | "model_name": "HTMLModel", 635 | "state": { 636 | "_dom_classes": [], 637 | "_model_module": "@jupyter-widgets/controls", 638 | "_model_module_version": "1.5.0", 639 | "_model_name": "HTMLModel", 640 | "_view_count": null, 641 | "_view_module": "@jupyter-widgets/controls", 642 | "_view_module_version": "1.5.0", 643 | "_view_name": "HTMLView", 644 | "description": "", 645 | "description_tooltip": null, 646 | "layout": "IPY_MODEL_f768eb9193774ea28185768e913147b9", 647 | "placeholder": "​", 648 | "style": "IPY_MODEL_9954acb0661f4e438c7baf7e4a43f493", 649 | "value": " 8192/? [00:00<00:00, 11911.67it/s]" 650 | } 651 | }, 652 | "3373c12efcad42459f1d367d75ea3d46": { 653 | "model_module": "@jupyter-widgets/base", 654 | "model_name": "LayoutModel", 655 | "state": { 656 | "_model_module": "@jupyter-widgets/base", 657 | "_model_module_version": "1.2.0", 658 | "_model_name": "LayoutModel", 659 | "_view_count": null, 660 | "_view_module": "@jupyter-widgets/base", 661 | "_view_module_version": "1.2.0", 662 | "_view_name": "LayoutView", 663 | "align_content": null, 664 | "align_items": null, 665 | "align_self": null, 666 | "border": null, 667 | "bottom": null, 668 | "display": null, 669 | "flex": null, 670 | "flex_flow": null, 671 | "grid_area": null, 672 | "grid_auto_columns": null, 673 | "grid_auto_flow": null, 674 | "grid_auto_rows": null, 675 | "grid_column": null, 676 | "grid_gap": null, 677 | "grid_row": null, 678 | "grid_template_areas": null, 679 | "grid_template_columns": null, 680 | "grid_template_rows": null, 681 | "height": null, 682 | "justify_content": null, 683 | "justify_items": null, 684 | "left": null, 685 | "margin": null, 686 | "max_height": null, 687 | "max_width": null, 688 | "min_height": null, 689 | "min_width": null, 690 | "object_fit": null, 691 | "object_position": null, 692 | "order": null, 693 | "overflow": null, 694 | "overflow_x": null, 695 | "overflow_y": null, 696 | "padding": null, 697 | "right": null, 698 | "top": null, 699 | "visibility": null, 700 | "width": null 701 | } 702 | }, 703 | "34c560af3a2c4560af7d0ff51e3ba192": { 704 | "model_module": "@jupyter-widgets/base", 705 | "model_name": "LayoutModel", 706 | "state": { 707 | "_model_module": "@jupyter-widgets/base", 708 | "_model_module_version": "1.2.0", 709 | "_model_name": "LayoutModel", 710 | "_view_count": null, 711 | "_view_module": "@jupyter-widgets/base", 712 | "_view_module_version": "1.2.0", 713 | "_view_name": "LayoutView", 714 | "align_content": null, 715 | "align_items": null, 716 | "align_self": null, 717 | "border": null, 718 | "bottom": null, 719 | "display": null, 720 | "flex": null, 721 | "flex_flow": null, 722 | "grid_area": null, 723 | "grid_auto_columns": null, 724 | "grid_auto_flow": null, 725 | "grid_auto_rows": null, 726 | "grid_column": null, 727 | "grid_gap": null, 728 | "grid_row": null, 729 | "grid_template_areas": null, 730 | "grid_template_columns": null, 731 | "grid_template_rows": null, 732 | "height": null, 733 | "justify_content": null, 734 | "justify_items": null, 735 | "left": null, 736 | "margin": null, 737 | "max_height": null, 738 | "max_width": null, 739 | "min_height": null, 740 | "min_width": null, 741 | "object_fit": null, 742 | "object_position": null, 743 | "order": null, 744 | "overflow": null, 745 | "overflow_x": null, 746 | "overflow_y": null, 747 | "padding": null, 748 | "right": null, 749 | "top": null, 750 | "visibility": null, 751 | "width": null 752 | } 753 | }, 754 | "3815582c26af4ee4a6031a103a196eff": { 755 | "model_module": "@jupyter-widgets/base", 756 | "model_name": "LayoutModel", 757 | "state": { 758 | "_model_module": "@jupyter-widgets/base", 759 | "_model_module_version": "1.2.0", 760 | "_model_name": "LayoutModel", 761 | "_view_count": null, 762 | "_view_module": "@jupyter-widgets/base", 763 | "_view_module_version": "1.2.0", 764 | "_view_name": "LayoutView", 765 | "align_content": null, 766 | "align_items": null, 767 | "align_self": null, 768 | "border": null, 769 | "bottom": null, 770 | "display": null, 771 | "flex": null, 772 | "flex_flow": null, 773 | "grid_area": null, 774 | "grid_auto_columns": null, 775 | "grid_auto_flow": null, 776 | "grid_auto_rows": null, 777 | "grid_column": null, 778 | "grid_gap": null, 779 | "grid_row": null, 780 | "grid_template_areas": null, 781 | "grid_template_columns": null, 782 | "grid_template_rows": null, 783 | "height": null, 784 | "justify_content": null, 785 | "justify_items": null, 786 | "left": null, 787 | "margin": null, 788 | "max_height": null, 789 | "max_width": null, 790 | "min_height": null, 791 | "min_width": null, 792 | "object_fit": null, 793 | "object_position": null, 794 | "order": null, 795 | "overflow": null, 796 | "overflow_x": null, 797 | "overflow_y": null, 798 | "padding": null, 799 | "right": null, 800 | "top": null, 801 | "visibility": null, 802 | "width": null 803 | } 804 | }, 805 | "3b9341e0dd1942a590a8a75b47c4c02a": { 806 | "model_module": "@jupyter-widgets/controls", 807 | "model_name": "DescriptionStyleModel", 808 | "state": { 809 | "_model_module": "@jupyter-widgets/controls", 810 | "_model_module_version": "1.5.0", 811 | "_model_name": "DescriptionStyleModel", 812 | "_view_count": null, 813 | "_view_module": "@jupyter-widgets/base", 814 | "_view_module_version": "1.2.0", 815 | "_view_name": "StyleView", 816 | "description_width": "" 817 | } 818 | }, 819 | "3bd8a63651ee4c538747d328c7b1e192": { 820 | "model_module": "@jupyter-widgets/controls", 821 | "model_name": "ProgressStyleModel", 822 | "state": { 823 | "_model_module": "@jupyter-widgets/controls", 824 | "_model_module_version": "1.5.0", 825 | "_model_name": "ProgressStyleModel", 826 | "_view_count": null, 827 | "_view_module": "@jupyter-widgets/base", 828 | "_view_module_version": "1.2.0", 829 | "_view_name": "StyleView", 830 | "bar_color": null, 831 | "description_width": "initial" 832 | } 833 | }, 834 | "45200009cb0f40449c05c889854d653c": { 835 | "model_module": "@jupyter-widgets/base", 836 | "model_name": "LayoutModel", 837 | "state": { 838 | "_model_module": "@jupyter-widgets/base", 839 | "_model_module_version": "1.2.0", 840 | "_model_name": "LayoutModel", 841 | "_view_count": null, 842 | "_view_module": "@jupyter-widgets/base", 843 | "_view_module_version": "1.2.0", 844 | "_view_name": "LayoutView", 845 | "align_content": null, 846 | "align_items": null, 847 | "align_self": null, 848 | "border": null, 849 | "bottom": null, 850 | "display": null, 851 | "flex": null, 852 | "flex_flow": null, 853 | "grid_area": null, 854 | "grid_auto_columns": null, 855 | "grid_auto_flow": null, 856 | "grid_auto_rows": null, 857 | "grid_column": null, 858 | "grid_gap": null, 859 | "grid_row": null, 860 | "grid_template_areas": null, 861 | "grid_template_columns": null, 862 | "grid_template_rows": null, 863 | "height": null, 864 | "justify_content": null, 865 | "justify_items": null, 866 | "left": null, 867 | "margin": null, 868 | "max_height": null, 869 | "max_width": null, 870 | "min_height": null, 871 | "min_width": null, 872 | "object_fit": null, 873 | "object_position": null, 874 | "order": null, 875 | "overflow": null, 876 | "overflow_x": null, 877 | "overflow_y": null, 878 | "padding": null, 879 | "right": null, 880 | "top": null, 881 | "visibility": null, 882 | "width": null 883 | } 884 | }, 885 | "4dc4b3a5bc7d4d1896a459836fc7610f": { 886 | "model_module": "@jupyter-widgets/base", 887 | "model_name": "LayoutModel", 888 | "state": { 889 | "_model_module": "@jupyter-widgets/base", 890 | "_model_module_version": "1.2.0", 891 | "_model_name": "LayoutModel", 892 | "_view_count": null, 893 | "_view_module": "@jupyter-widgets/base", 894 | "_view_module_version": "1.2.0", 895 | "_view_name": "LayoutView", 896 | "align_content": null, 897 | "align_items": null, 898 | "align_self": null, 899 | "border": null, 900 | "bottom": null, 901 | "display": null, 902 | "flex": null, 903 | "flex_flow": null, 904 | "grid_area": null, 905 | "grid_auto_columns": null, 906 | "grid_auto_flow": null, 907 | "grid_auto_rows": null, 908 | "grid_column": null, 909 | "grid_gap": null, 910 | "grid_row": null, 911 | "grid_template_areas": null, 912 | "grid_template_columns": null, 913 | "grid_template_rows": null, 914 | "height": null, 915 | "justify_content": null, 916 | "justify_items": null, 917 | "left": null, 918 | "margin": null, 919 | "max_height": null, 920 | "max_width": null, 921 | "min_height": null, 922 | "min_width": null, 923 | "object_fit": null, 924 | "object_position": null, 925 | "order": null, 926 | "overflow": null, 927 | "overflow_x": null, 928 | "overflow_y": null, 929 | "padding": null, 930 | "right": null, 931 | "top": null, 932 | "visibility": null, 933 | "width": null 934 | } 935 | }, 936 | "5ba9ed21d81a4642a60bf8a25043b231": { 937 | "model_module": "@jupyter-widgets/controls", 938 | "model_name": "ProgressStyleModel", 939 | "state": { 940 | "_model_module": "@jupyter-widgets/controls", 941 | "_model_module_version": "1.5.0", 942 | "_model_name": "ProgressStyleModel", 943 | "_view_count": null, 944 | "_view_module": "@jupyter-widgets/base", 945 | "_view_module_version": "1.2.0", 946 | "_view_name": "StyleView", 947 | "bar_color": null, 948 | "description_width": "initial" 949 | } 950 | }, 951 | "5cc2b2260d394c79abdac992228b0e31": { 952 | "model_module": "@jupyter-widgets/base", 953 | "model_name": "LayoutModel", 954 | "state": { 955 | "_model_module": "@jupyter-widgets/base", 956 | "_model_module_version": "1.2.0", 957 | "_model_name": "LayoutModel", 958 | "_view_count": null, 959 | "_view_module": "@jupyter-widgets/base", 960 | "_view_module_version": "1.2.0", 961 | "_view_name": "LayoutView", 962 | "align_content": null, 963 | "align_items": null, 964 | "align_self": null, 965 | "border": null, 966 | "bottom": null, 967 | "display": null, 968 | "flex": null, 969 | "flex_flow": null, 970 | "grid_area": null, 971 | "grid_auto_columns": null, 972 | "grid_auto_flow": null, 973 | "grid_auto_rows": null, 974 | "grid_column": null, 975 | "grid_gap": null, 976 | "grid_row": null, 977 | "grid_template_areas": null, 978 | "grid_template_columns": null, 979 | "grid_template_rows": null, 980 | "height": null, 981 | "justify_content": null, 982 | "justify_items": null, 983 | "left": null, 984 | "margin": null, 985 | "max_height": null, 986 | "max_width": null, 987 | "min_height": null, 988 | "min_width": null, 989 | "object_fit": null, 990 | "object_position": null, 991 | "order": null, 992 | "overflow": null, 993 | "overflow_x": null, 994 | "overflow_y": null, 995 | "padding": null, 996 | "right": null, 997 | "top": null, 998 | "visibility": null, 999 | "width": null 1000 | } 1001 | }, 1002 | "63cefcf9944d48cd8896bff26f85d8d0": { 1003 | "model_module": "@jupyter-widgets/controls", 1004 | "model_name": "IntProgressModel", 1005 | "state": { 1006 | "_dom_classes": [], 1007 | "_model_module": "@jupyter-widgets/controls", 1008 | "_model_module_version": "1.5.0", 1009 | "_model_name": "IntProgressModel", 1010 | "_view_count": null, 1011 | "_view_module": "@jupyter-widgets/controls", 1012 | "_view_module_version": "1.5.0", 1013 | "_view_name": "ProgressView", 1014 | "bar_style": "info", 1015 | "description": "", 1016 | "description_tooltip": null, 1017 | "layout": "IPY_MODEL_4dc4b3a5bc7d4d1896a459836fc7610f", 1018 | "max": 1, 1019 | "min": 0, 1020 | "orientation": "horizontal", 1021 | "style": "IPY_MODEL_053499a12c9a4c4ab3c2effba82aba70", 1022 | "value": 1 1023 | } 1024 | }, 1025 | "694a02b8ec6340b29c7bd53fd4a91e29": { 1026 | "model_module": "@jupyter-widgets/controls", 1027 | "model_name": "HBoxModel", 1028 | "state": { 1029 | "_dom_classes": [], 1030 | "_model_module": "@jupyter-widgets/controls", 1031 | "_model_module_version": "1.5.0", 1032 | "_model_name": "HBoxModel", 1033 | "_view_count": null, 1034 | "_view_module": "@jupyter-widgets/controls", 1035 | "_view_module_version": "1.5.0", 1036 | "_view_name": "HBoxView", 1037 | "box_style": "", 1038 | "children": [ 1039 | "IPY_MODEL_8a64b78684d54b6bbc4949ac797d2ebe", 1040 | "IPY_MODEL_2e4528b2db2a497c89907cd6366ad893" 1041 | ], 1042 | "layout": "IPY_MODEL_8047845f5cf24c9889c98d227bdbab79" 1043 | } 1044 | }, 1045 | "6e22ffc1689840a080a7a0306b6e40aa": { 1046 | "model_module": "@jupyter-widgets/controls", 1047 | "model_name": "HTMLModel", 1048 | "state": { 1049 | "_dom_classes": [], 1050 | "_model_module": "@jupyter-widgets/controls", 1051 | "_model_module_version": "1.5.0", 1052 | "_model_name": "HTMLModel", 1053 | "_view_count": null, 1054 | "_view_module": "@jupyter-widgets/controls", 1055 | "_view_module_version": "1.5.0", 1056 | "_view_name": "HTMLView", 1057 | "description": "", 1058 | "description_tooltip": null, 1059 | "layout": "IPY_MODEL_af5aba6f88f94c03b40f93feaf5513b4", 1060 | "placeholder": "​", 1061 | "style": "IPY_MODEL_3b9341e0dd1942a590a8a75b47c4c02a", 1062 | "value": " 9920512/? [00:20<00:00, 4620495.53it/s]" 1063 | } 1064 | }, 1065 | "71de708bde354564bb46cdbd2254557c": { 1066 | "model_module": "@jupyter-widgets/base", 1067 | "model_name": "LayoutModel", 1068 | "state": { 1069 | "_model_module": "@jupyter-widgets/base", 1070 | "_model_module_version": "1.2.0", 1071 | "_model_name": "LayoutModel", 1072 | "_view_count": null, 1073 | "_view_module": "@jupyter-widgets/base", 1074 | "_view_module_version": "1.2.0", 1075 | "_view_name": "LayoutView", 1076 | "align_content": null, 1077 | "align_items": null, 1078 | "align_self": null, 1079 | "border": null, 1080 | "bottom": null, 1081 | "display": null, 1082 | "flex": null, 1083 | "flex_flow": null, 1084 | "grid_area": null, 1085 | "grid_auto_columns": null, 1086 | "grid_auto_flow": null, 1087 | "grid_auto_rows": null, 1088 | "grid_column": null, 1089 | "grid_gap": null, 1090 | "grid_row": null, 1091 | "grid_template_areas": null, 1092 | "grid_template_columns": null, 1093 | "grid_template_rows": null, 1094 | "height": null, 1095 | "justify_content": null, 1096 | "justify_items": null, 1097 | "left": null, 1098 | "margin": null, 1099 | "max_height": null, 1100 | "max_width": null, 1101 | "min_height": null, 1102 | "min_width": null, 1103 | "object_fit": null, 1104 | "object_position": null, 1105 | "order": null, 1106 | "overflow": null, 1107 | "overflow_x": null, 1108 | "overflow_y": null, 1109 | "padding": null, 1110 | "right": null, 1111 | "top": null, 1112 | "visibility": null, 1113 | "width": null 1114 | } 1115 | }, 1116 | "7421dc7a0a874ab8835cdb30a73372ee": { 1117 | "model_module": "@jupyter-widgets/controls", 1118 | "model_name": "HTMLModel", 1119 | "state": { 1120 | "_dom_classes": [], 1121 | "_model_module": "@jupyter-widgets/controls", 1122 | "_model_module_version": "1.5.0", 1123 | "_model_name": "HTMLModel", 1124 | "_view_count": null, 1125 | "_view_module": "@jupyter-widgets/controls", 1126 | "_view_module_version": "1.5.0", 1127 | "_view_name": "HTMLView", 1128 | "description": "", 1129 | "description_tooltip": null, 1130 | "layout": "IPY_MODEL_a8773930a517488dbdbb30ac3bef7971", 1131 | "placeholder": "​", 1132 | "style": "IPY_MODEL_93f3ec87a84542ada9518c7771066cf3", 1133 | "value": " 44.7M/44.7M [00:00<00:00, 149MB/s]" 1134 | } 1135 | }, 1136 | "76bca89552e94a14a770b01d2ed5a632": { 1137 | "model_module": "@jupyter-widgets/base", 1138 | "model_name": "LayoutModel", 1139 | "state": { 1140 | "_model_module": "@jupyter-widgets/base", 1141 | "_model_module_version": "1.2.0", 1142 | "_model_name": "LayoutModel", 1143 | "_view_count": null, 1144 | "_view_module": "@jupyter-widgets/base", 1145 | "_view_module_version": "1.2.0", 1146 | "_view_name": "LayoutView", 1147 | "align_content": null, 1148 | "align_items": null, 1149 | "align_self": null, 1150 | "border": null, 1151 | "bottom": null, 1152 | "display": null, 1153 | "flex": null, 1154 | "flex_flow": null, 1155 | "grid_area": null, 1156 | "grid_auto_columns": null, 1157 | "grid_auto_flow": null, 1158 | "grid_auto_rows": null, 1159 | "grid_column": null, 1160 | "grid_gap": null, 1161 | "grid_row": null, 1162 | "grid_template_areas": null, 1163 | "grid_template_columns": null, 1164 | "grid_template_rows": null, 1165 | "height": null, 1166 | "justify_content": null, 1167 | "justify_items": null, 1168 | "left": null, 1169 | "margin": null, 1170 | "max_height": null, 1171 | "max_width": null, 1172 | "min_height": null, 1173 | "min_width": null, 1174 | "object_fit": null, 1175 | "object_position": null, 1176 | "order": null, 1177 | "overflow": null, 1178 | "overflow_x": null, 1179 | "overflow_y": null, 1180 | "padding": null, 1181 | "right": null, 1182 | "top": null, 1183 | "visibility": null, 1184 | "width": null 1185 | } 1186 | }, 1187 | "8047845f5cf24c9889c98d227bdbab79": { 1188 | "model_module": "@jupyter-widgets/base", 1189 | "model_name": "LayoutModel", 1190 | "state": { 1191 | "_model_module": "@jupyter-widgets/base", 1192 | "_model_module_version": "1.2.0", 1193 | "_model_name": "LayoutModel", 1194 | "_view_count": null, 1195 | "_view_module": "@jupyter-widgets/base", 1196 | "_view_module_version": "1.2.0", 1197 | "_view_name": "LayoutView", 1198 | "align_content": null, 1199 | "align_items": null, 1200 | "align_self": null, 1201 | "border": null, 1202 | "bottom": null, 1203 | "display": null, 1204 | "flex": null, 1205 | "flex_flow": null, 1206 | "grid_area": null, 1207 | "grid_auto_columns": null, 1208 | "grid_auto_flow": null, 1209 | "grid_auto_rows": null, 1210 | "grid_column": null, 1211 | "grid_gap": null, 1212 | "grid_row": null, 1213 | "grid_template_areas": null, 1214 | "grid_template_columns": null, 1215 | "grid_template_rows": null, 1216 | "height": null, 1217 | "justify_content": null, 1218 | "justify_items": null, 1219 | "left": null, 1220 | "margin": null, 1221 | "max_height": null, 1222 | "max_width": null, 1223 | "min_height": null, 1224 | "min_width": null, 1225 | "object_fit": null, 1226 | "object_position": null, 1227 | "order": null, 1228 | "overflow": null, 1229 | "overflow_x": null, 1230 | "overflow_y": null, 1231 | "padding": null, 1232 | "right": null, 1233 | "top": null, 1234 | "visibility": null, 1235 | "width": null 1236 | } 1237 | }, 1238 | "8a64b78684d54b6bbc4949ac797d2ebe": { 1239 | "model_module": "@jupyter-widgets/controls", 1240 | "model_name": "IntProgressModel", 1241 | "state": { 1242 | "_dom_classes": [], 1243 | "_model_module": "@jupyter-widgets/controls", 1244 | "_model_module_version": "1.5.0", 1245 | "_model_name": "IntProgressModel", 1246 | "_view_count": null, 1247 | "_view_module": "@jupyter-widgets/controls", 1248 | "_view_module_version": "1.5.0", 1249 | "_view_name": "ProgressView", 1250 | "bar_style": "success", 1251 | "description": "", 1252 | "description_tooltip": null, 1253 | "layout": "IPY_MODEL_f723b21b2c314b599f65d06b686693bf", 1254 | "max": 1, 1255 | "min": 0, 1256 | "orientation": "horizontal", 1257 | "style": "IPY_MODEL_3bd8a63651ee4c538747d328c7b1e192", 1258 | "value": 1 1259 | } 1260 | }, 1261 | "927c1878303a4601a439af11796c860d": { 1262 | "model_module": "@jupyter-widgets/controls", 1263 | "model_name": "IntProgressModel", 1264 | "state": { 1265 | "_dom_classes": [], 1266 | "_model_module": "@jupyter-widgets/controls", 1267 | "_model_module_version": "1.5.0", 1268 | "_model_name": "IntProgressModel", 1269 | "_view_count": null, 1270 | "_view_module": "@jupyter-widgets/controls", 1271 | "_view_module_version": "1.5.0", 1272 | "_view_name": "ProgressView", 1273 | "bar_style": "success", 1274 | "description": "", 1275 | "description_tooltip": null, 1276 | "layout": "IPY_MODEL_71de708bde354564bb46cdbd2254557c", 1277 | "max": 1, 1278 | "min": 0, 1279 | "orientation": "horizontal", 1280 | "style": "IPY_MODEL_c6d63cfbab0d43ef97fd112250844b4b", 1281 | "value": 1 1282 | } 1283 | }, 1284 | "93f3ec87a84542ada9518c7771066cf3": { 1285 | "model_module": "@jupyter-widgets/controls", 1286 | "model_name": "DescriptionStyleModel", 1287 | "state": { 1288 | "_model_module": "@jupyter-widgets/controls", 1289 | "_model_module_version": "1.5.0", 1290 | "_model_name": "DescriptionStyleModel", 1291 | "_view_count": null, 1292 | "_view_module": "@jupyter-widgets/base", 1293 | "_view_module_version": "1.2.0", 1294 | "_view_name": "StyleView", 1295 | "description_width": "" 1296 | } 1297 | }, 1298 | "9954acb0661f4e438c7baf7e4a43f493": { 1299 | "model_module": "@jupyter-widgets/controls", 1300 | "model_name": "DescriptionStyleModel", 1301 | "state": { 1302 | "_model_module": "@jupyter-widgets/controls", 1303 | "_model_module_version": "1.5.0", 1304 | "_model_name": "DescriptionStyleModel", 1305 | "_view_count": null, 1306 | "_view_module": "@jupyter-widgets/base", 1307 | "_view_module_version": "1.2.0", 1308 | "_view_name": "StyleView", 1309 | "description_width": "" 1310 | } 1311 | }, 1312 | "a18479993a7547d688eed72386289911": { 1313 | "model_module": "@jupyter-widgets/controls", 1314 | "model_name": "ProgressStyleModel", 1315 | "state": { 1316 | "_model_module": "@jupyter-widgets/controls", 1317 | "_model_module_version": "1.5.0", 1318 | "_model_name": "ProgressStyleModel", 1319 | "_view_count": null, 1320 | "_view_module": "@jupyter-widgets/base", 1321 | "_view_module_version": "1.2.0", 1322 | "_view_name": "StyleView", 1323 | "bar_color": null, 1324 | "description_width": "initial" 1325 | } 1326 | }, 1327 | "a42490f444564fd7ac946d852614792f": { 1328 | "model_module": "@jupyter-widgets/controls", 1329 | "model_name": "HBoxModel", 1330 | "state": { 1331 | "_dom_classes": [], 1332 | "_model_module": "@jupyter-widgets/controls", 1333 | "_model_module_version": "1.5.0", 1334 | "_model_name": "HBoxModel", 1335 | "_view_count": null, 1336 | "_view_module": "@jupyter-widgets/controls", 1337 | "_view_module_version": "1.5.0", 1338 | "_view_name": "HBoxView", 1339 | "box_style": "", 1340 | "children": [ 1341 | "IPY_MODEL_07dc9f747d104c11a43f01a95ed57902", 1342 | "IPY_MODEL_6e22ffc1689840a080a7a0306b6e40aa" 1343 | ], 1344 | "layout": "IPY_MODEL_3373c12efcad42459f1d367d75ea3d46" 1345 | } 1346 | }, 1347 | "a8773930a517488dbdbb30ac3bef7971": { 1348 | "model_module": "@jupyter-widgets/base", 1349 | "model_name": "LayoutModel", 1350 | "state": { 1351 | "_model_module": "@jupyter-widgets/base", 1352 | "_model_module_version": "1.2.0", 1353 | "_model_name": "LayoutModel", 1354 | "_view_count": null, 1355 | "_view_module": "@jupyter-widgets/base", 1356 | "_view_module_version": "1.2.0", 1357 | "_view_name": "LayoutView", 1358 | "align_content": null, 1359 | "align_items": null, 1360 | "align_self": null, 1361 | "border": null, 1362 | "bottom": null, 1363 | "display": null, 1364 | "flex": null, 1365 | "flex_flow": null, 1366 | "grid_area": null, 1367 | "grid_auto_columns": null, 1368 | "grid_auto_flow": null, 1369 | "grid_auto_rows": null, 1370 | "grid_column": null, 1371 | "grid_gap": null, 1372 | "grid_row": null, 1373 | "grid_template_areas": null, 1374 | "grid_template_columns": null, 1375 | "grid_template_rows": null, 1376 | "height": null, 1377 | "justify_content": null, 1378 | "justify_items": null, 1379 | "left": null, 1380 | "margin": null, 1381 | "max_height": null, 1382 | "max_width": null, 1383 | "min_height": null, 1384 | "min_width": null, 1385 | "object_fit": null, 1386 | "object_position": null, 1387 | "order": null, 1388 | "overflow": null, 1389 | "overflow_x": null, 1390 | "overflow_y": null, 1391 | "padding": null, 1392 | "right": null, 1393 | "top": null, 1394 | "visibility": null, 1395 | "width": null 1396 | } 1397 | }, 1398 | "af5aba6f88f94c03b40f93feaf5513b4": { 1399 | "model_module": "@jupyter-widgets/base", 1400 | "model_name": "LayoutModel", 1401 | "state": { 1402 | "_model_module": "@jupyter-widgets/base", 1403 | "_model_module_version": "1.2.0", 1404 | "_model_name": "LayoutModel", 1405 | "_view_count": null, 1406 | "_view_module": "@jupyter-widgets/base", 1407 | "_view_module_version": "1.2.0", 1408 | "_view_name": "LayoutView", 1409 | "align_content": null, 1410 | "align_items": null, 1411 | "align_self": null, 1412 | "border": null, 1413 | "bottom": null, 1414 | "display": null, 1415 | "flex": null, 1416 | "flex_flow": null, 1417 | "grid_area": null, 1418 | "grid_auto_columns": null, 1419 | "grid_auto_flow": null, 1420 | "grid_auto_rows": null, 1421 | "grid_column": null, 1422 | "grid_gap": null, 1423 | "grid_row": null, 1424 | "grid_template_areas": null, 1425 | "grid_template_columns": null, 1426 | "grid_template_rows": null, 1427 | "height": null, 1428 | "justify_content": null, 1429 | "justify_items": null, 1430 | "left": null, 1431 | "margin": null, 1432 | "max_height": null, 1433 | "max_width": null, 1434 | "min_height": null, 1435 | "min_width": null, 1436 | "object_fit": null, 1437 | "object_position": null, 1438 | "order": null, 1439 | "overflow": null, 1440 | "overflow_x": null, 1441 | "overflow_y": null, 1442 | "padding": null, 1443 | "right": null, 1444 | "top": null, 1445 | "visibility": null, 1446 | "width": null 1447 | } 1448 | }, 1449 | "ba3f6e39cfab49c2bdbf95eb8dcea867": { 1450 | "model_module": "@jupyter-widgets/controls", 1451 | "model_name": "HTMLModel", 1452 | "state": { 1453 | "_dom_classes": [], 1454 | "_model_module": "@jupyter-widgets/controls", 1455 | "_model_module_version": "1.5.0", 1456 | "_model_name": "HTMLModel", 1457 | "_view_count": null, 1458 | "_view_module": "@jupyter-widgets/controls", 1459 | "_view_module_version": "1.5.0", 1460 | "_view_name": "HTMLView", 1461 | "description": "", 1462 | "description_tooltip": null, 1463 | "layout": "IPY_MODEL_5cc2b2260d394c79abdac992228b0e31", 1464 | "placeholder": "​", 1465 | "style": "IPY_MODEL_d83b581aeb674ca293e3b67ecdc9059f", 1466 | "value": " 32768/? [00:00<00:00, 52827.20it/s]" 1467 | } 1468 | }, 1469 | "bdc046fad38343b3aba8fd0fbaffb9c9": { 1470 | "model_module": "@jupyter-widgets/controls", 1471 | "model_name": "IntProgressModel", 1472 | "state": { 1473 | "_dom_classes": [], 1474 | "_model_module": "@jupyter-widgets/controls", 1475 | "_model_module_version": "1.5.0", 1476 | "_model_name": "IntProgressModel", 1477 | "_view_count": null, 1478 | "_view_module": "@jupyter-widgets/controls", 1479 | "_view_module_version": "1.5.0", 1480 | "_view_name": "ProgressView", 1481 | "bar_style": "success", 1482 | "description": "100%", 1483 | "description_tooltip": null, 1484 | "layout": "IPY_MODEL_3815582c26af4ee4a6031a103a196eff", 1485 | "max": 46827520, 1486 | "min": 0, 1487 | "orientation": "horizontal", 1488 | "style": "IPY_MODEL_a18479993a7547d688eed72386289911", 1489 | "value": 46827520 1490 | } 1491 | }, 1492 | "c6d63cfbab0d43ef97fd112250844b4b": { 1493 | "model_module": "@jupyter-widgets/controls", 1494 | "model_name": "ProgressStyleModel", 1495 | "state": { 1496 | "_model_module": "@jupyter-widgets/controls", 1497 | "_model_module_version": "1.5.0", 1498 | "_model_name": "ProgressStyleModel", 1499 | "_view_count": null, 1500 | "_view_module": "@jupyter-widgets/base", 1501 | "_view_module_version": "1.2.0", 1502 | "_view_name": "StyleView", 1503 | "bar_color": null, 1504 | "description_width": "initial" 1505 | } 1506 | }, 1507 | "cb7a435d8db0472eb3388f5d0d53043f": { 1508 | "model_module": "@jupyter-widgets/controls", 1509 | "model_name": "HBoxModel", 1510 | "state": { 1511 | "_dom_classes": [], 1512 | "_model_module": "@jupyter-widgets/controls", 1513 | "_model_module_version": "1.5.0", 1514 | "_model_name": "HBoxModel", 1515 | "_view_count": null, 1516 | "_view_module": "@jupyter-widgets/controls", 1517 | "_view_module_version": "1.5.0", 1518 | "_view_name": "HBoxView", 1519 | "box_style": "", 1520 | "children": [ 1521 | "IPY_MODEL_bdc046fad38343b3aba8fd0fbaffb9c9", 1522 | "IPY_MODEL_7421dc7a0a874ab8835cdb30a73372ee" 1523 | ], 1524 | "layout": "IPY_MODEL_45200009cb0f40449c05c889854d653c" 1525 | } 1526 | }, 1527 | "d83b581aeb674ca293e3b67ecdc9059f": { 1528 | "model_module": "@jupyter-widgets/controls", 1529 | "model_name": "DescriptionStyleModel", 1530 | "state": { 1531 | "_model_module": "@jupyter-widgets/controls", 1532 | "_model_module_version": "1.5.0", 1533 | "_model_name": "DescriptionStyleModel", 1534 | "_view_count": null, 1535 | "_view_module": "@jupyter-widgets/base", 1536 | "_view_module_version": "1.2.0", 1537 | "_view_name": "StyleView", 1538 | "description_width": "" 1539 | } 1540 | }, 1541 | "da77a43ce62847e095c45f116b46d575": { 1542 | "model_module": "@jupyter-widgets/base", 1543 | "model_name": "LayoutModel", 1544 | "state": { 1545 | "_model_module": "@jupyter-widgets/base", 1546 | "_model_module_version": "1.2.0", 1547 | "_model_name": "LayoutModel", 1548 | "_view_count": null, 1549 | "_view_module": "@jupyter-widgets/base", 1550 | "_view_module_version": "1.2.0", 1551 | "_view_name": "LayoutView", 1552 | "align_content": null, 1553 | "align_items": null, 1554 | "align_self": null, 1555 | "border": null, 1556 | "bottom": null, 1557 | "display": null, 1558 | "flex": null, 1559 | "flex_flow": null, 1560 | "grid_area": null, 1561 | "grid_auto_columns": null, 1562 | "grid_auto_flow": null, 1563 | "grid_auto_rows": null, 1564 | "grid_column": null, 1565 | "grid_gap": null, 1566 | "grid_row": null, 1567 | "grid_template_areas": null, 1568 | "grid_template_columns": null, 1569 | "grid_template_rows": null, 1570 | "height": null, 1571 | "justify_content": null, 1572 | "justify_items": null, 1573 | "left": null, 1574 | "margin": null, 1575 | "max_height": null, 1576 | "max_width": null, 1577 | "min_height": null, 1578 | "min_width": null, 1579 | "object_fit": null, 1580 | "object_position": null, 1581 | "order": null, 1582 | "overflow": null, 1583 | "overflow_x": null, 1584 | "overflow_y": null, 1585 | "padding": null, 1586 | "right": null, 1587 | "top": null, 1588 | "visibility": null, 1589 | "width": null 1590 | } 1591 | }, 1592 | "e95daa37066d4e369e3541e1816e07c1": { 1593 | "model_module": "@jupyter-widgets/controls", 1594 | "model_name": "DescriptionStyleModel", 1595 | "state": { 1596 | "_model_module": "@jupyter-widgets/controls", 1597 | "_model_module_version": "1.5.0", 1598 | "_model_name": "DescriptionStyleModel", 1599 | "_view_count": null, 1600 | "_view_module": "@jupyter-widgets/base", 1601 | "_view_module_version": "1.2.0", 1602 | "_view_name": "StyleView", 1603 | "description_width": "" 1604 | } 1605 | }, 1606 | "ef04916378e54e12806b08b0bcfbb8fd": { 1607 | "model_module": "@jupyter-widgets/controls", 1608 | "model_name": "HBoxModel", 1609 | "state": { 1610 | "_dom_classes": [], 1611 | "_model_module": "@jupyter-widgets/controls", 1612 | "_model_module_version": "1.5.0", 1613 | "_model_name": "HBoxModel", 1614 | "_view_count": null, 1615 | "_view_module": "@jupyter-widgets/controls", 1616 | "_view_module_version": "1.5.0", 1617 | "_view_name": "HBoxView", 1618 | "box_style": "", 1619 | "children": [ 1620 | "IPY_MODEL_63cefcf9944d48cd8896bff26f85d8d0", 1621 | "IPY_MODEL_1b7ef3e5f8014502929183c185bde554" 1622 | ], 1623 | "layout": "IPY_MODEL_76bca89552e94a14a770b01d2ed5a632" 1624 | } 1625 | }, 1626 | "efa04fca6187462393be792a87847ead": { 1627 | "model_module": "@jupyter-widgets/controls", 1628 | "model_name": "HBoxModel", 1629 | "state": { 1630 | "_dom_classes": [], 1631 | "_model_module": "@jupyter-widgets/controls", 1632 | "_model_module_version": "1.5.0", 1633 | "_model_name": "HBoxModel", 1634 | "_view_count": null, 1635 | "_view_module": "@jupyter-widgets/controls", 1636 | "_view_module_version": "1.5.0", 1637 | "_view_name": "HBoxView", 1638 | "box_style": "", 1639 | "children": [ 1640 | "IPY_MODEL_927c1878303a4601a439af11796c860d", 1641 | "IPY_MODEL_ba3f6e39cfab49c2bdbf95eb8dcea867" 1642 | ], 1643 | "layout": "IPY_MODEL_34c560af3a2c4560af7d0ff51e3ba192" 1644 | } 1645 | }, 1646 | "f723b21b2c314b599f65d06b686693bf": { 1647 | "model_module": "@jupyter-widgets/base", 1648 | "model_name": "LayoutModel", 1649 | "state": { 1650 | "_model_module": "@jupyter-widgets/base", 1651 | "_model_module_version": "1.2.0", 1652 | "_model_name": "LayoutModel", 1653 | "_view_count": null, 1654 | "_view_module": "@jupyter-widgets/base", 1655 | "_view_module_version": "1.2.0", 1656 | "_view_name": "LayoutView", 1657 | "align_content": null, 1658 | "align_items": null, 1659 | "align_self": null, 1660 | "border": null, 1661 | "bottom": null, 1662 | "display": null, 1663 | "flex": null, 1664 | "flex_flow": null, 1665 | "grid_area": null, 1666 | "grid_auto_columns": null, 1667 | "grid_auto_flow": null, 1668 | "grid_auto_rows": null, 1669 | "grid_column": null, 1670 | "grid_gap": null, 1671 | "grid_row": null, 1672 | "grid_template_areas": null, 1673 | "grid_template_columns": null, 1674 | "grid_template_rows": null, 1675 | "height": null, 1676 | "justify_content": null, 1677 | "justify_items": null, 1678 | "left": null, 1679 | "margin": null, 1680 | "max_height": null, 1681 | "max_width": null, 1682 | "min_height": null, 1683 | "min_width": null, 1684 | "object_fit": null, 1685 | "object_position": null, 1686 | "order": null, 1687 | "overflow": null, 1688 | "overflow_x": null, 1689 | "overflow_y": null, 1690 | "padding": null, 1691 | "right": null, 1692 | "top": null, 1693 | "visibility": null, 1694 | "width": null 1695 | } 1696 | }, 1697 | "f768eb9193774ea28185768e913147b9": { 1698 | "model_module": "@jupyter-widgets/base", 1699 | "model_name": "LayoutModel", 1700 | "state": { 1701 | "_model_module": "@jupyter-widgets/base", 1702 | "_model_module_version": "1.2.0", 1703 | "_model_name": "LayoutModel", 1704 | "_view_count": null, 1705 | "_view_module": "@jupyter-widgets/base", 1706 | "_view_module_version": "1.2.0", 1707 | "_view_name": "LayoutView", 1708 | "align_content": null, 1709 | "align_items": null, 1710 | "align_self": null, 1711 | "border": null, 1712 | "bottom": null, 1713 | "display": null, 1714 | "flex": null, 1715 | "flex_flow": null, 1716 | "grid_area": null, 1717 | "grid_auto_columns": null, 1718 | "grid_auto_flow": null, 1719 | "grid_auto_rows": null, 1720 | "grid_column": null, 1721 | "grid_gap": null, 1722 | "grid_row": null, 1723 | "grid_template_areas": null, 1724 | "grid_template_columns": null, 1725 | "grid_template_rows": null, 1726 | "height": null, 1727 | "justify_content": null, 1728 | "justify_items": null, 1729 | "left": null, 1730 | "margin": null, 1731 | "max_height": null, 1732 | "max_width": null, 1733 | "min_height": null, 1734 | "min_width": null, 1735 | "object_fit": null, 1736 | "object_position": null, 1737 | "order": null, 1738 | "overflow": null, 1739 | "overflow_x": null, 1740 | "overflow_y": null, 1741 | "padding": null, 1742 | "right": null, 1743 | "top": null, 1744 | "visibility": null, 1745 | "width": null 1746 | } 1747 | } 1748 | } 1749 | } 1750 | }, 1751 | "nbformat": 4, 1752 | "nbformat_minor": 4 1753 | } 1754 | --------------------------------------------------------------------------------