├── .gitignore ├── 1_mlp.ipynb ├── 2_lenet.ipynb ├── 3_alexnet.ipynb ├── 4_vgg.ipynb ├── 5_resnet.ipynb ├── LICENSE ├── README.md ├── assets ├── alexnet.png ├── batch-norm.png ├── cifar10.png ├── filter-mnist.png ├── filter-mnist.xml ├── filtered-image.png ├── lenet5.png ├── lr-scheduler.png ├── mlp-mnist.png ├── mlp-mnist.xml ├── multiple-channel-mnist.png ├── multiple-channel-mnist.xml ├── multiple-filter-mnist.png ├── multiple-filter-mnist.xml ├── relu.png ├── resnet-blocks.png ├── resnet-pad.png ├── resnet-pad.xml ├── resnet-skip.png ├── resnet-table.png ├── single-filter.png ├── single-filter.xml ├── single-pool.png ├── single-pool.xml ├── subsample-mnist.png ├── subsample-mnist.xml ├── subsampled-image.png ├── vgg-resnet.png ├── vgg-table.png └── vgg.png └── misc ├── 4 - VGG.ipynb ├── 5 - ResNet.ipynb ├── 6 - ResNet - Dogs vs Cats.ipynb ├── conv order.ipynb ├── download_dogs-vs-cats.sh └── process_dogs-vs-cats.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | 106 | data/ 107 | .data/ 108 | models/ 109 | *.pt 110 | .vscode/ 111 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Ben Trevett 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # PyTorch Image Classification 2 | 3 | This repo contains tutorials covering image classification using [PyTorch](https://github.com/pytorch/pytorch) 1.7, [torchvision](https://github.com/pytorch/vision) 0.8, [matplotlib](https://matplotlib.org/) 3.3 and [scikit-learn](https://scikit-learn.org/stable/index.html) 0.24, with Python 3.8. 4 | 5 | We'll start by implementing a multilayer perceptron (MLP) and then move on to architectures using convolutional neural networks (CNNs). Specifically, we'll implement [LeNet](http://yann.lecun.com/exdb/lenet/), [AlexNet](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf), [VGG](https://arxiv.org/abs/1409.1556) and [ResNet](https://arxiv.org/abs/1512.03385). 6 | 7 | **If you find any mistakes or disagree with any of the explanations, please do not hesitate to [submit an issue](https://github.com/bentrevett/pytorch-image-classification/issues/new). I welcome any feedback, positive or negative!** 8 | 9 | ## Getting Started 10 | 11 | To install PyTorch, see installation instructions on the [PyTorch website](https://pytorch.org/). 12 | 13 | The instructions to install PyTorch should also detail how to install torchvision but can also be installed via: 14 | 15 | ``` bash 16 | pip install torchvision 17 | ``` 18 | 19 | ## Tutorials 20 | 21 | * 1 - [Multilayer Perceptron](https://github.com/bentrevett/pytorch-image-classification/blob/master/1_mlp.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-image-classification/blob/master/1_mlp.ipynb) 22 | 23 | This tutorial provides an introduction to PyTorch and TorchVision. We'll learn how to: load datasets, augment data, define a multilayer perceptron (MLP), train a model, view the outputs of our model, visualize the model's representations, and view the weights of the model. The experiments will be carried out on the MNIST dataset - a set of 28x28 handwritten grayscale digits. 24 | 25 | * 2 - [LeNet](https://github.com/bentrevett/pytorch-image-classification/blob/master/2_lenet.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-image-classification/blob/master/2_lenet.ipynb) 26 | 27 | In this tutorial we'll implement the classic [LeNet](http://yann.lecun.com/exdb/lenet/) architecture. We'll look into convolutional neural networks and how convolutional layers and subsampling (aka pooling) layers work. 28 | 29 | * 3 - [AlexNet](https://github.com/bentrevett/pytorch-image-classification/blob/master/3_alexnet.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-image-classification/blob/master/3_alexnet.ipynb) 30 | 31 | In this tutorial we will implement [AlexNet](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf), the convolutional neural network architecture that helped start the current interest in deep learning. We will move on to the CIFAR10 dataset - 32x32 color images in ten classes. We show: how to define architectures using `nn.Sequential`, how to initialize the parameters of your neural network, and how to use the learning rate finder to determine a good initial learning rate. 32 | 33 | * 4 - [VGG](https://github.com/bentrevett/pytorch-image-classification/blob/master/4_vgg.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-image-classification/blob/master/4_vgg.ipynb) 34 | 35 | This tutorial will cover implementing the [VGG](https://arxiv.org/abs/1409.1556) model. However, instead of training the model from scratch we will instead load a VGG model pre-trained on the [ImageNet](http://www.image-net.org/challenges/LSVRC/) dataset and show how to perform transfer learning to adapt its weights to the CIFAR10 dataset using a technique called discriminative fine-tuning. We'll also explain how adaptive pooling layers and batch normalization works. 36 | 37 | * 5 - [ResNet](https://github.com/bentrevett/pytorch-image-classification/blob/master/5_resnet.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-image-classification/blob/master/5_resnet.ipynb) 38 | 39 | In this tutorial we will be implementing the [ResNet](https://arxiv.org/abs/1512.03385) model. We'll show how to load your own dataset, using the [CUB200](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html) dataset as an example, and also how to use learning rate schedulers which dynamically alter the learning rate of your model whilst training. Specifially, we'll use the one cycle policy introduced in [this](https://arxiv.org/abs/1803.09820) paper and is now starting to be commonly used for training computer vision models. 40 | 41 | ## References 42 | 43 | Here are some things I looked at while making these tutorials. Some of it may be out of date. 44 | 45 | - https://github.com/pytorch/tutorials 46 | - https://github.com/pytorch/examples 47 | - https://colah.github.io/posts/2014-10-Visualizing-MNIST/ 48 | - https://distill.pub/2016/misread-tsne/ 49 | - https://towardsdatascience.com/visualising-high-dimensional-datasets-using-pca-and-t-sne-in-python-8ef87e7915b 50 | - https://github.com/activatedgeek/LeNet-5 51 | - https://github.com/ChawDoe/LeNet5-MNIST-PyTorch 52 | - https://github.com/kuangliu/pytorch-cifar 53 | - https://github.com/akamaster/pytorch_resnet_cifar10 54 | - https://sgugger.github.io/the-1cycle-policy.html 55 | -------------------------------------------------------------------------------- /assets/alexnet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/alexnet.png -------------------------------------------------------------------------------- /assets/batch-norm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/batch-norm.png -------------------------------------------------------------------------------- /assets/cifar10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/cifar10.png -------------------------------------------------------------------------------- /assets/filter-mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/filter-mnist.png -------------------------------------------------------------------------------- /assets/filter-mnist.xml: -------------------------------------------------------------------------------- 1 | 7b1bd6LK+r79aXqM9z3oHuw3h+LeGHEv5mQNtooioCCKn/5fpTGdpjLX7LnapOpZv5XRnUAhAldtnvsuCuqbWN+d2wc7XT8nnh99Ezjv/E1sfBMEXpdl9AenlK8pPP+asjqE3mvaz4RJePFfE7nX1GPo+dkvH8yTJMrD9NdEN4lj381/SbMPh+T068eCJPr1qKm98omEiWtHZOoi9PL1LVUT1J/pHT9cre9H5hX9tmVn3z/8eiXZ2vaS07sksflNrB+SJL8t7c51P8L07lxu+7X+YuvbiR38OP+dHYZLPdXlXdke96dhjVOC+mb5nRel1/wo7Oj4es2v55uXdwjo1FO8GO6utIzCP+QhYtS3HT8aJlmYh0mMtjtJnic79IEIbzBsd7s6JMfYqydRcrh+lRhcf959Ry0KV3jfPElRqp2lt1wMwrOPTty4HrJ2T+XuKWjZs3P7m1i7rQqtNF59E+rh3DDHJ+6pvUpq6Gcwma2bsxVacpvoV8eo15Y4fXw8Na4fMDqD+mQ+6tZrq25QW29DnFiLTpNWdEEL/Sbat37uGlyOeBvoMHhzY82N52tuJug7r+Ot3d2s5ixanNs+F177nC5DQ3ixehd7oR+fS6nsb2pZt73mvE5N6Zd6vrTGJ6etc047Cvs7vXwptbZvH2b4eO4sao7mYyk2Ra8UFG9+FtvSAh1XPfY5DxW6VthUJ9OtXQ+N5nYx2HWm7c3gaJiz4c41Fnptb68am/0hjCZPE+O4PXf5i7vmgmboC7q69NzTbLV2tLmCrmagx7sdZ4VC5MS42AiGkVv7uKf4g1V9ZJsTrX+q1WRu1qwtV9Fq1Wg2t2gx2a5q/Vaz2TWWy+turbwhot/ov3FBf1F5apVuze7PbluPbTEO0F/uRZX+8pu6q3Wtb6DF3rJbWxt9o9a+Lp6NHlp86nVRLoxR7i3q3e6ofkKL9UW91h1NTiMjxIvPo8kILYaTWi2bjUar1iqsN2tZEy2iA9abzex/x/5nxx7EgduSOtrZjNRr9vHBbpIX/YG+fHrB7YzZ8KZFZs5RvgdxeT7y/UhLnPlc01DKk6JpjlsUYVSobTWLGl7SwFXogWfc6PtSN3/e8GrinM9q4My1zPYEpYg2L+hIeWuATnKNSnrLevJQwpozj4vBdjpExdbo6GftKHtKcNlzcV9u5HoQRI7v94eX9HA+DxunvYP2NNa4WnQfnj8b9LV7oXy+aNNOvBFDFDSMRa5IuCXjFX8x0Lz+mAvitWb11QYnizm6Nnwlu3J22aGFTO05RdbBOeNa8UbytCCaejt5tx8Oc5Qoya5Ydly1PkbhqTBbYzFwTr3Rw0ubzy+EcTwO+LGAmveWVuwseSkOx6qG+RnnYxHuPVE8KFlcdi6BNUDXcXk56X1ZD6yiFAMpaJym6ArQh4vFsDznohrztnm4Nbevza5x0i+l1dltTJ3L8feWaFsQFb70PPy0q8otXbN0VdIEH7F12ysJnV4x5gpxjM4At6DWVFvjExQ6Z1dcr8wGV8SmLCqq0hQdXUod8cJpIiqPXhwJXl+el/qhUXLi1Ht+bkhBtFk+i/qLXgiZaT2ypmupvAql84Af6ufYtVTPLw66ljkrDbfyilWKvtLScNG2oh0ueYOJ7B0jVw6Ooq0GXc/xhFy0e/vdbi74T7iyZw4qmsb8srnlh4Vb/EDzrAW6tE1QRB19KeCWINfijrQsHt4mDZ8SJ8kyqyUliCquChwvS+iA9sC8XgEv5k4amP1QDvoeHyp5NPV3vdDe98RimjoFh2NRFl0C85R3VvsOzkO3UUtEoXky6+siNsrgM1qp2zXI847XWm9flJeRmBZHnBl7EbdSs4F82PUbju0IkwydYYx0ED/EYXOKlGWKilUS4GLVc46CIe1f0t1+3tddf3GrH88XeSTaJ9l9YESZlnFYm3nNl33BBealxSWi0sbA63pRdEph7qlLQUw9E6sGST8Y2lodSkd5k01flqJd04OHRzpzPrYmm2Qao0qELtxbyJiV3tiK00HTHVoIQr7xhQkORaIuHIfa5mWA5HBrdJwihqXZ6z882k7L/US5mJMhL56Gcv3WKhxz7SCiXFyqvqb5/TEKha1+Wz1KwzQYTGeXYWTKl1ZNG2y1yezhEb2TzyL+eJk4KKYoR+HwItanu0zkL0G/zRcmiiFOYpuYS08dXuq4+KleJ/N539IfHec2flOZzM+hG5fr4vIkDqPNXBazGFNS3RZuPkfSsCjKY9FvC/qWR2Gjcz4rFtaQ7vzWzIyUzVJUCiwpRSkcNR+sp8qhkveeukJw8s2DsdNQ+55h0XBILbPgN87TQF+bRrkYCC+7VpY47qnop8HzVAzSRhlgfkFt+3B9Nxbnaeq3cPWedF4iThOQuIpT1boc1Ku/xltG6kyYD3jbGoZycXlGBetZGW6wslmop0ZRPrRFuOWtP59Y8/RwreVpqM46ytAT9azAWdrKGloqtg110Tlvjpdg+tQ+q610yWv10QPVqqn1uG0y3aLLROXpjQWyNr6HWv/BkD8rc/VSK2cFFkt+R86Kw8lw/PGn1P9S3S53ykoPo0wXL+PB82XlomjoF3lSxFNNNidxsFzs95YuSSHnPyWP5nG9lpeON91aCWJhbmRxj2Piy2EnBm0Vaa1oUR4jjXdx2xQ9YSWFxa9zurZLUu0kJN3POydfz1WcK0WERJPQlMx6WlitUls0DgHW5cGrLi/Xg4FcnttJd/7w8uKGsrnZyk+yjNtif15e1aSHHam6QA2iL1rTPY63Rdq7tjh4i3lAKlaIbK6IGovVxpIe7cg2c3PK6UNfU2vPdVmr//tYgM5onI07NY0b1YNSX117Lnrjmdw8bHur1Qp3reB/RpbabhjjdeHeU/KzFwalxknsv21IDp5/+JmY4H1z3E+n4W4WsnvptccJd9z453dJr91NbT/Z+fkB8eVet2qK+uO1d+m1+0/g3lJOP3vTeO01bf2+J+2eaL/24K3evv9nJxdaeO3n+rjPy+6lq53hRZvteOtH8nw3aR2/80T/lu+t/MnranLI18kqie2o+TPVuALEHVLXLqifn+knuNsKaUbR2Ph5Xr52YNrHPEFJ63wXvW71z2FuvVte4q9CLG5rjfPrN19XyvtKjC7Xer+yfL/yc6fr2n2v29XhS/rLjsI7geR4cP1/1zvICdJrn6t9WPn5v/uoJOsfF4aDH9l5WPx6Nn+SqX9zqv+kI5PsXSQ6IP+2r/Pjjsx7L+U6z3FPde3WF+afc/+Ais13N4lzlB0/vKO7xf9XyQ83wb4+PF45tI5v+woy3lvG+6Nf4t7fpwdBce1tyCtevPZ4FIIVp1SK76fUj1dh7P+I/dyL7e9ZFt2+Fe13Su/HvK4e0yixvey6LHC8fl3ghOufzN6lkf+v6xV8FznuLOjaj2v3qoK2oGsLruX4vhYngfNasB/QZvAKV2kxOKK9QCf8Q/ygyXj77GeULp0oXd1bz3OliKHrzH+t+ll+SLb+vQ/8tbUNwiiqJNmvBchF9HDDTJSsXeh519botA5zf4Laa3zM08FOiRbKs7P1dQWfQIAy/rVh4rXHZBNi/UNRuZ8/wi+5xktkrmncRzn2aRkmckSGDVGVjrL/ozkmVSoWC1nEE1nUCiNEEnEQuP8PtVdFEhW+9//ja/+/nHcqx1reifw/yLvw/3BDqQvsZZ3w90LpV0gfYXyXb5/edHG/owk+hPhpFCWeFASfSPF9LcB30TXXd12iyqAtjiZL8oP82+9x/8qyKwlkVP+voy6wR51s7BlvMST2GIJrdX/LiX0tQxEaQ409hr/RTcIUQ5nBKPQbY+bYYshgTFGgMWQwpqhfyZANDSrQp67991OvthcMUP9Sv/UIhtX2gj7DD/pEGWdY1aAMMATnhaoalAGG0LwQoUEZYAjNCxEalAGG4LwQgzEFmhciFOUHPfJfzRCaFyL0IQMMv9QLPYJhtS4zwPBLnc0jGFb1IQMMwfmUqj6kz/B+BmAYEvqQAYbQfAqhDxlgCM6nsBdTJGg+hdCHCn2G0HwKoQ8ZYAjOp1TrMgMMwfmUqj5kgCE4n1LVhwwwhOZTCH3IAENoPoXQh/QZyuB8CnsxRYbmUwh9qNFnCM2nEPqQAYbgfEq1LjPAEJxPqepDBhiC8ylVfcgAQ2g+hdCHDDCE5lMIfcgAQ3A+hcGYAs2nVPWhSH+c6D3KwWEosMcQnE+R2GMIzqco7DEE51M09hhC8ylVfcgCQ2g+paoPWWAIzqcwGFOg+RRCH9Ifw6lA8ymEPmSAITifUq3L9Bmq4HxKVR8ywBCcT6nqQwYYQvMphD5kgCE0n0LoQwYYgvMpDMYUaD6F0If0x3Cq0HwKoQ8ZYAjOp1TrMgMMwfmUqj5kgCE4n1LVh/QZ3s8IDENCHzLAEJpPIfQhAwzB+RT2YooGzacQ+pD+GE4Nmk8h9CEDDMH5lGpdZoAhOJ9S1YcMMATnU6r6kAGG0HwKoQ8ZYAjNpxD6kD5DHZxPYS+m6NB8CqEP6Y/h1KH5FEIfMsAQnE+p1mUGGILzKVV9yABDcD6lqg8ZYAjNpxD6kAGG0HwKoQ8ZYAjOpzAXU+SvnQfmEQyZm9fhrWbAYcjcvA6yAM2nEOWQ+lgR+Y4MDkPm3u0sg5ujhSiH1O9JyfDmaGHuHZIyvDlamHtXlQxvjhbm3lUlf+0cLZ9RDulr7K+dceUzyiEDDKH7lK981szupaud4UWb7XjrR/J8N2kd8YTFBDHfW/mT19XkkK+TVRLbUfNnamW62Z+f6Sd4nvgrvI2f5+UrPfuYJ79y9s9hbr1bXr5+FV5unN+vlPeVGF2u9X7lus8P+b76c7fr2n2/29XhS/olyz7IxSw5Hlz/35a3+1wzuX1Y+fm/wXqbQIAsBgc/svOw+PVcPsrU6661w8Eu330gTcI4z9598xAn/CxdCv9x6Wr9Z59HC7cz+Fm83i7lT2otuBl7qi0f/SdEZXAz9hAtH31XB2/GHuaeKpPhzdjD3Oh1Gd6MPcyNXpfBzdhDlEP6rg7cjD1EOWSAIXRnzMDdJ3Az9hDl8AsZfmg/KAgbgtYHTP8SoCL/ClDgSYC89gHBt8SHI/xSXVOZD9eTfc2Tvn0wH64mOKKifB30Ly21FGTQnwFUGQNIQQP9GUCdMYAUBNAfAVR5xgBSUD9/BlBkDCAF6fNnAFkLIhTuCPwZQNaCyP34cAiyFkV4Cn2zj1WCIm2C4AxMtRZTJ0ihX/axWpA6QWh2hBCD1AlC8yOEGqROEJwhYS6SgHMkzEUScJaEuUgCzZMQelCmTJDGsxSP1YPUCYLzJNVaTJ0gNE9C6EHqBKF5EkIPUicIzpMwF0nAeRLmIgk4T8JcJIHmSQg9qNImCM2TEHqQOkFwnqRai2kTpDGK/bF6kDpBaJ6E0IPUCYLzJKxFEhrj1x+rB6kTBOdJmIsk0DwJoQd12gSheRJCD1InCM6TVGsxdYLQPAmhB6kThOZJCD1Im+D9OSw4BFmLJBI4T8JaJJHAeRLWIokEzZNU9aBIewSmBM2TVPUgfYLgPInOGkFonqSqB+kThOZJqnqQPkFwnoS5SALOk7AWSe45Cocga5FEhuZJCD1IewSmDM2TEHqQOkFwnqRai6kThOZJCD1InSA0T0LoQeoEwXkS5iIJOE/CXCQB50mYiyTQPAmhB2mPwLy/8woOwWotpk4QnCep1mLqBKF5EkIPUicIzZMQepA6QXCehLlIAs6TMBdJwHkS5iIJNE9C6EHaIzAVaJ6E0IPUCYLzJNVaTJvg/fhgCBJ6kDpBaJ6E0IPUCYLzJKxFEhWcJ2EtkqjgPAlrkUSDVgY11l5jq0FzdQRB2n3UGjRPQhCk7eo0aIqaIEi7HeTvhZCBue1u89T95ex2t/P8p7PUffwC5NcPvp+j7uOJNkTpIbPUEdPKadUpRfhKDt9O7XWvSia/ncZ/nu86NB9A1BzaTyDo0BUE9TF7OnQFQf0utw5dQVDvF9ahKwjqTorXvrRjuDINUKC5vut++2AaIEeTJZl7DHS+OjU29SmJv1b50oFenUubOvS75Phvhl4t6QJ16F+qVNko6fShCxShf83Ubhpzk8r8jqPI1naKF8OdjX2wgRGELnLetuNHwyQL8zCJ0XYnyfNkhz4Q4Q2G7W5X1/x6X7avP+++oxaFK7xvjk28YWep7+JrDsIzzjnjesjaPZW7p6Blz87tb2Lttiq00nj1TaiHc8Mcn7in9iqpoZ/BZLZuzlZoyW2iXx2jXlvi9PHx1Lh+wOgM6pP5qFuvrbpBbb0NcWItOk1a0QUt9Jto3/q5a3A5yldUAFp4c2PNjedrbiboO6/jrd3drOYsWpzbPhde+5wuQ0N4sXoXe6Efn0up7G9qWbe95rxOTemXer60xienrXNOOwr7O718KbW2bx9m+HjuLGqO5mMpNkWvFBRvfhbb0gIdVz32OQ8J71bYVCfTrV0PjeZ2Mdh1pu3N4GiYs+HONRZ6bW+vGpv9IYwmTxPjuD13+Yu75oJm6Au6uvTc02y1drS5gq5moMe7HWeFQuTEuBAJhpFb+7in+INVfWSbE61/qtVkbtasLVfRatVoNrdoMdmuav1Ws9k1lsvrbq28gcpQC/03Lugv8let0q3Z/dlt67EtxgH6y72o0l9+U3e1rvUNtNhbdmtro2/U2tfFs9FDi0+9LsqFMcq9Rb3bHdVPaLG+qNe6o8lpZIR48Xk0GaHFcFKrZbPRaNVahfVmLWuiRXTAerOZ/e/Y/+zYgzhwW1JHO5uRes0+PthN8qI/0JdPL7jVMRvetMjMOcr3IC7PR74faYkzn2Npbjwpmua4RRFGhdpWs6jhJQ1chR54xo2+L3Xz5w2vJs75rAbOXMtsT1CKaPOCjpS3Bugk16ikt6wnDyWsOfO4GGynQ1RsjY5+1o6ypwSXPRf35UauB0Hk+H5/eEkP5/Owcdo7aE9jjatF9+H5s0FfuxfK54s27cQbMUTC2ljkioRbMl7xFwPN64+5IF5rVl9tcLKYo2vDV7IrZ5cdWsjUnlNkHZwzrhVvJE8Loqm3k3f74TBHiZLsimXHVevjjh8WZmssBs6pN3p4afP5hTCOxwE/FlDz3tKKnSUvxeFY1TA/43wswr0nigcli8vOJbAG6DouLye9L+uBVZRiIAWN0xRdAfpwsRiW51xUY942D7fm9rXZNU76pbQ6u42pczn+XhT4WkFU+NLz8NOuKrd0zdJVSRN8xNZtryR0esWYK8QxOgPcglpTbY1PUOicXXG9MhtcEZuyqKhKU3R0KXXEC6eJqDx6cSR4fXle6odGyYlT7/m5IQXRZvks6i96IWSm9ciarqXyKpTOA36on2PXUj2/OOha5qw03MorVin6SkvDRduKdrjkDSayd4xcOTiKthp0PccTctHu7Xe7ueA/4cqeOahoGvPL5pYfFm7xA82zFujSNkERdfSlgFuCXIs70rJ4eJs0fEqcJMuslpQgqrgqcDx+cM+wB+b1Cngxd9LA7Idy0Pf4UMmjqb/rhfa+JxbT1Ck4HIuy6BKYp7yz2ndwHrqNWiIKzZNZXxexUQaf0UrdrkGed7zWevuivIzEtDjizNiLuJWaDeTDrt9wbEeYZOgMY6SD+CEOm9OV30lRsUoCXKx6zlEwpP1LutvP+7rrL2714/kij0T7JLsPjCjTMg5rM6/5si+4wLy0uERU2hh4XS+KTinMPXUpiKlnYtUg6QdDW6tD6ShvsunLUrRrevDwSGfOx9Zkk0xjVInQhXsLGbPSG1txOmi6QwtByDe+MMGhSNSF41DbvAyQZG6NjlPEsDR7/YdH22m5nygXczLkxdNQrt9ahWOuHUSUi0vV1zS/P0ahsNVvq0dpmAaD6ewyjEz50qppg602mT08onfyWcQfLxMHxRTlKBxexPp0l4n8Jei3+cJEMcRJbBNz6anDSx0XP9XrZD7vW/qj49zGbyqT+Tl043JdXJ7EYbSZy2IWY0qq28LN50gaFkV5LPptQd/yKGx0zmfFwhrSnd+amZGyWYpKgSWlKIWj5oP1VDlU8t5TVwhOvnkwdhpq3zMsGg6pZRb8xnka6GvTKBcD4WXXyhLHPRX9NHieikHaKAPML6htH67vxuI8Tf0Wrt6TzkvEaQISV3GqWpcDasmjW8UfqTNhPuBtaxjKxeUZFaxnZbjBymahnhpF+dAW4Za3/nxizdPDtZanoTrrKENP1LMCZ2kra2ip2DbURee8OV6C6VP7rLbSJa/VRw9Uq6bW47bJdIsuE5WnNxbI2vgeav0HQ/6szNVLrZwVWCz5HTkrDifD8cefUv9LdbvcKSs9jDJdvIwHz5eVi6KhX+RJEU812ZzEwXKx31u6JIWc/5Q8msf1Wl463nRrJYiFuZHFPY6JL4edGLRVpLWiRXmMNN7FbVP0hJUUFr/O6douSbWTkHQ/75x8PVdxrhQREk1CUzLraWG1Sm3ROARYlwevurxcDwZyeW4n3fnDy4sbyuZmKz/J+PHWlj8vr2rSw45UXaAG0Ret6R7H2yLtXVscvMU8IBUrRDZXRI3FamNJj3Zkm7k55fShr6m157qs1f99LEBnNM7GnZrGjepBqa+uPRe98UxuHra91WqFu1bwPyNLbTeM8bpw7yn52QuDUuMk9t82JAfPP/xMTPC+Oe6X0h7Uw6gp6g/5l+4uSXpLedfhxd9Hdb3v8XpL/IQuL4no4WJ09AkCfSit9yvL9ys/d7qufcKYFf5+u/z9oJW/mENK/bgw/PaYlT/M1N+4r1/txyQ7F4n+x7/t6vy4H/PeSbnO8xRdau3WFeafc/+Aysx3N4lzlBc/vKO7xf9XyQ83wbY+PF4xtI5v+woy3lvG+6Nf4t7fpwdBce1tyCtevPZ4FIEVp1SK76fUj1dh7P+I/dyL7e9ZFt2+Fe13Su/HvK4e0yixvey6LHB4GkS0wAnXP5m9SyP/X9cr+C5y3FnQtR/X3lUFbcEd59dCfF+Lk8B5LdWfcCdIkj7qH+d/iB+0GG+36j6hcEF7FE7ifofjhzcaPg8itHEjvwfxS2/W/PePepAE5qD/9496eIPMCvS3ZghOY/FbketLEUIbbStpzCH80hEJD0AosxayBA7aiGWZtQAkcNAevJPZCyfQRn0T2lOhjvBLPRAb2pM+9C/1TGxoT/rQob10hNCe9BFCe8KB0J7UEd5HhoJBSGhP+gihOSBCe9JHCM4BMRdOeGgOiNCeGnWE0BwQoSTpIwTngKoVmT5CcHeBqrqQPkJw94CqupA+QmjuhNCF9BFCcyeELqSO8P7NcBAyF04EaO6kqgtl2s9GC3dicBAKzCEE504k5hCCcycKcwjBuRONOYTQ3ElVFzKAEJo7qepCBhCCcyfshRNo7oTQhbTfJPL2klU4CKsVmT5CcO6kWpHpIwTnTqq6kD5CcO6kqgvpI4TmTghdSB8hNHdC6EL6CMG5E/bCCTR3QuhC6sNARWjuhNCF9BGCcyfVikwd4f0E4CCs6kL6CMG5k6oupI8QmjshdCF9hNDcCaEL6SME507YCyfQ3AmhC6kP0ZSguRNCF9JHCM6dVCsyfYTg3ElVF9JHCM6dVHUhdYT3yVHAICR0IX2E0NwJoQvpIwTnTpgLJzI0d0LoQupDNGVo7oTQhfQRgnMn1YpMHyE4d1LVhfQRgnMnVV1IHyE0d0LoQvoIobkTQhdSR3hvWeAgZC6cKNDcSVUXKtSHaCrQ3ElVFzKAEJw7kZhDCM6dKMwhBOdONOYQQnMnVV3IAEJo7qSqCxlACM6dsBdOoLkTQhdSH6J5n90VDsJqRaaPEJw7qVZk+gjBuZOqLqSPEJw7qepC+gihuRNCF9JHCM2dELqQPkJw7oS9cALOnTA3FYMKzp0wNxWDCs2dEKWQ+oCQe57CQcjcS5k1aO6EKIXUbz9p0NwJUQrpI4TmTqqlkP5LVjRw7oS5F05p4NwJc6+30MC5E+Zeb6FBdyf0HyDTNIIYrfmQv/0HsyHf5lD+mvmQBe3VhvztfMii+BfF4LfnQ77uWjsc7PLdB9IkjPPs3TcPccLP0qXwH5eu1n/2ebRwO4OfxevtUv6kxIEzc8w9vq2DM3PMPSymgzNzzD0spoMzc8wNTdfBmTnmhqbr4MwccwPhdHBmjrmBcDo4M8fcbXcdnJlj7ra7Du1WkyL/ilCSSYS89gHDt8RPgAjNn/wOxC8th+K9OaYyc60n+5onfftg5lpNcERFeRB0lTno0ByNojOHEJqjUXnmEEJzNKrIHEJojkZlLwBBczQqe+EEmqNR2Qsn0BwNoSRV6gjBOZpqRaaPEJyfqVZk6gjvKgsMQkIX0kcIzZ0QupA+QnDuhLlwwoNzJ8yFEx6cO2EvnEBzJ4Qu1KkjhOZOCF1IHyE4d1KtyPQRQnMnhC6kjxCaOyF0IXWE92+Gg5C5cCKAcyfMhZM7MTgImQsnAjR3UtWFMk8dITR3UtWFDCAE50505hBCcydVXcgAQmjupKoLGUAIzp2wF07AuRPmwsm9WsBByFw4EaG5E0IXitQRQnMnhC6kjxCcO6lWZPoIobkTQhfSRwjNnRC6kD5CcO6EvXACzp2wF07AuRP2wgk0d0LoQupDNO+P8sNBWK3I9BGCcyfVikwfITR3QuhC+gihuRNCF9JHCM6dsBdOwLkT9sIJOHfCXjiB5k4IXUh9iKYEzZ0QupA+QnDupFqRqSO8VwswCAldSB8hNHdC6EL6CMG5E+bCiQzOnTAXTmRw7oS9cALNnRC6kPoQTRmaOyF0IX2E4NxJtSLTRwjNnRC6kD5CaO6E0IXUEd7fIAgHIXPhRAHnTpgLJwo4d8JcOFGguZOqLlSoD9FUoLmTqi5kACE4d6IzhxCaO6nqQgYQQnMnVV3IAEJw7oS9cALOnTAXTu4nBAchc+FEheZONOZeh6tCcycEQuq91io0d0IgpG7wVGjupIqQ/jN4qkwQozVP3m3Ou7+cKe92ng+Z8U5UXw3F3854x99mMiIz9J/NeEdMUadVp2ySKzl8O7PXvSqZ/HYaf5Lv0CwVUXWoP6agQrNUBEL6MgKapSIQ0pcR0CwVgZC6jLjPuQMWIX1LpX2pparMjBNoru+63z6YGcfRZOyVHgKdr0yLJdGeLlXUvtSEUYEucMxB/1LbxkZJpz0lq6h9qdFjo6TTh/6lN66ozHamMTe3ivYbvixb2yleDHc2NsMGRhC6yH7bjh8NkyzMwyRG250kz5Md+kCENxi2u11d8+t92b7+vPuOWhSu8L45dvKGnaW+i685CM8454zrIWv3VO6egpY9O7e/ibXbqtBK49U3oR7ODXN84p7aq6SGfgaT2bo5W6Elt4l+dYx6bYnTx8dT4/oBozOoT+ajbr226ga19TbEibXoNGlFF7TQb6J96+euweUoX1EBaOHNjTU3nq+5maDvvI63dnezmrNocW77XHjtc7oMDeHF6l3shX58LqWyv6ll3faa8zo1pV/q+dIan5y2zjntKOzv9PKl1Nq+fZjh47mzqDmaj6XYFL1SULz5WWxLC3Rc9djnPOT/WmFTnUy3dj00mtvFYNeZtjeDo2HOhjvXWOi1vb1qbPaHMJo8TYzj9tzlL+6aC5qhL+jq0nNPs9Xa0eYo042BHu92nBUKkRPjQiQYRm7t457iD1b1kW1OtP6pVpO5WbO2XEWrVaPZ3KLFZLuq9VvNZtdYLq+7tfIGClEt9N+4oL+oErdKt2b3Z7etx7YYB+gv96JKf/lN3dW61jfQYm/Zra2NvlFrXxfPRg8tPvW6KBfGKPcW9W53VD+hxfqiXuuOJqeREeLF59FkhBbDSa2WzUajVWsV1pu1rIkW0QHrzWb2v2P/s2MP4sBtSR3tbEbqNfv4YDfJi/5AXz694FbHbHjTIjPnKN+DuDwf+X6kJc58riGPbTwpmua4RRFGhdpWs6jhJQ1chR54xo2+L3Xz5w2vJs75rAbOXMtsT1CKaPOCjpS3Bugk16ikt6wnDyWsOfO4GGynQ1RsjY5+1o6ypwSXPRf35UauB0Hk+H5/eEkP5/Owcdo7aE9jjatF9+H5s0FfuxfK54s27cQbMUSe2ljkioRbMl7xFwPN64+5IF5rVl9tcLKYo2vDV7IrZ5cdWsjUnlNkHZwzrhVvJE8Loqm3k3f74TBHiZLsimXHVevjjh8WZmssBs6pN3p4afP5hTCOxwE/FlDz3tKKnSUvxeFY1TA/43wswr0nigcli8vOJbAG6DouLye9L+uBVZRiIAWN0xRdAfpwsRiW51xUY942D7fm9rXZNU76pbQ6u42pczn+XhT4WkFU+NLz8NOuKrd0zdJVSRN8xNZtryR0esWYK8QxOgPcglpTbY1PUOicXXG9MhtcEZuyqKhKU3R0KXXEC6eJqDx6cSR4fXle6odGyYlT7/m5IQXRZvks6i96IWSm9ciarqXyKpTOA36on2PXUj2/OOha5qw03MorVin6SkvDRduKdrjkDSayd4xcOTiKthp0PccTctHu7Xe7ueA/4cqeOahoGvPL5pYfFm7xA82zFujSNkERdfSlgFuCXIs70rJ4eJs0fEqcJMuslpQgqrgqcDx+ksWwB+b1Cngxd9LA7Idy0Pf4UMmjqb/rhfa+JxbT1Ck4HIuy6BKYp7yz2ndwHrqNWiIKzZNZXxexUQaf0UrdrkGed7zWevuivIzEtDjizNiLuJWaDeTDrt9wbEeYZOgMY6SD+CEOm9OV30lRsUoCXKx6zlEwpP1LutvP+7rrL2714/kij0T7JLsPjCjTMg5rM6/5si+4wLy0uERU2hh4XS+KTinMPXUpiKlnYtUg6QdDW6tD6ShvsunLUrRrevDwSGfOx9Zkk0xjVInQhXsLGbPSG1txOmi6QwtByDe+MMGhSNSF41DbvAyQZG6NjlPEsDR7/YdH22m5nygXczLkxdNQrt9ahWOuHUSUi0vV1zS/P0ahsNVvq0dpmAaD6ewyjEz50qppg602mT08onfyWcQfLxMHxRTlKBxexPp0l4n8Jei3+cJEMcRJbBNz6anDSx0XP9XrZD7vW/qj49zGbyqT+Tl043JdXJ7EYbSZy2IWY0qq28LN50gaFkV5LPptQd/yKGx0zmfFwhrSnd+amZGyWYpKgSWlKIWj5oP1VDlU8t5TVwhOvnkwdhpq3zMsGg6pZRb8xnka6GvTKBcD4WXXyhLHPRX9NHieikHaKAPML6htH67vxuI8Tf0Wrt6TzkvEaQISV3GqWpcDasmjW8UfqTNhPuBtaxjKxeUZFaxnZbjBymahnhpF+dAW4Za3/nxizdPDtZanoTrrKENP1LMCZ2kra2ip2DbURee8OV6C6VP7rLbSJa/VRw9Uq6bW47bJdIsuE5WnNxbI2vgeav0HQ/6szNVLrZwVWCz5HTkrDifD8cefUv9LdbvcKSs9jDJdvIwHz5eVi6KhX+RJEU812ZzEwXKx31u6JIWc/5Q8msf1Wl463nRrJYiFuZHFPY6JL4edGLRVpLWiRXmMNN7FbVP0hJUUFr/O6douSbWTkHQ/75x8PVdxrhQREk1CUzLraWG1Sm3ROARYlwevurxcDwZyeW4n3fnDy4sbyuZmKz/J+Fmvlj8vr2rSw45UXaAG0Ret6R7H2yLtXVscvMU8IBUrRDZXRI3FamNJj3Zkm7k55fShr6m157qs1f99LEBnNM7GnZrGjepBqa+uPRe98UxuHra91WqFu1bwPyNLbTeM8bpw7yn52QuDUuMk9t82JAfPP/xMTPC+Oe6X0h7Uw6gp6g/51/tG2lvKuw4v/t7x977H6y3xE7q8VKKHi9EhKAj0obTeryzfr/zc6br2GQNXtNcb5n87cEW6DY/4g4Erf5ipv3Fnv9qPSXYuEv2Pf9vV+XE/5r2Tcp3nKbrU2q0rzD/n/gGVme9uEucoL354R3eL/6+SH26CbX14vGJoHd/2FWS8t4z3R7/Evb9PD4Li2tuQV7x47fEoAitOqRTfT6kfr8LY/xH7uRfb37Msun0r2u+U3o95XT2mUWJ72XVZ4PDUYWiBE65/MnuXRv6/rlfwXeS4s6BrP669qwragjvOr4X4vhYngfNaqh9xJ6gyTEnRPuof53+IH7QYbzc0PqFwQRvzIHG/w/HDGw2fBvF+6+O/DOJX3qzRoQ0klwTmEH7pGIZHIJSYQwhtILn0W1HlSxFCG0guacwhhPaYq8xeOIE2JltmL5xAG5MtsxdOoI3JrupClaOOEJw/ERhD+JancBBKzCEE504U5hCCcycacwihuZOqLmQAITR3UtWFDCAE507YCyfQ3AmhCwXqCKG5E0IX0kcIzp1UKzJ9hODcSVUXUkd4f94SDsKqLqSPEJo7IXQhfYTQ3AmhC+kjBOdOmAsnPDR3QuhC2s8USzw0d0LoQvoIwbmTakWmjxCcO6nqQvoIwbmTqi6kjxCaOyF0IXWE92+Gg5C5cCKAcyfMhZM7MTAICV1I+w0ckgDNnRC6kD5CcO6kWpHpIwTnTqq6kD5CcO6kqgvpI4TmTghdSB8hNHdC6EL6CMG5E+bCyf29vWAQErqQ9hBNSYTmTghdSB8hOHdSrcj0EYJzJ1VdSB8hOHdS1YX0EUJzJ4QupI8QmjshdCF9hODcCXvhBJo7qepCjfoQTRGaO6nqQvoI79UCDkKJOYTg3InCHEJw7kRjDiE0d1LVhQwghOZOqrqQAYTg3Al74QSaOyF0IfUhmhI0d0LoQvoIwbmTakWmjxCcO6nqQuoI79OTwEFY1YX0EUJzJ4QupI8QmjshdCF9hODcCXPhRIbmTghdSH2IpgzNnRC6kD5CcO6kWpHpIwTnTqq6kD5CcO6kqgvpI4TmTghdSB3hvVrAQchcOFHAuRPmwokCzZ0QupD6EE0FmjshdCF9hODcSbUi00cIzp1UdSF9hODcSVUX0kcIzZ0QupA+QmjuhNCF9BGCcyfMhRP1S91JZTr7QHN91/32wXT2jibj2zqPgc7axAqS+qV+hg501qZikNQvdUBMlHT6r0dVv9QzMVHSGYAOzWUR5Zb6bTQVmssiSiF9hNBcFlEKqXfaqeBcFnMvzlLBuSzmXtOhgnNZzL2m435CcBAy91CwxhPEaM2H/O0/mA35Nofy18yHLGmvpetv50OWby+CIovBb8+HfN21djjY5bsPpEkY59m7bx7ihJ+lS+E/Ll2t/+zzaOF2Bj+L19ul/EmJ+1LD+BntHv1KC+2WWbXdo/+ghwbOzDH30JsGzswxN8ReA2fmmBtir4Ezc8wN6NPAmTnmBvRp4Mwcc8MHdHBmjrnhAzq0AX2K/CtCRScR8toHDN8SPwEiNH/yOxC/uBxC8yeKyhxCaP7kDRk7CKH5E5VnDiE0f6KKzCGE5k9U9sIJNH+ishdOoPkTlbVw8jbSFQzCqi58Cy/0EILzJypzCMG5E505hNDcSVUXMoAQmjup6kIGEIJzJ+yFE3DuhL1wAs6dsBdOoLkTQheK1BFCcyeELqSO8F4N4CCsVmT6CKG5E0IX0kcIzZ0QupA+QnDuhLlwwoNzJ+yFE3DuhL1wAs2dELpQpo4QmjshdCF9hODcSbUi00cIzZ0QupA6wvs3w0FY1YX0EYJzJ8yFkzsxOAiZCycCOHfCXDgRoLkTQheq1BFCcyeELqSPEJw7qVZk+gihuRNCF9JHCM2dELqQPkJw7oS5cHLPUzgImQsnIjh3wlw4EaG5E0IXUh+iKUJzJ4QupI8QnDupVmT6CKG5E0IX0kcIzZ0QupA+QnDuhL1wAs6dsBdOwLkT5sLJ/d0GYBBWdaFGfYimBM2dVHUhAwjBuROdOYTQ3ElVFzKAEJo7qepCBhCCcyfshRNw7oS9cALOnbAXTqC5E0IXUh+iKUFzJ4QupI7wnqdwEFYrMn2E0NwJoQvpI4TmTghdSB8hOHfCXDiRwbkT9sIJOHfCXjiB5k4IXUh9iKYMzZ0QupA+QnDupFqR6SOE5k4IXUgd4f2NvHAQVnUhfYTg3Alz4UQB506YCycKOHfCXDhRvtSdVOYM9WRf86RvH8wZqgmOqCiPga6x9gJdWYHmZ6oI6b9fSYHmZwiE1C2hAs3PEAjpt54aQYzWPHm3Oe/+cqa823k+ZMY7WXm1IH874514m8mIzNB/NuMdMUWdVp0wUa/k8O3MXveqZPLbafxJvkMzYUTVof5gw/0E4CKkLiNUaCasipD+7XIVmgkjEFKXESo0E0YgpC4jVJomLNBc33U/MmGOJksy9xjoovArdIX2dKmy+qUmjA50iTnoX2rbmCjpKu0pWWX1S40eEyWdAehfequLSscaz7E2r5Ks/oYxy9Z2ihfDnY3dsIEZhC7y37bjR8MkC/MwidF2J8nzZIc+EOENhu1uV9cMe1+4rz/vvqMWhSu8b46tvGFnqe/iaw7CM84643rI2j2Vu6egZc/O7W9i7bYqtNJ49U2oh3PDHJ+4p/YqqaGfwWS2bs5WaMltol8do15b4vTx8dS4fsDoDOqT+ahbr626QW29DXFiLTpNWtEFLfSbaN/6uWtwOcpYVAJaeHNjzY3na24m6Duv463d3azmLFqc2z4XXvucLkNDeLF6F3uhH59Lqexvalm3vea8Tk3pl3q+tMYnp61zTjsK+zu9fCm1tm8fZvh47ixqjuZjKTZFrxQUb34W29ICHVc99jkPtUqtsKlOplu7HhrN7WKw60zbm8HRMGfDnWss9NreXjU2+0MYTZ4mxnF77vIXd80FzdAXdHXpuafZau1ocxRUjIEe73acFQqRE+NCJBhGbu3jnuIPVvWRbU60/qlWk7lZs7ZcRatVo9ncosVku6r1W81m11gur7u18gYStC3037igv0gktEq3Zvdnt63HthgH6C/3okp/+U3d1brWN9Bib9mtrY2+UWtfF89GDy0+9booF8Yo9xb1bndUP6HF+qJe644mp5ER4sXn0WSEFsNJrZbNRqNVaxXWm7WsiRbRAevNZva/Y/+zYw/iwG1JHe1sRuo1+/hgN8mL/kBfPr3gVsdseNMiM+co34O4PB/5fqQlznyOp0E2nhRNc9yiCKNCbatZ1PCSBq5CDzzjRt+XuvnzhlcT53xWA2euZbYnKEW0eUFHylsDdJJrVNJb1pOHEtaceVwMttMhKrZGRz9rR9lTgsuei/tyI9eDIHJ8vz+8pIfzedg47R20p7HG1aL78PzZoK/dC+XzRZt24o0YokbYWOSKhFsyXvEXA83rj7kgXmtWX21wspija8NXsitnlx1ayNSeU2QdnDOuFW8kTwuiqbeTd/vhMEeJkuyKZcdV6+OOHxZmaywGzqk3enhp8/mFMI7HAT8WUPPe0oqdJS/F4VjVMD/jfCzCvSeKByWLy84lsAboOi4vJ70v64FVlGIgBY3TFF0B+nCxGJbnXFRj3jYPt+b2tdk1TvqltDq7jalzOf5eFPhaQVT40vPw064qt3TN0lVJE3zE1m2vJHR6xZgrxDE6A9yCWlNtjU9Q6Jxdcb0yG1wRm7KoqEpTdHQpdcQLp4moPHpxJHh9eV7qh0bJiVPv+bkhBdFm+SzqL3ohZKb1yJqupfIqlM4DfqifY9dSPb846FrmrDTcyitWKfpKS8NF24p2uOQNJrJ3jFw5OIq2GnQ9xxNy0e7td7u54D/hyp45qGga88vmlh8WbvEDzbMW6NI2QRF19KWAW4JcizvSsnh4mzR8Spwky6yWlCCquCpwPB4natgD83oFvJg7aWD2Qznoe3yo5NHU3/VCe98Ti2nqFByORVl0CcxT3lntOzgP3UYtEYXmyayvi9gog89opW7XIM87Xmu9fVFeRmJaHHFm7EXcSs0G8mHXbzi2I0wydIYx0kH8EIfN6crvpKhYJQEuVj3nKBjS/iXd7ed93fUXt/rxfJFHon2S3QdGlGkZh7WZ13zZF1xgXlpcIiptDLyuF0WnFOaeuhTE1DOxapD0g6Gt1aF0lDfZ9GUp2jU9eHikM+dja7JJpjGqROjCvYWMWemNrTgdNN2hhSDkG1+Y4FAk6sJxqG1eBkgyt0bHKWJYmr3+w6PttNxPlIs5GfLiaSjXb63CMdcOIsrFpeprmt8fo1DY6rfVozRMg8F0dhlGpnxp1bTBVpvMHh7RO/ks4o+XiYNiinIUDi9ifbrLRP4S9Nt8YaIY4iS2ibn01OGljouf6nUyn/ct/dFxbuM3lcn8HLpxuS4uT+Iw2sxlMYsxJdVt4eZzJA2LojwW/bagb3kUNjrns2JhDenOb83MSNksRaXAklKUwlHzwXqqHCp576krBCffPBg7DbXvGRYNh9QyC37jPA30tWmUi4HwsmtlieOein4aPE/FIG2UAeYX1LYP13djcZ6mfgtX70nnJeI0AYmrOFWtywG15NGt4o/UmTAf8LY1DOXi8owK1rMy3GBls1BPjaJ8aItwy1t/PrHm6eFay9NQnXWUoSfqWYGztJU1tFRsG+qic94cL8H0qX1WW+mS1+qjB6pVU+tx22S6RZeJytMbC2RtfA+1/oMhf1bm6qVWzgoslvyOnBWHk+H440+p/6W6Xe6UlR5GmS5exoPny8pF0dAv8qSIp5psTuJgudjvLV2SQs5/Sh7N43otLx1vurUSxMLcyOIex8SXw04M2irSWtGiPEYa7+K2KXrCSgqLX+d0bZek2klIup93Tr6eqzhXigiJJqEpmfW0sFqltmgcAqzLg1ddXq4HA7k8t5Pu/OHlxQ1lc7OVn64jqVv+vLyqSQ87UnWBGkRftKZ7HG+LtHdtcfAW84BUrBDZXBE1FquNJT3akW3m5pTTh76m1p7rslb/97EAndE4G3dqGjeqB6W+uvZc9MYzuXnY9larFe5awf+MLLXdMMbrwr2n5GcvDEqNk9h/25AcPP/wMzHB++a4X0p7UBejpqg/5F+6u3ThLeVdhxevyWSP11vi47u87n367I9BQaAPpfV+Zfl+5edO17XPGLmivX7yb0euKLdXmv/ByJU/zNTfuLVf7cckOxeJ/se/7er8uB/z3km5zvMUXWrt1hXmn3P/gMrMdzeJc5QXP7yju8X/V8kPN8G2PjxeMbSOb/sKMt5bxvujX+Le36cHQXHtbcgrXrz2eBSBFadUiu+n1I9XYez/iP3ci+3vWRbdvhXtd0rvx7yuHtMosb3suixweGIOtICnREZ/MnuXRv6/rlfwXeS4s6BrP669qwragnvOr4X4vhYngfNaqh/RQV4Zp6QLH/WP8z/ED1oMgfu0TnIN2qAHifsdjh/eaPg8iNCGPfwexK+8WaNBezJWEphDCG0kuSQxhxDaSHLpt6LKlyKENpJc0phDCO3JWJm9cAJtULbMXDi5j9+Ag5C5cKJ/6aBsKoOtCCUpUYf+pY6GDvRqY0EfOjgHVG0s6CME54Cq2pM+QnAOqKo96SOE5oAI7UkfITQHRGhP+gjBOSD2wsmXOiA2tKdCGbpyP6H/ZujVxoI+dGiPvhLakz7CL/VMj0BY1Z70EYJzQFXtSR8hNAdEaE/6CKE5IEJ70kcIzgGxF06gOSBCSdJ+OFrhoDkgQhfSRwjtHhChC6kjvL8dAw7Cqi6kjxCcO6nqQvoIobkTQhfSRwjNnRC6kD5CcO6EvXACzZ1UdSHP0X6XiMJDsydVYcgCQ3D+RGKPITiDorDHEJxD0ZhjeP9mMAyr4pAFhtA8SlUdssAQnElhL6YI0FwKqQ9pjzlVBGg2hdSH9BmC8ylEXabPEJxPIfQhfYbgfAqhD+kzhOZTSH1InyE0n0LqQ+oM769HhcOQvZgiQvMppD6kPS5UEaH5FFIf0mcIzqcQdZk+Q3A+hdCH9BmC8ymEPqTPEJpPIfUhfYbQfAqpD+kzBOdTGIwp0HwKqQ+pj928ZyIchkRdps8QnE8h6jJ9huB8CqEP6TME51MIfUifITSfQupD+gyh+RRSH9JnCM6nMBhToPkUUh9SH8MpQfMppD6kzxCcTyHqMnWG94la4TAk9CF9huB8CqEP6TOE5lNIfUifITSfQupD+gzB+RQGYwo0n0LoQ576GE4Zmk8h9CEDDMH5lGpdZoAhOJ9S1YcMMATnU6r6kD7De66CYUjoQwYYQvMphD5kgCE4n8JeTFHA+RTWpodQFHA2hbXpIRQFmkshSiH1gSIKNJNClEL6CKF5FKIUUr8dpUCzKEQppI8QmkMhSiH1Ti8VnEFh7gVVKjh/wt67MFRw/oS9d2Go0P0JA8+YqRKBjNYUyt/+gwmUb9Muf80Uyor66kT+dgpl9fbGJbIc/PYUytdda4eDXb77QJqEcZ69++YhTvhZvN7eX1ktXq3/cAe0cDuHnwXs7WL+pMyBc3TsPeetgrN07D1PpoLzdOw9T6aCM3XsjVtXwbk69sat388IDkP2xsdp4Hwde+PjNHC+jr378Bo4X8fefXgN2o2ntxHO924ukUTIax8wfEv8BIjQjMrvQPzicgjNp7x1ELODEJpNUXTmEEJzKSrPHEJoJkUVWUN4rxZwEDIXTnRoFkVlLpzo0ByKylw40b/UoFTmvfVkX/Okbx/Me6sJjqgon6QkZerQwTmaatWnjxCcn6lWffoIofkZQknSRwjNzxBKkj5CcH6GvXACzs+wFk7UeycnHISshROVg+ZnCF2oUkcIzc8QupA+Qmg3XAhdSB8hNHdC6EL6CKG5E0IX0kcIzp2wF07AuRP2wgk4d8JeOIHmTghdqNNGeI9vcBBWKzJ9hODcSbUi00cIzZ0QupA+QmjuhNCF9BGCcyfshRNw7oS9cALOnbAXTqC5k6ou5DmeOkNo9qQqDFlgCM6f6MwxvH8zGIZVacgCQ2gOpaoNWWAIzqKwF1MEcB6FvZgigDMpDMYUaC6F1Ie0R3eqAjSbQupD+gzB+RSiLtNnCM2nkPqQPkNoPoXUh9QZ3k8ADkP2YooIzqewF1NEcD6FvZgiQvMppD6kPmZThOZTSH1InyE4n0LUZfoMofkUUh/SZwjNp5D6kD5DcD6FwZgCzqewF1Pu78KDw5C9mCJB8ymkPqQ+dlOC5lNIfUifITifQtRl+gyh+RRSH9JnCM2nkPqQPkNwPoXBmALOpzAYU8D5FAZjCjSfQupD6mM472cEhyFRl+kzBOdTiLpMnyE0n0LqQ/oMofkUUh/SZwjOpzAYU8D5FAZjCjifwmBMgeZTNNZetKvK0GwKgZB6L7YMzaUQCKkbvfusEHARUm8LFZ4gRmtqvds0eX85ud7tPB8ySZ6qvJaTv50kT75Ne0Rm6D+bJI+Y1E4jJtYRK1l8O7XX3Sq5/HYef5Lx0IxVte4w8CCDAs1YkQypKwkFmrEiGVKXEgo0Y0UypK8loBkrkiF9MfGlxqryyvlAc33X/fbBK+cdTZZk7jHU+epc8bQnt1SVL7VidKBXZ5enD/1LzRsbJZ329JlvvXH/zdCrJZ0+9C+9iUVlGhGNuZeWq6Q5m+R+ilJIu46uO/+V8K+w4iT2K2Rfk+woXMVo1UWcfJRuYIqha0e11w270POuhv+jLK10AqCNYY4Batwn3Zz8NZOED9TORzMvfmImke7vNZPIGvJ/NJPkD+zlF2cSaS9fM0n/XybdVtUPvNcXZxLpX+/NHfe/XHr98EfdXQ/KJrR6SJL8fQ8juub1c+L5+BP/Dw== -------------------------------------------------------------------------------- /assets/filtered-image.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/filtered-image.png -------------------------------------------------------------------------------- /assets/lenet5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/lenet5.png -------------------------------------------------------------------------------- /assets/lr-scheduler.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/lr-scheduler.png -------------------------------------------------------------------------------- /assets/mlp-mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/mlp-mnist.png -------------------------------------------------------------------------------- /assets/mlp-mnist.xml: -------------------------------------------------------------------------------- 1 | 7VvbcqM4EP0aP8bFVciPuTmzWzO1U5WHzTxtySAbbbDEgIjt/fqVQLLNJQ7Z2EbObFViUINAnO7TanXDyL1drh8ylMbfWISTkWNF65F7N3KcADriVwo2lcDzQCVYZCSqRPZO8Ej+wUpoKWlBIpzXTuSMJZykdWHIKMUhr8lQlrFV/bQ5S+p3TdECtwSPIUra0j9JxONKCp1gJ/+CySLWd7bBpDqyRPpk9SR5jCK22hO59yP3NmOMV3vL9S1OJHYal6rf9JWj24FlmPI+HZ7I5GH62xf+u/vj2/V89YczvQdX0FWD4xv9xDgSAKgmZVRsbjJW0AjL61iixTIeswWjKPnKWCqEthD+jTnfKPWhgjMhivkyUUfxmvCnvf0f8lJjX7Xu1urKZWOjG5Rnm6f9xl4v2dx1K1u7ftG1VLxohgnKcxJWwilJ9HDa0Ck0c1ZkIT6Al6uUyVG2wPzQieqKEs29WyjVPGC2xGLQ4oQMJ4iTl7q5IWW1i+15O8WKHaXbd+hZj/sFJYW6VUvxdS2vYsLxY4pKOFaC23WNzgWatyxhWdnXjRCGcwl0zjP2jPeOgBDi2fwQ6C8443h9ECR11NVcUs7EsVR7taOmp0TxHis9fd7RcfXbMF46fz5ADu2g3ySHYxY5tCGeihw+hpHXRQ7ozFwAjkMOz2uQw+tHji2pjk8OcyeXDxi509fIXbOM3PkMRu6DhpHDfka+FR4fV/e0uM5hiMPOmXUGfc8/aMz9cQWwjuvWKbyBqw1OhasHB3Me9kHX8V/n1QjlcTlSux6kqifZRagf9E9eX//km+WfvLd5JC4j1n34bQ6hPK0Wg3OylpgfJfh0G67HblPE6XI9J/M8vuGISe9UQ8wfGDFvYmxAYrpXARfqVYDhHGl5lY6o/awc0Rz9nyPv5khwoRwJLo0jwdDziN8C6DSEuGQ6eL2zQUeng+r6nREx6F04YjXWOG7DPqqRql4NE9kO4wNW0yPNZBTP/KEjXF0rGmIuumjq9c5RmTUT6XFfDkOGjta8wFCGnMvSj55kfWXuaC5lm3mvU88dPbKMZjFj6BjNb6/8AujJXjQteN5CTzwnr8NUT7gqw93PzioRSsiCimYo0MNCfiNRIyFKrtWBJYmi5DW91Lk5Z5Rr8nZU+D9cMQVuWy2d+fKTqaW92CjTQhbFRcbor6KXZrFueL3All5s69fTS7O+NLxeJh16EW1W8F/IjTXLU4OrRZuJsfPxdv7VActk4PkYmL76bSLWVQI9L2Kmr4ZaiA1d3wGmR8lNxLwOP3ZexEyvurYQA0MjZnrVtYXY4J6/HcKYhRiYNPzY0KwMTI8uWogNzcrAdM/fRMwbOroITPf8LcSGji4C0z1/M0npDZ2DC0x/+6KJmN/xQcF5ETO9Ft9CbPC50vToolU66Hhl+ayIQdOjiyZi7tB1aPj53hs86VdysPebUEpjhtSfYQ/vm8colbtkWX45elNurzULrC5K6ETqVzTDyXeWE06YTKjOGOds2ZFp5dJgbtQd7mLO5ceu1/LZnClec5wJq7oKGeVCM+OoCJ/l/4KNQ3E1Z0qKEoVpse0rvLTo7cv+4sf9iX+mmQNC9ExsENE4sif5C5htwMvVKsV0QSgeU8wjiq7yPKmuKvqtUn3PslmkCUNRXu47lj0pdyyn3ORomSb4r/IJhJKttTOB45QuRg4QR2SCuLQx3aJsPnvL6N6xAG5mPicd3sMd61dD6x7EOZVpddRX1g4Uf7JnqemmpX3OVL7t2mMAaxo6YTZfNHdfWFfF+N1n6u79vw== -------------------------------------------------------------------------------- /assets/multiple-channel-mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/multiple-channel-mnist.png -------------------------------------------------------------------------------- /assets/multiple-channel-mnist.xml: -------------------------------------------------------------------------------- 1 | 5Z1db1vXEUV/jR8D3G+e+xi7ifOQAEadoshTQYu0xIQWBYqO7f76XkW8+piRgxaJufdKn2xRFCkt8nBmkWfOfta+ePfx5X55dfHDbrXePmuq1cdn7d+eNU3dVu30z80ln24vWXTHC873m9XxSvcXvN78e328sDpe+n6zWl8/uuJht9seNlePLzzbXV6uzw6PLlvu97sPj6/2drd9fK9Xy/N1uuD12XKbL/3nZnW4uL20NIv7y79bb84v5nuuh/H2O++W85WPf8n1xXK1+/DgovabZ+2L/W53uP3fu48v1tsbeDOX25/79jPfvfvF9uvLw3/zA69+Gq/G/t2nl3///sfN19Xw9sXPP31VV+3xhn5dbt8f/+b+hufFcgL6289fXr0/3NzT5uN6e338Ww6fZkCH9cfp7p9fHN5tpwvq6b/Xh/3ul/WL3Xa3ny653F1O13z+drPdhouW28355fTl2fQHrKfLn/+63h82E/qvj994t1mtbu7m+YeLzWH9+mp5dnOfH6Yn2nTZfvf+crW++duq6avV8vrity9ufoG3u8vD8ZlUl+nrTGr+m6c7XH98cNGR3Mv17t36sP80XeX+iXz7I/PT+Pjlh/vnRD0/ZS8ePB/mB395fBqe393y/SM1/ef4YD39wP3r1e7Hf7xbvbzevP72+fqb8+bV9XdftfO9PXjc0mPzmNFTFB88bL+L8OHD96xpV/26rLr0WE/fKc2bdhjuoCfCTzwOn4XeLh5Dr/tMvTkt9LxYviD0L4HwiSfuiRE2dISjHGELR3hX3XUIOzrCVo6wpyPUl5OBjlBfThZ0hPpyUmgIR7u+cKQjlC/k+Za5COULucHZSUCo7wtnYlyE8r6wwdlJRCgvJw3OTiJCfTnB2UlEqC8nNDvp6lCR9QuZZicJoX4h0+wkIdQvZJqdRIT6vnDuqrgI5X3hEx80whDKy0lLs5OEUF5OWpqdJITyctLS7KQLH97r+8KWZicJoX4h4+wkItQvZJydBIQGfSHOTiJCfV+Is5OIUF5OOpydRITyctLh7CQilJeTDmcnfajI+oWMs5OIUL+QcXYSEeoXMs5OAkJ9X9jh7CQilPeFHc5OIkJ9OcHZSUSoLyc4O4kI5eVk/oU4CO3mTnqcndjNnfQ4O7GbO+lxdmI3d9Lj7MRu7qTH2Ynd3EmPsxO7uZMeZyd2cyc9zk7s5k56nJ3YzZ0MODuxmzsZcHZiN3cy4OzEbu5kwNmJ3dzJgLMTu7mTAWcndnMnA81Oeru5k4FmJwmhfiHT7CQh1C9kmp1EhPq+cF4GXITyvnBBs5OEUF5OFjQ7SQjl5WRBs5OEUF5OFrm1fr25PJ/oNdXbzfbmDMmm2r0//HZcZUCLOaXyD86QD48esyee9WN30ofspH1oOKTybTlbn52lh3r6zpvS32yT+FOYl9B1NfJlctLG1YJ5J2dO63QjwUFNsNAa3UiwyAnS+txAsKnkBGltbiQorz2F1uVGgvJKUmhvwUeC+kpy0nfgJf1PXYXio1/4J33PXnIafYKuf62gvcufEOpfLGjv8ieE+r6T5j4Rob7xnN9a4yKUF6CRZj8JobycjDT9SQjl5WSk+c9d52jTSY40AUoI9QuZtgcpIdQvZNoepIRQ3heOODsJCA36QpydRIT6coKzk4hQXU66eRlwEarLSVfh7KQL5US9kLsKZycRoX4h4+wkItQvZJydRITqvrCrcHYSEMr7wq7C2UlEqC8nODuJCPXlBGcnEaG+nODsZHDrC0+bg/4lEMoXMi7VPCGUL2RcqnlCKO8LcanmEaG+L8SlmieE+nKCs5OIUF9OcHYSEerLCc5O3MZgOlyqeUKoX8g4O3Eba+nmW+YilPeFDc5O3AZbupkYF6G8nOBSzRNCeTnBpZonhPpyQrOTxm1SpcOlmieE+oVMs5OEUL+QaXaSEOr7QpqdRIT6vhCXap4QyssJLtU8IZSXE1yqeUIoLye4VPM7FjZ9IS7VPCHUL2ScnbjNnXS4VPOEUN4X4lLNI0KDvhBnJ25zJx0u1TwhlJcTXKp5QigvJ7hU88Zu7gSXap4Q6hcyzk7s5k5wqeYJobwvxKWaR4T6vhCXap4Q6ssJzk7s5k5wqeYJob6c0OwkfhIvrya4UPN0WJIeIU1O0rkCeoQ0OUkjeHqENDlJu9X1CGlykjZ26RHi5MSvnODkxK+c4OVEj/Cvn74S86fVCW/daVPQLZirI+E6XGy6W/B8h0tNd8ud73Ch6W6x8x0uM90tdb7DRaa7hc53uMR0t8z57rSB6ZrEmxhurV/4ymjLEyXeuOXad7hQ9oRQ/2JB++gnIZT3nbhQ9ohQ33jiQtkTQnkBwoWyJ4TycoILZU8I5eVkQfOfu87RppN8ItcehlC/kGmf/SSE+oVM++wnIdT3hTg7ae36QpydRITycjJ/BMpFKC8nBWcnEaG8nBScnfShnOgXMs5OIkL9QsbZSUSoX8g4O4kI5X1hwdlJQKjvC3lR9xGhvpzg7CQi1JcTnJ1EhPJywou6X4RyIl/IvKj7iFC+kHlR9xGhfiHj7CQilPeFvKj7gFDfF/Ki7iNCfTnB2UlEqC8nODuJCPXlBGcndmMwvKh7t6mWnhd17zbW0vOi7t3mWnpe1L3bYEvPi7p3m2zpeVH3bqMtPS/q3m22pcdF3Tdukyo9Luo+IdQvZJqdJIT6hUyzk4RQ3hfiou4jQn1fiIu6Twjl5QQXdZ8QyssJLuo+IZSXE1zUfeM2d9Ljou4TQv1CxtmJ29xJj4u6Twj1fSHOTtzmTnpc1H1CKC8n8y1zEcrLSYOzE7e5k34mxkHoNnfS46LuE0L9QsbZidvcSc+LunebO+l5Ufducyc9L+rebe6k50Xdu82d9Lyoe7e5kx4XdR8/iZdXE1zSfTosSY+QJifpXAE9QpqcpBE8PUKanKTd6nqENDlJG7v0CHFy4ldOcHLiV07wcqJHeFI50Zw+HsKu5JHa/fwL/D9BV6em9h3OgCJCdTxW3+EMKObcqUO4+w5nQBFhI0eIM6CIUF+AcAYUEerLCc2A7CKk+04ZwnKiNqgJNVy/9JURlydKvonQ9S8WtI+AEkL5i8VszliE+t6zxxlQRCjvPXucAUWE8gLU4wwoIpSXkx5nQBGhvpzgDCiEpes7yR5nQBGhfiHTPgNKCPULmfYZUERo0Bfi7CQilPeF8ysJF6G8nAw4O4kI5eVkwNlJRCgvJwPOToZQkfULGWcnEaF+IePsJCLUL2ScnQSE+r5wwNlJRKjvC3F2EhHqywnOTiJCeTnhRd5HhPJywou8L6EiyxcyL/I+ItQvZJydRIT6hYyzk4BQ3xfyIu8jQnlfyIu8jwj15QRnJxGhvpzg7CQi1JcTmp00dtMwuMj7hFC+kHGR9wmhfCHjIu8jQn1fiIu8TwjlfSEu8j4h1JcTmp0khPpyQrOThFBfTmh2csfCpy+k2UlCqF/IODuxmzvBRd5HhPq+EBd5nxDK+0Jc5H1CKC8nuMj7hFBeTnCR9wmhvpzg7MRu7gQXeZ8Q6hcyzk7s5k5wkfcRoUFfiLMTt7mTARd5nxCqy8mAi7xPCNXlZMBF3ieE6nIy4CLvG7e5kwEXeZ8Q6hcyzk7c5k4GXuS929zJwIu8d5s7GXiR925zJ0OFsxO3uZOBF3nvNncy4CLv4zGTrbyc4CLv42lJBghpdhIPFjBASLOTOINngJBmJ3G7ugFCnJ34lROcnfiVE5yd+JUTup3oEc63/Fc+fjykXsmztYfmpD7jAV0dnzrMjDEvFgmhOidraHAGFAPv1GncQ4MzoIhQncY9NDgDigj1BYhmQAmhvpzQDMguS3polCksJ2qDWrve86TOpIm+idDlLxbzL8R5sYgI5S8WLe4zoJgULO89W5wBRYTy3rPFGVBEKC9ALc6AIkJ9OcEZkFtq+tDiDCikpus7yRZnQBGhfiHTPgNKCPULmfYZUESo7wvnz/W4COV9YYezk4hQXk46nJ1EhPJy0uHsJCKUl5MOZyeLUJH1CxlnJxGhfiHj7CQi1C9knJ0EhAZ9Ic5OIkJ9X4izk4hQXk54mfcRobyc8DLvI0J5OeFl3o+hIusXMs5OIkL9QsbZSUSoX8g4OwkI9X0hL/M+IpT3hbzM+4hQX05wdhIR6ssJzk4iQnk5wWXeN3bTMLjM+4RQvpBxmfcJoX4h0+wkItT3hbjM+4RQ3hfiMu8TQn05odlJQqgvJzQ7SQj15YRmJ43dpAou8z4hlC9kXOZ9QihfyLjM+4hQ3xfiMu8TQnlfiMu8Twjl5QSXeZ8Q6ssJzk7s5k5wmfeN3dwJLvM+IdQvZJyd2M2d8DLv7eZOeJn3dnMnvMx7u7kTXua93dwJL/Pebu6El3lvN3fCy7y3mzvhZd7bzZ3wMu/t5k54mfd2cye8zHu7uRNe5r3d3Akv895u7gSXeR+PmWzl5QSXeR9PSzJASLOTeLCAAUKancQZPAOENDuJ29UNEOLsxK+c4OzEr5zg7MStnCx4mfdu5aTg9lrbnWtdcHut7c61Lri91nbnWhfcXmu7c60Lbq+13bnWBbfX2u5c64Lba213onDB7bW2O1G44PZa250oXHB7re1OFC64vdZ2JwoX3F5ruxOFC26vtd1ZrgW319ruLNeC22ttd5Zrwe21tjvLteD2Wtud5Vpwe63tznItuL3WdqdoFtxea7tTNAtur7XdKZoFt9fa7hTNgttrbXeKZsHttbY7RbPw9lq7nV9YeHut3c4vLLy91m7nFxbcXmu78wsLbq+13fmFBbfX2u78woLba213clzB7bW2Ozmu8PZa2+3swu+1NqjIODtx29k18vZau+3sGiucnbjt7BornJ247ewaK5yduO3sGiucnbjt7BornJ247ewaK5yduO3sGiuanaTt6vpyQrOTtL9Qj5BmJ3bJtOPcEnAQuoW4jTXOTtzyTsYaZyd25aTG24ke4Unt5O1mu32x2+72v91wu+rXZdVNl18f9rtf1g++U5o37TD8Sa+ezWPo9SCHTvOZhLDIEdJ8JiJsKjlCnM9EhI0cIc5nIsJOjhDnMxGhvJzMt8xFKC8nDc1n6s6tqZmJcRHqn4U0n4kI9U1NQ/u0JSGUNzUNzk4iQnlT0+DsJCLUlxOcnUSE+nKCs5PBrqnB2UlEKH8Wzu8aYxHqm5oWZycRobypaXF2EhHKm5oWZycRobyctDg7iQj15QRnJ8WtqWlxdhIR6p+FODspdk0Nzk4iQn1Tg7OTiFDe1My/ABehvJx0ODuJCOXlpKPZyV35sGlqOpqdJIT6ZyHNTiJCfVPT0ewkIZQ3NR3NThJCfVNDs5OEUF9OaHaSEOrLCc1O7ljYNDXzdnEuQvmzsKfZSUSob2p6nJ3YbRTucXZit1G4x9mJ3UbhHmcndhuFe5yd2G0U7nF2YrdRuMfZid1G4R5nJ3YbhedXEi5CeVMz4OzEbqPwgLMTu43CA85O7DYKDzg7sdsojEuXjwj1TQ0uXT4h1Dc1ODux2yiMS5dPCPXlBGcndhuFeeny4c2uVl5OeOnynR1Cmp3EHf8GCGl2EvcXGiDE2UllhxBnJ37lBGcnfuWEbicGCE9qJ5oTD93CkUZeHr1bONLIy6N3C0caeXn0buFIIy+P3i4ciZdHbxeOxMujtzs7m5dHb5cvxcujt8uX4uXR2+VL8fLo7fKlcHn0CaG8qcHl0SeE8nKCy6P3C0fC5dHHcCR9U4PLo08I9c9CnJ3EiC55U4PLo08I5U0NLo8+IdQ3NTg7iQj15QRnJ3Zxhbg8+hhXKG9q6goXSJ8Yqp+GE0Ocn8TYTHVbMzHECUpkqO5rJoY4Q4kM1Y3NxBCnKJGhQU3BOYpbjPDEkCYpMUfYobehWUpiaPA8pGlKyrM26G1onpIY6nsbXDR9YqjvbfjZ9Pqawgunjwz1NYWXTu+2bXhiSPOUxNDgeUjzlMjQoLfBxc0nhga9Dc5T3LYOTwxxnuK2d3hiiPMUt83DdTXfNIeh2+7hiSHOU9y2D08McZ7itn94YojzFLcNxBNDnKe47SCeGOI8xW0L8cQQ5ylue4gnhjhPcdtEPDHEeYrbLuKJIc5T3LYR1xUufD4x1Pc2uPT5xFDf2+Di5xNDfU3B5c8nhvqawgugD+99tQY1heYpcU7PgSHNU+JYgANDmqfE/YcODHGeUvsxxHmKX03BxdDH920cGNI9xYHhST0lHI74tpytz86ePXE44pvSd331J1Wh5jF1eXbIRP2kZuNBXX0+/EQd50KBoTxvZGKIc6HIsNEzxLlQZKhOHJkY4lwoMjSoQjQXSgwNagrNhexCR+rqtKH3mm7ILZR3on5Se/Kgrn+96E/qW1+AoUEP2tM+F0oM9T1oj3MhtyjfiSHOhdyyfCeGOBdyC/OdGOJcyC1mZ2KIcyG3ROSJIc6F3CKR62r+jbAMDfrDgfa5UGKo7w8HnKe4pSJPDHGe4haLPDHEeYpbLvLEEOcpbhmCE0Ocp4RMWoP+cMB5SmRosJZxnlL8+kOcp0SG+v5w3snEZajvDxc4T4kM9TVlgfOUyFBfUxY4TwkMDfrDBc1T7uqwT3+4oHlKYmiwlmmeEhka9IcLmqckhgb9Ic1TEkOD/pDmKYmhvqbMr8hchvqaUmieEhka9IeF5il3MHz6w0LzlMTQYC3jPMVvBqbgPMVvBqbgPMVvBqbgPMVvBqbgPMVvBqbgPMVvBmY+KYPD0G+iZcR5it98yojzFL/5lBHnKX7zKSPOU/zmU0acp/jNp4w4T/GbTxlxnuI3nzLiPMVvPmXEeYrdfEo9rwQsQ31/WFc4T7GbT6krnKfYzafUFc5T7OZT6grnKXbzKXWF8xS7+ZS6wnmK3XxKXeE8xW4+pa5wnmI3n1LP+WwPkK1X5+vXxy93+8PF7nx3udx+c3/p88dQ76/z/W53daT38/pw+HTEt3x/2D0GfXufN3f0v5Lcr7fLw+bXxz/3FJfjj77abaZbvHsE2pjk3Qa017v3+7P18afu6eYbiqUt3tBhuT9fH9INfb3fLz89uNrVzRWuf+cX/kwc3Wd/r9+//vSf29/g/jlzB/cPLUWc7tqNOdXzU5PLUK8ZNU537cacpppAZ6hvTWqc7tqNOdU1TXfTcacGNYWmu+m4LgOGNN1NR1oYMKTpbhr7NGCI0904GmHAEOcpfjVlvmkOQ7+a0uA8xa+mzMg4DE9XU6Yv97vd4eHbPNOfe/HDbrW+ucZ/AA== -------------------------------------------------------------------------------- /assets/multiple-filter-mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/multiple-filter-mnist.png -------------------------------------------------------------------------------- /assets/multiple-filter-mnist.xml: -------------------------------------------------------------------------------- 1 | 7Z1bV+rYtkZ/zWrtnIdaLeSeR66iIqAgoi+nAUkgGAhCCJdfv2cQXDqHq461t5rx7ZVqVSVEufVkzvH1acz4oZVn27PlYDG5ilwv/KEq7vaHVvmhqgW1oIsv6Zbd8xZTs543jJeBe/yhXxs6wd47blSOW9eB663e/GAcRWEcLN5uHEXzuTeK32wbLJfR5u2P+VH49lUXg7FHNnRGg5BuvQvcePK81VatX9vrXjCenF65YDrP35kNTj98/CSrycCNNq82adUfWnkZRfHzrdm27IUpvBOX58fVfvPdlze29ObxRx7QvncWjjHbnd00ukFRMf3y9P6vgmMZx7cX706f2XMFguPdeTQXX0rLaD13vfSZFHEvWsaTaBzNB2EjihZiY0FsnHpxvDvuwME6jsSmSTwLj9/1tkHcf3X7Pn2qn8bxXmV7fObDnd3pzjxe7vqv79y/vvPrQYd7vx7lFtMdL+6OwsFqFYyeN9aC8PRm/GgeH99pwRb3nwmkH/u3bI+bVtF6OfL+DqiiHo/4eLAce/Hf/ahuOC/HgRhAXjTzxAcRD1164SAOkrfvZnA8kscvP/drZ4sbx/39T/b9y1tNBuH6+GLkYBCH7SK9GcwOI6V0+FpcLZ4HWwp8cLrjB9v0ECkl3jIOxBBqDIZe2I5WQRxEc/H9YRTH0ezVDxTDYJx+I04PotLxFSqTOE4HdzH9dGrN28beUhxpf4kRHovd8dNdjx7T/8bRz5F4NrUWrA8cauuXx6pG+mgjfbz4n/bkPS2WqjkaPAYF051P3IKzSszhzkz+2iy8+TiYez/nXuzOB3+tVuHzs4rHbRan1zzcXS/CaOCuDrdVpeAcbijq4ctqMFuE3v8dPsFfmqJsVcf+uZiPf6im+E56xB2OvNO9eeQPj4ficQcIIN727w8+epgcH6AfJ5jdac493t/8mq/E+/2pHQf65NV0pSrK1x1cDjm4zg97WD7CxCeP304Wq3gZPXrlKIyWv6YgX4xfadPgePyMBClv+c6BNQtcN32Z0mYSxF5nMTiM3Y0oVWROcweryeHO7yeI/2gvFZS3u8mge8lW3ttDX7aDNIXsoLYYweHqD91Dms5uDxXIHhJVTIAUGFTlf8TslERh4rn/m37OP3nXGQ6zXacV/sGuC/7gadE2NGZ77vQOvjoQf11M/Q8C5fGh7ShIU8/L8NLeDi+B/6eiW2bBOf3/7TM+R9/jk7xWEul5dYc879sneo7b5IkOe/bl4/1Hw1T9/xPw21393pB5tVu/ery8jIa/TXvvjpivGzKqDUbR+EBmLrw377xs/AKK7+Sx/wKK5nsUzS+cvrV8+pamWVtSD7Xg/HTUX9N3wfj3pm/L+vembwFusHv1Y4v0B1ZfM8Ebxn/hoFK/ORGZaAxNfgwtNIYfChrfyxAtZpgKP4Z0VY45Q5UdQxMtqJn8aopJ10eYM+RXU0w0eTX51RRTA2NI8qGePcMP/BqRF0N5LDNgCOcp8lhmwBDNU0g+ZMAQzVNIPmTAEM5TGNYUOE/hV1NOy3M4DPnVFAvNU0g+NLNniOYpJB8yYAjnKfJYZsAQzVNIPmTAEM1TSD5kwBDOUxjWFDhPYVhT4DyFYU1B8xSSD+3MGZ7eAQ5DeSwzYAjnKfJYZsAQzVNIPmTAEM1TSD5kwBDOUxjWFDhPYVhT4DyFYU1B8xQ5H76cN54hQzRPkfMhB4ZwnmKzY3j6axUYhnI+5MAQzVPkfMiBIZyn8KspDpyn8KspDpynMKwpaJ5C8mH253A6aJ5C8iEDhnCeIo9lBgzRPIXkQwYM0TyF5MPMGeqnvYrDkF1N0RU4T2FXU3QFzlPY1RRdQfMUkg8zP4dTV9A8heRDBgzhPEUeywwYonkKyYcMGKJ5CsmHDBjCeQrDmgLnKfxqyukazzgM+dWUd64zyJshyYeZn8Opw10EjuRDBgzhPEUeywwYonkKyYcMGKJ5CsmHDBjCeQrDmgLnKQxrCpynMKwpaJ7ysQvcfivD01PDMNTYXXdOV9E8RZP/LoABQzRPkTsqcGCI5ikau+vO6R9pG8SLIcOaguYpOsOaguYpOsOaguYpJB9mvwYL1/6A5EMGDNE8heTD7BnCNZAg+ZABQzhPYXfdOV2D8xR+NUVD8xSSDxkwRPMUkg8ZMETzFJIPs1+D1dA8heRDBgzRPIXkQwYM4TyF3XXndA3OU9hdd04/7VUchvxqio7mKSQfMmCI5ikkHzJgiOYpJB9mfo0gXUfzFJIPGTBE8xSSDxkwhPMUdted03U4T2F33Tldh/MUhjUFzVNIPsyeoYHmKSQfMmCI5ilyPsz+GkG6geYpcj7kwBDNU+R8yIEhnKewu+6cDtePXs6HHBjCeQrDmoLmKXI+5MAQzVPkfMiBIZqnkHyY/TmccP3oST5kwBDNU0g+ZMAQzlP4XXcOrh89yYcMGMJ5CsOaguYpJB8yYIjmKSQfMmCI5ikkH2Z/DidcP3qSDxkwRPMUkg+zZwjXj57kQwYM4TyF33Xn4PrRk3zIgCGap5B8yIAhmqeQfMiAIZqnkHyY/TmccP3oST5kwBDNU0g+ZMAQzlP4XXcOrh89yYfZM4TrR0/yIQOGaJ5C8iEDhmieQvIhA4ZonkLyYfbncML1oyf5kAFDNE8h+ZABQzhPkfMhA4ZwniLnQwYM4TyFYU1B8xSSD7NnCNePnuRDBgzRPEXOh1r253DC9aOX8yEHhmieIudDDgzhPEXnxxDOU0x+DOE8hWFNQfMUOR9yYIjmKXI+zJ6hAdc/RWfXc8E4PTUOQ3Y9FwwVzVPIcZj5uSLGCRkOQ3bXdjbg+qeQ4zDz30kZcP1TyHHIgCGap5DjMPO1LwOvfwq7a1UZcP1T5OMw+78BN+D6p8jHIQeG6J6iZp+x4fqnkOOQAUN0T8n+nHYDrn8KOQ4ZMET3lOzPnTPw+qewO5fYgOufQo7D7DM2XP8UchwyYIjuKQx+FwDXP0U+DjkwRPMUg998CNc/xeA3H8L1TzHYncNpwPVPMdmdw2nA9U8x2Z3DacD1TzEZ1hQ0TzEZ1hQ0TzEZ1hQ0T7HkvhXZr8HC9U8hDLNfP4Trn0IYZr/2Bdc/hTDMfj6E658iM2Twu1G4/imEYfY1Ba5/CmGYfU2B659CGDKoKWieQhgyqClonvKSJE51WaMMX2r1a4gvG7+CIpqpfIii+R5F8wspornKRyh+93hGcxXDYscQroeK4fBjiOYqZoEfQzRXMTV+DNFcxeRXU+B6qJgMawqaq5gMawq8qxjZM4QzFXksM2AI5ynyWGbAEM1TSD7MniFcDxWSDxkwhPMUfjUFrocKyYcMGMJ5Cr+aAtdDheRDK3uGaJ5C8iEDhnCeIo9lBgzRPIXkQwYM0TyF5EMGDOE8hV9NgeuhQvIhA4ZwnsKvpsD1UCH50MmeIZqnkHzIgCGcp8hjmQFDNE8h+ZABQzRPIfmQAUM4T2FYU+A8hWFNgfMUfjUFroeKnA/VQvYM0TxFzoccGMJ5isOPIZqnyPmQA0M0T5HzIQeGcJ7CsKbAeQrDmgLnKQxrCpqnkHyY/TmcDpqnkHyYOcOXKxfhMJTHMgOGaJ5C8iEDhmieQvIhA4ZwnsKuppgKnKcwrClwnsKwpqB5CsmHmZ/DaSponkLyIQOGcJ4ij2UGDNE8heTD7Bme3hEOQzkfMmAI5yn8akoBzlP41ZQCnKfwqykFNE8h+TDzczjNApqnkHzIgCGcp8hjmQFDNE8h+ZABQzRPIfmQAUM4T+FXU05PjcOQX01R4TyFX005IYNhSPJh5udwmnA96Uk+ZMAQzlPkscyAIZqnkHzIgCGap5B8yIAhnKcwrClwnsKwpsB5Cr+aAteT3mJ3TV0Tric9YZj9OjZcT3rCMHvXg+tJTxgymA/RPEVmmP057SZcT3rCkEFNQfMUwpBBTUHzFMKQQU1B8xTCkEFNQfMUuZ/6e/0yv7sfkgnflf5dit/cD8nE60v/AYrfPJ7h+tIbNj+GaK5isushbOL1pVf5MURzlZc6yIghmquYDGsKmquYDGsKnKswrCnwrpJ5D+GX3zbiMJTHMgOGaJ5C8iEDhmieQvIhA4ZwniLnQwYM4TyFYU2B8xSGNQXNU0g+ZMAQzVNIPrSzZ4jmKSQfMmCI5ikkH2bPEK4nPcmHDBjCeYqcDxkwhPMUfjUFryc9v5oC15Oe5EMGDNE8Rc6HqpI9QzRPkfMhB4ZoniLnQw4M0TxFzoccGMJ5is6OIV5Pen41Ba8nPb+aAteTXs6HHBiieQrJh2r2DNE8heRDBgzRPIXkQwYM0TyF5EMGDOE8Rc6HDBjCeQrDmgLnKfxqClxPepIPGTBE8xSSD7M/hxOvJ708lhkwRPMUkg8ZMETzFJIPGTCE8xQ5HzJgCOcpDGsKnKcwrClonkLyIQOGaJ5C8mH253Di9aSXxzIDhmieQvIhA4ZonkLyIQOGcJ4i50MGDOE8hWFNgfMUhjUFzVNIPmTAEM1TSD7M/hxOvJ708lhmwBDNU0g+zJyhBdeTnuRDBgzhPEXOhwwYwnkKu5pi4fWkZ1dTLLie9CQfMmCI5ilyPtQyP4fTwutJb/NjiOYpcj7kwBDNU+R8yIEhnKfo7Bji9aTnV1PwetLzqylwPenlfMiBIZqnWPI1MTL/Hb0F15OeMMx8HduC60lPGGbvenA96WWG2f+9ngXXk54wzPycdguuJz1hyKCmoHkKYZh9TTk9NS7D7GuKiuYpMkMGGfuEDIah3E/9vX6Z390PyYLvSv8uxW/uh2Th9aX/AMXvHs9ormI4/BiiuYrJroewhdeXXuPHEM1VXrptMGKI5iomv5oC15fe5FdT8r70n8AQ3lUy7yFswfWlJ/mQAUM0TyH5kAFDNE8h+ZABQzhPkfMhA4ZwnsKwpsB5CsOaguYpJB9mzxCvK708lp3sGaJ5CsmHDBiieQrJhwwYonkKyYcMGMJ5ipwPGTCE8xSGNQXOUxjWFDRPIfmQAUM0T5HzoVrIniGap8j5kAFDuJ70cj7kwBDNU+R8yIEhnKcY/BjCeQq/moLXk55hTUHzFDkfcmCI5ikkH2rZM0TzFJIPGTBE8xSSDxkwRPMUkg+zZ4jXk17OhwwYwnkKv5qC15OeX02B60lP8iEDhmieQvJh9udw4vWkl8cyA4ZonkLyIQOGaJ5C8iEDhnCeIudDBgzhPIVfTcHrSc+vpsD1pCf5kAFDNE8h+TD7czjxetLLY5kBQzRPIfmQAUM0TyH5kAFDOE+R8yEDhnCewrCmwHkKw5qC5ikkH2bPEK4nPcmH2Z/DideTXh7LDBiieQrJhwwYonkKyYcMGMJ5ipwPGTCE8xSGNQXOUxjWFDRPIfmQAUM0T5HzoZb9OZx4PekddgzhetLL+ZADQzRPkfMhB4ZwnmLwYwjnKfxqCl5PeoY1Bc1T5HzIgSGap1j8rqkL15OeMMx+HRuuJz1hmL3rwfWklxlm//d6NlxPesIw83Pabbie9IRh5jXFhutJTxhmXlNsuJ70hGHmNcXG60nPbs3BxutJL/W+fa9f5nf3Q7Lhu9K/S/Gb+yHZcH3pP0Lxu8czmquYCj+GaK5iquwY4vWl1/kxRHMV0+THEM1VTH41Ba8vPb+akvel/wSG8K6SeQ9hG64vPcmHDBiieQrJhwwYwnmKnA8ZMITzFDkfZs/w9NQ4DPnVFBXNU0g+ZMAQzVNIPmTAEM1T5HyoKtkzRPMUOR9yYIjmKXI+5MAQzlN0fgzhPMXkxxDOUxjWFDRPkfMhA4ZwPenlfMiBIZqnkHyoZs8QzVNIPmTAEM1TSD5kwBDOU+R8yIAhnKfI+ZABQzhPYVhT0DyF5EMGDNE8heRDBgzRPIXkQz1zhnA96Uk+ZMAQzVNIPmTAEM5T5HzIgCGcp8j5kAFDOE9hWFPQPIXkQwYM0TyF5EMGDNE8heTD7M/hhOtJT/IhA4ZonkLyYfYM8XrSy/mQAUM4T5HzIQOGcJ7Cr6bA9aQn+ZABQzRPIfmQAUM0TyH5MPtzOOF60pN8yIAhmqeQfMiAIZynyPmQAUM4T5HzYfYM8XrS86spcD3pST5kwBDNU0g+ZMAQzVPkfKhlfw4nXE96OR9yYIjmKXI+5MAQzlN0fgzhPMXkxxDOUxjWFDRPkfMhA4ZwPenlfMiBIZqnkHyY/TmccD3pST5kwBDNU0g+ZMAQzlPkfMiAIZynyPmQAUM4T2FYU9A8heRDBgzRPIXkQwYM4TyF3zV14XrSE4bZr2PD9aSXGTK4FgFcT3rCMPv5EK4nPWGY/TntcD3pCUMGNQXNUwhDBjUFzVNkhgzWD/F60rNbc9A+8rdmq8lgkd4MZoOx+FpKGQSjQdgYDL2wHa2COIjm4vvDKI6jmfiBMP1GaTB6HB/wl6MwWh6eSvMP/7x6jmIYjNPHxlG6LwarhTdKP7QfbNMdUTq8ZPG0VTltEbfdQTz4oRWf76q1xXz8Qy0HvVLrZqNcno2jovin2bmdVG/H4taoKv5XL5WL9+n2m/WmcviBUr1Z7vSuz8vF8blfnDwG6cZiuOnUwr240aiKx5a35yUlFjtWHAG19NuViXLTmyi3qjNz6+5kNLstDu9qyuhsm7hn28V9UFIf+hf7wZ2zvtrpu8a0uDo/myhuvWg2dk5837/ZDM8cZXgWBo2Zs3vY2WfeYHmbvt7oNqxe9270eUtzd6rp9rbamX4nXtdaNxRXHO61oGp1uo+DclCqPt41Z/Xu2bS5LrVu27NR6c4pPg3GlenTMgg7l53S+nF7XtiPJopfDTzVse7d0eZ2PBnaPTH5lJrOfDZT+oEaDufpUaSWSnH/aX5hes1x+XrQ6tiNTbFoKLfV4v04HI8r1eqjuBk9jouNWrV6Xrq/PzysFldELqmJ/0p78VUcT7XdqDho3D5/d32mzX3xVXmw9N8+0/l4UmyUxM2L+/PipNQoFc8ON7elC3Hz8uJc7IUbsffuyufn1+WNuFm+KxfPrzub61KQ3ry67lyLm0GnWFzdXl+Pa+OgXC2uquKmeMFytbrKX/ufvXZz7o9qet3etkLrsPsK/qwTJ42mc3/5kE47rYrbTVatntjv/ny3XRcaoR0Nez3bFlsuTdsejpIkCBPrzFqFFTeqpEPoE99xpeHp5/HVtGBFw+3W8oc9ezVwVTMJpw/ileJaU7zJiTjSa/1LV2yYKK31XfOx2xaHbanubO214Zr+/kmZN4xK7Ph+OPS8Rnu/WG637crmaSgeWZqkw+L80/fPVDztk7q72tvd+nyqBY64fxebejqTFUzvrmm7jRvFn0/sfsOqKIYWi8+WfpLZ7nY/EzdW1sUwWdXTPTPqz6e6a/th150Zs6d2OxYbdWOk7eojq3xT94KkVbvR/OHm4vrTjzavcKfezG/8wo0qpveancz6xr3WvrHslF9pu06CJ1fTluZqvqvv/X5TfI79w8ZpGI7fT3aar/uVTVd8AvHDyV17t401a14YtJbP0+1x2i1tnP2uX59NW44Sp88rKl/NDxNPv2p/2aeK+47ddyzdVj3BdnQ21sXbS26URLsR7yCdQftde5K+QbW+HWmTcauiJPOWoZmWWdWGjr4YanvF1sTx6M5D1W0YvZ2zrOwUreteXVV0P5zeX2nOg5Ooq1b/M0e6vTDGgb5tFtrOdj7qW66XLB17NRzb6Sxv9neaZ9bs9NDuh7P0yGt2DHcdjgx/rQ0s/9wdumqsDS6eZrOe6l2mg301FIdmqbefPu+Pfjrj+7bbvxMfbeonYd25V9OZILbndf0++fQ5qX0ZDaPVql/TI0E1HQpKIf0TntKg2Tp8goIWDxd+qxEYfsMtBGYcdr3ZRTB4utCS7mKYKGktWoV7v7WJ6+OneroPR5VipKnVTas8Sealnf8Vs9TzZzB6dbc2eXwwH661RbJOd8aTls5St01jOWtUhoOh2lmJdzgXOajQTstmd+zVF+Kwivz0sLoYrtWS/vSwmD31Gs7Iu3seH1d741obbIzRJ1aU7m4eFG/d6sNTovitfU2JNPMsBV52kqS+U3uuda9qC7eVpgbdWZbsidXW18Z01X241wZFx//0Stfq3fQ706g7F4NIfHD3zkhZOZVHrdusjtp9ASGeemonLUWao67b9vShKTJz7XrdFQx3rYvGp1fb7u6pY+5bnXZB27SN8vOssI7tpSb24r3l2bbXuBGlsNY4s9Z6e+E3u7f7dtgy9rWi3Xy0O7efXtHr8W1YWO87Q1FTzLW6fNDK3dlKK+z9xlkhaYkaMowGrZTLhdXel9PDz3LrK6/g9Z3PrnNTr2p2ettgNN9Nkv2l1g6nPUNbzVNK1qiWTp/XejtJduukcaY6jwVRNurbrdlPM+So9zzNXJvTe81M0kip6cF19ZPz1K5txheX56q/8VrL0swW8/sqDQ3LRb+VFKbDy6YzaZV2d031YVZbRcPRJmks/Kuu5i8qOz/l5xcfPz3f3Wi9xcKrpcO7U38IFVsV4Wq+sPr7pZjJw+eBf23dqr1mYdBvB0ayvxIH1pXZnqbJ5s7aVJLdp84Iz/vW63X6vcXyMMoXgXVbN9uu5qySdJfWVhV7oZ2VrLv6drre+93Ls61VW9wX7PL1J6bVln2hPEbdR/ExxfH0wkKojeeK2b/ZLmzNnrUv7m6TNCx5dWOVLDeloXfzJeN/Zz3ez8yxE4QrR9vfNK/245Gohl4SR8m8axutzty/v3t66ju6HijeZfTZPA6f5aHudh/7kWDRmhraU1oTH5YzzT+zRNYK73br0C6M0rkpvEyTVBp+h5vDvKQXN2p0/nXvyXNiK90rSShCk1rVW+VF0q/t7LvK0k9zuX/M5btJs2nstmfRee/Tj5dRYLSmj8alkf6BW83r7Q5p0k2N1LoTE6Kn9btPab1NFheHGSf9TmspUqwaDpQkrNyNp339s41s2mt1Faft2VbxqmzY5b+vBeId3axu6kVbuS77O2d8WLm4uLk1qsvHi/F4nC6tpP+WVovBKJin99XTSsmvVRixdR7NvZdvREvXW/7a+AnLWy9NUE+/uiuYPw2ywFU4Ldu9bVZtfdUSl/OR0wzzJa58iStf4vrDXjtf4sqXuPIlrnyJK1/iype48iWufIkrX+LKl7jyJa58iStf4sqXuH63xCX9lURB5bHE9YGzq/MlrnyJK1/i+sNeO1/iype48iWufIkrX+LKl7jyJa58iStf4sqXuPIlrnyJK1/iype4frfEpUlLXBqPJa4P/PF7vsSVL3HlS1x/2GvnS1z5Ele+xJUvceVLXPkSV77ElS9x5Utc+RJXvsSVL3HlS1z5Etfvlrjka4zqPJa4PnBtwnyJK1/iype4/rDXzpe48iWufIkrX+LKl7jyJa58iStf4sqXuPIlrnyJK1/iype48iWu3y1xGdISl/GlS1zpTB1F8avvnS0Hi8lVlM6SWvVf -------------------------------------------------------------------------------- /assets/relu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/relu.png -------------------------------------------------------------------------------- /assets/resnet-blocks.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/resnet-blocks.png -------------------------------------------------------------------------------- /assets/resnet-pad.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/resnet-pad.png -------------------------------------------------------------------------------- /assets/resnet-pad.xml: -------------------------------------------------------------------------------- 1 | 3Z1Rb5swEMc/TR4nBWwDfVyzrtO0SZsyadrTRMEFNhJHjtMk+/Qjw4RgmmmdhM76PxUOY+CX6/n+Pa6ZscXqcK/TTflR5bKehfP8MGNvZmEYBOFN8+NkObaWaM5bQ6Gr3A7qDcvql7TGubXuqlxuBwONUrWpNkNjptZrmZmBLdVa7YfDHlU9vOomLeTIsMzSemz9WuWmbK1JGPf2d7Iqyu7KQWQfeJV2g+2TbMs0V/sLE7ubsYVWyrRbq8NC1id4HZf2vLdXjp5vTMu1+ZcTPqvoexHmO1EfVl+C9w/3evftlZ3lKa139oHtzZpjR0Cr3TqXp0nmM3a7Lysjl5s0Ox3dN595YyvNqm72gmbzsarrhaqV/nMuy4VMct7Yt0arn/LiSBI+sChqjtgbkNrIw9UnC868GkeTaiWNPjZD7AncEj4Od/f959WZyouPqrOl1kOK87w9xGbDcnwB03Bapo9JJrPsOaYPieBiPgnThJgpA/TTICSGygEdNYiIoQpATw3nxFAjQE8lX6diRE+lXqgSQE9l1AvVDQLUxK8stRNYUFCpf/uDifUUCVTyPDWAUFQuVepENZhYU5FQJc9UAwhR5VIlX6wmVlU0VMlXKwhZ5VAlT1YDCF11XvR9SVchhJVLlTwCTKysaHyVPGHtro/lrOQZazixuqLxVvKUtaOI5a3kK1YIUbMaYaVeskIIgeViJc9aQwiFdV6iPMlaQwiF5VIlDwETKywaqvRZK4bEcrGSZ60Q1SsXK3nWyjA0louVesViEBWsEVbqJYthaCwHK3nWyiA01jmWepK1MgiJ5VIlDwEQbwa6VMmzVgYhsUZYqbNWBlHFcrHSZ60QGmuElXzFgqhjjbBSL1ndxFjeSp61cgiN5VvbFYeQWL41XnGItwT9a73CkFi+NV9xiCqWd+1XHENj+daAxSHqWN61YHEMjeVbExbH0FietWEJDInlWR+WgHhT0LtGLIEhsXzrxBIYVSzfWrEEhsbyrRdLYNSxfGvGEhgay7duLAGhsZhn3VgCQmK5VMlDAMSbgi5V8qw1CUYUZV7Ipd1V2pSqUOu0vuutt0PO/ZgPSm0s3R/SmKP9j4Xpzqgh+/aapwu9lKOWdWqqp+F5z1Gxp35SVTNjz/9aHtZNsVU7nUl7Vs92NBG/9spnN5FJdSHNaKLXWqfHi2Gb04Dt9RvmVxqirt7X38c3G+0d9B5zhvv/TtSxAPvdpJY+EUQp1MVKLn0iCKE+wkqd9kQQxdARVuq8J4IQ6i5WcukTQQh17tsfQaOJhToRVvJMAOKFUxcrfSYwsVInwkqeCUAUQ4VvUj2eWGURYaWOrTFEOdTFSh5b44lVFhFW6tgaQ5RDhW/t/fHEKosIK3lshVBZLlb62AqhskZYyWMrhMqKfGtCjSFU1ggreWyFUFkuVvLY2olpMKzTxdZmt/+etrai2H/bHbv7DQ== -------------------------------------------------------------------------------- /assets/resnet-skip.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/resnet-skip.png -------------------------------------------------------------------------------- /assets/resnet-table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/resnet-table.png -------------------------------------------------------------------------------- /assets/single-filter.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/single-filter.png -------------------------------------------------------------------------------- /assets/single-filter.xml: -------------------------------------------------------------------------------- 1 | 3VjBcpswFPwaXzqTGSRhjI+2k7SH9NJkUufUkZEMagWiQsS4X18JhEHYTpNOWju52Hr7JKG3uwwPRmiRVh8lzpPPglA+gh6pRuhyBOEUevrXANsG8INJA8SSkQYCHXDLflEL2nVxyQgtnIlKCK5Y7oKRyDIaKQfDUoqNO20tuHvVHMd0D7iNMN9HvzKikgYN4aTDP1EWJ+2VQTBtMiluJ9tKigQTselB6GqEFlII1YzSakG54a7lpVl3fSS7O5ikmXrOgvJ+eXd1f4dnwbK8Cb/cr/MKXNhdHjEvbcGbuppvNlGobcuEFGVGqNnMG6H5JmGK3uY4MtmNll5jiUq5joAeElwk9VwTrBnnC8GFrDdC6zCiUaTxQknxg/Yyq3Dsj83ua5Ep6wUQ6NiekkpFq6Plgx2p2oxUpFTJrZ5iF/hWBuvDcSvLpqeqhZKeoC2GrY/i3cYd1Xpg2X4B8/AY8/C9MQ/gmVGPjlGP3hv1A9MH3omZ948x77835oemPzn14z2GKdFPOhtmItN/c5d0IVUiYpFhfiNEbtn9TpXaWrJwqYQrBK2YWtrlZvzQG19W/WDbBpmubtkPHvpBt6iOulVkZp7u3dE1cs0MJXW+KdZU+LR+mhBRyoj++RGpsIypemLe9LAfJOVYsUf3HK+ubvBf1QUvVxecqbrwLag7Odm9C/7i3gXnoy56C+qGJ7t3n6kuOFN1/beg7nSv5ZG0KLn6dw0PGdOQ+IcanhCuUBAMGp7wdRqeXefSdvnBiRue9gvCAea9OtT5QEdovpKOEsHP0ryT1yRdFDVLMz0BhHlVU9Xm9Sg2/zmrKDcvzN4HU+7u/VnvPzcU1mnopuEgjdw0GqR9N63DAKfGEtmqyOujePvQwF9aWuWayPWHvVv7ZrIQ5izOdBhpK1CNz41RWIT5zCZSRoi5zEHXur52jfv6NvS9gQ2n+zYEkwM+hC/3oQ67zzd1rvcNDF39Bg== -------------------------------------------------------------------------------- /assets/single-pool.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/single-pool.png -------------------------------------------------------------------------------- /assets/single-pool.xml: -------------------------------------------------------------------------------- 1 | 3VjRkpowFP0aZ9qHzghBxEd13e3D9qXuWPepE8kV0gbChLBiv74JBAHR7dqxdfVFc89NQnLOyeRCD02j/EHgJPzCCbCe3Sd5D931bHvo2epXA9sSQKNhCQSCkhKyamBOf4EB+wbNKIG01VFyziRN2qDP4xh82cKwEHzT7rbmrP3UBAfQAeY+Zl30GyUyLFHPHtb4Z6BBWD3ZckdlJsJVZ7OTNMSEbxoQmvXQVHAuy1aUT4Fp7ipeynH3R7K7hQmI5VsGZIvl02zxhMfuMnv0vi7WSW59MrO8YJaZDSc0B/bd4KncVkQInsUE9Fz9HppsQiphnmBfZzdKeYWFMmIqslST4DQs+upgTRmbcsZFMRFaez74vsJTKfhPaGRW3sAZ6NnXPJbGCpar4u5Wq3WDkJA3ILP1B+ARSLFVXUzWMSoYGw4qVTYNUQ0UNvSsMGxsFOwmrplWDUP2CcTbR4i3b414y35nzKMjzKNbY37P8m7/wsQ7R4h3bo34fctfnPlBh2Eg6pYzYcxj9Tdpk86FDHnAY8weOU8Muz9Ayq0hC2eSt4WAnMqlGa7bz432Xd4MtlUQq90tm8FzM6gHFVE9ioz1zV4vXSH3VFNS5I8qmPJM+PDn21BiEYB8pZ+54jWJr/pBAMOSvrRLibOr6/5Xda3T1bXeh7r2Vao7vNjZtf7i7FoXUxddpbrexc7uG9W13oe6zlWqO+pUPALSjMl/V/CQAXjEOVTwePYKue5eweOdp+DZVS5Vje9euOCpvh4cYL5fhCrvqghNVkK1Alnw4OJIUx2vUv0X4fzD7mV4up8tpzgbtHv5m3YXtns9OZpzPnYcpZSTbdu0HWHOZ9M+BsKMBrEKfeUHUPhE+4D6mI1NIqKE6Mcc9GnbyW2rnt94Tn/PeKOu8azhAefZpztPhfXHmiLX+OKFZr8B -------------------------------------------------------------------------------- /assets/subsample-mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/subsample-mnist.png -------------------------------------------------------------------------------- /assets/subsample-mnist.xml: -------------------------------------------------------------------------------- 1 | 7Z1Zd6LKG+4/Ta91zkVnMch0WSBOMVEc4nDzXwiIKAItKOqnP4XRdJqy09mnE6ve7J3VnYgCFr8a3ucpiqpvorHe1zd2sniIXS/8JnDu/ptY/SYIvCZJ+E/xzuH8Ds+f3/E3gXt+7+cb/eDond/kzu9uA9dLf9kxi+MwC5Jf33TiKPKc7Jf37M0mzn/dbR6Hv35rYvse8UbfsUPy3VHgZovnd1VB+fl+wwv8xeWbeVl7/mRtX3Y+X0m6sN04f/WWaH4TjU0cZ8+v1nvDCwt6Fy7Px9V+8+lLwjZelL3nALuV+GvdDZer3soLpad1v7b9fj5Lmh0uF+y5+PrPm/EmW8R+HNmh+fNdfRNvI9crzsrhrZ/7tOM4wW/y+M2ll2WHc2ba2yzGby2ydXj+1NsH2fjV60lxqjvpvFXdn8982jhcNqJscxi/3pi83vh50GnrchTJ6HLB8XbjnK+4O9ESTVof6r32IECcPDeWk+88J1TOxc3e+F721q4V6ZznBbtXX3POh7oXrz2cKrzDxgvtLNj9WrbscxH1X/b7mYv4xTkjr2fqH9K+s8Pt+cuIjMblMSleButTFdBPf1GaPNeigp592ZgH+yK/9Z23yQJcN9r2zAu7cRpkQRzhz2dxlsXrVzugMPCLD7KiROjnb6gusqyotai4OqHm7TNvg4vNd1x1M5w/d+7WWRX//fjOwWcTasH2xKG2fTlWkIqjpeJ4/Ev84f1INoLs2KuAl91o4fJaupNnB3n3PU+8yA8i7y7yMjeyv6dp+HxWfFyeXL7ztLlNwth209NrgeO10wtOOP1J7XUSev87XcF3keP2gqbeJZH/TZDxJ/ja5qdyfNmK4vnsXLB/W/QKRt7+zWJy+VQ+Nx3nxlPgztv5z5YIJ/hOPDepi1cN0cu+n1G6NKJ0NU9ZXC5i+DqzX6t+mm3ilWfEYbzB70RxVJS8eRCGpbfscwFyMD1vc6VkrQPXPbVG+SLIvH5in2pzjqMQ0UK5dro4bRQJmOOMPzdMvPox2YRZ38kK9/NH+CXX+AqZayp3Lcc+LcNEjsiwLq7SYfovzbFKqWKxkEU8mUVY52AKAvd/0u3suSFy/29x6f/mrFM41rJO5N+fdcG/uJlUFf5OYi7vhD8LpV8xXQP5Kuc+veni3qMJrkL8NIoVnhQEn0jxdT34Johz1fEch6g0+JOZKlWkN23AR3O/ZdmtCGRU/3LUBfaok6094y1GhT2G4Frddzmx2zIUoTFU2WP4jm4SphhKDEYhCRpDBmOKDI0hgzFFuSVDNjSoQJ+6+vWpl9sLBqjf1G99BMNye0Gf4ZU+UcYZljUoAwzBeaGyBmWAITQvRGhQBhhC80KEBmWAITgvxGBMgeaFCEV5pUf+1gyheSFCHzLA8KZe6CMYlusyAwxv6mw+gmFZHzLAEJxPKetD+gwvKQDDkNCHDDCE5lMIfcgAQ3A+hb2YUoHmUwh9KNNnCM2nEPqQAYbgfEq5LjPAEJxPKetDBhiC8yllfcgAQ2g+hdCHDDCE5lMIfUifoQTOp7AXUyRoPoXQhyp9htB8CqEPGWAIzqeU6zIDDMH5lLI+ZIAhOJ9S1ocMMITmUwh9yABDaD6F0IcMMATnUxiMKdB8SlkfivTHiV6iHByGAnsMwfmUCnsMwfkUmT2G4HyKyh5DaD6lrA9ZYAjNp5T1IQsMwfkUBmMKNJ9C6EP6YzhlaD6F0IcMMATnU8p1mT5DBZxPKetDBhiC8yllfcgAQ2g+hdCHDDCE5lMIfcgAQ3A+hcGYAs2nEPqQ/hhOBZpPIfQhAwzB+ZRyXWaAITifUtaHDDAE51PK+pA+w0uKwDAk9CEDDKH5FEIfMsAQnE9hL6ao0HwKoQ/pj+FUofkUQh8ywBCcTynXZQYYgvMpZX3IAENwPqWsDxlgCM2nEPqQAYbQfAqhD+kz1MD5FPZiigbNpxD6kP4YTg2aTyH0IQMMwfmUcl1mgCE4n1LWhwwwBOdTyvqQAYbQfAqhDxlgCM2nEPqQAYbgfApzMUW67TowH8GQuXUdXmoGHIbMresgCdB8ClEOqY8VkS7I4DBkbm5nCdwaLUQ5pH5PSoK3Rgtzc0hK8NZoYW6uKgneGi3MzVUl3XaNls8oh/Q19m1XXPmMcsgAQ+g+5ZbPmtmtxF/rbrhc9VZeKD2t+7VtsWAxQcxzfa9/3ow32SL248gOzZ/vlhac/blPOy7WiT/BW3pZdjjTs7dZ/Cvn36IsvvtNkBsvtLNg5/1y1DUsp0PRZmMfXu2QxEGUpa/O3C3e+Jk/Mn89f2r/f/vjF88p+M3Rv3t685LcNN5uHO981FvJUP5wosze+F5GnOhUZF7g/k1NBLcKT7k1o//UpwRuFR6iNaPv1OCtwsPck2ISvFV4mBuRLsFbhYe5EekSuFV4iHJI36mBW4WHKIcMMITudhm4owRuFR6iHN6Q4VWnBk3YyOXuAokEyHNXCPJl8/JhCG+qa0pr3LqSp7qVb1fWuFWFmSjLt4N+01ILTQYp5UlLaQOEpoEUgTGA0ASQUp6ulDZAaOpHKff10wbIQ+vXIaKIQpsgNPFDhBHqBKH16RBxhDpBcFKmHEioEwSnZcqRhDpBaPc7iUiiUSYIblQmEUmoE4SmZohIQp0gODVTjiTUCYJTM+VIQp0gNFdXjiQiT5sgtB7tciShTxCamilHEuoEwY26KUcS+gTBqZnyvT3qBKG5OiKSiLQJguuiLkcS6gShqRkiklAnCE7NlCMJdYLg1Ew5ktAmyN92iqnSDfq56niOc+0G/UyVilWxPwQ6T/FB5d9Av2m5pQOd4pPNv+nHvalwZ6Ok33Bg82+g31Trs1HSaUNXriw70s+8BL/DE/DxRWa/Ev4VVhRHXons+S07DPwIbzoYnYff1wtkgWOH6PzBOnDd06NL17K09DgT/jDICoAq90kO5M+dgeptM4nsTz1nEllD/qWZJF1RRzfOJPIG9DmTpP8y6Sxpr9ygvXEmkb0h50yS/8uk887clS6rz8ql7qFhBtyeS7T7bNjcbTr84/j7O3RAurCT4mWwtosHUl8At+2ZF3bjNMiCuAA9i7MsXuMdwuID3XZW/gnv68h/+rmSSVnxsKx++gaUJp6TnXPk/J1V187sbyJ63hRqSeR/E4zgSe/0cu6+7scI/zz2hwtz6ONXAYd/NbcGmuC/+nE+WDrFDnrj0eg/WU0D+c05WqyC4k0U5v1aeMQv2iY+1tg3dS7DVPTigVX8U11wvacFNxS0tdtwF856iGajGufU9zu3vk8mgS5Mx62jPdK2D4fKob0002Z9wbkNJLcPWjYZ9/JZXeNm9TBor7XDFGeZvlSXG7v4xukwNK2nXiXqiLMHZzua9g6NijfvTMWOtYjtYbhsiknf5Dg/tkc/muHB5H/Yoz0X/DDCZL1JUKXftqTU6g2sim5Mg+pTM1sMlem9z/Xx96Rtt4Yvwzh4TW+LvU3N0pa7Ov4gzLSKGrvKyHV5lBs5f+QEL0cob6iLkW8g/dE0kNFv1nzkWzMLJ3Ud6JbeFKpK6jytRvhU3EMVdfGpNu6TMNZFXrQtE5krq41ayH/rHKbBIavZipGF+j2UopWh26iJTM5qolbTHyFD1zO8t2E3a8j0rbalI3/t60hvmUZuWM1ajHw06OFfq0CfoKZZL7K7FftD1NdRhkzDsNF/afkvLa/S0hxkCmogZSketUbn2MpzzZsvYwWF1aOiHbydbcbDdLyTfG3iLUeb3vbYUqrd7ZdnUs+FUS7UZM1ZD/iF3DkkQ+Uhqhpi+yHaGVxXcCN3NVjsZNUR5sLi3lwe9/K0O2rEuA2oZtGU0/xIfEy+DJUraRG0VXXS7k5n95I2H4mYQl2Za4JbUDio6vaYVPJio1+QqxWTO9U6YmoMH3e7xnStxf2Fqjw00g5uLSXoLN5Ky6L+5OLIWWss0zifo54ScPdmNF2pk+1u3NgU5WipyFJzNzUrk+cIOzc0odedtLQpP+4pnGgPvySZ57T00mmuosaKiwaFgdE9tW1Vcqd7Lj/KAZefvKWEnP916g6RlgTntON1D8cWJqDfTxSnjpSCxNMRlwet08CtziN+JRaLweneQ3cwMg5FjYqWBpeKjzvNHwjB+MkNc1dzhRVsGm+mJZoJVgeJ450iWtp6sPfluXecFDFrg92dtM1b1jFVNGHa9atiO7GgXudbaTFMxcv1vFNx57gQRGZx9cUjEDUvquJ41d3wM3z9qDHTv+b1n9NS30fVXBcU9ZCYRZRpiLhyqJMxbkn1BrYcX+dKibQsJ8dFBXmauMg5NypqAc79rDdID85DEUFmmyWOLL7pPfRzSNf1D9PyuOw0fF0RPXG2e24ZG4Ot2MU6NT9q6q4rYh5ouQ/GXyfnybSk98dNJ0qr6KHo7XDWf2798W650RGFJkYmTmr59OT59VZvKJmbVcv3/aJnovin25eOh3mwL7qEnjsf9Hjjeq/6m577pUbn7pqipyrxNsHay7xN0bcURP7rroufvSDPHSDc6eelL4noOLrSvfTbviT1HQ/3fd5T5Vd7k8jOo5tNoFaaLe1dID93AjVF1e4us9i/zKzF352/53fzl73rqD9MpqZob53jH0+pplbedboPnFjtWtkCMWPBp1foT5ur5BryCo3q7O2DbPzq9aQ41Z103qruz2c+bRwuGxG+3PHrjcnrjZ8HnbYuR7234cAETjXlDVLnWx3PVeAtospftkR/lZ/vGJ5a7uwnu+SJMPnH+wFvdPbjrUWWJfhCUXFpWNbvcSDFxeW7g2skzoY7d+usiv9+fOfgswm1YHuCUNu+HCtIxdFScTz+Jf7wfiQbQXbsVcDLbrRweS3dybODvPueJ17kB5F3F3mZG9nf0zR8Pis+Lk8u33na3CZhbLvp6bXAFU8b4hfFPWD8J7XXSej973QF33FM3wuaene6JyHjT4rm41R+L1tRPJ/9Eqv+qpUoDyapXJ2djL+7DEb+taX4rKaCwrjdv6JYXkXtOsWrje1nIaQwcPcGCG8YriiM2/07gAJjAEEMf/w75BXGkF/u/39l5u8KWLdkTuFBp78jqLJGUABGsLzAHX2CFB7b/juCrAUrGpPQ/B1B5mIPhccVP1Zx3nJC3KsEwdmeci2mTvCmrocNzUmdOYhnyz5Wc1JnTuGRyI/VnLQJ0piu6mM1J3WC0HwPoTmpEwTne1iLPTSmq/pYzXnLBQSuEoTmewjNSZ0gON9TrsXUCYLzPWU9SJ0guLs9ZT1InSC02z2EHqROEJonIfQgbYI0Jp37WD1InSA0T1LWg9ItF8u8ShCaJynrQfoEwXmSCmsEwXmS8pQW1AmC8yQqawSheZKyHqRPEJonKetB+gTBeRLmIgk0T0LowRtOM3Z9iDk0T0LoQeoEwXmSci2mThCcJynrQeoEwXmSsh6kThCaJyH0IHWC0DwJoQepEwTnSZiLJNA8CaEHaY/yrEDzJIQepE4QnCcp12LaBC8PMsMhWNaD1AmC8yRlPUidIDRPQuhB6gSheRJCD1InCM6TMBdJoHkSQg/SHoEpQfMkhB6kThCcJynXYuoEwXmSsh6kThCcJynrQdoEL98PhiChB6kThOZJCD1InSA4T8JaJJGheRJCD9IegSlD8ySEHqROEJwnKddi6gTBeZKyHqROEJwnKetB6gSheRJCD1InCM2TEHqQNsHLmm1wCLIWSRRonqSsB2XaIzCvrFLKOEGBNYLgPEmFNYLgPInMGkFwnkRljSA0T1LWg/QJQvMkZT1InyA4T8JcJIHmSQg9SHsE5qVdhkOwXIupEwTnScq1mDpBcJ6krAepEwTnScp6kDpBaJ6E0IPUCULzJIQepE4QnCdhLpKA8ySsrZ6ggvMkrK2eoELzJEQZpD3qQ4PmSYgySJ0gNE9ClEHad5o0aJ6EKIPUCULzJOUySH22FA2cJ2Ft1igNnCdhbaYKDZwnYW2mCg26J6H+VJimEsD+xQuPy/z17Pnd+t5/2P/b24uNv1j6cmH4p8uMy8ofTvTJC4xr4GwZa49Y85doDgcha8918Rw4Y8bag108B86ZsTaanOfAWTPWhpPzHDhvxtoYNp4DZ85YG8TGc+DcGWt3znkOnD1j7dY5z0G7ZySX+0o1EuGL2n3N8OXNT4AIzZ+8B+Jty+HF6FJZ+dSVPNWtfLuy8qkqzERZ/hjoSnmVIfrQoTkaRWAOITRHo5TXGaKPEJqjUco3POkjhOZoygFI4qkjhOZoyuGEAYTQHE05nDCAEJqjKYcTBhBCczTlcMIAQuh+RhJpI7ycGQxCIpzQRwjdnTCAELo7YQAhdHfCAELw7kSijhC8O6GPELw7oY8QvDuhjxC8O6GPELw7UWgjvEgCMAiJcEIfIXh3Qh8heHdCHyF4d0If4Tsicrqwk+JlsLaLIeV6gSBw7LBtz7ywG6dBFsQR/nwWZ1m8xjuExQe67az8E/1Xd5Tnp59X50Bh4BfHZsXYdv30DShNPCc7Z9j5O6uundnfRPS8KdSSyP8mGMGT3unl3H3djxH+eewPF+bQx68CDv9qbg00wX/143ywdIod9Maj0X+ymgbym3O0WAXFmyjM+7XwiF+0TXyssW/qXIazUS/Gl+Of6oLrPS24oaCt3Ya7cNZDNBvVOKe+37n1fTIJdGE6bh3tkbZ9OFQO7aWZNusLzm0guX3Qssm4l8/qGjerh0F7rR2mONP0pbrc2MU3ToehaT31KlFHnD0429G0d2hUvHlnKnasRWwPw2VTTPomx/mxPfrRDA8m/8Me7bnghxEm602CKv22JaVWb2BVdGMaVJ+a2WKoTO99ro+/J227NXwZxsFrelssHGqWttzV8QdhplXU2FVGrsuj3Mj5Iyd4OUJ5Q12MfAPpj6aBjH6z5iPfmlk4qetAt/SmUFVS52k1wqfiHqqoi0+1cZ+EsS7yom2ZyFxZbdRC/lvnMA0OWc1WjCzU76EUrQzdRk1kclYTtZr+CBm6nuG9DbtZQ6ZvtS0d+WtfR3rLNHLDatZi5KNBD/9aBfoENc16kd2t2B+ivo4yZBqGjf5Ly39peZWW5iBTUAMpS/GoNTrHVp5r3nwZKyisHhXt4O1sMx6m453kaxNvOdr0tseWUu1uvzyTei6McqEma856wC/kziEZKg9R1RDbD9HO4LqCG7mrwWInq44wFxb35vK4l6fdUSPGbUA1i6ac5kfiY/JlqFxJi6CtqpN2dzq7l7T5SMQU6spcE9yCwkFVt8ekkhcb/YJcrejwq3XE1Bg+7naN6VqL+wtVeWikHdxaStBZvJWWRf3JxZGz1limcT5HPSXg7s1oulIn2924sSnK0VKRpeZualYmzxF2bmhCrztpaVN+3FM40R5+STLPaeml01xFjRUXDYqZZHVPbVuV3Omey49ywOUnbykh53+dukOkJcE57Xjdw7GFCej3E8WpI6Ug8XTE5UHrNHCr84hficUkk7r30B2MjENRo6KlwaXi407zB0IwfnLD3NVcYQWbxptpiWaC1UHieKeIlrYe7H157h0nRczaKAonbfOWdUwVTZh2/arYTiyo1/lWWgxT8XI971TcOS4EkVlcfTHssOZFVRyvuht+hq8fNWb617z+c1rq+6ia64KiHhKziDINbH11dTLGLanewJbj61wpkZbl5LioIE8TFznnRkUtwLmf9QbpwXkoIshss8SRxTe9h34O6br+YVoel52GryuiJ852zy1jY7AVu1in5kdN3XVFzAMt98H46+Q8mZb0/rjpRGkVPRT9Hc76z60/3i03OqLQxMjESS2fnjy/3uoNJXOzavm+X/RMFP90+9LxMA/2RS/Qc+eDHm9cr+i+iOLIexklPzp32BQdRYm3CdZe5m2KjqUg8l93XfzsBXnuAOFOPx/Tm6S+YyTurR8GETWi++hfPOOBomp3l1k0Xh6G5+/O3/O7CQfeddQfZj9QtLfO8Y/nQFAr7zrdJ8+EwFdueueGynMyKnMPJ1V4GlXa2wfZ+NXrSXGqO+m8Vd2fz3zaOFw2InzB49cbk9cbPw86bV2Oem/jgQmcasubrM7l8bkivLXn87S9f9Eg/WWmvuPuU7nfn+ydJyLmH28NvNHvj7cWWZbgS0XFxWGFv8cxFZeZ7w6umDgv7tytsyr++/Gdg88m1ILtCUNt+3KsIBVHS8Xx+Jf4w/uRbATZsVcBL7vRwuW1dCfPDvLue554kR9E3l3kZW5kf0/T8Pms+Lg8uXznaXObhLHtpqfXAleMD8Yvitkc8J/UXieh97/TFXzH4X0vaOrd6faEjD8pWpFTIb5sRfF89kvQ+qvGgi8/lHx1dgH+TpSuNRif12JAuy9HLG5wlePVVvfzIEIbN/g+iDeNXNDGDRJLRNBHCG3cILFGBH2E0MYNEotE0EcIbdwgsUoEfYTQxg0Sy0RQR3gZ+wkHIXPhRII2bpBYKII+wnc4N6YQlnWhQn3yKQmcPxGYQwjOnVSYQwjOncjMIQTnTlTmEEJzJ2VdyABCaO6krAsZQAjOnTAXTi6NMxiEhC6kPh2kDM2dELqQPkJw7qRckekjBOdOyrqQPkJw7qSsC+kjhOZOCF1IHyE0d0LoQvoIwbkT9sIJNHdC6ELqy8fI0NwJoQupI7w86AwHYbki00cIzp2UdSF9hODcSVkX0kcIzZ0QupA+QmjuhNCF9BGCcyfshRNo7oTQhdQXdFOguRNCF9JHCM6dlCsyfYTg3ElZF1JHeFFZcBCWdSF9hNDcCaEL6SOE5k4IXUgfITh3wlw4UaG5E0IXUh+iqUJzJ4QupI8QnDspV2T6CMG5k7IupI8QnDsp60L6CKG5E0IXUkd4mQEADkLmwokGzp0wF040aO6krAtV6kM0NWjupKwLGUAIzp1UmEMIzp3IzCEE505U5hBCcydlXcgAQmjupKwLGUAIzp2wFk5eZsABg5DQhbSHaAocNHdC6EL6CMG5k3JFpo8QnDsp60L6CMG5k7IupI8QmjshdCF9hNDcCaEL6SME507YCyfQ3AmhC2kP0RQ4aO6E0IXUEfLg3Em5ItNHCM6dlHUhfYTg3ElZF9JHCM2dELqQPkJo7oTQhfQRgnMn7IUTaO6E0IW0h2gKPDR3QuhC+gjBuZNyRaaPEJw7KetC6ggvZ4aDsKwL6SOE5k4IXUgfITR3QuhC+gjBuRPmwolwU3dSWttrrjqe43y7srbXTJUqEvdB0FlbWAEn6etDZ20pBpyErw+dtamicRK+PnTWJpfGYQVaWGRtKktBgOayiFJIHaEIzWURpZB6p50IzmWxNnGWIIJzWaxN0yGI4FwWa9N0CCK4e0CsPRQsXBZT/W+J85Od4q/nz+9WEv/D/t/eXtZc/t2Tpf90QXNZ+cOJPnkpc0GEdhuMaMvoV0Rot8HKbRn9hzdEcAaNtQfZBBGcQWNu2PxFm8NByNyw+Qo4g8bcIL0KOIPG3CC9CjiDxtyQgAo4g8bckIAKtEF6cllaiyRCnrvC8OXNT4AIzZ+8B+KNyyE0f6KUp76ijxCaP1HKHRT0EULzJ0p56ivqCKWb+pPSvWNX8lS3cu3esSrMRFn+IOjlblb60KE5GiIASdQRQnM0RACijxCaoyECEH2E0BwNEYDoI4TmaIhwQh8heD+jUEcI3s/QRwjez9BHCN7PUEd4aZzhICyHE/oIwbsTjTpC8O6EPkLw7oQ+QvDuhD5C8O6EPkLo7kTlqSOE7k4YQAjdnTCAELo7oY/wou3hICwPIqGP8KbuhMrDjmKp6tN/rFe5qZ+hA73CHPSbOiAmSjr9x3qVrz9VQ7mkMwD9HS4rXdhJ8TJY28VzQnqBIHDssG3PvLAbp0EWxBH+fBZnWbzGO4TFB7rtrPxTfr3GfPp5dQ4UBn5xbFY8sKSfvgGliedk5yw+f2fVtTP7m4ieN4VaEvnfBCN40ju9nLuv+zHCP4/94cIc+vhVwOFfza2BJvivfpwPlk6xg954NPpPVtNAfnOOFqugeBOFeb8WHvGLtomPNfZNnctwNurFQ0P4p7rgek8Lbihoa7fhLpz1EM1GNc6p73dufZ9MAl2YjltHe6RtHw6VQ3tpps36gnMbSG4ftGwy7uWzusbN6mHQXmuHKc40fakuN3bxjdNhaFpPvUrUEWcPznY07R0aFW/emYodaxHbw3DZFJO+yXF+bI9+NMODyf+wR3su+GGEyXqToEq/bUmp1RtYFd2YBtWnZrYYKtN7n+vj70nbbg1fhnHwmt4Wa8iapS13dfxBmGkVNXaVkevyKDdy/sgJXo5Q3lAXI99A+qNpIKPfrPnIt2YWTuo60C29KVSV1HlajfCpuIcq6uJTbdwnYayLvGhbJjJXVhu1kP/WOUyDQ1azFSML9XsoRStDt1ETmZzVRK2mP0KGrmd4b8Nu1pDpW21LR/7a15HeMo3csJq1GPlo0MO/VoE+QU2zXmR3K/aHqK+jDJmGYaP/0vJfWl6lpTnIFNRAylI8ao3OsZXnmjdfxgoKq0dFO3g724yH6Xgn+drEW442ve2xpVS72y/PpJ4Lo1yoyZqzHvALuXNIhspDVDXE9kO0M7iu4EbuarDYyaojzIXFvbk87uVpd9SIcRtQzaIpp/mR+Jh8GSpX0iJoq+qk3Z3O7iVtPhIxhboy1wS3oHBQ1e0xqeTFRr8gVyumnKp1xNQYPu52jelai/sLVXlopB3cWkrQWbyVlkX9ycWRs9ZYpnE+Rz0l4O7NaLpSJ9vduLEpytFSkaXmbmpWJs8Rdm5oQq87aWlTftxTONEefkkyz2nppdNcRY0VFw0Ksat7atuq5E73XH6UAy4/eUsJOf/r1B0iLQnOacfrHo4tTEC/nyhOHSkFiacjLg9ap4FbnUf8SlTUAtFDdzAyDkWNipYGl4qPO80fCMH4yQ1zV3OFFWwab6YlmglWB4njnSJa2nqw9+W5d5wUMWujKJy0zVvWMVU0Ydr1q2I7saBe51tpMUzFy/W8U3HnuBBEZnH1xcT3NS+q4njV3fAzfP2oMdO/5vWf01LfR9VcFxT1kJhFlGmIuHKokzFuSfUGthxf50qJtCwnx0UFeZq4yDk3KmoBzv2sN0gPzkMRQWabJY4svuk99HNI1/UP0/K47DR8XRE9cbZ7bhkbg63YxTo1P2rqritiHmi5D8ZfJ+fJtKT3x00nSqvooejvcNZ/bv3xbrnREYUmRiZOavn05Pn1Vm8omZtVy/f9omei+Kfbl46HebAv+o2eOx/0eON6RfdFFEfeSwfR6NxhU3QtJd4mWHuZtym6ooLIf9118bMX5LkDhDv9fExvkvqOpytu/YCfIhPdR//iaWwUVbu7LEn8MsEJf3f+nt/NIvOuo/4wpY2ivXWOfzyxjVp51+k+e3obheZ0m7d5ZEpj7jk1RaVRpb19kI1fvZ4Up7qTzlvV/fnMp43DZSPCFzx+vTF5vfHzoNPW5aj3Nh6YwKm2vMnqfKP+uSK8saf41w3SX2bqO0YUlPv9yd55ImL+8dbAG/3+eGuRZQm+VFRcHFb4exxTcZn57uCKifPizt06q+K/H985+GxCLdieMNS2L8cKUnG0VByPf4k/vB/JRpAdexXwshstXF5Ld/LsIO++54kX+UHk3UVe5kb29zQNn8+Kj8uTy3eeNrdJGNtuenotcMUKFfhFsTYw/pPa6yT0/ne6gu84vO8FTb073Z6Q8SdFK3IqxJetKJ7Pfglaf9VY8CUFoF2dMYa/u0wu92uD8WktxuX+LZixFuXFeK5zvNrqfh5EaGPB3wfxlpFLvenYiY9AKDCHENpY8PKKRgwghDYWvLyiEQMIoY0FL69oxABCaGPByysaMYAQ2ljw8opGDCCENha8vKIRAwihjQUndCH1CQUvD+bAQViuyPQRgnMn5YpMHyE4d1LWhfQRgnMnZV1IHyE0d0LoQvoIobkTQhfSRwjOnbAXTr7+UnSEkqQ+KbB2Uz9DB3q5saAPHZwDKjcWtBGKl4oEB2FZe9JHCM4BlbUnfYTQHBChPekjhOaACO1JHyE4B8ReOPn6C34T2pP2Q+oi9/UX/Ca0J33o0O4BEdqTPkJo94AI7UkfITgHVNae1BFeZtMBg5DQnvQRQnNAhPakjxCcA2IunPDQHFBZSfIc7VlgRB6aBSoLQxYYQrsLVFaGLDCEdhuoLA1ZYAjOoajsMYRmUcrikAWG0DxKWR0ywPByZjgM2YspAjSXQupD2mNOxQsyOAyJukyfITifQtRl+gzB+RRCH9JnCM6nEPqQPkNoPoXUh/QZQvMppD6kzxCcT2EwpkDzKaQ+pD32VLzMUQKHIVGX6TME51OIukyfITifQuhD+gzB+RRCH9JnCM2nkPqQPkNoPoXUh/QZgvMpDMYUaD6F1IfUx4eK0HwKqQ/pMwTnU4i6TJ3hJUVwGBL6kD5DcD6F0If0GULzKaQ+pM8Qmk8h9SF9huB8CoMxBZpPIfUh9TGcFWg+hdSH9BmC8ylEXabPEJxPIfQhfYbgfAqhD6kzlKD5FFIf0mcIzaeQ+pA+Q3A+hb2YIkHzKYQ+5KmP4ZSg+RRCHzLAEJxPKddlBhiC8yllfcgAQ3A+pawPGWAIzacQ+pABhtB8CqEP6TO8tC5wGLIXU2RwPoW15SFEGZxNYW15CFGG5lKIUkh9oIgMzaQQpZA+QmgehSiF1G9HydAsClEK6SOE5lCIUki900sGZ1CYm6BKBudP2JsLQwHnT9ibC0OB7k8YeMZMIZH9i1dFl/nfZNDvVh//0wHf3l4LXf7tg5v/dBl0WfnTmT55BXRRAefT2Ht6WwFn1Nh7SkwB59TYe0pMAWfV2BuNroDzauyNRlfAmTX2Rr0p4Nwae6PeLgmAw5C9u+sqOLfG3t11VQDGUC53Xikkwhesrxm+vPkJEKEZlfdAvHE5hOZTlPIU9fQRQrMpisAcQmguRSlPUU8fITSTopRvKNFHSHOhTVfyVLfy7cpiR6owK+64fk4A0qhDh2ZqiABEHeElAXAQlgMQfYTQLA0RgOgjhOZoiABEHyF0P8NzPHWG0A0NCwyhOxoWGEK3NCwwhO5pWGAI7cYLGVNE6gyhexT6DF8e+IXDkIgp9BlCdyksMIRuU1hgCN+nSNQZwvcp9BnC9yn0GcL3KfQZwvcp9BnSvPkyVx3Pca7dfJmpUkXiPoY6z9qDfBXupsaGDnTWHv2rXLoEvjJ01h4WrPA3tU5slHT60N/htdKFnRQvg7VdPEOkFwgCxw7b9swLu3EaZEEc4c9ncZbFa7xDWHyg287KP+XXa8ynn1fnQGHgF8dmxcNM+ukbUJp4TnbO4vN3Vl07s7+J6HlTqCWR/00wgie908u5+7ofI/zz2B8uzKGPXwUc/tXcGmiC/+rH+WDpFDvojUej/2Q1DeQ352ixCoo3UZj3a+ERv2ib+Fhj39S5DGejXjxOhH+qC673tOCGgrZ2G+7CWQ/RbFTjnPp+59b3ySTQhem4dbRH2vbhUDm0l2barC84t4Hk9kHLJuNePqtr3KweBu21dpjiTNOX6nJjF984HYam9dSrRB1x9uBsR9PeoVHx5p2p2LEWsT0Ml00x6Zsc58f26EczPJj8D3u054IfRpisNwmq9NuWlFq9gVXRjWlQfWpmi6Eyvfe5Pv6etO3W8GUYB6/pbXEgqVnaclfHH4SZVlFjVxm5Lo9yI+ePnODlCOUNdTHyDaQ/mgYy+s2aj3xrZuGkrgPd0ptCVUmdp9UIn4p7qKIuPtXGfRLGusiLtmUic2W1UQv5b53DNDhkNVsxslC/h1K0MnQbNZHJWU3UavojZOh6hvc27GYNmb7VtnTkr30d6S3TyA2rWYuRjwY9/GsV6BPUNOtFdrdif4j6OsqQaRg2+i8t/6XlVVqag0xBDaQsxaPW6Bxbea5582WsoLB6VLSDt7PNeJiOd5KvTbzlaNPbHltKtbv98kzquTDKhZqsOesBv5A7h2SoPERVQ2w/RDuD6wpu5K4Gi52sOsJcWNyby+NennZHjRi3AdUsmnKaH4mPyZehciUtgraqTtrd6exe0uYjEVOoK3NNcAsKB1XdHpNKXmz0C3K1YnXiWkdMjeHjbteYrrW4v1CVh0bawa2lBJ3FW2lZ1J9cHDlrjWUa53PUUwLu3oymK3Wy3Y0bm6IcLRVZau6mZmXyHGHnhib0upOWNuXHPYUT7eGXJPOcll46zVXUWHHRoHj4VPfUtlXJne65/CgHXH7ylhJy/tepO0RaEpzTjtc9HFuYgH4/UZw6UgoST0dcHrROA7c6j/jV6Xk03XvoDkbGoahR0dLgUvFxp/kDIRg/uWHuaq6wgk3jzbREM8HqIHG8U0RLWw/2vjz3jpMiZm0UhZO2ecs6poomTLt+VWwnFtTrfCsthql4uZ53Ku4cF4LILK6+MC01L6rieNXd8DN8/agx07/m9Z/TUt9H1VwXFPWQmEWUaYi4cqiTMW5J9Qa2HF/nSom0LCfHRQV5mrjIOTcqagHO/aw3SA/OQxFBZpsljiy+6T30c0jX9Q/T8rjsNHxdET1xtntuGRuDrdjFOjU/auquK2IeaLkPxl8n58m0pPfHTSdKq+ih6O9w1n9u/fFuudERhSZGJk5q+fTk+fVWbyiZm1XL9/2iZ6L4p9uXjod5sC/6jZ47H/R443pF90UUR95LB9Ho3GFTdC0l3iZYe5m3Kbqigsh/3XXxsxfkuQOEO/18TG+S+o7HM278vF+FF4nuo3/xFDeKqt1dRsj9nPiEvzt/0e/ml3nfYX+Y7kbR3jzJP570Rq2873yfPPVNhb/pXXkqjwARFZv2sPcK//XvQgmlwRMV2hPDvCwe8ZWhC8xB//p3oYiSTv1+64XxV4ZeLumfBx1vbuI4ex13MaDFQ+x6xR7/Dw== -------------------------------------------------------------------------------- /assets/subsampled-image.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/subsampled-image.png -------------------------------------------------------------------------------- /assets/vgg-resnet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/vgg-resnet.png -------------------------------------------------------------------------------- /assets/vgg-table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/vgg-table.png -------------------------------------------------------------------------------- /assets/vgg.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bentrevett/pytorch-image-classification/33c9cfdb9097bc65b5800c1996f199f0ba520888/assets/vgg.png -------------------------------------------------------------------------------- /misc/4 - VGG.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import torch\n", 10 | "import torch.nn as nn\n", 11 | "import torch.optim as optim\n", 12 | "\n", 13 | "import torchvision.transforms as transforms\n", 14 | "import torchvision.datasets as datasets\n", 15 | "\n", 16 | "import random\n", 17 | "import time\n", 18 | "\n", 19 | "import numpy as np" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": 2, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "SEED = 1234\n", 29 | "\n", 30 | "random.seed(SEED)\n", 31 | "np.random.seed(SEED)\n", 32 | "torch.manual_seed(SEED)\n", 33 | "torch.cuda.manual_seed(SEED)\n", 34 | "torch.backends.cudnn.deterministic = True" 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "execution_count": 3, 40 | "metadata": {}, 41 | "outputs": [ 42 | { 43 | "name": "stdout", 44 | "output_type": "stream", 45 | "text": [ 46 | "Files already downloaded and verified\n", 47 | "Calculated means: [0.49139968 0.48215841 0.44653091]\n", 48 | "Calculated stds: [0.24703223 0.24348513 0.26158784]\n" 49 | ] 50 | } 51 | ], 52 | "source": [ 53 | "train_data = datasets.CIFAR10(root = 'data', \n", 54 | " train = True, \n", 55 | " download = True)\n", 56 | "\n", 57 | "means = train_data.data.mean(axis = (0,1,2)) / 255\n", 58 | "stds = train_data.data.std(axis = (0,1,2)) / 255\n", 59 | "\n", 60 | "print(f'Calculated means: {means}')\n", 61 | "print(f'Calculated stds: {stds}')" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": 4, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "train_transforms = transforms.Compose([\n", 71 | " transforms.RandomHorizontalFlip(),\n", 72 | " transforms.RandomRotation(10),\n", 73 | " transforms.RandomCrop(32, padding = 3),\n", 74 | " transforms.ToTensor(),\n", 75 | " transforms.Normalize(mean = means, \n", 76 | " std = stds)\n", 77 | " ])\n", 78 | "\n", 79 | "test_transforms = transforms.Compose([\n", 80 | " transforms.ToTensor(),\n", 81 | " transforms.Normalize(mean = means, \n", 82 | " std = stds)\n", 83 | " ])" 84 | ] 85 | }, 86 | { 87 | "cell_type": "code", 88 | "execution_count": 5, 89 | "metadata": {}, 90 | "outputs": [ 91 | { 92 | "name": "stdout", 93 | "output_type": "stream", 94 | "text": [ 95 | "Files already downloaded and verified\n", 96 | "Files already downloaded and verified\n" 97 | ] 98 | } 99 | ], 100 | "source": [ 101 | "train_data = datasets.CIFAR10('data', \n", 102 | " train = True, \n", 103 | " download = True, \n", 104 | " transform = train_transforms)\n", 105 | "\n", 106 | "test_data = datasets.CIFAR10('data', \n", 107 | " train = False, \n", 108 | " download = True, \n", 109 | " transform = test_transforms)" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": 6, 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [ 118 | "n_train_examples = int(len(train_data)*0.9)\n", 119 | "n_valid_examples = len(train_data) - n_train_examples\n", 120 | "\n", 121 | "train_data, valid_data = torch.utils.data.random_split(train_data, \n", 122 | " [n_train_examples, n_valid_examples])" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": 7, 128 | "metadata": {}, 129 | "outputs": [ 130 | { 131 | "name": "stdout", 132 | "output_type": "stream", 133 | "text": [ 134 | "Number of training examples: 45000\n", 135 | "Number of validation examples: 5000\n", 136 | "Number of testing examples: 10000\n" 137 | ] 138 | } 139 | ], 140 | "source": [ 141 | "print(f'Number of training examples: {len(train_data)}')\n", 142 | "print(f'Number of validation examples: {len(valid_data)}')\n", 143 | "print(f'Number of testing examples: {len(test_data)}')" 144 | ] 145 | }, 146 | { 147 | "cell_type": "code", 148 | "execution_count": 8, 149 | "metadata": {}, 150 | "outputs": [], 151 | "source": [ 152 | "BATCH_SIZE = 64\n", 153 | "\n", 154 | "train_iterator = torch.utils.data.DataLoader(train_data, \n", 155 | " shuffle = True, \n", 156 | " batch_size = BATCH_SIZE)\n", 157 | "\n", 158 | "valid_iterator = torch.utils.data.DataLoader(valid_data, \n", 159 | " batch_size = BATCH_SIZE)\n", 160 | "\n", 161 | "test_iterator = torch.utils.data.DataLoader(test_data, \n", 162 | " batch_size = BATCH_SIZE)" 163 | ] 164 | }, 165 | { 166 | "cell_type": "code", 167 | "execution_count": 9, 168 | "metadata": {}, 169 | "outputs": [], 170 | "source": [ 171 | "class VGGBlock(nn.Module):\n", 172 | " def __init__(self, in_channels, out_channels, batch_norm):\n", 173 | " super().__init__()\n", 174 | " \n", 175 | " modules = []\n", 176 | " modules.append(nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1))\n", 177 | " if batch_norm:\n", 178 | " modules.append(nn.BatchNorm2d(out_channels))\n", 179 | " modules.append(nn.ReLU(inplace=True))\n", 180 | " \n", 181 | " self.block = nn.Sequential(*modules)\n", 182 | " \n", 183 | " def forward(self, x):\n", 184 | " return self.block(x)" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": 10, 190 | "metadata": {}, 191 | "outputs": [], 192 | "source": [ 193 | "class VGG11(nn.Module):\n", 194 | " def __init__(self, output_dim, block, pool, batch_norm):\n", 195 | " super().__init__()\n", 196 | " \n", 197 | " self.features = nn.Sequential(\n", 198 | " block(3, 64, batch_norm), #in_channels, out_channels\n", 199 | " pool(2, 2), #kernel_size, stride\n", 200 | " block(64, 128, batch_norm),\n", 201 | " pool(2, 2),\n", 202 | " block(128, 256, batch_norm),\n", 203 | " block(256, 256, batch_norm),\n", 204 | " pool(2, 2),\n", 205 | " block(256, 512, batch_norm),\n", 206 | " block(512, 512, batch_norm),\n", 207 | " pool(2, 2),\n", 208 | " block(512, 512, batch_norm),\n", 209 | " block(512, 512, batch_norm),\n", 210 | " pool(2, 2),\n", 211 | " )\n", 212 | " \n", 213 | " self.classifier = nn.Linear(512, output_dim)\n", 214 | "\n", 215 | " def forward(self, x):\n", 216 | " x = self.features(x)\n", 217 | " x = x.view(x.shape[0], -1)\n", 218 | " x = self.classifier(x)\n", 219 | " return x" 220 | ] 221 | }, 222 | { 223 | "cell_type": "markdown", 224 | "metadata": {}, 225 | "source": [ 226 | "extra 64, 128, 256, 512 and 512" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": 11, 232 | "metadata": {}, 233 | "outputs": [], 234 | "source": [ 235 | "class VGG16(nn.Module):\n", 236 | " def __init__(self, output_dim, block, pool, batch_norm):\n", 237 | " super().__init__()\n", 238 | " \n", 239 | " self.features = nn.Sequential(\n", 240 | " block(3, 64, batch_norm),\n", 241 | " block(64, 64, batch_norm),\n", 242 | " pool(2, 2),\n", 243 | " block(64, 128, batch_norm),\n", 244 | " block(128, 128, batch_norm),\n", 245 | " pool(2, 2),\n", 246 | " block(128, 256, batch_norm),\n", 247 | " block(256, 256, batch_norm),\n", 248 | " block(256, 256, batch_norm),\n", 249 | " pool(2, 2),\n", 250 | " block(256, 512, batch_norm),\n", 251 | " block(512, 512, batch_norm),\n", 252 | " block(512, 512, batch_norm),\n", 253 | " pool(2, 2),\n", 254 | " block(512, 512, batch_norm),\n", 255 | " block(512, 512, batch_norm),\n", 256 | " block(512, 512, batch_norm),\n", 257 | " pool(2, 2),\n", 258 | " )\n", 259 | " \n", 260 | " self.classifier = nn.Linear(512, output_dim)\n", 261 | "\n", 262 | " def forward(self, x):\n", 263 | " x = self.features(x)\n", 264 | " x = x.view(x.shape[0], -1)\n", 265 | " x = self.classifier(x)\n", 266 | " return x" 267 | ] 268 | }, 269 | { 270 | "cell_type": "markdown", 271 | "metadata": {}, 272 | "source": [ 273 | "extra 256, 512 and 512" 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": 12, 279 | "metadata": {}, 280 | "outputs": [], 281 | "source": [ 282 | "class VGG19(nn.Module):\n", 283 | " def __init__(self, output_dim, block, pool, batch_norm):\n", 284 | " super().__init__()\n", 285 | " \n", 286 | " self.features = nn.Sequential(\n", 287 | " block(3, 64, batch_norm),\n", 288 | " block(64, 64, batch_norm),\n", 289 | " pool(2, 2),\n", 290 | " block(64, 128, batch_norm),\n", 291 | " block(128, 128, batch_norm),\n", 292 | " pool(2, 2),\n", 293 | " block(128, 256, batch_norm),\n", 294 | " block(256, 256, batch_norm),\n", 295 | " block(256, 256, batch_norm),\n", 296 | " block(256, 256, batch_norm),\n", 297 | " pool(2, 2),\n", 298 | " block(256, 512, batch_norm),\n", 299 | " block(512, 512, batch_norm),\n", 300 | " block(512, 512, batch_norm),\n", 301 | " block(512, 512, batch_norm),\n", 302 | " pool(2, 2),\n", 303 | " block(512, 512, batch_norm),\n", 304 | " block(512, 512, batch_norm),\n", 305 | " block(512, 512, batch_norm),\n", 306 | " block(512, 512, batch_norm),\n", 307 | " pool(2, 2),\n", 308 | " )\n", 309 | " \n", 310 | " self.classifier = nn.Linear(512, output_dim)\n", 311 | "\n", 312 | " def forward(self, x):\n", 313 | " x = self.features(x)\n", 314 | " x = x.view(x.shape[0], -1)\n", 315 | " x = self.classifier(x)\n", 316 | " return x" 317 | ] 318 | }, 319 | { 320 | "cell_type": "code", 321 | "execution_count": 13, 322 | "metadata": {}, 323 | "outputs": [], 324 | "source": [ 325 | "OUTPUT_DIM = 10\n", 326 | "BATCH_NORM = True\n", 327 | "\n", 328 | "vgg11_model = VGG11(OUTPUT_DIM, VGGBlock, nn.MaxPool2d, BATCH_NORM) \n", 329 | "vgg16_model = VGG16(OUTPUT_DIM, VGGBlock, nn.MaxPool2d, BATCH_NORM) \n", 330 | "vgg19_model = VGG19(OUTPUT_DIM, VGGBlock, nn.MaxPool2d, BATCH_NORM) " 331 | ] 332 | }, 333 | { 334 | "cell_type": "code", 335 | "execution_count": 14, 336 | "metadata": {}, 337 | "outputs": [ 338 | { 339 | "name": "stdout", 340 | "output_type": "stream", 341 | "text": [ 342 | "VGG11 has 9,231,114 trainable parameters\n", 343 | "VGG16 has 14,728,266 trainable parameters\n", 344 | "VGG19 has 20,040,522 trainable parameters\n" 345 | ] 346 | } 347 | ], 348 | "source": [ 349 | "def count_parameters(model):\n", 350 | " return sum(p.numel() for p in model.parameters() if p.requires_grad)\n", 351 | "\n", 352 | "print(f'VGG11 has {count_parameters(vgg11_model):,} trainable parameters')\n", 353 | "print(f'VGG16 has {count_parameters(vgg16_model):,} trainable parameters')\n", 354 | "print(f'VGG19 has {count_parameters(vgg19_model):,} trainable parameters')" 355 | ] 356 | }, 357 | { 358 | "cell_type": "code", 359 | "execution_count": 15, 360 | "metadata": {}, 361 | "outputs": [], 362 | "source": [ 363 | "model = VGG11(OUTPUT_DIM, VGGBlock, nn.MaxPool2d, BATCH_NORM) " 364 | ] 365 | }, 366 | { 367 | "cell_type": "code", 368 | "execution_count": 16, 369 | "metadata": {}, 370 | "outputs": [ 371 | { 372 | "data": { 373 | "text/plain": [ 374 | "VGG11(\n", 375 | " (features): Sequential(\n", 376 | " (0): VGGBlock(\n", 377 | " (block): Sequential(\n", 378 | " (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 379 | " (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 380 | " (2): ReLU(inplace=True)\n", 381 | " )\n", 382 | " )\n", 383 | " (1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 384 | " (2): VGGBlock(\n", 385 | " (block): Sequential(\n", 386 | " (0): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 387 | " (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 388 | " (2): ReLU(inplace=True)\n", 389 | " )\n", 390 | " )\n", 391 | " (3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 392 | " (4): VGGBlock(\n", 393 | " (block): Sequential(\n", 394 | " (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 395 | " (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 396 | " (2): ReLU(inplace=True)\n", 397 | " )\n", 398 | " )\n", 399 | " (5): VGGBlock(\n", 400 | " (block): Sequential(\n", 401 | " (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 402 | " (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 403 | " (2): ReLU(inplace=True)\n", 404 | " )\n", 405 | " )\n", 406 | " (6): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 407 | " (7): VGGBlock(\n", 408 | " (block): Sequential(\n", 409 | " (0): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 410 | " (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 411 | " (2): ReLU(inplace=True)\n", 412 | " )\n", 413 | " )\n", 414 | " (8): VGGBlock(\n", 415 | " (block): Sequential(\n", 416 | " (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 417 | " (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 418 | " (2): ReLU(inplace=True)\n", 419 | " )\n", 420 | " )\n", 421 | " (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 422 | " (10): VGGBlock(\n", 423 | " (block): Sequential(\n", 424 | " (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 425 | " (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 426 | " (2): ReLU(inplace=True)\n", 427 | " )\n", 428 | " )\n", 429 | " (11): VGGBlock(\n", 430 | " (block): Sequential(\n", 431 | " (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", 432 | " (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 433 | " (2): ReLU(inplace=True)\n", 434 | " )\n", 435 | " )\n", 436 | " (12): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n", 437 | " )\n", 438 | " (classifier): Linear(in_features=512, out_features=10, bias=True)\n", 439 | ")" 440 | ] 441 | }, 442 | "execution_count": 16, 443 | "metadata": {}, 444 | "output_type": "execute_result" 445 | } 446 | ], 447 | "source": [ 448 | "model" 449 | ] 450 | }, 451 | { 452 | "cell_type": "code", 453 | "execution_count": 17, 454 | "metadata": {}, 455 | "outputs": [], 456 | "source": [ 457 | "optimizer = optim.Adam(model.parameters())" 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "execution_count": 18, 463 | "metadata": {}, 464 | "outputs": [], 465 | "source": [ 466 | "criterion = nn.CrossEntropyLoss()" 467 | ] 468 | }, 469 | { 470 | "cell_type": "code", 471 | "execution_count": 19, 472 | "metadata": {}, 473 | "outputs": [], 474 | "source": [ 475 | "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')" 476 | ] 477 | }, 478 | { 479 | "cell_type": "code", 480 | "execution_count": 20, 481 | "metadata": {}, 482 | "outputs": [], 483 | "source": [ 484 | "model = model.to(device)\n", 485 | "criterion = criterion.to(device)" 486 | ] 487 | }, 488 | { 489 | "cell_type": "code", 490 | "execution_count": 21, 491 | "metadata": {}, 492 | "outputs": [], 493 | "source": [ 494 | "def calculate_accuracy(fx, y):\n", 495 | " preds = fx.argmax(1, keepdim=True)\n", 496 | " correct = preds.eq(y.view_as(preds)).sum()\n", 497 | " acc = correct.float()/preds.shape[0]\n", 498 | " return acc" 499 | ] 500 | }, 501 | { 502 | "cell_type": "code", 503 | "execution_count": 22, 504 | "metadata": {}, 505 | "outputs": [], 506 | "source": [ 507 | "def train(model, iterator, optimizer, criterion, device):\n", 508 | " \n", 509 | " epoch_loss = 0\n", 510 | " epoch_acc = 0\n", 511 | " \n", 512 | " model.train()\n", 513 | " \n", 514 | " for (x, y) in iterator:\n", 515 | " \n", 516 | " x = x.to(device)\n", 517 | " y = y.to(device)\n", 518 | " \n", 519 | " optimizer.zero_grad()\n", 520 | " \n", 521 | " fx = model(x)\n", 522 | " \n", 523 | " loss = criterion(fx, y)\n", 524 | " \n", 525 | " acc = calculate_accuracy(fx, y)\n", 526 | " \n", 527 | " loss.backward()\n", 528 | " \n", 529 | " optimizer.step()\n", 530 | " \n", 531 | " epoch_loss += loss.item()\n", 532 | " epoch_acc += acc.item()\n", 533 | " \n", 534 | " return epoch_loss / len(iterator), epoch_acc / len(iterator)" 535 | ] 536 | }, 537 | { 538 | "cell_type": "code", 539 | "execution_count": 23, 540 | "metadata": {}, 541 | "outputs": [], 542 | "source": [ 543 | "def evaluate(model, iterator, criterion, device):\n", 544 | " \n", 545 | " epoch_loss = 0\n", 546 | " epoch_acc = 0\n", 547 | " \n", 548 | " model.eval()\n", 549 | " \n", 550 | " with torch.no_grad():\n", 551 | " for (x, y) in iterator:\n", 552 | "\n", 553 | " x = x.to(device)\n", 554 | " y = y.to(device)\n", 555 | "\n", 556 | " fx = model(x)\n", 557 | "\n", 558 | " loss = criterion(fx, y)\n", 559 | "\n", 560 | " acc = calculate_accuracy(fx, y)\n", 561 | "\n", 562 | " epoch_loss += loss.item()\n", 563 | " epoch_acc += acc.item()\n", 564 | " \n", 565 | " return epoch_loss / len(iterator), epoch_acc / len(iterator)" 566 | ] 567 | }, 568 | { 569 | "cell_type": "code", 570 | "execution_count": 24, 571 | "metadata": {}, 572 | "outputs": [], 573 | "source": [ 574 | "def epoch_time(start_time, end_time):\n", 575 | " elapsed_time = end_time - start_time\n", 576 | " elapsed_mins = int(elapsed_time / 60)\n", 577 | " elapsed_secs = int(elapsed_time - (elapsed_mins * 60))\n", 578 | " return elapsed_mins, elapsed_secs" 579 | ] 580 | }, 581 | { 582 | "cell_type": "code", 583 | "execution_count": 25, 584 | "metadata": {}, 585 | "outputs": [ 586 | { 587 | "name": "stdout", 588 | "output_type": "stream", 589 | "text": [ 590 | "Epoch: 01 | Epoch Time: 0m 19s\n", 591 | "\tTrain Loss: 1.469 | Train Acc: 46.14%\n", 592 | "\t Val. Loss: 1.287 | Val. Acc: 54.75%\n", 593 | "Epoch: 02 | Epoch Time: 0m 19s\n", 594 | "\tTrain Loss: 1.035 | Train Acc: 63.19%\n", 595 | "\t Val. Loss: 0.973 | Val. Acc: 66.83%\n", 596 | "Epoch: 03 | Epoch Time: 0m 19s\n", 597 | "\tTrain Loss: 0.856 | Train Acc: 69.85%\n", 598 | "\t Val. Loss: 0.833 | Val. Acc: 70.41%\n", 599 | "Epoch: 04 | Epoch Time: 0m 19s\n", 600 | "\tTrain Loss: 0.761 | Train Acc: 73.57%\n", 601 | "\t Val. Loss: 0.780 | Val. Acc: 73.08%\n", 602 | "Epoch: 05 | Epoch Time: 0m 19s\n", 603 | "\tTrain Loss: 0.671 | Train Acc: 76.72%\n", 604 | "\t Val. Loss: 0.693 | Val. Acc: 75.95%\n", 605 | "Epoch: 06 | Epoch Time: 0m 19s\n", 606 | "\tTrain Loss: 0.619 | Train Acc: 78.62%\n", 607 | "\t Val. Loss: 0.648 | Val. Acc: 77.97%\n", 608 | "Epoch: 07 | Epoch Time: 0m 19s\n", 609 | "\tTrain Loss: 0.570 | Train Acc: 80.38%\n", 610 | "\t Val. Loss: 0.631 | Val. Acc: 78.34%\n", 611 | "Epoch: 08 | Epoch Time: 0m 19s\n", 612 | "\tTrain Loss: 0.523 | Train Acc: 82.01%\n", 613 | "\t Val. Loss: 0.606 | Val. Acc: 79.89%\n", 614 | "Epoch: 09 | Epoch Time: 0m 19s\n", 615 | "\tTrain Loss: 0.489 | Train Acc: 83.04%\n", 616 | "\t Val. Loss: 0.606 | Val. Acc: 79.96%\n", 617 | "Epoch: 10 | Epoch Time: 0m 19s\n", 618 | "\tTrain Loss: 0.454 | Train Acc: 84.42%\n", 619 | "\t Val. Loss: 0.575 | Val. Acc: 80.70%\n" 620 | ] 621 | } 622 | ], 623 | "source": [ 624 | "EPOCHS = 10\n", 625 | "\n", 626 | "best_valid_loss = float('inf')\n", 627 | "\n", 628 | "for epoch in range(EPOCHS):\n", 629 | " \n", 630 | " start_time = time.time()\n", 631 | " \n", 632 | " train_loss, train_acc = train(model, train_iterator, optimizer, criterion, device)\n", 633 | " valid_loss, valid_acc = evaluate(model, valid_iterator, criterion, device)\n", 634 | " \n", 635 | " if valid_loss < best_valid_loss:\n", 636 | " best_valid_loss = valid_loss\n", 637 | " torch.save(model.state_dict(), 'tut5-model.pt')\n", 638 | " \n", 639 | " end_time = time.time()\n", 640 | "\n", 641 | " epoch_mins, epoch_secs = epoch_time(start_time, end_time)\n", 642 | " \n", 643 | " print(f'Epoch: {epoch+1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')\n", 644 | " print(f'\\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc*100:.2f}%')\n", 645 | " print(f'\\t Val. Loss: {valid_loss:.3f} | Val. Acc: {valid_acc*100:.2f}%')" 646 | ] 647 | }, 648 | { 649 | "cell_type": "code", 650 | "execution_count": 26, 651 | "metadata": {}, 652 | "outputs": [ 653 | { 654 | "name": "stdout", 655 | "output_type": "stream", 656 | "text": [ 657 | "Test Loss: 0.487 | Test Acc: 83.41%\n" 658 | ] 659 | } 660 | ], 661 | "source": [ 662 | "model.load_state_dict(torch.load('tut5-model.pt'))\n", 663 | "\n", 664 | "test_loss, test_acc = evaluate(model, test_iterator, criterion, device)\n", 665 | "\n", 666 | "print(f'Test Loss: {test_loss:.3f} | Test Acc: {test_acc*100:.2f}%')" 667 | ] 668 | } 669 | ], 670 | "metadata": { 671 | "kernelspec": { 672 | "display_name": "Python 3", 673 | "language": "python", 674 | "name": "python3" 675 | }, 676 | "language_info": { 677 | "codemirror_mode": { 678 | "name": "ipython", 679 | "version": 3 680 | }, 681 | "file_extension": ".py", 682 | "mimetype": "text/x-python", 683 | "name": "python", 684 | "nbconvert_exporter": "python", 685 | "pygments_lexer": "ipython3", 686 | "version": "3.7.6" 687 | } 688 | }, 689 | "nbformat": 4, 690 | "nbformat_minor": 2 691 | } 692 | -------------------------------------------------------------------------------- /misc/6 - ResNet - Dogs vs Cats.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import torch\n", 10 | "import torch.nn as nn\n", 11 | "import torch.nn.functional as F\n", 12 | "import torch.optim as optim\n", 13 | "import torchvision\n", 14 | "import torchvision.transforms as transforms\n", 15 | "import torchvision.datasets as datasets\n", 16 | "\n", 17 | "import os\n", 18 | "import random\n", 19 | "import numpy as np" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": 2, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "SEED = 1234\n", 29 | "\n", 30 | "random.seed(SEED)\n", 31 | "np.random.seed(SEED)\n", 32 | "torch.manual_seed(SEED)\n", 33 | "torch.cuda.manual_seed(SEED)\n", 34 | "torch.backends.cudnn.deterministic = True" 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "execution_count": 3, 40 | "metadata": {}, 41 | "outputs": [], 42 | "source": [ 43 | "train_transforms = transforms.Compose([\n", 44 | " transforms.RandomHorizontalFlip(),\n", 45 | " transforms.RandomRotation(10),\n", 46 | " transforms.RandomCrop((224, 224), pad_if_needed=True),\n", 47 | " transforms.ToTensor(),\n", 48 | " transforms.Normalize((0.485, 0.456, 0.406),(0.229, 0.224, 0.225))\n", 49 | " ])\n", 50 | "\n", 51 | "test_transforms = transforms.Compose([\n", 52 | " transforms.CenterCrop((224, 224)),\n", 53 | " transforms.ToTensor(),\n", 54 | " transforms.Normalize((0.485, 0.456, 0.406),(0.229, 0.224, 0.225))\n", 55 | " ])" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": 4, 61 | "metadata": {}, 62 | "outputs": [], 63 | "source": [ 64 | "train_data = datasets.ImageFolder('data/dogs-vs-cats/train', train_transforms)\n", 65 | "valid_data = datasets.ImageFolder('data/dogs-vs-cats/valid', test_transforms)\n", 66 | "test_data = datasets.ImageFolder('data/dogs-vs-cats/test', test_transforms)\n", 67 | "\n", 68 | "#import os\n", 69 | "\n", 70 | "#print(len(os.listdir('data/dogs-vs-cats/train')))\n", 71 | "\n", 72 | "#n_train_examples = int(len(train_data)*0.9)\n", 73 | "#n_valid_examples = n_test_examples = len(train_data) - n_train_examples\n", 74 | "\n", 75 | "#train_data, valid_data = torch.utils.data.random_split(train_data, [n_train_examples, n_valid_examples])\n", 76 | "#train_data, test_data = torch.utils.data.random_split(train_data, [n_train_examples-n_valid_examples, n_test_examples])" 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "metadata": {}, 82 | "source": [ 83 | "https://github.com/facebook/fb.resnet.torch/issues/180\n", 84 | "https://github.com/bamos/densenet.pytorch/blob/master/compute-cifar10-mean.py" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": 5, 90 | "metadata": {}, 91 | "outputs": [ 92 | { 93 | "name": "stdout", 94 | "output_type": "stream", 95 | "text": [ 96 | "Number of training examples: 20000\n", 97 | "Number of validation examples: 2500\n", 98 | "Number of testing examples: 2500\n" 99 | ] 100 | } 101 | ], 102 | "source": [ 103 | "print(f'Number of training examples: {len(train_data)}')\n", 104 | "print(f'Number of validation examples: {len(valid_data)}')\n", 105 | "print(f'Number of testing examples: {len(test_data)}')" 106 | ] 107 | }, 108 | { 109 | "cell_type": "code", 110 | "execution_count": 6, 111 | "metadata": {}, 112 | "outputs": [], 113 | "source": [ 114 | "BATCH_SIZE = 64\n", 115 | "\n", 116 | "train_iterator = torch.utils.data.DataLoader(train_data, shuffle=True, batch_size=BATCH_SIZE)\n", 117 | "valid_iterator = torch.utils.data.DataLoader(valid_data, batch_size=BATCH_SIZE)\n", 118 | "test_iterator = torch.utils.data.DataLoader(test_data, batch_size=BATCH_SIZE)" 119 | ] 120 | }, 121 | { 122 | "cell_type": "markdown", 123 | "metadata": {}, 124 | "source": [ 125 | "https://discuss.pytorch.org/t/why-does-the-resnet-model-given-by-pytorch-omit-biases-from-the-convolutional-layer/10990/4\n", 126 | "https://github.com/kuangliu/pytorch-cifar/blob/master/models/resnet.py" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": 7, 132 | "metadata": {}, 133 | "outputs": [], 134 | "source": [ 135 | "device = torch.device('cuda')" 136 | ] 137 | }, 138 | { 139 | "cell_type": "code", 140 | "execution_count": 8, 141 | "metadata": {}, 142 | "outputs": [], 143 | "source": [ 144 | "import torchvision.models as models\n", 145 | "\n", 146 | "model = models.resnet18(pretrained=True).to(device)" 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": 9, 152 | "metadata": {}, 153 | "outputs": [ 154 | { 155 | "name": "stdout", 156 | "output_type": "stream", 157 | "text": [ 158 | "ResNet(\n", 159 | " (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n", 160 | " (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 161 | " (relu): ReLU(inplace)\n", 162 | " (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n", 163 | " (layer1): Sequential(\n", 164 | " (0): BasicBlock(\n", 165 | " (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 166 | " (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 167 | " (relu): ReLU(inplace)\n", 168 | " (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 169 | " (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 170 | " )\n", 171 | " (1): BasicBlock(\n", 172 | " (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 173 | " (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 174 | " (relu): ReLU(inplace)\n", 175 | " (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 176 | " (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 177 | " )\n", 178 | " )\n", 179 | " (layer2): Sequential(\n", 180 | " (0): BasicBlock(\n", 181 | " (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n", 182 | " (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 183 | " (relu): ReLU(inplace)\n", 184 | " (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 185 | " (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 186 | " (downsample): Sequential(\n", 187 | " (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n", 188 | " (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 189 | " )\n", 190 | " )\n", 191 | " (1): BasicBlock(\n", 192 | " (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 193 | " (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 194 | " (relu): ReLU(inplace)\n", 195 | " (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 196 | " (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 197 | " )\n", 198 | " )\n", 199 | " (layer3): Sequential(\n", 200 | " (0): BasicBlock(\n", 201 | " (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n", 202 | " (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 203 | " (relu): ReLU(inplace)\n", 204 | " (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 205 | " (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 206 | " (downsample): Sequential(\n", 207 | " (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n", 208 | " (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 209 | " )\n", 210 | " )\n", 211 | " (1): BasicBlock(\n", 212 | " (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 213 | " (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 214 | " (relu): ReLU(inplace)\n", 215 | " (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 216 | " (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 217 | " )\n", 218 | " )\n", 219 | " (layer4): Sequential(\n", 220 | " (0): BasicBlock(\n", 221 | " (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n", 222 | " (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 223 | " (relu): ReLU(inplace)\n", 224 | " (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 225 | " (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 226 | " (downsample): Sequential(\n", 227 | " (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n", 228 | " (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 229 | " )\n", 230 | " )\n", 231 | " (1): BasicBlock(\n", 232 | " (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 233 | " (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 234 | " (relu): ReLU(inplace)\n", 235 | " (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n", 236 | " (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", 237 | " )\n", 238 | " )\n", 239 | " (avgpool): AvgPool2d(kernel_size=7, stride=1, padding=0)\n", 240 | " (fc): Linear(in_features=512, out_features=1000, bias=True)\n", 241 | ")\n" 242 | ] 243 | } 244 | ], 245 | "source": [ 246 | "print(model)" 247 | ] 248 | }, 249 | { 250 | "cell_type": "code", 251 | "execution_count": 10, 252 | "metadata": {}, 253 | "outputs": [], 254 | "source": [ 255 | "for param in model.parameters():\n", 256 | " param.requires_grad = False" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": 11, 262 | "metadata": {}, 263 | "outputs": [ 264 | { 265 | "name": "stdout", 266 | "output_type": "stream", 267 | "text": [ 268 | "Linear(in_features=512, out_features=1000, bias=True)\n" 269 | ] 270 | } 271 | ], 272 | "source": [ 273 | "print(model.fc)" 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": 12, 279 | "metadata": {}, 280 | "outputs": [], 281 | "source": [ 282 | "model.fc = nn.Linear(in_features=512, out_features=2).to(device)" 283 | ] 284 | }, 285 | { 286 | "cell_type": "code", 287 | "execution_count": 13, 288 | "metadata": {}, 289 | "outputs": [], 290 | "source": [ 291 | "optimizer = optim.Adam(model.parameters())" 292 | ] 293 | }, 294 | { 295 | "cell_type": "code", 296 | "execution_count": 14, 297 | "metadata": {}, 298 | "outputs": [], 299 | "source": [ 300 | "criterion = nn.CrossEntropyLoss()" 301 | ] 302 | }, 303 | { 304 | "cell_type": "code", 305 | "execution_count": 15, 306 | "metadata": {}, 307 | "outputs": [], 308 | "source": [ 309 | "def calculate_accuracy(fx, y):\n", 310 | " preds = fx.max(1, keepdim=True)[1]\n", 311 | " correct = preds.eq(y.view_as(preds)).sum()\n", 312 | " acc = correct.float()/preds.shape[0]\n", 313 | " return acc" 314 | ] 315 | }, 316 | { 317 | "cell_type": "code", 318 | "execution_count": 16, 319 | "metadata": {}, 320 | "outputs": [], 321 | "source": [ 322 | "def train(model, device, iterator, optimizer, criterion):\n", 323 | " \n", 324 | " epoch_loss = 0\n", 325 | " epoch_acc = 0\n", 326 | " \n", 327 | " model.train()\n", 328 | " \n", 329 | " for (x, y) in iterator:\n", 330 | " \n", 331 | " x = x.to(device)\n", 332 | " y = y.to(device)\n", 333 | " \n", 334 | " optimizer.zero_grad()\n", 335 | " \n", 336 | " fx = model(x)\n", 337 | " \n", 338 | " loss = criterion(fx, y)\n", 339 | " \n", 340 | " acc = calculate_accuracy(fx, y)\n", 341 | " \n", 342 | " loss.backward()\n", 343 | " \n", 344 | " optimizer.step()\n", 345 | " \n", 346 | " epoch_loss += loss.item()\n", 347 | " epoch_acc += acc.item()\n", 348 | " \n", 349 | " return epoch_loss / len(iterator), epoch_acc / len(iterator)" 350 | ] 351 | }, 352 | { 353 | "cell_type": "code", 354 | "execution_count": 17, 355 | "metadata": {}, 356 | "outputs": [], 357 | "source": [ 358 | "def evaluate(model, device, iterator, criterion):\n", 359 | " \n", 360 | " epoch_loss = 0\n", 361 | " epoch_acc = 0\n", 362 | " \n", 363 | " model.eval()\n", 364 | " \n", 365 | " with torch.no_grad():\n", 366 | " for (x, y) in iterator:\n", 367 | "\n", 368 | " x = x.to(device)\n", 369 | " y = y.to(device)\n", 370 | "\n", 371 | " fx = model(x)\n", 372 | "\n", 373 | " loss = criterion(fx, y)\n", 374 | "\n", 375 | " acc = calculate_accuracy(fx, y)\n", 376 | "\n", 377 | " epoch_loss += loss.item()\n", 378 | " epoch_acc += acc.item()\n", 379 | " \n", 380 | " return epoch_loss / len(iterator), epoch_acc / len(iterator)" 381 | ] 382 | }, 383 | { 384 | "cell_type": "code", 385 | "execution_count": 18, 386 | "metadata": {}, 387 | "outputs": [ 388 | { 389 | "name": "stdout", 390 | "output_type": "stream", 391 | "text": [ 392 | "| Epoch: 01 | Train Loss: 0.198 | Train Acc: 91.95% | Val. Loss: 0.089 | Val. Acc: 96.80% |\n", 393 | "| Epoch: 02 | Train Loss: 0.136 | Train Acc: 94.40% | Val. Loss: 0.069 | Val. Acc: 97.50% |\n", 394 | "| Epoch: 03 | Train Loss: 0.128 | Train Acc: 94.68% | Val. Loss: 0.059 | Val. Acc: 97.70% |\n", 395 | "| Epoch: 04 | Train Loss: 0.119 | Train Acc: 95.03% | Val. Loss: 0.070 | Val. Acc: 97.30% |\n", 396 | "| Epoch: 05 | Train Loss: 0.118 | Train Acc: 94.95% | Val. Loss: 0.057 | Val. Acc: 97.73% |\n", 397 | "| Epoch: 06 | Train Loss: 0.121 | Train Acc: 94.95% | Val. Loss: 0.056 | Val. Acc: 97.70% |\n", 398 | "| Epoch: 07 | Train Loss: 0.117 | Train Acc: 95.11% | Val. Loss: 0.063 | Val. Acc: 97.46% |\n", 399 | "| Epoch: 08 | Train Loss: 0.110 | Train Acc: 95.44% | Val. Loss: 0.052 | Val. Acc: 97.93% |\n", 400 | "| Epoch: 09 | Train Loss: 0.116 | Train Acc: 95.14% | Val. Loss: 0.056 | Val. Acc: 97.77% |\n", 401 | "| Epoch: 10 | Train Loss: 0.114 | Train Acc: 95.36% | Val. Loss: 0.063 | Val. Acc: 97.46% |\n" 402 | ] 403 | } 404 | ], 405 | "source": [ 406 | "EPOCHS = 10\n", 407 | "SAVE_DIR = 'models'\n", 408 | "MODEL_SAVE_PATH = os.path.join(SAVE_DIR, 'resnet18-dogs-vs-cats.pt')\n", 409 | "\n", 410 | "best_valid_loss = float('inf')\n", 411 | "\n", 412 | "if not os.path.isdir(f'{SAVE_DIR}'):\n", 413 | " os.makedirs(f'{SAVE_DIR}')\n", 414 | "\n", 415 | "for epoch in range(EPOCHS):\n", 416 | " train_loss, train_acc = train(model, device, train_iterator, optimizer, criterion)\n", 417 | " valid_loss, valid_acc = evaluate(model, device, valid_iterator, criterion)\n", 418 | " \n", 419 | " if valid_loss < best_valid_loss:\n", 420 | " best_valid_loss = valid_loss\n", 421 | " torch.save(model.state_dict(), MODEL_SAVE_PATH)\n", 422 | " \n", 423 | " print(f'| Epoch: {epoch+1:02} | Train Loss: {train_loss:.3f} | Train Acc: {train_acc*100:05.2f}% | Val. Loss: {valid_loss:.3f} | Val. Acc: {valid_acc*100:05.2f}% |')" 424 | ] 425 | }, 426 | { 427 | "cell_type": "code", 428 | "execution_count": 19, 429 | "metadata": {}, 430 | "outputs": [ 431 | { 432 | "name": "stdout", 433 | "output_type": "stream", 434 | "text": [ 435 | "| Test Loss: 0.052 | Test Acc: 97.93% |\n" 436 | ] 437 | } 438 | ], 439 | "source": [ 440 | "model.load_state_dict(torch.load(MODEL_SAVE_PATH))\n", 441 | "\n", 442 | "test_loss, test_acc = evaluate(model, device, valid_iterator, criterion)\n", 443 | "\n", 444 | "print(f'| Test Loss: {test_loss:.3f} | Test Acc: {test_acc*100:05.2f}% |')" 445 | ] 446 | } 447 | ], 448 | "metadata": { 449 | "kernelspec": { 450 | "display_name": "Python 3", 451 | "language": "python", 452 | "name": "python3" 453 | }, 454 | "language_info": { 455 | "codemirror_mode": { 456 | "name": "ipython", 457 | "version": 3 458 | }, 459 | "file_extension": ".py", 460 | "mimetype": "text/x-python", 461 | "name": "python", 462 | "nbconvert_exporter": "python", 463 | "pygments_lexer": "ipython3", 464 | "version": "3.7.0" 465 | } 466 | }, 467 | "nbformat": 4, 468 | "nbformat_minor": 2 469 | } 470 | -------------------------------------------------------------------------------- /misc/conv order.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "- https://blog.paperspace.com/busting-the-myths-about-batch-normalization/\n", 8 | "- http://forums.fast.ai/t/questions-about-batch-normalization/230/2\n", 9 | "- https://www.quora.com/In-most-papers-I-read-the-CNN-order-is-convolution-relu-max-pooling-So-can-I-change-the-order-to-become-convolution-max-pooling-relu\n", 10 | "\n", 11 | "Therefore, order is: `conv -> pool -> relu -> BN -> drop -> conv`" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": null, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [] 20 | } 21 | ], 22 | "metadata": { 23 | "kernelspec": { 24 | "display_name": "Python 3", 25 | "language": "python", 26 | "name": "python3" 27 | }, 28 | "language_info": { 29 | "codemirror_mode": { 30 | "name": "ipython", 31 | "version": 3 32 | }, 33 | "file_extension": ".py", 34 | "mimetype": "text/x-python", 35 | "name": "python", 36 | "nbconvert_exporter": "python", 37 | "pygments_lexer": "ipython3", 38 | "version": "3.6.5" 39 | } 40 | }, 41 | "nbformat": 4, 42 | "nbformat_minor": 2 43 | } 44 | -------------------------------------------------------------------------------- /misc/download_dogs-vs-cats.sh: -------------------------------------------------------------------------------- 1 | mkdir data 2 | mkdir data/dogs-vs-cats 3 | kaggle competitions download -c dogs-vs-cats 4 | rm sampleSubmission.csv 5 | rm test1.zip 6 | unzip train.zip 7 | mv train data/dogs-vs-cats 8 | rm train.zip 9 | mkdir data/dogs-vs-cats/train/dog 10 | mkdir data/dogs-vs-cats/train/cat 11 | mkdir data/dogs-vs-cats/valid 12 | mkdir data/dogs-vs-cats/valid/dog 13 | mkdir data/dogs-vs-cats/valid/cat 14 | mkdir data/dogs-vs-cats/test 15 | mkdir data/dogs-vs-cats/test/dog 16 | mkdir data/dogs-vs-cats/test/cat 17 | python process_dogs-vs-cats.py 18 | -------------------------------------------------------------------------------- /misc/process_dogs-vs-cats.py: -------------------------------------------------------------------------------- 1 | import os 2 | import shutil 3 | import random 4 | 5 | SEED = 1234 6 | 7 | random.seed(SEED) 8 | 9 | TRAIN_DIR = 'data/dogs-vs-cats/train/' 10 | VALID_DIR = 'data/dogs-vs-cats/valid/' 11 | TEST_DIR = 'data/dogs-vs-cats/test/' 12 | 13 | assert os.path.exists(TRAIN_DIR) 14 | assert os.path.exists(VALID_DIR) 15 | assert os.path.exists(TEST_DIR) 16 | assert os.path.exists(os.path.join(TRAIN_DIR, 'cat')), 'Run download-dogs-vs-cats.sh first!' 17 | assert os.path.exists(os.path.join(TRAIN_DIR, 'dog')), 'Run download-dogs-vs-cats.sh first!' 18 | assert os.path.exists(os.path.join(VALID_DIR, 'cat')), 'Run download-dogs-vs-cats.sh first!' 19 | assert os.path.exists(os.path.join(VALID_DIR, 'dog')), 'Run download-dogs-vs-cats.sh first!' 20 | assert os.path.exists(os.path.join(TEST_DIR, 'cat')), 'Run download-dogs-vs-cats.sh first!' 21 | assert os.path.exists(os.path.join(TEST_DIR, 'dog')), 'Run download-dogs-vs-cats.sh first!' 22 | 23 | all_images = os.listdir(TRAIN_DIR) 24 | 25 | cats = [t for t in all_images if 'cat' in t and t.endswith('.jpg')] 26 | dogs = [t for t in all_images if 'dog' in t and t.endswith('.jpg')] 27 | 28 | random.shuffle(cats) 29 | random.shuffle(dogs) 30 | 31 | n_train_examples = int(len(all_images) * 0.8) // 2 32 | n_valid_examples = int(len(all_images) * 0.1) // 2 33 | 34 | train_cats = cats[:n_train_examples] 35 | valid_cats = cats[n_train_examples:n_train_examples+n_valid_examples] 36 | test_cats = cats[n_train_examples+n_valid_examples:] 37 | 38 | train_dogs = dogs[:n_train_examples] 39 | valid_dogs = dogs[n_train_examples:n_train_examples+n_valid_examples] 40 | test_dogs = dogs[n_train_examples+n_valid_examples:] 41 | 42 | for cat in train_cats: 43 | shutil.move(os.path.join(TRAIN_DIR, cat), os.path.join(TRAIN_DIR, 'cat', cat)) 44 | 45 | for cat in valid_cats: 46 | shutil.move(os.path.join(TRAIN_DIR, cat), os.path.join(VALID_DIR, 'cat', cat)) 47 | 48 | for cat in test_cats: 49 | shutil.move(os.path.join(TRAIN_DIR, cat), os.path.join(TEST_DIR, 'cat', cat)) 50 | 51 | for dog in train_dogs: 52 | shutil.move(os.path.join(TRAIN_DIR, dog), os.path.join(TRAIN_DIR, 'dog', dog)) 53 | 54 | for dog in valid_dogs: 55 | shutil.move(os.path.join(TRAIN_DIR, dog), os.path.join(VALID_DIR, 'dog', dog)) 56 | 57 | for dog in test_dogs: 58 | shutil.move(os.path.join(TRAIN_DIR, dog), os.path.join(TEST_DIR, 'dog', dog)) --------------------------------------------------------------------------------