├── .gitignore ├── 0-basics.ipynb ├── 1-fully-connected-binarized-mnist.ipynb ├── 2-fully-connected-w1a2-mnist.ipynb ├── 3-convolutional-binarized-gtsrb.ipynb ├── 4-convolutional-mixed-imagenet.ipynb ├── 7.png ├── LICENSE ├── QNN ├── __init__.py ├── im2col.py └── layers.py ├── README.md ├── conv-lowering-chetlur.png ├── gtsrb-w1a1.pickle ├── gtsrb_images ├── 50.jpg ├── left.jpg ├── right.jpg └── stop.jpg ├── imagenet_data ├── __init__.py ├── cat.jpg ├── classes.py ├── grouse.jpg └── husky.jpg ├── mnist-w1a1.pickle └── mnist-w1a2.pickle /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | 49 | # Translations 50 | *.mo 51 | *.pot 52 | 53 | # Django stuff: 54 | *.log 55 | local_settings.py 56 | 57 | # Flask stuff: 58 | instance/ 59 | .webassets-cache 60 | 61 | # Scrapy stuff: 62 | .scrapy 63 | 64 | # Sphinx documentation 65 | docs/_build/ 66 | 67 | # PyBuilder 68 | target/ 69 | 70 | # Jupyter Notebook 71 | .ipynb_checkpoints 72 | 73 | # pyenv 74 | .python-version 75 | 76 | # celery beat schedule file 77 | celerybeat-schedule 78 | 79 | # SageMath parsed files 80 | *.sage.py 81 | 82 | # dotenv 83 | .env 84 | 85 | # virtualenv 86 | .venv 87 | venv/ 88 | ENV/ 89 | 90 | # Spyder project settings 91 | .spyderproject 92 | .spyproject 93 | 94 | # Rope project settings 95 | .ropeproject 96 | 97 | # mkdocs documentation 98 | /site 99 | 100 | # mypy 101 | .mypy_cache/ 102 | 103 | alexnet-hwgq.pickle 104 | -------------------------------------------------------------------------------- /0-basics.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Introduction\n", 8 | "\n", 9 | "Welcome! In this set of tutorials you will learn about image classification using quantized neural networks (QNNs), and what kind of computations take place.\n", 10 | "\n", 11 | "## What We Are Trying To Do\n", 12 | "\n", 13 | "For the purposes of this tutorial, we will view the QNN as a gray box. We will put in an image, do some operations, and get out a *classification result* which tells us what the QNN thinks this image is. The twenty-thousand feet view of how this goes is something like this:\n", 14 | "\n", 15 | "1. We put in an image in the form of pixels, i.e. an array of numbers.\n", 16 | "2. We multiply those pixel values (numbers) with some other numbers, which are the neural network weights, add them together, and perform some other simple operations.\n", 17 | "4. We will repeat step 2 a couple of times with different weights.\n", 18 | "3. At the end, we will obtain an array of numbers, one number for each class that the QNN knows about. The class with the largest number is the QNN's best guess on what the image is, the second largest is the second best guess, and so on.\n", 19 | "\n", 20 | "We won't concern ourselves with *where* the weights come from -- this tutorial will simply provide you with several pre-trained QNNs for that purpose. If you'd like to know more about neural networks in general, [here](https://github.com/stephencwelch/Neural-Networks-Demystified) is a popular tutorial in Jupyter Notebook form with accompanying YouTube videos, alongside countless other resources on the Internet.\n", 21 | "\n", 22 | "## OK, Let's Do It!\n", 23 | "\n", 24 | "We'll start with a classical example in neural networks: classifying 28x28 grayscale images of digits (0 to 9). Let's load an image and see what it looks like first." 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": 1, 30 | "metadata": {}, 31 | "outputs": [ 32 | { 33 | "data": { 34 | "text/plain": [ 35 | "" 36 | ] 37 | }, 38 | "execution_count": 1, 39 | "metadata": {}, 40 | "output_type": "execute_result" 41 | }, 42 | { 43 | "data": { 44 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWEAAAFfCAYAAACfj30KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJztnXus9GtV37/r3ZfZt3e/r0I9R4tWEDE1jYQeK6V6lAYS\nWkzQRqNFE4KmaaiXGJJWY2J6qLYaNRDq5TQaKWhUEkSJtgGOhQgWFU6DxXKpGOlBVDjHc8Cz7zP7\nMk//mFlz1qy9nuf3m9nz27+Z2d9P8uR3mXlnfvPO7O+s+T7rWUtSSiCEENIOt9q+AEIIuclQhAkh\npEUowoQQ0iIUYUIIaRGKMCGEtAhFmBBCWoQiTAghLUIRJoSQFqEIE0JIi6y2fQEi8jQALwHwSQDd\ndq+GEEJmwgaALwXwUErps6U7NibCIvK9AP4tgHsB/AmA708p/a/gri8B8GtNXQchhLTIdwL49dId\nGrEjROTbAbwWwAMAnoeBCD8kIk8P7v7JJq6BEELmgE9W3aEpT/jVAH4hpfQrKaU/BfAqAMcAvju4\nLy0IQsiyUqlvMxdhEVkDcB+Ad+u5NCjV9i4AL5j18xFCyCLTRCT8dAArAB5z5x/DwB8mhBAyhClq\nhBDSIk2I8BMALgDc487fA+DRBp6PEEIWlpmLcErpDMAHAbxIz4mIDI//cNbPRwghi0xTecKvA/Am\nEfkggIcxyJbYAvCmhp6PEEIWkkZEOKX0lmFO8I9iYEN8CMBLUkqPN/F8hBCyqEjbjT5F5B9iYF8Q\nQsiycV9K6Y9Ld2B2BCGEtAhFmBBCWoQiTAghLUIRJoSQFqEIE0JIi1CECSGkRSjChBDSIhRhQghp\nEYowIYS0CEWYEEJahCJMCCEtQhEmhJAWoQgTQkiLUIQJIaRFKMKEENIiFGFCCGkRijAhhLQIRZgQ\nQlqEIkwIIS1CESaEkBahCBNCSItQhAkhpEUowoQQ0iIUYUIIaRGKMCGEtAhFmBBCWoQiTAghLUIR\nJoSQFqEIE0JIi1CECSGkRSjChBDSIhRhQghpEYowIYS0CEWYEEJahCJMCCEtQhEmhJAWoQgTQkiL\nUIQJIaRFKMKEENIiFGFCCGkRijAhhLQIRZgQQlqEIkwIIS1CESaEkBaZuQiLyAMi0nfjY7N+HkII\nWQZWG3rcjwB4EQAZHp839DyEELLQNCXC5ymlxxt6bEIIWRqa8oS/XET+WkQ+ISK/KiJf3NDzEELI\nQtOECL8fwCsBvATAqwA8E8Dvi8h2A89FCCELzcztiJTSQ+bwIyLyMIC/APBtAN446+cjhJBFpvEU\ntZTSHoA/A/Dspp+LEEIWjcZFWER2AHwZgM80/VyEELJoNJEn/NMi8vUi8vdE5J8AeBsGKWpvnvVz\nEULIotNEitozAPw6gKcBeBzA+wD845TSZxt4LkIIWWiamJh7+awfkxBClhXWjiCEkBahCBNCSItQ\nhAkhpEUowoQQ0iIUYUIIaZGmqqiRKRGRmZ5TUkpXu7AajzXJc8zyeghZZCjCLSIiI+HUfT9yt926\ndSt7f4sXu6rjEiml2mPaa9D96BwhywhFuEW8oEZbv186p+dLojapyNnb+v0+Ukro9/uV+6XnKW2j\n/aprJGSRoQi3iBfSlZWVS6Kq5+y2at9HpKV9JRI5f+7i4gL9fn+0tfv+XNXz+n0/9MuEQkyWHYpw\nS9go2AtpbqyurlZuSyI8iW0Q3XZxcXFpnJ+fh+dzz1XHytBoOncthCwTFOEW8UK8uroaCqsda2tr\n2eO1tTWsrKyMCdkk/m3V/vn5+aWhQuzHJP6xtTOsWPf7/dH/E4WYLCsU4RaxdoQXWyuwa2trtcb6\n+vpIhL0Q5/brToqllHB+fo6zszOcnZ2N9u05a6f45ygdiwj6/f5oC2B0rP9PFGKyrFCEW8JHwdZy\nyAlsnbG6unopsixtgXoTZgBweno6Ely7r9d+dnY2el2R6Nut2iYqtlaALeovE7KsUIRbJLIjvOjq\nfqfTQafTGduPjldXV0fiFmUu+GOgftbC6enpaKytrY327aSgFVT/fPZ5vQD79DqNkEvpd4QsAxTh\nFrF2hPV/feTb6XSwsbFRud3Y2MDa2tqYAJZGKWMhOtfr9UZDrRIvvnrf6Ln8ObUtLi4uxkQ2pTSK\nlCnAZNmhCLdEZEdYAbYRsArs5uZmcX9zc7MowjaNzIojUC+FTK9rbW0N3W53zAO2vm3dLwHr+/rn\nogCTmwJFuEWiSFhFzoqvCmydsb6+ns3ljY7riK8ed7vd0fVp1O5FWMmJfu56bMqbX3xiByfnrsY0\n/3+5f8P3YjZQhFvEZkbYyFcF1Yqvj3jt1nrCdewIK4JA9Yo13c9ZJ/qF0e12R3ZFToCjaynlHdtj\nTX0jV8P/H/psmEnSCu2/r9qSGIpwS2hk520IFbTNzU1sbW1ha2trzHqww0bLOkm3trZWaQn4CTKg\n3hJitU1s1kan00Gv18Pp6Sk2NjZwenqKXq83kSURiS1FePbUFctJ7axo6ONRiKuhCLeItyG8CG9v\nb2N7e/tSxJvb95FwlI3gsxVyE3LR1ka/er2aruZH1aRcFA3bxR+5c/xjno6SzeT3oxWQuWE/R37f\nPzeJoQi3RJSatr6+fikKViGuSk2zKW0+Jay0BeoX+bHXafOEo1EVgdtzkdjmtvyDno66NoN9P+oM\nP8lqBVi39PLLUIRbRIW4FAnv7OyMRNimrPlFGnpO84SjaDf38xGoXrKcUsLa2tpolVyn0xnt+9Vz\n9o8zF43bY4pw8+Te++i8/0K1qyJ16KRpNMnqn5eUoQi3RBQJW4tBI2ErwtEiDp9TbFfM1RlKHRH2\noljaz9kfkSBH1kMkwvSEp8eLbmQj6DZnMflFOQBGC3N8rrc+J9MMq6EIt0huYk4zItSK2NnZubSE\nOVc/wkbCkd9XioCrjqu8wcgrrBJh//OXItwMdeYH9JxOruqEa6/XG1uYYx9To2Hd+ue0Qsz3LoYi\n3BJ+oYa3I2wkrCJsJ8V8gR9fRQ2ol3pmyf2R6PmqmXJ7vm7UFdkRUXU2ZkdcjToTpXq+1+uh2+2O\nRhQB62MCyC64UZ+YlKEIt0huYs57wrdv3x6rrhaVuLS3AeWOFn4/OvZEHuIkYlvaL4kuRXg21PkC\n1eEX5XgBtl+eQCzC/f5Ty9IpxGUowi1hly1HdoTNjtjZ2RmrMVwq7G4jYUvdcyVy9kbutqqJwciO\nKNUoVvEgk5MT4WhiTe0uvyISGBdgO1EaTfhRgOtBEW6RXPU0a0moL2zLXZa6cFjPrk0iDzo3YVhH\nfOkJXw3fhqq01QnekgDbFDW9zUfBrIBXD4pwi+gH2s5Gd7tdnJyc4Pj4GIeHh+h0OgBQ2fooEuEq\nG0InTYDxn5RV+6Vz9g+27tZeky9sZIn6zvnXFB17Eag6vk7q2EC589F1517bysoKLi4usLKyko2A\n/XHVJKxGuqenp9kqelHWBBmHItwSPqrQWWgV4aOjo1HaWUppTGit9ZAT4SqrQPftH88k+z7KmaT2\nb+l2FeBInNVjtP+Hpa1/vqrtdVAnki99adr9Ol+Gut/v90cCXHeCNRJdL8j6+FaA1WK6uLi4VGGP\nXIYi3CJWhM/OzkYifHx8PJaGpn9AXnyjY9/os2pEolo1/E/N6BjIi7c/1j9Oe5t/jFu3bmU9SHvs\n9+tcg33uJplEgOv8coiEN7c/SXZEVSRsBTmKgC8uLrC6ujomwiQPRbglrB+qdoSNhFWE9WdkNCkX\n7du2QVVZCVaErZCW9vXY7+fO5fZzEVIkwPaxS18o9v/Vi3CdCP463vM6t5VeT06Eq15bnXztkh2R\n84/tNfl/6z1lEkMRbglvR9hI2KYGiciYCJeyIrwIRxGO3y8Jq9+f5bDPHQmL7axREt9osk/P6eNU\nRfJti3AkwDkhrvsF44cX29IXdE5wI1H2/+/6WdW60Nf1f7vIUIRbJOcJ+9VJOqFSEmL7b0o5oF6c\nJxVPm50Rbf256H4+jS76WR35u5HQRsLiLY4oYo8EuUkiAc6dy325+FFlC9lzVV9cUURcFQ1bEfZZ\nE1HfQRJDEW4J+8G1doRaEPaPRz/U0WINv3BDvdOqnFD9A/LCWbUfTQTWydrQa9U/2tXV1fCPMxLk\n6P+t9PNaf37nbJTcuabf76pzk0T5VoSr7J7SL4noeVNKReGNPkv2F50NCGhHVEMRbhFvR2iKmp/o\nsB/saIVcToSjug5+PxLT3H4pOyOaJLTD2wRWIPTYns/te6tFh73d/rsomo8slusW4dxxlVWgv3S8\nCFd9yXjBtfv+uEp4o8+SBhPahVs/BxTgaijCLeF/wukH135o9fbT09NK8fUiXGdYEZ4kmq2KyFdW\nBrUwfPrT2toagHEB7vf7IyGu+nmtYpJLq7KTRfo8VbZKGyKcE+CS1WLFV79sSp69P++fp2ob2RB1\nomD9rDISrg9FuEWsHeH/WPr9/uiDrSuY7LDNNiMRLnWp0H0V4arMi6p6FdE1XVxcXOryoehr9T4u\ngDCKiyb0vMVi06XsAoGcV+3PXffEXLRvRTiaSNXXbUW47shdR+7Yf9GVbC0rwHZSmZ5wPSjCLeEj\nYesBe684EuGc+N26dat2zV8rwlVes0a3ueptOiNuRdiKLDAeAasAegHw9/HiqWKkj29F14qvPlZu\nYtCfaysS9vvR5Glku+h7l4vs/blJyEW7kQjrrzib124/i4yEq6EIt0z0s85HD/bnoRU6HeoZTxsJ\n62NOEwnnInJfdN7XQrbbukKpkXBp5t4OFeGct23PXWcknIuC7eehzqiaI+j3+6MJUEvd1xpNCvro\nOLKGosk/koci3CL+w60J7ufn52MTTVaEV1ZWRiIaCWVdT9iKsApxncm2upGzCnE0fFH6nFBGx96v\nzHnd6g3X9bsnjRYnfZ+jbXQu53dH53NfcPY4yv6w9g8Qi3LJIom+8EpiTMpQhFsiJ8A+Atah4psT\nTCsmVVkR9g+mJHxR9FgSZy/UvgB9br+uSKp9EX2hRKIAIMzkiF7DPIiw/zyUPNmU0qVmrzr0dhVg\njfQjsbUZJdF1R9dVR3h91gXJQxFukegD7iNgOzteJZTqwUX+Xc7Ty/mKuUmsumIZWRX22O5Hopgb\nKsIlmyWyI0o2S1si7MU4Sg8rvY9a/F+3tuayFWDNSokiYJtt4aPlKFtjGjH2r5WMQxFuEf+HF0XA\neluVWNpR9TPW/qFMMsPuhbm0jeyJkn9ZFVlbEa7bmVlFOGed2G2TIqzvqX1v/TnderHN/aLp9/vY\n2tpCr9fD1tbWWDdq++WjGSpVGQpegHVrBTj6koiEmJ7wZFCEW8b+gehP6JxVUWfozLn9g4j2vQiX\nUsNyuajRF4C9LSd4uQg4d3+7ryJcJwNERLKZHv74OkXYvsf+3CRWUq/XG7Wk1/cTeKpZgM3V1tdX\n8oPtbZNEwqUvefv6SMzEIiwi9wP4dwDuA/CFAL45pfQ77j4/CuBfAbgL4A8A/JuU0p9f/XKXB/8h\ntx9+nwXgxbBq+a2PXqKIRkfVctc6zxnt17EYcgJcEm0rwqUOHFaE64ymRLgU9ZZEuCrzo98fdEW2\nETAwEGC1e87Ozka3TRoN22ssecKREDMSnoxpIuFtAB8C8AYAv+VvFJEfAvB9AF4B4JMA/iOAh0Tk\n76eUTqe/1OXDfsD9sUYvOWH0w86A+z+C0ogeq/Q8uZVskXBHXnF0XDdaXV1dzYqwRoQ5ES6l0jUp\nwvb90P3onI6c6JYyXPR9tALc6XRGImwjYUtJkP21lbIjrEVhv+j96yQxE4twSumdAN4JABK/iz8A\n4MdSSv99eJ9XAHgMwDcDeMv0l7p82Ek3+wHOiR1QXTvWPnb0R+5v84/pH6vO80f3KU3u+f0qv9iO\nfr9fS4TPzs4gItnJQL9/XRNzdUQ4J7g5/xUYtyC0a7e1KvQ+0XXpe5a7Xm9HTBIJk2pm6gmLyDMB\n3Avg3XoupbQvIh8A8AJQhMfQD6n+kZSE0G6jc7kUI7uNzuX8wUmeM9rmfORo5KLTKhG2wqv7dmsj\nw6rtvImwF90oDU8jYP3Cst26vVVhv/D1PbKfAe8V+19MkedbN0OClJn1xNy9ABIGka/lseFtxFHn\ng1r62dgWVdc0iZ+sEVyddDYV4Uh0/Tm1I+wChlzO8nVMzNX5dRJleUTn9BeTTQfULt3dbhenp6dj\ndoS+JyrEJQGOrnna9DQKcTXMjlgA5vFDXHVN+oetW2u7eGGOhMn/NFcRTimF/m8UFUeRXS7VamVl\nJfdSZvL/VBUF637ks1q/FZiso4a3ivx1+UlhEbn0/6pDO4Jr7eterzc6tl+C9ouCIlzNrEX4UQAC\n4B6MR8P3APjfM34usgD4iMueUwGwedKRN6n7Kp5RJoT/mW6fw0Zw+hw+yrsOEbavI3cuiib1C0v/\nHzWy9QVzfDaK/dVh/2/9/6u3JuqIr3aB6Xa7IzH2IkwhrsdMRTil9IiIPArgRQD+DwCIyC6A5wP4\n+Vk+F1ksIl9aUTHWPGl7P/9zOKXLizVyP4u9wPvHtfvn5+cNvvrqVXO56Fjx2S8ALnnbfvGJz/X2\n15ObmKsjwlaArQirBWLfD1JmmjzhbQDPxiDiBYBnichzAXwupfSXAF4P4EdE5M8xSFH7MQB/BeC3\nZ3LFZCEoRT42AyT3kzmyJuyy5ap8VR9tW4GPxL1p3z0nvv5cbiLUnxeRUcGeXCRsxdg+V+4LUfet\nt67i6qNgFWJrSehkoP1yZBRczTSR8FcD+D0MJuASgNcOz/8ygO9OKf2UiGwB+AUMFmv8TwD/PDFH\nmCAWZ+9L6taKpE3jy/mmPhqOvFD72Fbcr2Pys44QW6+8at9POPpaGD4KttZDbmJQfxXkIuHIkoh8\n4egLkcRMkyf8XgDFqeSU0msAvGa6SyLLhv/ZW4rEvCjYyTy7+ssKsPd9fSTsr0Uf0z7udYpwad/n\nUuv1+txqzSgpFUKyE6CR2NqJSrufE+GcH2wjZj8xR0+4GmZHkEbxEz/RbVYorfhGKW0+QvYLCUp/\n+F7QbKbGdYtwdAxgZLmsro4XY/crDn2qXcmOyE3I5f7/vB1RioJVhO2/yU2SkhiKMGkMK8AlIbZW\ng94vt3IQiDsSR+fsc1jBj5aDtyHC/pzPUrAZETYnWAXYF3H3BYn8F05kRUS/InwkfHZ2FkbBKsJR\nDQ/aEfWhCJNGyQmwnYSynrAK5SSTdjlv0z62Pl/u8a+LOoKkAmxXUtrVhVaA/WKWOilqeh25BRgl\nP9hnSJyenmZX9HlriMRQhEnj2OguNxk1ydaLbmkbZRjktteNFycb+astoV8e1o7w7aKq8oStJ6zP\nayPhaFVenRQ1zYwoLTChJ1wNRZhcG7k/xNz5uuJY+gOPbmtLdKtQsdXl2dGEnYptZEX4PGHrCeci\n4UiI/Wq5yA+2kXCVL08BLkMRJnNLU3+88yQK/gvBZkHYmhBaF8K2NNra2hrt631KLeet9RAtS9bj\nk5MTnJycjHzf0tJkm7HivXkKcD0owoRcE15w/QKMSHyt8Kro6v7t27exvb09Ot/pdMa8Ypv/rILo\n7QZrNejx0dHRmBDbBRlRbYhc0R5vDZEYijAh10Adv9uK8Pr6+qg0pUa9OlR4d3Z2sL29PRYNqwj7\n9DQAY1FwbhXc6ekpjo+PcXx8fEmEbSRct2oaBbgaijAh10RuklD3/SIMWx9Yo18VXju8LbG+vj7m\nDdu0v0iEfcrZ0dHRmAjb0pi54jy0IqaHIkxIw0R1HyIh9naEdshQO0JF9/bt29jZ2RkJs7cjbKaE\ntSO8H2xFWO2Hk5OTMBL2ldKi7AcK73RQhAm5BnIFi+ywXUb8hJyK7c7ODnZ2dnD79u1R9KvbjY2N\nMU/YFu+pioSt+KonnCtVWSXAFOXJoAgTco3kxLfkCasdYSPh3d3dMZHWff13pUhYF2REInx0dDRm\nR0SlKm3bJJtK50VYoRCXoQgTcg3k7Ae7X7IjokhYBdcPmx1hPWEVzVIkrAIcTczZvnVRwfbclpSh\nCBNyTZRsCK0NYSfmNLqNIuHbt29fWjlnxyTZEVEkHKWo5UpVAvUqxJEYijAhDVOamPNV4nwkrHaD\nTsJtb29jZ2cHu7u7Y6vkov3SxFxVJOxFWKul6aRe1ZJkim99KMKEXAO56NeLsV2WXGVH+OXJfjvN\nxJyKsIpvlR3B9kVXhyJMSMNEvm+0tZNxVdtOp3Op5nLdqmnRsmVbH8JaDz43mM07Zw9FmJCG8VXQ\nfAcM3V9fX7+UbmYL9UQ1IUpF6VV4gfGu075+sK+ilmueytSzZqAIE9IgObvB5gPrvk7C+ZQzXylN\nvd4qAVYi4Y3qCFd1sKb4NgNFmJCGqaoHbLMhIhH2q+CiEpVWjHMTZTnhzQmx79nH8pTNQBEmpGF8\n5oPNfrBDbQgduUjYl6jM1Ue2+bq5mr9efOtEwhTg2UIRJqRhovQzm4JmV775Qjx28YXvIaePHeGF\nMie+pciYRdqvB4owIQ0SFefRFDS75FgFWEfkC/tIuAormLmJuai1Ua5PXLREmVwdijAhDRN5wr5e\ncJUA+4k5m/9rt7n9yBOuI8SMhJuHIkxIw+TsCG9D2LrAOUvCRsJeDHPiG3nBORsiJ8BMUWsOijAh\nDeOL80SdM6JIOBJgK8RVnSx8wfW6Qhz5wj47gsyOamOJEHIlfJ6wn5izkXCd7IhSF2Xgcj3fKDOi\nypLw96EANwcjYUKuSKmBp10VlxNf275IhTialPM1goHLXm8u2rVF2m0H5WiJsi/SU6eXHJkeijAh\nU1DqF2ePI+vBtiyy1dG0aae1JHwU7BdlRBNqNt9Xt3t7ezg4OMDh4WFYuD3qIcdVc9cDRZiQKYmq\noflzNh3NT8T5CNi2tPe1I3yRduByfeDS2N/fx/7+/kiIS408o8m5qJknmQ0UYUKmwLclyu1H6Wh2\n+Fb21orwdoRfKWcjYV8FzZaf7PV6ODg4GImwjYRL9YJzE3RktlCECZmSqIykr+dbWhnn7Yitra2x\n+1kR9hNywFORcK59vR2Hh4ejKFgtCS3cXlUvmClqzUIRJmRCIvvB1ga2+7Y2hBVhmxFhI+Fcv7jI\njrCR8NnZ2Vjretsd4+TkZCS8utU+cjk7IpfSRgGePRRhQqbAl6i0AmxTyXwkXDUxp6LrK6yVPGEb\nCUddk/2+bWsfTcydn5+PZV34fYrwbKEIEzIlvltG1O/Ne8JVE3O+3nBUuCfnCasdcXx8PLIddNjo\n2G69J3x2dhbaD7QjmoMiTMgURP3hvBDb9LRcwR5vR9juG9HIZUdEkfDBwcFoQk6tCrUforxhjYRz\nOcEU3magCBMyBXUE2Bdsz0XCtp19NNEX9ZADnoqE1Y5QcVX74fDwEPv7+9jb27vUuFNH1MhTWyL5\n4kBWiCnIs4MiTMiUqBCXBHiSibmtra2wW0aug4bPE/Z2xMHBAfb29vDkk0+OCa1PYdNjjYRzQkvh\nbQaKMCETEk3KRdFw1D3Dty3y6Wv+eSLULvDt6zUa9i3sDw8Px5Yn6749Z1PTyPVCESZkCnINPKMI\nOFr5FhXiUaKf/9F5G8lGdSC8yOZWwzHroV0owoRMQckP9hkRUYsiuwLO+7y6LY1+vz8W0VYJcp0l\nyaQdKMKETIFfqBF1UvY1gf3Ci6gkZVR+MtpeXFxkxTY3StEwhbg9KMKETEguPa0UCVsh9naE7xcX\nFWL39oEV4TqRcKmTMgW4XSjChExJToAjTziKhK0VUaoRnOuKXCW+9lydzhkU4naYuLOGiNwvIr8j\nIn8tIn0ReZm7/Y3D83a8fXaXTEj7+PS0SIh9JkTOjvBCbCNh2/HCT7pViW/JkuCk3PwwTSS8DeBD\nAN4A4Lcy93kHgFcC0K/33hTPQ8jcEtWNqBMJRw07IwH20bD1c3V4AY7EWPdzXTcoxO0zsQinlN4J\n4J0AILlERqCXUnr8KhdGyLwSVVCr4wn7uhIlS8LbESq8Gs160fX7PgL2gsvylPNDU40+Xygij4nI\nn4rIgyLy+Q09DyGtEKWolaLhOtkRShQJeyGO0tOihRjRggzbzJPF2tuniYm5dwD4TQCPAPgyAD8B\n4O0i8oLEd5osCXUXa0Qdk30UbPOEgXhiznrCOT+4KhLWx/aRLyPhdpm5CKeU3mIOPyoiHwbwCQAv\nBPB7s34+QtqgarGGzxOOVszVnZSzYmrbF/liPKXUtKgYT7Ql109TdsSIlNIjAJ4A8Oymn4uQ68D3\nkyulqOXa1kcRMIBL4mtrQmgRdt8t2VZCs7aDtxkouPNJ43nCIvIMAE8D8Jmmn4uQWZObe875wVHh\nnigzwguxtQSs/6tRr9YKtgV6JhHiSIApxvPBxCIsItsYRLX66XyWiDwXwOeG4wEMPOFHh/f7SQB/\nBuChWVwwIdeBF197XBUJq/DmesX5zsnWilAv2HrAtoGn7R03bSTs90m7TBMJfzUG3m4ajtcOz/8y\ngO8B8FUAXgHgLoBPYyC+/z6ldHblqyXkGrCCG4lxKTMiV8KyVEFNibzgnB1h+8VNEgnr81goyO0y\nTZ7we1H2kv/Z9JdDyPwQiXGduhHWF+50OrUXagC4tDjD2xHaJXkaTxigAM8jrB1BiMGKbe7cpDnC\ndZYsA7jkCefsCC3UroI8aSRM5guKMCEFfARs9316mm1XXyXCVfnBUccM6wVrB+VJI2H7XGQ+oAgT\n4ogi30iAozzhunbEysrKpZ5xQJyi5u0ITVHzXZMnnZwj8wFFmJCAkgCX6kaUUtRKVgRw2Y7wDTxt\natrh4eHYQo0qAab4zi8UYUKGlNLSvBhHFdSiSNhaEVVCXMeOsBNzUdfkSSwJMh9QhAkx1Gk3n4uA\ncxNzfkIul54GICzUo5Gu76JcamNEAV4cKMKEDIna2Eej0+lgc3MTm5ub2NjYwMbGRljAPZeSphGv\niIxazGuBnbr94nzPOF8ZjUV5FgeKMCHAWITrU8/82NjYwNbWFra2tmoJsT52lAmh21u3bmXb2Oeq\nptn6wrk29mT+oQgTMkTF0toLkeWgIqzR8ObmZjES9lkQVoBFZBQVaxv7qtb1pe7JFxcXLNS+YFCE\nCcHlBRgq3z4wAAAalUlEQVR+ks0eb25ujomwRsIaDXsRjogEMmreGdUP1nNWeFmofXGhCBMyxDbu\n9Klmdl9FeGtrKxRgnxXhI9Pc8CKca9qpx7kuzPSEFwuKMCEYX4ThsxxUXNVyiCJhvS2qH5zr6+ab\nbdrmnaUecrZbho1+2cJ+MaEIEzLE2xG2MHun0xlFvN4PLkXCa2trY14tELcvsivkSo077X7UMZkd\nlBcPijAheCoSjuwIjX43NjbGrIgoMyLnCatAKnZlnG6n6RtnJ/n8PgV4MaAIEzLENu20feI0CrYC\nHEXDuewIm4Zm84Q1EtZMh6qOyT5KBi437owGmW8owoQMsbnCPhK2Iry9vV07T1g9YRVcRSNhW7ay\nlBMcRcT6OPYxoy2ZbyjChOCyH2w9YRVZb0Xk/GDf3l7Ftk6NCN81OZcrfH5+3vL/GJkVFGGy1JSq\nodnbbcSrgmuzIDQC1mHT03z7olu3nmo8Y8U2Vw8i6iE3TY1gsphQhMlSExXfiY5txGtF2PvAKsJ2\npVwkwraDss1+sCLs6wH7ThndbpcifAOgCJOlJeoH54feZrMbomjYCrCNgn02hK+QZv1gHwn76HeS\ndkVkeaAIk6XH+r25bV0BVhG2oq1CrJGw7xnn7QhfmtKWp6QI3zwowmSp8fV/c8MKcMmSUBH2S5nV\njvCF2u2CDJsLHNUHtiJ8cnJCEb4hUITJ0lLVnt7u5yLhSIC3t7fHcol91wzrCQMYyweOPOFIhBkJ\n3xwowmSpyQmw7YRhRTi3Os4LsRdyu/XZEZN0y9B9ivDNgSJMlppSe3obyXo7IhcJqwh7X9nv+5zg\n3MScj4Qn7Z5MFh+KMFlaIisiasoZFemx0XBkSfh0N5/6Fi1PrmreOWkHZbIcUITJUhMV5okacnoR\n9naEz47wz5GjTp6wtrHXDsrsnnyzoAiTpSXKEfZ2hBdgWyeiZEfULdReWjHnF2mwg/LNhCJMlgYf\nkdp+cb4iml1ssbGxMRbp+qpodhGGnXTzzTr9tt/vj/KCq5p22voQUe84tixaXijCZGGxousF2E7G\nWdvBR7l6vL29jZ2dnbG2RXYRhuYARwsx/LCi6QvzVBXoKXVQZnW05YQiTBYSX4DHbwFcyobwE29R\nTQhbnrLT6WSXI/uoN+rzdnFxMTbBFkXEuQ7K/rEYCS8vFGGycETCG53L9YvzdYFVgHd2dkI7QvN/\n7ZJkAJcE2HbJyHXLqGppb/+9FXO2K1peKMJkIfECHB37fnFRJLyzszMSX1srOGraWbIjfIF2jWht\nJJwTXnvOi2/UvJNCvFxQhMnC4vNyvRiX+sVpBGxF2HvFuRKVShQJW193mmLtuQ7K1v4gywVFmCw0\nOSH2doRPRbNCfPv2bWxvb49lS0R2RFUk7NPQbMW0OlkRVR2UGQkvJxRhsnD4qNefi+pFqBB7Ad7Z\n2RmJsAq1FeyqiTk/Keej4EnT06J0N79PlguKMFlYctGvXSUXTcxFQqwi7FfS2V5xkR0R1Qv2TTv9\nMuSSJVF3EQhZHijCZKHIZUZURcIlAdZI2FdWK5WorLIjqqyIyA/2HZTZPflmQBEmC4sXYtuuKGdH\nlCJhW44yGr5OcDQxV+qeXCdVjdw8KMJkobBZD1VDJ918Glq0LNlGu9bSUGzTTtuq3guulqLULfvG\nkSoowmShsF6vtQui7fb2Nu7cuYPd3d3sQoxJBNhOmEVlKW1VNFuiUjtmUIRJBEWYLBQ29Uwnz3JD\nRVijYb8Ywxbn0dVwkQD3+/3R/q1bt8KylFFVNDvYN47koAiThSG3Cs7n9trKaLdv3x6LhG27+mg1\nnBdi2ydO/eCqVkW+W4aNhLvdLkWYjEERJnOPzwfWymhalMeudPN1gL0nrLf5CmkrKyuXnktRO0Jv\ni/KBrR1hbYioeWev17skwuTmcqv6Lk8hIj8sIg+LyL6IPCYibxOR57j7dETk50XkCRE5EJG3isgX\nzPayyU0hKlHpI2FbB0Ij37t37+Lu3bvY3d0dCbGNhCM7wlsSPjc3lwWhwmr9YBXgw8PDsd5xtCOI\nZyIRBnA/gJ8F8HwALwawBuB3RWTT3Of1AL4RwLcA+HoAXwTgN69+qeSmEUWlOnnmc3/tEuQ7d+7g\nzp07uHv37qWJucgTjibmLNHquCgzIuoZp5GwRsM2ZY0iTIAJ7YiU0kvtsYi8EsDfALgPwPtEZBfA\ndwP4lyml9w7v810A/q+IfE1K6eGZXDVZeiIBjlbC+aLst2/fHg2tB+GbdnpPWAXYTsIBGIuC7bLh\nUqsi7wcfHh6ORctR3ziugrvZXNUTvgsgAfjc8Pi+4WO+W++QUvq4iHwKwAsAUIRJJTkB1m20Cs5H\nwru7u9je3r7UP87u+0hYBdh3sfCFdKLOyaWJuVIXDUbCZGoRlsFfxesBvC+l9LHh6XsBnKaU9t3d\nHxveRsjE5Cbmqjzhra2tsaI80bB1IRQ7CafHtqxkbnmyTVGzkbAvcelbGFGEbzZXiYQfBPCVAL5u\nRtdCSDgRZ7GesM2OsCKsfvDW1tYoi8IPW9hHJ+R8FgRQXbw9WqzhsyOijhu+fRFF+OYylQiLyM8B\neCmA+1NKnzY3PQpgXUR2XTR8z/A2QrJU9Y2LVsv5qmc+b1jv64vx+KI8Ni/YZkBEQ7MfbDaEirDN\ngNAR9Z/zNYLJzWViER4K8DcB+IaU0qfczR8EcA7gRQDeNrz/VwD4EgB/dLVLJctMrkWRP+frQ1iR\njQTXDlvYJ1qU4f1emwFhj20KWq4uhLcbWKSd5JhIhEXkQQAvB/AyAEcics/wpr2UUjeltC8ibwDw\nOhH5WwAHAH4GwB8wM4LUIVq5ZocX30iIvQDnqqFZgc9NutmoVvd9+pkX4SgPOBcBs0wlmTQSfhUG\n2RDvcee/C8CvDPdfDeACwFsBdAC8E8D3Tn+JZNkp1QT2+9ZKKEXA1u/1Apwr1ONF2E642a0tymNF\nWG/3reujlkXslkGUSfOEKxd3pJR6AL5/OAipTVSU3dcJtpFwTpBtJKzCmxNga0dYPzhXHU2zH+pE\nwlG3ZC/GhLB2BJkLSgJsh7UXSjZEtBLOD5954VfE2RrBNuq1hXmsONu6EFEKWtQ7jpEwoQiTuSGy\nI7yIej+4FAWvra2FQh5ZH9HEXLQU2QqwjYSjJcl2RZwXXloRRKEIk9aJsiC8cFpboU5mhIqxtzO8\nx2yJCrZbIdbFF4eHh5ei41J2BJt3khIUYTIXRHaEFWArwjnxjUYp06IqRU09YbsM+eDg4FKOsJ+Y\n856wz4KI9snNhSJM5obIjoisiKpo2C7i0Mf1W38OKGdHqAj70pTRIg2fHRGloeX2yc2DIkzmgjqR\ncCS+JU94dXX10nPkKGVH2EhY7QifvlZVJY2QHBRhcq1EQmiFNVrpZs9tb2+PNezUkpS+QHuuPb23\nAvTYdkius/ULOXKr5CjApAqKMGmU6Ge/P6dCq0Lqt3Z/d3d3rFuGFmmP+sX54jvRognd73a72awH\nbztEwuvT0Si+pC4UYdIYOQ82qgdhS1PaQjz+2DfutC3srQgrNve3NE5OTkZWQ6lTss8Fzi1PphCT\nulCESaPUyU6wkbCtgBaNnZ2dUSRse8b5LhlR7q+tAxxVRtNJt1I0HKWhRUuUKcCkLhRh0gh16kF4\nEVYB9h2T7bF2TvYi7CPhSIR9NTRbIc2LcE6AdXgR97WBKcKkLhRh0iiRAPutj4RVfFVk7dbveztC\nI2Gl1I7I7vvylN6K8JFwqVA7hZhMAkWYNIb1fn0tCLtvRVgjYRVajXx1f2trayw6tnaEtivKRcJ2\nAYavjqZFeaIOyVE07Iu0RwXbCakDRZg0Sin/10bC3g+2Imw7KKvo2qGTdn5izuf+2gUYNt1M84An\nsSOs2FbVCiakBEWYNEbJF/aLMbwdYQV4d3d31EFZ09F8JoXPE9YoXEXSesC2IaetjBZNzEVifHp6\nGnbJ8FtC6kARJo3g09BKRXlKdoQV4Tt37mBjYyPbvLNqYi4qT6liG4lvKRpmUR4yKyjCpDFKnrCv\nB+GzI3z35Dt37uDzPu/zsLGxcamwuy/ubu0Inx3hy1P6CDiXI+xFWIlqQFB8ySRQhEljeNGNRHNl\nZWVsYUYuT1gn4jqdTljaUvd9yyIrxLksicjzLS1Jvri4aPF/lSwbFGHSCL4we1QTQoefYLO1IHwB\n96pWRTb69taAzWgodVT26Wa+5gQhs4QiTBrBCnCuzKQOW4zHTrJZn9cLca5XXBQJ2wk0n9sbCbAO\nZjuQ64AiTBrBR8J28s1Guuvr62MLLmw0bEW41Csu6pxs8elkk0TCPhqmEJNZQxEmjeG7I0dFejQj\nwkfCdvFFFAnn2hVFLYu8L2xrSeQiYQowuS4owqQR7KScFWFfqEfzgq0A5yJhGw3nalHocwPjNYNt\nNOyL+VghztWDYO4vaQqKMGkEK5LWD44qpU3qCdfpGad4Ac7ZEVFpyigSphCTWUMRJo1g/eDIE/YV\n0ybJjqgrwACK2RE66pampACTJqAIk8bITczZSFgFOBLiyI6wS5IjC8Kei+yIqLYwJ+ZIm1CESSPk\nImFfI6IUCecm5uxz+P0oGq6yIzgxR9qEIkwaIUpRs9XSIjvCZ0fkJub885SIsiN8nnBdISakCSjC\nZGoiAdRzVnhVTP3SZFsPOLIiogk5L8IlSoV1oj5zUSNQGwEzGiZNQBEmE1HVtFP37QScTUfz7Yt8\nJOwn5fxkHCHLBkWY1KZuz7hbt25dyoSI0tI0ErZ94mxxdi/ChCwjFGEyEVWtinTfC7DPiLACrNXR\novQ0ijBZdijCpDa5Au1RXQe/PNnaEV6Ic3UjtDYwRZgsMxRhMhF+OXJU03dlZaVWJGz9YF/YJ7dU\nmZBlgyJMahNFwlGRdtszLvKEvR2hPrDNI6YdQW4KFGEyET7/17cW0v2SAHsh1kUZdkLOR8MUYbKs\nUIRJbXJ+sM8J1kUZ3o7Ipaitr69fEnO7ZSRMlhmKMJmIXCRsF2XYlXG5qmm2o3KufVFUupKQZYMi\nTGpTJxK2UXBuYs5Hwz4LIrdPyDJCESYTUScS9pNs0bDivLq6Wlkj2FZGq0NpyXLuMbgkmbQBRZjU\npq4nHBVk99kTVXWB60S+UQdkey5XEyIakUj7fUKagCJMJqJOJOwroEUTbbluydFzRd0ydFvaVwHO\nFekpRcks1kOuC4owqc0kkXBOiEtRsH0OL7yREFeNqgppJSHORdeEzBqKMJmIaLFGToBznZKjaNg+\ntn2uiEgwvaDaCFj3q4TYPrbfJ6QpKMKkNrnVcpN6wpEdoY9vn8eeU7xAejG12ygSrmtJ+EiYYkya\non6FbAAi8sMi8rCI7IvIYyLyNhF5jrvPe0Skb8aFiDw428smbVFathxZEWtra2NiHeX+lqyJiEiI\noxZGdQq35zops7syuS4mEmEA9wP4WQDPB/BiAGsAfldENs19EoBfBHAPgHsBfCGAH7z6pZK2KQlw\nFAnb81F2RDQxFwlwJMZ1u2XUEd5cdgQh18FEdkRK6aX2WEReCeBvANwH4H3mpuOU0uNXvjoyd1QJ\ncc6OKFkRJcGt8oVzKWfeD/ZNO+vYERRlch1MGgl77mIQ+X7Onf9OEXlcRD4sIj/uImWyoNQVYG9H\n+Gg4J8RejOtOzNWxInJ+cFWqmn1OQppg6ok5GfyFvB7A+1JKHzM3/RqAvwDwaQBfBeCnADwHwLde\n4TrJnHDVSNinqU2yJFlExroe1xXiSQXYP749JmTWXCU74kEAXwnga+3JlNIvmcOPisijAN4lIs9M\nKT1yhecjLWMFuEqIc8IbpabVJUo/8y3s7fb09HQ0tI29HTmfOEpbI6QpphJhEfk5AC8FcH9K6TMV\nd/8AAAHwbAAUYZLFi160vbi4wNnZGc7Pz3F2dlbcPz4+xt7eHvb397G/v4/Dw0McHR3h5OQE3W4X\nvV5vJNBWvHPibK+FkFkxsQgPBfibAHxDSulTNf7J8zDwjavEmtxgIksgsgrOz8/HItzT01OcnZ2N\nBPXs7Gx0/ujoCAcHB6OhInx8fDwSYSve0QQeJ+hI00wkwsN835cDeBmAIxG5Z3jTXkqpKyLPAvAd\nAN4O4LMAngvgdQDem1L6yOwumywbVSvg9JyKbK/XQ6/XG4mpHXru+PgYh4eHo6ECnIuES9kUFGDS\nFJNGwq/CIKp9jzv/XQB+BcApBvnDPwBgG8BfAvgNAP/pSldJlhob+UYr3+y+Rr3dbhfdbnckqH7b\n7XZxfHw8Et6jo6PRvo2ErV9ct7oaIbNk0jzhYkpbSumvALzwKhdEbia5TAc/bCR8cnIyimxVXO2x\n3fp9a0fkvGAvxoQ0AWtHkLkgl2rm91WENeq1Ua4fNiqOhvWRz8/PsxG496gJmSUUYdI6kfebqwWs\nUbAXYZ140+3h4SFOTk7G/ONo30bCkRdNT5g0DUWYzA0+Go5WvkWRsIrvwcEB9vf3R/snJyej+2sW\nRLSvnnApK4NCTJqCIkzmAh95WiG2ebyRJ3x0dITDw0Ps7++P8oL39vbQ7XbDRRr6OH7hRmQ70Iog\nTUMRJq1TEmC7Kk7FM2dHqPg++eSTePLJJ9Hr9cLly7l0NH9NpWNCZgVFmExErr6CtxJKmQ5eBHUl\nnI12c9GrzQU+OTkZRcFqSejqOI2ES+lu9hwhbUERJrWxk2Z2eXCv1xsrymPv78VVRVSj2O3t7dFK\nODts9GvPHR4eYm9vD3t7ezg4OLi0DFkFPBLb3HJoQtqEIkxqE4nw6enppYI8NgqObIRut4utra3R\npJqKta/hEJ2z1oNmQmj+r51o00i7apKNQkzahiJMahOJsI+A7X00m8FmNFgbYXNzE5ubm5dEuDRO\nTk7GUtGiWhA2Eo7SzuzrIaRtKMKkNpEIqwD7HF+NgLWYTq/XQ6fTwcnJCTqdDjY2NkZbFeFSRwzd\nt8uRc8uQq4rwMNOBzBMUYVIbL7K5CNhaEDadrNPpYH19/dIWQLEjsj32y5XtyEXCFGAyz1CESW28\n0FZZEGtra+h2u6N2R36r+1EWRS6zwq6Y0+jX7lsRzuX+RseEtAVFmNTGim0uAj47O8Pq6ipOT08v\nddrI7QPIpo/5rVocaj1EtYXtMuTcwgv7mghpE4owqY0VXH98fn5+qZ2RbXtUOtbHyk2k2WGtjlxX\nDWtf6GNHW0LmAYowqY0KpN2/uLi41DvOjqi3nL9NHy+KXP2xX8pc6q5clQ1BMSbzAEWY1MZHq75d\nfW5Ere2jNvdVdkFuZV5uS9EliwBFmNSGP+cJmT3FThmEEEKahSJMCCEtQhEmhJAWoQgTQkiLzIMI\nb7R9AYQQ0hCV+jYPIvylbV8AIYQ0xJdW3UHaTjcSkacBeAmATwLotnoxhBAyGzYwEOCHUkqfLd2x\ndREmhJCbzDzYEYQQcmOhCBNCSItQhAkhpEUowoQQ0iJzKcIi8r0i8oiInIjI+0XkH7V9TbNARB4Q\nkb4bH2v7uqZBRO4Xkd8Rkb8evo6XBff5URH5tIgci8j/EJFnt3Gt01D1+kTkjcF7+fa2rrcuIvLD\nIvKwiOyLyGMi8jYReY67T0dEfl5EnhCRAxF5q4h8QVvXPAk1X9973Pt2ISIPtnXNcyfCIvLtAF4L\n4AEAzwPwJwAeEpGnt3phs+MjAO4BcO9wfF27lzM12wA+BOB7AFxKsRGRHwLwfQD+NYCvAXCEwfu4\nfp0XeQWKr2/IOzD+Xr78ei7tStwP4GcBPB/AiwGsAfhdEdk093k9gG8E8C0Avh7AFwH4zWu+zmmp\n8/oSgF/EU+/dFwL4wWu+TnM1hWaIbQwA7wfwn82xAPgrAD/Y9rXN4LU9AOCP276OBl5XH8DL3LlP\nA3i1Od4FcALg29q+3hm9vjcC+K22r20Gr+3pw9f3deZ96gH4F+Y+XzG8z9e0fb1XfX3Dc78H4HVt\nX5uOuYqERWQNwH0A3q3n0uB/7V0AXtDWdc2YLx/+xP2EiPyqiHxx2xc0a0TkmRhEGPZ93AfwASzP\n+wgALxz+5P1TEXlQRD6/7QuagrsYRIafGx7fh0GdcfvefRzAp7CY751/fcp3isjjIvJhEflxFylf\nK/NW1P3pAFYAPObOP4bBt/Gi834ArwTwcQx+Ar0GwO+LyD9IKR21eF2z5l4MPvjR+3jv9V9OI7wD\ng5/ojwD4MgA/AeDtIvKCYeAw98igrcnrAbwvpaRzE/cCOB1+aVoW7r3LvD4A+DUAf4HBr7WvAvBT\nAJ4D4Fuv/SIxfyK81KSUHjKHHxGRhzH4MHwbBj9vyYKQUnqLOfyoiHwYwCcAvBCDn7uLwIMAvhKL\nOy9Rhb6+r7UnU0q/ZA4/KiKPAniXiDwzpfTIdV4gMH8Tc08AuMDAMLfcA+DR67+cZkkp7QH4MwAL\nkzVQk0cx8PJvxPsIAMM/3iewIO+liPwcgJcCeGFK6dPmpkcBrIvIrvsnC/Xeudf3mYq7fwCDz2sr\n791ciXBK6QzABwG8SM8Nf1K8CMAftnVdTSEiOxj8lK36kCwUQ0F6FOPv4y4GM9ZL9z4CgIg8A8DT\nsADv5VCgvgnAP00pfcrd/EEA5xh/774CwJcA+KNru8grUPH6Ip6HgX3Wyns3j3bE6wC8SUQ+COBh\nAK8GsAXgTW1e1CwQkZ8G8N8wsCD+LoD/gMEH/s1tXtc0iMg2BpGDtkt+log8F8DnUkp/iYEX9yMi\n8ucYVMj7MQyyXH67hcudmNLrG44HMPCEHx3e7ycx+FXz0OVHmx+G+bAvB/AyAEcior9W9lJK3ZTS\nvoi8AcDrRORvARwA+BkAf5BSeridq65P1esTkWcB+A4AbwfwWQDPxUBz3ptS+kgb19x6ekYmreR7\nMPjDPcHg2/er276mGb2uN2MgRCcYzDb/OoBntn1dU76Wb8Ag9efCjf9q7vMaDCY/jjEQp2e3fd2z\neH0YlCl8JwYC3AXw/wD8FwB/p+3rrvG6otd0AeAV5j4dDHJtn8BAhH8DwBe0fe2zeH0AngHgPQAe\nH34uP47BpOpOW9fMUpaEENIic+UJE0LITYMiTAghLUIRJoSQFqEIE0JIi1CECSGkRSjChBDSIhRh\nQghpEYowIYS0CEWYEEJahCJMCCEtQhEmhJAWoQgTQkiL/H9Kx/J9nkQFJgAAAABJRU5ErkJggg==\n", 45 | "text/plain": [ 46 | "" 47 | ] 48 | }, 49 | "metadata": {}, 50 | "output_type": "display_data" 51 | } 52 | ], 53 | "source": [ 54 | "from PIL import Image\n", 55 | "from matplotlib.pyplot import imshow\n", 56 | "import numpy as np\n", 57 | "\n", 58 | "# load image using PIL\n", 59 | "img = Image.open(\"7.png\")\n", 60 | "# convert to black and white\n", 61 | "img = img.convert(\"L\")\n", 62 | "# convert to numpy array\n", 63 | "img = np.asarray(img)\n", 64 | "# display\n", 65 | "% matplotlib inline\n", 66 | "imshow(img, cmap='gray')" 67 | ] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "metadata": {}, 72 | "source": [ 73 | "Looks like a seven to me, but to get a useful reminder of what images look like to a computer by default, let's have a look at the numpy array itself:" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": 2, 79 | "metadata": {}, 80 | "outputs": [ 81 | { 82 | "name": "stdout", 83 | "output_type": "stream", 84 | "text": [ 85 | "(28, 28)\n" 86 | ] 87 | }, 88 | { 89 | "data": { 90 | "text/plain": [ 91 | "array([[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 92 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 93 | " 0, 0],\n", 94 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 95 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 96 | " 0, 0],\n", 97 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 98 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 99 | " 0, 0],\n", 100 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 101 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 102 | " 0, 0],\n", 103 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 104 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 105 | " 0, 0],\n", 106 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 107 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 108 | " 0, 0],\n", 109 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 110 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 111 | " 0, 0],\n", 112 | " [ 0, 0, 0, 0, 0, 0, 84, 185, 159, 151, 60, 36, 0,\n", 113 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 114 | " 0, 0],\n", 115 | " [ 0, 0, 0, 0, 0, 0, 222, 254, 254, 254, 254, 241, 198,\n", 116 | " 198, 198, 198, 198, 198, 198, 198, 170, 52, 0, 0, 0, 0,\n", 117 | " 0, 0],\n", 118 | " [ 0, 0, 0, 0, 0, 0, 67, 114, 72, 114, 163, 227, 254,\n", 119 | " 225, 254, 254, 254, 250, 229, 254, 254, 140, 0, 0, 0, 0,\n", 120 | " 0, 0],\n", 121 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 17, 66,\n", 122 | " 14, 67, 67, 67, 59, 21, 236, 254, 106, 0, 0, 0, 0,\n", 123 | " 0, 0],\n", 124 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 125 | " 0, 0, 0, 0, 0, 83, 253, 209, 18, 0, 0, 0, 0,\n", 126 | " 0, 0],\n", 127 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 128 | " 0, 0, 0, 0, 22, 233, 255, 83, 0, 0, 0, 0, 0,\n", 129 | " 0, 0],\n", 130 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 131 | " 0, 0, 0, 0, 129, 254, 238, 44, 0, 0, 0, 0, 0,\n", 132 | " 0, 0],\n", 133 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 134 | " 0, 0, 0, 59, 249, 254, 62, 0, 0, 0, 0, 0, 0,\n", 135 | " 0, 0],\n", 136 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 137 | " 0, 0, 0, 133, 254, 187, 5, 0, 0, 0, 0, 0, 0,\n", 138 | " 0, 0],\n", 139 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 140 | " 0, 0, 9, 205, 248, 58, 0, 0, 0, 0, 0, 0, 0,\n", 141 | " 0, 0],\n", 142 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 143 | " 0, 0, 126, 254, 182, 0, 0, 0, 0, 0, 0, 0, 0,\n", 144 | " 0, 0],\n", 145 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 146 | " 0, 75, 251, 240, 57, 0, 0, 0, 0, 0, 0, 0, 0,\n", 147 | " 0, 0],\n", 148 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 149 | " 19, 221, 254, 166, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 150 | " 0, 0],\n", 151 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3,\n", 152 | " 203, 254, 219, 35, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 153 | " 0, 0],\n", 154 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 38,\n", 155 | " 254, 254, 77, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 156 | " 0, 0],\n", 157 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31, 224,\n", 158 | " 254, 115, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 159 | " 0, 0],\n", 160 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 133, 254,\n", 161 | " 254, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 162 | " 0, 0],\n", 163 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 61, 242, 254,\n", 164 | " 254, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 165 | " 0, 0],\n", 166 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 121, 254, 254,\n", 167 | " 219, 40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 168 | " 0, 0],\n", 169 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 121, 254, 207,\n", 170 | " 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 171 | " 0, 0],\n", 172 | " [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 173 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 174 | " 0, 0]], dtype=uint8)" 175 | ] 176 | }, 177 | "execution_count": 2, 178 | "metadata": {}, 179 | "output_type": "execute_result" 180 | } 181 | ], 182 | "source": [ 183 | "print(img.shape)\n", 184 | "img" 185 | ] 186 | }, 187 | { 188 | "cell_type": "markdown", 189 | "metadata": {}, 190 | "source": [ 191 | "It's all just numbers in a 28x28 array! Now let's see what the neural network says about this data. We will start by loading the QNN from the file it is stored in which is a [Python Pickle](https://wiki.python.org/moin/UsingPickle)." 192 | ] 193 | }, 194 | { 195 | "cell_type": "code", 196 | "execution_count": 3, 197 | "metadata": {}, 198 | "outputs": [ 199 | { 200 | "data": { 201 | "text/plain": [ 202 | "[,\n", 203 | " ,\n", 204 | " ,\n", 205 | " ,\n", 206 | " ,\n", 207 | " ,\n", 208 | " ,\n", 209 | " ,\n", 210 | " ,\n", 211 | " ]" 212 | ] 213 | }, 214 | "execution_count": 3, 215 | "metadata": {}, 216 | "output_type": "execute_result" 217 | } 218 | ], 219 | "source": [ 220 | "from QNN.layers import *\n", 221 | "import pickle\n", 222 | "\n", 223 | "qnn = pickle.load(open(\"mnist-w1a1.pickle\", \"rb\"))\n", 224 | "qnn" 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "metadata": {}, 230 | "source": [ 231 | "As you can see, the QNN consists of several *layers*. The QNN we loaded seems to contain four types of layers: BipolarThresholding, FullyConnected, ScaleShift and Softmax. We will cover what all these do in more detail later on. Right now, let's just see if it works! The QNN module that we just imported contains a function called predict:" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": 4, 237 | "metadata": {}, 238 | "outputs": [ 239 | { 240 | "name": "stdout", 241 | "output_type": "stream", 242 | "text": [ 243 | "[ 1.72944994e-04 6.73230017e-05 9.88439418e-05 1.12978784e-04\n", 244 | " 1.40739136e-04 3.00394310e-05 8.62074803e-06 9.98436276e-01\n", 245 | " 4.02650406e-06 9.28207609e-04]\n", 246 | "The QNN predicts this is a 7 with 99.843628 percent probability\n" 247 | ] 248 | } 249 | ], 250 | "source": [ 251 | "# get the predictions array\n", 252 | "res = predict(qnn, img)\n", 253 | "# return the index of the largest prediction\n", 254 | "winner_ind = np.argmax(res)\n", 255 | "# the sum of the output values add up to 1 due to softmax,\n", 256 | "# so we can interpret them as probabilities\n", 257 | "winner_prob = 100 * res[winner_ind]\n", 258 | "print(res)\n", 259 | "print(\"The QNN predicts this is a %d with %f percent probability\" % (winner_ind, winner_prob))" 260 | ] 261 | }, 262 | { 263 | "cell_type": "markdown", 264 | "metadata": {}, 265 | "source": [ 266 | "And our first image classification with a QNN is a success! In the following section, we will take a closer look at the computation that is taking place inside the .execute() functions for this network, and later we will cover more advanced types of networks." 267 | ] 268 | } 269 | ], 270 | "metadata": { 271 | "kernelspec": { 272 | "display_name": "Python 2", 273 | "language": "python", 274 | "name": "python2" 275 | }, 276 | "language_info": { 277 | "codemirror_mode": { 278 | "name": "ipython", 279 | "version": 2 280 | }, 281 | "file_extension": ".py", 282 | "mimetype": "text/x-python", 283 | "name": "python", 284 | "nbconvert_exporter": "python", 285 | "pygments_lexer": "ipython2", 286 | "version": "2.7.12" 287 | } 288 | }, 289 | "nbformat": 4, 290 | "nbformat_minor": 2 291 | } 292 | -------------------------------------------------------------------------------- /1-fully-connected-binarized-mnist.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# A Fully-Connected, Binarized Network\n", 8 | "\n", 9 | "In the previous section we successfully classified a single handwritten digit using a *binarized neural network*. Binarized Neural Networks (BNNs) are a subclass of QNNs, which use a single bit representing -1 or +1 for each weight and each activation. We will now go into more details of what happens inside such a network when we run an image through it.\n", 10 | "\n", 11 | "Let's start by re-loading our neural network and our test image, so that they are readily available." 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": 42, 17 | "metadata": {}, 18 | "outputs": [ 19 | { 20 | "data": { 21 | "text/plain": [ 22 | "[,\n", 23 | " ,\n", 24 | " ,\n", 25 | " ,\n", 26 | " ,\n", 27 | " ,\n", 28 | " ,\n", 29 | " ,\n", 30 | " ,\n", 31 | " ]" 32 | ] 33 | }, 34 | "execution_count": 42, 35 | "metadata": {}, 36 | "output_type": "execute_result" 37 | }, 38 | { 39 | "data": { 40 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWEAAAFfCAYAAACfj30KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJztnXus9GtV37/r3ZfZt3e/r0I9R4tWEDE1jYQeK6V6lAYS\nWkzQRqNFE4KmaaiXGJJWY2J6qLYaNRDq5TQaKWhUEkSJtgGOhQgWFU6DxXKpGOlBVDjHc8Cz7zP7\nMk//mFlz1qy9nuf3m9nz27+Z2d9P8uR3mXlnfvPO7O+s+T7rWUtSSiCEENIOt9q+AEIIuclQhAkh\npEUowoQQ0iIUYUIIaRGKMCGEtAhFmBBCWoQiTAghLUIRJoSQFqEIE0JIi6y2fQEi8jQALwHwSQDd\ndq+GEEJmwgaALwXwUErps6U7NibCIvK9AP4tgHsB/AmA708p/a/gri8B8GtNXQchhLTIdwL49dId\nGrEjROTbAbwWwAMAnoeBCD8kIk8P7v7JJq6BEELmgE9W3aEpT/jVAH4hpfQrKaU/BfAqAMcAvju4\nLy0IQsiyUqlvMxdhEVkDcB+Ad+u5NCjV9i4AL5j18xFCyCLTRCT8dAArAB5z5x/DwB8mhBAyhClq\nhBDSIk2I8BMALgDc487fA+DRBp6PEEIWlpmLcErpDMAHAbxIz4mIDI//cNbPRwghi0xTecKvA/Am\nEfkggIcxyJbYAvCmhp6PEEIWkkZEOKX0lmFO8I9iYEN8CMBLUkqPN/F8hBCyqEjbjT5F5B9iYF8Q\nQsiycV9K6Y9Ld2B2BCGEtAhFmBBCWoQiTAghLUIRJoSQFqEIE0JIi1CECSGkRSjChBDSIhRhQghp\nEYowIYS0CEWYEEJahCJMCCEtQhEmhJAWoQgTQkiLUIQJIaRFKMKEENIiFGFCCGkRijAhhLQIRZgQ\nQlqEIkwIIS1CESaEkBahCBNCSItQhAkhpEUowoQQ0iIUYUIIaRGKMCGEtAhFmBBCWoQiTAghLUIR\nJoSQFqEIE0JIi1CECSGkRSjChBDSIhRhQghpEYowIYS0CEWYEEJahCJMCCEtQhEmhJAWoQgTQkiL\nUIQJIaRFKMKEENIiFGFCCGkRijAhhLQIRZgQQlqEIkwIIS1CESaEkBaZuQiLyAMi0nfjY7N+HkII\nWQZWG3rcjwB4EQAZHp839DyEELLQNCXC5ymlxxt6bEIIWRqa8oS/XET+WkQ+ISK/KiJf3NDzEELI\nQtOECL8fwCsBvATAqwA8E8Dvi8h2A89FCCELzcztiJTSQ+bwIyLyMIC/APBtAN446+cjhJBFpvEU\ntZTSHoA/A/Dspp+LEEIWjcZFWER2AHwZgM80/VyEELJoNJEn/NMi8vUi8vdE5J8AeBsGKWpvnvVz\nEULIotNEitozAPw6gKcBeBzA+wD845TSZxt4LkIIWWiamJh7+awfkxBClhXWjiCEkBahCBNCSItQ\nhAkhpEUowoQQ0iIUYUIIaZGmqqiRKRGRmZ5TUkpXu7AajzXJc8zyeghZZCjCLSIiI+HUfT9yt926\ndSt7f4sXu6rjEiml2mPaa9D96BwhywhFuEW8oEZbv186p+dLojapyNnb+v0+Ukro9/uV+6XnKW2j\n/aprJGSRoQi3iBfSlZWVS6Kq5+y2at9HpKV9JRI5f+7i4gL9fn+0tfv+XNXz+n0/9MuEQkyWHYpw\nS9go2AtpbqyurlZuSyI8iW0Q3XZxcXFpnJ+fh+dzz1XHytBoOncthCwTFOEW8UK8uroaCqsda2tr\n2eO1tTWsrKyMCdkk/m3V/vn5+aWhQuzHJP6xtTOsWPf7/dH/E4WYLCsU4RaxdoQXWyuwa2trtcb6\n+vpIhL0Q5/brToqllHB+fo6zszOcnZ2N9u05a6f45ygdiwj6/f5oC2B0rP9PFGKyrFCEW8JHwdZy\nyAlsnbG6unopsixtgXoTZgBweno6Ely7r9d+dnY2el2R6Nut2iYqtlaALeovE7KsUIRbJLIjvOjq\nfqfTQafTGduPjldXV0fiFmUu+GOgftbC6enpaKytrY327aSgFVT/fPZ5vQD79DqNkEvpd4QsAxTh\nFrF2hPV/feTb6XSwsbFRud3Y2MDa2tqYAJZGKWMhOtfr9UZDrRIvvnrf6Ln8ObUtLi4uxkQ2pTSK\nlCnAZNmhCLdEZEdYAbYRsArs5uZmcX9zc7MowjaNzIojUC+FTK9rbW0N3W53zAO2vm3dLwHr+/rn\nogCTmwJFuEWiSFhFzoqvCmydsb6+ns3ljY7riK8ed7vd0fVp1O5FWMmJfu56bMqbX3xiByfnrsY0\n/3+5f8P3YjZQhFvEZkbYyFcF1Yqvj3jt1nrCdewIK4JA9Yo13c9ZJ/qF0e12R3ZFToCjaynlHdtj\nTX0jV8P/H/psmEnSCu2/r9qSGIpwS2hk520IFbTNzU1sbW1ha2trzHqww0bLOkm3trZWaQn4CTKg\n3hJitU1s1kan00Gv18Pp6Sk2NjZwenqKXq83kSURiS1FePbUFctJ7axo6ONRiKuhCLeItyG8CG9v\nb2N7e/tSxJvb95FwlI3gsxVyE3LR1ka/er2aruZH1aRcFA3bxR+5c/xjno6SzeT3oxWQuWE/R37f\nPzeJoQi3RJSatr6+fikKViGuSk2zKW0+Jay0BeoX+bHXafOEo1EVgdtzkdjmtvyDno66NoN9P+oM\nP8lqBVi39PLLUIRbRIW4FAnv7OyMRNimrPlFGnpO84SjaDf38xGoXrKcUsLa2tpolVyn0xnt+9Vz\n9o8zF43bY4pw8+Te++i8/0K1qyJ16KRpNMnqn5eUoQi3RBQJW4tBI2ErwtEiDp9TbFfM1RlKHRH2\noljaz9kfkSBH1kMkwvSEp8eLbmQj6DZnMflFOQBGC3N8rrc+J9MMq6EIt0huYk4zItSK2NnZubSE\nOVc/wkbCkd9XioCrjqu8wcgrrBJh//OXItwMdeYH9JxOruqEa6/XG1uYYx9To2Hd+ue0Qsz3LoYi\n3BJ+oYa3I2wkrCJsJ8V8gR9fRQ2ol3pmyf2R6PmqmXJ7vm7UFdkRUXU2ZkdcjToTpXq+1+uh2+2O\nRhQB62MCyC64UZ+YlKEIt0huYs57wrdv3x6rrhaVuLS3AeWOFn4/OvZEHuIkYlvaL4kuRXg21PkC\n1eEX5XgBtl+eQCzC/f5Ty9IpxGUowi1hly1HdoTNjtjZ2RmrMVwq7G4jYUvdcyVy9kbutqqJwciO\nKNUoVvEgk5MT4WhiTe0uvyISGBdgO1EaTfhRgOtBEW6RXPU0a0moL2zLXZa6cFjPrk0iDzo3YVhH\nfOkJXw3fhqq01QnekgDbFDW9zUfBrIBXD4pwi+gH2s5Gd7tdnJyc4Pj4GIeHh+h0OgBQ2fooEuEq\nG0InTYDxn5RV+6Vz9g+27tZeky9sZIn6zvnXFB17Eag6vk7q2EC589F1517bysoKLi4usLKyko2A\n/XHVJKxGuqenp9kqelHWBBmHItwSPqrQWWgV4aOjo1HaWUppTGit9ZAT4SqrQPftH88k+z7KmaT2\nb+l2FeBInNVjtP+Hpa1/vqrtdVAnki99adr9Ol+Gut/v90cCXHeCNRJdL8j6+FaA1WK6uLi4VGGP\nXIYi3CJWhM/OzkYifHx8PJaGpn9AXnyjY9/os2pEolo1/E/N6BjIi7c/1j9Oe5t/jFu3bmU9SHvs\n9+tcg33uJplEgOv8coiEN7c/SXZEVSRsBTmKgC8uLrC6ujomwiQPRbglrB+qdoSNhFWE9WdkNCkX\n7du2QVVZCVaErZCW9vXY7+fO5fZzEVIkwPaxS18o9v/Vi3CdCP463vM6t5VeT06Eq15bnXztkh2R\n84/tNfl/6z1lEkMRbglvR9hI2KYGiciYCJeyIrwIRxGO3y8Jq9+f5bDPHQmL7axREt9osk/P6eNU\nRfJti3AkwDkhrvsF44cX29IXdE5wI1H2/+/6WdW60Nf1f7vIUIRbJOcJ+9VJOqFSEmL7b0o5oF6c\nJxVPm50Rbf256H4+jS76WR35u5HQRsLiLY4oYo8EuUkiAc6dy325+FFlC9lzVV9cUURcFQ1bEfZZ\nE1HfQRJDEW4J+8G1doRaEPaPRz/U0WINv3BDvdOqnFD9A/LCWbUfTQTWydrQa9U/2tXV1fCPMxLk\n6P+t9PNaf37nbJTcuabf76pzk0T5VoSr7J7SL4noeVNKReGNPkv2F50NCGhHVEMRbhFvR2iKmp/o\nsB/saIVcToSjug5+PxLT3H4pOyOaJLTD2wRWIPTYns/te6tFh73d/rsomo8slusW4dxxlVWgv3S8\nCFd9yXjBtfv+uEp4o8+SBhPahVs/BxTgaijCLeF/wukH135o9fbT09NK8fUiXGdYEZ4kmq2KyFdW\nBrUwfPrT2toagHEB7vf7IyGu+nmtYpJLq7KTRfo8VbZKGyKcE+CS1WLFV79sSp69P++fp2ob2RB1\nomD9rDISrg9FuEWsHeH/WPr9/uiDrSuY7LDNNiMRLnWp0H0V4arMi6p6FdE1XVxcXOryoehr9T4u\ngDCKiyb0vMVi06XsAoGcV+3PXffEXLRvRTiaSNXXbUW47shdR+7Yf9GVbC0rwHZSmZ5wPSjCLeEj\nYesBe684EuGc+N26dat2zV8rwlVes0a3ueptOiNuRdiKLDAeAasAegHw9/HiqWKkj29F14qvPlZu\nYtCfaysS9vvR5Glku+h7l4vs/blJyEW7kQjrrzib124/i4yEq6EIt0z0s85HD/bnoRU6HeoZTxsJ\n62NOEwnnInJfdN7XQrbbukKpkXBp5t4OFeGct23PXWcknIuC7eehzqiaI+j3+6MJUEvd1xpNCvro\nOLKGosk/koci3CL+w60J7ufn52MTTVaEV1ZWRiIaCWVdT9iKsApxncm2upGzCnE0fFH6nFBGx96v\nzHnd6g3X9bsnjRYnfZ+jbXQu53dH53NfcPY4yv6w9g8Qi3LJIom+8EpiTMpQhFsiJ8A+Atah4psT\nTCsmVVkR9g+mJHxR9FgSZy/UvgB9br+uSKp9EX2hRKIAIMzkiF7DPIiw/zyUPNmU0qVmrzr0dhVg\njfQjsbUZJdF1R9dVR3h91gXJQxFukegD7iNgOzteJZTqwUX+Xc7Ty/mKuUmsumIZWRX22O5Hopgb\nKsIlmyWyI0o2S1si7MU4Sg8rvY9a/F+3tuayFWDNSokiYJtt4aPlKFtjGjH2r5WMQxFuEf+HF0XA\neluVWNpR9TPW/qFMMsPuhbm0jeyJkn9ZFVlbEa7bmVlFOGed2G2TIqzvqX1v/TnderHN/aLp9/vY\n2tpCr9fD1tbWWDdq++WjGSpVGQpegHVrBTj6koiEmJ7wZFCEW8b+gehP6JxVUWfozLn9g4j2vQiX\nUsNyuajRF4C9LSd4uQg4d3+7ryJcJwNERLKZHv74OkXYvsf+3CRWUq/XG7Wk1/cTeKpZgM3V1tdX\n8oPtbZNEwqUvefv6SMzEIiwi9wP4dwDuA/CFAL45pfQ77j4/CuBfAbgL4A8A/JuU0p9f/XKXB/8h\ntx9+nwXgxbBq+a2PXqKIRkfVctc6zxnt17EYcgJcEm0rwqUOHFaE64ymRLgU9ZZEuCrzo98fdEW2\nETAwEGC1e87Ozka3TRoN22ssecKREDMSnoxpIuFtAB8C8AYAv+VvFJEfAvB9AF4B4JMA/iOAh0Tk\n76eUTqe/1OXDfsD9sUYvOWH0w86A+z+C0ogeq/Q8uZVskXBHXnF0XDdaXV1dzYqwRoQ5ES6l0jUp\nwvb90P3onI6c6JYyXPR9tALc6XRGImwjYUtJkP21lbIjrEVhv+j96yQxE4twSumdAN4JABK/iz8A\n4MdSSv99eJ9XAHgMwDcDeMv0l7p82Ek3+wHOiR1QXTvWPnb0R+5v84/pH6vO80f3KU3u+f0qv9iO\nfr9fS4TPzs4gItnJQL9/XRNzdUQ4J7g5/xUYtyC0a7e1KvQ+0XXpe5a7Xm9HTBIJk2pm6gmLyDMB\n3Avg3XoupbQvIh8A8AJQhMfQD6n+kZSE0G6jc7kUI7uNzuX8wUmeM9rmfORo5KLTKhG2wqv7dmsj\nw6rtvImwF90oDU8jYP3Cst26vVVhv/D1PbKfAe8V+19MkedbN0OClJn1xNy9ABIGka/lseFtxFHn\ng1r62dgWVdc0iZ+sEVyddDYV4Uh0/Tm1I+wChlzO8nVMzNX5dRJleUTn9BeTTQfULt3dbhenp6dj\ndoS+JyrEJQGOrnna9DQKcTXMjlgA5vFDXHVN+oetW2u7eGGOhMn/NFcRTimF/m8UFUeRXS7VamVl\nJfdSZvL/VBUF637ks1q/FZiso4a3ivx1+UlhEbn0/6pDO4Jr7eterzc6tl+C9ouCIlzNrEX4UQAC\n4B6MR8P3APjfM34usgD4iMueUwGwedKRN6n7Kp5RJoT/mW6fw0Zw+hw+yrsOEbavI3cuiib1C0v/\nHzWy9QVzfDaK/dVh/2/9/6u3JuqIr3aB6Xa7IzH2IkwhrsdMRTil9IiIPArgRQD+DwCIyC6A5wP4\n+Vk+F1ksIl9aUTHWPGl7P/9zOKXLizVyP4u9wPvHtfvn5+cNvvrqVXO56Fjx2S8ALnnbfvGJz/X2\n15ObmKsjwlaArQirBWLfD1JmmjzhbQDPxiDiBYBnichzAXwupfSXAF4P4EdE5M8xSFH7MQB/BeC3\nZ3LFZCEoRT42AyT3kzmyJuyy5ap8VR9tW4GPxL1p3z0nvv5cbiLUnxeRUcGeXCRsxdg+V+4LUfet\nt67i6qNgFWJrSehkoP1yZBRczTSR8FcD+D0MJuASgNcOz/8ygO9OKf2UiGwB+AUMFmv8TwD/PDFH\nmCAWZ+9L6taKpE3jy/mmPhqOvFD72Fbcr2Pys44QW6+8at9POPpaGD4KttZDbmJQfxXkIuHIkoh8\n4egLkcRMkyf8XgDFqeSU0msAvGa6SyLLhv/ZW4rEvCjYyTy7+ssKsPd9fSTsr0Uf0z7udYpwad/n\nUuv1+txqzSgpFUKyE6CR2NqJSrufE+GcH2wjZj8xR0+4GmZHkEbxEz/RbVYorfhGKW0+QvYLCUp/\n+F7QbKbGdYtwdAxgZLmsro4XY/crDn2qXcmOyE3I5f7/vB1RioJVhO2/yU2SkhiKMGkMK8AlIbZW\ng94vt3IQiDsSR+fsc1jBj5aDtyHC/pzPUrAZETYnWAXYF3H3BYn8F05kRUS/InwkfHZ2FkbBKsJR\nDQ/aEfWhCJNGyQmwnYSynrAK5SSTdjlv0z62Pl/u8a+LOoKkAmxXUtrVhVaA/WKWOilqeh25BRgl\nP9hnSJyenmZX9HlriMRQhEnj2OguNxk1ydaLbmkbZRjktteNFycb+astoV8e1o7w7aKq8oStJ6zP\nayPhaFVenRQ1zYwoLTChJ1wNRZhcG7k/xNz5uuJY+gOPbmtLdKtQsdXl2dGEnYptZEX4PGHrCeci\n4UiI/Wq5yA+2kXCVL08BLkMRJnNLU3+88yQK/gvBZkHYmhBaF8K2NNra2hrt631KLeet9RAtS9bj\nk5MTnJycjHzf0tJkm7HivXkKcD0owoRcE15w/QKMSHyt8Kro6v7t27exvb09Ot/pdMa8Ypv/rILo\n7QZrNejx0dHRmBDbBRlRbYhc0R5vDZEYijAh10Adv9uK8Pr6+qg0pUa9OlR4d3Z2sL29PRYNqwj7\n9DQAY1FwbhXc6ekpjo+PcXx8fEmEbSRct2oaBbgaijAh10RuklD3/SIMWx9Yo18VXju8LbG+vj7m\nDdu0v0iEfcrZ0dHRmAjb0pi54jy0IqaHIkxIw0R1HyIh9naEdshQO0JF9/bt29jZ2RkJs7cjbKaE\ntSO8H2xFWO2Hk5OTMBL2ldKi7AcK73RQhAm5BnIFi+ywXUb8hJyK7c7ODnZ2dnD79u1R9KvbjY2N\nMU/YFu+pioSt+KonnCtVWSXAFOXJoAgTco3kxLfkCasdYSPh3d3dMZHWff13pUhYF2REInx0dDRm\nR0SlKm3bJJtK50VYoRCXoQgTcg3k7Ae7X7IjokhYBdcPmx1hPWEVzVIkrAIcTczZvnVRwfbclpSh\nCBNyTZRsCK0NYSfmNLqNIuHbt29fWjlnxyTZEVEkHKWo5UpVAvUqxJEYijAhDVOamPNV4nwkrHaD\nTsJtb29jZ2cHu7u7Y6vkov3SxFxVJOxFWKul6aRe1ZJkim99KMKEXAO56NeLsV2WXGVH+OXJfjvN\nxJyKsIpvlR3B9kVXhyJMSMNEvm+0tZNxVdtOp3Op5nLdqmnRsmVbH8JaDz43mM07Zw9FmJCG8VXQ\nfAcM3V9fX7+UbmYL9UQ1IUpF6VV4gfGu075+sK+ilmueytSzZqAIE9IgObvB5gPrvk7C+ZQzXylN\nvd4qAVYi4Y3qCFd1sKb4NgNFmJCGqaoHbLMhIhH2q+CiEpVWjHMTZTnhzQmx79nH8pTNQBEmpGF8\n5oPNfrBDbQgduUjYl6jM1Ue2+bq5mr9efOtEwhTg2UIRJqRhovQzm4JmV775Qjx28YXvIaePHeGF\nMie+pciYRdqvB4owIQ0SFefRFDS75FgFWEfkC/tIuAormLmJuai1Ua5PXLREmVwdijAhDRN5wr5e\ncJUA+4k5m/9rt7n9yBOuI8SMhJuHIkxIw+TsCG9D2LrAOUvCRsJeDHPiG3nBORsiJ8BMUWsOijAh\nDeOL80SdM6JIOBJgK8RVnSx8wfW6Qhz5wj47gsyOamOJEHIlfJ6wn5izkXCd7IhSF2Xgcj3fKDOi\nypLw96EANwcjYUKuSKmBp10VlxNf275IhTialPM1goHLXm8u2rVF2m0H5WiJsi/SU6eXHJkeijAh\nU1DqF2ePI+vBtiyy1dG0aae1JHwU7BdlRBNqNt9Xt3t7ezg4OMDh4WFYuD3qIcdVc9cDRZiQKYmq\noflzNh3NT8T5CNi2tPe1I3yRduByfeDS2N/fx/7+/kiIS408o8m5qJknmQ0UYUKmwLclyu1H6Wh2\n+Fb21orwdoRfKWcjYV8FzZaf7PV6ODg4GImwjYRL9YJzE3RktlCECZmSqIykr+dbWhnn7Yitra2x\n+1kR9hNywFORcK59vR2Hh4ejKFgtCS3cXlUvmClqzUIRJmRCIvvB1ga2+7Y2hBVhmxFhI+Fcv7jI\njrCR8NnZ2Vjretsd4+TkZCS8utU+cjk7IpfSRgGePRRhQqbAl6i0AmxTyXwkXDUxp6LrK6yVPGEb\nCUddk/2+bWsfTcydn5+PZV34fYrwbKEIEzIlvltG1O/Ne8JVE3O+3nBUuCfnCasdcXx8PLIddNjo\n2G69J3x2dhbaD7QjmoMiTMgURP3hvBDb9LRcwR5vR9juG9HIZUdEkfDBwcFoQk6tCrUforxhjYRz\nOcEU3magCBMyBXUE2Bdsz0XCtp19NNEX9ZADnoqE1Y5QcVX74fDwEPv7+9jb27vUuFNH1MhTWyL5\n4kBWiCnIs4MiTMiUqBCXBHiSibmtra2wW0aug4bPE/Z2xMHBAfb29vDkk0+OCa1PYdNjjYRzQkvh\nbQaKMCETEk3KRdFw1D3Dty3y6Wv+eSLULvDt6zUa9i3sDw8Px5Yn6749Z1PTyPVCESZkCnINPKMI\nOFr5FhXiUaKf/9F5G8lGdSC8yOZWwzHroV0owoRMQckP9hkRUYsiuwLO+7y6LY1+vz8W0VYJcp0l\nyaQdKMKETIFfqBF1UvY1gf3Ci6gkZVR+MtpeXFxkxTY3StEwhbg9KMKETEguPa0UCVsh9naE7xcX\nFWL39oEV4TqRcKmTMgW4XSjChExJToAjTziKhK0VUaoRnOuKXCW+9lydzhkU4naYuLOGiNwvIr8j\nIn8tIn0ReZm7/Y3D83a8fXaXTEj7+PS0SIh9JkTOjvBCbCNh2/HCT7pViW/JkuCk3PwwTSS8DeBD\nAN4A4Lcy93kHgFcC0K/33hTPQ8jcEtWNqBMJRw07IwH20bD1c3V4AY7EWPdzXTcoxO0zsQinlN4J\n4J0AILlERqCXUnr8KhdGyLwSVVCr4wn7uhIlS8LbESq8Gs160fX7PgL2gsvylPNDU40+Xygij4nI\nn4rIgyLy+Q09DyGtEKWolaLhOtkRShQJeyGO0tOihRjRggzbzJPF2tuniYm5dwD4TQCPAPgyAD8B\n4O0i8oLEd5osCXUXa0Qdk30UbPOEgXhiznrCOT+4KhLWx/aRLyPhdpm5CKeU3mIOPyoiHwbwCQAv\nBPB7s34+QtqgarGGzxOOVszVnZSzYmrbF/liPKXUtKgYT7Ql109TdsSIlNIjAJ4A8Oymn4uQ68D3\nkyulqOXa1kcRMIBL4mtrQmgRdt8t2VZCs7aDtxkouPNJ43nCIvIMAE8D8Jmmn4uQWZObe875wVHh\nnigzwguxtQSs/6tRr9YKtgV6JhHiSIApxvPBxCIsItsYRLX66XyWiDwXwOeG4wEMPOFHh/f7SQB/\nBuChWVwwIdeBF197XBUJq/DmesX5zsnWilAv2HrAtoGn7R03bSTs90m7TBMJfzUG3m4ajtcOz/8y\ngO8B8FUAXgHgLoBPYyC+/z6ldHblqyXkGrCCG4lxKTMiV8KyVEFNibzgnB1h+8VNEgnr81goyO0y\nTZ7we1H2kv/Z9JdDyPwQiXGduhHWF+50OrUXagC4tDjD2xHaJXkaTxigAM8jrB1BiMGKbe7cpDnC\ndZYsA7jkCefsCC3UroI8aSRM5guKMCEFfARs9316mm1XXyXCVfnBUccM6wVrB+VJI2H7XGQ+oAgT\n4ogi30iAozzhunbEysrKpZ5xQJyi5u0ITVHzXZMnnZwj8wFFmJCAkgCX6kaUUtRKVgRw2Y7wDTxt\natrh4eHYQo0qAab4zi8UYUKGlNLSvBhHFdSiSNhaEVVCXMeOsBNzUdfkSSwJMh9QhAkx1Gk3n4uA\ncxNzfkIul54GICzUo5Gu76JcamNEAV4cKMKEDIna2Eej0+lgc3MTm5ub2NjYwMbGRljAPZeSphGv\niIxazGuBnbr94nzPOF8ZjUV5FgeKMCHAWITrU8/82NjYwNbWFra2tmoJsT52lAmh21u3bmXb2Oeq\nptn6wrk29mT+oQgTMkTF0toLkeWgIqzR8ObmZjES9lkQVoBFZBQVaxv7qtb1pe7JFxcXLNS+YFCE\nCcHlBRgq3z4wAAAalUlEQVR+ks0eb25ujomwRsIaDXsRjogEMmreGdUP1nNWeFmofXGhCBMyxDbu\n9Klmdl9FeGtrKxRgnxXhI9Pc8CKca9qpx7kuzPSEFwuKMCEYX4ThsxxUXNVyiCJhvS2qH5zr6+ab\nbdrmnaUecrZbho1+2cJ+MaEIEzLE2xG2MHun0xlFvN4PLkXCa2trY14tELcvsivkSo077X7UMZkd\nlBcPijAheCoSjuwIjX43NjbGrIgoMyLnCatAKnZlnG6n6RtnJ/n8PgV4MaAIEzLENu20feI0CrYC\nHEXDuewIm4Zm84Q1EtZMh6qOyT5KBi437owGmW8owoQMsbnCPhK2Iry9vV07T1g9YRVcRSNhW7ay\nlBMcRcT6OPYxoy2ZbyjChOCyH2w9YRVZb0Xk/GDf3l7Ftk6NCN81OZcrfH5+3vL/GJkVFGGy1JSq\nodnbbcSrgmuzIDQC1mHT03z7olu3nmo8Y8U2Vw8i6iE3TY1gsphQhMlSExXfiY5txGtF2PvAKsJ2\npVwkwraDss1+sCLs6wH7ThndbpcifAOgCJOlJeoH54feZrMbomjYCrCNgn02hK+QZv1gHwn76HeS\ndkVkeaAIk6XH+r25bV0BVhG2oq1CrJGw7xnn7QhfmtKWp6QI3zwowmSp8fV/c8MKcMmSUBH2S5nV\njvCF2u2CDJsLHNUHtiJ8cnJCEb4hUITJ0lLVnt7u5yLhSIC3t7fHcol91wzrCQMYyweOPOFIhBkJ\n3xwowmSpyQmw7YRhRTi3Os4LsRdyu/XZEZN0y9B9ivDNgSJMlppSe3obyXo7IhcJqwh7X9nv+5zg\n3MScj4Qn7Z5MFh+KMFlaIisiasoZFemx0XBkSfh0N5/6Fi1PrmreOWkHZbIcUITJUhMV5okacnoR\n9naEz47wz5GjTp6wtrHXDsrsnnyzoAiTpSXKEfZ2hBdgWyeiZEfULdReWjHnF2mwg/LNhCJMlgYf\nkdp+cb4iml1ssbGxMRbp+qpodhGGnXTzzTr9tt/vj/KCq5p22voQUe84tixaXijCZGGxousF2E7G\nWdvBR7l6vL29jZ2dnbG2RXYRhuYARwsx/LCi6QvzVBXoKXVQZnW05YQiTBYSX4DHbwFcyobwE29R\nTQhbnrLT6WSXI/uoN+rzdnFxMTbBFkXEuQ7K/rEYCS8vFGGycETCG53L9YvzdYFVgHd2dkI7QvN/\n7ZJkAJcE2HbJyHXLqGppb/+9FXO2K1peKMJkIfECHB37fnFRJLyzszMSX1srOGraWbIjfIF2jWht\nJJwTXnvOi2/UvJNCvFxQhMnC4vNyvRiX+sVpBGxF2HvFuRKVShQJW193mmLtuQ7K1v4gywVFmCw0\nOSH2doRPRbNCfPv2bWxvb49lS0R2RFUk7NPQbMW0OlkRVR2UGQkvJxRhsnD4qNefi+pFqBB7Ad7Z\n2RmJsAq1FeyqiTk/Keej4EnT06J0N79PlguKMFlYctGvXSUXTcxFQqwi7FfS2V5xkR0R1Qv2TTv9\nMuSSJVF3EQhZHijCZKHIZUZURcIlAdZI2FdWK5WorLIjqqyIyA/2HZTZPflmQBEmC4sXYtuuKGdH\nlCJhW44yGr5OcDQxV+qeXCdVjdw8KMJkobBZD1VDJ918Glq0LNlGu9bSUGzTTtuq3guulqLULfvG\nkSoowmShsF6vtQui7fb2Nu7cuYPd3d3sQoxJBNhOmEVlKW1VNFuiUjtmUIRJBEWYLBQ29Uwnz3JD\nRVijYb8Ywxbn0dVwkQD3+/3R/q1bt8KylFFVNDvYN47koAiThSG3Cs7n9trKaLdv3x6LhG27+mg1\nnBdi2ydO/eCqVkW+W4aNhLvdLkWYjEERJnOPzwfWymhalMeudPN1gL0nrLf5CmkrKyuXnktRO0Jv\ni/KBrR1hbYioeWev17skwuTmcqv6Lk8hIj8sIg+LyL6IPCYibxOR57j7dETk50XkCRE5EJG3isgX\nzPayyU0hKlHpI2FbB0Ij37t37+Lu3bvY3d0dCbGNhCM7wlsSPjc3lwWhwmr9YBXgw8PDsd5xtCOI\nZyIRBnA/gJ8F8HwALwawBuB3RWTT3Of1AL4RwLcA+HoAXwTgN69+qeSmEUWlOnnmc3/tEuQ7d+7g\nzp07uHv37qWJucgTjibmLNHquCgzIuoZp5GwRsM2ZY0iTIAJ7YiU0kvtsYi8EsDfALgPwPtEZBfA\ndwP4lyml9w7v810A/q+IfE1K6eGZXDVZeiIBjlbC+aLst2/fHg2tB+GbdnpPWAXYTsIBGIuC7bLh\nUqsi7wcfHh6ORctR3ziugrvZXNUTvgsgAfjc8Pi+4WO+W++QUvq4iHwKwAsAUIRJJTkB1m20Cs5H\nwru7u9je3r7UP87u+0hYBdh3sfCFdKLOyaWJuVIXDUbCZGoRlsFfxesBvC+l9LHh6XsBnKaU9t3d\nHxveRsjE5Cbmqjzhra2tsaI80bB1IRQ7CafHtqxkbnmyTVGzkbAvcelbGFGEbzZXiYQfBPCVAL5u\nRtdCSDgRZ7GesM2OsCKsfvDW1tYoi8IPW9hHJ+R8FgRQXbw9WqzhsyOijhu+fRFF+OYylQiLyM8B\neCmA+1NKnzY3PQpgXUR2XTR8z/A2QrJU9Y2LVsv5qmc+b1jv64vx+KI8Ni/YZkBEQ7MfbDaEirDN\ngNAR9Z/zNYLJzWViER4K8DcB+IaU0qfczR8EcA7gRQDeNrz/VwD4EgB/dLVLJctMrkWRP+frQ1iR\njQTXDlvYJ1qU4f1emwFhj20KWq4uhLcbWKSd5JhIhEXkQQAvB/AyAEcics/wpr2UUjeltC8ibwDw\nOhH5WwAHAH4GwB8wM4LUIVq5ZocX30iIvQDnqqFZgc9NutmoVvd9+pkX4SgPOBcBs0wlmTQSfhUG\n2RDvcee/C8CvDPdfDeACwFsBdAC8E8D3Tn+JZNkp1QT2+9ZKKEXA1u/1Apwr1ONF2E642a0tymNF\nWG/3reujlkXslkGUSfOEKxd3pJR6AL5/OAipTVSU3dcJtpFwTpBtJKzCmxNga0dYPzhXHU2zH+pE\nwlG3ZC/GhLB2BJkLSgJsh7UXSjZEtBLOD5954VfE2RrBNuq1hXmsONu6EFEKWtQ7jpEwoQiTuSGy\nI7yIej+4FAWvra2FQh5ZH9HEXLQU2QqwjYSjJcl2RZwXXloRRKEIk9aJsiC8cFpboU5mhIqxtzO8\nx2yJCrZbIdbFF4eHh5ei41J2BJt3khIUYTIXRHaEFWArwjnxjUYp06IqRU09YbsM+eDg4FKOsJ+Y\n856wz4KI9snNhSJM5obIjoisiKpo2C7i0Mf1W38OKGdHqAj70pTRIg2fHRGloeX2yc2DIkzmgjqR\ncCS+JU94dXX10nPkKGVH2EhY7QifvlZVJY2QHBRhcq1EQmiFNVrpZs9tb2+PNezUkpS+QHuuPb23\nAvTYdkius/ULOXKr5CjApAqKMGmU6Ge/P6dCq0Lqt3Z/d3d3rFuGFmmP+sX54jvRognd73a72awH\nbztEwuvT0Si+pC4UYdIYOQ82qgdhS1PaQjz+2DfutC3srQgrNve3NE5OTkZWQ6lTss8Fzi1PphCT\nulCESaPUyU6wkbCtgBaNnZ2dUSRse8b5LhlR7q+tAxxVRtNJt1I0HKWhRUuUKcCkLhRh0gh16kF4\nEVYB9h2T7bF2TvYi7CPhSIR9NTRbIc2LcE6AdXgR97WBKcKkLhRh0iiRAPutj4RVfFVk7dbveztC\nI2Gl1I7I7vvylN6K8JFwqVA7hZhMAkWYNIb1fn0tCLtvRVgjYRVajXx1f2trayw6tnaEtivKRcJ2\nAYavjqZFeaIOyVE07Iu0RwXbCakDRZg0Sin/10bC3g+2Imw7KKvo2qGTdn5izuf+2gUYNt1M84An\nsSOs2FbVCiakBEWYNEbJF/aLMbwdYQV4d3d31EFZ09F8JoXPE9YoXEXSesC2IaetjBZNzEVifHp6\nGnbJ8FtC6kARJo3g09BKRXlKdoQV4Tt37mBjYyPbvLNqYi4qT6liG4lvKRpmUR4yKyjCpDFKnrCv\nB+GzI3z35Dt37uDzPu/zsLGxcamwuy/ubu0Inx3hy1P6CDiXI+xFWIlqQFB8ySRQhEljeNGNRHNl\nZWVsYUYuT1gn4jqdTljaUvd9yyIrxLksicjzLS1Jvri4aPF/lSwbFGHSCL4we1QTQoefYLO1IHwB\n96pWRTb69taAzWgodVT26Wa+5gQhs4QiTBrBCnCuzKQOW4zHTrJZn9cLca5XXBQJ2wk0n9sbCbAO\nZjuQ64AiTBrBR8J28s1Guuvr62MLLmw0bEW41Csu6pxs8elkk0TCPhqmEJNZQxEmjeG7I0dFejQj\nwkfCdvFFFAnn2hVFLYu8L2xrSeQiYQowuS4owqQR7KScFWFfqEfzgq0A5yJhGw3nalHocwPjNYNt\nNOyL+VghztWDYO4vaQqKMGkEK5LWD44qpU3qCdfpGad4Ac7ZEVFpyigSphCTWUMRJo1g/eDIE/YV\n0ybJjqgrwACK2RE66pampACTJqAIk8bITczZSFgFOBLiyI6wS5IjC8Kei+yIqLYwJ+ZIm1CESSPk\nImFfI6IUCecm5uxz+P0oGq6yIzgxR9qEIkwaIUpRs9XSIjvCZ0fkJub885SIsiN8nnBdISakCSjC\nZGoiAdRzVnhVTP3SZFsPOLIiogk5L8IlSoV1oj5zUSNQGwEzGiZNQBEmE1HVtFP37QScTUfz7Yt8\nJOwn5fxkHCHLBkWY1KZuz7hbt25dyoSI0tI0ErZ94mxxdi/ChCwjFGEyEVWtinTfC7DPiLACrNXR\novQ0ijBZdijCpDa5Au1RXQe/PNnaEV6Ic3UjtDYwRZgsMxRhMhF+OXJU03dlZaVWJGz9YF/YJ7dU\nmZBlgyJMahNFwlGRdtszLvKEvR2hPrDNI6YdQW4KFGEyET7/17cW0v2SAHsh1kUZdkLOR8MUYbKs\nUIRJbXJ+sM8J1kUZ3o7Ipaitr69fEnO7ZSRMlhmKMJmIXCRsF2XYlXG5qmm2o3KufVFUupKQZYMi\nTGpTJxK2UXBuYs5Hwz4LIrdPyDJCESYTUScS9pNs0bDivLq6Wlkj2FZGq0NpyXLuMbgkmbQBRZjU\npq4nHBVk99kTVXWB60S+UQdkey5XEyIakUj7fUKagCJMJqJOJOwroEUTbbluydFzRd0ydFvaVwHO\nFekpRcks1kOuC4owqc0kkXBOiEtRsH0OL7yREFeNqgppJSHORdeEzBqKMJmIaLFGToBznZKjaNg+\ntn2uiEgwvaDaCFj3q4TYPrbfJ6QpKMKkNrnVcpN6wpEdoY9vn8eeU7xAejG12ygSrmtJ+EiYYkya\non6FbAAi8sMi8rCI7IvIYyLyNhF5jrvPe0Skb8aFiDw428smbVFathxZEWtra2NiHeX+lqyJiEiI\noxZGdQq35zops7syuS4mEmEA9wP4WQDPB/BiAGsAfldENs19EoBfBHAPgHsBfCGAH7z6pZK2KQlw\nFAnb81F2RDQxFwlwJMZ1u2XUEd5cdgQh18FEdkRK6aX2WEReCeBvANwH4H3mpuOU0uNXvjoyd1QJ\ncc6OKFkRJcGt8oVzKWfeD/ZNO+vYERRlch1MGgl77mIQ+X7Onf9OEXlcRD4sIj/uImWyoNQVYG9H\n+Gg4J8RejOtOzNWxInJ+cFWqmn1OQppg6ok5GfyFvB7A+1JKHzM3/RqAvwDwaQBfBeCnADwHwLde\n4TrJnHDVSNinqU2yJFlExroe1xXiSQXYP749JmTWXCU74kEAXwnga+3JlNIvmcOPisijAN4lIs9M\nKT1yhecjLWMFuEqIc8IbpabVJUo/8y3s7fb09HQ0tI29HTmfOEpbI6QpphJhEfk5AC8FcH9K6TMV\nd/8AAAHwbAAUYZLFi160vbi4wNnZGc7Pz3F2dlbcPz4+xt7eHvb397G/v4/Dw0McHR3h5OQE3W4X\nvV5vJNBWvHPibK+FkFkxsQgPBfibAHxDSulTNf7J8zDwjavEmtxgIksgsgrOz8/HItzT01OcnZ2N\nBPXs7Gx0/ujoCAcHB6OhInx8fDwSYSve0QQeJ+hI00wkwsN835cDeBmAIxG5Z3jTXkqpKyLPAvAd\nAN4O4LMAngvgdQDem1L6yOwumywbVSvg9JyKbK/XQ6/XG4mpHXru+PgYh4eHo6ECnIuES9kUFGDS\nFJNGwq/CIKp9jzv/XQB+BcApBvnDPwBgG8BfAvgNAP/pSldJlhob+UYr3+y+Rr3dbhfdbnckqH7b\n7XZxfHw8Et6jo6PRvo2ErV9ct7oaIbNk0jzhYkpbSumvALzwKhdEbia5TAc/bCR8cnIyimxVXO2x\n3fp9a0fkvGAvxoQ0AWtHkLkgl2rm91WENeq1Ua4fNiqOhvWRz8/PsxG496gJmSUUYdI6kfebqwWs\nUbAXYZ140+3h4SFOTk7G/ONo30bCkRdNT5g0DUWYzA0+Go5WvkWRsIrvwcEB9vf3R/snJyej+2sW\nRLSvnnApK4NCTJqCIkzmAh95WiG2ebyRJ3x0dITDw0Ps7++P8oL39vbQ7XbDRRr6OH7hRmQ70Iog\nTUMRJq1TEmC7Kk7FM2dHqPg++eSTePLJJ9Hr9cLly7l0NH9NpWNCZgVFmExErr6CtxJKmQ5eBHUl\nnI12c9GrzQU+OTkZRcFqSejqOI2ES+lu9hwhbUERJrWxk2Z2eXCv1xsrymPv78VVRVSj2O3t7dFK\nODts9GvPHR4eYm9vD3t7ezg4OLi0DFkFPBLb3HJoQtqEIkxqE4nw6enppYI8NgqObIRut4utra3R\npJqKta/hEJ2z1oNmQmj+r51o00i7apKNQkzahiJMahOJsI+A7X00m8FmNFgbYXNzE5ubm5dEuDRO\nTk7GUtGiWhA2Eo7SzuzrIaRtKMKkNpEIqwD7HF+NgLWYTq/XQ6fTwcnJCTqdDjY2NkZbFeFSRwzd\nt8uRc8uQq4rwMNOBzBMUYVIbL7K5CNhaEDadrNPpYH19/dIWQLEjsj32y5XtyEXCFGAyz1CESW28\n0FZZEGtra+h2u6N2R36r+1EWRS6zwq6Y0+jX7lsRzuX+RseEtAVFmNTGim0uAj47O8Pq6ipOT08v\nddrI7QPIpo/5rVocaj1EtYXtMuTcwgv7mghpE4owqY0VXH98fn5+qZ2RbXtUOtbHyk2k2WGtjlxX\nDWtf6GNHW0LmAYowqY0KpN2/uLi41DvOjqi3nL9NHy+KXP2xX8pc6q5clQ1BMSbzAEWY1MZHq75d\nfW5Ere2jNvdVdkFuZV5uS9EliwBFmNSGP+cJmT3FThmEEEKahSJMCCEtQhEmhJAWoQgTQkiLzIMI\nb7R9AYQQ0hCV+jYPIvylbV8AIYQ0xJdW3UHaTjcSkacBeAmATwLotnoxhBAyGzYwEOCHUkqfLd2x\ndREmhJCbzDzYEYQQcmOhCBNCSItQhAkhpEUowoQQ0iJzKcIi8r0i8oiInIjI+0XkH7V9TbNARB4Q\nkb4bH2v7uqZBRO4Xkd8Rkb8evo6XBff5URH5tIgci8j/EJFnt3Gt01D1+kTkjcF7+fa2rrcuIvLD\nIvKwiOyLyGMi8jYReY67T0dEfl5EnhCRAxF5q4h8QVvXPAk1X9973Pt2ISIPtnXNcyfCIvLtAF4L\n4AEAzwPwJwAeEpGnt3phs+MjAO4BcO9wfF27lzM12wA+BOB7AFxKsRGRHwLwfQD+NYCvAXCEwfu4\nfp0XeQWKr2/IOzD+Xr78ei7tStwP4GcBPB/AiwGsAfhdEdk093k9gG8E8C0Avh7AFwH4zWu+zmmp\n8/oSgF/EU+/dFwL4wWu+TnM1hWaIbQwA7wfwn82xAPgrAD/Y9rXN4LU9AOCP276OBl5XH8DL3LlP\nA3i1Od4FcALg29q+3hm9vjcC+K22r20Gr+3pw9f3deZ96gH4F+Y+XzG8z9e0fb1XfX3Dc78H4HVt\nX5uOuYqERWQNwH0A3q3n0uB/7V0AXtDWdc2YLx/+xP2EiPyqiHxx2xc0a0TkmRhEGPZ93AfwASzP\n+wgALxz+5P1TEXlQRD6/7QuagrsYRIafGx7fh0GdcfvefRzAp7CY751/fcp3isjjIvJhEflxFylf\nK/NW1P3pAFYAPObOP4bBt/Gi834ArwTwcQx+Ar0GwO+LyD9IKR21eF2z5l4MPvjR+3jv9V9OI7wD\ng5/ojwD4MgA/AeDtIvKCYeAw98igrcnrAbwvpaRzE/cCOB1+aVoW7r3LvD4A+DUAf4HBr7WvAvBT\nAJ4D4Fuv/SIxfyK81KSUHjKHHxGRhzH4MHwbBj9vyYKQUnqLOfyoiHwYwCcAvBCDn7uLwIMAvhKL\nOy9Rhb6+r7UnU0q/ZA4/KiKPAniXiDwzpfTIdV4gMH8Tc08AuMDAMLfcA+DR67+cZkkp7QH4MwAL\nkzVQk0cx8PJvxPsIAMM/3iewIO+liPwcgJcCeGFK6dPmpkcBrIvIrvsnC/Xeudf3mYq7fwCDz2sr\n791ciXBK6QzABwG8SM8Nf1K8CMAftnVdTSEiOxj8lK36kCwUQ0F6FOPv4y4GM9ZL9z4CgIg8A8DT\nsADv5VCgvgnAP00pfcrd/EEA5xh/774CwJcA+KNru8grUPH6Ip6HgX3Wyns3j3bE6wC8SUQ+COBh\nAK8GsAXgTW1e1CwQkZ8G8N8wsCD+LoD/gMEH/s1tXtc0iMg2BpGDtkt+log8F8DnUkp/iYEX9yMi\n8ucYVMj7MQyyXH67hcudmNLrG44HMPCEHx3e7ycx+FXz0OVHmx+G+bAvB/AyAEcior9W9lJK3ZTS\nvoi8AcDrRORvARwA+BkAf5BSeridq65P1esTkWcB+A4AbwfwWQDPxUBz3ptS+kgb19x6ekYmreR7\nMPjDPcHg2/er276mGb2uN2MgRCcYzDb/OoBntn1dU76Wb8Ag9efCjf9q7vMaDCY/jjEQp2e3fd2z\neH0YlCl8JwYC3AXw/wD8FwB/p+3rrvG6otd0AeAV5j4dDHJtn8BAhH8DwBe0fe2zeH0AngHgPQAe\nH34uP47BpOpOW9fMUpaEENIic+UJE0LITYMiTAghLUIRJoSQFqEIE0JIi1CECSGkRSjChBDSIhRh\nQghpEYowIYS0CEWYEEJahCJMCCEtQhEmhJAWoQgTQkiL/H9Kx/J9nkQFJgAAAABJRU5ErkJggg==\n", 41 | "text/plain": [ 42 | "" 43 | ] 44 | }, 45 | "metadata": {}, 46 | "output_type": "display_data" 47 | } 48 | ], 49 | "source": [ 50 | "from PIL import Image\n", 51 | "import matplotlib.pyplot as plt\n", 52 | "import numpy as np\n", 53 | "\n", 54 | "# load image using PIL\n", 55 | "img = Image.open(\"7.png\")\n", 56 | "# convert to black and white\n", 57 | "img = img.convert(\"L\")\n", 58 | "# convert to numpy array\n", 59 | "img = np.asarray(img)\n", 60 | "# display\n", 61 | "% matplotlib inline\n", 62 | "plt.imshow(img, cmap='gray')\n", 63 | "\n", 64 | "from QNN.layers import *\n", 65 | "import pickle\n", 66 | "# load the qnn\n", 67 | "qnn = pickle.load(open(\"mnist-w1a1.pickle\", \"rb\"))\n", 68 | "qnn" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "We will also define a little function **showSrc** to peek at the internals of the QNN module as follows:" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": 43, 81 | "metadata": { 82 | "collapsed": true 83 | }, 84 | "outputs": [], 85 | "source": [ 86 | "import inspect\n", 87 | "\n", 88 | "def showSrc(what):\n", 89 | " print(\"\".join(inspect.getsourcelines(what)[0]))" 90 | ] 91 | }, 92 | { 93 | "cell_type": "markdown", 94 | "metadata": {}, 95 | "source": [ 96 | "## What's inside the predict function?\n", 97 | "\n", 98 | "So what does the **predict** function we call look like on the inside? It actually just runs the QNN layer by layer. We pass the image as the input to the first layer, passing the output of each layer as the input of the next one, and return the output from the final layer like this:" 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "execution_count": 44, 104 | "metadata": {}, 105 | "outputs": [ 106 | { 107 | "name": "stdout", 108 | "output_type": "stream", 109 | "text": [ 110 | "def predict(qnn, input_img):\n", 111 | " \"Predict the class of input_img using a quantized neural network.\"\n", 112 | " activations = input_img\n", 113 | " for layer in qnn:\n", 114 | " activations = layer.execute(activations)\n", 115 | " return activations\n", 116 | "\n" 117 | ] 118 | } 119 | ], 120 | "source": [ 121 | "showSrc(predict)" 122 | ] 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": {}, 127 | "source": [ 128 | "## Activation Quantization: Thresholding Layers \n", 129 | "OK, so **predict** just calls the implementation for each layer -- but what's going on inside each layer? We can see that this network starts with a **BipolarThresholdingLayer**. A key component in all neural networks are nonlinearities introduced by [activation functions](https://en.wikipedia.org/wiki/Activation_function). There are many types of activation functions, but in the QNN models we will look at, most activations will be *thresholding layers*. What does thresholding mean? In this context, it means a function that has a number of pre-defined threshold values, and as output it returns how many of the thresholds were \"crossed\" (were smaller than) the input. To understand this a bit better, let's define a simple thresholding function with a single stair step and plot it:" 130 | ] 131 | }, 132 | { 133 | "cell_type": "code", 134 | "execution_count": 45, 135 | "metadata": {}, 136 | "outputs": [ 137 | { 138 | "data": { 139 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgcAAAFdCAYAAACXXM43AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAFMhJREFUeJzt3X+Q7Xdd3/HX+96bEB2SzUwil4laLM0kXCsZ3QXFtigI\nAYEi0zoNrFAEKkordmanEaROB4baARkl+IMUlSJaYEfailNbmUsxguVX0V1JO01K0yYBaUzaBHax\nuWBC8ukf51x6PsvNze495+w5e/bxmDlzs9/z3e/ncz+5u/s83/M9Z6u1FgCA047MegIAwHwRBwBA\nRxwAAB1xAAB0xAEA0BEHAEBHHAAAnWOznsCoqrokyTOT3J7ky7OdDQAcKBck+ZYkJ1tr94xzoLmK\ngwzC4N2zngQAHGAvTPKecQ4wb3Fwe5K8613vyokTJ2Y8lYNjbW0t11133aynceBYt72zZufGuu2d\nNdu7m2++OS960YuS4c/SccxbHHw5SU6cOJHl5eVZz+XAWFpasl7nwLrtnTU7N9Zt76zZWMZ+Wt4F\niQBARxwAAB1xAAB0xMECWF1dnfUUDiTrtnfW7NxYt72zZrNVrbVZz+Grqmo5ycbGxoYLUQBgDzY3\nN7OyspIkK621zXGO5cwBANARBwBARxwAAB1xAAB0xAEA0BEHAEBHHAAAHXEAAHTEAQDQEQcAQEcc\nAAAdcQAAdMQBANARBwBARxwAAB1xAAB0xAEA0BEHAEBn3+Kgqn6qqh6sqjfv15gAwN7tSxxU1ROT\n/GiSG/djPADg3E09DqrqkUneleRHkmxNezwAYDz7cebgrUl+t7V2wz6MBQCM6dg0D15VL0jy7Ume\nMM1xAIDJmVocVNU3JXlLkqe31u7fy+eura1laWmp27a6uprV1dUJzhAADqb19fWsr69327a3tyd2\n/GqtTexg3YGrnpfkt5M8kKSGm48macNtj2g7Bq+q5SQbGxsbWV5ensq8AGARbW5uZmVlJUlWWmub\n4xxrmk8rfDDJ43dse2eSm5O8cWcYAADzYWpx0Fq7N8lNo9uq6t4k97TWbp7WuADAePb7HRKdLQCA\nOTfVVyvs1Fr7vv0cDwDYO79bAQDoiAMAoCMOAICOOAAAOuIAAOiIAwCgIw4AgI44AAA64gAA6IgD\nAKAjDgCAjjgAADriAADoiAMAoCMOAICOOAAAOuIAAOiIAwCgIw4AgI44AAA64gAA6IgDAKAjDgCA\njjgAADriAADoiAMAoCMOAICOOAAAOuIAAOiIAwCgIw4AgI44AAA64gAA6IgDAKAjDgCAjjgAADri\nAADoiAMAoCMOAICOOAAAOuIAAOiIAwCgIw4AgI44AAA64gAA6IgDAKAjDgCAjjgAADriAADoiAMA\noCMOAIDOVOOgql5TVZ+sqi9W1V1V9b6qumKaYwIA45n2mYMnJ/mlJN+V5OlJzkvygar6uimPCwCc\no2PTPHhr7dmjH1fVS5L87yQrST4yzbEBgHOz39ccXJykJfn8Po8LAOzSvsVBVVWStyT5SGvtpv0a\nFwDYm6k+rbDD9Um+Nclff7gd19bWsrS01G1bXV3N6urqlKYGAAfH+vp61tfXu23b29sTO3611iZ2\nsIccpOqXkzw3yZNba589y37LSTY2NjayvLw89XkBwKLY3NzMyspKkqy01jbHOdbUzxwMw+B5Sb73\nbGEAAMyHqcZBVV2fZDXJDyS5t6qOD+/abq19eZpjAwDnZtoXJL4iyUVJPpTkjpHbNVMeFwA4R9N+\nnwNvzwwAB4wf3gBARxwAAB1xAAB0xAEA0BEHAEBHHAAAHXEAAHTEAQDQEQcAQEccAAAdcQAAdMQB\nANARBwBARxwAAB1xAAB0xAEA0BEHAEBHHAAAHXEAAHTEAQDQEQcAQEccAAAdcQAAdMQBANA5NusJ\nAEzaF76Q3HjjrGfBInnKU2Y9g/0lDoCFc+21yTveMetZsEham/UM9pc4ABbOnXcmV1+dXH/9rGcC\nB5M4ABbO1lZyxRXJ5ZfPeiZwMLkgEVg4W1vJxRfPehZwcIkDYOGIAxiPOAAWjjiA8YgDYKHcd19y\n6pQ4gHGIA2ChbG8P/hQHcO7EAbBQtrYGf4oDOHfiAFgo4gDGJw6AhSIOYHziAFgo4gDGJw6AhbK1\nlVQlF14465nAwSUOgIWytZUsLSVHfHeDc+bLB1go3gAJxicOgIUiDmB84gBYKOIAxicOgIUiDmB8\n4gBYKOIAxicOgIUiDmB84gBYKOIAxicOgIUiDmB84gBYGPfdl5w6JQ5gXOIAWBjb24M/xQGMRxwA\nC8MvXYLJEAfAwhAHMBn7EgdV9eNVdVtVfamqPlFVT9yPcYHDRRzAZEw9Dqrq+Ul+Pslrk3xHkhuT\nnKyqS6c9NnC4iAOYjP04c7CW5Fdaa7/ZWvtvSV6R5FSSl+3D2MAhsrWVVCUXXjjrmcDBNtU4qKrz\nkqwk+f3T21prLckHk3z3NMcGDp+trWRpKTniaioYy7S/hC5NcjTJXTu235Xk0VMeGzhkvAESTMax\nWU/gTNbW1rK0tNRtW11dzerq6oxmBBwE4oDDYn19Pevr69227dNv9DEB046Du5M8kOT4ju3Hk9z5\nUJ903XXXZXl5eZrzAhaQOOCwONMD5s3NzaysrEzk+FN9WqG1dn+SjSRPO72tqmr48cemOTZw+IgD\nmIz9uGznzUleXlUvrqrHJXlbkq9P8s59GBs4RMQBTMbUrzlorb13+J4Gr8/g6YRPJXlma+3/THts\n4HARBzAZ+3JBYmvt+iTX78dYwOElDmAyvBoYWBjiACZDHAAL4b77klOnxAFMgjgAFsLpl3iLAxif\nOAAWgl+6BJMjDoCFIA5gcsQBsBDEAUyOOAAWgjiAyREHwELY2kqqkgsvnPVM4OATB8BC2NpKlpaS\nI76rwdh8GQELwRsgweSIA2AhiAOYHHEALARxAJMjDoCFIA5gcsQBsBDEAUyOOAAWgjiAyREHwEIQ\nBzA54gBYCOIAJkccAAfeffclp06JA5gUcQAceNvbgz/FAUyGOAAOPL90CSZLHAAHnjiAyRIHwIEn\nDmCyxAFw4IkDmCxxABx4W1tJVXLhhbOeCSyGY7OewEHQWvLhD896FsBD2dxMlpaSIx7uwESIg114\n4IHkqU+d9SyAs3n842c9A1gc4mAXjh5Nbrll1rMAzub48VnPABaHONiFquTyy2c9CwDYH56hAwA6\n4gAA6IgDAKAjDgCAjjgAADriAADoiAMAoCMOAICOOAAAOuIAAOiIAwCgIw4AgI44AAA64gAA6IgD\nAKAjDgCAjjgAADriAADoiAMAoCMOAICOOAAAOuIAAOiIAwCgM5U4qKrHVNXbq+rWqjpVVbdU1euq\n6rxpjAcATM6xKR33cUkqycuT/M8k35bk7Um+PsmrpjQmADABU4mD1trJJCdHNt1eVT+X5BURBwAw\n1/bzmoOLk3x+H8cDAM7BvsRBVV2e5JVJ3rYf4wEA525PTytU1RuSvPosu7QkJ1pr/33kc74xyfuT\n/FZr7R27GWdtbS1LS0vdttXV1ayuru5lugCwkNbX17O+vt5t297entjxq7W2+52rLklyycPsdmtr\n7SvD/S9L8gdJPtZae+kujr+cZGNjYyPLy8u7nhcAHHabm5tZWVlJkpXW2uY4x9rTmYPW2j1J7tnN\nvsMzBjck+aMkL9v71ACAWZjKqxWGZww+lOS2DF6d8KiqSpK01u6axpgAwGRM630Ork7y2OHtT4fb\nKoNrEo5OaUwAYAKm8mqF1tpvtNaO7rgdaa0JAwCYc363AgDQEQcAQEccAAAdcQAAdMQBANARBwBA\nRxwAAB1xAAB0xAEA0BEHAEBHHAAAHXEAAHTEAQDQEQcAQEccAAAdcQAAdMQBANARBwBARxwAAB1x\nAAB0xAEA0BEHAEBHHAAAHXEAAHTEAQDQEQcAQEccAAAdcQAAdMQBANARBwBARxwAAB1xAAB0xAEA\n0BEHAEBHHAAAHXEAAHTEAQDQEQcAQEccAAAdcQAAdMQBANARBwBARxwAAB1xAAB0xAEA0BEHAEBH\nHAAAHXEAAHTEAQDQEQcAQEccAACdqcdBVZ1fVZ+qqger6qppjwcAjGc/zhy8KcnnkrR9GAsAGNNU\n46CqnpXk6iTXJqlpjgUATMaxaR24qo4n+dUkP5DkS9MaBwCYrGmeOfj1JNe31v5kimMAABO2pzMH\nVfWGJK8+yy4tyYkk35/kkUl+9vSn7mWctbW1LC0tddtWV1ezurq6l8MAwEJaX1/P+vp6t217e3ti\nx6/Wdn+dYFVdkuSSh9nttiTvTfI3d2w/muQrSd7dWnvpQxx/OcnGxsZGlpeXdz0vADjsNjc3s7Ky\nkiQrrbXNcY61pzMHrbV7ktzzcPtV1U8k+emRTZclOZnkmiSf3MuYAMD+msoFia21z41+XFX3ZvDU\nwq2ttTumMSYAMBn7+Q6J3ucAAA6Aqb2UcVRr7TMZXHMAAMw5v1sBAOiIAwCgIw4AgI44AAA64gAA\n6IgDAKAjDgCAjjgAADriAADoiAMAoCMOAICOOAAAOuIAAOiIAwCgIw4AgI44AAA64gAA6IgDAKAj\nDgCAjjhYAOvr67OewoFk3fbOmp0b67Z31my2xMEC8EV0bqzb3lmzc2Pd9s6azZY4AAA64gAA6IgD\nAKBzbNYT2OGCJLn55ptnPY8DZXt7O5ubm7OexoFj3fbOmp0b67Z31mzvRn52XjDusaq1Nu4xJqaq\nfijJu2c9DwA4wF7YWnvPOAeYtzi4JMkzk9ye5MuznQ0AHCgXJPmWJCdba/eMc6C5igMAYPZckAgA\ndMQBANARBwBARxwAAB1xAAB05ioOquo5VfWJqjpVVZ+vqt/ecf83V9W/r6p7q+rOqnpTVc3V32EW\nqur8qvpUVT1YVVftuO+qqvrDqvpSVX2mqn5yVvOcB1X1mKp6e1XdOvx3dktVva6qztuxn3Xboap+\nvKpuG67JJ6rqibOe07yoqtdU1Ser6otVdVdVva+qrtixzyOq6q1VdXdV/XlV/euqetSs5jxvquqn\nht/D3jyyzZqdQVVdVlX/crgup6rqxqpa3rHP66vqjuH9/6GqLt/LGHPzg7WqfjDJbyb5F0ken+Sv\nJXnPyP1HkvxeBu/q+KQkP5zkJUlev99znUNvSvK5JN3rUqvqwiQnk9yWZDnJTyZ5XVX9yL7PcH48\nLkkleXmSb02yluQVSf7Z6R2s29eqqucn+fkkr03yHUluTHKyqi6d6cTmx5OT/FKS70ry9CTnJflA\nVX3dyD5vSfKcJD+Y5HuSXJbk3+zzPOfSMDR/NIN/V6Os2Q5VdXGSjyb5iwzeF+hEkn+U5Asj+7w6\nySszWNPvTHJvBl+v5+96oNbazG9Jjib50yQvOcs+z0pyf5JLR7b92HBBjs367zDDtXtWkv+awQ+9\nB5NcNXLf309y9+j6JHlDkptmPe95uiW5Nsn/sG5nXaNPJPmFkY8rgyB91aznNo+3JJcOvx7/xvDj\ni4bfzP/WyD5XDvf5zlnPd8Zr9cgkn07yfUn+IMmbrdlZ1+uNST78MPvckWRt5OOLknwpyTW7HWde\nzhwsZ1CEqarN4amQ36uqvzqyz5OS/JfW2t0j204mWUoyut+hUVXHk/xqkhdl8D9+pycl+cPW2ldG\ntp1McmVVLe3DFA+Ki5N8fuRj6zZi+JTLSpLfP72tDb7jfDDJd89qXnPu4gzO5J3+d7WSwVnP0TX8\ndJLPxhq+NcnvttZu2LH9CbFmZ/LcJH9cVe8dPoW1OXpWs6r+cpJHp1+3Lyb5T9nDus1LHDw2g0ci\nr83gaYLnZHBG4EPDUyjJ4C97147Pu2vkvsPo15Nc31r7k4e435o9jOHzcK9M8raRzdatd2kGZ/fO\ntCaHcT3Oqqoqg9PhH2mt3TTc/Ogk9w2/SY861GtYVS9I8u1JXnOGu4/Hmp3JYzM4u/npJM9I8s+T\n/GJV/d3h/Y/OIEzH+nqdahxU1RuGF5g81O2B4UU7p+fxM6213xn+sHtpBn/BvzPNOc6b3a5ZVf3D\nDE7H/ezpT53htGduD//WRj/nG5O8P8lvtdbeMZuZs4Cuz+B6ltVZT2SeVdU3ZRBRL2yt3T/r+Rwg\nR5JstNb+SWvtxtbaryX5tQyunZqYaf/K5p/L4NHt2dya4VMKSb76+yZba/dV1a1J/tJw051Jdl4d\nfXzkvkWxmzW7LclTMzhF9BeDBypf9cdV9e7W2kszWJfjOz53Edcs2f2/tSSDq32T3JDBo7sf27Hf\nYVq33bg7yQM585ocxvV4SFX1y0meneTJrbU7Ru66M8n5VXXRjkfCh3kNV5J8Q5LN+v/fxI4m+Z6q\nemWS70/yCGv2Nf4sIz8rh25O8reH/31nBg8Wj6c/e3A8yUOdZf4aU42DNvitUA/7m6GqaiODC0+u\nTPKx4bbzMvjtUp8Z7vbxJP+4qi4due7gGUm2k9yUBbGHNfuJJD89sumyDJ4XvybJJ4fbPp7kZ6rq\naGvtgeG2ZyT5dGtte3Kznr3drlvy1TMGNyT5oyQvO8Muh2bddqO1dv/wa/RpSf5t8tVT509L8ouz\nnNs8GYbB85J8b2vtszvu3kjylQzW7H3D/a/M4MHPx/dznnPkgxm8Mm3UOzP4QffGJP8rg4vQrVnv\noxn8rBx1ZYY/K1trt1XVnRms239Okqq6KINX0rx116PM+srLkaspr8vgQpOrk1yR5O0ZFNLS8P4j\nGbzM5f1JrsrgJRx3Jfmns577PNySPCZf+2qFizK4avU3MjjN+fwk/zfJ35v1fGe4TpcluSXJB4b/\nffz0zbqddd2uSXIqyYszeGXMr2QQY98w67nNwy2DpxK+kMFLGo+P3C7Ysc9tSZ6SwaPmjyb5j7Oe\n+zzdMvJqBWv2kGv0hAweTL8myV9J8kNJ/jzJC0b2edXw6/O5GQTY7wy/752/63Fm/Rcd+csczeD1\n+n+WZCuDR8EnduzzzUn+3fAb9V0ZPN9+ZNZzn4fbMA4eGI2D4fZvS/Lh4Tf2zya5dtZznfE6/fBw\nnUZvDyZ5wLo97Nr9gyS3Z/DKmI8necKs5zQvt9P/hs5we/HIPo/I4L0Q7h5+M/9XSR4167nP0y2D\nM3qjcWDNzrxOz87grMCpDF7K/rIz7PO6DB7knBr+PL18L2PU8CAAAEnm56WMAMCcEAcAQEccAAAd\ncQAAdMQBANARBwBARxwAAB1xAAB0xAEA0BEHAEBHHAAAnf8HOwpA82rwEyEAAAAASUVORK5CYII=\n", 140 | "text/plain": [ 141 | "" 142 | ] 143 | }, 144 | "metadata": {}, 145 | "output_type": "display_data" 146 | } 147 | ], 148 | "source": [ 149 | "def simpleThreshold(x):\n", 150 | " if x <= 10:\n", 151 | " return -1\n", 152 | " else:\n", 153 | " return +1\n", 154 | "\n", 155 | "x = range(-50,50)\n", 156 | "y = map(simpleThreshold, x)\n", 157 | "plt.plot(x, y)\n", 158 | "plt.ylim([-5, 5])\n", 159 | "plt.show()" 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "We can see that the returned value is -1 for anything smaller than or equal to 10. When the threshold (10) is crossed, the value changes to +1. For our current BNN, the values returned by the threshold function will be either -1 or +1, which are representable by a single bit. To see this in action, we can run our image through just the first layer:" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 46, 172 | "metadata": { 173 | "scrolled": true 174 | }, 175 | "outputs": [ 176 | { 177 | "data": { 178 | "text/plain": [ 179 | "" 180 | ] 181 | }, 182 | "execution_count": 46, 183 | "metadata": {}, 184 | "output_type": "execute_result" 185 | }, 186 | { 187 | "data": { 188 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWEAAAFfCAYAAACfj30KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XuMa1t9H/Dvb8bvx9jzOOdeWpICIUSJqiB6aVIabkJF\nJBoqkVSJkpJIiERVRfNQhNQSIaFeCm1QiEA0wK0ShUCiPCTyQEkr4KbQQHqTwG1vSsqjIQq9hCRw\nzxmP3/b29mv1D/u3Wd7e9tgz3rNsz/cjLXnb4+PZPj7nO2vWXr+1xBgDIiJy48D1CRAR3WYMYSIi\nhxjCREQOMYSJiBxiCBMROcQQJiJyiCFMROQQQ5iIyCGGMBGRQwnXJyAipwBeBuCLAHpuz4aIaCMy\nAJ4F4DFjzMWyJ8YWwiLyYwD+DYAHAfwZgJ8wxvzPiKe+DMCvxXUeREQO/RCAX1/2hFiGI0TkBwC8\nDcAjAF6ASQg/JiJnEU//YhznQES0Bb542RPiGhN+LYCfN8b8ijHmzwG8BkAXwI9EPJdDEES0ry7N\nt42HsIgkATwE4KP6mJks1fYRAC/a9PcjItplcfSEzwAcArgXevweJuPDREQ0xSlqREQOxRHCFQAj\nAA+EHn8AwNMxfD8iop218RA2xgwAPAngpfqYiMj0/h9v+vsREe2yuOYJvx3A+0TkSQBPYDJbIgfg\nfTF9PyKinRRLCBtj3j+dE/wmTIYhPgXgZcaY8zi+HxHRrhLXG32KyD/AZPiCiGjfPGSM+dNlT+Ds\nCCIihxjCREQOMYSJiBxiCBMROcQQJiJyiCFMROQQQ5iIyCGGMBGRQwxhIiKHGMJERA4xhImIHGII\nExE5xBAmInKIIUxE5BBDmIjIIYYwEZFDDGEiIocYwkREDjGEiYgcYggTETnEECYicoghTETkEEOY\niMghhjARkUMMYSIihxjCREQOMYSJiBxiCBMROcQQJiJyiCFMROQQQ5iIyCGGMBGRQwxhIiKHGMJE\nRA4xhImIHGIIExE5xBAmInKIIUxE5BBDmIjIIYYwEZFDDGEiIocYwkREDjGEiYgcYggTETm08RAW\nkUdEZBxqn9v09yEi2geJmF73MwBeCkCm94cxfR8iop0WVwgPjTHnMb02EdHeiGtM+OtF5G9F5Asi\n8qsi8jUxfR8iop0WRwh/AsCrAbwMwGsAPBvAH4pIPobvRUS00zY+HGGMecy6+xkReQLAXwH4fgDv\n3fT3IyLaZbFPUTPGNAD8BYDnxv29iIh2TewhLCIFAF8H4Ctxfy8iol0TxzzhnxWRbxeRvyci/xjA\nBzCZovYbm/5eRES7Lo4pas8E8OsATgGcA3gcwD8yxlzE8L2IiHZaHBfmXrnp1yQi2ldcO4KIyCGG\nMBGRQwxhIiKHGMJERA4xhImIHIprFTVakYjM3YYfuw2MMSvdEu0bhrAjIoKDgwMcHh7i8PAwOI56\nbN/DeDweYzweYzQaYTQaBcdRjxHtG4awIyKCw8NDJJNJJJNJJBKJhceHh4euTzdWw+EQw+EQg8EA\ng8EgOA4/Nh6P2SOmvcMQdkhDOJ1OI51OI5VKRR4nEvv9MQ0GA/i+D9/30e/3g2Pf93FwMLlsMR6P\nISIMYdo7+/2/e4vZPeF0Oo1sNotsNotMJhMc6/1UKuX6dGPl+z48z4Pneej1evA8D4lEYiaAh0Pu\nkEX7iSHsiIgEww4awvl8Hvl8HrlcLjjO5/NIp9OuTzdWnueh0+mg0+mg2+1GBvBtGBun24kh7EhU\nTzifz6NYLM61TCbj+nRj1el0kMlkkE6nkUwm5wK43+8HjxHtG4awIxrCOvZrh3CpVJppuVzO9enG\nqt1uLw3gXq/HnjDtLYawI3ZPOJPJIJfLoVAo4OjoCOVyGcfHx0HL5/d7e75sNhs5BKEBnEwmGcK0\ntxjCjiwbjiiVSjg+Psbp6SlOT09RLBZdn26s0un0wgDudDozAU20bxjCjlwWwicnJzg7O8OdO3dw\ndHTk+nRjlUwmAUQHsA5TsCdM+4oh7NDBwQESiQRSqdTckITdGy6VSq5PNVYiEhRm+L6PXq+HbreL\nfD4fjBenUikkEglWzW2xqBJzzuu+HEOYnFv0w6jX68H3/aBaDpjMKabtYYyZKzUPl52z5Hw5hjA5\nZ8+ZtkO43+9jMBgE5coHBwfo9/uuT5cso9Forsw86pYl54sxhMm5cE84m82iUChgOBwGPSh9zmAw\ncHy2ZBsMBkGpedQtwJLzyzCEyTldLc4ejrB7TxrAyWSS5ctbpt/vz5Sb6/HBwQFEhCXnK2AIk3M6\nHKEhrEMQwHwvmf+ht4teRO10OkilUsFMFmAyXqyzXTizZTGGMDlnB206nZ4bgrDHinmBZ7t4nodW\nqxXMXtEA1otx/X6f0wsvwRAm58Jhaz+mPeB8Pg/P84IeMm0H7QFrANtDEIPBYGZogqIxhMk5ex0N\nYD6A+/1+0BjC26XVas0U09gFN77vzwxPUDSGMDmnoWsfp1KpYMcNu/EK+3bJZrMzQxB2xaPneax2\nXAFDmJzT4NVbnfAf1Wi76FrX4QDudrssOV8RQ5icOzg44AI9OyqZTEYGcLvdDsaK+dkux78dIiKH\nGMJERA4xhImIHGIIExE5xBAmInKIIUxE5BBDmIjIIYYwEZFDDGEiIocYwkREDjGEiYgc4toRDo1G\no2Cbd8/z0Ol00Gq1kMlkZuru932HYREJ1o+4rBljli7wY7dVX1ef59o67023fVr1vXEBne3FEHZE\ntwoPh7AdwLo5oud5rk83VoeHh0gkEpc2/fvQHX7tprv62k23TVqlbQtdDGeVtup724YfMLTY9vzr\nu2UWhbC99J/2jLrdruvTjVUymUQqlVrYtNcHzC6ZaDfd3dduIoJ0Or30dXVB+W1g/5sIv5eopu9j\n0XsEwADeAQxhR6JCWANY/+Po13O5nOOzjVc6nUYmk5lr4b3mjDEzf2+9Xm+m6U6/2g4ODuZeM5vN\nYjQazQTwNi0UH7Uwevh9atP3E36P4U1SabvxE3IkHMLdbjcygPv9frDv2r7SPeRyuRxyuRwGg8FM\nAB8eHiKZTAa/Gdjb5+hvEN1uN9j1V48PDg6Qy+WC19bX1fFi+3W3xXg8nvnBHPW+tNl/Z/l8HsPh\nMAhgHYrhQvjbjyHsSDiENYCjesi6e8G+yuVy8DwPxWIxCEodgtCg1DDRvx97AXG9oNlut4PbdruN\nw8NDFAqFYKgi3ANOJpPBY9vAHu/u9/vBDxh9P/Z763Q6KBQKKBaL6Pf7GA6Hc785pFIphvAOWDuE\nReRhAP8WwEMAngHge4wxvxd6zpsA/EsAZQB/BOBfG2P+8vqnuz/Cv1br1Wv9T2j3hJLJpOOzjZcG\npQaJBrC9A7M+bowJLsTpbxDtdhvNZhPNZhONRiM4Pjw8RK/XmwlgAEEAp9PprQphYLYn3Ov1gh8w\n4ffWarXgeR76/T4Gg8HcEITu0ccQ3n5X6QnnAXwKwHsA/E74iyLyUwB+HMCrAHwRwH8A8JiIfKMx\npn/1U90vdgjbAaz/AdPpNDqdDtLp9N6P63meFwSJ9lQ1gNPp9MzwxHg8nukJe54XhHC9XketVkO9\nXke9Xsfh4eFMz9ruAduvu00hHO4Jd7tdtFotNBqNmffXaDSCH1xRAZzJZGZ6x7S91v7fbYz5MIAP\nA4BETz78SQBvNsb81+lzXgXgHoDvAfD+q5/qfrFDWMc6NYCTyWQQQolEYmuu3sfF9/25oNSr/vZY\nbrgnbA9HaAhXq9WgJRKJyDFg+3W3aQfn8L8D7Qm32200Gg3UajVcXFygWq2iVqvNnHs4gMMBTdtr\no10sEXk2gAcBfFQfM8Y0ReSTAF4EhvAMDRYNYx0X1ls93veJ9v1+f64HrFf+fd+f6SXbPeHwcISG\ncKVSwfn5eTCWHA7gbDaLXq+31T3hcAGPHcKVSgUXFxcLe8C5XC4YJ2YIb79N/577IACDSc/Xdm/6\nNZrS8NX/dEoDd9+D1zYcDoMfOHYPOJ/Pz4wVA5jrCetwRKvVmgnh+/fvI5lMRgawjkFvYwhrTzg8\nHNFsNlGr1VCtVnF+fo7z83MAXy10sQM4aqyYttd+DzbuCDsEtikQbor967cGT7vdDubAptPpYGim\n1+sFY756gUqnb3meF8yE0B9s9lQ2fd1cLjfzuslkcitKw4fDIRqNBtrtdhCk+sNHf0DpD5F+v4+j\noyMUCgXk8/mZvyutuLzqb1F2aXTUsX1fP4tGoxF8FuHPgT8Iltt0CD8NQAA8gNne8AMA/veGvxft\nCZ37a4/zttvtmfDVCsJer4eLiwvUarXI//hR48fhOcXNZnOuMGYbCmJGo1EwA6Lb7aLf72M8HgcB\nnM/ng2sIyWQSp6enODk5QblcRqlUQrFYDH7A2EG8Lv08wuXhUfcbjQYqlcpKnwdF22gIG2OeEpGn\nAbwUwP8BABE5AvCtAN69ye9F+yNq7q8GsPbmdCzY933UarXgP32z2Qx6jlEhbE8DtF/XXlNhNBoh\nm806/luYhJ9eiOt2u8FwwsHBAdLpNPL5PIwxSCQSyGazOD4+xsnJCY6Pj4NecbiXf5WyZXvc3W5R\n5dQ6TLIshLXnTNGuMk84D+C5mPR4AeA5IvJ8AFVjzF8DeAeAN4jIX2IyRe3NAP4GwO9u5Ixp74RL\ndXVutL2Ghl1B2Gg0gnZZTzhc+GKvTmd/fRuqErWn7/t+8Ou8MSboCetxJpNBsVhEqVQKesHaE9ah\nCft9XuU87M/DbjpspE0vGl72edBiV+kJvxDAH2ByAc4AeNv08V8G8CPGmLeKSA7Az2NSrPE/AHwX\n5wjTIuEQXhTA+p+61WrNNB0TDo9BrvO621CVaF+s1Wb3hBOJRFC4Mh6PUSwWZ5rdE75OCEf9UIxq\nnufNfRZa1Wd/HhyOWO4q84Q/jksWgzfGvBHAG692SnTbhHtei4JSizp0brDdVukJ26EUXqNBVx1z\nTUSCpusA6wwI+2signw+H1yY0xZXCGu5dLiFP4dlnwdF4+wIck7HIO0S7qgAbrfbGI1G8DwvaDqj\nYpUx4ajX1YKIbSgNF5FgtkYqlQqOFz2WzWYXNu05bzKEtWTabtojjmocjlgNQ5ic0//0Ok0s6kKd\nzh8ej8fBmKmOm9r3F82OiApgvVCXTqe3oirx4OAA2WwWuVwuuD08PMTh4WEwd9r+up67TkuzjzWw\nNxXCdum0tlqtFkylW/SZ8MLc5RjC5Jz+pw8HpVbP2WXcOoygU9qijjWE7R62vUKZlobbr7sNi58f\nHh6iWCwG5cg6BqxjwrlcDkdHRzg6OkKxWJw5/6jjq05RCw8PRVUlavm0Vh6u8nlQNIYwOaf/6e1q\nuHAJt/YI7YtXGrJ6ax/r60aVhke99jZUKOqCQwCCANZ5wjpFrVQqBdPS7L+XqL8rLX1fV3iRpHAI\na+l0pVJBr9db+jnoLUN4MYYwOadVWKPRaK5se1EZtw436HHULYCgh62r1V32ui7pankawIVCIZgd\nocUapVIJp6enuHPnDoDFf0/27brvcdmYsJZOVyoV3Lt3D/3+ZNLTqp8HzWMI09aw/yNv+nXt221l\nD7VoLxL46m7UOktCL9LFST8Lu3drb6pqF3DQ9bgfCCMiusUYwkREDjGEiYgcYggTETnEECYicogh\nTETkEEOYiMghhjARkUMMYSIihxjCREQOsWyZKGbhRdoX3U8mkzM7ZOgOyrospS7Ks03rXdD1MYSJ\nYqarmunaD4uOU6lUsIPyos07r7pQO20vhjBRjLTHG7U7Rvg4nU4HAVwul4MQzufz196yiLYXQ5go\nZtrT1Z0vtFcbvs1ms3M7KG9q807aXgxhohjZPWEN2vA2Rfat7pqht8VikSG85xjCRDHS3ZLtENad\nkXWoYdGtHmtIM4T3E0OYKGb2cISGsL1XnH0c3jU5PEuCIbx/GMJEMYoajsjn8ygWi8G4b7lcDi7E\nhceNwzsqM4T3D0OYKGY6HJFKpeZC+Pj4GCcnJzg5OUG5XA5mSyxrDOH9whAmipGOCYeHI+wQPj09\nxdnZGY6Pj2fmFIebzitmCO8XhjBRzJYNR5ycnODs7Ax37tzB6enpTAXdskb7gyFMdAXLAtIuR06l\nUigUCkHTWQ+LWlzG43Gwg/JlrdPpoNvtwvM89Ho9+L6Pfr8/swv0tu9cvUsYwkRr0iGGRcMFdstk\nMkEpsl0FZ8/9vYkhBmNMEKJRzf5atVrFxcUFarUaGo0GWq1WEMq+72MwGGA0GsV6vrcJQ5hoTfbc\nX7vZZcjastnsTClyuAoumUzeyIyH8XiM4XCIwWAw0/r9/txjtVotaBrCnU4nCOHhcMje8AYxhImu\nwC7AiJpSZpci6xS0RT3hmwzhfr8P3/fh+34w1BC+32g00Gg0UK/X0Wg00G63gxDW0GZPeHMYwkRr\nWlaKHD7Wi3BakGGHsD33N+6LbXYI93q9YHjB87yZY8/z0Gw20Ww20Wq1glu7JzwYDNgT3iCGMNGa\nVilFXlSOrMc3PSYcFcKdTifo5drH7XZ75li/rj1lHT+mzWAIE61pUQjbC+/orb32Qy6Xmzl2NRyh\nvd92uz3X4202m+h2u0HvOHxrX5hjT3gzGMJEa1oWwnYpcqlUCkqRFy1f6XI4QkNXx3711p6aZo8b\n93q9YEyYwxGbwxAmugK7Ci6XywWL8mgBhpYjHx0dRS7kbj920z1hDeF2ux2Eb7VaRa1WQ7VaDeYF\nh2dP6DGHIzaLIUy0pst6wsfHxzg7O8Pp6SnK5fKlWxu5GBPW8V7tCVerVVQqFVxcXMD3/YVziFmw\nsXkMYaI1LQvhcrkcrAdx9+5dlMvllTf6jJOGsA4r2GPC2gOuVCq4f/8+BoPBTAVduNqOAbxZDGGi\nK7AX5tEwzmQyQSAXCoXgAl0cNATDJceLHtOLbTorwp4F0W63g/HhZrPJoYYbxhAm2lHj8XhhGbI2\nfY4ONSyqgtOLbXTzGMJEO2o0Gs2VHEeVJmspcrVaRb1eR7PZRLvdRrfbDWY8aCky3TyGMNGOiipF\nXtR0FoROQ4vqCXPurxsMYaIdFZ7xEFWCrM0e8w2XIrMn7BZDmGhHjcdjDAYD+L4Pz/PmyoxXKUfm\n8pTurT05UUQeFpHfE5G/FZGxiLwi9PX3Th+32wc3d8pEBMyXIttzf2u1Gi4uLnB+fo779+/j/Pwc\nFxcXM8MROibM5SndukpPOA/gUwDeA+B3FjznQwBeDUAnP/pX+D5EtETU3N9WqzW3FGWj0UCv15sp\nRbaPuTylW2uHsDHmwwA+DACyeIa5b4w5v86JEdFyOjsiXAXXaDSCRdm1HLnf78+VItu37Am7E9eY\n8EtE5B6AGoD/DuANxphqTN+L6FaKWhlNL8DpcMTFxQUqlUpQbrzslhfm3IgjhD8E4LcBPAXg6wC8\nBcAHReRFhj9miTYmPBxh94Tr9frMmLD2chfd6jHdvI2HsDHm/dbdz4rIpwF8AcBLAPzBpr8f0W1l\njAl6sTpX2F4rWMeH6/W661OlJeJdugmAMeYpABUAz437exHRvLgXB6LriT2EReSZAE4BfCXu70VE\n8zgKuN3WHo4QkTwmvVr98focEXk+gOq0PYLJmPDT0+f9DIC/APDYJk6YiGifXGVM+IWYjO2aaXvb\n9PFfBvCjAL4ZwKsAlAF8GZPw/XfGmMG1z5aIaM9cZZ7wx7F8GOOfXv10iGjTRIRDElss9jFhInKL\nAbzdGMJERA4xhImIHGIIE+05zhPebgxhoj3HMeHtxkXdiULsnqMe248lEgkkEgkcHh4G7eDgYGYL\n+6uww/KyY2MMhsNhsACPbupprwVBu4EhTIRJyGqQaqguus1mszg7O8PJyQnK5TKKxSIKhQKy2Swy\nmQySySQSicSVwlh3R7Zvox4bjUY4Pz8PlqrULYt0eyNdppKL8mw/hjARJiF8eHgYBOiy21wuh5OT\nkyCEj46OUCgUkMvlkMlkkEqlgsBely7IY++cvOhWl6rUxdt1yyLuoLxbGMJEmA3hVCqFdDqNdDod\nHNu3+Xwe5XI5aHYI63Ou0hPW5SR1oXXf9xfe+r6Per0eLN5u94TDWxbRdmMIE2E2hDOZTDC0kM1m\n547z+TyOjo6CViqV5nrCiUTiWj1hXZZSd1EOH3ueh2aziUajsXQHZW5ZtP0YwkSYDeF0Oh2EbT6f\nRy6XmzkuFAqRTUM4mUxeeTjC3kFZF2rX1u1253ZR1nWD9Vh7whwT3h0MYSIsDuFisRi0QqEQ3OZy\nuchm94SvMhxhL9Ju76DcarVmmgauhrPddBt7DkfsBoYwEeZDWHu8xWIRpVJpphUKBWQymciWTqc3\n1hO2943TXZO1NZvNYHhCd0+2Gy/M7Q6GMBEmIayzH+ye8NHREcrlMo6Pj3F8fBxMSUulUkvbdaao\naU9YhyM0hPUiXK1WQ71eD3ZQXtQYwruBIUyEr/aEdRaEPRxRKpVwfHyM09NTnJ6eolgsBgUby9q6\nPWEdjrB7wuEQ1mlp1Wo1KNZY1HROMW03hjAR5mdH6HCE9oRPTk5wdnaGO3fuoFgsBoUd4WYXfWyi\nJ6ybdjabTdTrdVSrVVQqFZyfn89UyOlxVKPtxhCmW2VRMB4cHCCRSCCVSgXT0bQnHA7io6Oj2M5v\n1RCuVCqxnQPdLIYw7a2oUuRF9/P5fFCKXCqVggIMnSOsVXBckYw2jSFMeytcimyXH4fv5/N5nJ6e\nzlx803nBOuPhqhfbiJZhCNPesmc82GXHi0qRNYDtUmR7UR72hCkODGHaWzrsoBfbwuXI9mP5fD4Y\nhtBbuyd8nUV5iJZhCNPeCk8706o2uxTZLkm2K+KilqdkT5jiwBCmvWUPR2iPV9d5CIetliKHA5pj\nwhQ3hjDtraiesL0CmpYh6/jvoiELeziCIUybxhCmvRUeE9aesF2KrLeFQmHuop19fJ31IIiWYQjT\n3gqvB6E94WKxGBRfnJyc4PT0FIVCIXLqWvg+e8K0aQxh2ltR60FoT1jXg9BS5EKhEFmGHC7qYAjT\npjGEaW9FrQcR7gmfnZ3h7t27KBQKrk+XbimGMN0quiV9VIvDZYvr2G0wGKBSqaBaraJer6PZbM7t\nljEcDrmd/Z5hCBPFyN4tI9x052Rtvu+jWq3i4uICtVoNjUZjZvNO7hu3nxjCRDHTldF0t2Rt4fu9\nXm9m0XbtCYe3sWdPeL8whIliFF6o3d6CKLw1UbfbDXZQ1i2MooYjuEbwfmEIE8Usat+48Cadeqyb\neIY39LS3sWcI7xeGMFGMLttBObxtvR3I9lb37AnvL4YwUcyW7Runww/NZhOtVisYogjfMoT3F0OY\nKEY6Rc0eE7ZDuF6vBxfiGo3GzAW78C1DeD8xhIlipsMRy3ZQ1rnBOm3Nnr4WfowhvF8YwkQxCveE\nF4Xw+fk5arUaxuNxsFX9smPaHwxhopjpmLB9YU5DWIszzs/PcXFxMfPnwvOBOT94PzGEaafoDsr2\nTsr24jr2/WKxGCxXqWsG53K5YPfkRCJxo0tTGmOCZpcrj0ajoJdLtw9DmHaKLspjLzMZvq+tWCwG\nOyiXSqVgB+VsNhvslsE1gsk1hjDtlIODg5kdlJc17Qlf1hvm8pTkEkOYdoq9RrBuR7SoFQqFYAuj\nZT1hhjC5xBCmnRIOYd2MM6rpRp7aCoUCQ5i2zlqDYSLyehF5QkSaInJPRD4gIs8LPSctIu8WkYqI\ntETkt0Tk7mZPm24rezhCd8vQzTvDC7XfvXt3ZkzYHo5gCNO2WLcn/DCAdwL4X9M/+xYAvy8i32iM\n8abPeQeA7wLwvQCaAN4N4Lenf5boWqJ6wrplUbhpr1ebbvapxwxh2gZrhbAx5uX2fRF5NYD7AB4C\n8LiIHAH4EQD/whjz8elzfhjA/xWRbzHGPLGRs6ZbK9wTtkO4XC7PtHw+H+yWnE6nI48ZwuTadceE\nywAMgOr0/kPT1/yoPsEY83kR+RKAFwFgCNO1hHvCUcMRuoNyLpeb2Sk5mUzOHTOEybUrh7BM/uW+\nA8DjxpjPTR9+EEDfGNMMPf3e9GtE12L3hO3hiPDmnXfu3EE2mw3mAR8eHs4c248xhMml6/SEHwXw\nTQBevKFzIbpUeBv78HCEfWEum80Gf0Zv7WP7lsiVK4WwiLwLwMsBPGyM+bL1pacBpETkKNQbfmD6\nNaI5Go5R5cfhMuVSqTQ328GeeqYX3bRgIw5admyXHy96zPf9YJnKVquFTqcTrBFsb9zJdSFur7VD\neBrA3w3gO4wxXwp9+UkAQwAvBfCB6fO/AcDXAviT650q7SsdGogqQw4fl0qluWln9txffV6cPVzd\nLcPeRXnRse/7l25jz3Ujbre1QlhEHgXwSgCvANARkQemX2oYY3rGmKaIvAfA20WkBqAF4OcA/BFn\nRtAiIoJEIoFUKjVXjhy+r6XIJycnKJfLTqrg7OUpdbF1+9i+73le5A7K4S2L2BO+vdbtCb8Gk9kQ\nHws9/sMAfmV6/FoAIwC/BSAN4MMAfuzqp0j7Ti+26TivXXocvl8sFi8tRdbhi7hE7aAc3klZH+t2\nu8HuyctCmD3h22vdecKXVtgZY3wAPzFtRJfS4Qh7xoPdstnswlJkLUcO94TjFt43TndE1t2T7ab7\nx9k7KIfHhRnCtxfXjiDn7OEIe9qZ3fL5/MxxuN30cETU5p3tdju4tZvummy3brcL3/cxGAx4Ye6W\nYwiTc/ZwxGWlyNozjmquQ7jVaqHZbM407fVGNfaECWAI0xZYFsLhUuRcLjdTehxurkNYx361NZvN\nYMdkDV3f94PGECaGMDm3aDgiXIp8cnKCbDYblByHS5BvqhT5shCu1WqoVquoVqtoNBoYDAYzTXdO\ntu8zhG8vhjA5F3VhLqoK7uzsDNlsdqbkOFyCrC1ul4VwpVJBpVJBo9EI5hTbe8mF73NM+PZiCJNz\n4SlqUSF8584d3L17F+l0eq4MedH9uCzbxr5er8/soFyr1YI/o82+Hz6m24chTFtBy5K1J2vvI6fj\nvTpvOA72Lsj2bdSx53kz087sWRA6VU2b7/uxnC/tD4YwESYhrCXHWnZs39rH3W4XFxcXS0uRWQVH\nq2IIE2FDkZHzAAANcElEQVT2YltUObL9eLfbRbVaRa1WQ6PRQLPZnJn7y/UgaB0MYSLMhnC4DDlc\nmtztdoOV0TSE7Z6wznhgT5hWwRAmwvy0s3AZsl2a3Ol0ZsaE7aIMroxG62IIEyF67m+4DDlcmqwl\nyPZFObsAgz1hWgVDmAiLCzC0x3tZKbL2knU4gj1hWhVDmAhfDWEtK+52uzNzf3UMuF6vo9VqzZQe\na9NxY/aEaR0MYSIsHo5oNptzpcitVmuuFNluXA+C1sEQJsL87IioKjgtRW61WnOlyFGNPWFaBUOY\nCNGzI7QnbIfw/fv30Ww2F5YhhxvRZRjCFAt7HQe76dZDdtOF2nWt4HQ6Hewvl0gkYt+uSGlZslbH\n6dCCjvfaF+CINoUhTLHQ5Sntlc0W3T8+Pp7ZQdneN07DOO7lKYlcYQhTLHRlNHudX3v3ZPvxcrmM\n4+PjyBC+yX3jiFxgCFMsonZQDq+Gpre6ZGV4B2X9+k3soEzkCkOYYqHLUtq7ZehecPYuytlsFsVi\nEUdHRzO3dk9Yhy2I9hFDmGIRtW9ceMdke+fkqMducvNOIlcYwhSLVXZQ1p5vuJds3+r4MYcjaF8x\nhCkWUSGsoWvvnlwqlYIeb9S4MS/M0b5jCFMslvWEdSaE7qCsQWvPmrCPdUyYPWHaRwxhioW9jb0O\nLxQKBZRKJZTLZZyenuLs7Aynp6dIpVJzc4jD84o5HEH7iiFMsbisJ6zb2N+9exfJZHKlCjuifcQQ\nprUsCsvw1vM6w8Ge/WC3YrEYtESC/wzp9uK/flqZPTQQHjoIt9PT06AU+ejoCIVCAblcDplMJhh+\nODg4cP2WiJxjCNPKokqRFzUdctAZEFGlyAxhIoYwrSFcihzVdGqZliBHhbB9IY5jvXTbMYRpZVEX\n27TIItzsYgy7FDm8HgTRbccQppVpCKfT6WDaWfjiW1QZsja7Co7rQRBNMIRpZQcHB0EhhV0FZzft\n9drlx+FSZF2Uh3N/iRjCtIbwcIT2dMNLUZbL5ZnlKsNLV9oLtRPddgxhWplWsS2qgjs5OQlKksOl\nyKlUauaYPWGiCYYwrcweE9aecLFYRKlUCrYo0nJk7enapcd6bD9GdNsxhGll9piw9oR1HFhD+M6d\nO7h7924w/SxceqzH7AUTTTCEaWUanlqwoT1ie8H2o6MjlEolTj8jWhH/pxAROcQQppVw6IAoHgxh\nWokxxvUpEO0lhjARkUMMYSIih9YKYRF5vYg8ISJNEbknIh8QkeeFnvMxERlbbSQij272tOmmcUyY\nKB7r9oQfBvBOAN8K4DsBJAH8vohkrecYAL8A4AEADwJ4BoDXXf9UySWOCRPFY615wsaYl9v3ReTV\nAO4DeAjA49aXusaY82ufHRHRnrvumHAZk55vNfT4D4nIuYh8WkR+OtRTJiKiqStXzMlkkPAdAB43\nxnzO+tKvAfgrAF8G8M0A3grgeQC+7xrnSY5xTJgoHtcpW34UwDcB+Db7QWPML1p3PysiTwP4iIg8\n2xjz1DW+Hzm0LWPCeh6LbsPHqxoMBhgMBhgOhxgOhxiNRhiNRhiPxzDGbM37p/1zpRAWkXcBeDmA\nh40xX7nk6Z8EIACeC4AhTNdijAnC8bLbdYKz2+2iUqmgWq2i0Wig2Wyi0+nA8zz0er0goBnGtGlr\nh/A0gL8bwHcYY760wh95ASbjxpeFNdGlNGQ1FO3b8PE6gel5HqrVKqrVKur1OlqtFtrtNrrdLnzf\nR7/fx2g0YgjTxq0VwtP5vq8E8AoAHRF5YPqlhjGmJyLPAfCDAD4I4ALA8wG8HcDHjTGf2dxp003b\nljFhY0wQtL7vBwEZdTwej1d+3V6vh3q9jlqthnq9HtkT1h420Sat2xN+DSa92o+FHv9hAL8CoI/J\n/OGfBJAH8NcAfhPAf7zWWZJz29ID1J5wv99Hr9dDr9eD53lBs++PRqOVX9f3fTSbTTSbTTQajbme\nMIcjKC7rzhNeOqXNGPM3AF5ynRMiWibcE+52u+h0OkGz7w+Hw5Vft9/vo91uB63Vas30hDkcQXHh\nou60U/TCnPaENXSbzWbQe221Wmi1WhgMBiu/7mAwQLfbDZoGut0TZghTHBjCtFPG4zGGw+FMCLda\nrWAYwW79fn/l1x0Oh8HwRlTr9/sYDoccE6aNYwjTSrbpwpzOjvB9H57nBT1hvbCmzff9lV9XX7Pf\n70c29oQpLgxhWsm2hE9UT7jdbgchXK1WcXFxgYuLC/R6vbVedzQaBcUa4abFG9vy90D7gyFMO8Xu\nCS8K4UqlgvPzc3iet9Zrj8fjmWaMiXyMaJMYwrQ2LeMNh5T2FuPsMdq9YPvCXKvVQqPRmAnidXrC\nRK4whGll4aGATqeDdruNTCaDVCqFRCIRbHUf15b3jUYDFxcXqNVqM/N5Pc+bmcVAtCsYwrSy8Xgc\nXLzyPC+YmZBKpZBMJoPgHY/HsYVwq9UKQljLi3U+L6eS0S5iCNPKoi6K2T1gHZ4YDAaxhXC73Q5m\nQYQX2mEI0y5iCNPKNIR93w+GI+whCDuk45rS1u12g/DVAo1wCPMCGu0ShjCtTKdw6XBEuAds95Lj\nCmHP82ZKi9vtNnvCtNMYwrSycNBG9YC1hxxXCNvrRdglxrwwR7uKIUwrsy/M2QGsj2kAZzKZ2ELY\nnp6mK6bpMXvCtIsYwrQye0w4aggilUohnU4jlUrFdg76/XTdYLu0mCFMu4ghTCvT0LWPfd9HIpFA\nMplEIpEIWlyWlRbrmr8MYdolDGFamQav3h4cHCxscYkqJbYr9lheTLuGIUwr04Ajos2Jr8tCRESX\nYggTETnEECYicoghTETk0DaEcMb1CRARxeTSfNuGEH6W6xMgIorJsy57grieTykipwBeBuCLALgV\nAhHtgwwmAfyYMeZi2ROdhzAR0W22DcMRRES3FkOYiMghhjARkUMMYSIih7YyhEXkx0TkKRHxROQT\nIvIPXZ/TJojIIyIyDrXPuT6vqxCRh0Xk90Tkb6fv4xURz3mTiHxZRLoi8t9E5LkuzvUqLnt/IvLe\niM/yg67Od1Ui8noReUJEmiJyT0Q+ICLPCz0nLSLvFpGKiLRE5LdE5K6rc17Hiu/vY6HPbSQij7o6\n560LYRH5AQBvA/AIgBcA+DMAj4nImdMT25zPAHgAwIPT9mK3p3NleQCfAvCjAOam2IjITwH4cQD/\nCsC3AOhg8jnGt+L7Zi19f1Mfwuxn+cqbObVreRjAOwF8K4DvBJAE8PsikrWe8w4A/wzA9wL4dgB/\nB8Bv3/B5XtUq788A+AV89bN7BoDX3fB5WmdjzFY1AJ8A8J+s+wLgbwC8zvW5beC9PQLgT12fRwzv\nawzgFaHHvgzgtdb9IwAegO93fb4ben/vBfA7rs9tA+/tbPr+Xmx9Tj6Af2495xumz/kW1+d73fc3\nfewPALzd9blp26qesIgkATwE4KP6mJn8rX0EwItcndeGff30V9wviMivisjXuD6hTRORZ2PSw7A/\nxyaAT2J/PkcAeMn0V94/F5FHReTE9QldQRmTnmF1ev8hTNYZtz+7zwP4Enbzswu/P/VDInIuIp8W\nkZ8O9ZRv1LYt6n4G4BDAvdDj9zD5abzrPgHg1QA+j8mvQG8E8Ici8veNMR2H57VpD2LyDz/qc3zw\n5k8nFh/C5Ff0pwB8HYC3APigiLxo2nHYejLZjfUdAB43xui1iQcB9Kc/NG0799kteH8A8GsA/gqT\n39a+GcBbATwPwPfd+Eli+0J4rxljHrPufkZEnsDkH8P3Y/LrLe0IY8z7rbufFZFPA/gCgJdg8uvu\nLngUwDdhd69LXEbf37fZDxpjftG6+1kReRrAR0Tk2caYp27yBIHtuzBXATDCZMDc9gCAp2/+dOJl\njGkA+AsAOzNrYEVPYzKWfys+RwCY/uetYEc+SxF5F4CXA3iJMebL1peeBpASkaPQH9mpzy70/r5y\nydM/icm/Vyef3VaFsDFmAOBJAC/Vx6a/UrwUwB+7Oq+4iEgBk19lL/tHslOmgfQ0Zj/HI0yuWO/d\n5wgAIvJMAKfYgc9yGlDfDeCfGGO+FPrykwCGmP3svgHA1wL4kxs7yWu45P1FeQEmw2dOPrttHI54\nO4D3iciTAJ4A8FoAOQDvc3lSmyAiPwvgv2AyBPF3Afx7TP7B/4bL87oKEclj0nOQ6UPPEZHnA6ga\nY/4ak7G4N4jIX2KyQt6bMZnl8rsOTndty97ftD2CyZjw09Pn/Qwmv9U8Nv9q22M6H/aVAF4BoCMi\n+ttKwxjTM8Y0ReQ9AN4uIjUALQA/B+CPjDFPuDnr1V32/kTkOQB+EMAHAVwAeD4mmfNxY8xnXJyz\n8+kZC6aV/Cgm/3E9TH76vtD1OW3off0GJkHkYXK1+dcBPNv1eV3xvXwHJlN/RqH2S9Zz3ojJxY8u\nJuH0XNfnvYn3h8kyhR/GJIB7AP4fgP8M4I7r817hfUW9pxGAV1nPSWMy17aCSQj/JoC7rs99E+8P\nwDMBfAzA+fTf5ecxuahacHXOXMqSiMihrRoTJiK6bRjCREQOMYSJiBxiCBMROcQQJiJyiCFMROQQ\nQ5iIyCGGMBGRQwxhIiKHGMJERA4xhImIHGIIExE59P8B/y6ksT+5Vj0AAAAASUVORK5CYII=\n", 189 | "text/plain": [ 190 | "" 191 | ] 192 | }, 193 | "metadata": {}, 194 | "output_type": "display_data" 195 | } 196 | ], 197 | "source": [ 198 | "binarized_input = qnn[0].execute(img)\n", 199 | "plt.imshow(binarized_input.reshape(28,28), cmap='gray')" 200 | ] 201 | }, 202 | { 203 | "cell_type": "markdown", 204 | "metadata": {}, 205 | "source": [ 206 | "So the quantizatiion essentially took away all the gray tones and turned it into a black-and-white image! Let's have a look at the internals of the QNNBipolarThresholdingLayer to see what's going in there." 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "execution_count": 47, 212 | "metadata": {}, 213 | "outputs": [ 214 | { 215 | "name": "stdout", 216 | "output_type": "stream", 217 | "text": [ 218 | "class QNNBipolarThresholdingLayer(QNNThresholdingLayer):\n", 219 | " \"A 1-level QNNThresholdingLayer that returns -1 and +1 instead of 0 and 1.\"\n", 220 | " def __init__(self, thresholds):\n", 221 | " super(QNNBipolarThresholdingLayer, self).__init__(thresholds)\n", 222 | " if self.thresholds.shape[0] != 1:\n", 223 | " raise Exception(\"BipolarThresholdingLayer can only have one level\")\n", 224 | "\n", 225 | " def execute(self, v):\n", 226 | " # just the base implementation, but scaled by 2x-1 such that the output\n", 227 | " # is -1, +1 instead of 0, 1. this could have been done with a following\n", 228 | " # LinearLayer, but this way we keep the bipolar thresholding as a stand-\n", 229 | " # alone operation.\n", 230 | " ret = super(QNNBipolarThresholdingLayer, self).execute(v)\n", 231 | " return 2*ret - 1\n", 232 | "\n" 233 | ] 234 | } 235 | ], 236 | "source": [ 237 | "showSrc(QNNBipolarThresholdingLayer)" 238 | ] 239 | }, 240 | { 241 | "cell_type": "markdown", 242 | "metadata": {}, 243 | "source": [ 244 | "Not much going on here -- it is derived from a base class QNNThresholdingLayer, and it just calls the execute function of that, multiplies the output by 2 and subtracts 1 to convert 0s and 1s to -1s and +1s. What does the base class implementation do, though?" 245 | ] 246 | }, 247 | { 248 | "cell_type": "code", 249 | "execution_count": 48, 250 | "metadata": {}, 251 | "outputs": [ 252 | { 253 | "name": "stdout", 254 | "output_type": "stream", 255 | "text": [ 256 | "class QNNThresholdingLayer(QNNLayer):\n", 257 | " \"Given a set of thresholds, return the number of thresholds crossed.\"\n", 258 | " def __init__(self, thresholds):\n", 259 | " # we expect the thresholds array in the following format:\n", 260 | " # thresholds = [levels][channels]\n", 261 | " if thresholds.ndim == 1:\n", 262 | " self.thresholds = thresholds.reshape((len(thresholds),-1))\n", 263 | " elif thresholds.ndim == 2:\n", 264 | " self.thresholds = thresholds\n", 265 | " else:\n", 266 | " raise Exception(\"Thresholds array must be 1- or 2-dimensional\")\n", 267 | "\n", 268 | " def execute(self, v):\n", 269 | " # interpret as multi-channel image, where the number of channels is\n", 270 | " # decided as the number of threshold channels\n", 271 | " vr = v.reshape((self.thresholds.shape[1], -1))\n", 272 | " ret = np.zeros(vr.shape, dtype=np.int)\n", 273 | " for t in self.thresholds:\n", 274 | " for c in range(self.thresholds.shape[1]):\n", 275 | " ret[c] += map(lambda x: 1 if x == True else 0, vr[c] >= t[c])\n", 276 | " return ret.flatten()\n", 277 | "\n" 278 | ] 279 | } 280 | ], 281 | "source": [ 282 | "showSrc(QNNThresholdingLayer)" 283 | ] 284 | }, 285 | { 286 | "cell_type": "markdown", 287 | "metadata": {}, 288 | "source": [ 289 | "There's a bit of functional programming in there, but this is essentially a generalized version of the thresholding function we looked at. It allows specifying the thresholds (the x-coordinates of the stair step) as a list, though since our current network uses binarized activations, there will be only a single stair step -- we will look at the more general case in the next section. But also, when we are processing a vector of values instead of just a single value, it allows us to use different thresholds for each element of the vector (the second dimension of the thresholds parameter). \n", 290 | "\n", 291 | "So how do the thresholds look like for some of the QNNBipolarThresholdingLayers we have in the actual network?" 292 | ] 293 | }, 294 | { 295 | "cell_type": "code", 296 | "execution_count": 49, 297 | "metadata": {}, 298 | "outputs": [ 299 | { 300 | "name": "stdout", 301 | "output_type": "stream", 302 | "text": [ 303 | "First thresholding layer, parameter shape: (1, 1)\n", 304 | "First thresholding layer, threshold values: \n", 305 | "[[34]]\n" 306 | ] 307 | } 308 | ], 309 | "source": [ 310 | "print(\"First thresholding layer, parameter shape: \" + str(qnn[0].thresholds.shape))\n", 311 | "print(\"First thresholding layer, threshold values: \\n\" + str(qnn[0].thresholds))" 312 | ] 313 | }, 314 | { 315 | "cell_type": "code", 316 | "execution_count": 50, 317 | "metadata": {}, 318 | "outputs": [ 319 | { 320 | "name": "stdout", 321 | "output_type": "stream", 322 | "text": [ 323 | "Second thresholding layer, parameter shape: (1, 256)\n", 324 | "Second thresholding layer, threshold values: \n", 325 | "[[ 3 7 36 -36 36 -43 73 6 -19 10 -5 -34 8 -31\n", 326 | " -77 22 3 -29 -23 23 12 22 -39 61 23 -37 -53 -1\n", 327 | " 2 17 28 -33 -29 -1 48 -19 -13 10 -11 -63 66 32\n", 328 | " -7 18 -60 -23 -23 -29 -52 -10 -52 0 -2 7 13 53\n", 329 | " -18 -7 -6 99 29 7 -8 97 15 0 62 -23 -25 -33\n", 330 | " -53 -31 -1 -54 0 77 -26 39 35 41 22 -5 21 -37\n", 331 | " 1 10 10 59 3 -119 -12 -39 25 5 80 -41 -47 -10\n", 332 | " -23 -37 43 -32 29 -23 13 7 -41 24 58 29 18 -62\n", 333 | " -9 -23 -48 -16 -10 31 -39 0 -4 -15 -30 46 33 -51\n", 334 | " -41 8 -28 -28 -10 6 28 36 27 -11 13 -85 18 28\n", 335 | " 43 -36 36 -10 -49 20 -70 -22 -12 54 -12 -43 -53 -45\n", 336 | " -46 47 -8 69 -68 25 78 1 -48 19 2 26 -25 27\n", 337 | " -55 13 40 -5 -12 -30 10 -22 -43 6 24 -6 27 73\n", 338 | " 9 -12 -29 106 50 3 -21 32 -16 97 10 -38 5 7\n", 339 | " -37 55 -2 26 -18 -37 -10 15 -7 -24 25 -24 26 -52\n", 340 | " 9 17 -53 46 44 -5 29 61 94 -31 -2 -18 33 49\n", 341 | " 13 -22 -6 -39 -32 55 -48 10 -23 20 1 -73 61 30\n", 342 | " 62 -57 -24 30 45 16 -24 -78 7 64 -9 -23 -11 45\n", 343 | " -25 -15 67 70]]\n" 344 | ] 345 | } 346 | ], 347 | "source": [ 348 | "print(\"Second thresholding layer, parameter shape: \" + str(qnn[2].thresholds.shape))\n", 349 | "print(\"Second thresholding layer, threshold values: \\n\" + str(qnn[2].thresholds))" 350 | ] 351 | }, 352 | { 353 | "cell_type": "markdown", 354 | "metadata": {}, 355 | "source": [ 356 | "Here we can see that the first thresholding layer, which simply quantizes the input image, uses the same threshold value for all the pixels. The second thresholding layer receives a vector of 256 elements, and uses a different threshold value for each of them. Where do these thresholds come from? As you may have guessed, they are a product of the neural network training process, so you don't have to worry about that in this tutorial -- they are just magic numbers :-)\n", 357 | "\n", 358 | "## Fully Connected Layers\n", 359 | "What else is in our network besides thresholding? There are a total of 4 QNNFullyConnectedLayer instances, so let's have a look at that next. Such a Fully Connected (FC) layer uses a different *weight* for each element in its input vector, creating a *weighted sum* based on the input. They actually created multiple weighted sums using different weights, so their operation actually just corresponds to matrix-vector multiplication. We can see this in their implementation:" 360 | ] 361 | }, 362 | { 363 | "cell_type": "code", 364 | "execution_count": 51, 365 | "metadata": {}, 366 | "outputs": [ 367 | { 368 | "name": "stdout", 369 | "output_type": "stream", 370 | "text": [ 371 | "class QNNFullyConnectedLayer(QNNLayer):\n", 372 | " \"\"\"\n", 373 | " Fully-connected network layers as matrix-vector multiplication.\n", 374 | " Note that bias is not implemented, this can be done by adding a LinearLayer\n", 375 | " with A=1 B=bias following the QNNFullyConnectedLayer.\n", 376 | " \"\"\"\n", 377 | " def __init__(self, W):\n", 378 | " self.W = W\n", 379 | "\n", 380 | " def execute(self, v):\n", 381 | " return np.dot(self.W, v)\n", 382 | "\n" 383 | ] 384 | } 385 | ], 386 | "source": [ 387 | "showSrc(QNNFullyConnectedLayer)" 388 | ] 389 | }, 390 | { 391 | "cell_type": "markdown", 392 | "metadata": {}, 393 | "source": [ 394 | "There's actually nothing else in there -- just a call to the numpy dot product function. What about **self.W**, which contains the weight matrix? What does that look like?" 395 | ] 396 | }, 397 | { 398 | "cell_type": "code", 399 | "execution_count": 52, 400 | "metadata": {}, 401 | "outputs": [ 402 | { 403 | "name": "stdout", 404 | "output_type": "stream", 405 | "text": [ 406 | "First FC layer weight matrix shape: (256, 784)\n" 407 | ] 408 | }, 409 | { 410 | "data": { 411 | "text/plain": [ 412 | "array([[-1, -1, -1, ..., 1, 1, 1],\n", 413 | " [-1, 1, 1, ..., 1, 1, 1],\n", 414 | " [ 1, 1, -1, ..., -1, 1, 1],\n", 415 | " ..., \n", 416 | " [ 1, 1, -1, ..., 1, 1, 1],\n", 417 | " [ 1, -1, 1, ..., -1, 1, -1],\n", 418 | " [-1, -1, -1, ..., 1, -1, -1]], dtype=int8)" 419 | ] 420 | }, 421 | "execution_count": 52, 422 | "metadata": {}, 423 | "output_type": "execute_result" 424 | } 425 | ], 426 | "source": [ 427 | "print(\"First FC layer weight matrix shape: \" + str(qnn[1].W.shape))\n", 428 | "qnn[1].W" 429 | ] 430 | }, 431 | { 432 | "cell_type": "markdown", 433 | "metadata": {}, 434 | "source": [ 435 | "It also only contains -1 and +1 values, which come from the neural network training process (so again, *magic*). Since this is a 256x784 matrix, it will take in a 784-dimensional vector (so, a 28x28 image) and emit a 256-dimensional vector (not necessarily recognizable as an image anymore since it contains weighted sums of different pixels). What do the outputs look like? We can check that by running the image through the first two layers of our network:" 436 | ] 437 | }, 438 | { 439 | "cell_type": "code", 440 | "execution_count": 53, 441 | "metadata": {}, 442 | "outputs": [ 443 | { 444 | "name": "stdout", 445 | "output_type": "stream", 446 | "text": [ 447 | "First FC layer output shape: (256,)\n" 448 | ] 449 | }, 450 | { 451 | "data": { 452 | "text/plain": [ 453 | "array([ 6, -12, 90, -84, 16, -62, 128, -52, -58, 8, 32,\n", 454 | " -18, -36, -48, -54, 4, -4, -44, -34, 38, 10, 10,\n", 455 | " 10, 74, -28, -6, -92, 22, -18, 10, 26, 8, 10,\n", 456 | " 12, 38, -26, -26, -8, 14, -62, 94, 0, 50, 38,\n", 457 | " -42, 12, -56, 2, 8, -74, -34, -42, 32, -14, 90,\n", 458 | " 54, 36, -10, -52, 48, -34, 52, -36, 32, 66, 44,\n", 459 | " 116, 50, -18, -6, -12, 14, -62, 0, 20, 78, 18,\n", 460 | " 16, 18, 30, 54, -34, 36, -104, -8, -24, -6, 34,\n", 461 | " -12, -106, 30, -2, -30, 0, 6, -58, -104, -6, 42,\n", 462 | " -84, 46, 24, 0, -32, -48, 60, -24, 28, 52, 56,\n", 463 | " 46, 14, 36, -2, -20, -4, -44, 52, 22, -6, 30,\n", 464 | " -2, -4, 32, 72, -34, -46, 44, 0, -50, -6, 48,\n", 465 | " 42, 18, 46, -52, -28, -90, -6, 0, 68, -80, -24,\n", 466 | " 44, -40, 34, -110, 14, 16, 8, -52, -76, 6, -52,\n", 467 | " -54, 112, 30, 28, -90, 48, 86, 36, -36, 78, -38,\n", 468 | " -28, 20, -4, -42, 32, 14, -40, -24, -14, 20, -16,\n", 469 | " -16, 58, 2, -72, 8, 84, 22, -36, -18, 44, 66,\n", 470 | " 4, 44, 20, -4, 124, 18, -60, -32, 18, -42, 126,\n", 471 | " 38, 46, 6, 10, 0, 60, 6, 22, -8, 14, 18,\n", 472 | " -62, -46, 8, -62, 64, 20, -20, -26, 10, 30, -88,\n", 473 | " 26, -40, -2, 46, 50, -88, 42, -26, -60, 34, -78,\n", 474 | " 52, -52, 64, -80, -100, 108, 8, 8, -30, 48, 26,\n", 475 | " 38, 32, -14, -86, -10, 106, -82, 8, 32, 20, 40,\n", 476 | " -44, 72, 36])" 477 | ] 478 | }, 479 | "execution_count": 53, 480 | "metadata": {}, 481 | "output_type": "execute_result" 482 | } 483 | ], 484 | "source": [ 485 | "fc_0_output = predict(qnn[0:2], img)\n", 486 | "print(\"First FC layer output shape: \" + str(fc_0_output.shape))\n", 487 | "fc_0_output" 488 | ] 489 | }, 490 | { 491 | "cell_type": "markdown", 492 | "metadata": {}, 493 | "source": [ 494 | "As you can see, even just passing through the first FC layer, the output are no longer easy to interpret for a human. This is why deep neural networks are sometimes referred to as [black boxes](http://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning), although I chose to call them \"gray boxes\" in this tutorial. We can see exactly what kind of computation (matrix-vector, threshold...) is going on inside, and which numbers are passing through the layers, but it's not so easy to *make sense of* those numbers.\n", 495 | "\n", 496 | "> **Aside: XNOR-popcount for binarized networks.** Since we know that the input to this fully-connected layer is also going to be a vector of only -1 and +1 values, we can actually make an interesting observation: if we were to represent -1 values with a 0 bit and +1 values with a 1 bit, we could compute the entire matrix-vector product using binary [XNOR](https://en.wikipedia.org/wiki/XNOR_gate) and (popcount)[https://en.wikipedia.org/wiki/Hamming_weight] operations. Since popcount only adds up the resulting +1 values, you'll have to adjust the resulting sum to account for the -1 values as well, but this is still vastly simpler than computing everything in floating point numbers as we are doing here with numpy. This XNOR-popcount technique has been applied on [FPGAs](https://arxiv.org/abs/1612.07119), [CPUs](https://arxiv.org/abs/1705.09864) and [GPUs](https://arxiv.org/pdf/1705.07175.pdf) to deploy BNNs with very high performance.\n", 497 | "\n", 498 | "## A Pattern Starts to Emerge\n", 499 | "Putting together what we've learned so far, we can make out what is going on in most of this network, since it's just thresholding and FC layers one after another:" 500 | ] 501 | }, 502 | { 503 | "cell_type": "code", 504 | "execution_count": 54, 505 | "metadata": {}, 506 | "outputs": [ 507 | { 508 | "data": { 509 | "text/plain": [ 510 | "[,\n", 511 | " ,\n", 512 | " ,\n", 513 | " ,\n", 514 | " ,\n", 515 | " ,\n", 516 | " ,\n", 517 | " ]" 518 | ] 519 | }, 520 | "execution_count": 54, 521 | "metadata": {}, 522 | "output_type": "execute_result" 523 | } 524 | ], 525 | "source": [ 526 | "qnn[0:8]" 527 | ] 528 | }, 529 | { 530 | "cell_type": "markdown", 531 | "metadata": {}, 532 | "source": [ 533 | "\n", 534 | "1. The qnn[0] QNNBipolarThresholdingLayer binarizes our input image\n", 535 | "2. The qnn[1] QNNFullyConnectedLayer creates linear combinations of its input\n", 536 | "3. The qnn[2] QNNBipolarThresholdingLayer binarizes its inputs produced by the FC layer\n", 537 | "4. The qnn[3] QNNFullyConnectedLayer creates linear combinations of its input\n", 538 | "\n", 539 | "...and so on. Most deep neural networks today contain repeating patterns of layers like this. How do we know what kind of patterns and layers to put together? Again, that's a bit of black magic that is outside the scope of this tutorial, but you can read more about that [here](http://cs231n.github.io/convolutional-networks/#layerpat) if you want to know more.\n", 540 | "\n", 541 | "## The Final Pieces: Scaling and Softmax\n", 542 | "We're actually almost done going through our network. The two final layers are QNNScaleShiftLayer and QNNSoftmaxLayer, which are also relatively simple layers that are commonly found at the end of QNNs. Let's start with having a look at what the QNNScaleShiftLayer does:" 543 | ] 544 | }, 545 | { 546 | "cell_type": "code", 547 | "execution_count": 55, 548 | "metadata": {}, 549 | "outputs": [ 550 | { 551 | "name": "stdout", 552 | "output_type": "stream", 553 | "text": [ 554 | "class QNNScaleShiftLayer(QNNLayer):\n", 555 | " \"Using 1D vectors A and B, apply Ax+B to incoming x.\"\n", 556 | " def __init__(self, A, B):\n", 557 | " if A.shape != B.shape:\n", 558 | " raise Exception(\"QNNScaleShiftLayer A and B shapes do not match\")\n", 559 | " if A.ndim != 1:\n", 560 | " raise Exception(\"QNNScaleShiftLayer needs 1D vectors as parameters.\")\n", 561 | " self.A = A\n", 562 | " self.B = B\n", 563 | "\n", 564 | " def execute(self, v):\n", 565 | " # the outermost dimension is the channel dimension\n", 566 | " # reshape as inner dimension to apply transform\n", 567 | " vr = v.reshape((self.A.shape[0], -1)).transpose()\n", 568 | " return (self.A*vr+self.B).transpose().flatten()\n", 569 | "\n" 570 | ] 571 | } 572 | ], 573 | "source": [ 574 | "showSrc(QNNScaleShiftLayer)" 575 | ] 576 | }, 577 | { 578 | "cell_type": "markdown", 579 | "metadata": {}, 580 | "source": [ 581 | "The reshape and transpose in the **execute()** function may look a bit scary, but all this layer is doing is actually multiplying (scaling) its input by a vector **A**, then adding (shifting) this by a vector **B**. **A** and **B** are supposed to be 1-dimensional vectors, as you can see in the initializer. The transpose and reshape operations are done to make sure the scaling is applied along the channels dimension, that is, each element of the incoming input vector is scaled with a different number for a fully-connected layer. This will be a little different further on for convolutional neural networks, but we will come to that later on.\n", 582 | "\n", 583 | "To make things a little more concrete, let's have a look at the **A** and **B** of our QNNScaleShiftLayer:" 584 | ] 585 | }, 586 | { 587 | "cell_type": "code", 588 | "execution_count": 56, 589 | "metadata": {}, 590 | "outputs": [ 591 | { 592 | "name": "stdout", 593 | "output_type": "stream", 594 | "text": [ 595 | "Scale: [ 0.04175314 0.03991559 0.04339832 0.04174238 0.04294065 0.0413337\n", 596 | " 0.04307769 0.04429524 0.04124721 0.04182861]\n", 597 | "Shift: [-0.08773118 -0.06955352 0.04392026 0.07091018 -0.03852779 0.0639951\n", 598 | " -0.07374354 -0.02511922 0.11293755 0.00291254]\n" 599 | ] 600 | } 601 | ], 602 | "source": [ 603 | "print(\"Scale: \" + str(qnn[8].A))\n", 604 | "print(\"Shift: \" + str(qnn[8].B))" 605 | ] 606 | }, 607 | { 608 | "cell_type": "markdown", 609 | "metadata": {}, 610 | "source": [ 611 | "Once again, you can treat these as magic numbers coming from the training process. What's different from all the other magic numbers we have looked at so far is that these are *floating point numbers* as opposed to just -1 and +1. As this layer constitutes a very small part of the overall computation, this isn't a big deal for performance -- the vast majority of operations in our QNN are still quantized operations.\n", 612 | "\n", 613 | "Finally, let's have a look at the softmax layer. It computes something called the [softmax function](https://en.wikipedia.org/wiki/Softmax_function) on the network output, and its source code looks like this:" 614 | ] 615 | }, 616 | { 617 | "cell_type": "code", 618 | "execution_count": 57, 619 | "metadata": {}, 620 | "outputs": [ 621 | { 622 | "name": "stdout", 623 | "output_type": "stream", 624 | "text": [ 625 | "class QNNSoftmaxLayer(QNNLayer):\n", 626 | " \"Compute softmax values for each sets of scores.\"\n", 627 | " def execute(selv, v):\n", 628 | " e_x = np.exp(v - np.max(v))\n", 629 | " return e_x / e_x.sum()\n", 630 | "\n" 631 | ] 632 | } 633 | ], 634 | "source": [ 635 | "showSrc(QNNSoftmaxLayer)" 636 | ] 637 | }, 638 | { 639 | "cell_type": "markdown", 640 | "metadata": {}, 641 | "source": [ 642 | "What this essentially does is to transform the vector of output numbers so that they sum up to 1 while keeping the small numbers small and big numbers big, allowing us to interpret the softmax output as probabilities. In fact, you could take away the softmax layer and take the maximum of the output of the last layer directly, and you would still get a correct classification result by looking at the maximum value:" 643 | ] 644 | }, 645 | { 646 | "cell_type": "code", 647 | "execution_count": 58, 648 | "metadata": {}, 649 | "outputs": [ 650 | { 651 | "name": "stdout", 652 | "output_type": "stream", 653 | "text": [ 654 | "Last layer outupts without softmax: [ 2 -22 -14 -12 -4 -44 -68 196 -94 40]\n", 655 | "Predicted class: 7\n" 656 | ] 657 | } 658 | ], 659 | "source": [ 660 | "ret_no_softmax = predict(qnn[:8], img)\n", 661 | "print(\"Last layer outupts without softmax: \" + str(ret_no_softmax))\n", 662 | "print(\"Predicted class: %d\" % np.argmax(ret_no_softmax))" 663 | ] 664 | }, 665 | { 666 | "cell_type": "markdown", 667 | "metadata": {}, 668 | "source": [ 669 | "As you can see the predicted class is still the same since the index that has the maximum value does not change, but it's no longer easy to reason about how to interpret the *values* of those numbers.\n", 670 | "\n", 671 | "And that's it! We have now looked at what kind of computation takes place in this network every step of the way. It is somewhat fascinating that one can classify images by multiply-adding -1 and +1 values and comparing them against some thresholds, but that's the magic of binarized neural networks for you. In the next section we'll be looking at essentially the same network but using 2-bit activations instead of binarized ones." 672 | ] 673 | }, 674 | { 675 | "cell_type": "code", 676 | "execution_count": null, 677 | "metadata": { 678 | "collapsed": true 679 | }, 680 | "outputs": [], 681 | "source": [] 682 | } 683 | ], 684 | "metadata": { 685 | "kernelspec": { 686 | "display_name": "Python 2", 687 | "language": "python", 688 | "name": "python2" 689 | }, 690 | "language_info": { 691 | "codemirror_mode": { 692 | "name": "ipython", 693 | "version": 2 694 | }, 695 | "file_extension": ".py", 696 | "mimetype": "text/x-python", 697 | "name": "python", 698 | "nbconvert_exporter": "python", 699 | "pygments_lexer": "ipython2", 700 | "version": "2.7.12" 701 | } 702 | }, 703 | "nbformat": 4, 704 | "nbformat_minor": 2 705 | } 706 | -------------------------------------------------------------------------------- /2-fully-connected-w1a2-mnist.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# A Fully-Connected Network with Binarized Weights, 2-bit Activations\n", 8 | "\n", 9 | "This section will be a small increment of what we did in the previous one. We'll take a network that is almost exactly the same as the previous one, with one difference: it will be using 2-bit activations instead of binarized ones.\n", 10 | "\n", 11 | "What is the advantage of using 2-bit activations, you may wonder? Usually, the more bits of precision you use, the more accurate your QNN classification results you will get. For instance, if we apply the previous network on the 1000 first images from the MNIST test set, it will make 44 misclassifications, whereas the one with 2-bit activations will make 33 misclassifications. \n", 12 | "\n", 13 | "We will start by loading the new network and our previous test image with a digit 7 again:" 14 | ] 15 | }, 16 | { 17 | "cell_type": "code", 18 | "execution_count": 19, 19 | "metadata": {}, 20 | "outputs": [ 21 | { 22 | "data": { 23 | "text/plain": [ 24 | "[,\n", 25 | " ,\n", 26 | " ,\n", 27 | " ,\n", 28 | " ,\n", 29 | " ,\n", 30 | " ,\n", 31 | " ,\n", 32 | " ,\n", 33 | " ]" 34 | ] 35 | }, 36 | "execution_count": 19, 37 | "metadata": {}, 38 | "output_type": "execute_result" 39 | }, 40 | { 41 | "data": { 42 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWEAAAFfCAYAAACfj30KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJztnXus9GtV37/r3ZfZt3e/r0I9R4tWEDE1jYQeK6V6lAYS\nWkzQRqNFE4KmaaiXGJJWY2J6qLYaNRDq5TQaKWhUEkSJtgGOhQgWFU6DxXKpGOlBVDjHc8Cz7zP7\nMk//mFlz1qy9nuf3m9nz27+Z2d9P8uR3mXlnfvPO7O+s+T7rWUtSSiCEENIOt9q+AEIIuclQhAkh\npEUowoQQ0iIUYUIIaRGKMCGEtAhFmBBCWoQiTAghLUIRJoSQFqEIE0JIi6y2fQEi8jQALwHwSQDd\ndq+GEEJmwgaALwXwUErps6U7NibCIvK9AP4tgHsB/AmA708p/a/gri8B8GtNXQchhLTIdwL49dId\nGrEjROTbAbwWwAMAnoeBCD8kIk8P7v7JJq6BEELmgE9W3aEpT/jVAH4hpfQrKaU/BfAqAMcAvju4\nLy0IQsiyUqlvMxdhEVkDcB+Ad+u5NCjV9i4AL5j18xFCyCLTRCT8dAArAB5z5x/DwB8mhBAyhClq\nhBDSIk2I8BMALgDc487fA+DRBp6PEEIWlpmLcErpDMAHAbxIz4mIDI//cNbPRwghi0xTecKvA/Am\nEfkggIcxyJbYAvCmhp6PEEIWkkZEOKX0lmFO8I9iYEN8CMBLUkqPN/F8hBCyqEjbjT5F5B9iYF8Q\nQsiycV9K6Y9Ld2B2BCGEtAhFmBBCWoQiTAghLUIRJoSQFqEIE0JIi1CECSGkRSjChBDSIhRhQghp\nEYowIYS0CEWYEEJahCJMCCEtQhEmhJAWoQgTQkiLUIQJIaRFKMKEENIiFGFCCGkRijAhhLQIRZgQ\nQlqEIkwIIS1CESaEkBahCBNCSItQhAkhpEUowoQQ0iIUYUIIaRGKMCGEtAhFmBBCWoQiTAghLUIR\nJoSQFqEIE0JIi1CECSGkRSjChBDSIhRhQghpEYowIYS0CEWYEEJahCJMCCEtQhEmhJAWoQgTQkiL\nUIQJIaRFKMKEENIiFGFCCGkRijAhhLQIRZgQQlqEIkwIIS1CESaEkBaZuQiLyAMi0nfjY7N+HkII\nWQZWG3rcjwB4EQAZHp839DyEELLQNCXC5ymlxxt6bEIIWRqa8oS/XET+WkQ+ISK/KiJf3NDzEELI\nQtOECL8fwCsBvATAqwA8E8Dvi8h2A89FCCELzcztiJTSQ+bwIyLyMIC/APBtAN446+cjhJBFpvEU\ntZTSHoA/A/Dspp+LEEIWjcZFWER2AHwZgM80/VyEELJoNJEn/NMi8vUi8vdE5J8AeBsGKWpvnvVz\nEULIotNEitozAPw6gKcBeBzA+wD845TSZxt4LkIIWWiamJh7+awfkxBClhXWjiCEkBahCBNCSItQ\nhAkhpEUowoQQ0iIUYUIIaZGmqqiRKRGRmZ5TUkpXu7AajzXJc8zyeghZZCjCLSIiI+HUfT9yt926\ndSt7f4sXu6rjEiml2mPaa9D96BwhywhFuEW8oEZbv186p+dLojapyNnb+v0+Ukro9/uV+6XnKW2j\n/aprJGSRoQi3iBfSlZWVS6Kq5+y2at9HpKV9JRI5f+7i4gL9fn+0tfv+XNXz+n0/9MuEQkyWHYpw\nS9go2AtpbqyurlZuSyI8iW0Q3XZxcXFpnJ+fh+dzz1XHytBoOncthCwTFOEW8UK8uroaCqsda2tr\n2eO1tTWsrKyMCdkk/m3V/vn5+aWhQuzHJP6xtTOsWPf7/dH/E4WYLCsU4RaxdoQXWyuwa2trtcb6\n+vpIhL0Q5/brToqllHB+fo6zszOcnZ2N9u05a6f45ygdiwj6/f5oC2B0rP9PFGKyrFCEW8JHwdZy\nyAlsnbG6unopsixtgXoTZgBweno6Ely7r9d+dnY2el2R6Nut2iYqtlaALeovE7KsUIRbJLIjvOjq\nfqfTQafTGduPjldXV0fiFmUu+GOgftbC6enpaKytrY327aSgFVT/fPZ5vQD79DqNkEvpd4QsAxTh\nFrF2hPV/feTb6XSwsbFRud3Y2MDa2tqYAJZGKWMhOtfr9UZDrRIvvnrf6Ln8ObUtLi4uxkQ2pTSK\nlCnAZNmhCLdEZEdYAbYRsArs5uZmcX9zc7MowjaNzIojUC+FTK9rbW0N3W53zAO2vm3dLwHr+/rn\nogCTmwJFuEWiSFhFzoqvCmydsb6+ns3ljY7riK8ed7vd0fVp1O5FWMmJfu56bMqbX3xiByfnrsY0\n/3+5f8P3YjZQhFvEZkbYyFcF1Yqvj3jt1nrCdewIK4JA9Yo13c9ZJ/qF0e12R3ZFToCjaynlHdtj\nTX0jV8P/H/psmEnSCu2/r9qSGIpwS2hk520IFbTNzU1sbW1ha2trzHqww0bLOkm3trZWaQn4CTKg\n3hJitU1s1kan00Gv18Pp6Sk2NjZwenqKXq83kSURiS1FePbUFctJ7axo6ONRiKuhCLeItyG8CG9v\nb2N7e/tSxJvb95FwlI3gsxVyE3LR1ka/er2aruZH1aRcFA3bxR+5c/xjno6SzeT3oxWQuWE/R37f\nPzeJoQi3RJSatr6+fikKViGuSk2zKW0+Jay0BeoX+bHXafOEo1EVgdtzkdjmtvyDno66NoN9P+oM\nP8lqBVi39PLLUIRbRIW4FAnv7OyMRNimrPlFGnpO84SjaDf38xGoXrKcUsLa2tpolVyn0xnt+9Vz\n9o8zF43bY4pw8+Te++i8/0K1qyJ16KRpNMnqn5eUoQi3RBQJW4tBI2ErwtEiDp9TbFfM1RlKHRH2\noljaz9kfkSBH1kMkwvSEp8eLbmQj6DZnMflFOQBGC3N8rrc+J9MMq6EIt0huYk4zItSK2NnZubSE\nOVc/wkbCkd9XioCrjqu8wcgrrBJh//OXItwMdeYH9JxOruqEa6/XG1uYYx9To2Hd+ue0Qsz3LoYi\n3BJ+oYa3I2wkrCJsJ8V8gR9fRQ2ol3pmyf2R6PmqmXJ7vm7UFdkRUXU2ZkdcjToTpXq+1+uh2+2O\nRhQB62MCyC64UZ+YlKEIt0huYs57wrdv3x6rrhaVuLS3AeWOFn4/OvZEHuIkYlvaL4kuRXg21PkC\n1eEX5XgBtl+eQCzC/f5Ty9IpxGUowi1hly1HdoTNjtjZ2RmrMVwq7G4jYUvdcyVy9kbutqqJwciO\nKNUoVvEgk5MT4WhiTe0uvyISGBdgO1EaTfhRgOtBEW6RXPU0a0moL2zLXZa6cFjPrk0iDzo3YVhH\nfOkJXw3fhqq01QnekgDbFDW9zUfBrIBXD4pwi+gH2s5Gd7tdnJyc4Pj4GIeHh+h0OgBQ2fooEuEq\nG0InTYDxn5RV+6Vz9g+27tZeky9sZIn6zvnXFB17Eag6vk7q2EC589F1517bysoKLi4usLKyko2A\n/XHVJKxGuqenp9kqelHWBBmHItwSPqrQWWgV4aOjo1HaWUppTGit9ZAT4SqrQPftH88k+z7KmaT2\nb+l2FeBInNVjtP+Hpa1/vqrtdVAnki99adr9Ol+Gut/v90cCXHeCNRJdL8j6+FaA1WK6uLi4VGGP\nXIYi3CJWhM/OzkYifHx8PJaGpn9AXnyjY9/os2pEolo1/E/N6BjIi7c/1j9Oe5t/jFu3bmU9SHvs\n9+tcg33uJplEgOv8coiEN7c/SXZEVSRsBTmKgC8uLrC6ujomwiQPRbglrB+qdoSNhFWE9WdkNCkX\n7du2QVVZCVaErZCW9vXY7+fO5fZzEVIkwPaxS18o9v/Vi3CdCP463vM6t5VeT06Eq15bnXztkh2R\n84/tNfl/6z1lEkMRbglvR9hI2KYGiciYCJeyIrwIRxGO3y8Jq9+f5bDPHQmL7axREt9osk/P6eNU\nRfJti3AkwDkhrvsF44cX29IXdE5wI1H2/+/6WdW60Nf1f7vIUIRbJOcJ+9VJOqFSEmL7b0o5oF6c\nJxVPm50Rbf256H4+jS76WR35u5HQRsLiLY4oYo8EuUkiAc6dy325+FFlC9lzVV9cUURcFQ1bEfZZ\nE1HfQRJDEW4J+8G1doRaEPaPRz/U0WINv3BDvdOqnFD9A/LCWbUfTQTWydrQa9U/2tXV1fCPMxLk\n6P+t9PNaf37nbJTcuabf76pzk0T5VoSr7J7SL4noeVNKReGNPkv2F50NCGhHVEMRbhFvR2iKmp/o\nsB/saIVcToSjug5+PxLT3H4pOyOaJLTD2wRWIPTYns/te6tFh73d/rsomo8slusW4dxxlVWgv3S8\nCFd9yXjBtfv+uEp4o8+SBhPahVs/BxTgaijCLeF/wukH135o9fbT09NK8fUiXGdYEZ4kmq2KyFdW\nBrUwfPrT2toagHEB7vf7IyGu+nmtYpJLq7KTRfo8VbZKGyKcE+CS1WLFV79sSp69P++fp2ob2RB1\nomD9rDISrg9FuEWsHeH/WPr9/uiDrSuY7LDNNiMRLnWp0H0V4arMi6p6FdE1XVxcXOryoehr9T4u\ngDCKiyb0vMVi06XsAoGcV+3PXffEXLRvRTiaSNXXbUW47shdR+7Yf9GVbC0rwHZSmZ5wPSjCLeEj\nYesBe684EuGc+N26dat2zV8rwlVes0a3ueptOiNuRdiKLDAeAasAegHw9/HiqWKkj29F14qvPlZu\nYtCfaysS9vvR5Glku+h7l4vs/blJyEW7kQjrrzib124/i4yEq6EIt0z0s85HD/bnoRU6HeoZTxsJ\n62NOEwnnInJfdN7XQrbbukKpkXBp5t4OFeGct23PXWcknIuC7eehzqiaI+j3+6MJUEvd1xpNCvro\nOLKGosk/koci3CL+w60J7ufn52MTTVaEV1ZWRiIaCWVdT9iKsApxncm2upGzCnE0fFH6nFBGx96v\nzHnd6g3X9bsnjRYnfZ+jbXQu53dH53NfcPY4yv6w9g8Qi3LJIom+8EpiTMpQhFsiJ8A+Atah4psT\nTCsmVVkR9g+mJHxR9FgSZy/UvgB9br+uSKp9EX2hRKIAIMzkiF7DPIiw/zyUPNmU0qVmrzr0dhVg\njfQjsbUZJdF1R9dVR3h91gXJQxFukegD7iNgOzteJZTqwUX+Xc7Ty/mKuUmsumIZWRX22O5Hopgb\nKsIlmyWyI0o2S1si7MU4Sg8rvY9a/F+3tuayFWDNSokiYJtt4aPlKFtjGjH2r5WMQxFuEf+HF0XA\neluVWNpR9TPW/qFMMsPuhbm0jeyJkn9ZFVlbEa7bmVlFOGed2G2TIqzvqX1v/TnderHN/aLp9/vY\n2tpCr9fD1tbWWDdq++WjGSpVGQpegHVrBTj6koiEmJ7wZFCEW8b+gehP6JxVUWfozLn9g4j2vQiX\nUsNyuajRF4C9LSd4uQg4d3+7ryJcJwNERLKZHv74OkXYvsf+3CRWUq/XG7Wk1/cTeKpZgM3V1tdX\n8oPtbZNEwqUvefv6SMzEIiwi9wP4dwDuA/CFAL45pfQ77j4/CuBfAbgL4A8A/JuU0p9f/XKXB/8h\ntx9+nwXgxbBq+a2PXqKIRkfVctc6zxnt17EYcgJcEm0rwqUOHFaE64ymRLgU9ZZEuCrzo98fdEW2\nETAwEGC1e87Ozka3TRoN22ssecKREDMSnoxpIuFtAB8C8AYAv+VvFJEfAvB9AF4B4JMA/iOAh0Tk\n76eUTqe/1OXDfsD9sUYvOWH0w86A+z+C0ogeq/Q8uZVskXBHXnF0XDdaXV1dzYqwRoQ5ES6l0jUp\nwvb90P3onI6c6JYyXPR9tALc6XRGImwjYUtJkP21lbIjrEVhv+j96yQxE4twSumdAN4JABK/iz8A\n4MdSSv99eJ9XAHgMwDcDeMv0l7p82Ek3+wHOiR1QXTvWPnb0R+5v84/pH6vO80f3KU3u+f0qv9iO\nfr9fS4TPzs4gItnJQL9/XRNzdUQ4J7g5/xUYtyC0a7e1KvQ+0XXpe5a7Xm9HTBIJk2pm6gmLyDMB\n3Avg3XoupbQvIh8A8AJQhMfQD6n+kZSE0G6jc7kUI7uNzuX8wUmeM9rmfORo5KLTKhG2wqv7dmsj\nw6rtvImwF90oDU8jYP3Cst26vVVhv/D1PbKfAe8V+19MkedbN0OClJn1xNy9ABIGka/lseFtxFHn\ng1r62dgWVdc0iZ+sEVyddDYV4Uh0/Tm1I+wChlzO8nVMzNX5dRJleUTn9BeTTQfULt3dbhenp6dj\ndoS+JyrEJQGOrnna9DQKcTXMjlgA5vFDXHVN+oetW2u7eGGOhMn/NFcRTimF/m8UFUeRXS7VamVl\nJfdSZvL/VBUF637ks1q/FZiso4a3ivx1+UlhEbn0/6pDO4Jr7eterzc6tl+C9ouCIlzNrEX4UQAC\n4B6MR8P3APjfM34usgD4iMueUwGwedKRN6n7Kp5RJoT/mW6fw0Zw+hw+yrsOEbavI3cuiib1C0v/\nHzWy9QVzfDaK/dVh/2/9/6u3JuqIr3aB6Xa7IzH2IkwhrsdMRTil9IiIPArgRQD+DwCIyC6A5wP4\n+Vk+F1ksIl9aUTHWPGl7P/9zOKXLizVyP4u9wPvHtfvn5+cNvvrqVXO56Fjx2S8ALnnbfvGJz/X2\n15ObmKsjwlaArQirBWLfD1JmmjzhbQDPxiDiBYBnichzAXwupfSXAF4P4EdE5M8xSFH7MQB/BeC3\nZ3LFZCEoRT42AyT3kzmyJuyy5ap8VR9tW4GPxL1p3z0nvv5cbiLUnxeRUcGeXCRsxdg+V+4LUfet\nt67i6qNgFWJrSehkoP1yZBRczTSR8FcD+D0MJuASgNcOz/8ygO9OKf2UiGwB+AUMFmv8TwD/PDFH\nmCAWZ+9L6taKpE3jy/mmPhqOvFD72Fbcr2Pys44QW6+8at9POPpaGD4KttZDbmJQfxXkIuHIkoh8\n4egLkcRMkyf8XgDFqeSU0msAvGa6SyLLhv/ZW4rEvCjYyTy7+ssKsPd9fSTsr0Uf0z7udYpwad/n\nUuv1+txqzSgpFUKyE6CR2NqJSrufE+GcH2wjZj8xR0+4GmZHkEbxEz/RbVYorfhGKW0+QvYLCUp/\n+F7QbKbGdYtwdAxgZLmsro4XY/crDn2qXcmOyE3I5f7/vB1RioJVhO2/yU2SkhiKMGkMK8AlIbZW\ng94vt3IQiDsSR+fsc1jBj5aDtyHC/pzPUrAZETYnWAXYF3H3BYn8F05kRUS/InwkfHZ2FkbBKsJR\nDQ/aEfWhCJNGyQmwnYSynrAK5SSTdjlv0z62Pl/u8a+LOoKkAmxXUtrVhVaA/WKWOilqeh25BRgl\nP9hnSJyenmZX9HlriMRQhEnj2OguNxk1ydaLbmkbZRjktteNFycb+astoV8e1o7w7aKq8oStJ6zP\nayPhaFVenRQ1zYwoLTChJ1wNRZhcG7k/xNz5uuJY+gOPbmtLdKtQsdXl2dGEnYptZEX4PGHrCeci\n4UiI/Wq5yA+2kXCVL08BLkMRJnNLU3+88yQK/gvBZkHYmhBaF8K2NNra2hrt631KLeet9RAtS9bj\nk5MTnJycjHzf0tJkm7HivXkKcD0owoRcE15w/QKMSHyt8Kro6v7t27exvb09Ot/pdMa8Ypv/rILo\n7QZrNejx0dHRmBDbBRlRbYhc0R5vDZEYijAh10Adv9uK8Pr6+qg0pUa9OlR4d3Z2sL29PRYNqwj7\n9DQAY1FwbhXc6ekpjo+PcXx8fEmEbSRct2oaBbgaijAh10RuklD3/SIMWx9Yo18VXju8LbG+vj7m\nDdu0v0iEfcrZ0dHRmAjb0pi54jy0IqaHIkxIw0R1HyIh9naEdshQO0JF9/bt29jZ2RkJs7cjbKaE\ntSO8H2xFWO2Hk5OTMBL2ldKi7AcK73RQhAm5BnIFi+ywXUb8hJyK7c7ODnZ2dnD79u1R9KvbjY2N\nMU/YFu+pioSt+KonnCtVWSXAFOXJoAgTco3kxLfkCasdYSPh3d3dMZHWff13pUhYF2REInx0dDRm\nR0SlKm3bJJtK50VYoRCXoQgTcg3k7Ae7X7IjokhYBdcPmx1hPWEVzVIkrAIcTczZvnVRwfbclpSh\nCBNyTZRsCK0NYSfmNLqNIuHbt29fWjlnxyTZEVEkHKWo5UpVAvUqxJEYijAhDVOamPNV4nwkrHaD\nTsJtb29jZ2cHu7u7Y6vkov3SxFxVJOxFWKul6aRe1ZJkim99KMKEXAO56NeLsV2WXGVH+OXJfjvN\nxJyKsIpvlR3B9kVXhyJMSMNEvm+0tZNxVdtOp3Op5nLdqmnRsmVbH8JaDz43mM07Zw9FmJCG8VXQ\nfAcM3V9fX7+UbmYL9UQ1IUpF6VV4gfGu075+sK+ilmueytSzZqAIE9IgObvB5gPrvk7C+ZQzXylN\nvd4qAVYi4Y3qCFd1sKb4NgNFmJCGqaoHbLMhIhH2q+CiEpVWjHMTZTnhzQmx79nH8pTNQBEmpGF8\n5oPNfrBDbQgduUjYl6jM1Ue2+bq5mr9efOtEwhTg2UIRJqRhovQzm4JmV775Qjx28YXvIaePHeGF\nMie+pciYRdqvB4owIQ0SFefRFDS75FgFWEfkC/tIuAormLmJuai1Ua5PXLREmVwdijAhDRN5wr5e\ncJUA+4k5m/9rt7n9yBOuI8SMhJuHIkxIw+TsCG9D2LrAOUvCRsJeDHPiG3nBORsiJ8BMUWsOijAh\nDeOL80SdM6JIOBJgK8RVnSx8wfW6Qhz5wj47gsyOamOJEHIlfJ6wn5izkXCd7IhSF2Xgcj3fKDOi\nypLw96EANwcjYUKuSKmBp10VlxNf275IhTialPM1goHLXm8u2rVF2m0H5WiJsi/SU6eXHJkeijAh\nU1DqF2ePI+vBtiyy1dG0aae1JHwU7BdlRBNqNt9Xt3t7ezg4OMDh4WFYuD3qIcdVc9cDRZiQKYmq\noflzNh3NT8T5CNi2tPe1I3yRduByfeDS2N/fx/7+/kiIS408o8m5qJknmQ0UYUKmwLclyu1H6Wh2\n+Fb21orwdoRfKWcjYV8FzZaf7PV6ODg4GImwjYRL9YJzE3RktlCECZmSqIykr+dbWhnn7Yitra2x\n+1kR9hNywFORcK59vR2Hh4ejKFgtCS3cXlUvmClqzUIRJmRCIvvB1ga2+7Y2hBVhmxFhI+Fcv7jI\njrCR8NnZ2Vjretsd4+TkZCS8utU+cjk7IpfSRgGePRRhQqbAl6i0AmxTyXwkXDUxp6LrK6yVPGEb\nCUddk/2+bWsfTcydn5+PZV34fYrwbKEIEzIlvltG1O/Ne8JVE3O+3nBUuCfnCasdcXx8PLIddNjo\n2G69J3x2dhbaD7QjmoMiTMgURP3hvBDb9LRcwR5vR9juG9HIZUdEkfDBwcFoQk6tCrUforxhjYRz\nOcEU3magCBMyBXUE2Bdsz0XCtp19NNEX9ZADnoqE1Y5QcVX74fDwEPv7+9jb27vUuFNH1MhTWyL5\n4kBWiCnIs4MiTMiUqBCXBHiSibmtra2wW0aug4bPE/Z2xMHBAfb29vDkk0+OCa1PYdNjjYRzQkvh\nbQaKMCETEk3KRdFw1D3Dty3y6Wv+eSLULvDt6zUa9i3sDw8Px5Yn6749Z1PTyPVCESZkCnINPKMI\nOFr5FhXiUaKf/9F5G8lGdSC8yOZWwzHroV0owoRMQckP9hkRUYsiuwLO+7y6LY1+vz8W0VYJcp0l\nyaQdKMKETIFfqBF1UvY1gf3Ci6gkZVR+MtpeXFxkxTY3StEwhbg9KMKETEguPa0UCVsh9naE7xcX\nFWL39oEV4TqRcKmTMgW4XSjChExJToAjTziKhK0VUaoRnOuKXCW+9lydzhkU4naYuLOGiNwvIr8j\nIn8tIn0ReZm7/Y3D83a8fXaXTEj7+PS0SIh9JkTOjvBCbCNh2/HCT7pViW/JkuCk3PwwTSS8DeBD\nAN4A4Lcy93kHgFcC0K/33hTPQ8jcEtWNqBMJRw07IwH20bD1c3V4AY7EWPdzXTcoxO0zsQinlN4J\n4J0AILlERqCXUnr8KhdGyLwSVVCr4wn7uhIlS8LbESq8Gs160fX7PgL2gsvylPNDU40+Xygij4nI\nn4rIgyLy+Q09DyGtEKWolaLhOtkRShQJeyGO0tOihRjRggzbzJPF2tuniYm5dwD4TQCPAPgyAD8B\n4O0i8oLEd5osCXUXa0Qdk30UbPOEgXhiznrCOT+4KhLWx/aRLyPhdpm5CKeU3mIOPyoiHwbwCQAv\nBPB7s34+QtqgarGGzxOOVszVnZSzYmrbF/liPKXUtKgYT7Ql109TdsSIlNIjAJ4A8Oymn4uQ68D3\nkyulqOXa1kcRMIBL4mtrQmgRdt8t2VZCs7aDtxkouPNJ43nCIvIMAE8D8Jmmn4uQWZObe875wVHh\nnigzwguxtQSs/6tRr9YKtgV6JhHiSIApxvPBxCIsItsYRLX66XyWiDwXwOeG4wEMPOFHh/f7SQB/\nBuChWVwwIdeBF197XBUJq/DmesX5zsnWilAv2HrAtoGn7R03bSTs90m7TBMJfzUG3m4ajtcOz/8y\ngO8B8FUAXgHgLoBPYyC+/z6ldHblqyXkGrCCG4lxKTMiV8KyVEFNibzgnB1h+8VNEgnr81goyO0y\nTZ7we1H2kv/Z9JdDyPwQiXGduhHWF+50OrUXagC4tDjD2xHaJXkaTxigAM8jrB1BiMGKbe7cpDnC\ndZYsA7jkCefsCC3UroI8aSRM5guKMCEFfARs9316mm1XXyXCVfnBUccM6wVrB+VJI2H7XGQ+oAgT\n4ogi30iAozzhunbEysrKpZ5xQJyi5u0ITVHzXZMnnZwj8wFFmJCAkgCX6kaUUtRKVgRw2Y7wDTxt\natrh4eHYQo0qAab4zi8UYUKGlNLSvBhHFdSiSNhaEVVCXMeOsBNzUdfkSSwJMh9QhAkx1Gk3n4uA\ncxNzfkIul54GICzUo5Gu76JcamNEAV4cKMKEDIna2Eej0+lgc3MTm5ub2NjYwMbGRljAPZeSphGv\niIxazGuBnbr94nzPOF8ZjUV5FgeKMCHAWITrU8/82NjYwNbWFra2tmoJsT52lAmh21u3bmXb2Oeq\nptn6wrk29mT+oQgTMkTF0toLkeWgIqzR8ObmZjES9lkQVoBFZBQVaxv7qtb1pe7JFxcXLNS+YFCE\nCcHlBRgq3z4wAAAalUlEQVR+ks0eb25ujomwRsIaDXsRjogEMmreGdUP1nNWeFmofXGhCBMyxDbu\n9Klmdl9FeGtrKxRgnxXhI9Pc8CKca9qpx7kuzPSEFwuKMCEYX4ThsxxUXNVyiCJhvS2qH5zr6+ab\nbdrmnaUecrZbho1+2cJ+MaEIEzLE2xG2MHun0xlFvN4PLkXCa2trY14tELcvsivkSo077X7UMZkd\nlBcPijAheCoSjuwIjX43NjbGrIgoMyLnCatAKnZlnG6n6RtnJ/n8PgV4MaAIEzLENu20feI0CrYC\nHEXDuewIm4Zm84Q1EtZMh6qOyT5KBi437owGmW8owoQMsbnCPhK2Iry9vV07T1g9YRVcRSNhW7ay\nlBMcRcT6OPYxoy2ZbyjChOCyH2w9YRVZb0Xk/GDf3l7Ftk6NCN81OZcrfH5+3vL/GJkVFGGy1JSq\nodnbbcSrgmuzIDQC1mHT03z7olu3nmo8Y8U2Vw8i6iE3TY1gsphQhMlSExXfiY5txGtF2PvAKsJ2\npVwkwraDss1+sCLs6wH7ThndbpcifAOgCJOlJeoH54feZrMbomjYCrCNgn02hK+QZv1gHwn76HeS\ndkVkeaAIk6XH+r25bV0BVhG2oq1CrJGw7xnn7QhfmtKWp6QI3zwowmSp8fV/c8MKcMmSUBH2S5nV\njvCF2u2CDJsLHNUHtiJ8cnJCEb4hUITJ0lLVnt7u5yLhSIC3t7fHcol91wzrCQMYyweOPOFIhBkJ\n3xwowmSpyQmw7YRhRTi3Os4LsRdyu/XZEZN0y9B9ivDNgSJMlppSe3obyXo7IhcJqwh7X9nv+5zg\n3MScj4Qn7Z5MFh+KMFlaIisiasoZFemx0XBkSfh0N5/6Fi1PrmreOWkHZbIcUITJUhMV5okacnoR\n9naEz47wz5GjTp6wtrHXDsrsnnyzoAiTpSXKEfZ2hBdgWyeiZEfULdReWjHnF2mwg/LNhCJMlgYf\nkdp+cb4iml1ssbGxMRbp+qpodhGGnXTzzTr9tt/vj/KCq5p22voQUe84tixaXijCZGGxousF2E7G\nWdvBR7l6vL29jZ2dnbG2RXYRhuYARwsx/LCi6QvzVBXoKXVQZnW05YQiTBYSX4DHbwFcyobwE29R\nTQhbnrLT6WSXI/uoN+rzdnFxMTbBFkXEuQ7K/rEYCS8vFGGycETCG53L9YvzdYFVgHd2dkI7QvN/\n7ZJkAJcE2HbJyHXLqGppb/+9FXO2K1peKMJkIfECHB37fnFRJLyzszMSX1srOGraWbIjfIF2jWht\nJJwTXnvOi2/UvJNCvFxQhMnC4vNyvRiX+sVpBGxF2HvFuRKVShQJW193mmLtuQ7K1v4gywVFmCw0\nOSH2doRPRbNCfPv2bWxvb49lS0R2RFUk7NPQbMW0OlkRVR2UGQkvJxRhsnD4qNefi+pFqBB7Ad7Z\n2RmJsAq1FeyqiTk/Keej4EnT06J0N79PlguKMFlYctGvXSUXTcxFQqwi7FfS2V5xkR0R1Qv2TTv9\nMuSSJVF3EQhZHijCZKHIZUZURcIlAdZI2FdWK5WorLIjqqyIyA/2HZTZPflmQBEmC4sXYtuuKGdH\nlCJhW44yGr5OcDQxV+qeXCdVjdw8KMJkobBZD1VDJ918Glq0LNlGu9bSUGzTTtuq3guulqLULfvG\nkSoowmShsF6vtQui7fb2Nu7cuYPd3d3sQoxJBNhOmEVlKW1VNFuiUjtmUIRJBEWYLBQ29Uwnz3JD\nRVijYb8Ywxbn0dVwkQD3+/3R/q1bt8KylFFVNDvYN47koAiThSG3Cs7n9trKaLdv3x6LhG27+mg1\nnBdi2ydO/eCqVkW+W4aNhLvdLkWYjEERJnOPzwfWymhalMeudPN1gL0nrLf5CmkrKyuXnktRO0Jv\ni/KBrR1hbYioeWev17skwuTmcqv6Lk8hIj8sIg+LyL6IPCYibxOR57j7dETk50XkCRE5EJG3isgX\nzPayyU0hKlHpI2FbB0Ij37t37+Lu3bvY3d0dCbGNhCM7wlsSPjc3lwWhwmr9YBXgw8PDsd5xtCOI\nZyIRBnA/gJ8F8HwALwawBuB3RWTT3Of1AL4RwLcA+HoAXwTgN69+qeSmEUWlOnnmc3/tEuQ7d+7g\nzp07uHv37qWJucgTjibmLNHquCgzIuoZp5GwRsM2ZY0iTIAJ7YiU0kvtsYi8EsDfALgPwPtEZBfA\ndwP4lyml9w7v810A/q+IfE1K6eGZXDVZeiIBjlbC+aLst2/fHg2tB+GbdnpPWAXYTsIBGIuC7bLh\nUqsi7wcfHh6ORctR3ziugrvZXNUTvgsgAfjc8Pi+4WO+W++QUvq4iHwKwAsAUIRJJTkB1m20Cs5H\nwru7u9je3r7UP87u+0hYBdh3sfCFdKLOyaWJuVIXDUbCZGoRlsFfxesBvC+l9LHh6XsBnKaU9t3d\nHxveRsjE5Cbmqjzhra2tsaI80bB1IRQ7CafHtqxkbnmyTVGzkbAvcelbGFGEbzZXiYQfBPCVAL5u\nRtdCSDgRZ7GesM2OsCKsfvDW1tYoi8IPW9hHJ+R8FgRQXbw9WqzhsyOijhu+fRFF+OYylQiLyM8B\neCmA+1NKnzY3PQpgXUR2XTR8z/A2QrJU9Y2LVsv5qmc+b1jv64vx+KI8Ni/YZkBEQ7MfbDaEirDN\ngNAR9Z/zNYLJzWViER4K8DcB+IaU0qfczR8EcA7gRQDeNrz/VwD4EgB/dLVLJctMrkWRP+frQ1iR\njQTXDlvYJ1qU4f1emwFhj20KWq4uhLcbWKSd5JhIhEXkQQAvB/AyAEcics/wpr2UUjeltC8ibwDw\nOhH5WwAHAH4GwB8wM4LUIVq5ZocX30iIvQDnqqFZgc9NutmoVvd9+pkX4SgPOBcBs0wlmTQSfhUG\n2RDvcee/C8CvDPdfDeACwFsBdAC8E8D3Tn+JZNkp1QT2+9ZKKEXA1u/1Apwr1ONF2E642a0tymNF\nWG/3reujlkXslkGUSfOEKxd3pJR6AL5/OAipTVSU3dcJtpFwTpBtJKzCmxNga0dYPzhXHU2zH+pE\nwlG3ZC/GhLB2BJkLSgJsh7UXSjZEtBLOD5954VfE2RrBNuq1hXmsONu6EFEKWtQ7jpEwoQiTuSGy\nI7yIej+4FAWvra2FQh5ZH9HEXLQU2QqwjYSjJcl2RZwXXloRRKEIk9aJsiC8cFpboU5mhIqxtzO8\nx2yJCrZbIdbFF4eHh5ei41J2BJt3khIUYTIXRHaEFWArwjnxjUYp06IqRU09YbsM+eDg4FKOsJ+Y\n856wz4KI9snNhSJM5obIjoisiKpo2C7i0Mf1W38OKGdHqAj70pTRIg2fHRGloeX2yc2DIkzmgjqR\ncCS+JU94dXX10nPkKGVH2EhY7QifvlZVJY2QHBRhcq1EQmiFNVrpZs9tb2+PNezUkpS+QHuuPb23\nAvTYdkius/ULOXKr5CjApAqKMGmU6Ge/P6dCq0Lqt3Z/d3d3rFuGFmmP+sX54jvRognd73a72awH\nbztEwuvT0Si+pC4UYdIYOQ82qgdhS1PaQjz+2DfutC3srQgrNve3NE5OTkZWQ6lTss8Fzi1PphCT\nulCESaPUyU6wkbCtgBaNnZ2dUSRse8b5LhlR7q+tAxxVRtNJt1I0HKWhRUuUKcCkLhRh0gh16kF4\nEVYB9h2T7bF2TvYi7CPhSIR9NTRbIc2LcE6AdXgR97WBKcKkLhRh0iiRAPutj4RVfFVk7dbveztC\nI2Gl1I7I7vvylN6K8JFwqVA7hZhMAkWYNIb1fn0tCLtvRVgjYRVajXx1f2trayw6tnaEtivKRcJ2\nAYavjqZFeaIOyVE07Iu0RwXbCakDRZg0Sin/10bC3g+2Imw7KKvo2qGTdn5izuf+2gUYNt1M84An\nsSOs2FbVCiakBEWYNEbJF/aLMbwdYQV4d3d31EFZ09F8JoXPE9YoXEXSesC2IaetjBZNzEVifHp6\nGnbJ8FtC6kARJo3g09BKRXlKdoQV4Tt37mBjYyPbvLNqYi4qT6liG4lvKRpmUR4yKyjCpDFKnrCv\nB+GzI3z35Dt37uDzPu/zsLGxcamwuy/ubu0Inx3hy1P6CDiXI+xFWIlqQFB8ySRQhEljeNGNRHNl\nZWVsYUYuT1gn4jqdTljaUvd9yyIrxLksicjzLS1Jvri4aPF/lSwbFGHSCL4we1QTQoefYLO1IHwB\n96pWRTb69taAzWgodVT26Wa+5gQhs4QiTBrBCnCuzKQOW4zHTrJZn9cLca5XXBQJ2wk0n9sbCbAO\nZjuQ64AiTBrBR8J28s1Guuvr62MLLmw0bEW41Csu6pxs8elkk0TCPhqmEJNZQxEmjeG7I0dFejQj\nwkfCdvFFFAnn2hVFLYu8L2xrSeQiYQowuS4owqQR7KScFWFfqEfzgq0A5yJhGw3nalHocwPjNYNt\nNOyL+VghztWDYO4vaQqKMGkEK5LWD44qpU3qCdfpGad4Ac7ZEVFpyigSphCTWUMRJo1g/eDIE/YV\n0ybJjqgrwACK2RE66pampACTJqAIk8bITczZSFgFOBLiyI6wS5IjC8Kei+yIqLYwJ+ZIm1CESSPk\nImFfI6IUCecm5uxz+P0oGq6yIzgxR9qEIkwaIUpRs9XSIjvCZ0fkJub885SIsiN8nnBdISakCSjC\nZGoiAdRzVnhVTP3SZFsPOLIiogk5L8IlSoV1oj5zUSNQGwEzGiZNQBEmE1HVtFP37QScTUfz7Yt8\nJOwn5fxkHCHLBkWY1KZuz7hbt25dyoSI0tI0ErZ94mxxdi/ChCwjFGEyEVWtinTfC7DPiLACrNXR\novQ0ijBZdijCpDa5Au1RXQe/PNnaEV6Ic3UjtDYwRZgsMxRhMhF+OXJU03dlZaVWJGz9YF/YJ7dU\nmZBlgyJMahNFwlGRdtszLvKEvR2hPrDNI6YdQW4KFGEyET7/17cW0v2SAHsh1kUZdkLOR8MUYbKs\nUIRJbXJ+sM8J1kUZ3o7Ipaitr69fEnO7ZSRMlhmKMJmIXCRsF2XYlXG5qmm2o3KufVFUupKQZYMi\nTGpTJxK2UXBuYs5Hwz4LIrdPyDJCESYTUScS9pNs0bDivLq6Wlkj2FZGq0NpyXLuMbgkmbQBRZjU\npq4nHBVk99kTVXWB60S+UQdkey5XEyIakUj7fUKagCJMJqJOJOwroEUTbbluydFzRd0ydFvaVwHO\nFekpRcks1kOuC4owqc0kkXBOiEtRsH0OL7yREFeNqgppJSHORdeEzBqKMJmIaLFGToBznZKjaNg+\ntn2uiEgwvaDaCFj3q4TYPrbfJ6QpKMKkNrnVcpN6wpEdoY9vn8eeU7xAejG12ygSrmtJ+EiYYkya\non6FbAAi8sMi8rCI7IvIYyLyNhF5jrvPe0Skb8aFiDw428smbVFathxZEWtra2NiHeX+lqyJiEiI\noxZGdQq35zops7syuS4mEmEA9wP4WQDPB/BiAGsAfldENs19EoBfBHAPgHsBfCGAH7z6pZK2KQlw\nFAnb81F2RDQxFwlwJMZ1u2XUEd5cdgQh18FEdkRK6aX2WEReCeBvANwH4H3mpuOU0uNXvjoyd1QJ\ncc6OKFkRJcGt8oVzKWfeD/ZNO+vYERRlch1MGgl77mIQ+X7Onf9OEXlcRD4sIj/uImWyoNQVYG9H\n+Gg4J8RejOtOzNWxInJ+cFWqmn1OQppg6ok5GfyFvB7A+1JKHzM3/RqAvwDwaQBfBeCnADwHwLde\n4TrJnHDVSNinqU2yJFlExroe1xXiSQXYP749JmTWXCU74kEAXwnga+3JlNIvmcOPisijAN4lIs9M\nKT1yhecjLWMFuEqIc8IbpabVJUo/8y3s7fb09HQ0tI29HTmfOEpbI6QpphJhEfk5AC8FcH9K6TMV\nd/8AAAHwbAAUYZLFi160vbi4wNnZGc7Pz3F2dlbcPz4+xt7eHvb397G/v4/Dw0McHR3h5OQE3W4X\nvV5vJNBWvHPibK+FkFkxsQgPBfibAHxDSulTNf7J8zDwjavEmtxgIksgsgrOz8/HItzT01OcnZ2N\nBPXs7Gx0/ujoCAcHB6OhInx8fDwSYSve0QQeJ+hI00wkwsN835cDeBmAIxG5Z3jTXkqpKyLPAvAd\nAN4O4LMAngvgdQDem1L6yOwumywbVSvg9JyKbK/XQ6/XG4mpHXru+PgYh4eHo6ECnIuES9kUFGDS\nFJNGwq/CIKp9jzv/XQB+BcApBvnDPwBgG8BfAvgNAP/pSldJlhob+UYr3+y+Rr3dbhfdbnckqH7b\n7XZxfHw8Et6jo6PRvo2ErV9ct7oaIbNk0jzhYkpbSumvALzwKhdEbia5TAc/bCR8cnIyimxVXO2x\n3fp9a0fkvGAvxoQ0AWtHkLkgl2rm91WENeq1Ua4fNiqOhvWRz8/PsxG496gJmSUUYdI6kfebqwWs\nUbAXYZ140+3h4SFOTk7G/ONo30bCkRdNT5g0DUWYzA0+Go5WvkWRsIrvwcEB9vf3R/snJyej+2sW\nRLSvnnApK4NCTJqCIkzmAh95WiG2ebyRJ3x0dITDw0Ps7++P8oL39vbQ7XbDRRr6OH7hRmQ70Iog\nTUMRJq1TEmC7Kk7FM2dHqPg++eSTePLJJ9Hr9cLly7l0NH9NpWNCZgVFmExErr6CtxJKmQ5eBHUl\nnI12c9GrzQU+OTkZRcFqSejqOI2ES+lu9hwhbUERJrWxk2Z2eXCv1xsrymPv78VVRVSj2O3t7dFK\nODts9GvPHR4eYm9vD3t7ezg4OLi0DFkFPBLb3HJoQtqEIkxqE4nw6enppYI8NgqObIRut4utra3R\npJqKta/hEJ2z1oNmQmj+r51o00i7apKNQkzahiJMahOJsI+A7X00m8FmNFgbYXNzE5ubm5dEuDRO\nTk7GUtGiWhA2Eo7SzuzrIaRtKMKkNpEIqwD7HF+NgLWYTq/XQ6fTwcnJCTqdDjY2NkZbFeFSRwzd\nt8uRc8uQq4rwMNOBzBMUYVIbL7K5CNhaEDadrNPpYH19/dIWQLEjsj32y5XtyEXCFGAyz1CESW28\n0FZZEGtra+h2u6N2R36r+1EWRS6zwq6Y0+jX7lsRzuX+RseEtAVFmNTGim0uAj47O8Pq6ipOT08v\nddrI7QPIpo/5rVocaj1EtYXtMuTcwgv7mghpE4owqY0VXH98fn5+qZ2RbXtUOtbHyk2k2WGtjlxX\nDWtf6GNHW0LmAYowqY0KpN2/uLi41DvOjqi3nL9NHy+KXP2xX8pc6q5clQ1BMSbzAEWY1MZHq75d\nfW5Ere2jNvdVdkFuZV5uS9EliwBFmNSGP+cJmT3FThmEEEKahSJMCCEtQhEmhJAWoQgTQkiLzIMI\nb7R9AYQQ0hCV+jYPIvylbV8AIYQ0xJdW3UHaTjcSkacBeAmATwLotnoxhBAyGzYwEOCHUkqfLd2x\ndREmhJCbzDzYEYQQcmOhCBNCSItQhAkhpEUowoQQ0iJzKcIi8r0i8oiInIjI+0XkH7V9TbNARB4Q\nkb4bH2v7uqZBRO4Xkd8Rkb8evo6XBff5URH5tIgci8j/EJFnt3Gt01D1+kTkjcF7+fa2rrcuIvLD\nIvKwiOyLyGMi8jYReY67T0dEfl5EnhCRAxF5q4h8QVvXPAk1X9973Pt2ISIPtnXNcyfCIvLtAF4L\n4AEAzwPwJwAeEpGnt3phs+MjAO4BcO9wfF27lzM12wA+BOB7AFxKsRGRHwLwfQD+NYCvAXCEwfu4\nfp0XeQWKr2/IOzD+Xr78ei7tStwP4GcBPB/AiwGsAfhdEdk093k9gG8E8C0Avh7AFwH4zWu+zmmp\n8/oSgF/EU+/dFwL4wWu+TnM1hWaIbQwA7wfwn82xAPgrAD/Y9rXN4LU9AOCP276OBl5XH8DL3LlP\nA3i1Od4FcALg29q+3hm9vjcC+K22r20Gr+3pw9f3deZ96gH4F+Y+XzG8z9e0fb1XfX3Dc78H4HVt\nX5uOuYqERWQNwH0A3q3n0uB/7V0AXtDWdc2YLx/+xP2EiPyqiHxx2xc0a0TkmRhEGPZ93AfwASzP\n+wgALxz+5P1TEXlQRD6/7QuagrsYRIafGx7fh0GdcfvefRzAp7CY751/fcp3isjjIvJhEflxFylf\nK/NW1P3pAFYAPObOP4bBt/Gi834ArwTwcQx+Ar0GwO+LyD9IKR21eF2z5l4MPvjR+3jv9V9OI7wD\ng5/ojwD4MgA/AeDtIvKCYeAw98igrcnrAbwvpaRzE/cCOB1+aVoW7r3LvD4A+DUAf4HBr7WvAvBT\nAJ4D4Fuv/SIxfyK81KSUHjKHHxGRhzH4MHwbBj9vyYKQUnqLOfyoiHwYwCcAvBCDn7uLwIMAvhKL\nOy9Rhb6+r7UnU0q/ZA4/KiKPAniXiDwzpfTIdV4gMH8Tc08AuMDAMLfcA+DR67+cZkkp7QH4MwAL\nkzVQk0cx8PJvxPsIAMM/3iewIO+liPwcgJcCeGFK6dPmpkcBrIvIrvsnC/Xeudf3mYq7fwCDz2sr\n791ciXBK6QzABwG8SM8Nf1K8CMAftnVdTSEiOxj8lK36kCwUQ0F6FOPv4y4GM9ZL9z4CgIg8A8DT\nsADv5VCgvgnAP00pfcrd/EEA5xh/774CwJcA+KNru8grUPH6Ip6HgX3Wyns3j3bE6wC8SUQ+COBh\nAK8GsAXgTW1e1CwQkZ8G8N8wsCD+LoD/gMEH/s1tXtc0iMg2BpGDtkt+log8F8DnUkp/iYEX9yMi\n8ucYVMj7MQyyXH67hcudmNLrG44HMPCEHx3e7ycx+FXz0OVHmx+G+bAvB/AyAEcior9W9lJK3ZTS\nvoi8AcDrRORvARwA+BkAf5BSeridq65P1esTkWcB+A4AbwfwWQDPxUBz3ptS+kgb19x6ekYmreR7\nMPjDPcHg2/er276mGb2uN2MgRCcYzDb/OoBntn1dU76Wb8Ag9efCjf9q7vMaDCY/jjEQp2e3fd2z\neH0YlCl8JwYC3AXw/wD8FwB/p+3rrvG6otd0AeAV5j4dDHJtn8BAhH8DwBe0fe2zeH0AngHgPQAe\nH34uP47BpOpOW9fMUpaEENIic+UJE0LITYMiTAghLUIRJoSQFqEIE0JIi1CECSGkRSjChBDSIhRh\nQghpEYowIYS0CEWYEEJahCJMCCEtQhEmhJAWoQgTQkiL/H9Kx/J9nkQFJgAAAABJRU5ErkJggg==\n", 43 | "text/plain": [ 44 | "" 45 | ] 46 | }, 47 | "metadata": {}, 48 | "output_type": "display_data" 49 | } 50 | ], 51 | "source": [ 52 | "from PIL import Image\n", 53 | "import matplotlib.pyplot as plt\n", 54 | "import numpy as np\n", 55 | "\n", 56 | "# load image using PIL\n", 57 | "img = Image.open(\"7.png\")\n", 58 | "# convert to black and white\n", 59 | "img = img.convert(\"L\")\n", 60 | "# convert to numpy array\n", 61 | "img = np.asarray(img)\n", 62 | "# display\n", 63 | "% matplotlib inline\n", 64 | "plt.imshow(img, cmap='gray')\n", 65 | "\n", 66 | "from QNN.layers import *\n", 67 | "import pickle\n", 68 | "# load the qnn\n", 69 | "qnn = pickle.load(open(\"mnist-w1a2.pickle\", \"rb\"))\n", 70 | "qnn" 71 | ] 72 | }, 73 | { 74 | "cell_type": "markdown", 75 | "metadata": {}, 76 | "source": [ 77 | "## Thresholding for 2-bit Activations\n", 78 | "\n", 79 | "As you can see, there are exactly as many layers as there were in the previous one and the layer names are almost the same, with one exception: here we have QNNThresholdingLayer instead of QNNBipolarThresholdingLayer, since we now have 2-bit activations. This means each output element from the thresholding layer can have four different values: 0, 1, 2 or 3. \n", 80 | "\n", 81 | "Let's start by examining one of those thresholding layers to see what the parameters look like:" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": 20, 87 | "metadata": {}, 88 | "outputs": [ 89 | { 90 | "name": "stdout", 91 | "output_type": "stream", 92 | "text": [ 93 | "Threshold parameter shape: (3, 256)\n", 94 | "Thresholds for the first vector element: [ 5 49 78]\n", 95 | "Thresholds for the second vector element: [-23 25 58]\n" 96 | ] 97 | } 98 | ], 99 | "source": [ 100 | "print(\"Threshold parameter shape: \" + str(qnn[2].thresholds.shape))\n", 101 | "print(\"Thresholds for the first vector element: \" + str(qnn[2].thresholds[:, 0]))\n", 102 | "print(\"Thresholds for the second vector element: \" + str(qnn[2].thresholds[:, 1]))" 103 | ] 104 | }, 105 | { 106 | "cell_type": "markdown", 107 | "metadata": {}, 108 | "source": [ 109 | "Recall that the **thresholds** parameter of the QNNThresholdingLayer has two dimensions: the first dimension is the particular \"stair step\" location, and the second dimension is the particular input vector index that this threshold will be applied to. A few things worth pointing out about these thresholds:\n", 110 | "1. Thresholds can be totally different for different input vector elements. Here we can see that the thresholds for the second input vector element are lower than the thresholds for the first one. We will be generalizing this to *channels* in the next section for convolutional neural networks.\n", 111 | "2. Since we want four different values coming out of the thresholding function, we have three steps. In general, if you want to divide a space into N regions, you will need N-1 separators.\n", 112 | "3. Notice how the threshold values are always larger for the next step, but the difference between consecutive steps is not always the same. For instance, the thresholds for the first vector element have differences of 49-5=44, and then 78-49=29.\n", 113 | "\n", 114 | "Let's plot the threshold function for the first vector element to understand this a little better:" 115 | ] 116 | }, 117 | { 118 | "cell_type": "code", 119 | "execution_count": 21, 120 | "metadata": {}, 121 | "outputs": [ 122 | { 123 | "data": { 124 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgwAAAFdCAYAAABvvDXAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAFSNJREFUeJzt3X+Q7Xdd3/HXO7/4FbiJyZBAoAZ7Kb0lStkVRVpQixix\ngOPYQbfS8cdYQbF27gyCUi2UdoZilV8KRcQQa2Bb+kMbW+u1+KOMFkjdBSxDAioJSiCxuwkbuQnc\nkHz6xzkXzl2T+9lzs989u+c8HjNn7uz3fPf7+dx8snef8z3f8z3VWgsAwOmcNesJAAD7n2AAALoE\nAwDQJRgAgC7BAAB0CQYAoEswAABd58x6ApOq6qIkVya5KcnnZjsbADhQHpzk8iTHWmubu33wfRUM\nGcXCO2Y9CQA4wL47yTt3+6D7LRhuSpJrrrkmR44cmfFU2A1Hjx7N6173ullPg11iPeePNZ0f119/\nfV7wghck49+lu22/BcPnkuTIkSNZWlqa9VzYBYcOHbKWc8R6zh9rOpcGeUnfRY8AQJdgAAC6BAMA\n0CUYGNTKysqsp8Ausp7zx5qyU4KBQfnHaL5Yz/ljTdkpwQAAdAkGAKBLMAAAXYIBAOgSDABAl2AA\nALoEAwDQJRgAgC7BAAB0CQYAoEswAABdggEA6BIMAECXYAAAugQDANAlGACALsEAAHQJBgCga8+C\noap+vKrurarX7tWYAMDu2JNgqKqnJPnBJB/ai/EAgN01eDBU1flJrknyA0k+M/R4AMDu24szDG9K\n8uuttd/Zg7EAgAGcM+TBq+q7kvztJF895DgAwLAGC4aqekyS1yf5ptba3dN879GjR3Po0KFTtq2s\nrGRlZWUXZwgAB9Pq6mpWV1dP2ba1tTXomNVaG+bAVd+W5L8kuSdJjTefnaSNtz2obRu8qpaSrK2t\nrWVpaWmQeQHAPFpfX8/y8nKSLLfW1nf7+EO+JPHuJF+5bdvVSa5P8q+3xwIAsH8NFgytteNJPjK5\nraqOJ9lsrV0/1LgAwO7b6zs9OqsAAAfQoO+S2K619vf2cjwAYHf4LAkAoEswAABdggEA6BIMAECX\nYAAAugQDANAlGACALsEAAHQJBgCgSzAAAF2CAQDoEgwAQJdgAAC6BAMA0CUYAIAuwQAAdAkGAKBL\nMAAAXYIBAOgSDABAl2AAALoEAwDQJRgAgC7BAAB0nTPrCQDArN1xR7K+PutZnLkrrhh+DMEAwML7\nqZ9K3vjGWc/izF17bXLZZcOOIRgAWHg335w8/enJVVfNeiZn5lGPSj760WHHEAwALLzNzeQxj0kO\nH571TPYvFz0CsPA2NpKLLpr1LPY3wQDAwtvcTC6+eNaz2N8EAwALrbVRMDjDcHqCAYCFdvx4cuKE\nYOgRDAAstM3N0Z+C4fQEAwALbWNj9KdgOD3BAMBCO3mGwUWPpycYAFhoXpLYGcEAwELb3EzOOy95\n2MNmPZP9TTAAsNBOvqWyatYz2d8EAwALzV0ed0YwALDQ3OVxZwQDAAvNXR53RjAAsNAEw84IBgAW\nmmDYGcEAwELb2HANw04IBgAW1okTyWc/6wzDTggGABaWuzzunGAAYGEJhp0TDAAsLMGwc4IBgIV1\n8qOtXfTYJxgAWFibm8lZZyUXXDDrmex/ggGAhbW5mVx44SgaOL1B/xNV1U9U1XVVdUdV3VpVv1pV\nf2PIMQFgp9y0aeeGbqqnJ/m5JF+b5JuSnJvkt6rqIQOPCwBdgmHnzhny4K21b538uqq+N8lfJFlO\n8vtDjg0APe7yuHN7/arNBUlaktv2eFwA+CucYdi5Qc8wTKqqSvL6JL/fWvvIXo0LwH274Ybklltm\nPYvZuvnm5GlPm/UsDoY9C4Ykb07yt5L8nd6OR48ezaFDh07ZtrKykpWVlYGmBrBYTpxInvSk0Z+L\n7vLLZz2D6a2urmZ1dfWUbVtbW4OOWa21QQdIkqr6+STPTfL01tqfnWa/pSRra2trWVpaGnxeAIvq\n059OHv3o5K1vTb7xG2c9m9k566zkcY9LqmY9kwdufX09y8vLSbLcWlvf7eMPfoZhHAvfluTrTxcL\nAOydk7dEvuKK5PDh2c6Fg2HQYKiqNydZSfK8JMer6pLxU1uttc8NOTYA9+/kLZFd8MdODf0uiRcl\neUSS30vyqYnH8wceF4DTOHmGwVsK2amh78PgZpsA+5DPUGBafqEDLCCfocC0/K8CsIDcsIhpCQaA\nBbSxIRiYjmAAWECbmy54ZDqCAWABeUmCaQkGgAUkGJiWYABYQIKBaQkGgAVzzz3JbbcJBqYjGAAW\nzGc+k7TmokemIxgAFszJ20I7w8A0BAPAghEMnAnBALBgBANnQjAALBgfbc2ZEAwAC2ZzM3n4w5Pz\nzpv1TDhIBAPAgnEPBs6EYABYMIKBMyEYABaMYOBMCAaABbOx4aZNTE8wACwYZxg4E4IBYMEIBs6E\nYABYIK0JBs6MYABYIMePJydOCAamJxgAFsjJuzy66JFpCQaABeJzJDhT58x6AsB8u+ee5Lrrks9/\nftYzIUnW1kZ/CgamJRiAQf3mbybPec6sZ8GkhzwkeeQjZz0LDhrBAAzq5puTs85KbrghqZr1bEiS\nCy4YRQNMQzAAg9rcTC68MHn842c9E+CBcNEjMKiNDa+XwzwQDMCg3CQI5oNgAAa1uek9/zAPBAMw\nKGcYYD4IBmBQggHmg2AABuWiR5gPggEYzD33JLffLhhgHggGYDCf+czo45Rd9AgHn2AABuODjmB+\nCAZgMIIB5odgAAazsTH6UzDAwScYgME4wwDzQzAAg9ncTM4/PznvvFnPBHigBAMwGLeFhvkhGIDB\nuMsjzA/BAAzGXR5hfggGYDDOMMD8EAzAYAQDzA/BAAxGMMD8EAzAIFrzLgmYJ4IBGMRnP5ucOOEM\nA8wLwQAMwl0eYb4IBmAQggHmy54EQ1W9uKpurKq7qup9VfWUvRgXmB3BAPNl8GCoqu9M8rNJXpHk\nyUk+lORYVbkUCuaYYID5shdnGI4m+YXW2r9rrd2Q5EVJ7kzy/XswNjAjGxujD506//xZzwTYDYMG\nQ1Wdm2Q5yW+f3NZaa0neneTrhhwbmK2T92ComvVMgN1wzsDHvzjJ2Ulu3bb91iRPuL9v+sM/TO64\nY8hpAUP78Ie9HAHzZOhgOCMvfOHRJIe2bV0ZP4CD4rnPnfUMYD6trq5mdXX1lG1bW1uDjlmjVwgG\nOvjoJYk7k3xHa+3aie1XJznUWvv2bfsvJVn7tV9byxOfuDTYvIC9cdllyUMeMutZwGJYX1/P8vJy\nkiy31tZ3+/iDnmFord1dVWtJnpnk2iSpqhp//cb7+77HPjY5fHjImQEA09iLlyRem+TqcThcl9G7\nJh6a5Oo9GBsA2AWDB0Nr7V3jey68KsklST6Y5MrW2v8bemwAYHfsyUWPrbU3J3nzXowFAOw+nyUB\nAHQJBgCgSzAAAF2CAQDoEgwAQJdgAAC6BAMA0CUYAIAuwQAAdAkGAKBLMAAAXYIBAOgSDABAl2AA\nALoEAwDQJRgAgC7BAAB0CQYAoEswAABdggEA6BIMAECXYAAAugQDANAlGACALsEAAHQJBgCgSzAA\nAF2CAQDoEgwAQJdgAAC6BAMA0CUYAIAuwQAAdAkGAKBLMAAAXYIBAOgSDABAl2AAALoEAwDQJRgA\ngC7BAAB0CQYAoEswAABdggEA6BIMAECXYAAAugQDANAlGACALsEAAHQJBgCgSzAAAF2DBENVfXlV\nva2qPl5Vd1bVH1fVK6vq3CHGAwCGdc5Ax/2bSSrJP07yp0muSPK2JA9N8tKBxgQABjJIMLTWjiU5\nNrHppqr6mSQvimAAgANnL69huCDJbXs4HgCwS/YkGKrqcJIfSfKWvRgPANhdU70kUVWvTvKy0+zS\nkhxprX1s4nsuS/I/kvyH1tpVOxnn6NGjOXTo0CnbVlZWsrKyMs10AWAura6uZnV19ZRtW1tbg45Z\nrbWd71x1UZKLOrt9vLX2hfH+j07yu0n+d2vt+3Zw/KUka2tra1laWtrxvABg0a2vr2d5eTlJlltr\n67t9/KnOMLTWNpNs7mTf8ZmF30nyf5J8//RTAwD2i0HeJTE+s/B7SW7M6F0Rj6yqJElr7dYhxgQA\nhjPUfRieleQrxo8/H2+rjK5xOHugMQGAgQzyLonW2i+31s7e9jirtSYWAOAA8lkSAECXYAAAugQD\nANAlGACALsEAAHQJBgCgSzAAAF2CAQDoEgwAQJdgAAC6BAMA0CUYAIAuwQAAdAkGAKBLMAAAXYIB\nAOgSDABAl2AAALoEAwDQJRgAgC7BAAB0CQYAoEswAABdggEA6BIMAECXYAAAugQDANAlGACALsEA\nAHQJBgCgSzAAAF2CAQDoEgwAQJdgAAC6BAMA0CUYAIAuwQAAdAkGAKBLMAAAXYIBAOgSDABAl2AA\nALoEAwDQJRgAgC7BAAB0CQYAoEswAABdggEA6BIMAECXYAAAugQDANA1eDBU1XlV9cGqureqvmro\n8QCA3bcXZxh+Osknk7Q9GAsAGMCgwVBVz07yrCQvSVJDjgUADOecoQ5cVZckeWuS5yW5a6hxAIDh\nDXmG4e1J3txa+8CAYwAAe2CqMwxV9eokLzvNLi3JkSTfkuT8JK85+a3TjHP06NEcOnTolG0rKytZ\nWVmZ5jAAMJdWV1ezurp6yratra1Bx6zWdn4tYlVdlOSizm43JnlXkuds2352ki8keUdr7fvu5/hL\nSdbW1taytLS043kBwKJbX1/P8vJykiy31tZ3+/hTnWForW0m2eztV1X/JMk/m9j06CTHkjw/yXXT\njAkAzN4gFz221j45+XVVHc/oZYmPt9Y+NcSYAMBw9vJOj+7DAAAH1GBvq5zUWvtERtcwAAAHkM+S\nAAC6BAMA0CUYAIAuwQAAdAkGAKBLMAAAXYIBAOgSDABAl2AAALoEAwDQJRgAgC7BAAB0CQYAoEsw\nAABdggEA6BIMAECXYAAAugQDANAlGACALsHAoFZXV2c9BXaR9Zw/1pSdEgwMyj9G88V6zh9ryk4J\nBgCgSzAAAF2CAQDoOmfWE9jmwUly/fXXz3oe7JKtra2sr6/PehrsEus5f6zp/Jj43fngIY5frbUh\njntGquofJnnHrOcBAAfYd7fW3rnbB91vwXBRkiuT3JTkc7OdDQAcKA9OcnmSY621zd0++L4KBgBg\nf3LRIwDQJRgAgC7BAAB0CQYAoEswAABdMwmGqnp5Vf1BVR2vqtvuZ5/HVtV/H+9zS1X9dFWdtW2f\nb6iqtar6XFV9rKq+Z2/+BvRU1U1Vde/E456qeum2fb6qqt5TVXdV1Seq6sdmNV/6qurFVXXjeL3e\nV1VPmfWc6KuqV2z7Wby3qj4y8fyDqupNVbVRVX9ZVf+pqh45yznzJVX19Kq6tqpuHq/d8+5jn1dV\n1aeq6s6q+p9VdXjb8xdW1Tuqaquqbq+qt1XVw6ady6zOMJyb5F1J/u19PTkOg9/I6E6UT03yPUm+\nN8mrJva5PMl/S/LbSZ6U5A1J3lZVzxpu2kyhJfnJJJckuTTJo5L83Mknq+rhSY4luTHJUpIfS/LK\nqvqBvZ8qPVX1nUl+Nskrkjw5yYeSHKuqi2c6MXbqw/nSz+KlSf7uxHOvT/L3k3xHkmckeXSS/7zX\nE+R+PSzJB5P8cEb/rp6iql6W5EeS/GCSr0lyPKOfzfMmdntnkiNJnpnRWj8jyS9MPZPW2sweGYXA\nbfex/dlJ7k5y8cS2Fya5Pck5469fk+SPtn3fapLfmOXfyeOLa3Fjkh89zfM/lGTj5HqOt706yUdm\nPXeP+1yv9yV5w8TXleSTSV4667l5dNfuFUnW7+e5RyT5fJJvn9j2hCT3JvmaWc/d46+s171Jnrdt\n26eSHN22pnclef746yPj73vyxD5XJvlCkkunGX+/XsPw1CT/t7W2MbHtWJJDSZ44sc+7t33fsSRf\nN/z02KEfH5/mXK+ql1TV2RPPPTXJe1prX5jYdizJE6rq0N5Ok9OpqnOTLGd0Ni9J0kb/6rw7ft4O\nisePT2n/aVVdU1WPHW9fzuhM7uTafjTJn8Xa7ntV9biMzhhNrt8dSd6fL63fU5Pc3lr7wMS3vjuj\nsxVfO814+zUYLk1y67Ztt048d7p9HlFVDxpwbuzMG5J8V5JvSPKWJC/P6KzQSTtZY/aHi5Ocnfte\nL2u1/70vo5d0r0zyoiSPS/Ke8WvYlyY5Mf4lM8naHgyXZvSL/3Q/m5cm+YvJJ1tr9yS5LVOu8a59\nWmVVvTrJy06zS0typLX2sd0ak701zRq31l4/sf3DVXV3krdU1U+01u4edKLAF7XWjk18+eGqui7J\nJ5I8Pz6zhyns5sdb/0ySt3f2+fgOj3VLku1XYF8y/vPTE/tcch/73NFa+/wOx2E6D2SN35/R/2+X\nJ/nj3P/6Zfwc+8dGknty3+tlrQ6Y1tpWVX0syeGMTk2fV1WP2HaWwdoeDLdkdD3RJTn1LMMlST4w\nsc8p73oZvzz8ZZlyjXctGNrok7F269Ox3pvk5VV18cR1DN+cZCvJ9RP7PHvb933zeDsDeIBr/OSM\nLrw5eWrsvUn+VVWdPT49lozW76Otta0HNlN2U2vt7qpay+gK62uTpKpq/PUbZzk3pldV5yf560l+\nOclaRhe/PTPJr46ff0KSvxb/lu57rbUbq+qWjNbvj5Kkqh6R0bUJbxrv9t4kF1TVkyeuY3hmRqHx\n/mnG280zDDs2vuDmy5J8eZKzq+pJ46f+pLV2PMlvJflIkl8Zv2XkUUn+ZZKfnzid/ZYkL66q1yS5\nKqP/AP8gybfu3d+E+1JVT83of9jfTfKXSZ6W5LVJfmUiBt6Z5J8nuWq8hl+Z5EeT/NO9nzE78Nok\nV4/D4bokR5M8NMnVs5wUfVX1b5L8ekYvQ1yW5F9kFAn/vrV2R1X9UpLXVtXtGf28vjHJH7TWrpvV\nnPmS8bUmhzP6BZ8kXzH+nXlba+3PM3pb7E9W1Z8kuSmj35WfTPJfk6S1dkNVHUvyi1X1Q0nOy+gt\n7quttenOIs3orSFvz+gU5/bHMyb2eWxG91n4bEanWl6T5Kxtx3lGRoV8V0anuf/RrN/24tGS0dmE\n92Z0Uc3xjN4D/tIk527b74ok/yvJnRldlf2SWc/d47Tr+sPjf5DuGq/vV896Th47WrfV8S+Qu8Y/\nZ+9M8riJ5x80/gWykVEw/Mckj5z1vD2+uD5fn9HZ2e2/L6+a2OeVGb298s6M3m12eNsxLkhyTUZn\n6W9P8otJHjrtXGp8MACA+7Vf31YJAOwjggEA6BIMAECXYAAAugQDANAlGACALsEAAHQJBgCgSzAA\nAF2CAQDoEgwAQNf/B4xdUdb76DgtAAAAAElFTkSuQmCC\n", 125 | "text/plain": [ 126 | "" 127 | ] 128 | }, 129 | "metadata": {}, 130 | "output_type": "display_data" 131 | } 132 | ], 133 | "source": [ 134 | "def simpleThreshold(x):\n", 135 | " if x <= 5:\n", 136 | " return 0\n", 137 | " elif x <= 49:\n", 138 | " return 1\n", 139 | " elif x <= 78:\n", 140 | " return 2\n", 141 | " else:\n", 142 | " return 3\n", 143 | "\n", 144 | "x = range(-100,100)\n", 145 | "y = map(simpleThreshold, x)\n", 146 | "plt.plot(x, y)\n", 147 | "plt.ylim([-5, 5])\n", 148 | "plt.show()" 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "metadata": {}, 154 | "source": [ 155 | "You can see how the returned value goes from 0 to 3 as the input value increases, mapping any real number to one of its \"stair steps\". Notice that we can represent any of the values returned by this particular function using two bits: 0 is 00, 1 is 01, 2 is 10 and 3 is 11 in binary, so this is actually a two-bit [quantizer](https://en.wikipedia.org/wiki/Quantization_(signal_processing)). \n", 156 | "\n", 157 | "Let's look at one of the activations produced by our network to see this in action:" 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": 22, 163 | "metadata": {}, 164 | "outputs": [ 165 | { 166 | "data": { 167 | "text/plain": [ 168 | "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 169 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 170 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 171 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 172 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 173 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 174 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 175 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 176 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 3, 3, 1,\n", 177 | " 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 178 | " 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 0, 0, 0, 0, 0, 0, 0,\n", 179 | " 0, 0, 0, 0, 0, 1, 2, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 0, 0,\n", 180 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0,\n", 181 | " 3, 3, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 182 | " 0, 0, 0, 0, 1, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 183 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 184 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 3, 1, 0, 0, 0, 0, 0, 0,\n", 185 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 3, 1, 0, 0,\n", 186 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2,\n", 187 | " 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 188 | " 0, 0, 0, 0, 3, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 189 | " 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 190 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 3, 1, 0, 0, 0, 0, 0, 0, 0,\n", 191 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 0, 0, 0,\n", 192 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3,\n", 193 | " 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 194 | " 0, 0, 1, 3, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 195 | " 0, 0, 0, 0, 0, 0, 0, 3, 3, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 196 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0,\n", 197 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 3, 3, 1, 0, 0, 0,\n", 198 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 3,\n", 199 | " 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 200 | " 0, 0, 2, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 201 | " 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n", 202 | " 0, 0])" 203 | ] 204 | }, 205 | "execution_count": 22, 206 | "metadata": {}, 207 | "output_type": "execute_result" 208 | } 209 | ], 210 | "source": [ 211 | "qnn[0].execute(img)" 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "metadata": {}, 217 | "source": [ 218 | "There we go -- the activation only contains the values 0, 1, 2 and 3. Just to make sure this network actually works, let's run it all the way through:" 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "execution_count": 23, 224 | "metadata": {}, 225 | "outputs": [ 226 | { 227 | "name": "stdout", 228 | "output_type": "stream", 229 | "text": [ 230 | "[ 1.83423800e-04 2.80956471e-04 1.83361667e-03 8.53998803e-04\n", 231 | " 1.80900855e-04 4.23054777e-04 6.18959071e-05 9.95056417e-01\n", 232 | " 9.90050351e-05 1.02673042e-03]\n", 233 | "The QNN predicts this is a 7 with 99.505642 percent probability\n" 234 | ] 235 | } 236 | ], 237 | "source": [ 238 | "# get the predictions array\n", 239 | "res = predict(qnn, img)\n", 240 | "# return the index of the largest prediction\n", 241 | "winner_ind = np.argmax(res)\n", 242 | "# the sum of the output values add up to 1 due to softmax,\n", 243 | "# so we can interpret them as probabilities\n", 244 | "winner_prob = 100 * res[winner_ind]\n", 245 | "print(res)\n", 246 | "print(\"The QNN predicts this is a %d with %f percent probability\" % (winner_ind, winner_prob))" 247 | ] 248 | }, 249 | { 250 | "cell_type": "markdown", 251 | "metadata": {}, 252 | "source": [ 253 | "We won't look at this network in any more details, since everything else is almost exactly the same as the previous one. You can look inside the weights for the fully-connected layers, which will still consist of -1 and +1s, so we don't have to use the same quantization for the weights and activations. In fact, the particular quantization we use can potentially be different in each layer! The \"magic numbers\" may have changed, but the computation is the same.\n", 254 | "\n", 255 | "In the next section, we will start looking at more complicated (e.g. convolutional) network topologies, and apply them to datasets slightly more exciting than black-and-white handwritten digits." 256 | ] 257 | } 258 | ], 259 | "metadata": { 260 | "kernelspec": { 261 | "display_name": "Python 2", 262 | "language": "python", 263 | "name": "python2" 264 | }, 265 | "language_info": { 266 | "codemirror_mode": { 267 | "name": "ipython", 268 | "version": 2 269 | }, 270 | "file_extension": ".py", 271 | "mimetype": "text/x-python", 272 | "name": "python", 273 | "nbconvert_exporter": "python", 274 | "pygments_lexer": "ipython2", 275 | "version": "2.7.12" 276 | } 277 | }, 278 | "nbformat": 4, 279 | "nbformat_minor": 2 280 | } 281 | -------------------------------------------------------------------------------- /7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/7.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 2-Clause License 2 | 3 | Copyright (c) 2017, Yaman Umuroglu 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | * Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | * Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 17 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 18 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 19 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 20 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 21 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 22 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 23 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 24 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 25 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26 | -------------------------------------------------------------------------------- /QNN/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/QNN/__init__.py -------------------------------------------------------------------------------- /QNN/im2col.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | # adapted from A. Karpathy's CS231 im2col code 4 | # utilities to generate a patch matrix from a multichannel image 5 | # of shape (batches, channels, height, width) 6 | 7 | def get_im2col_indices(x_shape, field_height, field_width, padding=0, stride_y=1, stride_x=1): 8 | # First figure out what the size of the output should be 9 | N, C, H, W = x_shape 10 | assert (H + 2 * padding - field_height) % stride_y == 0 11 | assert (W + 2 * padding - field_width) % stride_x == 0 12 | out_height = (H + 2 * padding - field_height) / stride_y + 1 13 | out_width = (W + 2 * padding - field_width) / stride_x + 1 14 | 15 | i0 = np.repeat(np.arange(field_height), field_width) 16 | i0 = np.tile(i0, C) 17 | i1 = stride_y * np.repeat(np.arange(out_height), out_width) 18 | j0 = np.tile(np.arange(field_width), field_height * C) 19 | j1 = stride_x * np.tile(np.arange(out_width), out_height) 20 | i = i0.reshape(-1, 1) + i1.reshape(1, -1) 21 | j = j0.reshape(-1, 1) + j1.reshape(1, -1) 22 | 23 | k = np.repeat(np.arange(C), field_height * field_width).reshape(-1, 1) 24 | 25 | return (k, i, j) 26 | 27 | 28 | def im2col_indices(x, field_height, field_width, padding=0, stride_y=1, stride_x=1): 29 | """ An implementation of im2col based on some fancy indexing """ 30 | # Zero-pad the input 31 | p = padding 32 | x_padded = np.pad(x, ((0, 0), (0, 0), (p, p), (p, p)), mode='constant') 33 | 34 | k, i, j = get_im2col_indices(x.shape, field_height, field_width, padding, 35 | stride_y, stride_x) 36 | 37 | cols = x_padded[:, k, i, j] 38 | C = x.shape[1] 39 | cols = cols.transpose(1, 2, 0).reshape(field_height * field_width * C, -1) 40 | return cols 41 | 42 | 43 | def col2im_indices(cols, x_shape, field_height=3, field_width=3, padding=0, 44 | stride_y=1, stride_x=1): 45 | """ An implementation of col2im based on fancy indexing and np.add.at """ 46 | N, C, H, W = x_shape 47 | H_padded, W_padded = H + 2 * padding, W + 2 * padding 48 | x_padded = np.zeros((N, C, H_padded, W_padded), dtype=cols.dtype) 49 | k, i, j = get_im2col_indices(x_shape, field_height, field_width, padding, 50 | stride_y, stride_x) 51 | cols_reshaped = cols.reshape(C * field_height * field_width, -1, N) 52 | cols_reshaped = cols_reshaped.transpose(2, 0, 1) 53 | np.add.at(x_padded, (slice(None), k, i, j), cols_reshaped) 54 | if padding == 0: 55 | return x_padded 56 | return x_padded[:, :, padding:-padding, padding:-padding] 57 | -------------------------------------------------------------------------------- /QNN/layers.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import math 3 | from QNN.im2col import im2col_indices 4 | 5 | def convert(qnn): 6 | new_qnn = [] 7 | for L in qnn: 8 | if L.get_type() == "FullyConnectedLayer": 9 | Wnew = np.asarray(L.W, dtype=np.int8) 10 | new_qnn += [QNNFullyConnectedLayer(Wnew)] 11 | elif L.get_type() == "BipolarThresholdingLayer": 12 | new_qnn += [QNNBipolarThresholdingLayer(L.thresholds)] 13 | elif L.get_type() == "ThresholdingLayer": 14 | new_qnn += [QNNThresholdingLayer(L.thresholds)] 15 | elif L.get_type() == "LinearLayer": 16 | new_qnn += [QNNScaleShiftLayer(L.A, L.B)] 17 | elif L.get_type() == "PoolingLayer": 18 | new_qnn += [QNNPoolingLayer(L.idim, L.chans, L.k, L.s, L.poolFxn)] 19 | elif L.get_type() == "ConvolutionLayer": 20 | Wnew = L.W.reshape((L.ofm, L.ifm, L.k, L.k)) 21 | Wnew = Wnew.astype(np.int8) 22 | new_qnn += [QNNConvolutionLayer(Wnew, L.idim, L.pad, L.stride, L.padVal)] 23 | elif L.get_type() == "SoftmaxLayer": 24 | new_qnn += [QNNSoftmaxLayer()] 25 | elif L.get_type() == "ReLULayer": 26 | new_qnn += [QNNReLULayer()] 27 | else: 28 | raise Exception("Unrecognized layer type") 29 | return new_qnn 30 | 31 | # Quick and easy forward path definitions for a few layer types 32 | # Despite the QNN in the class names, these layers will work for any numpy-supported 33 | # data type, since the internals are mostly implemented with numpy function calls. 34 | # Assumptions: 35 | 36 | # 1. Data layout for the input image and all intermediate activations is [channels][rows][columns] in C style layout, that is to say, looks like this if laid out as a 1D array, 37 | # assuming three columns per row, two rows per channel, two channels img[2][2][3]: 38 | # pixel pixel pixel pixel pixel pixel pixel pixel pixel pixel pixel pixel 39 | # ^ ^ ^ ^ 40 | # row 0 row 1 row 0 row 1 41 | # ^ ^ 42 | # channel 0 channel 1 43 | 44 | # 2. All data going into and going between layers is exchanged as a flattened numpy vector (a one-dimensional array). It is the next layer's responsibility to reshape the input data as needed. 45 | 46 | # 3. Only a single image per execution (batch size = 1). 47 | 48 | def predict(qnn, input_img): 49 | "Predict the class of input_img using a quantized neural network." 50 | activations = input_img 51 | for layer in qnn: 52 | activations = layer.execute(activations) 53 | return activations 54 | 55 | class QNNLayer(object): 56 | "Base class for all layer types." 57 | 58 | def layerType(self): 59 | "Return the layer class name as a string." 60 | return self.__class__.__name__ 61 | 62 | def execute(self, v): 63 | "Forward-propagate given flat vector v through this layer." 64 | return v 65 | 66 | class QNNFullyConnectedLayer(QNNLayer): 67 | """ 68 | Fully-connected network layers as matrix-vector multiplication. 69 | Note that bias is not implemented, this can be done by adding a LinearLayer 70 | with A=1 B=bias following the QNNFullyConnectedLayer. 71 | """ 72 | def __init__(self, W): 73 | self.W = W 74 | 75 | def execute(self, v): 76 | return np.dot(self.W, v) 77 | 78 | class QNNThresholdingLayer(QNNLayer): 79 | "Given a set of thresholds, return the number of thresholds crossed." 80 | def __init__(self, thresholds): 81 | # we expect the thresholds array in the following format: 82 | # thresholds = [levels][channels] 83 | if thresholds.ndim == 1: 84 | self.thresholds = thresholds.reshape((len(thresholds),-1)) 85 | elif thresholds.ndim == 2: 86 | self.thresholds = thresholds 87 | else: 88 | raise Exception("Thresholds array must be 1- or 2-dimensional") 89 | 90 | def execute(self, v): 91 | # interpret as multi-channel image, where the number of channels is 92 | # decided as the number of threshold channels 93 | vr = v.reshape((self.thresholds.shape[1], -1)) 94 | ret = np.zeros(vr.shape, dtype=np.int) 95 | for t in self.thresholds: 96 | for c in range(self.thresholds.shape[1]): 97 | ret[c] += map(lambda x: 1 if x == True else 0, vr[c] >= t[c]) 98 | return ret.flatten() 99 | 100 | class QNNBipolarThresholdingLayer(QNNThresholdingLayer): 101 | "A 1-level QNNThresholdingLayer that returns -1 and +1 instead of 0 and 1." 102 | def __init__(self, thresholds): 103 | super(QNNBipolarThresholdingLayer, self).__init__(thresholds) 104 | if self.thresholds.shape[0] != 1: 105 | raise Exception("BipolarThresholdingLayer can only have one level") 106 | 107 | def execute(self, v): 108 | # just the base implementation, but scaled by 2x-1 such that the output 109 | # is -1, +1 instead of 0, 1. this could have been done with a following 110 | # LinearLayer, but this way we keep the bipolar thresholding as a stand- 111 | # alone operation. 112 | ret = super(QNNBipolarThresholdingLayer, self).execute(v) 113 | return 2*ret - 1 114 | 115 | class QNNScaleShiftLayer(QNNLayer): 116 | "Using 1D vectors A and B, apply Ax+B to incoming x." 117 | def __init__(self, A, B): 118 | if A.shape != B.shape: 119 | raise Exception("QNNScaleShiftLayer A and B shapes do not match") 120 | if A.ndim != 1: 121 | raise Exception("QNNScaleShiftLayer needs 1D vectors as parameters.") 122 | self.A = A 123 | self.B = B 124 | 125 | def execute(self, v): 126 | # the outermost dimension is the channel dimension 127 | # reshape as inner dimension to apply transform 128 | vr = v.reshape((self.A.shape[0], -1)).transpose() 129 | return (self.A*vr+self.B).transpose().flatten() 130 | 131 | class QNNPaddingLayer(QNNLayer): 132 | "A layer that adds padding around the edges of the image." 133 | def __init__(self, inDim, inChans, padCount, padVal): 134 | self.dim = inDim # input image dimension 135 | self.chans = inChans # number of input channels 136 | self.padCount = padCount # number of pixels to add on each edge 137 | self.padVal = padVal # value of pixels to be added to each edge 138 | 139 | def execute(self, v): 140 | img = v.reshape((self.chans, self.dim, self.dim)) 141 | padCounts = ((0, 0), 142 | (self.padCount, self.padCount), 143 | (self.padCount, self.padCount)) 144 | img = np.pad(img, padCounts, "constant", constant_values=self.padVal) 145 | return img.flatten() 146 | 147 | class QNNSlidingWindowLayer(QNNLayer): 148 | "Slide a window over a multichannel image (im2col)" 149 | def __init__(self, inDim, inChans, windowDim, stride=1): 150 | self.idim = inDim # input image dimension 151 | self.chans = inChans # channels in input image 152 | self.k = windowDim # window size 153 | self.s = stride # stride for next window 154 | 155 | def execute(self, v): 156 | # reshape the input vector into a 2D image 157 | img = v.reshape((1, self.chans, self.idim, self.idim)) 158 | # call im2col to get the sliding window result 159 | res = im2col_indices(img, self.k, self.k, padding=0, 160 | stride_y=self.s, stride_x=self.s) 161 | return res.flatten() 162 | 163 | class QNNConvolutionLayer(QNNLayer): 164 | "Convolution via im2col and matrix-matrix multiplication" 165 | def __init__(self, W, inDim, pad, stride, padVal=0): 166 | self.ofm = W.shape[0] # number of output channels 167 | self.ifm = W.shape[1] # number of input channels 168 | self.k = W.shape[2] # kernel (window) dimension 169 | self.idim = inDim # input image dimension 170 | self.padded_idim = inDim + 2*pad # input image dimension including padding 171 | self.odim = ((self.padded_idim - self.k) / stride) + 1 # output image dimension 172 | self.stride = stride # stride for sliding window 173 | self.pad = pad # number of padding pixels to add on each edge 174 | self.padVal = padVal # value of padding pixels 175 | if(W.shape[2] != W.shape[3]): 176 | raise Exception("Only square conv filters supported for now") 177 | # instantiate internal layer components 178 | self.layers = [] 179 | # add a padding layer, if padding is required 180 | if pad != 0: 181 | self.layers += [QNNPaddingLayer(self.idim, self.ifm, pad, padVal)] 182 | # add the sliding window layer 183 | self.layers += [QNNSlidingWindowLayer(self.padded_idim, self.ifm, self.k, self.stride)] 184 | # convert the kernel tensor to a matrix: 185 | # [ofm][ifm][k][k] to [ofm][ifm*k*k] 186 | self.W = W.reshape((self.ofm, self.ifm*self.k*self.k)) 187 | 188 | def execute(self, v): 189 | # execute internal padding/sliding window layers first 190 | vn = v 191 | for l in self.layers: 192 | vn = l.execute(vn) 193 | # reconstruct image matrix 194 | vn = vn.reshape((self.ifm*self.k*self.k, self.odim*self.odim)) 195 | # matrix-matrix multiply 196 | res = np.dot(self.W, vn) 197 | return res.flatten() 198 | 199 | class QNNPoolingLayer(QNNLayer): 200 | "Perform either max or average pooling." 201 | def __init__(self, inDim, inChans, poolSize, strideSize, poolFxn = "MAX"): 202 | self.idim = inDim 203 | self.chans = inChans 204 | self.k = poolSize 205 | self.s = strideSize 206 | self.odim = ((self.idim - self.k) / self.s) + 1 207 | self.poolFxn = poolFxn 208 | 209 | def execute(self, v): 210 | img = v.reshape((self.chans, self.idim, self.idim)) 211 | out_img = np.zeros((self.chans, self.odim*self.odim), dtype=np.float32) 212 | for c in range(self.chans): 213 | chan_img = img[c].reshape((1, 1, self.idim, self.idim)) 214 | # extract parts of image with sliding window 215 | wnd = im2col_indices(chan_img, self.k, self.k, padding=0, 216 | stride_y=self.s, stride_x=self.s) 217 | # each window is a column -- get the reduction along columns 218 | if self.poolFxn == "MAX": 219 | out_img[c]=wnd.max(axis = 0).flatten() 220 | elif self.poolFxn == "AVE": 221 | out_img[c]=wnd.mean(axis = 0).flatten() 222 | else: 223 | raise Exception("Unsupported pooling function") 224 | return out_img.flatten() 225 | 226 | 227 | class QNNSoftmaxLayer(QNNLayer): 228 | "Compute softmax values for each sets of scores." 229 | def execute(selv, v): 230 | e_x = np.exp(v - np.max(v)) 231 | return e_x / e_x.sum() 232 | 233 | class QNNReLULayer(QNNLayer): 234 | "Apply elementwise ReLU to the vector." 235 | def execute(self, v): 236 | return np.asarray(map(lambda x: x if x>0 else 0, v)) 237 | 238 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # qnn-inference-examples 2 | Jupyter notebook examples on image classification with quantized neural networks. The intent here is to give a better understanding of what kind of computations takes place when performing inference with a quantized neural network. 3 | 4 | So far, the following notebooks are available: 5 | 6 | 1. [Basics](0-basics.ipynb) for a gentle warmup 7 | 2. [Binarized, fully-connected MNIST](1-fully-connected-binarized-mnist.ipynb) for a deep dive inside a binarized fully-connected network 8 | 3. [Binary weights, 2-bit activations, fully-connected MNIST](2-fully-connected-w1a2-mnist.ipynb) for demonstrating what happens when we go to 2-bit activations 9 | 4. [Binarized, convolutional GTSRB](3-convolutional-binarized-gtsrb.ipynb) for an introduction to convolutional and pooling layers 10 | 5. [Mixed 1-bit weights/2-bit activations and 8-bit weights/activations ImageNet](4-convolutional-mixed-imagenet.ipynb) for a quantized AlexNet on the ImageNet dataset. 11 | -------------------------------------------------------------------------------- /conv-lowering-chetlur.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/conv-lowering-chetlur.png -------------------------------------------------------------------------------- /gtsrb_images/50.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/gtsrb_images/50.jpg -------------------------------------------------------------------------------- /gtsrb_images/left.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/gtsrb_images/left.jpg -------------------------------------------------------------------------------- /gtsrb_images/right.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/gtsrb_images/right.jpg -------------------------------------------------------------------------------- /gtsrb_images/stop.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/gtsrb_images/stop.jpg -------------------------------------------------------------------------------- /imagenet_data/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/imagenet_data/__init__.py -------------------------------------------------------------------------------- /imagenet_data/cat.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/imagenet_data/cat.jpg -------------------------------------------------------------------------------- /imagenet_data/classes.py: -------------------------------------------------------------------------------- 1 | imagenet_classes = {0: 'tench, Tinca tinca', 2 | 1: 'goldfish, Carassius auratus', 3 | 2: 'great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias', 4 | 3: 'tiger shark, Galeocerdo cuvieri', 5 | 4: 'hammerhead, hammerhead shark', 6 | 5: 'electric ray, crampfish, numbfish, torpedo', 7 | 6: 'stingray', 8 | 7: 'cock', 9 | 8: 'hen', 10 | 9: 'ostrich, Struthio camelus', 11 | 10: 'brambling, Fringilla montifringilla', 12 | 11: 'goldfinch, Carduelis carduelis', 13 | 12: 'house finch, linnet, Carpodacus mexicanus', 14 | 13: 'junco, snowbird', 15 | 14: 'indigo bunting, indigo finch, indigo bird, Passerina cyanea', 16 | 15: 'robin, American robin, Turdus migratorius', 17 | 16: 'bulbul', 18 | 17: 'jay', 19 | 18: 'magpie', 20 | 19: 'chickadee', 21 | 20: 'water ouzel, dipper', 22 | 21: 'kite', 23 | 22: 'bald eagle, American eagle, Haliaeetus leucocephalus', 24 | 23: 'vulture', 25 | 24: 'great grey owl, great gray owl, Strix nebulosa', 26 | 25: 'European fire salamander, Salamandra salamandra', 27 | 26: 'common newt, Triturus vulgaris', 28 | 27: 'eft', 29 | 28: 'spotted salamander, Ambystoma maculatum', 30 | 29: 'axolotl, mud puppy, Ambystoma mexicanum', 31 | 30: 'bullfrog, Rana catesbeiana', 32 | 31: 'tree frog, tree-frog', 33 | 32: 'tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui', 34 | 33: 'loggerhead, loggerhead turtle, Caretta caretta', 35 | 34: 'leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea', 36 | 35: 'mud turtle', 37 | 36: 'terrapin', 38 | 37: 'box turtle, box tortoise', 39 | 38: 'banded gecko', 40 | 39: 'common iguana, iguana, Iguana iguana', 41 | 40: 'American chameleon, anole, Anolis carolinensis', 42 | 41: 'whiptail, whiptail lizard', 43 | 42: 'agama', 44 | 43: 'frilled lizard, Chlamydosaurus kingi', 45 | 44: 'alligator lizard', 46 | 45: 'Gila monster, Heloderma suspectum', 47 | 46: 'green lizard, Lacerta viridis', 48 | 47: 'African chameleon, Chamaeleo chamaeleon', 49 | 48: 'Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis', 50 | 49: 'African crocodile, Nile crocodile, Crocodylus niloticus', 51 | 50: 'American alligator, Alligator mississipiensis', 52 | 51: 'triceratops', 53 | 52: 'thunder snake, worm snake, Carphophis amoenus', 54 | 53: 'ringneck snake, ring-necked snake, ring snake', 55 | 54: 'hognose snake, puff adder, sand viper', 56 | 55: 'green snake, grass snake', 57 | 56: 'king snake, kingsnake', 58 | 57: 'garter snake, grass snake', 59 | 58: 'water snake', 60 | 59: 'vine snake', 61 | 60: 'night snake, Hypsiglena torquata', 62 | 61: 'boa constrictor, Constrictor constrictor', 63 | 62: 'rock python, rock snake, Python sebae', 64 | 63: 'Indian cobra, Naja naja', 65 | 64: 'green mamba', 66 | 65: 'sea snake', 67 | 66: 'horned viper, cerastes, sand viper, horned asp, Cerastes cornutus', 68 | 67: 'diamondback, diamondback rattlesnake, Crotalus adamanteus', 69 | 68: 'sidewinder, horned rattlesnake, Crotalus cerastes', 70 | 69: 'trilobite', 71 | 70: 'harvestman, daddy longlegs, Phalangium opilio', 72 | 71: 'scorpion', 73 | 72: 'black and gold garden spider, Argiope aurantia', 74 | 73: 'barn spider, Araneus cavaticus', 75 | 74: 'garden spider, Aranea diademata', 76 | 75: 'black widow, Latrodectus mactans', 77 | 76: 'tarantula', 78 | 77: 'wolf spider, hunting spider', 79 | 78: 'tick', 80 | 79: 'centipede', 81 | 80: 'black grouse', 82 | 81: 'ptarmigan', 83 | 82: 'ruffed grouse, partridge, Bonasa umbellus', 84 | 83: 'prairie chicken, prairie grouse, prairie fowl', 85 | 84: 'peacock', 86 | 85: 'quail', 87 | 86: 'partridge', 88 | 87: 'African grey, African gray, Psittacus erithacus', 89 | 88: 'macaw', 90 | 89: 'sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita', 91 | 90: 'lorikeet', 92 | 91: 'coucal', 93 | 92: 'bee eater', 94 | 93: 'hornbill', 95 | 94: 'hummingbird', 96 | 95: 'jacamar', 97 | 96: 'toucan', 98 | 97: 'drake', 99 | 98: 'red-breasted merganser, Mergus serrator', 100 | 99: 'goose', 101 | 100: 'black swan, Cygnus atratus', 102 | 101: 'tusker', 103 | 102: 'echidna, spiny anteater, anteater', 104 | 103: 'platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus', 105 | 104: 'wallaby, brush kangaroo', 106 | 105: 'koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus', 107 | 106: 'wombat', 108 | 107: 'jellyfish', 109 | 108: 'sea anemone, anemone', 110 | 109: 'brain coral', 111 | 110: 'flatworm, platyhelminth', 112 | 111: 'nematode, nematode worm, roundworm', 113 | 112: 'conch', 114 | 113: 'snail', 115 | 114: 'slug', 116 | 115: 'sea slug, nudibranch', 117 | 116: 'chiton, coat-of-mail shell, sea cradle, polyplacophore', 118 | 117: 'chambered nautilus, pearly nautilus, nautilus', 119 | 118: 'Dungeness crab, Cancer magister', 120 | 119: 'rock crab, Cancer irroratus', 121 | 120: 'fiddler crab', 122 | 121: 'king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica', 123 | 122: 'American lobster, Northern lobster, Maine lobster, Homarus americanus', 124 | 123: 'spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish', 125 | 124: 'crayfish, crawfish, crawdad, crawdaddy', 126 | 125: 'hermit crab', 127 | 126: 'isopod', 128 | 127: 'white stork, Ciconia ciconia', 129 | 128: 'black stork, Ciconia nigra', 130 | 129: 'spoonbill', 131 | 130: 'flamingo', 132 | 131: 'little blue heron, Egretta caerulea', 133 | 132: 'American egret, great white heron, Egretta albus', 134 | 133: 'bittern', 135 | 134: 'crane', 136 | 135: 'limpkin, Aramus pictus', 137 | 136: 'European gallinule, Porphyrio porphyrio', 138 | 137: 'American coot, marsh hen, mud hen, water hen, Fulica americana', 139 | 138: 'bustard', 140 | 139: 'ruddy turnstone, Arenaria interpres', 141 | 140: 'red-backed sandpiper, dunlin, Erolia alpina', 142 | 141: 'redshank, Tringa totanus', 143 | 142: 'dowitcher', 144 | 143: 'oystercatcher, oyster catcher', 145 | 144: 'pelican', 146 | 145: 'king penguin, Aptenodytes patagonica', 147 | 146: 'albatross, mollymawk', 148 | 147: 'grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus', 149 | 148: 'killer whale, killer, orca, grampus, sea wolf, Orcinus orca', 150 | 149: 'dugong, Dugong dugon', 151 | 150: 'sea lion', 152 | 151: 'Chihuahua', 153 | 152: 'Japanese spaniel', 154 | 153: 'Maltese dog, Maltese terrier, Maltese', 155 | 154: 'Pekinese, Pekingese, Peke', 156 | 155: 'Shih-Tzu', 157 | 156: 'Blenheim spaniel', 158 | 157: 'papillon', 159 | 158: 'toy terrier', 160 | 159: 'Rhodesian ridgeback', 161 | 160: 'Afghan hound, Afghan', 162 | 161: 'basset, basset hound', 163 | 162: 'beagle', 164 | 163: 'bloodhound, sleuthhound', 165 | 164: 'bluetick', 166 | 165: 'black-and-tan coonhound', 167 | 166: 'Walker hound, Walker foxhound', 168 | 167: 'English foxhound', 169 | 168: 'redbone', 170 | 169: 'borzoi, Russian wolfhound', 171 | 170: 'Irish wolfhound', 172 | 171: 'Italian greyhound', 173 | 172: 'whippet', 174 | 173: 'Ibizan hound, Ibizan Podenco', 175 | 174: 'Norwegian elkhound, elkhound', 176 | 175: 'otterhound, otter hound', 177 | 176: 'Saluki, gazelle hound', 178 | 177: 'Scottish deerhound, deerhound', 179 | 178: 'Weimaraner', 180 | 179: 'Staffordshire bullterrier, Staffordshire bull terrier', 181 | 180: 'American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier', 182 | 181: 'Bedlington terrier', 183 | 182: 'Border terrier', 184 | 183: 'Kerry blue terrier', 185 | 184: 'Irish terrier', 186 | 185: 'Norfolk terrier', 187 | 186: 'Norwich terrier', 188 | 187: 'Yorkshire terrier', 189 | 188: 'wire-haired fox terrier', 190 | 189: 'Lakeland terrier', 191 | 190: 'Sealyham terrier, Sealyham', 192 | 191: 'Airedale, Airedale terrier', 193 | 192: 'cairn, cairn terrier', 194 | 193: 'Australian terrier', 195 | 194: 'Dandie Dinmont, Dandie Dinmont terrier', 196 | 195: 'Boston bull, Boston terrier', 197 | 196: 'miniature schnauzer', 198 | 197: 'giant schnauzer', 199 | 198: 'standard schnauzer', 200 | 199: 'Scotch terrier, Scottish terrier, Scottie', 201 | 200: 'Tibetan terrier, chrysanthemum dog', 202 | 201: 'silky terrier, Sydney silky', 203 | 202: 'soft-coated wheaten terrier', 204 | 203: 'West Highland white terrier', 205 | 204: 'Lhasa, Lhasa apso', 206 | 205: 'flat-coated retriever', 207 | 206: 'curly-coated retriever', 208 | 207: 'golden retriever', 209 | 208: 'Labrador retriever', 210 | 209: 'Chesapeake Bay retriever', 211 | 210: 'German short-haired pointer', 212 | 211: 'vizsla, Hungarian pointer', 213 | 212: 'English setter', 214 | 213: 'Irish setter, red setter', 215 | 214: 'Gordon setter', 216 | 215: 'Brittany spaniel', 217 | 216: 'clumber, clumber spaniel', 218 | 217: 'English springer, English springer spaniel', 219 | 218: 'Welsh springer spaniel', 220 | 219: 'cocker spaniel, English cocker spaniel, cocker', 221 | 220: 'Sussex spaniel', 222 | 221: 'Irish water spaniel', 223 | 222: 'kuvasz', 224 | 223: 'schipperke', 225 | 224: 'groenendael', 226 | 225: 'malinois', 227 | 226: 'briard', 228 | 227: 'kelpie', 229 | 228: 'komondor', 230 | 229: 'Old English sheepdog, bobtail', 231 | 230: 'Shetland sheepdog, Shetland sheep dog, Shetland', 232 | 231: 'collie', 233 | 232: 'Border collie', 234 | 233: 'Bouvier des Flandres, Bouviers des Flandres', 235 | 234: 'Rottweiler', 236 | 235: 'German shepherd, German shepherd dog, German police dog, alsatian', 237 | 236: 'Doberman, Doberman pinscher', 238 | 237: 'miniature pinscher', 239 | 238: 'Greater Swiss Mountain dog', 240 | 239: 'Bernese mountain dog', 241 | 240: 'Appenzeller', 242 | 241: 'EntleBucher', 243 | 242: 'boxer', 244 | 243: 'bull mastiff', 245 | 244: 'Tibetan mastiff', 246 | 245: 'French bulldog', 247 | 246: 'Great Dane', 248 | 247: 'Saint Bernard, St Bernard', 249 | 248: 'Eskimo dog, husky', 250 | 249: 'malamute, malemute, Alaskan malamute', 251 | 250: 'Siberian husky', 252 | 251: 'dalmatian, coach dog, carriage dog', 253 | 252: 'affenpinscher, monkey pinscher, monkey dog', 254 | 253: 'basenji', 255 | 254: 'pug, pug-dog', 256 | 255: 'Leonberg', 257 | 256: 'Newfoundland, Newfoundland dog', 258 | 257: 'Great Pyrenees', 259 | 258: 'Samoyed, Samoyede', 260 | 259: 'Pomeranian', 261 | 260: 'chow, chow chow', 262 | 261: 'keeshond', 263 | 262: 'Brabancon griffon', 264 | 263: 'Pembroke, Pembroke Welsh corgi', 265 | 264: 'Cardigan, Cardigan Welsh corgi', 266 | 265: 'toy poodle', 267 | 266: 'miniature poodle', 268 | 267: 'standard poodle', 269 | 268: 'Mexican hairless', 270 | 269: 'timber wolf, grey wolf, gray wolf, Canis lupus', 271 | 270: 'white wolf, Arctic wolf, Canis lupus tundrarum', 272 | 271: 'red wolf, maned wolf, Canis rufus, Canis niger', 273 | 272: 'coyote, prairie wolf, brush wolf, Canis latrans', 274 | 273: 'dingo, warrigal, warragal, Canis dingo', 275 | 274: 'dhole, Cuon alpinus', 276 | 275: 'African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus', 277 | 276: 'hyena, hyaena', 278 | 277: 'red fox, Vulpes vulpes', 279 | 278: 'kit fox, Vulpes macrotis', 280 | 279: 'Arctic fox, white fox, Alopex lagopus', 281 | 280: 'grey fox, gray fox, Urocyon cinereoargenteus', 282 | 281: 'tabby, tabby cat', 283 | 282: 'tiger cat', 284 | 283: 'Persian cat', 285 | 284: 'Siamese cat, Siamese', 286 | 285: 'Egyptian cat', 287 | 286: 'cougar, puma, catamount, mountain lion, painter, panther, Felis concolor', 288 | 287: 'lynx, catamount', 289 | 288: 'leopard, Panthera pardus', 290 | 289: 'snow leopard, ounce, Panthera uncia', 291 | 290: 'jaguar, panther, Panthera onca, Felis onca', 292 | 291: 'lion, king of beasts, Panthera leo', 293 | 292: 'tiger, Panthera tigris', 294 | 293: 'cheetah, chetah, Acinonyx jubatus', 295 | 294: 'brown bear, bruin, Ursus arctos', 296 | 295: 'American black bear, black bear, Ursus americanus, Euarctos americanus', 297 | 296: 'ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus', 298 | 297: 'sloth bear, Melursus ursinus, Ursus ursinus', 299 | 298: 'mongoose', 300 | 299: 'meerkat, mierkat', 301 | 300: 'tiger beetle', 302 | 301: 'ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle', 303 | 302: 'ground beetle, carabid beetle', 304 | 303: 'long-horned beetle, longicorn, longicorn beetle', 305 | 304: 'leaf beetle, chrysomelid', 306 | 305: 'dung beetle', 307 | 306: 'rhinoceros beetle', 308 | 307: 'weevil', 309 | 308: 'fly', 310 | 309: 'bee', 311 | 310: 'ant, emmet, pismire', 312 | 311: 'grasshopper, hopper', 313 | 312: 'cricket', 314 | 313: 'walking stick, walkingstick, stick insect', 315 | 314: 'cockroach, roach', 316 | 315: 'mantis, mantid', 317 | 316: 'cicada, cicala', 318 | 317: 'leafhopper', 319 | 318: 'lacewing, lacewing fly', 320 | 319: "dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk", 321 | 320: 'damselfly', 322 | 321: 'admiral', 323 | 322: 'ringlet, ringlet butterfly', 324 | 323: 'monarch, monarch butterfly, milkweed butterfly, Danaus plexippus', 325 | 324: 'cabbage butterfly', 326 | 325: 'sulphur butterfly, sulfur butterfly', 327 | 326: 'lycaenid, lycaenid butterfly', 328 | 327: 'starfish, sea star', 329 | 328: 'sea urchin', 330 | 329: 'sea cucumber, holothurian', 331 | 330: 'wood rabbit, cottontail, cottontail rabbit', 332 | 331: 'hare', 333 | 332: 'Angora, Angora rabbit', 334 | 333: 'hamster', 335 | 334: 'porcupine, hedgehog', 336 | 335: 'fox squirrel, eastern fox squirrel, Sciurus niger', 337 | 336: 'marmot', 338 | 337: 'beaver', 339 | 338: 'guinea pig, Cavia cobaya', 340 | 339: 'sorrel', 341 | 340: 'zebra', 342 | 341: 'hog, pig, grunter, squealer, Sus scrofa', 343 | 342: 'wild boar, boar, Sus scrofa', 344 | 343: 'warthog', 345 | 344: 'hippopotamus, hippo, river horse, Hippopotamus amphibius', 346 | 345: 'ox', 347 | 346: 'water buffalo, water ox, Asiatic buffalo, Bubalus bubalis', 348 | 347: 'bison', 349 | 348: 'ram, tup', 350 | 349: 'bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis', 351 | 350: 'ibex, Capra ibex', 352 | 351: 'hartebeest', 353 | 352: 'impala, Aepyceros melampus', 354 | 353: 'gazelle', 355 | 354: 'Arabian camel, dromedary, Camelus dromedarius', 356 | 355: 'llama', 357 | 356: 'weasel', 358 | 357: 'mink', 359 | 358: 'polecat, fitch, foulmart, foumart, Mustela putorius', 360 | 359: 'black-footed ferret, ferret, Mustela nigripes', 361 | 360: 'otter', 362 | 361: 'skunk, polecat, wood pussy', 363 | 362: 'badger', 364 | 363: 'armadillo', 365 | 364: 'three-toed sloth, ai, Bradypus tridactylus', 366 | 365: 'orangutan, orang, orangutang, Pongo pygmaeus', 367 | 366: 'gorilla, Gorilla gorilla', 368 | 367: 'chimpanzee, chimp, Pan troglodytes', 369 | 368: 'gibbon, Hylobates lar', 370 | 369: 'siamang, Hylobates syndactylus, Symphalangus syndactylus', 371 | 370: 'guenon, guenon monkey', 372 | 371: 'patas, hussar monkey, Erythrocebus patas', 373 | 372: 'baboon', 374 | 373: 'macaque', 375 | 374: 'langur', 376 | 375: 'colobus, colobus monkey', 377 | 376: 'proboscis monkey, Nasalis larvatus', 378 | 377: 'marmoset', 379 | 378: 'capuchin, ringtail, Cebus capucinus', 380 | 379: 'howler monkey, howler', 381 | 380: 'titi, titi monkey', 382 | 381: 'spider monkey, Ateles geoffroyi', 383 | 382: 'squirrel monkey, Saimiri sciureus', 384 | 383: 'Madagascar cat, ring-tailed lemur, Lemur catta', 385 | 384: 'indri, indris, Indri indri, Indri brevicaudatus', 386 | 385: 'Indian elephant, Elephas maximus', 387 | 386: 'African elephant, Loxodonta africana', 388 | 387: 'lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens', 389 | 388: 'giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca', 390 | 389: 'barracouta, snoek', 391 | 390: 'eel', 392 | 391: 'coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch', 393 | 392: 'rock beauty, Holocanthus tricolor', 394 | 393: 'anemone fish', 395 | 394: 'sturgeon', 396 | 395: 'gar, garfish, garpike, billfish, Lepisosteus osseus', 397 | 396: 'lionfish', 398 | 397: 'puffer, pufferfish, blowfish, globefish', 399 | 398: 'abacus', 400 | 399: 'abaya', 401 | 400: "academic gown, academic robe, judge's robe", 402 | 401: 'accordion, piano accordion, squeeze box', 403 | 402: 'acoustic guitar', 404 | 403: 'aircraft carrier, carrier, flattop, attack aircraft carrier', 405 | 404: 'airliner', 406 | 405: 'airship, dirigible', 407 | 406: 'altar', 408 | 407: 'ambulance', 409 | 408: 'amphibian, amphibious vehicle', 410 | 409: 'analog clock', 411 | 410: 'apiary, bee house', 412 | 411: 'apron', 413 | 412: 'ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin', 414 | 413: 'assault rifle, assault gun', 415 | 414: 'backpack, back pack, knapsack, packsack, rucksack, haversack', 416 | 415: 'bakery, bakeshop, bakehouse', 417 | 416: 'balance beam, beam', 418 | 417: 'balloon', 419 | 418: 'ballpoint, ballpoint pen, ballpen, Biro', 420 | 419: 'Band Aid', 421 | 420: 'banjo', 422 | 421: 'bannister, banister, balustrade, balusters, handrail', 423 | 422: 'barbell', 424 | 423: 'barber chair', 425 | 424: 'barbershop', 426 | 425: 'barn', 427 | 426: 'barometer', 428 | 427: 'barrel, cask', 429 | 428: 'barrow, garden cart, lawn cart, wheelbarrow', 430 | 429: 'baseball', 431 | 430: 'basketball', 432 | 431: 'bassinet', 433 | 432: 'bassoon', 434 | 433: 'bathing cap, swimming cap', 435 | 434: 'bath towel', 436 | 435: 'bathtub, bathing tub, bath, tub', 437 | 436: 'beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon', 438 | 437: 'beacon, lighthouse, beacon light, pharos', 439 | 438: 'beaker', 440 | 439: 'bearskin, busby, shako', 441 | 440: 'beer bottle', 442 | 441: 'beer glass', 443 | 442: 'bell cote, bell cot', 444 | 443: 'bib', 445 | 444: 'bicycle-built-for-two, tandem bicycle, tandem', 446 | 445: 'bikini, two-piece', 447 | 446: 'binder, ring-binder', 448 | 447: 'binoculars, field glasses, opera glasses', 449 | 448: 'birdhouse', 450 | 449: 'boathouse', 451 | 450: 'bobsled, bobsleigh, bob', 452 | 451: 'bolo tie, bolo, bola tie, bola', 453 | 452: 'bonnet, poke bonnet', 454 | 453: 'bookcase', 455 | 454: 'bookshop, bookstore, bookstall', 456 | 455: 'bottlecap', 457 | 456: 'bow', 458 | 457: 'bow tie, bow-tie, bowtie', 459 | 458: 'brass, memorial tablet, plaque', 460 | 459: 'brassiere, bra, bandeau', 461 | 460: 'breakwater, groin, groyne, mole, bulwark, seawall, jetty', 462 | 461: 'breastplate, aegis, egis', 463 | 462: 'broom', 464 | 463: 'bucket, pail', 465 | 464: 'buckle', 466 | 465: 'bulletproof vest', 467 | 466: 'bullet train, bullet', 468 | 467: 'butcher shop, meat market', 469 | 468: 'cab, hack, taxi, taxicab', 470 | 469: 'caldron, cauldron', 471 | 470: 'candle, taper, wax light', 472 | 471: 'cannon', 473 | 472: 'canoe', 474 | 473: 'can opener, tin opener', 475 | 474: 'cardigan', 476 | 475: 'car mirror', 477 | 476: 'carousel, carrousel, merry-go-round, roundabout, whirligig', 478 | 477: "carpenter's kit, tool kit", 479 | 478: 'carton', 480 | 479: 'car wheel', 481 | 480: 'cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM', 482 | 481: 'cassette', 483 | 482: 'cassette player', 484 | 483: 'castle', 485 | 484: 'catamaran', 486 | 485: 'CD player', 487 | 486: 'cello, violoncello', 488 | 487: 'cellular telephone, cellular phone, cellphone, cell, mobile phone', 489 | 488: 'chain', 490 | 489: 'chainlink fence', 491 | 490: 'chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour', 492 | 491: 'chain saw, chainsaw', 493 | 492: 'chest', 494 | 493: 'chiffonier, commode', 495 | 494: 'chime, bell, gong', 496 | 495: 'china cabinet, china closet', 497 | 496: 'Christmas stocking', 498 | 497: 'church, church building', 499 | 498: 'cinema, movie theater, movie theatre, movie house, picture palace', 500 | 499: 'cleaver, meat cleaver, chopper', 501 | 500: 'cliff dwelling', 502 | 501: 'cloak', 503 | 502: 'clog, geta, patten, sabot', 504 | 503: 'cocktail shaker', 505 | 504: 'coffee mug', 506 | 505: 'coffeepot', 507 | 506: 'coil, spiral, volute, whorl, helix', 508 | 507: 'combination lock', 509 | 508: 'computer keyboard, keypad', 510 | 509: 'confectionery, confectionary, candy store', 511 | 510: 'container ship, containership, container vessel', 512 | 511: 'convertible', 513 | 512: 'corkscrew, bottle screw', 514 | 513: 'cornet, horn, trumpet, trump', 515 | 514: 'cowboy boot', 516 | 515: 'cowboy hat, ten-gallon hat', 517 | 516: 'cradle', 518 | 517: 'crane', 519 | 518: 'crash helmet', 520 | 519: 'crate', 521 | 520: 'crib, cot', 522 | 521: 'Crock Pot', 523 | 522: 'croquet ball', 524 | 523: 'crutch', 525 | 524: 'cuirass', 526 | 525: 'dam, dike, dyke', 527 | 526: 'desk', 528 | 527: 'desktop computer', 529 | 528: 'dial telephone, dial phone', 530 | 529: 'diaper, nappy, napkin', 531 | 530: 'digital clock', 532 | 531: 'digital watch', 533 | 532: 'dining table, board', 534 | 533: 'dishrag, dishcloth', 535 | 534: 'dishwasher, dish washer, dishwashing machine', 536 | 535: 'disk brake, disc brake', 537 | 536: 'dock, dockage, docking facility', 538 | 537: 'dogsled, dog sled, dog sleigh', 539 | 538: 'dome', 540 | 539: 'doormat, welcome mat', 541 | 540: 'drilling platform, offshore rig', 542 | 541: 'drum, membranophone, tympan', 543 | 542: 'drumstick', 544 | 543: 'dumbbell', 545 | 544: 'Dutch oven', 546 | 545: 'electric fan, blower', 547 | 546: 'electric guitar', 548 | 547: 'electric locomotive', 549 | 548: 'entertainment center', 550 | 549: 'envelope', 551 | 550: 'espresso maker', 552 | 551: 'face powder', 553 | 552: 'feather boa, boa', 554 | 553: 'file, file cabinet, filing cabinet', 555 | 554: 'fireboat', 556 | 555: 'fire engine, fire truck', 557 | 556: 'fire screen, fireguard', 558 | 557: 'flagpole, flagstaff', 559 | 558: 'flute, transverse flute', 560 | 559: 'folding chair', 561 | 560: 'football helmet', 562 | 561: 'forklift', 563 | 562: 'fountain', 564 | 563: 'fountain pen', 565 | 564: 'four-poster', 566 | 565: 'freight car', 567 | 566: 'French horn, horn', 568 | 567: 'frying pan, frypan, skillet', 569 | 568: 'fur coat', 570 | 569: 'garbage truck, dustcart', 571 | 570: 'gasmask, respirator, gas helmet', 572 | 571: 'gas pump, gasoline pump, petrol pump, island dispenser', 573 | 572: 'goblet', 574 | 573: 'go-kart', 575 | 574: 'golf ball', 576 | 575: 'golfcart, golf cart', 577 | 576: 'gondola', 578 | 577: 'gong, tam-tam', 579 | 578: 'gown', 580 | 579: 'grand piano, grand', 581 | 580: 'greenhouse, nursery, glasshouse', 582 | 581: 'grille, radiator grille', 583 | 582: 'grocery store, grocery, food market, market', 584 | 583: 'guillotine', 585 | 584: 'hair slide', 586 | 585: 'hair spray', 587 | 586: 'half track', 588 | 587: 'hammer', 589 | 588: 'hamper', 590 | 589: 'hand blower, blow dryer, blow drier, hair dryer, hair drier', 591 | 590: 'hand-held computer, hand-held microcomputer', 592 | 591: 'handkerchief, hankie, hanky, hankey', 593 | 592: 'hard disc, hard disk, fixed disk', 594 | 593: 'harmonica, mouth organ, harp, mouth harp', 595 | 594: 'harp', 596 | 595: 'harvester, reaper', 597 | 596: 'hatchet', 598 | 597: 'holster', 599 | 598: 'home theater, home theatre', 600 | 599: 'honeycomb', 601 | 600: 'hook, claw', 602 | 601: 'hoopskirt, crinoline', 603 | 602: 'horizontal bar, high bar', 604 | 603: 'horse cart, horse-cart', 605 | 604: 'hourglass', 606 | 605: 'iPod', 607 | 606: 'iron, smoothing iron', 608 | 607: "jack-o'-lantern", 609 | 608: 'jean, blue jean, denim', 610 | 609: 'jeep, landrover', 611 | 610: 'jersey, T-shirt, tee shirt', 612 | 611: 'jigsaw puzzle', 613 | 612: 'jinrikisha, ricksha, rickshaw', 614 | 613: 'joystick', 615 | 614: 'kimono', 616 | 615: 'knee pad', 617 | 616: 'knot', 618 | 617: 'lab coat, laboratory coat', 619 | 618: 'ladle', 620 | 619: 'lampshade, lamp shade', 621 | 620: 'laptop, laptop computer', 622 | 621: 'lawn mower, mower', 623 | 622: 'lens cap, lens cover', 624 | 623: 'letter opener, paper knife, paperknife', 625 | 624: 'library', 626 | 625: 'lifeboat', 627 | 626: 'lighter, light, igniter, ignitor', 628 | 627: 'limousine, limo', 629 | 628: 'liner, ocean liner', 630 | 629: 'lipstick, lip rouge', 631 | 630: 'Loafer', 632 | 631: 'lotion', 633 | 632: 'loudspeaker, speaker, speaker unit, loudspeaker system, speaker system', 634 | 633: "loupe, jeweler's loupe", 635 | 634: 'lumbermill, sawmill', 636 | 635: 'magnetic compass', 637 | 636: 'mailbag, postbag', 638 | 637: 'mailbox, letter box', 639 | 638: 'maillot', 640 | 639: 'maillot, tank suit', 641 | 640: 'manhole cover', 642 | 641: 'maraca', 643 | 642: 'marimba, xylophone', 644 | 643: 'mask', 645 | 644: 'matchstick', 646 | 645: 'maypole', 647 | 646: 'maze, labyrinth', 648 | 647: 'measuring cup', 649 | 648: 'medicine chest, medicine cabinet', 650 | 649: 'megalith, megalithic structure', 651 | 650: 'microphone, mike', 652 | 651: 'microwave, microwave oven', 653 | 652: 'military uniform', 654 | 653: 'milk can', 655 | 654: 'minibus', 656 | 655: 'miniskirt, mini', 657 | 656: 'minivan', 658 | 657: 'missile', 659 | 658: 'mitten', 660 | 659: 'mixing bowl', 661 | 660: 'mobile home, manufactured home', 662 | 661: 'Model T', 663 | 662: 'modem', 664 | 663: 'monastery', 665 | 664: 'monitor', 666 | 665: 'moped', 667 | 666: 'mortar', 668 | 667: 'mortarboard', 669 | 668: 'mosque', 670 | 669: 'mosquito net', 671 | 670: 'motor scooter, scooter', 672 | 671: 'mountain bike, all-terrain bike, off-roader', 673 | 672: 'mountain tent', 674 | 673: 'mouse, computer mouse', 675 | 674: 'mousetrap', 676 | 675: 'moving van', 677 | 676: 'muzzle', 678 | 677: 'nail', 679 | 678: 'neck brace', 680 | 679: 'necklace', 681 | 680: 'nipple', 682 | 681: 'notebook, notebook computer', 683 | 682: 'obelisk', 684 | 683: 'oboe, hautboy, hautbois', 685 | 684: 'ocarina, sweet potato', 686 | 685: 'odometer, hodometer, mileometer, milometer', 687 | 686: 'oil filter', 688 | 687: 'organ, pipe organ', 689 | 688: 'oscilloscope, scope, cathode-ray oscilloscope, CRO', 690 | 689: 'overskirt', 691 | 690: 'oxcart', 692 | 691: 'oxygen mask', 693 | 692: 'packet', 694 | 693: 'paddle, boat paddle', 695 | 694: 'paddlewheel, paddle wheel', 696 | 695: 'padlock', 697 | 696: 'paintbrush', 698 | 697: "pajama, pyjama, pj's, jammies", 699 | 698: 'palace', 700 | 699: 'panpipe, pandean pipe, syrinx', 701 | 700: 'paper towel', 702 | 701: 'parachute, chute', 703 | 702: 'parallel bars, bars', 704 | 703: 'park bench', 705 | 704: 'parking meter', 706 | 705: 'passenger car, coach, carriage', 707 | 706: 'patio, terrace', 708 | 707: 'pay-phone, pay-station', 709 | 708: 'pedestal, plinth, footstall', 710 | 709: 'pencil box, pencil case', 711 | 710: 'pencil sharpener', 712 | 711: 'perfume, essence', 713 | 712: 'Petri dish', 714 | 713: 'photocopier', 715 | 714: 'pick, plectrum, plectron', 716 | 715: 'pickelhaube', 717 | 716: 'picket fence, paling', 718 | 717: 'pickup, pickup truck', 719 | 718: 'pier', 720 | 719: 'piggy bank, penny bank', 721 | 720: 'pill bottle', 722 | 721: 'pillow', 723 | 722: 'ping-pong ball', 724 | 723: 'pinwheel', 725 | 724: 'pirate, pirate ship', 726 | 725: 'pitcher, ewer', 727 | 726: "plane, carpenter's plane, woodworking plane", 728 | 727: 'planetarium', 729 | 728: 'plastic bag', 730 | 729: 'plate rack', 731 | 730: 'plow, plough', 732 | 731: "plunger, plumber's helper", 733 | 732: 'Polaroid camera, Polaroid Land camera', 734 | 733: 'pole', 735 | 734: 'police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria', 736 | 735: 'poncho', 737 | 736: 'pool table, billiard table, snooker table', 738 | 737: 'pop bottle, soda bottle', 739 | 738: 'pot, flowerpot', 740 | 739: "potter's wheel", 741 | 740: 'power drill', 742 | 741: 'prayer rug, prayer mat', 743 | 742: 'printer', 744 | 743: 'prison, prison house', 745 | 744: 'projectile, missile', 746 | 745: 'projector', 747 | 746: 'puck, hockey puck', 748 | 747: 'punching bag, punch bag, punching ball, punchball', 749 | 748: 'purse', 750 | 749: 'quill, quill pen', 751 | 750: 'quilt, comforter, comfort, puff', 752 | 751: 'racer, race car, racing car', 753 | 752: 'racket, racquet', 754 | 753: 'radiator', 755 | 754: 'radio, wireless', 756 | 755: 'radio telescope, radio reflector', 757 | 756: 'rain barrel', 758 | 757: 'recreational vehicle, RV, R.V.', 759 | 758: 'reel', 760 | 759: 'reflex camera', 761 | 760: 'refrigerator, icebox', 762 | 761: 'remote control, remote', 763 | 762: 'restaurant, eating house, eating place, eatery', 764 | 763: 'revolver, six-gun, six-shooter', 765 | 764: 'rifle', 766 | 765: 'rocking chair, rocker', 767 | 766: 'rotisserie', 768 | 767: 'rubber eraser, rubber, pencil eraser', 769 | 768: 'rugby ball', 770 | 769: 'rule, ruler', 771 | 770: 'running shoe', 772 | 771: 'safe', 773 | 772: 'safety pin', 774 | 773: 'saltshaker, salt shaker', 775 | 774: 'sandal', 776 | 775: 'sarong', 777 | 776: 'sax, saxophone', 778 | 777: 'scabbard', 779 | 778: 'scale, weighing machine', 780 | 779: 'school bus', 781 | 780: 'schooner', 782 | 781: 'scoreboard', 783 | 782: 'screen, CRT screen', 784 | 783: 'screw', 785 | 784: 'screwdriver', 786 | 785: 'seat belt, seatbelt', 787 | 786: 'sewing machine', 788 | 787: 'shield, buckler', 789 | 788: 'shoe shop, shoe-shop, shoe store', 790 | 789: 'shoji', 791 | 790: 'shopping basket', 792 | 791: 'shopping cart', 793 | 792: 'shovel', 794 | 793: 'shower cap', 795 | 794: 'shower curtain', 796 | 795: 'ski', 797 | 796: 'ski mask', 798 | 797: 'sleeping bag', 799 | 798: 'slide rule, slipstick', 800 | 799: 'sliding door', 801 | 800: 'slot, one-armed bandit', 802 | 801: 'snorkel', 803 | 802: 'snowmobile', 804 | 803: 'snowplow, snowplough', 805 | 804: 'soap dispenser', 806 | 805: 'soccer ball', 807 | 806: 'sock', 808 | 807: 'solar dish, solar collector, solar furnace', 809 | 808: 'sombrero', 810 | 809: 'soup bowl', 811 | 810: 'space bar', 812 | 811: 'space heater', 813 | 812: 'space shuttle', 814 | 813: 'spatula', 815 | 814: 'speedboat', 816 | 815: "spider web, spider's web", 817 | 816: 'spindle', 818 | 817: 'sports car, sport car', 819 | 818: 'spotlight, spot', 820 | 819: 'stage', 821 | 820: 'steam locomotive', 822 | 821: 'steel arch bridge', 823 | 822: 'steel drum', 824 | 823: 'stethoscope', 825 | 824: 'stole', 826 | 825: 'stone wall', 827 | 826: 'stopwatch, stop watch', 828 | 827: 'stove', 829 | 828: 'strainer', 830 | 829: 'streetcar, tram, tramcar, trolley, trolley car', 831 | 830: 'stretcher', 832 | 831: 'studio couch, day bed', 833 | 832: 'stupa, tope', 834 | 833: 'submarine, pigboat, sub, U-boat', 835 | 834: 'suit, suit of clothes', 836 | 835: 'sundial', 837 | 836: 'sunglass', 838 | 837: 'sunglasses, dark glasses, shades', 839 | 838: 'sunscreen, sunblock, sun blocker', 840 | 839: 'suspension bridge', 841 | 840: 'swab, swob, mop', 842 | 841: 'sweatshirt', 843 | 842: 'swimming trunks, bathing trunks', 844 | 843: 'swing', 845 | 844: 'switch, electric switch, electrical switch', 846 | 845: 'syringe', 847 | 846: 'table lamp', 848 | 847: 'tank, army tank, armored combat vehicle, armoured combat vehicle', 849 | 848: 'tape player', 850 | 849: 'teapot', 851 | 850: 'teddy, teddy bear', 852 | 851: 'television, television system', 853 | 852: 'tennis ball', 854 | 853: 'thatch, thatched roof', 855 | 854: 'theater curtain, theatre curtain', 856 | 855: 'thimble', 857 | 856: 'thresher, thrasher, threshing machine', 858 | 857: 'throne', 859 | 858: 'tile roof', 860 | 859: 'toaster', 861 | 860: 'tobacco shop, tobacconist shop, tobacconist', 862 | 861: 'toilet seat', 863 | 862: 'torch', 864 | 863: 'totem pole', 865 | 864: 'tow truck, tow car, wrecker', 866 | 865: 'toyshop', 867 | 866: 'tractor', 868 | 867: 'trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi', 869 | 868: 'tray', 870 | 869: 'trench coat', 871 | 870: 'tricycle, trike, velocipede', 872 | 871: 'trimaran', 873 | 872: 'tripod', 874 | 873: 'triumphal arch', 875 | 874: 'trolleybus, trolley coach, trackless trolley', 876 | 875: 'trombone', 877 | 876: 'tub, vat', 878 | 877: 'turnstile', 879 | 878: 'typewriter keyboard', 880 | 879: 'umbrella', 881 | 880: 'unicycle, monocycle', 882 | 881: 'upright, upright piano', 883 | 882: 'vacuum, vacuum cleaner', 884 | 883: 'vase', 885 | 884: 'vault', 886 | 885: 'velvet', 887 | 886: 'vending machine', 888 | 887: 'vestment', 889 | 888: 'viaduct', 890 | 889: 'violin, fiddle', 891 | 890: 'volleyball', 892 | 891: 'waffle iron', 893 | 892: 'wall clock', 894 | 893: 'wallet, billfold, notecase, pocketbook', 895 | 894: 'wardrobe, closet, press', 896 | 895: 'warplane, military plane', 897 | 896: 'washbasin, handbasin, washbowl, lavabo, wash-hand basin', 898 | 897: 'washer, automatic washer, washing machine', 899 | 898: 'water bottle', 900 | 899: 'water jug', 901 | 900: 'water tower', 902 | 901: 'whiskey jug', 903 | 902: 'whistle', 904 | 903: 'wig', 905 | 904: 'window screen', 906 | 905: 'window shade', 907 | 906: 'Windsor tie', 908 | 907: 'wine bottle', 909 | 908: 'wing', 910 | 909: 'wok', 911 | 910: 'wooden spoon', 912 | 911: 'wool, woolen, woollen', 913 | 912: 'worm fence, snake fence, snake-rail fence, Virginia fence', 914 | 913: 'wreck', 915 | 914: 'yawl', 916 | 915: 'yurt', 917 | 916: 'web site, website, internet site, site', 918 | 917: 'comic book', 919 | 918: 'crossword puzzle, crossword', 920 | 919: 'street sign', 921 | 920: 'traffic light, traffic signal, stoplight', 922 | 921: 'book jacket, dust cover, dust jacket, dust wrapper', 923 | 922: 'menu', 924 | 923: 'plate', 925 | 924: 'guacamole', 926 | 925: 'consomme', 927 | 926: 'hot pot, hotpot', 928 | 927: 'trifle', 929 | 928: 'ice cream, icecream', 930 | 929: 'ice lolly, lolly, lollipop, popsicle', 931 | 930: 'French loaf', 932 | 931: 'bagel, beigel', 933 | 932: 'pretzel', 934 | 933: 'cheeseburger', 935 | 934: 'hotdog, hot dog, red hot', 936 | 935: 'mashed potato', 937 | 936: 'head cabbage', 938 | 937: 'broccoli', 939 | 938: 'cauliflower', 940 | 939: 'zucchini, courgette', 941 | 940: 'spaghetti squash', 942 | 941: 'acorn squash', 943 | 942: 'butternut squash', 944 | 943: 'cucumber, cuke', 945 | 944: 'artichoke, globe artichoke', 946 | 945: 'bell pepper', 947 | 946: 'cardoon', 948 | 947: 'mushroom', 949 | 948: 'Granny Smith', 950 | 949: 'strawberry', 951 | 950: 'orange', 952 | 951: 'lemon', 953 | 952: 'fig', 954 | 953: 'pineapple, ananas', 955 | 954: 'banana', 956 | 955: 'jackfruit, jak, jack', 957 | 956: 'custard apple', 958 | 957: 'pomegranate', 959 | 958: 'hay', 960 | 959: 'carbonara', 961 | 960: 'chocolate sauce, chocolate syrup', 962 | 961: 'dough', 963 | 962: 'meat loaf, meatloaf', 964 | 963: 'pizza, pizza pie', 965 | 964: 'potpie', 966 | 965: 'burrito', 967 | 966: 'red wine', 968 | 967: 'espresso', 969 | 968: 'cup', 970 | 969: 'eggnog', 971 | 970: 'alp', 972 | 971: 'bubble', 973 | 972: 'cliff, drop, drop-off', 974 | 973: 'coral reef', 975 | 974: 'geyser', 976 | 975: 'lakeside, lakeshore', 977 | 976: 'promontory, headland, head, foreland', 978 | 977: 'sandbar, sand bar', 979 | 978: 'seashore, coast, seacoast, sea-coast', 980 | 979: 'valley, vale', 981 | 980: 'volcano', 982 | 981: 'ballplayer, baseball player', 983 | 982: 'groom, bridegroom', 984 | 983: 'scuba diver', 985 | 984: 'rapeseed', 986 | 985: 'daisy', 987 | 986: "yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum", 988 | 987: 'corn', 989 | 988: 'acorn', 990 | 989: 'hip, rose hip, rosehip', 991 | 990: 'buckeye, horse chestnut, conker', 992 | 991: 'coral fungus', 993 | 992: 'agaric', 994 | 993: 'gyromitra', 995 | 994: 'stinkhorn, carrion fungus', 996 | 995: 'earthstar', 997 | 996: 'hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa', 998 | 997: 'bolete', 999 | 998: 'ear, spike, capitulum', 1000 | 999: 'toilet tissue, toilet paper, bathroom tissue'} 1001 | -------------------------------------------------------------------------------- /imagenet_data/grouse.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/imagenet_data/grouse.jpg -------------------------------------------------------------------------------- /imagenet_data/husky.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maltanar/qnn-inference-examples/7087b974dcf87ecdf07a1b5e1ebf9b62ff4eeb27/imagenet_data/husky.jpg --------------------------------------------------------------------------------