├── .gitattributes ├── .gitignore ├── images ├── grizzly.jpg ├── jacobian_shapes.png ├── chapter1_cat_example.jpg └── basic-derivative-chain-rules.png ├── requirements.txt ├── export.pkl ├── catdog_classifier.pkl ├── README.md ├── LICENSE ├── voila_sample_bear_classifier.ipynb ├── voila_catdog_classifier.ipynb ├── Deep Learning Book Notes Ch 2 Exercises.ipynb ├── ExploreKaggle.ipynb ├── play-with-equations.ipynb └── CollectRealFingersData.ipynb /.gitattributes: -------------------------------------------------------------------------------- 1 | *.pkl filter=lfs diff=lfs merge=lfs -text 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .ipynb_checkpoints/* 3 | __pycache__/* 4 | 5 | -------------------------------------------------------------------------------- /images/grizzly.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/butchland/fastai_nb_explorations/master/images/grizzly.jpg -------------------------------------------------------------------------------- /images/jacobian_shapes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/butchland/fastai_nb_explorations/master/images/jacobian_shapes.png -------------------------------------------------------------------------------- /images/chapter1_cat_example.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/butchland/fastai_nb_explorations/master/images/chapter1_cat_example.jpg -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | fastcore==1.0.11 2 | fastai==2.0.12 3 | voila==0.1.21 4 | pillow>=7.1.0 5 | packaging==20.3 6 | ipywidgets==7.5.1 7 | -------------------------------------------------------------------------------- /images/basic-derivative-chain-rules.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/butchland/fastai_nb_explorations/master/images/basic-derivative-chain-rules.png -------------------------------------------------------------------------------- /export.pkl: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:dabcaa60d27152d1cea9d49d65f93337b269ffda9f246f27c8f7dc15aa558296 3 | size 87702859 4 | -------------------------------------------------------------------------------- /catdog_classifier.pkl: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:9f35e99812da764b0311a055a29e1d8887ab04561f094d44424d165c571b7dfd 3 | size 88296082 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # fastai_nb_explorations 2 | Explorations using jupyter notebooks and fastai2 3 | 4 | ### Run a sample app on binder 5 | 6 | * [sample voila bear classifier ![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/butchland/fastai_nb_explorations/master?urlpath=%2Fvoila%2Frender%2Fvoila_sample_bear_classifier.ipynb) 7 | 8 | * [sample voila pet classifier ![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/butchland/fastai_nb_explorations/master?urlpath=%2Fvoila%2Frender%2Fvoila_catdog_classifier.ipynb) -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Butch Landingin 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /voila_sample_bear_classifier.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "#hide\n", 10 | "from fastai.vision.all import *\n", 11 | "from ipywidgets import widgets \n", 12 | "from fastai.vision.widgets import *" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": 2, 18 | "metadata": {}, 19 | "outputs": [], 20 | "source": [ 21 | "#hide\n", 22 | "path = Path()\n", 23 | "# path.ls(file_exts='.pkl')" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 3, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "learn_inf = load_learner(path/'export.pkl')" 33 | ] 34 | }, 35 | { 36 | "cell_type": "markdown", 37 | "metadata": {}, 38 | "source": [ 39 | "### Creating a Notebook app from the model" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 4, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "btn_upload = SimpleNamespace(data = ['images/grizzly.jpg'])" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": 5, 54 | "metadata": {}, 55 | "outputs": [], 56 | "source": [ 57 | "img = PILImage.create(btn_upload.data[-1])" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": 6, 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "out_pl = widgets.Output()\n", 67 | "out_pl.clear_output()\n", 68 | "with out_pl: display(img.to_thumb(128,128))\n", 69 | "# out_pl" 70 | ] 71 | }, 72 | { 73 | "cell_type": "code", 74 | "execution_count": 7, 75 | "metadata": {}, 76 | "outputs": [ 77 | { 78 | "data": { 79 | "text/html": [], 80 | "text/plain": [ 81 | "" 82 | ] 83 | }, 84 | "metadata": {}, 85 | "output_type": "display_data" 86 | } 87 | ], 88 | "source": [ 89 | "pred,pred_idx,probs = learn_inf.predict(img)" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": 8, 95 | "metadata": {}, 96 | "outputs": [], 97 | "source": [ 98 | "lbl_pred = widgets.Label()\n", 99 | "lbl_pred.value = f'Prediction: {pred}; Probability: {probs[pred_idx]:.04f}'\n", 100 | "# lbl_pred" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": 9, 106 | "metadata": {}, 107 | "outputs": [], 108 | "source": [ 109 | "btn_run = widgets.Button(description='Classify')\n", 110 | "# btn_run" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": 10, 116 | "metadata": {}, 117 | "outputs": [], 118 | "source": [ 119 | "def on_click_classify(change):\n", 120 | " img = PILImage.create(btn_upload.data[-1])\n", 121 | " out_pl.clear_output()\n", 122 | " with out_pl: display(img.to_thumb(128,128))\n", 123 | " pred,pred_idx,probs = learn_inf.predict(img)\n", 124 | " lbl_pred.value = f'Prediction: {pred}; Probability: {probs[pred_idx]:.04f}'\n", 125 | "\n", 126 | "btn_run.on_click(on_click_classify)" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": 11, 132 | "metadata": {}, 133 | "outputs": [], 134 | "source": [ 135 | "#hide\n", 136 | "#Putting back btn_upload to a widget for next cell\n", 137 | "btn_upload = widgets.FileUpload()" 138 | ] 139 | }, 140 | { 141 | "cell_type": "code", 142 | "execution_count": 12, 143 | "metadata": {}, 144 | "outputs": [ 145 | { 146 | "data": { 147 | "application/vnd.jupyter.widget-view+json": { 148 | "model_id": "8cf0770ed9d04b13921e0940f7db2cb4", 149 | "version_major": 2, 150 | "version_minor": 0 151 | }, 152 | "text/plain": [ 153 | "VBox(children=(Label(value='Select your bear!'), FileUpload(value={}, description='Upload'), Button(descriptio…" 154 | ] 155 | }, 156 | "metadata": {}, 157 | "output_type": "display_data" 158 | }, 159 | { 160 | "data": { 161 | "text/html": [], 162 | "text/plain": [ 163 | "" 164 | ] 165 | }, 166 | "metadata": {}, 167 | "output_type": "display_data" 168 | } 169 | ], 170 | "source": [ 171 | "VBox([widgets.Label('Select your bear!'), \n", 172 | " btn_upload, btn_run, out_pl, lbl_pred])" 173 | ] 174 | }, 175 | { 176 | "cell_type": "code", 177 | "execution_count": null, 178 | "metadata": {}, 179 | "outputs": [], 180 | "source": [] 181 | } 182 | ], 183 | "metadata": { 184 | "jupytext": { 185 | "split_at_heading": true 186 | }, 187 | "kernelspec": { 188 | "display_name": "Python 3", 189 | "language": "python", 190 | "name": "python3" 191 | }, 192 | "language_info": { 193 | "codemirror_mode": { 194 | "name": "ipython", 195 | "version": 3 196 | }, 197 | "file_extension": ".py", 198 | "mimetype": "text/x-python", 199 | "name": "python", 200 | "nbconvert_exporter": "python", 201 | "pygments_lexer": "ipython3", 202 | "version": "3.7.7" 203 | }, 204 | "varInspector": { 205 | "cols": { 206 | "lenName": 16, 207 | "lenType": 16, 208 | "lenVar": 40 209 | }, 210 | "kernels_config": { 211 | "python": { 212 | "delete_cmd_postfix": "", 213 | "delete_cmd_prefix": "del ", 214 | "library": "var_list.py", 215 | "varRefreshCmd": "print(var_dic_list())" 216 | }, 217 | "r": { 218 | "delete_cmd_postfix": ") ", 219 | "delete_cmd_prefix": "rm(", 220 | "library": "var_list.r", 221 | "varRefreshCmd": "cat(var_dic_list()) " 222 | } 223 | }, 224 | "types_to_exclude": [ 225 | "module", 226 | "function", 227 | "builtin_function_or_method", 228 | "instance", 229 | "_Feature" 230 | ], 231 | "window_display": false 232 | } 233 | }, 234 | "nbformat": 4, 235 | "nbformat_minor": 4 236 | } 237 | -------------------------------------------------------------------------------- /voila_catdog_classifier.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "#hide\n", 10 | "from fastai.vision.all import *\n", 11 | "from ipywidgets import widgets \n", 12 | "from fastai.vision.widgets import *" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": 2, 18 | "metadata": {}, 19 | "outputs": [], 20 | "source": [ 21 | "#hide\n", 22 | "path = Path()\n", 23 | "# path.ls(file_exts='.pkl')" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 3, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "def is_cat(item):\n", 33 | " return 'cat' if item.name[0].isupper() else 'dog'" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": 4, 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "learn_inf = load_learner(path/'catdog_classifier.pkl')" 43 | ] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "metadata": {}, 48 | "source": [ 49 | "### A Naive Pet Guesser" 50 | ] 51 | }, 52 | { 53 | "cell_type": "code", 54 | "execution_count": 5, 55 | "metadata": {}, 56 | "outputs": [], 57 | "source": [ 58 | "btn_upload = SimpleNamespace(data = ['images/chapter1_cat_example.jpg'])" 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": 6, 64 | "metadata": {}, 65 | "outputs": [], 66 | "source": [ 67 | "img = PILImage.create(btn_upload.data[-1])" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": 7, 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "out_pl = widgets.Output()\n", 77 | "out_pl.clear_output()\n", 78 | "with out_pl: display(img.to_thumb(128,128))\n", 79 | "# out_pl" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": 8, 85 | "metadata": {}, 86 | "outputs": [ 87 | { 88 | "data": { 89 | "text/html": [], 90 | "text/plain": [ 91 | "" 92 | ] 93 | }, 94 | "metadata": {}, 95 | "output_type": "display_data" 96 | } 97 | ], 98 | "source": [ 99 | "pred,pred_idx,probs = learn_inf.predict(img)" 100 | ] 101 | }, 102 | { 103 | "cell_type": "code", 104 | "execution_count": 9, 105 | "metadata": {}, 106 | "outputs": [], 107 | "source": [ 108 | "lbl_pred = widgets.Label()\n", 109 | "lbl_pred.value = f'its probably a {pred} with a probability of {probs[pred_idx]*100.:.02f} percent'\n", 110 | "# lbl_pred" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": 10, 116 | "metadata": {}, 117 | "outputs": [], 118 | "source": [ 119 | "btn_run = widgets.Button(description='Classify')\n", 120 | "# btn_run" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": 11, 126 | "metadata": {}, 127 | "outputs": [], 128 | "source": [ 129 | "def on_click_classify(change):\n", 130 | " img = PILImage.create(btn_upload.data[-1])\n", 131 | " out_pl.clear_output()\n", 132 | " with out_pl: display(img.to_thumb(128,128))\n", 133 | " pred,pred_idx,probs = learn_inf.predict(img)\n", 134 | " lbl_pred.value = f'its probably a {pred} with a probability of {probs[pred_idx]*100.:.02f} percent'\n", 135 | "\n", 136 | "btn_run.on_click(on_click_classify)" 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": 12, 142 | "metadata": {}, 143 | "outputs": [], 144 | "source": [ 145 | "#hide\n", 146 | "#Putting back btn_upload to a widget for next cell\n", 147 | "btn_upload = widgets.FileUpload()" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": 13, 153 | "metadata": {}, 154 | "outputs": [ 155 | { 156 | "data": { 157 | "application/vnd.jupyter.widget-view+json": { 158 | "model_id": "1cafad554be742fcb4e0823339d451c8", 159 | "version_major": 2, 160 | "version_minor": 0 161 | }, 162 | "text/plain": [ 163 | "VBox(children=(Label(value='Upload your pet portrait!'), FileUpload(value={}, description='Upload'), Button(de…" 164 | ] 165 | }, 166 | "metadata": {}, 167 | "output_type": "display_data" 168 | }, 169 | { 170 | "data": { 171 | "text/html": [], 172 | "text/plain": [ 173 | "" 174 | ] 175 | }, 176 | "metadata": {}, 177 | "output_type": "display_data" 178 | } 179 | ], 180 | "source": [ 181 | "VBox([widgets.Label('Upload your pet portrait!'), \n", 182 | " btn_upload, btn_run, out_pl, lbl_pred])" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": null, 188 | "metadata": {}, 189 | "outputs": [], 190 | "source": [] 191 | } 192 | ], 193 | "metadata": { 194 | "jupytext": { 195 | "split_at_heading": true 196 | }, 197 | "kernelspec": { 198 | "display_name": "Python 3", 199 | "language": "python", 200 | "name": "python3" 201 | }, 202 | "language_info": { 203 | "codemirror_mode": { 204 | "name": "ipython", 205 | "version": 3 206 | }, 207 | "file_extension": ".py", 208 | "mimetype": "text/x-python", 209 | "name": "python", 210 | "nbconvert_exporter": "python", 211 | "pygments_lexer": "ipython3", 212 | "version": "3.7.7" 213 | }, 214 | "varInspector": { 215 | "cols": { 216 | "lenName": 16, 217 | "lenType": 16, 218 | "lenVar": 40 219 | }, 220 | "kernels_config": { 221 | "python": { 222 | "delete_cmd_postfix": "", 223 | "delete_cmd_prefix": "del ", 224 | "library": "var_list.py", 225 | "varRefreshCmd": "print(var_dic_list())" 226 | }, 227 | "r": { 228 | "delete_cmd_postfix": ") ", 229 | "delete_cmd_prefix": "rm(", 230 | "library": "var_list.r", 231 | "varRefreshCmd": "cat(var_dic_list()) " 232 | } 233 | }, 234 | "types_to_exclude": [ 235 | "module", 236 | "function", 237 | "builtin_function_or_method", 238 | "instance", 239 | "_Feature" 240 | ], 241 | "window_display": false 242 | } 243 | }, 244 | "nbformat": 4, 245 | "nbformat_minor": 4 246 | } 247 | -------------------------------------------------------------------------------- /Deep Learning Book Notes Ch 2 Exercises.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Chapter 2 Linear Algebra Exercises" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "### Exercises\n", 15 | "\n", 16 | "* [Source](https://www.deeplearningbook.org/linear_algebra.pdf) \n", 17 | " * Given the expression $ \\alpha \\pmb{u} $ for $ \\alpha \\in \\mathbb{R} $ and a unit vector $ \\pmb{u} \\in \\mathbb{R}^n $, such that varying $\\alpha$ defines a set of points (aka a line) along the direction of $\\pmb{u}$, and given an arbitrary $\\pmb{x} \\in \\mathbb{R}^n$, derive the expression defining $\\pmb{y}$ such that $\\pmb{y} = \\alpha \\pmb{u} $ where $\\pmb{y}$ is the point along the line that lies closest to the point $\\pmb{x}$.\n", 18 | " * Solution:\n", 19 | " * Defining the variable $d$ as the distance between $\\pmb{x}$ and $\\pmb{y}$, we can state that $ d = \\lVert \\pmb{x} - \\pmb{y} \\rVert $. \n", 20 | " * We can restate this as $ d = \\lVert \\pmb{x} - \\alpha\\pmb{u} \\rVert $, since $\\pmb{y} = \\alpha \\pmb{u} $ \n", 21 | " * In order to find the minimum $d$, we can vary $\\alpha$ such for the point at which $d$ is the minimum, the derivative of $d$ wrt to $\\alpha$ is zero; in other words $\\frac{dd}{d\\alpha} = 0$\n", 22 | " * We can also include the observation that squaring the distance $d$ does not change the point $\\pmb{y}$ closest to $\\pmb{x}$, since $d$ is always positive or zero.\n", 23 | " * so we can redefine $ d = \\lVert \\pmb{x} - \\alpha\\pmb{u} \\rVert^2 $ and at the point where $d$ is the minimum, the derivative of $d$ wrt to $\\alpha$ is still zero as above; in other words, the following still holds: $\\frac{dd}{d\\alpha} = 0$.\n", 24 | " * Expanding the equation\n", 25 | "$$\n", 26 | "0 = \\frac{dd}{d\\alpha} =\n", 27 | "\\frac{d\\lVert \\pmb{x} - \\alpha\\pmb{u} \\rVert^2}{d\\alpha} =\n", 28 | "\\frac{d}{d\\alpha}\\lVert \\pmb{x} - \\alpha\\pmb{u} \\rVert^2 =\n", 29 | "\\frac{d}{d\\alpha}[(\\pmb{x} - \\alpha\\pmb{u})^T(\\pmb{x} - \\alpha\\pmb{u})] =\n", 30 | "\\frac{d}{d\\alpha}[(\\pmb{x}^T - \\alpha\\pmb{u}^T)(\\pmb{x} - \\alpha\\pmb{u})] =\n", 31 | "\\frac{d}{d\\alpha}(\\pmb{x}^T\\pmb{x} - \\alpha\\pmb{u}^T\\pmb{x} - \\pmb{x}^T\\alpha\\pmb{u} + \\alpha\\pmb{u}^T\\alpha\\pmb{u}) =\n", 32 | "\\frac{d}{d\\alpha}(\\pmb{x}^T\\pmb{x} - \\alpha(\\pmb{u}^T\\pmb{x} + \\pmb{x}^T\\pmb{u}) + \\alpha^2\\pmb{u}^T\\pmb{u})\n", 33 | "$$\n", 34 | " Since the matrix multiplication transpose rule for vectors $x$ and $y$ states that $\\pmb{x}^T\\pmb{y} = \\pmb{y}^T\\pmb{x}$, then $\\pmb{u}^T\\pmb{x} = \\pmb{x}^T\\pmb{u}$ and since $\\pmb{u}^T\\pmb{u} = 1$ (as per definition of a unit vector), the above equation simplifies to \n", 35 | "$$\n", 36 | "0 = \\frac{dd}{d\\alpha} =\n", 37 | "\\frac{d}{d\\alpha}(\\pmb{x}^T\\pmb{x} - \\alpha(\\pmb{u}^T\\pmb{x} + \\pmb{u}^T\\pmb{x}) + \\alpha^2\\pmb{u}^T\\pmb{u}) =\n", 38 | "\\frac{d}{d\\alpha}(\\pmb{x}^T\\pmb{x} - 2\\alpha\\pmb{u}^T\\pmb{x} + \\alpha^2) =\n", 39 | "\\frac{d}{d\\alpha}\\pmb{x}^T\\pmb{x} -2\\frac{d}{d\\alpha}\\alpha\\pmb{u}^T\\pmb{x} + \\frac{d}{d\\alpha}\\alpha^2 = \n", 40 | "0 - 2\\pmb{u}^T\\pmb{x} + 2\\alpha\n", 41 | "$$\n", 42 | "Solving for $\\alpha$,\n", 43 | "$$\n", 44 | "0 = -2\\pmb{u}^T\\pmb{x} + 2\\alpha \\\\\n", 45 | "2\\pmb{u}^T\\pmb{x} = 2\\alpha \\\\\n", 46 | "\\alpha = \\pmb{u}^T\\pmb{x}\n", 47 | "$$\n", 48 | "So\n", 49 | "$$\n", 50 | "\\pmb{y} = \\alpha\\pmb{u} =\n", 51 | "\\pmb{u}^T\\pmb{x}\\pmb{u} =\n", 52 | "\\pmb{x}^T\\pmb{u}\\pmb{u}\n", 53 | "$$" 54 | ] 55 | }, 56 | { 57 | "cell_type": "markdown", 58 | "metadata": {}, 59 | "source": [ 60 | "* Source: Contributed by James\n", 61 | " * Given the definition of the Trace operator, $ Tr(\\pmb{C}) = \\sum_{i=1}^{n}C_{i,i} $, prove that $Tr(\\pmb{AB}) = Tr(\\pmb{BA}) $ for any $\\pmb{A} \\in \\mathbb{R}^{m \\times n} $ and $\\pmb{B} \\in \\mathbb{R}^{n \\times m}$\n", 62 | " * Solution:\n", 63 | " * Given the definition of a Matrix Product of $\\pmb{C} = \\pmb{AB}$, where $C_{i,j} = \\sum_{k=1}^{n}A_{i,k}B_{k,j}$, then $\\pmb{C} \\in \\mathbb{R}^{m \\times m} $\n", 64 | " * This means that for any diagonal element $C_{i,i}$ for $i \\in (1,n)$, since based on the alternative formulation that $ C_{i,j} $ is dot product of row $ i $ of $\\pmb{A}$ and column $ j $ of $ \\pmb{B} $, then $ C_{i,i} $ is dot product of row $ i $ of $\\pmb{A}$ and column $ i $ of $ \\pmb{B} $. In other words,\n", 65 | "$$\n", 66 | "C_{i,i} = \\sum_{k=1}^{n}A_{i,k}B_{k,i}\n", 67 | "$$\n", 68 | "and \n", 69 | "$$\n", 70 | "Tr(C) = \\sum_{i=1}^{m}C_{i,i} =\n", 71 | "\\sum_{i=1}^{m}\\sum_{k=1}^{n}A_{i,k}B_{k,i}\n", 72 | "$$\n", 73 | "Expanding the terms of $Tr(C)$, we get\n", 74 | "$$\n", 75 | "Tr(C) = \\\\\n", 76 | "\\sum_{i=1}^{m}\\sum_{k=1}^{n}A_{i,k}B_{k,i} = \\\\\n", 77 | "\\sum_{i=1}^{m}(a_{i,1}b_{1,i} + a_{i,2}b_{2,i} + \\dotsb + a_{i,n}b_{n,i}) = \\\\\n", 78 | "(a_{1,1}b_{1,1} + a_{1,2}b_{2,1} + \\dotsb + a_{1,n}b_{n,1}) + \\\\\n", 79 | "(a_{2,1}b_{1,2} + a_{2,2}b_{2,2} + \\dotsb + a_{2,n}b_{n,2}) + \\\\\n", 80 | "\\vdots \\\\\n", 81 | "(a_{m,1}b_{1,m} + a_{m,2}b_{2,m} + \\dotsb + a_{m,n}b_{n,m})\n", 82 | "$$\n", 83 | " * On the other hand let $\\pmb{D} = \\pmb{BA}$, so that $\\pmb{D} \\in \\mathbb{R}^{n \\times n} $ then\n", 84 | "$$\n", 85 | "D_{i,i} = \\sum_{k=1}^{m}B_{i,k}A_{k,i}\n", 86 | "$$\n", 87 | "and\n", 88 | "$$\n", 89 | "Tr(D) = \\sum_{i=1}^{n}D_{i,i} =\n", 90 | "\\sum_{i=1}^{n}\\sum_{k=1}^{m}B_{i,k}A_{k,i}\n", 91 | "$$\n", 92 | "Likewise, expanding the terms of $Tr(D)$, we get\n", 93 | "$$\n", 94 | "Tr(D) = \\\\\n", 95 | "\\sum_{i=1}^{n}\\sum_{k=1}^{m}B_{i,k}A_{k,i} = \\\\\n", 96 | "\\sum_{i=1}^{n}(b_{i,1}a_{1,i} + b_{i,2}a_{2,i} + \\dotsb + b_{i,m}a_{m,i}) = \\\\\n", 97 | "(b_{1,1}a_{1,1} + b_{1,2}a_{2,1} + \\dotsb + b_{1,m}a_{m,1}) + \\\\\n", 98 | "(b_{2,1}a_{1,2} + b_{2,2}a_{2,2} + \\dotsb + b_{2,m}a_{m,2}) + \\\\\n", 99 | "\\vdots \\\\\n", 100 | "(b_{n,1}a_{1,n} + b_{n,2}a_{2,n} + \\dotsb + b_{n,m}a_{m,n})\n", 101 | "$$\n", 102 | "Rearranging the terms since $ a_{i,j}b_{j,i} = b_{j,i}a_{i,j} $, we have\n", 103 | "$$\n", 104 | "(a_{1,1}b_{1,1} + a_{2,1}b_{1,2} + \\dotsb + a_{m,1}b_{1,m}) + \\\\\n", 105 | "(a_{1,2}b_{2,1} + a_{2,2}b_{2,2} + \\dotsb + a_{m,2}b_{2,m}) + \\\\\n", 106 | "\\vdots \\\\\n", 107 | "a_{1,n}(b_{n,1} + a_{2,n}b_{n,2} + \\dotsb + a_{m,n}b_{n,m})\n", 108 | "$$\n", 109 | "and by transposing terms from the columns into rows, we end up with \n", 110 | "$$\n", 111 | "(a_{1,1}b_{1,1} + a_{1,2}b_{2,1} + \\dotsb + a_{1,n}b_{n,1}) + \\\\\n", 112 | "(a_{2,1}b_{1,2} + a_{2,2}b_{2,2} + \\dotsb + a_{2,n}b_{n,2}) + \\\\\n", 113 | "\\vdots \\\\\n", 114 | "(a_{m,1}b_{1,m} + a_{m,2}b_{2,m} + \\dotsb + a_{m,n}b_{n,m})\n", 115 | "$$\n", 116 | "which is equal to \n", 117 | "$$\n", 118 | "\\sum_{i=1}^{m}(a_{i,1}b_{1,i} + a_{i,2}b_{2,i} + \\dotsb + a_{i,n}b_{n,i}) = \n", 119 | "\\sum_{i=1}^{m}\\sum_{k=1}^{n}A_{i,k}B_{k,i} =\n", 120 | "Tr(\\pmb{AB})\n", 121 | "$$\n", 122 | "So $Tr(\\pmb{D}) = Tr(\\pmb{BA}) = Tr(\\pmb{AB})$. Q.E.D. :D" 123 | ] 124 | }, 125 | { 126 | "cell_type": "markdown", 127 | "metadata": {}, 128 | "source": [ 129 | "* Derivation of PCA (to be completed)" 130 | ] 131 | }, 132 | { 133 | "cell_type": "code", 134 | "execution_count": null, 135 | "metadata": {}, 136 | "outputs": [], 137 | "source": [] 138 | } 139 | ], 140 | "metadata": { 141 | "kernelspec": { 142 | "display_name": "Python 3", 143 | "language": "python", 144 | "name": "python3" 145 | }, 146 | "language_info": { 147 | "codemirror_mode": { 148 | "name": "ipython", 149 | "version": 3 150 | }, 151 | "file_extension": ".py", 152 | "mimetype": "text/x-python", 153 | "name": "python", 154 | "nbconvert_exporter": "python", 155 | "pygments_lexer": "ipython3", 156 | "version": "3.7.7" 157 | }, 158 | "varInspector": { 159 | "cols": { 160 | "lenName": 16, 161 | "lenType": 16, 162 | "lenVar": 40 163 | }, 164 | "kernels_config": { 165 | "python": { 166 | "delete_cmd_postfix": "", 167 | "delete_cmd_prefix": "del ", 168 | "library": "var_list.py", 169 | "varRefreshCmd": "print(var_dic_list())" 170 | }, 171 | "r": { 172 | "delete_cmd_postfix": ") ", 173 | "delete_cmd_prefix": "rm(", 174 | "library": "var_list.r", 175 | "varRefreshCmd": "cat(var_dic_list()) " 176 | } 177 | }, 178 | "types_to_exclude": [ 179 | "module", 180 | "function", 181 | "builtin_function_or_method", 182 | "instance", 183 | "_Feature" 184 | ], 185 | "window_display": false 186 | } 187 | }, 188 | "nbformat": 4, 189 | "nbformat_minor": 4 190 | } 191 | -------------------------------------------------------------------------------- /ExploreKaggle.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "kernelspec": { 6 | "display_name": "Python 3", 7 | "language": "python", 8 | "name": "python3" 9 | }, 10 | "language_info": { 11 | "codemirror_mode": { 12 | "name": "ipython", 13 | "version": 3 14 | }, 15 | "file_extension": ".py", 16 | "mimetype": "text/x-python", 17 | "name": "python", 18 | "nbconvert_exporter": "python", 19 | "pygments_lexer": "ipython3", 20 | "version": "3.7.7" 21 | }, 22 | "colab": { 23 | "name": "ExploreKaggle.ipynb", 24 | "provenance": [], 25 | "collapsed_sections": [], 26 | "toc_visible": true, 27 | "include_colab_link": true 28 | } 29 | }, 30 | "cells": [ 31 | { 32 | "cell_type": "markdown", 33 | "metadata": { 34 | "id": "view-in-github", 35 | "colab_type": "text" 36 | }, 37 | "source": [ 38 | "\"Open" 39 | ] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "metadata": { 44 | "id": "XFqpp97M1mwf", 45 | "colab_type": "text" 46 | }, 47 | "source": [ 48 | "## Explore and download kaggle datasets\n", 49 | "\n", 50 | "**Note**: Notebook doesn't need GPU - use CPU runtime on Colab" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "metadata": { 56 | "id": "b2ROhPMyzLtI", 57 | "colab_type": "code", 58 | "outputId": "0bad8ea2-d584-4234-f691-76a1dc4a94b8", 59 | "colab": { 60 | "base_uri": "https://localhost:8080/", 61 | "height": 102 62 | } 63 | }, 64 | "source": [ 65 | "#colab\n", 66 | "!curl https://course.fast.ai/setup/colab | bash" 67 | ], 68 | "execution_count": 1, 69 | "outputs": [ 70 | { 71 | "output_type": "stream", 72 | "text": [ 73 | " % Total % Received % Xferd Average Speed Time Time Time Current\n", 74 | " Dload Upload Total Spent Left Speed\n", 75 | "100 322 100 322 0 0 379 0 --:--:-- --:--:-- --:--:-- 379\n", 76 | "Updating fastai...\n", 77 | "Done.\n" 78 | ], 79 | "name": "stdout" 80 | } 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "metadata": { 86 | "id": "5KiU7X2-zLtT", 87 | "colab_type": "code", 88 | "outputId": "da07c453-a2e0-49a6-c437-7bb89a3187d2", 89 | "colab": { 90 | "base_uri": "https://localhost:8080/", 91 | "height": 1000 92 | } 93 | }, 94 | "source": [ 95 | "#colab\n", 96 | "!pip install fastai2 --upgrade\n", 97 | "!pip install fastcore --upgrade\n", 98 | "!pip install nbdev --upgrade" 99 | ], 100 | "execution_count": 2, 101 | "outputs": [ 102 | { 103 | "output_type": "stream", 104 | "text": [ 105 | "Collecting fastai2\n", 106 | "\u001b[?25l Downloading https://files.pythonhosted.org/packages/bd/2b/9e0f6d3d928105dec034727ed723c33340f71a77ca253b67bbec99eacd3f/fastai2-0.0.16-py3-none-any.whl (184kB)\n", 107 | "\u001b[K |████████████████████████████████| 184kB 1.4MB/s \n", 108 | "\u001b[?25hRequirement already satisfied, skipping upgrade: matplotlib in /usr/local/lib/python3.6/dist-packages (from fastai2) (3.2.1)\n", 109 | "Requirement already satisfied, skipping upgrade: scikit-learn in /usr/local/lib/python3.6/dist-packages (from fastai2) (0.22.2.post1)\n", 110 | "Requirement already satisfied, skipping upgrade: torch>=1.3.0 in /usr/local/lib/python3.6/dist-packages (from fastai2) (1.4.0)\n", 111 | "Requirement already satisfied, skipping upgrade: spacy in /usr/local/lib/python3.6/dist-packages (from fastai2) (2.2.4)\n", 112 | "Requirement already satisfied, skipping upgrade: pandas in /usr/local/lib/python3.6/dist-packages (from fastai2) (1.0.3)\n", 113 | "Requirement already satisfied, skipping upgrade: fastprogress>=0.1.22 in /usr/local/lib/python3.6/dist-packages (from fastai2) (0.2.2)\n", 114 | "Requirement already satisfied, skipping upgrade: scipy in /usr/local/lib/python3.6/dist-packages (from fastai2) (1.4.1)\n", 115 | "Requirement already satisfied, skipping upgrade: torchvision>=0.5 in /usr/local/lib/python3.6/dist-packages (from fastai2) (0.5.0)\n", 116 | "Requirement already satisfied, skipping upgrade: requests in /usr/local/lib/python3.6/dist-packages (from fastai2) (2.21.0)\n", 117 | "Requirement already satisfied, skipping upgrade: pillow in /usr/local/lib/python3.6/dist-packages (from fastai2) (7.0.0)\n", 118 | "Requirement already satisfied, skipping upgrade: pyyaml in /usr/local/lib/python3.6/dist-packages (from fastai2) (3.13)\n", 119 | "Collecting fastcore\n", 120 | " Downloading https://files.pythonhosted.org/packages/3b/b5/aed836ce5b16ea1088a5d1a41d400bc051abf90bbef58bb74d8fd01a76af/fastcore-0.1.16-py3-none-any.whl\n", 121 | "Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (2.8.1)\n", 122 | "Requirement already satisfied, skipping upgrade: numpy>=1.11 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (1.18.2)\n", 123 | "Requirement already satisfied, skipping upgrade: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (0.10.0)\n", 124 | "Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (2.4.6)\n", 125 | "Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (1.2.0)\n", 126 | "Requirement already satisfied, skipping upgrade: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->fastai2) (0.14.1)\n", 127 | "Requirement already satisfied, skipping upgrade: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (3.0.2)\n", 128 | "Requirement already satisfied, skipping upgrade: srsly<1.1.0,>=1.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (1.0.2)\n", 129 | "Requirement already satisfied, skipping upgrade: blis<0.5.0,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (0.4.1)\n", 130 | "Requirement already satisfied, skipping upgrade: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (1.0.0)\n", 131 | "Requirement already satisfied, skipping upgrade: thinc==7.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (7.4.0)\n", 132 | "Requirement already satisfied, skipping upgrade: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (0.6.0)\n", 133 | "Requirement already satisfied, skipping upgrade: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (2.0.3)\n", 134 | "Requirement already satisfied, skipping upgrade: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (4.38.0)\n", 135 | "Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (46.1.3)\n", 136 | "Requirement already satisfied, skipping upgrade: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (1.1.3)\n", 137 | "Requirement already satisfied, skipping upgrade: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (1.0.2)\n", 138 | "Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->fastai2) (2018.9)\n", 139 | "Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from torchvision>=0.5->fastai2) (1.12.0)\n", 140 | "Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->fastai2) (2.8)\n", 141 | "Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->fastai2) (1.24.3)\n", 142 | "Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->fastai2) (2019.11.28)\n", 143 | "Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->fastai2) (3.0.4)\n", 144 | "Requirement already satisfied, skipping upgrade: dataclasses>='0.7'; python_version < \"3.7\" in /usr/local/lib/python3.6/dist-packages (from fastcore->fastai2) (0.7)\n", 145 | "Requirement already satisfied, skipping upgrade: importlib-metadata>=0.20; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from catalogue<1.1.0,>=0.0.7->spacy->fastai2) (1.6.0)\n", 146 | "Requirement already satisfied, skipping upgrade: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata>=0.20; python_version < \"3.8\"->catalogue<1.1.0,>=0.0.7->spacy->fastai2) (3.1.0)\n", 147 | "Installing collected packages: fastcore, fastai2\n", 148 | "Successfully installed fastai2-0.0.16 fastcore-0.1.16\n", 149 | "Requirement already up-to-date: fastcore in /usr/local/lib/python3.6/dist-packages (0.1.16)\n", 150 | "Requirement already satisfied, skipping upgrade: dataclasses>='0.7'; python_version < \"3.7\" in /usr/local/lib/python3.6/dist-packages (from fastcore) (0.7)\n", 151 | "Requirement already satisfied, skipping upgrade: numpy in /usr/local/lib/python3.6/dist-packages (from fastcore) (1.18.2)\n", 152 | "Collecting nbdev\n", 153 | "\u001b[?25l Downloading https://files.pythonhosted.org/packages/a0/c2/2631b32810211b6be5b17aff8db7135a8f29ff2a09a397cd9efbba490d48/nbdev-0.2.17-py3-none-any.whl (44kB)\n", 154 | "\u001b[K |████████████████████████████████| 51kB 893kB/s \n", 155 | "\u001b[?25hRequirement already satisfied, skipping upgrade: nbconvert>=5.6.1 in /usr/local/lib/python3.6/dist-packages (from nbdev) (5.6.1)\n", 156 | "Requirement already satisfied, skipping upgrade: nbformat>=4.4.0 in /usr/local/lib/python3.6/dist-packages (from nbdev) (5.0.5)\n", 157 | "Collecting fastscript\n", 158 | " Downloading https://files.pythonhosted.org/packages/55/0e/ecdc0213646bc82986884121109a38b50bbc2cd2c491bbbfdc7ae39228e3/fastscript-0.1.4-py3-none-any.whl\n", 159 | "Requirement already satisfied, skipping upgrade: pyyaml in /usr/local/lib/python3.6/dist-packages (from nbdev) (3.13)\n", 160 | "Requirement already satisfied, skipping upgrade: packaging in /usr/local/lib/python3.6/dist-packages (from nbdev) (20.3)\n", 161 | "Requirement already satisfied, skipping upgrade: defusedxml in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (0.6.0)\n", 162 | "Requirement already satisfied, skipping upgrade: entrypoints>=0.2.2 in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (0.3)\n", 163 | "Requirement already satisfied, skipping upgrade: testpath in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (0.4.4)\n", 164 | "Requirement already satisfied, skipping upgrade: mistune<2,>=0.8.1 in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (0.8.4)\n", 165 | "Requirement already satisfied, skipping upgrade: jupyter-core in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (4.6.3)\n", 166 | "Requirement already satisfied, skipping upgrade: traitlets>=4.2 in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (4.3.3)\n", 167 | "Requirement already satisfied, skipping upgrade: pandocfilters>=1.4.1 in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (1.4.2)\n", 168 | "Requirement already satisfied, skipping upgrade: bleach in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (3.1.4)\n", 169 | "Requirement already satisfied, skipping upgrade: pygments in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (2.1.3)\n", 170 | "Requirement already satisfied, skipping upgrade: jinja2>=2.4 in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (2.11.1)\n", 171 | "Requirement already satisfied, skipping upgrade: jsonschema!=2.5.0,>=2.4 in /usr/local/lib/python3.6/dist-packages (from nbformat>=4.4.0->nbdev) (2.6.0)\n", 172 | "Requirement already satisfied, skipping upgrade: ipython-genutils in /usr/local/lib/python3.6/dist-packages (from nbformat>=4.4.0->nbdev) (0.2.0)\n", 173 | "Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from packaging->nbdev) (1.12.0)\n", 174 | "Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from packaging->nbdev) (2.4.6)\n", 175 | "Requirement already satisfied, skipping upgrade: decorator in /usr/local/lib/python3.6/dist-packages (from traitlets>=4.2->nbconvert>=5.6.1->nbdev) (4.4.2)\n", 176 | "Requirement already satisfied, skipping upgrade: webencodings in /usr/local/lib/python3.6/dist-packages (from bleach->nbconvert>=5.6.1->nbdev) (0.5.1)\n", 177 | "Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from jinja2>=2.4->nbconvert>=5.6.1->nbdev) (1.1.1)\n", 178 | "Installing collected packages: fastscript, nbdev\n", 179 | "Successfully installed fastscript-0.1.4 nbdev-0.2.17\n" 180 | ], 181 | "name": "stdout" 182 | } 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "metadata": { 188 | "id": "Cy5gjJ7uzLuF", 189 | "colab_type": "code", 190 | "colab": {} 191 | }, 192 | "source": [ 193 | "#import packages\n", 194 | "from fastai2.vision.all import *" 195 | ], 196 | "execution_count": 0, 197 | "outputs": [] 198 | }, 199 | { 200 | "cell_type": "code", 201 | "metadata": { 202 | "id": "gZ4anwg6zLtf", 203 | "colab_type": "code", 204 | "outputId": "6f784bde-01bc-4514-d59e-8a0e37ad835e", 205 | "colab": { 206 | "base_uri": "https://localhost:8080/", 207 | "height": 122 208 | } 209 | }, 210 | "source": [ 211 | "#colab\n", 212 | "from google.colab import drive\n", 213 | "drive.mount('/content/drive')" 214 | ], 215 | "execution_count": 4, 216 | "outputs": [ 217 | { 218 | "output_type": "stream", 219 | "text": [ 220 | "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n", 221 | "\n", 222 | "Enter your authorization code:\n", 223 | "··········\n", 224 | "Mounted at /content/drive\n" 225 | ], 226 | "name": "stdout" 227 | } 228 | ] 229 | }, 230 | { 231 | "cell_type": "code", 232 | "metadata": { 233 | "id": "pL2weviczLto", 234 | "colab_type": "code", 235 | "colab": {} 236 | }, 237 | "source": [ 238 | "#colab\n", 239 | "gdrive = Path('/content/drive/My Drive/fastai_v4')" 240 | ], 241 | "execution_count": 0, 242 | "outputs": [] 243 | }, 244 | { 245 | "cell_type": "code", 246 | "metadata": { 247 | "code_folding": [], 248 | "id": "HvE-z-izzLtt", 249 | "colab_type": "code", 250 | "colab": {} 251 | }, 252 | "source": [ 253 | "#local\n", 254 | "# gdrive = Path('/Users/butch/Google Drive/fastai_v4') " 255 | ], 256 | "execution_count": 0, 257 | "outputs": [] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "metadata": { 262 | "id": "2NYz-QokzLt1", 263 | "colab_type": "code", 264 | "outputId": "72070c1c-0c3f-4ed8-d911-31c8c0ca01f3", 265 | "colab": { 266 | "base_uri": "https://localhost:8080/", 267 | "height": 224 268 | } 269 | }, 270 | "source": [ 271 | "### Install kaggle package\n", 272 | "#colab\n", 273 | "!pip install kaggle --upgrade\n", 274 | "\n", 275 | "### Add instructions on getting kaggle.json from kaggle account\n", 276 | "# get from kaggle account page after registering/logging in\n", 277 | "# save kaggle.json to $HOME/.kaggle directory\n" 278 | ], 279 | "execution_count": 7, 280 | "outputs": [ 281 | { 282 | "output_type": "stream", 283 | "text": [ 284 | "Requirement already up-to-date: kaggle in /usr/local/lib/python3.6/dist-packages (1.5.6)\n", 285 | "Requirement already satisfied, skipping upgrade: six>=1.10 in /usr/local/lib/python3.6/dist-packages (from kaggle) (1.12.0)\n", 286 | "Requirement already satisfied, skipping upgrade: python-dateutil in /usr/local/lib/python3.6/dist-packages (from kaggle) (2.8.1)\n", 287 | "Requirement already satisfied, skipping upgrade: requests in /usr/local/lib/python3.6/dist-packages (from kaggle) (2.21.0)\n", 288 | "Requirement already satisfied, skipping upgrade: certifi in /usr/local/lib/python3.6/dist-packages (from kaggle) (2019.11.28)\n", 289 | "Requirement already satisfied, skipping upgrade: tqdm in /usr/local/lib/python3.6/dist-packages (from kaggle) (4.38.0)\n", 290 | "Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from kaggle) (1.24.3)\n", 291 | "Requirement already satisfied, skipping upgrade: python-slugify in /usr/local/lib/python3.6/dist-packages (from kaggle) (4.0.0)\n", 292 | "Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->kaggle) (2.8)\n", 293 | "Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->kaggle) (3.0.4)\n", 294 | "Requirement already satisfied, skipping upgrade: text-unidecode>=1.3 in /usr/local/lib/python3.6/dist-packages (from python-slugify->kaggle) (1.3)\n" 295 | ], 296 | "name": "stdout" 297 | } 298 | ] 299 | }, 300 | { 301 | "cell_type": "code", 302 | "metadata": { 303 | "id": "C9YZZJRNzLt7", 304 | "colab_type": "code", 305 | "colab": {} 306 | }, 307 | "source": [ 308 | "#utility function\n", 309 | "escdrive = lambda x : x.as_posix().replace(' ','\\ ')" 310 | ], 311 | "execution_count": 0, 312 | "outputs": [] 313 | }, 314 | { 315 | "cell_type": "code", 316 | "metadata": { 317 | "id": "41LSSMsXzLt_", 318 | "colab_type": "code", 319 | "colab": {} 320 | }, 321 | "source": [ 322 | "#colab \n", 323 | "# install kaggle.json from gdrive\n", 324 | "!mkdir -p /root/.kaggle\n", 325 | "!cp {escdrive(gdrive/'kaggle'/'kaggle.json')} /root/.kaggle/." 326 | ], 327 | "execution_count": 0, 328 | "outputs": [] 329 | }, 330 | { 331 | "cell_type": "code", 332 | "metadata": { 333 | "id": "p0OIobYQ09IT", 334 | "colab_type": "code", 335 | "colab": {} 336 | }, 337 | "source": [ 338 | "# import kaggle\n", 339 | "from kaggle.api import KaggleApi" 340 | ], 341 | "execution_count": 0, 342 | "outputs": [] 343 | }, 344 | { 345 | "cell_type": "code", 346 | "metadata": { 347 | "id": "zKYQGbTvzLuL", 348 | "colab_type": "code", 349 | "outputId": "091eb08e-7c8e-40c8-b1fb-ba4b2e2d96bf", 350 | "colab": { 351 | "base_uri": "https://localhost:8080/", 352 | "height": 102 353 | } 354 | }, 355 | "source": [ 356 | "config = Config(); config.d" 357 | ], 358 | "execution_count": 11, 359 | "outputs": [ 360 | { 361 | "output_type": "execute_result", 362 | "data": { 363 | "text/plain": [ 364 | "{'archive_path': '/root/.fastai/archive',\n", 365 | " 'data_path': '/root/.fastai/data',\n", 366 | " 'model_path': '/root/.fastai/models',\n", 367 | " 'storage_path': '/root/.fastai/data',\n", 368 | " 'version': 2}" 369 | ] 370 | }, 371 | "metadata": { 372 | "tags": [] 373 | }, 374 | "execution_count": 11 375 | } 376 | ] 377 | }, 378 | { 379 | "cell_type": "code", 380 | "metadata": { 381 | "id": "t78JqigvzLuS", 382 | "colab_type": "code", 383 | "colab": {} 384 | }, 385 | "source": [ 386 | "archive_path = config.d['archive_path']\n", 387 | "data_path = config.d['data_path']\n", 388 | "model_path = config.d['model_path']" 389 | ], 390 | "execution_count": 0, 391 | "outputs": [] 392 | }, 393 | { 394 | "cell_type": "code", 395 | "metadata": { 396 | "id": "lF-ny9ZjzLuY", 397 | "colab_type": "code", 398 | "colab": {} 399 | }, 400 | "source": [ 401 | "# get api object, note: will error if kaggle.json not setup\n", 402 | "api = KaggleApi()" 403 | ], 404 | "execution_count": 0, 405 | "outputs": [] 406 | }, 407 | { 408 | "cell_type": "code", 409 | "metadata": { 410 | "id": "bGiLo4XLzLud", 411 | "colab_type": "code", 412 | "colab": {} 413 | }, 414 | "source": [ 415 | "# help(api)" 416 | ], 417 | "execution_count": 0, 418 | "outputs": [] 419 | }, 420 | { 421 | "cell_type": "code", 422 | "metadata": { 423 | "id": "lYQpo7qYzLuj", 424 | "colab_type": "code", 425 | "colab": {} 426 | }, 427 | "source": [ 428 | "# authenticate using kaggle.json\n", 429 | "api.authenticate()" 430 | ], 431 | "execution_count": 0, 432 | "outputs": [] 433 | }, 434 | { 435 | "cell_type": "code", 436 | "metadata": { 437 | "id": "DFDUpcAazLun", 438 | "colab_type": "code", 439 | "colab": {} 440 | }, 441 | "source": [ 442 | "# download utiltity function\n", 443 | "def kaggle_download(dset,path='.'):\n", 444 | " dpath = Path(path)/dset.replace('/','_')\n", 445 | " api.dataset_download_files(dset,path=dpath.as_posix())\n", 446 | " return dpath" 447 | ], 448 | "execution_count": 0, 449 | "outputs": [] 450 | }, 451 | { 452 | "cell_type": "code", 453 | "metadata": { 454 | "id": "sswsMtXqzLus", 455 | "colab_type": "code", 456 | "colab": { 457 | "base_uri": "https://localhost:8080/", 458 | "height": 34 459 | }, 460 | "outputId": "045f81e4-23cb-41c0-93d9-2fde92316075" 461 | }, 462 | "source": [ 463 | "cfg = api.read_config_file();cfg" 464 | ], 465 | "execution_count": 17, 466 | "outputs": [ 467 | { 468 | "output_type": "execute_result", 469 | "data": { 470 | "text/plain": [ 471 | "{'key': 'b9b9381e25077978056216e08da77b1d', 'username': 'butchland'}" 472 | ] 473 | }, 474 | "metadata": { 475 | "tags": [] 476 | }, 477 | "execution_count": 17 478 | } 479 | ] 480 | }, 481 | { 482 | "cell_type": "code", 483 | "metadata": { 484 | "id": "j9m12Vn8zLuw", 485 | "colab_type": "code", 486 | "outputId": "2bb88196-bc60-4d43-869f-471a0ddcffd8", 487 | "colab": { 488 | "base_uri": "https://localhost:8080/", 489 | "height": 34 490 | } 491 | }, 492 | "source": [ 493 | "api.get_default_download_dir()" 494 | ], 495 | "execution_count": 18, 496 | "outputs": [ 497 | { 498 | "output_type": "execute_result", 499 | "data": { 500 | "text/plain": [ 501 | "'/content'" 502 | ] 503 | }, 504 | "metadata": { 505 | "tags": [] 506 | }, 507 | "execution_count": 18 508 | } 509 | ] 510 | }, 511 | { 512 | "cell_type": "markdown", 513 | "metadata": { 514 | "id": "rVjuPvZF_JjI", 515 | "colab_type": "text" 516 | }, 517 | "source": [ 518 | "### Search for datasets by keywords\n", 519 | "\n" 520 | ] 521 | }, 522 | { 523 | "cell_type": "code", 524 | "metadata": { 525 | "id": "0ZagL64azLu0", 526 | "colab_type": "code", 527 | "colab": {} 528 | }, 529 | "source": [ 530 | "dsets = api.dataset_list(search='fingers')" 531 | ], 532 | "execution_count": 0, 533 | "outputs": [] 534 | }, 535 | { 536 | "cell_type": "code", 537 | "metadata": { 538 | "id": "JlyixoT6zLu4", 539 | "colab_type": "code", 540 | "outputId": "cae82799-9711-4b45-c37e-c426932ab264", 541 | "colab": { 542 | "base_uri": "https://localhost:8080/", 543 | "height": 34 544 | } 545 | }, 546 | "source": [ 547 | "len(dsets)" 548 | ], 549 | "execution_count": 20, 550 | "outputs": [ 551 | { 552 | "output_type": "execute_result", 553 | "data": { 554 | "text/plain": [ 555 | "13" 556 | ] 557 | }, 558 | "metadata": { 559 | "tags": [] 560 | }, 561 | "execution_count": 20 562 | } 563 | ] 564 | }, 565 | { 566 | "cell_type": "code", 567 | "metadata": { 568 | "id": "rTP65iLPzLu8", 569 | "colab_type": "code", 570 | "outputId": "45b4fcc6-a0b8-4474-cd64-efa822d5a49e", 571 | "colab": { 572 | "base_uri": "https://localhost:8080/", 573 | "height": 238 574 | } 575 | }, 576 | "source": [ 577 | "dsets" 578 | ], 579 | "execution_count": 21, 580 | "outputs": [ 581 | { 582 | "output_type": "execute_result", 583 | "data": { 584 | "text/plain": [ 585 | "[koryakinp/fingers,\n", 586 | " mrgeislinger/asl-rgb-depth-fingerspelling-spelling-it-out,\n", 587 | " feronial/turkish-sign-languagefinger-spelling,\n", 588 | " mahendran1/fingers,\n", 589 | " ardamavi/sign-language-digits-dataset,\n", 590 | " kyr7plus/emg-4,\n", 591 | " lgmoneda/br-coins,\n", 592 | " nagarjunk/hand-finger-dataset,\n", 593 | " gti-upm/multimodhandgestrec,\n", 594 | " niksanjp/number-of-fingers,\n", 595 | " roshea6/finger-digits-05,\n", 596 | " suresrkumar/hand-sign-language-dataset-with-annotation-in-json,\n", 597 | " cristeaioan/ffml-dataset]" 598 | ] 599 | }, 600 | "metadata": { 601 | "tags": [] 602 | }, 603 | "execution_count": 21 604 | } 605 | ] 606 | }, 607 | { 608 | "cell_type": "markdown", 609 | "metadata": { 610 | "id": "s_hHOU_RzLvA", 611 | "colab_type": "text" 612 | }, 613 | "source": [ 614 | "Based on visual inspection, get candidates" 615 | ] 616 | }, 617 | { 618 | "cell_type": "code", 619 | "metadata": { 620 | "id": "o7XwI2Fw_XAE", 621 | "colab_type": "code", 622 | "colab": {} 623 | }, 624 | "source": [ 625 | "\n", 626 | "candidate_dsets = ['koryakinp/fingers'\n", 627 | " ,'mahendran1/fingers'\n", 628 | " ,'roshea6/finger-digits-05',\n", 629 | " ]\n", 630 | "github_dset = ['https://github.com/ardamavi/Sign-Language-Digits-Dataset']\n" 631 | ], 632 | "execution_count": 0, 633 | "outputs": [] 634 | }, 635 | { 636 | "cell_type": "markdown", 637 | "metadata": { 638 | "id": "bwgvXqWwzLvF", 639 | "colab_type": "text" 640 | }, 641 | "source": [ 642 | "### Fetch and prep dataset 3" 643 | ] 644 | }, 645 | { 646 | "cell_type": "code", 647 | "metadata": { 648 | "id": "nWEpL_EnzLvM", 649 | "colab_type": "code", 650 | "colab": {} 651 | }, 652 | "source": [ 653 | "dset_arch = kaggle_download(candidate_dsets[2],path=archive_path)" 654 | ], 655 | "execution_count": 0, 656 | "outputs": [] 657 | }, 658 | { 659 | "cell_type": "code", 660 | "metadata": { 661 | "id": "68IcgxIHzLvV", 662 | "colab_type": "code", 663 | "outputId": "ea387f44-fd7d-4c52-a857-ca775fe8a1e3", 664 | "colab": { 665 | "base_uri": "https://localhost:8080/", 666 | "height": 52 667 | } 668 | }, 669 | "source": [ 670 | "# explore downloaded dset archive\n", 671 | "(dset_arch,dset_arch.ls())" 672 | ], 673 | "execution_count": 0, 674 | "outputs": [ 675 | { 676 | "output_type": "execute_result", 677 | "data": { 678 | "text/plain": [ 679 | "(Path('/root/.fastai/archive/roshea6_finger-digits-05'),\n", 680 | " (#1) [Path('/root/.fastai/archive/roshea6_finger-digits-05/finger-digits-05.zip')])" 681 | ] 682 | }, 683 | "metadata": { 684 | "tags": [] 685 | }, 686 | "execution_count": 23 687 | } 688 | ] 689 | }, 690 | { 691 | "cell_type": "code", 692 | "metadata": { 693 | "id": "x_2YchHtzLvZ", 694 | "colab_type": "code", 695 | "outputId": "2bff2a67-07c9-48b5-8374-3e906a614b5f", 696 | "colab": { 697 | "base_uri": "https://localhost:8080/", 698 | "height": 34 699 | } 700 | }, 701 | "source": [ 702 | "dset = Path(archive_path)/(dset_arch.name + '.zip');dset" 703 | ], 704 | "execution_count": 0, 705 | "outputs": [ 706 | { 707 | "output_type": "execute_result", 708 | "data": { 709 | "text/plain": [ 710 | "Path('/root/.fastai/archive/roshea6_finger-digits-05.zip')" 711 | ] 712 | }, 713 | "metadata": { 714 | "tags": [] 715 | }, 716 | "execution_count": 24 717 | } 718 | ] 719 | }, 720 | { 721 | "cell_type": "code", 722 | "metadata": { 723 | "id": "cAGhCjonzLve", 724 | "colab_type": "code", 725 | "colab": {} 726 | }, 727 | "source": [ 728 | "# cleanup dset and move to archive dir\n", 729 | "!mv {(dset_arch/'finger-digits-05.zip').as_posix()} {dset.as_posix()} " 730 | ], 731 | "execution_count": 0, 732 | "outputs": [] 733 | }, 734 | { 735 | "cell_type": "code", 736 | "metadata": { 737 | "id": "UUUjTjmlzLvi", 738 | "colab_type": "code", 739 | "colab": {} 740 | }, 741 | "source": [ 742 | "!rm -rf {dset_arch.as_posix()}" 743 | ], 744 | "execution_count": 0, 745 | "outputs": [] 746 | }, 747 | { 748 | "cell_type": "code", 749 | "metadata": { 750 | "id": "S5kWeevozLvm", 751 | "colab_type": "code", 752 | "outputId": "355ec102-3600-4c8e-d297-22e9e88b06ab", 753 | "colab": { 754 | "base_uri": "https://localhost:8080/", 755 | "height": 34 756 | } 757 | }, 758 | "source": [ 759 | "!ls -aldh {dset.as_posix()}" 760 | ], 761 | "execution_count": 0, 762 | "outputs": [ 763 | { 764 | "output_type": "stream", 765 | "text": [ 766 | "-rw-r--r-- 1 root root 19M Apr 9 07:34 /root/.fastai/archive/roshea6_finger-digits-05.zip\n" 767 | ], 768 | "name": "stdout" 769 | } 770 | ] 771 | }, 772 | { 773 | "cell_type": "code", 774 | "metadata": { 775 | "id": "c1eV_teizLvp", 776 | "colab_type": "code", 777 | "colab": {} 778 | }, 779 | "source": [ 780 | "!cp {dset.as_posix()} {escdrive(gdrive/'data'/dset.name)}" 781 | ], 782 | "execution_count": 0, 783 | "outputs": [] 784 | }, 785 | { 786 | "cell_type": "code", 787 | "metadata": { 788 | "id": "Rwj8XZ1nzLvu", 789 | "colab_type": "code", 790 | "outputId": "e04cf269-5a33-407b-f509-231800f82a6d", 791 | "colab": { 792 | "base_uri": "https://localhost:8080/", 793 | "height": 54 794 | } 795 | }, 796 | "source": [ 797 | "!ls -aldh {escdrive(gdrive/'data'/dset.name)}" 798 | ], 799 | "execution_count": 0, 800 | "outputs": [ 801 | { 802 | "output_type": "stream", 803 | "text": [ 804 | "-rw------- 1 root root 19M Apr 9 07:36 '/content/drive/My Drive/fastai_v4/data/roshea6_finger-digits-05.zip'\n" 805 | ], 806 | "name": "stdout" 807 | } 808 | ] 809 | }, 810 | { 811 | "cell_type": "code", 812 | "metadata": { 813 | "id": "cCB9xLJAzLvy", 814 | "colab_type": "code", 815 | "colab": {} 816 | }, 817 | "source": [ 818 | "" 819 | ], 820 | "execution_count": 0, 821 | "outputs": [] 822 | }, 823 | { 824 | "cell_type": "markdown", 825 | "metadata": { 826 | "id": "voywUfYV4QXA", 827 | "colab_type": "text" 828 | }, 829 | "source": [ 830 | "### Fetch and prep dataset 2" 831 | ] 832 | }, 833 | { 834 | "cell_type": "code", 835 | "metadata": { 836 | "colab_type": "code", 837 | "id": "lqnWMhnN4gA4", 838 | "colab": {} 839 | }, 840 | "source": [ 841 | "dset_arch = kaggle_download(candidate_dsets[1],path=archive_path)" 842 | ], 843 | "execution_count": 0, 844 | "outputs": [] 845 | }, 846 | { 847 | "cell_type": "code", 848 | "metadata": { 849 | "colab_type": "code", 850 | "outputId": "b477e2fb-2a2a-44a5-acad-76c8dcb28d6c", 851 | "id": "T-XVLawJ4QXf", 852 | "colab": { 853 | "base_uri": "https://localhost:8080/", 854 | "height": 52 855 | } 856 | }, 857 | "source": [ 858 | "# explore downloaded dset archive\n", 859 | "(dset_arch,dset_arch.ls())" 860 | ], 861 | "execution_count": 0, 862 | "outputs": [ 863 | { 864 | "output_type": "execute_result", 865 | "data": { 866 | "text/plain": [ 867 | "(Path('/root/.fastai/archive/mahendran1_fingers'),\n", 868 | " (#1) [Path('/root/.fastai/archive/mahendran1_fingers/fingers.zip')])" 869 | ] 870 | }, 871 | "metadata": { 872 | "tags": [] 873 | }, 874 | "execution_count": 31 875 | } 876 | ] 877 | }, 878 | { 879 | "cell_type": "code", 880 | "metadata": { 881 | "colab_type": "code", 882 | "outputId": "5fce40e9-8cd4-40b1-f32a-304b80ae74d8", 883 | "id": "nj6EbpuU4QXo", 884 | "colab": { 885 | "base_uri": "https://localhost:8080/", 886 | "height": 34 887 | } 888 | }, 889 | "source": [ 890 | "dset = Path(archive_path)/(dset_arch.name + '.zip');dset" 891 | ], 892 | "execution_count": 0, 893 | "outputs": [ 894 | { 895 | "output_type": "execute_result", 896 | "data": { 897 | "text/plain": [ 898 | "Path('/root/.fastai/archive/mahendran1_fingers.zip')" 899 | ] 900 | }, 901 | "metadata": { 902 | "tags": [] 903 | }, 904 | "execution_count": 32 905 | } 906 | ] 907 | }, 908 | { 909 | "cell_type": "code", 910 | "metadata": { 911 | "colab_type": "code", 912 | "id": "PrS1gsRp4QXv", 913 | "colab": {} 914 | }, 915 | "source": [ 916 | "# cleanup dset and move to archive dir\n", 917 | "!mv {(dset_arch/'fingers.zip').as_posix()} {dset.as_posix()} " 918 | ], 919 | "execution_count": 0, 920 | "outputs": [] 921 | }, 922 | { 923 | "cell_type": "code", 924 | "metadata": { 925 | "colab_type": "code", 926 | "id": "Ctg2pqlE4QX1", 927 | "colab": {} 928 | }, 929 | "source": [ 930 | "!rm -rf {dset_arch.as_posix()}" 931 | ], 932 | "execution_count": 0, 933 | "outputs": [] 934 | }, 935 | { 936 | "cell_type": "code", 937 | "metadata": { 938 | "colab_type": "code", 939 | "outputId": "b2a6ff41-8468-421b-9451-da8c0004aae9", 940 | "id": "x_ceNjL34QX7", 941 | "colab": { 942 | "base_uri": "https://localhost:8080/", 943 | "height": 34 944 | } 945 | }, 946 | "source": [ 947 | "!ls -aldh {dset.as_posix()}" 948 | ], 949 | "execution_count": 0, 950 | "outputs": [ 951 | { 952 | "output_type": "stream", 953 | "text": [ 954 | "-rw-r--r-- 1 root root 19M Apr 9 07:44 /root/.fastai/archive/mahendran1_fingers.zip\n" 955 | ], 956 | "name": "stdout" 957 | } 958 | ] 959 | }, 960 | { 961 | "cell_type": "code", 962 | "metadata": { 963 | "colab_type": "code", 964 | "id": "hXz6VZa54QYD", 965 | "colab": {} 966 | }, 967 | "source": [ 968 | "!cp {dset.as_posix()} {escdrive(gdrive/'data'/dset.name)}" 969 | ], 970 | "execution_count": 0, 971 | "outputs": [] 972 | }, 973 | { 974 | "cell_type": "code", 975 | "metadata": { 976 | "colab_type": "code", 977 | "outputId": "1d1ca751-cacb-478b-f918-ade2a6316ad6", 978 | "id": "8feKeTX34QYI", 979 | "colab": { 980 | "base_uri": "https://localhost:8080/", 981 | "height": 54 982 | } 983 | }, 984 | "source": [ 985 | "!ls -aldh {escdrive(gdrive/'data'/dset.name)}" 986 | ], 987 | "execution_count": 0, 988 | "outputs": [ 989 | { 990 | "output_type": "stream", 991 | "text": [ 992 | "-rw------- 1 root root 19M Apr 9 07:46 '/content/drive/My Drive/fastai_v4/data/mahendran1_fingers.zip'\n" 993 | ], 994 | "name": "stdout" 995 | } 996 | ] 997 | }, 998 | { 999 | "cell_type": "code", 1000 | "metadata": { 1001 | "id": "mwOtC-ar50hA", 1002 | "colab_type": "code", 1003 | "colab": {} 1004 | }, 1005 | "source": [ 1006 | "" 1007 | ], 1008 | "execution_count": 0, 1009 | "outputs": [] 1010 | }, 1011 | { 1012 | "cell_type": "markdown", 1013 | "metadata": { 1014 | "id": "7RmegeqD6Dz5", 1015 | "colab_type": "text" 1016 | }, 1017 | "source": [ 1018 | "### Fetch and prep dataset 1" 1019 | ] 1020 | }, 1021 | { 1022 | "cell_type": "code", 1023 | "metadata": { 1024 | "colab_type": "code", 1025 | "id": "2vF4_aOG6D0M", 1026 | "colab": {} 1027 | }, 1028 | "source": [ 1029 | "dset_arch = kaggle_download(candidate_dsets[0],path=archive_path)" 1030 | ], 1031 | "execution_count": 0, 1032 | "outputs": [] 1033 | }, 1034 | { 1035 | "cell_type": "code", 1036 | "metadata": { 1037 | "colab_type": "code", 1038 | "outputId": "31fb4eda-5dab-438a-ff08-5e2235f5b8c8", 1039 | "id": "AUlSE35e6D0V", 1040 | "colab": { 1041 | "base_uri": "https://localhost:8080/", 1042 | "height": 52 1043 | } 1044 | }, 1045 | "source": [ 1046 | "# explore downloaded dset archive\n", 1047 | "(dset_arch,dset_arch.ls())" 1048 | ], 1049 | "execution_count": 0, 1050 | "outputs": [ 1051 | { 1052 | "output_type": "execute_result", 1053 | "data": { 1054 | "text/plain": [ 1055 | "(Path('/root/.fastai/archive/koryakinp_fingers'),\n", 1056 | " (#1) [Path('/root/.fastai/archive/koryakinp_fingers/fingers.zip')])" 1057 | ] 1058 | }, 1059 | "metadata": { 1060 | "tags": [] 1061 | }, 1062 | "execution_count": 39 1063 | } 1064 | ] 1065 | }, 1066 | { 1067 | "cell_type": "code", 1068 | "metadata": { 1069 | "colab_type": "code", 1070 | "outputId": "fbd28f41-b67c-4b43-e6d8-d98864672985", 1071 | "id": "wNyh-U5X6D0d", 1072 | "colab": { 1073 | "base_uri": "https://localhost:8080/", 1074 | "height": 34 1075 | } 1076 | }, 1077 | "source": [ 1078 | "dset = Path(archive_path)/(dset_arch.name + '.zip');dset" 1079 | ], 1080 | "execution_count": 0, 1081 | "outputs": [ 1082 | { 1083 | "output_type": "execute_result", 1084 | "data": { 1085 | "text/plain": [ 1086 | "Path('/root/.fastai/archive/koryakinp_fingers.zip')" 1087 | ] 1088 | }, 1089 | "metadata": { 1090 | "tags": [] 1091 | }, 1092 | "execution_count": 40 1093 | } 1094 | ] 1095 | }, 1096 | { 1097 | "cell_type": "code", 1098 | "metadata": { 1099 | "colab_type": "code", 1100 | "id": "9KCAgy2F6D0j", 1101 | "colab": {} 1102 | }, 1103 | "source": [ 1104 | "# cleanup dset and move to archive dir\n", 1105 | "!mv {(dset_arch/'fingers.zip').as_posix()} {dset.as_posix()} " 1106 | ], 1107 | "execution_count": 0, 1108 | "outputs": [] 1109 | }, 1110 | { 1111 | "cell_type": "code", 1112 | "metadata": { 1113 | "colab_type": "code", 1114 | "id": "D8GFBGSJ6D0q", 1115 | "colab": {} 1116 | }, 1117 | "source": [ 1118 | "!rm -rf {dset_arch.as_posix()}" 1119 | ], 1120 | "execution_count": 0, 1121 | "outputs": [] 1122 | }, 1123 | { 1124 | "cell_type": "code", 1125 | "metadata": { 1126 | "colab_type": "code", 1127 | "outputId": "abce61a9-9a30-4c85-e626-6b0832d08b1b", 1128 | "id": "Mgda4Ghk6D0v", 1129 | "colab": { 1130 | "base_uri": "https://localhost:8080/", 1131 | "height": 34 1132 | } 1133 | }, 1134 | "source": [ 1135 | "!ls -aldh {dset.as_posix()}" 1136 | ], 1137 | "execution_count": 0, 1138 | "outputs": [ 1139 | { 1140 | "output_type": "stream", 1141 | "text": [ 1142 | "-rw-r--r-- 1 root root 364M Apr 9 07:48 /root/.fastai/archive/koryakinp_fingers.zip\n" 1143 | ], 1144 | "name": "stdout" 1145 | } 1146 | ] 1147 | }, 1148 | { 1149 | "cell_type": "code", 1150 | "metadata": { 1151 | "colab_type": "code", 1152 | "id": "g4SHXTT16D01", 1153 | "colab": {} 1154 | }, 1155 | "source": [ 1156 | "!cp {dset.as_posix()} {escdrive(gdrive/'data'/dset.name)}" 1157 | ], 1158 | "execution_count": 0, 1159 | "outputs": [] 1160 | }, 1161 | { 1162 | "cell_type": "code", 1163 | "metadata": { 1164 | "colab_type": "code", 1165 | "outputId": "857307ca-f43f-48db-8a5f-c368d2f0e923", 1166 | "id": "CNHipVUs6D06", 1167 | "colab": { 1168 | "base_uri": "https://localhost:8080/", 1169 | "height": 54 1170 | } 1171 | }, 1172 | "source": [ 1173 | "!ls -aldh {escdrive(gdrive/'data'/dset.name)}" 1174 | ], 1175 | "execution_count": 0, 1176 | "outputs": [ 1177 | { 1178 | "output_type": "stream", 1179 | "text": [ 1180 | "-rw------- 1 root root 364M Apr 9 07:49 '/content/drive/My Drive/fastai_v4/data/koryakinp_fingers.zip'\n" 1181 | ], 1182 | "name": "stdout" 1183 | } 1184 | ] 1185 | }, 1186 | { 1187 | "cell_type": "markdown", 1188 | "metadata": { 1189 | "id": "66v6ryku6nrs", 1190 | "colab_type": "text" 1191 | }, 1192 | "source": [ 1193 | "### download github dataset" 1194 | ] 1195 | }, 1196 | { 1197 | "cell_type": "code", 1198 | "metadata": { 1199 | "id": "CGtebMjR_ssO", 1200 | "colab_type": "code", 1201 | "colab": {} 1202 | }, 1203 | "source": [ 1204 | "git_url = github_dset[0] + '.git'" 1205 | ], 1206 | "execution_count": 0, 1207 | "outputs": [] 1208 | }, 1209 | { 1210 | "cell_type": "code", 1211 | "metadata": { 1212 | "id": "rUOwm6GbACX9", 1213 | "colab_type": "code", 1214 | "colab": {} 1215 | }, 1216 | "source": [ 1217 | "dset = 'ardamavi/Sign-Language-Digits-Dataset'.replace('/','_')" 1218 | ], 1219 | "execution_count": 0, 1220 | "outputs": [] 1221 | }, 1222 | { 1223 | "cell_type": "code", 1224 | "metadata": { 1225 | "id": "5ZXcn0Gc7O8I", 1226 | "colab_type": "code", 1227 | "outputId": "c0c325f6-c210-400f-c642-9ebccc471f55", 1228 | "colab": { 1229 | "base_uri": "https://localhost:8080/", 1230 | "height": 104 1231 | } 1232 | }, 1233 | "source": [ 1234 | "!git clone {git_url}" 1235 | ], 1236 | "execution_count": 0, 1237 | "outputs": [ 1238 | { 1239 | "output_type": "stream", 1240 | "text": [ 1241 | "Cloning into 'Sign-Language-Digits-Dataset'...\n", 1242 | "remote: Enumerating objects: 2089, done.\u001b[K\n", 1243 | "remote: Total 2089 (delta 0), reused 0 (delta 0), pack-reused 2089\n", 1244 | "Receiving objects: 100% (2089/2089), 15.07 MiB | 6.28 MiB/s, done.\n", 1245 | "Resolving deltas: 100% (658/658), done.\n" 1246 | ], 1247 | "name": "stdout" 1248 | } 1249 | ] 1250 | }, 1251 | { 1252 | "cell_type": "code", 1253 | "metadata": { 1254 | "id": "zLglc-Y-87TA", 1255 | "colab_type": "code", 1256 | "colab": {} 1257 | }, 1258 | "source": [ 1259 | "!rm -rf Sign-Language-Digits-Dataset/.git\n" 1260 | ], 1261 | "execution_count": 0, 1262 | "outputs": [] 1263 | }, 1264 | { 1265 | "cell_type": "code", 1266 | "metadata": { 1267 | "id": "Ohu-AqMk75Yr", 1268 | "colab_type": "code", 1269 | "colab": {} 1270 | }, 1271 | "source": [ 1272 | "!tar cvzf ardamavi_Sign-Language-Digits-Dataset2.tgz Sign-Language-Digits-Dataset -C Sign-Language-Digits-Dataset" 1273 | ], 1274 | "execution_count": 0, 1275 | "outputs": [] 1276 | }, 1277 | { 1278 | "cell_type": "code", 1279 | "metadata": { 1280 | "id": "bs6wFhME8px1", 1281 | "colab_type": "code", 1282 | "colab": {} 1283 | }, 1284 | "source": [ 1285 | "!cp ardamavi_Sign-Language-Digits-Dataset.tgz {escdrive(gdrive/'data')}" 1286 | ], 1287 | "execution_count": 0, 1288 | "outputs": [] 1289 | }, 1290 | { 1291 | "cell_type": "code", 1292 | "metadata": { 1293 | "id": "SS25rmUhBiZV", 1294 | "colab_type": "code", 1295 | "colab": {} 1296 | }, 1297 | "source": [ 1298 | "" 1299 | ], 1300 | "execution_count": 0, 1301 | "outputs": [] 1302 | } 1303 | ] 1304 | } -------------------------------------------------------------------------------- /play-with-equations.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Notes on The Matrix Calculus You Need for Deep Learning\n", 8 | "\n", 9 | "These are my notes from the article [\"The Matrix Calculus you need for Deep Learning\"](https://explained.ai/matrix-calculus/index.html) by Terence Parr and Jeremy Howard \n", 10 | "as part of a fastai study group being conducted May-Oct 2020 (ongoing).\n", 11 | "\n", 12 | "_(Status: unfinished)_\n", 13 | "\n", 14 | "#### Affine function for a Neuron\n", 15 | "\n", 16 | "$$ \n", 17 | "z(x) = \\sum \\limits _{i} ^{n} w_{i}x_{i} + b = w \\cdot x + b \n", 18 | "$$" 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "#### Common loss function\n", 26 | "$$\n", 27 | "\\frac {1}{N} \\sum \\limits _{x} (target(x) - activation(x))^{2} = \n", 28 | "\\frac {1}{N} \\sum \\limits _{x} (target(x) - max(0, \\sum \\limits _{i}^{\\vert x \\vert} w_{i}x_{i} + b))^2 \n", 29 | "$$" 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "#### Table of Derivatives\n", 37 | "\n", 38 | "![basic derivative chain rules](https://raw.githubusercontent.com/butchland/fastai_nb_explorations/master/images/basic-derivative-chain-rules.png)\n" 39 | ] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "metadata": {}, 44 | "source": [ 45 | "#### Partial Derivative Example\n", 46 | "\n", 47 | "Consider the function $ f(x,y) = 3x^{2}y $\n", 48 | "\n", 49 | "The derivative of _f_ wrt to _x_ is:\n", 50 | "\n", 51 | "$\\frac{\\partial f}{\\partial x} = $\n", 52 | "$\\frac{\\partial} {\\partial x} 3x^{2}y = $\n", 53 | "$3y\\frac{\\partial}{\\partial x}x^{2} = $\n", 54 | "$ 6yx $\n", 55 | "\n", 56 | "The derivative of _f_ wrt to _y_ is:\n", 57 | "\n", 58 | "$\\frac{\\partial f}{\\partial y} = $\n", 59 | "$\\frac{\\partial} {\\partial y} 3x^{2}y = $\n", 60 | "$3x^{2}\\frac{\\partial}{\\partial y}y = $\n", 61 | "$ 3x^{2} $\n" 62 | ] 63 | }, 64 | { 65 | "cell_type": "markdown", 66 | "metadata": {}, 67 | "source": [ 68 | "#### Gradient of $ f(x,y) $\n", 69 | "\n", 70 | "$ \\nabla f(x,y) = [\\frac{\\partial f(x,y)}{\\partial x},\\frac{\\partial f(x,y)}{\\partial y}] = $\n", 71 | "$ [6yx, 3x^{2}] $\n" 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "### Matrix calculus\n", 79 | "\n", 80 | "Given the functions $ f(x,y) = 3x^{2}y $ and \n", 81 | "$ g(x,y) = 2x + y^{8} $\n", 82 | "\n", 83 | "\n", 84 | "$ \\frac{\\partial g(x,y)}{\\partial x} = $\n", 85 | "$ \\frac{\\partial 2x}{\\partial x} + \\frac{\\partial y^{8}}{\\partial x} = $\n", 86 | "$ 2\\frac{\\partial x}{\\partial x} + 0 = $\n", 87 | "$ 2 \\times 1 = 2 $\n", 88 | "\n", 89 | "and\n", 90 | "\n", 91 | "$ \\frac{\\partial g(x,y)}{\\partial y} = $\n", 92 | "$ \\frac{\\partial 2x}{\\partial y} + \\frac{\\partial y^{8}}{\\partial y} = $\n", 93 | "$ 0 + 8y^{7} = $\n", 94 | "$ 8y^{7} $\n", 95 | "\n", 96 | "giving us the gradient of $ g(x,y) $ as\n", 97 | "\n", 98 | "$ \\nabla g(x,y) = $\n", 99 | "$ [2,8y^{7}] $" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": {}, 105 | "source": [ 106 | "### Jacobian matrix (aka Jacobian)\n", 107 | "\n", 108 | "**_numerator layout_(used here)** :\n", 109 | "* rows: _equations(f,g)_ , \n", 110 | "* columns: _variables(x,y)_ \n", 111 | "\n", 112 | "\n", 113 | "$$\n", 114 | "J = \n", 115 | "\\begin{bmatrix}\n", 116 | "\\nabla f(x,y) \\\\\n", 117 | "\\nabla g(x,y)\n", 118 | "\\end{bmatrix} =\n", 119 | "\\begin{bmatrix}\n", 120 | "6yx & 3x^{2} \\\\\n", 121 | "2 & 8y^{7}\n", 122 | "\\end{bmatrix} \n", 123 | "$$\n", 124 | "\n", 125 | "_denominator layout_ - rows: _variables(x,y)_ , columns: _equations(f,g)_\n", 126 | "\n", 127 | "$$\n", 128 | "\\begin{bmatrix}\n", 129 | "6yx & 2 \\\\\n", 130 | "3x^{2} & 8y^{7}\n", 131 | "\\end{bmatrix} \n", 132 | "$$\n" 133 | ] 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "metadata": {}, 138 | "source": [ 139 | "### Generalization of the Jacobian\n", 140 | "\n", 141 | "Combine $ f(x,y,z) \\Rightarrow f(x) $ where **`x`** is a vector (aka $ \\vec{x} $) and _x_ are scalars. e.g. $ x_{i} $ is the $i^{th}$ element of vector **x**. \n", 142 | "\n", 143 | "Assume vector **x** is a column vector (vertical vector) by default of size $ n \\times 1 $. \n", 144 | "\n", 145 | "$$\n", 146 | "x = \n", 147 | "\\begin{bmatrix}\n", 148 | "x_{1} \\\\\n", 149 | "x_{2} \\\\\n", 150 | "\\vdots \\\\\n", 151 | "x_{n}\n", 152 | "\\end{bmatrix} \n", 153 | "$$" 154 | ] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": {}, 159 | "source": [ 160 | "For multiple scalar-valued functions, combine all into a vector just like the parameters. \n", 161 | "\n", 162 | "Let $ y = f(x) $ be a vector of `m` scalar-valued functions that each take a vector **x** of length $ n = \\vert x \\vert $ where $ \\vert x \\vert $ is the count of elements in **x**.\n", 163 | "\n", 164 | "Each $ f_{i} $ function within **f** returns a scalar.\n", 165 | "\n", 166 | "$$\n", 167 | "\\begin{matrix}\n", 168 | "y_{1} = f_{1}(x) \\\\\n", 169 | "y_{2} = f_{2}(x) \\\\\n", 170 | "\\vdots \\\\\n", 171 | "y_{m} = f_{m}(x)\n", 172 | "\\end{matrix} \n", 173 | "$$\n", 174 | "\n", 175 | "For instance, given $ f(x,y) = 3x^{2}y $ and $ g(x,y) = 2x + y^{8} $,\n", 176 | "then \n", 177 | "\n", 178 | "$ y_{1} = f_{1}(x) = 3x_{1}^{2}x_{2} $ (substituting $ x_{1} $ for _x_, $ x_{2} $ for _y_ )\n", 179 | "\n", 180 | "$ y_{2} = f_{2}(x) = 2x_{1} + x_{2}^{8} $\n", 181 | "\n", 182 | "\n" 183 | ] 184 | }, 185 | { 186 | "cell_type": "markdown", 187 | "metadata": {}, 188 | "source": [ 189 | "For the identity function $ y = f(x) = x $ it will be the case that $ m = n $ :\n", 190 | "$$\n", 191 | "\\begin{matrix}\n", 192 | "y_{1} = f_{1}(x) = x_{1} \\\\\n", 193 | "y_{2} = f_{2}(x) = x_{2} \\\\\n", 194 | "\\vdots \\\\\n", 195 | "y_{m} = f_{m}(x) = x_{n}\n", 196 | "\\end{matrix} \n", 197 | "$$\n", 198 | "\n", 199 | "So for the identity function, we will $ m = n $ functions and parameters.\n", 200 | "\n", 201 | "Generally, the Jacobian matrix is the collection of all $ m \\times n $ possible partial derivatives (_m_ rows and _n_ columns), which is a stack of _m_ gradients with respect to **x**:\n", 202 | "\n", 203 | "$$\n", 204 | "\\frac {\\partial y}{\\partial x} \n", 205 | "=\n", 206 | "\\begin{bmatrix}\n", 207 | "\\nabla f_{1}(x) \\\\\n", 208 | "\\nabla f_{2}(x) \\\\\n", 209 | "\\dotsb \\\\\n", 210 | "\\nabla f_{m}(x)\n", 211 | "\\end{bmatrix} \n", 212 | "=\n", 213 | "\\begin{bmatrix}\n", 214 | "\\frac{\\partial}{\\partial x}f_{1}(x) \\\\\n", 215 | "\\frac{\\partial}{\\partial x}f_{2}(x) \\\\\n", 216 | "\\dotsb \\\\\n", 217 | "\\frac{\\partial}{\\partial x}f_{m}(x) \\\\\n", 218 | "\\end{bmatrix} \n", 219 | "=\n", 220 | "\\begin{bmatrix}\n", 221 | "\\frac{\\partial}{\\partial x_{1}}f_{1}(x) & \\frac{\\partial}{\\partial x_{2}}f_{1}(x) & \\dotsb & \\frac{\\partial}{\\partial x_{n}}f_{1}(x) \\\\\n", 222 | "\\frac{\\partial}{\\partial x_{1}}f_{2}(x) & \\frac{\\partial}{\\partial x_{2}}f_{2}(x) & \\dotsb & \\frac{\\partial}{\\partial x_{n}}f_{2}(x) \\\\\n", 223 | "\\dotsb \\\\\n", 224 | "\\frac{\\partial}{\\partial x_{1}}f_{m}(x) & \\frac{\\partial}{\\partial x_{2}}f_{m}(x) & \\dotsb & \\frac{\\partial}{\\partial x_{n}}f_{m}(x) \\\\\n", 225 | "\\end{bmatrix} \n", 226 | "$$\n" 227 | ] 228 | }, 229 | { 230 | "cell_type": "markdown", 231 | "metadata": {}, 232 | "source": [ 233 | "Each $ \\frac{\\partial}{\\partial x}f_{i}(x) $ is a horizontal _n_-vector b/c the partial derivative wrt to the vector **x**, whose length $ n = \\vert x \\vert $. The _width_ of the Jacobian is _n_ if we take the partial derivative with respect to **x** because there are _n_ parameters we can wiggle, each potentially changing the function's value. Therefore, the Jacobian is always _m_ rows for _m_ equations.\n", 234 | "\n", 235 | "##### Jacobian Shapes \n", 236 | "![jacobian shapes](https://raw.githubusercontent.com/butchland/fastai_nb_explorations/master/images/jacobian_shapes.png)" 237 | ] 238 | }, 239 | { 240 | "cell_type": "markdown", 241 | "metadata": {}, 242 | "source": [ 243 | "#### An Example: Jacobian of the identity function\n", 244 | "Given the identity function $ \\pmb{f}(x) = \\pmb{x} $, with $ f_{i}(x) = x_{i} $, the Jacobian of the identity function has _n_ functions and each function has _n_ parameters held in a single vector **x**. The Jacobian is, therefore, a square matrix since $ m = n $:\n", 245 | "\n", 246 | "$$\n", 247 | "\\frac{\\partial y}{\\partial x} \n", 248 | "=\n", 249 | "\\begin{bmatrix}\n", 250 | "\\frac{\\partial}{\\partial x}f_{1}(x) \\\\\n", 251 | "\\frac{\\partial}{\\partial x}f_{2}(x) \\\\\n", 252 | "\\dotsb \\\\\n", 253 | "\\frac{\\partial}{\\partial x}f_{m}(x) \\\\\n", 254 | "\\end{bmatrix} \n", 255 | "=\n", 256 | "\\begin{bmatrix}\n", 257 | "\\frac{\\partial}{\\partial x_{1}}f_{1}(x) & \\frac{\\partial}{\\partial x_{2}}f_{1}(x) & \\dotsb & \\frac{\\partial}{\\partial x_{n}}f_{1}(x) \\\\\n", 258 | "\\frac{\\partial}{\\partial x_{1}}f_{2}(x) & \\frac{\\partial}{\\partial x_{2}}f_{2}(x) & \\dotsb & \\frac{\\partial}{\\partial x_{n}}f_{2}(x) \\\\\n", 259 | "\\dotsb \\\\\n", 260 | "\\frac{\\partial}{\\partial x_{1}}f_{m}(x) & \\frac{\\partial}{\\partial x_{2}}f_{m}(x) & \\dotsb & \\frac{\\partial}{\\partial x_{n}}f_{m}(x) \\\\\n", 261 | "\\end{bmatrix} \n", 262 | "=\n", 263 | "\\begin{bmatrix}\n", 264 | "\\frac{\\partial}{\\partial x_{1}}x_{1} & \\frac{\\partial}{\\partial x_{2}}x_{1} & \\dotsb & \\frac{\\partial}{\\partial x_{n}}x_{1} \\\\\n", 265 | "\\frac{\\partial}{\\partial x_{1}}x_{2} & \\frac{\\partial}{\\partial x_{2}}x_{2} & \\dotsb & \\frac{\\partial}{\\partial x_{n}}x_{2} \\\\\n", 266 | "\\dotsb \\\\\n", 267 | "\\frac{\\partial}{\\partial x_{1}}x_{n} & \\frac{\\partial}{\\partial x_{2}}x_{n} & \\dotsb & \\frac{\\partial}{\\partial x_{n}}x_{n} \\\\\n", 268 | "\\end{bmatrix}\n", 269 | "$$\n", 270 | "\n", 271 | "And since $ \\frac{\\partial}{\\partial x_{j}}x_{i} = 0 $ for $ j \\ne i $ and $ \\frac{\\partial}{\\partial x_{j}}x_{i} = 1 $ for $ j = i $\n", 272 | "\n", 273 | "$$\n", 274 | "=\n", 275 | "\\begin{bmatrix}\n", 276 | "\\frac{\\partial}{\\partial x_{1}}x_{1} & 0 & \\dotsb & 0 \\\\\n", 277 | "0 & \\frac{\\partial}{\\partial x_{2}}x_{2} & \\dotsb & 0 \\\\\n", 278 | " & & \\ddots \\\\\n", 279 | "0 & 0 & \\dotsb & \\frac{\\partial}{\\partial x_{n}}x_{n} \\\\\n", 280 | "\\end{bmatrix} \n", 281 | "=\n", 282 | "\\begin{bmatrix}\n", 283 | "1 & 0 & \\dotsb & 0 \\\\\n", 284 | "0 & 1 & \\dotsb & 0 \\\\\n", 285 | " & & \\ddots \\\\\n", 286 | "0 & 0 & \\dotsb & 1 \\\\\n", 287 | "\\end{bmatrix}\n", 288 | "= I \n", 289 | "$$\n", 290 | "\n", 291 | "(_I_ is the identity matrix with the ones down the diagonal)" 292 | ] 293 | }, 294 | { 295 | "cell_type": "markdown", 296 | "metadata": {}, 297 | "source": [ 298 | "### Derivatives of vector element-wise binary operators\n", 299 | "\n", 300 | "_Element-wise binary operations on vectors_ - applying an operation to the first element of each vector to get the first element of the output, then apply to the second items of each vector to get the second item of the output, and so forth.\n", 301 | "\n" 302 | ] 303 | }, 304 | { 305 | "cell_type": "markdown", 306 | "metadata": {}, 307 | "source": [ 308 | "#### Generalized notation for element-wise binary operations\n", 309 | "$$\n", 310 | "\\pmb{y} = \\pmb{f}(w) \\bigcirc \\pmb{g}(x) \n", 311 | "$$\n", 312 | "\n", 313 | "where $ m = n = \\vert y \\vert = \\vert w \\vert = \\vert x \\vert $\n", 314 | "\n", 315 | "Reminder: $ \\vert x \\vert $ is the number of items in _x_" 316 | ] 317 | }, 318 | { 319 | "cell_type": "markdown", 320 | "metadata": {}, 321 | "source": [ 322 | "Zooming in $ \\pmb{y} = \\pmb{f}(\\pmb{w}) \\bigcirc \\pmb{g}(\\pmb{x}) $\n", 323 | "gives :\n", 324 | "\n", 325 | "$$\n", 326 | "\\begin{bmatrix}\n", 327 | "y_{1} \\\\\n", 328 | "y_{2} \\\\\n", 329 | "\\vdots \\\\\n", 330 | "y_{n}\n", 331 | "\\end{bmatrix}\n", 332 | "=\n", 333 | "\\begin{bmatrix}\n", 334 | "f_{1}(w) \\bigcirc g_{1}(x) \\\\\n", 335 | "f_{2}(w) \\bigcirc g_{2}(x) \\\\\n", 336 | "\\vdots \\\\\n", 337 | "f_{n}(w) \\bigcirc g_{n}(x) \n", 338 | "\\end{bmatrix} \n", 339 | "$$" 340 | ] 341 | }, 342 | { 343 | "cell_type": "markdown", 344 | "metadata": {}, 345 | "source": [ 346 | "#### Jacobian of Elementwise Binary Operations\n", 347 | "The general case for the Jacobian of **y** wrt **w** is the square matrix:\n", 348 | "$$\n", 349 | "J_{W} =\n", 350 | "\\frac{\\partial y}{\\partial w} =\n", 351 | "\\begin{bmatrix}\n", 352 | "\\frac{\\partial}{\\partial w_{1}}(f_{1}(w) \\bigcirc g_{1}(x)) & \\frac{\\partial}{\\partial w_{2}}(f_{1}(w) \\bigcirc g_{1}(x)) & \\dotsb & \\frac{\\partial}{\\partial w_{n}}(f_{1}(w) \\bigcirc g_{1}(x)) \\\\\n", 353 | "\\frac{\\partial}{\\partial w_{1}}(f_{2}(w) \\bigcirc g_{2}(x)) & \\frac{\\partial}{\\partial w_{2}}(f_{2}(w) \\bigcirc g_{2}(x)) & \\dotsb & \\frac{\\partial}{\\partial w_{n}}(f_{2}(w) \\bigcirc g_{2}(x)) \\\\\n", 354 | "\\dotsb \\\\\n", 355 | "\\frac{\\partial}{\\partial w_{1}}(f_{n}(w) \\bigcirc g_{n}(x)) & \\frac{\\partial}{\\partial w_{2}}(f_{n}(w) \\bigcirc g_{n}(x)) & \\dotsb & \\frac{\\partial}{\\partial w_{n}}(f_{n}(w) \\bigcirc g_{n}(x)) \n", 356 | "\\end{bmatrix} \n", 357 | "$$\n" 358 | ] 359 | }, 360 | { 361 | "cell_type": "markdown", 362 | "metadata": {}, 363 | "source": [ 364 | "The general case for the Jacobian of **y** wrt **x** is the square matrix:\n", 365 | "$$\n", 366 | "J_{X} =\n", 367 | "\\frac{\\partial y}{\\partial x} =\n", 368 | "\\begin{bmatrix}\n", 369 | "\\frac{\\partial}{\\partial x_{1}}(f_{1}(w) \\bigcirc g_{1}(x)) & \\frac{\\partial}{\\partial x_{2}}(f_{1}(w) \\bigcirc g_{1}(x)) & \\dotsb & \\frac{\\partial}{\\partial x_{n}}(f_{1}(w) \\bigcirc g_{1}(x)) \\\\\n", 370 | "\\frac{\\partial}{\\partial x_{1}}(f_{2}(w) \\bigcirc g_{2}(x)) & \\frac{\\partial}{\\partial x_{2}}(f_{2}(w) \\bigcirc g_{2}(x)) & \\dotsb & \\frac{\\partial}{\\partial w_{n}}(f_{2}(w) \\bigcirc g_{2}(x)) \\\\\n", 371 | "\\dotsb \\\\\n", 372 | "\\frac{\\partial}{\\partial x_{1}}(f_{n}(w) \\bigcirc g_{n}(x)) & \\frac{\\partial}{\\partial x_{2}}(f_{n}(w) \\bigcirc g_{n}(x)) & \\dotsb & \\frac{\\partial}{\\partial x_{n}}(f_{n}(w) \\bigcirc g_{n}(x)) \n", 373 | "\\end{bmatrix} \n", 374 | "$$\n" 375 | ] 376 | }, 377 | { 378 | "cell_type": "markdown", 379 | "metadata": {}, 380 | "source": [ 381 | "### Diagonal Jacobians\n", 382 | "In a Diagonal Jacobian, all elements off the diagonal are zero, $ \\frac{\\partial}{\\partial w_{j}}(f_{i}(w) \\bigcirc g_{i}(x)) = 0 $ where $ j \\ne i $\n", 383 | "\n", 384 | "This will be the case when $ f_{i} $ and $ g_{i} $ are constants wrt $ w_{j} $:\n", 385 | "\n", 386 | "$$ \n", 387 | "\\frac{\\partial}{\\partial w_{j}}f_{i}(w) =\n", 388 | "\\frac{\\partial}{\\partial w_{j}}g_{i}(x) = 0\n", 389 | "$$\n", 390 | "\n", 391 | "Regardless of the operation $ \\bigcirc $, if the partial derivatives go to zero, $ 0 \\bigcirc 0 = 0 $ and the partial derivative of a constant is zero.\n", 392 | "\n", 393 | "These partial derivatives go to zero when $f_{i} $ and $ g_{i} $ are not functions of $ w_{j} $.\n", 394 | "\n", 395 | "Element-wise operations imply that $ f_{i} $ is purely a function of $ w_{i} $ and $ g_{i} $ is purely a function of $ x_{i} $.\n", 396 | "\n", 397 | "For example, $ \\pmb{w} + \\pmb{x} $ sums $ w_{i} + x_{i} $. \n", 398 | "\n", 399 | "Consequently, $ f_{i}(w) \\bigcirc g_{i}(x) $ reduces to $ f_{i}(w_{i}) \\bigcirc g_{i}(x_{i}) $ and the goal becomes $ \\frac{\\partial}{\\partial w_{j}}f_{i}(w_{i}) = 0$ and $ \\frac{\\partial}{\\partial w_{j}}g_{i}(x_{i}) = 0 $\n", 400 | "\n", 401 | "Notice that $ f_{i}(w_{i}) $ and $ g_{i}(x_{i}) $ look like constants to the partial differentiation wrt to $ w_{j} $ when $ j \\ne i $\n", 402 | "\n", 403 | "#### Element-wise diagonal condition\n", 404 | "_Element-wise diagonal condition_ refers to the constraint that $ f_{i}(w) $ and $ g_{i}(x) $ access at most only $ w_{i} $ and $ x_{i} $, respectively." 405 | ] 406 | }, 407 | { 408 | "cell_type": "markdown", 409 | "metadata": {}, 410 | "source": [ 411 | "### Jacobians under an element-wise diagonal condition\n", 412 | "\n", 413 | "Under this condition, the elements along the diagonal of the Jacobian are $ \\frac{\\partial}{\\partial w_{i}}(f_{i}(w_{i}) \\bigcirc g_{i}(x_{i})) $:\n", 414 | "\n", 415 | "$$\n", 416 | "\\frac{\\partial y}{\\partial w} = \n", 417 | "\\begin{bmatrix}\n", 418 | "\\frac{\\partial}{\\partial w_{1}}(f_{1}(w_{1}) \\bigcirc g_{1}(x_{1})) & & \\\\\n", 419 | "& \\frac{\\partial}{\\partial w_{2}}(f_{2}(w_{2}) \\bigcirc g_{2}(x_{2})) & \\huge0 \\\\\n", 420 | "\\dotsb \\\\\n", 421 | "\\huge0 & & \\frac{\\partial}{\\partial w_{n}}(f_{n}(w_{n}) \\bigcirc g_{n}(x_{n}))\n", 422 | "\\end{bmatrix}\n", 423 | "$$" 424 | ] 425 | }, 426 | { 427 | "cell_type": "markdown", 428 | "metadata": {}, 429 | "source": [ 430 | "More succinctly, we can rewrite the following as:\n", 431 | "$$\n", 432 | "\\frac{\\partial y}{\\partial w} = \n", 433 | "diag \\left(\\frac{\\partial}{\\partial w_{1}}(f_{1}(w_{1}) \\bigcirc g_{1}(x_{1})),\n", 434 | "\\frac{\\partial}{\\partial w_{2}}(f_{2}(w_{2}) \\bigcirc g_{2}(x_{2})),\n", 435 | "\\dotsb ,\n", 436 | "\\frac{\\partial}{\\partial w_{n}}(f_{n}(w_{n}) \\bigcirc g_{n}(x_{n}))\\right)\n", 437 | "$$\n", 438 | "\n", 439 | "and\n", 440 | "\n", 441 | "$$\n", 442 | "\\frac{\\partial y}{\\partial x} = \n", 443 | "diag \\left(\\frac{\\partial}{\\partial x_{1}}(f_{1}(w_{1}) \\bigcirc g_{1}(x_{1})),\n", 444 | "\\frac{\\partial}{\\partial x_{2}}(f_{2}(w_{2}) \\bigcirc g_{2}(x_{2})),\n", 445 | "\\dotsb ,\n", 446 | "\\frac{\\partial}{\\partial x_{n}}(f_{n}(w_{n}) \\bigcirc g_{n}(x_{n}))\\right)\n", 447 | "$$\n", 448 | "\n", 449 | "where $ diag(x) $ constructs a matrix whose diagonal elements are taken from the vector $ \\pmb{x} $ \n" 450 | ] 451 | }, 452 | { 453 | "cell_type": "markdown", 454 | "metadata": {}, 455 | "source": [ 456 | "If the general function $ \\pmb{f}(w) $ in the binary element-wise operation is just the vector $ \\pmb{w} $, then we know that $ f_i(w) $ reduces to $ f_i(w_i) = w_i $\n", 457 | "\n", 458 | "As an example, $ \\pmb{w} + \\pmb{x} $ fits our element-wise diagonal condtion because $ \\pmb{f}(\\pmb{w}) + \\pmb{g}(\\pmb{x}) $ has scalar equations $ y_i = f_i(w) + g_i(x) $ that reduce to just $ y_i = f_i(w_i) + g_i(x_i) = w_i + x_i $ with the partial derivatives:\n", 459 | "\n", 460 | "\n", 461 | "\n", 462 | "$$\n", 463 | "\\frac{\\partial}{\\partial w_i}(f_i(w_i) + g_i(x_i)) =\n", 464 | "\\frac{\\partial}{\\partial w_i}(w_i + x_i) = \n", 465 | "\\frac{\\partial w_i}{\\partial w_i} + \\frac{\\partial x_i}{\\partial w_i} = \n", 466 | "1 + 0 =\n", 467 | "1\n", 468 | "$$\n", 469 | "\n", 470 | "and\n", 471 | "\n", 472 | "$$\n", 473 | "\\frac{\\partial}{\\partial x_i}(f_i(w_i) + g_i(x_i)) =\n", 474 | "\\frac{\\partial}{\\partial x_i}(w_i + x_i) = \n", 475 | "\\frac{\\partial w_i}{\\partial x_i} + \\frac{\\partial x_i}{\\partial x_i} = \n", 476 | "0 + 1 =\n", 477 | "1\n", 478 | "$$\n", 479 | "Which gives us:\n", 480 | "$$\n", 481 | "\\frac{\\partial (w + x)}{\\partial w} =\n", 482 | "\\frac{\\partial (w + x)}{\\partial x} =\n", 483 | "I\n", 484 | "$$\n", 485 | "\n", 486 | "\n", 487 | "\n" 488 | ] 489 | }, 490 | { 491 | "cell_type": "markdown", 492 | "metadata": {}, 493 | "source": [ 494 | "Given the simplicity of this special case $f_i(\\pmb{w}) $ reducing to $ f_i(w_i) $, we can derive the Jacobians for common element-wise binary operations:\n", 495 | "\n", 496 | "##### Addition _(wrt to w)_\n", 497 | "$$\n", 498 | "\\frac{\\partial (\\pmb{w} + \\pmb{x})}{\\partial \\pmb{w}} =\n", 499 | "diag \\left(\\dotsb \\frac{\\partial (w_i + x_i)}{\\partial w_i} \\dotsb \\right) =\n", 500 | "diag \\left(\\dotsb \\frac{\\partial w_i}{\\partial w_i} + \\frac{\\partial x_i}{\\partial w_i} \\dotsb \\right) =\n", 501 | "diag \\left(\\dotsb 1 + 0 \\dotsb \\right) =\n", 502 | "diag \\left(\\dotsb 1 \\dotsb \\right) =\n", 503 | "diag(\\vec{1}) =\n", 504 | "I\n", 505 | "$$\n", 506 | "##### Subtraction _(wrt to w)_\n", 507 | "$$\n", 508 | "\\frac{\\partial (\\pmb{w} - \\pmb{x})}{\\partial \\pmb{w}} =\n", 509 | "diag \\left(\\dotsb \\frac{\\partial w_i - x_i}{\\partial w_i} \\dotsb \\right) =\n", 510 | "diag \\left(\\dotsb \\frac{\\partial w_i}{\\partial w_i} - \\frac{\\partial x_i}{\\partial w_i} \\dotsb \\right) =\n", 511 | "diag \\left(\\dotsb 1 - 0 \\dotsb \\right) =\n", 512 | "diag \\left(\\dotsb 1 \\dotsb \\right) =\n", 513 | "diag(\\vec{1}) =\n", 514 | "I\n", 515 | "$$\n", 516 | "\n", 517 | "##### Elementwise Multiplication (aka Hadamard product) _(wrt to w)_\n", 518 | "$$\n", 519 | "\\frac{\\partial (\\pmb{w} \\otimes \\pmb{x})}{\\partial \\pmb{w}} =\n", 520 | "diag \\left(\\dotsb \\frac{\\partial w_i \\times x_i}{\\partial w_i} \\dotsb \\right) =\n", 521 | "diag \\left(\\dotsb (x_i \\times \\frac{\\partial w_i}{\\partial w_i}) + (w_i \\times \\frac{\\partial x_i}{\\partial w_i}) \\dotsb \\right) =\n", 522 | "diag \\left(\\dotsb (x_i \\times 1 ) + (w_i \\times 0) \\dotsb \\right) =\n", 523 | "diag \\left(\\dotsb x_i \\dotsb \\right) =\n", 524 | "diag(\\pmb{x})\n", 525 | "$$\n", 526 | "\n", 527 | "##### Elementwise Division _(wrt to w)_\n", 528 | "$$\n", 529 | "\\frac{\\partial (\\pmb{w} \\oslash \\pmb{x})}{\\partial \\pmb{w}} =\n", 530 | "diag \\left(\\dotsb \\frac{\\partial (\\frac{w_i}{x_i})}{\\partial w_i} \\dotsb \\right) =\n", 531 | "diag \\left(\\dotsb (\\frac{1}{x_i} \\times \\frac{\\partial w_i}{\\partial w_i}) + (w_i \\times \\frac{\\partial \\frac{1}{x_i}}{\\partial w_i}) \\dotsb \\right) =\n", 532 | "diag \\left(\\dotsb (\\frac{1}{x_i} \\times 1 ) + (w_i \\times 0) \\dotsb \\right) =\n", 533 | "diag \\left(\\dotsb \\frac{1}{x_i} \\dotsb \\right)\n", 534 | "$$\n", 535 | "\n" 536 | ] 537 | }, 538 | { 539 | "cell_type": "markdown", 540 | "metadata": {}, 541 | "source": [ 542 | "\n", 543 | "##### Addition _(wrt to x)_\n", 544 | "$$\n", 545 | "\\frac{\\partial (\\pmb{w} + \\pmb{x})}{\\partial \\pmb{x}} =\n", 546 | "diag \\left(\\dotsb \\frac{\\partial (w_i + x_i)}{\\partial x_i} \\dotsb \\right) =\n", 547 | "diag \\left(\\dotsb \\frac{\\partial w_i}{\\partial x_i} + \\frac{\\partial x_i}{\\partial x_i} \\dotsb \\right) =\n", 548 | "diag \\left(\\dotsb 0 + 1 \\dotsb \\right) =\n", 549 | "diag \\left(\\dotsb 1 \\dotsb \\right) =\n", 550 | "diag(\\vec{1}) =\n", 551 | "I\n", 552 | "$$\n", 553 | "##### Subtraction _(wrt to x)_\n", 554 | "$$\n", 555 | "\\frac{\\partial (\\pmb{w} - \\pmb{x})}{\\partial \\pmb{x}} =\n", 556 | "diag \\left(\\dotsb \\frac{\\partial (w_i - x_i)}{\\partial x_i} \\dotsb \\right) =\n", 557 | "diag \\left(\\dotsb \\frac{\\partial w_i}{\\partial x_i} - \\frac{\\partial x_i}{\\partial x_i} \\dotsb \\right) =\n", 558 | "diag \\left(\\dotsb 0 - 1 \\dotsb \\right) =\n", 559 | "diag \\left(\\dotsb -1 \\dotsb \\right) =\n", 560 | "diag(-\\vec{1}) =\n", 561 | "-I\n", 562 | "$$\n", 563 | "\n", 564 | "##### Elementwise Multiplication (aka Hadamard product) _(wrt to x)_\n", 565 | "$$\n", 566 | "\\frac{\\partial (\\pmb{w} \\otimes \\pmb{x})}{\\partial \\pmb{x}} =\n", 567 | "diag \\left(\\dotsb \\frac{\\partial (w_i \\times x_i)}{\\partial x_i} \\dotsb \\right) =\n", 568 | "diag \\left(\\dotsb (x_i \\times \\frac{\\partial w_i}{\\partial x_i}) + (w_i \\times \\frac{\\partial x_i}{\\partial x_i}) \\dotsb \\right) =\n", 569 | "diag \\left(\\dotsb (x_i \\times 0) + (w_i \\times 1) \\dotsb \\right) =\n", 570 | "diag \\left(\\dotsb w_i \\dotsb \\right) =\n", 571 | "diag(\\pmb{w})\n", 572 | "$$\n", 573 | "\n", 574 | "##### Elementwise Division _(wrt to x)_\n", 575 | "$$\n", 576 | "\\frac{\\partial (\\pmb{w} \\oslash \\pmb{x})}{\\partial \\pmb{x}} =\n", 577 | "diag \\left(\\dotsb \\frac{\\partial (\\frac{w_i}{x_i})}{\\partial x_i} \\dotsb \\right) =\n", 578 | "diag \\left(\\dotsb (\\frac{1}{x_i} \\times \\frac{\\partial w_i}{\\partial x_i}) + (w_i \\times \\frac{\\partial}{\\partial x_i}(\\frac{1}{x})) \\dotsb \\right) =\n", 579 | "diag \\left(\\dotsb (\\frac{1}{x_i} \\times 0) + (w_i \\times -\\frac{1}{x_{i}^{2}}) \\dotsb \\right) =\n", 580 | "diag \\left(\\dotsb \\frac{-w_i}{x_i^2} \\dotsb \\right)\n", 581 | "$$\n", 582 | "\n", 583 | "_Note_:\n", 584 | "$$\n", 585 | "\\frac{\\partial (\\frac{w_i}{x_i})}{\\partial x_i} =\n", 586 | "w_i \\times \\frac{\\partial (\\frac{1}{x_i})}{\\partial x_i} =\n", 587 | "w_i \\times \\frac{\\partial x_{i}^{-1}}{\\partial x_i} =\n", 588 | "w_i \\times (-1) \\times x_{i}^{-2} =\n", 589 | "\\frac {-w_i}{x_{i}^{2}}\n", 590 | "$$\n", 591 | "\n", 592 | "\n", 593 | "\n" 594 | ] 595 | }, 596 | { 597 | "cell_type": "markdown", 598 | "metadata": {}, 599 | "source": [ 600 | "### Derivatives involving scalar expansion\n", 601 | "\n", 602 | "To multiply or add scalars to vectors, we expand the scalar to a vector and\n", 603 | "perform an elementwise binary operation.\n", 604 | "\n", 605 | "Given a scalar $ z $ and a vector $ \\pmb{x} $, the partial derivatives for elementwise\n", 606 | "operations for the function $ \\pmb{y} = \\pmb{x} \\bigcirc z $ wrt to $ \\pmb{x} $ is given by:\n", 607 | "\n", 608 | "$$\n", 609 | "\\frac{\\partial \\pmb{y}}{\\partial \\pmb{x}} =\n", 610 | "diag \\left( \\dotsb \\frac{\\partial}{\\partial x_i}(f_i(x_i) \\bigcirc g_i(z)) \\dotsb \\right)\n", 611 | "$$\n", 612 | "\n", 613 | "where $ \\pmb{g}(z) = \\vec{1}z $ is a vector function of the appropriate length where $ g_i(z) = z $ and $ \\pmb{f}(\\pmb{x}) $ is a vector function such that $ f_i(x_i) = x_i $" 614 | ] 615 | }, 616 | { 617 | "cell_type": "markdown", 618 | "metadata": {}, 619 | "source": [ 620 | "This results in the following (based on the previous derivations for elementwise binary operations)\n", 621 | "\n", 622 | "##### Addition wrt x :\n", 623 | "$$\n", 624 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{x} + z) =\n", 625 | "I\n", 626 | "$$\n", 627 | "Since \n", 628 | "$$\n", 629 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{x} + z) =\n", 630 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{f}(\\pmb{x}) + \\pmb{g}(z) =\n", 631 | "diag \\left(\\dotsb \\frac{\\partial x_i + z}{\\partial x_i} \\dotsb \\right) =\n", 632 | "diag(\\vec{1}) =\n", 633 | "I\n", 634 | "$$\n", 635 | "Since for a given $ y_i = f_i(x) + g_i(z) = x_i + z $, \n", 636 | "\n", 637 | "$$\n", 638 | "\\frac{\\partial y_i}{\\partial x_i} =\n", 639 | "\\frac{\\partial}{\\partial x_{i}}(f_i(x) + g_i(z)) =\n", 640 | "\\frac{\\partial x_i}{\\partial x_i} + \\frac{\\partial z}{\\partial x_i} =\n", 641 | "1 + 0 =\n", 642 | "1\n", 643 | "$$ \n", 644 | "and $ \\frac{\\partial}{\\partial x_{i}}(f_j(x) + g_j(z)) = 0 + 0 $ for all $ i \\ne j $ \n", 645 | "\n", 646 | "##### Subtraction wrt x:\n", 647 | "$$\n", 648 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{x} - z) =\n", 649 | "I\n", 650 | "$$\n", 651 | "Since \n", 652 | "$$\n", 653 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{x} - z) =\n", 654 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{f}(\\pmb{x}) - \\pmb{g}(z) =\n", 655 | "diag \\left(\\dotsb \\frac{\\partial x_i - z}{\\partial x_i} \\dotsb \\right) =\n", 656 | "diag(\\vec{1}) =\n", 657 | "I\n", 658 | "$$\n", 659 | "Since for a given $ y_i = f_i(x) - g_i(z) = x_i - z $, \n", 660 | "\n", 661 | "$$\n", 662 | "\\frac{\\partial y_i}{\\partial x_i} =\n", 663 | "\\frac{\\partial}{\\partial x_{i}}(f_i(x) - g_i(z)) =\n", 664 | "\\frac{\\partial}{\\partial x_{i}}x_i - \\frac{\\partial}{\\partial z} =\n", 665 | "1 - 0 =\n", 666 | "1\n", 667 | "$$ \n", 668 | "and $ \\frac{\\partial}{\\partial x_{i}}(f_j(x) - g_j(z)) = 0 - 0 $ for all $ i \\ne j $ \n", 669 | "\n", 670 | "##### Multiplication wrt x :\n", 671 | "$$\n", 672 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{x} \\times z) =\n", 673 | "diag(z)\n", 674 | "$$\n", 675 | "Since\n", 676 | "$$\n", 677 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{x} \\times z) =\n", 678 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{f}(\\pmb{x}) \\otimes \\pmb{g}(z)) = \n", 679 | "diag \\left( \\dotsb \\frac{\\partial}{\\partial x_{i}}(x_{i} \\times z) \\dotsb \\right) =\n", 680 | "diag(z)\n", 681 | "$$\n", 682 | "Since for $ y_i = x_i \\times z $, \n", 683 | "$$\n", 684 | "\\frac{\\partial y_i}{\\partial x_i} = \n", 685 | "\\frac{\\partial}{\\partial x_{i}}(x_{i} \\times z) = \n", 686 | "(z \\times \\frac{\\partial x_{i}}{\\partial x_{i}}) + (x_i \\times \\frac {\\partial z}{\\partial x_{i}}) =\n", 687 | "(z \\times 1) + (x_i \\times 0)\n", 688 | "= z\n", 689 | "$$\n", 690 | "for all $ i = j $ and $ \\frac{\\partial y_j}{\\partial x_i} = 0 $ for all $ j \\ne i $\n", 691 | "since $ \\frac{\\partial x_j}{\\partial x_i} = 0 $ and $ \\frac{\\partial z}{\\partial x_i} = 0 $\n", 692 | "##### Division wrt x:\n", 693 | "$$\n", 694 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\frac{\\pmb{x}}{z}) =\n", 695 | "diag \\left( \\frac{1}{z} \\right)\n", 696 | "$$\n", 697 | "Since \n", 698 | "$$\n", 699 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\frac{\\pmb{x}}{z}) =\n", 700 | "\\frac{\\partial}{\\partial \\pmb{x}}(\\pmb{x} \\times \\frac{1}{z}) =\n", 701 | "diag \\left( \\frac{1}{z} \\right)\n", 702 | "$$\n" 703 | ] 704 | }, 705 | { 706 | "cell_type": "markdown", 707 | "metadata": {}, 708 | "source": [ 709 | "#### WRT to Z however,\n", 710 | "\n", 711 | "##### Addition wrt z :\n", 712 | "$$ \n", 713 | "\\frac{\\partial}{\\partial z}(\\pmb{x} + z) =\n", 714 | "\\begin{bmatrix}\n", 715 | "\\frac{\\partial}{\\partial z}(x_1 + z) \\\\\n", 716 | "\\frac{\\partial}{\\partial z}(x_2 + z) \\\\\n", 717 | "\\vdots \\\\\n", 718 | "\\frac{\\partial}{\\partial z}(x_n + z)\n", 719 | "\\end{bmatrix} =\n", 720 | "\\begin{bmatrix}\n", 721 | "\\frac{\\partial}{\\partial z}x_1 + \\frac{\\partial z}{\\partial z} \\\\\n", 722 | "\\frac{\\partial}{\\partial z}x_2 + \\frac{\\partial z}{\\partial z} \\\\\n", 723 | "\\vdots \\\\\n", 724 | "\\frac{\\partial}{\\partial z}x_n + \\frac{\\partial z}{\\partial z}\n", 725 | "\\end{bmatrix} =\n", 726 | "\\begin{bmatrix}\n", 727 | "0 + 1 \\\\\n", 728 | "0 + 1 \\\\\n", 729 | "\\vdots \\\\\n", 730 | "0 + 1 \\\\\n", 731 | "\\end{bmatrix} =\n", 732 | "\\begin{bmatrix}\n", 733 | "1 \\\\\n", 734 | "1 \\\\\n", 735 | "\\vdots \\\\\n", 736 | "1 \\\\\n", 737 | "\\end{bmatrix} =\n", 738 | "\\vec{1}\n", 739 | "$$\n", 740 | "\n", 741 | "##### Subtraction wrt z:\n", 742 | "$$ \n", 743 | "\\frac{\\partial}{\\partial z}(\\pmb{x} - z) =\n", 744 | "\\begin{bmatrix}\n", 745 | "\\frac{\\partial}{\\partial z}(x_1 - z) \\\\\n", 746 | "\\frac{\\partial}{\\partial z}(x_2 - z) \\\\\n", 747 | "\\vdots \\\\\n", 748 | "\\frac{\\partial}{\\partial z}(x_n - z)\n", 749 | "\\end{bmatrix} =\n", 750 | "\\begin{bmatrix}\n", 751 | "\\frac{\\partial}{\\partial z}x_1 - \\frac{\\partial z}{\\partial z} \\\\\n", 752 | "\\frac{\\partial}{\\partial z}x_2 - \\frac{\\partial z}{\\partial z} \\\\\n", 753 | "\\vdots \\\\\n", 754 | "\\frac{\\partial}{\\partial z}x_n - \\frac{\\partial z}{\\partial z}\n", 755 | "\\end{bmatrix} =\n", 756 | "\\begin{bmatrix}\n", 757 | "0 - 1 \\\\\n", 758 | "0 - 1 \\\\\n", 759 | "\\vdots \\\\\n", 760 | "0 - 1 \\\\\n", 761 | "\\end{bmatrix} =\n", 762 | "\\begin{bmatrix}\n", 763 | "-1 \\\\\n", 764 | "-1 \\\\\n", 765 | "\\vdots \\\\\n", 766 | "-1 \\\\\n", 767 | "\\end{bmatrix} =\n", 768 | "\\vec{-1}\n", 769 | "$$\n", 770 | "##### Multiplication wrt z :\n", 771 | "$$\n", 772 | "\\frac{\\partial}{\\partial z}(\\pmb{x} \\times z) =\n", 773 | "\\begin{bmatrix}\n", 774 | "\\frac{\\partial}{\\partial z}(x_1 \\times z) \\\\\n", 775 | "\\frac{\\partial}{\\partial z}(x_2 \\times z) \\\\\n", 776 | "\\vdots \\\\\n", 777 | "\\frac{\\partial}{\\partial z}(x_i \\times z) \\\\\n", 778 | "\\vdots \\\\\n", 779 | "\\frac{\\partial}{\\partial z}(x_n \\times z)\n", 780 | "\\end{bmatrix} =\n", 781 | "\\pmb{x}\n", 782 | "$$\n", 783 | "Since\n", 784 | "$$\n", 785 | "\\frac{\\partial}{\\partial z}(x_i \\times z) =\n", 786 | "(x_i \\times \\frac{\\partial z}{\\partial z}) + (z \\times \\frac{\\partial x_i}{\\partial z}) =\n", 787 | "(x_i \\times 1) + (z \\times 0) =\n", 788 | "x_i\n", 789 | "$$\n", 790 | "##### Division wrt z:\n", 791 | "$$\n", 792 | "\\frac{\\partial}{\\partial z}(\\frac{\\pmb{x}}{z}) =\n", 793 | "\\frac{\\partial}{\\partial z}(\\pmb{x} \\times \\frac{1}{z}) =\n", 794 | "\\begin{bmatrix}\n", 795 | "\\frac{\\partial}{\\partial z}(x_1 \\times \\frac{1}{z}) \\\\\n", 796 | "\\frac{\\partial}{\\partial z}(x_2 \\times \\frac{1}{z}) \\\\\n", 797 | "\\vdots \\\\\n", 798 | "\\frac{\\partial}{\\partial z}(x_i \\times \\frac{1}{z}) \\\\\n", 799 | "\\vdots \\\\\n", 800 | "\\frac{\\partial}{\\partial z}(x_n \\times \\frac{1}{z})\n", 801 | "\\end{bmatrix} =\n", 802 | "\\pmb{x}\\frac{-1}{z^2} =\n", 803 | "\\frac{- \\pmb{x}}{z^2} =\n", 804 | "$$\n", 805 | "Since \n", 806 | "$$\n", 807 | "\\frac{\\partial}{\\partial z}(x_i \\times \\frac{1}{z}) =\n", 808 | "(x_i \\times \\frac{\\partial}{\\partial z}\\frac{1}{z}) + (\\frac{1}{z} \\times \\frac{\\partial x_i}{\\partial z}) =\n", 809 | "(x_i \\times \\frac{-1}{z^2}) + (\\frac{1}{z} \\times 0) =\n", 810 | "\\frac{-x_i}{z^2}\n", 811 | "$$\n" 812 | ] 813 | }, 814 | { 815 | "cell_type": "markdown", 816 | "metadata": {}, 817 | "source": [ 818 | "### Vector Sum Reduction\n", 819 | "\n", 820 | "Let $ y = sum(f(\\pmb{x})) = \\sum \\limits_{i=1}^{n}f_i(\\pmb{x}) $, where each function $ f_i $ could use all values in the vector, not just $ x_i $. The sum is over the results of the function, not the parameter. The gradient ($ 1 \\times n $ Jacobian) of vector summation is: \n", 821 | "\n", 822 | "$$\n", 823 | "\\frac{\\partial y}{\\partial \\pmb{x}} = \n", 824 | "\\begin{bmatrix}\n", 825 | "\\frac{\\partial y}{\\partial x_1}, \\frac{\\partial y}{\\partial x_2}, \\dotsb , \\frac{\\partial y}{\\partial x_n}\n", 826 | "\\end{bmatrix} =\n", 827 | "\\begin{bmatrix}\n", 828 | "\\frac{\\partial}{\\partial x_1}\\sum_i f_i(x), \\frac{\\partial}{\\partial x_2} \\sum_i f_i(x), \\dotsb , \\frac{\\partial}{\\partial x_n}\\sum_i f_i(x)\n", 829 | "\\end{bmatrix} =\n", 830 | "\\begin{bmatrix}\n", 831 | "\\sum_i \\frac{\\partial}{\\partial x_1}f_i(x), \\sum_i \\frac{\\partial}{\\partial x_2} f_i(x), \\dotsb , \\sum_i \\frac{\\partial}{\\partial x_n}f_i(x)\n", 832 | "\\end{bmatrix}\n", 833 | "\n", 834 | "$$" 835 | ] 836 | }, 837 | { 838 | "cell_type": "markdown", 839 | "metadata": {}, 840 | "source": [ 841 | "For example, given $ y = sum(x) $, the function $ f_i(x) = x_i $ and the gradient is:\n", 842 | "\n", 843 | "$$\n", 844 | "\\nabla y = \n", 845 | "\\begin{bmatrix}\n", 846 | "\\sum_i \\frac{\\partial f_i(x)}{\\partial x_1}, \\sum_i \\frac{\\partial f_i(x)}{\\partial x_2}, \\dotsb , \\sum_i \\frac{\\partial f_i(x)}{\\partial x_n}\n", 847 | "\\end{bmatrix} =\n", 848 | "\\begin{bmatrix}\n", 849 | "\\sum_i \\frac{\\partial x_i}{\\partial x_1}, \\sum_i \\frac{\\partial x_i}{\\partial x_2}, \\dotsb , \\sum_i \\frac{\\partial x_i}{\\partial x_n}\n", 850 | "\\end{bmatrix} =\n", 851 | "\\begin{bmatrix}\n", 852 | "\\frac{\\partial x_1}{\\partial x_1}, \\frac{\\partial x_2}{\\partial x_2}, \\dotsb , \\frac{\\partial x_n}{\\partial x_n}\n", 853 | "\\end{bmatrix} =\n", 854 | "\\begin{bmatrix}\n", 855 | "1, 1, \\dotsb , 1\n", 856 | "\\end{bmatrix} =\n", 857 | "\\vec{1}^T\n", 858 | "$$\n", 859 | "Since $ \\frac{\\partial}{\\partial x_j}x_i = 0 $ for $ i \\ne j $." 860 | ] 861 | }, 862 | { 863 | "cell_type": "markdown", 864 | "metadata": {}, 865 | "source": [ 866 | "As another example, given $ y = sum(\\pmb{x}z) $ then $ f_{i}(x,z) = x_{i}z $\n", 867 | "The gradient wrt to $ \\pmb{x} $ is:\n", 868 | "$$\n", 869 | "\\frac{\\partial y}{\\partial \\pmb{x}} =\n", 870 | "\\begin{bmatrix}\n", 871 | "\\sum_i\\frac{\\partial}{\\partial x_1}x_{i}z,\n", 872 | "\\sum_i\\frac{\\partial}{\\partial x_2}x_{i}z,\n", 873 | "\\dotsb,\n", 874 | "\\sum_i\\frac{\\partial}{\\partial x_n}x_{i}z,\n", 875 | "\\end{bmatrix} =\n", 876 | "\\begin{bmatrix}\n", 877 | "\\frac{\\partial}{\\partial x_1}x_{1}z,\n", 878 | "\\frac{\\partial}{\\partial x_2}x_{2}z,\n", 879 | "\\dotsb,\n", 880 | "\\frac{\\partial}{\\partial x_n}x_{n}z,\n", 881 | "\\end{bmatrix} =\n", 882 | "\\left[ z,z, \\dotsb , z \\right] =\n", 883 | "\\vec{z}^T\n", 884 | "$$\n", 885 | "where $ \\vert \\pmb{x} \\vert = \\vert \\vec{z} \\vert = n $.\n", 886 | "\n", 887 | "With respect to the scalar z, the resulting is a $ 1 \\times 1 $\n", 888 | "$$\n", 889 | "\\frac{\\partial y}{\\partial z} =\n", 890 | "\\frac{\\partial}{\\partial z}\\sum_{i=1}^{n}x_{i}z =\n", 891 | "\\sum_i \\frac{\\partial}{\\partial z} x_{i}z =\n", 892 | "\\sum_i x_i =\n", 893 | "sum(\\pmb{x})\n", 894 | "$$" 895 | ] 896 | }, 897 | { 898 | "cell_type": "markdown", 899 | "metadata": {}, 900 | "source": [ 901 | "#### Vector Chain Rules\n", 902 | "\n", 903 | "Three types of chain rules:\n", 904 | "\n", 905 | "* _single-variable chain rule_ - derivative of a scalar function with respect to a scalar\n", 906 | "* _single-variable total-derivative chain rule_ - derivative of a scalar function wrt to scalar _but with \"many paths\"_\n", 907 | "* _vector chain rule_ - full vector chain rules\n", 908 | "\n", 909 | "##### Single Variable Chain Rule\n" 910 | ] 911 | }, 912 | { 913 | "cell_type": "markdown", 914 | "metadata": {}, 915 | "source": [ 916 | "Given $ y = f(g(x)) $, the scalar derivation rule can be expressed by defining an intermediate variable $ u $ such that:\n", 917 | "$$\n", 918 | "y = f(u)\n", 919 | "$$\n", 920 | "and\n", 921 | "$$\n", 922 | "u = g(x)\n", 923 | "$$\n", 924 | "The derivative of\n", 925 | "$$\n", 926 | "\\frac{dy}{dx} = \\frac{dy}{du}\\frac{du}{dx}\n", 927 | "$$\n", 928 | "As an example, given:\n", 929 | "$$\n", 930 | "y = sin(x^2)\n", 931 | "$$\n", 932 | "then \n", 933 | "$$\n", 934 | "\\frac{dy}{dx} = \n", 935 | "\\frac{d}{du}sin(u) \\times \\frac{d}{dx}(x^2) =\n", 936 | "cos(u) \\times 2x =\n", 937 | "2x cos(x^2) \n", 938 | "$$\n", 939 | "where \n", 940 | "$$\n", 941 | "u = x^2\n", 942 | "$$\n", 943 | "\n", 944 | "###### Another example: \n", 945 | "\n", 946 | "Given $ y = f(x) = ln(sin(x^3)^2) $ let: \n", 947 | "$$\n", 948 | "\\begin{align*}\n", 949 | "u_1 &= f_1(x) = x^3 \\\\\n", 950 | "u_2 &= f_2(u_1) = sin(u_1) \\\\\n", 951 | "u_3 &= f_3(u_2) = u_2^2 \\\\\n", 952 | "u_4 &= f_4(u_3) = ln(u_3) \\\\\n", 953 | "y &= u_4 &\n", 954 | "\\end{align*}\n", 955 | "$$\n", 956 | "\n", 957 | "Such that\n", 958 | "$$\n", 959 | "\\begin{align*}\n", 960 | "\\frac{d}{dx}u_1 &= \\frac{d}{dx}x^3 = 3x^2 \\\\\n", 961 | "\\frac{d}{du_1}u_2 &= \\frac{d}{du_1}sin(u_1) = cos(u_1)\\\\\n", 962 | "\\frac{d}{du_2}u_3 &= \\frac{d}{du_2}u_2^2 = 2u_2\\\\\n", 963 | "\\frac{d}{du_3}u_4 &= \\frac{d}{du_3}ln(u_3) = \\frac{1}{u_3}\\\\\n", 964 | "\\frac{dy}{du_4} &= \\frac{du_4}{du_4} = 1\n", 965 | "\\end{align*}\n", 966 | "$$\n", 967 | "\n", 968 | "And combining the intermediate variables :\n", 969 | "$$\n", 970 | "\\frac{dy}{dx} = \\frac{dy}{du_4}\\frac{du_4}{du_3}\\frac{du_3}{du_2}\\frac{du_2}{du_1}\\frac{du_1}{dx} =\n", 971 | "1 \\times \\frac{1}{u_3} \\times 2u_2 \\times cos(u_1) \\times 3x^2 =\n", 972 | "\\frac{6u_2x^2cos(u_1)}{u_3}\n", 973 | "$$\n", 974 | "\n", 975 | "Substituting the derivatives results in\n", 976 | "$$\n", 977 | "\\frac{dy}{dx} = \n", 978 | "\\frac{6sin(u_1)x^2cos(x^3}{u_2^2} =\n", 979 | "\\frac{6sin(x^3)x^2cos(x^3)}{sin(u_1)^2} =\n", 980 | "\\frac{6sin(x^3)x^2cos(x^3)}{sin(x^3)^2} =\n", 981 | "\\frac{6x^2cos(x^3)}{sin(x^3)}\n", 982 | "$$\n", 983 | "\n", 984 | "##### Single variable total derivative chain rule\n", 985 | "$$\n", 986 | "\\frac{\\partial f(u_1,\\dotsb,u_{n+1})}{\\partial x} =\n", 987 | "\\sum \\limits_{i=1}^{n+1}\\frac{\\partial f}{\\partial u_i}\\frac{\\partial u_i}{\\partial x}\n", 988 | "$$\n", 989 | "\n", 990 | "As an example, given: $ y = f(x) = x + x^2 $, we can restate the function as:\n", 991 | "$$\n", 992 | "\\begin{align*}\n", 993 | "u_1(x) &= x^2 \\\\\n", 994 | "u_2(x,u_1) &= x + u_1 \\\\\n", 995 | "y &= f(x) = u_2(x,u_1)\n", 996 | "\\end{align*}\n", 997 | "$$\n", 998 | "Applying the total derivative rule:\n", 999 | "$$\n", 1000 | "\\frac{dy}{dx} =\n", 1001 | "\\frac{\\partial f(x)}{\\partial x} =\n", 1002 | "\\frac{\\partial u_2(x,u_1)}{\\partial x} =\n", 1003 | "\\frac{\\partial u_2}{\\partial x}\\frac{\\partial x}{\\partial x} + \\frac{\\partial u_2}{\\partial u_1}\\frac{\\partial u_1}{\\partial x} = \n", 1004 | "$$" 1005 | ] 1006 | }, 1007 | { 1008 | "cell_type": "markdown", 1009 | "metadata": {}, 1010 | "source": [ 1011 | "### Questions and Exercises\n", 1012 | "\n", 1013 | "1. Compute the Jacobian of the following function $ \\pmb{f}(\\pmb{w}) $:\n", 1014 | "\n", 1015 | " 1. such that $ f_{i}(w) = w_{i}^{2} $: \n", 1016 | "$$\n", 1017 | "\\frac{\\partial \\pmb{y}}{\\partial \\pmb{w}} =\n", 1018 | "\\begin{bmatrix}\n", 1019 | "\\frac{\\partial}{\\partial w_1}f_1(w) & \\frac{\\partial}{\\partial w_2}f_1(w) & \\dotsb & \\frac{\\partial}{\\partial w_n}f_1(w) \\\\\n", 1020 | "\\frac{\\partial}{\\partial w_1}f_2(w) & \\frac{\\partial}{\\partial w_2}f_2(w) & \\dotsb & \\frac{\\partial}{\\partial w_n}f_2(w) \\\\\n", 1021 | "& & \\ddots \\\\\n", 1022 | "\\frac{\\partial}{\\partial w_1}f_n(w) & \\frac{\\partial}{\\partial w_2}f_n(w) & \\dotsb & \\frac{\\partial}{\\partial w_n}f_n(w) \n", 1023 | "\\end{bmatrix} =\n", 1024 | "\\begin{bmatrix}\n", 1025 | "\\frac{\\partial}{\\partial w_1} w_{1}^{2} & \\frac{\\partial}{\\partial w_2}w_{1}^{2} & \\dotsb & \\frac{\\partial}{\\partial w_n}w_{1}^{2} \\\\\n", 1026 | "\\frac{\\partial}{\\partial w_1} w_{2}^{2} & \\frac{\\partial}{\\partial w_2}w_{2}^{2} & \\dotsb & \\frac{\\partial}{\\partial w_n}w_{2}^{2} \\\\\n", 1027 | "& & \\ddots \\\\\n", 1028 | "\\frac{\\partial}{\\partial w_1} w_{n}^{2} & \\frac{\\partial}{\\partial w_2}w_{n}^{2} & \\dotsb & \\frac{\\partial}{\\partial w_n}w_{n}^{2} \n", 1029 | "\\end{bmatrix} =\n", 1030 | "\\begin{bmatrix}\n", 1031 | "2w_1 & 0 & \\dotsb & 0 \\\\\n", 1032 | "0 & 2w_2 & \\dotsb & 0 \\\\\n", 1033 | "& & \\ddots \\\\\n", 1034 | "0 & 0 & \\dotsb & 2w_n\n", 1035 | "\\end{bmatrix} =\n", 1036 | "diag \\left( \\dotsb 2 w_i \\dotsb \\right) \n", 1037 | "$$\n", 1038 | "\n", 1039 | " 1. such that $ f_{i}(w) = w_{i}^{i} $:\n", 1040 | "$$\n", 1041 | "\\frac{\\partial \\pmb{y}}{\\partial \\pmb{w}} =\n", 1042 | "\\begin{bmatrix}\n", 1043 | "\\frac{\\partial}{\\partial w_1}f_1(w) & \\frac{\\partial}{\\partial w_2}f_1(w) & \\dotsb & \\frac{\\partial}{\\partial w_n}f_1(w) \\\\\n", 1044 | "\\frac{\\partial}{\\partial w_1}f_2(w) & \\frac{\\partial}{\\partial w_2}f_2(w) & \\dotsb & \\frac{\\partial}{\\partial w_n}f_2(w) \\\\\n", 1045 | "& & \\ddots \\\\\n", 1046 | "\\frac{\\partial}{\\partial w_1}f_n(w) & \\frac{\\partial}{\\partial w_2}f_n(w) & \\dotsb & \\frac{\\partial}{\\partial w_n}f_n(w) \n", 1047 | "\\end{bmatrix} =\n", 1048 | "\\begin{bmatrix}\n", 1049 | "\\frac{\\partial}{\\partial w_1} w_{1}^{1} & \\frac{\\partial}{\\partial w_2}w_{1}^{1} & \\dotsb & \\frac{\\partial}{\\partial w_n}w_{1}^{1} \\\\\n", 1050 | "\\frac{\\partial}{\\partial w_1} w_{2}^{2} & \\frac{\\partial}{\\partial w_2}w_{2}^{2} & \\dotsb & \\frac{\\partial}{\\partial w_n}w_{2}^{2} \\\\\n", 1051 | "& & \\ddots \\\\\n", 1052 | "\\frac{\\partial}{\\partial w_1} w_{n}^{n} & \\frac{\\partial}{\\partial w_2}w_{n}^{n} & \\dotsb & \\frac{\\partial}{\\partial w_n}w_{n}^{n} \n", 1053 | "\\end{bmatrix} =\n", 1054 | "\\begin{bmatrix}\n", 1055 | "1w_{1}^{0} & 0 & \\dotsb & 0 \\\\\n", 1056 | "0 & 2w_{2}^{1} & \\dotsb & 0 \\\\\n", 1057 | "& & \\ddots \\\\\n", 1058 | "0 & 0 & \\dotsb & nw_{n}^{n-1}\n", 1059 | "\\end{bmatrix} =\n", 1060 | "diag \\left( \\dotsb iw_{i}^{i-1} \\dotsb \\right) \n", 1061 | "$$\n", 1062 | " 1. such that $ f_{i}(w) = \\sum \\limits _{j} ^{m} w_{j}^{i} $ where $ n = m = \\vert w \\vert $:\n", 1063 | "$$\n", 1064 | "\\frac{\\partial \\pmb{y}}{\\partial \\pmb{w}} =\n", 1065 | "\\begin{bmatrix}\n", 1066 | "\\frac{\\partial}{\\partial w_1}f_1(w) & \\frac{\\partial}{\\partial w_2}f_1(w) & \\dotsb & \\frac{\\partial}{\\partial w_n}f_1(w) \\\\\n", 1067 | "\\frac{\\partial}{\\partial w_1}f_2(w) & \\frac{\\partial}{\\partial w_2}f_2(w) & \\dotsb & \\frac{\\partial}{\\partial w_n}f_2(w) \\\\\n", 1068 | "& & \\ddots \\\\\n", 1069 | "\\frac{\\partial}{\\partial w_1}f_n(w) & \\frac{\\partial}{\\partial w_2}f_n(w) & \\dotsb & \\frac{\\partial}{\\partial w_n}f_n(w) \n", 1070 | "\\end{bmatrix} =\n", 1071 | "\\begin{bmatrix}\n", 1072 | "\\frac{\\partial}{\\partial w_1} \\sum \\limits _{j} ^{n} w_{j}^{1} & \\frac{\\partial}{\\partial w_2}\\sum \\limits _{j} ^{n} w_{j}^{1} & \\dotsb & \\frac{\\partial}{\\partial w_n}\\sum \\limits _{j} ^{n} w_{j}^{1} \\\\\n", 1073 | "\\frac{\\partial}{\\partial w_1} \\sum \\limits _{j} ^{n} w_{j}^{2} & \\frac{\\partial}{\\partial w_2}\\sum \\limits _{j} ^{n} w_{j}^{2} & \\dotsb & \\frac{\\partial}{\\partial w_n}\\sum \\limits _{j} ^{n} w_{j}^{2} \\\\\n", 1074 | "& & \\ddots \\\\\n", 1075 | "\\frac{\\partial}{\\partial w_1} \\sum \\limits _{j} ^{n} w_{j}^{n} & \\frac{\\partial}{\\partial w_2}\\sum \\limits _{j} ^{n} w_{j}^{n} & \\dotsb & \\frac{\\partial}{\\partial w_n}\\sum \\limits _{j} ^{n} w_{j}^{n} \n", 1076 | "\\end{bmatrix} =\n", 1077 | "\\begin{bmatrix}\n", 1078 | "\\frac{\\partial}{\\partial w_1}w_{1}^{1} & \\frac{\\partial}{\\partial w_2}w_{2}^{1} & \\dotsb & \\frac{\\partial}{\\partial w_n}w_{n}^{1} \\\\\n", 1079 | "\\frac{\\partial}{\\partial w_1}w_{1}^{2} & \\frac{\\partial}{\\partial w_2}w_{2}^{2} & \\dotsb & \\frac{\\partial}{\\partial w_n}w_{n}^{2} \\\\\n", 1080 | "& & \\ddots \\\\\n", 1081 | "\\frac{\\partial}{\\partial w_1}w_{1}^{n} & \\frac{\\partial}{\\partial w_2}w_{2}^{n} & \\dotsb & \\frac{\\partial}{\\partial w_n}w_{n}^{n} \\\\\n", 1082 | "\\end{bmatrix} =\n", 1083 | "\\begin{bmatrix}\n", 1084 | "1w_{1}^{0} & 1w_{2}^{0} & \\dotsb & 1w_{n}^{0} \\\\\n", 1085 | "2w_{1}^{1} & 2w_{2}^{1} & \\dotsb & 2w_{n}^{1} \\\\\n", 1086 | "& & \\ddots \\\\\n", 1087 | "nw_{1}^{n-1} & nw_{2}^{n-1} & \\dotsb & nw_{n}^{n-1}\n", 1088 | "\\end{bmatrix} =\n", 1089 | "\\begin{bmatrix}\n", 1090 | "1 & 1 & \\dotsb & 1 \\\\\n", 1091 | "2w_{1} & 2w_{2} & \\dotsb & 2w_{n} \\\\\n", 1092 | "& & \\ddots \\\\\n", 1093 | "nw_{1}^{n-1} & nw_{2}^{n-1} & \\dotsb & nw_{n}^{n-1}\n", 1094 | "\\end{bmatrix} \n", 1095 | "$$\n", 1096 | "Since\n", 1097 | "$$\n", 1098 | "\\frac{\\partial}{\\partial w_i} \\sum \\limits _{j} ^{n} w_{j}^{i} =\n", 1099 | "\\frac{\\partial}{\\partial w_i} \\left( w_{1}^{i} + w_{2}^{i} + \\dots + w_{i}^{i} + \\dots + w_{n}^{i} \\right) =\n", 1100 | "\\left( \\frac{\\partial}{\\partial w_i}w_{1}^{i} + \\frac{\\partial}{\\partial w_i}w_{2}^{i} + \\dots + \\frac{\\partial}{\\partial w_i}w_{i}^{i} + \\dots + \\frac{\\partial}{\\partial w_i}w_{n}^{i} \\right) =\n", 1101 | "\\left( 0 + 0 + \\dots + iw_{i}^{i-1} + \\dots + \\right) =\n", 1102 | "iw_{i}^{i-1}\n", 1103 | "$$\n", 1104 | "\n", 1105 | "\n" 1106 | ] 1107 | }, 1108 | { 1109 | "cell_type": "code", 1110 | "execution_count": null, 1111 | "metadata": {}, 1112 | "outputs": [], 1113 | "source": [] 1114 | }, 1115 | { 1116 | "cell_type": "code", 1117 | "execution_count": null, 1118 | "metadata": {}, 1119 | "outputs": [], 1120 | "source": [] 1121 | }, 1122 | { 1123 | "cell_type": "code", 1124 | "execution_count": null, 1125 | "metadata": {}, 1126 | "outputs": [], 1127 | "source": [] 1128 | }, 1129 | { 1130 | "cell_type": "code", 1131 | "execution_count": null, 1132 | "metadata": {}, 1133 | "outputs": [], 1134 | "source": [] 1135 | }, 1136 | { 1137 | "cell_type": "code", 1138 | "execution_count": null, 1139 | "metadata": {}, 1140 | "outputs": [], 1141 | "source": [] 1142 | }, 1143 | { 1144 | "cell_type": "code", 1145 | "execution_count": null, 1146 | "metadata": {}, 1147 | "outputs": [], 1148 | "source": [] 1149 | }, 1150 | { 1151 | "cell_type": "code", 1152 | "execution_count": null, 1153 | "metadata": {}, 1154 | "outputs": [], 1155 | "source": [] 1156 | }, 1157 | { 1158 | "cell_type": "code", 1159 | "execution_count": null, 1160 | "metadata": {}, 1161 | "outputs": [], 1162 | "source": [] 1163 | } 1164 | ], 1165 | "metadata": { 1166 | "kernelspec": { 1167 | "display_name": "Python 3", 1168 | "language": "python", 1169 | "name": "python3" 1170 | }, 1171 | "language_info": { 1172 | "codemirror_mode": { 1173 | "name": "ipython", 1174 | "version": 3 1175 | }, 1176 | "file_extension": ".py", 1177 | "mimetype": "text/x-python", 1178 | "name": "python", 1179 | "nbconvert_exporter": "python", 1180 | "pygments_lexer": "ipython3", 1181 | "version": "3.7.7" 1182 | }, 1183 | "varInspector": { 1184 | "cols": { 1185 | "lenName": 16, 1186 | "lenType": 16, 1187 | "lenVar": 40 1188 | }, 1189 | "kernels_config": { 1190 | "python": { 1191 | "delete_cmd_postfix": "", 1192 | "delete_cmd_prefix": "del ", 1193 | "library": "var_list.py", 1194 | "varRefreshCmd": "print(var_dic_list())" 1195 | }, 1196 | "r": { 1197 | "delete_cmd_postfix": ") ", 1198 | "delete_cmd_prefix": "rm(", 1199 | "library": "var_list.r", 1200 | "varRefreshCmd": "cat(var_dic_list()) " 1201 | } 1202 | }, 1203 | "types_to_exclude": [ 1204 | "module", 1205 | "function", 1206 | "builtin_function_or_method", 1207 | "instance", 1208 | "_Feature" 1209 | ], 1210 | "window_display": false 1211 | } 1212 | }, 1213 | "nbformat": 4, 1214 | "nbformat_minor": 4 1215 | } 1216 | -------------------------------------------------------------------------------- /CollectRealFingersData.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "CollectRealFingersData.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [], 9 | "toc_visible": true, 10 | "authorship_tag": "ABX9TyOdbPgVFpaaS5LVewk+xjYy", 11 | "include_colab_link": true 12 | }, 13 | "kernelspec": { 14 | "name": "python3", 15 | "display_name": "Python 3" 16 | } 17 | }, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "metadata": { 22 | "id": "view-in-github", 23 | "colab_type": "text" 24 | }, 25 | "source": [ 26 | "\"Open" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "metadata": { 32 | "id": "LZ7xi9zsSsb7", 33 | "colab_type": "text" 34 | }, 35 | "source": [ 36 | "# Data Exploration notebooks for Fingers datasets" 37 | ] 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "metadata": { 42 | "id": "r4i9cOODIxh0", 43 | "colab_type": "text" 44 | }, 45 | "source": [ 46 | "### Run using a CPU Runtime (no GPU needed)" 47 | ] 48 | }, 49 | { 50 | "cell_type": "markdown", 51 | "metadata": { 52 | "id": "llTUM3bWTLJO", 53 | "colab_type": "text" 54 | }, 55 | "source": [ 56 | "## Environment setup" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "metadata": { 62 | "id": "_hl8hkvmDkNW", 63 | "colab_type": "code", 64 | "outputId": "519bf64d-ce78-4646-bf8c-4d2eb6c9a56a", 65 | "colab": { 66 | "base_uri": "https://localhost:8080/", 67 | "height": 102 68 | } 69 | }, 70 | "source": [ 71 | "!curl https://course.fast.ai/setup/colab | bash" 72 | ], 73 | "execution_count": 1, 74 | "outputs": [ 75 | { 76 | "output_type": "stream", 77 | "text": [ 78 | " % Total % Received % Xferd Average Speed Time Time Time Current\n", 79 | " Dload Upload Total Spent Left Speed\n", 80 | "\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 322 100 322 0 0 1634 0 --:--:-- --:--:-- --:--:-- 1634\n", 81 | "Updating fastai...\n", 82 | "Done.\n" 83 | ], 84 | "name": "stdout" 85 | } 86 | ] 87 | }, 88 | { 89 | "cell_type": "code", 90 | "metadata": { 91 | "id": "TteGcyYhEEi_", 92 | "colab_type": "code", 93 | "colab": { 94 | "base_uri": "https://localhost:8080/", 95 | "height": 1000 96 | }, 97 | "outputId": "797a0a12-82b6-4a22-bcc9-e7c8c1e608b3" 98 | }, 99 | "source": [ 100 | "!pip install fastai2 --upgrade\n", 101 | "!pip install fastcore --upgrade\n", 102 | "!pip install nbdev --upgrade" 103 | ], 104 | "execution_count": 2, 105 | "outputs": [ 106 | { 107 | "output_type": "stream", 108 | "text": [ 109 | "Collecting fastai2\n", 110 | "\u001b[?25l Downloading https://files.pythonhosted.org/packages/bd/2b/9e0f6d3d928105dec034727ed723c33340f71a77ca253b67bbec99eacd3f/fastai2-0.0.16-py3-none-any.whl (184kB)\n", 111 | "\r\u001b[K |█▉ | 10kB 18.8MB/s eta 0:00:01\r\u001b[K |███▋ | 20kB 1.8MB/s eta 0:00:01\r\u001b[K |█████▍ | 30kB 2.3MB/s eta 0:00:01\r\u001b[K |███████▏ | 40kB 1.7MB/s eta 0:00:01\r\u001b[K |█████████ | 51kB 1.9MB/s eta 0:00:01\r\u001b[K |██████████▊ | 61kB 2.2MB/s eta 0:00:01\r\u001b[K |████████████▌ | 71kB 2.4MB/s eta 0:00:01\r\u001b[K |██████████████▎ | 81kB 2.5MB/s eta 0:00:01\r\u001b[K |████████████████ | 92kB 2.8MB/s eta 0:00:01\r\u001b[K |█████████████████▉ | 102kB 2.8MB/s eta 0:00:01\r\u001b[K |███████████████████▋ | 112kB 2.8MB/s eta 0:00:01\r\u001b[K |█████████████████████▍ | 122kB 2.8MB/s eta 0:00:01\r\u001b[K |███████████████████████▏ | 133kB 2.8MB/s eta 0:00:01\r\u001b[K |█████████████████████████ | 143kB 2.8MB/s eta 0:00:01\r\u001b[K |██████████████████████████▊ | 153kB 2.8MB/s eta 0:00:01\r\u001b[K |████████████████████████████▌ | 163kB 2.8MB/s eta 0:00:01\r\u001b[K |██████████████████████████████▎ | 174kB 2.8MB/s eta 0:00:01\r\u001b[K |████████████████████████████████| 184kB 2.8MB/s \n", 112 | "\u001b[?25hRequirement already satisfied, skipping upgrade: matplotlib in /usr/local/lib/python3.6/dist-packages (from fastai2) (3.2.1)\n", 113 | "Requirement already satisfied, skipping upgrade: pandas in /usr/local/lib/python3.6/dist-packages (from fastai2) (1.0.3)\n", 114 | "Requirement already satisfied, skipping upgrade: fastprogress>=0.1.22 in /usr/local/lib/python3.6/dist-packages (from fastai2) (0.2.2)\n", 115 | "Requirement already satisfied, skipping upgrade: spacy in /usr/local/lib/python3.6/dist-packages (from fastai2) (2.2.4)\n", 116 | "Requirement already satisfied, skipping upgrade: requests in /usr/local/lib/python3.6/dist-packages (from fastai2) (2.21.0)\n", 117 | "Requirement already satisfied, skipping upgrade: pyyaml in /usr/local/lib/python3.6/dist-packages (from fastai2) (3.13)\n", 118 | "Requirement already satisfied, skipping upgrade: scikit-learn in /usr/local/lib/python3.6/dist-packages (from fastai2) (0.22.2.post1)\n", 119 | "Requirement already satisfied, skipping upgrade: scipy in /usr/local/lib/python3.6/dist-packages (from fastai2) (1.4.1)\n", 120 | "Requirement already satisfied, skipping upgrade: pillow in /usr/local/lib/python3.6/dist-packages (from fastai2) (7.0.0)\n", 121 | "Requirement already satisfied, skipping upgrade: torch>=1.3.0 in /usr/local/lib/python3.6/dist-packages (from fastai2) (1.4.0)\n", 122 | "Collecting fastcore\n", 123 | " Downloading https://files.pythonhosted.org/packages/3b/b5/aed836ce5b16ea1088a5d1a41d400bc051abf90bbef58bb74d8fd01a76af/fastcore-0.1.16-py3-none-any.whl\n", 124 | "Requirement already satisfied, skipping upgrade: torchvision>=0.5 in /usr/local/lib/python3.6/dist-packages (from fastai2) (0.5.0)\n", 125 | "Requirement already satisfied, skipping upgrade: numpy>=1.11 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (1.18.2)\n", 126 | "Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (2.8.1)\n", 127 | "Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (2.4.6)\n", 128 | "Requirement already satisfied, skipping upgrade: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (0.10.0)\n", 129 | "Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai2) (1.2.0)\n", 130 | "Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->fastai2) (2018.9)\n", 131 | "Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (46.1.3)\n", 132 | "Requirement already satisfied, skipping upgrade: thinc==7.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (7.4.0)\n", 133 | "Requirement already satisfied, skipping upgrade: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (1.0.2)\n", 134 | "Requirement already satisfied, skipping upgrade: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (3.0.2)\n", 135 | "Requirement already satisfied, skipping upgrade: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (2.0.3)\n", 136 | "Requirement already satisfied, skipping upgrade: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (0.6.0)\n", 137 | "Requirement already satisfied, skipping upgrade: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (1.0.0)\n", 138 | "Requirement already satisfied, skipping upgrade: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (4.38.0)\n", 139 | "Requirement already satisfied, skipping upgrade: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (1.1.3)\n", 140 | "Requirement already satisfied, skipping upgrade: blis<0.5.0,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (0.4.1)\n", 141 | "Requirement already satisfied, skipping upgrade: srsly<1.1.0,>=1.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai2) (1.0.2)\n", 142 | "Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->fastai2) (3.0.4)\n", 143 | "Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->fastai2) (2019.11.28)\n", 144 | "Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->fastai2) (2.8)\n", 145 | "Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->fastai2) (1.24.3)\n", 146 | "Requirement already satisfied, skipping upgrade: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->fastai2) (0.14.1)\n", 147 | "Requirement already satisfied, skipping upgrade: dataclasses>='0.7'; python_version < \"3.7\" in /usr/local/lib/python3.6/dist-packages (from fastcore->fastai2) (0.7)\n", 148 | "Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from torchvision>=0.5->fastai2) (1.12.0)\n", 149 | "Requirement already satisfied, skipping upgrade: importlib-metadata>=0.20; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from catalogue<1.1.0,>=0.0.7->spacy->fastai2) (1.6.0)\n", 150 | "Requirement already satisfied, skipping upgrade: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata>=0.20; python_version < \"3.8\"->catalogue<1.1.0,>=0.0.7->spacy->fastai2) (3.1.0)\n", 151 | "Installing collected packages: fastcore, fastai2\n", 152 | "Successfully installed fastai2-0.0.16 fastcore-0.1.16\n", 153 | "Requirement already up-to-date: fastcore in /usr/local/lib/python3.6/dist-packages (0.1.16)\n", 154 | "Requirement already satisfied, skipping upgrade: numpy in /usr/local/lib/python3.6/dist-packages (from fastcore) (1.18.2)\n", 155 | "Requirement already satisfied, skipping upgrade: dataclasses>='0.7'; python_version < \"3.7\" in /usr/local/lib/python3.6/dist-packages (from fastcore) (0.7)\n", 156 | "Collecting nbdev\n", 157 | "\u001b[?25l Downloading https://files.pythonhosted.org/packages/a0/c2/2631b32810211b6be5b17aff8db7135a8f29ff2a09a397cd9efbba490d48/nbdev-0.2.17-py3-none-any.whl (44kB)\n", 158 | "\u001b[K |████████████████████████████████| 51kB 1.7MB/s \n", 159 | "\u001b[?25hRequirement already satisfied, skipping upgrade: pyyaml in /usr/local/lib/python3.6/dist-packages (from nbdev) (3.13)\n", 160 | "Requirement already satisfied, skipping upgrade: packaging in /usr/local/lib/python3.6/dist-packages (from nbdev) (20.3)\n", 161 | "Requirement already satisfied, skipping upgrade: nbformat>=4.4.0 in /usr/local/lib/python3.6/dist-packages (from nbdev) (5.0.5)\n", 162 | "Collecting fastscript\n", 163 | " Downloading https://files.pythonhosted.org/packages/55/0e/ecdc0213646bc82986884121109a38b50bbc2cd2c491bbbfdc7ae39228e3/fastscript-0.1.4-py3-none-any.whl\n", 164 | "Requirement already satisfied, skipping upgrade: nbconvert>=5.6.1 in /usr/local/lib/python3.6/dist-packages (from nbdev) (5.6.1)\n", 165 | "Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from packaging->nbdev) (1.12.0)\n", 166 | "Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from packaging->nbdev) (2.4.6)\n", 167 | "Requirement already satisfied, skipping upgrade: jsonschema!=2.5.0,>=2.4 in /usr/local/lib/python3.6/dist-packages (from nbformat>=4.4.0->nbdev) (2.6.0)\n", 168 | "Requirement already satisfied, skipping upgrade: traitlets>=4.1 in /usr/local/lib/python3.6/dist-packages (from nbformat>=4.4.0->nbdev) (4.3.3)\n", 169 | "Requirement already satisfied, skipping upgrade: jupyter-core in /usr/local/lib/python3.6/dist-packages (from nbformat>=4.4.0->nbdev) (4.6.3)\n", 170 | "Requirement already satisfied, skipping upgrade: ipython-genutils in /usr/local/lib/python3.6/dist-packages (from nbformat>=4.4.0->nbdev) (0.2.0)\n", 171 | "Requirement already satisfied, skipping upgrade: bleach in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (3.1.4)\n", 172 | "Requirement already satisfied, skipping upgrade: pygments in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (2.1.3)\n", 173 | "Requirement already satisfied, skipping upgrade: mistune<2,>=0.8.1 in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (0.8.4)\n", 174 | "Requirement already satisfied, skipping upgrade: entrypoints>=0.2.2 in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (0.3)\n", 175 | "Requirement already satisfied, skipping upgrade: jinja2>=2.4 in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (2.11.1)\n", 176 | "Requirement already satisfied, skipping upgrade: pandocfilters>=1.4.1 in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (1.4.2)\n", 177 | "Requirement already satisfied, skipping upgrade: testpath in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (0.4.4)\n", 178 | "Requirement already satisfied, skipping upgrade: defusedxml in /usr/local/lib/python3.6/dist-packages (from nbconvert>=5.6.1->nbdev) (0.6.0)\n", 179 | "Requirement already satisfied, skipping upgrade: decorator in /usr/local/lib/python3.6/dist-packages (from traitlets>=4.1->nbformat>=4.4.0->nbdev) (4.4.2)\n", 180 | "Requirement already satisfied, skipping upgrade: webencodings in /usr/local/lib/python3.6/dist-packages (from bleach->nbconvert>=5.6.1->nbdev) (0.5.1)\n", 181 | "Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from jinja2>=2.4->nbconvert>=5.6.1->nbdev) (1.1.1)\n", 182 | "Installing collected packages: fastscript, nbdev\n", 183 | "Successfully installed fastscript-0.1.4 nbdev-0.2.17\n" 184 | ], 185 | "name": "stdout" 186 | } 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "metadata": { 192 | "id": "FyIRmd1AFGPO", 193 | "colab_type": "code", 194 | "colab": { 195 | "base_uri": "https://localhost:8080/", 196 | "height": 122 197 | }, 198 | "outputId": "8e400be6-66d2-4f4c-9189-82ceb253b25c" 199 | }, 200 | "source": [ 201 | "from google.colab import drive\n", 202 | "drive.mount('/content/drive')" 203 | ], 204 | "execution_count": 3, 205 | "outputs": [ 206 | { 207 | "output_type": "stream", 208 | "text": [ 209 | "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n", 210 | "\n", 211 | "Enter your authorization code:\n", 212 | "··········\n", 213 | "Mounted at /content/drive\n" 214 | ], 215 | "name": "stdout" 216 | } 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "metadata": { 222 | "id": "VVy2lfJKFPzq", 223 | "colab_type": "code", 224 | "colab": {} 225 | }, 226 | "source": [ 227 | "from fastai2.vision.all import *" 228 | ], 229 | "execution_count": 0, 230 | "outputs": [] 231 | }, 232 | { 233 | "cell_type": "code", 234 | "metadata": { 235 | "id": "rNqytLLDFoAc", 236 | "colab_type": "code", 237 | "colab": {} 238 | }, 239 | "source": [ 240 | "escdrive = lambda x : x.as_posix().replace(' ','\\ ')" 241 | ], 242 | "execution_count": 0, 243 | "outputs": [] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "metadata": { 248 | "id": "GU9zH5p9GI8J", 249 | "colab_type": "code", 250 | "colab": {} 251 | }, 252 | "source": [ 253 | "gdrive = Path('/content/drive/My Drive/fastai_v4')" 254 | ], 255 | "execution_count": 0, 256 | "outputs": [] 257 | }, 258 | { 259 | "cell_type": "code", 260 | "metadata": { 261 | "id": "TFD-j5nxGisi", 262 | "colab_type": "code", 263 | "outputId": "2484501d-7f1b-4227-e917-3b4dbbfe4f10", 264 | "colab": { 265 | "base_uri": "https://localhost:8080/", 266 | "height": 34 267 | } 268 | }, 269 | "source": [ 270 | "config = Config()\n", 271 | "data_path = config.d['data_path']\n", 272 | "archive_path = config.d['archive_path']\n", 273 | "model_path = config.d['model_path']\n", 274 | "(data_path, archive_path,model_path)" 275 | ], 276 | "execution_count": 7, 277 | "outputs": [ 278 | { 279 | "output_type": "execute_result", 280 | "data": { 281 | "text/plain": [ 282 | "('/root/.fastai/data', '/root/.fastai/archive', '/root/.fastai/models')" 283 | ] 284 | }, 285 | "metadata": { 286 | "tags": [] 287 | }, 288 | "execution_count": 7 289 | } 290 | ] 291 | }, 292 | { 293 | "cell_type": "markdown", 294 | "metadata": { 295 | "id": "PODEpAkqbAon", 296 | "colab_type": "text" 297 | }, 298 | "source": [ 299 | "## Collect Realworld Pics as Test Data" 300 | ] 301 | }, 302 | { 303 | "cell_type": "code", 304 | "metadata": { 305 | "id": "zCJXbr3bc0FZ", 306 | "colab_type": "code", 307 | "colab": {} 308 | }, 309 | "source": [ 310 | "from IPython.display import display, Javascript\n", 311 | "from google.colab.output import eval_js\n", 312 | "from base64 import b64decode\n", 313 | "\n", 314 | "def take_photo(filename='photo.jpg', quality=0.8):\n", 315 | " js = Javascript('''\n", 316 | " async function takePhoto(quality) {\n", 317 | " const div = document.createElement('div');\n", 318 | " const capture = document.createElement('button');\n", 319 | " capture.textContent = 'Capture';\n", 320 | " div.appendChild(capture);\n", 321 | "\n", 322 | " const video = document.createElement('video');\n", 323 | " video.style.display = 'block';\n", 324 | " const stream = await navigator.mediaDevices.getUserMedia({video: true});\n", 325 | "\n", 326 | " document.body.appendChild(div);\n", 327 | " div.appendChild(video);\n", 328 | " video.srcObject = stream;\n", 329 | " await video.play();\n", 330 | "\n", 331 | " // Resize the output to fit the video element.\n", 332 | " google.colab.output.setIframeHeight(document.documentElement.scrollHeight, true);\n", 333 | "\n", 334 | " // Wait for Capture to be clicked.\n", 335 | " await new Promise((resolve) => capture.onclick = resolve);\n", 336 | "\n", 337 | " const canvas = document.createElement('canvas');\n", 338 | " canvas.width = video.videoWidth;\n", 339 | " canvas.height = video.videoHeight;\n", 340 | " canvas.getContext('2d').drawImage(video, 0, 0);\n", 341 | " stream.getVideoTracks()[0].stop();\n", 342 | " div.remove();\n", 343 | " return canvas.toDataURL('image/jpeg', quality);\n", 344 | " }\n", 345 | " ''')\n", 346 | " display(js)\n", 347 | " data = eval_js('takePhoto({})'.format(quality))\n", 348 | " binary = b64decode(data.split(',')[1])\n", 349 | " with open(filename, 'wb') as f:\n", 350 | " f.write(binary)\n", 351 | " return filename" 352 | ], 353 | "execution_count": 0, 354 | "outputs": [] 355 | }, 356 | { 357 | "cell_type": "code", 358 | "metadata": { 359 | "id": "J3dnmdY9c0EP", 360 | "colab_type": "code", 361 | "colab": { 362 | "base_uri": "https://localhost:8080/", 363 | "height": 514 364 | }, 365 | "outputId": "bebb6a3f-9d69-4ede-edd7-11a70a064966" 366 | }, 367 | "source": [ 368 | "from IPython.display import Image\n", 369 | "try:\n", 370 | " filename = take_photo()\n", 371 | " print('Saved to {}'.format(filename))\n", 372 | " \n", 373 | " # Show the image which was just taken.\n", 374 | " display(Image(filename))\n", 375 | "except Exception as err:\n", 376 | " # Errors will be thrown if the user does not have a webcam or if they do not\n", 377 | " # grant the page permission to access it.\n", 378 | " print(str(err))" 379 | ], 380 | "execution_count": 9, 381 | "outputs": [ 382 | { 383 | "output_type": "display_data", 384 | "data": { 385 | "application/javascript": [ 386 | "\n", 387 | " async function takePhoto(quality) {\n", 388 | " const div = document.createElement('div');\n", 389 | " const capture = document.createElement('button');\n", 390 | " capture.textContent = 'Capture';\n", 391 | " div.appendChild(capture);\n", 392 | "\n", 393 | " const video = document.createElement('video');\n", 394 | " video.style.display = 'block';\n", 395 | " const stream = await navigator.mediaDevices.getUserMedia({video: true});\n", 396 | "\n", 397 | " document.body.appendChild(div);\n", 398 | " div.appendChild(video);\n", 399 | " video.srcObject = stream;\n", 400 | " await video.play();\n", 401 | "\n", 402 | " // Resize the output to fit the video element.\n", 403 | " google.colab.output.setIframeHeight(document.documentElement.scrollHeight, true);\n", 404 | "\n", 405 | " // Wait for Capture to be clicked.\n", 406 | " await new Promise((resolve) => capture.onclick = resolve);\n", 407 | "\n", 408 | " const canvas = document.createElement('canvas');\n", 409 | " canvas.width = video.videoWidth;\n", 410 | " canvas.height = video.videoHeight;\n", 411 | " canvas.getContext('2d').drawImage(video, 0, 0);\n", 412 | " stream.getVideoTracks()[0].stop();\n", 413 | " div.remove();\n", 414 | " return canvas.toDataURL('image/jpeg', quality);\n", 415 | " }\n", 416 | " " 417 | ], 418 | "text/plain": [ 419 | "" 420 | ] 421 | }, 422 | "metadata": { 423 | "tags": [] 424 | } 425 | }, 426 | { 427 | "output_type": "stream", 428 | "text": [ 429 | "Saved to photo.jpg\n" 430 | ], 431 | "name": "stdout" 432 | }, 433 | { 434 | "output_type": "display_data", 435 | "data": { 436 | "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAYEBQYFBAYGBQYHBwYIChAKCgkJChQODwwQFxQYGBcU\nFhYaHSUfGhsjHBYWICwgIyYnKSopGR8tMC0oMCUoKSj/2wBDAQcHBwoIChMKChMoGhYaKCgoKCgo\nKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCj/wAARCAHgAoADASIA\nAhEBAxEB/8QAHAAAAwEBAQEBAQAAAAAAAAAAAAECAwUEBgcI/8QAOBAAAgIBAwMEAQQBAwQCAQUB\nAAECESEDEjEEQVEFImFxgRMykaEGB0KxFCPB0VLhYhUlU3Lw8f/EABkBAQEBAQEBAAAAAAAAAAAA\nAAEAAgMEBf/EACERAQEBAAMAAwEBAQEBAAAAAAABEQIhMRJBUWFxAyIT/9oADAMBAAIRAxEAPwD+\neIS8o0tpYJiotY5DLxZ25dunpptP+x23JSFxXyUlSy0wtZs+xKTwmljI3PwuRbE3d54InF838DMv\nQinBSVtK6FOPtp8Bbkkt2CpJyirXBbi7RtU0lLKflC/SSVxVGyaqqyRKX0anOmJ2P90uOw21eKoJ\nNtbaslqS/wD+Du+nqttN5yipSw78GS3eRtt8mc2s3opS9zruVtT7prwRGnNRu2VKoSy3RqxLUsW8\nMlajpeEUktrpii1bvj6M+rCT7ZfYOMpOyLuWL5NMPg3mHxLu2/kqU5O1RN85LTjj6Bn/ABDSayJf\n/KNu/gau3h1Zc17Y1yWmE6SSqidq5SCTlaTi+eTWkkleR+jccbrW46tcWjyxrdfc93qENurF/wDy\nR4qSsxyxYuCs303KNUYafg9SxBGNGdK5z+OC4pJMUGaNcs1bo8S3WUK28J4B8UGNuP3G4hUlPcrf\n4K1W5psI6jil9kc/yHtGaUYqs4YRdtJcO2XsVYsKcaxg00SWM9iIzqWU8M1cr5/oU1alVfZcf6pA\n2nyG61WP4Ek1VjcU+GGSAOTkiVFN5LUVi2gms0s0M5mM93v2qjWEmnbSJaaaklQ5TVpfAXKre122\n/ghRp2uA09Snnj6KT3JJGZ/5RacnGO35BvLE3VIteWuSsnrNnaItW08Dabdscq5qmZ7pcJhP2Ekr\nnxwgaaW3sVG/GSZftbbNTlolKqyOD9wJccieGmjVvymN41nLbH7Mnbbod2s35HF19PBznQxMVXKy\niovc+A2W21zVCtxGWU4fCwQpO8Jl5ppEp82ODlDSbq/sJcfYtzi88C3bkHxtElqm33XBTlUXgiOp\nfCCV/gLP1YL3ZJunSwacpiq7fdGup0ZMJXWWqFSXA1msg8Bk+ii3lZyVCTSp2S5NIMtqhkl6GjVd\npckxdPjJplrgmSfYLk6PFW++wSW5NNBFP+SlJcXkz54r/GLVSk44vkPdbqshK9+AypZOm6Yp2vtm\nbtOy2pcgrfIfJrHvtNqsD4K2qsGTrvVnP1mU79wO4t/yJIee5qncXFt1mg5TWXZK4zRenSWTHjOs\n+MIuMqXNicU5VZUOGmNv6qlpu2iVG3llNvdVYK27U2n/AEalmKdhRSSp5Btpe5qvoyulZSluat4K\n8VmHF7ryOly5Eywm0KVtOuaNSWi9nahO0uRzqTTbx3M9sk7lx2NYxTtP8DeqNKDvCspaTUcvj5FD\n2vg01P245YW99CbWGYytMqMW7tkwTWPzk2+PJrnT/qVDd3JqjSKdPPwUoJypPBjVf4msbc3REcvN\n0U5K3TyhNUsVwPHr0W41Scryvoz1FJNvDCEpJJtUvk03bo2nTK7onLtzOuVJNO2c+ScZZOr1y9uT\nlatto58u63ur0+TdP2nn03f4N4N2khnHFraCxg0km06r+TOLp5NISuXGOStVsJWsvIK1lK7Lmrpr\ngI5TaxRq3pmUVux3BrtYPDwLbJu3wYg8Lj+BrKTyLnDLwouvo18jobt/LIVusjzS2tL7Jbd0+EMN\nq2mnYLMsVQN81wyFKpcF3Rpyi+LCGG23yVGbvOGWkm19jbhlRbcb5REkva39FrFrtYONp5wwlzsa\nzjHPk0i9rSocXTW36FJ+7LK8tEuiWX3WQUmqSXA90U1kJtUq5Dfppk5OT5BJrN0Wv2g4txfYugTy\n+eCYJtyTawVFcWx06b4HydEU1Km0TON1Xcp5eMlfDM7hvJliKoce3glJ8s0XKTdF6LdC1MUqIbvh\noH7bsSkmzUhi1JtOSruiVG32wNpuL2kx9qbfNBKr0ajeGJQpYKTi+XTGub+TNtFtZyUov/6Hml5N\nZST5ZnKqWRlt9a0ndYDO106Gk6XcFfI39FTFyWAak1eC9rtPCoTuqKctDJQlWWxxuPlmmaJdrOGa\n06qLtCq2SluvJSfdnKztGzH9v8mknTBKLeTU6Uwk0vyTNOTVYouTWUqxwTN+AnLto4212YnF3lpB\nCbVYpLwPem84LtfJ7t3te13ZO27zkHGkqRL/AHYRSb0y0wk7oenteXj7M5KT5KSTwWKr9jT7/kUn\nik8EPTr+BVSM530pVbkpZotNds/RnBK8ji915yNiwOT3XwVKT2pJ3gmluZpdZXgv8TJK1kS01XL5\nsq/NlJJ08/R0Nuo20n3LaSQp8KkTFpZ8D6OlpqVReAurJ9s26XPc0ULjf9GbRmJb4Hvwrz5E1l35\nB7Tdz6ZqnTusCtpqhRjjObLnFY22sDs+1VQ90b7pkuVuVdiMq1fA9yzjLC8e2p/DiuG/JVKKWH/J\nPKqmaKmlYW4LEyXtVEwzkqc40ku2BQSq2QyR5+vX/bXemcjWw18na61XoySu6xRx9SN0/BmtQoLD\nPRpfR5YprPY9GnJqKrsG9HHohhr4LjHflXXBknKrSXJopNRpY7jnWi5PFtpKsWJN7cY8k0nx3Goe\n1rhvgcEUmnSdCm7ST4BrbSd8ULP3QWQ6cWk/gFJ7rE47WrQ+W0i6HRp+7NExe6+xSWQpN2PUWSh0\nuHbI5RSaXIvLrBS4LBJraitKaSyTKKclXCGoqnSYfXaxWqtyTi6RKxi8iTuNJ9+QX7qvLLL4sOL7\nXwZNvd5N4pbfc+CUopuk8+RlikxEOS3JJ2yXiXNFS2tO8sMNEnuTSFbqgWKGsqsF1BiVKmu9GmrO\n4ullkulwrYpN/hstlRJpKxqWLJ2yz4Kj7o+AuGpbux/PkSjd5CadJXQ/TXQtU939sIqNsGm1hrGA\nXtTdvgpRat0mZNY5KtN5sHt4QTpM1G3fyWpqqHxSQnDa/hm5n2p2G4vDVhONxbSocvb9lq2srBnl\nfw9fSIt3yqFbXARaUsockuwf6vipSdVkmUnbpMai6b/4JS2vJmWTxE5Pblcgn7S5NPFEVT+B2U5A\nsA5KsFVgztJUiWYUm5R5JivcrfYHDa7t5K23TwdOsGQ8pCbpJsai21lL7HJWqOfUMxO600kJfJSV\nPgKp/wBlKuq9u9J1kl2hOKd3eSmk63N0auTxByuIK+TOVU1bCMm8O6LOlI2uWcuyd6/3MIvKtBPb\ntboxJ2sJSt4Xcpe1W/BnGKu0az/ak+50sg5ds5PO5XXIT98at/zRWLWSa9zaXIzIvDWFta4QKXtx\nfAvLv4NobdqC8jfEw9ypjklHxgE0pYJk25U8qvBfbOtIOOKXJblUsGEdsXnwXiV+GivGHNCabdku\nTTXgSik3n8DpHSZB9rU2l+122UpO3jF+RJ0mKtyvuHWgSlGMlhDjTshQppPKY1BK3VP7NbxNyeLk\nlH3U7EptxM7bj+aLUHLEeQyfY/qHT7YvkOFh2aLTe2nXnBns22otvwMsq3RqyahlM42pl3wdqUXP\nT+Ti6q23ZjnY1xKHJvptMxg47ecmmm0ng5SaseqHz9lNN/CMm3SUSraWbOnHpY0h8sdJ3zyRHlFX\nVfY3GbA8VfkN1tpGepLwxpcOykmdpq1Sbb/kISy26J4dfFE+UGT7Ea4ptchuteDNJJt5zgqmmHxi\nKcouKVK7Ekks2kEUnJvNin7m0MkFi93h8lSunnBEYqlnI3dO2FkMJQ2347Di9r7WTKlihx93bJGl\nJSlfZsp+2KwmyWqlbl2wLdfyx6U0U2r7hdruFO0HHwa6iw4t89kNt8xYk2rXYbaox6oVtxzz8EuW\n1+6xpq6ZbSmhyT1I3cNPJSmkqrjNk7XFtPsEku/FWFka6OLxzm7E25NhGsU8DpbsMOoKrTpRflu+\nSJVQlBvvhFbUl7inxX+Jp9mG1tjdXjsDjauxuVE8YyCnboEnw8htSfBdKcdKUk3TuyvGSKi2qzjw\nOKjaqJWRrwOErt8FOXApyw0TGKtNy5NZLOx21ukSmm8gqbyLUSbuJynGaJD9t3fBM3z8EK0uxW3c\nzckjVhW4xeCU7vsaNZqyJKgokOXaiuDJOm8lNtpvkM6NgeWlYPdeCU/c21WPJaljCLqRYd0s8g2v\nOSG/gcYJ8vsWT0yY9Vxbqq+ipJJJeMCpJsTlw1xRZvTN00k0hcNYBNJ2O1JXXHk1mGT7XcazzYSi\nmnTElFZVIUvcucGPj30Lova6KSsz2Nxec0OONPPNZN4po2Nyu6j4oqksLP4FaaWfkbeLjJKmXa3S\nSVPHyXBrCJhJNPc0OLxgOzIT03ubBR24d/kbxUr+CtOpL6N94sRSd+UaNJNL5FsqTTpohZk88FO4\nJ/FxUVqX8VYSjWbeQaUVyElu+u2S9GX01TVJEJO6sSqDas09u2+7N2SLERTUsuzV5VLBEYvdmsMH\nHKdquKCs5qpKljuNyqOG01/Yk4uLTddgnFNxrivJWNXzAtRtN2/gUZLdfdGkox9tNVkzlFb1xT5C\nyUSnGTSX8HH6nataaXbFM684qsYo5nWJLVbvNBynTUjywXFm2mkjDJvpdjnFW2msWzVUKKSQRTz3\nNbLGbq1JK7E65Ft4yDjUc5GTVBtTp5Be2127D0/ammKUtyRrN6NVBOpbuLwJJuxRbqvkaxLmzNme\nsiOWU3imwkku6oSppYyW6Z4rbFRdN5FSpusrAnW91wOVV2yHYwnjjI0iY+6OPId8FJ+o5c2Ldtt3\nVBJU8CUG2auE07St2KkmNuorcLuZk0kuU1dmqiqd5fYzgqTHq83f4H7W6E0pOwkk3glL3NilzZX0\nXibgkrTdlQl8sINVxbJiliu5e+qRTdt2yUuV58BLt9jVvhFYsS8yoItJsajTdvKCSecr+BxHd8dw\nnVZ8kRcrylRUnTToLxxE3m4j3rER1hOkZ7XzgZJXTMjRLwNtVkmDTdMcoqngzZ3jMtKKTbaQpvbK\nvkaTiuEOk5Z4G9CTsnFS+iZpWjRUlyLapReayU5Nds1+/F0XdLLJap5kJwTjyNmnD5TZUGk+SVp4\nryH6OHkzcYsNxubafYmKr5Evb/FD3cV3M+E6j3QUnhE47i/ZmOCkUn6LS58lJx8kZmr7Cm0lyN40\nqcVJ8opxUI4M4R2+KLWZcoL0XtfN4TM5yiqTyN4llYJm03YSdrMGHG0S+VnA4QbzmgV1nJrcqtC2\ntYRou1BCqxRMpSir22G74vY1UFzV9kKUaTIhKTeeC5ZDbKO0Sp+BYuqpFPTSpjg1J1ybvIWYIxjT\nwWqUKX8CcVFc5vwTlJNO68hulU47sJcFN7IvBClJ54KlFuPus1L+jv1HufcFURq0mKk/n8GtW/TS\nVSX9lRXFJEQdf7fgp3yZtzofxnq1ykSpO0k7LaaeaaZVrdjHwdJej/hZ7UXCW1JJKuQ3Nol0rM93\noLtP9qX8CinbTfPahaUlFp13Keo3dJh34O1QWGmu4NJqmEZ+BLu35LRZUSXuw8Hg9QqOqnzaOg5R\ntpco8PXxeG/oLf1p4JU2zTQSu7IcEsFRw0kct/C9EZdkaxVLPHwYpJJYNf3cGpcFPdlOh7t2KVIU\nYNy+EXOMYqornua6WFe6XBO2peMlw9qsmS3NOsjOWL7U4d01RGm9raf/AAKKe15G8VhtvwW71U01\nFgzTUa7lt0nS7ErOTMudClas0SXx5Ik1WATd4NXB4aey8YCLba48lNZTFH2vLuIafVNJprCf2TK1\nwyG6dsTlSvuOaMVLPcFHyJxtNlpK8+CtwwP9uLIllLgpyjdPuKaSMaiSVFUtqdkpU5UKOf3KkPRk\n0/a3gUsYoUGO7aNRVarFkRdP5KawQ4uzN5b0saNJq+xMnS+hPx2GkqK3EOSZPasqym/aJJtothw4\ntvNCnhjlKsBaaGLE6fOS5K3/AOiVSXA3zgN7WJbbWEDVrDplcLBk5OUqVmt1YpqUe/8AJUcxdBGV\nv3dyqS4YWm2pUHv3fI9t3YRbuxb6u0Y5WrsOSTpEynXHJSalZE1TqslL+jKMSlwKVRkWuGJtOyti\nypy4ulwLnsiouov5E2077BafRbikkqJcbywTfwVVr5GUi8Ulkz7mkarKJeXgJcqx6XJtFwSWecEx\narLdsFh2PosOnQpNptY/gttNNN/wJLc7pl/q+sZvJtT2u/BLjnj5BSbaQ3b4orisg7bvwLcld8g5\neFyY7Bydp1wLSbV3SHFUqZLTVuxnfSW4722uBtpfRnHUadKshzLJv4XynxXdVRUm2qruQqVOxxlf\nFMrArKQ9yUJKrkgTsmTfdIZx0HGflDu7JSt1wmOkuHbN2RXsbm40+V3FujF5bspNJpNPnwTLbJuX\n/guP4ltJr9zFJ3w0RGXCp5Kik3ykOWLs3XDoSe283jgHKnkqCTlfwW9KHFx/T9uGvgUnj6HP9lRw\njOMl5yMmpoo14/J5eu/cuyPTyjy9ZH2rGEc7NZc/Ub4SwOGXTwE039BBo5eNY9EWvtI1TpIxi6ia\nKWaZr041jL2pc3yGo7i6w6Ih3wVKopW+SyM2pTdUxq0u4Xx4HdL4sTA3J9lwTuqXgN7TLiluba5W\nCzFZjObuWEVFrbQSzJ4CKwzVyxmFT+TSDVq6sz3NPKwFO26Czo2G26zihtyzRMp2hRw33sZOlnQ8\nJtWFKs1+Slt22+SZU13KULV01eBcZVPsOuKJm0vLM52oG7ryKXKbCLvlDlldqH7axMJ5xxZcm5Nd\n6wTUaSQ21F0VnfQLa6XgiCe/KeDTfF4yGLxdjpEpUNO3ZMvlYCLpOu5nAHJ1XexK3zygkndrI4LL\n8osOBSWU+xamsJIzcPnBWFFU+1FZKg8u2NUuVyxLPCFqSaL+LVtxryZykuyRVprgmk0WYczs9PN+\nAUabYrw0ildpdiG7STxkiV7roqeHVsemlJO3gihzpK3yXarILh8Ezk37ewenVWuwNx5lyZ7Xu5HK\nIWRSITb7lVJvGawFKsjg9rx3GYpCt8U7Jd90W5rh8jk08hn2vEdiW5RyXWcMpFuHWSm/A7a7DquR\nYTKZU9Uk5fVjSSWWwhaVWhz70kat7xm8bom7S24kCcrS7UZK1L47GmaZeNSBSa5lQ5Zl7WgUFttg\n06uPJbF50Sf/AOSZW7y0TCC5d2uclOK8FemKmUn2dkxtvKsvljmmo+xWynSkvrN4fFFp3WWyopNP\ncs/ZlJSjO/8AadfVe1yqTaLSaqjKMbk2zScmpd+Cs+lOlq8IUmqV82S+V5+xPCuk8mZBGqyrfImq\nymKMrj9CjLenlccGsvqp6j+G7RnabqmjTdSohNNP23Lyak6S3GqpiT7SDdLascOhTV5XnwE37U0b\n0msYZayrwQk9vHDKUZXxjsa6XolNp0k3jI4NOPiwjVPyJqnz37gLpvduWVSwZ9Sr0pJ8nok47c8n\nm1fdpybwZ9MmuXPdF0wV7XaHrdiIuu2DlZ2cejSdxds2VKjHTS5S5Ncq8cCKpOqQZaqTsqWaq+CV\nHnJrpKf7cDjmOX+CXcU9o6zkMwWiVOTrCGqYOOMiji0x9MpYbu+PISlStfYSqn2scaUa5LUUX+11\nguUk28/+CKtNsKw6DdGDD4YL6TyPtwCVdy1eEn2dA2msEaldqeRwquDXx6059rTaTS7hJW8hdNYF\nPJz3tDdSrAJ3H4Hin2FLKqxUTTXISVU12Em0/kuFt5G0VPtlJN+MobecAo1L4KaSpotlRPdsVkq0\n4+GPxzRSV4aK9KCLaYN5b8iva/IN7uKoy16Jqo2iE+1A7bzxRVfux2NSYD3LhMTe61JWvkUY+5d2\nVJJ3ZXDqGkS1lU2abfayEqryMp9W44xhk7ppcDUqVPn7Hd5SeQgwRqae7kNtLuQsT7pGm5U8ozy1\neo302rDloUqbWSs1dcD8ejlP3W22RLPcblLa13fJPZIJMUmKrAo2rBQp2m/4Jc23SLN8X2mVylyP\n6BUuR4bL5ZDYW5rgFIr9N5rhEvA7KpEy1Guw0935BtSlhAqTormJ62pU3ga39wk3hRBXFpuzN7hv\nRxu8/QN/aJbcn8gk+947FJhtOLrGcgrxz/IndqkDnnh2mM7ZzWibWO1BfFomWUnYounmS44LNZnq\nr2pUhObYN3LDHhJceB+mqTT738iu3TLV3Tap4JSVmuPJlUWlJc8ZKbUnclZDaTy0KEneeBze1jSU\nbppOibw0sFXaVPIoySbvuXGmFFSin8lpZb7/AASpOXCyV2tsrv2Kncr7OyrSpVz3JcVSebY8080a\n3o4pKuBRWc4yG5v/AHCbymgjPYcs1HjkrcsZ/AoVK8pP5JrMlwvg10YrbghKSlHiryVK1G008cMp\nO1nH4LVYvDTTX5MtT9uFfYpyp/gjUlhc8+TMo1zNSNt8GVZRtrfukZRnRxtumN9PCw6vyaO7TsjT\nlcVeGaq2+xqX9S1wrG3jJEd0pKMfwfR6HoTjoQnrZlLLXZFempxvJ87Hv3QLNs63UdCouUY4dnK1\nYOFrgZylHL/nYUm7pdhJvDdWCdpXyxz4VP8AofLjBNuTtdioVV8mae2VrhlcZTKzCveksLBKluaS\n7i3JSav+h1TtWExG3muAl3FK7bspu4u3l9qBnGbp1gqMfgSX0F0ht6NOWEDcn4E5NpNfwOLbTTMr\nwtzTVoUqcW7zZUsImr78mp+mqdKhK6Cfm6/ANyUMZDVn2Sk0/wBoKbwngIypZKl7opurH5Hooypr\nwU3csIlv5Qk3KqlgL2Oop0s/0DysGNNvlsp3XBqyFafZA5VXyTF7ZcdipNWuMB5VVRnngTlFSfIm\n7SaRLp9nZSbdKt2bSYbk2rEk13FKF1fKZXBptprC4CL8WxpecEuorBmUzuKbb5Tohr29yoyd54Cb\nTXI6Nwl7c5LcrT+iVlUnjuKTSVXkN1fK/aXJpvFjTt2wi13eAc4u9iwW/TS20qt8meeUuCZZqwvA\nTozKNyqQuXYSjnkbwis0G5tRaTWeSU2N8K0vyKLflFISd3ZcfDomV3bLhKKfayo8ehp2qdBfyOcr\nVLlE3SfkzJQLalS4NVK420TGTtN4TXcmbbfNLxQ+nFPKXBMYtvDVhpppPc7b4LktvctkuKBQrTrw\nNqqUfIozVOxwlF5TXIzRnZNe6mNJJvHcUpRVXljUreO47Qcml+CEotdxvUz/AORQ1FK1fBrD8Q4Y\nu6FjdS4NJaiSWCf1IylmFI1Pyr/Rap4sIri0XCSzh0DnTQbfIsw8LiLJj2b/AIBSuTfwD3duC7+x\nhtW8MEuzyS8SbscZtRzkfj0cEk2/ak/tiSadvARlK03SQnLc2mjU1a227ad4omXLXkNy2Vz4J3Nt\n/BmS/bKdlXasuLvFItTVe5K6MnKMntSoe6Wy7JpGbqS4ordFJVVodR2qw8c/tz9dcp3k8ceWevrF\nFSqP5PLF1Zy5THXxpp8nqhWMHmhJukuTaMspP+SzQ7PoHSR6v1TRg1cVln3WtoqGk14Pnv8AA+mb\nnr6zT7RX/J9R1O6Kwr5NydO3/J8t18Nkm33PnesjHdJJfk+r6/Tvdaq8nynXR2bqqqeUc7crrZ+v\nDHNK/gbxzbaJ0v2p/JrqZ4+jUtrycvei7LHHclRVtp8i3YdtDv5+TfYw1nnkLfCVsU5Ywxxe3lma\nrIazdumKSfkamna4dkyaTayU1HPi0kCafKQk04tZGmv4DwGqXEUDfjDE5Ldhgpbk65LN7WJTSfFs\nptKsDSS75oSlfLLThunHIkwk2lglv2ryUmnOgW6awTGV/kaTrguUwYKS5J7cjbT57D9pCjYkrbHF\neeLM0reBu00k6LDIqSTdpkNU33RpH2rmxYcstFNjQhJK1QbrvC/BEpKNvtZUWmroroo5du0PwS5O\n84G9TFhhO6lkFBNuxJppMSmk8cjlxG40x7Ullic74dEuWcuw7Viv9zJlpu7vkrevuwlLGQ7izUbP\n4HGoxoeaEnTZdqxLy2KKNpNMxc4p03Q5b4bF4qV89iVTY4tUKld2WYsNK6T+uAkklSQX7sWvoTbb\nbtmNqhxgnG5OhN4aXAn9iTfAwvSp90v6DdcrxySpRi3SBe58Gs1Z00lJvxQm3FctJBFx8A3WF/wG\n4xGsXcbaeBuSrC/sye5Wr/gM0kHxO4NydqiopbXXfBG1xVmcdScf9uLOsm+Lu9t0vK+A3qMvFE72\n8Mcl8Is/Vn6FJPGClST2oy21nhGidR8jZiwcPOfour+jKTfZotStd7L6WHVvukLUSXf+wc1S+gUl\nRTdZuqjFVl2G1tEN1m+RKVpjeOj03G28MqHHHBP6q72EZ7pNrihm/bWqTW1/IuXn6Jtp1yhqV0xy\nqKi3H8FSpQvuSpYd9nkJzSi2qX9mZLvY9OORuKtPKM4+5unx8GlWtrZq9CpTjyxykmsPgbgvAtiS\ntItieLqo1JSy75PHJ+99jodZFqCb/g8Lit92cudlLaCXJcb7Iz0ms/Btp1fJjjcp1+kf6f6X/wCz\nKbVSepI7fVw2yd1dHk/xKH/T+i6EVw47r82bddqb9Rrs+52n678HC9Ue6Tikr8nyPqqcXJpYbaPp\nvUZRhF0238nyvqM99q1XwcrO9dOV6eDTlUadFJNvn5EsVXCBTzl8G5d8eTlQtOk35yRHEn/Be9bV\nVi3JqkO2eszpU1UexFfZTkqJbwngJTp7fJO6Jpu9rp8nk1sLAfKp6v1IYEpJvD+DwfqNulwVHUae\nGGrHtTinzkPbF/uPHLUTt1ZenNSvNV5KXQ9Mmuwk1RnHLNVFpG5iVFXf0KMVTsrsssmKeaZn/FKU\nvbF7UUpruQ5Phu6Jcvg1eOtNb7BT/Bm75RpCTlHP0Zs6WCVJUgckweHlEVSilWA4yCRcEnfdr5Hs\nT7kK0Enl8ob/ABe+G0o4Y/bTr+jO8q3f4L9q8ZC9NYmTSxX8kr3ftrBpKVxwTCNO6GcutGLinS+E\nRVW6afyNTdusCV1lmdCsOqdYG1jgztp4K3cjeinvwDeR7k2/gmea20MmlpCmu35Bozhd8Mair5Zj\nOxilFvnhBsi2+A3NKiectFq+g0ouldrsROXZFOSt9hOv/wDM1uztubTi/sJOkNNJYXJE1YdMqVZs\nbSRKe3A73VyFhnb0Si7txBtdlX5FO5dn+RbFFVJJJlP6lRoduuCXNKGLwUsRp/8AJmjD3J+RTvm6\nRKl48+B05N3wdZkGK3PGbQ/07dpGftTr+MFbnnHC7Fd+movbt7Ilzd5X4JTpLnPkbpu5Ky2z1WaF\n7sOufBUWnh4+Am7VpJCi0ub/AINb8mcW0skt0uEG9V5f0J5WUwksWWHX7WorK/scY7fz5DdshTVl\nXapJIbb9LsTVpX28Gcfa1djzJ8qh6qwqaGbEWHir8lVWYpEqK7t/kc1GKRdgRkq+RSxFPsOKW20u\nAcvbTQyoU3urOLI7PBbkr/8AROLfyM5dj7bQitr2t5r8A1l5M4am3CyhtqeK+TPLdCrltdZJuUYv\nA1i0LfUlapDIXm6ve9P3cKVnib9x0uoalptNdzn6lJqlg586RB0275PZ00VOcErcpNJJHijng7/+\nKdL/ANR610sErUZbn+MmZ2ZNfqHSaEtDoNKFP2wSuvB4+o5dWfRN/p9Ok/Bw+vjFKTWDpepjrx2P\nlPVnF6ve14PmOv8A3N1SPpfUtJ73JSyz5TrG3O+xjt05XpmmlFESWc4Q06j5yKTUr5LjMeXlDpMI\npReGhds/QRws57D6s6Dascoul854HtVk7Yq32KdiUni+eDzTk02qN5uKeDLUkqkki59H15q+B7S2\nqJbwc9rRcFaSy2iJZNtGOGx4s2PVpJV8ml4JhGMYK+fkjF3ao1LtWatylLixKUoKmhw4arHkU5Jc\nl/ErDbxzkTSUar8hCV8pruh4a47l3Fi4xpEuSQqkvoFLnHAISlcflEJpvKZbV5T7kv8Aaa41d6uS\nuOCax3bFF0leUPdzS/gCaTv3IT/oW74KgtyqyzFrNNLyXuwvsbioSV3ZLHZUpPPAStxwJ/mnyC9u\nfJnDiVuWKuxy3LITeMCi7+wy+qh5vnIlFN8f2OUksSQ0l2RrbICTrhZFeSk4uNUkxOKWUg1aTlOs\nDi3eWhQb8OglVJVn6DfojUbv200S7V2O6dJDknWURmiNUO8k9sCjblQ/QqpeUytNWvon9PuLc6qv\nkzts6E6b7pN4QSlIFcWrRTtteC+WHlcTmsoHua4Kbp0yYu3l5+B2+r5HBNLLYbmnzTRSXexSS281\nbGctva+Wpbcsv/gI2mWmlhOxOUdyTtm7ab4pttKuw+I33JhSbriy5S9v7QrFQ5Om2DdtVwEm3H2r\nKFFVLa+y7Gp1FLod93z8BCWK3OwcVbtlxjeOw3k1arO128ERblbWBOcUnG3afAQlxV+AjPZSi9yY\nJ4W5JNGu22rfcJ6Vt1JUM5/ovJD4TClKLdWDxzwOLpfY3lfo/LDUsVHBKtN/+yl+5OgnFJJ8FL+n\nYiNOWW19I0kvhmeM5f8ABcnhU88ByjN7LhZyyk1BXVMzadsTk8V/yOamqlbsJL3ZM48/nwaSaw+5\nXpMdeNwdYPBqNrB0tRquTnan1TOfPWhBs+5/0x6X9Treo6l37Eo5+T4XTbTP1H/T7Tj0voz1Wvfq\nzcvwsGZq+312tJ1TePBwuu3zk1ij29T1O+6lizl9ZqVFurwa+Tvx1wfVJSUZU6aeT5fqlcu3Pc+h\n9T1d7l/B811EnuebdmOVo53IxnaTpY+iIjlLGU8jWFyb248+nKnhEyXYE3u4wW2lANSU6jT7jVVW\nQjteLsjjnsx2rU6iVYyZ028xNU03kp1tK79mVi9O1gS0U0b8LkFmSk3kzh15v01x3LjFpJI3tVwh\nRe4pAIt1m/4KklJ4RG735NHSsrc8JJY7IHG7bFabaHf/AAW1nQ1heRqKq0yeytjTaNdkrfArv7KT\nxloSzLkFKUovuN4Sq/4HOS4QQfngpbZp0k07KjcY4YNRd/ZFUqLdO6VO7tlRdeBdssI1eRv9HauU\n7djp1hB7XjIm6eDKtN2kGa/+hO9r8gvLDavkJJtENNJWXcRKSfF/yU5Vazab7jVLuaJwb+SJZdVy\nanLej0V3wwuXkqq+hYQfL8UpR3JO33Kcn9kqSlwwp/8Ayv8AAWfdVoTHubdWJUlljTt4C1aSbbaK\nqib93Be61TMnWbbt8kvck2atc8mbvvZuX8OPVLPP0QpNPC/kUpPja2Cu6af8DOOQXjnalbbvv4KU\nP6J4y2yXKTWG19o1mhsuH8hy0mrMk2o5G3KsB8cEiks/tQ0qdOid+2ryNtvizUtWU1FRk2hSdvDZ\nNzwEItu5J/Y2fZbQV8MTi7lJsLqnGxSblza+zMFgjNPCG5JLlEXXBpHas1eByIlFZe1NMt0opKkR\nvpU7oSpu+xS6PV274vBMm3fHH0Dlh1ZnWW6NTjFi+zfYcWlGssmLk1TvA3KksN/gbCmdOuUxxTa9\n1qhrdF3JFSeCtt6CKXcafi3REE03utrtgpUvh/Q38C3N8OKQopOCSSsUs0qZpoac9TUjDS05TlJ0\nkkExSaiCfwVxM+k0/wDG56em59TOpVeyK4Od1PpyjqS/Tul5M/KWtThXI28tpnh6i3J5po6+toOD\npp8HK6qG3UlaYcrqzEdPCWpqQgsyk1FH7B6f0r6PodLRkswik8n5h/jOj/1HrXSQaxGak/xk/X5z\nj+ji8vwHFrj05PUTmpPP0jna+pJRlKadHs62Uv127tXweDrnP/p5pXbvgbO3fXG9S19N2rpvwfP6\nklLUfLOj6hFtbpXZypYd33Odktxy50OrzwHfgly88Fr4N2dOPiKcpXf2VjY7yw72O8ZZVVHDxwEp\nJcsard5Q3TTqy1IWcrgrtWCY+1Dy1x/Rq3abA8rBK3NZHVYLhKrpGbcCGkkuRpusIc3b+BVWEU8J\nJu+B5dWsCbvkcW0NVoSWLKopTw6XBm3Jt8UZm0QWxkxb4ZVurodukpJZKSaXx8hHLTKlF7VSG3RE\nNu+EGXkEvgabXIX+NFTyKTaVvsNyk/JMk288FIlRxJ3VFW2uCILPLsLp4bK9r1ok1G+PyJ5eeSG3\nxZVpruHxU49jOFtf5JfJUZSaxf5RF022UWK2p5YkkuGhSlfCZcIIP9XxpYxeGN7XxyTKlKmwTSXB\nZqgcsNMl5XA5JttLgUcVY5nh8So5pI2UccEd7uhtyV5C3Vmjh5SYbSZyb7gnJLBg9hqmVDy3gh7n\nkItrk1hOepnglNOrTC7bG77GvIzrb3XaK3NOmvyEZ5zKyJSV/wDszG7ylW/mxuSSrkhqTdtqu9FR\ntRyrNY56cU5XwsdxqVS/20EdRNYWKIn5VF3uGcsWxfqU8JPsRHUX+4bkk7SNSfp1e65YQakm3UVn\nkmE2+2GNydjmM/ZrcooKfdttltNUTNtrkJVvY2vGKfwVJtq2ybbacmsE7k24mvVauM6TT5ByUlTV\nE1V2wvOEHXolO/hscHa7ju8PC8kOs7WWrTrL5DjtZenSVvNBKWE+S48tVqd9uvPI4y9tIVLzQrSj\nlmt/Boe7us9gV7mn2FJ+5yTuJV8usojKuPumlHnsvJ+nf45/jz6LpYas1evqRTn8Xmj4z/CulXW/\n5B08ZpOEL1Gvrj+z9i04XF0wyHqPlvUNKUN99sHC6rQjK2q/B9T61B00u+WfL9RJx3LLzycb1W+P\nJxOq6dtSdJ18HznX6Li26wfW9Q8NI+c6+lNxsuXR5ZXp/wALS0+q1uolxGO1fbPsl6lGUaT48nwP\nQ6j6fQqDVylbo6Ol1bS5qzXHl1gkx9RqdQ5yxVM8vW6qWlnxyjm6PU71UpuuCet1f+2knY9Ojnep\n6qak28c5OO3lu8ns66bfCR5Fl9kZkc+VLL5SE5KP2EnkHDcb1zP9VNcZ8hv8r+CXFRxgH/A3D00U\nvbhCk+K4JWEgUnu+DH2BKWA3NpJPDDC4oE1wl3NeRaVNTSRtGSUaozcn8CU03lGcvJHK9zrh5JUm\nmrWL7FOclwOL3ZdcjfO0UlhNcAq5S7DnflCtXgpbi1SflUhX4oV1HPIKnWC1HuaaCbuqtA6J2pY7\nBCtXS8fJSd/+qM0sLJSKwG14Erm6tLvkJVVpkrDw8mZ2ZV6knFLuZrc8tml3jApJ2anLFpQ3MHab\na5KVIp008oLRrP8AUaSwrDStrJLqkNfuw19DvTW9Li+aE+4RltlxyOTT4sx2zspLhUrbKi3FKyFP\nKQSq+cj63KJ+7sISdFOuaHxlLaUasmLee5VK2weOEOmdmkqdjlIjlO6saijFPUOl4Qk2mDxxTFfL\nbRnD1ipyaWMMmMr5Ssbe5OnwZpN8mp4pBJVmJUZMTTiuwottWat6Vx6rjTurCO1q6Fi80O80k8FO\nmbFJYaXBE4+GyoScbVEyVybSGe6zh6WaUrQ5Qkq/kKknfA0/krbaYNkZOmTNLCjlIV1KxRT3OW60\na8Wmr7ppfQ043aZUpWu5nFpYrual02tXqWkvyNNOq5MpUm6CMttJdjM4ysZrR0o+7+SVurDE5XKr\nqzSGH3wa8IinLnNBJU8fwFrNvJm7d5x9F6cabJVy/NDhH3U2mZrc5UpOuw6qaebQWUYqaawk6Jt7\nUy5TbS/8ExxGmv5KdQGpNPNE7E5YaZW1POP4PR0/Sauu4x0opybpIdkUjzU1KmUotH0vQf4rr61T\n13+nFZ2tW2e+f+PrQVy2tPP7QvNucNL/AE36eur6nqpJ3BKEW/nk/TehmpRk91ZPzz06Wr6Zpz0+\nmpKTt2u59B/jPrE9Tr10vURjGU/2STwzneenHU9V0d9+Mny/WdO3uSWG7PtvUdFyhbSTPmvUEo3H\nuXL+KR8v1HTtd3S5PmPWIVrfZ9pr8Sxijg9P6ZL1P1vS0E7073TfOFyvzwY7tyuk468voPok+q01\n1HUJx0P9q4cv/o9XqnRaXT6V6UUqdH3ut0sOk0EktsUqSXZHxPrmq09qw3fyN4WeN/Bwoa1NtOid\nXqJKPKeDya16cmu1mOpq48hNc71TnLdLkjvhijJS7BWK7G5HK3acln/6GvLeBS7E5dpOvI5rNmrb\nW53/AMFOmqMW3XyaL9rxTCxYnUq8MUKfDKarbZFVaNTww6TbLUUkmQ6SzKht+xJMLKFYqmvyS6Tx\nZKTk+WU1xV8D1CdLldxPHCKi65wJvIaFNWuMi27X8iTyS2u9h39LFS4CCq7aFK3VDTrkfpCv4BL3\nOrK30m+SG3J4wMmnFR/GBumuCLoalX2Zsow8VSB2+5VquCYqpf8Agz4gOL4slyfwDkkhxSC/dzQN\n1xY/33bGsDSSykFU02g54RLfeRn1qKfu4JyuRwkm7obfgu50zmB3Fcck88UOc7SQrSq2OdGCKaeS\nk0vkndufwKMfBZ+m3tV4whNU+VRaW2NXXcindF0LQ6fKC1G8D20iJO+wemTTUk0wcU1wKCz3sqdo\nzeKoVXhEtLwCm+MWNW+DU44ZEtXeceRVjBatLLsll/FlavLw2OMvcOO35sl4zR2kONG+6RK1qdbQ\nUpUqQbLjmrCz9ZyxS1U13/gndd5z9EStyuxt0lnLL4yeKLvdGryQ1OPz+AXwy1+1ZNeKw4TpK3y8\nkykrwhN8JMG6dBmMt4OLhbaTM2s/+RRVv4FJ8p8dgkyrMLLmtsuOTZS+bMq28YKTp2a3emjqrpK7\nLt19/BCp55YSbTqOfovelYe51arHlCk3eOQgnJ+55Oh6T6V1fqeq9LpdPdStyqkkOZWcrnxTfc9O\nnpas4NqLcV3o/TfRP8M6fpVHV1YvUmkszWUz19d0ehBNfp1XwY5888b+Gvy/07otbX6iMdntk6X2\nz9R9N9A6fotWGjpS3ShHLay2zgdXox09Vz0ai1nHkfpHqmtp+rqerqyk9RrdfdnPlzta/wDnj7/S\n6BRzKODi+vfp6ctsE7vKs+mnqxn026D5R8Z6zqyn1Em+3ljl9XH1yZ5lVozWqtHWhqRXui07/Jq1\naT/8nO6t7G325MXi3/H6w1+v0Wlq83BSPl/VYNaksf0fWf4+49T6P07k2l+lFf0cT1rTgpTSlhcH\nSTpjjO8r4/qIyp0sdzo/4hpQerr6yWcRyjLqIRTdM6H+Lpf9Jq7LSUzEm8nWdPT67NKDSTwuD4H1\nNrW15Sil4yfbeqJ6l23R8R6il02vJSi9vY6cp23vT5v1CDeGqds50lTalKjv9Vpx1XjxZx+r6dqd\n1+Tlnbjym9vOrXDRpDPJntcZ/BblWF/wbscc0SaSyiY03YJp4fJMqjVDJZ0sxd5WOCnJ9yL8CttL\nBWBe7z/wS1cmwapdyU3zkIF7aWPAv9ueRp+SWsrih39KotK+2AUqoiPn/kttNpdgFgbu8jlVJ2Q9\nqfLKSuwJJpyVdh9/IpRe61Ym3F4RqT8RvHwEW3LyTJ3nNMqDTXyh6OKzw1RKT5E065oaeOzLQJOg\nvOFkTWbDjkr/AA0//QN0ruwTi0rBKNYujP8AoFu8Kq7jf7cjUva/BG5J54GVeKgnxY7zQm08ol1z\nYXsrcvAlm1fwZtpSwzSLoqYF7R7nx2FJprwS0v8A5ELFrhN3ki/ch06RLXjsUUWljyS20xwtZsG1\nlsPs026w3Y1NN0jNUwtBn6viqUqsmLr6GmkEs9mQmmpS+AnNpEOTQnlLgcbxUVm0VHdG7oiKfaip\nNtFb2sDl8BdGS3WU3gc1bW1NO4yK/cs8oU181bG4yWU7TR0it7TCTundFYt7VyKFbso13q0g5eip\narlES5ysGmonLlpfZnNPMbTXwa4SerOwqatIe6lQo4TjS45ReykrNWwW/Sdt0VttcFOSt+CL3Ol/\nRiapE8BVy7+C4Q53cBhNJKrNfLQnblWNxbrJW5MmSdJp/BRQ/wBqd8oNKV9mlyDjJLJ7fTuml1XW\naOjC71JKP0Oxp3P8Q/x2fq+utXWjJdInV3W5n7H6P6Dp9HoacNPSjCCVJKiPQOh0um6TQ0dONQ04\npcV+T6L9aOnHlUjjyvyq8eDq+njpaX0j5H1lJSkkss+36vS/W0nJPDVnzHqnR1lPv4D3prcfGdXp\nXbkufk4PVN6GvGUeYvdg+s67ScdzksWfPdZFW/CKR148n6J6H1Eer9J05Otzjmjj+rdJtm8Ef4F1\nK1NDU0XK9suPB1/WNFNNt+TfGbGL0+SeittvDXyc/rpp6dVhKrZ1dV7b9qycbrXqPe/9r7BeOqPt\nf8X9Ulqeh6LTalGO1/jBl6h1G9u3fc4P+L9cl089HxLj7Pf1bvKfkJn2s72vH1GqlF3zZ0v8U1kv\n+ognh5PmutnTkm27Z7v8V6uOn10tG6epHH4DqXXR9R1+E6yj5D1rp46rclFvUPruqcXpu2cTrdNJ\nN4fZYN7DI+Pend14o82p063SUo2dxdNU243af9Gmt0qUXiznembcfF9Ro1J4weSV7sLB9B6hofuc\nVTON1Gm4c8lK48o8q/ddAsf+hu7yOKwjXycyddgTsdJN2xKSXBe+JVbu7oKjFKnyJS4sTdchlBN2\nlzaG5Ytolye4Gm1WWMhzF5rglccWFOggqY3Ih+BydpD72hVXyXpNNtW0ybzwPdigv22rBaO/wK1u\nugtqsWh0q4GobrVIS3U+eCqJld1bGYDjbjbFJtv6BJ1yVttKiU7JJNYFuV4eRv2ol1fBTsnbdYYL\nN2U8KhXSwgNoXekLc+KG3d1gIfLA/R7YySb58CaVZRTimuSfxZlnewmlHCJbt8FXhAlnjsa6V0+C\nX+4p545Iz82UMU00ndEe68Mtr2tpkJ3l2MpgTceR7r5Le18kSUbwzOkk80jSOeSE0mNST5wHoJt3\n7qQNU/7Ke2SSBuN8luAt2OKJUb7uirXgV2sYHj21Ipv4Jq0NceSGm+A4+qzHp5jTyGd39Di1FrGG\nK4m2LdEpUrBTlh1j6C05ZRTkmkqr4KU6N98omvdhA0nG/ngHJJ2uBkXQi3Fuln6KUl3RKbc8d/JM\nrk80Mk+1MXOduqQk6XklxoFFRNdfS2L/AFJKSwKSbd0wdUq5DfeGVyeBaW3tYlJoFnmwm84ozAre\n39n1n+m/RPq/8k0W1cdKLm15PkYqW9NZP1H/AEi6CcdbW6yf7JLYo+fkOWSOnGR+gw1YaFwqUVfc\njrNdV7Z33o9vq/RQ1tGMqax2PnNVamm3Fu0YsyOnx3x9f0clPp4Refac71XRuGPsv0fqoz6eDeGl\nTs26qV8/Q8XOx8b6loJQlf3wfJ9epLU2OKprsfd+qaO93R8h6nDbLdQ3PTK8f+LdTPovWVH9unP9\nx+hdbFamjdYaPyzqJ/pdTDUi3HK4P0bpuq/W6LT91txyZ4cu8b5d9uB6npbJulVHC9QjUVSyfSeo\nxadv+T57rZKTeO1Kxt7Zjnem60tDV1YLLk9x2+n1566rDPnb/T6xSvlcXyd7091p3Ginfrrb0XX6\nVRaaz9nHjqanTdTp62k2nF2drrWm223+Ti6ubqsMzykjUj7Xo+qj1ehGcXHPK8Mw6lSlhpKj4vo/\nV9bo/WdLS0ZKWnNpTi//APcn20Z/qxbqvsZynjG9ua9OtR5Tf0GqtkPLPbqaUU1Lu2ePq41F0guD\nl24HqEXvbrk+b65SSS/B9N1cm3K7aPnfUF7KvuHyjPK9OY3i2Pcr/BMlVq+ewJJM1krivcpK12En\nlk2gTV01gsjPiktzu+HYpJtX2saq2kC8MvFoSvLxWC6jSzlktt14Eqeb/sP6rT+LslsdK8UKSpLJ\ndI9yivgTkq+gmnjuSkmsmpkh1SdjTrnJFU0vmy213QXtaJXJYVBGFctjTurH3eA3OkXC4ZDaeaZd\nrx8ikMuIJ4qxJ1wJLy8FKopjsMuE3bsTbXCbLbx2Fak8FLAUW3J7lgcnt4G2lgTjuyFzdKIvdeCo\n4TsNqi+XkJN4Q7Fqt1PgN1A5LvFMhNcNBgabr8CUm+SKTfct0ldBsVLtYKYQ5zhDlTSRafE7vsKV\neAdXaG3jwa6PhJV8iznA1NJj/UV4SaM06zdlD5fCQ0q8F0dRXLoM7VgqN2P45DkOk520Lgp3YYWA\nlXyNStYFKVPArRPJqZ9iXXqeUZ1teS5Tbi3VV/ZlufNHXjL9rFtpvCopxojf48eC5T3U6K8c8FmG\nsJNkuOe1Mn9R3UopJGkZptYM5yhJ7lHixOLu3/Q56lJ0v5EtTc05YwMlrJrixJpvPYd+GQ5Jcl3q\nkXWMExjnGQi3LjgpWuywb/lOBXXeykuzQozt3SLU4tcf2Z5S70sez0roNTrus0en0YvfqOlg/d/8\nS9M0vR+hh00LceXLy+58R/pP6VHU0dT1LVmoyt6cU8r5P1KGppQhhp+DFmu3Hj006yf/AGG1wfK+\npTcYOSV12s7XVdRHbKpUmuD5j1LXi72uzLrxj1ei+qQ0db9PUtKbw2d/rde9O1x8M/ONfX97cW1R\n9R0OtOXpWk3b9iechLVz4H1evl7Wz571WG+Em8fZ6+s6xx1FXfk5PqXqGktOScm+1eQ538csxw+q\n5lLtE+o9B6lz9N0s1g+I6zqJSlKko/R9H6Fr7Oi0YLNK3kzNlbk3p3Otuak1lM+a6mVSkm3g7ur1\nG3Tbw34Z89109+pJrvZ0+U+xJ25XWzjpvcknI63p/Xaf6MdrV0cHr7UXud4zR89pTn+o9jkn8Mzr\nV6fe9Z1UWmm+eDkdX1aimlz2Ob0/62rFRuT2+T36fRuf+2X5Rd1veng6KDn6loyeJOdtn7T0HS6O\nr0kVKKuj8ql0n6K/WUMqvg/WPSJxn0elOCxKKeTUcucx5Ot6DShFtOWD571S1NbE+OT6nr1KSkuz\nPmfUlTzfBq8WI+d65VmVc0fP+qJLKPouupxd/jJ8/wCpy9rbZz+KvjjtbnZLx5KTz8sTlTyayuSV\nufCG7a4K3YdIhuuEagWltXAbnYKdpC3VILFglb4TCL4L3JkYVushKsXCOfjsEl7skqdUPc3wGX1Y\nJNOrEvzyO7FnwX0rDknSYllPAm/hl8K+5qXICX0Jcu7YPNBHDZRrFbbYsLhhF4dihXOAowRp3yOf\nFVyFpO0sitybKz7GFGGVZptrCJnaqrE5uXDfAZal12FJNxIi2pZKnLjBfE4a4yZN5wi0rY9yprbk\nosCprNZE1xQO/FBke2swK+4W7eAbbjlVRG7JfH7WNeRJU7JU2n3ocpqltsO18S1O3JTjhciWVlB+\np8BtWJcLdBCO1lKVvKJ3ZapGtVirriwf0Jt7XQJvGDKkEdyzQO7wXdoiTarJLDpuyZ4GpX5ByS5V\nhh+OCCtWyti5WCN18cGiaSzwPZkHu2t2DrareBNyXItzuqR65x0E3T9pppu1d/giGXlFypcNBb9K\n/wBNqyZJeQTe220ClnhMMwZWkZJ8oGneGQvJabw06Xgxy/jMqHhcdxWnxlmup7uSYuldDL1pwor8\nFXUcolSbbodxr5Jr0o/ku/jsQ8p1yNWoqzeh+v8A+kepp9X6HqdNqaihqaepaS5pn2fVejaluWnr\nyX0fkX+mHWy6f1+Oi5VDUi1Xln7r0uvGWhTVs53u2O3G3OnyHWdD1kIuP67eO6OH1fp/UtOL12q+\nD9A62K1LdcHC6vTq7SOV443OV18Rq9BrxUl+q3jsju+h6+ppemR0dZPfG07HrxpvyeDqOoejK7M7\nnTp68Xq85acpNvl4PnNbXc5NZo7PrE/19JuPPKOHp6TVt80V77Z+Oub1mtK8rufSem6q09KFLLXY\n4XXdPKbW10dXom1oRcsNIfRmPf1XUy2pZVnL15Pnc8G+rq20ua+Tz6kJSXD8sqMrm9ZNNS4to8vp\nnR73+2mbdRFvXWnG22z7P/HfQ9TW0o6n6bS+UZ+N1X9c70/09qnt5O7pdF7HUPwfSaPoqglSWEet\n9BGGk3sd0d5/z6Hyj8+9Y6PZoyUFTq0fZ/4pJy9F0ZTf/cUdrXk8HrnTRelKFK2uTb0LqF08IQlK\noSSQSYeXcdPrMqTXg+U9UjJN7m+T6vr9TapOLXB8z6tNS07X/I8unLMfK9cnl1Z8/wCpLdB4dH0P\nWavua7M4HX98nOGuPJVLvgSVvgc097CPP0bccNV4FNJeQTw6E8rKKoWrHV9+RRG1/CHQUbfOSmrw\nkx3S+CVKrM3tKrsEU42Rut4NE3bTI6LVO0TuBvx3ElS5CA27ryPdWGTHLGnTXA4TrPwJxzd4G5XX\ngW6r/ouwUnSpFKK28g7Y08ZK3olGrTtg/wA2JvsF+eC7QT82EsLBS9yJfmg05pqVp+Q2u85CGbwV\nPLwFoxDbVUwirbbBqngLfnBf4cNW15JfOR3XAc5bKrKJO7yyaWOSk1nISiqtNMZYZBSfmwSUcZYO\n1wxR/c7ZYDk7SpCjH4HlvgfGDOZ4anzgHihxbvIS4HBJtTubtJAnSoFhhJjMbw1aB5fA4yY0sXYV\nmys5t4+OBrKyElcqbwUlGNobZIU4WQTb44Ku+w0l5RmVf4ptJinTTrkcqvKsqotYX2em8sYv6UUk\nhSfhDlSJcq4CbavTc1tramvsN6wtuBqpPMQqKurH+NBStPCBNpdqFGim1SSRdayN6byv5HJxXCJx\nNt1kVVloOtFJzXCjkINOfjAl+7ii0lbZuyY0rf7nSwOTjuyRJJqu43FNq74DIPK7H+Kas16/0H6b\nab1UrR/QXp+pcK70fz7/AIrJaXrnQzae1asc18n7voOelJNRdUcuXvTrx7dTWa2O1/RxurabfY6L\n1nOGbOX10d0Mcle3WON1slTafc4fWVKStHX62tKLuWa4OF12vnGEjjZ21O3g6tW2laXameJaV2r/\nALN+p1dzSSt+Q6eCit0lm+5W50bHmnoqMXJq2erpp6c+laik2scEa0ozlsgnzR6ek6VSb5RqXpaw\n6bQvUlOaVdlRetFRcv4Pa+kcVui6o26H0fX62UU40pZsJdWxp/hnoL631CXW60E9GPti6zuP1Dou\nj0tGCjGCTS8E+genaPp3Q6ehpL2xXfydLU2wt9+MHbhP1x53XO6iGzc14PJraiWnTjeLPb1FODV8\nnK1tSm0+yOtsYnHXM9UhHVTtZSPl5OcdLUiv3Rvb9ne9T6ja7vucPqpQbdd0cuTf8erQ9Sl1PR1q\nY1I4l8nK63UuMlbz2PHpdU+n62em09k3h+DbqtRTi3HLRy5cq145HVS84OT1ySTbWTq9U6jK1lfF\n0cT1DU3W2+QlY5OXJ+6uWDSbwJ1YSa7c0a38cKW5JtNDtPngFWeLBRd9mhtiwm0uAsUsPgfbJRHa\nayCYk08eBuknWQiw3JWsA3bJTVcZHiiVhyRN3hlQkrSp/Y5Rjl9w3KoiqWAVK75HHw7ocljCNbAX\nbCVIIJOWSUkpFKXwP+NfR8PkVt8A8jv4M2YNS+VaQ87LQNeUDeK7FqEJUht/wS07wONt8YHDRdP7\nHaFFptWml8DeLrgzejpSmtvYWn7mEUmykkuEauSDQ3mqJcslppolxS8mN/TpSniqCNZFhmkY1THp\nYzfdDjVFNK+BOMUuB2DsJ0xfdDpJcBHD4wH9RNpA8q0U2kroN25NLBW9akqVYaHh5SJaWQUq4L1q\nBugfBT2vtTHKh2C26hPwgbzm7LhFPyiZrJn2lW61wFKiafYEq7FgjWfuax2BRfFt9wTRDlK+Dv3e\nhYtxVGbTSqi3J0qFGTd7voeOqTFRaSy8iSdrImvFFKXtVrJeHNTKNNd/oNr7FRlHul5CMlJPFDtn\nYykt0YCtvFdhp0+MDk1Je0f6viUlSq+w43ttLHkTk1WUVuUol9KQXbBOSbq8qghUnaKVpBeqse30\necl6h0ua/wC5H/lH9DaUm4r5R/OvQzel1OlPvGaf9n9C6eo3pwlFLMUznzb49PY6ppLNHG9R1HpR\nk2s1ydF9Q0mmsnI9Sf6kXudWG5Hfi+X67V1dXUbjxwc/U0Zza326fk7XU6UNHCy+eDnas1pxt93R\nyt710z8cz9Jb2l24Hqy2rallHo0ou7ePtnnnKOprvuvJnl2JLq+k0XKptJZOx0ej7G6yznaOrFVH\n/wAHY6KUXp82A5Qfo3JQpvdJI+x9I6BRUWllI+b6dRlraTq/ckfb9FH9PTTqsHX/AJ8be2OVbO0v\nbg8+rqSjiTNNTVpS4PD1GpuSzk7/AC+mKz19alzxg4/WaslbTbPZryS3cnJ6jUUZSvi8B8mpMc/1\nV7tO3jBwZa2HBvPY6nX6rlcW8fR891cq1cUmjny5L0uti4yWortIcdaOpprPbJlPV3adOqaPBLV2\ntqLpfBzva2xfW6iVq8M+b6rVlq6uP2pUenr+rcm4/wAngcndmpxxx53Q17sA012uxKavKByt2sDl\nYym+cYsTul4Gndg28KnRYilkTbLT8onG2xhwQTbeSkmlZCRUZUVgwVlWEm48IPa021kWpLCpBL2Z\nBFtIdtWmKEnRTq7sb6vCzS4Hnm+Cd10XikgxUbrXgVXiyltXINxT5ovAz47mke/gzk8j3usDyByt\nvkKaVWS3eRoy1lTNy7M0hjl5CLxfhibV/Y7vSyhy2ukGWUku6YpPGDKmoXtl5KbyyU6fAbk2bxWC\nNvKGm2XaS+CE324M7qSyrbq2ClQ2vDLcPgtk27yWqXbIpO1gofQs8J4Eue4KT2j3Nch50sEnjuRZ\nblfCIvJqGQO+wLjKKclVEt34Ie1VeLFnvwO7XA9ySyFV1Lk4k7tzWHRVqQ3JWPi7Cb7DzQrV8Dlx\nh/2ZMh01FiqwV8XkJXX5PXuG+B2qQtvkqXGHZO62rKTRO2kFVWOTwwu1yTJmPtZnhJLm8iTyVHgl\nxe7DOk5TxnV4pXy8EyjVK+5LusvgpZyZvRk0OlLOSoU41RE+yLpRQ3uK9Go0gTpJNjirgvJlJts5\ny95Vr0Q/cneD92/xbr/+s9H6STkm3ppO+cH4Rp/uR+mf6adVu6HX0HK3pzTV9k0Z5tca/SX0r2bm\n+3k5fqEIaUW2+x7NHWlLTzJtHg6+9TEuA2X134x8/wBVF6kt74XHY4uvp6ms9sbpH0vUaDftV0eH\nX0tPRjfcLHWcs6cjq4rRUIt5fk88NCWnHc0rZ09DSj1etJzzGPcOu22oxSaqjF6a3HM0k97xx8nX\n9OuEHu7/ACeGEtsrrg6nTuL0rfbIdufLXV6CEVOEmsb1Z9s5KOilij4roJxnoQpf7rPqJ9R/2c+D\nrwueuXLsuo1b/a/g5/WarhBZ5DW1r02c3qtbdHFlaxE9R1La5z3OP12u1Je7l8D6jXe6VZOT1mtv\ne5Sdop/WvE9TrNxd2qOP1MlKbds36nWThJXbRy9bqKTirbfkxqnJKltnK21fyc7r+oUJOGnL3P8A\novq+p/STbz4OPNucnJ8sM71z5clSdrkVNhTv7Kd3wMYSo8WTte7DLp0uwU1x3GWjS4Y0218IVOSK\nimip1MkxbbVM0aVuxVw+EM0J20ueRKFXkp58lP8AaVXiEl2GqYfFD20mnVmVEvnAqzzyWovbXCJ2\ntPORiG35Kr5E0wVVXkto0+LCUVVkvLKa/kUW1gl/Io3myqaYU4EuzIqn8FqxV8FPSOxLKppA1ZdK\nXSWB23wFOnyOOA1VFYtg6Lau8kySteB+UqhU28cFU1Y1aWPInJ2vkztSO5SWMsdPlkvnkb212q8c\nCaXkSuqQJYzyU6ZzBWAStfI2+y4BRaeS39aS01gVZLlDvZNDKd6OUaENLIS4sO2ZRFvwDy6FFlNd\n0zVuNeEo08v+wXIqefIK1yGpQqZTft5JzQHqrad2hOVDc/ghy7HrkY+WtE7FGu+QUbh80SsWZ+WK\nNsVTJnT4aJ+WJ1eDMnepUe6QLErFH8FRp3fI25VaMPA0s4QnSTQJozbrM/RxzRTeME2vyEWngd2G\n203NJURasvbd2TtSyXGsxpGSSwfX/wCm3WS0/XX09LbrQd35WUfGRayfVf6dxT/yjp8Z2yr+A55n\nbctlfsujBPTq+/BnqaKkvmhx1HBNNUVDVg19HCV6ON6c3qdNpdjkdRoPU1GmrTO/1DU3VHj1/wBP\nThJtpyryb+W+tTXJ1dnSaMYacUm1k82lpPWlK7/Jrrt6+omotJYNnOOjBcBW9rm9SlCW3usPBXT6\n1Km/5wY9Z1EdsmjxaGvvny1kNFt+313pevBaMk2rs7E+sUuljJZwfH9Fr7ISt9zorqq0afAfKOV6\nrq/9Spabs5urrKnn+w0uqhLp6tJrmzmdX1EdN3H+uCnKUXtnqyjFybfLZxOr1ak/B6Os6yNvDeOD\ni9T1acpOvagnLR5Fams4t5s5fVa222nm7Fr9RSbOZr6kpvPHgYxbPpOvq/qzKUFS7GcYZRtTofti\n000orFsaSpYyJprFDWSvfjOhJX8BV1QbaVsSdclCW3a1XcbSLUlVNZQmrldGvQTUeOQUUkwu38hy\ny7Qkr4DZS8jrHYG+bC6qSjHNjlFXZNdx3gpPswY8DilWUT89xq7wPq7PUUU0LbFXww5TvLQRSk1e\nCkrIUFl0Lb7rLwlRNtFqg2J2vgHFN5BJvmwlHPyjP20TirXA3hqhpUsikmamLdJpNicaeC0sPAqp\npNB0p0GlRnFNSzwaJpYY3XIdQ1GCXp55G3axeB9xyRSlJUuwtivBbVqmRVM1IrQ44phHS+io8Wxv\nKM1fJnKKXAlG12LkkrJWEPx07o/Trv8A0DtvLsakrRVpGbLF3UPsqHSa4G+UwfwSzGbjTGoJ3Zo2\nmvknh8GpehKUdOueApFbntJWTPvdJKNy+BySRcXGvnvgU0jM7Ooik+RO7opJSVKwUWpZGzKt1KjZ\nOzOXRdYx2ElfJ7bcY49lG7VMrUTruLbnCY5txqmea7yrWITebKYllWFMZaCSa7lcfkcVZTRm8qtZ\n21+Bpse3OUKqofo4Tu2CZai2scClCr4H5XxFbtUwv5CvI1FXwPnbKbyfdf6TaX6v+RKT5hpyZ8Vt\nSzR+g/6NQv17qW//AODH8mOdvKGP0zq4qF2vg561Uk3fOeTtddBxTvijkakITwopfRzv/mdu3/Ov\nO9felVHg17lJr5PfPTWlFvsc/qJ7FfYuNdflKx1H+nHH/JzOr13tfur6DruppLuc3qNd7bthfWpc\nYdRrJ2k7DpnFK5M8uorla45K0+zcqKz8HLk6T1dq5dG8ur/7VcHNlNNKn/ZE9XhPgesYr36fWODS\nt01TPH1fWb4tN88BJKaW2zy6sUo5Diz8sjyz1pJO239ng6nWhGDb5fY9HVaq04vczidRN6k74pmu\nOOdup1NRzfOCavIKkvkqKpWarnWsVgrw/Alx9kwak+Rgi223bD/baCUkmJzVBhErbwwiCknbVC3R\ni+Xyaga4aebIdu6Y3KKWHZLaTrOQnpDjXcatfQm18gmsux+QUvAbcoUZX/yViWWw0E0tpHblly45\nFFxXLGNSGo+eCXCuH3G5PyqDcmE1acaXL5Bva2sV9BGSp5FJ7m6NaA3fcNr7tAqw27Iu5dyajaUk\nuCH7qYmo2NVSozMZolK0xKbWMZG0k7X5JeXwhkijSLzkUnnyK1VKifJYaUoqfCoeY45Kj7VdCWX2\nJQk3jBSdP8g0078i3JP3d+DP+NKpZbEnXhhLKWSeMp4GS1k2/hEpOrxyNtOs/wD2J8clOjEtX3Go\nrbVg6QReeR7q01HPwSq3Zy+TWsWmTUb+Q3ejLgclWCJydfJWp2UcEr/8imenTjLyOTfNBtvCpFuK\nrlGbWemd0uCVPKVFSdxqxJcUObD6aC1n/wBFJruRJq+MFx7Vhtq8DVuN3kzVmimkh5cVOhHTxafO\nSZQdqqIg6G5bex35bo/xXCV8j+zPduse7LM/Ey/q4o0/bn4MrSzkamrdtleO+H09sebyNRTkryhK\ncaE5pcGc/BWr2tPCTJpXnlGamUmruwvHGdxq4qKwLbYnP2jjPGWF6Nuk4JtcBJbXgUZJXTGpwbe5\n0MoUoqeD9H/0Y0V/+udROqX6P/k/O47ex+t/6Oenyj02v1cotfqSUYu+Uuf7M8r01j9O6vpIa2l7\nkcHq/S3C3pyVXg+rTitOmkcrr25JqKC9w8Px8f1+hracJbpRq+xwutjqaqUYxdI+q6mMnJ90eF6U\nXmUaRj4tzY+R1enkotyi/wAnM1NKc1e2j7LrdiUoRiv4Odq6O51VJLshkanKvnP+lmotfgJdK4Ur\nVI62ovijxa0fcqseMF5ZWOjBReVdmn6ab/a85Li0lnFeSm6itofYvK2sZJRjwkc3rJVKk8Ht1JnH\n9Q1afLf0F96HKuT1mpOUtrdpGCheStT3SbeBwaOvxma42sYwpuwlF1hG9ZI1ZbYGbYp2w/Uav+CI\ntq6C7QkWlW55yG4hU+w2kBEW03T5L5fJC4yGBtAbdlbqpvkSqvkTSbyUu9H1pKZKlyTPtQJ44HAt\nzruH6jqicdxXmgTRSbWSG8gmEqLRardWAcnZF5GsvJeIOTvktzdYZmwodKtzTRTlLJGBWBO3TyNS\na4ZLwMRjRajr5Yv1PknbwyWjIq3q00OUnfJEorA5CV72lTYpTrgz+RrOCqjWMnJEytvLIWMIG3u7\nmZCtSq0xqVLDM+eQtVmzQq3qNsam0Z4YWWKNXqq8Ije07ROLEs2Ua9aLUfZhvfJlWSkKafqWJTyR\nih8ozb9KL/Wp8BLU3JdjPaJJlZBY0XxQnKVgnVgn3C8lBdIamTLIhnZjTcqyiVK8UhWn5sFHNjv0\nfVzjtITV5NJrcvlCUUmj3W9MwRp8g0kXsx8mTi0cfRirwS0/A9rlZaWEmO4Yl0kSXNIlRMXnIcSq\nQLzbKr+BOK7GLytGC8Bklc8mqraZ5XFIlOuR+1rl/Qk7Q1HJeFpC5zSjl9kf0r/g/p69O9H6Hp1/\nt01a+Xl/2z8I/wAE9LXqf+UdFozi3pRl+pOvCz/6P6X9N0UoRM3lvQvj2bE4Ln5Of1UMvGDsSiow\nOdryS58mvoca4k+lXhcnO6rQS3NrB9LqKBw/VJKmk6SHpucnzXU6cXN8Hg6hpJuJ7+q1Em8WcnVk\nm2jGNa8GrqXJpJ/wZzSpXhouUktZpLFeTLVld8DbIc6eeaW5eCZyUYX8UazftZz+o1ZRbofnBZXn\n6nWqLVnA63Xa1Gkz29XrUpX4OLqS3yt8h1Wb4T1HLMhKbXBMlaKUeC1zxf6r8mc5OXLJkrElh0Wf\nbUCRS5Qo3WQbzgzqDaSCNMloStcMdWtK8CqiNzTKlK1ktUhpq8FYZlFdy457EsNitIaiyXGxiN1e\nGFYDY0hpYyFuiFasIq28jaEo7eC2JTSSsm6Q+xNOinaxV2w7CUWkVtxkLZES4JxuodWGzIyk8MTG\n40yXzyUq9XeEiWhtDSt54KUYUkHkfDYnQrAisJrJm3wh1dEotruhLLENJduS8awcMNt1THsdWCbT\nMwHTjYm0NNvkbhaHc9WM20ymknyEYdypLwHyhxm1kaqhvKQpKjWok1ZT7Ijb3NNuLC9drtOLG2Ku\nGPDDdWBMd3QopDpmVgaQnTFTGoj414zbaLjFtNhKDwEW0dNlgUraeRp5BpZSFxHmz350xrTKSpku\n6ugSwic97OeSle6rKhbM4p5dDc5djl/04/hmLcMq3gVf8glKSyS7icsQqmTLDwXLjgzk7lgLq1KW\nclXnl0TJWELoN0/S4pZGo5fyQk0z3endLPres0Om0o7tTVmoL8kn6l/o96Z+l0+r18/3az2Q/wD6\np/8As/Y+hwlZ8z/j3p+l0PR9P02iqhpQSWD6jQSi40Y9rFrfXdRu+Tk9Ryvhnv6qdJYON1Wtu48m\nrTxxn1OqoZs4PqWtvTVnr6rUk7qzh+oas/8Ab35C1qTt4Os1HF44OVqSbme2cpNPd5PHqR9zeMBu\net6xmsp8M8nUNpu2erWltgvJ49fUV2yi1hqSajy8HO6vWu2nw6PR1evUHWLRxeq1qvOGQtrx9bqt\nzaPE3Vl603OVmf2PjFA1xkXLoOVjkvUJU3gzk6ZTdUQ7eTU6Gq3W6HsdMUIO1bPTHTpLuUmjXnjH\nyOnbR6qxTjkyap3Rr4tPO4tSyD4ybyj3oh6dpGbmi6yg200NOpUEYtWvkUk1KzNHbVMN2Sd1rgmT\nyUa8aN97FJ3wTlo0UcGpIqhN2XYmnG7RO4xQG23yWspGG7Ja1KFLlfYG2lbbszlPIbsdxsh9ablY\nRMt2bKep4LEueXdiTtE700JTopRjTkeDPeLfnJJpfIlkjdn4HGWS1RoqjWBXbIU/KHuV4KFa4yWo\nrnJmpYNNN2nSKdpqmqGooye5cKh70l8ms/ANSKsamlaBZEoZyZ/lKlW3H2RTlmzSdUkkJJpZLJF2\nhx4oEk+SmuCpJUki3Ewk1dIdvaLUhTE7SD1qU3bJrPIJtoqKt0XirP8A3cmix35E0k3gfKtGrdBy\nk19CUhcrIJNq0ZhvfarC0SreB1VF4Fxi07E00y01X4Ilng+jusYcZPhCUmubsndtsay8p2Xwzsqi\n3V2NVyC8LgNtv4Zw58pqw/1KD9zw+BuMbzeBJJWY2VWz6OVJZMZVeDTVnFppIw7nKwyKi854HVEp\nUy18ZTDZDpx9zR91/pf6aur9d/6icN2n0yTvxLsfDpVlH7p/pt6Wuj/x7SnKKWrrvfJ/0v6DdOfb\n7voNPiSXc6kqilSPL0UK04peD1TftdcjxkjG65/X6+2L8nJnJXcj3ddC3ufBydWe6s4HlI3JMebq\n5xSdM4PqE3eM4Pd1kvfJWzk9S1lrkxZinTz6juLs8M7zf0Xr6rW5HknrNc9xs3so6iSUWn4OdqyU\nsXwbdVrr/fjJ4NXUagpJ4CHHk62fukl4OV1Eva7PZ1U5ZfnByuq1LaSGRnlGWpzggJSwQ5cYHyMr\nXOB8EJ1QW3dErBzyTL4Lp1Q4abdFIzYrp1bVnoy/wRGO1I004TcsJ0a0mrfxZnOLb4OgumTrz9Ch\n0722Z+TUjxrSbwy46F+fJ7paVJt9iNNK7ox72ceB6FcmWrp4dHQ1YU/hZMpQVWaWa5cotdmJJ3k9\ns4XKux5tWO2dFGbDgm3R6NP9ttKzzRdM9ejmsWa491RnNOaPLqqng6ihfY8+roU+Pk1eOBz1gpJN\nWPUVTa8Erg5VYGhpCZQ4UspZ8ElqLrCComsE1ZTFdSJJodDayDVJUWKDsLsAPBeLA+EVCu5PKChS\n1Lk20XVmCRrpumrHMWPQ7oUqzSEp2JakbHwKTx+SrMt8RLUpOgv6Ptcm6Xd/A021+DL9RPllRnHN\nsdmdtNMoIvdVsUnG+eURaXctlStZUkTDvfFBLKuyFLDCQxaruVGuFyZ0XCObG4vTlFvgmKNG8irx\neTO9HGbaTpJjiy4winlWKeKruUu9HCrPGQa8opKSeU0JywWds6//2Q==\n", 437 | "text/plain": [ 438 | "" 439 | ] 440 | }, 441 | "metadata": { 442 | "tags": [] 443 | } 444 | } 445 | ] 446 | }, 447 | { 448 | "cell_type": "code", 449 | "metadata": { 450 | "id": "W0dMFtgcPAr5", 451 | "colab_type": "code", 452 | "colab": {} 453 | }, 454 | "source": [ 455 | "!mv photo.jpg test1_1R.jpg" 456 | ], 457 | "execution_count": 0, 458 | "outputs": [] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "metadata": { 463 | "id": "tf_43ZaKjn3x", 464 | "colab_type": "code", 465 | "colab": {} 466 | }, 467 | "source": [ 468 | "!mkdir -p data/fingers/testset/orig\n", 469 | "!mv *.jpg data/fingers/testset/orig" 470 | ], 471 | "execution_count": 0, 472 | "outputs": [] 473 | }, 474 | { 475 | "cell_type": "code", 476 | "metadata": { 477 | "id": "owPSHr1Z9bAJ", 478 | "colab_type": "code", 479 | "colab": {} 480 | }, 481 | "source": [ 482 | "dataset = 'testset'\n", 483 | "path = Path(data_path)/'fingers'" 484 | ], 485 | "execution_count": 0, 486 | "outputs": [] 487 | }, 488 | { 489 | "cell_type": "markdown", 490 | "metadata": { 491 | "id": "BQPZ-su3JJER", 492 | "colab_type": "text" 493 | }, 494 | "source": [ 495 | "## Backup realworld testset to GDrive" 496 | ] 497 | }, 498 | { 499 | "cell_type": "code", 500 | "metadata": { 501 | "colab_type": "code", 502 | "outputId": "841c45ba-4966-4cf1-8722-2a4feaec332a", 503 | "id": "ufEe2OXh-Aqp", 504 | "colab": { 505 | "base_uri": "https://localhost:8080/", 506 | "height": 34 507 | } 508 | }, 509 | "source": [ 510 | "!tar -czf {(Path(data_path)/(dataset + '.tgz')).as_posix()} -C {Path(data_path).as_posix()} {path.as_posix()} " 511 | ], 512 | "execution_count": 0, 513 | "outputs": [ 514 | { 515 | "output_type": "stream", 516 | "text": [ 517 | "tar: Removing leading `/' from member names\n" 518 | ], 519 | "name": "stdout" 520 | } 521 | ] 522 | }, 523 | { 524 | "cell_type": "code", 525 | "metadata": { 526 | "id": "AN0a2BoG-2zP", 527 | "colab_type": "code", 528 | "colab": {} 529 | }, 530 | "source": [ 531 | "!cp data/testset.tgz {escdrive(gdrive/'data'/'fingers_testset.tgz')}" 532 | ], 533 | "execution_count": 0, 534 | "outputs": [] 535 | } 536 | ] 537 | } --------------------------------------------------------------------------------