├── .gitignore ├── 100_Days_of_ML_Code_Instructions.MD ├── LICENSE ├── OpenCV_in_Colab.ipynb ├── README.md ├── cheatsheets ├── Beginners-Python.jpeg ├── Matplotlib.pdf ├── Pandas-DataFrame-Notes.pdf ├── Python-for-DataScience-Seaborn.pdf ├── pytorch-cheatsheet.jpg └── rl-cheatsheet.pdf ├── code ├── Celsius_to_Fahrenheit_PyTorch.ipynb ├── Export_Model_to_ONNX.ipynb ├── Intro_to_pandas.ipynb ├── Linear_Algebra │ ├── 1_vector_coding.ipynb │ ├── 2_matrices_in_python.ipynb │ ├── 3_matrix_addition.ipynb │ ├── 4_matrix_multiplication.ipynb │ └── guide_to_mathematical_notation.ipynb ├── Robot_Localization.ipynb ├── cv │ ├── Contour detection and features.ipynb │ ├── Finding Edges and Custom Kernels.ipynb │ ├── Hough_lines.ipynb │ └── models.py ├── data_exploration │ ├── Nationalities_data_exploration.ipynb │ └── datasets │ │ ├── 2018_POBLACIO_NACIONALITAT.csv │ │ └── 2018_lloguer_preu_trim.csv ├── knn-lodash.js ├── load_volumetric_data.py └── ml-sklearn.py ├── content ├── Attention-mechanisms.md ├── Batch-size.md ├── CNN.md ├── Computer-Vision.md ├── Conda.md ├── Data-augmentation.md ├── Day-10.md ├── Debug-jupyter.md ├── Defining-network-structure.md ├── Downloading-Datasets.md ├── Dropout.md ├── Embeddings.md ├── FCNNs.md ├── Feather.md ├── GPU.md ├── Generate-requirements.md ├── High-bias-and-high-variance.md ├── Hyperparameters.md ├── Image-Captioning.md ├── Jobs-in-Computer-Vision.md ├── Jupyter-extensions.md ├── Jupyter-server.md ├── LSTM.md ├── Layer-shapes.md ├── Learning-rate.md ├── Localization.md ├── Machine-learning-concepts.md ├── Momentum.md ├── Motion.md ├── NLTK.md ├── OpenCV.md ├── Probability.md ├── PyTorch.md ├── RNN.md ├── Region-Proposal-Algorithms.md ├── Regularization.md ├── Reinforcement-Learning.md ├── Training-Dev-and-Testing-Sets.md ├── Visualizing-Feature-Maps.md ├── Weighted-Loss-Functions.md ├── YOLO.md ├── bad-constraint-matrix.md ├── fast-ai.md ├── index.md └── sklearn.md └── resources ├── Calculate-Layer-Shapes-Helper.xlsx ├── aisat.png ├── aisat2.png ├── aisat3.png ├── kernels.png └── mse.png /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | -------------------------------------------------------------------------------- /100_Days_of_ML_Code_Instructions.MD: -------------------------------------------------------------------------------- 1 | # 100DaysOfMLCode 2 | 3 | ## Overview 4 | 5 | These are the instructions for [this](https://www.youtube.com/watch?v=cuQMBj1cWPo&t=7s) video on Youtube by Siraj Raval for the #100DaysofMLCode Challenge. 6 | 7 | ## Motivation 8 | 9 | Machine Learning is the most transformative technology of our time. Whether its helping us discover new drugs for major diseases, 10 | fighting fraud, generating music, improving supply chain efficiency, the list of applications are truly endless. In order for us as a community 11 | to be able to make valuable contributions to the world, we need to master this technology. This is a call to action, a battle cry, a spark that 12 | will light a movement to radically improve the state of humanity. 100 Days of ML Code is a committment to better your understanding of 13 | this powerful tool by dedicating at least 1 hour of your time everyday to studying and/or coding machine learning for 100 days. 14 | 15 | ## Eligibility 16 | 17 | - Everyone is eligible, even people who've never coded before 18 | 19 | ## The 3 Rules 20 | 21 | - Make a public pledge to code or study machine learning for minimum 1 hour every day for the next 100 days via your favorite social platform using the [#100DaysofMLCode](https://twitter.com/sirajraval/status/1014758160572141568) Hashtag. 22 | - Make a public log of your work. Update it daily. Here is a GitHub example [template](https://gist.github.com/llSourcell/43194e486a92532bc343f7837b178eda). Another one is [here](https://github.com/LordSomen/100DaysOfML/blob/master/Log.md). You can also 23 | make a blog or vlog. 24 | - If you see someone make a post using the #100DaysofMLCode hashtag, encourage them via a 'like', 'share', or comment! 25 | 26 | ## Project Ideas 27 | 28 | - Siraj's pick is [Preventing the Spread of Dengue Fever](https://www.drivendata.org/competitions/44/dengai-predicting-disease-spread/). 29 | The task is to predict the number of dengue cases each week (in each location) based on environmental variables describing changes 30 | in temperature, precipitation, vegetation, and more. An understanding of the relationship between climate and dengue dynamics can improve research 31 | initiatives and resource allocation to help fight life-threatening pandemics. You can literally save lives with machine learning. 32 | 33 | - There are many other projects out there though. Pick an industry that excites you, find a problem they have, locate a relevant dataset, apply AI to that dataset, and monetize the solution. 34 | 35 | ## Learning Resources 36 | 37 | - [Siraj Raval on Youtube](https://youtube.com/c/sirajraval) 38 | - [Tensorflow Documentation](https://www.tensorflow.org/tutorials/) 39 | - [A bunch of online courses](https://github.com/josephmisiti/awesome-machine-learning/blob/master/courses.md) 40 | - [A bunch of public dataset](https://github.com/awesomedata/awesome-public-datasets/blob/master/README.rst) 41 | 42 | ## Awards 43 | 44 | - Siraj Raval will give a shoutout to some of the Wizards who succesfully complete the challenge, notify him if you've completed 45 | it via Twitter [@sirajraval](https://twitter.com/sirajraval) 46 | - By the end of 100 days, you'll be expected to have a project you've contributed to, whether your own or another. The more impact 47 | you've had, the more likely you'll get a shoutout. Impact comes in the form of good documentation of your journey so that others can follow, 48 | a project that improves the lives of other people, or real progress in your own ability to code machine learning. 49 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 George Studenko 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /OpenCV_in_Colab.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "OpenCV in Colab.ipynb", 7 | "provenance": [], 8 | "authorship_tag": "ABX9TyO/UZgZxECitKVJ8BhQydgi", 9 | "include_colab_link": true 10 | }, 11 | "kernelspec": { 12 | "name": "python3", 13 | "display_name": "Python 3" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "markdown", 19 | "metadata": { 20 | "id": "view-in-github", 21 | "colab_type": "text" 22 | }, 23 | "source": [ 24 | "\"Open" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "metadata": { 30 | "id": "NXa6dcOm0hEG" 31 | }, 32 | "source": [ 33 | "import numpy as np\n", 34 | "import pandas as pd\n", 35 | "import cv2 as cv2\n", 36 | "from google.colab.patches import cv2_imshow # for image display\n", 37 | "from skimage import io\n", 38 | "from PIL import Image \n", 39 | "import matplotlib.pylab as plt" 40 | ], 41 | "execution_count": null, 42 | "outputs": [] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "metadata": { 47 | "colab": { 48 | "base_uri": "https://localhost:8080/", 49 | "height": 35 50 | }, 51 | "id": "zZiTpBeI0oMc", 52 | "outputId": "2843826b-fec8-426a-b371-02a81e360dff" 53 | }, 54 | "source": [ 55 | "cv2.__version__" 56 | ], 57 | "execution_count": null, 58 | "outputs": [ 59 | { 60 | "output_type": "execute_result", 61 | "data": { 62 | "application/vnd.google.colaboratory.intrinsic+json": { 63 | "type": "string" 64 | }, 65 | "text/plain": [ 66 | "'4.1.2'" 67 | ] 68 | }, 69 | "metadata": { 70 | "tags": [] 71 | }, 72 | "execution_count": 4 73 | } 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "metadata": { 79 | "id": "qLJCLk4v0rxv" 80 | }, 81 | "source": [ 82 | "cv2_imshow()" 83 | ], 84 | "execution_count": null, 85 | "outputs": [] 86 | } 87 | ] 88 | } -------------------------------------------------------------------------------- /cheatsheets/Beginners-Python.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/cheatsheets/Beginners-Python.jpeg -------------------------------------------------------------------------------- /cheatsheets/Matplotlib.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/cheatsheets/Matplotlib.pdf -------------------------------------------------------------------------------- /cheatsheets/Pandas-DataFrame-Notes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/cheatsheets/Pandas-DataFrame-Notes.pdf -------------------------------------------------------------------------------- /cheatsheets/Python-for-DataScience-Seaborn.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/cheatsheets/Python-for-DataScience-Seaborn.pdf -------------------------------------------------------------------------------- /cheatsheets/pytorch-cheatsheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/cheatsheets/pytorch-cheatsheet.jpg -------------------------------------------------------------------------------- /cheatsheets/rl-cheatsheet.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/cheatsheets/rl-cheatsheet.pdf -------------------------------------------------------------------------------- /code/Celsius_to_Fahrenheit_PyTorch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "Celsius to Fahrenheit PyTorch.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [], 9 | "authorship_tag": "ABX9TyNeePCLlDC1JN6kz7pDg7mW", 10 | "include_colab_link": true 11 | }, 12 | "kernelspec": { 13 | "name": "python3", 14 | "display_name": "Python 3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "view-in-github", 22 | "colab_type": "text" 23 | }, 24 | "source": [ 25 | "\"Open" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "metadata": { 31 | "id": "60XriFRSwCpQ", 32 | "colab_type": "code", 33 | "colab": {} 34 | }, 35 | "source": [ 36 | "" 37 | ], 38 | "execution_count": 0, 39 | "outputs": [] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "metadata": { 44 | "id": "tFu0XWgowMUK", 45 | "colab_type": "code", 46 | "colab": {} 47 | }, 48 | "source": [ 49 | "import numpy as np\n", 50 | "import torch\n", 51 | "import torch.nn as nn\n", 52 | "from torch.autograd import Variable\n", 53 | "from torch import optim" 54 | ], 55 | "execution_count": 0, 56 | "outputs": [] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": { 61 | "id": "zY2LNl-r3yRj", 62 | "colab_type": "text" 63 | }, 64 | "source": [ 65 | "𝑓=𝑐×1.8+32" 66 | ] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "metadata": { 71 | "id": "JY7wxI3uwepJ", 72 | "colab_type": "code", 73 | "colab": {} 74 | }, 75 | "source": [ 76 | "celsius = np.array([[-40], [-10], [0], [8], [15], [22], [38]], dtype=float)\n", 77 | "fahrenheit = np.array([[-40], [14], [32], [46.4], [59], [71.6], [100.4]], dtype=float)\n", 78 | "\n", 79 | "test_set = np.array([[71], [94], [117]], dtype=float)\n", 80 | "test_set_result_should_be = np.array([[159.8], [201.2], [242.6]], dtype=float)\n" 81 | ], 82 | "execution_count": 0, 83 | "outputs": [] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "metadata": { 88 | "id": "95CJm06p547U", 89 | "colab_type": "code", 90 | "colab": {} 91 | }, 92 | "source": [ 93 | "celsius = Variable(torch.from_numpy(celsius))\n", 94 | "fahrenheit = Variable(torch.from_numpy(fahrenheit))\n", 95 | "\n", 96 | "test_set = Variable(torch.from_numpy(test_set))\n", 97 | "test_set_result_should_be = Variable(torch.from_numpy(test_set_result_should_be))" 98 | ], 99 | "execution_count": 0, 100 | "outputs": [] 101 | }, 102 | { 103 | "cell_type": "code", 104 | "metadata": { 105 | "id": "FznK-6K9_rqa", 106 | "colab_type": "code", 107 | "colab": { 108 | "base_uri": "https://localhost:8080/", 109 | "height": 34 110 | }, 111 | "outputId": "f996e485-3069-4e4f-de9e-83224e9d86cc" 112 | }, 113 | "source": [ 114 | "celsius[0]" 115 | ], 116 | "execution_count": 7, 117 | "outputs": [ 118 | { 119 | "output_type": "execute_result", 120 | "data": { 121 | "text/plain": [ 122 | "tensor([-40.], dtype=torch.float64)" 123 | ] 124 | }, 125 | "metadata": { 126 | "tags": [] 127 | }, 128 | "execution_count": 7 129 | } 130 | ] 131 | }, 132 | { 133 | "cell_type": "code", 134 | "metadata": { 135 | "id": "Bfogi_QY36Hv", 136 | "colab_type": "code", 137 | "colab": {} 138 | }, 139 | "source": [ 140 | "linear_model = nn.Linear(1,1)" 141 | ], 142 | "execution_count": 0, 143 | "outputs": [] 144 | }, 145 | { 146 | "cell_type": "code", 147 | "metadata": { 148 | "id": "BoLK3nxK4Nv9", 149 | "colab_type": "code", 150 | "colab": { 151 | "base_uri": "https://localhost:8080/", 152 | "height": 34 153 | }, 154 | "outputId": "81014ac1-cb32-4743-ddf8-72bf8add2b00" 155 | }, 156 | "source": [ 157 | "linear_model" 158 | ], 159 | "execution_count": 9, 160 | "outputs": [ 161 | { 162 | "output_type": "execute_result", 163 | "data": { 164 | "text/plain": [ 165 | "Linear(in_features=1, out_features=1, bias=True)" 166 | ] 167 | }, 168 | "metadata": { 169 | "tags": [] 170 | }, 171 | "execution_count": 9 172 | } 173 | ] 174 | }, 175 | { 176 | "cell_type": "code", 177 | "metadata": { 178 | "id": "UI3Dv1YJ4dLa", 179 | "colab_type": "code", 180 | "colab": { 181 | "base_uri": "https://localhost:8080/", 182 | "height": 34 183 | }, 184 | "outputId": "b0dddfa6-de1a-4ff4-c05b-6303141561d8" 185 | }, 186 | "source": [ 187 | "print(f'Model weight before training is: {linear_model.weight.item()}')" 188 | ], 189 | "execution_count": 12, 190 | "outputs": [ 191 | { 192 | "output_type": "stream", 193 | "text": [ 194 | "Model weight is: -0.8136303424835205\n" 195 | ], 196 | "name": "stdout" 197 | } 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "metadata": { 203 | "id": "H7G3a-CU4fts", 204 | "colab_type": "code", 205 | "colab": { 206 | "base_uri": "https://localhost:8080/", 207 | "height": 34 208 | }, 209 | "outputId": "fba6fa4c-e944-4cca-bcf2-fe8d54f6b0c7" 210 | }, 211 | "source": [ 212 | "print(f'Model bias before training is: {linear_model.bias.item()}')" 213 | ], 214 | "execution_count": 13, 215 | "outputs": [ 216 | { 217 | "output_type": "stream", 218 | "text": [ 219 | "Model bias is: -0.17612791061401367\n" 220 | ], 221 | "name": "stdout" 222 | } 223 | ] 224 | }, 225 | { 226 | "cell_type": "code", 227 | "metadata": { 228 | "id": "AfQ6wacg4iLZ", 229 | "colab_type": "code", 230 | "colab": {} 231 | }, 232 | "source": [ 233 | "linear_model.double()\n", 234 | "y = linear_model(celsius[0])" 235 | ], 236 | "execution_count": 0, 237 | "outputs": [] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "metadata": { 242 | "id": "dJa1xeY24yC4", 243 | "colab_type": "code", 244 | "colab": { 245 | "base_uri": "https://localhost:8080/", 246 | "height": 34 247 | }, 248 | "outputId": "3f5ec0b0-dd85-4e09-bb93-29b185f9c437" 249 | }, 250 | "source": [ 251 | "print(f'The prediction for -40 C without training the model is: {round(y.item(),2)} F and should really be {fahrenheit[0].item()} F')" 252 | ], 253 | "execution_count": 20, 254 | "outputs": [ 255 | { 256 | "output_type": "stream", 257 | "text": [ 258 | "The prediction for -40 C without training the model is: 32.37 F and should really be -40.0 F\n" 259 | ], 260 | "name": "stdout" 261 | } 262 | ] 263 | }, 264 | { 265 | "cell_type": "code", 266 | "metadata": { 267 | "id": "IrF4GI_s91hF", 268 | "colab_type": "code", 269 | "colab": {} 270 | }, 271 | "source": [ 272 | "criterion = nn.MSELoss()\n", 273 | "sgd_optimizer = optim.SGD(linear_model.parameters(),lr = 0.001)" 274 | ], 275 | "execution_count": 0, 276 | "outputs": [] 277 | }, 278 | { 279 | "cell_type": "code", 280 | "metadata": { 281 | "id": "W4lK_r8M-EPp", 282 | "colab_type": "code", 283 | "colab": { 284 | "base_uri": "https://localhost:8080/", 285 | "height": 1000 286 | }, 287 | "outputId": "55d14af7-d907-459b-e607-cfb981870167" 288 | }, 289 | "source": [ 290 | "epochs = 600\n", 291 | "print_every = 50\n", 292 | "steps = 0\n", 293 | "\n", 294 | "for epoch in range(epochs):\n", 295 | " running_loss = 0\n", 296 | " \n", 297 | " linear_model.train()\n", 298 | " \n", 299 | " # train\n", 300 | " for index in range(len(celsius)):\n", 301 | " steps += 1\n", 302 | " \n", 303 | " # Set gradients to zero\n", 304 | " sgd_optimizer.zero_grad()\n", 305 | " \n", 306 | " y_hat = linear_model.forward(celsius[index])\n", 307 | " loss = criterion(y_hat,fahrenheit[index])\n", 308 | " loss.backward()\n", 309 | " sgd_optimizer.step()\n", 310 | " \n", 311 | " running_loss += loss.item()\n", 312 | " \n", 313 | " if steps % print_every == 0:\n", 314 | " linear_model.eval()\n", 315 | " \n", 316 | " print(\"Epoch: {}/{}... \".format(epoch+1, epochs),\n", 317 | " \"Loss: {:.3f}\".format(running_loss/print_every))\n", 318 | " \n", 319 | " running_loss = 0\n", 320 | " linear_model.train()" 321 | ], 322 | "execution_count": 29, 323 | "outputs": [ 324 | { 325 | "output_type": "stream", 326 | "text": [ 327 | "Epoch: 8/600... Loss: 0.000\n", 328 | "Epoch: 15/600... Loss: 0.000\n", 329 | "Epoch: 22/600... Loss: 0.000\n", 330 | "Epoch: 29/600... Loss: 0.000\n", 331 | "Epoch: 36/600... Loss: 0.000\n", 332 | "Epoch: 43/600... Loss: 0.000\n", 333 | "Epoch: 50/600... Loss: 0.000\n", 334 | "Epoch: 58/600... Loss: 0.000\n", 335 | "Epoch: 65/600... Loss: 0.000\n", 336 | "Epoch: 72/600... Loss: 0.000\n", 337 | "Epoch: 79/600... Loss: 0.000\n", 338 | "Epoch: 86/600... Loss: 0.000\n", 339 | "Epoch: 93/600... Loss: 0.000\n", 340 | "Epoch: 100/600... Loss: 0.000\n", 341 | "Epoch: 108/600... Loss: 0.000\n", 342 | "Epoch: 115/600... Loss: 0.000\n", 343 | "Epoch: 122/600... Loss: 0.000\n", 344 | "Epoch: 129/600... Loss: 0.000\n", 345 | "Epoch: 136/600... Loss: 0.000\n", 346 | "Epoch: 143/600... Loss: 0.000\n", 347 | "Epoch: 150/600... Loss: 0.000\n", 348 | "Epoch: 158/600... Loss: 0.000\n", 349 | "Epoch: 165/600... Loss: 0.000\n", 350 | "Epoch: 172/600... Loss: 0.000\n", 351 | "Epoch: 179/600... Loss: 0.000\n", 352 | "Epoch: 186/600... Loss: 0.000\n", 353 | "Epoch: 193/600... Loss: 0.000\n", 354 | "Epoch: 200/600... Loss: 0.000\n", 355 | "Epoch: 208/600... Loss: 0.000\n", 356 | "Epoch: 215/600... Loss: 0.000\n", 357 | "Epoch: 222/600... Loss: 0.000\n", 358 | "Epoch: 229/600... Loss: 0.000\n", 359 | "Epoch: 236/600... Loss: 0.000\n", 360 | "Epoch: 243/600... Loss: 0.000\n", 361 | "Epoch: 250/600... Loss: 0.000\n", 362 | "Epoch: 258/600... Loss: 0.000\n", 363 | "Epoch: 265/600... Loss: 0.000\n", 364 | "Epoch: 272/600... Loss: 0.000\n", 365 | "Epoch: 279/600... Loss: 0.000\n", 366 | "Epoch: 286/600... Loss: 0.000\n", 367 | "Epoch: 293/600... Loss: 0.000\n", 368 | "Epoch: 300/600... Loss: 0.000\n", 369 | "Epoch: 308/600... Loss: 0.000\n", 370 | "Epoch: 315/600... Loss: 0.000\n", 371 | "Epoch: 322/600... Loss: 0.000\n", 372 | "Epoch: 329/600... Loss: 0.000\n", 373 | "Epoch: 336/600... Loss: 0.000\n", 374 | "Epoch: 343/600... Loss: 0.000\n", 375 | "Epoch: 350/600... Loss: 0.000\n", 376 | "Epoch: 358/600... Loss: 0.000\n", 377 | "Epoch: 365/600... Loss: 0.000\n", 378 | "Epoch: 372/600... Loss: 0.000\n", 379 | "Epoch: 379/600... Loss: 0.000\n", 380 | "Epoch: 386/600... Loss: 0.000\n", 381 | "Epoch: 393/600... Loss: 0.000\n", 382 | "Epoch: 400/600... Loss: 0.000\n", 383 | "Epoch: 408/600... Loss: 0.000\n", 384 | "Epoch: 415/600... Loss: 0.000\n", 385 | "Epoch: 422/600... Loss: 0.000\n", 386 | "Epoch: 429/600... Loss: 0.000\n", 387 | "Epoch: 436/600... Loss: 0.000\n", 388 | "Epoch: 443/600... Loss: 0.000\n", 389 | "Epoch: 450/600... Loss: 0.000\n", 390 | "Epoch: 458/600... Loss: 0.000\n", 391 | "Epoch: 465/600... Loss: 0.000\n", 392 | "Epoch: 472/600... Loss: 0.000\n", 393 | "Epoch: 479/600... Loss: 0.000\n", 394 | "Epoch: 486/600... Loss: 0.000\n", 395 | "Epoch: 493/600... Loss: 0.000\n", 396 | "Epoch: 500/600... Loss: 0.000\n", 397 | "Epoch: 508/600... Loss: 0.000\n", 398 | "Epoch: 515/600... Loss: 0.000\n", 399 | "Epoch: 522/600... Loss: 0.000\n", 400 | "Epoch: 529/600... Loss: 0.000\n", 401 | "Epoch: 536/600... Loss: 0.000\n", 402 | "Epoch: 543/600... Loss: 0.000\n", 403 | "Epoch: 550/600... Loss: 0.000\n", 404 | "Epoch: 558/600... Loss: 0.000\n", 405 | "Epoch: 565/600... Loss: 0.000\n", 406 | "Epoch: 572/600... Loss: 0.000\n", 407 | "Epoch: 579/600... Loss: 0.000\n", 408 | "Epoch: 586/600... Loss: 0.000\n", 409 | "Epoch: 593/600... Loss: 0.000\n", 410 | "Epoch: 600/600... Loss: 0.000\n" 411 | ], 412 | "name": "stdout" 413 | } 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "metadata": { 419 | "id": "FkUhaQf3CoSL", 420 | "colab_type": "code", 421 | "colab": { 422 | "base_uri": "https://localhost:8080/", 423 | "height": 34 424 | }, 425 | "outputId": "97bd6ab3-e011-408b-91a1-f3d592d3aab2" 426 | }, 427 | "source": [ 428 | "print(f'Model weight after training is: {linear_model.weight.item()}')" 429 | ], 430 | "execution_count": 31, 431 | "outputs": [ 432 | { 433 | "output_type": "stream", 434 | "text": [ 435 | "Model weight after training is: 1.79999959976287\n" 436 | ], 437 | "name": "stdout" 438 | } 439 | ] 440 | }, 441 | { 442 | "cell_type": "code", 443 | "metadata": { 444 | "id": "bHdZVEK0C1iL", 445 | "colab_type": "code", 446 | "colab": { 447 | "base_uri": "https://localhost:8080/", 448 | "height": 34 449 | }, 450 | "outputId": "ec90c5f9-9394-4d5d-d085-735d2921997e" 451 | }, 452 | "source": [ 453 | "print(f'Model bias after training is: {linear_model.bias.item()}')" 454 | ], 455 | "execution_count": 24, 456 | "outputs": [ 457 | { 458 | "output_type": "stream", 459 | "text": [ 460 | "Model bias after training is: 31.958300538443208\n" 461 | ], 462 | "name": "stdout" 463 | } 464 | ] 465 | }, 466 | { 467 | "cell_type": "code", 468 | "metadata": { 469 | "id": "G1LrHSasFcuj", 470 | "colab_type": "code", 471 | "colab": {} 472 | }, 473 | "source": [ 474 | "def run_tests():\n", 475 | " for index in range(len(test_set)):\n", 476 | " prediction = linear_model(test_set[index])\n", 477 | " label = test_set_result_should_be[index]\n", 478 | " print(f'Prediction:{prediction.item()} Label: {label.item()}')\n" 479 | ], 480 | "execution_count": 0, 481 | "outputs": [] 482 | }, 483 | { 484 | "cell_type": "code", 485 | "metadata": { 486 | "id": "3ulFFYSgC3Mz", 487 | "colab_type": "code", 488 | "colab": { 489 | "base_uri": "https://localhost:8080/", 490 | "height": 68 491 | }, 492 | "outputId": "aabe92e2-3a4d-4d0b-a577-758cbc8890ce" 493 | }, 494 | "source": [ 495 | "run_tests()" 496 | ], 497 | "execution_count": 26, 498 | "outputs": [ 499 | { 500 | "output_type": "stream", 501 | "text": [ 502 | "Prediction:159.73611782591695 Label: 159.8\n", 503 | "Prediction:201.12893187678873 Label: 201.2\n", 504 | "Prediction:242.5217459276605 Label: 242.6\n" 505 | ], 506 | "name": "stdout" 507 | } 508 | ] 509 | }, 510 | { 511 | "cell_type": "code", 512 | "metadata": { 513 | "id": "5bo641rMFYHv", 514 | "colab_type": "code", 515 | "colab": {} 516 | }, 517 | "source": [ 518 | "def run_tests():\n", 519 | " for index in range(len(test_set)):\n", 520 | " prediction = linear_model(test_set[index])\n", 521 | " prediction = round(prediction.item(),2)\n", 522 | " label = test_set_result_should_be[index]\n", 523 | " print(f'Rounded Prediction:{prediction} Label: {label.item()}')" 524 | ], 525 | "execution_count": 0, 526 | "outputs": [] 527 | }, 528 | { 529 | "cell_type": "code", 530 | "metadata": { 531 | "id": "SPBy04M0Jyr9", 532 | "colab_type": "code", 533 | "colab": { 534 | "base_uri": "https://localhost:8080/", 535 | "height": 68 536 | }, 537 | "outputId": "9a0e0530-7e19-4920-ff93-b0ca4c2086c4" 538 | }, 539 | "source": [ 540 | "run_tests()" 541 | ], 542 | "execution_count": 28, 543 | "outputs": [ 544 | { 545 | "output_type": "stream", 546 | "text": [ 547 | "Rounded Prediction:159.74 Label: 159.8\n", 548 | "Rounded Prediction:201.13 Label: 201.2\n", 549 | "Rounded Prediction:242.52 Label: 242.6\n" 550 | ], 551 | "name": "stdout" 552 | } 553 | ] 554 | }, 555 | { 556 | "cell_type": "code", 557 | "metadata": { 558 | "id": "xfKCZlQXJ0A2", 559 | "colab_type": "code", 560 | "colab": { 561 | "base_uri": "https://localhost:8080/", 562 | "height": 68 563 | }, 564 | "outputId": "fae1aaa9-cb8e-4a3a-fc42-006d8d737560" 565 | }, 566 | "source": [ 567 | "run_tests()" 568 | ], 569 | "execution_count": 30, 570 | "outputs": [ 571 | { 572 | "output_type": "stream", 573 | "text": [ 574 | "Rounded Prediction:159.8 Label: 159.8\n", 575 | "Rounded Prediction:201.2 Label: 201.2\n", 576 | "Rounded Prediction:242.6 Label: 242.6\n" 577 | ], 578 | "name": "stdout" 579 | } 580 | ] 581 | }, 582 | { 583 | "cell_type": "code", 584 | "metadata": { 585 | "id": "NBf3uUR7LwpA", 586 | "colab_type": "code", 587 | "colab": {} 588 | }, 589 | "source": [ 590 | "" 591 | ], 592 | "execution_count": 0, 593 | "outputs": [] 594 | } 595 | ] 596 | } -------------------------------------------------------------------------------- /code/Export_Model_to_ONNX.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Export model to onnx" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": { 14 | "ExecuteTime": { 15 | "end_time": "2018-12-23T14:29:51.552827Z", 16 | "start_time": "2018-12-23T14:29:51.129934Z" 17 | } 18 | }, 19 | "outputs": [], 20 | "source": [ 21 | "import torch\n", 22 | "from torch.autograd import Variable\n", 23 | "from model_62 import Net" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 2, 29 | "metadata": { 30 | "ExecuteTime": { 31 | "end_time": "2018-12-23T14:29:54.276292Z", 32 | "start_time": "2018-12-23T14:29:51.569888Z" 33 | } 34 | }, 35 | "outputs": [], 36 | "source": [ 37 | "# Create the object model\n", 38 | "model = Net()\n", 39 | "\n", 40 | "# Load model's parameters\n", 41 | "model_checkpoint = '62_classes_acc_8778.pt'\n", 42 | "model.load_state_dict(torch.load('models/{0}'.format(model_checkpoint)))\n", 43 | "\n", 44 | "# Export the model\n", 45 | "dummy_input = Variable(torch.randn(1, 1, 28, 28),requires_grad=True)\n", 46 | "torch.onnx.export(model, dummy_input, \"mnist2.onnx\")" 47 | ] 48 | } 49 | ], 50 | "metadata": { 51 | "kernelspec": { 52 | "display_name": "Python [default]", 53 | "language": "python", 54 | "name": "python3" 55 | }, 56 | "language_info": { 57 | "codemirror_mode": { 58 | "name": "ipython", 59 | "version": 3 60 | }, 61 | "file_extension": ".py", 62 | "mimetype": "text/x-python", 63 | "name": "python", 64 | "nbconvert_exporter": "python", 65 | "pygments_lexer": "ipython3", 66 | "version": "3.6.6" 67 | }, 68 | "varInspector": { 69 | "cols": { 70 | "lenName": 16, 71 | "lenType": 16, 72 | "lenVar": 40 73 | }, 74 | "kernels_config": { 75 | "python": { 76 | "delete_cmd_postfix": "", 77 | "delete_cmd_prefix": "del ", 78 | "library": "var_list.py", 79 | "varRefreshCmd": "print(var_dic_list())" 80 | }, 81 | "r": { 82 | "delete_cmd_postfix": ") ", 83 | "delete_cmd_prefix": "rm(", 84 | "library": "var_list.r", 85 | "varRefreshCmd": "cat(var_dic_list()) " 86 | } 87 | }, 88 | "types_to_exclude": [ 89 | "module", 90 | "function", 91 | "builtin_function_or_method", 92 | "instance", 93 | "_Feature" 94 | ], 95 | "window_display": false 96 | } 97 | }, 98 | "nbformat": 4, 99 | "nbformat_minor": 2 100 | } 101 | -------------------------------------------------------------------------------- /code/Linear_Algebra/1_vector_coding.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Vectors in Python\n", 8 | "\n", 9 | "In the following exercises, you will work on coding vectors in Python.\n", 10 | "\n", 11 | "Assume that you have a state vector \n", 12 | "$$\\mathbf{x_0}$$\n", 13 | "\n", 14 | "representing the x position, y position, velocity in the x direction, and velocity in the y direction of a car that is driving in front of your vehicle. You are tracking the other vehicle.\n", 15 | "\n", 16 | "Currently, the other vehicle is 5 meters ahead of you along your x-axis, 2 meters to your left along your y-axis, driving 10 m/s in the x direction and 0 m/s in the y-direction. How would you represent this in a Python list where the vector contains `` in exactly that order? \n", 17 | "\n", 18 | "\n", 19 | "### Vector Assignment: Example 1" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": 1, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "## Practice working with Python vectors\n", 29 | "\n", 30 | "## TODO: Assume the state vector contains values for \n", 31 | "## Currently, x = 5, y = 2, vx = 10, vy = 0\n", 32 | "## Represent this information in a list\n", 33 | "x0 = [5,2,10,0]\n" 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "### Test your code\n", 41 | "\n", 42 | "Run the cell below to test your code. \n", 43 | "\n", 44 | "The test code uses a Python assert statement. If you have a code statement that resolves to either True or False, an assert statement will either:\n", 45 | "* do nothing if the statement is True\n", 46 | "* throw an error if the statement is False\n", 47 | "\n", 48 | "\n", 49 | "A Python assert statement\n", 50 | "will output an error if the answer was not as expected. If the\n", 51 | "answer was as expected, then nothing will be outputted." 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": 2, 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "### Test Cases \n", 61 | "### Run these test cases to see if your results are as expected\n", 62 | "### Running this cell should produce no output if all assertions are True\n", 63 | "\n", 64 | "assert x0 == [5, 2, 10, 0]" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "### Vector Assignment: Example 2\n", 72 | "\n", 73 | "The vehicle ahead of you has now moved farther away from you. You know that the vehicle has moved 3 meters forward in the x-direction, 5 meters forward in the y-direction, has increased its x velocity by 2 m/s and has increased its y velocity by 5 m/s. \n", 74 | "\n", 75 | "Store the change in position and velocity in a list variable called xdelta" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": 4, 81 | "metadata": {}, 82 | "outputs": [], 83 | "source": [ 84 | "## TODO: Assign the change in position and velocity to the variable\n", 85 | "## xdelta. Remember that the order of the vector is x, y, vx, vy\n", 86 | "\n", 87 | "xdelta = [3,5,2,5]" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": 5, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "### Test Case \n", 97 | "### Run this test case to see if your results are as expected\n", 98 | "### Running this cell should produce no output if all assertions are True\n", 99 | "\n", 100 | "assert xdelta == [3, 5, 2, 5]" 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "### Vector Math: Addition\n", 108 | "\n", 109 | "Calculate the tracked vehicle's new position and velocity. Here are the steps you need to carry this out:\n", 110 | "\n", 111 | "* initialize an empty list called x1\n", 112 | "* add xdelta to x0 using a for loop\n", 113 | "* store your results in x1 as you iterate through the for loop using the append method" 114 | ] 115 | }, 116 | { 117 | "cell_type": "code", 118 | "execution_count": 14, 119 | "metadata": {}, 120 | "outputs": [], 121 | "source": [ 122 | "## TODO: Add the vectors together element-wise. For example, \n", 123 | "## element-wise addition of [2, 6] and [10, 3] is [12, 9].\n", 124 | "## Place the answer in the x1 variable. \n", 125 | "##\n", 126 | "## Hint: You can use a for loop. The append method might also\n", 127 | "## be helpful.\n", 128 | "\n", 129 | "x1 = []\n", 130 | "for idx in range(len(x0)):\n", 131 | " x1.append(x0[idx]+xdelta[idx])\n" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "execution_count": 15, 137 | "metadata": {}, 138 | "outputs": [], 139 | "source": [ 140 | "### Test Case \n", 141 | "### Run this test case to see if your results are as expected\n", 142 | "### Running this cell should produce no output if all assertions are True\n", 143 | "assert x1 == [8, 7, 12, 5]" 144 | ] 145 | }, 146 | { 147 | "cell_type": "markdown", 148 | "metadata": {}, 149 | "source": [ 150 | "### Vector Math: Scalar Multiplication\n", 151 | "\n", 152 | "You have your current position in meters and current velocity in meters per second. But you need to report your results at a company meeting where most people will only be familiar with working in feet rather than meters. Convert your position vector x1 to feet and feet/second.\n", 153 | "\n", 154 | "This will involve scalar multiplication. The process for coding scalar multiplication is very similar to vector addition. You will need to:\n", 155 | "* initialize an empty list\n", 156 | "* use a for loop to access each element in the vector\n", 157 | "* multiply each element by the scalar\n", 158 | "* append the result to the empty list" 159 | ] 160 | }, 161 | { 162 | "cell_type": "code", 163 | "execution_count": 18, 164 | "metadata": {}, 165 | "outputs": [ 166 | { 167 | "data": { 168 | "text/plain": [ 169 | "[8, 7, 12, 5]" 170 | ] 171 | }, 172 | "execution_count": 18, 173 | "metadata": {}, 174 | "output_type": "execute_result" 175 | } 176 | ], 177 | "source": [ 178 | "## TODO: Multiply each element in the x1 vector by the conversion\n", 179 | "## factor shown belowand store the results in the variable s. \n", 180 | "## Use a for loop\n", 181 | "\n", 182 | "meters_to_feet = 1.0 / 0.3048\n", 183 | "x1feet =[]\n", 184 | "for idx in range(len(x1)):\n", 185 | " x1feet.append(x1[idx]*meters_to_feet)\n", 186 | "x1" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": 17, 192 | "metadata": {}, 193 | "outputs": [], 194 | "source": [ 195 | "### Test Cases\n", 196 | "### Run this test case to see if your results are as expected\n", 197 | "### Running this cell should produce no output if all assertions are True\n", 198 | "x1feet_sol = [8/.3048, 7/.3048, 12/.3048, 5/.3048]\n", 199 | "\n", 200 | "assert(len(x1feet) == len(x1feet_sol)) \n", 201 | "for response, expected in zip(x1feet, x1feet_sol):\n", 202 | " assert(abs(response-expected) < 0.001)" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": {}, 208 | "source": [ 209 | "### Vector Math: Dot Product\n", 210 | "\n", 211 | "The tracked vehicle is currently at the state represented by \n", 212 | "$$\\mathbf{x_1} = [8, 7, 12, 5] $$.\n", 213 | "\n", 214 | "Where will the vehicle be in two seconds?\n", 215 | "\n", 216 | "You could actually solve this problem very quickly using Matrix multiplication, but we have not covered that yet. Instead, think about the x-direction and y-direction separately and how you could do this with the dot product.\n", 217 | "\n", 218 | "#### Solving with the Dot Product\n", 219 | "You know that the tracked vehicle at x1 is 8m ahead of you in the x-direction and traveling at 12m/s. Assuming constant velocity, the new x-position after 2 seconds would be\n", 220 | "\n", 221 | "$$8 + 12*2 = 32$$\n", 222 | "\n", 223 | "The new y-position would be\n", 224 | "$$7 + 5*2 = 17$$\n", 225 | "\n", 226 | "You could actually solve each of these equations using the dot product:\n", 227 | "\n", 228 | "$$x_2 = [8, 7, 12, 5]\\cdot[1, 0, 2, 0] \\\\\\ \n", 229 | "= 8\\times1 + 7\\times0 + 12\\times2 + 5\\times0 \\\\\\\n", 230 | "= 32$$\n", 231 | "\n", 232 | "$$y_2 = [8, 7, 12, 5]\\cdot[0, 1, 0, 2] \\\\\\ \n", 233 | "= 8\\times0 + 7\\times1 + 12\\times0 + 5\\times2 \\\\\\\n", 234 | "= 17$$\n", 235 | "\n", 236 | "Since you are assuming constant velocity, the final state vector would be \n", 237 | "\n", 238 | "$$\\mathbf{x_2} = [32, 17, 12, 5]$$\n", 239 | "\n", 240 | "#### Coding the Dot Product\n", 241 | "\n", 242 | "Now, calculate the state vector $$\\mathbf{x_2}$$ but with code. You will need to calculate the dot product of two vectors. Rather than writing the dot product code for the x-direction and then copying the code for the y-direction, write a function that calculates the dot product of two Python lists.\n", 243 | "\n", 244 | "Here is an outline of the steps:\n", 245 | "* initialize an empty list\n", 246 | "* initialize a variable with value zero to accumulate the sum\n", 247 | "* use a for loop to iterate through the vectors. Assume the two vectors have the same length\n", 248 | "* accumulate the sum as you multiply elements together\n", 249 | "\n", 250 | "You will see in the starter code that x2 is already being calculated for you based on the results of your dotproduct function" 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": 21, 256 | "metadata": {}, 257 | "outputs": [], 258 | "source": [ 259 | "## TODO: Fill in the dotproduct() function to calculate the \n", 260 | "## dot product of two vectors.\n", 261 | "##\n", 262 | "\n", 263 | "## Here are the inputs and outputs of the dotproduct() function:\n", 264 | "## INPUTS: vector, vector\n", 265 | "## OUTPUT: dot product of the two vectors\n", 266 | "## \n", 267 | "##\n", 268 | "## The dot product involves mutliplying the vectors element\n", 269 | "## by element and then taking the sum of the results\n", 270 | "##\n", 271 | "## For example, the dot product of [9, 7, 5] and [2, 3, 4] is \n", 272 | "## 9*2+7*3 +5*4 = 59\n", 273 | "## \n", 274 | "## Hint: You can use a for loop. You will also need to accumulate\n", 275 | "## the sum as you iterate through the vectors. In Python, you can accumulate \n", 276 | "## sums with syntax like w = w + 1\n", 277 | "\n", 278 | "x2 = []\n", 279 | "\n", 280 | "def dotproduct(vectora, vectorb):\n", 281 | " \n", 282 | " # variable for accumulating the sum\n", 283 | " result = 0\n", 284 | " \n", 285 | " # TODO: Use a for loop to multiply the two vectors\n", 286 | " # element by element. Accumulate the sum in the result variable\n", 287 | " for idx in range(len(vectora)):\n", 288 | " result += vectora[idx] * vectorb[idx]\n", 289 | " \n", 290 | " return result\n", 291 | " \n", 292 | "x2 = [dotproduct([8, 7, 12, 5], [1, 0, 2, 0]), \n", 293 | " dotproduct([8, 7, 12, 5], [0, 1, 0, 2]),\n", 294 | " 12,\n", 295 | " 5]" 296 | ] 297 | }, 298 | { 299 | "cell_type": "code", 300 | "execution_count": 22, 301 | "metadata": {}, 302 | "outputs": [], 303 | "source": [ 304 | "### Test Case\n", 305 | "### Run this test case to see if your results are as expected\n", 306 | "### Running this cell should produce no output if all assertions are True\n", 307 | "assert x2 == [32, 17, 12, 5]" 308 | ] 309 | }, 310 | { 311 | "cell_type": "code", 312 | "execution_count": null, 313 | "metadata": {}, 314 | "outputs": [], 315 | "source": [] 316 | } 317 | ], 318 | "metadata": { 319 | "kernelspec": { 320 | "display_name": "Python 3", 321 | "language": "python", 322 | "name": "python3" 323 | }, 324 | "language_info": { 325 | "codemirror_mode": { 326 | "name": "ipython", 327 | "version": 3 328 | }, 329 | "file_extension": ".py", 330 | "mimetype": "text/x-python", 331 | "name": "python", 332 | "nbconvert_exporter": "python", 333 | "pygments_lexer": "ipython3", 334 | "version": "3.6.3" 335 | } 336 | }, 337 | "nbformat": 4, 338 | "nbformat_minor": 2 339 | } 340 | -------------------------------------------------------------------------------- /code/Linear_Algebra/2_matrices_in_python.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Coding Matrices\n", 8 | "\n", 9 | "Here are a few exercises to get you started with coding matrices. The exercises start off with vectors and then get more challenging\n", 10 | "\n", 11 | "### Vectors" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": 1, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "### TODO: Assign the vector <5, 10, 2, 6, 1> to the variable v\n", 21 | "v = [5, 10, 2, 6, 1]" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "The v variable contains a Python list. This list could also be thought of as a 1x5 matrix with 1 row and 5 columns. How would you represent this list as a matrix?" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 2, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "### TODO: Assign the vector <5, 10, 2, 6, 1> to the variable mv\n", 38 | "### The difference between a vector and a matrix in Python is that\n", 39 | "### a matrix is a list of lists.\n", 40 | "\n", 41 | "### Hint: See the last quiz on the previous page\n", 42 | "\n", 43 | "mv = [v]" 44 | ] 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "metadata": {}, 49 | "source": [ 50 | "How would you represent this vector in its vertical form with 5 rows and 1 column? When defining matrices in Python, each row is a list. So in this case, you have 5 rows and thus will need 5 lists.\n", 51 | "\n", 52 | "As an example, this is what the vector $$<5, 7>$$ would look like as a 1x2 matrix in Python: \n", 53 | "```python\n", 54 | "matrix1by2 = [\n", 55 | " [5, 7]\n", 56 | "]\n", 57 | "```\n", 58 | "\n", 59 | "And here is what the same vector would look like as a 2x1 matrix:\n", 60 | "```python\n", 61 | "matrix2by1 = [\n", 62 | " [5], \n", 63 | " [7]\n", 64 | "]\n", 65 | "```" 66 | ] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "execution_count": 3, 71 | "metadata": {}, 72 | "outputs": [], 73 | "source": [ 74 | "### TODO: Assign the vector <5, 10, 2, 6, 1> to the variable vT\n", 75 | "### vT is a 5x1 matrix\n", 76 | "vT = [\n", 77 | " [5],\n", 78 | " [10],\n", 79 | " [2],\n", 80 | " [6],\n", 81 | " [1]\n", 82 | "]" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "### Assigning Matrices to Variables" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": 4, 95 | "metadata": {}, 96 | "outputs": [], 97 | "source": [ 98 | "### TODO: Assign the following matrix to the variable m\n", 99 | "### 8 7 1 2 3\n", 100 | "### 1 5 2 9 0\n", 101 | "### 8 2 2 4 1\n", 102 | "\n", 103 | "m = [[8, 7, 1, 2, 3],\n", 104 | " [1, 5, 2, 9, 0],\n", 105 | " [8, 2, 2, 4, 1]\n", 106 | " ]" 107 | ] 108 | }, 109 | { 110 | "cell_type": "markdown", 111 | "metadata": {}, 112 | "source": [ 113 | "### Accessing Matrix Values" 114 | ] 115 | }, 116 | { 117 | "cell_type": "code", 118 | "execution_count": 8, 119 | "metadata": {}, 120 | "outputs": [ 121 | { 122 | "data": { 123 | "text/plain": [ 124 | "[[8, 7, 1, 2, 3], [1, 5, 2, 9, 5], [8, 2, 2, 4, 1]]" 125 | ] 126 | }, 127 | "execution_count": 8, 128 | "metadata": {}, 129 | "output_type": "execute_result" 130 | } 131 | ], 132 | "source": [ 133 | "### TODO: In matrix m, change the value \n", 134 | "### in the second row last column from 0 to 5\n", 135 | "### Hint: You do not need to rewrite the entire matrix\n", 136 | "m[1][4]=5\n", 137 | "m" 138 | ] 139 | }, 140 | { 141 | "cell_type": "markdown", 142 | "metadata": {}, 143 | "source": [ 144 | "### Looping through Matrices to do Math\n", 145 | "\n", 146 | "Coding mathematical operations with matrices can be tricky. Because matrices are lists of lists, you will need to use a for loop inside another for loop. The outside for loop iterates over the rows and the inside for loop iterates over the columns.\n", 147 | "\n", 148 | "\n", 149 | "Here is some pseudo code\n", 150 | "```python\n", 151 | "for i in number of rows:\n", 152 | " for j in number of columns:\n", 153 | " mymatrix[i][j]\n", 154 | "```\n", 155 | "\n", 156 | "To figure out how many times to loop over the matrix, you need to know the number of rows and number of columns. \n", 157 | "\n", 158 | "\n", 159 | "If you have a variable with a matrix in it, how could you figure out the number of rows? How could you figure out the number of columns? The [len](https://docs.python.org/2/library/functions.html#len) function in Python might be helpful." 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "### Scalar Multiplication" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 15, 172 | "metadata": {}, 173 | "outputs": [], 174 | "source": [ 175 | "### TODO: Use for loops to multiply each matrix element by 5\n", 176 | "### Store the answer in the r variable. This is called scalar\n", 177 | "### multiplication\n", 178 | "###\n", 179 | "### HINT: First write a for loop that iterates through the rows\n", 180 | "### one row at a time\n", 181 | "###\n", 182 | "### Then write another for loop within the for loop that\n", 183 | "### iterates through the columns\n", 184 | "###\n", 185 | "### If you used the variable i to represent rows and j\n", 186 | "### to represent columns, then m[i][j] would give you\n", 187 | "### access to each element in the matrix\n", 188 | "###\n", 189 | "### Because r is an empty list, you cannot directly assign\n", 190 | "### a value like r[i][j] = m[i][j]. You might have to\n", 191 | "### work on one row at a time and then use r.append(row).\n", 192 | "r = []\n", 193 | "\n", 194 | "for row in range(len(m)):\n", 195 | " new_row = []\n", 196 | " for col in range(len(m[row])):\n", 197 | " new_row.append(m[row][col] * 5)\n", 198 | " r.append(new_row)" 199 | ] 200 | }, 201 | { 202 | "cell_type": "markdown", 203 | "metadata": {}, 204 | "source": [ 205 | "### Printing Out a Matrix" 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "execution_count": 24, 211 | "metadata": {}, 212 | "outputs": [ 213 | { 214 | "name": "stdout", 215 | "output_type": "stream", 216 | "text": [ 217 | "8\t7\t1\t2\t3\t\n", 218 | "1\t5\t2\t9\t5\t\n", 219 | "8\t2\t2\t4\t1\t\n", 220 | "\n" 221 | ] 222 | } 223 | ], 224 | "source": [ 225 | "### TODO: Write a function called matrix_print() \n", 226 | "### that prints out a matrix in\n", 227 | "### a way that is easy to read.\n", 228 | "### Each element in a row should be separated by a tab\n", 229 | "### And each row should have its own line\n", 230 | "### You can test our your results with the m matrix\n", 231 | "\n", 232 | "### HINT: You can use a for loop within a for loop\n", 233 | "### In Python, the print() function will be useful\n", 234 | "### print(5, '\\t', end = '') will print out the integer 5, \n", 235 | "### then add a tab after the 5. The end = '' makes sure that\n", 236 | "### the print function does not print out a new line if you do\n", 237 | "### not want a new line.\n", 238 | "\n", 239 | "### Your output should look like this\n", 240 | "### 8 7 1 2 3 \n", 241 | "### 1 5 2 9 5 \n", 242 | "### 8 2 2 4 1\n", 243 | "\n", 244 | "def matrix_print(matrix):\n", 245 | " matrix_str = ''\n", 246 | " for row in range(len(matrix)):\n", 247 | " row_str = ''\n", 248 | " for col in range(len(matrix[row])):\n", 249 | " row_str += str(matrix[row][col])+ '\\t'\n", 250 | " matrix_str += row_str + '\\n'\n", 251 | " print(matrix_str)\n", 252 | " return\n", 253 | "\n", 254 | "m = [\n", 255 | " [8, 7, 1, 2, 3],\n", 256 | " [1, 5, 2, 9, 5],\n", 257 | " [8, 2, 2, 4, 1]\n", 258 | "]\n", 259 | "\n", 260 | "matrix_print(m)" 261 | ] 262 | }, 263 | { 264 | "cell_type": "markdown", 265 | "metadata": {}, 266 | "source": [ 267 | "### Test Your Results" 268 | ] 269 | }, 270 | { 271 | "cell_type": "code", 272 | "execution_count": 17, 273 | "metadata": {}, 274 | "outputs": [], 275 | "source": [ 276 | "### You can run these tests to see if you have the expected\n", 277 | "### results. If everything is correct, this cell has no output\n", 278 | "\n", 279 | "assert v == [5, 10, 2, 6, 1]\n", 280 | "assert mv == [\n", 281 | " [5, 10, 2, 6, 1]\n", 282 | "]\n", 283 | "\n", 284 | "assert vT == [\n", 285 | " [5], \n", 286 | " [10], \n", 287 | " [2], \n", 288 | " [6], \n", 289 | " [1]]\n", 290 | "\n", 291 | "assert m == [\n", 292 | " [8, 7, 1, 2, 3], \n", 293 | " [1, 5, 2, 9, 5], \n", 294 | " [8, 2, 2, 4, 1]\n", 295 | "]\n", 296 | "\n" 297 | ] 298 | }, 299 | { 300 | "cell_type": "code", 301 | "execution_count": 18, 302 | "metadata": {}, 303 | "outputs": [], 304 | "source": [ 305 | "assert r == [\n", 306 | " [40, 35, 5, 10, 15], \n", 307 | " [5, 25, 10, 45, 25], \n", 308 | " [40, 10, 10, 20, 5]\n", 309 | "]" 310 | ] 311 | }, 312 | { 313 | "cell_type": "markdown", 314 | "metadata": {}, 315 | "source": [ 316 | "### Print Out Your Results" 317 | ] 318 | }, 319 | { 320 | "cell_type": "code", 321 | "execution_count": 19, 322 | "metadata": {}, 323 | "outputs": [ 324 | { 325 | "name": "stdout", 326 | "output_type": "stream", 327 | "text": [ 328 | "[5, 10, 2, 6, 1]\n", 329 | "[[5, 10, 2, 6, 1]]\n", 330 | "[[5], [10], [2], [6], [1]]\n", 331 | "[[8, 7, 1, 2, 3], [1, 5, 2, 9, 5], [8, 2, 2, 4, 1]]\n", 332 | "[[40, 35, 5, 10, 15], [5, 25, 10, 45, 25], [40, 10, 10, 20, 5]]\n" 333 | ] 334 | } 335 | ], 336 | "source": [ 337 | "### Run this cell to print out your answers\n", 338 | "print(v)\n", 339 | "print(mv)\n", 340 | "print(vT)\n", 341 | "print(m)\n", 342 | "print(r)" 343 | ] 344 | }, 345 | { 346 | "cell_type": "code", 347 | "execution_count": null, 348 | "metadata": {}, 349 | "outputs": [], 350 | "source": [] 351 | } 352 | ], 353 | "metadata": { 354 | "kernelspec": { 355 | "display_name": "Python 3", 356 | "language": "python", 357 | "name": "python3" 358 | }, 359 | "language_info": { 360 | "codemirror_mode": { 361 | "name": "ipython", 362 | "version": 3 363 | }, 364 | "file_extension": ".py", 365 | "mimetype": "text/x-python", 366 | "name": "python", 367 | "nbconvert_exporter": "python", 368 | "pygments_lexer": "ipython3", 369 | "version": "3.6.3" 370 | } 371 | }, 372 | "nbformat": 4, 373 | "nbformat_minor": 2 374 | } 375 | -------------------------------------------------------------------------------- /code/Linear_Algebra/3_matrix_addition.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Matrix Addition\n", 8 | "\n", 9 | "In this exercises, you will write a function that accepts two matrices and outputs their sum. Think about how you could do this with a for loop nested inside another for loop." 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 1, 15 | "metadata": {}, 16 | "outputs": [ 17 | { 18 | "data": { 19 | "text/plain": [ 20 | "[[9, 24, 6.1],\n", 21 | " [12.5, 18.2, 14.8],\n", 22 | " [4.8, 2.5, 13],\n", 23 | " [16, 10, 6],\n", 24 | " [4, 2, 12],\n", 25 | " [7, 36, 2]]" 26 | ] 27 | }, 28 | "execution_count": 1, 29 | "metadata": {}, 30 | "output_type": "execute_result" 31 | } 32 | ], 33 | "source": [ 34 | "### TODO: Write a function called matrix_addition that \n", 35 | "### calculate the sum of two matrices\n", 36 | "###\n", 37 | "### INPUTS:\n", 38 | "### matrix A _ an m x n matrix\n", 39 | "### matrix B _ an m x n matrix\n", 40 | "###\n", 41 | "### OUPUT:\n", 42 | "### matrixSum _ sum of matrix A + matrix B\n", 43 | "\n", 44 | "def matrix_addition(matrixA, matrixB):\n", 45 | "\n", 46 | " # initialize matrix to hold the results\n", 47 | " matrixSum = []\n", 48 | " \n", 49 | " # matrix to hold a row for appending sums of each element\n", 50 | " row = []\n", 51 | " for row in range(len(matrixA)):\n", 52 | " new_row = []\n", 53 | " for col in range(len(matrixA[row])):\n", 54 | " new_row.append(matrixA[row][col] + matrixB[row][col])\n", 55 | " matrixSum.append(new_row)\n", 56 | " \n", 57 | " # TODO: write a for loop within a for loop to iterate over\n", 58 | " # the matrices\n", 59 | " \n", 60 | " # TODO: As you iterate through the matrices, add matching\n", 61 | " # elements and append the sum to the row variable\n", 62 | " \n", 63 | " # TODO: When a row is filled, append the row to matrixSum. \n", 64 | " # Then reinitialize row as an empty list\n", 65 | " \n", 66 | " return matrixSum\n", 67 | "\n", 68 | "### When you run this code cell, your matrix addition function\n", 69 | "### will run on the A and B matrix.\n", 70 | "\n", 71 | "A = [\n", 72 | " [2,5,1], \n", 73 | " [6,9,7.4], \n", 74 | " [2,1,1], \n", 75 | " [8,5,3], \n", 76 | " [2,1,6], \n", 77 | " [5,3,1]\n", 78 | "]\n", 79 | "\n", 80 | "B = [\n", 81 | " [7, 19, 5.1], \n", 82 | " [6.5,9.2,7.4], \n", 83 | " [2.8,1.5,12], \n", 84 | " [8,5,3], \n", 85 | " [2,1,6], \n", 86 | " [2,33,1]\n", 87 | "]\n", 88 | "\n", 89 | "matrix_addition(A, B)" 90 | ] 91 | }, 92 | { 93 | "cell_type": "markdown", 94 | "metadata": {}, 95 | "source": [ 96 | "### Vectors versus Matrices\n", 97 | "\n", 98 | "What happens if you run the cell below? Here you are adding two vectors together. Does your code still work?" 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "execution_count": 2, 104 | "metadata": {}, 105 | "outputs": [ 106 | { 107 | "ename": "TypeError", 108 | "evalue": "object of type 'int' has no len()", 109 | "output_type": "error", 110 | "traceback": [ 111 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", 112 | "\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)", 113 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mmatrix_addition\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", 114 | "\u001b[0;32m\u001b[0m in \u001b[0;36mmatrix_addition\u001b[0;34m(matrixA, matrixB)\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mrow\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmatrixA\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[0mnew_row\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m \u001b[0;32mfor\u001b[0m \u001b[0mcol\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmatrixA\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mrow\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 21\u001b[0m \u001b[0mnew_row\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmatrixA\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mrow\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mcol\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mmatrixB\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mrow\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mcol\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 22\u001b[0m \u001b[0mmatrixSum\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_row\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", 115 | "\u001b[0;31mTypeError\u001b[0m: object of type 'int' has no len()" 116 | ] 117 | } 118 | ], 119 | "source": [ 120 | "matrix_addition([4, 2, 1], [5, 2, 7])" 121 | ] 122 | }, 123 | { 124 | "cell_type": "markdown", 125 | "metadata": {}, 126 | "source": [ 127 | "Why did this error occur? Because your code assumes that a matrix is a two-dimensional grid represented by a list of lists. But a horizontal vector, which can also be considered a matrix, is a one-dimensional grid represented by a single list.\n", 128 | "\n", 129 | "What happens if you store a vector as a list of lists like [[4, 2, 1]] and [[5, 2, 7]]? Does your function work? Run the code cell below to find out." 130 | ] 131 | }, 132 | { 133 | "cell_type": "code", 134 | "execution_count": 3, 135 | "metadata": {}, 136 | "outputs": [ 137 | { 138 | "data": { 139 | "text/plain": [ 140 | "[[9, 4, 8]]" 141 | ] 142 | }, 143 | "execution_count": 3, 144 | "metadata": {}, 145 | "output_type": "execute_result" 146 | } 147 | ], 148 | "source": [ 149 | "matrix_addition([[4, 2, 1]], [[5, 2, 7]])" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "metadata": {}, 155 | "source": [ 156 | "### Test your Code\n", 157 | "Run the cell below. If there is no output, then your results are as expected." 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": null, 163 | "metadata": { 164 | "collapsed": true 165 | }, 166 | "outputs": [], 167 | "source": [ 168 | "assert matrix_addition([\n", 169 | " [1, 2, 3]], \n", 170 | " [[4, 5, 6]]) == [[5, 7, 9]]\n", 171 | "\n", 172 | "assert matrix_addition([\n", 173 | " [4]], [\n", 174 | " [5]]) == [[9]]\n", 175 | "\n", 176 | "assert matrix_addition([[1, 2, 3], \n", 177 | " [4, 5, 6]], \n", 178 | " [[7, 8, 9], \n", 179 | " [10, 11, 12]]) == [[8, 10, 12], \n", 180 | " [14, 16, 18]]" 181 | ] 182 | } 183 | ], 184 | "metadata": { 185 | "kernelspec": { 186 | "display_name": "Python 3", 187 | "language": "python", 188 | "name": "python3" 189 | }, 190 | "language_info": { 191 | "codemirror_mode": { 192 | "name": "ipython", 193 | "version": 3 194 | }, 195 | "file_extension": ".py", 196 | "mimetype": "text/x-python", 197 | "name": "python", 198 | "nbconvert_exporter": "python", 199 | "pygments_lexer": "ipython3", 200 | "version": "3.6.3" 201 | } 202 | }, 203 | "nbformat": 4, 204 | "nbformat_minor": 2 205 | } 206 | -------------------------------------------------------------------------------- /code/Linear_Algebra/4_matrix_multiplication.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Matrix Multiplication\n", 8 | "\n", 9 | "Matrix multiplication involves quite a few steps in terms of writing code. But remember that the basics of matrix multiplicaiton involve taking a row in matrix A and finding the dot product with a column in matrix B.\n", 10 | "\n", 11 | "So you are going to write a function to extract a row from matrix A, extract a column from matrix B, and then calculate the dot product of the row and column.\n", 12 | "\n", 13 | "Then you can use these functions to output the results of multiplying two matrices together.\n", 14 | "\n", 15 | "Here is a general outline of the code that you will be writing. Assume you are calculating the product of \n", 16 | "\n", 17 | "$$A\\times{B}$$\n", 18 | "\n", 19 | "* Write a nested for loop that iterates through the m rows of matrix A and the p columns of matrix B\n", 20 | "* Initialize an empty list that will hold values of the final matrix\n", 21 | "* Starting with the first row of matrix A, find the dot product with the first column of matrix B\n", 22 | "* Append the result to the empty list representing a row in the final matrix\n", 23 | "* Now find the dot product between the first row of matrix A and the second column of matrix B\n", 24 | "* Append the result to the row list\n", 25 | "* Keep going until you get to the last column of B\n", 26 | "* Append the row list to the output variable. Reinitialize the row list so that it is empty.\n", 27 | "* Then start on row two of matrix A. Iterate through all of the columns of B taking the dot product\n", 28 | "* etc..." 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "# Breaking the Process down into steps\n", 36 | "\n", 37 | "Rather than writing all of the matrix multiplication code in one function, you are going to break the process down into several functions:\n", 38 | "\n", 39 | "**get_row(matrix, row_number)**\n", 40 | "\n", 41 | "Because you are going to need the rows from matrix A, you will use a function called get_row that takes in a matrix and row number and then returns a row of a matrix. We have provided this function for you.\n", 42 | "\n", 43 | "**get_column(matrix, column_number)**\n", 44 | "\n", 45 | "Likewise, you will need the columns from matrix B. So you will write a similar function that receives a matrix and column number and then returns a column from matrix B.\n", 46 | "\n", 47 | "**dot_product(vectorA, vectorB)**\n", 48 | "\n", 49 | "You have actually already written this function in a prevoius exercise. The dot_product function calculates the dot product of two vectors. \n", 50 | "\n", 51 | "**matrix_multiply(matrixA, matrixB)**\n", 52 | "\n", 53 | "This is the function that will calculate the product of the two matrices. You will need to write a nested for loop that iterates through the rows of A and columns of B. For each row-column combination, you will calculate the dot product and then append the result to the output matrix." 54 | ] 55 | }, 56 | { 57 | "cell_type": "markdown", 58 | "metadata": {}, 59 | "source": [ 60 | "# get_row\n", 61 | "\n", 62 | "The first function is the get_row function. We have provided this function for you. \n", 63 | "\n", 64 | "The get_row function has two inputs and one output.\n", 65 | "\n", 66 | "INPUTS\n", 67 | "* matrix\n", 68 | "* row number\n", 69 | "\n", 70 | "OUTPUT\n", 71 | "* a list, which represents one row of the matrix\n", 72 | "\n", 73 | "In Python, a matrix is a list of lists. If you have a matrix like this one:\n", 74 | "```python\n", 75 | "m = [\n", 76 | " [5, 9, 11, 2],\n", 77 | " [3, 2, 99, 3],\n", 78 | " [7, 1, 8, 2]\n", 79 | "]\n", 80 | "```\n", 81 | "\n", 82 | "then row one would be accessed by \n", 83 | "```\n", 84 | "m[0]\n", 85 | "```\n", 86 | "\n", 87 | "row two would be \n", 88 | "```\n", 89 | "m[1]\n", 90 | "```\n", 91 | "\n", 92 | "and row three would be \n", 93 | "```\n", 94 | "m[2]\n", 95 | "```" 96 | ] 97 | }, 98 | { 99 | "cell_type": "code", 100 | "execution_count": 1, 101 | "metadata": {}, 102 | "outputs": [], 103 | "source": [ 104 | "## TODO: Run this code cell to load the get_row function\n", 105 | "## You do not need to modify this cell\n", 106 | "\n", 107 | "def get_row(matrix, row):\n", 108 | " return matrix[row]" 109 | ] 110 | }, 111 | { 112 | "cell_type": "markdown", 113 | "metadata": {}, 114 | "source": [ 115 | "# Getting a Column from a Matrix\n", 116 | "\n", 117 | "Since matrices are stored as lists of lists, it's relatively simple to extract a row. If A is a matrix, then\n", 118 | "```\n", 119 | "A[0]\n", 120 | "```\n", 121 | "\n", 122 | "will output the first row of the matrix.\n", 123 | "```\n", 124 | "A[1]\n", 125 | "```\n", 126 | "\n", 127 | "outputs the second row of the matrix.\n", 128 | "\n", 129 | "But what if you want to get a matrix column? It's not as convenient. To get the values of the first column, you would need to output:\n", 130 | "```\n", 131 | "A[0][0]\n", 132 | "A[1][0]\n", 133 | "A[2][0]\n", 134 | "...\n", 135 | "A[m][0]\n", 136 | "```\n", 137 | "\n", 138 | "For matrix multiplication, you will need to have access to the columns of the B matrix. So write a function called get_column that receives a matrix and a column number indexed from zero. The function then outputs a vector as a list that contains the column. For example\n", 139 | "```\n", 140 | "get_column([\n", 141 | " [1, 2, 4], \n", 142 | " [7, 8, 1], \n", 143 | " [5, 2, 1]\n", 144 | " ], \n", 145 | " 1)\n", 146 | "```\n", 147 | "\n", 148 | "would output the second column\n", 149 | "```\n", 150 | "[2, 8, 2]\n", 151 | "```\n" 152 | ] 153 | }, 154 | { 155 | "cell_type": "markdown", 156 | "metadata": {}, 157 | "source": [ 158 | "# get_column\n", 159 | "\n", 160 | "The get_column function is similar to the get_row function except now you will return a column.\n", 161 | "\n", 162 | "Here are the inputs and outputs of the function\n", 163 | "\n", 164 | "INPUTS \n", 165 | "* matrix\n", 166 | "* column number\n", 167 | "\n", 168 | "OUPUT\n", 169 | "* a list, which represents a column of the matrix\n", 170 | "\n", 171 | "Getting a matrix column is actually more difficult than getting a matrix row.\n", 172 | "\n", 173 | "Take a look again at this example matrix:\n", 174 | "```python\n", 175 | "m = [\n", 176 | " [5, 9, 11, 2],\n", 177 | " [3, 2, 99, 3],\n", 178 | " [7, 1, 8, 2]\n", 179 | "]\n", 180 | "```\n", 181 | "\n", 182 | "What if you wanted to extract the first column as a list [5, 3, 7]. You can't actually get that column directly like you could with a row.\n", 183 | "\n", 184 | "You'll need to think about using a for statement to iterate through the rows and grab the specific values that you want for your column list.\n" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": 4, 190 | "metadata": {}, 191 | "outputs": [], 192 | "source": [ 193 | "### TODO: Write a function that receives a matrix and a column number.\n", 194 | "### the output should be the column in the form of a list\n", 195 | "\n", 196 | "\n", 197 | "### Example input:\n", 198 | "# matrix = [\n", 199 | "# [5, 9, 11, 2],\n", 200 | "# [3, 2, 99, 3],\n", 201 | "# [7, 1, 8, 2]\n", 202 | "# ]\n", 203 | "#\n", 204 | "# column_number = 1\n", 205 | "\n", 206 | "### Example output:\n", 207 | "# [9, 2, 1]\n", 208 | "#\n", 209 | "\n", 210 | "def get_column(matrix, column_number):\n", 211 | " column = []\n", 212 | " for row in range(len(matrix)):\n", 213 | " column.append(matrix[row][column_number])\n", 214 | " return column" 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "execution_count": 5, 220 | "metadata": {}, 221 | "outputs": [], 222 | "source": [ 223 | "### TODO: Run this code to test your get_column function\n", 224 | "assert get_column([[1, 2, 4], \n", 225 | " [7, 8, 1], \n", 226 | " [5, 2, 1]], 1) == [2, 8, 2]\n", 227 | "\n", 228 | "assert get_column([[5]], 0) == [5]" 229 | ] 230 | }, 231 | { 232 | "cell_type": "markdown", 233 | "metadata": {}, 234 | "source": [ 235 | "### Dot Product of Two Vectors\n", 236 | "\n", 237 | "As part of calculating the product of a matrix, you need to do calculate the dot product of a row from matrix A with a column from matrix B. You will do this process many times, so why not abstract the process into a function?\n", 238 | "\n", 239 | "If you consider a single row of A to be a vector and a single row of B to also be a vector, you can calculate the dot product. \n", 240 | "\n", 241 | "Remember that for matrix multiplication to be valid, A is size m x n while B is size n x p. The number of columns in A must equal the number of rows in B, which makes taking the dot product between a row of A and column of B possible.\n", 242 | "\n", 243 | "As a reminder, the dot product of `` and `` is equal to \n", 244 | "\n", 245 | "`a1*b1 + a2*b2 + a3*b3 + a4*b4`\n" 246 | ] 247 | }, 248 | { 249 | "cell_type": "code", 250 | "execution_count": 9, 251 | "metadata": {}, 252 | "outputs": [], 253 | "source": [ 254 | "### TODO: Write a function called dot_product() that\n", 255 | "### has two vectors as inputs and outputs the dot product of the \n", 256 | "### two vectors. First, you will need to do element-wise\n", 257 | "### multiplication and then sum the results. \n", 258 | "\n", 259 | "### HINT: You wrote this function previously in the vector coding\n", 260 | "### exercises\n", 261 | "\n", 262 | "def dot_product(vector_one, vector_two):\n", 263 | " dot_prod = 0\n", 264 | " for idx in range(len(vector_one)):\n", 265 | " dot_prod += vector_one[idx] * vector_two[idx]\n", 266 | " return dot_prod" 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": 10, 272 | "metadata": {}, 273 | "outputs": [], 274 | "source": [ 275 | "### TODO: Run this cell to test your results\n", 276 | "\n", 277 | "assert dot_product([4, 5, 1], [2, 1, 5]) == 18\n", 278 | "assert dot_product([6], [7]) == 42" 279 | ] 280 | }, 281 | { 282 | "cell_type": "markdown", 283 | "metadata": {}, 284 | "source": [ 285 | "### Matrix Multiplication\n", 286 | "\n", 287 | "Now you will write a function to carry out matrix multiplication\n", 288 | "between two matrices.\n", 289 | "\n", 290 | "If you have an m x n matrix and an n x p matrix, your result will be m x p. \n", 291 | "\n", 292 | "Your strategy could involve looping through an empty n x p matrix and filling in each elements value." 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": 28, 298 | "metadata": {}, 299 | "outputs": [], 300 | "source": [ 301 | "### TODO: Write a function called matrix_multiplication that takes\n", 302 | "### two matrices,multiplies them together and then returns\n", 303 | "### the results\n", 304 | "### \n", 305 | "### Make sure that your function can handle matrices that contain\n", 306 | "### only one row or one column. For example,\n", 307 | "### multiplying two mat------------rices of size (4x1)x(1x4) should return a\n", 308 | "### 4x4 matrix\n", 309 | "\n", 310 | "def matrix_multiplication(matrixA, matrixB):\n", 311 | " \n", 312 | " ### TODO: store the number of rows in A and the number\n", 313 | " ### of columns in B. This will be the size of the output\n", 314 | " ### matrix\n", 315 | " ### HINT: The len function in Python will be helpful\n", 316 | " m_rows = len(matrixA)\n", 317 | " p_columns = len(matrixB[0]) \n", 318 | " \n", 319 | " \n", 320 | " # empty list that will hold the product of AxB\n", 321 | " result = []\n", 322 | " \n", 323 | "\n", 324 | " \n", 325 | " ### TODO: Write a for loop within a for loop. The outside\n", 326 | " ### for loop will iterate through m_rows.\n", 327 | " ### The inside for loop will iterate through p_columns.\n", 328 | " for row in range(m_rows):\n", 329 | " row_result = []\n", 330 | " current_row = get_row(matrixA,row)\n", 331 | " for col in range(p_columns):\n", 332 | " current_col = get_column(matrixB,col)\n", 333 | " dot_prod = dot_product(current_row,current_col)\n", 334 | " row_result.append(dot_prod)\n", 335 | " #print(dot_prod)\n", 336 | " result.append(row_result)\n", 337 | " \n", 338 | " ### TODO: As you iterate through the m_rows and p_columns,\n", 339 | " ### use your get_row function to grab the current A row\n", 340 | " ### and use your get_column function to grab the current\n", 341 | " ### B column.\n", 342 | " \n", 343 | " \n", 344 | " ### TODO: Calculate the dot product of the A row and the B column\n", 345 | " \n", 346 | " \n", 347 | " ### TODO: Append the dot product to an empty list called row_result. \n", 348 | " ### This list will accumulate the values of a row \n", 349 | " ### in the result matrix\n", 350 | " #row_result = []\n", 351 | " \n", 352 | " ### TODO: After iterating through all of the columns in matrix B,\n", 353 | " ### append the row_result list to the result variable. \n", 354 | " ### Reinitialize the row_result to row_result = []. \n", 355 | " ### Your for loop will move down to the next row \n", 356 | " ### of matrix A.\n", 357 | " ### The loop will iterate through all of the columns \n", 358 | " ### taking the dot product\n", 359 | " ### between the row in A and each column in B.\n", 360 | " \n", 361 | " ### TODO: return the result of AxB\n", 362 | " \n", 363 | " \n", 364 | " return result" 365 | ] 366 | }, 367 | { 368 | "cell_type": "code", 369 | "execution_count": 29, 370 | "metadata": {}, 371 | "outputs": [ 372 | { 373 | "name": "stdout", 374 | "output_type": "stream", 375 | "text": [ 376 | "[[25, 5], [10, 2]]\n" 377 | ] 378 | } 379 | ], 380 | "source": [ 381 | "a = matrix_multiplication([[5], [2]], [[5, 1]])\n", 382 | "print(a)" 383 | ] 384 | }, 385 | { 386 | "cell_type": "code", 387 | "execution_count": 30, 388 | "metadata": {}, 389 | "outputs": [], 390 | "source": [ 391 | "### TODO: Run this code cell to test your results\n", 392 | "assert matrix_multiplication([[5], [2]], [[5, 1]]) == [[25, 5], [10, 2]]\n", 393 | "assert matrix_multiplication([[5, 1]], [[5], [2]]) == [[27]]\n", 394 | "assert matrix_multiplication([[4]], [[3]]) == [[12]]\n", 395 | "assert matrix_multiplication([[2, 1, 8, 2, 1], [5, 6, 4, 2, 1]], [[1, 7, 2], [2, 6, 3], [3, 1, 1], [1, 20, 1], [7, 4, 16]]) == [[37, 72, 33], [38, 119, 50]]" 396 | ] 397 | }, 398 | { 399 | "cell_type": "code", 400 | "execution_count": null, 401 | "metadata": {}, 402 | "outputs": [], 403 | "source": [] 404 | }, 405 | { 406 | "cell_type": "code", 407 | "execution_count": null, 408 | "metadata": {}, 409 | "outputs": [], 410 | "source": [] 411 | } 412 | ], 413 | "metadata": { 414 | "kernelspec": { 415 | "display_name": "Python 3", 416 | "language": "python", 417 | "name": "python3" 418 | }, 419 | "language_info": { 420 | "codemirror_mode": { 421 | "name": "ipython", 422 | "version": 3 423 | }, 424 | "file_extension": ".py", 425 | "mimetype": "text/x-python", 426 | "name": "python", 427 | "nbconvert_exporter": "python", 428 | "pygments_lexer": "ipython3", 429 | "version": "3.6.3" 430 | } 431 | }, 432 | "nbformat": 4, 433 | "nbformat_minor": 2 434 | } 435 | -------------------------------------------------------------------------------- /code/Linear_Algebra/guide_to_mathematical_notation.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Becoming \"Wikipedia proficient\"\n", 8 | "\n", 9 | "The goal of this course is **not** for you to memorize how to calculate a dot product or multiply matrices. The goal is for you to be able to do something useful with a wikipedia page like their [article on Kalman Filters](https://en.wikipedia.org/wiki/Kalman_filter), even if requires some additional research and review from you.\n", 10 | "\n", 11 | "But these pages are usually written in the notation of **linear algebra** and not the notation of computer programming. \n", 12 | "\n", 13 | "In this notebook you will learn something about how to navigate the notation of linear algebra and how to translate it into computer code. " 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": {}, 19 | "source": [ 20 | "## Analyzing The Dot Product Equation\n", 21 | "At the time I'm writing this, the wikipedia article on the [dot product](https://en.wikipedia.org/wiki/Dot_product) begins with a section called **Algebraic Definition**, which starts like this:\n", 22 | "\n", 23 | "> The dot product of two vectors $\\mathbf{a} = [a_1, a_2, \\ldots, a_n]$ and $\\mathbf{b} = [b_1, b_2, \\ldots, b_n]$ is defined as: \n", 24 | "> \n", 25 | "> $$\\mathbf{a} \\cdot \\mathbf{b} = \\sum _{i=1}^{n}a_{i}b_{i}=a_{1}b_{1}+a_{2}b_{2}+\\cdots +a_{n}b_{n}$$\n", 26 | "\n", 27 | "If you don't know what to look for, this can be pretty unhelfpul. Let's take a look at three features of this equation which can be helpful to understand...\n", 28 | "\n", 29 | "### Feature 1 - Lowercase vs uppercase variables\n", 30 | "This equation only uses lowercase variables. In general, lowercase variables are used when discussing **vectors** or **scalars** (regular numbers like 3, -2.5, etc...) while UPPERCASE variables are reserved for matrices. \n", 31 | "\n", 32 | "### Feature 2 - Bold vs regular typeface for variables\n", 33 | "A variable in **bold** typeface indicates a vector or a matrix. A variable in regular typeface is a scalar.\n", 34 | "\n", 35 | "\n", 36 | "### Feature 3 - \"...\" in equations\n", 37 | "When you see three dots $\\ldots$ in an equation it means \"this pattern could continue any number of times\"\n", 38 | "\n", 39 | "#### EXAMPLE 1 - APPLYING FEATURES 1, 2, and 3\n", 40 | "When you see something like $\\mathbf{a} = [a_1, a_2, \\ldots, a_n]$ you can infer the following:\n", 41 | "\n", 42 | "1. **$\\mathbf{a}$ is a vector**: since a is bold it's either a vector OR a matrix, but since it's also lowercase, we know it can only be a vector.\n", 43 | "\n", 44 | "2. **$\\mathbf{a}$ can have any length**: since there's a $\\ldots$ in the definition for $\\mathbf{a}$, we know that in addition to $a_1$ and $a_2$ there could also be $a_3$, $a_4$, and so on... \n", 45 | "\n", 46 | "3. **The values in the $\\mathbf{a}$ vector are scalars**: since $a_1$ is lowercase and non-bold we know that it must be a scalar (regular number) as opposed to being a vector or matrix." 47 | ] 48 | }, 49 | { 50 | "cell_type": "markdown", 51 | "metadata": {}, 52 | "source": [ 53 | "### Feature 4 - $\\Sigma$ Notation\n", 54 | "The symbol $\\Sigma$ is the uppercase version of the greek letter \"sigma\" and it is an instruction to perform a sum.\n", 55 | "\n", 56 | "**When you see a $\\Sigma$ you should think \"for loop!\"**\n", 57 | "\n", 58 | "In the case of the dot product, the sigma instructs us to sum $a_ib_i$ for $i=1,2, \\ldots, n$. And in this case $n$ is just the length of the $\\mathbf{a}$ and $\\mathbf{b}$ vectors.\n", 59 | "\n", 60 | "How this for loop works is best explained with an example. Take a look at the `dot_product` function defined below. Try to read through the comments and really understand how the code connects to math.\n" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "**The MATH**\n", 68 | "\n", 69 | "The dot product of two vectors $\\mathbf{a} = [a_1, a_2, \\ldots, a_n]$ and $\\mathbf{b} = [b_1, b_2, \\ldots, b_n]$ is defined as: \n", 70 | "\n", 71 | "$$\\mathbf{a} \\cdot \\mathbf{b} = \\sum _{i=1}^{n}a_{i}b_{i}=a_{1}b_{1}+a_{2}b_{2}+\\cdots +a_{n}b_{n}$$" 72 | ] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "execution_count": 4, 77 | "metadata": { 78 | "collapsed": true 79 | }, 80 | "outputs": [], 81 | "source": [ 82 | "# The CODE\n", 83 | "\n", 84 | "def dot_product(a, b):\n", 85 | " # start by checking that a and b have the same length.\n", 86 | " # I know they SHOULD have the same length because they\n", 87 | " # each are DEFINED (in the first line above) to have n \n", 88 | " # elements. Even though n isn't specified, the fact that \n", 89 | " # a goes from 0 to n AND b does the same (instead of going\n", 90 | " # from 0 to m for example) implies that these vectors \n", 91 | " # always should have the same length.\n", 92 | " if len(a) != len(b):\n", 93 | " print(\"Error! Vectors must have the same length!\")\n", 94 | " return None\n", 95 | " \n", 96 | " # let's call the length of these vectors \"n\" so we can\n", 97 | " # be consistent with the mathematical notation\n", 98 | " n = len(a)\n", 99 | " \n", 100 | " # Since we want to add up a bunch of terms, we should\n", 101 | " # start by setting the total to zero and then add to \n", 102 | " # this total n times.\n", 103 | " total = 0\n", 104 | " \n", 105 | " # now we are going to perform the multiplication!\n", 106 | " # note that the algebraic version goes from 1 to n. \n", 107 | " # The Python version of this indexing will go from \n", 108 | " # 0 to n-1 (recall that range(3) returns [0,1,2] for example).\n", 109 | " for i in range(n): \n", 110 | " a_i = a[i]\n", 111 | " b_i = b[i]\n", 112 | " total = total + a_i * b_i\n", 113 | " \n", 114 | " return total\n", 115 | "\n" 116 | ] 117 | }, 118 | { 119 | "cell_type": "code", 120 | "execution_count": 3, 121 | "metadata": {}, 122 | "outputs": [ 123 | { 124 | "name": "stdout", 125 | "output_type": "stream", 126 | "text": [ 127 | "52\n" 128 | ] 129 | } 130 | ], 131 | "source": [ 132 | "# let's see if it works\n", 133 | "a = [3,2,4]\n", 134 | "b = [2,5,9]\n", 135 | "\n", 136 | "# a*b should be 3*2 + 2*5 + 4*9\n", 137 | "# or... 6 + 10 + 36 \n", 138 | "# 52\n", 139 | "\n", 140 | "a_dot_b = dot_product(a,b)\n", 141 | "print(a_dot_b)" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": null, 147 | "metadata": { 148 | "collapsed": true 149 | }, 150 | "outputs": [], 151 | "source": [] 152 | } 153 | ], 154 | "metadata": { 155 | "kernelspec": { 156 | "display_name": "Python 3", 157 | "language": "python", 158 | "name": "python3" 159 | }, 160 | "language_info": { 161 | "codemirror_mode": { 162 | "name": "ipython", 163 | "version": 3 164 | }, 165 | "file_extension": ".py", 166 | "mimetype": "text/x-python", 167 | "name": "python", 168 | "nbconvert_exporter": "python", 169 | "pygments_lexer": "ipython3", 170 | "version": "3.6.3" 171 | } 172 | }, 173 | "nbformat": 4, 174 | "nbformat_minor": 2 175 | } 176 | -------------------------------------------------------------------------------- /code/Robot_Localization.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Sense and Move\n", 8 | "\n", 9 | "In this notebook, let's put all of what we've learned together and see what happens to an initial probability distribution as a robot goes trough cycles of sensing then moving then sensing then moving, and so on! Recall that each time a robot senses (in this case a red or green color)it gains information about its environment, and everytime it moves, it loses some information due to motion uncertainty.\n", 10 | "\n", 11 | "\n", 12 | "\n" 13 | ] 14 | }, 15 | { 16 | "cell_type": "markdown", 17 | "metadata": {}, 18 | "source": [ 19 | "First let's include our usual resource imports and display function." 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": 1, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "# importing resources\n", 29 | "import matplotlib.pyplot as plt\n", 30 | "import numpy as np" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "A helper function for visualizing a distribution." 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": 2, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "def display_map(grid, bar_width=1):\n", 47 | " if(len(grid) > 0):\n", 48 | " x_labels = range(len(grid))\n", 49 | " plt.bar(x_labels, height=grid, width=bar_width, color='b')\n", 50 | " plt.xlabel('Grid Cell')\n", 51 | " plt.ylabel('Probability')\n", 52 | " plt.ylim(0, 1) # range of 0-1 for probability values \n", 53 | " plt.title('Probability of the robot being at each cell in the grid')\n", 54 | " plt.xticks(np.arange(min(x_labels), max(x_labels)+1, 1))\n", 55 | " plt.show()\n", 56 | " else:\n", 57 | " print('Grid is empty')\n" 58 | ] 59 | }, 60 | { 61 | "cell_type": "markdown", 62 | "metadata": { 63 | "collapsed": true 64 | }, 65 | "source": [ 66 | "### QUIZ: Given the list motions=[1,1], compute the posterior distribution if the robot first senses red, then moves right one, then senses green, then moves right again, starting with a uniform prior distribution, `p`.\n", 67 | "\n", 68 | "`motions=[1,1]` mean that the robot moves right one cell and then right again. You are given the initial variables and the complete `sense` and `move` function, below." 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": 3, 74 | "metadata": {}, 75 | "outputs": [], 76 | "source": [ 77 | "# given initial variables\n", 78 | "p=[0.2, 0.2, 0.2, 0.2, 0.2]\n", 79 | "# the color of each grid cell in the 1D world\n", 80 | "world=['green', 'red', 'red', 'green', 'green']\n", 81 | "# Z, the sensor reading ('red' or 'green')\n", 82 | "measurements = ['red', 'green']\n", 83 | "pHit = 0.6\n", 84 | "pMiss = 0.2\n", 85 | "\n", 86 | "motions = [1,1]\n", 87 | "pExact = 0.8\n", 88 | "pOvershoot = 0.1\n", 89 | "pUndershoot = 0.1\n", 90 | "\n", 91 | "# You are given the complete sense function\n", 92 | "def sense(p, Z):\n", 93 | " ''' Takes in a current probability distribution, p, and a sensor reading, Z.\n", 94 | " Returns a *normalized* distribution after the sensor measurement has been made, q.\n", 95 | " This should be accurate whether Z is 'red' or 'green'. '''\n", 96 | " q=[]\n", 97 | " # loop through all grid cells\n", 98 | " for i in range(len(p)):\n", 99 | " # check if the sensor reading is equal to the color of the grid cell\n", 100 | " # if so, hit = 1\n", 101 | " # if not, hit = 0\n", 102 | " hit = (Z == world[i])\n", 103 | " q.append(p[i] * (hit * pHit + (1-hit) * pMiss))\n", 104 | " \n", 105 | " # sum up all the components\n", 106 | " s = sum(q)\n", 107 | " # divide all elements of q by the sum to normalize\n", 108 | " for i in range(len(p)):\n", 109 | " q[i] = q[i] / s\n", 110 | " return q\n", 111 | "\n", 112 | "\n", 113 | "# The complete move function\n", 114 | "def move(p, U):\n", 115 | " q=[]\n", 116 | " # iterate through all values in p\n", 117 | " for i in range(len(p)):\n", 118 | " # use the modulo operator to find the new location for a p value\n", 119 | " # this finds an index that is shifted by the correct amount\n", 120 | " index = (i-U) % len(p)\n", 121 | " nextIndex = (index+1) % len(p)\n", 122 | " prevIndex = (index-1) % len(p)\n", 123 | " s = pExact * p[index]\n", 124 | " s = s + pOvershoot * p[nextIndex]\n", 125 | " s = s + pUndershoot * p[prevIndex]\n", 126 | " # append the correct, modified value of p to q\n", 127 | " q.append(s)\n", 128 | " return q\n", 129 | "\n", 130 | "\n", 131 | "## TODO: Compute the posterior distribution if the robot first senses red, then moves \n", 132 | "## right one, then senses green, then moves right again, starting with a uniform prior distribution.\n", 133 | "\n", 134 | "## print/display that distribution" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": 4, 140 | "metadata": {}, 141 | "outputs": [ 142 | { 143 | "name": "stdout", 144 | "output_type": "stream", 145 | "text": [ 146 | "[0.21157894736842112, 0.1515789473684211, 0.08105263157894739, 0.16842105263157897, 0.3873684210526316]\n" 147 | ] 148 | }, 149 | { 150 | "data": { 151 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAG5ZJREFUeJzt3XuYJHV97/H3hwVcboEIexJhl4sBgyvxuiLRRImSE0ABk2gENYoSOcdI1KPGS6KoqNFcNF5CYogXCEaQaGI2HjyEJKBHI8Iiii7IcQWEdRFWFAGJIvo9f1RNpRm6Z3p2p6Zh5v16nnmmq+pX1d+qru5P16+6q1NVSJIEsM2kC5Ak3XsYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEwpiRvSPKhLZz3+CSfmWH6J5M8d1jbJLcneeCW3O8ca9whyT8n+V6Svx9znguT/E7ftY2479OTvLmnZe+bpJJsuwXz7t0+Zsv6qO3eZmu21Rzvp9vXZno+zff235rn/Tzd/8jn/2yvK1tqUYdCkmuT/Ge7YW9M8sEkO0+6rumq6oiqOmPEtJ2r6mro94UQeBrwM8DuVfX06RMn/eSYT32GWVVd1z5mP+5j+aMs1Ivzvd3WbP8khybZ2EddW2rw+b9QFnUotI6qqp2BRwKPBl47vUEaS2FbzGQf4P9V1V2TLmSpv7BJk3wOLJkXwqr6JvBJ4CDo3i2+JclngTuABybZM8naJN9JsiHJC6YtZnmSjyS5LckXkjxsakKSVyf5ejvtiiS/Pm3eJHlP2z3z1SRPGpgw8p1r++5v/yQnAs8CXtke+fxzkt9P8rFp7d+T5J0jlvXg9r5uSbI+ydHt+DcCJwPPaJd9wrT5Dgf+YGD6lwYm75Pks+16/0uSPQbmOyTJf7T396Ukhw6rq217bZJXJbkc+H6SbUfVO2CPJOe39/2pJPsMLO+xSS5pt/clSR7bjn8L8MvAX7Tr8hejagKen2RTkhuSvHxg2dsMPN43Jzknyf3baXd7x97W/6YZttFzknyjXc7r2u1w2Iht9OQklyW5Ncn1Sd4wMPnT7f9b2vX6xSHzj6y7nf73Sb7VbrNPJ3nIwLQdkry9rfV7ST6TZIeBxT8ryXVJvp3kD0dt0JmWM5f9ZYblz2n7D8y3E83rw57t9rs9yZ7t5O2T/G07//okawbm2zPJx5JsTnJNkhfPUNvuaZ63t7b75Jtz967iSvKiJF8DvjYwbv+B+de2818M/Nxct89YqmrR/gHXAoe1t1cB64E3tcMXAtcBDwG2BbYDPgX8JbAceDiwGXhS2/4NwI9oulm2A14BXANs105/OrAnTdA+A/g+8IB22vHAXcD/aud9BvA94P4DtfzOQNvPDKxDAfu3t08H3jww7QHt/ezWDm8L3AQ8asi22A7YQPPivj3wROA24OcH1u9DM2zLe0xv6/468CBgh3b4be20vYCbgSPbbfKr7fCKGR6rL7aP0w5j1Ht6O/x44H7Au6a2G3B/4LvAb7fb5Lh2ePfp23tELfu22/0sYCfgF9p9YWpfeilwEbCyve+/Bs6aNu+2Y2yj1cDtwC+16/hnNPvYYSPqOrStZRvgocCNwFOH3e+I+UfW3U5/PrBLO+2dwBcHpp3a1r4XsAx4bNtu6n7/pl2/hwE/BB48ooZRy5lxf2GG58iIx27W7T9i+24cst//oK1rGfBW4KJ22jbApTRvqLYHHghcDfzaiOWf3f7t2D7213PP5/r5NPvvDkOe/2cD59DskwcB3xy1HbbqdXO+F3hv+qN5obkduAX4Bs0L/tTGvhA4ZaDtKuDHwC4D494KnD6wc1w0MG0b4Abgl0fc9xeBYwZ24k1ABqZfDPz2bDs8M4RCO+6TwAva208BrhhRzy8D3wK2GRh3FvCGgfXbklB47cDw7wL/p739KuDMae3PA547w2P1/DnUezpw9sC0ndvHbxVNGFw8bfmfA46fvr1H1LJvu90PHBj3J8D729tX0r5ZaIcfQPNivi3DX5RGbaOTufuL8o7AnYwIhSF1vhP482k1zxQKI+se0na3dnm70uzr/wk8bIZttXLavn3skLYzLWfG/YWtC4Wh23/IvIcyPBT+dWB4NfCf7e3HANdNa/8a4INDlr2s3dY/PzDuzdzzuf7EafMVsP/A/IP75B+N2g5b87cU+m6fWlX/OmLa9QO39wS+U1W3DYz7BrBmWPuq+kmak1J7QtMNALyMZqeE5kVq8DD1m9U+kgPL3pOtdwbwQpp3as8GzhzRbk/g+qr6ybQa9trK+//WwO07aNYbmnMUT09y1MD07YALZljW9MdjtnoHH4/bk3ynnW/Pti0zzDuOwXq+QfMuHZp1+8ckg7X9mOZE/TCjttGe3H0d7khy86hikjwGeBvNu8Ttad5hj/VJsdnqTvIt4C00R7wrgKk2e7T3s5zmHfcoo9Zx0B4zLGdL9pdxjVPbXOZf3nZP7UPT3XTLwPRlwP8dsowVNG8aBvep64e0GzZu1PzT9/F5sWTOKYww+CK9Cbh/kl0Gxu1Nc4g2ZdXUjTQnplcCm9q+7L8BTqLpotgN+AqQgXn3SjI4vHd7n1ta75SPAw9NchDNkcLfjZh3E7Aqdz+hPn395nrfM7me5p3fbgN/O1XV28a8j3HqHXw8dqY57N7U/u0zbdmD8467LqsGbg8+XtcDR0xbt+XVnLeaixto9qGpddgB2H2G9h8G1gKrqmpX4L381z42zjrNVPczgWOAw2iODvadKgv4Nk0Xytb2Yc+0nC3ZX+bbluzj10yreZeqOnJI2800XcgrB8atGtJuVA1T80/fJ+fdUg+FTlVdD/wH8NYky5M8FDiBu7/IPirJb7TvEl5K03d6EU0fX9E8cCR5Hu0J7QH/DXhxku2SPB14MHDuHMu8kabfcrDuHwAfpXnBuLiqrhsx7+dpzj+8sq3hUOAomn7Kce9734z/Ka0PAUcl+bUky9ptemiSlbPOOX69Ryb5pSTbA28CPt8+jucCD0ryzDQnrJ9Bc9j/iYF1Gee7H69LsmN7wvV5wEfa8e8F3tK+GSDJiiTHjLlegz5Ks40e267DG7n7G4npdqE5mv1BkoNpXsinbKZ5dz/Tes1U9y40+/PNNN1YfzQ1U3u09gHgHe2J1WVJfjHJ/eaysrMsZ2v3l/lwI7B7kl3HbH8xcGuaD0js0NZ9UJJHT29YzUdk/wF4Q7tPHQg8Z9zChsy/GnjuuPPPhaFwd8fRvEPaBPwj8PqqOn9g+j/RnCSeOon5G1X1o6q6Ang7Tb/1jTTdDJ+dtuzPAwfQvFt6C/C0qhrZVTDC+4HV7aczPj4w/oz2Pkd1HVFVdwJHA0e0Nfwl8Jyq+uqY9z3VTXFzki/M1rh9cT6G5kTxZpp3Vb/PmPvcmPV+GHg98B3gUTSfzqLdrk8BXk7zIvdK4ClV9e12vncBT0vy3STvnqGMT9Gc7P434M+q6l8G5l8L/EuS22jeGDxmnPWato7rgd+jCbobaE6c30Tz4jzM7wKntPd5Ms1Jx6ll3UGzX3223T8OGTL/THX/LU13xDeBK9ppg14BfBm4hGZ7/zFb9voxdDlbu7/Mh3bfOgu4ut2GM3bvti/UR9F8KOUamv30fTRHWsOc1E77Fs1z9SxGP9aj5t+5nf904INzmHdsuXs3t+6LkuwNfBX42aq6ddL1aMu0XWC3AAdU1TWTrkf9SvLHNM/ZXt7xbymPFO7j2u6cl9F8EsdAuI9JclTbHbATzUdSv0zzSSwtMkkOTPLQNA6m6Z7+x0nXNV1voZDkA0luSvKVEdOT5N1pviR2eZJH9lXLYtW+kNxK85nu10+4HG2ZY/ivk+MH0HyU08P3xWkXmvMC36fp+ns7TZf0vUpv3UdJHk/zHYG/rarpJ11JciRNf+qRNP2a76qqOffLSpLmT29HClX1aZoTSaMcQxMYVVUXAbsleUBf9UiSZjfJL6/txd2/iLGxHXfD9IZprvtzIsBOO+30qAMPPHBBCpSkxeLSSy/9dlWtmK3dJENh2Oexh/ZlVdVpwGkAa9asqXXr1vVZlyQtOknG+gb0JD99tJG7fztvJXP/hq8kaR5NMhTWAs9pP4V0CPC9qrpH15EkaeH01n2U5Cyaqw7u0V447vU0F7iiqt5LcymCI2m+MXoHzWUEJEkT1FsoVNVxs0wv4EV93b8kae78RrMkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqWMoSJI6hoIkqdNrKCQ5PMlVSTYkefWQ6XsnuSDJZUkuT3Jkn/VIkmbWWygkWQacChwBrAaOS7J6WrPXAudU1SOAY4G/7KseSdLs+jxSOBjYUFVXV9WdwNnAMdPaFPBT7e1dgU091iNJmkWfobAXcP3A8MZ23KA3AM9OshE4F/i9YQtKcmKSdUnWbd68uY9aJUn0GwoZMq6mDR8HnF5VK4EjgTOT3KOmqjqtqtZU1ZoVK1b0UKokCfoNhY3AqoHhldyze+gE4ByAqvocsBzYo8eaJEkz6DMULgEOSLJfku1pTiSvndbmOuBJAEkeTBMK9g9J0oT0FgpVdRdwEnAecCXNp4zWJzklydFts5cDL0jyJeAs4Piqmt7FJElaINv2ufCqOpfmBPLguJMHbl8BPK7PGiRJ4/MbzZKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSeoYCpKkjqEgSer0GgpJDk9yVZINSV49os1vJbkiyfokH+6zHknSzMYKhSQfS/LkJGOHSJJlwKnAEcBq4Lgkq6e1OQB4DfC4qnoI8NKxK5ckzbtxX+T/Cngm8LUkb0ty4BjzHAxsqKqrq+pO4GzgmGltXgCcWlXfBaiqm8asR5LUg7FCoar+taqeBTwSuBY4P8l/JHleku1GzLYXcP3A8MZ23KAHAQ9K8tkkFyU5fNiCkpyYZF2SdZs3bx6nZEnSFphLd9DuwPHA7wCXAe+iCYnzR80yZFxNG94WOAA4FDgOeF+S3e4xU9VpVbWmqtasWLFi3JIlSXO07TiNkvwDcCBwJnBUVd3QTvpIknUjZtsIrBoYXglsGtLmoqr6EXBNkqtoQuKSMeuXJM2jcY8U3ldVq6vqrVOBkOR+AFW1ZsQ8lwAHJNkvyfbAscDaaW0+DvxKu7w9aLqTrp7jOkiS5sm4ofDmIeM+N9MMVXUXcBJwHnAlcE5VrU9ySpKj22bnATcnuQK4APj9qrp5zJokSfNsxu6jJD9Lc3J4hySP4L/OE/wUsONsC6+qc4Fzp407eeB2AS9r/yRJEzbbOYVfozm5vBJ4x8D424A/6KkmSdKEzBgKVXUGcEaS36yqjy1QTZKkCZmt++jZVfUhYN8k9+jiqap3DJlNknQfNVv30U7t/537LkSSNHmzdR/9dfv/jQtTjiRpkmbrPnr3TNOr6sXzW44kaZJm6z66dEGqkCTdK4zz6SNJ0hIxW/fRO6vqpUn+mXtezI6qOnrIbJKk+6jZuo/ObP//Wd+FSJImb7buo0vb/59qL2p3IM0Rw1XtD+dIkhaRcS+d/WTgvcDXaa5/tF+S/1FVn+yzOEnSwhorFIC3A79SVRsAkvwc8L8BQ0GSFpFxL51901QgtK4G/D1lSVpkZvv00W+0N9cnORc4h+acwtPx19EkadGZrfvoqIHbNwJPaG9vBn66l4okSRMz26ePnrdQhUiSJm/cTx8tB04AHgIsnxpfVc/vqS5J0gSMe6L5TOBnaX6J7VM0v8R2W19FSZImY9xQ2L+qXgd8v70e0pOBX+ivLEnSJIwbCj9q/9+S5CBgV2DfXiqSJE3MuF9eOy3JTwOvA9bS/BLb63qrSpI0EWOFQlW9r735KeCB/ZUjSZqksbqPkuye5D1JvpDk0iTvTLJ738VJkhbWuOcUzqa5rMVvAk8Dvg18pK+iJEmTMe45hftX1ZsGht+c5Kl9FCRJmpxxjxQuSHJskm3av9+iuUqqJGkRme2CeLfRXAAvwMuAD7WTtgFuB17fa3WSpAU127WPdlmoQiRJkzfuOQWSHA08vh28sKo+0U9JkqRJGfcjqW8DXgJc0f69pB0nSVpExj1SOBJ4eFX9BCDJGcBlwKv7KkyStPDG/fQRwG4Dt3ed70IkSZM37pHCW4HLklxA80mkxwOv6a0qSdJEzBoKSQJ8BjgEeDRNKLyqqr7Vc22SpAU2a/dRVRXw8aq6oarWVtU/jRsISQ5PclWSDUlGnn9I8rQklWTNHGqXJM2zcc8pXJTk0XNZcJJlwKnAEcBq4Lgkq4e02wV4MfD5uSxfkjT/xg2FX6EJhq8nuTzJl5NcPss8BwMbqurqqrqT5qJ6xwxp9ybgT4AfjF21JKkX455oPmILlr0XcP3A8EbgMYMNkjwCWFVVn0jyilELSnIicCLA3nvvvQWlSJLGMdu1j5YD/xPYH/gy8P6qumvMZWfIuBpY9jbAnwPHz7agqjoNOA1gzZo1NUtzSdIWmq376AxgDU0gHAG8fQ7L3gisGhheCWwaGN4FOAi4MMm1NJ9uWuvJZkmanNm6j1ZX1S8AJHk/cPEcln0JcECS/YBvAscCz5yaWFXfA/aYGk5yIfCKqlo3h/uQtERlWF/EIlcL0E8y25HCj6ZuzKHbaLD9ScB5wJXAOVW1Pskp7cX1JEn3MrMdKTwsya3t7QA7tMOh+QrDT800c1WdC5w7bdzJI9oeOlbFkqTezPZ7CssWqhBJ0uTN5YJ4kqRFzlCQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSp9dQSHJ4kquSbEjy6iHTX5bkiiSXJ/m3JPv0WY8kaWbb9rXgJMuAU4FfBTYClyRZW1VXDDS7DFhTVXckeSHwJ8Az+qupryXfe1VNugJJ9yV9HikcDGyoqqur6k7gbOCYwQZVdUFV3dEOXgSs7LEeSdIs+gyFvYDrB4Y3tuNGOQH45LAJSU5Msi7Jus2bN89jiZKkQX2GwrDOmqGdGUmeDawB/nTY9Ko6rarWVNWaFStWzGOJkqRBvZ1ToDkyWDUwvBLYNL1RksOAPwSeUFU/7LEeSdIs+jxSuAQ4IMl+SbYHjgXWDjZI8gjgr4Gjq+qmHmuRJI2ht1CoqruAk4DzgCuBc6pqfZJTkhzdNvtTYGfg75N8McnaEYuTJC2APruPqKpzgXOnjTt54PZhfd6/JGlu/EazJKnT65GCpIWxFL+YqX54pCBJ6hgKkqSOoSBJ6nhOYZFbin3NXgRQ2nIeKUiSOoaCJKljKEiSOoaCJKljKEiSOoaCJKljKEiSOoaCJKljKEiSOoaCJKljKEiSOoaCJKljKEiSOoaCJKnjpbO16CzFy4VL88UjBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBUlSx1CQJHV6DYUkhye5KsmGJK8eMv1+ST7STv98kn37rEeSNLPeQiHJMuBU4AhgNXBcktXTmp0AfLeq9gf+HPjjvuqRJM2uzyOFg4ENVXV1Vd0JnA0cM63NMcAZ7e2PAk9K/IkUSZqUPn95bS/g+oHhjcBjRrWpqruSfA/YHfj2YKMkJwIntoO3J7mql4r7tQfT1msJWGrrvNTWF1znBbWVb5n3GadRn6EwrPzagjZU1WnAafNR1KQkWVdVayZdx0Jaauu81NYXXOfFqM/uo43AqoHhlcCmUW2SbAvsCnynx5okSTPoMxQuAQ5Isl+S7YFjgbXT2qwFntvefhrw71V1jyMFSdLC6K37qD1HcBJwHrAM+EBVrU9yCrCuqtYC7wfOTLKB5gjh2L7quRe4T3d/baGlts5LbX3BdV504htzSdIUv9EsSeoYCpKkjqHQs9ku9bEYJflAkpuSfGXStSyEJKuSXJDkyiTrk7xk0jX1LcnyJBcn+VK7zm+cdE0LJcmyJJcl+cSka+mDodCjMS/1sRidDhw+6SIW0F3Ay6vqwcAhwIuWwOP8Q+CJVfUw4OHA4UkOmXBNC+UlwJWTLqIvhkK/xrnUx6JTVZ9mCX3fpKpuqKovtLdvo3nB2GuyVfWrGre3g9u1f4v+UytJVgJPBt436Vr6Yij0a9ilPhb1i8VS117p9xHA5ydbSf/abpQvAjcB51fVol9n4J3AK4GfTLqQvhgK/RrrMh5aHJLsDHwMeGlV3TrpevpWVT+uqofTXK3g4CQHTbqmPiV5CnBTVV066Vr6ZCj0a5xLfWgRSLIdTSD8XVX9w6TrWUhVdQtwIYv/PNLjgKOTXEvTFfzEJB+abEnzz1Do1ziX+tB9XHu59/cDV1bVOyZdz0JIsiLJbu3tHYDDgK9Otqp+VdVrqmplVe1L81z+96p69oTLmneGQo+q6i5g6lIfVwLnVNX6yVbVvyRnAZ8Dfj7JxiQnTLqmnj0O+G2ad45fbP+OnHRRPXsAcEGSy2ne/JxfVYvyI5pLjZe5kCR1PFKQJHUMBUlSx1CQJHUMBUlSx1CQJHUMBS1ZSX4myYeTXJ3k0iSfS/LrI9rumeSjI6ZdmOQeP+SeZLskb0vytSRfaa8qesQsNXXLSnJtkj22ZN2kLdXbz3FK92btF84+DpxRVc9sx+0DHD2k7bZVtYnmd8Tn4k00n+c/qKp+mORngCdsXeVSvwwFLVVPBO6sqvdOjaiqbwDvAUhyPM3VMJcDOyV5PvCJqjqo/QbvB2kuh34lsMP0hSfZEXgBsF9V/bBd/o3AOe30/w68Ebgf8HXgeQNXHZUmxu4jLVUPAb4wS5tfBJ5bVU+cNv6FwB1V9VDgLcCjhsy7P3DdsAvjtV1CrwUOq6pHAuuAl82xfqkXHilIQJJTgV+iOXp4dDv6/Koa9rsQjwfeDVBVl7eXepiLQ2iOMj7b9GKxPc1lQaSJMxS0VK0HfnNqoKpe1L6DXzfQ5vszzD/b9WE2AHsn2aX94Z1BoQmc4+ZSsLQQ7D7SUvXvwPIkLxwYt+OY834aeBZA+xsCD53eoKruoLly6rvbK+SS5AFJng1cBDwuyf7t+B2TPGiL10SaR4aClqRqrgT5VOAJSa5JcjFwBvCqMWb/K2DnttvolcDFI9q9FtgMXJHkKzSfdtpcVZuB44Gz2mVcBBy4NesjzRevkipJ6nikIEnqGAqSpI6hIEnqGAqSpI6hIEnqGAqSpI6hIEnq/H+LHmRV7hoEnwAAAABJRU5ErkJggg==\n", 152 | "text/plain": [ 153 | "" 154 | ] 155 | }, 156 | "metadata": { 157 | "needs_background": "light" 158 | }, 159 | "output_type": "display_data" 160 | } 161 | ], 162 | "source": [ 163 | "for idx in range(len(motions)):\n", 164 | " p = sense(p,measurements[idx])\n", 165 | " p = move(p,motions[idx])\n", 166 | "print(p)\n", 167 | "display_map(p)" 168 | ] 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "metadata": {}, 173 | "source": [ 174 | "### Clarification about Entropy\n", 175 | "\n", 176 | "The video mentions that entropy will go down after the update step and that entropy will go up after the measurement step. \n", 177 | "\n", 178 | "In general, **entropy measures the amount of uncertainty**. Since the update step increases uncertainty, then entropy should increase. The measurement step decreases uncertainty, so entropy should decrease.\n", 179 | "\n", 180 | "Let's look at our current example where the robot could be at five different positions. The maximum uncertainty occurs when all positions have equal probabilities $[0.2, 0.2, 0.2, 0.2, 0.2]$ \n", 181 | "\n", 182 | "Following the formula $$\\text{Entropy} = \\Sigma (-p \\times log(p))$$we get $$-5 \\times (.2)\\times log(0.2) = 0.699$$\n", 183 | "\n", 184 | "Taking a measurement should decrease uncertainty and thus decrease entropy. Let's say after taking a measurement, the probabilities become [0.05, 0.05, 0.05, 0.8, 0.05]. Now the entropy decreased to 0.338. Hence a measurement step should decrease entropy whereas an update step should increase entropy." 185 | ] 186 | } 187 | ], 188 | "metadata": { 189 | "kernelspec": { 190 | "display_name": "Python 3", 191 | "language": "python", 192 | "name": "python3" 193 | }, 194 | "language_info": { 195 | "codemirror_mode": { 196 | "name": "ipython", 197 | "version": 3 198 | }, 199 | "file_extension": ".py", 200 | "mimetype": "text/x-python", 201 | "name": "python", 202 | "nbconvert_exporter": "python", 203 | "pygments_lexer": "ipython3", 204 | "version": "3.6.3" 205 | } 206 | }, 207 | "nbformat": 4, 208 | "nbformat_minor": 2 209 | } 210 | -------------------------------------------------------------------------------- /code/cv/models.py: -------------------------------------------------------------------------------- 1 | ## TODO: define the convolutional neural network architecture 2 | 3 | import torch 4 | import torch.nn as nn 5 | import torch.nn.functional as F 6 | # can use the below import should you choose to initialize the weights of your Net 7 | import torch.nn.init as I 8 | 9 | 10 | class Net(nn.Module): 11 | 12 | def __init__(self): 13 | super(Net, self).__init__() 14 | 15 | ## TODO: Define all the layers of this CNN, the only requirements are: 16 | ## 1. This network takes in a square (same width and height), grayscale image as input 17 | ## 2. It ends with a linear layer that represents the keypoints 18 | ## it's suggested that you make this last layer output 136 values, 2 for each of the 68 keypoint (x, y) pairs 19 | 20 | # As an example, you've been given a convolutional layer, which you may (but don't have to) change: 21 | # 1 input image channel (grayscale), 32 output channels/feature maps, 5x5 square convolution kernel 22 | 23 | #(96x96x1) 24 | self.conv1 = nn.Conv2d(1, 32, 4) 25 | #(93x93x32) 26 | self.pool1 = nn.MaxPool2d(2) 27 | #(46x46x32) 28 | self.dropout1 = nn.Dropout(p=0.1) 29 | 30 | #(46x46x32) 31 | self.conv2 = nn.Conv2d(32, 64, 3) 32 | #(44x44x64) 33 | self.pool2 = nn.MaxPool2d(2) 34 | #(22x22x64) 35 | self.dropout2 = nn.Dropout(p=0.2) 36 | 37 | #(22x22x64) 38 | self.conv3 = nn.Conv2d(64, 128, 2) 39 | #(21x21x128) 40 | self.pool3 = nn.MaxPool2d(2) 41 | #(10x10x128) 42 | self.dropout3 = nn.Dropout(p=0.3) 43 | 44 | #(10x10x128) 45 | self.conv4 = nn.Conv2d(128, 256, 1) 46 | #(10x10x256) 47 | self.pool4 = nn.MaxPool2d(2) 48 | #(5x5x256) 49 | self.dropout4 = nn.Dropout(p=0.4) 50 | 51 | self.fc1 = nn.Linear(6400,1000) 52 | self.dropout5 = nn.Dropout(p=0.5) 53 | 54 | self.fc2 = nn.Linear(1000,1000) 55 | self.dropout6 = nn.Dropout(p=0.6) 56 | 57 | self.output = nn.Linear(1000,136) 58 | 59 | ## Note that among the layers to add, consider including: 60 | # maxpooling layers, multiple conv layers, fully-connected layers, and other layers (such as dropout or batch normalization) to avoid overfitting 61 | 62 | 63 | 64 | def forward(self, x): 65 | ## TODO: Define the feedforward behavior of this model 66 | ## x is the input image and, as an example, here you may choose to include a pool/conv step: 67 | ## x = self.pool(F.relu(self.conv1(x))) 68 | 69 | x = self.conv1(x) 70 | x = F.elu(x) 71 | x = self.pool1(x) 72 | x = self.dropout1(x) 73 | 74 | x = self.conv2(x) 75 | x = F.elu(x) 76 | x = self.pool2(x) 77 | x = self.dropout2(x) 78 | 79 | x = self.conv3(x) 80 | x = F.elu(x) 81 | x = self.pool3(x) 82 | x = self.dropout3(x) 83 | 84 | x = self.conv4(x) 85 | x = F.elu(x) 86 | x = self.pool4(x) 87 | x = self.dropout4(x) 88 | 89 | #(6400) 90 | x = x.view(x.size(0), -1) 91 | 92 | x = self.fc1(x) 93 | x = F.relu6(x) 94 | x = self.dropout5(x) 95 | 96 | x = self.fc2(x) 97 | x = F.relu6(x) 98 | x = self.dropout6(x) 99 | 100 | x = self.output(x) 101 | 102 | # a modified x, having gone through all the layers of your model, should be returned 103 | return x 104 | -------------------------------------------------------------------------------- /code/data_exploration/datasets/2018_POBLACIO_NACIONALITAT.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/code/data_exploration/datasets/2018_POBLACIO_NACIONALITAT.csv -------------------------------------------------------------------------------- /code/knn-lodash.js: -------------------------------------------------------------------------------- 1 | const outputs = [ 2 | [ 10, .5, 16, 1], 3 | [200, .5, 16, 4], 4 | [350, .5, 16, 4], 5 | [600, .5, 16 , 5] 6 | ]; 7 | 8 | const k = 3; 9 | 10 | function distance(pointA, pointB) { 11 | return Math.abs(pointA - pointB); 12 | } 13 | 14 | function getTrainTestSets(data,testSetSize){ 15 | const shuffled = _.shuffle(data); 16 | 17 | const testSet = _.slice(shuffled,0,testSetSize); 18 | const trainingSet = _slice(shuffled,testCount); 19 | 20 | return [testSet, trainingSet]; 21 | } 22 | 23 | function knn(data, point){ 24 | return _.chain(data) 25 | .map(row => [distance(row[0],point),row[3]]) 26 | .sortBy(row => row[0]) 27 | .slice(0,k) 28 | .countBy(row => row[1] 29 | .toPairs() 30 | .sortBy(row => row[1] 31 | .last() 32 | .first() 33 | .parseInt(), 34 | .value(); 35 | } 36 | 37 | function getAccuracy(trainingSet, testSet){ 38 | let numberCorrect = 0; 39 | const testSetSize = testSet.length; 40 | for(let i = 0; i < testSetSize; i++){ 41 | const prediction = knn(trainingSet, testSet[i][0]); 42 | if(prediction === testSet[i][3]){ 43 | numberCorrect++; 44 | } 45 | } 46 | return numberCorrect / testSetSize; 47 | } -------------------------------------------------------------------------------- /code/load_volumetric_data.py: -------------------------------------------------------------------------------- 1 | # Let’s load a sample CT scan using the volread function in the imageio module, which takes a directory 2 | # as argument and assembles all 'DICOM' (Digital Imaging Communication and Storage) files in a series in 3 | # a NumPy 3D array. 4 | 5 | import imageio 6 | 7 | vol_arr = imageio.volread(dirname, 'DICOM') 8 | print(vol_arr.shape) 9 | 10 | # (256, 256, 50) 11 | 12 | # Also in this case, the layout is different from what PyTorch expects. imageio outputs a W x H x D 13 | # array, with no channel information. So we’ll first have to transpose and then make room for the 14 | # channel dimension using unsqueeze: 15 | 16 | vol = torch.from_numpy(col_arr).float() 17 | vol = torch.transpose(vol, 0, 2) 18 | vol = torch.unsqueeze(vol, 0) 19 | 20 | vol.shape 21 | 22 | torch.Size([1, 1, 50, 256, 256]) 23 | 24 | # At this point we could assemble a 5D dataset by stacking multiple volumes along the batch direction 25 | -------------------------------------------------------------------------------- /code/ml-sklearn.py: -------------------------------------------------------------------------------- 1 | from sklearn import model_selection 2 | import pandas as pd 3 | 4 | rent_prices_evolution_2018 = ['datasets/2018_lloguer_preu_trim.csv', ',', 'utf-8'] 5 | 6 | nationalities = ['datasets/2018_POBLACIO_NACIONALITAT.csv',';', 'latin1'] 7 | csv = pd.read_csv(nationalities[0], sep=nationalities[1], encoding =nationalities[2]) 8 | print(csv.head) 9 | 10 | -------------------------------------------------------------------------------- /content/Attention-mechanisms.md: -------------------------------------------------------------------------------- 1 | ## Attention Mechanisms 2 | Allows the network to pay more attention to the most important parts, so in an image it will find the most important pixels or in language transltation it will know what words to focus on and the order it has to process them. 3 | 4 | The inputs are passed in to an encoder, the encoder will generate the context (hidden states for all sequences) and will pass in the context to the decoder network and the decoder will output the sequence output. 5 | 6 | In computer vision the encoder will be a Convolutional Neural Network that will produce the feature vectors, in the [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://arxiv.org/pdf/1502.03044.pdf) paper it will use one of the latest convolution layers in the CNN of size 14 x 14 x 512 (that is 14 x 14 pixels image x 512 features) , flatten each feature into a 1-dimentional (14 x 14 = 196 x 1) vertical vector and end up with a matrix of 512 features resized size 196 x 512 which becomes the context vector. 7 | 8 | The context vector will be sent as the input of the decoder. 9 | 10 | There are 2 types of attention mechanisms, Additive attention and Mutiplicative attention. Not too long ago a new type have been implementet, The transformer, which uses Self-attention in the encoder and then Self-attention and Encoder-Decoder attention on the Decoder. 11 | 12 | Paper: [Attention Is All You Need](https://arxiv.org/abs/1706.03762) 13 | -------------------------------------------------------------------------------- /content/Batch-size.md: -------------------------------------------------------------------------------- 1 | ## Batch Size 2 | Batch size refers to the number of training examples utilized in one step or iteration. 3 | 4 | One step or iteration is one step of gradiend decent (one update of weights and parameters) 5 | 6 | The batch size can be either: 7 | 8 | * The same number of the total number of samples which makes **one step = an epoch**, this is called **batch mode** 9 | 10 | * A number greater than one but smaller than the total dataset samples, so for example for a dataset of 1000 images and a batch size of 100 then **10 steps = 1 epoch** thi sis called **mini-batch mode** 11 | 12 | * Exactly **one** in this case the gradient and the network parameters are updated after each sample this is called **stochastic mode**. 13 | -------------------------------------------------------------------------------- /content/CNN.md: -------------------------------------------------------------------------------- 1 | # Defining a CNN Network architecture 2 | Convolutional Neural networks will make use of the following types of layers: 3 | * Convolutional Layers 4 | * Maxpooling layers 5 | * Fully connected (linear) layers 6 | 7 | To define a network in PyTorch you create a class: 8 | 9 | **class Net**(nn.Module) 10 | 11 | Define the layers on the __init__ function 12 | 13 | ``` 14 | def __init__(self, n_classes): 15 | super(Net,self).__init__() 16 | # 1 input channel (grayscale image), 32 outputs or feature maps, 5x5 conv. kernel 17 | self.conv1 = nn.Conv2d(1,32,5) 18 | 19 | # pool with kernel_size = 2, stride = 2 20 | self.pool = nn.MaxPool2d(2,2) 21 | 22 | # fully connected layer, input_size, number of output classes 23 | self.fc1 = nn.Linear(32*4,n_classes) 24 | ``` 25 | 26 | The feedforward behavior is defined in the **forward** function which takes the image tensor x as input: 27 | 28 | ``` 29 | def forward(self,x): 30 | # one conv/relu + pool layers 31 | x = self.pool(F.relu(self.conv1(x))) 32 | 33 | # prep for linear layer by flattening the feature maps into feature vectors 34 | x = x.view(x.size(0), -1) 35 | # linear layer 36 | x = F.relu(self.fc1(x)) 37 | 38 | # final output 39 | return x 40 | ``` 41 | 42 | ## CNN's Glossary 43 | * **CNNs:** Convolutional neural network. That is, a network which has at least one convolutional layer. A typical CNN also includes other types of layers, such as pooling layers and dense layers. 44 | * **Convolution:** The process of applying a kernel (filter) to an image 45 | * **Kernel / filter:** A matrix which is smaller than the input, used to transform the input into chunks 46 | * **Padding:** Adding pixels of some value, usually 0, around the input image 47 | * **Pooling:** The process of reducing the size of an image through downsampling.There are several types of pooling layers. For example, average pooling converts many values into a single value by taking the average. However, maxpooling is the most common. 48 | * **Maxpooling:** A pooling process in which many values are converted into a single value by taking the maximum value from among them. 49 | * **Stride:** the number of pixels to slide the kernel (filter) across the image. 50 | * **Downsampling:** The act of reducing the size of an image -------------------------------------------------------------------------------- /content/Computer-Vision.md: -------------------------------------------------------------------------------- 1 | ## Computer Vision 2 | 3 | ## Types of Features and Image Segmentation 4 | 5 | **Types of features:** Edges, Corners and Blobs. 6 | 7 | **Corner Detector:** Intersection of 2 edges, can be calculated with Sobel operators (Sobel x and Sobel y) Gx and Gy (G for Gradient) 8 | 9 | **Dilation** (add pixels to the boundaries of an object) and **erosion** (removes pixels along object boundaries) can be combined to fill in gaps in the image or eliminate noise. 10 | 11 | Some combined operations are: **opening**, which is erosion followed by dilation, This is useful in noise reduction in which erosion first gets rid of noise (and shrinks the object) then dilation enlarges the object again. 12 | 13 | **Closing** is the reverse combination of opening; it’s dilation followed by erosion, which is useful in closing small holes or dark areas within an object. 14 | 15 | **Image Contouring:** Allow us to get the area, perimeter, center and bounding rectangle of an image. It can be obtained with a binary thresholded image with black and white pixels (inverted so the background is black) in openCV you can use cv2.findContours method. 16 | 17 | **K-means Clustering**: Separates an image into segments by clustering data points that have similar traits. K-means is an unsupervised learning method. 18 | 19 | ## Computer vision algorithms 20 | 21 | **ORB Algorithm**: Oriented Fast and Rotated Brief, creates feature vectors from detected keypoints and is invariant to rotations, changes in illumination, and noise. 22 | 23 | **HOG Algorithm**: Histogram of Oriented Gradients works by creating histograms of the distribution of gradient orientations in an image and then normalizing them in a very special way. This special normalization is what makes HOG so effective at detecting the edges of objects even in cases where the contrast is very low. These normalized histograms are put together into a feature vector, known as the HOG descriptor, that can be used to train a machine learning algorithm, such as a Support Vector Machine (SVM), to detect objects in images based on their boundaries (edges). 24 | 25 | **Convolutional Neural Networks**: this is a oversimplified example of an CNN architecture: 26 | Input Image -> Convolutional Layers -> Pooling layers -> Feature maps -> Fully Connected (linear) layer -> Class scores -> Predicted class 27 | 28 | Image Pre-processing: 29 | 30 | 1. Load image 31 | 2. Trimming 32 | 3. Resizing 33 | 4. Sharpening 34 | 5. Extending 35 | 6. Negating 36 | 7. Gray scaling 37 | 38 | 39 | -------------------------------------------------------------------------------- /content/Conda.md: -------------------------------------------------------------------------------- 1 | ## Conda 2 | I've gone through a lot of trouble setting up my enviroments, most of the times the problem is that it wont show up on my Jupyter Notebooks as Kernels so I cannot access them so here is a quick couple of commands to run to create an enviroment and make it show up as a Kernel in Jupyter. 3 | 4 | To create an enviroment with Python 3.6 5 | ```conda create --name myEnviromentName python=3.6``` 6 | 7 | To activate the enviroment 8 | ```source activate myEnviromentName``` 9 | 10 | To set the environment up as jupyter kernel 11 | ```python -m ipykernel install --user --name myEnviromentName --display-name "Python (myEnviromentName)"``` 12 | 13 | If an error comes up saying No Module name 'decorator' 14 | ```pip install decorator``` 15 | -------------------------------------------------------------------------------- /content/Data-augmentation.md: -------------------------------------------------------------------------------- 1 | ## Data augmentation 2 | In order to improve models without adding new data, we can do data augmentation, this will work well for images that do make sense to make transformations on top of, so for example it is not a good idea to do it for text or numbers because text the orientation of characters in these cases are important. 3 | 4 | But for example for satellite images or other types of pictures then it does make sense to do it. 5 | 6 | To do data augmentation we can do transformations like rotation, increasing and decreasing brightness, image cropping, flipping, image location, scaling, etc. -------------------------------------------------------------------------------- /content/Day-10.md: -------------------------------------------------------------------------------- 1 | I've been working on the facial keypoints detection project, read this paper [Facial Key Points Detection using Deep Convolutional Neural Network - NaimishNet](https://arxiv.org/pdf/1710.00977.pdf) and took it as a starting point for my network architecture. 2 | 3 | I'm using MSELoss as my loss function and Adam with default parameters as the optimizer (LR=0.001, \beta=(0.9,0.999)) and now trying to get the best performance possible on the network. 4 | 5 | This is my architecture so far: [models.py](code/cv/models.py) 6 | 7 | This is the best I could get so far: 8 | 9 | ``` 10 | # With MSELoss and optim.SGD(net.parameters(),0.001,momentum=0.9,nesterov=True) 11 | Epoch: 1, Batch: 10, Avg. Loss: 0.028066457714885473 12 | Epoch: 1, Batch: 20, Avg. Loss: 0.02104236278682947 13 | Epoch: 1, Batch: 30, Avg. Loss: 0.022216709144413472 14 | Epoch: 1, Batch: 40, Avg. Loss: 0.029279047250747682 15 | Epoch: 1, Batch: 50, Avg. Loss: 0.01847806265577674 16 | Epoch: 1, Batch: 60, Avg. Loss: 0.024008906725794078 17 | Epoch: 1, Batch: 70, Avg. Loss: 0.022673950437456368 18 | Epoch: 1, Batch: 80, Avg. Loss: 0.025031224079430103 19 | Epoch: 1, Batch: 90, Avg. Loss: 0.035366627294570206 20 | Epoch: 1, Batch: 100, Avg. Loss: 0.016411050129681824 21 | Epoch: 1, Batch: 110, Avg. Loss: 0.0172842089086771 22 | Epoch: 1, Batch: 120, Avg. Loss: 0.02427839869633317 23 | Epoch: 1, Batch: 130, Avg. Loss: 0.023034661076962948 24 | Epoch: 1, Batch: 140, Avg. Loss: 0.022957767266780137 25 | Epoch: 1, Batch: 150, Avg. Loss: 0.016886541806161403 26 | Epoch: 1, Batch: 160, Avg. Loss: 0.02071635639294982 27 | Epoch: 1, Batch: 170, Avg. Loss: 0.018030618131160737 28 | Epoch: 1, Batch: 180, Avg. Loss: 0.022987023554742338 29 | Epoch: 1, Batch: 190, Avg. Loss: 0.02365094544366002 30 | Epoch: 1, Batch: 200, Avg. Loss: 0.021768472623080015 31 | Epoch: 1, Batch: 210, Avg. Loss: 0.022931396309286357 32 | Epoch: 1, Batch: 220, Avg. Loss: 0.020561089925467967 33 | Epoch: 1, Batch: 230, Avg. Loss: 0.02189907068386674 34 | Epoch: 1, Batch: 240, Avg. Loss: 0.023978890758007763 35 | Epoch: 1, Batch: 250, Avg. Loss: 0.01856830148026347 36 | Epoch: 1, Batch: 260, Avg. Loss: 0.019809638615697622 37 | Epoch: 1, Batch: 270, Avg. Loss: 0.02397430771961808 38 | Epoch: 1, Batch: 280, Avg. Loss: 0.027975480444729327 39 | Epoch: 1, Batch: 290, Avg. Loss: 0.02225145073607564 40 | Epoch: 1, Batch: 300, Avg. Loss: 0.021185639034956693 41 | Epoch: 1, Batch: 310, Avg. Loss: 0.052688654605299236 42 | Epoch: 1, Batch: 320, Avg. Loss: 0.02006478551775217 43 | Epoch: 1, Batch: 330, Avg. Loss: 0.01888055559247732 44 | Epoch: 1, Batch: 340, Avg. Loss: 0.027196702361106873 45 | Epoch: 2, Batch: 10, Avg. Loss: 0.017323469463735818 46 | Epoch: 2, Batch: 20, Avg. Loss: 0.01812070393934846 47 | ``` 48 | 49 | 50 | -------------------------------------------------------------------------------- /content/Debug-jupyter.md: -------------------------------------------------------------------------------- 1 | # Debug in Jupyter notebooks 2 | ## Variable inspector 3 | A quick way to inspect variables can be done with the jupyter extension ```Variable Inspector``` if a more line by line debug is required then you can either use pdb or what I'm using now is ```PixieDebugger``` that gives me a more traditional way to debug code 4 | 5 | ## Debug with PixieDebugger 6 | 7 | Install PixieDust using the following pip command: 8 | ```pip install pixiedust``` 9 | 10 | Import it: 11 | ```import pixiedust``` 12 | 13 | TO use PixieDebugger for a specific cell, add: 14 | ```%%pixie_debugger``` 15 | to the top of the cell and run it. -------------------------------------------------------------------------------- /content/Defining-network-structure.md: -------------------------------------------------------------------------------- 1 | ## Defining a network structure 2 | The hardest part if to develop an intuition to decide on what layers you will include in a given network since the network will perform different depending on the complexity of the task, some recommended things to try are: 3 | 4 | * Change the number of convolutional layers 5 | * Increase the size of convolutional kernels for larger images 6 | * Change loss and optimization functions 7 | * Change hyperparameters such as learning rate and momentum 8 | * Add layers to prevent overfitting 9 | * Change the batch_size of the data loader to see how larger batch sizes can affect training 10 | 11 | Make notes on how the loss changes and tune it from there. -------------------------------------------------------------------------------- /content/Downloading-Datasets.md: -------------------------------------------------------------------------------- 1 | ## Kaggle 2 | To Download datasets from Kaggle you can use the: 3 | ```kaggle-cli``` 4 | 5 | You can install it with pip: 6 | ```pip install kaggle-cli``` 7 | 8 | or to upgrade it using: 9 | ```pip install kaggle-cli --upgrade``` 10 | 11 | The only problem with it is that it will download all of the dataset files, which can be huge (more than 20 GBs). 12 | 13 | Another way to do it is with the [CurlWGet](https://chrome.google.com/webstore/detail/curlwget/jmocjfidanebdlinpbcdkcmgdifblncg?hl=en) Chrome extension, that way you can download only the file you want directly into your project folder. 14 | 15 | You can also use symbolic links to set your data folder in different drives or outside of your project folder. 16 | 17 | ```ls -l``` will show where folders and links are pointing to. 18 | 19 | ### How to submit it to kaggle 20 | 21 | Kaggle will provide with the strucutre of the csv to create and submit: 22 | 23 | You can create a csv file using Pandas 24 | ```df.to_csv('file.gz', compression='gzip', index=False)``` 25 | 26 | 27 | 28 | 29 | -------------------------------------------------------------------------------- /content/Dropout.md: -------------------------------------------------------------------------------- 1 | ## Dropout 2 | Will randomly turn on and off nodes in each layer (with some specified probability) each epoch during the feedforward and backpropagation, that means that the node that was disabled will not contribute to the prediction and will neither get the weighst updated during backpropagation, this will help the model generalize better and increase accuracy on the test dataset. By turning off nodes randomly it makes all nodes work better as a team by making sure no node is too weak or too strong. 3 | 4 | Dropout layers often come near the end of the network; placing them in between fully-connected layers for example can prevent any node in those layers from being too strong. 5 | 6 | Dropout is specified as part of the network inside the ```__init__``` method, example: 7 | ```self.dropout = nn.Dropout(p=0.2)``` 8 | 9 | Dropout will be defined with a probability of a node being dropped, so when ```ps=0.25``` there is a 25% probability of that node to be dropped. 10 | 11 | The bigger the network and the more parameters a network is more likely to overfit, to fix that having a higher dropout probability might help to avoid overfitting. 12 | 13 | Some standard dropout probs are 0.25 for the first layer, 0.5 for a second layer and they are placed before linear fully connected layers. 14 | 15 | In practice I've seen up to 6 dropout layers increasing on each layer from 0.1 probability to 0.6 for the last dropout layer. -------------------------------------------------------------------------------- /content/Embeddings.md: -------------------------------------------------------------------------------- 1 | ## Embeddings 2 | 3 | An embedding is a mapping from discrete objects, such as words, to vectors of real numbers. 4 | 5 | For example, a 300-dimensional embedding for English words could include: 6 | 7 | ``` 8 | blue: (0.01359, 0.00075997, 0.24608, ..., -0.2524, 1.0048, 0.06259) 9 | blues: (0.01396, 0.11887, -0.48963, ..., 0.033483, -0.10007, 0.1158) 10 | orange: (-0.24776, -0.12359, 0.20986, ..., 0.079717, 0.23865, -0.014213) 11 | oranges: (-0.35609, 0.21854, 0.080944, ..., -0.35413, 0.38511, -0.070976) 12 | ``` 13 | The individual dimensions in these vectors typically have no inherent meaning. Instead, it's the overall patterns of location and distance between vectors that machine learning takes advantage of. 14 | 15 | Embeddings are important for input to machine learning. Classifiers, and neural networks more generally, work on vectors of real numbers. They train best on dense vectors, where all values contribute to define an object. However, many important inputs to machine learning, such as words of text, do not have a natural vector representation. Embedding functions are the standard and effective way to transform such discrete input objects into useful continuous vectors. 16 | 17 | Embeddings are suitable for any categorical variables. 18 | 19 | They are actually just hidden units in one or more layers, you will need to decide on the number of units, each unit of the embedding will be one parameter to track, the bigger the number of units the more likely the network is to overfit. 20 | 21 | The dot product of two embeddings is a measure of their similarity. 22 | 23 | ## Google Definition 24 | A categorical feature represented as a continuous-valued feature. Typically, an embedding is a translation of a high-dimensional vector into a low-dimensional space. For example, you can represent the words in an English sentence in either of the following two ways: 25 | 26 | * As a million-element (high-dimensional) sparse vector in which all elements are integers. Each cell in the vector represents a separate English word; the value in a cell represents the number of times that word appears in a sentence. Since a single English sentence is unlikely to contain more than 50 words, nearly every cell in the vector will contain a 0. The few cells that aren't 0 will contain a low integer (usually 1) representing the number of times that word appeared in the sentence. 27 | * As a several-hundred-element (low-dimensional) dense vector in which each element holds a floating-point value between 0 and 1. This is an embedding. 28 | 29 | Embeddings are trained by backpropagating loss just like any other parameter in a neural network. 30 | 31 | ## Embedding space 32 | The d-dimensional vector space that features from a higher-dimensional vector space are mapped to. Ideally, the embedding space contains a structure that yields meaningful mathematical results; for example, in an ideal embedding space, addition and subtraction of embeddings can solve word analogy tasks. 33 | 34 | [Word Embedding Visual Inspector](https://ronxin.github.io/wevi/) 35 | 36 | 37 | -------------------------------------------------------------------------------- /content/FCNNs.md: -------------------------------------------------------------------------------- 1 | # Fully- Convolutional Neural Networks 2 | 3 | In fully convolutional networks the last fully connected layer (linear layer) gets replaced by a 1 x 1 convolution. This allows to preserve the spatial information and removes the constraint of the input size image, in CNNs the fully connected layer shape is different for every image size but not in this case. 4 | 5 | A Fully-Convolutional Neural Network consists of an Encoder (the convolutions that we normally use in a CNN) followed by a 1 x 1 convolution and a decoder which is the opposite of the encoder, it is called a transposed convolution or deconvolution, but it is not the same a in the context of a transposed matrix. A transposed convolution is essentially a a reverse convolution where the forward and backward passes are swapped 6 | 7 | The goal of the encoder is to extract features from the image the goal of the decoder is to upscale the output of the encoder. 8 | 9 | FCNNs can also skip layers to preserve information and segment images, all layers must be added by the end of the network to have all the information. 10 | -------------------------------------------------------------------------------- /content/Feather.md: -------------------------------------------------------------------------------- 1 | # Feather 2 | 3 | ## What is Feather 4 | Feather is a fast, lightweight, and easy-to-use binary file format for storing data frames. It has a few specific design goals: 5 | 6 | * Lightweight, minimal API: make pushing data frames in and out of memory as simple as possible 7 | * Language agnostic: Feather files are the same whether written by Python or R code. Other languages can read and write Feather files, too. 8 | * High read and write performance. When possible, Feather operations should be bound by local disk performance. 9 | 10 | ## How to install Feather 11 | ```pip install feather-format``` 12 | 13 | ## How to use Feather 14 | ``` 15 | import feather 16 | 17 | path = 'data.feather' 18 | 19 | # To write a feather file 20 | feather.write_dataframe(df, path) 21 | 22 | # To load a feather file 23 | df = feather.read_dataframe(path) 24 | ``` -------------------------------------------------------------------------------- /content/GPU.md: -------------------------------------------------------------------------------- 1 | # GPU 2 | The GPU performance state APIs are used to get and set various performance levels on a per-GPU basis. P-States are GPU active/executing performance capability and power consumption states. 3 | 4 | P-States range from P0 to P15, with P0 being the highest performance/power state, and P15 being the lowest performance/power state. Each P-State maps to a performance level. Not all P-States are available on a given system. The definition of each P-States are currently as follows: 5 | 6 | * P0/P1 - Maximum 3D performance 7 | * P2/P3 - Balanced 3D performance-power 8 | * P8 - Basic HD video playback 9 | * P10 - DVD playback 10 | * P12 - Minimum idle power consumption 11 | 12 | To see the current state and memory available I use the command: 13 | ```nvidia-smi``` 14 | 15 | It can be very useful to see the maximum batch size to use without getting a CUDA out of memory error. 16 | 17 | -------------------------------------------------------------------------------- /content/Generate-requirements.md: -------------------------------------------------------------------------------- 1 | ## Generate requirements.txt file 2 | ```pip install pipreqs``` 3 | 4 | ```pipreqs /path/to/project``` 5 | -------------------------------------------------------------------------------- /content/High-bias-and-high-variance.md: -------------------------------------------------------------------------------- 1 | ## Bias and Variance 2 | When looking at the train set error and comparing to the cross validation error we could have the following situations: 3 | 4 | Train set error: 1% (the model is doing very well) 5 | Cross Val error: 11% 6 | 7 | There is a big variance in error between the training set and cross val. set, this is an example of **High Variance**, and it happens because the model cannot generalize well so it makes a lot of mistakes when using new data. 8 | 9 | Train set error: 15% (the model is not doing very well) 10 | Cross Val error: 16% (similar to the training set, not big variance) 11 | 12 | This is an example of **High Bias**, the model needs to train more to get better 13 | 14 | Train set error: 15% (the model is not doing very well) 15 | Cross Val error: 30% (big variance) 16 | 17 | This is an example of both previous cases together, the model is not performing well on the training set so that is already an example of high bias and the variance with the dev set is huge so that adds high variance as well. So in this example we have High Bias and High Variance. 18 | 19 | Train set error: 1% (the model is doing very well) 20 | Cross Val error: 2% (similar to the training set, not big variance) 21 | 22 | This is a perfect example of **Low Bias** and **Low Variance** 23 | 24 | Base error or human error is the error % a person will have when performing the same task 25 | 26 | ## Basic recipe for ML 27 | * Does the model have high bias? (training data performance) 28 | Try a bigger network, train longer (more epochs), try a different NN architecture, keep doing it until the training data fits well. 29 | 30 | * Does the model have High Variance? (Dev Set performance) 31 | Add more data, data augmentation, try regularization, NN architecture. 32 | 33 | For High variance add data 34 | For High Bias improve the network 35 | 36 | 37 | -------------------------------------------------------------------------------- /content/Hyperparameters.md: -------------------------------------------------------------------------------- 1 | ## Hyperparameters 2 | Is a variable that we need to set to a value before we can train a Neural Network. 3 | 4 | There are no magic numbers, it all depends on the architecture, data and problem to solve, etc. 5 | 6 | ### Optimizer Hyperparameters 7 | 8 | ```learning rate``` 9 | Is the most important hyperparameter of all, typical values are: 0.1, 0.01, 0.001, 0.0001, 0.00001 and so on. 10 | 11 | If the error diminishes very slowly then increase the learning rate. 12 | 13 | If the error bounces back from going smaller, then bigger and the smaller again, then decrease the learning rate. 14 | 15 | Another technique is called Learning Rate Decay which just consists in decreasing the learning rate every _x_ epochs. 16 | 17 | Adaptive Learning are algorithms that will decrease and increase the learning rate when needed. 18 | 19 | Some Adaptive Learning Optimizers are: 20 | * [AdamOptimizer](https://pytorch.org/docs/stable/optim.html?highlight=adam#torch.optim.Adam) 21 | * [AdagradOptimizer](https://pytorch.org/docs/stable/optim.html?highlight=adagrad#torch.optim.Adagrad) 22 | 23 | ```mini batch size``` 24 | Small mini batches will have more noise and be slower 25 | Big mini batches could result in worse accuracy 26 | 27 | Typical mini batch sizes are 1,2,4,8,16,23,64,128,256 with 32 being a good starting point. 28 | 29 | ```epochs``` 30 | Epochs are also known as number of iterations. 31 | 32 | To choose the best number of epochs we should check the Validation Error and train while the validation keeps decreasing. 33 | 34 | ### Model Hyperparameters 35 | ```numbers of hidden units``` 36 | For the number of hidden units, we need to have enough hidden units to learn the function, so for a simple function the hidden layes will be less than for a complex one. 37 | 38 | However if we add too many hidden units the network will have too much capacity that it will just memorize the training set and that will lead to overfitting. 39 | 40 | So if your model is overfitting then you could reduce the number of hidden units, or use regularization like L2 regularization or Dropout. 41 | 42 | ```number of layers``` 43 | As for the number of hidden layers, looks like 3 hidden layers works quite well, and while adding more can be benefitial but rarely helps much more, the exception being Convolutional Neural Networks. 44 | 45 | ```models specific hyperparameters``` 46 | For RNNs deciding wether to use a normal RNN cell, a LSTM cell or GRU cells, the number of layers to stack and the embeddings dimentions. 47 | -------------------------------------------------------------------------------- /content/Image-Captioning.md: -------------------------------------------------------------------------------- 1 | ## Image Captioning Project 2 | In this project I will train a network with the COCO Dataset (**C**ommon **O**bjects in **Co**ntext). This dataset contains images and a set of 5 different captions per image. 3 | 4 | I will train a CNN-RNN model by feeding it with the image and captions so the network will learn to generate captions given an image. Once trained I will give the trained network an image and expect to get a caption that describes that image. 5 | 6 | ### Image transformations 7 | As always the images will need some preprocessing: 8 | 9 | * Resize to a smaller size for example 256 x 256 pixels 10 | * Random crop to 224 x 224 standard size for most pre-trained models like Densenet or 299 x 299 for Inception (v3) 11 | * Randomly flipping the image 12 | * And the most important transformation convert it to an Image Tensor ```transforms.ToTensor()``` 13 | * If using a pre-trained model, normalize the images using mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225] 14 | 15 | ### Captions tokenization 16 | I will use NLTK to tokenize the captions from the dataset. 17 | First set the caption to lower case and then tokenize words. 18 | ```nltk.tokenize.word_tokenize(caption.lower())``` 19 | 20 | I will use 3 special tags: `````` to signal that is is the start of the caption`````` for the end of the caption and `````` for unknown words, these words will be words with very low relevance, words that do not appear that many times given a threshold. 21 | 22 | These special tags will be at the beginning of the vocab dictionary and their corresponign indices will be 0 for ```start```, 1 for ```end``` and 2 for ```unk```. 23 | 24 | So a simple phrase like ```This is a short text``` might be tokenized as follow: 25 | 26 | ```['', 'This', 'is', 'a', 'short', 'text', '']``` 27 | 28 | Then the words will be replaced for the corresponding indexes in the vocabulary, so it might become something like this: 29 | 30 | ```[0, 3, 98, 754, 3, 396, 1]``` 31 | 32 | 33 | 34 | 35 | 36 | -------------------------------------------------------------------------------- /content/Jobs-in-Computer-Vision.md: -------------------------------------------------------------------------------- 1 | # Jobs in Computer Vision: 2 | 3 | ## How can I get a job in Computer Vision industry? 4 | 5 | Work on your own Computer vision projects, the best way to show your expertise is by showing what you have done rather that trying to convince others by talking about how much you know. So create something and share it with the world, post code in Github public repositories, or blog about your findings in Medium. 6 | 7 | You can work on projects with gadgets like the AWS Deeplens or if you are into hardware why not try something with the Goole AIY Vision Kit. Maybe buy a Raspbery Pi and some servos and do your own miniature Self-driving car!, document the process and share it online! 8 | 9 | Attend to events like meetups, conferences or study groups related to Computer Vision, Deep Learning and related fields, good old in-person networking will definitely increase your chances or getting a job in the industry! 10 | 11 | I have also saw mobile developers who are working with computer vision optimized for mobiles (iOS/Android), which means the algorithms used must be really optimized to run fast on mobilde devices and avoid draining the battery. 12 | 13 | So be creative, see on what fields computer vision can be applied, cross check your current skills with computer vision and see if you can find a good match. 14 | 15 | ## How do I search for computer vision jobs? 16 | 17 | While searching for "Computer Vision jobs" might give some results, chances are there are a lot more offers that you won't find with these keywords. 18 | 19 | You should also try other keywords like for example: Machine Vision, robotics vision, Image processing, Image Analysis or Image processing, or search for skills instead of job titles, like image feature extration, or more technical keywords like Simultaneous Location and Mapping (SLAM), HOG, ORB and so on, be creative! 20 | 21 | ## Computer Vision applications 22 | 23 | Computer vision can be applied to many fields in the real world, a big field it which Computer vision is being used a lot is medicine with image analysis, identify lessons or diseases and much more. 24 | 25 | Self driving cars is also a growing field making use of computer vision from detecting a pedestrian to slow down, detect cars to know when to switch lines, detect trafic signs to adjust speed, stop, and so on. 26 | 27 | There other known applications like text translation or image recognition, from general recognition for example to detect a bolt, a cup and a mobile on a table to more specific stuff like to classify the species of insects or type of flower on the image. 28 | 29 | Other interesting fields like hand recognition to translate hand language or detecting emotions and so on, I'm sure there are so many more application and future applications to come in the near future. 30 | 31 | This is a very fast changing field, with new algorithms and papers being published every day, so make sure to stay up to date by reading what is going on in the field in a daily basis. 32 | 33 | ## Where to start looking for a Computer Vision job? 34 | 35 | Here are some basic links where you can start, but don't forget to do some more google searches for computer vision jobs! 36 | 37 | [LinkedIn Jobs Search](https://www.linkedin.com/jobs/search/?keywords=computer%20vision&location=Worldwide&locationId=OTHERS.worldwide) 38 | 39 | [Glassdoor Job Search](https://www.glassdoor.com/Job/jobs.htm?suggestCount=0&suggestChosen=false&clickSource=searchBtn&typedKeyword=computer+vision&sc.keyword=computer+vision&locT=&locId=&jobType=) 40 | 41 | [indeed Job Search](https://www.indeed.com/q-Computer-Vision-jobs.html) 42 | 43 | [Pilot.AI Jobs](https://pilotai.workable.com/) 44 | 45 | [stackoverflow Jobs Search](https://stackoverflow.com/jobs/developer-jobs-using-computer-vision) 46 | 47 | -------------------------------------------------------------------------------- /content/Jupyter-extensions.md: -------------------------------------------------------------------------------- 1 | # Jupyter Notebook Extension 2 | 3 | To install the packages required with conda use the following commands: 4 | 5 | ``` 6 | conda install -c conda-forge jupyter_contrib_nbextensions 7 | conda install -c conda-forge jupyter_nbextensions_configurator 8 | conda install -c conda-forge jupyterthemes 9 | ``` 10 | 11 | Then run ```jupyter notebook``` 12 | 13 | Open the Extensions configurations panel ```http://localhost:8888/nbextensions``` 14 | 15 | Enable the extensions and enjoy them! 16 | 17 | Some extensions to autoformat code require autopep8 and yapf 18 | 19 | ``` 20 | conda install -c conda-forge yapf 21 | conda install -c conda-forge autopep8 22 | ``` 23 | 24 | Official documentation [here](https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/install.html) 25 | 26 | ## With pip 27 | You can do the same with pip 28 | ``` 29 | pip install jupyter_contrib_nbextensions 30 | pip install jupyter_nbextensions_configurator 31 | pip install jupyterthemes 32 | jupyter contrib nbextension install 33 | ``` 34 | -------------------------------------------------------------------------------- /content/Jupyter-server.md: -------------------------------------------------------------------------------- 1 | ## Configure a Jupyter Notebook Server 2 | ### Configure the server 3 | 1. On the server you first create the config with: 4 | ```jupyter notebook --generate-config``` 5 | 6 | 1. Then open the file that was created and change these 2 settings: 7 | 8 | ```c.NotebookApp.allow_origin = '*' #allow all origins``` 9 | 10 | ```c.NotebookApp.ip = '0.0.0.0' # listen on all IPs``` 11 | 12 | 1. Save the file. 13 | 14 | 1. Set a password to protect the server with: 15 | ```jupyter notebook password``` 16 | 17 | 1. Start the server with: 18 | ```jupyter notebook --ip 0.0.0.0 --port 8888``` 19 | 20 | 1. Open a browser on the client machine and open the url: 21 | 22 | ```192.168.1.100:8888``` 23 | 24 | You might need to open the port 8888 on the server if the firewall is blocking it, for ubuntu run: 25 | ```sudo ufw allow 8888``` 26 | 27 | This will enable tcp:8888 port, which is ur default jupyter port. 28 | 29 | To access it remotely you will need to forward the port on the router to the server IP. I recommend configuring an static IP on the server or set a DHCP rule in the router so it assigns the same IP to the server every time. 30 | -------------------------------------------------------------------------------- /content/LSTM.md: -------------------------------------------------------------------------------- 1 | ## LSTM Cells 2 | 3 | LSTM Cells will replace hidden layers on a Recurrent Neural Network, and can be stacked so you can have multiple hidden layers, all of them being LSTM cells. 4 | 5 | It is comprised of 4 gates with 2 inputs and 2 outputs: 6 | 7 | **Learn Gate:** it takes the **Short term memory** and the **Event** and combines them with a ```than``` function and then ignores a part of it by multiplying it by an _ignore factor_ it. To calculate the _ignore factor_ it combines de Shor term memory with the event multiplies it by the _Ignore Weights_ and activates using a ```sigmoid``` function and outputs the **new Short Term Memory**. 8 | 9 | **Forget Gate:** it takes the **Long term memory** and multiplies it by a _forget factor_ ft to calculate the _forget factor_ it combines de Shor term memory with the event multiplies it by the _Forget Weights_ and activates using a ```sigmoid``` function. 10 | 11 | **Remember Gate:** it takes the **Forget Gate** output and adds it to the **Learn Gate** outputs the **new Long Term Memory**. 12 | 13 | **Use Gate**: or output gate, will take the **Forget Gate** and activate it with ```tanh``` then Take the **Shor Term Memory** and activate it with ```sigmoid``` and then multiplies them, and that is the output. 14 | 15 | ## LSTM in PyTorch 16 | 17 | To define a LSTM: 18 | ```lstm = nn.LSTM(input_size=input_dim, hidden_size=hidden_dim, num_layers=n_layers)``` 19 | 20 | To initialize the hidden state: 21 | ```h0 = torch.randn(1, 1, hidden_dim)``` 22 | ```c0 = torch.randn(1, 1, hidden_dim)``` 23 | 24 | We will need to wrap everything in Variable, input is a tensor 25 | ```inputs = Variable(inputs)``` 26 | ```h0 = Variable(h0)``` 27 | ```c0 = Variable(c0)``` 28 | 29 | Get the outputs and hidden state 30 | ```out, hidden = lstm(inputs, (h0, c0))``` 31 | 32 | For Natural Language 33 | 34 | ## Basic LSTM Network 35 | The first layer of a LSTM Network should always be an embedding layer which will take the vocabulary dictionary size as the input. 36 | 37 | Before we initialize the network we need to define the ```vocabulary``` that is simply a dictionary (with unique words) and each word will have a numerical index, to do so we can use the following function: 38 | 39 | ``` 40 | def add_word(word,dictionary): 41 | if word not in dictionary: 42 | dictionary[word] = len(dictionary) 43 | ``` 44 | 45 | In this example we will use a vocabulary and a different dictionary for the outputs 46 | 47 | ``` 48 | training_data = [ 49 | ("The cat ate the cheese".lower().split(), ["DET", "NN", "V", "DET", "NN"]), 50 | ("She read that book".lower().split(), ["NN", "V", "DET", "NN"]), 51 | ("The dog loves art".lower().split(), ["DET", "NN", "V", "NN"]), 52 | ("The elephant answers the phone".lower().split(), ["DET", "NN", "V", "DET", "NN"]) 53 | ] 54 | 55 | word2idx = {} 56 | tag2idx = {} 57 | 58 | for sent, tags in training_data: 59 | # create a dictionary that maps words to indices 60 | for word in sent: 61 | add_word(word,word2idx) 62 | # create a dictionary that maps tags to indices 63 | for tag in tags: 64 | add_word(tag,tag2idx) 65 | ``` 66 | 67 | ## Defining the network and feedforward function 68 | 69 | ``` 70 | class LSTMTagger(nn.Module): 71 | 72 | def __init__(self, embedding_dim, hidden_dim, vocab_size, tagset_size): 73 | super(LSTMTagger, self).__init__() 74 | 75 | self.hidden_dim = hidden_dim 76 | self.word_embeddings = nn.Embedding(vocab_size, embedding_dim) 77 | self.lstm = nn.LSTM(embedding_dim, hidden_dim) 78 | self.hidden2tag = nn.Linear(hidden_dim, tagset_size) 79 | self.hidden = self.init_hidden() 80 | 81 | def init_hidden(self): 82 | return (torch.zeros(1, 1, self.hidden_dim), 83 | torch.zeros(1, 1, self.hidden_dim)) 84 | 85 | def forward(self, sentence): 86 | embeds = self.word_embeddings(sentence) 87 | lstm_out, self.hidden = self.lstm(embeds.view(len(sentence), 1, -1), self.hidden) 88 | tag_outputs = self.hidden2tag(lstm_out.view(len(sentence), -1)) 89 | tag_scores = F.log_softmax(tag_outputs, dim=1) 90 | return tag_scores 91 | ``` 92 | 93 | Apart from LSTMs there are also other types of architectures that also work well, like for exampla the **Gated Recurrent Unit** or **GRU**, the GRU combines the forget and learn gate into a update gate and returns only a "New Working Memory" instead of a long-term and short-term -------------------------------------------------------------------------------- /content/Layer-shapes.md: -------------------------------------------------------------------------------- 1 | ## Layer Shapes Calculation in CNNs 2 | 3 | Calculating the shape of layers have always been a hard thing for me, today I found this: 4 | 5 | For any convolutional layer, the output feature maps will have the specified depth (a depth of 10 for 10 filters in a convolutional layer) and the dimensions of the produced feature maps (width/height) can be computed as the input image width/height, W, minus the filter size, F, divided by the stride, S, all + 1. 6 | 7 | The equation looks like this: 8 | ```output_dim = (Width - Filter + 2*Padding) / Stride + 1```. 9 | 10 | Where: 11 | ```Filter```: can be also referred as the ```Kernel```. 12 | ```Padding```: Could be zero, in that case ```2 x 0 = 0``` which will simplify the equation to: 13 | ```output_dim = (Width - Filter) / Stride + 1```. 14 | 15 | For a pool layer with a size 2 and stride 2, the output dimension will be reduced by a factor of 2. Read the comments in the code below to see the output size for each layer. 16 | 17 | So for example for an input of 28 x 28 pixels in grayscale (1,28,28) 18 | 19 | Where 1 is the channels (since it is grayscale it is only 1) when applying one convolutional layer like this: ```nn.Conv2d(1, 10, 3)``` 20 | 21 | Where ```nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding)``` so 1 channel in, 10 out and a kernel of 3 x 3 then the output layer will have a shape of (10,26,26) ```10``` because that is what we specified as output channel and 26 because it is the result of the formula ```output_dim = (W-F)/S + 1``` that being translated into ```=((28-3)/1)+1``` (28 pixels - 3 of the kernel (or filter) / by the stride which we didn't specify so by default that is 1 and then + 1) = 26 22 | 23 | If then we apply a MaxPooling layer with a kernel of 2 x 2 and stride of 2 x 2 then the formula is easier just divide it by 2! 24 | 25 | So if we apply this ```nn.MaxPool2d(2, 2)``` then we just need to divide the previous result ```26``` that is ```26 / 2 = 13``` that will give us a layer of shape ```(10,13,13)``` 10 since we keep the same number of outputs so we still take 10 as number of inputs. 26 | 27 | If at any moment the formula yields a result with decimals, the number will be rounded down (just get rid of the decimal part) 28 | -------------------------------------------------------------------------------- /content/Learning-rate.md: -------------------------------------------------------------------------------- 1 | ## Learning rates 2 | **Cosine Annealing**: uses cos/2 (so half of the cosine function) to decrease the learning rate as it trains. 3 | 4 | We can use Cosine Annealing on Stochastic Gradiend Descent with _warm restarts_, that means that when a cycle ends the learning rate will jump again to the highest learning rate (restart on top of the cosine function) and if the loss didn't increase it means we just landed on a good part of of the gradient which is quite flat. 5 | 6 | I found this [artile](https://www.jeremyjordan.me/nn-learning-rate/) on learning rates which is quite interesting, there is a section talking about _Stochastic Gradient Descent with Warm Restarts (SGDR)_ 7 | 8 | ## Finding the optimal Learning Rate 9 | 10 | The Fast.AI framework implemented the [Cyclical Learning Rates for Training Nueral Networks paper](https://arxiv.org/pdf/1506.01186.pdf) learn.lr_find() which in simple words starts with a very small learning rate and starts doubling it until the loss get worse so then it stops. When you reach that step you can plot the results and see the best learning rate when the loss was still improving and before it overshoots. 11 | 12 | 13 | -------------------------------------------------------------------------------- /content/Localization.md: -------------------------------------------------------------------------------- 1 | ## Localization 2 | Robot localization in essence is based in two main steps: 3 | ```Sense``` and ```Move``` 4 | 5 | It will start with a initial belief (or prior) of maximum confusion where the probability distribution will be uniform (flat, which means it has the same value everywhere) 6 | 7 | Then it will start cycling through sensor measurements (```Sense```) and movements (```Move```) 8 | 9 | When the robot moves it loses information and when it senses it gains information. 10 | 11 | If the robot will move enough steps without sensing (updating beliefs) the probability distribution will be uniform (max confusion state) 12 | 13 | Entropy measures the amount of uncertainty. 14 | 15 | -------------------------------------------------------------------------------- /content/Machine-learning-concepts.md: -------------------------------------------------------------------------------- 1 | ### Key ML Terminology 2 | 3 | * **Feature**: features are the input variables we feed into a network, it can be as simple as a signle number or more complex as an image (which in reality is a vector of numbers, where each pixel is a feature) 4 | * **Label**: is the thing we are predicting, it is normally refered as ```y``` 5 | * **Prediction**: or predicted value if the value we predict with a previously trained model for a given output and it is refered as ```y'``` 6 | 7 | ### Regression vs. classification: 8 | * A regression model predicts continuous values. 9 | * A classification model predicts discrete values. 10 | 11 | #### Linear Regression 12 | 13 | Is a method for finding the straight line or hyperplane that best fits a set of points. 14 | 15 | Line formula: 16 | y = wx + b 17 | 18 | Where: 19 | 20 | w = Weights 21 | x = Input features 22 | b = Bias 23 | 24 | Some convenient loss functions for linear regression are: 25 | **L2 Loss** = (y - y')2 26 | 27 | **Mean Square Error**: is the average squared loss per example over the whole dataset. To calculate MSE, sum up all the squared losses for individual examples and then divide by the number of examples 28 | 29 | ![MSE](../resources/mse.png "MSE formula") 30 | 31 | When training a model we want to minimize the loss as much as possible to make the model more accurate. 32 | 33 | [Go Back](../README.md) -------------------------------------------------------------------------------- /content/Momentum.md: -------------------------------------------------------------------------------- 1 | ## Momentum 2 | In gradient descent momentum uses a constant \beta between 0 and 1, and it is used to calculate the next step size, it will weight previous steps, so the previous step matthers a lot and the weight for each previous step will then decrease and this is done by using the the constant \beta so: 3 | * the previos step will be multiplied by 1 4 | * the step before that \times \beta 5 | * the one before that \times \beta2 6 | * the one before that \times \beta3 7 | * and so on... 8 | 9 | Momentum is specified in the optimizer, example: 10 | 11 | ```optimizer = optim.SGD(net.parameters(),lr=0.001,momentum=0.9)``` -------------------------------------------------------------------------------- /content/Motion.md: -------------------------------------------------------------------------------- 1 | ## Motion in Computer Vision 2 | Motion can be tracked with a 2D motion vector. A vector has a direction and magnitude which will determine the direction and amount of movement between one frame and the next one. 3 | 4 | First we will have to defive special points to track like for example intersections or corners once we localize those features in both frames we can track the motion vector. The magniture of the vector (how much it moved) can be found by the Pythagorean theorem: 5 | a2 + b2 = c2 6 | 7 | so the magnitude = sqrt(a2 + b2) 8 | 9 | as for the direction we can calculate the angle: 10 | 11 | angle = tan-1(b/a) 12 | 13 | Knowing those 2 things we can track an object -------------------------------------------------------------------------------- /content/NLTK.md: -------------------------------------------------------------------------------- 1 | ## NLTK 2 | Stands for Natural Language Toolkit. 3 | 4 | Tokenization is just splitting sentences in a list of words. 5 | 6 | ### Word Tokenization with Python built in functions 7 | ```word = text.split()``` 8 | 9 | ### Word Tokenization with NLTK 10 | ``` 11 | from nltk.tokenize import word_tokenize 12 | words = word_tokenize(text) 13 | ``` 14 | 15 | ### Sencentes Tokenization with NLTK 16 | ``` 17 | from nltk.tokenize import sent_tokenize 18 | words = sent_tokenize(text) 19 | ``` 20 | 21 | [NLTK Documentation](http://www.nltk.org/api/nltk.tokenize.html) -------------------------------------------------------------------------------- /content/OpenCV.md: -------------------------------------------------------------------------------- 1 | # OpenCV 4.1.1 2 | ## Installing OpenCV 4 in Mac Os 3 | 1. Install Anaconda 4 | 1. Create a new environment 5 | 1. Open the new environment in the terminal 6 | 1. Run: ```conda install -c conda-forge/label/main opencv``` 7 | 1. Run: ```conda install notebook``` 8 | 1. Run: ```Jupyter notebook``` 9 | 1. In the notebook ```import cv2``` 10 | 1. Check the version with ```cv2.__version__``` 11 | 1. You should see something like 4.1.1 12 | 13 | ## Problems when showing an image or video 14 | If when opening an image or playing a video/camera stream it just crashes everytime, then there is a simple trick to fix it, after destroying all windows sentence just add: ```cv2.waitKey(1)``` 15 | -------------------------------------------------------------------------------- /content/Probability.md: -------------------------------------------------------------------------------- 1 | ## Probability Basic Concepts 2 | ### Independent Events 3 | The probability of an event does not affect the probability of the next event. So for example tossing a coin does not affect the probability of the next flip. 4 | 5 | ### Dependent Events 6 | Two events are dependent when the probability of one influences the likelihood of the other event. 7 | 8 | ### Joint Probability 9 | Is the probability of two or more **indepentent events** happening together. It is calculated by multiplying the probability of each independent event. So the probility of tossing a coin and getting Heads twice in a row is calculated like this 10 | ```P(H) * P(H) = 0.5 * 0.5 = 0.25``` 11 | 12 | ## Probability Distributions 13 | If we use probability distributions for localization, when the distribution is uniform (all values are the same) it is known as the state of Max Confusion, as it is equally possible to be located on anywhere. -------------------------------------------------------------------------------- /content/PyTorch.md: -------------------------------------------------------------------------------- 1 | ## PyTorch 2 | We can define our own PyTorch modules, to do so we need to inherit from ```nn.Module``` 3 | 4 | To have a fully functional PyTorch layer we can create a constructor and call the parent class constructor. 5 | ```def __init__(self): 6 | super().__init__() 7 | ``` 8 | 9 | Then all we need to do is to define the forward function and return the results of a forward pass. 10 | 11 | ## PyTorch Squeeze 12 | Is used to delete one rank 13 | 14 | 15 | 16 | -------------------------------------------------------------------------------- /content/RNN.md: -------------------------------------------------------------------------------- 1 | ## Recurrent Neural Networks (RNNs) 2 | Will mantain states, the output depdends on the current input as well as previous ones. 3 | 4 | This means that the input of the next step will include their own inputs + the output of the hidden layer of the previous step (the memory cell) and we will refer to it instead of ```h``` (for hidden layer) to ```s``` for State (that refers to the current state or memory) 5 | 6 | The state will add a new matrix of weights Ws 7 | 8 | In RNNs we will use Backpropagation Through Time (BPTT) 9 | 10 | It is recommended to use the unfolded model to visualize the chain rule when it comes to calculate BPTT, as it will be necessary to accumulate gradients and the unfolded model will help visualize all paths that need to be calculated before acummulating them. 11 | 12 | With only one step, we will need to update 3 Weight matrices, Wy, Ws and Wx 13 | 14 | The current state (St) depends on the current input and previous states that will be activated 15 | 16 | St = activation(Xt * Wx + St-1 * Ws) 17 | 18 | The current output will be St multiplied by the output weight matrix. 19 | 20 | yt = St * Wy 21 | or with and activation 22 | yt = activation(St * Wy) 23 | 24 | where ```activation``` is any activation function like Sigmoid, ReLu, etc. 25 | 26 | We cannot accumulate more than 8 or 10 time steps of memory with this alone, because with BPTT we will suffer of the vanishing gradient problem, to avoid this issue we will need to use Long Short-Term Memory cells (LSTMs) 27 | -------------------------------------------------------------------------------- /content/Region-Proposal-Algorithms.md: -------------------------------------------------------------------------------- 1 | ## Region Proposal Algorithms 2 | R-CNN is a Region Convolutional Neural Network 3 | 4 | It is used to produce a limited set of cropped regions to analyze ROIs (Regions of Interest) 5 | 6 | The R-CNN is the least sophisticated region-based architecture 7 | 8 | The next one is the Fast R-CNN algorithm. Fast R-CNN is about 10 times as fast to train as an R-CNN because it only creates convolutional layers once for a given image and then performs further analysis on the layer 9 | 10 | And then we have the Faster R-CNN which is the faster of all of the three, the main difference with Fast R-CNN is that it does a quick binary classification on the RoIs which will classify if there is an object on the feature map or not before actually running the classification. -------------------------------------------------------------------------------- /content/Regularization.md: -------------------------------------------------------------------------------- 1 | ## Regularization 2 | Lamda is the regularization parameter 3 | We can use L2 regularization which is also refered as Weight Decay 4 | 5 | Regularization will help prevent overfitting 6 | 7 | \lambda 8 | 9 | -------------------------------------------------------------------------------- /content/Reinforcement-Learning.md: -------------------------------------------------------------------------------- 1 | ## Reinforcement Learning 2 | 3 | **Reinforcement Learning** optimizes an agent for sparse, time delayed labels called **rewards** in an environment. 4 | 5 | **Markov Decision Processes** or *MDP* for short, are a mathematical framework for modeling decisions using states, actions and rewards. 6 | 7 | **Q Learning** is a strategy that finds the optimal action selection **policy** for any *MDP*. 8 | 9 | 10 | 11 | -------------------------------------------------------------------------------- /content/Training-Dev-and-Testing-Sets.md: -------------------------------------------------------------------------------- 1 | ## Train set, Cross Validation set (Dev set) and Test set 2 | You want to have 3 different sets: 3 | 4 | 1. Train set: data used to train your network 5 | 2. Cross Validation set: used to test algorithms while deciding on the architecture of the network 6 | 3. Test set: this is the secret set, and the network should never see this data during training, you can only use this set once the network was trained to see how it performs with new (never seen) data 7 | 8 | For small datasets (100 to 10000) the ratio is normally 9 | 1. 60% for the training set 10 | 2. 20% for the cross validation set 11 | 3. 20% for the test set 12 | 13 | For bigger datasets 14 | 1. 98% for the training set 15 | 2. 1% for the cross validation set 16 | 3. 1% for the test set 17 | 18 | -------------------------------------------------------------------------------- /content/Visualizing-Feature-Maps.md: -------------------------------------------------------------------------------- 1 | ## Visualizing Feature Maps 2 | 3 | When a Convolutional Neural Network trains, it is actually adjusting the weights of each node, then each node will become a kernel (or filter) that will be applied to an image, each filter will play a purpose, it can be to detect horizontal o vertical edges, blur out noise, detect patterns, etc. 4 | 5 | Here we can see some filters that my CNN created during training applied to an image. 6 | 7 | 8 | ![Kernels](https://github.com/george-studenko/100_Days_of_ML_Code/blob/master/resources/kernels.png "Kernels applied to an image") -------------------------------------------------------------------------------- /content/Weighted-Loss-Functions.md: -------------------------------------------------------------------------------- 1 | ## Weighted Loss Functions 2 | For image classification and localization we need to use 2 loss functions on the same network!, to calculate the predicted class we would use categorical cross entropy but to find the bounding box (which is a regression problem) we need to use something like an SME, L1 Loss or smooth L1 Loss. 3 | 4 | The way to do it is to use a weighted sum of classification and regression losses (ex. ```0.5*cross_entropy_loss + 0.5*L1_loss```); the result is a single error value with which we can do backpropagation. This does introduce a hyperparameter: the loss weights. 5 | -------------------------------------------------------------------------------- /content/YOLO.md: -------------------------------------------------------------------------------- 1 | ## YOLO (You Only Look Once) 2 | It is a real time object detection algorithm which prioritizes speed and recognition. 3 | 4 | YOLO will output the probabilities of the classes as well as the bounding box parameters (x,y,w,h) where x and y determines the coordinates of the center of the box and w and h are the width and height of the box. It will also return a probability ```pc``` which is the probability of an object existing in the current box of the grid. 5 | 6 | So a basic output of YOLO for a network trained on 3 different classes (c1, c2 and c3) might look like this 7 | ```y = [pc, c1, c2, c3, x, y, w, h]``` 8 | where pc, c1, c2 and c3 are probabilities. 9 | 10 | In a grid cell the coordinates system is as used in computer vision where x goes left to right (0 to 1) and y goes from the top to the bottom (0 to 1) so if you think of a square the top left corner coordinate will be ```(0,0)``` and the bottom right corner will be ```(1,1)``` 11 | 12 | The width and height will be a percentage between 0 and 1 this means that if ```w = 0.1``` the width of the bounding box is 10% of the width of the image and the same applies to the heigh if ```h = 0.4``` then the bounding box is 40% of the actual height of the whole image. 13 | 14 | ### Intersection Over Union (IoU) 15 | ```IoU = Area of Intersection / Area of Union of both bounding boxes``` 16 | that will give us the area where the 2 boxes overlap. The area of the union is the area of box boxes unified shape. 17 | 18 | IoU can only return values from 0 (when the boxes do not overlap at all) and 1 (a perfect match, that is when 2 boxes are overlaping on the whole area) 19 | 20 | ### Non-Maximal Suppression 21 | Will get rid of overlapping bounding boxes that do not have the maximum probability for object detection ```pc``` 22 | once that only the box with the highest ```pc``` is left all other boxes that have a high IoU with the box that is left will also be removed as they are most likely different bounding boxes for the same object. 23 | 24 | ### Anchor Boxes 25 | If there are multiple objects overlapping we will need to define different size anchor boxes, the given size will determine the shape of the box (aspect ratio) 26 | 27 | That way we will be able to detect overlapping objects with different aspect ratios, this also means that the vector will have more parameters, a set of parameters per anchor box, so for only 1 object we will have the same as we defined before 28 | ```y = [pc, c1, c2, c3, x, y, w, h]``` 29 | but for 2 anchor boxes we will have the parameters doubled 30 | ```y = [pc, c1, c2, c3, x, y, w, h, pc, c1, c2, c3, x, y, w, h]``` 31 | 32 | 33 | Here you can find the [YOLOv3 Paper](https://pjreddie.com/media/files/papers/YOLOv3.pdf) 34 | 35 | ![AI Saturdays YOLO](../resources/aisat.png) 36 | 37 | ![AI Saturdays YOLO](../resources/aisat2.png) 38 | 39 | ![AI Saturdays YOLO](../resources/aisat3.png) 40 | -------------------------------------------------------------------------------- /content/bad-constraint-matrix.md: -------------------------------------------------------------------------------- 1 | ## This is a bad Omega constraint matrix 2 | 3 | It is definitely not correct, as we can see the landmarks positions are almost all with zeros, used this to debug my code, it is a singular matrix (does not have an inverse matrix) 4 | 5 | [[ 6. 1. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0. 0.] 6 | [ 1. 6. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0.] 7 | [-1. 0. 6. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. 0. 0. 0. 0. 0. 0.] 8 | [ 0. 0. 0. 5. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. 0. 0. 0. 0. 0.] 9 | [ 0. -1. -1. 0. 6. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. -1. 0. 0. 0. 0. 0. 0.] 10 | [ 0. 0. 0. 0. 0. 4. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. -1. 0. 0. 0. 0. 0.] 11 | [ 0. 0. 0. -1. -1. 0. 4. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0.] 12 | [ 0. 0. 0. 0. 0. 0. 0. 2. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0.] 13 | [ 0. 0. 0. 0. 0. -1. -1. 0. 4. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0.] 14 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 2. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0.] 15 | [ 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 4. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0.] 16 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 2. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0.] 17 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 4. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0.] 18 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 2. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0.] 19 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 5. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. -1. 0. 0. 0. 0. 0. 0.] 20 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 3. 0. 0.-1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. -1. 0. 0. 0. 0. 0.] 21 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 6. 0.-1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. -1. 0. 0. 0. 0. 0. 0.] 22 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 4. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. -1. 0. 0. 0. 0. 0.] 23 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 6. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. -1. 0. 0. 0. 0. 0. 0.] 24 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 4. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. -1. 0. 0. 0. 0. 0.] 25 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1.-1. 0. 8. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0. 0.] 26 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 6. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0.] 27 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 8. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0. 0.] 28 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 6. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0.] 29 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 8. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0. 0.] 30 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 6. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0.] 31 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 8. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0. 0.] 32 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 6. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0.] 33 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 8. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0. 0.] 34 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 6. 0. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0.] 35 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 8. 0. -1. 0. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0. 0.] 36 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 6. 0. 0. -1. 0. 0. 0. 0. 0. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0.] 37 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 5. 0. -1. 0. 0. 0. 0. 0. 0. 0. -1. 0. -1. 0. 0. 0. 0. 0.] 38 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 3. 0. 0. -1. 0. 0. 0. 0. 0. 0. -1. 0. -1. 0. 0. 0. 0.] 39 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 6. 0. -1. 0. 0. 0. 0. -1. -1. 0. -1. 0. 0. 0. 0. 0.] 40 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 4. 0. 0. -1. 0. 0. 0. -1. -1. 0. -1. 0. 0. 0. 0.] 41 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 8. 0. -1. 0. -1. -1. -1. -1. -1. 0. 0. 0. 0. 0.] 42 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 6. 0. 0. -1. -1. -1. -1. -1. -1. 0. 0. 0. 0.] 43 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. -1. 0. 2. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] 44 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] 45 | [-1. 0. -1. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0.-1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. 0. 0. 0. 0. -1. -1. 0. 0. 13. 0. 0. 0. 0. 0. 0. 0. 0. 0.] 46 | [-1. -1. -1. -1. -1. -1. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. -1.-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. 0. 0. -1. 0. -1. -1. 0. 0. 0. 30. 0. 0. 0. 0. 0. 0. 0. 0.] 47 | [-1. -1. -1. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. 0. -1. -1. -1. -1. 0. 0. 0. 0. 29. 0. 0. 0. 0. 0. 0. 0.] 48 | [-1. -1. -1. -1. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. -1. 0.-1. 0. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. 0. -1. 0. -1. -1. -1. 0. 0. 0. 0. 0. 24. 0. 0. 0. 0. 0. 0.] 49 | [-1. -1. 0. -1. 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. -1. 0. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. 0. -1. 0. -1. -1. 0. 0. 0. 0. 0. 0. 23. 0. 0. 0. 0. 0.] 50 | [ 0. -1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. -1. 0. 0. 0. 0. 0. 0. 0. 10. 0. 0. 0. 0.] 51 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] 52 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] 53 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] 54 | [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]] -------------------------------------------------------------------------------- /content/fast-ai.md: -------------------------------------------------------------------------------- 1 | ## Fast.AI 2 | 3 | ### Update Feb-2019 4 | **Deep learning V3 Kernels** 5 | 6 | [Lesson 1 Pets](https://www.kaggle.com/hortonhearsafoo/fast-ai-v3-lesson-1) 7 | [Lesson 2 Download](https://www.kaggle.com/init27/fastai-v3-lesson-2) 8 | [Lesson 2 SGD](https://www.kaggle.com/init27/fastai-v3-lesson-2-sgd) 9 | [Lesson 3 Camvid-tiramisu](https://www.kaggle.com/hortonhearsafoo/fast-ai-v3-lesson-3-camvid-tiramisu) 10 | [Lesson 3 Camvid](https://www.kaggle.com/hortonhearsafoo/fast-ai-v3-lesson-3-camvid) 11 | [Lesson 3 Head-Pose](https://www.kaggle.com/hortonhearsafoo/fast-ai-v3-lesson-3-head-pose) 12 | [Lesson 3 Planet](https://www.kaggle.com/hortonhearsafoo/fast-ai-v3-lesson-3-planet) 13 | [Lesson 3 Tabular](https://www.kaggle.com/hortonhearsafoo/fast-ai-v3-lesson-3-imdb) 14 | [Lesson 4 Collab](https://www.kaggle.com/init27/fastai-v3-lesson4-collab) 15 | [Lesson 4 Tabular](https://www.kaggle.com/init27/fastai-v3-lesson-4-tabular) 16 | [Lesson 5 SGD-MNIST](https://www.kaggle.com/hortonhearsafoo/fast-ai-v3-lesson-5-sgd-mnist) 17 | [Lesson 6 Pets-more](https://www.kaggle.com/init27/fastai-v3-lesson-6-pets) 18 | [Rossmann data clean](https://www.kaggle.com/init27/fastai-v3-rossman-data-clean) 19 | [Lesson 6 Rossmann](https://www.kaggle.com/init27/fastai-v3-lesson-6-rossman) 20 | [Lesson 7 Human-numbers](https://www.kaggle.com/init27/fastai-v3-lesson-7-human-numbers) 21 | [Lesson 7 Resnet MNIST](https://www.kaggle.com/init27/fastai-v3-lesson-7-resnet-mnist 22 | 23 | ###Update Nov-2018 24 | The part one of the Deep Learning Course is now available with Kaggle Kernels, until last week they were not working as it was making user of the new fastai libraries which was fixed recently. 25 | 26 | Here is a list of the notebooks that you can fork and modify on your own: 27 | 28 | 1. [Lesson 1](https://www.kaggle.com/hortonhearsafoo/fast-ai-lesson-1) 29 | 2. [Lesson 2](https://www.kaggle.com/hortonhearsafoo/fast-ai-lesson-2) 30 | 3. [Lesson 3](https://www.kaggle.com/hortonhearsafoo/fast-ai-lesson-3) 31 | 4. [Lesson 4](https://www.kaggle.com/hortonhearsafoo/fast-ai-lesson-4) 32 | 5. [Lesson 5](https://www.kaggle.com/hortonhearsafoo/fast-ai-lesson-5) 33 | 6. [Lesson 6 SGD](https://www.kaggle.com/hortonhearsafoo/fast-ai-lesson-6-sgd) 34 | 7. [Lesson 6 RNN](https://www.kaggle.com/hortonhearsafoo/fast-ai-lesson-6-rnn) 35 | 8. [Lesson 7 CIFAR10](https://www.kaggle.com/hortonhearsafoo/fast-ai-lesson-7-cifar10) 36 | 9. [Lesson 7 CAM](https://www.kaggle.com/hortonhearsafoo/fast-ai-lesson-7-cam) 37 | -------------------------------------------------------------------------------- /content/index.md: -------------------------------------------------------------------------------- 1 | ## 100DaysOfMLCode Index 2 | 3 | [Attention mechanisms](content/Attention-mechanisms.md) 4 | [Batch size](content/Batch-size.md) 5 | [CNNs](content/CNN.md) 6 | [Computer Vision](content/Computer-Vision.md) 7 | [Conda](content/Conda.md) 8 | [Data Augmentation](content/Data-augmentation.md) 9 | [Defining a network structure](content/Defining-network-structure.md) 10 | [Downloading Datasets](content/Downloading-Datasets.md) 11 | [Dropout](content/Dropout.md) 12 | [Embeddings](content/Embeddings.md) 13 | [FastAI](content/fast-ai.md) 14 | [Filtered Images](resources/kernels.png) 15 | [Facia-Keypoints-Detector notes](content/Day-10.md) 16 | [GPU States](content/GPU.md) 17 | [High bias & high variance](content/High-bias-and-high-variance.md) 18 | [Hyperparameters](content/Hyperparameters.md) 19 | [Image Captioning Project Notes](content/Image-Captioning.md) 20 | [Intro to Pandas Lab](code/Intro_to_pandas.ipynb) 21 | [Jobs in Computer Vision](content/Jobs-in-Computer-Vision.md) 22 | [Layer Shapes](content/Layer-shapes.md) 23 | [Learning Rates](content/Learning-rate.md) 24 | [Localization](content/Localization.md) 25 | [LSTM cells](content/LSTM.md) 26 | [Momentum](content/Momentum.md) 27 | [Machine Learning Concepts](content/Machine-learning-concepts.md) 28 | [NLTK](content/NLTK.md) 29 | [Probability](content/Probability.md) 30 | [PyTorch Anatomy](https://freecontent.manning.com/pytorch-crash-course-part-1/) 31 | [PyTorch notes](content/PyTorch.md) 32 | [Pandas DataFrame Cheatsheet](cheatsheets/Pandas-DataFrame-Notes.pdf) 33 | [Region Proposal Algorithms](content/Region-Proposal-Algorithms.md) 34 | [Regularization](content/Regularization.md) 35 | [RNNs](content/RNN.md) 36 | [Training Dev and Testing sets](content/Training-Dev-and-Testing-Sets.md) 37 | [Visualizing Feature Maps](content/Visualizing-Feature-Maps.md) 38 | [Weighted Loss Functions](content/Weighted-Loss-Functions.md) 39 | [YOLO](content/YOLO.md) 40 | 41 | ### Code 42 | [Architecture](code/cv/models.py) 43 | [Contour detection and features](code/cv/Contour%20detection%20and%20features.ipynb) 44 | [Finding Edges and Custom Kernels](code/cv/Finding%20Edges%20and%20Custom%20Kernels.ipynb) 45 | [Guide to Mathematical Notation](code/Linear_Algebra/guide_to_mathematical_notation.ipynb) 46 | [Hough Lines](code/cv/Hough_lines.ipynb) 47 | [Matrices](code/Linear_Algebra/2_matrices_in_python.ipynb) 48 | [Matrix Addition](code/Linear_Algebra/3_matrix_addition.ipynb) 49 | [Matrix Multiplication](code/Linear_Algebra/4_matrix_multiplication.ipynb) 50 | [Robot Localization](code/Robot_Localization.ipynb) 51 | [Vectors](code/Linear_Algebra/1_vector_coding.ipynb) 52 | 53 | ### Repositories 54 | [Detecting edges live with webcam repo](https://github.com/george-studenko/Live-Sketch-with-Computer-Vision) 55 | [Facial Keypoints Detector Project](https://github.com/george-studenko/Facia-Keypoints-Detector) 56 | -------------------------------------------------------------------------------- /content/sklearn.md: -------------------------------------------------------------------------------- 1 | # Scikit Learn 2 | ## Logistic Regression 3 | Is used for classification problems, outputs probabilities if > 0.5 the data is labeled 1 else it is labeled 0. 4 | Produces a linear decision boundary. 5 | 6 | ``` 7 | from sklearn.linear_model import LogisticRegression 8 | from sklearn.model_selection import train_test_split 9 | 10 | logreg = LogisticRegression() 11 | 12 | X_train, X_text, y_train, y_test = train_test_split(X,y,test_size=0.2,random_state=42) 13 | 14 | logreg.fit(X_train,y_train) 15 | 16 | y_pred= logreg.predict(X_test) 17 | ``` 18 | 19 | ## ROC Curve 20 | Stands for Receiver Operating Characteristics curve 21 | 22 | ``` 23 | from sklearn.metrics import roc_curve 24 | 25 | # Returns the probability of a given sample being in a particular class (in this case class 1, as is the 2nd column) 26 | y_pred_prob = logreg.predict_proba(X_test)[:,1] 27 | 28 | false_positives_rate, true_positives_rate, thresholds = roc_curve(y_test, y_pred_prob) 29 | 30 | plt.plot(fpr,tpr, label = 'Logistic Regression') 31 | plt.xlabel(False Positive Rate') 32 | plt.ylabel(True Positive Rate') 33 | plt.titel('Log Reg ROC Curve') 34 | plt.show() 35 | ``` 36 | 37 | -------------------------------------------------------------------------------- /resources/Calculate-Layer-Shapes-Helper.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/resources/Calculate-Layer-Shapes-Helper.xlsx -------------------------------------------------------------------------------- /resources/aisat.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/resources/aisat.png -------------------------------------------------------------------------------- /resources/aisat2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/resources/aisat2.png -------------------------------------------------------------------------------- /resources/aisat3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/resources/aisat3.png -------------------------------------------------------------------------------- /resources/kernels.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/resources/kernels.png -------------------------------------------------------------------------------- /resources/mse.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/george-studenko/100_Days_of_ML_Code/63b200d63e5bf3cd3ddf5acdd7103dd0c074a3e8/resources/mse.png --------------------------------------------------------------------------------