├── .github └── FUNDING.yml ├── LearningPyTorch.png ├── Pytorch Tutorial ├── .ipynb_checkpoints │ └── PyTorch-IV-checkpoint.ipynb ├── PyTorch-I.ipynb ├── PyTorch-II.ipynb ├── PyTorch-III.ipynb ├── PyTorch-IV.ipynb ├── PyTorch-V.ipynb ├── PyTorch-VI.ipynb ├── README.md ├── SampleImages │ ├── cat1.jpg │ ├── cat2.jpg │ └── dog.jpg └── data │ ├── anna.txt │ └── hour-fixed.csv └── README.md /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | 3 | github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2] 4 | patreon: startingBrain 5 | open_collective: # Replace with a single Open Collective username 6 | ko_fi: mayurjain 7 | tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel 8 | community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry 9 | liberapay: # Replace with a single Liberapay username 10 | issuehunt: # Replace with a single IssueHunt username 11 | otechie: # Replace with a single Otechie username 12 | custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2'] 13 | -------------------------------------------------------------------------------- /LearningPyTorch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mayurji/Learning-PyTorch/fd36b79d3e944285286c769d94f7cbd442f6b8c9/LearningPyTorch.png -------------------------------------------------------------------------------- /Pytorch Tutorial/.ipynb_checkpoints/PyTorch-IV-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### PyTorch\n", 8 | "\n", 9 | "PyTorch is a popular open source machine learning library based on Torch library. Pytorch provides three set of libraries, i.e., torchvision, torchaudio, torchtext for Computer Vision, Audio and Text respectively.\n", 10 | "\n", 11 | "It provides two high-level features:\n", 12 | "\n", 13 | "* Tensor computation (like NumPy) with strong GPU acceleration.\n", 14 | "* Deep neural networks built on a type-based autograd system.\n", 15 | "\n", 16 | "**Topic Covered**\n", 17 | "\n", 18 | " - Handling Time Series Data\n", 19 | " - Handling Text Data" 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "metadata": {}, 25 | "source": [ 26 | "### Importing Libraries" 27 | ] 28 | }, 29 | { 30 | "cell_type": "code", 31 | "execution_count": null, 32 | "metadata": {}, 33 | "outputs": [], 34 | "source": [ 35 | "import os\n", 36 | "import numpy as np\n", 37 | "\n", 38 | "import torch\n", 39 | "\n", 40 | "from PIL import Image\n", 41 | "import matplotlib.pyplot as plt\n", 42 | "\n", 43 | "import warnings\n", 44 | "warnings.filterwarnings(\"ignore\")" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "### Working on Time Series Data\n", 52 | "\n", 53 | "We use [Bike sharing](https://www.kaggle.com/c/bike-sharing-demand/data) data from kaggle given for a two year period. Though, the objective of the dataset is to predict the count of bikes rented during each hour, but here we are going to learn how to handle time series dataset using pytorch.\n", 54 | "\n", 55 | "* Loading dataset using Numpy\n", 56 | "* Skip the column names\n", 57 | "* Convert string to float" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "# 2 Years Data\n", 67 | "bikes_numpy = np.loadtxt(\"data/hour-fixed.csv\",dtype=np.float32,\n", 68 | " delimiter=\",\",skiprows=1, converters={1: lambda x: float(x[8:10])})" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "* Convert numpy to pytorch tensor.\n", 76 | "* Check after how many elements the next(second) records starts from zero offset." 77 | ] 78 | }, 79 | { 80 | "cell_type": "code", 81 | "execution_count": null, 82 | "metadata": {}, 83 | "outputs": [], 84 | "source": [ 85 | "print(f'Total Obervations: {bikes_numpy.shape} \\n')\n", 86 | "bikes = torch.from_numpy(bikes_numpy)\n", 87 | "print(f'A Jump of 17 element in 1st dimension is required to go from one observation to next observation {bikes.stride()}')" 88 | ] 89 | }, 90 | { 91 | "cell_type": "markdown", 92 | "metadata": {}, 93 | "source": [ 94 | "### Preparing the Dataset into features\n", 95 | "\n", 96 | "* **Convert the hour based into days as 1st dim, 24 hours as 2nd dim and 17 features as 3rd dim. It is quite common to reshape the timeseries data to find seasonality or trends.**\n", 97 | "* 17520 / (2 * 365 * 24) = 1.0. (2 years, 365 Days, 24 hours - Total Number of Observations.)\n", 98 | "* The original data is presented on hour bases.\n", 99 | "* We have 2 years data, 730 days, each day has 24 hours and each hour has 17 columns/features.\n", 100 | "* We convert the hours into days, which becomes our records or rows. Each row or record is further segregated among hours our second dim.\n", 101 | "* Each row in second dim contains 17 elements." 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": {}, 108 | "outputs": [], 109 | "source": [ 110 | "daily_bikes = bikes.view(-1, 24, bikes.shape[1])\n", 111 | "print(f'Reshaped Time Series Dataset: {daily_bikes.shape} \\n')\n", 112 | "print(f'Stride for the current data structure: {daily_bikes.stride()}')" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "Stride : 24 * 17 = 408\n", 120 | "\n", 121 | "N: 730 days\n", 122 | "\n", 123 | "* Reshaping the data to make it available for training purpose.\n", 124 | "* Converting Class into One Hot encoding\n", 125 | "* Stride value (408, 17, 1), it means to jump from one day (24x17 elements) to next day records, we need to jump 408 elements (feature values)." 126 | ] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "metadata": {}, 131 | "source": [ 132 | "**Since prediction requires count of rent biker per hour, we transpose the feature dimensions.** Below we are performing a series of steps.\n", 133 | "\n", 134 | "- Selecting 24 rows.\n", 135 | "- Creating empty matrix with 24 rows for weather feature.(categorical variable)\n", 136 | "- Creating One-Hot Encoder Vector for weather feature." 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": null, 142 | "metadata": {}, 143 | "outputs": [], 144 | "source": [ 145 | "daily_bikes = daily_bikes.transpose(1, 2)\n", 146 | "print(f'After Transpose: {daily_bikes.shape, daily_bikes.stride()} \\n')\n", 147 | "first_day = bikes[:24].long()\n", 148 | "weather_onehot = torch.zeros(first_day.shape[0], 4) #4 is count of categories\n", 149 | "print(f'Weather Feature (categorical): {first_day[:,9]}')" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "metadata": {}, 155 | "source": [ 156 | "**Would encourage reader's to take up previous pytorch notebooks to understand better.**" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "print(f'One-Hot Encoded Weather Feature \\n{weather_onehot.scatter_(dim=1,index=first_day[:,9].unsqueeze(1).long() - 1,value=1.0)}')" 166 | ] 167 | }, 168 | { 169 | "cell_type": "markdown", 170 | "metadata": {}, 171 | "source": [ 172 | "* **Combining the one hot encoded matrix with feature matrix.**\n", 173 | "* **Creating Empty matrix to accomdate all the weather feature as one-hot encoder.**\n", 174 | "* **Creating Zero Matrix for Class label.**" 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": null, 180 | "metadata": {}, 181 | "outputs": [], 182 | "source": [ 183 | "\"\"\"Using cat method, we concat two matrix. Combining first 24 obsersations and \n", 184 | "its feature with weather's one-hot encoded vectors.\n", 185 | "\"\"\"\n", 186 | "\n", 187 | "combinedFeature = torch.cat((bikes[:24], weather_onehot), 1) \n", 188 | "print(f'After combining features (17 feature is turned into 21 feature after one-hot encoding): {combinedFeature.shape}\\n')\n", 189 | "\n", 190 | "daily_weather_onehot = torch.zeros(daily_bikes.shape[0], 4, daily_bikes.shape[2])\n", 191 | "print(f'Zero Matrix to accomdate weather\\'s one-hot encoding {daily_weather_onehot.shape}')" 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "metadata": {}, 197 | "source": [ 198 | "**Turning Zero Empty matrix into One-Hot Encoded feature.**" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": null, 204 | "metadata": {}, 205 | "outputs": [], 206 | "source": [ 207 | "daily_weather_onehot.scatter_(1, daily_bikes[:,9,:].long().unsqueeze(1) - 1, 1.0)" 208 | ] 209 | }, 210 | { 211 | "cell_type": "markdown", 212 | "metadata": {}, 213 | "source": [ 214 | "**Combining the existing and one-hot encoded features.**" 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "execution_count": null, 220 | "metadata": {}, 221 | "outputs": [], 222 | "source": [ 223 | "daily_bikes = torch.cat((daily_bikes, daily_weather_onehot), dim=1)\n", 224 | "print(f'Time Series Dataset: {daily_bikes.shape}')" 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "metadata": {}, 230 | "source": [ 231 | "**Keeping Temp values between [0, 1] with min-max Scaling.**" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": null, 237 | "metadata": {}, 238 | "outputs": [], 239 | "source": [ 240 | "temp = daily_bikes[:, 10, :]\n", 241 | "temp_min = torch.min(temp)\n", 242 | "temp_max = torch.max(temp)\n", 243 | "daily_bikes[:, 10, :] = ((daily_bikes[:, 10, :] - temp_min) / (temp_max - temp_min))\n", 244 | "print(f'Min-Max Scaled: {daily_bikes[:,10,:]}')" 245 | ] 246 | }, 247 | { 248 | "cell_type": "markdown", 249 | "metadata": {}, 250 | "source": [ 251 | "****Standardization of Temperature Feature****" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": null, 257 | "metadata": {}, 258 | "outputs": [], 259 | "source": [ 260 | "# temp = daily_bikes[:, 10, :]\n", 261 | "# daily_bikes[:, 10, :] = ((daily_bikes[:, 10, :] - torch.mean(temp))\n", 262 | "# / torch.std(temp))\n", 263 | "# print(f'Standardized Temperature Feature: {daily_bikes[:,10,:]}')" 264 | ] 265 | }, 266 | { 267 | "cell_type": "markdown", 268 | "metadata": {}, 269 | "source": [ 270 | "### Working with Text" 271 | ] 272 | }, 273 | { 274 | "cell_type": "markdown", 275 | "metadata": {}, 276 | "source": [ 277 | "### Character Level Conversion\n", 278 | "\n", 279 | "There are 128 Ascii character, we convert our text/letter into index or integer. Letters not present in ASCII are turned into 0." 280 | ] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": null, 285 | "metadata": {}, 286 | "outputs": [], 287 | "source": [ 288 | "with open(\"data/anna.txt\", encoding='utf-8') as f:\n", 289 | " text = f.read()" 290 | ] 291 | }, 292 | { 293 | "cell_type": "markdown", 294 | "metadata": {}, 295 | "source": [ 296 | "* Converting text document into list of lines.\n", 297 | "* Fetching One Line by Index\n", 298 | "* Creating Matrix based on length of line and total number of characters possible." 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "execution_count": null, 304 | "metadata": {}, 305 | "outputs": [], 306 | "source": [ 307 | "lines = text.split(\"\\n\")\n", 308 | "print(f'Total Number of Lines: {len(lines)} \\n')\n", 309 | "line = lines[100]\n", 310 | "letter_t = torch.zeros(len(line), 128)\n", 311 | "print(f'Number of characters in a line x Total No. of Characters Possible: {letter_t.shape}')" 312 | ] 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "metadata": {}, 317 | "source": [ 318 | "**Creating Character Matrix for a line. All ASCII characters are features, and each character from a line is separate observation. Since each observation is one character, we have that particular feature as 1 and other features as 0.**" 319 | ] 320 | }, 321 | { 322 | "cell_type": "code", 323 | "execution_count": null, 324 | "metadata": {}, 325 | "outputs": [], 326 | "source": [ 327 | "line" 328 | ] 329 | }, 330 | { 331 | "cell_type": "code", 332 | "execution_count": null, 333 | "metadata": {}, 334 | "outputs": [], 335 | "source": [ 336 | "for i, letter in enumerate(line.lower().strip()):\n", 337 | " letter_index = ord(letter) if ord(letter) < 128 else 0\n", 338 | " #print(letter_index)\n", 339 | " letter_t[i][letter_index] = 1\n", 340 | " \n", 341 | "print(f'Character Matrix: {letter_t[0]}')" 342 | ] 343 | }, 344 | { 345 | "cell_type": "code", 346 | "execution_count": null, 347 | "metadata": {}, 348 | "outputs": [], 349 | "source": [ 350 | "print(f'Column Number of the Feature In Character Matrix: {(letter_t[0]==1).nonzero(as_tuple=True)} \\n')\n", 351 | "\n", 352 | "print(f'First Character of the line: {ord(line[0])}')" 353 | ] 354 | }, 355 | { 356 | "cell_type": "markdown", 357 | "metadata": {}, 358 | "source": [ 359 | "### Word Level Conversion\n", 360 | "\n", 361 | "* Clean the text\n", 362 | "* Sorting the text\n", 363 | "* Mapping the words into integer" 364 | ] 365 | }, 366 | { 367 | "cell_type": "code", 368 | "execution_count": null, 369 | "metadata": {}, 370 | "outputs": [], 371 | "source": [ 372 | "def clean_words(input_str):\n", 373 | " punctuation = '.,;:\"!?”“_-'\n", 374 | " word_list = input_str.lower().replace('\\n',' ').split()\n", 375 | " word_list = [word.strip(punctuation) for word in word_list]\n", 376 | " return word_list" 377 | ] 378 | }, 379 | { 380 | "cell_type": "markdown", 381 | "metadata": {}, 382 | "source": [ 383 | "**Cleaning and Tokenization**" 384 | ] 385 | }, 386 | { 387 | "cell_type": "code", 388 | "execution_count": null, 389 | "metadata": {}, 390 | "outputs": [], 391 | "source": [ 392 | "words_in_line = clean_words(line)\n", 393 | "line, words_in_line" 394 | ] 395 | }, 396 | { 397 | "cell_type": "markdown", 398 | "metadata": {}, 399 | "source": [ 400 | "15K+ words are present in the text after cleaning. We are mapping each word with an unique id." 401 | ] 402 | }, 403 | { 404 | "cell_type": "code", 405 | "execution_count": null, 406 | "metadata": {}, 407 | "outputs": [], 408 | "source": [ 409 | "word_list = sorted(set(clean_words(text)))\n", 410 | "word2index_dict = {word: i for (i, word) in enumerate(word_list)}\n", 411 | "print(f'Total Number Of Unique Words: {len(word2index_dict.keys())}')" 412 | ] 413 | }, 414 | { 415 | "cell_type": "markdown", 416 | "metadata": {}, 417 | "source": [ 418 | "* Creating a Matrix where columns size is length of all words and each row is length of each document." 419 | ] 420 | }, 421 | { 422 | "cell_type": "code", 423 | "execution_count": null, 424 | "metadata": {}, 425 | "outputs": [], 426 | "source": [ 427 | "word_t = torch.zeros(len(words_in_line), len(word2index_dict))\n", 428 | "word_t.shape" 429 | ] 430 | }, 431 | { 432 | "cell_type": "markdown", 433 | "metadata": {}, 434 | "source": [ 435 | "**Index of each word on the column axis**" 436 | ] 437 | }, 438 | { 439 | "cell_type": "code", 440 | "execution_count": null, 441 | "metadata": {}, 442 | "outputs": [], 443 | "source": [ 444 | "for i, word in enumerate(words_in_line):\n", 445 | " word_index = word2index_dict[word]\n", 446 | " word_t[i][word_index] = 1\n", 447 | " print('{:2} {:4} {}'.format(i, word_index, word))" 448 | ] 449 | }, 450 | { 451 | "cell_type": "code", 452 | "execution_count": null, 453 | "metadata": {}, 454 | "outputs": [], 455 | "source": [ 456 | "print(f'Word Matrix: \\n{word_t}')" 457 | ] 458 | }, 459 | { 460 | "cell_type": "markdown", 461 | "metadata": {}, 462 | "source": [ 463 | "### Thanks For Reading. For Feedback, reach out on Github. Please don't spam." 464 | ] 465 | } 466 | ], 467 | "metadata": { 468 | "interpreter": { 469 | "hash": "dca0ade3e726a953b501b15e8e990130d2b7799f14cfd9f4271676035ebe5511" 470 | }, 471 | "kernelspec": { 472 | "display_name": "Python 3", 473 | "language": "python", 474 | "name": "python3" 475 | }, 476 | "language_info": { 477 | "codemirror_mode": { 478 | "name": "ipython", 479 | "version": 3 480 | }, 481 | "file_extension": ".py", 482 | "mimetype": "text/x-python", 483 | "name": "python", 484 | "nbconvert_exporter": "python", 485 | "pygments_lexer": "ipython3", 486 | "version": "3.7.6" 487 | } 488 | }, 489 | "nbformat": 4, 490 | "nbformat_minor": 4 491 | } 492 | -------------------------------------------------------------------------------- /Pytorch Tutorial/PyTorch-I.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### PyTorch\n", 8 | "\n", 9 | "PyTorch is a popular open source machine learning library based on Torch library. Pytorch provides three set of libraries, i.e., torchvision, torchaudio, torchtext for Computer Vision, Audio and Text respectively. \n", 10 | "\n", 11 | "It provides two high-level features:\n", 12 | "\n", 13 | "* Tensor computation (like NumPy) with strong GPU acceleration\n", 14 | "* Deep neural networks built on a type-based autograd system\n", 15 | "\n", 16 | "We'll not dwell on history of deep learning libraries like talking about caffee, tensorflow, maxnet, etc. We'll directly jump on to PyTorch and work on it.\n", 17 | "\n", 18 | "In this notebook, We cover basics of pytorch tensors. Things will get more interesting with every next notebook.\n", 19 | "\n", 20 | "### Topics Covered\n", 21 | "\n", 22 | "* Image is a tensor?\n", 23 | "* Creating Scalar, Vector, and Tensor.\n", 24 | "* Mathematical operation on tensors.\n", 25 | "* Convert vector to matrices.\n", 26 | "* Accessing elements in a tensors.\n", 27 | "* Selecting rows/columns of a tensors.\n", 28 | "* Elementwise and matrix multiplication between tensors.\n", 29 | "* Combining Elements in a tensor.\n", 30 | "* Dot Product of a tensor.\n", 31 | "* Matrix-Vector Multiplication.\n", 32 | "\n", 33 | "### Note:\n", 34 | "\n", 35 | "Used Torch 1.8.1 " 36 | ] 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "metadata": {}, 41 | "source": [ 42 | "### Installation" 43 | ] 44 | }, 45 | { 46 | "cell_type": "code", 47 | "execution_count": null, 48 | "metadata": {}, 49 | "outputs": [], 50 | "source": [ 51 | "#!pip install torch==1.8.1" 52 | ] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": {}, 57 | "source": [ 58 | "### Importing Libraries" 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": null, 64 | "metadata": {}, 65 | "outputs": [], 66 | "source": [ 67 | "import os\n", 68 | "import numpy as np\n", 69 | "\n", 70 | "import torch\n", 71 | "\n", 72 | "from PIL import Image\n", 73 | "import matplotlib.pyplot as plt" 74 | ] 75 | }, 76 | { 77 | "cell_type": "markdown", 78 | "metadata": {}, 79 | "source": [ 80 | "## Image is a tensor?\n", 81 | "\n", 82 | "Before we start working with pytorch tensor, we'll see a sample of image and find what it consists of." 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": null, 88 | "metadata": {}, 89 | "outputs": [], 90 | "source": [ 91 | "Image.open('SampleImages/dog.jpg')" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "image = Image.open('SampleImages/dog.jpg')\n", 101 | "print(f'Image Size with Channels {np.asarray(image).shape}')" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "metadata": {}, 107 | "source": [ 108 | "There are some series of steps done in the above cell.\n", 109 | "\n", 110 | "* First, we are loading the dog's image using PIL Library.\n", 111 | "* Second, we are converting the PIL object into array using Numpy.\n", 112 | "* Then, we are finding the shape of the image. Shape refers to width, height, and channels. \n", 113 | "\n", 114 | "From an image shape, we can notice:\n", 115 | " - 1200px Width\n", 116 | " - 602px Height\n", 117 | " - 3 Channel (RGB color channel)\n", 118 | "\n", 119 | "px - pixels" 120 | ] 121 | }, 122 | { 123 | "cell_type": "markdown", 124 | "metadata": {}, 125 | "source": [ 126 | "**Channels of an Image**\n", 127 | "\n", 128 | "Below we can see all the three channels of the image separately, and each image is of dimension (1, width, height)." 129 | ] 130 | }, 131 | { 132 | "cell_type": "code", 133 | "execution_count": null, 134 | "metadata": {}, 135 | "outputs": [], 136 | "source": [ 137 | "\"\"\"Plotting each channel of the image separately\"\"\"\n", 138 | "\n", 139 | "figure, plots = plt.subplots(ncols=3, nrows=1, figsize=(15, 15))\n", 140 | "for i, subplot in zip(range(3), plots):\n", 141 | " subplot.imshow(np.asarray(image)[:, :, i])\n", 142 | " subplot.set_axis_off()\n", 143 | "plt.show()" 144 | ] 145 | }, 146 | { 147 | "cell_type": "code", 148 | "execution_count": null, 149 | "metadata": {}, 150 | "outputs": [], 151 | "source": [ 152 | "\"\"\"\n", 153 | "Using numpy's rollaxis, we are shifting the channel dimension to the first dimension.\n", 154 | "Remember this variable as we'll use it later for manipulating images.\n", 155 | "\"\"\"\n", 156 | "np_image = np.rollaxis(np.asarray(image), 2, 0)\n", 157 | "#np_image" 158 | ] 159 | }, 160 | { 161 | "cell_type": "markdown", 162 | "metadata": {}, 163 | "source": [ 164 | "On uncommenting np_image above, we'll see a tensor of numbers. Each image is represented by a set of numbers aligned in matrix format with value ranging between 0 to 255. Where 0 refers to black and 255 refers to white. \n", 165 | "\n", 166 | "**From here onwards, we'll create tensor similar to image tensor using PyTorch and work with it.**\n", 167 | "\n", 168 | "### Creating a Scalar, Vector, and Tensor.\n", 169 | "\n", 170 | "Scalar: It is a zero rank tensor, simply referred as a numeric value. It is also known as tensor with no axis.\n", 171 | "\n", 172 | "Vector: It is a first rank tensor, simply referred as a list of numeric values. It is also known as tensor with one axis.\n", 173 | "\n", 174 | "Tensor/Matrix: It is a tensor with rank greater than one. It is also known as tensor with more than one axis." 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": null, 180 | "metadata": { 181 | "jupyter": { 182 | "outputs_hidden": true 183 | } 184 | }, 185 | "outputs": [], 186 | "source": [ 187 | "\"\"\"Scalar\"\"\"\n", 188 | "scalar = torch.tensor(3.14)\n", 189 | "print(f'Scalar Declaration: {scalar.shape}\\n')\n", 190 | "\n", 191 | "\"\"\"Vector\"\"\"\n", 192 | "vector = torch.tensor([0.2126, 0.7152, 0.0722])\n", 193 | "print(f'Vector Declaration: {vector.shape}\\n')\n", 194 | "\n", 195 | "\"\"\"Tensor\"\"\"\n", 196 | "tensor = torch.randn(3, 5, 5) #In analogy with Image, we can think of [channels, rows, columns]\n", 197 | "print(f'Tensor Declaration: {tensor.shape}\\n')\n", 198 | "\n", 199 | "\"\"\"\n", 200 | "If we compare a tensor with an image tensor, an image consists of CxWxH, but to keep multiple \n", 201 | "images inside a tensor, we must add a separate dimension called as Batch dimension.\n", 202 | "Creating a tensor with two images with dimension as follows (3 channel, 5 width, 5 height).\n", 203 | "\"\"\"\n", 204 | "\n", 205 | "batch = torch.randn(2, 3, 5, 5) #[batch, channels, rows, columns]\n", 206 | "print(f'Adding batch dimension: {batch.shape}')" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": {}, 212 | "source": [ 213 | "## Playing With Tensors" 214 | ] 215 | }, 216 | { 217 | "cell_type": "markdown", 218 | "metadata": {}, 219 | "source": [ 220 | "**Scalar Multiplication**" 221 | ] 222 | }, 223 | { 224 | "cell_type": "code", 225 | "execution_count": null, 226 | "metadata": { 227 | "jupyter": { 228 | "outputs_hidden": true 229 | } 230 | }, 231 | "outputs": [], 232 | "source": [ 233 | "x = torch.tensor([0.3])\n", 234 | "y = torch.tensor([1.5])\n", 235 | "\n", 236 | "print(f'Multiplying two scalar values x and y: {(x * y).item()}')" 237 | ] 238 | }, 239 | { 240 | "cell_type": "markdown", 241 | "metadata": {}, 242 | "source": [ 243 | "**Creating a Vector using arange**" 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": null, 249 | "metadata": { 250 | "jupyter": { 251 | "outputs_hidden": true 252 | } 253 | }, 254 | "outputs": [], 255 | "source": [ 256 | "vector = torch.arange(20) \n", 257 | "print(f'Vector of size 20: {vector}')" 258 | ] 259 | }, 260 | { 261 | "cell_type": "markdown", 262 | "metadata": {}, 263 | "source": [ 264 | "**Converting a Vector into matrix using reshape.**" 265 | ] 266 | }, 267 | { 268 | "cell_type": "code", 269 | "execution_count": null, 270 | "metadata": { 271 | "jupyter": { 272 | "outputs_hidden": true 273 | } 274 | }, 275 | "outputs": [], 276 | "source": [ 277 | "matrix = vector.reshape(4, 5)\n", 278 | "print(f'Matrix of shape 4x5: \\n{matrix}')" 279 | ] 280 | }, 281 | { 282 | "cell_type": "markdown", 283 | "metadata": {}, 284 | "source": [ 285 | "**Transpose of a matrix.**" 286 | ] 287 | }, 288 | { 289 | "cell_type": "code", 290 | "execution_count": null, 291 | "metadata": { 292 | "jupyter": { 293 | "outputs_hidden": true 294 | } 295 | }, 296 | "outputs": [], 297 | "source": [ 298 | "print(f'Transpose of a matrix: \\n{matrix.T}') # Shape 4x5 becomes 5x4." 299 | ] 300 | }, 301 | { 302 | "cell_type": "markdown", 303 | "metadata": {}, 304 | "source": [ 305 | "**Accessing elements of a tensor.**" 306 | ] 307 | }, 308 | { 309 | "cell_type": "code", 310 | "execution_count": null, 311 | "metadata": { 312 | "jupyter": { 313 | "outputs_hidden": true 314 | } 315 | }, 316 | "outputs": [], 317 | "source": [ 318 | "print(f'Accessing second row and third column: {matrix[1, 2]}') #index starts from zero.\n", 319 | "print(f'Accessing first row and first column: {matrix[0, 0]}') #index starts from zero.\n", 320 | "print(f'Accessing last row and last column: {matrix[3, 4]}') #index starts from zero." 321 | ] 322 | }, 323 | { 324 | "cell_type": "markdown", 325 | "metadata": {}, 326 | "source": [ 327 | "**Selecting row/column**" 328 | ] 329 | }, 330 | { 331 | "cell_type": "code", 332 | "execution_count": null, 333 | "metadata": { 334 | "jupyter": { 335 | "outputs_hidden": true 336 | } 337 | }, 338 | "outputs": [], 339 | "source": [ 340 | "print(f'Selecting second Row completely: \\n{matrix[1,:]}\\n')\n", 341 | "print(f'Selecting second Column completely: \\n{matrix[:,1]}')" 342 | ] 343 | }, 344 | { 345 | "cell_type": "markdown", 346 | "metadata": {}, 347 | "source": [ 348 | "**Creating complex matrix with arbitrary no. of axes.**" 349 | ] 350 | }, 351 | { 352 | "cell_type": "code", 353 | "execution_count": null, 354 | "metadata": {}, 355 | "outputs": [], 356 | "source": [ 357 | "complexMatrix = torch.arange(150).reshape(2, 3, 5, 5)\n", 358 | "#complexMatrix" 359 | ] 360 | }, 361 | { 362 | "cell_type": "markdown", 363 | "metadata": {}, 364 | "source": [ 365 | "**Creating a Copy of a matrix using Clone method.**" 366 | ] 367 | }, 368 | { 369 | "cell_type": "code", 370 | "execution_count": null, 371 | "metadata": { 372 | "jupyter": { 373 | "outputs_hidden": true 374 | } 375 | }, 376 | "outputs": [], 377 | "source": [ 378 | "matrixTwo = matrix.clone()\n", 379 | "print(f'Cloned Matrix: \\n{matrixTwo}')" 380 | ] 381 | }, 382 | { 383 | "cell_type": "markdown", 384 | "metadata": {}, 385 | "source": [ 386 | "**Elementwise Multiplication of matrices. Elementwise multiplication of \n", 387 | "two matrices is called as their Hadamard product (math notation ⊙).**" 388 | ] 389 | }, 390 | { 391 | "cell_type": "code", 392 | "execution_count": null, 393 | "metadata": { 394 | "jupyter": { 395 | "outputs_hidden": true 396 | } 397 | }, 398 | "outputs": [], 399 | "source": [ 400 | "print(f'Elementwise Multiplication of two matrices: \\n{matrix * matrixTwo}')" 401 | ] 402 | }, 403 | { 404 | "cell_type": "markdown", 405 | "metadata": {}, 406 | "source": [ 407 | "**Combining Elements along axes.**" 408 | ] 409 | }, 410 | { 411 | "cell_type": "code", 412 | "execution_count": null, 413 | "metadata": { 414 | "jupyter": { 415 | "outputs_hidden": true 416 | } 417 | }, 418 | "outputs": [], 419 | "source": [ 420 | "print(f'Given Matrix and its shape: \\n{matrix}, {matrix.shape} \\n')\n", 421 | "print(f'Sum of all elements of the given matrix: {matrix.sum()} \\n')\n", 422 | "print(f'Adding along row axis and its shape: \\n{matrix.sum(axis=1)}, {matrix.sum(axis=1).shape}\\n')\n", 423 | "print(f'Adding along column axis and its shape: \\n{matrix.sum(axis=0)}, {matrix.sum(axis=0).shape}')" 424 | ] 425 | }, 426 | { 427 | "cell_type": "markdown", 428 | "metadata": {}, 429 | "source": [ 430 | "**In the above summation along the axis, we lose the dimension about the matrix. \n", 431 | "To retain the dimension information, we can use keep dims parameter.**" 432 | ] 433 | }, 434 | { 435 | "cell_type": "code", 436 | "execution_count": null, 437 | "metadata": { 438 | "jupyter": { 439 | "outputs_hidden": true 440 | } 441 | }, 442 | "outputs": [], 443 | "source": [ 444 | "print(f'Adding along row axis and its shape: \\n{matrix.sum(axis=1, keepdims=True)}, {matrix.sum(axis=1, keepdims=True).shape}\\n')\n", 445 | "print(f'Adding along column axis and its shape: \\n{matrix.sum(axis=0, keepdims=True)}, {matrix.sum(axis=0, keepdims=True).shape}')" 446 | ] 447 | }, 448 | { 449 | "cell_type": "markdown", 450 | "metadata": {}, 451 | "source": [ 452 | "**Performing matrix multiplication using matmul() function.\n", 453 | "Reason for .T is to match the dimension, (4x5)x(5x4).**" 454 | ] 455 | }, 456 | { 457 | "cell_type": "code", 458 | "execution_count": null, 459 | "metadata": {}, 460 | "outputs": [], 461 | "source": [ 462 | "print(f'Matrix Multiplication: \\n{torch.matmul(matrix, matrixTwo.T)}')" 463 | ] 464 | }, 465 | { 466 | "cell_type": "markdown", 467 | "metadata": {}, 468 | "source": [ 469 | "**Dot Product: is a sum over the products of the elements at the same position.**" 470 | ] 471 | }, 472 | { 473 | "cell_type": "code", 474 | "execution_count": null, 475 | "metadata": {}, 476 | "outputs": [], 477 | "source": [ 478 | "x = torch.arange(4, dtype = torch.float32) # declaring tensor as float type values.\n", 479 | "y = torch.ones(4,dtype=torch.float32) #Using ones function, we create a vector of ones.\n", 480 | "\n", 481 | "print(f' x.y = {torch.dot(x, y)}')" 482 | ] 483 | }, 484 | { 485 | "cell_type": "markdown", 486 | "metadata": {}, 487 | "source": [ 488 | "**Matrix-Vector Multiplication**" 489 | ] 490 | }, 491 | { 492 | "cell_type": "code", 493 | "execution_count": null, 494 | "metadata": {}, 495 | "outputs": [], 496 | "source": [ 497 | "A = torch.arange(20).reshape(4,5)\n", 498 | "x = torch.arange(5)\n", 499 | "\n", 500 | "print(f'Matrix-Vector Multiplication: \\n{torch.mv(A,x)}')" 501 | ] 502 | }, 503 | { 504 | "cell_type": "markdown", 505 | "metadata": {}, 506 | "source": [ 507 | "### Thanks For Reading. For Feedback, reach out on Github. Please don't spam." 508 | ] 509 | } 510 | ], 511 | "metadata": { 512 | "interpreter": { 513 | "hash": "dca0ade3e726a953b501b15e8e990130d2b7799f14cfd9f4271676035ebe5511" 514 | }, 515 | "kernelspec": { 516 | "display_name": "Python 3.7.6 64-bit ('base': conda)", 517 | "name": "python3" 518 | }, 519 | "language_info": { 520 | "name": "python", 521 | "version": "" 522 | } 523 | }, 524 | "nbformat": 4, 525 | "nbformat_minor": 4 526 | } -------------------------------------------------------------------------------- /Pytorch Tutorial/PyTorch-II.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### PyTorch\n", 8 | "\n", 9 | "PyTorch is a popular open source machine learning library based on Torch library. Pytorch provides three set of libraries, i.e., torchvision, torchaudio, torchtext for Computer Vision, Audio and Text respectively.\n", 10 | "\n", 11 | "It provides two high-level features:\n", 12 | "\n", 13 | "* Tensor computation (like NumPy) with strong GPU acceleration\n", 14 | "* Deep neural networks built on a type-based autograd system\n", 15 | "\n", 16 | "### Topics Covered\n", 17 | "\n", 18 | "* **Named Tensors**\n", 19 | "\n", 20 | " - How to declare named dimensions?\n", 21 | " - Manipulating using named dimensions.\n", 22 | " - Renaming dimensions.\n", 23 | "\n", 24 | "* **Tensor Storage**\n", 25 | "\n", 26 | " - View storage object of a tensor.\n", 27 | " - Accessing and modifying a tensor using storage location.\n", 28 | " - Storage Offset.\n", 29 | " \n", 30 | "* **Stride**\n", 31 | "\n", 32 | " - Find the stride of a tensor.\n", 33 | " - Storage and Stride.\n", 34 | " - Accessing elements using stride and index.\n", 35 | " - Comparing index and stride based element." 36 | ] 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "metadata": {}, 41 | "source": [ 42 | "### Importing Libraries" 43 | ] 44 | }, 45 | { 46 | "cell_type": "code", 47 | "execution_count": null, 48 | "metadata": {}, 49 | "outputs": [], 50 | "source": [ 51 | "import os\n", 52 | "import numpy as np\n", 53 | "\n", 54 | "import torch\n", 55 | "\n", 56 | "from PIL import Image\n", 57 | "import matplotlib.pyplot as plt\n", 58 | "\n", 59 | "import warnings\n", 60 | "warnings.filterwarnings(\"ignore\")" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "### Importing Image" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "image = Image.open('SampleImages/dog.jpg')\n", 77 | "np_image = np.rollaxis(np.asarray(image), 2, 0)\n", 78 | "image" 79 | ] 80 | }, 81 | { 82 | "cell_type": "markdown", 83 | "metadata": {}, 84 | "source": [ 85 | "### Named Tensor\n", 86 | "\n", 87 | "* It assigns name to a dimension, and manipulate the tensors using those names." 88 | ] 89 | }, 90 | { 91 | "cell_type": "markdown", 92 | "metadata": {}, 93 | "source": [ 94 | "**How To Declare Named Dimensions?**" 95 | ] 96 | }, 97 | { 98 | "cell_type": "code", 99 | "execution_count": null, 100 | "metadata": {}, 101 | "outputs": [], 102 | "source": [ 103 | "\"\"\"Consider the below tensor as weights with three values.\"\"\"\n", 104 | "\n", 105 | "namedTensorWgts = torch.tensor([0.2126, 0.7152, 0.0722], names=['channels'])\n", 106 | "print(f'Named Tensor: {namedTensorWgts}')" 107 | ] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "execution_count": null, 112 | "metadata": {}, 113 | "outputs": [], 114 | "source": [ 115 | "\"\"\"Converting Image Array into PyTorch Tensor.\"\"\"\n", 116 | "torchImage = torch.from_numpy(np_image)" 117 | ] 118 | }, 119 | { 120 | "cell_type": "markdown", 121 | "metadata": {}, 122 | "source": [ 123 | "Below, we are assigning first dim as channel name, second dim as row and third dim as columns. It assigns name from right to left because the batch dimension is assigned as None. Batch Dimension varies according to users preference." 124 | ] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "execution_count": null, 129 | "metadata": {}, 130 | "outputs": [], 131 | "source": [ 132 | "torchNamed = torchImage.refine_names(..., 'channels', 'rows', 'columns')\n", 133 | "print(\"Named Image Tensor:\", torchNamed.shape, torchNamed.names)" 134 | ] 135 | }, 136 | { 137 | "cell_type": "code", 138 | "execution_count": null, 139 | "metadata": {}, 140 | "outputs": [], 141 | "source": [ 142 | "weightsAligned = namedTensorWgts.align_as(torchNamed)\n", 143 | "print(f'Aligning weight as image dimensions {weightsAligned.shape}, {weightsAligned.names}')" 144 | ] 145 | }, 146 | { 147 | "cell_type": "markdown", 148 | "metadata": {}, 149 | "source": [ 150 | "**Manipulating Using Named Dimensions.**" 151 | ] 152 | }, 153 | { 154 | "cell_type": "code", 155 | "execution_count": null, 156 | "metadata": {}, 157 | "outputs": [], 158 | "source": [ 159 | "\"\"\"Combining Channels to turn RGB into Gray Scale.\"\"\"\n", 160 | "\n", 161 | "grayNamed = (torchNamed * weightsAligned).sum('channels')" 162 | ] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "execution_count": null, 167 | "metadata": {}, 168 | "outputs": [], 169 | "source": [ 170 | "plt.imshow(grayNamed, cmap='gray')" 171 | ] 172 | }, 173 | { 174 | "cell_type": "markdown", 175 | "metadata": {}, 176 | "source": [ 177 | "**Renaming the dimensions**" 178 | ] 179 | }, 180 | { 181 | "cell_type": "code", 182 | "execution_count": null, 183 | "metadata": {}, 184 | "outputs": [], 185 | "source": [ 186 | "grayPlain = grayNamed.rename(None)\n", 187 | "print(f'Renaming Gray Tensor as None: {grayPlain.shape}, {grayPlain.names}')" 188 | ] 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "metadata": {}, 193 | "source": [ 194 | "### Tensor Storage\n", 195 | "\n", 196 | "Tensor Storage makes pytorch quite fast. It assigns block of continous memory for each tensor (matrix or vector) and whenever operation like dimension changes are done, it happens within the same memory without assigning to any other block of memory." 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "**View Storage Object Of A Tensor**" 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": null, 209 | "metadata": {}, 210 | "outputs": [], 211 | "source": [ 212 | "points = torch.tensor([[4.0, 1.0], [5.0, 3.0], [2.0, 1.0]])\n", 213 | "print(f'No. Of Elements in a Tensor: {points.storage().size()}')" 214 | ] 215 | }, 216 | { 217 | "cell_type": "markdown", 218 | "metadata": {}, 219 | "source": [ 220 | "**Accessing and modifying value of a tensor using storage location.**" 221 | ] 222 | }, 223 | { 224 | "cell_type": "code", 225 | "execution_count": null, 226 | "metadata": {}, 227 | "outputs": [], 228 | "source": [ 229 | "points = torch.tensor([[4.0, 1.0], [5.0, 3.0], [2.0, 1.0]])\n", 230 | "points_storage = points.storage()\n", 231 | "points_storage[0] = 2.0\n", 232 | "print(f'Changing Value at Position 0 Of a Tensor from 4 to 2: \\n{points}')" 233 | ] 234 | }, 235 | { 236 | "cell_type": "markdown", 237 | "metadata": {}, 238 | "source": [ 239 | "**Storage Offset**\n", 240 | "\n", 241 | "Tensor's offset in the underlying storage is in terms of number of storage elements (not bytes)." 242 | ] 243 | }, 244 | { 245 | "cell_type": "code", 246 | "execution_count": null, 247 | "metadata": {}, 248 | "outputs": [], 249 | "source": [ 250 | "points = torch.tensor([[4.0, 1.0], [5.0, 3.0], [2.0, 1.0]])\n", 251 | "print(f\"Tensor's Offset: {points.storage_offset()} \\n\")\n", 252 | "\n", 253 | "first_point = points[0]\n", 254 | "print(f'Offset Of First Element Of The Tensor: {points[0]}')\n", 255 | "print(f\"First Element's Offset {first_point.storage_offset()} \\n\")\n", 256 | "\n", 257 | "second_point = points[1]\n", 258 | "print(f\"Offset Of Second Element Of The Tensor: {points[1]}\")\n", 259 | "print(f\"Second Element's Offset {second_point.storage_offset()}\")" 260 | ] 261 | }, 262 | { 263 | "cell_type": "markdown", 264 | "metadata": {}, 265 | "source": [ 266 | "Create a Tensor 'a' and reference it on 'b' tensor. Check if the storage location is same for both a and b." 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": null, 272 | "metadata": {}, 273 | "outputs": [], 274 | "source": [ 275 | "# Creating 'a' Tensor\n", 276 | "a = torch.arange(9).view(3,3)\n", 277 | "b = a.view(3,3)" 278 | ] 279 | }, 280 | { 281 | "cell_type": "code", 282 | "execution_count": null, 283 | "metadata": {}, 284 | "outputs": [], 285 | "source": [ 286 | "#id is the unique id of a Specific Object\n", 287 | "id(a.storage)==id(b.storage)" 288 | ] 289 | }, 290 | { 291 | "cell_type": "markdown", 292 | "metadata": {}, 293 | "source": [ 294 | "### Stride \n", 295 | "\n", 296 | "It is the jump necessary to go from one element to the next one in the\n", 297 | "specified dimension `dim`. A tuple of all strides is returned when no\n", 298 | "argument is passed in. Otherwise, an integer value is returned as the stride in\n", 299 | "the particular dimension.\n", 300 | "\n", 301 | "**Find the Stride of a Tensor**\n", 302 | "\n", 303 | "In the below example, 'a' is a single dimension tensor. So every element is located next to each other by distance of 1 and their storage location is stored next to each other." 304 | ] 305 | }, 306 | { 307 | "cell_type": "code", 308 | "execution_count": null, 309 | "metadata": {}, 310 | "outputs": [], 311 | "source": [ 312 | "a = list(range(9))\n", 313 | "a = torch.tensor(a)\n", 314 | "print(a.stride(dim=0))" 315 | ] 316 | }, 317 | { 318 | "cell_type": "code", 319 | "execution_count": null, 320 | "metadata": {}, 321 | "outputs": [], 322 | "source": [ 323 | "\"\"\"Convert Vector Into Matrix\"\"\"\n", 324 | "a = a.view(3,3)\n", 325 | "print(a)" 326 | ] 327 | }, 328 | { 329 | "cell_type": "markdown", 330 | "metadata": {}, 331 | "source": [ 332 | "**Storage and Stride**" 333 | ] 334 | }, 335 | { 336 | "cell_type": "code", 337 | "execution_count": null, 338 | "metadata": {}, 339 | "outputs": [], 340 | "source": [ 341 | "storage = a.storage()\n", 342 | "print(f'Storage Object of \\'a\\' Tensor:\\n{storage}')" 343 | ] 344 | }, 345 | { 346 | "cell_type": "markdown", 347 | "metadata": {}, 348 | "source": [ 349 | "For 'a' tensor, the stride at dim=0 is 3 because at dim=0, each block occupies 3 elements, so the next block(sub-vector) is present at 3 storage places from first block." 350 | ] 351 | }, 352 | { 353 | "cell_type": "code", 354 | "execution_count": null, 355 | "metadata": {}, 356 | "outputs": [], 357 | "source": [ 358 | "print(a.stride(dim=0))" 359 | ] 360 | }, 361 | { 362 | "cell_type": "markdown", 363 | "metadata": {}, 364 | "source": [ 365 | "For 'a' tensor, the stride at dim=1 is 1 because each element is present next to each other with storage places next to each other." 366 | ] 367 | }, 368 | { 369 | "cell_type": "code", 370 | "execution_count": null, 371 | "metadata": { 372 | "jupyter": { 373 | "outputs_hidden": true 374 | } 375 | }, 376 | "outputs": [], 377 | "source": [ 378 | "print(f'Elements in Ist Dimension are present next to each other: {a.stride(dim=1)}')" 379 | ] 380 | }, 381 | { 382 | "cell_type": "markdown", 383 | "metadata": {}, 384 | "source": [ 385 | "**Accessing Elements Using Stride and Index**" 386 | ] 387 | }, 388 | { 389 | "cell_type": "code", 390 | "execution_count": null, 391 | "metadata": {}, 392 | "outputs": [], 393 | "source": [ 394 | "idx = (2, 1)\n", 395 | "item = a[idx].item()\n", 396 | "print(f'Element to find: {item}')" 397 | ] 398 | }, 399 | { 400 | "cell_type": "code", 401 | "execution_count": null, 402 | "metadata": {}, 403 | "outputs": [], 404 | "source": [ 405 | "stride = a.stride()\n", 406 | "print(f'Stride at Each Dimension: {stride}')" 407 | ] 408 | }, 409 | { 410 | "cell_type": "markdown", 411 | "metadata": {}, 412 | "source": [ 413 | "Comparing Index and Stride Based Item." 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "execution_count": null, 419 | "metadata": {}, 420 | "outputs": [], 421 | "source": [ 422 | "loc = idx[0]*stride[0] + idx[1]*stride[1]\n", 423 | "print(f'Is Element at indexed location is same as stride based location: {storage[loc]==item}')" 424 | ] 425 | }, 426 | { 427 | "cell_type": "markdown", 428 | "metadata": {}, 429 | "source": [ 430 | "### Thanks For Reading. For Feedback, reach out on Github. Please don't spam." 431 | ] 432 | } 433 | ], 434 | "metadata": { 435 | "interpreter": { 436 | "hash": "dca0ade3e726a953b501b15e8e990130d2b7799f14cfd9f4271676035ebe5511" 437 | }, 438 | "kernelspec": { 439 | "display_name": "Python 3.7.6 64-bit ('base': conda)", 440 | "name": "python3" 441 | }, 442 | "language_info": { 443 | "name": "python", 444 | "version": "" 445 | } 446 | }, 447 | "nbformat": 4, 448 | "nbformat_minor": 4 449 | } -------------------------------------------------------------------------------- /Pytorch Tutorial/PyTorch-III.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### PyTorch\n", 8 | "\n", 9 | "PyTorch is a popular open source machine learning library based on Torch library. Pytorch provides three set of libraries, i.e., torchvision, torchaudio, torchtext for Computer Vision, Audio and Text respectively.\n", 10 | "\n", 11 | "It provides two high-level features:\n", 12 | "\n", 13 | "* Tensor computation (like NumPy) with strong GPU acceleration\n", 14 | "* Deep neural networks built on a type-based autograd system\n", 15 | "\n", 16 | "### Topics Covered\n", 17 | "\n", 18 | "- Introducing Batch Dimension.\n", 19 | "- Load Batch Of Images (Not Recommended Approach).\n", 20 | "- Normalization\n", 21 | " - Resize.\n", 22 | " - Standardization.\n", 23 | " - Plotting.\n", 24 | "- Creating One-Hot Encoding.\n", 25 | " - Convert Vector Into One-Hot Encoded Matrix.\n", 26 | " - Sample Example On Scatter_ with Zero and One Dimension.\n", 27 | " - Filter observation based on Condition.\n", 28 | "- Norm\n", 29 | " - L2 Norm\n", 30 | " - L1 Norm" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "### Importing Libraries" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "import os\n", 47 | "import numpy as np\n", 48 | "\n", 49 | "import torch\n", 50 | "\n", 51 | "from PIL import Image\n", 52 | "import matplotlib.pyplot as plt\n", 53 | "\n", 54 | "import warnings\n", 55 | "warnings.filterwarnings(\"ignore\")" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "### Importing Image" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": null, 68 | "metadata": {}, 69 | "outputs": [], 70 | "source": [ 71 | "image = Image.open('SampleImages/dog.jpg')\n", 72 | "np_image = np.rollaxis(np.asarray(image), 2, 0)\n", 73 | "image" 74 | ] 75 | }, 76 | { 77 | "cell_type": "markdown", 78 | "metadata": {}, 79 | "source": [ 80 | "### Working with Images" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": null, 86 | "metadata": { 87 | "jupyter": { 88 | "outputs_hidden": true 89 | } 90 | }, 91 | "outputs": [], 92 | "source": [ 93 | "print(f'Numpy Image Shape: {np_image.shape}')\n", 94 | "torchImage = torch.from_numpy(np_image)\n", 95 | "print(f'Convert Numpy to PyTorch: {torchImage.shape}')" 96 | ] 97 | }, 98 | { 99 | "cell_type": "markdown", 100 | "metadata": {}, 101 | "source": [ 102 | "* Pytorch format for image is C * H * W\n", 103 | "* Convert numpy into PyTorch Format\n", 104 | "* If the image is not in required format, then we can use Permute to align the dimension as required by pytorch, it moves dimension as mentioned in permute.\n", 105 | "\n", 106 | "If you've built a deep learning model, during training, we should mention the batch size of the data. Now, we have seen that an image has three dimension. So there must be another dimension(batch), which needs to be included.\n", 107 | "\n", 108 | "**Note: Sometimes images also have an alpha channel indicating transparency.**" 109 | ] 110 | }, 111 | { 112 | "cell_type": "markdown", 113 | "metadata": {}, 114 | "source": [ 115 | "### Introducing Batch Dimension\n", 116 | "\n", 117 | "**Since we have only one image, we have batch size of 1. Unsqueeze introduces the dimension along with position in the argument. Standard practice is to keep the batch dimension at first position.**" 118 | ] 119 | }, 120 | { 121 | "cell_type": "code", 122 | "execution_count": null, 123 | "metadata": {}, 124 | "outputs": [], 125 | "source": [ 126 | "print(f'Batch Dimension at First Position: {torchImage.unsqueeze(0).shape}')\n", 127 | "print(f'Batch Dimension at Second Position: {torchImage.unsqueeze(1).shape}')\n", 128 | "print(f'Batch Dimension at Last Position: {torchImage.unsqueeze(-1).shape}')" 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "metadata": {}, 134 | "source": [ 135 | "### Load Batch Of Images (Not Recommended Approach)\n", 136 | "\n", 137 | "Declaring Zero matrix of an Image Dimension with constant resolution and batch dimension. Here, we are creating block of tensor to handle the batch of images. But torchvision library provides dataloader to load bulk of images. We will create dataloader in the next notebook." 138 | ] 139 | }, 140 | { 141 | "cell_type": "code", 142 | "execution_count": null, 143 | "metadata": {}, 144 | "outputs": [], 145 | "source": [ 146 | "\"\"\"Creating a batch of three image.\"\"\"\n", 147 | "batch_size = 3\n", 148 | "imageBatch = torch.zeros(batch_size, 3, 256, 256, dtype=torch.int16)" 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "metadata": {}, 154 | "source": [ 155 | "Now, we can insert three images into this batch tensor. \n", 156 | "\n", 157 | "* Moving a set of three images of different dimensions into imageBatch.\n", 158 | "* We should resize the dimensions of the image to imageBatch dimension.\n", 159 | "\n", 160 | "A bunch of things are happening in the below cell\n", 161 | "\n", 162 | "- We are only selecting \"jpg\" files.\n", 163 | "- Iterating through image folder and generating corresponding path for each image.\n", 164 | "- Resizing each image w.r.t imageBatch." 165 | ] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "execution_count": null, 170 | "metadata": {}, 171 | "outputs": [], 172 | "source": [ 173 | "data = 'SampleImages'\n", 174 | "filenames = [name for name in os.listdir(data) if os.path.splitext(name)[-1] == '.jpg']\n", 175 | "pilImages = [Image.open(os.path.join(data, f)) for f in filenames[:]]\n", 176 | "f = lambda: [img.resize((256, 256)) for img in pilImages]" 177 | ] 178 | }, 179 | { 180 | "cell_type": "markdown", 181 | "metadata": {}, 182 | "source": [ 183 | "**Converting the PIL to Numpy Array, from Numpy to Torch. There is other way around for conversion from PIL to Torch image using torchvision library.**" 184 | ] 185 | }, 186 | { 187 | "cell_type": "code", 188 | "execution_count": null, 189 | "metadata": {}, 190 | "outputs": [], 191 | "source": [ 192 | "for i, file in enumerate(f()):\n", 193 | " file = np.asarray(file)\n", 194 | " file = torch.from_numpy(file).permute(2, 0, 1)\n", 195 | " imageBatch[i] = file" 196 | ] 197 | }, 198 | { 199 | "cell_type": "markdown", 200 | "metadata": {}, 201 | "source": [ 202 | "**We have three images in a imageBatch Tensor with resized dimension to 256x256.**" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": null, 208 | "metadata": {}, 209 | "outputs": [], 210 | "source": [ 211 | "imageBatch.shape" 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "metadata": {}, 217 | "source": [ 218 | "### Normalization\n", 219 | "\n", 220 | "Normalization is a common Machine Learning concept applied on the features to scale down the values. Normalization on images can be done by finding the image's mean and standard deviation.\n", 221 | "\n", 222 | "**One way to normalize a grayscale image is image/=255.0**\n", 223 | "\n", 224 | "In the below cell, we perform **Standardization**. A series of steps are followed here\n", 225 | "\n", 226 | "- Convert Int type tensor to float tensor.\n", 227 | "- Get the number of channels in the Image.\n", 228 | "- Iterate through each channel.\n", 229 | "- Calculate mean and standard deviation by combining three image's channel iteratively." 230 | ] 231 | }, 232 | { 233 | "cell_type": "code", 234 | "execution_count": null, 235 | "metadata": {}, 236 | "outputs": [], 237 | "source": [ 238 | "### Second way to Normalize, It is technically called as Standardization.\n", 239 | "imageBatch = imageBatch.float()\n", 240 | "\n", 241 | "n_channels = imageBatch.shape[1]\n", 242 | "for c in range(n_channels):\n", 243 | " mean = torch.mean(imageBatch[:, c])\n", 244 | " std = torch.std(imageBatch[:, c])\n", 245 | " imageBatch[:, c] = (imageBatch[:, c] - mean) / std" 246 | ] 247 | }, 248 | { 249 | "cell_type": "markdown", 250 | "metadata": {}, 251 | "source": [ 252 | "### Plotting" 253 | ] 254 | }, 255 | { 256 | "cell_type": "code", 257 | "execution_count": null, 258 | "metadata": {}, 259 | "outputs": [], 260 | "source": [ 261 | "f, axarr = plt.subplots(1,2, figsize=(10,10))\n", 262 | "axarr[0].imshow(np.asarray(pilImages[0]))\n", 263 | "axarr[1].imshow(imageBatch[0].permute(1,2,0))\n", 264 | "axarr[0].set(xlabel=\"Original\")\n", 265 | "axarr[1].set(xlabel=\"Edited-256x256 and Standardized\")\n", 266 | "plt.show()" 267 | ] 268 | }, 269 | { 270 | "cell_type": "markdown", 271 | "metadata": {}, 272 | "source": [ 273 | "### One Hot Encoding\n", 274 | "\n", 275 | "To handle categorical variables like class names or text feature etc, we use one hot encoding.\n", 276 | "We create a vector of 25 rows and then plug a value 1 at each index along the dimension mentioned in _scatter. " 277 | ] 278 | }, 279 | { 280 | "cell_type": "markdown", 281 | "metadata": {}, 282 | "source": [ 283 | "**Converting Vector Into One-Hot Encoded Matrix**" 284 | ] 285 | }, 286 | { 287 | "cell_type": "markdown", 288 | "metadata": {}, 289 | "source": [ 290 | "Steps to convert a vector into one-hot encoding.\n", 291 | "\n", 292 | "- Create a Vector.\n", 293 | "- Create zero matrix with unique categories present in vector.\n", 294 | "- Converting 1D Vector into 2D Matrix Using unsqueeze.\n", 295 | "- torch scatter_ writes values on the indices as mentioned by vector.\n", 296 | "- scatter_ method's first argument of 1 refers to dimension. \n", 297 | " - If one, it sets 1.0 along column dimension.\n", 298 | " - If zero, it sets 1.0 along row dimension.\n", 299 | "- Mapping value 1.0 to Indices." 300 | ] 301 | }, 302 | { 303 | "cell_type": "code", 304 | "execution_count": null, 305 | "metadata": {}, 306 | "outputs": [], 307 | "source": [ 308 | "vector = torch.randint(0, 5, (25,))\n", 309 | "print(vector)" 310 | ] 311 | }, 312 | { 313 | "cell_type": "code", 314 | "execution_count": null, 315 | "metadata": {}, 316 | "outputs": [], 317 | "source": [ 318 | "target_onehot = torch.zeros(vector.shape[0], len(vector.unique()))\n", 319 | "target_onehot.scatter_(1, vector.unsqueeze(1), 1.0)" 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "metadata": {}, 325 | "source": [ 326 | "### Sample Example On Scatter_ with Zero and One Dimension" 327 | ] 328 | }, 329 | { 330 | "cell_type": "code", 331 | "execution_count": null, 332 | "metadata": {}, 333 | "outputs": [], 334 | "source": [ 335 | "src = torch.arange(1, 11).reshape((2, 5))" 336 | ] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "execution_count": null, 341 | "metadata": {}, 342 | "outputs": [], 343 | "source": [ 344 | "index = torch.tensor([[0, 1, 2, 0]])\n", 345 | "torch.zeros(3, 5, dtype=src.dtype).scatter_(0, index, src)" 346 | ] 347 | }, 348 | { 349 | "cell_type": "code", 350 | "execution_count": null, 351 | "metadata": {}, 352 | "outputs": [], 353 | "source": [ 354 | "index = torch.tensor([[0, 1, 2, 0]])\n", 355 | "torch.zeros(3, 5, dtype=src.dtype).scatter_(1, index, src)" 356 | ] 357 | }, 358 | { 359 | "cell_type": "code", 360 | "execution_count": null, 361 | "metadata": {}, 362 | "outputs": [], 363 | "source": [ 364 | "index = torch.tensor([[0, 1, 3], [0, 1, 4]])\n", 365 | "torch.zeros(3, 5, dtype=src.dtype).scatter_(1, index, src) " 366 | ] 367 | }, 368 | { 369 | "cell_type": "markdown", 370 | "metadata": {}, 371 | "source": [ 372 | "**Filtering records based on condition.**\n", 373 | "\n", 374 | "Consider we have 25 observations with 10 features and a target variable, we can assume the previously created vector as target variable. If we can filter observations based on target value as mentioned in the below cell." 375 | ] 376 | }, 377 | { 378 | "cell_type": "code", 379 | "execution_count": null, 380 | "metadata": {}, 381 | "outputs": [], 382 | "source": [ 383 | "\"\"\"\n", 384 | "We find the index of the vector with values less than or equal to two \n", 385 | "and utilize those indexes to get relevant observation.\n", 386 | "\"\"\"\n", 387 | "data = torch.randn(25, 10)\n", 388 | "print(f'Total Observation: {data.shape}')\n", 389 | "bad_index = vector<=2\n", 390 | "bad_data = data[bad_index]\n", 391 | "print(f'Filtered Observations: {bad_data.shape}')" 392 | ] 393 | }, 394 | { 395 | "cell_type": "markdown", 396 | "metadata": {}, 397 | "source": [ 398 | "### Calculating Norm of a Vector\n", 399 | "\n", 400 | "**In Machine Learning, we often hear about norm of a vector. It refers to magnitude or length of a vector in the vector space.**" 401 | ] 402 | }, 403 | { 404 | "cell_type": "markdown", 405 | "metadata": {}, 406 | "source": [ 407 | "**L2 Norm**" 408 | ] 409 | }, 410 | { 411 | "cell_type": "code", 412 | "execution_count": null, 413 | "metadata": {}, 414 | "outputs": [], 415 | "source": [ 416 | "u = torch.tensor([3.0, -4.0])\n", 417 | "print(f'L2 Norm of a Vector: {torch.norm(u, p=2)}')" 418 | ] 419 | }, 420 | { 421 | "cell_type": "markdown", 422 | "metadata": {}, 423 | "source": [ 424 | "**L1 Norm**" 425 | ] 426 | }, 427 | { 428 | "cell_type": "code", 429 | "execution_count": null, 430 | "metadata": {}, 431 | "outputs": [], 432 | "source": [ 433 | "u = torch.tensor([3.0, -4.0])\n", 434 | "print(f'L1 Norm of a Vector: {torch.norm(u, p=1)}')" 435 | ] 436 | }, 437 | { 438 | "cell_type": "markdown", 439 | "metadata": {}, 440 | "source": [ 441 | "### Thanks For Reading. For Feedback, reach out on Github. Please don't spam." 442 | ] 443 | } 444 | ], 445 | "metadata": { 446 | "interpreter": { 447 | "hash": "dca0ade3e726a953b501b15e8e990130d2b7799f14cfd9f4271676035ebe5511" 448 | }, 449 | "kernelspec": { 450 | "display_name": "Python 3", 451 | "language": "python", 452 | "name": "python3" 453 | }, 454 | "language_info": { 455 | "codemirror_mode": { 456 | "name": "ipython", 457 | "version": 3 458 | }, 459 | "file_extension": ".py", 460 | "mimetype": "text/x-python", 461 | "name": "python", 462 | "nbconvert_exporter": "python", 463 | "pygments_lexer": "ipython3", 464 | "version": "3.7.6" 465 | } 466 | }, 467 | "nbformat": 4, 468 | "nbformat_minor": 4 469 | } 470 | -------------------------------------------------------------------------------- /Pytorch Tutorial/PyTorch-IV.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### PyTorch\n", 8 | "\n", 9 | "PyTorch is a popular open source machine learning library based on Torch library. Pytorch provides three set of libraries, i.e., torchvision, torchaudio, torchtext for Computer Vision, Audio and Text respectively.\n", 10 | "\n", 11 | "It provides two high-level features:\n", 12 | "\n", 13 | "* Tensor computation (like NumPy) with strong GPU acceleration.\n", 14 | "* Deep neural networks built on a type-based autograd system.\n", 15 | "\n", 16 | "**Topic Covered**\n", 17 | "\n", 18 | " - Handling Time Series Data\n", 19 | " - Handling Text Data" 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "metadata": {}, 25 | "source": [ 26 | "### Importing Libraries" 27 | ] 28 | }, 29 | { 30 | "cell_type": "code", 31 | "execution_count": null, 32 | "metadata": {}, 33 | "outputs": [], 34 | "source": [ 35 | "import os\n", 36 | "import numpy as np\n", 37 | "\n", 38 | "import torch\n", 39 | "\n", 40 | "from PIL import Image\n", 41 | "import matplotlib.pyplot as plt\n", 42 | "\n", 43 | "import warnings\n", 44 | "warnings.filterwarnings(\"ignore\")" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "### Working on Time Series Data\n", 52 | "\n", 53 | "We use [Bike sharing](https://www.kaggle.com/c/bike-sharing-demand/data) data from kaggle given for a two year period. Though, the objective of the dataset is to predict the count of bikes rented during each hour, but here we are going to learn how to handle time series dataset using pytorch.\n", 54 | "\n", 55 | "* Loading dataset using Numpy\n", 56 | "* Skip the column names\n", 57 | "* Convert string to float" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "# 2 Years Data\n", 67 | "bikes_numpy = np.loadtxt(\"data/hour-fixed.csv\",dtype=np.float32,\n", 68 | " delimiter=\",\",skiprows=1, converters={1: lambda x: float(x[8:10])})" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "* Convert numpy to pytorch tensor.\n", 76 | "* Check after how many elements the next(second) records starts from zero offset." 77 | ] 78 | }, 79 | { 80 | "cell_type": "code", 81 | "execution_count": null, 82 | "metadata": {}, 83 | "outputs": [], 84 | "source": [ 85 | "print(f'Total Obervations: {bikes_numpy.shape} \\n')\n", 86 | "bikes = torch.from_numpy(bikes_numpy)\n", 87 | "print(f'A Jump of 17 element in 1st dimension is required to go from one observation to next observation {bikes.stride()}')" 88 | ] 89 | }, 90 | { 91 | "cell_type": "markdown", 92 | "metadata": {}, 93 | "source": [ 94 | "### Preparing the Dataset into features\n", 95 | "\n", 96 | "* **Convert the hour based into days as 1st dim, 24 hours as 2nd dim and 17 features as 3rd dim. It is quite common to reshape the timeseries data to find seasonality or trends.**\n", 97 | "* 17520 / (2 * 365 * 24) = 1.0. (2 years, 365 Days, 24 hours - Total Number of Observations.)\n", 98 | "* The original data is presented on hour bases.\n", 99 | "* We have 2 years data, 730 days, each day has 24 hours and each hour has 17 columns/features.\n", 100 | "* We convert the hours into days, which becomes our records or rows. Each row or record is further segregated among hours our second dim.\n", 101 | "* Each row in second dim contains 17 elements." 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": {}, 108 | "outputs": [], 109 | "source": [ 110 | "daily_bikes = bikes.view(-1, 24, bikes.shape[1])\n", 111 | "print(f'Reshaped Time Series Dataset: {daily_bikes.shape} \\n')\n", 112 | "print(f'Stride for the current data structure: {daily_bikes.stride()}')" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "Stride : 24 * 17 = 408\n", 120 | "\n", 121 | "N: 730 days\n", 122 | "\n", 123 | "* Reshaping the data to make it available for training purpose.\n", 124 | "* Converting Class into One Hot encoding\n", 125 | "* Stride value (408, 17, 1), it means to jump from one day (24x17 elements) to next day records, we need to jump 408 elements (feature values)." 126 | ] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "metadata": {}, 131 | "source": [ 132 | "**Since prediction requires count of rent biker per hour, we transpose the feature dimensions.** Below we are performing a series of steps.\n", 133 | "\n", 134 | "- Selecting 24 rows.\n", 135 | "- Creating empty matrix with 24 rows for weather feature.(categorical variable)\n", 136 | "- Creating One-Hot Encoder Vector for weather feature." 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": null, 142 | "metadata": {}, 143 | "outputs": [], 144 | "source": [ 145 | "daily_bikes = daily_bikes.transpose(1, 2)\n", 146 | "print(f'After Transpose: {daily_bikes.shape, daily_bikes.stride()} \\n')\n", 147 | "first_day = bikes[:24].long()\n", 148 | "weather_onehot = torch.zeros(first_day.shape[0], 4) #4 is count of categories\n", 149 | "print(f'Weather Feature (categorical): {first_day[:,9]}')" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "metadata": {}, 155 | "source": [ 156 | "**Would encourage reader's to take up previous pytorch notebooks to understand better.**" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "print(f'One-Hot Encoded Weather Feature \\n{weather_onehot.scatter_(dim=1,index=first_day[:,9].unsqueeze(1).long() - 1,value=1.0)}')" 166 | ] 167 | }, 168 | { 169 | "cell_type": "markdown", 170 | "metadata": {}, 171 | "source": [ 172 | "* **Combining the one hot encoded matrix with feature matrix.**\n", 173 | "* **Creating Empty matrix to accomdate all the weather feature as one-hot encoder.**\n", 174 | "* **Creating Zero Matrix for Class label.**" 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": null, 180 | "metadata": {}, 181 | "outputs": [], 182 | "source": [ 183 | "\"\"\"Using cat method, we concat two matrix. Combining first 24 obsersations and \n", 184 | "its feature with weather's one-hot encoded vectors.\n", 185 | "\"\"\"\n", 186 | "\n", 187 | "combinedFeature = torch.cat((bikes[:24], weather_onehot), 1) \n", 188 | "print(f'After combining features (17 feature is turned into 21 feature after one-hot encoding): {combinedFeature.shape}\\n')\n", 189 | "\n", 190 | "daily_weather_onehot = torch.zeros(daily_bikes.shape[0], 4, daily_bikes.shape[2])\n", 191 | "print(f'Zero Matrix to accomdate weather\\'s one-hot encoding {daily_weather_onehot.shape}')" 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "metadata": {}, 197 | "source": [ 198 | "**Turning Zero Empty matrix into One-Hot Encoded feature.**" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": null, 204 | "metadata": {}, 205 | "outputs": [], 206 | "source": [ 207 | "daily_weather_onehot.scatter_(1, daily_bikes[:,9,:].long().unsqueeze(1) - 1, 1.0)" 208 | ] 209 | }, 210 | { 211 | "cell_type": "markdown", 212 | "metadata": {}, 213 | "source": [ 214 | "**Combining the existing and one-hot encoded features.**" 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "execution_count": null, 220 | "metadata": {}, 221 | "outputs": [], 222 | "source": [ 223 | "daily_bikes = torch.cat((daily_bikes, daily_weather_onehot), dim=1)\n", 224 | "print(f'Time Series Dataset: {daily_bikes.shape}')" 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "metadata": {}, 230 | "source": [ 231 | "**Keeping Temp values between [0, 1] with min-max Scaling.**" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": null, 237 | "metadata": {}, 238 | "outputs": [], 239 | "source": [ 240 | "temp = daily_bikes[:, 10, :]\n", 241 | "temp_min = torch.min(temp)\n", 242 | "temp_max = torch.max(temp)\n", 243 | "daily_bikes[:, 10, :] = ((daily_bikes[:, 10, :] - temp_min) / (temp_max - temp_min))\n", 244 | "print(f'Min-Max Scaled: {daily_bikes[:,10,:]}')" 245 | ] 246 | }, 247 | { 248 | "cell_type": "markdown", 249 | "metadata": {}, 250 | "source": [ 251 | "****Standardization of Temperature Feature****" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": null, 257 | "metadata": {}, 258 | "outputs": [], 259 | "source": [ 260 | "# temp = daily_bikes[:, 10, :]\n", 261 | "# daily_bikes[:, 10, :] = ((daily_bikes[:, 10, :] - torch.mean(temp))\n", 262 | "# / torch.std(temp))\n", 263 | "# print(f'Standardized Temperature Feature: {daily_bikes[:,10,:]}')" 264 | ] 265 | }, 266 | { 267 | "cell_type": "markdown", 268 | "metadata": {}, 269 | "source": [ 270 | "### Working with Text" 271 | ] 272 | }, 273 | { 274 | "cell_type": "markdown", 275 | "metadata": {}, 276 | "source": [ 277 | "### Character Level Conversion\n", 278 | "\n", 279 | "There are 128 Ascii character, we convert our text/letter into index or integer. Letters not present in ASCII are turned into 0." 280 | ] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": null, 285 | "metadata": {}, 286 | "outputs": [], 287 | "source": [ 288 | "with open(\"data/anna.txt\", encoding='utf-8') as f:\n", 289 | " text = f.read()" 290 | ] 291 | }, 292 | { 293 | "cell_type": "markdown", 294 | "metadata": {}, 295 | "source": [ 296 | "* Converting text document into list of lines.\n", 297 | "* Fetching One Line by Index\n", 298 | "* Creating Matrix based on length of line and total number of characters possible." 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "execution_count": null, 304 | "metadata": {}, 305 | "outputs": [], 306 | "source": [ 307 | "lines = text.split(\"\\n\")\n", 308 | "print(f'Total Number of Lines: {len(lines)} \\n')\n", 309 | "line = lines[100]\n", 310 | "letter_t = torch.zeros(len(line), 128)\n", 311 | "print(f'Number of characters in a line x Total No. of Characters Possible: {letter_t.shape}')" 312 | ] 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "metadata": {}, 317 | "source": [ 318 | "**Creating Character Matrix for a line. All ASCII characters are features, and each character from a line is separate observation. Since each observation is one character, we have that particular feature as 1 and other features as 0.**" 319 | ] 320 | }, 321 | { 322 | "cell_type": "code", 323 | "execution_count": null, 324 | "metadata": {}, 325 | "outputs": [], 326 | "source": [ 327 | "line" 328 | ] 329 | }, 330 | { 331 | "cell_type": "code", 332 | "execution_count": null, 333 | "metadata": {}, 334 | "outputs": [], 335 | "source": [ 336 | "for i, letter in enumerate(line.lower().strip()):\n", 337 | " letter_index = ord(letter) if ord(letter) < 128 else 0\n", 338 | " #print(letter_index)\n", 339 | " letter_t[i][letter_index] = 1\n", 340 | " \n", 341 | "print(f'Character Matrix: {letter_t[0]}')" 342 | ] 343 | }, 344 | { 345 | "cell_type": "code", 346 | "execution_count": null, 347 | "metadata": {}, 348 | "outputs": [], 349 | "source": [ 350 | "print(f'Column Number of the Feature In Character Matrix: {(letter_t[0]==1).nonzero(as_tuple=True)} \\n')\n", 351 | "\n", 352 | "print(f'First Character of the line: {ord(line[0])}')" 353 | ] 354 | }, 355 | { 356 | "cell_type": "markdown", 357 | "metadata": {}, 358 | "source": [ 359 | "### Word Level Conversion\n", 360 | "\n", 361 | "* Clean the text\n", 362 | "* Sorting the text\n", 363 | "* Mapping the words into integer" 364 | ] 365 | }, 366 | { 367 | "cell_type": "code", 368 | "execution_count": null, 369 | "metadata": {}, 370 | "outputs": [], 371 | "source": [ 372 | "def clean_words(input_str):\n", 373 | " punctuation = '.,;:\"!?”“_-'\n", 374 | " word_list = input_str.lower().replace('\\n',' ').split()\n", 375 | " word_list = [word.strip(punctuation) for word in word_list]\n", 376 | " return word_list" 377 | ] 378 | }, 379 | { 380 | "cell_type": "markdown", 381 | "metadata": {}, 382 | "source": [ 383 | "**Cleaning and Tokenization**" 384 | ] 385 | }, 386 | { 387 | "cell_type": "code", 388 | "execution_count": null, 389 | "metadata": {}, 390 | "outputs": [], 391 | "source": [ 392 | "words_in_line = clean_words(line)\n", 393 | "line, words_in_line" 394 | ] 395 | }, 396 | { 397 | "cell_type": "markdown", 398 | "metadata": {}, 399 | "source": [ 400 | "15K+ words are present in the text after cleaning. We are mapping each word with an unique id." 401 | ] 402 | }, 403 | { 404 | "cell_type": "code", 405 | "execution_count": null, 406 | "metadata": {}, 407 | "outputs": [], 408 | "source": [ 409 | "word_list = sorted(set(clean_words(text)))\n", 410 | "word2index_dict = {word: i for (i, word) in enumerate(word_list)}\n", 411 | "print(f'Total Number Of Unique Words: {len(word2index_dict.keys())}')" 412 | ] 413 | }, 414 | { 415 | "cell_type": "markdown", 416 | "metadata": {}, 417 | "source": [ 418 | "* Creating a Matrix where columns size is length of all words and each row is length of each document." 419 | ] 420 | }, 421 | { 422 | "cell_type": "code", 423 | "execution_count": null, 424 | "metadata": {}, 425 | "outputs": [], 426 | "source": [ 427 | "word_t = torch.zeros(len(words_in_line), len(word2index_dict))\n", 428 | "word_t.shape" 429 | ] 430 | }, 431 | { 432 | "cell_type": "markdown", 433 | "metadata": {}, 434 | "source": [ 435 | "**Index of each word on the column axis**" 436 | ] 437 | }, 438 | { 439 | "cell_type": "code", 440 | "execution_count": null, 441 | "metadata": {}, 442 | "outputs": [], 443 | "source": [ 444 | "for i, word in enumerate(words_in_line):\n", 445 | " word_index = word2index_dict[word]\n", 446 | " word_t[i][word_index] = 1\n", 447 | " print('{:2} {:4} {}'.format(i, word_index, word))" 448 | ] 449 | }, 450 | { 451 | "cell_type": "code", 452 | "execution_count": null, 453 | "metadata": {}, 454 | "outputs": [], 455 | "source": [ 456 | "print(f'Word Matrix: \\n{word_t}')" 457 | ] 458 | }, 459 | { 460 | "cell_type": "markdown", 461 | "metadata": {}, 462 | "source": [ 463 | "### Thanks For Reading. For Feedback, reach out on Github. Please don't spam." 464 | ] 465 | } 466 | ], 467 | "metadata": { 468 | "interpreter": { 469 | "hash": "dca0ade3e726a953b501b15e8e990130d2b7799f14cfd9f4271676035ebe5511" 470 | }, 471 | "kernelspec": { 472 | "display_name": "Python 3", 473 | "language": "python", 474 | "name": "python3" 475 | }, 476 | "language_info": { 477 | "codemirror_mode": { 478 | "name": "ipython", 479 | "version": 3 480 | }, 481 | "file_extension": ".py", 482 | "mimetype": "text/x-python", 483 | "name": "python", 484 | "nbconvert_exporter": "python", 485 | "pygments_lexer": "ipython3", 486 | "version": "3.7.6" 487 | } 488 | }, 489 | "nbformat": 4, 490 | "nbformat_minor": 4 491 | } 492 | -------------------------------------------------------------------------------- /Pytorch Tutorial/PyTorch-V.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### PyTorch\n", 8 | "\n", 9 | "PyTorch is a popular open source machine learning library based on Torch library. Pytorch provides three set of libraries, i.e., torchvision, torchaudio, torchtext for Computer Vision, Audio and Text respectively.\n", 10 | "\n", 11 | "It provides two high-level features:\n", 12 | "\n", 13 | "* Tensor computation (like NumPy) with strong GPU acceleration.\n", 14 | "* Deep neural networks built on a type-based autograd system.\n", 15 | "\n", 16 | "**Topic Covered**\n", 17 | "\n", 18 | "- Building simple model.\n", 19 | "- Computing derivative.\n", 20 | "- Training and computing loss.\n", 21 | "- Using Autograd.\n", 22 | "- Turn tensor to a learnable parameter.\n", 23 | "- Training and Updating parameters using Autograd.\n", 24 | "\n", 25 | "**Note**: I am assuming the reader has basic understanding of simple machine learning model." 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "### Importing Libraries" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": null, 38 | "metadata": {}, 39 | "outputs": [], 40 | "source": [ 41 | "import os\n", 42 | "import numpy as np\n", 43 | "\n", 44 | "import torch\n", 45 | "\n", 46 | "from PIL import Image\n", 47 | "import matplotlib.pyplot as plt\n", 48 | "\n", 49 | "import warnings\n", 50 | "warnings.filterwarnings(\"ignore\")" 51 | ] 52 | }, 53 | { 54 | "cell_type": "markdown", 55 | "metadata": {}, 56 | "source": [ 57 | "### How to create a Model\n", 58 | "\n", 59 | "* Creating linear regression\n", 60 | "* Mapping of x and y.\n", 61 | "* Turing List into Tensor.\n", 62 | "* Create a function for basic line equation\n", 63 | "* Create a function to measure the loss using mean square error.\n", 64 | "\n", 65 | "\n", 66 | "Declaring vectors t_c (temperature in celsius) and t_u (unknown variable along which the temp in celsius changes).\n", 67 | "To be precise, t_u is the x, i.e. feature and t_c is the target/label." 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "t_c = [0.5, 14.0, 15.0, 28.0, 11.0, 8.0, 3.0, -4.0, 6.0, 13.0, 21.0]\n", 77 | "t_u = [35.7, 55.9, 58.2, 81.9, 56.3, 48.9, 33.9, 21.8, 48.4, 60.4, 68.4]\n", 78 | "t_c = torch.tensor(t_c)\n", 79 | "t_u = torch.tensor(t_u)" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "\"\"\"Python function returns the line equation, where w represents the weight vector and b represents intercept(bias term)\"\"\"\n", 89 | "\n", 90 | "def model(t_u, w, b):\n", 91 | " return w*t_u + b" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "\"\"\"Loss function - To find difference between estimated vs actual value. \n", 101 | "Loss function is called as Mean Squared Error\"\"\"\n", 102 | "\n", 103 | "def loss(t_p, t_c):\n", 104 | " return torch.mean((t_p-t_c)**2)" 105 | ] 106 | }, 107 | { 108 | "cell_type": "code", 109 | "execution_count": null, 110 | "metadata": {}, 111 | "outputs": [], 112 | "source": [ 113 | "\"\"\"Initializing Weight and Bias\"\"\"\n", 114 | "\n", 115 | "w = torch.ones(())\n", 116 | "b = torch.zeros(())" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": null, 122 | "metadata": {}, 123 | "outputs": [], 124 | "source": [ 125 | "t_p = model(w, t_u, b)\n", 126 | "Loss = loss(t_p, t_c)\n", 127 | "print(f'Loss on Untrained Model: {Loss}')" 128 | ] 129 | }, 130 | { 131 | "cell_type": "markdown", 132 | "metadata": {}, 133 | "source": [ 134 | "In Machine Learning, the objective is to build a optimized function to estimate the acual target. Here, the optimized function is a function of weight, bias, and input. Thus, we optimize the weight and bias of the function, to accurately estimate the target.\n", 135 | "\n", 136 | "To optimize the parameters, we use the popular gradient descent algorithm, the idea is to compute the rate of change of the loss with respect to each parameter, and modify each parameter in the direction of decreasing loss.\n", 137 | "\n", 138 | " The loss function represents the difference between our estimated output against the actual ground truth." 139 | ] 140 | }, 141 | { 142 | "cell_type": "markdown", 143 | "metadata": {}, 144 | "source": [ 145 | "**Loss Rate Of Change In W and b**\n", 146 | "\n", 147 | "This is saying that in the neighborhood of the current values of w and b, a unit\n", 148 | "increase in w leads to some change in the loss. If the change is negative, then we need\n", 149 | "to increase w to minimize the loss, whereas if the change is positive, we need to\n", 150 | "decrease w." 151 | ] 152 | }, 153 | { 154 | "cell_type": "code", 155 | "execution_count": null, 156 | "metadata": {}, 157 | "outputs": [], 158 | "source": [ 159 | "delta = 0.1\n", 160 | "\n", 161 | "loss_rate_of_change_w = (loss(model(t_u, w + delta, b), t_c) - loss(model(t_u, w - delta, b), t_c)) / (2.0 * delta)\n", 162 | "\n", 163 | "learning_rate = 1e-2\n", 164 | "w = w - learning_rate * loss_rate_of_change_w\n", 165 | "\n", 166 | "print(f'Updated W: {w}')" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": null, 172 | "metadata": {}, 173 | "outputs": [], 174 | "source": [ 175 | "loss_rate_of_change_b = \\\n", 176 | "(loss(model(t_u, w, b + delta), t_c) -\n", 177 | "loss(model(t_u, w, b - delta), t_c)) / (2.0 * delta)\n", 178 | "b = b - learning_rate * loss_rate_of_change_b\n", 179 | "print(f'Updated b: {b}')" 180 | ] 181 | }, 182 | { 183 | "cell_type": "markdown", 184 | "metadata": {}, 185 | "source": [ 186 | "### Computing Derivative\n", 187 | "\n", 188 | "To compute derivative of the loss w.r.t parameters, we can apply chain rule(partial derivatives). Would encourage users to read through basic calculus.\n", 189 | "\n", 190 | "d_loss / d w = (d loss / d t_p) * (d t_p / d w)" 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": null, 196 | "metadata": {}, 197 | "outputs": [], 198 | "source": [ 199 | "def d_loss(t_p, t_c):\n", 200 | " dsq_diff = 2 * (t_p - t_c) / t_p.size(0)\n", 201 | " \n", 202 | " return dsq_diff" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": null, 208 | "metadata": {}, 209 | "outputs": [], 210 | "source": [ 211 | "\"\"\"Differentiating linear equation w.r.t w parameter, linear equation is our objective function,\n", 212 | " d(w*t_u + b)/ dw = t_u + 0 = t_u\"\"\"\n", 213 | "\n", 214 | "def d_model_dw(w, t_u, b):\n", 215 | " return t_u " 216 | ] 217 | }, 218 | { 219 | "cell_type": "code", 220 | "execution_count": null, 221 | "metadata": {}, 222 | "outputs": [], 223 | "source": [ 224 | "\"\"\"Differentiating linear equation w.r.t b parameter, d(w*t_u + b)/ db = 0 + 1.0 = 1.0\"\"\"\n", 225 | "\n", 226 | "def d_model_db(w, t_u, b):\n", 227 | " return 1.0" 228 | ] 229 | }, 230 | { 231 | "cell_type": "code", 232 | "execution_count": null, 233 | "metadata": {}, 234 | "outputs": [], 235 | "source": [ 236 | "\"\"\"grad_fn - Updating the weight and bias using learning rate after calculating the loss.\"\"\"\n", 237 | "\n", 238 | "def grad_fn(t_u, t_c, t_p, w, b):\n", 239 | " \n", 240 | " dloss_dtp = d_loss(t_p, t_c)\n", 241 | " dloss_dw = dloss_dtp * d_model_dw(t_u, w, b)\n", 242 | " dloss_db = dloss_dtp * d_model_db(t_u, w, b)\n", 243 | " \n", 244 | "\n", 245 | " return torch.stack([dloss_dw.sum(), dloss_db.sum()])" 246 | ] 247 | }, 248 | { 249 | "cell_type": "markdown", 250 | "metadata": {}, 251 | "source": [ 252 | "**Creating Training Loop**\n", 253 | "\n", 254 | "To iterate over the training data to learn parameters(w and b) by updating it based on the loss." 255 | ] 256 | }, 257 | { 258 | "cell_type": "code", 259 | "execution_count": null, 260 | "metadata": {}, 261 | "outputs": [], 262 | "source": [ 263 | "def training_loop(n_epochs, learning_rate, params, t_u, t_c):\n", 264 | " for epoch in range(0, n_epochs + 1):\n", 265 | " w, b = params\n", 266 | " t_p = model(t_u, w, b) #Forward pass\n", 267 | " \n", 268 | " Loss = loss(t_p, t_c)\n", 269 | " grad = grad_fn(t_u, t_c, t_p, w, b) #Backward pass\n", 270 | " \n", 271 | " params = params - learning_rate * grad\n", 272 | " if epoch % 10 == 0:\n", 273 | " print('Epoch %d, Loss %f' % (epoch, float(Loss)))\n", 274 | " print(f'params: {params}')\n", 275 | " print(f'grad: {grad}\\n')\n", 276 | " return params" 277 | ] 278 | }, 279 | { 280 | "cell_type": "markdown", 281 | "metadata": {}, 282 | "source": [ 283 | "**We are iterating through objective function for 100 epochs, meaning in each epoch we pass the entire dataset through objective function. We calculate the loss and update the parameters in each iteration.**" 284 | ] 285 | }, 286 | { 287 | "cell_type": "code", 288 | "execution_count": null, 289 | "metadata": {}, 290 | "outputs": [], 291 | "source": [ 292 | "params = training_loop(n_epochs = 100, learning_rate = 1e-2, \n", 293 | "\n", 294 | " params = torch.tensor([1.0, 0.0]), t_u = t_u, t_c = t_c)" 295 | ] 296 | }, 297 | { 298 | "cell_type": "markdown", 299 | "metadata": {}, 300 | "source": [ 301 | "After each iteration, we see the change in the parameter value of W and b w.r.t loss. But we can notice that the parameters are not updated after the 20th epoch, meaning, that the parameters are at its best minimum possible in the current experiment setup.\n", 302 | "\n", 303 | "### PyTorch - Autograd Function\n", 304 | "\n", 305 | "Pytorch provides a **.grad** characteristic to each tensor. If a tensor is created as **require_grad = True**, then that tensor turns into learnable parameter. We can check if the parameters are getting updated after executing loss.backward(). \n", 306 | "\n", 307 | "Loss.backward() tells pytorch to update the all learnable parameters to update weight based on loss. We can turn **.grad** to zero by **zero_()** because if we don't turn the **.grad to zero then grad values gets accumulated into .grad after each epoch.**" 308 | ] 309 | }, 310 | { 311 | "cell_type": "code", 312 | "execution_count": null, 313 | "metadata": { 314 | "scrolled": true 315 | }, 316 | "outputs": [], 317 | "source": [ 318 | "params = torch.tensor([1.0, 0.0],requires_grad=True)\n", 319 | "print(f'Learnable Parameter: {params.grad is None}')" 320 | ] 321 | }, 322 | { 323 | "cell_type": "code", 324 | "execution_count": null, 325 | "metadata": {}, 326 | "outputs": [], 327 | "source": [ 328 | "Loss = loss(model(t_u, *params), t_c)\n", 329 | "Loss.backward()\n", 330 | "print(f'Parameter\\'s Gradient: {params.grad}')" 331 | ] 332 | }, 333 | { 334 | "cell_type": "code", 335 | "execution_count": null, 336 | "metadata": {}, 337 | "outputs": [], 338 | "source": [ 339 | "if params.grad is not None:\n", 340 | " params.grad.zero_()\n", 341 | "print(f'Turning Gradient to Zero: {params.grad}')" 342 | ] 343 | }, 344 | { 345 | "cell_type": "code", 346 | "execution_count": null, 347 | "metadata": {}, 348 | "outputs": [], 349 | "source": [ 350 | "def training_loop_AG(n_epochs, learning_rate, params, t_u, t_c):\n", 351 | " \n", 352 | " for epoch in range(1, n_epochs+1):\n", 353 | " \n", 354 | " if params.grad is not None:\n", 355 | " params.grad.zero_()\n", 356 | " \n", 357 | " t_p = model(t_u, *params)\n", 358 | " Loss = loss(t_p, t_c)\n", 359 | " Loss.backward()\n", 360 | " \n", 361 | " with torch.no_grad():\n", 362 | " params -= learning_rate * params.grad\n", 363 | " \n", 364 | " if epoch % 10 == 0:\n", 365 | " print(f'epoch {epoch}, Loss: {Loss}')\n", 366 | " \n", 367 | " return params" 368 | ] 369 | }, 370 | { 371 | "cell_type": "code", 372 | "execution_count": null, 373 | "metadata": {}, 374 | "outputs": [], 375 | "source": [ 376 | "\"\"\"scaling Down the feature.\"\"\"\n", 377 | "t_un = 0.1 * t_u " 378 | ] 379 | }, 380 | { 381 | "cell_type": "code", 382 | "execution_count": null, 383 | "metadata": {}, 384 | "outputs": [], 385 | "source": [ 386 | "training_loop_AG(n_epochs = 100, learning_rate = 1e-2,\n", 387 | "params = torch.tensor([1.0, 0.0], requires_grad=True), t_u = t_un, t_c = t_c)" 388 | ] 389 | }, 390 | { 391 | "cell_type": "markdown", 392 | "metadata": {}, 393 | "source": [ 394 | "From the above result, we can notice at the 100th epoch, the scratch code we've written earlier produces a loss of loss 29.66, while the result from the autograd gives a loss of 22.14. \n", 395 | "\n", 396 | "Increase the number of epochs for better results, because we can see that after each epoch the loss is continuously decreasing." 397 | ] 398 | }, 399 | { 400 | "cell_type": "markdown", 401 | "metadata": {}, 402 | "source": [ 403 | "### Thanks For Reading. For Feedback, reach out on Github. Please don't spam." 404 | ] 405 | } 406 | ], 407 | "metadata": { 408 | "interpreter": { 409 | "hash": "dca0ade3e726a953b501b15e8e990130d2b7799f14cfd9f4271676035ebe5511" 410 | }, 411 | "kernelspec": { 412 | "display_name": "Python 3", 413 | "language": "python", 414 | "name": "python3" 415 | }, 416 | "language_info": { 417 | "codemirror_mode": { 418 | "name": "ipython", 419 | "version": 3 420 | }, 421 | "file_extension": ".py", 422 | "mimetype": "text/x-python", 423 | "name": "python", 424 | "nbconvert_exporter": "python", 425 | "pygments_lexer": "ipython3", 426 | "version": "3.7.6" 427 | } 428 | }, 429 | "nbformat": 4, 430 | "nbformat_minor": 4 431 | } 432 | -------------------------------------------------------------------------------- /Pytorch Tutorial/PyTorch-VI.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### PyTorch\n", 8 | "\n", 9 | "PyTorch is a popular open source machine learning library based on Torch library. Pytorch provides three set of libraries, i.e., torchvision, torchaudio, torchtext for Computer Vision, Audio and Text respectively.\n", 10 | "\n", 11 | "It provides two high-level features:\n", 12 | "\n", 13 | "* Tensor computation (like NumPy) with strong GPU acceleration.\n", 14 | "* Deep neural networks built on a type-based autograd system.\n", 15 | "\n", 16 | "**Topic Covered**\n", 17 | "\n", 18 | "- Using Pytorch Optimizer.\n", 19 | "- Splitting Dataset.\n", 20 | "- Training without no_grad.\n", 21 | "- Training with no_grad.\n", 22 | "- Creating Polynomial Model.\n", 23 | "- Building Neural Network Using nn.Module.\n", 24 | "- Building Neural Network With One Hidden Layer.\n", 25 | "- Finding total number of parameters in the model.\n", 26 | "- Building Sequential Model using OrderedDict and Named Layers.\n", 27 | "- Training and Predicting on validation set.\n", 28 | "\n", 29 | "**Note**: I am assuming the reader has basic understanding of simple machine learning model. Would encourage the user to read notebook V to have clear understanding of what are the different terms like model, loss function, autograd etc." 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "### Importing Libraries" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": null, 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "import os\n", 46 | "import numpy as np\n", 47 | "from collections import OrderedDict\n", 48 | "\n", 49 | "import torch\n", 50 | "from torch import optim\n", 51 | "import torch.nn as nn\n", 52 | "\n", 53 | "import matplotlib.pyplot as plt\n", 54 | "\n", 55 | "import warnings\n", 56 | "warnings.filterwarnings(\"ignore\")" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "### Optimizer\n", 64 | "\n", 65 | "Optimizer are the algorithms used to update the weights. Pytorch provides wide range of optimizer from SGD to ADAM & many more. Check docstring for more details.\n", 66 | "* Iterate through epochs.\n", 67 | "* Training the model and finding the predicted value.\n", 68 | "* Pass the predicted value to loss function to calculate the loss.\n", 69 | "* Turn the existing parameter/weights to zero.\n", 70 | "* calculate the weights using optimizer based on loss.\n", 71 | "* Apply the weights update to parameters." 72 | ] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "execution_count": null, 77 | "metadata": {}, 78 | "outputs": [], 79 | "source": [ 80 | "\"\"\"\n", 81 | "Declaring vectors t_c (temperature in celsius) and t_u (unknown variable along which the temp in celsius changes).\n", 82 | "To be precise, t_u is the x, i.e. feature and t_c is the target/label.\n", 83 | "\"\"\"\n", 84 | "\n", 85 | "t_c = [0.5, 14.0, 15.0, 28.0, 11.0, 8.0, 3.0, -4.0, 6.0, 13.0, 21.0]\n", 86 | "t_u = [35.7, 55.9, 58.2, 81.9, 56.3, 48.9, 33.9, 21.8, 48.4, 60.4, 68.4]\n", 87 | "t_c = torch.tensor(t_c)\n", 88 | "t_u = torch.tensor(t_u)" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": null, 94 | "metadata": {}, 95 | "outputs": [], 96 | "source": [ 97 | "\"\"\"Python function returns the line equation, where w represents the weight vector and b represents intercept(bias term)\"\"\"\n", 98 | "\n", 99 | "def model(t_u, w, b):\n", 100 | " return w*t_u + b" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": null, 106 | "metadata": {}, 107 | "outputs": [], 108 | "source": [ 109 | "\"\"\"Loss function - To find difference between estimated vs actual value. \n", 110 | "Loss function is called as Mean Squared Error\"\"\"\n", 111 | "\n", 112 | "def loss(t_p, t_c):\n", 113 | " return torch.mean((t_p-t_c)**2)" 114 | ] 115 | }, 116 | { 117 | "cell_type": "code", 118 | "execution_count": null, 119 | "metadata": {}, 120 | "outputs": [], 121 | "source": [ 122 | "def training_loop_optim(n_epochs, optimizer, params, t_u, t_c):\n", 123 | " for epoch in range(1, n_epochs+1):\n", 124 | " \n", 125 | " t_p = model(t_u, *params)\n", 126 | " Loss = loss(t_p, t_c)\n", 127 | " optimizer.zero_grad()\n", 128 | " \n", 129 | " Loss.backward()\n", 130 | " optimizer.step()\n", 131 | " \n", 132 | " if epoch % 10 == 0:\n", 133 | " print(f'Epoch: {epoch}, Loss: {Loss}')\n", 134 | " \n", 135 | " return params" 136 | ] 137 | }, 138 | { 139 | "cell_type": "code", 140 | "execution_count": null, 141 | "metadata": {}, 142 | "outputs": [], 143 | "source": [ 144 | "params = torch.tensor([1.0, 0.0], requires_grad=True)\n", 145 | "learning_rate = 1e-2\n", 146 | "optimizer = optim.SGD([params], lr=learning_rate)" 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": null, 152 | "metadata": {}, 153 | "outputs": [], 154 | "source": [ 155 | "\"\"\"scaling Down the feature.\"\"\"\n", 156 | "t_un = 0.1 * t_u" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "training_loop_optim(n_epochs=100, optimizer=optimizer,params= params, t_u = t_un,t_c= t_c)" 166 | ] 167 | }, 168 | { 169 | "cell_type": "markdown", 170 | "metadata": {}, 171 | "source": [ 172 | "### Splitting Dataset into Train and Validation\n", 173 | "\n", 174 | "* We use indexing to split the dataset. Using random selection of indices, we select the observations." 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": null, 180 | "metadata": {}, 181 | "outputs": [], 182 | "source": [ 183 | "n_samples = t_u.shape[0]\n", 184 | "n_val = int(0.2 * n_samples)\n", 185 | "shuffled_indices = torch.randperm(n_samples)\n", 186 | "train_indices = shuffled_indices[:-n_val]\n", 187 | "val_indices = shuffled_indices[-n_val:]\n", 188 | "print(f'Indices of Training Samples: {train_indices}')\n", 189 | "print(f'Indices of validation Samples: {val_indices}')" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "* Scaling Down the training and validation samples by 0.1" 197 | ] 198 | }, 199 | { 200 | "cell_type": "code", 201 | "execution_count": null, 202 | "metadata": {}, 203 | "outputs": [], 204 | "source": [ 205 | "train_t_u, train_t_c = t_u[train_indices], t_c[train_indices]\n", 206 | "val_t_u, val_t_c = t_u[val_indices], t_c[val_indices] \n", 207 | "train_t_un = 0.1 * train_t_u\n", 208 | "val_t_un = 0.1 * val_t_u" 209 | ] 210 | }, 211 | { 212 | "cell_type": "markdown", 213 | "metadata": {}, 214 | "source": [ 215 | "In the below cell, we write the training_loop_grad method to train the model and simultaneously validate the model on validation dataset.\n", 216 | "\n", 217 | "Note, While performing an inference on the validation set, we should disable the gradients calculation. Also note the loss at the end of 100th epoch.\n", 218 | "\n", 219 | "[PyTorch Documentation](https://pytorch.org/docs/stable/generated/torch.no_grad.html)\n", 220 | ">Disabling gradient calculation is useful for inference, when you are sure that you will not call Tensor.backward(). It will reduce memory consumption for computations that would otherwise have requires_grad=True" 221 | ] 222 | }, 223 | { 224 | "cell_type": "code", 225 | "execution_count": null, 226 | "metadata": {}, 227 | "outputs": [], 228 | "source": [ 229 | "def training_loop_grad(n_epochs, optimizer, params, train_t_u, val_t_u, train_t_c, val_t_c):\n", 230 | " for epoch in range(1, n_epochs + 1):\n", 231 | " train_t_p = model(train_t_u, *params)\n", 232 | " train_loss = loss(train_t_p, train_t_c)\n", 233 | " val_t_p = model(val_t_u, *params)\n", 234 | " val_loss = loss(val_t_p, val_t_c)\n", 235 | "\n", 236 | " optimizer.zero_grad()\n", 237 | " train_loss.backward()\n", 238 | " optimizer.step()\n", 239 | "\n", 240 | " if epoch % 10 == 0:\n", 241 | " print(f\"Epoch {epoch}, Training loss {train_loss.item():.4f},\"\n", 242 | " f\" Validation loss {val_loss.item():.4f} \\n\")" 243 | ] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": null, 248 | "metadata": {}, 249 | "outputs": [], 250 | "source": [ 251 | "params = torch.tensor([1.0, 0.0], requires_grad=True)\n", 252 | "learning_rate = 1e-2\n", 253 | "optimizer = optim.SGD([params], lr=learning_rate)" 254 | ] 255 | }, 256 | { 257 | "cell_type": "code", 258 | "execution_count": null, 259 | "metadata": {}, 260 | "outputs": [], 261 | "source": [ 262 | "training_loop_grad(n_epochs = 100, optimizer = optimizer,\n", 263 | "params = params, train_t_u = train_t_un, val_t_u = val_t_un,\n", 264 | "train_t_c = train_t_c, val_t_c = val_t_c)" 265 | ] 266 | }, 267 | { 268 | "cell_type": "markdown", 269 | "metadata": {}, 270 | "source": [ 271 | "In the below cell, we write the training_loop_no_grad method to train the model and simultaneously validate the model on validation dataset. Here, by using torch.no_grad(), we disable the gradients.\n", 272 | "\n", 273 | "Check out `torch.set_grad_enabled` for enabling and disabiling gradients" 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": null, 279 | "metadata": {}, 280 | "outputs": [], 281 | "source": [ 282 | "def training_loop_no_grad(n_epochs, optimizer, params, train_t_u, val_t_u,train_t_c, val_t_c):\n", 283 | " for epoch in range(1, n_epochs + 1):\n", 284 | " train_t_p = model(train_t_u, *params)\n", 285 | " train_loss = loss(train_t_p, train_t_c)\n", 286 | " \n", 287 | " with torch.no_grad():\n", 288 | " val_t_p = model(val_t_u, *params)\n", 289 | " val_loss = loss(val_t_p, val_t_c)\n", 290 | " assert val_loss.requires_grad == False\n", 291 | "\n", 292 | " optimizer.zero_grad()\n", 293 | " train_loss.backward()\n", 294 | " optimizer.step()\n", 295 | " \n", 296 | " if epoch % 10 == 0:\n", 297 | " print(f\"Epoch {epoch}, Training loss {train_loss.item():.4f},\"\n", 298 | " f\" Validation loss {val_loss.item():.4f} \\n\")" 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "execution_count": null, 304 | "metadata": {}, 305 | "outputs": [], 306 | "source": [ 307 | "training_loop_no_grad(n_epochs = 100, optimizer = optimizer,\n", 308 | "params = params, train_t_u = train_t_un, val_t_u = val_t_un,\n", 309 | "train_t_c = train_t_c, val_t_c = val_t_c)" 310 | ] 311 | }, 312 | { 313 | "cell_type": "markdown", 314 | "metadata": {}, 315 | "source": [ 316 | "### Polynomial Equation - w2 * t_u ** 2 + w1 * t_u + b" 317 | ] 318 | }, 319 | { 320 | "cell_type": "code", 321 | "execution_count": null, 322 | "metadata": {}, 323 | "outputs": [], 324 | "source": [ 325 | "\"\"\"\n", 326 | "Three parameters - w2, w1, b. Input t_u and its square(t_u**2).\n", 327 | "\n", 328 | "For instance, equation can be thought as 2X^2 + 4X + 5, the coefficient (2, 4, 5) are (w2, w1, b).\n", 329 | "\n", 330 | "We are using in-built methods for loss calculation and for updating the parameters.\n", 331 | "\"\"\"\n", 332 | "\n", 333 | "params = torch.randn(3,) \n", 334 | "params.requires_grad=True\n", 335 | "criterion = torch.nn.MSELoss()\n", 336 | "optimizer = optim.Adam([params], lr=1e-2)" 337 | ] 338 | }, 339 | { 340 | "cell_type": "code", 341 | "execution_count": null, 342 | "metadata": {}, 343 | "outputs": [], 344 | "source": [ 345 | "def polyModel(t_u, w1, w2, b):\n", 346 | " return w2 * t_u ** 2 + w1 * t_u + b" 347 | ] 348 | }, 349 | { 350 | "cell_type": "code", 351 | "execution_count": null, 352 | "metadata": {}, 353 | "outputs": [], 354 | "source": [ 355 | "def train(n_epochs, optimizer, params, train_t_u, val_t_u,train_t_c, val_t_c):\n", 356 | " for epoch in range(1, n_epochs + 1):\n", 357 | " w1, w2, b = params\n", 358 | " train_t_p = polyModel(train_t_u, w1, w2, b)\n", 359 | " train_loss = criterion(train_t_p, train_t_c)\n", 360 | " \n", 361 | " with torch.no_grad():\n", 362 | " val_t_p = polyModel(val_t_u, *params)\n", 363 | " val_loss = criterion(val_t_p, val_t_c)\n", 364 | " assert val_loss.requires_grad == False\n", 365 | "\n", 366 | " optimizer.zero_grad()\n", 367 | " train_loss.backward()\n", 368 | " optimizer.step()\n", 369 | " if epoch % 10==0:\n", 370 | " print(f'Epoch: {epoch}, Loss: {train_loss}, Parameters: {params} \\n')" 371 | ] 372 | }, 373 | { 374 | "cell_type": "code", 375 | "execution_count": null, 376 | "metadata": {}, 377 | "outputs": [], 378 | "source": [ 379 | "train(n_epochs = 100, optimizer = optimizer, params=params,\n", 380 | "train_t_u = train_t_un, val_t_u = val_t_un,\n", 381 | "train_t_c = train_t_c, val_t_c = val_t_c)" 382 | ] 383 | }, 384 | { 385 | "cell_type": "markdown", 386 | "metadata": {}, 387 | "source": [ 388 | "**Note:** With increasing complexity of the model(polynomial), the performance of the model on that training data is improved. In polynomial model at epoch 10, we have a training loss of 4.48, while for the linear model at epoch 10, we have a training loss of 20.08." 389 | ] 390 | }, 391 | { 392 | "cell_type": "markdown", 393 | "metadata": {}, 394 | "source": [ 395 | "### Building Neural Nets Using nn.Module\n", 396 | "\n", 397 | "**nn.Module** is the base class of all neural network modules. Here, the neural network modules refers to the layers like linear layers, convolution layers, LSTM layers, etc, which are required to build the neural network architecture.\n", 398 | "\n", 399 | "For our previous linear model, we used one feature and a target variable. Here, we'll build the same model using nn.Module." 400 | ] 401 | }, 402 | { 403 | "cell_type": "code", 404 | "execution_count": null, 405 | "metadata": {}, 406 | "outputs": [], 407 | "source": [ 408 | "\"\"\"Dummy Linear Equation model with one sample with one feature. \n", 409 | "We can notice that by declaring the Linear Layer, \n", 410 | "we automatically create weight tensor and the bias term is optional\"\"\"\n", 411 | "\n", 412 | "x = torch.ones(1)\n", 413 | "l_model = nn.Linear(1,1, bias=True)\n", 414 | "l_model(x)\n", 415 | "print(f'Weight of the Linear Layer: {l_model.weight.item()} \\n')\n", 416 | "print(f'Bias of the Linear Layer: {l_model.bias.item()}')" 417 | ] 418 | }, 419 | { 420 | "cell_type": "markdown", 421 | "metadata": {}, 422 | "source": [ 423 | "**Model which takes batch of samples with one feature**" 424 | ] 425 | }, 426 | { 427 | "cell_type": "code", 428 | "execution_count": null, 429 | "metadata": {}, 430 | "outputs": [], 431 | "source": [ 432 | "\"\"\"\n", 433 | "Creating a set of sample with one value.\n", 434 | "\"\"\"\n", 435 | "x = torch.ones(10, 1)\n", 436 | "print(f'Passing set of samples through Linear Layer: \\n{l_model(x)}')" 437 | ] 438 | }, 439 | { 440 | "cell_type": "markdown", 441 | "metadata": {}, 442 | "source": [ 443 | "### Building Neural Network with One Hidden Layer\n", 444 | "\n", 445 | "* Pytorch has Sequence Module similar to Keras, We can add layers in sequence, in the order of operation.\n", 446 | "* Finding Total number of parameters in the model.\n", 447 | "* Assigning names to layers, it is helpful when you want to retrain particular layer.\n", 448 | "* Training the network with optimizer and criterion function." 449 | ] 450 | }, 451 | { 452 | "cell_type": "code", 453 | "execution_count": null, 454 | "metadata": {}, 455 | "outputs": [], 456 | "source": [ 457 | "x = t_u.unsqueeze(1)\n", 458 | "y = t_c.unsqueeze(1)" 459 | ] 460 | }, 461 | { 462 | "cell_type": "markdown", 463 | "metadata": {}, 464 | "source": [ 465 | "**Sequential Model**\n", 466 | "\n", 467 | "* First Linear Layer is the hidden layer with 10 neurons.\n", 468 | "* Tanh is the activation function.\n", 469 | "* Second Linear layer is the output layer with 1 neuron (Label)." 470 | ] 471 | }, 472 | { 473 | "cell_type": "code", 474 | "execution_count": null, 475 | "metadata": {}, 476 | "outputs": [], 477 | "source": [ 478 | "seq_model = nn.Sequential(\n", 479 | " nn.Linear(1, 10),\n", 480 | " nn.Tanh(),\n", 481 | " nn.Linear(10, 1)\n", 482 | " )" 483 | ] 484 | }, 485 | { 486 | "cell_type": "code", 487 | "execution_count": null, 488 | "metadata": {}, 489 | "outputs": [], 490 | "source": [ 491 | "print(f'Total Number Of Parameters: {sum(p.numel() for p in seq_model.parameters() if p.requires_grad)}')" 492 | ] 493 | }, 494 | { 495 | "cell_type": "code", 496 | "execution_count": null, 497 | "metadata": {}, 498 | "outputs": [], 499 | "source": [ 500 | "for name, param in seq_model.named_parameters():\n", 501 | " print(f'Parameter Detail: {name, param.shape}')" 502 | ] 503 | }, 504 | { 505 | "cell_type": "markdown", 506 | "metadata": {}, 507 | "source": [ 508 | "**Sequential Model Using OrderedDict With Named Layers**" 509 | ] 510 | }, 511 | { 512 | "cell_type": "code", 513 | "execution_count": null, 514 | "metadata": {}, 515 | "outputs": [], 516 | "source": [ 517 | "seq_model = nn.Sequential(OrderedDict([\n", 518 | " ('hidden_linear', nn.Linear(1, 9)),\n", 519 | " ('hidden_activation', nn.Tanh()),\n", 520 | " ('output_linear', nn.Linear(9, 1))\n", 521 | " ]))\n", 522 | "print(seq_model)" 523 | ] 524 | }, 525 | { 526 | "cell_type": "code", 527 | "execution_count": null, 528 | "metadata": {}, 529 | "outputs": [], 530 | "source": [ 531 | "\"\"\"Named Layers\"\"\"\n", 532 | "for name, param in seq_model.named_parameters():\n", 533 | " print(name, param.shape)" 534 | ] 535 | }, 536 | { 537 | "cell_type": "code", 538 | "execution_count": null, 539 | "metadata": {}, 540 | "outputs": [], 541 | "source": [ 542 | "\"\"\"Random Initialization Of Weight and Bias Term\"\"\"\n", 543 | "\n", 544 | "print(f'Weight Initialized: {seq_model.output_linear.weight}')\n", 545 | "print(f'Bias Initialized: {seq_model.output_linear.bias.item()}')" 546 | ] 547 | }, 548 | { 549 | "cell_type": "code", 550 | "execution_count": null, 551 | "metadata": {}, 552 | "outputs": [], 553 | "source": [ 554 | "def training_loop(n_epochs, optimizer, model, loss_fn, t_u_train, t_u_val,\n", 555 | "t_c_train, t_c_val):\n", 556 | " for epoch in range(1, n_epochs + 1):\n", 557 | " t_p_train = model(t_u_train)\n", 558 | " loss_train = loss_fn(t_p_train, t_c_train)\n", 559 | " with torch.no_grad():\n", 560 | " t_p_val = model(t_u_val)\n", 561 | " loss_val = loss_fn(t_p_val, t_c_val)\n", 562 | " optimizer.zero_grad()\n", 563 | " loss_train.backward()\n", 564 | " optimizer.step()\n", 565 | "\n", 566 | " if epoch == 1 or epoch % 10 == 0:\n", 567 | " print(f\"Epoch {epoch}, Training loss {loss_train.item():.4f},\"\n", 568 | " f\" Validation loss {loss_val.item():.4f}\")" 569 | ] 570 | }, 571 | { 572 | "cell_type": "markdown", 573 | "metadata": {}, 574 | "source": [ 575 | "**Using Trained Sequential Model to predict on the validation samples.**" 576 | ] 577 | }, 578 | { 579 | "cell_type": "code", 580 | "execution_count": null, 581 | "metadata": {}, 582 | "outputs": [], 583 | "source": [ 584 | "optimizer = optim.SGD(seq_model.parameters(), lr=1e-3)\n", 585 | "\n", 586 | "training_loop(n_epochs = 100, optimizer = optimizer,model = seq_model, loss_fn = nn.MSELoss(),\n", 587 | "t_u_train = train_t_un.unsqueeze(1), t_u_val = val_t_un.unsqueeze(1),\n", 588 | "t_c_train = train_t_c.unsqueeze(1), t_c_val = val_t_c.unsqueeze(1))\n", 589 | "\n", 590 | "print('\\nValidation Set\\'s Predicted Output: ', seq_model(val_t_un.unsqueeze(1)),\"\\n\")\n", 591 | "print('Validation Set\\'s Ground Truth:', val_t_c.unsqueeze(1), '\\n')\n", 592 | "print('Hidden Tensor Weights:', seq_model.hidden_linear.weight.grad)" 593 | ] 594 | }, 595 | { 596 | "cell_type": "markdown", 597 | "metadata": {}, 598 | "source": [ 599 | "### Thanks For Reading. For Feedback, reach out on Github. Please don't spam." 600 | ] 601 | } 602 | ], 603 | "metadata": { 604 | "interpreter": { 605 | "hash": "dca0ade3e726a953b501b15e8e990130d2b7799f14cfd9f4271676035ebe5511" 606 | }, 607 | "kernelspec": { 608 | "display_name": "Python 3", 609 | "language": "python", 610 | "name": "python3" 611 | }, 612 | "language_info": { 613 | "codemirror_mode": { 614 | "name": "ipython", 615 | "version": 3 616 | }, 617 | "file_extension": ".py", 618 | "mimetype": "text/x-python", 619 | "name": "python", 620 | "nbconvert_exporter": "python", 621 | "pygments_lexer": "ipython3", 622 | "version": "3.7.6" 623 | } 624 | }, 625 | "nbformat": 4, 626 | "nbformat_minor": 4 627 | } 628 | -------------------------------------------------------------------------------- /Pytorch Tutorial/README.md: -------------------------------------------------------------------------------- 1 | # Learning PyTorch 2 | 3 |

4 |

PyTorch-I

19 | 20 |

21 |

PyTorch-II

43 | 44 |

45 |

PyTorch-III

46 |
47 | 48 | - Introducing Batch Dimension. 49 | 50 | - Load Batch Of Images (Not Recommended Approach). 51 | 52 | - Normalization 53 | 54 | - Resize. 55 | - Standardization. 56 | - Plotting. 57 | 58 | - Creating One-Hot Encoding. 59 | 60 | - Convert Vector Into One-Hot Encoded Matrix. 61 | - Sample Example On Scatter_ with Zero and One Dimension. 62 | - Filter observation based on Condition. 63 | 64 | - Norm 65 | - L2 Norm 66 | - L1 Norm 67 | 68 |

69 | 70 |

71 |

PyTorch-IV

72 |
73 | 74 | - Handling Time Series Data. 75 | - Handling Text Data. 76 |

77 | 78 |

79 |

PyTorch-V

90 | 91 |

92 |

PyTorch-VI

-------------------------------------------------------------------------------- /Pytorch Tutorial/SampleImages/cat1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mayurji/Learning-PyTorch/fd36b79d3e944285286c769d94f7cbd442f6b8c9/Pytorch Tutorial/SampleImages/cat1.jpg -------------------------------------------------------------------------------- /Pytorch Tutorial/SampleImages/cat2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mayurji/Learning-PyTorch/fd36b79d3e944285286c769d94f7cbd442f6b8c9/Pytorch Tutorial/SampleImages/cat2.jpg -------------------------------------------------------------------------------- /Pytorch Tutorial/SampleImages/dog.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mayurji/Learning-PyTorch/fd36b79d3e944285286c769d94f7cbd442f6b8c9/Pytorch Tutorial/SampleImages/dog.jpg -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![Learning PyTorch](LearningPyTorch.png) 2 | --------------------------------------------------------------------------------