├── Concepts ├── Boosting.ipynb └── README.md ├── LICENSE ├── README.md ├── SpeedUpYourAlgorithms ├── 1) PyTorch.ipynb ├── 2) Numba.ipynb ├── 3) Prallelization.ipynb ├── 4) Dask.ipynb └── README.md └── Tackle ├── BigData-IncrementalLearningAndDask.ipynb ├── HowToVisualize.ipynb └── README.md /Concepts/README.md: -------------------------------------------------------------------------------- 1 | These are files related to my posts on DataScience, MachineLearning, Statistics, Probability etc. 2 | 3 | 1. [Clearing air around Boosting](https://towardsdatascience.com/clearing-air-around-boosting-28452bb63f9e) 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Puneet Grover 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # MediumPosts 2 | Repo for all files related to my Medium Posts 3 | 4 | Medium Profile: [@grover.puneet1995](https://medium.com/@grover.puneet1995) 5 | -------------------------------------------------------------------------------- /SpeedUpYourAlgorithms/1) PyTorch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "Speed_Up_Your_Algo_PyTorch.ipynb", 7 | "version": "0.3.2", 8 | "provenance": [], 9 | "collapsed_sections": [ 10 | "nJyq4ZJ3CL5-", 11 | "IOpLFtlLj_Sf", 12 | "zLBPSTrUkEDN", 13 | "ma-vz8-qCSwK", 14 | "70ana3V1EIFi", 15 | "4oxBruH9pPFV", 16 | "ZrV3b8zWGgoX", 17 | "KGE6gB_CJ4Oi" 18 | ] 19 | }, 20 | "kernelspec": { 21 | "name": "python3", 22 | "display_name": "Python 3" 23 | }, 24 | "accelerator": "GPU" 25 | }, 26 | "cells": [ 27 | { 28 | "metadata": { 29 | "id": "FkDg9fic_5Rl", 30 | "colab_type": "text" 31 | }, 32 | "cell_type": "markdown", 33 | "source": [ 34 | "This is a part of series I am writing, named \"**Speed Up Your Algorithms**\". \n", 35 | "\n", 36 | "Here is the list of posts with files I have written:\n", 37 | "[Github-SpeedUpYourAlgorithms](https://github.com/PuneetGrov3r/MediumPosts/tree/master/SpeedUpYourAlgorithms)" 38 | ] 39 | }, 40 | { 41 | "metadata": { 42 | "id": "nJyq4ZJ3CL5-", 43 | "colab_type": "text" 44 | }, 45 | "cell_type": "markdown", 46 | "source": [ 47 | "# Initial" 48 | ] 49 | }, 50 | { 51 | "metadata": { 52 | "id": "IOpLFtlLj_Sf", 53 | "colab_type": "text" 54 | }, 55 | "cell_type": "markdown", 56 | "source": [ 57 | "### Torch, Torchvision:" 58 | ] 59 | }, 60 | { 61 | "metadata": { 62 | "id": "m9K9aQp_CEw5", 63 | "colab_type": "code", 64 | "outputId": "a89fda99-0c33-4dbd-f6a3-6b3d240e0d41", 65 | "colab": { 66 | "base_uri": "https://localhost:8080/", 67 | "height": 34 68 | } 69 | }, 70 | "cell_type": "code", 71 | "source": [ 72 | "!cat /usr/local/cuda/version.txt" 73 | ], 74 | "execution_count": 0, 75 | "outputs": [ 76 | { 77 | "output_type": "stream", 78 | "text": [ 79 | "CUDA Version 9.2.148\n" 80 | ], 81 | "name": "stdout" 82 | } 83 | ] 84 | }, 85 | { 86 | "metadata": { 87 | "id": "srtGQ9C1AdLT", 88 | "colab_type": "code", 89 | "colab": {} 90 | }, 91 | "cell_type": "code", 92 | "source": [ 93 | "!pip install http://download.pytorch.org/whl/cu92/torch-0.4.1-cp36-cp36m-linux_x86_64.whl\n", 94 | "!pip install torchvision" 95 | ], 96 | "execution_count": 0, 97 | "outputs": [] 98 | }, 99 | { 100 | "metadata": { 101 | "id": "zLBPSTrUkEDN", 102 | "colab_type": "text" 103 | }, 104 | "cell_type": "markdown", 105 | "source": [ 106 | "### Pycuda:" 107 | ] 108 | }, 109 | { 110 | "metadata": { 111 | "id": "8RpbRcfdeuk7", 112 | "colab_type": "code", 113 | "colab": {} 114 | }, 115 | "cell_type": "code", 116 | "source": [ 117 | "# https://medium.com/@iphoenix179/running-cuda-c-c-in-jupyter-or-how-to-run-nvcc-in-google-colab-663d33f53772\n", 118 | "# https://developer.nvidia.com/cuda-90-download-archive?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1704&target_type=deblocal\n", 119 | "!apt update -qq;\n", 120 | "!wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda-repo-ubuntu1704-9-0-local_9.0.176-1_amd64-deb\n", 121 | "!mv cuda-repo-ubuntu1704-9-0-local_9.0.176-1_amd64-deb cuda-repo-ubuntu1704-9-0-local_9.0.176-1_amd64.deb \n", 122 | "!dpkg -i cuda-repo-ubuntu1704-9-0-local_9.0.176-1_amd64.deb\n", 123 | "!apt-key add /var/cuda-repo-9-0-local/7fa2af80.pub\n", 124 | "!apt-get update -qq;\n", 125 | "!apt-get install cuda gcc-5 g++-5 -y -qq;\n", 126 | "!ln -s /usr/bin/gcc-5 /usr/local/cuda/bin/gcc;\n", 127 | "!ln -s /usr/bin/g++-5 /usr/local/cuda/bin/g++;\n", 128 | "#!apt install cuda-9.2;" 129 | ], 130 | "execution_count": 0, 131 | "outputs": [] 132 | }, 133 | { 134 | "metadata": { 135 | "id": "pUBT0zfUFlz5", 136 | "colab_type": "code", 137 | "colab": {} 138 | }, 139 | "cell_type": "code", 140 | "source": [ 141 | "# http://alisonrowland.com/articles/installing-pycuda-via-pip\n", 142 | "# https://codeyarns.com/2015/07/31/pip-install-error-with-pycuda/\n", 143 | "import os\n", 144 | "PATH = os.environ[\"PATH\"]\n", 145 | "os.environ[\"PATH\"] = \"/usr/local/cuda-9.2/bin:/usr/local/cuda/bin:\" + PATH\n", 146 | "os.environ[\"LD_LIBRARY_PATH\"] = \"/usr/local/cuda/lib64\"\n", 147 | "os.environ[\"CUDA_ROOT\"] = \"/usr/local/cuda/\"" 148 | ], 149 | "execution_count": 0, 150 | "outputs": [] 151 | }, 152 | { 153 | "metadata": { 154 | "id": "He-T3N_uGB8i", 155 | "colab_type": "code", 156 | "colab": {} 157 | }, 158 | "cell_type": "code", 159 | "source": [ 160 | "!pip -q install --ignore-installed pycuda" 161 | ], 162 | "execution_count": 0, 163 | "outputs": [] 164 | }, 165 | { 166 | "metadata": { 167 | "id": "ma-vz8-qCSwK", 168 | "colab_type": "text" 169 | }, 170 | "cell_type": "markdown", 171 | "source": [ 172 | "# Import" 173 | ] 174 | }, 175 | { 176 | "metadata": { 177 | "id": "u0CQHYVcKjKs", 178 | "colab_type": "code", 179 | "colab": {} 180 | }, 181 | "cell_type": "code", 182 | "source": [ 183 | "import torchvision\n", 184 | "import torch\n", 185 | "import numpy as np\n", 186 | "import pandas as pd\n", 187 | "from torch import nn\n", 188 | "\n", 189 | "import matplotlib.pyplot as plt" 190 | ], 191 | "execution_count": 0, 192 | "outputs": [] 193 | }, 194 | { 195 | "metadata": { 196 | "id": "dWWAJMSVDjLN", 197 | "colab_type": "code", 198 | "colab": {} 199 | }, 200 | "cell_type": "code", 201 | "source": [ 202 | "import pycuda.driver as cuda\n", 203 | "cuda.init()" 204 | ], 205 | "execution_count": 0, 206 | "outputs": [] 207 | }, 208 | { 209 | "metadata": { 210 | "id": "70ana3V1EIFi", 211 | "colab_type": "text" 212 | }, 213 | "cell_type": "markdown", 214 | "source": [ 215 | "# 2. How to check the availability of cuda?" 216 | ] 217 | }, 218 | { 219 | "metadata": { 220 | "id": "aJTVCSj2A9jS", 221 | "colab_type": "code", 222 | "outputId": "a6079b17-92c3-4ef6-f6dc-cbfedaa0dbe1", 223 | "colab": { 224 | "base_uri": "https://localhost:8080/", 225 | "height": 34 226 | } 227 | }, 228 | "cell_type": "code", 229 | "source": [ 230 | "torch.cuda.is_available() # We are on GPU instance, so cuda is already installed." 231 | ], 232 | "execution_count": 0, 233 | "outputs": [ 234 | { 235 | "output_type": "execute_result", 236 | "data": { 237 | "text/plain": [ 238 | "True" 239 | ] 240 | }, 241 | "metadata": { 242 | "tags": [] 243 | }, 244 | "execution_count": 8 245 | } 246 | ] 247 | }, 248 | { 249 | "metadata": { 250 | "id": "T--ZU71EEkFr", 251 | "colab_type": "text" 252 | }, 253 | "cell_type": "markdown", 254 | "source": [ 255 | "# 3. How to get more info on your cuda devices?" 256 | ] 257 | }, 258 | { 259 | "metadata": { 260 | "id": "h5ujkSp2nVOe", 261 | "colab_type": "code", 262 | "outputId": "13140521-a763-42e3-83dd-b1eaef2de072", 263 | "colab": { 264 | "base_uri": "https://localhost:8080/", 265 | "height": 34 266 | } 267 | }, 268 | "cell_type": "code", 269 | "source": [ 270 | "torch.cuda.current_device() # Get Id of current cuda device" 271 | ], 272 | "execution_count": 0, 273 | "outputs": [ 274 | { 275 | "output_type": "execute_result", 276 | "data": { 277 | "text/plain": [ 278 | "0" 279 | ] 280 | }, 281 | "metadata": { 282 | "tags": [] 283 | }, 284 | "execution_count": 9 285 | } 286 | ] 287 | }, 288 | { 289 | "metadata": { 290 | "id": "dfyIYR9rE4Pg", 291 | "colab_type": "code", 292 | "outputId": "9cdc0288-a47a-4dd1-8c1b-0000380f3a68", 293 | "colab": { 294 | "base_uri": "https://localhost:8080/", 295 | "height": 34 296 | } 297 | }, 298 | "cell_type": "code", 299 | "source": [ 300 | "cuda.Device(0).name() # get name of device using pycuda" 301 | ], 302 | "execution_count": 0, 303 | "outputs": [ 304 | { 305 | "output_type": "execute_result", 306 | "data": { 307 | "text/plain": [ 308 | "'Tesla K80'" 309 | ] 310 | }, 311 | "metadata": { 312 | "tags": [] 313 | }, 314 | "execution_count": 10 315 | } 316 | ] 317 | }, 318 | { 319 | "metadata": { 320 | "id": "PdfRQs65m_7I", 321 | "colab_type": "code", 322 | "outputId": "5a2569b1-d95e-4e3a-a059-969fa14c384b", 323 | "colab": { 324 | "base_uri": "https://localhost:8080/", 325 | "height": 34 326 | } 327 | }, 328 | "cell_type": "code", 329 | "source": [ 330 | "torch.cuda.get_device_name(0) # or using torch" 331 | ], 332 | "execution_count": 0, 333 | "outputs": [ 334 | { 335 | "output_type": "execute_result", 336 | "data": { 337 | "text/plain": [ 338 | "'Tesla K80'" 339 | ] 340 | }, 341 | "metadata": { 342 | "tags": [] 343 | }, 344 | "execution_count": 11 345 | } 346 | ] 347 | }, 348 | { 349 | "metadata": { 350 | "id": "A8ALco5IngOu", 351 | "colab_type": "code", 352 | "outputId": "5f44950f-b69a-4e7c-80c3-667e3efcd728", 353 | "colab": { 354 | "base_uri": "https://localhost:8080/", 355 | "height": 67 356 | } 357 | }, 358 | "cell_type": "code", 359 | "source": [ 360 | "# Simple class for getting info on all cuda compatible devices\n", 361 | "import pycuda.autoinit\n", 362 | "\n", 363 | "class aboutCudaDevices():\n", 364 | " def __init__(self):\n", 365 | " pass\n", 366 | " \n", 367 | " def num_devices(self):\n", 368 | " \"\"\"Return number of devices connected.\"\"\"\n", 369 | " return cuda.Device.count()\n", 370 | " \n", 371 | " def devices(self):\n", 372 | " \"\"\"Get info on all devices connected.\"\"\"\n", 373 | " num = cuda.Device.count()\n", 374 | " print(\"%d device(s) found:\"%num)\n", 375 | " for i in range(num):\n", 376 | " print(cuda.Device(i).name(), \"(Id: %d)\"%i)\n", 377 | " \n", 378 | " def mem_info(self):\n", 379 | " \"\"\"Get available and total memory of all devices.\"\"\"\n", 380 | " available, total = cuda.mem_get_info()\n", 381 | " print(\"Available: %.2f GB\\nTotal: %.2f GB\"%(available/1e9, total/1e9))\n", 382 | " \n", 383 | " def attributes(self, device_id=0):\n", 384 | " \"\"\"Get attributes of device with device Id = device_id\"\"\"\n", 385 | " return cuda.Device(device_id).get_attributes()\n", 386 | " \n", 387 | " def __repr__(self):\n", 388 | " \"\"\"Class representation as number of devices connected and about them.\"\"\"\n", 389 | " num = cuda.Device.count()\n", 390 | " string = \"\"\n", 391 | " string += (\"%d device(s) found:\\n\"%num)\n", 392 | " for i in range(num):\n", 393 | " string += ( \" %d) %s (Id: %d)\\n\"%((i+1),cuda.Device(i).name(),i))\n", 394 | " string += (\" Memory: %.2f GB\\n\"%(cuda.Device(i).total_memory()/1e9))\n", 395 | " return string\n", 396 | "\n", 397 | "# You can print output just by typing its name (__repr__):\n", 398 | "aboutCudaDevices()" 399 | ], 400 | "execution_count": 0, 401 | "outputs": [ 402 | { 403 | "output_type": "execute_result", 404 | "data": { 405 | "text/plain": [ 406 | "1 device(s) found:\n", 407 | " 1) Tesla K80 (Id: 0)\n", 408 | " Memory: 12.00 GB" 409 | ] 410 | }, 411 | "metadata": { 412 | "tags": [] 413 | }, 414 | "execution_count": 12 415 | } 416 | ] 417 | }, 418 | { 419 | "metadata": { 420 | "id": "BAyU2477owkl", 421 | "colab_type": "code", 422 | "outputId": "4a4f096c-1c28-46a3-a391-974d5d2d1403", 423 | "colab": { 424 | "base_uri": "https://localhost:8080/", 425 | "height": 34 426 | } 427 | }, 428 | "cell_type": "code", 429 | "source": [ 430 | "# Lets check how much memory is allocated:\n", 431 | "torch.cuda.memory_allocated()" 432 | ], 433 | "execution_count": 0, 434 | "outputs": [ 435 | { 436 | "output_type": "execute_result", 437 | "data": { 438 | "text/plain": [ 439 | "0" 440 | ] 441 | }, 442 | "metadata": { 443 | "tags": [] 444 | }, 445 | "execution_count": 13 446 | } 447 | ] 448 | }, 449 | { 450 | "metadata": { 451 | "id": "Equ6KD99oEMT", 452 | "colab_type": "code", 453 | "outputId": "7333a930-b7f7-4b13-8ea8-d1dd1015b0f2", 454 | "colab": { 455 | "base_uri": "https://localhost:8080/", 456 | "height": 34 457 | } 458 | }, 459 | "cell_type": "code", 460 | "source": [ 461 | "# Let allocate some arrays to GPU:\n", 462 | "mat_a = torch.cuda.FloatTensor((100., 100.))\n", 463 | "mat_a" 464 | ], 465 | "execution_count": 0, 466 | "outputs": [ 467 | { 468 | "output_type": "execute_result", 469 | "data": { 470 | "text/plain": [ 471 | "tensor([100., 100.], device='cuda:0')" 472 | ] 473 | }, 474 | "metadata": { 475 | "tags": [] 476 | }, 477 | "execution_count": 14 478 | } 479 | ] 480 | }, 481 | { 482 | "metadata": { 483 | "id": "O6b6mf59oXhe", 484 | "colab_type": "code", 485 | "outputId": "6ff6867f-f802-4d3a-d4f9-f15041a8ee6d", 486 | "colab": { 487 | "base_uri": "https://localhost:8080/", 488 | "height": 34 489 | } 490 | }, 491 | "cell_type": "code", 492 | "source": [ 493 | "# Lets check how muc memory is allocated now:\n", 494 | "torch.cuda.memory_allocated()" 495 | ], 496 | "execution_count": 0, 497 | "outputs": [ 498 | { 499 | "output_type": "execute_result", 500 | "data": { 501 | "text/plain": [ 502 | "512" 503 | ] 504 | }, 505 | "metadata": { 506 | "tags": [] 507 | }, 508 | "execution_count": 15 509 | } 510 | ] 511 | }, 512 | { 513 | "metadata": { 514 | "id": "Al9L1fBooqx-", 515 | "colab_type": "code", 516 | "outputId": "ef47df7f-07e3-49b1-fd75-2a45e3a8d9ad", 517 | "colab": { 518 | "base_uri": "https://localhost:8080/", 519 | "height": 34 520 | } 521 | }, 522 | "cell_type": "code", 523 | "source": [ 524 | "# Memory used by Cache Allocator:\n", 525 | "torch.cuda.memory_cached()" 526 | ], 527 | "execution_count": 0, 528 | "outputs": [ 529 | { 530 | "output_type": "execute_result", 531 | "data": { 532 | "text/plain": [ 533 | "1048576" 534 | ] 535 | }, 536 | "metadata": { 537 | "tags": [] 538 | }, 539 | "execution_count": 16 540 | } 541 | ] 542 | }, 543 | { 544 | "metadata": { 545 | "id": "V0JshoUtDPM6", 546 | "colab_type": "code", 547 | "colab": {} 548 | }, 549 | "cell_type": "code", 550 | "source": [ 551 | "# These memory methods are only available for GPUs. And that's where they are actually needed." 552 | ], 553 | "execution_count": 0, 554 | "outputs": [] 555 | }, 556 | { 557 | "metadata": { 558 | "id": "4oxBruH9pPFV", 559 | "colab_type": "text" 560 | }, 561 | "cell_type": "markdown", 562 | "source": [ 563 | "# 4. How to store Tensors and run Models on GPU?" 564 | ] 565 | }, 566 | { 567 | "metadata": { 568 | "id": "54xGuTWvo5Z3", 569 | "colab_type": "code", 570 | "outputId": "8b36a64e-f254-4af8-e78f-00848db43ea6", 571 | "colab": { 572 | "base_uri": "https://localhost:8080/", 573 | "height": 34 574 | } 575 | }, 576 | "cell_type": "code", 577 | "source": [ 578 | "# For storing something on CPU:\n", 579 | "Mat_cpu = torch.FloatTensor([1., 2.])\n", 580 | "Mat_cpu" 581 | ], 582 | "execution_count": 0, 583 | "outputs": [ 584 | { 585 | "output_type": "execute_result", 586 | "data": { 587 | "text/plain": [ 588 | "tensor([1., 2.])" 589 | ] 590 | }, 591 | "metadata": { 592 | "tags": [] 593 | }, 594 | "execution_count": 18 595 | } 596 | ] 597 | }, 598 | { 599 | "metadata": { 600 | "id": "4HzPhPO5EIVm", 601 | "colab_type": "code", 602 | "outputId": "ee17ef47-f8c7-4f0a-b912-fbfaa9b037a6", 603 | "colab": { 604 | "base_uri": "https://localhost:8080/", 605 | "height": 34 606 | } 607 | }, 608 | "cell_type": "code", 609 | "source": [ 610 | "# To put this on GPU:\n", 611 | "Mat_gpu = Mat_cpu.cuda()\n", 612 | "Mat_gpu, Mat_cpu" 613 | ], 614 | "execution_count": 0, 615 | "outputs": [ 616 | { 617 | "output_type": "execute_result", 618 | "data": { 619 | "text/plain": [ 620 | "(tensor([1., 2.], device='cuda:0'), tensor([1., 2.]))" 621 | ] 622 | }, 623 | "metadata": { 624 | "tags": [] 625 | }, 626 | "execution_count": 19 627 | } 628 | ] 629 | }, 630 | { 631 | "metadata": { 632 | "id": "4EXubaSKETq2", 633 | "colab_type": "code", 634 | "outputId": "6392d2b2-e7e6-4251-fd11-28e2ce904a4b", 635 | "colab": { 636 | "base_uri": "https://localhost:8080/", 637 | "height": 34 638 | } 639 | }, 640 | "cell_type": "code", 641 | "source": [ 642 | "# Or you could have done:\n", 643 | "Mat_gpu2 = torch.cuda.FloatTensor([1., 2.])\n", 644 | "Mat_gpu2" 645 | ], 646 | "execution_count": 0, 647 | "outputs": [ 648 | { 649 | "output_type": "execute_result", 650 | "data": { 651 | "text/plain": [ 652 | "tensor([1., 2.], device='cuda:0')" 653 | ] 654 | }, 655 | "metadata": { 656 | "tags": [] 657 | }, 658 | "execution_count": 20 659 | } 660 | ] 661 | }, 662 | { 663 | "metadata": { 664 | "id": "WX3ROV-vFAMG", 665 | "colab_type": "code", 666 | "colab": {} 667 | }, 668 | "cell_type": "code", 669 | "source": [ 670 | "# Putting Model on GPU:\n", 671 | "sq = nn.Sequential(\n", 672 | " nn.Linear(20, 20),\n", 673 | " nn.ReLU(),\n", 674 | " nn.Linear(20, 4),\n", 675 | " nn.Softmax()\n", 676 | ")" 677 | ], 678 | "execution_count": 0, 679 | "outputs": [] 680 | }, 681 | { 682 | "metadata": { 683 | "id": "oiLwGOdtGJqM", 684 | "colab_type": "code", 685 | "colab": {} 686 | }, 687 | "cell_type": "code", 688 | "source": [ 689 | "model = sq.cuda()" 690 | ], 691 | "execution_count": 0, 692 | "outputs": [] 693 | }, 694 | { 695 | "metadata": { 696 | "id": "PfZlJIUUGSoJ", 697 | "colab_type": "code", 698 | "outputId": "7e2f2c17-32e1-4cc2-c1ba-b0383899db44", 699 | "colab": { 700 | "base_uri": "https://localhost:8080/", 701 | "height": 34 702 | } 703 | }, 704 | "cell_type": "code", 705 | "source": [ 706 | "# To check if model is on GPU:\n", 707 | "# From the discussions here: discuss.pytorch.org/t/how-to-check-if-model-is-on-cuda\n", 708 | "next(model.parameters()).is_cuda" 709 | ], 710 | "execution_count": 0, 711 | "outputs": [ 712 | { 713 | "output_type": "execute_result", 714 | "data": { 715 | "text/plain": [ 716 | "True" 717 | ] 718 | }, 719 | "metadata": { 720 | "tags": [] 721 | }, 722 | "execution_count": 23 723 | } 724 | ] 725 | }, 726 | { 727 | "metadata": { 728 | "id": "FwIa7hAk7DDb", 729 | "colab_type": "text" 730 | }, 731 | "cell_type": "markdown", 732 | "source": [ 733 | "# CPU vs GPU" 734 | ] 735 | }, 736 | { 737 | "metadata": { 738 | "id": "_4QjDwoa1Xw7", 739 | "colab_type": "code", 740 | "outputId": "60ac43b7-81a3-459b-e212-4237c7746f72", 741 | "colab": { 742 | "base_uri": "https://localhost:8080/", 743 | "height": 34 744 | } 745 | }, 746 | "cell_type": "code", 747 | "source": [ 748 | "model_cpu = nn.Sequential(\n", 749 | " nn.Linear(20, 20),\n", 750 | " nn.ReLU(),\n", 751 | " nn.Linear(20, 4),\n", 752 | " nn.Softmax()\n", 753 | ").float()\n", 754 | "model_gpu = model.float()\n", 755 | "next(model_cpu.parameters()).is_cuda, next(model_gpu.parameters()).is_cuda" 756 | ], 757 | "execution_count": 0, 758 | "outputs": [ 759 | { 760 | "output_type": "execute_result", 761 | "data": { 762 | "text/plain": [ 763 | "(False, True)" 764 | ] 765 | }, 766 | "metadata": { 767 | "tags": [] 768 | }, 769 | "execution_count": 37 770 | } 771 | ] 772 | }, 773 | { 774 | "metadata": { 775 | "id": "s21Imowu3woJ", 776 | "colab_type": "code", 777 | "colab": {} 778 | }, 779 | "cell_type": "code", 780 | "source": [ 781 | "cpu_inp = torch.ones((100000, 20), dtype=torch.float32)\n", 782 | "gpu_inp = torch.ones((100000, 20), dtype=torch.float32).cuda()" 783 | ], 784 | "execution_count": 0, 785 | "outputs": [] 786 | }, 787 | { 788 | "metadata": { 789 | "id": "6zTaBUiS4E5u", 790 | "colab_type": "code", 791 | "outputId": "895c3847-96b3-4a3d-8ca2-6e65f5f66535", 792 | "colab": { 793 | "base_uri": "https://localhost:8080/", 794 | "height": 87 795 | } 796 | }, 797 | "cell_type": "code", 798 | "source": [ 799 | "%timeit res = model_cpu(cpu_inp)" 800 | ], 801 | "execution_count": 0, 802 | "outputs": [ 803 | { 804 | "output_type": "stream", 805 | "text": [ 806 | "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py:91: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.\n", 807 | " input = module(input)\n" 808 | ], 809 | "name": "stderr" 810 | }, 811 | { 812 | "output_type": "stream", 813 | "text": [ 814 | "10 loops, best of 3: 28 ms per loop\n" 815 | ], 816 | "name": "stdout" 817 | } 818 | ] 819 | }, 820 | { 821 | "metadata": { 822 | "id": "jX0ENIk_4K5D", 823 | "colab_type": "code", 824 | "outputId": "4e24e52b-2caf-434b-bb15-318ebb7cbb0a", 825 | "colab": { 826 | "base_uri": "https://localhost:8080/", 827 | "height": 87 828 | } 829 | }, 830 | "cell_type": "code", 831 | "source": [ 832 | "%%timeit \n", 833 | "for i in range(10): # Because we don't have enough memory on GPU\n", 834 | " res = model_gpu(gpu_inp[i*1000:i*1000+1000])" 835 | ], 836 | "execution_count": 0, 837 | "outputs": [ 838 | { 839 | "output_type": "stream", 840 | "text": [ 841 | "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py:91: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.\n", 842 | " input = module(input)\n" 843 | ], 844 | "name": "stderr" 845 | }, 846 | { 847 | "output_type": "stream", 848 | "text": [ 849 | "100 loops, best of 3: 3.61 ms per loop\n" 850 | ], 851 | "name": "stdout" 852 | } 853 | ] 854 | }, 855 | { 856 | "metadata": { 857 | "id": "7-U_851D4UWx", 858 | "colab_type": "code", 859 | "outputId": "2d2f6fd4-4d32-4b32-ad59-794e28b0f0a2", 860 | "colab": { 861 | "base_uri": "https://localhost:8080/", 862 | "height": 417 863 | } 864 | }, 865 | "cell_type": "code", 866 | "source": [ 867 | "fig, ax = plt.subplots(1, 1)\n", 868 | "ax.plot([\"a\", \"b\"], [28, 3.61], \"-o\")\n", 869 | "ax.set_xticklabels([\"CPU Model\", \"GPU Model\"])\n", 870 | "fig.suptitle(\"CPU vs GPU\", fontsize=17)\n", 871 | "ax.set_ylabel(\"Time (ms)\")\n", 872 | "ax.set_xlabel(\"Model Type\")" 873 | ], 874 | "execution_count": 0, 875 | "outputs": [ 876 | { 877 | "output_type": "execute_result", 878 | "data": { 879 | "text/plain": [ 880 | "Text(0.5,0,'Model Type')" 881 | ] 882 | }, 883 | "metadata": { 884 | "tags": [] 885 | }, 886 | "execution_count": 54 887 | }, 888 | { 889 | "output_type": "display_data", 890 | "data": { 891 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfIAAAF/CAYAAACyv0vWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XlcVXX+x/HXuZddUVYBEXcEFDVX\nxDWXXFvcMjfS1GmZyWymZWrMprLG0bYx2xUzNbXUNAtNM7MxF8wtRXFXxAVERQHZ4f7+sPjFJCIK\nXC68n49Hf3A495zPbR7Nm/f5nnuuYbFYLIiIiIhNMll7ABEREbl1CnIREREbpiAXERGxYQpyERER\nG6YgFxERsWEKchERERumIBe5Tdu3b+fxxx+nU6dOhIaG0rlzZ8aPH8+3335baL8vv/ySoKCgQv8E\nBwcTHh7Ok08+yZEjRwr2PX36NEFBQcybN++650xJSSEoKIhZs2aV5Vu7obS0ND766CMGDx5MmzZt\naNasGZ07d+bJJ59k27Ztf9i/R48ef3j/zZs3595772XevHnk5eUV7Pvcc8/Rtm3bIs/92muvERQU\nVCbvS8TW2Fl7ABFb9sEHHzBz5kyGDh3KO++8g6+vL+fOnWPFihVMmjSJBx54gFdeeaXQa+bNm0fj\nxo0ByMnJ4fjx47z11lsMHz6clStXEhAQYI23UiJnzpxh3Lhx5Ofn89hjj9GqVSvMZjPHjx9n/vz5\njB07lueff54xY8YUel2XLl2YNm1awc+XLl1iw4YNTJ8+nfj4eKZMmVLeb0XE5inIRW7R5s2b+c9/\n/sNTTz3Fww8/XLDd39+ftm3bUqdOHWbOnMnAgQNp3bp1we/d3Nzw9vYu+Ll27doEBgbStWtXlixZ\nwjPPPFOu7+NWPPXUU+Tk5LB8+XLc3d0LttetW5euXbvyl7/8hWXLljFkyBCqV69e8HsHB4dC793b\n25ugoCDOnj3LkiVL+Otf/1pofxEpni6ti9yiuXPn4u/vz4QJE677+/Hjx7Nx48ZCIV4UHx8fPDw8\nOH36dKnMFh8fT1BQEHPnzv3D7x577DF69uyJxWIhISGBp59+mi5duhAaGkrXrl2ZMmUKKSkpRR57\nx44d7N69m4kTJxYK8d+YTCZmzZrF119/fdOhHBwcTG5uLgkJCTf/JkUEUJCL3JLc3Fx+/vlnunTp\ngsl0/f+MHBwc8PX1vanjXb58mcuXL1O7du1SmS8gIIBWrVqxZs2aQttTUlLYtGkTAwcOxDAMnn76\naQ4ePMjMmTP57rvvmDFjBjt27ODZZ58t8tjR0dEAdOvWrch97OxKdrHv5MmTmM1matWqVaLXiYgu\nrYvcksuXL5OVlYW/v/9tH+v06dNMnToVR0dH7r///lKY7pr77ruPl156iTNnzhTMuXbtWnJzcxk4\ncCAA+/btK3Tp38/Pj8jISNLS0oo8bmJiIk5OTnh4eNz2jFlZWWzYsIEvvviCQYMGUaNGjds+pkhV\noyAXuQWGYdzya4cPH17Q4nNzc8nOziY0NJS5c+fSsGHD0hqRfv368dprr7FmzZqCy/9RUVG0adOm\n4Ia6u+66iy+++ILs7Gy6d+9O+/bti70qYDKZCt1h/psPP/yQjz76qNC2Rx55hEcffbTg540bN9Kq\nVauCnzMyMnBxcWHkyJH89a9/veX3KlKVKchFboGbmxsuLi6cPHmyxK99++23adSoEXAtFGvWrPmH\nJmo2m294jN+C9EaXsN3c3LjzzjsLgjwpKYnt27fz8ssvF+wzbdo0WrduzapVq1i1ahUWi4Xw8HD+\n8Y9/FMz4v2rXrk1OTg5nz54tFPrDhw+nX79+BT/37t2bnJycQq8NCwvjpZdeKvj5t5vf/vd9mEwm\nbvTFjPn5+SW+fC9SWWmNXOQWmM1mwsLC2LhxI1lZWdfdx2KxsHDhwj/cOObn50e9evWoV68eAQEB\n172c7O7ujslk4sKFC9c99m83hRW3Bn/fffcRExNDfHw8a9aswd7evlDY2tvbM3LkSJYsWUJ0dDSv\nv/46R48e5U9/+lORQdqpUycA1q1bV2i7m5tbwfuqV6/edV/r7OxcaB8/P7/rBrKXlxfp6elkZGRc\n9zjnzp3Dx8fnhu9dpKpQkIvcogkTJnDp0iVef/316/4+MjKSqVOnsnv37hIf28nJibCwMFavXk16\nevoffj9v3jycnJzo0qXLDY/TrVs33Nzc+P7771m9ejW9evUquJM8OTmZr776iqtXrwJQvXp1BgwY\nwJgxYzhz5gwXL1687jGbNWtG165d+eCDD4iPj7/uPseOHSvJ2/2Drl27kp+fz/Lly6977E2bNnHX\nXXfd1jlEKgvzS7+/ziUiN6127do4OTnxwQcfcOjQITw8PDAMg+PHjzNr1iwiIyN58sknGTp0KACx\nsbF8//33DB8+vNBnqYvSrFkzFi9ezObNm/H19S049jvvvMPKlSuZPHkyYWFhNzyG2Wzm9OnT/Pjj\nj8TExPD3v/+dunXrAtfWp8eNG8ehQ4cICAjAYrFw6NAhZs+ejYeHB+PHjy/yXoBOnTqxYcMGPv30\nU+zt7alWrRoZGRkcPnyYBQsWMGXKFGrXrs2kSZPw9PQE4NNPP8XLy4sBAwbc1L/bxMRE5s2bh8lk\nwtXVlZSUFDZu3MjkyZPx8PBg+vTpODk5FXsskcrOsNxoIUpEirVr1y7mzZvHrl27uHz5Ml5eXjRr\n1owxY8bQvn37gv2+/PJLnn/+eVauXElISMhNHTs+Pp7333+fzZs3c/HiRapXr06LFi0YN24c4eHh\nNz3fiBEj8PHxYePGjYU+Lrd3715mzpxJTEwMV69excvLi/DwcCZNmlTsZfvs7GwWL17MmjVrOHr0\nKJmZmbi7uxMcHMxdd93FwIEDcXBwKNi/R48eBAcH8/7779/U3BaLhWXLlvHFF19w9OhRcnJyqFOn\nDr179+bhhx/Wg2NEfqUgFxERsWFaIxcREbFhCnIREREbpiAXERGxYQpyERERG6YgFxERsWEKchER\nERumIBcREbFhCnIREREbpiAXERGxYQpyERERG6YgFxERsWEKchERERumIBcREbFhCnIREREbpiAX\nERGxYQpyERERG6YgFxERsWEKchERERumIBcREbFhCnIREREbpiAXERGxYQpyERERG6YgFxERsWEK\nchERERumIBcREbFhCnIREREbZmftAW4kKSm11I/p7u5CcnJ6qR9XRESqptLOFW9v1xLtX+UauZ2d\n2dojiIhIJWLtXKlyQS4iIlKZKMhFRERsmIJcRETEhinIRUREbJiCXERExIYpyEVERGyYglxERMSG\nKchFRERsWIV+sltpij6QSNTWk5y9mE5tTxcGhNcnrKmPtccSERG5LVUiyKMPJPLRqv0FP59Oulrw\ns8JcRERsWZW4tB619WQR2+PKdQ4REZHSViWC/OyF6z/M/tzFq+U8iYiISOmqEkFe28vlutu93ZzL\neRIREZHSVSWCfEB4/etuP5+cwZroOPLzLeU7kIiISCmpEkEe1tSHR+5tRh3v6phNBnW8q9O7XQDV\nne1Y+sMxpn22U5fZRUTEJhkWi6XC1tGkpNRSP6a3t2vBcVPTs/nsu8Nsjz2PvZ2JQV0a0rtdACaT\nUernFRGRyun3uVJaxyuJKtHIi+Lq4sCj94Xy54GhODmY+eKHo/z7s10kXLr+zXEiIiIVTZVu5L+X\nkp7NZ+sO8/PBa+18SLdG9GpbB5Ohdi4iIkVTI68garg48NjAUB4bGIqjvZkl3x9hxme7SExWOxcR\nkYpLQf4/2gXX4tUJYbQJ8ubw6Sv8M3I763fEk19xL1yIiEgVpiC/jhrVHPjzwFAeva8ZDvZmFq0/\nwuuLdnP+coa1RxMRESlEa+TFuHI1mwVrD7HrcBIO9ibuv7Mx3Vv7a+1cREQArZFXeDWrOfCXQaE8\nfE9T7M0mPvvuMG8s3k2S2rmIiFQAauQlcCUti/lrD7H7yAUc7c0M696Ibq3UzkVEqjI1chtSs7oj\njw9uzp/uboqd2WDBusO8uWQPF66onYuIiHUoyEvIMAzCQ315ZXwYLRt5EhuXzJTI7WzcfYYKfHFD\nREQqKQX5LXJ3deSJoS0YPyAEs2Ewf+0h3vp8DxevZFp7NBERqUIU5LfBMAw6Nfdj6oQwWjTyZP/J\nZKZERvPfX86qnYuISLlQkJcCd1dHJg1twbj+IRgGzFtzkLe/+IVLKWrnIiJSthTkpcQwDDq38GPq\n+DBCG3oQc+ISUyKj2aR2LiIiZUhBXso8ajjx1/tbMrZfMACfrDnIf5buJTk1y8qTiYhIZaQgLwOG\nYdC1ZW2mjg+jWQMP9h2/yAtzotm875zauYiIlCoFeRnyqOHE34a1ZEzfICwWC5FRscxcpnYuIiKl\nR0FexgzDoNsd/rwyvj0h9dzZe+wiU+ZEsyVG7VxERG6fgryceNV05unhdxDRJ4i8fAtzvoll1vJ9\nXE5TOxcRkVunZ61bwYXLGXyy5iCxcclUc7Jj5F1N6NDUB0PPbBcRsTl61noV5OXmzFPD72B07ybk\n5lmY/fUB3v1yH1euZlt7NBERsTF21h6gqjIZBj1a1yG0oSefRMWy+8gFDsdvY1TvJoSFqJ2LiMjN\nUSO3slpuzjwzshWj7mpCTl4+H686wPsrYkhROxcRkZugRl4BmAyDnm3q0LyhB3OjYtl5OIlD8ZcZ\n3bsJ7UN8rD2eiIhUYGrkFUgtdxeeHdWaEb0Cyc7J48Ov9vP+yhhS0tXORUTk+tTIKxiTYXBX2wBa\nNPQkcnUsOw6e59CpZCJ6B9E2uJa1xxMRkQqmTD9+NmPGDHbu3Elubi6PPPIIGzZsYP/+/bi5uQEw\nfvx47rzzziJfX1k/fnaz8vMtrN8Rz/L/HicnN5/2IbUYdVcTXF0crD2aiIj8ytofPyuzRr5t2zaO\nHDnC559/TnJyMoMGDaJDhw787W9/o3v37mV12krFZDLo3b4uzRt5Mnd1LNtjz3MwLpmIPsG0CfK2\n9ngiIlIBlNkaebt27Zg5cyYANWrUICMjg7y8vLI6XaXm51mN50e1YVj3xqRn5fHein18tGo/aRk5\n1h5NRESsrFye7Pb555+zY8cOzGYzSUlJ5OTk4OnpyZQpU/Dw8Cjydbm5edjZmct6PJsSn5jKzCW7\nOXQqGTdXR/4ytCUdQv2sPZaIiFhJmQf5+vXr+eijj5g7dy4xMTG4ubkREhLCxx9/TEJCAi+++GKR\nr63qa+RFyc+3sHb7KVZsOkFuXj7hzXwY0asJ1Z3trT2aiEiVY+018jL9+NmmTZv48MMPmT17Nq6u\nroSHhxMSEgJAjx49OHz4cFmevtIymQz6dajHPx9qRwM/V7buT2TKnGj2HLlg7dFERKSclVmQp6am\nMmPGDD766KOCu9QnTpxIfHw8ANHR0QQGBpbV6asEf69q/COiDUO6NeRqZg7vLN/LnG8OcDVTa+ci\nIlVFmd21vnr1apKTk3nyyScLtg0ePJgnn3wSZ2dnXFxcmDZtWlmdvsowm0wMCK/PHY29mBMVy5aY\nBA6cvMTYfsG0aORl7fFERKSM6WtMK5G8/HzWbDvFVz+dIC/fQqfmvozoGYiLk9bORUTKirXXyPVk\nt0rEbDJxd8dr7TwyKpbN+xI4cDKZsf2Cad7Q09rjiYhIGdCz1iuhOrWqM/nBNgzs0oCUq9m8/cUv\nfLI6lvTMXGuPJiIipUyX1iu5U4mpzI2K5dT5NNxdHXmofzChDdTORURKi7UvrauRV3J1fVx5YUxb\n7ut8rZ2/9fkvzFtzkIwstXMRkcpAjbwKOZWYypxvYjmdlIZnDUfG9g+hWf2in6wnIiLFUyOXclPX\nx5UXx7blno71SU7N5s0le5j/rdq5iIgtUyOvok4mpBAZFcuZpKt41nBiXP9gQtTORURKTI1crKK+\nbw1eHNOOuzvWIzk1i9eX7GHBukNkZqudi4jYEjVy4cS5a+387IWreNV0Ylz/EILruVt7LBERm2Dt\nRq4gFwBycvNZtfkEq7fFYbFAz9Z1GHpnIxwd9DWyIiI3oiC/AQV5+Tt+NoXIqAOcu5iOt9u1dh5U\nV+1cRKQoCvIbUJBbR05uHis3neDb7afAAj3b1mFIt0Y42qudi4j8LwX5DSjIrevYmStERsWScCmd\nWu7OjOsfQpMAN2uPJSJSoSjIb0BBbn3ZOdfa+drtpwC4q10Ag7o2VDsXEfmVgvwGFOQVx9HTV4iM\nOkBicgY+7s6MH9CUxnVqWnssERGrU5DfgIK8YsnKyWPFf4/z3c/xAPRpX5eBXRrgoHYuIlWYgvwG\nFOQV0+H4y8xdHcv55Ax8PVwYPyCERv5q5yJSNSnIb0BBXnFl5eSx/MdjfL/jNBjQ99d2bm+ndi4i\nVYu1g1yPaJVb4mhvZmSvJjw7shVeNZ1YE32Klz75meNnU6w9mohIlaJGLrctKzuPZT8e4/udpzEM\n6BdWj/s6N8DeTn8nikjlp0YuNs/Rwcyou5rw7IhWeNZwYvW2OF6Z9zMnzqmdi4iUNTVyKVWZ2bks\n23iMDbvOYDIM+nWoy72d1M5FpPJSI5dKxcnBjtG9g3hm+B24uzoStTWOVz79mbgE/fEkIlIW1Mil\nzGRk5bJ04zE27r7WzgeE1+OeTvWxM+vvRxGpPNTIpdJydrTjwT5BPDX8DtxdHfh6y0lembeDU4n6\nQ0pEpLSokUu5yMjK5YsfjvLjnrOYTdfa+d0d1c5FxPapkUuV4Oxox5i+wfztgZbUrO7Aqs0nefVT\ntXMRkdulRi7lLj0zl883HGHT3nOYTQb3dKpP/w711M5FxCapkUuV4+Jkx0P9Q3jy/pbUqObAyk0n\neG3+Tk6fT7P2aCIiNkeNXKwqPTOHJd8f5ad919r5vZ0b0L9DXcwm/Y0pIrZBjVyqNBcne8YNCOHJ\n+1vg6mLPiv8e57X5OzmTpHYuInIz1MilwriamcPi9UfYEpOAndngvs4N6Bumdi4iFZsaucivqjnZ\nM+HupjwxpAXVnOxZ/uNx/rVgJ2cuXLX2aCIiFZaCXCqcOwK9mDohjPBmPpw4l8rLn/zMmm1x5OdX\n2ItHIiJWoyCXCqm6sz1/uqcZEwc3x8XJjqUbj/GvhTs5d1HtXETk97RGLhVeWkYOi747zLYDidiZ\nTQzu2pDe7QIwmQxrjyYiojVykeJUd7bn4Xub8ZdBzXFxNPPFD0eZ9tlOEi6lW3s0ERGrUyMXm5Ka\nns1n3x1me+x57O1MDOnakF5t1c5FxHrUyEVKwNXFgUfvC+XPA0NxcjCzZMNR/r1oF4lq5yJSRSnI\nxSa1Da7F1AlhtA2uxdHTV/jn3O1893M8+RX3ApOISJlQkIvNquHiwJ8HhvLofc1wsDez+PsjzPhs\nF4nJauciUnUoyMXmtQ/xYeqEMNo08ebwr+18/Q61cxGpGhTkUinUrObAnweF8si9zbA3m1i0/giv\nL9rN+csZ1h5NRKRM6a51qXSupGUxf+0hdh+5gKO9maF3NqJ7a39Mhu5sF5HSp7vWRUpZzeqOPD64\nOQ/f0xQ7s8Fn3x3mjcW7uaB2LiKVkIJcKiXDMOjQzJepE8K4o7EXB09dZsrc7fyw+wwV+CKUiEiJ\nKcilUnOr7sjEIc2ZcHcIZsNgwdpDvPn5Hi5cUTsXkcqhTNfIZ8yYwc6dO8nNzeWRRx6hefPmPPvs\ns+Tl5eHt7c3rr7+Og4NDka/XGrmUpuTULOZ/e5Bfjl3EycHMsB6N6dayNobWzkXkNlh7jbzMgnzb\ntm1ERkYye/ZskpOTGTRoEOHh4XTt2pV+/frx1ltv4evry8iRI4s8hoJcSpvFYmFLTAKL1h8hIyuX\nZvXdGdsvBM+aTtYeTURsVKUN8ry8PLKysnBxcSEvL4+OHTtSrVo1vv32WxwcHNi9ezdz585l1qxZ\nRR5DQS5lJTk1i3lrDrLv+LV2PrxnIF1a+Kmdi0iJWTvI7UrtzP/DbDbj4uICwLJly+jatSs//fRT\nwaV0T09PkpKSbngMd3cX7OzMpT5bSf8lSeXj7e3Ka3/uxPrtp5izKoZ5aw6y98QlJt5/B15uztYe\nT0RsjDVzpcyC/Dfr169n2bJlzJ07l969exdsv5kLAcll8KhNNXL5vTsaevDKuPbMW3OQXQfP8+cZ\nGxjeszGdm6udi8jNsXYjL9O71jdt2sSHH37I7NmzcXV1xcXFhczMTAASExOpVatWWZ5e5KZ41HDi\nr8NaMrZfMBaLhU9WH2Tmsr0kp2ZZezQRkWKVWZCnpqYyY8YMPvroI9zc3ADo2LEja9euBWDdunV0\n6dKlrE4vUiKGYdC1ZW2mjg+jaX139h67yAtzotm875w+dy4iFVqZ3ez2+eefM2vWLBo0aFCw7d//\n/jcvvPACWVlZ1K5dm2nTpmFvb1/kMXSzm1iDxWLhx1/O8vmGo2Rl59GykScP9g3G3dXR2qOJSAVk\n7Uvreta6SBEuXM7gkzUHiY1LppqTHSN7NaFDMx+tnYtIIdYOcj3ZTaQIXm7OPD38DiL6BJGbZ2H2\nNweYtXwfV9K0di4iFYeCXOQGDMOgeyt/XhnfnuC6buw5eoEX5kSzbX+C1s5FpEJQkIvcBG83Z54e\n0YpRdzUhJy+fj78+wHsrYrhyNdvao4lIFVfmnyMXqSxMhkHPNnVo3tCDuasPsutwEofjLzO6dxPa\nBdfS2rmIWIUauUgJ1XJ34dmRrRjRK5DsnDw+/Go/76+MIUXtXESsQI1c5BaYDIO72gbQopEnc6Ni\n2XkoiUOnLhPRJ4h2wXrQkYiUHzVykdvg4+7C30e1ZnjPa+38g5Ux19p5utq5iJQPNXKR22QyDHq3\n+/92vuPgeQ6dSiaidxBt1c5FpIypkYuUEl8PF54b1ZoHejQmMzuP91fG8OFXMaRl5Fh7NBGpxNTI\nRUqRyWTQp33dgna+PfY8B+OSiegTTJsgb2uPJyKVkBq5SBnw86zG86PbMKx7Y9Kz8nhvxT4+XrVf\n7VxESp0auUgZMZkM+oZda+eRUbFsO5BIbFwyD/YNolWg2rmIlA41cpEyVturGv+IaM3QOxtxNTOH\nWcv3MfvrA1zNVDsXkdunRi5SDswmE/071KNlYy/mRh1g6/4EDsRdYkzfYO5o7GXt8UTEhqmRi5Qj\nf69q/COiDUO6NSQtPYd3lu0l8psDpKudi8gtUiMXKWdmk4kB4fVp2diLyKhYNscksP/kJcb2C6ZF\nI7VzESkZNXIRK6njXZ3JEW0Y1KUBqek5/GfpXuZGxZKemWvt0UTEhhiWCvylyklJqaV+TG9v1zI5\nrsjtiD+fRmTUAU4lpuHu6shD/YIJbehp7bFE5CaUdq54e7uWaP8bBnlCQgJz585l06ZNnD17FgB/\nf3+6dOnC2LFj8fPzu71pi6Egl6okNy+f1Vvj+HrLSfLyLXRt6ccDPQJxdtQKmEhFVmGDfNmyZURG\nRjJixAg6duxI7dq1ATh79ixbtmxhyZIljB8/niFDhtz+1EVQkEtVdCoxlcioWOLPp+FRw5GH+oXQ\nrIGHtccSkSJYO8iL/FP/yJEjrFq1Cnt7+0LbGzduTOPGjRk+fDhvvvnmrU0pIkWq6+PKlDFt+WbL\nSaK2xvHm53vodkdthnVvrHYuIn9Q7Br5lStXOH/+PIGBgWzatIm9e/cybNgwvL3L/slUauRS1cUl\npBIZdYDTSVfxrOHEQ/2DaVpf7VykIrF2Iy/2rvVnnnmG8+fPc/LkSf7973/j5ubG5MmTb3lAEbl5\n9XxdeXFsO+7pWJ/k1CzeWLKH+WsPkZGlO9tF5JpigzwjI4NOnTrx7bffMnr0aEaNGkVOjh5eIVJe\n7MwmBnVtyAtj2uDvVY2Nu8/wz7nbiY1LtvZoIlIB3FSQX7p0ibVr13LnnXdisVi4cuVKecwmIr9T\n37cGL45tx4DwelxMyeT1xbtZuO4Qmdlq5yJVWbFBfs8999C7d286dOiAn58f7733HmFhYeUxm4j8\nD3s7E0O6NeKFB9tS26saG3ad4cXI7Rw6pXYuUlWV+IEwqampuLqWbCH+VulmN5Gi5eTmsfKnE3wb\nfQqLBXq2qcPQbo1wdDBbezSRKsXaN7sVG+Rbtmxh0aJFpKam8vtd58+ff2sTloCCXKR4x85eYW5U\nLOcuplPLzZlxA0JoEuBm7bFEqowKH+S9e/fmsccew9fXt9D28PDwkk9XQgpykZuTk5vHik0nWLv9\nFFigV9sABndriKO92rlIWbN2kBf7dIn69eszaNCgWx5IRMqevZ2ZYd0b07qJN5FRsXy3I569xy4w\nbkAIgXXUzkUqs2Ib+fr16/nhhx9o1aoVdnb/n/sDBw4s8+HUyEVKLjsnjxWbjrNuezwAd7ULYHDX\nhjionYuUiQrfyD/88EOcnZ3Jzs4u2GYYRrkEuYiUnIO9mQd6BNK6iTdzo2JZ93M8vxy7yPgBITT2\nr2nt8USklBXbyEeMGMHixYvLa55C1MhFbk9WTh4r/nuc736OBwP6tKvLwC4N1M5FSpG1G7n5pZde\neulGOyQnJ5OTk0OtWrUwDAOLxYLFYsEwjNuZ86akp2cXv1MJVavmWCbHFamI7MwmQht6ElLPncOn\nLvPLsYvsPJxEfT9XPFydrD2eSKVQ2rlSrZpjifYvtpG3atWKjIyMazv/GuSGYRAbG3vrU94kNXKR\n0pOVncfyH4+xfudpDAP6htVlYOcG2NupnYvcDms38iKDPCcn5w9fYXor+9wOBblI6Tt0Kpm5q2NJ\nupxJba9qjB8QQgO/GtYeS8RmWTvIi3xE64QJEzhx4kSRLzx27BgTJkwo0clExPqC6rrzyrgwerau\nw9kLV3l1/g6W/3iMnNx8a48mIregyEZ+5MgRnn32WXx9fenSpQt+fn4AnDt3jk2bNpGYmMj06dMJ\nDAwss+HUyEXKVmxcMp+sjuXClUz8vaoxTu1cpMSs3chvuEZusVj4/vvv+e9//0tCQgIAvr6+dO3a\nlZ49e5b5DW8KcpGyl5mdy9IfjvHD7jOYDIP+4fW4t1N97MzFfqeSiFDBg9zaFOQi5efAyUt8svog\nF1MyqeNdjfEDmlLPt3y+IEnElinIb0BBLlK+MrJyWfrDUTbuOYvJMLi7Yz3u7qh2LnIj1g5y/dcp\nIgWcHe14sG8wTz1wB26uDqyOuQtjAAAgAElEQVTafJKpn+7gVKL++BWpqG4qyPPz80lKSirrWUSk\ngmjWwIOp48Po2tKP+PNpTP10B1/9dILcPN3ZLlLRFBvkW7dupVevXkRERADwr3/9ix9++KHMBxMR\n63J2tGNsvxD+NqwlNao58NVPJ3h1/g7iz6dZezQR+Z1ig/ztt9/miy++wNvbG4BHH32UDz74oMwH\nE5GKIbShJ1PHh9G5hR+nEtN4Zd7PrNqsdi5SURQb5C4uLnh5eRX87OHhUaZPcxORisfFyY5x/UN4\n8v6WuLrYs3LTCV6bv5PTSWrnItZWbJA7OTmxfft2AK5cucKiRYtwdLy5B7ofPnyYXr16sXDhQgCe\ne+457rnnHiIiIoiIiGDjxo23PrmIlLsWjTx5dUIYnZr7EpeYysuf/Mw3W06Sl692LmItxX787Ny5\nc7z00ktER0fj4OBAmzZtmDx5MnXq1LnhgdPT03nkkUeoX78+QUFBjB49mueee44+ffrQvXv3mxpO\nHz8Tqbh+OXqBed8e5EpaNvV9XRl/d1P8vapZeyyRcmftj5/ZFbeDn58fH330UYkHcXBwYPbs2cye\nPbvErxWRiq9lYy9enRDGou+OsHV/Ai9/sp37Ojegb1hdzCZ9slWkvBTbyLds2cKiRYtITU3l97vO\nnz//pk4wa9Ys3N3dCxp5UlISOTk5eHp6MmXKFDw8PIp8bW5uHnb6ikWRCm/7/gTeXbqH5NQsmtR1\n48nhrQnw0VPhRMpDsUHeu3dvHnvsMXx9fQttDw8Pv6kT/D7It27dipubGyEhIXz88cckJCTw4osv\nFvlaXVoXsR1pGTksWn+YbfsTsTObGNS1AX3a1cVkKtvvZBCxtgp/ab1+/foMGjTolgf6vd+Hf48e\nPXjppZdK5bgiYn3Vne15+J5mtA2qxfxvD7L0h2PsOpTEuAEh+Hlq7VykrBQb5MOGDWPy5Mm0atUK\nO7v/333gwIElPtnEiRN59tlnCQgIIDo6uky/AlVErKN1E2+aBLjx2XeHiT6QyEuf/MygLg3p3S5A\n7VykDBR7aX3o0KE4OzsXurRuGAYzZsy44YFjYmKYPn06Z86cwc7ODh8fH0aPHs3HH3+Ms7MzLi4u\nTJs2DU9PzyKPoUvrIrZt56HzzF97iNT0HBr712TcgBB8PVysPZZIqbL2pfVig3zEiBEsXrz4toa6\nVQpyEduXkp7NZ+sO8/PB89jbmRjStSG92qqdS+Vh7SAv9jMiPXr0YNu2bWRnZ5Ofn1/wj4jIzajh\n4sBjA0N5bGAojvZmlmw4yvRFu0hMTrf2aCKVQrGNvFWrVmRkZBR+kWEQGxtbpoOBGrlIZZNyNZsF\n6w6x81ASDnYmhtzZiJ5t6mAy1M7Fdlm7kRcb5NakIBepfCwWCz8fPM/CdYdJy8ihSYAb4/oHU8td\na+dimypskC9fvpwhQ4Ywc+bM675w0qRJJZ+uhBTkIpXXlavZLFh7iF2Hk3CwN3H/nY3p3tpf7Vxs\njrWDvMg1ctOvj1g0m83X/UdE5HbUrObAXwaF8vC9TbE3m/jsu8O8sXg3SZczin+xiBQo8nPkv4X1\n448/Xm7DiEjVYhgGHZr6ElLXnflrD7H7yAVejNzOsO6N6NZK7VzkZhTZyJctW1aec4hIFVazuiOP\nD27On+5uip3ZYMG6w7y5ZA8X1M5FiqWvKBKRCsEwDMJDfZk6IYw7GnsRG5fMlLnb2bj7DBX4nlwR\nqyvyZrfmzZtf96lrFosFwzDYuHFjWc+mm91EqiiLxcKWmAQWrz9CelYuTeu7M7ZfMF41na09msgf\nWPtmtyLXyJs2bcpbb7112wOJiJSUYRh0au5H0/oefPrtQfYeu8iLkdt5oEdjurasjaG1c5ECRQa5\ng4MD/v7+5TmLiEgh7q6OTBrags37Elj8/WE+/fbaw2TG9gvGo4aTtccTqRCKXCNv0aJFec4hInJd\nhmHQuYUfU8eHEdrQg5gTl5gSGc1/fzmrtXMR9GQ3EbEhFouFn/aeY8mGI2Rk5RHa0IOxfdXOxbqs\nvUauu9ZFxGYYhkGXlrWZOj6MZg08iDl+iSmR2/lp7zm1c6myFOQiYnM8ajjxt2EtGdM3CIvFwtzV\nscxctpfk1CxrjyZS7nRpXURs2oUrGcxbc5ADJ5NxcbRjRK9AOob66s52KTe6tC4ichu8ajrz1AN3\n8GCfIPIsFiKjYpm1fB+X09TOpWpQkIuIzTMMgztb+TN1XHtC6rmz5+gFpsyJZuv+BK2dS6WnIBeR\nSsPLzZmnht/B6N5NyM2zMPvrA7z75T6uqJ1LJVbkA2FERGyRyTDo0boOoQ09+SQqlt1HLnA4/jKj\nejchLMRHa+dS6aiRi0ilVMvNmWdGtmLUXU3Iycvn41UHeG9FDFeuZlt7NJFSpUYuIpWWyTDo2aYO\nzRt6MDcqll2Hkzgcf5nRvZvQPsTH2uOJlAo1chGp9Gq5u/DsqNaM6BVIdk4eH361n/dX7CMlXe1c\nbJ8auYhUCSbD4K62AbRo6Enk6lh2HEriUPxlInoH0Ta4lrXHE7llauQiUqX4eLjw3MjWDO/RmMzs\nPN5fGcMHK2NIVTsXG6VGLiJVjslk0Lt9XVo09mJuVCw/HzzPoVPJRPQJok2Q2rnYFjVyEamyfD1c\neG5Ua4Z1b0x6Vh7vrYjho1X7ScvIsfZoIjdNjVxEqjSTyaBvWF1aNvYkMiqW6AOJxMYlM6ZPEK2a\neFt7PJFiqZGLiAB+ntX4x+g23H9nI9Izc5n15T4+/lrtXCo+NXIRkV+ZTAb9OtQrWDvftj+R2JPJ\nPNg3iFaBaudSMamRi4j8D3+vavwjojVDujXkamYOs5bvY/bXB7iaqXYuFY8auYjIdZhNJgaE1+eO\nxl7MiYpl6/4EYuMuMaZvMC0be1l7PJECauQiIjfg712dFx5sw+CuDUlNz2Hmsr1ERh0gXe1cKgg1\nchGRYphNJu7ueK2dR0bFsnlfAgdOJjOmbzAtGnlaezyp4tTIRURuUp1a1Zn8YBsGdWlAytVs/rP0\nF+aujiU9M9fao0kVZlgsFou1hyhKUlJqqR/T29u1TI4rIlVL/Pk0Ir85wKnzabi7OvJQ/2BCG6id\nV0WlnSve3q4l2l+NXETkFgTUqs4LY9pyX+dr7fytz39h3pqDZGSpnUv5UiMXEblNpxJTmfNNLKeT\n0vCs4cjYfiE0a+Bh7bGknKiRi4jYuLo+rrw4ti33dqpPcmo2b36+h/nfqp1L+VAjFxEpRXEJqURG\nHeB00lU8azjxUP9gmtZXO6/M1MhFRCqRer6uTBnTjrs71iM5NYs3luxhwbpDZGarnUvZUCMXESkj\nJ86lEBkVy9kLV/Gq6cS4/iEE13O39lhSyqzdyBXkIiJlKCc3n1WbT7B6WxwWC/RsXYehdzbC0cFs\n7dGklCjIb0BBLiKVxfGzKURGHeDcxXS83a6186C6aueVgYL8BhTkIlKZ5OTmsfKnE3wbfQqLBXq1\nqcOQbmrntk5BfgMKchGpjI6duUJkVCwJl9Kp5e7MuP4hNAlws/ZYcosU5DegIBeRyio7J4+Vm06w\ndvspAHq1DWBwt4Y42qud2xprB3mZfvzs8OHD9OrVi4ULFwJw7tw5IiIiGDlyJJMmTSI7O7ssTy8i\nUmE52JsZ1qMxz49uQy0PF77bEc9Lc7dz9PQVa48mNqbMgjw9PZ2pU6cSHh5esO2dd95h5MiRLFq0\niHr16rFs2bKyOr2IiE1oXKcmLz/Ujt7tAjifnMG0hTv5fMMRsnPyrD2a2IgyC3IHBwdmz55NrVq1\nCrZFR0fTs2dPALp3787WrVvL6vQiIjbDwd7M8J6B/H1Ua7zdnVm7PZ6XPvmZY2fUzqV4dmV2YDs7\n7OwKHz4jIwMHBwcAPD09SUpKuuEx3N1dsLMr/fWikq4/iIiUB29vV9qE+rFgdSxf/3ScaQt3MrBb\nY0b1DcZBa+cVmjVzpcyCvDg3c49dcnJ6qZ9XN7uJSEU3sFN9QgJqMnd1LF9uPMrWfWcZNyCERrVr\nWns0uY5KfbPb/3JxcSEzMxOAxMTEQpfdRUTk/wXVdeeVcWH0bFOHcxfT+deCnSzbeIyc3HxrjyYV\nTLkGeceOHVm7di0A69ato0uXLuV5ehERm+LoYGbUXU14dkQrPGs4sXpbHC/P+5kT51KsPZpUIGX2\nOfKYmBimT5/OmTNnsLOzw8fHhzfeeIPnnnuOrKwsateuzbRp07C3ty/yGPocuYjINZnZuSzbeIwN\nu85gMgz6dajLvZ0aYG+nL7G0NmtfWtcDYUREbEjsyUt8suYgF65k4u9djfEDQqjvW8PaY1VpCvIb\nUJCLiPxRRlYuSzceY+Pua+28f3g97u1UHzuz2rk1WDvI9b+6iIiNcXa048E+QTw1/A7cXR34ZstJ\nXpm3g7gElZSqSI1cRMSGZWTl8sUPR/lxz1nMJoMB4fW4u6PaeXlSIxcRkVvm7GjHmL7B/O2BltSs\n7sCqzSeZ+ukOTiWqsFQVauQiIpVEemYun284wqa95zCbDO7pWJ/+4fXUzsuYGrmIiJQKFyc7Huof\nwl+HtaRGNQdW/nSCV+fv4PT5NGuPJmVIQS4iUsk0b+jJ1PHt6dzcj1OJabw872e+3nKSvHw9Fa4y\nUpCLiFRCLk72jBsQwpP3t8DVxZ4V/z3Oq/N3cjpJ7byy0Rq5iEgldzUzhyXrj7A5JgE7s8F9nRvQ\nN6wuZpO6XGnQGrmIiJSpak72jL+7KU8MbUE1Z3uW/3icfy3YyZkLV609mpQCNXIRkSokLSOHxesP\ns3V/InZmg4FdGtKnfYDa+W1QIxcRkXJT3dmeP93TjImDm+PiZM+yjceYtnAX5y6qndsqNXIRkSoq\nLSOHRd8dZtuBROzMJgZ1bUCfdnUxmQxrj2ZT1MhFRMQqqjvb8/C9zfjLoOa4OJpZ+sMxpn22U+3c\nxqiRi4gIqenZfPbdYbbHnsfezsTgrg25q22A2vlNUCMXERGrc3Vx4NH7QvnzwFCcHMx8vuEo/160\ni8RL6dYeTYqhRi4iIoWkpGezcN1hdhw8j4OdicHdGtGrbR1Mhtr59aiRi4hIhVLDxYE/Dwzl0fua\n4WBvZsn3R5jx2S4Sk9XOKyI1chERKVLK1WwWrDvEzkNJONiZGHpnI3q0UTv/PTVyERGpsGpUu9bO\nH7m3GfZ2JhatP8Lri3Zz/nKGtUeTX6mRi4jITbmSlsX8tYfYfeQCjvZmht7ZiO6t/at8O1cjFxER\nm1CzuiOPD27Ow/c0xc5s8Nl3h3lj8W6S1M6tSo1cRERK7HJaFvO/PcSeo9fa+bDujejWqmq2czVy\nERGxOW7VHZk4pDl/urspZpPBgnWHeXPJHi5cUTsvbwpyERG5JYZhEB7qy9QJYbRs5ElsXDJTIrez\ncc8ZKvDF3kpHQS4iIrfF3dWRJ4a2YPyAEEyGwfxvD/HW53u4eCXT2qNVCQpyERG5bYZh0Km5H69O\nCKN5Q0/2n0xmSmQ0//3lrNp5GVOQi4hIqXF3deTJ+1vwUP9gDAPmrTnI21/8wqUUtfOyoiAXEZFS\nZRgGXVrUZur4MEIbeBBz4hJTIqPZtFftvCwoyEVEpEx41HDir8NaMrZfMBYLfLL6IDOX7SU5Ncva\no1UqCnIRESkzhmHQteW1dt6svjt7j13khTnRbN53Tu28lCjIRUSkzHnWdOJvD9zBg32DyLdYiIyK\nVTsvJQpyEREpF4ZhcOcd/kwd356Qetfa+ZQ50WyJUTu/HQpyEREpV141nXl6+B1E9AkiL9/CnG9i\nmbV8H5fT1M5vhYJcRETKnWEYdG/lzyvj2xNc1409Ry8wZU40W/cnqJ2XkIJcRESsxtvNmadHtGLU\nXU3Iyctn9tcHePfLfVy5mm3t0WyGnbUHEBGRqs1kGPRsU4fmjTz5JCqW3UcucOR0NKPuakL7kFoY\nVfAb1UpCjVxERCqEWm7OPDOyFSN7BZKdk8dHq/bz/soYUtTOb0iNXEREKgyTYdCrbQDNG3kyNyqW\nnYeSOHTqMqN7N6F9iI+1x6uQ1MhFRKTC8XF34e+jWjO857V2/uFXv7bzdLXz/6VGLiIiFZLJMOjd\nLoCWjTyJXB3LjoPnOXQqmYjeQbQNrmXt8SoMNXIREanQfDxceG5kax7o0ZjM7DzeXxnDh1/FkKp2\nDqiRi4iIDTCZDPq0r0uLX9fOt8ee52BcMhF9gmkT5G3t8axKjVxERGyGn2c1nh/dhmHdG5Oelcd7\nK/bx0ar9pGXkWHs0q1EjFxERm2IyGfQN+7Wdr44l+kAisXHJjOkTRKsmVa+dq5GLiIhNqu1VjedH\nt+b+OxuRnpnDrC/3MfvrqtfO1chFRMRmmU0m+nWoR4vGXsyNOsDW/YkciEtmTN9g7mjsZe3xyoVh\nKcen00dHRzNp0iQCAwMBaNKkCVOmTCly/6Sk1FKfwdvbtUyOKyIi1pWXn8+30adYuekEefkWOob6\nMqJXINWc7Mv0vKWdK97eriXav9wbefv27XnnnXfK+7QiIlLJmU0mBoTXp2VjLyKjYtkSk8CBk5cY\n2y+YFo0qbzvXGrmIiFQqdbyrMzmiDYO6NiQ1PYf/LN3L3KhY0jMr59p5uV9af/nll6lbty5Xrlzh\n8ccfp1OnTkXun5ubh52dubzGExGRSubE2Sv8Z8lujp+5gmdNJyYOu4M2wZXrme3lGuSJiYns3LmT\nfv36ER8fz4MPPsi6detwcHC47v5aIxcRkduVm5fP6q1xfL3lJHn5Frq08OOBHoG4OJXO6rK118jL\n9dK6j48P/fv3xzAM6tati5eXF4mJieU5goiIVDF2ZhP3dm7AlDFtCahVnU17zzElMpqYExetPVqp\nKNcgX7VqFZGRkQAkJSVx8eJFfHwq1yUOERGpmOr6uDJlTFvu7VSflKvZvPX5L8xbc5CMrFxrj3Zb\nyvXSelpaGk8//TQpKSnk5OTw+OOP061btyL316V1EREpC3EJqURGHeB00lU8azgytn8Izep73NKx\nrH1pvVyDvKQU5CIiUlZy8/L5evNJorbGkW+xcGcrf+6/sxHOjiVbO1eQ34CCXEREytrJhBQiv4nl\nzIWreNZwYlz/YEJK0M6tHeT6HLmIiFRp9X1r8OLYdgwIr0dyahavL9nDgnWHyMy2jbVzNXIREZFf\nnTiXQmRULGcvXMWrphPjB4QQVNf9hq9RIxcREakgGvjV4J9j29K/Qz0upmQyfdFuPvvuMFnZedYe\nrUhq5CIiItdx7OwV5kbFcu5iOrXcnHmof/B127m1G7mCXEREpAg5uXms2HSCtdtPgQV6tq3DkG6N\ncLT//8eHK8hvQEEuIiIVwdEzV4iMiiXxUjq13J0Z1z+E5NQsorae5OzFdGp7ujAgvD5hTW//IWcK\n8mIoyEVE5FZk5+SxYtNx1m2Pp6jgfOTeZrcd5rrZTUREpAw42Jt5oEcgz41ujZ3ZuO4+UVvjynkq\nBbmIiEiJBNZxIz//+p383MWr5TyNglxERKTEantVu+52P8/rby9LCnIREZESGhBev4jt9cp3EKB0\nvlVdRESkCvnthraorXGcu3gVP89qDAivVyp3rZeU7loXERG5Ddb+HLkurYuIiNgwBbmIiIgNU5CL\niIjYMAW5iIiIDVOQi4iI2DAFuYiIiA1TkIuIiNgwBbmIiIgNU5CLiIjYsAr9ZDcRERG5MTVyERER\nG6YgFxERsWEKchERERumIBcREbFhCnIREREbpiAXERGxYQpyERERG1ahgvzkyZM8/PDDDB06lMGD\nBzN16lSys7MB6NGjByNHjiQiIoJhw4Yxc+ZMAKKjo3niiScKHWfWrFksXLiw0LbTp08TFBTEnj17\nCm0fMmQIzz333E3Nd/XqVXr06FHk7683i4iIWE9cXByPPvoo999/P/fffz+TJk3i0qVLAHz55Zd0\n69aNiIgIRo8eTUREBEePHgUgIiKCw4cPFzpWWFjYH47/3HPPMX78+ELbfvjhB4KCgjh9+vRNzTh9\n+nS+/PLLIn9/vVl+r8IEeV5eHhMnTmTChAksW7aM5cuXA/Dee+8V7DN79mwWLFjAkiVL2LVrFzt2\n7CjROQICAvjmm28Kfo6LiyMlJaV03oCIiFQov8+VpUuXsnTpUpo1a8Zrr71WsE///v1ZsGABCxcu\nZOLEibz66qslPs/p06cL/jgAWL16NQEBAaXyHm6GXbmdqRibN2+mYcOGtG/fHgDDMHjmmWcwmf74\nt4bJZKJ58+bExcVRp06dmz5Hy5Yt2bJlC3l5eZjNZqKioujUqROZmZnAtUb99ttvY2dnh4+PD9Om\nTSM7O5uJEyeSlZVFmzZtCo61Y8cO3nrrLezs7PDz82Pq1Km3+W9ARERK0+bNmwkMDKRt27YF2yZM\nmEBRDzRt2bIlcXFxJT5P586dWbNmDaNGjSIzM5OTJ0/i5+cHQE5ODi+++CLx8fFkZ2fzxBNP0Llz\nZ7766ivmzJmDj48PTk5OBAYGkpeXx5QpU4iPjyc3N5cnnniC8PDwYs9fYRr58ePHCQkJKbTNyckJ\nBweHP+ybmZlJdHQ0zZs3L9E57O3tadmyJdHR0QB8//33dOvWreD3//znP3n77bdZuHAhNWvW5Ouv\nv+arr74iMDCQRYsWFZrv1Vdf5f3332f+/Pl4enry7bfflmgWEREpW8ePHycoKKjQNpPJhNlsvu7+\nP/zwQ4lzBaB3795ERUUBsHHjRjp27Fjwu6ioKBwcHFi4cCGzZs1i6tSpWCwW3n77bebNm8cHH3xQ\n8MfD119/jbe3NwsWLOC9997jX//6102dv8I0csMwyMvLu+E+f/rTnwr+Bxg2bBhNmjQpCOXrHe96\n+vbtyzfffIOXlxc+Pj64uLgAcPnyZQzDKPgrKiwsjJ9//pn8/HzatWsHUHC14MKFC8TFxTFx4kQA\n0tPTcXd3x8fHp4TvWkREyorJZCI3N7fg58cee4y0tDQSEhJYtWoVcO0yeExMDBaLBW9vbyZPnlzk\n8YrKFX9/f3Jycjh79iyrV6/mscceY9euXQDExMQUrK37+Pjg4ODApUuXqFatGp6engC0bt0agN27\nd7Nz586C12ZlZRXcJ3YjFSbIGzZsyGeffVZoW3Z2NidPnqRJkybAtTXyatWqFdrHw8PjD+vcly5d\n+sNfYb8JDw/nlVdewdvbmz59+hRsNwyj0OWWnJycgm2/Xd7Pz88HrjX7WrVqsWDBgkLHLuqPChER\nKX+BgYHMnz+/4OcPPvgAuHbz9G//f96/f3/+/ve//+G17u7uhbLl0qVLeHt7F3muPn36sGLFCk6c\nOPGHq8u/z5bs7GwMwyi0bPzb7+3t7Xn00Ue5++67S/I2K86l9U6dOnHmzBk2bNgAXAvN119/ndWr\nV9/wdfXr1ychIaHg0sSlS5eIjo4u+Avnfzk4ONCuXTuWL19e6A70mjVrYhgGZ8+eBWD79u2EhobS\noEEDYmJigP8P6po1awIU3N24YMECDh48eKtvXUREykCHDh1ISEgoyBWA/fv3c/Xq1SIvr/8mPDy8\noLUDLF26lK5duxa5f58+fZg/f/4f9mnevHlBdpw7dw6TyYSbmxupqamkpKSQk5NT0MBbtmzJ999/\nD8DFixd56623bup9VphGbjKZiIyM5MUXX+Tdd9/FwcGBjh078vjjj9/wdfb29rzxxhtMmTIFi8WC\nxWLhhRdewMvLq8jX9O3bl0uXLuHq6lpo+9SpU3nqqaews7MjICCAAQMGkJ6ezl/+8hfGjBlT6Ga3\n1157jeeff76gnT/wwAPs3r379v4liIhIqTEMgzlz5vDKK6/w3nvvYW9vj4uLCx988AFOTk43fO0D\nDzzAm2++yfDhwzGbzTRq1Ijnn3++yP0DAgKoU6dOoSu9AAMGDGD79u1ERESQk5PDK6+8gslk4vHH\nH2f06NH4+/sTGBgIQL9+/di2bRvDhw8nLy+v2PwreJ/6PnIRERHbVWEurYuIiEjJKchFRERsmIJc\nRETEhinIRUREbJiCXERExIYpyEVswG/f3rd48eJC23fs2EFQUFCJHka0dOnSYr/xLyIigi1bthTa\nNmPGjIJvHwwNDSUiIoKIiAhWrlx5829EREpdhfkcuYjcWP369fnyyy8ZMWJEwbYvv/ySBg0alMv5\nn332WeDaHxUjR478w5MNRcQ61MhFbEStWrUwDIMjR44AkJGRwc6dO2nRokXBPsuWLWPo0KFERETw\n5JNPkpaWBsBnn33G3XffzYQJEwo9uOjgwYM89NBDREREMHz4cA4cOFDiuVJTU+nYsSNXr14Frj2C\nslOnTly+fJmmTZvy7rvvEhERweDBgwu+U7k0zisi1yjIRWzIfffdx/LlywFYu3YtXbt2LXhm89mz\nZ5k1axbz5s1jwYIF+Pn5MW/ePFJTU3nnnXdYsGABc+bMITk5ueB4zzzzDC+//DILFizgpZde4oUX\nXijxTK6urnTr1o21a9cC8NNPP9GhQwfc3NzIy8sjMDCQBQsWMGLECN55551SO6+IXKMgF7Eh/fr1\nY82aNeTm5rJixQruvffegt8dOHCAZs2aUb16deDat/Xt27ePuLg4/P39cXd3Byj4JqaLFy9y4sQJ\nJk+eTEREBK+99hppaWkFXyZREsOHD2fFihUArFmzhqFDhxb8rnPnzsC1b3g6evRoqZ5XRLRGLmJT\nPDw8aNq0KcuWLSMpKemG351ssVgKvsHv91+/+FtgOjg4YG9vXypr3S1btiQ1NZXjx49z5MgROnTo\nUGiO3xiGUarnFRE1chGbc9999/H2228zYMCAQttDQ0PZv39/wbr4li1baNmyJXXr1uX06dOkpKRg\nsVjYunUrcO2SeJ06dfjxxx8BOHHiBO++++4tzzVs2DAmT55M7969C/3hsG3bNgB27txJUFBQqZ9X\npKpTIxexMT169ODFF18sdFkdwNfXl0mTJvHQQw/h4OCAr68vf/vb33BxceHRRx9l1KhR+Pv74+/v\nT2ZmJgDTp0/n1Vdf5Rc0dMoAAACRSURBVOOPPyY3N7fYj6XdyL333su0adP4z3/+U2j7gQMHWLx4\nMVeuXGH69Omlfl6Rqk7ffiYipWLNmjWsX7+eN998s2BbUFAQ+/fvx85OnUGkrOi/LhG5bRMnTuTi\nxYsFd6X/X7t2QAIAAIAwrH9rWwiHrcQRBH4scgAIc3YDgDAhB4AwIQeAMCEHgDAhB4CwAV0oBg/9\n7QrmAAAAAElFTkSuQmCC\n", 892 | "text/plain": [ 893 | "" 894 | ] 895 | }, 896 | "metadata": { 897 | "tags": [] 898 | } 899 | } 900 | ] 901 | }, 902 | { 903 | "metadata": { 904 | "id": "ZrV3b8zWGgoX", 905 | "colab_type": "text" 906 | }, 907 | "cell_type": "markdown", 908 | "source": [ 909 | "# 5. How to select and work on GPU(s) if you have multiple of them?" 910 | ] 911 | }, 912 | { 913 | "metadata": { 914 | "id": "g4B86rMUGbHI", 915 | "colab_type": "code", 916 | "outputId": "ccb3379d-0cd3-4753-e7b2-d1c225388fc9", 917 | "colab": { 918 | "base_uri": "https://localhost:8080/", 919 | "height": 35 920 | } 921 | }, 922 | "cell_type": "code", 923 | "source": [ 924 | "# Sadly we only have one GPU here, so we will look into the dynamics with that GPU only.\n", 925 | "cuda0 = torch.device('cuda:0')\n", 926 | "\n", 927 | "x = torch.Tensor([1., 2.]).to(cuda0)\n", 928 | "x" 929 | ], 930 | "execution_count": 0, 931 | "outputs": [ 932 | { 933 | "output_type": "execute_result", 934 | "data": { 935 | "text/plain": [ 936 | "tensor([1., 2.], device='cuda:0')" 937 | ] 938 | }, 939 | "metadata": { 940 | "tags": [] 941 | }, 942 | "execution_count": 56 943 | } 944 | ] 945 | }, 946 | { 947 | "metadata": { 948 | "id": "uvNDslc6IvUC", 949 | "colab_type": "code", 950 | "colab": {} 951 | }, 952 | "cell_type": "code", 953 | "source": [ 954 | "# To change the default Device:\n", 955 | "torch.cuda.set_device(0) # '0' is the Id of device" 956 | ], 957 | "execution_count": 0, 958 | "outputs": [] 959 | }, 960 | { 961 | "metadata": { 962 | "id": "bmsLfUk3JegG", 963 | "colab_type": "code", 964 | "outputId": "b5744a38-5744-4a8c-9401-7a0e8fd561a3", 965 | "colab": { 966 | "base_uri": "https://localhost:8080/", 967 | "height": 35 968 | } 969 | }, 970 | "cell_type": "code", 971 | "source": [ 972 | "y = torch.Tensor([3., 4.]).to(cuda0)\n", 973 | "z = x + y\n", 974 | "z # This tensor is stored on the same device as x and y" 975 | ], 976 | "execution_count": 0, 977 | "outputs": [ 978 | { 979 | "output_type": "execute_result", 980 | "data": { 981 | "text/plain": [ 982 | "tensor([4., 6.], device='cuda:0')" 983 | ] 984 | }, 985 | "metadata": { 986 | "tags": [] 987 | }, 988 | "execution_count": 58 989 | } 990 | ] 991 | }, 992 | { 993 | "metadata": { 994 | "id": "KGE6gB_CJ4Oi", 995 | "colab_type": "text" 996 | }, 997 | "cell_type": "markdown", 998 | "source": [ 999 | "# 6. Data Parallelism?" 1000 | ] 1001 | }, 1002 | { 1003 | "metadata": { 1004 | "id": "cyVerBqXJxwB", 1005 | "colab_type": "code", 1006 | "colab": {} 1007 | }, 1008 | "cell_type": "code", 1009 | "source": [ 1010 | "# Again we only have one GPU, so we will only use that to see into Dynamics\n", 1011 | "module = nn.Sequential(\n", 1012 | " nn.Linear(20, 20),\n", 1013 | " nn.ReLU(),\n", 1014 | " nn.Linear(20, 4),\n", 1015 | " nn.Softmax()\n", 1016 | ").cuda()\n", 1017 | "\n", 1018 | "inp = torch.ones((30, 20)).cuda()" 1019 | ], 1020 | "execution_count": 0, 1021 | "outputs": [] 1022 | }, 1023 | { 1024 | "metadata": { 1025 | "id": "Ik8h9tc3LTKQ", 1026 | "colab_type": "code", 1027 | "outputId": "575bf4f1-3933-4d89-8c1c-0837ccd9ba44", 1028 | "colab": { 1029 | "base_uri": "https://localhost:8080/", 1030 | "height": 52 1031 | } 1032 | }, 1033 | "cell_type": "code", 1034 | "source": [ 1035 | "# Replicate module to devices in device_ids\n", 1036 | "replicas = nn.parallel.replicate(module, [0])\n", 1037 | "\n", 1038 | "# Distribute input to devices in device_ids\n", 1039 | "inputs = nn.parallel.scatter(inp, [0])\n", 1040 | "\n", 1041 | "# Apply the models to corresponding inputs\n", 1042 | "outputs = nn.parallel.parallel_apply(replicas, inputs)\n", 1043 | "\n", 1044 | "# Gather result from all devices to output_device\n", 1045 | "result = nn.parallel.gather(outputs, 0)" 1046 | ], 1047 | "execution_count": 0, 1048 | "outputs": [ 1049 | { 1050 | "output_type": "stream", 1051 | "text": [ 1052 | "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py:91: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.\n", 1053 | " input = module(input)\n" 1054 | ], 1055 | "name": "stderr" 1056 | } 1057 | ] 1058 | }, 1059 | { 1060 | "metadata": { 1061 | "id": "9hosvC37Lp1i", 1062 | "colab_type": "code", 1063 | "outputId": "1cb95020-60a9-484f-da99-13bd912cd37f", 1064 | "colab": { 1065 | "base_uri": "https://localhost:8080/", 1066 | "height": 35 1067 | } 1068 | }, 1069 | "cell_type": "code", 1070 | "source": [ 1071 | "result.shape # Output from module" 1072 | ], 1073 | "execution_count": 0, 1074 | "outputs": [ 1075 | { 1076 | "output_type": "execute_result", 1077 | "data": { 1078 | "text/plain": [ 1079 | "torch.Size([30, 4])" 1080 | ] 1081 | }, 1082 | "metadata": { 1083 | "tags": [] 1084 | }, 1085 | "execution_count": 63 1086 | } 1087 | ] 1088 | }, 1089 | { 1090 | "metadata": { 1091 | "id": "vR4KkgxCLtWC", 1092 | "colab_type": "code", 1093 | "outputId": "50f9bfb4-1829-4634-fa67-d8626269ea27", 1094 | "colab": { 1095 | "base_uri": "https://localhost:8080/", 1096 | "height": 70 1097 | } 1098 | }, 1099 | "cell_type": "code", 1100 | "source": [ 1101 | "# Or we could have done:\n", 1102 | "model = nn.DataParallel(model, device_ids=[0])\n", 1103 | "result = model(inp)\n", 1104 | "result.shape" 1105 | ], 1106 | "execution_count": 0, 1107 | "outputs": [ 1108 | { 1109 | "output_type": "stream", 1110 | "text": [ 1111 | "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py:91: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.\n", 1112 | " input = module(input)\n" 1113 | ], 1114 | "name": "stderr" 1115 | }, 1116 | { 1117 | "output_type": "execute_result", 1118 | "data": { 1119 | "text/plain": [ 1120 | "torch.Size([30, 4])" 1121 | ] 1122 | }, 1123 | "metadata": { 1124 | "tags": [] 1125 | }, 1126 | "execution_count": 64 1127 | } 1128 | ] 1129 | } 1130 | ] 1131 | } -------------------------------------------------------------------------------- /SpeedUpYourAlgorithms/2) Numba.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "SpeedUpYourAlgorithms_Numba.ipynb", 7 | "version": "0.3.2", 8 | "provenance": [] 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "accelerator": "GPU" 15 | }, 16 | "cells": [ 17 | { 18 | "metadata": { 19 | "id": "QPIVdxjZ_6az", 20 | "colab_type": "text" 21 | }, 22 | "cell_type": "markdown", 23 | "source": [ 24 | "\n", 25 | "This is a part of series I am writing, named \"**Speed Up Your Algorithms**\".\n", 26 | "\n", 27 | "Here is the list of posts with files I have written: [Github-SpeedUpYourAlgorithms](https://github.com/PuneetGrov3r/MediumPosts/tree/master/SpeedUpYourAlgorithms)" 28 | ] 29 | }, 30 | { 31 | "metadata": { 32 | "id": "tI78SzIeTZSH", 33 | "colab_type": "text" 34 | }, 35 | "cell_type": "markdown", 36 | "source": [ 37 | "# Initial" 38 | ] 39 | }, 40 | { 41 | "metadata": { 42 | "id": "vF74OXegST3A", 43 | "colab_type": "code", 44 | "colab": {} 45 | }, 46 | "cell_type": "code", 47 | "source": [ 48 | "!pip install numba" 49 | ], 50 | "execution_count": 0, 51 | "outputs": [] 52 | }, 53 | { 54 | "metadata": { 55 | "id": "YmsI2Ou7SyTX", 56 | "colab_type": "code", 57 | "colab": {} 58 | }, 59 | "cell_type": "code", 60 | "source": [ 61 | "# https://medium.com/@iphoenix179/running-cuda-c-c-in-jupyter-or-how-to-run-nvcc-in-google-colab-663d33f53772\n", 62 | "# Needed for using cuda with Numba (https://numba.pydata.org/numba-doc/dev/user/installing.html#installing-using-pip-on-x86-x86-64-platforms)\n", 63 | "# Numba only supports cuda till 9.1\n", 64 | "!apt update -qq;\n", 65 | "!wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64-deb\n", 66 | "!mv cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64-deb cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb\n", 67 | "!dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb\n", 68 | "!apt-get update -qq;\n", 69 | "!apt-get install cuda-8-0 gcc-5 g++-5 -y -qq;\n", 70 | "!ln -s /usr/bin/gcc-5 /usr/local/cuda/bin/gcc;\n", 71 | "!ln -s /usr/bin/g++-5 /usr/local/cuda/bin/g++;" 72 | ], 73 | "execution_count": 0, 74 | "outputs": [] 75 | }, 76 | { 77 | "metadata": { 78 | "id": "_26iiW8S4ECp", 79 | "colab_type": "code", 80 | "colab": {} 81 | }, 82 | "cell_type": "code", 83 | "source": [ 84 | "import os\n", 85 | "os.environ[\"NUMBAPRO_CUDA_DRIVER\"] = \"/usr/lib64-nvidia/libcuda.so\"\n", 86 | "os.environ[\"NUMBAPRO_NVVM\"] = \"/usr/local/cuda-8.0/nvvm/lib64/libnvvm.so\"\n", 87 | "os.environ[\"NUMBAPRO_LIBDEVICE\"] = \"/usr/local/cuda-8.0/nvvm/libdevice/\"\n", 88 | "#os.environ[\"LD_LIBRARY_PATH\"] = \"/usr/local/cuda/lib64\"" 89 | ], 90 | "execution_count": 0, 91 | "outputs": [] 92 | }, 93 | { 94 | "metadata": { 95 | "id": "qvkZoVOv9u_F", 96 | "colab_type": "code", 97 | "colab": {} 98 | }, 99 | "cell_type": "code", 100 | "source": [ 101 | "!numba -s" 102 | ], 103 | "execution_count": 0, 104 | "outputs": [] 105 | }, 106 | { 107 | "metadata": { 108 | "id": "9wh6C6ac7pZJ", 109 | "colab_type": "text" 110 | }, 111 | "cell_type": "markdown", 112 | "source": [ 113 | "# Import" 114 | ] 115 | }, 116 | { 117 | "metadata": { 118 | "id": "5vndypxM7rGN", 119 | "colab_type": "code", 120 | "colab": {} 121 | }, 122 | "cell_type": "code", 123 | "source": [ 124 | "import os\n", 125 | "import numpy as np # Numba supports many functions from numpy (https://numba.pydata.org/numba-doc/dev/reference/numpysupported.html)\n", 126 | "from numba import jit, njit, vectorize, cuda\n", 127 | "import math # Numba supports many functions from math (http://numba.pydata.org/numba-doc/0.17.0/reference/pysupported.html)\n", 128 | "\n", 129 | "import matplotlib.pyplot as plt" 130 | ], 131 | "execution_count": 0, 132 | "outputs": [] 133 | }, 134 | { 135 | "metadata": { 136 | "id": "3bnRKV978fFN", 137 | "colab_type": "text" 138 | }, 139 | "cell_type": "markdown", 140 | "source": [ 141 | "# 4. Using basic numba functionalities (Just @jit it!)" 142 | ] 143 | }, 144 | { 145 | "metadata": { 146 | "id": "_6DEmDigpMfv", 147 | "colab_type": "code", 148 | "colab": {} 149 | }, 150 | "cell_type": "code", 151 | "source": [ 152 | "a = np.ones((1, 100), dtype=np.float64)\n", 153 | "b = np.ones((100, 1), dtype=np.float64)" 154 | ], 155 | "execution_count": 0, 156 | "outputs": [] 157 | }, 158 | { 159 | "metadata": { 160 | "id": "V91pNrvmpG-3", 161 | "colab_type": "code", 162 | "colab": {} 163 | }, 164 | "cell_type": "code", 165 | "source": [ 166 | "#\n", 167 | "# Simple Python function\n", 168 | "#\n", 169 | "\n", 170 | "def func(a, b):\n", 171 | " for i in range(100000):\n", 172 | " constant = math.pow((a@b)[0][0], 1./2)/math.exp((a@b)[0][0]/1000)\n", 173 | " a = np.array([[constant]*100], dtype=np.float64)\n", 174 | " return a" 175 | ], 176 | "execution_count": 0, 177 | "outputs": [] 178 | }, 179 | { 180 | "metadata": { 181 | "id": "G9URbB5dpNhJ", 182 | "colab_type": "code", 183 | "colab": { 184 | "base_uri": "https://localhost:8080/", 185 | "height": 34 186 | }, 187 | "outputId": "425aadd9-4eff-4223-802a-887a7acc70a4" 188 | }, 189 | "cell_type": "code", 190 | "source": [ 191 | "%timeit res = func(a, b)" 192 | ], 193 | "execution_count": 8, 194 | "outputs": [ 195 | { 196 | "output_type": "stream", 197 | "text": [ 198 | "1 loop, best of 3: 775 ms per loop\n" 199 | ], 200 | "name": "stdout" 201 | } 202 | ] 203 | }, 204 | { 205 | "metadata": { 206 | "id": "baBmONpCioUe", 207 | "colab_type": "code", 208 | "colab": {} 209 | }, 210 | "cell_type": "code", 211 | "source": [ 212 | "#\n", 213 | "# Numba with nopython = True\n", 214 | "#\n", 215 | "\n", 216 | "@njit # or @jit(nopython=True)\n", 217 | "def njit_func(a, b):\n", 218 | " for i in range(100000):\n", 219 | " constant = math.pow((a@b)[0][0], 1./2)/math.exp((a@b)[0][0]/1000)\n", 220 | " a = np.array([[constant]*100], dtype=np.float64)\n", 221 | " return a" 222 | ], 223 | "execution_count": 0, 224 | "outputs": [] 225 | }, 226 | { 227 | "metadata": { 228 | "id": "-d90HRMllKwr", 229 | "colab_type": "code", 230 | "colab": { 231 | "base_uri": "https://localhost:8080/", 232 | "height": 34 233 | }, 234 | "outputId": "6849decb-b67f-48d0-a1d2-cb74dfa802e7" 235 | }, 236 | "cell_type": "code", 237 | "source": [ 238 | "%timeit res = njit_func(a, b)" 239 | ], 240 | "execution_count": 18, 241 | "outputs": [ 242 | { 243 | "output_type": "stream", 244 | "text": [ 245 | "10 loops, best of 3: 109 ms per loop\n" 246 | ], 247 | "name": "stdout" 248 | } 249 | ] 250 | }, 251 | { 252 | "metadata": { 253 | "id": "WUEcagVshnon", 254 | "colab_type": "code", 255 | "colab": {} 256 | }, 257 | "cell_type": "code", 258 | "source": [ 259 | "#\n", 260 | "# Basic Numba compiler with type information provided\n", 261 | "#\n", 262 | "\n", 263 | "@jit('float64(float64, float64)')\n", 264 | "def jit_func(a, b):\n", 265 | " for i in range(100000):\n", 266 | " constant = math.pow((a@b)[0][0], 1./2)/math.exp((a@b)[0][0]/1000)\n", 267 | " a = np.array([[constant]*100], dtype=np.float64)\n", 268 | " return a" 269 | ], 270 | "execution_count": 0, 271 | "outputs": [] 272 | }, 273 | { 274 | "metadata": { 275 | "id": "TbYs5rtup5KX", 276 | "colab_type": "code", 277 | "colab": { 278 | "base_uri": "https://localhost:8080/", 279 | "height": 34 280 | }, 281 | "outputId": "a317b078-295a-47e3-8c86-8f538b62e25c" 282 | }, 283 | "cell_type": "code", 284 | "source": [ 285 | "%timeit res = jit_func(a, b)" 286 | ], 287 | "execution_count": 17, 288 | "outputs": [ 289 | { 290 | "output_type": "stream", 291 | "text": [ 292 | "10 loops, best of 3: 109 ms per loop\n" 293 | ], 294 | "name": "stdout" 295 | } 296 | ] 297 | }, 298 | { 299 | "metadata": { 300 | "id": "RPMMOCq2uuPQ", 301 | "colab_type": "code", 302 | "colab": { 303 | "base_uri": "https://localhost:8080/", 304 | "height": 417 305 | }, 306 | "outputId": "648f342c-7328-41f5-c39d-f79e5bd40162" 307 | }, 308 | "cell_type": "code", 309 | "source": [ 310 | "fig, ax = plt.subplots(1, 1)\n", 311 | "ax.plot([\"a\", \"b\", \"c\"], [821, 447, 440], \"-o\") # Results without caching\n", 312 | "ax.set_xticklabels([\"pyFunc\", \"Jit\", \"Njit\"])\n", 313 | "fig.suptitle(\"Basic Numba Functionalities\", fontsize=17)\n", 314 | "ax.set_ylabel(\"Time (ms)\")\n", 315 | "ax.set_xlabel(\"Method\")" 316 | ], 317 | "execution_count": 27, 318 | "outputs": [ 319 | { 320 | "output_type": "execute_result", 321 | "data": { 322 | "text/plain": [ 323 | "Text(0.5,0,'Method')" 324 | ] 325 | }, 326 | "metadata": { 327 | "tags": [] 328 | }, 329 | "execution_count": 27 330 | }, 331 | { 332 | "output_type": "display_data", 333 | "data": { 334 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfIAAAF/CAYAAACyv0vWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XlYVPXiBvD3zMKOwOCwugcisihG\n5kYJamomYrmUouWSqWl11WvmFma3rrd7+3VdU/OaS2WKipi5lVnmghkmiyjigiAKA7LvMPP7w5xE\nQRBnODPM+3keHpkzZ868Ax7eM9/vmRlBo9FoQEREREZJInYAIiIiajwWORERkRFjkRMRERkxFjkR\nEZERY5ETEREZMRY5ERGREWORU5OYN28evLy8tF+dOnVCjx498Morr2DPnj16uc8VK1bAy8sLBQUF\njd5Geno6vLy88NRTTyEnJ6fWdby8vLBr165G38ej0MVjasj26/p68cUX9XK/j+ru7+XLL78ULcO4\nceMwbNgw7eWQkBBMnz69zvVjYmLg5eWFH374oSnikQmRiR2ATIeNjQ0OHDgAANBoNMjOzsa2bdsw\nd+5cVFRUYOTIkTq9v4kTJ+Lll1+Gra3tY2+ruLgYy5Ytw7/+9S8dJDN8Bw4cgI2NzQPLZTJx/mR8\n//33WLt2rfagz9XVFb/++mutGcUSGRlZ4+ezfv16nD17FqtXrwYABAQE4Ndff4WdnZ1YEamZ4jNy\najKCIECpVEKpVMLJyQmdO3fGBx98AKVSicOHD+v8/qytraFUKiEIwmNva+zYsdizZw9Onz6tg2SG\nz9HRUfu7uvfLwcFBlDyxsbE1LkulUiiVSlhaWoqSpzYKhQItWrTQXv79999rXG9mZgalUgkzM7Om\njkbNHIucDMK9fwABYMuWLRg6dCh8fX0RGBiIMWPG4NSpUzXWOXLkCEaPHo3AwEB07doVoaGh2L59\nu/b62oahT5w4gZdffhldunRBnz59MHfuXGRlZdWbr3///ujbty+WLFmCysrKOtera8j3/izz5s3D\ngAEDEBMTg2HDhsHf3x+DBg3C8ePHkZKSgrFjx6Jr164YMGAADh48+MD9XLt2DePHj0eXLl3w9NNP\n4/3330dFRYX2+pycHLz33nvo3bs3fHx88Mwzz2D+/Pm4fft2vY+1IXbt2gUvLy8kJSXVWF7bcPPi\nxYsRFRWFQYMGaR/nvn37atzu+vXrmDFjBgIDAxEYGIgJEyYgLi5Ou80tW7bgwoUL8PLywooVK2r9\nOWdlZWHu3Lno2bMnfH19ERwcjI8//hhlZWU18k2aNAk///wzwsLC4O/vj5CQEGzatKlGnvj4eLz+\n+usIDAyEr68vnnvuOaxevRpqtbrOn8m9Q+shISH46aef8OOPP2qnXmobWk9PT8ff/vY3BAcHw8/P\nD0OGDMG3335bY7spKSmYNm0aevXqBT8/P4SEhGDZsmU1ft9k2ljkJJrCwkJ89tlnKCoqwvjx47XL\no6Oj8eGHH2L48OE4ePAgvv32W7Rq1QpTp05FRkYGgDtFNnPmTPj6+mL79u3Yu3cvRo8ejffffx+H\nDh2q9f5+//13TJ48GX5+fti5cyeWL1+OxMRETJky5aF/oO9auHAh0tLSsHHjRp08/oKCAnz++edY\nunQptm/fDrlcjvnz5yMiIgJvvvkmdu3aBVdXVyxYsAClpaU1brtkyRKEh4cjOjoaM2fOxPbt27F8\n+XLt9XPmzMGJEyfw2Wef4YcffsC///1vnDp1Cu+9955Osj+K3377DYcOHcL//d//aR/TvHnzkJmZ\nCQDIy8tDeHg4CgsL8b///Q/ffvstbGxsMGHCBKSnp2PFihUICAiAp6cnfv31V0ycOPGB+6ioqMCr\nr76KP/74A8uWLcP+/fvxt7/9DZGRkZg7d26NdVNTU7Fhwwa8//77iI6ORrdu3fDRRx8hPj4ewJ1p\nlIkTJ6KyshJfffUVDh48iBkzZmDVqlUN/t1HRkbCxcUFQUFB+PXXX/H8888/sE5BQQHGjh2Lixcv\n4qOPPkJ0dDRCQ0Px/vvvY9u2bdr1pkyZgqKiIqxfvx6HDh3CwoULsXfvXpOZ5qH6cY6cmkxhYSEC\nAgIA3JkjLy0thZ2dHT766CP4+/tr1wsODsbevXvRsWNH7bIpU6Zoh7bDwsJw/vx5VFVVITQ0FB06\ndABwZ/i7S5cucHFxqfX+169fj9atW2PBggXaZREREdi+fTtUKhWcnZ0fmr9169Z44403sHr1arzw\nwgtwc3Nr9M8CuFNg7777Ljp16gQAGD58OJYtW4ZZs2ahV69e2sf01ltvITU1Vbve3eXPPfccAKBt\n27Y4fvw4oqKiMGfOHADAP/7xDwDQZnR1dcWgQYOwefNmVFVVNelc9+3bt7Fr1y7tMPjEiRNx4sQJ\nnD9/Hs7Ozti1axdUKhW2b9+u/d0tXboUH3zwAdLT09GjRw/I5XLtcDoA5Obm1riPw4cP48qVK9i6\ndSueeuopAHd+Xzdu3MB///tf3Lx5E66urgCAmzdv4quvvtL+vqdPn469e/fi3Llz8PPzg7m5OXbs\n2AE7OzvtVIK7uzs2b96MX375BZMmTar3MSsUCkilUu1wem0iIyNx69Yt7NmzR/u7feONN5CYmIjV\nq1dj9OjRuH37Nm7cuIGXX34ZPj4+AO78Ltu3bw+JhM/D6A4WOTUZa2tr7N69W3s5Pz8fMTExWLBg\nAZKSkjB79mwAgKWlJY4fP4758+fjxo0bKCsrw93P9snLywMAPPnkk3B0dMTMmTMxevRo9OzZE/7+\n/vD19a3z/uPi4hAUFFRj2VNPPaX9w98Qr7/+unbE4O5JTI0ll8vh5eWlvXz3JKjOnTs/sOz+s9QD\nAwNrXPbx8cGRI0eQn58POzs7VFdXY/369YiJiUFOTg6qq6tRWVmJyspKlJSUPDCVcb9nn3221uUj\nRoyocSDUEN7e3jXmsh0dHQH89buMi4uDi4tLjQMwe3t7fPrppw2+j/j4eEilUnTt2rXG8oCAAGg0\nGpw/f15b5G5ubjUO2u7PI5PJkJWVhX/9619ISkrSLi8rK6txcPm4zp49C6VSWeMADQB69+6NgwcP\nQqVSQalUIjAwECtXrkRWVhaCgoIQGBiI9u3b6ywHGT8WOTUZiUSCtm3b1ljm7+8PS0tLfPjhhxg2\nbBg8PDywbNkybN68GZMnT8bAgQNhZ2eHzMxMjBs3Tns7Z2dnREZG4n//+592mNzOzg4jR47EO++8\nA7lc/sD9FxQUwNra+rEeg5mZGRYvXoyJEyfip59+QnBwcKO3ZWFhUeNEvLvf31t6d5fd/yGF9xfx\n3cdVWloKmUyGsWPHQqPRYO7cufDy8oK5uTm2bNmCLVu2NCjbV199VevPqjGvALCysqpx+f7HVFhY\n+Ni/l8LCQlhYWDzwe797VntxcXGD88THx+O1115Dly5dsGTJEri7u0Mmk2HOnDk6nZcuLCxEdna2\ndpTqrurqagBAZmYmnJycsH79emzduhUHDhzA1q1bIZPJ0K9fPyxYsABOTk46y0PGi0VOovP19YVG\no0FycjI8PDwQHR2N/v374+9//7t2ndpO0nJzc8PChQuxcOFCpKamIjIyEuvXr4e5uTneeuutB9Z3\ndHREfn7+Y+ft3bs3Bg0ahKVLl6Jnz541rqvrDPl7i0QXiouLYW9vr71cVFQE4E5xxcTEIDMzE6tW\nrUL//v216zxKCbVq1eqhz9rrOsBozONUKBRITk5+5Nvdq0WLFigtLUVFRUWNs8ILCwu11zfUvn37\noNFosGbNmho/44KCAlhYWDxWzvszu7i4PHCi3V13Rw2srKwwZcoUTJkyBbdv38bhw4fx73//G7Nm\nzcLWrVt1loeMFydZSHSXLl0C8NcfroqKigde5rRz584al5OSknD06FHt5bZt22L27Nnw9PTUnu18\nPz8/P5w5c6bGiW1//PEHXnnlFW2Ghpo/fz7y8vKwZs2aGsvvFsb9Bx73v3zqcd1/Bn98fDzc3Nxg\nY2OjPav+3p9hYWGh9iTA+8u3Me4+M7/3cebl5eHKlSuPvC0/Pz9kZWXh6tWr2mWlpaUIDw+v8WZB\nD8vt7+8PtVr9wEu+fv/9d0ilUu38ckNUVlZCKpXWeL33mTNncO3atUf+2T1s/W7dukGlUsHCwgJt\n27bVfllaWsLW1hYWFha4desWoqKitP9nFQoFRo8ejdDQ0Dr/n5PpYZFTk9FoNFCpVNqv69evIyoq\nCp988gm6d++Obt26AbjzB+7HH3/EmTNncPXqVXz00UeQSCSQSqWIi4tDdnY2zp49ixkzZmDz5s1I\nTU1Feno6IiMjcfXq1QeeJd81ZcoUqFQqLFiwAKmpqfjjjz8QERGBsrKyR55zdHZ2xsyZM7Fhw4Ya\ny21tbdG+fXvs3bsXsbGxuHLlCpYtW6aTkYB7ffPNNzhy5AiuX7+OL7/8EseOHdO+65qfnx/kcjm+\n/PJLpKam4syZM3jttde0J8fFxMQ89ghB586dIZfLsXHjRly+fBmJiYmYM2dOo4Z6X3rpJSiVSsyZ\nMwdJSUm4fPkyFi5ciKSkJO3/CTs7O6SnpyM2NhbXr19/YBv9+vWDp6cnFi9ejBMnTiAtLQ07duzA\nhg0b8OKLL9Z5wlltunXrhsrKSqxduxbp6en4/vvvsXTpUgQHByMjIwOXL19+6EsQ77Kzs0NycjIS\nEhJw8+bNB66/m2vmzJmIiYnBjRs38PPPPyM8PFx7vkhBQYH2lQwXLlzAzZs3cfz4cRw5cqTO/+dk\neji0Tk2mqKgIffr00V62srKCu7s7Jk+ejPDwcO1wbUREBBYuXIjXX38d1tbWGDZsGObOnQtLS0t8\n8803qKiowMqVK1FZWYkdO3bg008/hSAIaNWqFebMmYPXXnut1vv39/fH559/jhUrVmDo0KGwtbVF\nr1698O677zbqLO5x48Zh165dDwwLf/LJJ4iIiMCECRNga2uLESNGYPz48fjggw8e+T7q8uGHH+If\n//gH4uLiYGFhgfDwcEybNg3AnSmHjz76CMuXL8fQoUPRvn17zJo1C35+foiNjcXs2bPxySefYNCg\nQY2+fzc3N3z44YdYuXIlwsLC4O7ujrfeekt79vijsLa2xpYtW7Bs2TKEh4cDuHPy3qZNm9C6dWsA\nwKuvvor4+HiMHz8eo0ePxoQJE2psw8zMDBs3bsQnn3yCWbNmobCwEC4uLpg4cSKmTp36SHkGDx6M\nhIQEbN68GWvXrkVAQAA+++wz5OTk4Ny5cxg9ejS+++67erczZcoULFmyBK+88grefvtt+Pn51bi+\nRYsW+Oqrr/Cf//wHb731FgoLC9GyZUsMGjRIOzXUsWNHrF27FmvXrkV4eDjKy8vh5OSEfv361Tp9\nRKZJ0OhinI2IiIhEwaF1IiIiI8YiJyIiMmIsciIiIiPGIiciIjJiLHIiIiIjxiInIiIyYixyIiIi\nI8YiJyIiMmIsciIiIiPGIiciIjJiLHIiIiIjxiInIiIyYixyIiIiI8YiJyIiMmIsciIiIiPGIici\nIjJiLHIiIiIjxiInIiIyYixyIiIiI8YiJyIiMmIsciIiIiPGIiciIjJiLHIiIiIjxiInIiIyYixy\nIiIiI8YiJyIiMmIysQM0hkpVqNPtOThYITe3RKfbJGqOuK8QNYyu9xWl0rbO6/iMHIBMJhU7ApFR\n4L5C1DBNua+wyImIiIwYi5yIiMiIsciJiIiMGIuciIjIiLHIiYiIjBiLnIiIyIixyImIiIwYi5yI\niMiIGeU7u+lKzPlM7Dt5DRk5JXBztMKQnu3wdGdnsWMRERE1mMkWecz5TKyNTtReTlcVay+zzImI\nyFiY7ND6vpPX6lie2qQ5iIiIHofJFnlGdu1vZn8zp7iJkxARETWeyRa5W0urWpe7Olo3cRIiIqLG\nM9kiH9KzXa3Ln+ni2rRBiIiIHoPJFvnTnZ3xRqgPWiltIJUIcLA1BwCcSLiFyiq1yOmIiIgaxmTP\nWgfulPnTnZ2hVNoiK6sAG/Yl4UTCLXx75BLCn/MSOx4REVG9TPYZ+f0EQcC457zg3tIaR2Jv4HRS\nptiRiIiI6sUiv4e5mRTTwnxhLpdi4/4LPIOdiIgMHov8Pm4trfHqYC+UV1RjdVQCyiurxY5ERERU\nJxZ5LXp0dkFwgDtuqIrx1aFkseMQERHViUVeh5f7eaCtsy1+jb+JY3EZYschIiKqFYu8DnKZFNOG\n+8LSXIath5KRllUkdiQiIqIHsMgfwsneEpOHeKOySo3Vu+NRWl4ldiQiIqIa9PY68uLiYrz77rvI\nz89HZWUl3nzzTSiVSkRERAAAvLy8sGTJEgDAF198gQMHDkAQBMyYMQPPPvusvmI9soCOSgzq3gYH\nTl/Hl/svYOowHwiCIHYsIiIiAHos8t27d6N9+/aYPXs2MjMz8eqrr0KpVGL+/Pnw9/fH7Nmz8fPP\nP6NDhw74/vvvsW3bNhQVFWHMmDHo06cPpFKpvqI9shef7YCUjHz8diELHVvbo9+TrcSOREREBECP\nQ+sODg7Iy8sDABQUFMDe3h43btyAv78/ACA4OBgnT55ETEwMgoKCYGZmBoVCAXd3d6SkpOgrVqPI\npBJMDfWBjaUc2368hKs3C8SOREREBECPz8iHDBmCXbt2YcCAASgoKMCaNWvwwQcfaK93dHSESqWC\nvb09FAqFdrlCoYBKpYKXV91vkergYAWZTLfP2JVK23qv//u4QESsP4m10Yn4bFZf2FqZ6TQDkTGo\nb18hojuaal/RW5Hv2bMHbm5u2LBhAy5cuIA333wTtrZ/PSiNRlPr7epafq/c3No/S7yxlEpbqFSF\n9a7XWmGJob3aIfr4NSz78jRmjvCHhPPlZEIauq8QmTpd7ysPOyjQ29B6bGws+vTpAwDo1KkTysvL\nkZubq70+MzMTTk5OcHJyQnZ29gPLDVVo7/bo3M4B5y7n4EDMdbHjEBGRidNbkbdt2xbnzp0DANy4\ncQPW1tZ44okncObMGQDAoUOHEBQUhB49euDo0aOoqKhAZmYmsrKy4OHhoa9Yj00iETBlqA/sbMyw\n6+cruHg9t/4bERER6YmgachYdiMUFxdj/vz5yMnJQVVVFd5++20olUosXrwYarUaXbp0wXvvvQcA\n2LJlC/bu3QtBEPDOO++gZ8+eD922rof2GjMEkpyWh399fRa21nJETOgOO2vOl1Pzx6F1ooZpyqF1\nvRW5PhlCkQPA/lOp2HH0MrzbOmD26K6QSDhfTs0bi5yoYZrFHLkpGPh0G3T1aImk1FxEH78qdhwi\nIjJBLPLHIBEETBziDccWFth7/BoSruaIHYmIiEwMi/wx2VjKMX24L6RSAeuiz+N2QZnYkYiIyISw\nyHWgvWsLjA7xRFFpJT7fk4iqarXYkYiIyESwyHUkpJs7uns7IeVGPnb+fFnsOEREZCJY5DoiCAJe\nHdQJzgorHDydhthkldiRiIjIBLDIdcjSXIY3w3xhJpNgw74kZOWVih2JiIiaORa5jrVyskH4c14o\nLa/Cmt0JqKyqFjsSERE1YyxyPejj74o+fq5IzSzENz8a1keyEhFR88Ii15Oxz3VEK6U1jp69gVOJ\nt8SOQ0REzRSLXE/M5VJMH+4HczMpNh24iIzsYrEjERFRM8Qi1yMXhRUmDO6E8spqrI5KQHkF58uJ\niEi3WOR61t3bGf26tUJGdjE2H7wII/yMGiIiMmAs8iYwKsQD7V1tcTLxFo7F3RQ7DhERNSMs8iYg\nl0kwbZgvrC1k2HooGdcz+TGQRESkGyzyJtLS3hKTXuiMqmo1Vu9OQElZldiRiIioGWCRN6GuHi0x\nuEcbZOWVYuP+JM6XExHRY2ORN7EXn+mAjq3s8PtFFX44ky52HCIiMnIs8iYmlUjwxjBftLCSY/tP\nKbh8I1/sSEREZMRY5CJwsDXHlFAfqNUarNmTgKLSSrEjERGRkWKRi6RzOwWGBbXH7YJyrN97HmrO\nlxMRUSOwyEX0Qq928GmvQPyVHHx/MlXsOEREZIRY5CKSCAJeH9oZDrbm2H3sCpJSc8WORERERoZF\nLrIWVmaYNswXEkHA2uhE5BeVix2JiIiMCIvcAHi0ssNLzz6BguIKrI1ORLVaLXYkIiIyEixyAzGw\ne2sEeLbEhet5iDp2Vew4RERkJFjkBkIQBEwa4o2WdhbYdzIVcZdzxI5ERERGQKavDe/YsQPR0dHa\nywkJCfD19UVJSQmsrKwAAO+++y58fX3xxRdf4MCBAxAEATNmzMCzzz6rr1gGzcpCjunDffHRlt+x\nfm8iIiZ0h6OdhdixiIjIgAmaJnjD79OnT2P//v1ISUnBokWL0LFjR+11aWlpePvtt7Ft2zYUFRVh\nzJgx2LdvH6RSaZ3bU6l0++lhSqWtzrf5OH46ewNbDl5EB7cWmDe2G2RSDpyQYTC0fYXIUOl6X1Eq\nbeu8rkkaYtWqVZg+fXqt18XExCAoKAhmZmZQKBRwd3dHSkpKU8QyWH27uqFHZ2dcySjAjp8uix2H\niIgMmN6G1u+Ki4uDq6srlEolAGD58uXIzc3FE088gfnz5yM7OxsKhUK7vkKhgEqlgpeXV53bdHCw\ngkxW9zP2xnjY0Y4YZoUHYtZnP+PwmTQ86eOC3v5uYkciAmB4+wqRoWqqfUXvRR4ZGYnhw4cDAMaP\nHw8vLy+0adMG77//Pr766qsH1m/ISH9ubolOMxrqcOEbQztj6eYz+O+2WNhZSOHsYCV2JDJxhrqv\nEBmaZjW0HhMTg4CAAADAgAED0KZNGwBASEgIkpOT4eTkhOzsbO36mZmZcHJy0ncso+CutMH4gV4o\nLa/Gmt0JqKisFjsSEREZGL0WeWZmJqytrWFmZgaNRoPXXnsNBQUFAO4UvKenJ3r06IGjR4+ioqIC\nmZmZyMrKgoeHhz5jGZVevq54posbrmcV4esfLokdh4iIDIxeh9ZVKpV2/lsQBIwaNQqvvfYaLC0t\n4ezsjJkzZ8LS0hKjRo1CeHg4BEFAREQEJBKepX2vMf09ce1mAX45l4GOre3Qy9dV7EhERGQgmuTl\nZ7rW3F9+VpvM3BJ88OVvqFZrsGh8INyVNmJHIhNkDPsKkSFoVnPkpBvODlaYMNgbFZVqrI5KQFlF\nldiRiIjIALDIjUhgJyf0D2yFmzkl2HzgYoPO8CciouaNRW5kRgV74Am3Fjh1PhNH/8gQOw4REYmM\nRW5kZFIJpg7zhbWFDN/8kIzUW5yvJCIyZSxyI+RoZ4HXh/qgqlqDVbvjUVJWKXYkIiISCYvcSPk/\n4YghPdsiO78MG/Ylcb6ciMhEsciNWFhQe3RqY4+zl7Jx6Lc0seMQEZEIWORGTCqR4I1QH7SwNkPk\n0ctISc8XOxIRETUxFrmRs7Mxx9RQH6g1GqzZk4CCkgqxIxERURNikTcDndo6YHhQB+QWlmP93vNQ\nc76ciMhksMibied7toVfB0ckXr2N705cEzsOERE1ERZ5MyERBLw+tDMULcyx59hVnL92W+xIRETU\nBFjkzYiNpRzThvlCIhGwLjoRuYXlYkciIiI9Y5E3M0+422FUsAcKSiqxNjoR1Wq12JGIiEiPWOTN\nUP/AVnjSS4nktDzs+uWK2HGIiEiPWOTNkCAImDDYG04Olth/6jr+SMkWOxIREekJi7yZsrKQYXqY\nL2RSCTZ8dx7ZeaViRyIiIj1gkTdjbZxtMXaAJ4rLqrBmTwIqqzhfTkTU3LDIm7lnurihp48Lrt4s\nxPYjKWLHISIiHWORN3OCIGD8QC+4t7TGj7HpOJ2UKXYkIiLSIRa5CTA3k2JamC/M5VJ8uf8Cbt0u\nETsSERHpCIvcRLi1tMarg7xQVlGN1bvjUV5ZLXYkIiLSARa5Cenh44K+Ae5IVxXjq8PJYschIiId\nYJGbmFf6eaCtsy1+jbuJX+Nuih2HiIgeE4vcxMhlUkwb7gtLcxm2HrqI9KwisSMREdFjYJGbICd7\nS0wa4o2KKjVWRSWgtLxK7EhERNRILHIT1a2jEgO7t0bm7RJsOnABGo1G7EhERNQIMn1teMeOHYiO\njtZeTkhIwDfffIOIiAgAgJeXF5YsWQIA+OKLL3DgwAEIgoAZM2bg2Wef1VcsusdLzz6ByzcKcDop\nC56t7NHvyVZiRyIiokckaJrgqdjp06exf/9+pKSk4O9//zv8/f0xe/ZshIaGokOHDnj77bexbds2\nFBUVYcyYMdi3bx+kUmmd21OpCnWaT6m01fk2jcXtgjJEbPwNpeVVmD/uSbR3bSF2JDJgpryvED0K\nXe8rSqVtndc1ydD6qlWr8Prrr+PGjRvw9/cHAAQHB+PkyZOIiYlBUFAQzMzMoFAo4O7ujpQUvpVo\nU1G0sMCU0M5QqzVYvTsBxWWVYkciIqJHoLeh9bvi4uLg6uoKqVSKFi3+erbn6OgIlUoFe3t7KBQK\n7XKFQgGVSgUvL686t+ngYAWZrO5n7I3xsKOd5i5YaYuM22XYdvgithy6hAUTukMiEcSORQbKlPcV\nokfRVPuK3os8MjISw4cPf2B5XSP6DRnpz83V7VuMcrgQ6B/ghnPJWTh9/ha27kvE4B5txY5EBoj7\nClHDNKuh9ZiYGAQEBEChUCAvL0+7PDMzE05OTnByckJ2dvYDy6lpSSQCpoT6wM7GDDt/voLktLz6\nb0RERKLTa5FnZmbC2toaZmZmkMvl6NChA86cOQMAOHToEIKCgtCjRw8cPXoUFRUVyMzMRFZWFjw8\nPPQZi+pgZ22GqaE+AIDP9ySgoLhC5ERERFQfvQ6tq1SqGvPf8+fPx+LFi6FWq9GlSxf06tULADBq\n1CiEh4dDEARERERAIuHL28Xi1cYBLz7bAZFHL2NtdCJmj+7K+XIiIgPWJC8/0zW+/Ey/1BoNVkTG\n4dzlHIT2boewoA5iRyIDwX2FqGGa1Rw5GR+JIGDSC53h2MICe49fQ8LVHLEjERFRHVjkVCsbSzmm\nhflCIhGwLvo8bheUiR2JiIhqwSKnOnVwa4GX+3miqLQSn0cnoqpaLXYkIiK6D4ucHiqkmzue6uSE\nlPR87Pr5ithxiIjoPixyeihBEPDa4E5wdrDEgdPXcTZZJXYkIiK6B4uc6mVpLsP04X6QyyT4Yl8S\nsvJKxY5ERER/YpFTg7R2skFrRmYNAAAgAElEQVT4cx1RWl6FNbsTUFlVLXYkIiICi5weQZC/G/r4\nuSI1sxDbfuQn1BERGQIWOT2Ssc91RCulNX46ewOnEm+JHYeIyOSxyOmRmMulmBbmC3MzKTYduIib\nOcViRyIiMmkscnpkro7WmDC4E8orq7F6dwLKKzhfTkQkFhY5NUp3b2eEdHPHjexibDl0sUGfI09E\nRLrHIqdGGx3iifautjiRcAvH4m6KHYeIyCSxyKnR5DIJpg3zhZW5DF8dTsb1TH4qFhFRU2OR02Np\naW+JyS90RmWVGqujElBSViV2JCIik8Iip8fW1bMlBj/dBlm5pfhyfxLny4mImhCLnHTixWc7oGMr\nO5y5qMIPv6eLHYeIyGSwyEknpBIJ3hjmC1srObYfScHljHyxIxERmQQWOemMg605poT6QK3WYE1U\nAopKK8WORETU7LHISad82ikwrE973C4oxxffnYea8+VERHrFIiede6F3O/i0VyDucg72n0oVOw4R\nUbMme9iVt27dwv/+9z8cO3YMGRkZAAB3d3cEBQXhtddeg6ura5OEJOMiEQS8PrQzlmz8Dbt+uYIn\n3OzQqa2D2LGIiJqlOp+RR0ZGYsKECWjVqhVWrFiBkydP4uTJk1i+fDnc3d0xadIk7Ny5symzkhFp\nYWWGqcN8IEDA2uhE5BeVix2JiKhZqrPIL126hOjoaIwfPx4eHh6wsrKClZUVPDw8MH78eERFRSE5\nObkps5KR8WxljxF9n0B+cQXWRidCreZ8ORGRrgmaet69Iz8/H1lZWfD09MSxY8cQFxeHUaNGQalU\nNlXGB6hUun0rUKXSVufbpDs0Gg1W7orH2UvZeKFXW7z4zBNiR6LHwH2FqGF0va8olbZ1XlfvyW5/\n//vfkZWVhWvXruGf//wn7O3tsWDBAp2Fo+ZNEARMGuKNlnYW+O5EKuIu54gdiYioWam3yEtLS9G7\nd28cOHAA4eHhGDt2LCorG/b64OjoaISGhuLFF1/E0aNHMW/ePAwdOhTjxo3DuHHjcPToUe16L730\nEkaOHIkdO3Y81gMiw2NlIcf04b6QSQV88d153C4oEzsSEVGz8dCz1oE7RX779m0cPHgQq1evhkaj\nQX5+/e/alZubi1WrVmHnzp0oKSnBihUrAACzZs1CcHCwdr2SkhKsWrUKkZGRkMvlGDFiBAYMGAB7\ne/vHeFhkaNq5tMAr/Tyx5VAy1kQl4N2x3SCT8tWPRESPq96/pEOHDsVzzz2HHj16wNXVFatWrcLT\nTz9d74ZPnjyJnj17wsbGBk5OTli6dGmt6507dw5+fn6wtbWFhYUFunXrhtjY2Ed/JGTw+ga44+nO\nzricUYDIo5fFjkNE1CzUe7Lb/QoLC2FrW/ek+13r1q3DlStXkJeXh4KCAsycORN79uyBSqVCZWUl\nHB0dsWjRIhw/fhzx8fGYP38+AOCzzz6Dq6srRo8eXee2q6qqIZNJHyU2GYjS8irM+uxnpGcV4b1X\nn0IvfzexIxERGbV6h9ZPnDiBr7/+GoWFhTU+nnLz5s31bjwvLw8rV65ERkYGxo8fj48//hj29vbw\n9vbGunXrsHLlSgQEBNS4TUOOK3JzS+pd51HwTNymNWVoZ3y46Qw+2xaLFhZSODtYiR2JGoj7ClHD\nNOVZ6/UWeUREBKZNmwYXF5dHulNHR0cEBARAJpOhTZs2sLa2RseOHeHo6AgACAkJQUREBAYOHIjs\n7Gzt7bKystC1a9dHui8yLq2UNhg30Asb9iVhze4EzB/3JMzkHGEhImqMeufI27Vrh+HDh6Nnz541\nvurTp08fnDp1Cmq1Grm5uSgpKcHixYuRlpYGAIiJiYGnpye6dOmC+Ph4FBQUoLi4GLGxsQgMDHz8\nR0YGrbefK57p4orrWUX45sdLYschIjJa9T4jHzVqFBYsWKB9dn1XWFjYQ2/n7OyMgQMHYtSoUQCA\nhQsXwtraGu+88w4sLS1hZWWFjz/+GBYWFpg9ezYmTZoEQRDw5ptvNmgOnozfmP4dcfVmIX7+IwMd\nW9mjp++jjfoQEVEDTnYbMWIELC0tawytC4KAf/3rX3oPVxe+s1vzkXm7BEu+/A1qjQaLxgfCXWkj\ndiR6CO4rRA1jUHPkcrkcW7Zs0VkYons5K6ww8XlvrI5KwOqoBCx6NRAWZvX+tyQioj/VO0ceEhKC\nU6dOoaKiAmq1WvtFpCuBnZzQP7AVbuaUYPPBiw165QIREd1R71Of1atXo7S0FMCdIXWNRgNBEJCU\nlKT3cGQ6RgV74EpGAU4lZqJjK3v0DXAXOxIRkVGos8grKyshl8tx9uzZOm98dx2ixyWTSjBtmC8i\nNp7G1z8ko71rC7R14UmPRET1qXNoffLkybh69WqdN7x8+TImT56sl1BkmhztLPD60M6oqtZgdVQ8\nSsoa9uE8RESmrM5n5AsXLsSsWbPg4uKCoKAguLq6AgBu3ryJY8eOITMzE8uWLWuyoGQa/J9oiSE9\n22LfyVRs2JeEGS/6QRAEsWMRERmsh778TKPR4Mcff8Qvv/yCW7duAQBcXFzwzDPPoF+/fqL9geXL\nz5q3arUa//7mD1xMy8PLIR54rnsbsSPRn7ivEDVMU7787JE/NMUQsMibv7yickRs/A3FpZV4d0w3\neLSyEzsSgfsKUUM1ZZHzA6HJINnbmOONUB+oNRqs2ZOAwpIKsSMRERkkFjkZLO+2DggL6oDcwnKs\n33seauMbPCIi0rsGFblarYZKpdJ3FqIHDOnZFn4dHJFw9Tb2nbgmdhwiIoNTb5GfPHkS/fv3x7hx\n4wAAH330EX766Se9ByMCAIkg4PWhnaFoYY6oX68i6dptsSMRERmUeov8//7v/7B9+3YolUoAwNSp\nU7FmzRq9ByO6y8ZSjqnDfCERBKyNTkRuYbnYkYiIDEa9RW5lZYWWLVtqLysUCr6bGzU5D3c7jAz2\nQEFJJdZGJ6Ka7/dPRASgAUVuYWGB06dPAwDy8/Px9ddfw9zcXO/BiO43ILAVnuyoRHJaHnb/Uve7\nDhIRmZJ6i/z999/Hhg0bEB8fjwEDBuDYsWP44IMPmiIbUQ2CIGDC895wsrfE96dS8UdKttiRiIhE\nxzeEAd/kwthczyzEh5t/h7lcgvcnPIWWdpZiRzIZ3FeIGqYp3xCm3o8xPXHiBL7++msUFhbW+Jzo\nzZs36yYd0SNq42yLsQM8senARayJSsR74d0gk/ItEYjINNVb5BEREZg2bRpcXFyaIg9RgzzTxQ3J\naXk4mZiJb4+kYOyAjmJHIiISRb1F3q5dOwwfPrwpshA1mCAIGD+wE1Izi/Dj7+no2NoeT3VyEjsW\nEVGTq7fIR40ahQULFiAgIAAy2V+rh4WF6TUYUX3MzaSYHuaLDzb9ho3fJ6G1kw1cFFZixyIialL1\nFvnnn38OS0tLVFT89aEVgiCwyMkguLW0xquDOmH93vNYvTsBC8c/CTO5VOxYRERNpt4il8vl2LJl\nS1NkIWqUnj4uuJSWh6N/ZGDr4WRMfN5b7EhERE2m3lN9Q0JCcOrUKVRUVECtVmu/iAzJK/090dbZ\nFr/G3cSvcTfFjkNE1GTqfUa+evVqlJaW1lgmCAKSkpL0ForoUcllUkwb7oslG3/D1kMX0c7FFq2c\nbMSORUSkd3xDGPBNLpqT3y+qsGp3PFwUVlj0aiAszes9VqVHwH2FqGEM4g1hdu7ciZdeegn//e9/\na73+7bfffvxkRDr2pJcSzz3VGod+S8OmAxfwRqgPBEEQOxYRkd7UWeQSyZ3pc6m08WcAR0dH44sv\nvoBMJsNbb70FLy8vzJ07F9XV1VAqlfjkk09gZmaG6OhobNq0CRKJBKNGjcLIkSMbfZ9EI/o+gcsZ\n+TidlIWOre0R0q2V2JGIiPSmzqH16OhohIaGNnrDubm5ePnll7Fz506UlJRgxYoVqKqqwjPPPIPB\ngwfj008/hYuLC8LCwjB8+HBERkZCLpdjxIgR2Lp1K+zt7evcNofWqT63C8oQsfE3lFVU4b3wJ9He\ntYXYkZoF7itEDdOUQ+t1nrUeGRn5WHd68uRJ9OzZEzY2NnBycsLSpUsRExODfv36AQCCg4Nx8uRJ\nnDt3Dn5+frC1tYWFhQW6deuG2NjYx7pvIkULC0wZ2hnV1RqsiUpAcVml2JGIiPRCb2cCpaeno6ys\nDFOnTkVBQQFmzpyJ0tJSmJmZAQAcHR2hUqmQnZ0NhUKhvZ1CoYBKpXroth0crCCT6fZNPx52tEPG\nKVhpixu5pfj2cDK2Hr6EBRO6c75cB7ivEDVMU+0rdRb52bNn0bdv3weWazQaCIKAo0eP1rvxvLw8\nrFy5EhkZGRg/fnyNT0+r62T5hpxEn5tbUu86j4LDhc3XgAB3xCWrEJN4C1v2JWLw023FjmTUuK8Q\nNYxBnLXeuXNnfPrpp42+U0dHR+37s7dp0wbW1taQSqUoKyuDhYUFMjMz4eTkBCcnJ2RnZ2tvl5WV\nha5duzb6fonuJZEImBLqg4iNp7Hz6BU84WaHjq3rPv+CiMjY1DlHbmZmBnd39zq/6tOnTx+cOnUK\narUaubm5KCkpQa9evXDw4EEAwKFDhxAUFIQuXbogPj4eBQUFKC4uRmxsLAIDA3X3CMnk2VmbYWqo\nDzTQ4PM9CSgorqj/RkRERqLOZ+T+/v6PtWFnZ2cMHDgQo0aNAgAsXLgQfn5+ePfdd/Htt9/Czc0N\nYWFhkMvlmD17NiZNmgRBEPDmm2/C1pZzcKRbXm0c8OIzHbDz5ytYtzcRs0Z1hUTC+XIiMn58Zzdw\n3s9UqDUarIiMw7nLOQjt3Q5hQR3EjmR0uK8QNYxBvPyMqLmRCAImvdAZji0ssPf4NSRevS12JCKi\nx8YiJ5NiYynHtDBfSCQC1u1NRG5hudiRiIgeC4ucTE4HtxYYHeKBwpJKrNmTgKpqfiwvERkvFjmZ\npH5PtkJgJyekpOdj1y9XxI5DRNRoLHIySYIgYMLgTnB2sMSBmOs4e+nh7yZIRGSoWORksizNZZg+\n3A9ymQQbvkuCKq9U7EhERI+MRU4mrbWTDcIHdERJeRVWRyWgsorz5URkXFjkZPKCuriht58LUm8V\nYtuRS2LHISJ6JCxyIgDhz3nBXWmNn2JvIOZ8pthxiIgajEVOBMBcLsX0MF+Ym0nx5f4LuJlTLHYk\nIqIGYZET/cnV0RoTBndCeWU1VkcloLyyWuxIRET1YpET3aO7tzOCu7njhqoYWw9ehBF+FAERmRgW\nOdF9Xg7xRDsXWxxPuIVf426KHYeI6KFY5ET3kcskmBbmCytzGbYeTsb1TH7aFxEZLhY5US2U9paY\n9II3KqvUWBOVgNLyKrEjERHVikVOVIcATyUGPd0Gmbml2Ph9EufLicggsciJHuLFZzrAs5UdzlxU\n4cff08WOQ0T0ABY50UPIpBJMHeYLWys5vj2SgssZ+WJHIiKqgUVOVA8HW3NMCfWBWq3B51EJKCqt\nFDsSEZEWi5yoAXzaKRDapz1yCsrxxXfnoeZ8OREZCBY5UQMN7dUOPu0cEHc5B/tPpYodh4gIAIuc\nqMEkEgGvD/WBvY0Zdv1yBRev54odiYiIRU70KFpYm2HqMF8IEPD5nkTkF1eIHYmITByLnOgRdWxt\njxF9n0B+cQXWRSdCreZ8ORGJh0VO1AgDu7dGV4+WSErNRdSvV8WOQ0QmjEVO1AiCIGDSC95oaWeB\n705cQ/yVHLEjEZGJYpETNZK1hRzTh/tCJhWwfu953C4oEzsSEZkgmb42HBMTg7fffhuenp4AgI4d\nO6K4uBiJiYmwt7cHAEyaNAl9+/ZFdHQ0Nm3aBIlEglGjRmHkyJH6ikWkU+1cWuCVfp7YcigZa/Yk\n4N0x3SCT8viYiJqO3oocALp3747ly5drL8+bNw+zZs1CcHCwdllJSQlWrVqFyMhIyOVyjBgxAgMG\nDNCWPZGh6xvgjotpeTidlIXIo5fxcj9PsSMRkQkR/anDuXPn4OfnB1tbW1hYWKBbt26IjY0VOxZR\ngwmCgFcHdYKLwgqHfkvD7xdVYkciIhOi12fkKSkpmDp1KvLz8zFjxgwAwNatW7Fx40Y4Ojpi0aJF\nyM7OhkKh0N5GoVBApXr4H0IHByvIZFKdZlUqbXW6PTI9Cyc+jVn//QUb9yehSydnuLa0FjuSXnBf\nIWqYptpX9Fbk7dq1w4wZMzB48GCkpaVh/PjxWLp0KVq2bAlvb2+sW7cOK1euREBAQI3bNeQzn3Nz\nS3SaVam0hUpVqNNtkumxkgkY91xHbNiXhA//dwoLxj0JuY4POMXGfYWoYXS9rzzsoEBvQ+vOzs54\n/vnnIQgC2rRpg5YtW6Jdu3bw9vYGAISEhCA5ORlOTk7Izs7W3i4rKwtOTk76ikWkV739XBHk74rr\nmUX45odLYschIhOgtyKPjo7Ghg0bAAAqlQo5OTn45z//ibS0NAB3zmr39PREly5dEB8fj4KCAhQX\nFyM2NhaBgYH6ikWkd2MHdEQrpQ2O/pGBk4m3xI5DRM2coGnIWHYjFBUVYc6cOSgoKEBlZSVmzJgB\nc3NzfPLJJ7C0tISVlRU+/vhjODo64sCBA9iwYQMEQUB4eDhCQ0Mfum1dD+1xuJB0LfN2CZZ8+RvU\nGg0WvfoU3JvJfDn3FaKGacqhdb0VuT6xyMkYnLmQhdVRCXB1tMLiV5+CuZnxz5dzXyFqmGYxR05k\n6gI7OaH/k61wM6cEmw9eaNCJnEREj4pFTqRHo0I80N61BU4mZuLncxlixyGiZohFTqRHMqkE08J8\nYG0hw9eHLyH1FoeliUi3WOREetbSzhKTX+iMqmo11kQloKSsSuxIRNSMsMiJmkAXj5YY0rMtsvJK\n8b/vkzhfTkQ6wyInaiJhQe3h1doesckqHP4tTew4RNRMsMiJmohUIsEbw3zQwtoMO45eRsqNfLEj\nEVEzwCInakL2NuZ4I9QHao0Ga6ISUFhSIXYkIjJyLHKiJubd1gFhfdojt7Ac6/eeh5rz5UT0GFjk\nRCIY0qsdfDsokHD1NvadTBU7DhEZMRY5kQgkgoDXX+gMB1tzRB27gqTUXLEjEZGRYpETicTWygzT\nwnwhEQSsjU5EXlG52JGIyAixyIlE5OFuh5F9n0BBcQXW7klEtVotdiQiMjIsciKRDXiqNZ7sqMTF\ntDxEHbsqdhwiMjIsciKRCYKACc97Q2lvgX0nU3EuJVvsSERkRFjkRAbAykKG6WF+kEkl+OK788jO\nLxU7EhEZCRY5kYFo62KLMQM8UVxWhc/3JKKqmvPlRFQ/FjmRAXm2ixt6+jjjSkYBth9JETsOERkB\nFjmRAREEAeMGesHV0Qo//J6O3y5kiR2JiAwci5zIwFiYyTB9uB/M5BJs/D4JmbdLxI5ERAaMRU5k\ngNxbWuPVgZ1QVlGNVbsTUFFZLXYkIjJQLHIiA9XT1wXPdnVDuqoIXx1OFjsOERkoFjmRARvT3xNt\nnG1wLO4mjsffFDsOERkgFjmRAZPLpJge5gtLcym2HLyIdFWR2JGIyMCwyIkMnJODFSY+742KKjVW\n705AaXmV2JGIyICwyImMwJNeTnjuqda4dbsEmw5cgEajETsSERkImb42HBMTg7fffhuenp4AgI4d\nO2Ly5MmYO3cuqquroVQq8cknn8DMzAzR0dHYtGkTJBIJRo0ahZEjR+orFpHRGtH3CVzOyMfppCx4\ntbZHcLdWYkciIgOg12fk3bt3x5YtW7BlyxYsWrQIy5cvx5gxY/D111+jbdu2iIyMRElJCVatWoUv\nv/wSW7ZswaZNm5CXl6fPWERGSSaVYNowX9hYyvHNj5dw9WaB2JGIyAA06dB6TEwM+vXrBwAIDg7G\nyZMnce7cOfj5+cHW1hYWFhbo1q0bYmNjmzIWkdFQtLDA60M7o7pagzVRCSguqxQ7EhGJTG9D6wCQ\nkpKCqVOnIj8/HzNmzEBpaSnMzMwAAI6OjlCpVMjOzoZCodDeRqFQQKVSPXS7Dg5WkMmkOs2qVNrq\ndHtE+hKitEXG7VJ8+0Myth6+hAUTukMQhCa7f+4rRA3TVPuK3oq8Xbt2mDFjBgYPHoy0tDSMHz8e\n1dV/vTtVXSfrNOQkntxc3b5lpVJpC5WqUKfbJNKnAd3ccS45CzGJt7B133kMerpNk9wv9xWihtH1\nvvKwgwK9Da07Ozvj+eefhyAIaNOmDVq2bIn8/HyUlZUBADIzM+Hk5AQnJydkZ2drb5eVlQUnJyd9\nxSJqFiQSAW+E+sDO2gyRRy8jOY3nlRCZKr0VeXR0NDZs2AAAUKlUyMnJwYsvvoiDBw8CAA4dOoSg\noCB06dIF8fHxKCgoQHFxMWJjYxEYGKivWETNhp2NOaYO84EGGny+JwEFJRViRyIiEQgaPb0gtaio\nCHPmzEFBQQEqKysxY8YMeHt7491330V5eTnc3Nzw8ccfQy6X48CBA9iwYQMEQUB4eDhCQ0Mfum1d\nD+1xuJCM2b6T17Dz5yvwaeeAv43qColEf/Pl3FeIGqYph9b1VuT6xCIn+otao8HyyDjEXc7BsD7t\nMaxPe73dF/cVooZpFnPkRNQ0JIKAyS90hmMLc0T/ehWJ126LHYmImhCLnKgZsLGUY2qYLyQSAeui\nE5FbWC52JCJqIixyombiCTc7jArxQGFJJT7fk4CqarXYkYioCbDIiZqR/k+2QmAnJ1xKz8fuX66I\nHYeImgCLnKgZEQQBEwZ3grODJfbHXMfZSw9/l0QiMn4scqJmxtJchmlhvpDLJNjwXRJUeaViRyIi\nPWKREzVDbZxtMXZAR5SUV2FNVAIqqzhfTtRcsciJmqkgf1f09nXBtVuF+PbIJbHjEJGesMiJmilB\nEBA+0AvuSmscib2B00mZYkciIj1gkRM1Y+ZyKaaH+cLcTIqN+y/gZk6x2JGISMdY5ETNnKujNV4b\n1AnlFdVYHZWA8srq+m9EREaDRU5kAp7u7Izgbu64oSrG1kMXxY5DRDrEIicyES+HeKKtiy2Ox9/C\nsXMZYschIh1hkROZCLlMgulhvrAyl2Hr4WSkZRWJHYmIdIBFTmRClPaWmPSCNyqr1Fi9Ox6l5VVi\nRyKix8QiJzIxAZ5KDOreBpm5pdi4/wI0Go3YkYjoMbDIiUzQi892gGcrO5y5kIUjsTfEjkNEj4FF\nTmSCZFIJpg7zha2VHNt+vIQrGQViRyKiRmKRE5koB1tzTBnqA7VagzVRCSgqrRQ7EhE1AoucyIT5\ntFdgaO92yCkow4bvzkPN+XIio8MiJzJxob3bo3M7B5y7nIMDMdfFjkNEj4hFTmTiJBIBU4b6wN7G\nDLt+voKL13PFjkREj4BFTkRoYW2GqcN8AQCfRyciv7hC5ERE1FAsciICAHRsbY+X+nZAflEF1kUn\nQq3mfDmRMWCRE5HWoO5t0NWjJZJSc7Hn16tixyGiBmCRE5GWIAiY9II3WtpZ4LsT15BwJUfsSERU\nDxY5EdVgbSHHtDBfSKUC1u09j9sFZWJHIqKH0GuRl5WVoX///ti1axfmzZuHoUOHYty4cRg3bhyO\nHj0KAIiOjsZLL72EkSNHYseOHfqMQ0QN1N61BV7u54mi0kp8vicRVdVqsSMRUR1k+tz4mjVrYGdn\np708a9YsBAcHay+XlJRg1apViIyMhFwux4gRIzBgwADY29vrMxYRNUBwgDuS0/JwOikLkUcv4+V+\nnmJHIqJa6K3IL1++jJSUFPTt27fOdc6dOwc/Pz/Y2toCALp164bY2FiEhIToKxYRNZAgCHh1UCdc\nzyzCod/S8PtFFXKLyuHmaIUhPdvh6c7OYkckIuixyJctW4ZFixYhKipKu2zr1q3YuHEjHB0dsWjR\nImRnZ0OhUGivVygUUKlU9W7bwcEKMplUp3mVSludbo+ouRjUqx2+/O48cv6cK09XFWNtdCJatLDA\nMwGtRE5HZLiaqlf0UuRRUVHo2rUrWrdurV02bNgw2Nvbw9vbG+vWrcPKlSsREBBQ43YN/Vzk3NwS\nneZVKm2hUhXqdJtEzcUPMam1Ll+98xz+uJAFmVSATCq5598730ulEsilEkjvWy77c9md6+4su/f7\nu+tJBAGCIDTxoyXSDV33ysMOCvRS5EePHkVaWhqOHj2KW7duwczMDB988AG8vb0BACEhIYiIiMDA\ngQORnZ2tvV1WVha6du2qj0hE1EgZ2bUfOBeXVuHwmTS93a8A3DkYkAmQSh48UHj4QcOf/0okkMn+\nPHiQ1H37+7dV1/V3D0DurifhgQYZAL0U+Weffab9fsWKFXB3d8c333yD1q1bo3Xr1oiJiYGnpye6\ndOmChQsXoqCgAFKpFLGxsZg/f74+IhFRI7m1tEK6qviB5c4KS0wN9UVVtfrOl1qDqio1qqo1qFar\nUVmlRrVa8+f1mr/Wu+f76moNKqvVqK6xXPPndWpUVmvuXHd322o1SsqrtN9XV2tQLeI70EkE4c6B\nwt0DDdmdg4f7RxwePOi4e8BQcxRCu57kz/VkdR2APHhwUevIh4SjGqZAr2et32vs2LF45513YGlp\nCSsrK3z88cewsLDA7NmzMWnSJAiCgDfffFN74hsRGYYhPdthbXTiA8vD+nRAWxfx91e1RlPrgUDN\ng4LaDySqqjWoUqtrHIDUf9BR2/V/LausVqOkXP3XtqrUEPPNbrWjDJK/DjTuPei49+DgYQcXDxyc\nSCR/buPPdWT3HIDUd6ByzyiKRNK8DjRizmdi38lryMgpabITQwVNQyemDYiu57M5R070cHf+OKXi\nZk4xXB2tMaRnW5613kAajQZqjebOgcK9oww1vq/74KGug44732v+HJmo+0Cm+s+Di+rarv9zpEPM\nUQ1BQK0jCXdGI+496Kj/XIu6pkO0UyySP6+v5fvaRjTu3r6hUygx5zNrPeh9I9TnsfeXJp8jJ6Lm\n5enOzni6szMPehtBEARIBQFSCQC5bl9toysazZ0pihrTIVV/Fn0tUyAPTofUctBw90DjvqmR2g8u\nHtxWRWlljQMdtYjPORGBE6UAAAfUSURBVP+a3qjlHArJXwcaqZlFtd5+38lUvR74ssiJiEycIAja\ncjJU6nvPt1D/daBRXV33+Rh3D0DqmxqpcaCiPdejrgOQO/9WVFajpKyqxnbqcjPnwXNMdIlFTkRE\nBk8iEWAmkcJMLnaS2mk0GizecBo3sh8sbVdHa73et+EefhERERkJQRDwQq92tV43pGdbvd43n5ET\nERHpwN158KY+MZRFTkREpCNinBjKoXUiIiIjxiInIiIyYixyIiIiI8YiJyIiMmIsciIiIiPGIici\nIjJiLHIiIiIjxiInIiIyYixyIiIiI2aUn0dOREREd/AZORERkRFjkRMRERkxFjkREZERY5ETEREZ\nMRY5ERGREWORExERGTEWORERkREzuSKfN28ehg4dinHjxmm/kpKSxI5FZLAmT56M3r1746effsLX\nX38NADhw4IDIqYgMQ3p6Ory9vXHhwgXtsl27dmHdunVYvHgxAOBvf/sbysrKkJGRgbi4OJ1nkOl8\ni0Zg1qxZCA4OFjsGkVH44osvMG/ePADAmDFjAADr1q3DoEGDxIxFZDA8PDzwn//8B+vXr9cua9my\nJf6/vfsLafr74zj+nIkkSNlKKbzIsFqZf5LUSivBgogiKYNYYlGGRSkVFUImthIyk+zC/gghmpoS\ntiBCCbyRAgvUdBdCWZNAyZCsxDJhut9F/Eb+fub39wvN3F6Pq/H57HzOOYPx4pyN805PTwegqKgI\ngOfPn/Pt2zciIiImtX+3CXKr1crTp08ZHBykt7cXs9nMy5cvuXr1KgDnz5+fMLxTU1PJyclh+fLl\nVFZW8unTJ2JjY6mqqsJgMGC329m6dSsZGRl0dHRgsVgwGAxERUWRlZX1p6YpMm16enq4cuUK8+fP\n59WrV2RkZFBcXDzdwxKZdqtWrWJoaIimpibWr1/vur57926sViuJiYlUVVVRXFyMt7c3ixYtYvPm\nzZPWv1ttrb9584Zbt25RXl7OzZs3sdlsDA8PMzo6SmtrKxs3bvy/n2mz2cjPz6empoaKigoA8vLy\nsFgs1NTU8PHjR3p6eiZ7KiJ/rcOHD+Pn56cQF/nJqVOnuH79Or869XzOnDns2rWL/fv3T2qIgxut\nyAFiYmLw9vbGaDTi7+9PeHg4jY2NBAQEEB0djY+PDwDXrl2jtLTU1a6wsPCXzwwNDcXX13fMta6u\nLlasWAFAQUHBFMxERERmkuDgYEJDQ6mrq/vjfbtVkI+OjrpeO51OzGYzZWVlBAUFsWPHDte9f/qN\n3OFwuF57e//3R+Tl5VYbGSLjGhgYYPbs2fj4+DA6OsqsWbOme0gif7Xjx4+TlpZGSkrKuNkxVdwq\nkdra2hgZGaG/v5+vX78SFhbGhw8fsNlsxMTETNjWz8+Pvr4+AFpbWyd8b0hICO3t7QCcO3eOt2/f\nTs4ERP4iFouFhoYGnE4ndrvdtaMF/HL7UMSTLViwgC1btlBTUzPufYPBMGahOFncKsiDgoI4ceIE\nBw4c4OTJk3h5eREfH09YWBgGg2HCtnv37uXixYukp6cTGBg44Xuzs7PJz8/HbDYzd+5cQkJCJnMa\nIn+FzMxMysvLMZvNJCQkjPkOrVy5kj179kzj6ET+TocOHaK3t3fce1FRUdy5c4dHjx5Nap9uU4/c\narXS2dk55h/kTqeTgwcPYrFYWLx48TSOTkREPElXVxdZWVncv39/yvtyqxX5z7q7u0lOTiYuLk4h\nLiIif8z379/JzMwkISHhj/TnNityERERT+S2K3IRERFPoCAXERGZwRTkIiIiM5iCXMRDdHd3YzKZ\nqK6uHnO9ubkZk8nEixcvftm2sbGRz58/A5CYmMi7d+9+awwOhwOTyfRbbUVkfApyEQ8SHByM1Wod\nc81qtbJkyZIJ25WVlfHly5epHJqI/Ca3OqJVRCYWGBjI8PAwnZ2dLFu2jKGhIVpaWoiMjASgrq6O\nyspKnE4nRqORvLw86uvraW5u5syZM1y+fBmAx48f09LSQk9PD7m5ucTFxdHV1UVubi5OpxOHw8Hp\n06eJjo7Gbrdz9uxZfH19Wbt27XROX8QtaUUu4mGSkpJ48OABAE+ePGHTpk14eXnx/v17bt++TVlZ\nGdXV1cTGxlJSUsK+ffsICAigsLCQpUuXAmA0GiktLeXYsWPcvXsX+FEV0Gw2U1FRwYULF1yHM924\ncYPk5GQqKyu1rS4yBRTkIh5m27Zt1NfX43A4ePjwITt37gTAx8eHvr4+0tLSSE1Npa6uzlV/4D/F\nxsYCsHDhQgYGBgBob28nPj4eAJPJxODgIP39/bx+/Zo1a9YAsG7duqmenojH0da6iIcxGo2EhoZS\nW1tLX18f4eHhwI8gj4iIoKSk5B+f8XNlp3+fKTVePQODwYDT6XRVDBwZGZmMKYjIT7QiF/FASUlJ\nFBUVsX37dte1oaEhbDabaxVeX19PQ0MD8L9VbYqMjOTZs2cAdHR04O/vz7x58wgJCaGtrQ2Apqam\nqZiOiEdTkIt4oMTERJxOp2tbHX78ES47O5sjR46QkpJCbW0tq1evBmDDhg0cPXp0whK/OTk53L9/\nn9TUVC5dukRBQQHwo0bzvXv3SEtLw263/9E6zSKeQGeti4iIzGBakYuIiMxgCnIREZEZTEEuIiIy\ngynIRUREZjAFuYiIyAymIBcREZnBFOQiIiIz2L8A/N1IHUukVsUAAAAASUVORK5CYII=\n", 335 | "text/plain": [ 336 | "" 337 | ] 338 | }, 339 | "metadata": { 340 | "tags": [] 341 | } 342 | } 343 | ] 344 | }, 345 | { 346 | "metadata": { 347 | "id": "xnOl22rJqHTm", 348 | "colab_type": "text" 349 | }, 350 | "cell_type": "markdown", 351 | "source": [ 352 | "# 5. The @vectorize wrapper" 353 | ] 354 | }, 355 | { 356 | "metadata": { 357 | "id": "QKNgTakJp_L8", 358 | "colab_type": "code", 359 | "colab": {} 360 | }, 361 | "cell_type": "code", 362 | "source": [ 363 | "#\n", 364 | "# Making Ufuncs with Numba.\n", 365 | "#\n", 366 | "\n", 367 | "@vectorize\n", 368 | "def vec_func(a, b):\n", 369 | " # Now we can pass arrays too, and operate\n", 370 | " # inside like they are scalars:\n", 371 | " for i in range(100000):\n", 372 | " a = math.pow(a*b, 1./2)/math.exp(a*b/1000)\n", 373 | " return a\n", 374 | "# This is similar to functions before, for comparison. But..." 375 | ], 376 | "execution_count": 0, 377 | "outputs": [] 378 | }, 379 | { 380 | "metadata": { 381 | "id": "hobQGwJCrjEA", 382 | "colab_type": "code", 383 | "colab": { 384 | "base_uri": "https://localhost:8080/", 385 | "height": 34 386 | }, 387 | "outputId": "6f89dc77-a79c-43dc-bfee-9b6ba34e9fce" 388 | }, 389 | "cell_type": "code", 390 | "source": [ 391 | "%timeit res = vec_func(a, b)" 392 | ], 393 | "execution_count": 20, 394 | "outputs": [ 395 | { 396 | "output_type": "stream", 397 | "text": [ 398 | "1 loop, best of 3: 35.4 s per loop\n" 399 | ], 400 | "name": "stdout" 401 | } 402 | ] 403 | }, 404 | { 405 | "metadata": { 406 | "id": "RrOcEpPgrn6B", 407 | "colab_type": "code", 408 | "colab": {} 409 | }, 410 | "cell_type": "code", 411 | "source": [ 412 | "#\n", 413 | "# This is slow because previously we were doing some\n", 414 | "# operations on 1,00,000 scalars obtained by multiplying\n", 415 | "# (a@b), but now we are multiplying individual elements \n", 416 | "# of a and b for 1,00,000 times. Also numba is taking care\n", 417 | "# of broadcasting too. So, in this case we are applying this\n", 418 | "# loop for 100 times.\n", 419 | "#\n", 420 | "res.shape # Previously it was (1, 100)" 421 | ], 422 | "execution_count": 0, 423 | "outputs": [] 424 | }, 425 | { 426 | "metadata": { 427 | "id": "nZSgu8R8sJaM", 428 | "colab_type": "code", 429 | "colab": {} 430 | }, 431 | "cell_type": "code", 432 | "source": [ 433 | "@vectorize(['float64(float64, float64)'], target='parallel')\n", 434 | "def vecParallel_func(a, b):\n", 435 | " for i in range(100000):\n", 436 | " a = math.pow(a*b, 1./2)/math.exp(a*b/1000)\n", 437 | " return a" 438 | ], 439 | "execution_count": 0, 440 | "outputs": [] 441 | }, 442 | { 443 | "metadata": { 444 | "id": "Z0piW2xJtgHc", 445 | "colab_type": "code", 446 | "colab": { 447 | "base_uri": "https://localhost:8080/", 448 | "height": 34 449 | }, 450 | "outputId": "b7fe8658-0237-451b-dca1-78571cb6e8ac" 451 | }, 452 | "cell_type": "code", 453 | "source": [ 454 | "%timeit res = vecParallel_func(a, b)" 455 | ], 456 | "execution_count": 23, 457 | "outputs": [ 458 | { 459 | "output_type": "stream", 460 | "text": [ 461 | "1 loop, best of 3: 18.6 s per loop\n" 462 | ], 463 | "name": "stdout" 464 | } 465 | ] 466 | }, 467 | { 468 | "metadata": { 469 | "id": "rLiJPLY0tjvs", 470 | "colab_type": "code", 471 | "colab": {} 472 | }, 473 | "cell_type": "code", 474 | "source": [ 475 | "@vectorize(['float64(float64, float64)'], target='cuda')\n", 476 | "def vecCuda_func(a, b):\n", 477 | " for i in range(100000):\n", 478 | " a = math.pow(a*b, 1./2)/math.exp(a*b/1000)\n", 479 | " return a" 480 | ], 481 | "execution_count": 0, 482 | "outputs": [] 483 | }, 484 | { 485 | "metadata": { 486 | "id": "naP5t1btub6C", 487 | "colab_type": "code", 488 | "colab": { 489 | "base_uri": "https://localhost:8080/", 490 | "height": 34 491 | }, 492 | "outputId": "a9231757-14c6-46a2-841b-8e9105d20caf" 493 | }, 494 | "cell_type": "code", 495 | "source": [ 496 | "%timeit res = vecCuda_func(a, b)" 497 | ], 498 | "execution_count": 25, 499 | "outputs": [ 500 | { 501 | "output_type": "stream", 502 | "text": [ 503 | "1 loop, best of 3: 282 ms per loop\n" 504 | ], 505 | "name": "stdout" 506 | } 507 | ] 508 | }, 509 | { 510 | "metadata": { 511 | "id": "ymlt0rKIuhf-", 512 | "colab_type": "code", 513 | "colab": {} 514 | }, 515 | "cell_type": "code", 516 | "source": [ 517 | "# Woah!!" 518 | ], 519 | "execution_count": 0, 520 | "outputs": [] 521 | }, 522 | { 523 | "metadata": { 524 | "id": "fH42xHl5xNiZ", 525 | "colab_type": "code", 526 | "colab": { 527 | "base_uri": "https://localhost:8080/", 528 | "height": 417 529 | }, 530 | "outputId": "0d9443e8-a58f-4c59-9c14-90c1a888bad9" 531 | }, 532 | "cell_type": "code", 533 | "source": [ 534 | "fig, ax = plt.subplots(1, 1)\n", 535 | "ax.plot([\"a\", \"b\", \"c\"], [35.8, 19.2, 0.568], \"-o\")\n", 536 | "ax.set_xticklabels([\"Vectorize\", \"Vectorize Parallel\", \"Vectorize Cuda\"])\n", 537 | "fig.suptitle(\"@vectorize wrapper\", fontsize=17)\n", 538 | "ax.set_ylabel(\"Time (sec)\")\n", 539 | "ax.set_xlabel(\"Method\")" 540 | ], 541 | "execution_count": 162, 542 | "outputs": [ 543 | { 544 | "output_type": "execute_result", 545 | "data": { 546 | "text/plain": [ 547 | "Text(0.5,0,'Method')" 548 | ] 549 | }, 550 | "metadata": { 551 | "tags": [] 552 | }, 553 | "execution_count": 162 554 | }, 555 | { 556 | "output_type": "display_data", 557 | "data": { 558 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfoAAAF/CAYAAAChaAsiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XlcVOX+B/DPDMMumwgoKqjowCCi\nIIK4o+aSu1kZhuZ2vZUtt3tTuyq5m+btVvYrk1A0c0krs3C7buUGiJrIJqIIigiIbLLDPL8/vHIj\nQUhnOMPweb9evF5xOOc5X4YeP3Oe85xnZEIIASIiItJLcqkLICIiIu1h0BMREekxBj0REZEeY9AT\nERHpMQY9ERGRHmPQExER6TEGPem9O3fu4NNPP8ULL7wAHx8fdO3aFf7+/pg6dSq++eYblJWVSV2i\nxty6dQuurq4ICwuTuhQi0hEyPkdP+mznzp344IMP0Lt3b4wfPx5KpRJmZmbIy8vDhQsXsH37dlRU\nVGDDhg3o2LGjJDWGhITg4sWL+Pzzz5+6raqqKty7dw8tWrSAqampBqojoqaOQU96a926dQgPD8f6\n9evh4eFR6z6VlZUIDg7GuXPnsG/fPknC8a9//SvkcrlGgp6I6I84dE966ejRo9i9ezd27twJDw8P\nnDx5Es8//zw8PT0xePBg/PDDD/j4448xefJkLF++HMbGxtizZw8AYM+ePXB1dUVsbGyNNisrK9G7\nd28sWLCg+vsvvvgCY8aMQffu3dGnTx8sWrQIeXl5NY6Li4vD9OnT4eXlhd69e+P1119HSkoKAGDw\n4ME4fvw4jh49CldXV3z//fcAgKysLMybNw/+/v7w8PBAQEAAVq9ejdLS0up2g4KCMHPmTGzYsAE9\ne/bEv//970eG7hcsWABXV9davyIjI6vbOnjwICZPngwfHx/07NkTs2bNQlJSUp2v77///W/4+PhA\nrVZXb/vmm2/g6uqK0NDQGvv2798fa9asQWRkJFxdXbF//35MmDABHh4eKC8vBwB8/fXXGDNmDDw8\nPODj44PAwEBERERUt/Hw99qxYwdWrlwJf39/eHp64uWXX8bVq1er91uwYAEGDhyIixcvYuLEiejW\nrRv69euHzz77DL+/pikpKcGaNWswYsQIeHp6YuDAgVizZk2N13fBggUYMWIEvv/+e/j7++Mf//hH\nna8HkU4TRHpoxIgRYu/evUIIIQ4cOCDc3NzExx9/LFJTU0VERIQYPXq0GDp0qFi1apUQQojNmzeL\nGTNmCCGEKCwsFN26dRNr166t0eaJEyeEUqkUERERQgghli1bJtzd3cWWLVtEamqqOHHihAgICBAv\nvviiUKvVQgghbty4Iby8vMTf/vY3kZCQIOLi4kRgYKAYMGCAKCwsFDk5OWLAgAFi5syZIisrS5SU\nlIiysjIxYsQI8cwzz4hffvlFpKWliR9//FF4e3uLN954o7qel19+WQwZMkS8/vrr4urVqyIvL0/c\nvHlTKJVKsXnzZiGEEAUFBSIrK6v6KyMjQ4wfP14MGzZMFBQUCCGEOHjwoFAqlWLhwoUiOTlZXLp0\nSUydOlX4+vqKO3fu1Pr6njt3TiiVSnH58uXqbXPnzhWDBg0Sf/nLX6q3JScnC6VSKc6cOSMiIiKE\nUqkUo0ePFuHh4SI9PV2o1Wrx448/CqVSKUJDQ8WtW7dEcnKyePfdd0X37t1Fenq6EEJU/14DBw4U\n69evF9evXxeRkZFiyJAhIiAgQJSVlQkhhJg/f77w9PQUgYGBIjIyUly/fl189NFHQqlUiu3bt1fX\nNWfOHOHt7S1++OEHkZqaKsLDw4Wvr6/429/+Vr3P/PnzRd++fcUrr7wi4uLiRE5OToP+3yPSNQx6\n0jvnzp0Tvr6+ory8XJSUlAg/Pz+xYsWKGvscOnRIKJVKcejQISGEEL/88osYPnx49c/ffvttMXjw\n4BrH/OMf/xABAQFCrVaL7OxsoVKpqt8oPHT8+HGhVCrFqVOnhBAP3gx4e3uLkpKS6n2Sk5PFO++8\nIxISEoQQQgQEBIhXX321+uc///yzUCqVIioqqkbbn3/+uXB1dRW3b98WQjwIejc3N5GdnV29zx+D\n/o+WLFkivL29RXJycvW20aNHiwkTJtTYLycnR3h6eoqPPvqo1nYqKiqEt7e32LRpkxBCiKqqKtGr\nVy/x5ZdfCm9vb1FZWSmEEGLbtm2iR48eoqysrDroly5dWqOtgoICceXKlRrbrl69KpRKpfjhhx9q\n/F5Tpkypsd/D1+rkyZNCiAfhrFQqxdmzZ2vsN3z4cPHiiy8KIYS4fPmyUCqVYsuWLTX22bZtm1Aq\nlSIlJaVGWzExMbW+BkRNBYfuSe9ERkbCz88PhoaGOHXqFHJzc/HKK6/U2Ecuf/C/fs+ePQE8GMpV\nKBTVPx83bhxu3bqFmJgYAEBpaSmOHj2KcePGQSaTISYmBlVVVejXr1+Ndnv37g0DA4PqYf+YmBio\nVCqYmJhU7+Pi4oJ//etfcHNzq7X+y5cvw8DAAD169Kix3cvLC0IIxMfHV29r06YNWrVq1aDXZdeu\nXdixYwfWrVsHFxcXAMD9+/eRlJSEvn371ti3ZcuWUKlUj9y+eEihUMDf3796+D8+Ph4lJSUIDAyE\nWq1GXFwcAODs2bPw9/eHkZFR9bF/nC9hamqK06dPY9KkSfD394eXlxcmTZoEAI/cBunVq1eN7z09\nPQEAN2/erN4ml8ur/66/3+/hPhcvXgSAR/52ffr0AYAav7NcLkfXrl1rfQ2ImgpF/bsQNS2ZmZlo\n27YtAOD69euwtLSs/v6hX3/9Fe3bt4etrS0AICUlpcas+379+qFly5Y4cOAAPD09cfz4cRQVFWH8\n+PEAgMLCQgDA3Llzq980PFRVVYXMzEwAQEFBATp06PCn6i8sLISJiQkMDQ1rbG/RogUAoKioqHqb\nhYVFg9qMjo7G8uXL8dZbbyEgIKB6+/379wEAmzdvxrZt22ocU1ZWhk6dOtXZZv/+/bFu3Tqo1Wqc\nOXMGPXr0QIsWLeDl5YXIyEh4eHggKioK77zzTo3jLC0ta3y/Zs0abN26FbNmzcLw4cNhZWWFzMxM\nBAUFPXLOPx5rbm4O4MHr/JCpqekjr525uTny8/MB/O9vN3HiRMhkskfO8fBv9/C4P/59iZoaBj3p\nnd9fmZuamtaYhAUA165dw759++Dv7w8AUKvV2LdvH+bMmVOjjVGjRuHQoUOYP38+wsPD4e3tDWdn\nZwCAlZUVAGDVqlW1zuh/GMq2trbVAdNQlpaWKCkpQXl5eY0r4YcB9cewq09GRgbefPNNDB48GK++\n+mqNn1lYWEAmkyEwMBBTpkx55Njfv5Z/1K9fPwQHByMxMRERERHw8/MDAPj6+iIyMhL+/v7Iz8/H\ngAEDHlvfvn37MHToULz77rvV2+7du1frvr9/kwP87zWxtrau3lZWVoaqqioYGBjU2O/hPg//dl99\n9RXs7OweOcfDnxPpC75VJb3j5ORUPWPc398fRUVF+Oabb3D//n2cOnUKCxcuRN++fVFcXIzCwkIs\nXboUZmZmGD16dI12xo0bh/T0dERFReGXX37BhAkTqn/m6ekJhUKB9PR0ODs7V3+1b98e5eXlaNmy\nJQCgW7duiI+Prw4k4MEM8pdeeglnzpyp3vb7NyOenp5Qq9U4f/58jXrOnz8PAwODPzWUXFpaitdf\nfx22trb44IMPHvm5ubk5lEolrl+/XuP3cHZ2RmVlZa1B+FDbtm3RqVMnnD59GufPn0fv3r0BPBhe\nP3/+PM6ePYvOnTvD0dHxsTWWl5fDxsamxrbvvvuu1n1/PxMfAC5dugQA1bcigAdPQ5w7d67Gfpcv\nX0bnzp0BPLgFAjx4suH3v6+9vT2Amm8aiPQBg570TkBAAKKionDz5k107twZK1euRFhYGAYMGICt\nW7di7dq1CAoKwvXr1zFw4EAUFxcjJCSkxhUg8CCkO3XqhA8++AAymQwjR46s/lnLli0xefJkbNiw\nAbt370ZqaioSEhLw3nvvYdKkSUhNTQUATJs2DXK5HH//+99x7do1JCYmYvHixcjIyKgeCbCyskJS\nUhJiY2ORkZGBIUOGoEuXLggODsaZM2dw8+ZN7N69G6GhoZg4ceJjw/eP/vnPfyIlJQVLly5FUVER\nsrOzq78eXh2/9tprOHXqFNatW4ekpCTcuHEDISEhGDNmDA4ePPjY9vv164cdO3ZALpeje/fuAB68\nURFCYOfOnfVezQOAt7c3jh49iujoaKSkpGDVqlWQy+UwMDBATEwM7t69W71vamoqPvnkE1y/fh1R\nUVH45JNP0KFDhxr35I2NjfHxxx8jIiICKSkp+Oijj3Djxg0899xzAAB3d3cEBARg1apV2L9/P27e\nvIlLly7hzTffxOTJk//0CAyRzpN2LiCRdixatEi88MILorCw8LH7FRYWVs8Qr80XX3whlEpljceu\nHqqsrBRffvmlGDZsmOjatavw8vISM2fOFJcuXaqx36VLl0RQUJDo0aOH8PX1FX/961/FjRs3qn++\nf/9+4efnJzw8PERISIgQQoisrCzx7rvvCj8/P+Hu7i4GDx4sPv30U1FeXl593MsvvyzGjh1b41x/\nnHWvVCrr/Pr000+rjzt8+LCYNGmS6Natm/Dw8BATJkwQ+/bte+xrJ8SDpxWUSmX1o4kPTZs2TSiV\nSnH69OnqbQ9n3f/nP/+psW9aWpqYOnWq6NGjh+jbt69Yu3atqKioEKtWrRLdunUTr7/+evXvtXHj\nRrFixQrRu3dv4eHhIaZMmVLjCYL58+eLnj17iujoaDFx4kTh4eEh+vbtK/7v//6vxjlLSkrEmjVr\nxKBBg4S7u7vo1auXePPNN8W1a9ceaYuoqePKeKSXysrK8NZbb+Hq1auYM2cOBg4cCAcHBwghkJub\ni7i4OBw/fhzh4eH49NNPq+8vk266desWhgwZgvfee++RJyh+b8GCBThy5Aiio6MbrzgiHcfJeKSX\njI2N8cUXX+Cnn37Cjh07sHTpUgAP7oWr1Wq4uLigf//+2L59e437u0RE+oZBT3pLJpNh7NixGDt2\nLMrLy5GbmwvgwWQrY2NjiasjImocHLonIiLSY5x1T0REpMcY9ERERHqMQU9ERKTHGPRERER6jEFP\nRESkxxj0REREeoxBT0REpMcY9ERERHqMQU9ERKTHGPRERER6jEFPRESkxxj0REREeoxBT0REpMcY\n9ERERHqMQU9ERKTHGPRERER6jEFPRESkxxj0REREeoxBT0REpMcY9ERERHqMQU9ERKTHGPRERER6\njEFPRESkxxRSF/A42dmFGm/TxsYMubnFGm+XSN+wrxA1jKb7ip2dhcbaAprhFb1CYSB1CURNAvsK\nUcPoel9pdkFPRETUnDDoiYiI9BiDnoiISI8x6ImIiPQYg56IiEiPMeiJiIj0GIOeiIhIjzHoiYiI\n9JhOr4ynSZHxmQg/ewO3c4rhaGuGUf4d4OfuIHVZREREWtUsgj4yPhNf7our/v5WdlH19wx7IiLS\nZ81i6D787I06tqc2ah1ERESNrVkE/e27tX/YQEZOUSNXQkRE1LiaRdA7tjKrdXtLS+NGroSIiKhx\nNYugH+Xfodbt2XmlCA2Px/2SisYtiIiIqJE0i8l4DyfchZ9NRUZOEdrYmsNXZY/oK1k4ffkOLiXn\n4KUhXdC7qwNkMpnE1RIREWmOTAghtNFwSUkJFixYgJycHJSVleG1117DoUOHEBcXB2trawDAzJkz\nMWjQoDrbyM4u1HhddnYW1e1WqdX4z7lb2HvqOsor1OjawQZBw11hb1P7UD9Rc/L7vkJEddN0X7Gz\ns9BYW4AWg37//v1IT0/H7NmzkZ6ejhkzZsDLywvDhw9HQEBAg9rQdtA/dDevBF8fTsLl6zkwVMgx\nrl9HDOvVHgqDZnFng6hWDHqihtH1oNfa0P2zzz5b/d8ZGRlwcNDd59VbWZvi7ec9cS4xC9v/k4Q9\nJ64hIi4Tr4x0QydHS6nLIyIiemJau6J/aPLkybhz5w42bNiAsLAwZGdno6KiAra2tli8eDFatmxZ\n57GVlVVQKAy0Wd4j7heXIyw8HociUiGTAaP6dkTQSBXMTAwbtQ4iIiJN0HrQA0BCQgLmzZuHf/7z\nn7C2toZKpcLGjRtx584dBAcH13lcYw3d1ybpZh62HExERk4xbCyM8fIzSngp7TReD5Gu4tA9UcPo\n+tC91m5Cx8bGIiMjAwCgUqlQVVUFpVIJlUoFABg8eDCSkpK0dfqnpmxvjSXTfTGuX0cUFpdj/feX\n8dn3l5FbWCZ1aURERA2mtaCPjo7Gpk2bAAB3795FcXExgoODcfPmTQBAZGQkunTpoq3Ta8TDiXlL\nZ/hC2c4KF5KysTAkAscu3IJa+wMhRERET01rQ/elpaVYuHAhMjIyUFpairlz58LMzAwffvghTE1N\nYWZmhtWrV8PW1rbONqQcuv8jtRA4FZOBb48lo7isEi6Olpg2wg3t7FtovEYiXcChe6KG0fWh+0a5\nR/+kdCnoH8q/X4YdR68iKiELBnIZRvg5YUyfDjAybNxJg0TaxqAnahhdD3o+KP4nWbUwxl/HeeDt\n57vDuoUxws+mIjg0CvE37kldGhER0SMY9E/I08UWK2b5Ybhve2Tnl2Ddzt/w1c/xKCwul7o0IiKi\nas1irXttMTYywIuDu6C3e2uEHUzEmdg7iLmWgxcHd0Yfj9ZcN5+IiCTHK3oNcG5tgUVTe2LykC6o\nqFQjNDwB63b+hszcYqlLIyKiZo5BryEGcjmG9WqP5bN84elii4TUXASHRiH87A1UVqmlLo+IiJop\nBr2GtbIyxVuTPPHqeA+YGivw3S/XsTTsHK6l50tdGhERNUMMei2QyWTo5WaPVbP9MKiHI9Kzi7Dq\n6/PYdvgKSsoqpS6PiIiaEQa9FpmZGGLqCDcsmOKN1rZmOHYhHQtDInD+SrbUpRERUTPBoG8ED9fN\nH9+/I+6XVOD/friM9d/F4F5BqdSlERGRnuPjdY3EUCHH2L4d0cvNHlsPXsHFq3eRkJqLiQM6YbB3\nO8jlfBSPiIg0j1f0jayNrTnmBXph+kg3GMhl2H7kKlZ+fR43s+5LXRoREekhBr0EZDIZ+nd3xMrZ\nvdHb3QEpGQVYuvkcdp9IRllFldTlERGRHmHQS8jS3Ah/GdsV77zQHS0tjXEgIg3BoZGIS+G6+URE\npBkMeh3g0ckWy2f5YaSfE3Lyy/CvXb8h5Kc4FHDdfCIiekqcjKcjjA0N8HxAZ/i5OyDsQCLOxmX+\nd938LujbjevmExHRk+EVvY5xcrDAoqk+eGloF1SqBTbtT8CHOy7izj2um09ERH8eg14HyeUyPOPT\nHitn+aFH51ZITMtDcGgUfjrDdfOJiOjPYdDrsJaWJnjjuW54bbwHzE0V+OHX61i6+RySb3HdfCIi\nahgGvY6TyWTwcbPHylm9EeDVFrfvFmHVtvPYeugKiksrpC6PiIh0HIO+iTAzUSBouCvee7kn2rYy\nx4mL6Vj4VSSiE7MghJC6PCIi0lEM+iamczsrvD+9FyYM6ISikkp8vjcW67+7zHXziYioVny8rglS\nGMgxpk8H+LrZY8vBRPyWfBcJabmY2L8ThvTkuvlERPQ/vKJvwhxamuHdl7ww41kVFHIZdhy9ipVf\nRyMts1Dq0oiISEcw6Js4mUyGfp5tsPIvveHftTVSMgqxLCwa3x5PRlk5180nImruGPR6wtLMCLPH\nuOPvL/aArZUxDkamYXFoJGKv50hdGhERSYhBr2e6dmyJZTP98GxvZ9wrKMNH317Cxn1xKCjiuvlE\nRM0RJ+PpIWNDA0wa5FK9bn5EfCYuX8/B8wGd0d+zDdfNJyJqRnhFr8fa27fAwqCemPKMEpVqgbAD\niVi7/SIycoqkLo2IiBoJg17PyeUyDOnZDitn+cGrSytcuZmH9zdFYd+pFFRUct18IiJ9JxNaWlat\npKQECxYsQE5ODsrKyvDaa6/Bzc0N8+bNQ1VVFezs7PDhhx/CyMiozjayszX/mJidnYVW2m0qzl/J\nxjf/uYK8++VoY2uGaSPcoGxvLXVZpIOae18haihN9xU7OwuNtQVoMej379+P9PR0zJ49G+np6Zgx\nYwa8vb0xYMAAjBw5Eh999BFat26NwMDAOttg0GtHcWklvv/1Go5fSIcAMLCHI54f5AIzE0OpSyMd\nwr5C1DC6HvRaG7p/9tlnMXv2bABARkYGHBwcEBkZiSFDhgAAAgICcPbsWW2dnh7DzESBl4e54p9B\nPdHWzhy//HYbC0MiEZWQyXXziYj0jNZn3U+ePBl37tzBhg0bMH369OqheltbW2RnZz/2WBsbMygU\nBhqvSdPvlpoqOzsL+HRzxA8nkrHz8BVs+DEO0Ul38epET9i3NJO6PNIB7CtEDaPLfUXrQb9z504k\nJCTg3XffrXG12JArx9zcYo3Xw+HIRw3ybANVeytsPXgF0QmZeG3tMUzo3xFDfNrBQM75ms0V+wpR\nwzTbofvY2FhkZGQAAFQqFaqqqmBubo7S0gefspaZmQl7e3ttnZ7+JAcbM/xjcg/MGq2CoUKOnceS\nsWLLeaTe4T/0RERNmdaCPjo6Gps2bQIA3L17F8XFxejTpw8OHToEADh8+DD69++vrdPTE5DJZOjj\n0QYrZ/uhj0drpGYWYtmWc9h17CrXzSciaqK0Nuu+tLQUCxcuREZGBkpLSzF37lx4eHhg/vz5KCsr\ng6OjI1avXg1Dw7pnenPWvbTib9zD1kNXkJVbAltLEwQNV8LTpZXUZVEjYV8hahhdH7rXWtBrAoNe\neuUVVfjpzA0cjExDlVrAV2WPl4YqYWVe9/oHpB/YV4gaRteDnmvd02MZGRrguYEu8FM5YMvBREQl\nZCH2+j28MLgz+nm2gZzr5hMR6TROqaYGaWffAu8F9cTLw5QQ+O+6+d9cwO27XDefiEiXMeipweQy\nGQZ7t8OKWb3RU2mHpFv5WLI5CntPXue6+UREOopBT3+ajYUxXp/YDW9M7AYLMyPsO30DSzZH4Upa\nrtSlERHRHzDo6Yl5Ke2wYpYfhvRshzs5xViz/SLCDiSgqLRC6tKIiOi/GPT0VEyNFZjyjBL/nNoT\n7exa4NdLGVi4MQKR8Vw3n4hIFzDoSSNcHK0Q/IoPnh/kgtLyKny5Lw4f747B3bwSqUsjImrWGPSk\nMQoDOUb2dsayWX7o2sEGl6/nYFFo5H+fwedkPSIiKTDoSePsrU3xzos9MHuMO4wUBvj2eDKWb4nG\njTsFUpdGRNTsMOhJK2QyGfy7tsaqv/RGv25tkJZ5H8u3RGPHkasoLa+UujwiomaDQU9a1cLUEDNG\nqfDuS16wtzbFf6JvYvFXkbiUfFfq0oiImgUGPTUKlbMNls30xeg+HZB3vxyf7InB53tjkXe/TOrS\niIj0Gte6p0ZjqDDAxAGd4Keyx5aDVxCdmIW4lHt4fpALBvRw5Lr5RERawCt6anRt7VpgwcveCBru\nCkBg66Er+OCbC0jnuvlERBrHoCdJyGUyBHi1xYpZveHjaofkW/lYsikKP/x6HRWVVVKXR0SkNxj0\nJCkbC2O8NqEb3nzOE1YtjPDTmRsI3nQOialcN5+ISBMY9KQTenRpheUz/TDUpx2ycouxdsdFbNqf\ngPslXDefiOhpMOhJZ5gaKxA4VIlFU33Q3r4FTsVkYGFIBCLi7nDdfCKiJ8SgJ53TsY0lgl/xwQsB\nnVFWXoWNP8Xjo28vIYvr5hMR/WkMetJJBnI5Rvg5YfksP3h0aom4lHsI/ioSByJTUVnFdfOJiBqK\nQU86zc7aFH97vjv+MtYdxkYG2H38GpZviUZKBtfNJyJqCAY96TyZTIbe7q2xcnZv9Pdsg5tZ97Fi\nSzS2/ycJJWVcN5+I6HEY9NRktDA1xPRnVZgf6AWHlmY4cv4WFn0ViYtXs6UujYhIZzHoqclxdbLB\n0hm9MLZvBxQUlWP9d5fxfz9cRm4h180nIvojrnVPTZKhwgDj+3dCL5UDth5MxPkr2Yi/cQ+TBrpg\noFdbrptPRPRfvKKnJq1tK3PMn+KNaSNcAcjw9eEkfLDtAm5l35e6NCIincCgpyZPLpNhYI+2WDnb\nD73c7JGcno+lm8/h+1+vcd18Imr2GPSkN6xbGOPV8R54a5InrFsY4eczqVgcGoWEG/ekLo2ISDIM\netI73Tu3wvJZfhjWqz2y80rw4c7fEPpzPAqLy6UujYio0THoSS+ZGCkweUgXLJ7mAyeHFjgdewcL\nQyJxNpbr5hNR8yITWvxXb+3atTh//jwqKysxZ84cHDt2DHFxcbC2tgYAzJw5E4MGDarz+OzsQo3X\nZGdnoZV2SXdVqdU4En0LP5y8jvIKNbp2sEHQcFfY25hJXZpOY18hahhN9xU7OwuNtQVo8fG6iIgI\nXL16Fbt27UJubi4mTJiA3r1745133kFAQIC2Tkv0CAO5HMN9ndBTaYevDyfh8vUcLA6Nwrh+HTGs\nV3soDDiwRUT6S2tB36tXL3h6egIALC0tUVJSgqoqzoAm6bSyNsXbz3viXGIWth+5ij0nriEiLhPT\nRrrCxdFK6vKIiLRCq0P3D+3atQvR0dEwMDBAdnY2KioqYGtri8WLF6Nly5Z1HldZWQWFwkDb5VEz\ndL+4HGHh8TgUkQqZDBjVpyOCnlXBzMRQ6tKIiDRK60F/5MgRfPnll9i0aRNiY2NhbW0NlUqFjRs3\n4s6dOwgODq7zWN6jJ21LupmHLQcTkZFTDBsLY0x5RglvpZ3UZekE9hWihtH1e/RavTl58uRJbNiw\nASEhIbCwsIC/vz9UKhUAYPDgwUhKStLm6YnqpWxvjSXTfTGuX0cUFpfjs+8v47PvuW4+EekPrQV9\nYWEh1q5diy+//LJ6lv0bb7yBmzdvAgAiIyPRpUsXbZ2eqMEMFXKM69cRS2f4QtneGheSsrEwJAJH\nz9+CWs1H8YioadPa0P2uXbuwfv16dOzYsXrbxIkTsW3bNpiamsLMzAyrV6+Gra1tnW1w6J4am1oI\nnIrJwLfHklFcVolOjpZ4ZYQb2tm3kLq0Rse+QtQwuj503yiT8Z4Ug56kkl9Ujp1HryIyPhMGchlG\n+DlhTJ8OMDJsPpND2VeIGkbpbTOfAAAgAElEQVTXg54PEBPVwsrcCHPGdsXfXugOGwtjhJ9NRXBo\nFOK4bj4RNTEMeqLH6NbJFstn+mGErxPu5pfiXzt/Q8hP8SjguvlE1ERobcEcIn1hbGSAFwZ3hp+7\nA8IOJuJs3B1cvp6DFwd3Rh+P1pDJZFKXSERUJ17REzWQc2sLLJraE5OHdEFFpRqh4QlYt/M3ZOYW\nS10aEVGdGPREf4KBXI5hvdpjxSw/dHexRUJqLoJDo/DzmRuorFJLXR4R0SMY9ERPwNbKBG9O8sRr\n4z1gZqzA979ex9Kwc0hOz5e6NCKiGhj0RE9IJpPBx80eK2f7YVAPR6RnF2H11+fx9eErKC6tlLo8\nIiIADHqip2ZmYoipI9ywYIo3Wtua4fiFdCz6KgLnr2RBh5epIKJmgkFPpCEP180f378j7pdU4P9+\niMVn31/GvYJSqUsjomaMj9cRaZChQo6xfTvCV+WArQcTcfHqXcSn5uK5AZ0w2Lsd5HI+ikdEjYtX\n9ERa0LqlGd59yQvTn3WDQi7D9iNXsfLr80jL5JKyRNS4GPREWiKTydDf0xErZ/dG764OSMkowLKw\naOw+kYyyiiqpyyOiZoJBT6RlluZG+MuYrnjnhe5oaWmMAxFpWPxVJGJTcqQujYiaAQY9USPx6GSL\n5bP8MNLPCfcKyvDRrkvY+FMcCoq4bj4RaQ8n4xE1ImNDAzwf8GDd/C0HExERl4nL13LwwuDO6Net\nDdfNJyKN4xU9kQScHCywMMgHLw3tgkq1wOb9ifhwx0Xcucd184lIsxj0RBKRy2V4xqc9Vs7yQ4/O\nrZCYlofg0Cj8dDqF6+YTkcYw6Ikk1tLSBG881w2vT/CAuakCP5xMwZLN53D1Vp7UpRGRHmDQE+kA\nmUyGnq72WDmrNwK82yLjbhFWb7uArYeuoLi0QuryiKgJY9AT6RAzEwWChrnivZd7om0rc5y4mI6F\nIZGITuS6+UT0ZBj0RDqoczsrvD+9FyYO6ISi0kp8vjcWn+6JQU4+180noj+HQU+koxQGcozu0wHL\nZ/pC5WyDS9dysOirSBw+dxNqNa/uiahhZKKe8UC1Wo3Y2FjcunULANCuXTt4eHhALtf+e4TsbM2v\nC25nZ6GVdom0SQiBM7F3sOtYMu6XVKBDawtMG+EG59YWWjsn+wpRw2i6r9jZabZf1xn0arUaoaGh\nCAsLg6OjI9q0aQMAuH37Nu7cuYNXXnkFM2bM0GrgM+iJaiooLseuo8k4G3cHcpkMw3q1x7h+HWFs\nZKDxc7GvEDVMkw36WbNmoWvXrnjllVdgY2NT42e5ubkICwtDfHw8QkJCNFrQ7zHoiWoXl3IPWw8l\nIjuvFK2sTBA03BXdOtlq9BzsK0QN02SDPiYmBp6engAeDBs+XJqzsrISCoXikX20gUFPVLeyiir8\ndPoGDkWloUot4OfugMlDusDK3Egj7bOvEDWMrgd9nePuDwP84MGDePXVV6u3BwYG4uDBgzX2IaLG\nZ2xogEmDXBD8Si90crREZHwmFoVE4NdLt/koHhFVq/cGe1hYGD788MPq7zdt2oTNmzdrtSgiarj2\n9i3wz5d7YsozSlSpBcIOJGLt9ovIyCmSujQi0gH1Br0QAhYW/xtGaNGiBT9hi0jHyOUyDOnZDitm\n+cGrSytcuZmH9zdFYd+pFFRUct18ouas3o+p9fDwwNtvvw1fX18IIXDy5El4eHg0qPG1a9fi/Pnz\nqKysxJw5c9CtWzfMmzcPVVVVsLOzw4cffggjI83cTySih+vme+L8lWx8858r2HsqBZEJmZg2wg3K\n9tZSl0dEEqj3OXohBPbt24eYmBjIZDJ4eXlh5MiR9T5WFxERgdDQUISEhCA3NxcTJkyAv78/BgwY\ngJEjR+Kjjz5C69atERgYWGcbnIxH9ORKyirx/S/XcezCLQgAA3s4YtIgF5ibGDboePYVoobR9cl4\n9QY9ACQlJSEtLQ1Dhw5FQUEBLC0t6224qqoKZWVlMDMzQ1VVFfr06QNzc3McPHgQRkZGuHjxIjZt\n2oT169fX2QaDnujpXUvPR9jBRKRnF8HS3AiBQ7ugl5t9vbfg2FeIGkbXg77eofuwsDD8/PPPKC8v\nx9ChQ/H555/D0tISr7322mOPMzAwgJmZGQBgz549GDBgAE6dOlU9VG9ra4vs7OzHtmFjYwaFQjsL\ngRA1F3Z2FvDp5ogfTiRj5+Er2PBjHKKT7uLViZ6wb2lW77FEVD9d7iv1Bv3PP/+Mb7/9FtOmTQMA\nzJs3D5MnT6436B86cuQI9uzZg02bNmHYsGHV2xvy+E9ubnGDzvFn8CqFmqtBnm3g3t4KWw9dQXRC\nJl5dexQT+nfCUJ92MKjlVhz7ClHD6PoVfb2z7s3NzWvcj5fL5Q1e9vbkyZPYsGEDQkJCYGFhATMz\nM5SWPvj0rczMTNjb2z9h2UT0JOxtzPD3F3tg1mgVjBQG2HUsGSu2nEfqHQY6kb6q94reyckJn332\nGQoKCnD48GHs378fLi4u9TZcWFiItWvXIiwsDNbWD2b79unTB4cOHcK4ceNw+PBh9O/f/+l/AyL6\nU2QyGfp4tEG3Trb49lgyTsfewbIt5/CMT3uM798Rl5JzEH72Bm7nFMPR1gyj/DvAz91B6rKJ6AnV\nOxmvoqICW7duRWRkJIyMjNCzZ09MmTKl3sfidu3ahfXr16Njx47V2z744AMsWrQIZWVlcHR0xOrV\nq2FoWPcMYE7GI9K++Bv3sPXQFWTllqCFqSHul1Q8ss+csV0Z9kR10PWh+wbNur9//z5atGiB7Oxs\npKamwtvbmx9TS6RHyiuq8NOZGwg/m1rrz9vZtcCymb6NXBVR06DrQV9vWi9fvhwHDhxAXl4eAgMD\nsW3bNixZskSjRRCRtIwMDfDcQBfI63jijsvpEjVd9QZ9fHw8nn/+eRw4cADjx4/Hxx9/jNTU2t/1\nE1HT5tjKvNbtrW0f/xgeEemuBq11DwAnTpzA4MGDAQDl5eXarYqIJDHKv0Ot24tKKpCWyVteRE1R\nvUHfsWNHjBo1CkVFRVCpVNi7dy+srKwaozYiamR+7g6YM7Yr2tm1gIFchnZ25vDo1BJ598uxfEs0\nDkSmQs2PwCVqUuqcjFdRUQFDQ0NUVVUhKSkJLi4uMDIyQmxsLJycnGBpaVm9j7ZwMh6RdH7fV2Kv\n5yA0PAH5ReVQOdtg5igVWlqaSFwhkW5ospPxZs2ahZSUFBgYGEClUlU/Tufh4QFLS0tcu3YNs2bN\n0mgxRKSbPDrZYulMX3h1aYWE1Fy8vykK5xKzpC6LiBqgziv6q1evYt68eWjdujX69++PNm3aAAAy\nMjJw8uRJZGZmYs2aNejSpYvWiuMVPZF0ausrQgj8cuk2dh69ivIKNfp6tEbgM0qYGte79haR3tL1\nK/rHPkcvhMDRo0fx66+/4s6dOwCA1q1bY8CAARgyZEi9n371tBj0RNJ5XF+5c68YX+6LQ+qdQthZ\nm2D2mK7o3JZzd6h5atJBLzUGPZF06usrlVVq/HgqBfvPpkImk2F0H2eM6duh1g/IIdJnuh707JFE\n9EQUBnI8N9AF8wK9YGNhhH2nb+CDby4gK69E6tKI6HcY9ET0VFydbLB0hi/83B1wLb0A72+KwqmY\njAZ9FDURaV+Dgl6tViM7O1vbtRBRE2VmYog5Y7ti9hh3yGXApv0J+OLHuFo/IIeIGle9QX/27FkM\nHToUQUFBAIBVq1bh+PHjWi+MiJoe/66tsXS6L7q0s0J0Yhbe3xSFhNRcqcsiatbqDfp///vf+Pbb\nb2FnZwcA+Otf/4ovvvhC64URUdPUytoU8wO9MWFAJ+TfL8e6HRfx7fFkVFSqpS6NqFmqN+jNzMzQ\nqlWr6u9btmyp1dXwiKjpk8tlGNOnA/4Z1BN2NqY4GJmGlVujcfsuPwWPqLHVG/QmJiaIiooCAOTn\n52P79u0wNjbWemFE1PR1crTEkum9MKB7G6Rl3cfSsHM4duEWJ+oRNaJ6n6PPyMjAkiVLEBkZCSMj\nI/Ts2RMLFy5Eu3bttF4cn6Mnko6m+8r5K9kIO5CAotJKeLrYYvqzKliZG2msfSKp6Ppz9Fwwh4hq\npY2+kltYhk3h8Yi7kQtLM0PMGKWCp0ur+g8k0mFNPujPnDmD7du3o7CwsMZw29atWzVaSG0Y9ETS\n0VZfUQuBI9G3sOdEMiqrBAK82+KFgM4wNjTQ+LmIGoOuB329n0SxZMkSvPrqq2jdurVGT0xEzZNc\nJsOwXu2hcrbBxp/icPxCOhJTczFnbFc4OWj2HzgiakDQd+jQARMmTGiMWoioGWlv3wLB03yw+8Q1\nHIm+heVbovHcQBcM820PuZY/MIuoOal36P7IkSM4fvw4vLy8oFD8733B+PHjtV4ch+6JpNOYfSX2\neg5CwxOQX1QOlbMNZo5SoaWlSaOcm+hp6frQfb1BP2nSJJiamtYYupfJZFi7dq1GC6kNg55IOo3d\nVwqKy7HlQCIuXr0LcxMFpo5wQy83+0Y7P9GT0vWgr3fo3tDQEF9//bVGT0pE9EeWZkaYO7Ebfrl0\nGzuPXsUXe2MR0601AocqYWpc7z9VRFSHehfMGTx4MCIiIlBeXg61Wl39RUSkaTKZDIN6tMX7r/SC\nc2sLnL58B0s2RyE5PV/q0oiarHqH7r28vFBSUvPzpWUyGRISErRaGMCheyIpSd1XKqvU+PFUCvaf\nTYVMJsOYvh0wuo8zDOT8dG3SLbo+dM8Fc4ioVrrSV66k5SLk53jcKyiDS1tLzB7TFfbWplKXRVSt\nyQb9d999h+eeew6ffPJJrQe+9dZbGi2kNgx6IunoUl8pLq3A14eTEBmfCWMjA7z8jBJ9PFpDxsfw\nSAfoetDXOQYm/+/wmIGBQa1fRESNxczEEHPGdsXsMe6Qy4DQ8AR88WMc7pdUSF0akc6rcyrrwzCf\nO3duoxVDRPQ4/l1bo0tbK4T8HI/oxCxcS8/HrNHuUDnbSF0akc6q84p+z549T914UlIShg4dim3b\ntgEAFixYgDFjxiAoKAhBQUE4ceLEU5+DiJqXVtammB/ojQkDOiH/fjnW7biIb48no7KKTwMR1UZr\nD6cWFxdj+fLl8Pf3r7H9nXfeQUBAgLZOS0TNgFwuw5g+HdC1Q0ts/CkOByPTEH/jHv4ypiscW5lL\nXR6RTqkz6C9evIhBgwY9sl0IAZlMVu/VuJGREUJCQhASEvK0NRIR1aqToyWWTO+FHUeu4mRMBpaG\nncOLgzsjwKstJ+oR/VedQe/u7o6PPvroyRtWKGqsjf/Qtm3bsHnzZtja2mLx4sVo2bJlnW3Y2JhB\nodD8xD9Nz2gk0ldNpa/Mm+aL/pdvY/23v2Hb4SQk3szHmy/2gI0F18unxqHLfaXOoDcyMkLbtm01\nerJx48bB2toaKpUKGzduxGeffYbg4OA698/NLdbo+QHdemSISJc1tb7SubUFlkz3RWh4PKITMjF3\n7THMGKWCp0srqUsjPddkH6/z9PTU6IkAwN/fHyqVCsCDpXWTkpI0fg4iar5sLIzxzos9MHlwZxSX\nVeLj3THYdvgKyiuqpC6NSDJ1Bv27776r8ZO98cYbuHnzJgAgMjISXbp00fg5iKh5k8tkGObrhMXT\neqFtK3Mcu5COpWHnkJbZdEYniDRJa0vgxsbGYs2aNUhPT4dCoYCDgwNefvllbNy4EaampjAzM8Pq\n1atha2tbZxtcGY9IOvrQV8orqrDnxDUcOX8LBnIZnhvogmG+7SHnRD3SIF0fuuda90RUK33qK7HX\ncxAanoD8onKonG0wc5QKLS05UY80Q9eDnh8DRUR6z6OTLZbO9EWPzq2QkJqL9zdF4VxiltRlETUK\nBj0RNQuWZkZ447lumDrcFRWVanyxNxah4fEoKauUujQirdLaynhERLpGJpNhkFdbuDpZY+NP8Th9\n+Q6SbuZh9piu6NzWSuryiLSCV/RE1Oy0sTXHwqCeGOXvjLt5pfhg2wX8eCoFVWqul0/6h0FPRM2S\nwkCO5wa6YF6gF6wtjPDjqRR88M0FZOWVSF0akUYx6ImoWXN1ssGyGb7wVdnjWnoBlmyKwunLGdDh\nB5KI/hQGPRE1e2Ymhpgztitmj3aHTAaEhifgix/jUFRaIXVpRE+Nk/GIiPBgop6/R2t0aWeFjT/H\nIzoxC9fS8zFrtDtUzjZSl0f0xHhFT0T0O62sTTE/0AsT+ndE/v1yrNtxEd8eT0ZlFSfqUdPEoCci\n+gMDuRxj+nbEe0HesLMxxcHINKzYGo3bd4ukLo3oT2PQExHVwcXRCkum90J/zzZIy7yPZWHncOzC\nLU7UoyaFQU9E9BgmRgpMf1aF1yd4wFAhx7bDSfhkTwwKisqlLo2oQRj0REQN0NPVHstm+sG9gw1i\nruUgODQSMdfuSl0WUb0Y9EREDWRjYYx3XuyByYM7o7isEh/vjsG2w1dQXlEldWlEdWLQExH9CXKZ\nDMN8nbBoqg/atjLHsQvpWBp2DmmZ+vGRvqR/GPRERE/AycECi6f5YGjPdsjIKcbyLdE4GJkGNSfq\nkY5h0BMRPSEjQwMEPqPE317oDnNTQ3x7PBn/2vkb7hWUSl0aUTUGPRHRU+rWyRbLZvqiR+dWSEjN\nxfubohCdmCV1WUQAGPRERBphaWaEN57rhqnDXVFRqcbne2MRGh6PkrJKqUujZo5r3RMRaYhMJsMg\nr7ZwdbLGxp/icfryHSTdzMNfxnSFS1srqcujZopX9EREGtbG1hwLg3pilL8z7uaVYvW2C/jxVAqq\n1Fwvnxofg56ISAsUBnI8N9AF8wK9YG1hhB9PpeCDby4gK69E6tKomWHQExFpkauTDZbO8IWvyh7X\n0guwZFMUTl/O4Hr51GgY9EREWmZuYog5Y7ti9mh3AEBoeAK++DEORaUVEldGzQEn4xERNQKZTAZ/\nj9bo3M4KIT/HIzoxC9fS8zFrtDtUzjZSl0d6jFf0RESNyM7aFPMDvTChf0fk3y/Huh0Xsft4Miqr\nOFGPtINBT0TUyAzkcozp2xHvBXnDztoUByLTsGJrNG7fLZK6NNJDDHoiIom4OFphyYxe6O/ZBmmZ\n97Es7ByOX7jFiXqkUQx6IiIJmRgpMP1ZFV6f4AFDhRxfH07Cp3tiUFBULnVppCe0GvRJSUkYOnQo\ntm3bBgDIyMhAUFAQAgMD8dZbb6G8nP8jExEBQE9Xeyyb6Qf3Dja4dC0HwaGRiLl2V+qySA9oLeiL\ni4uxfPly+Pv7V2/79NNPERgYiO3bt8PZ2Rl79uzR1umJiJocGwtjvPNiD7w4uDOKyyrx8e4YbDt8\nBeUVVVKXRk2Y1oLeyMgIISEhsLe3r94WGRmJIUOGAAACAgJw9uxZbZ2eiKhJkstkGO7rhEVTfdC2\nlTmOXUjH0rBzSMsslLo0aqK09hy9QqGAQlGz+ZKSEhgZGQEAbG1tkZ2d/dg2bGzMoFAYaLw2OzsL\njbdJpI/YV6RjZ2cBD1cHhP0ch59PpWDF1mgEjXTH+IEukMtlUpdHf6DLfUWyBXMaMqs0N7dY4+e1\ns7NAdjbfGRPVh31FN0zs1xFdHC0RGp6AzT/HIeLybcwcpUJLSxOpS6P/0nRf0fSbhkaddW9mZobS\n0lIAQGZmZo1hfSIiql23TrZYNtMXPTq3QkJqLt7fFIXoxCypy6ImolGDvk+fPjh06BAA4PDhw+jf\nv39jnp6IqMmyNDPCG891w9ThrqioVOPzvbEIDY9HSVml1KWRjpMJLa3MEBsbizVr1iA9PR0KhQIO\nDg5Yt24dFixYgLKyMjg6OmL16tUwNDSssw1tDBtyOJKoYdhXdFdGThE27otHamYh7KxN8JcxXeHS\n1krqspotXR+611rQawKDnkg67Cu6rbJKjb0nU3AgIhUymQxj+3bAqD7OMJBzHbTGputBz/8jiIia\nIIWBHJMGueDdl7xgbWGEvadSsOabi8jKK5G6NNIxDHoioibMzdkGS2f4wldlj+T0fCzZFIXTlzO4\nXj5VY9ATETVx5iaGmDO2K2aNVgEAQsMTsOHHOBSVVkhcGekCyZ6jJyIizZHJZOjj0QZd2lkj5Od4\nnEvMQnJ6PmaNdofK2Ubq8khCvKInItIjdtammB/ohfH9OyL/fjnW7biI3ceTUVmllro0kgiDnohI\nzxjI5RjbtyPeC/KGnbUpDkSmYcXWaNy+WyR1aSQBBj0RkZ5ycbTCkhm90N+zDdIy72NZ2Dkcv3CL\nE/WaGQY9EZEeMzFSYPqzKrw23gOGCjm+PpyET/fEoKCoXOrSqJEw6ImImgEfN3ssm+kHlbMNLl3L\nQXBoJGKu3ZW6LGoEDHoiombCxsIYf5/cAy8O7oziskp8vDsG2w5fQXlFldSlkRYx6ImImhG5TIbh\nvk5YNNUHjq3McexCOpZtiUZaJpc71lcMeiKiZsjJwQLB03wwpGc73L5bhOVbonEwMg1qTtTTOwx6\nIqJmysjQAFOeUeLt57vD3NQQ3x5Pxr92/obcwjKpSyMNYtATETVzni62WDbDFz06t0JCai6CQyMR\nnZgldVmkIQx6IiKCpbkR3niuG6YOd0VFpRqf743FpvAElJRVSl0aPSWudU9ERAAerJc/yKstXJ2s\nsXFfPE5dzkDSzTzMHuMOl7ZWUpdHT4hX9EREVEMbW3MsnNoTz/Z2RnZeCVZvu4B9p1JQpeZ6+U0R\ng56IiB6hMJBj0iAXvPuSF6wtjLD3VArWfHMRWXklUpdGfxKDnoiI6uTmbIOlM3zhq7JHcno+lmyK\nwunLGVwvvwlh0BMR0WOZmxhiztiumDVaBQAIDU/Ahh/jUFRaIXFl1BCcjEdERPWSyWTo49EGXdpZ\nI+SneJxLzEJyej5mjXaHytlG6vLoMXhFT0REDWZnbYr5U7wwvn9H5N8vx7odF7H7eDIqqzhRT1cx\n6ImI6E8xkMsxtm9HvBfkDTtrUxyITMPKreeRkVMkdWlUCwY9ERE9ERdHK7w/vRf6ebZBamYhlm4+\nh+MXbnGino5h0BMR0RMzNVZgxrMqvDbeA4YKOb4+nIRP98SgoKhc6tLovxj0RET01Hzc7LFsph9U\nzja4dC0HwaGRiLl2V+qyCAx6IiLSEBsLY/x9cg+8ENAZxWWV+Hh3DLYdvoLyiiqpS2vWGPRERKQx\ncpkMI/ycsGiqDxxbmePYhXQs2xKNtMxCqUtrthj0RESkcU4OFgie5oMh3u1w+24RVmyNxsHINKg5\nUa/RMeiJiEgrjAwNMGWYEm8/3x1mJob49ngy/rXzN+QWlkldWrMiE434HERkZCTeeustdOnSBQCg\nVCqxePHiOvfPztb8UI+dnYVW2iXSN+wrpEkFReUIO5CI35LvwtxEgWkj3ODjZi91WRqh6b5iZ2eh\nsbYACZbA9fX1xaefftrYpyUiIglZmhvhjee64cRvt7Hr6FV8vjcW/bq1wUtDu8DUmKuxaxNfXSIi\nahQymQwBXm3h5mSNjfvicepyBpJu5mH2GHe4tLWSujy91ehD90uXLoWTkxPy8/Mxd+5c9O3bt879\nKyuroFAYNFZ5RETUSCoq1fjmYAK+P5EMmUyGyc+44oUhXWBgwKljmtaoQZ+ZmYnz589j5MiRuHnz\nJqZOnYrDhw/DyMio1v15j55IOuwr1BgSU3MR8nM8cgvL0LmtFWaNcYe9tanUZf0pun6PvlHfOjk4\nOODZZ5+FTCaDk5MTWrVqhczMzMYsgYiIdIibsw2WzfSFr8oeyen5WLIpCqcvZ3C9fA1q1KDft28f\nQkNDAQDZ2dnIycmBg4NDY5ZAREQ6xtzEEHPGdsWs0SoAQGh4Ajb8GIei0gqJK9MPjTp0f//+ffzj\nH/9AQUEBKioqMHfuXAwcOLDO/Tl0TyQd9hWSQnZeCUJ+ikdyej5sLIwxe7Q73JxtpC7rsXR96L5R\ng/7PYtATSYd9haRSpVYj/Gwq9p26ASEERvg5YcKATlDo6EQ9XQ963XzViIio2TKQyzG2b0e897I3\n7KxNcSAyDSu3nkdGTpHUpTVJDHoiItJJLm2t8P70Xujn2QapmYVYuvkcjl9M50S9P4lBT0REOsvU\nWIEZz6rw2ngPGCrk+PrQFaz/7jIKisqlLq3JYNATEZHO83Gzx9IZvlA52+C35LsIDo1EzLUcqctq\nEhj0RETUJLS0NMHfJ/fACwGdUVxWiY93X8I3h5NQXlEldWk6jUFPRERNhlwmwwg/Jyya6gPHVuY4\neuEWlm2JRlomnxCpC4OeiIiaHCcHCwRP88EQ73a4fbcIK7ZG42BkGtScqPcIBj0RETVJRoYGmDJM\nibef94SZsQLfHk/Gv3b+htzCMqlL0ykMeiIiatI8XVph2Uw/dHexRUJqLoJDIxGdmCV1WTqDQU9E\nRE2epbkR3pzkiaDhrqioVOPzvbHYFJ6AkrJKqUuTnELqAoiIiDRBJpMhwKst3Jys8eW+OJy6nIGk\nm3mYPcYdLm2tpC5PMryiJyIivdLG1hyLpvpgZG8nZOeVYPW2C9h3KgVVarXUpUmCQU9ERHpHYSDH\n84M6492XvGDVwgh7T6VgzTcXkZ1XInVpjY5BT0REesvN2QbLZvrCV2WP5PR8vL8pCqcvZzSr9fIZ\n9EREpNfMTQwxZ2xXzBqtAgCEhidgw49xKCqtkLiyxsHJeEREpPdkMhn6eLRBl3bWCPkpHucSs5Cc\nno/Zo93h5mwjdXlaxSt6IiJqNuysTTF/ihfG9+uI/Pvl+HDHRew+kYzKKv2dqMegJyKiZsVALsfY\nfh3x3svesLM2xYGINKzceh4ZOUVSl6YVDHoiImqWXNpa4f3pvdCvWxukZhZi6eZzOH4xXe8m6jHo\niYio2TI1VmDGKBVeG+8BQ4UcXx+6gvXfXUZBUbnUpWkMg56IiJo9Hzd7LJ3hC5WzDX5LvovgTVGI\nuZYjdVkawaAnIiIC0NLSBH+f3AMvBHRGcWkFPt59Cd8cTkJ5RZXUpT0VBj0REdF/yWUyjPBzwqKp\nPmhja4ajF25h2ZZopN3o+MoAAAzuSURBVGUWSl3aE2PQExER/YGTgwXef6UXhni3w+27RVixNRoH\nI9OgboIT9Rj0REREtTAyNMCUYUq8/bwnzIwV+PZ4Mv618zfkFpZJXdqfIhM6/BxBdrbmh0rs7Cy0\n0i6RvmFfIfqfgqJybN6fgEvXcmBuosC0EW6oUguEn72B2znFcLQ1wyj/DvBzd3jqc9nZWTx9wb/D\noCeiWrGvENUkhMCJi+nYdSwZ5ZW1r6Q3Z2zXpw57TQc9h+6JiIgaQCaTIcC7Hd6f3guGiv9v795j\nqq7/OI4/4Sc6xZlggv42FKeEikJXTWwuTdcEDa8DhFOb1rR0XbzBdEzDC+a9Vk5jMhXSaKZ/oCCp\nMJsLQ8AkdN7CFM1NNFRABDl8fn8wzy8MDBVPh+Pr8d/58D2f2z4f3t/P9/s930/j4XNvzgU71+qf\nKdCLiIg8hO5d3LE28W58R3yNrgK9iIjIQ/rvs+6Npnfv0nj6v8nugX758uWEh4cTERFBYWGhvYsX\nERF5bKFDfJtI72nfijSDXfejz83N5cKFC6SmpvLbb7+xYMECUlNT7VkFERGRx3bvgbu9ORe4cr2S\n7l3cCR3Ss0Weum9pdg30OTk5jBw5EoDevXtz8+ZNKioq6Nixoz2rISIi8tgG9/dmcH9vh/+Fil0D\n/bVr1wgICLB99vT0pLS0tMlA7+HRgTZt/tPi9Wjpny6IOCvNFZHmceS5YtdAf79/+gl/WdntFi/T\n0c+8RByF5opI87T0XGnVv6P38vLi2rVrts9Xr16la9eu9qyCiIjIU8WugX7o0KFkZmYCcOLECby8\nvHR/XkRE5Amy66X7F198kYCAACIiInBxcWHRokX2LF5EROSpY/d79HPnzrV3kSIiIk8tvRlPRETE\niSnQi4iIODEFehERESfm0PvRi4iIyOPRil5ERMSJKdCLiIg4MQV6ERERJ6ZALyIi4sQU6EVERJyY\nAr2IiIgTU6AXERFxYg4d6MPDwykqKmqQtmbNGpKSkpqdx759+5p97K5du9i/f3+zjxd5XC0xxsG+\n4zw2NpaxY8disViIiorio48+oqKi4pHzAxgxYgSVlZXExsaSnZ3d5HEWi4UzZ848VlniHFrj3AHY\nvHkz48ePJzIykoiICH7++edmf3fChAlcunTpoct06EA/ZswYMjIyGqT98MMPhIaGNjuPr7/+utnH\nTpgwgVGjRjX7eJHH1RJjHOw/zmfPnk1ycjLffPMNfn5+bNu27bHyE3lYrXHupKWlcfToUVJTU9mx\nYwcJCQnMnz+fmzdvPnKezWH33eseRkhICJGRkcybNw+AoqIivLy8KC8vZ968ebi4uODu7s6KFSvo\n1KkTiYmJZGZm4urqyuzZsykqKuL06dPMmjWLL7/8kpUrV1JQUIDVaiUqKopx48ZhsVjw8/MDwMPD\nAw8PD7y9vW3/uK5cuUJwcDDx8fGsW7eOvLw8rFYr0dHRjBkz5l/rG3EOTY1xb29vzp07R3x8vMOP\n88DAQPbu3QtAQkIChYWFVFdXExkZyeTJk4mNjcXNzY0bN26QkJDAnDlzuH37Nnfu3CEuLo7AwMC/\n5Wm1WomLi6OkpITa2lo+/PBDhgwZ0pJdL61ca5w7ycnJLF++nLZt2wLQq1cv0tLS6NSpE7Gxsbz5\n5psMHz6c7OxsMjMzWbFiBUuXLuXYsWP06tWLu3fvAnDq1Ck+/fRT2rRpg6urK59//jmdO3duurOM\ng5s6dao5fvy4McaYlStXmtTUVPP222+b8+fPG2OMSUlJMRs2bDDnz583EydONFar1fz+++9mwYIF\nxhhjBg0aZIwxJjc317z77rvGGGMqKyvNG2+8YcrLy010dLTZvn27McaYL774wiQnJ9vKLi8vN+PG\njTMXL140R48eNXPmzDHGGFNdXW1CQkJMVVWVXfpAnFtjY9wY47DjPCYmxmRlZdk+x8fHm5SUFHPn\nzh2zdetWY4wxVVVVZujQobbjV61aZYwxpri42Ozfv98YY8xPP/1kZs2aZYwxZvjw4aaiosKW9+7d\nu83atWuNMcZcv37djBkzxhhjTHR0tDl9+vRj9LY4k9Y2d4KDg01dXV2jbfnrvMrKyjIxMTHm7Nmz\nZvz48cZqtZo//vjDBAQEmJKSEnP48GFz4sQJY4wx69evN9u2bXtgPzn0ih7qL8+kp6cTGBhIVlYW\n3377LQkJCcTFxQFQU1PDwIEDOXnyJEFBQbi6utKzZ0+WLVvWIJ+ioiJeeeUVADp06ECfPn24cOEC\nQKMrCoD4+HimTp2Kj48PGRkZHD9+HIvFAkBdXR2lpaX4+Pg8qabLU6KxMQ5QWFjosON87dq1JCUl\nUVdXR2BgIJMnT6Zt27bcvHmTiIgI3NzcKCsrsx1/r+xnn32WDRs2sHnzZmpqaujQoUOjdTp27Bj5\n+fkUFBQAUF1dTU1NTfM7VZ4KrW3uGGMwxuDi4tKs9p07d85W5+7du9vy6tKlC6tXr+bOnTtcvXqV\nsWPHPjAfhw/0o0aNYuPGjYSGhuLr68szzzxD+/bt2bZtW4POyszMpK6ursl87u/Yu3fv4upa/4iC\nm5vb345PS0vDxcXF1oFt27Zl0qRJTJ8+vSWaJWLT2BgHHHqcz549m+HDhzdIy83N5ciRIyQnJ+Pm\n5sYLL7xg+9u9srdu3Yq3tzerVq3i119/ZeXKlY3m7+bmxowZM3R7TB6otc0dHx8fTp48yYABA2xp\np06donfv3g3Kr62tBepPDO7VAbDVf9myZbz33nsMGzaMzZs3c/v27SbLBAd/GA+gY8eO+Pv7s2nT\nJluH9u3blx9//BGAvXv3kpOTQ0BAAAUFBdTW1nLt2jVmzpwJ1HcUwIABA2xPN1ZWVnLx4kV69uzZ\naJklJSUkJSXZzgih/owuOzuburo6qqurWbJkyRNrszxdGhvj0PrGeVlZGd26dcPNzY2DBw9itVr/\ntgovKyujR48eABw4cMB2z/F+QUFBHDx4EIDr16+zdu3aZtdDnh6tbe688847fPbZZ7bAXFxczMcf\nf8ytW7dwd3entLQUgPz8fKD+Hv6JEycwxnD58mUuX74MwI0bN+jRowc1NTUcOnSoyXl0j8Ov6AHG\njh3L/PnzWb16NQALFy4kLi6OxMRE2rVrx5o1a+jcuTNhYWFER0djjOGTTz4BoF+/fkyaNImdO3cy\nYMAAoqKiqK2tZc6cOU1eNkxMTKS8vJz3338fgB49erBs2TIGDx5MeHg4xhimTJlin8bLU+H+MQ6t\nb5wHBweTmJhIdHQ0I0eO5PXXX2fx4sUNjgkLCyMmJoZ9+/YRFRXFnj17+P777/+W1+jRozly5AgR\nERFYrVZmzZrV7HrI06U1zZ2QkBAqKysJDw+nU6dOtGvXjvXr19OlSxfCwsKYO3cumZmZ9OvXD6g/\nYXnuuecIDw/H19eXvn37AhAdHc3MmTPx8fHBYrEQHx9PSEiI7e/30370IiIiTszhL92LiIjIo1Og\nFxERcWIK9CIiIk5MgV5ERMSJKdCLiIg4MQV6ESdz6dIl/P392bFjR4P0vLw8/P39H7hb1qFDh7hx\n4wZQv6PcvTeDPaza2lr8/f0f6bsi0rIU6EWckK+vL7t27WqQtmvXLnr16vXA723ZsuWJ76QlIvbV\nKl6YIyIPx8vLi+rqas6ePYufnx9VVVXk5+cTFBQEQHp6OikpKRhj8PT0ZOnSpWRkZJCXl8fcuXNJ\nSEgAYM+ePeTn53P58mUWLVpEcHAw58+fZ9GiRRhjbC8WefnllykuLmbevHm0b9+ewYMH/5vNF5G/\n0IpexEmFhYXZ3jqXmZnJsGHDcHV15cqVK2zcuJEtW7awY8cOBg0axKZNm5gyZQpdu3Zl9erV9OnT\nBwBPT0+SkpL44IMPbNtyLl26lMjISJKTk1m8eDExMTEAfPXVV0ycOJGUlBRdthdxIAr0Ik5q9OjR\nZGRkUFtby+7du3nrrbeA+s03SktLmTZtGhaLhfT0dNs7tu83aNAgALp168atW7cAOH78OEOHDgXA\n39+fiooK/vzzT86cOcNLL70EwKuvvvqkmycizaRL9yJOytPTk/79+7Nz505KS0sZOHAgUB/oAwMD\n2bRp0z/m0abN//9F3HtbdmNbbLq4uDTYactqtbZEE0SkBWhFL+LEwsLCWLduHaGhoba0qqoqCgsL\nbav4jIwMDhw4ANQH7HtbZDYlKCiIw4cPA3Dy5Ek6d+6Mh4cHvXv35pdffgEgJyfnSTRHRB6BAr2I\nExsxYgTGGNtle6h/UG/hwoVMnz6dqKgodu7cyfPPPw/Aa6+9xowZMygoKGgyz7i4OL777jssFgtL\nliyx7Sk/c+ZMtm/fzrRp0yguLm5wNUBE/j3avU5ERMSJaUUvIiLixBToRUREnJgCvYiIiBNToBcR\nEXFiCvQiIiJOTIFeRETEiSnQi4iIOLH/AU4NpTixzvPUAAAAAElFTkSuQmCC\n", 559 | "text/plain": [ 560 | "" 561 | ] 562 | }, 563 | "metadata": { 564 | "tags": [] 565 | } 566 | } 567 | ] 568 | }, 569 | { 570 | "metadata": { 571 | "id": "P1VRyAOtzrYD", 572 | "colab_type": "text" 573 | }, 574 | "cell_type": "markdown", 575 | "source": [ 576 | "# 6. Running your functions on GPU\n" 577 | ] 578 | }, 579 | { 580 | "metadata": { 581 | "id": "J3_Zv57yxsBV", 582 | "colab_type": "code", 583 | "colab": {} 584 | }, 585 | "cell_type": "code", 586 | "source": [ 587 | "@cuda.jit\n", 588 | "def cudaKernal_func(a, b, result): # cuda.jit does not return result yet\n", 589 | " pos = cuda.grid(1)\n", 590 | " if (pos < a.shape[1]) and (pos < b.shape[0]):\n", 591 | " for i in range(100000):\n", 592 | " result[pos] = math.exp(a[0][pos]*b[pos][0])" 593 | ], 594 | "execution_count": 0, 595 | "outputs": [] 596 | }, 597 | { 598 | "metadata": { 599 | "id": "S4pShvAI1YjR", 600 | "colab_type": "code", 601 | "colab": {} 602 | }, 603 | "cell_type": "code", 604 | "source": [ 605 | "result = np.zeros((100,), dtype=np.float64)" 606 | ], 607 | "execution_count": 0, 608 | "outputs": [] 609 | }, 610 | { 611 | "metadata": { 612 | "id": "RxlLDdbT1gFf", 613 | "colab_type": "code", 614 | "colab": { 615 | "base_uri": "https://localhost:8080/", 616 | "height": 34 617 | }, 618 | "outputId": "d5419ec2-d734-4635-9e90-e6f50b47ec05" 619 | }, 620 | "cell_type": "code", 621 | "source": [ 622 | "threadsperblock = 32\n", 623 | "blockspergrid = (100 + 31) // 32 # blockspergrid = (array.size + (threadsperblock - 1)) // threadsperblock\n", 624 | "\n", 625 | "%timeit cudaKernal_func[threadsperblock, blockspergrid](a, b, result)" 626 | ], 627 | "execution_count": 30, 628 | "outputs": [ 629 | { 630 | "output_type": "stream", 631 | "text": [ 632 | "1 loop, best of 3: 66.2 ms per loop\n" 633 | ], 634 | "name": "stdout" 635 | } 636 | ] 637 | }, 638 | { 639 | "metadata": { 640 | "id": "I0Q2FOAk1rXw", 641 | "colab_type": "code", 642 | "colab": { 643 | "base_uri": "https://localhost:8080/", 644 | "height": 34 645 | }, 646 | "outputId": "a12d6f7d-acd0-4277-d0c5-f3760eadd5f2" 647 | }, 648 | "cell_type": "code", 649 | "source": [ 650 | "result.shape" 651 | ], 652 | "execution_count": 31, 653 | "outputs": [ 654 | { 655 | "output_type": "execute_result", 656 | "data": { 657 | "text/plain": [ 658 | "(100,)" 659 | ] 660 | }, 661 | "metadata": { 662 | "tags": [] 663 | }, 664 | "execution_count": 31 665 | } 666 | ] 667 | }, 668 | { 669 | "metadata": { 670 | "id": "1DjchmIF13_j", 671 | "colab_type": "code", 672 | "colab": {} 673 | }, 674 | "cell_type": "code", 675 | "source": [ 676 | "#\n", 677 | "# Here, we have only used it for 1D arrays. You can use it for any Tensor. For eg:\n", 678 | "# For 2D array operations you would have used: x, y = cuda.grid(2)\n", 679 | "#" 680 | ], 681 | "execution_count": 0, 682 | "outputs": [] 683 | }, 684 | { 685 | "metadata": { 686 | "id": "e1snDXNG3c9A", 687 | "colab_type": "code", 688 | "colab": {} 689 | }, 690 | "cell_type": "code", 691 | "source": [ 692 | "@cuda.jit(device=True)\n", 693 | "def cudaDevice_func(a, b):\n", 694 | " for i in range(100000):\n", 695 | " a = math.exp(a*b)\n", 696 | " return a" 697 | ], 698 | "execution_count": 0, 699 | "outputs": [] 700 | }, 701 | { 702 | "metadata": { 703 | "id": "dgv5aV6n3wgv", 704 | "colab_type": "code", 705 | "colab": {} 706 | }, 707 | "cell_type": "code", 708 | "source": [ 709 | "@cuda.jit\n", 710 | "def cudaKernal_func2(a, b, result): # cuda.jit does not return result yet\n", 711 | " pos = cuda.grid(1)\n", 712 | " if (pos < a.shape[1]) and (pos < b.shape[0]):\n", 713 | " result[pos] = cudaDevice_func(a[0][pos], b[pos][0])" 714 | ], 715 | "execution_count": 0, 716 | "outputs": [] 717 | }, 718 | { 719 | "metadata": { 720 | "id": "Ro7kzwPS32vv", 721 | "colab_type": "code", 722 | "colab": { 723 | "base_uri": "https://localhost:8080/", 724 | "height": 50 725 | }, 726 | "outputId": "2c5866c2-7e8c-4cc5-ebcf-fd4492a7464f" 727 | }, 728 | "cell_type": "code", 729 | "source": [ 730 | "%timeit cudaKernal_func2[threadsperblock, blockspergrid](a, b, result)" 731 | ], 732 | "execution_count": 35, 733 | "outputs": [ 734 | { 735 | "output_type": "stream", 736 | "text": [ 737 | "The slowest run took 5.08 times longer than the fastest. This could mean that an intermediate result is being cached.\n", 738 | "1 loop, best of 3: 41.4 ms per loop\n" 739 | ], 740 | "name": "stdout" 741 | } 742 | ] 743 | } 744 | ] 745 | } -------------------------------------------------------------------------------- /SpeedUpYourAlgorithms/README.md: -------------------------------------------------------------------------------- 1 | My medium posts which goes along with above Jupyter notebooks: 2 | 3 | 1. [Speed Up your Algorithms Part 1 — PyTorch](https://towardsdatascience.com/speed-up-your-algorithms-part-1-pytorch-56d8a4ae7051) 4 | 5 | 2. [Speed Up your Algorithms Part 2— Numba](https://towardsdatascience.com/speed-up-your-algorithms-part-2-numba-293e554c5cc1) 6 | 7 | 3. [Speed Up your Algorithms Part 3 — Parallelization](https://towardsdatascience.com/speed-up-your-algorithms-part-3-parallelization-4d95c0888748) 8 | 9 | 4. [Speed Up your Algorithms Part 4 — Dask](https://towardsdatascience.com/speeding-up-your-algorithms-part-4-dask-7c6ed79994ef) 10 | -------------------------------------------------------------------------------- /Tackle/README.md: -------------------------------------------------------------------------------- 1 | My medium posts with which these notebooks go along: 2 | 3 | 1. [How to learn from BigData Files on low memory - Incremental Learning](https://towardsdatascience.com/how-to-learn-from-bigdata-files-on-low-memory-incremental-learning-d377282d38ff) 4 | 2. [Make your Data Talk!- From 0 to Hero in visualization with matplotlib and seaborn](https://medium.com/@grover.puneet1995/make-your-data-talk-13072f84eeac) 5 | --------------------------------------------------------------------------------