├── README.md ├── Python Lab.ipynb └── Folder └── Logistic_Regression.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # Data-Science-Fundamentals-Course 2 | Practical exercises from Data Science Fundamentals by DataRoot Labs 3 | 4 | Coursera ML 5 | https://dataplatform.cloud.ibm.com/analytics/notebooks/v2/93948b4b-f7f5-478e-90ab-278d6fdd2756/view?access_token=067f9493dcb3f02baa726bf6aba5f3f1db94d597aa9f3ea5a67915057406faf8 6 | 7 | Kaggle Housing Prices Competition for Kaggle Learn Users 8 | https://www.kaggle.com/dmytro21/exercise-xgboost 9 | -------------------------------------------------------------------------------- /Python Lab.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "colab_type": "text", 7 | "id": "KOP7kgV5PSl2" 8 | }, 9 | "source": [ 10 | "# Python and algorithms" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": { 16 | "colab_type": "text", 17 | "id": "0R6uyhNyjPMc" 18 | }, 19 | "source": [ 20 | "##### Сomplete this part of test without using any libs (your code must not contain keyword 'import')" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": { 26 | "colab_type": "text", 27 | "id": "GC_iufBgjPMd" 28 | }, 29 | "source": [ 30 | "#### 1. Implement function str_to_dict(some_str) which returnes dictionary, where keys are string characters, and values are their quantity in the string:\n", 31 | "\n", 32 | "
1 point
" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": 60, 38 | "metadata": { 39 | "colab": { 40 | "autoexec": { 41 | "startup": false, 42 | "wait_interval": 0 43 | } 44 | }, 45 | "colab_type": "code", 46 | "id": "DDou3xupjPMe" 47 | }, 48 | "outputs": [], 49 | "source": [ 50 | "def str_to_dict(some_str): \n", 51 | " \"\"\"\n", 52 | " :param some_str: str\n", 53 | " :return: dict\n", 54 | " \"\"\"\n", 55 | " # YOUR CODE HERE\n", 56 | " dict_1 = dict()\n", 57 | " for elem in some_str:\n", 58 | " if elem not in dict_1:\n", 59 | " dict_1[elem] = 1\n", 60 | " else:\n", 61 | " dict_1[elem] += 1\n", 62 | " return dict_1" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": 61, 68 | "metadata": { 69 | "colab": { 70 | "autoexec": { 71 | "startup": false, 72 | "wait_interval": 0 73 | } 74 | }, 75 | "colab_type": "code", 76 | "id": "FPjc0Ckxsdbl" 77 | }, 78 | "outputs": [ 79 | { 80 | "name": "stdout", 81 | "output_type": "stream", 82 | "text": [ 83 | "Str to dict: {'d': 1, 'a': 2, 't': 3, 'r': 2, 'o': 2, '_': 1, 'u': 1, 'n': 1, 'i': 2, 'v': 1, 'e': 1, 's': 1, 'y': 1}\n" 84 | ] 85 | } 86 | ], 87 | "source": [ 88 | "print('Str to dict:', str_to_dict('dataroot_university'))" 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "metadata": { 94 | "colab_type": "text", 95 | "id": "nurux2Btsdbn" 96 | }, 97 | "source": [ 98 | "Expected Output: \n", 99 | "\n", 100 | " \n", 101 | " \n", 102 | " \n", 103 | " " 104 | ] 105 | }, 106 | { 107 | "cell_type": "markdown", 108 | "metadata": { 109 | "colab_type": "text", 110 | "id": "ELtxJQQLjPMi" 111 | }, 112 | "source": [ 113 | "#### 2. Implement function sec_smallest(numbers) which returns second smallest item in the list, without using the built-in sorting methods (your code mustn't contain such words as 'sort', 'sorted'):\n", 114 | "\n", 115 | "
1 point\n", 116 | " " 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": 1, 122 | "metadata": { 123 | "colab": { 124 | "autoexec": { 125 | "startup": false, 126 | "wait_interval": 0 127 | } 128 | }, 129 | "colab_type": "code", 130 | "id": "yJVJiK9jjPMj" 131 | }, 132 | "outputs": [], 133 | "source": [ 134 | "def sec_smallest(numbers):\n", 135 | " \"\"\"\n", 136 | " :param numbers: list[int]\n", 137 | " :return: int \n", 138 | " \"\"\"\n", 139 | " # YOUR CODE HERe\n", 140 | " smallest = min(numbers)\n", 141 | " sec_smallest = max(numbers)\n", 142 | " for elem in numbers:\n", 143 | " if sec_smallest > elem and elem > smallest:\n", 144 | " sec_smallest = elem\n", 145 | " \n", 146 | " return sec_smallest" 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": 2, 152 | "metadata": { 153 | "colab": { 154 | "autoexec": { 155 | "startup": false, 156 | "wait_interval": 0 157 | } 158 | }, 159 | "colab_type": "code", 160 | "id": "pg4ry_8Csdbq" 161 | }, 162 | "outputs": [ 163 | { 164 | "name": "stdout", 165 | "output_type": "stream", 166 | "text": [ 167 | "Sec_smallest: -2\n" 168 | ] 169 | } 170 | ], 171 | "source": [ 172 | "print('Sec_smallest:', sec_smallest([1, 2, -8, -8, -2, 0]))" 173 | ] 174 | }, 175 | { 176 | "cell_type": "markdown", 177 | "metadata": { 178 | "colab_type": "text", 179 | "id": "qVlYH9axsdbs" 180 | }, 181 | "source": [ 182 | "Expected Output: \n", 183 | "
Str to dict: {'d': 1, 'a': 2, 't': 3, 'r': 2, 'o': 2, '_': 1, 'u': 1, 'n': 1, 'i': 2, 'v': 1, 'e': 1, 's': 1, 'y': 1}
\n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " " 188 | ] 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "metadata": { 193 | "colab_type": "text", 194 | "id": "tzgLZ6VijPMo" 195 | }, 196 | "source": [ 197 | "#### 3. Implement function prime_nums(n) that returns list of numbers which are simple and < n:\n", 198 | "\n", 199 | "
1 point
" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": 3, 205 | "metadata": { 206 | "colab": { 207 | "autoexec": { 208 | "startup": false, 209 | "wait_interval": 0 210 | } 211 | }, 212 | "colab_type": "code", 213 | "id": "jvVD9N32jPMp" 214 | }, 215 | "outputs": [], 216 | "source": [ 217 | "def prime_nums(n):\n", 218 | " \"\"\"\n", 219 | " :param n: int\n", 220 | " :return: list[int]\n", 221 | " \"\"\"\n", 222 | " # YOUR CODE HERE\n", 223 | " prime_numbers = list()\n", 224 | "# prime_numbers = [2, 3]\n", 225 | " \n", 226 | " for i in range(2, n):\n", 227 | " for k in range(2, i):\n", 228 | " if i % k == 0:\n", 229 | " break\n", 230 | " else:\n", 231 | " prime_numbers.append(i)\n", 232 | " \n", 233 | " return prime_numbers" 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": 4, 239 | "metadata": {}, 240 | "outputs": [ 241 | { 242 | "name": "stdout", 243 | "output_type": "stream", 244 | "text": [ 245 | "Prime numbers: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29]\n" 246 | ] 247 | } 248 | ], 249 | "source": [ 250 | "print('Prime numbers:', prime_nums(30))" 251 | ] 252 | }, 253 | { 254 | "cell_type": "markdown", 255 | "metadata": { 256 | "colab": { 257 | "autoexec": { 258 | "startup": false, 259 | "wait_interval": 0 260 | } 261 | }, 262 | "colab_type": "code", 263 | "id": "Zf9DMkN9sdbw" 264 | }, 265 | "source": [ 266 | "print('Prime numbers:', prime_nums(30))" 267 | ] 268 | }, 269 | { 270 | "cell_type": "markdown", 271 | "metadata": { 272 | "colab_type": "text", 273 | "id": "pYutFqaAsdbz" 274 | }, 275 | "source": [ 276 | "Expected Output:\n", 277 | "
Sec_smallest: -2
\n", 278 | " \n", 279 | " \n", 280 | " \n", 281 | " " 282 | ] 283 | }, 284 | { 285 | "cell_type": "markdown", 286 | "metadata": {}, 287 | "source": [ 288 | "**4. Implement function max_sum_index(tuples), which returnes index of tuple in the list with maximum sum of elements:**\n", 289 | "\n", 290 | "
1 point
" 291 | ] 292 | }, 293 | { 294 | "cell_type": "code", 295 | "execution_count": 5, 296 | "metadata": {}, 297 | "outputs": [], 298 | "source": [ 299 | "def max_sum_index(tuples): \n", 300 | " '''\n", 301 | " :param tuples: list[tuple]\n", 302 | " :return: int\n", 303 | " '''\n", 304 | " # YOUR CODE HERE\n", 305 | " max_sum_index = 0\n", 306 | " max_sum = 0\n", 307 | " for i in range(0, len(tuples)):\n", 308 | " small_list =tuples[i]\n", 309 | " if max_sum < sum(small_list):\n", 310 | " max_sum = sum(small_list) \n", 311 | " max_sum_index = i\n", 312 | "\n", 313 | "\n", 314 | " return max_sum_index" 315 | ] 316 | }, 317 | { 318 | "cell_type": "code", 319 | "execution_count": 6, 320 | "metadata": {}, 321 | "outputs": [ 322 | { 323 | "name": "stdout", 324 | "output_type": "stream", 325 | "text": [ 326 | "Index: 1\n" 327 | ] 328 | } 329 | ], 330 | "source": [ 331 | "print(\"Index:\", max_sum_index([(10, 20), (40, 32), (30, 25)]))\n" 332 | ] 333 | }, 334 | { 335 | "cell_type": "markdown", 336 | "metadata": {}, 337 | "source": [ 338 | "Expected Output: \n", 339 | "
Prime numbers:[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
\n", 340 | " \n", 341 | " \n", 342 | " \n", 343 | " " 344 | ] 345 | }, 346 | { 347 | "cell_type": "markdown", 348 | "metadata": {}, 349 | "source": [ 350 | "**5. Implement function gcd(x, y), which returns the greatest common divisor of n and m.**\n", 351 | "
1 point
" 352 | ] 353 | }, 354 | { 355 | "cell_type": "code", 356 | "execution_count": 7, 357 | "metadata": {}, 358 | "outputs": [], 359 | "source": [ 360 | "def gcd(a, b): \n", 361 | " '''\n", 362 | " :params m,n: int\n", 363 | " :return: int\n", 364 | " '''\n", 365 | " # YOUR CODE HERE\n", 366 | " while a != 0 and b != 0:\n", 367 | " if a > b:\n", 368 | " a = a % b\n", 369 | " else:\n", 370 | " b = b % a\n", 371 | " return a + b\n" 372 | ] 373 | }, 374 | { 375 | "cell_type": "code", 376 | "execution_count": 8, 377 | "metadata": {}, 378 | "outputs": [ 379 | { 380 | "name": "stdout", 381 | "output_type": "stream", 382 | "text": [ 383 | "GCD: 20\n" 384 | ] 385 | } 386 | ], 387 | "source": [ 388 | "print(\"GCD:\", gcd(160, 180))" 389 | ] 390 | }, 391 | { 392 | "cell_type": "markdown", 393 | "metadata": {}, 394 | "source": [ 395 | "Expected Output: \n", 396 | "
Index: 1
\n", 397 | " \n", 398 | " \n", 399 | " \n", 400 | " " 401 | ] 402 | }, 403 | { 404 | "cell_type": "markdown", 405 | "metadata": {}, 406 | "source": [ 407 | "#### 6. Implement recursive sum of the list:\n", 408 | "
1 point
" 409 | ] 410 | }, 411 | { 412 | "cell_type": "code", 413 | "execution_count": 9, 414 | "metadata": {}, 415 | "outputs": [], 416 | "source": [ 417 | "def recursive_list_sum(data_list):\n", 418 | " \"\"\"\n", 419 | " :param data_list: list[list]\n", 420 | " \"\"\"\n", 421 | " # YOUR CODE HERE\n", 422 | " some_list = []\n", 423 | " list_sum = 0\n", 424 | " for i in reversed(range(len(data_list))):\n", 425 | " print(data_list)\n", 426 | " if type(data_list[i]) == type(some_list):\n", 427 | " data_list[i] = sum(data_list[i])\n", 428 | " \n", 429 | " return sum(data_list)" 430 | ] 431 | }, 432 | { 433 | "cell_type": "code", 434 | "execution_count": 10, 435 | "metadata": {}, 436 | "outputs": [ 437 | { 438 | "name": "stdout", 439 | "output_type": "stream", 440 | "text": [ 441 | "[1, 2, [3, 4], [5, 6], [7, 8, 9]]\n", 442 | "[1, 2, [3, 4], [5, 6], 24]\n", 443 | "[1, 2, [3, 4], 11, 24]\n", 444 | "[1, 2, 7, 11, 24]\n", 445 | "[1, 2, 7, 11, 24]\n", 446 | "The sum of a list is 45\n" 447 | ] 448 | } 449 | ], 450 | "source": [ 451 | "print('The sum of a list is', recursive_list_sum([1, 2, [3,4],[5,6], [7, 8, 9]]))" 452 | ] 453 | }, 454 | { 455 | "cell_type": "markdown", 456 | "metadata": {}, 457 | "source": [ 458 | "Expected Output:\n", 459 | "
GCD: 20
\n", 460 | " \n", 461 | " \n", 462 | " " 463 | ] 464 | }, 465 | { 466 | "cell_type": "markdown", 467 | "metadata": {}, 468 | "source": [ 469 | "#### 7. Implement decorator which returns function signature and it's return value:\n", 470 | "
1 point
" 471 | ] 472 | }, 473 | { 474 | "cell_type": "code", 475 | "execution_count": 173, 476 | "metadata": {}, 477 | "outputs": [], 478 | "source": [ 479 | "def debug(func):\n", 480 | " \"\"\"\n", 481 | " :param func: function\n", 482 | " \"\"\"\n", 483 | " # YOUR CODE HERE\n", 484 | " def wrapper(a, b):\n", 485 | " c = func.__name__ + \"(%s\" %a + \", %s)\" %b + \" was called and returned \" + \"%s\" %func(a, b)\n", 486 | " return c\n", 487 | " return wrapper\n", 488 | " " 489 | ] 490 | }, 491 | { 492 | "cell_type": "code", 493 | "execution_count": 174, 494 | "metadata": {}, 495 | "outputs": [ 496 | { 497 | "data": { 498 | "text/plain": [ 499 | "'add(3, 4) was called and returned 7'" 500 | ] 501 | }, 502 | "execution_count": 174, 503 | "metadata": {}, 504 | "output_type": "execute_result" 505 | } 506 | ], 507 | "source": [ 508 | "@debug\n", 509 | "def add(a, b):\n", 510 | " return a + b\n", 511 | " \n", 512 | "add(3, 4)\n" 513 | ] 514 | }, 515 | { 516 | "cell_type": "markdown", 517 | "metadata": {}, 518 | "source": [ 519 | "Expected Output:\n", 520 | "
The sum of a list is 45
\n", 521 | " \n", 522 | " \n", 523 | " " 524 | ] 525 | }, 526 | { 527 | "cell_type": "markdown", 528 | "metadata": { 529 | "colab_type": "text", 530 | "id": "p96mEq8vjPMr" 531 | }, 532 | "source": [ 533 | "#### 8. Implement class Conv, that contains method to_roman(self, n), which converts decimal numbers to Roman numerals:\n", 534 | "\n", 535 | "
2 points
" 536 | ] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "execution_count": 46, 541 | "metadata": { 542 | "colab": { 543 | "autoexec": { 544 | "startup": false, 545 | "wait_interval": 0 546 | } 547 | }, 548 | "colab_type": "code", 549 | "id": "weYiaNWojPMs" 550 | }, 551 | "outputs": [], 552 | "source": [ 553 | "class Conv:\n", 554 | " def __init__(self):\n", 555 | " self.val = [\n", 556 | " 1000, 900, 500, 400,\n", 557 | " 100, 90, 50, 40,\n", 558 | " 10, 9, 5, 4, 1\n", 559 | " ]\n", 560 | "\n", 561 | " self.syb = [\n", 562 | " 'M', 'CM', 'D', 'CD',\n", 563 | " 'C', 'XC', 'L', 'XL',\n", 564 | " 'X', 'IX', 'V', 'IV',\n", 565 | " 'I'\n", 566 | " ]\n", 567 | " \n", 568 | " def to_roman(self, num): \n", 569 | " \"\"\"\n", 570 | " :param self:\n", 571 | " :param n: int\n", 572 | " :return: str \n", 573 | " \"\"\"\n", 574 | " # YOUR CODE HERE\n", 575 | " \n", 576 | " roman_num = \"\"\n", 577 | " i = 0\n", 578 | " while num > 0:\n", 579 | " for _ in range(num // self.val[i]):\n", 580 | " roman_num += self.syb[i]\n", 581 | " num -= self.val[i]\n", 582 | " i += 1\n", 583 | " return roman_num\n", 584 | " \n", 585 | " \n", 586 | "# roman = \"\"\n", 587 | "# for i in range(len(self.val)):\n", 588 | "# if num//self.val[i] == 0:\n", 589 | "# continue\n", 590 | "# else: \n", 591 | "# roman.append(self.syb[])\n", 592 | " " 593 | ] 594 | }, 595 | { 596 | "cell_type": "code", 597 | "execution_count": 47, 598 | "metadata": { 599 | "colab": { 600 | "autoexec": { 601 | "startup": false, 602 | "wait_interval": 0 603 | } 604 | }, 605 | "colab_type": "code", 606 | "id": "NdffBhZesdb6" 607 | }, 608 | "outputs": [ 609 | { 610 | "name": "stdout", 611 | "output_type": "stream", 612 | "text": [ 613 | "Converted: XLIV\n" 614 | ] 615 | } 616 | ], 617 | "source": [ 618 | "print('Converted:', Conv().to_roman(44))" 619 | ] 620 | }, 621 | { 622 | "cell_type": "markdown", 623 | "metadata": { 624 | "colab_type": "text", 625 | "id": "lF6Acd9gsdb8" 626 | }, 627 | "source": [ 628 | "Expected Output:\n", 629 | "
'add(3, 4) was called and returned 7'
\n", 630 | " \n", 631 | " \n", 632 | " \n", 633 | " " 634 | ] 635 | } 636 | ], 637 | "metadata": { 638 | "colab": { 639 | "collapsed_sections": [], 640 | "default_view": {}, 641 | "name": "DataRoot University _ Data Science Module 1 _ Test 1.ipynb", 642 | "provenance": [], 643 | "version": "0.3.2", 644 | "views": {} 645 | }, 646 | "kernelspec": { 647 | "display_name": "Python 3", 648 | "language": "python", 649 | "name": "python3" 650 | }, 651 | "language_info": { 652 | "codemirror_mode": { 653 | "name": "ipython", 654 | "version": 3 655 | }, 656 | "file_extension": ".py", 657 | "mimetype": "text/x-python", 658 | "name": "python", 659 | "nbconvert_exporter": "python", 660 | "pygments_lexer": "ipython3", 661 | "version": "3.7.6" 662 | } 663 | }, 664 | "nbformat": 4, 665 | "nbformat_minor": 1 666 | } 667 | -------------------------------------------------------------------------------- /Folder/Logistic_Regression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "colab_type": "text", 7 | "id": "LTjV_QhNIgxE" 8 | }, 9 | "source": [ 10 | "# Logistic Regression\n", 11 | "\n", 12 | "Welcome to your next lab! You will build a logistic regression classifier. \n", 13 | "\n", 14 | "You will classify pictures with and without cats.\n", 15 | "\n", 16 | "**Instructions:**\n", 17 | "- Do not use loops (for/while) in your code, unless the instructions explicitly ask you to do so.\n", 18 | "\n", 19 | "**You will learn to:**\n", 20 | "- Build the general architecture of a learning algorithm, including:\n", 21 | " - Initializing parameters\n", 22 | " - Calculating the cost function and its gradient\n", 23 | " - Using an optimization algorithm (gradient descent) \n", 24 | "- Gather all three functions above into a main model function, in the right order." 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": { 30 | "colab_type": "text", 31 | "id": "q5s5aPjlIgxG" 32 | }, 33 | "source": [ 34 | "## 1 - Packages ##\n", 35 | "\n", 36 | "First, let's run the cell below to import all the packages that you will need during this assignment. \n", 37 | "- [numpy](www.numpy.org) is the fundamental package for scientific computing with Python.\n", 38 | "- [h5py](http://www.h5py.org) is a common package to interact with a dataset that is stored in HDF5 format.\n", 39 | "- [matplotlib](http://matplotlib.org) is a famous library to plot graphs in Python." 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 60, 45 | "metadata": { 46 | "colab": { 47 | "autoexec": { 48 | "startup": false, 49 | "wait_interval": 0 50 | } 51 | }, 52 | "colab_type": "code", 53 | "id": "ip12qta2IgxG" 54 | }, 55 | "outputs": [], 56 | "source": [ 57 | "import numpy as np\n", 58 | "import h5py\n", 59 | "import matplotlib.pyplot as plt\n", 60 | "\n", 61 | "%matplotlib inline" 62 | ] 63 | }, 64 | { 65 | "cell_type": "markdown", 66 | "metadata": { 67 | "colab_type": "text", 68 | "id": "3ADfa7UWIgxJ" 69 | }, 70 | "source": [ 71 | "## 2 - Overview of the Problem set ##\n", 72 | "\n", 73 | "**Problem Statement**: You are given a dataset containing:\n", 74 | " - a training set of m_train examples labeled as (y=1) or (y=0)\n", 75 | " - a test set of m_test examples as (y=1) or (y=0)\n", 76 | " - each example is of shape (number of features = 64 * 64 * 3, 1)\n", 77 | "\n", 78 | "You will build a simple algorithm that can correctly classify training examples depending on your particular dataset.\n", 79 | "\n", 80 | "Let's get more familiar with the dataset. Load the data by running the following code." 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": 61, 86 | "metadata": { 87 | "colab": { 88 | "autoexec": { 89 | "startup": false, 90 | "wait_interval": 0 91 | } 92 | }, 93 | "colab_type": "code", 94 | "id": "SZyUOo3hIgxK" 95 | }, 96 | "outputs": [], 97 | "source": [ 98 | "# Loading the data\n", 99 | "\n", 100 | "def load_data():\n", 101 | " train_dataset = h5py.File(\"train_cats.h5\", \"r\")\n", 102 | " train_set_x = np.array(train_dataset[\"train_set_x\"][:]) # your train set features\n", 103 | " train_set_y = np.array(train_dataset[\"train_set_y\"][:]) # your train set labels\n", 104 | "\n", 105 | " test_dataset = h5py.File(\"test_cats.h5\", \"r\")\n", 106 | " test_set_x = np.array(test_dataset[\"test_set_x\"][:]) # your test set features\n", 107 | " test_set_y = np.array(test_dataset[\"test_set_y\"][:]) # your test set labels\n", 108 | "\n", 109 | " classes = np.array(test_dataset[\"list_classes\"][:]) # the list of classes\n", 110 | " \n", 111 | " # Images have (64(width), 64(height), 3(RGB)) dimensions, we need to have all the pixels as features\n", 112 | " test_set_x = test_set_x.reshape(test_set_x.shape[0], -1).T # shape ((209)examples, (64)width, (64)height, (3)RGB) -> (64 * 64 * 3, 209)\n", 113 | " train_set_x = train_set_x.reshape(train_set_x.shape[0], -1).T # shape ((209)examples, (64)width, (64)height, (3)RGB) -> (64 * 64 * 3, 209)\n", 114 | "\n", 115 | " train_set_y = train_set_y.reshape((1, train_set_y.shape[0]))\n", 116 | " test_set_y = test_set_y.reshape((1, test_set_y.shape[0]))\n", 117 | " \n", 118 | " return train_set_x, train_set_y, test_set_x, test_set_y, classes\n", 119 | "\n", 120 | "train_set_x, train_set_y, test_set_x, test_set_y, classes = load_data()" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": 62, 126 | "metadata": { 127 | "colab": { 128 | "autoexec": { 129 | "startup": false, 130 | "wait_interval": 0 131 | } 132 | }, 133 | "colab_type": "code", 134 | "id": "ui3z2vXaIgxM" 135 | }, 136 | "outputs": [ 137 | { 138 | "name": "stdout", 139 | "output_type": "stream", 140 | "text": [ 141 | "(12288, 209) (1, 209) (12288, 50) (1, 50) (2,)\n" 142 | ] 143 | } 144 | ], 145 | "source": [ 146 | "print (train_set_x.shape, train_set_y.shape, test_set_x.shape, test_set_y.shape, classes.shape)" 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "metadata": { 152 | "colab_type": "text", 153 | "id": "eMnm8hHYIgxO" 154 | }, 155 | "source": [ 156 | "### Example of a picture" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": 63, 162 | "metadata": { 163 | "colab": { 164 | "autoexec": { 165 | "startup": false, 166 | "wait_interval": 0 167 | } 168 | }, 169 | "colab_type": "code", 170 | "id": "DEKTIoXAIgxO" 171 | }, 172 | "outputs": [ 173 | { 174 | "name": "stdout", 175 | "output_type": "stream", 176 | "text": [ 177 | "y = [1], it's a 'cat' picture.\n" 178 | ] 179 | }, 180 | { 181 | "data": { 182 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD7CAYAAACscuKmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO29aaxl13Ue+O1zzp3vm1/NZLE4i6QskTKtwbTVNEUKatktAW2pE8doKAERwoC74aDTiKQOEiRBAth/YgdB4DTRdkc/3JHsJI4EwXHMZkuw3S1LouaBojgVyZrrDffdeTjn7P5xb931rVX1qp5UVffRuvsDCrXv2+ees8+w71nfXmt9y3nvERAQ8JOPaL8HEBAQMBuEyR4QMCcIkz0gYE4QJntAwJwgTPaAgDlBmOwBAXOC65rszrkPOOdecM695Jz7xI0aVEBAwI2H+3H97M65GMAPATwB4BSArwL4Fe/992/c8AICAm4Ukuv47jsBvOS9fwUAnHOfBvBhALtO9kIc+3IyPmQca6OimMhQ7A+Q+kR9cRyr7VxE+4z0/pOkIOMoJtQuq+3K1Sr1FfX+nUNAAGOU5dN2Su08189wmqbTdqGon81SUZ5N/lZ+2TzIpm3nMtU3HI0/nzt1Do3NxhUf1OuZ7McAvEGfTwF419W+UE4S/PSxYwCAxXpV9d1yaH3aHo5Gqs9Dxp5T31K9rrZL6pVpOypVVN/K4SPT9tHjh6ftw7feo7a7/6F3TNtHbr1V779QQMCPDk8PvruMOP71/gE9t9Obts9vd6ftbneottvcuDhtHz1RU3133irPZurlenRyPQ9GaEzbcdRSfW9c3AIA/N33/91dx3o9k/1Kd+kyTuCcewrAUwBQMm/igICA2eF6JvspAPzquwXAGbuR9/5pAE8DwFKl7Mvl8dsxMUd29DtRKWvzWaFcmjbjon7T8ktjNNK/rDtb29N2UpR9nLhHWwClsnxut3ZU39LKmow3mPRXBb/N04ze7Lm+bvY5eLPDvs2KiZjTB5bl79GqtlyPHz02bZfN8514uQhqHthnLJPnPfd6H4uV8fHiaPc19+tZjf8qgLudc7c754oA/iaAz13H/gICAm4ifuzfVe996pz7nwD8VwAxgN/33n/vho0sICDghuK6jCjv/Z8A+JMbNJaAgICbiNkyJg9g4p6IvOYjzWZ72l5c1KvsUSw8pkg8PSroBb9qRbj4wOm+Qlm+1+t2pu3XX31Nbbd+9Oi0ffS241c+jzmFDcnwPt+1L0vZJUV/1x4jlMvCJONEPxPRm3BdxJ7nc98Xh9QLJ+VZSszz12ltTtvvfe9Pqb633H4nfYp2aQN9ung+0mtS0aXjXeWShXDZgIA5QZjsAQFzgpma8XEcYWlhbKKvLGlTfdgfTNv9fl/1lUpigvdHEsRQW9Sut/5AopSSsj61SoWi5ioSNXfxwlm13df+6kvT9tLqiupbXl7FvIHN1izVNmw2ooixLFV9w5FsO6LvDek7ADCgW12q6vtZrYh7KY7eHCa9ZRZvv0e8z7cckeel0x2o7Ta3xIxfXVhWfRG9c/kKWxqTxHJ9OoZOpPFo8v3dw9/Dmz0gYE4QJntAwJwgTPaAgDnBTDm7AxBjzNnKxs1SWxMOPxzqBIBCTMOkTIrUhMQOB/K9otfcEC1xhVSdcJ+lZc2fanXh84X4Rshs2328ObjnXsG00fJVPxRXUN7rqb7+UK5/n3xvg5H2vWWU7FFd0CGmo0UJXV5d0skj+wWbkfnqKXG9Xdi+MG0XTcYkIlrTiPQ18KotnyLzrCzEcj1KuV4/GmHcl7jdp3R4swcEzAnCZA8ImBPM1ox3QKU0/n1pd9uqb6W4OG3XSnpYQzIDOYIut64JMvGzWJvxvR659lLJC66uLKntFlZkHJW6NvEZKdEEKzKQ0WfrCnGQ7xWcydqj8+HvGR0EJLTdLLPvjN4IfCbXdNDV7lLO5+6nYqoPUm3CtjuSA769rbMMS6R5ULxbssaSREencYZdsaCfHSuScr2wEXSnzsiYv/29l6btXk9TzO0tMfE/9NHHVN/y6oFpu0iRd868iyOarva8Bm58L4LrLSAgIEz2gIB5wUzNeO89hpMV9CTWh964KKb10oIWlCiWZGVzNBTzKDaJMBmZtOWKlawSkzBKZH+RMXu2LkqkU9+sMOekY9ei0K/YWNLuKqaUc2RyxiXVl5CSA5vuNpKqROO4bHGfqYBnWrB38PGYMsRmLxldn+1GR/VtN8U8H5EZnxt61SUzvtvTUWc9ugg7DREfWV7WK/ODntyLw0cPqL7jxySx6UaY9FYb4mcfvnvafuD+Q9O2jRRsdYS2rh3Q1DHiBBd6PixDY9IXOa2dOML4WXJX8faEN3tAwJwgTPaAgDlBmOwBAXOCmXL2JImxvjJ2Z/VMlFxSEB5tdC2UBneBMuB2jHvDRXI6aab5X83J/mursiZw5Oghtd3iMvGpXGdyLVVlHWCxKrzxqs6vq3RafuVV3+64iqr+1Tbca5cCrz+kxr3Waoic8dkL26qvS9sWSSfdx9r1NugL129ua3nkdl+ekQtnJDuxXNGPLXviTr9movB++m3T9p233z5tp8aHdrYhazUjI2XO96liRE7/8395dtr+7vMvTNu8zjTehzyrhw6vqb5Vcv+WSVC1WtVrOizOUjHu6UtLYIO+fu4Z4c0eEDAnCJM9IGBOMFMzPnJuqglfKGhDNRuJyVytalOsTwkvBfJzra4sqO2GZJqVYlO9hZJaOu0utTUVeOs7RQ+svqQj6FRSyE1IaNnrHt1VPt2ALoVRV9xrzVNa6GODtPhbbR0RyS6kItGrPNXXOxuKud/v6X30KQpvp0WuTvPUlsjavXhWm+C9jtCEOBJ7f2ldm9J1qiA0NMkk3QFFCprkq9uPS2Rfj8a71Wiq7ba2NqbtVlNTiNFQ9s8RcO2d02q7NJVrkKWaYvb742d6a1NTIUZ4swcEzAnCZA8ImBOEyR4QMCeYKWfP0gzbG2OeZ90KMXHxZkO7cUoV4fDttnCwYsWIAHAYqSF2FRIzLNSEn5WNC+PsmVPSZ7LeTtwuawQ3g7O/GdHuCE88t9FQfWfPy+emWfsAlRRudWktxSyl9LqyftLvdXUfHZuz6BITn5yRUIbPNGc/+dLJafvCpNIpACzcclhtd/c9d0zbd504ofruoArAvBYBAJtNyXq7nyJYbajriFzB9991r+pbrUmIb06Zm422ngc5hdVmRoC/Nwnf/ss//Tx2wzXf7M6533fOXXDOfZf+tuqce8Y59+Lk/5Wr7SMgIGD/sRcz/t8B+ID52ycAPOu9vxvAs5PPAQEBb2Jc04z33v+5c+6E+fOHATw6aX8KwBcBfPyaR3MO0STcqbags9JKVAZos6FNpSFlTYGS+8+f12Yla5j1i9qsbJJJePC42JJFoy+/tibaXv2uzuTac4jbX3MMSSfu9SaZ8WWdaXWBIt4GfX29L4mUAECxJPcsN9KAbLoPTZZhj9x+vP/U2MhZwuW+9f3cviAur1dOS9u/qquLf/1rU8MVd92ly3790i8+MW2fOHqL6nvmmS9P21/6ylen7dxo7PcG8qz+w3/0a6rvtreJ+4515+qldbVdARQqaOhE6sf3rFLS9Jjx4y7QHfLenwWAyf8Hf8z9BAQEzAg3fYHOOfcUgKcAoG7ihQMCAmaHH3eyn3fOHfHen3XOHQFwYbcNvfdPA3gaAI4sL/iVSdSbgynnSeZd3SzZ9iDRQo6EJw5VtchFP5d9OlsuiJJp+hRV9fIPvq82y+h7jzz+ocvO5xKulkhi5YYZrBm3n0yAR9gypYpOnZYV5lc3KfKwrIUhmt0fTNsu1Sb4Yk1M/lpFDEiOXgSAdluoQLenadOlqDBAy4THzorhkXmbWi08+bzZleeov6Uj3KJIznP73EXV1+nItn/7Y7+q+h5/33um7bvuFRO/0dDRgKdPi+R0saopbAdynmXSJXRei7PwAxMZo/xSQOrNEK/4HICPTdofA/DZH3M/AQEBM8JeXG//HsCXANzrnDvlnHsSwG8CeMI59yKAJyafAwIC3sTYy2r8r+zS9b4bPJaAgICbiJlG0AF+mvU0NCV+tzZJ7MC4eMpUYnlEbiGb+cOfopI2WhZWKGquLry/YsQtz7x2cto+d+pV1VdbElcI65/vtA3XJFHMJNHrD7weERsX0uqSaNaXbfmgG4wuCUN8/ZuvqL5TrwnfzBdE8KFj3DqDoYy3BJP1llHG2obw8s0t7S7dpuywtrmOKYlIZHRNTaweALlWaaoj6FpD4ulUCrxvBCoyeh5bDa1f32jJ53JF389H3/feafvW4xKV99rrX1HbcWmo7ki7/Rq5ZOCVI7nGJWhhzbKXZ9U+O5jUMQi68QEBAWGyBwTMC2ZqxudZjnZn7KKxIgCOIuN6Te3GSUckhFAWM6o3NNrc5J4pjLSJPySzdUhJBAvrOkrp9vskSWH9gE6W+No3vjVtnzktQg6bm5tqu2JJaMdCXbtZuDSU62k30c8+8u5p+8QdJ6btONYumB+n4pM1706dlaSQ114+p/o6bRIPaZMZbK5pbyjnmZnIuG0yhVPS2G9ua5dXl0z3dkvf95j8sY4EHjJToXdIlWBTU15qqy3bjsiktxWAu2Tip4Zi7rSEhvyXzz2r+hobcu/f+vb7p+03XtbCE//3M381bT/yC8dU331vke/xfUqhx5GTu7rg9dQdYXTZ9y3Cmz0gYE4QJntAwJwgTPaAgDnBbAUnI4dqdSI4mRvdaxL5q9SM4KTiU6QhX9RcNma3S6p5nS/I9zYuCF+1ogsRucrueeBnVN8rL744bX/9K1+ftrNc88RVWgeoGPHMvCCfFxL9W3vmtHDnCunSL5my0kVy3xUSfR13o/O9geZ/L78g7rbOlnZ5jVLZvy/IuY0MZ2dZ/czrc8mJV3vimrkJk+4TF+8OzL3wsm1K6xuZVc4nrfSGqbFGmhfK3ZsbF51z9D0jfMLbnjOZln/+hS9N289/V8KH73vbPWq7X/u1j07bd5/4KdVXpLpttkyzGgdGtJ2+04XJ/Am13gICAsJkDwiYF8zUjHdRhMqkhE3RqBj0yBQb9nVmFOvI7zTZdaN/q1YWKRrOmMhxUU41roiJnJR0BF0hE/fPqy9+R/W98YZEPp07Kya3M66xPpWlygumDBDRl3f89P2q7zWTbXUJ6wdW1eeFNYm4Wl/TimBlKk/EBt2pN7R77dTLEh046OqIxZSuAZuOvUybiCmJTWQm02/UlfsUeTHBBwNTnoieg9xERA4pCs9TxFtuzPgWUbtNbZ3DU7ZjRCIXuUkoGxCFaHc0neDsO6u+0W3Lc8ZCHEnBZl3K905+X9OyRSovfnhN3L1RpKP1RnQvLJVJJhlyVysXHt7sAQFzgjDZAwLmBDM149M0xcXJSnjJVrlMxOzZ2dGJCB3Sjxuy2WdWHmOKtCsYLbIBRdeVKVpv0ZQBOnBcJIVzIx5w+uTr0zbro+VGTEFFasWaJqyRjHVjW6/sJkWhKxeJdqwd0qIRixSVF12WECHXICMT9rWXdVJP46LosWWppgkoimR2EeK5iI0J2yLTOjFVef1Aos6cF7N4NNRRg/2emPV2hZwTYeDlmm71tanapWGZLpUsFdEz0BvqaL0mR/J19BhzWsWPDF3JMnlGahVpb5zXMtCRe03Ga2jC2deEvv3UQ2+Ztu+5V6/o1xblvg+c3kc8Fb0IZnxAwNwjTPaAgDlBmOwBAXOC2breXIRCcex6K1etQIVw+GHfRmpJJFVEZXeHJqIrHYnro93RXDapibutuyFCC76os7D6X/3mtH3XW9+q+jY3hOc2O+RmMS4ST9GAscmgWjgk4gT9geaGg46M646HH5D2Hbep7cpXUellQcsu6d6ffu11tV23JeftI/2bX184MW1/8Bfl2H0ThfdHn5E1h0FXX0dQZFya7u5WHdA+h4b3Nymr0RVkjBd72n3nSSxyaN139IwUKIuub9cOyPVrM+JySukrRHrdokau1RqJeyRmLSUmLu1tVh2V1frOV0W//vSrp9R2R0kc4/htOnPu2KGj47FeRew0vNkDAuYEYbIHBMwJZmrGJ0mMtUk0WNdU7GSBivWDWlDi/Ia4f5bIHK/bKDwy2XJjRhUWxa3FyS633aHL+YwogebUaR3Rdn5D3Ckt0iOvV/SxfF/2sVjWiR9xRFpnbW1KliKmCWKCp6ZiJyeZOHOenAixsyPur3MmOm9Emm5RoaX6VlbkXpw4IZVsB0M9jpUlOdYZSi6aDHLaZPeX3UejL5+3+/p+tumZqEdsnupzZtM9tbqEdO36dOyBoQysS18yrtQS6RmuLWr6uUKfudxBYWh0CS/I9T7bOK/6LhI1jZlSGZO8viou0fc8ppO0PviRcY2D3OvzZ4Q3e0DAnCBM9oCAOUGY7AEBc4IZ68YD7hLPNnxkk0JHy0afnGt7NUhnPDHa6pyUlWcmK4jcdyM67Z6pPXb83jun7dKiLk77F1+XkNOLW8Jzc69dNfVUPlfq2i3H7rB0oC9/BcLhn/uGZNwtr+k1jDtPiDusXDbXgDjqi+S6eaOhOWSUU0llcy9SJ9tqr5zm1P2hcM+B6cuovHA3lRvTNElv7aGn7fQ4BrR+wk9E0Ty1PYqRzYzyZUYKGwPi8/b5cLQOUjW1Bjlst9XS6yxDcgNyecFiSQ+yTPusmlDuMoWKl6hGwlt+RmdFvv9vSF2Wtz6gBTDKlXH2I7umLfZS/ulW59wXnHPPO+e+55z7jcnfV51zzzjnXpz8v3KtfQUEBOwf9mLGpwD+vvf+PgDvBvDrzrn7AXwCwLPe+7sBPDv5HBAQ8CbFXmq9nQVwdtJuOeeeB3AMwIcBPDrZ7FMAvgjg41fbV5am2Nocu68iIy7BmUWdlnYfXIq6A4B+nwQNetp9UlsQ91rbaLI3KVIrolLPg+FrartGR7Khjtx+p+rbbojp3ldRXJp2ONJtq5e0q6lFY65WjE3rZPwxlab2xtXEBuhOT1OI7W3RMX/5pZPTdp5ZrXWKRMz1vTi1Jdfgwo5cx1ZLH2uDrlVatvr4sk/PWu4FU6qbTHdnzPiEzFu2kWtVPd72gFxq5jwduWfTnF10RqOQhrXT0felQMeul005r6rce86IK5a0OV0hs75S1X3H7xW6+K7/7l3T9gPveYva7taDx6ftBAuqz+fjcXi/+/v7R1qgc86dAPAQgC8DODT5Ibj0g3Bw928GBATsN/Y82Z1zdQD/EcDf8943r7U9fe8p59xzzrnnuqPdHf4BAQE3F3ua7M65AsYT/Q+89/9p8ufzzrkjk/4jAC5c6bve+6e99w977x+uFma++B8QEDDBNWefG8dj/h6A5733/5K6PgfgYwB+c/L/Z6+1Lw+H3I/5T9WUZXbkMmi1tJuoRsosjrTiB33NywuxcKZa1bikyIdUXpC+8pLWdV+sybhqRr++uS0KOrzGEJvaY8s1GaPPTRaWF45XTU0GWCY8bJUUdJYWNT8bELd99ayuKfbyD344bZ96Q1xjNguLVX76Rmv95MmT0/anPvPctG1VZjp9MfCKJhMvJ3dVRC7SgnGrFjijzNSLy2gfKYXfrixo3rzRlHFdlvRF38vz3cNqVWabUXtxxNkrRX3sBVonKtF2JeN6K9DX7nvnrarvA39HQl/XbxNVojzRYbUgF3THGMmdibs3u0q47F5etY8A+B8BfMc5dyn/83/DeJL/oXPuSQCvA/joLt8PCAh4E2Avq/F/id0Ljbxvl78HBAS8yTB7Eh2Pfzc6XWOCE583SUdoNCTbLGEzyphs7G5zpoRPROIYSSKmZCHWBzt0TAQC1o8cVX0VEirol8QNtVAz7hiyR7eMuGBGmvX9nnZDFUcylgqFiZlgL2xSOeQXf/hD1ffKD09O282m0KHMlDJOSGADhmoMevK9b39TzPgk1ttVC6zlbjTO2WTO6V1hrneBMhARGdOabPIRZcdFq/pYxXj3pSdHF49dY7ENNCMTvGzWlkokWJEZ8z+J5XtFEtiwrrf12+W5+vlf/nnVd/yYRETWKhItmTlN8ypOhEEriRYhdfH4mY6j3ad0iI0PCJgThMkeEDAnmK0Z7z3yib6XFa9ot+Tz8qrWMU8psWSHTNjLEmFY58skZjhKlmhTac94U2vUcyLBqQ0dobe1LQINlbqswtYq2qwcUnJHw5jgRUjUWbWuywCtUlmnFmnnb2xoDfKUqsZeuKhFKS5SldFul6IGjaleoeqv1UgvyWQU3Tikazo05Z9qVE7JUgFe7U55pduYyLwKnneNJjutpMPLsTPziqqQuAQivY+YlpuU5rsRPomJChTNGGv0sdvXevPbDYqMOyxCH6kZ48lzUn6rl51RfeWKrM6XyXTPsaG286oybk315ZPkJY8gXhEQMPcIkz0gYE4QJntAwJxgxq43Dz/hgOVyyfTI744zYVBVSuiPKRPKChXkxLdHhl9WyBUSUwTdwsqy2u7YcdHjfvG85vNZKq69CBQBaGL+t3LZf57oc6lQVlKponlXla7J8rKsW1ixyJ2mrB00tnWaQo+y6rqku17xRrSSuG1S1m6ixVjG4RaETw56pn5ZX8Zlgt8w4Ag6up8lozzBWvFWvz4itxaLQFrhiRpFLJa29T5Scptx9p0zaylOrfdolCnas1rR60Ts7k1p/MNYr+PsdOQ+5V73RU4iRIug+gk2ko+jHrGp+rYwjrZLoV3a6ji79gQEBPxEIUz2gIA5wUzNeA+HbBJNVa3pUsZJUT73BjrqrE/leAoxR5aZ6DQy76plvf+YBA8GIzHnqok2qY4eFx350w1d5rhE2mFZX8zPdqRN5Ix05CuxphORk+M5oxfWbAptOH367LRdW9KuyHZP3D+ttjb1vCN6kZC+vKE1Kf3ORyYaeqkuxysfuk+Ota2FPrbakuiYG5oAchMpLXNTBjsnlxpTNACoUTKJJw354UCThsW67LNYMOdJ9nqBPXnmvsSeKaA+F9bCWzTRkkNKiCquU4TbyqLabrUgz9VW44jqO7chY1k5LO7YzJjkOSjpxunkqAN+/LlghFQY4c0eEDAnCJM9IGBOECZ7QMCcYOaut0t1ylLjPhkSTxpYYQH63B5KOGGUaDdIty/ZWp22DmtMSJSCuX1nSYes/r9//qVp+wevaPGAEXHFnMMtq4aXU92wyGSDMZ/vD/WaQ5E4a7Mt5Zs73bbarpuzgIfexyhl16T8lo+MqGSRsqNyI76BkmT75TmtARR0plUUy/XvpXqM7EUrsAiiORaH0lYKms9z6es2rdukRqcTVcpsM05APrcShcEmhrOrcZnQ3w6Ji1oxkoTWCLaptt5fffkbevdOvvfw429TfdtNESDprsqxikV9rD6FWhe8XpOqYZwtF+M6dOMDAgJ+MhAme0DAnGC2rjfvkQ7HpslmR+vMsQtmZUWb1iw1N6KIq9HIunukWTAuGJYEc/TBBDqhQClOQ+OC2aYAspVV2W5kdL8iGlelqvXU65QhV6tpN0mXXGo9sioX6nq7BrmhBvY6klAEWY4YjTSt6ZNe31LV6MIVxXTPyVTPErNdQlGJxn3XJZdXTFl1uck2Y12LqtFkZ9rkKOIvM/ryrE9XMK8vx6WVyHU6yvU+WCgjNfv31LfR0NcxJvfmnYt3T9v3PHCP3gdd79vuuFv1JXTez5+VqMTjR3SE5YjFQxKt75r6sfmfQ2dqMsKbPSBgThAme0DAnGCmZnycJFg9MF417Pe1eEWvK+ZRf6hNkYSi3MolLudjEyLEzBzZiC76XStWZCWzb5Jpug0ZV5rrlc2IovziopiHI1NqqrQgpnupok1fXr2t1PWKKm/bGcr4bTkirnY66GqzErT6X6AlcTYjAaCfyjWu5Eb2OBa+kpXumrbztk4MgpdxDYxnoUf3kBfZ7SI4J7sUEj0OT9FpLOc8Guh726eEH1vhlbXgcnpeYq8HkucyXiteMaKkHmckuWPigQkJ2x04qKMeC/Rsri3qGqgHSLTk5Ktyz77T0ceqyma466AeY8+NBTGyYMYHBASEyR4QMCcIkz0gYE4wW9db7tGbCPZZRs1C3rocMhCTcEFK/rWREY0YEA8dGhdPyu4winCrGM6eFuTYDUN/KlQNKh3IdsOS5t6HV4Sf3UF64QBwywHJVrr1uO7jWhytnpzbVkvz8q1Wk76h1xWGlKFVSZiz6991PrXe0IovijjGKPv+tN3tvqK26w9krcJcRnAC25Ci0wpGoILdirkRLWHXaonqJ1mP67DP2+lHulyUY7dIaJTdhgDg/e6RZ47WDhJTarxG6y4bJF764g9e0vsoyQX5G0/+iup74DZaC8LJafuF5/U6yNrtcg1uXdduSheNXXb+esQrnHNl59xXnHPfcs59zzn3Tyd/v90592Xn3IvOuc8454rX2ldAQMD+YS9m/ADAY977twN4EMAHnHPvBvBbAH7be383gG0AT968YQYEBFwv9lLrzQO4lOVQmPzzAB4D8Lcmf/8UgH8C4Hevtq8sS7FzSXvd1nhyZMab0lCO3WgkXjGwJjhpjBVMkkyN3Fo1qry5dmRNbVdaFJPqfN5QfRdPc7KEjP/QYa1j98C9J+hYuq9NFKWxpfffTklLjbTOlqo6kipdEXO3ta0TUBJy2XHkly2HFbNe32VuKLrembjhIiMWwhoSywt6jGe7cp49SmJZqGtzeYFETBpdTd8ycnMp899Y3COiIQUbEsmJMeTms9VYVfKVEaGL6dnMY/3MVWj8dzzwlmn71rt1BF1SleMdOaifiVoi1O7uO6QUVKuroyNZL3+7r6/VSvXSPq8zEcY5F08quF4A8AyAlwE0vJ/GiZ4CcGy37wcEBOw/9jTZvfeZ9/5BALcAeCeA+6602ZW+65x7yjn3nHPuuX562bJcQEDAjPAjud689w0AXwTwbgDLzk0Tjm8BcGaX7zztvX/Ye/9wOdndxAgICLi5uCZnd84dADDy3jeccxUAj2O8OPcFAB8B8GkAHwPw2WvtK4oi1Gv1K/Z5cpVFRuCAM6UWFoTfDHNtKQyJw8dGzLFQlUyuFoVyXjynQ0CXSByje7Gl+rp94XWlovxOrhjRhQtnpK5XResO4kb4DZsAACAASURBVMH7hZNVCtoYatN5jiib7cKO5uWDXDiw/bVeXhAOycsira7xI1JoamaELToU/ltKyC1k7ktMtfZKZb1GcvyA8NI3zksm14LJ9OMaa5GpOVeg6+rZ5TUyopIUPhwZzp6R26xOazWJEeB0xOcj40bMTUYfg3Xk2UW3aWrwrR2R5/5gTbuM+cotViWU9vBtuibAa6T32TR18S55FYe7l3rbk5/9CIBPOedijJ+tP/Tef945930An3bO/XMA3wDwe3vYV0BAwD5hL6vx3wbw0BX+/grG/D0gIOCvAWasGw9kE1O1Qmb1GGIq2Wy24UjcDM2muKtiE7nG3+tl2k0EWhzskXsqMe6k2IupFxn34GAk5m2FMvFGO3q85QPSt7SozzMiN2J9WWdGtSDmepnccJm5HptNsdV6xgWTUVTh+rpwiIO1qtpui7TrWh1DJ8jlU2Ghj9xkI5IJbiQFcfSAHLvZEqpULmrK0xtRxlpsNOXJPeaJdkRm7WcwoLJLmbVjZdsVykZsbGqKViQz3l5vfjbLxmVXXxJaefGiPJs//L6ONlw4J/e9P9DCEzEOTdsjiqCrVE+r7UYjeZbaJsp0OHl20nx3Oz7ExgcEzAnCZA8ImBPM1Ix3GK/IA0Cjqc0orqpp8iGU6ECXNeiMqENSkJXe2IiRpU7M1iValS0muowOV+IsxXo1NBuJmZ3H69N2ua418w4dlr5CRZvxW1ty3me29TVIyINQZ/XlWNOVCmmpDU1WSIFWrXOiIbbU1OKCjCtyevn5YEU+H18Us3XY1avSDYpwy02YRZsEJiIWpTDiD8OURSP0GEeZfObhZ4Zelckr4M3+VxflmSglJIZhtisRLdvp68SjIl3TRVPW6d4Hf2rarq+K1Pax2+9U29VW5Tnrx7epvnP0TO9AqEZS1Fp1B9bkPJtNo4VXmFCvPJR/CgiYe4TJHhAwJwiTPSBgTjBTzp6mOTYmnLVa1r8zzZZEri0uaB5do/LLMblnMkPuk7LwFWfcOBlFZzHL3ejrDDtPIpMnX99SfUrHfF0ixGqrmpdvbsr3erneP2ewRcYdVqS6RlxqODfXKqK+slFYHJGLkcUgvCmHzJ9sGeIThySKa4kiHlfa2q1zprW7uOGAogELJDyRmvyIAt2XgllnicilxuGAlZq+3jUSlXx9S7u1Mk9a8SRe4U30ZUo1ArpGPHOZnqu1de0uffH5l6ftrZ1vTdub53UE3dJBya587InHVV+BpmEWCS+v10xpLyqRfeqcLk22sJ5NzsO4nAnhzR4QMCcIkz0gYE4wUzM+ihyqE7eRNdnWyTwaGTOqUiVXGbkWNnd0EkubKrdGsXZBZJmYnH1yw8U1LV6xviyulZVtnbTzRlPM4iqZ7n6kE1WyTPZRW9L7iCIxaWOvr0FMQguefE1Wm21A7rYo0gkosTLXpT206cVkPheNGyopC9VY4Cq357fVdiU6lg06S8mMzyhBKU9MMg2Z57ERleeSUhkJbJTNdhc35N52esaMJbGJlKINK0ajnulF2ejYFalk170PvVX15SV5Nrcb4krdOHdObVeqChWNIr3/wUAuXrMl93PD1D7o9aWvmmgqE+eTZ/+KieaT4+7eFRAQ8JOEMNkDAuYEYbIHBMwJZsvZ4wjVSahqo6HFFhMnvMiKBZw9L24Gpms+MiV+KfSyasoLry2Ty6sibrN+pLlPsysuwKaOxkVC/GyRxCuc0a935O6xIhqNHckoc4a/ViH7qZLwpTOCkDHxTasb74gD89fMLpARR3VOE73tvnwu9mntQO8CERFEZ+5ZRusKGbu5DLmPaR+2DlyuSmZLZ2eg3ZmHVmVdoTfSIchEh3HsoNy/9o7ejl1vizUdnnzo6MFp+4F3aM6e0fuy1ZK1m+6dR9V2xTI9ZyYzrbEjc+HVl07T3/Wa1NHD4hLNu/r5bk3covlod6GN8GYPCJgThMkeEDAnmHkE3dbmxEw2ZYAGZNp0utp9skCCB7Vlia6LjZBAkolpUzQ66aUilUPO5HvnNrSpxCIMudFEq5Cbq5ByCWFtmuakez8qau3vHunYlSt6jB3SV0/Lcs6ub8QxKPKuZCIFWUCN6ZA1kfljZMz4Dum37xRIyKJg3GZkgnvjRkyVDjvrC+pxsLScudxg7yyX86ol+r4fO0iZaG5d9W3Rveh2xcxmARNAl4teWtSRje/6b3522q4u6f3/7//6307bm5tijnd3dMbkAaICD//cz6u+tWWhDV86941p+6VXfqC2u+XYg9N2biIz82R8PpauMcKbPSBgThAme0DAnGCmZnwSR1hdGZvh3UFX9XG01PqSFoNo92XbHTLLSkb4LKN9NEd6/1ugJAgqDRUleh8VSrqpGIGNTpEEH8hsNYv2WK1QMkqqKUlK5mPu9BijAunwsSx2qhNOBrSinXvdVyvJufGqfcUkzCCWfWS5SRoiHbd2S65BOjB6fbSCHRk64cnEV4k7BUO9OILOeC6KysSnaEATDPgGab9tG222zS0x3T15OyytqS4KPXzkifeqvp95/H3TdrOnr/f7f/mXp21eje939b3lhKWNLZ1gxZFxa4elsm/RyG73R/JsJiUTQTfxlURXmdLhzR4QMCcIkz0gYE4QJntAwJxgtoKTziGZ8MgVUxdpa0syqpotzbtS8vgkxEN73Z7ZjjK5SkaUsCJ8sEjtPjSH3CG3X6Ol3RuFGpUlpq9xOV4AWCgKrzu7o11vIBHFfl+Tz5iyoWKq4xM5zWVLpOVuThMRabl3qJRxCn0uEdX/7Q61S63TFZ5eisn1FhmxCoqS89DnwmW2OdLOasNDZcfpfXDJac7M65vtOiT6YaMS2RdVJaHOpKKj5N79+M9N20985EOqr7Yoa0hLbb2Os0KCnD3S2+929H3v0NrH2Ze/q/ua4v5t0/4HZv3hJJXlGhhRzEFv/Lm1rSNTGXt+s0/KNn/DOff5yefbnXNfds696Jz7jHOueK19BAQE7B9+FDP+NwA8T59/C8Bve+/vBrAN4MkbObCAgIAbiz2Z8c65WwD8IoB/AeB/cWPR7ccA/K3JJp8C8E8A/O7V9pNmKba2NwAAvb42CStUyikxbqIe6cPX6xLdtLBUU9v1h2LmGCl0xLG4KkbECzZNQgRL0sVGxCB2Ylr7SEzJeKjPJaVdtkz0W5HGkUEbQ2VyQ2UD2WdcM9GAZAn3ekY/nHNO2L1kK7B6MT+deQyG5NsaUMmnNDFmvDo1K0pBen107Nxm09Af0pG+ViOKqiySW67vdSIJ04RyUR9g9YA8I0fXRUjk1nuPq+3e9dgjNCZ9nhtnpHzq9oYWpdg8L7pwzYZQ0ca2dq/t0OeWSQLrscuOTPWR8TEOyfWZjswzN0nGGpjngbHXN/vvAPgHkDu6BqDh/fSqnwJwbI/7CggI2Adcc7I7534JwAXv/df4z1fY9IqCOM65p5xzzznnnutbaaSAgICZYS9m/CMAPuSc+yCAMoBFjN/0y865ZPJ2vwXAmSt92Xv/NICnAWCtVrqKQlZAQMDNxF7qs38SwCcBwDn3KID/1Xv/q865PwLwEQCfBvAxAJ+91r7iKMby4tjlVkh0OGGWyu9ApayzjpKS8MsulRqOjTClLlerwwm75ALLiZNlkeZ/sRNeV3aGG6YkGkHD73a1m6UVy/iTyHBZCs9N7U8fcdZiSYynnq2H3BGOl5pabwn5BBMWhEz1OCIyzrp9K3xJ2xHfHpm1iTplLi5clsEnY2Yt+765ZxG517qpvt6ctVem9LgOdhfzuOfew6rv4C3y+eCaiJoWKvqcG2demrbPv6JdXj165gamzsCQ+lIS1Sjk+jwXKYuxvqZFSEdLVKtOKY5Ag7oiIxJ6adOTp89iN1xPUM3HMV6sewljDv9717GvgICAm4wfKajGe/9FAF+ctF8B8M4bP6SAgICbgRlH0El2VG1Bu8065F7bNu4wjpobkPjDwGiRFah0b6liIrUKVBqYzXFjfr5+UcbR08PAoRUxkTl7K0q0C21APq/EuKSG5L5KjOvN07Y7beEJUU2bbMWRHNsKZ7iYSlpnFFFoXG+Njux/mGnKUyDhj0oiJngvM1F4VCK7ZqjGQlX6Xj2zMW1vGNpRJT9iavxyZYpy61GWZL2mr9sv/vdPTNuPP6rfP5UqR8pxOSm1mb5LxnzOmMrYslFEj4YkPtLvG5duV8QshkNNYYdDcZex6Edu6RtlWlqd/nxi/v/Jn30FuyHExgcEzAnCZA8ImBPM1IzPco/WpERTqaoT8zl4KjWJDs22rD6XSX+sVLFmMJmjZhW8SBp0Wy1ZPd9u6VXTzpBM37I2byMacoNW/hcTLbZRo5JDiYnGSiiZJk31+EckuJFRwowz5u2AVsXZQwAAOfWNaMW2Z1bB+z2iQ5lefR7RKrsrkiS0EYnbGcm4qgVtVy7RpSvQO6Xd0hFeXaIaR5Z1clSZq/JSddKSscEHIxl/1XhyauTJyVS0nn7GMrqfIxMPkpOQSGquY7fPGnc9amszvteTezvoaTO+T3R0SPuzUXJD+pwa74pE0FkpFUF4swcEzAnCZA8ImBOEyR4QMCeYKWcfpSkubI6zf6KGPvTaskQVLdW1sACIU0YkklCqafddOhQO3Dalc1oj+V1jTlYs67WDCkW45QXDqReoFBILapifzIQ18U1J5YwoawodMVakTLGIMv8KZv1hSKKHdSMGkZE4ZU6ZYpm3EXS7l10CCU8MiSvnmT4W08btHRP9NpBrUGQ3pXEZLa8LuS/HWgSkR2s3KYlW9gf6WH/xhf9v2s56Otvs0QclP8vR9e61zRoGCZm2TZRcjz73Orv3tcld2re8nLbjzDYAGFCGIz+bLPwJAENyxRnKPuXw3bbWq2eEN3tAwJwgTPaAgDnBTM34UrGI20/cAkBrbQFQ4gojkyFSrYiJ32pL4n9rU++jSDbi0PyOVSiirkKRds2OtmH7pEG3elTTiYii7UaUPJIY29SR+exzPY4++RjTWJuSA3LrFLjkldGG71HUnK/qW5hy0glF8hkZOzgykYsFTWX4bDxt1zPVauOEXVn6AD26/jzCYslW3pV9vrHRVn39hO4Z0Td2hQGA68p5/uVfaH03UFXXuw6TgInR3et35Rp3jL5bh0zwYcckyRCl6PYH1DYJM0Pp4yhQAOiTKEVG52Yj6IYpJxfp8V+y+E0ukUJ4swcEzAnCZA8ImBOEyR4QMCeYKWf3eT7lxHGiQ1EHJBa53dRus05XiMhqnTTTY82piYqr8EcA6OzI/ptt+Y3b2Nbcp1Yj8QpTe4zrtBXI5ZUmmsdxRGjq9RiHsYxruab5ax7JNYlJxGDY0Vw2g4SE9jKzD8rKIul5+KKpo0ZcPDHi87yPnF1Bhicm5LPLjFikJ3cbe9TSgV5/aLdkkBcb+jyLdQqNpvLcrEkPALGXc+s29b341jdfnrbzt98+bR9e1SHOHVqp6I709Wj25Lx7bf1cXQr/BjS3b7e1oEmfQmn7JqS1R59V1pupv5zRGoyJKJ+uY6RXIe3hzR4QMCcIkz0gYE4wWzMeHrkfm/GNhjZzMnY7meyqtSUxhUtlsdV3WtqWYcGAvnF5DUeyz/aQI9W0GVyqiSmdpMa8Ja34QpUiuqBdNdFAxpiWTfRbJmZs2SQoFXIZy0JFxrvZ0BsOiEIkIz3+lEy/1HHkmjbjE5Y6M5FrrDvnScveGVUHT+eSmkw0kpZDqSD7GFpt+NHuUWGFAZm3NPyCcQEqIQdDa/pNMa1Pvib6bLExkQcDoRAdU6q7Sc9qx0TQdboyRtbXs1p1A4qaS42WnyPaxBr7sXkXxxSZmZs54ieUyjkbDikIb/aAgDlBmOwBAXOCGWvQRShMRCQOHdCr8b1em9o6soxjuvpkpjkTueZHbJtqc4ZLIVXLso9aSQsmOEpA6fS0uVgpsjabtLsmPG1I5hZHTo3HLO3tlk6WKAzJPKfyTHmkf5M9mXr9gameSia5K9JJD/W5cGklP9Imp+MV4ZxENIyJmNL3skiPY7gbLTP74CixoRGUKNIYU7LOi2Yl2hGdcLF+JhJ6xHubEn35RqTN/cPr4uFYquiIwrjPwhn64BU63nBAUaBGl7DP0YZDfQ24zFOaMYUy14rezZet1E8+x1Ew4wMC5h5hsgcEzAnCZA8ImBPMlLPneYbWpDxtkmjOVK4tTNu9VLs+upSFlKeUIWSE3QtViX6zlYF7xL9HA+E1Uay3jKlMz9Br/rdakP2v1VdkfCZaypGaRdHojHvyE7WMeGGeyrgutIQPV0oLaruY3Fz9nt4/lzVyJDbR6xo/H513pajHD+LRHLVlLgeIKiNLjIuR1iqKFGmXGgGMPmeKmfWNKr2KMsr0y8wahiPBjqQQmz7i0RSt1zQnUy0dnLaPr2lRlCLdp6IREimxqIajjEMjFlLwCW1nnjl6WlNaI8mM+5h5un2+L1H9q1D2PddnPwmgBSADkHrvH3bOrQL4DIATAE4C+B+899u77SMgIGB/8aOY8b/gvX/Qe//w5PMnADzrvb8bwLOTzwEBAW9SXI8Z/2EAj07an8K4BtzHr/WlS+6bzGtX0A6VfOp1tHleX1mftmNyw22nOnJtsyXuu75xb7RIZMAVxN22WDd0oizRerFxn5RK4p5ZX1qbtrOqNrP7NI7UJOTsUOJEc6DNVk+Rcd2R3JrcRK7FYI04sw8ypz25cfpGc40KzaJmklMGVJZqSNSiaF2d1GcC9JCSpvyAtN+6o93N+FGqx5hFcg0yoiujmn5Hlcl2LSZGR9+RJh+dZxRrd+MOJeHslPU+ilw1yulja6+ip7baTLvRDB1ilybvz5tErJwooLXW+di7Ya9vdg/gz5xzX3POPTX52yHv/VkAmPx/cNdvBwQE7Dv2+mZ/xHt/xjl3EMAzzrkf7PUAkx+HpwCgWpzpemBAQABhT2927/2Zyf8XAPwxxqWazzvnjgDA5P8Lu3z3ae/9w977h8tWczkgIGBmuOar1jlXAxB571uT9vsB/DMAnwPwMQC/Ofn/s9faV577qe724uqy6mMdwqEJ3zx3/uK0XSEek5uQR87oH5jMoiqJZeQUDlkq6bDd1WXh4iuHD6m+mDgf8+vDCytquxYRqlZHu7U8RWKOUsOVKaS1T9ls3mSKRRGVnzZcOSJB+yFl93VNWG2lQHx7aLh4cmWhCOboAFS62XBgxDPpXvQh420ZEcVOj9ZdjE56SteYu4apdX/tzodTcoc5IsS2nltnRzj7GRMSe2CtTp/MtaLPOTnEvHG95XRf7BoM83lv/ZsMelea3SPPr+Jzm2AvdvUhAH88WURIAPxf3vs/dc59FcAfOueeBPA6gI/uYV8BAQH7hGtOdu/9KwDefoW/bwJ4380YVEBAwI3HTFfMojhCZWkcnWR15ioV8QU5oy3uKJqsTyZnbVHru1XJtGmbiLEMYhLWF8i9VtCXYHlZTPIHH3pI9fVaUlpnSO7Bna52AfZHVDLJ+qTIFVcyl79I5YY5Sqy9Y6LwSG++Y+hKiew7F8l55t5kztEuWYceAIa0bU7UYuSNW2vEriDjNqMDcPRbx+vtRhSd5k1561Esx0uJvjmj+Z6zOEaq7zuX8+KMMFv22XPJZuMS3WjJ/hcS/T1H+8nJBLeuZTb3r+Yky2l/3lrm9MXLrP1rW/EhNj4gYF4QJntAwJwgTPaAgDnBzKNc/OT3JYptzS/hawWTEbe8JOGoXVKxaQ00V+6SvvzAiEUuLAt/LZQkHLS+XFfbFSvSt7io3YNF4oZbNN7U1AZrUunejlHdyTPieCPNPR0p6JQoK20n0lw2Z4UYw6OrmYy/UhBiVzBeSk9qNH3zm99JyfVJKi15YhRcaFjDVLtL+fZ2qS5eZms2kwssyW04rnDnrCDPgDMlmwcUKhoZ3s+ltSMOvzXHcsSxM70LDHtyz5pFHUJdYx9YzllpV2HmNpR2t067HZF4fxlJv7SXoFQTEDD3CJM9IGBOMHMzPppEGRUibYpxCZy8UFV9jkr3glww3a42HTebYj73M31qri99SSrm+cKSjn47evwOGYcRaSyTa6xHOuBbFy6q7bi8z44RZOj1xWxdhnbLRcocFfMwKehxOCrd20+0W65KmXpLpP7AevsA0O9I30XjOmyR26xIJrPVda+Szn3fUJKIRDsyyqIzSWlIM3bfmSwvMvFZKt4bE7xP161Usm5KcocRhYpL5vmgthUJTai3aChmVpAIzMjJ92wSGmes2fGzHc9ZddlVtPgtLu3yam698GYPCJgThMkeEDAnmKkZn+U5WhPTu1bRCShsjba6uppnj4QLnJMhx8auKVE0XKWsqUBUkpXklCKwVpa1GX/nXfdN253GpurbOrchY2xIBKAziSoLVAl2WNS/p92uJMac39G68b4ly8ALddFBKxndswElTsRGz6xO1vpKXc65aFaRhxRNtjM0K+mkf07yccjM9S6QrZqYyLKchDMKlFiSGM0/Xp23+vgj2kdOkYIlIwhSItOa9QXH+6TIONq9cU5gSLQvNokwGem6l03pqX5BrmtUJw1EkwDFZnxuk1jYXI9Zp19v55U2oL4X00Sbq9j64c0eEDAnCJM9IGBOECZ7QMCcYLacPcuw09wCAGxv6t+ZhHwy1ZrOZkvJFcLZVM7wkzgS/hSb+msFcufdcuuRaXt1Qdd6O7C+Om2fPfO66nvphRem7SGV5C2V9XiH5BoznjGsEJ9vDnXn+U3h8EzX1hY13+6RqGTRULQSRc2VqRx1wWT3VYhjH/M6irB/XrL7YjqBPDelnekWJokeCItNFMgNFxm+PaQItCjWJDWjd9FoIBw4MVF4nFk4MmUCc856o4tlhUPKJGJyGZ+ncQwMZ3ctyX4s0RpJUtcipJ5ctZdH0F25Fp4tkc18PDfPfj4RCwmut4CAgDDZAwLmBTM14+MoQn1i3gwH2t0z6FMJ4UgPa3FJTKxmV0zMzU0d+bXdEXOxWjZuOYq2O39eXGrv+Bntejt37qzso67LAJUoOu1iU/YRtfS5JFVyeVmTjRR26+va1CuRXnmvy3p6xvVGgg8V41IDmbgDSsIxAW6ol+V3frGu3ZSVLUq0oftkdcw5os4+SBHRKO5zJsmkynTLXKsd1sQfcVKMPmdPR7BRfr2BnCfThEqsR1wgN19shFHZ7cd6egCQEA1Jh0I1uBQZAERUW8A3m6qPz1sJVlg3GguCmBMdXTLjg+stICAgTPaAgDlBmOwBAXOCmXJ274FLmgnVunZ5pRC+s73dUH05uVO65DKJTd2t5QXhci7R4bjMN6v1pWl7a1vXlVtoSkjsd77zguo7f/68jJHquUVGzLHsZYxLFc0vUwpzLCaaAy9UKdyXXJE7RlSy35HPmTMiIETlhiP6nqnlW62QmIdRtqgQZ2332DWmXYUsYumMbrkjTpwT93aGaybE2XMjFpnQ99KM1mfiywrL0YE1Z+1l8hywwGdq1kFGJOaBijkXup+mAjdyx9dAxuWNjn6F6iTYpLduU57BjIUpMz3GlLT4s0wP5FKdvMDZAwICwmQPCJgXzDiCLsfO9tjt0OroQ/cpcyzJdzexWMMtMVFhOWXEweil1anE8q1HT0zblZp2ryGjrCOT/bSwStr2lM3WNRr1uZPxclQfABTJ7GPNdAAokHBbJxOaUDRCCxnZ6qn5vU7peMUaCSsYV1OHBDZqVaNjV5Zr0KcExMRow3PJIW/M84xoCGfteWMGs3hFauhKTOIVnu5FamzpIZfDsuIVdRnHgNxkRWMGR16uW26yGNk1FhkzuUiRiFGZRTrM1CLKVlvR2oZ9en7YzZeaeZDReaemvHV6o8x459yyc+4/OOd+4Jx73jn3HufcqnPuGefci5P/V669p4CAgP3CXs34fwXgT733b8G4FNTzAD4B4Fnv/d0Anp18DggIeJNiL1VcFwG8F8DfBgDv/RDA0Dn3YQCPTjb7FIAvAvj41fbl4acSu30jA80BdVWzOlyiiDRWVb5gxB86XsyhalWvqKZOIt4c6ZLV7Go5tSslverbIZN2kcz/qtnHoEcr9cZsLZI53TaSyKBV8KQo5+xMRFeRovAiU8Y1olX3AiV3LBlZbC615G1SCD0VBaIdsQnDG40owcU8SQWKNkxohXxgKsaqhfRE798N5Vwy+l7X64jFKkl8mwA6xBWSiCbTPTVCGUOiCd7UXWLtEOMHABcSdpT9k5syqwW677GJeqwsi2eqcU70DDOTdJOlbMabUlkTz8v1mvF3ALgI4P90zn3DOfd/TEo3H/Len50c4CyAg3vYV0BAwD5hL5M9AfAOAL/rvX8IQAc/gsnunHvKOfecc+45G7ccEBAwO+xlsp8CcMp7/+XJ5/+A8eQ/75w7AgCT/y9c6cve+6e99w977x9OgqMvIGDfsJf67Oecc2845+713r+AcU3270/+fQzAb07+/+w1j+YcMBFUKBs1vRLxroGJPuq2uKyTfE8zN53sb8Ua6pQ516fspG9/+2tqu2JV+FOU63WF9rZE1zWa0rdsxDYS+lVrmIy4LBIubgUUBj0WR5TxL1W1G5H5X2ZDuoiXDjlKzGmeuLwuHL5oBC37VPLp/Hm6BkYZokD8sGCEJB3VhnI0poLhyo7cjXHJRL/R5SlB3J79vr6mo1w+90YmKjGja0frFL1cPx8xXYOimRY5XeNhX1/vmMpGxSmJhRh3qU8oe9BE+RWp7BdnTPa6ek2K3W2Zcb1dKjPt/e7m81797P8zgD9wzhUBvALg72BsFfyhc+5JAK8D+Oge9xUQELAP2NNk995/E8DDV+h6340dTkBAwM3CzMs/5ZOIqaJxV/XJzTA0Lh5PGmNlMo9ir022IZn4iUkyWV4W3blShdxmdW0id9ti4jcaOiEnI225Kple7aZOpukOxNTb6RmzihJX6ibKivXNMtJ+q5hkF08JKM0dTTV2ikIpKuR2KnV0lF+pKrTmCNmU/gAABS1JREFU2O3HVN+5s9vTdoeWYpzRiCtREovNTfHkHnNktppTQZxwxKLGKOGkIflibiq1DlO5BgWTADUid1VRlWfV5nhK52LyceDc7s8mV86q0Rid0UDMyM06Mu7HiNysC5Qw029rM77dkmdzaKjucEK9ListxcfZtScgIOAnCmGyBwTMCcJkDwiYE8w4681jpznmFsargAIRtpqp05aT64ZDTNsmOymjYMZqrNcEyhTXeGDtwLS9tKyz3l5t/VCOOzLckLhWlUIek5omou0B1YEztd6cpzWHSH8vpyyniEI281zzv8WacLyzF4ybktYIuHR0c0fXz4tImHKhqq9BbUHEPeIaiVAYrhzxGolxqXkSfODRp4az+4xddJq1F0lwJKZA5qHh2yPFX/UjnVJILyftFYpmXWhIpZIN7y3SwxmbdRYXk6Al3U+f6evRo3hwZ9zCMQlolstyb5fX19R22xdlDWlkyokPJvu/mustvNkDAuYEYbIHBMwJ3NWyZG74wZy7COA1AOsANq6x+c3Gm2EMQBiHRRiHxo86jtu89weu1DHTyT49qHPPee+vFKQzV2MI4wjjmOU4ghkfEDAnCJM9IGBOsF+T/el9Oi7jzTAGIIzDIoxD44aNY184e0BAwOwRzPiAgDnBTCe7c+4DzrkXnHMvOedmpkbrnPt959wF59x36W8zl8J2zt3qnPvCRI77e86539iPsTjnys65rzjnvjUZxz+d/P1259yXJ+P4zES/4KbDORdP9A0/v1/jcM6ddM59xzn3Tefcc5O/7cczctNk22c22d045+/fAPhvAdwP4Fecc/fP6PD/DsAHzN/2Qwo7BfD3vff3AXg3gF+fXINZj2UA4DHv/dsBPAjgA865dwP4LQC/PRnHNoAnb/I4LuE3MJYnv4T9GscveO8fJFfXfjwjN0+23Xs/k38A3gPgv9LnTwL45AyPfwLAd+nzCwCOTNpHALwwq7HQGD4L4In9HAuAKoCvA3gXxsEbyZXu1008/i2TB/gxAJ/HOK19P8ZxEsC6+dtM7wuARQCvYrKWdqPHMUsz/hiAN+jzqcnf9gv7KoXtnDsB4CEAX96PsUxM529iLBT6DICXATS8n4qqzer+/A6AfwCRfF/bp3F4AH/mnPuac+6pyd9mfV9uqmz7LCe7FSIBgLl0BTjn6gD+I4C/571v7scYvPeZ9/5BjN+s7wRw35U2u5ljcM79EoAL3ntW/dyv5+QR7/07MKaZv+6ce+8MjmlxXbLt18IsJ/spALfS51sAnJnh8S32JIV9o+GcK2A80f/Ae/+f9nMsAOC9b2BczefdAJadm1bHnMX9eQTAh5xzJwF8GmNT/nf2YRzw3p+Z/H8BwB9j/AM46/tyXbLt18IsJ/tXAdw9WWktAvibAD43w+NbfA5jCWxgr1LY1wnnnAPwewCe997/y/0ai3PugHNuedKuAHgc44WgLwD4yKzG4b3/pPf+Fu/9CYyfh//He/+rsx6Hc67mnFu41AbwfgDfxYzvi/f+HIA3nHP3Tv50Sbb9xozjZi98mIWGDwL4Icb88B/O8Lj/HsBZACOMfz2fxJgbPgvgxcn/qzMYx89hbJJ+G8A3J/8+OOuxAHgbgG9MxvFdAP948vc7AHwFwEsA/ghAaYb36FEAn9+PcUyO963Jv+9dejb36Rl5EMBzk3vznwGs3KhxhAi6gIA5QYigCwiYE4TJHhAwJwiTPSBgThAme0DAnCBM9oCAOUGY7AEBc4Iw2QMC5gRhsgcEzAn+f3iinn/TQLPTAAAAAElFTkSuQmCC\n", 183 | "text/plain": [ 184 | "
" 185 | ] 186 | }, 187 | "metadata": { 188 | "needs_background": "light" 189 | }, 190 | "output_type": "display_data" 191 | } 192 | ], 193 | "source": [ 194 | "index = 42\n", 195 | "plt.imshow(train_set_x[:,index].reshape((64, 64, 3)))\n", 196 | "print (\"y = \" + str(train_set_y[:, index]) + \", it's a '\" + classes[np.squeeze(train_set_y[:, index])].decode(\"utf-8\") + \"' picture.\")" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": { 202 | "colab_type": "text", 203 | "id": "4PR2XRmcIgxQ" 204 | }, 205 | "source": [ 206 | "Many software bugs in machine learning come from having matrix/vector dimensions that don't fit. If you can keep your matrix/vector dimensions straight you will go a long way toward eliminating many bugs. \n", 207 | "\n", 208 | "**Exercise:** Find the values for:\n", 209 | " - m_train (number of training examples)\n", 210 | " - m_test (number of test examples)" 211 | ] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "execution_count": 64, 216 | "metadata": { 217 | "colab": { 218 | "autoexec": { 219 | "startup": false, 220 | "wait_interval": 0 221 | } 222 | }, 223 | "colab_type": "code", 224 | "id": "bM5XLGe8IgxR" 225 | }, 226 | "outputs": [ 227 | { 228 | "name": "stdout", 229 | "output_type": "stream", 230 | "text": [ 231 | "Number of training examples: m_train = 209\n", 232 | "Number of testing examples: m_test = 50\n", 233 | "\n", 234 | "train_set_x shape: (12288, 209)\n", 235 | "train_set_y shape: (1, 209)\n", 236 | "test_set_x shape: (12288, 50)\n", 237 | "test_set_y shape: (1, 50)\n" 238 | ] 239 | } 240 | ], 241 | "source": [ 242 | "### START CODE HERE ### (≈ 2 lines of code)\n", 243 | "m_train = train_set_x.shape[1]\n", 244 | "m_test = test_set_x.shape[1]\n", 245 | "### END CODE HERE ###\n", 246 | "\n", 247 | "print (\"Number of training examples: m_train = \" + str(m_train))\n", 248 | "print (\"Number of testing examples: m_test = \" + str(m_test))\n", 249 | "\n", 250 | "print (\"\\ntrain_set_x shape: \" + str(train_set_x.shape))\n", 251 | "print (\"train_set_y shape: \" + str(train_set_y.shape))\n", 252 | "print (\"test_set_x shape: \" + str(test_set_x.shape))\n", 253 | "print (\"test_set_y shape: \" + str(test_set_y.shape))" 254 | ] 255 | }, 256 | { 257 | "cell_type": "markdown", 258 | "metadata": { 259 | "colab_type": "text", 260 | "id": "fCkW_m4bIgxU" 261 | }, 262 | "source": [ 263 | "**Expected Output for m_train and m_test**: \n", 264 | "
Converted:XLIV
\n", 265 | " \n", 266 | " \n", 267 | " \n", 268 | " \n", 269 | " \n", 270 | " \n", 271 | " \n", 272 | " \n", 273 | " \n", 274 | "
m_train 209
m_test 50
\n" 275 | ] 276 | }, 277 | { 278 | "cell_type": "markdown", 279 | "metadata": { 280 | "colab_type": "text", 281 | "id": "r-AuahEtIgxV" 282 | }, 283 | "source": [ 284 | "### \"Standardization\"\n", 285 | "To represent color images, the red, green and blue channels (RGB) must be specified for each pixel, and so the pixel value is actually a vector of three numbers ranging from 0 to 255.\n", 286 | "\n", 287 | "One common preprocessing step in machine learning is to center and standardize your dataset, meaning that you substract the mean of the whole numpy array from each example, and then divide each example by the standard deviation of the whole numpy array. But for picture datasets, it is simpler and more convenient and works almost as well to just divide every row of the dataset by 255 (the maximum value of a pixel channel).\n", 288 | "\n", 289 | "Let's standardize our dataset." 290 | ] 291 | }, 292 | { 293 | "cell_type": "code", 294 | "execution_count": 65, 295 | "metadata": { 296 | "colab": { 297 | "autoexec": { 298 | "startup": false, 299 | "wait_interval": 0 300 | } 301 | }, 302 | "colab_type": "code", 303 | "id": "C1NrOEyMIgxW" 304 | }, 305 | "outputs": [], 306 | "source": [ 307 | "train_set_x = train_set_x / 255.\n", 308 | "test_set_x = test_set_x / 255." 309 | ] 310 | }, 311 | { 312 | "cell_type": "markdown", 313 | "metadata": { 314 | "colab_type": "text", 315 | "id": "VrfXbSL7IgxZ" 316 | }, 317 | "source": [ 318 | "\n", 319 | " What you need to remember:\n", 320 | "\n", 321 | "Common steps for pre-processing a new dataset are:\n", 322 | "- Figure out the dimensions and shapes of the problem (m_train, m_test, ...)\n", 323 | "- Reshape the datasets such that each example is now a vector of size (number of features, 1)\n", 324 | "- \"Standardize\" the data" 325 | ] 326 | }, 327 | { 328 | "cell_type": "markdown", 329 | "metadata": { 330 | "colab_type": "text", 331 | "id": "_0zb_21DIgxa" 332 | }, 333 | "source": [ 334 | "## 3 - General Architecture of the learning algorithm ##\n", 335 | "\n", 336 | "**Mathematical expression of the algorithm**:\n", 337 | "\n", 338 | "For one example $x^{(i)}$:\n", 339 | "\n", 340 | "Compute Linear Regression: $z^{(i)} = w^T x^{(i)} + b \\tag{1}$\n", 341 | "Pass it through the activation function: $\\hat{y}^{(i)} = a^{(i)} = sigmoid(z^{(i)})\\tag{2}$\n", 342 | "We will use log-loss as a loss function:\n", 343 | "\n", 344 | "$$\\mathcal{L}(a^{(i)}, y^{(i)}) = - y^{(i)} \\log(a^{(i)}) - (1-y^{(i)} ) \\log(1-a^{(i)})\\tag{3}$$\n", 345 | "\n", 346 | "The cost is then computed by summing over all training examples:\n", 347 | "$$ J(w, b) = \\frac{1}{m} \\sum_{i=1}^m \\mathcal{L}(a^{(i)}, y^{(i)})\\tag{4}$$\n", 348 | "\n", 349 | "**Key steps**:\n", 350 | "In this exercise, you will carry out the following steps: \n", 351 | " - Initialize the parameters of the model\n", 352 | " - Learn the parameters for the model by minimizing the cost \n", 353 | " - Use the learned parameters to make predictions (on the test set)\n", 354 | " - Analyse the results and make a conclusion" 355 | ] 356 | }, 357 | { 358 | "cell_type": "markdown", 359 | "metadata": { 360 | "colab_type": "text", 361 | "id": "ryjne1TVIgxb" 362 | }, 363 | "source": [ 364 | "## 4 - Building the parts of our algorithm ## \n", 365 | "\n", 366 | "The main steps for building a learning algorithm:\n", 367 | "1. Define the model structure (such as number of input features) \n", 368 | "2. Initialize the model's parameters\n", 369 | "3. Loop:\n", 370 | " - Calculate current loss (forward propagation)\n", 371 | " - Calculate current gradient (backward propagation)\n", 372 | " - Update parameters (gradient descent)\n", 373 | "\n", 374 | "You often build 1-3 separately and integrate them into one function we call `model()`.\n", 375 | "\n", 376 | "### 4.1 - Helper functions\n", 377 | "\n", 378 | "**Exercise**: Implement `sigmoid()`. As you've seen in the figure above, you need to compute $sigmoid( w^T x + b) = \\frac{1}{1 + e^{-(w^T x + b)}}$ to make predictions. Use `np.exp()`." 379 | ] 380 | }, 381 | { 382 | "cell_type": "code", 383 | "execution_count": 66, 384 | "metadata": { 385 | "colab": { 386 | "autoexec": { 387 | "startup": false, 388 | "wait_interval": 0 389 | } 390 | }, 391 | "colab_type": "code", 392 | "id": "LHdT7SEIIgxc" 393 | }, 394 | "outputs": [], 395 | "source": [ 396 | "# GRADED FUNCTION: sigmoid\n", 397 | "\n", 398 | "def sigmoid(z):\n", 399 | " \"\"\"\n", 400 | " Compute the sigmoid of z\n", 401 | "\n", 402 | " Arguments:\n", 403 | " z -- A scalar or numpy array of any size.\n", 404 | "\n", 405 | " Return:\n", 406 | " s -- sigmoid(z)\n", 407 | " \"\"\"\n", 408 | " ### START CODE HERE ### (≈ 1 line of code)\n", 409 | " s = 1 / (1 + np.exp(-1 * z))\n", 410 | "\n", 411 | " ### END CODE HERE ###\n", 412 | "# 1 / (1 + np.exp(-1 * (w.transpose() * z + b)))\n", 413 | " return s" 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "execution_count": 67, 419 | "metadata": { 420 | "colab": { 421 | "autoexec": { 422 | "startup": false, 423 | "wait_interval": 0 424 | } 425 | }, 426 | "colab_type": "code", 427 | "id": "I0FcrFkCIgxd" 428 | }, 429 | "outputs": [ 430 | { 431 | "name": "stdout", 432 | "output_type": "stream", 433 | "text": [ 434 | "sigmoid([0, 2]) = [0.5 0.88079708]\n" 435 | ] 436 | } 437 | ], 438 | "source": [ 439 | "print (\"sigmoid([0, 2]) = \" + str(sigmoid(np.array([0,2]))))" 440 | ] 441 | }, 442 | { 443 | "cell_type": "markdown", 444 | "metadata": { 445 | "colab_type": "text", 446 | "id": "2Ciaq0cuIgxg" 447 | }, 448 | "source": [ 449 | "**Expected Output**: \n", 450 | "\n", 451 | "\n", 452 | " \n", 453 | " \n", 454 | " \n", 455 | " \n", 456 | "
sigmoid([0, 2]) [ 0.5 0.88079708]
" 457 | ] 458 | }, 459 | { 460 | "cell_type": "markdown", 461 | "metadata": { 462 | "colab_type": "text", 463 | "id": "A4XFeomFIgxg" 464 | }, 465 | "source": [ 466 | "### 4.2 - Initializing parameters\n", 467 | "\n", 468 | "**Exercise:** Implement parameter initialization in the cell below. You have to initialize w as a vector of zeros. If you don't know what numpy function to use, look up `np.zeros()` in the Numpy library's documentation." 469 | ] 470 | }, 471 | { 472 | "cell_type": "code", 473 | "execution_count": 68, 474 | "metadata": { 475 | "colab": { 476 | "autoexec": { 477 | "startup": false, 478 | "wait_interval": 0 479 | } 480 | }, 481 | "colab_type": "code", 482 | "id": "IBhZbikRIgxg" 483 | }, 484 | "outputs": [], 485 | "source": [ 486 | "# GRADED FUNCTION: initialize_with_zeros\n", 487 | "\n", 488 | "def initialize_with_zeros(dim):\n", 489 | " \"\"\"\n", 490 | " This function creates a vector of zeros of shape (dim, 1) for w and initializes b to 0.\n", 491 | " \n", 492 | " Argument:\n", 493 | " dim -- size of the w vector we want (or number of parameters in this case)\n", 494 | " \n", 495 | " Returns:\n", 496 | " w -- initialized vector of shape (dim, 1)\n", 497 | " b -- initialized scalar (corresponds to the bias)\n", 498 | " \"\"\"\n", 499 | " \n", 500 | " ### START CODE HERE ### (≈ 2 lines of code)\n", 501 | " w = np.zeros((dim, 1))\n", 502 | " b = 0\n", 503 | " ### END CODE HERE ###\n", 504 | "\n", 505 | " assert(w.shape == (dim, 1))\n", 506 | " assert(isinstance(b, float) or isinstance(b, int))\n", 507 | " \n", 508 | " return w, b" 509 | ] 510 | }, 511 | { 512 | "cell_type": "code", 513 | "execution_count": 69, 514 | "metadata": { 515 | "colab": { 516 | "autoexec": { 517 | "startup": false, 518 | "wait_interval": 0 519 | } 520 | }, 521 | "colab_type": "code", 522 | "id": "MXukxYQ8Igxj" 523 | }, 524 | "outputs": [ 525 | { 526 | "name": "stdout", 527 | "output_type": "stream", 528 | "text": [ 529 | "w = [[0.]\n", 530 | " [0.]]\n", 531 | "b = 0\n" 532 | ] 533 | } 534 | ], 535 | "source": [ 536 | "dim = 2\n", 537 | "w, b = initialize_with_zeros(dim)\n", 538 | "print (\"w = \" + str(w))\n", 539 | "print (\"b = \" + str(b))" 540 | ] 541 | }, 542 | { 543 | "cell_type": "markdown", 544 | "metadata": { 545 | "colab_type": "text", 546 | "id": "fSwCgelAIgxl" 547 | }, 548 | "source": [ 549 | "**Expected Output**: \n", 550 | "\n", 551 | "\n", 552 | "\n", 553 | " \n", 554 | " \n", 555 | " \n", 557 | " \n", 558 | " \n", 559 | " \n", 560 | " \n", 561 | " \n", 562 | "
w [[ 0.]\n", 556 | " [ 0.]]
b 0
" 563 | ] 564 | }, 565 | { 566 | "cell_type": "markdown", 567 | "metadata": { 568 | "colab_type": "text", 569 | "id": "Xsnj0T3mIgxn" 570 | }, 571 | "source": [ 572 | "### 4.3 - Forward and Backward propagation\n", 573 | "\n", 574 | "Now that your parameters are initialized, you can do the \"forward\" and \"backward\" propagation steps for learning the parameters.\n", 575 | "\n", 576 | "**Exercise:** Implement a function `propagate()` that computes the cost function and its gradient.\n", 577 | "\n", 578 | "**Hints**:\n", 579 | "\n", 580 | "Forward Propagation:\n", 581 | "- You get X\n", 582 | "- You compute $A = \\sigma(w^T X + b) = (a^{(1)}, a^{(2)}, ..., a^{(m-1)}, a^{(m)})$\n", 583 | "- You calculate the cost function: $J = -\\frac{1}{m}\\sum_{i=1}^{m}y^{(i)}\\log(a^{(i)})+(1-y^{(i)})\\log(1-a^{(i)})$\n", 584 | "\n", 585 | "Here are the two formulas you will be using: \n", 586 | "\n", 587 | "$$ \\frac{\\partial J}{\\partial w} = \\frac{1}{m}X(A-Y)^T\\tag{5}$$\n", 588 | "$$ \\frac{\\partial J}{\\partial b} = \\frac{1}{m} \\sum_{i=1}^m (a^{(i)}-y^{(i)})\\tag{6}$$" 589 | ] 590 | }, 591 | { 592 | "cell_type": "code", 593 | "execution_count": 115, 594 | "metadata": { 595 | "colab": { 596 | "autoexec": { 597 | "startup": false, 598 | "wait_interval": 0 599 | } 600 | }, 601 | "colab_type": "code", 602 | "id": "gNoCaKL-Igxn" 603 | }, 604 | "outputs": [], 605 | "source": [ 606 | "# GRADED FUNCTION: propagate\n", 607 | "\n", 608 | "def propagate(w, b, X, Y):\n", 609 | " \"\"\"\n", 610 | " Implement the cost function and its gradient for the propagation explained above\n", 611 | "\n", 612 | " Arguments:\n", 613 | " w -- weights, a numpy array of size (number of features, 1)\n", 614 | " b -- bias, a scalar\n", 615 | " X -- data of size (number of features, number of examples)\n", 616 | " Y -- true \"label\" vector (containing 0 if non-cat, 1 if cat) of size (1, number of examples)\n", 617 | "\n", 618 | " Return:\n", 619 | " cost -- negative log-likelihood cost for logistic regression\n", 620 | " dw -- gradient of the loss with respect to w, thus same shape as w\n", 621 | " db -- gradient of the loss with respect to b, thus same shape as b\n", 622 | " \n", 623 | " Tips:\n", 624 | " - Write your code step by step for the propagation. np.log(), np.dot()\n", 625 | " \"\"\"\n", 626 | " \n", 627 | " m = X.shape[1]\n", 628 | " # FORWARD PROPAGATION (FROM X TO COST)\n", 629 | " ### START CODE HERE ### (≈ 2 lines of code)\n", 630 | " A = sigmoid(np.dot(w.transpose(), X) + b) # compute activation\n", 631 | " cost = sum(-1 / (m) * sum(Y * np.log(A) + (1 - Y) * (np.log(1 - A)))) # compute cost\n", 632 | " ### END CODE HERE ###\n", 633 | "\n", 634 | " # BACKWARD PROPAGATION (TO FIND GRAD)\n", 635 | " ### START CODE HERE ### (≈ 2 lines of code)\n", 636 | " dw = (1 / m) * np.dot(X, (A - Y).transpose())\n", 637 | " db = sum(sum(A - Y) * 1 / m)\n", 638 | " ### END CODE HERE ###\n", 639 | " \n", 640 | " assert(dw.shape == w.shape)\n", 641 | " assert(db.dtype == float)\n", 642 | " cost = np.squeeze(cost)\n", 643 | "# assert(cost.shape == ())\n", 644 | " \n", 645 | " grads = {\"dw\": dw,\n", 646 | " \"db\": db}\n", 647 | " \n", 648 | " return grads, cost" 649 | ] 650 | }, 651 | { 652 | "cell_type": "code", 653 | "execution_count": 116, 654 | "metadata": { 655 | "colab": { 656 | "autoexec": { 657 | "startup": false, 658 | "wait_interval": 0 659 | } 660 | }, 661 | "colab_type": "code", 662 | "id": "pap_G48zIgxq" 663 | }, 664 | "outputs": [ 665 | { 666 | "name": "stdout", 667 | "output_type": "stream", 668 | "text": [ 669 | "dw = [[0.99845601]\n", 670 | " [2.39507239]]\n", 671 | "db = 0.0014555781367842635\n", 672 | "cost = 5.8015453193945525\n" 673 | ] 674 | } 675 | ], 676 | "source": [ 677 | "w, b, X, Y = np.array([[1.],[2.]]), 2., np.array([[1.,2.,-1.],[3.,4.,-3.2]]), np.array([[1,0,1]])\n", 678 | "grads, cost = propagate(w, b, X, Y)\n", 679 | "print (\"dw = \" + str(grads[\"dw\"]))\n", 680 | "print (\"db = \" + str(grads[\"db\"]))\n", 681 | "print (\"cost = \" + str(cost))" 682 | ] 683 | }, 684 | { 685 | "cell_type": "markdown", 686 | "metadata": { 687 | "colab_type": "text", 688 | "id": "wricPbBUIgxt" 689 | }, 690 | "source": [ 691 | "**Expected Output**:\n", 692 | "\n", 693 | "\n", 694 | " \n", 695 | " \n", 696 | " \n", 698 | " \n", 699 | " \n", 700 | " \n", 701 | " \n", 702 | " \n", 703 | " \n", 704 | " \n", 705 | " \n", 706 | " \n", 707 | "\n", 708 | "
dw [[ 0.99845601]
\n", 697 | " [ 2.39507239]]
db 0.00145557813678
cost 5.801545319394553
" 709 | ] 710 | }, 711 | { 712 | "cell_type": "markdown", 713 | "metadata": { 714 | "colab_type": "text", 715 | "id": "oqQ9az4RIgxu" 716 | }, 717 | "source": [ 718 | "### 4.4 - Optimization\n", 719 | "- You have initialized your parameters.\n", 720 | "- You are also able to compute a cost function and its gradient.\n", 721 | "- Now, you want to update the parameters using gradient descent.\n", 722 | "\n", 723 | "**Exercise:** Write down the optimization function. The goal is to learn $w$ and $b$ by minimizing the cost function $J$. For a parameter $\\theta$, the update rule is $ \\theta = \\theta - \\alpha \\text{ } \\partial\\theta$, where $\\alpha$ is the learning rate." 724 | ] 725 | }, 726 | { 727 | "cell_type": "code", 728 | "execution_count": 117, 729 | "metadata": { 730 | "colab": { 731 | "autoexec": { 732 | "startup": false, 733 | "wait_interval": 0 734 | } 735 | }, 736 | "colab_type": "code", 737 | "id": "Hmcks9o6Igxu" 738 | }, 739 | "outputs": [], 740 | "source": [ 741 | "# GRADED FUNCTION: optimize\n", 742 | "\n", 743 | "def optimize(w, b, X, Y, num_iterations, learning_rate, print_cost=False):\n", 744 | " \"\"\"\n", 745 | " This function optimizes w and b by running a gradient descent algorithm\n", 746 | " \n", 747 | " Arguments:\n", 748 | " w -- weights, a numpy array of size (number of features, 1)\n", 749 | " b -- bias, a scalar\n", 750 | " X -- data of shape (number of features, number of examples)\n", 751 | " Y -- true \"label\" vector (containing 0 if non-cat, 1 if cat), of shape (1, number of examples)\n", 752 | " num_iterations -- number of iterations of the optimization loop\n", 753 | " learning_rate -- learning rate of the gradient descent update rule\n", 754 | " print_cost -- True to print the loss every 100 steps\n", 755 | " \n", 756 | " Returns:\n", 757 | " params -- dictionary containing the weights w and bias b\n", 758 | " grads -- dictionary containing the gradients of the weights and bias with respect to the cost function\n", 759 | " costs -- list of all the costs computed during the optimization, this will be used to plot the learning curve.\n", 760 | " \n", 761 | " Tips:\n", 762 | " You basically need to write down two steps and iterate through them:\n", 763 | " 1) Calculate the cost and the gradient for the current parameters. Use propagate().\n", 764 | " 2) Update the parameters using gradient descent rule for w and b.\n", 765 | " \"\"\"\n", 766 | " \n", 767 | " costs = []\n", 768 | " \n", 769 | " for i in range(num_iterations):\n", 770 | " \n", 771 | " \n", 772 | " # Cost and gradient calculation (≈ 1 line of code)\n", 773 | " ### START CODE HERE ### \n", 774 | " grads, cost = propagate(w, b, X, Y)\n", 775 | " ### END CODE HERE ###\n", 776 | " \n", 777 | " # Retrieve derivatives from grads\n", 778 | " dw = grads[\"dw\"]\n", 779 | " db = grads[\"db\"]\n", 780 | " \n", 781 | " # update rule (≈ 2 lines of code)\n", 782 | " ### START CODE HERE ###\n", 783 | " w = w - learning_rate * dw\n", 784 | " b = b - learning_rate * db\n", 785 | " ### END CODE HERE ###\n", 786 | " \n", 787 | " # Record the costs\n", 788 | " if i % 100 == 0:\n", 789 | " costs.append(cost)\n", 790 | " \n", 791 | " # Print the cost every 100 training iterations\n", 792 | " if print_cost and i % 100 == 0:\n", 793 | " print (\"Cost after iteration %i: %f\" %(i, cost))\n", 794 | " \n", 795 | " params = {\"w\": w,\n", 796 | " \"b\": b}\n", 797 | " \n", 798 | " grads = {\"dw\": dw,\n", 799 | " \"db\": db}\n", 800 | " \n", 801 | " return params, grads, costs" 802 | ] 803 | }, 804 | { 805 | "cell_type": "code", 806 | "execution_count": 118, 807 | "metadata": { 808 | "colab": { 809 | "autoexec": { 810 | "startup": false, 811 | "wait_interval": 0 812 | } 813 | }, 814 | "colab_type": "code", 815 | "id": "rsM2ZmsJIgxx" 816 | }, 817 | "outputs": [ 818 | { 819 | "name": "stdout", 820 | "output_type": "stream", 821 | "text": [ 822 | "w = [[0.19033591]\n", 823 | " [0.12259159]]\n", 824 | "b = 1.9253598300845747\n", 825 | "dw = [[0.67752042]\n", 826 | " [1.41625495]]\n", 827 | "db = 0.21919450454067657\n" 828 | ] 829 | } 830 | ], 831 | "source": [ 832 | "params, grads, costs = optimize(w, b, X, Y, num_iterations=100, learning_rate=0.009, print_cost=False)\n", 833 | "\n", 834 | "print (\"w = \" + str(params[\"w\"]))\n", 835 | "print (\"b = \" + str(params[\"b\"]))\n", 836 | "print (\"dw = \" + str(grads[\"dw\"]))\n", 837 | "print (\"db = \" + str(grads[\"db\"]))" 838 | ] 839 | }, 840 | { 841 | "cell_type": "markdown", 842 | "metadata": { 843 | "colab_type": "text", 844 | "id": "e5NToBRrIgxy" 845 | }, 846 | "source": [ 847 | "**Expected Output**: \n", 848 | "\n", 849 | "\n", 850 | " \n", 851 | " \n", 852 | " \n", 854 | " \n", 855 | " \n", 856 | " \n", 857 | " \n", 858 | " \n", 859 | " \n", 860 | " \n", 861 | " \n", 863 | " \n", 864 | " \n", 865 | " \n", 866 | " \n", 867 | " \n", 868 | "\n", 869 | "
w[[ 0.19033591]\n", 853 | " [ 0.12259159]]
b 1.92535983008
d [[ 0.67752042]\n", 862 | " [ 1.41625495]]
db 0.219194504541
" 870 | ] 871 | }, 872 | { 873 | "cell_type": "markdown", 874 | "metadata": { 875 | "colab_type": "text", 876 | "id": "umCJBHMOIgxz" 877 | }, 878 | "source": [ 879 | "**Exercise:** The previous function will output the learned w and b. We are able to use w and b to predict the labels for a dataset X. Implement the `predict()` function. There are two steps to computing predictions:\n", 880 | "\n", 881 | "1. Calculate $\\hat{Y} = A = \\sigma(w^T X + b)$\n", 882 | "\n", 883 | "2. Convert the entries of $A$ into 0 (if $\\sigma(z)$ <= 0.5) or 1 (if $\\sigma(z)$ > 0.5), stores the predictions in a vector `Y_prediction`. If you wish, you can use an `if`/`else` statement in a `for` loop (though there is also a way to vectorize this). " 884 | ] 885 | }, 886 | { 887 | "cell_type": "code", 888 | "execution_count": 121, 889 | "metadata": { 890 | "colab": { 891 | "autoexec": { 892 | "startup": false, 893 | "wait_interval": 0 894 | } 895 | }, 896 | "colab_type": "code", 897 | "id": "uJNjSBerIgx0" 898 | }, 899 | "outputs": [], 900 | "source": [ 901 | "# GRADED FUNCTION: predict\n", 902 | "\n", 903 | "def predict(w, b, X):\n", 904 | " '''\n", 905 | " Predict whether the label is 0 or 1 using learned logistic regression parameters (w, b)\n", 906 | " \n", 907 | " Arguments:\n", 908 | " w -- weights, a numpy array of size (number of features, 1)\n", 909 | " b -- bias, a scalar\n", 910 | " X -- data of size (number of features, number of examples)\n", 911 | " \n", 912 | " Returns:\n", 913 | " Y_prediction -- a numpy array (vector) containing all predictions (0/1) for the examples in X\n", 914 | " '''\n", 915 | " \n", 916 | " m = X.shape[1]\n", 917 | " Y_prediction = np.zeros((1,m))\n", 918 | " \n", 919 | " # Compute vector \"A\" predicting the probabilities of a cat being present in the picture\n", 920 | " ### START CODE HERE ### (≈ 1 line of code)\n", 921 | " A = sigmoid(np.dot(w.transpose(), X) + b)\n", 922 | " ### END CODE HERE ###\n", 923 | " \n", 924 | " # Convert probabilities A[0,i] to actual predictions p[0,i]\n", 925 | " ### START CODE HERE ### (≈ 4 lines of code, but can be done in one line using numpy)\n", 926 | " ### Variable with probabilities called Y_prediction\n", 927 | " Y_prediction = np.where(A<=0.5, 0, 1)\n", 928 | " ### END CODE HERE ###\n", 929 | " \n", 930 | " assert(Y_prediction.shape == (1, m))\n", 931 | " \n", 932 | " return Y_prediction" 933 | ] 934 | }, 935 | { 936 | "cell_type": "code", 937 | "execution_count": 122, 938 | "metadata": { 939 | "colab": { 940 | "autoexec": { 941 | "startup": false, 942 | "wait_interval": 0 943 | } 944 | }, 945 | "colab_type": "code", 946 | "id": "ClYaM7C4Igx1" 947 | }, 948 | "outputs": [ 949 | { 950 | "name": "stdout", 951 | "output_type": "stream", 952 | "text": [ 953 | "predictions = [[1 1 0]]\n" 954 | ] 955 | } 956 | ], 957 | "source": [ 958 | "w = np.array([[0.1124579],[0.23106775]])\n", 959 | "b = -0.3\n", 960 | "X = np.array([[1.,-1.1,-3.2],[1.2,2.,0.1]])\n", 961 | "print (\"predictions = \" + str(predict(w, b, X)))" 962 | ] 963 | }, 964 | { 965 | "cell_type": "markdown", 966 | "metadata": { 967 | "colab_type": "text", 968 | "id": "BUT39WZxIgx3" 969 | }, 970 | "source": [ 971 | "**Expected Output**: \n", 972 | "\n", 973 | "\n", 974 | " \n", 975 | " \n", 978 | " \n", 981 | " \n", 982 | "\n", 983 | "
\n", 976 | " predictions\n", 977 | " \n", 979 | " [[ 1 1 0]]\n", 980 | "
\n" 984 | ] 985 | }, 986 | { 987 | "cell_type": "markdown", 988 | "metadata": { 989 | "colab_type": "text", 990 | "id": "sb7g-flaIgx4" 991 | }, 992 | "source": [ 993 | "\n", 994 | " What to remember:\n", 995 | "You've implemented several functions that:\n", 996 | "- Initialize (w,b)\n", 997 | "- Optimize the loss iteratively to learn parameters (w,b):\n", 998 | " - computing the cost and its gradient \n", 999 | " - updating the parameters using gradient descent\n", 1000 | "- Use the learned (w,b) to predict the labels for a given set of examples" 1001 | ] 1002 | }, 1003 | { 1004 | "cell_type": "markdown", 1005 | "metadata": { 1006 | "colab_type": "text", 1007 | "id": "tvX5qoSZIgx6" 1008 | }, 1009 | "source": [ 1010 | "## 5 - Merge all functions into a model ##\n", 1011 | "\n", 1012 | "You will now see how the overall model is structured by putting together all the building blocks (functions implemented in the previous parts) together, in the right order.\n", 1013 | "\n", 1014 | "**Exercise:** Implement the model function. Use the following notation:\n", 1015 | " - Y_prediction_test for your predictions on the test set\n", 1016 | " - Y_prediction_train for your predictions on the train set\n", 1017 | " - w, costs, grads for the outputs of optimize()" 1018 | ] 1019 | }, 1020 | { 1021 | "cell_type": "code", 1022 | "execution_count": 128, 1023 | "metadata": { 1024 | "colab": { 1025 | "autoexec": { 1026 | "startup": false, 1027 | "wait_interval": 0 1028 | } 1029 | }, 1030 | "colab_type": "code", 1031 | "id": "RWkqw8icIgx6" 1032 | }, 1033 | "outputs": [], 1034 | "source": [ 1035 | "# GRADED FUNCTION: model\n", 1036 | "\n", 1037 | "def model(X_train, Y_train, X_test, Y_test, num_iterations=2000, learning_rate=0.5, print_cost=False):\n", 1038 | " \"\"\"\n", 1039 | " Builds the logistic regression model by calling the function you've implemented previously\n", 1040 | " \n", 1041 | " Arguments:\n", 1042 | " X_train -- training set represented by a numpy array of shape (number of features, m_train)\n", 1043 | " Y_train -- training labels represented by a numpy array (vector) of shape (1, m_train)\n", 1044 | " X_test -- test set represented by a numpy array of shape (number of features, m_test)\n", 1045 | " Y_test -- test labels represented by a numpy array (vector) of shape (1, m_test)\n", 1046 | " num_iterations -- hyperparameter representing the number of iterations to optimize the parameters\n", 1047 | " learning_rate -- hyperparameter representing the learning rate used in the update rule of optimize()\n", 1048 | " print_cost -- Set to true to print the cost every 100 iterations\n", 1049 | " \n", 1050 | " Returns:\n", 1051 | " d -- dictionary containing information about the model.\n", 1052 | " \"\"\"\n", 1053 | " \n", 1054 | " ### START CODE HERE ###\n", 1055 | " \n", 1056 | " # initialize parameters with zeros (≈ 1 line of code)\n", 1057 | " w, b = initialize_with_zeros(X_train.shape[0])\n", 1058 | "\n", 1059 | " # Gradient descent (≈ 1 line of code)\n", 1060 | " parameters, grads, costs = optimize(w, b, X_train, Y_train, num_iterations, learning_rate, print_cost=False)\n", 1061 | " \n", 1062 | " # Retrieve parameters w and b from dictionary \"parameters\"\n", 1063 | " w = parameters[\"w\"]\n", 1064 | " b = parameters[\"b\"]\n", 1065 | " \n", 1066 | " # Predict test/train set examples (≈ 2 lines of code)\n", 1067 | " Y_prediction_test = predict(w, b, X_test)\n", 1068 | " Y_prediction_train = predict(w, b, X_train)\n", 1069 | "\n", 1070 | " ### END CODE HERE ###\n", 1071 | "\n", 1072 | " # Print train/test Errors\n", 1073 | " print(\"train accuracy: {} %\".format(100 - np.mean(np.abs(Y_prediction_train - Y_train)) * 100))\n", 1074 | " print(\"test accuracy: {} %\".format(100 - np.mean(np.abs(Y_prediction_test - Y_test)) * 100))\n", 1075 | "\n", 1076 | " \n", 1077 | " d = {\"costs\": costs,\n", 1078 | " \"Y_prediction_test\": Y_prediction_test, \n", 1079 | " \"Y_prediction_train\" : Y_prediction_train, \n", 1080 | " \"w\" : w, \n", 1081 | " \"b\" : b,\n", 1082 | " \"learning_rate\" : learning_rate,\n", 1083 | " \"num_iterations\": num_iterations}\n", 1084 | " \n", 1085 | " return d" 1086 | ] 1087 | }, 1088 | { 1089 | "cell_type": "markdown", 1090 | "metadata": { 1091 | "colab_type": "text", 1092 | "id": "9AFjdI_3Igx9" 1093 | }, 1094 | "source": [ 1095 | "Run the following cell to train your model." 1096 | ] 1097 | }, 1098 | { 1099 | "cell_type": "code", 1100 | "execution_count": 139, 1101 | "metadata": { 1102 | "colab": { 1103 | "autoexec": { 1104 | "startup": false, 1105 | "wait_interval": 0 1106 | } 1107 | }, 1108 | "colab_type": "code", 1109 | "id": "L_CByy5GIgx9" 1110 | }, 1111 | "outputs": [ 1112 | { 1113 | "name": "stdout", 1114 | "output_type": "stream", 1115 | "text": [ 1116 | "train accuracy: 99.04306220095694 %\n", 1117 | "test accuracy: 70.0 %\n" 1118 | ] 1119 | } 1120 | ], 1121 | "source": [ 1122 | "d = model(train_set_x, train_set_y, test_set_x, test_set_y, num_iterations=2000, learning_rate=0.005, print_cost=True)" 1123 | ] 1124 | }, 1125 | { 1126 | "cell_type": "markdown", 1127 | "metadata": { 1128 | "colab_type": "text", 1129 | "id": "j5knJ1HiIgyA" 1130 | }, 1131 | "source": [ 1132 | "**Expected Output**: \n", 1133 | "\n", 1134 | " \n", 1135 | " \n", 1136 | " \n", 1137 | " \n", 1138 | " \n", 1139 | " \n", 1140 | " \n", 1141 | " \n", 1142 | " \n", 1143 | " \n", 1144 | " \n", 1145 | " \n", 1146 | " \n", 1147 | " \n", 1148 | " \n", 1149 | " \n", 1150 | " \n", 1151 | "
Cost after iteration 0 0.693147
$\\vdots$
$\\vdots$
Train Accuracy 99.04306220095694 %
Test Accuracy 70.0 %
\n", 1152 | "\n", 1153 | "\n" 1154 | ] 1155 | }, 1156 | { 1157 | "cell_type": "markdown", 1158 | "metadata": { 1159 | "colab_type": "text", 1160 | "id": "eAGGv-UOIgyA" 1161 | }, 1162 | "source": [ 1163 | "**Comment**: Training accuracy is close to 100%. This is a good sanity check: your model is working and has high enough capacity to fit the training data. Test accuracy is 70%. It is actually not bad for this simple model, given the small dataset we used and that logistic regression is a linear classifier.\n", 1164 | "\n", 1165 | "Also, you see that the model is clearly overfitting the training data. Later you will learn how to reduce overfitting, for example by using regularization. Using the code below (and changing the `index` variable) you can look at predictions on pictures of the test set." 1166 | ] 1167 | }, 1168 | { 1169 | "cell_type": "code", 1170 | "execution_count": 140, 1171 | "metadata": { 1172 | "colab": { 1173 | "autoexec": { 1174 | "startup": false, 1175 | "wait_interval": 0 1176 | } 1177 | }, 1178 | "colab_type": "code", 1179 | "id": "6F96ZtEaIgyB" 1180 | }, 1181 | "outputs": [ 1182 | { 1183 | "name": "stdout", 1184 | "output_type": "stream", 1185 | "text": [ 1186 | "y = 1, you predicted that it is a \"cat\" picture.\n" 1187 | ] 1188 | }, 1189 | { 1190 | "data": { 1191 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD7CAYAAACscuKmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO29aYxk2XUe+J2IF3vumbVXdVV3s9kLW1STbFNNkZZbpBZKMkRgQBmSjQE105ieH5qBhLFhkjYwkAdjQPpjyQMMBDSshSPIIiVZEgnKkk21uFii3c1qks3e9+pasqqyqnLP2OPd+ZGRcb9zIiMqu6sqss04H5DI++LeuO+++96Nd84953xHQghwOBzf/8js9wAcDsdo4Ivd4RgT+GJ3OMYEvtgdjjGBL3aHY0zgi93hGBPc0GIXkY+LyEsi8qqIfOZmDcrhcNx8yNu1s4tIFsDLAH4cwHkA3wLwCyGE52/e8BwOx81CcgPf/SCAV0MIrwOAiHwewCcADFzs8/Nz4cTJY+i2V3UZEjIazaaqO/PGmVjXiHV9fUiG6oYNnSv3/mPHLUNIB/Yw7NTDfluFxl8qFHrlickp1e7AgQO9ciYz9Gx7HIdul3Y6vfJWdatXbrXaqt3yymqv3G7pe6buDZULOf3IJUm85pAOHi931+nodp003osko4VVGTA/9iXXpmu2dzBLfdr5Tqkf7rPdTlW7LH0vyQ1edkH1N7DZwO9tVetoNJu7XvSNLPZjAM7R8XkAPzTsCydOHsPjX/8iACCX0zcln53slc+8eVbVPfKL/3Ov/Mor8ZS5JKvalYrFWJc1N51/COhmvhXJhm9svVmLn0PfWH4w7cPRaND5gq4r0vjvf9cdvfJHHv5J1e5/+V8f7ZUr5aKqUz9CnRaPanC7VC/irc3NXvnJJ7/ZK1+8clW1+8Iff6lXvnzxvKor5unRyuV7xduPLqh2B6YrvXKjrn8weO54Da9t1VW7ja14L+Yqej7yhVyvLNRJu9VS7a6sbOzaDgAmK+VeuVQsqLp6o9Ert9rxB+PKlQ3Vbnqi1CsvHNZzEOjHin902m19XzISn/d26Ki6dmv7+K+/8SQG4UZ09t1+PfpWjog8KiKnReT0tavLN3A6h8NxI7iRN/t5ACfo+DiARdsohPAYgMcA4Pix+fDv/p9fBQBMzUyrdpOTUTS9RuIhANRIlOSfk4z5BdbH5rdogFxvf506nfgr2zS//kq9kPhNK6VoCfZtxh5QJ1kjweQK8bYlhbyq4+8J9FtIY/BcZQrxbZvLxz6slJLJstpk7wWJrSSqd1L9RmrRm8xe51Y9vjWr9Na3qsBBUnnSrL3vVKS5CeZ5SHKxj8mJsqorFmmOze3MgMZF4zhxW0W1S7Lx2oZJk9kMzYF5rtqkRol5T+e7ba1qq8f69vEtAHeJyO0ikgfw8wC+dJ3vOByOfcLbfrOHENoi8r8B+E8AsgB+J4Tw3E0bmcPhuKm4ETEeIYT/COA/3qSxOByOW4gbWuxvFRkBSrltHS3b0Tuq9ZW4y751dV3Vsa7ydv0COqQbst5Ya+gd4GYz6unWFMRHWk/X7YbpTbqtaUfXxn1sbuhd8OeeiNrSxMSEqkuSuPvMJqNcQe9SSybe+kxG68pN2lVutao0JqOXK1OnMVfRYaK+ptvVWvFcVbPLXsjFcc1NRT06TbX1o0b3LBf0tag9DDYBFkqq2aFC7N/q84xg7nWRdup5vyfJ5VQ7fm5T81ylYffzWWtTLon3zK6DtHvuW6WzOxyO/47gi93hGBOMVIyfmpvFj/38JwH0m1mERJszr15RdX/yF9/tlVnisRJ9i0xlVePtxaI7m0Haph2L7n2iKZmakmR3hw/ACqqDj/qkN742ZbLTF5onUT1vzY/UVtgrrNlQ7UKIInPLONXUyeTVIceOPo9FZXpTVQjqRsVys6VNb5VCrJs2DjEpOf6sb8bxblS1uM9ecweM2SxD6spexXP7XKlv2TnIsFmRzLZmTrOkXlh1SDLxhAmti6xpx89mx6gy6PYxTIP0N7vDMSbwxe5wjAl8sTscY4KR6uydTgdrG2sAgEJBuxNOVKJLYierdTIGq7INYzarN4boXaTLsOnDmjDYNTJvzCe5bJyuZod0YNF6aFDmHj0ObXizUXv0PWpYnpxT7d79gZ/qlUslraMOPtuQVkHrf41aDCxZXH+8V17d1Hp/lvY+rBWRg4Z4H2GiaNx7qW5pRQePsHvyZCnq87NT+tlRUWkmAEpFkfF9NzeGH4OsDaJSVlZdp4KgyEU4Z/vH7iZAQLsWc11fhB310f+W3rkAN705HGMPX+wOx5hgpGL8+vIKHv/3fw4AqBhRrFSOEUOXjQedJU3YgRXB2drTZ9ViEZk8xoomgmqC4sMtEQKbZ9oN/lxDi2XW62wwSQITLfDXOsbbcG0tehumnRnTR/xiNolzmmS1+CzCpiAzDjapZQab19jsZFUSNifxDK1tVlU79h4LJiJunp6RQj6qVJnEiNk0sIaJVGSVLSEPNDFegyqa0jrhDRGN6414Pp6fXF4/V4HE/2DnaoDnZMc8352Onh/Vx45K5aY3h8Phi93hGBOMVIyvTE7ggz/yIQBAo7mp6hKJQ0nlgqob6BVkPudAgXyiLy1H4jq3S8zOKxMjWDWBvfBY3MpmtIjMBARWBGxkovzf5wTFgSXUR21TB8I8/18/3ytXyjoQhnf082TxKJYmVbtsbrCI3ySOt3pVE4moc7GaM2SHmUXwiiHbmCS6JvvmGeTV1jHzJtRuw1gM8iROs/Eg2Mk3Z2YwHVTHWC6urUZilcmJeG2ForbkZOnqgnGdVEeKnMV4cOaZTk2PeMf6MUzl8De7wzEm8MXucIwJfLE7HGOCkersuVIeR+/f5o1nPQ7QJp7mlImIYyYE+nkqGA+3MhH+Ke8uaD06S/318YxzsNYQRnjWr/u5xKNub73TNPHC3n5rJ6c19fC97/+JXtl6+aUp8erTeLOJIa8QJq/Qj0GDzEnZxWfpO9oEqEyFRlWcLEfPvkML871ysWDve/ziFlFCA9CKqdKb9bwVc7GPSlmTbGbUfeeoRWMSJe/IzBDvOkuc0WnHyiz1IWaMlp6awd6GeSLssPtJ7AFox7EXknl/szscYwJf7A7HmGCkYny9Wsfz334JAHDwqDYF5UpRdFq5qk1Ng+I5smLE/QF8Y9vHu3fYNl5bSAebN/h7LHqlbd0He21ZEV9JptYjTY0/NiwY89rC8btjXV57Imq1gb3kbADKYDQpACX/8pnYx6r2bGTTm52qHIujRPRhVSNWh9gbDQDyRORQJLNTn5mPHoOCDFYT0g6b8gZnyLGccEodMuL48cOzsQ89KtMHlc19Z8KKHBGT9FvR2NPOqCE7Yr1z0DkcDl/sDseYwBe7wzEmGKnO3qrWcfGZbZ19a1GTLnBE0oVzWmdP2T+Sk6C+BQ55dnVlnczqYJxa1/4U8ukUEYI5F+tTeWNi1IQbQ6LeSPfa2tQEnM8//eVe2ZJXMEkh5y9LcpaIMZrisomNiIsurM1mjFKz+jZfp+VC57TKvL9RMOYknu5KSc9VhnXl7GC9n8/Vl7uvvfuzk0Dv93DKcJvmWV+bPkMhH/tpqDTNg3MN9hOr0DxS/63W4L0gyym/87wMy1hw3Te7iPyOiCyJyLP02ZyIfEVEXun+nx3Wh8Ph2H/sRYz/PQAfN599BsDjIYS7ADzePXY4HO9gXFeMDyF8Q0ROmY8/AeDhbvlzAL4G4NPX66s8M4EP/MyHAQA5k02YzRvN77ym6zLRi4sloFZHk1okHTZbaA8j9kay5jBGOkQ8TwfYAI2THHLkJfaBBx5UdZubMdrv2edfUnXadEjjNebBdCPmua9v6Zz3oRPNV4pcIqtvtYqOMmml0zSOv7bJuotJc5wMJsBQXHthUI1WxWwq5nSAx1jLqAzKNGbUBI5wZK9K60HHuQSs9apB4nSr3Rdyt1txF/IUPtDn7qhnLvZv+2DR3XqIDo/i28bb3aA7FEK4CADd/wffZj8Oh2NEuOW78SLyqIicFpHTqyub1/+Cw+G4JXi7u/GXReRICOGiiBwBsDSoYQjhMQCPAcC9P3AqJOVtj69cSYumOeJLq8xprzCWelgcytjUR0N+utq0o59R7L+WdIHEOSPuM1eYPrcWK0vFuPN93913qbpiMU552QSFPPfyq/FcJI5OHziq2t33of+BBqXptNM0ivFJJu6qi1jKadqBN6mKGo0YkHJ+9au98lZNUz0fO3ygVz73htntV/cplq24qUkprDrBQSbx86IhJuF70RcAld394em/74OFcJHBInKT+BHZaNT/LO5uWQCAPKkvrA71pasaYonqWZhuAQfdlwB8qlv+FIAvvs1+HA7HiLAX09sfAvivAO4WkfMi8giAXwPw4yLyCoAf7x47HI53MPayG/8LA6o+dpPH4nA4biFG6kFX26ji2W98BwAwOa+9toqlqF9efvOiqrOmrR0Yi5HytBvGG48h0VqKFKAvMiqWWb+0ljw+tJ5lhdJ0r3zP3XeoulotequpNECm/04gHVt0pFgQqlPEDYa8gjzoxJrDiECTo+/SpuZ8v/vOE73ymddeV3Wba9ELkvdL8mafIpDOu2rIK5oUTXh8Ic5bzujsQUWDDU6phSFRkYP62+4jltuG7XLxykqvPDMd95pyE+Y6h6TiapM5L8kPNgGqFFKmj7i/5FFvDsfYwxe7wzEmGKkYn81kMFncFh9nyyVdx95B7cEiD0OMCYPlrbSPWDsW2+0oOlrznU4TZQkIOCNo7NAm5el0dk9XBQCVialeeWNNmxjffcfJXpkzmq4tax79Z/7uj3tla8YBB0vkiZMvr8V45lyz6YiazXi8dCGmmqrWjVmLzKV5k+6IVaqg0hsNHC42q82BdZrYQ/cxjLCBUyglKpWV/o7yjOvrbrD4zFyKTNjR13JAEBUAdMi0l2PCkT4TIAcDmRF27dM3FAjjcDi+P+CL3eEYE/hidzjGBCPV2fOlIk7efy8A4Mixw6ouU4hayBWTdneQmaRjiB45Usx+Reh3LSODf+PYbXJYdBz3P8zN05IMlMtRT5+ePaDqtjbWeuUjh6L568AB3e6Oe+/vla3bZKse4w8K+ZjOOWPy0bFb7cbaiqp75ttP98qXl2K7+YPHVbs2EVNubOi4h7aKWKNyx2ibNMfD5pFNVy2Tupgj1uxWTYf2VtiVdri7rOmDOrVRdUcPxTlukt7f9/xxPoJgTXv0zGHwvkJI9/Y8DoK/2R2OMYEvdodjTDBSMT6kQLsr+SVBi6Zox1S7ncbgyCjNFrB3fm/lkcZpnMzPHUtzWgDXp2uoQVnxk8gOWtqcVCxGE9jUtGbzulKIkWNNIqhYunxZtTv9XLxtzYY28+UorVO5Ej3eZmfnVbt6Pc7xy89qspCzZ6PZb2omqlvFojYVblJ03MamFuOVEYpNRkZUV1PXR84Wi4rdzTZTaY51HZsYh3nNMRGHtWt10sGmWub8K5LpLWtThiesQthnM7NruT/6bjCRyB4c6PzN7nCMC3yxOxxjgpGK8Y1qDa899T0AQH3Z8F1koqh0+VXtMcbinQzwqgI0DXR/5hzOu8R9WLGM6oZQSbO4n2S0wM98d01tWIBK4WP50kiMb1yJ83PuDT1Xr70aj20W1xbtkBdLUWVIEstZFlWNxMit5cm4wzxD4n9lQqehunb5bK+8aYJYJotxXCr7qI1qokktGdrtQDebxefE8q8N8dAbxDHS74HGpBF9lb2i3Y1PlLhO3nq53amedxujps0YLKqrLxpdZlCwGMPf7A7HmMAXu8MxJvDF7nCMCUaqsxfLJdzz/vcCABaOajNOJ0Rd85xJ2TxI30kteSF/x6YqUmVFL6H7CLolg/W16cmYcrpY0CT4pULUldeNd1p1I6Y9tpYm1v+Kldj/+oZOlZzPx4hBJv0AgHYavfCQieNq2bTSzIUedF2xQh5jaqr0BsTS1Xht9YY2MZYL8Vrq5BEpYu8Le9Dpe9FKd4+cs+Aaq7pqE6A6sR6HhIGV/PzZPQdOM83Ph9l+0MQZfe/YAfayvsC5vac72w3+Znc4xgS+2B2OMcFoySuSBFNzCwCAmTltxmk14lBmp7UYnyczRqUUAzqsNxNzk1nzDJM8cLu8EcHZ08mqAiymsbmtUDB8evl4XNtcU3VXL5+P5yoYrnU63wSpCS3De5aj3Fk2GChPdbmEiRW0uB/SqGpkjGjK5rwtUiEaVa1OnDsfTaQ2FRebQbMkwloTIBNKNM21qMCYIRKsskgNcyEbkoZqGEEFn8By0K1uRpMjP5tFw7U3zOw3yEM09KmYHBikR7mj0g7LAuVvdodjTOCL3eEYE/hidzjGBCPV2dvNgGsXtk0VE0Wjs7eiDnJt6ZqqK1J63eOHY8LYRqOu2rH+l7V52gbkiEuSwZFF/QQH5A6ZJVKEVI8DrHsafZjNhUx8adsy0UKlos2U5VI8tn00yT+XI69yJh0ynyuliEPblvO+bTW1S+zlpei22zGEEqxf6n2QPl/UXinpi1RkEshhxA2DCR8GnMpEl1mT62ByCZsH4PK1uI9RKkadvWVuLecQtPp2GNDO5hoMGGI77M63NV+qMQys2elS5ISIfFVEXhCR50Tkl7ufz4nIV0Tkle7/2ev15XA49g97EePbAP5pCOFeAA8B+CURuQ/AZwA8HkK4C8Dj3WOHw/EOxV5yvV0EcLFb3hCRFwAcA/AJAA93m30OwNcAfHpoX2lAu7ot7l06d17Vfe53/79e+Y3Xz6k6kp4xNR3F//U1E1LGKX6HifFkhrPUY1yXZHOmLrtrOZ/XpjfOMd00pprlldXYf06b/bgfTgUMI3KyqG7F81KJRHfqz5J5VKuRbCKXM/x0JOK3SE2wRBlbFOlmxVue1gKlNMolRnwmTrqi4Z5X0WzUvzV/sTmvL5UxqVR8nVbcZ3XI1rHIb5+rPJ07n4vtSua+8PNiI+e0hyg9Y30PJ5sHrYi/fd3JEC69t7RBJyKnALwPwBMADnV/CHZ+EA4O/qbD4dhv7Hmxi8gEgP8A4FdCCOvXa0/fe1RETovI6bX1jet/weFw3BLsabGLSA7bC/0PQgh/2v34sogc6dYfAbC023dDCI+FEB4MITw4PTW5WxOHwzECXFdnl22F4rcBvBBC+DdU9SUAnwLwa93/X7zu2SSLNLedevdv/+6/qaqXXnizV24bt8lsJg5zi4gN80bXzA4xW7DSXiB9Kps1eiIZQiwLDOvArMtOlLXba7MZ69hlFQBq9ahvl0S7jhaIjJKv2bKSNJvRVFYsmv0C1uXYRdOYANdJypqdmVZ1KREsblQjaaWNvmsPiUob5Ona147KW3UdOcf9H2VX0T7zEqc5Nvq2yu9Gn5s55T0N++yo0xn2GN4/YLfgtokkBHHKZ4Jl2uEBx7nv9xAebKbcMblZ7n3GXuzsHwbwPwJ4RkS+2/3sX2B7kf+RiDwC4CyAn9tDXw6HY5+wl934v8VggtqP3dzhOByOW4WRetCtb6zjbx7/SwDAd574pqrLktCSNeYZYQoCNj8Yk5SKdBsizrEpyxICMAGlNX20yRxWYJHeuEulFAFmSR0KHGVnfkI5xfL0bBSt19eWVTsh0X12fk7VZZW5kEV6Pcb1tRiNlzPpnNNO7KNWJa+5Pq7ywWmX2ENveGYiEs9NJ+wNxuJ53pi1tHiuz5bkBm1L6XOpaDyxYnwUye0Y+WhtPao8K2tV1U6nWzZmswGpENKgx86P8OH5KVXX6JpFrSej+v7AGofD8X0FX+wOx5hgpGJ8vbqFF7/3FABAOlq8zSe8a2rElwGitZjdbFVn0/SQiK+8zmxABO1mlsvaw61DWTSZd71U0kE9V65eiedNtIi8SdYEu3Naot34RjOKgYcOH1XtFmai6G4JPJot9q7LUTstOlZrsf9GU98Lnu+URM5yWV/n9FQUJS9d1cFLDM3NMJg0PZ8bTDgy8EswKQH6mScGjGQPaU9732LeeP095sevN6KVJLSsOM1qpFUxuRWvA5tWbGAXvdRTw+KA/M3ucIwJfLE7HGMCX+wOx5hgpDp7p9PB2vK2yUeM7qMMRlZX4eOU7RTG044ji4yZiMkMm+TNVChqIsZmiyK+tsy+ApmoGo3YXyejo+/aZCosmDxwgX5fbzuudfHp6eiJd+7KG73yWk3/Jt9558leeXNtS9V12pxeOM5BsaT3Hzaal3rletWkfe7EOanXoh5qOdMnKnG8NhpMkSiEsHsZWnO2+wphwLtoqLb99isHtmKzWc4QZk6UOVIxPgcpLJmHSjqt+w+8D6VOrAdCfbCXJgAUu96kw8g7/M3ucIwJfLE7HGOCkYrxAumlM7YmIyYB6De97e5iZNtxUEtiCAKUNxaJizZNcIaDDYwnX7nEXOtRTCubYJRmnUgjDH/95ETkj6vVdMgvm3UmKtGDbmJSm+/OX4hBQ3MVzQbG3nsZur1rGzoN1QSpDDat08HpqF6UKQ3V+cWLqt0wvrNBbGnDBOmOIaVoEbEFi6d9/AwqtdLg1E3DPh/mhZchLjzLS5in+8v8gnaMymuur39qR2pOu4/XL5bt3He6puBhabL8ze5wjAl8sTscYwJf7A7HmGCkOntA6OlhNhipQ/oZMoZQMMPusqQjmd+qlHQ+q7nkSDdv07kaLW1eK5WjTl2taT51Nj3lCiVqp3nji5MzvbKxIipSybyZhFo99jNH0WyypXW3l998sVe+4+RxVdchM06rFs917uKiajd3eL5XruT1fK+tRZdeTW6p9zdUqmSbPptdWDFEp+aot9S6h3Z2bWcjuzjVtdXQU2rL+ydW98YQl9uMDDCNQZsL7fjVGNkldggpZEeZKQcTsBTzei8o110jbnpzOBy+2B2OccFoTW8iKOxE5/RxYlO0j01HS0H87G2UMd5MzF3OZhtAawZCBA+NpjY75fJk+jBi2QrxtrWYZ66o0zNVSrH/CZO6t1yO4n/GpORtkAdWtRpViLkpw3FHXG1XL19WdaVCvKWr5A24tqo97e68/c5eecmkYm63I2EF871Z7nkWrS1vfEriKN+lrNVrCCboTYmtbEnNmC4GBsfBmMOoE/uWYzOXHSKbY21drRbvBZ+rY0ku+Lk1SmaO1gLzvlvyCn5aEiOuT3U9+awnI8Pf7A7HmMAXu8MxJhipGM/o836jXfYksaQUPEzmqtPt8sThZj2MmGxiaiqSMDRtuk3yjFuY0xTLV64QKQXJc3MLOhnOJqVFmj64oOqYsGJ2Tnu/1SkrbZLEa2kaHoSJyUgaEVJtMShQSqmNrdhf0QT8XF2JKkkwtMecoimwCqWHocR66ymYo/tbFP7cBrvweW2ASCwX6Vz5xDy2/CwZmb5AKpt65oxKkssM9uDkwKnUePltkLoVmCcPBsNUDY7zAquR+lxqjgdv1A+Ev9kdjjGBL3aHY0zgi93hGBOM1oMuADs8fPbEKemNbRNMxWmUWb/JtLXZrNmOlYlJDdWmus5GNEOVDIlitRr13DSrveuaZM5jksk0Yz3LqlQ2JB20z3Do8CEzxni+deIgD+Y6jx2NewQvv/C8PjdN0PpG3Ds4ettJ1Y6jDDM2hVRgb7V4M65euaqabRGnvPVqYy+xEpkfrdcg25NslCGnVhIicOzzQCPlPmNISNkUNcirDzA5BwzhCEcFbhidvUbjapN5LTX9Z+m+WP2ayVlSImexJkX25KvW9F5TKdl+dm4o6k1EiiLypIg8LSLPici/6n5+u4g8ISKviMgXRKRvT8LhcLxzsBcxvgHgoyGEHwTwAICPi8hDAH4dwG+EEO4CsALgkVs3TIfDcaPYS663AGAnMiLX/QsAPgrgH3c//xyAXwXwW8P6ymQyPQIIwy2hs2jatE4kkguZ4TqpCYigTi23HPOkFwuxP+vNlBKH2zXDhS4k3lWmo0ktNaLTwkKsm5jQaarnlLnNeFmRqJeQm1gw4u3R47f1yqvEUQ8AWUo3NTsdA3JOnjym2l24GEXyTGLNoGTmork6cGBetcvRfUkSm/GWglMo5ZUUDCcfif/z01qlqjHPGnH9d6yIPCBTqz1mjz+xplkuGz0yR31kzQmanF2WPi+K9ZIbMChzzE90w6izDeJR3GoYz8+uzN/p3IAYvz0WyXYzuC4B+AqA1wCshtBLIHYewLFB33c4HPuPPS32EEInhPAAgOMAPgjg3t2a7fZdEXlURE6LyGlm33Q4HKPFWzK9hRBWAXwNwEMAZkRkR7Y6DmBxwHceCyE8GEJ4MG/ioR0Ox+hwXZ1dRA4AaIUQVkWkBODHsL0591UAnwTweQCfAvDF6/XV6aRY3djWna2ra4FcXY0nIzJkkiqQ72WxoPXyTpt5u7XCMzsT9c0tivKqTGqdenk5nisxA6lMRvfZScq3Vm/UVLs6kVBYE8z6Rtw7yBoTz/R0jJ6bIf01b67zwOEjvfJD/0Cn7n3q61/vlf/+j3y0V5asFryqtThXltiQCULY/IWMNriwm/BtNW2mzHbinJy9FFNOT1Q0fz2b5aYrOnrwwEwcB7vIZmx+vgGEjQCQcrSZUuDte46OTR85JgI1UYxzpIzXaR4bbd0Hz05fwCfp92wqLJmNLaYdbRs377WN7XMPIwHdi539CIDPyXYWxQyAPwohfFlEngfweRH5vwF8B8Bv76Evh8OxT9jLbvz3ALxvl89fx7b+7nA4/jvAiD3oUjS6aW2lqWWZej1GD1nxlokuphBNV+2OJmTIUXrkVfJAAwDMxLqtZhR16tc0n3q2oM0/jPI0qwKxfwlapFpdjmJrKa+nuJiPnOzra3rDcnoyiuu5IhFWZHQfbYpEK03oyLkT9z3QK08uRA+9WlXP1eRUNMsFM/4WpXA+d+5Cr1zd0jz3WxtrdKTF1lY2iuvnlogco6Ovma1+hZyZKxKZJ8rx/lUqWq2ZpONyWasaReL8y5F50D5jLMZb4gnOcZA3e8wTxAVXIJNd26iA7EnZMtxybLplhapuCEHY688K6ztm2yEOdO4b73CMC3yxOxxjgpGTV+yIM31ORKYVI0s7p/NEKLGysqrazR2MO+v1qiZ1yBIJXb0axdGi4Y87dce7euVz5zYmGa4AACAASURBVM6oOqauFqKVXlvT42BZqlHT6gSrKxMlvbO7vBy94Q7lo4+S4ZbA0pXo2ddp6uts0vmWLsV0TW3jbci02MHUbW1FKmmmj2byDgCYmonzffzkCVXXIA+vq9fi95avLat2G+vxXMvrmpI7K/G4kicPyJwNdiGiDKM2sZdfvhhVgamK5vWbnozPQbms020VyIuwj9uQ1IFMhngJh7BL9NURWNy3noJEKYiGEfF7R0NILPzN7nCMCXyxOxxjAl/sDseYYLQ6eyaLTJf0oT8NUEQwhoVSJZrDao0Gfa69xzgN08K8JoG8cjXyq0+R2alQ0h50q6tRH06yJp0zeXExKePGxqZqd4BSN2WNF9TFi0u9crmkzUQNIpxsNigtsyG0LFeIC938XFcmor6ZtqOeyzz3ALC5Gce8PmTPYY2IPtbX9XVO0nXOzWtizXwxmt7uvCty1PNeBwA0ac9hbVWPY+lynKtLS7G8uqp57pvkvZet6/4LuXjdZSJ82NjUXo9XlqMZMWeiAFV6cWuyo/2kPD0vTbvRIgM8+Qx49G3TBXPK5wyzxQ6hxzAOfX+zOxxjAl/sDseYYLTpn0KKbGdbVM1kBovImaw2fQQSjzYaUbbZIcLYQacdf7vWTGbV8mT0fms0omh65OhR1e6N11/plY+dfLfun4IM1tej512zqYNA+Bc0MRzntVoUH69cW1N1R49Ecb1EpqF6XXu/NSjwplDUgSUcCMKEHZZgg0X1Ulmboa6SV+HZ89GDLjGZQ1XGULFqGaduooCWgr5n0zPRA/D4MZ2R9r0/GL8XSPyvben5WFmN83jpkk6HtXgxmh/5uq5V9fORKg9OfS15uqGlvH5uS2SWKyRE5iFabeIebbLXdEAaLRurI0oV0JWdXh+e/snhGHv4Ync4xgS+2B2OMcFoo95AhHhBhw9lKPIqdLQLaKsddbRMNg651tC6W0IkDytN7abKBBBCevSSiXorkm6fM6SVCfGHX3uD8r4Z+1dK7qfW9MauwNUtbcq6vBTbNpvRPDhh3DeZ7ICJMgDt3mr3Cxg1cmddWdFzsLh4qVdu1Dn/nNZXha7NpuDW6jylPLYmV26Y2d2cBABZyn1XqejIxENHo67/nh94r6pjM26D9nHsNV+6GK/5LEX6AcAi1a2ua7PfMu2f8PzkDeFkgWxieXOd/IxkqZ19cphMxT5WO8tqSMZmf7M7HOMCX+wOx5hg5OmfdijNrCWIudryVhThtmT+6rS0uB+Iq65txKg2pffJkriVbmlR/dBtd8dxGDIF9pRbX4uRc6WS7iNPPOm5xJJsxnFtbGiRkFNDbSpxTrtSzc5GEd+qEDkia2ARuWO40Nl8tb6mTYD1ehRN89Sf4VxQPPd9nAmDxPNhcmZfJKTsWmV7YHUla1QXnoMSRTjOLhxQ7e589z298g+bKED2bFxd1XN1+XI09S1eiOL+BTL5AcDSUlT7Vmvaey8otS8+A0Vj5ivSpRWNJ18vEHCId56/2R2OMYEvdodjTDB68oquGB76dqkj+gURzuJKGTv7KC9IVA0DgvsBdDidT1bv7M5QQEfBiFGvXIy7tG1KeFGYnlbtONAhkx1MtJAaCucWBYUkieaWY6QUIVGs6J36vBLj4+f1jt61Z9G0bZJ3sGdiSnx3lpuNRx/C4F32QWU7RrtTD/U9DkbR7crklZdsaBE5T16JTOBhs86qKzP3LEc05xNThvPv5O27dtJsa69KDjy6ZrLhLi7GlAsXyGPx4mVNFrJGloCVulZh0fUwbHYGU0n7m93hGBP4Ync4xgS+2B2OMcGIdfZAUVla7+LAfGs9CKwMcdE0ZJWyz8KjPLrib9zsAR1pVSAzWr2uPdxYn0rIky9vPcvY0ymrp7hAXOjWu45NfSvXol7XodTLgLI+Yr2qdUNOVR3I4y8YvvYqmX/KZW06zDQp3bJE/fXqsjHRdeK5F/qMb7vr6XafZVjkHN9fvSVgiBgpFdKbz7+s6m6bigQnCwfJ3NbWUWmBHhibDqtFacWGpVficRVyOpKwUIrHCwcPqbq777u/V06pfzaBAprY9MplHd234wH42kv6+hl7frN30zZ/R0S+3D2+XUSeEJFXROQLIpK/Xh8Oh2P/8FbE+F8G8AId/zqA3wgh3AVgBcAjN3NgDofj5mJPYryIHAfwMwD+NYD/Q7Zlr48C+MfdJp8D8KsAfus6PfVEaEsQwBKtzcTJxxwcYduBveb6Iv+JW7wQPanmjmi+cxBJwuKiDoioVqNYdfLkydifIdEoFdl0pcc4RWLlNRuEQypE2oliZmKDHkh8vrpmg4HitUk7jvfgnObrm56K3HvNlvEYa8U+i6Vodrp85VXVbn0rBhudOKHncaC5rc+uOixIZkB/pt1VSkv1xOIZVdc8E01l5clYzhvPQ+YbrEyYZUGnS40Y3yYzaJVSbPWJ+5nB4+dnMyHe+8mCJiaZJKIPZfJDNGn/4R9+AYOw1zf7bwL454h8ePMAVkNMEnYewLHdvuhwON4ZuO5iF5F/CGAphPAUf7xL011TyonIoyJyWkROt82miMPhGB32IsZ/GMDPishPYzsf/BS23/QzIpJ03+7HASzu9uUQwmMAHgOASrk0JMekw+G4ldhLfvbPAvgsAIjIwwD+WQjhn4jIHwP4JIDPA/gUgC9ery9B5OC2fOdhiEmDdTd2y+zLF0eRQP0qe/xg+mDUOApFrW+nZK66cklHLk1Wop5bpJTKnE8MAFiC2TTkiDOUU8xGrLHuPDkR+09MbjN2x2WyCgAoFKP7b4tIFVstY6IjF9Mtk86Z9eM6k1yY6DiOeiskhqSRzIgyJI+wuodDAuL0l/RhldyM5w5qM2UmiXMqJHw2TM62djO2Swp6WSRZzudm6uiQTWUdEzkXlMlYj5/JSLjhYGNmv9l5aDThTpPrthiMT2N7s+5VbOvwv30DfTkcjluMt+RUE0L4GoCvdcuvA/jgzR+Sw+G4FRitB50IMl0bW1aseU0121NdxvKesRhvOklyUWw9eDyaLfJGRGaespkpnc45T2atHKeCymsxnq8sY0gG2hSVNDOro+WuXYseUpWJKI7n8lrV2NqK4mKzrjc9251ozuPUxhnj5cdcFg2T9nl6OqorL77yRq+8vqHF/YMLMf3Txz72MVU3RWaiJ598olfuGE++hAg79iCJ7ooj85E3sNC4U9XVzjzTK79B0X0nb79ftSvNxWuxzw4L0H0Kye4SuFJxAAyXoWktsDl52HT0jyPs/vkeh+BwOL6P4Ivd4RgTjDb9EwKSzO6CxlDxgwMpaAfbisjKu84IQTPzh3vlKQ4sSbVYub5GYrChkm5RZk5RgRPaksBBFXljFuCW5bImzqhWo0fatasxm2wuOaza1RtxZ33TZCMtVyhAZyKqHS2TErROqaFmjTpRrcc5WSPevQmTJuoDD74/9mGy5vK9yVKqL/ZCBPQcVAxldpsCUNQM241oEplP3nmHqsveGVW2LaLuTvP60c/mOfjHBliRmN0n4rMIvtunu1QasFee6j7oZ0cFhBmdp6c2DFlI/mZ3OMYEvtgdjjGBL3aHY0wwcsLJHU3Dmib4qJ+UcG/khQyOYgKAgyeiSYYd1zpG314g/bXT1Kamq6tRj2bChxC0PpymUV+tGo7wbJb0XjP+I4cjqcHyctw7WFvV0XGNZtRlW3XtGVdtxZnMZqNZrmjSLR86FIkcgtlXeOVM9Hw+fORIr/y+Dzyo2t11R9SHD8xZgkze04jjWF9fVa3+7r98s1e+7z33qrr774/Hgfpomyg9Aen26Yaqy1HkWInIIoslbVZlElIx9xOg1NGmRunzFDGZmpYybFOK+9NJEgb30ZeBO929guBvdodjTOCL3eEYE4yeN36A5N0X0D8Aw0R3tm5MzOr0PrPEB88EFZWyJgi4/+6YBujVV7Wo9/qbkcyiRtlNKyb9U0pioOVab5P60mhqcXF6IaoQBw7E8dZqmvOdSS+28tr7rU2H09ORsGJ+blK1K5fjtZ09f0nVHSJ1YmMzqglf/ou/Uu0++bM/GQ/SD6s6EPfe/Hz0TtvYNFlQry31yq++rO/tbScjIcY0cfPb+Wh3OF2Vvp+sKml+N90Hp+ziuQF00NOwzLjK09N60CmTmq5KFekKe9DZvAgqIsycu5uPYYiJz9/sDseYwBe7wzEm8MXucIwJRh711tOhhugW/V9jXSUWLflfksTos8MndPRTjiLA2mS6yhkyAuZyX1iYV3U1crfc2Iw630HDAz45GV1Ag0m33CSOcy4DQJ3MZjOFqCdytB0AJLm4R7A1oU17y1ciwcQsRXLNzmrCyQ0ii6xM6rpnn4/Ekl//+td65ZxhvrzzXe/qlYvG9Vcoyu6HP/TDvfICRcoBUO7TbePSmyMCDCbfmJrS+w91itprmznlPnOUu69gyBxT2sdpNU2etiTed+tCnScSEM4fZ4lG1WbVMDucIq8wJmgVfbe7zj4M/mZ3OMYEvtgdjjHBaKPeRJB0iR46RlQCmSqkj0CO0wAN9qCbnIli98LhI6pOFM8XcX03tNhH0nlf6qat9Sgir2zEhmK45Ngsl5j0v1PH4rhYTAV0RNg6DWTCpGcKlJLJphduEf8daznNlhb7WsTB9trrb6q6b37z73rlPEWDzUxrcf/Y8Zg6y5Jj8NmY7+49975H93E4RvS9+aYex9WVaGJU3mlGCmbTZyOroxiZv529JbcowhAAsjSPPF4AyNEJO4YhuVbjfgalq8JQ05tWUzMDG8oAkgsgqiHDtGN/szscYwJf7A7HmGC0u/EBPd/+Pk84To9jspsOCn7JmmCXQ7QDnzfkBCy6zxHH2rvu0GmLlimd0uJFHYBSoh3ndohi38WzZ1U7Fhcnjeh75FAkeWBiCACoUBqpVaZtNmoNE3i0jUVC8rsH4WwZjzHmfnv66aehEftkVebO23XKoaNHj/bKzYb25GNxl3eprZhZLsd7ce99WsRfXVnulc9dONcrrxvCjgJZK7JZEyRDzxWTm2SN2tEkQpAtQ//NRBx2Fz9P/IN8X/q40tWg7AfUVojIoo9LOtb1ifGpB8I4HI4ufLE7HGMCX+wOx5hg5FFvfXaTLqyvkD5kXSjWVSZ0qp+Fw1GHzFgPozTqkCUyrRTzWnc7eTSa76pVTYTwkY/GKK9vn45c6LUtncqYbV62/3wx6nx2X6FN+wpr6zE6rGq44SeISLJs0ldVZqIOfOXq1djOkEWuLkd9+MQx7QHIqaNBZr58UXvytSg1VDunTanao4v3XCxpSby3ZqsG8/MxclEkzuPv/8HnVbt33xNJJu11FknHLhRiHyb7E7LkHdiXzov2HGqGjIRNeIUin0vfF6XP28d7oEfdMHJLQ3KaXj/qba/52c8A2ADQAdAOITwoInMAvgDgFIAzAP5RCGFlUB8Oh2N/8VbE+B8NITwQQtjhJvoMgMdDCHcBeLx77HA43qG4ETH+EwAe7pY/h+0ccJ++7rd2xJlOnyxDRe0VxmaMLAWuHDyhTUGcOdSmEmq2omkog2hCW1vX5i82vV1b1SJbpRJFxIsXIuHD9IL21jtyNHqFTVS06Fsgsd6OsUAkCVNE1rCyogWmV197PfZhAnk4g+zkRBQrD5qgnksXoory7rs01/qRGnmdkfhsudmWV6J50JJGsIk0Q/L5MH7B1Kh4LQpqefX1aHo7++brqt3ySlRX1tc1OcZDH/qhXvn48ajm2ZRdpSIHtJigEiaUEK2WMTlJvR6fsS0j7udJNSgaET9LQVqasGLvYnya3jwPugDgP4vIUyLyaPezQyGEi9snCBcBHBz4bYfDse/Y65v9wyGERRE5COArIvLiXk/Q/XF4FAAKhfx1WjscjluFPb3ZQwiL3f9LAP4M26maL4vIEQDo/l8a8N3HQggPhhAezA/h73I4HLcW1119IlIBkAkhbHTLPwHg/wLwJQCfAvBr3f9fvF5fhVIJd3Sjns69rvWuDunU/e6hUXcpknvlwaO3qXacftl643Yob9giETy8dv6KascRbDPTOgfaGqVUPnksai2VSU2mUCTTWCGnddmpidi21NLmqgbphtXNaNI5dPCoaneUCCEXFy+ousWLUX+tVKKenjOpqdnNM1/QUXUbK5d75YvL8Zqtnvu7f/anvfLUlHYLniDSxlIS56BV17rm1iZHjWmFs0imstXVeJ9O3an3ai4txv0T+z555pnvxnaX4lzdfvu7VDsm+MyaTnhfoVi0+ejiPgbvn7C5zh6vrOl9BT4fmwpzOeM2TuW34y67l1ftIQB/1r3gBMC/DyH8lYh8C8AficgjAM4C+Lk99OVwOPYJ113sIYTXAfzgLp9fA/CxWzEoh8Nx8zFSJbqQL+Fdt9+/feKsFgkXz53plW1a3JRS/0wvRLNWsaCHn5CJJ2uIIaYPHIt1JNHWL+mthgPELz9txPMjJOp98IMxFZI1kCwtRVOZ5LTYNzUVzWuVjFYhNtpRvXiKqjbNXfrZBx7ulT9iOOtffvG5Xvnp70UR1vLdVaZjKqSq4X7baEfzY5qL0XLBRJQ1mteorFWSUhKPG414XzZW11S79fVoAkxMpFgyE9WogzNxHitT2ox4/o1XeuVcQZvGihRJWK9Hc9j3vvdt1W5hIXrrnbpd8xdOTsVxtA15BROtsCnV8vUlFGVnozo5ndXahvbaZLDZtm//q6sCOm+8w+Hwxe5wjAt8sTscY4KR6uytZg1LZ54FAByZ1/zhU6VoTnnljYuq7uip6M6ZkA7cqGvSwCniaz9+yphWDkZTWaMa9cZySevlczPRhFRItLmK9wTYtndtWachzleijjc1rVMZl0qUU6xk+OYp9XBxIeqGa4Zlpq1IJfWOwd13398rN5txfhbPazJHZOP3tmpaT8zTpsZCJUYWzkxo89qJg9FN+MhBnVuvRBFg2i1Ym96apOvXDQnkVjXuHRRK8d4uLWvT1cFjcR/Hug+zBy6b0KandcQkD/Hs2TdUXYWu+8AB7Sg6QabUDLt8GxYlTluNjtb7WRfnvAWNht4H2SAzZcf0keveM8u9z/A3u8MxJvDF7nCMCUbMGw/kuqLx7IwWo2aSaNYqTOkosnffc3evvEoRTqsmGqxYieKWStEMTSKRIHp3HTDRYDkSowrGfMdpfi+QyS6X1+avuVnyvAta3MpKbFue0KIvW01un4umyVpDc6GnKRE4GrE4QymwWKRv1TSJ4sKhSLRZN2SRly9HNapG5ItZE6aXksmoUTckkER8mcvFuW+19HjrtSiq1hpGBCXPuwzNcehcVs1mJjj1lBkjTWqL+d87ek4rlSiOJ4a8Iksqz+UlrWJeuxZJQObJfFcyJBo8LJtCio1lKZGs5Ix5LZAVt248EatdFcimRGP4m93hGBP4Ync4xgQjFeMz2QSl7u50Jq9JHSYp6OTQMb1Dni/FtrOzUVRKjdN/mTOJmuB+TjfVYGKB1PCM0wZ8q21EJRKncxQ8MjerVRImrLCeVJwh9OolLRIWaBc/RyJyx3hFMZd7ktceesxrzsaDYslkgiU+uWJRZ2DNk8WDOdc2t/RuuWQGkzrwtFbJehBMXiROc5Ur6j4W5qKKtboSvfXKxmuQBWFLgMFibZ6eibYhqGg04rVJRlthODjKknRk6F5coKCkxARALRyIz23eBBRpQg+aU2h02vHZyRov06Qb6NSXj4HHOrDG4XB8X8EXu8MxJvDF7nCMCUars2cyKHX12a1N7QXVInKJnIn8YVNIlnShnNGfrpFua1MlMxkl67WTE1qXZc8nS3w5RdFP01ROTBQT845bHS9XiCaZxor2vGsQT32nGXVlGwXIOmRo6v43SdPrtKI+3DEmmbQe53ti0ujz2TiPJRp/3pCE1iiS7rnnNBlJicgyJijfHZNDAsDxU5GYY2pae1VuUi48Np1mRe8xsBdex+zBpKSb8xxks3o+EmrXbGgzIm2fIDT1nkCnE02TJdpLaBvT3osvvdwrl40n4pHD0SuPr9OSSvJ+T8Y8EzFPgke9ORxjD1/sDseYYKRifC5fwJGT2wEqZ87oYIN6LYo9k3ntfZSwKExmkawJVOEUO5YrLCUOMG2e0b93QXlZVQfWsWlvtaYDVZCN402MmYVTBQfLT04iaMoc+EYlCSSOZoy4yKYXJmso5C2zb+zTerVVKlFMTktxTInhsSuTGF+tXlN1T33jK73yAUrtfOc9Oi1zoxr7Xzx/3owj3sNAXnjW3JgltanVNEFDNMYMzVvHcMQJ4nHbGL2evhLPdzXVZtZyEuc/Q8+LmP7Z/W3rrJ6r5OUYmPXeO6NIf+qgVq84bVnDXKeEwZ5zvfFdt4XD4fi+gC92h2NM4Ivd4RgTjFRnzxdLuO2u7UisakvrmiuXYxRZpaR1ds5/VaBIoHxW61bsQpiY4H7mpU/IzbO5ofXyNI3tqm1tgtmgXF4TTCSZ0fpwoRTNch1D9JgI5RRrW72RzT9EXGBNb6Rvw+iGrFd32nG8NoIK5N6aK2jzXZZIFEtE2DhpCDiZQGF2RpvDSpT959zrZ3vlJ/76G6pddSOa4u59//tV3e3vua9XZvNmp23uO+3dFEvGLJeNui2brvrcSumar9b0fDy3Et12WybXM7sh58kFPLT0c9WoxeNWW7v7dii33hvfjubXk9PaNPu+k3FdLEwZYpXMYJPbDvzN7nCMCXyxOxxjgpEnXwtdkfzwoWPq8wVKH5QkWiy+ciVGh22sLPbK1aDND1mKCpKmVhPaRNCQkLhcyOvfu5QONzqa1KFMJsCUTHsho6O1qiQiZxMtEtYUX/5g0yGnOUZOe1wVK9E80zQeYy2KMMuSN2DeEGw0mpS2yHgKUhYqtLmdEWE54ivJ6us8/3L0GPvW3/xVr2zJNtgk+Ma3/5uqyxLxxx0/8N543qIeL3vJiZlTToOdJROmJeyAxD6qQc9Vjjzj0qZWy5jQpJ1yOm4T2ZaN35O2fm7ZottM4zy+cFVz0L15NYr1p6b18333ke3zNcz4GHt6s4vIjIj8iYi8KCIviMiHRGRORL4iIq90/89evyeHw7Ff2KsY/28B/FUI4R5sp4J6AcBnADweQrgLwOPdY4fD8Q7FXrK4TgH4EQC/CAAhhCaApoh8AsDD3WafA/A1AJ8e1le71cLVxW0vqdqG3mmcIDF+/rAW8StUt7kaPZhWli+pdvUNSi1UMNxvpbj7mqFd/NTwqlVJ/M+Z38KJqbjTW6Fd6tSm3GH6YkOE0CHyBhvMkJCXGIQ86HLaa6tVi9eWn9GWi0xCZA35uHueGiKORjPOVT7RaggPi9MJ9WUWYmKInO7jnvsj/93z3/ybXrle0xYO9gBs1/QO9rlnKEUTBUrd8cAHVLsiZYztiAn4acVBZ5N4z4pGdalVo2jdTPW1JBkiEslo8bkTYl2zQYE2xvqRIIr1kjEqRJs87wLTletmW53Yx7NX9LP50sXtXfy16o1x0N0B4AqA3xWR74jIv+umbj4UQrgIAN3/B4d14nA49hd7WewJgPcD+K0QwvsAbOEtiOwi8qiInBaR0+tra9f/gsPhuCXYy2I/D+B8COGJ7vGfYHvxXxaRIwDQ/b+025dDCI+FEB4MITw4RTxzDodjtNhLfvZLInJORO4OIbyE7Zzsz3f/PgXg17r/v3jdvtK0xy8+Pat53SfnInHBmknrmyW9aP7AqV65Mq1515998m975e/+7TdV3dYqkWUoIgetGLGHVDC/hQ//zCd75fd/iLy9DIliSJlgUetQ7PRXKOrpz9D+QSCTWsekAaoS+SImjBmKUkpRICHabd1HgxgZlla1rsxOeWmDzEQNbTLqcBoqc531rTh3E5PxRz5tmyg9Jos0UYAt0u8vPP+9XrltePTv+ns/1CuXjJcf3xo2m/Xp1GSa3GzrOiYLsR6LGfJca5HeD0Naks1ROvFER+0luajDp3TNdo8k7ZAXYaqfzWbY3k/qDHl/79XO/r8D+AMRyQN4HcD/hG2p4I9E5BEAZwH83B77cjgc+4A9LfYQwncBPLhL1cdu7nAcDsetwmg96ER6HmVT8zqDaW0ritmbqzqt09xCFPkrM1HcL3e0Z9lDD/9Urzwzr9M6/eXnf79XXr98pVdmPnnA8olrOer0X3+5V37xW0/EVobfjY8tqUBCZq5j92kihzLx2nGgxlZVi88rW1HUa+B5VddkUgq6NivGnzoa53HDcP6dXYxqQm4rpjcqLGtyiTaJ5GmwZA0UbNSM/dsUUhyAosRlAHwrWmQSvfTqi6pdk7zY7vrgQ6puYpZ47WhObZqkVkredSbQRonTNiiJPRhTNiMa0y8ofZUJ4AJ5VWaJDKNjshSrCTF8gD3HxD77aIT7xjscYwJf7A7HmMAXu8MxJhhtymZIj889NdFabSKzmD9wWNXNkM6+thx1yKzRqTmH1oMf+Undx3x08Pv21/+yV36ZdG8AaNaiGSQYM0trK7oSbNTjOMSa3tSB1g3zRBpY39ROh8006uJbFG22XtX6NlWpSDkAyJFOnDA5hjGbrS/G42O36f2TtfORbCLtxDG1m5uqXYdMjC2z98Emx8D5y2wqMu2bi0HgyLCWIT65+mbkrG+aaLZ7Pvz3e+XJGdbf9bzVaa6qDRMJaZ5VU9krZmQA+QiANt2npGDckzO0DCmS0N7bDpFMWkLL0O5O0BDiSX+zOxxjAl/sDseYQMIQ0emmn0zkCoA3ASwAuHqd5rca74QxAD4OCx+Hxlsdx8kQwoHdKka62HsnFTkdQtjNSWesxuDj8HGMchwuxjscYwJf7A7HmGC/Fvtj+3RexjthDICPw8LHoXHTxrEvOrvD4Rg9XIx3OMYEI13sIvJxEXlJRF4VkZGx0YrI74jIkog8S5+NnApbRE6IyFe7dNzPicgv78dYRKQoIk+KyNPdcfyr7ue3i8gT3XF8octfcMshItkuv+GX92scInJGRJ4Rke+Ka0i7jgAAAuVJREFUyOnuZ/vxjNwy2vaRLXYRyQL4fwH8FID7APyCiNw3/Fs3Db8H4OPms/2gwm4D+KchhHsBPATgl7pzMOqxNAB8NITwgwAeAPBxEXkIwK8D+I3uOFYAPHKLx7GDX8Y2PfkO9mscPxpCeIBMXfvxjNw62vYQwkj+AHwIwH+i488C+OwIz38KwLN0/BKAI93yEQAvjWosNIYvAvjx/RwLgDKAbwP4IWw7byS73a9beP7j3Qf4owC+jG2esP0YxxkAC+azkd4XAFMA3kB3L+1mj2OUYvwxAOfo+Hz3s/3CvlJhi8gpAO8D8MR+jKUrOn8X20ShXwHwGoDVEHo5l0Z1f34TwD9HpF+Y36dxBAD/WUSeEpFHu5+N+r7cUtr2US52G+8EWCqYMYGITAD4DwB+JYSwfr32twIhhE4I4QFsv1k/CODe3ZrdyjGIyD8EsBRCeIo/HvU4uvhwCOH92FYzf0lEfmQE57S4Idr262GUi/08gBN0fBzA4oC2o8CeqLBvNkQkh+2F/gchhD/dz7EAQAhhFdvZfB4CMCPSS0UzivvzYQA/KyJnAHwe26L8b+7DOBBCWOz+XwLwZ9j+ARz1fbkh2vbrYZSL/VsA7urutOYB/DyAL43w/BZfwjYFNrBHKuwbhWwTy/02gBdCCP9mv8YiIgdEZKZbLgH4MWxvBH0VwA5f9i0fRwjhsyGE4yGEU9h+Hv4mhPBPRj0OEamIyOROGcBPAHgWI74vIYRLAM6JyN3dj3Zo22/OOG71xofZaPhpAC9jWz/8lyM87x8CuAighe1fz0ewrRs+DuCV7v+5EYzjI9gWSb8H4Lvdv58e9VgAvBfAd7rjeBbA/9n9/A4ATwJ4FcAfAyiM8B49DODL+zGO7vme7v49t/Ns7tMz8gCA09178+cAZm/WONyDzuEYE7gHncMxJvDF7nCMCXyxOxxjAl/sDseYwBe7wzEm8MXucIwJfLE7HGMCX+wOx5jg/wfQZ4fzWZDUZgAAAABJRU5ErkJggg==\n", 1192 | "text/plain": [ 1193 | "
" 1194 | ] 1195 | }, 1196 | "metadata": { 1197 | "needs_background": "light" 1198 | }, 1199 | "output_type": "display_data" 1200 | } 1201 | ], 1202 | "source": [ 1203 | "# Example of a picture that was classified.\n", 1204 | "index = 7\n", 1205 | "plt.imshow(test_set_x[:,index].reshape((64, 64, 3)))\n", 1206 | "print (\"y = \" + str(test_set_y[0,index]) + \", you predicted that it is a \\\"\" + classes[int(d[\"Y_prediction_test\"][0,index])].decode(\"utf-8\") + \"\\\" picture.\")" 1207 | ] 1208 | }, 1209 | { 1210 | "cell_type": "markdown", 1211 | "metadata": { 1212 | "colab_type": "text", 1213 | "id": "iAHWKyttIgyD" 1214 | }, 1215 | "source": [ 1216 | "Let's also plot the cost function and the gradients." 1217 | ] 1218 | }, 1219 | { 1220 | "cell_type": "code", 1221 | "execution_count": 141, 1222 | "metadata": { 1223 | "colab": { 1224 | "autoexec": { 1225 | "startup": false, 1226 | "wait_interval": 0 1227 | } 1228 | }, 1229 | "colab_type": "code", 1230 | "id": "4t7Vx6ffIgyE" 1231 | }, 1232 | "outputs": [ 1233 | { 1234 | "data": { 1235 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxV9Z3/8dcnCUlISEJWliSEHQoqopFFbMWlFq11acWlWq22pZvT6TKLv19nWsdO59dlZjpttdOxrp1a99aitbUuqEVECMgiAhLWhLCEPeyEfH5/nBO8xJuQQO69Se77+XjcR3LP+d57P/fk5r7P+v2auyMiIskrJdEFiIhIYikIRESSnIJARCTJKQhERJKcgkBEJMkpCEREkpyCQHokM/uTmd2S6DpEugMFgXQqM1tnZhcnug53v9TdH050HQBm9qqZfT4Or5NhZg+Y2R4z22xm3zxB+2+E7XaHj8uImDfYzGaZ2X4zWxH5NzWzz5rZUTPbG3GbGsO3JjGmIJBux8zSEl1Ds65UC3AnMAKoAC4A/sHMpkVraGYfA+4ALgIGA0OBf4lo8ijwNlAIfBt4ysyKI+a/6e59Im6vdu5bkXhSEEjcmNnlZrbIzHaZ2RwzOyNi3h1mttrMGszsXTO7OmLeZ83sDTP7iZntAO4Mp802s383s51mttbMLo14zLG18Ha0HWJmr4ev/ZKZ3WNmv2nlPUw1s1oz+0cz2ww8aGb5ZvacmdWHz/+cmZWF7b8PfBi4O1xzvjucPtrMXjSzHWa20syu7YRFfDPwPXff6e7LgV8Bn22l7S3A/e6+zN13At9rbmtmI4GzgO+6+wF3fxpYCnyqE2qULkhBIHFhZmcBDwBfJFjL/B9gZsTuiNUEX5h5BGumvzGzARFPMRFYA5QA34+YthIoAn4E3G9m1koJbbX9LTAvrOtO4DMneDv9gQKCNe8ZBP9HD4b3BwEHgLsB3P3bwF+B28M159vNLBt4MXzdEuAG4BdmNjbai5nZL8LwjHZbErbJBwYCiyMeuhiI+pzh9JZt+5lZYThvjbs3tPFc481sm5m9Z2b/3MW2jKSDFAQSL18A/sfd33L3o+H++0PAJAB3f9Ld69y9yd0fB1YBEyIeX+fuP3f3Rnc/EE5b7+6/cvejwMPAAKBfK68fta2ZDQLOAb7j7ofdfTYw8wTvpYlgbflQuMa83d2fdvf94Zfn94Hz23j85cA6d38wfD8LgaeBa6I1dvevuHvfVm7NW1V9wp+7Ix66G8hppYY+UdoStm85r+VzvQ6cRhBinyIIsr9v4/1KF6cgkHipAL4VuTYLlBOsxWJmN0fsNtpF8EVTFPH4mijPubn5F3ffH/7aJ0q7ttoOBHZETGvttSLVu/vB5jtmlmVm/2Nm681sD8EXZV8zS23l8RXAxBbL4kaCLY2TtTf8mRsxLRdoiNK2uX3LtoTtW8477rncfY27rw1DeylwF62EmHQPCgKJlxrg+y3WZrPc/VEzqyDYn307UOjufYF3gMjdPLHqJncTUGBmWRHTyk/wmJa1fAsYBUx091zgI+F0a6V9DfBai2XRx92/HO3FzOyXLc7QibwtAwj3828CxkU8dBywrJX3sCxK2y3uvj2cN9TMclrMb+25nOP/VtLNKAgkFnqZWWbELY3gi/5LZjbRAtlm9vHwyyab4MukHsDMbiXYIog5d18PVBEcgE43s8nAJzr4NDkExwV2mVkB8N0W87cQnJXT7DlgpJl9xsx6hbdzzOxDrdT4pRZn6ETeIvfb/xr4p/Dg9WiC3XEPtVLzr4HPmdmY8PjCPzW3dff3gEXAd8O/39XAGQS7rzCzS82sX/j7aOCfgT+0YzlJF6UgkFh4nuCLsfl2p7tXEXwx3Q3sBKoJz1Jx93eB/wDeJPjSPB14I4713ghMBrYD/wo8TnD8or3+C+gNbAPmAn9uMf+nwDXhGUU/C48jXAJcD9QR7Lb6IZDBqfkuwUH39cBrwI/d/c8AZjYo3IIYBBBO/xEwK2y/nuMD7HqgkuBv9QPgGnevD+ddBCwxs30Ef+vfAf92irVLApkGphE5npk9Dqxw95Zr9iI9krYIJOmFu2WGmVmKBRdgXQk8k+i6ROJF5/6KBGfr/I7gOoJa4Mvu/nZiSxKJH+0aEhFJcto1JCKS5LrdrqGioiIfPHhwossQEelWFixYsM3di6PN63ZBMHjwYKqqqhJdhohIt2Jm61ubp11DIiJJTkEgIpLkFAQiIkkupkFgZtPCQTeqzeyOKPN/EvY4uSjs13xXLOsREZEPitnB4rAL3nuAjxJcpDPfzGaG/coA4O7fiGj/N8D4WNUjIiLRxXKLYAJQHfZdfhh4jODS/dbcQDBOqoiIxFEsg6CU4wf4qA2nfUDYH/0Q4JVW5s8wsyozq6qvr4/WRERETlIsgyDaQBWt9WdxPfBUOIzgBx/kfq+7V7p7ZXFx1OshTmhxzS5++OcVJ/VYEZGeLJZBUMvxIz2VEfS9Hs31xHi30OLaXfz3q6tZXKPj0SIikWIZBPOBEWY2xMzSCb7sPzAouJmNAvIJBiWJmavHl5KVnspv5rZ6cZ2ISFKKWRC4eyPBGLQvAMuBJ9x9mZndZWZXRDS9AXjMY9wNak5mL64aX8rMxXXs2n84li8lItKtxPQ6And/3t1Huvswd/9+OO077j4zos2d7v6Bawxi4aaJFRxqbOKpBbXxeDkRkW4hqa4sHjMwl7Mr8nnkrQ00NWkcBhERSLIgAPjMpArWbtvHnNXbE12KiEiXkHRBcOnp/SnITud/565LdCkiIl1C0gVBRloq0yvLeGn5VjbtPpDockREEi7pggDgxgkVNLnz6LyaEzcWEenhkjIIBhVmcf7IYh6bt4EjR5sSXY6ISEIlZRBAcNB4a8MhXnx3S6JLERFJqKQNgqmjSijt21tXGotI0kvaIEhNMT49cRBzVm+neuveRJcjIpIwSRsEANedU06vVOORt7RVICLJK6mDoKhPBpeeNoCnFtSy/3BjossREUmIpA4CgJsmVdBwsJFnF7fWQ7aISM+W9EFwzuB8RvXL4X/nrifGHaCKiHRJSR8EZsZNkwbxzsY9LK7dnehyRETiLumDAOCq8aVka9AaEUlSCgLeH7TmWQ1aIyJJSEEQummSBq0RkeSkIAh9aEAulRX5/Gbueg1aIyJJRUEQ4aZJFazbvp83Vm9LdCkiInGjIIhwbNCaN3XQWESSh4IgQkZaKtdWlvPS8i0atEZEkoaCoIUbJw7CgUff2pDoUkRE4kJB0EJ5QRZTRxbz6PwaDVojIklBQRDFZyZXUN9wiL8s06A1ItLzKQiiOH+kBq0RkeShIIgiNcW4cdIg3lyzneqtDYkuR0QkpmIaBGY2zcxWmlm1md3RSptrzexdM1tmZr+NZT0dcW1lMGjNb+bqoLGI9GwxCwIzSwXuAS4FxgA3mNmYFm1GAP8HmOLuY4Gvx6qejmoetObphRq0RkR6tlhuEUwAqt19jbsfBh4DrmzR5gvAPe6+E8Ddt8awng77zORg0JqZizRojYj0XLEMglKgJuJ+bTgt0khgpJm9YWZzzWxaDOvpsMoKDVojIj1fLIPAokxr+W2aBowApgI3APeZWd8PPJHZDDOrMrOq+vr6Ti+0NWbGTZMrWFa3h0U1u+L2uiIi8RTLIKgFyiPulwEt97HUAn9w9yPuvhZYSRAMx3H3e9290t0ri4uLY1ZwNFcfG7RGB41FpGeKZRDMB0aY2RAzSweuB2a2aPMMcAGAmRUR7CpaE8OaOqxPRhpXn1XKs0vq2LlPg9aISM8TsyBw90bgduAFYDnwhLsvM7O7zOyKsNkLwHYzexeYBfy9u2+PVU0n66ZJFRzWoDUi0kNZdzsIWllZ6VVVVXF/3em/nMPWhkPM+tZUUlKiHf4QEem6zGyBu1dGm6cri9vpxokVrN++n7lru9wGi4jIKVEQtNO00/qTk5nGU1XaPSQiPYuCoJ0ye6XyiXEDef6dTew5eCTR5YiIdBoFQQdcW1nOwSNN/HHJpkSXIiLSaRQEHTCuLI8RJX14oqrmxI1FRLoJBUEHmBnXVpbz9oZd6p5aRHoMBUEHXTW+lNQU40kdNBaRHkJB0EHFORlcOLqEpxdu1JjGItIjKAhOwvSzy9i29xCvrYxfB3giIrGiIDgJF4wuoahPOk8u0EFjEen+FAQnoVdqClePL+Xl5VvZtvdQossRETklCoKTNL2ynMYm55m3Nya6FBGRU6IgOEkj++UwrrwvT1bVavQyEenWFASnYPrZZazc0sDSjbsTXYqIyElTEJyCT4wbSEZaiq40FpFuTUFwCvJ692Laaf2ZuaiOg0eOJrocEZGToiA4RddWlrPnYCN/eXdLoksRETkpCoJTNHloIaV9e/Okdg+JSDelIDhFKSnGNWeXMbt6Gxt3HUh0OSIiHaYg6ATXnF2GOzytwe1FpBtSEHSC8oIszh1WyFMLamlq0jUFItK9KAg6yfTKMjbs2M9ba3ckuhQRkQ5REHSSaWMHkJORpo7oRKTbURB0kt7pqVw+biDPL91Egwa3F5FuREHQia6tLNPg9iLS7SgIOtGZ5X0ZrsHtRaSbURB0IjNj+tllLNywi+qtexNdjohIu8Q0CMxsmpmtNLNqM7sjyvzPmlm9mS0Kb5+PZT3xcPVZ4eD2OmgsIt1EzILAzFKBe4BLgTHADWY2JkrTx939zPB2X6zqiZeSnEwuGFXM7xZupFGD24tINxDLLYIJQLW7r3H3w8BjwJUxfL0uY3plOfUNh3jtPQ1uLyJdXyyDoBSI3D9SG05r6VNmtsTMnjKz8mhPZGYzzKzKzKrq67v+l+uFo0sozE7nySp1OSEiXV8sg8CiTGvZ/8KzwGB3PwN4CXg42hO5+73uXunulcXFxZ1cZuc7Nrj9ii1s1+D2ItLFxTIIaoHINfwyoC6ygbtvd/fmb8pfAWfHsJ64ml5ZzpGjzjOL6k7cWEQkgWIZBPOBEWY2xMzSgeuBmZENzGxAxN0rgOUxrCeuRvXPYVxZHk9W1WhwexHp0mIWBO7eCNwOvEDwBf+Euy8zs7vM7Iqw2dfMbJmZLQa+Bnw2VvUkwjWV5azY3MA7G/ckuhQRkVZZd1tbrays9KqqqkSX0S67Dxxhwvdf4rpzyrnrytMSXY6IJDEzW+DuldHm6criGMrr3YuPje3PM29v1OD2ItJlKQhirHlw+xc1uL2IdFEKghg7d1gwuL06ohORrkpBEGMpKcanwsHt6zS4vYh0QQqCOJiuwe1FpAtTEMRBeUEWk4YW8KQGtxeRLkhBECc3TBjEhh37eWm5DhqLSNeiIIiTj58+gMGFWfzslVW60lhEuhQFQZykpabw1QuG887GPcxauTXR5YiIHKMgiKOrxpdSXtCbn75cra0CEekyFARx1Cs1ha9OHc7iml28vmpbossREQEUBHH3ybPKKO3bm5++9J62CkSkS1AQxFl6WgpfnjqMhRt2MWf19kSXIyKiIEiE6ZVl9M/N5Kcvr0p0KSIiCoJEyEhL5ctThzFv7Q7mrtFWgYgkloIgQa47p5ySnAx+pq0CEUmwdgWBmU1vzzRpv8xeqXzx/GHMWb2d+et2JLocEUli7d0i+D/tnCYd8OkJgyjqk66tAhFJqLS2ZprZpcBlQKmZ/SxiVi7QGMvCkkHv9FRmfGQo//b8ChZu2MlZg/ITXZKIJKETbRHUAVXAQWBBxG0m8LHYlpYcbpxYQX5WL36urQIRSZA2twjcfTGw2Mx+6+5HAMwsHyh3953xKLCny85I4/MfHsqPX1jJktpdnFHWN9EliUiSae8xghfNLNfMCoDFwINm9p8xrCup3Dy5grzevfjZy9WJLkVEklB7gyDP3fcAnwQedPezgYtjV1ZyycnsxefOG8JLy7fwzsbdiS5HRJJMe4MgzcwGANcCz8WwnqR1y7mDyclM4+5XtFUgIvHV3iC4C3gBWO3u881sKKCjm50or3cvbp0yhD8v28yKzXsSXY6IJJF2BYG7P+nuZ7j7l8P7a9z9U7EtLfncNmUwfTLS+Lm2CkQkjtp7ZXGZmf3ezLaa2RYze9rMymJdXLLpm5XOLedW8PzSTaza0pDockQkSbR319CDBNcODARKgWfDaW0ys2lmttLMqs3sjjbaXWNmbmaV7aynx/rceUPp3SuVu2dpq0BE4qO9QVDs7g+6e2N4ewgobusBZpYK3ANcCowBbjCzMVHa5QBfA97qUOU9VEF2Op+ZXMGzi+tYXb830eWISBJobxBsM7ObzCw1vN0EnKj/5AlAdXg84TDwGHBllHbfA35EcPWyAF/48FDS01K4R1sFIhIH7Q2C2whOHd0MbAKuAW49wWNKgZqI+7XhtGPMbDzBVcptnpJqZjPMrMrMqurr69tZcvdV1CeDmyZW8IdFdazbti/R5YhID9feIPgecIu7F7t7CUEw3HmCx1iUaccG6TWzFOAnwLdO9OLufq+7V7p7ZXFxm3ukeowZHxlKWorxi1e1VSAisdXeIDgjsm8hd98BjD/BY2qB8oj7ZQSd2DXLAU4DXjWzdcAkYKYOGAdKcjO5YcIgfrdwIzU79ie6HBHpwdobBClhZ3MAhH0OtdlhHTAfGGFmQ8wsHbie4MwjANx9t7sXuftgdx8MzAWucPeqDr2DHuxL5w8jxYxfvLo60aWISA/W3iD4D2COmX3PzO4C5hAc4G2VuzcCtxNckbwceMLdl5nZXWZ2xakUnSz652Vy3TnlPLWgho27DiS6HBHpoczdT9wKCE/9vJBg3//L7v5uLAtrTWVlpVdVJc9Gw8ZdB5j641lcf84gvnfVaYkuR0S6KTNb4O5Rd72faPfOMeEXf0K+/JNZad/eXHN2OY/Pr+GrFwynf15moksSkR6mvbuGJIG+MnUYTe788jUdKxCRzqcg6AbKC7L45Fml/HbeBhZu0MBwItK5FATdxN9dMooBeZncfP88FqxXGIhI51EQdBMluZk8NmMShX3SueWBeSxYvyPRJYlID6Eg6EYG5PXm8RmTKeqTHm4ZKAxE5NQpCLqZ/nmZPDZjMiW5wW6iqnUKAxE5NQqCbqh/XiaPfmESJbmZ3PLAPOYrDETkFCgIuqlgy2AS/cIwmLdWYSAiJ0dB0I31Cw8g98/L5LMPzuOtNScaIkJE5IMUBN1cSW4mj31hEgPyMrn1ofkKAxHpMAVBD1CSm8mjM4Iw+OyD85mrMBCRDlAQ9BAlOUEYlOb35tYH5/PmaoWBiLSPgqAHKckJziYqy+/NbQ/NZ87qbYkuSUS6AQVBD1Ock8FvI8OgWmEgIm1TEPRAxTkZPDpjEoMKsrjt4fm8oTAQkTYoCHqooj7BlkFFQTa3PaQwEJHWKQh6sCAMJjKkKAiD2asUBiLyQQqCHq6wTwaPfD4Mg4fnc8+sag43NiW6LBHpQhQESaCwTwaPfmESF3+ohB+/sJLLf/5X9VwqIscoCJJEfnY6v7jxbO67uZK9Bxu55pdv8u3fL2X3gSOJLk1EEkxBkGQuHtOPF795PreeO4RH523go//5Gs8v3YS7J7o0EUkQBUESys5I4zufGMMfvnoexTkZfOWRhXz+4So27jqQ6NJEJAEUBEns9LI8/vDVKXz7sg8xZ/V2Pvqfr3H/7LUcbdLWgUgyURAkubTUFL7wkaH85RsfYeKQAr733Ltcdc8bvLNxd6JLE5E4URAIAOUFWTzw2XO4+9Pj2bT7IFfcPZt/fe5d9h1qTHRpIhJjCgI5xsy4/IyBvPzN87nunEHcN3stl/zkdWat2Jro0kQkhmIaBGY2zcxWmlm1md0RZf6XzGypmS0ys9lmNiaW9Uj75GX14v998nSe/NJkeqencutD8/nqbxeyteFgoksTkRiwWJ02aGapwHvAR4FaYD5wg7u/G9Em1933hL9fAXzF3ae19byVlZVeVVUVk5rlgw41HuV/XlvD3a9Uk5GWwhfPH8qtU4aQnZGW6NJEpAPMbIG7V0abF8stgglAtbuvcffDwGPAlZENmkMglA3odJUuJiMtla9dNII/f/3DTBxawL//5T3O//EsHpi9loNHjia6PBHpBLEMglKgJuJ+bTjtOGb2VTNbDfwI+Fq0JzKzGWZWZWZV9fX1MSlW2ja0uA/33XIOv/vKuYwoyeGu597lwn9/lcfnb6DxqPouEunOYhkEFmXaB9b43f0edx8G/CPwT9GeyN3vdfdKd68sLi7u5DKlI84alM+jMybxyOcnUpybyT8+vZSP/uR1Zi6uo0nXH4h0S7EMglqgPOJ+GVDXRvvHgKtiWI90oinDi3jmK+dy72fOJj01ha89+jaX/eyvvLx8i7qrEOlmYhkE84ERZjbEzNKB64GZkQ3MbETE3Y8Dq2JYj3QyM+OSsf15/m8/zH9ddyYHjhzlcw9X8cn/nqPxkkW6kZid+uHujWZ2O/ACkAo84O7LzOwuoMrdZwK3m9nFwBFgJ3BLrOqR2ElNMa4aX8rHzxjAk1W1/OzlVXz6V29x3vAi/u5jozizvG+iSxSRNsTs9NFY0emjXd/BI0f5zdz13DOrmp37j3DJmH5865JRjOqfk+jSRJJWW6ePKggkZhoOHuGB2eu4769r2Hu4kSvHDeSL5w/jQwNyE12aSNJREEhC7dx3mF++vppfz1nPgSNHmTK8kM+dN4SpI0tISYl2cpmIdDYFgXQJu/Yf5rfzNvDwnHVs2XOIocXZ3DZlCJ86q4ze6amJLk+kR1MQSJdyuLGJ55du4r7Za3hn4x76ZvXipokV3Dy5gpLczESXJ9IjKQikS3J35q3dwf2z1/Li8i2kpRifOGMgt503hNNK8xJdnkiP0lYQqOcwSRgzY+LQQiYOLWTdtn08NGcdT1TV8Lu3NzJpaAGfP28oF47WcQSRWNMWgXQpu/cf4bH5G3hozjo27T7IkKJsbpsymE+dXUZWutZbRE6Wdg1Jt3PkaBN/emcz9/91DYtrd5PXuxc3TBjEDRPKqSjMTnR5It2OgkC6LXdnwfqd3D97LS8s20yTw8QhBVxbWc6lp/fXVoJIOykIpEfYtPsAv1u4kSeqali/fT99MtK4/IwBTK8s56xBfTHTsQSR1igIpEdxd+av28kTVTX8cckmDhw5yrDibK6tLOfqs0opydEpqCItKQikx9p7qJE/LqnjyapaqtbvJDXFuGBUCdMry7hwdAm9UmM6LLdIt6EgkKSwun4vT1bV8vTCWuobDlHUJ52rx5cyvbKckf3U4Z0kNwWBJJXGo0289l49T1bV8tLyLTQ2OePK+zL97DIuO30ABdnpiS5RJO4UBJK0tu09xDNvBweY39uyl9QU49xhhXz89AF8bGx/8hUKkiQUBJL03J1ldXt4fukm/rh0E+u371coSFJREIhEaA6FPy7dxPMtQuHyMwZwyRiFgvQ8CgKRVkQLhbQU49zhRXz89P4KBekxFAQi7RAZCn9csokNOxQK0nMoCEQ6qDkUnlsSbCls2BHsPqqsyOeiD5Vw4egShhX30dXM0m0oCEROgbvzzsY9/OmdTbyyYisrNjcAMKggiwtHl3DB6BImDikgs5dGWZOuS0Eg0ok27jrArBVbmbViK2+s3sbBI01kpacyZXgRF44Othb6aaQ16WIUBCIxcvDIUd5cvZ1XVmzllRVb2bjrAABjB+ZyUbi1MK6srwbXkYRTEIjEgbvz3pa9vLxiC7NWbGXB+p00ORRmpzN1VLClMGV4IX2zdMBZ4k9BIJIAO/cd5vVV9byyYiuvrqxn94EjmAVbC1OGFXHu8CImDC6gd7qOLUjsKQhEEqzxaBOLanbxRvV23li9jbc37OTIUadXqjF+UD7nDS9iyvBCzijrqx5TJSYUBCJdzP7Djcxft5M51dt4Y/U2ltXtwR2y01OZOLSQc4cVMmV4EaP65ej4gnSKtoIgpuP8mdk04KdAKnCfu/+gxfxvAp8HGoF64DZ3Xx/LmkS6gqz0NM4fWcz5I4uBYDfSm2u280b1NuaEB58hOL4weVhhuMVQRFl+b127IJ0uZlsEZpYKvAd8FKgF5gM3uPu7EW0uAN5y9/1m9mVgqrtf19bzaotAkkHdrgPHQuGN6m1sbTgEwIC8TCoHF3DO4HzOGVzAyH45pGqLQdohUVsEE4Bqd18TFvEYcCVwLAjcfVZE+7nATTGsR6TbGNi3N9Mry5leWY67U711L2+u2c68tTuYt3Y7zy6uAyAnM42zK4JQOGdwAWeU5enCNumwWAZBKVATcb8WmNhG+88Bf4o2w8xmADMABg0a1Fn1iXQLZsaIfjmM6JfDzZMH4+7U7jxA1fodzFu7k6p1O3h15UoA0lNTOL0sLwyGfM6uyNfpqnJCsQyCaNurUfdDmdlNQCVwfrT57n4vcC8Eu4Y6q0CR7sjMKC/Iorwgi6vHlwHBMYYF63cyf90O5q/bwf2z1/DL14J/lVH9cqgMdyWdWd6XisIsHWeQ48QyCGqB8oj7ZUBdy0ZmdjHwbeB8dz8Uw3pEeqz87HQuHtOPi8f0A4IrnhfX7KJq/U7mrd3BzEV1PPLWBgD6ZvViXFlfziwPbuPK+2r4ziQXy4PFaQQHiy8CNhIcLP60uy+LaDMeeAqY5u6r2vO8Olgs0nFHm5z3tjSwqGYXi2t2sahmF+9taaAp/PcfVJDFuPK+jCvLY/ygvowdqGMNPU3CriMws8uA/yI4ffQBd/++md0FVLn7TDN7CTgd2BQ+ZIO7X9HWcyoIRDrHvkONLN24+1gwLK7ZRd3ugwCkpRijB+QwrizYYhhf3pdhxX10TUM3pgvKRKRdtu45GIRCbRAOS2p203CoEYA+GWmMGZDL2NJcxg7M47TSXIYV99GV0N1Ewi4oE5HupSQ3k0vG9ueSsf0BaGpy1mzby6KaYMthWd1uHp23gYNHmgBIT0thdP8cxg7MY+zAXE4rzWN0/xztVupmtEUgIh1ytMlZu20vy+r28M7G3Syr28Oyuj3sPnAEgNQUY3hxH8YOzGVMGA5jBuaSm9krwZUnN+0aEpGYar62IQiF3cd+btnz/omAgwqyGN0/h9H9cxjVP5dR/XMYXJhFmnYtxYV2DYlITEVe2zDttP7Hptc3HDoWDO/W7WHF5j28tARYykgAAAylSURBVHzLsbOVMtJSGNGvD6P65YYBEQRFcU6GrnWII20RiEhcHTxylOqte1mxuYGVm/eEPxuO9acEkJ/VKwyFYMthVP8cRvXLITtD664nS1sEItJlZPZK5bTSPE4rzTtu+s59h4+Fw8otDSzf1MATVTXsP3z0WJvSvr0ZVtKHESV9GN58K+5Dvi6IOyUKAhHpEvLDLrcnDys8Nq2pKTj2sGLzHlZubqC6fi/VW/cyb+32Y2cuART1SWdYcRAMQUjkMLykD/1ytYupPRQEItJlpaQYgwqzGFSYdeyUVggCYuOuA1Rv3XvstmprA88urmPPwcZj7XIy0hgWsfUwtCibocXZlBdkkZGmU1ybKQhEpNtJSXn/4PQFo0uOTXd36vceOi4gqrfu5fX36nlqQe37jzcoy89iSFE2Q8JwaP59YF7vpLuCWkEgIj2GmVGSk0lJTibnDis6bt7uA0dYt20fa7ftY034c+22vVSt28G+iOMQGWkpDC4MgyEMiKFhSBRkp/fIXU0KAhFJCnm9ewUd65X3PW66u1PfcCgiHPaxpn4fq7Y28PKKLRw5+v6ZlTkZaZQXZFER7q6qKMimojC4PyCvd7cdLU5BICJJzcwoyc2kJDeTSUMLj5vXeLSJjbsOsCYMhw3b97F+x35Wbm7gpeXHh0SvVKM8vzkgshhUmE1FGBrlBVldutsNBYGISCvSUlOoKMymojCbC0YdP+9ok7Np9wE2bN/P+h37Wb99Pxt27GP99v0sWLfzWGd9zfrnZlJe0Jvy/CzK8ntTlp9FWXh/QF5mQq+wVhCIiJyE1BQLvszzszi3xTx3Z+f+I6zfvo8NYUis376f2p37eWvtDp5ZdODY1dXNz9U/N5Oy/N6UFwRBcSwwCrLon5sZ091OCgIRkU5mZhRkp1OQnc74QfkfmH/kaBObdx+kZsd+anceoGZn+HPHfmav2saWhoNEdvqQlmIM7Nubb10ykivPLO30ehUEIiJx1is15djpr9EcajxK3a6D1O7cT82OA8HPnQco6pMRk3oUBCIiXUxGWuqx6xriQf2/iogkOQWBiEiSUxCIiCQ5BYGISJJTEIiIJDkFgYhIklMQiIgkOQWBiEiS63aD15tZPbD+JB9eBGzrxHI6m+o7Narv1HX1GlXfyatw9+JoM7pdEJwKM6ty98pE19Ea1XdqVN+p6+o1qr7Y0K4hEZEkpyAQEUlyyRYE9ya6gBNQfadG9Z26rl6j6ouBpDpGICIiH5RsWwQiItKCgkBEJMn1yCAws2lmttLMqs3sjijzM8zs8XD+W2Y2OI61lZvZLDNbbmbLzOxvo7SZama7zWxRePtOvOoLX3+dmS0NX7sqynwzs5+Fy2+JmZ0Vx9pGRSyXRWa2x8y+3qJN3JefmT1gZlvN7J2IaQVm9qKZrQp/fnDMwqDdLWGbVWZ2S5xq+7GZrQj/fr83s76tPLbNz0KMa7zTzDZG/B0va+Wxbf6/x7C+xyNqW2dmi1p5bFyW4Slx9x51A1KB1cBQIB1YDIxp0eYrwC/D368HHo9jfQOAs8Lfc4D3otQ3FXgugctwHVDUxvzLgD8BBkwC3krg33ozwYUyCV1+wEeAs4B3Iqb9CLgj/P0O4IdRHlcArAl/5oe/58ehtkuAtPD3H0arrT2fhRjXeCfwd+34DLT5/x6r+lrM/w/gO4lchqdy64lbBBOAandf4+6HgceAK1u0uRJ4OPz9KeAiM7N4FOfum9x9Yfh7A7Ac6PzRqGPrSuDXHpgL9DWzAQmo4yJgtbuf7JXmncbdXwd2tJgc+Tl7GLgqykM/Brzo7jvcfSfwIjAt1rW5+1/cvTG8Oxco68zX7KhWll97tOf//ZS1VV/43XEt8Ghnv2689MQgKAVqIu7X8sEv2mNtwn+G3UBhXKqLEO6SGg+8FWX2ZDNbbGZ/MrOxcS0MHPiLmS0wsxlR5rdnGcfD9bT+z5fI5desn7tvgmAFACiJ0qYrLMvbCLbwojnRZyHWbg93Xz3Qyq61rrD8PgxscfdVrcxP9DI8oZ4YBNHW7FueI9ueNjFlZn2Ap4Gvu/ueFrMXEuzuGAf8HHgmnrUBU9z9LOBS4Ktm9pEW87vC8ksHrgCejDI70cuvIxK6LM3s20Aj8EgrTU70WYil/waGAWcCmwh2v7SU8M8icANtbw0kchm2S08MglqgPOJ+GVDXWhszSwPyOLnN0pNiZr0IQuARd/9dy/nuvsfd94a/Pw/0MrOieNXn7nXhz63A7wk2vyO1ZxnH2qXAQnff0nJGopdfhC3Nu8zCn1ujtEnYsgwPTF8O3OjhzuyW2vFZiBl33+LuR929CfhVK6+d0M9i+P3xSeDx1tokchm2V08MgvnACDMbEq41Xg/MbNFmJtB8dsY1wCut/SN0tnB/4v3Acnf/z1ba9G8+ZmFmEwj+TtvjVF+2meU0/05wUPGdFs1mAjeHZw9NAnY37wKJo1bXwhK5/FqI/JzdAvwhSpsXgEvMLD/c9XFJOC2mzGwa8I/AFe6+v5U27fksxLLGyONOV7fy2u35f4+li4EV7l4bbWail2G7JfpodSxuBGe1vEdwNsG3w2l3EXzoATIJdilUA/OAoXGs7TyCTdclwKLwdhnwJeBLYZvbgWUEZ0DMBc6NY31Dw9ddHNbQvPwi6zPgnnD5LgUq4/z3zSL4Ys+LmJbQ5UcQSpuAIwRrqZ8jOO70MrAq/FkQtq0E7ot47G3hZ7EauDVOtVUT7Ftv/gw2n0U3EHi+rc9CHJff/4afryUEX+4DWtYY3v/A/3s86gunP9T8uYtom5BleCo3dTEhIpLkeuKuIRER6QAFgYhIklMQiIgkOQWBiEiSUxCIiCQ5BYHEhJnNCX8ONrNPd/Jz/99orxUrZnZVrHowNbO9MXreqWb23Ck+x0Nmdk0b8283s1tP5TWka1AQSEy4+7nhr4OBDgWBmaWeoMlxQRDxWrHyD8AvTvVJ2vG+Yi68ErazPAB8rROfTxJEQSAxEbGm+wPgw2Ff7N8ws9SwL/z5YWdiXwzbT7VgnIbfElxEhJk9E3bUtay5sy4z+wHQO3y+RyJfK7zS+cdm9k7Y//t1Ec/9qpk9ZUEf/I9EXHn8AzN7N6zl36O8j5HAIXffFt5/yMx+aWZ/NbP3zOzycHq731eU1/h+2EHeXDPrF/E610S02RvxfK29l2nhtNkE3R40P/ZOM7vXzP4C/LqNWs3M7g6Xxx+J6CQv2nLy4IrkdeHV29KNdebagUg0dxD0Kd/8hTmDoEuKc8wsA3gj/IKCoA+W09x9bXj/NnffYWa9gflm9rS732Fmt7v7mVFe65MEHZSNA4rCx7wezhsPjCXoh+YNYIqZvUvQdcFod3eLPjjLFIJO7CINBs4n6BBtlpkNB27uwPuKlA3Mdfdvm9mPgC8A/xqlXaRo76WKoD+eCwmuGm7Z983ZwHnufqCNv8F4YBRwOtAPeBd4wMwK2lhOVQS9b847Qc3ShWmLQOLtEoJ+ihYRdL9dCIwI581r8WX5NTNr7iaiPKJda84DHvWgo7ItwGvAORHPXetBB2aLCL7M9wAHgfvM7JNAtD53BgD1LaY94e5NHnQ7vAYY3cH3Fekw0Lwvf0FY14lEey+jgbXuvsqD7gJ+0+IxM939QPh7a7V+hPeXXx3wSti+reW0laBLBenGtEUg8WbA37j7cR2rmdlUYF+L+xcDk919v5m9StBH1ImeuzWHIn4/SjA6V2O4W+Migs7KbidYo450gKB32kgt+2Vx2vm+ojji7/fzcpT3/ycbCVfUwl0/6W29l1bqihRZQ2u1XhbtOU6wnDIJlpF0Y9oikFhrIBiSs9kLwJct6IobMxtpQa+MLeUBO8MQGE0wJGazI82Pb+F14LpwH3gxwRpuq7ssLBgTIs+Drqq/TrBbqaXlwPAW06abWYqZDSPoVGxlB95Xe60j2J0DwYhb0d5vpBXAkLAmCHpnbU1rtb4OXB8uvwHABeH8tpbTSLpib5rSIdoikFhbAjSGu3geAn5KsCtjYbimW0/0IRz/DHzJzJYQfNHOjZh3L7DEzBa6+40R038PTCbo6dGBf3D3zWGQRJMD/MHMMgnWkr8Rpc3rwH+YmUWsua8k2O3Uj6DnyYNmdl8731d7/SqsbR5Bz6VtbVUQ1jAD+KOZbQNmA6e10ry1Wn9PsKa/lKA3z9fC9m0tpynAv3T43UmXot5HRU7AzH4KPOvuL5nZQ8Bz7v5UgstKODMbD3zT3T+T6Frk1GjXkMiJ/RvBGAhyvCLgnxNdhJw6bRGIiCQ5bRGIiCQ5BYGISJJTEIiIJDkFgYhIklMQiIgkuf8Plyb1PRXzwOYAAAAASUVORK5CYII=\n", 1236 | "text/plain": [ 1237 | "
" 1238 | ] 1239 | }, 1240 | "metadata": { 1241 | "needs_background": "light" 1242 | }, 1243 | "output_type": "display_data" 1244 | } 1245 | ], 1246 | "source": [ 1247 | "# Plot learning curve (with costs)\n", 1248 | "costs = np.squeeze(d[\"costs\"])\n", 1249 | "plt.plot(costs)\n", 1250 | "plt.ylabel(\"cost\")\n", 1251 | "plt.xlabel(\"iterations (per hundreds)\")\n", 1252 | "plt.title(\"Learning rate =\" + str(d[\"learning_rate\"]))\n", 1253 | "plt.show()" 1254 | ] 1255 | }, 1256 | { 1257 | "cell_type": "markdown", 1258 | "metadata": { 1259 | "colab_type": "text", 1260 | "id": "6cfRp012IgyG" 1261 | }, 1262 | "source": [ 1263 | "**Interpretation**:\n", 1264 | "You can see the cost decreasing. It shows that the parameters are being learned. However, you see that you could train the model even more on the training set. Try to increase the number of iterations in the cell above and rerun the cells. You might see that the training set accuracy goes up, but the test set accuracy goes down. This is called overfitting. " 1265 | ] 1266 | }, 1267 | { 1268 | "cell_type": "markdown", 1269 | "metadata": { 1270 | "colab_type": "text", 1271 | "id": "GUrvwHOxIgyH" 1272 | }, 1273 | "source": [ 1274 | "## 6 - Further analysis (optional/ungraded exercise) ##\n", 1275 | "\n", 1276 | "Congratulations on building your first image classification model. Let's analyze it further, and examine possible choices for the learning rate $\\alpha$. " 1277 | ] 1278 | }, 1279 | { 1280 | "cell_type": "markdown", 1281 | "metadata": { 1282 | "colab_type": "text", 1283 | "id": "-rQquOSlIgyH" 1284 | }, 1285 | "source": [ 1286 | "#### Choice of learning rate ####\n", 1287 | "\n", 1288 | "**Reminder**:\n", 1289 | "In order for Gradient Descent to work you must choose the learning rate wisely. The learning rate $\\alpha$ determines how rapidly we update the parameters. If the learning rate is too large we may \"overshoot\" the optimal value. Similarly, if it is too small we will need too many iterations to converge to the best values. That's why it is crucial to use a well-tuned learning rate.\n", 1290 | "\n", 1291 | "Let's compare the learning curve of our model with several choices of learning rates. Run the cell below. This should take about 1 minute. Feel free also to try different values than the three we have initialized the `learning_rates` variable to contain, and see what happens. " 1292 | ] 1293 | }, 1294 | { 1295 | "cell_type": "code", 1296 | "execution_count": 142, 1297 | "metadata": { 1298 | "colab": { 1299 | "autoexec": { 1300 | "startup": false, 1301 | "wait_interval": 0 1302 | } 1303 | }, 1304 | "colab_type": "code", 1305 | "id": "IfRrjGvOIgyI" 1306 | }, 1307 | "outputs": [ 1308 | { 1309 | "name": "stdout", 1310 | "output_type": "stream", 1311 | "text": [ 1312 | "learning rate is: 0.01\n", 1313 | "train accuracy: 99.52153110047847 %\n", 1314 | "test accuracy: 68.0 %\n", 1315 | "\n", 1316 | "-------------------------------------------------------\n", 1317 | "\n", 1318 | "learning rate is: 0.001\n", 1319 | "train accuracy: 88.99521531100478 %\n", 1320 | "test accuracy: 64.0 %\n", 1321 | "\n", 1322 | "-------------------------------------------------------\n", 1323 | "\n", 1324 | "learning rate is: 0.0001\n", 1325 | "train accuracy: 68.42105263157895 %\n", 1326 | "test accuracy: 36.0 %\n", 1327 | "\n", 1328 | "-------------------------------------------------------\n", 1329 | "\n" 1330 | ] 1331 | }, 1332 | { 1333 | "data": { 1334 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXhU5dnH8e89M5nsmWwQyEZAdoiChEXcRRFX1CqC2mqr0s361talLrWttdbaTWutLVprtYBrK1RpccO6sgRQ2XdIwhpIyJ5Mluf940ySSZgsJBkmk7k/13WuzFnmzD2BzG+e85zzHDHGoJRSKnTZAl2AUkqpwNIgUEqpEKdBoJRSIU6DQCmlQpwGgVJKhThHoAs4XsnJySYrKyvQZSilVFBZvXr1YWNMP1/rgi4IsrKyyM3NDXQZSikVVERkT1vr9NCQUkqFOA0CpZQKcRoESikV4oKuj0Apb263m+3bt1NVVRXoUnqVyMhIhg4ditPpDHQpKghoEKigtn37dhwOBwMHDkREAl1Or2CMoby8nG3btjFmzJhAl6OCgB4aUkGtqqqKmJgYDQEvIkJMTAxVVVVs2bIl0OWoIKBBoIKehsCxRAQR4c0336S0tDTQ5aheToOgm1bvKWbFziOBLkOpNpWVlQW6BNXLaRB0w5HyGr7x/Cru/ee6QJeiAmzZsmWcccYZTJ06lSeffPKY9TU1NXzzm99k6tSpXHLJJeTn5wNQVFTE1VdfzdChQ7nvvvt6vC5tLanO0CDohl/+ZzMlVbXsPFxBWXVtoMtRAVJfX899993H/Pnz+eCDD1i0aBFbt25tsc3ChQuJj4/n008/5dZbb+Xhhx8GICIigrvuuosHH3wwEKUrBWgQdNnynUd4bXUB4zLiAdiwT4/Dhqq1a9eSlZXFoEGDcDqdzJw5k6VLl7bYZunSpVxzzTUAXHrppXz88ccYY4iKimLy5MmEh4cHonSlAD19tEvcdQ088MZ60hMieXLOeM58bBnr95YwZUhSoEsLab//Xz7bCnv2eoJh/SK54+yMdrc5cOAAqampTfMDBw5kzZo1bW7jcDiIi4ujqKiIpCT9P6MCz68tAhGZISJbRGS7iPzIx/pMEVkmImtF5EsRudif9fSUZz/eyfZD5fzs8jFkJEYx0BXBur0lgS5LBYiv+363PjbfmW2UChS/tQhExA48BVwAFACrRGSxMWaj12YPAK8YY54WkdHAEiDLXzX1hPyiSv7w3jYuHJPCtFEpAIxNc2kQ9AIdfXP3l4EDB7Jv376m+f379zNgwACf26SmplJXV0dpaSkJCQknulSlfPJni2ASsN0Ys9MY4wZeAma22sYAcZ7HLmAfvZgxhp8s3oBNhJ9c1nzFZnaai12HKyivqQtgdSpQxo0bx65du8jLy8PtdrNo0SKmT5/eYpvp06fz6quvAvDmm29yxhlnaItA9Rr+7CNIA/K95guAya22+Snwtoh8D4gGzve1IxGZC8wFyMzM7PFCO2vphoO8v/kQ9188itT4yKblY9PiMAY27C1hsvYThByHw8EvfvELrrvuOurr65k9ezYjRozgscce45RTTuHCCy9kzpw53H777UydOpX4+HiefvrppudPmjSJ8vJy3G43S5cuZeHChQwfPjyA70iFGn8Gga+vO60PlM4BnjfG/FZETgNeFJGxxpiGFk8yZh4wDyAnJ+fYg60nQEVNHT/79wZGDojlptOzWqwbm+YCYP2+Ug2CEDVt2jSmTZvWYtndd9/d9DgiIoJ58+b5fO7KlSv9WptSHfHnoaECwPugbTrHHvq5GXgFwBjzGRABJPuxpi57/N2t7C+p5hdXjiXM3vLX1j82gpS4cNZrP4FSKgj5MwhWAcNEZLCIOIHZwOJW2+QB0wBEZBRWEBT6saYu2bS/lOc+2c2cSRlMGJToc5ts7TBWSgUpvwWBMaYOuA1YCmzCOjtog4g8JCKXezb7IXCriHwBLARuMr7OswughgbD/f9ahysyjHtmjGxzu7FpLnYUllOhHcZKqSDj1wvKjDFLsE4J9V72oNfjjcDp/qyhu17OzWdN3lF+c80pxEe1fZOP7DQXxsDG/aVMzPLdalBKqd5Ih5hox5HyGh79z2YmD07kK6emtbtttqfDeF2BHh5SSgUXDYJ2PLJkMxU1dTx8xdgOz/nuHxdB/1jtMFZKBR8NgjYs33mE19cUMPesIQxLie3Uc/QK49DV1WGoAZ588kmmTp3KGWecwQcffNC0/I477iA7O5tzzz33RLwFFcI0CHzwHlTue+cN6/TzGjuMK93aYRxKujMM9datW1m0aBHLli1jwYIF3HvvvdTX1wNw7bXXMn/+/BP+flTo0SDw4ZmPrEHlHpo5hkinvdPPy05z0WCs001V6OjOMNRLly5l5syZhIeHk5mZSVZWFmvXrgVgypQpOh6ROiF0GOpW8osqefJ9a1C580amHNdzvTuM27reQPlP3CeP4DiyqUf3WZc0itLT279zWHeGod6/fz8TJkxo8dwDBw704DtQqmPaIvDS1qBynZUSF05yTDjr9mqLIJR0ZxhqHZ5a9QbaIvDSOKjcA5e0HFSus0SE7LQ4PXMoQDr65u4v3RmGOjU19ZjnpqQcX0tUqe7SFoFHufegclOzuryf7DQX2w6VUeWu77niVK/WnWGop0+fzqJFi6ipqSEvL49du3Yxfvz4QLwNFcI0CDwef6dxULlsHPau/1rGejqMN2qHccjwHob67LPP5rLLLmsahrqx03jOnDkUFxczdepU5s2bx333Wa2XESNGcNlll3HOOedw3XXX8cgjj2C3WycofPvb3+ayyy5jx44dTJgwgQULFgTsPaq+TXrZ0D4dysnJMbm5uT26z437Srnsjx8zKyedX151crf2tb+kitN++T4/u3wMN3ajZaE6Z/Xq1S06alWzffv28b///Y9rr72WtLT2r4xXfZ+IrDbG5PhaF/ItgoYGwwNvdDyoXGcNiIsgKdqp/QRKqaAR8kHQOKjc/RePandQuc4SEb3CWCkVVEI6CA57DSp3VQeDyh0Pq8O4nOpa7TBWSvV+IR0Ev1yymUp3Hb+4suNB5Y7H2DQX9Q1GrzBWSgWFkA2Cz3ZYg8rdeuYQhvbv3KBynZWd7rmHsR4eUkoFgZAMAnddAz9edPyDynVWqiuCxGin9hMopYJCSAZBVweV66zmDmM9NBQq/DEMdVv7fO6555g6dSqpqakcOXLEr+9LhQa/BoGIzBCRLSKyXUR+5GP970Xkc8+0VUSO+rMesAaV+8N725gxZsBxDyp3PLLT4th2sEw7jEOAP4ahbm+fEydO5OWXXyY9Pf2Ev1fVN/ktCETEDjwFXASMBuaIyGjvbYwxdxhjxhljxgFPAv/0Vz2e1+PBRetx2ISfXD664yd0w9hUF3UNhs0Hyvz6Oirw/DEMdXv7zM7OJiMj44S/T9V3+XPQuUnAdmPMTgAReQmYCWxsY/s5wE/8WA9LNxxg2ZZCHrhkFANdxz+o3PEYm9bcYTwuI96vr6UsT29+mh1lO3p0nyfFnsS3R3673W38NQx1R/tUqqf489BQGpDvNV/gWXYMERkEDAbeb2P9XBHJFZHcwsLCLhVzsOQwL7z1Z0YNjOvWoHKdlZ4QSXxUmJ45FAL8MQy1Dk+tTiR/tgh8/a9ta2Cj2cBrxhifB9SNMfOAeWCNNdSVYn7/+tdZn7KL6YmHqKofR6y9Z08Zbc0aklqvMD6ROvrm7i/+Goa6o30q1VP82SIoALwPZKYD+9rYdjaw0I+1cPtlf+DqKnin6CMu/+elLNm5xOe3rp40Ns3F1oNl1NRph3Ff5o9hqDuzT6V6ij+DYBUwTEQGi4gT68N+ceuNRGQEkAB85sdaSO03mAevfIkFB4+SUl3GPR/dw63v3Mrukt1+e83sNBe19YYt2mHcp/ljGOq29gnw7LPPMmHCBPbv38/555/PD3/4w4C9d9U3+HUYahG5GHgcsAPPGWN+ISIPAbnGmMWebX4KRBhjjjm91JduD0O97jXqX7+ZV7Mv4omaPdTU13Bz9s3ckn0L4fbwru/Xh/yiSs58bBm/uHIs108e1KP7VhYdhrptOgy18tbeMNR+vVWlMWYJsKTVsgdbzf/UnzUcI/tq7HvXMHv5U5x/6e/4deVW/vzFn3lr51vcP/l+Tk87vcdeKj0hElekdhgrpXq3kLyymAt+BoPOIPm/9/GrYdfxzPRnsIudb737Le78350cqjzUIy9jXWEcpx3GSqleLTSDwB4G1/wNIhPh5RuY4hrO65e/znfHfZdlecu4/I3Lmb9pPnUNdd1+qbFpLrYcKMNd19ADhStfgu0ueyeCMUZ/L6rTQjMIAGL6w7UvQtkBeP1mnGLnW6d8i3/N/Bfj+o3j0ZWPct1b17GucF23Xqaxw3jrQe0w9ofIyEjKy8v1Q8+LMYaysjJqa2sDXYoKEn7tI+j10nPg4l/Dv/8Plv0Cpj1IZlwmT5//NG/veZvHVj7G9UuuZ9aIWdx+6u3EOeOO+yWyPVcYr9tb0nS1seo5Q4cOZdOmTZSWluoFVx7GGGpra9m1axfGGGy20P2+pzontIMAYMJNsHc1fPRbSB0Poy5DRLgw60JOTz2dpz5/igWbF/DOnne4M+dOLh1y6XF94GQmRhEX4WDd3hLm+O9dhCyn08nQoUN5/vnnqaurIyoqKtAl9RoVFRVERUURH69DnKj26VcFgIt+Damnwr++DYXNo0bGOGO4Z9I9vHTJS6TFpHHfx/dxy9u3sLNkZ6d33TgktZ455D/R0dHMmjWL1NRUREQnz5Sens61115LZKR/x9VSwc+v1xH4Q7evI2hLSQH85WyISoRb3oOIloeB6hvqeX3b6zy+5nGq6qr4+pivM/fkuUQ4Ijrc9S+XbOJvn+xm/c8uxOnQ7FVKnXjtXUegn0qNXOnWmURHdsAb34ZWAWm32Zk1YhaLr1jMjKwZPLPuGa5cdCUfFXzU4a7Hprlw1zdoh7FSqlfSPgJvg8+CCx6Ct++Hj38HZx576X5yZDK/PPOXXDH0Ch5e/jDfee87jEocxYDoAfSL7Ee/qH5NP5Mjk+kf1Z/RqTGANSS1dhgrpXobPTTUmjHw+s2w/p9ww+swdFqbm7rr3by48UVWHljJocpDHK46zNGaY2+yZhc79bXRuMKTGDcwk+TI5ObA8AqNpMgkwmxh/ntvSqmQ1d6hIQ0CX9wV8Oz5ULYf5n4ACVmdf2q9m8NVhymsKuRwpfXzUOUhXv18IzXmKIP613Oo8hDF1cWYVqNyC0JCRAL9IvuRHJVMcoQVDkkRSSRGJpIUkdQ0Hx8ej93W8/dbVkr1TQEbayhoOaPh2n/AM+fCyzfAN94GZ+dOS3TanaTGpJIa03IgtLL9G/n7Z3v45KYLCbPbqG2opaiqqCk0GlsUjQFyqOoQ24u3c6T6iM8rnG1iIz48vikYkiKTSIxoGRZN4RGRRJhdWxpKKd80CNqSdBJc9SwsmAVv3gFX/hm6ccHS2DQX7roGth0sZ3RqHGG2MFKiU0iJTmn3ecYYSt2lHKk+QlFVEUeqj3Ck6kjTz6Jqa1n+oXyKqouoqqvyuZ84Z1xTWCRGJJIQnkBChGfyPE6MSGya1+BQKnRoELRn+HQ451744BFImwCT53Z5V9le9zAendr5K5RFBFe4C1e4iyGuIR1uX1lbeUxYtA6R7Ue3c7T6KEdrjh5zeKpRbFgs8RHxVkCEWwERHxHf9LgxOOLD40mMSCTSEalX9ioVpDQIOnLWXbBvLSy9FwZkw6DTurSbrKRoYsKtK4xnTczo+AldFBUWRVRYFBmxHb9GfUM9Je4SiquLranG+llUXcTRmqMUVRdRXF3M/or9bCzaSHF1MbUNvsevCbeH4wp3ER8eT3x4vM/HCREJLZbHOmOxiZ7BrFSgaRB0xGaDq/4C886FV2+Euf+DuIFd2I0wJjWwQ1IfrXTzu3e2cuuZQ8hIjMJuszcdKuoMYwwVtRU+Q6O4upijNVYro6SmhB1HdzQ9rvd9K2psYiPOGdciLFqHhsvpamoRuZwu4sLjiHJEaetDqR6kQdAZES6r8/jZ8+GVr8FNb4HDedy7yU5z8eLyPdTVN+Cwn/hvwi9+tocXPtvDZzuO8Pp3phIXcXz9ACJCjDOGGGcMGXSuVWOMoay2jJLqkqagaAwI7/mjNUc5WHmQLcVbKKkpabOvA8AhDuLC44hzxh0TEo0/fa2Lc8bhsOl/eaVa07+KzkoZDTP/CK993TpMdMlvj3sXY9Nc1NQ1sL2wnJEDjn8k0+6obzAsXJnHkORodh2u4LYFa3nuxhy/B5KIEOe0PoQ7Gx4A1XXVTYFR6i6ltKaUEndJ03xJjfW4xF1CYWUhO47uoKSmhPLa8nb3GxMWY9XjCYZYZ+yxPz3rGqfGZT19K1Olegu/BoGIzACewLpn8bPGmEd9bDML+ClggC+MMdf5s6ZuGXuV1V/w6R+sQerGX398T28ckrqg5IQHwQdbDrGvpJo/33AqRytr+dE/1/HQmxt5aObYE1pHZ0U4IhjgGMCA6AHH9by6hjrK3GVNIeEdHN5hUuYuo9Rdyp7SPZTWlFLqLqW6vrrdfTttznYDxBXuItYZS0xYDLHO2JZTWKyeiaV6Lb8FgYjYgaeAC4ACYJWILDbGbPTaZhhwL3C6MaZYRPr7q54eM+0nsP9z65TSlNHW0NWdNCQ5mminnfV7S7gmx38dxr7MX5FHv9hwpo1KIcxuY+fhCuZ9uJOT+sVw49SsE1qLPzlsjqazmo6Xu95Nqbu0KSTK3GWU1jTPe68rdZdyuOowu0p2NS1v6wysRhH2CGKcMS3CIdYZ27wsLPaYAGkMlZiwGKLCorRzXfmFP1sEk4DtxpidACLyEjAT2Oi1za3AU8aYYgBjTM/cLNif7A64+m8w7xx4+atW53F0UqeeanUYu054h3FBcSXLthzitnOHEuY5FHTPjJHsLKzgZ//eQGZSFOeO6P0Z7G9Ou5PkyGSSI5OP+7kNpoGK2grK3eWUukspry2nzF3WYmq9rNRdyt7yvU2P2zojq5EgxIRZfTTRYdFNAdG4LMYZQ2xYbMt1zub1sWGxRDujdRgTdQx/BkEakO81XwBMbrXNcAAR+QTr8NFPjTH/bb0jEZkLzAXIzMz0S7HHJToZZr0Az82w+gxu+KcVEJ0wNs3FgpUntsP45VX5CDB7UvPvzm4Tnpg9jmv+/BnfW7CW1789lREDYk9IPX2RTWxN3+IHcvxnlQHU1Nc0h4bbExq11nxFbUWLMGkMnSPVR8gry2t6jrvB3eHrNLZMYsKsQGn66Ww1HxZDtLPlvPcyp/34T5hQvZM/g8DX+X2t284OYBhwDpAOfCQiY40xLUZuM8bMA+aBNdZQz5faBWmnwqW/g0XftabzHoD4jg/3ZKfHUf1JAzsKK07IB29tfQMvrcrn3BH9SYtveYOS6HAHf70ph5l//IRvPL+KRbedTnKMdogGSrg9nPDI8C61SBq5692U15ZbQVJbRoW7grJaKyQal7cIk9pyKmorKCovosLdPN/WKb/ewmxhRIdFHxsmjmiindHWz7CWU1RYVNO23o81VALLn0FQAC1OE0kH9vnYZrkxphbYJSJbsIJhlR/r6jnjb7DuX/DJE7DuVRh9OUz5LmRMbPMp3vcwPhFB8O7GgxSW1XD9FN8tqYGuSJ69MYdZf/mMuS/ksuDWKUSE6WB2wcppd5Jo7/y1Ib4YY6iur25qdTQGRnltOZW1lU1h0Rgq3oFSWFnI7trdVNRWUFFb0WEHfCOHzdEUKFFhUT7DJCosynrsiG66cNJ7vnGbKEeUniZ8nPz521oFDBORwcBeYDbQ+oygN4A5wPMikox1qKjz94HsDc7/CeR8A1b+BVa/ABv+BekTYcp3YNTlxxwyGpwcQ5Snw/jqCel+L2/+ijzS4iM5e3jbfQAnp8fz+1nj+Pb8Ndz92pc8MXucXrAVwkSESEckkY7IbrVOwDqLq7Kukgq3FQwVdRVNIeFragyaytpKSqpL2Fu7l8raSirqrHUddcg3CreHW8Hg8AqIxoDxCoxjHjus7SIdkS3WhdvD+/TfhN+CwBhTJyK3AUuxjv8/Z4zZICIPAbnGmMWeddNFZCNQD9xljDnir5r8Jj4Dpj8MZ98Dny+A5U9bfQeuDJg0F079GkRaNxC3e64wPhH3MN51uIKPtx/mzunDsdva/098UfZA7rpwBL9euoUh/aL5/vnD/V6f6vscNkfT9Rjd1WAaqK6rtoLFKzyq6qpahkldpRUereaPVh+1gsUzX1lXSYNp6NRr28VOlCOKyLDIprDoKDwal/l6HOWIIsIR0WvOAvNr+8kYswRY0mrZg16PDfADzxT8wmNh8jdh4i2wdSks/xO882P44FHrmoPJ34KkkxiT6uLlVfnUN5gOP6C7Y+HKPBw2YVYnT1X9zjknsbOwgsff3cbg5GhmjkvzW21KHS+b2Jq+2Xe3pQLWIbCquiqq6qpahIP3T+8g8d6ucfmhykPW8i60WoCmlleLkGkjOCIdkZyWehojEkd0+723pgfS/MFmh5EXW9P+L6wWQu7fYOUzMOIizk28hudrnewsLGdYin/6Capr63k1N58LRqfQPy6iU88RER65aiz5RZXc9dqXZCRGcWrm8Z+Pr1QwEJGmYEmic6eAd6R1uDSGRGVdJVW1xy5rWue1rKKugsPVh5vDp7ayqa/lQeeDfgkCvUPZiVJ2AFY9C7nPQeURNjQMomrCN8m55BZw9PyZOm+s3cv3X/6cf9w8mTOGHd+3p6IKN1c89QmV7jr+9Z3TyUjs3E15lFL+Ud9QT3V9NXaxE+Ho3Be71tq7Q1nvOEAVCmIHWKeY3rGBhkv/QLjUk7P2Png8G/73GFQc7tGXW7Aij6ykKKaedPzfdBKjnTx300Rq6hq45e+5lFW3f6GTUsq/7DY70WHRXQ6BjmgQnGhhkdhybuSelHn8POFh6x4Hy34Bvx8Di78HhzZ1+yW2Hixj5e4irpucia2LfRBD+8fw9PUT2F5YzvcWrqWuvnOdakqp4KNBECDZ6fEsPDKM+uteg++sgFNmw5evwJ+mwItXwrZ3oaFrH74LVuThtNu4ekL3xjM6Y1gyD80cwwdbCnn4re4HlFKqd9IgCJCxaS4q3fXsOlwO/UfCZU/AHRvhvB/DwY0w/yvw+Fj4732Qv7LToVDpruP1NQVcnD2AxOjuX615/eRB3HzGYJ7/dDcvfra72/tTSvU+etZQgHhfYTy0v+fMoegkOOtOmHo7bFoM61+HVc/A8qcgLg1GXwFjroC0HOvOaT68+cV+yqrruG7yoB6r9b6LR7HrcAU//fdGMpOiOXt4vx7bt1Iq8LRFECAn9YsmIszG+r2lx650OCH7apizEO7aDlfOg4GnWKHw1wvabSnMX7GHYf1jmJjVc6d92m3CH+aMZ1j/GG6bv4ZtB8t6bN9KqcDTIAgQh93GqIGduIdxhAtOubaDULgX8leyvqCYLwpKuH5yZo9fDh8T7uCvN00kPMzON/6+iiPlNT26f6VU4GgQBFB2mouN+0ppaOjktRxthsKz8NcLSP/7JH7qfJGrU/Z3uaO5PWnxkTzztQkcKq3hmy+upqau4xEqlVK9nwZBAI1Nc1FeU8euIxXH/+RWoVB16Z9Y487kBvu7xPzjohYthZ4MhfGZCfx21ink7inmR6+vI9guSFRKHUs7iwOoscN4/d4STuoX0/UdRbh4re4MflwTz79vPZnsis+sUVBXPWuNdxSXBqNnwpgr2+1o7qxLT05lV2EFv31nK0OSo/netGHd2p9SKrA0CAJoWP8Ywh021hWUdGuAN2MM85fvYWxaHGOHpIPMgpNnQXUpbP1vy1CITYVh58NJ02DI2RDZtU7l284bys7DVhgM7hfNpSendrl+pVRgaRAEUKc7jDuwJu8omw+U8ciV2S07iSPirEDwDoVNi2HDIljzAogNUk+Fk86DodMgbQLYO3c/WxHh0a9kk19UyQ9f+YL0hCjGZcR3630opQJD+wgCLDvNxYbj6TD2Yf6KPcSEO7h8XDvfyhtD4dp/wN074Rtvw1l3W2Hw0W/guQvhsSHw0vVW66FoV4evG+6w85evTqB/XDi3vpBLSaWOSaRUMNIgCLBsT4fxnqLKLj3/aKWbN7/czxXjU4kJ72QDz+6AzMlw7r1wyztWMMx6AcZeBfu/hLd+CH8YB0+Mgzd/AJvetFoUPiTFhPP09RM4Ul7D797Z0qX3oJQKLD00FGBj0qw7N63bW8Lg5Ojjfv5rqwtw1zVw3aRuXEkcmWB1Jo+eCcZY92He8b41ffES5P4VxA4Zk6zDSCedB6njrfsuYJ39dP3kQby4fA+zJmYwJtXV9VqUUiecBkGADU+JxemwsX5vCZefcnwdrsYYFqzM49TMeEandv9WgACIQPJQa5o8F+rcULDSCoXt78GyR6zRUiPiYcg5TcFw5/QRvLVuPw8u2sCr3zyty6OeKqVOPL8GgYjMAJ7Aumfxs8aYR1utvwn4NdbN7QH+aIx51p819TZhdhujBsSyruD4O4yX7yxiZ2EFv73mFD9U5uFwQtYZ1jTtQeu+CTs/aG4xbHwDAFfSUF5LPZmndvRn6cdw0ZmnWaGilOr1/BYEImIHngIuAAqAVSKy2BizsdWmLxtjbvNXHcFgbJqLxV/swxhzXENDzF+xB1dkGJecPNCP1bUSnWyNg5R9tXUYqXCz1VLY9SGD89/jt84SeP/PNKxMwZZ5GgyaCplTIGVs06EkpVTv4s8WwSRguzFmJ4CIvATMBFoHQcjLTnMxf0Uee45UktXJfoLCshqWbjjAV6dkEREWoA9YEeg/ypqm3oY0NLB1/SpeeHkh14YVkF2wqqnFgDPW6mMYdBpknmadqhoWGZi6lVIt+DMI0oB8r/kCYLKP7b4iImcBW4E7jDH5rTcQkbnAXIDMzEw/lBpYY72GpO5sELy6Op/aesN1k3vR78NmY/jJk2FnFDNX5PHm985kdFQJ5H1mTXs+g/cf9mwbZnU4NwZDxmSISgxs/UqFKH8Gga9jHK1Plv83sNAYUyMi3wL+Dpx3zJOMmQfMA+vm9VrYflIAACAASURBVD1daKANT4nFabc6jC/rRIdxQ4NhwYo8pgxJZGj/bgxN4Sd3Th/BW1/u5yeL1/PKN09DGi9qA6gsssY/yvsU8pbDZ3+CT56w1vUbZR1GajycFN+LQk6pPsyfQVAAeN8rMR3Y572BMeaI1+wzwK/8WE+v5XTYGDkwlvX7Otdh/OG2QgqKq7hnxkg/V9Y18VFO7pkxkh/9cx3/WruXq05Nb14ZlQgjZlgTQG0V7F3THAzrX4fVf7PWxaVbh5PSTrWugB54CoT3vuBTKth1KghE5BpjzKsdLWtlFTBMRAZjnRU0G7iu1T4GGmP2e2YvB0L2xrhjUl0sWbe/Ux3G81fkkRTt5MIxA05QdcdvVk4GC1fl88iSzZw/OoW4iDaGrgiLhKzTrQmgoR4ObrBCIe9TKMiFDf+01okN+o20QiHNM/UfY53ZpJTqss62CO4FWn/o+1rWxBhTJyK3AUuxTh99zhizQUQeAnKNMYuB20XkcqAOKAJuOs76+4zsNBcLV+aRX1RFZlJUm9vtL6ni/c2HmHvWEJyO3nthuM0m/HzmGGY+9Qm/f2crP7lsTCefaIeBJ1vT5LnWsvJC2LcG9q62Wg9b/wOf/8NaZw+HAdnNrYa0CZA0tNsjrCoVStoNAhG5CLgYSBORP3itisP68G6XMWYJsKTVsge9Ht+LFSghz/sexu0Fwcur8mkwhjkTe//x85PT45kzKZMXPtvDrJwMRg3s4kVvMf1g+IXWBNZpq0f3WKGwbw3sXQtr58PKedb68DhIHefVcphgDcWt1zUo5VNHLYJ9QC7WYZvVXsvLgDv8VVQoGj4ghjC7sG5vSZvXBdTVN/DSynzOHNav3bDoTe6aPoL/rNvPg4s8Hcc98WEsAglZ1jT2KmtZQz0c3mqFw97VVkB89hQ0eAbCi+7fHAqN/Q0x/bpfi1J9QLtBYIz5AvhCRBYYY2oBRCQByDDGFJ+IAkNFuMPOiAGxrG9nSOr3Nx/iQGk1P5vZycMsvUBCtJO7Z4zk3n+u443P93Ll+PSOn9QVNnvzNQ3jr7eW1dXAgfWeVoMnILYupenktZgB1mGlpulkSBysF76pkNPZPoJ3PMfyHcDnQKGI/M8Y8wP/lRZ6stNcLFl3oM0O4/kr8kiJC2fayP4BqK7rrs3J4KWVeVbH8agUYtvqOO5pjnBIn2BNjapLYf8XcGBd87RzGTR4jnSGRUPK6Jbh0H80OIOjBaZUV3Q2CFzGmFIRuQX4mzHmJyLypT8LC0Vj01wsXJlPQXEVGYktP3jyjlTy4bZCbj9vGA57cHWE2mzCQzPHcsWfPuHxd7fx40tHB66YiDgYfKY1NaqrgcItLcNh/euQ+5y1XmxWB/SAbGuojAEnW49jUwLzHpTqYZ0NAoeIDARmAff7sZ6Q5n0P49ZBsHBVHgLMnpTh45m93ykZ8cyemMnzn+7mmpx0Rg7oodFSe4IjvPlMpUbGQEl+y3AoWGUFRKPo/q0OLWVD4knW/R6UCiKd/R/7ENZpoJ8YY1aJyBBgm//KCk0jBsTisFkdxhdlN3cYu+saeDU3n2mjUhjoCt7xee6+cAT/WW8NVf3y3Ck903HsLyLWlc3xmTDykublVUfh4PqWAeHdKW13QvJwq6+i30jrsFL/kRCfpae0ql6rU0HguXDsVa/5ncBX/FVUqAp32BmeEnvMPYzf3niAw+Vuru9N4wp1QUK0k7suHMH9/1rP4i/2MXNcWqBLOn6R8c3Dcjeqc1tnLB1YB4Wb4NAmyFsB67wuswmL8gTE6OZO7f6j9LRW1St09sridOBJ4HSsUy4+Bv7PGFPgx9pCUnaai7c3tuwwnr88j/SESM4aFvynO86emMnLq/J5+K1NnDey/4nrOPYnhxMGjLUmb9WlVt9DYzgc2uS569uC5m3C4zwth5HNIdFvFMT014BQJ0xnDw39DVgAXOOZv8Gz7AJ/FBXKxqa7eDk3n71Hq0hPiGL7oXI+23mEuy4c0Sfu+mX3dBxf+adPeOLdbTwQyI5jf4uIg4yJ1uStssi6j8OhjXBosxUQm96ENS80bxOZ2Nxq6DfSak0kD4fYARoQqsd1Ngj6GWP+5jX/vIh83x8FhTrvDuP0hCgWrszDYRNm5QRnJ7Ev4zLiuTYng799uptrcjIYMSA20CWdWFGJ1girg6Y2LzMGKgq9wmGjFRZfvgI1pc3bOWMheZgnGIY1B0TiYKvTW6ku6GwQHBaRG4CFnvk5wJF2tlddNNKrw/icEf15bXUBF44dQL/YvvVHfveMkfxn/QEeXLSel3p7x/GJIGIdDorpb90LupExULoPjmyDw9usvojDW2H3R/DlS17Pt1tXWicPaxkQycP1Pg+qQ50Ngm8AfwR+j9VH8CnwdX8VFcoiwuwMS4ll3d5S3vpyPyVVtUHfSexLoqfj+IE3grjj+EQQAVeaNQ05p+W6mjI4sh0Ob28OiMPbYMcyqK9p3i4q6dgWRPIwiB+kV1EroPNB8HPgxsZhJUQkEfgNVkCoHpadFse7mw5RXl3LkORoThuSFOiS/GLOpExeWpXHI0s2MW1UCjHhev79cQmPte7yljq+5fKGejia17IFcXgbbF4ClV79ELYwqxWRdJJ1/UPSEM/Pk6x7QejpriGjs395J3uPLWSMKRKR8e09QXVddpqLV3ILKKpw88Alo/rsYRO7Tfj5zLFc+adP+cN727jv4lGBLqlvsNmtPoPEwTB8est1lUWegNgCR3ZA0Q44shN2/g/qqpq3s4d79tEqIBKHQGyqhkQf09kgsIlIQqsWgX5985Mxng5jp8PG1RP8NEhbLzE+M4FrczJ47uNdXDMhnWEpIdZxfKJFJULmZGvy1tAAZfs9weAVEEU7YPu7LQ81OSI9ITHEqzXh+alnNQWlzn6Y/xb4VERew+ojmAX8wm9VhbjRA+NwOmxcmj2Q+Ki+f/etu2c0X3G84NbJfbYF1KvZbM19EYPParmuoR5K9x4bEIVbrNFcG6+qBmvQvoQsKygahwpP8DyOz9S7yfVSYkzn7gUvIqOxbiwvwHvGmI3+LKwtOTk5Jjc3NxAvfUJ9nn+UwUnRuKL6wAVXnfDiZ7v58aINPDlnPJedkhroclRn1ddZYzJ5B0Tx7uaprtprYwFXuldAZHkFxmCITNDWhB+JyGpjTI7PdZ0Ngt4iVIIg1NQ3GC7/48ccLq/hvR+eox3HfUFDA5Qf9ITCruZwKPI8rjjUcvtwFyQMOjYgErKsALGHxpcif2kvCPz61yYiM4AnsO5Z/Kwx5tE2trsaayyjicYY/ZQPQY1XHH/l6U958r1t3Ksdx8HPZoO4gdY06LRj17srWrYeGgPi0CbY+l+odzdvK3brsFX8oObBAL2n2FQd9bUb/PabExE78BTWMBQFwCoRWdz6kJKIxAK3Ayv8VYsKDhMGJXDNhHT++vEurslJZ2h/7Tju05zRkDLGmlprqPd0Xnu1JI7mWdOOZdY6vI5m2BzWAH7xmb7DIi5Vr5lohz8jdBKw3TNSKSLyEjATaN238HPgMeBOP9aigsQ9F41k6YYDPLhoA/Nv0Y7jkGWzW4eDXOktbyLUqK4GSgqaw+HoHq+geM8TFN778wRFwqCWYeHKgPgMiB0Y0oee/BkEaUC+13wB0OKcNc+1CBnGmDdFpM0gEJG5wFyAzMy+d5WtapYcE86dF47gwUUbeGvdfi49WTuOlQ+OcOuU1aSTfK+vrbbOdPIOiKN5ULwHtr0L5Qdabi82Kwwaw8eVboWE93xEfJ/tzPZnEPj6jTW15UTEhjVkxU0d7cgYMw+YB1ZncQ/Vp3qp6ycP4qWV+Tz85ibOHdGfaO04VscrLKLjoCgpsIKidK/1uLGFsXcNbPp3yz4KsAb8axEUrcIiLjVoWxX+/AsrALyHzEwH9nnNxwJjgQ88zf8BwGIRuVw7jEOb3Sb8/IoxfOXpz3jy/e386KKRgS5J9TVhEZA81Jp8aWiwRoMtKbBOj20MipJ8a9q3Bipbj7spLVsVcanW4ShXmvUzLhViUnplX4U/g2AVMExEBgN7gdnAdY0rjTElQHLjvIh8ANypIaAAJgxK5KrxaTz3yS6+dtogUuOD9xadKgjZbBCbYk3pE3xv4670tCa8g8LTqtj/OWxZ0uo6Cqyzn2IHesLBExRxaS1DIwBh4bcgMMbUichtWPc6tgPPGWM2iMhDQK4xZrG/Xlv1DXdcMJw3v9zPE+9u41dXn9zxE5Q6kZxRzcN++2IMVBVb4VC6zwqN0r3Nj/d/CVv+23KMJ2gOi7hUa2pqXaRCWo7Vud3D9IIy1av97N8b+Punu3n7jrMZ2j8m0OUo1bMaw8I7IEq8Hjf+rK20tr/095DTtUGfA3ZBmVLd9d1zh/LKqnx++/YWnr6hjSa6UsFKxBoIMCoRBmT73sYYqD5qhUJ0f7+UoWPJql4tOSacW84cwn/WH+CL/KOBLkepE0/EGocpZQzE9PPLS2gQqF7vljMHkxjt5LGlmwNdilJ9kgaB6vViI8L47rlD+WT7ET7edjjQ5SjV52gQqKBww5RM0uIj+dV/NxNsJzgo1dtpEKigEO6wc8cFw1m3t4T/rD/Q8ROUUp2mQaCCxpXj0xjWP4bfLN1CXX1DoMtRqs/QIFBBw24T7rpwBDsPV/Da6oJAl6NUn6FBoILKBaNTGJ8Zz+PvbqO6tj7Q5SjVJ2gQqKAiItwzYyQHSqt54bPdgS5HqT5Bg0AFnSlDkjh7eD+eWraDkqraQJejVNDTIFBB6a4LR1BSVcszH+4MdClKBT0NAhWUxqa5uOyUVP768S4OlVV3/ASlVJs0CFTQ+uEFw6mtb+CP728PdClKBTUNAhW0spKjuXZiBgtW5LHnSEWgy1EqaGkQqKB2+7RhOOzC797ZGuhSlApaGgQqqKXERfD10wez6PN9bNhXEuhylApKGgQq6H3rrJOIi3Dwm6VbAl2KUkHJr0EgIjNEZIuIbBeRH/lY/y0RWScin4vIxyIy2p/1qL7JFRXGd84dyrIthazYeSTQ5SgVdPwWBCJiB54CLgJGA3N8fNAvMMZkG2PGAY8Bv/NXPapvu/G0LFLiwnls6RYdplqp4+TPFsEkYLsxZqcxxg28BMz03sAYU+o1Gw3oX7Dqkkinnf+bNpzVe4p5b9OhQJejVFDxZxCkAfle8wWeZS2IyHdFZAdWi+B2XzsSkbkikisiuYWFhX4pVgW/a3LSGZwcza+XbqG+Qb9TKNVZ/gwC8bHsmL9OY8xTxpiTgHuAB3ztyBgzzxiTY4zJ6dfPPzdvVsEvzG7jh9OHs+VgGYs+3xvocpQKGv4MggIgw2s+HdjXzvYvAVf4sR4VAi4eO5CxaXH87p2t1NTpMNVKdYY/g2AVMExEBouIE5gNLPbeQESGec1eAmzzYz0qBNhswt0XjqSguIqFK/ICXY5SQcFvQWCMqQNuA5YCm4BXjDEbROQhEbncs9ltIrJBRD4HfgDc6K96VOg4c1gypw1J4sn3t1NeUxfocpTq9STYTrXLyckxubm5gS5D9XJr84q58k+f8oMLhnP7tGEdP0GpPk5EVhtjcnyt0yuLVZ80PjOBC8ekMO/DnRRVuANdjlK9mgaB6rPunD6CSncdf1qmw1Qr1R4NAtVnDUuJ5SunpvPC8j3sPVoV6HKU6rU0CFSf9v0LhoOBJ97VYaqVaosGgerT0uIj+eppg3htdQHbDpYFuhyleiUNAtXnfffcoUQ5HfzmbR2mWilfNAhUn5cY7WTuWUNYuuEga/OKA12OUr2OBoEKCTefMZikaCe/+u9mHaZaqVY0CFRIiA538L3zhrJ8ZxEfbTsc6HKU6lU0CFTImDM5k/SESB5bupkGHaZaqSYaBCpkhDvs/OCC4azfW8orufkdP0GpEKFBoELKzHFpTBmSyP1vrGfJuv2BLkepXkGDQIUUu0149saJjMuI5/aFa1m64UCgS1Iq4DQIVMiJCXfw/NcnMjbNxW0L1vDepoOBLkmpgNIgUCEpNiKMF26exKiBcXz7H2v4YIve8F6FLg0CFbLiIsJ48RuTGZYSw9wXV/PRtsJAl6RUQGgQqJDmigrjHzdPZkhyNLf8PZdPd+g1Bir0aBCokJcQ7WT+LZMZlBTFzc/nsmLnkUCXpNQJ5dcgEJEZIrJFRLaLyI98rP+BiGwUkS9F5D0RGeTPepRqS1JMOPNvmUJqfARff34VubuLAl2SUieM34JAROzAU8BFwGhgjoiMbrXZWiDHGHMy8BrwmL/qUaoj/WLDWXjrFAbERXDT31axRgeoUyHCny2CScB2Y8xOY4wbeAmY6b2BMWaZMabSM7scSPdjPUp1qH9cBAtunUJSjJMb/7qSLwuOBrokpfzOn0GQBnhfx1/gWdaWm4H/+FohInNFJFdEcgsL9cwO5V8DXFYYuKLCuOHZFazfWxLokpTyK38GgfhY5nOkLxG5AcgBfu1rvTFmnjEmxxiT069fvx4sUSnf0uIjWXjrFGIjwrjhryvYuK800CUp5Tf+DIICIMNrPh3Y13ojETkfuB+43BhT48d6lDouGYlRLLh1MpFhdm746wq2HNBbXaq+yZ9BsAoYJiKDRcQJzAYWe28gIuOBv2CFgF7aqXqdQUnRLLh1Cg6bcP2zy9l+SMNA9T1+CwJjTB1wG7AU2AS8YozZICIPicjlns1+DcQAr4rI5yKyuI3dKRUwg5OjWTh3CiDMeWYFOwvLA12SUj1Kgu22fTk5OSY3NzfQZagQtO1gGbPnLcdhF16eexpZydGBLkmpThOR1caYHF/r9MpipTppWEos82+djLuugeueWU5+UWXHT1IqCGgQKHUcRg6I4x+3TKbCXc/secspKNYwUMFPg0Cp4zQm1cU/bp5MaXUt1z2zgv0lVYEuSalu0SBQqguy0128ePNkiivczJm3nIOl1YEuSaku0yBQqovGZcTz/DcmUVhWw5xnlnOoTMNABScNAqW6YcKgBJ7/xiQOlFQze95yXly+h80HSmloCK6z8VRo09NHleoBy3ce4Qcvf86+EqtVEBfhICcrkZysBCZmJZKd5iIizB7gKlUoa+/0UceJLkapvmjKkCQ++dF55BdVsWp3Ebl7ili1u5j3N1sXzDvtNk5Od5GTlcjErAQmDEogPsoZ4KqVsmiLQCk/Kqpws3pPMbm7i1i1u4h1e0uorbf+5kakxDa1GHKyEkiLj0TE11iNSnVfey0CDQKlTqAqdz1fFBz1BEMxa/YUU1ZTB8BAV0RTiyFnUCIjBsRit2kwqJ6hh4aU6iUinXamDEliypAkAOobDFsOlDUdSlq1q4h/f2EN0hsb4WDCoAROSY8nMzGKjMQoMhIjSYmNwKYBoXqQBoFSAWS3CaNT4xidGsfXTsvCGENBcVVTMOTuLuJ/Wwvxbrg77TbSEiJJT4gkPcEKh4wET1AkRJIY7dRDTOq4aBAo1YuIiOebfxRXjrfu3FpTV8++o9XkF1WSX1xJflEV+cWVFBRVsnTfAYoq3C32EeW0e4KhMSisgMhIjCI9IZLYiLBAvDXVi2kQKNXLhTvsDE6OZnAbo52W19RR0BgQXmFRUFzJZzuOUOGub7F9fFQYGQlRpMZHMCAughSX9XNAXAQDXNYU5dSPhlCi/9pKBbmYcAcjB8QxckDcMeuMMRRX1h7TmsgvqmRHYQWfbj/S1FntLTbC0RQMKXHHBkaKK5zk6HDtq+gjNAiU6sNEhMRoJ4nRTk7JiPe5TUVNHQdKqzlYUs2B0upWj2vYdvAwheU11Le6WtphE/rHhjcFRIonOPrFhJMcG05yjJN+seEkRjlx2HUQg95Mg0CpEBcd7uCkfjGc1C+mzW3qGwyHy2s44AmIg6XVLR5vPVjGR9sOU+6jdSECiVFOkmPCSY51WkHRFBbNgdEvJpzEaA2NQNAgUEp1yG4TUjzf+k9pZ7vymjoOl9VwuLyGwsaf5W4Ol9dwuKyGwvIaVucVc7jMTVVt/THPF4GEKGdTOCTHNE9J0U4Sop0kRoeRGG21NGIjHHp4qgf4NQhEZAbwBGAHnjXGPNpq/VnA48DJwGxjzGv+rEcp5V8x4Q5iwh2duo1nRU2dFRCe0CgsdzeFiDW5WZt3lMPlNVS6jw0NsAIqISqMxGgnCVHOpsNgjfNJMc3LE6KdJEU7dcwnH/wWBCJiB54CLgAKgFUistgYs9FrszzgJuBOf9WhlOqdosMdRIc7GJTUcWhUuusoqnBTXFFLUaWboooaiipqKa5wc6TCTXGFm6JKN9sOlVNc4aa40k1bA8BGhtmbwiI+Koz4KCfxkWE+HofhinR6foYR1ocPWfmzRTAJ2G6M2QkgIi8BM4GmIDDG7Pasa/BjHUqpIBfldBDldJCe0LntGxoMpdW1zSHROFW6W4VHLQXFVRytdFNSVdtmeIDV2nF5hUR8pBNXVFhzcHjNu6LCiIsIIy4yjGinvddf4OfPIEgD8r3mC4DJXdmRiMwF5gJkZmZ2vzKlVJ9ms4n17T7KCf0695yGBkNZTR1HK90crazlaFVtU0AcrfRMVW5KPOs2l5Q2ratrJ0HsNiEuwkFcZGM4WIHSGBSN61xe65vXhRERZvN7kPgzCHxV3qUR7owx84B5YA06152ilFLKF5tNcHk+kAcldf55xhgq3PXNAVJZS2l1LaVVjT/rKKnyXlbHodJySqtrKamqpbq2/QMiTrutKRy+f8FwLj8ltZvv9Fj+DIICIMNrPh3Y58fXU0qpE05EmjrJO3voyltNXT1l1XVNIVFS1TJEGgOjtKqWhCj/DA/izyBYBQwTkcHAXmA2cJ0fX08ppYJOuMNOeIyd5JjwgNXgt25wY0wdcBuwFNgEvGKM2SAiD4nI5QAiMlFECoBrgL+IyAZ/1aOUUso3v15HYIxZAixptexBr8ersA4ZKaWUCpC+e2KsUkqpTtEgUEqpEKdBoJRSIU6DQCmlQpwGgVJKhTgNAqWUCnFiTHCN2CAihcCeLj49GTjcg+X4WzDVG0y1QnDVG0y1QnDVG0y1QvfqHWSM8TnyUtAFQXeISK4xJifQdXRWMNUbTLVCcNUbTLVCcNUbTLWC/+rVQ0NKKRXiNAiUUirEhVoQzAt0AccpmOoNplohuOoNplohuOoNplrBT/WGVB+BUkqpY4Vai0AppVQrGgRKKRXiQiYIRGSGiGwRke0i8qNA19MWEckQkWUisklENojI/wW6ps4QEbuIrBWRNwNdS3tEJF5EXhORzZ7f8WmBrqk9InKH5//BehFZKCIRga7Jm4g8JyKHRGS917JEEXlHRLZ5fnbhvl09r41af+35v/CliPxLROIDWWMjX7V6rbtTRIyIJPfU64VEEIiIHXgKuAgYDcwRkdGBrapNdcAPjTGjgCnAd3txrd7+D+sGRL3dE8B/jTEjgVPoxTWLSBpwO5BjjBkL2LHu9NebPA/MaLXsR8B7xphhwHue+d7geY6t9R1grDHmZGArcO+JLqoNz3NsrYhIBnABkNeTLxYSQQBMArYbY3YaY9zAS8DMANfkkzFmvzFmjedxGdYHVVpgq2qfiKQDlwDPBrqW9ohIHHAW8FcAY4zbGHM0sFV1yAFEiogDiKKX3ffbGPMhUNRq8Uzg757HfweuOKFFtcFXrcaYtz13UwRYTi+5UVYbv1eA3wN3Az16lk+oBEEakO81X0Av/3AFEJEsYDywIrCVdOhxrP+cDYEupANDgELgb57DWM+KSHSgi2qLMWYv8Busb3/7gRJjzNuBrapTUowx+8H6YgP0D3A9nfUN4D+BLqItnlv87jXGfNHT+w6VIBAfy3r1ebMiEgO8DnzfGFMa6HraIiKXAoeMMasDXUsnOIBTgaeNMeOBCnrPYYtjeI6tzwQGA6lAtIjcENiq+iYRuR/rsOz8QNfii4hEAfcDD3a0bVeEShAUABle8+n0sia2NxEJwwqB+caYfwa6ng6cDlwuIruxDrmdJyL/CGxJbSoACowxjS2s17CCobc6H9hljCk0xtQC/wSmBrimzjgoIgMBPD8PBbiedonIjcClwPWm915YdRLWF4IvPH9r6cAaERnQEzsPlSBYBQwTkcEi4sTqcFsc4Jp8EhHBOoa9yRjzu0DX0xFjzL3GmHRjTBbW7/V9Y0yv/NZqjDkA5IvICM+iacDGAJbUkTxgiohEef5fTKMXd257WQzc6Hl8I7AogLW0S0RmAPcAlxtjKgNdT1uMMeuMMf2NMVmev7UC4FTP/+luC4kg8HQG3QYsxfpDesUYsyGwVbXpdOCrWN+sP/dMFwe6qD7ke8B8EfkSGAc8EuB62uRpubwGrAHWYf299qohEURkIfAZMEJECkTkZuBR4AIR2YZ1hsujgayxURu1/hGIBd7x/K39OaBFerRRq/9er/e2hJRSSp0IIdEiUEop1TYNAqWUCnEaBEopFeI0CJRSKsRpECilVIjTIFABJSKfen5mich1Pbzv+3y9lr+IyBUi8qDn8fMicrWfXmd3d0aeFJFz2hslVkT6ich/u7p/FXw0CFRAGWMar5TNAo4rCDyjyranRRB4vZa/3A38yc+v4ZNYeuTv2RhTCOwXkdN7Yn+q99MgUAElIuWeh48CZ3ou6rnDc3+DX4vIKs9Y8d/0bH+O534NC7AuskJE3hCR1Z5x++d6lj2KNWrn5yIy3/u1PB+av/aM8b9ORK712vcH0ny/gvmeK3oRkUdFZKOnlt/4eB/DgRpjzGGvxWeJyKcisrOxddD627iI/FFEbvI83i0iPxORNZ66RnqWJ4nI256B8v6CZ+wsTytqk4j8CeuiswwRmS4in3n28apnzKrG+3FsFpGPgau8Xv9srwsX14pIrGfVG8D1XfgnVcHIGKOTTgGbgHLPz3OAN72WzwUe8DwOB3Kxxlo5B2uwuMFe2yZ6fkYC64Ek7337eK2vYI1DbwdSsIZyGOjZdwnWOC42rCs7zwASgS00X4AZ7+N90Pl2tgAAAwNJREFUfB34rdf888Crnv2MxhoG3df7/CNwk+fxbuB7nsffAZ71PP4D8KDn8SVYAyYmY7WiGoApnnXJwIdAtGf+HqxByiKwRt8dhhUirzTWAPwbON3zOAZweB6nAesC/f9DpxMzaYtA9VbTga+JyOdYw3AnYX2QAaw0xuzy2vZ2EfkCazz5DK/t2nIGsNAYU2+MOQj8D5jote8CY0wD8DnWh20pUA08KyJXAb7GpBmINcS1tzeMMQ3GmI1YgdMZjYMMrva8Nlj3UPgHgDHmLaDYa/s9xpjlnsdTsELnE8/v7UZgEDASa/C6bcYY07gvj0+A34nI7VgB1zg2/yGsEU9VCNAgUL2VYH07HueZBpvmsfgrmjYSOQdrlM7TjDGnAGuxvgF3tO+21Hg9rsf6hlyHdXOj17FusuKrI7XKx+t676vxNeto+XfX1nPqsYbNbtTWWDAVXo8FeMfrdzbaGNM4Ro3P5xtjHgVuwWpNLW88HOWpq6qN11R9jAaB6i3KsAb/arQU+LZYQ3IjIsPF901kXECxMabS8yE2xWtdbePzW/kQuNbTD9EP6xv3yrYK8xxndxljlgDfxxqsrrVNwNC2316TPcBoEQkXERfWiKId+RDP8XoRuQho6x7Ay4HTRWSoZ9soT9/FZmCwiJzk2W5O4xNE5CRjjWz5K6zDb41BMBzrMJsKAY6ON1HqhPgSqPMc4nke697CWVhjrgvWYRdftzz8L/AtsUYT3YL1YdhoHvCliKwxxnh3fP4LOA34Auub8t3GmANe34ZbiwUWiXXjeAHu8LHNh8BvRUQ8h198Msbki8grnve7DasF05GfAQtFZA3WYSyf96s1xhR6Op4Xiki4Z/EDxpitnk70t0TkMPAxMNaz/vsici5WC2QjzXfoOhd4qxO1qT5ARx9VqoeIyBPAv40x7wa6lu4SkQ+BmcaY4g43VkFPDw0p1XMewbrBfFDzHC77nYZA6NAWgVJKhThtESilVIjTIFBKqRCnQaCUUiFOg0AppUKcBoFSSoW4/wdmpkemCuQo9QAAAABJRU5ErkJggg==\n", 1335 | "text/plain": [ 1336 | "
" 1337 | ] 1338 | }, 1339 | "metadata": { 1340 | "needs_background": "light" 1341 | }, 1342 | "output_type": "display_data" 1343 | } 1344 | ], 1345 | "source": [ 1346 | "learning_rates = [0.01, 0.001, 0.0001]\n", 1347 | "models = {}\n", 1348 | "for i in learning_rates:\n", 1349 | " print (\"learning rate is: \" + str(i))\n", 1350 | " models[str(i)] = model(train_set_x, train_set_y, test_set_x, test_set_y, num_iterations=1500, learning_rate=i, print_cost=False)\n", 1351 | " print ('\\n' + \"-------------------------------------------------------\" + '\\n')\n", 1352 | "\n", 1353 | "for i in learning_rates:\n", 1354 | " plt.plot(np.squeeze(models[str(i)][\"costs\"]), label=str(models[str(i)][\"learning_rate\"]))\n", 1355 | "\n", 1356 | "plt.ylabel('cost')\n", 1357 | "plt.xlabel('iterations (hundreds)')\n", 1358 | "\n", 1359 | "legend = plt.legend(loc='upper center', shadow=True)\n", 1360 | "frame = legend.get_frame()\n", 1361 | "frame.set_facecolor('0.90')\n", 1362 | "plt.show()" 1363 | ] 1364 | }, 1365 | { 1366 | "cell_type": "markdown", 1367 | "metadata": { 1368 | "colab_type": "text", 1369 | "id": "OvTnLTqGIgyK" 1370 | }, 1371 | "source": [ 1372 | "**Interpretation**: \n", 1373 | "- Different learning rates give different costs and thus different predictions results.\n", 1374 | "- If the learning rate is too large (0.01), the cost may oscillate up and down. It may even diverge (though in this example, using 0.01 still eventually ends up at a good value for the cost). \n", 1375 | "- A lower cost doesn't mean a better model. You have to check if there is possibly overfitting. It happens when the training accuracy is a lot higher than the test accuracy.\n", 1376 | "- We usually recommend that you: \n", 1377 | " - Choose the learning rate that better minimizes the cost function.\n", 1378 | " - If your model overfits, use other techniques to reduce overfitting. \n" 1379 | ] 1380 | } 1381 | ], 1382 | "metadata": { 1383 | "colab": { 1384 | "collapsed_sections": [], 1385 | "default_view": {}, 1386 | "name": "Logistic_Regression.ipynb", 1387 | "provenance": [], 1388 | "version": "0.3.2", 1389 | "views": {} 1390 | }, 1391 | "kernelspec": { 1392 | "display_name": "Python 3", 1393 | "language": "python", 1394 | "name": "python3" 1395 | }, 1396 | "language_info": { 1397 | "codemirror_mode": { 1398 | "name": "ipython", 1399 | "version": 3 1400 | }, 1401 | "file_extension": ".py", 1402 | "mimetype": "text/x-python", 1403 | "name": "python", 1404 | "nbconvert_exporter": "python", 1405 | "pygments_lexer": "ipython3", 1406 | "version": "3.7.6" 1407 | } 1408 | }, 1409 | "nbformat": 4, 1410 | "nbformat_minor": 1 1411 | } 1412 | --------------------------------------------------------------------------------