├── BasicClassification.ipynb ├── CNN.ipynb ├── ClassifyingImages.ipynb ├── Clustering.ipynb ├── EMSegmentation.ipynb ├── EMTopicModel.ipynb ├── GLMnet.ipynb ├── HiDimClassification.ipynb ├── MeanField.ipynb ├── PCA.ipynb ├── README.md ├── Regression.ipynb └── SGDSVM.ipynb /BasicClassification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# * Prerequisites" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "You should familiarize yourself with the `numpy.ndarray` class of python's `numpy` library.\n", 15 | "\n", 16 | "You should be able to answer the following questions before starting this assignment. Let's assume `a` is a numpy array.\n", 17 | "* What is an array's shape (e.g., what is the meaning of `a.shape`)? \n", 18 | "* What is numpy's reshaping operation? How much computational over-head would it induce? \n", 19 | "* What is numpy's transpose operation, and how it is different from reshaping? Does it cause computation overhead?\n", 20 | "* What is the meaning of the commands `a.reshape(-1, 1)` and `a.reshape(-1)`?\n", 21 | "* Would happens to the variable `a` after we call `b = a.reshape(-1)`? Does any of `a`'s attributes change?\n", 22 | "* How do assignments in python and numpy work in general?\n", 23 | " * Does the `b=a` statement use copying by value? Or is it copying by reference?\n", 24 | " * Is the answer to the previous question change depending on whether `a` is a numpy array or a scalar value?\n", 25 | " \n", 26 | "You can answer all of these questions by\n", 27 | "\n", 28 | " 1. Reading numpy's documentation from https://numpy.org/doc/stable/.\n", 29 | " 2. Making trials using dummy variables." 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "# *Assignment Summary" 37 | ] 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "metadata": {}, 42 | "source": [ 43 | "The UC Irvine machine learning data repository hosts a famous collection of data on whether a patient has diabetes (the Pima Indians dataset), originally owned by the National Institute of Diabetes and Digestive and Kidney Diseases and donated by Vincent Sigillito. You can find this data at https://www.kaggle.com/uciml/pima-indians-diabetes-database/data. This data has a set of attributes of patients, and a categorical variable telling whether the patient is diabetic or not. For several attributes in this data set, a value of 0 may indicate a missing value of the variable.\n", 44 | "\n", 45 | "* **Part 1-A)** Build a simple naive Bayes classifier to classify this data set. We will use 20% of the data for evaluation and the other 80% for training. \n", 46 | "\n", 47 | " There are a total of 768 data-points. You should use a normal distribution to model each of the class-conditional distributions. You should write this classifier yourself (it's quite straight-forward).\n", 48 | "\n", 49 | " Report the accuracy of the classifier on the 20% evaluation data, where accuracy is the number of correct predictions as a fraction of total predictions.\n", 50 | "\n", 51 | "* **Part 1-B)** Now adjust your code so that, for attribute 3 (Diastolic blood pressure), attribute 4 (Triceps skin fold thickness), attribute 6 (Body mass index), and attribute 8 (Age), it regards a value of 0 as a missing value when estimating the class-conditional distributions, and the posterior.\n", 52 | "\n", 53 | " Report the accuracy of the classifier on the 20% that was held out for evaluation.\n", 54 | "\n", 55 | "* **Part 1-C)** Now install SVMLight, which you can find at http://svmlight.joachims.org, to train and evaluate an SVM to classify this data.\n", 56 | "\n", 57 | " You don't need to understand much about SVM's to do this as we'll do that in following exercises. You should NOT substitute NA values for zeros for attributes 3, 4, 6, and 8.\n", 58 | " \n", 59 | " Report the accuracy of the classifier on the held out 20%" 60 | ] 61 | }, 62 | { 63 | "cell_type": "markdown", 64 | "metadata": {}, 65 | "source": [ 66 | "# 0. Data" 67 | ] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "metadata": {}, 72 | "source": [ 73 | "## 0.1 Description" 74 | ] 75 | }, 76 | { 77 | "cell_type": "markdown", 78 | "metadata": {}, 79 | "source": [ 80 | "The UC Irvine's Machine Learning Data Repository Department hosts a Kaggle Competition with famous collection of data on whether a patient has diabetes (the Pima Indians dataset), originally owned by the National Institute of Diabetes and Digestive and Kidney Diseases and donated by Vincent Sigillito. \n", 81 | "\n", 82 | "You can find this data at https://www.kaggle.com/uciml/pima-indians-diabetes-database/data. The Kaggle website offers valuable visualizations of the original data dimensions in its dashboard. It is quite insightful to take the time and make sense of the data using their dashboard before applying any method to the data." 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "## 0.2 Information Summary" 90 | ] 91 | }, 92 | { 93 | "cell_type": "markdown", 94 | "metadata": {}, 95 | "source": [ 96 | "* **Input/Output**: This data has a set of attributes of patients, and a categorical variable telling whether the patient is diabetic or not. \n", 97 | "\n", 98 | "* **Missing Data**: For several attributes in this data set, a value of 0 may indicate a missing value of the variable.\n", 99 | "\n", 100 | "* **Final Goal**: We want to build a classifier that can predict whether a patient has diabetes or not. To do this, we will train multiple kinds of models, and will be handing the missing data with different approaches for each method (i.e., some methods will ignore their existence, while others may do something about the missing data)." 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "## 0.3 Loading" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": 46, 113 | "metadata": {}, 114 | "outputs": [], 115 | "source": [ 116 | "%matplotlib inline\n", 117 | "import pandas as pd\n", 118 | "import numpy as np\n", 119 | "import seaborn as sns\n", 120 | "import matplotlib.pyplot as plt\n", 121 | "\n", 122 | "from utils import test_case_checker" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": 47, 128 | "metadata": {}, 129 | "outputs": [ 130 | { 131 | "data": { 132 | "text/html": [ 133 | "
\n", 134 | "\n", 147 | "\n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | " \n", 180 | " \n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " \n", 188 | " \n", 189 | " \n", 190 | " \n", 191 | " \n", 192 | " \n", 193 | " \n", 194 | " \n", 195 | " \n", 196 | " \n", 197 | " \n", 198 | " \n", 199 | " \n", 200 | " \n", 201 | " \n", 202 | " \n", 203 | " \n", 204 | " \n", 205 | " \n", 206 | " \n", 207 | " \n", 208 | " \n", 209 | " \n", 210 | " \n", 211 | " \n", 212 | " \n", 213 | " \n", 214 | " \n", 215 | " \n", 216 | " \n", 217 | " \n", 218 | " \n", 219 | " \n", 220 | " \n", 221 | " \n", 222 | " \n", 223 | " \n", 224 | "
PregnanciesGlucoseBloodPressureSkinThicknessInsulinBMIDiabetesPedigreeFunctionAgeOutcome
061487235033.60.627501
11856629026.60.351310
28183640023.30.672321
318966239428.10.167210
40137403516843.12.288331
\n", 225 | "
" 226 | ], 227 | "text/plain": [ 228 | " Pregnancies Glucose BloodPressure SkinThickness Insulin BMI \\\n", 229 | "0 6 148 72 35 0 33.6 \n", 230 | "1 1 85 66 29 0 26.6 \n", 231 | "2 8 183 64 0 0 23.3 \n", 232 | "3 1 89 66 23 94 28.1 \n", 233 | "4 0 137 40 35 168 43.1 \n", 234 | "\n", 235 | " DiabetesPedigreeFunction Age Outcome \n", 236 | "0 0.627 50 1 \n", 237 | "1 0.351 31 0 \n", 238 | "2 0.672 32 1 \n", 239 | "3 0.167 21 0 \n", 240 | "4 2.288 33 1 " 241 | ] 242 | }, 243 | "execution_count": 47, 244 | "metadata": {}, 245 | "output_type": "execute_result" 246 | } 247 | ], 248 | "source": [ 249 | "df = pd.read_csv('diabetes.csv')\n", 250 | "df.head()" 251 | ] 252 | }, 253 | { 254 | "cell_type": "markdown", 255 | "metadata": {}, 256 | "source": [ 257 | "## 0.1 Splitting The Data" 258 | ] 259 | }, 260 | { 261 | "cell_type": "markdown", 262 | "metadata": {}, 263 | "source": [ 264 | "First, we will shuffle the data completely, and forget about the order in the original csv file. \n", 265 | "\n", 266 | "* The training and evaluation dataframes will be named ```train_df``` and ```eval_df```, respectively.\n", 267 | "\n", 268 | "* We will also create the 2-d numpy array `train_features` whose number of rows is the number of training samples, and the number of columns is 8 (i.e., the number of features). We will define `eval_features` in a similar fashion\n", 269 | "\n", 270 | "* We would also create the 1-d numpy arrays `train_labels` and `eval_labels` which contain the training and evaluation labels, respectively." 271 | ] 272 | }, 273 | { 274 | "cell_type": "code", 275 | "execution_count": 48, 276 | "metadata": {}, 277 | "outputs": [], 278 | "source": [ 279 | "# Let's generate the split ourselves.\n", 280 | "np_random = np.random.RandomState(seed=12345)\n", 281 | "rand_unifs = np_random.uniform(0,1,size=df.shape[0])\n", 282 | "division_thresh = np.percentile(rand_unifs, 80)\n", 283 | "train_indicator = rand_unifs < division_thresh\n", 284 | "eval_indicator = rand_unifs >= division_thresh" 285 | ] 286 | }, 287 | { 288 | "cell_type": "code", 289 | "execution_count": 49, 290 | "metadata": {}, 291 | "outputs": [ 292 | { 293 | "data": { 294 | "text/html": [ 295 | "
\n", 296 | "\n", 309 | "\n", 310 | " \n", 311 | " \n", 312 | " \n", 313 | " \n", 314 | " \n", 315 | " \n", 316 | " \n", 317 | " \n", 318 | " \n", 319 | " \n", 320 | " \n", 321 | " \n", 322 | " \n", 323 | " \n", 324 | " \n", 325 | " \n", 326 | " \n", 327 | " \n", 328 | " \n", 329 | " \n", 330 | " \n", 331 | " \n", 332 | " \n", 333 | " \n", 334 | " \n", 335 | " \n", 336 | " \n", 337 | " \n", 338 | " \n", 339 | " \n", 340 | " \n", 341 | " \n", 342 | " \n", 343 | " \n", 344 | " \n", 345 | " \n", 346 | " \n", 347 | " \n", 348 | " \n", 349 | " \n", 350 | " \n", 351 | " \n", 352 | " \n", 353 | " \n", 354 | " \n", 355 | " \n", 356 | " \n", 357 | " \n", 358 | " \n", 359 | " \n", 360 | " \n", 361 | " \n", 362 | " \n", 363 | " \n", 364 | " \n", 365 | " \n", 366 | " \n", 367 | " \n", 368 | " \n", 369 | " \n", 370 | " \n", 371 | " \n", 372 | " \n", 373 | " \n", 374 | " \n", 375 | " \n", 376 | " \n", 377 | " \n", 378 | " \n", 379 | " \n", 380 | " \n", 381 | " \n", 382 | " \n", 383 | " \n", 384 | " \n", 385 | " \n", 386 | "
PregnanciesGlucoseBloodPressureSkinThicknessInsulinBMIDiabetesPedigreeFunctionAgeOutcome
01856629026.60.351310
18183640023.30.672321
218966239428.10.167210
30137403516843.12.288331
45116740025.60.201300
\n", 387 | "
" 388 | ], 389 | "text/plain": [ 390 | " Pregnancies Glucose BloodPressure SkinThickness Insulin BMI \\\n", 391 | "0 1 85 66 29 0 26.6 \n", 392 | "1 8 183 64 0 0 23.3 \n", 393 | "2 1 89 66 23 94 28.1 \n", 394 | "3 0 137 40 35 168 43.1 \n", 395 | "4 5 116 74 0 0 25.6 \n", 396 | "\n", 397 | " DiabetesPedigreeFunction Age Outcome \n", 398 | "0 0.351 31 0 \n", 399 | "1 0.672 32 1 \n", 400 | "2 0.167 21 0 \n", 401 | "3 2.288 33 1 \n", 402 | "4 0.201 30 0 " 403 | ] 404 | }, 405 | "execution_count": 49, 406 | "metadata": {}, 407 | "output_type": "execute_result" 408 | } 409 | ], 410 | "source": [ 411 | "train_df = df[train_indicator].reset_index(drop=True)\n", 412 | "train_features = train_df.loc[:, train_df.columns != 'Outcome'].values\n", 413 | "train_labels = train_df['Outcome'].values\n", 414 | "train_df.head()" 415 | ] 416 | }, 417 | { 418 | "cell_type": "code", 419 | "execution_count": 50, 420 | "metadata": {}, 421 | "outputs": [ 422 | { 423 | "data": { 424 | "text/html": [ 425 | "
\n", 426 | "\n", 439 | "\n", 440 | " \n", 441 | " \n", 442 | " \n", 443 | " \n", 444 | " \n", 445 | " \n", 446 | " \n", 447 | " \n", 448 | " \n", 449 | " \n", 450 | " \n", 451 | " \n", 452 | " \n", 453 | " \n", 454 | " \n", 455 | " \n", 456 | " \n", 457 | " \n", 458 | " \n", 459 | " \n", 460 | " \n", 461 | " \n", 462 | " \n", 463 | " \n", 464 | " \n", 465 | " \n", 466 | " \n", 467 | " \n", 468 | " \n", 469 | " \n", 470 | " \n", 471 | " \n", 472 | " \n", 473 | " \n", 474 | " \n", 475 | " \n", 476 | " \n", 477 | " \n", 478 | " \n", 479 | " \n", 480 | " \n", 481 | " \n", 482 | " \n", 483 | " \n", 484 | " \n", 485 | " \n", 486 | " \n", 487 | " \n", 488 | " \n", 489 | " \n", 490 | " \n", 491 | " \n", 492 | " \n", 493 | " \n", 494 | " \n", 495 | " \n", 496 | " \n", 497 | " \n", 498 | " \n", 499 | " \n", 500 | " \n", 501 | " \n", 502 | " \n", 503 | " \n", 504 | " \n", 505 | " \n", 506 | " \n", 507 | " \n", 508 | " \n", 509 | " \n", 510 | " \n", 511 | " \n", 512 | " \n", 513 | " \n", 514 | " \n", 515 | " \n", 516 | "
PregnanciesGlucoseBloodPressureSkinThicknessInsulinBMIDiabetesPedigreeFunctionAgeOutcome
061487235033.60.627501
137850328831.00.248261
210168740038.00.537341
30118844723045.80.551311
47107740029.60.254311
\n", 517 | "
" 518 | ], 519 | "text/plain": [ 520 | " Pregnancies Glucose BloodPressure SkinThickness Insulin BMI \\\n", 521 | "0 6 148 72 35 0 33.6 \n", 522 | "1 3 78 50 32 88 31.0 \n", 523 | "2 10 168 74 0 0 38.0 \n", 524 | "3 0 118 84 47 230 45.8 \n", 525 | "4 7 107 74 0 0 29.6 \n", 526 | "\n", 527 | " DiabetesPedigreeFunction Age Outcome \n", 528 | "0 0.627 50 1 \n", 529 | "1 0.248 26 1 \n", 530 | "2 0.537 34 1 \n", 531 | "3 0.551 31 1 \n", 532 | "4 0.254 31 1 " 533 | ] 534 | }, 535 | "execution_count": 50, 536 | "metadata": {}, 537 | "output_type": "execute_result" 538 | } 539 | ], 540 | "source": [ 541 | "eval_df = df[eval_indicator].reset_index(drop=True)\n", 542 | "eval_features = eval_df.loc[:, eval_df.columns != 'Outcome'].values\n", 543 | "eval_labels = eval_df['Outcome'].values\n", 544 | "eval_df.head()" 545 | ] 546 | }, 547 | { 548 | "cell_type": "code", 549 | "execution_count": 51, 550 | "metadata": {}, 551 | "outputs": [ 552 | { 553 | "data": { 554 | "text/plain": [ 555 | "((614, 8), (614,), (154, 8), (154,))" 556 | ] 557 | }, 558 | "execution_count": 51, 559 | "metadata": {}, 560 | "output_type": "execute_result" 561 | } 562 | ], 563 | "source": [ 564 | "train_features.shape, train_labels.shape, eval_features.shape, eval_labels.shape" 565 | ] 566 | }, 567 | { 568 | "cell_type": "markdown", 569 | "metadata": {}, 570 | "source": [ 571 | "## 0.2 Pre-processing The Data" 572 | ] 573 | }, 574 | { 575 | "cell_type": "markdown", 576 | "metadata": {}, 577 | "source": [ 578 | "Some of the columns exhibit missing values. We will use a Naive Bayes Classifier later that will treat such missing values in a special way. To be specific, for attribute 3 (Diastolic blood pressure), attribute 4 (Triceps skin fold thickness), attribute 6 (Body mass index), and attribute 8 (Age), we should regard a value of 0 as a missing value.\n", 579 | "\n", 580 | "Therefore, we will be creating the `train_featues_with_nans` and `eval_features_with_nans` numpy arrays to be just like their `train_features` and `eval_features` counter-parts, but with the zero-values in such columns replaced with nans." 581 | ] 582 | }, 583 | { 584 | "cell_type": "code", 585 | "execution_count": 52, 586 | "metadata": {}, 587 | "outputs": [], 588 | "source": [ 589 | "train_df_with_nans = train_df.copy(deep=True)\n", 590 | "eval_df_with_nans = eval_df.copy(deep=True)\n", 591 | "for col_with_nans in ['BloodPressure', 'SkinThickness', 'BMI', 'Age']:\n", 592 | " train_df_with_nans[col_with_nans] = train_df_with_nans[col_with_nans].replace(0, np.nan)\n", 593 | " eval_df_with_nans[col_with_nans] = eval_df_with_nans[col_with_nans].replace(0, np.nan)\n", 594 | "train_features_with_nans = train_df_with_nans.loc[:, train_df_with_nans.columns != 'Outcome'].values\n", 595 | "eval_features_with_nans = eval_df_with_nans.loc[:, eval_df_with_nans.columns != 'Outcome'].values" 596 | ] 597 | }, 598 | { 599 | "cell_type": "code", 600 | "execution_count": 53, 601 | "metadata": {}, 602 | "outputs": [ 603 | { 604 | "name": "stdout", 605 | "output_type": "stream", 606 | "text": [ 607 | "Here are the training rows with at least one missing values.\n", 608 | "\n", 609 | "You can see that such incomplete data points constitute a substantial part of the data.\n", 610 | "\n" 611 | ] 612 | }, 613 | { 614 | "data": { 615 | "text/html": [ 616 | "
\n", 617 | "\n", 630 | "\n", 631 | " \n", 632 | " \n", 633 | " \n", 634 | " \n", 635 | " \n", 636 | " \n", 637 | " \n", 638 | " \n", 639 | " \n", 640 | " \n", 641 | " \n", 642 | " \n", 643 | " \n", 644 | " \n", 645 | " \n", 646 | " \n", 647 | " \n", 648 | " \n", 649 | " \n", 650 | " \n", 651 | " \n", 652 | " \n", 653 | " \n", 654 | " \n", 655 | " \n", 656 | " \n", 657 | " \n", 658 | " \n", 659 | " \n", 660 | " \n", 661 | " \n", 662 | " \n", 663 | " \n", 664 | " \n", 665 | " \n", 666 | " \n", 667 | " \n", 668 | " \n", 669 | " \n", 670 | " \n", 671 | " \n", 672 | " \n", 673 | " \n", 674 | " \n", 675 | " \n", 676 | " \n", 677 | " \n", 678 | " \n", 679 | " \n", 680 | " \n", 681 | " \n", 682 | " \n", 683 | " \n", 684 | " \n", 685 | " \n", 686 | " \n", 687 | " \n", 688 | " \n", 689 | " \n", 690 | " \n", 691 | " \n", 692 | " \n", 693 | " \n", 694 | " \n", 695 | " \n", 696 | " \n", 697 | " \n", 698 | " \n", 699 | " \n", 700 | " \n", 701 | " \n", 702 | " \n", 703 | " \n", 704 | " \n", 705 | " \n", 706 | " \n", 707 | " \n", 708 | " \n", 709 | " \n", 710 | " \n", 711 | " \n", 712 | " \n", 713 | " \n", 714 | " \n", 715 | " \n", 716 | " \n", 717 | " \n", 718 | " \n", 719 | " \n", 720 | " \n", 721 | " \n", 722 | " \n", 723 | " \n", 724 | " \n", 725 | " \n", 726 | " \n", 727 | " \n", 728 | " \n", 729 | " \n", 730 | " \n", 731 | " \n", 732 | " \n", 733 | " \n", 734 | " \n", 735 | " \n", 736 | " \n", 737 | " \n", 738 | " \n", 739 | " \n", 740 | " \n", 741 | " \n", 742 | " \n", 743 | " \n", 744 | " \n", 745 | " \n", 746 | " \n", 747 | " \n", 748 | " \n", 749 | " \n", 750 | " \n", 751 | " \n", 752 | " \n", 753 | " \n", 754 | " \n", 755 | " \n", 756 | " \n", 757 | " \n", 758 | " \n", 759 | " \n", 760 | " \n", 761 | " \n", 762 | " \n", 763 | " \n", 764 | " \n", 765 | " \n", 766 | " \n", 767 | " \n", 768 | " \n", 769 | " \n", 770 | " \n", 771 | " \n", 772 | " \n", 773 | " \n", 774 | " \n", 775 | " \n", 776 | " \n", 777 | " \n", 778 | " \n", 779 | "
PregnanciesGlucoseBloodPressureSkinThicknessInsulinBMIDiabetesPedigreeFunctionAgeOutcome
1818364.0NaN023.30.672321
4511674.0NaN025.60.201300
510115NaNNaN035.30.134290
7812596.0NaN0NaN0.232541
8411092.0NaN037.60.191300
..............................
598616262.0NaN024.30.178501
599413670.0NaN031.21.182221
605110676.0NaN037.50.197260
606619092.0NaN035.50.278661
612112660.0NaN030.10.349471
\n", 780 | "

186 rows × 9 columns

\n", 781 | "
" 782 | ], 783 | "text/plain": [ 784 | " Pregnancies Glucose BloodPressure SkinThickness Insulin BMI \\\n", 785 | "1 8 183 64.0 NaN 0 23.3 \n", 786 | "4 5 116 74.0 NaN 0 25.6 \n", 787 | "5 10 115 NaN NaN 0 35.3 \n", 788 | "7 8 125 96.0 NaN 0 NaN \n", 789 | "8 4 110 92.0 NaN 0 37.6 \n", 790 | ".. ... ... ... ... ... ... \n", 791 | "598 6 162 62.0 NaN 0 24.3 \n", 792 | "599 4 136 70.0 NaN 0 31.2 \n", 793 | "605 1 106 76.0 NaN 0 37.5 \n", 794 | "606 6 190 92.0 NaN 0 35.5 \n", 795 | "612 1 126 60.0 NaN 0 30.1 \n", 796 | "\n", 797 | " DiabetesPedigreeFunction Age Outcome \n", 798 | "1 0.672 32 1 \n", 799 | "4 0.201 30 0 \n", 800 | "5 0.134 29 0 \n", 801 | "7 0.232 54 1 \n", 802 | "8 0.191 30 0 \n", 803 | ".. ... ... ... \n", 804 | "598 0.178 50 1 \n", 805 | "599 1.182 22 1 \n", 806 | "605 0.197 26 0 \n", 807 | "606 0.278 66 1 \n", 808 | "612 0.349 47 1 \n", 809 | "\n", 810 | "[186 rows x 9 columns]" 811 | ] 812 | }, 813 | "execution_count": 53, 814 | "metadata": {}, 815 | "output_type": "execute_result" 816 | } 817 | ], 818 | "source": [ 819 | "print('Here are the training rows with at least one missing values.')\n", 820 | "print('')\n", 821 | "print('You can see that such incomplete data points constitute a substantial part of the data.')\n", 822 | "print('')\n", 823 | "nan_training_data = train_df_with_nans[train_df_with_nans.isna().any(axis=1)]\n", 824 | "nan_training_data" 825 | ] 826 | }, 827 | { 828 | "cell_type": "markdown", 829 | "metadata": {}, 830 | "source": [ 831 | "# 1. Part 1 (Building a simple Naive Bayes Classifier)" 832 | ] 833 | }, 834 | { 835 | "cell_type": "markdown", 836 | "metadata": {}, 837 | "source": [ 838 | "Consider a single sample $(\\mathbf{x}, y)$, where the feature vector is denoted with $\\mathbf{x}$, and the label is denoted with $y$. We will also denote the $j^{th}$ feature of $\\mathbf{x}$ with $x^{(j)}$.\n", 839 | "\n", 840 | "According to the textbook, the Naive Bayes Classifier uses the following decision rule:\n", 841 | "\n", 842 | "\"Choose $y$ such that $$\\bigg[\\log p(y) + \\sum_{j} \\log p(x^{(j)}|y) \\bigg]$$ is the largest\"\n", 843 | "\n", 844 | "However, we first need to define the probabilistic models of the prior $p(y)$ and the class-conditional feature distributions $p(x^{(j)}|y)$ using the training data.\n", 845 | "\n", 846 | "* **Modelling the prior $p(y)$**: We fit a Bernoulli distribution to the `Outcome` variable of `train_df`.\n", 847 | "* **Modelling the class-conditional feature distributions $p(x^{(j)}|y)$**: We fit Gaussian distributions, and infer the Gaussian mean and variance parameters from `train_df`." 848 | ] 849 | }, 850 | { 851 | "cell_type": "markdown", 852 | "metadata": {}, 853 | "source": [ 854 | "# Task 1" 855 | ] 856 | }, 857 | { 858 | "cell_type": "markdown", 859 | "metadata": {}, 860 | "source": [ 861 | "Write a function `log_prior` that takes a numpy array `train_labels` as input, and outputs the following vector as a column numpy array (i.e., with shape $(2,1)$).\n", 862 | "\n", 863 | "$$\\log p_y =\\begin{bmatrix}\\log p(y=0)\\\\\\log p(y=1)\\end{bmatrix}$$\n", 864 | "\n", 865 | "Try and avoid the utilization of loops as much as possible. No loops are necessary.\n", 866 | "\n", 867 | "**Hint**: Make sure all the array shapes are what you need and expect. You can reshape any numpy array without any tangible computational over-head." 868 | ] 869 | }, 870 | { 871 | "cell_type": "code", 872 | "execution_count": 54, 873 | "metadata": { 874 | "deletable": false, 875 | "nbgrader": { 876 | "cell_type": "code", 877 | "checksum": "071dcd7013b592e1fc344ddc31bedc4e", 878 | "grade": false, 879 | "grade_id": "cell-540952d95c213032", 880 | "locked": false, 881 | "schema_version": 3, 882 | "solution": true, 883 | "task": false 884 | } 885 | }, 886 | "outputs": [], 887 | "source": [ 888 | "def log_prior(train_labels):\n", 889 | " \n", 890 | " # your code here\n", 891 | " num0 = 0\n", 892 | " num1 = 0\n", 893 | " for label in train_labels:\n", 894 | " if label == 0:\n", 895 | " num0 += 1\n", 896 | " else:\n", 897 | " num1 += 1\n", 898 | " log_py = np.log([[num0/len(train_labels)],[num1/len(train_labels)]])\n", 899 | " \n", 900 | " assert log_py.shape == (2,1)\n", 901 | " \n", 902 | " return log_py" 903 | ] 904 | }, 905 | { 906 | "cell_type": "code", 907 | "execution_count": 55, 908 | "metadata": { 909 | "deletable": false, 910 | "editable": false, 911 | "nbgrader": { 912 | "cell_type": "code", 913 | "checksum": "20d512227df8d765b37255ebdca0bbde", 914 | "grade": true, 915 | "grade_id": "cell-7c3e85395bcc5892", 916 | "locked": true, 917 | "points": 1, 918 | "schema_version": 3, 919 | "solution": false, 920 | "task": false 921 | } 922 | }, 923 | "outputs": [], 924 | "source": [ 925 | "some_labels = np.array([0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1])\n", 926 | "some_log_py = log_prior(some_labels)\n", 927 | "assert np.array_equal(some_log_py.round(3), np.array([[-0.916], [-0.511]]))\n", 928 | "\n", 929 | "# Checking against the pre-computed test database\n", 930 | "test_results = test_case_checker(log_prior, task_id=1)\n", 931 | "assert test_results['passed'], test_results['message']" 932 | ] 933 | }, 934 | { 935 | "cell_type": "code", 936 | "execution_count": 56, 937 | "metadata": {}, 938 | "outputs": [ 939 | { 940 | "data": { 941 | "text/plain": [ 942 | "array([[-0.41610786],\n", 943 | " [-1.07766068]])" 944 | ] 945 | }, 946 | "execution_count": 56, 947 | "metadata": {}, 948 | "output_type": "execute_result" 949 | } 950 | ], 951 | "source": [ 952 | "log_py = log_prior(train_labels)\n", 953 | "log_py" 954 | ] 955 | }, 956 | { 957 | "cell_type": "markdown", 958 | "metadata": {}, 959 | "source": [ 960 | "# Task 2" 961 | ] 962 | }, 963 | { 964 | "cell_type": "markdown", 965 | "metadata": {}, 966 | "source": [ 967 | "Write a function `cc_mean_ignore_missing` that takes the numpy arrays `train_features` and `train_labels` as input, and outputs the following matrix with the shape $(8,2)$, where 8 is the number of features.\n", 968 | "\n", 969 | "$$\\mu_y = \\begin{bmatrix} \\mathbb{E}[x^{(0)}|y=0] & \\mathbb{E}[x^{(0)}|y=1]\\\\\n", 970 | "\\mathbb{E}[x^{(1)}|y=0] & \\mathbb{E}[x^{(1)}|y=1] \\\\\n", 971 | "\\cdots & \\cdots\\\\\n", 972 | "\\mathbb{E}[x^{(7)}|y=0] & \\mathbb{E}[x^{(7)}|y=1]\\end{bmatrix}$$\n", 973 | "\n", 974 | "Some points regarding this task:\n", 975 | "\n", 976 | "* The `train_features` numpy array has a shape of `(N,8)` where `N` is the number of training data points, and 8 is the number of the features. \n", 977 | "\n", 978 | "* The `train_labels` numpy array has a shape of `(N,)`. \n", 979 | "\n", 980 | "* **You can assume that `train_features` has no missing elements in this task**.\n", 981 | "\n", 982 | "* Try and avoid the utilization of loops as much as possible. No loops are necessary." 983 | ] 984 | }, 985 | { 986 | "cell_type": "code", 987 | "execution_count": 57, 988 | "metadata": { 989 | "deletable": false, 990 | "nbgrader": { 991 | "cell_type": "code", 992 | "checksum": "48bacfdecfbecc35ccca01e2e264d3ef", 993 | "grade": false, 994 | "grade_id": "cell-9482e9412e53e401", 995 | "locked": false, 996 | "schema_version": 3, 997 | "solution": true, 998 | "task": false 999 | } 1000 | }, 1001 | "outputs": [], 1002 | "source": [ 1003 | "def cc_mean_ignore_missing(train_features, train_labels):\n", 1004 | " N, d = train_features.shape\n", 1005 | " \n", 1006 | " # your code here\n", 1007 | " pos = np.sum(train_labels)\n", 1008 | " neg = N - pos\n", 1009 | " train_labels = train_labels.reshape(-1,1)\n", 1010 | " \n", 1011 | " positives = train_features * train_labels\n", 1012 | " train_opps = (train_labels == 0)\n", 1013 | " pos_avgs = np.sum(positives, axis=0) / pos\n", 1014 | " pos_avgs = pos_avgs.reshape(-1,1)\n", 1015 | " \n", 1016 | " negatives = train_features * train_opps\n", 1017 | " neg_avgs = np.sum(negatives, axis=0) / neg\n", 1018 | " neg_avgs = neg_avgs.reshape(-1,1)\n", 1019 | " \n", 1020 | " mu_y = np.hstack((neg_avgs, pos_avgs))\n", 1021 | " assert mu_y.shape == (d, 2)\n", 1022 | " return mu_y" 1023 | ] 1024 | }, 1025 | { 1026 | "cell_type": "code", 1027 | "execution_count": 58, 1028 | "metadata": { 1029 | "deletable": false, 1030 | "editable": false, 1031 | "nbgrader": { 1032 | "cell_type": "code", 1033 | "checksum": "af330da15d19ecdf406ae2d28bbf3a36", 1034 | "grade": true, 1035 | "grade_id": "cell-f3045a00bb2c1146", 1036 | "locked": true, 1037 | "points": 1, 1038 | "schema_version": 3, 1039 | "solution": false, 1040 | "task": false 1041 | } 1042 | }, 1043 | "outputs": [], 1044 | "source": [ 1045 | "some_feats = np.array([[ 1. , 85. , 66. , 29. , 0. , 26.6, 0.4, 31. ],\n", 1046 | " [ 8. , 183. , 64. , 0. , 0. , 23.3, 0.7, 32. ],\n", 1047 | " [ 1. , 89. , 66. , 23. , 94. , 28.1, 0.2, 21. ],\n", 1048 | " [ 0. , 137. , 40. , 35. , 168. , 43.1, 2.3, 33. ],\n", 1049 | " [ 5. , 116. , 74. , 0. , 0. , 25.6, 0.2, 30. ]])\n", 1050 | "some_labels = np.array([0, 1, 0, 1, 0])\n", 1051 | "\n", 1052 | "some_mu_y = cc_mean_ignore_missing(some_feats, some_labels)\n", 1053 | "\n", 1054 | "assert np.array_equal(some_mu_y.round(2), np.array([[ 2.33, 4. ],\n", 1055 | " [ 96.67, 160. ],\n", 1056 | " [ 68.67, 52. ],\n", 1057 | " [ 17.33, 17.5 ],\n", 1058 | " [ 31.33, 84. ],\n", 1059 | " [ 26.77, 33.2 ],\n", 1060 | " [ 0.27, 1.5 ],\n", 1061 | " [ 27.33, 32.5 ]]))\n", 1062 | "\n", 1063 | "# Checking against the pre-computed test database\n", 1064 | "test_results = test_case_checker(cc_mean_ignore_missing, task_id=2)\n", 1065 | "assert test_results['passed'], test_results['message']" 1066 | ] 1067 | }, 1068 | { 1069 | "cell_type": "code", 1070 | "execution_count": 59, 1071 | "metadata": {}, 1072 | "outputs": [ 1073 | { 1074 | "data": { 1075 | "text/plain": [ 1076 | "array([[ 3.48641975, 4.91866029],\n", 1077 | " [109.99753086, 142.30143541],\n", 1078 | " [ 68.77037037, 70.66028708],\n", 1079 | " [ 19.51358025, 21.97129187],\n", 1080 | " [ 66.25679012, 100.55980861],\n", 1081 | " [ 30.31703704, 35.1492823 ],\n", 1082 | " [ 0.42825926, 0.55279904],\n", 1083 | " [ 31.57283951, 37.39712919]])" 1084 | ] 1085 | }, 1086 | "execution_count": 59, 1087 | "metadata": {}, 1088 | "output_type": "execute_result" 1089 | } 1090 | ], 1091 | "source": [ 1092 | "mu_y = cc_mean_ignore_missing(train_features, train_labels)\n", 1093 | "mu_y" 1094 | ] 1095 | }, 1096 | { 1097 | "cell_type": "markdown", 1098 | "metadata": {}, 1099 | "source": [ 1100 | "# Task 3" 1101 | ] 1102 | }, 1103 | { 1104 | "cell_type": "markdown", 1105 | "metadata": {}, 1106 | "source": [ 1107 | "Write a function `cc_std_ignore_missing` that takes the numpy arrays `train_features` and `train_labels` as input, and outputs the following matrix with the shape $(8,2)$, where 8 is the number of features.\n", 1108 | "\n", 1109 | "$$\\sigma_y = \\begin{bmatrix} \\text{std}[x^{(0)}|y=0] & \\text{std}[x^{(0)}|y=1]\\\\\n", 1110 | "\\text{std}[x^{(1)}|y=0] & \\text{std}[x^{(1)}|y=1] \\\\\n", 1111 | "\\cdots & \\cdots\\\\\n", 1112 | "\\text{std}[x^{(7)}|y=0] & \\text{std}[x^{(7)}|y=1]\\end{bmatrix}$$\n", 1113 | "\n", 1114 | "Some points regarding this task:\n", 1115 | "\n", 1116 | "* The `train_features` numpy array has a shape of `(N,8)` where `N` is the number of training data points, and 8 is the number of the features. \n", 1117 | "\n", 1118 | "* The `train_labels` numpy array has a shape of `(N,)`. \n", 1119 | "\n", 1120 | "* **You can assume that `train_features` has no missing elements in this task**.\n", 1121 | "\n", 1122 | "* Try and avoid the utilization of loops as much as possible. No loops are necessary." 1123 | ] 1124 | }, 1125 | { 1126 | "cell_type": "code", 1127 | "execution_count": 60, 1128 | "metadata": { 1129 | "deletable": false, 1130 | "nbgrader": { 1131 | "cell_type": "code", 1132 | "checksum": "865730f8532366280665d029bc3b2ce5", 1133 | "grade": false, 1134 | "grade_id": "cell-410ce572204e37df", 1135 | "locked": false, 1136 | "schema_version": 3, 1137 | "solution": true, 1138 | "task": false 1139 | } 1140 | }, 1141 | "outputs": [], 1142 | "source": [ 1143 | "def cc_std_ignore_missing(train_features, train_labels):\n", 1144 | " N, d = train_features.shape\n", 1145 | " \n", 1146 | " # your code here\n", 1147 | " positive_rows = train_labels == 1\n", 1148 | " negative_rows = train_labels == 0\n", 1149 | " positives = train_features[positive_rows,:]\n", 1150 | " negatives = train_features[negative_rows,:]\n", 1151 | "\n", 1152 | " pos = np.std(positives, axis = 0)\n", 1153 | " neg = np.std(negatives, axis = 0)\n", 1154 | " sigma_y = np.column_stack((neg, pos))\n", 1155 | " assert sigma_y.shape == (d, 2)\n", 1156 | " return sigma_y" 1157 | ] 1158 | }, 1159 | { 1160 | "cell_type": "code", 1161 | "execution_count": 61, 1162 | "metadata": { 1163 | "deletable": false, 1164 | "editable": false, 1165 | "nbgrader": { 1166 | "cell_type": "code", 1167 | "checksum": "1d389f545cb36b2404ec6069b0d5801a", 1168 | "grade": true, 1169 | "grade_id": "cell-d91cfef7c658f962", 1170 | "locked": true, 1171 | "points": 1, 1172 | "schema_version": 3, 1173 | "solution": false, 1174 | "task": false 1175 | } 1176 | }, 1177 | "outputs": [], 1178 | "source": [ 1179 | "some_feats = np.array([[ 1. , 85. , 66. , 29. , 0. , 26.6, 0.4, 31. ],\n", 1180 | " [ 8. , 183. , 64. , 0. , 0. , 23.3, 0.7, 32. ],\n", 1181 | " [ 1. , 89. , 66. , 23. , 94. , 28.1, 0.2, 21. ],\n", 1182 | " [ 0. , 137. , 40. , 35. , 168. , 43.1, 2.3, 33. ],\n", 1183 | " [ 5. , 116. , 74. , 0. , 0. , 25.6, 0.2, 30. ]])\n", 1184 | "some_labels = np.array([0, 1, 0, 1, 0])\n", 1185 | "\n", 1186 | "some_std_y = cc_std_ignore_missing(some_feats, some_labels)\n", 1187 | "\n", 1188 | "assert np.array_equal(some_std_y.round(3), np.array([[ 1.886, 4. ],\n", 1189 | " [13.768, 23. ],\n", 1190 | " [ 3.771, 12. ],\n", 1191 | " [12.499, 17.5 ],\n", 1192 | " [44.312, 84. ],\n", 1193 | " [ 1.027, 9.9 ],\n", 1194 | " [ 0.094, 0.8 ],\n", 1195 | " [ 4.497, 0.5 ]]))\n", 1196 | "\n", 1197 | "# Checking against the pre-computed test database\n", 1198 | "test_results = test_case_checker(cc_std_ignore_missing, task_id=3)\n", 1199 | "assert test_results['passed'], test_results['message']" 1200 | ] 1201 | }, 1202 | { 1203 | "cell_type": "code", 1204 | "execution_count": 62, 1205 | "metadata": {}, 1206 | "outputs": [ 1207 | { 1208 | "data": { 1209 | "text/plain": [ 1210 | "array([[ 3.1155426 , 3.75417931],\n", 1211 | " [ 25.96811899, 32.50910874],\n", 1212 | " [ 18.07540068, 21.69568568],\n", 1213 | " [ 15.02320635, 17.21685884],\n", 1214 | " [ 95.63339586, 139.24364214],\n", 1215 | " [ 7.50030986, 6.6625219 ],\n", 1216 | " [ 0.29438217, 0.37201494],\n", 1217 | " [ 11.67577435, 11.01543899]])" 1218 | ] 1219 | }, 1220 | "execution_count": 62, 1221 | "metadata": {}, 1222 | "output_type": "execute_result" 1223 | } 1224 | ], 1225 | "source": [ 1226 | "sigma_y = cc_std_ignore_missing(train_features, train_labels)\n", 1227 | "sigma_y" 1228 | ] 1229 | }, 1230 | { 1231 | "cell_type": "markdown", 1232 | "metadata": {}, 1233 | "source": [ 1234 | "# Task 4" 1235 | ] 1236 | }, 1237 | { 1238 | "cell_type": "markdown", 1239 | "metadata": {}, 1240 | "source": [ 1241 | "Write a function `log_prob` that takes the numpy arrays `train_features`, $\\mu_y$, $\\sigma_y$, and $\\log p_y$ as input, and outputs the following matrix with the shape $(N, 2)$\n", 1242 | "\n", 1243 | "$$\\log p_{x,y} = \\begin{bmatrix} \\bigg[\\log p(y=0) + \\sum_{j=0}^{7} \\log p(x_1^{(j)}|y=0) \\bigg] & \\bigg[\\log p(y=1) + \\sum_{j=0}^{7} \\log p(x_1^{(j)}|y=1) \\bigg] \\\\\n", 1244 | "\\bigg[\\log p(y=0) + \\sum_{j=0}^{7} \\log p(x_2^{(j)}|y=0) \\bigg] & \\bigg[\\log p(y=1) + \\sum_{j=0}^{7} \\log p(x_2^{(j)}|y=1) \\bigg] \\\\\n", 1245 | "\\cdots & \\cdots \\\\\n", 1246 | "\\bigg[\\log p(y=0) + \\sum_{j=0}^{7} \\log p(x_N^{(j)}|y=0) \\bigg] & \\bigg[\\log p(y=1) + \\sum_{j=0}^{7} \\log p(x_N^{(j)}|y=1) \\bigg] \\\\\n", 1247 | "\\end{bmatrix}$$\n", 1248 | "\n", 1249 | "where\n", 1250 | "* $N$ is the number of training data points.\n", 1251 | "* $x_i$ is the $i^{th}$ training data point.\n", 1252 | "\n", 1253 | "Try and avoid the utilization of loops as much as possible. No loops are necessary." 1254 | ] 1255 | }, 1256 | { 1257 | "cell_type": "markdown", 1258 | "metadata": {}, 1259 | "source": [ 1260 | "**Hint**: Remember that we are modelling $p(x_i^{(j)}|y)$ with a Gaussian whose parameters are defined inside $\\mu_y$ and $\\sigma_y$. Write the Gaussian PDF expression and take its natural log **on paper**, then implement it.\n", 1261 | "\n", 1262 | "**Important Note**: Do not use third-party and non-standard implementations for computing $\\log p(x_i^{(j)}|y)$. Using functions that find the Gaussian PDF, and then taking their log is **numerically unstable**; the Gaussian PDF values can easily become extremely small numbers that cannot be represented using floating point standards and thus would be stored as zero. Taking the log of a zero value will throw an error. On the other hand, it is unnecessary to compute and store $p(x_i^{(j)}|y)$ in order to find $\\log p(x_i^{(j)}|y)$; you can write $\\log p(x_i^{(j)}|y)$ as a direct function of $\\mu_y$, $\\sigma_y$ and the features. This latter approach is numerically stable, and can be applied when the PDF values are much smaller than could be stored using the common standards." 1263 | ] 1264 | }, 1265 | { 1266 | "cell_type": "code", 1267 | "execution_count": 63, 1268 | "metadata": { 1269 | "deletable": false, 1270 | "nbgrader": { 1271 | "cell_type": "code", 1272 | "checksum": "335f5b8746a99280ca50e0d2dbca5375", 1273 | "grade": false, 1274 | "grade_id": "cell-773a3cddb6c45cf8", 1275 | "locked": false, 1276 | "schema_version": 3, 1277 | "solution": true, 1278 | "task": false 1279 | } 1280 | }, 1281 | "outputs": [], 1282 | "source": [ 1283 | "def log_prob(features, mu_y, sigma_y, log_py):\n", 1284 | " N, d = features.shape\n", 1285 | " \n", 1286 | " # your code here\n", 1287 | " part1 = np.sum(np.log(1 / (sigma_y.T * (np.sqrt(2 * np.pi)))), axis = 1)\n", 1288 | " part2_neg = np.power(features - mu_y.T[0,:],2) / (2* np.power( sigma_y.T[0,:],2)) \n", 1289 | " log_neg = np.sum(part2_neg, axis=1)\n", 1290 | " log_neg -= part1[0] + log_py[0]\n", 1291 | " part2_pos = np.power(features - mu_y.T[1,:],2) / (2* np.power( sigma_y.T[1,:],2)) \n", 1292 | " log_pos = np.sum(part2_pos, axis=1)\n", 1293 | " log_pos -= part1[1] + log_py[1]\n", 1294 | " log_p_x_y = -np.column_stack((log_neg, log_pos))\n", 1295 | " \n", 1296 | " assert log_p_x_y.shape == (N,2)\n", 1297 | " return log_p_x_y" 1298 | ] 1299 | }, 1300 | { 1301 | "cell_type": "code", 1302 | "execution_count": 64, 1303 | "metadata": { 1304 | "deletable": false, 1305 | "editable": false, 1306 | "nbgrader": { 1307 | "cell_type": "code", 1308 | "checksum": "372496f883a755a2b19cd88b06cab33a", 1309 | "grade": true, 1310 | "grade_id": "cell-a8c2e2a5d88902b0", 1311 | "locked": true, 1312 | "points": 1, 1313 | "schema_version": 3, 1314 | "solution": false, 1315 | "task": false 1316 | } 1317 | }, 1318 | "outputs": [], 1319 | "source": [ 1320 | "some_feats = np.array([[ 1. , 85. , 66. , 29. , 0. , 26.6, 0.4, 31. ],\n", 1321 | " [ 8. , 183. , 64. , 0. , 0. , 23.3, 0.7, 32. ],\n", 1322 | " [ 1. , 89. , 66. , 23. , 94. , 28.1, 0.2, 21. ],\n", 1323 | " [ 0. , 137. , 40. , 35. , 168. , 43.1, 2.3, 33. ],\n", 1324 | " [ 5. , 116. , 74. , 0. , 0. , 25.6, 0.2, 30. ]])\n", 1325 | "some_labels = np.array([0, 1, 0, 1, 0])\n", 1326 | "\n", 1327 | "some_mu_y = cc_mean_ignore_missing(some_feats, some_labels)\n", 1328 | "some_std_y = cc_std_ignore_missing(some_feats, some_labels)\n", 1329 | "some_log_py = log_prior(some_labels)\n", 1330 | "\n", 1331 | "some_log_p_x_y = log_prob(some_feats, some_mu_y, some_std_y, some_log_py)\n", 1332 | "\n", 1333 | "assert np.array_equal(some_log_p_x_y.round(3), np.array([[ -20.822, -36.606],\n", 1334 | " [ -60.879, -27.944],\n", 1335 | " [ -21.774, -295.68 ],\n", 1336 | " [-417.359, -27.944],\n", 1337 | " [ -23.2 , -42.6 ]]))\n", 1338 | "\n", 1339 | "# Checking against the pre-computed test database\n", 1340 | "test_results = test_case_checker(log_prob, task_id=4)\n", 1341 | "assert test_results['passed'], test_results['message']" 1342 | ] 1343 | }, 1344 | { 1345 | "cell_type": "code", 1346 | "execution_count": 65, 1347 | "metadata": {}, 1348 | "outputs": [ 1349 | { 1350 | "data": { 1351 | "text/plain": [ 1352 | "array([[-26.96647828, -31.00418408],\n", 1353 | " [-32.4755447 , -31.39530914],\n", 1354 | " [-27.14875996, -31.51999532],\n", 1355 | " ...,\n", 1356 | " [-26.29368771, -29.09161966],\n", 1357 | " [-28.19432943, -30.08324788],\n", 1358 | " [-26.98605248, -30.80571318]])" 1359 | ] 1360 | }, 1361 | "execution_count": 65, 1362 | "metadata": {}, 1363 | "output_type": "execute_result" 1364 | } 1365 | ], 1366 | "source": [ 1367 | "log_p_x_y = log_prob(train_features, mu_y, sigma_y, log_py)\n", 1368 | "log_p_x_y" 1369 | ] 1370 | }, 1371 | { 1372 | "cell_type": "markdown", 1373 | "metadata": {}, 1374 | "source": [ 1375 | "## 1.1. Writing the Simple Naive Bayes Classifier" 1376 | ] 1377 | }, 1378 | { 1379 | "cell_type": "code", 1380 | "execution_count": 66, 1381 | "metadata": {}, 1382 | "outputs": [], 1383 | "source": [ 1384 | "class NBClassifier():\n", 1385 | " def __init__(self, train_features, train_labels):\n", 1386 | " self.train_features = train_features\n", 1387 | " self.train_labels = train_labels\n", 1388 | " self.log_py = log_prior(train_labels)\n", 1389 | " self.mu_y = self.get_cc_means()\n", 1390 | " self.sigma_y = self.get_cc_std()\n", 1391 | " \n", 1392 | " def get_cc_means(self):\n", 1393 | " mu_y = cc_mean_ignore_missing(self.train_features, self.train_labels)\n", 1394 | " return mu_y\n", 1395 | " \n", 1396 | " def get_cc_std(self):\n", 1397 | " sigma_y = cc_std_ignore_missing(self.train_features, self.train_labels)\n", 1398 | " return sigma_y\n", 1399 | " \n", 1400 | " def predict(self, features):\n", 1401 | " log_p_x_y = log_prob(features, mu_y, sigma_y, log_py)\n", 1402 | " return log_p_x_y.argmax(axis=1)" 1403 | ] 1404 | }, 1405 | { 1406 | "cell_type": "code", 1407 | "execution_count": 67, 1408 | "metadata": {}, 1409 | "outputs": [], 1410 | "source": [ 1411 | "diabetes_classifier = NBClassifier(train_features, train_labels)\n", 1412 | "train_pred = diabetes_classifier.predict(train_features)\n", 1413 | "eval_pred = diabetes_classifier.predict(eval_features)" 1414 | ] 1415 | }, 1416 | { 1417 | "cell_type": "code", 1418 | "execution_count": 68, 1419 | "metadata": {}, 1420 | "outputs": [ 1421 | { 1422 | "name": "stdout", 1423 | "output_type": "stream", 1424 | "text": [ 1425 | "The training data accuracy of your trained model is 0.7671009771986971\n", 1426 | "The evaluation data accuracy of your trained model is 0.7532467532467533\n" 1427 | ] 1428 | } 1429 | ], 1430 | "source": [ 1431 | "train_acc = (train_pred==train_labels).mean()\n", 1432 | "eval_acc = (eval_pred==eval_labels).mean()\n", 1433 | "print(f'The training data accuracy of your trained model is {train_acc}')\n", 1434 | "print(f'The evaluation data accuracy of your trained model is {eval_acc}')" 1435 | ] 1436 | }, 1437 | { 1438 | "cell_type": "markdown", 1439 | "metadata": {}, 1440 | "source": [ 1441 | "## 1.2 Running an off-the-shelf implementation of Naive-Bayes For Comparison" 1442 | ] 1443 | }, 1444 | { 1445 | "cell_type": "code", 1446 | "execution_count": 69, 1447 | "metadata": {}, 1448 | "outputs": [ 1449 | { 1450 | "name": "stdout", 1451 | "output_type": "stream", 1452 | "text": [ 1453 | "The training data accuracy of your trained model is 0.7671009771986971\n", 1454 | "The evaluation data accuracy of your trained model is 0.7532467532467533\n" 1455 | ] 1456 | } 1457 | ], 1458 | "source": [ 1459 | "from sklearn.naive_bayes import GaussianNB\n", 1460 | "gnb = GaussianNB().fit(train_features, train_labels)\n", 1461 | "train_pred_sk = gnb.predict(train_features)\n", 1462 | "eval_pred_sk = gnb.predict(eval_features)\n", 1463 | "print(f'The training data accuracy of your trained model is {(train_pred_sk == train_labels).mean()}')\n", 1464 | "print(f'The evaluation data accuracy of your trained model is {(eval_pred_sk == eval_labels).mean()}')" 1465 | ] 1466 | }, 1467 | { 1468 | "cell_type": "markdown", 1469 | "metadata": {}, 1470 | "source": [ 1471 | "# Part 2 (Building a Naive Bayes Classifier Considering Missing Entries)" 1472 | ] 1473 | }, 1474 | { 1475 | "cell_type": "markdown", 1476 | "metadata": {}, 1477 | "source": [ 1478 | "In this part, we will modify some of the parameter inference functions of the Naive Bayes classifier to make it able to ignore the NaN entries when inferring the Gaussian mean and stds." 1479 | ] 1480 | }, 1481 | { 1482 | "cell_type": "markdown", 1483 | "metadata": {}, 1484 | "source": [ 1485 | "# Task 5" 1486 | ] 1487 | }, 1488 | { 1489 | "cell_type": "markdown", 1490 | "metadata": {}, 1491 | "source": [ 1492 | "Write a function `cc_mean_consider_missing` that\n", 1493 | "* has exactly the same input and output types as the `cc_mean_ignore_missing` function,\n", 1494 | "* and has similar functionality to `cc_mean_ignore_missing` except that it can handle and ignore the NaN entries when computing the class conditional means.\n", 1495 | "\n", 1496 | "You can borrow most of the code from your `cc_mean_ignore_missing` implementation, but you should make it compatible with the existence of NaN values in the features.\n", 1497 | "\n", 1498 | "Try and avoid the utilization of loops as much as possible. No loops are necessary." 1499 | ] 1500 | }, 1501 | { 1502 | "cell_type": "markdown", 1503 | "metadata": {}, 1504 | "source": [ 1505 | "* **Hint**: You may find the `np.nanmean` function useful." 1506 | ] 1507 | }, 1508 | { 1509 | "cell_type": "code", 1510 | "execution_count": 70, 1511 | "metadata": { 1512 | "deletable": false, 1513 | "nbgrader": { 1514 | "cell_type": "code", 1515 | "checksum": "ed57e96c9d1d8044ce805a98adaacbbb", 1516 | "grade": false, 1517 | "grade_id": "cell-6ab8c367d427a588", 1518 | "locked": false, 1519 | "schema_version": 3, 1520 | "solution": true, 1521 | "task": false 1522 | } 1523 | }, 1524 | "outputs": [], 1525 | "source": [ 1526 | "def cc_mean_consider_missing(train_features_with_nans, train_labels):\n", 1527 | " N, d = train_features_with_nans.shape\n", 1528 | " \n", 1529 | " # your code here\n", 1530 | " train_labels = train_labels.T\n", 1531 | " pos_train_labels = train_labels == 1 \n", 1532 | " neg_train_labels = train_labels == 0\n", 1533 | " positives = train_features_with_nans[pos_train_labels, :]\n", 1534 | " pos_mean = np.nanmean(positives, axis=0) \n", 1535 | " pos_mean = pos_mean.reshape(-1,1)\n", 1536 | " negatives = train_features_with_nans[neg_train_labels, :]\n", 1537 | " neg_mean = np.nanmean(negatives, axis=0) \n", 1538 | " neg_mean = neg_mean.reshape(-1,1)\n", 1539 | " mu_y = np.hstack((neg_mean, pos_mean))\n", 1540 | " \n", 1541 | " assert not np.isnan(mu_y).any()\n", 1542 | " assert mu_y.shape == (d, 2)\n", 1543 | " return mu_y" 1544 | ] 1545 | }, 1546 | { 1547 | "cell_type": "code", 1548 | "execution_count": 71, 1549 | "metadata": { 1550 | "deletable": false, 1551 | "editable": false, 1552 | "nbgrader": { 1553 | "cell_type": "code", 1554 | "checksum": "ec2c7c4cbb59a66bc04e3afcdd1d7701", 1555 | "grade": true, 1556 | "grade_id": "cell-b340557154da9804", 1557 | "locked": true, 1558 | "points": 1, 1559 | "schema_version": 3, 1560 | "solution": false, 1561 | "task": false 1562 | } 1563 | }, 1564 | "outputs": [], 1565 | "source": [ 1566 | "some_feats = np.array([[ 1. , 85. , 66. , 29. , 0. , 26.6, 0.4, 31. ],\n", 1567 | " [ 8. , 183. , 64. , 0. , 0. , 23.3, 0.7, 32. ],\n", 1568 | " [ 1. , 89. , 66. , 23. , 94. , 28.1, 0.2, 21. ],\n", 1569 | " [ 0. , 137. , 40. , 35. , 168. , 43.1, 2.3, 33. ],\n", 1570 | " [ 5. , 116. , 74. , 0. , 0. , 25.6, 0.2, 30. ]])\n", 1571 | "some_labels = np.array([0, 1, 0, 1, 0])\n", 1572 | "\n", 1573 | "for i,j in [(0,0), (1,1), (2,3), (3,4), (4, 2)]:\n", 1574 | " some_feats[i,j] = np.nan\n", 1575 | "\n", 1576 | "some_mu_y = cc_mean_consider_missing(some_feats, some_labels)\n", 1577 | "\n", 1578 | "assert np.array_equal(some_mu_y.round(2), np.array([[ 3. , 4. ],\n", 1579 | " [ 96.67, 137. ],\n", 1580 | " [ 66. , 52. ],\n", 1581 | " [ 14.5 , 17.5 ],\n", 1582 | " [ 31.33, 0. ],\n", 1583 | " [ 26.77, 33.2 ],\n", 1584 | " [ 0.27, 1.5 ],\n", 1585 | " [ 27.33, 32.5 ]]))\n", 1586 | "\n", 1587 | "# Checking against the pre-computed test database\n", 1588 | "test_results = test_case_checker(cc_mean_consider_missing, task_id=5)\n", 1589 | "assert test_results['passed'], test_results['message']" 1590 | ] 1591 | }, 1592 | { 1593 | "cell_type": "code", 1594 | "execution_count": 72, 1595 | "metadata": {}, 1596 | "outputs": [ 1597 | { 1598 | "data": { 1599 | "text/plain": [ 1600 | "array([[ 3.48641975, 4.91866029],\n", 1601 | " [109.99753086, 142.30143541],\n", 1602 | " [ 71.41538462, 75.34693878],\n", 1603 | " [ 27.53658537, 32.11188811],\n", 1604 | " [ 66.25679012, 100.55980861],\n", 1605 | " [ 30.85025126, 35.31826923],\n", 1606 | " [ 0.42825926, 0.55279904],\n", 1607 | " [ 31.57283951, 37.39712919]])" 1608 | ] 1609 | }, 1610 | "execution_count": 72, 1611 | "metadata": {}, 1612 | "output_type": "execute_result" 1613 | } 1614 | ], 1615 | "source": [ 1616 | "mu_y = cc_mean_consider_missing(train_features_with_nans, train_labels)\n", 1617 | "mu_y" 1618 | ] 1619 | }, 1620 | { 1621 | "cell_type": "markdown", 1622 | "metadata": {}, 1623 | "source": [ 1624 | "# Task 6" 1625 | ] 1626 | }, 1627 | { 1628 | "cell_type": "markdown", 1629 | "metadata": {}, 1630 | "source": [ 1631 | "Write a function `cc_std_consider_missing` that\n", 1632 | "* has exactly the same input and output types as the `cc_std_ignore_missing` function,\n", 1633 | "* and has similar functionality to `cc_std_ignore_missing` except that it can handle and ignore the NaN entries when computing the class conditional means.\n", 1634 | "\n", 1635 | "You can borrow most of the code from your `cc_std_ignore_missing` implementation, but you should make it compatible with the existence of NaN values in the features.\n", 1636 | "\n", 1637 | "Try and avoid the utilization of loops as much as possible. No loops are necessary." 1638 | ] 1639 | }, 1640 | { 1641 | "cell_type": "markdown", 1642 | "metadata": {}, 1643 | "source": [ 1644 | "* **Hint**: You may find the `np.nanstd` function useful." 1645 | ] 1646 | }, 1647 | { 1648 | "cell_type": "code", 1649 | "execution_count": 73, 1650 | "metadata": { 1651 | "deletable": false, 1652 | "nbgrader": { 1653 | "cell_type": "code", 1654 | "checksum": "3001dfb41f62e3925b7edde741bb1776", 1655 | "grade": false, 1656 | "grade_id": "cell-927753c6215c5646", 1657 | "locked": false, 1658 | "schema_version": 3, 1659 | "solution": true, 1660 | "task": false 1661 | } 1662 | }, 1663 | "outputs": [], 1664 | "source": [ 1665 | "def cc_std_consider_missing(train_features_with_nans, train_labels):\n", 1666 | " N, d = train_features_with_nans.shape\n", 1667 | " \n", 1668 | " # your code here\n", 1669 | " positive_rows = train_labels == 1\n", 1670 | " negative_rows = train_labels == 0\n", 1671 | " positives = train_features_with_nans[positive_rows,:]\n", 1672 | " negatives = train_features_with_nans[negative_rows,:]\n", 1673 | "\n", 1674 | " pos = np.nanstd(positives, axis = 0)\n", 1675 | " neg = np.nanstd(negatives, axis = 0)\n", 1676 | " sigma_y = np.column_stack((neg, pos))\n", 1677 | " \n", 1678 | " assert not np.isnan(sigma_y).any()\n", 1679 | " assert sigma_y.shape == (d, 2)\n", 1680 | " return sigma_y" 1681 | ] 1682 | }, 1683 | { 1684 | "cell_type": "code", 1685 | "execution_count": 74, 1686 | "metadata": { 1687 | "deletable": false, 1688 | "editable": false, 1689 | "nbgrader": { 1690 | "cell_type": "code", 1691 | "checksum": "2c8b25848847a54241cfbaba98b8d83d", 1692 | "grade": true, 1693 | "grade_id": "cell-d67179c6dea81502", 1694 | "locked": true, 1695 | "points": 1, 1696 | "schema_version": 3, 1697 | "solution": false, 1698 | "task": false 1699 | } 1700 | }, 1701 | "outputs": [], 1702 | "source": [ 1703 | "some_feats = np.array([[ 1. , 85. , 66. , 29. , 0. , 26.6, 0.4, 31. ],\n", 1704 | " [ 8. , 183. , 64. , 0. , 0. , 23.3, 0.7, 32. ],\n", 1705 | " [ 1. , 89. , 66. , 23. , 94. , 28.1, 0.2, 21. ],\n", 1706 | " [ 0. , 137. , 40. , 35. , 168. , 43.1, 2.3, 33. ],\n", 1707 | " [ 5. , 116. , 74. , 0. , 0. , 25.6, 0.2, 30. ]])\n", 1708 | "some_labels = np.array([0, 1, 0, 1, 0])\n", 1709 | "\n", 1710 | "for i,j in [(0,0), (1,1), (2,3), (3,4), (4, 2)]:\n", 1711 | " some_feats[i,j] = np.nan\n", 1712 | "\n", 1713 | "some_std_y = cc_std_consider_missing(some_feats, some_labels)\n", 1714 | "\n", 1715 | "assert np.array_equal(some_std_y.round(2), np.array([[ 2. , 4. ],\n", 1716 | " [13.77, 0. ],\n", 1717 | " [ 0. , 12. ],\n", 1718 | " [14.5 , 17.5 ],\n", 1719 | " [44.31, 0. ],\n", 1720 | " [ 1.03, 9.9 ],\n", 1721 | " [ 0.09, 0.8 ],\n", 1722 | " [ 4.5 , 0.5 ]]))\n", 1723 | "\n", 1724 | "# Checking against the pre-computed test database\n", 1725 | "test_results = test_case_checker(cc_std_consider_missing, task_id=6)\n", 1726 | "assert test_results['passed'], test_results['message']" 1727 | ] 1728 | }, 1729 | { 1730 | "cell_type": "code", 1731 | "execution_count": 75, 1732 | "metadata": {}, 1733 | "outputs": [ 1734 | { 1735 | "data": { 1736 | "text/plain": [ 1737 | "array([[ 3.1155426 , 3.75417931],\n", 1738 | " [ 25.96811899, 32.50910874],\n", 1739 | " [ 12.26342359, 12.1982786 ],\n", 1740 | " [ 9.87753687, 10.37284304],\n", 1741 | " [ 95.63339586, 139.24364214],\n", 1742 | " [ 6.38703834, 6.21564813],\n", 1743 | " [ 0.29438217, 0.37201494],\n", 1744 | " [ 11.67577435, 11.01543899]])" 1745 | ] 1746 | }, 1747 | "execution_count": 75, 1748 | "metadata": {}, 1749 | "output_type": "execute_result" 1750 | } 1751 | ], 1752 | "source": [ 1753 | "sigma_y = cc_std_consider_missing(train_features_with_nans, train_labels)\n", 1754 | "sigma_y" 1755 | ] 1756 | }, 1757 | { 1758 | "cell_type": "markdown", 1759 | "metadata": {}, 1760 | "source": [ 1761 | "## 2.1. Writing the Naive Bayes Classifier With Missing Data Handling" 1762 | ] 1763 | }, 1764 | { 1765 | "cell_type": "code", 1766 | "execution_count": 76, 1767 | "metadata": {}, 1768 | "outputs": [], 1769 | "source": [ 1770 | "class NBClassifierWithMissing(NBClassifier):\n", 1771 | " def get_cc_means(self):\n", 1772 | " mu_y = cc_mean_consider_missing(self.train_features, self.train_labels)\n", 1773 | " return mu_y\n", 1774 | " \n", 1775 | " def get_cc_std(self):\n", 1776 | " sigma_y = cc_std_consider_missing(self.train_features, self.train_labels)\n", 1777 | " return sigma_y" 1778 | ] 1779 | }, 1780 | { 1781 | "cell_type": "code", 1782 | "execution_count": 77, 1783 | "metadata": {}, 1784 | "outputs": [], 1785 | "source": [ 1786 | "diabetes_classifier_nans = NBClassifierWithMissing(train_features_with_nans, train_labels)\n", 1787 | "train_pred = diabetes_classifier_nans.predict(train_features_with_nans)\n", 1788 | "eval_pred = diabetes_classifier_nans.predict(eval_features_with_nans)" 1789 | ] 1790 | }, 1791 | { 1792 | "cell_type": "code", 1793 | "execution_count": 78, 1794 | "metadata": {}, 1795 | "outputs": [ 1796 | { 1797 | "name": "stdout", 1798 | "output_type": "stream", 1799 | "text": [ 1800 | "The training data accuracy of your trained model is 0.7182410423452769\n", 1801 | "The evaluation data accuracy of your trained model is 0.7142857142857143\n" 1802 | ] 1803 | } 1804 | ], 1805 | "source": [ 1806 | "train_acc = (train_pred==train_labels).mean()\n", 1807 | "eval_acc = (eval_pred==eval_labels).mean()\n", 1808 | "print(f'The training data accuracy of your trained model is {train_acc}')\n", 1809 | "print(f'The evaluation data accuracy of your trained model is {eval_acc}')" 1810 | ] 1811 | }, 1812 | { 1813 | "cell_type": "markdown", 1814 | "metadata": {}, 1815 | "source": [ 1816 | "# 3. Running SVMlight" 1817 | ] 1818 | }, 1819 | { 1820 | "cell_type": "markdown", 1821 | "metadata": {}, 1822 | "source": [ 1823 | "In this section, we are going to investigate the support vector machine classification method. We will become familiar with this classification method in week 3. However, in this section, we are just going to observe how this method performs to set the stage for the third week.\n", 1824 | "\n", 1825 | "`SVMlight` (http://svmlight.joachims.org/) is a famous implementation of the SVM classifier. \n", 1826 | "\n", 1827 | "`SVMLight` can be called from a shell terminal, and there is no nice wrapper for it in python3. Therefore:\n", 1828 | "1. We have to export the training data to a special format called `svmlight/libsvm`. This can be done using scikit-learn.\n", 1829 | "2. We have to run the `svm_learn` program to learn the model and then store it.\n", 1830 | "3. We have to import the model back to python." 1831 | ] 1832 | }, 1833 | { 1834 | "cell_type": "markdown", 1835 | "metadata": {}, 1836 | "source": [ 1837 | "## 3.1 Exporting the training data to libsvm format" 1838 | ] 1839 | }, 1840 | { 1841 | "cell_type": "code", 1842 | "execution_count": 79, 1843 | "metadata": {}, 1844 | "outputs": [], 1845 | "source": [ 1846 | "from sklearn.datasets import dump_svmlight_file\n", 1847 | "dump_svmlight_file(train_features, 2*train_labels-1, 'training_feats.data', \n", 1848 | " zero_based=False, comment=None, query_id=None, multilabel=False)" 1849 | ] 1850 | }, 1851 | { 1852 | "cell_type": "markdown", 1853 | "metadata": {}, 1854 | "source": [ 1855 | "## 3.2 Training `SVMlight`" 1856 | ] 1857 | }, 1858 | { 1859 | "cell_type": "code", 1860 | "execution_count": 80, 1861 | "metadata": {}, 1862 | "outputs": [ 1863 | { 1864 | "name": "stdout", 1865 | "output_type": "stream", 1866 | "text": [ 1867 | "Scanning examples...done\n", 1868 | "Reading examples into memory...100..200..300..400..500..600..OK. (614 examples read)\n", 1869 | "Setting default regularization parameter C=0.0000\n", 1870 | "Optimizing....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done. (1781 iterations)\n", 1871 | "Optimization finished (141 misclassified, maxdiff=0.00099).\n", 1872 | "Runtime in cpu-seconds: 0.19\n", 1873 | "Number of SV: 375 (including 369 at upper bound)\n", 1874 | "L1 loss: loss=335.23204\n", 1875 | "Norm of weight vector: |w|=0.03179\n", 1876 | "Norm of longest example vector: |x|=871.75350\n", 1877 | "Estimated VCdim of classifier: VCdim<=769.24695\n", 1878 | "Computing XiAlpha-estimates...done\n", 1879 | "Runtime for XiAlpha-estimates in cpu-seconds: 0.00\n", 1880 | "XiAlpha-estimate of the error: error<=60.75% (rho=1.00,depth=0)\n", 1881 | "XiAlpha-estimate of the recall: recall=>10.53% (rho=1.00,depth=0)\n", 1882 | "XiAlpha-estimate of the precision: precision=>10.58% (rho=1.00,depth=0)\n", 1883 | "Number of kernel evaluations: 71356\n", 1884 | "Writing model file...done\n", 1885 | "\n" 1886 | ] 1887 | } 1888 | ], 1889 | "source": [ 1890 | "from subprocess import Popen, PIPE\n", 1891 | "process = Popen([\"./svmlight/svm_learn\", \"./training_feats.data\", \"svm_model.txt\"], stdout=PIPE, stderr=PIPE)\n", 1892 | "stdout, stderr = process.communicate()\n", 1893 | "print(stdout.decode(\"utf-8\"))" 1894 | ] 1895 | }, 1896 | { 1897 | "cell_type": "markdown", 1898 | "metadata": {}, 1899 | "source": [ 1900 | "## 3.3 Importing the SVM Model" 1901 | ] 1902 | }, 1903 | { 1904 | "cell_type": "code", 1905 | "execution_count": 81, 1906 | "metadata": {}, 1907 | "outputs": [], 1908 | "source": [ 1909 | "from svm2weight import get_svmlight_weights\n", 1910 | "svm_weights, thresh = get_svmlight_weights('svm_model.txt', printOutput=False)\n", 1911 | "\n", 1912 | "def svmlight_classifier(train_features):\n", 1913 | " return (train_features @ svm_weights - thresh).reshape(-1) >= 0." 1914 | ] 1915 | }, 1916 | { 1917 | "cell_type": "code", 1918 | "execution_count": 82, 1919 | "metadata": {}, 1920 | "outputs": [], 1921 | "source": [ 1922 | "train_pred = svmlight_classifier(train_features)\n", 1923 | "eval_pred = svmlight_classifier(eval_features)" 1924 | ] 1925 | }, 1926 | { 1927 | "cell_type": "code", 1928 | "execution_count": 83, 1929 | "metadata": {}, 1930 | "outputs": [ 1931 | { 1932 | "name": "stdout", 1933 | "output_type": "stream", 1934 | "text": [ 1935 | "The training data accuracy of your trained model is 0.7703583061889251\n", 1936 | "The evaluation data accuracy of your trained model is 0.7402597402597403\n" 1937 | ] 1938 | } 1939 | ], 1940 | "source": [ 1941 | "train_acc = (train_pred==train_labels).mean()\n", 1942 | "eval_acc = (eval_pred==eval_labels).mean()\n", 1943 | "print(f'The training data accuracy of your trained model is {train_acc}')\n", 1944 | "print(f'The evaluation data accuracy of your trained model is {eval_acc}')" 1945 | ] 1946 | }, 1947 | { 1948 | "cell_type": "code", 1949 | "execution_count": null, 1950 | "metadata": {}, 1951 | "outputs": [], 1952 | "source": [] 1953 | }, 1954 | { 1955 | "cell_type": "code", 1956 | "execution_count": null, 1957 | "metadata": {}, 1958 | "outputs": [], 1959 | "source": [] 1960 | } 1961 | ], 1962 | "metadata": { 1963 | "kernelspec": { 1964 | "display_name": "Python 3", 1965 | "language": "python", 1966 | "name": "python3" 1967 | }, 1968 | "language_info": { 1969 | "codemirror_mode": { 1970 | "name": "ipython", 1971 | "version": 3 1972 | }, 1973 | "file_extension": ".py", 1974 | "mimetype": "text/x-python", 1975 | "name": "python", 1976 | "nbconvert_exporter": "python", 1977 | "pygments_lexer": "ipython3", 1978 | "version": "3.7.6" 1979 | } 1980 | }, 1981 | "nbformat": 4, 1982 | "nbformat_minor": 4 1983 | } 1984 | -------------------------------------------------------------------------------- /MeanField.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 3, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "%matplotlib inline\n", 10 | "%load_ext autoreload\n", 11 | "%autoreload 2\n", 12 | "\n", 13 | "import matplotlib.pyplot as plt\n", 14 | "import numpy as np\n", 15 | "import os\n", 16 | "import pandas as pd\n", 17 | "\n", 18 | "from scipy.special import expit\n", 19 | "\n", 20 | "from utils import test_case_checker, perform_computation, show_test_cases" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "# 0. Data " 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "metadata": {}, 33 | "source": [ 34 | "Since the MNIST data (http://yann.lecun.com/exdb/mnist/) is stored in a binary format, we would rather have an API handle the loading for us. \n", 35 | "\n", 36 | "Pytorch (https://pytorch.org/) is an Automatic Differentiation library that we may see and use later in the course. \n", 37 | "\n", 38 | "Torchvision (https://pytorch.org/docs/stable/torchvision/index.html?highlight=torchvision#module-torchvision) is an extension library for pytorch that can load many of the famous data sets painlessly. \n", 39 | "\n", 40 | "We already used Torchvision for downloading the MNIST data. It is stored in a numpy array file that we will load easily." 41 | ] 42 | }, 43 | { 44 | "cell_type": "markdown", 45 | "metadata": {}, 46 | "source": [ 47 | "## 0.1 Loading the Data" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": 4, 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "if os.path.exists('mnist.npz'):\n", 57 | " npzfile = np.load('mnist.npz')\n", 58 | " train_images_raw = npzfile['train_images_raw']\n", 59 | " train_labels = npzfile['train_labels']\n", 60 | " eval_images_raw = npzfile['eval_images_raw']\n", 61 | " eval_labels = npzfile['eval_labels']\n", 62 | "else:\n", 63 | " import torchvision\n", 64 | " download_ = not os.path.exists('./mnist')\n", 65 | " data_train = torchvision.datasets.MNIST('mnist', train=True, transform=None, target_transform=None, download=download_)\n", 66 | " data_eval = torchvision.datasets.MNIST('mnist', train=False, transform=None, target_transform=None, download=download_)\n", 67 | "\n", 68 | " train_images_raw = data_train.data.numpy()\n", 69 | " train_labels = data_train.targets.numpy()\n", 70 | " eval_images_raw = data_eval.data.numpy()\n", 71 | " eval_labels = data_eval.targets.numpy()\n", 72 | "\n", 73 | " np.savez('mnist.npz', train_images_raw=train_images_raw, train_labels=train_labels, \n", 74 | " eval_images_raw=eval_images_raw, eval_labels=eval_labels) " 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 5, 80 | "metadata": {}, 81 | "outputs": [], 82 | "source": [ 83 | "noise_flip_prob = 0.04" 84 | ] 85 | }, 86 | { 87 | "cell_type": "markdown", 88 | "metadata": {}, 89 | "source": [ 90 | "# Task 1" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "Write the function `get_thresholded_and_noised` that does image thresholding and flipping pixels. More specifically, this functions should exactly apply the following two steps in order:\n", 98 | "\n", 99 | "1. **Thresholding**: First, given the input threshold argument, you must compute a thresholded image array. This array should indicate whether each element of `images_raw` is **greater than or equal to** the `threshold` argument. We will call the result of this step the thresholded image.\n", 100 | "2. **Noise Application (i.e., Flipping Pixels)**: After the image was thresholded, you should use the `flip_flags` input argument and flip the pixels with a corresponding `True` entry in `flip_flags`. \n", 101 | "\n", 102 | " * `flip_flags` mostly consists of `False` entries, which means you should not change their corresponding pixels. Instead, whenever a pixel had a `True` entry in `flip_flags`, that pixel in the thresholded image must get flipped. This way you will obtain the noised image.\n", 103 | "3. **Mapping Pixels to -1/+1**: You need to make sure the output image pixels are mapped to -1 and 1 values (as opposed to 0/1 or True/False).\n", 104 | "\n", 105 | "`get_thresholded_and_noised` should take the following arguments:\n", 106 | "\n", 107 | "1. `images_raw`: A numpy array. Do not assume anything about its shape, dtype or range of values. Your function should be careless about these attributes.\n", 108 | "2. `threshold`: A scalar value.\n", 109 | "3. `flip_flags`: A numpy array with the same shape as `images_raw` and `np.bool` dtype. This array indicates whether each pixel should be flipped or not.\n", 110 | "\n", 111 | "and return the following:\n", 112 | "\n", 113 | "* `mapped_noised_image`: A numpy array with the same shape as `images_raw`. This array's entries should either be -1 or 1." 114 | ] 115 | }, 116 | { 117 | "cell_type": "code", 118 | "execution_count": 21, 119 | "metadata": { 120 | "deletable": false, 121 | "nbgrader": { 122 | "cell_type": "code", 123 | "checksum": "d7a43224809fd8d0963612527c0b97c7", 124 | "grade": false, 125 | "grade_id": "cell-8537fe703ac9bd5d", 126 | "locked": false, 127 | "schema_version": 3, 128 | "solution": true, 129 | "task": false 130 | } 131 | }, 132 | "outputs": [], 133 | "source": [ 134 | "def get_thresholded_and_noised(images_raw, threshold, flip_flags):\n", 135 | " \n", 136 | " # your code here\n", 137 | " mapped_noised_image = np.where(np.logical_xor(images_raw >= threshold, flip_flags), 1, -1)\n", 138 | " \n", 139 | " assert (np.abs(mapped_noised_image)==1).all()\n", 140 | " return mapped_noised_image.astype(np.int32)" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": 22, 146 | "metadata": { 147 | "deletable": false, 148 | "editable": false, 149 | "nbgrader": { 150 | "cell_type": "code", 151 | "checksum": "9d437b2f579e514e2afb64f057b9b7cc", 152 | "grade": true, 153 | "grade_id": "cell-a93db968174effe4", 154 | "locked": true, 155 | "points": 0.2, 156 | "schema_version": 3, 157 | "solution": false, 158 | "task": false 159 | } 160 | }, 161 | "outputs": [ 162 | { 163 | "name": "stdout", 164 | "output_type": "stream", 165 | "text": [ 166 | "The reference and solution images are the same to a T! Well done on this test case.\n" 167 | ] 168 | }, 169 | { 170 | "data": { 171 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAAElCAYAAABgRJorAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3da7gcdZXv8d8iFxIT5BpCCLdwG+VmJFvEg46ZB1RgRMIBHFQUFA2oHOEMDiLOCDgyKI+onAPCAc0E5CYSQZzBo8AIjI54TDQSEEQgCLmw2VzCJQaBZJ0XVQ2Vpru6dnd19+ru7+d58mTvrtVVq+uydv27q2uZuwsAAAAA0F0bdDsBAAAAAACDMwAAAAAIgcEZAAAAAATA4AwAAAAAAmBwBgAAAAABMDgDAAAAgAAYnAHAgDKzqWZ2h5k9Z2bndTufTjOzh83sgCafu52ZPW9mY8rOC0DvMrP5ZvblFp5/j5nNLjEl9BgGZ30qPelYk548PJYWi8ltXmZLBQnA6LV4rM+V9ISk17v7KW1Ms23MbBszW2BmT5jZM2a2xMyObcNy1hvIufsj7j7Z3de2YVluZjuXPV+g15jZlWY2r+qxd5rZk2Y2rY3LHW9m55nZsrS2LjWzb7RhOa85b3L33d39tjYs6zYz+3jZ80X5GJz1t0PcfbKkmZLeLOnzXc4HQHs0e6xvL+n37u6jXaCZjR3tc9rku5IeVfJaNpf0EUnDXc0IQFk+I+lgM3uXJJnZBEmXSjrF3VeWsYA6tezzkoYk7SNpI0l/I+m3ZSwPaITB2QBw98ck/UTJiZskycxOM7MH08uZfm9mh2Wm/cnMZqU/H52+i7tb+vvHzeyGRss0sx3S533UzB41s6fN7AQze4uZ3WVmq8zsgkz8Tmb2H+m7YU+k75Ztkpm+t5n9Ns33+2b2vey7TWb2XjNbnM73v8xsr1bXG9Br6hzr+6bHxCoz+13lchkzmy/pGEmnpu8MH2BmG2Rqw5Nmdq2ZbZbGV47p48zsEUn/kT7+MTO7Nz3Gf2Jm22eW7elx/8d0+oVmZpnpn0ifW6lDe6ePb51+GjaSvmP9mZyX/RZJ8919tbu/7O6/dfcfZ5bxvvQyoVXpO8dvrDWT6newzWy2mS1Lf/6upO0k/ShdV6dm1sfYTM43mtlTZvaAmX0iM68z03V5efpa7zGzodyNuf5zv29mV6TPXWJmu5rZ583s8bS+vjsT/9HMOn3IzI6vmt+pZrbSzFak9fyVT+nMbEMz+5qZPWJmw2Z2sZlNLJIn0A7u/qSk/yHpEjObJOkMSQ+6+/y8Y9uqPn3OHt+VY9vMPmdmj0n61xqLfouk6919hScedvfLM/N7Y7rMVWkO76uVv5kda2Y/r3rMzWxnM5sr6UN6tQb/KJ3+yqf06TH5zfR4XZH+vGHV6zglrQUrzeyjRdZr5rmnZp47x8wONrP70zp2eiZ+HzP7Zfp6V5rZBWY2PjP93Wb2B0uuXviWmd1umU/pLOfvBF6LwdkAMLNtJB0k6YHMww9KeoekjSWdJekKe/USgdslzU5//mtJD0l6Z+b320ex+LdK2kXS30n6pqQvSDpA0u6S3m9mlfmapHMkbS3pjZK2lXRmmv94SddLmi9pM0lXS8oOJveWNE/S8UreOf8/km6sFDBgUFQf62Y2XdK/S/qykmPns5IWmNkUdz9W0pWSzk0vz7tFybvUc5Qc71tLelrShVWLeaeSY/Q9ZjZH0umS/rukKZL+U8nxmfVeJSc6b5L0fknvSXM7Uskx/hFJr5f0PklPmtkGkn4k6XeSpkvaX9LJZvaeOi/7TkkXmtlRZrZd1frYNc3n5DS/m5QMsMa/djb1ufuHJT2i9BNKdz+3RtjVkpYpWW9HSPoXM9s/M/19kq6RtImkGyVd8Jo51HeIkk8IN1Xy7v1PlPz9ni7pS0pqXsXjStb56yV9VNI37NVB74GS/l5JDd5Zr9b1iq9K2lXJ4H7ndP5fHEWeQOnc/fuSFik5xuZKOr6EY3srJTVx+3Se1e6U9Pdm9ikz29NsvTeVximpUT+VtKWSweOVZvZXo3xdl2j9GnxIjbAvSNpXyTH5JiWf5P1j1evYWMmxepySWrhpwRS2kjRBrx7nl0o6WtIsJeeHXzSzHdPYtZL+p6QtJL1NSV3+lCSZ2RaSrlPyaePmkv4g6b9VFlLw7wSy3J1/ffhP0sOSnpf0nCSXdKukTXLiF0s6NP35OEk3pj/fK+njkq5Jf/+TpL3rzGO+pC+nP++QLnd6ZvqTkv4u8/sCSSfXmdccSb9Nf/5rScslWWb6zzPLukjSP1c9/w+S3tnt7cA//rX7X96xLulzkr5bFf8TScekP79yzKa/3ytp/8zv0yS9JGls5pjeMTP9x5KOy/y+gaQ/S9o+/d0lvT0z/VpJp2XyOKnG63mrpEeqHvu8pH+t8/o3lfQVSfcoOYFYLOkt6bR/knRtVX7LJc3OrLsD6qyL2ZKWVa3nAzK/V9bHWCVvJq2VtFFm+jlKPtGTkkHoLZlpu0lak7NNXdLOmefenJl2SLq9x6S/b5TG16zvkm6orGclb2Kdk5m2c2VZSt4gWy1pp8z0t0la2u19nH/8kzQ13e8r+3KjY/uVYyj9/ZXjOz22X5Q0IWd5YyR9WtIvJP1F0gq9WjffIekxSRtk4q+WdGaNZR0r6edV884e3+vVnfSxbF16UNLBmWnvkfRw5nWskTQ2M/1xSfvWeU23Sfp41XOr68hbM/GLJM2pM6+TlXyyKCVvsP0yM82UXGpeWVbu3wn+vfYfn5z1tznuvpGSg/ANSt7xkCSZ2Ufs1csAV0naIzP9dknvMLOtlBSo70naz8x2UPIOzeJR5JD97seaGr9PTvPZ0syuMbPlZvaspCsy+WwtabmnR3Xq0czP20s6pfJa0tezbfo8YBDUO9a3l3Rk1bHxdiWDrlq2l3R9JvZeJYOOqZmY6mPv/Ez8U0r+ME/PxDyW+fnPSo95Jcfog3Vy2Loq59OrcniFuz/t7qe5++5pzGJJN6TvdG+t5A2lSuy6NP/ptebVgq0lPeXuz2Ue+5Py18MEK/69veq6+YS/eiOSNen/lVp6kJndmV6WtErSwVq/lma3X/bnKZJeJ2lRZr3/3/RxoKvcfVjJzYvuSR9q9dgecfcXcpa31t0vdPf9lHzafbakeemlk1tLejRdZkX18V6W9V5n+nP23OZJd38583u2xjbyZI06Uu8cbVcz+zdLbjr1rKR/UZ26kp6rLcvMp8jfCWQwOBsA7n67kndnviZJ6bW+l0o6UdLm7r6JpLuVHCxy9weUHOCfkXRHesLxmJKP/n9eVZDKco6Sd232cvfXK/lovXIZwUpJ07OXFSg5sat4VNLZ7r5J5t/r3J2PzTFQqo91JcfGd6uOjUnu/pU6s3hU0kFV8RPcfXl2MVXxx1fFT3T3/yqQ7qOSdqrz+NKqeW7k7gc3mqG7P6HktW+t5JKlFUpODCRJaQ3ZVsk77NVWKxmcVGxVPfucRa+QtJmZbZR5bLs6y2mb9FLuBUrWwdS0tt+k9WvpNpmnZOvoE0pOxnbPrPeNPbnRDBBNo2P7z2r+eF4/0H2Nu1+o5DLv3dJlb5tegl1R73hfr66kb3qPJo/1Xme6nBUFUy/TRZLuk7RLeo52uurUlXRbZOtMK38nBhKDs8HxTUnvMrOZkiYpKQgjUvIFciWfnGXdrmTwVvl+2W1Vv5dtIyWXLKxKvyfzD5lpv1Ty7v2JZjbWzA5Vct11xaWSTjCzt1pikpn9bdWJEjAossf6FZIOMbP3mNkYM5tgyRfBt6nz3IslnV35sraZTUmPt3oulvR5M9s9jd84/S5ZEd+W9Fkzm5Uetzuny/1/kp615Mv6E9O89zCzt9SaiZl9NZ0+Nj3mPynpAU9uJHCtpL81s/3T74mcouQSpVonBYuV3BVus/QE6uSq6cOSdnzt0yR3fzSd5znpOt5LyeXhVxZcF2UZL2lDJbX9ZTM7SNK7M9OvlfRRS25m8Dplvk+Wvul2qZLvqG0pJd9ZzPmuH9BNjY7txZI+mNaPA/Xa71fmMrOT01o5Ma0txyg5T/mtpF8pGXSdambjLLnJ0iFKvlNa7XeSdjezmZbcafLMqul160rqakn/mNbiLZQcs1eM5rWUZCNJz0p63szeoKTOVvy7pD0tuaHIWCWXg2YHoa38nRhIDM4GhLuPSLpc0j+5++8lnadk0DMsaU8l11Vn3a7kYLyjzu9lO0vS3pKeUXKg/yCT+4tKvkh6nKRVSj5V+zclhVjuvlDSJ5R8wf5pJTdDOLZNeQKhVR3rj0o6VMm7nCNK3sH8B9Wv/ecruVnFT83sOSVfin9rzrKuV3ITiWvSS13uVnJDkiJ5fl/JpUJXKfm+3A2SNksvszlEyRfglyr5ROfbSi6pruV1Sm4YtErJzYu2V3LzDbn7H5TUi/+dzucQJTf1eLHGfL6r5ETqYSVf9P9e1fRzlJwkrTKzz9Z4/geUfA9tRZrPGe5+c946KFt6lcNnlJy4Pi3pg0q2Z2X6jyX9L0k/U1Inf5lO+kv6/+fSx+9Mt+ctkkZ1kwOgEwoc2yelj61SckfEhneZrrJGyXnSY+n8Py3pcHd/KF3G+5TUuickfUvSR9z9vhp53q/kpj23SPqjku/LZ31H0m5pXamV45clLZR0l6Qlkn6TPtZpn1VST55T8ibOK/UxvWLhSEnnKrm3wG5Kcq6cozX9d2JQ2fpf4wF6g5n9StLF7l7rFrgAgAbS78/cLWnDqu+tAEBT0ss9l0n6kLv/rNv59CI+OUNPMLN3mtlWmcsL9lLyZXUAQEFmdpiZjbfkdttflfQjBmYAWpFeOr9J+r3XyvfR7uxyWj2LwRl6xV8pueToGSXXlh/h7iu7mxIA9JzjlVzi+qCS7/J+Mj8cABp6m5KaUrnEdI67r8l/CurhskYAAAAACIBPzgAAAAAgAAZnAAAAABBA1wZnZnagmf3BzB4ws9O6lUdRZvawmS0xs8VmtrDb+VQzs3lm9riZ3Z15bDMzu9nM/pj+v2k3c8yqk++ZZrY8XceLzaxh09lOMbNtzexnZnavmd1jZielj4dcxzn5hl3HUVCbykVtai9q02ChPpWL+tQ+1KYWcunGd87MbIyk+yW9S8ntNn8t6QNp/62QzOxhSUNpP4dwzOyvlTRxvtzd90gfO1fSU+7+lbSIb+run+tmnhV18j1T0vPu/rVu5laLmU2TNM3df2NJo9tFkuYo6acWbh3n5Pt+BV3HEVCbykdtai9q0+CgPpWP+tQ+1KbmdeuTs30kPZBp5neNkkapaJK73yHpqaqHD5V0WfrzZUp2shDq5BuWu69099+kPz8n6V5J0xV0Hefki3zUppJRm9qL2jRQqE8loz61D7Wped0anE2X9Gjm92WKX5xd0k/NbJGZze12MgVNrdxuPv1/yy7nU8SJZnZX+tF9iI+6q5nZDpLeLOlX6oF1XJWv1APruIuoTZ0R/ripIfxxQ23qe9Snzgh/7NQQ+tihNo1OtwZnVuOx6Pf038/d95Z0kKRPpx8to1wXSdpJ0kxJKyWd1910XsvMJktaIOlkd3+22/k0UiPf8Ou4y6hNqCX8cUNtGgjUJ9QS+tihNo1etwZnyyRtm/l9G0krupRLIe6+Iv3/cUnXK7m8ILrh9BrayrW0j3c5n1zuPuzua919naRLFWwdm9k4JQfsle7+g/ThsOu4Vr7R13EA1KbOCHvc1BL9uKE2DQzqU2eEPXZqiXzsUJua063B2a8l7WJmM8xsvKSjJN3YpVwaMrNJ6ZcDZWaTJL1b0t35zwrhRknHpD8fI+mHXcylocrBmjpMgdaxmZmk70i6192/npkUch3XyzfyOg6C2tQZIY+beiIfN9SmgUJ96oyQx049UY8dalMLuXTjbo2SZMmtKL8paYykee5+dlcSKcDMdlTyjo8kjZV0VbR8zexqSbMlbSFpWNIZkm6QdK2k7SQ9IulIdw/xRdI6+c5W8rGxS3pY0vGV65K7zczeLuk/JS2RtC59+HQl1yOHW8c5+X5AQddxFNSmclGb2ovaNFioT+WiPrUPtamFXLo1OAMAAAAAvKprTagBAAAAAK9icAYAAAAAATA4AwAAAIAAGJwBAAAAQAAMzgAAAAAggK4OzsxsbjeXP1rk217k2169lm+39dr6It/2It/26rV8u6nX1hX5thf5tlc38u32J2c9tYFEvu1Gvu3Va/l2W6+tL/JtL/Jtr17Lt5t6bV2Rb3uRb3sN3OAMAAAAAKAON6HeYostfIcddnjl95GREU2ZMuWV3xctWqRZs2blzqNTMbWmk29/5rto0SJJ6ql8m5lHG/N9wt2n5AYFZ2be6vrq9vFFvuRbL99u59LNfCX1fH2Kcu4U8G8P+ZJvafPodL7KqU0tDc7M7EBJ50saI+nb7v6VvPihoSFfuHBh3vzUKJ9OxUTKpUhMpFyKxETLRVJP5RsllzRmkbsP5QZ1wWjqk5l5H24X8u1iDLk0H1PmciSFq0+9eu4UaR8pEhMplyIxkXIpEkMuzcc0qk1NX9ZoZmMkXSjpIEm7SfqAme3W7PwAoCzUJwARUZsANNLKd872kfSAuz/k7i9KukbSoeWkBQAtoT4BiIjaBCBXK4Oz6ZIezfy+LH1sPWY218wWmtnCkZGRFhYHAIU1rE/Z2tTRzAAMMs6dAORqZXBmNR57zQWW7n6Juw+5+1D2C3UA0EYN61O2NnUoJwDg3AlArlYGZ8skbZv5fRtJK1pLBwBKQX0CEBG1CUCuVgZnv5a0i5nNMLPxko6SdGM5aQFAS6hPACKiNgHINbbZJ7r7y2Z2oqSfKLkd7Dx3vyfvOYsWLXrltuX1NJreyZhIuRSJiZRLkZhIuRSJIZfe0Ux96sftQr7djSGX5mOoTa+KdO4UaR8pEhMplyIxkXIpEkMurcXUfW4nm1BH6dVRJCZSLkViIuVSJCZaLhJ9zlqICddHaLSMPmfhYyLlUiSGXJqPKXM5CtjnbLSinDtF2keKxETKpUhMpFyKxJBL8zGNalMrlzUCAAAAAErC4AwAAAAAAmBwBgAAAAABMDgDAAAAgAAYnAEAAABAAAzOAAAAACAABmcAAAAAEEBH+5yZWecWBqBTer6PELUJ6FvUJwAR1a1NYzuZxaxZsxShkWKRmEi5FImJlEuRmGi5SDShbiWmH/TjdiHf7sX0Yi5FRMp3UGpTlHOnSPt0kZhIuRSJiZRLkZho5xlR1kuRmEavh8saAQAAACAABmcAAAAAEACDMwAAAAAIgMEZAAAAAATA4AwAAAAAAmBwBgAAAAAB0OcMQKvoIwQgKuoTgIjq1iY+OQMAAACAAGhC3QO5FImJlEuRmGi5SDShbiWmH0TaLkVEyndQ9/teyTdSLkViqE3ri3LuFKlReZGYSPt0kZhIuRSJIZfmY2hCDQAAAAA9gMEZAAAAAATA4AwAAAAAAmBwBgAAAAABMDgDAAAAgAAYnAEAAABAADShBtAqmrwCiIr6BCCiurWJPmc9kEuRmEi5FImJlotEn7NWYvpBP24X8u1eDLk0H0NtWl+Uc6dI+0iRmEi5FImJlEuRGHJpPoY+ZwAAAADQAxicAQAAAEAADM4AAAAAIAAGZwAAAAAQAIMzAAAAAAiAwRkAAAAABMDgDAAAAAACoAk1gFbR5BVAVNQnABHRhHq0MZFyKRITKZciMdFykWhC3UpMP+jH7TKI+RYRKd9By6VIDLVpfVHOnSLtI0ViIuVSiWkkWr69sn4j5VIkptG+0NLgzMwelvScpLWSXu71d6cA9A/qE4CIqE0A8pTxydnfuPsTJcwHAMpGfQIQEbUJQE3cEAQAAAAAAmh1cOaSfmpmi8xsbq0AM5trZgvNbOHIyEiLiwOAwnLrU7Y2dSE3AIOLcycAdbV6WeN+7r7CzLaUdLOZ3efud2QD3P0SSZdI0tDQEHccAtApufUpW5u4GxqADuLcCUBdLX1y5u4r0v8fl3S9pH3KSAoAWkV9AhARtQlAnqYHZ2Y2ycw2qvws6d2S7i4rMQBoFvUJQETUJgCNNN2E2sx2VPKOj5RcHnmVu5/d4Dl8NA/0n3BNXkdbn6hNQN8KVZ84dwKQKr8Jtbs/JOlNo3lOlEaKRWIi5VIkJlIuRWKi5SLRhLqVmGiaqU/9uF0GMd8iIuXbidcUaTsWiaE2rS/KuVOkfaRITKRcKjGNRMu3lUbKFYO6rVtZd9xKHwAAAAACYHAGAAAAAAEwOAMAAACAABicAQAAAEAADM4AAAAAIAAGZwAAAAAQAIMzAAAAAAig6SbUTS2MRopAPwrV5LUZ1Cagb1GfAERUfhPqZkRppFgkJlIuRWLKXM7SpUtzY2bMmKEzzjgjN+ass87SvvvuW3f6nXfeqQ02yP/gdt26dYVirrnmmtyYo446Sr/4xS/qTt9vv/0kKfc1nXXWWYXWS69t635t9NqMftwug5hvJDShbi6G2rS+KOdOkfaRIjGRcqnEREIT6u7F0IQaAAAAAHoAgzMAAAAACIDBGQAAAAAEwOAMAAAAAAJgcAYAAAAAATA4AwAAAIAA6HMGoFX0EQIQFfUJQET0ORttTKRcisQUnUdevy8p6fk1ffr03Jjly5drzJgxuTFr167N7VG2bt06HXHEEbnzuO6663J7pUlJv7QiueTFrF27VpIa5vupT30qdznf+ta3Qm3rQe3d0qx+3C79li+aF2U7FomhNq0vyrlTpH2kSAx/B3tHr21r+pwBAAAAwABgcAYAAAAAATA4AwAAAIAAGJwBAAAAQAAMzgAAAAAgAAZnAAAAABAAgzMAAAAACIAm1ABaRZNXAFFRnwBERBPq0cZEyqUSU2mWXMuYMWP00EMP5c5jxx131MYbb5wb88wzzxRq6nz44YfnxixYsED33Xdf3elveMMbcl+PlLymv/zlL7kxG264oY4++ujcmCuuuKKUJtRXXHFF7nKOPvrogd03+0E/bpcymjr3434Uaf2Wpdf2zUZ6bZ9qpyjnTtH2Ealz+32nRFu/Zei1fbMImlADAAAAwABgcAYAAAAAATA4AwAAAIAAGJwBAAAAQAAMzgAAAAAgAAZnAAAAABAAgzMAAAAACIAm1ABaRZNXAFFRnwBERBPq0cZEyqUSc+utt9advv/++xdqHl0k5kMf+lBuzJVXXqkXXnghN2bChAmlrN/bbrstN2b27Nktv+5KE+qddtqpbsyDDz6o1atX5y5n0qRJA7tv9oN+3C6R8u2USOu3k3ptW3eqQXo/iHLuFKl+FYmJVpuk3jtOy9Br2zpCjWt4WaOZzTOzx83s7sxjm5nZzWb2x/T/TQtlCgAloj4BiIjaBKBZRb5zNl/SgVWPnSbpVnffRdKt6e8A0GnzRX0CEM98UZsANKHh4Mzd75D0VNXDh0q6LP35MklzSs4LABqiPgGIiNoEoFnN3q1xqruvlKT0/y3rBZrZXDNbaGYLR0ZGmlwcABRWqD5la1NHswMwqDh3AtBQ22+l7+6XuPuQuw9NmTKl3YsDgEKytanbuQBAFudOwOBqdnA2bGbTJCn9//HyUgKAllCfAEREbQLQULODsxslHZP+fIykH5aTDgC0jPoEICJqE4CGGjahNrOrJc2WtIWkYUlnSLpB0rWStpP0iKQj3b36i6+15kUjRaD/dK3Ja1n1idoE9K2u1CfOnQA0ULc2NRyclWloaMgjNFIsEtPpXM4///zcmJNOOkkbbFD/g85169YVasb8pS99KTfmi1/8YssNpqXy1u/MmTNzYxYvXlxaE+q8fXNoaGhg980CMV0bnJXFzLwPt0vPNWhtJMq6q8R0Cvtm8zHq4ptHZYly7hRpH6nERBJp/XZKlP0h4r5ZoAl13drU9huCAAAAAAAaY3AGAAAAAAEwOAMAAACAABicAQAAAEAADM4AAAAAIAAGZwAAAAAQAIMzAAAAAAigo33OaKQI9KWe7yNEbQL6FvUJQER1a9PYTmYxa9as3Ea/0RrElbWciy++ODfmhBNOyG0wLSVNpidOnFh3+po1a/Sxj30sdx7z5s3TSy+9lBszbty40l533rLGjRuXuy9ISePnyZMn58Y8//zzhdbdBRdcUHf6iSeeKCm/mWK/7pu91vCynfpxuxRoghlKr63fTikjl2jrpd+a8bZTlHOnfj2+iui19dspnagHlfmUse46kUuRmEa5cFkjAAAAAATA4AwAAAAAAmBwBgAAAAABMDgDAAAAgAAYnAEAAABAAAzOAAAAACAA+pwBaBV9hABERX0CEBF9zkYbU3Qea9asyY2ZOHGipk+fnhuzfPlyjRkzJjdm7dq1uX3M5s2b19F19+STT+bGbL755jrggAPqTr/lllsKveYiMZ/85CdzYy666CK9+OKLdaePHz9eUu/0WIqUSyWmH/TjdonU5yzS+o1kUGvGIG7rZkU5d+rXvyuR9vtoIu0PvZJLkRj6nAEAAABAD2BwBgAAAAABMDgDAAAAgAAYnAEAAABAAAzOAAAAACAABmcAAAAAEACDMwAAAAAIgCbUAFpFk1cAUVGfAEREE+rRxhSdx+rVq3NjJk2aVFqz5Weeeabu9I033rhQLvPnz8+NOfbYY3XwwQfnxtx0003aZJNNcmNWrVqlDTao/8HsunXrCr3mcePG5ca89NJLWrRoUW7MrFmzCjUD7JUGh5FyqcT0g37cLmXkWwT7UW29tq2j7A/UpvVFOXeKVL8qMY2wH9XXT/tDp/9W0YQaAAAAAAYAgzMAAAAACIDBGQAAAAAEwOAMAAAAAAJgcAYAAAAAATA4AwAAAIAAGJwBAAAAQAA0oQbQKpq8AoiK+gQgouabUJvZPEnvlfS4u++RPnampE9IGknDTnf3mxrNK0ojxSIxReexZs2a3JiJEydqm222yY1ZtmxZoYbMZTR1Litmxx13zI156KGHNGPGjLrTly5dWmg5RdZdWc0Ae6nZYpRcKjHdUmZ96sft0qkm1Kit17Z1lBhq0/qinDtF2kcqMWheP+0PkXIpElNGE+r5kg6s8fg33H1m+q9hcQGANpgv6hOAeOaL2gSgCQ0HZ+5+h6SnOpALAIwK9QlARNQmAM1q5YYgJ5rZXWY2z9FIvxQAAAq3SURBVMw2LS0jAGgd9QlARNQmALmaHZxdJGknSTMlrZR0Xr1AM5trZgvNbOHIyEi9MAAoS6H6lK1NnUwOwMDi3AlAQ00Nztx92N3Xuvs6SZdK2icn9hJ3H3L3oSlTpjSbJwAUUrQ+ZWtTZzMEMIg4dwJQRFODMzOblvn1MEl3l5MOALSG+gQgImoTgCKK3Er/akmzJW1hZssknSFptpnNlOSSHpZ0fBtzBICaqE8AIqI2AWgWTagBtIomrwCioj4BiKj5JtRlitJIsUhMmctZunRpbsyMGTM0derU3Jjh4eGGTaj32muv3HncddddOvfcc3NjTj31VA0PD+fGTJ06Vc8//3xuzOTJk3XYYYfVnX799dcXakJ99tln58Z84QtfoAl1gJh+0I/bpd+aUPfa/tpr2zpKDLVpfVHOnSLtI5WYSNrdmLhs/bQ/RMqlSEwZTagBAAAAAG3G4AwAAAAAAmBwBgAAAAABMDgDAAAAgAAYnAEAAABAAAzOAAAAACAABmcAAAAAEABNqAG0iiavAKKiPgGIiCbUo42JlEuRmE7ncv/99+fG7LrrrrnTJeU21ZaSxtoLFizIjTn88MNpQh0gph/043Yh39oxZemV9RsplyIxEbd1N0U5d4q0jxSJiZRLkZhO1aYiuRSJibbuouRSJIYm1AAAAADQAxicAQAAAEAADM4AAAAAIAAGZwAAAAAQAIMzAAAAAAiAwRkAAAAABECfMwCtoo8QgKioTwAios/ZaGMi5VIkptO5LFmyJDdmzz33zO1jtm7dOo0ZMyZ3HmvXrtXq1atzYyZNmkSfswAx/aAft0sZPXUi5Rttf+2V/SFSLkViIm7rbopy7hTx70qkfOlz1r2YSLkUiaHPGQAAAAD0AAZnAAAAABAAgzMAAAAACIDBGQAAAAAEwOAMAAAAAAJgcAYAAAAAATA4AwAAAIAAaEINoFU0eQUQFfUJQEQ0oR5tTKRcisREyqUSQxPq9sREyqUS0w86tV06kUuRmIj7Ua81Ju6V9RsplyIxEbd1N3Xy3KmRKPtIkZhIuRSJoQl1f+RSJIYm1AAAAADQAxicAQAAAEAADM4AAAAAIAAGZwAAAAAQAIMzAAAAAAiAwRkAAAAABMDgDAAAAAACoAk1gFbR5BVAVNQnABE134TazLaVdLmkrSStk3SJu59vZptJ+p6kHSQ9LOn97v503rxoQt2+mE7nsmTJktyYPffckybUbYqJlEslphvKrE1S55pQR4mJlEuRmIj7a6+s30i5FIkps1l7t/TjuVOkfaRITKRcisTQhLp3cimi3U2oX5Z0iru/UdK+kj5tZrtJOk3Sre6+i6Rb098BoFOoTQCioj4BaErDwZm7r3T336Q/PyfpXknTJR0q6bI07DJJc9qVJABUozYBiIr6BKBZo7ohiJntIOnNkn4laaq7r5SSIiRpy7KTA4AiqE0AoqI+ARiNwoMzM5ssaYGkk9392VE8b66ZLTSzhSMjI83kCAB1lVGb2pcdgEHGuROA0So0ODOzcUqKy5Xu/oP04WEzm5ZOnybp8VrPdfdL3H3I3YemTJlSRs4AIKm82tSZbAEMEs6dADSj4eDMkluKfEfSve7+9cykGyUdk/58jKQflp8eANRGbQIQFfUJQLMa3kpf0n6SPixpiZktTh87XdJXJF1rZsdJekTSke1JEQBqojYBiIr6BKApNKEG0CqavAKIivoEIKLmm1CXKUojxSIxkXIpEtPpXG644YbcmDlz5tCEuk0xkXKpxPSDMl5ntO3Sa/sRTajbExMplyIxEbd1NxU5dyoi0nbptf2IJtTtmUc/5lIkpowm1AAAAACANmNwBgAAAAABMDgDAAAAgAAYnAEAAABAAAzOAAAAACAABmcAAAAAEACDMwAAAAAIgCbUAFpFk1cAUVGfAEREE+rRxkTKpUhMp3NZuXJlbsy0adNyp0vKbVItJY2qaUIdO5dKTD+ItF2K6FTD0yj7WsT9tVeO00i5FImJuK27Kcq5U6drRqvLirRPF4mhCXV/5FIkhibUAAAAANADGJwBAAAAQAAMzgAAAAAgAAZnAAAAABAAgzMAAAAACIDBGQAAAAAEQJ8zAK2ijxCAqKhPACKiz9loYyLlUiQmUi6VmD322KPu9LvvvltjxozJncfatWu1dOnS3JgZM2bQ5yxATD/ox+1CvrVjytIr6zdiPei1bd1NUc6dIu3TRWIi5VIkhj5n7c+liAjbmssaAQAAACAABmcAAAAAEACDMwAAAAAIgMEZAAAAAATA4AwAAAAAAmBwBgAAAAABMDgDAAAAgABoQg2gVTR5BRAV9QlARDShHm1MpFyKxETKpRJz66231p2+//77F2pCfcQRR+TGXHfddRoeHs6NmTp1Kk2o2xzTD/pxu5Bv8zFlGNR1FynffhDl3CnSPlIkJlIuRWI6vU/30/qNlEuRGJpQAwAAAEAPYHAGAAAAAAEwOAMAAACAABicAQAAAEAADM4AAAAAIAAGZwAAAAAQAIMzAAAAAAiAJtQAWkWTVwBRUZ8ARNR8E2oz21bS5ZK2krRO0iXufr6ZnSnpE5JG0tDT3f2mvHlFaaRYJCZSLkViIuVSiXnhhRfqTp8wYYI++MEP5s7jqquuKtSo+oQTTsiNufjii/Xiiy/WnT5+/HhJNKFuJaYbyqxNUn815CwSU+a2jZRvr+3TERqedjKXIjG9uB1rLLvvzp0i1a9KTCPR8o3UhLqISMdpr+2brdTkhoMzSS9LOsXdf2NmG0laZGY3p9O+4e5fKzAPACgbtQlAVNQnAE1pODhz95WSVqY/P2dm90qa3u7EACAPtQlAVNQnAM0a1Q1BzGwHSW+W9Kv0oRPN7C4zm2dmm5acGwAUQm0CEBX1CcBoFB6cmdlkSQsknezuz0q6SNJOkmYqeXfovDrPm2tmC81s4cjISK0QAGhaGbWpY8kCGCicOwEYrUKDMzMbp6S4XOnuP5Akdx9297Xuvk7SpZL2qfVcd7/E3YfcfWjKlCll5Q0ApdWmzmUMYFBw7gSgGQ0HZ5bcUuQ7ku51969nHp+WCTtM0t3lpwcAtVGbAERFfQLQrCJ3a9xP0oclLTGzxeljp0v6gJnNlOSSHpZ0fFsyBIDaqE0AoqI+AWgKTagBtIomrwCioj4BiKj5JtRlitJIsUhMpFyKxETKpUhMoybVUtKo+qyzzsqNOeOMMwo1qh4eHq47ferUqZJ6p8FhpFwqMf2gH7cLTajbG9NIlHwjrrtIzW2ji3LuFGkfqcQ0Ei3fSPt9pHz7KZciMY220ahupQ8AAAAAaA8GZwAAAAAQAIMzAAAAAAiAwRkAAAAABMDgDAAAAAACYHAGAAAAAAEwOAMAAACAAGhCDaBVNHkFEBX1CUBENKEebUykXIrERMqlSEy0XCSaULcS0w9o6hw7JlIuRWLIpfkYatP6Onnu1EiUfaRITKRcisREyqVIDLk0H0MTagAAAADoAQzOAAAAACAABmcAAAAAEACDMwAAAAAIgMEZAAAAAATA4AwAAAAAAqDPGYBW0UcIQFTUJwARxehzJukJSX/q8DIBtNf23U6gBNQmoD9RnwBEVLc2dfSTMwAAAABAbXznDAAAAAACYHAGAAAAAAEwOAMAAACAABicAQAAAEAADM4AAAAAIID/Dxv/3UOisvt+AAAAAElFTkSuQmCC\n", 172 | "text/plain": [ 173 | "
" 174 | ] 175 | }, 176 | "metadata": { 177 | "needs_background": "light" 178 | }, 179 | "output_type": "display_data" 180 | }, 181 | { 182 | "name": "stdout", 183 | "output_type": "stream", 184 | "text": [ 185 | " Enter nothing to go to the next image\n", 186 | "or\n", 187 | " Enter \"s\" when you are done to recieve the three images. \n", 188 | " **Don't forget to do this before continuing to the next step.**\n", 189 | "s\n" 190 | ] 191 | } 192 | ], 193 | "source": [ 194 | "\n", 195 | "def test_thresh_noise(x, seed = 12345, p = noise_flip_prob, threshold = 128): \n", 196 | " np_random = np.random.RandomState(seed=seed)\n", 197 | " flip_flags = (np_random.uniform(0., 1., size=x.shape) < p)\n", 198 | " return get_thresholded_and_noised(x, threshold, flip_flags)\n", 199 | "\n", 200 | "(orig_image, ref_image, test_im, success_thr) = show_test_cases(test_thresh_noise, task_id='1_V')\n", 201 | "\n", 202 | "assert success_thr" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": 23, 208 | "metadata": { 209 | "deletable": false, 210 | "editable": false, 211 | "nbgrader": { 212 | "cell_type": "code", 213 | "checksum": "9ddfa2bdbc3cdfaccac1c378121cc61d", 214 | "grade": true, 215 | "grade_id": "cell-cad4a05d0f97d19d", 216 | "locked": true, 217 | "points": 0.8, 218 | "schema_version": 3, 219 | "solution": false, 220 | "task": false 221 | } 222 | }, 223 | "outputs": [], 224 | "source": [ 225 | "# Checking against the pre-computed test database\n", 226 | "test_results = test_case_checker(get_thresholded_and_noised, task_id=1)\n", 227 | "assert test_results['passed'], test_results['message']" 228 | ] 229 | }, 230 | { 231 | "cell_type": "markdown", 232 | "metadata": {}, 233 | "source": [ 234 | "## 0.2 Applying Thresholding and Noise to Data" 235 | ] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": 24, 240 | "metadata": {}, 241 | "outputs": [], 242 | "source": [ 243 | "if perform_computation:\n", 244 | " X_true_grayscale = train_images_raw[:10, :, :]\n", 245 | "\n", 246 | " np_random = np.random.RandomState(seed=12345)\n", 247 | " flip_flags = flip_flags = (np_random.uniform(0., 1., size=X_true_grayscale.shape) < noise_flip_prob)\n", 248 | " initial_pi = np_random.uniform(0, 1, size=X_true_grayscale.shape) # Initial Random Pi values\n", 249 | "\n", 250 | " X_true = get_thresholded_and_noised(X_true_grayscale, threshold=128, flip_flags=flip_flags * 0)\n", 251 | " X_noised = get_thresholded_and_noised(X_true_grayscale, threshold=128, flip_flags=flip_flags)" 252 | ] 253 | }, 254 | { 255 | "cell_type": "markdown", 256 | "metadata": {}, 257 | "source": [ 258 | "# Task 2" 259 | ] 260 | }, 261 | { 262 | "cell_type": "markdown", 263 | "metadata": {}, 264 | "source": [ 265 | "Write a funciton named `sigmoid_2x` that given a variable $X$ computes the following:\n", 266 | "\n", 267 | "$$f(X) := \\frac{\\exp(X)}{\\exp(X) + \\exp(-X)}$$\n", 268 | "\n", 269 | "The input argument is a numpy array $X$, which could have any shape. Your output array must have the same shape as $X$.\n", 270 | "\n", 271 | "**Important Note**: Theoretically, $f$ satisfies the following equations:\n", 272 | "\n", 273 | "$$\\lim_{X\\rightarrow +\\infty} f(X) = 1$$\n", 274 | "$$\\lim_{X\\rightarrow -\\infty} f(X) = 0$$\n", 275 | "\n", 276 | "Your implementation must also work correctly even on these extreme edge cases. In other words, you must satisfy the following tests.\n", 277 | "* `sigmoid_2x(np.inf)==1` \n", 278 | "* `sigmoid_2x(-np.inf)==0`.\n", 279 | "\n", 280 | "**Hint**: You may find `scipy.special.expit` useful." 281 | ] 282 | }, 283 | { 284 | "cell_type": "code", 285 | "execution_count": 25, 286 | "metadata": { 287 | "deletable": false, 288 | "nbgrader": { 289 | "cell_type": "code", 290 | "checksum": "9467e4711dfb00723c4a04f641d3a4bb", 291 | "grade": false, 292 | "grade_id": "cell-baba53895e886588", 293 | "locked": false, 294 | "schema_version": 3, 295 | "solution": true, 296 | "task": false 297 | } 298 | }, 299 | "outputs": [], 300 | "source": [ 301 | "def sigmoid_2x(X):\n", 302 | " \n", 303 | " # your code here\n", 304 | " output = expit(2*X)\n", 305 | " \n", 306 | " return output" 307 | ] 308 | }, 309 | { 310 | "cell_type": "code", 311 | "execution_count": 26, 312 | "metadata": { 313 | "deletable": false, 314 | "editable": false, 315 | "nbgrader": { 316 | "cell_type": "code", 317 | "checksum": "05703316807b847b94777d150f813ef1", 318 | "grade": true, 319 | "grade_id": "cell-4e87b3b9548c3052", 320 | "locked": true, 321 | "points": 1, 322 | "schema_version": 3, 323 | "solution": false, 324 | "task": false 325 | } 326 | }, 327 | "outputs": [], 328 | "source": [ 329 | "assert sigmoid_2x(+np.inf) == 1.\n", 330 | "assert sigmoid_2x(-np.inf) == 0.\n", 331 | "assert np.array_equal(sigmoid_2x(np.array([0, 1])).round(3), np.array([0.5, 0.881]))\n", 332 | "\n", 333 | "\n", 334 | "# Checking against the pre-computed test database\n", 335 | "test_results = test_case_checker(sigmoid_2x, task_id=2)\n", 336 | "assert test_results['passed'], test_results['message']" 337 | ] 338 | }, 339 | { 340 | "cell_type": "markdown", 341 | "metadata": {}, 342 | "source": [ 343 | "# 1. Applying Mean-field Approximation to Boltzman Machine's Variational Inference Problem" 344 | ] 345 | }, 346 | { 347 | "cell_type": "markdown", 348 | "metadata": {}, 349 | "source": [ 350 | "# Task 3" 351 | ] 352 | }, 353 | { 354 | "cell_type": "markdown", 355 | "metadata": {}, 356 | "source": [ 357 | "Write a `boltzman_meanfield` function that applies the mean-field approximation to the Boltzman machine. \n", 358 | "\n", 359 | "Recalling the textbook notation, $X_i$ is the observed value of pixel $i$, and $H_i$ is the true value of pixel $i$ (before applying noise). For instance, if we have a $3 \\times 3$ image, the corresponding Boltzman machine looks like this: \n", 360 | "\n", 361 | "```\n", 362 | " X_1 X_2 X_3\n", 363 | " / / /\n", 364 | " H_1 ------ H_2 ------ H_3\n", 365 | " | | |\n", 366 | " | | |\n", 367 | " | | |\n", 368 | " | X_4 | X_5 | X_6\n", 369 | " |/ |/ |/ \n", 370 | " H_4 ------ H_5 ------ H_6\n", 371 | " | | |\n", 372 | " | | |\n", 373 | " | | |\n", 374 | " | X_7 | X_8 | X_9\n", 375 | " |/ |/ |/ \n", 376 | " H_7 ------ H_8 ------ H_9\n", 377 | "``` \n", 378 | "\n", 379 | "Here, we a adopt a slightly simplified notation from the textbook and define $\\mathcal{N}(i)$ to be the neighbors of pixel $i$ (the pixels adjacent to pixel $i$). For instance, in the above figure, we have $\\mathcal{N}(1) = \\{2,4\\}$, $\\mathcal{N}(2) = \\{1,3,5\\}$, and $\\mathcal{N}(5) = \\{2,4,6,8\\}$.\n", 380 | "\n", 381 | "\n", 382 | "With this, the process in the textbook can be summarized as follows:\n", 383 | "\n", 384 | "```\n", 385 | "1. for iteration = 1, 2, 3, ....,\n", 386 | " 2. Pick a random pixel i.\n", 387 | " 3. Find pixel i's new parameter as\n", 388 | "```\n", 389 | "$$\\pi_i^{\\text{new}} = \\frac{\\exp(\\theta_{ii}^{(2)} X_i + \\sum_{j\\in \\mathcal{N}(i)} \\theta_{ij}^{(1)} (2\\pi_j -1))}{\\exp(\\theta_{ii}^{(2)} X_i + \\sum_{j\\in \\mathcal{N}(i)} \\theta_{ij}^{(1)} (2\\pi_j -1)) + \\exp(-\\theta_{ii}^{(2)} X_i - \\sum_{j\\in \\mathcal{N}(i)} \\theta_{ij}^{(1)} (2\\pi_j -1))} .$$\n", 390 | "```\n", 391 | " 4. Replace the existing parameter for pixel i with the new one.\n", 392 | "```\n", 393 | "$$\\pi_i \\leftarrow \\pi_i^{\\text{new}}$$\n", 394 | "\n", 395 | "Since our computational resources are extremely vectorized, we will make the following minor algorithmic modification and ask you to implement the following instead:\n", 396 | "\n", 397 | "```\n", 398 | "1. for iteration = 1, 2, 3, ....,\n", 399 | " 2. for each pixels i:\n", 400 | " 3. Find pixel i's new parameter, but do not update the original parameter yet.\n", 401 | "```\n", 402 | "$$\\pi_i^{\\text{new}} = \\frac{\\exp(\\theta_{ii}^{(2)} X_i + \\sum_{j\\in \\mathcal{N}(i)} \\theta_{ij}^{(1)} (2\\pi_j -1))}{\\exp(\\theta_{ii}^{(2)} X_i + \\sum_{j\\in \\mathcal{N}(i)} \\theta_{ij}^{(1)} (2\\pi_j -1)) + \\exp(-\\theta_{ii}^{(2)} X_i - \\sum_{j\\in \\mathcal{N}(i)} \\theta_{ij}^{(1)} (2\\pi_j -1))} .$$\n", 403 | "```\n", 404 | " 4. Once you have computed all the new parameters, update all of them at the same time:\n", 405 | "```\n", 406 | "$$\\pi \\leftarrow \\pi^{\\text{new}}$$\n", 407 | "\n", 408 | "We assume that the parameters $\\theta_{ii}^{(2)}$ have the same value for all $i$ and denote their common value by scalar `theta_X`. Moreover, we assume that the parameters $\\theta_{ij}^{(1)}$ have the same value for all $i,j$ and denote their common value by scalar `theta_pi`.\n", 409 | "\n", 410 | "The `boltzman_meanfield` function must take the following input arguments:\n", 411 | "1. `images`: A numpy array with the shape `(N,height,width)`, where \n", 412 | " * `N` is the number of samples and could be anything,\n", 413 | " * `height` is each individual image's height in pixels (i.e., number of rows in each image),\n", 414 | " * and `width` is each individual image's width in pixels (i.e., number of columns in each image).\n", 415 | " * Do not assume anything about `images`'s dtype or the number of samples or the `height` or the `width`.\n", 416 | " * The entries of `images` are either -1 or 1.\n", 417 | "2. `initial_pi`: A numpy array with the same shape as `images` (i.e. `(N,height,width)`). This variable is corresponding to the initial value of $\\pi$ in the textbook analysis and above equations. Note that for each of the $N$ images, we have a different $\\pi$ variable.\n", 418 | "\n", 419 | "3. `theta_X`: A scalar with a default value of `0.5*np.log(1/noise_flip_prob-1)`. This variable represents $\\theta_{ii}^{(2)}$ in the above update equation.\n", 420 | "\n", 421 | "4. `theta_pi`: A scalar with a default value of 2. This variable represents $\\theta_{ij}^{(1)}$ in the above update equation.\n", 422 | "\n", 423 | "5. `iterations`: A scalar with a default value of 100. This variable denotes the number of update iterations to perform.\n", 424 | "\n", 425 | "The `boltzman_meanfield` function must return the final $\\pi$ variable as a numpy array called `pi`, and should contain values that are between 0 and 1. \n", 426 | "\n", 427 | "**Hint**: You may find the `sigmoid_2x` function, that you implemented earlier, useful.\n", 428 | "\n", 429 | "**Hint**: If you want to find the summation of neighboring elements for all of a 2-dimensional matrix, there is an easy and efficient way using matrix operations. You can initialize a zero matrix, and then add four shifted versions (i.e., left-, right-, up-, and down-shifted versions) of the original matrix to it. You will have to be careful in the assignment and selection indices, since you will have to drop one row/column for each shifted version of the matrix.\n", 430 | " * Do **not** use `np.roll` if you're taking this approach." 431 | ] 432 | }, 433 | { 434 | "cell_type": "code", 435 | "execution_count": 27, 436 | "metadata": { 437 | "deletable": false, 438 | "nbgrader": { 439 | "cell_type": "code", 440 | "checksum": "0cd68da6ffa91cc0796f53f49b7ab71a", 441 | "grade": false, 442 | "grade_id": "cell-e47949a00f04759c", 443 | "locked": false, 444 | "schema_version": 3, 445 | "solution": true, 446 | "task": false 447 | } 448 | }, 449 | "outputs": [], 450 | "source": [ 451 | "def boltzman_meanfield(images, initial_pi, theta_X=0.5*np.log(1/noise_flip_prob-1), theta_pi=2, iterations=100):\n", 452 | " if len(images.shape)==2:\n", 453 | " # In case a 2d image was given as input, we'll add a dummy dimension to be consistent\n", 454 | " X = images.reshape(1,*images.shape)\n", 455 | " else:\n", 456 | " # Otherwise, we'll just work with what's given\n", 457 | " X = images\n", 458 | " \n", 459 | " pi = initial_pi\n", 460 | " # your code here\n", 461 | " for i in range(iterations):\n", 462 | " left = np.pad(pi, ((0,0), (0,0), (1,0)), mode='constant')[:, :, :-1]\n", 463 | " right = np.pad(pi, ((0,0), (0,0), (0,1)), mode='constant')[:, :, 1:]\n", 464 | " up = np.pad(pi, ((0,0), (1,0), (0,0)), mode='constant')[:, :-1, :]\n", 465 | " down = np.pad(pi, ((0,0), (0,1), (0,0)), mode='constant')[:, 1:, :]\n", 466 | " L = theta_pi * np.where(left==0, left, 2 * left - 1)\n", 467 | " R = theta_pi * np.where(right==0, right, 2 * right - 1)\n", 468 | " U = theta_pi * np.where(up==0, up, 2 * up - 1)\n", 469 | " D = theta_pi * np.where(down==0, down, 2 * down - 1)\n", 470 | " \n", 471 | " pi = sigmoid_2x(theta_X * images + (L+R+U+D))\n", 472 | " \n", 473 | " return pi.reshape(*images.shape)" 474 | ] 475 | }, 476 | { 477 | "cell_type": "code", 478 | "execution_count": 28, 479 | "metadata": { 480 | "deletable": false, 481 | "editable": false, 482 | "nbgrader": { 483 | "cell_type": "code", 484 | "checksum": "5753989d3e0dad787a6833c39262fb89", 485 | "grade": true, 486 | "grade_id": "cell-6291d0a80ccca660", 487 | "locked": true, 488 | "points": 0.2, 489 | "schema_version": 3, 490 | "solution": false, 491 | "task": false 492 | } 493 | }, 494 | "outputs": [ 495 | { 496 | "name": "stdout", 497 | "output_type": "stream", 498 | "text": [ 499 | "The reference and solution images are the same to a T! Well done on this test case.\n" 500 | ] 501 | }, 502 | { 503 | "data": { 504 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAAElCAYAAABgRJorAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de7gkBXnn8d9vmAvIoAww3G8RMBG8IIyoi0byeAMjgrvBaBJFF4UkssoGo0gukqwG9REvuxJdUAIogrgIYqKraBTiRl1nFAXECyhymWE4wCCgszDDvPtH1WFqmtPddbqru9/u/n6eZ5453fV293uqq95Tb3d1v44IAQAAAABGa8GoEwAAAAAA0JwBAAAAQAo0ZwAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAMKVs72L7GtsP2D5r1PkMm+1bbL+wx9vubftB21s1nReA8WX7fNvv6uP2N9g+osGUMGZoziZUedCxvjx4uLMsFksH/Jh9FSQA89fnvn6ipLslPT4iTh1gmgNje0/bl9m+2/avbF9n+3UDeJwtGrmIuDUilkbEIwN4rLC9f9P3C4wb2xfZPq/luufbvsf2bgN83MW2z7J9e1lbf2H7gwN4nMccN0XEQRHxjQE81jdsv6Hp+0XzaM4m29ERsVTSwZKeIekdI84HwGD0uq/vI+lHERHzfUDbC+d7mwH5pKTbVPwuO0p6raS1I80IQFPeLOmltl8kSba3lnSupFMjYk0TD9Cmlr1D0gpJh0naTtLvSfp+E48HdENzNgUi4k5JX1Zx4CZJsn2a7ZvL05l+ZPsVlWW/tH1o+fOflK/iHlhefoPtK7o9pu19y9u93vZtttfZ/lPbz7T9Q9v32f5IJX4/2/9avhp2d/lq2faV5YfY/n6Z72dtf6b6apPtl9m+trzff7f9tH7XGzBu2uzrzy73ifts/2D2dBnb50s6XtLbyleGX2h7QaU23GP7Uts7lPGz+/QJtm+V9K/l9f/Z9o3lPv5l2/tUHjvK/f5n5fKzbbuy/I3lbWfr0CHl9buX74bNlK9Yv7nDr/1MSedHxK8jYmNEfD8ivlR5jJeXpwndV75y/OS57qT1FWzbR9i+vfz5k5L2lvSFcl29rbI+FlZyvtL2vbZvsv3Gyn2dUa7LC8vf9QbbKzo+mVve9rO2P1Xe9jrbT7L9Dtt3lfX1xZX411fW6c9tn9Ryf2+zvcb26rKeP/oune0ltt9v+1bba21/zPY2dfIEBiEi7pH0XySdY3tbSe+UdHNEnN9p33bLu8/V/Xt237b9dtt3SvqnOR76mZIuj4jVUbglIi6s3N+Ty8e8r8zh5XPlb/t1tr/Zcl3Y3t/2iZL+WJtr8BfK5Y++S1/ukx8q99fV5c9LWn6PU8tasMb26+us18pt31a57bG2X2r7p2UdO70Sf5jtb5W/7xrbH7G9uLL8xbZ/4uLshX+0fbUr79K5w98JPBbN2RSwvaekoyTdVLn6ZknPk/QESX8n6VPefIrA1ZKOKH/+XUk/l/T8yuWr5/Hwz5J0gKQ/lPQhSX8l6YWSDpL0Stuz92tJZ0raXdKTJe0l6Ywy/8WSLpd0vqQdJF0sqdpMHiLpPEknqXjl/H9KunK2gAHTonVft72HpH+R9C4V+85bJV1me3lEvE7SRZLeV56e91UVr1Ifq2J/313SOklntzzM81Xsoy+xfayk0yX9R0nLJf2biv2z6mUqDnSeLumVkl5S5nacin38tZIeL+nlku6xvUDSFyT9QNIekl4g6RTbL2nza39b0tm2X2V775b18aQyn1PK/L6oosFa/Ni7aS8iXiPpVpXvUEbE++YIu1jS7SrW2x9I+gfbL6gsf7mkSyRtL+lKSR95zD20d7SKdwiXqXj1/ssq/n7vIenvVdS8WXepWOePl/R6SR/05qb3SEl/oaIG76/NdX3WeyU9SUVzv395/387jzyBxkXEZyWtUrGPnSjppAb27V1V1MR9yvts9W1Jf2H7z20/1d7iRaVFKmrUVyTtrKJ5vMj2b8/z9zpHW9bgo+cI+ytJz1axTz5dxTt5f93yezxBxb56gopauKxmCrtK2lqb9/NzJf2JpENVHB/+re0nlrGPSPqvknaS9BwVdfnPJcn2TpL+l4p3G3eU9BNJ/2H2QWr+nUBVRPBvAv9JukXSg5IekBSSviZp+w7x10o6pvz5BElXlj/fKOkNki4pL/9S0iFt7uN8Se8qf963fNw9KsvvkfSHlcuXSTqlzX0dK+n75c+/K+kOSa4s/2blsT4q6b+13P4nkp4/6ueBf/wb9L9O+7qkt0v6ZEv8lyUdX/786D5bXr5R0gsql3eTtEHSwso+/cTK8i9JOqFyeYGk30jap7wckp5bWX6ppNMqebxljt/nWZJubbnuHZL+qc3vv0zSeyTdoOIA4lpJzyyX/Y2kS1vyu0PSEZV198I26+IISbe3rOcXVi7Pro+FKl5MekTSdpXlZ6p4R08qmtCvVpYdKGl9h+c0JO1fue1VlWVHl8/3VuXl7cr4Oeu7pCtm17OKF7HOrCzbf/axVLxA9mtJ+1WWP0fSL0a9jfOPf5J2Kbf72W2527796D5UXn50/y737Yclbd3h8baS9CZJ/0fSQ5JWa3PdfJ6kOyUtqMRfLOmMOR7rdZK+2XLf1f17i7pTXletSzdLemll2Usk3VL5PdZLWlhZfpekZ7f5nb4h6Q0tt22tI8+qxK+SdGyb+zpFxTuLUvEC27cqy6ziVPPZx+r4d4J/j/3HO2eT7diI2E7FTvg7Kl7xkCTZfq03nwZ4n6SnVJZfLel5tndVUaA+I+lw2/uqeIXm2nnkUP3sx/o5Li8t89nZ9iW277B9v6RPVfLZXdIdUe7VpdsqP+8j6dTZ36X8ffYqbwdMg3b7+j6SjmvZN56roumayz6SLq/E3qii6dilEtO67324En+vij/Me1Ri7qz8/BuV+7yKffTmNjns3pLz6S05PCoi1kXEaRFxUBlzraQryle6d1fxgtJs7KYy/z3muq8+7C7p3oh4oHLdL9V5PWzt+p/ba62bd8fmLyJZX/4/W0uPsv3t8rSk+yS9VFvW0urzV/15uaTHSVpVWe//u7weGKmIWKviy4tuKK/qd9+eiYj/1+HxHomIsyPicBXvdr9b0nnlqZO7S7qtfMxZrft7U7b4Pcufq8c290TExsrlao3t5p456ki7Y7Qn2f5nF186db+kf1CbulIeq91euZ86fydQQXM2BSLiahWvzrxfkspzfc+VdLKkHSNie0nXq9hZFBE3qdjB3yzpmvKA404Vb/1/s6UgNeVMFa/aPC0iHq/irfXZ0wjWSNqjelqBigO7WbdJendEbF/597iI4G1zTJXWfV3FvvHJln1j24h4T5u7uE3SUS3xW0fEHdWHaYk/qSV+m4j49xrp3iZpvzbX/6LlPreLiJd2u8OIuFvF7767ilOWVqs4MJAklTVkLxWvsLf6tYrmZNaurXff4aFXS9rB9naV6/Zu8zgDU57KfZmKdbBLWdu/qC1r6Z6Vm1Tr6N0qDsYOqqz3J0TxRTNANt327d+o9/15y8CI9RFxtorTvA8sH3uv8hTsWe329y3qSvmi93zy2OL3LB9ndc3Um/RRST+WdEB5jHa62tSV8rmo1pl+/k5MJZqz6fEhSS+yfbCkbVUUhBmp+AC5infOqq5W0bzNfr7sGy2Xm7adilMW7is/J/OXlWXfUvHq/cm2F9o+RsV517POlfSntp/lwra2f7/lQAmYFtV9/VOSjrb9Ettb2d7axQfB92xz249Jevfsh7VtLy/3t3Y+Jukdtg8q459Qfpasjo9LeqvtQ8v9dv/ycf+vpPtdfFh/mzLvp9h+5lx3Yvu95fKF5T7/Z5JuiuKLBC6V9Pu2X1B+TuRUFacozXVQcK2Kb4XboTyAOqVl+VpJT3zszaSIuK28zzPLdfw0FaeHX1RzXTRlsaQlKmr7RttHSXpxZfmlkl7v4ssMHqfK58nKF93OVfEZtZ2l4jOLHT7rB4xSt337Wkl/VNaPI/XYz1d2ZPuUslZuU9aW41Ucp3xf0ndUNF1vs73IxZcsHa3iM6WtfiDpINsHu/imyTNalretK6WLJf11WYt3UrHPfmo+v0tDtpN0v6QHbf+Oijo7618kPdXFF4osVHE6aLUJ7efvxFSiOZsSETEj6UJJfxMRP5J0loqmZ62kp6o4r7rqahU74zVtLjft7yQdIulXKnb0z1Vyf1jFB0lPkHSfinfV/llFIVZErJT0RhUfsF+n4ssQXjegPIHUWvb12yQdo+JVzhkVr2D+pdrX/g+r+LKKr9h+QMWH4p/V4bEuV/ElEpeUp7pcr+ILSerk+VkVpwp9WsXn5a6QtEN5ms3RKj4A/wsV7+h8XMUp1XN5nIovDLpPxZcX7aPiyzcUET9RUS/+R3k/R6v4Uo+H57ifT6o4kLpFxQf9P9Oy/EwVB0n32X7rHLd/tYrPoa0u83lnRFzVaR00rTzL4c0qDlzXSfojFc/n7PIvSfrvkr6uok5+q1z0UPn/28vrv10+n1+VNK8vOQCGoca+/ZbyuvtUfCNi12+ZbrFexXHSneX9v0nSf4qIn5eP8XIVte5uSf8o6bUR8eM58vypii/t+aqkn6n4vHzVJyQdWNaVuXJ8l6SVkn4o6TpJ3yuvG7a3qqgnD6h4EefR+liesXCcpPep+G6BA1XkPHuM1vPfiWnlLT/GA4wH29+R9LGImOsrcAEAXZSfn7le0pKWz60AQE/K0z1vl/THEfH1UeczjnjnDGPB9vNt71o5veBpKj6sDgCoyfYrbC928XXb75X0BRozAP0oT53fvvzc6+zn0b494rTGFs0ZxsVvqzjl6Fcqzi3/g4hYM9qUAGDsnKTiFNebVXyW9886hwNAV89RUVNmTzE9NiLWd74J2uG0RgAAAABIgHfOAAAAACABmjMAAAAASGBkzZntI23/xPZNtk8bVR512b7F9nW2r7W9ctT5tLJ9nu27bF9fuW4H21fZ/ln5/7JR5ljVJt8zbN9RruNrbXcdOjsstvey/XXbN9q+wfZbyutTruMO+aZdx1lQm5pFbRosatN0oT41i/o0ONSmPnIZxWfObG8l6aeSXqTi6za/K+nV5fytlGzfImlFOc8hHdu/q2KI84UR8ZTyuvdJujci3lMW8WUR8fZR5jmrTb5nSHowIt4/ytzmYns3SbtFxPdcDLpdJelYFfPU0q3jDvm+UknXcQbUpuZRmwaL2jQ9qE/Noz4NDrWpd6N65+wwSTdVhvldomJQKnoUEddIurfl6mMkXVD+fIGKjSyFNvmmFRFrIuJ75c8PSLpR0h5Kuo475IvOqE0NozYNFrVpqlCfGkZ9GhxqU+9G1ZztIem2yuXblb84h6Sv2F5l+8RRJ1PTLrNfN1/+v/OI86njZNs/LN+6T/FWdyvb+0p6hqTvaAzWcUu+0his4xGiNg1H+v1mDun3G2rTxKM+DUf6fWcOqfcdatP8jKo58xzXZf9O/8Mj4hBJR0l6U/nWMpr1UUn7STpY0hpJZ402nceyvVTSZZJOiYj7R51PN3Pkm34djxi1CXNJv99Qm6YC9QlzSb3vUJvmb1TN2e2S9qpc3lPS6hHlUktErC7/v0vS5SpOL8hubXkO7ey5tHeNOJ+OImJtRDwSEZsknatk69j2IhU77EUR8bny6rTreK58s6/jBKhNw5F2v5lL9v2G2jQ1qE/DkXbfmUvmfYfa1JtRNWfflXSA7d+yvVjSqyRdOaJcurK9bfnhQNneVtKLJV3f+VYpXCnp+PLn4yV9foS5dDW7s5ZeoUTr2LYlfULSjRHxgcqilOu4Xb6Z13ES1KbhSLnftJN5v6E2TRXq03Ck3HfaybrvUJv6yGUU39YoSS6+ivJDkraSdF5EvHskidRg+4kqXvGRpIWSPp0tX9sXSzpC0k6S1kp6p6QrJF0qaW9Jt0o6LiJSfJC0Tb5HqHjbOCTdIumk2fOSR832cyX9m6TrJG0qrz5dxfnI6dZxh3xfraTrOAtqU7OoTYNFbZou1KdmUZ8Gh9rURy6jas4AAAAAAJuNbAg1AAAAAGAzmjMAAAAASIDmDAAAAAASoDkDAAAAgARozgAAAAAggZE2Z7ZPHOXjzxf5Dhb5Dta45Ttq47a+yHewyHewxi3fURq3dUW+g0W+gzWKfEf9ztlYPUEi30Ej38Eat3xHbdzWF/kOFvkO1rjlO0rjtq7Id7DId7CmrjkDAAAAAGjIQ6h32mmn2HfffR+9PDMzo+XLlz96edWqVTr00EM73sewYuZaPi351pEp3yZykTr/Ttny7eU+Bpjv3RGxvGNQcraj3/U16npAvuTbLt9R5zLKfCWNfX3KcuyU8G8P+ZJvY/cx7HzVoTb11ZzZPlLShyVtJenjEfGeTvErVqyIlStXdro/dctnWDGZcqkT0+Tj1JEp3yZykTr/TtnyzZJLGbMqIlZ0DBqB+dQn2zGBzwv5jjCGXHqPafjvWbr6NK7HTpm2kToxmXKpE5Mplzox5NJ7TLfa1PNpjba3knS2pKMkHSjp1bYP7PX+AKAp1CcAGVGbAHTTz2fODpN0U0T8PCIelnSJpGOaSQsA+kJ9ApARtQlAR/00Z3tIuq1y+fbyui3YPtH2StsrZ2Zm+ng4AKita32q1qahZgZgmnHsBKCjfpqzuT6c9JgTLCPinIhYERErqh+oA4AB6lqfqrVpSDkBAMdOADrqpzm7XdJelct7SlrdXzoA0AjqE4CMqE0AOuqnOfuupANs/5btxZJeJenKZtICgL5QnwBkRG0C0NHCXm8YERttnyzpyyq+Dva8iLih021WrVrV9ava63yV+7BiMuVSJ6apx6kjU77jtn4nLZeMeqlPk/i8kO9oY8il9xhq02aZjp0ybSN1YjLlUicmUy51Ysilv5i2tx3mEOqmZnXU0dTsq0E/TlMxmeY31InJlovEnLM+YtLNEZovM+csfUymXOrEkEvvMU0+jhLOOZsv5pz1FpMplzoxmXKpE0Muvcd0q039nNYIAAAAAGgIzRkAAAAAJEBzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQAM0ZAAAAACRAcwYAAAAACQx1zpnt4T0YgGEZ+zlC1CZgYlGfAGTUtjYtHGYWhx56qDIMUqwTkymXOjGZcqkTky0XiSHU/cRMgkl8Xsh3dDHk0nsMtWlLWY6dMm0jdWIy5VInJlMudWLIpfeYbrWJ0xoBAAAAIAGaMwAAAABIgOYMAAAAABKgOQMAAACABGjOAAAAACABmjMAAAAASIA5ZwD6xRwhAFlRnwBk1LY28c4ZAAAAACTAEOoxyKVOTKZc6sRky0ViCHU/MZNgEp8X8h1dDLn0HkNt2lKWY6dM20idmEy51InJlEudGHLpPYYh1AAAAAAwBmjOAAAAACABmjMAAAAASIDmDAAAAAASoDkDAAAAgARozgAAAAAgAYZQA+gXQ14BZEV9ApBR29o0lnPO6pjGmQlZcqkTky2XOsi3v3yym6Rtuk5MplzqxGTKpU4MufQeQ23aEnPOeovJlEudmEy51Ikhl95jmHMGAAAAAGOA5gwAAAAAEqA5AwAAAIAEaM4AAAAAIAGaMwAAAABIgOYMAAAAABKgOQMAAACABBhCDaBfDHkFkBX1CUBGkzWEmoF2uXOpE5MtF6nzEGIGpHeOmQST+LyQ7+hiyKX3GGrTlrIcO2XaRurEZMqlTkymXOrEkEvvMd1qU1/Nme1bJD0g6RFJG8f91SkAk4P6BCAjahOATpp45+z3IuLuBu4HAJpGfQKQEbUJwJz4QhAAAAAASKDf5iwkfcX2KtsnzhVg+0TbK22vnJmZ6fPhAKC2jvWpWptGkBuA6cWxE4C2+j2t8fCIWG17Z0lX2f5xRFxTDYiIcySdI0krVqzgG4cADEvH+lStTXwbGoAh4tgJQFt9vXMWEavL/++SdLmkw5pICgD6RX0CkBG1CUAnPTdntre1vd3sz5JeLOn6phIDgF5RnwBkRG0C0E3PQ6htP1HFKz5ScXrkpyPi3V1uw1vzwORJN+R1vvWJ2gRMrFT1iWMnAKXmh1BHxM8lPX0+t8kySLFOTKZc6sQMeyDnuOWbSaZ1N6mDXnupT9SM3DHDzmXDhg0dYxYtWqRtttmm7fL169d3vP18LFu2rOPydevWaccdd2y7/J577knzPNaJmeQh1ON87JRpG6kTkymXOjHDqk1Sc/Wp39okNVOfMj2PdWK61Sa+Sh8AAAAAEqA5AwAAAIAEaM4AAAAAIAGaMwAAAABIgOYMAAAAABKgOQMAAACABGjOAAAAACCBnodQ9/RgDFIEJlGqIa+9oDYBE4v6BCCj5odQ9yLLIMU6MZlyqRMz7Fwwt0l8rqdle5jE52XS8t24cWPH+1i4cOHQhq9mM2nPNbVpsyzHTpm2kToxw86l3/o0jbVJGs/nmiHUAAAAADAFaM4AAAAAIAGaMwAAAABIgOYMAAAAABKgOQMAAACABGjOAAAAACAB5pwB6BdzhABkRX0CkBFzzuYbkymXOjF157lMy1yYVnXW3bhsD5lymY2ZBJP4vGTKd9OmTR1jFixYoA0bNrRdvmjRIi1ZsqTjfTz00EMdlw9bnZlrnX5nqfi9x+25zpTvJMhy7JRpG6kTM6zaJE1nfWqiNjUVk2m7qxPDnDMAAAAAGAM0ZwAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkADNGQAAAAAkwBBqAP1iyCuArKhPADJiCPV8YzLlUicm47DNcVm/dYdQ1zGt2+YkmMTnJVO+y5Yt6xizbt26jsub9PDDD3dcvnjx4rFbv+OSS50YatOWshw7ZdpG6sRQmwYbM47PdZYYhlADAAAAwBigOQMAAACABGjOAAAAACABmjMAAAAASIDmDAAAAAASoDkDAAAAgARozgAAAAAgAYZQA+gXQ14BZEV9ApARQ6jnG5NxiC9DqAcTU3cIdaZ8s+QyGzMJJvF5yZTvsCxdurTj8gcffHAi1++45FInZhy3u0HKcuyUaRupE5NxG+lUn4ZVm5qKIZfeY/oeQm37PNt32b6+ct0Otq+y/bPy/87j1QFgAKhPADKiNgHoVZ3PnJ0v6ciW606T9LWIOEDS18rLADBs54v6BCCf80VtAtCDrs1ZRFwj6d6Wq4+RdEH58wWSjm04LwDoivoEICNqE4Be9fptjbtExBpJKv/fuV2g7RNtr7S9cmZmpseHA4DaatWnam0aanYAphXHTgC6GvhX6UfEORGxIiJWLF++fNAPBwC1VGvTqHMBgCqOnYDp1Wtzttb2bpJU/n9XcykBQF+oTwAyojYB6KrX5uxKSceXPx8v6fPNpAMAfaM+AciI2gSgq65DqG1fLOkISTtJWivpnZKukHSppL0l3SrpuIho/eDrXPfFIEVg8oxsyGtT9YnaBEyskdQnjp0AdNG2NnVtzpq0YsWKyDBIsU5MplzqxAx7kOIkrV+GUPcdM7LmrCm2YwKfl7HLt5tNmzZ1XL5gwQJt2LChY8yiRYumbv1myqVOTMPb1NjXpyzHTpm2kToxw6xNUv/1aVi1qakYcuk9plttGvgXggAAAAAAuqM5AwAAAIAEaM4AAAAAIAGaMwAAAABIgOYMAAAAABKgOQMAAACABGjOAAAAACCBoc45Y5AiMJHGfo4QtQmYWNQnABm1rU0Lh5nFoYceqiYGKdYxiQPtumlq3dUxSet3modQ1zHM7WqUMj0v47Yd1Ynpdzj0sGp/UzHT/FxnyncSNHXslOl5yZRvU4Pr65jG9TttudSJ6ba9cFojAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkADNGQAAAAAkQHMGAAAAAAkw5wxAv5gjBCAr6hOAjCZrzhkzE3q/j6ZM0vqd5jlnzBLabBKfl3HLt5tly5Z1XL5u3bo0v1Om9Zsplzox1KYtZTl2yrSN1IkZ9nbUb33KtO7qxJBL7zHMOQMAAACAMUBzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACDKEG0C+GvALIivoEICOGUM83JlMudWIYQt1fLhJDqPuJmQST+LyMW75NyPI7ZVq/mXKpE5Ntmxq1LMdOmbaROjEZtyPW7+TnUieGIdQAAAAAMAZozgAAAAAgAZozAAAAAEiA5gwAAAAAEqA5AwAAAIAEaM4AAAAAIAGaMwAAAABIgCHUAPrFkFcAWVGfAGTU+xBq2+dJepmkuyLiKeV1Z0h6o6SZMuz0iPhit/vKMkixTkymXOrEZBqkWCefjMNtM+U7LrnMxoxKk/VpEp+Xccu3CRs2bOi4fNGiRVO3fjPlUicm2zbVi0k8dsq0jdSJybgddapPw6pNTcWQS+8xTQyhPl/SkXNc/8GIOLj817W4AMAAnC/qE4B8zhe1CUAPujZnEXGNpHuHkAsAzAv1CUBG1CYAvernC0FOtv1D2+fZXtZYRgDQP+oTgIyoTQA66rU5+6ik/SQdLGmNpLPaBdo+0fZK2ytnZmbahQFAU2rVp2ptGmZyAKYWx04AuuqpOYuItRHxSERsknSupMM6xJ4TESsiYsXy5ct7zRMAaqlbn6q1abgZAphGHDsBqKOn5sz2bpWLr5B0fTPpAEB/qE8AMqI2AaijzlfpXyzpCEk72b5d0jslHWH7YEkh6RZJJw0wRwCYE/UJQEbUJgC9Ygg1gH4x5BVAVtQnABn1PoS6SVkGKdaJyZRLnZiMAznHZf3OrpdxyncYg7ebfqzsqBmjj2nCkiVLOi5/6KGHhjKoOtP6zZRLnZhs29SoZTl2yrSN1InJuB11qk/Dqk1NxUzqcz0uQ6gBAAAAAANGcwYAAAAACdCcAQAAAEACNGcAAAAAkADNGQAAAAAkQHMGAAAAAAnQnAEAAABAAgyhBtAvhrwCyIr6BCAjhlDPNyZTLnViJnUgJ0Ooc+cyGzMJJvF5Gbd8M9lmm206Ll+/fn3HgbEMi+09ZlK3qV5lOXbKtI3UiZnU7ajf2iQ1U58m9blmCDUAAAAAQBLNGQAAAACkQHMGAAAAAAnQnAEAAABAAjRnAAAAAM7/fm8AAAgmSURBVJAAzRkAAAAAJMCcMwD9Yo4QgKyoTwAyYs7ZfGMy5VInZhxndQwTc84GGzMJJvF5Gbd8ly5d2nb5gw8+2PH2GdWZR7Rx48aOMQsXLpzI53oa51P1KsuxU6ZtpE7MsGqTNJ31qYna1FRMpu2uTgxzzgAAAABgDNCcAQAAAEACNGcAAAAAkADNGQAAAAAkQHMGAAAAAAnQnAEAAABAAjRnAAAAAJAAQ6gB9IshrwCyoj4ByIgh1PONyZRLnZhMudSJyTgcdFzW77AHtE7LoNdMz8u4bUfjlu+CBZ1PGtm0aZOWLFnSMeahhx7quLyuOoNgM627cXuuJ0GWY6dM20idmEy51InJVpukzvWpidrUVEym57FODEOoAQAAAGAM0JwBAAAAQAI0ZwAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkABDqAH0iyGvALKiPgHIqPch1Lb3knShpF0lbZJ0TkR82PYOkj4jaV9Jt0h6ZUSs63RfWQYp1onJlEudmEy5zMZ0k22A6Lis30y5zMaMQpO1SWIIdfaYYefS7zDYJgfBjtu6y5TvqEzisVOmbaROTKZc6sQMqzZJzdWncVt3WWKaGEK9UdKpEfFkSc+W9CbbB0o6TdLXIuIASV8rLwPAsFCbAGRFfQLQk67NWUSsiYjvlT8/IOlGSXtIOkbSBWXYBZKOHVSSANCK2gQgK+oTgF7N6wtBbO8r6RmSviNpl4hYIxVFSNLOTScHAHVQmwBkRX0CMB+1mzPbSyVdJumUiLh/Hrc70fZK2ytnZmZ6yREA2mqiNg0uOwDTjGMnAPNVqzmzvUhFcbkoIj5XXr3W9m7l8t0k3TXXbSPinIhYERErli9f3kTOACCpudo0nGwBTBOOnQD0omtz5uIrRT4h6caI+EBl0ZWSji9/Pl7S55tPDwDmRm0CkBX1CUCvun6VvqTDJb1G0nW2ry2vO13SeyRdavsESbdKOm4wKQLAnKhNALKiPgHoCUOoAfSLIa8AsqI+Acio9yHUTcoySLFOTKZc6sRkyqVOTMYho+OyfjPlMhszCSbxeSHfwcY8/PDDbZcvXry44+3nYxLX3bj9fRilLMdOmbaROjGZcqkTM6zaJDVXnyZx3Y3LEGoAAAAAwIDRnAEAAABAAjRnAAAAAJAAzRkAAAAAJEBzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQAEOoAfSLIa8AsqI+AciIIdTzjcmUS52YTLnUick4ZHRc1m+mXGZjJsEkPi/kO7qYYdWmurlkWS91YqhNW8py7JRpG6kTkymXOjHj+Hd7WtcdQ6gBAAAAYArQnAEAAABAAjRnAAAAAJAAzRkAAAAAJEBzBgAAAAAJ0JwBAAAAQALMOQPQL+YIAciK+gQgI+aczTcmUy51YjLlUicm26wOabzmnNUxbvNSRi3TNj1u+yD5zr182bJlHe9j3bp1HZfPmsZ1R23aLMuxU6ZtpE5MplzqxAw7lybq07SuO+acAQAAAMAUoDkDAAAAgARozgAAAAAgAZozAAAAAEiA5gwAAAAAEqA5AwAAAIAEaM4AAAAAIAGGUAPoF0NeAWRFfQKQEUOo5xuTKZc6MZlyqROTcYDouKzfJtddtudglNgHc8dkyqVOTN0hr6y7wT7OJMhy7JRpG6kTkymXOjEZh1BnyndccqkTwxBqAAAAABgDNGcAAAAAkADNGQAAAAAkQHMGAAAAAAnQnAEAAABAAjRnAAAAAJAAzRkAAAAAJMAQagD9YsgrgKyoTwAy6n0Ite29JF0oaVdJmySdExEftn2GpDdKmilDT4+IL3a6ryyDFOvEZMqlTkymXOrEDHvIaJ1hgOOyfjPlMhszCk3WJokh1NljMuVSJ4Zceo8Z99pUPvbEHTtl2kbqxGTKpU5MplzqxJBL7zHdalPX5kzSRkmnRsT3bG8naZXtq8plH4yI99e4DwBoGrUJQFbUJwA96dqcRcQaSWvKnx+wfaOkPQadGAB0Qm0CkBX1CUCv5vWFILb3lfQMSd8przrZ9g9tn2d7WcO5AUAt1CYAWVGfAMxH7ebM9lJJl0k6JSLul/RRSftJOljFq0NntbndibZX2l45MzMzVwgA9KyJ2jS0ZAFMFY6dAMxXrebM9iIVxeWiiPicJEXE2oh4JCI2STpX0mFz3TYizomIFRGxYvny5U3lDQCN1abhZQxgWnDsBKAXXZszF18p8glJN0bEByrX71YJe4Wk65tPDwDmRm0CkBX1CUCv6nxb4+GSXiPpOtvXltedLunVtg+WFJJukXTSQDIEgLlRmwBkRX0C0BOGUAPoF0NeAWRFfQKQUe9DqJuUZZBinZhMudSJyZRLnZhsuUgMoe4nZhJM4vNCvqOLIZfeY6hNW8py7JRpG6kTkymXOjGZcqkTQy69x3SrTfP6Kn0AAAAAwGDQnAEAAABAAjRnAAAAAJAAzRkAAAAAJEBzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQAEOoAfSLIa8AsqI+AchosoZQ1zGNA+2GObQzU74Moe49po5pGfSa6XkZt+2IfMmlyRiGUG+JIdS9xWTKpU5MplzqxJBL7zEMoQYAAACAMUBzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQAM0ZAAAAACRAcwYAAAAACTDnDEC/mCMEICvqE4CMcsw5k3S3pF8O+TEBDNY+o06gAdQmYDJRnwBk1LY2DfWdMwAAAADA3PjMGQAAAAAkQHMGAAAAAAnQnAEAAABAAjRnAAAAAJAAzRkAAAAAJPD/ASZGVU+GSolwAAAAAElFTkSuQmCC\n", 505 | "text/plain": [ 506 | "
" 507 | ] 508 | }, 509 | "metadata": { 510 | "needs_background": "light" 511 | }, 512 | "output_type": "display_data" 513 | }, 514 | { 515 | "name": "stdout", 516 | "output_type": "stream", 517 | "text": [ 518 | " Enter nothing to go to the next image\n", 519 | "or\n", 520 | " Enter \"s\" when you are done to recieve the three images. \n", 521 | " **Don't forget to do this before continuing to the next step.**\n", 522 | "s\n" 523 | ] 524 | } 525 | ], 526 | "source": [ 527 | "def test_boltzman(x, seed = 12345, theta_X=0.5*np.log(1/noise_flip_prob-1), theta_pi=2, iterations=100): \n", 528 | " np_random = np.random.RandomState(seed=seed)\n", 529 | " initial_pi = np_random.uniform(0,1, size=x.shape)\n", 530 | " return boltzman_meanfield(x, initial_pi, theta_X=theta_X, \n", 531 | " theta_pi=theta_pi, iterations=iterations)\n", 532 | " \n", 533 | "(orig_image, ref_image, test_im, success_is_row_inky) = show_test_cases(test_boltzman, task_id='3_V')\n", 534 | "\n", 535 | "assert success_is_row_inky" 536 | ] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "execution_count": 29, 541 | "metadata": { 542 | "deletable": false, 543 | "editable": false, 544 | "nbgrader": { 545 | "cell_type": "code", 546 | "checksum": "dfbdbd77c48ab7c23ce00913b90050b8", 547 | "grade": true, 548 | "grade_id": "cell-e7b59624d7ab9ec3", 549 | "locked": true, 550 | "points": 0.8, 551 | "schema_version": 3, 552 | "solution": false, 553 | "task": false 554 | } 555 | }, 556 | "outputs": [], 557 | "source": [ 558 | "# Checking against the pre-computed test database\n", 559 | "test_results = test_case_checker(boltzman_meanfield, task_id=3)\n", 560 | "assert test_results['passed'], test_results['message']" 561 | ] 562 | }, 563 | { 564 | "cell_type": "markdown", 565 | "metadata": {}, 566 | "source": [ 567 | "## 2. Tuning the Boltzman Machine's Hyper-Parameters" 568 | ] 569 | }, 570 | { 571 | "cell_type": "markdown", 572 | "metadata": {}, 573 | "source": [ 574 | "Now, with the `boltzman_meanfield` function that you implemented above, here see the effect of changing hyper parameters `theta_X` and `theta_pi` which were defined in Task 3. \n", 575 | "\n", 576 | "- We set `theta_X` to be `0.5*np.log(1/noise_flip_prob-1)` where `noise_flip_prob` was the probability of flipping each pixel. Try to think why this is a reasonable choice. (This is also related to one of the questions in the follow-up quiz).\n", 577 | "- We try different values for `theta_pi`. \n", 578 | "\n", 579 | "For each value of `theta_pi`, we the apply the denoising and compare the denoised images to the original ones. We adopt several statistical measures to compare original and denoised images and to finally decide which value of `theta_pi` is better. Remember that during the noising process, we chose some pixels and decide to flip them, and during the denoising process we essentially try to detect such pixels. Let `P` be the total number of pixels that we flip during the noise adding process, and `N` be the total number of pixels that we do not flip during the noise adding process. We can define:\n", 580 | "\n", 581 | "- True Positive (`TP`). Defined to be the total number of pixels that are flipped during the noise adding process, and we successfully detect them during the denoising process. \n", 582 | "- True Positive Rate (`TPR`). Other names: sensitivity, recall. Defined to be the ratio `TP / P`.\n", 583 | "- False Positive (`FP`). Defined to be the number of pixels that were detected as being noisy during the denosing process, but were not really noisy. \n", 584 | "- False Positive Rate (`FPR`). Other name: fall-out. Defined to be the ratio `FP/N`.\n", 585 | "- Positive Predictive Value (`PPV`). Other name: precision. Defined to be the ratio `TP / (TP + FP)`.\n", 586 | "- `F1` score. Defined to be the harmonic mean of precision (`PPV`) and recall (`TPR`), or equivalently `2 TP / (2 TP + FP + FN)`. \n", 587 | "\n", 588 | "Since we fix `theta_X` in this section and evaluate different values of `theta_pi`, in the plots, `theta` refers to `theta_pi`." 589 | ] 590 | }, 591 | { 592 | "cell_type": "code", 593 | "execution_count": 15, 594 | "metadata": {}, 595 | "outputs": [], 596 | "source": [ 597 | "def get_tpr(preds, true_labels):\n", 598 | " TP = (preds * (preds == true_labels)).sum()\n", 599 | " P = true_labels.sum()\n", 600 | " if P==0:\n", 601 | " TPR = 1.\n", 602 | " else:\n", 603 | " TPR = TP / P\n", 604 | " \n", 605 | " return TPR\n", 606 | "\n", 607 | "def get_fpr(preds, true_labels):\n", 608 | " FP = (preds * (preds != true_labels)).sum()\n", 609 | " N = (1-true_labels).sum()\n", 610 | " if N==0:\n", 611 | " FPR=1\n", 612 | " else:\n", 613 | " FPR = FP / N\n", 614 | " return FPR\n", 615 | "\n", 616 | "def get_ppv(preds, true_labels):\n", 617 | " TP = (preds * (preds == true_labels)).sum()\n", 618 | " FP = (preds * (preds != true_labels)).sum()\n", 619 | " if (TP + FP) == 0:\n", 620 | " PPV = 1\n", 621 | " else:\n", 622 | " PPV = TP / (TP + FP)\n", 623 | " return PPV\n", 624 | "\n", 625 | "def get_f1(preds, true_labels):\n", 626 | " TP = (preds * (preds == true_labels)).sum()\n", 627 | " FP = (preds * (preds != true_labels)).sum()\n", 628 | " FN = ((1-preds) * (preds != true_labels)).sum()\n", 629 | " if (2 * TP + FP + FN) == 0:\n", 630 | " F1 = 1\n", 631 | " else:\n", 632 | " F1 = (2 * TP) / (2 * TP + FP + FN)\n", 633 | " return F1" 634 | ] 635 | }, 636 | { 637 | "cell_type": "code", 638 | "execution_count": 16, 639 | "metadata": {}, 640 | "outputs": [], 641 | "source": [ 642 | "if perform_computation:\n", 643 | " all_theta = np.arange(0, 10, 0.2).tolist() + np.arange(10, 100, 5).tolist()\n", 644 | "\n", 645 | " tpr_list, fpr_list, ppv_list, f1_list = [], [], [], []\n", 646 | "\n", 647 | " for theta in all_theta:\n", 648 | " meanfield_pi = boltzman_meanfield(X_noised, initial_pi, theta_X=0.5*np.log(1/noise_flip_prob-1), theta_pi=theta, iterations=100)\n", 649 | " X_denoised = 2 * (meanfield_pi > 0.5) - 1\n", 650 | "\n", 651 | " predicted_noise_pixels = (X_denoised != X_noised)\n", 652 | " tpr = get_tpr(predicted_noise_pixels, flip_flags)\n", 653 | " fpr = get_fpr(predicted_noise_pixels, flip_flags)\n", 654 | " ppv = get_ppv(predicted_noise_pixels, flip_flags)\n", 655 | " f1 = get_f1(predicted_noise_pixels, flip_flags)\n", 656 | "\n", 657 | " tpr_list.append(tpr)\n", 658 | " fpr_list.append(fpr)\n", 659 | " ppv_list.append(ppv)\n", 660 | " f1_list.append(f1)" 661 | ] 662 | }, 663 | { 664 | "cell_type": "code", 665 | "execution_count": 17, 666 | "metadata": {}, 667 | "outputs": [ 668 | { 669 | "data": { 670 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4UAAAFgCAYAAAAMxty4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAN1wAADdcBQiibeAAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3zb1fX/8dfxih2PLGeH7MEIEMpeZZQCBcoquy2r/QEt0JYWukspdJdCKR203xYomxYKLaNswiYkjAABsshO7AwntuPE8Tq/P+5HieLYsRVLlmy/n4+HHpY++uijI8vW1fnce881d0dERERERER6pqx0ByAiIiIiIiLpo6RQRERERESkB1NSKCIiIiIi0oMpKRQREREREenBlBSKiIiIiIj0YEoKRUREREREejAlhSIiIiIiIj2YkkIREREREZEeTEmhJMzMvB2Xw9MQ19S4528ws/lmdqOZlSTxOQ6Pjj85up1nZteY2ZRm+42O9jshWc/dRlwL4157nZnNNbNfmVnhDhzrIjM7OQkx5ZpZhZndvJ193jezxzv6XNGxbm/H3+Xt0b5uZpcl4TkHRe//6I4eS0S6tuizoKXPnWfa+fhOazdaaDM+MrMfmVleEp/j/Oj4RdHtFj8vm7erqdbsvdloZh+a2XfMLGcHjvXtZHzfMbPB0feWb7Vyf6w9/VNHnys63lRru728Jpl/k2Y2MTpm32S8Bkm+hP8BRIAD464XAM8BPwUei9v+QadGtMXzwPcJf9v7AtcBOwGnJen4bxFe//zodh7wY2Ah8E7cfiui/T5K0vO2xz3AzVFMhwE/AgYAX07wOBcB7wMPdyQYd683sweB083sG+7eGH+/me0G7Ab8qiPPE+c64Ja42z8H+gJfjdu2KknPFTOI8P5PJfwNiEjPVgkc28K2TBRrM3oBRxA+y/oAVybp+I8R2sEN0e3WPi+bt6ud4bfAA4TvMCcAvwRyCd9lEvFt4A+E17TD3L3czJ4Hzopia+4YoB9wb0eeJ85XgfgT5rcBHxPa0ZilJDdPmEh4/28H1iXxuJIkSgolYe7+eux67AwgMD9+ezwzyway3b2uE8KriIvj5ain7DozG+juHU4I3L0KaPF1NttvU3v2S7IVca/9RTMbDpxvZhe5e1MnxxJzLyEpPRx4ttl9ZwO1dDD5jHH3+cR9qTCzCiCrtb9LEZEUaOhCnznxbcYLZjYCuMTMrnJ37+jBoza3zXa3ve1qki2Me+3PRycpzyXxpDCZ7gX+bmbjovYs3lmEJO3lZDyRu2914t7MaoBVzf92NQqmZ9HwUUm6aBjfDDM72cxmEb747x8NG1jdwv7bDOUzsy+b2Swz22Rmi8zs2zsYzpvRz9HRcaeY2bNmtsHM1prZ3WY2uNlzf8/M5plZrZmVm9kTZjYkuq/5MJfq6OdtcUMuRjcfcmFm/zCzN1p47ZdFw1diw2uyzOy70fNvMrM5ZnbeDr72mYQzwAPjnu9bZjbdzCqj1/aImY2Pu38qsDdwXtzrOT/u/kTfl6mEXtOzWrjvTOBRd6+Ojr1b9LuuMLOaaEjPpTv20tsl28x+bmarzGylmf3RzHrF72BmI83sviimDWb2pJlNiu4bDbwX7fp87PcV3VdoZn8ws9nR4xZEx0/aUGYR6TrMbKiZ3WpmH0ef+XPM7KfWxnBNMzvRzN6MPhPXmtk0Mzss7v5kthlvAoVAaXTsI6Pni7WFf4o7ERwb0ni9mS2Onnu5mT0Ue00WN3y0jc/L5tMyXjCzf7bwu4g9l0W3883s12a2JHr+mWZ23A6+9pmEUUXxz/dLM3vPzNab2VIL3xeGxN2/kDAa58dx7eXh0X078r78G9hEs/bSzPKBE4H7Ysm6mR1iZi+ZWVV0ecfMTt/B194evc3sL9F3h6Vm9hMz2yqHMLPJZvaYmVVHl39Z3Hcn4JFo1wXR72phdN8O/W9I8ikplFQZDfwa+AVwHLCgvQ80s6uAPxN6kE6Irl9nOzYHbHT0s8zMBhKSlN7AOcDlhGGWT8c1YucShp/eQBiu8RVgHqGhbMmR0c+fEoa/HEhIgpq7D9jXzMY2234G8Ji7r49u3wz8EPgrcDzwEHCr7dh4/pGEpDU+ER9BGOpyEvD/gGzgFTPrE93/VcKQ18fjXs9jsGPvS9RD+U/gVDPLjW03s32A8Ww9FOa/QCPwBUIDeDNQvAOvu72+BQyLnu83wMXA1+Ni7E84KzsJuITwXhUCz5hZAeF9/ny0+6Vs+X1B+BvLBn4AfIYwlPdI4F8pfD0ikgHMLKfZxQiJVgXwTcLw0t8AFxA+51o7zjjCEMfngM8SPm8eBfrH7ZbMNmM0UAdUmNmuwBOE9uNzhGF/50TxxHwviulHwKeBbxCGyma3cOztfV42dx9wgsXNiY9+h6cD/4zrxXwAOJ8wVeCzwHTgv9Zsjn87jWTb7ymDomMfT3htY4HnLIx+AjiF8Hr/Hvd63oruS/h9cfd1hN9585OoJxDawnsBLJxcfJQw3PNzhOkxdxKmS6TKr4H10XPdBVxN3LQcCyeXXwHygS8S3pfdgEei9+4ttgxLPpXwuzolup3w/4akiLvrossOX4AiwIHz47bdHm2b0mzfa4DVLRzDgcui6yWED54fN9vnWqCMMAy1tVimAg8ShkX3Ag4lDLeYDhhhzsA6oCTuMftFz392dPsPwIPbeY7Do/0nt/b6o+2jo+0nRLdzCI3rd+P2GQ40AadFt8dHt89rdqw7gOltvA8LCfMQcggJybHAWuA723lMNmE+RTVwbtz2GcDtzfbtyPuyf/S7OD5u2/WExjQ/ul0a7bN7Ev82HwCmtnKfAy822/Yw8Hrc7euANUD/uG39orgvjW5Pjo51eBux5AAHR/uOTNZr1EUXXTLnErVx3sLlqBb2zSEkWbVAXrStebtxGrBmO8+XzDbjhOiz7YHo/vuAufGf7YQTYw4cGN1+FPjtdp7j/Gj/ouh2i5+XbNuuDgQagLPi9jkw2mef6PanotuHNTvWi8C/2njtDnwteu3FhKkMm+Kfr4XHZBPabAc+Gbd9NXBNEt+XM6Pn2DVu2wPA7Ljb+0T7FCfxb3ebdr/Z3+Qdzba/Q+i5jN2+E5gd+1uOtk0gnOg9Prp9QnSs0W3Ess3/hi6dc1FPoaTKMnd/p+3dtnEgoTfmX/FnWglnSgcTerq251SgnvBh8iKh4fu8h0+a/YCnPMxfAMDd34j2OSTa9A5wXDQ0Yr+4M4Id4u4NhKEhZ8ZtPh2oYUuBnk8RGpKHmr32Z4Ep7Yjlm4TXXgP8D3je3bcq4mJmB5jZ02a2htDobiAkthPbOPYOvy/uPo1wRvPMKAYjfLl4yN1ro90qgCXALWZ2ppkNaiOeZHiq2e0P2Pp1HAU8DVTFvd5qwhCrfdo6uJl90czeNrP1hPclNhekrd+1iHRdlYQiZ/GXaRZ8w8w+MLONhM+EuwknMEe2cqz3gD4Wph8cbdtWk05mm/EIoc2MDdnfj/AZHV8g7EFCuxHfXp5voQLnHtFne4d5mIv4HFu3l2cSahfMiG4fRTgh+UoLr73Nz2fgJsJrryIU3Pmju98Xv4OZfcbMXjWzSsLrXhrd1dZneEfel0cIJ2DPimIoIoy2ih9VMz/a5x4zO8k6p5pne9rLh4CmuNe7gPD9arvvxw7+b0gKKCmUVCnfwceVRj9nET4YYpfno+07tfSgOM8RGuG9CD08h7j7nOi+oa3EVc6W4Ti3EoaPngFMA8rN7LokJYf3ERqEWINyJvBfd98Y3S4lnI2sZOvXfjvhzNnQNo5/F+G1H06oJHaKmX0ldqeZjSR8sBthqOTB0f4rCUM+tqej78t9wEkW5kYcFO2/uZHzMMz0aEIjfythuO9LZrZXG8ftiObVz+rY+vdQSniP6ptdjqCN12tmpxDOCr9GSP4PYMtQmbZ+1yLSdTW4+4xml2rC8MPfEr44n0RIumIJWIufCe4+O9p3LGFI/2ozuyeaCgHJazP2IIyg+ay7x9rIbdrLKEFcw5b28qfAHwnTDmYCS8zs6yTHfcBnzKzEwty104H74+4vBYaw7efzNbTdHkEYorgvIZl5FLjC4uYjmtm+hCkNSwnDIQ8kfI5D+9rLHXpf3H1D9LyxIaQnEUb03Be3z1pCe5lLmJ6xysJcvubTU5KpPe3ld9j2/RhL2+9Hwv8bkhqqPiqp0lLlslrCcgmbmVm/ZvtURD9PoOUEbnYbz7s27kxicysIcwSaG0xUkCZKTm4EbjSznQhzIH4GLGPr5Q52xFRC0nOmmd1BGFb5i7j7KwhnIw8mnGVsbmUbxy+Pe+0vmNko4Fozu8PdawhDSnsDJ0W3ic7m9W/5cFvp6PtyLyHZPo6QVK2iWTVSd/8I+Fw09/BQwlIVj5nZCE9P9dQKQuN8XQv3VbewLd7pwDR337wchsUVhxCRHud0wrDGH8Q2RPP2tsvdHyN8DvYhzE37HWGu1Vkkt81obpv2Mjo5OiB6XqKRHlcDV5vZBMLc69+Z2Wx3f6Kt19aGhwjz1k8CFhHmf8cnhRWEdnlH19RdHHvtZvYioVf2N2b2v2hk0SmEdurM6DZRm9oeHX1f7gXOMbNPEN7nt6P2cTN3fw04NprffhShDsI9bElcO1sF4T37Wwv3bVNgsJkd+t+Q5FNSKJ1pKVBsZsPdfVm07ehm+7wGbASGRY1hMk0DvmJmxb6l4uW+hDHz25R5dvclwC/N7AKgtQ+o2DIbbZ7NcvcmM3uA0PtUSxi2Et9wPkc4u9jH3Z9u1yvavu8RXvOXgN8TzjY2ERqrmDPY9nOg+RlA6OD74u7vm9n7hHkCBxMagIZW9q0nTOaPNXJ92ZKUdqZnCb+fWXG9uc219v4XEOaoxPs8ItJTdegzwd0rCcMFD2NLgZZktxnxphFGm3w/bgjpqYT2oqX2cq6ZXUno4YkVqWkukfZyrZk9RWgvFwEfuvu7cbs8SygWtr55wpQoD2vq/ojQ6/ZZwsnAAqA+lhBGWnq/WmovO/q+PElo8y4hfEf64XZi30go5jKZ0Oany7OEOaNvNvudxVN7meGUFEpneoKQWNxqZr8FxhA+9DZz93Vmdg1wU3RW7kXCMOeJwBHufgo77gZCNdEnzexXhLl0vyScIXwQwMz+Qvgwfp0w9OMIwmTp77R0QHevM7MFwBlR0lMLvNvSvpH7gcuAKwjzNTav3ejus83sFuA+M/s1YeJ3PqGC10R3T2gRend/w8yeJgyL+SNbGqrbzOzv0XGvZNthIR8Bx5jZMYShQgvcfU0S3pd7CcONjGYL8JrZHoTiM/cT5h/2I/zOZ7p7RbTP1cDV7t5Zn1s3ECqTPmdmNxPOSg8mVKx92d3vBRYT/qbPi+ad1Ednn58G/mhmPyB8uTqOMM9ERHqmp4Gvmdk0wpywzxMKkrTKzC4mJIBPAMsJbdHphKHpSW8zmvkp8DbwsJn9mTB/7FfAk1EvFWb2EGGUzduEz8HTCN8rX2zlmK19XrbmfsJ0gkpCEbh4TxOSp6ej9nwWoSDaFEIBs0QTpAcJbd9VhKTwaeAbZvY7wjy/gwjtQXMfAceb2ROEeX6zO/q+REnqg4Q1fmHrHlLM7HjgQkJxtMWEAjgXE9r42D5/JxTh2e7fWBJdA7xB6NW+ldA7OJxQlfZ2d5/KlhFFF5vZfcAGd3+PHfjfkBTpzKo2unS/C61XH53Ryv6fIXx4bwBeAnYhrvpo3H5fIDQ2GwlVNKcB32wjlqlEldO2s89ehA/ODYRk6B5gcNz95xPKKldE+7wLfCnu/sOJq5IWbTs62q82um80zarIxe1rhA9xB45pIT4jjK+fRThztgp4gbjqoK28roXA9S1s/yRbV1c9l/Chu5GQ+O7f/LGEOQDPEBri5u9twu9L3GPHRMdbDFiz+wYRqpd9HP0eywiJ48i4fa4JH1nt/ttsq/po87+5a2hWHZcwZOk2wpDZTdHv6i5gt7h9Pg/MIZwF9WhbNiHJXUnoEX6QLVVYT2jva9BFF126zqWlz5C4+4qiz5KK6PI3tlRjjFXd3KrdYMuSQMujz8UFhMSsV9xxk9pmNNvnU9FnfG30WfYnokqi0f1XERKeSsKQ+mmE6Qmx+88nrvpotK2lz8vDadauRtuLCe2wA5NaiK8X8BPCslF1UbvxBHGVrlt5Xdt8/kfbz2Xr6qrfJhRAqyG0iROaP5awru/r0T6bK6vu6PsSd9wjouO93MJ9kwjt25Lo2EsJ01viK2XfDixM4G+3reqjzb/L3E6z73nAzlFcFYTvCPOAvwAj4vb5FqHntyEWH+3439Clcy4WvSEiIiIiIiLSA6n6qIiIiIiISA+mpFBERERERKQHU1IoIiIiIiLSgykpFBERERER6cGUFIqIiIiIiPRgSgpFRERERER6sG6/eL2Zac0NEZEewt0t3TF0FWofRUR6lu21kd0+KQTQWowiIt2fmfLBRKl9FBHpGdpqIzV8VEREREREpAdTUigiIiIiItKDKSkUERERERHpwZQUioiIiIiI9GBKCkVERERERHowJYUiIiIiIiI9mJJCERERERGRHkxJoYiIiIiISA+mpFBERERERKQHy0l3ACLt4e7MLq9m6uxVvLGggk0NjdvdvzAvh/3G9Ofg8aVMGlxMVpZ1UqQiIiIiIh33n3eW8c8ZS7jw4DF8apfBKX0uJYWSsdZvauDluat5Yc5Kps5exYrK2oQe/9QH5QAMKMzjwHEDOHh8KQeNG8DI/r0xU5IoIiIiIplr3sr1vDJvDafsNSLlz6WkUDKGuzOnfD1TZ4ckcMaiCuobHQAz2GtkX46YNIhDJ5TSr3fedo9VXlXLq/PX8Or81by9eB2PvruCR99dAcDwvgVMHl7CmNIixpYWMmZgIWNKCxlQmKdkUUREREQywur1dQAMKNr+995kUFIoGeHx91bws8c+ZNm6jZu39S/M47CJAzl80kAOnTCQ/oXt/4cYXVrI/mMHcMWnJ1KzqYE3Flbw6rzVvDp/DR+sqIqep3yrxxT3ymHSkGLOPWg0x+8+lGwNORURERGRNFmzfhMApYW9Uv5c5u4pf5J0MjPv7q+xq3tk5nK+ft/bOLDniL4cPmkgh08axB7D+6RkLmDlxnrmr1rPwtU1LFhdw8era1iwqoaFa2rYUBfmKo4fVMTlR47nhD2GKTkU6SLMDHfXP2w7qX0UEclsp/35VWYsWsur3z2SYX0LOnSsttpIJYWSVo+/t4LL730bd+d3Z+3FiXsOS1ss7s5r89fwu2fn8saCCgDGDizk8iPH89k9hpGTrWK9IplMSWFi1D6KiGS2I66fyoLVNXx03bHk52Z36FhKCtXoZawn3i/jsnveosmdG86Ywsl7DU93SJu9Nn8NNz07h9c/jpLD0kKuOmYSx04eonmHIhlKSWFi1D6KiGS23a95End4/yfHdPhYbbWR6vqQtHj6g3Iuu+ctGt35zWl7ZlRCCHDguAHcd9GB3H/RARw8fgAfr67hK3e/xal/fpUZCyvSHZ6IiIiIdGObGhqprm3olCIzoKRQ0uC5j8r56t1v0ujOrz63B5/bO/VldnfU/mMHcPeXD+CuL+3PrkNLeHvxOk675TUuvnMG81etT3d4IiIiItINra2pB8LSap1BSaF0qqmzV3LJnW9R3+j84pTdOWOfndIdUrscMqGURy8/hBvP3JPhfQt4clY5R9/4Ijc9M5emJg2/EhEREZHkWR1VHu3fCZVHQUmhdKIX56ziojvfpK6xiZ+dMpmz9huZ7pASkpVlnLLXCJ791mF87zM7k5+TxY3PzOHiu96kurY+3eGJiIiISDexpiasUViq4aPSnbwybzX/744Z1DU0cd1Ju/H5/UelO6Qdlp+bzcWHjePhSw9mTGkhT39Qzsl/fEXDSUVEREQkKWJrFGpOoXQbr85fzZf+MZ1NDU1c89ld+eKBo9MdUlJMGFzMw5cezJE7D2L+qhpO/sMrPPNBebrDEhEREZEubs360FM4QMNHpTuY9vEavnT7DGrrm/jh8btw/sFj0h1SUvUpyOVv5+7D5UeOp3pTA1++YwYX3TGD1+avQaXeRURERGRHxIaPdlZPYU6nPIv0SNMXVnDB7dPZWN/I94/bmS8fOjbdIaVEVpbxraMnsduwPvzw4fd56oNynvqgnF2GlnDBwaM5cc9hHV5wVERERER6js3DRzupp1CL10tKvLloLef+fRo1dY1859id+crh49IdUqfY1NDIY++u4NZXFvD+sioAeuVksedOfdl3dD/2Gd2ffUb1ozg/N82RinQ/Wrw+MWofRUQy14W3T+e5j1byv68fyi5DSzp8vLbaSPUUStJNnb2Sy+55m5q6Rq46ZlKPSQgBeuVkc+onRnDKXsOZsWgtt7+6kJfmrOKNBRW8saACmE9BbjbnHjiKiz45lgFFnXP2R0RERES6js4uNKOeQkmqf7y6kJ88Mosmh6uOmcSlR4xPd0hp19TkzF25nukLQ2L45KwyNjU00Tsvm/MOGs2XDhlDqZJDkQ5TT2Fi1D6KiGSuQ371HEvXbmTezz5DTnbHy8C01UYqKZSkaGhs4rpHP+Afry0iLyeL35y2BydNGZ7usDJSeVUtf546n3veWExdQxNmsMfwPhwyoZRDJwxkv9H9ycrS91qRRCkpTIzaRxGRzLXLj56gV24W71x9dFKOp6RQjV7KVdfWc9k9b/PCnFUMKMzjr+fuzd6j+qc7rIxXVlnLX16czxPvl7Gisnbz9sMmDuQP5+yleYciCVJSmBi1jyIimWlDXQO7Xv0k4wYW8uy3Dk/KMZUUqtFLqSUVG/jSP6Yzp3w9EwYVcev5+7JT/97pDqtLcXfmr6rh5bmruGvaYuatXM/OQ4q59fx9Gda3IN3hiXQZSgoTo/ZRRCQzLanYwKG/fp79xvTnnxcfmJRjqtCMpMxbi9dy0R0zWL2+jk9GvVsl6t1KmJkxflAR4wcVcereI/jqXW/x8rzVnPzHV7ju5Mn0KQi/0+F9C5Rwi4iIiHRzsTUKSzupyAwoKZQd9N+Zy7nyXzOpa2ji3ANHcfUJuyZlEmxPV5Kfy20X7MsPH3qf+2cs4eI739x8X3aW8bdz9+GInQe1+3jlVbXMKa9u9f69R/Wjd54+BkREREQyRazyaP/Cbp4UmlkucCNwTrTpbuAKd29oYd/hwB+BQwEHngcuc/fyTgpX4rg7v392Hjc+M4csg2s+uyvnHzwm3WF1K7nZWfzyc7vziVF9mbagAoBNDU089u4KvnH/Ozx6+SHt6jF8+O1lfPff71Jb39TqPqMH9OaRyw/R/EURERGRDLFmfegp7KyF6yF9PYU/BA4Bdotu/w/4PnBtC/v+iZAMjgKMkEDeBJyV+jAlXm19I9998F0efmc5Rb1yuPnsvRLqtZL2MzPO3HckZ+47cvO24X0/5K8vfswld73Jg185iPzcbD5etZ455eu3efwr81Zz5+uLyDI4cc9h9M7L3mafD1dUMXNpJVf/ZxY3njklpa9HRERERNpndU3oKewJw0cvJPQMrgAws58B19NyUjgG+KW7r4/2vR/4XmcFKsGa9Zu46M43eXPRWob3LeDv5+/DzkNK0h1Wj/LtYyYxc8k6pi2o4MLbp7NuQz0frKhqdf/+hXncfPZeHDy+tMX7122o4zM3vcRDby9jZP/ejB9UxM5DipkwuDhVL0FE2pDISJq4xxQA7wGl7t439VGKiEgqVcR6CjtxHetOTwrNrB8wAngnbvM7wEgz6+Pulc0ecgNwupk9RugpPBt4bDvHvwb4cVKD7uHmlldz4T+ms6RiI1N26stfz92bQcX56Q6rx8nJzuLmc/bihN+/zKvz1wAwYVARB44bQE7W1vM5C/Ky+Pz+o7ZbvbRv7zxuPHMKZ//f69z07FwACvOyeeHbR1DaiR9CIrKVREbSxFwLLAVaPgMkIiJdSqzQTGfOKez0JSnMbCdgMTDQ3VdH2wYCK4Gd3H1ps/0nALcDsXqsrwPHuHvr1TO2frxKbnfAi3NWcendb1G9qYET9hjK9afvSX7utkMRpfPMKa/muY9WcsSkQUwa0vFevZfmruLleat5f1klr8xbw0WfHMv3j9slCZGKdK7usCSFmS0h9Aw+EN0+Hbje3Ue1sv8ngDuBbwL3J9JTqPZRRCQzffHv03hp7mqe+eYnGT8oOSO42moj01EuMjYBqk/cttj1rRI9M8sCngZeAYqiy8vAkymOUYA7X1/EBbdPp3pTA187cjy/P2svJYQZYOLgYi45bFxSEkKAQycM5Huf2YUbz5hCr5ws7nhtIaujqlci0nnaGknTwv45wP8BlwJt/tOa2TVm5rFLksIWEZEkS0ehmU5PCt19LWGYS3xliynAkhaGjvYnFJj5vbtvcPcNwM3AgWamYTIp4u5c+8gH/Ojh98k248Yz9+SbR08iK6tLn4CXNgwqyecLB4yitr6J7//7PW59eQGzy9rVIS8iyVEU/VwXty12vaWzQN8C3nX3qe05uLtf4+4Wu+x4mCIikkprajaRnWWb16ruDOlaWO424AdmNsTMhhDmS/yt+U7R8NJ5wKVmlm9m+YQzoktjQ08l+d5Zso5bX1lA/8I87v5/+3PKXiPSHZJ0kksOG0d+bhZPfVDOtY9+wEl/fJnpCyvSHZZIT5HISJpxhPbwyk6IS0REOom7U1FTR7/eeZ3aIZOupPA64DXgw+jyKvBzADO7xcxuidv3JOATwDJgBbAfcGKnRtvDLFqzAYAz9tmJfUf3T3M00pkGFvfi7i8fwA+P34XzDgy9hhfeNp33lzXvxBeRZEtwJM2hwEBglpmVAf8GSsyszMz265SARUQk6apqG6hv9E5djgLStCSFu9cTznBe2sJ9lzS7/QFwTCeFJkBZVS0AQ0pUgbIn2ntUP/Ye1Q8IFUpvenYu5936Bg985SDGlBamOTqRbi82kuaV6HaLI2mA+4En4m4fFD12CrAmpRGKiEjKrInqOgzo5KQwXT2FksHKKqOksI+WnejpvnHUBM4/aDRraur44t+n8fGq9W0/SEQ6ol0jadx9o7uXxS5ARdjsZdGJVxER6YJiy1F0ZpEZUFIoLVhZHZLCQSVKCns6M+PqE3bls3sOY+najZ5HFNUAACAASURBVBzzuxf5xeMf0tikwoUiqeDu9e5+qbv3iy6XxRaud/dLmo+miXvcVC1cLyLS9cUqj3bmGoWgpFBasLmnUEmhAFlZxg1n7MlVx0wiLzuLv7z4MfdPX5LusERERES6nTU1YfhoZ88pVFIo2yiv2oRZKDoiApCbncWlR4zn3osOwAx++9Rsqmo1Qk1EREQkWWaXVfPQW8sA6N/Jw0fTUmhGMldTk1NeVUtpUS9ys3XOQLa2x4i+nPaJEfzrzaV86fbpjBoQCs/s1K83Xzp0DM9+WM7Lc1ezo4NLswxGDShk4uBiJg0uZkS/Aq2PKSIiIt3ayqpafvG/j3j4nWW4w7A++RwyvnOXZFdSKFup2FBHQ5MzWJVHpRVXHTOJJ94vY/rCtUxfuHbz9r+8OJ8NdY1Jfa6C3GwmDi4KSeKQ4s0/BxX3wkzJooiIiHR9P3j4fZ7+oJwBhXlcesR4ztl/JPm52Z0ag5JC2YrmE0pbBpXk88QVn2TRmhoA3OHeNxbz6LsrGDuwkG8fM4mSgtwdOnZ9o7Ng1Xpml69nTnk1c8qqmbm0kplLt16irU9BLpMGFzNxSBGTBhczIepZ7NfJk7JFREREOsLdmbGwgrzsLKZedTjF+Tv2HaqjlBTKVsqjNQoHKymU7Rjet4DhfQs23z54fCnfOrqGYX3z6ZXTsTNbh00cuPm6u7OisjYkiOXVzC5bv/n6GwsreGNhxVaPHVjcKySLg4uZNCT0ME4YXExRL33UiYiISOYpq6pl7YZ6Jg8vSVtCCEoKpZktC9crKZTEpGJhezNjWN8ChvUt4PBJgzZvb2xyllRsYHbUmzg7ShQ/XlXDy/NW8/K81VsdZ0S/AnYbVsLpe+/EETsPIlvzFEVERCQDzFpWBcBuQ/ukNQ4lhbKV8qpQBlc9hZLJsrOM0aWFjC4t5JjdhmzeXtfQxILVNdski4srNrB07UaenFXOTv0L+OIBozhjn53o21vDTUVERCR9Zi0PSeGuw0rSGoeSQtlKeTSncHAfJYXS9eTlZDFpSChGw55btm+oa+DFOau547WFvDp/DT9//CN++9QcTp4ynPMOGp32D2IRERHpmWYtD3UTdlNSKJlEw0elO+qdl8Oxk4dw7OQhzC2v5o7XFvHgW0u5f8YS7p+xhH1H9+O8g0ZzzG5DtBSLiIiIdJpZy6swg12GKimUDFKupFC6uQmDi7nu5MlcdewkHnxzKXe8tmjz8hqDS3rx+f1HcdZ+OzGoWP8DIiIikjqVG+pZtm4jY0sLKUxzUTwlhbKV8qpaeuVkUVKgPw3p3kryc7ng4DGcd+BoXpq3mjteXchzs1dyw9NzuPm5uRy3+1DOO2g0e+3UV2siioiISNLNWhGGju6SAdNY9M1fNqutb2TthnpGDeitL8HSY2RlGYdNHMhhEweyaE0Nd72+iPunL+E/7yznP+8sZ/fhfTjvoNGcsMfQTl9IVkRERLqvD6IiM+meTwhKCiXOSlUelR5u1IBCfnD8rlzx6Yn8553l/OPVhby3rJIr/zWTax+ZxU79e9O/MI9+vfPifubSrzCP/r3zws/CPPr2zu3weo0iIiLSvc3anBSmdzkKUFIoccqrtXC9CITCNGfvN5Kz9t2JNxZUcMdri3hyVtnmD+/2KOqVQ35uNu3pdM/LzmJon3yG9i1gWJ98hvbJZ0ifAob1zWdonwIGFOaRpbUVRUREupVMqTwKSgolTlllrMhMrzRHIpIZzIz9xw5g/7EDaGhsonJjPWs31LN2Qx0VNXWsramjYkP0syZue/SzamN9u56nvqmJZes2wqK1Ld6fl5PFCXsM5een7K4hrCIiIt1AbX0j81fVMLikF6VF6f/uraRQNotVHlVPoci2crKzGFDUiwEp+OCua2hiZXUtKyprWb5uIysqa1mxbiPLK2tZUbmRRas38O+3lrF83Ub+eu4+lOTnJj0GERER6Tyzy6ppbHJ2TfNSFDFKCmWzzT2FWrhepFPl5WQxol9vRvTr3eL9q9dv4vzb3uD1jys46y+v848L92NgcfrPKoqIiMiOyaT5hABapVk2K69WoRmRTFRa1It7/98BHDRuAB+sqOK0W15l8ZoN6Q5LREREdlAmzScEJYUSp7xSC9eLZKri/Fxuu2Bfjtt9CIvWbODUP7+6uUERERGRrkU9hZKxyqI5hYNUaEYkI/XKyebmsz/B5/cfyer1mzjrL6/z+sdr0h2WiIiIJKCxyfmorIriXjmM6FeQ7nAAJYUScXfKqmrpX5in9dVEMlh2lvHTkyfz9U9NoHpTA+fe+gZPvL8Cd2/3MWrrGxPaX0RERJJn+bqN1NY3MXFIccYsOaVCMwJA5cZ66hqaGFSqXkKRTGdmXPHpiQwoyuPH/53FJXe9RUl+DmMGFjG2tJAxcZeGJmdueTVzV67f/HPp2o2M7N+bq46ZxAl7DMXas5iiiIiIJEVltGTVgMK8NEeyhZJCAbYMHVXlUZGu49wDRzOouBe/f3YeH69ez8wl65i5ZN12H2MGg0t6sbhiA5ff+zZ/e+ljvnfcLhwwdkAnRS0iItKzxdYxLinInCWmlBQKEL9wvZJCka7k2MlDOXbyUJqanPLqWhasquHj1TUsiC7ZWcaEQUVMGFzEhEHFjBtYREFeNq/OX80vHv+ImUsrOeuvr3PULoP4zrE7M2FwcbpfkoiISLdWVRslhRm07rCSQgG0cL1IV5eVZQztU8DQPgUcNL60zf0PGlfKfy49mEfeXc5vnpzNMx+u5JkPV3LohFLO3m8kn951MLnZmnYuIiKSbLHho33UUyiZprxKaxSK9DRZWcZJU4Zz7OQh3PnaIm59eQEvzV3NS3NXU1rUi9P3GcFRuwwmN3vLnMPsLGPi4GIljCIiIjuoamMDACUFmZOKZU4kklZb5hSq0IxIT9MrJ5svHzqWCw4ewwtzVnLPtMU899FK/jx1Pn+eOn+b/Yf3LeCiT47ljH12oiBP1YpFREQSoeGjkrFiC9erp1Ck58rOMo7ceTBH7jyYFZUb+ef0pXxUVrXVPmvW1/HGwgp+/N9Z/P7ZuVxw8GjOP3gMRb3UnIiIiLSHCs1IxtrcU6ikUESAoX0K+PpRE1q8b9bySm554WMee3c51z81hxmL1nL7Bft1coQiIiJdUybOKdSkEAHCnMLcbKNf78xZL0VEMtNuw/pw89l78fyVhzOyf2+mzl7Fm4vWpjssERGRLqGqNvPmFCopFOobm1hTs4lBxflkZWkRaxFpn1EDCrnsyPEA3PTs3DRHIyIi0jVsHj6aQXMKlRQKK6s34a6F60UkcafsNZyd+hfw4pxVvL1YvYUiIiJt2VxoRsNHJZNo4XoR2VG52VlcdoR6C0VERNqramMD2VlGYQZV8FZSKKyMiswMKtFyFCKSuFM/MYIR/QqYOnsVM5esS3c4aWVmF5jZM2b2bnT7MDM7I91xiYhI5qjcWE9Jfg5mmTNtS0mhqPKoiHRIbnYWl0a9hb/vwb2FZvYD4ArgPmBktHkFcFXaghIRkYxS19DExvrGjBo6CgkmhWaWa2Y/MbN5ZlYZbTvWzC5NTXjSGbYsXK+kUER2zOc+MYLhfQt49qOVvLe0Mt3hpMuXgePc/W+AR9vmAePSF5KIiGSS6gxcuB4S7yn8NXAIcAlbGrwPgYuTGZR0Li1cLyIdlZeTxSWHh9zn3umL0xxN2hQSegZhSxuZC2xKTzgiIpJpMnE5Ckg8KTwdOM3dnwGaANx9EVuGyUgXVF4Vvq8oKRSRjjhy50EAvL+sx/YUvg40HzlzIfBKGmIREZEMlIkL1wMkmqIasGGrDWZFQHXSIpJOV645hSKSBMP65NOvdy4fraimvrGJ3OweN239CuAZMzsPKDKzV4HBwFHpDUtERDJFJq5RCIn3FD4PXNds23eAp5MTjnQ2d6esqpaS/BwKMqgsroh0PWbG5OF9qGtsYm75+nSH0+ncfT6wC/AL4PvATcAe7r4grYGJiEjGyMQ1CiHxpPAK4FAzWwWUmNkywhnQbyc9MukU1Zsa2FDXqKGjIpIUuw3rA8D7y3veEFIz+66717r7A+5+vbvf7+41ZqY2UkREgLBGIUBJfheeU+juq9z9QOAzwFnAqcDB7r46FcFJ6q1U5VERSaLJw0sAmNUz5xV+v5Xt3+nUKEREJGPFegq79JxCM/uTu3/V3WcAM+K2/8HdL0t6dJJyZZUqMiMiyTN5c09hVZoj6Txmtkd0NcvMdifMv48ZB2zs/KhERCQTxQrNdPXho19oZfvZiRwkWu/wD2ZWEV1uNrNWE1QzO9HM3jGzGjNbbmaXJBS1tEoL14tIMo3s35viXjl8sLyKxiZv+wHdwzvA20ABMDO6Hdv2B+Dq9IUmIiKZJFMLzbSrp9DMToyuZpvZZ9n2LGii44R+SFjvcLfo9v8Iw26ubeG5jwX+REhIXwJKCNXcJAlilUcHl/RKcyQi0h1kZRm7Dith2oIKFqxez/hBxekOKeXcPQvAzKa5+/4dOZaZ5QI3AudEm+4GrnD3hhb2vRk4GehDqAL+L+Db7l7XkRhERCR1uvo6hTdFl3zg93G3bySsXfi1BJ/3QuCn7r7C3VcAPwO+1Mq+1wHXuvtUd29097Xu/lGCzyet2JIUqqdQRJJj8vBoCOmynjOEFKCjCWEk/qTpbsChtD5X8U/Azu5eAkwB9kSF30REMlqm9hS2Kyl09zHuPgZ4MHY9uoxz94Pc/dH2PqGZ9QNGEIbWxLwDjDSzPs32LQT2JlQ6/cjMyszsfjMbsp3jX2NmHru0N66eqqxShWZEJLlixWZ64iL2Zna2mf3FzB40s3/HLgkcot0nTd39Q3evidvUBEzY8ehFRCTVMnXx+kSrj56RhOcsin6ui9sWu958nFE/wlDVLwLHAOOBeuDO7cR4jbtb7JKEeLs1LVwvIsk2uYcuS2Fm1wI3ABsIVboXAQcDS9r5+HafNI17zHfNrBpYSegpvHk7x9dJUxGRNOsW6xSaWS8z+56ZPWlmb5rZW7FLAoeJrWgc38DFrle3su/v3X2Ru68Hfgx8KupFlA4qq6olO8sYUKQ5hSKSHGMHFpGfm8WsZVU09ZxiMwDnAse4+xVAXfTzRGBUOx+fyElTANz9l+5eDOwK3AKUtXZwnTQVEUm/qo0N5GVn0Ssn0XqfqZVoNDcQCr48DkwC/gH0Bv7T3gO4+1pgKWH+Q8wUYIm7Vzbbdx2wGGjpW4UatA5qbHJWVW9iYFEvsrP06xSR5MjOMnYdWkL1pgaWrN2Q7nA6U393fze6Xm9mOe4+DTi8nY9P5KTpVtz9Q0Ll09vb+VwiIpIGVbX1lBTkYJZZ370TTQpPBo5395uAhujnKbS/wYu5DfiBmQ2J5gd+H/hbK/v+FfiamQ03swJCae9no15D6YDV6zfR5DBY8wlFJMl6aLGZRWY2Pro+BzjLzI4DarbzmM0SOWnailw0p1BEJGPV1jdS19CUcUNHIfGksNDdF0bXa80sPzo7uXeCx7kOeA34MLq8CvwcwMxuMbNb4vb9JfAs4QzoEkLP5BcTfD5pweYiM1qOQkSSrIfOK/wFYZkmCEss3UIYSbPNckvb0a6TpmZWZGYXmFlfC3YnVC59skOvQEREUiZTK49CO9cpjDPXzPZ095nAe8AVZrYOWJ3IQdy9Hrg0ujS/75JmtxuBb0UXSSItRyEiqbJbD6xA6u73xF1/Mioc0yvBkS3XAQMIJ0whrFO4+aRpdOxLCNMqzgGuB3oRCs08SJh3LyIiGShTi8xA4knh99kyEf57wL2Eye+XtPoIyVhKCkUkVSYMKiYvO4tZy6tw94ybO9EZ3L3ezHY3s2vd/YT2PoZ2nDSNlqL4dNKCFRGRlKvcGC1cn59ZC9dD4ktSPO3ur0TXZ7j7BHcf4u4PpyY8SaUyLUchIimSl5PFpCHFVNTUMXNp9+4tNLP8aLmHR8zs+mho58hofcJXaOeSFCIi0r1lck9hh2uhmtmpZvZu23tKpimr3ARo4XoRSY1jJw8B4At/m8YLc1alOZqU+h3wOcKQz88QRtG8QqiePdbdv5LG2EREJENUZejC9dDOpNDMSs3sdjN7z8weMLOhZra3mb0J/Bm4p61jSOZZWR0bPqpCMyKSfF89fBzfOGoC6zc1cOHt07ln2uJ0h5QqxwPHuvu3o+vHAxe7+zfcfUV6QxMRkUyRyYVm2ttTeDMwgpAADiBMZn8YuBMY5e6/TE14kkqx6qOaUygiqWBmfOOoidx45p5kGXz/ofe46/VF6Q4rFUrcfRlAVKF7g7s/nt6QREQk01TVRnMKCzJvTmF7Izoc2NXd15rZg8AK4AB3fyNlkUnKlVXVUpiXTXEGnq0Qke7jlL1GMLg4n3P+No0/PT+Ps/cbSXZWtyo8Y2ZWDMReVH2z27h7j1qwUUREttUdegp7R4vq4u7lwHolhF3bhroGqmsbtHC9iHSKg8aXcuiEUpZX1vLcRyvTHU6yFQHrgLXRpU/c7dhPERHp4WKFZjJxTmF7ewqzo4VxY2c9vdlt3F3FZrqQ8qpQZGZwsZJCEekcn99/FC/NXc1dry/i07sOTnc4yTQm3QGIiEjmq9yYudVH25sU9gbeIS4JBGbGXXcgO1lBSerF5hOq8qiIdJajdhnEkJJ8Xpy7isVrNjByQO90h5QU7t4tJ0qKiEhyVXX1dQrdPcvds6OfLV2UEHYxWrheRDpbTnYWZ+83Ene4+w3lUSIi0rN063UKpWvasnC9lqMQkc5z1n47kZ1l/HP6EmrrG9MdjoiISKfpDoVmpJtRT6GIpMPgknyO3nUwazfU87/3tYSfiIj0HFW1DRTkZpOXk3kpWOZFJJ1ic1KoOYUi0sm+cMAoAP45fWmaIxEREekc7k7lxvqMXKMQlBT2WJsLzainUEQ62YFjB1CYl817yypx93SHkzRmlmtmPzGzeWZWGW071swuTXdsIiKSXhvqGmls8owcOgo7mBRaMDTZwUjnKa/ahBkMLNacQhHpXFlZxoTBxazf1MCydRvTHU4y/Ro4BLiEUJUb4EPg4rRFJCIiGSGTi8xAgkmhmRWb2R1ALTAv2naymV2biuAkNZqanJXVtQwo7EVutjqLRaTzTRpcDMCc8uo0R5JUpwOnufszQBNsXq5iZFqjEhGRtIstR5GJC9dD4j2FNxPWI5wM1EXbXgfOTGZQkloVG+qob3SG9FEvoYikx8QhISmcXbY+zZEklQEbttpgVgR0q8xXREQSt3nh+gxcoxASTwqPBb7s7nOJhsa4exkwONmBSepoPqGIpNvOQ7plT+HzwHXNtn0HeDoNsYiISAbZvBxFhvYUJpqqbmr+GDMbAFQkLSJJuVjl0UFKCkUkTSZGw0c/KutWSeEVwH/NbBVQYmbLgMXAZ9MbloiIpNvmOYUZWmgm0aTwEeBPZnY5gJn1IkysfyjZgUnqlFdtAtRTKCLpU1qUR//CPOavXE9DYxM53WB+s7uvAg40s32BUcASYLq7N6U3MhERSbdYT2F3mVP4HaAXsBroC6wHSoCrkxyXpFBZlYaPikh6mRkTBxdR19jEwjUb2n5AF2BmZ5hZnrtPd/cH3H2aEkIREYGwcD3QPdYpdPcadz8DGArsB4xw99PdvSYl0UlKlFdq4XoRSb9uWIH0x0C5mf2fmR2S7mBERCRzbCk00w16Cs3s22Y2zN1XufsMdy9PVWCSOrGewsElqj4qIukzaUgJALO7ybxCd98N+DRQAzxgZh+b2bVmNiHNoYmISJpleqGZRIePHg58bGZPm9kXzKx3CmKSFCvX8FERyQCThhQB3ScpBIhOmH4DGA5cDhwEfJTeqEREJN0yvdBMosNHjyMswvs48E3CMJk7zOxTqQhOUqO8qpZeOVkZO9FVRHqGCd1v+CgAZpYHnAJcDBwCvJDeiEREJN1WVYdCj317Z+b374TLvbn7Sne/0d0/ARwADASeSnpkkhK19Y2s3VDPkD75mFm6wxGRHqwkP5dhffJZuKaG2vrGdIfTYWZ2iJn9BSgHrgVeBya4+5HpjUxERNJpY10j7y+rol/vXIb3LUh3OC3aoRrgZjbUzK4E7iEMjflHUqOSlFkZLUcxuFhDR0Uk/SYOKabJYd7K9ekOJRn+TVjP92h339Xdf+7uS9IdlIiIpNebi9ZS19jEgeMGkJWVmZ0yiRaa+YKZPQUsAD4F/AoY4u4XpiI4Sb7yalUeFZHM0c0qkA5z96+5+/R0ByIiIpnj1fmrAThw7IA0R9K6RBfKuBK4EzjX3ctSEI+kWFllrMiMKo+KSPpNGhKSwq5abMbMjnP3x6Obx7U2LN/d/9t5UYmISCZ57eM1ABw4rjTNkbQuoaTQ3aekKhDpHOWbl6NQT6GIpN/EqKdwdtftKfw1ofgawE2t7OOAkkIRkR6ouraed5dWMqi4F+MGFqY7nFa1mRSa2SXufkt0/Wut7efuv09mYJIasZ5CJYUikgnGDyoiy2BOF+0pdPfJcdfHpDMWERHJPNMXVtDY5Bw0bkBGF3lsT0/hicAt0fVTWtnHASWFXUB5VA53iOYUikgGyM/N5sYzpzCiX2ZWY0uEmT3m7se3sP2/7n5iOmISEZH0enVebOho5s4nhHYkhdHahLHrR6Q2HEm18kotXC8imeWkKcPTHUKyHNrK9kM6NQoREckYsfmEB2XwfEJIcE6hmb3n7ru3sP0dzTfsGsqiOYWDVGhGRCQp4qZW5LYwzWIcoMJsIiI90NqaOj5YUcWIfgXs1L93usPZrkSrj45uZfvIDsYhncDdKa+qpV/vXHrlZKc7HBGR7iI2tSKXradZNBEWsj+/swMSEZH0m7ZgDe6ZvRRFTLuSQjO7IbqaG3c9ZizwcVKjkpSo3FjPpoYmxpRmbuUjEZGuJja1wsyud/cr0x2PiIhkhlfnR0NHx3eTpBDoF/3MirsO4SzoB8A3kxmUpEZs6KiKzIiIJJ8SQhERifdalBQeODaz5xNCO5NCd78AwMzecvebUxuSpEqZisyIiCSVmS1295HR9bWEatzbcPf+nRqYiIik1crqWuauXM/Y0sIu0SHTnnUKi909toDUP8yspKX93L0qqZFJ0q2sCstRDFJSKCKSLOfEXT85bVGIiEhG2dxLmOFLUcS0p6dwGRBLBNex7VlQi7apckmG2zx8VEmhiEhSuPvLcddfSGcsIiKSGWrrG7njtUVA90oKd4u7PiZVgUjqbZlTqOUoRESSzcx+ADzh7m+a2eHAQ0AD8Dl3fzGtwYmISKdobHK+ft/bvLloLbsP78NRuwxOd0jt0p7F65fEXV8Uf5+ZDQQa3H1tCmKTJIstXD+oWD2FIiIpcAkQm3d/LfBjoBq4HtgvXUGJiEjncHd++PD7PDmrnNEDenPbBfuSn9s1BlNmJbKzmf3RzA6Irp8OLAfKzexzqQhOkqu8WtVHRURSqI+7V5lZMbAH8Ed3vw2YmOa4RESkE/zumbnc+8ZiBhb34s4v7U9pUdcZnZfo4vWnArGS298FzgAqgZuAB5MYl6RAWeUmcrON/r3z0h2KiEh3VGZmBwO7Aq+4e6OZFRGWbxIRkW5mY10jc8qr+aisijcXreWfM5ZS3CuH2y/Yl5369053eAlJNCksdPeNZlYKjHb3hwDMbGTyQ5Nkqm9sYk3NJob1KSAry9IdjohId/QT4HmgDvhstO0oYGbaIhIRkaRydx5/r4ybn5vL7PJqPK4EZ15OFn89dx92G9YnfQHuoESTwgVmdg4wgdDwYWZ9CQ1gu5lZLnAjW0p53w1c4e4N23lMAfAeUOrufROMu8dbWb0Jdxhc0nW6sUVEuhJ3v9vMHoqub4g2vwq8nr6oREQkWT5cUcU1/53FtAUVAAzvW8DOQ4rZeWgxOw8pYZ/R/RjapyDNUe6YRJPCK4HbCUngKdG2E4DpCR7nh8AhbKls+j/g+4SJ+a25FlgKlCb4XAKUV2k+oYhIJ8gGjjezEYQ26/FE1vFt70lTM+sF/IHQE1lKWD7q1+5+a8dfgoiIxKuoqeOGp2dzz7TFNDlMGlzMjz+7KweN7z5pSUJJobs/DQxvtvn+6JKICwmN3AoAM/sZoTpbi0mhmX0COA745g48l7Cl8uhgrVEoIpISZjYFeBJYCywARgM3mdmx7v52Ow/T3pOmOcAKQlL4MbA/8D8zW+ruT3XkdYiIyBZL127gpD+8wpqaOvoU5PKtoydyzn4jyclOqF5nxku0pxAzGwWcDcTOgt7bfKmKNh7fL3rsO3Gb3wFGmlkfd69stn8O8H/Ape08/jWEMuASRwvXi4ik3O+Bn7v7TbENZnZ5tP3Qdh6jXSdN3b0GuDpu0+tm9jwhoVRSKCKSBO7O9/79Hmtq6jhxz2H85MTd6FfYPQs2JrokxaeAD4AjCENkDgdmmdlRCRymKPq5Lm5b7HpxC/t/C3jX3ae25+Dufo27W+ySQFzdWiwpVE+hiEjKTCYM6Yz3p2h7m9o6adrGY/MJayG+u519rjEzj13aE5OISE92//QlvDR3NRMGFfGb0/fotgkhJJgUAr8CznX3Y9z9K+5+LHAu8OsEjrE++hnfwMWuV8fvaGbjCD2EVyIdsrJqE6CkUEQkhcqAA5pt2y/a3h6JnjQFwMwM+BswF/h3a/vppKmISPstX7eRnz32IVkGvzl9T3rldI1F6HdUosNHxwEPNdv2MNDuie3uvtbMlgJTgPnR5inAkuZDRwnDbQYSeiMB8oASMysDTnT3NxKMv8cqq1ShGRGRFPs5YV7fncBCwpzCzwOXt/Px8SdNV8ddh2YnTWOihPDPwCTgKHfXmogiIh0UGzZavamBiw8by5Sduv/CB4n2FC5gy9pLMcdH2xNxG/ADMxtiZkMIk+j/1sJ+9wNjCEnjFODLhIZxCtDeSfvCluqjWpJCRCQ13P0uQlG0XMI0i1zCCcw72/n4tYS5+lPiNrd20jSWEP6R0Bt5WoZcZwAAIABJREFUdEv7iIhI4h54cykvzFnF2IGFXHHUxHSH0ykS7Sn8LvBwNJl9IeEs6OHAqQke5zpgAPBhdPtuwhlWzOwWAHe/xN03AhtjDzKzinCXt3cojhDOdpRV1VKcn0PvvIRrC4mISBvM7MvA7sCb7n5RBw4VO2n6SnS7tZOmEOYvHgwcGSWUIiLSQWWVtVz76AeYwW9O24P83O49bDQm0SUpnjKzycBZhMnwrwBfc/f523/kNsepJ8wV3KaiqLtfsp3HTQW6f/9tkq3f1MCGukYmDCpqe2cREUmImf2cUDX0JeAsMxvh7j/fwcO166RpVAn8q8AmYFE0xQLgru21oyIi0jp35wcPvUd1bQNfOmQMe4/qn+6QOk27ksKoqtkPic6CAr+IEjvpArRwvYhISp0DfMrdZ5nZnsB9RIlcotp70jRaCkrFYkREOqC+sYm55et5f3kl7y+rZObSSmYuWcfoAb258uhJ6Q6vU7W3p/B3wCeBxwhrFJagiqBdRlmlKo+KiKRQf3efBeDuM81sULoDEhGRljU2Of+duYw7XlvErOVV1DVsXZ+rtKgXN5w5hYK8njFsNKa9SeEJwEHuvtjM/kxYGFdJYRdRpiIzIiKdST14IiIZpqnJeXJWGTc8PYe5K0Ox5769c9lvdH92G17C7sP7MHlYH0b2701WVs/7GG9vUlji7osB3P9/e/ceX3ddH3789c69aZLe07TSUgqlLSAg46qoWPihIvN+d9PN6WQTN92c7qdsIjh105/ztomT/WA/p+IdJ8hP5VYZiIJQAWlLaQsU2iS90SRtkyY5n/2RkzYUSpP25Hxzznk9H4/zOPneTt959JN88v5+vp/3J63LL7CrErH38VFHCiVpPDTlC6ENm7rfNimlypmYIkkTSEqJW1dv5rM/W83vNnYBcMZR0/nr8xdz2oJpjJiTXdFGXWgmIprZd/ezar9tUkpdBY5NBbJvOQqTQkkaBy/JOgBJ0tPd/cg2PnXDKn7z6FCB5pPmTeWD5x/L2cfMNBncz2iTwibgyRHbMWI7gARU1oO3JcSF6yVp/KSUlmcdgyTpqX674Une/G93MpBLLGlr5q/PX8x5S1tNBg9gtEnhUeMahcaVI4WSJEmqFN29/bzvW/cykEt86GWLuehFR1fkPMGxGFVSmC99rRLV0dVHdVUws8lCM5IkSSpfKSUuufYBHtu2i5ceP5s/e/HRjg6OQlXWAWh8DeYSm3v6mNVUT7V3SCRJklTGvn/PE/xoxUbmTmngH193ognhKJkUlrktPX0M5hKznU8oSZKkMrZucw9//6MHqAr4/Jufx9TGuqxDKhkmhWVuuMjM7GYfHZWk8RYRfxwRN0bEffntF0fEG7OOS5LKXd/AIO/71r3s2jPIX557LKcf5UpAY2FSWOb2rlHoSKEkjauI+CjwAeAaYH5+9ybgbzILSpIqxKdvWMXvNnZx+lHTuXjZMVmHU3LGlBRGRG1EfDwiHo6IHfl9L4uI945PeDpcVh6VpKJ5F3BBSulKhpZqAngYODq7kCSp/N20soOrbn+EqY21fOHNJ1tH4xCMdaTwn4CzgYvY1+GtBN5TyKBUOO3DI4UmhZI03iYzNDII+/rIWqAvm3Akqfx1dPXywe/+FoB/et2JzJkyKeOIStNYk8I3AK9PKd0I5GDvchXzn/UqZaaja+hvEUcKJWnc3Qns/+TMO4HbM4hFksreYC7x/mtWsH1XP+8460jOP74t65BK1mgXrx8WwK6n7IhoAroLFpEKat+cQgvNSNI4ez9wU0S8A2iKiDuA2cB52YYlSeVl3eYerr9vEz++byMPdfSwpK2Z/33B0qzDKmljTQpvAS4HPjRi34eBnxcsIhXU3uqjjhRK0rhKKa2LiKXAhcACYANwXUppZ6aBSVIZeGTLTq6/fxPX37eJBzd17d1/3JwWvvzW59FQW51hdKVvrEnhB4D/iojNQEtEPAE8Bvx+wSNTQbR39dJYV01T/Vj/qyVJY5VS6gW+BxARs4A6wKRQksZgz0COrt5+tvbs4ZbVnVx330YeeGJfInhMaxMXnjiHVzx3DotmN2cYafkYU6aQUtoMnBURpwFHMnQX9K6UUm48gtPh2bVngO7eARbOnEyEVZgkaTxFxL8AX08p3RkRbwC+CaSIeEtK6fsZhydJ4yqlxO7+QXb2DbJ7zyA79wywa88AO/sGn/q+Z5CdfUN/o+7Y3U/X7v6h9978++4BdvcPPu3zF86czIUnzuHCk+ZyrIlgwR3S8FFK6S7grgLHogKzyIwkFdVrgQ/mv/5b4I3ADuALgEmhpLLS1dvPf63YyHd/8zgPd3Szq3+QlA5+3bOprgpaGmpobamnpaGWKZNqOWneFC48cS5L2pod5BhHY0oKI+IW9pXZfoqU0rKCRKSCGZ5P6ML1klQUk1NKuyNiJrAgpfRDgIiwQrekspBS4tfrt/Htuzfwk/s30ds/9LBgc0MNc6dMorGuOv+qYXL9fu911TTW1+w93tJQw5RJtbTkX1Mm1TK5rtrELyNjHSm8dr/tOcAfAlcXJBoV1HDl0dYWK49KUhGsj4i3AosYKsxGREwF9mQalSQdps7uXr7/myf47t0bWLdlaJr05Lpq3nzaPN542jyeN2+qyVyJG+ucwi/svy8irgE+VbCIVDAdLlwvScX0QYZuku4BXpPfdyFOt5BUggYGc9y6ejPX3LWBW1Z3Mpgbeljw946cxptOnccrTpzDZAsZlo1C/E/eD7ygAJ+jAms3KZSkokkp/Rx4zn67v51/SVJJWL9lJ9+5ewPf/83jdHYP1aeYMbmO157yHN502jyOabXISzka65zCE/fb1cjQ46NrCxaRCmZ4pHC2cwolaVxERMsoT+0f10Ak6TDs3jPIDQ9s4tt3beBX67cBUBVwzuJZvPm0eSxbMpu6mqqMo9R4GutI4QqGCs0MPzS8E7gHeEchg1JhuHC9JI27JzlAAba8yB93VWVJE0pv/yB3rN3Czx8cWgewu3cAgCOmTeJNp87j9acewZwpkzKOUsUy1qSwBdiZ0uEWnFUxdHT1EQGtzRaakaRxclTWAUjSaLXv6OXmVZ3ctLKD29du2Vs9tK66ileeNJc3nTaPsxbOoKrKojGVZtRJYURUAZsZSgx9DGaCy+USnd29zJhcT221w/2SNB5SSo9mHYMkHUgul7jviR3cvLKDm1Z18ruNXXuPNdfXcO5zZ3Pu0laWLWllamNdhpEqa6NOClNKuYh4GJgOdIxfSCqEbbv20D+YaJviKKEkFUtEnAWcA8xk31QLUkp/lVVMkirLzr4BbluzhZtXdXDzqs1s6enbe2zBjEbOXTqbc5e0cuqC6c4T1F5jfXz0CuD7EfGPwAYgN3wgpXRfIQPT4dk7n7DZ+YSSVAwR8V7gM8BPgZcDNwDnAz/KMi5J5W/Dtl3cvKqTG1d28Kt129gzOPQnenVVcObC6Zy7ZDbLlraycOZk1xPUMxpVUhgR16eUXgF8Kb9r/w7OSfQTTGe3lUclqcjeD7w8pbQ8IranlF4TERcAb8g6MEnlZTCXuPex7dyUnx/4UEfP3mNTJtVyweI2li2dzYsXzWJKY22GkapUjHak8IUAKSXHmEtE+46hRwVco1CSimZ2Sml5/usUQ7fjbwC+nmFMksrEjt39/OKhzdy8qpNbV3eyfde+Eh+LWptYtrSVc5fM5pT5U6mxnoTGqBCL12sCcuF6SSq69oiYm1LaCKxnaG7hZkZMtZCk0djZN8B//PIR1nT08Pj2XWzt2cNj23YxkBtaAKC2OnjhopksWzKUCM6f0ZhtwCp5o00K6yLifYyYNL+/lNIXCxOSCqEjP6ewtcVCM5JUJF8BzgB+CHwO+Fl+/z9kFpGkkpBS4os3Pcy2nX0sbmvhqtvXs6az5ynnzGyq45zFrZy3tJWzF82iqd6xHRXOaFtTDfDaZzmeAJPCCaQjP6ewzTmFkjSuIuKilNIVKaV/zm9PSil9IyJ+ATSllFZmHKKkInp06056+gaYPrmOmU31XHnbelZu6uKSC5fSeoACgDev6uSfb3zoKftedOws3vOihcyb1khrSz0NtZbv0PgZbVK4K6X0knGNRAU1XH3Ux0cladz9E0PVuYc9AUxPKW3IKB5JGfn0Dau4YvnavdstDTV09Q4AcM9j23n5CW2sau9mxWNP0jeQY0pjLcfObmL95p0AvOvso6irqeKU+dN4yZJWql1EXkXiuHOZ6ujqpb6miimTrDglSeNs/7/a/CtOqkDf/NVjXLF8LS0NNZx+1Awe3bqTNZ09nLZgGrOa6/nJ/e187bb1AEyqraZlUi3bd+7h9oe3AvDCRTP56CuWumSEMjHapNDWWUJ6+wfZvquf+dMb/cUiSeMvHWRbUpn71bqt/N2PHqC2Ovja20/ljIUzANi1Z4DGuhpyucTNz+tkd/8g86c3cvzcFmqqq+gfzLGmo4eHOrp58bGz/LtNmRlVUphSah7vQFQ4m7tdjkKSiqguIv5ixHbDftsWY5PKWPuOXt77zXsYzCU+9foT9yaEAI11Q39qV1UF5x03+2nX1lZXcdzcFo6b21K0eKVn4uOjZWh4OQoXrpekorgTeM2I7V/tt20xNqkMpJR4cFMXtdVV5FLippWd3LF2C/dt2EF33wB/cOZ83njqvKzDlA6JSWEZ2ldkxuUoJGm8pZTOyToGSePv336xjk/dsOpp+xtqq3jlSXP5+wuPzyAqqTBMCstQx/BIoY+PSpIkHbZ7H9vOZ366mvqaKs5cOIPe/kHOXDiD/3XcbJa0NVNTXZV1iNJhMSksQyaFkiRJhdHR1ctfXHMvA7nEZa86gbeeMT/rkKSCMyksQ+1d+UIzzimUJEk6JIO5xFW3r+fzN66hp2+AC57bxltOd86gypNJYRnqyM8pnN1sUihJknQoPvPT1VyxfC3VVcGfvmghHzjvWJeMUNnK5AHoiKiNiC9HxLb860sR8bQENSLqI+JrEbE+IrojYlVEvDOLmEvJcPXRVgvNSJIkjdktqzu5Yvlamhtq+PHFZ/ORC5Yyqa4667CkcZPVrNhLgLOB4/OvFwIfeYbzaoBNwHlAC/BHwP+JiPOLE2bpSSnR0dXLtMZaGmr95SVJpWS0N03z514cEXdHRF9EXFvsWKVy09s/yL2Pbefae5/gr7/zWwA+8/oTXUNQFSGrx0ffCXwgpbQJICL+AfgscNnIk1JKO4G/H7Hrzoi4haGE8mdFirWk7NjdT99AjqNmTs46FEnS2I28aQpwA0M3TS97hnM3Ap9g6MbpEUWJTipDd67bynfvfpyf/q6dnr6BvfvfcdaRvOyEORlGJhVP0ZPCiJjGUOe1YsTuFcD8iJiSUtrxLNc2AKcD3xzfKEtXu5VHJamUjeqmKUBK6Qf5c07GpFAas5WbuvjUDav4xUObAaivqeLcJa0c3drE4tnNvOrkuRlHKBVPFiOFTfn3J0fsG/66GXjGpDCGZvZeCawBfnCgD4+IS4GPHXaUJWrfwvUmhZJUSg7npukoP/9SKrh/lIYN5hJfuPEhvnTLw6QEC2Y08t6XHMPLnzuHpnprMKoyZdHye/LvU4AtI74G6H6mC/IJ4VeAxcB5KaXcgT48pXQpcOmIa9PhhVtaOvPLUcx2OQpJKjWHdNN0tCq9f5QAOrt7ef81K7hj7Vaa6mv44PnH8tYzjqSuxsXnVdmKnhSmlLZHxOPAycDa/O6TgQ3PdBc0nxD+C0OPjZ57uHdKy93w46OOFEpSyRnzTVNJB9c/mOOeR7fz7bs38JP7N9Hbn2NJWzP/+rZTWDir6eAfIFWArMbIrwI+GhG357c/wtCjoc/ky8ALgGUppe3FCK6U7ZtT6HIUklRKxnrTVNKBdXb38uWbH+aex7bzUHsPewaHHjKbXFfNO19wFB962WKrtEsjZJUUXg7MAFbmt78BfBIgIq4ASCldFBFHAn8O9AGPjlgw9D9TShcVNeIS0WmhGUkqZaO+aZpfqmL4VZUvxpZLKe0pSqTSBHX9fZu45Nr72b6rH4BpjbWc8ZzpXHjiHC48cS6TnTcoPU0mPxUppX7gvfnX/scuGvH1o0Dsf44ObO/jo84plKRSNKqbpvljl/DUwjG7geXAOcUIVJpounr7+btrH+BHKzYCQ0tKXHTO0bS1NDBiYEHSM4iUynueeUSkcv8eRzr1EzeyY/ceVl/+cqqq/AUoqXJEBCklf/GNUqX1jypv9z3+JBd/814e27aLOVMa+MzrT+LsRTOzDkuaMA7WRzp+Xkb6B3Ns3dnH3CmTTAglSVJZ6+0f5OHOHpY/tJnP3/gQ/YOJlx3fxj++7kSmNNZmHZ5UUkwKy8jm7j5SssiMJEkqT4O5xPfveZz/+9/rWdPZw2BuaLS7rrqKj7/yON5+1pE+KiodApPCMuJ8QkmSVI5SSvz8wQ4+89PVrOkcWr2ltbmepXNaWDKnmVef/ByWzmnJOEqpdJkUlpGOHVYelSRJpa1vYJDfPLqdux/ZzvotO1m/ZSePbt25t5ro6UdN58MvW8LvHTkt40il8mFSWEbaXY5CkiSVoMFc4j/vfJQbV3Zw1yPb6O3PPeV4XU0Vp8yfyvvOXcQ5x87yEVGpwEwKy0hHVx8AbSaFkiSphHzxpjV84aY1ADTX13D20lk8/+gZLJnTzIIZk2lrabCInjSOTArLSIcjhZIkqcT8ev02vnTzGhrrqrnyHady+oLp1FRXZR2WVFFMCstI+w4LzUiSpNKxY1c/77/mXnIJLnvVCTz/aNcWlLLgbZgysm+k0CUpJEnSxJZS4m9/cB8bd/TyypPm8rpTnpN1SFLFMiksIx1dvTQ31NBY5wCwJEma2K65awM3PNDOEdMm8YnXnGDxGClDJoVloru3n517Bi0yI0mSJrx1m3v4+I9/R3VV8MW3PI+WhtqsQ5Iqmklhmehw4XpJklQiPvfzh+jtz/EXyxZxynzXG5SyZlJYJtp3DC1H0dpsUihJkiautZt7uP7+TcxsquNPX7Qw63AkYVJYNvaNFFpkRpIkTVz/estaUoJ3vXAhk+qqsw5HEiaFZaN9OCl0TqEkSZqgHtu6i2tXPMHUxlr+4Mwjsw5HUp5JYZkYHilsNSmUJEkT1FeWr2Uwl3jnC46iqd5q6dJEYVJYJvYuXG9SKEmSJqBNO3bzvd9soLm+hnc8f0HW4UgawaSwTHR0DxWasfqoJEmaiL66fB39g4m3P/9IpkxyCQppIjEpLBMdO3qpCpjZZKEZSZI0sXR29/KtXz/GpNpq/uRsK45KE41JYRkYzCU29/Qxq7me6qrIOhxJkqSnuPK29fQN5PiDM+czfXJd1uFI2o9JYRnY0tPHYC45n1CSJE043717A1feto76mire/UJHCaWJyKSwDAxXHp1tUihJkiaQr//yEf7me/dRFcHn3niyVdKlCcpawGVgb+VRi8xIkqQJ4qvL1/KpG1ZRV1PFV952CucunZ11SJIOwKSwDDhSKEmSJoqUEp+/cQ1fuGkNk2qr+drbT+XsRTOzDkvSszApLAPtJoWSJCkjO3b18/Dmbh7u7GFNRw8PbNzBneu20Vxfw1V/fBqnLpiedYiSDsKksAx0dOXXKDQplCRJRXLr6k4uv+5B1m7e+bRjM5vqueqPTuO5R0zJIDJJY2VSWAaGHx9tm+IahZIkaXxt27mHy697kB/e+wQAC2dO5tjZzSya3cQxrU0sam3m6NbJ1NdUZxyppNEyKSwDw4VmrOglSZLGS0qJH63YyGXXPci2nXuY3VLPZa86gZce35Z1aJIOk0lhGWjv6qWxrprmev87JUlS4T2+fRcf/eEDLH9oMwBvO2M+H375EloaajOOTFIhmEWUuF17BujuHWDhzMlERNbhSJKkMjKYS/zHHY/w2Z+tZteeQRbOmsynX3sipx9l8RipnJgUlrjhIjNWHpUkSYW0qr2LD3//fn674UlqqoKLX3IMFy87hoZa5wpK5caksMQNzyec3WKRGUmSdPh6+wf58s0Pc8XytQzkEifPm8qnX/dclrS1ZB2apHFiUljiOrvzSeEURwolSdLhuXPdVj7yg/tZt2UnjXXVfPSli3n7WQuornKKilTOTApL3PBIoWsUSpKkQ5VS4nM/f4gv3fwwAOcsnsUnXn0CR0xrzDgyScVgUlji2rtMCiVJ0qFLKfHpG1bx1V+so6WhhstffQKvPGmuBeykCmJSWOKGF653jUJJkjRWKSU++ZOVfO229UxtrOU//+QMTnjOlKzDklRkJoUlbrj6aJtzCiVJ0hiklPjE9Sv59/8eSgi/8a4zOH6uCaFUiUwKS1z7jl4ioLXZ6qOSJGl0Ukpcdt2DXHX7I0xrrOUb7zqT4+ZaXVSqVCaFJSyXS3R29zJjcj211VVZhyNJkkpASomP//hBrr7jEaZPruOb7z7D5SakCmdSWMK27dpD/2ByjUJJkjQqKSU+9l+/4//98lFmTK7jm+8+k8VtzVmHJSljJoUlrMPKo5IkKW/Hrn7Wbemhtz9H38Dg3ve+/hy9+fffPv4k1923iZlNQwnhsbNNCCWZFJa04aTQheslSao8KSVWtXdz86pObl3dyW8e3U4uHfy6mU31fOvdZ7DIhFBSnklhCWvfka886kihJEkVYdeeAW5/eCu3rO7k1lWdbNzRu/dYa3M9p8yfRmN9NfU11TTUVj3tvaG2mmVLWpnt3w6SRjApLGHDC9c7p1CSpPI0mEus39LDf6/Zws2rN3Pnuq3sGcgBEAHPmz+VZYtbecmSVo6f2+KC85IOiUlhCevcmxR6t0+SpFK3feceVrZ3sWpTN6vau1i5qZuHOrrpyyeBAC0NNbz0+DaWLZnFi49tZfrkugwjllQuMkkKI6IW+Gfgrfld3wA+kFIaOJxzK83wSKEL10tSebB/rAz9gznWbd7JqvYuHty0Lwns6Op72rlTJtVy8rypnHLkNJYtaeV586ZS4zJUkgosq5HCS4CzgePz2zcAHwEuO8xzK0p7fh7B7GaTQkkqE/aPE1xKif7BRHdvP129A3T39tOdfx/afuq+7v32dfUO8OSuPQzsVxGmuipY1NrEkjktLGlr5rg5LSyZ00xbS4OPhEoad5HSKMpUFfofjdjA0N3M7+W33wB8NqV05OGce4B/K/3lt+4pXPATyP//XTu5BKsvf5kdhqSKFxGklEr6l6H9Y+EkYCCXGBxMQ++5XP49PeV9YDC3377ciGueun8gv/9w1FYHUxvrOHZ2E0vaWliaTwKPaW2ioba6MN+8JO3nYH1k0UcKI2IacASwYsTuFcD8iJiSUtpxKOeOuOZS4GMj9127YmPhvoEJ5sQjppgQSlIZsH+cWGqrg5qqKmqqgprqoKG2huqqoX211UFzQy3NDTX5V+3e95Zn2Dd8XktDLfU1Vfbbkiacoo8URsQ84DFgVkppS37fLKATmJdSevxQzn2Wfy/9ev3Wwn8jE8TitmZaGmqzDkOSMlfqI4X2j4VXk0/iqvOJ3VBSF3uTu5rqp24PH6+qKtlmJEnPaMKNFAI9+fcpwJYRXwN0H8a5B3TaguljDFGSpKKzf5QkZaLo5atSStuBx4GTR+w+Gdiw/+MuYzlXkqRSZv8oScpKVjWNrwI+GhFtEdHGULW0KwtwriRJpcz+UZJUdFktSXE5MANYmd/+BvBJgIi4AiCldNHBzpUkqczYP0qSii6TJSmKKSJSuX+PkqTSLzRTbPaPklQ5DtZHZvX4qCRJkiRpAjAplCRJkqQKZlIoSZIkSRXMpFCSJEmSKphJoSRJkiRVMJNCSZIkSapgWa1TWFQRViiXJGl/9o+SJABSSr4O8gJunaifPdbrx3L+wc491OPPsj9l/X9dCW1gLNeM5rxnO+dQjk3UdmAbsA34OvQ2ksXn+nNR+m0gi3ZQrDZwkP9r20AJtIHD+T+eiG3Ax0clSZIkqYKZFI7O1RP4s8d6/VjOP9i5h3p8LDFMFFdP4M8+lOtHe81oznu2cw712ER09QT+7EO5frTXjOa8ZzvnUI+pdFw9QT/3UK4f7TWjOe/ZzjnUYxPV1RP4s8d6/WjPH815BzvnQMdHG8NEcvUE/uyxXj+W8w927qEeH0sMBRH5oUgpcxFxaUrp0qzjULZsB7INSE/nz4VsAxrPNmBSKEmSJEkVzMdHJUmSJKmCmRRKkiRJUgUzKZQkSZKkCmZSqJIQEe+OiDsi4taIWJh1PCquiKiNiNsj4smIeH3W8aj4IuKsiPhlRCyPiOsjYmrWMUkTgf2j7CNViD7SpFATXkRMB94FvAj4G+DT2UakDAwArwc+n3UgysyjwLkppRcDPwbem3E8UubsH5VnH6nD7iNrCh6SVHhnALemlAaAuyLi2KwDUnGloTLJmyIi61CUkZTSxhGbA/mXVOnsH2UfqYL0kY4Uqigi4uKIuDsi+iLi2v2O1UbElyNiW/71pYgYecNiGrB9xLbttgQdZhtQGShEG4iIGcCfA/9erLil8WT/KLCPVPZ9pL88VCwbgU8AX3uGY5cAZwPH518vBD4y4vh2YOSz0blxilHj63DagMrDYbWBiGgEvgtcnFLaMr6hSkVj/yiwj1TGfaRJoYoipfSDlNK1wDM10ncCn0gpbUopbQL+AfiTEcd/Bbw4Iqoj4hRgzfhHrEI7zDagMnA4bSB/R/Qa4EsppTuKErBUBPaPAvtIZd9HOvSsTEXENOAIYMWI3SuA+RExJaW0I6W0LSL+A7gN6MdfhGVlNG0gf953gFOBnog4PaX0oeJHq/EwyjbwFoaKabRExF8C16eUPlP8aKXisH8U2EeqeH2kSaGy1pR/f3LEvuGvm4EdACmlK4ArihiXime0beCNxQxKRXXQNpBS+jrw9aJGJWXL/lFgH6ki9ZE+Pqqs9eTfp4zYN/x1d5FjUTZsA7INSE/nz4XAdqAitQGTQmUqpbQdeBw4ecTuk4GHMb4mAAADEklEQVQNw49EqLzZBmQbkJ7OnwuB7UDFawMmhSqKiKiJiAaGHlmuioiGiKjLH74K+GhEtEVEG0PVlK7MKlaND9uAbAPS0/lzIbAdKPs24JxCFcslwMdGbO8GlgPnAJcDM4CV+WPfAD5ZzOBUFLYB2Qakp/PnQmA7UMZtIFJKhfw8SZIkSVIJ8fFRSZIkSapgJoWSJEmSVMFMCiVJkiSpgpkUSpIkSVIFMymUJEmSpApmUihJkiRJFcykUJIkSZIqmEmhJEmSJFUwk0JJkiRJqmAmhVIJiYgFEZEiYmrWsUiSNJHYR0qHzqRQmkAiomfEazAi+kZs31CAz780Iq4tRKySJBWTfaQ0fmqyDkDSPimlpuGvI+JW4NqU0udH7FtQ/KgkScqefaQ0fhwplErT70fEwxHxZERcHRG1wwci4pSIuCUituXPeXd+/6uBjwAXDt9Zze8/PyLujogdEbEpIv41IiZl821JknTY7COlMTIplErTK4BTgOOA84C3AUREG/Bz4CvALODVwMcj4tyU0rXAJ4HrUkpNI+647gbeDUwHXgC8BPirIn4vkiQVkn2kNEYmhVJpujSl1JVS2gjcAPxefv8fAr9IKX0npTSYUnoAuAp464E+KKV0W0rp3vz564CvAueMc/ySJI0X+0hpjJxTKJWm9hFf7wSGK60tAC6IiCdHHK8GbjvQB0XEacCngOcCkxj6vbC6kMFKklRE9pHSGDlSKJWXDcAPU0pTR7yaU0oX5I/nnuGabwG3AAtTSi0MzamIIsUrSVKx2EdKB2BSKJWXrwPLIuJ1EVGbf52cv9MJ0AEcGRHVI65pAZ5MKe2MiKXAnxU7aEmSisA+UjoAk0KpjKSUngBeCrwH2MRQB/cvDHVqAN8FuoAtIx6feQ/wwXyltSuAa4oatCRJRWAfKR1YpJSyjkGSJEmSlBFHCiVJkiSpgpkUSpIkSVIFMymUJEmSpApmUihJkiRJFcykUJIkSZIqmEmhJEmSJFUwk0JJkiRJqmAmhZIkSZJUwUwKJUmSJKmCmRRKkiRJUgX7H2Q+vpezV1VGAAAAAElFTkSuQmCC\n", 671 | "text/plain": [ 672 | "
" 673 | ] 674 | }, 675 | "metadata": { 676 | "needs_background": "light" 677 | }, 678 | "output_type": "display_data" 679 | } 680 | ], 681 | "source": [ 682 | "if perform_computation:\n", 683 | " fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(12,4), dpi=90)\n", 684 | "\n", 685 | " ax=axes[0]\n", 686 | " ax.plot(all_theta, tpr_list)\n", 687 | " ax.set_xlabel('Theta')\n", 688 | " ax.set_ylabel('True Positive Rate')\n", 689 | " ax.set_title('True Positive Rate Vs. Theta')\n", 690 | " ax.set_xscale('log')\n", 691 | "\n", 692 | " ax=axes[1]\n", 693 | " ax.plot(all_theta, fpr_list)\n", 694 | " ax.set_xlabel('Theta')\n", 695 | " ax.set_ylabel('False Positive Rate')\n", 696 | " ax.set_title('False Positive Rate Vs. Theta')\n", 697 | " ax.set_xscale('log')" 698 | ] 699 | }, 700 | { 701 | "cell_type": "code", 702 | "execution_count": 18, 703 | "metadata": {}, 704 | "outputs": [ 705 | { 706 | "data": { 707 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4UAAAEcCAYAAABwJhECAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAN1wAADdcBQiibeAAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3gU1frA8e+76ZSE0JEWpCigXlRCExQFFEXqTwS7qCgK3qtX7KIIivVawCt2RS+CDREpokgTkCa9iKC00CGBkED6+f0xs2FZE0jIJrPl/TzPPNmdmZ15N8mePe+cM+eIMQallFJKKaWUUqHJ5XQASimllFJKKaWco0mhUkoppZRSSoUwTQqVUkoppZRSKoRpUqiUUkoppZRSIUyTQqWUUkoppZQKYZoUKqWUUkoppVQI06RQKaWUUkoppUKYJoVKKaWUUkopFcI0KVRFJiLDRcR4LHtFZKqIXFDI/s1F5AsR2S8iGSLyh4iMEJHyhezfwt5/r4hkichuEflERJoVIbYaIvKGiPwpIpkikiIiM0TkqpK+b6VU0RVQTriXWR779BORSSKyx952u4MhlyoRWSci359i+1QR2eijcxX2u/dc5tr7bhORV31wzkj7vC1K/AaUKgUFfC52i8g3ItKwDM9/sBj7327HWaE047LP9ZaIHBKRiEK2DxWRXBGp5cNzzvX4W+TYZdG7IlLNV+coZjwn/b5FJMF+fq0T8Tgp3OkAVMA5AnS1HycAI4CfRKSpMSbZvZOIXA5MA1YB9wN7gZbAE8DVInK5MSbNY/8+wERgPvAgsAuoA9wILATiCwtIRM4B5gDpwKvABiAWuAaYIiKtjDGrS/zOlVJF5VlOeK5zuw6r/JgK3FVGMTllAvC0iMQbY1I8N4hIPHAlMNJH5/oA+MHj+f3AFUBvj3WpPjqXWyTwDLANq7xXyh95lklnY33mfhaR5saY9FI+9wdAoReGCjANaAscK51wTjIBGIxVDk0rYHt/YK4xZo+PzzsHqz4YDlwEPAc0BDr7+DyqGDQpVMWVY4xZbD9eLCLbgF+xCtvPAUSkHDAe+A24whiTbe8/T0R+ApZjFQAP2PufBYzDKpxuN8YYj/N9XoSrNeOBZKCdMcazwvO9iIwFDp/RO7WJSIwx5nhJjqFUiPEsJwrSzxiTZ1+Z9fukUETCgDBjTNYZvHwCVnnXB/jQa9v/ARFYF8RKzBiTBCS5n4vIdUDmaf4WSoUC77rLDuAXrIvHX3nv7Mvvfe/PZRH2PwAc8MW5i2ARsB0r+TspKRSRRsDFwMBSOG+yx99jgV1vfEFEzjLG7C6F86ki0O6jqqTcLXB1Pdb1BWoBT3okhAAYY9ZgJXF32YUAWJXCSOAhr4TQ/ZqphZ1cRC7FKrQe90oI889njNlh7ztXRL72en1Hu5vAefZzd7eBm0TkUxE5jJVcjhORpQWcf4iIHPfoduASkcdEZIvdjfUPEbmtsPiVCkXGmLwzfa2I3Cki6+3P3UERmScizT22x4jIyyKy3f4MbhWRFzy2h9nduXbY29eLyI1e5/hERJaLSC8RWQ9kAK3tbT3tbRlidXV/ubCuV/Z7/QtYilXp8tYfWG6M2Wwfu46IfClWl/vjYnWH91UrYoFE5EERSRKry/1EEanktb2y3bVrn/2eF4lIa49djto/P/boEpZgv/ZFEVkrImn2OcaLSM3SfD9KFdFv9s8EyO9O/R8RGSYiSdgt6kX9TheR3iKy1P7cHhKR6SJS3952UvdREYkQkVc9yqDdIvKtiETa2//WfVREqtr1kEMicsyuz7T0imGbfdxTfqY92XWuL4CeIhLttbk/kA18U5S4S+hvdUkRibbL1532+VaLyDXeLxSRgXY5k2GXU1+LSJy9ra2ITLFjTReRVSJykw/iDUraUqhKqp79c6vHukuBFGPM/EJeMxm4HavLwALgMqyKUZH73Hu4DMgFZp1ux2J6FZiEleDmAlHAdBE5267kuV0PTPPoCjsGuA2rW+0KoAvwkYgcOlVyq1SwERHv75fcgi76FPOYlwLvAE9j9VCIxepm5a4ACPCdvW4kVsWvNtDB4zAjgEeAZ4FlWK1140XEGGMmeOyXALxs778P2Coi12O1/L2L1fWpIfAC1gXWoacIfQLwqohUN8bst2OtAXS0Y3H7FIgB7sbq4XA2cG5Rfjdn6HpgjX2+OsBrwCjgPjvGKKyytRLwMLAfuBeYJSKNjTF7sbqnzsZqDXW3NLi7mlW3j7cbqAY8BMwWkfONMbml+L6UOp0E++dej3U3Auux/v/d5ddpv9NF5Basz+5ErHJHsD4X1bBa4bw9DtwEPIZVd6qJ1WIZdop4JwONsMqZg1ifxzkicqExZovHfqf8TBdiAlY51A07AbT1B37w6PZ+JnEXVT0gj5N/X18DrbC6p/+J9d6miEhLY8wqABF5Cutv8zbW76Sc/T4qYHUZro91C9I7WBf3LsG6gJXnVd4rAGOMLroUaQGGYxVG4fbSEPgJWAlEeez3A7DyFMdpARisLmQAvwMTzjCmd4A9Rdx3LvC117qOdizn2c8T7Offeu0Xbr/3xzzW1cYqxK6znzeyn9/m9dpPgWVO//100aUsFrucMAUsnQvYt4K97fYiHnso8Nsptl9lH69HIdsrY917/IzX+unAJo/nn9jHaeGxTrAqLB97vfYO4DhQ5RRx1cK6uDTYY90Qu7yo47EuDejuw7/Fq8C2QrZtw6pohXusewPY6/H8TiALaOyxLtx+3SvF+RtiVRxr2/te6vT/qS6hs/D3uksTrHvaUoFa9j7bsC5mRHu87rTf6VgXhHYBk053fo/nU4H/nGL/2+3PSQX7eVf7+WUe+5TH6mL6rse6036mT3HODcBXHs+b2+e8oahxF+PvMRcr+QzHuuDezo59rMc+nbzfs71+vjtOrItVx4DXinhesc/5LjD7FL/vBPv5tU7/75b1ot1HVXFVwepOkA1sAS4E+hhjMkt43JK0IJSo9aEQJ/WtN8bkYLUc9vNY3RergunetxPWF8i3IhLuXoCfgRZi3ZekVCg4AiR6LUuKcwDPz5DHZ2cVcKGIvC4ilxbQbekKrHtVphRy2POwriR730P0BdBERKp7rNtl7KvRtiZYV7O/9Pp8zwai7WMXyFiDNMzj5PKjHzDfWPcbua3Cuq/mdhGpR+mbY5dtbhuA6h6/185Yra1bPd4vWO/lpK5rBRGRq+3upkeAHE7cV9XEN+ErVWSedZdNWK3w/czJA6j8bIzJ8HhelO/0c4CzgI+LEcsq4HYReURELrB7OJxKK+CAMWaee4WxBseZCrT32vd0n+nCTAS6eXRZ7Y+VcHmWpcWN+1T6YP0tMrBa8vYB//TY3hmrFXdhAb97d9nTFqtnRaG/exGJF5HRIrKdE3//u9EyqECaFKriclf22gD3YN0L+LmIeP4v7cJqsi9MfY/93D/PtAK0C6hWQF/4ktpXwLqJWF8E7sKkHzDFnLgZvSrW1fAjnCh8srFaHcKxWguUCgU5xpjlXsvR07/MItY9aZ6foT8BjDGzgAFYXdTnAgdF5G05Mc1NFU50XSyI+zPo/fl2P48vYJ1bVfvndK/Y3F3n63JqE4D2Yt03WAerG5N396V+WANxvQ5st+9/6XSa45aE9yBcWVhX090VyKpYZX221zKA07xfEUnEqlAmAbdgVeDa2Jt9XV4rdTruuktLrG6VCcaYGV77FPSZP913ehV73+KMzvkc8F+sLp2rgZ0i8q9T7F+rgNjc8Vb2Wne6z3RhJmAlWD3s5+76jefIrMWN+1RmY/092gMvYSW+z3lsr4rVPdW77BnOibKnKL/7T+z38grWCKuJwEdoGVQgvadQFVeOMWa5/XiJiBzH6krRF+tqO1jN+3eISHtjzIICjtEDq4XNfaP3XOBJEalsPKa1KKK5WP3JO1HwcMqeMvh7wehdoLoV1Po4F+vKVT8R+RRr4IkXPLYnY10NvwTr6qK3/aeJTyll2Y315e2W3xPBGDMOGCfWnFZ9sBKoVKz7XA5x6osvnve6HfJYX8P+6Vn+eJcB7m13Y3WZ97a1gHWevsGqUF2PVUnLxbpn5sQJjdmFdSXehVVJGo51D009Y8whyl4yVpJ6bwHbTtc7pDdW97Z+xt13yx54QykHeNZdClPQZ/503+kV7cdFvuhrt0Y+jTVVTWNgEPCGiGwyxvxQwEv2YJVZ3mpwcpl1xowxm0XkN6C/iGwCGuN1n/QZxH0qKR5/j4V2ef6AiLxljNmJ9b52Ab1OcQx3mVgLq3vwSezGgm7AEGPMOx7rtUGsEPqLUSX1P6wbsx/1WPcVViH2vPdgE2KN8nkL8L5HC9uHWFeACpxIWUS6FXZyY8wvWMnlKBGp6L1dRM4XEfdVpST+PmhDl8KOXcC58rAqcf2wKnapnDwn2Gysq4pxBbSSLDdnNpy9UiHHGJPl9dlZW8A+B4wx72INK9/MXv0zUFkKn8ZmHVaXqL5e668H/jDWUPCF2YRVSUko5PN9yqTNvuA1E6tbVn/gx8JeY4zJM9Zw7c9idXd1Kpn6Geu+qh0FvF/338RdrnlfeY8Bst0JoU1H/VOBpCjf6e5y4YxGGTfWyMNDsS6yNCtktyVYXUAvda8Qa/T2bliD9fnKBKz7sgdhtTgWmugVMe7ieMb++aD982eslsK0gn739j6/Yt3PXdjvPgrr75d/AcuuJ/YoZP+Qpy2FqkSMMUZERmGN3tfJGPOzMeaYPeTvNGCuiIzG6uZwMdaIfauBYR7H2C0itwMT7G5VH2EVsrWxErDLKLxFD6yKxhxguYi8zonJ66/Cml+nNbAT+Ba4095nGnC5vU9xfIE1QMSDWIPR5Cd6xphNIvIOMFFEXsa6wh6NdcN2E2OM38/HplRZEJFmWBUJdyLRUkTS8LpvpoDXPYtVFszFujJ8IVb58Ji9y09YidfnIuIeLbAW1sAm9xhjkkXkDeApEcnB+oz2wRpB74ZTxWyseRUfAj4TkVhgBlZCdDbW1ezrjDGnm2x6AtaUPGBdHPN8b3F27J8Cf2BVaB7C6p2w0d7nVqzysaExpqBRDX3tU6wK4lwReRX4C6vLViuswSteN8ZkichW4HoRWYfVI2MN1t/iAfv3/T3WYBI3l0HMSvlEUb7T7XLhEaw60Hisz7jBur95QkGtkyLyLdbF7JVYSc11WPXxAkdsN8bMFJGFwBci4u4RMRTrwssrPnzLX9jHuxP4yPtC9unitnsC/AncYYz5tDgnNsYkicg4YKBddrvL8p9E5CWsxodYrIEKo40xjxtjDos1Zc/z9j2T07HKzW7As8aYXSKyDKtlMxWrtfcxrO7AscX83YSGMx2hRpfQW/AaQctjfRhWJWam1/rzgC+xuhBl2vuMAMoXcvwL7f33YbUc7sZqibyoCLHVBN7EqrRkAilYBUofr/0ex0oQj9rH7kHBo48WOOoUVrevHfY+VxWy/QGsAizTfu/zgFud/vvpoktZLIWVEwXsU9AIpXNP87prsa4gH8BKPjZhfcmLxz4xWL0OkuzP4FbgeY/tYVgtcDuxkroNwE1e5/kEa5qcgmK4Gqt1Mh2rt8AqrHthwk8Vu/3a8vbrjgMVvbZFAe/b7+kYVtI7FTjfY5/b7d9TQhH/FqcbffRVr3Xu41fwWBdnl63u31cS1qBbl3jscyVWIpjhGR/WMPc77fc8C6tLmsHqzuX4/6ouobEUsUz62+fBXl+k73Ssi0u/2Z+BQ1gXnusXdH6sqROWYyUnR7FaAnt6bC/oc1gN6yJNil1+zAMST/ceCjrWaX4P8yh8tOjTxZ1A0UYinovXSPD2+rOx6n6P28+jsMrqLXbZsxer9bKb1+vuwSrHM+19vgRi7W2NsFp807Hqbo8U8Pc46XdECI8+KvYvQCmllFJKKaVUCNJ7CpVSSimllFIqhGlSqJRSSimllFIhTJNCpZRSSimllAphmhQqpZRSSimlVAjTpFAppZRSSimlQpgmhUoppZRSSikVwoJ+8noR0Tk3lApCxhhxOoaS0LJJqeAU6GUTaPmkVLA6VfkU9EkhgM7FqFRwEQn4OhegZZNSwSZYyia3s846i8mTJ5OYmOh0KEqpEjpd+aTdR5VSfis5OZlrr72W5cuXOx2KUkqFlAEDBrB7924uvfRSJk6c6HQ4SqlSFhIthUqpwLNx40a6d+/On3/+CcDUqVMdjkgppULHhx9+yPnnn8/QoUO54YYbqFq1Kp07d3Y6LKVUKZFg774kIibY36NSwWbGjBn079+f1NRU+vTpw6effkr58uXzt4tIwN+3o2WTUsEnGMomOLl8+uGHH5g4cSIfffQRLpd2MFMqUJ2ufNKkUCnlN4wxvP766zz88MPk5eUxbNgwhg8f/reKSDBUvLRsUir4BEPZBKcunxYsWEC9evWoV69eGUellCqJ05VPZX7JR0SGiMhyEckUkcmn2TdCRN4SkWR7GSMi2uVVqSD19ddf89BDDxEZGcmECRMYMWKEXplWSoU8f6k7bdu2je7du5OYmMiiRYt8cUillJ9wora1G3gOeL8I+z4FtAea20sH4InSC00p5aQ+ffpw11138csvv9C/f3+nw1FKKX/hF3Wn2rVrc+ONN7J//34uv/xyxo0b54vDKqX8gGPdR0VkONDCGNPrFPvsBB40xnxtP+8LvGqMqV+M82gXLaX82Jo1azhy5AgdOnQo8muCoYtWsJdNGdm5/Hkgjc370vhj31H+2JdGUsox2jeqyv1XNCauXITTISrlc6VdNvlL3Wns2LHcf//95ObmUufS62nS/R4iIyKIDBMiwlzWEu466Xl4mBDp3hbmIiLc63mYEBl+8vOTttnHPOl5/jrrebj7dS4XLldAf0Uo5XOnK5/8tiumiMQDdYBVHqtXAfVEJM4Yc6SQ1w0Hnin9CJVSJTV58mRuvvlmoqKiWLduHbVq1XI6JFVMxhg27TvKpr1H8xPAzfvT2H4onbwC6pS/7z3KNyuSeKBzE25sXY+IMO0erJSvlFXd6d577yWqSh0G3nYjSfO/5FDSX1Tr/QQSHlmS8H0q3CWnTDbd68JdntvtbR6J60nP3etc9r726yI9jx3u9byQ83qeSxNY5Q/8tqVQROoCO4BqxpiD9rpqwH6grjEmqYjnCeqr8UoFImMML7zwAk8++SQAzz//PI8//niRJ37WlkL/MXLqBj5csPVv62vFRdO4RkWaVK9AkxoVaVyjApXLRzJ27p98sXwnxkDDauV5qlszOp5TLegm/VahyemWwrKuO23evJnuPXqQ2KoVY8a+T06eITs3j+wcQ1ZunvXYXrJyzEnPs3NPPM/KNWTn5J38PDcvf537eY79upOO/bdzGbLs1+XkWcfNyrUWfy1yw1zyt4T0pJbV8BPJZaRnshnu9dzrcWS413OvY538upO3F5bohmkCG7ACtqUQSLN/xgEHPR4DHC37cJRSvnD8+HHuuusuPv/8c8qXL89nn31G7969nQ5LnaG1u6yGhxta1eMfdeJobCeAsdEFdw998f8u4Na2CTw3bQOL/jzEgE+W0aFxVZ7q1oxzalYsy9CVCkZlWndq3LgxSxYvJjo6mqgoq5Xw8OHD1KpSyden8olcO2nNyk84PZ7bCWZ2XiHbCklAC0pGsz2T1xyv57l55BR0Xvt5elYO2bmG3IK6WvgBEf6enJ4iAQ13ubd7J6AnEtuTnnsnq+He2wo6lpVEeye24S7RC47F4LdJoTEmRUSSgBbAn/bqFsDOwro/KKX8W2pqKl26dGHp0qXUq1ePKVOm8I9//MPpsFQJJKdnAfBkt6ZUiCraV0qzs2IZf1drZm3cz6jpG/ll80GufnM+N7Sqx9ArzyG+vP90QVMqkDhRd4qLi8t/vGrVKjp27MhLL73EPffcUxqnK5EwlxDmCiM6IszpUE7LncD+PYm0H+ecnJh6J7p/b409kXwWlpD+LVkt5FzudZk5eaRlWkmsv4r0aO30TGbDw1zERofzQOcmXNqkmtNh+oUyTwrtYZHdi0tEooE8Y0xWAbt/DDwpIgvt508AH5RNpEopX6tYsSJNmjQhIiKCSZMmUb16dadDUiWUkp5FZLiL8pHFq2SJCF2a1eCyJtX4bPF23pz1B+OX7GDdriNMuu8S7aKklIdAqTtt2rSJY8eOMWjQINatW8frr79OeLjftj/4tUBKYI0xf09APboEZ+UWkKwWsi3rFIltQd2QC+p6nOPdWmsnzMcyc8nKzTsp9ts/XsoT1zTlzvYNQr5VsczvKSzkZuZ5xpiOIvIOgDFmkL1vBPAGcKO933jgAWNMTjHOFxT37SgVyPbu3UvNmjUByMjIQESIioo64+PpPYX+IS/P0OjJ6VSvGM3iJzqV6Fgp6Vnc/vFSVicd4dkezbmtXYJvglSqDJVW2RRIdacFCxbQp08fDhw4QOfOnfnyyy+Jj48/o2Mp5WvGmPx7X39cv49Hv1lDZk4e/3dRHZ7vfV5AJOFn6nTlk2MDzZSVYKh4KRWo8vLyGD58OG+88QYLFy7k/PPP98lxNSn0DynpWVw48iea1Ypl+r+KPqVIYTbsTqX7WwuIiQjj54cuo0ZstA+iVKrsBEPZBCUvn7Zt20bPnj1Zs2YNjRs35vvvv+ecc87xYYRK+caapMPc/elv7E3NoEXdSrx3y8VUD9LvntOVTzoWuFKqVKSnp9O3b19GjhyJiLBv3z6nQ1I+dsi+n7Cyj+4BbHZWLAPaJZCWmcOI7zf45JhKqbKXkJDAwoUL6dmzJ5s3b+aTTz5xOiSlCnRBnUpMGXIJF9arxKqdh+n+1gJW7zzsdFiO0KRQKeVzO3bsoH379kyaNImGDRuyePFiOnfu7HRYyseSfZwUAjzYpQlnxUUzbe0e5mza77PjKqXKVoUKFZg0aRIfffQRI0eOdDocpQpVPTaaiXe34bqL67AvNZO+7/7K5JW7nA6rzGlSqJTyqUWLFpGYmMiqVau44oorWLJkCU2bNnU6LFUKSiMpLB8VzvAezQF4+rt1HM/K9dmxlVJly+VyMWDAgPzBZhYsWMB9991HVlZB4+Mo5Zyo8DBeue4Chl3bjJzcPB74YhUvzNjot1ODlAZNCpVSPvXVV1+xf/9+Bg0axA8//ECVKlWcDkmVktJICgGubF6TLs1qsDP5OGNmb/bpsZVSzsjLy2PIkCGMHTuWLl26cODAAadDUuokIsKd7RvwyYBWxEaH8+68v7hr3DJSM7KdDq1MaFKolPKpV155hUmTJjF27FgiIgqewFwFh5RjpZMUAjzboznlIsN4b/5f/LHP53NuK6XKmMvlYtq0abRs2ZL58+fTqlUr1q5d63RYSv3NpU2q8d2Q9jSqXoE5mw7Q678L+etAmtNhlTpNCpVSJZKamsp1113Hr7/+CkB4eDi9e/d2OCpVFg6llV5SeFalGP7dpQk5eYYnv11LXgh14VEqWNWuXZt58+bRr18/tm3bRrt27ZgyZYrTYSn1Nw2qlufb+9rR6dzq/HUgnZ7/XciapOAegEaTQqXUGfvrr79o27Yt33zzDU888QSBPsWCiESIyFsikmwvY+xJowvat7aITBaRQyJyUES+EpEaZR2zk0qzpRDg9nYJNKsVy7JtKXz1285SOYdSqmyVK1eOCRMmMHLkSNLS0ujVqxcLFy50Oiyl/qZidATv3dqSey49m6MZOTz6zdqgvsdQk0Kl1BmZO3curVq1YsOGDXTt2pVvv/0WkYCfnuspoD3Q3F46AE8Usu/b9s/6QAMgCniztAP0J76eksJbeJiLUX3ORwRemPE7Hy/cytKtyaRlFnkObqWUHxIRnnrqKb755htuv/122rVr53RIShUozCU8dvW5tKwfz8Y9qXy+ZLvTIZUanbxeKVVs7733HoMHDyYnJ4cHH3yQV155hbCwsDI7f2lNEC0iO4EHjTFf28/7Aq8aY+oXsO8a4EVjzOf285uAx40x5xXxXAFfNnUfs4C1u46w/KnOVK0QVWrnGT5lPZ8s2pb/XAQSqpSn2Vmx9PjHWVzVvGapnVup4tDJ60tmwYIFNGrUiJo19TOt/Mv63UfoPmYBFaMjmDO0Y6ldDC1NOnm9Usqnpk+fzj333IOI8OGHH/Laa6+VaUJYWkQkHqgDrPJYvQqoJyJxBbzkNaCviMSJSCXgBmBa6UfqP9yjj1aKKd0BhZ6+thmfDEjkoS5N6Nq8JrUrxbD1YDrT1uzhXxNXkp2bV6rnV0qVvs2bN9OtWzcSExNZsWKF0+EodZLmZ8VxU+v6HDmezSszNzkdTqnQpFApVSxdu3bl7rvvZvbs2dxxxx1Oh+NLFeyfnneSux9XLGD/hUB1IAVIBioDzxV2cBEZLiLGvfggXsclp2dRqVwE4WGl+1Xicgkdz6nO/Z0a884tF7Pg0StY/fSVJCbEk5Gdp6OTKhUE6tatS69evUhKSqJ9+/Z89dVXToek1EkeurIJ8eUimLhsR1AOOqNJoVLqtDZt2sS8efMAa1jxd999l/bt2zsclc+5x5v2bBV0Pz4p6xARF/ATVmJYwV4WADMLO7gxZrgxRtyLz6J2yPGsXI5n51K5nDNdaOLKRdAyoTIA63YdcSQGpZTvREdH88knn/DKK6+QkZHB9ddfz7PPPktenvYEUP6hUrlIHr7qXIyBZ6asD7pRsTUp9HL4WBbDJq9jwtIdHM/KdTocpRz3448/0rp1a3r16sWOHTucDqfUGGNSgCSghcfqFsBOY4x31lEZa4CZ0caYY8aYY8AYoK2IVC2TgB2WXMojjxbF+bWtnH2tJoVKBQURYejQoXz//fdUrFiR4cOH069fP7KzQ2PycOX/+iXW5fzacazccZhvViQ5HY5PaVLoZcTUDXy2eDuPT1pL2xd/5sUZv7Mz+ZjTYSlV5owxjB49mquvvpojR45w7733UqdOHafDKm0fA0+KSE0RqYk18ugH3jsZYw4CW4DBIhItItHAYCDJ3hb0ku05CuP9IilMdSwGpZTvdevWjV9//ZUGDRpQpUoVwsMLnBlIqTIX5hKe7dkcgJd++J0jx4PngoUmhR4W/3WISSt2AdDxnGqkZeTwzrw/6fDyHG76YLEmhypkZGVlcc899/Cvf/2LiIgI/qDlVokAACAASURBVPe//zFq1ChcrqAvMkYCvwIb7WURMApARN4RkXc89u0JXATsAvYArYAeZRqtg9wthVUcTArrxMcQFxPBxj2pOtiMUkGmefPmLFu2jDFjxuRPd5SaqheAlPMuqhfPdRfX4WBaFm/M+sPpcHwm6Gt4xTF97R4AXuxzPp8MaMWCR6/gn1c04qy4aBZuOcTdn/2mXUpV0Dt27BhdunTh/fffp2bNmsybN4+bbrrJ6bDKhDEm2xgz2BgTby9DjDE59rZBxphBHvtuMMZcZYypYu97hTFmpXPRl63k9EzA2ZZCEeH82nFk5eSxeV/a6V+glAooVapUISLCGt142bJl1K9fn//9738OR6UUPNr1XCpGhfPpr9vZtDc4BjvTpNDDniMZgDXsLEDNuGj+feU5zH/kcto1rMLGPak8/d06J0NUqtTFxMTQsGFDLrroIpYtW0br1q2dDkn5oeR0q8uMky2FAOfZXUh1sBmlgtu6des4evQot9xyC4899hi5uXqRXjmnWsUoHuzShNw8wzNT1hHo8w6DJoUn2WsnhTXjok9aHx7mYswNFxIXE8GklbvIzNGCSAWf/fv3A1bry9ixY/nll19C4R5CdYbcLYVOT+Drvq9wza7gGx5cKXXCgAEDmDFjBpUqVeKll16id+/eHD0aHC00KjDd2rY+59SoyOK/kpm6Zo/T4ZSYJoUe9qZmEBEmBV75rlIhin/UrURungmaZmKlwBpQ5uWXX6Zhw4asXGn1foyKiqJcuXIOR6b8mXvieie7j4IONqNUKOnSpQtLliyhSZMmfP/997Rr146tW7c6HZYKUeFhLob3sAadGTV9I+mZOQ5HVDKaFNqycvI4mJZJ9YrRuFwFTyHW5mxrTqznp20kN8jmJlGhKSMjg9tuu41HH32U3NxckpKCa3hlVXrcSaHT3UfrVtbBZpQKJU2aNGHx4sV06dKFdevW8dlnnzkdkgphbRtW4doLarHnSAb/nbPF6XBKRJNC2/6jGRgDtby6jnq6s30DmtWKZcnW5ID/wyu1d+9eLr/8cj777DPq1KnDwoUL6d69u9NhqQCR31Lo0OT1bjrYjFKhJz4+nunTp/P+++/z1FNPOR2OCnFPdmtKTEQY7//yF38dCNzvIU0KbYXdT+gpKjyMMTdeSExEGK/99Ac9/7uQDxdsZV9qRlmFqZRPrFy5ksTERBYvXkybNm1YtmwZF154odNhqQCS31JYwdmkEHSwGaVCUXh4OHfddVf+VEmzZ8/moYceIicnsLvwqcBTKy6G+zs1IjvXMGLqhoAddEaTQtteO7GrGVt4UgjQsFoFxtxwIQlVyrF652FGTt1Amxd+pu87i3hr9mY27tH7WpR/M8bwxRdfkJSUxC233MKcOXOoWbOm02GpAJOcnkVUuIuYiDCnQ/G4r1CTQqVCUW5uLvfeey+vvfYa1157LYcP68BTqmzd2b4BDaqWZ+6mA8zauN/pcM6IJoW2orQUunVuVoM5QzsyZcglDOzQgBoVo1m2LYVXf/yDq9/8haVbk0s7XKXOmIjw/PPPM3HiRMaNG0d09On/55XylJtnOHw8myrlI/MnlXaSJoVKhbawsDBmzJhB8+bNmTlzJm3atGHz5s1Oh6VCSFR4GE9e0xSA8Uu2OxzNmdGk0HY0w+puEBsTUaT9RYQL6lTiyW7NWPTYFUy9vz03tKoLwPerd5danEqdiWPHjnHzzTczf/58wPoC7devn19U6FXgOXwsC2OcH3nUzT3YzAYdbEapkHX22WezaNEirr32WjZt2kTr1q2ZNWuW02GpENK2YRUAth865nAkZ0aTQtvxbGvuwTPpCuVyCefVjmPolefgEvhh/V5ytGKi/ERSUhIdOnRg/PjxDB06NGD7uiv/kXLMup/Q6TkK3USE82rH6mAzSoW42NhYJk+ezCOPPEJKSgpdu3ZlxYoVToelQkT5qHCqlI9kV8px8gJwlgJNCm3Hs6yksFzkmd8fU6VCFJc0qsqBo5lMWrnLV6EpdcaWLFlCYmIiK1as4LLLLmP69OnaOqhK7FCafyWFoIPNKKUsYWFhvPTSS4wbN44BAwboIGqqTNWpXI6s3Dz2HQ28QSgdSQpFJEJE3hKRZHsZIyLhhexbW0Qmi8ghETkoIl+JSA1fx3TMTgpjSpAUAlx3cR0AXpi+kbw8w+SVu3jwi1WMW7StpCEqVSzjx4/nsssuY+/evQwcOJAff/yRqlWrOh2WCgL+1lIIel+hCn7+WHfyZ7feeivvv/9+/oXQ+fPnc+jQIYejUsGuXuVyAOwIwC6kTrUUPgW0B5rbSwfgiUL2fdv+WR9oAEQBb/o6oOPZ1j2F5SILLF+LrNv5tYgKd5FyLJuhX63mgS9W8e3KXTwzZT2//qmFkSobc+bM4eabbyY7O5s333yTd999l8hI/6nAq8B2yJ6OorLDcxR6uqB2JUCTQhXU/K7uFCg2bNjANddcQ6tWrdiwYYPT4aggVjc+BoCdKccdjqT4nEoK7wCeM8bsMcbsAZ4H7ixk3wbAl8aYNGPMUeAL4DxfB3TMB91HAcLDXDzbozlAfhfS+HLW4DU3frCYZdt0ZFJV+jp27MigQYOYMWMG//znP7XLqPKpFHdS6AdzFLq5B5vZuCdV7+lWwcrv6k6Bon79+nTt2pW//vqLNm3aMH36dKdDUkGqrt1SuDNZWwpPS0TigTrAKo/Vq4B6IhJXwEteA/qKSJyIVAJuAKad4vjDRcS4l6LGld991Adzbl19Xi2qVoiiesUoHr7qHJY+2ZlWCZUxBoZNXqcDfahSsW3bNmbPng1YA2+MHTuWK6+80uGoVDBytxRW8aPuo+7BZjJz8ti8XwebUcHFX+tOgaJ8+fJ8+eWXPPPMMxw9epRrr72WV199VetjyufqxttJYYomhUVRwf7pObOo+3HFAvZfCFQHUoBkoDLwXGEHN8YMN8aIeylqUBnZvrmnECCuXAQ/P3QZCx+7gsGXNyIizMWnd7aiWsUoft97lGXbUkp8DqU8/fLLLyQmJtKrVy+2bNnidDgqyLlbCuP9qPsonBhsRruQqiDkl3WnQOJyuRg+fDhffvkl0dHRPPzwwwwYMIDc3FynQ1NBpG5lq/toUnKQdx+1b3J+VkS2iMgRe11XERlcjMO4L+F6XtlyPz7qdT4X8BNW4VbBXhYAM4sTd1H4qvuoW1xMBBFhJ3690RFhDLqsIQDjft3mk3MoBfDhhx/SqVMnDh48yIABA0hISHA6JBXk8lsK/aj7KHgMNpOkSaEKOn5ZdwpEffv2ZcGCBdSuXZvY2FjCwnxT71MK4KxKMbgkNFoKX8a6yXkQ4G5z3wjcU9QDGGNSgCSghcfqFsBOY4z3N3llrJukRxtjjhljjgFjgLYi4tNhFHPt+UTCXaXXeHplM2vgr2lr9pTaOVToyMnJ4d///jd33XUXxhjeffdd3nzzTcLDSzZYklKn4x591N9aCt2Dzei92yrY+GvdKVBddNFFrFixgtdeey1/3dGjR0/xCqWKJiLMRa24GPamZpCZE1it0MXNgPoC1xljZgF5AMaY7UC9Yh7nY+BJEakpIjWxRs/6wHsnY8xBYAswWESiRSQaGAwk2dsCSp34GCpGhxMTEab92FWJZGZm0r17d15//XWqVKnCrFmzuPvuu50OS4WI5LQsRKCSnyWFdSvH0KRGBX7fe5T1u7W1UAWdkKw7lZbq1avnX0RdtGgRCQkJfPvttw5HpYJB3coxGAO7AmwE0uImhQKc1B4qIhXw6rpQBCOBX7FaGTcCi4BR9vHeEZF3PPbtCVwE7AL2AK2AHsU8n18QERpWq8Dx7Fwe/WaN0+GoABYZGUlCQgLNmzdn6dKlXHbZZU6HpEKEMYZD6VlUiokgzOVftx6JCP0SrWuUXy7b6XA0SvlcSNadysKKFStISUmhT58+PPfcc3rhXpXIicFmgjspnINVKHl6FKvvepEZY7KNMYONMfH2MsQYk2NvG2SMGeSx7wZjzFXGmCr2vlcYY1YWM+6ixOTrQxbo+d7nUb1iFF8uT+L3vallck4VPA4etC7yigijR49m0aJFnH322Q5HpULJ8excMnPy/Griek+9L6xNRJjw7cpd+QOIKRUM/LHuFCyGDBnC5MmTqVChAsOGDePGG2/k+PHAqtAr/xGo01IUNyl8EOggIgeAWBHZBXQGHvF5ZA4p7encmp8Vx/6jmQB0feMXuo3+RefUUkXy9ttv06BBA5YuXQpAREQEsbGxDkelQs2hNHuOQj9NCiuXj+TK5jVJzchh5vq9ToejlAoQPXr0YOHChdSvX5+JEydy6aWXsmvXLqfDUgHIPQJpoA02U6yk0BhzwBjTFrga6A/0AS7RPurF457MHmD97lSGfL4y4G5GVWUnOzub++67j8GDB5OVlcXWrVudDkmFMPcgM/6aFAL0a1kXgIlLtQupUqroLrjgApYuXUr79u1Zvnw548ePdzokFYDc3UcDbVqK4k5J8TaAMWa5MeZrY8wSY0yeiLxVOuEFp5VPX8kDnRtzS5v6JFQpxw/r9zLw0984nqWJoTrZoUOHuOqqqxg7diw1atRg7ty59OvXz+mwVAhzT0fhz0lh+0ZVqV0phl//OsT2Q+lOh6OUCiDVq1fn559/ZuzYsQwdOtTpcFQAyu8+GswthcDNhay/oaSBOK2sbyl+oHMTRvY6jy8HteWcGhWZ/8cBnv5uXRlHofzZxo0bad26NXPmzKFFixYsXbqUtm3bOh2WCnEpAZAUulxC35Z1APhyubYWKqWKJzIykkGDBuGypyn78ccfGTZsGHl5eruPOr1qFaKICncF5z2FItJDRHoAYSLS3f3cXh4Egmbs77IeS696xWgm3t2GKuUj+XpFEut2Bc2vUpXQl19+yZ9//kmfPn1YsGAB9eoVd+YXpXwvOd0/5yj01rdlXUTgq+VJet+2UuqMZWdnc/fdd/Pcc89x3XXXkZaW5nRIys+5XELt+BhSjmWTnpnjdDhFVtSWwjftJRoY7fH8day5C/9ZKtGFiPjykTzQpQnGwKjpG3UoZAXAsGHDGD9+PF999RXly5d3OhylgBNJYZUK/p0U1q4UQ4fG1dh/NJN5fxxwOhylVICKiIhgxowZNGrUiG+//ZZLLrmE7du3Ox2W8nMxEWEAZAfQRckiJYXGmAbGmAbAN+7H9tLQGNPOGDO1lOMMejck1qVhtfIs+vMQs3/f73Q4ygGZmZkMHDiQn3/+GQCXy8WNN96Y331FKX+QnN99NMrhSE6vf6I94IzOWaiUKoGmTZuyZMkSOnXqxJo1a0hMTGThwoVOh6X8WG6e1cDj8rP5fE+luKOPXl9agTjN6ca58DAXT1zTFIDnp2/U7k4hZv/+/XTq1IkPPviABx98UO9bUH4rPyn08+6jAJ2b1qBy+Uhm/76ffakZToejlApglStXZsaMGQwePJgDBw5w+eWXs2HDBqfDUn4qze42Wj4y3OFIiq64o49GicjjIjJTRH4TkRXupbQCLGtS2hMVnsIV51andYPK/HUgnR5vLWTL/qOOxaLKzurVq/OvOrZq1YqZM2dq66DyW/lJoZ93HwWIDHdx3cV1yM0zfKGthUqpEoqIiOCtt95i7Nix3HnnnTRt2tTpkJSfSs/MISYijLBgbSkEXsMagXQ6cA4wDigHfOfjuEKSiPDQledQvWIUG/ak0uftRWzaq4lhMJs8eTKXXHIJO3bs4KabbmLu3LnUqlXL6bCUKlTyscBpKQS4sZU1QNOEpTu0B4ZSyicGDRrE2LFj8xsS5s2bx5EjOlCgOiE9M5fyUYHTSgjFTwp7Ad2MMW8COfbP3kBHXwdW1kyZT0pRsFYNKrPg0Su4s30DUjNyuPWjJSQF2DwnqmgWL15M7969SU9PZ9SoUXz22WfExMQ4HZZSp5ScnkVMRBgxkWFOh1IkCVXL06FxVfYcydD7tZVSPrd69Wquvvpq2rZty5YtW5wOR/mBrJw8snLzqBAVGN+TbsVNCssbY7bZjzNEJNoYsxG42LdhOccfGnkjw1081a0p17esw77UTG79cCmH0jKdDkv5WOvWrbn33nv59ttvefzxxx3tuqxUUeTk5nHkeLZfz1FYkJvb1Afgf0t2OByJUirYJCQkcPnll580t7AKbe5pKIK9pXCziPzDfrwWeFBE7gUO+jYsJSKM6n0+XZrV4K+D6dz+8bL8m1ZV4Nq9ezezZs0CrL/x22+/Ta9evRyOSrmJSISIvCUiyfYyRkQKLdXtuVpXiUi6iOwWkUFlGW9ZO3w8G2P8e+L6gnQ6tzq14qKZ/8cBth9KdzocpVQQiYuLY8qUKTz00EMkJydz5ZVX8s477zgdlnJQWogkhU8AFezHjwN3AM8A//ZlUMoSHuZizA0X0qpBZdbuOsJNHyyh+5gFDJu8joPachhwli9fTmJiIj179tQRy/zXU0B7oLm9dMAq9/5GRLoCbwMPALH2/nPLJEqH5E9cH2BJYXiYi/6J1r2Fny/V1kKllG+FhYXx6quv8tFHHyEi3HvvvQwZMkTnnQ5R6VlWUlghmJNCY8xPxpiF9uPlxpjGxpiaxpjJpRNe2fHXz210RBgf3NaS82rHsnrnYdbuOsJni7dz+atz+WThVh04IUBMnDiRDh06sHv3bvr370+jRo2cDkkV7A7gOWPMHmPMHuB54M5C9h0JjDDGzDXG5BpjUowxv5dZpA7In7g+wJJCgH6JdQlzCV8tTyIzJ9fpcJRSQWjAgAHMmTOHatWqERMTo7eFhKhQ6T76NyLSR0TW+CIYf+CPn9/Y6Agm3XsJr1x3AZefU41O51bnWFYuw7/fQI+3FnLgqLYa+qu8vDyGDRvGDTfcQFZWFq+//joffPABkZGBV6kOdiISD9QBVnmsXgXUE5E4r33LY91LHSsiv4vIXhH5QkRqnuL4w0XEuJfSeA+lLb+lMEBGHvVUMy6aLk1rkJyexYy1e50ORykVpC655BJWrVrFiy++mL8uPV27rYeStEzrwmNQDjQjIlVF5BMRWSsiX4tILRG5WER+A8YCn5dumCoy3EXflnX5eEArPrw9kan3t6dl/Xg27Enl/8YuYm2SDoXsb3Jycujbty/PPfccsbGxTJs2jQceeECvHPovd9f4wx7r3I8reu0bjzUu1S3AVUAjIBv4rLCDG2OGG2PEvfgm5LKV31IYAHMUFsQ94Mw78/7Mfy9KKeVrZ511FmFhVkIwb948EhIS+OGHHxyOSpWV9ACcuB6K3lI4BusK+ligCvANMBmrAlTfGPPiKV6rSkHTWrH8767WtEqozI7kY/R+eyGjf96s3Un9SHh4OPXq1aNRo0YsXryYrl27Oh2SOrU0+6dnq6D7sfeEoe59Rxtjthtj0rDur+5ktyIGpUBuKQRo17AKF9eP5/e9R+nx1gI27kl1OiSlVJBbvHgxBw8epFu3brzxxht6n2EICPaBZjoCfY0xbwP9gTbA/xlj3jDGZJRWcGUpED+j0RFhfHpnK5645lwiwly89tMf3DFuOakZ2U6HFtKSk5PzH7/yyissXbqUpk2bOhhR6BCR2iJS2WtdvIicdbrXGmNSgCSghcfqFsBOY8wRr30PAzugwAlOA7IVsCjcSWGgjT7q5nIJn93Zim4X1CIp5Th93l7EjLV7nA5LKRXEHn30USZMmEBkZCQPPvggAwcOJCtLeyoEM3dLYbAONFPOrjBhjNkHpBljlpZeWM4JtK590RFh3H1pQ6b9sz1NalRg/h8HuG7sInYm64T3Tvjkk09ISEhg4cKFgNVaGB8f73BUIWUSUNdrXX2s3g1F8THwpIjUtO8PfAL4oJB93wP+aSeiMcDTwM92q2FQCvTuowDlIsN564YLefiqc8jIyeXe8StY9KfOqqSUKj39+/dn/vz51KpViw8//JDOnTtz4MABp8NSpSTYB5oJE5HzReQCEbkAMJ7P7XXKQWdXq8DX97bj0ibV+GNfGr3fXsjKHSlOhxUycnNzeeSRRxgwYADHjh1j8+bNTocUqs4xxqz2WrcaKGpT7UjgV2CjvSwCRgGIyDsi4jn51IvAz/bxdwLlsO4xDFopxwK7+6ibiDD48kY826M5AJNW7HI4IqVUsEtMTGTZsmW0bNmSX375hYkTJzodkiol7oFmygfjQDNYlZ1VHktFrIqQ+/nKUolOFUtsdAQf3daSm9vU42BaFv3fW8zUNbudDivopaam0qNHD1555RXi4+P58ccfuf32250OK1QdEZEaXutqcOIewFMyxmQbYwYbY+LtZYgxJsfeNsgYM8hj31xjzEPGmKr20tcYE9TDWh5KC9wpKQrS68LaRIQJszbu0/uxlVKlrnbt2sybN4///ve/DBkyxOlwVCkJ6u6jxhiXMSbM/lnQElipcBALD3Mxsud5PH1tM7Jy8xjy+Ur+O2eL3thcSv7880/atm3L9OnTOffcc1m6dClXXHGF02GFsu+Bj0WkNlj3GALvA1McjSpIpBzLwiUQFxPhdCg+ERsdQbuGVTl8LJulW5NP/wKllCqhcuXKcd999+XfrjR16lReeOEFracFkWDvPqoCiIhwR/sGvH9LS8pFhvHKzE3c9vEy1u/WaSt87auvvmLDhg107dqVxYsX66T0znscyAB2isgxrMFgsoFHHY0qCBhjOJSeRXy5SFyuwLr3+lS6nmdNLTlzfVA38iql/NDx48cZOHAgTzzxBDfffDPHjx93OiTlA2nB3FKoAlPnZjX4alBb6lcpx/w/DtBt9AL+NXElOw7pIDS+8sgjjzBu3DimTp1KXFzc6V+gSpUx5qgxpg9QC7gMqGWM6WOM8Z5SQhXTsaxcsnLyiA+SrqNuXZrVQARmrt9HXp5eqVdKlZ2YmBhmzJhB3bp1+fzzz+nYsSN79uiIyIEuPUtbCgNasDbbNz8rjp8evIwRPZtTtUIk363azRX/mcvT363jwNFMp8MLODk5OTzwwAP5k9C6XC5uvfXW/ElqlX8wxuwzxiwzxux3OpZgEejTURSmaoUoEutXZm9qBmt2aW8KVTrsUYo/EpHVIvKX5+J0bMpZLVq0YNmyZbRr146lS5eSmJjIihUrnA5LlUCgDjQTWClsKQuw2SiKLDLcxa1tE/i/i+rw0YKtvDv/Lz79dTtf/5bEne0bMPDSs4mNDo57hEpTcnIy/fr1Y9asWUybNo2NGzcSHq4fIaeJyAJjTHv78UoKnjsQY8xFZRpYkMlPCgN85NGCXHVeTZZuS+aHdXtpUbeS0+Go4PQ/4BjwEpDucCzKz9SoUYPZs2dzzz33MG7cONq3b8/atWtp2LCh06GpMxCoA82cUbRi3R1b0xhzRm3cIhIBvA7caK8aDzzoHuWvgP17ACOAxsARYIQx5p2C9lWFKx8Vzv2dGnNTm/q8PWcLn/66nTGzt/C/xdsZfHkjbm5Tn+iIwLqqUVZ+//13evTowebNmzn//POZMmWKJoT+422Px284FkWQy08KA3iOwsJc2awGI6duYOb6vTza9ZyAm69WBYSLgarGmDOetVzrTsEtKiqKjz/+mPPOO4/t27drQhjA0jNzcAnEBFiduljdR0Wkooh8ijWQwxZ7XS8RGVHM8z4FtAea20sHrEmiCzpnV6xK3wNArL3/3GKeT3moXD6Sp65txpyHO9L34jocOZ7Nc9M2csWrc/ly+U4dmt3LzJkzadOmDZs3b6Znz54sWrSIhIQEp8NSNmPM5x5PtxhjxnkvgE4cWULB3FJYt3I5zqsdy9aD6WzeX6TZS5QqrvVY9zqXhNadgpyIMHToUEaPHp2/bt68eRw7pmNBBJK0zBzKR4YH3AXG4t5TOAYIA84D3Fe7FgP9inmcO4DnjDF77NbG54E7C9l3JNbVrbn2vGApxpjfi3m+0wrOOwpPrXalGF7p+w9mPnApVzarwe4jGTzy9Rq6vvkLM9fvDdr7LItj9erVXHPNNRw5coTHH3+cSZMmUaFCBafDUoWbUcj6qWUaRRAK1nsK3a5qZo1C+tOGfQ5HooLUJOB7ERkoIj08l2Icw+/qTqp0uJOJZcuWcdVVV9G+fXt27tzpcFSqKIwxpGfmBNwgM1D8pLArcJcxZjN2HmVP1uw9WXShRCQeqIM16b3bKqCeiMR57Vseq8tFrIj8LiJ7ReQLEal5iuMPFxHjXor8zoDAyud9p3GNirx3a0sm3deO1g0qs2V/Gvd89ht9xi5i8V+HnA7PURdccAGDBw9m/PjxjBo1CpdLx2byc3/7GItIVSDXgViCyqEgTwrbNaoKwModhx2ORAWp+4CKWC17b3osRery7s91J1V6zj77bNq2bcvKlStJTExk8eLFToekTiMjO488E3iDzEDxk8JMvO5DFJEqQHFm/XU3s3h+87ofV/TaNx6rkncLcBXQCGvOsc8KO7gxZrgxRtxLMeIKeRfVi2fi3W34ZEAiTWvFsnLHYfq/t5jbPloaUnMcHjhwgFmzZgHW1brRo0dz4403nuZVykkikiIiyUA5EUn2XIA9wDcOhxjwUoI8KWxWKxaXwNpdmhQq3zPGNChkObuIh9C6UwiqUqUKP/74I/fccw/79u2jY8eOfPZZoX9G5QcCdY5CKH5S+D3wtohUAhCRKOBl4NtiHMN9w4bnlS33Y++5xNz7jjbGbDfGpAHPAJ3sK2HKx0SEjudUZ9r97XmzfwvqVS7HPHuOw39OWMn2Q8E9aNratWtp1aoV3bt3Z/Xq1U6Ho4quF9AH68JVb4+lJ9DcGDPIwdiCQrC3FMZEhtGkRkX2pWayPzXD6XBUEBIRl4i0EZHrRKS1iBSnDqZ1pxAVERHB2LFjeeutt8jJyeHWW2/lscce01t8/JR75NFQ6D76KBAFHAQqYRU8scDTRT2AMSYFSAJaeKxuAew0xhzx2vcwsIOCb/nz6ZUs/WydzOUSeraozax/X8bIns2pWiGKKat30+k/5uxrAgAAIABJREFU83j6u3XsPxp8laYpU6bQrl07tm3bRs+ePWncuLHTIakiMsbMM8bMBerYj93LL8aYP5yOLxikHAvupBDg/NpWHXutzleofExEGgBrgZlY9/v9CKwVkSK1FPpr3UmVDRFh8ODB/PDDD1SqVInw8MAbxCRUpIVKUmiMSTfGXI81glYrrApYX2NMcZuPPgaeFJGadh/3J4APCtn3PeCf9sSvMVgJ6M/2lS+f0g/Y30WGu7ilbQLzH+nI0CubEBMRxqe/bueyl+fynx83kZqR7XSIJWaM4aWXXqJXr16kpaUxYsQIJkyYQLly5ZwOTRXfvSJyMYCIdLS7lR4QkUudDizQBftAMwAX1LGSwjVJmhQqn/sv1kBY1YwxTYFqwDR7fVH5Zd1JlZ3OnTuzZs0aRow4Mei/jkzqXwJ1jkIo/pQUj4jIWcaYA8aY5caYMx2mbSTwK7DRXhYBo+xzvCMinvPovAj8DKwGdgLlsPrJqzJULjKcIVc0Zv4jlzOwQwNyjWHM7C1c9vIc/vPjJpZvSyY7AKeyyMvL47bbbuOxxx4jJiaGr7/+mmHDhukFgsA1iBPTT4zA6jL1CPCqYxEFieT0LMpFhgX1XKbn17EmrteWQlUKWgFPuOcptH8Os9cXldadFHXr1s0f9G7WrFk0bNiQ+fPnOxyVckvPcrcUBt53ZXHT2I7ACBH5BRgHTDLGFPsShTEmGxhsL97bBnk9zwUeshflsPjykTzZrRm3X9KAN2f9wde/JTFm9hbGzN5Chahw2pxdhQ6Nq9K+cVXOrlre75Mrl8tFrVq1qFu3Lt999x0XXnih0yGpkokzxqSKSEXgAuByY0yuiLzudGCBLDs3jyPHs6kTH+N0KKXq3JoVCXcJa5KOYIzx+/JLBZTDWAO+bPBYdzYnDxxzSlp3Ut7mz5/P3r176dSpE2+//TYDBw50OqSQl5ZpDXYeCt1HrwHqAdOBfwP7RORTEelUGsGVJROSMxWeudqVYnj5un8w+6GOPNWtKZc1qUZOXh6zNu7jmSnr6fSfeVzy4mxGTt1AXp7//W4PHz7xPTxq1ChWrFihCWFw2CsilwD9gYV2QlgBCLxmbD9y+JjVTTyYu44CREeEcU7NihxMy2RfaqbT4ajgMhaYKSIPi0hfEXkY+MFer9QZGTFiBO+//z4iwt13382//vUvcnJynA4rpKVl2N1HI4M8KQQwxuw3xrxujLkIaIPVL/5Hn0fmAL0mXHwJVctzV4ezGXdHK1Y/cyWfD2zNfR0bckGdOPakZvDhgq1s2uc9MJqzvv76axISEpg7dy4AYWFhVK1a1dmglK88C8wBXudEl9HOWF2o1BkKhfsJ3U7cV6hTUyjfMcb8B3gS6IJVTnUBhhljtGu7KpG77rqLWbNmUbVqVUaPHk23bt1ISUlxOqyQ5R5rI65chMORFN8ZzcQtIrVEZCjwOdAOqyupCnFR4WG0a1iVR7qey5Qh7bmlTX0AdiT7x03QeXl5PPvss/Tt25fU1FTWr1/vdEjKx4wx47FGRq5ujJljr14E3OBcVIEvPyksF/xJ4fm19b5C9f/s3Xd4FOX2wPHvSQ8khITeIfQqKCC9KCqKgFgQFBULiqBwr/2H9drbxYIiIlewICLSVaQoINKrdKRIlZ4AKaS/vz92E2MkZDfZ3dlyPs8zT2ZnZ3dOJpvZ98zb3MMY87kx5mpjTBP7z8+tjkn5hy5durBmzRqaNWvGggULmDZtmtUhBayz521JYZkI30sKnarbFJFBwJ1AF2x3498AZhpjzrshNo/SKSlcr0asbfTOQ16QFKampjJ48GCmTZtGdHQ0U6ZMoVevXlaHpdygYD9nY8wJq2LxF4FUU5g7LYWOQKpKSkSaGmO22ddbFLafMWaz56JS/qpOnTqsWLGCzz//XPsWWig3KYyJ9POkEHgM+AK40xhzzA3xWErHFHCtGnG2QSmsTgoPHz5M37592bBhA/Hx8cyZM4emTZtaGpNyHRE5aIypaV9P5MJzc2GMifNoYH4kwT5HYWwAJIUNKkcRFhzEliM62IwqsVVAtH19UyH7GMD3hilUXik6Oprhw/8ah2jGjBn8+eefDB8+XK9lHnIut6bQ35NCY0zLovdSyqZGnK2m0Ormo99++y0bNmyga9eufPvtt9p/0P/clm/9Bsui8GMJybaksFwAJIXhIcE0qhLN5sNnOXLmPNVjdb5SVTzGmOh868XqrqNUcSUlJTFkyBASEhLYunUrY8aMITTU9xIVX/NXTaHvDTRTZMQiMtQYM86+PqKw/Ywx77syMOX7asaVIjRY+HXPKSavPsDtl9eyJI6RI0cSExPD7bffTliY/xdqA40x5td8D6saY6YU3EdEBngwJL+TGEA1hWBrQrr58Fk2Hz6rSaFyCxFpBWQaY7ZaHYvyT9HR0Xz//ff069ePjz/+mJ07d+qNcQ/w5ZpCR+5c9cm33q+QxefvzmuXQteLjgjllX7NAXh65laenrmFjCz3zwyQk5PD//3f/zFnzhwARIS7775bE8LA8HEh28d6NAo/czolcGoKAdrWsbU0XrDN73pJKIuIyGwR6WRfH45tAKxVIjL04q9UqvjatWvHmjVraNWqFUuXLqVt27Y6yJ6b+fVAM/a5CXPXu7s3HGuJTkrhcv1b1yC+fGmGfrmByasP8vvxJMbefhkVosPdcrykpCRuv/125s6dS40aNbjmmmsID3fPsZT3EJEyf61KNH+fYaYukOn5qPxHYkpg1RRe1aQSpcOC+XHbMZLSMon2wS935XXaA6vt68OxTZVzBpgFjLMqKOX/atSowbJlyxg8eDDffvst7du3Z8eOHVSrVs3q0PzSubQswkOCiAj1va7CTrVxF5EthWwvrAO1UrSuHcfchzvSonoMa/cn0ueDX90yB9j+/fvp0KEDc+fOpUGDBixatEgTwsBxBkgESuVbz13WAGOsC833BVpNYamwEK5rXoW0zBzmbdHaQuUS4caYTBGpBsQZY5bbRyatZHVgyv+VLl2aqVOn8sILLzBkyBBNCN3EGMPZ85k+OfIoOD9PYe1CttcsYRzKz1WJieSbB9pzY6tqHD2bxi3jVjJr4xGXvf+yZcto06YNW7du5aqrrmLVqlU0aNDAZe+vvF4dbDWCp4D4fEttINoY87J1ofm+hJR0goPEJ5vDFNdNl1UH4Nv1hy2ORPmJ7SLyf8CzwHwAEakIpFgalQoYQUFBPP/887z99tt523755RfS09MtjMq/pGRkk51jfDYpdGhoHBEZbV8NzbeeKx7Y59KoLKDzFLpfRGgw/+1/CU2qluHVH3bwr6mb2PbnWZ7s2YiQ4OIPzLZr1y6uvPJKMjMzGTlyJG+//TYhIb436pMqPmPMAfuq3nV3MWMMiSmZxJYKJSgocJrYt60dR424SNbsT+Dg6VRqltMBZ1SJDAM+wNaU/W77tmuABZZFpAJS7tQUK1asoEePHrRu3ZqZM2dSqZJ+fZaULw8yA47XFMbal6B867FADLAd6O+W6DwtcMo7lhER7uscz2f3tCUmMpRPlv3B3ZPWcsY+umFxNGzYkIceeojx48fz7rvvakIY4ESkvYj8n4j8V0RG5y5Wx+WrktOzyMjOCYiJ6/MLChJubGWrLZy+QWsLVckYYzYaYzoaY7oZY/6wb/vCGHOX1bGpwBQfH8+ll17KypUradOmDZs2aU+wkjp69jwAsaV88/vSoaTQGHO3MeZu4NHcdftyrzFmlDHG52sKlWd1rl+BOQ91pGGlaJbtPkXfD5ez61iSw68/c+YMCxcuzHs8evRohgwZ4o5QlQ+xj+r3E9AW22AOdYAHgMpWxuXLElNsdz599UuuJG669K8mpMnpWRZHo3yNiDTNt96isMXKGFXgqly5MkuWLGHQoEEcOnSIjh07MmPGDKvD8mmr9iUAcFmtWIsjKZ4ik0L7SH65PhORMhda3Bij8lO1ypVmxrAO9GxamQOnU+k3djk/bi16UIfdu3fTrl07evfuzbp16zwQqfIh/wKuNcb0A87bf94CaKeJYjqdYjt15aICLymsWa4UneqV58iZ81z73i+s+SPB6pCUb1mVb31TIctGC+JSCoCIiAg+//xzXn/9dc6fP89NN93Eyy9rF/ziWrn3NAAd6pazOJLicaSmMP9oIAVH9UvMt83HaadCK5QOD2Hs7ZfyyFUNSM3IZuiX63ln4e/k5Fz47/HTTz9x+eWXs2vXLq666iodTEYVVMkYs9S+bsTWeWIef59vVTkhb+L6AKwpBHh/YCt6Nq3MoYTz3Dp+JUO/WM/8bcdIz8q2OjTl5Ywx0fnWgwpZfG/ceuVXRIQnn3yS2bNnExUVhdFBNoolPSubtfsTiI4IoWlV36wrc6TzVdN863XcFYg30C6F1ggKEkZcWZ9GlaP599RNvPfTbnYcPcfoW1sSFW77iBpjGDt2LCNHjiQ7O5snnniCV199leBg/T5Vf3NMRKoaY/4E/gC6ASeBHEuj8mGnkwNrOoqC4kqH8dGgS5m16Qj/mbudH7cd48dtx6hXMYrpQzsQU8o3BxRQniUi4UCOMSYz37ZQIMgYoy0ZlOV69+7Nli1bqFWrVt628+fPExkZaWFUvmPjwTOkZ+XQuX6FEg2eaKUiozbGHMq3fiD/AqQC5/KN/KdUsV3dtDKzhnekdrlSLNh+nH4fLmf/qRSMMQwbNoyHHnqI4OBgPv/8c9544w1NCNWFfARcbl8fjW1kv43AWEdeLCKhIvKBiCTYlzEictGbZyISKSJ7RMT1k296gbyawgBNCsF2J71fq+qs+r8rGTfoUi6rFcueE8mMnLqR7EJaNShVwHygTYFtbbG1ZFDKK9SuXTtvdNJ58+bRoEED1qxZY3FUvmGFjzcdBecnr/9QRNrZ128B/gSOi8hN7gjOk7S23DvUrxTN7OGd6NKgArtPJLNy32lEhPLly1OpUiWWLFnCHXfcYXWYyksZY94xxsy0r0/GNmVOC2PM8w6+xTNAJ2wtJJoCnYFRRbzmRcBvh6fMnbg+0EYfvZCI0GB6NqvCZ/e0pV7FKJbsOslb83eRma0V0apILfh7H0Psj1taEItSRVq4cCGHDx+mS5cufPXVV1aH4/VW7j0FQId6AZIUAjcCv9nXn8I2FUVP4AUXxmQZ0fajXiGmVCgTB7fhvzfUZ2DbmgD85z//4bfffqN9+/YWR6e8mYh8n/+xMeaQMWaHiMxx8C3uAV42xhw1xhwFXgHuvcjxLgWuA14rbszeLlGTwn+ICg/h4zsuIzo8hHFL99LqxYX8e+omHaFUXcx5IKrAtiig+PMxKeVG//3vf3nvvffIzMzk9ttv55lnniEnR2+AXUhqRhYbD56hXOkwGlSMLvoFXsrZpLC0Mea8iJQHahtjZhpjfgZquiE2FcDm/ziPIde1Y8EC27y+QUFBOrGqckTnQrZ3KuqFIhILVMc2ImCuTUBNEYm5wP4hwCfYpr4osk+QiLwgIiZ3KWp/b5GgSeEF1a0QxaR72nBFo4pk5eQwc+MR3lv0u9VhKe/1M/COiIRBXh/D/wKLLY1KqUKICCNGjOCHH34gJiaGV155hZtuuonk5GSrQ/M6a/cnkpVjaF+3HEFBvlvD5GxS+IeI3IatELQYQETKone6lIsYY3j77be5/vrrSUxM1MlUlUNEZISIjABCc9fzLe8BRc918tdd/Px9A3PXL3Tr71FgszFmiSMxGmNeMMZI7uLIa7yBJoWFu6xWHJ8ObsPSx7tTOiyYicv3s+eEFpjUBT0GNAdOicgObANgXQL829KolCrCNddcw6pVq6hXrx6zZs1i5syZVofkVc6lZfKu/YZgh7rlLY6mZBwZfTS/x4BJ2JLAfvZt1wNrXRiTJXzmtr0fS09PZ+jQoUyaNInIyEgmTZpE//79rQ5L+Ybc61FovnWwjTp6HBjswHvkluZjgFP51gGS8u8oInWx3RxrVYxYfYomhUWrVCaCEVfW57V5O/nP3G18fk/bvMEalAIwxhwXkcuxDS5TEzgArDU6/r/yAY0aNWLNmjVMnjyZQYMGWR2O1zibmsmdE9fw26EztKpZlhtaVbU6pBJxKik0xiwEqhXYPNW++DzRSSksc/z4cW688UZWrFhBtWrVmD17NpdddpnVYSkfYYzpDiAibxtjHivmeySKyGFsAz/stW9uCRwyxpwtsHtnoAKwzV74DwPKiMgxoI8xxm+Ga0tIySAqPITwEB3t92Lu7liHqWsPsWz3KR6btplX+jUjIlTPmfqLMcbYrzHiT9cIFRhiY2N56KGH8h5PnTqVlJQU7rnnHgujstbQL9fz26EztKkdy6eD21AqzNm6Nu/idPQiUgsYiK3vzWFgik5JoUpq1qxZrFixgrZt2zJr1iyqVKlidUjKN70nImWNMWfsfXaGAZnAOGOMI6OATASeFpHl9sejgAkX2G8q8GO+xx3sr20JnC529F4mMzuHc2lZ1IjTeaqKEhYSxAe3Xco9k9YyfcNhfjt8hmuaVuK65lVoWvUfXVJVgBGRKsDXQEcgDYgSkf7AVcaYIZYGp5STEhMTGTJkCElJSWzbto0333wz4KYJ23MiiZX7TlOnfGkm3d2W0uG+nRCC81NSXAlsB7oDwdgmht4mIj1cH5oKJPfffz8TJkxgyZIlmhCqkpjBXwNfvYFtNNF7gHccfP1LwEpgh31ZAbwKICLjRGQcgDHmvDHmWO4CJNg2m2P5J6f2dX+NPBpucSS+oUnVMsx5uCNta8ex50QyHy7eS6/3f2X45A0cSki1OjxlrY+A1dj6LudeI34CrrQsIqWKKTY2ljlz5hAXF8fo0aPp3bs3Z88WbFDj32Zv+hOAmy+r7hcJIdiaMDi+s8g64DVjzPR8224EnjHGXOqG+EpMRBxqst/qxQWkZeaw46WeHohKGWN45ZVXaNy4MTfd5PPTXCoPExEuNFiLiCQCcfZmWn8C7bH1FdxqjPGquw2OXpustPPYOXq+u4zuDSsw8e62VofjM4wx7D2ZzJJdJxn/yz5OJKVTKiyYp65txKDLa/n06HTq4i5ybToJVDHGZIlIgjEmzr79rDHG66qSfeH6pKy3d+9e+vTpw/bt22nUqBFz586lXr16VofldsYYur61hIMJqSx7ojs14kpZHZJDCrs+5XJ29NG6QMFhh2ZhmyDamaBCReQDEUmwL2Psw7tf7DWRIrJHRM5cbL+S0HEBPCM1NZXbbruNZ599luHDh5OaqnfQlcvkAGEi0gI4Z2/ansA/5wdTDkhI1prC4hAR6lWM5r7O8Sx9vDsPda9HelYOz83exl0T12itYWA6C8Tl3yAi1bENhOUQby07qcBVt25dVq5cyXXXXcfOnTu5/PLLOXHihNVhud2mQ2c4mJDKpTXL+kxC6Ahn6zv/AHoDs/Nt62Xf7oxnsM0b1tT+eB62vjsvXuQ1L2Lrw+jb470GuCNHjtC3b1/Wr19P7dq1mTNnDqVK+c8/lLLcUuAboBx/3cCqB/j/t5QbJKTmJoWhFkfiuyLDgnnsmob0bFaZf0/dxLLdp+j61mK6NaxIi+ox3N2xDjGRen4DwFfAZBF5BBARaQCMxjaiu6O07KS8TpkyZZgzZw5PPfUUISEhVKxY0eqQ3C636WjflgXH3vRtztYUPgVMEZHvReRDEfkeW8fpp5x8n3uAl40xR40xR4FXgHsL21lELgWuA15z8jjKi6xZs4Y2bdqwfv16OnfuzJo1a2jevLnVYSn/ci+wFVgEvGzf1gB437KIfFiC9il0mWbVYpj7cCf+1aM+saXC+HnnCd5dtJtHpupcrAHiRWAjtn7KMcB6bP2W33TiPbTspLxScHAwb731Fq+++mretmXLlpGZ6Tdd7PNkZefw3eajBAcJvVp4Va+UEnMqKTTGLACaAcsBsf9sYYyZ7+h7iEgstpFL838TbgJqisg/2tXbm0Z8gm1OsHQH3v8FETG5i6Nxact599q/fz9du3bl6NGj3HvvvSxatIgKFSpYHZbyM8aYRGPM0/aJ4lPs2743xrxndWy+6K+kUGuyXCEiNJh/9WjA8qeuYNrQ9lSJieCnnSfYfTyp6BcrnyUiwcADwHPGmGigIlDGGPO4g6Mie23ZSan8cudnXbJkCVdccQU9e/YkISHB4qhca+W+05xKTqdTvfKUj/KvG6YOJYUiEiEiL4vIbGAQ8JYxZpgx5lVjzN6iXl9Abt+e/O3bc9ejL7D/o8BmY8wSR97cXhiU3MWZwLRLofvUrl2bESNG8M477/DJJ58QFqYTYSv3EpFzVsfg67Sm0D0iQoNpUzuOezvVAWDskr3ooB7+yxiTDbxijEmzPz5VjFFcvLbspFRBderUoWnTpvz888+0bduWHTt2WB2Sy/zVdNS3J6q/EEdrCt8FbgR+xzZHYUmaIiTbf+a/s5W7/rfbpSJSF9tdrmJNRq2slZyczMKFC/Mev/HGG/zrX//Ku5OklJvpB62EtKbQvQa0rUlMZCgzNx5h2OQNpGVmWx2Scp/5JZy+S8tOymfUqlWLX3/9lX79+rF3717atWvHvHnzrA6rxDKzc5i/9RjhIUFc3bSy1eG4nKNJ4fVAT2PM49gGlrmhuAc0xiRi6/TcMt/mlsAhY0zBSU46AxWwzYV4DNscZGVE5JiIuHR8dL1J61oHDhygU6dO9OrVi+XLlxf9AqWU19GaQveKCg/h08FtqBEXybytx5iy5qDVISn3SQJmicgMEXlXREbnLo682FvLTkoVJioqim+//ZZnnnmGc+fOcf311/Puu+9aHVaJHEk8T1J6FpfUKEuUn8xNmJ+jSWEZY8xBAGPMPiC2hMedCDwtIpVFpDK20bMmXGC/qUAdbBe+lsB92C6sLbF12HYprcFyjeXLl9OmTRt+++03OnfuTOPGja0OSQWma60OwNflJYWltLm3u1xWK5aPB7UGYNKK/WTn6B1KP1URmIZtaooYbOWo3MVRXll2UqowQUFBvPTSS0yZMoWwsDDS04vs3urVDieeB6BGrH+Omu9wmisi0fzVHCuowGOMMc7033kJ25DxuY2MJwOv2o8zzv5+Q40x54Hz+WJIsD1ljjlxLOVBkyZN4oEHHiAjI4Phw4fzzjvvEBqqTc+U5xljfrU6Bl+XkJJBSJBQJtL/7oh6kyZVy9AuPo5V+xLo//FKrmxckQe61CVYJ7n3eSLSGtt8zlWB/UAfY8zWYr6dlp2UTxowYABt2rQhPv6vac3T09MJD/etViiHE21zzFaPjbQ4EvdwtKYwCluH5kT7EpPvce5PhxljMo0xw40xsfblodwRuOwXtKGFvG6JMaasM8dSnvPEE09w9913k52dzdixY/nggw80IVSWE5EwEdlndRy+xhhDYmoGsaXDtBWFBzzQtS4A6w8k8uaPu7jtk1UcPK2T3PuBt7DNndocmINzU1D8jZadlC+rW7du3nfJ7Nmzady4MZs3b7Y4Kucc0qQQsDVDiM+31LnAuk/Tkd9KLiYmhri4OBYuXMiDDz5odThK5RKgttVB+Jqk9Cwys402HfWQ7g0r8vOjXZn+YHsaVynD6j8S6PX+MvafSrE6NFUyLYCnjDHbsDX3bFnE/kr5ve+++44//viDDh06MHv2bKvDcVhe89E4/2w+6lBSaIw5UNTi7kA9Qe+FOy85OTlvfdSoUWzZsoXu3btbGJEKRCKyobAFWIVOReq0xLxBZjQp9JT4ClFcViuOWcM7cEe7WiSlZ/HYtN+0n6FvCzXGZAAYY1KBCIvjUcpy48eP5+WXXyYlJYV+/frx2muv+UTlTG5S6K81hdpRRBXb4sWLufXWW5k4cSK9evVCRKha1f/mbVE+oSHwMvDnBZ4LA8Z5Nhzfd1qTQsuEhwTzXO8mbDyUyLoDiXy7/hC3tqlpdViqeMJEZES+xxEFHmOMed/DMSllKRHh6aefpkmTJgwaNIhRo0axdetWJkyYQGSk9yZchxJSCQkSKpfxz3s7mhSqYhk3bhwPP/wwWVlZrF69ml69elkdkgpsm4DdxphvCz4hIuHAx54PybdpTaG1QoODeKlvM/qNXcGT07ew42gSiakZ1KsQxfQNhykVFkLDytGkZWaz63gSbWvH8dqNzbX/p/dZBfTL93h1gccG0KRQBaR+/fqxYsUK+vTpw1dffUW/fv24+eabrQ7rgtIyszmRlE6NuEhCgh3tfedbNCm08/5Ka++QmZnJv//9bz788ENCQ0P59NNPufvuu60OS6l3gYRCnssE9EPqpNyawlhNCi3TqmYsvS+pytzf/mTSiv3/eH770b8G/d53MoUZG47Qq0UVHuxWl/oVozRB9ALGmG5Wx6CUN7vkkktYu3Yt33zzjdcmhAB/nrE3HS3rn/0JQZPCv9Pvz4tKTEykf//+LFq0iIoVKzJjxgw6duxodVhKAZQzxkzLfSAikfZh2THG5ACfWRaZj8qdo7CcJoWWeqVfM7rUL4+IsPXIWX7ceoxnrm9Mm9px7D2RTHhoMCfOpTHsqw1kZOcwc+MRZm48QlhIEBWiwulYrxxXN6lMp/rliQgNtvrXUUqpf6hYsSIPPfRQ3uPJkycTGhpK//79LYzq7w7lDTLjvc1bS8qppFBEQoFngNuBCsaYGBHpCdQ1xnzojgCV95gzZw6LFi2iRYsWzJkzh1q1alkdklK53uTv/QaPAHEWxeIXErWm0CuUiQjlltY1ALj5suq80Kdp3nOV8vVr+WFEZ4JE+HHrMeZvO8bxc2kcPXueb9Yd5pt1h4kMDaZLg/Jc1aQyVzaqqH9XpZRXOnHiBEOGDOH8+fNs27aN559/nqAg65tr/jVHodYU5noT2/DKQ4Hcvjs77Ns1KfRzd911F5mZmQwYMICoqCirw1Eqv4L1/FrvX0KntabQpzSuUgaAhpWjGdmjPgBnUzNZvOsEC7YfY+muk8zfdpz5244TJNCmdhxXNanE1U1cUHoAAAAgAElEQVQqU7Oc/xZylFK+pWLFinz77bcMHDiQF198kW3btvHZZ59RunRpS+Py95FHwfmk8BaguTEmUURywDZdhYj4/rBoBi1GFmCM4f3336dixYoMHDgQgPvuu8/iqJS6oILdgrWbcAnl1RTqPIU+K6ZUKDe0qsYNraqRlpnNyn2nWbDtOIt2HGf1Hwms/iOBl7/fQaPK0VzdpBJXNalMs2pltC+iUspS1113HStXrqRPnz5Mnz6dvXv3Mnv2bGrWtC7dOJigNYUFCZD6tw0iUUCSyyKykH4N/iUjI4Phw4czYcIEYmNj6dWrF2XKlLE6LKUKo8O+u1heTWGUJoX+ICI0mO4NK9K9YUVeyWnGb4fPsGD7cRZuP87OY0nsPJbE+z/voUpMBFc1qcS1zarQLj5OE0SllCWaNGnC6tWrueWWW1i8eDGXX34527dvJzY21pJ4Dp62pT+1/LhlhbNJ4WLgJeCJfNueBBa6LCJluZMnT3LTTTexbNkyqlSpwuzZszUhVN5Oh313scRUW1JYtlSoxZEoVwsKElrVjKVVzVie7NmIfSeTWWhPENcfTOTzlQf4fOUBGlSKYnCHOvRrVY3IMB2kRinlWeXKlWP+/PmMHDmSChUqWJYQAhw4nUJEaBAVo8Mti8HdnE0K/w3MEZGTQBkROQIcBHq7PDIP09ajNlu2bKFPnz7s37+f1q1bM2vWLKpVq2Z1WEpdlA777noJyRlEh4cQHqLJgL+LrxDFA12jeKBrXU4lp/PTjuNMWXOITYfOMGrmFt6cv5MBbWpyZ/taVC3rv/1plFLeJzQ0lLFjx2LMX71Cfv31V9q3b09wsGe+n86mZnIuLYsGlfx7qh+nhvMxxpw0xrQHrgMGADcCHY0xp9wRnPKso0eP0qFDB/bv38+tt97K0qVLNSFUKgBlZOWQlJ6lI1QGoPJR4dzapiazhndk5rAO9G1ZleS0LMYt3UvnNxczbPJ61u5P+FsBTSml3C03GVuwYAFdu3alb9++nDt3rohXucaBhBQAasb5b9NRKOY8hcaYtcBaF8diOX/O/h1RpUoVRowYQalSpRg1alTAnw+lAlVu09E4TQoDWm4T01HXNebLVQf4avVBfthyjB+2HKNZtTIM7lCH3pdU0dpkpZTH1KlTh/r16/P999/ToUMH5syZQ3x8vFuPuenQGQBqxlk7Aqq7iTN3+0RkMYWM6meMucJVQbmSiBhHfsdmz88nOEj47fmrPRCV90hLS2PZsmVcddVVVoeilMNEBGOMT9+1cPTaZIUdR89x7XvLuLJRRf43uI3V4SgvkZaZzdzf/mTi8v1sP2q7Qx9XOow65UsTHRFCdESo/WcIZfKtlwoLITI0mFJhwUSGBVMqLIRSYcHERIYSEepfCaU/XJvAu69PSp09e5YBAwbw448/Uq5cOaZPn07Xrl1dfpzsHMOHi/fwzqLfMQYm3NmaHk0qufw4nlLU9cnZmsJZBR5XAe4AJjn5Pl7HdvHz+eu4U44ePcoNN9zA+vXrWbhwId27d7c6JKWUF0jQievVBUSEBnNL6xrcfFl11vyRwKQV+5m/7RjrD2QU6/2CBJpXi6F93fK0r1uONrVjKRVWrAZMSqkAEhMTw9y5c3niiSd455136NGjB2PHjmXIkCEuO8bZ85mMmLKRpb+fpFRYMK/f1MKnE0JHOHX1Nca8V3CbiHwNvOayiCwUSK0l169fT9++fTly5Ajt27enSZMmVoeklPISCTpxvboIEeHy+HJcHl+O1IwszqRmkpSWRVKa7ee5tNzHtm2pGdmcz8gmNTOb8xlZpGZkk5qRzZ9nzvPb4bP8dvgs45buJSRIaFmjLHd2qE2fS6pa/WsqpbxYSEgIo0ePpmnTpjz44IOkpKS47L33nUzmvs/Xse9kCg0qRTH29suoVzHKZe/vrVxxS24L0NEF76M8ZNq0adx1112cP3+eu+66i48//pjwcP8dYlcp5RytKVSOsjUFLV5RwhjDvlMprNx7mpX7TrNq72nWHUhk3YFEcnIMN7TSgc6UUhd377330qlTJxo0aJC3LSMjg7Cw4n1/JaRk0PeD5SSlZ9GjcUXeHdCKqPDAaMHg1OijItKiwNIO29xfe90TnnK1l156if79+5OWlsbbb7/NxIkTNSFUSv1NblKoA80odxIR6laIYlC7Wnx426Wse6YHY2+/lOAg4bFpv7F45wmrQ1RK+YCGDRvmDY44bdo0WrRowa5du5x+nx1Hz3HDh7aEsG3tOMbf0TpgEkJwMikENgEb7T83YZu0vhlwl4vj8rhA6U4dGRlJdHQ03333HY8++qiOMKqU+oe8pLCUJoXKc0SE65pX4c2bWpCVY3hw8nrW7U+wOiyllA+ZMWMGu3btol27dixcuNDh132xcj99P1zOwYRUujSowNhBlxIUFFhlZGeTwjJAiDEmyL5EG2O6GmM2uyM4T/PXP33+dtaPPvoo27dv57rrrrMwIqWUN0vInZIiSpNC5Xk3XVadZ3o1Ji0zh3smrWXnMc/MRaaU8n2TJ0/mqaee4syZM1x77bWMGTOmyHlVt/95jmdnbyMzO4enr2vMpMFtKB8VeK3oHE4KRSQIOIlr+iEqD1m1ahX169dn5syZgO1ObPXq1S2OSinlzRKStaZQWeu+zvEM61aXc2lZ3Pm/NWw+fKbIgp1SSgUFBfHaa6/xxRdfEBISwogRIxg6dCgZGYWPklynfGmqlY3EGGhRPSbgaghzOZwUGmNygD1AnPvCUa705Zdf0q1bN44ePcqyZcusDkcprycioSLygYgk2JcxIvKPG2EiEi4in4jIHyKSJCI7ReQeK2J2h7zmo1pTqCz0+DUNGdi2BieS0unzwXK6vLWY/8zdxsq9p0nLzNYkUSlVqEGDBrF06VIqV67M+PHjWbBgQaH7RoYF09s+4vELc7dzIinNU2F6FWdr/cYB00XkDeAQkJP7hK83IfWn75acnBxGjRrFG2+8QXBwMGPGjGH48OFWh6WUL3gG6AQ0tT+eB4wCXiywXwhwFOgB7AMuB+aJyGFjTOHfPD4iITWD0GAhOoA62CvvIyK8fENz6laIYu5vf/Lb4bNMXL6ficv3AxAcJJQKC6Z0WAg9m1Xm6V6NCQ12tleMUspfXX755axdu5ZZs2Zx/fXXX3Tfa5pW4pt1h9hx9BwDx6/ip0e7eSZILyKO3GkTke+NMb1EJKeQXYwxJti1obmGiBhHfsfGz/5IZFgwG569ygNRuU9SUhK33347c+fOpWzZskybNo0ePXpYHZZSLiUiGGNc3r5DRA4B/zbGfGt/fAvwtjGmlgOvnQFsNcY85+CxHLo2eZoxhvpPzyOudBhrntZrh/Iex86msWjHcRbtOM6B06mkZmSRmp5NSkYWOQY61C3HR4MuIyYy1LIY3XVt8jRvvT4pVVKTJk2ifPnyF0wSTyWn0/rlRUSFh7D1P9dYEJ17FXV9cvQ2cGcAY4zegvNyP/zwA3PnzqVBgwZ5P5VSRRORWKA6tpGVc20CaopIjDHm7EVeGwG0Bb66yD4vAM+7Jlr3OZeWRVaO0ekolNepHBPBoHa1GNTu7/do/jxznnsmrWXF3tPc9NEKJg5uQ424UhZFqZTyVocOHeKBBx4gMzOT119/nccffzxvFP6ElAyGfbkBgLoVSlsZpmU0yfMzt956K+PHj2f16tWaECrlnCj7zzP5tuWuRxf2IrF9o0wAdgMzCtvPGPOCMUZyl5IG6y46R6HyNVXLRvLtgx3o3rACe04k0/uDX3nthx3sOKqjliql/lKjRg0mT55MREQETz75JIMHDyYtLY2ktExu+HA5a/Yn0LxaDOPvbG11qJZwtKYwTEQe5iKzNhhj3nf0oCISCrwD3GbfNBlbk62sAvuFAx9g67dTHjgCvGmM+dTRYznK+PBMhRMmTCA0NJS77rJNFzlkyBCLI1LKJyXbf8YAp/KtAyRd6AX2hPAjoCHQwz4gl0/TpFD5oqjwED65szWv/LCDSSv28/Ev+/j4l33UqxhFrbhSlC0VRtWyEdx+eS0qx0RYHW6xeGPZSSlfc/PNN1O3bl369OnD559/zo6dv1PpxlEcTAyifsUovnmgPZFhXtkjzu0cTQpDgBsv8rwBHE4K8dLBHLz21n0hsrKyeOyxx3jvvfeIiori+uuvp1y5claHpZRPMsYkishhoCWw1765JXDoQk1H7Qnhh9iajV55sealvkSTQuWrQoKDeL53Ux7oUpc5vx1hxoYj7DyWxJ4TyXn7TFj2Bw90jef+LvGUCvO5gZS8suyklK9p1aoVPy1dQbeevVi7ZhXBO++l6cMTeKVf84BNCMHxpDDVGNPdhce9B9vdraMAIvIK8DYFLmzGmBQg/6ANq0RkMbaLYkBf2M6cOcOtt97KggULKF++PNOnT9eEUKmSmwg8LSLL7Y9HYWsaeiEfAB2BK4wxiZ4IzhMSNSlUPq5yTAT3d6nL/V3qcvxcGqeTMzhzPoNffj/Fp7/+wbuLdvP95qPMfbgTEaE+VQDUspNSJZCSnsW0dYdY8vtJluw6SWif/1D6xzG0bdmcmaOuJaaUdYNUeQOP9yksajCHIl6bO5hDodNfiMgLImJyF0fj8qVBtn7//XfatWvHggULaNasGWvWrKFLly5Wh6WUP3gJWAnssC8rgFcBRGSciIyzr9cChmFrNnpARJLtyzhrwnad05oUKj9SqUwETaqWoUPd8jx1bSN+erQrl9WKZfeJZL5cdcDq8BzmrWUnpXxFdo7hvs/W8cLc7SzZdRKA2pXKsnr+TH764t28hHDFihUBOweqozWFrmxZWdRgDhdsguXMYA7AC/le5/BfVnyg/eipU6do164diYmJ9O7dm8mTJxMdXegYGEopJxhjMoHh9qXgc0PzrR/A91qcOyQxVZNC5b9qxJXitRub0/PdX/hw8R76t6lBmQifqB3w2rKTUt5s5d7TPDxlAxWiI9hx9ByNKkfzYt9mtKpZ9h/zmn733Xf06dOHW265hYkTJ1KqVGCNYuxQTaExxpVZR/7BHCiw7shgDjf4w2AOxVW+fHlGjhzJk08+ycyZMzUhVEq51Olke1JYSpNC5Z8aVIrmxkurk5iaySe/7LM6HEdp2UkpJ+08do6Bn6ziVHIGO46eo1XNsowbdBlt68T9IyEEqF27NrVr1+abb76hS5cuHDlyxIKorePx5qP2vje5gznkcnQwh6v9ZTAHZ2RkZPDTTz/lPX7uued4/fXXCQ72qb4QSikfkFdTGKVJofJf/76qAWEhQUxY9gcnktKsDqdIWnZSynnL95zOW7+/SzwzHuxA7fKFz0GYv0vW+vXrad26NatXr/ZEqF7BqnkKcwdzqCwilXFsMIer3DmYg7e2kzh16hRXX301V199NfPnzwfIm2hTKaVcLa9PodYUKj9WrWwkd7arxfnMbKasPmR1OI7yurKTUt4qNSOLJbtO5D1+4pqGDpWfy5cvz8KFCxkyZAjHjh2ja9eufPXVV+4M1WtYlRR66WAO3pVsbdu2jbZt27J06VIuueQSmjRpYnVISik/lzv6aKz2KVR+rm/LagCsO5BgcSQO89Kyk1LeJTElg9s+Wc2y3adoWCma1aOuJOQCzUULExYWxscff8x7771HZmYmZ88GRkW7JZP06GAORfvuu++47bbbSEpK4uabb2bSpEmULl14lbdSSrlCQkoG0REhF+xvoZQ/aVQlmvCQIDYdOkNOjiEoyLuLG1p2Uuri0rOymbbuMGMX7+HPs2lcViuWT+9qU6ypJkSEESNG0KNHj79VymRmZhIa6hODUzlNv/W90OjRo+nTpw9JSUk8//zzTJ06VRNCpZTbpWdlk5yeRTmtJVQBIDQ4iBbVY0hKy2LfqRSrw1FKFVNaZjafrdhP1zeX8Mysrfx5No3rW1Thy3svL/Hcg/kTwsmTJ9O6dWv2799fwoi9kyU1hV7JizoVhoSEEBERwWeffcYtt9xidThKqQCRmJIJaNNRFTha1ijL2v2JbDyYSL2KUUW/QCnlNc6ez2T6+sOMW7qXE0npAFzdpBIjrqxPs2oXnb7TacYYpk6dyubNm2nTpg0zZ86kU6dOLj2G1bSmMB8rx29JTU3NW3/44YfZsWOHJoRKKY86nWL7UtWaQhUoWtWMBWDjoTNF7KmU8ia7jiXR6Y2fefG77ZxISufaZpX5fkQnxt/Z2uUJIdiak86YMYMRI0Zw6tQprrjiCj799FOXH8dKmhR6gY0bN9K4cWOmTp0K2D54tWrVsjgqpVSgya0p1InrVaBoVbMsABsPalKolC8wxvDj1mPcM2ktSWlZxFcozY//6sxHgy6jaVXXJ4P5hYSE8N577/Hxxx9jjOHee+/lkUceISsry63H9RRNCi02ffp0OnXqxMGDB/82F6FSSnlabk2hNh9VgaJKTCSVy0Sw69g5UtL9o2CnlL/aeuQsA8avYuiX6zly5jx9W1Zlwb+60KhyGY/Gcf/997Nw4ULi4uJ45513WLp0qUeP7y7ap9DOeLhToTGGl19+meeeew4R4bXXXuPJJ5/0aAxKKZVf7nQU2nxUBZKWNcry47ZjbDlylnbx5awORylVwPFzabw1fxfTNxzGGGhQKYpnejWhS4MKlsXUrVs31q5dy7x587jyyisti8OVNCnMx1NdClNTU7n33nv5+uuviYqK4ssvv6Rv374eOrpSSl1YQu4chTpxvQogrWraksKNB89oUqiUl5m96QhPTt9MWmYOcaXDeOSqBgxoU8OpeQfdJT4+nuHD/5ohZsKECcTHx3PFFVdYGFXxaVJogYULF/L1119Tu3Zt5syZQ/Pmza0OSSmlSEi11xRGaVKoAkfuYDPrDyQAda0NRimVZ/fxJEZ+vQmAHo0rMvrWlpSJ8M45Avfs2cPQobbpQseMGcODDz5ocUTOsz7N9hLGg61H+/bty/jx41mzZo0mhEopr6E1hSoQNa8WQ1R4CIt2nGDMT7sxniwQKKUu6HBiKg9O3pD3+OleTbw2IQSoV68e//vf/wgODmbYsGEMHz6czMxMq8NyiiaF+bhzSoopU6YwYcKEvMdDhgyhQgXr2kIrpVRBCXl9CsMtjkQpz4kMC2bcoMsoHRbMfxf+zqiZWzh73rcKc0r5iz0nkhj+1Qa6vbWEPSeSaV4thiWPdaNO+dJWh1aku+66i8WLF1OxYkXGjh1Lz549OX36tNVhOUyTQjfLycnh6aef5rbbbuPhhx/m6NGjVoeklFIXlFdTWNp778Yq5Q6d6pfn6/vbUz4qjClrDtHlzcVMWv6H1WEpFTCOnU3jrfk76TH6F77ffJSsHMOIK+ox/cEO1PaBhDBXhw4dWLt2LZdccgk///wzHTp0IC0tzeqwHKJ9Ct0oOTmZO+64g1mzZhETE8M333xDlSpVrA5LKaUuKCElk7DgIKLC9atBBZ7m1WP4fkRn3l20m2/WHeKFudtpVKWMDj6jlJvk5BhW7D3Nl6sOsHDHcbJzDGHBQdzRvhYjrqxPTKRv3qCsWbMmv/76K3fddRcdOnQgIiLC6pAcot/8dq7uQXDgwAH69OnD5s2bqVevHnPnzqVRo0YuPopSSrlGTo4hMTWD8lFhiDvb0ivlxSqVieC1G5tzWa1YHpv2GxOW7dOkUCkXO5uaybT1h/hq9UH2nUoBbFMh9W9Tg0HtalGtbKTFEZZcVFQU06ZN+9v36erVq2nbtq3XfsdqUpiPuGhSinPnztGuXTuOHTtGjx49mDp1KnFxcS55b6WUcoektCyycwxx2p9QKfpcUpW35+9i0Y4T7DmRTL2KUVaHpJRPy8kxrPrjNNPWHeaHLUdJz8oBoHWtWO5oX4uezSoTHhJscZSuFRT0Vy+9GTNmcNNNN3H33Xfz0UcfER7ufd+1mhS6QZkyZRg5ciRHjhzhnXfeISRET7NSyrudTkkHIE77EypFWEgQgzvW5vV5O/nfr/t47cYWVoeklE86evY83647zLT1hzmYkApA6bBgbr68JoPa1aJxlTIWR+gZNWvWpGrVqkycOJHff/+dGTNmULFiRavD+hvNVlwkOzubpUuX5k1Y+eSTT3pt9bBSShWUaJ+jUGsKlbIZ2LYmY37azfQNR3jkqoZUiNb/DaUckZGVw6Idx/lm3SF++f0kOfY+Wm1qx9K/dQ16tahCqbDASkFat27N2rVr6devH8uXL6dNmzbMnTuXFi2854ZTYP1FLqIk8xKdPXuW2267jXnz5jF79mx69+6tCaFSyqecTs6djkLnKFQKICYylEHtanHsXBoZ2TlWh6OU19t1LIlv1h1i5sYjeaNZV4gO56ZLq9O/dXXiKwR2M+yqVauyZMkS7r33XqZMmUKHDh348ssvueGGG6wODdCk8G+Kk8ft2bOHPn36sGPHDpo0aUKTJk1cH5hSSrmZTlyv1D89dW0jvcmr1EWcS8tk7m9/8s26w/x26AwAIUHC1U0q0b91Dbo1rEBIsM6AlysyMpLJkyfTvHlznn76aa+ax1CTwhL4+eefueWWW0hISOC6665jypQplCkTGG2jlVL+JSG3+WiUJoVK5dKEUKl/Msaw5o8Epq49xA9bj5KWaatJj69Qmltb1+DGS6trc+uLEBH+7//+j969e9OsWbO87VlZWZaOQ6JJYTF9/PHHDB8+nOzsbB577DFef/11goP9a9QkpVTgSLA3H43TmkKllFL5GGNIycjmdHI6q/adZuLy/ew8lgRAqbBgbm1dg/5tqnNpzVi9keKE/Anhp59+yrhx45g1axZVq1a1JB5NCu2c7VFojCE4OJgJEyYwePBgd4SklFIek1dTqH0KlVIqoGRl53AwIZV9J1PYdyqZfSdTOHo2jYSUDE4np3MqJYOMrL/3q21RPYZBl9eiV4sqlA7XdKIkjDFMnjyZtWvX0qZNG2bPnk3r1q09Hof+FfMp6t5GWloaERERAAwdOpSrr76a+Ph49wemlFJultunUJNCpZTyTwkpGew7aUv69p5KZu8JWxJ48HQqWTkXrh4RsbUgiYsLo1xUGDViSzGgbQ2tFXQhEeGHH37gwQcfZOLEiXTu3JmJEycyYMAAj8ahSaGDduzYQZ8+fXjuuee44447ADQhVEr5jURNCpVSym+cOJfGtxsO22r/Tiaz71QKZ1IzL7hv+ahw4iuUpm6F0sSXjyK+Qmmqx5aiXFQYsaXCCA7S5M/dwsPD+d///kezZs14/PHHGThwINu2beM///kPQUGeGahHk0K7i81IMW/ePAYMGMC5c+eYN29eXlKolFL+4rQ9KSxbSievV0opX3cuLZM3f9yV9zgsJIiGlaLtyZ8t8YuvEEWd8qWJidTrvjcQER555BEaN27MgAEDePnll+nZsycdO3b0yPE1KcynYDW4MYZ3332Xxx57jJycHJ599lleeOEFa4JTSik3SkzJICYylFAdOlwppXxezbjSPHd9k7wksGrZSK3x8xHXXnstq1atYvHixR5LCEGTwkKlp6czbNgwPv30UyIiIixp26uUUp6QlplNSkY2dcpHWB2KUkopFwgLCeKeTnWsDkMVU+PGjWncuHHe4/Hjx9OsWTM6dOjgtmPqLeFCLF26lE8//ZQqVarwyy+/aEKolPJbiTryqFJKKeWVtm7dyoMPPkj37t357LPP3HYcS5JCEQkVkQ9EJMG+jBGRC9ZaOrOvK1199dWMHz8+b3hYpZT/84Vrkzucts9RGKtzFCrltQL1+qRUoGvWrBljxowhOzubwYMH8/jjj5Odne3y41hVU/gM0Aloal86A6NcsG+JJGxfzrhx4/IeDxkyhGrVqrnjUEop7+SV1yZ3y60pLKc1hUp5s4C8PimlYNiwYcyfP5/Y2Fjefvtt+vbty7lz51x6DDEXG3bTTUTkEPBvY8y39se3AG8bY2qVZN9CjmWK+h2NMcR1G8yZXz4nJCSEPXv2UKuWQ2+vlLKAiGCMcXmPeU9fm16cu811wZfAH6dS+HnnCYZ2rctT1zayOhylfJa7rk329/aqspNSyvN2795Nnz592LlzJ5dccglr164lNNSx0WOLuj55vCmBiMQC1YFN+TZvAmqKSIwx5mxx9s33mheA5x2NJy0tjfvuu48zv0wmKCyCb77+ShNCpQKQFdem//36h+t+AReoVa6U1SEopS7A28pOSilr1K9fn5UrVzJw4EB69erlcELoCI/XFIpIDeAgUMEYc8q+rQJwAqhhjDlcnH0vcrxC73YdPXqUG264gTVr1lCxSjVe+egL7uvbvYS/oVLK3dxxN96Ka9O8LX+68lcokciwEDrULadTUihVAm5sxeA1ZSellPVycnIQkbzp9NatW0fr1q0v+hqvqykEku0/Y4BT+dYBkkqwr1NSUlJo164dBw8epEOHDsyYMYNKlSqV5C2VUr7N49emns2qOBmiUipAeUXZSSnlHYKC/rqB+/XXXzNw4EAefvhhRo8eTUhI8dI7j98SNsYkAoeBlvk2twQOFWzS4My+zipdujQjR47k7rvv5ueff9aEUKkA5y3XJqWUKkivT0qpwlSvXp3y5cszZswYrr32WhITE4v1PlYNNPMicD1wnX3TD8AsY8yLJdm3kGPlNYHIycnhl19+oVu3boBtgBn7PsX9VZRSFnBjEy1Lrk1KKf/g5oFm9PqklLqg/fv306dPH7Zs2UL9+vWZM2cOjRr9feC4oq5PVnUeeQlYCeywLyuAVwFEZJyIjHNkX2ekpKTQv39/unfvzvTp07EfSxNCpVR+Hr82KaWUg/T6pJS6oNq1a7NixQr69u3L7t27adeuHfPnz3fqPSypKfQkETEHDhygb9++bNq0ifj4eObOnUuTJk2sDk0pVUzuvBvvKXonXin/4w/XJtDrk1K+Kicnh2effZZXX32VL774gkGDBuU9V9T1KSCSwkqVKnH8+HG6d+/OtGnTKFeunNVhKaVKwB8KXgc/LQMAAA/8SURBVFroUsr/+MO1CfT6pJSv27RpEy1btvzbNk0KRQzA0KFDef/99106n4dSyhr+UPDSQpdS/scfrk2g1yel/JE3TknhcR9++CHDhg2zOgyllFJKKaWU8joBU1OolPIvvn43Xq9NSvknX782gV6flPJXAd181Bn25hI+dTH3tZh9LV7QmD3FF2P2FBFZYozp5u7XFrVvYc87s73gtgs8dvvnwBPns7jnsrDnirPNU/9TxT2f+tn0D754XnwtZl+LFzRmT3FlzFZNSaGUUkoppZRSygtoUqiUUt5vkodeW9S+hT3vzPaC24o6pjuU5JiOvrao/S72/IWeK8k2dyvuMZ15XVH7Fva8M9sLbivqmEop5Te0+Wg+IvKCMeYFq+Nwhq/F7GvxgsbsKb4Ys3I9/Ry4jp5L19LzeWG+eF58LWZfixc0Zk9xZcyaFCqllFJKKaVUANPmo0oppZRSSikVwDQpVEoppZRSSqkApkmhUkoppZRSSgUwTQqVUkp5PREZIiIrRGSJiMRbHY8vE5FQEVkuImdE5Gar4/F1ItJeRFaKyFIR+V5Eylodk1JKOSugkkL7F+EHIpJgX8aISEhJ9/WGmEUkXEQ+EZE/RCRJRHaKyD3eGm+B10SKyB4ROeOpOAsc36mYRaSPiGwSkRQR+VNEhnoyXnsMznyWq4nILBE5LSKnRGSaiFTycLwPicg6EUkXkVlF7OsV/3vKe4hIHHAf0AV4HHjd2oh8XhZwM/Cu1YH4iQPAlcaYrsBcYLjF8biUr5WdfK3c5EzMBV6jZScnadnp4gIqKQSeAToBTe1LZ2CUC/Z1J0fjCAGOAj2AMsBg4L8icrVnwsxTnPP2InDYzXFdjMMxi0hPYCzwL2znuSmwxCNR/p0z53ms/WctoA4QDrzn7gAL+BN4GfjEgX295X9PeY/LgSXGmCxjzFqggdUB+TJjc9TqOPyFMeZPY0yq/WGWffEnvlZ28rVyE2jZyVO07HQxxpiAWYBDwM35Ht8CHCjpvt4S8wVeOwN40ZvjBS4FtgHXAGd84HOxFrjfijhLEPNm4LZ8j28HtloU9wvALFd+hnTxjQV4CFgHpBf8DAChwAdAgn0ZA4Tke/424Kl8jzdb/ftYvZTkfObb74X8/2uBvLjofJYDNgDlrf59XHxufKrs5GvlpuLErGUnj8QccGWngKkpFJFYoDqwKd/mTUBNEYkp7r7uVJI4RCQCaIvtQ+0RzsZrr9b+BFtTm3SPBPnPGJz5XJQGLgPK2JuZHBORqSJS2XMRF+tzMRq4RURixNbXZSDwvfsjdZ63/O8pt7jYHc+i7nAmAvn7aeW4KUZfUpLzqf6pROdTREoB04CHjDGn3Buq5/ha2cnXyk3242rZyQO07FS0gEkKgSj7z/xtr3PXo0uwrzsVKw4REWACsBvbXS9PcTbeR7Hd8V/izqCK4EzMsYAAd2C7O1cPyAS+cGeAF+DseV4OVMRWsE4A4rAVfryRt/zvKRczxswwxswCLlRgvgd42Rhz1NiaNb4C3Jvv+dVAVxEJFpFLsV3bAloJz6cqoCTn015I/xoYY4xZ4ZGAPcfXyk6+Vm4CLTt5ipadihBISWGy/Wf+jDl3PakE+7qT03HYL2wfAQ2BG4wxnryj7nC8IlIX212uxzwQ18UU53PxvjHmgDEmGXgeuNJ+J8xTnDnPQcBCbBe3KPvyKzDfzTEWl7f87ykPceQOpzEmAfgMWAa8A/yfp+P0FY7eMRaRb4A7gedE5E3PRuk7HDyfA7ENgjRSbKPjPu7hMN3J18pOvlZuAi07eYqWnYoQMEmhMSYRW4fclvk2twQOGWPOFndfd3I2DvuF7UNszR+u9mSs4HS8nYEKwDYROYbtzlwZe7OCth4JGKc/F2eAg4C5wFuJ24IswMnzHIetk/T7xphUYxsMYQzQXkTKeyRgJ3jL/57yKIfucBpjxhljOhhjuhpj9ngsOt/j6Pnsb4yJN8a0MMY84bHofE+R59MY84Uxpqwxppt9ecujEbqRr5WdfK3cBFp2cluQBWjZqWgBkxTaTQSeFpHK9rbMo7A1Fyjpvu7kTBwfAB2Bq+wfECs4Gu9UbKM5tbQv92G7m9ES2OihWHM5c47HAyPsQxVHAs8BP9nvfHmSQzHb+7bsAYaLSIS9z8Rw4LAn+72ISIj92CFAkD2WsEJ295b/PeUZ3lC74E/0fLqWnk/fKzv5WrkJtOzkKVp2uhhPjp5j9YJtBLEPsbUPTsR2MQixPzcOGOfIvt4YM7Y7GgZIw/YllruM88Z4L/C6blg3gpYzn4tg4L/Y+p2cwjawQGUvj7kJtiYPp+37/gy08nC8L9g/n/mXJYXE6xX/e7q49bNQcHTHQ8BN+R7fDBy0OlZfWPR86vn0wDnxqbKTo/HiJeUmZ89xgdd1Q8tO7oo54MpOYn8jpZRSym3sg3GEYBvJsQXQH8gxxmSIyIvA9cB19t1/wFYwf9GSYH2Ank/X0vOplAp0Ts10r5RSShXTM9gGF8h1HliK7U73S9jmeNthf24y8Kong/NBej5dS8+nUiqgaU2hUkoppZRSSgWwQBtoRimllFJKKaVUPpoUKqWUUkoppVQA06RQKaWUUkoppQKYJoVKKaWUUkopFcA0KVRKKaWUUkqpAKZJoSqUiNQWESMiZa2OpSgi0llEDhexzzYRud5TMSmllFJK+YuC5UIRmSQi71odl3INTQoDhIgsEZF0EUnOtwyzKJZu9otKbhwHReQ1ESn259EYs8wYUz3fMZaIyL8K7NPUGPNdSWK/kHwXyWQRSRKRIyLysYiUcuI9jIi0dHVsSimllPI/Bcp1CSKyVERaWx2X8l2aFAaWJ40xUfmWsRbGcjY3DqAXcA9wn4XxuEJ1Y0w00A7oCDxlcTxKKaWU8l9P2stRlYHVwAyL41E+TJPCACcij4jIbnsN114Reegi+14lIpvt+x4XkY/yPVdXROaKyEkROSAizzha82eM2QIsA5rb32uQiOwQkTMi8quItMp3nNvzxXtERJ61b+8mImfs6/+F/2/v7mO9LOs4jr8/AQIpB8iIighmNka6iWbZYjp8mAY+QWYRjh7m0B5mrcelPZGkzLkmaRS1tVBIykixSHLNINr4Qwz6w2kaS+UUpgN5MGLKw6c/ruvUjx/nyDnHc2ye3+e1/TbOfd/nPtd9jXPv+72u73UdzgRuqiNoa+rxJyXNlDSktvOspud7RNLs+u83SPqppG31s0jS0G4+TzuwBnhnU7sfru3eKmmBJNVzD9bLNtT2Xvdy+zQiIiJag+0XgduB8ZLGqPiMpL/UWGqdpMkd10tqk/S9Go/skbRR0vh6rttxYQwsCTDjKeAcoI0yU3ezpKldXHs7cHOdDTsBWAYgaTjwAPB7YBwlIZsNfLw7DZB0CnAWsEnSmcAPgKuBMcBK4H5JIyUdCywFrqxtOAn4bfP9bH+BkmR2zIxObzq/H/g5MLehDafXtt9bk7VfAf8ETqQkq6cAX+vm80ykzH4+3nD4OeD9lH6+BLgKmFPb8+56zXtre298uX0aEf1Pr6J11xExcNWY4UpgO7AT+GT9+mLg9ZQZxF9LOqZ+y1JKfPMeYBQlJtlXz/UkLowBJElha1lYR4w6Psfa/qXtdhdrgfuBaV18/37gREljbO+1vaEevwjYafsW2y/a3gp8l5r0dGFkbcNO4C7gNspL6iPActvrbe+3vYjygruwoQ2TJbXZ3mV7Yy/74g7gcknD6tdzgZW29wGnA28HvmT737Z3ADce5XkAnpK0F3iifr7ZccL2GtuP137+M7CCrvsZetenEdGHdPga7IM6fF32mj64/3xJq/qirRHRkhbWKqm9wIeBWbYPAJ8GvmH7r7YP2L4VGA6cIWksMAu4yvY224dsb7a9HaCHcWEMIEkKW8u1tkc1fPbWssZNknbWF8sMyqhSZ2YBJwOPSdos6YP1+ETg5MaEE/gOpca9K7trG0bbnmR7gW0DbwGebLr2Ccp6vb2UUa9LgfZaWnp2L/oB2w9SZgIvkTSYMgt3R8PzjAKea3ielcDYo9x2AnBcbeMU4HUdJyRdIGmDpO2SdgOfoOt+7mhDT/s0IvpQ4xpsDq8+OKICISLi/+Ba26OA8cA2SlUTlBhieVMMMZoSY00AXqiDzUfoYVwYA0iSwhYm6a2UktAvA2Pqi+U+QJ1db3uT7csoL4cFwJ11xKkd+FNTwtlm+6ReNOvvlJdZo4n1OLYfsN3xgvoFcE8X6+wOdeNnLaPMEL6PUjaxvh5vB55tep6RNTB8SXVkbTUlibwFoJZr3A38EBhneySwhMP72U236ss+jYj+dbGkLTX4WippSMcJSadJWquyO+AWSfPq8ZnAdcBFHbOP9fj5kh6StFvS05K+X0vDIiI6ZfsfwDzKXgpvpsQQlzfFEK+1vYJSHjq0Yw1ho57GhTGwJClsbcdRftGfBQ5JmgGc39mFko6RNFfSaNuHgF311AFgNTBW0qckDZM0SNIkSdN60ablwBWSpkoaLOka4HjgPkljJc2SNKL+3D3AwS7u8wzwtqP8rGWU5/0cpWS1IzHbCGyV9G1JI1RMkNSTmYGbgOl1reJQYBiww/YLks7gyDLQ5vb2ZZ9GRP+6EDgNeAdwHnAFgKQ3Ar+jrJMeA8wEviXpXNurKGXpqxtmI6EMUM2jVBpMBc4GPv8KPktEvArZ3gSsoww2LQaulzQJ/ruxzKWSRth+BrgXWCLpTZJeI+lUScfTg7gwBp4khS3M9iPADZTNTHYAH6JssNKVOcAWSc9T1gDOsb3D9r8ogdC5lNLPHcCd9KLU0fYfgGuAH9f7zAam295F+f/6WcoI2G5KzfwHapLabBFwXh257/RvE9bSiQ2UBdXLGo4fpJSAjgMerT/rN5RF2d19jm2U0bbrbT9f2/ojSXuAr1I2umn0deDWWq7xlb7s04jod/Nt76m/9407D88F1tu+y/ZB2w8DP+El1ga7/M3VzfX6v1EqDKb1c/sjYmC4gbI5zCrKPg1317jjUQ5/73yUEks9RBnkXwIM70VcGAOI/jc5EhEREZ2RtA5YVTe/6jg2kbLmeXQduELSImCU7Y9JWkwJ0PY13GoQ8EfbMyTNB6bYntlwz3cBCym7Hg8HBgOP2T6ViIiIfpKZwoiIiP7RDtzTtK5nRF0XDZ2vfV4BrAVOsN1GKQXLep6IiOhXSQojIiL6xzLgHEmXSRpSP1PqbCCUtcQTJA1q+J42YFfdHXoy5e+NRURE9KskhREREf2g7gh4AXA18DQlCVxMSfyg7KC8B9het36nXvvFuhvpEuBnr2ijIyKiJWVNYURERERERAvLTGFEREREREQLS1IYERERERHRwpIURkREREREtLAkhRERERERES0sSWFEREREREQLS1IYERERERHRwpIURkREREREtLAkhRERERERES0sSWFEREREREQLS1IYERERERHRwv4DNZQyQ0vzns0AAAAASUVORK5CYII=\n", 708 | "text/plain": [ 709 | "
" 710 | ] 711 | }, 712 | "metadata": { 713 | "needs_background": "light" 714 | }, 715 | "output_type": "display_data" 716 | } 717 | ], 718 | "source": [ 719 | "if perform_computation:\n", 720 | " fig, axes = plt.subplots(nrows=1, ncols=3, figsize=(12,3), dpi=90)\n", 721 | "\n", 722 | " ax=axes[0]\n", 723 | " ax.plot(fpr_list, tpr_list)\n", 724 | " ax.set_xlabel('False Positive Rate')\n", 725 | " ax.set_ylabel('True Positive Rate')\n", 726 | " ax.set_title('ROC Curve')\n", 727 | " ax.set_xlim(-0.05, 1.05)\n", 728 | " ax.set_ylim(-0.05, 1.05)\n", 729 | " ax.plot(np.arange(-0.05, 1.05, 0.01), np.arange(-0.05, 1.05, 0.01), ls='--', c='black')\n", 730 | "\n", 731 | " ax=axes[1]\n", 732 | " ax.plot(all_theta, f1_list)\n", 733 | " ax.set_xlabel('Theta')\n", 734 | " ax.set_ylabel('F1-statistic')\n", 735 | " ax.set_title('F1-score Vs. Theta')\n", 736 | " ax.set_xscale('log')\n", 737 | "\n", 738 | " ax=axes[2]\n", 739 | " ax.plot(tpr_list, ppv_list)\n", 740 | " ax.set_xlabel('Recall')\n", 741 | " ax.set_ylabel('Precision')\n", 742 | " ax.set_title('Precision Vs. Recall')\n", 743 | " ax.set_xlim(-0.05, 1.05)\n", 744 | " ax.set_ylim(-0.05, 1.05)\n", 745 | " ax.plot(np.arange(-0.05, 1.05, 0.01), 1-np.arange(-0.05, 1.05, 0.01), ls='--', c='black')\n", 746 | " None" 747 | ] 748 | }, 749 | { 750 | "cell_type": "code", 751 | "execution_count": 19, 752 | "metadata": {}, 753 | "outputs": [ 754 | { 755 | "name": "stdout", 756 | "output_type": "stream", 757 | "text": [ 758 | "Best theta w.r.t. the F-score is 0.8\n" 759 | ] 760 | } 761 | ], 762 | "source": [ 763 | "if perform_computation:\n", 764 | " best_theta = all_theta[np.argmax(f1_list)]\n", 765 | " print(f'Best theta w.r.t. the F-score is {best_theta}')" 766 | ] 767 | }, 768 | { 769 | "cell_type": "markdown", 770 | "metadata": {}, 771 | "source": [ 772 | "Now let's try the tuned hyper-parameters, and verify whether it visually improved the Boltzman machine." 773 | ] 774 | }, 775 | { 776 | "cell_type": "code", 777 | "execution_count": 20, 778 | "metadata": { 779 | "deletable": false, 780 | "editable": false, 781 | "nbgrader": { 782 | "cell_type": "code", 783 | "checksum": "9ffddbff83f528b6590d47a23414c8dd", 784 | "grade": true, 785 | "grade_id": "cell-00c232dc99ca3fdd", 786 | "locked": true, 787 | "points": 0, 788 | "schema_version": 3, 789 | "solution": false, 790 | "task": false 791 | } 792 | }, 793 | "outputs": [ 794 | { 795 | "name": "stdout", 796 | "output_type": "stream", 797 | "text": [ 798 | "The reference and solution images are the same to a T! Well done on this test case.\n" 799 | ] 800 | }, 801 | { 802 | "data": { 803 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAAElCAYAAABgRJorAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3debxkdXnn8e9XWkQBBaTZmi0CJoIL0i3q4EJebmBEcCYYTaLooE0SGWWCUSSLZEaD+hKXGYkOKGlQBHEQxERH0SjEiTp2KwrYLqAtSzfNZRNUJg7wzB/nFH1ucavq3Fqfc+7n/Xr1q2t5qs5TvzrnueepOnV+jggBAAAAAGbrYbNOAAAAAABAcwYAAAAAKdCcAQAAAEACNGcAAAAAkADNGQAAAAAkQHMGAAAAAAnQnAHAEmV7V9tX2r7H9hmzzmfabG+w/fwhH7u37V/a3mrceQFoLttrbL9jhMdfa/vwMaaEhqE5a6lyp+PecufhlrJYbDfhZY5UkAAs3ojb+mpJt0l6dEScPME0J8b2nrYvtn2b7V/Yvtr2ayawnHmNXETcEBHbRcT9E1hW2N5/3M8LNI3t822f03Xbc23fbnv3CS53a9tn2L6prK0/s/3+CSznIftNEXFQRHxtAsv6mu3Xjft5MX40Z+12VERsJ+lgSU+V9LYZ5wNgMobd1veR9IOIiMUu0PayxT5mQj4u6UYVr+Wxkl4tafNMMwIwLm+U9GLbL5Ak29tIOlvSyRGxaRwL6FHL3iZplaRDJW0v6XclfXccywMGoTlbAiLiFklfVLHjJkmyfYrt68vDmX5g+2WV+35ue2V5+Y/LT3EPLK+/zvalg5Zpe9/yca+1faPtO23/ie2n2f6+7btsf6gSv5/tfy4/Dbut/LRsh8r9h9j+bpnvp21/qvppk+2X2L6qfN5/tf3kUccNaJoe2/ozym3iLtvf6xwuY3uNpOMkvaX8ZPj5th9WqQ23277I9k5lfGebPt72DZL+ubz9P9peX27jX7S9T2XZUW73PynvP9O2K/e/vnxspw4dUt6+R/lt2Fz5ifUb+7zsp0laExG/ioj7IuK7EfGFyjJeWh4mdFf5yfETFnqS7k+wbR9u+6by8scl7S3pc+VYvaUyHssqOV9m+w7b19l+feW5TivH8rzytV5re1XfN3P+Yz9t+xPlY6+2/Xjbb7N9a1lfX1iJf21lTH9q+4Su53uL7U22N5b1/MFv6Ww/wvZ7bd9ge7Ptj9h+ZJ08gUmIiNsl/SdJZ9neVtLbJV0fEWv6bdvu+va5un13tm3bb7V9i6R/WGDRT5N0SURsjMKGiDiv8nxPKJd5V5nDSxfK3/ZrbH+967awvb/t1ZL+SFtq8OfK+x/8lr7cJj9Qbq8by8uP6HodJ5e1YJPt19YZ18pj31J57DG2X2z7x2UdO7USf6jtb5Svd5PtD9neunL/C23/yMXRC39v+wpXvqVzn78TeCiasyXA9p6SjpR0XeXm6yU9W9JjJP2tpE94yyECV0g6vLz8HEk/lfTcyvUrFrH4p0s6QNIfSPqApL+U9HxJB0l6ue3O81rS6ZL2kPQESXtJOq3Mf2tJl0haI2knSRdIqjaTh0g6R9IJKj45/x+SLusUMGCp6N7Wba+Q9E+S3qFi23mzpIttL4+I10g6X9J7ysPzvqziU+pjVGzve0i6U9KZXYt5ropt9EW2j5F0qqR/L2m5pH9RsX1WvUTFjs5TJL1c0ovK3I5VsY2/WtKjJb1U0u22Hybpc5K+J2mFpOdJOsn2i3q87G9KOtP2K2zv3TUejy/zOanM7/MqGqytH/o0vUXEqyTdoPIbyoh4zwJhF0i6ScW4/b6kv7P9vMr9L5V0oaQdJF0m6UMPeYbejlLxDeGOKj69/6KKv98rJP0XFTWv41YVY/5oSa+V9H5vaXqPkPTnKmrw/tpS1zveLenxKpr7/cvn/5tF5AmMXUR8WtI6FdvYakknjGHb3k1FTdynfM5u35T057b/zPaT7HkfKj1cRY36kqRdVDSP59v+7UW+rrM0vwYftUDYX0p6hopt8ikqvsn7q67X8RgV2+rxKmrhjjVT2E3SNtqynZ8t6Y8lrVSxf/g3th9Xxt4v6T9L2lnSM1XU5T+TJNs7S/qfKr5tfKykH0n6d52F1Pw7gaqI4F8L/0naIOmXku6RFJK+ImmHPvFXSTq6vHy8pMvKy+slvU7SheX1n0s6pMdzrJH0jvLyvuVyV1Tuv13SH1SuXyzppB7PdYyk75aXnyPpZkmu3P/1yrI+LOm/dj3+R5KeO+v3gX/8m/S/ftu6pLdK+nhX/BclHVdefnCbLa+vl/S8yvXdJf0/Scsq2/TjKvd/QdLxlesPk/RrSfuU10PSsyr3XyTplEoeb1rg9Txd0g1dt71N0j/0eP07SnqXpGtV7EBcJelp5X1/LemirvxulnR4Zeye32MsDpd0U9c4P79yvTMey1R8mHS/pO0r95+u4hs9qWhCv1y570BJ9/Z5T0PS/pXHXl6576jy/d6qvL59Gb9gfZd0aWecVXyIdXrlvv07y1LxAdmvJO1Xuf+Zkn4263Wcf/yTtGu53nfW5UHb9oPbUHn9we273LZ/I2mbPsvbStIbJP1vSf8maaO21M1nS7pF0sMq8RdIOm2BZb1G0te7nru6fc+rO+Vt1bp0vaQXV+57kaQNlddxr6RllftvlfSMHq/pa5Je1/XY7jry9Er8OknH9Hiuk1R8sygVH7B9o3KfVRxq3llW378T/HvoP745a7djImJ7FRvh76j4xEOSZPvV3nIY4F2Snli5/wpJz7a9m4oC9SlJh9neV8UnNFctIofqbz/uXeD6dmU+u9i+0PbNtu+W9IlKPntIujnKrbp0Y+XyPpJO7ryW8vXsVT4OWAp6bev7SDq2a9t4loqmayH7SLqkErteRdOxayWme9v7YCX+DhV/mFdUYm6pXP61ym1exTZ6fY8c9ujK+dSuHB4UEXdGxCkRcVAZc5WkS8tPuvdQ8YFSJ/aBMv8VCz3XCPaQdEdE3FO57efqPw7buP7v9rrr5m2x5UQk95b/d2rpkba/WR6WdJekF2t+La2+f9XLyyU9StK6yrj/r/J2YKYiYrOKkxddW9406rY9FxH/t8/y7o+IMyPiMBXfdr9T0jnloZN7SLqxXGZH9/Y+LvNeZ3m5um9ze0TcV7lerbGD3L5AHem1j/Z42//o4qRTd0v6O/WoK+W+2k2V56nzdwIVNGdLQERcoeLTmfdKUnms79mSTpT02IjYQdI1KjYWRcR1KjbwN0q6stzhuEXFV/9f7ypI43K6ik9tnhwRj1bx1XrnMIJNklZUDytQsWPXcaOkd0bEDpV/j4oIvjbHktK9ravYNj7etW1sGxHv6vEUN0o6sit+m4i4ubqYrvgTuuIfGRH/WiPdGyXt1+P2n3U95/YR8eJBTxgRt6l47XuoOGRpo4odA0lSWUP2UvEJe7dfqWhOOnbrfvo+i94oaSfb21du27vHciamPJT7YhVjsGtZ2z+v+bV0z8pDqnX0NhU7YwdVxv0xUZxoBshm0Lb9aw2/Pc8PjLg3Is5UcZj3geWy9yoPwe7otb3Pqyvlh96LyWPe6yyXs7Fm6uP0YUk/lHRAuY92qnrUlfK9qNaZUf5OLEk0Z0vHByS9wPbBkrZVURDmpOIH5Cq+Oau6QkXz1vl92de6ro/b9ioOWbir/J3MX1Tu+4aKT+9PtL3M9tEqjrvuOFvSn9h+ugvb2v69rh0lYKmobuufkHSU7RfZ3sr2Ni5+CL5nj8d+RNI7Oz/Wtr283N56+Yikt9k+qIx/TPlbsjo+KunNtleW2+3+5XL/j6S7XfxY/5Fl3k+0/bSFnsT2u8v7l5Xb/J9Kui6KEwlcJOn3bD+v/J3IySoOUVpop+AqFWeF26ncgTqp6/7Nkh730IdJEXFj+Zynl2P8ZBWHh59fcyzGZWtJj1BR2++zfaSkF1buv0jSa12czOBRqvyerPzQ7WwVv1HbRSp+s9jnt37ALA3atq+S9Idl/ThCD/19ZV+2Typr5SPL2nKciv2U70r6loqm6y22H+7iJEtHqfhNabfvSTrI9sEuzjR5Wtf9PetK6QJJf1XW4p1VbLOfWMxrGZPtJd0t6Ze2f0dFne34J0lPcnFCkWUqDgetNqGj/J1YkmjOloiImJN0nqS/jogfSDpDRdOzWdKTVBxXXXWFio3xyh7Xx+1vJR0i6RcqNvTPVHL/jYofkh4v6S4V36r9o4pCrIhYK+n1Kn5gf6eKkyG8ZkJ5Aql1bes3Sjpaxaeccyo+wfwL9a79H1Rxsoov2b5HxY/in95nWZeoOInEheWhLteoOCFJnTw/reJQoU+q+L3cpZJ2Kg+zOUrFD+B/puIbnY+qOKR6IY9SccKgu1ScvGgfFSffUET8SEW9+O/l8xyl4qQev1ngeT6uYkdqg4of+n+q6/7TVewk3WX7zQs8/pUqfoe2sczn7RFxeb8xGLfyKIc3qthxvVPSH6p4Pzv3f0HSf5P0VRV18hvlXf9W/v/W8vZvlu/nlyUt6iQHwDTU2LbfVN52l4ozIg48y3SXe1XsJ91SPv8bJP2HiPhpuYyXqqh1t0n6e0mvjogfLpDnj1WctOfLkn6i4vfyVR+TdGBZVxbK8R2S1kr6vqSrJX2nvG3a3qyintyj4kOcB+tjecTCsZLeo+LcAgeqyLmzjzb034mlyvN/xgM0g+1vSfpIRCx0ClwAwADl72eukfSIrt+tAMBQysM9b5L0RxHx1Vnn00R8c4ZGsP1c27tVDi94soofqwMAarL9Mttbuzjd9rslfY7GDMAoykPndyh/99r5Pdo3Z5xWY9GcoSl+W8UhR79QcWz570fEptmmBACNc4KKQ1yvV/Fb3j/tHw4AAz1TRU3pHGJ6TETc2/8h6IXDGgEAAAAgAb45AwAAAIAEaM4AAAAAIIGZNWe2j7D9I9vX2T5lVnnUZXuD7attX2V77azz6Wb7HNu32r6mcttOti+3/ZPy/x1nmWNVj3xPs31zOcZX2R446ey02N7L9ldtr7d9re03lbenHOM++aYd4yyoTeNFbZosatPSQn0aL+rT5FCbRshlFr85s72VpB9LeoGK021+W9Iry/m3UrK9QdKqcj6HdGw/R8UkzudFxBPL294j6Y6IeFdZxHeMiLfOMs+OHvmeJumXEfHeWea2ENu7S9o9Ir7jYqLbdZKOUTGfWrox7pPvy5V0jDOgNo0ftWmyqE1LB/Vp/KhPk0NtGt6svjk7VNJ1lcn8LlQxUSqGFBFXSrqj6+ajJZ1bXj5XxUqWQo9804qITRHxnfLyPZLWS1qhpGPcJ1/0R20aM2rTZFGblhTq05hRnyaH2jS8WTVnKyTdWLl+k/IX55D0JdvrbK+edTI17do53Xz5/y4zzqeOE21/v/zqPsVX3d1s7yvpqZK+pQaMcVe+UgPGeIaoTdORfrtZQPrthtrUetSn6Ui/7Swg9bZDbVqcWTVnXuC27Of0PywiDpF0pKQ3lF8tY7w+LGk/SQdL2iTpjNmm81C2t5N0saSTIuLuWeczyAL5ph/jGaM2YSHptxtq05JAfcJCUm871KbFm1VzdpOkvSrX95S0cUa51BIRG8v/b5V0iYrDC7LbXB5D2zmW9tYZ59NXRGyOiPsj4gFJZyvZGNt+uIoN9vyI+Ex5c9oxXijf7GOcALVpOtJuNwvJvt1Qm5YM6tN0pN12FpJ526E2DWdWzdm3JR1g+7dsby3pFZIum1EuA9netvxxoGxvK+mFkq7p/6gULpN0XHn5OEmfnWEuA3U21tLLlGiMbVvSxyStj4j3Ve5KOca98s08xklQm6Yj5XbTS+bthtq0pFCfpiPlttNL1m2H2jRCLrM4W6MkuTgV5QckbSXpnIh450wSqcH241R84iNJyyR9Mlu+ti+QdLiknSVtlvR2SZdKukjS3pJukHRsRKT4IWmPfA9X8bVxSNog6YTOccmzZvtZkv5F0tWSHihvPlXF8cjpxrhPvq9U0jHOgto0XtSmyaI2LS3Up/GiPk0OtWmEXGbVnAEAAAAAtpjZJNQAAAAAgC1ozgAAAAAgAZozAAAAAEiA5gwAAAAAEqA5AwAAAIAEZtqc2V49y+UvFvlOFvlOVtPynbWmjRf5Thb5TlbT8p2lpo0V+U4W+U7WLPKd9TdnjXqDRL6TRr6T1bR8Z61p40W+k0W+k9W0fGepaWNFvpNFvpO15JozAAAAAICmPAm17YELW7lyZd/7161bN5WYhe6fm5vT8uXLp55LJ2aQbPk2ZXw7Y9ukfId5jgnme1tELO8blJztGHW8Zr19kS/59sp31rnMMl9Jja9PO++8c+y7774PXudvD/mS73hymWW+6lObRmrObB8h6YOStpL00Yh414D4gQsblI/tqcRMazmLiRkkW75NGd/O2DYp3yy5lDHrImJV36AZWEx9sh0tfF/Id4Yx5DJ8zJj/bqarT4vdd1q1alWsXbu23/M17n0h39y51Ikhl+FjBtWmoQ9rtL2VpDMlHSnpQEmvtH3gsM8HAONCfQKQEbUJwCCj/ObsUEnXRcRPI+I3ki6UdPR40gKAkVCfAGREbQLQ1yjN2QpJN1au31TeNo/t1bbX2u79nTwAjNfA+kRtAjADi953mpubm1pyAGZvlOZsoR9BPeQAy4g4KyJWZTvmG0CrDaxP1CYAM7DofafqyQgAtN8ozdlNkvaqXN9T0sbR0gGAsaA+AciI2gSgr1Gas29LOsD2b9neWtIrJF02nrQAYCTUJwAZUZsA9LVs2AdGxH22T5T0RRWngz0nIq4dNaE6p4yfVkymXOrIlm/TxrdJ+WbKJaNh6lMb3xfynW0MuQwfQ23aYt26dWne30zrSJ2YTLnUicmUS50YchktpudjpzkJ9TTn6qijbXMmZMmlTky2XKTmrA+Zcilj0s0jtFhmnrP0MZlyqRNDLsPHjHlfoPH1iXnOhovJlEudmEy51Ikhl+FjBtWmUQ5rBAAAAACMCc0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkADNGQAAAAAkQHMGAAAAAAlMdZ4z29NbGIBpafw8QtQmoLWoTwAy6lmblk0zi5UrVyrDRIp1YjLlUicmUy51YrLlIjEJ9SgxbdDG94V8ZxdDLsPHUJvmy7LvlGkdqROTKZc6MZlyqRNDLsPHDKpNHNYIAAAAAAnQnAEAAABAAjRnAAAAAJAAzRkAAAAAJEBzBgAAAAAJ0JwBAAAAQALMcwZgVMwjBCAr6hOAjHrWJr45AwAAAIAEmIS6AbnUiZn2pJ2TnoBvXMtZTC5NWR8y5dKJaYM2vi/kO7sYchk+hto0X5Z9p0zrSJ2YTLnUicmUS50Ychk+hkmoAQAAAKABaM4AAAAAIAGaMwAAAABIgOYMAAAAABKgOQMAAACABGjOAAAAACABJqEGMComeQWQFfUJQEY9axPznDUglzoxmXKpE5MtF4l5zkaJaYM2vi/kO7sYchk+hto0X5Z9p0zrSJ2YTLnUicmUS50Ychk+hnnOAAAAAKABaM4AAAAAIAGaMwAAAABIgOYMAAAAABKgOQMAAACABGjOAAAAACABmjMAAAAASIBJqAGMikleAWRFfQKQEZNQLzYmUy51YjLlUicmWy4Sk1CPEtMGbXxfyHd2MeQyfAy1ab4s+06Z1pE6MZlyqROTKZc6MeQyfMyg2jRSc2Z7g6R7JN0v6b6mfzoFoD2oTwAyojYB6Gcc35z9bkTcNobnAYBxoz4ByIjaBGBBnBAEAAAAABIYtTkLSV+yvc726oUCbK+2vdb22rm5uREXBwC19a1P1do0g9wALF3sOwHoadTDGg+LiI22d5F0ue0fRsSV1YCIOEvSWZK0atUqzjgEYFr61qdqbeJsaACmiH0nAD2N9M1ZRGws/79V0iWSDh1HUgAwKuoTgIyoTQD6Gbo5s72t7e07lyW9UNI140oMAIZFfQKQEbUJwCBDT0Jt+3EqPvGRisMjPxkR7xzwGL6aB9on3SSvi61P1CagtVLVJ/adAJTGPwl1RPxU0lMW85gsEynWicmUS52YTLnUicmWi8Qk1KPEZDNMfWrj+0K+s4shl+FjqE3zZdl3yrSO1InJlEudmEy51Ikhl+FjBtUmTqUPAAAAAAnQnAEAAABAAjRnAAAAAJAAzRkAAAAAJEBzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQwNCTUA+1MCZSBNoo1SSvw6A2Aa1FfQKQ0fgnoR5GlokU68RkyqVOTMYJOZsyvkxCPXpMG7TxfWlavtOy1MY3Uy51Ypq4Tk1Sln2nTOtInZgmrkdZxq5ODLkMH8Mk1AAAAADQADRnAAAAAJAAzRkAAAAAJEBzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQAPOcARgV8wgByIr6BCCjHPOc1ZFpDoJs8/tMa56zTHN+1NHGfDOtm00b32G18X1pWr5t07TXnGV9YJ2aj3nOhothPeqvaa+7betmPxzWCAAAAAAJ0JwBAAAAQAI0ZwAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkEC6ec7qzKswrZhp5lLHOPIdx3KyaWO+mdbNpo3vsNr4vjQt37Zp2mvOtD6wTm2xbt26NNtypnWkTgzrUW9Ne91tWzf7mWpzlmUixToxmXKpE9O0jUxq3gSI49LGdbMN2vi+ZMq3adr4mgZp47rZBln2nTLVrzoxbV1H2via6mjbutkPhzUCAAAAQAI0ZwAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkADNGQAAAAAk4EHn6h/rwuzpLQzAtKyLiFWzTmIU1CagtahPADLqWZuYhLrP/XW0cRLXpk0OydjNPqYN2vi+tG2i3yxj14lpkmxj17Z1c5Ky7DtlWkfqxEz7/c80vk2TaewyrJsDD2u0fY7tW21fU7ltJ9uX2/5J+f+Og54HAMaN+gQgI2oTgGHV+c3ZGklHdN12iqSvRMQBkr5SXgeAaVsj6hOAfNaI2gRgCAObs4i4UtIdXTcfLenc8vK5ko4Zc14AMBD1CUBG1CYAwxr2bI27RsQmSSr/36VXoO3VttfaXjs3Nzfk4gCgtlr1qVqbppodgKWKfScAA038VPoRcVZErIqIVcuXL5/04gCglmptmnUuAFDFvhOwdA3bnG22vbsklf/fOr6UAGAk1CcAGVGbAAw0bHN2maTjysvHSfrseNIBgJFRnwBkRG0CMNDASahtXyDpcEk7S9os6e2SLpV0kaS9Jd0g6diI6P7h60LPxUSKQPvMbJLXcdUnahPQWjOpT+w7ARigZ20a2JyN06pVq2LQRIp1tHGyxWnJNAEfYzdcTKaJFMuYmTVn42I7Wvi+TG0i+HFp2/hOU9vGbox/Hxpfn+rsOzXtfeHv/2SeYzEx09LGsRvT34eetWniJwQBAAAAAAxGcwYAAAAACdCcAQAAAEACNGcAAAAAkADNGQAAAAAkQHMGAAAAAAnQnAEAAABAAlOd54yJFIFWavw8QtQmoLWoTwAy6lmblk0zi5UrVyrDRIp1YpgMcLIx45xwfFoyjV2WXDoxbdDG94VJqGcXw9gNH0Ntmi/LvlOmdaQTMy1tHN9paePYTfrvA4c1AgAAAEACNGcAAAAAkADNGQAAAAAkQHMGAAAAAAnQnAEAAABAAjRnAAAAAJAA85wBGBXzCAHIivoEICPmOVtsTKZcOjGDtDHfaWrK+jDteU6yvU+Tkul9WarrUdPGd1raOHaZ1s3ssuw7ZVpH6sRMqzaNK6aJ630bxy7DuslhjQAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkADNGQAAAAAkQHMGAAAAAAkwCTWAUTHJK4CsqE8AMlp6k1DX0bYJ7bJNvjqOySHHpU4uTZnMMtN614lpgza+L+Q7fMy0tHHsMuXbBkxCPVxMplzqxGRcp5syvplyqRPDJNQAAAAA0AA0ZwAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkADNGQAAAAAkwCTUAEbFJK8AsqI+Acho+EmobZ8j6SWSbo2IJ5a3nSbp9ZLmyrBTI+Lzg54ry0SKdWIy5VInJlMudWKyTaRYJybT2GXJpRMzK+OsT218X8h3+JhpaePYZcp3Vtq475RpHakTkymXOjEZ1+mmjG+mXOrEjGMS6jWSjljg9vdHxMHlv4HFBQAmYI2oTwDyWSNqE4AhDGzOIuJKSXdMIRcAWBTqE4CMqE0AhjXKCUFOtP192+fY3nFsGQHA6KhPADKiNgHoa9jm7MOS9pN0sKRNks7oFWh7te21ttfOzc31CgOAcalVn6q1aZrJAViy2HcCMNBQzVlEbI6I+yPiAUlnSzq0T+xZEbEqIlYtX7582DwBoJa69alam6abIYCliH0nAHUM1ZzZ3r1y9WWSrhlPOgAwGuoTgIyoTQDqqHMq/QskHS5pZ9s3SXq7pMNtHywpJG2QdMIEcwSABVGfAGREbQIwLCahBjAqJnkFkBX1CUBGw09CPU5ZJlKsE5MplzoxmXLpxEzLuCYDbMr4ZsqlE9MGbXxfyHfhmGlp49g1Ld82yLLvlGkdqROTKZc6MZkmmJaaNb6ZcqkTM45JqAEAAAAAE0ZzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACTEINYFRM8gogK+oTgIyaMwl1HZkmtKsjU77TnNhzWpMtMgn17GPaoI3vy1LMN5OmjV2WGGrTfExCPVxMplw6MZm0aXwz5VInhkmoAQAAAKABaM4AAAAAIAGaMwAAAABIgOYMAAAAABKgOQMAAACABGjOAAAAACAB5jkDMCrmEQKQFfUJQEbNmeesKXMQdGTKt41zdUzzvc40vk3JpRPTBm18X5ZivtPUlPHNlEudGGrTfFn2nTKtI3ViMuXSiZmWpTa+mXKpE8M8ZwAAAADQADRnAAAAAJAAzRkAAAAAJEBzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQAM0ZAAAAACTAJNQARsUkrwCyoj4ByIhJqBcbkymXOjFMkjh8DLR791EAAAcYSURBVJNQjx7TBm18X9qW7zS1aXwz5VInhto0X5Z9p0zrSJ2Ytv4dXKrj25Rc6sQwCTUAAAAANADNGQAAAAAkQHMGAAAAAAnQnAEAAABAAjRnAAAAAJAAzRkAAAAAJEBzBgAAAAAJMAk1gFExySuArKhPADIafhJq23tJOk/SbpIekHRWRHzQ9k6SPiVpX0kbJL08Iu7s91xZJlKsEzPtiQmnlW8m08y3aZNQD5JlO+nEzMI4a5PUrkmH68RkyqUTk0mbxjdTLnVi2rBOtXHfKdM6UiemDevRQto4vm3KpU7MOCahvk/SyRHxBEnPkPQG2wdKOkXSVyLiAElfKa8DwLRQmwBkRX0CMJSBzVlEbIqI75SX75G0XtIKSUdLOrcMO1fSMZNKEgC6UZsAZEV9AjCsRZ0QxPa+kp4q6VuSdo2ITVJRhCTtMu7kAKAOahOArKhPABajdnNmeztJF0s6KSLuXsTjVttea3vt3NzcMDkCQE/jqE2Tyw7AUsa+E4DFqtWc2X64iuJyfkR8prx5s+3dy/t3l3TrQo+NiLMiYlVErFq+fPk4cgYASeOrTdPJFsBSwr4TgGEMbM5cnFLkY5LWR8T7KnddJum48vJxkj47/vQAYGHUJgBZUZ8ADGvgqfQlHSbpVZKutn1Vedupkt4l6SLbx0u6QdKxk0kRABZEbQKQFfUJwFCYhBrAqJjkFUBW1CcAGQ0/CfU4ZZlIsU5MplzqxEx7IsU2jW/GSaibkksnpg3a+L40Ld9xyPKaMo1vplzqxGRbp2Yty75TpnWkTkzG9YjxbX8udWLGMQk1AAAAAGDCaM4AAAAAIAGaMwAAAABIgOYMAAAAABKgOQMAAACABGjOAAAAACABmjMAAAAASIBJqAGMikleAWRFfQKQEZNQLzYmUy51YqY92WabxpdJqEePaYM2vi+Z8p2WpTq+TcmlTkwT17tJyrLvlGkdqROTcR1ZiuO71HKpE8Mk1AAAAADQADRnAAAAAJAAzRkAAAAAJEBzBgAAAAAJ0JwBAAAAQAI0ZwAAAACQAPOcARgV8wgByIr6BCAj5jlbbEymXOrEZJqrQ8o3vnVkyrcpuXRi2qCN70umfKdlqY5vU3KpE9PE9W6Ssuw7ZVpH6sRkXEeW4vgutVzqxDDPGQAAAAA0AM0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkADNGQAAAAAkQHMGAAAAAAkwCTWAUTHJK4CsqE8AMmIS6sXGZMqlEzMtS218O2PbpHyz5NKJaYM2vi9tm+g3y9jViSGX4WOauG5OUpZ9p0zrSJ2YTBNMS+0cX3IZLoZJqAEAAACgAWjOAAAAACABmjMAAAAASIDmDAAAAAASoDkDAAAAgARozgAAAAAgAZozAAAAAEiASagBjIpJXgFkRX0CkNHwk1Db3kvSeZJ2k/SApLMi4oO2T5P0eklzZeipEfH5fs+VZSLFOjFLeRLfNo4vk1BPNmYWxlmbJCYQ7RWTSZaxqxNDLsPHNL02lctu3b5TpnWkE5NJG8e3KflmyqVOzKB1d2BzJuk+SSdHxHdsby9pne3Ly/veHxHvrfEcADBu1CYAWVGfAAxlYHMWEZskbSov32N7vaQVk04MAPqhNgHIivoEYFiLOiGI7X0lPVXSt8qbTrT9fdvn2N5xzLkBQC3UJgBZUZ8ALEbt5sz2dpIulnRSRNwt6cOS9pN0sIpPh87o8bjVttfaXjs3N7dQCAAMbRy1aWrJAlhS2HcCsFi1mjPbD1dRXM6PiM9IUkRsjoj7I+IBSWdLOnShx0bEWRGxKiJWLV++fFx5A8DYatP0MgawVLDvBGAYA5szF6cU+Zik9RHxvsrtu1fCXibpmvGnBwALozYByIr6BGBYdc7WeJikV0m62vZV5W2nSnql7YMlhaQNkk6YSIYAsDBqE4CsqE8AhsIk1ABGxSSvALKiPgHIaPhJqMcpy0SKdWIy5dKJGSRbvk0Z387YNinfLLl0Ytqgje9Lpol+M+XbtPFtUy51YqhN82XZd8q0jtSJmVZtGldMprGrE0Muw8cMWjcXdSp9AAAAAMBk0JwBAAAAQAI0ZwAAAACQAM0ZAAAAACRAcwYAAAAACdCcAQAAAEACNGcAAAAAkACTUAMYFZO8AsiK+gQgIyahXmxMplzqxGTKpU5MpglwmYR69Jg2aOP7Qr6ziyGX4WOoTfNl2XfKtI7UicmUS52YTLnUiSGX4WOYhBoAAAAAGoDmDAAAAAASoDkDAAAAgARozgAAAAAgAZozAAAAAEiA5gwAAAAAEmCeMwCjYh4hAFlRnwBklGOeM0m3Sfr5lJcJYLL2mXUCY0BtAtqJ+gQgo561aarfnAEAAAAAFsZvzgAAAAAgAZozAAAAAEiA5gwAAAAAEqA5AwAAAIAEaM4AAAAAIIH/D6fUHeInbbf4AAAAAElFTkSuQmCC\n", 804 | "text/plain": [ 805 | "
" 806 | ] 807 | }, 808 | "metadata": { 809 | "needs_background": "light" 810 | }, 811 | "output_type": "display_data" 812 | }, 813 | { 814 | "name": "stdout", 815 | "output_type": "stream", 816 | "text": [ 817 | " Enter nothing to go to the next image\n", 818 | "or\n", 819 | " Enter \"s\" when you are done to recieve the three images. \n", 820 | " **Don't forget to do this before continuing to the next step.**\n", 821 | "s\n" 822 | ] 823 | } 824 | ], 825 | "source": [ 826 | "if perform_computation:\n", 827 | " def test_boltzman(x, seed = 12345, theta_X=0.5*np.log(1/noise_flip_prob-1), theta_pi=best_theta, iterations=100): \n", 828 | " np_random = np.random.RandomState(seed=seed)\n", 829 | " initial_pi = np_random.uniform(0,1, size=x.shape)\n", 830 | " return boltzman_meanfield(x, initial_pi, theta_X=theta_X, \n", 831 | " theta_pi=theta_pi, iterations=iterations) > 0.5\n", 832 | "\n", 833 | " (orig_image, ref_image, test_im, success_is_row_inky) = show_test_cases(test_boltzman, task_id='4_V')" 834 | ] 835 | }, 836 | { 837 | "cell_type": "code", 838 | "execution_count": null, 839 | "metadata": {}, 840 | "outputs": [], 841 | "source": [] 842 | }, 843 | { 844 | "cell_type": "code", 845 | "execution_count": null, 846 | "metadata": {}, 847 | "outputs": [], 848 | "source": [] 849 | } 850 | ], 851 | "metadata": { 852 | "kernelspec": { 853 | "display_name": "Python 3", 854 | "language": "python", 855 | "name": "python3" 856 | }, 857 | "language_info": { 858 | "codemirror_mode": { 859 | "name": "ipython", 860 | "version": 3 861 | }, 862 | "file_extension": ".py", 863 | "mimetype": "text/x-python", 864 | "name": "python", 865 | "nbconvert_exporter": "python", 866 | "pygments_lexer": "ipython3", 867 | "version": "3.7.6" 868 | } 869 | }, 870 | "nbformat": 4, 871 | "nbformat_minor": 4 872 | } 873 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ### Applied-Machine-Learning 2 | --------------------------------------------------------------------------------