├── README.md ├── Markov Chain Monte Carlo (MCMC) Introduction.ipynb └── Gaussian Processes made Easy.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # Self-learning Tutorials 2 | 1. Gaussian Processes 3 | 2. Markov Chain Monte Carlo 4 | -------------------------------------------------------------------------------- /Markov Chain Monte Carlo (MCMC) Introduction.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Markov Chain Monte Carlo (MCMC) Introduction\n", 8 | "MCMC is used to sample from complex probability distributions. Sampling from a simple distribution like the uniform $U(0,1)$ is easy enough using pseudorandom number generators, but things get harder for more complex distributions.\n", 9 | "\n", 10 | "For more complex distributions, such as the normal, more complex, but still relatively simple, approaches like the inverse transform can be used to perform samplgin , provided the CDF can be calculated in close form.\n", 11 | "\n", 12 | "For an even more complex distribution, $p(x)$, the CDF cannot be calculated, and a more sophisticated algorithm must be used. To obtain sample $x$ from $p(x)$, Monte Carlo is used." 13 | ] 14 | }, 15 | { 16 | "cell_type": "markdown", 17 | "metadata": {}, 18 | "source": [ 19 | "## Metropolis-Hastings (MH)\n", 20 | "The *Metropolis-Hastings* (MH) algorithm is the populat MCMC method. It starts with a point $x$ on the target distribution $p(x)$, whose form is known. A proposal value $x^*$ is obtained by sampling from the proposal distribution $q(x^*|x), which depends on the current value $x$. The proposal $x^*$ is \"accepted\" as a sample with probability:" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "$$A(x,x^*) = min \\left\\{1, \\frac{p(x^*)q(x|x^*)}{p(x)q(x^*|x)}\\right\\}$$" 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "metadata": {}, 33 | "source": [ 34 | "If the proposal is rejected, then the current value $x$ is used as the sample again. The final list of values, from $x$'s and $x^*$'s, will be distributed according to $p(x)$." 35 | ] 36 | }, 37 | { 38 | "cell_type": "markdown", 39 | "metadata": {}, 40 | "source": [ 41 | "The pseudo-code is shown below:" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "1. Initialize $x_0$\n", 49 | "2. For $i = 0$ to $N-1$\n", 50 | " * Sample $x^*$ ~ $q(x^*|x_i)$\n", 51 | " * Sample $u$ ~ $U(0,1)$\n", 52 | " * **If**: $u < A(x_i,x^*) = min \\left\\{1, \\frac{p(x^*)q(x_i|x^*)}{p(x_i)q(x^*|x_i)}\\right\\}$ \n", 53 | " $x_{i+1} = x^*$ \n", 54 | " **else**: \n", 55 | " $x_{i+1} = x_i$" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "This algorithm results in $N$ samples from the target distribution $p(x)$. The values jumps around, guided by the proposal distribution and the acceptance probability, and the sample eventually converge to the desired distribution.\n", 63 | "\n", 64 | "Note that the proposal distribution depends only on the current state $x$, and none before. Since the probability of the next step only depends on the current state, the sequence forms a Markov Chain." 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "### Technical Note\n", 72 | "Without getting too deep into the theory, MCMC aims to find a Markov Chain whose stationary distribution is the distribution we need to sample from. This requires care selection of the proposal distribution $q(x)$, but detailed discusssion is beyond our scoep, and we'll show an example instead." 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "## Example\n", 80 | "We will use MH to sample from the distribution $p(x) = 0.3e^{-0.2x^2}+0.7e^{-0.2(x-10)^2}$. This is plotted below to give a sense of its shape:" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": 37, 86 | "metadata": { 87 | "collapsed": true 88 | }, 89 | "outputs": [], 90 | "source": [ 91 | "import numpy as np\n", 92 | "import matplotlib.pyplot as plt\n", 93 | "import time\n", 94 | "%matplotlib inline" 95 | ] 96 | }, 97 | { 98 | "cell_type": "code", 99 | "execution_count": 38, 100 | "metadata": { 101 | "collapsed": true 102 | }, 103 | "outputs": [], 104 | "source": [ 105 | "# define the pdf\n", 106 | "p = lambda x: 0.3*np.exp(-0.2*x**2)+0.7*np.exp(-0.2*(x-10)**2)" 107 | ] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "execution_count": 39, 112 | "metadata": {}, 113 | "outputs": [ 114 | { 115 | "data": { 116 | "text/plain": [ 117 | "Text(0,0.5,'p(x)')" 118 | ] 119 | }, 120 | "execution_count": 39, 121 | "metadata": {}, 122 | "output_type": "execute_result" 123 | }, 124 | { 125 | "data": { 126 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xl0HPWZ7//3o32XJVmytViSN7Bl\nMAZs4yw4YTckAZIAgZCELBMmZ8Jv7iQzcwPJ7zJzGebMZOY3QzJ3MpkQIIEkbDdAcIKD2QMJGCyM\nsS3ZxrJsyZJlrbb2rdXP74+uNo0sWWt1dbee1zl93F1dVf2UtXxU3++3viWqijHGGDNdcV4XYIwx\nJrpZkBhjjJkRCxJjjDEzYkFijDFmRixIjDHGzIgFiTHGmBmxIDHGGDMjFiTGGGNmxILEGGPMjCR4\nXUA4zJ8/X8vLy70uwxhjosrbb7/dpqr5E603J4KkvLycyspKr8swxpioIiJ1k1nPmraMMcbMiAWJ\nMcaYGbEgMcYYMyMWJMYYY2bEgsQYY8yMuBokIrJJRPaLSI2I3D7G+xtFZIeI+ETkupDlF4nIzpDH\ngIhc67z3cxE5FPLeGjePwRhjzOm5NvxXROKBHwGXAQ3AdhHZrKrVIavVA18G/iZ0W1V9GVjj7CcX\nqAGeC1nlb1X1127VbowxZvLcPCNZD9Soaq2qDgGPAteErqCqh1V1F+A/zX6uA36vqn3ulWqMmYk/\nHmhjR/1xr8swHnEzSIqBIyGvG5xlU3Uj8MioZf8oIrtE5B4RSR5rIxG5VUQqRaSytbV1Gh9rjJmM\nV99r5ZafvcXNP32T3Q2dXpdjPBDRne0iUgicDWwNWXwHsAJYB+QC3xlrW1W9V1XXqura/PwJr/A3\nxkzD/mPdfPNXO1hekEFuehJfe3A7R0/0e12WCTM3g6QRWBTyusRZNhU3AE+p6nBwgao2acAg8DMC\nTWjGmDBr7R7kqz/fTmpSPA98eR0/+8o6+odG+OrPt9Mz6PO6PBNGbgbJdmC5iCwWkSQCTVSbp7iP\nmxjVrOWcpSAiAlwL7JmFWo0xU/RXj71DR+8Q99+yjqJ5qZyxIJMf3XweB1p6+H+f2u11eSaMXAsS\nVfUBtxFoltoLPK6qVSJyl4hcDSAi60SkAbge+ImIVAW3F5FyAmc0fxi161+JyG5gNzAfuNutYzDG\njO1Y5wB/qmnntouXcXZJ9snlG8/I5wsXlPL7PcfoHxrxsEITTq7O/quqW4Ato5bdGfJ8O4Emr7G2\nPcwYnfOqevHsVmmMmaqX97cAcFnFglPeu7RiAQ++UcfrB9u4ZOWp75vYE9Gd7caYyPTi3hZKclJZ\nXpBxynvrF+eSnhTPi/taPKjMeMGCxBgzJQPDI/yppo2LVxQQ6Kr8oOSEeC5cns/L+1pQVQ8qNOFm\nQWKMmZJtte30D49w8YqCcde5eEUBTZ0D7G3qDmNlxisWJMaYKXlpXwupifFsWJI37jofX5HvrNsc\nrrKMhyxIjDGTpqq8uLeFjyybT0pi/LjrFWSmcE5JtvWTzBEWJMaYSTvQ0kPjif7TNmsFXbSigJ1H\nTtDeMxiGyoyXLEiMMZP24t7AGcZkguSSFQtQhVf221x3sc6CxBgzaS/va2FVURYLs1MmXHdVURYF\nmcm8tN+at2KdBYkxZlIGhkfYUX+cC5dPbhLUuDjhwuX5vFnbbsOAY5wFiTFmUt5r7sbnV1aHTIky\nkbOLs2jrGaKl2/pJYpkFiTFmUqqOdgGBJqvJWlWc7Wxr9ymJZRYkxphJqTraSUZyAoty0ia9zYqF\nmYFtG7vcKstEAAsSY8ykVB3toqIwi7i4U6dFGU9mSiLleWknz2ZMbLIgMcZMaMSv7GvqpmIKzVpB\nq4qyqW6yIIllFiTGmAkdauulf3hkSv0jQRVFWdR39NE1MDzxyiYqWZAYYyYU7CxfVTT5EVtBwfCp\ntuatmGVBYoyZUPXRLpLi41i+4NT7j0wkGD7WTxK7LEiMMROqOtrFGQszSIyf+q+M/MxkCjKTbQhw\nDLMgMcaclqpSdbSTVYVTb9YKWlWUZU1bMczVIBGRTSKyX0RqROT2Md7fKCI7RMQnIteNem9ERHY6\nj80hyxeLyJvOPh8TkSQ3j8GYua6pc4DjfcPTGrEVVFGUxYGWHgaGR2axMhMpXAsSEYkHfgRcCVQA\nN4lIxajV6oEvAw+PsYt+VV3jPK4OWf594B5VXQYcB74268UbY06qnsYV7aOtKspmxK8caO6ZrbJM\nBHHzjGQ9UKOqtao6BDwKXBO6gqoeVtVdgH8yO5TADaIvBn7tLHoQuHb2SjbGjFZ1tAsRWFk4kyDJ\ncvZl/SSxyM0gKQaOhLxucJZNVoqIVIrINhEJhkUecEJVfRPtU0RudbavbG21+yEYM11VRztZnJdO\nenLCtPexKCeNzOQEG7kVoyK5s71MVdcCnwd+ICJLp7Kxqt6rqmtVdW1+/uSmvTbGnKrqaNeM+kcg\nMKX8yqIsOyOJUW4GSSOwKOR1ibNsUlS10fm3FngFOBdoB+aJSPBPoynt0xgzNd0DwzSe6J9Rs1ZQ\nRWEW+451271JYpCbQbIdWO6MskoCbgQ2T7ANACKSIyLJzvP5wEeAag18B74MBEd43QI8PeuVG2MA\nqG3tBWBZwdQvRBxtaUEGfUMjNHfZvUlijWtB4vRj3AZsBfYCj6tqlYjcJSJXA4jIOhFpAK4HfiIi\nVc7mK4FKEXmXQHD8s6pWO+99B/i2iNQQ6DO5361jMGauq20LjLJaMj99xvsK7qO21UZuxZrp955N\ngqpuAbaMWnZnyPPtBJqnRm/3OnD2OPusJTAizBjjstrWXuIESvMmfw+S8SzJDwTJwbZePrxs/oz3\nZyJHJHe2G2M8Vtvay6LcNJIT4me8r4VZKaQlxdsZSQyyIDHGjOtga8+sNGsBiAiL56ef7HcxscOC\nxBgzJr9fOdzey5L8mXe0By3JzzjZ72JihwWJMWZMRzv7GRj2n+zbmA1L5qfTcLzf5tyKMRYkxpgx\nHWoLNEEtmT+bZyTpqEJde9+s7dN4z4LEGDOmYF/G0lk8I1nqNJNZh3tssSAxxoyptrWHjOQE8jOT\nZ22fi4PXkrRZh3sssSAxxoyptq2XJfnpBCbdnh3pyQkszErhoJ2RxBQLEmPMmGpbe2dt6G+oJfk2\nBDjWWJAYY07RPzRC44n+WR36GxQIkh6bvDGGWJAYY05xcsTWLHa0By2Zn0HXgI/23qFZ37fxhgWJ\nMeYUwYsGF7vQtLU4Pzh5ozVvxQoLEmPMKYK/5N0IkqXzbQhwrLEgMcacora1h6LsFNKSZn+C8OKc\nVJIS4k42n5noZ0FijDlFYOjv7He0A8THCeV5aRy0pq2YYUFijPkAVeVQa68rHe1BS+bb5I2xxILE\nGPMBbT1DdA/6XLmGJGhJfjr17X34RvyufYYJHwsSY8wH1LUHmpzKXAyS8rx0fH7l6IkB1z7DhI8F\niTHmA4Iz85blzvz2uuMJ3rq3rsP6SWKBq0EiIptEZL+I1IjI7WO8v1FEdoiIT0SuC1m+RkTeEJEq\nEdklIp8Lee/nInJIRHY6jzVuHoMxc01dRx9xAiU57gVJWTBIbDr5mDD7Y/scIhIP/Ai4DGgAtovI\nZlWtDlmtHvgy8DejNu8DvqSqB0SkCHhbRLaq6gnn/b9V1V+7Vbsxc1ldey+F2YEhum5ZkJlCUkIc\n9R0WJLHAtSAB1gM1qloLICKPAtcAJ4NEVQ87732gx01V3wt5flREWoB84ATGGFfVtfedPGNwS1yc\nUJqbdrI/xkQ3N5u2ioEjIa8bnGVTIiLrgSTgYMjif3SavO4Rkdm7WYIxhvoO94MEAn0w1rQVGyK6\ns11ECoFfAF9R1eBZyx3ACmAdkAt8Z5xtbxWRShGpbG1tDUu9xkS77oFhOnqHKM11b8RWUGleGvUd\nfTYLcAxwM0gagUUhr0ucZZMiIlnAM8D3VHVbcLmqNmnAIPAzAk1op1DVe1V1raquzc/Pn9YBGDPX\nnByxFaYzkr6hEVp7Bl3/LOMuN4NkO7BcRBaLSBJwI7B5Mhs66z8FPDS6U905S0ECt227Ftgzq1Ub\nM4cFO79LXRz6G1SWFzjrqbfmrajnWpCoqg+4DdgK7AUeV9UqEblLRK4GEJF1ItIAXA/8RESqnM1v\nADYCXx5jmO+vRGQ3sBuYD9zt1jEYM9eE84yk1IYAxww3R22hqluALaOW3RnyfDuBJq/R2/0S+OU4\n+7x4lss0xjjqO3rJTU8iMyXR9c8qyUlFJHDdioluEd3ZbowJr3AM/Q1KToinKDuVehsCHPUsSIwx\nJ9W197k6NcpoZXlpdkYSAyxIjDEADPpGaOrspzTP/aG/QWV5adbZHgMsSIwxADQc78ev7k7WOFpp\nbjrtvUP0DPrC9plm9lmQGGOA94fhhquPJPSzbKqU6GZBYowB3v9lXhrGIAler2LNW9HNgsQYAwSG\n4aYlxZOfEb7p606ekViHe1SzIDHGAIGzgtLcNAKTRoRHZkoiuelJdlFilLMgMcYAgbOCcEyNMlpp\nbhr1dqfEqGZBYozB79ewTR8/WlmeTScf7SxIjDE0dw8w5POH9RqSoLLcNI6e6GfI5594ZRORLEiM\nMe9P1uhF01ZeOn6FxhP9Yf9sMzssSIwxJ6eP96ppK7QGE30sSIwx1Lf3ER8nFM1LDftnv38tiXW4\nRysLEmMM9R19FM1LITE+/L8SCjKTSU6IszOSKGZBYoyh3qOhvwAi4gwBtiCJVhYkxhgnSMI/YivI\nhgBHNwsSY+a47oFhOnqHPDsjAViUm8aRjj5U1bMazPRZkBgzxwWblLwMktLcNHqHRmjvHfKsBjN9\nrgaJiGwSkf0iUiMit4/x/kYR2SEiPhG5btR7t4jIAedxS8jy80Vkt7PP/5BwTgxkTAw64uHQ3yAb\nAhzdXAsSEYkHfgRcCVQAN4lIxajV6oEvAw+P2jYX+DvgAmA98HcikuO8/WPg68By57HJpUMwZk4I\n9k0s8viMBGw6+Wjl5hnJeqBGVWtVdQh4FLgmdAVVPayqu4DRcyNcATyvqh2qehx4HtgkIoVAlqpu\n00Bj6kPAtS4egzExr76jj3lpiWSnJnpWQ0mOnZFEMzeDpBg4EvK6wVk2k22LnefT2acxZgxeDv0N\nSkmMZ2FWio3cilIx29kuIreKSKWIVLa2tnpdjjERKxKCBAJ3ZjxiZyRRyc0gaQQWhbwucZbNZNtG\n5/mE+1TVe1V1raquzc/Pn3TRxswlvhE/jcf7IyNIctOos/uSRCU3g2Q7sFxEFotIEnAjsHmS224F\nLheRHKeT/XJgq6o2AV0issEZrfUl4Gk3ijdmLmjqHMDnV09HbAWV5abR3DXIwPCI16WYKXItSFTV\nB9xGIBT2Ao+rapWI3CUiVwOIyDoRaQCuB34iIlXOth3APxAIo+3AXc4ygL8A7gNqgIPA7906BmNi\nXbBz28sRW0GlTpg1HLfmrWiT4ObOVXULsGXUsjtDnm/ng01Voes9ADwwxvJK4KzZrdSYuSnYuR0J\nTVvBMKtr72NZQabH1ZipiNnOdmPMxOo7+kiMFwqzwz99/GjBm2rZEODoM6UgEZF050JDY0wMqO/o\npSQnjfg47yeIyE1PIj0p3oYAR6HTBomIxInI50XkGRFpAfYBTSJSLSL/KiLLwlOmMcYNkTL0F5zp\n5PPSbQhwFJrojORlYClwB7BQVRepagHwUWAb8H0R+YLLNRpjXKCq1LVHTpAAlOamUmdBEnUm6my/\nVFWHRy90RlA9ATwhIt7Nq2CMmbbO/mG6B3wRMfQ3qCwvnVf2t+L3K3ER0NxmJue0ZyTBEBGRS0e/\nF5yRd6ygMcZEvkiYrHG0RblpDPr8tHQPel2KmYLJdrbfKSI/djrbF4jIb4FPuVmYMcZdwSak8jzv\n7ow4WnlecAiwXeEeTSYbJB8jcPHfTuCPwMOqet3pNzHGRLLDbYFf1pHURxIMtcMWJFFlskGSQ2Ba\n+IPAIFBmN5QyJrodbu9lYVYKqUmRM6K/MDuFxHjhsA0BjiqTDZJtwLOquglYBxQBf3KtKmOM6+ra\n+yifHzlnIwAJ8XEsyk2zpq0oM9kpUi5V1XoAVe0H/lJENrpXljHGbXXtvVy6coHXZZyiPC+dQ212\nRhJNJrogsRwgGCKhVPVVCRhzrixjTOTqHhimrWeIsgjqaA8qywuckQRugmqiwURnJP8qInEEpmp/\nG2gFUoBlwEXAJQTurd4w7h6MMREnOPR3cYQ1bUHgjKRvaITWnkEKMlO8LsdMwmmDRFWvF5EK4Gbg\nq8BCoJ/AtPBbgH9U1QHXqzTGzKrgqKhIPCMpn++M3GrrsyCJEhN2tqtqNXA38FsCAXKIwD1Cfm0h\nYkx0Cg79jaSr2oOC15LYEODoMdnO9geBLuA/nNefBx4CbnCjKGOMuw6397EgK5m0JFdvSTQtxfNS\nSYgTG7kVRSb7XXSWqlaEvH5ZRKrdKMgY47669t6IbNaC94cAH7aRW1FjsteR7BCRDcEXInIBUOlO\nScYYtx1u7zvZhBSJyvLSrGkrikz2jOR84HURCQ4DLgX2i8huQFV1tSvVGWNmXc+gj9buwZOd2pGo\nPC+dysPHUVVsEo3IN9kg2eRqFcaYsAn2PUTSZI2jleWl0TPoo61niPzMZK/LMROYVNOWqtad7jHe\ndiKySUT2i0iNiNw+xvvJIvKY8/6bwQsgReRmEdkZ8vCLyBrnvVecfQbfK5jeoRszNwWvIYnEEVtB\nwbMl63CPDlO6Z/tUOPd2/xFwJVAB3ORckxLqa8BxVV0G3AN8H0BVf6Wqa1R1DfBF4JCq7gzZ7ubg\n+6ra4tYxGBOLDrVF/hnJ+7MAW4d7NHAtSAjMFlyjqrWqOgQ8Clwzap1rCAwtBvg1cMkYswrf5Gxr\nDDUtPfz3Hw5ysLXH61KiVl17L/mZyaQnR97Q36CSnFTi4+Tk9S4msrn5nVQMHAl53QBcMN46quoT\nkU4gD2gLWedznBpAPxOREQK3+71bx5iUR0RuBW4FKC0tncFhmEhwsLWH/3jxAJvfPYoq/Muz+7hm\nTTG3XbyMpfkZXpcXVSJ9xBZAYnwcJTmpNnIrSrh5RjJjzjDjPlXdE7L4ZlU9G7jQeXxxrG1V9V5V\nXauqa/Pz88NQrXHL1qpjXPbvf+C5qmZu3biE57+1ka9fuIRn9wSXH/O6xKhS194b0c1aQWV56Sf7\nc0xkczNIGoFFIa9LnGVjriMiCUA20B7y/o3AI6EbqGqj82838DCBJjQTo1q6Brj9iV2cVZzNH79z\nEXdcuZLlCzK546qVvPadi6goyuL2J3fT0m2z9UxG35CP5q7IHvobVJ6XxuE2mwU4GrgZJNuB5SKy\nWESSCITC5lHrbAZucZ5fB7wUbKZyZh2+gZD+ERFJEJH5zvNE4JPAHkxMUlVuf3I3fUMj/PsNa8jL\n+OAw0PkZydxzwxp6Bn1898nd9gtnEqJhxFZQeV463YM+OnqHvC7FTMC1IFFVH3AbsJXAZI+Pq2qV\niNwlIlc7q90P5IlIDfBtIHSI8EbgiKrWhixLBraKyC4C949vBH7q1jEYbz26/Qgv7Wvh9itXsKxg\n7H6Q5Qsy+Z9XnMkLe1t4vPLImOuY9x2OghFbQcG7Nx6yDveI5+qwDVXdQmC6+dBld4Y8HwCuH2fb\nV4ANo5b1ErjK3sS4Ix19/MPvqvnw0jxu+VD5adf96kcW88LeZu76bTUfXjqfRbmR/9e2V4Kj3Zbk\nR36QBAdR1Lb2srY81+NqzOlEdGe7mbv+86UaRvzKv15/DnFxp58iIy5O+P+uPwefX/mvV2rCVGF0\nqmnpoXheakTO+jtaSU4aSQlx1NhQ74hnQWIiTlvPIE/tbOS680sonpc6qW1KctL4zHklPLmjkfae\nQZcrjF41rT0sHaeZMNLExwlL5qdT02JBEuksSEzE+eW2OoZ8fr760cVT2u5rHy1n0OfnV2/WT7zy\nHOT3KwdbelkWRdfdLC3IsItPo4AFiYkoA8Mj/OKNOi5ZUTDlCw2XFWRy0Zn5PPTGYQaGR9wpMIo1\ndQ3QPzzC0oLI7x8JWpqfwZGOPvt6RjgLEhNRnt7ZSHvvEF+7cGpnI0F/duES2nqG2Pzu0VmuLPoF\nm4ii6YxkWUEGfrWRW5HOgsREDFXlvtcOUVGYxYeW5E1rHx9emseKhZnc/9ohu65klGCQREsfCbwf\neta8FdksSEzEePVAGwdaevizCxdP+2ZGIsKfXbiE/c3d/LGmbeIN5pCDrT3MS0skLz3J61ImbUl+\nOiJYh3uEsyAxEePx7UfIS0/ik6uLZrSfT51TSG56Eo9ttwsUQ9W09LAsPyOq7jiYkhhPSU6qBUmE\nsyAxEaF7YJgX9jbzydWFJCXM7NsyOSGeT5xdyAt7m+kZ9M1ShdHvYEtPVM6UvCw/g4Ot1kcSySxI\nTER4ds8xBn1+rl5TPCv7u/bcIgaG/WzdYzMDAxzvHaK9d2jcqWYi2dL8DGpbexjxW59XpLIgMRHh\n6Z1HKc1N47zSebOyv/NKcyjJSeU3O0dPOD03BTurozFIlhVkMOjzc/REv9elmHFYkBjPtXQN8PrB\nNq5ZUzRr7fciwjVrivhTTZtNMU/IiK1obNpyws/6SSKXBYnx3G93NeFXuGaWmrWCrl1TjF/hmV1N\ns7rfaFTT0kNyQhzFOZObciaSBMPPgiRyWZAYzz29s5GzirNmvdll+YJMKgqz+M1OuzjxYGsPi+en\nEz/BBJiRKCc9ibz0JLuWJIJZkBhP1bb2sKuhk2tn+Wwk6Npzi3j3yIk5f2V0TWtPVPaPBC0tyLAz\nkghmQWI89fTOo4jAp86Z2bUj47n6nGJEYPMcPisZGB6h4Xh/VAfJsoIMalp7bLaCCGVBYjz17J5j\nrCvPZUFWiiv7X5idwrqyXH6/Z+72k9S29qIanR3tQUvzMzjRN2y33Y1QFiTGM4faetnf3M2mVQtd\n/ZzLVy1g37Fu6trnZvNWTRQP/Q0K1n7AmrcikgWJ8czWqsDFglec5W6QXOEEVfDz5pp9TV0kxElU\n3F53PGcuyARg/7FujysxY3E1SERkk4jsF5EaEbl9jPeTReQx5/03RaTcWV4uIv0istN5/HfINueL\nyG5nm/+QaJo4yHzAs3uOsboke9J3QZyuRblpnFWcxbNz9Cr36qYulhVkkJwQ73Up07YgK5nc9CSq\nj3Z5XYoZg2tBIiLxwI+AK4EK4CYRqRi12teA46q6DLgH+H7IewdVdY3z+EbI8h8DXweWO49Nbh2D\ncU9TZz87j5w4ebbgtk2rFrKj/gTNXXPv4sTqo11UFGV5XcaMiAirirKobrIgiURunpGsB2pUtVZV\nh4BHgWtGrXMN8KDz/NfAJac7wxCRQiBLVbdpYPjGQ8C1s1+6cdtzVc0AbHK5WSso+DnPzbHmrdbu\nQVq6B6kojO4gAagozGJ/czfDI36vSzGjuBkkxUDoPN4NzrIx11FVH9AJBO9otFhE3hGRP4jIhSHr\nN0ywTwBE5FYRqRSRytbW1pkdiZl1z+45xrKCjLCNJFpWkMmS/HSenWNBstf5Cz7az0ggcAxDPj+1\nNhNwxInUzvYmoFRVzwW+DTwsIlP6SVDVe1V1raquzc/Pd6VIMz0dvUO8dbjD9dFao21atZBttR2c\n6Js7Q0iDTUGxckYCUN3U6XElZjQ3g6QRWBTyusRZNuY6IpIAZAPtqjqoqu0Aqvo2cBA4w1m/ZIJ9\nmgj3wt5mRvwatmatoE1nLWTEr7ywtyWsn+ulqqNdFM9LZV5a9NwVcTyL56eTnBBnHe4RyM0g2Q4s\nF5HFIpIE3AhsHrXOZuAW5/l1wEuqqiKS73TWIyJLCHSq16pqE9AlIhucvpQvAU+7eAzGBVv3HKN4\nXiqrwtzccnZxNkXZKXNq9Fb10U5WxsDZCEBCfBwrFmZSZUEScVwLEqfP4zZgK7AXeFxVq0TkLhG5\n2lntfiBPRGoINGEFhwhvBHaJyE4CnfDfUNUO572/AO4DagicqfzerWMws6930MdrNW1cvmpB2G/5\nKiJcvmohrx1opW8o9u+c2Dfko7atN+yB7aYKZ+SWTZUSWRLc3LmqbgG2jFp2Z8jzAeD6MbZ7Anhi\nnH1WAmfNbqUmXP7wXitDPn/Yhv2OdvmqBfz89cO8+l5b2JvWwm3/sW5UY6OjPaiiMItH3jpCU+cA\nRS5ff2QmL1I7202Meq7qGDlpiawty/Hk89eX55KdmjgnhgHHUkd7UDAUrZ8ksliQmLAZHvHz4r4W\nLlm5gIR4b771EuLjuGRlAS/ua4n56xGqj3aRmZJASRTezGo8Zy7MQgS7MDHCWJCYsNlW2073gM+z\nZq2gK1YtpLN/mO2HOiZeOYpVN3VRUZgV9r4oN2UkJ1Cel25nJBHGgsSEzXNVzaQmxnPh8vme1rFx\neT4piXExPYnjiF/Z19QdU/0jQRWFNlVKpLEgMWHh9yvPVR9j4xnzSUn0dvLA1KR4Llyez3PVzTE7\n+udwey/9wyMx1T8SVFGURX1HH10Dw16XYhwWJCYsdjV20tw16HmzVtAVqxbS1DnAnsbY/Ms2eK3F\nqqJsjyuZfcGzrH1NNqV8pLAgMWGxteoY8XHCxSsKvC4FgEtWFBAfJzHbvLXryAmSEuKi+mZW4wle\nF/PukRMeV2KCLEiM61SVZ/ccY8OS3IiZqiMnPYn15YFb8MZi81Zl3XHWlMwjKSH2fsQLMlMozU2j\nsi62B0tEk9j7LjMRZ9+xbg619XLlWYVel/IBV529kIOtvTF3+9b+oRH2NHZyfrk31+qEw9ryHCoP\nH4/JPwKikQWJcd2W3U3ESfjuPTJZV5y1EBF4ZleT16XMqp1HTuDzK+tiOEjWlefS3jvEoTabUj4S\nWJAYV6kqz+xu4oLFeczPSPa6nA8oyExhfXkuW3bHVpBUHg40+ZxfmutxJe4JhmTl4eMeV2LAgsS4\n7L3mHmpbe7lqdWQ1awVddXYhB1p6ONAcOyOAKuuOc+aCTLLTEr0uxTVL8zPISUtk+2HrJ4kEFiTG\nVc/sbkKEsN/EarKudJq3tuyOjdFbI35lR91x1sZwsxYEZnI+vyyXt+vsjCQSWJAYV/1+dxPry3PJ\nz4ysZq2ggqwU1pXFTvPW/mPmGV1kAAAS6ElEQVTddA/6Yj5IINC8VdvWS1vPoNelzHkWJMY1B5q7\nOdDSwycitFkr6MqzF7K/uZuaGBi9FRwSu7YsdvtHgtaWB47R+km8Z0FiXHOyWSvCRmuNFhyWHAtn\nJdsPH2dhVkpMzfg7nrOKs0hKiDs5uMB4x4LEuEJV+d2uJtaV51KQmeJ1Oae1MDuFtWU5/Pbdo1F/\nXcLbhztYW54TUzP+jic5IZ41JfPYbv0knrMgMa7Y1dBJTUsPnz632OtSJuXac4s50NIT1XNvNZ7o\n52jnAOvKY79ZK2hteQ5VjZ1z4tbJkcyCxLjiiR0NJCfERXz/SNCnVheRlBDHEzsavC5l2k5eP+LR\n3Se9sK48F59f2WnzbnnK1SARkU0isl9EakTk9jHeTxaRx5z33xSRcmf5ZSLytojsdv69OGSbV5x9\n7nQekTELoDlp0DfC5nePcvmqhWSlRMe1DNlpiVy2cgFP72xkyBedd058vaadzOQEVizM9LqUsDmv\nLIc4gW0H270uZU5zLUhEJB74EXAlUAHcJCIVo1b7GnBcVZcB9wDfd5a3AZ9S1bOBW4BfjNruZlVd\n4zxa3DoGMz0v72vhRN8wnz0vOpq1gj57fjHH+4Z5eX/0fUv5/cqL+1rYeGa+Z7cx9kJ2aiJry3J5\nYW/0fc1iiZvfceuBGlWtVdUh4FHgmlHrXAM86Dz/NXCJiIiqvqOqR53lVUCqiETmhQjmFL9+u5GC\nzGQ+uszbOyFO1cbl+czPSOaJt6OveevdhhO09Qxy2coFXpcSdpdWFFDd1EXjiX6vS5mz3AySYuBI\nyOsGZ9mY66iqD+gE8kat81lgh6qGXnX0M6dZ63/JXBieEkXaewZ5ZX8L155bHHV/GSfEx3HtmiJe\n3t9CR++Q1+VMyQt7m4mPEy46c+619F7qhOcL1c0eVzJ3RfRPuoisItDc9echi292mrwudB5fHGfb\nW0WkUkQqW1tb3S/WAPD0zqP4/MpnzyvxupRp+ez5JQyPKJt3NnpdypQ8X93M+vLcmJ5fazxL8jNY\nkp/OC3stSLziZpA0AotCXpc4y8ZcR0QSgGyg3XldAjwFfElVDwY3UNVG599u4GECTWinUNV7VXWt\nqq7Nz8+flQMyp6eq/N+3GzirOIszo7TDd2VhFhWFWTxe2RA115TUtffyXnMPl1bMvWatoMtWLmBb\nbbvdx90jbgbJdmC5iCwWkSTgRmDzqHU2E+hMB7gOeElVVUTmAc8At6vqn4Iri0iCiMx3nicCnwT2\nuHgMZgreOtTB3qYublpf6nUpM3LzhlKqm7qojJIL3YIdzZeunHvNWkGXVixgeER59T1rffCCa0Hi\n9HncBmwF9gKPq2qViNwlIlc7q90P5IlIDfBtIDhE+DZgGXDnqGG+ycBWEdkF7CRwRvNTt47BTM0D\nfzrEvLREPnNudDZrBX3m3BKyUxN54I+HvC5lUl6obuaMBRmU5aV7XYpnzivNITc9yfpJPJLg5s5V\ndQuwZdSyO0OeDwDXj7Hd3cDd4+z2/Nms0cyO+vY+nqtu5i8+vpTUpHivy5mR1KR4Pn9BKT/5w0GO\ndPSxKDfN65LG1dk3zFuHO/jzjUu8LsVT8XHCxSsKeK7qGMMjfhKjbKBHtLP/bTMrHnzjMPEifHFD\nudelzIovfagMEeHB1w97XcppvfJeCyN+ndP9I0GXrlxA14DPZgP2gAWJmbHugWEe236ET6wuZGF2\nZE/QOFmF2alcdXYhj20/Qs9g5M7j9Nt3m8jPTGZNyTyvS/Hchcvnk5wQx+92HZ14ZTOrLEjMjP3f\nygZ6Bn185SOLvS5lVn31I+V0D/r4deWRiVf2QMPxPl7a18z155cQF2eXU6UnJ/CJ1YX85p1Gum30\nVlhZkJgZGfL5+dnrhzi/LIc1i2Lrr+JzS3M4r3QeD/zpMMMjkTf/1sNv1gNw84YyjyuJHF/6UDm9\nQyM89U50XQcU7SxIzIw88lY9Rzr6ue2iZV6X4orbLl5GfUcfj7xV73UpHzDoG+Gx7Ue4ZOUCiufF\n/k2sJmvNonmsLsnmF2/URc11QLHAgsRMW/fAMD988QAfWpLHx8+MzYs+LzqzgAsW5/LDFw5EVF/J\nlt1NtPcO8aUP2dnIaF/YUMaBlh621dqdE8PFgsRM272v1tLRO8QdV62I2TvyiQh3XLWS9t4h7v3D\nwYk3CJOH3qhjyfx0PrI0uibGDIerzyliXloiv9xW53Upc4YFiZmW5q4BfvpaLZ86p4jVMT5iaM2i\neXxydSE/fe0QzV0DXpfDnsZO3qk/wRc2lFkn+xhSEuO5Ye0itlYdi4iv11xgQWKm5Z7n32PEr/zt\n5Wd6XUpY/O0VZ+Lz+/nBC+95XQoP/PEQqYnxfPb86J5BwE03X1DKiCoPvXHY61LmBAsSM2Vv1rbz\nWOURvrChjNK8yL3qezaV5aXzxQ3lPLr9CNtqvbsb3zv1x3nynUa++KEyslPn3ky/k1WWl84nzi7k\nvtcOcaSjz+tyYp4FiZmSzv5hvvXYTspy0/ibOXI2EvTXl59BeV46335sJ5194b9OYcSv3Pl0FQuy\nkvnLS5aH/fOjzXevWkmcCHc/U+11KTHPgsRMmqryvad209I9yA9vPJf0ZFenaos46ckJ/OBza2jp\nHuS7v9kd9uGlj26vZ3djJ9+9aiUZc+z/fjqK5qXy/1yyjK1VzfzBZgV2lQWJmbSn3mnkd7ua+NZl\nZ3BOjF18OFnnLJrHty47g2d2NfHEjvBd9Ha8d4h/3bqfCxbncvU5RWH73Gj3tY8uZvH8dP5+cxWD\nvhGvy4lZFiRmUnY1nOB//WYP68tz+cbHlnpdjqe+8bGlXLA4lzuf3sOOevcnCFRV/uGZaroHfNx1\nzVkxO9TaDckJ8fzdpyo41NbLf79S63U5McuCxEyo6mgnX7z/LXLSk/jhTWuIn+NDTuPjhP9z07nk\nZyZzy/1vsavhhKuf958v1fDkjka++fGlUXvnSS99/MwCrj6niB+8+B7P7GryupyYZEFiTmv/sW6+\neP9bpCfF88jXN1CYbdNxABRkpfDI1zcwLz2RL9z3JnsaO135nEffquffnn+Pz5xXzLcuO8OVz5gL\n/uW61ZxfmsO3HtvJmx6OuotVFiRmXNtq27n5vm0kxAkPf31DRN/gyQtF81J55OsbyExJ5Ob73pz1\nu/M9V3WM7z61m4+dkc/3P7vamrRmICUxnvtuWcui3FS+/lAl7zV3e11STLEgMacYHvHzb8/t56af\nbiMzJZGHv76B8vlz9zaup1OSk8ajt26geF4qf/ZQJXc8uYveGc7JNegb4Z+27OXPf/k2Z5fM479u\nPs/u+DcL5qUl8fOvrCc5MZ7rfvw6v99tzVyzRebCDJlr167VyspKr8uICm/XHefuZ6p5p/4E159f\nwt9fvWrODfOdjiGfn39//j1+8upBFuWkcdtFy7h6TREpiVO77fCexk7++vF32d/czecvKOV7V620\n//9ZVtfey18+8g7vNnRy0/pS7vxkRdTfHtotIvK2qq6dcD03g0RENgE/BOKB+1T1n0e9nww8ROA+\n7O3A51T1sPPeHcDXgBHgL1V162T2ORYLktNTVd442M5/vlzD6wfbyU1P4n9fvYpP2TDTKXvrUAd3\nPr2Hfce6mZ+RxOcvKOPiFQVUFGaRlDD2WUXPoI9ndh3l8coG3q47Tn5mMv/y2dVctKIgzNXPHcHg\n/+8/HKR4Xipf+Ug5n1u3iMwUmy0glOdBIiLxwHvAZUADsB24SVWrQ9b5C2C1qn5DRG4EPq2qnxOR\nCuARYD1QBLwABHsaT7vPsViQnKpn0Mc79cd5vrqZ56ubaeocoCAzmVs3LuHzF5SSlmR/BU+XqvL6\nwXbu/+MhXtrXAkByQhyrS7LJS08mLSmepIQ4jnYOUNvaQ+OJflRhaX46n1u3iBvWLmJeWpLHRzE3\nvH6wjR88f4C3DneQmZzAp88r5mNn5HPBkjy76JPICJIPAX+vqlc4r+8AUNV/Cllnq7POGyKSABwD\n8oHbQ9cNrudsdtp9jmUuBYmqMujz0zvoo2vAR2f/MB29gxw9MUBTZz917X1UH+3iUHsvqpCSGMfG\n5flcsWohn1hdOOWmGHN6LV0DVNYd5+2647x75ARdA8P0DY0wMOxnQVYyS/IzWDI/nY1n5HNe6Tzr\nUPfIziMnuO+1Wp6vbmbQ5ychTlhVlMXSgsDXpywvnbyMJPLSk8lJSyQtOYG0xPiYn315skHiZuQW\nA6E3u24ALhhvHVX1iUgnkOcs3zZq22Ln+UT7nDXfe2o3bx2a/ZvjjBfdoaH+gXU08FpVUcCvit8f\neO3zBx7DI36GfH4GfePfEjY+TijMTqGiMItrzy3m7OJsNizJs/ZhFxVkpXDV2YVcdXah16WY01iz\naB7/+fnzGBgeYUfdcf50sI136k/wek07T55mBoPkhDiS4uNITIgjIU6IjxPiJPCvCMSJIAAheRMa\nPeP94TCb8XT/Letcn1w1Zs/dRORW4FaA0tLSae2jaF4qyxdkzGZZJ8l43yqn+YYTQATiRRAR4iQQ\nDgnxQkJcHMkJcSQnxpOSGEdaYjzZaYlkpyYyLy2JouxU8jOT5/zFhMacTkpiPB9eNp8PL3v/hmG9\ngz4ajvfT3jtIR+8Qx/uG6R/y0Ts4wsDwCEMjfnwjgT/mRvzKiCp+vzp//AX+8Asa/QfiWHTcPzWn\nZ7y+udnkZpA0AotCXpc4y8Zap8Fp2som0Ol+um0n2icAqnovcC8EmramcwDfjNH7kBtjJi89OcGZ\nUcBmFRiPm1G1HVguIotFJAm4Edg8ap3NwC3O8+uAlzTQvrMZuFFEkkVkMbAceGuS+zTGGBNGrp2R\nOH0etwFbCQzVfUBVq0TkLqBSVTcD9wO/EJEaoINAMOCs9zhQDfiAb6rqCMBY+3TrGIwxxkzMLkg0\nxhgzpsmO2rJ5F4wxxsyIBYkxxpgZsSAxxhgzIxYkxhhjZsSCxBhjzIzMiVFbItIK1E1z8/lA2yyW\n46VYOZZYOQ6wY4lUsXIsMz2OMlXNn2ilOREkMyEilZMZ/hYNYuVYYuU4wI4lUsXKsYTrOKxpyxhj\nzIxYkBhjjJkRC5KJ3et1AbMoVo4lVo4D7FgiVawcS1iOw/pIjDHGzIidkRhjjJkRC5JxiMj1IlIl\nIn4RWTvqvTtEpEZE9ovIFV7VOB0i8vci0igiO53HVV7XNBUissn5f68Rkdu9rmcmROSwiOx2vg5R\nNauoiDwgIi0isidkWa6IPC8iB5x/c7yscTLGOY6o/BkRkUUi8rKIVDu/u/6Hs9z1r4sFyfj2AJ8B\nXg1dKCIVBKa7XwVsAv5LRKLtXrX3qOoa57HF62Imy/l//hFwJVAB3OR8PaLZRc7XIdqGmv6cwPd/\nqNuBF1V1OfCi8zrS/ZxTjwOi82fEB/y1qlYAG4BvOj8frn9dLEjGoap7VXX/GG9dAzyqqoOqegio\nAdaHt7o5az1Qo6q1qjoEPErg62HCTFVfJXAPoVDXAA86zx8Erg1rUdMwznFEJVVtUtUdzvNuYC9Q\nTBi+LhYkU1cMHAl53eAsiya3icgu57Q+4psfQsTC/30oBZ4TkbdF5Favi5kFC1S1yXl+DFjgZTEz\nFK0/IwCISDlwLvAmYfi6zOkgEZEXRGTPGI+o/it3guP6MbAUWAM0Af/mabFz20dV9TwCTXXfFJGN\nXhc0W5xbZkfrkNCo/hkRkQzgCeCvVLUr9D23vi6u3Wo3GqjqpdPYrBFYFPK6xFkWMSZ7XCLyU+B3\nLpczmyL+/34qVLXR+bdFRJ4i0HT36um3imjNIlKoqk0iUgi0eF3QdKhqc/B5tP2MiEgigRD5lao+\n6Sx2/esyp89IpmkzcKOIJIvIYmA58JbHNU2a840U9GkCgwqixXZguYgsFpEkAoMeNntc07SISLqI\nZAafA5cTXV+LsWwGbnGe3wI87WEt0xatPyMiIsD9wF5V/feQt1z/utgFieMQkU8D/wfIB04AO1X1\nCue97wFfJTBK4q9U9feeFTpFIvILAqfsChwG/jyk/TTiOUMxfwDEAw+o6j96XNK0iMgS4CnnZQLw\ncDQdi4g8AnycwOyyzcDfAb8BHgdKCcy2fYOqRnRH9jjH8XGi8GdERD4KvAbsBvzO4u8S6Cdx9eti\nQWKMMWZGrGnLGGPMjFiQGGOMmRELEmOMMTNiQWKMMWZGLEiMMcbMiAWJMcaYGbEgMcYYMyMWJMZ4\nQETWOZMCpjhXuVeJyFle12XMdNgFicZ4RETuBlKAVKBBVf/J45KMmRYLEmM84swXth0YAD6sqiMe\nl2TMtFjTljHeyQMygEwCZybGRCU7IzHGIyKymcBdHhcDhap6m8clGTMtc/p+JMZ4RUS+BAyr6sPO\nvehfF5GLVfUlr2szZqrsjMQYY8yMWB+JMcaYGbEgMcYYMyMWJMYYY2bEgsQYY8yMWJAYY4yZEQsS\nY4wxM2JBYowxZkYsSIwxxszI/w+tpuSEpwJ3cwAAAABJRU5ErkJggg==\n", 127 | "text/plain": [ 128 | "" 129 | ] 130 | }, 131 | "metadata": {}, 132 | "output_type": "display_data" 133 | } 134 | ], 135 | "source": [ 136 | "# plot the target distribution\n", 137 | "x_1 = np.linspace(-10,20,100)\n", 138 | "\n", 139 | "# normalization constant, calculated using Wolfram Alpha\n", 140 | "Z = 4\n", 141 | "\n", 142 | "plt.plot(x_1,p(x_1)/4)\n", 143 | "plt.xlabel('x')\n", 144 | "plt.ylabel('p(x)')" 145 | ] 146 | }, 147 | { 148 | "cell_type": "markdown", 149 | "metadata": {}, 150 | "source": [ 151 | "Now that we have a sense of the target distribution, let's look at the proposal distribution $w(x)$. In this example we'll use a Gaussian: $q(x^*|x) = N(x^*,\\sigma^2)$ where $\\sigma = 10$." 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": 40, 157 | "metadata": { 158 | "collapsed": true 159 | }, 160 | "outputs": [], 161 | "source": [ 162 | "# proposal distribution pdf\n", 163 | "sigma = 10\n", 164 | "q = lambda x, x_s: (1/(2*np.pi*sigma**2)**0.5)*np.exp(-(x_s-x)**2/(2*sigma**2))\n", 165 | "\n", 166 | "# sample from the proposal distribution\n", 167 | "q_sample = lambda x: np.random.normal(loc=x, scale=sigma)" 168 | ] 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "metadata": {}, 173 | "source": [ 174 | "Now let's define the acceptance probability $A(x,x^*)$" 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": 41, 180 | "metadata": { 181 | "collapsed": true 182 | }, 183 | "outputs": [], 184 | "source": [ 185 | "# Acceptance probability\n", 186 | "A = lambda x, x_s: min(1,((p(x_s)*q(x_s,x))/(p(x)*q(x,x_s))))" 187 | ] 188 | }, 189 | { 190 | "cell_type": "markdown", 191 | "metadata": {}, 192 | "source": [ 193 | "## Let's run the algorithm" 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": 42, 199 | "metadata": { 200 | "collapsed": true 201 | }, 202 | "outputs": [], 203 | "source": [ 204 | "# number of samples\n", 205 | "N = 10000\n", 206 | "\n", 207 | "# initialize the sample\n", 208 | "x = np.zeros(N)" 209 | ] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "execution_count": 43, 214 | "metadata": {}, 215 | "outputs": [ 216 | { 217 | "name": "stdout", 218 | "output_type": "stream", 219 | "text": [ 220 | "fraction done:, 0.0\n", 221 | "fraction done:, 0.1\n", 222 | "fraction done:, 0.2\n", 223 | "fraction done:, 0.3\n", 224 | "fraction done:, 0.4\n", 225 | "fraction done:, 0.5\n", 226 | "fraction done:, 0.6\n", 227 | "fraction done:, 0.7\n", 228 | "fraction done:, 0.8\n", 229 | "fraction done:, 0.9\n" 230 | ] 231 | } 232 | ], 233 | "source": [ 234 | "# time the simulation\n", 235 | "t0 = time.time()\n", 236 | "\n", 237 | "# run the MCMC algorithm\n", 238 | "for i in range(N-1):\n", 239 | " \n", 240 | " # sample from uniform for the acceptance probability\n", 241 | " u = np.random.uniform(low=0, high=1)\n", 242 | " \n", 243 | " # sample from the proposal distribution\n", 244 | " x_s = q_sample(x[i])\n", 245 | " \n", 246 | " # accept the proposal with probability A\n", 247 | " if u < A(x[i], x_s):\n", 248 | " x[i+1] = x_s\n", 249 | " else:\n", 250 | " x[i+1] = x[i]\n", 251 | " \n", 252 | " # print status updates \n", 253 | " if (i % (N/10)) == 0:\n", 254 | " print(\"fraction done:,\", float(i)/N)" 255 | ] 256 | }, 257 | { 258 | "cell_type": "code", 259 | "execution_count": 36, 260 | "metadata": {}, 261 | "outputs": [ 262 | { 263 | "name": "stdout", 264 | "output_type": "stream", 265 | "text": [ 266 | "time in seconds: 83.99115800857544\n" 267 | ] 268 | }, 269 | { 270 | "data": { 271 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3XmcFPWZ+PHPUz0HDPcx3DPcCKgI\niHe8FTAxohs1Hq/EZJM12cS9snFj4m/VmGQ3ukc2m5isxsQrGmXNGlGJeB8xHgwIozAcwwBzMMMM\nMMAwd3c/vz+6GttxYHqG7qmu6uf9evWL7qpvVT81zczT37NEVTHGGGMcrwMwxhiTGSwhGGOMASwh\nGGOMcVlCMMYYA1hCMMYY47KEYIwxBrCEYIwxxmUJwRhjDGAJwRhjjCvH6wB6Y/To0TplyhSvwzDG\nGF9Zs2bNHlUt7KmcrxLClClTKCkp8ToMY4zxFRHZmUw5azIyxhgDWEIwxhjjsoRgjDEGsIRgjDHG\nZQnBGGMMkGRCEJGlIrJZRMpF5JZu9p8jImtFJCwiVyZsP19E1iU82kTkcnffgyKyPWHf/NRdljHG\nmN7qcdipiISAe4CLgWpgtYisUNWNCcUqgS8B3048VlVfBea75xkJlAMvJBS5WVWfPJYLMMYYkxrJ\nzEM4FShX1QoAEXkcWAYcTgiqusPdFz3Kea4E/qiqLX2O1hiTVvFb6oqIx5EYLyTTZDQRqEp4Xe1u\n661rgN912fYjESkVkZ+ISH4fzmmMSZGWjjBX/s/bfPWhEsKRo323M0HVL53KIjIeOBFYlbD5u8Bs\n4BRgJPCdIxx7o4iUiEhJQ0ND2mM1JhupKjc/WcqanY28vKmeH60s8zok44FkEkINUJTwepK7rTeu\nBp5S1c74BlWt1Zh24AFiTVOfoKr3qeoiVV1UWNjjUhzGmD74xWvbeK60lu8snc2Xz5rCA2/tYHlJ\nVc8HmkBJpg9hNTBTRKYSSwTXANf18n2uJVYjOExExqtqrcQaKy8HPuzlOY0xKfDSxt38+wubWTZ/\nAl8/dxqRqLJ19yH+31MfMr1wMCdPHuF1iKaf9FhDUNUwcBOx5p4yYLmqbhCRO0XkMgAROUVEqoGr\ngHtFZEP8eBGZQqyG8XqXUz8qIh8AHwCjgR8e++UYY3ojGlVu/cMHzB0/lLs+Nw8RISfk8PPrFjB2\nWD63PW3f07JJUqudqupKYGWXbbclPF9NrCmpu2N30E0ntKpe0JtAjTGpt7H2ILsPtnPzktkMyA0d\n3j68II9rTy3m7uc3s/tgG2OHDvAwStNfbKayMVns9S2xgRrnzBr9iX3nzRoDwBtbbDBHtrCEYEwW\ne31LA8dPGMqYIZ+sAcwZP4TCIfmHk4YJPksIxmSpg22drN3ZyLmzuh+9JyKcO6uQN7fuIRLVfo7O\neMESgjFZ6s/lewhH9YgJAeDcWYUcaO1kXdX+fozMeMUSgjFZ6vUtDQzJz2HhUYaVnj1zNI5gzUZZ\nwlf3VDbGpIaq8vrmBs7q/DO5P7j6iOWGA/O5g9e3DOdbF8/qvwCNJ6yGYEwWKq8/xK4DbZzrrO+x\n7Lmh9ZRW72dfc0c/RGa8ZAnBmCx0eLhpqLTHsuc6pajCm1ut2SjoLCEYk4Ve29zAzDGDmSh7eyx7\nolQwoiCX1zdbQgg6SwjGZJlwJMrqHfv41MxPTkbrTkiUs2aM5u2KnpOH8TdLCMZkme17mmkPRzlx\n4rCkjzlh4jBqD7Sxv8X6EYLMEoIxWWZj7UEA5owfmvQx8bJltU1piclkBksIxmSZstomckPC9MLB\nSR8zZ/wQADbVHUxXWCYDWEIwJstsqjvI9MLB5OUk/+tfODifUYPyKKu1hBBklhCMyTJltQeZ24vm\nIoitazRn/FA21VmTUZBZQjAmi+xr7mD3wfZe9R/EzR43hM11TYQj0TREZjKBJQRjssgmt8lnttsn\n0Btzxg+lPRxlx97mVIdlMoQlBGOySF9GGMXFk4iNNAouSwjGZJFNdU2MHpzP6MH5vT52xpjB5Dhi\nHcsBZgnBmCxSVnvw8BDS3srPCTFjzGDrWA4wSwjGZInOSJStuw/1eoRRotnjhlgNIcCSuh+CiCwF\nfgqEgPtV9cdd9p8D/BcwD7hGVZ9M2BcBPnBfVqrqZe72qcDjwChgDfAFVbV58cYcqzu6X5Jie3Qi\nHZF/Y/bb/wjvvdWnU88ZP5Q/rNvF/pYOhhfkHUuUJgP1WEMQkRBwD3AJMBe4VkTmdilWCXwJeKyb\nU7Sq6nz3cVnC9ruAn6jqDKAR+Eof4jfGJKlMJwMwRyr7fI7ZtoRFoCXTZHQqUK6qFe43+MeBZYkF\nVHWHqpYCSQ1QFhEBLgDiNYmHgMuTjtoY02tl0WJyCTNddvX5HHMOjzSyZqMgSiYhTASqEl5Xu9uS\nNUBESkTkHRGJ/9EfBexX1XBP5xSRG93jSxoabD12Y/qqTIuZIdXkSqTP54gvYWFrGgVTf3QqT1bV\nRcB1wH+JyPTeHKyq96nqIlVdVFhYmJ4IjckCZdHJx9RcBB8tYWFNRsGUTEKoAYoSXk9ytyVFVWvc\nfyuA14AFwF5guIjEO7V7dU5jTO8c0EHUM4LjnOpjPtdx44awtb6JaFRTEJnJJMkkhNXATBGZKiJ5\nwDXAimROLiIjRCTffT4aOAvYqKoKvApc6Ra9AXi6t8EbY5KzXccBMFVqj/lcU0cPoq0zyu6mtmM+\nl8ksPSYEt53/JmAVUAYsV9UNInKniMSHkJ4iItXAVcC9IrLBPXwOUCIi64klgB+r6kZ333eAb4lI\nObE+hV+n8sKMMR/Z4SaEaSlKCBC785oJlqTmIajqSmBll223JTxfTazZp+txfwZOPMI5K4iNYDLG\npFlFdBwOUYqkvm8nSJjbMEVHAT9j+wN/xZk5r3RT9kDf3sN4zmYqG5MFdug4Jsge8iXcc+EejGcf\n+XQcrnWY4LCEYEwW2KHjmCp1KTmXI8pk2X24X8IEhyUEYwJONdapPCVFCQFindNWQwgeSwjGBNw+\nhtDEoJQmhClSR6WOJaKSsnMa71lCMCbgtut4AKaltIZQRwe57NLRKTun8Z4lBGMCbns01rST0hqC\nEzuX9SMEiyUEYwJuh44jRIRJkrq1wOId1NaPECyWEIwJuB06jiJpOKZF7boaw34KaLMaQsBYQjAm\n4FI9wghAJNYEZTWEYLGEYEyAqcZqCKlOCBBrNop3WJtgsIRgTIDVM5wWBqRsUlqiKVJHlRbSqaGU\nn9t4wxKCMQGWylVOu5oidUQIUa12n5KgsIRgTIDtiMYTQuprCNOcWJKxfoTgsIRgTIBt13Hk0ckE\n2ZPyc8f7JWykUXBYQjAmwGJDTusJServbjaSJobQbDWEALGEYEyAbdfxaWkugtjQ09hII0sIQWEJ\nwZiAiqqwU8emZchp3BRLCIFiCcGYgKpjBO3kpT0h1Oho2jWpmy+aDGcJwZiAqtQxABT39baZSSh2\n6lEcW/U0ICwhGBNQVdF+SAjuuatsLkIgJJUQRGSpiGwWkXIRuaWb/eeIyFoRCYvIlQnb54vI2yKy\nQURKReTzCfseFJHtIrLOfcxPzSUZYwCqdAxCNC1DTuOK3BVU47UR4289NvyJSAi4B7gYqAZWi8gK\nVd2YUKwS+BLw7S6HtwBfVNWtIjIBWCMiq1R1v7v/ZlV98lgvwhjzSVVayAT2kpfCVU67GksjeXRa\nDSEgkukJOhUoV9UKABF5HFgGHE4IqrrD3RdNPFBVtyQ83yUi9UAhsB9jTFpV6piU3gOhO44ok6SB\nKh2b1vcx/SOZJqOJQFXC62p3W6+IyKlAHrAtYfOP3Kakn4hIfm/PaYw5siodQ7GTvv6DuCKptxpC\nQPRLp7KIjAceAb6sqvFaxHeB2cApwEjgO0c49kYRKRGRkoaG9H7bMSYo2jSXekYcbuNPpyJpsIQQ\nEMkkhBqgKOH1JHdbUkRkKPAccKuqvhPfrqq1GtMOPECsaeoTVPU+VV2kqosKC+0/nTHJiK9Ams4R\nRnHFUs9+hnBQB6b9vUx6JZMQVgMzRWSqiOQB1wArkjm5W/4p4OGuncdurQEREeBy4MPeBG6MObIq\nd9RPUT8khKLDQ09tpJHf9ZgQVDUM3ASsAsqA5aq6QUTuFJHLAETkFBGpBq4C7hWRDe7hVwPnAF/q\nZnjpoyLyAfABMBr4YUqvzJgsVmkJwfRBUvPNVXUlsLLLttsSnq8m1pTU9bjfAr89wjkv6FWkxpik\nVWkhA2inkANpf694P4X1I/ifzVQ2JoDiQ05F0v9ew6SFoTTb5LQAsIRgTABVaWG/dCjHFUu9NRkF\ngCUEYwJGNdae3x9DTuOKpN5qCAFgCcGYgNnPYA5R0C8dynHFUk+1jiaq/dBGZdLGEoIxAfPRCKP+\nqyFMkgY6yKOe4f32nib1LCEYEzBVhyel7e639yy2oaeBYAnBmICp8qCGUGT3RQgESwjGBEyljmEk\nBxksbf32nhNlD0LUOpZ9zhKCMQFTrYX9WjsAyJcw42ikKmo1BD+zhGBMwFTqmH4dYRRXZHMRfM8S\ngjEBEokqNTraEoLpE0sIxgRI7YFWwuT06yzluGKnnjpG0B5O3y07TXpZQjAmQKr2tQL9O8Iorkga\nUBxqGlv7/b1NalhCMCZAqhpbgP5Z9rqrw6ueWkLwLUsIxgRI9b4WhCjjZW+/v/ekeELY19Lv721S\nwxKCMQFS1djKePaRJ/3fjj+WRvLoPFxLMf5jCcGYAKna13L4m3p/c0SZKHuo3mdNRn5lCcGYAKlu\nbPWk/yBukjRQbTUE37KEYExAtIcj7G5qo8jxpoYA7lwE61T2LUsIxgRETWMrqt4MOY0rkgb2NXfQ\n3B72LAbTd5YQjAmI+DdzL5uMDq96as1GvpRUQhCRpSKyWUTKReSWbvafIyJrRSQsIld22XeDiGx1\nHzckbD9ZRD5wz/nfIv1xO3Bjgis+3NPrGkIsFms28qMeE4KIhIB7gEuAucC1IjK3S7FK4EvAY12O\nHQncDpwGnArcLiIj3N2/BP4KmOk+lvb5KowxVDW2kBsSxtLoWQxFNhfB15KpIZwKlKtqhap2AI8D\nyxILqOoOVS0Fol2OXQK8qKr7VLUReBFYKiLjgaGq+o6qKvAwcPmxXowx2ax6XysThw/EEfUshhE0\nUZAXsiYjn0omIUwEqhJeV7vbknGkYye6z3s8p4jcKCIlIlLS0OBdVdiYTFfV2ELRyAJPYxCBohEF\n1mTkUxnfqayq96nqIlVdVFhoN98w5kiqG1uZNMLbhABQNHKgzUXwqWQSQg1QlPB6krstGUc6tsZ9\n3pdzGmO6aG4Ps6+5g6KRA70OhUkjCqhubCXWGmz8JJmEsBqYKSJTRSQPuAZYkeT5VwGLRWSE25m8\nGFilqrXAQRE53R1d9EXg6T7Eb4whYZXTjKghFHCoPcz+lk6vQzG91GNCUNUwcBOxP+5lwHJV3SAi\nd4rIZQAicoqIVANXAfeKyAb32H3AD4glldXAne42gG8A9wPlwDbgjym9MmOyyOH7IHjchwBQNCJW\nS7GOZf/JSaaQqq4EVnbZdlvC89V8vAkosdxvgN90s70EOKE3wRpjund4DsII75uM4kmpal8r8yYN\n9zga0xsZ36lsjOlZVWMLBXkhRg7K8zoUJlkNwbcsIRgTAFX7Wpk0YiCZMOF/yIBchhfk2uQ0H7KE\nYEwAVDe2ZESHclzRiAJb9dSHLCEY43OqStU+7yelJSoaOZBqqyH4jiUEY3xuf0snzR2Rw233maBo\nRAHV+1uJRm0ugp9YQjDG5w7PQcigGsKkkQV0hKPUN7V7HYrpBUsIxvhcpds0U5xBCSEei4008hdL\nCMb43M69mZcQJruxxGMz/mAJwRifq9zbwujBeQzKT2qeab+YMHwgjkDl3mavQzG9YAnBGJ+r3NeS\nUbUDgLwchwnDBx5uzjL+YAnBGJ/LxIQAsSasnZYQfMUSgjE+1h6OsOtAK8WjBnkdyidMHlVApfUh\n+ErmNDoaY47ujmGf2FQTHYfqfzL5jW/BW296ENSRFY8cxN7mDg61hxmcQf0b5sjsUzLe6uaP3JHL\nHkhfHD61U8cCUCz1HkfySfFmrMq9LcydMNTjaEwyrMnIGB+rdBPCZNntcSSfNHmUmxD22Ugjv7CE\nYIyPVeoYBtBOIfu9DuUTig8nBOtH8AtrMjLp0ZumINNnO3UMxVJPBqx6/QlD3WWwbXKaf1gNwRgf\nq9SxFGdgc1Hc5JEFVkPwEUsIxviUaqzJaHIGdijHFY8aZAnBRywhGONTDQynjfyMHGEUVzxyIDWN\nrYQjUa9DMUlIKiGIyFIR2Swi5SJySzf780XkCXf/uyIyxd1+vYisS3hERWS+u+8195zxfWNSeWHG\nBN1HQ04zucloEOGosmt/m9ehmCT0mBBEJATcA1wCzAWuFZG5XYp9BWhU1RnAT4C7AFT1UVWdr6rz\ngS8A21V1XcJx18f3q2rmfs0xJgNVauw7VCYOOY2zkUb+kkwN4VSgXFUrVLUDeBxY1qXMMuAh9/mT\nwIXyybt9X+sea4xJgZ3RMQhRJsoer0M5ovjktJ02F8EXkkkIE4GqhNfV7rZuy6hqGDgAjOpS5vPA\n77pse8BtLvrnbhKICShVeD0yj1s6v8obkRNRu8tin1TqWCawl3wJex3KEY0bOoC8HMfWNPKJfpmH\nICKnAS2q+mHC5utVtUZEhgC/J9ak9HA3x94I3AhQXFzcH+GaNFoTncldndfwns4hhzCPRy7gNNnI\nP+U+wcnOVq/D85VKHUOxk9ktrY4jFI2wZbD9IpkaQg1QlPB6krut2zIikgMMA/Ym7L+GLrUDVa1x\n/20CHiPWNPUJqnqfqi5S1UWFhYVJhGsy1VORs/hcx/ep0PHcmfMA6/P/iu/nPMg2ncDnOr7PHyJn\neR2ir8TmIGR2QgB3GWyrIfhCMglhNTBTRKaKSB6xP+4rupRZAdzgPr8SeEU11hAgIg5wNQn9ByKS\nIyKj3ee5wKXAh5jAqtfh3N75JRbJZt7I/we+mPMig6SdG3Je4I38f2ChbOH2zhuoV5vhnIxDOoA9\nDMvoEUZxk925CGptgxmvx4Tg9gncBKwCyoDlqrpBRO4UkcvcYr8GRolIOfAtIHFo6jlAlapWJGzL\nB1aJSCmwjlgN41fHfDUmI6nCrZ1/STu53J17LwXS/rH9BdLO3bn30Uoe/9z5ZetTSEKVxmrLfqgh\nFI0s4FB7mMaWTq9DMT1Iqg9BVVcCK7tsuy3heRtw1RGOfQ04vcu2ZuDkXsZqfOqZ6Bm8GF3E93Ie\nZZpT122ZGc4uvpXzJD8OX8dz0dO4NPRuP0fpLzszeJXTria7I4127G1m5KA8j6MxR2MzlU1a7dGh\n3N75JU6Scr4SWnnUsl8NreQk2cZtnV9mrw7ppwj9qULHAzBVuk+wmWRqYexubtsbbOhpprOEYNLq\nV+HPcJAC/i33XkJy9LagHIlyd+697Gcwvwp/pp8i9KcKnUAhjQyRVq9D6VHxyAJyHKFizyGvQzE9\nsIRg0qZV83g8cj5LnBJmOV0HpnXvOKeaxU4JT0TOp01z0xyhf1VExzNNar0OIym5IYfikQVUWA0h\n41lCMGmzInImBxjMDTmrenXcDaFVNDKEFZEz0xSZ/23X8Uxz/JEQAKYVDrKE4AN2gxyTFqrwYGQx\ns2Unp8qmXh17ulPGcVLJQ5HFXBV6PSNv/uKlRh1MI0Myt4bQzc2RpnZexxuRxURvH46T2HRo98nO\nKFZDMGmxWo+jTKdwQ+iFXv9BF4Evhl5gg05ljc5KT4A+Fu9QztiE0I1pUksHedRo1xVtTCaxhGDS\n4qHwEoZxiMtDb/Xp+CtCbzGUZh4ML0lxZP63LToB8FlCcJu3KnSCx5GYo7GEYFKu7kAbz0dP4fOh\n1xgoHX06R4G0c3XoNZ6PnsJuHZ7iCP2tQseTS5giH0xKi5smu4CPajcmM1lCMCn3xOoqoghfCL14\nTOf5YuhFIjgsj5yXmsACYruOo1h2kyP+uQvZaA4yhGZLCBnOEoJJKVXlmdJdnCqbKHIajulcxU49\np8hmno2c3nPhLFKhE3zVXASxfqFpUmtNRhnOEoJJqS27D1Fef4hLQ++k5HyfCb3LZi1ma7TrLTiy\nU0SFnTrWdwkB3IQQtRpCJrOEYFLq2dJdOAJLQ++l5HyXhN7FIcozVksAoFoL6SDXnwnBqaWWUbRo\nvtehmCOwhGBSRlV5trSWM6aPolAOpuScY+QApzllPBs9w5ZP5qNROn6alBYXT2LbdZzHkZgjsYRg\nUmbDroNs39PMpfNS2058qfM2FTqBTXVNKT2vH1W4f0zjo3b8xEYaZT5LCCZlni2tJeQIS45P7TfA\npaHVhIjwbKn//gimWoVOYBiHGIn/kmN8ZVZLCJnLEoJJCVXluQ92cdaM0Slf836UNHGms4FnS2uz\nvtmoQsczVWp9uZzHAOlkIg1URG2kUaayhGBSorT6AFX7Wrl0Xnq+/V3qvMPOvS1s2JWavgm/8tMq\np92Z5tRaDSGDWUIwKbHyg1pyQ8KSuenpMFwSWk2OIzxb6t8/hsfqkA5gNyOZ7sMO5bjYXITxdpvU\nDGUJwaTEixt3c/q0UQwrSM89DIZLM6dPG8VLZZl/y8h02eHjDuW4aVJLMwNpwJYjyUS2/LVJXjfL\nGgNsi46nouM/+NL+n8Mdx7ZcxdFcNGcMdzyzke17mpk6elDa3idTbfPhKqddxZNZeXQCY0L7PY7G\ndGU1BHPMXoqeDMCFobVpfZ+L5sZuLP9yltYStkSLyCHMFB/cR/lIZjnVAGzVSR5HYrqTVEIQkaUi\nsllEykXklm7254vIE+7+d0Vkirt9ioi0isg69/E/CcecLCIfuMf8t4gfx00YgJciCzletjNR9qb1\nfSaNKGDO+KG8uDE7E8ImLWaa1JIvYa9D6bMx7Gc4TWzSYq9DMd3oMSGISAi4B7gEmAtcKyJzuxT7\nCtCoqjOAnwB3Jezbpqrz3cfXE7b/EvgrYKb7WNr3yzBe2atDWKOzuMhJb+0g7uI5YyjZ2Uhjc9+W\n1fazTdEiZkul12EcExGY7VRSFi3yOhTTjWRqCKcC5apaoaodwOPAsi5llgEPuc+fBC482jd+ERkP\nDFXVdzQ2sPxh4PJeR28890pkAVEcLg6t6Zf3u2juWCJR5dXN/rkXQCocaO2khkJmO/5OCACzpYot\nWkRUrVEg0ySTECYCVQmvq91t3ZZR1TBwAIjfK2+qiLwvIq+LyNkJ5at7OKfxgZeiCxnHXo6XHf3y\nfidMGMbYoflZN9pos7tsxxyp6qFk5psjlbQwgCot9DoU00W6O5VrgWJVXQB8C3hMRIb25gQicqOI\nlIhISUPDsa2vb1KrTXN5IzqPi0Jr+23mrOMIF84Zy+ubG2gPR/rnTTPAprrYhLxA1BDcayizfoSM\nk0xCqAESG/wmudu6LSMiOcAwYK+qtqvqXgBVXQNsA2a55ROHGXR3Ttzj7lPVRaq6qLDQvlFkkrej\nx9PKAC5y+qe5KO7iOWNp7ojwTsW+fn1fL22qa2IYhxiH/695llQjRK1jOQMlkxBWAzNFZKqI5AHX\nACu6lFkB3OA+vxJ4RVVVRArdTmlEZBqxzuMKVa0FDorI6W5fwxeBp1NwPaYfvRhdSAFtnOFs7Nf3\nPWP6KAryQryURaONNtUeZLZU+nINo64GSgdTpY5NUUsImabHhOD2CdwErALKgOWqukFE7hSRy9xi\nvwZGiUg5saah+NDUc4BSEVlHrLP566oa/4rzDeB+oJxYzeGPKbom0w9UYx3K5zil/T4MckBuiE/N\nGM3LZbuzYrG7aFTZXNfEnAA0F8XNliqrIWSgpGYqq+pKYGWXbbclPG8DrurmuN8Dvz/COUuAE3oT\nrMkcG3QKdYziQud/PXn/i+aM5YWNuymrbWLuhF51S/lOdWMrzR0RZucEKCE4lfwxfArN7WEG5duC\nCZnCZiqbPnk5ugAhyvmhdZ68/3mzY/1J2TBruexwh7L/RxjFzZZKFIctu/13X4cgs4Rg+uTlyELm\nyzZGp+hWmb01ZsgATioazsubgj8fYVNtEyKxztigmO0On91sd8HLKJYQTK/V63BKdToXpXntop5c\nNHsM66v309DU7mkc6bap7iCTRxZQIMG5zknSwCBa7baoGcYa70yvvRKZD8AF/bRcxWFdVlu9IDqZ\n/9B/5dW7ruLqnNe7lD3Qj4Gl16a6JmaPGxobehEQjijHSRVltTYfNZNYDcH02kvRhUyk4XC13ytz\nZScT2MNL0YWexpFOLR1hduxtZvb4IV6HknKznUo21TVlxUgxv7CEYHqlTXN5K3oCF4be93xMvAhc\nEHqfP0VPpE3Tc2Mer23ZfQhVYjWEgJkjlRxo7aTuYJvXoRiXJQTTK/HZyRf2d3PREVzorKWFAbwT\nneN1KGmxqTbWaT8nkDWEWA2zrDa775OdSSwhmF55yZ2dfJpT5nUoAJzhbGQgbbwc0Gaj9dUHGDIg\nh6IRBV6HknJzZCciUFodnP4ev7OEYJIWVeHFyMmc45QyQDq9DgeAAdLJ2c4HvBg5OZDLKb9f2cj8\nouE4TvCubbC0cdzYIayttFtpZgpLCCZp63Q69YxgSWi116F8zJJQCXWMolSneR1KSjW1dbJ5dxMn\nTx7hdShps6B4BO9XNhKNWsdyJrCEYJL2QmQROYS5wPFmdvKRXOisJUSEVZFFXoeSUuuq9qMKC4uD\nmxAWFg+nqS3MtoZDXodisIRgkqSqvBBdxBnORoZJs9fhfMxwaeZ0p4xV0VO8DiWl1u7cjwjMLx7u\ndShpE6/9rNnZ6HEkBiwhmCSV1x+iQiew2CnxOpRuLXFWU6ETKI9O8DqUlFlb2cjMMYMZOiCYQ2oB\npo4exIiCXNZWWkLIBJYQTFJWbagDYHEoMxNCPK5V0WA0G0WjyvuVjYFuLgIQERYUj7CO5QxhCcEk\nZdWG3SyQrYyVzPzFHSeNnCTlvBAJRrNRxZ5DHGwLszDAHcpxC4uHU15/iAMtmTFyLZtZQjA9qtnf\nygc1BzJudFFXS0KrWa/TqdWRXodyzNbujCXeoNcQ4KNrfL/Kmo28ZgnB9OiFeHNRhvYfxC1x43sh\nAKON1lY2MmxgLtNGD/I6lLTGh/C2AAARU0lEQVQ7qWg4jmDNRhnAEoLp0fMf1jFzzGCmOXVeh3JU\n051aZkg1fwzAaKO1lY0sKA7mhLSuBuXncNy4obxvHcues4Rgjqr2QCvv7djHpfP8MXrnM867vBud\nQ90B/y6YdqC1ky27D2VFc1HcwuLhrKvcT8QmqHnKEoI5qmfW70IVls33R0JYFnoLxeHZ0l1eh9Jn\n66qyp/8gbmHxCJraw2yttxvmeMkSgjmqP7y/i5OKhjPFJ23Z05w65sk2/rCuxutQ+mzNzkZE4KSi\nYT0XDoiFNkEtIyR1xzQRWQr8FAgB96vqj7vszwceBk4G9gKfV9UdInIx8GMgD+gAblbVV9xjXgPG\nA63uaRaravBvkOsjW3c3sbH2ILd/dq7XofTKstBb/KBmOuX1h5gxZrDX4RzdHZ/8o/9G+/c5CRjy\n4+v6Px6PTBlVwIRhA3hjSwPXnzbZ63CyVo81BBEJAfcAlwBzgWtFpOtfiK8Ajao6A/gJcJe7fQ/w\nWVU9EbgBeKTLcder6nz3Yckgwzy9bheOwGfmjfc6lF75bOgdHIEVPqwl7NUhrNfpnB/KrPWi0k1E\nOG/2GP60dQ/t4YjX4WStZJqMTgXKVbVCVTuAx4FlXcosAx5ynz8JXCgioqrvq2q8MXcDMNCtTZgM\np6o8vb6Gs2aMZsyQAV6H0ytjZD9nTh/NH9bt8t3tGV+PnoTiZNwCgv3hguPG0NwRoWSHNRt5JZmE\nMBFIvHlutbut2zKqGgYOAKO6lPkcsFZV2xO2PSAi60Tkn0W6vyGjiNwoIiUiUtLQ0JBEuCYV1lbu\np2pfK8vm+/Mm6MvmT6ByX8vhDlq/eCWygEIaOV52eB1Kvztzxijychxe2WSNBV7pl05lETmeWDPS\n1xI2X+82JZ3tPr7Q3bGqep+qLlLVRYWFhekP1gCx5pb8HIclx4/1OpQ+WXLCOPJyHJ5e55/RRmF1\neCM6j/NC63HEXzWbVCjIy+H0aaN4dbMlBK8k06lcAxQlvJ7kbuuuTLWI5ADDiHUuIyKTgKeAL6rq\ntvgBqlrj/tskIo8Ra5p6uI/XYVKorTPCivW7uGjuWIb4dKXNoQNyuXjOWJ5eV8Mtl8xmQG7I65B6\ntFZncpBBXOC873Uo/adLp/r54SV8P3wDO2+byWQnITHcYbfZ7A/J1BBWAzNFZKqI5AHXACu6lFlB\nrNMY4ErgFVVVERkOPAfcoqpvxQuLSI6IjHaf5wKXAh8e26WYVHm2tJbGlk6uP63Y61COyXWnFdPY\n0slzpbVeh5KUVyPzySHMWU72/irEk+Gr0fkeR5KdekwIbp/ATcAqoAxYrqobROROEbnMLfZrYJSI\nlAPfAm5xt98EzABuc/sK1onIGCAfWCUipcA6YjWMX6XywkzfPfz2DmaOGcwZ07p2A/nLmdNHMb1w\nEA+/s9PrUJLyanQBpzibGSqtPRcOqMlOPdNkF69EF3gdSlZKah6Cqq4EVnbZdlvC8zbgqm6O+yHw\nwyOc9uTkwzRplVBtXxedTmnHD7gz5wHk+5cd5aDMJyJ84fTJ3PHMRkqr9zNvUubeeWyXjmSTFvO9\n0KNeh+K58511PBK5iBbNp0Daez7ApIzNVDYf83D4YgbRyhWhP3kdSkr8xcmTKMgL8fDbmV1LeDUS\nayLJqv6DI7jAeZ8O8vhz9HivQ8k6lhDMYft0CM9GT+cvQm8yJCDNFkMH5HLFgomsWL+Lfc0dXodz\nRCujp1Msu5ku/hkVlS6nOJsYQgvPRU7zOpSsk1STkckOT0TOo4M8vhB60etQjl1CM9gXo5N4NHw3\ny//1y3w959ku5bwfvbIjOpa3oifw7Zwn6H42TnbJkwjLQm+xPHIut+vDDJdmr0PKGlZDMAB0aIjf\nhi/idGcDsxz/LflwNMc51ZwmG3kkfDEdmnnDTx+PnE+ICFeFXvc6lIxxbegVOsjjqcinvA4lq1hC\nMAA8ETmfGgr5WujZngv70NdznqGGQpZHzvM6lI/p0BBPRs7lQmdtxt6v2gvHOzs5SbbxeOR8fLb6\niK9ZQjC0ah4/C1/BKbKJ85z1XoeTFuc561kkm/lZ+AraNHMm270UPZk9DOPa0Cteh5Jxrg29zGYt\nZq3O9DqUrGEJwfBI5GLqGcG3c5cHtg1bBG7OfYLdjOSRyMVeh3PY7yIXMJEGznFKvQ4l43w29DaD\naOV3kQu8DiVrWELIck1tnfwyfBlnO6Wc5mzyOpy0Os3ZxNlOKb8IL6NJB3odDpV7W3gzOo/P57xG\nKAvXLurJIGnnstCfeTZyOgdaO70OJytYQshyv/nTDhoZws05T3gdSr+4OecJGhnCbyJLvQ6Fx96r\nxCHK1aHXvA4lY10XeoU28nlqbbXXoWQFSwhZbNf+Vn71ZgVLnPeY52z3Opx+Mc/ZzhLnPX4V/gy7\ndKRncew+2MZDf97BJc67jBNb//9ITnS2s1C28IvXttHcHvY6nMCzhJClVJXv/L6UqCrfy3nM63D6\n1a05jxHF4Z86v+bZDXTufn4zkajynSypmR2LW3Mfpb6pnXtf39ZzYXNMLCFkqUffreTNrXv43qfn\nfHyZ4SxQ7NTzvZxH+VP0RH7rwcJ3pdX7+f3aav7yU1MpzrKffV+c7GzlsydN4L43K9i1Pxgz6DOV\nJYQstHNvM/+ysoyzZ472/RLXfXV96GXOcdbzLys3sWNP/82EVVXufGYjowfn883zp/fb+/rdd5Ye\nhyrc/XywBz54zRJClumMRPn2/64nJMJdn5vHEe5cGngicHfufeSGhH/83/V0hKP98r7PfVBLyc5G\nbl4yy7c3H/LCpBEFfPXsqfxh3S7er7Q+l3SxhJBFolHln54sZfWORu68/HgmDPd+6KWXxkkjP7zi\nRNbsbOQflq8jEk1vf8LOvc3c9vQG5o4fypUnF/V8gPmYvz5vBoVD8rn5yVIbhpomlhCyhKpy57Mb\neer9Gr69eBZXLJjkdUgZ4bKTJvC9T8/mudJabn3qg7R1Mu9v6eDLD64mqso91y8k5GRnzexYDM7P\n4b+vWcDOvc3c9NhawpH+qdVlE1vtNEv89OWtPPjnHXzlU1P55vkzvA4no9x4znQOtob5+avlDM7P\n4dbPzDm2prQu9wlu1xy+1nEL1TqT3+b9C1N/vvkYI85eZ0wfxY+uOJF/erKUO57ZwA+WnZC1zZ7p\nYAkh4Fo7Ity+4kOWl1Rz5cmTuPXTx/jHLqD+cfEsmto6uf9P29m5r4V/u3Iewwvyjvm8LZrPtzu/\nzrs6l5/m/pxTHUsGfZKQZK8GtoWu4d53LmN8yd18I7Ti40uuZMCS5n5lCSGo7hjGtuh4vtH5d2zR\nSdwUepq///D3OBusmt0dEeGOy46naGQBdz2/iU//9E1+eu0CTpnS98lrH0Yn87edf8N2Hcf3ch5l\nWejPKYw4u30n5wlqtJB/C1/Dpmgx/5p7P4OlzeuwfM8SQgAdaO3k152f4/7IZxhABw/m3s25IVs8\nrSciwlfPnsZpU0dx0+/WcvW9b3PJCeP42jnTOako+fsxH9BB/DZyIT8Nf44RNPFo7r9wZmhjGiPP\nPo4o/537c+ZGdvDv4c+zoWMKP8v9Gcc7mX2r1EwnyXSiichS4KdACLhfVX/cZX8+8DBwMrAX+Lyq\n7nD3fRf4ChAB/lZVVyVzzu4sWrRIS0pKkr64bLNrfyvLS6r49Z+209QW5hLnXW7LfYTxss/r0Hzn\nkA7gF+FlPBK5iCYGcaqUsfjTf8EpU0Zy/ISh5IQ+Ph5j76F2NtU18X9ra3h2bQXt5LHYWc2Pc+9n\npDR5dBXZ4Z3obP6m42/YwzAWOyX89V//A/N7kcCzgYisUdVFPZbrKSGISAjYAlwMVAOrgWtVdWNC\nmW8A81T16yJyDXCFqn5eROYCvwNOBSYALwGz3MOOes7uWEL4uLbOCGW1B3lv+z7++GEd66piN1hZ\nPHcsf1/+ZeY6lR5H6H+HdACPRy7gt5GL2KHjABiYG2LkoDwG5oXICznUHmilsSU2DHJwfg7Lws9z\nXehl+7bajxp1MA+El/JQZDEHGMwC2coFofc5xynlRNmO091qslnU15DKhHAGcIeqLnFffxdAVf81\nocwqt8zbIpID1AGFwC2JZePl3MOOes7uBDUhqCrhqBKOKJ3RKB3hKG2dEdo6IzS3R9jf2smB1k72\nHmpn18q7qdFCtutYtuokwm6r34lSwdLQe3zaeY+pTp3HVxRMdTqC96KzWRedzgEdRCsDaCOXsdLI\nDKlhhuxikbOZQdLudahZK57An46cyQc6DYDBtDBddjFddjHVqaOQ/YySg4z86u8ZlJ/DoLycWHLP\ncch1HHJDQsiRQA2+SDYhJNOHMBGoSnhdDZx2pDKqGhaRA8Aod/s7XY6d6D7v6Zwp89e/XcMbWxrS\ndXoAEtNqYo7VhD2qbjmNbY8qvZ4Mlc9iJsoeiqSeC533OcHZzklOhTUL9YNx0shlobe5LPS216GY\nIxgsbXw1ZyVfzVnJXh3Cn6InsiY6i206gbeiJ/B/0XM+KvzLnj/HkCM4AoIgwsdGMwkfvfj49vR4\n5m8+xbTCwWk6e0zGdyqLyI3Aje7LQyLS13F7o4E9qYnKW1uCcy1BuQ6wa8lAB4Ga0fB8AK6F0dN/\ncEyfyeRkCiWTEGqAxHn2k9xt3ZWpdpuMhhHrXD7asT2dEwBVvQ+4L4k4j0pESpKpMvlBUK4lKNcB\ndi2ZKijX0l/XkczSFauBmSIyVUTygGuAFV3KrABucJ9fCbyisc6JFcA1IpIvIlOBmcB7SZ7TGGNM\nP+qxhuD2CdwErCI2RPQ3qrpBRO4ESlR1BfBr4BERKQf2EfsDj1tuObARCAPfVNUIQHfnTP3lGWOM\nSVZS8xCCQERudJuffC8o1xKU6wC7lkwVlGvpr+vImoRgjDHm6Gz5a2OMMUAWJAQRuUpENohIVEQW\nddn3XREpF5HNIrLEqxj7QkTuEJEaEVnnPj7tdUy9ISJL3Z97uYjc4nU8x0JEdojIB+7n4KuZkyLy\nGxGpF5EPE7aNFJEXRWSr++8IL2NMxhGuw5e/IyJSJCKvishG92/X37nb0/65BD4hAB8CfwG8kbjR\nXVbjGuB4YCnwC3eZDj/5iarOdx8rvQ4mWe7P+R7gEmAucK37efjZ+e7n4Lchjg8S+/+f6BbgZVWd\nCbzsvs50D/LJ6wB//o6EgX9U1bnA6cA33d+PtH8ugU8Iqlqmqt1NZlsGPK6q7aq6HSgntuaSSb9T\ngXJVrVDVDuBxYp+H6Weq+gaxkYGJlgEPuc8fAi7v16D64AjX4UuqWquqa93nTUAZsRUe0v65BD4h\nHEV3S3JMPELZTHWTiJS61eWMr9YnCMLPPpECL4jIGndmvd+NVdVa93kdMNbLYI6RX39HABCRKcAC\n4F364XMJREIQkZdE5MNuHr7+1tnDdf0SmA7MB2qB//A02Oz2KVVdSKwJ7Jsick5PB/iFO8HUr0MR\nff07IiKDgd8Df6+qBxP3petzyfi1jJKhqhf14bBkluTwVLLXJSK/Ap5NcziplPE/+95Q1Rr333oR\neYpYk9gbRz8qo+0WkfGqWisi44F6rwPqC1XdHX/ut98REckllgweVdX/czen/XMJRA2hj460rIYv\nuP8h4q4g1nnuF4FZukREBonIkPhzYDH++iy6k7gUzQ3A0x7G0md+/R0RESG2+kOZqv5nwq60fy6B\nn5gmIlcAPyN2f4b9wLqE+zDcCvwlsV79v1fVP3oWaC+JyCPEqsIK7AC+ltC+mPHcIYD/xUdLl/zI\n45D6RESmAU+5L3OAx/x0LSLyO+A8Yiuc7gZuB/4ALAeKgZ3A1aqa0R22R7iO8/Dh74iIfAp4E/gA\niN8E/XvE+hHS+rkEPiEYY4xJTjY3GRljjElgCcEYYwxgCcEYY4zLEoIxxhjAEoIxxhiXJQRjjDGA\nJQRjjDEuSwjGGGMA+P9uFqOknPxDDwAAAABJRU5ErkJggg==\n", 272 | "text/plain": [ 273 | "" 274 | ] 275 | }, 276 | "metadata": {}, 277 | "output_type": "display_data" 278 | } 279 | ], 280 | "source": [ 281 | "# Plot the resulting samples against the target distribution\n", 282 | "plt.plot(x_1,p(x_1)/4)\n", 283 | "weights = np.ones_like(x)/float(len(x)) # normalize the bins\n", 284 | "plt.hist(x, weights=weights, bins=20)\n", 285 | "\n", 286 | "print(\"time in seconds:\", time.time()-t0)" 287 | ] 288 | }, 289 | { 290 | "cell_type": "markdown", 291 | "metadata": {}, 292 | "source": [ 293 | "As expected, the histogram of the samples approximates the target distribution. The states travel around the target distribution and both modes have been vistited and sampled from." 294 | ] 295 | }, 296 | { 297 | "cell_type": "markdown", 298 | "metadata": { 299 | "collapsed": true 300 | }, 301 | "source": [ 302 | "## References\n", 303 | "http://www.cs.princeton.edu/courses/archive/spr06/cos598C/papers/AndrieuFreitasDoucetJordan2003.pdf" 304 | ] 305 | }, 306 | { 307 | "cell_type": "code", 308 | "execution_count": null, 309 | "metadata": { 310 | "collapsed": true 311 | }, 312 | "outputs": [], 313 | "source": [] 314 | } 315 | ], 316 | "metadata": { 317 | "kernelspec": { 318 | "display_name": "Python 3", 319 | "language": "python", 320 | "name": "python3" 321 | }, 322 | "language_info": { 323 | "codemirror_mode": { 324 | "name": "ipython", 325 | "version": 3 326 | }, 327 | "file_extension": ".py", 328 | "mimetype": "text/x-python", 329 | "name": "python", 330 | "nbconvert_exporter": "python", 331 | "pygments_lexer": "ipython3", 332 | "version": "3.6.3" 333 | } 334 | }, 335 | "nbformat": 4, 336 | "nbformat_minor": 2 337 | } 338 | -------------------------------------------------------------------------------- /Gaussian Processes made Easy.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Gaussian Processes made Easy" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "I've been reading about Gaussian proccesses and thought I'd share what I've learned. This introduction is on the technical side but hopefully still accessible. \n", 15 | "\n", 16 | "Gaussian processes are Bayesian non-parametric models which can be used for regression and classification, but this article will focus on regression.\n", 17 | "\n", 18 | "In supervised learning, we have some training data $\\mathbf{X}$ and $\\mathbf{y}$ and assume the outputs were produced from the inputs by a function $y=f(x)$. In a Bayesian approach, we use the training data to infer the distribution over functions, $p(f|,\\mathbf{X}^{\\checkmark},\\mathbf{y}^{\\checkmark})$, then use this to make predictions given new inputs $\\mathbf{x_{*}}^{\\checkmark}$ by integrating over all functions $f$: " 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "$$p(y_{*}|\\mathbf{x_{*}^{\\checkmark}},\\mathbf{X}^{\\checkmark},\\mathbf{y}^{\\checkmark}) = \\int p(y_{*}\\vert f,\\mathbf{x_{*}}^{\\checkmark})p(f|\\mathbf{X}^{\\checkmark},\\mathbf{y}^{\\checkmark})df$$" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "The use check marks $\\checkmark$ to indicate that the variable is known. This is a trick from my high school physics teacher and it's very useful. \n", 33 | "\n", 34 | "The usual regression approach is to assume some parametric form of the function $f$. For example, linear regression assumes that $f$ follows normal distribution with mean $\\mathbf{w}^{T}\\mathbf{x}$ and variance $\\sigma^2$: " 35 | ] 36 | }, 37 | { 38 | "cell_type": "markdown", 39 | "metadata": {}, 40 | "source": [ 41 | "$$p(y_{*}|\\mathbf{x_{*}^{\\checkmark}},\\theta^{\\checkmark})=\\mathcal{N}(y_{*}|\\mathbf{w}^{T\\checkmark}\\mathbf{x}_{*}^\\checkmark,\\sigma^{2\\checkmark})$$" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "The parameters of the distribution, $\\theta$ are the weights $\\mathbf{w}$ and $\\sigma^2$. While assuming parametric representations of the function $f$ is one approach, Gaussian processes provide a way to perform Bayesian inference over the functions $f$ themselves, without assuming any paramatric form of the distribution. Thus instead of inferring $p(\\theta|\\mathbf{X},\\mathbf{y})$, we can infer over the functions themselves: $p(f|\\mathbf{X},\\mathbf{y})$." 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "This is a suprising statement. While it is clear to picture a distribution over the values of some parameters, it is less clear to picture *a distribution over functions*. However, we actual only need to be able to define the distribution of the function at some finite, but arbitrary, set of points $\\mathbf{x}_1\\dots \\mathbf{x}_N$, instead of the whole domain of $f(\\mathbf{x})$. This is highly convenient, and in fact makes the whole inference possible. It relies on the key assumptions that the function at the set of the values is distributed according to a multivariate Gaussian. This defines a **Gaussian process**, or **GP**:" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "$$p(f(\\mathbf{x}_1)\\dots f(\\mathbf{x}_N)) \\sim \\mathcal{N}(\\mathbf{x}|\\mathbf{\\mu}(\\mathbf{x}),\\mathbf{\\mathcal{\\Sigma}}(\\mathbf{x}))$$" 63 | ] 64 | }, 65 | { 66 | "cell_type": "markdown", 67 | "metadata": {}, 68 | "source": [ 69 | "This is simply a multivariate Gaussian distribution, except the mean function $\\mathbf{\\mu}(\\mathbf{x})$ and covariance function $\\mathbf{\\mathcal{\\Sigma}}(\\mathbf{x})$ are functions of the data points, which we chose. Notice that it is necessary to choose the points $\\mathbf{x}_1\\dots \\mathbf{x}_N$ to define our probability density, instead of using infinitely many points, and these points in turn determine the mean and covariance functions. This fact will be used later on." 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "It is important to note that..." 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "metadata": {}, 82 | "source": [ 83 | "## Note on Multivariate Gaussians" 84 | ] 85 | }, 86 | { 87 | "cell_type": "markdown", 88 | "metadata": {}, 89 | "source": [ 90 | "Gaussian processes are useful largely because of the nice properties of the multivariate Gaussian (MVN) distribution. Specifically, it is useful that the conditional distribution of a joint Gaussian distribution is also a Gaussian. This is a powerful property. Stated mathematically, suppose the vector-valued random variables $x_A$ and $x_B$ are jointly Gaussian:" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "$$\n", 98 | "\\begin{pmatrix}\n", 99 | "x_A \\\\\n", 100 | "x_B\n", 101 | "\\end{pmatrix}\n", 102 | "\\sim \\mathcal{N}{\\left(\n", 103 | "\\begin{pmatrix}\n", 104 | "\\mu_A \\\\\n", 105 | "\\mu_B\n", 106 | "\\end{pmatrix}\n", 107 | ",\n", 108 | "\\begin{pmatrix}\n", 109 | "\\Sigma_{AA} & \\Sigma_{AB}\\\\\n", 110 | "\\Sigma_{AB}^{T} & \\Sigma_{BB}\\\\\n", 111 | "\\end{pmatrix}\n", 112 | "\\right)}\n", 113 | "$$" 114 | ] 115 | }, 116 | { 117 | "cell_type": "markdown", 118 | "metadata": {}, 119 | "source": [ 120 | "where $\\mu_A, \\mu_B$ are the corresponding means and $\\Sigma_{A}, \\Sigma_{AB}, \\Sigma_{AB}^{T}, \\Sigma_{BB}$ are the corresponding covariance matrices. Notice that $x_A$ and $x_B$ are simply vectors that would be obtained by splitting one larger vector $x$ i.e. $x$ is the concatenation of $x_A$ and $x_B$." 121 | ] 122 | }, 123 | { 124 | "cell_type": "markdown", 125 | "metadata": {}, 126 | "source": [ 127 | "Here is the **key property**. When you condition one variable on another, the resulting distribution is also a Gaussian:" 128 | ] 129 | }, 130 | { 131 | "cell_type": "markdown", 132 | "metadata": {}, 133 | "source": [ 134 | "$$p(x_A|x_B)=\\mathcal{N}(x_A|\\mu_{*},\\Sigma_{*})$$\n", 135 | "\n", 136 | "where the new mean and covariance are given by\n", 137 | "$$\\mu_{*} = \\mu_A + \\Sigma_{AB}\\Sigma_{BB}^{-1}(x_B-\\mu_B)$$\n", 138 | "\n", 139 | "$$\\Sigma_{*} = \\Sigma_{AA} - \\Sigma_{AB}\\Sigma_{BB}^{-1}\\Sigma_{AB}^T$$" 140 | ] 141 | }, 142 | { 143 | "cell_type": "markdown", 144 | "metadata": {}, 145 | "source": [ 146 | "Thus the resulting distribution is a Gaussian with mean and covariance calculated simply from the means and covariances of the joint distribution. While this is a nice result, deriving the equations for $\\mu_{*}$ and $\\Sigma_{*}$ involve some intense linear algebra and I go into it. This conditional distribution is essential for our GP regression example below." 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "metadata": {}, 152 | "source": [ 153 | "## Coveriance" 154 | ] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": {}, 159 | "source": [ 160 | "Before starting the regression example, it should be noted that the covariance $\\Sigma(x)$ will be given by $\\Sigma_{ij}=\\kappa(x_i,x_j)$ where $\\kappa$ is a positive definite kernel function, which measures the similarity between the two points. The kernel is designed to capture the notion that if two points $x_i$ and $x_j$ are close to each other, or similar, then so should the outputs $f(x_i)$ and $f(x_j)$. This is encoded in the covariance matrix." 161 | ] 162 | }, 163 | { 164 | "cell_type": "markdown", 165 | "metadata": {}, 166 | "source": [ 167 | "## GP Regression" 168 | ] 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "metadata": {}, 173 | "source": [ 174 | "Now it's time to demonstrate GP regression in action. First, let's choose the test points $\\mathbf{X}_{*}$." 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": 1, 180 | "metadata": { 181 | "collapsed": true 182 | }, 183 | "outputs": [], 184 | "source": [ 185 | "import numpy as np\n", 186 | "import matplotlib.pyplot as plt\n", 187 | "\n", 188 | "# number of data points\n", 189 | "n = 50\n", 190 | "\n", 191 | "# create the data points\n", 192 | "x_s = np.linspace(-5,5, n).reshape(-1,1)" 193 | ] 194 | }, 195 | { 196 | "cell_type": "markdown", 197 | "metadata": {}, 198 | "source": [ 199 | "Now let's sample the prior to get a sense of the function $f$ at these points:" 200 | ] 201 | }, 202 | { 203 | "cell_type": "markdown", 204 | "metadata": {}, 205 | "source": [ 206 | "$$p(f_{*}|X_{*})=\\mathcal{N}(f_{*}| \\mu_{*},K_{**})$$\n", 207 | "\n", 208 | "where\n", 209 | "$\\mu = 0$ by design and $K = \\kappa(X_{*},X_{*})$, which is simply a covariance matrix, which is defined by the data, which we selected. I chose the x-values to have mean 0 because the mean plays a very minor role and it simplifies the calculations. The covariance matrix is far more important. Let's samples some values to see how $f$ prior behaves over the x-values that we selected." 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": 3, 215 | "metadata": { 216 | "collapsed": true 217 | }, 218 | "outputs": [], 219 | "source": [ 220 | "# First, we need to define the covariance function (RBF kernel)\n", 221 | "def kernel(a, b, param=0.1):\n", 222 | " sqdist = np.sum(a**2,1).reshape(-1,1) + np.sum(b**2,1) - 2*np.dot(a, b.T)\n", 223 | " return np.exp(-.5 * (1/param) * sqdist)" 224 | ] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "execution_count": 19, 229 | "metadata": {}, 230 | "outputs": [ 231 | { 232 | "data": { 233 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEICAYAAABLdt/UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXmcXFWV+L+39r2r9707SSfpLJ2FJJAQCJtBiLigDiCg\n4oozjvMb1GEERcVRBx0cXEbUQR0UFFkUQVDWQAIkhJB933rf9+6qrn25vz+qqqnuru6uqq5Od9r3\n/Xzq013v3eXUe1Xn3XvuOecKKSUKCgoKCnMH1UwLoKCgoKCQWRTFrqCgoDDHUBS7goKCwhxDUewK\nCgoKcwxFsSsoKCjMMRTFrqCgoDDHUBT7OYIQ4m4hxO9mWo6ZRAghhRALM9zmRUKI00KIISHEtZls\nO0155kU/p2amZckEQoibhRAvzrQcf28oin2WEFUssVdYCOGJe3/zTMs3h/kP4KdSSouU8qmz3bkQ\nokEIsXka27cKIe6L9uMSQjQJIf4ohFgfV0ZGzw0JIVqj5dWZ6F9K+Xsp5bsz0ZZC8iiKfZYQVSwW\nKaUFaALeF3fs96m0NVdGe2eJSuBoohMiwjn7GxFC6IFXgBXAewEbsBR4FNgyqviq6HfvXcBNwGcz\n0H/a38Nz/drPNMqFO7fQCSEeEkI4hRBHhRDrYieiI7KvCCEOAS4hhEYIUSKE+JMQolsIUS+E+H9x\n5VVCiDuEELVCiF4hxONCiJxEnQoh8oQQzwohBoQQfUKI12M/urg2nEKIY0KID8bV+4QQYocQ4ofR\nunVCiI3R481CiC4hxC1x5X8jhPiFEOKlaHvbhRCV48ikF0L8IDoC7YzWM04m76g2aoEFwDPR0ape\nCLFNCPFdIcQOwA0siF7Hv0TbOiOE+GxcG3cLIZ4QQvwuKvNhIcRiIcSd0c/XLIRIOGIVQjwMVMT1\n/+9xp2+OfrYeIcTX0rlvwMeAMuBaKeURKWVISumSUv5RSnl3ogpSyhPA60DNODJLIcT/i97LHiHE\nvXHfhfj73QvcHT32Rlz9jUKIt4UQg9G/G+POjbn243wuhcmQUiqvWfYCGoDNo47dDXiB9wBq4B5g\n16g6B4BywEjkob0X+AagI/IjqQOuipb/V2AXkR++Hvhf4A/jyHMP8AtAG31tAkT03HVASbS/GwAX\nUBw99wkgCHwyKvN3iMxG7o/2+W7ACVii5X8TfX9J9PyPgTfi5JDAwuj/PwT+AuQAVuAZ4J7J5J3s\nWgPbojIuBzTR+q8BPwMMwGqgG7hi1H25Klr+IaAe+Fq07meB+mTvNTAv+jl/Gb2PqwAfsDSN+/Yo\n8Jskvm/x13UZ0AF8eoKyr0avewVwCvjMqPv9L9FrYYweeyN6PgfoJ/LA0QA3Rt/njnftZ/q3eK6+\nZlwA5ZXgpoyv2F+Oe78M8Iyq86m49+uBplFt3Ak8GP3/OPCuuHPFQADQJJDnP4CnYz/+SWQ/AHwg\n+v8ngNNx51ZEFUNh3LFeYHX0/98Aj8adswAhoDz6XgILAUHkAVIVV/bCmAJNUd4R1zqqXP4j7n15\nVAZr3LF7Ygozel9eijv3PmAIUEffW6Ny25Psf160fFncsd3AR9K4by8D34t7vxoYABzAybjjMnqs\nH6gl8gBWjSOvBK6Oe/95YGvc/R79nfsE7yj2jwG7R51/E/hEomuvvNJ/KaaYc4uOuP/dgGGUHbM5\n7v9KoCRqjhgQQgwAXwUK487/Oe7ccSIKrJCx3AucAV6MTsHviJ0QQnxcCHEgrp0aIC+ubmfc/x4A\nKeXoY5ZEn0FKOQT0EZkRxJMPmIC9cf0+Hz0+obxJEn8dS4A+KaUz7lgjUBr3fvTn6ZFShuLew8jP\nmAyj73Wsfir3rZeI4gdASnlASmkHPkRktB/PGilltpSySkp5l5QyPIFs8denkZH3p5nxKYmWj2f0\ntZyovkKSKIp9bhGfqrOZyAjWHveySinfE3d+y6jzBill65hGpXRKKb8spVwAvB/4khDiXVH79y+B\nLxCZTtuBI0RG1OlSHvtHCGEhMn1vG1Wmh4jCXB4ne5aMLP6NK28KMsRfxzYgRwhhjTtWAYy5TmmS\nanrVpO8bsBV4txDCPHUxR1Ae938FI+/PRJ+njciDKZ7R11JJN5sBFMU+d9kNOEVkQdUohFALIWqE\nEOdHz/8C+G5scVIIkS+E+ECihoQQ7xVCLBRCCGCQyAgxDJiJ/BC7o+U+yTiLbinwHiHExUIIHfBt\nIusII0Zx0dHkL4EfCiEKon2XCiGumkTelIn2vRO4RwhhEEKsBD4NZCqmoJPUFgmTvm9E7P3tREb4\nNdHvgAFYN075ZLldCJEthCgnYvN/LMl6fwMWCyFuEpHF/RuImBSfnaI8CqNQFPscJWoKeC8Ru2o9\nkVHur4CsaJEfE1l8fFEI4SSyILc+QVMAi4jYa4eI2ER/JqV8VUp5DPjv6LFOIjb0HVMU/RHgm0RM\nMGuBj45T7itEzC27hBCOqHzVE8k7BZluJGL7bgP+DHxTSvnyFNqL5x7grqhp5d+SKJ/0fZNSeoHL\ngWPAX4na1oHzgeunIPPTRBbmD0Tb/XUylaSUvUS+k18mYib6d+C9UsqeKciikICYZ4OCwowjhPgN\n0CKlvGumZVFIjBBCAouklGdmWhaF8VFG7AoKCgpzjCkr9qjdcbcQ4qCIBM18KxOCKSgoKCikx5RN\nMdEFKrOUckgIoQXeAP5VSrkrEwIqKCgoKKTGlHOKyMiTYSj6NhbppxjuFRQUFGaIjCSLEpFMcHuJ\nRAXeL6V8K0GZW4FbAcxm89olS5ZkomsFBQWFvxv27t3bI6XMn6xcRr1ihBB2Iu5g/yKlPDJeuXXr\n1sk9e/ZkrF8FBQWFvweEEHullJPGIWTUK0ZKOUAkQdDVmWxXQUFBQSF5MuEVkx8dqSMiaVOvBE5M\ntV0FBQUFhfTIhI29GPht1M6uAh6XUiohwgoKCgozRCa8Yg4B52VAFgUFBQWFDKBEniooKCjMMRTF\nrqCgoDDHUBS7goKCwhxDUewKCgoKcwxFsSsoKCjMMRTFrqCgoDDHUBS7goKCwhxDUewKCgoKcwxF\nsaeKqxdCgZmWQkFBQWFcFMWeCs5O+MlqeO0HMy2JgoKCwrgoij0Vtv0n+BxwJlMb1CsoKChkHkWx\nJ0vXCdj3EOht0H4AfEOT11FQUFCYARTFniwvfxN0FrjmvyEchJbdMy2RgoKCQkIUxZ4M9a/Dqefh\n4i9C9RYQamjcOdNSKSgoKCQkI3uezmnCYXjp62Argw3/BFojFK+Chh0zLZmCgoJCQpQR+2QcfRLa\n9sMVd0WUOsC8i6B1DwQ8MyubgoKCQgIUxT4RQR9s/RYUrYCVN7xzvPIiCPmhde/MyaagoKAwDopi\nn4jdD8BAE1z5bVDFXaqKCwGhmGMUFBRmJYpiHw93H7x2LyzcDFWXjzxntENRDTQqil1BQWH2oSj2\n8Xj9v8HrgM3fSny+8iJo3g1B/9mVS0FBQWES5rRiD8swroAr9Yr9DREzzOqbIyPzRFReBEFPZGFV\nQUFBYRYxpxX7oyce5co/Xpm6cn/1noiv+hVfG79M5cbIX8Uco6CgMMuY04p9a9NWnH4neztT8F4J\neODY07D6JrCVjF/OnAf5SxTFrqCgMOuYs4rdHXCzvytiJnmr/a3kKza8ETGxLHnP5GUrL4KmtyAU\nTFNKBQUFhcwzZxX73s69BMIBLFpLaor91POgNUHlxZOXrdwIfid0HEpfUAUFBYUMM2cV+862nehU\nOm5cciMn+0/S7+2fvJKUcOpFWHA5aA2Tl6+8KPJXMccoKCjMIuasYt/Vvou1hWu5pOwSAHZ3JJGN\nses4DDbB4ncn14mtGHIWKAnBFBQUZhVTVuxCiHIhxKtCiGNCiKNCiH/NhGBTodPVyZmBM2ws2UhN\nXg1mrTk5c8zpFyJ/FyWp2CEyam/cGUkWpqCgoDALyMSIPQh8WUq5DNgA/LMQYlkG2k2bXe27ALiw\n5EI0Kg3rCtclN2I/9QIUrZzYG2Y08y4G7wB0HUtTWgUFBYXMMmXFLqVsl1Lui/7vBI4DpVNtdyrs\nbNtJriGXRdmLALig6AIaHY10uDrGr+Tug+a3YPHVqXWm+LMrKCjMMjJqYxdCzAPOA1JwQ8ksYRlm\nV/suNpRsQCUiH2998XrgnZF8Qs5sBRmGxVel1qG9ArIqFMWuoKAwa8jYRhtCCAvwJ+A2KaUjwflb\ngVsBSkpKqKury1TXI6gbqqPP28dCzcLhPtRSTZY2i62nt7JStTJhvfz9f8JkyKbRZ4cUZcvPXYGx\n7nWaamtBiCl/BgUFBYVELFu27LxkymVkxC6E0BJR6r+XUj6ZqIyU8gEp5Top5bqcnJxMdJuQA/0H\nAFhlXzV8TCVU1GTVcGjgEFLKsZXCQUytO3CXXgwi9UviLVyLxtuHdrA+bbkVFBQUJkOlUiWloDLh\nFSOAXwPHpZT3TbW9qXJg4ACVpkpy9bkjjq+yr6LP30erp3VMHX33YdR+B+6yTWn16SlcC4Cha19a\n9RUUFBQySSZG7BcBHwOuEEIciL6SiMfPPL6Qj2ODx1iVvWrMuZX2iAnm0MDYKFFzy2tIocFdvCGt\nfoPWcoLGPIwdyo5KCgoKM08mvGLekFIKKeVKKeXq6OtvmRAuVY45jhGQAc6zjzVDFRmKyNfnJ1Ts\nptY38BauQeqs6XUsBN7CtZEReyJTzywnLMP8te2v3HnwThyBMcsjCgoK5xhzKvL0QP8BNELD8qzl\nY84JIVhpX8nhwcOEZGj4uGaoDd3AGdxlSeSGmQBP4Ro07i40Qy1Tauds0+Ju4WuHvsYDtQ9wzHGM\n447jMy2SgoLCFJlTin1//36W2ZahV+sTnl9pX8lQcIgGV8PwMVPLGwC4o6kH0sVbsAYAQ9fBKbVz\ntgjJEH9q/hO37buNRncjt1bdCkCTq2mGJVNQUJgqc0ax9/n7aHQ3sjp79bhlVmStAODgwDvK19T6\nGgFrOQFb5ZT6D9gqkUKF1tE4pXbOBvVD9dx+4HYeaniIdTnruH/t/VxTcg15+jya3IpiV1A418mY\nH/tMc7A/oqwnUuy5+lzKjGUcGjjEh8o+hAh4MLS/jbP6uqkLoNYSNJegdcxexSil5LGmx3i8+XEs\nGgv/vuTfuSj/ouHzFaYKmt3NMyihgoJCJpgziv3AwAFsGhvzzfMnLLfSvpKtnVsJhANkdexGFfZH\n/NczQMBWgdY5exV7k7uJPzT9gQtzL+Tziz6PTWsbcb7CVMHhgcgahFqoZ0hKBQWFqTInTDFSSg70\nH2BV9qrhNALjsdK+El/YxynnKUytrxPWmIb90KdKwFoeGbHPUs+YFndkYfe6iuvGKHWACnMFARmg\nwzNBTh0FBYVZz5xQ7I3uRgYCA6y2j2+GibEiawUCwaH+g5ha3sBTsgHU2ozIEbBVoAq6UXt7M9Je\npmn3tgNQbChOeL7CVAGg2NkV5iRNria6fd0zLcZZYU4o9lgagYns6zEsWgsLLAs40rsbjbsT1xS9\nYeIJ2CKKcbba2ds8bdi1dkwaU8LzZaYyAMXOrjDnCIQD3HX4Lv7r+H/NtChnhTmh2Pf376fMWEae\nPi+p8qvsqzjhbsAtBJ7SiyavkCQB6+xW7O2edkqM4+eaN6qNFOgLlBG7wpxjV+8uBgODnHKeom5o\nehIQzibOecXuD/s55jiW1Gg9xkr7SoJI3spbQMiY3MMgGYKWEqTQoHHOzhFvm6eNYmNiM0yMCnOF\n4suuMOd4vv158vR56FQ6nmt/bqbFmXbOecV+wnECf9iflH09xlLbUjRS8pbFnllhVBoClpJZ6Rnj\nDroZCAxMOGKHiJ291dM6IjpXQeFcpsXdwpHBI2wp3sKm/E1s79qOK+iaabGmlXNesTe6IgFBi6yL\nkq5jCrhZ5fWxRx3MuDxBW8WsNMVMtnAao9xUTlAGafe0nw2xFBSmnRc6XkAt1Gwu3MyW4i34wj62\ndW2babGmlXNesbe4W7BoLGRps5Kuo+s/w1K/n6awK3F+9ikQ8WVvnnUujzFFncyIHRTPGIW5gS/k\n49XOV9mQuwG7zs4i6yKqLFU81/5cxn/7s4lzPkCpxdNCqbEUkcLORfr+U5QGQ3ikH2fQmdCnO10C\n1gpUQQ9qTzchU0HG2p0qbZ42gElt7OWmcgSCJlcTG/M2ng3RFBSmjZ09O3EGnVxd/M5exluKt/DT\n0z/lmONYwoSB082gN8iRDg8ObwiHL4Qz+hr0hhjyh7hqsZ0rFyU/UE3EOa/YW92trMlZk1IdXf9p\nClURl79Ob2dmFXucy+NsUuztnnZydDkY1IYJy+nVegoNhcqIXWFO8ELHC5QYS4bzRAFsyt/Eg3UP\n8lz7czOi2O/d3s7OxqHh92oBNoMam15NnyeIy9/3963Yh4JD9Af6KTOWpVRP13+KAlM50EWXtysl\n+/xkDLs8OpvwFq3LWLtTpc3bNqkZJka5qVzxZVc452l0NXLccZxPzv/kiBm9QW3g8sLLeb79eQb8\nA9h1GXaimIBASLKv1cXmhTY+dX4BNr0Ko1Y1LN//7urkySP9BEISrTr9/ZPPaRt7qzuyzV0ssCYp\nwkF0A7Xk2auByIg9kwTNRUiVFq1jdinGdk/7pGaYGBWmCto8bQTDmV9cVlA4W7zQ/gJaoeWKwivG\nnLu6+GqCMsjLnS+fVZmOdbrxBiWXLrBRZNVi0qlHPHSqC4wEwpK6Pu+U+jmnFXuLJ5L7JBXFrnU0\nIsIBtDlLsWqsdPm6MiuUSk3AWjarXB5dQReDgUFKDMmN2CvMFQRlcNgur6AwHXiDYe7d3kZDvy/z\nbYe8vNr1KhvzNiY0tZabyqnJquGF9hfOqmvvnhYXKgGrShJHfy/Jj5hKT3b/PSt2dwsaoaHQUJh0\nHV3/KQD82YsoMBRkfMQOccnAZgnJLpzGiHnGKOYYhelkb4uL504O8o0XWxjyZ1a5vtH9Bu6Qe8Si\n6Wi2FG+hy9fF/v79Ge17Iva0ulhWYMSiS5w9tdCixaZXc7LbM6V+znnFXmIsSSnFrK7/NFKlwW+b\nT4F+mhS7rSISfSrDGW87HZJ1dYxRaixFhUpZQFWYVva2utCqBe0OP99/tY1wBt0Pn29/nnJTOUtt\nS8ctsz53PXatnefbn89YvxPh8IY41e1lbZl53DJCCJYUGP7OR+xRV8dU0Pefwp81H9RaCg2FdPu6\nM+/Lbq1AFfKhdmfYzJMmsRF7kaEoqfKKZ4zC2WBfq4vzSkz844ZCdjQO8ejBzGRFrR2q5fTQaa4u\nunpCN2itSsvmos3s6dtDl3f6f6v72lxIYN0Eih2gOt9IQ78PTyD9geE5q9gD4Uje8JQWTomM2P3Z\niwEoMBTgD/vpD/RnVrZZluWx3dtOnj5v3L1gE5GJnDHagdpZmzdHYWbpHgrQNOBnbamZD9Vkc3mV\njf97u5u9LVMP9X+h/QV0Kh2XFV42admriq6K1Ol4Ycr9TsbeFhdmnYol+cYJy1XnGwhLONOb/qj9\nnFXs7Z52woRTUuwq7wAad9ewYo/Z5jP9tI53eZwNtHnakl44jVFuKqfd204gHEi736JXbqPsmY9g\naN+ddhsKc5N9bREFvqbUjBCCf7ukmAq7nu+80krnUPrfOXfQzfbu7WzK34RFY5m0fIGhgHU563ip\n46UpfdcnQ0rJnhYXq0tMqFUTuzHGFP9UzDHnrGJv9URdHVPwYdcNnAYYo9gzbWcPmQsJq/WzZ8Se\ngqtjjApTBSEZGr7OqaIZakM71AIyRPHWL2BqeiWtdhTmJntbXNgNaubnRGaRRq2Kb11ZSiAkuful\nFvyh9MwQ27u34w15h0fiyXB18dUMBgZ5q/ettPpMhlZHgM6hAOvKJn/Y5Jg05Js1U1pAPWcVe2yb\nt1JT8jZ2XX9MsUcCkgr0kcjQjC+gChVBa9msUOzOgBNn0Jn0wmmM4ZwxaZpjDJ17AOh410/x5Syh\ncPvtWE8/lVZbCnMLKSNBOueVmlHF2cDL7Xq+clkxJ7u93L8zvd/knr49lBpLWWxdnHSd87LPw6qx\ncmDgQFp9JiVXSyTSdF3pxPb1GIvzp7aAeu4qdk8LubpcjOqJ7VXx6PtOETTkEDLmRt6r9di19sz7\nshMxx8wGU0yqro4xSk0Rz5h0XR6NHfsI6e14C9fQfuX/4ilaT/6b3yLr6ENptacwd2jo99PnCbE2\ngZLbNN/Gjatzeeb4AM+dHEi57RZ3C/PN81PKHaUWaqosVdQ6a1PuL1n2trgotmopsSW3DeeSfCMt\ng36GfOm5gZ67it3dQrmpPKU6uoFTw2aYGNPmy26rQOtsgfDM5jWPpetNdcSuU+koMhal7Rlj6NyD\nt3ANCBVSa6Tjih8zVPlucvf+kOx9P5l12S8Vzh774+zrifjUunzWlJr40Rsd9LqTt3v7w366vF0p\nzeJjVFmqaHI3TYudPRiW7G9zsza6npAM1VMMVDonFbuUMuLqmMoNDAfR9tcOm2FiFBoKp8XVKWCt\nQIQDaNyZf2ikQpunDRWqpF0d46k0Vaal2NVD7WiHWvEUro07qKVr03/iWPxhso88SN6u78z4Q09h\nZtjb4qLUpqXImnj0qlYJPrkun0BIpqTYhh0qUswdBRHFHpRBGlwNKdedjBNdHtyB8LCb44N1D/K1\nQ1+b8CFSHVtA7UnPzp4RxS6E+D8hRJcQ4kgm2puMPn8f3pA3pRuodTShCvvHjtj1BfT4ejIeVjxb\nXB7bPRFXR60quSlgPOWmcjo8HfjD/pTqGTv3AuCNV+wAKjU9679G/4pPYzv9JNkHfpayTArnNsGw\n5GC7e9zReoxKe2RRtak/+e9ebN0tVRdogIXWhUDEBz7TxNIInFdqxh1081z7cxwZPMKjTY+OW8eq\nV1Nq03KyK70Re6ayO/4G+ClwVgyoMbtvKjcwPpVAPIWGQoIySJ+vj3xDfsZkjFfsnpINGWs3Vdo8\nyWd1HE2FuYIwYVrdrcy3zE+6nqFzLyGdbcy1BkAI+s/7AhpXB1nHfodz8YcJWtKTby6RdfQhrLXP\nENYYCWsthLUmpNZMOPpyVVyOP3fZTIs5ZWKj10T29XgsejW5Jg2NA8nnkYl5cKXzfS/QF2DRWKZH\nsbe6qM43YNWrebljJ76wjyW2Jfyp+U+syV4zburgxflGjna40+ozIyN2KeVrQF8m2kqGYVfHlBT7\naaTQ4M9aMOL4sC97hhdQQ8Z8whrDjC6gSinTcnWMke5uSsbOvcP29fHketT8USSC7P0/TUu2uYTG\n2UpO9DqEtWZUgSF0g/UY23djqX0G++Ffk7/rOzMsZWbY1+pCAKtLJvcOqczWpZQgrMXdQr4+f9I9\nBxIhhGCBZUHGFfuQL8SJLs/wg2xr51ZKjaV8c/k3KTQU8qOTPxp3/9Ul+Qa6XEH63KlnWT1rNnYh\nxK1CiD1CiD19fVN7BrS4WzCpTWRrs5Ouo+s/PZxKIJ4Cw3S5PAoC1nI0M2iKcQQcuEKutEfssTw8\nqSh2tbsLrbN5rBkmjr8c6+e7b0ue0L4Xa/1z6HuOpiXfXCFn//8ghZr2zffTceXPaXvPQ7R84E80\n/cNzNN74On1rb0PfexyNM72YgtnE3lYXi/IM2AyT53eqtOtpGvAnnfKj1dOacoqReBZaFtLoaszo\nAur+NhdhCevKLLR52jjmOMYVhVdg0pj4UvWX6PH18EDtAwnrDtvZ0/BnP2uKXUr5gJRynZRyXU5O\nzpTaanFHFk5T3Q4vkWkgX5+PQEzbAupMjtjbvBFXx3QVu1alpdhQnJLLo6EjYl/3FCbeZKSuz8vP\ndnWRbVTzncEteLV2cvbe93frJaPvPoyl4QUGl39s3B23XBXvAsDctHVKfYVkiF09u/jm4W9y086b\nzkp+lHg8gTDHOj2T2tdjVGTr8QTCdLsmH7Gm5VAxitgCaqOrMe02RrO31YVRq2JZoZGtnVtRoeLy\ngssBqLZVc0PFDWzr2sbr3a+Pqbsoz4BKpOcZc056xbR4Wig3Ju/qqPINonF3jlk4hYjyytHlTKPL\nYyvM0IYVsayOxYb0TDGQes4YY+ceQlpLwoeoNxjm2y+3YtWp+NkH56MzWvmt7gaMnfswNW9LW8Zz\nFinJ3XMfQUMuA8s/MW6xoLUUb+4yzI0vpdVNv7+fx5se57O7P8s9x++h0dWIK+RiT9+eNAVPj4Pt\nbkKSSe3rMebZdQBJ2dnTcagYTZWlCsjsAuqeFheri00IEebVzlc5L/s8cvW5w+evq7iOxdbF/PzM\nz+nx9Yyoa9SqqLTrZ/eIPVO4g276/H1TijgdTaGhcFoUe9BagZBBNK72jLedDDFXx1Ty1Y+mwlRB\nh7cDXyg5W6chZl9XjZ1q37+zk6YBP3deXkqhRcsNq3L5r96LcZorydn3Y5jGXB2zEXPTVgzdB+hf\n/XmkNvHGCzFclZsx9BxBM5T85ifHB49z7/F7+fTuT/P7xt9TbirnzqV38uv1v6ZAX8DBgYNT/Qgp\nsT+apremKLmgworsiGdMYxKeMVPxiIlRZCjCrDFnTLG3Ofy0OQKsLTNzsP8gvf5e3lX4rhFl1ELN\nl6q/RCgc4scnf0x4VKrv6mgEaqoZaDPl7vgH4E2gWgjRIoT4dCbaTcTwrkmp5IiJecTkJA4zLjAU\nTE/06Qy7PLZ52igwFKBRpe/8VGGqQCKTyhmjdnejczQmtK9vq3Pw1xMDfGR17nA+6vcuzcas1/Fz\nzcfRORqxnXoybTnPOUIBcvb9BL+9CufC909a3FW5GQBzkjl3Xul8hTsO3cH+gf1cU3INP1v7M761\n4ltsyNuAWqhZaV/J4cHDZ3X3oL2tLlYUGdFrklM79ugGz01JLKDG9MJUbOxCCKrMVZwZOpN2G/HE\nMlWuKzPzcufLWDVWLsi9YEy5YmMxn6n6DIcGD/FM6zMjzlXnGxjwhugcSm3WnymvmBullMVSSq2U\nskxK+euJyvvC6W+Flc6TWdd/mpAhm5AhN+H5An0Bvb7ejO/xOdOKvd3TnrZ9PUYsujeZBVRDZ8y+\nPlKxdzj9/Pdr7SwrMPLJde+4lBq1Kq5bmcPPOpfQm7uW7IO/QPidU5L3XMF28nG0zmZ6134Rxnnw\nhmSIHd0t/EQ6AAAgAElEQVQ7+MrBr/Cjjqdx5VQnZY45OniU+0/fz4qsFTx4wYN8esGnx8xwV9lX\n4Qq6qBuqy8jnmYw+d5C6Pl/S9nWIKNrKbF1SpphWTytGtZEc3dTW76qsVRlbQN3b6qLArMFu8vFW\n71tcUnDJuPEkmws3sz53PQ81PDQiSCrdBdQZMcX0+tJPqN/qaUUt1ClFUur6T+HLXgzjLLYWGgoJ\nEx5j45oqIUMuYY1pRhZQpZS0edvSdnWMUWIsQSM0SdnZjZ17CGst+HOqh48Fw5LvbG0DCV+7ogTN\nqJSlH1iejVmn5kfio6h8g2Qf/r8pyXsuoPI5yD70AO7iDXhKNo457wv5eK7tOT6/5/P814n/osvb\nxUsdL/HVvBw03YdQu8Y3G7Z72rnn2D0UGAq4Y+kd4+bgX2lfCXDWzDHDaQSScHOMp9Kup7F/cs+Y\nVnfEIyYVh4pExBZQp7rJTCgcSXS2tszM6z2vE5RBNhduHre8EIIvLPoCZo2Zh+rfCQdakKtHo0p9\nAXVGFLs37B1jS0qWFncLxYbi5M0L4RC6gbGpBOKZVpdHW8WMjNgHAgN4Q96U87CPRqPSUGIsSXLE\nvg9vweoRI9Df7unmWJeHL19STLFNN6aORafmg8uz+V1LIR1lW7AdfyQlO/K5iP3wL1H5nfSt++KI\nwYYj4OCxpsf47Nuf5Re1v8CqsfKVpV/hVxf8ilvm3cIrgVbuyM9FP86ofSgwxLePfhuAbyz/Bhbt\n+Cli7To7883zOdh/dhT7vlYXFp2KRXmp+ZhXZOtx+EIMeCc2GbV4WqZkX4+x0BKNQJ1iQrBTPV6G\n/GHWlVnY2rmV+eb5LLAsmLCOTWvj0oJLOThwEHcwEpikU6uoyjWcGyP2sAynnTUw5uqYLFpnE6qQ\nb0LFPl1BShBzecz8LkK63uNoB8afRseyOk7VFAMRc8xk90vt6UE3WD/CzXFfq4tHDvTyniV2Lqsa\nu1N8jA+vyEGvEdwXvA6EIHv//VOWebaicbaQdeJRnAvfP8JLa2vnVj6z+zM80vgICy0L+e6K73Lv\n6nvZmLcRtVDzofIP8cn5n+QFi5l7Op8eYzYMhoN8/8T36fR2cueyO5Oaqa2yr+KY41hSC+PBsOQn\nOzq487kmOpyppZiQUrK3xcV5JeZJN5kYTWV21DNmAju7J+Shx9czJft6jCJDEWa1ecp29tfrnQgg\nL7uL2qHaMYum47EhdwNBGRyxwXZ1voFT3d6U9oSdMa+Yk46TKdcJhoO0e9tTyur4jkdM9bhl8vR5\nqFBN38bWQ22Z9fiQkqJXv0jxi7ei8iQO9hp2dZyiKQag0lxJp7dz3Ag5iIzWAbxFa6MiSn78Rgfl\ndh3/fOHEXjlZBg3vX5bNnxr1tFTdiLX+b+TsuQ9914E5lygsZ99PkCoN/as/P3zMG/LyYN2DVJor\n+fGaH/ONmm9QY68ZY1a4tuxa/tmwlFc0fn5w9DvDdmApJQ/UPsChgUP886J/HjdEfbi/YJhdTUPU\nZK0kKIMcdxyfsLzLH+Krzzfz1NF+DrS5+cwf63nx1EDygUOOAF2uIGsm2eszEfOiOWMaB8Z/mMQG\nMZkYsWciArXfE+Spo31cVmXjrf5taISGSwsuTaruEtsSrBrriE0/qvOMuAJhWgaTf6DOiGJXCzUn\nnCdSrtfh7SAkQyk9mXX9p6KpBMbPdaIWavL0edOk2MsRMpRR84Ku9xgadycaby/5O+9OGNzT5mlD\nLdTDZqapsNS2FImcUAEYOvYQ1pjw5SwB4Eyvj+ZBP/+wIgejdvKv2fUrc1GrBD/0vhdX2aVknfgD\npc9/kso/Xkn+jrsxNb2KCKS/o8xsQN91AEvjSwwuu2VEMNLLHS/jDDr51PxPMc88b8I2rln8Ob7S\n28+bA/u598S9BMIB/tL6F17oeIF/KP8Hrii8YsL69X1e/unP9Xz1+WYe2G5GjXpCO3u3K8C//qWR\nfa0ubr+kmAevX0BVrp7vbWvn7pdaGfRO7nCwtzUyIEjWfz2ePLMGk1Y14Yi91Z36bmoTUWWposHV\nkPYC6qMHevGHJDefZ2d713bOzzkfm3b8GWs8aqHm/Nzz2dO/Z3hWVl2QegrfGVHsBrUhrRF7uh4x\ngax5oB5r342n0FA4baYYyKxnjLllO1Ko6F/5Ocytr2M7+diYMm2eNgoNhajF5KHbk1FtrUYjNBwZ\nHD95p7Fz7wj7+qu1DtQCNs23JtVHjknDe6rtPHPGz+EL7qXh+lfo3PQ93MUbMDW/QtG2L1H5+OUU\nbP93RGDqGx6fbdSeXgpfu5OAuYiB5R8fPh6SIZ5ufZpqazVLs5ZO2k4gaz7Xqwr4N7+Ft3rf4s5D\nd/Jg/YNszNvIzZU3j1tPSsmzx/v5pz834PCG+MwF+fS5VPjdFWxt3ZNwK7q6Pi9feKqBdmeAe64u\nZ8sSO0VWHfe9t5Jb1xfwZpOTTz1Rx66moQll3t/qosCioTTJTSbiEUJQYdfRNIFnTIunBRWqjMxO\n4Z0F1HTMxd2uAE8d6+fdi7LoDB/CEXSwuWj8RdNErM9Zjyvo4uhgJNVGpV2PQSNSsrPPjGJXGWjx\ntOAMpObalq4Pu28C+3qM6dxwA8iond3UvA1vwWr6V30Od+nF5Oz5IdqoySlGu3fqro4x9Go9i62L\nx1XsKk8fusE6PEUR+7qUkm11DtaWmckyJO9D/5HVuUgJjx3sReqsuOZfRfem/6Tx+q20Xfm/OBde\ni6XxJWyn/pSRz3XWCAUo3PZvqHwDdF5234hgpJ09O+nydfHBsg8m3dxQ5ZV8vPU4/1RxM6edp1lo\nWchti29DNU7SNZc/xHdeaeO+1ztYUWTilx9ewE2r83jwuirm6WsYCDfz2ScPc6zzHcWxt8XFvz7d\nSFjCj99Xyfnl7yzEqlWCj6zK5ecfnE+2UcNXn2/mh6+3c7zLw2t1Dp441MtPd3bw9Reb+dyT9exo\ncLKmJPlNJkZTma2fMEipxd1CgaEgrdTUiYil8E3Hzv67fT1IKfn42jy2dm4lW5fNednnpdTG6uzV\n6FQ63uqLmGPUKsGivNS2ypuxETvASWdqo/ZWdys5uhxMmomj9GKofA60ro6EqQRGU2gopN/fn3Lu\n8ckI67MJaS0ZG7FrhtrQ95/GXX4ZCEHXxm8R1lkpfP1ORDBy44ezOk4hlcBolmctp9ZZO7xaH4+x\na2T+9RPdXjqcAS5bkNz0M0ahRctVi7P464mBkRntVFq8xRfQu/4OPIXryDr+yLkTpSolebu/h6H7\nAN0b78afuzTulOTPLX+mxFiSMHBlPFyVmxFIrvPCD1b/gG+t+Na4bo2nuj187sl6ttc5+PT5+Xz/\nPeXkmCIPW5tBzb+s2YQQErf6NP/ydAP37+zkmWP93PFcE4VWLfdfO4+F43iyVOUa+Nm187h+ZQ7P\nHh/gn59q4O6XW/n5ri6eOzlI62CAbKOa9yyx85HViWNIkqHSrqfXHRx3m7hWT2tG7OsxigxFmNSm\nlD1j2h1+/nZigGuWZKPXu9jbt5fLCy5PedZsUBtYbV/N7t7dw+sY1flGzvTMcsWuV+tRoUrZHNPi\naUlttD4wcSqBeGIbW2c8MZIQBG3lGVPssZwqrrLIYkzYmEP3xd9GN1BLzt4fApG8Gb6wL2MjdoCa\nrBrChDnhGLs2YujYS1hjxBdVWq/WOtCqBBfPS84ME89HVufhD0meH2e/y4HlH0fj7sTSkF7elLON\n7eQT2E4/SX/Np3DNv2rEucODh6kdquUDpR9I6ccfyFqAP2s+lsatLLIuwqxJbLt+8kgfX3i6gUBI\n8sP3VXLzeXkjNo+GyOjUpDZx8fIO3rfMzp+O9PHDNzpYXWLmR++vpMAy8ShYp1HxjxsK+cWH5vPt\nd5fxwIfm8/Qti3n2E4v5v+sW8L0tFXxxUzEV9sQPnmSoyB4/Z0xYhmnztGXMvg6gEqq0FlAf2teD\nWiW4+bxcXut+jTDhpL1hRrM+dz3dvm7qXfVAxDPGF5rlXjEqVMwzz0tpAVVKmbKr42Q5YuIZdnmc\n5VkeTc3b8WctIBg18QB4Si5kYNnHyDr5OKbmbRl1dYyxxLYEtVAnNMcYO/fizV8NKi3hqBnm/HIz\nFn3q9v2yLB3LC428UutIeN5TehH+rPlkHXt41meENHTsIffte3GVbhrhBRPjqZanyNJmDWf7Sxoh\nIrljOveM6xV1osvDT3d2sq7MwgMfns+KosSzXLVQs8K+gmOOw9x2cTE/el8lt15QwD1byrHokr9/\ni/IMXDTPysK8yIYSUw0Uiqdygpwx3b5u/GH/lLI6JmKhZSENroako9GbBny8dHqQDyzPJs+sZX//\nfspN5WnPJNblrEMghr1jYhGoyTJj7o7VtmpOOU8lnauiP9CPO+ROzdVxoJaQzkbIOPnOSDHF3umb\nJpdHVzuEpmY+UPmdGDv34Sof6zrVd94X8OUsIX/nt+gYjMyEMrWYBJHp4SLrojGKXeXtRzdwZtjN\n8Winhx5XkMsn8FufjMurbNT1+RJvsiBUDC77KPq+Exg63k67j+lG42ylcPvtBGzldG367pikaI2u\nRvb27+W9Je8d14wyEa7KzQgZxtz8asLzbzQ4UQm48/KSSdc5VtlX0eHtoMPTwcpiEx9ZnTsmQngm\nKbJo0alFwhF7zCMmEz7s8VRZqgjIQNIRqL/Z041eo+LGVbkEwgGODh4dju5NB7vOzlLb0mHFXmrT\nprT4PGOKfYltCd6QN+mUsDGPmFRuoHawjkDWgnFTCcSTrctGIzTTt7G1DE951G5seQMhgxH7+mjU\nOro23YMIeRmofRKN0JCnz5tSf6OpyarhzNAZvKF3bH0x//VYfphXax3o1IILK8aPepyMyxbYUIlI\nW4kYWnANQUMO9mMPp93HdCICbgq3fRFkiI7Lf4jUjTVJPdXyFHqVnquLr06rD799EX5rBebGlxOe\n39HgZHWJCWsSs6aYAjo0cCgtWaYbtUpQbtcldHlMx6EiGVJJ4Vvb62VbnZMP12RjN2o46TiJP+xn\ntalqSjJckHsB9a56Or2dCCF4+CMLk647c4rdGvF3jjfHdLsCBMOJp9dpuToO1k/ovx6PSqimzTPG\nmx/54Rjbd0+pHXPLNoKGXHx5NQnPB7Lm0Xv+7bT5uylVmTPi6hjP8qzlhGRohJ3d2LmXsNqAL3c5\nobBke52DDRUWTClM40eTY9KwusTEK2cGEwbBSLUeR/X1mFrfQDuQ+T0qp4QMk7/jG+gGauna9D2C\ntsoxRXp9vbzW/RqbizYn7d88BiFwzbsSY8fbqLz9I041DfhoHPCzsTK5NY4yYxk5uhwODBxIT5az\nQGw3pdG0uFuwaqzpX8dxKDYWY1Qbk1Ls//d2NxadiutXRhaID/XuRiVhy7avo3Gkv2nH+tz1AOzu\nTV1vzJhiLzQUkqXN4qTjJN2uAD/Y3saNj5zh6y80E0qg3Fs8LRjUBnJ1ya2uq7z9qL39BOwT52eI\np0BfMC0j9qCtAr+tElPL2F1SkiYUwNS6E3fZJePuJQrgXPhBGvVGKgKZ9xpZaluKChWHBw8PHzN0\n7o08uNRaDra76feEJkwfkCxXVGXR6ghwehxPAEf19YTVerKO/W7KfWUKjbOFopf+CUvTVvrW/Cue\n0rEJvgCeaXuGsAzzgdIPTKk/V8VmhAxhHrVJyc6GiF/5RUkuXgshWGVfxaGBQ2nncJpuKrN1dDgD\neAIj5Wv1tKa8m1oyqISKKksVZ5y1/M+ODh492MupHs8Y3XSs08ObTUPcsCoXi16N2t3F8aa/sNzv\nxyohf9d/pr0WVGIsodxUPiIKNWn50+oxAwghWGSpZnf3UT7+aC0vnXZwQbmFt5pdPLine0z5Vncr\nZcaypG+gbjCympzsiB2mb8MNAHfpxRg79yAC6e06buzciyowhDuBfT2eMJJmjYp5nsGMLy4a1UYW\nWhcOB06ofA50/aeH7evbah0YNIINUzDDxNg034pGBVvHMceEDdkMVb0Pa91fUXvSzxaaEcIhso49\nTNlfrsPQe5Tu9V9lcNnHEhZ1BV083/48G/M2TmkDFAB/TjVBYx6GjpE7Ie1odLIoz0DhJB4t8ayy\nr8IZdI5IGTubqIx61TSPsrPH9MJ0UGWpom6onj8f7eGBt7r4xycb+NDDp/jGiy38+UgfDf0+HtzT\njd2g5kM1OWgdjdie/wRH1WFqCi+jd+0XMXbsxlL3zOSdjcP63PUcHTyacszPjCh2CTx9tI+9tXm4\nZBcXzIPfXL+A715VxnuX2HnkQO8Y+2qq2du0g5EEWf6sFEbshgIcQQeeUOZD191llyDCAYztqT99\nIeLmGFYb8BSvn7Bcl7cLP5KFbgdqd+YfUjVZNZx2nsYX8mHoPoBA4i1YQzAsea3eyYWVVgxJbqQw\nEVa9mvPLLGyrdYyb/Ghw2UchHEwYeXu20PafpuS5W8jdcx+e4gtofv8fcVZfN+66zgsdL+AJeVIK\nSBoXIfDlr8TQ884Mqs8d5Finh4vnpfZwPdtpfFNleDelOHPMUHCI/kB/xhdOY1RZqggRINfey2M3\nL+RrV5SwaZ6V2l4v/7Ozk089UcfeVhc3rs7FPniMkuc+wX51gJAQLCvfgnPxh/HmryL37ftQeRN7\nL03G+tz1hAmnvI3hjCj2hn4fP97RSZE24oZ4zWoHxTYdQgj+5aIiagqN/Ne2tmGH/HSyt+kG6wlr\njITMyedtnzZfdsBbcB5hrRlT6xupV5YSc/N2PCUbkJqJ057GwqAXBAIjfvCZoiarhqAMcsJxAkPn\nPqRKiy+vhv2tLhy+0JS8YUZzxUIb3a4gRzoSP2gDtkrc5ZdiO/kEIniW88iE/GTvv5+yZ29C62qj\nc9M9dF7+owm/b4FwgGdbn2Vl1srh6MYYQ74Qve7UzWfevBVonc3DimNnoxNJ8maYGLn6XMpN5bNW\nsZfadKhFJMujyu9E13viHY+YDLs6xijSRmb7i8t6yTdredfCLP7t0hJ+f+NCfv+RKm6/pJiPnpfL\nTfbjFL94K2Gtma2Lt6BT6VhiWwJCRfeFX0cVdJG75760ZFhoWUi2Ljtlc8yMKHYBfPeqMn707khK\n0vgIVK1acPeVZVgNar7+YgsDniA7uncAkZwlyaIdqIuYYVKwvU2nLztqbSTvScvrKZtIdH0n0bg7\ncCXyhhlFTLHPCwr03ePndkmXmJ39yOCRSP71vBqkxsCrtQ7MWhUXpJHBbzw2VlrRq8W43jEAg8s+\nhto3gKU2/eluqgi/k7K/3kT24V8xNP8qmt//J1zzr570u7a9azu9/t4xo/V9rS5uebyWf3yyAX8w\nNRu3L38FAPqeiHlsR4OTYquW+dmpu1Cusq/i6ODRjOwelGm0akFplg519xFKn7me0r/eRFdbxNVz\nukwxZzosyJAem3XsnsXFNh1bltj5f7lvU7b9iwSs5bRt+Q0H3PUssy1Dp4oEVQXsVQzUfBJr3V8x\ntr2ZsgwqoWJ9znr29+9PKSp+RhT7vGx9dMpuYL55/phoxhyThm+/u4x+T5C7X27i8ebHqbJUpeQX\nqhusJ5CCfR3ifNmny85edgkaTze6vtQibk0t25EI3KWbJi3b7G4mR5eDIXsx+mkYsZs0JhZYFnB0\n4BD63uN4C9cQCEneaHCycZ4VXQbMMDGMWhUXVlrYXudIuKAOkZmQN6+GrGO/P2spfm0nn0A3UEvH\nZffRffF3CBuyJ63jC/n4Q9MfqLJUDecOCYUlv93bze1/bUKjEvS6g7x4ejAlWXw5y5BCjaH7EG5/\niH1tbi6aZ01rMXGVfRX+sD9hdPGMIyWf0r3M13u/AkSSofXV/RmNUE95rWI8ttUNoQmW0htK7Nli\naH+LgjfuwluwmrarfkW3WkWTu4lV9lUjyg2s+DR+WyV5u76b1sxyfe56vGFvSu6oM7Z4GqPaVs1p\n5+kxgUrV+Ua+fEkxx9w76fR2ckPFDUl/WYV/CI27M2XFnqXNQqfSTUuQEkSiJgFMral5x5ibt+HL\nX0XYOPl+js3uZspN5fjyV6DvOw4Z3scVIuaYU87T+AjhLVjDnpYhhvzhjJphYlyxMIsBb4h9beNk\ndBSCwWUfQ+dswtTyWsb7H0PIT9aJP+Au3oC7IvmI0WfbnqXH18Mn5n8CIQR97iB3PNfMb/f2cOWi\nLH5zfRXV+QYeO9g77kMsEVJrxJ+9CH3PYd5ucREIyZTNMDFqsmpQoZp15hgRcJP/xtf4hOMX7Awt\np37L7+m44kfUa1SUB8Nop2GG0ecOsr/NxTxTFfWu+oSBlFknHiNoyKFj80+ROuuw4h09AJVqPT0b\n7kI71Er2wQdSlmWFfQVGtTElc8yMK/Yl1iX4wr6Eq/FXLLSQW7KdkLeYnq7JE3nF0DkibaWycAoR\nT53pcnkECBlz8eYuT8ntUe3qQN93ImG06WiklMOK3ZtXgyroRTcNft41WTUECHHIYMBbsIpXax1Y\n9aq08m1PxgVlZsxaFa+cGd8c46q4goClJJJmYJqx1v0NjaeHweW3JF3HEXDwx+Y/si5nHSvtKznY\n5uJzT9ZzuMPN7ZcU85XLijFqVXxkVS6tjgCvN6TmAeHNW4Gh5wg76wfIMqipKUwt/DyGSWNisW3x\nrFLs2oE6Sv/2MSz1z7On/FN8MnA7zV4TQWs5tbYiFnhd5E3BpXA8ttc5CEvYWLwEf9g/JoWv2tOD\nqeU1hqrej4xGDh8cOIhVY2W+ZeyA0lu0DsfCa8k69nDKM3atSsua7DUp+bPPvGK3RQOVEkz/Xu9+\nHTddlIXfw493dHCiK7lpTGzLuFR82GNMp8sjgLtsE/qeI0mvkpubt0fqJWFf7/H14A17IyP2vKjt\ntTvz0YRLs5YiJOzKLsErjOxoHGLTPCtadebD0HUaFRfPt/JGvTNhznAAVBoGl96MsWv/sK15WpBh\nso49hC+7elLvpHgeb3ocb8jLx+fdwiP7e/jyX5swalXcf+08tiyxD89EL55npSxLx6MHepPenQgi\ndnZVwEVX8yk2VFhS3n4unpVZKznjPJPUdnnTjbn+BUr/9lHUvn46Nv+MvhWfRaKisd9PMBykLeig\nKLsGa92z2E4+kdG+X6l1sCBHz0VFkd2o3uwZaR+31D6DkCGci64FIoOqQwOHqMmqGTcwsG/tFwnp\n7eS9+R8pmw3X565nIJA4MV4iZlyx5+vzydZlj8n0GJIhnmh6gkpTJd+7ZAs5Rg3f29aW1OKSbrAe\nqdISsKS+Wj5dG27EcJdtQiAxte5IqrypeRt+W2Vks5BJiI0qyk3lBC2lhPR2DD2ZX0C1CB1L/AH2\nGE281TyEJxBOKSjJH/ZzbPBYwhTAibiiyoYrEGZ38/gbbDir3k9Ya8Z2/JGk5UgVU+sb6AbrGVz+\n8aQX5ds97TzX/hybizazp87Cr97u5tIFNn7xwXlU5Y70cFKrBDesyuFUj5d9rcnHO8Qe4otDp9PK\nqBnPPPM8woRp9bROqZ0pEQ6Rs+c+Cl+/A3/2Ylqu+QOekg2U23UIIlkeO72dBGWQvMprcJdeTO6e\ne9F3Z2am0eEMcLTTwxVVNoqNxWzI3cBfWv/yji+5lNhO/xlP4RoC0cjiNk8bPb4eVmWvGrfdsN5G\n7/m3Y+g9hu3EoynJtDZnbUqR5DOu2IUQLLEuGZPpcWfPTlo8LVxfcT12o5YvX1pM04A/YfDSaLSD\ndZELrkp+k4cYBYYCXEEXQ8GJd4VJF3/OEoLGPEwtk7s9Cr8TY+eepEbrAM2edxQ7QuDNWzEtC6j6\nnqOs83o4Kod4vaGfLIOa80rGN8OEZZi6oTqebH6Sbx7+Jje/eTN3HrqT/zn9P0n1t6bUTJZBzasT\nmGOkzhLZiKPhRdTu6XkwZx39LUFTEUPzrky6zsMND6MWam6svJEdDU4W5Oi564qScVMuXLkoi1yT\nhj8c7Em6j4CtApfKwjr1GdZO0SsplmQv2eRXmUb4nRS98v+wH3uYweobaLvql4TMkcVRvUZFkVVL\nY79vOEdMqamcrou/S9BUROG221F7kr9u47GtLvI9iw1Wbqy8EU/Iw1OtTwGRaGutsxnnwne8m2L2\n9dELp6NxzXs37pKNZB/+FYSS93KxaCzctfyupMvPuGKHyAJqp7eTAX9kqhGWYR5vepxyUzkb8yJh\n2eeXWXjvEjtPHO7jaOfEo5lIjpjUzTAwzS6PAEKFu/QiTG07J90swtS6ExEODuden4xmdzNZ2qzh\nvBm+/Bq0gw0If2o228kwdu3jfK8PvwxxdOAkNUXGhNP/PX17uPf4vdyy6xa+uP+L/Lbht/T7+7mq\n6Co2F25mZ89OjgxMPqNQqwSXLrCys9E5JqQ8nsElHwEZyvi0HEDfcwRj5z4Gl90MSe7Uc9Jxkh09\nO7i27FqMws7RTg/ryibeSUinVvHhFTnsa3UnvRWaRHAwvJALdbVTDg4rMZagFuq0toWbKlpHI6V/\n+zjG9t10b7iL3vV3jLnWldmRnDHxPuxhvY3Oy36Ayu+kYPu/T3kTllfODLK0wECJLeKyOM88j4vy\nLuLZ1mcZ9A9iPfMUIa0FV+U7udYPDhwkX58/+eY2QjC49CbUvgHMKS72r8lek3TZWaHYR9vZd/Xu\nosndxPXl14/Y7utzGwrIM2v4/rZ2fOOYZETQi8bZkrJHTIxYkNL02tkvQRUYwtA1wdRRhrGdeoKQ\nIRtffnJunrGF0xi+vBUIJIYM250NnftYoS9BIOiTp6jOG7lYJ6XksabH+PbRb3PUcZS1OWu5bfFt\nPLj+QX6y9id8puoz3Fp1K3n6PH5d9+ukUjdfUZWFLyTZ2Tj+QypoLcNdfhm2U38c3k0qU2QdfYiQ\n1oJjUXIRo1JKHqx/ELvWzgfLPsjhDjeBsExqgfl9S+2YdSr+cCC5VAmne7y8FaiiPNg05f1gNSoN\nJcaSpLOuZgpj25uU/C0Sk9B+5c9xLv5wwnKVdh3NA35a3C1ka7OxaCIRtv6canouvAtj135y9t+f\nto2wFAgAACAASURBVBxNAz7O9Pq4vCprxPEbK2/EH/bzZOMfMDe+zNCC9yA1ke99SIY4PHiYlfaV\nSXnueYo3EDQVYD3zdNpyTsasUOxVlio0QsNJ50nCMsxjTY9RYizhovyLRpQz69TcfkkJLYN+/m8c\nk4zW0YhA4k9j4RTOwogd8BSvR6o0E3rHZB39LcbOvfSd9y9jcnknIt4jJkYsC6Q+k3b2cBBD1wG0\nBWsp0FWgNtVTnf+OrTgQDvCjUz/ikcZHuKzgMn55/i+5rfo2Li+8nBzdO+6aerWeW+bdQp2rjlc6\nX5m025oiI/lmzYTeMcDwaMhS/7f0P+MoNM4WzE1bcS7+B6Q2OVPHrt5dHHcc58bKGzGqjextdaFV\nCVYUT76to1mn5gPLsnm93knL4OTT9R2NQxyQCxHIjCweV5gqzt6IXUpsxx+haOsXCJqKaH3P7/BG\n985NRGW2nkBYUj80dtOdoQXXMDTvamyn/pj2qP3VWgcCuGzByLWKMlMZlxZcyt86XqSXwAgzTN1Q\nHUPBoUnNMMOo1Dir3oexbee0mQ0zotiFEFcLIU4KIc4IIe5Itb5OpWOBZQEnHCfY3bubBlcD15df\nn3CxYG2ZmfcttfPHQ30c6RhrknknR0x6I3aLxoJRbZw2X3YAqTXjKVw7rj+7vucIOft/xlDllTgX\nXptUm/2BflxB1wjFHtZZ8dvmZdTOrus7iSroxluwhiy5CLWxkQW5kfvkCDj45uFvsq1rGzdV3sRt\ni2+bcIPhTfmbWGJbwsMND0+6kKoSgsuqbLzdMsTD+3o40ObCm2DW5i1ciy+7OrIvaoZc4LKO/S6y\nwcfSG5MqHwwHeajhIcqMZVxZFLHH721xUVNkTNpU8qGaHDRqwWMHJx+172hwEohGoGYijUS5qZxO\nb+f0e8aEg+S9+R/kvX0v7rJLadvyIEHrxA4Pkd2UJG3eloQpRlyV70IVcKX1gJNS8mqtg1XFJvLM\nY7+3N1TcQEiG+N/CefhzlwwfH89/fSKcVe9HyDCW2mdTljMZpqzYhRBq4H5gC7AMuFEIsSzVdqqt\n1ZwZOsOjTY9SZCjikoJLxi37ufUFFFq1fH9b25gft26wHilUw6vVqSKEoMhQRIenI636yeIu3YRu\nsB6Ns2Vk/wEXBa9/laApj54Lv56090WzK27hNA5fXk1kxJ4hJWfsimys4S1cQ9A9H6EK0BVooM3T\nxlcOfoWTzpN8ufrLSQWUCSH4zILPMBgY5Inmye3iH1yezfwcA7/Z082Xnm3ifQ+e5PN/rufnb3by\nWr2DQW8wGrB0E7qB2rQTrsWj8vZjPfM0Q/OvIWQqSKrOCx0v0OZp45b5t6AWavrcQer6fCn5+eeY\nNFy9OOv/t3ff8W1X5+LHP0fLsmR5753YzrSJMwlkElYClFWgjG5o4RY66AZ6WyiFDnr7a2/poBd6\nuaWhrEJYIcxMVjY4O7Fjx/Gesi1Lssb5/SHbsWPZliU7cuzzfr38AlvS18eK/PjoOc95Dm8dsQ7Z\nQ6aqrYuyZifFU9N8f8QbRiewj3lljJQk7HiE6GPraCm6jbqVvw3o3VB2rAGhteHw2vw2BbSnLkQi\ngvq3L21ycqK1a9CNdjmdrVzd3sGLEV4anKcyBp+2fkqOKYc4w/A7kHu4o7OxJ8/1pWPG4ohHKWVI\nH8B5wJt9Pr8HuGeox8z3/Sj9Po5ctVIWPlkoC58sHHCbBGn93OdkaWmpLC0tHfL29ieuCenx4b69\n7f9uCurxJZ85TxY+WSh3Htzp9/b2z35mVMbX8fiVfm9fd0GyPP+f58vXdr8W1PVfWJEgt+3bFvT4\n1s65VM7+z/Xy/b0Hx+Tfp/2aNQE9vuRwyaDjW//RvjF7/dRce71s+8ct0vXL3HH9+h7Pt//o6Q/l\n1B8P/vptXXurPPlQit/bP15TfEbGX1hYKAOJy6ORiskA+ibkTnZ/rR8hxNeFEDuFEH77T8YYfIsV\nofZ96OnDfjaKKluPpez1oB7b4bYRpYsiVh/r9/bBDj0eEenFWLfH7016jYHfFP+GmTEzg7q0Rgie\nPP5k0EM7LzsKj4T/2jI277S8hsDq9NdWrPX7dYNWkJ8wdGfOobiGaTNg0mtwJBahC7I9rOLLrw9a\nLio9WI6/QXTmKr83J0QM3+7jTBIj2eHm9wJCXAesllLe1v35F4BzpZR3DfaYoqIi+fLLA1eE/3z0\nzyxKWMSC+MEXT/qyu7zc9kIZQsAT100lQuNhytrzaZ39BVrmfSu4Hwhfmd6D+x/kl+f8klkxI84q\nBSx+x2+JOfwc5Z/bhNbRTOZrN+KMm0bNJX8bcQ3+vZ/cixcvv5rzq/43eF3k/msZbdOuo3nh90Ma\nr77lGFmvXk/9kp+z0XABP3qjkpuWHcMqDnNH3h1E6YM/YOO5E8+xtmItDxY9GPQhwC8faOEP22r5\nwfI0rshxk/3vy2gruNZXNhcEy+HnSfr4Yaov+Z8hF/R6HGo7xI8/+TFr0tZwe/7tAEgpufHpY8xM\njuT+i0fehfBoo4PbXzxOfoJv23qDzY3V0b+K6NaFSdwyNxFD82EyX7uRumUPY5uyZsTfq687d95J\nhimDe2fdG9J1TmdoOkT6m1/FFZNL9SVPIPUjb39w57bfUen5kH8vfQa9duA6nLFmO+lv307tqj/4\nThwLwP66Tr75cgU/XpnOJdNiBtweVfoqye//lOpLH6c6bgq377idpUlLiTfE82Lli6w9by0m3fAL\n46dL/OB+osrfpuL6t5H64R9/1VVXUVJSMmx+djRm7FVA38RuZvfXRuwbBd8IOKiDr/vfN5ekUN3m\nYudJG/r2kwjp9h1gHYIcUw7gO0l+LHVmLme/TrB2/yMkbf0xUmioX/pQUBurTq+I6aXR0xU/Y1R2\noEbW9eTX53O4wVdOeNPUy/j+jO+HFNQBrsq4iuSI5IDLH/35zMxYzkkz8eeP6qjzxtAxZTWW0lfQ\nBFHHL9wOYg88hSNhFo7ug7qH4vK6ePToo8Qb4vlC7qnTkyqtXTTY3EH30SlINHLFjFi8EhLNepZP\nsfCVBUn8aGUav708mydvmMrNxb7jIrti8/DqjKPyb51lyhr1yhhtZz2p730LryGa2gv+EFRQB7Bp\njuLpzOV4s/+KIWfyHLxaI5HVHwV8zY2lbei1giWDHFBiOfoSXZZsHMnziDfEsyZtDZvqNrGlfgsF\nloKggjpAe95VaNydgx5KHqzRCOw7gAIhxBQhhAG4EXhlFK4bkPkZURh1wldO1hpaRUyPxIhEzFoz\nFZ1jG9gb46dxd0oSz7bvpqztCI3n/SeeqGE2OPhh7bLS5m7zH9gBR2IhhuZDIW/cMNbvwm1KxW1O\n43CjncwYA1ERo3NgdoQ2gi9N+RLltnLeqQ3uRa4Rgu8vT8PlkfxhWy2tM25C47ZjOfrSiK+VsP1X\n6NsraZl7V0AL2C9UvkBlZyXfKPhGv1/yXSd9deWh7Aj97vI0Hr9uKg+vzuLuZWl8YV4il06LZV6G\nmezYiFOL1BodzoRZo7aAWmuvHVEP8KEIVyep730bjctG7YX/jceUFNR1WrtaafFU4+mcyu5q/5VU\nUhuBI3luwAuoXinZVNrG4qwozH52BOut5UTW76G94Jre18K1mddi0Biod9YHXubohzO5mC5LNpZj\n64K+hj8hB3YppRu4C3gTOAg8J6Ucw05M/em1gnPSTOyusvXm14PdnNRDCEG2OXvMZ+yPHX+CBq0W\njZS8kTkPW07gW9X76tsjxh9nYhEajxNDy9Ggx4qUGOt2Y0+ZB0JwpMHRr359NCxJXMKs6FmsrViL\nzR3cRpvMGANfWZDE+xUdvNWagT1lvq8vxwjaF1uOriP62Mu0FN2GPf28Ye9/wnaCFypfYHnS8gHv\nOHdX2Uiz6Ht3MY41Z+I5RDQfGtF2dX+yzdl48VJtrw59UF4PyVvvwdByhLrlv6IrLvBOrafrOW83\nSTONPVWDv0bsaedisJYFVCd+pMFBs93Dsin+++xYjq1DCh0deVf0fi3WEMsVGb7Pi+OKR/Ij9CcE\n7flXEVm/B13b6MWbUaljl1Kul1JOk1LmSSkfGo1rjsT8DDMnWrvwNh3DZU4NKFc1nBxTDhW2CkJd\ngxjM5vrNbGnYwi0JKyjGxEZj8DPfYQN7T41zCCcq6dpPoLM34kiZR3Onmwabm2mJoxvYhRDcOvVW\nrC4r79QF/9b0uqJ4picZ+e/3a6nOuxG9rQZz5aaAHmtoPkzC9l9hT11Ey5w7hr2/R3p49OijRGoj\nuW3qbf1v80r2VneOSTvjwTiSihBely+4h6C3Z0yIO1CFy07i9l9hPrmFpoU/wJ45/GExQ9ln3YdR\nY2R+0gzfbl6P/99Pe/pigIBm7dsrOxDAwiw//04eF1Glr9KZtRxPZGK/m27IuoEfzvghs6JDW4fr\nyLsCKTRYRvEUsHGx8zRU87p/cWRTWcj59R455hxsHhuNXaE3FTpdg6OBx449xnTLdK6e9W3m5d3C\nCfvJoGdHlfZKIrWRJBgS/N7uNqfhNsaHtFHpVH59Xm8Pk+lJweVIh5JvyScvKo+t9SM7jKQvrUbw\nwxVp2Lo8PFw+DVdUBnF7/4KuY+jnV3S1k7L5B3gN0dQt+2VAO37XV6/ncPthbsu7rbeyq8ehBjs2\nlzfkxlwj0bvbOMR0TEZkBho0QefZtbY64nf9gex/+3aCWmfeQtuMG0MaE/hm7DOiZzA/w4LDLTk0\nSD+drrgCPBGxAQZ2G9OTjMQYB65tmWo+ROdopj3/qgG3RWgjWJK0JKjTqvrymJKxp5/vC+yjdArY\nhAjsU+IjiDcKou0nQs6v98gxj94CqscrKW9x4pUSr/Ty+yO/x4OHu6ffjVZoWZzgm1181Bj4Yk9f\nlZ2VZJoyB3+BCXFqo1KQjHW78RjjcEXncrjBgUb4FvbGwrKkZRztOEqNfeBZk4GaEm/klrmJvFtm\nY2vut9F11pHx2o2DL1JJSfL7P0PXUU3dit8EdFpVvaOef5b/k/lx81mRtGLA7btO2hBAcXro7yAD\n5TEl4zKnhrwDVa/RkxaZNuLAbmg6QNLW+8h+8QpiDvwDe+pCqlb/L00LvhfSeMC3s7mis4LCmEKK\n080IfKkuv4QGe9oiX2Af4l231eHmYL2dRVn+F03NxzfgMcTQmTZ8Si4U7flXouusJ7ImuBhwugkR\n2DVCcFGKjQjpDLqr4+lGozLG7vLy4r5mPv9MKV99voxbny/j13ueYZ91H1+b+jXSIn0LpUnGJPKi\n8vioKfjAPlgapocz6RwMbeVonEP3WhmMsX439uTu/Hqjg+zYCCL1Y/PyWZq4FPAdtBKKm4sTmRof\nwQ8O5HLkkrW4LNmkbP4BiR/9YkCTsJgDT2Gu3Ejz/G/jTB4+Zyql5M/H/gzAHfl3+P2juqvKxrRB\nZoJjyZl4zqi0kcgyZfVv3ysl2s4G9NZyDI37MdbuwFS5iaiy9UQfeoa0N28l8/VbMFduom3G56i8\n+hXqV/7W93yGOKuFU/n1wphCLBFaChKN7BlkARV8zbZ09sbeNiP+7DxpQwLnZg8M7MJtx1y5CVvO\nRaANrKNnsGyZK/BExGI5NkjdifSO6OSlM/uKG0PLY+qhDk5oMhiNrQJR+igSDAlBBXarw81L+1pY\nt7+FNqeHwpRIri2MY/3xA3zY8QIaeyF1NXPoiPP0VpUsTljM2oq1NHc192uWNZwOVwctXS3DBnZH\nz1v0pv0BLQj2pe2oQd9RjXXmLUgpOdww+AxnNCQZk5gVPYutDVu5IfuGoK+j1wp+sCKNO9eV88f9\nUXxv9f8Sv/dPxO7/P4z1e6lb9itccflE1O0hfvd/Y8tehXXm5wO69uaGzexp2cPXpn6NZOPAVgOd\nXR4O1Nm5YY7/9NhYciYVElXxFlp744C88EhkmbLY3rQdl9eFXqMn8cOfEz1E9YbLnErTgu/Sln81\n0hDagR/+7LPuw6AxkG/JB3wp2BdKmrC7vH4nGT2nXEXWfIwrNs/vNbdX2oiO0PpdLzKd3IrGbadj\nyqWj+FMMQmugY8oaoo+8gMbRijciBr21jMjaHUTW7sBYtxutsxXIDuhyEyawFxp8Ow4/aE/himHu\nG6hcc67fs1gHU9PWxfMlzbxxqBWnR3J+ThQ3zkmgMNVEl7eLLa7n6HJaSPHcwhM7Gnl6bzOXz4jl\ns0XxvYH946aPWZMW+OaSnsM1sk1D/4M7E2YjEUQ0low4sEfW+TYLO1Lm02Bz02L3jPrC6emWJS3j\nsdLHKLeVk2vODfo605MiuWJmLG8csnL7uckw/zvY084ladtPyFj/eVqKv0HMgadwR6VTf/79Ac0s\nO1wdPF76ONMt01mT7v/f6pOaTjySM7pw2sPReyxiyYgO3D5dtulUZUyBo5PoY+ton3o59vTz8erN\n3R8mpN6MV2fCY4wPaF0iWD359Z7GcnPTTTzzSRP76jpZmDlwouGOSsdlySKy5mPaZt484HavlOyo\n7GBhltnveQJRxzfgjkzEkRx4H/RQtOdfRcyhf5H67l3obLXoHL4GcC5zGp2Zy7GnLoB/Px7QtSZM\nYE+wV9BELB/UaUctsOeYc/ik9RPcXje6YTYNbTnexi/e9e3Luig/hs/NSejuROfzj+P/4ETnCX42\n+2fMi5/FsUYHz33axIv7mlm3v5nPFsaTEpE28sA+TEVMD2mIwhUzhYgRVsaIrg7i9v4FlyWLrth8\nDlf4cppjsXDa15LEJfxP6f+wtWFrSIEd4NJpsbxyoJVt5e2snh6LPf08qj7zHEnb/pOEXf8PrzaC\n6ov+FPAsc0PtBtrd7dyRf8egx5XtqrIRoRVBHywdiq74GUiNjojG0AJ739OUzv30WTyGGBoX/WhM\nZuPDaXe1U24r56acUx02C1NN6DSwp8p/YAfoTFvsa9PhdQ04tONoo4NWh8fvu0/R1Y6pahtt068f\n0z9WfXXFT6czdREG63HsaefiSF2IPXXhaR0vJ1lg11vLqDHmsLfahssjR+Vg5RxzDm7ppspe1buY\n6s97x6w8vLGaGcmR/OzCDJKiTnsBtR/l1epXuTztcubF+/765ycauXdVBrcuSubJnQ08+2kz0enT\naIjZRrurA0uAOzkrOysxaAwkRQy/4cOZWITp5GbfYlKAOc/EHb9B11lP9eq/g0bLkQYHWgF5CRHD\nPzgEMYYY5sTNYWvDVj6f8/mQKg9mJBlJs+jZWNrG6um+XjqeyARqL3oUy9F1uM2pAddWu7wuXqt6\njblxc5kaNfh6zq4qG0VpJgwhnmgUDKkz4oybHvIO1AyTrzKmpn47pqr3aZr3nbAEdYADbQeQSApj\nCnu/FqnXMDM5kj3VQ9ezxxx5noiGfThT5va7rbfM0U/VkvnERoTXRUfu6lH7GQJRe8ljo3KdcbV4\naj6+AX1r6cgfKCUG63G8cVNwuCUH6wM7Umw4PTPFofLsbx5p5eGN1RSlmvjNmqwBQR1ge9N2NGi4\nJfeWAbelROn50cp0/nx1LknMxYuHu97e4LfXvD89C6d9T5oajCOpEK2zFV1HYB0fzBVvYyl9ldai\nW3tPcTrc6GBKfAQRZyBgLUtaRp2jjiPtR0K6jhCCC/Ki2VVlo9XeZ6OS0NA+7VrsGecHfK3N9Ztp\ncbVwTcbgJyk12lxUtHSFJQ3Tw5lU5KuCCqF8zqAxkGpMpab+I9yRibTNCH69I1Q9+fVplv5/gOdl\nmDnS4KDd6f/ndKQuRAoNJj9ljx9X+ha3YyMHzm+jyt/EFZXRWz56thk3gV3XVkHK1ntIf+tr6NpH\nVmKltTegcXUQlTYNjRiiBGqEMiIz0ArtoK0FXjvYwm821TA33cwv12QNekDxPus+8ix5mHWD/6LP\nSI7kr5ctx6SJoU3zCd96pYKH3qvik2obW8raeHFfM3/7uJ6H36viu69V8KVnS/n7jvqAKmJ69ATn\n2JK/D/sLr+2sJ/HDh3AkzKblHN/GGyklRxocTBvjNEyPxQmL0Qt9yNUxABfkReOVsPl48Oe/eqWX\nl06+xBTzlCEblfW8/sIa2BOL0LjtGKxBTJT6yNGYKPfaaDnna71HwYXDvtZ9TLNMG3Bwy9x0MxLf\nmoY/3ohonAmzBpQRtjk8HBqkzFHjaCay5mM6ci8dlWqecBg3gd1y7BWk0IDXS+o7d6FxtAT8WEN3\njxhNQh7TEo3sGqXArtfoyYjM8Dtjf3FfM7/bWsuiLDMPXZo56Mk4Do+DI+1HKIopGvb7aTValiUv\nxmA5wk3F0Ww53s7dr53g/neqePSDOv5d0sy+OjsujyRCJ3j60yoanY0BB/auuGm0zv4S0cdeInnr\nvYNvO5dekt7/GcLrpGHZQ725yZp2F21OD9PHeOG0h1lnZn78fLY1bAu6MViPqfER5MQa2FgaXLkn\nwO6W3Zy0n+TqzKuHTA3tquok1qhl6hinq4bi6C7ZHOr4xWFJLzOayzmh19E8dbRWrkauw93Bcdtx\nv79DM5MjMerEkJM5e9q5RDTuQ3R19H5tV5UNr4RFfnabmiveQUgPtjNRDTNGxkdg97qxlL5KZ/oS\nalf9Hp2tltSN3wn4QGJ9T4+Y2KnMzzBzsN6OrWt0dnDlmHMGBPZnP2ni0Q/qWJIbxQOXZA6ZRz3Y\ndhC3dFMUO3xgB1icuBiH10FxXjVPfS6PX67O4n8+O4WXvljAhlun8/RN+fzxqlweuTybSJNvV2yg\ngR2gef53aJp/N1EVb5H67jf9Hn4cfegZTDUf0bTge/1OojrS3dFxevKZCewAy5OW0+Jq6a1hDpYQ\nglX50ZTUdNLQEVwztHUn15FgSOits/dHSsnuKhtzM8xowjjbc0elY09dhOXIv4NOx5gr3mVaWy1u\nIagZgx3YgTpg9eXXZ8fMHnCbXisoSjUNm2cX0kNk3a7er22v7CA6QssMP+8+o45voCtmKl2xBaPz\nA4TBuAjskdUforM30F5wNc7kYhqWPUREQwnJ2+4L6EVpsJbhMVjwGBOYn2nGKwd/azZSOaYc6p31\nvU2pnt7TyGMf17NyqoWfXZSJQTv0U1jSWoJWaJkZHdgBFEUxRZi0Jj5q+ojkKD3nZkeRl+Db5NJ3\nlhhj1HF+vu9ntLaNrFbaOvuL1C95gMi6XaS/9bV+h3DoW0t9Nd2Zy2kv6H9S/OEGO3qNYErcmQvs\nC+IXYNQaRyUds3JqNBLYVDbyWfux9mOUWEu4MuPKISukylu6aOp0syCMaZgebdOvQ2+rIbL6g5E/\n2Osmbu+fyTamApy5w6392G/dj07oBuTXe8zNMFPR0kVzp/9Gb46k7ja+3ekYr5Rsr+xgfubAMket\nrY7I+j2+2vWzNA0D4ySwRx9bh8cYR2eGr0GQLecimhZ8D/OJ90jY9bthH6+3Hvf1iBGCWSmRRGhF\nb7vUUPVUw5ywnaCy1ckTOxq4YGo0963KQOen9vV0JdYSplmmEakNLD+p1+hZEL+AHc07hk0/JMc3\ng9Ty3G6JZ5gTdk7XkXcldRf8Dn3rcdI3fAVdexV4XCRvvQ+pN9F43k8HvLAPNziYmhAxKhVHgYrQ\nRnBu/Ll80PgBrhDbDmfFRlCQaAwqHbOuah0mrYlLUi8Z8n4fVvhy+AvOYH+YwdiyVuKOTCT6yAsj\nfqyl9DUMbeXEzL4Dgei/A/UM22fdx3TLdCK0/lNb87pbNgw6a9cacKTM6+0bc6zJQYvdw7l+0jBR\n5W8C+PLrZ7GwB3aNoxlT5Rbap17eb9tu26xbsM68hZiDT/tOiR+CwVrW2yPGoNX0tvEdDT2Bvbyz\nnH/va0anEdx5forfDQ2n63R3cqz9WED59b4WJyzG6rJyqG3oDn01jioS9OmUNbl5+6h1RN8DfAd9\n1Fz8F7TOFtI3fJmkDx8gouUwDef9FE9k/3cBXik50jj6rXoDsTx5OR3uDva27A35WhfkRXOowUFV\nW+Btbescdbzf8D6Xpl067IEKm8ramJ0S6bc66ozT6GnPvxrTya3DNkDrx9NF3CeP4UiYjSfnIlKM\nKWGbsdvcNso6yvymYXrkJRixRGgCaON7HG1nPdtP+O63wE/tu7n8TZwJM3FHD17efDYIe2C3lL2O\nkG7a868ecFvTgu/SkX0h8Tt/h7nibb+P1zha0DpacMWeqimel2GmorWLBltoMzyA5IhkIrWRHG0r\n583DVi7MjybeFFj5/4G2A3jxBpxf7zE3bi56oR+2d0xlZyUzY3OYmWzkiR0NONzeEX0f8DX6r770\n74DAUvY6bQXX0pm1csD9Tlq76HR5mZ545isj5sTOwaKzsKVhS8jXWjnVd3bpphHM2l+tehUhBFek\nD72AWNnq5FiTs/d7jAdt0z4LQmA5+mLAj4k+8gK6ztreQ0ayTdlhC+wH2w7ixduvfv10Wo1gTpp5\n0IM3oG97gY/YXtnBtETjgN9jXVsFxqYDZ7x2fSyEN7BLieXoOhyJRf57OQgNDUt/gTPpHJK2/oT0\n9V/0fbzxpe6PL5P29n8A/U9N6ikzG+oveKCEEL4dqI2lOD2S688JvI9LSWsJOqFjumX6iL6nSWdi\nTtwcPmr8aNB+8E6PkzpHHVmmLO5YnEJTp5vnP20a0ffp4YrLp3rNkzQX/8egXfjCsXDaQ6/Rc37i\n+Wxv2o7T4wzpWqkWPbNTIgNOx3S4Oni79m2WJS0jMWLoviuby3xpmOVTw7OJxx+POZXOjGVYjq4D\nz/ATHeHqJK7kceypC3uDYZYpi2p7Ne4RHFYyWnry6zOiZwx5v3kZJmrbXdQM8k6sK64AjzGO6JL/\nQzYc9FsNE1X+FgAduUOn284GYQ3sEY37MFjL/PY67iF1Rmov+D223Ivx6qN8/Sl0pu4PIx5jLLbs\nVTiSTnXkm5oQQYxRy66q0VlAzYrMptF1kvmZJqbEBx7YSqwlzIieMWhucCiLExZT76znuO2439ur\n7FVIJFmmLIpSTSzLtfCvvU2DLiANxx2VTus5Xx/0kJLDDQ4itIKc2PCU8C1NWorD62BH846QHptP\nowAAGOxJREFUr7UqL5qyZifHm4f/I7GhdgMOr4OrMwa+ozzdprI2ilIjSTKPgzRMH23Tr0PnaMJc\nuXHY+8aWPI7W0UJznyMBs0xZuKWbGkfwbZSDVWItocBSMOzv0Nx0X6DePVieXWioP/8BpL2Fl/U/\n4cudTyJcfTYySknU8Q3Yk+fiMaeO1vDDJqyB3XLsZbxa47ALFV5jLA1Lf0HtxX+m9uK/9Pn4K7UX\n/5W6lf+FNJzKl2mEYF66md1VtlE5AcnRmQJaO6tHcFBKh6uDso6yEefXeyyMX4gGDR83+T8ooGcx\nq6fU8evnJuPySJ7c2RDU9xvO4QY7+YnGgNYWxsLsmNnEGeJGJR2zYmo0GjF8dYzL6+K16tcoji1m\nStTQff5PtDopa3ayYhylYXrY08/HFZVO9OHnh7yfseZjYvc9SVv+Vb2b2cB3TB6EfprSSHW6Oylt\nLx0yv94jO9ZAgknHniEmc/bMZdyb8hde5ALyK/5F5ivXEtld569vPYbBWoZtAqRhIIyBXbjtRJVv\nwJZzUb+gPFrmZ5pp6nRT3hLa2Y9SSj6t8J2ME2MJPGjus+5DIkecX+8Ra4hlRvQM1p1cx4P7H+SF\nyhfYb93fm4qo7KxEK7S9Pd0zYgxcNTuO9YdbA5qJjoTHKznWFJ6F0x5aoWVp4lJ2Ne+iw90x/AOG\nEG/SUZxu4r1j1iH/8G9p2EJLVwvXZA7ePqDHptI2BLB8kHMzw0poaCv4LJF1O3v3fJxOY28medtP\ncMXk0rTwR/1uy4zMRCDOeJ79UPsh3xpVAJMjIQRz03317IP9m3qlZHOVYH3Gd6i69O9InYm0975F\n8uYfEnPwX0ihpSPnotH+McIibIHdXPEOGpdtyDRMKHqOywu1OmZ3VScnG3x59ZGUfJVYS/z2thiJ\nOwvuZFnyMmrttTxV/hT3fnovN394Mz/c+0O2NWwjzZjWb4v1F+YlEqnX8NjHdUF/T39OtDpxuOWY\nd3QczorkFbilmw8bPwz5WhfkRVPV5uJoo/9NcF7pZd3JdeSacwM6hX5TWTtFaSYSx1kapkd7/tVI\njc5/6aP0kvzBT9E426hf/iukvv+/c4Q2gmRj8hkP7Pta96EV2mHz6z3mpptpsXsGncyVNjlptvu6\nOTpT5nLyimdonnsXpsrNRB97CXvqooBOzjobhC2wW469jMuShSNl/phcP9WiJyNaH3J7gedLmoiL\nsBBvSBhRb/aS1hJmRs8c0NtiJDJNmdxVcBd/WvAnnlr8FPfNuo8rM65EK7Q0OZsGvEWNMer4/NxE\ntlfaRq2OH3z5dSCsM3aA/Kh8MiIz2Fg3fK54OMtyo9EKeG+QRdQPGj/gROcJrs28dtjOksebnZS3\nOFk5jhZNT+eNjMeWfRFRpa8i3P2b5MUc+CemqvdpXvC9QbtcjkVljJSSQ22HqOysxCsHVnTts+4j\nPyofozaw193cnqKJQfLs2yt97/R6+8No9bQW3crJK5+nPe8ztJ5zm9/HnY3C0rZXeLqIrNvVb4Fm\nLMzLMPPOsTa63N6g2qeWtzjZXmnjKwsSKTPkDNoM7HTWLisVnRUsSwrtRPa+ovXRLEpYxKKERQC4\nvW6/vcCvKYzjmU+aeOuoddQOUT5Yb8ek15AZYxiV6wVLCMHK5JWsrVhLnaOOFGNK0NeKNmpZmBXF\nptI2vn5ucr/t/x7p4V8V/yLLlMXSpMHbB/TYXNaThhl/+fW+2qZfR1T5Bszlb9HR/U45onE/8Xv+\niC17la/3+CCyTFnsadmDR3oG7UE/Uk9XPM1zlc8BEKmNJD8qnwJLAQWWArJN2RzrOBbQonWPVIue\nNIuedftbOFBnx+7yYnd7cXT/t77DTX5CxIAyR3d0Ng1Lfj4qP9N4EZYZu9bZihQa2vM+M6bfZ9kU\nC3aXl/crgsvJvlDSjEEruHJWHDnmHE52ngyo5Guf1dcHe6gOgKHSaXR+Z5IGrYbidBN7h8g1jtTB\nejszkyPD2vukx4pk36HRm+s3h3ytVXnR1NvcHKjrP4Pd2rCVk/aT3JR907BBTErJ5rI25qSZAt7f\nEC6O5Hl0xeb1LqKKrg6St/wYjzGRBj87jfvqrYwJ4YDxvl6uepnnKp9jVcoqvj3t26xMXonD4+CV\nqlf49cFfc+euO/FIz5D16/5cPjMWu8vLkUYHDTYXHq8k2qglJzaCZbkWbj83+MnA2SQsr0SN04o9\n/UI8poFnRY6meRlmUqL0rD/UygV5I5tNtdh9uzlXT4shxqgj15yLW7qptlf3VgkMpsRaglFrJC/K\n/zmLY6043czmsnaq21xkhDjLtru8lDU7ubl47M44HYkUYwqFMYVsrN/I9VnXh3QAx/k5URi0gjeP\nWClM9ZV5eqSHZyqeYYp5CuclDn+EYHmLk4rWLq4uPAtys0LQNu06Erf/GkPTAWL3P4XOVkP1pY/j\njYgZ8qE91VeVnZVkmjJDGsa7de/y97K/syRxCXcV3IVWaFmVsgrwVSKV28o50n6E1q7WERcf3Fyc\nyM3FwZ/zOlGEZcYuvK4xWzTtSyMEa6bHsKvKNujGhcG8vL8Fl0fy2SLfL2yOqbu1QAB59pLWEmZH\nzx72OL2xMne43hkjcKTRgVf62qOOFyuTV1Jtr+Zox9GQrmMyaFk9PYY3DrdyuME3a99Yt5EaRw03\n5dwU0OElm8ra0QhYljt+8+t9tU+9HK/OSPKWe4gq30DLnNtxJhcP+7ieYB5qnv3Dxg959MijFMcW\nc/f0uwe8I9Jr9BRYCrg8/XJuyb0lpDWqySw8i6caHbbMFWfkW62eHosA3jjcGvBjnG4vrxxoYXF2\nFNndG3IyTZlo0AybZ2/uauak/SSFseE7eSUrxkB8pJa9Q2yxDtTB7jTFeArs5yeej0FjYFPdppCv\ndduiZOIidTyyuQa7u4tnTzxLQVQBi+IXDftYKSWbSs+ONEwPabDQkbsGQ/sJ7KkLaS38akCPi9RG\nkhyRHFIzsE9aPuG3h35LgaWAe2bdo4L2GApLYPdExPVr+DWWkqP0LMwys+GINeAOiO8cs9Lq8HBD\nn/YBeo2eDJP/Qzf62tfqy68HuzFpNAghKE43j0qe/UC9nfRovd/jw8LFrDOzKH4RWxq2hNzxMcqg\n5dtLUylrdvLI7pepd9Zzc+7NAaV4ypqdVFq7xlVvmEC0Fn2FjtzV1C/9xYgOas4yZVHWURbU9zzc\ndpiHDzxMRmQGP53904ArXZTghBTYhRDXCyH2CyG8QogFgT7ObRr+4OXRdNmMWBptbnacHH4R1eWR\nPLO3ifyECOak9d9en2MaeOjG6T61fopZax7yoOMzoTjdRLPdQ6U1+A1aUvrOj501jmbrPS5IuYB2\ndzu7W3aHfK2luRaWTjGys+MVppqmMzd27vAPwrdzVSN8i/RnE7cli/rlvxzxGteC+AVU2atGHNxP\n2E7w4P4HiTXEcn/R/UQFeFC7ErxQZ+z7gGuB0Pd5j6Hzsi3ERWpZf2j41rYv7mumqs3FbYuSB8za\ncsy+Qzc63YOnOEpaS5gdM3vUSsKC1dM7Y08I6ZgGm5umTve4SsP0KI4tJkYfw6b6TaNyvVn5nyD0\nbXTWX0wg73F81TDtzE03j6t3M2NpWdIydELHe3XvBfyYLm8XD+x/AJ1Gx8+Lfk684SxYZJ4AQgrs\nUsqDUsrDozWYsaLXCi4piOHDivYhm2Q1d7p5ancji7Oj/B5y29ObfbA8e4OzgVpHbdBtBEZTerSe\nJLOOvSEsoB6o9+XXZ6WMv8Cu0+hYnrSc7U3b6XCF1mLA4XHwRt1LZOhncbQyk/WHhl+PKW1yctLa\nxcq8s2u2HgqL3sKihEVsrt8ccApsW8M2Gp2NfGfad0Lad6CMzBnLsQshvi6E2CmE2Nnc3Dz8A0bZ\nmhmxeCS8dWTwWfsTO+rp8nj5j8X+36LmmnMB3yKQv9ONSlpLgPDm13v05Nk/qe7EG2Se/WCdHb1W\nMHUEHS3PpJXJK3FLN+83vh/SddZXr6fV1co3Z36RuekmHvuonsZhevn3pGGWniXVMKPlwpQLaXO3\nBZwCW1+9nszIzIDaMiijZ9jALoR4Rwixz8/HiOoVpZR/k1IukFIuiI8/82/HsmMjKEqNZP3hVr8L\niocb7Gw4bOXawniyBmlNmxSRRGZkJv868S9u/fhW/lb6Nw5aD/Zuhy6xlmDRWXpn9uFWnG6i1eGh\nvCW4pmAH6u1MSzSe0aPwRiIvKo/MyMyQ0jGd7k5ePPki8+LmMTNmJt9dlobLK/nD+7WDLjy7PJJN\nZW3MzzATY5wcaZgec+PmEquP5d26d4e975H2IxztOMpl6ZeFtN9AGblhX5VSyonR7gzfIuqvN9Xw\naa2938KolJI/flBHbKSWL8wbfHODRmj43dzfsbN5J1sbtvJWzVu8Xv06iRGJLE1cyt6WvRTGFAZU\n/3wm9NSz763uHPGs2+WRHG10cNWsuLEY2qgQQnBBygU8Vf4UtfZaUiNH3kf71epXaXe3c3POzYCv\nS+aXFyTxt4/r2Xq8neXdFS/tTg/bKzv4oLyD7ZUd2FxevjjvzBYBjAdaoWVl8kperX4Va5eVGMPg\nG5ter36dSG0kFyRfcAZHqEC4T1A6w5ZPicas1wzIob57rI0DdXZuW5iM2TD0omeENoIlSUv48awf\n84/F/+Du6XeTa87lterXaO5qHldvOVMtBlIt+qA2KpU1O+jySGaOw/x6XyuSViAQQc3aP2r8iGdP\nPMt5CedRYCno/fr1RfEUJBr57/dref7TJr77WgXX/OMID71Xzd4aGyumWnjo0kwuLji7yhxHy6qU\nVXikZ8je+K1drWxr2Maq5FXDnhOrjL6Q3kcKIa4B/ggkAa8LIfZKKcft8d6Reg2r8qN564iVb56f\nQlSEFrvLy2Mf1zM9ycil04feVn06k87EyuSVrExeSYergwNtB5gbF1ip3JlSnGbi/Yp2vFKOqNdL\nT/+U8Vjq2FeSMYnCmEI21W/ic9mfC/gt//am7Txy6BHyo/L51rRv9btNqxH8YHkad7x0nL98VM+U\nuAhumpPAeTkWZiQbx0XPnHDKMeeQF5XHe3Xv8ZkM//2e3q59G7d0c1n6ZWd4dAqEXhXzkpQyU0oZ\nIaVMGc9BvcflM2JxeiTvdrdrfXpvI02dbu46PyWkX9gofRSLEhaNu910xelm2p1eSptGlmc/WG8n\nwaQjyTz+c8grk1dS46jhcHtgBVo7m3fy64O/Zop5Cj8r/JnfGWV+opG/XjOFf96YxxPXT+XWRcnM\nShkfjdDGgwtTLqTMVsbxjoEHd3ikhw21G5gTOyfkvjJKcCZVKgagINFIfkIE6w+1Ut3WxXOfNnNR\nfjSzUybm28VTefaRpWN6OjqeDYtevS0GAkjH7G7ezS8P/JIccw73F92PWTd4a+P8RCPp0eFtVTxe\n9da01w+sad/etJ1GZyOXpanZerhMusAuhGDN9FiONjq4/+2TaAV87dyx7TIZTklRvgNHRrJRyepw\nU9XmGvdpmB4mnYnFCYvZWLeR/y37X8o6yvxWtOxp2cPDBx4m25TNA4UPEKVTOyCDFa2PZmH8QjbX\nbx7Qynp99XqSIpJYmLAwTKNTJl1gB7ioIAa9VnCsycnNxYnj7lT50VacbqakpjPgXjkHuzcmjfeF\n074+n/t5imKLeLX6Ve7eczff3P1NXqh8gXpHPQB7W/by8IGHyTRl8kDRA1j0k6v+fCysSlmF1WXt\nV9N+wnaCT62fsjptddh3X09m4z+BOgYsEVouLojh05rOfo2+JqridBOvH2rlaKODGQHMwg/U2dEI\nmJY4Pjcm+ZNiTOEns39Cm6uN9xvfZ3P9Zp4qf4qnyp9iZvRMSjtKSTOm8fOinxOtn5zVLKNtXtw8\nYvQxvFf3Xu/JXutr1qMXei5OuTjMo5vcJmVgB7h7aSoeKTFoJ/6bluI+9eyBBPaD9Q6mxkcQqT/7\nnptofTRr0tawJm0NdY46NtdvZnP9ZjIiM7i/8H4V1EeRTqNjRfIK1levp83Vhk7o2Fi/kWVJy4as\nb1fG3qQN7FqNQMv4XxgcDQkmPdmxBvbW2LixOGHI+3ql5FC9nVX5Z38ATDGmcEP2DdyQfUO4hzJh\nXZhyIa9UvcKW+i0IIXB4HKrEcRyYtIF9silON/HWEStur0SnGfwP2onWLmwu77hs/KWMP7nmXKaa\np/Ju3bs4vU6mWab12+ylhMfZ915bCUpxuhmHW/YeATeY8XhikjK+rUpZRZmtjCp7FZenXx7u4Sio\nwD5pFKedyrMP5UC9HUuEhswQD8FWJo/lScvRCi0x+hiWJC4J93AUVCpm0oiN1DElLoK91Z3cMkTX\ng56NSWqHpRKoGEMMX8r9EnGGuHG383qyUoF9EilON7H+UCtdHq/faqDOLl+L38nWY1wJ3VWZI+ri\nrYwxlYqZRIrTTTg9kq3H2/3efrjRgVeOzxOTFEUJnArsk8j8DDPZsQYeeq+a32yqps3R/xSonoXT\nGUkqsCvK2UwF9knEZNDy2LVTuKk4gbeOWvnK86VsLG3r7atyoN5OVoyBaKPaCq4oZzMV2CeZCJ2G\nry1K5q/XTCE5Ss+D71Zx35snqetwcbDertIwijIBqMA+SeUnGnn0qlz+Y3Eye6ttfPnZUlrsHlW/\nrigTgArsk5hWI7j+nASeuG4qRWkmNOJUXxlFUc5eqtxRIS3awK/XZNHm9BBjVC8JRTnbqRm7AvgO\nIFFBXVEmBhXYFUVRJhgV2BVFUSYYFdgVRVEmGBXYFUVRJhgV2BVFUSYYFdgVRVEmGBXYFUVRJhgV\n2BVFUSaYkAK7EOIRIcQhIcSnQoiXhBCxozUwRVEUJTihztjfBgqllOcAR4B7Qh+SoiiKEoqQAruU\n8i0ppbv704+AzNCHpCiKooRiNHPsXwXeGMXrKYqiKEEYtuuTEOIdINXPTfdJKV/uvs99gBtYO8R1\nvg58HSA9PT2owSqKokxmXq/XG8j9RM+xaMESQnwZuB24UErZGchjFixYIHfu3BnS91UURZlshBC7\npJQLhrtfSH1ahRCrgR8CKwIN6oqiKMrYCjXH/ihgAd4WQuwVQvx1FMakKIqihCCkGbuUMn+0BqIo\niqKMDrXzVFEUZYJRgV1RFGWCUYFdURRlglGBXVEUZYJRgV1RFGWCUYFdURRlglGBXVEUZYJRgV1R\nFGWCUYFdURRlglGBXVEUZYJRgV1RFGWCUYFdURRlglGBXVEUZYJRgV1RFGWCUYFdURRlglGBXVEU\nZYJRgV1RFGWCUYFdURRlglGBXVEUZYJRgV1RFGWCUYFdURRlglGBXVEUZYJRgV1RFGWCUYFdURRl\nglGBXVEUZYJRgV1RFGWCUYFdURRlglGBXVEUZYIJKbALIR4UQnwqhNgrhHhLCJE+WgNTFEVRghPq\njP0RKeU5Uspi4DXgp6MwJkVRFCUEIQV2KWVbn0/NgAxtOIqiKEqodKFeQAjxEPBFwApcMMT9vg58\nvfvTDiHE4VC/d4gSgcYwj2G8UM/FKeq5OEU9F6eMl+ciJ5A7CSmHnmQLId4BUv3cdJ+U8uU+97sH\nMEopfzaSUYaLEGKnlHJBuMcxHqjn4hT1XJyinotTzrbnYtgZu5TyogCvtRZYD5wVgV1RFGWiCrUq\npqDPp1cBh0IbjqIoihKqUHPsvxJCTAe8QAVwR+hDOmP+Fu4BjCPquThFPRenqOfilLPquRg2x64o\niqKcXdTOU0VRlAlGBXZFUZQJRgV2QAjxPSGEFEIkhnss4SKEeEQIcai7RcRLQojYcI/pTBNCrBZC\nHBZCHBNC/Djc4wkXIUSWEGKjEOKAEGK/EOLb4R5TuAkhtEKIPUKI18I9lkBM+sAuhMgCLgFOhHss\nYfY2UCilPAc4AtwT5vGcUUIILfAnYA0wC7hJCDErvKMKGzfwPSnlLGAxcOckfi56fBs4GO5BBGrS\nB3bg/wE/ZJK3Q5BSviWldHd/+hGQGc7xhMEi4JiUskxK2QU8g6+Ed9KRUtZIKXd3/387voCWEd5R\nhY8QIhO4HHg83GMJ1KQO7EKIq4AqKeUn4R7LOPNV4I1wD+IMywAq+3x+kkkczHoIIXKBucDH4R1J\nWP0e3+TPG+6BBCrkXjHj3VAtEYB78aVhJoVA2kMIIe7D91Z87ZkcmzL+CCGigH8D3zmt4d+kIYS4\nAqiXUu4SQqwM93gCNeED+2AtEYQQRcAU4BMhBPhSD7uFEIuklLVncIhnzHDtIYQQXwauAC6Uk2+D\nQxWQ1efzzO6vTUpCCD2+oL5WSvliuMcTRkuAK4UQlwFGIFoI8U8p5efDPK4hqQ1K3YQQ5cACKeV4\n6OB2xgkhVgO/A1ZIKRvCPZ4zTQihw7dofCG+gL4DuFlKuT+sAwsD4Zvp/B/QLKX8TrjHM150z9i/\nL6W8ItxjGc6kzrEr/TwKWIC3u0/E+mu4B3QmdS8c3wW8iW+x8LnJGNS7LQG+AKzqfi3s7Z6xKmcJ\nNWNXFEWZYNSMXVEUZYJRgV1RFGWCUYFdURRlglGBXVEUZYJRgV1RFGWCUYFdURRlglGBXVEUZYL5\n/z2oE0ewSxj0AAAAAElFTkSuQmCC\n", 234 | "text/plain": [ 235 | "" 236 | ] 237 | }, 238 | "metadata": {}, 239 | "output_type": "display_data" 240 | } 241 | ], 242 | "source": [ 243 | "# Find the mean and covariance of the x-values\n", 244 | "mu_s = np.repeat(0,n).reshape(n,1) # by assumption\n", 245 | "K_ss = kernel(x_s, x_s)\n", 246 | "\n", 247 | "# From each of the 50 points, take 5 samples from our prior\n", 248 | "f_s_prior = np.random.multivariate_normal(mu_s[:,0], K_ss, size=(3)).T\n", 249 | "\n", 250 | "# find the standard deviation to demonstrate the prediction uncertainty\n", 251 | "s2_prior = np.diag(K_ss)\n", 252 | "stdv_prior = (s2_prior**0.5).reshape(n,1)\n", 253 | "\n", 254 | "plt.plot(x_s,f_s_prior)\n", 255 | "plt.gca().fill_between(x_s.flat, (mu_s-2*stdv_prior)[:,0], (mu_s+2*stdv_prior)[:,0], color=\"#dddddd\")\n", 256 | "plt.plot(x_s, mu_s, 'r--', lw=2)\n", 257 | "plt.axis([-5, 5, -3, 3])\n", 258 | "plt.title('Three samples from the GP prior')\n", 259 | "plt.show()" 260 | ] 261 | }, 262 | { 263 | "cell_type": "markdown", 264 | "metadata": {}, 265 | "source": [ 266 | "The function output $f$ at each $x$ value is normally distributed with mean zero and variance defined by $K_{**}$ (the meaning of the double stars will become clear shortly). The grey region indicates a 2 standard deviation region for the prediction of $f$. The uncertainty is contant across all points $x$, reflecting the fact that in the prior, $f(x_i)$ at each point $x_i$ follows the same distribution. \n", 267 | "\n", 268 | "The covariance encodes how correlated different values of $f(x_i)$ are with each other. The kernel captures the notion that for values $x_i$ that are close to each other, $f(x_i)s$ should also be close to each other, resulting in a smooth function. The kernel can be tuned to increase or decrease the smoothness.\n", 269 | "\n", 270 | "Notice that this is much different from the, say, linear regression setting, where each output $f(x_i)$ was independent of each other. In this setting, the $f(x_i)'s$ are correlated as given in the covariance matrix." 271 | ] 272 | }, 273 | { 274 | "cell_type": "markdown", 275 | "metadata": {}, 276 | "source": [ 277 | "Now, 6 data points are observed and the posterior predictive distribution is updated. The data points are noise-less observations of the true function, a sine function. The prior distribution" 278 | ] 279 | }, 280 | { 281 | "cell_type": "markdown", 282 | "metadata": {}, 283 | "source": [ 284 | "$$p(f_{*}|X_{*}^{\\checkmark})$$" 285 | ] 286 | }, 287 | { 288 | "cell_type": "markdown", 289 | "metadata": {}, 290 | "source": [ 291 | "becomes the posterior predictive distribution after observing data $X$ and $f$:" 292 | ] 293 | }, 294 | { 295 | "cell_type": "markdown", 296 | "metadata": {}, 297 | "source": [ 298 | "$$p(f_{*}|X_{*}^{\\checkmark},X^{\\checkmark},f^{\\checkmark})= ?$$" 299 | ] 300 | }, 301 | { 302 | "cell_type": "markdown", 303 | "metadata": {}, 304 | "source": [ 305 | "By the asusmptions of a GP, the function at the 50 test points, $f_{*}$, and at the 6 observed outputs $f^{\\checkmark}$ form a joint Gaussian distribution:" 306 | ] 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "metadata": {}, 311 | "source": [ 312 | "$$\n", 313 | "\\begin{pmatrix}\n", 314 | "f^{\\checkmark} \\\\\n", 315 | "f_{*}\n", 316 | "\\end{pmatrix}\n", 317 | "\\sim \\mathcal{N}{\\left(\n", 318 | "\\begin{pmatrix}\n", 319 | "\\mu^{\\checkmark} \\\\\n", 320 | "\\mu_{*}^{\\checkmark}\n", 321 | "\\end{pmatrix}\n", 322 | ",\n", 323 | "\\begin{pmatrix}\n", 324 | "K^{\\checkmark} & K_{*}^{\\checkmark}\\\\\n", 325 | "K_{*}^{T\\checkmark} & K_{**}^{\\checkmark}\\\\\n", 326 | "\\end{pmatrix}\n", 327 | "\\right)}\n", 328 | "$$" 329 | ] 330 | }, 331 | { 332 | "cell_type": "markdown", 333 | "metadata": {}, 334 | "source": [ 335 | "where $f$ and all means and covariances are known since they are functions of the data $X^{\\checkmark}$ and $X_{*}^{\\checkmark}$, which are also known.\n", 336 | "\n", 337 | "Now, we use the helpful property of Gaussians by conditioning on the data points that we observed:" 338 | ] 339 | }, 340 | { 341 | "cell_type": "markdown", 342 | "metadata": {}, 343 | "source": [ 344 | "$$p(f_{*}|X_{*}^{\\checkmark},X^{\\checkmark},f^{\\checkmark})= \\mathcal{N}(\\mu_{post},K_{post})$$\n", 345 | "\n", 346 | "where $\\mu_{post}$ and $K_{post}$ are given by:\n", 347 | "$$\\mu_{post} = \\mu_{*} + K_{*}^{T}K^{-1}(f-\\mu)$$\n", 348 | "\n", 349 | "$$K_{post} = K_{**} - K_{*}^{T}K^{-1}K_{*}$$" 350 | ] 351 | }, 352 | { 353 | "cell_type": "markdown", 354 | "metadata": {}, 355 | "source": [ 356 | "All values on the right hand side are known, so we're done! **We now have the predictive posterior distrbution - all that's left is to sample from it.** The fact that the function values form a joint Gaussian allowed us to use the nice property to condition on the observed values. Now that we have everything we need, let's sample from the posterior and plot it." 357 | ] 358 | }, 359 | { 360 | "cell_type": "code", 361 | "execution_count": 8, 362 | "metadata": { 363 | "scrolled": true 364 | }, 365 | "outputs": [ 366 | { 367 | "data": { 368 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEICAYAAABLdt/UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXeYXVW5/z/r9Dq9l9RJTyBA6L2qiIIoiooKYvvZ0OsV\nu6CXYruKXfSqiCggIiLFAoEQIEAakDbpmWR6OzOn971+f+xzJmdmzpk5bWaSYX+eJ0/m7LL22u3d\n7/qud71LSCnR0NDQ0Jg96Ga6AhoaGhoaxUUz7BoaGhqzDM2wa2hoaMwyNMOuoaGhMcvQDLuGhobG\nLEMz7BoaGhqzDM2wzwBCiFuFEPfNdD1mEiGEFEK0FLnMs4UQ+4QQPiHEVcUsO8/6zEucp2Gm63I8\nIYT4lRDiGzNdj+MZzbBPAQnDkvynCCGCKb/fP9P1m8V8G/iZlNIhpfz7dB9cCNEmhLhkCst3CiF+\nmDiOXwhxRAjxVyHE6SnbyMQ6nxCiM7G9fqrqlKaO64QQHymkDCnlJ6SU/1OsOr0R0Qz7FJAwLA4p\npQM4ArwtZdmfcilL8/ZyYi6wM90KoXLcPu9CCDPwDLAKuAIoAZYBDwBvGbP5iYln72LgfcBHp7Gq\nBVHIR0h7V45y3D7oswCTEOJeIYRXCLFTCLEmuSLhkX1JCLEN8AshDEKIBiHEw0KIfiHEISHEZ1O2\n1wkhviyEOCCEGBRC/EUIUZHuoEKIKiHE40KIYSGESwjxfNLgpZThFULsEkK8I2W/64UQLwohfpTY\n96AQ4qzE8nYhRJ8Q4kMp29+TaFI/lSjvOSHE3Ax1MgshfpDwQHsT+1knq++YMg4AC4DHEt6qOeE9\n3i6EeBEIAAsS1/EfibL2CyE+mlLGrUKIh4QQ9yXqvF0IsVgI8ZXE+bULIS7LcA5/BOakHP/mlNXv\nT5zbgBDia/ncN+ADQBNwlZRyh5QyLqX0Syn/KqW8Nd0OUsrdwPPAygx1lkKIzybu5YAQ4vspz4JO\nCPF1IcThxLnfK4QoTayzJK7RYOK+bBJC1AohbgfOBX6WuAY/S2y/NPEcuIQQe4QQ706pwz1CiF8K\nIZ4UQviBCxPLbkvZ5qOJe+VK3LuGMefwKSHEPmBfhmv3xkNKqf2bwn9AG3DJmGW3AiHgckAP3Am8\nPGaf14BmwIr6Ad4CfBMwoRqwg8CbEtvfBLyM+uKbgbuB+zPU507gV4Ax8e9cQCTWXQM0JI73HsAP\n1CfWXQ/EgBsSdb4NtTXy88QxLwO8gCOx/T2J3+cl1v8YeCGlHhJoSfz9I+AfQAXgBB4D7pysvpNd\na2Bdoo4rAENi//XALwALsBroBy4ac1/elNj+XuAQ8LXEvh8FDmV7r4F5ifP8TeI+ngiEgWV53LcH\ngHuyeN5Sr+tyoAe4cYJtn01c9znAXuAjiXUfBvajPmsO4G/AHxPrPp64R7bEs3AKUJJyzT+Scgw7\n0J54bgzAScAAsDzlOXEDZ6M+d5bEstsS6y9KbH9y4hr9FFg/5hyeSpyDdabf92Pl34xXYLb/G/uy\nJ5bdCjyd8ns5EByzz4dTfp8OHBlTxleA3yf+bgUuTllXD0QBQ5r6fBt4NPnyT1L314ArE39fD+xL\nWbcq8VLVpiwbBFYn/r4HeCBlnQOIA82J3xJoAQTqB2RhyrZnkjCgOdZ31LVOGJlvp/xuTtTBmbLs\nThIGM3FfnkpZ9zbAB+gTv52Jepdlefx5ie2bUpZtBK7N4749DXwn5fdqYBjwAHtSlsvEsiHgAOoH\nWJehvhJ4c8rvTwJrE3+vBT6Zsm5Jsm6oRn8DcEKaMtcx2rC/B3h+zDZ3A7ekPCf3jll/D0cN+2+B\n7415jqLAvJRzuGgq3t3j+Z8mxcwcPSl/BwCLGK0Rtqf8PRdoSDR7h4UQw8BXgdqU9Y+krGtFNWC1\njOf7qJ7YfxJN8C8nVwghPiiEeC2lnJVAVcq+vSl/BwGklGOXOdKdg5TSB7hQWwSpVKN6fltSjvuv\nxPIJ65slqdexAXBJKb0pyw4DjSm/x57PgJQynvIbRp9jNoy918n9c7lvg6iGHwAp5WtSyjLgalRP\nNpWTpZTlUsqFUsqvSymVCeqWen0Oc/T+NCR+p64zJOr2R+DfwANCiC4hxPeEEMYM5c8FTh/z7L4f\nqMtQh7GMqkfiORpk9D2baP83JFpnw7FLatrNdlQPdlGGbdtRPfwXJy1UNWpfAL4ghFgJPCOE2IRq\nPH+D2uH2kpQyLoR4DdWjzpfm5B9CCAdqc7lrzDYDqAZzhZSyM9v6SinXZlmH1OvYBVQIIZwpxn0O\nMO64eZJrqtSs7xuqB/0tIYRdSunPvWoZaeZoh/Mcjt6fLlSjTMq6GNArpYwB30rUZx7wJLAH1bse\new3ageeklJdOUIeJrtuoeggh7EAlo++ZlqJ2DJrHfnywEfAKtUPVKoTQCyFWCiFOTaz/FXC7SHRO\nCiGqhRBXpitICHGFEKJFCCFQtc04oKBqoRJVc0YIcQMZOt1y4HIhxDlCCBPwP6j9CKO8q4Q3+Rvg\nR0KImsSxG4UQb5qkvjmTOPYG4M5EB+AJwI1AscYU9KJq0tmS9X1D1fu7UT38lYlnwAKsybB9tnxR\nCFEuhGhG1fwfTCy/H/i8EGJ+4qN8B/CglDImhLhQCLFKqBEsHlRpJHlPxl6Dx4HFQogPCCGMiX+n\nCiGWZVm/+4EbhBCrhRoZdAfwipSyrZCTnu1ohv04ICEFXIGqqx5C9XL/DyhNbPJj1M7H/wghvKgd\ncqenKQpgEape6wNeAn4hpXxWSrkL+N/Esl5UDT0bT3Ii/gzcgirBnAJcl2G7L6G2GF4WQngS9Vsy\nUX0LqNN7UbXvLuARVK336QLKS+VO4OsJyeG/s9g+6/smpQwBFwK7gCdIaOvAqcC70+2TJY+idsy/\nlij3t4nlv0OVXNajPnMh4DOJdXXAXxN1aAWeS2ybPKd3CSGGhBA/SbSMLgOuRb3mPcB3GS8fpSVx\nb74BPIz6YVuYKEtjApLREBoaRUUIcQ/QIaX8+kzXRSM9QggJLJJS7p/pumgUF81j19DQ0JhlFGzY\nE1rlRiHE60IdaPOtYlRMQ0NDQyM/CpZiEp1adimlLxHy9AJwk5Ty5WJUUENDQ0MjNwoOd5Tql8GX\n+JkcHagJ9xoaGhozRFHi2BNhT1tQRxL+XEr5SpptPgZ8DMBut5+ydOnSYhxaQ0MjDQPBAXoDvQgE\nLWUtmPSmma6SRhHYsmXLgJSyerLtihoVI4QoQw0h+4yUckem7dasWSM3b95ctONqaGiM5ovPfZGN\nPRvxR/1cPv9yvn32t2e6ShpFQAixRUo56diFokbFSCmHUZMKvbmY5WpoaORGq6uVk2tO5upFV/PY\ngcfo9nXPdJU0ppFiRMVUJzx1hJpq9VJgd6Hlamho5Icv4uOw5zDLKpdxw4obALhn5z0zWymNaaUY\nHns98KxQc4dvQs2O93gRytXQ0MiD3S7Vr1peuZx6Rz1XLLyCh/c9zEBwYIZrpjFdFGzYpZTbpJQn\nSSlPkFKulFJqYp6Gxgyya3AXAEsr1ACFD6/8MJF4hPt2vaGn2X1DoY081dCYZbS6Wqmx1VBlVTMu\nzy+dz2XzLuOBPQ/gDrtnuHYa04Fm2DU0Zhmtg60sr1g+atlHV30Uf9TP/bvvn6FaaUwnmmHX0JhF\nBKIBDnkOsaxydFbcJRVLOK/pPO5rvY9ANDBDtdOYLjTDrqExi9g7tBdFKiyvXD5u3UdXfRR32M1D\nex+agZppTCeaYdfQmEUkO06XVYyfx2J1zWpOrTuVP+z8A+F4eLqrpjGNaIZdQ2MW0epqpcJSQY2t\nJu36j676KP3Bfh7d/+g010xjOtEMu4bGLKJ1sJVllctQk66O54z6M1hVtYrf7fgdMSU2zbXTmC40\nw66hMUsIx8McGD4wLiImFSEE1y27jk5fJ62DrdNYO43pRDPsGhqzhP1D+4nJ2LiImLEsLl8MwBHv\nkemolsYMoBl2DY1Zws7BnQBpI2JSaXI2AdDubZ/yOmnMDJph19CYJbS6WikxldBgb5hwO4vBQo21\nRjPssxjNsGtozBIm6zhNpcnZRIe3YxpqpTETaIZdQ2MWEFWi7B3aO2HHaSrNzmbNY5/FaIZdQ2MW\ncHD4IFElOmnHaZJmZzP9wX6CseAU10xjJtAMu4bGLCA54nSyjtMkzc5mAE2OmaVohl1DYxawa3AX\ndqN9xGBPRnI7TY6ZnWiGXUNjFtDqamVpxVJ0IrtXWjPssxvNsGtoHOfElTh7XHvSJv7KRKm5FKfR\nqRn2WYpm2DU0jnPaPG2E4qGs9XVQUwtoIY+zF82wa2gc50yUqncitJDH2Ytm2DU0jnN2De7Corcw\nv3R+Tvs1O5vp8nVpWR5nIZph19A4zml1tbKkYgl6nT6n/ZqdzcRkjB5/zxTVTGOm0Ay7hsZxjCIV\ndrt25yzDgBYZM5vRDLuGxnFMu7cdf9SftuNUURTi8XjGfTXDPnsxzHQFNDQ08mek4zQllYCUEo/H\ng8vlAqCyshKn0zkuOViNrQajzliUyBhFUXC5XOh0OgwGw6h/Op3mP043mmHX0DiOaR1sxagzsrBs\nIVJKAoEAAwMDKIqClBKAwcFBfD4fNTU1GAxHX3m9Tk+jo7EoHnsgEMDr9SKlHPUBSdbBZDJRWVmJ\n1Wot+Fgak3PcGHYpJfF4nFgsRiwWG/k7Go0Si8UoLy/HbrfPdDU1NKaVXa5dLC5fjBJV6OzvJBqN\njhjTJFJKQqEQ7e3t47z3YoU8ut3ukeOOPT5AJBKhp6cHk8lEVVUVZrO54GNqZKZgwy6EaAbuBWoB\nCfxaSvnjQstNEo1G8Xq9eDyeUQ/M2IdnYGAAm82WVS5qDY3ZgKIotA62ck7NOXR1daU1qKlIKRkc\nHMTv91NdXY3BYKDZ2cyW3i3jPO1ciEajRCKRSbeTUhIOh+nq6sJqtVJZWYnRaMzrmBoTUwyPPQZ8\nQUq5VQjhBLYIIZ6SUu7Kt0BFUfD7/bjd7rQeSKZ9AoHAlHvtgUAAj8dDbW2t9hHRmBEikQher5dO\nVyeeiIcGU0NW7wioxjUYDNLe3k59fT3NzmYCsQCukItKa2Ve9UlKMNmSlIyCwSAOh4OysjLNwBeZ\ngg27lLIb6E787RVCtAKNQFaGXUo5Iq9Eo1GCwSCBQGBkXQ71wOVyTanXHo1G6e3tBcDv9+NwOKbk\nOBoaqUgpiUaj+Hw+fD4f8XgcKSWd/k4Aai21eZXp8XhGRcbkY9iT5eSDlBKv14vP50MIgclkwmq1\nYrFYMJvNWqdrARRVYxdCzANOAl6ZaLtYLEZHRwexWAxFURBCIIRASpmTMU9XbjgcxmKx5F1GJhRF\nobu7e1SHlM1m0x4+jZxI9gkln/VkJ2fy/9Q+pHg8PrI8+X6k0htSnYw6S11edQkGgzSXHjXsq2tW\n51VGIe8sMHItQqEQoVBo5Fz1ev1IVI1Op0MIgU6nQ6/Xj9iMscz2VrTD4chKkiiaYRdCOICHgc9J\nKcd9woUQHwM+BtDQ0DBKkyvUoKeW43K5aGiYeDLffMrt6ekhFjs69FpRFNxuN+Xl5UU9lsbsIx6P\n4/P58Hg8xGKxtEY6G318LD0hdcRoPh57sl51tjoEIu+Qx7F9X8UgWV7y45YLs9mwSympr6+fk822\nRTHsQggjqlH/k5Tybxkq9Wvg1wCrVq0q7pOQQjgcJhKJYDKZilamy+UiHA6PWialZHh4mJKSEvT6\n3IZya8x+kn0+Ho9nlBcKuUmME9ET6qHcVI5Zn1+EiRACGZXU2GryioyJx+MEg8fW1HrF/sgcg2T1\n5SpYRxDqJ/K3QKuU8oeFllcoUkqGhoaKVl66iJzUYyUHgWhoJPF4PBw+fJj+/n5CoRAwNQanN9RL\nvaU+7/2THan5hjz6fL68j60xtRRDID4b+ABwkRDitcS/y4tQbt4EAoFRskm+hMNhBgYGJnwpfT5f\nVqFeGm8M4vE4g4ODRZMXJ6I72J23DJOkEMOeGruucWxRjKiYF8iyeTBdSClxu91UVuYXvgVqR2xq\nZ+lExxoYGCi6rq9xfDI8PDwtxi6iRHBFXHl3nCaJRqM0OZoYDA0SiAawGW1Z7RcOh3PWvzWmj1kb\n0uHxeFAUJa99k52l2e4fDoePOa1RY/qJxWJ5h/7lSl+oD4mkzlqYYRdCjHj9uXjtU9FpqlE8Zq1h\nB/J+yaLRKNFoNOvtpZT09/drD/obnKGhoWl7BgqNiEkipaTaWA2QdWSMoiiavn6MM2sNezJqJZ8X\nbWwETDbE43G8Xm/O+2nMDmKx2LQau0Jj2FMpF2rIbrYee3IAocaxy6w17KAa93xetlAolPMHIRkh\nk6/8o3F843K5prXF1hPqwawzU2YsK7gsszRTYirJ2rBrnabHPrPesOfTPM7HY08eb3h4OK99NY5f\notEofr9/Wo/ZE+yhzlJXlAE5Qgga7dml743FYloU2HHArDbsoEokyVjibJBS5v3gSimn/QXXmHmS\n4Y3TSW+ot2B9PYmUknprfVaGXes0PT6YEcPe75++WdGTgzCyJRqNFuQFZZuNUmN2EIlEpj0iSkpJ\nT6in4IiYVGpMNXT7u4kqmYMGCkn4pTG9zIhh90emN/41lxev0GamECJvKUfj+GMmvPXh6DBhJVyU\njtMk1aZq4jJOj68n4zZ+v19zWo4TZsSwR+KS4eD0ee2RSCTrBzKfjtOxaIb9jUEyG+F0U6xQx1SS\nqQkyyTHJ4ADNsB8fzJjGvrN3epuv2Xrihb6ouUo/GsUjdZ7P6WAmvHVQO06hOKGOSZKyTibDHgwG\ntZGmxxEzYtjNRNneM72xsNl40ckJDabjWIWSnKRA4yidnZ20t7cTCASm3OCGQqEZiw7pDfUiENRY\naopWZrmxHKPOmNGwa9768cWMGPZGnYsdPdPn1San4pqMYiQOg/zySOeK1+sdlT3wjU7qxOa9vb10\ndnZO6bWZSb25J9RDhakCk654qal1QkedpS6tYQ+FQkVxeDQmxzTYiiHP3PipzIhht8kAnQMuwrHp\nG8yTzUteLE97qjtQU1MTFzNF8fFMMBgciWZKhqx2d3fT3d09JZ71TMptPaEe6q35p+vNRJ2ljiOe\nI+OWT2eqhDcyen8PDf++kcotPyq4rBkx7ALJWWxjd//0eZuKokzqkRej4xQYmeZrqvB6vSMjXDVv\nSsXn86WdlSgYDNLZ2Ulvb2/RjFOxJLt86Qn1FLXjNEmdtY5OX+eo6xSJRN7QrUIp5bRF8VVt+j66\nWBCDJ/cUymOZEcMuhZ5L9VvYMY06uxBi0ge0mA/wVOXTGDuattgTixyPTPYhTUpxxWpFFUuyy4dw\nPMxQZKioHadJ6sx1BONBBkODI8umKw3xsYKUkiPDYR5vHeKOZzp575/387Z79rK5Y2rzANnan8N+\n5Bni5jKMvk4o8JoXdTLrbJEmJ5fqX+Wv3R44qWp6jpl4+R0OR8b1xfTCkgOVij0HY6q3nsTv9xOP\nx9+wU/Rl80FO3v9iTHQ+k+GsxUz+NZbUyJgqaxWxWOwNM5J63UEPz+73sL0nwHBI9dArrHpW1dsY\nOORlW3eANU3pbUehiGiQyo3fIVK2EO+Ct1G59S50YTeKJf88QDNi2OMmJ06GsPRtRZHz0U3TBLQT\n6aJT4YVFo9Gizr06Ue4bt9tNRUVF0Y51PJFtR2YwGKSsrPCkWcWS7PJhKmLYkyQ/Fm3DbZxUc9Ib\nxltvGwrz7ac7qbYbOK3ZwQn1Nk6ot9FYYkQIwY0PHWT/4NR9zMu33Y3R30Pnm36HPqzmmjL4Ookc\nb4ZdMdqJ6gKcG9tM29AVLKgo3IvKhmg0iqIo6HTjFaip8MJCoVBRDbvP50ubPTI5Y1RZWVnac5vt\nZOtVFusez6TmPJUee62lFoGgbbjtDZWG+u87XRj1gruvnk+ZdbxJXFhp5rXuqZFWjUP7KN11H56W\nqwjXnoTJtUdd7usiUrUi73JnxgoIHe6a07lMv5kdU3TB0h5WiIwREuFweMQ7ebH/RV4ZfKWgYxW7\nAzWbkX9vlBcxleTHOhuklAW3zGa647Q71I1Vb6XEWFL0so06I1XmKg67D79hcsL4wnH+s9fNxQtL\n0hp1gJYqCwP+WPFHy0uF6pdvQzE5cZ18EwBRRyOgeuyFMGPunTL/QhrFIJ727dN2zIlGhSaNcCAW\n4Cd7f8Idu+7gqZ6nCjpeMQ17Jm89SSETixSKy+WasVGJuXZSF+q1z3QEUm+ot2jpetNRZ6mjw9fx\nhpFh/rlnmFBM8o6VmWXMlkpVUThQZDnGue/vWPq3Mbjm8yN6ujQ5iJtKMfi6Cip7xgx7oPk8FHQ0\n9L8wrcdNZ9hTU/VuGNhASAkx3z6fn+37WUHGPRaLFWXijWzzdCiKMu2z20QiEYaHh+nv75/W4yZJ\nF+aYiWKke5jpXOQ9wakJdUxSZ60bSVkw24krkr/vHGJVnZVFVZnl4IWVZgD2DRbPUdMFXVRs/THB\n2lPwLXjbqHUxR4MaGVNI+QXtXQCKpYJOx0rOjm+k3zd9XlCq5JIktXn+dO/TNFmb+N7q73Fy+ckF\nGfdiDVSazFtPMhOhj0l9OxgMTvtHRVGUnK9voa2omew4VaQy4rFPFY3WRtxRN8Ph2T9hzCvtPrq9\n0Qm9dYBSi4Eau4EDRTTslVt+iC4WYOCMr8GY1lfU0XD8euwAvqbzWaY7Qtvhg9N63LHN6aQX1hHo\noNXTysV1F2PSmfjK8q8UZNyLobPnmlUvGo1Oa+decupBKSV9fX3TOjVgKBTKWZLIJdNnpmPOFEOR\nIaIyOqWGfZFjEQD7fPum7BjHCo/sGKLabuCcec5Jt11YaWF/kQy7pWcTzoNPMLzieqKl88etjzka\nVcMu83+XZtSwW5dcAoCp7ZlpPe5YLy/pha3tXYsOHRfWXKjWqwjGvdCmf7beepLp9NqT+VlSjz04\nODjBHsUln3wtE3WgT0Yhs2sVg5FQR+vUSTEtzhZ06Njj2TNlxzgWaBsKs6XTz5XLyzHoJncOWqos\ntA9HipIGpbT1fmLWaoZX3Zh2fczRiE6JoA/m/y7NqGFXSufQpp/LgqEXp+2Y6RKChUIh4jLOs73P\nsqZiDeWm8pF1hRr3dNJPLuSTp2O60gz4/f5RHnNy8vDp8GoLmYYw3/oVOrtWoSQNezJ3+lRg0VuY\na5/LXu/eKTvGscAjO9QQx7cuyy5WfGGlGUXCQVfh0qrRc5hw1QqkIb2uH3Wo97eQyJgZD3puKz+b\nlfFdhLzT5+mlvthJL2yraytD0SEuqbtk3PZjjfuL/bl9iPINscsmv006pmtSba/XmzY/S19f35Tr\n0PlOQVhIB+pMT6DSE+xBh44q89SO1l7sXMw+7z6UAqSAYxlfOM5T+9QQx1JLdkN5Fo1ExhTotEgF\ng6+TqLM54yaxRMijsQCdfcYNe3TeReiFxLt7+uSY1LS6yf+f7n2aUmMpp5SfknafpHGfb5/PfYfv\nIy6zD+/L1yBEIpG8PcSp7siMx+MZZYl4PD7lclAhOdfzvR8z2XEKaqhjlbkKo844pcdZ7FyMP+6n\nK1hYB96xSjYhjmOpcxqxG3UF6+z64AC6eJiosynjNjFHA3Cce+z1C0+kS1bg7Fg3bcdMjVYJh8MM\nR4bZ5NrEhTUXYtBl/oKbdCauab6GrmAXLw+8nNWxCulALUTPjcfjU9qRGQgEMn50kiNhp1KPTnba\n5kO++fJnOsthT6inKB2n/kicR3e66MsQjbbYuRiAPd7ROrutYz0lrX8uOEHVTJJtiONYhBAsrLIU\nnFrAmMh3H5vAY5cGCzFrVUGRMUUx7EKI3wkh+oQQO3Ld12Y2sNF4Ggu9mxGx6clxndocD4fDrOtb\nR1zGubj24kn3PaPqDBqsDTzc8XDW3lshTf98PUQhxJTq7OlkmFSmUpJRFKWgj0Y+YagzPeIUEul6\ni9Bx+vMNvfz4xV7ed/9+vv10Bzt7R7d+mmxN2PQ29nqO6uz64AA1z3+Vqk3fp2TPXwquw0zx8hE1\nxPHqHLz1JC2VZg4Ohogr+T/TxkRK3ok8doCYvf6YkGLuAd6c787d1edhJoKps7Bh/LmQNLaBQICn\ne55miXMJc+xzJt1PL/S8o+kdHPAd4PXh17M6Vr56cKGa7lR5zIqiZOW9RqPRKRmaPlFrIRvyaUXN\ndMdpIBbAHXUX7LFv7fTzr71u3r68jGtWVbC5w89nHj3MJ//extr9bqJxiU7oWORcNMpjr9j6E0Q8\nTLD2FCo3fR9rV3Yt1mONR3a6sg5xHEtLpYVQTNLpyf+9Mvg6kMJAzD5xB7ga8jjDUoyUcj3gynd/\n87zT8Ugbyv6ni1GdrEjGM+9y7eJI4AiX1I7vNM3EhTUXUm4q5+GOh7PafiY8RCnllHX2ZWtYp2pm\n+2JMSzfdqQgKpS/cBxSW/CscU/jh8900lhj5f2fU8vEzannw/Yu46exaAhGF25/p4uK7DrH6GsFT\n957GQc8RWpbV8/7zB3AeeIyfbvokPRf9mGjpfGrW34zBc7hYpzcttA2F2doZ4Mrl5eizCHEcSzFS\nCxi97cQc9TCB5AsQdTZi8PeAkl/gxbRp7EKIjwkhNgshNrtco78BK+qdPKOsprL3BVCmJ+eIEIJA\nIMBTPU9h0pk4p/qcrPc16oxc2Xgl24a3sc87+UCOfIxsMXKvTJUxymUYPxQ3JXK289dORq6tqCnv\nOJVywgEpyWH+haQT+MOWAbo8Uf7rvHrMBvXVtxp1XLmigt+/ewG9D51KbMhOxaW7iA4tROjj2Oft\n4Kdv+SKdnnq++u8vIY12ei68C4Seumc+hy5y/CSeyzXEcSxzy80YdBTUgWrwdhB1TCzDgNqBKmQc\nQ6Avr+NMm2GXUv5aSrlGSrlmbN7waoeRTabTscXcWPqzkzeKQd9QH+v71nN21dnYDLac9n1T3Zuw\nG+w83J6d156rzl4MGWUqNOF8wgWLKQkVs6xcrs9Ud5xWvXIHjU9el7GfKRnDnq/Hvn8gxF+2DXL5\nklJOarCPW68TgtDBGvoePpV4yICx2grAeefdw5qG1/jvp27DH1Unmog5G+m94AcYvR3UrP9y3l7l\ndBJTJM97nfoXAAAgAElEQVQe8HDBguxDHMdi1AvmlpsLCnk0etsn1dcBYvZkZEx+OvuMR8Ukcdee\nSRQ91vbnilamiAUx972Wdp2Ukmc7niUQD+QkwySxGWxcXn85Lw++TEdg8lnFczUMhQ59B1ULL3Zk\nzFij7ol6RnKEp6PYozWLIcMkyfaeTHXHqXFoH869D2MebKVi60/TbtMT6sFusOM05q4NxxXJD9Z3\nU2rR8/EzJvb4ZcSAb+s8LPOCxFx1zFv1DM+1nc0DO945artQ7SkMnPEVbF0bqNxyV851mm62dQfw\nRRTOm5/79UulpdLCvoH8DLsu7EYf8RIryRwRkyTqLCx97zFj2FsaqtiqLMLQtbFoZTo33kXjv25Q\nQ7TSsLZnLXWWOlaU5pfQ/oqGKzDqjDzS8cik2yqKkpO8UgwZpZDh85kYK8Pctecuvrn9mxPuU0xJ\nqFhl5dLyKGQ8QTZUvP4rpNGOd+HbKN19f9qOyUKSf/1th4u9AyE+c1YdTnPm6RMNRNERx7NlHnZ/\niG/8Zz9DoSif/dd3gfHn7110Ne5l76O09U849/0tr7pNFxsOezHpBSc3jm+t5EJLpYWhYBxXIPdW\nitGrOoBZSTH2OqTQ5Z3lsVjhjvcDLwFLhBAdQoj0SRAmYFWdlZeV5diH9+al2wWjCtu6Azy0bZDb\n1nby0Qd2Ydr3OAFppmrT9zHsGv3g9YZ62ebexsW1F+f90paZyrik9hLW9a1jMDzxyNlsJtNOpVgG\nrJiGfeww/v5wP1uHttIT6mEgPDAtdShmWdnej6nsODUN7sJ+5BmGl1/HwOlfIVIyj+oNt457B3pC\n+aXr7fZE+P3mfs6Y4+D8BRN7q9/jZp7iUmoCLj704jO8719t3PflA/zv4Kd5L3/GwvgP4eApnydQ\nfyZVr9yJpXdrzvWbDqSUvNjm45RGO1ZjYSavJZHCNx+d3eBNhjpO7rGjMxKz1cysFCOlfK+Usl5K\naZRSNkkpf5trGXPLzew0rkKHgrlnS9b7HRkO87GHD3LF7/fwuccO88uX+9jRE+Bd1s2UiAD3NnyD\n55QTaNx0G7d+ai0LF85n4cIFXPW1V5GK4JvvvJ6FCxdw+ulz8IXjPLRtkA8+eICr/rCXTz/axnfX\ndfGnVwdYf9DDQVdoXBKgq5quQpEK/+j8x2TXKGsPsRgz/STLKaYhHGvg1vWuQ6J67xMljYrFYkWR\nT6SUGVs9IhbCNNia0+CZbAdxTWXHacVrvyRuKsW97H1Ig5X+c25DHxygcuN3j9ZTxukL9eXssUsp\n+dELPQgh+Nw5E0/OYXvqKT7PXZzL88zhMG9q3kT/KhtRveCS2Hr+zPvpph5DxxjZUWeg7/zvErPV\nUbH5f3Oq33Rx0BWm1xflrHmFT0a9MBEZk49hPzo4qTGr7WMFpO+dkTlP06ETghWrzyC81Yh7/0uY\n5lww6T5SSn76Yi893igfPKWKJdUWFldZqbAZqP/3d4gqzVx2yeUc6j+bTY98mt+e9iXic/w89Pcb\ncJzwPIF9JxF11WGs8qKc3Ma7/9ROKCZZVWdldb2NDk+EzR1+/r3XPXJMAdx4ajXvO0nN11FrqeXc\n6nP5V8+/eFfzuybUQLM17MmY6WIYk2LP4pSsUzIb5tKSpRz0HaTV08rZ1Wen3S8pCZnN5oKOH4vF\nMl6X8td+SdmuewnWnoLr5JsIV6+atLxkK8pmm7jjfKo6Ts39r2PrfIHBkz6DNKnPTbhqBcOrPkL5\ntrsJNF+Af+4luMIuYjKWs2F/er9HjVM/q5YaR+Y0BIbOTqpvvhmAL/MdFqzsYPXJe7ip6ots+thf\nOe8Pi7l6cwcleDA1jjdKismJe+m1VG3+Acah/UTLW3Kq51Sz4bAXAZw5pzB9HcBh1lPvNLI/D53d\n6O0gZq1GGqxZbR9zNGLtzm9szzFj2AHevLyW7a8tprRrE4qU6CaRSF5p97Ol08+nzqzlnauORtoY\n3W1Ye7cyePJnQQjm15Rx0o+fZN3H38Jv624ndGOAA007GNj4fmqvfRnL3EGUqI6LFpZw1YoKWsYM\nNQ5GFTrcEdqHw6zd7+G3m/pZVG3h1CbVA7i6+Wqe63+Of3b/k3fPeXfG+k40mXYqxfSyixVqmMzc\nmKTV00p3qJtr5lzD2p6144afj6UYhj1jB6YSx3HoSSKlCzC6D9H4zw/im3MxQyd9mmjpvIzlJQcq\nTWTYi9V6Skf5q78gZqnAs/S9o5YPnXAj1s7nqXr5dkI1q+kOdwO5hTr6I3F+8VIvy2usvH15eeYN\no1FqbroJvcfDv41v5W75MfZceiqbu1Zz95OfZcFpG3n4aiPf3byRBRUDPCU8iISDIq1HDZRv/luo\n3HIXzgOP4Vrz+RyuwtTzYpuPZbWqw1cM1NzsuctzhiwjYpJEHY04Av0Qj4DelNOxjpnOU1DDiWTT\n6SyMt7Fh98SRJjFF8quXe2kqNY17cJ37/44UBnwLj0455QmXcelvHudg71y+s+V7PP7VPVRXSQxl\nAYaeXUrnLy7mv89vGGfUQY31XVRl4aKWUr5xSSPzys3c8UwX/X7V0Myzz2NNxRoe63yMcDzzDc9W\nZy9GREySXDttMzG2Tmt712LRWTir6iyWlCzhoO9gxnMv1mCpTNfF2rMJQ3CAodX/j/Z3/APXiZ/A\n1vUSTf94F1Uv34Y+kHnavslaUVPVcWrp2YytZyPulTcgjWM8OJ2R/rP/BxELUrXh2/QmYtjrrdmn\n693eE8QdivPhU6snHIxjPHIEQ3s7sbo6lrz0LQ7/9bs0lnRTf+Pn2bVhmCr9Aix1+9i+ew9PbfJg\nffFFmi+9lLKf/3xUOYq1gkDTOTgOPnFMhT/2+6LsHQhx1tzCZZgkLZVmOt0RgtHcIs6M3o4Jc8SM\nJeZoQCAx+rtzreKxZdgB6pedhU5Idm1dTzSe2bg90TrEkeEIHz+9BqM+5cGNR3Ee+AeB5vOIW0en\nNz0h+DrGX7tZ9B83TQNRrnrARefdF9C4MYwSyu6LaDHouOXSRsIxhdvWdhJL5I14Z9M78cQ8PNH9\nRMZ9s9XZi9n01+l0RWkBpIYZhuIhXhh4gbOrz8aqt7K0ZCkxGeOA70DG/Ytl2NPhOPgEcaODQNO5\nSKOd4RM/zpF3PIZnyTU49z9K89/fnjEyarKP6JR0nEpJ+Wu/IGatxrP4XWk3iZYtwHXyZ7F3Ps9Q\n9/qc0/Xu6g2gE7CsZuJmf3ThQjoff5yeX/8apbwc+5G1hKpWEa4+AYCLmlaCLsoDrbsAUGw2DN3d\nlP32txgOjx556l34dgyhQaxdL2Vdz6lmwxG1lXn23MJlmCQLKy1I4KAr+/dUxIIYgv05eexHszzm\nrrMfc4Y9Ur2KmM7MkvB2ntydPvWrLxLnni0DrK63jfsS2zueQx8awtNy1cgyEQjwR67jGS5mkdzP\noUoHt316Du++fB1vlU+wjRP4Dl/KuuNtTpmZL5xXz/aeIL/bpHqDy0uXs6ZiDQ8deQh3xJ1x32xG\nTRYzZrpYMdip0TAbBjYQioe4uPZi/m9jH396SQ0h2+3dnXH/YtQhnWEX0SD2I8/gn3spUn9U6lGs\nFQye9iXar/wboeqTqNz8w4wvyERSSzAYLHrHqbXrJax9rzK86iMZJ1sA8Cy9lmDdaQz0b6bGVIFe\nZA5VHEtrX4gFFeaMUSD67m7KfvELiMeJV1cTWbECfaAPy+AuAs3nj2x3SfNKAP7Vto24IgmfdBLe\nd7wDEYlQeccdo8oMNJ5D3FyG88DEgQTTyUuHvTSWGJlTlpuUMRHJrJC56OwGrxq2mKsUA/nFsh9z\nhh29iUjNai4w7eberQNpmzt/fnUQTyjOJ86sGddMdu57hJitlmDDWSPLpNXKHI4QxMLXDd/k6tvn\ns0FewrLqvSxx7EMi+BLfo+orX4Es9dSLW0p527IyHnh9kA2H1dC06+dfTyge4oEjD2TcL6mzZ6JY\n0kmSYsy7CqON39peNf5/iWMZT+weZkeXnlJ9Lbs9mQ17Mc4rnQG2daxDFwvgW3B5+n2czfSfdQsI\nQemuP6bdZqLrU3SPPeGtR+31eBZdNfG2QkffWbfSYdDTHMo+TXFckbT2BVlem8Fbj8Wo+dznqPjf\n/6X8Rz8aWWxLDA70N18wsqzeWo9V58QnDrH+kPqcu26+GcVux/7001iff/5ouXojvgWXY29/Dl1o\n5ifDDkTivNoZ4Kx5zqLKadV2AyVmfU46+9GImMkTDSaJW6uROgNG72ww7ECo/lTmK4cRwSEe3j46\nr0y3J8LD211ctriUxVWjH1yDrwtr10t4W64EnZ6SP/4RndsNQnAjv2UFO/nxskuJWSS7N74dALnS\nxJU8SgArJQ89RO2nPoXI0hB+6sxaWirNfHddFz3eCM22Zt5c/2b+1f0v2gPtafeZTGefCk23UCkm\nHo+PeK29oV52uHdwce3FbO9VdVybUYfP08Ruz56M3m2haYQzjaJ1HnySmK2OUO3Jmetvr8U3/3Kc\n+/6OLjS6FTiRPJbvDFYTYet4DsvgToZP+GhWHWJRey0HTGYWe/sxDu3P6hhHhiMEogrLM8gwZb/6\nFdbNm4nV1uK+4YaR5faO54g6m4mWLhhZJoRgeekSzPYO7n9tQA05ralh6FOfAsD50EOjyvYufDtC\nieJo+3dWdZ1KNnb4iSqyqDIMJHKzV5pzCnk0+hKDk3Lw2NHpidnrZ4cUAxCsWwPAB2sP8eDrg7hD\nR1+u32zsQyfgw6dWj9vPuV9tAnpbrsT+xBNU3XorDe98J8RiDFfN5xALsC/dhBIz0Pb6pezoW8bl\ni/7DP7mcd5X+i3hJCfann8b54INZ1dNk0HHLpU0oCnzr6U6iccm1c67Fqrdyz6F70u4zWRKrqUi1\nm2/a4NT9kx+bZ3qfQSC4sOZC1h/0YtYLvnZRA35PM+7o8ITpBQo5t3Rpc/VBVc/1LngLiIkf5eEV\nH0IXD1G6e/y9DQaDuN1uXC4X/f399PT00NnZSUdHR3E/slKh/LVfEnU24114RVa79IR6CBGnJS4p\n23VvVvvs6lOfr3T6ur63l7Jf/QqAvu9/H6WyEgAR9WPt3oi/+XwYc85LS5agGPvYP+Rmc6cqybmv\nv56+73+fvhSPHyBSsYRw+WIcx4Acs6HNS4lZz4pMLZcCaKm0cMgVzjo3u8HTTtzkRDGX5nScqKMB\ng3+WGPZw5XIUg5V3le0nEFW4/zV1VOeOngDrDnp5z4mVVNvHxOUqcZz7HyXYcCZKyETVrbcC4P7w\nh8Fg4JVXjnDgwEHOufZFVlQsYn9rD00Xns7FLS9ycPd2fra1ia4HHsB9/fV4PvCBrOvaWGLii+fX\ns6c/xN2v9FJqKuWaOdew2bWZ14bS56mZqAO10MmvM1GIDJL0tBWp8EzvM5xQdgKV5ipeaPNyarOD\nM+c6WVa6FIDXXLvSllGoJJTuo2Bv+zdCxvEteOvk51C2AH/T+ZTsfgARHX394/E4g4ODDA8P4/V6\nCQQChMPhog2sSmJrX4d5aC9DJ34cspzers3fBkBz7Tk4Dv4TvT/zhzNJa2+QErOeptLxLYLyH/0I\nXTCI/7LLCJ19dNyBreslhBIl0HT+uH3UGZUkFWVd/OX1RAvabMZ39dWg16MbHkaXMseud+HbsQzu\nyrqFMRXEFMkr7T7OnOvIK0XvZCystBCJS44MZ+esqBExOXjrCWKOxrzSChyThh2dkVDNSdQMv8pl\ni0t5ZOcQfb4ov3y5l0qbgfecWDluF2v3SxgCPXgXXUXVrbeid7kInHUW3vcejREOxoPs8+4byQ0T\naDoHocRGcnNElyxh8BvfAJ0OfW8v1nXrsqrueQtKuHplOX/bMcQLbV6uaLiCWkstvzv4u7Rzo06k\ns0/V8PVCZJBk5MgO9w76wn1cXHsxrX1BBgMxzk0kVfrUKSci42aeaMucnbOQc0vX6nAcfJJw+RKi\nZQuzKmN45fXoI26c+yfP7TMV2LpeIm504JuX/Zw0h3yH0KGjYvmNgKS09U+T7rOrL8SyGkva1kbg\nsssIL1nCYGJA0kjd2tcRN5USqlk9bp8WpzrgaH5DL693+0eNvrauW0fzJZdQ8d2jI2V989+CFAac\nBx7L9jSLzo6eAN6wwplFDHNMZVGV2lGfbaZHo68ju1QCY4g5GtCHhhDR3FJVH5uGHVWOMbkP8pEV\nOpBw85NHaO0LceOp1Wl7+kv2PULcUg7bgjj++U8Uu52BO+8c1axs9bSioLCqVB2VGKo+gbjRga3z\nhVFl6bu6aL7sMmpvugl97+QeEsDHT1f19h+u78YfFnxo3oc4HDjM2p6147bNpLNPVRbBQlMLJA3y\n2t612PV2zqg8g+cPeTHo4Mw56ouzqMpGmZhPW2AvvRnm0izEAx77UTC627AM7szYaZq2jJrVBGtO\nUjtRlemf5s40tJ9I+SLQZR/dcjhwmAZrA/qSefjmXUbJ3ocnzKXki8Q5PBTO2HEauOgiOp94gtj8\n+UcXKjFsHS8QaDo37QQQDoODJmsTceMRYgq09h1t8cTmzEHn9eJ86CFM27erxR0DMe0vtvkw6sXI\nIMJi01xmxqgX2ensShSDrzsvw340MiY3OeaYNeyhulMBmON7nbcvL+fIcISWSjOXLR6vUemDA9ja\n1+Nd8DbCq04geOqpDH75y8SaRjd9drp3ohd6lpaosoHRbCPYeJZq2FMMTryhgeAZZ6Dz+ai87bas\n6mvUC758YQO+SJy7XujhzMozWVayjPsO30cgNvprm0lnL2Y0zFgKkUGi0SiBWIANAxs4p/ocTDoT\nzx/ycnKjHUdKtsBzGlYizN3836b0HceQ/zmO/eA5Dj2JFDp883ObkdG98nqM/h4ch6a5c09KTMMJ\nw54Dh3yHmGufC4B7xYfQxQI49/414/a7+4JIxuvr1vXrKf/hDxE+3zgN3dL3GvqIe1SY41gWlyym\nP3YAgWR7z9FnN7pgAe7rr0dISdW3vgWJluhMxrRLKdlw2FuUpF+ZMOgE88vNWc2mZPD3IGSMWBZZ\nHceSjGU35qizH7OGPVyxFMVox9q7ifefVMkZcxx8/tz6tGkGHAceR8gY3pariM2bR/ef/zxKgkmy\nfXg7LY4WLHq1mWq32wk2nYshOIDJNTpUb+CWW1CsVhxPPol1/fqs6rygwsL1p1Sz/pCXdQe9fHjB\nh3FH3Wmn0Euns0/VHKWFlJ1MvPXCwAtElAgX117M/sEw3d7oiAyTZE3lCoSQPNe5K2MTNZ96jBvW\nLyWOg08SrDuNuK0mp7ICjecQKVtI2c4/5JQwrFAM/m50UV9Oht0f89MX7mO+XfWuIxVLCNSfQWnr\nn9Vh5mlo7QsigKWphj0Wo/L22yn/+c9xPjz+WbS1r0PqjAQSIcLpJJzFzsV4Yh7mVPvY1j362R36\n9KeJVVdjefVVnA+oob4zGdPeNqQ+n8UcbZqOlkoz+wYmTxA3kq43T40djsbBZ8sxa9jRGQjWnIyl\nZzNlVgN3vLk5/Sg6KXHue4RIeyMVX/4u+v5+0OnGeSWheIj9vv2sKjuaHMput+OvPwuJGCfHxBsa\nGPrMZwCouuWWrEMg33NiJUurLfzkxV6q9As4v/p8Hu14lL7Q6Cmu0unsxUwlMJZ8ZZBk4q1nep+h\nydrEYudinj/kQSfGj+ZTO9nA4jjCbzaOn9Ir39QCyTokMfe/jtHXmZMMM4LQMbziekzD+7GOuedT\niWlInUIxUpZ9gqzDfnVk5zz7vJFl7hUfwhAcwHkw/QjnXb1B5pabcZiOtqScf/kLpv37ic6Zg+fa\na0fvICX2jvUE609DGm2JRXKccV/iXAJAXVUPu/qCo6JBpNPJ4C23AGDdsEFdOIUx7VElygHfAXpD\nvUSU8R+4F9vUmP+p0teTtFRZ8ITjDPgnlptG0vWWZB/DniRuqUAxWHIepHTsGnYgVLcGk+cw+gnm\n/bO1P4uppw3Dw0ewP/UUtrXjNW2A3Z7dxGWclaUrR5aZTCakrZJw1QpsHc+P28f94Q8TWbSIyOLF\niJSRlxOh16mSTCim8IP13Vw37zqEEPyxbfTgmHQ6+1RPmJyPDBKNRukMdNLqaR3JXf/8IS8n1Nso\ns47WYx1GB822Zhpru9nY7mdr5/hrls85jg11dBx8EkVvwT/nopFlwajC3oHssmf65r+JqL2Osh2/\nz7ku+WIaViNEcjHsyYiYeY55I8uC9acTrlhK6c57x82RKqWktS/E8pqjo1mF10tFIiTRdfPNMCYR\nm9F9EKO3fdSgpHQe+1z7XMw6MyZbO8GoMk5b9r/5zXTdey99Pz06A9RUxbQ/0vEI//Xqf/GxTR/j\nmhev4X0b3scnN3+Sr237Gt9v/T5P96xlabWFSlt2kUf5MrdcvZaTRcYYvR0oevO4FCdZIQQxewPG\n2aKxw9F4dkuG/OwGbwfVL95C/CkjOref4Jln4n3Pe9Juu929HR26EX3dYDAghECv1xNoPAfzwA50\nodGDoTAa6XrwQXrvvnsk3jcb5pSZufHUal4+4mPrYRNXNV7F+v71o6bQS6ezT7Vhz0cGiUQibBhQ\nvbALai/g8FCYw8MRzp2XftDH0pKleDhItV3Hr1/pQxnTSsinDqNaN/Eojrb/EGi+AGk8OhvOr17u\n5ZOPtNE+nMU11BlxL/8A1r5XMU/THLumob1EHQ1IU/ZeZJu/DbvBTpUpxSAIwfCKD2LytGHrGC0R\ndrgjeMJxlqV0nJb96lfoXS5Cp5yC/83j+yPs7esACDSdN7LMYDCMy3ipF3paHC0Myr2AZFv3mD4i\nIdTwSSHQd3VhfvXVKYtpb/W0Umep4zOLPsN1c6/jgtoLmGefhyIVtg/vwGV/iNPmTn3i2qYSNZy0\nwzO5YY85Gicda5GJfPKyH9OGPVK+hLjJibVn07h1Ihakdt0XEK8H0b8+hGK30/+d74yTYJLscO9g\nkXMRVr360CdTyJpMJgJN5yKQ2NJ09Cilamet4dAhyn/wg6x12XeuqmBVnZWfb+jl5JILAdg6NHqG\nmVSdfSrTwybLz9ewH/Idos5SR4WpgucTw8rPyTB35FLnUnwxH1efFGfvQIh1Bzyj1ucjCaXW29b1\nIvqIG2+KDDMcjPHvvW4UCQ++PvFMVkm8Le8gbiqlbMc9OdUlX0xD+3Py1kE17PPt88d50P65lxK1\n11O24w+jliejVVJHnAYuvJDQ6tUMfvWrad8NW/tzhCpXjOqrMBqNOJ3jh+GfW3Mu7cE2qqv3saMn\nfevItHMnzW96EzWf/SzC7y96TLuUkn3efawsXckldZdwzZxr+NjCj3Hzspu588Q7Od/2aYQuiqV0\nZ1GONxGVdgNmvaDTPZlhzy1d71iijsbZJcWg0xOqORlr7+bRy6Wk6pU7MQ/sQXlB/TKni4JJEoqH\nRh4GUJuaFovaXDWZTEQqlhKzVKaVYwCIRGi47jrKf/lL7I9lF5urE4KbL2ggLiW/fzlGg6WBbcPb\nRm2T6ommG1lZbPJpEUQiEY4EjjDHpuqD6w95WF5rHT9ALEGyRVRe3kVTqYkn94zWV/NJLZBab8fB\nJ4lbygk2nDGy7LHWISJxyZomO//Z56Y/Q7hlKtJoxbP0Pdjb12EcPphTfXImHsHoOUykfHHWuyhS\n4bD/8EhEzCh0BtzLP4Cl/7VRk7Xv6gtiN+pGJAKA8Jo1dP31r4RXj49P1wf6sQxsJ5Aiw4Dq9Fit\n4/uzLq29lEZrI6LySbb1eNN+oCNLlhBdsABjVxfld92lxrTrjFS/9O1x6RzyoS/chzfmHYmtH8vB\nnlpEtIrt/gzvchHRCUFjqYmOiQy7lBjyjGFPEnM0oI/60IU9k2+crFveR5sCXhl8hf7w6NzZwbpT\nMXrb0ft7RpY59z2M88BjDJ30CTr/8jdcX/hC2iiYJJn0dVC9E6HTE2w8B2vXhvRxtyYTQzfdBEDl\nHXcgvNnNydpYYuLjp9ewucNPiVzCdvd2Yinlp+rsUxkRkyQfwx6MBOkMdjLHPocuT4T9g+EJZ3pv\nsDbgMDjY69vNCfU29g+MH0mb67kmWzIi4sXW/hy+eW8aGbkZiSn8fecQpzfb+a9z61AkPDQmv1Am\n3EuvRdFb1CiTKcTkPoSQ8Zw89t5QLyElNBIRMxZvy1Vqi2PnUa99V2+QpTVWdEJg2biRyltuQTc4\nmLEVm3Rk/ClhjkII9Z1IRI2lYtAZ+ND8DxEUPQQsr9CezqAZDPTffjtSp6P0nnsw7Oug99w7Mbn2\n0PDP6zF4J55nYTL2e1XPf5FjfHSRIiU7eoLM0Z+pDqYLZe6bKxaNJaYJPXZ9cABdLJRTHvax5JPl\n8Zgx7K8Ovcodu+7gz22jX7JQQme39qheu7l/O1UvfIfIniaGVn2EeEMDw5/8ZMaHF1QZJlVfl1KO\nMuygjkLVR7xY+relLcP7rncROvlkDP39VKaMspuMty0v55RGOzvaGtWWg2/fyLpUnT1dREzZtl9T\n/fzXsj7WZOQqgyiKQoe/g7iMM9c2lxfa1A9aJn0dQCd0LHEuYY9nD4sqzXjCcfpSogZyjYxJTf5l\n69yATomMil1/er+HoWCca06opM5p4qKFJTzeOjQqv1DGsi3lBJrOU7XqKQx9HImIySHU8ZD/EDA6\nIiYVtcXxbmztz2F0HyIYVTjoCrOsxoJx715qP/EJSu+7j9I/ps9oCWqYY9TRSHTMByf5TpSUlIxr\nRZ5WcRoLbcswVT/Flq70sldk5UrcN9yAUBSqv/Y1Ao3n033Z3ejDbhr++SHMA/nLJPt8+zAIQ9qW\nzCFXGG9Y4byaCwB4ru+5vI+TLY2lJrq9kYw5YwwFhDomieUxSOmYMOy+mI+f7lV707cObUVJ6e2P\nlC8ibirF2rMZXchF7bP/jXwshumBXVTefkemIkexw72DFmcLNoPaIaTT6UampzMajaqBrT8dKQyZ\nQ+B0Ovpvuw1pNFJy//3Ynnoqq2PrhOBTZ9US9CwABK8Pje6sS+rsY42daWAn5a/fjfPQk0XTJ4UQ\nOdFilq0AACAASURBVOn40WiUI4EjAMyxz2H9IS8tlWbqSybOSri0ZClHAkdoLFejcPYN5B/9kypR\nmQd3oOjNhCuXA+pH4qHtg7RUmjmpQb23711dRSgm+fvO7Jr9wYbTMQT7MbqnTo4xDe1D0ZlyCndr\n87ehQzcigaXDveRapN5MxdafsrcvgCJhjXBTf8MN6N1u/JdcwtCnP512XxENYu1+RR2UlGK8pZQj\nht1sNo8z7EIIPr7oBnQGH0/3Z5Ylhz73OaKNjUTnzUMEg4RrVtP1lnuQBgv1//nIuI7fbNnn3cd8\nx3yMaXLtvJ7o0D2naS4rSlbwbN+zUxY+nKSp1ERMIeNo62S63sIMe2KQ0vHmsf/u4O8Yigzx1oa3\nMhwd5qAv5SUTOkJ1p2Dp2Ujt+q+g/3cXum1eFIcDzwTyS5JwPDxKX4ejMgyAXq9XJ0g2OQnVnoSt\nI3Nsc3TJEga//nV8V1xB8KyzMm43lnnlZuaWlmGKNWXU2UfJE0qM6pdvI24uR+oMlOz7W9bHmojk\npNLZEolEOOI/gl7oMSs17OoNct78kkn3S7aMYsYjCMZPSJBLHVL1ePPATlWnTrzUG9v9HB6K8K5V\nlSMGaH6FmbPmOnhkx1BWU5cF61WtPpkvaCowDe8nWrYg7XD9TLT526i31mPWZ54nVrFWMHTix7G3\nP4uy7z+UBT286SufwtDTQ3DNGvp+/GMwpD+mtfsldEpkVJgjjHZ6hBA4HOOjeJaULKEsdgodPM1g\nOL3XLm02Oh99lL6f/hTpVFt40dJ5dL3lD0RL51P77OcnHEGb9nylwgHfAVoc6SWtbd0Bah1G6pxG\nLqy9kM5g56gWcsnuBynZ/eC4MNFCaEwkWsskxxi97UihJ2ZvyPsYirmEuNFxfHnsm12bWdu7lnc2\nv5N3N6sTQY+LHqldg9HfjfWpFxHrAkidjt6f/IToosmbtrs9u4nJ2CjDnuw4TWJIPPyBxnMwD+9D\n78s8x6Dn/e+n7667kHY7IhjMOr79vPkl+NwL2O3dQyh+1NAJIQgEAqNizEt334/ZtZvB07+Ev/ki\nHAefQEwwl2q2jPuATEI0Gh3JVfLKEbXOY0ebpmORcxE6dBwM7KG5zDTOY8+UWz0dIxKVEsfs2k24\nasXIur9ud1FpM3DhwtEfm/etrsITjvN46+Ree8zRQMQ5B1v3FBr2ob15RcRkkmFScS+/jnDFMi7p\n+AnnR9sx9fcRWbyY3l//GmnJPDuTvf054qaScUm/DGM+BOmiYwDOL78GKeL87kDmpGRKuToXsXH/\nfqq/+EUM7e3ErVV0XfZ/BBvOovrl2ynf+tOsZbDOYCfBeJBFzvHvvZRqqoMT6tWW21lVZ2HSmVjX\nuw5QUzxXbvoBVRu/Q+2zn0cXzjzLWS40lapORqYOVKO3g5i9DvSFxdTnmuVxRg27L+rjZ/t+xlzb\nXN4z5z2UmcpocbSw2TU6CiZYfzoEFOQTqvc2+LWvETw/c16LVJL6+rKSZYBqSFM9dkjR2RvPAdSQ\nuowIAUJgaGuj4Z3vpPorX8nqwTxvvpOYfyFxGWOX+2hqWyklHo9n5OUx+Loof+0X+BvPxT/nEryL\n3oE+4sF2+JmszncycpFBwuEwh/2HmWNTZZi5ZaZREReZsOqtzLXPVXX2Ksu4wSy5tByS2xndh9DF\ngoQrVcN+YDDElk4/V68sHz3nLbC81srqehsPbXcRiWfhtTecgaV3C8SLnxhMFxrGEBzISV8PxAL0\nhnqzMuzoDPSd8U1sio93z3uFrgcfpPv3vx8J002LEsfWsV593sdIGsl3IYnJZBrx4FM5u3EeUdeZ\nvDj4zMgI2bSEw9R++tM4//Y3Gq+8Euu6dUijjZ4Lf4Rn0dWU7/idOtgqC/Z5Ve87XcdpuzvCUDDO\niQnDbjfYOa3iNNb3ryeqRHEc+AdCxhhecT22rg00Pn4t5gz9ablQYTVgNerozBDLrkbE5C/DJIk5\nG48fj/03B3+DJ+rhpiU3jWhmp1Scwj7vPjzRo6E9UedcOt79F3p+8UuGb7wRz4c+lPUxdrh3sNCx\ncERfB8YZ9uTvaOl8oo7GzGGPKQhF+f/snXd8W4W5v5+jvb33tpM4e++EhIRAQ0kgjNCWUgoUaBmX\n9rIpXCiFshpa2kIpZaUtBQpcRlgBAoQkZIcsnMSxndiO421LtqwtnfP7Q5Y8NCzZYt7f8/n4H50p\n+Zz3vOcd3xdlQwOGd97B9I9/DLl+aaqaTMVoBEnBfsvAOHuwA1WSSNvxIAAdc24DQcCRMxuPIQ9T\ngqRm4/HYrU4rLc4WslQFHGiyx+StBxhrGkultZLSVBVtNm9IMjNew67u8CfcAh77Kwc60SgEVoxL\nCbvdj6al0W7zsqFq6BIxR85cZF4Hmi+hWSnYcRqHYQ8YykgVMQOQJNS/e5pdm4pZYPsQhbETX3Z2\n1E30dR8gd1lCyhyhr78jQKRwzKg0DbKuZcjR8o/jUa5/tZrG//wH29KlyLu6yL7iClL+9CdARvvc\nO7EVLiV13+Mx5ZGqrdVoZBrydHkhywINUwGPHWBJ1hKsXiufd+7GVPU6jqwZdM74JY3L14IgI3f9\nz0g69PyIEueCIJBnUkb22LtPDEuHfTCeOCcpfW2GfUfHDja2bmR1wWrKDH162jNSZiAiss+8D9Xh\nw2RdcQWpDz2EO7Ucx6lL6YzQaBEOl8/FUetRJib3hWH6J4cCqFQqv8csCNjzFqJt3jlk6MNTWkpb\nb3VM2gMPoN4Tvjs2gCAILC5Ox2svZK851IBIkoS+foN/Kv3Ua4IJEwQZ1lGr0DbvQtFdH8vXjkqs\nlTGSJHG8+zgSEraeDEQJTokhvh5grGksDp+DZFM7MDCBGmtlTP+mLXV7BaLSgMdURLvNw8c1XZxZ\nnoxRHV4Cd2aentHpGl7a3zHklBtH9kwkQY62aUesXy9mVOajQHxSAkNVxATxekl98EEy173O1C2H\nsHlySN92b1TtbmPVa2RuuRNn2njs+QsHLAuUOoZsEyYcI5cJjE9PQ21dxh7znpCigP6ISUm0PPkk\nnTfcAEDKn/9M0nPPgSDQNucORJWBzM/+Z8g3puqeasoMZWGHeu9vspOqlZNn6jv/aSnTSFImsan+\ndZTWE3SPPh/wOwcnz3oRe/4ppO1+hKyNN8RVIz6YvKTwJY8ytxW5uwtPHHNOI+E15CHzxa7Q+rUY\ndp/k4/GqxynRl7C6YPWAZaOMoxjbrmDibfeTv2IF+k8+wfj66whRpg5FotJaGRJfD0gJ9Kf/xWzP\nX4jM64woY9Af25lnYrn8cgSvl6zrr0fW3h51/VNKjHhto6izHx/wRgL+Gu20nQ/jSh1L19iBSWHr\nqLORBBmmqpF77bE2CPl8vmBFTHNHGqlaOaPShg7DBBhr9CdQXQq/kapuH2jIYzHsPp+vX0VMBa60\ncSDIeL3CjCj5u3sjIQgCF01No6HLzebayH0Hc+YUUjpuCtvqZ3D0vb2UlZUO+JszZ2Q3pcpcjU+d\nHJdOSFBKQB1+G1l3NymPPELhokUkP/00Prmc68/7NZ0r7kdhayZ172OhG0kSyfufJGPbvThy5tJ0\nxlNIitAmpHCGXaVSIZeHGtPJOVpaTs4iXZXB2uNrB1SzhZ60DMu119K8di2OefPovugiAERVEm1z\n70TdeYSUg09F3NwrejluOx4xvr6/yc6UHP2Ae1suyFmUsYgdPUfpVCdjL+rTFhLVJlpOfYT2mTeh\na9hC3js/Qtl1PPL5RyHfpKLJ6sE7yIFQJKAiJoDXGPqWEo2EGHZBEJYLglApCEK1IAi3DbV+u6sd\nm9fGr8p/hWJQpUDK8y/w0q0HmPlpFZJSSdell3LivfeQwnTCDcU+874B8XUIfdWEvpJHAGfWTES5\nJuZyrM5bbsE5YwaiVou8O/pTvzxDg1H0q+QNro5J3fsYcmcnbXPvDKme8OkysectxFDzVkIGRMQS\nBgl0nKpkKiqbdUzK0cXVGZulySJJmUSt4yiZBgVVg+LsscxhDT6AfG7U5qO40ibg8Ii8dcjMwmIj\nuUOUXS4sNpKfpOLpI69z9a6ruWrXVVyx8wou33E5l+64lEu2X0L6XUvJ//mtvFe/kJm5e0nW9CVc\n1fkddFpHVi6nslT5wzBx/Ha1tlqKdcUDf2+PB0WdP0QjKZWY/vUvFC0tuEtKuPcnd9E67xQ82dPo\nLr8Q05GXBmrgiF7St99H6v6/YS1dQfPSR4NKjv2RJCkkeRrAaAwNw03K1iFJSuYZV3PMdiymunHH\nwoU0/etfSFotgs1G3rnnYljzBs7jY0j+8O+oW8LHvevt9bhFd9iO0yarh3abl8k5oTZiWfJ0PILE\nusLpSIMrjASB7vE/pnH5s8idFpIqItf8RyMvSYUoQbN14L0ZkOtNSCjGEJ9hH7FSjiAIcuBx4HSg\nAdglCMI6SZLCD78EvLZu7q5ZxtT976Csq0NZW0v3RRfRs2oVrilTEIDXT0km9fY/UVC+MNJuoiJJ\nEts6tjExeSJ6RV8HXTjDLpPJ/CWPkoSk0GAvONXf2Trl54ia8DHcIEolLX/9K6JajWQ0IjObQS5H\nNIWGLWSCwCm549jgUfN55z4WZvi/m7ptP6bKV+ge9yPc/ao++mMdfR76hk3oGjZj76dqGC+xasZ4\nPB7qbHVkq/Op6BGZNDnUEERDEIRgnH102vkhJY/g98gjGZLAOQCozVUIohdX+njWV1rocYusnhzZ\nWw8glwmcMcHBy51vYBCLGZdciAwZMqH3DxkvfqIief7brCvJ5RcegaUlm3jt8NkkL64kY+YR3NWp\nCPa+Y0lard9Iu90IoogkCCCX+6WiBycZJRGVpRrrqHOjn6gkgSgi2O34DHrqbHVcah6H6eDzyNvb\nkbe0oPv4Y8TkZBrWr0fSaum46y68+fl0T5vB8/84ygW9+jCd0/8LXcOnZGy9h4YVLyFIPjI33Y6+\n4VPMk36Geeq1ER8y/UsdB2MwGLBYLAMexuMytShk4OueQqm+lJdPvMzizMXIhhK76j2+4d13UVdU\noK7oa1jKXXs+bQ/8np6zVyHr6EDZ0ICo1dLcs5uUbi9jFAX+36vfd+iLrw/slAWY3LifUW437+q8\nLApZ6seVMQlH9sxQ6ZIYCcyWbehyD5gzG6xhH8aAjcF49TlxrZ8ICbTZQLUkSccABEF4CTgHiGjY\nS5tcrPrtiwM+c06dCqtW4Zo8mUMfvcvdtTfzQ00DP4ywj6E4YT9Bo6ORlbkrg58JghDWsIM/RBMw\nJObJV6CvfZ/kin/SOeOXQx7Ll977yixJZNx2G+qKClp//3uc8+aFrLu4NIX1e0vZ3dHrUYkeMrbd\nh0+XSefUayIew563AK82A1PVayMy7AB2u53U1OiG0e12U2erI1Puf9uZlB2fYQe/hveOjh1MS/Oy\ntc6NwyMOmGjjdrujGvbAYO9Ap6I9ZTyvbu1kfKaWCVlDn48kSVT4XkAQNbhOXMaFY8cG647BX4an\nenoX417NJ6dwG20OF5PmvcqmUYU83Hg3lz3S24DzZt8+j+/bh2Q0krpmDcnPPBNyzNq9exFNJlIf\nfpikZ59FkDyY5M9ilP0TBIG6rVv92z/4IEnPPus36P2M5da9n+AUnSz+tJ701wZ6kO6UFOQdHfjS\n0/2DpIGjzXa8IsFReJJST/vcO8n56DrS9jyKuqMCddtB2mffRvfY8MqnAaL9L5RK5YB7BECtkDEm\nQ8sXLU4unH8ea46sYVfnLuakzYl6nADW1atxTp2KZudONHv2oN25FUVTG9ruLfSwCt2nn5J5880A\nXNz7B6fR+OKLOGfPxvDGG6TfdReXiXCxCPon5CAINK9di2vyZPRvvkHanQ/woiDgEGowKDYjE2Q0\nP/usf/m6daT/9rcACF4XMq+Dovum0/zc2pDl/Rm8faEksdAlovmrgFrRt3/N+5uQ/tFD4Z8XRd1+\nqP0HcI/KiOl3hcQY9jyg/yy0BiDkPysIwlXAVQCTlApspy7BU1Tk/ysuHlCTrisYzaiOUewx7+GH\nRcMz7ds7/HXJ/S+y/lICg1GpVMGL1pNcRk/JckyVL2EZ/xNE7dDeIYDQ04O8rQ1FUxM5P/kJXZdf\nTueNNw7QwJ6YpUXtHkOX701anC2Mrt2IylJN86l/GCBDG4JMgXXUOSQffAa5rRmfPnrlQzTcbjde\nrzfqjdxh66DT3UkyWeiUMkpTY4+vBwg0Kmn0J5DIoKbDycTsvmEObrc7RB528HmCP77u06Sw3Wyi\nydrNlXNim5r0adunHOqu4MyMn/FOrZorXqnh8lkZnDcpjYyHHyL5qaf4HYCl9w+44bpdJNvfoO0v\nBXhQ4FEoEARQKwZ6uZJSiahWI4ii3zj39iFIAY/X60XovZ4ErweBQSE0SQpuE9hO0mppavI/xLxz\nF9CtysGXno4vLQ3XxIm4pk0L8bYDio7j+mmwO/IWYC09i6QjLyLKVLQufhhb0bIhf69I90YAo9GI\n2Wwe4LVPztbx6sEOZiTPJVOdyRsNb8Rs2AE8o0fjGT0a649/DED6+jsxNr2DtXk3otGIa+JEBIeD\nLstJNG4RvQvE3rCs4PEgs9nQAlqAQNqm93dVtR1FsPvQAP5fp2vAcsHtRm4e2Osgd3RBUJsodDkQ\ndnlqmOWKnnYEu4jcbh5y+1iWu2VjgNh0kL580eJeJEn6O/B3gEmTJkktf/tb1PVnpszkpfqX6PZ0\nY1LGXo0RYFv7NsqN5aSp+3TUA/rr4VCpVNj6NRtZJl+FofZ9kivW0jnzhpiOKRmNNP7nPyQ/8QQp\njz1G8jPPoN2yhda//AVPmb/yZ/68ImSnLiBzxZucdd0JPi1fR41nBnOX/AyA9HQvO3aEr36xjlpF\nysGnMVa/iWXKz2M6p3AIgkBPTw/JyckR16mx1ADQ3pnOhCwtcln8ypNlhjJkyHDIa4EMqvsZdhg6\ngRoMxXRU4EqbwO6TdlRygfmFQ2ua93h7ePbYs4wxjuEa01R+2foKvPE2l7bcysbj5TxUPhmjycQ/\nu8/jMxZg1hpYdtarjGcvbTmv8ZfFP+DWl+zox7SSvnIfv16Sy7LRfbXh5ptvxtzrTQbpZ/A6b7kF\n8fQkUg48Td0FG5AUGpAkpF5hrc6bb6bzppv6wji9HK57ARky1Ksupv38oR+mh1odZBmUIUMlOmbe\nBPhDeM6s6UPuB8KHKftjMBjo7BxoWCZma3lpP1S1uTk772yePvY0ld2VlJvKYzrmYDpOuwPt2wfI\n2Ho3DStfxn766bh8Ln649YecX3A+FxdfHFzXumoVdacs47JXarhiZjrnjEsGSULsLc9UlrTi+3UO\nDWe/yoNH/0iLs4VHpj6C1Bsmta1cSe3S3rdfSaTgjVXY8xfimjQpdHk/xN58Q//lN71Tj1Et5+5l\necHlsnIPPY+dR/uc24fcPpblklIJq1eHrB+ORCRPTwL9pcvyez8bEdNTpyMhhXShxkKLs4VjtmPM\nSx8YConmkQQU7QJ4korpKfk+psqXkdvbIm4XZkdYrr+expdfxlNUhKKxsc/DcLlob1fQvXsGosdI\n8fy3mJR1iGf2/iS4eXt75Get15iHPWcuxuo3QBz+4GtJkrBGUagURZHanloAGjtShxWGAdDINZQY\nSjjhPIpJLQ+Js9vt9ojaNYHuVMFjR9l1HFfaBPY22piUrUOlGPqy/Xftv7F6rNxmPYXCFSvJe/Kv\n5LXUc7+vgpNdHi5sK+KBv73L5bJn+FfWanZcns5/Cs7i0c52cvZOJ/l7/6H4pqtx1BoYna7h6V2t\nuLxDNDv1lswCoFCgsh3Hm1qAmJSGpNcjGQwDlqNUhsTlY5ES6M/hFscA/fXg76dJpm3hfTEb9Uil\njv1RKBQhjtHE3pDYgWYHp2Wdhl6h542Tb8R0zHBISi2tC36LwtZM2q5HAH/5p4gYWhGjVLK3R4ZV\nrad8dBZicrK/21WpRObsRN/0KT2TV+HLyGbW6DOpUpmpULT4f3dAUqsRU1P9f2npOEtno7F9EX55\nv79wy425GRzxaYPLBa8ThacNd+6omLaPZbkUJoEdiUQY9l3AaEEQSgRBUAE/BEY8MmWUYRRJyiQ+\n74zfsG9v94dh5qbNHfB5NI8k3EVtnnwlgugluWJt3OfgmjqVhrffpvm55/Dl5oIoUrhgAfuYwpra\n37B8o4I84+dY3Tpe+uL8mPdrHX0eSlsz2hG2wHu93ohGNSD+pRZ0SN4kJmXHX5EUoNxYTlVPFaPS\nVVQNmuguSRItLS1hq2MCc07VnYcRJJEO0ziOdbqCYl/RqLZW817Te/xX01hmXP1rZD092Bcvpukf\n/yDt9ut57sJS5pYm8fR+Czk/3ULWRdtAFNj6r/Np7MrnqkaRhqd/i658D9k/+AO/mJtJa4+X176I\n7TU4gMpSHVdjEsQuJQDQ5fTSavMyOiOydEA8DGXYIdQ5MmnklKSoOdhsR6fQsTx7Odvbt9PkiCzL\nMRSuzKl0jb8EU/XrqNsOBDtOw2nEHGiyo1fKKEsd+BsYa95GEL10j/YnruemzUUj0/B87fNY3OFn\nsDqyZ6LsORn3tCLwJ1Bbejx4fP5rOSCxm4ga9uEwYsMuSZIXuA54HzgMvCxJ0ojHl8gEGdNSprHX\nvBefFJ93uq1jGyX6EnK0fZnkaIlTGFjyGMBrKsRatgJj5atR565GQtLp/HFRQNHQgKynhykc4Fqe\n4JF/b+at2w5R8VYaVreJTFp4hBu4i3swPfcchldfRff++yirqgbs01awGJ86OSHCYD09PWE/D2jE\n6MhFIRMGTryPk3JTOU6fk6zUDo53OoMXfgC32405TJwxGF/vTZzu9Pi7MKflRclD4O+R+Fv130hS\nJnH+NjsyhwPr+efT/NRTOBYuBKWSFK2Cu5blc8/peSgMLjydepr/tQBPu4kPj53KaSWf0v3Z2Zg3\nnUfK/LcpSfcwr9DAC3s7YpIDBv+EL2V3Pe7kL0lKgL5Zm8Up0WPjsRCt1LE/4e6hSTlaKlr8A67P\nyj0LmSDjrZOxDaSJhHnylb0Trp6juqeaFFXKgLBqgP1NdiZmDwoVShLGo/+LI3ManmR/CFQj13Bl\n2ZUctR7l+s+vDzp//QlIhGuGUR2TZ/KXPDZZeyUwEljqOBwSUscuSdK7kiSNkSSpTJKk3yVinwDT\nU6bT7e0OiuvHQqe7kyPdR0K8dYgeiolU6mWZdCWCJJJ88NmYzyEc3sJCavft4xQ2cRsPsC9rLN06\nGe9m+C+8fBq4gT9yD78h/b77yLz1VrKvuQbjq4MU8OQqrGUr0Z3YhNwR2xi4cEQLx7hcLupt9bgd\nmYzJ0KKOIfQRicB0e5WuHq8IdeZQr72rq2vAmEDoq3NXt1fg0eewrVWJTiljTHp07/TD5g85bqnk\n8tLLMT/0e9ruvdc/MjFMbuWUEhMf/lcpm+7M5ej+RmpqjrHi+gmkaLuo3/Y2L928GEHpZn3Teq6a\nk4nDK/LPPdGb0AIoLccQkHCnxN5xGpASiNWwB37LwuT4E9uDCaicDkWwS7sfk7J1ODwiNR1O0tRp\nLMpcxIaWDVg9sQ2kCYek1AUnXFV3HQqrD2N2eKm3uJkyqMxR07IHlbUe6+jzBny+LHsZf5j2B9JU\naTxw+AH+fPTP2L19Xbru5FH41MnB2Q/x0L/kERKjwz4SvnZ1x2hMS5mGgBBXnH1Hxw4kpJD4ejgp\ngcGE81i8xjyso87BVPVaVNXHWJA0GrZwCg9xG93fL+LCP4zllSX+i7KJHG7i99zLnXRdcgnWc8/F\ndtppdF15JQDaTZvIvvRS1J9/jnX0uQiS19+wNAK8Xm/YLtTmnmasXivmrnQmDzO+HiBbk02SMgmb\nvBYgpFEJ+kIy/RUuA4lVdUcFrvQJ7G30K/dFS+Ja3BZ6nlnDuntOcqpsAqhUWC+6KLS+vB9KuTBg\nn46c2QBom7aTr8tnRsoM3mt6j9wkGWeNTWbdIXP0UWi9qHuHa7jiCMXU2mqBGDVi8HvsarlAlmFk\nyoEQvdSxP+Gco8A1cqB3Duo5eefgEl2sb1o/onPqGvtDuhRaGlytYTtOw+nDAJiO/i8+lTFsJVCh\nvpCHpz7M6oLVfNLyCb/8/JdUdPUGGAQZjqwZaJp3xa0fk9er8hiQFlB3HsarSUNURy5Q+DL5Rht2\nk9LEGOMY9nQO3d4fYHv7dnK1uSEDCuRyecTmiwCRPHrzpJ8BEilfhNYtD4fRqdUsKtyGxlyEPK8S\nFG6ayOURbuIu7qXj7rtpW7OGlr//PVgjn/TMM+g2byZv9WrS/utuXN0lGI+9M6LzkCQpbDimule4\nyuvMGlF8HfwhsHJjOQ3OKjQKIWyjEviTpf3j7R6PB5nTjLLnJB3GsTR0uZk+RBjmxJrrueW54xSc\n6EK/ceOwzlfUpOJKHRvMYazMW4nFY2Fz22Z+OiMDlULGUzuGDsupLNWIcg3eOJpTjtuOR5USGEy9\nxUVBsmpYFUuDGaoiJkC4kGVGrwb6/iZ/VVmxvphpKdN4u/FtPCPolBY1KXxeeioAY5ShYZgDTXY0\nCoEx/XIMMqcZff1H9JSu8FcihfsOMiUXF1/MA1MeQCbIuOPAHaw9vhaP6MGZPQulrTnu4dFJGgVG\ntSz40Ne07sOVOSWujuNE8o027OBXe6zuqY6Y8OiP1WPlYNdB5qbNDXldHKpGN9o6PkMO3aPPw1j1\n5rASK4O5fNq/8IpyKnafjyB3kzQnikxwL62PPor5mmsQDQZ0n32G+o/7UT19AGVn1ZDbRmNwOCYg\n/gUgurJiagQainJTOScdJylO94VI+PbH5XLR1dUVFP9Sd/h73Pb7SgGiJk6dj/2Oc57x/47t99xD\nzwUXDPt8HTlz0bQdQPDYmZo8lUJdIetOriNFK+eHU9LYXGvlYHNkoS0ApbkKd3IZyMKX14YjrJRA\ntPXNroSEYWKpiOm/briS4Rl5evaetAf1UlblrcLisYx4PN3eDP8A8JkndoYs299kZ0KWDkW/NGaI\n/AAAIABJREFUB5uxeh2C6KF7UBgmHGNNY3l0+qOckX0Grze8zl0H76Itw9/5PZxwTJ5JxcluN3J7\nK8qekyFa918l33jDPj3FX/a417x3yHV3de7CJ/lCwjAQOlwjHINLHvtjmXg5kiAj+UBkoaJYyMpw\n8tMpL/LO0e9Ru2EVkiSQNLvv4k9Pj1D+l5KC+cYbqd+0CfP11yPqdEgpcvS1Gwas1+ps5Z3Gd2h2\nNIfdz2B8Pt8AiQGfz0edrQ65aKTYlIZJE7thikQgzp6W0kh1hwsxwmuuJEmYzebgHFh1RwUSAh9Z\nC0jSyCmJ0CQlqz7K6D8/B8DJh+6n++KLw64XK46cOQiiF03LHgRBYGXeSo7bjvNF1xesnpxKmk7B\n37aHr+YJoLZUxRVfFyWROltdzPF1h0ektcdLUQISp/EYdgjvAM0u0GP3iFT0PvCmJE+hRF/C6w2v\nRxcHG4JKdwu5qMivfguZs8+563b6ON7pGqAPo+yuI+XA33Fkz8YT42+vlWu5ZvQ13Fh+I0etR7m5\n9kkadWnDSqDmJ6lo6HKjafV3lTszp8W9j0TxjTfsZYYyf9ljDHH27R3bSVOlhZRFhRuuEY5oF7dP\nn4V1zPkYa94KqrYNh32vvECOsYU5V59O5edd6KQCkso/p7q6hpqaYxGbkwKISUmYf/lL6rZuxXnR\nfAwn/Emq9U3ruX3/7Vy560r+XvN3Xqp/KabzGRyOCWjEeJ1ZTBxhGCZAYKKSQncCh0eksTvy63kg\n3i4Igj9xaipia5PE1FwdsggP3e0nPqI+S0XVOafiuiB623wsOLOmIcrVwalKizMWY1KYWHdyHRqF\njMtnZXC41cn2+vBVRXJHB3Kn2T/GL0ZanC04RSfFhuKY1j9h8ecgihLgsceSf+pPuLDN9Dw9cgF2\nNvjDMYIgsCp/FQ2OhmH1ogSotlYzKmk8Mq8DU+V/gp8fbLYj0acPI3gdZG68GUmuonXBPXEfZ1Hm\nIv5nwv/Q4mrlp5lJNLftHkacXUVbjxdly15EuQZX6vCatBLBN96wywQZ01OmD1n26PA52Gvey9z0\nuWFFiGIx7AqFIqoXZpl4GZJMSdbGG5E7YquOGIyx6g282vTgtKZy/SREdR2VnbENXw7g0qt5J7uc\nJ+tacJ27kLVfPEa3p5sfF/2Y6SnT2WfeF/MgX6vVGlzX5XZRZ6vH48wKSUoNF41cQ7G+GKvgn2Ub\nKc7eH9HnQ91xCLNpHG02L9Nzw8fXu9xdPKrZzD1/WoX8nkcTcr6SXI0zc1pwDqparmZ5znJ2de6i\nydHEGaOTMKnlbDwWvupD1Zs4jUeDPZA4jbkiprfUsTCGiVZDEWupY4BwlTF6lZyJ2Tp2nuh72C1M\nX0iaKo03G94cvIuY6HJ30epqpTR1Krb8RSQdeQnB40/QHmi2o5QLjOuNr6ftfAiVpZrWhfcNW25j\naspU7p98Py6ZgstTFFQ3Dz1wpz/5SSokQNG8zz8QJszA7a+Kb7xhB3+c3eq1hkjd9ufzzs9xi27m\npYWGYSC2rH80dTsAny6DliV/QGk9Qe76y4MlTbEit7ehO7mFntIVQWnepTnzEQSR904MHWfvz2+/\n+C2/tWxm+WudzD7cw+vri3hsxmNcWHghp2ScgtljDhqLoeg/C7W+qx635EJ0ZQ1o/x8p5aZyGpzV\nyAUxbGXMYOT2VhSOdg4L/nLQcPXrMouFzrt/jqzHxuXlv4A4OvOGwpEzF1XXsWD/wpm5ZyIX5LzV\n+BZymcDcQgPb6qwhGtzgl+oF4grF1NpqERAo0hXFtH692YVM8Md1R0qspY4BIjlJswr01HS4aLf5\n38gUMgUr81ZyoOsANT01cZ9XYBD1aMNoLBMvQ+6y+Luu8SdOx2VqUSlkGKrXYap+E8ukn+HIWxD3\ncfpTZihjzZibMIgid1T/Ka7CjVyTCi1ODF1Hv9b4OnxLDPv0lOmkq9N58NCDYRsLwB+GMSlMjE8a\nH7Is3HCNSAz1AHDkzqPp9CeRubrJXX9ZcEJOLBiOvY0g+bCOOif42cKc8eBJYX/31pj3c8J+goNd\nB1ld9APyLp2CpBDIe2M9hnffBfyeB4QOBY9E/3DM0U7/90mS5SWkjC5AubEch89BfoY5Jo89MApv\ns6OIDL1iwGScAPLf3clpL+/iyZdECvWJ7fBz5Pr7IALVMamqVBZmLOSj5o/o8fawsMRIj1tkf2No\nElVlrsarTUfUxCYeB/6h63navJilBGotLvJMqpB5r8MhnjBMYP1wb4NzCvwaLbsa+jSXzsg+A61c\nOyyvvdpajYBAqaEUV+ZUHJnTSDr0T+xOF1XtTiZna1Gaq0jf8QCO7FmYp/wi7mOEIz1jJmvNHgol\nBfdV3MfHLbHNG85PUjFVVoMM3/837LGgV+j5/dTfU6Av4IHDD/BK/SsDLiyP6GFX5y5mp80OOzYr\n1lIuiC1k48qYTOPyZ0CQkfP+Fahb9w29Y0nCWP2mvxsuqTj4sVwmI02aiZkjIVOVIrGxdSMyZJyV\nexa2eechnOH/ful33IHi5ElSVakU64tjSjgHCIRjjnX5wyUTU4ujbxAnAaXHlJRGqjucQ4aJ1O0V\nSIKCde3ZTMvVh1Y57dlNwWvv4ZELaG55IKHnCv4ZpV5t+oD5t2fnnY1TdLKheQMz8/Wo5QJbwkxn\nUpmr4grD9Hh7ONh1kFlps2Lept7sjmmweCzEcs33J1JlTGmqmjSdgl39wjF6hZ5lWcv4rP0zOlzx\nNdRV91STr8sPziu2TLwMpa2ZnoNvIUowI0Mg69NbEFUGWk+5P64KpKgIAvqMmTzT2snEpIn86eif\noo7+C2BUy1mgrEJEwJUxJTHn0o9IRQfh+FYYdvB7TL+b9DsWZSzi+brn+UPlH3D1ziXdb9mPw+cI\nWw0jCEJMFTEBYn0IeJLLaFz+HD51CjkfXo12iAHYmta9qLrrsI5aFbJsumk+CCIfN28Z8riiJPJp\n66dMTZlKsioZW+FpSLPUuGeWIbdaSerVCJ+eMp3D3Ydx+GIbKRiYQ3rYcgzRnczU7NC64ZGQrcnG\npDAhaOowO3x02KO35qs7KrAaS2lzypmWNygk5PGgvd2vuFlx8feRl08Ms4cRIsiwFyxGd/Kz4Pzb\nMkMZE0wTeLvxbZRyiVkFBj6rsw684UQvyq5jcWnE7OroreaKEEYcjMcncbLbTVFyYipi4jXsEP5h\nIAgCs/L17G6wDZgze1buWfgkX1wNS5IkUWWtGtBx6shbiDt5FEU1z6OUiSytfQSltZ7WUx6Ia/Rg\nLDiyZ5Jsb+e3BZdgVBj5qOWjmLabq6jihKwAUZW4sCD4jfoP/h17B/63xrCDP4l1Q/kNXFx0MZva\nNnHHwTvodHeyrX0bWrmWKcnhn5LxXLjRSh4H4zXk0rj8OTxJxWR/cgP6Y+9FXNdY/QaiUo+t6PSQ\nZfNzx+BzZfBR09Dj+A51HaLN1capmacCIGpT/d2SK5V03HILHb/+NeDv2vVKXg5aDsb0XQKlhsd6\n6hFd2cNWdIyEIAiUm8qxSL0J1GhxdklC3XGIY0r/TT1tUOJU8+zfSas5SXOWHtPNDyX0PPtjK1iC\nzOsYMOT67LyzaXO1sb19OwuLjbTbvFS29X0XpfUEMp8rrvj6to5tpKnSwnZXhuNktxtRSkziFOIP\nxUBkB2h2gYEetxjUiQfI0eYwO20265vWB52xoWh3t2PxWAaOwhME2sb9lBx3HS+YniCp/n3MU68J\narwkEkfvPk2t+5iXPo8dnTuGPnfRx3ixkt1S7NVQsXK80zWkM9Sfb5VhB7+BWF24mtvG3Ua9rZ6b\n9t7E9o7tzEqdhTJMFjracI1wxFMdAH7D2njGUzgzp5C15dcUvnIGeW9fRPZH15Hx2W9I+fwvmA6/\ngL7uQ3qKv4ekDC0hHJ+tx2edTL3ryJCvqxvbNqKRawZo4diKzkAlNuJYfSooFMg6OpjaKEMj08RV\nama1WTF7GlH4smOujza89hopa9agPD70IOByYzntnkaQ2alqj3yTKKz1yN1WdrpLyE9SkTko1v9q\niZkDpVqa7r4DhjELN1Yc2bMRlQZ09X0x1llps8jWZLOucR3zigzIBQaEYwI1zO6U2ErdgtVcaeGr\nucIR0IhJhMcOwzPs4SpjAGbk65EJDKiOAViRu4Jubzeb2mKbJRxIWg5+2L0rzqNBSmeW8zPseQux\nTLws7nOPBa+xEK8uE23zbk7JOAWnz8nuzui17aquGnSSnS2u0TiHkniOk6Ea4gbzrTPsAealz+PB\nKQ/6h0Z4e8KKfkFsUgL9iZQYioakMtC87HE6pl/fO8IuHbmzE23TNpIr/kn6rt8jeF1hu+EEQSAn\nPZVc2SxAYmt75CSqW3SztW0r89LmDUiy2QqXIAly9LUfoKyuJv+ss8i77nqm6cbFFWdvcjYhCT7y\nNYURa8YHnLvdTvLjj5PyxBMULFtG9qWXovv44+CEmsEEhi9kpTdFTaCq2/0dp+stBSHdprW2Wp7R\n7uH5v15L0vdGXrMeFbkSe95C9Cc+BdHvLckFOStyV3Ck+wjN7mNMzdWz5XifYTccewe3sTDmGvZg\nNVeYMGIk6ntr2AsSVMMeafhMNCI5S0a1nHGZ2gEJVIBJSZMo1hfz1sm3hry/ejw9vFD3AmNNY0N6\nUtZVWvmb8hIcmdNoXXAvxPgwjBtBwJE9C03LbiaYxpOsTGZLe/RQaSDXtksqD2rGJIqDTQ4y9LE7\nnd9aww5QaihlzdQ1XD3qauamhzfs8SROIfqUpWhIcjVdEy+jff7dtJz2Z06e9QL1F7zP8Yt3UHvh\nx9RfsD7soGpJkjCZTExJL0Fy5bC5LXKsflfHLmw+WzAME0DUpODIno2h7kM8BQWIycko6+q4/N12\nmp3NMWtjV3QGEqelQ68sikg6Hc1r1+IePRpRrUa3eTPZV15J5vXXh90k0KiUlNQQteRR01GBT6Zm\nvyd3QBhG9/77yH/5M/LsKi4q/nFM32mk2AqXIndZ0LT1Jc9OyzoNtUzNh80fsrDYwIkuN3VmFwrr\nSbQtu+kpWxmzRsi2jm0Rq7kiUWd2k2VQDpgfO1ziqRjrTzQHaHaBgco2J2ZHX+hAEARW5q6kzl7H\nwa7o4cF/1/mHpPyi7BcD3mJqzS4ONjvQjf8+TcufRdR8uQJbzqwZKJydaLrrWJC+gN2duweoQQ5G\n07oPpzqdBikjoYZdkiQONNvjCo9+qw07QIoqheU5y8NWw0BsUgKDiTccExVBhqhJwacLP6dTqVQi\nl8uZnGvA3TWZSmslrc7wIlMbWzeSqkplUvKkkGW24tNRWk+g6jlG+z3+zrvZz39Efqsr5nDM3vZj\nSJLAvNzo6oLJjz1Gxo03gs+Ht6CAhvXrqd+6lY7bb8dTUID9dH8eQWY2k7JmTbCDTyvXUqQvQlTX\n0Wz1YHWF9+zV7RU0asrwIWdqr8cub2oi6babWPxJLbdWlw1rXOJwsOfNR5Sp0Nd/EvxMp9AxL30e\nW9q2MKvQ7zhsqbViOPY2EgLW0rNi2rdbdPuHP6fPiXj9hqPe4kqIlAAMLwwD0R2g2QX+h/HuQV77\nosxFJCmTWHcy8hyemp4a1jet58zcMykxDLwO3zlsQSGD75UnRdg6sTiy/VVK2ubdLMxYGPx/RSIo\n/IVAQ3fiDHuj1UOH3cukOBoGv/WGPRpDDdeIxHCqBIaLvncG5rSCJDzdkwHCvvJ1e7rZY97DooxF\nYY2ArWAJkqDAUPsBzjlzsK5ahczt5p5/d7I3xilUx6x1SJ50JmQOvHFMR14m5/0rkTktJP/lL6T+\n8Y8Y3noLzZ6+5g0xOZmuK67gxEcf0bNiBQDGl18m5YknSHnkkeB65cZyLOJxQAyfQBW9qDqPsF8s\nozRVTbJWAV4vqmsvQ9tt5+CMAnJ/EX/L+HCRlHocOXP8cfZ+HuqSzCXYfDZqnJ8zLlPDluPdGGve\nxpE9C58hJ8oe+9hv3o/T52R+2vyYz8cnStRb3AkR/4KRXeuRth2driFZIw+Js6tkKpbnLGd3524a\nHaFieqIk8mT1kxiVRi4qumjAMrdX5IMqCwuLjaRov5pRzV5DHh59NpqW3Yw1jSVNlRbxjVpub0Vp\na8KTNY0UrTyhHvvBgDxxHBIf32nDHm/iNMBXZdgFQUCn8z+Fi9KNpCqz0ItFbGkLNeyb2zbjk3ws\nyVoSdl+iJhlHzmz0dR+AJNF5++34TCbSRS01zftjkk/t8DVgJHdA04vp8Auk73wAbfMu8m5ZTeqj\njyLJZLStWYNz9uzQncjl/nme+CfQS3I5KU88gen55wF/nN0lOZCp2vioujtkopLu5GfIfE429JQE\nwzD2h28jZ38V5hQt2sf/g1z+1bZq2wuXoLQ1oeqsDH42KXkSaao0Pmn5hIXFRkwd+1H2NNAz6uyY\n97utYxt6uT7sG1gkWno8uH0ShV9jqWOASE6TTBCYma9n14mBZY8AZ+b4O3jfbnw7ZLuPWz6m0lrJ\npSWXYlAMHFi+6bgVq0tkxbiUYZ9v3AgCzqyZaJt3IwNOyTiFvea99HhCNYI0vfF1Z+bUXpXH4csV\nD+Zgsx2jWhZX38J32rDLZLJhxcvjKXkcKYGbQ6PRMDFbi6d7CjU9NZy0D9SD/rT1U4r1xVG1RHqK\nz0DZ04i64xC+9HROvvYaO55+iE61h8Pdh6OeR5fLgVfWTr62r4PTdOQ/pO/6PbaCJdiOzUH5zjEk\nQaD1kUfoWRVajz8Y+9KltN1/PwBpv/kNuvffDyo9Ti5p5d0jFq549Rh7Aq/soofUPY9i1RXwtncm\n0/J0HKnbSuGL6xAFsPzpcWQZWUMeN14EQRjw/xYEAZlMFow/2woWIwky9Cf6qmPkgpwlWUvYa97L\nxDwvF8g34ZZpsRWETp0Ph1f0srNjJ7PSwldzRSKQOC3+GksdA0SqjAGYU2ig2+Xj6KAkeYoqxd/B\n2/IRNm9fqMbqsfKP4/9gvGk8SzJDnZe3DpvJMymDobmvCkf2LOQuC0pLDQszFuKVvGzvCO1+17Tu\nQ1RocKeOIS9JlViPvdnBpOzIInjh+E4b9uF6IyO52ONBq9UGbwyVSsWELB2dbf4Ea/9wTKOjkUpr\nJYszF0fdn71gCZJMgb72AwC8JSVMSp5MjkXC++o/o2776YlqBEFiYqo/rmmsfJn0nQ9iy19My6KH\nsC7+AZJKjnCeBqbE/krYc8EFdP73fyNIEhm33kq+x4BRYaQkp4X7lxfgkyRufree325oQHbwZVTd\ntaxLuwJRUGA0NfGbk3/mVw8uoOHeu5AWRP/+w0Uul5Ofn09RURElJSWUlJRQXFxMQUEBkiQhalJx\nZkxBV79xwHZLMpcgIlJj38QKxQ42KeaHLWcNR0VXBVavNWI1VyTqzL3iX1+DquNgom07M1+PAAO6\nUAOszF2J0+dkQ0uf5PS/6/5Nj7eHq8quCnlY1PUmTc8amxKXcUsEjty5SDIFyYf+xSjDKLI12WHD\nMZrWfbjSJ4FMSX6Sig67F4dn5CWPnXYvDV3uuPtKvtOGfTiJU0hw8jQCgiAE4+vgNy4TsrRI3iTy\nVGMGhGM2tm5EQGBxRnTDJqpN2HPmBcMxAAabh9d/fZRzH3oDZWVl2O0kSeLNpleRJDlLCyZhPPq/\nZOx4AFv+IjpG/cpf9rd0KfUbN+I8dToZn92N0hK7qJPl2muxXHUVLU8+iZSURLmpnEprJXMLDTx7\nQSmXzUzni7pmkvb9jVrDNJ63TKQ0w8y253+FUW7g6mVr8P7opzEfL140Gk0wiT3Ycw8YL3vhUtSW\nqgGSzfm6fMqN5XzS+C46HDzVMz/mYdfbOrahlqmZnjI9rnOtt7hI0coTopMfeDMZLiqVKmJlTJJG\nQXmGhp0nbCHLRhlHMd40nrdPvo1P8lFtrWZ903q+n/v9kIQpwDtHvtqkaX98ukws4y/BWPMWmtZ9\nLMxYyAHLAbrcXcF1BI8dlfkozl4Zgbze+aeJ8NoD9evxKq1+Zw17vFICg7f9sr12SZKC8fXAMcdl\nG1DKBUzeGdTb66mz1SFJEp+2fsrk5Mlhp7QPxlZ8OkpbM+r2LwC/fnvlGbOQixLJd94OYqgXsb7p\nfdrZR4HvbMY1biFj+33Y8xZi61pEwenfw9A7UNuXlUvLqWsQFTqyPrkBwR3jsGJBoPPWW3HOmQPA\nObtcuE/U0OPtQaWQ8ZPpGbw57hOSBBvXdKzmiLmFs3fcxYOPVvLv/9XH9L2Hy1DXSSBUZiv0h1j6\nV8cALMlaQq3XzOfGXHaK5WyrC6/R3h9REtnevp3pKdNjFv0KUGdOXOJ0uKWOAYYqDZ5dYOBImyPs\nw+7svLNpdbWyo2MHT9Y8SZIyiR8V/ShkPbdX5IOjXV9p0nQwlklX4NFnk77jAU5Jm4eIyNaOvn4T\ndftBBKlP+Cu/V3EzEZUxB5v94/9GDzHEfTDfWcMOI0uC9je6XwYBD7E/eq2GsRkaujrGI0PG5rbN\nHLEeodnZHFK7HglbwamIMhVJh58PftZ94810mOQYP9+P4bXXBqxfb6vn6WPP4O0ZzW0qFRnb7sWe\nO59u4Wwybvs1gteL3NynFe/TZdK6+GGUPY1kbrkT4pyOo1+3jlX3vsgTf6jjip9vZ/LyDpb/bAsf\nN7/Dla75fGGvY477Ea5+tRYA6cyhY/kjJdp1otFoEAQBryEXV0o5+vqBSn+n6sagEiXW5ZSRoVfx\nWRhRsMFUdldi9piZnx57NQz4nYG6b0CpY6z7mFWgR5Toy6H0Y3babDLVmTxW9RhHrUfDJkzBnzTt\ndvm+2qTpICSllo5ZN6O2VDGpfjv52vwBb9Sa1n1ICDgz/FVtifTYA/LEijjn2n6nDftwEqcBAjf0\nl0X/MEz/Y07I0nGsVcWEpElsadvCxpaNqGSqmGOxksqIZfLPMNR+gK7XuyzImcRff+zv4Et76CFk\nFv+IMbfo5pHKRxAkNQtax7KoZg32nHlYkn5C5n/9CsHrxXLVVXRdeeWAYzizptMx8wb0DZviHhXo\nWLQIZ1kpo0+62Lnx51hqZvN29VU8nJHEjtEnubrudf74ZCVKUeIRbsCxJHwVUKIYqnKqf+WHrXAJ\n6rYDA4as5NRvZIndzgaxnXlF/o7LoWKr2zq2oRAUzEyNT+Ok0+HF5ha/EaWOAaKVE4/N0GJUy0K6\nUMGffD4r9yxsXhvjTeMjOi5vH7GQ+zUkTQdjL1iCPW8BaQf+xqLkaVR0VQTlPzSt+3GnjELqFf7S\nKmWk6RTBwdbDpcfto6bDNSzdpu+sYY+WsY8FtVodt7RArPQvc+yPSqViQrYOjyhRpp5Nk7OJDS0b\nmJs2NyhdGguWiZfhShlD+o77kbm6EASBthWns2ecCcf0aaw4PYOyshJ+fNNaam21yKuWs8b3D/Y0\nTmHatX9C96NrkTmddK9eTectt4Q9RvfYH2ItPYuU/U+iO7Ex5nMTk5NpWfsP2qaPx6wwoBChyONl\n+b8vpOKKvSx6M5Ocbjs7mcXtJF6OdzBDSU70jyPbC5cgIKE70TujVpIwVK9juSKHbp+NjMxq3D6J\nXQ2RwzGSJLGtfRtTU6bG9T+FvsRpIsbhDbfHYzBqtTrifSaXCczMM7DzRE9Yydkzss9gUcYirh19\nbdh91JldHGiys+JrSJqGIAi0z7oFfB5WNB1GCsh/iD407QdwZgzUX09EZUxFs6N3/N//N+xBhhtf\nDyCXy0fk8Q9FuJtKpVIxPtN/3nLHJOSCHK/kjTkME0SmpG3+PcidFtJ2+5uDpqVO5+pf5bNlzU1U\ndBawvOzP7Fr3EI/e5eKlmn8ieCUufGUtVb7RrJV+im3ZMtrvuy9ya7wg0D73Dtyp5WR/8t9kffLf\nwbj+UPhyc7G+8hZp3i72XL6Q9lWlPP7JjSDJeJhbuI6/sJK38PDl9xMMZdz651vcyaPxGPKD4Rh1\n+wFU1nomFJ9PijKF457PMKnlA7RjBnPMdoxWV2vMEr39CYh/FSYgFCNJ0ojvERg6nDO7UI/Z4fc8\nB6NT6Lhx7I3k6/LDbvt1Jk3D4TUV0jXxUiYd/5gydRab2zajslQj89hwDRqskW9ScnKEMfaDzXbk\nAozLjF/o7jtp2BPljSTiwo+033AeilwuJ1WnJM+kpLpVzoyUGaSoUoITkeLBnTYWy8RLMda8hfbk\nFqYmT8WplrPXvBdFUhvzp/0Fr0zGafVVlL1Ug7DGxgzLbiRk/Dd/pOWxx4KNRpGQFFqazvg75slX\noWnZQ967PyH7w1+gad4V0yDgH09+mRkFB7hl6304vH6vZB3n8DjX0Uri69XDEcv/OLiOIGArXIK2\neSeC24qx+i1EhQZn8ekszlzMHvMeZhZJbK/vCTsyD2Br+1ZkyJidFqa5awjqLW70ShnpupEnERPl\nuAwlmjcr3x83X7u7La7yv0DSdMHXmDQNh2XiZXgMeZxpbqXSWomlyR9rHzwxqSxNg9nhCw4dHw4H\nm+2MydAMSxPoO2nYIX7xr3D0rzNPFIIgYDCEJokCKJVKxmfpqGixc93o63hoykNx6Yj0xzz5StzJ\nZWRsu5dkZJQZytjTuYf8K+/gpdNT+Mm0P8AyNd0mIyZfN4/yK8ZxCBAgxsSaqDJinno19ee/R8f0\nX6EyV5P7wVXkvvdTf8giQnJV8Dh44LR72HlyOi8evGBY32+kxOoA9H8Q2wqWIIhe9PUfY6h9H1vh\nMiSlnqVZS/FJPoxpB+lxi1REkFnd3r6dickTh6V1U2dxUZgyshBjAG2C5I6HagJM1Sn4xdxMdpzo\n4Zo3jgcbrKLR0uPhfz5ooNvlY+XXmDQNh6TQ0DH7Fs7q8M873tKxHa8uE69+oIzEgmJ/vH1zlLe3\naLi9IkdancOeizAiwy4IwmpBECoEQRAFQUi82v0ISEQt+pfhsUuSFPWmUqvVTMjSYnb4sDm1ZGlG\n4LnKVf6QjKOdtN1/ZHrKdCqtlRgmbsP1+hU8sewRvphXTkn+Xk7nA57iStrIGNahJKUuqoe+AAAa\nqUlEQVSerok/5cT579A253bkzg6yP/kVxS8uJP/N88n+6DrSt/+OpIPPoj/+Hql7/0y+qZEb3r8f\n6WvyL2KVnOhv/F0Zk/FqUknb8ygyTw/WspUAFOmLKDOUccy9BZkAe8PMQq231dPgaBhWGAb8A6wT\nkTgVBCFhhh2GDsdcODmNh79fiMXh4+rXa9l0LPwISFGSePOQmctfOcbBZjvXL8gKkW3+JmDPX0RK\n9gImujx85G3yx9cHPWwzDUrGZ2r5dJiG/UibE48oDduwj9T6fQGcBzw5wv0klERJAnwZtewKhSLq\nQ0etVjOx959Z0eIgd4RT6F3pE+gafwnJFWuZl3UHLwPW3Ut5IfdllHIP1zh+hbm6iA2UsYHQ6U7x\nIsnVWMsvxDr6XPR1G9C0f4GipwmFrQl12xfI+zV2vFyxis9ODM/IJYJYJScGhBtkcuwFp2Kqeg2P\nPnvA9J6lmUt56thTFGV2sq+p74bscHXwUctHfNj8ITJkESWmo2F1+eh0+BKSOIXEOi1qtRqnM/qA\n8ul5ep48r4R7NjTwmw0nuXCygytnZyLvLeNr6HKzZlMTB5rszMjTc+OibLKNX50YX7x0zLqF5Z/8\nhDUpRqpTSgn3/rWo1MjftrfS2O2O+z4+0Cv8NTEO4a/+jMiwS5J0GPjKdFViJVEXbUAkyeUafpxs\nMOHKHPujUqkoTlGjU8qoaLFz+uiRJ47MU36O7sRGFu17lhvn/zcN/znO/Nm7+UX3DRypnYTk/hJi\nmDIltpIzsZWcOeBjwWNDYWtGYWvhtqe+H3UX6emxjwIbDrGW+wUSqB6PX9jJVrgEU9VrvbrrfW8b\nizIX8dzx5zCm7+Xw4RS2te3k49YP2d25GxGRycmTuXrU1aSqUuM+10QmTgVBSGh3daACbagqskyD\nkj+uLOKJba28fKCTo21Ofr00l4+qu3ludxtKucDNi3JYXp70jbMpg/Ea85hTuAqsH/GRUuTcMOss\nKvEb9k3HrfxwSnxNdgeb7RSnqEnSDO//9JVlJQRBuAq4CiA3N/fLPE5C4usBtFptwgx7pDLH/qhU\nKmS9mfCKltgGUQ+FpNDQNv9uctdfzg/2v4h29l6ePHQx60tnYjsSm8RsopCUejzJZXiSy9iw2QJY\nvtLj9yceB0Cj0QQNuyNnHu0zb/Ib9n6YlCZmpM5gv/kzVKU7efBIF8nKZM7NP5fTs08nRzv837re\n0lvqmADxr2glisMhnnp4lVzGLxdmMy5Tyx83N/HDF6oRJVhQbOCXC7JJ13+1yp0jQTXlOsbuqWKz\nvSqsYc82qijP0LDpWHdcht0nSlS0ODht1PBnDgwZ3BQEYYMgCF+E+TsnngNJkvR3SZJmSpI0MzU1\nfo8lHhJp2BPdqDSUMZHJZMhkMiZkaTne6cLmDj+MIl5cmVPpHncR2ta9HGybxG/MP0PyynDUhA4A\n+bI95W8C8ToAA64DmZzu8T9GVIfeeN/P+T4u0YnoymKW8mqemf0Ml5RcMiKjDv7EqVIukG0YmeGL\nxbmIl+GMkzxjTBKPrSpmdoGBu07L47en53+rjDoAgox52adR01NDs6M57CqLSowcaXPSbI1dxrem\n04ndIw6rfj3AkB67JEnLhr33r4GRKtYNRqPRJKxRKdaHhFKpZEK2FlGCI61OZuRHD9/ESue0axEV\nWoxl55D/Vhuj03Xcd/jE0Bt+R4nHsMe67tSUqby64BV+ue4ErS0CClliXorrzS4KklTBmPRISHRR\nQCBX4Ysw6zYSZWka7l9ekNBz+aqZlz6P544/x9aOrZyXHzrTeFGJiad2trH5eDerJ8fmtR9s8r+p\nDzdxCt/BcsdEa6kPV9N9MIPVHKOhVqsZl6lFACpa4ptOHg1JocU87VoOOlJps3lZXPrVjJf7phLP\n/zUer1QhUzAlR8+RNkdCpFsB6izuhCVOv4xBMl+V1PU3jSxNFqMMo9jaFn4IfV6SilFp6riqYw42\n28k2KskcwdvZSMsdzxUEoQGYB7wjCML7I9lfIkhkGCZAIjwcSZLiMuxGtYKSVDX7muwJlzb49JgV\npUxgXlHkevrvOvFKTsQ7bWharg6vCF8k4MHs9Iq0WD0JEf8aqdRGJL6M++7bwvz0+VT1VEWcVbyo\nxMShFgdtPUOHYyRJ4mCcg6vDMSLDLknS65Ik5UuSpJYkKUuSpO+N6GxGyEikeqORiEalcGqOkQgY\nkIXFRvY12nlkU1PICLnhIkoSm451MyNfj0H15UkmfNMZjiGK59qamK1DLsC+MPXs8XLC4kYCChOQ\nOE1k/Xp/vqwHxreBgErntvZtYZcvLu1tVopB+fNElxuzw8ekYZY5BvjOhWK+qR670WiMed3Aa/8l\nM9L58bQ03q3s4tZ362Me4hCNyjYnrTYvp5bGfj7fNYbrAMSTSNcqZYzN1LKvMVTZMF4C3ZpFI5xz\nmujGpP58lQPgv2nkaHMo1ZfyWftnYZcXJKspTVXzaYTGrP4Md7DGYL5Thj3RidMAI91nPPF16Ivr\nywSBn83K5PYluVS0OLjujdohW7IlSaLe4qKhyx3Wy994rBuFDOYX/d817DA8QxSv0zA1V0dlmxP7\nCCubas0uZEKfzvdwkSTpSwuZ/F+NsQeYnz6fSmsl7a72sMsXlRj5otlBhz16OOZgk4NkjZyCEf6v\nvznqOglAoVCMaNRXJEbaqCSXy+O+8FUqFQ6HPzt++ugkck1K7vqggWvfqOU3y/JDKmWarR4+ru5i\nQ3U3tb3NLDIBMvRKckxKck0qco1KNtZ0MyNPj0H9fzcMM1wHIN6mnmm5ev69t4ODzQ7mFA4/n1Fv\n9ncuquQju7aVSuWXcn+A3xnRaDTBa/b/GvPT5/N83fNsa9/GyryVIcsXlZpYu6edLcetnDMhfLn3\n0XYHm2utzCnQjzis9Z3y2L/MBM5IXmHj8dYDDP4uE7J0PL6qhEy9klvfq2fdITNdTi9vHjJz/Zu1\nXPRiNU/vasOgknH9gixuPTWHi6elMylbi9snsa3OytO72mizeVmWgG7WbzPDrZyKd2TihCwtSpkw\n4nBMTaeT4m9wfD2A0Wj8Pxtnz9PlUaQr8mu0h6E4RU1RsipidczJLje3v3cCbdo2Zo2NfZ5wJL5T\nHvuXJbMb2HcsbdODGUrNMRLh2rSzjUr+ck4R933c+P/au/cYuerrDuDfcx9zZ+7szK73OTO7g1mD\nATuLi1Ubii3s1qCIFrdIOCpq1KKIpjQVhTyokKmFqyhpVYmmQVUiRRCklMQSSUqjpCgIiBSFUGS6\nBoODHUIMwfba3sS7a+94X/PY+fWP2bsP787Mfd+ZO+cjIVjvMPvz3TtnfnN+53d+ePK1UfzH/46i\nLCo3zae392DPtcma/TVmi2VMzJaQSbT2x2Yn90k0GkWhYK7PtqZI2NQXw9vn7S+gnssVcC5XxD1D\nzjb1eZlfN+i67tnhNM1gR/cOPHf6OYznx9c8p3fXhiQOHR3DxdnSilbEEzMlPPrj05hPvAqx7n/w\njd8A0+IC9g3ss/1GGZoZu9VyNKvsblSyO65q/48ekfGljw/gb2/pxZ9v6cLT+wbxzCcG8cmt3XWb\nJsVUCf3J6tULrTDbctpywupO5JvSOn49Nocpm3l241i57Q43qbl1sEYtRjqmVe3o3gEBgcPjh9f8\n/q7BBMoCK87FnSrMY/+Lp3FJfgvoegG3dt2K3T278e2Pvo1nPnwGZYtnChtCM2P3cmEIqNy0iqKg\nVLJWmRKP28uX1doQI0uEey02FTLDGGfQsy6vx+HkPrGzgPrsW5VufXYWrIfPTCGVUDHgcDHN6xPB\nDIlEAvl8PvB7KAhXxa/CQGwAr4+9jrsyd636/oZODQPtEfzsw8vYu2kdCqUyDr40glOzv0R8/Xdx\nXfIGfP76z0OVVLRH2vGjsz/CZHESD1/3MFTJ2qfs0MzY651d6QarsxG7aRjj/3WzA5+Zn9fT09MQ\nL0gv/95me7BXY3Vsm3tjUGWyVc9enBc4em4G2wecL6Z5nYYxtHo6ZmfPTpyYPIFLhdXN7YgIuwYT\nOHpuGhdnS/iXn57DsfHfoP3q7yAdS+HA5gPQZA0SSbh/8H7cd/V9ePXCq/jy8S9jdt7aonRoArsf\ndbR2Nio5+WjqZ22wLMvQdT3wsjVZlpHJZDx7k1YUxVGQtJpaiyiVhm52FlBP/HYGs8UytjlMw/iR\nXzfIstzyu1DLKFdPx2yopGMeeeE0fn7mLHqvfRa6ouHg0EEk1KVPdESEfdl9eGjjQzh26Rge/8Xj\nyBXr18EbQhPY/bhxrd6wTnes+vUCISJ0dnb6GgCqjSOdTkOWZc/GEcRZuFszOj4YzyM3Zy3PPjwy\nDYmArf3Om8D5mftu5eqY9fp6ZGKZqpuVNnZFkU6o+GhyEpnrnsW8NI2DQwernpR2R+oO7N+8H6em\nT2H/O/tNjyMUgd1JysMKK7NZIrK023QtbvfNrkaSpMWSTC/OeTWDiJBKpRZnw7quezIONwKc5QXU\nTBwCwLEq56BWMzwyjY/1xRy3fvA7rRePx1s2HUNE2NG9A+9eeheTy04LW/79T9/SiWs2fQ/TOI/9\nm/ZjQ9uGms95S9ct+OLQF3GpaP7sglAE9lgs5suNa+VjeL2zTc3wIxVDROjo6FgMVJqm+f6iJCJ0\ndXWtuF6xWMz1cbh1CIvV57ihJ4aoYq2e/eJsCb8em8P2AecTFr8mCIZWT8fs7N6JMsp4Y+KNxT+7\nVLiE18dex9MfPI3/mngcvyufwIMbH8TWdVtNPefm9s34yk1fMT2Gpq+KISKsW+ffSea6rpvagRqN\nRh3nif2oYiAiJJNL7XsVRbHVW9vJz08kEivGAFT+7naqkGpxunBqsPp7UWXCx/p0Swuobxpljlnn\n+XW3D9YwI5FIoFAotOTMfTA+iFQ0hRfPvYiTl0/i+ORxjMyOAAAiUgQ3JG/Avuw+7OnbY+l5rRzW\n0vSBXVVVX2cHZjYquZUaMppVebVN+8rZukHTNMzMuNcHvtbPj0aj6Opau3RT13XkcuYXjOpxq3LK\nTouJrRkd3xy+gEuzJXTE6r/shkem0R6VsbHbndSR33Rdx9jY2n1Two6IcFvPbfj+me9jdG4Um5Kb\nsKdvD4bah7ChbYPl0kU7mjqwG4HJT7FYDJ2dnZiYmKga3K30Xq8nkUhgbm7Os5nPlTNloPKinJ2d\n9Xy2Jcsy+vr6qqYJdF3H5cuXXRuHm6ktq2fh/l6mMmt+5/xM3QNOykLgyMgUfr8/DsmFFEoQnRcV\nRUEkEjG9Szds7r3qXuzu3Y1MLAOZ/O/L1NQ5dqtdE93S3t6OdDpddfYXiURcS6N4VRdspGDW+jv4\nNcNLJBI1Z9BuHktoPJ9brG48u34xz17/k9AH43lcnJ13nIYBgu2T3srVMaqkIqtnAwnqQBMHdiJC\ne3t7YDdONBrFwMDAqoZSblfoSJLkWelftU87ftSym8n9SpLk2lgkSXI1ZadpmqU3CkUibEnrphZQ\nh0emAMCV+nWnlVlOtHJ1TNCaNrADa6cR/KQoCvr7+1eVCLr9KcKLmU8ikaj6qcLrvjtWfoZb19KL\nlhNG7b9ZN2fbcOpSAf93Zqrm446cmcY1XRq6dGdvaqqqBhrYFUUJfMNbq2rawB6Px32pGqlHkiT0\n9fUtVubY6b1ej9vpGDNrE350AjQTFN2qq49Go67fL5qmWbpOezd1YH1HBE++Nlr1kOvZYhnv/nbG\ncZkjEaG3tzfwVEgrp2OC1JSBPYhF01qM8aTT6aoVHk64nY7Rdb1u3b/VTThWWFkbcSPP7uX9YmXW\nHpElfGFXGqOXi/jWmxfWfMzRc9MolZ2lYYwUTCMcV+fHxkG2WlMGdk3TGuKmvVIsFvNsMdetmY+x\nNlGP2wuXy1nZvOVGWsjLdrKRSMRSnfiNKR1/uqkDz/9iAu+PrS5jHT4zjahCGHJwmLHRIqIRKIri\n665XVtF0gb3RZut+cSsdY3YR0WhT7IVIJGKpntzJm6Ufi+xWc+1/c3MvOqIy/u1n5zFfXvk7PTIy\nhZsycdvH4BERuru7Pe90agXP2v3XOL99k7ysEmlkbvy9rQY5L2a5dqqGnObZvV5AVFXV0ptPmybj\noZ0pnBzP4/l3Jxb//GyugLO5oqM0TCQSCaQEuJZ6Za3MfU11tavtlGwVbqRjrAQ5rxpxWd3i7qSa\nRdd1XxbZrc7adw0msGN9G7515ALO5yqbeI4sVMvYPS3J6KnfaK8PRVE8bcXMVmu6Kx1k+VbQnKZj\nrFaGeNGqwU5tut3mXX6m7RRFsfRJhIjw8M4UiAhPvjYKIQSGR6Ztn5ZkbDhrxLUnoPJJgoO7f5rm\nKhvNvlr5xnCSjrmy2ZcZTg+lWIvdTwF2jhhUFMXXPkLr1q2zNMbeNhV/vb0HwyPTeOn9SUenJfnd\nDM8OI7g32ieKMGr4KElEkGUZ6XS6JRdNr2Q3HWOny59bbW6XP5/d/K/VN7QgFtkVRbH8ifLuzeuw\nqTeKr/68Utu+PWt9odFIwTTDpIeDuz8c3QlE9AQRvUdEx4joB0Tk6ivJWGjLZrMtffr5cnZbsNp9\nQ3BzoVoIYfv3aGfTVxCLiFZn7bJE+MJtaZSFgEyVLpBWGG/YQbTmtUvTNA7uHnP6Fv8KgCEhxBYA\n7wN4zPmQKiRJQiqVapqZiF/s1GQ76Rni5kYlTdNs/y6tHtsXVCWGLMuLKRGz1+2arij+7tY+3HNj\nJ+ImT0syJj39/f01O2Q2Kg7u3nJUqCyEeHnZl4cBfMLZcJZmIBzQq7PayldVVduLam6dqORGczSz\n7YTNbsLySkdHB5LJJKamppDL5VAsFuuO+Z6h+huKjCCYTCbR0dHREC01nDCC+8WLF1fcz9w4zDk3\nd6DcD+C7Zh9sHFZhnMcYiUSgaZrl/hutyMrHbqdBztj56UZfbafpArP3RSQSCbz5lCRJSCaTSCaT\nKBaLyOVyNXvLG39+5QzWeI3Isoz29vbQ1YRrmoZUKgUhBEqlEmZnZzEzM7Mi0C8/2IaDvjl1AzsR\n/QRAao1vHRBC/HDhMQcAlAAcqvE8DwB4AACy2SzS6TRUVW36WUcQjHSM2ZOVnOaao9Go48DuRgte\no0VyvdOrGm2RXVVVdHV1obOzE/l8fvG4PyHEioAlhIAsy6v+aYV0BRFBVVWoqopkMrkY6EulEsrl\nMoQQi/9e/t9AawX78fHxUTOPqxvYhRB31Po+EX0KwF4At4saV1gI8RSApwBg27ZtghdDnTGbjonH\n445neLFYzPFJRm4tZMbjcUxNVTbyrDWeoM74NMM4CpDVtzzQsyUTExMT9R/lMBVDRHcCeBTAbiGE\n94dkskXGjsr5+fmqAddO7fpanDYEc/Okq+7ubiSTSeTzeczNzSGfz6NYLK7IP7fCDJexWpzm2L8G\nQAPwysKL6bAQ4jOOR8XqkiQJ2WwWuVyu6vmrbp0aZKQE5ufnbT+HWzNVo7Ze07TFNy0hBAqFAgqF\nQsPO1hnzk9OqmGvdGgizzlgYbWtrw8TEBKamplYsOLnZ1bCjo6PmAd61eNnbHVgZ7BljTbDzlNUn\nyzJ6enrQ398PTdMWg6ib7VKNNxCrATqoA8cZa2XcAT9EjO3aMzMzKBaLrvdT7+7uRrFYRD6fNz1z\nF0JweoQxn/GMPWSMGbIXJX9EhFQqZfoNw0gH8Qk6jPmLAzuzRJIkpNPpuiWUxiHfXpwByxirjQM7\ns0xRFKTT6TXz7Uat9sDAAKdgGAsIB3Zmi6Zpq5pPGT3B0+k0p18YCxAHdmabruvo7Kw0r5JlGZlM\npqWPLmSsUfC0ijliLI7GYrFQNadirJlxYGeOcZ06Y42Fp1iMMRYyHNgZYyxkOLAzxljIcGBnjLGQ\n4cDOGGMhw4GdMcZChgM7Y4yFDAd2xhgLGQ7sjDEWMhzYGWMsZDiwM8ZYyHBgZ4yxkOHAzhhjIcOB\nnTHGQoYDO2OMhQwHdsYYCxkO7IwxFjIc2BljLGQ4sDPGWMg4CuxE9CUiOkZEbxPRy0SUcWtgjDHG\n7HE6Y39CCLFFCHETgBcAHHRhTIwxxhxwFNiFELllX8YBCGfDYYwx5pTi9AmI6J8B3AdgEsAf1Xjc\nAwAeWPhyioh+5fRnO9QNYCzgMTQKvhZL+Fos4WuxpFGuxXozDyIhak+yiegnAFJrfOuAEOKHyx73\nGICoEOKfrIwyKER0RAixLehxNAK+Fkv4Wizha7Gk2a5F3Rm7EOIOk891CMCPATRFYGeMsbByWhWz\ncdmXdwN4z9lwGGOMOeU0x/6vRHQ9gDKAUwA+43xIvnkq6AE0EL4WS/haLOFrsaSprkXdHDtjjLHm\nwjtPGWMsZDiwM8ZYyHBgB0BEjxCRIKLuoMcSFCJ6gojeW2gR8QMi6gh6TH4jojuJ6FdEdJKI9gc9\nnqAQUZaIfkpEJ4joOBF9NugxBY2IZCI6SkQvBD0WM1o+sBNRFsDHAZwOeiwBewXAkBBiC4D3ATwW\n8Hh8RUQygK8D+GMAmwH8BRFtDnZUgSkBeEQIsRnAHwB4sIWvheGzAH4Z9CDMavnADuCrAB5Fi7dD\nEEK8LIQoLXx5GMBAkOMJwM0ATgohPhRCFAA8h0oJb8sRQpwXQry18N+XUQlo/cGOKjhENADgLgDf\nDHosZrV0YCeiuwGcFUK8E/RYGsz9AF4MehA+6wdwZtnXI2jhYGYgoqsBbAXwRrAjCdSTqEz+ykEP\nxCzHvWIaXa2WCAD+EZU0TEsw0x6CiA6g8lH8kJ9jY42HiNoAPA/gc1c0/GsZRLQXwO+EEG8S0R8G\nPR6zQh/Yq7VEIKIbAQwCeIeIgErq4S0iulkIMerjEH1Trz0EEX0KwF4At4vW2+BwFkB22dcDC3/W\nkohIRSWoHxJC/HfQ4wnQTgB/RkR/AiAKIElE3xFC/GXA46qJNygtIKKPAGwTQjRCBzffEdGdAP4d\nwG4hxIWgx+M3IlJQWTS+HZWAPgzgk0KI44EOLABUmen8J4AJIcTngh5Po1iYsf+DEGJv0GOpp6Vz\n7GyFrwFIAHhl4USsbwQ9ID8tLBz/PYCXUFks/F4rBvUFOwH8FYA9C/fC2wszVtYkeMbOGGMhwzN2\nxhgLGQ7sjDEWMhzYGWMsZDiwM8ZYyHBgZ4yxkOHAzhhjIcOBnTHGQub/Af5JqizaKCzBAAAAAElF\nTkSuQmCC\n", 369 | "text/plain": [ 370 | "" 371 | ] 372 | }, 373 | "metadata": {}, 374 | "output_type": "display_data" 375 | } 376 | ], 377 | "source": [ 378 | "# observe 6 points of training data\n", 379 | "x = np.array([-4,-3,-2,-1,0,2]).reshape(6,1)\n", 380 | "f = np.sin(x)\n", 381 | "\n", 382 | "# apply the kernel function to get the covariance\n", 383 | "K = kernel(x,x) # K\n", 384 | "K_s = kernel(x,x_s) # K_*\n", 385 | "K_ss = K_ss # we already calculated this before from x_s\n", 386 | "K_i = np.linalg.inv(K) # inverse of K\n", 387 | "\n", 388 | "# find the new parameters, using the property of conditional joint Gaussians\n", 389 | "mu_post = mu_s + np.dot(K_s.T,K_i).dot(f)\n", 390 | "K_post = K_ss - np.dot(K_s.T,K_i).dot(K_s)\n", 391 | "\n", 392 | "# use the parameters to sample from the posterior distribution\n", 393 | "# From each of the 50 points, take 6 samples from our prior\n", 394 | "f_s_post = np.random.multivariate_normal(mu_post[:,0], K_post, size=(3)).T\n", 395 | "\n", 396 | "# find the standard deviation to demonstrate the prediction uncertainty\n", 397 | "s2_post = np.diag(K_post)\n", 398 | "stdv_post = (s2_post**0.5).reshape(n,1)\n", 399 | "\n", 400 | "# Plot the function\n", 401 | "plt.plot(x, f, 'bs', ms=8)\n", 402 | "plt.plot(x_s,f_s_post)\n", 403 | "plt.gca().fill_between(x_s.flat, (mu_post-2*stdv_post)[:,0], (mu_post+2*stdv_post)[:,0], color=\"#dddddd\")\n", 404 | "plt.plot(x_s, mu_post, 'r--', lw=2)\n", 405 | "plt.axis([-5, 5, -3, 3])\n", 406 | "plt.title('Three samples from the GP posterior')\n", 407 | "plt.show()" 408 | ] 409 | }, 410 | { 411 | "cell_type": "markdown", 412 | "metadata": {}, 413 | "source": [ 414 | "By conditioning on the 6 known data points, the posterior collapsed around the data points, as desired, since we know these are the true values. The uncertainty, shown by the grey region, becomes smaller near the data points and larger away from them." 415 | ] 416 | }, 417 | { 418 | "cell_type": "code", 419 | "execution_count": null, 420 | "metadata": { 421 | "collapsed": true 422 | }, 423 | "outputs": [], 424 | "source": [] 425 | } 426 | ], 427 | "metadata": { 428 | "kernelspec": { 429 | "display_name": "Python 3", 430 | "language": "python", 431 | "name": "python3" 432 | }, 433 | "language_info": { 434 | "codemirror_mode": { 435 | "name": "ipython", 436 | "version": 3 437 | }, 438 | "file_extension": ".py", 439 | "mimetype": "text/x-python", 440 | "name": "python", 441 | "nbconvert_exporter": "python", 442 | "pygments_lexer": "ipython3", 443 | "version": "3.6.3" 444 | } 445 | }, 446 | "nbformat": 4, 447 | "nbformat_minor": 2 448 | } 449 | --------------------------------------------------------------------------------