├── .gitignore ├── LICENSE ├── README.md └── genadv1.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | .ipynb_checkpoints 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | genadv_tutorial is licensed under the Simplified "2-clause" BSD License: 2 | 3 | Copyright (c) 2015: Eric Jang. 4 | 5 | Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 6 | 7 | Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 8 | Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 9 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 10 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # genadv_tutorial 2 | 3 | Tutorial on Generative Adversarial Models. See the [blog post](http://blog.evjang.com/2016/06/generative-adversarial-nets-in.html). 4 | 5 | Eric Jang 6 | 7 | 30 Dec 2015 8 | 9 | License: BSD Clause 2 10 | -------------------------------------------------------------------------------- /genadv1.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Generative Adversarial Nets\n", 8 | "\n", 9 | "Training a generative adversarial network to sample from a Gaussian distribution. This is a toy problem, takes < 3 minutes to run on a modest 1.2GHz CPU." 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 1, 15 | "metadata": {}, 16 | "outputs": [ 17 | { 18 | "name": "stderr", 19 | "output_type": "stream", 20 | "text": [ 21 | "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\h5py\\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n", 22 | " from ._conv import register_converters as _register_converters\n" 23 | ] 24 | } 25 | ], 26 | "source": [ 27 | "import tensorflow as tf\n", 28 | "import numpy as np\n", 29 | "import matplotlib.pyplot as plt\n", 30 | "import seaborn as sns # for pretty plots\n", 31 | "from scipy.stats import norm\n", 32 | "%matplotlib inline" 33 | ] 34 | }, 35 | { 36 | "cell_type": "markdown", 37 | "metadata": {}, 38 | "source": [ 39 | "Target distribution $p_{data}$" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 2, 45 | "metadata": {}, 46 | "outputs": [ 47 | { 48 | "data": { 49 | "text/plain": [ 50 | "[]" 51 | ] 52 | }, 53 | "execution_count": 2, 54 | "metadata": {}, 55 | "output_type": "execute_result" 56 | }, 57 | { 58 | "data": { 59 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl0FOeZ7/Hvo24taEVoAySBQIhFBhuMAAMG7za2E7Cd2MGeZJzYGY9nwiQzSU7iTHKdHOfOnUwykziTONfxZLlZvITYSYY42Hhf2XdbbFrYBAi0gSSEtu7n/tEtT1sI1IJuVS/P5xwduqre6n76IP26+q236hVVxRhjTHxIcLoAY4wxw8dC3xhj4oiFvjHGxBELfWOMiSMW+sYYE0cs9I0xJo5Y6BtjTByx0DfGmDhioW+MMXHE7XQB/eXm5mpJSYnTZRhjTFTZsmVLo6rmDdYu4kK/pKSEzZs3O12GMcZEFRE5GEw7694xxpg4YqFvjDFxxELfGGPiiIW+McbEkaBCX0SWiMheEakWkYfO0+7jIqIiUhGw7mv+/faKyE2hKNoYY8yFGXT0joi4gMeAG4A6YJOIrFLVXf3aZQCfBzYErCsHlgOXAGOBV0Rksqp6QvcWjDHGBCuYI/25QLWq1qpqN/AMsGyAdt8Gvgt0BqxbBjyjql2quh+o9j+fMcYYBwQzTr8QOBywXAfMC2wgIrOAYlV9XkS+3G/f9f32LbzAWk2cq2loZ21NEy2nuxmdlcJVk/MoyExxuixjokowoS8DrPtgYl0RSQB+AHx6qPsGPMcDwAMA48aNC6IkE0/qWjr41qpdvLL7+IfWuxKE5XOK+cqSqWSNSHSoOmOiSzChXwcUBywXAUcDljOA6cAbIgIwGlglIkuD2BcAVX0CeAKgoqLCZmo3H1hb3cjfPbkVj1f5p+snc8flhRRkpnCg6TS/XX+QpzYc4t3qRn527xwm5ac7Xa4xEU9Uz5+xIuIG9gHXAUeATcA9qlp5jvZvAF9W1c0icgnwFL5+/LHAq0DZ+U7kVlRUqN2GwQC8W93Iff9vEyU5afz0U7MpyU07q83mA808+NstiAi//9v5A7YxJh6IyBZVrRis3aAnclW1F1gBrAF2AytVtVJEHvEfzZ9v30pgJbALeBH4nI3cMcGobWjnwd9soSQnjacfuOKcYV5RMoqn/+YKPF7l3l9upLWzZ5grNSa6DHqkP9zsSN909nhY+uN3aGjrYtWKKykelTroPpsPNLP8ifVcNy2fxz85G39XozFxI2RH+sYMtx+9VsW+4+18/xMzgwp88B3xf2XJFNZUHmfVjrNOGxlj/Cz0TUR5/8gpHn+zljtnF3HNlPwh7Xv/lROZWTySR/68i5bT3WGq0JjoZqFvIoaq8sjzuxg5IpFv3Fo+5P1dCcJ3PjaDU2d6+N5Le8NQoTHRz0LfRIyXdx1n4/5m/vGGyWSlXti4+6mjM7ln3jh+t+kwtQ3tIa7QmOhnoW8igser/NuLeyjNS2P5nOLBdziPf7i2jGR3Av9uR/vGnMVC30SEF94/Rk3Dab504xQSXRf3a5mXkcxnF01k9Xv17D7WGqIKjYkNFvrGcarKT16vYWJuGjddMjokz3nfwhJSk1z811u1IXk+Y2KFhb5x3FtVjew61sqDV5XiSgjN+PqRqUksnzOOVTuOcuTkmZA8pzGxwELfOO7xN2oYk5XCbbNCewPW+xdNAODnb+8P6fMaE80s9I2jqo63sa62ib+eX0KSO7S/joUjR/DRy8byu02HaO/qDelzGxOtLPSNo57ccIgkVwJ3VRSF5fk/NX88p7s9rNpuV+kaAxb6xkGnu3p5bksdt8wYTU56clheY1bxSKaNyeTJDQeJtPtMGeMEC33jmFU7jtLW1cun5o8P22uICPfMG0fl0VZ21J0K2+sYEy0s9I1jfrfpMFMKMrh8XHZYX+e2mWNJTXLx1IaDYX0dY6KBhb5xRE1DO9sPn+RjswvDfhvkjJREbp0xhtXv1dPZY9M5mPhmoW8c8adtR0gQWDYztMM0z+X2WYW0d/Xy8q7jgzc2JoYFFfoiskRE9opItYg8NMD2B0XkPRHZLiLviEi5f32JiJzxr98uIo+H+g2Y6OP1Kn/cdoSFk3IpyEwZltecNzGH0Zkp/GnbkWF5PWMi1aChLyIu4DHgZqAcuLsv1AM8paozVHUm8F3g+wHbalR1pv/nwVAVbqLXpgPN1LWc4Y7Lh+coH3y3XV42ayxv7mugqb1r2F7XmEgTzJH+XKBaVWtVtRt4BlgW2EBVA+9qlQbY2DhzTn/cdoTUJFfI7rMTrNtnFdLrVZ7feWxYX9eYSBJM6BcChwOW6/zrPkREPiciNfiO9D8fsGmCiGwTkTdFZNFALyAiD4jIZhHZ3NDQMITyTbTp8Xh54f16biwvIDXJPayvPXV0JlNHZ/Cn7dbFY+JXMKE/0NCKs47kVfUxVS0Fvgp8w7/6GDBOVWcBXwSeEpHMAfZ9QlUrVLUiLy8v+OpN1FlX08SpMz3cMmOMI6//0cvGsu3QSY6dspuwmfgUTOjXAYGzWhQB57um/RngNgBV7VLVJv/jLUANMPnCSjWx4IX360lLcrF4sjMf7kum+7qUXny/3pHXN8ZpwYT+JqBMRCaISBKwHFgV2EBEygIWbwWq/Ovz/CeCEZGJQBlgNziPU70eLy9V1nPttAJSEl2O1FCal87kgnResNA3cWrQ0FfVXmAFsAbYDaxU1UoReURElvqbrRCRShHZjq8b517/+sXAThHZATwLPKiqzSF/FyYqbDzQTNPpbm6ePrwncPtbMn0Mmw4009Bmo3hM/AnqTJqqrgZW91v3cMDjL5xjv+eA5y6mQBM7XnivnpTEBK6e4ux5m5unj+Y/X63ipV31/NW88N33x5hIZFfkmmHh8SovVtZzzZT8YR+109/U0RmU5KRav76JSxb6ZlhsO9RCQ1vXBydSnSQiLJk+xjeSqKPH6XKMGVYW+mZYvLrnBO4E4eop+U6XAvhG8fR6ldf22r14THyx0DfD4rXdJ5hTMoqsEYlOlwLApYVZ5KYn89oeuxjQxBcLfRN2h5s72Hu8jeumRcZRPkBCgnDNlDze3HuCHo/X6XKMGTYW+ibsXttzAoBrp0ZO6ANcNy2f1s5ethxscboUY4aNhb4Ju1f3nGBibhoT89KdLuVDrizLI9ElvO7/UDImHljom7A63dXL+pqmiDvKB0hPdnPFxBxetdA3ccRC34TVO9WNdHu8XBtB/fmBrpmST/WJdg41dThdijHDwkLfhNWru4+TkeJmTskop0sZUN/J5df22NBNEx8s9E3YeL3K63sbWDw5j0RXZP6qjc9JozQvzbp4TNyIzL9EExP21LfR0NbFNRFyQda5XDs1nw21zXR09zpdijFhZ6FvwuatKt+FT4vKch2u5PyumpxPt8fLhlq7AayJfRb6Jmzermpg6ugMCjJTnC7lvCpKskl2J3zwIWVMLLPQN2HR0d3Lpv0tEX+UD5CS6GLexBzermp0uhRjws5C34TFhv3NdHu8jk2LOFSLy3KpPtHO0ZM2d66JbUGFvogsEZG9IlItIg8NsP1BEXlPRLaLyDsiUh6w7Wv+/faKyE2hLN5Errf2NZDsTojYoZr9LSrzfTi9bV08JsYNGvr+OW4fA24GyoG7A0Pd7ylVnaGqM4HvAt/371uOb07dS4AlwE/65sw1se3tqkbmTcxxbC7coZpckE5BZjJvWRePiXHBHOnPBapVtVZVu4FngGWBDVS1NWAxDVD/42XAM6rapar7gWr/85kYdvTkGapPtLM4Cvrz+4gIi8ryeLe6EY9XB9/BmCgVTOgXAocDluv86z5ERD4nIjX4jvQ/P8R9HxCRzSKyuaHBvl5Hu74ukmjpz++zqCyXkx09vH/klNOlGBM2wYS+DLDurEMhVX1MVUuBrwLfGOK+T6hqhapW5OVFV1CYs721r5HRmSmU5UfWXTUHc+Uk3zcT69c3sSyY0K8DigOWi4Cj52n/DHDbBe5ropzHq7xT3ciislxEBvrMj1w56clML8y0fn0T04IJ/U1AmYhMEJEkfCdmVwU2EJGygMVbgSr/41XAchFJFpEJQBmw8eLLNpHqvSOnOHWmh0VR1rXTZ1FZHlsPttDeZbdkMLFp0NBX1V5gBbAG2A2sVNVKEXlERJb6m60QkUoR2Q58EbjXv28lsBLYBbwIfE5VPWF4HyZCvFvtO0peWJrjcCUXZlFZLr1eZV1Nk9OlGBMW7mAaqepqYHW/dQ8HPP7Cefb9F+BfLrRAE13W1TQxdXQGOenJTpdyQWaPzyYlMYG1NY3cUF7gdDnGhJxdkWtCpqvXw6YDzSwojZ6hmv0lu13MKRnF2mo70jexyULfhMy2Qyfp6vUyP0q7dvrML81h73HfbaGNiTUW+iZk1tY0kSAwd0J03HrhXPq+qayvtaN9E3ss9E3IrKtpZEZhFlkjEp0u5aJMH5tJRoqbtTU2dNPEHgt9ExId3b1sO3SS+VHcn9/H7Upg3oQc1toIHhODLPRNSGw60EKvV1kQ5f35fRaU5nCwqYO6lg6nSzEmpCz0TUisrWkk0SVUlGQ7XUpILPTfksGO9k2ssdA3IbG+polZxdmkJgV16UfEm1yQTk5akl2kZWKOhb65aKfO9PDekVNRP1QzkIgwvzSHtTWNqNqtlk3ssNA3F23j/ma8Ssz05/dZOCmX461d1DScdroUY0LGQt9ctLU1jaQkJjBz3EinSwmpvg+xdTZ008QQC31z0dbVNFExfhTJ7uiYGjFY40alUjhyhJ3MNTHFQt9clMb2LvbUt8VUf34fEWFBaQ7rapvw2hSKJkZY6JuL0nergljrz++zYFIOJzt62HWsdfDGxkQBC31zUdbWNJGe7GZGYZbTpYRF3314bOimiRVBhb6ILBGRvSJSLSIPDbD9iyKyS0R2isirIjI+YJtHRLb7f1b139dEt3U1TcybMAq3KzaPHwoyUyjNS+NdO5lrYsSgf6ki4gIeA24GyoG7RaS8X7NtQIWqXgo8C3w3YNsZVZ3p/1mKiRnHTp1hf+PpmOzPD7SgNJeN+5vp8XidLsWYixbM4dlcoFpVa1W1G9/E58sCG6jq66rad5OS9fgmQDcxrq/LI5onTQnGgtIcOro97Kw76XQpxly0YEK/EDgcsFznX3cu9wMvBCyniMhmEVkvIrddQI0mQq2taSI7NZGpozOcLiWsrpjo+yZjs2mZWBBM6MsA6wYcvyYinwQqgO8FrB6nqhXAPcCjIlI6wH4P+D8YNjc0NARRknGaqm/y8PmlOSQkDPQrEjuy05IoH5Np4/VNTAgm9OuA4oDlIuBo/0Yicj3wdWCpqn4wz5yqHvX/Wwu8Aczqv6+qPqGqFapakZeXN6Q3YJxxqLmDIyfPMH9ibPfn91lQmsOWQy109nicLsWYixJM6G8CykRkgogkAcuBD43CEZFZwE/xBf6JgPXZIpLsf5wLLAR2hap445y+o95YmDQlGAsm5dDd62XroRanSzHmogwa+qraC6wA1gC7gZWqWikij4hI32ic7wHpwO/7Dc2cBmwWkR3A68B3VNVCPwasrWkiPyOZ0rw0p0sZFnNKRuFKEBuvb6JeUDc/V9XVwOp+6x4OeHz9OfZbC8y4mAJN5PH15zdy5aRcRGK7P79PRkoiMwqzWFvTxJecLsaYixCbV9SYsKo60U5je3fMD9Xsb0FpDjsOn6S9q9fpUoy5YBb6ZsjWVvuuTo31i7L6W1CaS69X2XSg2elSjLlgFvpmyNbWNFGUPYLiUalOlzKsZo/PJsmVYP36JqpZ6Jsh8XiV9bVNLIyzrh2AEUkuZo0byVq7D4+JYhb6Zkh2H2ultbOXBZPiq2unz4LSXCqPtnKyo9vpUoy5IBb6Zkj6jnLj5aKs/hZMykEV1tdav76JThb6ZkjW1jQxKT+d/MwUp0txxGVFIxmR6Ppg8hhjoo2Fvglaj8fLxv3NMTtLVjCS3AnMmTDK+vVN1LLQN0HbWXeSjm5PXIc++Mbr7zveTkNb1+CNjYkwFvomaGurmxCBeRPiO/T7zmessy4eE4Us9E3Q1tY0UT4mk+y0JKdLcdQlYzPJSHGzzrp4TBSy0DdB6ezxsOVQS9x37QC4XQnMm5Bj99c3UclC3wRl68EWunu9cXfrhXNZUJrDwaYO6lo6Bm9sTASx0DdBWVvThCtBmFMyyulSIkLfxWl2SwYTbSz0TVDW1jRyaVEWGSmJTpcSESbnZ5CTlmShb6KOhb4ZVHtXLzvqTll/foCEBOGKUl+/vuqAU0YbE5GCCn0RWSIie0WkWkQeGmD7F0Vkl4jsFJFXRWR8wLZ7RaTK/3NvKIs3w2PT/mY8Xo27++cPZkFpDvWtnRxosn59Ez0GDX0RcQGPATcD5cDdIlLer9k2oEJVLwWeBb7r33cU8E1gHjAX+KaIZIeufDMc1tY0kuRKYPZ4+68L1PchaFfnmmgSzJH+XKBaVWtVtRt4BlgW2EBVX1fVvsOd9UCR//FNwMuq2qyqLcDLwJLQlG6Gy9qaJi4fP5KURJfTpUSUkpxUxmSl2NBNE1WCCf1C4HDAcp1/3bncD7xwgfuaCNNyuptdx1qta2cAIsL80hzW1zTh9Vq/vokOwYT+QDNfD/gbLiKfBCqA7w1lXxF5QEQ2i8jmhoaGIEoyw2XD/iZUsZO45zB/Yg5Np7vZd6LN6VKMCUowoV8HFAcsFwFH+zcSkeuBrwNLVbVrKPuq6hOqWqGqFXl5ecHWbobBupomUpNcXFo00ulSIlLfxWprq62Lx0SHYEJ/E1AmIhNEJAlYDqwKbCAis4Cf4gv8EwGb1gA3iki2/wTujf51Jkq8U93InJJRJLltdO9AirJTGZ+Tav36JmoM+pesqr3ACnxhvRtYqaqVIvKIiCz1N/sekA78XkS2i8gq/77NwLfxfXBsAh7xrzNR4NipM9Q0nGZRmfXnn8+C0hw21DbR6/E6XYoxg3IH00hVVwOr+617OODx9efZ9xfALy60QOOct6t8QxGvtNA/r/mluTy98TCVR1u5rNi6wUxks+/s5pzeqWokLyOZKQUZTpcS0frur29dPCYaWOibAXm9yrvVjVw5KReRgQZhmT55GclMLki3i7RMVLDQNwPadayVptPdXDnJunaCsXBSLhv3N9PZ43G6FGPOy0LfDOidauvPH4rFk/Po6vWy6YCNUzCRzULfDOidqkYmF6RTkJnidClRYd6EUSS5Enhrn11caCKbhb45S2ePh40Hmrlykl0oF6zUJDdzJmR/MOLJmEhloW/OsulAM929XhufP0SLyvLYU9/G8dZOp0sx5pws9M1Z3qlqJNElzJtoUyMOxeIy3zcjO9o3kcxC35zlrapGZo/PJjUpqGv3jN/U0Rnkpidbv76JaBb65kMa2rrYfayVRWXWnz9UCQnC4rJc3qlutFstm4hloW8+pO8CIxuff2EWTc6l+XQ3lUdbnS7FmAFZ6JsPeXNvA9mpiUwvzHK6lKjUN+LprSrr4jGRyULffMDrVd7Y18BVk/NwJditFy5EXkYy5WMyrV/fRCwLffOBnUdO0Xy6m2um5jtdSlRbPDmPrYdaaO/qdboUY85ioW8+8MbeE4hgJ3Ev0uKyXHo8ynq766aJQBb65gOv721gZvFIRqUlOV1KVJtdkk1qkos39p0YvLExwyyo0BeRJSKyV0SqReShAbYvFpGtItIrIh/vt83jn03rgxm1TORpbO9iZ91JrpliXTsXK9ntYlFZLq/tPoGqDd00kWXQ0BcRF/AYcDNQDtwtIuX9mh0CPg08NcBTnFHVmf6fpQNsNxHgrX0NqMLVU6xrJxSunZrP0VOd7Klvc7oUYz4kmCP9uUC1qtaqajfwDLAssIGqHlDVnYBNEhql3tjbQG56EtPH2lDNUOj7xvTaHuviMZElmNAvBA4HLNf51wUrRUQ2i8h6EbltSNWZYeHxKm/ua+Cqyfkk2FDNkMjPTGFGYZaFvok4wYT+QCkwlI7KcapaAdwDPCoipWe9gMgD/g+GzQ0NNr55uG0/3MKpMz1cM9W6dkLp2qn5bD3UQvPpbqdLMeYDwYR+HVAcsFwEHA32BVT1qP/fWuANYNYAbZ5Q1QpVrcjLs+AZbm/sbSBBYJHdPz+krpuWjyq8aaN4TAQJJvQ3AWUiMkFEkoDlQFCjcEQkW0SS/Y9zgYXArgst1oTHq7tPcPm4bLJSE50uJaZMH5tFbnoyr+620DeRY9DQV9VeYAWwBtgNrFTVShF5RESWAojIHBGpA+4Efioilf7dpwGbRWQH8DrwHVW10I8gdS0d7DrWyg3lBU6XEnMSEoRrp+bx1r4Gejw2xsFEhqBumK6qq4HV/dY9HPB4E75un/77rQVmXGSNJoxe2XUcwEI/TK6dms/KzXVsOdjCFRNznC7HGLsiN969tOs4k/LTmZiX7nQpMenKsjwSXWKjeEzEsNCPY6c6etiwv9mO8sMoPdnNFRNzeGXXcbs610QEC/049tre43i8yo0W+mF14yWjqW08TfWJdqdLMcZCP569vOs4+RnJXFY00ulSYtpN5QWIwIvv1ztdijEW+vGqs8fDG3sbuL68wK7CDbP8zBQuH5fNi5UW+sZ5Fvpxal1NEx3dHuvPHyZLLhlN5dFWDjd3OF2KiXMW+nHqpV31pCW5WFBqwwiHw02XjAZgjR3tG4dZ6MehXo+Xl3cd5+op+SS7XU6XExfG5aRSPibT+vWN4yz049DG/c00tndz66VjnC4lriyZPpoth1o40drpdCkmjlnox6Hn3ztGapLLZskaZkumj0bVd0GcMU6x0I8zvR4vL75fz3XTChiRZF07w6ksP52JuWnWxWMcZaEfZ9bVNtF8uptbZ1jXznATEW6ZMYa1NY00tHU5XY6JUxb6ceYvO4+RluSyuXAdsmzmWLwKf9kZ9JQUxoSUhX4c6fF4ebGynhvKC0hJtK4dJ5QVZDB1dAardljoG2dY6MeRd6sbOdnRw62XjnW6lLi2bGYhWw+d5FCTXahlhp+Ffhz5845jZCS7WVSW63Qpce2jl/nOp/zZuniMA4IKfRFZIiJ7RaRaRB4aYPtiEdkqIr0i8vF+2+4VkSr/z72hKtwMTUd3Ly+8f4xbLx1jXTsOK8pOpWJ8Nqu2W+ib4Tdo6IuIC3gMuBkoB+4WkfJ+zQ4Bnwae6rfvKOCbwDxgLvBNEcm++LLNUK2prKej28Mdl581wZlxwNKZY9l7vI099a1Ol2LiTDBH+nOBalWtVdVu4BlgWWADVT2gqjuB/hOB3gS8rKrNqtoCvAwsCUHdZoj+sPUIxaNGUDHePnMjwS0zxuBKEP647YjTpZg4E0zoFwKHA5br/OuCEdS+IvKAiGwWkc0NDQ1BPrUJVv2pTt6pbuT2WUV2G+UIkZuezNWT8/jj1iP02qTpZhgFE/oDpUSw874Fta+qPqGqFapakZdn48dD7b+3H0EV7pgV7Ge1GQ53zSnmRFsXb+6zAx0zfIIJ/TqgOGC5CAj2DNTF7GtCQFV5bmsds8dnU5Kb5nQ5JsC1U/PJTU9i5ebDgzc2JkSCCf1NQJmITBCRJGA5sCrI518D3Cgi2f4TuDf615lhsv3wSfYdb+djdgI34iS6Erh9ViGv7j5BY7vdlsEMj0FDX1V7gRX4wno3sFJVK0XkERFZCiAic0SkDrgT+KmIVPr3bQa+je+DYxPwiH+dGSZPbThEWpKLpTPtgqxIdFdFMb1e5U92QtcME3cwjVR1NbC637qHAx5vwtd1M9C+vwB+cRE1mgt06kwPf955lNtnFZGeHNR/tRlmZQUZzBo3kt9tOsz9V05AxE60m/CyK3Jj2H9vP0Jnj5d75o5zuhRzHndVFFN1op2th046XYqJAxb6MUpVeWrDIWYUZjGjKMvpcsx5fPSysaQnu/nt+oNOl2LigIV+jNp66CR76tu4Z54d5Ue69GQ3H59dxPM7j9p99k3YWejHqF+tPUB6spuPXmYncKPBX88fT49HeXrjIadLMTHOQj8GHTt1htXvHeMTc4rtBG6UmJiXzuLJeTy54SA9doWuCSML/Rj0q7UH8ary6QUlTpdihuDTC8ZzvLWLNZU2h64JHwv9GNPR3cvTGw9x0yWjKR6V6nQ5ZgiunpzP+JxUfvHOflSDvdOJMUNjoR9jnttSx6kzPXx20QSnSzFDlJAgfPbKCWw9dJKN++0aRhMeFvoxxONVfv7Ofi4rHsnl4+wWytHozopictKS+L9v1jhdiolRFvox5PmdRznQ1MGDiyfalZ1RKiXRxX1XTuCNvQ1UHj3ldDkmBlnoxwivV/nRa9VMKcjgpktGO12OuQifvGI86cluHn+z1ulSTAyy0I8RL7xfT/WJdlZcO8kmSolyWSMS+asrxvGXnUepbWh3uhwTYyz0Y4DvKL+K0rw0bpkxxulyTAh89sqJJLtd/OCVKqdLMTHGQj8GvFhZz576Nv7h2jJcdpQfE/IykrnvyhL+vOOo9e2bkLLQj3I9Hi/ffXEPkwvS+cildpQfSx5YXEpmipt/X7PX6VJMDAkq9EVkiYjsFZFqEXlogO3JIvI7//YNIlLiX18iImdEZLv/5/HQlm+e3niIA00dPHTzVNwu+wyPJVkjEnnw6lJe39vApgM2bt+ExqApISIu4DHgZqAcuFtEyvs1ux9oUdVJwA+AfwvYVqOqM/0/D4aobgO0dfbww1equGLiKK6Zku90OSYMPr2ghLyMZP73X3bj9dpVuubiBXNoOBeoVtVaVe0GngGW9WuzDPiV//GzwHViA8XD7rHXa2g63c3Xbp5m4/JjVGqSm68umcqOwyd5dmud0+WYGBBM6BcChwOW6/zrBmzjn1P3FJDj3zZBRLaJyJsisugi6zV+1Sfa+NnbtXx8dhGXFY90uhwTRnfMKuTycSP57ot7OHWmx+lyTJQLJvQHOoTs/z3zXG2OAeNUdRbwReApEck86wVEHhCRzSKyuaGhIYiS4puq8r/+VElqkouHbp7qdDkmzBIShEeWTafpdDePvrLP6XJMlAsm9OuA4oDlIuDoudqIiBvIAppVtUtVmwBUdQtQA0zu/wKq+oSqVqgTHs0xAAALS0lEQVRqRV5e3tDfRZxZteMo62qb+MqSqeSmJztdjhkG0wuzuHvuOH697iDv1dkQTnPhggn9TUCZiEwQkSRgObCqX5tVwL3+xx8HXlNVFZE8/4lgRGQiUAbYteUX4URbJ99cVcllxSO52yY8jytfvWkquelJfPn3O+jutYlWzIUZNPT9ffQrgDXAbmClqlaKyCMistTf7OdAjohU4+vG6RvWuRjYKSI78J3gfVBVbezZBVJVvvbce5zp9vAfd15mF2LFmazURP71jhnsPd7Gj16zK3XNhQlqLj1VXQ2s7rfu4YDHncCdA+z3HPDcRdZo/H6/pY5X95zgG7dOY1J+utPlGAdcO7WAj11exE/eqOGaqfl2C20zZHY1T5SoOt7Gt1ZVMnfCKO5baBOkxLOHP1rOmKwU/uGpbZzs6Ha6HBNlLPSjwOmuXv7uya2kJrn40d2z7C6acS5rRCI/vudyTrR18uXf77SpFc2QWOhHOFXla394j9qGdn64fBYFmSlOl2QiwMzikXzt5mm8svs4j71e7XQ5JopY6Ee4R1+pYtWOo3zpxiksnJTrdDkmgnxmYQm3zRzLv7+0j+d39h9FbczALPQj2LNb6vjhq1XcObuIv7+61OlyTIQREb7zsUuZPT6bL63cwZaDLU6XZKKAhX6EWlNZz0PP7WThpBz+zx0z7N46ZkApiS6e+NRsRmel8JlfbuT9I3bhljk/C/0I9Mqu46x4aivTC7N4/JOzSbRbJpvzyElP5snPziM92c2nfr6BvfVtTpdkIpilSYRZteMof/fkFsrHZPLr++eSkZLodEkmChRlp/LU31xBkjuBu366js12/31zDhb6EUJV+a+3avn809uYVZzNr++bR6YFvhmCktw0nn1wAaPSkvirn23gpcp6p0syEchCPwJ0dPfypd/v4F9W7+bWGWP49f1zyUq1wDdDVzwqlWcfnM/U0Rn87W+38Ogr+/DY5CsmgIW+w6pPtHP7Y2v547YjfOG6Mn509yxSEl1Ol2WiWE56Mk8/cAW3zyzk0Veq+PQvN9LY3uV0WSZCWOg7pMfj5bHXq7nlP9+mob2LX31mLv90w2S72taERGqSm/+46zL+9Y4ZbNjfzPXff5M/bK2zq3dNcDdcM6H1bnUj335+F3vq27hlxmi+tfQS8jPsSlsTWiLC3XPHUTE+m68+t5MvrtzBH7cd4Z9vmca0MWfNZWTihETaJ39FRYVu3rzZ6TLCYuuhFn7w8j7ermqkcOQIHv5oOTddMtrpskwc8HiV36w7wA9eqaK1s4fbZxWy4ppJTMyzu7XGChHZoqoVg7az0A+v7l4vL+86zs/fqWXroZOMTE1kxTWT+OQV463v3gy7Ux09/OTNan757gF6PF6um5rPfQsncMXEHOtajHIW+g7yeJUtB1v40/YjrH7vGCc7ehifk8pnFpRwZ0UxacnWq2ac1dDWxW/WH+S36w/SfLqbwpEjuG3WWD5y6Vimjs6wK8CjUEhDX0SWAD8EXMDPVPU7/bYnA78GZgNNwCdU9YB/29eA+wEP8HlVXXO+14rG0Pd6lf1Np9m4v5m3qxp4t7qJU2d6GJHo4obyAm6bNZarJufbTFcm4nT2eHjh/WP8adtR3q5qwKswOjOFqybnsbAsl1nFIynKHmEfAlEgZKHvn+N2H3ADvgnQNwF3q+qugDZ/D1yqqg+KyHLgdlX9hIiUA08Dc4GxwCvAZFX1nOv1Ijn0vV6lvrWTA02nOdTUQU1DO+8dOcX7R1pp7+oFfH8wi8pyWTQ5j+um5ttRvYkaJ9o6eWNPA2/sO8HbVY20dfp+p3PTk5lZnMWk/AxK89IozU9nYm4aWSMS7cMgggQb+sEk0lygWlVr/U/8DLAM2BXQZhnwLf/jZ4Efi++3YRnwjKp2Afv9c+jOBdYF+0ZCweNVer1eej3q+/F66fUqnT0eTnd5ON3dy+mu3g89bunoobG9i8a2Lt+/7d0cb+2kK2BC6iR3AtPGZHL7rEJmFGZx+fiRlOal2x+CiUr5GSncNaeYu+YU0+PxsudYG9sPt7Dt8EneqzvFm/sa6PH8z0HiiEQXBZnJFGSmUJCZQm56Mpkj3GSmJJI5IpHMFDcZKYmkJrlITkwg2e0i2Z1AkjuBZLdvOdEl9vcyzIIJ/ULgcMByHTDvXG1UtVdETgE5/vXr++1beMHVnkfz6W5uevQtPF6lx+MLeI9X6fF6udDTFqPSkshNT/If6YykIDOZktw0SnLSGJ+TypisEdZlY2JSoiuBGUVZzCjK4lPzfet6PV4Ot5yh5kQ7B5pOU3+qk/rWTo63drLtcAstp3s++MY7FK4EwSWCyNmPE0RI8K9LEEhI8G0LJPzPirO39VsOaHDWX+559u3/wRSuv/qpYzL50d2zwvTsPsGE/kDvr3+MnqtNMPsiIg8ADwCMGzcuiJLOluxO4Pppvn5zd0ICiS7B5f/XnZCA2yW4EwS3K8H/r5DidpGW7CIt2U1qktv3OMlNWrKbjBS33d3SmABuVwITctOYkJt2zja9Hi/tXb20numltbOH1jM9nOnx0N3rpavXS1evx/dvj+9xd68Xjype9XWferz+x6p4tW9Z8Xrxtet3S4nApf5d1f2DJnDz2dvOs2+/xnp2hIVMcfaIsD13n2BCvw4oDlguAvpP09PXpk5E3EAW0BzkvqjqE8AT4OvTD7b4QGnJbv71jksvZFdjTIi4XQmMTE1iZGqS06WYcwjmUHYTUCYiE0QkCVgOrOrXZhVwr//xx4HX1PfRuQpYLiLJIjIBKAM2hqZ0Y4wxQzXokb6/j34FsAbfkM1fqGqliDwCbFbVVcDPgd/4T9Q24/tgwN9uJb6Tvr3A5843cscYY0x42cVZxhgTA4IdsmlnKo0xJo5Y6BtjTByx0DfGmDhioW+MMXHEQt8YY+JIxI3eEZEG4KDTdVyAXKDR6SKGmb3n+GDvOTqMV9W8wRpFXOhHKxHZHMxwqVhi7zk+2HuOLda9Y4wxccRC3xhj4oiFfug84XQBDrD3HB/sPccQ69M3xpg4Ykf6xhgTRyz0w0BEviwiKiK5TtcSbiLyPRHZIyI7ReSPIjLS6ZrCQUSWiMheEakWkYecrifcRKRYRF4Xkd0iUikiX3C6puEiIi4R2SYizztdSzhY6IeYiBTjm0T+kNO1DJOXgemqeimwD/iaw/WEnIi4gMeAm4Fy4G4RKXe2qrDrBb6kqtOAK4DPxcF77vMFYLfTRYSLhX7o/QD4CgNMCxmLVPUlVe2bGHU9vtnRYs1coFpVa1W1G3gGWOZwTWGlqsdUdav/cRu+EAzL/NaRRESKgFuBnzldS7hY6IeQiCwFjqjqDqdrcch9wAtOFxEGhcDhgOU64iAA+4hICTAL2OBsJcPiUXwHbV6nCwmXYObINQFE5BVg9ACbvg78M3Dj8FYUfud7z6r63/42X8fXJfDkcNY2TGSAdXHxTU5E0oHngH9U1Van6wknEfkIcEJVt4jI1U7XEy4W+kOkqtcPtF5EZgATgB0iAr5ujq0iMldV64exxJA713vuIyL3Ah8BrtPYHANcBxQHLBcBRx2qZdiISCK+wH9SVf/gdD3DYCGwVERuAVKATBH5rap+0uG6QsrG6YeJiBwAKlQ12m7aNCQisgT4PnCVqjY4XU84iIgb30nq64AjwCbgHlWtdLSwMBLfkcuvgGZV/Uen6xlu/iP9L6vqR5yuJdSsT99crB8DGcDLIrJdRB53uqBQ85+oXgGswXdCc2UsB77fQuBTwLX+/9ft/iNgE+XsSN8YY+KIHekbY0wcsdA3xpg4YqFvjDFxxELfGGPiiIW+McbEEQt9Y4yJIxb6xhgTRyz0jTEmjvx/vJJ2mkye+h4AAAAASUVORK5CYII=\n", 60 | "text/plain": [ 61 | "
" 62 | ] 63 | }, 64 | "metadata": {}, 65 | "output_type": "display_data" 66 | } 67 | ], 68 | "source": [ 69 | "mu,sigma=-1,1\n", 70 | "xs=np.linspace(-5,5,1000)\n", 71 | "plt.plot(xs, norm.pdf(xs,loc=mu,scale=sigma))\n", 72 | "#plt.savefig('fig0.png')" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": 3, 78 | "metadata": {}, 79 | "outputs": [], 80 | "source": [ 81 | "TRAIN_ITERS=10000\n", 82 | "M=200 # minibatch size" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": 4, 88 | "metadata": {}, 89 | "outputs": [], 90 | "source": [ 91 | "# MLP - used for D_pre, D1, D2, G networks\n", 92 | "def mlp(input, output_dim):\n", 93 | " # construct learnable parameters within local scope\n", 94 | " w1=tf.get_variable(\"w0\", [input.get_shape()[1], 6], initializer=tf.random_normal_initializer())\n", 95 | " b1=tf.get_variable(\"b0\", [6], initializer=tf.constant_initializer(0.0))\n", 96 | " w2=tf.get_variable(\"w1\", [6, 5], initializer=tf.random_normal_initializer())\n", 97 | " b2=tf.get_variable(\"b1\", [5], initializer=tf.constant_initializer(0.0))\n", 98 | " w3=tf.get_variable(\"w2\", [5,output_dim], initializer=tf.random_normal_initializer())\n", 99 | " b3=tf.get_variable(\"b2\", [output_dim], initializer=tf.constant_initializer(0.0))\n", 100 | " # nn operators\n", 101 | " fc1=tf.nn.tanh(tf.matmul(input,w1)+b1)\n", 102 | " fc2=tf.nn.tanh(tf.matmul(fc1,w2)+b2)\n", 103 | " fc3=tf.nn.tanh(tf.matmul(fc2,w3)+b3)\n", 104 | " return fc3, [w1,b1,w2,b2,w3,b3]" 105 | ] 106 | }, 107 | { 108 | "cell_type": "code", 109 | "execution_count": 5, 110 | "metadata": {}, 111 | "outputs": [], 112 | "source": [ 113 | "# re-used for optimizing all networks\n", 114 | "def momentum_optimizer(loss,var_list):\n", 115 | " batch = tf.Variable(0)\n", 116 | " learning_rate = tf.train.exponential_decay(\n", 117 | " 0.001, # Base learning rate.\n", 118 | " batch, # Current index into the dataset.\n", 119 | " TRAIN_ITERS // 4, # Decay step - this decays 4 times throughout training process.\n", 120 | " 0.95, # Decay rate.\n", 121 | " staircase=True)\n", 122 | " #optimizer=tf.train.GradientDescentOptimizer(learning_rate).minimize(loss,global_step=batch,var_list=var_list)\n", 123 | " optimizer=tf.train.MomentumOptimizer(learning_rate,0.6).minimize(loss,global_step=batch,var_list=var_list)\n", 124 | " return optimizer" 125 | ] 126 | }, 127 | { 128 | "cell_type": "markdown", 129 | "metadata": {}, 130 | "source": [ 131 | "# Pre-train Decision Surface\n", 132 | "\n", 133 | "If decider is reasonably accurate to start, we get much faster convergence." 134 | ] 135 | }, 136 | { 137 | "cell_type": "code", 138 | "execution_count": 6, 139 | "metadata": {}, 140 | "outputs": [], 141 | "source": [ 142 | "with tf.variable_scope(\"D_pre\"):\n", 143 | " input_node=tf.placeholder(tf.float32, shape=(M,1))\n", 144 | " train_labels=tf.placeholder(tf.float32,shape=(M,1))\n", 145 | " D,theta=mlp(input_node,1)\n", 146 | " loss=tf.reduce_mean(tf.square(D-train_labels))" 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": 7, 152 | "metadata": {}, 153 | "outputs": [], 154 | "source": [ 155 | "optimizer=momentum_optimizer(loss,None)" 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "execution_count": 8, 161 | "metadata": {}, 162 | "outputs": [], 163 | "source": [ 164 | "sess=tf.InteractiveSession()\n", 165 | "tf.global_variables_initializer().run()" 166 | ] 167 | }, 168 | { 169 | "cell_type": "code", 170 | "execution_count": 9, 171 | "metadata": {}, 172 | "outputs": [], 173 | "source": [ 174 | "# plot decision surface\n", 175 | "def plot_d0(D,input_node):\n", 176 | " f,ax=plt.subplots(1)\n", 177 | " # p_data\n", 178 | " xs=np.linspace(-5,5,1000)\n", 179 | " ax.plot(xs, norm.pdf(xs,loc=mu,scale=sigma), label='p_data')\n", 180 | " # decision boundary\n", 181 | " r=1000 # resolution (number of points)\n", 182 | " xs=np.linspace(-5,5,r)\n", 183 | " ds=np.zeros((r,1)) # decision surface\n", 184 | " # process multiple points in parallel in a minibatch\n", 185 | " for i in range(r//M):\n", 186 | " x=np.reshape(xs[M*i:M*(i+1)],(M,1))\n", 187 | " ds[M*i:M*(i+1)]=sess.run(D,{input_node: x})\n", 188 | "\n", 189 | " ax.plot(xs, ds, label='decision boundary')\n", 190 | " ax.set_ylim(0,1.1)\n", 191 | " plt.legend()" 192 | ] 193 | }, 194 | { 195 | "cell_type": "code", 196 | "execution_count": 10, 197 | "metadata": {}, 198 | "outputs": [ 199 | { 200 | "data": { 201 | "text/plain": [ 202 | "Text(0.5,1,'Initial Decision Boundary')" 203 | ] 204 | }, 205 | "execution_count": 10, 206 | "metadata": {}, 207 | "output_type": "execute_result" 208 | }, 209 | { 210 | "data": { 211 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8VFXex/HPL5OZdBIgoZcECQEkdAFFUKyIiGIB7G1h1XXtupZ117L62Nb2WOHZxbIIVhQQ6wqCoihdpIYeagADJCQh5Tx/3EkcQsokmclN7vzer9e8ptwz9/6GkG/OnHvvuWKMQSmllLOE2V2AUkqpwNNwV0opB9JwV0opB9JwV0opB9JwV0opB9JwV0opB9JwVxUSkc9E5Ooqlr8mIg/6ua65IvKHwFXnHxG5XES+9KOd35+lsRKRh0TkP3bXoeqPhnsIEZHNInKGP22NMecYY970vu8aEfmu3PIbjDGPBqCmh0SkUEQOeW/rROQlEWld13UbY6YYY87yo11APkt5Pp8tx3tbLSIXBXo7SlVEw101BO8aY+KAZsBooBWwOBAB3wC8a4yJNcbEArcB/xGRlnYXVRMi4rK7BlVzGu4hqrQ3LiLPiMhvIrJJRM7xWT5XRP4gIt2A14ATvb3PbO/yN0TkH97HTUVklohkedc1S0Ta1bQmY0yhMeZXYCyQBdzpU89IEVkmItkiskBEevosay8iH3m3v09EXvL9jN7HIiLPicgeETkgIitEpEf5z+J9Pl5EMkRkv4jMEJE2PsuMiNwgIuu9n/VlERE/P98XwCHguOq2JSLJ3m2F+7QtG97y4+eXIiLfer8NfQUk+tYiIu+LyC7vv8U8ETneZ9kbIvKqiMwWkVzgDhHZXa6Wi0RkmT+fW9lDwz20DQTWYv3iPwX8q3xQGWNWAzcAP3h7oAkVrCcMmAx0BDoAecBLtS3KGFMMfAIMARCRvsC/gT8CzYHXgRkiEuHtVc4CtgDJQFtgWgWrPQsYCnQBErD+gOwr30hETgP+BxgDtPaut/z6RgInAL287c6u7jN5/7icC3iAVTXYVlWq+vm9Ayz2LnsUKL//5DMgFWgBLAGmlFt+GfAYEAf8L9a/1Zk+y68A3q5BraqeabiHti3GmEneMH0TK2BqPGRgjNlnjPnQGHPYGHMIKxROqWNtO7CGaQDGA68bYxYaY4q9+wIKgEHAAKANcLcxJtcYk2+M+a6C9RViBVVXQIwxq40xOytodznwb2PMEmNMAXAf1reWZJ82Txhjso0xW4E5QO8qPscY77edXGAG8LgxJrsG26pKhT8/EemA9cfnQWNMgTFmHjDT943GmH8bYw55t/sQ0EtE4n2afGKM+d4YU2KMyfeu/woAEWmG9QftHT/rVDbQcA9tu0ofGGMOex/G1nQlIhItIq+LyBYROQjMAxLqOFbbFtjvfdwRuNM7JJPtDcv2WKHeHivkiqpamTHmG6xvEy8Du0Vkoog0qaBpG6wedOn7crB6rW192uzyeXyYqv/N3jPGJBhjorGGY64SkT/WYFtVqezn1wb4zRiT69O2bDsi4hKRJ0Rkg/fntdm7yHfoZlu5bf0HOE9EYrG+acyv5I+jaiA03JU/qps69E4gDRhojGmCNfwB4NdYdHkiEgacB8z3vrQNeMwbkqW3aGPMVO+yDr7jwZV+CGNeNMb0A47HGp65u4JmO7D+mJTWEoM1FLS9Np+l3PY3Yw2HnOfHtkqDOdpnFa383NROoKl3faU6+Dy+DDgfOAOIxxrOgqN/Xkf9zI0x24EfsHZ4X4kOyTR4Gu7KH7uBdiLiqWR5HNY4e7b3K/vfa7MREXF7d+BOxQqyZ72LJgE3iMhA79h1jIicKyJxwE9YYfaE9/VIERlcwbpP8L7fjRWc+UBxBWW8A1wrIr1FJAJ4HFjoDeY6EWsn83Dg1+q2ZYzJwgr5K7w97evw2RFbFWPMFmAR8LCIeETkZH7/gwLWz6sA61tCtHe7/ngLuAdIB6b7+R5lEw135Y9vsAJpl4jsrWD580AUsBf4Efi8husfKyI5QDbWuPQ+oJ8xZgeAMWYR1rj7S8BvQAZwjXdZMVZwdQa2AplYO0vLa4L1R+I3rCGKfcAz5RsZY/4LPAh8iPVH4zhgXA0/zzGfzfv5fga+Bx72c1vjsb5d7MP6trGgBtu9DGuH636sP7Zv+Sx7C+vfYDvWzt0f/VzndKxvGtPLDfmoBkj0Yh1KKX+JyAbgj8aYr+2uRVVNe+5KKb+IdXatwfompxq4andCKaWUiMwFugNXGmNKbC5H+UGHZZRSyoF0WEYppRzItmGZxMREk5ycbNfmlVKqUVq8ePFeY0xSde1sC/fk5GQWLVpk1+aVUqpREpEt1bfSYRmllHIkDXellHIgDXellHIgPc5dqQassLCQzMxM8vPz7S5F1bPIyEjatWuH2+2u1fs13JVqwDIzM4mLiyM5ORk/L/ikHMAYw759+8jMzCQlJaVW69BhGaUasPz8fJo3b67BHmJEhObNm9fpG5uGu1INnAZ7aKrrz13DXSmlHEjDXSmlHEjDXSkVFHPnzmXkyJFVtlm2bBmzZ8+up4pCi4a7Uso2Gu7Bo4dCKtVIPDzzV1btOBjQdXZv04S/n3d8lW02b97M8OHDGThwIEuXLqVLly689dZbREdHH9P2888/57bbbiMxMZG+ffuWvf7TTz9x2223kZeXR1RUFJMnTyYlJYW//e1v5OXl8d1333HfffeRkpJyTLu0tLSAfuZQoT13pVS11q5dy4QJE1ixYgVNmjThlVdeOaZNfn4+48ePZ+bMmcyfP59du3aVLevatSvz5s1j6dKlPPLII9x///14PB4eeeQRxo4dy7Jlyxg7dmyF7VTtaM9dqUaiuh52MLVv357BgwcDcMUVV/Diiy9y1113HdVmzZo1pKSkkJqaWtZu4sSJABw4cICrr76a9evXIyIUFhZWuB1/26nqac9dKVWt8sdcV3YMdmWvP/jggwwbNoyVK1cyc+bMSk/O8bedql614S4i/xaRPSKyspLlIiIvikiGiKwQkb4VtVNKNV5bt27lhx9+AGDq1KmcfPLJx7Tp2rUrmzZtYsOGDWXtSh04cIC2bdsC8MYbb5S9HhcXx6FDh6ptp2rOn577G8DwKpafA6R6bxOAV+tellKqIenWrRtvvvkmPXv2ZP/+/dx4443HtImMjGTixImce+65nHzyyXTs2LFs2T333MN9993H4MGDKS4uLnt92LBhrFq1it69e/Puu+9W2k7VnF8XyBaRZGCWMaZHBcteB+YaY6Z6n68FTjXG7Kxqnf379zd6JSalqrZ69Wq6detmaw2bN29m5MiRrFxZ4Zd3FUQV/fxFZLExpn917w3EmHtbYJvP80zva8cQkQkiskhEFmVlZQVg00oppSoSiKNlKtqDUuHXAWPMRGAiWD33AGxbKRVkycnJx/TaR48ezaZNm4567cknn+Tss8+uz9JUFQIR7plAe5/n7YAdAVivUqqBmj59ut0lqGoEYlhmBnCV96iZQcCB6sbblVJKBVe1PXcRmQqcCiSKSCbwd8ANYIx5DZgNjAAygMPAtcEqVimllH+qDXdjzKXVLDfAnwJWkVJKqTrTM1SVUn576KGHeOaZZ2r13pNOOqnK5SNGjCA7O7tW6/Z1zTXX8MEHH9R5PTURGxtbr9vzh84to5SqFwsWLKhyeahM/WuMwRhDWFhw+9bac1dKVemxxx4jLS2NM844g7Vr15a9vmHDBoYPH06/fv0YMmQIa9asAWD37t2MHj2aXr160atXr7JQL+3d7ty5k6FDh9K7d2969OjB/PnzAeuQy7179wLw7LPP0qNHD3r06MHzzz8PWCdTdevWjfHjx3P88cdz1llnkZeXV2HNX3/9NUOGDKFLly7MmjULsGatvPbaa0lPT6dPnz7MmTMHsKY5uPnmm8veO3LkSObOnVtW8wMPPECvXr0YNGgQu3fvBmDTpk2ceOKJnHDCCTz44INl783JyeH000+nb9++pKen88knnxxV+0033UTfvn159NFHuf3228veN2nSJO64444a/2yqoj13pRqLz+6FXb8Edp2t0uGcJypdvHjxYqZNm8bSpUspKiqib9++9OvXD4AJEybw2muvkZqaysKFC7npppv45ptvuOWWWzjllFOYPn06xcXF5OTkHLXOd955h7PPPpsHHniA4uJiDh8+fMw2J0+ezMKFCzHGMHDgQE455RSaNm3K+vXrmTp1KpMmTWLMmDF8+OGHXHHFFcfUvXnzZr799ls2bNjAsGHDyMjI4OWXXwbgl19+Yc2aNZx11lmsW7euyn+e3NxcBg0axGOPPcY999zDpEmT+Otf/8qtt97KjTfeyFVXXVW2XrCmYJg+fTpNmjRh7969DBo0iFGjRgHWtMmTJ0/mlVdeITc3l549e/LUU0/hdruZPHkyr7/+epW11JSGu1KqUvPnz2f06NFlF+YoDaqcnBwWLFjAJZdcUta2oKAAgG+++Ya33noLAJfLRXx8/FHrPOGEE7juuusoLCzkggsuoHfv3kct/+677xg9ejQxMTEAXHjhhcyfP59Ro0aRkpJS1r5fv35s3ry5wrrHjBlDWFgYqampdOrUiTVr1vDdd9/x5z//GbAmOevYsWO14e7xeMouFdivXz+++uorAL7//ns+/PBDAK688kr+8pe/ANaQy/3338+8efMICwtj+/btZb39jh07MmjQIABiYmI47bTTmDVrFt26daOwsJD09PQqa6kpDXelGosqetjBVNE0viUlJSQkJLBs2bIar2/o0KHMmzePTz/9lCuvvJK7776bq666qmx5VfNdRURElD12uVyVDstUNEVxZesNDw+npKSk7LnvNMNut7tsXS6Xi6Kiokq3ATBlyhSysrJYvHgxbreb5OTksvWV/rEq9Yc//IHHH3+crl27cu21gT+CXMfclVKVGjp0KNOnTycvL49Dhw4xc+ZMAJo0aUJKSgrvv/8+YAXy8uXLATj99NN59VVrctji4mIOHjz60oBbtmyhRYsWjB8/nuuvv54lS5Ycs82PP/6Yw4cPk5uby/Tp0xkyZEiN6n7//fcpKSlhw4YNbNy4kbS0NIYOHcqUKVMAWLduHVu3biUtLY3k5GSWLVtGSUkJ27Zt46effqp2/YMHD2batGkAZesEa8riFi1a4Ha7mTNnDlu2bKl0HQMHDmTbtm288847XHpplUec14r23JU9jIHcvbAvAw5uh5w9kJtl3YoKoKQITAlENoHIBEjoAC26W2PEkU3srj5k9O3bl7Fjx9K7d286dux4VMhOmTKFG2+8kX/84x8UFhYybtw4evXqxQsvvMCECRP417/+hcvl4tVXX+XEE08se9/cuXN5+umncbvdxMbGlg3h+G7zmmuuYcCAAYDVw+3Tp0+lQzAVSUtL45RTTmH37t289tprREZGctNNN3HDDTeQnp5OeHg4b7zxBhEREQwePJiUlBTS09Pp0aPHUdd+rcwLL7zAZZddxgsvvMBFF11U9vrll1/OeeedR//+/enduzddu3atcj1jxoxh2bJlNG3a1O/P5i+/pvwNBp3yN4Qc3g87l8OuFbBrpRXo+zZAwYGj24WFQ3QiuKPA5bZeyz8Iefuh+Ii3jRs6ngTpF0P6JVZbB2sIU/6q4Bk5ciS33347p59+eoXL6zLlr/bcVeAYAwcyrRDfueL3+4OZv7dp0hYSU6HnJdC8s3VL6AAxSVYPvaJjf42BgztgzyrYPB/WzIYZf4avH4bTH4Q+V0KYq/4+p1J1lJ2dzYABA+jVq1elwV5XGu6q9g7tgu1LYMfS32+HreOUkTBongodT4RWPa3hlNa9ILpZzbcjAvFtrVvqmXDGw7D5O5jzOMy8FVa8D5dMhtgWgf18SgVJQkJCtUfq1JWGu/JP7t6jQ3zHUjjknfxTwiCpG6QNh9a9rVvL48ETHZxaRCBlCCTPhmVT4NO7YNJpcPVMaJYSnG3ayBhT6YWnlXPVdchcw10dzRhrB+eulbB7pTVWvmMZHNjqbSDWsErKUGjTF9r0sXrlwQryqohAnyugZQ94+wJ4YySM/y/Etar/WoIkMjKSffv20bx5cw34EGKMYd++fURGRtZ6HbpDNZQV5sGe1bD7VyvISwM932fypqbJv4d4mz7W0EpDPFpl5wr499nWH5qrZ0G4x+6KAqKwsJDMzMyjjr1WoSEyMpJ27drhdruPel13qCpL6c7Ifeth73rrSJW93scHtlF2RUR3DLTsDsePtoZUWqVbhx42xCCvSOuecP7L8MG1MP+fMOw+uysKCLfbTUqK84aaVPBpuDd2pceLH9gK2aW3bT6Pt0Jh7u/t3TGQ2Bk6DITmV0CLrtawRtOUio9UaUx6XAhrPoXvnrUOlUxMtbsipWyj4d7QFBdBwUHI+w3yD3hv2ZCTBTm7vbc9kLvHus/ZAyWFR68jMh7iO0CzTtDpFOtww8RUSOwCca2tsWqnOvtxWP8VfP0QjJtSbXOlnKrxhfsPr8Ccx8Dl8bm5ITzCuj/qdd9lHp/lPm3DPdbJMxLmc3N576Xc6z43AFMMJcW/3/s+NsXWWZYlJVCUB4X53nvvrSj/98eFh62TdfKz4UhO5Z9dwiCmhXXIX2wLa9gktgXEtrKOFU9oD/HtISqhfn4WDVFcSxh0A3z7pLUPoVUPuytSyhaNL9xb9YC+V1tnLB5zK7ROXS8uhCO5ULzfely6vMinXfERKC6on5rD3NaZlO4oCI/0eRxljWnHtbJO4ImMt25RPo9LbzEtrGPE9WSd6g28weoEzP+ndfy7UiGo8YV7ylDrFgjGWL3r4kLAeHvcJd6b8XlcwQ1j9fDDXN6ev/exhHnvva+HuTSQ61t0M+h7Ffz0ujVspSc3qRDU+MI9kES8wzPu6tuqxqXf1fDjy7DsHTj5NrurUareNfLDI5SqRFIatB8Iv7xvdyVK2ULDXTnX8aOtk7L2ZthdiVL1TsNdOVc365JwrPrY3jqUsoGGu3Ku+LbWlAkZX9tdiVL1TsNdOVunYbDtJ+s8AqVCiIa7crbjTrNOKNv8nd2VKFWvNNyVs7UfaJ0stmme3ZUoVa803JWzhXugbV/IrP6K9ko5iYa7cr72A6yLjhTm2V2JUvVGw105X/uB1jQTO5baXYlS9cavcBeR4SKyVkQyROTeCpZ3EJE5IrJURFaIyIjAl6pULbU7wbrP1Ct/qdBRbbiLiAt4GTgH6A5cKiLdyzX7K/CeMaYPMA54JdCFKlVrMYkQ18Y6W1WpEOFPz30AkGGM2WiMOQJMA84v18YApddjiwd2BK5EpQKgVTrs+sXuKpSqN/6Ee1tgm8/zTO9rvh4CrhCRTGA28OeKViQiE0RkkYgsysrKqkW5StVSqx6wd5110RSlQoA/4V7RNdlMueeXAm8YY9oBI4C3ReSYdRtjJhpj+htj+iclJdW8WqVqq1W6tVM1a43dlShVL/wJ90ygvc/zdhw77HI98B6AMeYHIBJIDESBSgVES+/l9vassrcOpeqJP+H+M5AqIiki4sHaYTqjXJutwOkAItINK9x13EU1HE2TrStj7V1vdyVK1Ytqw90YUwTcDHwBrMY6KuZXEXlERLxzqnInMF5ElgNTgWuMMeWHbpSyj8sNTVNgn4a7Cg1+XWbPGDMba0ep72t/83m8Chgc2NKUCrDELtpzVyFDz1BVoSOxM+zfaF0IXSmH03BXoaN5KhQfgewtdleiVNBpuKvQ0byzdb9vo711KFUPNNxV6Gja0brXnrsKARruKnTEtgKXR8NdhQQNdxU6wsIgvj1kb7W7EqWCTsNdhZaEDhruKiRouKvQ0rQj/KbDMsr5NNxVaEnoAIf3wpFcuytRKqg03FVoSSg9YmZb1e2UauQ03FVoiWtt3R/S68koZ9NwV6GliTfcD+60tw6lgkzDXYUW7bmrEKHhrkKLOwqimmrPXTmehrsKPXFt4JCGu3I2DXcVepq0hoM6LKOcTcNdhZ641tpzV46n4a5CT5M2kLMHigvtrkSpoNFwV6EnrhVgIFev4a6cS8NdhZ6YFta9hrtyMA13FXpikqz7HA135Vwa7ir0xCRa99pzVw6m4a5CT2zpsMwee+tQKog03FXo8cRCeKT23JWjabir0CNi7VTVMXflYBruKjTFJGrPXTmahrsKTTFJOuauHE3DXYWm2CTI3Wt3FUoFjYa7Ck0xSdawjDF2V6JUUGi4q9AU1QxKiqDgkN2VKBUUfoW7iAwXkbUikiEi91bSZoyIrBKRX0XkncCWqVSARTW17vN+s7cOpYIkvLoGIuICXgbOBDKBn0VkhjFmlU+bVOA+YLAx5jcRaRGsgpUKiOhm1n3eb9C0o721KBUE/vTcBwAZxpiNxpgjwDTg/HJtxgMvG2N+AzDG6GEIqmEr67nvt7cOpYLEn3BvC2zzeZ7pfc1XF6CLiHwvIj+KyPCKViQiE0RkkYgsysrSY4yVjaJ8eu5KOZA/4S4VvFb+EINwIBU4FbgU+D8RSTjmTcZMNMb0N8b0T0pKqmmtSgVOac/9sPbclTP5E+6ZQHuf5+2A8hegzAQ+McYUGmM2AWuxwl6phqlsWCbb3jqUChJ/wv1nIFVEUkTEA4wDZpRr8zEwDEBEErGGaTYGslClAircY00gpsMyyqGqDXdjTBFwM/AFsBp4zxjzq4g8IiKjvM2+APaJyCpgDnC3MWZfsIpWKiCimuoOVeVY1R4KCWCMmQ3MLvfa33weG+AO702pxiGqqfbclWPpGaoqdEU11R2qyrE03FXoim6mPXflWBruKnTpmLtyMA13FbqivD33khK7K1Eq4DTcVeiKagqmBI7ozJDKeTTcVegqnTxMd6oqB9JwV6FLp/1VDqbhrkJXRBPrvuCgvXUoFQQa7ip0RXrDPV/DXTmPhrsKXdpzVw6m4a5Cl/bclYNpuKvQpT135WAa7ip0hbmsaX+1564cSMNdhbaIJlBwwO4qlAo4DXcV2iKbaM9dOZKGuwptEU10zF05koa7Cm3ac1cOpeGuQpv23JVDabir0KY9d+VQGu4qtEU0gXw9WkY5j4a7Cm2RTaC4AIoK7K5EqYDScFehLSLeutehGeUwGu4qtEXqFATKmTTcVWgrnV9Gx92Vw2i4q9AW6R2W0Z67chgNdxXadNpf5VAa7iq06bS/yqE03FVoi4iz7gsO2VuHUgGm4a5CW2m4H8mxtw6lAkzDXYU2lxtcEVCg4a6cRcNdKU+M9tyV4/gV7iIyXETWikiGiNxbRbuLRcSISP/AlahUkEXEas9dOU614S4iLuBl4BygO3CpiHSvoF0ccAuwMNBFKhVUnjjtuSvHCfejzQAgwxizEUBEpgHnA6vKtXsUeAq4K6AVqpCQX1jMgg17WbhxPxuycsnKKQBjaB4bQUpiDANSmnFy50RiIvz5L1tDOiyjHMif35S2wDaf55nAQN8GItIHaG+MmSUilYa7iEwAJgB06NCh5tUqx8n87TD/N38THy3J5GB+ER5XGJ2SYkiKi0BE2HUgnwUb9vKv7zYR43ExqndbJgztREpiTOCKiIjVk5iU4/gT7lLBa6ZsoUgY8BxwTXUrMsZMBCYC9O/f31TTXDlYbkERL/53PZMXbMYYw/AerbmkXzsGpDQj0u06qm1BUTGLt/zGR0u2M31pJu8v2salAzpw19lpxEe5616MJxYO7qj7epRqQPwJ90ygvc/zdoDvb0Ic0AOYKyIArYAZIjLKGLMoUIUq51i85TfueG8ZW/cf5sI+7bjjrC60TYiqtH1EuIuTjkvkpOMSuWd4Gv/73wze+WkrX63azdOX9GRIalLdCoqIgyO5dVuHUg2MP0fL/AykikiKiHiAccCM0oXGmAPGmERjTLIxJhn4EdBgVxV6+4fNjHn9B4pLDNPGD+KfY3pVGezltYiL5NELevDRjScRE+Hiyn/9xEvfrMeYOnwR9MToGarKcaoNd2NMEXAz8AWwGnjPGPOriDwiIqOCXaByhuISw98+WcmDn/zKqV2S+OzWIQzs1LzW6+vVPoFPbxnCBb3b8MyX67h12jKOFJXUbmWeWGuHal3+QCjVwPh16IExZjYwu9xrf6uk7al1L0s5SVFxCbe/t5yZy3cwfkgK957TDVdYRbtyaibS7eK5sb3p0iqOpz5fS05BEa9c3veYMftqRcRCSREUH4HwiDrXpVRDoGeoqqAqLC7hlmlLmbl8B38Z3pUHzu0ekGAvJSLcdGpnHhvdgzlr93D9mz+TX1hcs5V4Yq17PZFJOYiGuwoaYwx/+XAFs3/ZxV/P7caNpx4XtG1dPrAjT1/ci+8z9nHL1KUUFddgiKY03I/ouLtyDg13FTRPf7GWj5Zs544zu/CHIZ2Cvr2L+7Xj7+d158tVu/nrxyv938kaoT135TxBON1PKXj7xy28MncDlw7owJ9P61xv2712cAp7cwp4ec4G2iREccvpqdW/qaznrodDKufQcFcB9+PGfTw041dO79qCR88/Hu/5D/XmrrPS2JGdz7NfraNb6yac2b1l1W/QYRnlQDosowJqR3Yef5qyhI7No3l+XG/CXfX/X0xE+J8L0+nZLp7b311Gxp5qQluHZZQDabirgMkvLOaG/yymoKiEiVf2Jy4yAFMD1FKk28VrV/Qj0h3G+LcWczC/sPLGOiyjHEjDXQXMo7NWsSLzAM+O6UXnFrF2l0ObhCheubwfW/cf5oHpVexg1UvtKQfScFcB8fnKXUxZuJUJQztx1vGt7C6nzICUZtxxZhdmLt/B+4syK27k8c4wqVMQKAfRcFd1tvNAHvd+tIL0tvHcdVaa3eUc44ZTjuOk45rz9xm/Vjz+Hh4BYW7tuStH0XBXdVJcYrj9XWtelxfG9cYT3vD+S7nChOfG9ibK4+Lmd5ZWfAZrRKyOuStHaXi/iapRee3bDfy4cT8PjTqeTkn2j7NXpmWTSP55SS/W7DrEs1+tO7aBR6+jqpxFw13V2tpdh3j+63Wc29O60EZDN6xrCy4b2IFJ8zeyaPP+oxd6YqBQe+7KOTTcVa0Ulxju+XAFcZFuHhlV/ycq1db9I7rRNiGKO99fzuEjRb8vcEfDkcP2FaZUgGmBBK7ZAAARxUlEQVS4q1qZ/P0mlm/L5qFRx9M8tvFMkxsbEc7TF/diy77DPPnZmt8XeGKgUMNdOYeGu6qxLftyeebLtZzRrQXn9Wxtdzk1duJxzbnmpGTe/GELCzL2Wi+6o3WHqnIUDXdVI8YY7v3wF9xhYTx6QY9GMxxT3l+GdyUlMYa7P1hBbkEReDTclbNouKsaeffnbfywcR/3jehG63j/r33a0ER5XDx1cU+2Z+fxzJdrwa3DMspZNNyV33YdyOexT1czqFMzxp3Q3u5y6uyE5GZcOagjbyzYTFaBS3vuylE03JVfjDH89eOVFJaU8MSFPQkL4KXy7HTP8DRaxkXyzcYcjPbclYNouCu/zFqxk69X7+bOM9NIToyxu5yAiYt0848LepCZI0jxESguqv5NSjUCGu6qWvtzj/DQjF/p1S6eawcn211OwJ3RvSUpbZIA2Lhjt83VKBUYGu6qWo/OWsWBvEKevLinLRffqA9n9rKu8fr4J0soKfHz2qtKNWDO/E1VATNnzR6mL93OTcM607VVE7vLCZq4uHgAMrbvZsrCLTZXo1TdabirSh3KL+SB6b/QpWUsfxp2nN3lBJcnGoDBHaJ54rM17MjOs7kgpepGw11V6snP17DzYD5PXtSTiHCX3eUEl9sK91uGtKHEwF8/ruLKTUo1AhruqkILN+7jPz9u5brBKfTp0NTucoLPezWmlpHF3HlWF75Zs4cZy3fYXJRStafhro6RX1jMvR/9Qodm0dx5Vhe7y6kf3p47hYe5dnAKvdon8PDMVezLKbC3LqVqScNdHeP5r9ezaW8uT1yYTrQn3O5y6kfpdVSPHMYVJjx1UU8O5RfyyKxV9talVC1puKuj/JJ5gEnzNzLuhPac1DnR7nLqT1nP3ZqCIK1VHH8a1plPlu3gv6v12HfV+PgV7iIyXETWikiGiNxbwfI7RGSViKwQkf+KSMfAl6qC7UhRCXd/sJzmMR7uG9HN7nLql/doGd8Ldtx0amfSWsbxwPSVHMovtKkwpWqn2nAXERfwMnAO0B24VES6l2u2FOhvjOkJfAA8FehCVfC9MjeDNbsO8fjodOKj3HaXU7/c3mEZn/llPOFhPHlxT/YcyucJ3wt7KNUI+NNzHwBkGGM2GmOOANOA830bGGPmGGNKfyt+BBr+BTXVUVbvPMhL32RwQe82nNG9pd3l1D9XOLg8x8wM2bt9AtcNTmHKwq38uHGfTcUpVXP+hHtbYJvP80zva5W5HvisogUiMkFEFonIoqysLP+rVEFVVFzCPR+sICHazd/PO97ucuxTyaX27jirCx2aRXPvhyvILyy2oTClas6fcK9obtcKz+4QkSuA/sDTFS03xkw0xvQ3xvRPSkryv0oVVBPnb+SX7Qd45PweNI3x2F2OfdwxFV4kO9oTzhMXprN532Ge+3qdDYUpVXP+hHsm4HtlhnbAMWd3iMgZwAPAKGOMHhzcSGTsOcTzX69nRHorRqQ3vuuhBpQnuuxomfJO6pzIuBPaM2neRpZvy67nwpSqOX/C/WcgVURSRMQDjANm+DYQkT7A61jBvifwZapgKB2Oifa4eHhUD7vLsZ87usKee6n7RnSjRVwkd7y3TIdnVINXbbgbY4qAm4EvgNXAe8aYX0XkEREZ5W32NBALvC8iy0RkRiWrUw3Ia99uYMnWbB4edTxJcRF2l2O/SsbcS8VHuXnmkl5syMrVo2dUg+fX6YfGmNnA7HKv/c3n8RkBrksF2YrMbJ7/ej0je7ZmVK82dpfTMLij4XDVR8ScnJrINScl88aCzZzRrSUnp4bQiV6qUdEzVENQ3pFibnt3GYmxETx2QToizrgeap15oqvsuZf6y/CuHJcUw90fLOdAnp7cpBomDfcQ9MRnq9mYlcs/x/QiPjrETlaqSiVHy5QX5XHx3NjeZB0q4O+frKyHwpSqOQ33EPPtuize/GEL1w1OYXAozR3jD080HMnxq2nPdgn8+bRUPl62g0+WbQ9yYUrVnIZ7CNl9MJ873l1Gl5ax3DM8ze5yGh63f8Mypf407Dj6d2zK/R/9wqa9FR9CqZRdNNxDRFFxCbdMXcrhI8W8fFlfIt0Ov7JSbXhioCgfSvw7zDHcFcaLl/bBHR7GTVOW6OGRqkHRcA8RL/x3PQs37ecfF/QgtWWc3eU0TD4X7PBXm4Qonh3Ti9U7D/Kozv2uGhAN9xAwb10WL83J4JJ+7bion87pVqkKpv31x2ldW/LHoZ2YsnArM/XSfKqB0HB3uMzfDnPbu8tIbRHLI+frWahVKpv2t+bj53ednUbfDgnc++EK1u8+FODClKo5DXcHO3ykiPFvLaawqIRXLu9HlEfH2atUy547gNsVxsuX9yXKE84f3lpE9uEjAS5OqZrRcHcoYwx3vb+cNbsO8uJlfejcItbukhq+Ci7YUROt46N4/cp+7MzO5+Z3llJUXBLA4pSqGQ13h/rfbzKY/csu7junK8PSWthdTuNQ1nOv/WGN/To25R+je/Bdxl4em706QIUpVXMhcmn70DJ9aSbPfrWO0X3aMn5IJ7vLaTxqcbRMRcb0b8+anYf49/eb6NAsmmsHpwSgOKVqRsPdYb5dl8Xd76/gxE7NeeIinTemRjzeYZlajLmX98C53diRnccjs1aRFBfByJ46OZuqXzos4yArMrO58T+LSW0Zx+tX9SMiXHeg1khZz73uZ5u6woTnx/XmhI7NuOPd5SzI2FvndSpVExruDrF+9yGunfwzzWI8vHntCTSJ1AnBaqwOR8tUJNLtYtJV/UlOjGbC24tZuvW3gKxXKX9ouDvA+t2HuHTSj7jChLevH0iLJpF2l9Q41eE498rER7t587oBNI/1cNW/fmKJBryqJxrujVxpsIeJMHXCIFISY+wuqfEK90BYOBTmBXS1reOjmDZhEM1iPVytAa/qiYZ7I7Z062+Mnfh7sB+XpMey15k7OuDhDr8HfGkPfsEGHYNXwaXh3kh9s2Y3l01aSGxEOO/98UQN9kBxR9fpOPeqWAF/Im0SIrn63z8xQ+ehUUGk4d4ITVm4hfFvLea4FjF8eONJJOtQTOC4o4LScy/VKj6S9284ib4dmnLL1KW89u0GjDFB254KXRrujUhBUTH3fbSCB6av5OTOiUybcCJJcRF2l+Usnpg6n8RUnfgoN29dP4CRPVvzxGdr+PPUpRw+UhTUbarQoycxNRLbs/O4+Z0lLN2azU2nHsedZ6XhCtMTlALOHRX0cAeICHfxv5f2oXubJjzzxVrW787h1Sv60kmH11SAaM+9gTPGMH1pJsOfm8e6XYd45fK+3DO8qwZ7sLijAnace3VEhJtO7cxb1w1kz6F8zn3xO97+cYsO06iA0HBvwPYcsmYXvP3d5aS1iuOzW4cyIr213WU5mzv4wzLlnZyayGe3DqV/clMe/HglV0/+me3ZwRv3V6FBw70BKiouYfL3mzj9mW/5atVu7j47jXf/eCIdmkfbXZrzBXmHamVaxUfy1nUDePT84/lp0z7O+Oe3vDwng4IivS6rqh0dc29AjDF8uWo3z365jrW7DzEkNZFHzu+hJybVJ090vffcS4kIV56YzLCuLXh01iqe/mItHyzO5PYzuzAyvTVhOhSnakDDvQEoKTF8uy6L575ex4rMA3RKjOHVy/syvEcrndWxvrntC/dS7ZpG8/qV/Zm7dg//M3sNt0xdykvfrOe2M7pw9vGtdH+L8ouGu41yC4r4cEkmbyzYzMasXNomRPHUxT25sE9bwl06YmYLd3S97VCtzqlpLRiamsSnv+zk+a/XcdOUJbRvFsWVgzoytn8H4qN1cjhVOQ33elZUXML8jL18snQ7X67azeEjxfRqF8/zY3szIr01nnANdVu5o6G4AEqKIcz+KZPDwoTzerVhRHprvvh1F28s2Mzjs9fwzy/XcUb3lozq1YZT05J0emd1DA33evBb7hHmrc/i27VZzF2Xxf7cIzSJDOf83m24pH97+nZoaneJqpQ7yrovPAwRcfbW4sMVJoxIb82I9Nas2nGQqT9t5dNfdvLpip3ERYYzLK0FQ7skMTQ1UWcFVYCGe8AVlxg27c1h6dZslm7LZunWbNbsOogx0DTazZDUJEb2bM0p2ttqmErndC/Ma1Dh7qt7myY8ekEP/nZed77P2MvM5Tv5dt2esrlqurSMpW+HpvRun0DvDgl0TorVYb4Q5Fe4i8hw4AXABfyfMeaJcssjgLeAfsA+YKwxZnNgS204CoqK2X2ggJ0H8th1MJ9Ne3PJ2JNDxp4cNu3NpaDIuup9XGQ4vdsncNvpXTglLYn0tvG6M6yhc9f9Itn1xe0K49S0Fpya1oKSEsOqnQeZtz6LhRv389nKXUz7eRsAHlcYyYnRdG4RS+ekWJITY2gdH0Xr+EhaxUcS6dZOhhNVG+4i4gJeBs4EMoGfRWSGMWaVT7Prgd+MMZ1FZBzwJDA2GAX7MsZQYqzecomxbtZj6wiUEmMoNoaSEsqWGQNFJSXkF5aQX1RMQdl9sfVaYTH5hcXkFBRxIK+Q7MOF1n1eIQfzCtmbU8DenCPl/o2gXdMoOifFMiQ1kdSWcfTtkECnxFg9fK2xcfv03BuRsDChR9t4erSN56ZTrd+NTXtzWbYtm7W7D7FhTw6rdhzk85W7KCl3AmyzGA+JsR4SojzER7tJiHKTEO0mPspNtCecKI+LKLeLSLeLSHcYUW4XUR4XnvAwwsMEV1jpvfjch+FyWc/DRCj9NRARBOt3pvS5Cg5/eu4DgAxjzEYAEZkGnA/4hvv5wEPexx8AL4mImCCcRz1p3kae/HwNxcYK6mCKcruI9/mP3qFZNH06JNA6PopW8ZG09t7aJkQT5dHejyOUhXvDOGKmtkSETkmxx8xVk19YzI7sPHYeyGfngXx2Hchjx4F89uUUcCCvkG37D7PS26nJK6y/E6jKwp7f/wCUvi4IHLXc+5rP+xqbv5/XnbEndAjqNvwJ97bANp/nmcDAytoYY4pE5ADQHDjqigQiMgGY4H2aIyJra1O0zRIp97lCQKh95kQePiGUPi+E3s8YbPzM4x6FcbV/e0d/GvkT7hX9bSzfZ/anDcaYicBEP7bZYInIImNMf7vrqE+h9plD7fOCfmYn8mcXeibQ3ud5O6D8JWTK2ohIOBAP7A9EgUoppWrOn3D/GUgVkRQR8WB9m5hRrs0M4Grv44uBb4Ix3q6UUso/1Q7LeMfQbwa+wDoU8t/GmF9F5BFgkTFmBvAv4G0RycDqsddhOKnBa9TDSrUUap851D4v6Gd2HNEOtlJKOY+etqaUUg6k4a6UUg6k4V4HInKXiBgRSbS7lmASkadFZI2IrBCR6SKSYHdNwSIiw0VkrYhkiMi9dtcTbCLSXkTmiMhqEflVRG61u6b6IiIuEVkqIrPsriUYNNxrSUTaY03JsNXuWurBV0APY0xPYB1wn831BIXPVBvnAN2BS0Wku71VBV0RcKcxphswCPhTCHzmUrcCq+0uIlg03GvvOeAeKjhZy2mMMV8aY4q8T3/EOtfBicqm2jDGHAFKp9pwLGPMTmPMEu/jQ1hh19beqoJPRNoB5wL/Z3ctwaLhXgsiMgrYboxZbnctNrgO+MzuIoKkoqk2HB90pUQkGegDLLS3knrxPFbnrMTuQoJF53OvhIh8DbSqYNEDwP3AWfVbUXBV9XmNMZ942zyA9TV+Sn3WVo/8mkbDiUQkFvgQuM0Yc9DueoJJREYCe4wxi0XkVLvrCRYN90oYY86o6HURSQdSgOXe6UrbAUtEZIAxZlc9lhhQlX3eUiJyNTASON3BZx/7M9WG44iIGyvYpxhjPrK7nnowGBglIiOASKCJiPzHGHOFzXUFlJ7EVEcishnob4xx7Ix63ou1PAucYozJsrueYPHOi7QOOB3YjjX1xmXGmF9tLSyIxOqhvAnsN8bcZnc99c3bc7/LGDPS7loCTcfclT9eAuKAr0RkmYi8ZndBweDdaVw61cZq4D0nB7vXYOBK4DTvz3aZt0erGjntuSullANpz10ppRxIw10ppRxIw10ppRxIw10ppRxIw10ppRxIw10ppRxIw10ppRzo/wHN32tjiOmFPgAAAABJRU5ErkJggg==\n", 212 | "text/plain": [ 213 | "
" 214 | ] 215 | }, 216 | "metadata": {}, 217 | "output_type": "display_data" 218 | } 219 | ], 220 | "source": [ 221 | "plot_d0(D,input_node)\n", 222 | "plt.title('Initial Decision Boundary')\n", 223 | "#plt.savefig('fig1.png')" 224 | ] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "execution_count": 11, 229 | "metadata": {}, 230 | "outputs": [], 231 | "source": [ 232 | "lh=np.zeros(1000)\n", 233 | "for i in range(1000):\n", 234 | " #d=np.random.normal(mu,sigma,M)\n", 235 | " d=(np.random.random(M)-0.5) * 10.0 # instead of sampling only from gaussian, want the domain to be covered as uniformly as possible\n", 236 | " labels=norm.pdf(d,loc=mu,scale=sigma)\n", 237 | " lh[i],_=sess.run([loss,optimizer], {input_node: np.reshape(d,(M,1)), train_labels: np.reshape(labels,(M,1))})" 238 | ] 239 | }, 240 | { 241 | "cell_type": "code", 242 | "execution_count": 12, 243 | "metadata": { 244 | "scrolled": true 245 | }, 246 | "outputs": [ 247 | { 248 | "data": { 249 | "text/plain": [ 250 | "Text(0.5,1,'Training Loss')" 251 | ] 252 | }, 253 | "execution_count": 12, 254 | "metadata": {}, 255 | "output_type": "execute_result" 256 | }, 257 | { 258 | "data": { 259 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt8XHWd//HXZ2YySZMmbdOm0HvaUigFKZdYQOQiF7coFlxRW3UBxQVXuyL6c8VVWJd1f+tPXVlxUekPr6vcZF2tUC0KKMqlNKVceqElpKVNLzRt2rTN/fLZP+YkTNO0naSTTnLO+/l45NE5Z76Z+ZwceOeb7/nO95i7IyIi4RLLdQEiIpJ9CncRkRBSuIuIhJDCXUQkhBTuIiIhpHAXEQkhhbsMeWYWN7P9ZjY5m21FhjLTPHc51sxsf9pmIdACdATbN7r7z499VUfPzL4KTHT363Jdi0gi1wVI9Lj78K7HZrYR+Li7/+FQ7c0s4e7tx6I2kbDQsIwMOmb2VTN7wMzuM7N9wEfM7Fwze9bM9pjZNjO708zygvYJM3MzKw+2fxY8/1sz22dmz5jZ1L62DZ6/3MzWm1m9mX3HzJ4ys+v6cUynmNmfgvpfNrN3pz13hZmtDd6/xsxuDvaPNbMlwffUmdmT/f2ZSvQo3GWwei9wLzACeABoB24CxgDnAXOBGw/z/R8CbgVKgU3Av/S1rZmNBR4EPh+87wZgTl8PxMySwMPAI0AZcDPwgJmdEDT5EXC9uxcDpwF/CvZ/HqgOvuf4oEaRjCjcZbD6i7v/xt073b3J3Ze7+zJ3b3f3amARcOFhvv8hd6909zbg58Dp/Wh7BfCCu/86eO4OYGc/juU8IAl8w93bgiGo3wLzg+fbgFlmVuzude7+fNr+8cBkd2919z8d9Moih6Bwl8Fqc/qGmc00s0fMbLuZ7QVuJ9WbPpTtaY8bgeGHaniYtuPT6/DU7IOaDGrvaTywyQ+cvfA6MCF4/F5gHrDJzP5oZmcH+78WtHvMzF4zs8/3470lohTuMlj1nMZ1N7AKOMHdS4DbABvgGrYBE7s2zMx4M5D7YiswKfj+LpOBLQDBXyTzgLGkhm/uD/bvdfeb3b0cuAr4gpkd7q8VkW4KdxkqioF6oMHMTubw4+3Z8jBwppm9x8wSpMb8y47wPXEzK0j7ygeeJnXN4HNmlmdmFwPvAh40s2Fm9iEzKwmGfvYRTAsN3nd68EuhPtjf0fvbihxI4S5DxeeAa0mF392kLrIOKHd/A/gg8C1gFzAdWElqXv6hfARoSvta5+4twHuAK0mN2d8JfMjd1wffcy3wejDcdD3wN8H+k4DHgf3AU8C33f0vWTtACTV9iEkkQ2YWJzXEcrW7/znX9YgcjnruIodhZnPNbEQwvHIrqeGV53JclsgRKdxFDu/tpOaa7yQ1t/6qYJhFZFDTsIyISAip5y4iEkI5WzhszJgxXl5enqu3FxEZklasWLHT3Y80JTezcDezucC3gThwj7t/rcfzdwDvCDYLgbHuPvJwr1leXk5lZWUmby8iIgEzez2TdkcM92D6113AZaQ+er3czBa7+5quNu5+c1r7vwfO6HPFIiKSNZmMuc8Bqty92t1bSX00+srDtF8A3JeN4kREpH8yCfcJHLiIUw2HWF/DzKYAU0l9qk5ERHIkk3DvbXGmQ82fnE9q+dRe178wsxvMrNLMKmtrazOtUURE+iiTcK8BJqVtTyT1EezezOcwQzLuvsjdK9y9oqzsiBd7RUSknzIJ9+XADDObGtxRZj6wuGcjMzsJGAU8k90SRUSkr44Y7sGNiRcCS4G1wIPuvtrMbjezeWlNFwD3uz7yKiKScxnNc3f3JcCSHvtu67H9leyVdXib6xpZ/8Y+Ljn5uGP1liIiQ8qQXH7gmh8+x/U/qaS+sQ39oSAicrAhGe7b6psAmH37o3ztt6/kuBoRkcFnyIV7c1sHzW2d3ds/fnpj7ooRERmkhly4V+3Yn+sSREQGvSEX7s1tB34+SiPuIiIHG3Lh3tB6YLi3tnfS2t55iNYiItE05MK9saUdgMJkvHvfP/9mda7KEREZlIZcuHf13MuK87v3vVizJ1fliIgMSkMu3BtbUz33McPfDPf2Do28i4ikG3Lh3tCS6rnfesWs7n0dnQp3EZF0ObuHan99aM5kLj15LCeMHc6k0mFsrmtSuIuI9DDkwn1EYR4jCvMAsGCp+XaFu4jIAYbcsEw6C24jop67iMiBhna4B/8q3EVEDjS0w900LCMi0puhHe7Bv13TI0VEJGVIh3tXuje29no/bhGRyBrS4W5HbiIiEklDO9ztzXjvuVqkiEiUDe1wT3tc39SWszpERAabIR3u6YuH7WlUuIuIdMko3M1srpmtM7MqM7vlEG0+YGZrzGy1md2b3TJ7950FZ3D5qccDsKex9Vi8pYjIkHDEcDezOHAXcDkwC1hgZrN6tJkBfBE4z91PAT4zALUeZPTwfD550QmAhmVERNJl0nOfA1S5e7W7twL3A1f2aPO3wF3uvhvA3Xdkt8xDK8pP3bRD0yFFRN6USbhPADanbdcE+9KdCJxoZk+Z2bNmNre3FzKzG8ys0swqa2tr+1dxD4XJ1NpnDfogk4hIt0zCvbfp5D0/758AZgAXAQuAe8xs5EHf5L7I3SvcvaKsrKyvtfaqMOi5N6nnLiLSLZNwrwEmpW1PBLb20ubX7t7m7huAdaTCfsAV5qXCvesmHiIiklm4LwdmmNlUM0sC84HFPdr8CngHgJmNITVMU53NQg8lEY+RTMRobNOwjIhIlyOGu7u3AwuBpcBa4EF3X21mt5vZvKDZUmCXma0BngA+7+67BqrongqTce7+UzU3P/DCsXpLEZFBzdxzs1xuRUWFV1ZWZuW1Lvj6E2yqawRg49fenZXXFBEZjMxshbtXHKndkP6EapfjSvKP3EhEJEJCEe4lBXm5LkFEZFAJRbjHYlr8V0QkXSjC/cYLpuW6BBGRQSUU4V5RXsqNF04jPxGKwxEROWqhScNkPEZbR2euyxARGRRCE+558RidDh2duZnaKSIymIQq3AE2B/PdRUSiLDThXtfQAsD1P1me40pERHIvNOHeEKwKubmuKceViIjkXmjCfW9wJ6ZWXVQVEQlPuJcWJbsf66KqiERdaML9lstnMmPscAD2t2j5XxGJttCEe2EywTXnTgHQfHcRibzQhDu8OR1S4S4iUReqcE90hXu7xtxFJNpCFe558dTqkJoxIyJRF6pwTwY99/ZOhbuIRFuowj1PwzIiIkDYwj1Y8vfJV2tzXImISG5lFO5mNtfM1plZlZnd0svz15lZrZm9EHx9PPulHlnXmPs3lq7LxduLiAwaiSM1MLM4cBdwGVADLDezxe6+pkfTB9x94QDUmLGuYRkRkajLJA3nAFXuXu3urcD9wJUDW1b/JHQvVRERILNwnwBsTtuuCfb19D4ze8nMHjKzSb29kJndYGaVZlZZW5v9cfF2rSkjIgJkFu69dYd7puhvgHJ3Pw34A/CT3l7I3Re5e4W7V5SVlfWt0gw0t3Vk/TVFRIaiTMK9BkjviU8EtqY3cPdd7t4SbP5/4KzslNc3Jx1XnIu3FREZdDIJ9+XADDObamZJYD6wOL2BmY1L25wHrM1eiZkbW1LAe2aPZ9qYoly8vYjIoHHE2TLu3m5mC4GlQBz4obuvNrPbgUp3Xwx82szmAe1AHXDdANZ8WMl4jJZ2fUJVRKLtiOEO4O5LgCU99t2W9viLwBezW1r/5Ocp3EVEQjcxPNVz14VVEYm20IW7eu4iImEM93iM1vZO3DXnXUSiK3zhnhcHtKa7iERb6MK9a013Dc2ISJSFLtzz81KHdNpXHs1xJSIiuRO+cE+E7pBERPosdEmYVLiLiIQv3EVEJITh3tKmC6kiIqELdy37KyISwnAvLsjLdQkiIjkXunB/7xkTmBos+duhOzOJSESFLtxjMePqsyYC0KZPqYpIRIUu3OHNG2Wr5y4iURXOcA+WIGjvULiLSDSFMtzz4qmee1unhmVEJJpCGe6JmHruIhJt4Qz3rp67LqiKSESFMty7hmXadUFVRCIqo3A3s7lmts7MqszslsO0u9rM3Mwqsldi33UNy7RqTXcRiagjhruZxYG7gMuBWcACM5vVS7ti4NPAsmwX2VfDCxIAPPLS1hxXIiKSG5n03OcAVe5e7e6twP3Alb20+xfg60BzFuvrl/NPGANAh+6jKiIRlUm4TwA2p23XBPu6mdkZwCR3fziLtfVbIh6jKBmnWStEikhEZRLu1su+7i6xmcWAO4DPHfGFzG4ws0ozq6ytrc28yn4oyItrhUgRiaxMwr0GmJS2PRFIH8wuBk4F/mhmG4FzgMW9XVR190XuXuHuFWVlZf2vOgOpcFfPXUSiKZNwXw7MMLOpZpYE5gOLu55093p3H+Pu5e5eDjwLzHP3ygGpOEP5iRjN7eq5i0g0HTHc3b0dWAgsBdYCD7r7ajO73czmDXSB/ZWfF9ddmUQkshKZNHL3JcCSHvtuO0Tbi46+rKNXkBejRT13EYmoUH5CFaAgoQuqIhJd4Q33vJguqIpIZIU43NVzF5HoCm24a7aMiERZaMO9QLNlRCTCQh3uGpYRkagKbbjn58Vo1pK/IhJRoQ33gkSc1vZOOnXDDhGJoNCGe35e6tBa1HsXkQgKbbgXJOIA+pSqiERSaMO9rDgfgNdqG3JciYjIsRfacH9reSkAa7bW57gSEZFjL7ThXpjfNSyjMXcRiZ7QhnvXmLvmuotIFIU23PPiRszQ4mEiEkmhDXczIz8R12wZEYmk0IY7aNlfEYmukIe71pcRkWgKdbjnJ2KaLSMikRTqcFfPXUSiKtThrp67iERVRuFuZnPNbJ2ZVZnZLb08/wkze9nMXjCzv5jZrOyX2nf56rmLSEQdMdzNLA7cBVwOzAIW9BLe97r7W9z9dODrwLeyXmk/FOTFtaa7iERSJj33OUCVu1e7eytwP3BlegN335u2WQQMikXU8xMxWtRzF5EISmTQZgKwOW27Bji7ZyMz+xTwWSAJXNzbC5nZDcANAJMnT+5rrX1WkBfXmLuIRFImPXfrZd9BPXN3v8vdpwNfAL7c2wu5+yJ3r3D3irKysr5V2g/quYtIVGUS7jXApLTticDWw7S/H7jqaIrKlgLdR1VEIiqTcF8OzDCzqWaWBOYDi9MbmNmMtM13A69mr8T+S8bj1DW0an0ZEYmcI4a7u7cDC4GlwFrgQXdfbWa3m9m8oNlCM1ttZi+QGne/dsAq7oPFL24B4L5lm3JciYjIsZXJBVXcfQmwpMe+29Ie35TlurJiwZzJfOfxKsx6u2wgIhJeof6E6o0XTgd0k2wRiZ5Qh3tRMnU3pqod+3NciYjIsRXqcO8ajnmwsibHlYiIHFuhDncRkagKfbh/8qLpJGK6oCoi0RL6cE8mYrR3Op2dg2K5GxGRYyIS4Q7Q2qFPqopIdIQ/3OOpQ9QCYiISJeEP966eu8JdRCIk/OEe17CMiERP+MNdPXcRiSCFu4hICIU/3OMKdxGJntCHe35ean2Z1g4tHiYi0RH6cC8uSK1qvLepPceViIgcO6EP91GFSQB2N7bmuBIRkWMn9OFeGoR7XYPCXUSiI/ThXlyQIGawp7Et16WIiBwzoQ/3WMwYVZikTsMyIhIhoQ93gJGFeexRuItIhGQU7mY218zWmVmVmd3Sy/OfNbM1ZvaSmT1mZlOyX2r/lRYl2d2gYRkRiY4jhruZxYG7gMuBWcACM5vVo9lKoMLdTwMeAr6e7UKPxsjCpGbLiEikZNJznwNUuXu1u7cC9wNXpjdw9yfcvTHYfBaYmN0yj86owjyFu4hESibhPgHYnLZdE+w7lOuB3/b2hJndYGaVZlZZW1ubeZVHaVQwLOOuuzGJSDRkEu693YC015Q0s48AFcA3enve3Re5e4W7V5SVlWVe5VEaVZiktaOTx9buOGbvKSKSS5mEew0wKW17IrC1ZyMzuxT4EjDP3VuyU152dH2Q6eM/raS5TWvMiEj4ZRLuy4EZZjbVzJLAfGBxegMzOwO4m1SwD7ru8cjCvO7H+5q1xoyIhN8Rw93d24GFwFJgLfCgu682s9vNbF7Q7BvAcOAXZvaCmS0+xMvlRGlRsvtxS7t67iISfolMGrn7EmBJj323pT2+NMt1ZdXIwvRw17ruIhJ+kfiEaknBm7/DNOYuIlEQiXAfW1LAe2aPB9RzF5FoiES4A3xozmRAPXcRiYbIhHt+XupQb/zpihxXIiIy8KIT7onUoe5r0VRIEQm/yIR7W4eWHhCR6IhMuJ82YQSQmvPe0elsq2/KcUUiIgMnMuEeixkL5kxif0s7d/x+Pef+2+Ns3aOAF5Fwiky4AyTjMVrbO/nPJ6oAqN03qJbAERHJmkiF+9xTxx2wHbPeFrwUERn6IhXu5WMKD9hWtotIWEUq3EsK8g7Y1r07RCSsIhXuhcn4AdttnVqKQETCKVLhbj3GYdq0zoyIhFSkwh3gfWe+ee9ufbBJRMIqcuFeVpzf/fgTP1vBz559PYfViIgMjMiFezLx5iHvb2nny79alcNqREQGRuTCXbMfRSQKIhfu7T1myJw2cUSOKhERGTjRC/ceF1HVkxeRMIpcuPecIfNiTT3b65tzVI2IyMDIKNzNbK6ZrTOzKjO7pZfnLzCz582s3cyuzn6Z2dNzWAbg8w+9mINKREQGzhHD3cziwF3A5cAsYIGZzerRbBNwHXBvtgvMtt7mtq/dtjcHlYiIDJxMeu5zgCp3r3b3VuB+4Mr0Bu6+0d1fAgb9Rz5jPQbZJ4wcRqc+yyQiIZNJuE8ANqdt1wT7+szMbjCzSjOrrK2t7c9LHLX/886TuPbcKd3bl806jrqGVn76zMac1CMiMhASGbTpbUJJv/q67r4IWARQUVGRk/7yqKIk/3zlqUwZXcTwggTPbagD4KuPrOWac8tzUZKISNZl0nOvASalbU8Etg5MOcfOx94+lQ9UTKKtIzWS1NreyZf+52Wa2zpyXJmIyNHLJNyXAzPMbKqZJYH5wOKBLevYSQ/zny/bxPf++BruTmuwYmTlxjr2NbflqjwRkX45Yri7ezuwEFgKrAUedPfVZna7mc0DMLO3mlkN8H7gbjNbPZBFZ9PHzpt6wHZDSzvfWLqOE7/8W3Y3tHL195/hU/euzFF1IiL9k8mYO+6+BFjSY99taY+XkxquGXLOnjb6gO17/rKh+/G24MNNq7fUH9OaRESOVuQ+odoX2/c2AbrXqogMPQr3w9iyJ9Vz73kHJ3fn2epduG7CKiKDVEbDMmH3k4/NobPTqdnTxK1p67s//GJqUlDtvhYefmkr7zp1HF99ZC3DCxLc+dir/Pv7Z/O+s4bkaJSIhJzCHbjwxLLux99cuo76ptTsmGXBHHiAhfeu5PHPlfDDp94ck6/euR+Azk5nxabdvLW89BhVLCJyeBqW6aEr2Hvz+q7GA7aDKfL8fNnrvP/7z/DY2jcGsjQRkYwp3Pvgoz9efsD29//0Glv2NHHrr1MzPzfVNfb2bSIix5zC/RAqv3xpRu3O+9rj3Y93N7S++f0b63jvd5/SJ15FJCcU7odQUpDX5++58/Eqym95hOa2Dq7+/jOs3LSHDy56lpWbdne3aWhpp0PLUIrIANMF1R4+dt5UNu9uJJl48/fe9z9yFk9V7eS/nn09o9eYeevvuh+/uHkP7/3u08w8vpjW9k6qdzbwyYum8w9zZ/L8pt2MLc5n/Rv7OHPyKEYWJg96rY5OJ95znWIRkSOwXM3Vrqio8MrKypy8d6Y27Wpk9PAkRfkJKjfWcfX3n+F3nzmfuf/xZwC+9YHZfPbB/t3F6eWvvJO3fOXR7u1LZo7lw+dMZu22fXzqHScAsPjFrXz6vpX86KNv5fwTxpCI6w8tkagzsxXuXnHEdgr3vrvp/pVMLxvOpy+ZwY59zbS0dfKDv2ygfHQhyzfu5pxppd0XWfvj7y6aztxTjufzD73I+jdS0y0/9Y7pNLR0cPmpx/P4KzuYP2cyz23YxQcqJtHp8OdXa3nb9DHEY6aevkiIKdxzrLp2Pxf/+58AeOxzF3JJ8DjbCvJiNLcdeAOsq8+ayK1XzGLEsDzcnVd37GfEsDyefm0nV86eQCxmtLZ3HjD01KW5rYPVW+s5a4rm7IsMRpmGu8bcB8i0suHdj6enPc62nsEO8NCKGuZMLWXH3ma++ej6A557cHkN50wbzR1/WN899g/Q2NpOze4mrrrrKRpbUzN8fnb92bx9xpju7925v4VHXtrGNedOOWhJBhEZXNRzH0DltzxCPGa89n/fxZ7GVlrbO+l0GJYXZ/btj/Lld5/Mkpe3sWVPE/PfOplkIsZxJQXcu+x1nt+0h7HF+XztfW/hYz8++Oc0sjCPa84t587HXj2qGvPi1utNw7s8f+tlPPPaLp6t3nXABeX3nTmRkmEJ5p5yPGdPG01zWwd/WPsGc6aWMra4gC17mli6ajsfOnsyyXiMXQ2tlBXn97m+3Q2tjBiWR0xDTSKAhmUGhR17m8mLxxhVdPAsGHc/ZO+3vrGNhfc9z1evSt0O8Mq7nuK44nwWXVPBPX+uZsueJm67YhZmxlcfXsOzG3axasteThlfwuqtew96vUtPPo7zZ4zhgeWbWbPt4OcPp7QoSV3a/P3e3LngDP7xly+zv6Wd0qIky/7xEk77yqM0BXP8r3tbOT9+eiO3X3kKc6aWMvP4ku7jLEjG+N2q7RQlE1w66zjcnea2ToYl49Q3tjH79kf5+4tP4HPvPKlPdYuElcI9gjbXNXL+15/gghPLeHJ9LfdcU8HHf1rJ4oXncdrEkezY18ycf33soO/7yDmT+dmzmzJ6jymjCw9ahqGvJo4axrSy4Ty5/uCbpI8qzGN3YxsL5kxi1Za9vByspX/21FKmlRWxdU8z588Yw4UnlvG3P63kjg+ezhmTR7FjbzMPLN/M2u17ec9p4zmrfBR/WldLRXkpN92/khsvmM67TxvX/T6rt9bz5PqdfOLCaZgZT1ftZM7UUs1IkkFP4R5h7s7GXY1MHVN00HPb6pu4N7id4M2Xncjpk0Zy9tRSvvN4Fa/u2Me73jKOSaMK+frSV7j1ilnMGFvMhp37mTZmOLGYseL1Ot73vWc4f8YYRhYm+c2Lub+d7icvms53//haRm1PPG549wwkgA+dPZlLZo7l+p9UcuqEEv71qrdw8rgS/uXhNRw/ooBXtu9j9dZ6qmsbWDBnEhNHFbL4ha1cd1457zhpLKOHJ/mf57ewYVcDiZhRXduA43z3w2cdto4de5spK86nvqmNtg7vdcjK3enodP3CkQMo3GXAPL9pN6dPHEl7p7Otvonh+QkW/bmamy89kbXb9vLe7z7N+BEFPHXLxXzkB8t4qmoXHz2vnJKCPF7YvIe7Pnwmf3l1J/VNrexqaOW+5zaxua6Jc6aVcsMF0/jVyq2s2baXqh37+eszJ/DL57fk+pD7LP2vobOmjGLF67u56vTxDEvG+e2q7expPHCBuimjC5k2pohzpo1m5aY91De18Uz1LgCKCxJ85tITMeCik8r41QtbeblmD7e95xTWbd/HO2aWsWDRs1xwYhnzZo+nrDifx9bu4MrTx2NmdHY6sZjR0t7Bwy9uo3xMEcl4jJnjimlp72R4fmpexfb6Zva3tHPC2IGbAACpi/eFSc3l6C+Fu+TMIy9t45TxJZSPKaK5rYOWtk5GFPZ9OYcum+saKS5IsHrrXj58zzLmnnI83/+bs9i6p4mOTufOx17lFytq+Pr7TuPWX6/i2reVk4zH+PA5k9lQ28DLW+opyk/wx3W1TCsr4uEXt7I1uIViujMnj+T5TXsO2Hfd28p5aEUN+1va+11/rpw6oYRVW1LXWG65fCY/emoDb+xtOajdmOH5VEwZxe9WbwfgxgumcfeT1dx0yQx+8JcN3cde+eVLWVZdx4ad+ynKT7CprpE39jYzqjDJz5dtYnRRkk9cOJ1lG+r4w9o3+GDFJGYcN5zpZcPpdOeXK7fw5Ppa9jW389ubzuf4kgJGFSXZsLOB4oIEo4uSdHQ6dz9ZzbnTR7Nu+z5GFyWZPnY4U0cXUd/UxvKNdVw26ziqdzYwbUwRm+oamVxaSFuH09rx5i+qdO5O9c4GRhcluz8FXtfQysgheqFe4S6h4+78bNkm3v2WcZSmXaRu7+jk2eo6zjthdMZTNDftaqQgGWNUYZK8tGGPxtZ2PnP/C1TvbOA3C9/OsGS8+7lVW+r57+druHjmWLbXN/OlX63is5edyBmTRtLY1sHJx5cwZniSt/7rH9jd2Ma0siJumTuTV7bv4/Vdjbxn9jj2t7Sz8N6VB1z8vnjmWE4dX8Kdj1cB8IGKiSzbUMdHzp7Cd/9Yxe7GNooLEuxrHphfMOWjC9l4lNdRsqGkIMHeLBzjVaeP55Xt+3hl+74jtr3ubeWs3LSbF2tS13YSMev+HEjFlFGcOmEEb5kwgjXb9rJxZwNTxxQRjxs1dU3kxVNtJ5cWUj66iKod+6lvauMXKzbzt+dPY3djK6VF+bg7yXiM4oIEF888jqrafZwzbXS//3rJarib2Vzg20AcuMfdv9bj+Xzgp8BZwC7gg+6+8XCvqXCXsGrr6CQRs4x+0XT9/5c+fHK4tj1fs7PTeWLdDs47YQwFeXFWbamntChJImZ894+vMWt8CfNmj+fVN/azbMMu5s0ez+bdTVTX7ueMySOZMrqIvHiMxtZ2nttQxynjR/DTZzby+zVv8Mr2fXzvw2fyk2c28mx1HUXJOF+4fCZ7m9r45qPr+eszJ1Bd28DJ40q477nUENSlJ4+lsbWDp1/bRWlRkhPGDue54KY3IwvzGFWY6qkfyohheQfdU2Hm8cXsbWrr9a+toeoLc2fydxdN79f3Zi3czSwOrAcuA2qA5cACd1+T1uaTwGnu/gkzmw+8190/eLjXVbiLDF4NLe00tLYztrjgqF+rvaOTBytruPzU4xlVlOT3a97grCmjKC1KsmlXIw+t2MwH50xmwshh3b/s2jtTF5PzE7HuX2gv19R336x+xLA8anY3UVyQ4JTxJTywfDMnHl9MXizGqKI88uKgoDnJAAAFu0lEQVQxtuxpYtyIAgqTCXbtb2HK6CIef2UH2+ubGFGYZN7s8TS3ddDe6dz/3CbKivM5ZXwJYJQUJHi9rpHxI4fxyxU1vPu0ccRjxt6mdmIxGFWYZNmGXbzjpLH85+NVrN2+l2ljhnNcST4njyshLx5j9PAky6rraGrroCAvTjJu1O5L1XHF7HHkJ+KH+IkdXjbD/VzgK+7+V8H2FwHc/d/S2iwN2jxjZglgO1Dmh3lxhbuISN9lGu6ZzLGaAGxO264J9vXaxt3bgXpgdC9F3WBmlWZWWVt78BxnERHJjkzCvbdBwJ498kza4O6L3L3C3SvKysp6+RYREcmGTMK9BpiUtj0R6PnJle42wbDMCKAuGwWKiEjfZRLuy4EZZjbVzJLAfGBxjzaLgWuDx1cDjx9uvF1ERAbWESdaunu7mS0ElpKaCvlDd19tZrcDle6+GPgB8F9mVkWqxz5/IIsWEZHDy2gWvbsvAZb02Hdb2uNm4P3ZLU1ERPpLKxKJiISQwl1EJIRytraMmdUCrx+xYe/GADuzWM5QoGOOBh1zNBzNMU9x9yPOJc9ZuB8NM6vM5BNaYaJjjgYdczQci2PWsIyISAgp3EVEQmiohvuiXBeQAzrmaNAxR8OAH/OQHHMXEZHDG6o9dxEROQyFu4hICA25cDezuWa2zsyqzOyWXNeTLWY2ycyeMLO1ZrbazG4K9pea2e/N7NXg31HBfjOzO4Ofw0tmdmZuj6B/zCxuZivN7OFge6qZLQuO94FgsTrMLD/YrgqeL89l3f1lZiPN7CEzeyU41+dG4BzfHPw3vcrM7jOzgjCeZzP7oZntMLNVafv6fG7N7Nqg/atmdm1v75WJIRXuwS3/7gIuB2YBC8xsVm6rypp24HPufjJwDvCp4NhuAR5z9xnAY8E2pH4GM4KvG4DvHfuSs+ImYG3a9v8D7giOdzdwfbD/emC3u58A3BG0G4q+DfzO3WcCs0kde2jPsZlNAD4NVLj7qaQWH5xPOM/zj4G5Pfb16dyaWSnwT8DZwBzgn7p+IfSZuw+ZL+BcYGna9heBL+a6rgE61l+Tum/tOmBcsG8csC54fDepe9l2te9uN1S+SN0b4DHgYuBhUjd92Qkkep5vUquSnhs8TgTtLNfH0MfjLQE29Kw75Oe46y5tpcF5exj4q7CeZ6AcWNXfcwssAO5O239Au758DameO5nd8m/IC/4UPQNYBhzn7tsAgn/HBs3C8LP4D+AfgM5gezSwx1O3aoQDjymjWzkOctOAWuBHwVDUPWZWRIjPsbtvAb4JbAK2kTpvKwj3eU7X13ObtXM+1MI9o9v5DWVmNhz4b+Az7r73cE172TdkfhZmdgWww91XpO/upaln8NxQkQDOBL7n7mcADbz5Z3pvhvwxB0MKVwJTgfFAEakhiZ7CdJ4zcajjzNrxD7Vwz+SWf0OWmeWRCvafu/svg91vmNm44PlxwI5g/1D/WZwHzDOzjcD9pIZm/gMYGdyqEQ48pjDcyrEGqHH3ZcH2Q6TCPqznGOBSYIO717p7G/BL4G2E+zyn6+u5zdo5H2rhnskt/4YkMzNSd7Ra6+7fSnsq/RaG15Iai+/af01w1f0coL7rz7+hwN2/6O4T3b2c1Hl83N0/DDxB6laNcPDxDulbObr7dmCzmZ0U7LoEWENIz3FgE3COmRUG/413HXNoz3MPfT23S4F3mtmo4K+edwb7+i7XFyD6ccHiXcB64DXgS7muJ4vH9XZSf369BLwQfL2L1HjjY8Crwb+lQXsjNXPoNeBlUrMRcn4c/Tz2i4CHg8fTgOeAKuAXQH6wvyDYrgqen5bruvt5rKcDlcF5/hUwKuznGPhn4BVgFfBfQH4YzzNwH6nrCm2keuDX9+fcAh8Ljr8K+Gh/69HyAyIiITTUhmVERCQDCncRkRBSuIuIhJDCXUQkhBTuIiIhpHAXEQkhhbuISAj9L1F7S5R3tBIwAAAAAElFTkSuQmCC\n", 260 | "text/plain": [ 261 | "
" 262 | ] 263 | }, 264 | "metadata": {}, 265 | "output_type": "display_data" 266 | } 267 | ], 268 | "source": [ 269 | "# training loss\n", 270 | "plt.plot(lh)\n", 271 | "plt.title('Training Loss')" 272 | ] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "execution_count": 13, 277 | "metadata": {}, 278 | "outputs": [ 279 | { 280 | "data": { 281 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XlcVXX+x/HXh8uqIKiAGy6ogBuKiEuZWzZl5lhWak6rlc7ktC+2b/arqaZpqsmmdBrTxrTFLDXb09Q0SwV3VNxxxwUXFoH7/f1xABFZLngvl3v9PB8PHnf7nu/9IPi+X77nnO8RYwxKKaW8i4+7C1BKKeV8Gu5KKeWFNNyVUsoLabgrpZQX0nBXSikvpOGulFJeSMNdKaW8kIa7Ukp5IQ13pZTyQr7ueuPw8HDTqlUrd729Ukp5pJUrV2YYYyIqa+e2cG/VqhUrVqxw19srpZRHEpGdjrTTaRmllPJCGu5KKeWFNNyVUsoLuW3OXSlVuby8PNLT08nJyXF3KaqGBQYGEhUVhZ+fX7W213BXqhZLT08nJCSEVq1aISLuLkfVEGMMhw8fJj09nejo6Gr1odMyStViOTk5NGzYUIP9AiMiNGzY8Lz+YtNwV6qW02C/MJ3vz13DXSmlvJCGu1JKeSENd6WUSyxcuJAhQ4ZU2CYlJYX58+fXUEUXFg13pZTbaLi7jh4KqZSHeH7uejbsPe7UPjs0rcezf+xYYZsdO3YwaNAgevbsSXJyMrGxsUybNo06deqc0/abb77h/vvvJzw8nMTExOLnf/vtN+6//36ys7MJCgpiypQpREdH88wzz5Cdnc2SJUt4/PHHiY6OPqddXFycU7/nC4WO3JVSldq0aRNjx45lzZo11KtXj3feeeecNjk5OYwZM4a5c+eyePFi9u/fX/xau3btWLRoEcnJyUyYMIEnnngCf39/JkyYwMiRI0lJSWHkyJFltlPVoyN3pTxEZSNsV2revDm9e/cG4KabbuKtt97i4YcfPqtNamoq0dHRxMTEFLebNGkSAJmZmdx6661s2bIFESEvL6/M93G0napcpSN3EfmviBwUkXXlvC4i8paIpInIGhFJLKudUspzlT7murxjsMt7/umnn2bAgAGsW7eOuXPnlntyjqPtVOUcmZb5ABhUwetXAjGFX2OBf59/WUqp2mTXrl0sW7YMgBkzZnDJJZec06Zdu3Zs376drVu3FrcrkpmZSbNmzQD44IMPip8PCQnhxIkTlbZTVVdpuBtjFgFHKmhyNTDNWH4FwkSkibMKVEq5X/v27Zk6dSqdO3fmyJEj3HXXXee0CQwMZNKkSVx11VVccskltGzZsvi18ePH8/jjj9O7d28KCgqKnx8wYAAbNmwgISGBjz/+uNx2qurEGFN5I5FWwDxjTKcyXpsHvGyMWVL4+EfgUWNMhZdZSkpKMnolJqUqtnHjRtq3b+/WGnbs2MGQIUNYt67MmVnlQmX9/EVkpTEmqbJtnXG0TFmTbGV+YojIWBFZISIrDh065IS3VkopVRZnHC2TDjQv8TgK2FtWQ2PMJGASWCN3J7y3UsrFWrVqdc6ofdiwYWzfvv2s51555RWuuOKKmixNVcAZ4T4HuFtEZgI9gUxjzD4n9KuUqqVmz57t7hJUJSoNdxGZAfQHwkUkHXgW8AMwxrwLzAcGA2lAFjDaVcUqpZRyTKXhbowZVcnrBvir0ypSSil13nT5AaWU8kIa7kophz333HO89tpr1dr24osvrvD1wYMHc+zYsWr1XdJtt93GZ599dt79VEVwcHCNvp8jdG0ZpVSNWLp0aYWvXyhL/xpjMMbg4+PasbWO3JVSFXrxxReJi4vjsssuY9OmTcXPb926lUGDBtGtWzf69OlDamoqAAcOHGDYsGF06dKFLl26FId60eh237599O3bl4SEBDp16sTixYsB65DLjIwMAF5//XU6depEp06deOONNwDrZKr27dszZswYOnbsyOWXX052dnaZNf/www/06dOH2NhY5s2bB1irVo4ePZr4+Hi6du3KggULAGuZg7vvvrt42yFDhrBw4cLimp988km6dOlCr169OHDgAADbt2/noosuonv37jz99NPF2548eZKBAweSmJhIfHw8X3755Vm1jxs3jsTERF544QUeeOCB4u0mT57Mgw8+WOWfTUV05K6Up/j6Mdi/1rl9No6HK18u9+WVK1cyc+ZMkpOTyc/PJzExkW7dugEwduxY3n33XWJiYli+fDnjxo3jp59+4t5776Vfv37Mnj2bgoICTp48eVafH330EVdccQVPPvkkBQUFZGVlnfOeU6ZMYfny5Rhj6NmzJ/369aN+/fps2bKFGTNmMHnyZEaMGMGsWbO46aabzql7x44d/Pzzz2zdupUBAwaQlpbGxIkTAVi7di2pqalcfvnlbN68ucJ/nlOnTtGrVy9efPFFxo8fz+TJk3nqqae47777uOuuu7jllluK+wVrCYbZs2dTr149MjIy6NWrF0OHDgWsZZOnTJnCO++8w6lTp+jcuTOvvvoqfn5+TJkyhffee6/CWqpKw10pVa7FixczbNiw4gtzFAXVyZMnWbp0KcOHDy9um5ubC8BPP/3EtGnTALDZbISGhp7VZ/fu3bn99tvJy8vjmmuuISEh4azXlyxZwrBhw6hbty4A1157LYsXL2bo0KFER0cXt+/WrRs7duwos+4RI0bg4+NDTEwMrVu3JjU1lSVLlnDPPfcA1iJnLVu2rDTc/f39iy8V2K1bN77//nsAfvnlF2bNmgXAzTffzKOPPgpYUy5PPPEEixYtwsfHhz179hSP9lu2bEmvXr0AqFu3Lpdeeinz5s2jffv25OXlER8fX2EtVaXhrpSnqGCE7UplLeNrt9sJCwsjJSWlyv317duXRYsW8dVXX3HzzTfzyCOPcMsttxS/XtF6VwEBAcX3bTZbudMyZS1RXF6/vr6+2O324scllxn28/Mr7stms5Gfn1/uewBMnz6dQ4cOsXLlSvz8/GjVqlVxf0UfVkXuvPNOXnrpJdq1a8fo0c4/PUjn3JVS5erbty+zZ88mOzubEydOMHfuXADq1atHdHQ0n376KWAF8urVqwEYOHAg//63tfJ3QUEBx4+ffWnAnTt3EhkZyZgxY7jjjjtYtWrVOe/5xRdfkJWVxalTp5g9ezZ9+vSpUt2ffvopdrudrVu3sm3bNuLi4ujbty/Tp08HYPPmzezatYu4uDhatWpFSkoKdrud3bt389tvv1Xaf+/evZk5cyZAcZ9gLVkcGRmJn58fCxYsYOfOneX20bNnT3bv3s1HH33EqFEVnk5ULRruSqlyJSYmMnLkSBISErjuuuvOCtnp06fz/vvv06VLFzp27Fi88/DNN99kwYIFxMfH061bN9avX39WnwsXLiQhIYGuXbsya9Ys7rvvvnPe87bbbqNHjx707NmTO++8k65du1ap7ri4OPr168eVV17Ju+++S2BgIOPGjaOgoID4+HhGjhzJBx98QEBAAL179yY6Opr4+Hgefvjhs679Wp4333yTiRMn0r17dzIzM4ufv/HGG1mxYgVJSUlMnz6ddu3aVdjPiBEj6N27N/Xr16/S9+cIh5b8dQVd8lepytWGJX+V6wwZMoQHHniAgQMHlvm6u5f8VUopVQXHjh0jNjaWoKCgcoP9fOkOVaWUqmFhYWGVHqlzvnTkrlQt566pU+Ve5/tz13BXqhYLDAzk8OHDGvAXGGMMhw8fJjAwsNp96LSMUrVYVFQU6enp6GUpLzyBgYFERUVVe3sNd6VqMT8/P6Kjo91dhvJAOi2jlFJeSMNdKaW8kIa7Ukp5IQ13pZTyQhruSinlhTTclVLKC2m4K6WUF9JwV0opL6ThrpRSXkjDXSmlvJCGu1JKeSENd6WU8kIa7kop5YU03JVSygtpuCullBdyKNxFZJCIbBKRNBF5rIzXW4jIAhFJFpE1IjLY+aUqpZRyVKXhLiI2YCJwJdABGCUiHUo1ewr4xBjTFbgBeMfZhSqllHKcIyP3HkCaMWabMeY0MBO4ulQbA9QrvB8K7HVeiUopparKkcvsNQN2l3icDvQs1eY54DsRuQeoC1zmlOqUUkpViyMjdynjudKXYh8FfGCMiQIGAx+KyDl9i8hYEVkhIiv0gr9KKeU6joR7OtC8xOMozp12uQP4BMAYswwIBMJLd2SMmWSMSTLGJEVERFSvYqWUUpVyJNx/B2JEJFpE/LF2mM4p1WYXMBBARNpjhbsOzZVSyk0qDXdjTD5wN/AtsBHrqJj1IjJBRIYWNnsIGCMiq4EZwG3GmNJTN0oppWqIIztUMcbMB+aXeu6ZEvc3AL2dW5pSSqnq0jNUlVLKC2m4K6WUF9JwV0opL6ThrpRSXkjDXSmlvJCGu1JKeSENd6WU8kIa7kop5YU03JVSygtpuCullBfScFdKKS+k4a6UUl5Iw10ppbyQhrtSSnkhDXellPJCGu5KKeWFNNyVUsoLabgrpZQX0nBXSikvpOGulFJeSMNdKaW8kIa7Ukp5IQ13pZTyQhruSinlhTTclVLKC2m4K6WUF9JwV0opL6ThrpRSXkjDXSmlvJCGu1JKeSENd6WU8kIOhbuIDBKRTSKSJiKPldNmhIhsEJH1IvKRc8tUSilVFb6VNRARGzAR+AOQDvwuInOMMRtKtIkBHgd6G2OOikikqwpWSilVOUdG7j2ANGPMNmPMaWAmcHWpNmOAicaYowDGmIPOLVMppVRVOBLuzYDdJR6nFz5XUiwQKyK/iMivIjKorI5EZKyIrBCRFYcOHapexUoppSrlSLhLGc+ZUo99gRigPzAK+I+IhJ2zkTGTjDFJxpikiIiIqtaqlFLKQY6EezrQvMTjKGBvGW2+NMbkGWO2A5uwwl4ppZQbOBLuvwMxIhItIv7ADcCcUm2+AAYAiEg41jTNNmcWqpRSynGVhrsxJh+4G/gW2Ah8YoxZLyITRGRoYbNvgcMisgFYADxijDnsqqKVUkpVTIwpPX1eM5KSksyKFSvc8t7Ky506DKlz4dhuaNgG2g2BwHrurkoppxCRlcaYpMraVXqcu1IeZeUH8O1TcPoE1rEABoKfh2snQet+bi5OqZqjyw8o72AMfP8MzL0PopLgL0vg2aNw+7cQFAbTh8P2xe6uUqkao+GuvMOyt+GXNyHpdrhpFjSOBxFo0QtGfw31W8Jnt8NJPb9OXRg03JXn27kUvnsaOlwNg/8BPrazX6/TAEZMg5xj1uheqQuAhrvybDnHYfafrZH51e+ATzm/0pHt4aK7YfUM2LOqZmtUyg003JVn+/F5yEyHYZMgILjitn0ehIBQWPLPmqlNKTfScFeea/86WPFf6H4ntOhZefuAEOh+B2ycCxlprq9PKTfScFeeyRj45jEIDIP+jzu+Xa+7wMcXVrzvutqUqgU03JVn2voj7FgMA56wdpg6KjgS4q6ENZ9AQZ7r6lPKzTTclecxBhb8DUKbQ+KtVd8+4U+QlQFbvnd+bUrVEhruyvOk/QB7VkDfh8HXv+rbt70M6kbAmo+dX5tStYSGu/IsxsDCv0FYC0i4sXp92Pyg3VXWh0R+rnPrU6qW0HBXnmXXMtizEnrfb4V0dcVdBadPwvZFzqtNqVpEw115lmUTIagBdBl1fv1E9wW/upD6lXPqUqqW0XBXnuPwViuMu98B/nXOry+/QGg7EDZ/Y031KOVlNNyV5/j139ZUTPcxzumv7UA4sQ8yNjunP6VqEQ135RmyjkDKdIgfDiGNnNNn6/7W7bafndOfUrWIhrvyDCnTIS8Leo1zXp/1W0FYS9i20Hl9KlVLaLir2s8YWDkVonpA407O7bt1f+tM14J85/arlJtpuKvab9cyOLwFulXjbNTKRPeF3ONwYK3z+1bKjTTcVe23cioE1IOOw5zfd4te1u3u35zft1JupOGuarfso7DhC4i/HvzrOr//0Cio1wx2L3d+30q5kYa7qt3WfAr5OdVbIMxRzXvoyF15HQ13VXsZA6umQpMu0DTBde/TvCdk7obMPa57D6VqmIa7qr32rIID61w7agdr5A6QrqN35T003FXtteoD8KtjnbjkSo07g2+QTs0or6Lhrmqn3BOwdhZ0vBYC67n2vWx+1tTPnlWufR+lapCGu6qd1s2CvFOuOba9LE0TYP8asBfUzPsp5WIa7qp2WjkVItpDVPeaeb8mCdbyBofTaub9lHIxDXdV++xfC3tXWaN2kZp5zyZdrNu9KTXzfkq5mEPhLiKDRGSTiKSJyGMVtLteRIyIJDmvRHXBWTkVbAHQeWTNvWd4rLVTdZ+Gu/IOlYa7iNiAicCVQAdglIh0KKNdCHAvoKf6qerLy4Y1n0CHoVCnQc29r80XGsfDvtU1955KuZAjI/ceQJoxZpsx5jQwE7i6jHYvAK8COU6sT11oNnwJuZmuP7a9LE26wL41YLfX/Hsr5WS+DrRpBuwu8Tgd6FmygYh0BZobY+aJyMNOrE9dIHLyCli6NYO2P/ybAN9mjP1KgCU0DA4gOrwuPaIbcEnbcOoGOPIrW01NE+D3yXBkG4S3dd37KFUDHPmfUtYereKLToqID/BP4LZKOxIZC4wFaNGihWMVKq+WfjSL/yzezuer0onI3cmPASm8H3gr9YL8EBH2Z+awdGsG7y/ZTl1/G0MTmjG2b2uiw12wiFijjtbtgXUa7srjORLu6UDzEo+jgL0lHocAnYCFYh3Z0BiYIyJDjTErSnZkjJkETAJISkrSqxJfwE7l5vPWj1uYsnQHxhgGdWrCePkWs9mXO+5+ijuCI4vb5uYXsHLnUT5ftYfZyel8umI3o3q04OEr4ggN8nNeURHtQHzg4AboeI3z+lXKDRwJ99+BGBGJBvYANwB/KnrRGJMJhBc9FpGFwMOlg12pIit3HuXBT1LYdSSLa7tG8eDlsTQL9oHXv4C4K6FEsAME+Nq4uE04F7cJZ/ygOP71Yxof/baL7zcc4O/DO9MnJsI5hfkFQYPWVrgr5eEq3aFqjMkH7ga+BTYCnxhj1ovIBBEZ6uoClXf5cNkORry3jAK7YeaYXvxjRBeahQVB6leQdRgSb6tw+8iQQF64phOf33UxdQNs3Pz+b7z90xaMcdIfgpEd4ICGu/J8Du2dMsbMB+aXeu6Zctr2P/+ylLcpsBuen7ueact2MrBdJG/ckEBIYIkplVVTIbQ5tBngUH9dmofx1b19eGzWGl77bjObD5zkteFd8Pc9z/PyGnWEjXPhdBb41zm/vpRyIz1DVblcfoGd+z9OYdqynYzpE82kW5LODvYj22HbQuh6M/jYHO430M/GP0cmMH5QHHNW7+Uv/1tJTt55rg0T2QEwcCj1/PpRys003JVL5RXYuXdmMnNX7+XRQe148qoO2HxKHYC1aqq1I7PrTVXuX0QY178tLw7rxIJNB7lj6u/nF/BFR8zovLvycBruymWMMTw6aw3z1+7nqavac1f/Nuc2yj8Nyf+D2EEQ2qza73Vjz5b8/fou/JJ2mHtnJJNfUM0Tkeq3spYh0Hl35eE03JXL/P3bTXy+ag8P/iGWO/u0LrvRpvlw6hB0u+283+/6blE8+8cOfLfhAE99sa56O1l9bBARBwfXn3c9SrmTC0/3UxeyD3/dyTsLtzKqRwvuubSCE4JWfgD1oqDtZU5539G9o8k4mcvEBVtpGhbEvQNjqt5Jo46w5Xun1KOUu+jIXTndr9sO89yc9QxsF8kLV3dEylu298g22LYAEm+p0o7Uyjx8eRzDujbj9e838/2GA1XvILI9nDoIWUecVpNSNU3DXTnV3mPZ/HX6Klo2rMMbNyTga6vgV2zVNBAbJN7s1BpEhL9dG0/nqFAe+DiFtIMnqtZBeKx1m7HZqXUpVZM03JXT5OQV8Jf/rSQ3386km0sd7lhayR2p9Zo6vZZAPxvv3tSNQD8fxkxbyfGcPMc31nBXXkDDXTnNC/M2sCY9k9dHdKFtZHDFjTd95bQdqeVpGhbEOzd2Y9eRLJ6cXYUdrGEtrIuFHNrkstqUcjUNd+UU36zbz/TluxjbtzWXd2xc+QYrP7DOSG070KV19YhuwIN/iGXu6r18uiLdsY18bBAeAxlbXFqbUq6k4a7O277MbB77fA3xzUJ5+PK4yjc4vNU6I9XJO1LL85d+bbi4TUOenbPe8fn38FjI0JG78lwa7uq8FNgND3ycwul8O2/ekODY2i6/TQYfPyvca4DNR/jnyASC/G3c/VGyY2ewhsfC0Z2QpxcWU55Jw12dl3d/3sqv247w3NCOtI6oZJ4dIPeEtSO14zUQ4sD0jZM0qhfIP4Z3IXX/CV7/3oEdpRGxgIHDaS6vTSlX0HBX1bZp/wne+GEzV3VuwvBuUY5ttHomnD4BPf/i2uLKMKBdJH/q2YLJi7exYkclx7DrETPKw2m4q2opsBvGz1pDSKAfE4ZWcKJSSXY7LH8PmnWDqCTXF1mGJwa3p1lYEA99upqs0/nlN2zYFhANd+WxNNxVtUz5ZTurdx/juaEdaRgc4NhG2xbA4S3Q48+uLa4CwQG+/P36Luw8nMUrX1ewrK9fkHVIpIa78lAa7qrKdh4+xWvfbeKy9pH8sXMTxzdc/h7UjXT79UkvatOQ2y5uxdRlO1mallF+w4g4OKThrjyThruqEmMMj81ai5+PDy9c08mx6RiwDn/c8h0kjQZfB0f6LvTooHZEh9flkc/WcCq3nOmZ8FjrLw17NZcPVsqNNNxVlXz8+26WbTvM44Pb0yQ0yPENl/4LbP6QdIfriquCIH8br17fmT3Hsnntu3KOZw+PhfwcyNxVs8Up5QQa7sph+zNzePGrjfRq3YAbujd3fMOTByHlI0gYBSGNXFdgFXVv1YCbe7Xkg6U7SN519NwGxUfM6JmqyvNouCuHGGN46ot15NntvHxtZ3xKXyqvIsvfg4LTcNE9riuwmsYPiqNRSCCPzVrL6fxS0y8RhWfb6hozygNpuCuHzFuzjx82HuChP8TRKryu4xvmnoTfJ0P7IRBewUU73CQk0I//u6YTmw6c4N2ft579Yp0GUKehNe+ulIfRcFeVOnLqNM/NWU+XqFBG925VtY1XTYOcTOh9v0tqc4bLOjRiSOcmvP1T2rlrzzTUBcSUZ9JwV5V6Yd4GMrPzeOX6zhVffKO0/NOwbCK07O22k5Yc9ewfOxLkb+PRWWux20ssDayrQyoPpeGuKrQg9SCzk/cwbkBb2jWuV7WNU/4Hx9Ohz4OuKc6JIkICeHpIB1buPMr05TvPvBAea11yL7uMHa5K1WIa7qpcJ3LyeHL2WmIbBfPXAW2qtnF+Liz6B0T1gDauXbPdWa5LbEafmHBe/jqVvceyrSfDCy+wnaELiCnPouGuyvXKN6nsO57DK9d1JsC3iuuuJxeO2vs/Bo6e6ORmIsJLw+KxG3jqi8IrNxUdDqk7VZWH0XBXZVq+7TD/+3UXt/eOpmuL+lXbOD8XFr8OzXtCm0tdU6CLNG9Qh4cuj+Wn1IPMWb0Xwlpaa8/rGjPKw2i4q3Pk5BXw2OdraVEYdFWW/KHHjdpLGt07mi7Nw3h+7gYOZxdAg9a6U1V5HA13dY43ftjC9oxTvHxtPHX8fau2ce5J+PlVaN4LWg9wTYEuZvMRXr2uMydy8pgwb4MeMaM8koa7Osva9EwmL97GDd2bc3Hb8Kp3sOxtOHkA/jDBI0ftReIah/DXAW35MmUvO6QpHNkGBRWs/65ULeNQuIvIIBHZJCJpIvJYGa8/KCIbRGSNiPwoIi2dX6pytdP5dh75bDUN6/rz+OD2Ve/gxAH45S1oPxRa9HR+gTVsXP+2xDUKYdrmALDnwbGdlW+kVC1RabiLiA2YCFwJdABGiUiHUs2SgSRjTGfgM+BVZxeqXO+dhWmk7j/BS8PiCQ3yq3oHC1+Cgly47Dlnl+YW/r4+vHJ9Z1KyC/+C0Z2qyoM4MnLvAaQZY7YZY04DM4GrSzYwxiwwxmQVPvwVcPCCmqq22LjvOG//lMY1CU25rEM1Vm48mGotNZB0BzSs4jHxtVhC8zAu7tELgJ2bUtxcjVKOcyTcmwG7SzxOL3yuPHcAX5f1goiMFZEVIrLi0KFDjlepXCq/wM74z9YQVsePZ//YseodGANfPwL+IdBvvPMLdLNxg5M4TBjr1qwgJ6/A3eUo5RBHwr2svWKmjOcQkZuAJODvZb1ujJlkjEkyxiRFREQ4XqVyqUmLt7F2TyYTru5E/br+Ve9g3SzYvggGPg11q7ETtpar4++Lb2Qskad38c8fdGpGeQZHwj0dKHllhihgb+lGInIZ8CQw1BiT65zylKulHTzBGz9sYXB8YwbHV+F6qEVyjsO3T0KTBEi63fkF1hKhzTvQwe8AkxdtY/XuY+4uR6lKORLuvwMxIhItIv7ADcCckg1EpCvwHlawH3R+mcoViqZj6vjbeH5op+p1suAl69DHIa+DTxWXKPAkDWOoW5BJTPBpHvwkRadnVK1XabgbY/KBu4FvgY3AJ8aY9SIyQUSGFjb7OxAMfCoiKSIyp5zuVC3y7s9bWbXrGM8P7UhESDUuWr1rOSx/F7rfAc26Ob/A2qRwjZlX+wex9dApXv461c0FKVUxh04/NMbMB+aXeu6ZEvcvc3JdysXWpB/jjR+2MKRzE4Z2aVr1Dk5nwRd3QWhzrzn0sUKFq0N2CTrIbRd344OlO7isfSMuifG+fQzKO+gZqheg7NMF3P9xCuHBAbx4TTxSnTNJf/o/OLIVrn4bAkKcX2RtE9YCbAGQsYVHB7WjTURdHvlsNZnZee6uTKkyabhfgF7+eiPbDp3iHyO6EFqnGicrbV8Mv74D3cdA637OL7A28rFZx+9nbCHI38Y/RyZw6EQuz365zt2VKVUmDfcLzM+bDzF12U5u7x1N7+qsHXPyEMy6Exq2vTCmY0pq2LZ4XffOUWHcc2kMX6Ts5cuUPW4uTKlzabhfQA4cz+HBj1OIbRTM+EFxVe/AbofZYyHnGAz/AAKCnV5jrRYeC0e2W9eGBf46oA1JLevzxOdr2Z5xys3FKXU2DfcLRH6BnXtnJJN1uoCJf0ok0K8ahy0u+Qds/QkGvQyNq3nopCcLjwVTAEe3A+Br8+GtUV1xtoZbAAAQPUlEQVTx8/Vh3PRVenikqlU03C8Qb/64heXbj/B/13QiplE1doCmfgU/vQjxw6HbbU6vzyOEt7VuS6zt3jQsiNdHdGHjvuO8MG+DmwpT6lwa7heARZsP8faCNIZ3i+K6btVY023/Wpg1Bpp2haH/8uh12s9Lw6KLZZ+9BMGl7Rrx576tmb58F3NXn3PytlJuoeHu5dKPZnH/xynERAYz4epqTKUc3wczRkFgKNzwEfgFOb9ITxFYD0KalHlVpoeviCOxRRiPzVrDlgMn3FCcUmfTcPdiWafzGTNtJXn5dt65sRtB/lWcZ886Ah8Og+yjMOojqFeNtWe8TYkjZkrys/kw8cZEgvx9uXPaCo5lnXZDcUqdoeHupYwxPPzpalL3H+etP3WlbWQVj2zJPQHTr7cuL3fDR9aUjLJ2qmZstpY5LqVJaBDv3dyNfcdyuPujZPIL7G4oUCmLhruX+tdPacxfu5/Hr2zHgLjIqm2ckwnTh8PeFBg+5cI5UckR4THWv8+pjDJf7tayPv83rBNL0jJ4cf7GGi5OqTOqeGl75QlmJ6fz+vebGda1GWP6tK7axkVTMQfWwXX/gXZXuaZITxVeYqdqcNnXJBiR1JzUfSf47y/badGgDqN7R9dggUpZdOTuZX7efIhHPl3DRa0b8vJ1VVw35tgumDIYDm6EkdOh07WuK9RTFa4OWdn1VJ+8qj2DOjZmwrwNzFujR9Comqfh7kXWpB/jrv+tJKZRCO/d0o0A3yrsQN39G0y+FI7vhZs+g7hBrivUk9WLAt+gMo+YKcnmI7xxQwLdWzbgwY9XszSt7GkcpVxFw91LbDlwgtFTfqdBXX+mju5OvUAHFwQzxrqw9QdDwD8Y7vwBovu6tlhP5uMDEbFwqPL59EA/G5NvSaJVeB3GfriS5F1Ha6BApSwa7l5gy4ETjJr8KzYf4cM7ehJZL9CxDXMy4bPbYc490KIXjPnJCi5VsciOcGC9Q01D6/gx9fYeNAz255b3f2OVBryqIRruHq4o2H1EmDG2F9HhdR3bcNvP8G4f2PAlDHwGbp4NdRq4tlhv0aijdWnBco6YKa1JaBAzx/aiQbA/t2rAqxqi4e7BkncdZeSkM8HeJsKBY9mzjsAX42DaUBAfGP019HnIu69/6myNOli3Do7e4UzAF43gl27VOXjlWhruHuqn1AP8afJyggN8+eTPF1Ue7Hk5sPRt+FcirPnYCvRxy6BFz5op2Js0KlzG4WDVFgqzAv4imoYFcut/f2OOrkOjXEiPc/dA05fv5Jkv19O+SQhTbutR8cWt83OtMP/5VcjcDW0GwuUvWFMLqnqCI6FuhHUuQBU1Dg3k079czNhpK7h3RjJ7j2Xz576tq3epQ6UqoOHuQXLzC3huznpm/LabfrERTLwxkeCAcn6EWUdg5RRY/p41P9wkwbreaev+NVmy94rsUKVpmZJCg/yYdkcPHvpkNS9/ncq6PZm8en1n6vjrf0flPPrb5CH2HMvm7o9WkbzrGOP6t+Ghy+Ow+ZQa7dkLYOsCSJlurb9ekAttLoVh71mhrqND52nUCVb81/o3r8b+igBfG/8a1ZUOTevx2reb2HLgJP++KZHWjuw38UR2u/X7mJ8LBafBng9is/7tfGwl7vuBzU9/V51Aw72WM8bwRcoenvliPXZjeOfGRAbHl1idMS/bOvJl01ew6Ws4dQiC6kO3WyHx1gvzikk1oVEHyM+2LrtXdBGPKhIRxvVvS+dmYdwzYxVXvbWEJ65qz009W3jGNI29AI7ugGM7rbObj+2CzHTIOmytJJp1xLrNPWFdwcpRPr7gVxf864J/ncLbYPCrAwEh1u93UJh1Gxh25nHJ+351LvgPCA33WuzgiRyen7OBr9buI6llfV4fkUCLegI7lsCOX2DnEtj9uxUy/iEQcxl0uBriBoNvBfPw6vwV7bM4uL7a4V7kkphwvr6vL498tpqnv1jH9xsO8Ldr42kWVovWzjfGWnJh5y/WgnIH1lnLVORlnWkjNqjXFOqGQ1ADqB9tHV4bEAK2APD1B1vhl4+vFfh2e+FtvvVhYc+D01lw+hTknbJuix6fOmStUpp91LqOr6lg1U2bP9RpaNVRp4F1v/i26Pmi5wrv+wd71QeCmDKWLq0JSUlJZsWKFW5579ouv8DOR79s5tMfl9G4YB+jY7LoVXcfPvvXWmuJGzsg1tRAq94Q8wdo1UcDvSblZcNLzeCSB2Dg007p0hjD/37dyYvzNyIId1/aljv7RFdtGQlnOnUYNn9jfe1cClmFh28G1bd+9xrHQ2R7aNAawlpASFOw1dB40W6H0ycg+5gV9NlHS90v/Msh60jhXxJHzvxFUd6Hgs2/7NAPKvGhUPr5gJAa/0AQkZXGmKRK23lcuC9/Dxa+bF0RyDew7NtynwsCv8Bzb0u/5htgzfv5+J75Op8fYEFe4QjklDXSOX3SGo1kH7VGI1kZcCoDc+oQx/bvIP/wDiLM4bP7CG1+5j9U067Q8iLrP5lyn39fYh05c/PnTu02/WgWL8zbwLfrDxAdXpcH/hDLkPgm+JTex+IKOZmw9jNY9znsWmoFYUhTa59Ny4utwUT9aM8d4drt1gdA1pEzgZ91+MyHQPGUUonns4+U/4Hg41fqr4IGZXwYlHr+PD8QvDfcty6A1HnWcdv52dZtXhbk51ijqbNuC1+z551/weJTGPRFoW87c2tM4Z+WBdatMWfu2/Otr0rk+YVwsCCE9Px6HPVvSpvYjrSN64TUbwURcRrktdGXd1s7rsdvc0nYLdx0kL/NT2XTgRPENgrm/stiuaJj43N3pDvD7t/h9/9YZyznZ0N4HHQYai353CTBc8PcGYo+EEqHfvGHQYm/EEr+pVDRB8JVr1X7QvOOhrvnzbm3GWB9VYW9oIzgzy68n332B0V+trVH355vjbjtJQK6vC/xKfwqtedffKxbvxI7hkrsKMr2Cear7QVMXnmcTRmnaRYWxH2DY7i2azN8bXp+Wa3XtCskf2jtSKzf0und94+LpG9MBF+t3ccbP2xm3PRVNG8QxM29WjIyqQWhdRxcHK48xsCW7+GXN6y59IB60OUGSLwZmiZe2IFeko/PmemYhm0c28Zuh9zMs6eGSk4PNXL9gQ6eN3L3cPkFdhanZfBl8h6+23CArNMFdIkKZXTvaAbHN8HfV0PdY+xZBZMHwPCp0PEal75Vgd3w7fr9fLB0B79tP0KArw+XdWjE0C5N6R8XUfV5+e2L4funYW+ytYzxRX+FxFsgwEsPxfQi3jty90BHT51m0ZZD/LzpEAs3H+LIqdPUC/Tl6oSmDE9qTmILnXLxSI06Wn9i7012ebjbfITB8U0YHN+EDXuPM+O3XXy1dh9frdlHSKAvA+Ii6RsbQd+Y8IpXBT20Cb57GrZ8a+3Hufod6DzC2sekvIpDI3cRGQS8CdiA/xhjXi71egAwDegGHAZGGmN2VNSnt47cC+yG7RknSd51jOTdx0jedYzU/ccxBurX8aNPTARDOjehX3VGW6r2ea8fBIbCrXNq/K3zCuz8kpbB3NX7+HnzQTJOngYgtlEwiS3qk9A8jIQWYbSNCMbX5MHi12HxP6xjwPs+BD3+bB1EoDyK00buImIDJgJ/ANKB30VkjjGm5KpJdwBHjTFtReQG4BVgZPVKr/1y8ws4kJnLvsxs9h/PYXvGKdIOniTt4Em2Z5wiN9/akRIS6EtC8zDuHxhLv7gI4puFumZnmHKfZomwdpY1x+pTs1NqfjYf+sdF0j8uErvdsGHfcRZtOcTybUf4et1+Zv6+G4Duvlt5zX8SLe272Rh+Bdu6PUmDyCiaZBbQOLSAQD8dZHgjR6ZlegBpxphtACIyE7gaKBnuVwPPFd7/DHhbRMS4eELfGIPdWKNlu7G+rPtgL3yuwBjsdopfMwby7XZy8uzk5BeQW3xbYD2XV0BOXgEnc/PJzM7jWFaedZudx/HsPDJO5haPkIqIQFT9INpGBNMnJpyYRiEktgijdXhwzRy+ptwnqoe1DMHBDW49G9jHR+jULJROzUIZ19/6v7H94HGyF/yD9qlvc1QaMj7gaT7b0x57ejrWOM3SoK4/4cH+hAX5E1rHj7AgP8Lq+BEa5Ecdf1+C/G0E+dkI9LMR6OdDkJ+NIH8b/r4++PoINp+iWylx64PNZj32EaHov4GIIJzZV+sRZ+J6KEfCvRmwu8TjdKD0OrHFbYwx+SKSCTQEnL5o9eRF23jlm1QKjBXUrhTkZyO0xC96iwZ16NoijCahQTQODaRJ4VezsDoE+evo54LU8mLrdteyWrXUg5w8QOtvxsL2n6HTdTQc8gavBtZjQl4Be49lsy8zh32ZOezPzGZvZg6HT+aSmZ3H7iNZrCsc1GTnVWHJgPOttyjsOfMBUPS8IHDW64XP4bkH9Dz7xw6M7N7Cpe/hSLiX9c9XOlYdaYOIjAXGFj48KSKbHHj/2iYcF3xo1XIX2vdc9e/3+ZK/2rXNlMKvCl1oP2Nw4/d8wwtwQ/U3d+i4W0fCPR1oXuJxFFD6KgNFbdJFxBcIBY6U7sgYMwmY5EhhtZWIrHBkZ4Y3udC+5wvt+wX9nr2RI3uAfgdiRCRaRPyxPnBKHxowB7i18P71wE+unm9XSilVvkpH7oVz6HcD32IdCvlfY8x6EZkArDDGzAHeBz4UkTSsEft5/MWhlFLqfDl0EpMxZj4wv9Rzz5S4nwMMd25ptZZHTytV04X2PV9o3y/o9+x13Lb8gFJKKdfRhUyUUsoLabifBxF5WESMiIS7uxZXEpG/i0iqiKwRkdkiEubumlxFRAaJyCYRSRORx9xdj6uJSHMRWSAiG0VkvYjc5+6aaoqI2EQkWUTmubsWV9BwryYRaY61JMMud9dSA74HOhljOgObgcfdXI9LlFhq40qgAzBKRDq4tyqXywceMsa0B3oBf70Avuci9wEb3V2Eq2i4V98/gfGUcbKWtzHGfGeMKbriyK9Y5zp4o+KlNowxp4GipTa8ljFmnzFmVeH9E1hh18y9VbmeiEQBVwH/cXctrqLhXg0iMhTYY4xZ7e5a3OB24Gt3F+EiZS214fVBV0REWgFdgeXuraRGvIE1OKvgKtueTddzL4eI/AA0LuOlJ4EngMtrtiLXquj7NcZ8WdjmSaw/46fXZG01yKFlNLyRiAQDs4D7jTHH3V2PK4nIEOCgMWaliPR3dz2uouFeDmPMZWU9LyLxQDSwunBFuyhglYj0MMbsr8ESnaq877eIiNwKDAEGevHZx44steF1RMQPK9inG2Oce7Xv2qk3MFREBgOBQD0R+Z8x5iY31+VUepz7eRKRHUCSMcZrF10qvFjL60A/Y8whd9fjKoXrIm0GBgJ7sJbe+JMxZr1bC3MhsUYoU4Ejxpj73V1PTSscuT9sjBni7lqcTefclSPeBkKA70UkRUTedXdBrlC407hoqY2NwCfeHOyFegM3A5cW/mxTCke0ysPpyF0ppbyQjtyVUsoLabgrpZQX0nBXSikvpOGulFJeSMNdKaW8kIa7Ukp5IQ13pZTyQhruSinlhf4ffbroZXm8U1EAAAAASUVORK5CYII=\n", 282 | "text/plain": [ 283 | "
" 284 | ] 285 | }, 286 | "metadata": {}, 287 | "output_type": "display_data" 288 | } 289 | ], 290 | "source": [ 291 | "plot_d0(D,input_node)\n", 292 | "#plt.savefig('fig2.png')" 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": 14, 298 | "metadata": {}, 299 | "outputs": [], 300 | "source": [ 301 | "# copy the learned weights over into a tmp array\n", 302 | "weightsD=sess.run(theta)" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": 15, 308 | "metadata": {}, 309 | "outputs": [], 310 | "source": [ 311 | "# close the pre-training session\n", 312 | "sess.close()" 313 | ] 314 | }, 315 | { 316 | "cell_type": "markdown", 317 | "metadata": {}, 318 | "source": [ 319 | "# Build Net\n", 320 | "\n", 321 | "Now to build the actual generative adversarial network" 322 | ] 323 | }, 324 | { 325 | "cell_type": "code", 326 | "execution_count": 16, 327 | "metadata": {}, 328 | "outputs": [], 329 | "source": [ 330 | "with tf.variable_scope(\"G\"):\n", 331 | " z_node=tf.placeholder(tf.float32, shape=(M,1)) # M uniform01 floats\n", 332 | " G,theta_g=mlp(z_node,1) # generate normal transformation of Z\n", 333 | " G=tf.multiply(5.0,G) # scale up by 5 to match range\n", 334 | "with tf.variable_scope(\"D\") as scope:\n", 335 | " # D(x)\n", 336 | " x_node=tf.placeholder(tf.float32, shape=(M,1)) # input M normally distributed floats\n", 337 | " fc,theta_d=mlp(x_node,1) # output likelihood of being normally distributed\n", 338 | " D1=tf.maximum(tf.minimum(fc,.99), 0.01) # clamp as a probability\n", 339 | " # make a copy of D that uses the same variables, but takes in G as input\n", 340 | " scope.reuse_variables()\n", 341 | " fc,theta_d=mlp(G,1)\n", 342 | " D2=tf.maximum(tf.minimum(fc,.99), 0.01)\n", 343 | "obj_d=tf.reduce_mean(tf.log(D1)+tf.log(1-D2))\n", 344 | "obj_g=tf.reduce_mean(tf.log(D2))\n", 345 | "\n", 346 | "# set up optimizer for G,D\n", 347 | "opt_d=momentum_optimizer(1-obj_d, theta_d)\n", 348 | "opt_g=momentum_optimizer(1-obj_g, theta_g) # maximize log(D(G(z)))" 349 | ] 350 | }, 351 | { 352 | "cell_type": "code", 353 | "execution_count": 17, 354 | "metadata": {}, 355 | "outputs": [], 356 | "source": [ 357 | "sess=tf.InteractiveSession()\n", 358 | "tf.global_variables_initializer().run()" 359 | ] 360 | }, 361 | { 362 | "cell_type": "code", 363 | "execution_count": 18, 364 | "metadata": {}, 365 | "outputs": [], 366 | "source": [ 367 | "# copy weights from pre-training over to new D network\n", 368 | "for i,v in enumerate(theta_d):\n", 369 | " sess.run(v.assign(weightsD[i]))" 370 | ] 371 | }, 372 | { 373 | "cell_type": "code", 374 | "execution_count": 19, 375 | "metadata": {}, 376 | "outputs": [], 377 | "source": [ 378 | "def plot_fig():\n", 379 | " # plots pg, pdata, decision boundary \n", 380 | " f,ax=plt.subplots(1)\n", 381 | " # p_data\n", 382 | " xs=np.linspace(-5,5,1000)\n", 383 | " ax.plot(xs, norm.pdf(xs,loc=mu,scale=sigma), label='p_data')\n", 384 | "\n", 385 | " # decision boundary\n", 386 | " r=5000 # resolution (number of points)\n", 387 | " xs=np.linspace(-5,5,r)\n", 388 | " ds=np.zeros((r,1)) # decision surface\n", 389 | " # process multiple points in parallel in same minibatch\n", 390 | " for i in range(r//M):\n", 391 | " x=np.reshape(xs[M*i:M*(i+1)],(M,1))\n", 392 | " ds[M*i:M*(i+1)]=sess.run(D1,{x_node: x})\n", 393 | "\n", 394 | " ax.plot(xs, ds, label='decision boundary')\n", 395 | "\n", 396 | " # distribution of inverse-mapped points\n", 397 | " zs=np.linspace(-5,5,r)\n", 398 | " gs=np.zeros((r,1)) # generator function\n", 399 | " for i in range(r//M):\n", 400 | " z=np.reshape(zs[M*i:M*(i+1)],(M,1))\n", 401 | " gs[M*i:M*(i+1)]=sess.run(G,{z_node: z})\n", 402 | " histc, edges = np.histogram(gs, bins = 10)\n", 403 | " ax.plot(np.linspace(-5,5,10), histc/float(r), label='p_g')\n", 404 | "\n", 405 | " # ylim, legend\n", 406 | " ax.set_ylim(0,1.1)\n", 407 | " plt.legend()" 408 | ] 409 | }, 410 | { 411 | "cell_type": "code", 412 | "execution_count": 20, 413 | "metadata": {}, 414 | "outputs": [ 415 | { 416 | "data": { 417 | "text/plain": [ 418 | "Text(0.5,1,'Before Training')" 419 | ] 420 | }, 421 | "execution_count": 20, 422 | "metadata": {}, 423 | "output_type": "execute_result" 424 | }, 425 | { 426 | "data": { 427 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VMX6wPHvZNNJSIAAQRJIKKEmtFCUIogFEFFsiKKASLUrYruWn17rvWK54oUAF1ARrAiiUlRQQEWB0AkQQgskgQAJ6WV3fn+cEEJI2YTdbHZ5P8+zz+6enTPnXdi8OztnzozSWiOEEMK1uDk6ACGEELYnyV0IIVyQJHchhHBBktyFEMIFSXIXQggXJMldCCFckCR3UasppSYrpVKUUplKqQaOjqcqlFJzlFLP2bqsENZQMs5d2JNS6hDQGDADBcDvwCSt9VEr9vUAzgK9tNbb7BznTGBU0VNPQAF5Rc/Xaa0H2/P4QtiaJHdhV0XJ/QGt9U9KKW/gI6C+1voWK/YNAY4CHlrrwioeV2F8vi3ViPlloJXWelQFZdyrGpMQNUm6ZUSN0VrnAl8B7c9tU0p5KaX+rZQ6UtT9MlMp5aOUigD2FhVLU0r9UlT+KqXU30qp9KL7q0rUtVYp9ZpSagOQDbRQSgUopeYqpZKUUseUUv9USpmqGrtSqpVSSiulxiqljgCrlFJuSqmvlFLJSqm0ouO3K7HPp0VfFCilrlVKHVJKTVNKnVRKHVdK3VfNsg2VUt8rpc4qpf5SSr2ulFpb1fckXJskd1FjlFK+wAjgzxKb3wIigM5AK6Ap8KLWeh/QoahMoNb6GqVUfeB74AOgATAd+L5UX/y9wATAHzgMLAAKi+ruAlwPPHAJb6Mf0Ba4sej5cqA1EAzsBD6pYN8QwAe4ApgE/FcpVbcaZf8LpGF0d90PjK7umxGuS5K7qAnfKqXSMPrPrwP+BcVdJ+OBx7XWp7XWGcDrwF3l1HMjsF9r/YnWulBrvQiIA24qUWa+1npXUZdJfWAw8JjWOktrfQJ4t4L6rfGS1jpba52jtbZoredrrTOKfpW8DHRTStUpZ99c4J9a6wKt9TKMPv2IqpQtOg9xC8YXYI7WurIvFHGZcnd0AOKycEtRn7sJuBn4VSnVHrAAvsBmI88DxonM8rpNrsBojZd0GKO1f07JE7XNAQ8gqUT9bqXKVFXxvkXv5w3gdiAI4/1Q9DirjH1TtdbmEs+zAb9yjlNe2cYY/z4l38NRoFcV3oO4DEjLXdQYrbVZa/0NxsiZPkAqkAN00FoHFt0CtNblJbzjGAm7pGbAsZKHKfH4KEaLN6hE/XW11h2oJn3hCIT7gCHANUAARtcPGF9Q9pKC8SUSUmJbqB2PJ5yUJHdRY5ThZqAesKdoJMts4F2lVKOiMk2VUjeUU8UPGF0Tdyul3JVSIzBOzi4vq7DWOglYBbyjlKpbdAK0pVLqahu9JX+ML49TGL9AXrNRveXSWhcA3wL/V3TiuQPnh3AKUUySu6gJ3ymlMjH63F8DRmutdxW99jQQD/yplDoL/AS0KasSrfUpYCjwJEZCnQYM1VqnVnDs+zDGre8GzmCM1mlyye/IMA/j18RxYBfGGP6aMBnjhHJKUQyLOD8mXwhAxrkL4fSUUu9gjCga5+hYRO0hLXchnIxSqr1SKrKom6sXMBZY4ui4RO0io2WEcD51gYUY3UspwJta6zLPO4jLl3TLCCGEC5JuGSGEcEEO65YJCgrSYWFhjjq8EEI4pc2bN6dqrRtWVs5hyT0sLIxNmzY56vBCCOGUlFKlr9Iuk3TLCCGEC5LkLoQQLkiSuxBCuCAZ5y5ELVZQUEBiYiK5ubmODkXUMG9vb0JCQvDw8KjW/pLchajFEhMT8ff3JywsjBLTFgsXp7Xm1KlTJCYmEh4eXq06pFtGiFosNzeXBg0aSGK/zCilaNCgwSX9YpPkLkQtJ4n98nSp/++S3IUQwgVJchdCCBckyV0IYRdr165l6NChFZbZunUrP/zwQw1FdHmR5C6EcBhJ7vYjQyGFcBL/990udh8/a9M6219Rl5duqni98EOHDjFo0CB69uxJbGwsERERfPzxx/j6+l5UdsWKFTz22GMEBQXRtWvX4u1//fUXjz32GDk5Ofj4+DBv3jzCw8N58cUXycnJYf369Tz77LOEh4dfVK5NmzJXXRSVkJa7EKJSe/fuZcKECWzfvp26devy0UcfXVQmNzeX8ePH891337Fu3TqSk5OLX2vbti2//fYbsbGxvPLKKzz33HN4enryyiuvMGLECLZu3cqIESPKLCeqR1ruQjiJylrY9hQaGkrv3r0BGDVqFB988AFTp069oExcXBzh4eG0bt26uFxMTAwA6enpjB49mv3796OUoqCgoMzjWFtOVE5a7kKISpUec13eGOzytr/wwgsMGDCAnTt38t1335V7cY615UTlKk3uSqn/KaVOKKV2lvO6Ukp9oJSKV0ptV0p1LaucEMJ5HTlyhD/++AOARYsW0adPn4vKtG3bloMHD3LgwIHicuekp6fTtGlTAObPn1+83d/fn4yMjErLiaqzpuU+HxhUweuDgdZFtwnAfy89LCFEbdKuXTsWLFhAVFQUp0+fZvLkyReV8fb2JiYmhhtvvJE+ffrQvHnz4temTZvGs88+S+/evTGbzcXbBwwYwO7du+ncuTOff/55ueVE1Vm1QLZSKgxYrrXuWMZrs4C1WutFRc/3Av211kkV1RkdHa1lJSYhKrZnzx7atWvn0BgOHTrE0KFD2bmzzB/vwo7K+v9XSm3WWkdXtq8t+tybAkdLPE8s2nYRpdQEpdQmpdSmkydP2uDQQgghymKL0TJlnUEp8+eA1joGiAGj5W6DYwsh7CwsLOyiVvvw4cM5ePDgBdveeustbrjhhpoMTVTAFsk9EQgt8TwEOG6DeoUQtdSSJUscHYKohC26ZZYB9xWNmukFpFfW3y6EEMK+Km25K6UWAf2BIKVUIvAS4AGgtZ4J/AAMAeKBbGCsvYIVQghhnUqTu9Z6ZCWva+BBm0UkhBDikskVqkIIq7388sv8+9//rta+V111VYWvDxkyhLS0tGrVXdKYMWP46quvLrmeqvDz86vR41lD5pYRQtSI33//vcLXL5epf7XWaK1xc7Nv21pa7kKICr322mu0adOGa6+9lr179xZvP3DgAIMGDaJbt2707duXuLg4AFJSUhg+fDidOnWiU6dOxUn9XOs2KSmJfv360blzZzp27Mi6desAY8hlamoqANOnT6djx4507NiR9957DzAupmrXrh3jx4+nQ4cOXH/99eTk5JQZ808//UTfvn2JiIhg+fLlgDFr5dixY4mMjKRLly6sWbMGMKY5eOihh4r3HTp0KGvXri2O+fnnn6dTp0706tWLlJQUAA4ePMiVV15J9+7deeGFF4r3zczMZODAgXTt2pXIyEiWLl16QexTpkyha9euvPrqqzz++OPF+82ePZsnnniiyv83FZGWuxDO4sdnIHmHbesMjoTBb5b78ubNm1m8eDGxsbEUFhbStWtXunXrBsCECROYOXMmrVu3ZuPGjUyZMoVffvmFRx55hKuvvpolS5ZgNpvJzMy8oM7PPvuMG264geeffx6z2Ux2dvZFx5w3bx4bN25Ea03Pnj25+uqrqVevHvv372fRokXMnj2bO++8k6+//ppRo0ZdFPehQ4f49ddfOXDgAAMGDCA+Pp4ZM2YAsGPHDuLi4rj++uvZt29fhf88WVlZ9OrVi9dee41p06Yxe/Zs/vGPf/Doo48yefJk7rvvvuJ6wZiCYcmSJdStW5fU1FR69erFsGHDAGPa5Hnz5vHRRx+RlZVFVFQUb7/9Nh4eHsybN49Zs2ZVGEtVSXIXQpRr3bp1DB8+vHhhjnOJKjMzk99//5077rijuGxeXh4Av/zyCx9//DEAJpOJgICAC+rs3r07999/PwUFBdxyyy107tz5gtfXr1/P8OHDqVOnDgC33nor69atY9iwYYSHhxeX79atG4cOHSoz7jvvvBM3Nzdat25NixYtiIuLY/369Tz88MOAMclZ8+bNK03unp6exUsFduvWjdWrVwOwYcMGvv76awDuvfdenn76acDocnnuuef47bffcHNz49ixY8Wt/ebNm9OrVy8A6tSpwzXXXMPy5ctp164dBQUFREZGVhhLVUlyF8JZVNDCtqeypvG1WCwEBgaydevWKtfXr18/fvvtN77//nvuvfdennrqKe67777i1yua78rLy6v4sclkKrdbpqwpisur193dHYvFUvy85DTDHh4exXWZTCYKCwvLPQbAwoULOXnyJJs3b8bDw4OwsLDi+s59WZ3zwAMP8Prrr9O2bVvGjrX9CHLpcxdClKtfv34sWbKEnJwcMjIy+O677wCoW7cu4eHhfPnll4CRkLdt2wbAwIED+e9/jclhzWYzZ89euDTg4cOHadSoEePHj2fcuHFs2bLlomN+++23ZGdnk5WVxZIlS+jbt2+V4v7yyy+xWCwcOHCAhIQE2rRpQ79+/Vi4cCEA+/bt48iRI7Rp04awsDC2bt2KxWLh6NGj/PXXX5XW37t3bxYvXgxQXCcYUxY3atQIDw8P1qxZw+HDh8uto2fPnhw9epTPPvuMkSMrHHFeLZLchRDl6tq1KyNGjKBz587cdtttFyTZhQsXMnfuXDp16kSHDh2KTx6+//77rFmzhsjISLp168auXbsuqHPt2rV07tyZLl268PXXX/Poo49edMwxY8bQo0cPevbsyQMPPECXLl2qFHebNm24+uqrGTx4MDNnzsTb25spU6ZgNpuJjIxkxIgRzJ8/Hy8vL3r37k14eDiRkZFMnTr1grVfy/P+++8zY8YMunfvTnp6evH2e+65h02bNhEdHc3ChQtp27ZthfXceeed9O7dm3r16lXp/VnDqil/7UGm/BWicrVhyl9hP0OHDuXxxx9n4MCBZb7u6Cl/hRBCVEFaWhoRERH4+PiUm9gvlZxQFUKIGhYYGFjpSJ1LJS13IYRwQZLchRDCBUlyF0IIFyTJXQghXJAkdyGEcEGS3IUQwgVJchdCCBck49yFcBJv/fUWcafjbFpn2/ptebrH0xWWOXToEIMGDaJnz57ExsYSERHBxx9/XDxTZEk//PADTzzxBEFBQXTt2pWEhITi+dRFzZKWuxCiUnv37mXChAls376dunXr8tFHH11UJjc3l4kTJ/Ljjz+yfv16Tp486YBIxTnSchfCSVTWwran0NBQevfuDcCoUaP44IMPmDp16gVl4uLiaNGiBeHh4QCMHDmSmJiYGo9VGKTlLoSoVFnzo5fmqEkIRdkkuQshKnXkyBH++OMPABYtWkSfPn0uKtO2bVsSEhKKV0f6/PPPazJEUYokdyFEpdq1a8eCBQuIiori9OnTTJ48+aIyPj4+fPTRRwwaNIg+ffrQuHHji5bYEzVH+tyFEJVyc3Nj5syZlZYbMGAAcXFxaK158MEHiY6udNpxYSfSchdC2Mzs2bPp3LkzHTp0ID09nYkTJzo6pMuWtNyFEBUKCwtj586dF2wbPnw4Bw8evGDbW2+9xeOPP87jjz9ek+GJckhyF6KW01qXOTrFkZYsWeLoEFzepY4+km4ZIWoxb29vTp06JcMMLzNaa06dOoW3t3e165CWuxC1WEhICImJiXK152XI29ubkJCQau8vyV2IWszDw6P4ik8hqsKqbhml1CCl1F6lVLxS6pkyXm+mlFqjlIpVSm1XSg2xfahCCCGsVWlyV0qZgBnAYKA9MFIp1b5UsX8AX2ituwB3ARfPKiSEEKLGWNNy7wHEa60TtNb5wGLg5lJlNFC36HEAcNx2IQohhKgqa5J7U+BoieeJRdtKehkYpZRKBH4AHi6rIqXUBKXUJqXUJjlBJIQQ9mNNci9rgG3pcVkjgfla6xBgCPCJUuqiurXWMVrraK11dMOGDaserRBCCKtYk9wTgdASz0O4uNtlHPAFgNb6D8AbCLJFgEIIIarOmuT+N9BaKRWulPLEOGG6rFSZI8BAAKVUO4zkLv0uQgjhIJUmd611IfAQsBLYgzEqZpdS6hWl1LCiYk8C45VS24BFwBgtl9QJIYTDWHURk9b6B4wTpSW3vVji8W6gt21DE0IIUV0yt4wQQrggSe5CCOGCJLkLIYQLkuQuhBAuSJK7EEK4IEnuQgjhgiS5CyGEC5LkLoQQLkiSuxBCuCBJ7kII4YIkuQshhAuS5C6EEC5IkrsQQrggSe5CCOGCJLkLIYQLkuQuhBAuSJK7EEK4IEnuQgjhgiS5CyGEC5LkLoQQLkiSuxBCuCBJ7kII4YIkuQshhAuS5C6EEC5IkrsQQrggSe5CCOGCJLkLIYQLkuQuhBAuSJK7EEK4IEnuQgjhgqxK7kqpQUqpvUqpeKXUM+WUuVMptVsptUsp9ZltwxRCCFEV7pUVUEqZgBnAdUAi8LdSapnWeneJMq2BZ4HeWuszSqlG9gpYCCFE5axpufcA4rXWCVrrfGAxcHOpMuOBGVrrMwBa6xO2DVMIIURVWJPcmwJHSzxPLNpWUgQQoZTaoJT6Uyk1qKyKlFITlFKblFKbTp48Wb2IhRBCVMqa5K7K2KZLPXcHWgP9gZHAHKVU4EU7aR2jtY7WWkc3bNiwqrEKIYSwkjXJPREILfE8BDheRpmlWusCrfVBYC9GshdCCOEA1iT3v4HWSqlwpZQncBewrFSZb4EBAEqpIIxumgRbBiqEEMJ6lSZ3rXUh8BCwEtgDfKG13qWUekUpNayo2ErglFJqN7AGeEprfcpeQQshhKiY0rp093nNiI6O1ps2bXLIsYWLy0qFPd9B+lGo3wLa3wxe/o6OSgibUEpt1lpHV1au0nHuQjiVTf+DVS9AfiYoN9AW+OlluG0utLja0dEJUWNk+gHhGrSG1S/C8schJBomrYcXTsG41eBTHxbeAYc2ODpKIWqMJHfhGv78CDa8D9HjYNQ3EBwJbm4Q2gPuXwH1msOXoyFLTgWJy4Mkd+H8Dq2Hlc9Du5tgyL/BzXTh67714Y75kJMGK8qcGkkIlyPJXTi3vAz4djLUD4fhs4zWelkad4Dej8COLyBpW83GKIQDSHIXzu2nlyE9EW6ZCZ51Ki571SPgHQhrXq+R0IRwJEnuwnml7DJGx3QfD816Vl7eJxB6ToJ9K+DUAfvHJ4QDSXIXzklrWPkceNWF/lXoR+82Btzc4e+5dgtNiNpAkrtwTvE/QcJa6P+sccLUWnWbGCdety6Ewny7hSeEo0lyF85Ha1j7JgQ2g+7jqr5/p5GQmwYHfrZ9bELUEpLchfM58Asc2wR9ngCTR9X3b3mNcWHTjq9sH5sQtYQkd+FctIZf34K6IdD57urVYfIw5pvZ+wMU5Ng2PiFqCUnuwrkcWgdHN0Kfx8Ddq/r1tLsJCrLh4DrbxSZELSLJXTiXPz4C3yDocu+l1RPWBzx8Yf9K28QlRC0jyV04j1MHjDHq3ceBh/el1eXuBS36w75VRlePEC5GkrtwHn/FGGPUo6sxQqYsra+H9CNwMs429QlRi0hyF84hNx1iP4WOt4F/Y9vU2epa4z5hrW3qE6IWkeQunEPsQmMBjl6TbFdnYCjUCzNmlRTCxUhyF7WfxQJ/zYJmV8IVXWxbd/M+cHiDcQwhXIgkd1H7JayBM4egx3jb1x3WB3LOwIndtq9bCAeS5C5qvy0LjCtK2w61fd1hvY176ZoRLkaSu6jdMk9C3A/G1aiXctFSeQKbGbfDktyFa5HkLmq3bYvAUgBd77PfMUJ7QeIm+9UvhANIche1l9aw5WMj+TZsY7/jhERDRhKkH7PfMYSoYZLcRe11+Hc4tR+6jbbvcZpGG/fHpPUuXIckd1F7bVlgrLTU/hb7Hie4I5g8pWtGuBRJ7qJ2yjkDu5dC5B3g6WvfY7l7QXAUHNts3+MIUYMkuYvaafuXUJhr/y6Zc5p2g+OxYC6smeMJYWeS3EXto7XRJdOkMzTpVDPHDIk25neXScSEi5DkLmqfY1sgZad9hz+Wdm5ag6RtNXdMIexIkruofbbMNxbSiLyj5o5Zv4VxzOQdNXdMIezIquSulBqklNqrlIpXSj1TQbnblVJaKRVtuxDFZSUvA3Z8DR1uBe+6NXdcNxM07iDJXbiMSpO7UsoEzAAGA+2BkUqp9mWU8wceATbaOkhxGdn5NRRk1dyJ1JKCI43kLiszCRfgbkWZHkC81joBQCm1GLgZKD2N3qvA28BUm0YoLgu5BWZ+P5BK219moT2aM2VpPrCeBn5ehAfVoUd4ffq0CqKOlzUf2WoKjoRN/4O0I1Cvuf2OI0QNsOYvpSlwtMTzRKBnyQJKqS5AqNZ6uVKq3OSulJoATABo1qxZ1aMVLifxTDZz1h3kmy2JXJGXwAqv3cz0foC6Ph4opUhOz+X3A6nMXX+QOp4mhnVuyoR+LQgPqmP7YIKjjPvkHZLchdOzJrmrMrYV/25VSrkB7wJjKqtIax0DxABER0fLb9/LWFZeIR/8vJ95vx9Ca82gjk14Rn+HPuDJpEeeZ5Jv/eKyeYVmNh8+wzdbjrEkNpEvNx1lZI9mTL2hDQE+HrYLqlF7UG5Gcm9nh+mFhahB1iT3RCC0xPMQ4HiJ5/5AR2CtUgogGFimlBqmtZbrucVFNh8+wxNfbOXI6Wxu7RLCE9dH0LQO8M4yaHcTlEjsAF7uJq5qGcRVLYOYNqgN//k5ns/+OsLq3Sn8644o+rZuaJvAPH2hQSs5qSpcgjWjZf4GWiulwpVSnsBdwLJzL2qt07XWQVrrMK11GPAnIIldlOmTPw5x56w/MFs0i8f34p07O9E00MeYaiA3HbpWfCK1kb83r97SkW8mX0UdLxP3zv2LD3/Zj7bVSdDgSEjebpu6hHCgSpO71roQeAhYCewBvtBa71JKvaKUGmbvAIVrMFs0Ly7dyQtLd9E/oiE/PtqXni0anC+weQHUC4ewvlbV1yk0kO8f6cstna/g36v28ejireQX2mAd1EbtIf0o5J699LqEcCCrhh5orX8Afii17cVyyva/9LCEKyk0W3j8i218t+044/uG88zgdpjcSpzKSd0PR36HgS+Bm/XX1Xl7mHh3RGcigv15e8VeMvMK+eiernh7mKofbKN2xv3JvRDavfr1COFgcoWqsKsCs4VHFsfy3bbjPD2oLc/f2P7CxA7GPDJu7tD5nirXr5RiSv9WvDa8I2v2nmDcgr/JLTBXP+CGbY17mWNGODlJ7sJutNY8/fV2ftiRzD9ubMfk/i0vLlSYD1sXQcQg8G9c7WPd07M5/7q9ExviT/HIolgKzdXsoqkXBu7ektyF05PkLuzmXyv38s2WYzxxXQQP9G1RdqG930N2KnQbc8nHu71bCC/d1J5Vu1P4x7c7q3eS1c0EQa3hxJ5LjkcIR7Lj5X7icvbJn4f5aO0BRvZoxsPXtCq/4OYFEBAKLa+xyXHH9g4nNTOPGWsOcEWgD48MbF31Shq2M5b4E8KJSctd2NyfCad4edkuBrZtxKs3d6Do+oeLnTkECWugyyijxWwjU69vw/AuTZm+eh+rd6dUvYJGbeFsooyYEU5NkruwqeNpOTy4cAvNG/jy3l2dcTdV8BGL/dS4IrTLKJvGoJTijVsjiQoJ4PHPtxJ/IqNqFTQsMWJGCCclyV3YTG6BmUmfbiav0ELMvdH4e1cwNYC5wEjura6FgBCbx+LtYWLmqG54e7gx/uPNnM0tsH7nhm2MezmpKpyYJHdhM68u3832xHSm39mJVo38Ki4c9z1kJEG3sXaL54pAHz66pxtHTmfz/JIqnGCVETPCBUhyFzaxYmcyCzceYUK/FlzfIbjyHf6eAwHNIOIGu8bVI7w+T1wXwXfbjvPlpkTrdnIzQVCEjJgRTk2Su7hkSek5PPPNdiKbBjD1+jaV73AiDg6tg+ixNj2RWp5JV7fkqpYNeGnZLuv73xu1k5a7cGqS3MUlMVs0j39uzOvy/l2d8XS34iP19xwwedbYAtgmN8W7Izrj42nioc9irbuCNSgCzh6DvEz7ByiEHUhyF5dk5q8H+DPhNC8P60CLhpX0s4OxRuq2xcYaqXWC7B9gkcZ1vXnnjk7EJWcwffW+yncIKhoffyrevoEJYSeS3EW17U3O4L2f9nFjVBPu6GbliJdtiyE/A3qMt29wZRjQthF392zG7HUJbDp0uuLCQRHGfep++wcmhB1IchfVYrZopn29HX9vD14ZVsGFSiVpbXTJNOkMTbvZP8gyPDekHU0DfXjyy21k5xeWX7B+C2MM/ilJ7sI5SXIX1TJvw0G2HU3j5WEdaODnZd1OB38zTlL2GA/WfBnYgZ+XO/+6vROHT2Xz1o8VnDB194LAZtJyF05LkruossOnsvj3qr1c264RN0U1sX7HPz6EOg2h4+32C84KV7ZswJirwljwx2F+j08tv2CD1tJyF05LkruoEq01z3y9Aw83N169paN13TFgXMq/fxX0mAAe3vYN0gpPD2pLeFAdnvpqO1l55XTPBEVAajxYbLDCkxA1TJK7qJLP/z7KHwmneHZIO5oE+Fi/4x8fgrsPRI+zX3BV4ONp4u3boziWlsO/V5Uzh0xQKyjMMYZECuFkJLkLqyWn5/La93vo1aI+d3UPtX7HzBOw7XPoPBLqNKi8fA3pHlafe3s1Z/7vh4g9cubiAg3ODYeUrhnhfCS5C6torfnHtzspsFh489Yo3EovlVeRv2aDOR96PWi/AKtp2qA2NPb35pmvd1y8wPa5se5yUlU4IadL7tkF2fyW+Jujw7jsLN+exE97UnjyujaEBdWxfsf8bGP4Y5shRjdHLePv7cE/b+nI3pQMZv564MIX/RqDV11J7sKm1hxZQ545z+7HcbrkPmfHHB78+UE+j/vc0aFcNk5n5fPysl10CglgbO+wqu28ZQHknIarHrZLbLZwbfvGDI1qwoe/xF8494xS0KCVdMsIm5m3cx6PrHmET3Z/YvdjOV1yn9hpIv1D+vPPjf9k3s55jg7nsvDq8t2k5xTw1u1RFS++UVpBLqx/D8L6QvMr7RegDbx0Uwd8PE08/fUOLJYSUwMHtZaWu7hkWms+jP2Q6ZunMyhsEKMhPv1dAAAgAElEQVQ7jLb7MZ0uuXuZvJg+YDqDwwYzffN0/hP7n+othCyssibuBEtijzFlQCvaBtet2s6xn0BmMvR7yj7B2VBDfy9eGNqezYfPsHDj4fMvNGhtjJbJz3JccMKpaa15+++3mbV9Fre2vpU3+76Jh1sFC9nYiNMldwAPNw/e6PsGt7W+jZjtMbz191tYtIxFtrWM3AKeX7KDiMZ+PDigZdV2Lsw3Wu2hPSG8n30CtLHbujalb+sg3vwxjuNpOcZGmUBMXAKzxczLf7zMp3s+ZVS7Ubx85cuYamCaa3DS5A5gcjPx0pUvcW/7e1m4ZyEv//4yZosVU7kKq721Io6ks7m8dVsUXu5V/EBuW2QsMt1vmsOmGqgqpRSvD4/EouEf3xat3CQjZkQ1FZgLeGbdM3yz/xsmdZrEtO7TrL/ozwacNrmD8cf4VPRTTO40mSXxS5j22zQKzFVYK1OUa2PCKT798wj39w6nS7N6Vdu5MB/WvQNXdIFWA+0ToJ2E1vflyesj+CXuBMu2HTcmEENJchdVkluYy2NrH2PFoRU82e1JHuz8YI0mdnDy5A5Ggp/SeQpTo6ey6vAqHlv7GLmFuY4Oy6nlFph55psdNCtKdFW2eT6kHYYB/3CaVntJY3uH0yk0kP/7bjen8twgMFRGzAirZRVk8eDPD7IucR0v9HqBMR3HOCQOp0/u54zuMJoXr3yRdYnrmPLzFLIK5ARYdb33034Opmbx5q2R+Hq6V23nvEz47W1jhIyTtdrPMbkp3r4tiozcAl5ZvrtojhkrFvgQl730vHQmrJrA5pTNvN73de5sc6fDYnGZ5A5wR8QdvNH3DbakbGH8qvGk56U7OiSnsyMxndnrErireyhXtarGSkl/zICsk3Dty07Zaj+nTbA/Dw5oxdKtxzmimsKpAzKBmKhQak4q96+8nz2n9/BO/3cY2mKoQ+OxKrkrpQYppfYqpeKVUs+U8foTSqndSqntSqmflVLNbR+qdW5scSPT+08n7nQc96+8n9ScCqZ0FRfIL7Tw1FfbaFDHk2eHtKt6BVmp8PsH0O4mCIm2fYA1bEr/VrRp7M+iBG8oyIaM444OSdRSyVnJjF0xlqMZR/lw4IcMbOb4X62VJnellAmYAQwG2gMjlVLtSxWLBaK11lHAV8Dbtg60Kq5pdg0zBs7gaMZRxq4YS1JmkiPDcRofrY0nLjmD14dHEuBTjXG4a980kuA1L9o+OAfwdHfjrduj2JrT0NggXTOiDEfOHmH0j6NJzUll1nWzuOqKqxwdEmBdy70HEK+1TtBa5wOLgZtLFtBar9FaZxc9/ROwckFN+7nyiiuJuS6GUzmnGL1iNEfOHnF0SLXanqSzfPhLPLd0voJr2zeuegXJO2HTXGNK34bVOAlbS3UODaRH914AHIzb6uBoRG2z/8x+Rq8YTXZhNnNvmEuXRl0cHVIxa5J7U+BoieeJRdvKMw74sawXlFITlFKblFKbTp48aX2U1dS5UWfm3DCH3MJcRq8Yzf4zMuKhLIVmC9O+2k6grwcv3dSh6hVoDT9OA+9AGPCc7QN0sIlDepGBL1tj/yK3QK6lEIZdqbsYu3Isbrgxf9B82jco3aHhWNYk97LOipV5vb9SahQQDfyrrNe11jFa62itdXTDhg2tj/IStG/QnvmD5uOGG2NXjmVn6s4aOa4ziVmXwI5j6bxyc0fq1fGsegW7voHDG2DgC+Bb3/YBOpivlwc0aE2jvCO8+5N0zQjYnLKZcavG4efhx/zB82kZWMUruGuANck9ESi5MkMIcNGZJaXUtcDzwDCttf3ns6yCFoEtmD94Pn4efjyw6gE2JW9ydEi1RvyJDN77aT9DIoMZElmF9VDPycuAVS9AcBR0tf9kSI7iH9Kejl4nmP1bAtuOpjk6HOFA64+tZ9LqSTTybcSCQQsI9a/CwjU1yJrk/jfQWikVrpTyBO4ClpUsoJTqAszCSOwnbB/mpQv1D2XBoAU08m3EpJ8msf7YekeH5HDnumN8PU3837CO1avk51fg7HG48R2ooTkzHCKoNQGFJwnz0zzxxVbpnrlMrT68mod/eZiwgDDm3TCPxnWqcX6qhlSa3LXWhcBDwEpgD/CF1nqXUuoVpdSwomL/AvyAL5VSW5VSy8qpzqEa12nM/EHzaRHQgod/eZjVh1c7OiSHmvnrAbYcSeP/hnWgob9X1Ss4/Af8FQM9J0JoD9sHWJsEGSeJpw/05cDJLN78Mc7BAYmatuzAMqb+OpUODTow94a5NPCpPUtGlsWqce5a6x+01hFa65Za69eKtr2otV5W9PharXVjrXXnotuwimt0nPre9Zlzwxw6NujI1F+nsjR+qaNDcojtiWm899N+hkY1YVinK6peQUEuLHsIApvBNS/YPsDapii5d/Y5yZirwpj/+yHW75drKC4Xi+MW8/z65+ke3J2Y62Ko61nF6a8dwKWuULVWXc+6zLpuFt2Du/OPDf9gUdwiR4dUo3LyzTz2+VaC/Lx47ZbI6k1otOafxjS4N70PXn62D7K2qRcOygSp+3h6UFtaNqzDU19tIz1HJqpzdXN3zOW1ja/RP7Q/MwbOwNfD19EhWeWyTO4Avh6+zBg4gwGhA3h94+vM2THH0SHVmDd/3EPCySzeubMTAb7VuFjpwC/w+3+g21hoeY3tA6yN3D2hXhic2o+Pp4l3R3TmZEYeLy2V0VeuSmvNB1s+4L0t7zE4fDDT+0/Hy1SN7ksHuWyTOxirOr3T/x2GhA/h/S3v8/6W911+Vadf951kwR+Hub93OL2rM3dMViosmQQN28INr9s+wNosKKJ46t+okEAevqY13249ztKtxxwcmLA1i7bw5l9vMnvHbG5rfRtv9HmjRlZPsqUqTvnnejzcPHi9z+v4evgyZ8ccsgqyeKbHM7gp1/veSzmbyxOfbyWisR/TBrWpegUWC3w7GXLSYNQ34OkcP09tJqg1HPgZLGZwM/HggJas23+S577ZQVRIIOFBdRwdobABs8XMS7+/xNIDSxndfjRPRj9Z43Ox24LrZbBqMLmZeLHXi4xuP5pFcYt4ccOLFFoKHR2WTRWaLTyyKJbsfDMz7u6Kt0c1hi3+9jbsXwXX/xOCqzl00pkFRYA535irHnA3ufHByC54uLsxZeEWGR7pAgrMBUz7bRpLDyxlSqcpTpvYQZJ7MaUUT0Y/yZTOU1h6YKnLrer0/s/72XjwNP+8pSOtG/tXvYI9y2HtG9Dpbugx3vYBOoOiETMlV2W6ItCH6Xd2Yk/SWV5dvttBgQlbyC3M5ZE1j7Dq8CqmRk9lcufJTpvYQZL7BZRSTO40maein2L14dU8suYRcgpzHB3WJftt30k+XBPPHd1CuK1bNeZ0O7EHlkyEK7rC0Hedep72S1LOeqrXtG3MxH4tWLjxCN9tk2mBnVFmfiaTf5rMhmMbeOnKlxjdwfmvtpbkXob7OtzHS1e+xIZjG5j802Qy8zMdHVK1JZ7J5rHPt9K6kR+v3FyNrpT0Y/Dp7eDhCyM+BQ9v2wfpLHzrg2+DMqf+nXpDG7o2C+SZr7ezPyXDAcGJ6krPS2f8qvHEnojlzb5vcnvE7Y4OySYkuZfj9ojbeavfW2w7sY3xq8aTlut884lk5xcy/uPNFBRa+Oiebvh4VrGfPfs0fHor5KbDqK8goKLJQC8TJUbMlORhcmPGPV3x8XTngY83kZad74DgRFWl5qQyZsUY9p3Zx7v932VIiyGODslmJLlXYHD4YN4d8C77zuxj7MqxTrWqk9aaqV9uIy75LB/c3YVWjap4oVFeBnw2Ak4nwMjPoEkn+wTqbIJal7toR5MAH2bd242ktFwe+iyWQrMsy1ebJWUmMWbFGI5lHmPGtTMY0GyAo0OyKUnulegf2p8Z187gWOYxRv84muOZztGn+p9f4vlhRzLPDm7LgDaNqrZzbjp8cisc2wy3zYXwfvYJ0hkFRUB2qvGrpgzdmtfjn8M7sj4+ldd+2FPDwQlrHT57mPtW3MfpnNPEXBdDrya9HB2SzV3249yt0atJL2Kui2HKT1MYvWI0s6+bTVhAmN2PW2Ap4GzeWdLz0knLSyu+P5t/tvh5el46req14u62dxPgFQDAkthEpq/ex/AuTRnft0XVDppzxkjsyTvgjvnQvtZOE+QYDYpOqp6KB9+yJ0u7MzqUuKQM/rfhIM3q+zK2d3gNBli7nM49zcI9CzmUfohAr0ACvAKKbyWfB3oFUtezLu5u9k9Je0/vZeLqiVi0hbk3zKVdg2qsF+wElKOuyIyOjtabNjnXvOpxp+OYuHoiCsWs62bRpr51FwKZLWYy8jNIz0+/IClfkLDzihJ2fnrx65kF5Z/INSkTAV4B+Hn4cSTjCHU86nB327tp6TWERxfuo3tYfebf3x0v9yr0s585BAvvhDMH4c6Poc1g6/e9XJxOgA+6wLD/QNf7yi1mtmgeXLiFlbuT+c/ILgyNqsbkbE4sNSeV+Tvn88W+L8gtzCXUP5TMgkzS89Ix6/KvB/D38KeuV90Lvwg8Awj0DiTA88IvhnNl/D39rb7ocMfJHUz6aRLe7t7Mvn42LQKq2PipBZRSm7XWla5AL8m9ihLSExi/ajy5hbk81/M5FOp8wi6VvM8l7oz8DHTZi1ehUOc/zKU+vCW3l271+Hn4FY/B3Xt6LzHbY1h9eDUWiwf++f1YNOJpwgKDrX9jR/+GRXeBpQBGLITwvrb453I9Fgu80dRYmGTwmxUWzS0wc9/cv9h6NI35Y7tzVXWme3AyKVkpzNs1j6/2fUWBpYAh4UMYHzmeFoFGErVoS3GSL93IseffUKBXIPnmfF7f+Dr1vOsx5/o5hPg7fKnnapHkbkeJGYmMXzWexMzEC7ZXpdVx7r4qrY6K7E/JYMT/voV6P2P2icXT5MntEbcztuNYGvlW0OeuNWyeDz8+DXWbwN1futQC13YRM8CYCXP0d5UWTc8u4I5Zv3M8LZdPxvWgS7N6NRBgzTueeZy5O+ayJH4JWmtuankTD0Q+QLO6zWxS/7lfv6V/3Vb112+LgBbEXBdTqxfZqIwkdzvLzM8kPi2+OJn7e/o7bGKh/SkZjJz9J25K8fnEK3HzTGX29tksT1iOSZkY3no44zqOo4lfqWX08jLgu8dg51fQYgDcNgfquH7r8pItfRD2roBpB6wqnpSew10xf3I6M58F43rQ1YUS/NGzR5mzcw7L4peBguGthjMuchxN/WrHsNmS563O5p8lol6E00zZWx5J7peJkol90YRetGx4fsjj0YyjzN0xl6UHjAVJbm55Mw9EPmD8HD2wBr57FNKPwoDnoc8T4CaDp6zyx0ew8lmYuh/8rBuJ5GoJ/mD6QebsmMP3Cd9jUiZui7iN+zveT3CdKnQFimqR5H4ZiD1yhnELNuHudnFiLykpM4m5O+fyzf5vsGgLQz2CGH8gluYBYTDsQ2h+Zc0G7uwOrIFPboF7v4WW1o+NTkrPYWTMn6Rm5hNzXzeuaul8v5Liz8QTsyOGlYdW4unmyR1t7mBsh7E09G3o6NAuG9Ymd2mqOalf4lK4e/ZG/Lzc+WLileUmdoAmfk34R7eprGh2ByMzslmRm8yw0Ct4OnIABwKqOAZeQOMOxv2Jqk0U1iTAh8UTruSKQG9G/+8vljnRPDRxp+N4Yu0TDF82nLVH1zKmwxhW3LaCad2nSWKvpWScuxNauPEwLy7dRbsm/swb06Pixa0LcmHbIlg/nUZpR3i61XWM6/cEH5/4k8V7F/Pj4VVc1/w6JkRNsHpo52XPrxH4BkFK1WeBDA7w5stJVzHh4008siiW42k5TOzXotbOPrgrdRczt89k7dG1+Hn4MSFqAve2u5dA70BHhyYqId0yTiSv0MzLy3ax6K+jXB3RkBn3dMXPq5zv56xTEPsx/PlfyEyBK7rAwJcu6EY4k3uGT3Z/wmdxn5FVkMU1odcwsdNE2jdoX0PvyIktuAnyMmHCmmrtnldo5skvtrF8exJDo5rw9u1R+HrWnrbW1hNbmbV9FuuPraeuZ11GtR/FPe3ucYqFoV2d9Lm7mGNpOTz02RZij6QxpX9Lnry+DSa3Uq09ixkS1kLsJxD3vbGwRIsB0OcxCL+63Kl60/PSWbhnIZ/u+ZSM/Az6hfRjYtREohpG2f+NOasfn4EtC+DZY9U+Ea215r+/HuDfK/fSupE//x3VlRYVdK/VhE3Jm5i1fRZ/Jv1JPa963NfhPu5qcxd+npcQl9ZgLjCuoTAXGJ9TSwFoC7i5GzeTR9FjD3AzXb7TSltBkruL0Frz7dZjvPjtLixa8687OjEkssSQxoJcOPgr7PkO9v5ozHviUw+iRkCXe6u0YlJGfgaL4xbz8e6PSctL48omVzKp0yS6Nu5qh3fm5LZ8DMsehoe3QIOWl1TV+v2pPLxoC7kFFp67sR2jejar0W4arTUbkzcya9ssNqVsooF3A8Z2HMsdEXdUPGzQYoH0I8YsmemJcPYYnD1u3HLTjDmKzt2qtLKZAk8/41oCzzrG4+LnfuDlb3zGfeoZ0zCfe1zy5u48C1lXlSR3F3AiI5f/W7ab73ckEd28HtPv7Ewzf+B4LBxaD4fWQeLfUJgLnv4QcT20uwkiBl/SvOvZBdl8vvdz5u+az+nc03QP7s6kqEl0D+5ea/uGa9yxzTD7GmOahvY3X3J1yem5PPXVNtbtT6VfREPeuDWSpoE+Ngi0fFprNhzfwMxtM9l2chuNfBpxf+T93Nb6NrzdS31+LBY4tR+O/AmJf0HKLji5DwqyzpdRbuDfxLj51gfvgPM3D98yWuduRtK3FBa16IseF+ZBQbZxHUZ+FuRnGvfnnuemG18eFX1hePhemOx9GxTd6p9/7FO/xPP6xheHE3y+Jbk7sUKzhc827OOrnzcQbE5iTEQ+vXyP4Za8w/gD0xZAQXAkhPWFltcY0wXYuLWSU5jDV/u+Yt7OeZzMOUmXRl2YGDWRq664SpJ8Qa4xDcFVj8C1L9mkSq01n/55mNd+2INC8dA1rXigb3jV5gey8jhrj65l1vZZ7Dq1iyZ1mjCu4zhuaX0LXqYSn6GsVNi/GvatMH4d5pwxtvvUhyZR0LDt+VtgKPgFg6mGzhtobST9nDPGLfv0+cfFtzTIOW28ln3q/ONypjHA5FmU8M99CZT+Ijj3BVHiy8IBXwium9w3zoK1b4KHD7h7l3HvDe4+xr2HbzllSt/7Fu1XYpvJs6iVYTLuL+U/0GI+3/oo2QLJOwtZJyHzJGSdQGee4MyJRMynDtHAcgo3VeL/pm6IkcybREGTzsbYdJ+auRAmz5zHN/u/4X87/0dyVjKRQZFMjJpIv5B+l3eSn9nHGDVz37c2rTbxTDavLt/Nyl0phAfV4fHrIhga2QS30udYqsiiLfx85GditscQdzqOEL8QHoh8gGEth+FhKrq6Ojcddn0L2xbDkT8AbSTtVtcan7nQntCglVO0cMtksRit/gsS/qmi2+nz9yW355wpalCVwc2j1C+C+qW+CMrY7uV/Sf9+rpvcD6yBuOVGy6kwp4z7olth7vltZhusiqPczp/8KZn03dwBZfxE1Oaik0XmEo+LtleiwMOfFEtdjhX4k+4ZTHjrjrRqG4mq38L4Y/Ktf+nv4RIVmAtYemApc3bM4VjmMdrVb8fEqIkMaDbAJvPjOJ1lDxvnOqYdtEuyW7v3BG/8EMfelAwiGvvx2LUR3NAh+OIT6ZUwW8ysOryKmO0xxKfFE1Y3jPFR4xkSPuT8FLvHthgjq/YsM/52giKg420QMQiCoy7vq5dLfiHknC71RXDqwl8HJb80KvpCuPHf0G1MtcJx3eReHRbzhcm+INfo0yvMLfFFUOq+uA/QfL4vUJsv3mYpPH/WX5nOJ/7iLwOT8SvAs+TJoTrg5UeO8uGHhEJitmSwN7WApoE+PHpta27t0hR3U+39YyqwFPB9wvfM2TGHw2cP0yqwFROjJtI3pC91POo4Oryas+l/sPxxeHQb1AuzyyEsFs33O5J476d9HDiZRWh9H+7t1ZwR0c0I8K14LqPM/EzWHF1DzPYYDp09RMuAlkyImsANYTdgcjMZXRv7VsCGD+DI78Z5m04joPPdxmLozto6rw0sFshLL0r2pb8ITkHboRBSaX4ukyT3WqrQbGFdfCpLY4+xancK2flmOoUEMLZ3OEMim+DpXnuTemmFlkJWHlpJzPYYEtITAGNmzGC/YIJ9gwmuE0yTOk0IrhNcfGvs2xhPk6eDI7eR47EQ099Y1KTDcLseymzRrNyVzPzfD/HXwdN4eZjp3daDbi0UwQ1ySM05QXJ2MklZSaRkpZCclVw8I2JEvQgmRk3k2ubXnv+FlfAr/PwKHNsEgc2g52ToMgq8ZRx7bWdtcq89V024sDNZ+fy2/yS/7j3J2n0nOZ2VT11vd27ufAV3RIc67SRS7m7u3NjiRgaFDWLD8Q3Ep8WTlJlEcnYyKVkp7EjdQVrexQuLN/BucFHSP3drUqcJDbwbGC3L2q5Re+NX2fFYmyZ3s8XMyZyTJGcln79lJ5OUmYRqmkxI3STS88/wdyH8XWI5V19TAE39mxDqH0qP4B4E1wkmol4EV15x5fmknhoPP06DAz9D3abGoiOd7q65E6Gixsj/qI2ZLZqDqZnEHkkj9mgasUfSiEs+i9ZQz9eDvq0bMjSqCVe3aWjzURCOYnIz0S+kH/1CLl5rNacwh5SsFJKykoqT1LmElZCewIbjG8gpzLlgH3flTiPfRhcl/ZKP63rWdfzJXHcvY56Z41ut3kVrTVpeGslZyRf+m2Se/7c5kX3iotWKfN19i/8N2tVvR3CdYBr6NCY1zZfYBPj7QCEpGZACZDb2w7dZPcJDA2kQFIjFAm6WXFg/Hda/awwYuP416P7AJQ2ZFbWbVd0ySqlBwPuACZijtX6z1OtewMdAN+AUMEJrfaiiOp25Wyav0ExKeh5J6Tkkn83lYGoW8ScyiT+RycHULPIKjRMp/t7udA4NJLp5fa5u05DIpgFVPhnm6rTWnM0/e3Er9Vziy0omJTuFwlJjmn3cfWjs2/iipN/It1HNzqv/50ew9wfjXMq5IXYa8pUixaRINrmRZFIku7mRYnIj2eRGbqnPgIfWNDZbCC4008RsJrjocXDhuceF+GuNKv5bPXec88+L/4q18ditnOF+e4JuIKHrs9Rv3IwmAd4EB3jj7eEajYzLhc363JVSJmAfcB2QCPwNjNRa7y5RZgoQpbWepJS6CxiutR5RUb22SO5aayzaaC1btHEzHhsnoixaY9Yai4Xi17SGQouF3AILuYVm8orvzca2AjO5BWYy8wpJzykgLbvAuM8p4GxOAamZeaRmXjj6RikIqedDq4Z+tGrkR+vG/nRtFkiLIL9LHr4mjCF8p3JOXdDaTcpKIiU7pfhxak6qo8MskwIaunkRbPIh2M3HuDf50KToPtjNh/omb9yUKip9bkdVogYrnpfaptGkZReSfDaX1Kx8zmTl80tuBMvSW2Ip9Sdfv44nQX6eBPp4EuDrQaCPB4G+HgT4eODr6Y6PpwkfDxPeHia8Pdzw8TDh42nC090NdzeFye3cvSpx74bJZDx3Uwq34rAVqkSoDv/15YRsmdyvBF7WWt9Q9PxZAK31GyXKrCwq84dSyh1IBhrqCiqvbnKf/VsCb62Iw6w19j4X7ONhIqDEBz3Ax4MGfp40CfAhOMCbJkW3poG++HhK68eRCswFpGSncDLn5EWtfEdwdzO6lmr8l0QlcgvMHE/LISk9l6T0XJLTcziensupzLwLGzPZBeQUVD6E11aKkz3nvwDObVeo899pJbeV2M/ZvHRTe0Z0r94ShLY8odoUOFrieSLQs7wyWutCpVQ60AC4oDmllJoATCh6mqmU2mvF8WubIEq9r8vA5faeL7f3C/Kea9Rdr8Jd1d+9uTWFrEnuZX03lm4zW1MGrXUMEGPFMWstpdQma741Xcnl9p4vt/cL8p5dkTWDqhOB0BLPQ4DSS8gUlynqlgkATtsiQCGEEFVnTXL/G2itlApXSnli/JpYVqrMMmB00ePbgV8q6m8XQghhX5V2yxT1oT8ErMQYCvk/rfUupdQrwCat9TJgLvCJUioeo8V+Cd1JtZ5TdytV0+X2ni+39wvynl2Ow6YfEEIIYT/OM5GJEEIIq0lyF0IIFyTJ/RIopaYqpbRSKsjRsdiTUupfSqk4pdR2pdQSpVSgo2OyF6XUIKXUXqVUvFLqGUfHY29KqVCl1Bql1B6l1C6l1KOOjqmmKKVMSqlYpdRyR8diD5Lcq0kpFYoxJcMRR8dSA1YDHbXWURhTUTzr4HjsomiqjRnAYKA9MFIp1d6xUdldIfCk1rod0At48DJ4z+c8CuxxdBD2Ism9+t4FplHugoyuQ2u9Smt97pr+PzGudXBFPYB4rXWC1jofWAxc+urXtZjWOklrvaXocQZGsmvq2KjsTykVAtwIzHF0LPYiyb0alFLDgGNa622OjsUB7gd+dHQQdlLWVBsun+jOUUqFAV2AjY6NpEa8h9E4K2ctPOcn87mXQyn1ExBcxkvPA88B19dsRPZV0fvVWi8tKvM8xs/4hTUZWw2yahoNV6SU8gO+Bh7TWp91dDz2pJQaCpzQWm9WSvV3dDz2Ism9HFrra8varpSKBMKBbUXTlYYAW5RSPbTWyTUYok2V937PUUqNBoYCA1346mNrptpwOUopD4zEvlBr/Y2j46kBvYFhSqkhgDdQVyn1qdZ6lIPjsim5iOkSKaUOAdFaa5edUa9osZbpwNVa65OOjsdeiuZF2gcMBI5hTL1xt9Z6l0MDsyNltFAWAKe11o85Op6aVtRyn6q1HuroWGxN+tyFNT4E/IHVSqmtSqmZjg7IHopOGp+bamMP8IUrJ/YivYF7gWuK/m+3FrVohZOTlrsQQrggabkLIYQLkuQuhBAuSJK7EEK4IEnuQtH8RJEAAAAcSURBVAjhgiS5CyGEC5LkLoQQLkiSuxBCuKD/BzSql2khiymTAAAAAElFTkSuQmCC\n", 428 | "text/plain": [ 429 | "
" 430 | ] 431 | }, 432 | "metadata": {}, 433 | "output_type": "display_data" 434 | } 435 | ], 436 | "source": [ 437 | "# initial conditions\n", 438 | "plot_fig()\n", 439 | "plt.title('Before Training')\n", 440 | "#plt.savefig('fig3.png')" 441 | ] 442 | }, 443 | { 444 | "cell_type": "code", 445 | "execution_count": 21, 446 | "metadata": {}, 447 | "outputs": [ 448 | { 449 | "name": "stdout", 450 | "output_type": "stream", 451 | "text": [ 452 | "0.0\n", 453 | "0.1\n", 454 | "0.2\n", 455 | "0.3\n", 456 | "0.4\n", 457 | "0.5\n", 458 | "0.6\n", 459 | "0.7\n", 460 | "0.8\n", 461 | "0.9\n" 462 | ] 463 | } 464 | ], 465 | "source": [ 466 | "# Algorithm 1 of Goodfellow et al 2014\n", 467 | "k=1\n", 468 | "histd, histg= np.zeros(TRAIN_ITERS), np.zeros(TRAIN_ITERS)\n", 469 | "for i in range(TRAIN_ITERS):\n", 470 | " for j in range(k):\n", 471 | " x= np.random.normal(mu,sigma,M) # sampled m-batch from p_data\n", 472 | " x.sort()\n", 473 | " z= np.linspace(-5.0,5.0,M)+np.random.random(M)*0.01 # sample m-batch from noise prior\n", 474 | " histd[i],_=sess.run([obj_d,opt_d], {x_node: np.reshape(x,(M,1)), z_node: np.reshape(z,(M,1))})\n", 475 | " z= np.linspace(-5.0,5.0,M)+np.random.random(M)*0.01 # sample noise prior\n", 476 | " histg[i],_=sess.run([obj_g,opt_g], {z_node: np.reshape(z,(M,1))}) # update generator\n", 477 | " if i % (TRAIN_ITERS//10) == 0:\n", 478 | " print(float(i)/float(TRAIN_ITERS))" 479 | ] 480 | }, 481 | { 482 | "cell_type": "code", 483 | "execution_count": 22, 484 | "metadata": {}, 485 | "outputs": [ 486 | { 487 | "data": { 488 | "text/plain": [ 489 | "" 490 | ] 491 | }, 492 | "execution_count": 22, 493 | "metadata": {}, 494 | "output_type": "execute_result" 495 | }, 496 | { 497 | "data": { 498 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8VFX+//HXoYTQa5ASMICgVAEjiAgiRRFZUdQVxa6LZfVndym6oq5lla/r2hXLuvZeEREEFFgBgyK910hJCCVACKSc3x/nTiZlwqRMygzv5+ORx8xt5547987nnvs5d26MtRYREYkcVSq6AiIiEloK7CIiEUaBXUQkwiiwi4hEGAV2EZEIo8AuIhJhFNhFRCKMAruISIRRYBcRiTDVKmKlTZo0sXFxcRWxahGRsLVo0aJd1tqYYPNVSGCPi4sjISGhIlYtIhK2jDGbizKfUjEiIhFGgV1EJMIosIuIRJgKybGLiBRFRkYGiYmJpKenV3RVylV0dDSxsbFUr169RMsrsItIpZWYmEjdunWJi4vDGFPR1SkX1lpSUlJITEykTZs2JSpDqRgRqbTS09Np3LjxMRPUAYwxNG7cuFRXKQrsIlKpHUtB3ae02xxegX3fH7D6u4quhYhIpRZegf21QfD+pRVdCxGRSi28Avv+7e5V/4BbRCrQpk2b6NKlS8BpN9xwAytWrChSOf/5z3+49dZbQ1k1IIR3xRhjqgIJwB/W2uGhKjcgmw2mapmuQkSkJF577bWKrkJIb3e8HVgJ1AthmYFlZ0IVBXaRY8lDXy9nxbbUkJbZqUU9HvxT56DzPf3007zxxhuAa5FfcMEFZGZmcvXVV/Pbb7/RoUMH/vvf/1KrVi0GDBjApEmTiI+PD1jWm2++yeOPP07z5s3p0KEDNWrUCOk2QYhSMcaYWOA8oHxOVdmZ5bIaEZFFixbx5ptvsmDBAubPn8/kyZPZs2cPq1evZsyYMSxZsoR69erx4osvBi1r+/btPPjgg8ybN4/p06cXOWVTXKFqsT8D3AfULWwGY8wYYAxA69atS7e27KzSLS8iYacoLeuyMHfuXC688EJq164NwMiRI5kzZw6tWrWib9++AFxxxRU8++yz3HPPPUcta8GCBQwYMICYGPfk3UsvvZQ1a9aEvM6lbrEbY4YDSdbaRUebz1r7qrU23lob79uoYqvbwitMgV1Eyoct5GaN/PeaF/Xe8/K4Lz8UqZi+wPnGmE3AB8BAY8w7ISg3wJpud6+6K0ZEykn//v354osvSEtL4+DBg3z++ef069ePLVu28PPPPwPw/vvvc8YZZwQtq3fv3syePZuUlBQyMjL4+OOPy6TOpQ7s1tpx1tpYa20cMAqYaa29otQ1C8TXYapUjIiUk549e3LNNdfQq1cvevfuzQ033EDDhg3p2LEjb731Ft26dWP37t3cfPPNOcsU1ipv3rw5EydOpE+fPgwePJiePXuWSZ3D6yFgxjsPKRUjIuXorrvu4q677sozrrCOz5SUFBo1alRoWddeey3XXnttSOuXX0gDu7V2NjA7lGXmoRa7iFRiQ4YMoWvXriV+KmOohFmL3QvsNrti6yEiEsD06dNz3vfu3ZvDhw/nmf7222/TtWvXMq9HmAV2pWJEJDwsWLCgwtYdXs+KUSpGRCSo8ArsSsWIiAQVXoG9ii8Vo8AuIlKY8Arsvhy7UjEiIoUKs8DuS8UosItIxQnV89jLSnjdFaPOUxGp5CLteexlT52nIseuqWNhx9LQltmsK5z7RNDZQvk89tdff51//vOftGjRgvbt21OjRg2ef/75kG5WmKVi1HkqIuUrlM9j37ZtG4888gjz589n+vTprFq1qkzqHF4t9irqPBU5ZhWhZV0WQvk89oULF3LmmWfmPEvmkksuqZzPYy9X6jwVkXIWyuexF1ZWqIVXYK+iHLuIlK9QPo+9V69e/Pjjj+zZs4fMzEw+/fTTMqlzeKVidB+7iJSz3M9jBwo8j/3GG2+kffv2RXoee8uWLRk/fjy9e/emRYsWdOrUifr164e8zmEW2JWKEZHyF8rnsV9++eWMGTOGzMxMLrzwQs4+++yQ1hXCNRWTrVSMiFQ+RXke+8SJE+nevTtdunShTZs2XHDBBSGvR5i12HW7o4hUXkV5HvukSZPKvB5hFth9eSv9M2uRY4W1tkh3nFQ2pXkee2nvngmvVAzezlWLXeSYEB0dTUpKSrndJlgZWGtJSUkhOjq6xGWEWYvdl4o5dnayyLEsNjaWxMREkpOTK7oq5So6OprY2NgSLx9mgV0tdpFjSfXq1Sv8H0OHo/BKxfha7Mqxi4gUKjwDu1rsIiKFCq/Ars5TEZGgwiuwq/NURCSoUgd2Y0y0MWahMeZ3Y8xyY8xDoahYIStzr2qxi4gUKhR3xRwGBlprDxhjqgNzjTFTrbXzQ1B2Xia8LjBERCpCqQO7db8cOOANVvf+yihXoha7iEgwIWkCG2OqGmMWA0nAdGttgd/SGmPGGGMSjDEJJf6xQU4qRjl2EZHChCSwW2uzrLXdgViglzGmS4B5XrXWxltr42NiYkq2IuXYRUSCCmnS2lq7F5gNDA1luTl0H7uISFChuCsmxhjTwHtfExgMlM2/3tYvT0VEggrFXTHNgbeMMVVxJ4qPrLXfhKDcAJSKEREJJhR3xSwBeoSgLsHpB0oiIkGF143h6jwVEQkqzAK7cuwiIsGEV2BXjl1EJKjwCuzKsYuIBKXALiISYcIssCsVIyISTHgGdnWeiogUKrwCuzpPRUSCCq/Arhy7iEhQYRbY1WIXEQkmzAK7fqAkIhJMeAV25dhFRIIKr8Cu57GLiAQVpoFdqRgRkcKEWWDX/zwVEQkmzAK7Ok9FRIIJr8CuzlMRkaDCK7ArFSMiElSYBna12EVEChNegR28PLta7CIihQnPwK4Wu4hIocIzsGdnVXQtREQqrfAL7NVrQsahiq6FiEilFX6BPaoOHDlY0bUQEam0Sh3YjTGtjDGzjDErjTHLjTG3h6JihYqqA0f2l+kqRETCWbUQlJEJ3G2t/dUYUxdYZIyZbq1dEYKyC4qqDYcPlEnRIiKRoNQtdmvtdmvtr977/cBKoGVpyy1UrUaQllJmxYuIhLuQ5tiNMXFAD2BBKMvNo14LSN1WZsWLiIS7kAV2Y0wd4FPgDmttaoDpY4wxCcaYhOTk5JKvqG4LOJgMWRklL0NEJIKFJLAbY6rjgvq71trPAs1jrX3VWhtvrY2PiYkp+crqtQAs7N9R8jJERCJYKO6KMcDrwEpr7dOlr1IQ9Vq4VwV2EZGAQtFi7wtcCQw0xiz2/oaFoNzAajVyr8XtQLUWsvUoAhGJfKG4K2autdZYa7tZa7t7f9+GonIB1WzoXrf95l7TU+Hzm2HfHwXn3TQXpv/d3R654GV4uCGkrIfkNWVWPRGRihaK+9jLV71Y95qd6V5/fh5+fw8aHg8DxkLSKkjbBTuXw9T73DzzniXniZDPx7uHiN2fDNWiyr36IZGVCTMfgV5/gfqxFV2bo9uX6E6wJ4+q6JqEnrX+R0lHkox02L4YWp9W0TUpvt0boWFcZO6XYgi/RwpUi3Kt9vR97ou16C03vnYT9/rf8+E/5/mDOpDnMb++J0OunwlLPs77TzvSdvuvBMrahtmQebjg+NTt/m0qzB+LYN4z8K/OsGtd8dcdKCV1eD9M/Zv7YhwIctfS4QMweRBsW+yG03YX/mC2GRPh8xsL9ons3Qqb/1dw/mWfwkdXB92EIknfV/R5szLh7ZGw4Ud3FXg0SSvhma7wcj/YtdY/PvMIHNrrPpdA+xYKfk5JK+E/w91neGiv+0vbXfR6F1VGOhzclXfctAmwcLI75v741X0Xpk2Ady+GN86B5NVFLz8r4+jfnYxDof1hYVZmwX+4k7QSnu0OPzwc+NgqK9bCqinBj5tyFH4tdoBajV2OPfEXOOAFjF3rYFIHOLCzaGV8NgYO74PF78Kf34Lo+jDlLlj+uZt+3tNw6vVlU//dG+G/I9z7dgPhsg/9Vw+fXg+b58HX/88N3/YrNG7nX/ZgCsx/wT/8+hDodL4LJGeNhwatvXVscAdc7mV3LocVX8GPT0CdZnDdVGjU1k1b+KpLVy142Q1f9SW0HeBfNmmVawkdOeDK+SMBvhsHQx+D14ZAl4tg5CvuOT6f3wQrv4KOf4KVX7vl18+Eky9zLanERfDaQDf+3g1Qu7F/PZ9c517nvwQnDYcGrfzTFr8P0fXgxGHBW2QzHoK5T8PFb0KXkf7xCyfDt/e494MfgjPucO9TE2H9D+4P4NQb4Lz/y1umtW76Oxf5xz0fDyMnw6xHYc8m//gO58IZd0Lr3v5l50yCmf+AAeNhwN/c+J+fh01z4Mk2edc1sRgnpX2J8OGV7jbgP/0bThiUd3ryGnjhVPf+gV1Qtbp/3eD2Y3YGNGoHu9f7l9uxFGo1gUO74bd3oEo16PNXfz+XT8YheLSZe9/5QveZ+/bPzhXwUh//vJd9CCcOzbt8ViakrINfXoMhD0NUrcK3NfOw+/w3zYE2/eHqr/3Tkla617lPu7/462F4Ke/n+N/z8P0EF3Oun573+5TwplvP3i1uuPtouODFvMv7gv5xnaFRvn1choytgH8zFx8fbxMSEkpewOtnQ7UacFzXvEGupGJPhRtmwMT6ecfn/nJlHIIt8yF5FXS9xH+F4NtxSSsgMx363uGCj8/hA7DkQ7eO5t3cuC3zXYvIJ+Yk+Kv3m66nO7sg49P+HBj9kWsN/vCQ/8tYmAf3upPDsk/d8Pjt7ouy4Ud3NZNb9VowYbt7P/NR+OnJwNu/9Rd4fbB//EnDYdU3Bdddr6V7rPK+rYHrdu5TULMBfPYX/7huo9wJIWedufZB3RZwt/dlnTzInUwA/vy2O5kVZsErea/Yxm5xJ+6sTHikcd55/74bqlSFxAR4LV9APO0WGPq4e5+RDh9c5k5QxTH6E1jznTtOEl73j/edsD8YHfizfCDFpUOWfeYCcZeRLqh9fqPrT+o0Ai58xR2PuQMn5Dtu0+HR4/zD13wLcX3zBuPiGDDOpTx9tv0Grw7IO8/N/3OBDGDKPfDL5MLrt3M5vHS6f/iUa12atceVsH8bxHSEpif5p791Pmz80T/89z1QxUs8/Po2fHVr3nXdvRrqetuZeRiei4d9W+D856HnlW58xiFY8pE7Rtqe6e/Hm3K3O9n4NIyD23/PtR354kX++gD8+BTM+gfUbgq3Jbh1lIIxZpG1Nj7YfOHZYq/XArb/Drs3Qes+sOXn0pWX+AscSSs4Pm031Kjnvngf50oPLPsMzn/WXTXs3pj3YDq4C065xp2dp01wVwQ+133vWry+loVP8ip/vjZ/S3TtNHdALv8ieFAHN48vqINL29SOcQdufhneNlsLezYWnH4wxZ1Acwd1CByIAFIDdGDnNvXevMOtTnP5d3CX8rkDPrgv9o9Pur6EP3I1BH560p0kk9fA9Afc59fkROh+mTtJ5knDAWunQ9eLYe/mgnXatQaadgx8++z8F10Lcs9meP6Uo29bYd69OPD4DbOhYZvCP8tHGrvWcpqXPpn3TN7pSz+C+OvyBh4f37G0aa5LS+a2broL7POeLdZm5PDt4wNJ7iQT6ES37TcX2A8kw68B0opbF0KrXgUbDACL3nSvv72dd/ywSa7c3EEdIGk5NOvqUlz7Eing+wfgosnu+Hq8FWR5KbKvbnVXIDUbwvuX+udv099draesL/jZ+q7IsrP8V7b5/f4e9LjCvT98AOb+y63nYBI80Rr+Mgta9gy8bAiFZ4t92gR/kBv6BHw39ujzF0Wzru7S82jqt3JPl0xeWXBa7aYQc6K7RMyv6yWw9OOjl10vFuq3hK0hehpD7Rh3ad60k8uV2kJy4Oc85lq4gYJeINENIH1v6et31v2uJTrjQbhlvktRFBbkcmt/Nqz93j8cVQfqNneX8kf7l4ntBrn9ezDJDbc9CzbMclcMabtg3YzAyzVq63KnabvcevZvL/ImBjRysjt+253lgtuUu4Mvc85jMG18wfF1W7iTH0CD4/37cNgk1yiZ/Zh/3kvecum2w6kw6MHCTziFOb6v+18ISSvdiXbh5IIn8pt/dlcPHc93KcYpdwc+7gJ9j7qPztsI8mlyIuwKkOsfOdml/BqfAHWaBv7e+bTo4RqCvv61WxbAe3/2Pi8DWNcaz51KK0zz7q5xlpKvb6tOM5cWjqoLHc5xx9nGn9y0G35wV1++fX3zz3Bcp+DrCqCoLfbw6zwF94XwaX92ycu59F2X44WCQb3/ve7MW70WtDzFXdLduQyu+84/z6l/gb63w2UfwL1rXc4W3EHb7VIY8aJLjVz0Gpz2V5en6zYKTsjVSpmwE3rd6FIUgYJ6+3OgWk3/cGyvgvMAjJntLoF9bk1wX+DdG1xAOmGIf9qIF2Hg/e79tPGBWzq+y1GfNv3h9iVw3bS84wc9GLg+gQzIFZxOucafmnrxNNeq7pPryqfdQOh3N3TIlY+tGgWj3ndXPicNhzZnwpVfuEvc+3e69EaPK11+/P4kl84Y8YILgOt/8Ad1gOH/cjnlJR+4jsO2A/LWtfnJbh27N7igHtMRbinlleHJl0O3P7sW85IPXZqiUbujLzPiRZfXbu+l7vre7u7ouvRdf4rhrPvhjiXwt83uWP32HhfUm3mf7/FnuNRN+yHuOH/3Yn9fTFFd8alLTR3c5TrEq0a5tInPuD9csGo30PWvfHNHwaB+9xq3/w7u8gf12jFwxzKXm75zed7j7sY5cOtCFxhb9faP73e3+xwve99dNaesg84jKaBmI/e5paf6g/rfNrnUznXT3PewWRf3/vbf4b6N7ngZ8jDE9XOpxZy6/OSuBrcvdus7/f/5p8V0hHtWQ++b3SPFl33irkYwrp8pNt4dkzfNhU4XlMs/CgrPFjvAL6+7y8EBY+GhBiUr49ZF0OQE14ueug2GP+MuvzuN8HeSZGX4O5t8dix1B2TdADnKlPWu9VQ1X5YrO9sdXL7xvvxc/k6yuc+4Vmz/e93B1fZMN37NNKgW7Q6S7++HhDf8ywyb5FpRAF/f4ep++m1uOPcted/f71IAN8114zfPc2metgPc3RhPtXUt2QFj/be6rfvBfdlyXz766t4wDm77zbXgazWCz250gfK6aS5dNXCCu/z0Gf4v+OZO1+q58UeX6nnK67z1XaKu+BIWv+c6Ln23cq74Ej66yp28WvQo+JkXxeafXb/I4vdch9fYLe6SescS1/8RVdu/XTf/7NJ9NRu4k97cf7lgUq+FayX+/n7esqtFw4Qd7sqoURuXN37zXLj8Q3csPNvdzfdAitv/K76Cj650qZgxs1z/R2a6a8luXww9r3Lpwd8/cLngalGuBZ66zQUiH2th5zJo2tmf1808AqunuJbjCYNcEGoY547h/Ttc53SrXq4vaMNsl2Ic/JA75gaMcyfwec/CmqmuvI7nu6DkOw7TUyHriPssDx+A2Y+7+sac6KanbnP7q2Gc23/ZWS4d1HYAjPJa5NnZbnsXv+s6wuvnCqDZ2e73JlAwX734fZeiufKLwLcqW+uPBbcscPvL19+1aa6ri287imrvFrdfjHHlZ2W4FnutRrB6Krw/yqXEhv/LlZ/6h7t6rxrl0j5Vo1wfTogUtcUevoE9N98Xsv99BTsAj2b8NveFrgg7l7svXacRJVt+5qMu4NgsGLsVatQpfZ0yj7jOz/wnpfw2zXO3k512E9So6x9/JM21oHLfybJjqbsDqcNQd5fIOxfBJW/6g/ayz9wJ8vjTOapQ3TOenQWH9vg7v3P79b+w6lu4/IPCl8/KdF/YKtVcnjW2lwtMR2sBr57qgmq818K11qV+jusC9ZqXbntKK2V93js9fLYudEGpRffSr8MXY4q6/xa84tI+Pa8q/rrWz3Qnn84XFH/Zkkha5U7m1WqUy+qOzcA+frtrcc98xJ01c196g8vjdRrh8mtQvFvKpOQi9Yc8IuUssu+KKUxULZenrRoFp93sLqOe6+kuNTtf4C4PpfwpqIuUq8gK7ADVo6Gv17HRuJ3rsKnTNG9wGfmavzNFRCTCRF5gz6/ucQXHdbuk/OshIlJOIiOw37dRl/siIp7ICOz5n10hInIMC88fKImISKEU2EVEIowCu4hIhFFgFxGJMArsIiIRRoFdRCTCKLCLiEQYBXYRkQijwC4iEmEU2EVEIkxIArsx5g1jTJIxZlkoyhMRkZILVYv9P8DQYDOJiEjZC0lgt9b+BOwORVkiIlI6yrGLiESYcgvsxpgxxpgEY0xCcnJyea1WROSYU26B3Vr7qrU23lobHxMTU16rFRE55igVIyISYUJ1u+P7wM/AicaYRGPM9aEoV0REii8k/xrPWntZKMoREZHSUypGRCTCKLCLiEQYBXYRkQijwC4iEmEU2EVEIowCu4hIhFFgFxGJMArsIiIRRoFdRCTCKLCLiEQYBXYRkQijwC4iEmEU2EVEIowCu4hIhFFgFxGJMArsIiIRRoFdRCTCKLCLiEQYBXYRkQijwC4iEmEU2EVEIowCu4hIhFFgFxGJMArsIiIRRoFdRCTChCSwG2OGGmNWG2PWGWPGhqJMEREpmVIHdmNMVeAF4FygE3CZMaZTacsVEZGSCUWLvRewzlq7wVp7BPgAGBGCckVEpARCEdhbAltzDSd640REpAKEIrCbAONsgZmMGWOMSTDGJCQnJ4dgtSIiEkgoAnsi0CrXcCywLf9M1tpXrbXx1tr4mJiYEKxWREQCCUVg/wVob4xpY4yJAkYBX4WgXBERKYFqpS3AWptpjLkVmAZUBd6w1i4vdc1ERKRESh3YAay13wLfhqIsEREpHf3yVEQkwiiwi4hEGAV2EZEIo8AuIhJhFNhFRCKMAruISIRRYBcRiTAK7CIiEUaBXUQkwiiwi4hEGAV2EZEIo8AuIhJhFNhFRCKMAruISIRRYBcRiTAK7CIiEUaBXUQkwiiwi4hEGAV2EZEIo8AuIhJhFNhFRCKMAruISIRRYBcRiTAK7CIiEUaBXUQkwpQqsBtjLjHGLDfGZBtj4kNVKRERKbnSttiXASOBn0JQFxERCYFqpVnYWrsSwBgTmtqIiEiplVuO3RgzxhiTYIxJSE5OLlEZWdmWeet2hbhmIiKRJWhgN8bMMMYsC/A3ojgrsta+aq2Nt9bGx8TElKiyk+dsYPRrC5i1KqlEy4uIHAuCpmKstYPLoyJFkbgnDYCt3quIiBQUVrc71qhWFYDDGdkVXBMRkcqrtLc7XmiMSQT6AFOMMdNCU63AqlVxnbRZ1pblakREwlpp74r5HPg8RHUpxnrLe40iIuEjrFIx7y/cAsCXi/+o4JqIiFReYRXYU9MzAVi1Y38F10REpPIKq8A+8KSmBcZt3Z1G3Ngp7ExNLzDtpzXJPDplBQBpRzL5feveMq+jiEhFK1WOvbwF+n3rqFfnAzBw0myWPzyU9ckHSD2UQecW9bnqjYUApB7KZMX2VJb+sY85951F4zpR1IoKq00PW0cys9myO40Tmtap6KqIHDPCKrpl5+o13XXgME3q1OCPvYcAuPTU1lhrGfR/PxZY7sOErTnv+z05C4BvbjuDLi3rl3GNy8bqHfs555mf+Peo7ozo3jJk5R7JzCaqWtEu4uasTaZP28ZUq1r4/Jt2HWTApNkALJwwiKZ1o0NRzTKzPz2D2lHVqFKl5I/I2LEvndo1qlKnRrVK/6gNa22eOqamZ3DoSBZ3friY+jWr89xlPY66fyuj5P2HaVInqtJ/9mUtrPZaeq771296e1GeaQ1rVafbxO+LXNak71fT+7EZ7DpwOGfc1t1pvDBrHeM+W8qRzLK9V/6yV+fz9vzNvD1/M1nZ/hPWO/M3Ezd2Cg9/vYLhz80hI6tgPc55xj1z7fYPFpOwaXfO+M0pB7FFuGVo4cbdXPbqfNIzsnLGrdyeSof7p/LMjDWc9tgPpKZnBFzWWsuPa5K58vWFPDdzHR8lbCVu7BTixk7h2R/WkpVt2Zt2hG+XbufJaatyltublsG+NH+Zb8zdyAnjvyUpXwrt7Z83cd1/fgm6DcEcOJzJvR//TvL+w3nGH8nMJjU9b10A9qVl0HXi95z33Fze/nlTToMhP1/qL27sFHo+Mp1/z1iLtZaMrGwmfrWc0x7/ga4Tv6fNuG+56KX/sX3fIbbuTsvZx6t37GftTn8f0YHDmfzf96tZtHkPJ94/lbixU3hh1jq27i7ej/COZGbzv3W7Ct3/q3ak8u6CzXnGDZg0m7++9ytPfreKbhOnMeCp2fR+7Af+tz6Fqct28MpPG/LMn79say3Z3nZZa/nitz/Ysa9gSnRzykEysrKZtnzHUY/P1PQMpizZzqodqQXWk3+5pNR0Ug7k3bdrd+7n1Edn8MpPG7jl3UUB61JcWdmWbXsPkZGVnacOud8fzszi5/UpbN93KM/03J9NeTMVsdL4+HibkJBQ7OX+8t8Epq/YmTM8f9wgTnv8h1BWrchaNqhZ6Je/MmnVqCZbd1fuejasVZ09aYFPJIG0bFCTg0cy2ZuWwfGNa9G+aV1mrNwZfMEALjkllo8XJZZo2eJo37QOa5MO5AyfGteQXzbtKZN19WrTiIUb/Sf8v/Rrw+Q5G8tkXQB/H96Jh79ZkTN8bpdmTF22o9D5T46tz++J+0q0rtG9W/Pugi05w+POPYnHp64qdP6HR3Tm718uP2qZQzs347vlhdc3t2B1796qAYuD9OV9d0c/TmpWr0jry88Ys8haG/QR6WEV2F+YtY6npq3OGR4/7CQe+7bwnSoiUtl8fsvp9GjdsETLFjWwh1UqJv9JaNrykrXSAqkbHbi74cmLu9ErrhHN60fz071nMeueATnTfpkwmBb1Xd546cSz+fvwTnmmATx5UTc2Pj4sZ/w3t51Bh+MCdyQue+icQuv32IVdi7wtANPu6M83t51B3ehqPHNpd8YPOylnWlS+vOnyo6x37t/Oom2T2lSv6nKWk68q+f9Tueb0uJz3/29Qe767o1/O8D8v6so71/cusMxb1/XiqYu75RnXvVUD3r6+Fx/d2Ievbu3LhseG8frV8fxv7EB+fWAIUdWqMLJHS2pHVc2SoIEmAAALZklEQVRZ5oXLe/LS6J45wx2b+1tMCycM4uTYwP0tr10Vz5d/7cujF3bhxjPbFnub548blGf4zWtPZUx/V8553ZofddnbB7XnwT914s7BHZh595mM6d+WiX/qxILxg/jir335/s7+vHzFKXRuUY8nL+7GeV395f36wBBm5zpWHx+Z9/h5Mt9nGszHN/XhkQu6ANCvfRM2PXEec+47K2f6cfVqsO7Rc/Mss/ofQwuUU8VAkzo1WPnwUBZOGMR1fdswonsLHhnRuUAH+73nnMjU2/uxZOLZeb5DAPPGDqROjWo0rVuDf4/qztp86/a5c3AHlj90Dt/cdgbXn9GG0b1b89Lonjwxsisvje7J17eewS8TBrPqkaE8dH5nbjqzHQvGDyqwvmFdmwFwcqsGrH303DyfLcB7N/iP3fuGnsimJ86jV1wjAF70jrsJwzrywZjTShzUiyOsWuz/nrGWf81YA7jLTZe/LHma4a3rejF7dRIP/qkzACu2pTLu86W8c30v6kZXL3S5fWkZ1ImuRtUAnWzZ2TZo51t2tqXt+G8B2PTEeXmmxY2dAsAdg9sz6tTWxNStUWA9r83ZwD+mrMwZXjB+EMfVC94xOWt1Eiu2pXLLgHYFOpc+XZTI3R//TruY2nzx175H3f6+T8zMSUOt/sdQZqxIYmiXZlhrqVa1Cht3HeS5mWuZMKwjp/xjBgCDOx7HpEu6MerV+Tx5cTe6xTYA4Ovft9GjdQNiG9bKs/3v3dCb009oEnSbiuu9BVv4bcsenrrk5DzjrbW0Gef2yde3nkGHZnVynk2U2/Z9h3hu5jqmr9iZk79/8uJu/Dm+FfvSMkhNz2B98gG27k7j5FYN6BbbgJEvziPtSBZP/7k7nVrkvQRP2p9Or0ddOvGSU2IBeOBPnah3lM+/OKy1ZGZbqnsn89wdpr7PemTPlnz26x/c2L8t1atW4flZ67j4lFgeH9k1Z7lg6yiss/KXTbu55OWf6di8HlNv7xdwntw27TrIvPW76NuuCXFNaueZlpmVTWa2Jbp6wf0CsDftCN0fnu7Kyfe9KgvD/j2HFdtT+fimPpzqBfGyFpGpmKenr+HZH9YC0LlFPZL3H+bUuEZMWbq9yGUMPKkpM73H/pbHzi/Muws2c+JxdYnPd0CsS9rP0Gfm8MPdZ3J849qFLO3/Uv5071m0blyr1PXZffAIPR+Zzv3ndeSGfkdvmW7be4hHv13J9We0oWeQ1sc9H//OJ4sSueb0OCae3zloPVZuT+Whr5fz5jW9qBkV+AtcVnyfaVGOi6xsS/L+w9SuUfWoJ0E4euAD+GRRIv3aNynSyTmUfIFp4+PDmL0mmQEdYkJ+N0l2tuWJ71ZxRe/jQ3KcBhM3dgo9Wjfg81v6lvm6jmRms2jzHvq0a1zm6/KJyMB+8zuLCnTK3Ni/bYHe+6MZ0b0FXy7eRqPaUfz6wJBi16GyKE4QKqojmdlUr2pC+uXOzrZ88MtWzu/egjo1KvfdtUOe/pGuLevz9KXdK7oq5WLfoQySUtNpf1zdiq5KyKxLOkCz+tGV/lgrqaIG9rDa+kA97d/nuktmWNdmfLs0cO92ywY1ueiUWIZ3a86Xi7cRXcT7tSura06Po30hufqSKuo97MVRpYrh8t6tQ15uWZh+15kVXYVyVb9mderXDE3Kp7LQD+Gc8I5uuA4WnxdHn5LTgfTwiM50aenymY9e2IV5Ywdy15AO1PbO5O3C/ACYeH5nRvc+vqKrISKVUFi12AOJj8ub4330wi6cfkJjLu/VmqFdmvHE1FWM7BGbM71lg5q8OLonfduFvmNORKQyCKsW+x2D2xe4LbFhrSgAfGnhBrWiGN37eIwxNK0bzdN/7l6gE25Y1+bUrxVZl6AiIj5hFtg7sHRi3nuuq3gRfdBJx1VElUREKp2wTsXMuKs/VasYfrr3LJrWq1HR1RERqRTCqsXu8/ktpxN/fEPaxbgO0NaNaxX6owURkWNNWLbYe7RuyCc3n17R1RARqZTCssUuIiKFU2AXEYkwCuwiIhFGgV1EJMKUKrAbY54yxqwyxiwxxnxujGkQqoqJiEjJlLbFPh3oYq3tBqwBxpW+SiIiUhqlCuzW2u+ttZne4Hwg9mjzi4hI2Qtljv06YGoIyxMRkRII+gMlY8wMoFmASROstV9680wAMoF3j1LOGGCMN3jAGLO6sHmDaALsKuGy4UrbfGzQNh8bSrPNRXpWd6n/g5Ix5mrgJmCQtTatVIUVbX0JRfkPIpFE23xs0DYfG8pjm0v1SAFjzFDgb8CZ5RHURUQkuNLm2J8H6gLTjTGLjTEvh6BOIiJSCqVqsVtrTwhVRYrh1QpYZ0XTNh8btM3HhjLf5lLn2EVEpHLRIwVERCJMWAV2Y8xQY8xqY8w6Y8zYiq5PSRljWhljZhljVhpjlhtjbvfGNzLGTDfGrPVeG3rjjTHmWW+7lxhjeuYq62pv/rXeHUqVmjGmqjHmN2PMN95wG2PMAq/+HxpjorzxNbzhdd70uFxljPPGrzbGnBN4TZWDMaaBMeYT79EbK40xfSJ9Pxtj7vSO62XGmPeNMdGRtp+NMW8YY5KMMctyjQvZfjXGnGKMWeot86wxvv/qXETW2rD4A6oC64G2QBTwO9CpoutVwm1pDvT03tfFPY6hE/AkMNYbPxb4p/d+GO7HXwY4DVjgjW8EbPBeG3rvG1b09gXZ9ruA94BvvOGPgFHe+5eBm733twAve+9HAR967zt5+74G0MY7JqpW9HYdZXvfAm7w3kcBDSJ5PwMtgY1AzVz795pI289Af6AnsCzXuJDtV2Ah0MdbZipwbrHqV9EfUDE+yD7AtFzD44BxFV2vEG3bl8AQYDXQ3BvXHFjtvX8FuCzX/Ku96ZcBr+Qan2e+yvaHe+TED8BA4BvvoN0FVMu/j4FpQB/vfTVvPpN/v+eer7L9AfW8IGfyjY/Y/ewF9q1esKrm7edzInE/A3H5AntI9qs3bVWu8XnmK8pfOKVifAeMT6I3Lqx5l549gAXAcdba7QDea1NvtsK2Pdw+k2eA+4Bsb7gxsNf6nzeUu/452+ZN3+fNH07b3BZIBt700k+vGWNqE8H72Vr7BzAJ2AJsx+23RUT2fvYJ1X5t6b3PP77IwimwB8oxhfUtPcaYOsCnwB3W2tSjzRpgnD3K+ErHGDMcSLLWLso9OsCsNsi0sNlmXAu0J/CStbYHcBB3iV6YsN9mL688Apc+aQHUBs4NMGsk7edgiruNpd72cArsiUCrXMOxwLYKqkupGWOq44L6u9baz7zRO40xzb3pzYEkb3xh2x5On0lf4HxjzCbgA1w65hmggTHG93uK3PXP2TZven1gN+G1zYlAorV2gTf8CS7QR/J+HgxstNYmW2szgM+A04ns/ewTqv2aSN4n5RZ728MpsP8CtPd616NwHS1fVXCdSsTr4X4dWGmtfTrXpK8AX8/41bjcu2/8VV7v+mnAPu9SbxpwtjGmoddSOtsbV+lYa8dZa2OttXG4fTfTWjsamAVc7M2Wf5t9n8XF3vzWGz/Ku5uiDdAe19FU6VhrdwBbjTEneqMGASuI4P2MS8GcZoyp5R3nvm2O2P2cS0j2qzdtvzHmNO8zvCpXWUVT0R0QxeysGIa7g2Q97umSFV6nEm7HGbhLqyXAYu9vGC63+AOw1ntt5M1vgBe87V4KxOcq6zpgnfd3bUVvWxG3fwD+u2La4r6w64CPgRre+GhveJ03vW2u5Sd4n8Vqinm3QAVsa3cgwdvXX+Dufojo/Qw8BKwClgFv4+5siaj9DLyP60PIwLWwrw/lfgXivc9vPe7RLaY49dMvT0VEIkw4pWJERKQIFNhFRCKMAruISIRRYBcRiTAK7CIiEUaBXUQkwiiwi4hEGAV2EZEI8/8BEXaNl1SWdQkAAAAASUVORK5CYII=\n", 499 | "text/plain": [ 500 | "
" 501 | ] 502 | }, 503 | "metadata": {}, 504 | "output_type": "display_data" 505 | } 506 | ], 507 | "source": [ 508 | "plt.plot(range(TRAIN_ITERS),histd, label='obj_d')\n", 509 | "plt.plot(range(TRAIN_ITERS), 1-histg, label='obj_g')\n", 510 | "plt.legend()\n", 511 | "#plt.savefig('fig4.png')" 512 | ] 513 | }, 514 | { 515 | "cell_type": "code", 516 | "execution_count": 23, 517 | "metadata": {}, 518 | "outputs": [ 519 | { 520 | "data": { 521 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8VFX+//HXyWTSG5IEkJZQQg2EjtJEXEREFKWIYhdcEDt2bPBzV921L4ioXxRFRECkiNjoTaXXBEIIEGpCSS+TmfP74yYhCWmQmUwy+Tx355EpZ+58RpL3nDn33HOV1hohhBCuxc3ZBQghhLA/CXchhHBBEu5CCOGCJNyFEMIFSbgLIYQLknAXQggXJOEuhBAuSMJdCCFckIS7EEK4IHdnvXBwcLAOCwtz1ssLIUSNtHXr1iStdUh57ZwW7mFhYWzZssVZLy+EEDWSUupIRdrJsIwQQrggCXchhHBBEu5CCOGCnDbmLoQon8ViISEhgaysLGeXIqqYl5cXjRo1wmw2X9HzJdyFqMYSEhLw9/cnLCwMpZSzyxFVRGvN2bNnSUhIIDw8/Iq2IcMyQlRjWVlZ1K1bV4K9llFKUbdu3Up9Y5NwF6Kak2CvnSr77y7hLoQQLkjCXQghXJCEuxDCIVavXs2QIUPKbLNjxw6WL19eRRXVLhLuQginkXB3HJkKKUQN8cbSvew7kWLXbba9OoDXbmlXZpv4+HgGDRpEjx492L59OxEREcyePRsfH59L2q5YsYInn3yS4OBgOnfuXHD/X3/9xZNPPklmZibe3t7MmjWL8PBwXn31VTIzM1m/fj0vvvgi4eHhl7Rr1aqVXd9zbSE9dyFEuWJiYhg3bhy7du0iICCA6dOnX9ImKyuLsWPHsnTpUtatW8epU6cKHmvdujVr165l+/btTJkyhZdeegkPDw+mTJnCqFGj2LFjB6NGjSqxnbgy0nMXooYor4ftSI0bN6ZXr14AjBkzho8++ohJkyYVaRMdHU14eDgtW7YsaDdz5kwAkpOTue+++zh48CBKKSwWS4mvU9F2onzl9tyVUv+nlDqjlNpTyuNKKfWRUipWKbVLKdW5pHZCiJqr+Jzr0uZgl3b/K6+8Qv/+/dmzZw9Lly4t9eCcirYT5avIsMyXwKAyHr8JaJl3GQd8UvmyhBDVydGjR9m0aRMAc+fOpXfv3pe0ad26NYcPH+bQoUMF7fIlJyfTsGFDAL788suC+/39/UlNTS23nbh85Ya71notcK6MJrcCs7VhMxCklGpgrwKFEM7Xpk0bvvrqKzp06MC5c+cYP378JW28vLyYOXMmN998M71796Zp06YFjz333HO8+OKL9OrVC6vVWnB///792bdvH1FRUcybN6/UduLyKa11+Y2UCgOWaa3bl/DYMuAtrfX6vNt/AM9rrcs8zVLXrl21nIlJiLLt37+fNm3aOLWG+Ph4hgwZwp49JY7MCgcq6d9fKbVVa921vOfaY7ZMSYNsJX5iKKXGKaW2KKW2JCYm2uGlhRBClMQes2USgMaFbjcCTpTUUGs9E5gJRs/dDq8thHCwsLCwS3rtw4YN4/Dhw0Xue/vtt7nxxhursjRRBnuE+xJgolLqO6AHkKy1PmmH7QohqqlFixY5uwRRjnLDXSk1F7gOCFZKJQCvAWYArfUMYDkwGIgFMoAHHFWsEEKIiik33LXWo8t5XAOP2q0iIYQQlSbLDwghhAuScBdCVNjrr7/Of//73yt67rXXXlvm44MHD+bChQtXtO3C7r//fhYsWFDp7VwOPz+/Kn29ipC1ZYQQVWLjxo1lPl5blv7VWqO1xs3NsX1r6bkLIcr05ptv0qpVK2644QZiYmIK7j906BCDBg2iS5cu9OnTh+joaABOnz7NsGHD6NixIx07diwI9fze7cmTJ+nbty9RUVG0b9+edevWAcaUy6SkJADee+892rdvT/v27fnggw8A42CqNm3aMHbsWNq1a8fAgQPJzMwssebff/+dPn36EBERwbJlywBj1coHHniAyMhIOnXqxKpVqwBjmYOJEycWPHfIkCGsXr26oOaXX36Zjh070rNnT06fPg3A4cOHueaaa+jWrRuvvPJKwXPT0tIYMGAAnTt3JjIyksWLFxepfcKECXTu3JmpU6fy1FNPFTzvs88+4+mnn77sf5uySM9diJri5xfg1G77brN+JNz0VqkPb926le+++47t27eTm5tL586d6dKlCwDjxo1jxowZtGzZkj///JMJEyawcuVKHn/8cfr168eiRYuwWq2kpaUV2ea3337LjTfeyMsvv4zVaiUjI+OS15w1axZ//vknWmt69OhBv379qFOnDgcPHmTu3Ll89tlnjBw5koULFzJmzJhL6o6Pj2fNmjUcOnSI/v37Exsby7Rp0wDYvXs30dHRDBw4kAMHDpT5nyc9PZ2ePXvy5ptv8txzz/HZZ58xefJknnjiCcaPH8+9995bsF0wlmBYtGgRAQEBJCUl0bNnT4YOHQoYyybPmjWL6dOnk56eTocOHXjnnXcwm83MmjWLTz/9tMxaLpeEuxCiVOvWrWPYsGEFJ+bID6q0tDQ2btzIiBEjCtpmZ2cDsHLlSmbPng2AyWQiMDCwyDa7devGgw8+iMVi4bbbbiMqKqrI4+vXr2fYsGH4+voCcPvtt7Nu3TqGDh1KeHh4QfsuXboQHx9fYt0jR47Ezc2Nli1b0qxZM6Kjo1m/fj2PPfYYYCxy1rRp03LD3cPDo+BUgV26dOG3334DYMOGDSxcuBCAe+65h+effx4whlxeeukl1q5di5ubG8ePHy/o7Tdt2pSePXsC4Ovry/XXX8+yZcto06YNFouFyMjIMmu5XBLuQtQUZfSwHamkZXxtNhtBQUHs2LHjsrfXt29f1q5dy08//cQ999zDs88+y7333lvweFnrXXl6ehZcN5lMpQ7LlLREcWnbdXd3x2azFdwuvMyw2Wwu2JbJZCI3N7fU1wCYM2cOiYmJbN26FbPZTFhYWMH28j+s8j388MP861//onXr1jzwgP0PD5IxdyFEqfr27cuiRYvIzMwkNTWVpUuXAhAQEEB4eDjz588HjEDeuXMnAAMGDOCTT4yVv61WKykpRU8NeOTIEUJDQxk7diwPPfQQ27Ztu+Q1f/zxRzIyMkhPT2fRokX06dPnsuqeP38+NpuNQ4cOERcXR6tWrejbty9z5swB4MCBAxw9epRWrVoRFhbGjh07sNlsHDt2jL/++qvc7ffq1YvvvvsOoGCbYCxZHBoaitlsZtWqVRw5cqTUbfTo0YNjx47x7bffMnp0mYcTXREJdyFEqTp37syoUaOIiorijjvuKBKyc+bM4YsvvqBjx460a9euYOfhhx9+yKpVq4iMjKRLly7s3bu3yDZXr15NVFQUnTp1YuHChTzxxBOXvOb9999P9+7d6dGjBw8//DCdOnW6rLpbtWpFv379uOmmm5gxYwZeXl5MmDABq9VKZGQko0aN4ssvv8TT05NevXoRHh5OZGQkkyZNKnLu19J8+OGHTJs2jW7dupGcnFxw/913382WLVvo2rUrc+bMoXXr1mVuZ+TIkfTq1Ys6depc1vuriAot+esIsuSvEOWrDkv+CscZMmQITz31FAMGDCjxcWcv+SuEEOIyXLhwgYiICLy9vUsN9sqSHapCCFHFgoKCyp2pU1nScxdCCBck4S6EEC5Iwl0IIVyQhLsQQrggCXchhHBBEu5CCOGCJNyFEMIFyTx3IWqIt/96m+hz0XbdZuurWvN89+fLbBMfH8+gQYPo0aMH27dvJyIigtmzZxesFFnY8uXLefrppwkODqZz587ExcUVrKcuqpb03IUQ5YqJiWHcuHHs2rWLgIAApk+ffkmbrKwsHnnkEX7++WfWr19PYmKiEyoV+aTnLkQNUV4P25EaN25Mr169ABgzZgwfffQRkyZNKtImOjqaZs2aER4eDsDo0aOZOXNmldcqDNJzF0KUq6T10Ytz1iKEomQS7kKIch09epRNmzYBMHfuXHr37n1Jm9atWxMXF1dwdqR58+ZVZYmiGAl3IUS52rRpw1dffUWHDh04d+4c48ePv6SNt7c306dPZ9CgQfTu3Zt69epdcoo9UXVkzF0IUS43NzdmzJhRbrv+/fsTHR2N1ppHH32Url3LXXZcOIj03IUQdvPZZ58RFRVFu3btSE5O5pFHHnF2SbWW9NyFEGUKCwtjz549Re4bNmwYhw8fLnLf22+/zVNPPcVTTz1VleWJUki4C1HNaa1LnJ3iTIsWLXJ2CS6vsrOPZFhGiGrMy8uLs2fPyjTDWkZrzdmzZ/Hy8rribUjPXYhqrFGjRiQkJMjRnrWQl5cXjRo1uuLnVyjclVKDgA8BE/C51vqtYo83Ab4CgvLavKC1Xn7FVQkhADCbzQVHfApxOcodllFKmYBpwE1AW2C0UqptsWaTge+11p2AO4FLF54QQghRZSoy5t4diNVax2mtc4DvgFuLtdFAQN71QOCE/UoUQghxuSoyLNMQOFbodgLQo1ib14FflVKPAb7ADXapTgghxBWpSM+9pDlYxXfdjwa+1Fo3AgYDXyulLtm2UmqcUmqLUmqL7CASQgjHqUi4JwCNC91uxKXDLg8B3wNorTcBXkBw8Q1prWdqrbtqrbuGhIRcWcVCCCHKVZFw/xtoqZQKV0p5YOwwXVKszVFgAIBSqg1GuEvXXAghnKTccNda5wITgV+A/RizYvYqpaYopYbmNXsGGKuU2gnMBe7XctSFEEI4TYXmuefNWV9e7L5XC13fB/Syb2lCCCGulCw/IIQQLkjCXQghXJCEuxBCuCAJdyGEcEES7kII4YIk3IUQwgVJuAshhAuScBdCCBck4S6EEC5Iwl0IIVyQhLsQQrggCXchhHBBEu5CCOGCJNyFEMIFSbgLIYQLknAXQggXJOEuhBAuSMJdCCFckIS7EEK4IAl3IYRwQRLuQgjhgiTchRDCBUm4CyGEC5JwF0IIFyThLoQQLkjCXQghXJCEuxBCuCAJdyGEcEES7kII4YIk3IUQwgVJuAshhAuqULgrpQYppWKUUrFKqRdKaTNSKbVPKbVXKfWtfcsUQghxOdzLa6CUMgHTgH8ACcDfSqklWut9hdq0BF4EemmtzyulQh1VsBBCiPJVpOfeHYjVWsdprXOA74Bbi7UZC0zTWp8H0FqfsW+ZQgghLkdFwr0hcKzQ7YS8+wqLACKUUhuUUpuVUoNK2pBSapxSaotSaktiYuKVVSyEEKJcFQl3VcJ9uthtd6AlcB0wGvhcKRV0yZO0nqm17qq17hoSEnK5tQohhKigioR7AtC40O1GwIkS2izWWlu01oeBGIywF0II4QQVCfe/gZZKqXCllAdwJ7CkWJsfgf4ASqlgjGGaOHsWKoQQouLKDXetdS4wEfgF2A98r7Xeq5SaopQamtfsF+CsUmofsAp4Vmt91lFFCyGEKJvSuvjwedXo2rWr3rJly+U/8fQ+OL0XglsaFw9f+xcnhBDVlFJqq9a6a3ntyp3nXu1EL4NVb168HdgYgiPyLi0hpJVx3TcEVEn7goUQwvXVvHDv9QS0uQWSDkDiAeNn0gHYNhss6RfbeQVdDP2QiIvXg5qCqea9bSGEuBw1L+XcPSG0jXEpzGaD1BOQGANJBy+GfuxvsOObi+1MHnBV80KB30qGeIQQLqfmhXtp3NwgsJFxaTGg6GOZ5yEpNi/w88L/1B7YvxS07WK7/CGe/KGdkFZG+PvWrdr3IoQQleQ64V4W7zrQuJtxKSw3G84dNgI/MS/4E2PgyEbIzbzYzqeuEfIhEUV/BjaScf3aIPk4HNtsdBDOx0PGWbBkgDXH+Cbp7g2efuBXz7j418+7XA0BVxuPCVHFake4l8bdE0JbG5fCbDZIPpY3rh9zcYhn3xLIPHexndm36E7c/J7+VeFgMlftexH2dWo37J4P+xYbgZ7P/2rwDTaG8Ny9jA5CVjIkpUDamaL7ffJ5BhohH9Ag72dD42d++AdcbXRApKMg7Kh2h3tp3NygTlPj0vIfRR9LT8oL/EK9/fgNsGteoee7FxrXb1V0p66M61dv8Rtg9b8hfp3x79isP3R/BJpeAyFtwOxV9vOzUyH1NKSeNC4pxyEl/+cJYypv2mkuWcHD3dv40PAMAK8A8PQvet3DL+/im3cp4bqnn9HhkAkDAgn3y+cbbFzCehW9Pzv14o7c/N7+mWiIXg7aerGdjOtXT0c2GqF+eC34hsLAN6Hj6Mv/d/H0Ny7BLUpvY7UYAZ9yomj4Z5w1fo+yko3Hz8ZCVopxnzW74jW4e5XwIVD8Z6HrPnWNbxX+ed8uPP0v7z2LaknC3V48/aFhZ+NSWG4OnIu7snH9ui2NcX03U9W+l9rkyCZY/a+LoX7jv6HL/eDh47jXNJkv7vyvKKsFctILXdKKXS9+O+96durF62lnCj2WBrlZJb+Wh39e2DfI+wYbDlc1M4Yb64Qb3yZEtSfh7mjuHhUc188L/32Ljdk9+dzMENQ47w8sHOqEFb0uwzxXJn4DrHkbDq/JC/V/QZcHHBvqlWEyg3eQcbEXa64R8hlnjW8RqSfzfp4yphWnnICYnyG92PLcPsFQtzmEtIbQthenJsuBg9VKzVt+wNVpbYzrJ8XA2UNw/rAxo+d8vHE9K7loe99QI+iDmhTaWdfw4nXfEGMfgjD+2x76A9a+C0c3Gv9tej0JXR+svqFeHWSnGr+D5+Lyfh/jjJlDifuLdkR86hr7JULbQIOOcHWU8QEgkwvsqqLLD0i41zQZ5y4G/fn4i8GffMzoaVlzirZ3MxtfrwMbGtPzfIIv7jcouB5iXPeu45ofBGlnYOdc2PoVnDtkfOj1egI63SOhXhlaG/9tz+yDM/uNsD+Td8lJM9qYPKF+e2gQZYR9gygj/CXwr5iEe21ks+V9xT5eaGddoZ+pJ41vBVkXStmAKjRDo6RZG/kzN3zB7HNxx1z+dbOPEZZm37yfPs75mp6bDSd3wZH1xg7thL8BDU2ugc73Qfs7jOEy4Rg2m/EhemIHnNyR93Mn5KQaj5s8jaBv1A0a94DG3Y2Oh6gQCXdROqvF+BBIT4KMJONnelLebI2UvBkaKcYQUHbebI38+4p/MyhP4aAv7QPAw6+ENsXaFp/eZ7NBdrJRY+YF45tL/tDB6b1gsxjtGnSEVjdDu9uMmUnCOWw249/m5A44sd34wD2x/eLvU1ATaNQ9L+y7Qb320rsvhYS7cIzcbGO2hSUDcjKMg3Zy0gtdz8h7LP3iz8LXizyvWLvCU0YvlzIZAVEnDBp0MHqFjbpJj7A6y/+GdexPSPgLjv1lfLsE44O9UVdo2huaXmv8W5Z3jEEt4bpL/grncvc0Llxl3+1qbfTiCn8A5KRdvF48+JWbMUzkHWT89KsnB+/UNO6eRZcF0RqSE4ywP/aXsdN79b8BbSz417CrEfRhvYwevswUK5P03IUQ1VfmeTi6GY5sMKavntxpfNC7uRs7Z8N65fXur6k1B1/JsIwQwvVkpxo9+/gNRuAf32bsX1EmaNgFmvWD8H7GTlp3T2dX6xAS7kII15eTYYzXH14LcWvgxDZjGW93b2jSE8L7GoHfIMpljvSWcBdC1D5ZyUav/vBa4+jjM/uM+70CIayP0asP72vMnKqhR9PKDlUhRO3jFQitBxsXMA6yyg/6uDXGOZgB/Opf7NWH9zOW+HAxEu5CCNflFwqRw40LGEdzx63JC/tVsPt74/6rmkPz/sYSz+F9jA+JGk6GZYQQtZPWxlIJcauNoI/fYBx/kb9zNj/sG3WtVgdUyZi7EEJcjtwcY+fsoVVG2J/Ybuyc9fCHsN4Xwz64pVPH6yXchRCiMjLPG+P1+WGff7rFgEbQ7Lq8sL/OWHyvCskOVSGEqAzvOtD2VuMCxtpFcauMsI9eCju+Me6vHwnNrzd69U2uqTbLJEjPXQghLpfNaqx2GbcSDq02DqyyWYxTHDa55uIQTr32dl9GW4ZlhBCiqmSnGafOzO/ZJ+437vcJLjSE0984r0IlybCMEEJUFU8/iBhoXMA46Xn+LJxDq2DPAuP+4Agj5KPuMta0dyAJdyGEsLeABhA12rhobZxjID/ot82GqztVj3BXSg0CPgRMwOda67dKaTccmA9001rLmIsQQihlnGqwfnu49jGwZFXJy5Y70q+UMgHTgJuAtsBopVTbEtr5A48Df9q7SCGEcBlmryqZUVOR3bjdgVitdZzWOgf4Dri1hHZTgXeAqvlYEkIIUaqKDMs0BI4Vup0A9CjcQCnVCWistV6mlJpkx/pELZFlsbLxUBJ/xp3jUGI6iWnZoDV1/TwJD/ale/hV9G4RjK+n7CYSoiIq8pdS0nG2BfMnlVJuwPvA/eVuSKlxwDiAJk2aVKxC4dISzmfw+brD/LAtgZSsXDxMbjQL8SXE3xOlFKeSs9h4KIkv1h/G18PE0KiGjOvbjPBgOcWaEGWpSLgnAIXXw2wEnCh02x9oD6xWxnoL9YElSqmhxXeqaq1nAjPBmOdeibpFDZeenctHfxxk1sZ4tNYMat+AEV0a0T38KrzMRU+qkJ1rZeuR8/yw7TiLticwf8sxRndvwqQbWxHoXX0WdBKiOin3ICallDtwABgAHAf+Bu7SWu8tpf1qYFJ5s2XkIKbaa+uR8zz9/Q6Onsvg9k6NeHpgBA2DvCv03DOpWXz8Ryzf/nWUED9P/jOiA31ahji4YiGqj4oexFTuDlWtdS4wEfgF2A98r7Xeq5SaopQaWvlSRW3y9aZ4Rn66CatN893Ynrw7smOFgx0g1N+Lqbe154fx1+LraeKeL/7ifysP4qwjrYWormT5AVElrDbNG0v3MnvTEQa0DuWDO6Pw96rckEqWxcoLC3fx444TDO14Nf8d0REPd/uu4yFEdSPLD4hqI9dq46nvd7J05wnG9gnnhZvaYHKr/HrYXmYT74+KIqK+P++siCEtO5fpd3e+ZMxeiNpIujnCoSxWG49/t52lO0/w/KDWvHxzW7sEez6lFBOua8Gbw9qzKuYMD331N1kWq922L0RNJeEuHEZrzfMLd7F89ykm39yG8dc1d9hr3d2jKf8Z3pENsWd5fO52cq02h72WEDWBhLtwmP/8EsMP247z9D8ieLhPM4e/3vAujXjtlrb8uu80k3/cIztZRa0mY+7CIb7efITpqw8xunsTHru+RZW97gO9wklKy2baqkNcHeTN4wNaVtlrC1GdSLgLu9scd5bXl+xlQOtQpt7aDlXFJxOeNLAVJy5k8d5vB2jTIIB/tK1Xpa8vRHUgwzLCrk5cyOTROdtoWteHD+6Mwt1U9b9iSin+fXskHRoF8tS8HcSeSa3yGoRwNgl3YTdZFiv//GYr2bk2Zt7TtdLz2CvDy2xixpgueJndGDt7KylZFqfVIoQzSLgLu5m6bB+7EpJ5b2RHWoT6Obscrg7yZvrdXTh6LoOXF8kOVlG7SLgLu1ix5xRz/jzKuL7NGNiuvrPLKdA9/Cqe/kcES3eeYP6WBGeXI0SVkXAXlXYyOZMXfthFZMNAJg1s5exyLvHPfs25tnldXluyV8bfRa0h4S4qxWrTPDVvBzm5Nj68M6paru1iclO8PyoKbw8TE7/dLkewilqh+v0lihplxppDbI47x+tD29EsxPnj7KWpF+DFuyM6En0qlfd+O+DscoRwOAl3ccViTqXywe8HuLmDcaKN6q5/61Du6tGEz9bFsSX+nLPLEcKhJNzFFbHaNM8t3IW/l5kpQ6v+QKUr9dLgNjQM8uaZ+TvJyMl1djlCOEyNC/eUnBRWxK9wdhm13qwNh9l57AKvD21HXT9PZ5dTYX6e7vxneEeOnM3g7Z+jnV2OEA5T48L9yz1f8uyaZ5m+Y7rMW3aSI2fT+e+vMdzQJpRbOjRwdjmX7Zrmdbn/2jC+2nSEjbFJzi5HCIeoceE+Pmo8tza/lU92fsLrm14n1yZfrauS1poXFu7G7ObG1Nva15jhmOKeH9Sa8GBfnl2wi/Rs+R0SrqfGhbvZzczUXlN5pMMj/HDwBx5f+TgZlgxnl1VrzPv7GJvizvLi4DY0CKz4uU+rG28PE+8M78DxC5n899cYZ5cjhN3VuHAHY2GoiZ0m8krPV9hwYgMP/fIQZzPPOrssl3cqOYs3f9pPz2ZXcWe3xs4up9K6hV3FPT2b8uXGeLYfPe/scoSwqxoZ7vlGthrJB9d9QOyFWO75+R6Ophx1dkkuS2vN5B/3YLHZeOv2DrjZ8VR5zvTcoFbU8/fihYW7ycmVszcJ11Gjwx2gf5P+fH7j56TmpHLPz/ewO3G3s0tySct2neT3/ad55h+tCAv2dXY5duPvZeb/3daemNOpzFhzyNnlCGE3NT7cATqGdOTrm77G292bh359iLUJa51dkks5l57D60v20rFRIA/0CnN2OXZ3Q9t6DOnQgP+tjJW1Z4TLcIlwBwgLDOObwd8QHhjO4ysfZ+GBhc4uyWVMXbaP5EwLbw/v4JSTb1SF125ph7eHiecX7sZmkym2ouZzqb/UYO9gZt04i55X9+T1Ta8zbcc0mQtfSauiz7Bo+3Em9G9B6/oBzi7HYUL8PXllSFu2HjnPnD+POLscISrNpcIdwMfsw8fXf8xtLW5jxs4ZvLbxNSw2OQvPlUjNsvDyot1E1PPj0f7NnV2Ow93RuSF9Wgbz1s/RnLiQ6exyhKgUlwt3MObCT7l2Co90eIRFsYtkLvwVentFNCdTsnj7jg54upucXY7DKaX417BIbBom/yhnbhI1m0uGO1ycC//qNa+y8cRGHvzlQZkLfxn+jDvLN5uP8mCvcDo1qePscqpM46t8eGZgBCujz7Bk5wlnlyPEFXPZcM83ImIEH/b/kEMXDslc+ArKslh54YfdNMkLutrmgV7hdGwcxBtL93E2LdvZ5QhxRVw+3AGua3wdX9z4BWk5aYxZPkbmwpfjg98Pcjgpnbduj8THw93Z5VQ5k5vinTs6kJplYcqyfc4uR4grUivCHaBDSAe+Hvw1vmZfHvr1IdYcW+Pskqql3QnJfLYujju7NebaFsHOLsdpWtX359H+LVi84wR/7D/t7HKEuGwVCnel1CClVIxSKlYp9UIJjz+tlNq22c1+AAAWmElEQVSnlNqllPpDKdXU/qVWXtOApnw9+GtjLvyqx1lwYIGzS6qUhNQEPtz2IYtjF9tlezm5Np5dsJO6vh68OLiNXbZZk024rgWt6vnz8qI9pGbJjCtRs5Qb7kopEzANuAloC4xWSrUt1mw70FVr3QFYALxj70LtJX8u/LVXX8sbm96ocXPhtdZsPLGRx1Y+xuAfBvP57s+ZvGEy7299H5uu3Noo01fHEn0qlX8NiyTQ22ynimsuD3c33h7egTOpWbwlJ/YQNUxFeu7dgVitdZzWOgf4Dri1cAOt9Sqtdf5cw81AtT6hpo/Zh4+u/4hhLYYxY+cMXt34arWfC59uSWdu9FxuXXwrj/z2CLsSdzG2w1h+veNXRkaM5P/2/B8vrHuBHGvOFW1//8kU/rcyltuiruaGtvXsXH3NFdU4iAd7hTPnz6NsjpPZVqLmqMjesobAsUK3E4AeZbR/CPi5pAeUUuOAcQBNmjSpYImOYXYz88a1b1DPtx4zds4gKTOJd/u9i4/Zx6l1FXck5Qhzo+eyOHYxaZY02tdtz796/4sbw27Ew+QBwOSek2ng14APt31IYkYiH/T/gEDPwAq/Rq7VxnMLdhHkY+a1W9o56q3UWE8PjODXfad5YeEuVjzZFy+z68/5FzVfRXruJa3tWuI4hlJqDNAV+E9Jj2utZ2qtu2qtu4aEhFS8SgdRSvFo1KO8ds1rbDyxkQd+eYCkTOefds2mbaxNWMs/f/8nQxYNYV7MPPo17secwXOYO2QutzS/pSDYwXgfD0c+zFt93mJH4g7u+/k+TqRVfI72zHVx7D6ezJRb21PH16P8J9QyPh7uvHV7JPFnM3j/9wPOLkeICqlIzz0BKHxmhkbAJcmhlLoBeBnop7WuUZODh0cMJ8Q7hElrJnHP8nuY8Y8ZNA2o+n3CqTmp/Bj7I99Ff8fR1KOEeIcwIWoCIyJGEOxd/syVm5vdTKhPKE+sfIK7l9/N9AHTaVO37B2jsWdS+eD3gwyOrM/gyJp3PtSqcm2LYO7s1pjP1sYxuH0DOjYOcnZJQpRJlbczUSnlDhwABgDHgb+Bu7TWewu16YSxI3WQ1vpgRV64a9euesuWLVdat0PsStzFxD8mAvC/Af+jQ0iHKnndQxcOMTd6LksOLSEzN5OokCjuanMXNzS5AbPp8ndsxp6PZfwf40nJTuHd696ld8PeJbbLtdoY+ekm4pLS+e2pfoT4e1b2rbi05EwLN76/Fl9PEz893keGZ4RTKKW2aq27lteu3GEZrXUuMBH4BdgPfK+13quUmqKUGprX7D+AHzBfKbVDKbWkErU7TZG58L84di681Wblj6N/8PCvD3Pb4ttYdHARA5sOZN6QeXw9+GtuCr/pioIdoEWdFswZPIcmAU2Y+MdEfjj4Q4ntZqw5xLajF3hjaDsJ9goI9Dbz3xEdOZSYLrNnRLVXbs/dUapjzz1fUmYSE/+YyP5z+5ncczIjIkbYbdsXsi7wQ+wPzIuex4n0E9T3rc+oVqO4o+Ud1PGy7xou6ZZ0nln9DBtObOCRDo/waNSjKGXsQtmVcIHbp29kUPv6fDy6U8H9onyvL9nLlxvj+eahHvRuWXsP9BLOUdGeu4R7KTIsGTyz5hnWH19/STBeiZhzMXwb/S0/xf1EtjWbbvW7cVfru7iu8XW4uznuEH+LzcLUTVNZFLuIoc2H8vo1r5NrdePmj9eRkW3llyf7Eugjc9ovR2aOlSEfryMjx8qKJ/vKMQGiSlU03GvfwiEVlD8XfuqmqXy661NOZ5zm1WtexexW8T9ki83CyqMr+Xb/t2w7sw0vkxe3NL+F0a1HE1Gnahbkyp/y2cCvAdN3TOdMxhlCMsYSl5jOnId7SLBfAW8PE++PiuL26Rt5bfEePrizk7NLEuISEu5lyA/G+r71+WTnJyRmJvJev/fKnQt/NvMsCw8uZF7MPM5knKGhX0MmdZ3EbS1uu6z55/ailGJ8x/Fc7Xs1r258DUtmPKOveY1etXjtmMrq0CiIx65vyfu/H6B/61BujWro7JKEKELCvRxKKSZETaCeTz2mbp7KA788wLQB00qcmrgnaQ/f7v+WFfErsNgsXNPgGl7p+Qp9GvbB5Ob8mRU9Q2/E7fQx3EO+4q+c1zlwfnqVfYNwRY/2b866g4m89MNuOjQKIjzY19klCVGg1qwKWVl3RNzBR9d/xOHkw4xZPob45HgALFYLy+KWcfdPdzP6p9H8cfQP7mh5B4tvW8zMgTO5rvF11SLYc602Hp+7nayUFvy316egNff9fB+bT252dmk1lrvJjY9Gd8Ls7saEOdvIslidXZIQBWSH6mXanbibR/94FI3mtha3sfTQUs5mnaVpQFNGtx7Nrc1vxc/Dz9llXuLdX2P4eGUs747oyB1dGnEq/RTjfx9PfEo8U66dwi3Nb3F2iTXWyujTPPjlFu7u0YQ3h0U6uxzh4uw2z10UFRkSyTeDv8Hfw5+v9n5F27pt+eSGT1hy2xLubnN3tQz2tQcS+d+qWEZ0acQdXYw13er71mf2TbPpEtqFl9a/xMxdM2vU6pjVyfWt6/FI32bM+fMoS+XUfKKakJ77FUrLSSM1J5UGftX7kP2E8xkM/d8Ggv08WPxob7w9ig4RWawWXtv4GkvjlnJHyzuY3HOyQ6dmuiqL1caoTzcRcyqVHx/tRct6/s4uSbgo6bk7mJ+HX7UP9oycXMbO3ool18b0u7tcEuwAZpOZN3u/ydjIsSw8uJDHVj5GhiWjhK2JsphNbky7uzPeHu48PHsLFzKubOllIexFwt1Faa2ZNH8n0adS+OiuTrQILX24SCnF450f59VrXmXTiU3cv+L+arE6Zk3TINCbT+/pwskLWUz8dju51sqdPEWIypBwd1Efr4xl+e5TvHhTa/q3Cq3Qc0ZEjOCj6z8iPiWeu3+6m7gLcQ6u0vV0aVqH/zesPetjk3hz+X5nlyOqGa01e5L2kJyd7PDXknB3QYu2J/DebwcY1qkhY/s0u6zn9m3Ul1mDZpFtzWbMz2PYcqrm7hdxlpFdG/Ngr3BmbYhn1obDzi5HVANpOWl8H/M9I5eNZPRPo/kx9keHv6aEu4tZcyCRZ+fv4ppmdXnrjsgrWg+nXd12zLl5DsHewYz7bRw/Hy7xxFqiDC/f3IZB7eozZdk+lu2SGTS1UX4v/bWNr3H9/OuZunkqWmsm95jM7S1vd/jry2wZF7Ir4QJ3ztxM07q+zHukJwFelVs3Jjk7mSdWPcHW01t5usvT3N/uflk98jJkWazc+8Vf7Dh2gS8f6Ma1stxDrZCWk8ZPcT+x4OACos9F4+3uzaCwQYyIGEH74PaV/huSVSFrmYOnU7lz5ma8PUz8MP5aQgO87LLdbGs2k9dPZkX8Cu5sdScvdH+hWhxxW1MkZ1gY8elGTlzI4uuHutOpiX2XdRbVg9aavWf3Mv/AfH4+/DOZuZm0qtOKEREjGNxsMP4e9psaK6tC1iIHT6cy+rPNmNwUXz/Uw27BDuBp8uTtvm/TwK8Bs/bM4lTGKd7p+w7e7t52ew1XFuhj5qsHu3PnzM3c+8VffPVQdzpLwLuM1JxUlsctL9JLvyn8Joa3HG6XXnplSM+9hssPdjelmDuuJ81DHHeE7Nzoubz111u0q9uOj6//mLredR32Wq7mZHImd87czLm0HAn4Gi5/LH3BwQWX9NJvbnazw49Sl2GZWmD70fM89NUW3N0cH+z5Vh5dyfNrnyfYO5hPbviEsMAwh7+mqziZnMnomZtJSsth5r1duLa5jMHXJKX10kdEjKBd3XZV1kuXcHdxK6NP8+ic7YT4ezL7we6EVeFys7sSd/HYysewaRsfX/8xUaFRVfbaNd2p5Czu/b8/OZyUzrsjoxja8WpnlyTKUFIvvfVVrY2x9PDBTllLSsLdhc358wivLt5Lmwb+zLq/u1NObn0s5Rjj/xjPqfRTvNXnLW5oekOV11BTJWdaGDd7C38ePscLN7Xmkb7NZBZSNZPfS59/YD4x52PwdvdmcPhghkcMr9Jeekkk3F1Qdq6V15fsZe5fx+gXEcK0uzvj5+m8feLns87z2MrH2JW4i+e6PceYtmOcVktNk51r5Znvd7Js10mGdGjAO8M74OMh8xucKb+XPv/AfFbEr6gWvfSSSLi7mOMXMpn47Ta2H73AhOua88zAVpjcnN/by8rN4sV1L/L70d8J9Q4l1KfkSz2feoT6hFabP5DqQGvNJ2sO8d9fYmgZ6s8nYzrTrAr2m4iiUnNSjXnpBxZUu156SSTcXYTWmh93HOfVH/di05r/jOjI4MjqtRql1WZlXsw8os9FcybjDKczTnMm4wwpOSmXtPVx9ykS9oXDP8QnhFCfUIK9g2vVssPrDybx2NxtZFlsvHRzG8b0aFLtAsXVaK3ZnbSbBQcWVOteekkk3F3AmdQs3liyj592n6Rr0zq8NzKKJnXLPjl3dZKZm0liRmJB2Odf8m8nZiRyJvMMubbcIs9zU27U9apbYs+/8MXP7OcyIXgqOYtnF+xk3cEk+kaE8O/bI2kYJMcS2FtpvfQRESNoW7dtjfh9knCvwXKtNr7efIT3fj1Adq6NJ25oyT/7Na8WwzD2ZtM2zmedvyT4i98u6VuAt7t3qaHv7e6Nj9kHH3efS657u3tXy6NstdZ8s/kIby7fj0Ix8foWPNwnHE/36ldrdWKxWUjNSSUlO4WUnBTjek7KJbfPZZ1j88nNZOZm0uaqNgyPGF7te+klkXCvgbTW/LrvNO/9eoCY06n0aRnMlFvbE16F0xyrq/xvASUFf8GlhG8BpfEyeeFjvhj2+dd93H3KvF7Sh0X+dS93L9xU5dfiSzifwdRl+/hl72nCg3156h8RDIlsgJsLfrjny8rNuhjKhYK5+O3ibVJzUsnILfvkMh5uHgR4BhDgEUCn0E41qpdeEgn3GsRm06w5kMj7vx9gV0IyzYJ9efbGVgxqX7/G/gI6g03bSM5OJt2STkZuBhmWDDJzM4tcz8zNJMOSQUZuRoWvX478Dwtvd2/c3dwxKRMmNxPuyv2S2yY3U8F9hR8zKRNmNzOnk3PYfjSF8+m51PHxoltYMBGhQbibLj4//7nFbxfelkZj0zY0GuP/Gq01NmwF583Nv6/En/nXi98u4Wf+v0Px+7Jysy4J5cK3c2xln7nKx92HAM8A/D38CfAIKLj4e/gXBHfBxbPo417u9luOozqQtWVqgPTsXBZuS+DLjfHEJabTMMibd4Z34PZODXE3yWrMl8tNuVHHqw51vOx3aL9N28jKzTKC3pJZJPwLf3AU/1DIys0iV+eSa8vFarNi1VZy9cXrObYcrLlF78u15bXXVqw24zH3ACv+PhbSrRZWnbKx+rTVbu+tKikU/h7+F8PZM4DmPs0vCeQi4Z0X5v4e/pjdKrfCaW0k4V7Fcq021sUmsXj7cX7dd5qMHCsdGwXywagoBkc2wMNdQr06cVNuxrCL2QecuH/TatP8svcUX26M56/DSXi6Q/82wQxqH8o1LYIwuemCDwmrzYpFW7DarNi0DaUUBf9TpfxEgTLeb+G2+f8NCt/Ov+6G28X78raT37bwfUopzG5muwxZiYqTcK8C59NzWHswkTUxiaw+kMi59BwCvNy5NepqRnRtLItIiXKZ3BSDIxswOLIB+06kMPevo/y0+yQrdp/D38ud/q1C6RsRQt+WwVxtx1VBRc1VoTF3pdQg4EPABHyutX6r2OOewGygC3AWGKW1ji9rm6465m61aQ4npbH96AW2H7vA9qMXiD6VgtZQx8dMn5YhDOnQgH6tQmQWhKgUi9XGhtgklu48yZoDZ0hKM8atI+r50blJHaIaBxHVJIgWIX4yzOdC7LZDVSllAg4A/wASgL+B0VrrfYXaTAA6aK3/qZS6EximtR5V1nZrcrhn51o5nZzNyeRMTqVkcTgpndgzacSeSeNwUjrZucZZ7/293IlqHETXplfRr1UIkQ0DXXI6o3A+m02z72QKaw8m8mfcOXYcu0BypgUAD5MbYcE+tAj1o0WIH2HBvjQI9KZBoBf1A73wMksnoyax5w7V7kCs1joub8PfAbcC+wq1uRV4Pe/6AuB/SimlHTwVR2uNTRu9ZZs2LsZ145fdpjVWrbHZKHhMa8i12ciy2MjKtZJd8NNq3GexkmWxkpadS3KmhQsZFuNnpoWUTAtJadkFPaR8SkGjOt60CPGjT8tgWtbzp3OTIJoF+7n09DVRfbi5Kdo3DKR9w0AmXGf8bRxOSmfHsQvEnE7l0Jk09p1IYcWeU9iK/VVe5etBsJ8HQd4eBPqYCfI2E+RjJtDbjI+HO94eJrzNJrzMJrzMbnibTXh7mPBwd8PdTWFyy/+pCv10w2QybrspRf6fgTEOb/zN5N8WjlGRcG8IHCt0OwHoUVobrXWuUioZqAsk2aPIwj5bG8fbK6KxaiOoHcnbbCKw0C96k6t86NQkiAaB3tQP9KJB3qVhkA/eHtL7EdWHUopmIX6XrFWTZbFy4kImJ5OzOJmcxankTE4kZ3E2LZvkTAvHzmWwJ69Tk2mpupk5BWHPxQ+A/Pvzd/ZefJxCO22rrES7eu2Wtozq1sShr1GRcC/pP1/xWK1IG5RS44BxeTfTlFIxFXj96iYYB3xoVXO17T3XtvcL8p6r1J1T4c4rf3rTijSqSLgnAI0L3W4EnCilTYJSyh0IBM4V35DWeiYwsyKFVVdKqS0VGe9yJbXtPde29wvynl1RRXah/w20VEqFK6U8MD5wlhRrswS4L+/6cGClo8fbhRBClK7cnnveGPpE4BeMqZD/p7Xeq5SaAmzRWi8BvgC+VkrFYvTYK/GNQwghRGVV6CAmrfVyYHmx+14tdD0LGGHf0qqtGj2sdIVq23uube8X5D27HKctHCaEEMJx5LA1IYRwQRLulaCUmqSU0kqpYGfX4khKqf8opaKVUruUUouUUkHOrslRlFKDlFIxSqlYpdQLzq7H0ZRSjZVSq5RS+5VSe5VSTzi7pqqilDIppbYrpZY5uxZHkHC/QkqpxhhLMhx1di1V4Degvda6A8ZSFC86uR6HyFtqYxpwE9AWGK2UauvcqhwuF3hGa90G6Ak8Wgvec74ngP3OLsJRJNyv3PvAc5RwsJar0Vr/qrXOP8XRZoxjHVxRwVIbWuscIH+pDZeltT6ptd6Wdz0VI+waOrcqx1NKNQJuBj53di2OIuF+BZRSQ4HjWuudzq7FCR4EfnZ2EQ5S0lIbLh90+ZRSYUAn4E/nVlIlPsDonNmcXYijyHrupVBK/Q7UL+Ghl4GXgIFVW5FjlfV+tdaL89q8jPE1fk5V1laFKrSMhitSSvkBC4EntdaXno3chSilhgBntNZblVLXObseR5FwL4XW+oaS7ldKRQLhwM68Fe0aAduUUt211qeqsES7Ku395lNK3QcMAQa48NHHFVlqw+UopcwYwT5Ha/2Ds+upAr2AoUqpwYAXEKCU+kZrPcbJddmVzHOvJKVUPNBVa+2yiy7lnazlPaCf1jrR2fU4St66SAeAAcBxjKU37tJa73VqYQ6kjB7KV8A5rfWTzq6nquX13CdprYc4uxZ7kzF3URH/A/yB35RSO5RSM5xdkCPk7TTOX2pjP/C9Kwd7nl7APcD1ef+2O/J6tKKGk567EEK4IOm5CyGEC5JwF0IIFyThLoQQLkjCXQghXJCEuxBCuCAJdyGEcEES7kII4YIk3IUQwgX9fy7vuGVlkZdtAAAAAElFTkSuQmCC\n", 522 | "text/plain": [ 523 | "
" 524 | ] 525 | }, 526 | "metadata": {}, 527 | "output_type": "display_data" 528 | } 529 | ], 530 | "source": [ 531 | "plot_fig()\n", 532 | "#plt.savefig('fig5.png')" 533 | ] 534 | } 535 | ], 536 | "metadata": { 537 | "kernelspec": { 538 | "display_name": "Python 3", 539 | "language": "python", 540 | "name": "python3" 541 | }, 542 | "language_info": { 543 | "codemirror_mode": { 544 | "name": "ipython", 545 | "version": 3 546 | }, 547 | "file_extension": ".py", 548 | "mimetype": "text/x-python", 549 | "name": "python", 550 | "nbconvert_exporter": "python", 551 | "pygments_lexer": "ipython3", 552 | "version": "3.6.5" 553 | } 554 | }, 555 | "nbformat": 4, 556 | "nbformat_minor": 1 557 | } 558 | --------------------------------------------------------------------------------