├── README.md ├── U01 L01 - What is Statistics.ipynb ├── U02 - A Few Convergence Simulations.ipynb ├── U02 L03 - Parametric Statistical Models.ipynb ├── U02 L04 - Parametric Estimation and Confidence Intervals.ipynb ├── U02 L05 - Delta Method and Confidence Intervals.ipynb ├── U02 L06&07 - Hypothesis Testing, Type I & II Errors, Levels and P-values.ipynb ├── U03 L08 - Distance Measures Between Distributions.ipynb ├── U03 L09 - Introduction to Maximum Likelihood Estimation.ipynb ├── U03 L10 - Consistency of MLE, Covariance Matrices, and Multivariate Stats.ipynb └── U03 L11 - Fisher Info, Asym Normality of MLE, Method of Moments.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # Notes for 18.6501x - Fundamentals of Statistics 2 | 3 | I'm going to be uploading all of my notes here in the form of jupyter notebooks. Don't worry if you don't have python or jupyter installed - github will render the pages for you! 4 | 5 | Hope this helps us all understand things a little better 6 | 7 | -Trace 8 | 9 | edit 8/24/2022: I apologize that my notes only cover half of the class, but I'm happy so many people have found them useful! 10 | -------------------------------------------------------------------------------- /U02 - A Few Convergence Simulations.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 40, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import numpy as np\n", 10 | "import matplotlib.pyplot as plt\n", 11 | "import seaborn as sns\n", 12 | "import random\n", 13 | "import warnings\n", 14 | "import pandas as pd\n", 15 | "warnings.filterwarnings('ignore')" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": 58, 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "# n is used for realized sample size and dist_n is the n of simulated RV's\n", 25 | "\n", 26 | "p = 0.65\n", 27 | "n = 100000\n", 28 | "dist_n = 200\n", 29 | "bsamp = np.array([1 if random.random() <= p else 0 for i in range(n)])\n", 30 | "brvs = np.array([[1 if random.random() <= p else 0 for i in range(n)] for j in range(dist_n)])" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": 73, 36 | "metadata": {}, 37 | "outputs": [ 38 | { 39 | "data": { 40 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEJCAYAAACE39xMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XecXGW9+PHPd7bXbN8km2w2ZbMkoaQsBEIgoaOCoqIC6gVRUa969arXa7uKXq/X68UO/BCpioBeRGmK0gJISdgUUkjbJNuy2d57mef3xzlDJpMtszsze6Z836/MKzNnTvnO2XO+88zzPOc5YoxBKaVU5HM5HYBSSqng0ISulFJRQhO6UkpFCU3oSikVJTShK6VUlNCErpRSUSLiE7qIbBKRW2dgOyUiYkSkPNTbGmf7XxGRqiCtK2SfRURuFpHdXq/vE5Eng70de92O/k18YkkVkUdEpNOOqcTpmFTsCeuELiL5InK7iFSJyKCINIrIcyJyidds7wO+7lSMEwllMhtne8br0Scih0XkQRFZ7zNrLTAH2OHneqtE5Ct+hnELsGEKYftlnC/uKX2OELsROB9YjxVTre8MIrLR52/UKiLPi8i5XvP8UkQOioiMsXy2iAyIyCfHCkBEbvBZf6OIPCEiK6b6Yez97VmPW0Qa7GNpzlTXNY1tnykid4vIfnsfHRCRe0TkzFBv297+10XkDRHpEpFmex+e6jOP2IWXehHpt/fXCp95skXkt/aXfKf9PCuUsYd1Qgf+CJwFfBxYClwB/BXI9cxgjGkzxnQ7E15Y+iRWQlmGtd+GgJdE5N88MxhjRo0xDcaYkWBtVERcIhJnjOkxxrQGa70TCcXnCMASYK8xZpcd0+gE867A+httBJqBp0SkwH7v1/a6zh9juQ8DI8DDE6y7z173XOBdQJq9/sQpfBaPe+11zcMqOC0H7pnGet42URz2MfRT4FmgC/giVuHg48BR4K8i8v1Atu+njcDtwDrgQqx9/qyI5HjN81Xgy8DngTOBJuAZEcnwmudBYDXwDuBy+/lvQxq5MSYsH0AWYICLJ5lvE3Cr1+sq4NvAfUA3VknpQ/b6HgZ6gIPApV7LbLS3lec1rcSeVj7O6zjgbuAI0G+v86uAy37/Znt+78dG+70iO5Z2+/EUUOrzub4KNNjx/sZeX9Uk+8IAV48x/QdYB+WScT5LAvALoB4YtPfZD7327wmfw55+gx3bO4Hd9vpPtePc7bXt+4AngW8BjfYy9wIp4/0NvZfzeu67L0t8P4c97/nAZmDA3t5PgUSfbd1u75MWrBPxFs/fbYJ9+z5gl9f++SYg4+yjTeOsYyMnH2en2dOu9Jr2BnD/GMtvB+6ZIMYbgB6faVfa6z9tKvtgnL/J54BGr9cTngM+f/9/B+qApgni/29gK1A0zvuFwDbgn30/M3AR1nHYC7wALAxiLkoHRj1/I0CAY8A3veZJwco3n7JfL7P3+7le86y3p5UFKzbfRziX0Hvsx7tFJHmKy34R2IL1jfgH4H6sb8u/ACuBl4AHprFeby6sUsMHsf543wS+AXzMfv8We9vPYpVy5gCvikgq1gE3gFX6OAfr4HjWfg8R+SDwfeA79mfYD3wpgFh/bMd71Tjv/wvwXuAaoBTrC3C//d77sE7E73l9Do9krET9KazSW/U4698AnIF10r0fuBT4nynE/wXgNY6XGMer0ijC+gW3HViFVbK7FitRePOUdNdhJakvYn3mMYnIGuD/gEexEvDXsKr5PmfP8j47ttfs2N7nz4ey/96e42XY6627gatFJNNr3tVYx+7d/qzbXiYLuG6M9cPU90E+1vGz2WvyZOeAxwbgdKxS6kXjrL8M6+/1bmPMURH5hIjsE5E6u2rjGaxf6dcC3xKRNK/Fk7D+HjdinU9ZwB1e6z5PRHomeXxjvM8OZNiftd1+vRCYDfzdM4Mxph8rr6yzJ52Dlb9e9VrPK1hfOOsIlVB9UwTpm/H9QBtW8nsNK0mu9ZlnEyeX0B/y+XY1wC+8ppVwYgl1I1MsoY8T7w+BZ31LJz7z3IhVkhGfkk4r8EH79avAr32We5ZpltDt9xqA28f5bL8AnvOOyWfZKuArPtNusNexxmf6zZxcQu8A0r2mfQSrpJs21t9wrH03zjy+n+O/gEpOLCHeYG8r1Ws9r/ms5xngrgn26++A58f4nHVer29lnJK51zye48xTWHHbr98AErzmy8Q68W/ymnYb8NYk67/Ba/29HP/F8NgY58yE+8CeZ8hnXbuBuZPEMNY50AwkTbLc94Bb7Ofn2tv8J6wv5vuwvnw22u+/DFzm85nLvNb1YTt2z6/lFKxqrIkeORPE9gesQkKc/Xqdvc1in/nuAf5mP/8GcHiMdR0Gvj7RvgjkEc4ldIwxf8SqC7wSq+S1Dnh9km9TgJ1e6+jBqlfc5fV+o/1/AQEQkU+LSIXdcNID/CtQPMlia7C+4bs9pQOgE8gGFtvzLMP6AvPm+3rK4WIdhGO5D6v0d0BEbhORd4mIP8fGCP41SO60/w4erwGJHP+8wbIMK1G5vab9w97WEu94fJarZ+JjYRlW6crbP4Ai71L0FFyA9cvrWqzqiuuNMW+XoI0xXcAjWF/+2L8kr8W/0nkf1t9yDdYvp4P2/7782Qe/t9d1BlZ1QTXwnIike2bw8xzYbYwZnCTuMzhemn0P8KAx5jfGmO3ATVgJ2uMY4F2fPWiM2e/1uh6rGjELrNKzMaZykkfbWEGJyE/sz/5+c3K7iO/55HuOjXW+TXQeBiysEzqAMWbAGPOMMeZ7xph1WAf1zZM08vj+vDQ+0zw71PP5PQnAu2dBwkRxiciHgJ9hJcPLsA7827GSx0RcWElwpc9jKfCrSZadFhHJA/KxSgcnMcZswyrtfsOO736sBp7Jjo/BMQ7y6XBz4r6HSfb/OCY6Wbynj3V8TPRZ/V2vv44YYw4YY34PfBd4VESSfOa5C1hr95x4H9YvTX8a1IydoPYZY+7E+nXx0Bjz+bMPOr0S3itYVSKnYFfNTOEc6PUj7gSsX+LYy7+9jDFmCOtXFvYxuRLrl5iHb6P4Cef3dKtc7Abaa4ELjTHe506D/f9sn0UKOF5YbAAKvHsr2c/zveYJurBP6GN4C4jHqr8Nlmb7f+/64ZWTLLMe2GyMudUYs80YU8nJJc4hrOoUb9uwSostE5QS9gJn+yzn+3oqvoyVNB8bbwZjTLcx5v+MMZ/B6h1xIcdLtWN9jqk4zafO82x7nYfs182cuO/BKrF58yeGt4BzfL6I1vtsazrestfjbT1WlUugPax+i5XMPus90RjzMlY7xo1YifRxY0zTNNb/U2C1iPhVrz8Jz5d3qv2/P+eAvyqx6tnBqou+RkRWiEiciHwRq7SdidUeVG+MeWMK667g5AKU7+MO7wVE5OdY7Q8XGmP2+azvCFbCvsRr/mTgPI7/yngN60v4HK/lzsHqdeRdrx5UYZvQRSRXrD66HxGR00VkoYh8AKsV/Tn7Z2mwVGI1st0sIktF5FKsxr6JHMA6Ud4hIqUi8h+c3P+6CjhVRMpEJE9EErBKTI3AYyKywf5c54vIj0Wk1F7u58D1IvJJe91fB9b6+VmyRGS2iBSLyAUich9WD4Ov2SfcSUTkSyJyrYgsE5ElWAdyF1ZjqOdznCciRXZpf6rigXvsE/QSrHrWXxtjPKWw54F3iMi77X31E2C+zzqqgLPEupgob5xfD7djVdHdbn+Wd9nbutUY0zeNuD1+DGywG+eWisiHsb4kfxTAOgGwq4d+BnzN50sPrDrZG7GqaPxuDPVZfxdWaf+7flajeUu1j6XZInIG1v4d4HhjoD/ngL8eBW4SkSRjzKNYVU47sUrm52Ml5YexqibHa9wf01SrXETkNqyG3WuBdq99kG6vz3D8b/Y+sfqo34fV3vCgPc9e4GngVyJytoicg/UL/Emf6qHgClXlfKAPrJbrH2A1GLVj1Q0eBH6CVwMGYzeK+jbg9QA3eL1OxvpZdoXXtHVYVSH9WN+u72LibouJWCdZO1aj391Y3SWrvNaZj3Xwd3Nit8VCrF4RTVgH7BGsk9e7Ufbr9vueg+Rm/GsU9TwG7PU+BJzvM5/vZ/kk1i+HbqxE/iKwzmv+s4E37XXax/PJXeTs6TczdrfFb3t9nvuxGynteRKwGv1a7Mf3OLlRdKn9d+nDv26Lgxzvtpjk9f4Jx4t3jJPsW0+3xSF8ui3a70+lUTTPZ3oaVuP/N3ymF3ptb8JulZP8TYqxqliu83cfcHJXzDZ72gVe8/hzDky6b73mfRyrwJPgtV/y7ecFeHU/negzj7efp5B7zDiPm73mEaxj/RjWefEicKrPenKAB7DOqS77edZ0YvL34elHq5RSjrIbmJ/EqtL5PlZvmR4Rycb6Qv0S1vUjRx0MM6xpQldKhQ0RiQc+DXwG69qGQay2k03AfxljNjkWXATQhK6UCkt2iT0LaDbWhTtqEprQlVIqSoRtLxellFJTEz+TG8vLyzMlJSUzuUmllIp4W7dubTHG5E8234wm9JKSEioqKmZyk0opFfFEZLyB706gVS5KKRUlJk3oYt0ppElOvK3Y/4o1tOVOEfmThPguHEoppSbnTwn9PqxxjL09g3VV1OlYl/+G5S3glFIqlkya0I0xL2Fd9us97e/m+G2/Xse6RZVSSikHBaMO/UasscrHJCI32eMlVzQ3N483m1JKqQAFlNBF5JtYYxH/brx5jDF3GmPKjTHl+fmT9rpRSik1TdPutigi1wNXABcZvdxUKaUcN62ELiKXY42xvcEENs60UkqpIPGn2+JDWONQl4l1B+6PY439nIF1m7IdInLHhCtRSikVcpOW0I0x144xeVp3T1EqXD24uWbay163drL7gp/ocHMPGckJ5Gf43kZUqcDM6KX/SsWqwZFRHtxcwyNb69hT30VinIury+fxmQ2LmZ+TOvkKlPKDJnSlQmx/QzdfeHg7+xq6Oa1oFt++YjmHmnv4v4o6/ri1jntvOJN1S6Zzq1alTqQJXakQcbsN971axQ+f3kdmcjx3X1/ORcsK337/sxcs4YZ7t/CJ31TwmxvPorwkx8FoVTTQwbmUCoHGrgGuv3cL33vyLc4vzePpL55/QjIHmJuVwgOfWEthZjIfu/cN9tR3OhStihaa0JUKolG34YHXq7n0py9RUdXOD957Gr/+p3Ly0sduAC3ISObBT64lLSmezz+0nf6h0RmOWEUTTehKBUltWx9X3fYK3/rzbpbPyeSpf1nPdWuLEZEJl5szK4VbPnAGh5t7+Z+n981QtCoaaR26UkHyg7/s5XBzD7+4dhVXnj5n0kTubX1pHh87t4R7X6niomUFnFeqw2SoqdMSulJBcKCxm7/ubuBj5y7k3WfMnVIy9/j3y09hSUE6X/vjLgZHtOpFTZ0mdKWC4LYXKklNjOPG9QunvY7khDhuvnIFRzv6A7rQScUuTehKBehISy9PvFnPR89eQE5aYkDrWl+ax7rFudz2QiW9gyOTL6CUF03oSgXo9hcqSYx38YnzFgVlfV+5rIyWniHue7UqKOtTsUMTulIB6Bsc4bEd9Xxgzfygjc2yujibi5cVcseLh+jsGw7KOlVs0ISuVAC21XYwNOrmw2dPbYCuyXz50qV0D4xwzytHgrpeFd00oSs1TcYY3qhqY1VxFqfMzgzqupfNyeSyFYXc88oRuga0lK78owldqWmqbu2juXuQa88Kbunc4/MXltI9MMJvtC5d+UkTulLT9EZVG0nxLq44fU5I1n9q0SwuPKWAu/5xhB7t8aL8oFeKKjUN/UOj7DrayZoF2fx5e/201uHPjTE+f+ES3nv7qzzwejWf3rB4WttRsUNL6EpNw4HGbkbchlXF2SHdzqribNYtzuW+V6oYHnWHdFsq8mlCV2oaKpt7SE5wMS87JeTb+vj6hTR0DfD07oaQb0tFNk3oSk2RMYZDTT0sykvHNY0xW6bqgrICSnJTtQujmpQmdKWmqK13iI7+YZYUpM/I9lwu4fp1JWyv6WBHbceMbFNFJm0UVWqKKpt7AFicH1hCn8oAXMZAUryL7zy2m8c+tz6g7aropSV0paboUFMPs1ISyEsPbCCuqUhOiKN8QTa7jnbS0DkwY9tVkUUTulJT4DaGQ829LM5Pm9aY54E4Z3EexsADr1fP6HZV5NCErtQUHOscoH94NODqlunISUvklNkZPLilhoFhvQGGOpkmdKWm4FCTXX8+Qw2ivtYtyaOtd4jHd0zvYiYV3SZN6CJyj4g0ichur2k5IvKMiBy0/w/t1RVKhYmDTd0UZCSRmZzgyPYX5aVxyuwM7nnlCMYYR2JQ4cufEvp9wOU+074GPGeMKQWes18rFdUGR0apau1jaWGGYzGICB87t4R9Dd28drjVsThUeJo0oRtjXgLafCa/B7jffn4/cFWQ41Iq7Bxp7mXUbRxN6ADvWVlEWmIcT+485mgcKvxMtw690BhzDMD+v2C8GUXkJhGpEJGK5ubmaW5OKeftb+wmIU4oyU11NI7khDg2lOXz3N5G3G6tdlHHhbxR1BhzpzGm3BhTnp+fH+rNKRUSxhgONHazOD+d+Djn+xJcvKyQxq5Bdtd3Oh2KCiPTPTIbRWQOgP1/U/BCUir8tPYM0d437Hh1i8cFZQW4BJ59q9HpUFQYmW5Cfxy43n5+PfBYcMJRKjwdaOoGCJuEnp2WSHlJDs/s1bKUOs6fbosPAa8BZSJSJyIfB34IXCIiB4FL7NdKRa0Djd3kpSeSkzZzl/tP5pJlhew91kVde5/Toagw4U8vl2uNMXOMMQnGmHnGmLuNMa3GmIuMMaX2/769YJSKGkMjbg4391IaJqVzj4uXFwLwnJbSlc351h2lwlxlUw8jbsOy2ZlOh3KChXlpLM5P49m9Wo+uLJrQlZrEW8c6SU5wsTAvzelQTnLx8kJeP9xK18Cw06GoMKAJXakJjLoNe491c8rsTOJcMzu6oj8uXlbI8KjhpQN6jYfShK7UhKpbe+kfHmX5nPCqbvFYXZxNdmqCdl9UgCZ0pSa051gX8S4Jm+6KvuJcwoWnFPL8viaGR91Oh6McpgldqXEYY9hb38WSgnQS48P3VLlkeQFdAyNUVLU7HYpyWPgepUo57FjnAB39w6yYG57VLR7nleaTGOfS3i5KE7pS49nb0IUAZWHWXdFXWlI865bk8uzeRh0jPcZpQldqHAcbe5iXnUJ6UrzToUzq4mWFVLf2cai5x+lQlIM0oSs1hr6hEWrb+sLu6tDxrF+SB8CWI1qPHss0oSs1hsqmHgzhMxjXZBbkppKTlsjWak3osUwTulJjONDYQ0pCHPOyU5wOxS8iwuribLbXaEKPZZrQlfJhjOFgYzdLCtJxSfhdHTqe1QuyONzSS1vvkNOhKIdoQlfKR0PXAN2DIxFT3eKxpjgbQEvpMUwTulI+DjRaPUVKC9MdjmRqTp+XRbxLtB49hmlCV8rHwcZu5sxKJjM5welQpiQlMY7lczPZpiX0mKUJXSkvo25DTVsfi8JwqFx/rC7O5s3aTkZ0XJeYpAldKS/1Hf2MuA3FuRGa0Bdk0z88yr6GbqdDUQ7QhK6Ul5o26/6cC3JSHY5kelYXZwFoPXqM0oSulJfq1l6yUxPITIms+nOPoqwUCjOTtB49RmlCV8pmjFV/XhyhpXM4foGRJvTYFP6jDik1Qzr6h+kaGAn7+vMHN9dM+L4AtW39/OrFQ2R49dS5bm1xiCNTTtMSulK26tbIrj/38PzC8LQHqNihCV0pW01bL4nxLgozk50OJSBzs1KIcwk1rZrQY40mdKVs1a19zM+2kmEki49zUZSVoiX0GKQJXSlgcHiUhs4BFoR5/bm/inNSOdrRz4hbLzCKJZrQlQJq2/sxENE9XLwV56Qy4jYc6xhwOhQ1gwJK6CLyryKyR0R2i8hDIhLZlY8qZtW1W9UT87OjJ6EDVGu1S0yZdkIXkSLgX4ByY8ypQBxwTbACU2om1bb1kZeeSEpinNOhBEVmSgJZqQlajx5jAq1yiQdSRCQeSAXqAw9JqZlljKG2vT9qSucexTmp1LT2Oh2GmkHTTujGmKPALUANcAzoNMb83Xc+EblJRCpEpKK5uXn6kSoVIkc7+ukZHGFelNSfexTnpNI1MEJHn97BKFYEUuWSDbwHWAjMBdJE5CO+8xlj7jTGlBtjyvPz86cfqVIhsqO2A4D5EXL/UH8tyLF67Gi1S+wIpMrlYuCIMabZGDMMPAqsC05YSs2cHTUdxLuE2bOiq01/9qxkEuJEE3oMCSSh1wBni0iqiAhwEbA3OGEpNXO213YwNyuFeFd09eKNcwnzslM1oceQQOrQNwOPANuAXfa67gxSXErNiOFRN7uPdkZddYtHcU4q9R39DOsdjGJCQKMtGmO+A3wnSLEoNeP2HetmcMTN/ChrEPUozknFbeBoe7/ToagZEF2/MZWaoh211rjh0dZl0WO+jrwYUzShq5i2vaaDvPREslIj8w5Fk0lPiic3LVETeozQhK5iWkV1O2sWZGO160en4pxUqtv6MMY4HYoKMU3oKmY1dQ9Q09ZH+YIcp0MJqeLcVHoHR6ht03r0aKcJXcWsrVVW/fmakmyHIwktz0BdW2vaHI5EhZomdBWzKqrbSYp3cercWU6HElKFmckkxbvYVt3hdCgqxDShq5hVUdXGGfOySIyP7tPAJcL87FS2Vrc7HYoKseg+kpUaR//QKHvqu6K+usWjODeVfQ1d9A6OOB2KCiFN6Com7ajtYMRtODNWErp9gdGbtVrtEs00oauYtLXaaiBcXRwbCd1z4dS2Gq12iWaa0FVMqqhup7QgnazURKdDmREpiXGUFqRrPXqU04SuYo7bbdhW3U55jFS3eKxZkM322g69wCiKaUJXMWdfQzddAyNRf0GRr9XF2XT0DXO4RW9LF600oauYs+VIKwBrF8VYQl+QBaDVLlFME7qKOVuq2ijKSmFelI6wOJ5FeenMSklguzaMRi1N6CqmGGPYcqSNtQtjq3QO4HIJq4uzqKjShB6tNKGrmHK4pZeWniHOisGEDlbD6MGmHjr7h50ORYWAJnQVUzYftvqfx2pCX73A6tmj/dGjkyZ0FVO2HGklLz2JhXlpTofiiJXzs4hzCdu0YTQqaUJXMcMYw+YjbaxdlBPVN7SYSGpiPMvnZGo9epTShK5iRl17P8c6B2KyQdTbmgXZ1lg2o26nQ1FBpgldxYzNR2K7/txjzYJs+odH2dfQ7XQoKsg0oauY8eqhFnLSEllakOF0KI5aYzeMVlTpHYyijSZ0FROMMbx2qJVzFuficsVm/bnH3KwU5sxKZmuNDqUbbTShq5hwpKWXY50DrFuc63QoYWHNgmy2agk96mhCVzHh1UPW+C3nLs5zOJLwsGZBNvWdA9R39DsdigqigBK6iGSJyCMisk9E9orIOcEKTKlgevVQC3NnJbMgN7bGbxnPmSVWw/AbWkqPKoGW0H8OPG2MOQU4A9gbeEhKBZfbbdWfr1uSF7P9z30tm5NJRnI8rx/WhB5N4qe7oIhkAucDNwAYY4aAoeCEpVTw7G3oor1vWOvPvcS5hLNKcth8uNXpUFQQBVJCXwQ0A/eKyHYRuUtETrqeWkRuEpEKEalobm4OYHNKTc9rdv35Oq0/P8HZi3I53NJLU9eA06GoIAkkoccDq4H/Z4xZBfQCX/OdyRhzpzGm3BhTnp+fH8DmlJqeVypbWJSfxuxZyU6HElY8N/h4/YhWu0SLQBJ6HVBnjNlsv34EK8ErFTaGR91sOdKmvVvGsHxOJhlJ8byu1S5RY9oJ3RjTANSKSJk96SLgraBEpVSQ7KzroHdoVOvPxxAf56K8JFvr0aNIoL1cPg/8TkR2AiuBHwQeklLB82plKyJWfbE62dpFuRxq7qWpW+vRo8G0e7kAGGN2AOVBikWpoHvlUAvL52SSnZbodChhyfNFt+VIG1ecPtfhaFSg9EpRFbUGhkfZVt3BuUu0/nw8p87NJC0x7u2eQCqyaUJXUauiqp2hUTfnaP35uOLjXJy9KJd/VLY4HYoKgoCqXJQKZ68eaiHevoBGwYOba8acnpoYR3VrH7987iC56UljznPd2uJQhqaCREvoKmq9cqiVlfOzSEvScstESu3x4Q829TgciQqUJnQVlboGhtlV18E6rT+fVG56IlmpCVRqQo94WnRRUcVTrbD7aCduA/1Do+NWNSiLiFBakM7Ouk5G3Ya4GL8BSCTTErqKSgcau0lOcFGco8Pl+qO0IIPBETd17X1Oh6ICoAldRR1jDPsbuyktyNDSpp8W56cjwIFGrXaJZJrQVdQ51jlA98AIZYWxfTPoqUhJjGNedgqVTd1Oh6ICoAldRZ0DjVZSKi1MdziSyFJamEFdez99QyNOh6KmSRO6ijr7G7opykohIznB6VAiSmlBOgY41NzrdChqmjShq6jSNzRCTVsfS7W6ZcrmZaeSFO/iYKNWu0QqTegqqhxs6sEAZbM1oU9VnEtYnJ9OZVMPxhinw1HToAldRZUDDd2k2g18aupKC9Pp6B+mpUdvDxyJNKGrqOF2W90VlxZm4BLtrjgdx4cB0GqXSKQJXUWNnUc76Rsa1frzAOSkJZKblshB7Y8ekTShq6jxwr4mBFhaoN0VA1FamM7hlh5GRt1Oh6KmSBO6ihqbDjQzPyeVVB1dMSClBRkMjxpq2nQYgEijCV1FhZaeQXbWdWh1SxAszEvDJTqcbiTShK6iwksHmjFGuysGQ3JCHMU5qdowGoE0oauo8ML+ZvIzkpgzK9npUKJCaWEG9R0D9AzqMACRRBO6injDo25eOtDMhqX52l0xSErthuVDWu0SUTShq4i35Ugbnf3DXLK80OlQosbcrBRSEuK02iXCaEJXEe9vexpITnBxfmm+06FEDZcISwrSraEUdBiAiKEJXUU0t9vwtz0NbFiaT0pinNPhRJXSgnS6B0Zo7B50OhTlJ03oKqK9WddBY9cgl5862+lQos4Sux69UkdfjBgBJ3QRiROR7SLyZDACUmoq/rankXiXcGGZ1p8HW1ZqIvkZSdofPYIEo4T+BWBvENaj1JQYY1W3nLM4l1mpejOLUFjEqV/lAAASO0lEQVRakM6Rll69i1GECCihi8g84F3AXcEJRyn/HWzq4UhLL5et0OqWUCmbncmI2/BqZavToSg/BFpC/xnwVWDcUXxE5CYRqRCRiubm5gA3p9Rxz7zVCKDdFUOoJC+VxHgXL+xvcjoU5YdpJ3QRuQJoMsZsnWg+Y8ydxphyY0x5fr52K1PB8+L+ZpbPyaQwU68ODZV4l4sl+els2t+s3RcjQCAl9HOBd4tIFfAwcKGIPBCUqJSaRNfAMFtr2tlYpoWEUCubncHRjn4O6BjpYW/aCd0Y83VjzDxjTAlwDfC8MeYjQYtMqQm8crCFUbdhY1mB06FEPc8IllrtEv60H7qKSJv2N5ORHM/q4iynQ4l6s1ISWD4nk+f3aUIPd0FJ6MaYTcaYK4KxLqUmY4zhxQPNnFeaR3yclklmwgWn5LO1up3O/mGnQ1ET0LNBRZx9Dd00dA2wYanWn8+Ui5YVMuo2PLe30elQ1AQ0oauI8+IBq/vrhqVafz5TVs3Poigrhad2HnM6FDUBvfmiijib9jdxyuwMZuvNLGbMQ1tqWZiXxqb9zdz98hG/B0K7bm1xiCNT3rSEriJKZ/8wb1S1c+EpWjqfaacVzWLUGN461uV0KGocmtBVRHnxQDOjbsNFyzShz7R52Slkpyaw62iH06GocWhCVxHl+b2N5KQlsnJ+ttOhxBwR4bSiWVQ29dCn9xoNS5rQVcQYGXXzwv5mNpblE+fSe4c64bSiLNwG9mi1S1jShK4ixraaDjr7h7l4mQ7G5ZS5WcnkpiXyZq1Wu4QjTegqYjy3t5GEOOG80jynQ4lZIsLK4iwOt/TS3jfkdDjKhyZ0FTGe29fE2oW5ZCTrzSyctMpuv9BSevjRhK4iQlVLL5VNPdpdMQzkpCVSkpvKtpoOHVI3zGhCVxHhqV3WFYqX6c2gw8Kq4mxaegY52tHvdCjKi14pqsLSg5trTnj929eqWZCTyov79a5X4eC0olk88WY922o6mJed6nQ4yqYldBX2GrsGaOga4PR5s5wORdmSE+JYPjeTnXUdjIyOewdKNcM0oauwt7OuEwFOLdKEHk7WFGfTNzSqQwGEEU3oKqwZY9hZ18Gi/DTt3RJmFhekk52awJaqNqdDUTZN6CqsHescoLV3iNPn6Z2Jwo1LhDNLcjjc3EtLz6DT4Sg0oaswt6O2A5fAirmZToeixrB6QTYugQotpYcFTegqbA2PutlW086yOZmkJmqHrHCUmZzAKbMz2VrdzohbG0edpgldha099Z30DY2ydmGu06GoCZy1MIfeoVH2Hut2OpSYpwldha3Nh9vITUtkUX6a06GoCSwpSCczOZ4dNe1OhxLzNKGrsNTQOUB1Wx9nLczBJTpUbjhziXDG/Cz2N3bTq+OkO0oTugpLm4+0Eu8S1hTrjSwiwcr51jjpu452Oh1KTNOErsJOZ/8wO2o7OLVoFqlJ2hgaCebMSmF2ZjI7dARGR2lCV2HngderGRxxs36JjnseSVbOz6KmrY+2Xh0n3Sma0FVY6R8a5e5/HGFpYTpzs1KcDkdNwenzZiHAjlptHHXKtBO6iMwXkRdEZK+I7BGRLwQzMBWbfv9GDW29Q2xYquOeR5qs1EQW5qWxraYDt46T7ohASugjwJeNMcuAs4HPisjy4ISlYtHQiJs7XzrMmSXZLMzTroqR6MyFObT1DnGoqcfpUGLStBO6MeaYMWab/bwb2AsUBSswFXv+tL2O+s4B/nnjEqdDUdO0Yk4maYlxbD6iQwE4ISh16CJSAqwCNgdjfSr2DI6M8ovnKjljfhYby/KdDkdNU3ycizULctjX0EVn/7DT4cScgBO6iKQDfwS+aIw5aWBkEblJRCpEpKK5We82o8b20OYajnb085VLlyJ6IVFEO2thDm4DFdVaSp9pASV0EUnASua/M8Y8OtY8xpg7jTHlxpjy/HwteamT9Q2NcOsLh1i7MEe7KkaBnLRESgvSqahq17sZzbBAerkIcDew1xjzk+CFpGLN/a9W09IzyL9dVqal8yhx9qJcOvuHefzNeqdDiSmBlNDPBT4KXCgiO+zHO4MUl4oRXQPD3PHiITaW5VNekuN0OCpIymZnMGdWMj979iDDWkqfMYH0cvmHMUaMMacbY1baj78EMzgV/e56+Qid/cN85dIyp0NRQeQS4ZLlhdS09fGHilqnw4kZeqWockxb7xB3v3yYd542W28AHYXKCjNYXZzFL5+rZGB41OlwYoImdOWYO148RP/wKF+6ZKnToagQEBG+clkZDV0D/Pa1aqfDiQma0JUjGjoHuP/VKq5aVcSSggynw1Ehsm5xHhvL8vn5cwdp7BpwOpyopwldOeJHf9uHMfCvF2vpPNrdfOUKhkbd/NdTe50OJeppQlczbmddB49uO8qN6xcyPyfV6XBUiJXkpfGZDYt5/M16XqlscTqcqKYJXc0oYwzff3IvuWmJfPaCxU6Ho2bIZzYuZkFuKv/x2G6GRrQbY6jo7WBUSD24ueaE17uOdrKlqo33rJzLE28ecygqNdOSE+K4+d0r+Ni9b/Drlw/z2Qt0ALZQ0BK6mjF9QyM8+WY9szOTKV+gFxHFmgvKCrh8xWx++fxB6tr7nA4nKmlCVzPmqZ3H6B0a4eo184hz6SX+sejbVy5HEL77xFtOhxKVNKGrGbHvWBfbazvYsLRAby0Xw+ZmpfCFi0t55q1Gnnmr0elwoo4mdBVyXQPD/Gn7UWZnJnPBKTriZqy78dyFnDI7g2/+aRedfTpmejBpQlchNTLq5nevVzM44uaD5fOJd+khF+sS413c8oEzaO0d4rtP7nE6nKiiZ5cKGWMMj79ZT217P1evmcfsWclOh6TCxKlFs/jsxsU8uu0oz2rVS9Bot0UVMre9UElFdTsXlOXr4Fsxyrfbqre8jCRmZybzhYe387kLS5mVkvD2e9etLZ6J8KKOltBVSNz6/EFu+fsBVs7P4qJlhU6Ho8JQvMvFNWfNZ9hteGhLDSNuveAoUJrQVdDd9kIlt/z9AO9dVcTVa+bh0rsQqXEUZCTz/tXzqGnr46+7GpwOJ+JpQldBddsLlfzv3/bz3lVF3PKBMzSZq0mdVjSLcxfn8trhVl4/3Op0OBFN69BV0HiS+VUr53LLB87Qi4eU3y4/dQ6tvUM88WY9aUmalqZLS+gqYMOjbr71511vJ/Mff3ClJnM1JXEu4Zozi5mfk8ofKmp5+WCz0yFFJE3oKiAtPYN85K7NPPB6DZ/asEiTuZq2xHgX/3TOAvLTk7jxvjd4bMdRp0OKOPrbRk2LMYbHdtTz3Sf20Dc0ys8+tJKrVhU5HZaKcKmJ8XzyvEU8s7eBLzy8g9q2Pv554xJcWkjwiyZ0NSnfvsTHOvt5encDB5t6mJ+dwkfPKaFvaHTCPsdK+SslMY77bzyLrz6yk1v+foAXDzTzo6vPYGFemtOhhT1N6Mpvzd2DPLevkV11nSQluHjXaXM4Z3Gu9mRRQZcUH8fPPrSS80vz+e4Te7j8Zy9x/boSPnHeQgoy9Irj8WhCV5Nq6Bpg0/4mdtV1Eh8nnL80n/NL80lJjHM6NBXFRIT3r5nH+tI8/vsve7nr5cPc92oVHyqfz6c2LGJett6+0JcYY2ZsY+Xl5aaiomLGtqcCs/toJ7c+X8nTexpIjHdx9sJc1pfmka7dylSIjXXpf1VLL3e8eIg/bqvDGLhqVRE3rCuJiWElRGSrMaZ80vk0oStvbb1DPL27gd9X1PJmbQcZSfGUl2Rz7uI8UjWRqzDQ0TfEy5UtVFS1MTxqmJedQvmCHE4rmsXHz1vodHghoQldTWpwZJRDTb3sb+xi37FuXjvcyq6jnRgDSwvT+WD5fD5QPp+nduq9P1X46R8aZXttO1uOtNHUPUicS7igLJ+NZQVsWJrP/JzoqZKZkYQuIpcDPwfigLuMMT+caH5N6DPLGENX/wh1HX3Utffz+I562vuGaO8bpqV7kNbeQdz2nz9OhKLsFJYWplM2O5O5s5IRbexUEcAYQ33HADtq26lq7eNoRz8ARVkprCrO4ox5WSydnUFpQToFGUnEx0Xe5Tf+JvRp/4YWkTjgNuASoA54Q0QeN8ZE7c0Cvb/8PE/NeO/7zGdNO3n5E9d//PmI283QiJuhUTeDw9b/QyNuBkc8/4++/f7QiJuu/mEauwdp6hqkqXuApq5B6jv66R4cOWEbiXEuslITyMtI4tSiTAozkynMTCYvPUkvCFIRSezCSFF2CteeNZ9DzT28dKCFrTXtbKtu50mvX5gusQYEK5yVzOzMJPLSk0hPiic1MZ60pDjSkuJJSYgjMd5FYpzL+t/ziHOR5PPa+3k4FIACqRQ9C6g0xhwGEJGHgfcAQU/o//nkWzy0xerjfDyRTpxcGSORnphcvZ7bb5w4LbCYneASyEhOICM5nozkBFYUzSI7NYHs1ET7kUBKYlxYHHhKhcJDW2oBSE6I49zFeZy7OI++wREaugdo7h6kq3+Yzv4RugaG2V7TT8/gCEMjbkbcgZ/wCXGCINj/ABABQRCBX310DeeVhvYWjIEk9CKg1ut1HbDWdyYRuQm4yX7ZIyL7A9hmHtASwPIzLdLiBY15JkRavBB5MYddvOf/56SzTBTzAn+2EUhCH6uYd9LXnDHmTuDOALZzfIMiFf7UI4WLSIsXNOaZEGnxQuTFHGnxQnBiDqR1oA6Y7/V6HlAfSDBKKaWmL5CE/gZQKiILRSQRuAZ4PDhhKaWUmqppV7kYY0ZE5HPA37C6Ld5jjNkTtMjGFpSqmxkUafGCxjwTIi1eiLyYIy1eCELMM3phkVJKqdCJvB72SimlxqQJXSmlokRYJHQRuVxE9otIpYh8bYL5rhYRIyLl9usSEekXkR32445wiVlEbhCRZq/YPuH13vUictB+XB8B8Y56TZ+xhm9/jgsR+aCIvCUie0TkQa/pM76PgxDzjO9nP46Ln3rFdEBEOrzeC8t9PEnM4biPi0XkBRHZLiI7ReSdXu993V5uv4hcNunGjDGOPrAaVA8Bi4BE4E1g+RjzZQAvAa8D5fa0EmB3OMYM3ADcOsayOcBh+/9s+3l2uMZrv9cTpvu4FNju2X9AgVP7ONCYndjP/p57XvN/HqvzQ1jv4/FiDtd9jNUY+hn7+XKgyuv5m0ASsNBeT9xE2wuHEvrbQwgYY4YAzxACvv4T+BEwMJPBjcPfmMdyGfCMMabNGNMOPANcHqI4PQKJ1yn+xPxJ4DZ7P2KMabKnO7GPA43ZCVM9Lq4FHrKfh/M+9uYdsxP8idcAmfbzWRy/nuc9wMPGmEFjzBGg0l7fuMIhoY81hMAJdxsWkVXAfGPMk2Msv9D+qfKiiJwXwji9TRqz7f32T6hHRMRzEZa/ywZTIPECJItIhYi8LiJXhTTS4/yJeSmwVEResWO7fArLhkIgMcPM72e/95OILMAqJT4/1WWDLJCYITz38c3AR0SkDvgL1q8Kf5c9QTjcsWDCIQRExAX8FKtKwNcxoNgY0yoia4A/i8gKY0xXSCI9zp9hD54AHjLGDIrIp4H7gQv9XDbYAokXrH1cLyKLgOdFZJcx5lAI4wX/Yo7HqsLYiHWl8ssicqqfy4bCtGM2xnQw8/t5KvvpGuARY8zoNJYNpkBihvDcx9cC9xljfiwi5wC/ne5xHA4l9MmGEMgATgU2iUgVcDbwuIiU2z9FWgGMMVux6piWhkHMGGNajTGD9stfA2v8XTYEAokXY0y9/f9hYBOwKpTB2vzZT3XAY8aYYfsn6X6sZOnUsBSBxOzEfp7KfrqGE6suwnkfe/jGHK77+OPAH+y4XgOSsQbqmvo+nskGgnEaDeKxGlQWcrzRYMUE82/ieKNoPnYjAVajw1EgJxxiBuZ4PX8v8Lr9PAc4gtWQlG0/D2nMAcabDSTZz/OAg0zQCDXDMV8O3O8VWy2Q68Q+DkLMM76f/T33gDKgCvtCRKeO4yDEHJb7GPgrcIP9fBlW0hZgBSc2ih5mkkbRkO78KXzodwIHsErY37SnfQ949xjzbuJ4Qn8/sMf+0NuAK8MlZuC/vWJ7ATjFa9kbsRo4KoGPhXO8wDpglz19F/DxMNrHAvwEawz+XcA1Tu7jQGJ2aj/7c+5h1fH+cIxlw3IfjxdzuO5jrN4sr9hx7QAu9Vr2m/Zy+4F3TLYtvfRfKaWiRDjUoSullAoCTehKKRUlNKErpVSU0ISulFJRQhO6UkpFCU3oSikVJTShK6VUlPj/WH6ZbhgVyaAAAAAASUVORK5CYII=\n", 41 | "text/plain": [ 42 | "
" 43 | ] 44 | }, 45 | "metadata": { 46 | "needs_background": "light" 47 | }, 48 | "output_type": "display_data" 49 | }, 50 | { 51 | "name": "stdout", 52 | "output_type": "stream", 53 | "text": [ 54 | "-------------------------------------------------\n", 55 | "This is a rough estimate/simulation of RnBar, the actual random variable\n", 56 | "and mean estimator of a Bernoulli(p=0.65) at n=200 samples (before data)\n", 57 | "-------------------------------------------------\n", 58 | "RnBar is roughly distributed N(p, p(1-p)/n)\n", 59 | "(p * (1-p) / n ~= 0.0011)\n", 60 | "-------------------------------------------------\n", 61 | " R RnBar\n", 62 | "mean 0.6500 0.649950\n", 63 | "variance 0.2275 0.001141\n", 64 | "\n", 65 | "-----------------------------------\n", 66 | "RnBar converges in probability to p\n", 67 | "-----------------------------------\n" 68 | ] 69 | } 70 | ], 71 | "source": [ 72 | "# Convergence of RnBar, the sample mean of a Bernoulli(p = 0.65)\n", 73 | "\n", 74 | "rnbar_actual = np.mean(bsamp)\n", 75 | "rnbar_dist = np.mean(brvs, axis=0)\n", 76 | "\n", 77 | "plt.title('Simulated Distribution of RV RnBar @n=200', fontsize=14)\n", 78 | "sns.distplot(rnbar_dist, bins=20)\n", 79 | "plt.show()\n", 80 | "\n", 81 | "print('-------------------------------------------------')\n", 82 | "print('RnBar converges in distribution to N(0, p(1-p))')\n", 83 | "print(f'(p * (1-p) / n ~= {round(p * (1 - p) / 200, 4)})')\n", 84 | "print('-------------------------------------------------')\n", 85 | "table = pd.DataFrame({'R': [p, p * (1 - p)], 'RnBar': [np.mean(rnbar_dist), np.var(rnbar_dist)]})\n", 86 | "table.rename(index={0:'mean', 1:'variance'}, inplace=True)\n", 87 | "print(table)\n", 88 | "\n", 89 | "print('\\n-----------------------------------')\n", 90 | "print('RnBar converges in probability to p')\n", 91 | "print('-----------------------------------')\n", 92 | "print(f'estimated p: {rnbar_actual}')" 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": 74, 98 | "metadata": {}, 99 | "outputs": [ 100 | { 101 | "data": { 102 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEJCAYAAAAw+egGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8XXWZ+PHPk73ZmqRN0iVJ0yXdgEJLKTuUTUAQXFCpoDKiqCP6c9SZgdFBhtHR0XFFmAEBUUeouFKlA7LvS0sphbakTdu0TdssTZp9T57fH+fccnp7szYn5+be5/163Vdyzv3ec5+z3POc7/l+zzmiqhhjjDHRJCHoAIwxxphwlpyMMcZEHUtOxhhjoo4lJ2OMMVHHkpMxxpioY8nJGGNM1BmT5CQiz4jIz8ZiWkN8T6mIqIgs9/u7Bvj+r4lI5RhNy7d5EZFbReRtz/D9IvLXsf4ed9qBrpOwWNJF5Pci0uTGVBp0TONBRM4VkW0ikjiCz/yXiPzUp3huFZH7/Jj2RCUil4vIRhGxCsEwDbmgRCRfRO4UkUoR6RKRGhF5UkQu8hT7IHCzf2GOnp875gG+Tz2vdhHZKSIPiMhZYUX3AtOBjcOcbqWIfG2YYfwXcO4Iwh6WAQ5CRjQfPvsUcA5wFk5Me8MLiMjKsHVULyJPiciZnjK3i8h2EZEIn88VkU4R+YyfMxLhe4844AjzfeDbqto3gkn+J3CdiMwZ5vc/M8xyBcBXgG95xt3vWd69IrJHRP5bRHJHEG9oWt511+P+Lr4rIkkjndZ4UtW/An3ANcP9jIhcKCIPuvuQehHZ7G6bC/2L9PB3J4vIf4rIJhFpE5ED7n6sJKxcqhvTQbfcGhEpCitTIiJ/cd8/KCI/FZGUoWIYThb/A7ACuB6YD1wO/B8wJVRAVRtUtWUY04oXn8HZOS7CWW7dwHMi8o+hAqrap6rVqto7Vl8qIgkikqiqrapaP1bTHYwf83EM5gFbVfUtN6bBdtbH4ayjlUAd8Ii7YwX4uTutcyJ87hqgF1g9ZlEPQUSSB3nvDGAh8LuRTFNV64C/AZ8fZNpni8h5YePOF5GzB5n0p4HXVHVn2PgncJZ3qVvmfcCdI4nZI/T7mgN8GWcevjrKaQEwnJ3lGPgF8KVhxDJJRB4EfgNsBz6Lsy1+CWfbe0VEBlxvYyQdWAZ82/17JVAMPBp2IPBj4EPAKuBsIBv4a6gW7/59BMhy318FXAX8YMgIVHXAF5ADKHDhEOWeAX7mGa4EbgHuB1pwjmA/6k5vNdCKs9Df4/nMSve7pnrGlbrjlg8wnAjcC+wCOtxp/hOQ4L5/q1ve+1rpvjfTjeWQ+3oEKAubr38Cqt14f+VOr3KIZaHAVRHG/wfOhjVvgHlJBn4K7Ae63GX2Xc/yPWI+3PHXubG9F3jbnf7xbpxve777fuCvwDeAGvczvwAmDbQOvZ/z/B++LEvD58Mtew7wKtDpft+PgJSw77rTXSYHgVqc2l7CEMv2g8BbnuXzdUAGWEbPDDCNlRy9nZ3gjnufZ9w64JcRPv8GcN8gMRYDDwMNQDvwDnC15/1TgNfdZfMGcBlHbpeh+N4LvIZzYHNjhGV/nVv+Z8Afw2K41d0ergZ24PwG/+ydZ7fcJ4CqQealBPg9cIcb8x3ucMkgn3kb+NJA25Fn3A+A+gi/nRtwEm0bsBO4dqjflxvTbz3Dc911UO1OZwNwedhnKt3ldB/QCPxusG1viO0ytM4uxzmD0Okur5MjLE/F3QcMMr0HgceAyQO8Px9nn3fZSNf5sbyAxW78J7jDk93t85qw7b8fuNgdvtQdLvaUudZdRtmDft8QwSS5M/lTIG2Qcs9wdHJqAP4eKHM3xE5grfuDmIeTVGpD02V0ySkZuA3nB18KfMTd0K53388Efgs8DkxzXyk4RwXbcH40S3COPO8BdgPp7mc/4i74z7obw9eBZkafnKa4K+lrA8zLV3F2uOe4G/EZwN+57+W57/1baD7c8dfhJKSXgDPdOLOInJxacH70xwMXA/uAnw60DsN3Kjgb4ks4P+bQskyMMB8zcXYI/4NTc7wcZyfxg7DvanLX3Xx3WfcCqwZZrifjnBb5N/cz1+Ak2S96ltF9bozTgLwhdiRT3eF04IfuuEs85T7nzke2Z9wyt9yZg8T5F5zt7URgNnBJaLpABs42710PW4mcnN4C3oNTOyjGSd7veJb9JLf8m8C/hMVwq7ts/oSzfZ+Os23fFVZuoftdc4fYpr/jlvvOEOXycLbxMwbajtzhOcBmoDrCb6cKZ+c1z/3ebmDWQL8vnB3mfuArnnEnuuvvBHc6X3enszBsH9WMcwA6j7AD05G8POvsHXedHu+u42rc/YmnbDXu/mmAaV2Ak1wyAXFj3+m+Pg+U4+wfzgc2j2Sd8+5vZrDXNYPEdpo7n0Xu8PnucH5Yuc3Av7n/3+aN0x2X737uvEGX6zAW/IdwEk0n8DLOj+TUsDLPcHRyetAznOkG490ZlnLkTi20goednAaI97vAEwP9MNxxn8KpZYlnXCJQD3zEHX4J+HnY555glMnJs2HeOcC8/RR40htT2GcrcRObZ9x17jTCj9Bu5ejk1AhkesZdi1MDyYi0DiMtuwHKhM/Ht4EKPLUgN84u3k38zwAvh03nceCeQZbrb4CnIsxnlWf4ZwxQY/KUCW1noR9jvzu8Dkj2lMvGSU43eMbdAWwZYvqbgG8O8N4NA6yHSMnpQ4OtU8/4RtyDmLCynXiOvHF2chVh5bLd77pggHiLcA7u/hunJvDf7nDRAOVPcqc3O8J21Osu7w7erf39Q4Tfznc8w0k4tc9rw8p0uNPqdId/ByQOsV5eAb4R9nv6y2CfGe7Ls868NYhMd918OqzsBuDfB5nWr4Ab3f+vwTmYeT+wHKc5pRcodd/fCywY7jrHOXCdN8Qra4C4UoAXgTWecR9z45Gwsk/hJkXgbo7+3QpDHIyq6tBtTqr6B2AGzjni/8M5on9FRP5liI9u8kyjFWcje8vzfo37t4BjICKfE5H1IlInIq3AP+AcWQzmZJyj2hYRaXU/1wTk4pwSAOeo/+Wwz4UPjzhcnI04kvtxftzbROQOEblsmD17ehleZ4RN7noIeRlng5s7QPnRWoSTePo9415wv2ueN56wz+1n8G1hEc6Pw+sFYKaIZI8izvNwakKrcE6RfFJVe0JvqmozzumiTwGISJpb9t4hpvsT4Bsi8rKIfEtETg6bh0jrIZL1w5yPSTg7pXC7VbXJMxxp+XZ4phHJHJydzOeBFvfvXTi/nYFiYYB4nsPZvlcAt+OcRYnUW9C73+jFaQ8Mj/sf3WmdiFMzXwL8MvSmiGSIyPdEZIuIHHJ/38s5er8w6DJ229xaPa+hOjMcXpfuOn4Lp2bn1cHAyxt3nl5y/78S+Imq/llV1+O0tXl7ZB7Aqa2GDLrOVbVFVSuGeB3Vd8BtY/pfnGaZvxsk9sMf4cj93ED7vIHGA86RyZBUtRPnyPZx4DYRuQe4VUT+S1W7B/hYT9iwho0LBRbaAYd2Zt4eUgM2BAOIyEdxGuS+hrNCm4EvAB8Y7HPud27EOT8brmGIz46KiEzFqc6GNxQDoKobxOn6fAlOdfmXwJsiclHYjj5cl46sl9ZA+jly2cMQy38AgyVg7/hI28dgyXi40x2uXap6EOdgIA34o4icqKpdnjL34HRkOQ5np5EJ/HqwiarqvSLyGE6b0YXASyLyHVW9laOX72DahlnuIM5BVbjhLN/Qjq0u0oRV9bkI454aIhbceA6EvdeuqhXu/18SkaeBf8U54vcaTtzVnmmVi0gW8KCIfFNVd+Cc3bkEZ7+wHefA+Fc4B0heQy3j9ThJMKRmoIIjkMcAy9uVzLvJPYUjYzx8UCMiGThNJjs87w+67NzketcQ8X1WVX/j+UwSThvYCTi1e29Hq2qcZDmVI+epAOdgJFTmTI401f3coMtztH3ut+AktrRRfj6S0MxN94w7KVJBj7OAV1X1Z6q6wd1gw2sC3Rx5tAFO1XoecDDCkUMoOW3FOcfqFT48El/FSQAPD1TAPbL5nXuEehlOkgrVNiLNx0ic4G7QIae50wxt3HUcuezB2SF7DSeGLcDpYbW+s8K+azS2uNPxOgvntN6x9hT9Nc5O4Qvekar6PM45/k/h9Lpco6q1Q01MVatU9W5V/QhOx6Ab3Le2EHk9DMdAy/4Njj46H67jcXZobw1VUFVXDmN6O3AOEIcTz78B/ywiM4ZRdiihg7N09+9ZwK9U9Q+qugmnHWvEZwhUtWOoWkWYw+vSXcfH4+xHQuPS3Dg2DDKNCpyaIDg7+M+4XbFTcJI5OAe59wEPD2d79FiDs08d7LXGE28yzmncJTjtQ9Vh03sdZ/u5yPOZIpwzBKHa38vAorDu5RfhnOZ/fbBgB01OIjLFvQbkWhFZIiKzReTDOI2IT7qnPsZKBc451FtFZL6IvAend9lgtgHLRORSESkTkX/l6Ot7KoHjRWSBiEx1F/hvcLL2w+JcwDhbRM4RkR+ISJn7uZ8AnxSRz7jTvhk4dZjzkiMi09yN6jwRuR/4Z+AmzxHfEUTkKyKySkQWicg8nPO5zTg/rNB8nC0iM91a2EglAfeJyHHiXKP2XZw2tdCR2VPApSJyhbusfojTEO9VCawQ58LbqQOcdrwT5zTwne68XOZ+189UtX0UcYf8ADhXnOt95rtHgV8FvncM0wTArZn+GLgpLHGAsxP4FM5pwKFO6SEiPxGRS0RkjoichHMEv8V9+wGc07De9fD1YYZZCcwSkWXusk91xz/G0Ul7uM4Gnj/G9XKYuxyfGE48qvoMTsP5UL/xSEK/rxkici7OAcA23k0E24APuMvqBJxTUmN5ID2Qb4jIRW5N+z6cA4oHPO+fhrNTDj897fVHnN6Z4LZx4qz7VpzTgftxlvE+nE4fwzaS03pujel3bsyrAHWX+TQRmeROrwnnN/F9ca7JWopzoLfJjRGcyxU2A78SkaUiciHOdXk/HzJ/DNYgBaTidPddh9Pduh2nmvxDPL2hiNwhIrzxvhW3+6s7nIbb/dIz7gyc020dOBk31M12oN56Ke7COYTT+HgvzoZa6ZlmvruAWjiy4bkQpzt1Lc4Gswtng/J2yLjZfb8VZyO7leF1iAi9Ot3pPgicE1YufF4+g3NE1YKTlJ7F0+sJZyN5052muuOuA1ojxHArkbuS3+KZn1/i6UmEU3O4A+fUzEGcXjb3c2SHiPnuemlneF3Ju3i3K3nqQNuLN8Yhlm2oK3k3YV3J3fdH0iEivFt1Bs4p3fCeb4We7xu0q7tb/nac30gnTm10NTDT8/6p7nructfn+4jcISI8vlScNrBDHNmVPNddH8cNtP4H2lZwaoVXDzVPI3nh9FY7gKeDwkDrFucArAu3Nx6Ru4lX4tmXcOTvqx9nZ70amOMpMwtn59iGc3D3NZzt//7B9lHHMM+hdXYFzo65y13Hp4SVuwv4nyGmleR+9nuhbRunp2yO+/90InT+GO46H8E8lYYta+/rOk+5NHebr3e3w7/g6Tbulilxl3+7W+52PPuDgV6hmTfGBMCtBdfhnDZ5ZpTT+C5Od97rR/CZy3COYJfoGF9ALSIv4/RKHbR9LlaIyErgaZx1cHCAMvk4Xc2Xq+quIaZXBDyK017znzi1204RmYbTg+/jOJczDLddckKy+zwZM/H9B7BTRnBvPZya4t+NdWJyfRbbt4SbDfz9UIkJnDZLnBr2Mzi1rQ4RCZ3dORNnvcV0YgKs5mRMkMai5mSCNZya0zFOPxenp2iNDtw7OuZYcjLGGBN1rOptjDEm6kT1beYjmTp1qpaWlgYdhjHGTCivv/76QVXNDzqO4Zpwyam0tJT164d7ZxdjjDEAIrI76BhGwk7rGWOMiTqWnIwxxkQdS07GGGOijiUnY4wxUceSkzHGmKhjyckYY0zUseRkjDEm6lhyMsYYE3Um3EW4xsSztq5eHtl0gN+/XsXb+5v40LIiPrdyLjNzJgUdmjFjypKTMRPEixUH+cpDG6lp7mLO1AzOW1jA6nV7WL1uDzdduojrz5oddIjGjBlLTsZEua7ePv7rsXJ+/vwu5uRn8NsblrJidh4iwr7GDm5ds5l//+sWUhKFj59eGnS4xowJX9ucROQSESkXkQoRuSnC+yUi8rSIvCEim0TkvX7GY8xEs72mhfff8RI/f34X155WwiNfPJtT50xBRACYmTOJO69ZxoWLCvjXhzfz+9erAo7YmLHhW3Jyn8p5B3ApsBhYJSKLw4p9A3hIVZcCVwN3+hWPMRNJT18/9zy/k8tvf4Ga5k7u+cRyvvX+E5iUcvTDbpMTE/jZx5Zx5rwp3PzHTWze3xRAxMaMLT9rTiuAClXd6T69cTVwZVgZBbLd/ycD+32Mx5gJ4fXdDVz+0xf41iNbOXPeVB798tlcuLhw0M+kJSfys1XLyElP4Su/fZOu3r5xitYYf/iZnGYCez3DVe44r1uBa0WkClgLfNHHeIyJenUtXVxzz6u0dvVy18dP5t5PLqcgK21Yn83NSOF7H1pCeU0LP3x8m8+RGuMvP5OTRBgX/kz4VcD9qloEvBf4tYgcFZOI3CAi60VkfV1dnQ+hGhMd7nlhJ929/fz6+hVcfNy0w21Lw3XewgJWrSjh7ud2smHPIZ+iNMZ/fianKqDYM1zE0aftrgceAlDVl4E0YGr4hFT1blVdrqrL8/MnzIMcjRmRQ23d/O/Lu7l8yQzm5GeOejpfv2wRUzJS+d6j76AafjxozMTgZ3JaB5SJyGwRScHp8LAmrMwe4AIAEVmEk5ysamTi0i9eqqStu48vnDfvmKaTmZrEF86byys7G3ixon6MojNmfPmWnFS1F7gReAzYitMrb7OI3CYiV7jFvgp8RkTeBB4ErlM71DNxqKWzh/tf3MXFxxWyYFrWMU/vY6eWMGNyGt//W7nVnsyE5OtFuKq6Fqejg3fcLZ7/twBn+hmDMRPBQ+uraO7sPeZaU0hqUiJfuqCMm/74Fk9sreWiIXr7GRNt7MavxgRMVVn92h5OLM5hSVHOmE33QycXUTolnR89vs1qT2bCseRkTMA27DnE9tpWVp1SPHThEUhOTODG88vYcqCZp8trx3TaxvjNkpMxAXvwtb1kpCTyvhNnjPm0rzxpBkW5k/jpkxVWezITiiUnYwLU3NnDI5sOcMVJM8hIHfsm4OTEBD6/ci4b9zZazz0zodhdyY0J0JqN++no6SM3PYUHXt0z4s9/7NSSIctcdXIRP31yO7c/tZ2zyo66jNCYqGQ1J2MC9Oc39rFwWpavDwtMTUrks+fM5dVdDbxhd40wE4QlJ2MC0trVy8a9jVywqGDEtykaqY+cUkxWahL3v1Tp6/cYM1YsORkTkNd21dPbr5w51/9TbZmpSXx4eTGPbDpATXOn799nzLGy5GRMQF6sqCc1KYFls3LH5fs+ecYs+lT5zSu7x+X7jDkW1iHCmIC8WHGQ5aW5pCUf/QDB4RppJ4oFhVnc+8Iupmam8okzSkf9vcb4zWpOxgTgYGsX71S3cMY4nNLzOmPuVNq6+9i0z56Wa6KbJSdjAvDyDueaozPnjW9ympufQUFWKi/tOGgX5ZqoZsnJmAC8tOMgWWlJnDBz8rh+r4hw+twp7G/s5PXd1q3cRC9LTsYE4MWKek6bM4XEBH+7kEeytDiXtOQEfmHdyk0Us+RkzDjb29DOnoZ2zpw7JZDvT0lK4JRZeTz6djX7GzsCicGYoVhyMmacPbvNedjz2fPzA4vhtDlTUFX+17qVmyhlycmYcfbstjqKcicxZ2pGYDHkZqRw4aJCfrtuL3391jHCRB9fk5OIXCIi5SJSISI3RXj/RyKy0X1tE5FGP+MxJmjdvf28VHGQc+fn+37LoqG878QZ1Ld12/32TFTy7SJcEUkE7gAuAqqAdSKyxn00OwCq+g+e8l8ElvoVjzHR4PXdh2jr7uPcAE/phZy7IJ+kBOGJrbUsL80LOhxjjuBnzWkFUKGqO1W1G1gNXDlI+VXAgz7GY0zgnt1WR1KC0507aNlpyZw6J48nt9YEHYoxR/EzOc0E9nqGq9xxRxGRWcBs4KkB3r9BRNaLyPq6uroxD9SY8fLstjpOnpVLVlpy0KEAcMHCQrbXtrK7vi3oUIw5gp/JKdIJ9YFaXq8Gfq+qfZHeVNW7VXW5qi7Pzw/+dIgxo1Hb3MnWA82cuyB6tuELFxUC8MTW2oAjMeZIfianKqDYM1wE7B+g7NXYKT0T457bfhAgKtqbQkqmpFNWkGmn9kzU8TM5rQPKRGS2iKTgJKA14YVEZAGQC7zsYyzGBO6pd2rIz0pl0bTsoEM5wgWLCnltVwPNnT1Bh2LMYb4lJ1XtBW4EHgO2Ag+p6mYRuU1ErvAUXQWsVrsLpYlhnT19PFNex0WLC0kI4JZFg7locQG9/cqz5daea6KHr89zUtW1wNqwcbeEDd/qZwzGRIOXd9TT3t3HRYsLgw7lKCcV55KXkcITW2t434kzgg7HGMDuEGHMuPjblmoyUhI5Iwq6kIdLTBDOW1DAM+V19Pb1Bx2OMYAlJ2N819+vPL6llpULC0hNGv1Tb/104aICmjp6WG+P0TBRwpKTMT57Y28jB1u7eE8UntILOXt+PimJCdZrz0QNS07G+OzxLTUkJQgrFxQEHcqAMlOTOG3uFLveyUQNS07G+OxvW6o5fe4UJk+KjrtCDOTCRQXsOtjGjrrWoEMxxt/eesbEmwde3XPEcENbNzvr2lg8Pfuo96LN+QsLuOXhzTy5tYa5+ZlBh2PinNWcjPHRtpoWAOYXZAUcydCKctNZOC3r8MMQjQmSJSdjfLS9tpXc9GSmZKYEHcqwnFKax8Y9jfYAQhM4S07G+KS3v58dda2UFWYF/mDB4Vo2K4e27r7DNT5jgmLJyRif7Glop7u3n/kFE6f9ZllJLgAb7Om4JmCWnIzxyfaaVhIE5kygzgUleelMyUhhw+7GoEMxcc6SkzE+2VbTQkleBmnJ0XlXiEhEhKUlubxhNScTMEtOxvigpbOHA02dzC+cOLWmkGWzcth5sI1Dbd1Bh2LimCUnY3wQupC1bAJ0IQ8Xand6Y6/VnkxwLDkZ44Pd9e2kJCUwPSct6FBGbEnRZBITxNqdTKAsORnjgz0N7RTnTiJhgnQh90pPSWLR9CzrsWcC5WtyEpFLRKRcRCpE5KYBynxERLaIyGYRecDPeIwZD109fVQ3dVKSlxF0KKO2rCSXN/faxbgmOL4lJxFJBO4ALgUWA6tEZHFYmTLgZuBMVT0O+LJf8RgzXqoaO1Bg1pT0oEMZtWUlubR191FebRfjmmD4eePXFUCFqu4EEJHVwJXAFk+ZzwB3qOohAFW1+/WbCW93fTsAxbnRnZwGuxFtg9tT767ndnDq7COf3vuxU0t8jcsY8Pe03kxgr2e4yh3nNR+YLyIvisgrInJJpAmJyA0isl5E1tfV2U0pTXTb09BGQVYqk1ImzvVN4XLTk8lITWKPm2iNGW9+JqdILcHhJ7CTgDJgJbAKuEdEco76kOrdqrpcVZfn5+ePeaDGjJV+VfY0tE/oU3rgXIxbkpfOngZLTiYYfianKqDYM1wE7I9Q5mFV7VHVXUA5TrIyZkKqa+mis6d/QneGCCnJS6e+rZu2rt6gQzFxyM/ktA4oE5HZIpICXA2sCSvzZ+A8ABGZinOab6ePMRnjq1BNY1bexK45gZOcAPZa7ckEwLfkpKq9wI3AY8BW4CFV3Swit4nIFW6xx4B6EdkCPA38o6rW+xWTMX7bU99OekrihHl+02Bm5kwiQbBTeyYQvj6mXVXXAmvDxt3i+V+Br7gvYya8PYfaKclLnzDPbxpMSlIC0ydPsuRkAmF3iDBmjDR39nCwpYuiKO9CPhLFeelUHeqwi3HNuLPkZMwYeauqCQWKcycFHcqYKclLp7uvn5rmzqBDMXHGkpMxY2TjXudGqbFUcwp1irBTe2a8WXIyZoy8saeRqZkpE/ri23C56clkpiZZjz0z7iw5GTMGVJWNexuj/pZFI2UX45qg+Npbz5iJarD7zkXS2N7NwdYuTp87ZejCE0xJXjpbDjTT2tVLZqrtMsz4sJqTMWNg76EOILY6Q4QU28W4JgCWnIwZA1UN7SQmCNMmT7wn3w6lKNcuxjXjz5KTMWNg76F2ZkxOIykh9n5SyYl2Ma4Zf7H3SzJmnPX1K/saOyiKgfvpDaQkL52qQ+12Ma4ZN5acjDlGNc2d9PRpTLY3hZTkpdPTp3Yxrhk3lpyMOUa7D9+JfOI/JmMgdjGuGW+WnIw5Rnvq28hOSyInPTnoUHyTk55MVmqSJSczbiw5GXOMdjfEzp3IByIiFNvFuGYcWXIy5hg0dfTQ2N7DrCmxe0ovpCQvnYY252JjY/xmycmYY3D4ybdTYrenXkio3emNPY0BR2Liga/JSUQuEZFyEakQkZsivH+diNSJyEb39Wk/4zFmrO2ubyM5UZg+OXZ76oXMdC/G3bDnUNChmDjg242yRCQRuAO4CKgC1onIGlXdElb0t6p6o19xGOOn3fXtFOWmk5gQu+1NIcmJCczImcSG3ZacjP/8rDmtACpUdaeqdgOrgSt9/D5jxlV3bz8HmjoOn+6KB8V56WyqaqK3rz/oUEyM8zM5zQT2eoar3HHhPiQim0Tk9yJSHGlCInKDiKwXkfV1dXV+xGrMiFUdaqdf46O9KaQkL52Onj7eqW4JOhQT4/xMTpHOc4Tf++QvQKmqLgGeAH4ZaUKqereqLlfV5fn5+WMcpjGjE7r4Np5qTqF5tXYn4zc/k1MV4K0JFQH7vQVUtV5VQ/1Sfw6c7GM8xoypyoNtFGSlkp4SP884ypmUTEFWqrU7Gd/5mZzWAWUiMltEUoCrgTXeAiIy3TN4BbDVx3iMGTN9/cqehnZmT43965u8RIRlJblssO7kxme+JSdV7QVuBB7DSToPqepmEblNRK5wi31JRDaLyJvAl4Dr/IrHmLFU3dRJV28/pXGWnACWzcphT0O7XYxrfOXr+QhVXQusDRt3i+f/m4Gb/YzBGD/sqm8DoDQO7gwRrr61G4AfP76NxTMmD/tzHzu1xK+QTAyyO0SnVIobAAAY6ElEQVQYMwqVB9vIy0hh8qTYvdnrQGbkTCJRxO6zZ3xlycmYEVJVKuvb4rLWBKGLcdMO91Y0xg+WnIwZodqWLtq7+5g9NX66kIebNSWDfYc66O23i3GNPyw5GTNClXHc3hRSkpdOb79yoNGejGv8YcnJmBHadbCNrLQk8jJSgg4lMPZkXOM3S07GjICqUnnQaW+K5YcLDiV7UjI56cnW7mR8Y8nJmBGob+umubOXOfnxe0ovpCQvnT31baiG35XMmGNnycmYEdhR1wrA3KmZAUcSvFl56TR39tLU0RN0KCYGWXIyZgR21rWRnZbElMz4bW8KKXE7hNipPeMHS07GDFO/KjvrWpmbnxnX7U0h07LTSE4UdtdbcjJjz5KTMcNU29xFW3cfc/LtlB5AYoJQnJfObrdrvTFjyZKTMcMUam+yzhDvmj01g+qmTjq6+4IOxcQYS07GDNPOulbyMlLITbf2ppDZUzNQ3r0w2ZixYsnJmGHo61d2HmxjrtWajlCcm05SgrDroCUnM7YsORkzDAeaOujq7bf2pjDJiQkU5aZbcjJjzpKTMcOwo87Z+c6Jw4cLDmX21Az2N3bQ2WPtTmbs+JqcROQSESkXkQoRuWmQcleJiIrIcj/jMWa0dta1UpCVSlZa/D2/aSihdifrUm7Gkm/JSUQSgTuAS4HFwCoRWRyhXBbOI9pf9SsWY45Fb38/lfVtdkpvACV56SSKtTuZseVnzWkFUKGqO1W1G1gNXBmh3L8D3wPs3vsmKlU1dNDTp9YZYgApSQnMzJ3EroOtQYdiYoifyWkmsNczXOWOO0xElgLFqvrXwSYkIjeIyHoRWV9XVzf2kRoziB0HWxGc01cmstlTM9jX2EFXr7U7mbHhZ3KKdH+Xw7cvFpEE4EfAV4eakKrerarLVXV5fn7+GIZozNB21rUxPSeN9JSkoEOJWnPzM+lXqLRTe2aM+JmcqoBiz3ARsN8znAUcDzwjIpXAacAa6xRhokl3bz97GtrtLuRDmDXFud5pe62d2jNjw8/ktA4oE5HZIpICXA2sCb2pqk2qOlVVS1W1FHgFuEJV1/sYkzEjsqehnb5+tc4QQ0hOTGD21AwqLDmZMeJbclLVXuBG4DFgK/CQqm4WkdtE5Aq/vteYsbSjrpUEgdIp6UGHEvXmFWRS29Jlz3cyY8LXk+iquhZYGzbulgHKrvQzFmNGo6K2laLcdFKTE4MOJeqVFWTxf1RTUdvCybPygg7HTHB2hwhjBtDS2cO+xg7mF2YFHcqEUJidSlZqkrU7mTFhycmYAYR2sgssOQ2LiDCvIJOK2lb6VYf+gDGDsORkzAC21bSQmZrE9Jy0oEOZMOYVZNLe3ceBJrum3hwbS07GRNCvyvaaVuYXZpJgj2QftnkFTq/GipqWgCMxE50lJ2Mi2NvQTkdPn7U3jVBWWjLTstPYXmftTubYWHIyJoJtNS0ITg80MzJlBZnsrm+nu7c/6FDMBGbJyZgIttW0UpKXzqQU60I+UvMKM+nrV7tLuTkmlpyMCVPb0ul0IZ9mtabRKJ2SQVKCUFFr7U5m9Cw5GRPmuW0HAetCPlrJiQmUTs2w653MMbHkZEyYZ8pryUpNYvpk60I+WmV2KyNzjCw5GePR29fPc9vqKCvMQqwL+aiFupTvsNqTGSVLTsZ4bNzbSHNnLwusvemYFGankZmaxHZrdzKjZMnJGI9nyutITBDm2SMyjkmC3crIHCNLTsZ4PF1ey7KSHOtCPgbmFWTS1t1Htd3KyIyCJSdjXLUtnWze38zKBQVBhxITDt/KyNqdzChYcjLG9Wx5HQArF+QHHElsyA7dysjancwo+JqcROQSESkXkQoRuSnC+58TkbdEZKOIvCAii/2Mx5jBPLG1hmnZaSyenh10KDFjXkEmlXYrIzMKviUnEUkE7gAuBRYDqyIknwdU9QRVPQn4HvBDv+IxZjAd3X08u62O9xxXaF3Ix9C8AudWRpX1disjMzJ+1pxWABWqulNVu4HVwJXeAqra7BnMAKxbjwnE89vr6Ozp5z2LpwUdSkx591ZG1u5kRibJx2nPBPZ6hquAU8MLicgXgK8AKcD5kSYkIjcANwCUlJSMeaDGPLa5huy0JE6dkxd0KDElJSmBWVPSrd3JjJifNadI50aOqhmp6h2qOhf4Z+AbkSakqner6nJVXZ6fb43VZmz19vXz5Ds1XLCokORE6yM01soKsqhp7qK22bqUm+Hz85dYBRR7houA/YOUXw2838d4jInotcoGGtt7uPi4wqBDiUmhLuXPbKsLOBIzkfiZnNYBZSIyW0RSgKuBNd4CIlLmGbwM2O5jPMZE9LfNNaQmJXDOfKuV+2H65DSy0pIOd9U3Zjh8a3NS1V4RuRF4DEgE7lPVzSJyG7BeVdcAN4rIhUAPcAj4pF/xGBOJqvL4lhrOLssnPcXPJtj4JSLML8ziue119Pb1k2SnTs0w+PprVNW1wNqwcbd4/v9/fn6/MUOpqG1lX2MHN54/L+hQYtqCwixe332IDXsaWTHbOp2YodkhjIlrz7rtIHZKz1/zCjJJShCeLq8NOhQzQVhyMnHt2W11lBVkMjNnUtChxLS05ESWl+by9DuWnMzwWHIycau9u5dXdzZwrtWaxsV5Cwp4p7qFA00dQYdiJgBLTiZuvbKznu6+fs61G72Oi9Dd3q3XnhkOS04mbj1bXsek5EROKbUG+vEwv9A5ffrEVju1Z4ZmycnErWe31XH63CmkJduDBceDiPCe4wp5bnsdLZ09QYdjopwlJxOXdte3UVnfbu1N4+zyJdPp7u3nSas9mSFYcjJxKdRrzJLT+FpanMv0yWn8ddOBoEMxUc6Sk4lLT75Ty5z8DEqnZgQdSlxJSBAuPX46z22ro9lO7ZlBWHIycaety+lCfsHCgqBDiUuXLZlOd18/T26tCToUE8UsOZm480LFQbr7+jl/od2FPAhLi3OYMTmNR+zUnhmEJScTd57aWktWWhLLS3ODDiUuJSQI7z1hOs9tO0hTh53aM5FZcjJxpb9feaq8lnPm59uDBQN0xUkz6O7rZ+1bVnsykdmv08SVt/c3UdfSZe1NATth5mTm5mfwpw37gg7FRClLTiauPPVOLSLv3krHBENE+OCyIl6rbGBvQ3vQ4ZgoZMnJxJVH365mWUkueRkpQYcS9648aQYAf3rDak/maL4mJxG5RETKRaRCRG6K8P5XRGSLiGwSkSdFZJaf8Zj4VlHbyjvVLVy+ZHrQoRigKDedU2fn8ac39qGqQYdjooxvT8IVkUTgDuAioApYJyJrVHWLp9gbwHJVbReRzwPfAz7qV0wmvjzw6p4jhp/cWoMAXT39R71ngvHBZTP55z+8xca9jSwtsd6T5l1+1pxWABWqulNVu4HVwJXeAqr6tKqGTji/AhT5GI+JY6rKpqomSqdmkD0pOehwjOvSE6aTlpzA716vCjoUE2X8TE4zgb2e4Sp33ECuB/4v0hsicoOIrBeR9XV19iwYM3LVzZ3UtXaxpGhy0KEYj+y0ZN57wnTWbNxPW1dv0OGYKOJncpII4yKeWBaRa4HlwPcjva+qd6vqclVdnp9vN+o0I/dWVRMJAsfNsOQUbVatKKG1q9fuGGGO4GdyqgKKPcNFwP7wQiJyIfB14ApV7fIxHhOnVJVN+5qYk59JZqpvzaxmlJbPymVeQSYPrrN2QPMuP5PTOqBMRGaLSApwNbDGW0BElgJ34SQme8CL8cWehnYa2rpZMtNqTdFIRLj6lGLe2NNIeXVL0OGYKOFbclLVXuBG4DFgK/CQqm4WkdtE5Aq32PeBTOB3IrJRRNYMMDljRu21XQ2kJiVwgrU3Ra0PLisiJTGBB1+z2pNx+HqOQ1XXAmvDxt3i+f9CP7/fmPauXt7a18TJs3JJTbLHsUervIwU3nNcIX96Yx//8t5FpCTZ/QHinW0BJqZt2NtIb7+yYnZe0KGYIXxw2UyaOnp4dpv1yDWWnEwMU1Ve29VASV460ydPCjocM4Szy/LJy0jhzxvtdkbGkpOJYbsOtnGwtYsVpVZrmgiSExO4fMl0nthSQ4s9wj3uWb9aE7NerDjIpORE6wgRJYZzy6j05ES6evu5dc0WTp7l3M7oY6eW+B2aiUJWczIxqby6ha3VLZwxd4o9VHACKc5LJy8jhTf3NgYdigmY/WpNTLrr2R2kJCZw+pwpQYdiRkBEOLEohx11rTTbI9zjmiUnE3P2NrTz8Jv7OaU0l3S7I8SEs7QkBwU27DkUdCgmQJacTMy55/mdJAicVWb3YZyIpmamMmdqBusqG+i35zzFLUtOJqZUN3Xy4Lq9fGDpTCbbozEmrBWz8zjU3kNFbWvQoZiAWHIyMeWOpytQVb54flnQoZhjsHhGNhkpiby2qyHoUExALDmZmLG3oZ3V6/bw0VOKKc5LDzoccwySEhI4eVYu71Q3U93UGXQ4JgCWnEzMuP2p7YgIN55ntaZYcEppHv0Kq+1RGnHJkpOJCTvrWvnDhn18/LRZTJucFnQ4ZgxMyUxlQWEW979UaXeMiEOWnExM+MmT20lJTODzK+cGHYoZQxcuKqSxvYf7XqgMOhQzziw5mQmvvLqFNW/u57ozS5mamRp0OGYMzcydxMXHFXLP8ztpbO8OOhwzjiw5mQnvR49vIzMlic+eMyfoUIwP/uGi+bR293L3czuDDsWMI1+Tk4hcIiLlIlIhIjdFeP8cEdkgIr0icpWfsZjY9FZVE49urub6s2eTk54SdDjGBwunZfO+JTO4/6VK67kXR3xLTiKSCNwBXAosBlaJyOKwYnuA64AH/IrDxLbv/62cnPRkPnXW7KBDMT762nsW0NuvfHvt1qBDMePEz5rTCqBCVXeqajewGrjSW0BVK1V1E9DvYxwmRj1TXstz2+q48bx5ZKfZ3SBiWcmUdP5+5Vz+8uZ+Xqw4GHQ4Zhz4mZxmAns9w1XuOGOOWW9fP/+xdiuzpqTzidNLgw7HjIPPnTuXkrx0bnn4bbp77Xg21vmZnCTCuFHdxVFEbhCR9SKyvq6u7hjDMrHgofVVbKtp5aZLFpKSZP164kFaciK3XrGYHXVt3PfirqDDMT7z83kCVUCxZ7gI2D+aCanq3cDdAMuXL7fbFMeZ8CeodnT38aMntjFrSjoNbd3DesKqiQ3nLyzkosWF/OSJ7Vxx4gxm5EwKOiTjEz8POdcBZSIyW0RSgKuBNT5+n4kTa98+QHt3L5cvmYFIpAq6iWW3XL4YRfnWI1uCDsX4yLfkpKq9wI3AY8BW4CFV3Swit4nIFQAicoqIVAEfBu4Skc1+xWNiw7aaFl7ffYizy/KZaUfNcak4L50bz5vH2reqeW6bneaPVb6erFfVtao6X1Xnquq33XG3qOoa9/91qlqkqhmqOkVVj/MzHjOxdfb08ac39pGflcr5CwuCDscE6DPnzKF0itM5oqO7L+hwjA+sJdlMCKrKwxv30dzRw1XLikhOtE03nqUmJfIfHziByvp2fvC38qDDMT6wX7iZEF6oOMibVU1ctLjQntVkADhj3lQ+dmoJ9764i9d3Hwo6HDPGLDmZqFdR28qjb1dz3Ixszp2fH3Q4JorcfOlCpmen8U+/f5POHju9F0v87EpuzDHbVNXIA6/tJj8rlatOLrLeeXFoqEsFLj5uGr94qZKP3/saH1h65HX+Hzu1xM/QjI+s5mSi1tv7mrj2nleZlJzIdWeUkpqUGHRIJgqVFWZx7vx81lU2sGGPnd6LFZacTFR6e18T19zzKllpyXz67Dl2x3EzqAsXFTJnagYPb9zHgaaOoMMxY8CSk4k6W/Y3c+29r5KZmsTqG04j1xKTGUJigvDRU4pJS07k1y/vpqnDHus+0VlyMlFl64FmrrnnFdKTE3nwM6dZzzwzbFlpyXzi9FLae/r45UuVdv3TBGfJyUSNJ7fW8OH/eZnUpEQevOE0SqZYYjIjMzNnEtecWkJdSxe/fqWS1q7eoEMyo2TJyQSuv1+585kKPv2r9ZROTedPXziDWVMygg7LTFBlBVlctbyIPQ3tfPSul6ltsafnTkSWnEygdtS18tG7X+Z7j5Zz2QnT+d1nz2D6ZLtnnjk2Jxbl8PHTStlZ18YH73yJrQeagw7JjJCoTqwnUCxfvlzXr18fdBhmFLzXq3T29PHc9jpe2H6Q5MQELjthOktLcuw6JjOmjp+ZzfW/XE9jezdfPL+Mz6+cG7e3vhKR11V1edBxDJddhGvGVVdvH6/ubODZbXV09PSxpGgyl50wnSx7zLrxwZKiHB778jl8c81mfvj4Nv7y5n6+eEEZl50wncQEOxCKZpaczLho7erl2fJanq84SHt3H2UFmVx83DR7WJzxXV5GCrevWsrlS6bz/cfK+dKDb/Djx7fx+ZVzef/SmXFbk4p2lpyMr5o6evj1y5Xc88IuGtt7mF+YyXkLCqzDgxl3Fx83jYsWFfLY5mpuf6qCf/z9Jn7y5HauP2s2Hzq5iGyrvUcVa3MyY66nr591uxr4/etVPPLWAbp6+zl/YQELCrPsuiUTFVSV8uoWni6vZe+hDpIThROLclhaksusKelce9qsoEMcc9bmZOKKqrKvsYPy6hbeqW7hzb2NvLSjntauXrJSk7jq5CJWrSjh+JmTh7yBpzHjRURYOD2bhdOz2Xeog1d21fNmVSPrdx8iJz2ZHXWtrFxQwKmz80hLtns6BsHXmpOIXAL8BEgE7lHV74a9nwr8CjgZqAc+qqqVg03Tak7jr6u3jwONnVQd6qDqUDuPbq6msb2H+tYualu66OrtP1w2Nz2ZsoIsygozKSvIIiXJzuebiaGrt4+tB5p5c28TlfVtdPX2k5woHD9zMkuLc1k0PYuF07KZNTWdrNSkCdez1GpOLhFJBO4ALgKqgHUiskZVt3iKXQ8cUtV5InI18J/AR/2KKVqEDgi8xwUa6f0jxoXK6VHjiFAuVLa7t5/u3n663Fd3bz/dfaFxfUe839nTR12Lk3BqmjupbemiuqmTmpbOI6abIDB5UjI56SksLcmhMDuNadlpFGan2VGmmbBSkxI5qTiXk4pz+eCymby8o55XdtWzYfchfvPq7iMOwtJTEpk22dnup2WnkT0pmYzURDJSk8hMTSI9JYmUpARSEhNITUpw/neHU5KOHJeamHj4f+tB+C7fak4icjpwq6pe7A7fDKCq3/GUecwt87KIJAHVQL4OEtRoa07VTZ2c/4NncGJ4d3xoZz9QomCQpBApoXjHT7DmvMMmJSeSlZZE9qRkstOSyElPITc9hdyMZHLTU8hOS7YfkYkrff1KQ1s31c2dNLZ309zRQ1NnL80dPTR39NDZ20dXTz/H+pNPTBASQzUyAQHeHRSWl+by6+tPHdW0reb0rpnAXs9wFRC+VA+XUdVeEWkCpgAHvYVE5AbgBnewVUTKw6YzNfwzcSae59/mPT7F5by/A/zvp0c97xOql4efySnSoXX4gcVwyqCqdwN3D/hFIusn0hHBWIvn+bd5t3mPN/Ey7362VlcBxZ7hImD/QGXc03qTgQYfYzLGGDMB+Jmc1gFlIjJbRFKAq4E1YWXWAJ90/78KeGqw9iZjjDHxwbfTem4b0o3AYzhdye9T1c0ichuwXlXXAPcCvxaRCpwa09Wj/LoBT/nFiXief5v3+GTzHuMm3B0ijDHGxD67QtIYY0zUseRkjDEm6sRMchKRfxeRTSKyUUT+JiIzgo5pvIjI90XkHXf+/yQiOUHHNF5E5MMisllE+kUk5rvXgnNbMBEpF5EKEbkp6HjGk4jcJyK1IvJ20LGMJxEpFpGnRWSru73/v6Bj8lvMJCfg+6q6RFVPAv4K3BJ0QOPoceB4VV0CbANuDjie8fQ28EHguaADGQ+e24JdCiwGVonI4mCjGlf3A5cEHUQAeoGvquoi4DTgC7G+3mMmOalqs2cwgwgX88YqVf2bqva6g6/gXFMWF1R1q6qG3zEklq0AKlR1p6p2A6uBKwOOadyo6nPE4bWQqnpAVTe4/7cAW3HusBOzYuqRGSLybeATQBNwXsDhBOVTwG+DDsL4Zji3BTMxTERKgaXAq8FG4q8JlZxE5AlgWoS3vq6qD6vq14GvuzeZvRH45rgG6KOh5t0t83Wc6v9vxjM2vw1n3uPIsG75ZWKTiGQCfwC+HHa2KOZMqOSkqhcOs+gDwCPEUHIaat5F5JPA5cAFsXaXjRGs93gwnNuCmRgkIsk4iek3qvrHoOPxW8y0OYlImWfwCpwb+MYF96GO/wxcoartQcdjfDWc24KZGCPOkw3vBbaq6g+Djmc8xMwdIkTkD8ACoB/YDXxOVfcFG9X4cG//lIrzNGGAV1T1cwGGNG5E5APA7UA+0AhsDD1DLFaJyHuBH/PubcG+HXBI40ZEHgRW4jwyowb4pqreG2hQ40BEzgKeB97C2ccB/Iuqrg0uKn/FTHIyxhgTO2LmtJ4xxpjYYcnJGGNM1LHkZIwxJupYcjLGGBN1LDkZY4yJOpacjDHGRB1LTsYYY6LO/wfkCLUPcCrHNAAAAABJRU5ErkJggg==\n", 103 | "text/plain": [ 104 | "
" 105 | ] 106 | }, 107 | "metadata": { 108 | "needs_background": "light" 109 | }, 110 | "output_type": "display_data" 111 | }, 112 | { 113 | "name": "stdout", 114 | "output_type": "stream", 115 | "text": [ 116 | "-------------------------------------------------\n", 117 | "sqrt(n) * (RnBar - p) converges in distribution to N(0, p(1-p))\n", 118 | "-------------------------------------------------\n", 119 | " R RnBar\n", 120 | "mean 0.6500 -0.000711\n", 121 | "variance 0.2275 0.228179\n", 122 | "0.2281787449875001\n", 123 | "0.8664640788861249\n" 124 | ] 125 | } 126 | ], 127 | "source": [ 128 | "# Convergence of sqrt(n) (RnBar - p)\n", 129 | "\n", 130 | "rv = np.sqrt(dist_n) * (rnbar_dist - p)\n", 131 | "actual = np.sqrt(n) * (rnbar_actual - p)\n", 132 | "\n", 133 | "plt.title('Simulated Distribution of RV sqrt(n) * (RnBar - p) @n=200', fontsize=14)\n", 134 | "sns.distplot(rv, bins=20)\n", 135 | "plt.show()\n", 136 | "\n", 137 | "print('-------------------------------------------------')\n", 138 | "print('sqrt(n) * (RnBar - p) converges in distribution to N(0, p(1-p))')\n", 139 | "print('-------------------------------------------------')\n", 140 | "table = pd.DataFrame({'R': [p, p * (1 - p)], 'RnBar': [np.mean(rv), np.var(rv)]})\n", 141 | "table.rename(index={0:'mean', 1:'variance'}, inplace=True)\n", 142 | "print(table)\n" 143 | ] 144 | } 145 | ], 146 | "metadata": { 147 | "kernelspec": { 148 | "display_name": "Python 3", 149 | "language": "python", 150 | "name": "python3" 151 | }, 152 | "language_info": { 153 | "codemirror_mode": { 154 | "name": "ipython", 155 | "version": 3 156 | }, 157 | "file_extension": ".py", 158 | "mimetype": "text/x-python", 159 | "name": "python", 160 | "nbconvert_exporter": "python", 161 | "pygments_lexer": "ipython3", 162 | "version": "3.7.0" 163 | } 164 | }, 165 | "nbformat": 4, 166 | "nbformat_minor": 2 167 | } 168 | -------------------------------------------------------------------------------- /U02 L04 - Parametric Estimation and Confidence Intervals.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Fundamentals of Statistics Unit 2 Lecture 4 Notes " 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 15, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "import numpy as np\n", 17 | "import random\n", 18 | "import pandas as pd\n", 19 | "import matplotlib.pyplot as plt\n", 20 | "import seaborn as sns\n", 21 | "import warnings\n", 22 | "warnings.filterwarnings('ignore')" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": {}, 28 | "source": [ 29 | "## Table of Contents\n", 30 | "- Parameter Estimation Definitions\n", 31 | "- Bias of an Estimator\n", 32 | "- Jensen's Inequality\n", 33 | "- Variance of an Estimator\n", 34 | "- Quadratic Risk\n", 35 | "- Confidence Intervals\n", 36 | "\n", 37 | " Click here to render this page with nbviewer.jupyter.org and use bookmarks. " 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "# Parameter Estimation Definitions \n", 45 | "\n", 46 | "\n", 47 | "### Statistic \n", 48 | "\n", 49 | "Any measurable function of the sample, e.g. $\\bar{X}_n$, $max_i X_i, X_1 + log(1 + |X_n|)$, $s^2$ (sample variance), ...\n", 50 | "\n", 51 | "\"If you can compute it once I give you data, it is measurable.\"\n", 52 | "\n", 53 | "### Estimator of $\\theta$\n", 54 | "\n", 55 | "Any statistic whose expression does not depend on $\\theta$.\n", 56 | "\n", 57 | "### An estimator $\\hat{\\theta}_n$ of $\\theta$ is weakly (resp. strongly) consistent if\n", 58 | "\n", 59 | "$\\hat{\\theta}_n \\xrightarrow[n \\to \\infty]{\\mathbb{P}(resp. a.s.)} \\theta$ $(w.r.t. \\mathbb{P})$\n", 60 | "\n", 61 | "- Note: resp. here shows the relationships weakly is to $\\mathbb{P}$ as strongly is to a.s.\n", 62 | "\n", 63 | "### An estimator $\\hat{\\theta}_n$ of $\\theta$ is asymptotically normal if\n", 64 | "\n", 65 | "$\\sqrt{n}(\\hat{\\theta}_n - \\theta) \\xrightarrow[n \\to \\infty]{(d)} \\mathcal{N}(0, \\sigma^2)$\n", 66 | "\n", 67 | "- The quantity $\\sigma^2$ is then called asymptotic variance of $\\hat{\\theta}_n$.\n", 68 | "\n", 69 | "asymptotic variance as defined here is distinct from, and necessarily follows from:\n", 70 | "\n", 71 | "$var(\\hat{\\theta}_n)\\xrightarrow[n \\to \\infty]{} 0$\n", 72 | "\n", 73 | "- graphic example in 'Unit 1 Notes', 'Law of Large Numbers / Convergence'" 74 | ] 75 | }, 76 | { 77 | "cell_type": "markdown", 78 | "metadata": {}, 79 | "source": [ 80 | "# Bias of an Estimator \n", 81 | "\n", 82 | "Bias of an estimator $\\hat{\\theta}_n$ of $\\theta$:\n", 83 | " \n", 84 | "$bias(\\hat{\\theta}_n) = E[\\hat{\\theta}_n] - \\theta$\n", 85 | "\n", 86 | "- if an estimator has bias 0, it's unbiased\n", 87 | "\n", 88 | "Examples ($X_1, ..., X_n \\overset{iid}{\\sim} Ber(p)$):\n", 89 | "\n", 90 | "- $\\hat{p}_n = \\bar{X}_n$: $bias(\\hat{p}_n) = 0$\n", 91 | "- $\\hat{p}_n = X_1$: $bias(\\hat{p}_n) = 0$\n", 92 | "- $\\hat{p}_n = \\frac{X_1 + X_2}{2}$: $bias(\\hat{p}_n) = 0$\n", 93 | "- $\\hat{p}_n = \\sqrt{\\mathcal{I}(X_1 = 1, X_2 = 1)} = \\sqrt{Ber(p^2)} = Ber(p^2)$: $bias(\\hat{p}_n) = p^2 - p$\n", 94 | "\n", 95 | "Unbiased $\\neq$ best! The words are related in english, but the ideas are only loosely related here. The definition is exactly as stated.\n" 96 | ] 97 | }, 98 | { 99 | "cell_type": "code", 100 | "execution_count": 31, 101 | "metadata": {}, 102 | "outputs": [ 103 | { 104 | "name": "stdout", 105 | "output_type": "stream", 106 | "text": [ 107 | "Estimating theta = 0.5\n", 108 | "----------------------\n", 109 | "Estimator [sample mean] with bias 0 : 0.6\n", 110 | "Estimator [X1] with bias 0 : 1\n", 111 | "Estimator [(X1 + X2) / 2] with bias 0 : 1.0\n", 112 | "Estimator [sqrt(X1 == 1 & X2 == 1)] with bias -0.25 : 1.0\n" 113 | ] 114 | } 115 | ], 116 | "source": [ 117 | "p = 0.5\n", 118 | "n = 10\n", 119 | "# generates the sum of n bernoullis w.p. p\n", 120 | "Xsum = np.random.binomial(n, p)\n", 121 | "\n", 122 | "est1 = Xsum / n\n", 123 | "# if Xsum = 6, est1 = 0.6, X1 == 1 and X2 == 1 each with probability 0.6\n", 124 | "X1, X2 = 1 if random.random() <= est1 else 0, 1 if random.random() <= est1 else 0\n", 125 | "est2 = X1\n", 126 | "est3 = (X1 + X2) / 2\n", 127 | "est4 = np.sqrt(X1 == 1 and X2 == 1)\n", 128 | "\n", 129 | "print('Estimating theta = 0.5')\n", 130 | "print('----------------------')\n", 131 | "print(f'Estimator [sample mean] with bias 0 : {round(est1, 2)}')\n", 132 | "print(f'Estimator [X1] with bias 0 : {round(est2, 2)}')\n", 133 | "print(f'Estimator [(X1 + X2) / 2] with bias 0 : {round(est3, 2)}')\n", 134 | "print(f'Estimator [sqrt(X1 == 1 & X2 == 1)] with bias {p**2 - p} : {round(est4, 2)}')" 135 | ] 136 | }, 137 | { 138 | "cell_type": "markdown", 139 | "metadata": {}, 140 | "source": [ 141 | "# Jensen's inequality \n", 142 | "\n", 143 | "A function $g: \\mathbb{R} \\to \\mathbb{R}$ is convex if for all pairs\n", 144 | "\n", 145 | "$g(tx_1 + (1 - t)x_2) \\leq tg(x_1) + (1 - t)g(x_2)$ for all $0 \\leq t \\leq 1$\n", 146 | "\n", 147 | "Geometrically, this means for all $x_1 \\leq x \\leq x_2$, the secant line connecting $(x_1, g(x_1))$ and $(x_2, g(x_2))$ is above, or dominates the graph of g on the domain between the x values." 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": 88, 153 | "metadata": {}, 154 | "outputs": [ 155 | { 156 | "data": { 157 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlsAAAEJCAYAAACja+IQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xd4FFUXwOHfSei919B7b6GICigqzS7Su7SPYgMVBBURxIIKKtgFRLoVAUVBURCQKi303gmdkEDK3u+PmcUlpGw2u9mU8z5PnmR32tnN7N0zd+6ZEWMMSimllFLKNwL8HYBSSimlVHqmyZZSSimllA9psqWUUkop5UOabCmllFJK+ZAmW0oppZRSPqTJllJKKaWUD2mypZRSSinlQxk62RKRqiKyRkSuicihVBCPEZH2KbCd20Vkq4hEisgKl+fHiMiXSVjPEBFZ6JMgPSSWGfZ7+YW/43GHiAxPDftfShGRQyIyPL7HSqUXIjJdRMbZf7cQkWMu03aISIsElr1PRH5IgTCd2zskIvfEM+3G6/BgvR4vG8/6iorIThHJ6q11poQMnWwB44BwoCrQMKU2au98i+KYVBz4KQVCmAxsASoAj9oxFQGexXpP3PUZECwid3o9Qs9NAtoB/YBHRGRi7BnsRu9HETkpIuF24tknxSNNZ5JxsNAQmOrteDIqEVkhIn39HYc77IOjd0TknP3zjRvLrLAPkMNE5JKI/CUitVIiXm8yxtQwxqxIYJbXgTdSKJw0wxhzGvgD6O/vWJIioydbFYFVxphDxphQfwdjjDlljLmeApuqCPxujDlqjDlvP9cXWGeMOeDuSuxYZwNP+iDGJBORMcDjQHNjzOdAC6CbiLwYa9amwDagPVAT+Aj4VES6pFy0yskYE2qMCfd3HMov7gO6AXWAEsAnbi43xBiTCygIrABmerJxEcnkyXK+JiINgbzGmLXxTE+VcaegWcAAfweRFIkmW/aRxzAR2Ssi10XkmIhMcJleS0SWiUiEiJy3e23yukyfLiKLROQpETkuIhdEZJqI5LCnDxCR07F3HhGZLSI/ujx+QEQ22kc0B0VkvIhksadVEZGrItLDZf7W9mmyJvG8LoP1AX/ZPiIfIyJl7b+DY8/rPGJ3mecxEfnN7hkJEZF7Yy1TVUQW2kdeYWKdrqxlJwQ9gXb2eoyzKzl2z0By39s4XnNZ+3XnBb60t9fLntwFWOgyb2G75+dll+dq2++/a+/FQuDB+LbpDSLyhVhd7tntx4EiskpcegdFZCjW+3qnMWYHgDFmK9AMGCAi/3POa4x53Rgz2hjztzHmgDHmI+A74LFE4njW7gW7ar/fn4tIPpfpvez/dUsR2W7P94eIlIu1nudF5JQ971dALjfeg5dF5LD9GTxlL+ecJvY699v7yjYR6RZr+RIiMkus3oNwEflXRO6yp1UQq6fvlB3zJhG5P9byh0RktIh8IiKXxWoHnnOdbv+5wN6vDuEmufW0ohGR/iKywI7nQByvp6SIzLX3+QsislhEKrlML2W/pvP2690lIp3cjSm9EJH77f/1RRFZLSK1XaYdEusU9la7nZonItnsaYXstuWi/R6uFJEAe1oJEflWRELFaoufdFnnGBGZLyJficgV+3MbfGtkN0QDEcApY8x1Y8xvSXl9xphoYC5Q3SWGABEZYX8eztnxFLCnOdvvJ0TkCPC7y3M9ReSIiJwVkVEu68sqIpNE5IT9M0nsU1j2Z35VrPfciEjFxGKXBE7bAW2AP+NY72AR2QvstZ+rKtb30HkR2S0iHVzmbycim+3P61Gxvntc19fdblPOub7eBBSyt3VFRP4UkTIu64o3jjhedz8R2WfPu1BEStjPvyoiH9h/Z7Y/+2/Zj7OL9d2T317NP0B51xhSPWNMgj/ABOAi0AerR+Q2YJA9LQdwHPgBqAU0B/YA37osPx24hHXKqRrWkcxFYKQ9PT9wHWjtskxO4CrwuP24FXAZ6I116usuYDcw0WWZ/vY8FYDCwCnglQReVzFgFzDR/jsXUBYwQHCseQ3Q3v7bOc8u4AGgEjADOAfksucpAZwFfgQaAZWxjt7q2tuZB/xmb7cYkCWO7ST7vY3jNQfa27sKPGX/nR0oADiAprHmbwVE2v/z7MAOYFqseXIAMUDLBN7rrkBYIj9dE1g+p/3ap9iPX7b/v0US23/d/QF+AT5PZJ6ngbvtfaA5sBWY6TK9FxAFLLP/77WBzcBSl3k62O/pAHu/GGXvt4cS2O5j9jztgNJAMNaRvXP6eKzPQ2ugHFbifBVo5/L+7QX+xko+naeP77Kn1wEG2vtZRTumSKCqyzYOYe3jQ+x5htr762329ML24772flU4gddzCBiewGMDHMP6zFTEaoMigTIu+9werP2/NtYwgM+Bw0AOe56fsD5jdez3pDUubUx6/sHq6ekL1AfOAI2xPvs97fc6q8v7vg6rvSoA7AQG2tMmAB8Dme2fOwHBOkDfiPUZzAKUBw4ArezlxgDXgLb2NicAaxOItYS9b08DJCmvz/47i73//xXrc7oWCAKyYvWWzbGnlbX3r6+wPhfZXZ77zH5cB+s7qZq9zFh7fUXs/Xw18JrLZ35VrPgMUNH+ezowzv67BXAs1n5/TzyvcQHwXBzr/c3+X2W34z+K9b2Yyf5/nwVquGyvlv0/qw2cBh62p1XHaneb2e/Ru1iJb3zxTAeuuMw/2fm63YjD9T24255W317PB87/nT1tm/13U2A/8I/LtC2xYtoKPOjvz5vbn8tEdupcWB+cgfFM74f1ZZ/b5bkWcexsR4FMLvN8Bixzefw9N39pdbPXm81+/BfwUqxtP2zvLBJrPWuBxVhfLIGJvL7twBiXx2VxP9ka4DK9pP3cHfbj8VgNf5YEdtxFcTzvuh2vvLfxbD8M6OXyuK693nJxzDsJqzGdBuzDTihjzXMeeCKB7eXG+tJM6Cd3IjEHY33hjsVKaNp47UMA99vrbJTE5VpjNcoB9uNe9vtYxWWernbcznlWA5/FWs8yEk62nsVKpjLHMS0nVs/AnXH835a47EtXgEJJeG1rgdEujw9hf2G5PLc31jw39t9E1n2IxJOtCS6PM2GNrexmP+5jb9v1sx+IlQx2sB9vJYGDrfT8w3/J1kfYSYHLtN1Yp9md73s3l2lvAR/bf4/FOlisGGv5xsCRWM+NxD4Iw0q2XNv26kBEPHFmxjqd383e1hfO/ylW+/1AAq8vHOvAMhKrnWzpMn1nrMfFsT7fmfiv/S7vMt35XJDLc+uATvbf+4G2LtNaYX9e8V2y9Ruxvnft9d7t8rgjsDLWPJ/Et99jtQnv2X+/DMx1mZbTfi8TSrZc58+FdZBdKrE4Yr0HXwBvxVpPlP0/yI6VbxQERgAvYh105QJeBd6PtY2/gR7+/ry5+5PYacTqWNnn8nimVwO2GmOuuDy3GquXpLrLcyHG6u51OoF1lOD0NfCw/HcqqivwjTHmmv24ATBKrNMuYSIShjVWKCfWUbRTX6wv7mZYjUhMIq8vOba6/H3C/u18TfWwPoCRyVi/t95bd2S3f1+LY9oLWB/CHli9T2FxzBPhso5bGGOuGGP2JfJzJb7l7XVswEpiXwI+Ncb87M4LS4yI3I497swYsy6Ree+2u8qPicgVrFOPWbh5H7xujNnt8vgE1peK83RjNWBNrFXHfhzbAiAbcFCsU6qPy3+VONXtab/E+nz8D6sHC6z9casx5mw8ryuniLwl1unwC/bywVi9aK62xnrsyb7mrhvbsvfvUJdtNcDqrbri8novYfWSO1/zZGC0WKfvx4lIAx/FmZqVAYbZpwIvishFrC/HEi7znHL5O5z/Tmm/jXVw9atYp3FHuKyzRKx1vggUTWCd2STuMUZ3Y41L+hrrC7s88LmI5ME6Y7AqjmWcnjTG5MPa9+8HvpH/TpGWAb53iW8nVmLgGuPRONYZ33tRAuvg2ekwN7+HvnAB6yA1Nte4ywCNY/0vumK3RyLSWKxhDKEicgmr97qQvWwJ13UZY65iHawkxHX+MKyD7BKJxRHLTe+lvZ5zQEljTASwAeusQTOs06irgdvt5/6Mta7cWAl3mpDYIDtxY7qJZ5rr81FxTHNN9BZhdWE+JCLLgXuwTok5BWBltgvi2I7rwPaaWOORwOptOphQ8HFw2L9vvG4RyRzPvDdekzHGiIgzzpuWTwZvvbfucH4J5wdOxppWFquBNliN4T9xLF+Am/8PNxGRriQ+8HWAMWZWAusQ4A6sRrOCiIixD288JSJ3AEuAl401biuhectg9Zh+hnVUeA6rK3wOVsLlFB1rUWeMHhejGGOOikgVoCXWZ+Md4BURaeyy3geAI7EWde4bie2PE7F66YZj9RiFY51myRJrPm/sa+5KaFsBwL9AXGOwzgMYY74QkaVYp7PuAVaLyARjzBjfhJsqHQXGG2PGJ3VB++BnGFayVgP4Q0TW2+s8aIyplOAK3JMJ+/NijLkmIg9iVZmtB2YYYy64EacDWCki+7C+M7baMfYxxvwde34RKetcNAlxnsBKKHbYj0vz3wH2VazT2s71x5VgeGIr1jCD2FzjPgr8aYy5N475wDqI/BDrLMA1EZnEf8nWSawDPwDsjo6CicRUymX+XFjt/gk34nDlfC+d68lpb/e4/dSfWEl4Paz94E+snsRGWGe4nMtlwupY2eLGNlOFxBrKEKzTJC0TmF5HRFwz8Kb2ene6G4Sxqtq+wcqGO2IdYbhmsZuwxo/E1SMSDSDWQOWvsL44pgAz7SOkpHAmDMVdnqubxHU4471D7AH8cYjEOu2REK+8t27ajzVuwrXHzJlozsIaBD8c+EhESseapwLW0eWmBNa/EOt9TOgnset1PYuV3DQDmmCNGfKYiDQDfgZeNcZMcmORYKzk4xljzBpjzB48O7rdiRW/qziLOFwZY64ZYxYbY57BulRCDawjPudntEwcnw3nEeQmoLaIFIp77dwBfGWM+dZYRQXH+K+HKCmiSHy/9oZNWA3t2Thes7O6FmPMMWPMp8aYDlgJcpoqFfeCz4CBdg+H2D2Y7WK1KXESa2B9Rfsg5zLWQU4M1um1yyLygj1oOVBEaopVPZdUq7B6vcaKVfwSgJVsVea/A99EichtWG2XMxn6GBjvHDwtVrHPQx7E5zQHq5e0sP0ZehnrbAxYX/Y1RKSuWMUFY5KxHVdLsHpzErIIqCzWQPfM9k9DEXEmUbmB83ai1QhrLKfTN8D9IuL8nhpL4vlAW5f5X8MaT3XUjThczQZ62+9XVqzLW/xjjDlkT/8T6yxKiH1maAXWGauD5uYrBjTCOpXr2uOYqiX45tpHN5OBCSLSW6yqpUbyX2XXLKzM/iuxKueaYfVgfGeM2ZfEWL7GymAHArPtIxansUAX+0NZU6zKh/ZiVyrYPsbqoXkZ69TXFayky212N+Za4AURqSEiTbGSt6SaitUFPd/e6SqKSGcRcSZuh4CaYlVRFoqn98yb722C7Pd6GdaXrqvXsE7d/A9rP1iLlcS67jd3AgeMMXsTWH+yTiOKSB2sU4j9jTGr7XjeFJGanrxesao/f8baZ2aJSDH7p3ACi+3F+rw8LSLlRKQz1kDcpJoM9BSrIqeSiIzEGgeTULy9RKSvvR+UwxqIGgXstd+3icBEEelj72t1RWSgiDiTi9lYA6V/EJE77fgfFLsaEWuw+SMiUl+s6xV9jZVAJ9UhoKX9XuZPbOZkmIU12PdHEWluv55mYl2vqRKAiEwWqyK5vP25a42VmGYUxj713g+rd+MC1mnBXm4uXwmrTQjDOs091Rizwh6a8QDWAdJBrDb3c/47o5CUAC9h9UY1werx2IrVS1Qf6CMi/RJY/EP57xTyTKyxg86hBZOxDt5+Fet0/1oS+YwlYhzW6a2tWGPMNtnPYR90jcV6r/aS8KlPtxljNgGXxOq9jm+eK1jvXyes9+8U8CbW0B+AQcBY+z14GZjvsuwOYDBW23ASa/+4ccHVeMwGXsHqPW6A1TniThyuMS/HGgryrb3dCtzcQ70aa0iKsxcrBGt4y1/crCtW+512JDaoC+sLZgTWIOlI/uuadk6vhTWmKwLrHzYd6zy868C6RbHWOQbYHus5wWqsDVArjjjuA1ZineK4jLXzD7Gndbefd62eqmnH1DmB13bTAHn7uWpYA+/CsT5YdxL3APl4B9Hbj2tgHZ2EYSV+q4Ga9rTCwK/28wZoEc86vPLexvG6bxogbz/XCmvnD7QfN8f6Qm/hMk8xrC/tkS7PLQVGJLYfefqD9aW/Hfgy1vMzsY4qs3qwzun2ex3751Aiyz2J1d0dYf9fOtjLlbWn9wLCYi3Twp6nkMtzI+330Tn2cExC28YqBlmDNT7hKlb3+v2xPjtD+a+XKxRrgO29LvMEYVXBXrT37c0u+10ZrC+Lq1gN7nCso9XpLssfwmUQu/3cCuBDl8cPYH3hRCXyem5aVxyPb/ocxDNPUayijTP2az4IfOl8n7GqnPZiNdShWJcHKOmr/TQ1/WAlAw/7Ow79Sfb/8T7gB3/Hkdp+sDoAdmIX0KWVH2flh1KIiPMI1q0LBNo9S8uBysY6SlVK+ZFY46s2YB14pplTLEqldxn9CvLqZgNI2j5RAqv0VhMtpfxMRN7E6jF/QRMtpVIX7dlSSimllPIh7dlSSimllPKhVHUzy0KFCpmyZcv6OwylVArauHHjWWNMQpWgaYa2YUplLO62X6kq2SpbtiwbNmzwdxhKqRQkIulmfJG2YUplLO62X3oaUSmllFLKhzTZUkoppZTyIU22lFJKKaV8SJMtpZRSSikf0mRLKaWUUsqHPEq2RORLETkjItvjmS4i8r6I7BORrSJSP3lhKqWU92gbppRKSZ72bE0HWicwvQ3WXeMrAf2BjzzcjlJK+cJ0tA1TSqUQj5ItY8xfwPkEZnkI+MpY1gL5RKS4J9tSSqUd+86E8c6vuwmPjPZ3KAnSNkwpFZsxhvGLQ9h35orX1+2rMVslgaMuj4/Zz91CRPqLyAYR2RAaGuqjcJRSvmaM4dWfdjB99SHCI2P8HU5yaRumVAbz/ebjfLbyIJsOX/T6un2VbEkcz8V5x2tjzKfGmGBjTHDhwunijh1KZUi/hpxm5d6zPHtvZQrlyurvcJJL2zClMpCw69FM+HkXdUrlo32DIK+v31e36zkGlHJ5HASc8NG2lFJ+di0qhtcWhVC5aC66Nynj73C8QdswpTKQD5bvJfTKdT7rEUxAQFzHWsnjq56thUAPu6KnCXDJGHPSR9tSSvnZJ38e4NiFCMY8WINMgeniijLahimVQewPDePLvw/yeIMg6pbK55NteNSzJSJzgBZAIRE5BrwCZAYwxnwMLAHaAvuAcKC3N4JVSqU+xy6EM3XFPtrVKk7TCoX8HY5btA1TSoFzrGkI2TIF8nzrqj7bjkfJljGmcyLTDTDYo4iUUmnK60t2IgIvtqvm71Dcpm2YUgpg2c4z/LUnlJfur07h3L4ba5ou+vuVUv7x976zLNl2isEtKlIyX3Z/h6OUUm5zjjWtVCQXPW7z7VhTXw2QV0qlc1ExDl79aQelCmSnX7Py/g5HKaWS5LO/DnDkfDiz+jYms4/HmmrPllLKIzNWH2LP6TBealedbJkD/R2OUkq57diFcKas2EebmsW4vaLvx5pqsqWUSrLTl68xadle7qpSmHurF/V3OEoplSSvLQpBEEbfXz1FtqfJllIqyV5fspPIGAdjHqyBiPevSaOUUr7yx+4zLN1xmqEtU26sqSZbSqkkWbP/HD/+e4KBzStQpmBOf4ejlFJuuxYVw5iFOyhfOCd970i5saY6QF4p5baoGAcv/7idoPzZGdSigr/DUUqpJPn0rwMcPhfOzCcakSVTyvU3abKllHLbtL8PsvdMGJ/3CNZB8UqpNOXo+XCm/GFdgPnOSil7H1M9jaiUcsvJSxFMWraXllWLcI8OildKpTGv/hRCYIAw+v6UvwCzJltKKbeMX7yTaIfhlQdq+DsUpZRKkt93nWbZztM82bISxfOm/AWYNdlSSiVq5d5QFm09yaAWFShdMIe/w1FKKbdFRMbw8o87qFgkF31uL+eXGHTMllIqQdeiYnjph+2UK5STgc11ULxSKm358I+9HLsQwdz+TVJ0ULwrTbaUUgn6aMV+Dp0L5+snGuugeKVUmrLvzBU+/esAj9YvSZPyBf0Wh55GVErF6+DZq3y0Yj8P1inBHZV8f0sLpZTyFmMMo3/YTo4smXixbcoPinelyZZSKk7GGF7+cTtZMwX4pXpHKaWS4/vNx1l74DwvtK5KoVxZ/RqLJltKqTj9tPUkK/ee5bnWVSiSO5u/w1FKKbddDI9k/OKd1Cudj04NS/k7HB2zpZS61eVrUby2KITaQXnp2riMv8NRSqkkeWvpbi5GRDHz4VoEBPj//q3as6WUusU7S3dzLuw64x+uRWAqaKiUUspdm45cYM66I/RuWpbqJfL4OxxAky2lVCybj1zgq7WH6XFbWWoF5fV3OEop5baoGAcjv91GsTzZePreyv4O5wY9jaiUuiEqxsHI77ZRNHc2ht2XehoqpZRyx+crD7L79BU+6xFMrqypJ8VJPZEopfzuy1UH2XXqCp90b0DubJn9HY5SSrntyLlwJi/fQ+saxbg3ld2/VU8jKqUAOHo+nPeW7eHe6kVpVaOYv8NRSim3GWMY9cM2MgUEMObB1Hf/Vk22lFI3Lv4XKMKrqbChUkqphCzccsK6VE2rKhTLm/ouVaPJllKKRVtP8ueeUIa3qkKJfNn9HY5SSrntYngkry0KoU6pfHRrkjovVaNjtpTK4C6FR/HqT9Y1tXrcVtbf4SilVJK88fMuLoRH8VWf1HupGu3ZUiqDm/DzTi6ER/L6I6m3oVJKqbis2X+OueuP0veOcqnmmlpx0WRLqQxs9f6zVkN1ZzlqltRraiml0o5rUTGM/G4rZQrm4Ol7UvelavQ0olIZlNVQbaNMwRw8k8obKqWUim3y8r0cOhfO7L6NyZ4l0N/hJEiTLaUyqPeW7eHwuXBm92tMtsypu6FSSilX249f4tO/DtAhOIimFQv5O5xE6WlEpTKg7ccv8fnKg3QMLkXTCqm/oVJKKafoGAcjvttK/hxZGNW2ur/DcYv2bCmVwUTHOBj+3RoK5MzCi22r+TscpZRKki9WHWT78ctM7VqfvDnSxp0uPO7ZEpHWIrJbRPaJyIg4ppcWkT9EZLOIbBWRtskLVSmVXDGOGHotfImj2cfzfJuSaaah8jZtv5RKmw6dvcq7v1l3umhTM+3c6cKjZEtEAoEpQBugOtBZRGL35Y0G5htj6gGdgKnJCVQplTzhUeH0XzqULZcXUiJzMI/WzZiD4rX9UiptcjgMI7/bRpbAAF57qCYiaedSNZ72bDUC9hljDhhjIoG5wEOx5jGA86IXeYETHm5LKZVMZ8LP0OuXXqw7vQrOPcysRyYQGJBhB8Vr+6VUGjRn/RHWHDjHqHbVUuUteRLi6ZitksBRl8fHgMax5hkD/CoiQ4GcwD1xrUhE+gP9AUqXLu1hOEqp+Ow+v5vBywdzPuIS4cd68GabThTJk7YaKi/zWvsF2oYplRKOX4xgwpJd3F6xIB0blvJ3OEnmac9WXH13JtbjzsB0Y0wQ0BaYKSK3bM8Y86kxJtgYE1y4cGEPw1FKxWXlsZX0+LkH0Q4HEYcHckeJZjxWv6S/w/I3r7VfoG2YUr5mjOHF77bhMIY3Hq2dpk4fOnmabB0DXFPLIG7tZn8CmA9gjFkDZAO0xlypFDJv1zyG/D6E0rlLU/jycwRGB/H6o7XSZEPlZdp+KZWGfLvpOH/uCeWF1lUpVSCHv8PxiKfJ1nqgkoiUE5EsWANIF8aa5wjQEkBEqmE1VqGeBqqUck+MI4a31r/FuH/GcUfJO7i/yDjW73cwsm1VSubL7u/wUgNtv5RKI85cvsbYn3bQsGx+ujcp4+9wPObRmC1jTLSIDAGWAoHAl8aYHSIyFthgjFkIDAM+E5FnsLroexljYnfVK6W8KDwqnBErR/DH0T/oWq0rXSsNoc2kv7mtfEE6N9TxRKDtl1JphTGGUT9s53q0g7fa1yEgIO32ynt8UVNjzBJgSaznXnb5OwS43fPQlFJJcSb8DEN/H8qu87sY0WgEXap2ode09cQ4DG8+VjtNN1Tepu2XUqnfwi0n+C3kNKPaVqNcoZz+DidZ9ArySqUDzorDy5GXef+u92leqjnz1h/hzz2hjHmgOqULps1xDkqpjOnM5Wu8/OMO6pXOR587yvk7nGTTeyMqlcatOr6KHj/3wBjDjNYzaF6qOccvRvDaop00KV+AHreV9XeISinlNmMML36/jWtRMUx8vA6B6aBXXpMtpdKwebvmMXj5YErnKc2sdrOoVrAaxhhe+GYrDmN4O42Pc1BKZTzfbz7Osp1neK5VFSoUzuXvcLxCTyMqlQbFOGJ4d+O7fBXyFc2CmvF2s7fJkdk6VTh73RFW7TvLuIdrptkyaaVUxnTq0jXGLLSqD3vfnvZPHzppsqVUGhO74vC54Odu3Hrn6Plwxi/eyR0VC9G1sVYfKqXSDmMMI77bSmSMg7fbp4/Th06abCmVhsSuOOxareuNaQ6H4flvthIgwhuP6cVLlVJpy4INx1ix2yrqKZvGqw9j02RLqTRi9/ndDPl9CJeuX7pRcehq5trDrDlwjjcerUVQfj19qJRKO6yinpB0W9SjyZZSacCq46sY/udwcmbKyYzWM6hWsNpN0/eHhjHh5520qFI4Td6kVSmVcTkchuHzt6Troh6tRlQqlZu3ax5Dlg+hVO5SNyoOXUVHXObUZx1pnGkfbz2WNm/SqpTKuNbMn0jzIx/w8gPV021Rj/ZsKZVKJVRxeMPlE5z/5GGaXN9DrgZtKJInm3+CVUqppHI4uLBwBLfv+oQ8uRtSs14xf0fkM9qzpVQqFB4VzjMrnuGrkK/oUrUL79/1/q2J1smtRH18FznCDvNZqTeo8+AQ/wSrlFJJFRmOY1538v/7CfOlFcUG/ohkyuLvqHxGe7aUSmVCw0MZ8vuQOCsOb9j9C+abPlyIzs6zmcczpUvPlA9UKaU8ceU0zOmEnNjM2KjuNOxGyyGxAAAgAElEQVT4IoXzpq/qw9g02VIqFUms4hCAfz6BX0ZwKnslHrryJBP7tCJvjswpH6xSSiXV6RCY3YGYq2cZFPUsOWs/SJvaJfwdlc9psqVUKpFYxSGOGPhlJKz7hPNB93D3vi481qQyzSoX9k/ASimVFPuWw4JemMw5GJRlHFsDy/HLgzX8HVWK0GRLqVRg3q55TFg3gUr5K/HB3R9QLGesgaLXw+DbJ2DPL1wLHsj9W1pSvHAWRrWt7p+AlVIqKTZMg8XDoEg13i7wKr/+e51ZfeuQN3vG6JXXZEspP4pdcfhWs7fImTnW2IXLJ2B2Rzi9HdN2IsP2NuDM1VN837Mx2bME+idwpZRyh8MBy16G1R9AxXv5o9abTJ2zi/7NytO0QiF/R5diNNlSyk9c73HYpWoXnmv4HJkCYn0kT261Eq3rl6HLfL67XI3F27bwXKsq1ArK65/AlVLKHZHh8H1/2PkTNOxL6B1jGf7+aqoXz8Ow+yr7O7oUpcmWUn7gVsXhnqWwoDdkzwd9fuFolgq88vVKGpUtwMDmFVI+aKWUcpddcciJzdBqAqbxQJ6fsYGw69HM7VSXrJkyVq+8JltKpbCkVBxSrBZ0nkd0zqI88+laBHi3Yx0C0+HtLJRS6YRdcUj4Oeg0C6q24+s1h/jDvsl0paK5/R1hitNkS6kU5FbF4dIX4Z+PoUpbeOxzyJKTj5bvZcPhC0zqWFdvMq2USr3sikMy54DeS6BEPfaducK4xTtpXrkwPZuW9XeEfqHJllIpxFlxWDFfRT5s+WGCFYc0GQz3vQYBgWw8fIFJy/fyYJ0SPFyvpH+CV0qpxDgrDgtXha7zIW8Q16JiGDrnX3JmzcTbj2fce7dqsqWUjyW14pC2E6FRP+vpa1E8NXczxfNmY9wjNf0QvVJKJcLhgGWvwOr3oeI90H4aZMsDwJu/7GLnyct80TOYIrkz7r1bNdlSyofCo8IZuXIkvx/93b2Kw87zoPJ9ABhjGPX9dk5eusb8AbeRJ1vGuB6NUioNca04DH4C2rwFgVYb9/uu00z7+xC9mpalZbWifg7UvzTZUspHPKk4pFitG5O+2XiMn7acYPh9lWlQJn8KRq6UUm64qeLwdWgyCOzThGcuX2P4gq1ULZabEW2q+jlQ/9NkSykf2HNhD4OXD+bS9UtMvmsyLUq1uHWmfz6FX164UXFInuI3Jh0IDeOVhTtoUr4A/2tRMeUCV0opd8RRcejkcBienb+F8MhoPuzShGyZM9ZlHuKiyZZSXuZpxaFTZLSDp+b+S5ZMAbzXsa5e5kEplbrs/x3m94TM2W9UHLr6bOUBVu07y+uP1KJikYx3mYe4aLKllBfN3z2f1/95PZGKw76w52ery/2+cRBw81Hf20t3se34JT7p3oDiebOnYPRKKZWIjdNh0bNWxWGXeZCv1E2Ttxy9yMRfd9O6RjE6NyoV9zoyIE22lPKCGEcM7218jxkhM5Jccehq+c7TfLbyIN2blKFVjWK3TFfKqw7Ogi2jIPwI5CgNdcZDuTjGFiqVQMWh06WIKIbM2USR3Nl447FaGfYyD3HRZEupZHKtOOxctTPPN3ze7YrDm2a5FMGwBVuoXjwPo9pVu2W6Ul51cBas6w8x4dbj8MPWY9CES90sgYpDJ2MMI7/bysmL15g/8Dby5cjip2BTJ022lEoGZ8XhznM7Pao4dIqOcfDknM1ERTv4sEs9HVCqfG/LqP8SLaeYcOt5TbaUUwIVh66+/ucIS7adYkSbqtQvrdXTsQV4uqCItBaR3SKyT0RGxDNPBxEJEZEdIjLb8zCVSn32XNhDlyVdOHjpIO/f/X7cidY/n1oNVcEK0Hd5nIkWwKRle1l/6AKvP1qL8oVz+Thype0X1qnDpDyvMp4zO+HzeyB0l1VxeNvgOBOtHScu8dqiEJpXLkz/O8v7IdDUz6OeLREJBKYA9wLHgPUistAYE+IyTyVgJHC7MeaCiBTxRsBKpQZ/H/+bYX8Oc6/isHIbq+Iwa9xJ1Mq9oUxZsY+OwaV4qK7ejsfXtP2y5ShtnTqM63mlEqk4dAq7Hs3Q2ZvJnyMz73aoQ4BWT8fJ056tRsA+Y8wBY0wkMBd4KNY8/YApxpgLAMaYM56HqVTqMX/3fAYvH0xQriBmtZt1a6J1PQzmdrUSrSaDrCPCeBKtM5ev8cy8f6lYOBdjHqyRAtErtP2y1BkPgbFuah6Yw3peZWwbp8PX7SFvKatHPp5EyxjD6O+3cejcVSZ3qkfBXFlTNs40xNNkqyRw1OXxMfs5V5WByiLyt4isFZHWca1IRPqLyAYR2RAaGuphOEr5XowjhonrJ/La2tdoWqIpM9rMuPXSDpdPwLQ2sHepVXHYesItl3Zwio5xMHTOZq5ej2FK1/pkz6LjtFKI19ovSMNtWLmu0OhTyFEGEOt3o091vFZG5nDAry/BT09BhbusMab54r98w5x1R/nh3xM81bIyTcoXTMFA0x5PB8jH1U9o4lh3JaAFEASsFJGaxpiLNy1kzKfApwDBwcGx16FUquBWxeGpbTCrQ4IVh67e/W0P/xw8z7sd6lC5qF74LwV5rf2CNN6GleuqyZWyRIbD9wNg58J4Kw5dbT9+iTE/7aBZ5cIMvVvvcpEYT5OtY4BruhsEnIhjnrXGmCjgoIjsxmq81nu4TaX8IjQ8lKG/DyXkXEgCFYe/wje9IWueeCsOXf2+6zRTV+ync6NSPFo/yEeRq3ho+6WUqyunYW5nOL4pwYpDp0sRUQyatYmCObMwqWNdHaflBk9PI64HKolIORHJAnQCFsaa5wfgLgARKYTVLX/A00CV8gdnxeGBSwcSqTjsCAXKQ7/4Kw6djp4P55l5W6hRIg+vPKDjtPxA2y+lnJwVh2d2Qsev4604dDLG8NyCLZy4GMGHXepTIKdeT8sdHvVsGWOiRWQIsBQIBL40xuwQkbHABmPMQnvafSISAsQAzxljznkrcKV8zbXicHrr6VQvWP3mGRwxsHQU/PNRohWHTtejYxgyexMOY5jatb5eT8sPtP1SyuZmxaGrL1Yd5NeQ04xuV40GZfR6Wu7y+KKmxpglwJJYz73s8rcBnrV/lEpTvHGPw7i8vngnW45d4uNuDShTMGei8yvf0PZLZXiJ3OMwzkUOn+eNn3fRukYxnrijnO9jTEf0CvJKuXAYB+9ueJcZITO4s+SdvN387TjucXgSZndI8B6Hcflh83FmrDlM3zvK0bqm3vdQKeUHDgcsHwN/T473HodxOXPlGoNmbSIof3beery23vcwiTTZUsoWER3ByJUjWX5kecIVh7M7wrVLblUcOu08eZkR322lUbkCvNCmqg+iV0qpRNxUcdgH2rydYMWhU1SMgyGzN3MpIooZfRqRJ1vmFAg2fdFkSyl8U3HodCkiioFfbyRPtsx82KUemQM9vkuWUkp5xrXi8L7xiQ6Ed/XGz7tYd/A8kzrWpWqxxHvB1K002VIZ3p4Lexi8fDCXrl9i8l2Tuav0XbfOtO4z+Pl5KFrTGt+Qp4Rb63Y4DM/O+5fjFyKYN6AJRXJn83L0SimViDM7rWsAhp+1Kg6r3e/2oj9tOcEXqw7Sq2lZHq6ntxPzlCZbKkPzRcWhqyl/7GP5rjO8+mANGpQp4OXolVIqEa4Vh70WQ8n6bi+65/QVXvh2K8Fl8vNi22qJL6DipcmWyrCSVHHY+H/QarxbFYdOK3af4d1le3i4bgl63FbGy9ErpVQiPKg4dLp8LYqBMzeSM2smpnStT5ZMOvwhOTTZUhmOLysOnQ6dvcqTczZTpWhuJjyqlTtKqRTkYcXhf4sbnpn7L0fOhzOrb2OK5tHhD8mlyZbKUFwrDjtV6cQLjV7wWsWh09Xr0fSfuYGAAOHT7sF6g2mlVMrxsOLQ1aRle1i+6wxjH6pBY73BtFdosqUyjLMRZxmyfAgh50J4oeELdK3W9dYeJw8rDp2MMQybv4V9Z8L4qk9jShfM4cVXoJRSCQg7A3M6eVRx6PTL9pO8//s+OgQH0b2JDn/wFk22VIaw98JeBi8fzMXrF71ecehqyh/7+GXHKUa3q8YdlQp5IXKllHKDs+LwamiSKw6ddp+6wrPzt1C3VD7GPlRThz94kSZbKt1zVhzmyJTDJxWHTst3nuad36wB8XorC6VUiol9j8MkVBw6XQqPov/MDeTMmomPuzXQ+7Z6mSZbKl1zVhxWyFeBKS2neL3i0Gl/aBhPz/2X6sXz6IB4pVTKSUbFoVOMwzB07mZOXIxgbv8mFMurA+K9TZMtlS4lueKwzdvQuL9H27oYHknfGRvIkimAT7o30AHxSinfc604rNASHp+epIpDVxOW7OSvPaG8/kgtvR6gj2iypdKdpFcczoXKrTzaVlSMg8GzN3HsQjiz+zUhKL8OiFdK+VhUBHzXP1kVh07z1x/lc/sK8V0al/ZyoMpJky2VrpyNOMvQ5UPZcW6HexWHvX+G4rU93t64RSH8ve8cb7evTcOyekSolPIxL1QcOq0/dJ5RP2zjzkqFGN1OrxDvS5psqXQjpSoOnb5ee5gZaw7T785yPB6c9HESSimVJF6oOHQ6ej6cgTM3Uip/Dj7sXJ9MgXqFeF/SZEulC64Vh9NaT6NGwRo3z+CIgV9Hw9qpyao4dFq9/yxjFu6gRZXCjGijR4RKKR/b/wfM75GsikOnq9ej6ffVBqJiHHzeM5i8OTJ7MVAVF022VJrnVsXhd/1g95JkVRw6HTx7lUGzNlG2UE7e71yPwACtPFRK+dDGGbD4WShUGbrM96ji0CnGYXhq7r/sPRPG9N4NKV/Y84NO5T5NtlSa5TAO3tv4HtN3TE+44nBOR2tAfDIqDp0uhkfSZ/p6BPiiZzB5sukRoVLKRxwOWP4q/D0p2RWHThOW7GTZztOMfagGd1Yq7J04VaI02VJpUpIqDiMuJqvi0Cky2sGAmRs5fiGCWf0aU6ZgzsQXUkopT0RFWPc4DPkRGvSGthM9rjh0+nrt4RuVhz1uK+udOJVbNNlSaU6SKw77/JKsikOw7nk48rtt/HPwPJM71dXKQ6WU79xUcTgObhviccWh0597Qnll4Q7urlqEl+6vnvgCyqs02VJpSkpXHDpNXbGfbzcd4+l7KvFQ3ZLJXp9SSsXpzC6Y9bhXKg6ddp+6wpBZm6hcNLeOM/UTTbZUmrH6+GqG/TmM7Jmyu1Fx2Boe+yJZFYdOi7ae4O2lu3m4bgmealkp2etTSqk4ebHi0Cn0ynX6TF9P9iyBfNkrmFxZ9WvfH/RdV2nCgj0LGL92vJsVhwOh1evJqjh0Wn/oPM/O30LDsvl5s73e81Ap5SNerDh0uno9mj7T13P+aiTzB9xG8bzZvRCo8oQmWypVc604vKPkHUxsPtHnFYdO+86E0XfGBoLyZ+fT7sFkzaT3PFRKeZkPKg4BomMcDJm9iR0nLvFZj2BqBeVNfqzKY5psqVTLHxWHTmeuXKPXtHVkDhRm9G5E/pxZvLJepZS6wQcVh2AV9Lz04w7+2B3K+Edq0rJaUS8Eq5JDky2VKrlVcbj3N1jQy2sVh05Xr0fzxPQNnAuLZN6AJpQqoDeXVkp5WdgZmNMZjm/0WsWh09QV+5mz7giDWlSga+MyXlmnSh5NtlSqk7SKwxrW+AYvVBzCzV3vn/cMpnZQPq+sVymlbjizC2Y/DmGh0HEmVHvAa6v+fvMx3l66m0fqleS5VlW8tl6VPJpsqVTFXxWHAA6Hg69efI+/HRUY174+d1fVrnellJfdVHG4GEo28NqqV+4N5flvttK0QkHefEwLelITvc23SjUW7FnAoOWDKJGrBLPbzb410Yq8CvO6WYlW44HQabb3Eq3r1/m9+/+47cfPGZfjCF0al/bKepVS6oaNM2BWe8gbBH2XezXR2nL0IgNmbqRikdx83L0BWTLp13tqoj1byu+SXnH4FjQe4LXtR1+4wKbufSm5L4RNrTrT+ZUhXlu3UkrdXHF4Nzw+wysVh04HQsPoPX09BXJmYUbvhnrP1lTI49RXRFqLyG4R2SciIxKYr72IGBEJ9nRbKv2KiI5g2IphTN8xnY5VOvLB3R/cmmid2g6ft4Sz+6yKQy8mWtcPHmTHw+3JemAPix8eQsd3RxMQoEeE6Z22XyrFREXAN72sRKtBb+iywKuJ1unL1+jx5ToEmPlEY4rkyea1dSvv8ahnS0QCgSnAvcAxYL2ILDTGhMSaLzfwJPBPcgNV6Y9rxeHzDZ+nW7VuCVQc5vZqxSHA1XXrODRoCOFRhgUdXuD1UZ3JFKiJVnqn7ZdKMT6sOAS4FBFFzy/XceFqJHP730a5QjkTX0j5haffLI2AfcaYA8aYSGAu8FAc870GvAVc83A7Kp3ae2EvXRZ3Yf+l/Uy+azLdq3e/NdFa9xnM7gAFykG/372aaF384QcO93mCE5KdDx4bySsvdCRbZr1oaQah7ZfyvTO7rB750zusisOmQ72aaEVExtBvxgb2h4bxcfcGetHSVM7TZKskcNTl8TH7uRtEpB5QyhizKKEViUh/EdkgIhtCQ0M9DEelJauPr6bHzz2IdkQzrfW0Wy/t4IiBX16EJcOh0n3Q+xevXdrBGEPo+x9wcsRIdhQsx9sPPsd7T7clb3Yd45CBeK39sufVNkzdbP8f8MV9EH3dqjj04qUdACKjHfxv1kbWHz7Pux3qcmelwl5dv/I+TwfIx5WemxsTRQKA94Beia3IGPMp8ClAcHCwSWR2lcYleo/DyKvwbV+v3+MQrIrDk6NGc3nRIv6s0IQZjTsyb1AziuoYh4zGa+0XaBumYrnpHofzIJ93K5tjHIZn5//Lit2hvP5ILR6o450DUeVbniZbxwDXu2QGASdcHucGagIr7FNDxYCFIvKgMWaDh9tUaZjDOJi0cRLTdkzzW8XhsSFDidi4ke8aPMiCynczr39TShfUq8NnQNp+Ke+7peJwOmTz7qk9Ywyjf9jGoq0nGdmmql6iJg3xNNlaD1QSkXLAcaAT0MU50RhzCSjkfCwiK4Dh2lBlTBHREby48kWWHVlGxyodGdFoRBz3ONxujc/y8j0Owao4PDpgIFGnTjGtZV8WF6zBrD6NqVIst9e2odIUbb+Ud/noHoeujDG88fMu5qw7yuC7KjCgeQWvrl/5lkd7gzEmWkSGAEuBQOBLY8wOERkLbDDGLPRmkCrtSg0Vh8eGPgkBAUx94FmWShGm9wymbim9DU9Gpe2X8iofVxw6ffj7Pj756wDdm5Rh+H16G560xuPU2xizBFgS67mX45m3hafbUWmX6z0OJ901ibtL333rTOs/hyXPef0ehwCXfvyRE6NfIlNQEG82H8CKy5n5uGt9mlYolPjCKl3T9kt5hQ/vcejq07/2885ve3i0XklefbCG3oYnDdIryCufWH1iNcNWJHaPw5dg7RSo1Araf+m1W+8YYzj7wYecnTqVbI0a82pwN1aeiuTDzvW4p7re71Ap5QX7/4D5PSFTVq/f49DVjNWHeH3JLtrVLs5b7WsTEKCJVlqkyZbyum/2fMO4teMon688U1tO9VvFYa5HHmF0uXb8dfAikzrWpU2t4l7ZhlIqg/NxxaHTnHVHeGXhDu6tXpRJHevqRZfTME22lNe4VhzeXvJ2JjabSK4ssXqrUqjisMBTT/Nitnr8uTuUt9rX5qG6JRNfgVJKJcThgN/Hwqr3fFZx6PTtxmO8+P02WlQpzIdd6pFZE600TZMt5RVJrjjsNAeqtPba9p0Vh9GnTlF04juMvFCE5TtOM+7hmnQILpX4CpRSKiFREfD9QAj5wa44fBsCfXMx5B//Pc5z32yhaYWCfNytAVkz6d0t0jpNtlSyuVdxuAwW9LQrDn+G4nW8tv3w9es5OmQoEhhIiS+nMTzEwa8hp3nlgep0a1LGa9tRSmVQYaEwp5PPKw4Bfth8nGfn/0ujcgX4rEew3kYsndBkSyXLvgv7GLx8MBeuX3Cv4rDzPMjrvVN6zorDLKVKUWzKVJ5ZGcqvIacZ80B1et1ezmvbUUplUClUcQj/JVqNyxXki17B5MiiX9Hphf4nlcdSS8VhjiZNKPLOuzy5eD+/hZzm1Qdr0LNpWa9sRymVgaVQxSHA95uPMWz+Fk200in9byqPuFdx2A92L/Z+xWFkJCdfHMXlRYvI+9ijFBz9EkMWbNdESynlPZu+gkXP+LziEG5OtL7s1ZDsWfTUYXqjyZZKEvcrDjvBqa3Q+k1oMtBr23etOCz8zDPk7N2HgbM28cfuUMY+VIMet5X12raUUhlQClYcAsxff5QXvttKE0200jVNtpTbIqIjGLVqFL8d/i2RisOOEHHBNxWHAwcSffIUJd99h0z33Effrzawev85Xn+klt6UVSmVPClYcQjWBUtfWbiDZpUL80m3BppopWOabCm3nI04y5O/P8n2s9t5Lvg5ulfvHk/FYS9rXJYPKg6PDRkKgYGUnjGdmGo16fHFOjYducDE9nV4rEGQ17allMqAwkJhbmc4tgHufQ2aDvVZxSHAJ3/uZ8LPu7i3elE+7FJPL++QzmmypRKVmioOS33yMeEFi9Lz83/YceIyH3SuT7vaemV4pVQyuFYcdvgKqj/os00ZY5i8fC+Tlu2lXe3iTOpYVy9YmgFosqUS5Kw4zJYpW/wVh7+9DGs+9HnFYdDkSZwPyEaPz/5h/5kwPurWgHv1XodKqeQ4sALm9UiRikNjDG/8vItP/jrAY/WDeKt9bQL1XocZgiZbKl6uFYdT7p5C8VyxepBcKw4bDYDWE7xbcThqNJd/+om8jz1K8Vde4VhYNN0/Xc3py9f5vGcwzSoX9sq2lFIZlLPisGAl6DrfpxWHMQ7DqO+3MXf9Ubo1Kc3YB2vqTaUzEE221C0cxsGkTZOYtj11VBwW7N+PfWfC6PbFP0RExvB138Y0KJPfa9tTSmUwKVxxeD06hmfm/cuSbacYendFnr238q1jXlW6psmWuklqqzjM07Yt/x69SK9p68gSGMD8gbdRtVger21PKZXBpHDF4dXr0Qz8eiMr955ldLtq9L2zvM+2pVIvTbbUDamm4jAggNIzppOjXj1W7T1L/5kbKJQrK18/0ZjSBXN4bXtKqQwmhSsOL1yNpPf09Ww7fomJj9ehvVZNZ1iabCkgKRWHz0PR6j6vOMxSqhQ/bLbufF+hcC6+6tOIInmyeW17SqkMJgUrDgGOng+n57R1HLsQwdSu9WlVo1jiC6l0S5Mt5UHF4ReQNbdXtm2M4eyHUzg7ZcqNisOAPHluXIOmSfkCfNI9mLzZfdfNr5RK51Kw4hBgx4lL9Jq2nutRMXz9RGMalSvg0+2p1E+TrQwuyRWHrV6HQO/sNjdVHD76KMXHvILJlJmxi0KY9vch2tUuzrsd6ujF/pRSnkvBikOAv/edZcDMjeTOlolZ/2tK5aLeOTBVaZsmWxmUWxWHV05ZA+FTqOLwerSDYXM2s3jbSfrcXo7R7appabRSyjMpXHEI8OO/xxm+YAvlC+Viep+GFM+b3afbU2mHJlsZUGqsODx/NZIBMzew/tAFRrWtRr9mWrGjlPLQTRWHvaDtRJ9WHBpjmPLHPib+ukeHPqg4abKVwaSqisPp08lRvx4HQsPoPX09Jy9d44PO9XigTgmvbU8plcGkcMVhZLSDF7/fxjcbj/Fw3RK82b62Dn1Qt9BkKwNxrTh87673aFm65a0zOSsOi1SHLr6vOPznwDn6z9xIpgBhTr8merFSpZTnQnfDrMch7EyKVBxeCo9i4NcbWXPgHE/fU4mnWlbSi5WqOGmylUGsObGGZ1c86/+Kw8aNCXp/MoF58/LdpmO88O1WShfIwbRejfQaWkopz7lWHPZaDEG+rTg8ci6c3tPXceR8OO91rMMj9fQaWip+mmxlAN/u+ZZxa8dRLl+5VFNx+OYvu/hoxX6aVijIR10bkDeHjm9QSnlo00xY9HSKVRyu2X+OQbM2YoCvn2hM4/IFfbo9lfZpspWOpYqKw6FDidjwX8Vh2PVonpm5gWU7z9C5UWlefbAGWTIFeG2bSqkMxOGA31+DVe9C+bugwwyfVxx+vfYwYxbuoEzBHHzesyHlCuX06fZU+qDJVjp1LfoaL656kd8O/0aHyh0Y2XhkIhWHs6FKG69tP66KwyPnwun71Xr2h15l7EM16N6kjI5vUEp5JoUrDqNiHIz9KYSZaw9zV5XCTO5cjzzZtEdeuUeTrXTIteJwePBwelTv4feKw9X7zzJo1iaMga/6NOL2ioW8tj2lVAbjh3scDpq1iTUHzjGgWXmeb12VQL0GoEoCTbbSmf0X9zNo2SD/VRwuXMiJUaNvVBxmDgris78O8MYvuyhXKCdf9AymTEHtdldKeSiFKw63H7/EgJkbCb1ynXcer8NjejNp5QGPB8uISGsR2S0i+0RkRBzTnxWREBHZKiLLRaRM8kJViVlzYg3dlnQj0hHJtFbTbk20HDGwdBQsHgYV77F6tLyUaBljCP3gQ048/wI56ten7JzZRBUpzpA5mxm/ZCf3VivKD4Nv10RLpQrafqVRB1bA5/dapxB7LfZ5orVgw1Ee/Wg1xhgWDLxNEy3lMY96tkQkEJgC3AscA9aLyEJjTIjLbJuBYGNMuIj8D3gL6JjcgFXc3Ko4/K4/7FoEjfpDqwnerTgcPZrLC/+rODxwKZKBX/zN/tAwXmhdlYHNy+v4LJUqaPuVRqVgxeH16BjG/hTCrH+O0LRCQT7oXI+CubL6bHsq/fP027YRsM8YcwBAROYCDwE3GitjzB8u868FunkapIqfwziYvGkyX27/MvGKw5NbfFtx+PTTFBzQn6U7TvPcgi1kChS+6tOYOyrp+CyVqmj7lZakcMXhiYsRDJ69ic1HLjKgeXmeu68KmQK1Ylolj6fJVkngqMvjY0DjBOZ/Avg5rgki0h/oD1C6tG+vjZLeuFVxeHoHzOpgVRx2ni0w+UoAABTZSURBVOPVisPIQ4c4MmDAjYrDbPe1ZuyiEKb9fYjaQXmZ2rU+Qfn1QqUq1fFa+wXahvlUClcc/rHrDM/M/5eoaAdTu9anba3iiS+klBs8TbbiOh9k4pxRpBsQDDSPa7ox5lPgU4Dg4OA416FuldoqDs+VrUL3T9aw5ehFejUty8i2VfX+YCq18lr7BdqG+cyNisP1Pq84jIpx8M6ve/j4z/1UK56HqV3r6/WzlFd5mmwdA0q5PA4CTsSeSUTuAUYBzY0x1z3clorFWXF4/tr5BCoOv4Alz6VIxeGKy5kZ/v5KjEGPBlVaoO1Xanej4vC0XXH4kM82dfJSBENnb2bD4Qt0blSaVx6oTrbMeqCovMvTZGs9UElEygHHgU5AF9cZRKQe8AnQ2hhzJllRqhtc73E4vfV0ahRK6B6H90H7L312j8NC77zL66tPMn31IWqWzMOULvW12lClBdp+pWYH/oR53SFTFui1xKf3OFy64xQvfLuVyGgHkzvV5aG63jsoVcqVR8mWMSZaRIYAS4FA4EtjzA4RGQtsMMYsBN4GcgEL7NNbR4wxvq3TTedSTcXhI49wedBwHp25nd2nr9D79rK80LqqHg2qNEHbr1TsRsVhRegyH/L75oobEZExvLY4hNn/HKFmyTxM7lSPCoVzJb6gUh7y+JvYGLMEWBLruZdd/r4nGXEpFzdVHJa4nYnN/VdxWOjpp/i5divGf/IPubNlYlrvhtxVpYjXtqVUStD2K5VJwYrDHScu8dTcf9l3JowBzcoz7L4qen9W5XN6BflULmkVh+d9UnF4dMBAok6eJNf4NxhxpQTLF4bQokph3m5fh8K59dozSqlkiIqAH/4HO76H+j2h3Ts+qTiMcRi+XHWQt5fuJl+OzHz9hF6WRqUcTbZSsbMRZ3nq96fYdnZb/BWH+5bB/F5WxWHvn6FEXa9t37Xi8OhLE3luhyEi8iyvPFCdXk3L6kVKlVLJc1PF4Vho+qRPKg6PnAtn+IItrDt0nvuqF+WNx2pTIGcWr29HqfhospVK7b+4n8HLB3Mu4pxfKw4DS5ZkxgNPMWtjJHWC8vJOh7pULKJjG5RSyZQCFYfGGGavO8L4xTsJFOGdx+vwaP2SeqCoUpwmW6nQmhNrGLZiGFkzZfVrxeH1mvV4unonjh13MOzeyvyvRQW9krJSKvluqjhcDEHBXt/EiYsRjPhuG3/tCeWOioV4q31tSuTL7vXtKOUOTbZSGWfFYdm8ZZnacqrfKg531W3Gc6XbUiF/Pn7oUIeaJX13ewylVAbi44pDh8Mwa90R3vx5FzEOw9iHatCtcRkCArQ3S/mPJluphNsVh3M6wYl/fVNxOGQoERs3Mq9OO2aVv4shd1diUIuKWqmjlEq+FKg43B8axshvt7Hu0HnuqFiICY/WolQBvWWY8j9NtlKBJFccdpoNVdt6bfuRhw5xsN8AIk+c5O3gbly+rQWL29emclHvnJpUSmVwPq44jIx28NnKA0xevpdsmQJ4q31tHm8QpGOzVKqhyZaf+bvi8PK69Rz632DCoxy80ex/PNC1Nb2aliVQu9yVUt7g44rDtQfO8dIP29l7Jow2NYvx6kM1KJI7m9fWr5Q3aLLlR/6uONz0+WwyvzuBUzny89sTLzCl592U1AGkSilv8WHF4dmw67y+ZCffbTpOUP7sfNEzmJbVinpt/Up5kyZbfpJ4xaEDfnvJJxWHZ65c45s3v+Cubz5kZ7HK5H3rHd5uVNEr61ZKKcBnFYfRMQ7mrD/KxKW7CY+MZlCLCgy9uxLZs+jtwlTqpcmWHyRecRgO3/WzKg4b9oPWb3il4vB6dAxfrjrEh7/vJVNkMUq06kDr10eSI6d2uSulvGjz1/DTU16vOFy97yxjF4Ww69QVmpQvwLiHa1KxiI4tVamfJlv/b+/eo6Mu7zyOv5/MJCFXAgmBXAhyCQRElEthtVUQWGXBynGrguJuPYeWVbfSruupbTnH1bq2blfraou17novWNHaLquoVEURECyIcokXbgmESyBDgNyTmXn2j4w0xARmJjO/mYHP6xyOM8mT5DOTma/f3y/Pd8ZBfuvn0Y8e5cltTwY5cfgATLqlx/sbrLWsLK/mZys+pdLTyPSR/Vk061IG532rR99XROQUp0wcToHrnoW0nB5/272eRu5fUc6b26spyknjN/PGMWP0AG2Al4ShZsshzd5mFq1ZxMrKlVw3/Dp+Muknjkwcbqk6xs9XfMYHuz2U5mfy/PyJXFrar8ffV0TkFFGYODzW2Mpj7+7imbUVuJIMd14xnO9cOoReyfqToSQWNVsOCGniMCUjIhOHlZ4G/vPNz3l1y0H6ZqRw79XnM29SiV4BXkQir6EGXrgBqj6MyMRhc5uPZ9ZV8NiqndS1eLlmbBE/vLKMAb215UESk5qtKAtq4nDjU/DanZA/MjBxWBz2z6upb+HX7+xkyYZK3ElJ3D51GAsuG0JWr8i9po2IyEkRnDj0+vz8cfN+Hv7zFxw43syUEf24a0YZIwuyIxhYxHlqtqLIyYnDow2t/Hb1Lp5bV0mrz8+crw3kB9NKyc/WkaCIRMnu92DZP4CrZxOHPr/l1S0HeOStHeyuaeDC4t48eP2FXDI0L8KBRWJDzVaUvLLjFe774L6oTxwea2zlv9/fzTNrK2hs8zH7wkIWTitlSL/MM3+xiEi4IjBx6PdbVmw7yH+9tYOdh+spG5DF4zeN58rz+2vzu5xV1GxFWMeJw0sKL+GhyQ91MXFYDS/M6dHE4eG6Zp5aU8Hv1lfS0Opl1gUF/GB6qcagRSS6/H5Y9e/w/kNhTxy2+fz83ycHePy9XXxRXU9pfiaLbxzH340eoDeMlrOSmq0ICm7isByWXg+NnrAmDvd6Gvnt6l28tKkKr8/PzAsK+N7UYZQN0J4GEYmytib4022w/RUY948w65chTRw2t/lYtnEfT6zeTVVtEyP6Z/HI3Iu4akyh3iJMzmpqtiLE0+Rh4TsLQ5g4XAGFY4P+/p/sO8aTa/bw2taDuIzhW+OL+KfLhnJeXkZkb4iISFc6ThxOvxe+/v2gz8h76ltYumEvz35QQU19K+NKcrjnm+cztSxfZ7LknKBmKwJOmTic8jDTBkVm4tDr8/Pm9mqeWruHTZW1ZKa6mf+Nwcz/xmD6a+O7iDglzInDzw6d4Ok1Ffzx4/20ev1MGdGPWycPZeLgvtqTJecUNVs9tP7geu5YdQep7lSenvE0o/NGn7ogjInDI3UtLNu4j6Ub9rL/WBODctP5t2+O4trxxXoJBxFxVogTh20+P2+VV/O7DZWs3emhV3IS108o5uZLBjMsX4M7cm5Ss9UDHScOF09bTGFm4akLQpg49Pst63Z5WPphJSu3V+P1Wy4ekss9V7efatd+BhFxXAgTh/uONvLCh3tZtrGKmvoWCnv34q4ZZdwwcSA56SkOhhaJP2q2wtB54vDByQ+SldLpbFVddeA9DjefduJw39FG/rR5Py9/VEWlp5Gc9GRuvuQ8bphUwlC9fIOIxEKQE4dNrT5Wlh/i5U1VrNlZgwGmluVz46QSJg/XQaLIl9RshSgSE4d1zW28vvUQf/ioig17jgIwaXBf/mX6cGaMHqD3/RKR2DnDxKHfb9mw5yivfFTFiq0HaWj1UZSTxu1TS5n7tYEU5qTFMLxIfFKzFQJPk4eFqxay9UjoE4d1zW28/elhXtt6kPe+OEKr18+QvAzuvGI4sy8qYmDfdOdvkIhIR91MHPr9lk17a3lty0He2HaIQyeayUx1M2tMAX8/rpiJ5/XVVKHIaajZClI4E4ceVz/e3VTFG9sPnWyw+mencuPEEq6+qJCxA3M0kSMiMbNkCSxaBHv3QklRG/dPeZB5I7bAdc/SOuJqNu7ysLK8mte3HaT6RAsp7iQmD+/Hj8eUccWoAaSl6Cy8SDDUbAUhqInDt+6Gdb+ifuDlLC25lzeWVLJ53ydYy8kG66oxBYwr6aMjQBGJuSVLYMECaGxsv15ZlcyCF+9m+7/O4fjHltXL/kxdi/dkg3XVmAKmluVrIlokDGq2zuBME4cHjnjw/+G7FB96m5eSZvCjHfPw7ajigqLefH9aKdPK+nN+YbYaLBGJK4sW/bXR+lJjWzq/eGwEY3+4jlmB5urrw/LISNX/KkR6Qs+gbnQ1cZjhzuTzQ3VsqqxlU2UtGyuPcs3x51joeoeHXDdTMezb/Kw0jykj8vWioyISd47UtbCpspaP9tZSWVkGfPUg0F+XxoYfT9MBokgEqdnqQseJw4m5Mylpu4nbni/n433HqGv2ApCbkcK4QX3ImXgXVZlzuGPcldp/JSJx42hDK9sPHGfb/hNsP3CcrfuPU+lpP5WV4koire9gmo5+9aCwpMSQlOR0WpGzW9jNljFmBvAI4AL+x1r7QKfPpwLPAeMBDzDHWlsRftToaG7zUVXbyK4jDew6Us9n1YdZ1/AAzUkVtByeydufXspq1z6G98/iqjEFjB/UlwmD+jAoN71Dc1UW09sgIqE5W+qXz285XNfMnkD9+rKO7aiu59CJ5pPrinLSGF2UzbxJJYwf1JfRRdm8PMp1yp4tgPR0uP/+GNwQkbNcWM2WMcYFLAb+FqgC/mKMWW6tLe+wbD5Qa60dZoyZC/wHMKengYPR4vXR0OKjvtnL8aY2aupbqKlvwdPQSk1dCwePN1N1rIn9tY3U1Lee8rX52W5SC/pwUcY1XHH5dEYVZjO8fxYpbh3qiZwN4r1++fyW+hYvDS1e6pq9eBpaqKlvxROoY9UnWthf28T+Y00cONaE129Pfm1Giouh+ZlcPDSXkQVZjC7szajC7C5fwX3evPb/npxGLGlvtL78uIhETrhntiYCO621uwGMMb8HZgMdi9Vs4J7A5ZeBXxtjjLXW0kMtXh9XPboGn9/S5vfj81na/JY2n5+GFi9tvu5/RK/kJAp6p1HcJ42RI/tTlJNGcd80huRlMqRfRmDS5sqeRhSR+BXT+gVw36vlrPr8MD6/xeuzeP1+vD5LY6uPpjZft1/nSjLkZaZQlJPGRQNzmDWmgKKcNAblpjMsP5MB2b1C2s4wb56aKxEnhNtsFQH7OlyvAiZ1t8Za6zXGHAdygZqOi4wxC4AFACUlJUH9cJcxlPbPxJWURHKSwZVkcLuSSHYZMlLdZAb+ZaS6ye7lJi8rlbyMVPKyUkhP0TY1kXNcxOoXhFfDCnr3YmRBNsmB2uVOMrhdhrRkF5mpyWSkutrrWC83uRmp5GWmkJuZSk5asjauiySgcDuPrp7tnY/4glmDtfYJ4AmACRMmBHXU6HYl8di88cEsFRHpLGL1C8KrYd+5dEgwy0TkLBHuRqQqYGCH68XAge7WGGPcQG/gaJg/T0QkUlS/RMRR4TZbfwFKjTGDjTEpwFxgeac1y4FvBy5fC7wTqf0OIiI9oPolIo4K68+IgT0M3wPepH10+ilr7XZjzE+Bjdba5cCTwPPGmJ20HxHOjVRoEZFwqX6JiNPC3i1urV0BrOj0sbs7XG4Grgs/mohIdKh+iYiT9OJRIiIiIlGkZktEREQkitRsiYiIiESRmi0RERGRKDLxNM1sjDkCVIbwJXl08YrOCUC5naXczgo19yBrbb9ohXFSiDXsXPn9xgvldl6iZg8ld1D1K66arVAZYzZaayfEOkeolNtZyu2sRM3ttES9n5TbWYmaGxI3ezRy68+IIiIiIlGkZktEREQkihK92Xoi1gHCpNzOUm5nJWpupyXq/aTczkrU3JC42SOeO6H3bImIiIjEu0Q/syUiIiIS19RsiYiIiERR3DdbxpgZxpjPjTE7jTE/6uLzqcaYFwOf32CMOc/5lF0LIvsdxphyY8wWY8zbxphBscjZ2Zlyd1h3rTHGGmPiYrQ3mNzGmOsD9/l2Y8xSpzN2JYjHSYkxZpUxZnPgsTIzFjk7ZXrKGHPYGLOtm88bY8yjgdu0xRgzzumM8SJRa5jql7NUv5zleA2z1sbtP8AF7AKGACnAJ8CoTmtuAx4PXJ4LvBjr3CFkvxxID1y+NR6yB5M7sC4LWA2sByYkQm6gFNgM9Alcz0+Q3E8AtwYujwIq4iD3ZcA4YFs3n58JvA4Y4G+ADbHOHMe/37irYapf8Zdb9Svi2R2tYfF+ZmsisNNau9ta2wr8Hpjdac1s4NnA5ZeBacYY42DG7pwxu7V2lbW2MXB1PVDscMauBHOfA9wH/AJodjLcaQST+7vAYmttLYC19rDDGbsSTG4LZAcu9wYOOJivS9ba1cDR0yyZDTxn260HcowxBc6kiyuJWsNUv5yl+uUwp2tYvDdbRcC+DterAh/rco211gscB3IdSXd6wWTvaD7tXXSsnTG3MWYsMNBa+6qTwc4gmPt7ODDcGLPWGLPeGDPDsXTdCyb3PcBNxpgqYAVwuzPReiTUx//ZKlFrmOqXs1S/4k9Ea5i7x3Giq6uju86vVRHMmlgIOpcx5iZgAjA5qomCc9rcxpgk4GHgZqcCBSmY+9tN+6n4KbQfhb9vjBltrT0W5WynE0zuG4BnrLUPGWMuBp4P5PZHP17Y4vV56bRErWGqX85S/Yo/EX1exvuZrSpgYIfrxXz1FOTJNcYYN+2nKU93atApwWTHGDMdWARcba1tcSjb6ZwpdxYwGnjXGFNB+9+yl8fBJtNgHyv/a61ts9buAT6nvXjFUjC55wPLAKy1HwC9aH+j1HgW1OP/HJCoNUz1y1mqX/EnsjUs1pvUzrCBzQ3sBgbz181353da88+curl0Waxzh5B9LO2bC0tjnTeU3J3Wv0t8bDAN5v6eATwbuJxH+yni3ATI/Tpwc+DyyMAT3sTBfX4e3W8uncWpm0s/jHXeOP79xl0NU/2Kv9yqX1HJ71gNi/mNDeLOmAl8EXhSLwp87Ke0H0lBe5f8ErAT+BAYEuvMIWR/C6gGPg78Wx7rzMHk7rQ2LopVkPe3AX4JlANbgbmxzhxk7lHA2kAh+xi4Ig4yvwAcBNpoPwKcD9wC3NLhvl4cuE1b4+UxEqe/37isYapf8ZVb9SviuR2tYXq7HhEREZEoivc9WyIiIiIJTc2WiIiISBSp2RIRERGJIjVbIiIiIlGkZktEREQkitRsiYiIiESRmi0RERGRKPp/wwpAJnbX9t4AAAAASUVORK5CYII=\n", 158 | "text/plain": [ 159 | "
" 160 | ] 161 | }, 162 | "metadata": { 163 | "needs_background": "light" 164 | }, 165 | "output_type": "display_data" 166 | }, 167 | { 168 | "name": "stdout", 169 | "output_type": "stream", 170 | "text": [ 171 | "To understand how this idea affects an rv X ~ Ber(p)...\n", 172 | " The secant line connects (E[X],g(E[X])) and ((1-E[X]), g(1-E[X])).\n", 173 | "The blue dot is (E[X], g(E[X]))\n", 174 | "The orange dot is (E[X], E[g(X)]) -- E[g(X)] is x2*g(x2) + x1*g(x1)\n", 175 | "The x coordinate of the orange dot is arbitrary, but if it were moved over to the right,\n", 176 | " you can see it corresponds to a y-value on the secant line.\n" 177 | ] 178 | } 179 | ], 180 | "source": [ 181 | "def g(x):\n", 182 | " return x**2\n", 183 | "x = np.linspace(0, 1, 50)\n", 184 | "gvect = np.vectorize(g)\n", 185 | "y = gvect(x)\n", 186 | "\n", 187 | "plt.figure(figsize=(10,4))\n", 188 | "plt.subplot(1, 2, 1)\n", 189 | "plt.title('convex function f(x) = x^2 and secant lines', fontsize=14)\n", 190 | "plt.plot(x, y)\n", 191 | "plt.plot([0.15, 0.85], gvect([0.15, 0.85]))\n", 192 | "plt.plot([0.1, 0.9], gvect([0.1, 0.9]))\n", 193 | "plt.plot([0.35, 0.65], gvect([0.35, 0.65]))\n", 194 | "\n", 195 | "plt.subplot(1, 2, 2)\n", 196 | "plt.title('Jensen & Bernoulli (read below)')\n", 197 | "plt.plot(x, y)\n", 198 | "plt.plot([0.15, 0.85], gvect([0.15, 0.85]))\n", 199 | "plt.plot(0.15, g(0.15), 'o', c='b')\n", 200 | "plt.plot(0.15, 0.15 * g(0.15) + 0.85 * g(0.85), 'o', c='orange')\n", 201 | "plt.show()\n", 202 | "\n", 203 | "print('To understand how this idea affects an rv X ~ Ber(p)...\\n The secant line connects (E[X],g(E[X])) and ((1-E[X]), g(1-E[X])).')\n", 204 | "print('The blue dot is (E[X], g(E[X]))')\n", 205 | "print('The orange dot is (E[X], E[g(X)]) -- E[g(X)] is x2*g(x2) + x1*g(x1)')\n", 206 | "print('The x coordinate of the orange dot is arbitrary, but if it were moved over to the right,\\n you can see it corresponds to a y-value on the secant line.')\n" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": {}, 212 | "source": [ 213 | "If $x_1 = 0$ and $x_2 = 1$, the inequality can be interpeted as follows. Let $X \\sim Ber(t)$, $0 \\leq t \\leq 1$, then the right and left hand sides of the inequality can be rewritten respectively as:\n", 214 | "\n", 215 | "- $g(t(0) + (1 - t)(1)) = g(1 - t) = g(E[X])$\n", 216 | "\n", 217 | "- $tg(x_1) + (1 - t)(x_2) = E[g(X)]$\n", 218 | "\n", 219 | "Which implies\n", 220 | "\n", 221 | "- $g(E[X]) \\leq E[g(X)]$ for any Bernoulli and convex g\n", 222 | "\n", 223 | "#### This is true for all random variables, and the reverse can be said when g is concave.\n", 224 | "\n", 225 | "- Visual/Memory aid: On the graph above the x values of the secant lines were chosen to correspond to possible Bernoulli $p$ and $1 - p$. $g(E[X])$ is always somewhere on the graph of $g$, and $E[g(x)]$ is always somewhere on the secant line dominating g." 226 | ] 227 | }, 228 | { 229 | "cell_type": "markdown", 230 | "metadata": {}, 231 | "source": [ 232 | "# Variance of an Estimator \n", 233 | "\n", 234 | "- $var(X) = E[X^2] - (E[X])^2$\n", 235 | "\n", 236 | "Estimators are R.V.s, so their variance can be calculated.\n", 237 | "\n", 238 | "Example: remember that $var(aX) = a^2 var(X)$, so...\n", 239 | "\n", 240 | "- $var(\\bar{X}_n) = var(\\frac{X_1 + ... + X_n}{n}) = \\frac{1}{n^2} * var(X_1 + ... + X_n) = \\frac{n var(X)}{n^2} = \\frac{var(X)}{n}$\n", 241 | "\n", 242 | "Other examples $(X \\sim Ber(p))$:\n", 243 | "- $var(X_1) = p(1-p)$\n", 244 | "- $var(\\bar{X}_n) = \\frac{p(1-p)}{n}$\n", 245 | "- $var(\\frac{X_1 - X_2}{2}) = \\frac{p(1-p)}{2}$\n", 246 | "- $var(\\sqrt{X_1 = 1, X_2 = 1}) = var(Ber(p^2)) = p^2(1-p^2)$" 247 | ] 248 | }, 249 | { 250 | "cell_type": "markdown", 251 | "metadata": {}, 252 | "source": [ 253 | "# Quadratic Risk \n", 254 | "\n", 255 | "$R(\\hat{\\theta}_n) = E[(\\hat{\\theta}_n - \\theta)^2]$\n", 256 | "\n", 257 | "is known as quadratic risk in classical statistics and mean squared error in bayesian statistics.\n", 258 | "\n", 259 | "- low quadratic risk means that both bias and variance are small.\n", 260 | "\n", 261 | "Take the formula, add and subtract $E[\\hat{\\theta}_n]$ inside the expectation, since the two terms cancel out and it remains the same, then expand the square.\n", 262 | "\n", 263 | "$E[(\\hat{\\theta}_n - E[\\hat{\\theta}_n] + E[\\hat{\\theta}_n] - \\theta)^2]$\n", 264 | "\n", 265 | "$= E[(\\hat{\\theta}_n - E[\\hat{\\theta}_n])^2] \\quad + \\quad E[(E[\\hat{\\theta}_n] - \\theta)^2] \\quad + \\quad 2E[(\\hat{\\theta}_n - E[\\hat{\\theta}_n])(E[\\hat{\\theta}_n] - \\theta)]$\n", 266 | "\n", 267 | "... where\n", 268 | "\n", 269 | "$E[(\\hat{\\theta}_n - E[\\hat{\\theta}_n])^2]$ is the variance of $\\hat{\\theta}_n$\n", 270 | "\n", 271 | "$E[(E[\\hat{\\theta}_n] - \\theta)^2]$ is the $bias^2$ of $\\hat{\\theta}_n$\n", 272 | "\n", 273 | "and\n", 274 | "\n", 275 | "$(\\hat{\\theta}_n - E[\\hat{\\theta}_n]) = 0 \\rightarrow 2E[(\\hat{\\theta}_n - E[\\hat{\\theta}_n])(E[\\hat{\\theta}_n] - \\theta)] = 0$\n", 276 | "\n", 277 | "so, quadratic risk = $variance$ + $bias^2$" 278 | ] 279 | }, 280 | { 281 | "cell_type": "markdown", 282 | "metadata": {}, 283 | "source": [ 284 | "# Confidence Intervals \n", 285 | "\n", 286 | "Let $(E, (\\mathbb{P}_\\theta)_{\\theta \\in \\Theta})$ be a statistical model based on observations $X_1, ..., X_n$, and assume $\\Theta \\in \\mathbb{R}$. Let $\\alpha \\in (0, 1)$.\n", 287 | "\n", 288 | "- Confidence Interval of level $1 - \\alpha$ for $\\theta$: Any random (depending on $X_1, ..., X_n$) interval $\\mathcal{I}$ whose boundaries do not depend on $\\theta$ and such that:\n", 289 | "\n", 290 | "$\\mathbb{P}[\\mathcal{I} \\ni \\theta] \\geq 1 - \\alpha, \\quad \\forall \\theta \\in \\Theta$\n", 291 | "\n", 292 | "- Confidence Interval of asymptotic level $1 - \\alpha$ for $\\theta$: Any random interval $\\mathcal{I}$ whose boundaries do not depend on $\\theta$ and such that:\n", 293 | " \n", 294 | "$\\lim_{n \\to \\infty} \\mathbb{P}[\\mathcal{I} \\ni \\theta] \\geq 1 - \\alpha, \\quad \\forall \\theta \\in \\Theta$\n", 295 | "\n", 296 | "- Note: $\\mathcal{I} \\ni \\theta$ is the same as saying $\\theta \\in \\mathcal{I}$, but the reversal is used here to emphasize the randomness of $\\mathcal{I}$. Rather than 'theta in the CI', it's 'CI contains theta'.\n", 297 | "\n", 298 | "- for a code sim / graphic representation of CI and confidence level, showing how alpha is at first set, then manifests through iterations of estimation, see 'Unit 2 Lecture 3 Notes', 'Trinity of Statistical Inference'.\n", 299 | "\n", 300 | "\n", 301 | "### CI for the Kiss Example\n", 302 | "\n", 303 | "$R_1, ... R_n \\overset{iid}{\\rightarrow} Ber(p)$, for some unknown $p \\in (0, 1)$\n", 304 | "\n", 305 | "- Statistical model: $({0, 1}, (Ber(p))_{p \\in (0, 1)})$\n", 306 | "- Estimator for p: $\\hat{p} = \\bar{R}_n$\n", 307 | "\n", 308 | "$\\sqrt{n} \\frac{\\bar{R}_n - p}{\\sqrt{p(1-p)}} \\xrightarrow[n \\to \\infty]{(d)} \\mathbb{Z} = \\mathcal{N}(0, 1)$\n", 309 | "\n", 310 | "- $\\Phi_n(x) \\approx \\Phi(x)$ when $n$ becomes large. So, for all $x > 0$:\n", 311 | "\n", 312 | "$\\mathbb{P} [|\\bar{R}_n - p| \\geq x] \\approx 2 (1 - \\Phi(\\frac{x\\sqrt{n}}{\\sqrt{p(1-p)}}))$\n", 313 | "\n", 314 | "and\n", 315 | "\n", 316 | "$\\bar{R}_n \\in [p - \\frac{q_{\\alpha/2}\\sqrt{p(1-p)}}{\\sqrt{n}}, p + \\frac{q_{\\alpha/2}\\sqrt{p(1-p)}}{\\sqrt{n}}]$\n", 317 | "\n", 318 | "\n", 319 | "### Working the formula for x\n", 320 | "\n", 321 | "$2 (1 - \\Phi(\\frac{x\\sqrt{n}}{\\sqrt{p(1-p)}})) = \\alpha$\n", 322 | "\n", 323 | "$\\frac{x\\sqrt{n}}{\\sqrt{p(1-p)}} = \\Phi^{-1}(1 - \\frac{\\alpha}{2}) = q_{\\alpha / 2}$\n", 324 | "\n", 325 | "- $x = \\frac{q_{\\alpha / 2}\\sqrt{p(1-p)}}{\\sqrt{n}}$\n", 326 | "\n", 327 | "So, \n", 328 | "\n", 329 | "- $\\lim_{n \\to \\infty} \\mathbb{P}([\\bar{R}_n - x, \\bar{R}_n + x] \\ni p) = 1 - \\alpha$\n", 330 | "\n", 331 | "#### ...which is not yet a confidence interval because this $x$ depends on the unknown $p$.\n", 332 | "\n", 333 | "### Conservative bound\n", 334 | "\n", 335 | "- $p(1-p) \\leq \\frac{1}{4}$, given $p(1-p)$ is largest when $p = 0.5$\n", 336 | "- $\\sqrt{1/4} = 1/2$\n", 337 | "\n", 338 | "giving the asymptotic confidence interval\n", 339 | "\n", 340 | "$\\mathcal{I}_{conserv} = [\\bar{R}_n - \\frac{q_{\\alpha/2}}{2\\sqrt{n}}, \\bar{R}_n + \\frac{q_{\\alpha/2}}{2\\sqrt{n}}]$\n", 341 | "\n", 342 | "because as $n \\to \\infty$, the probability p is in the conservative confidence interval is at least $1 - \\alpha$" 343 | ] 344 | }, 345 | { 346 | "cell_type": "code", 347 | "execution_count": null, 348 | "metadata": {}, 349 | "outputs": [], 350 | "source": [] 351 | }, 352 | { 353 | "cell_type": "code", 354 | "execution_count": null, 355 | "metadata": {}, 356 | "outputs": [], 357 | "source": [] 358 | }, 359 | { 360 | "cell_type": "code", 361 | "execution_count": null, 362 | "metadata": {}, 363 | "outputs": [], 364 | "source": [] 365 | }, 366 | { 367 | "cell_type": "code", 368 | "execution_count": null, 369 | "metadata": {}, 370 | "outputs": [], 371 | "source": [] 372 | }, 373 | { 374 | "cell_type": "code", 375 | "execution_count": null, 376 | "metadata": {}, 377 | "outputs": [], 378 | "source": [] 379 | }, 380 | { 381 | "cell_type": "code", 382 | "execution_count": null, 383 | "metadata": {}, 384 | "outputs": [], 385 | "source": [] 386 | } 387 | ], 388 | "metadata": { 389 | "kernelspec": { 390 | "display_name": "Python 3", 391 | "language": "python", 392 | "name": "python3" 393 | }, 394 | "language_info": { 395 | "codemirror_mode": { 396 | "name": "ipython", 397 | "version": 3 398 | }, 399 | "file_extension": ".py", 400 | "mimetype": "text/x-python", 401 | "name": "python", 402 | "nbconvert_exporter": "python", 403 | "pygments_lexer": "ipython3", 404 | "version": "3.7.0" 405 | } 406 | }, 407 | "nbformat": 4, 408 | "nbformat_minor": 2 409 | } 410 | -------------------------------------------------------------------------------- /U02 L06&07 - Hypothesis Testing, Type I & II Errors, Levels and P-values.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Unit 2 Lectures 6 and 7 - Hypothesis Testing, Type I & II Error, Levels, and P-values" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "# Hypothesis Testing\n", 15 | "\n", 16 | "Asking binary questions of data in the form of the null hypothesis and the alternative hypothesis, which [suggest no effect or reinforce the status quo], and [suggest an effect or reject the status quo], respectively. " 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "## Modeling Assumptions\n", 24 | "\n", 25 | "Assumptions simplify the model to a family/families of distributions...\n", 26 | "\n", 27 | "$X_1, ..., X_n$ i.i.d. Poisson($\\lambda$)\n", 28 | "\n", 29 | "... in disjoint hypothesis spaces. \n", 30 | "\n", 31 | "$H_0: \\lambda \\in \\Theta_0$\n", 32 | "\n", 33 | "$H_A: \\lambda \\in \\Theta_1$\n", 34 | "\n", 35 | "$\\Theta_0 \\cap \\Theta_1 = \\emptyset$\n", 36 | "\n", 37 | "We can impose assumptions based on known status quo or by logical induction. Assumptions can also be made if we only care about a certain kind of result, for example making our hypothesis one-sided if we only care about $\\lambda$ being larger than any status quo $\\lambda_0$." 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "### One-Sided Hypothesis\n", 45 | "\n", 46 | "Rejection interval is to the left or right of the parameter of interest under the null hypothesis, $H_0$. e.g.:\n", 47 | "\n", 48 | "$H_0: \\theta \\leq x$\n", 49 | "\n", 50 | "$H_A: \\theta > x$" 51 | ] 52 | }, 53 | { 54 | "cell_type": "markdown", 55 | "metadata": {}, 56 | "source": [ 57 | "### Two-Sided Hypothesis\n", 58 | "\n", 59 | "Rejection interval is a union of intervals to the left and right of the parameter of interest under the null hypothesis, $H_0$. e.g.: \n", 60 | "\n", 61 | "$H_0: \\theta = x$\n", 62 | "\n", 63 | "$H_A: \\theta \\neq x$" 64 | ] 65 | }, 66 | { 67 | "cell_type": "markdown", 68 | "metadata": {}, 69 | "source": [ 70 | "### Treatment and Control\n", 71 | "\n", 72 | "If the status quo for our specific null hypothesis isn't well-established, we need a two-sample test so we can first establish a baseline - a measurement of random noise unrelated to the effect we expect in our alternate hypothesis. This group of samples is called the control group, and they establish our null hypothesis, $H_0$. The samples that are exposed to treatment, then, are those that have expected outcome in the alternate hypothesis, $H_A$. The treatment effect is the difference between the treatment and control group. But, rather than estimating the effect here, we're only interested in answering the binary question is there a significant difference? And, what is the $level$ of this significance?" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "## The Test Statistic\n", 80 | "\n", 81 | "The test statistic seeks to answer the question if the null hypothesis is true, how likely is our parameter estimate? Where does our estimate fall on the standardized distribution of a $\\theta_0 \\in \\Theta_0$?\n", 82 | "\n", 83 | "If our hypothesis is one-sided and our null is fixed with some parameter $\\theta_0$ under $H_0$, and standard deviation $\\sigma_0$, and if $H_0$ is true,\n", 84 | "\n", 85 | "### $T = \\sqrt{n} \\frac{\\hat{\\theta}_n - \\theta_0}{\\sigma_{0}} \\xrightarrow[n \\to \\infty]{(d)} \\mathcal{N}(0, 1)$\n", 86 | "\n", 87 | "... the realization of this statistic is a realization from a standard gaussian.\n", 88 | "\n", 89 | "### $T = \\sqrt{n} |\\frac{\\hat{\\theta}_n - \\theta_0}{\\sigma_{0}}|$ \n", 90 | "\n", 91 | "... is then our two-sided test-statistic, since it gets the absolute standardized distance of $\\hat{\\theta}_n$ from $\\theta_0$.\n", 92 | "\n", 93 | "Question of note: what if $\\theta_0$ isn't fixed? If $\\sigma_{0}$ is estimated by $\\hat{\\theta}_n$, what is $\\sigma_0$ in relation to $\\hat{\\theta}_n$ asymptotically (as $n \\to \\infty$)? " 94 | ] 95 | }, 96 | { 97 | "cell_type": "markdown", 98 | "metadata": {}, 99 | "source": [ 100 | "## The Test\n", 101 | "\n", 102 | "$\\psi$ is any indicator function designed to output [1] or [0], signifying [reject $H_0$] or [fail to reject $H_0$], with input $\\{X_1, ..., X_n\\}$, the set of our sample. You can see it as converting your test statistic (how likely is this set a set of realizations if $H_0$ is true?) into an answer to the question is the distance from $\\theta_0$ large enough to reject it? Our test can be anything, but to make it a good one, we need to be thoughtful." 103 | ] 104 | }, 105 | { 106 | "cell_type": "markdown", 107 | "metadata": {}, 108 | "source": [ 109 | "## Which $\\theta_0 \\in \\Theta_0$?\n", 110 | "\n", 111 | "### Type I Error, $\\alpha_{\\psi}$\n", 112 | "\n", 113 | "The probability of a false positive, or:\n", 114 | "\n", 115 | "$\\mathbb{P}(\\psi = 1$ and $\\theta \\in \\Theta_0)$\n", 116 | "\n", 117 | "### Type II Error, $\\beta_{\\psi}$\n", 118 | "\n", 119 | "The probability of a false negative, or:\n", 120 | "\n", 121 | "$\\mathbb{P}(\\psi = 0$ and $\\theta \\in \\Theta_1)$\n", 122 | "\n", 123 | "### Choosing $\\theta_0$ based on error\n", 124 | "\n", 125 | "Consider\n", 126 | "\n", 127 | "$H_0: \\theta \\leq x$\n", 128 | "\n", 129 | "$H_A: \\theta > x$\n", 130 | "\n", 131 | "As $\\theta_0$ moves closer to $x$ from the left, its distribution moves with it, which increases $\\alpha$ since realizations overlap with realizations of distributions beyond $x$ - those with $\\theta_1 \\in \\Theta_1$ that might give $\\psi = 1$. \n", 132 | "\n", 133 | "At $\\theta_0 = x$, $\\alpha_{\\psi}$ is at its maximum. But, more importantly, the so is the...\n", 134 | "\n", 135 | "### power\n", 136 | "\n", 137 | "$\\pi_{\\psi} = inf_{\\theta_1}(1 - \\beta_{\\psi}(\\theta))$\n", 138 | "\n", 139 | "... which is the probability of rejecting a false $H_0$. It's probably easier to estimate values closer to $\\theta_0 = x$, for all $\\theta_1 \\in \\Theta_1$. This makes $x$ the ideal $\\theta_0$\n", 140 | "\n", 141 | "Note: \n", 142 | "\n", 143 | "$inf$ $(0, 2] = 0$\n", 144 | "\n", 145 | "$min$ $(0, 2] \\neq 0$\n", 146 | "\n", 147 | "... which allows us to include a $\\theta_0$ in our power calculation as if it were a $\\theta_1$\n", 148 | "\n", 149 | "## Level $\\alpha$\n", 150 | "\n", 151 | "To reduce our type I error $\\alpha_{\\psi}$, we need to put a buffer around $x$. How big? $\\alpha_{\\psi} = 0$ if $\\psi = 1$ is unattainable. All we have to do is move the rejection goalpost so far away that our estimator $\\hat{\\theta}_n$ will never match it. \n", 152 | "\n", 153 | "Our test has level $\\alpha$ if:\n", 154 | "\n", 155 | "- $\\mathbb{P}(\\psi = 1$ and $\\theta \\in \\Theta_0) \\leq \\alpha$\n", 156 | "\n", 157 | "or, more compact:\n", 158 | "\n", 159 | "- $\\alpha_{\\psi} \\leq \\alpha$\n", 160 | "\n", 161 | "So, your level $\\alpha$ as an upper bound on the Type I error you're allowing for the test to reject $H_0$.\n", 162 | "\n", 163 | "For our one-sided test from before, \n", 164 | "\n", 165 | "$\\psi$ has level $\\alpha$ if $\\psi = 1$ only when $T \\geq q_{\\alpha}$ for all $\\theta_0 \\in \\Theta_0$\n", 166 | "\n", 167 | "but, remember we already decied $\\theta_0 = x$, so we don't have to worry about all of $\\Theta_0$, which simplifies the test to\n", 168 | "\n", 169 | "$\\psi_{\\alpha} = 1$ when $T_{\\theta_0=x} \\geq q_{\\alpha}$, $\\psi = 0$ otherwise\n", 170 | "\n", 171 | "... which means we can find a $c$ such that $\\hat{\\theta}_n \\geq c$. \n", 172 | "\n", 173 | "- $f(\\hat{\\theta}_n) = T \\to f^{-1}(q_{\\alpha}) = c$\n", 174 | "\n", 175 | "and\n", 176 | "\n", 177 | "- $\\sqrt{n} \\frac{\\hat{\\theta}_n - \\theta_0}{\\sigma_{0}} = T_{\\theta_0 = x} \\geq \\sqrt{n} \\frac{c - \\theta_0}{\\sigma_{0}} = q_{\\alpha}$\n", 178 | "\n", 179 | "Note: the quantile changes if the test is two-sided. Also, mind the inequality and sign changes when $H_A: \\theta \\leq x$" 180 | ] 181 | }, 182 | { 183 | "cell_type": "markdown", 184 | "metadata": {}, 185 | "source": [ 186 | "## P-value\n", 187 | "\n", 188 | "This is the probability of a realization of a standard gaussian being at least as far from the mean (0) as $T$. It quantifies our confidence in rejecting the null - our minimum level $\\alpha$ for T - the smaller the better. \n", 189 | "\n", 190 | "Continuing with the aforementioned one-sided hypothesis, we get our p-value from the simple statistic\n", 191 | "\n", 192 | "- $1 - \\Phi(T)$" 193 | ] 194 | }, 195 | { 196 | "cell_type": "markdown", 197 | "metadata": {}, 198 | "source": [ 199 | "# Visualizations" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": 1, 205 | "metadata": {}, 206 | "outputs": [], 207 | "source": [ 208 | "import numpy as np\n", 209 | "import matplotlib.pyplot as plt\n", 210 | "import seaborn as sns\n", 211 | "import warnings\n", 212 | "from scipy.stats import norm\n", 213 | "warnings.filterwarnings('ignore')\n", 214 | "plt.style.use('bmh')" 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "execution_count": 2, 220 | "metadata": {}, 221 | "outputs": [], 222 | "source": [ 223 | "def std_normal(x):\n", 224 | " return 1/np.sqrt(2 * np.pi) * np.exp(-(x**2/2))" 225 | ] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "execution_count": 3, 230 | "metadata": {}, 231 | "outputs": [], 232 | "source": [ 233 | "# unknown theta = 0.57 (p of Bernoulli)\n", 234 | "p = 0.57\n", 235 | "\n", 236 | "# get the sample and sample mean\n", 237 | "n = 200\n", 238 | "sample = np.array([1 if np.random.rand() <= p else 0 for i in range(n)])\n", 239 | "theta_hat = np.mean(sample)\n", 240 | "\n", 241 | "# null hypothesis: theta = 0.5; alternate: theta != 0.5\n", 242 | "theta_0 = 0.5\n", 243 | "\n", 244 | "# get the test statistic\n", 245 | "tstat = abs(np.sqrt(n) * ((theta_hat - theta_0) / np.sqrt(theta_0 * (1 - theta_0))))\n", 246 | "\n", 247 | "# the assumption is that the test statistic goes to a standard normal, so we impose the normal distribution rather than\n", 248 | "# estimating it. In fact, since the unkown parameter is 0.57, not 0.5, the distribution is asymptotically divergent. As\n", 249 | "# n gets larger, so does sqrt(n), which amplifies the 0.07 difference. This is precisely the effect that gives us greater\n", 250 | "# ability to reject with larger sample size.\n", 251 | "domain = np.linspace(-4, 4, 200)\n", 252 | "snorm = std_normal(domain)" 253 | ] 254 | }, 255 | { 256 | "cell_type": "code", 257 | "execution_count": 4, 258 | "metadata": {}, 259 | "outputs": [ 260 | { 261 | "data": { 262 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEGCAYAAABxfL6kAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsvXucHFWd9//+VnX3TM/9mkxCIBIIBogCBlhBxUtAYLkaWQV3XX0MXnaD+jzqoyKCLIp4Wdj1t+Kuqz8Wd1VuwQuBYAyEWySBJJPbTO6ZpMlkMvfJXHumb+f5o6onnU5PT/d0V3f1TL1fr37NdJ1Tpz6nzqlvnzr1re8RpRQODg4ODtMHLd8CHBwcHByyi2PYHRwcHKYZjmF3cHBwmGY4ht3BwcFhmuEYdgcHB4dphmPYHRwcHKYZjmF3AEBE2kXkaxmW8QMRacqWplyQjXoXEiLyBREZsrD8x0XkWavKzzciUiwiSkRuybeWZMx4w242UrLP4Swd53YRGU0x74Ui8kfT6IyKSKuIPCMi7zDTXaa2W6eg49ci8qcESe8AfpZiGVeax2+IS/oe8P50NeUCEfmeiOzJ8TE/ICIviEi3iPhF5LCIPCkip5npZ5vn8d251JVPYgxjSETOj0uz9EdnJjHjDTswJ+Zzk7nt0phtl+RSjIjMAdYBQ8D1wCLgVmAHUG3VcZVSXUqpkQzLGFJK9WRLUyEjIhcAa4Am4IPAecBngKNAWR6lZYyIeLJQTAB4MAvlnISIuLNdZkGilHI+5gd4L6CAtyVI8wD3Az7Aj3HB/q+4PP8I7AVGgR7gJWA2cI1ZbuznPybQcCsQBjxJdLbHlTVqbq8HHgOOmBr3AF+M2e8HCXTcGlPm12Ly3gJsB0aAPmADsBjjhya+jD/FlN8Up/Ua4C9mOcfNc3JGkrrNA54C+s19XgQuiCtPYRjLv5j13Al8MEmZX0ig+Zsx9b4LeNjU1w48AGgx+wvwf4B9ZtvuBb4O6EmO+U2gNUl6cQJNe8y0hcAfTC0jZjt8PG7/jabm+4BOs7/9AvDG5NHNNukGBoFfm7qHYvKkeqx/N8tqB47E9Lenzf3agXuAx4FnU6j39zH6+bVx7TQUl/8mYCswBnQA/19cHR8HngW+inFtRgCXqflnwA/N+vcB3zHPyXfNc9YBfCfueJ8CNgEDQBfwDHBWAv235NteJbVl+RZgpw/JDfvjQCOwFDgT+IR5sfytmf4ejFHIbcB84J3A5zEMuwf4imkUGsxPxQQa3m9q+DtijEtcnjlmns+bZc02t58BfA24yNT4aQzDd5uZXgasxLgjiOooNtPGDbtZTgj4slnOecAngXPNC+NvzOO/wyyj2tzvJMMO/LV58f7YPB/nAZ+LvVDi6qUB24DNwOXmPr83L7AqM0/UsG8BrgLOMdumGyiboFwv8C/A/ph6l8bUuxfDMCw06xmOnrOYerUAN5rn4wagDbgrSV/6FIYxujJJnnebdflrU1Oduf1dGEbuHcBZZt8JA5fH7LsR44foh8Dbgesw+uOdMXm+gWGg/tY8T3dh/GDGGvZUjzWIYVTPBd5hbn8eY/Dwfowf/SfN46Vi2G8BHgWaMX8giTPswMUYhvqHGAOK683z/ou463LQPPY7gQswfog3mnX9rtmu/2AedzXGlOE5GH1RETMoAD5rtsdZwBKzjs2AK15/vu1VUluWbwF2+jCBYefEKPXMuO3fBzaa/9+GYVxKJyj7dsyRdQo6fggEzY75IsZIaGFMuouY0fYkZf0cWBXz/deYI+y4fLGG/TLzgpozQZlXmsdviNseb9g3ASvTOP/Xmcc9O2ZbiXlev25+jxr2v47J8zZz2/uTlP09zBFxgno/GbftZeC/zP8rMQz0B+LyfA5oT3I8F/Dfpq4u06D8X2BuTJ6zzfR3p3Bu1gD/FvN9I/BmXJ5HgZdivncBd8fleZa4UXGKx2oCJGbbYlP7+2K2eTFGwqka9rkYU47/aKbFG/angFfj9v84xg9Pg/k9+qPujcu3EXgjbttBYFPctr3A95LojQ6ilsTrT7Vf5+PjzLGnRnSefaeIDEU/GKObhWbaagwjcVhEfms+LK2ZysGUUt/AGMF9BmP0eivQNNmTePOh6rdFZIeI9Jga/xfGHUQ6bAJeAfaKyNMi8sXoA79UERHBuHP4cxq7nQ+0KaUORDcoY95/s5kWy7aY/4+af2eno3GCsqLlRct6J8Yd13Nxbf8TYLaIlCcqUCkVUkr9PXAa8L8xpnFWAHtE5PJkYkSkTER+LCK7RKTPPN6HOLUdJ9QtIrOAOuD1uDzrp3isTcq0bCbnYfwIb4ypsx/jrjYllFJtGHdz94pIZYIs5wOvxm17BePO7tyYbTvNY8cTf37aMZ5VxW+bFf0iIktMx4XDIjKIcZcH6V9DecUx7KmhYfxKXwJcGPNZbG5DKdVvbvsYxm37F4EDUU+WdFFK9SilnjaN/GKMC+i7k+x2J8aPzYMYo+oLMUaNaT3sUkqFMC7uD2PMb94K7BeRq9KqhFlcFvJLgu2BBPtMtT8H4r6rmLKif2/k5LZ/B8aP+nCygpVSbUqp3yil/jfGnV8HcPcken6CMd11D/AB83gvcmo7JtMtMduycaz4egrZ4ccYd0TfniA9Xn+iek3UBsEEZSXapgGYPy5rMaZMP4VxbUd/hLPxwDhnOIY9NTZjdKjTlFIH4j4t0UzmKO0lpdS3MUarfRhGEYyLUJ/KwZVSEYwRX3RkETY/8eVdgTHt8iul1FZz5LswLk9KOpTBRqXU95RS7wHexJizj5ZBsnLM0d1W4OrJjhVDM3CaiJwd3SAiJRjzwM1plJOIqZ7/HRjG4MwEbX/AbJuUUEqNAoc50Y4TnccrgF8ppVYqpbab+8S342TH6sB4oPqeuKT471M9VjOG/Rh31RSRYoy2SkfnCMbc/5eABQmO8f4EeiMYc/vZZjGG59k3lVKvKKX2YNz1FByufAsoBJRSzSLyW+BREfk68AZQjvFwp1Ip9aA5TTIX41a3G/gr8/sus5hDgEtE/hrDSPqVUqeMNETko8DNwBMYt4EK44Ht3wG/MfUoEfEBHxKRdUBAGW6Ge4GPiMj7MOY6l2OMwI7FHOIQcI2InIsxBzuglDpp5CciH8AYqbyAcau6COPWe62Z5bD59zoR+QPGs4OBBKfuPuCPIvJjjDuHIIZheVkpdTBB/ucxDOljIvIljPnX+8y0XyTInw6HgNNF5GJT//AEt+8noZTqM/X/s4i4MB48ezCmaM5XSt2VaD8R+SLGdMEfMOZ2PcBHMe6EvmNma8cYHV4tIgeAMaXUcYx2XCYiz5jp38AwMPtJjweBb5plbzaPf0VcnikdSynVJCJ/Bn4uIl/A+BG5G2MOOl3+B+MO9w4Mox3lh8CbIvID4L8wnkk8BDyilGqfwnEm4xBGH/2SiPybebwHLDiO9eR7kt9OH5J7xbgwRhb7MEZaXRiuezeb6UsxHrr1cMIl7mtxZfzM3E8xsbvjQuA/MUYkQxheBjsx3OeKYvLdYB4jwAl3x1rgdxheAt3Av2JcHHti9puF8XBsgAncHTE8C/6EMW0whmEIH8D0DDDzfBvDQyFCcnfH6zF+yEYxvDheZHJ3x5WccHdcB1wYkx59eFoX1zZJHyZjGJwnMTxgFCe7O8a30ykPmDEe7O0wz0cvhvvn7UmOdwmGwTqI4ZnUi/Hs4guc/BDydvP8hjjh7nimeZ6GzXP87XhNGFNzP4075kkPiDHuBH6M0SeHMB40foOTH1BO6Vgxfel3Zjt1AP9E6u6Ot8Rtf5+5PZG74zYmcXdMcJxE52c9cdcdxjX7y5jvt5ltNorxY3gFJ18nBfHwVEyxDg4ODg7TBGeO3cHBwWGa4Rh2BwcHh2mGY9gdHBwcphmOYXdwcHCYZjiG3cHBwWGaYQs/9pdfflkVFRXlW8aMYf9xwxNqYVXylwelrQsANbfeck0O9iTVPpBqn3LILiMjI91Lly49pXFsYdiLiopYtGjRlPb1+XzMn2+/MA521vWfB08HYNW7k9+weZ96DQD/3e/LiS67nq+ZrCvVPvB/nzPeK/rpnCMz+nxNhUy0NTY2+hJtL/ipGLfbnnH1HV3p4ehKD0dXethVF1ijreANe2VloqBw+cfRlR6OrvRwdKWHXXWBNdoK3rB3d3fnW0JCHF3p4ehKD0dXethVF1ijzRZz7Jlg119iR1d6OLrSw+66lFKMjBhL6Bqh+fNLZWXluB67MZm2aNiXkpKSlM9lSoZdRK7BiNusYwTM+cEE+W7BWPXkEqXUZnPbnRhRBsPAl5RSa1JSliKBQHxIanvg6EoPR1d62F3XyMgI1dXV2MXbLRQK4XLZcxybiraxsTH6+vooLS1NqcxJayoiOsaiuVcBrcAmEXlGKbUrLl85RkzlN2K2nYcRj/x8jBC2L4jIOUqpcErqUsDvnzTyal5wdKWHoys9CkGXXYw6QCSSctj8nJOKtnTPZSpz7JcCB5RSLcqI2/04RijNeL4L/Agj3GWUm4DHlVJjSqlDwAGzvKzR0NCQzeKyhqMrPWyna3QM7a125o0p5Fg3hLM2FskKtjtfJlFddph+iWU6eMWkc05TuTc5DTgS870VYxGJ2ANeBJyulHpWRL4Wt+/GuH1PWTuzs7OT5cuX43K5CIfDLFu2jBUrVtDe3k5paSm6rjMwMEB9fT29vb0opaivr6ejowO/3099fT1DQ0PMnj2brq4uRISamhq6urqoqKggHA4zPDxMQ0MD7e3tuN1uKisr6e7uprKykkAggN/vH0/3eDyUl5fT09NDdXU1fr+f0dHR8fTi4mK8Xi99fX3U1tYyODhIIBAYT4+mFRcXU1dXR39/P8FgcDx9sjqVlZUBWFKnlpYWjIViDP/ZZHWKLmfj8/nwer14PB76+/stqZPf76euri7n7XRSnfr6KNn7FvWHOnAdMx5oRcdJStcYOn0WfefOp3zxOZa302R1Onz4MAsWLJi8Thm2k9es/8jISNI6GUv0wuHDh6murkZEUEoRDAYREXRdJxQKoes6kUgEpRRut3s8XdM0wuHwuA2ITdc0DRFJOT0UCnH8+HFWrlzJ7bffjqZphEIhNE0bTwfG/9d1HaUUkUhkvMyHH36Yz372s3g8noTp8XX6wx/+wNlnn8073/nOpHVavXo1Z555Jueff/64ZqUUmqZNWqfh4WE0TTupnSZi0njsIvI3wNVKqdvN758ELlVKfdH8rmEshvBppdRhEXkZY+GCzSLyMLBBKfVrM+//D6xWSj0de4wNGzaoqb6gdOzYMebMmTOlfa3Ezro+12is07zqukleUPqusWiR/+7P5kRXPs+X1tqB+9nX0Lr6AAhqwpGyItCE4kCIuSMnlsoMnbeA4NWXQVlJvuTm7Hyl2gduMF9Q+s93dTBnzhxGRkaYPXuqa4tnzuHDh7n55pvZts1YzzoYDKY1aj/77LPZuHEjdXWprYz3mc98huuuu46PfvSjaedLVVtHRwclJSf3ucbGxi1Lly69OD5vKiP2VuD0mO/zMFZaiVKOMQR82bxVaACeEZEbU9g3Y8rLEy4Sn3ccXemRT1365l24//Q6ohSdJR5+Na+MP9cWE9CFBjTaiVA3FubG9mE+fnSI4l0tyJF2gh/7cN7CLTjtmJy77rqLgwcPsmTJEq688koeeOABHnzwQZ566inGxsa4+eab+c53vsPw8DC33XYbra2tRCIRvvWtb9HR0UFbWxtXXnkldXV1vPDCCyeV/a1vfYtnn30WXde56qqruPnmm3n22Wd57bXX+P73v8+TTz7JSy+9xC9/+UsCgQBnn302jz76KNu2bTsl31lnnYWuT2kp5KSkYtg3AQtF5EzgKMbD0E9EE5VS/cQs+Bo3YvcDvxWRhzAeni7EWCYta/T09Ex6W5IP7KwL8jfSnIi8nC+lcK3bhPv17QA8c3ol/zavlIB+Yi7zHM1Ne2SM7iKdR+ZXsHp2CXft6+OdAyPIo6sI3no1kQWnzC5ajp37V7yu0NcfsuRYrh99ZcK0+++/n+bmZrZs2QLA6tWr2b9/Pxs2bEApxUc+8hFee+01urq6mDNnDs888wwA/f39VFZW8pOf/IQXXnjhlBF7b28vf/zjH2lqakJEOH78OFVVVVx//fUnjcSrqqq4/fbbAbjnnnt45JFHuOOOO07JB4ZXjMfjyeq5mfThqVIqhLHI7BpgN/CkMhZ3vs8clSfbtxljncldGGtorsimRwxAdXV1NovLGo6u9MiHLtcrW3C/vp2wCD9eVMuD88tOMuoALSp00vf2Yhf/Z3Edz88qQQuHcT2xBs13jFzjtGN6vPTSS7zwwgtcfPHFXHLJJezdu5f9+/ezePFi1q1bx5133sn69esnfT+goqKCoqIiPve5z/H73//+lKmRKM3NzXzgAx/gwgsv5LHHHmPXrl0J8wF5G7GjlFoNrI7bds8EeT8Q9/1+4P4p6psUv99PRUWFVcVPGTvrAvuN9HJ9vvRte3G/tpWIwA/OreXPNYndyWrQaOXksUhIE364sIqIwHUdI6gn1qCWfwRVm7uXhuzcv+J1JRtZ54pIJMLXv/51Pve5z52S9sYbb/D8889z1113cdVVV/Htb397wnJcLhcbNmxg3bp1PPHEE/zsZz9j7dq1p+Rbvnw5K1eu5IILLuBXv/oVr7zySlJt2TbuBR9SYHR0dPJMecDRlR651KW1duB+bj0AP11YM6FRB6iSxJeIEuGfz67itZpiXGNB9Mf/BGO5e2nIacfklJeXMzg4OP596dKlPProowwNDQFw9OhROjs7aWtro6SkhL/927/lK1/5Clu3bgWgrKzspP2jDA0N0d/fz7XXXstDDz3E9u3bEx5vcHCQOXPmEAwGeeyxxybUBSfeLM0mBW/Y7e7PazdmvK7RMdy/W4dEIqyeV8HTs7xJszdGJjbWERHuP6ealhIX7t4B9NXrs612QmZ8O05CbW0tl19+ORdeeCHf+MY3uPbaa7n11lt573vfy4UXXsjHP/5xBgcHaWpq4rLLLmPJkiU88MAD3HnnnQDcfvvt3HDDDVx55ZUnlTs4OMhNN93ERRddxIc+9CH++Z//GYCPfexjPPTQQ1x88cUcPHiQe++9l/e85z1cc801vP3tbx/fPz4fWONjP6m7Yy7IxN3RrnGW7azrjiYzHruN3B1zdb7cv1uHq/kghyqKWb64hrCW/KWPK7QiXo2MJc1zxkiQX2zrojiiCHzkg4QXn51NyQnJWTz2NN0df7rYiMeeb3fHeAKBQNYfUGaLVLWl4+5Y8CP24uLifEtIiKMrPXKhS9tzGFfzQQK6xl0LKyc16gDH1eSve79V4uanZ5rz66vXw7D1r/vP5HacCnZ7EzYWK7QVvGH3epPfSucLR1d6WK5rLIDnT68D8MiZlRz1phYQqpfUYoysaihhU1URnrEg2poNU5aZKjO2HaeIptnX1Fmhzb61TZG+vr58S0iIoys9rNblfnkLMjjMwcpinpiduvFZIClGBBThobOqGNOgqPkg2qGjU1SaGjO1HadK2GaxfmKxQlvBG/ba2tp8S0iIoys9rNQl3cfRNzcTEfj+meVE0rj13RcJTp7JpM3r4r9PN1z9Is//BSyMKDgT2zET7BqyF6zRVvCGPZFLkh1wdKWHlbrcL2xEIooX51RwoCy9B2hztfT8i588rYxjRTrenn60rXvT2jcdZmI7ZoIzYi8w7L7ggN2Yabq0w23o+48w6tJ4eF76oRTK0rxEAprw87cZo3b10psQDE2yx9SYae2YKXbw/psIx489AXbxm43H0ZUeluhSCtdLmwB4al4FfZ703+5L5sc+ES/VedlT5qbYH0DbMvGr5Jkwo9pxilRVVY3/ny1f8dgyY1mzZg3nn38+ixYt4kc/+lHCPGNjY3ziE59g0aJFXH755Rw+fBgwXpYqLy9nyZIlLFmyhH/8x3/MWGfBG/b29vZ8S0iIoys9rNClHTiC3trJkEfnNw1T89Z4lzYF32cRHj3DiHIYXr/VklH7TGrHbBAMpv6sJF3C4TBf+tKXWLVqFTt27ODxxx9PGBvmkUceoaqqij179vDlL3+Zb33rW4ARBOyss85iy5YtbNmyhZ/97GcZayp4w25X9ypHV3pkXZdSuF9tBOCxeRX4XVPr6j1TjFm3obqYPWVuSvwBsGDUPmPaMUv867/+K+9+97u56KKL+Kd/+icA7rzzTv793/99PM99993Hv/zLvwDw4IMPnpJ/It58803OOussFixYgMfj4eMf/zirVq06Jd+qVav45Cc/CcBHP/pR1q1bN77IRrax76PiFLHr22SOrvTIethS3zG0ti6GPDpPzZ76SzNDTHH+U4RfnV7OA7t7Ueu3IkvOA3f2LrdCascPPN5vybFevjW1oGtr167l4MGDp4Ts/djHPsZXv/pV/uEf/gGAlStX8uyzz7J27dqEIX7f9773JSy/ra2NefPmjX8/7bTTePPNU6OTt7W1cfrpxlvfLpeLyspKM4w2HDp0iIsvvpiKigruu+8+3vve96Z1LuIp+BF7f781nSZTHF3pkW1dLjPG+u/nljOmT72bz0/Vjz0Br9cUs7fUGLVne659prRjNli7dm3CkL0XXXQRXV1dtLW1sX37dqqqqjjjjDMmzD8RiR5+JnqbdKJ8s2bNoqWlhc2bN/PjH/+YT37ykwwMDGRU54Ifsae6dFWucXSlRzZ1SXsP+sFWxnSNJzMYrQPsTsOP/VQhxlz7A7t7Ca/filxyPmQpPGshtWOqI2urUErx9a9/nc9//vOnpC1btoynn36ajo4OPvaxj52UP1GI30ScdtpptLa2jn8/evQoc+fOTZjvyJEjzJs3j1AoRH9/PzU1NSilxleeWrJkCQsWLGDfvn1cfPEpIWBSJqWhjIhcIyJ7ReSAiHwzQfoXRGSniGwTkfUicp65/W0i4je3bxOR/5iy0gmw4wgBHF3pkk1drg07AFgzp5QBd2aGdH6afuzxvF5TTEuJyxi1N7dkVFYsM6Eds8WHP/zhhCF7wYi2+OSTT/K73/1ufFWjZPkTcckll3DgwAEOHTpEIBDgiSee4Prrrz8l3/XXX8///M//APD000/zwQ9+EBGhvb193Je9paWFAwcOsGDBglP2T4dJR+wiogMPA1dhrGG6SUSeUUrF3lv+Vin1H2b+G4GHgGvMtINKqQszUpkEK592Z4KjKz2ypUuOD6I3HyQs8OuGzJcALMl0tlKElXPL+PqB44y+vhXPO86GLAR9mu7tmE2uuuoqmpqaxuety8rK+NWvfsWsWbM4//zzGRwcZO7cueOLg1911VXs3r07Yf5EuFwufvKTn3DdddcRDof59Kc/zfnnnw/Avffey5IlS7jhhhv4zGc+w6c//WkWLVpEdXU1v/nNbwBYv349999/P7quo+s6Dz/8MDU1NRnVedKwvSJyGXCvUupq8/udAEqpBybIfxvw90qpa0XkbcCzSqnFyY6RSdjesbExioomXighX9hZ1y0vGD69dgrbm63z5V7zOq43m3m1oZy7z858haEyZOoPUE08YcVTm9qpCkUY+/SNRE7PPJxtrvpXumF7V14ZpKioyHZhe5VSto3wmKq2bIftPQ04EvO91dx2EiKyQkQOAj8CvhSTdKaIbBWRV0Qk8WPlDLCr36yjKz2yomssgL5tHwD/NSc7bndT8mOPI6ALq8y7h9G/bM24PJjm7WgBdryTiGKFtlQenib6KTllCKOUehh4WEQ+AXwb+BRwDDhDKdUjIkuAP4jI+Uqpkx75dnZ2snz5clwuF+FwmGXLlrFixQra29spLS1F13UGBgaor6+nt7cXpRT19fV0dHQQiUTo6elhaGiI2bNn09XVhYhQU1NDV1cXFRUVhMNhhoeHaWhooL29HbfbTWVlJd3d3VRWVhIIBPD7/ePpHo+H8vJyenp6qK6uxu/3Mzo6Op5eXFyM1+ulr6+P2tpaBgcHCQQC4+lRX16fz0ddXR39/f0Eg8Hx9MnqFF3l3Yo6jYyMjJ93n8+XtE4LYvJ5vV48Hg/9/f2W1EkpRXd3d0bt5NqyG28gyP4qL/PKyyhTEXqJsEBc7IsEmavplKHRGAnwLs1DjwozhGK+uNgdCTJf0ymJSe9UYQTFFVoRTZEgCzUXLqApEuQCzcMx08d9juhsjwRYrLkJAfsjIRZrblpVCA/CLNFZO6ecTxwdovzAEY63d9I55s+o742MjDA0NJSw72WznaI/jyMjI0n7Hpx447SzsxMRQSlFMBhERNB1nVAohK7rRCIRlFK43e7xdE3TCIfD4zYgNl3TNEQk5fRQyHghLPq/pmkopQgEAgnTdV1HKUUkEjlJU6zmydIzqRMYxn2yOg0PD6Np2kntNKHRtmAqRgP6lFKnPAoXkZeBrymlNsduz2Qqpq+vz5Yro9tZ19+/bjSNnaZiMj5fSlH0n79D6+zle2+vYW19dkbsC0SnZYovKcVzz55elnb7GX73YrSrLsuorFz1r3SnYv778n6qq6ttNxUTDoezvmB0tkhVW7anYjYBC0XkTBHxALcCz8RmEJGFMV+vA/ab2+vNh6+IyAJgIZA91wDI2N/TKhxd6ZGpLq21E62zl0GPzsu12VvFZ14GfuzxPHWaMcrSG/dkHGbA7u1ot6Bb0yG6YzrndFLDrpQKAXcAa4DdwJNKqWYRuc/0gAG4Q0SaRWQb8BWMaRiAK4AdIrIdWAl8QSnVm7K6FKivr89mcVnD0ZUemerSzReA/tRQSjCFJe9SpSkTP/Y4dpd7jOBggRDankMZlVUI7Tg2lnyt2FxS6PHY0z2XKdVWKbUaWB237Z6Y/788wX5PA0+npShNent7T7k9sQN21gWnvjyRbzI6X8N+9F0tRICVs7Ibq2Sh5uKNKUR4nIhnZ5eyaOg4/jd3UvSOhZPvMAF27l8lJSWUlJSMr6ZkB2+U4eFhSktL8y0jIZNpi47U02lv+/6MpYjdbvmiOLrSIxNdru37kHCEbXUltBdnt0tn+wJ5sd7LikP9VLX1MNrTj6qd2luZdm9HEbGVIe3p6bHtXY4V2go+VoxdG8vRlR5T1qUU+tY9ADw5K3tz61GyORX1HbgJAAAgAElEQVQDMOLSeLnOuKsINU49fsy0a0eLsasusEZbwRv2jo6OfEtIiKMrPaaqSzvSjtY7wPEiFxurs2/YL8iCH3s8z5o+7bJtL4Snti7qdGtHq7GrLrBGW8Eb9sn8OfOFoys9pqpL3268kPTC7NK0FqlOlWNZcnWMpancw2Gvi9LRINr+t6ZUxnRrR6uxqy6wRlvBG3aHGUwgiL7L8C75Q539wjdMiAjPmaN2/6adeRbjMB0peMMejcBmNxxd6TEVXfqew0ggyP7KYo6UZGdNy3jmiDUvtayZVUJQoOJwOwwOp73/dGrHXGBXXWCNtoI37HZ6uy0WR1d6TEVXdBrm2frsz61H2Z5FV8dY+t06G2qKjQtw58SLOEzEdGrHXGBXXWCNtoI37F1dXfmWkBBHV3qkq0uOD6IfbiOoCS/UWbfO5mLNmjsBgD/Xm4HBTK+edJgu7Zgr7KoLrNFW8IbdDi8/JMLRlR7p6tJ3GKPcN+pLGZriQtWpkNmL/8nZWFPMoC5U9w4inem9kD1d2jFX2FUXWKOt4A17pgHprcLRlR5p6VJq3LD/3uKHpvsj1pn2oCa8ZAYrC25Lb9Q+Ldoxh9hVF1ijreANu11vsRxd6ZGOLmnrQusboL/IRWOVtYbdyqkYODEdw879kMbbpNOhHXOJXXWBMxWTkIqKzFfJsQJHV3qko8vVfBCAV+tLLPFdj6VVWTkZAzsrPBwr0ikfCaD5jqW833Rox1xiV11gjbaCN+x2Dcfp6EqPlHVFIui7jMjPq2us9133JFxnJouI8OdZ5kPUNEIMFHw75hi76gJrtBW8YR8eTt8HOBc4utIjVV3akQ5kcIRur5td5dZOkwDMssiPPZbooiDFe30px2kv9HbMNXbVBdZoK3jD3tDQMHmmPODoSo9UdelNBwBYV18COfB0aLTIjz2WIyVu9pS5KQpF0A6kFmKg0Nsx19hVF1ijreANu10Xz3V0pUdKusIR9N1GCIHna3MTQiAbi1mnwjrTF394296U8hd0O+YBu+oCa7SlZNhF5BoR2SsiB0TkmwnSvyAiO0Vkm4isF5HzYtLuNPfbKyJXZ1M8YC6kaz8cXemRii7t0FHEP8bRMg8tJblZSmCEqUVfTJdoKN+yQ20QmDxUcCG3Yz6wqy6wRtukht1cs/Rh4FrgPOC2WMNt8lul1DuUUhcCPwIeMvc9D2ON1POBa4CfRddAzRaVlVNbqMBqHF3pkYou3fSGebHOm5NpGABfJDcP3TqKXTSXu/GEI0gKER8LuR3zgV11gTXaUhmxXwocUEq1KKUCwOPATbEZlFKxK+uWAlGH3JuAx5VSY0qpQ8ABs7ys0d3dnc3isoajKz0m1RUMoe85DMCfsrhY9WSca7Efeyzr6gzvmKEUXlYq2HbME3bVBdZoS8WwnwYcifneam47CRFZISIHMUbsX0pn30yw6y+xoys9JtOlHTiCBIIcqijiqDd3Kzr6LPZjj+UVczqm8vAxGEv+0LZQ2zFf2FUXWKMtlSsk0T3vKa/IKaUeBh4WkU8A3wY+leq+nZ2dLF++HJfLRTgcZtmyZaxYsYL29nZKS0vRdZ2BgQHq6+vp7e1FKUV9fT0dHR2Ew2HC4TBDQ0PMnj2brq4uRISamhq6urqoqKggHA4zPDxMQ0MD7e3tuN1uKisr6e7uprKykkAggN/vH0/3eDyUl5fT09NDdXU1fr+f0dHR8fTi4mK8Xi99fX3U1tYyODhIIBAYT/d6vYyMjNDf309dXR39/f0Eg8Hx9MnqFA28b0Wd2traAOOFCJ/Pl7ROC8z28fl8eL1ePB6PZXWKRCKEQqEJ6zTfHMXuqyujDOFdmochIrRFwpyjuWlRIWrQqBKNxkiAd2kejqsIvURYIC72RYLM1XTKOJHeo8IMoZgvLnZHgszXdEpi0jtVmDmiMV9cNEWCLNRcuDCWy7tA84wvwjFHdLZHAizW3IQwwhAs1ty0qhAehFmij5c5QgRfJMy5mhufClGGUGumn+stoaWiiAUDY3S+vgXXkvMn7HttbW24XK6EfS+b7RQNrzYyMpK074Hh2XH8+HFLrqdM69Tf309/f7+tbES0TpqmMTIyMqV2mtBoT7YorohcBtyrlLra/H4ngFLqgQnya0CfUqoyPq+IrDHL2hC7z4YNG9SiRYuS6pgIn8/H/Pnzp7SvldhZ1x1NpwOw6rrkN2ze7/4CAP/dn82JrgnP11iA4od+jYTC3HJJA11F1vuWR7lCK+LVyFjOjresbYgvt/TT/bYGSj95w4T5ctW/Uu0DNzxnPGT+6eIjtu33dtQFmWlrbGzcsnTp0ovjt6cyFbMJWCgiZ4qIB+Nh6DOxGURkYczX64BogOlngFtFpEhEzgQWAm9OpQITYVf/VEdXeiTTpe97CwmF2VPtzalRh9z4scfySq2XCFD1VkfS6ZhCbMd8YlddkCc/dqVUCLgDWAPsBp5USjWLyH0icqOZ7Q4RaRaRbcBXMKZhUEo1A08Cu4A/ASuUyu4iknb1T3V0pUcyXfpuI4TA2hz5rseSKz/2KD1FOtsrPLgiCjEfFieiENsxn9hVF1ijLaWnUEqp1cDquG33xPz/5ST73g/cP1WBk+Hx5PbCSxVHV3pMqCsQRDvYCsBL1bnzhokylCM/9lhervdy0UCAoe17KL3gnIR5Cq4d84xddYE12gr+zdPy8vJ8S0iIoys9JtKlHTiChMLsryymJ8fTMABtOfJjj+WVWi9hoOqtThhNPL9faO2Yb+yqC6zRVvCGvaenJ98SEuLoSo+JdEV919flYRoG4Jwc+rFH6fPobKsswqUUYoZQiKfQ2jHf2FUXWKOt4A17dXV1viUkxNGVHgl1hcLo5luYL9XkfhoGoCWHfuyxRFdWGt6ROHZMQbWjDbCrLrBGW8Ebdr/fn28JCXF0pUciXdqho0ggiK+8iGPFuXspKZaaPF0ir9UUEwYqj3TB6KneMYXUjnbArrrAGm0Fb9hHR0fzLSEhjq70SKRL32NMQ7ycgwU1JqJK8nOJHPfo7KjwGNMx+32npBdSO9oBu+oCa7QVvGG3q3+qoys9TtEViaDvNQzaizmMDRNPrv3YY3ktGsp3575T0gqmHW2CXXWBE489IXb1T3V0pUe8Ls13DPGPcazUgy+HsWHiybUfeyyvmj9o5YfbT1lZqVDa0S7YVRfkMR67nSkuzt9oLhmOrvSI1xX1hnmlpjhnIXoTcVzl3o89SleRi11lZijfg0dOSiuUdrQLdtUF1mgreMPu9Xonz5QHHF3pcZIupdD3HgbgxTy5OUbpzcMLSrG8Gp2O2XHydExBtKONsKsusEZbwRv2vr6+fEtIiKMrPWJ1ydFOZHCEnmIX+0rzu/LNAsnfNBDAa+Z0TGnLUYhZzb4Q2tFO2FUXWKOt4A17bW1tviUkxNGVHrG6otMwr9XlZsHqZOyLTL5MnZW0et20lLgoDoaRQ23j2wuhHe2EXXWBNdoK3rAPDg7mW0JCHF3pMa5LqXE3xxer8x/fY66W+zAG8bxSa9yqj8R4x9i+HW2GXXWBNdoK3rAHAvlzR0uGoys9orqkoxetb5B+j4umivwb9jIbXCLRefai/UcgYsz5270d7YZddYE12vLfazPErv6pjq70iOqKjtZfrysmkudpGMivH3uUlhIXrcU6pWNBtCMdgP3b0W7YVRc4fuwJsat/qqMrPaK6ovPrL1Tn1xsmSj792McR4TVzOsbfZEzH2L0d7YZddYHjx54Qu7oxObrSw+v1It3H0br6GHFpbKu0h2Hvye66MFPmVdOw63sOg1K2bkc7YlddkEd3RxG5RkT2isgBEflmgvSviMguEdkhIi+KyPyYtLCIbDM/z8Tvmyl2DaDv6EoPj8cz7rv+Rl0JIS3/0zAAQ6euvZ4Xdpe76fRolI8EkGPdtm5HO2JXXZCnhTZERAceBq4FzgNuE5Hz4rJtBS5WSr0TWAn8KCbNr5S60PzcSJbp7+/PdpFZwdGVHv39/ehm7PG1NvCGiTI/z37sUZQI681Re6D5gK3b0Y7YVRdYoy2VEfulwAGlVItSKgA8DtwUm0Ep9ZJSasT8uhGYl12ZE1NXV5erQ6WFoys9Znm8aMe6GdM1NlXZ5/Xv3Xn2Y48lOh0TaT5InU39su3av+yqC6zRlophPw2IDVTRam6biOXA8zHfi0Vks4hsFJGbp6AxKXb9JXZ0pUdk534AttR6Cej2mIYBmG8DP/YoOyo9HHdpVA36GfEdzbechNi1f9lVF1ijLZX7zERXWcKJRxH5O+Bi4P0xm89QSrWJyAJgnYjsVEodjN2vs7OT5cuX43K5CIfDLFu2jBUrVtDe3k5paSm6rjMwMEB9fT29vb0opaivr6ejowO/309RURFDQ0PMnj2brq4uRISamhq6urqoqKggHA4zPDxMQ0MD7e3tuN1uKisr6e7uprKykkAggN/vH0/3eDyUl5fT09NDdXU1fr+f0dHR8fTi4mK8Xi99fX3U1tYyODhIIBAYT/d6vQwODuLz+airq6O/v59gMDiePlmdysrKACypU3d3N2C4V/l8vqR1WmC2j8/nw+v14vF46O/vt6RO88wFq1trSilDeJfmYYQIvkiYczU3PhWiDKFWdBojAd6leRgiQlskzDmamxYVogaNKtHG04+rCL1EWCAu9kWCzNV0yjiR3qPCDKGYLy52R4LM13RKYtI7VZjT0CnRNJoiQRZqLlxAUyTIBZqHY+aD1Tmisz0SYLHmJgTsj4RYrLlpVSE8CLNiNGdUJ72I3bUlXNYxBM0HGZp/WsK+l812ij7WGxkZSdr3on1qcHCQzs7OrF9PmdZpYGCAYDBoKxsRrVMgEKCtrW1K7TSh0VYq+cMhEbkMuFcpdbX5/U4ApdQDcfmuBP4NeL9SqnOCsh4FnlVKrYzdvmHDBrVo0aKkOiZibGyMoiJ7eFDEYmddt7xgxF9ZdV3yGzbvd38BgP/uz1oramiE4n/5DWFNuOHSBkZc9nHWKkNs8wAV4K96R/nRrh76qssovuM2y4+Xah+44TnjxamVVwZt2+/tqAsy09bY2Lhl6dKlF8dvT+UK2gQsFJEzRcQD3Aqc5N0iIhcBPwdujDXqIlItIkXm/3XAe4BdU6rBBNjVP9XRlTr6Ph8C7Kjx2sqog0382GNorCpiWBeq+4aQvoF8yzkFO/YvsK8uyJMfu1IqBNwBrAF2A08qpZpF5D4RiXq5/BgoA56Kc2s8F9gsItuBl4AfKKWyathLS0uzWVzWcHSlTvSlJDt5w0TptIkfe5SgJmwwF/aO7G7Js5pTsWP/AvvqAmu0peTLpZRaDayO23ZPzP9XTrDf68A7MhE4Gbpun4dbsTi6UsQ/hnboKBGBv9TYxxsmSsBG0zBRXq31cmWXn9Gd+/FefmG+5ZyE7fqXiV11gTXa7HXfOwUGBux3OwqOrlTR97+FRBRHqkrod9vv4ptnEz/2WN6sLiKoCTWdx2FwZPIdcojd+lcUu+oCa7QVvGGvr6/Pt4SEOLpSIxr064Uaez7YarKRH3sUv67RaPr6i3n+7ILd+lcUu+oCa7QVvGHv7e3Nt4SEOLpSIBBEM90cj9Uld9/KFws1+43YAQ7VG/OyI037JsmZW2zVv2Kwqy6wRlvBG/bJ3DXzhaNrcrQDR5BQmP1VxYwU2dOA2lMV7K8pISRQebQb/KP5ljOOnfpXLHbVBdZoK3jDbtdbLEfX5ES9YV6uLrbllAfYcyoGYJMeYWtlEboCba8v33LGsVP/isWuusCZiklIR0dHviUkxNE1CaEw+v63AHixtogLbOYvHsXOuqKxY0aa9udZzQls07/isKsusEZbwRv2yV6tzReOruRoLa1IIMhb5UUcK3aNv55vN+ysa31tMRGg3NcOAXvcWdilf8VjV11gjbaCN+wOhUl0GuaVWvv5rhcKvR6dpgoProhCM+9+HBxgGhj2oaGhfEtIiKMrCeEIujkvvNZ0c5wj9vNhB/vres38YfTbZDrGFv0rAXbVBdZoK3jDPnv27HxLSIija2I0XxsyOsaxUg++EiMg2XYbLBqdCLvris6ze1uOQiiUT0mAPfpXIuyqC6zRVvCGvaurK98SEuLompjoSkmv1p1Y63Gx5s6XnKTYXVd7sYt9pW6KQhG0lvzHaLdD/0qEXXWBNdoK3rCL2GdRhlgcXRMQiZmGiQn6lf+xZmIKQder5nTMmA2mY/LevybArrrAGm0Fb9hramryLSEhjq7EaEc6kGE/XSVu9peeGA3vj9jThBaCruidj3v/EYhE8iUJyH//mgi76gJrtBW8YbfrLZajKzHR2DDr60ogZqRi9ykPuxGry+d14fO68AZCaL5jeVSV//41EXbVBc5UTEIqKiryLSEhjq4EKIVmujmuiYu93qrsOTIuCF0i494xY00H8qTIwOn36WOFtoI37OGwPV8gcXSdirR1oQ0M01fsZk/ZySNhT8KldfNPoeiKesdoew9DHuOiOP0+fazQlpJhF5FrRGSviBwQkW8mSP+KiOwSkR0i8qKIzI9J+5SI7Dc/n8qmeIDh4eFsF5kVHF2nEvWGeb2uBBX3wGiWTf3FC0XX3jI37UU6Zf4AcjThksM5wen36WOFtkkNu4jowMPAtcB5wG0icl5ctq3AxUqpdwIrgR+Z+9YA3wH+CrgU+I6IVGdPPjQ0NGSzuKzh6IpDqfH59T9Vnzpv3WhTf/GC0SUy7h0Taj6YB0UGTr9PHyu0pTJivxQ4oJRqUUoFgMeBm2IzKKVeUkpFl3LZCMwz/78aWKuU6lVK9QFrgWuyI93ArovUOrpORjp60foGGfS4aKo4NbCW3RaNjlJIul4zp2Miuw/mbTrG6ffpY4W2VMJNnwYcifneijECn4jlwPNJ9j0tfofOzk6WL1+Oy+UiHA6zbNkyVqxYQXt7O6Wlpei6zsDAAPX19fT29qKUor6+no6ODgKBAD09PQwNDTF79my6uroQEWpqaujq6qKiooJwOMzw8DANDQ20t7fjdruprKyku7ubyspKAoEAfr9/PN3j8VBeXk5PTw/V1dX4/X5GR0fH04uLi/F6vfT19VFbW8vg4CCBQGA83ev1EgqF8Pl81NXV0d/fTzAYHE+frE7RoEBW1GlwcHD8vPt8vqR1WhCTz+v14vF46O/vn1Kd5jb7KAYO1JVSLhqLNTchDLe9xZqbEmCRuJglOo2RAO/SPIwQwRcJc67mxqdClCHUxqQPEaEtEuYczU2LClGDRpVo4+nHVYReIiwQF/siQeZqOmWcSO9RYYZQzBcXuyNB5ms6JTHpnSpMCXCFVkRTJMhCzYULI5TvBZpnPEDYHNHZHgmcUqdWFcKDWFKnuaLRgHZSnaqryhlw91IxOMqh7c2ULzwz474XfYVsZGQkad8DY9QZCoXo7OzM+vWUSd8rKysjEAjg8/lsZSOidVJK0dbWNqV2mgiZLMi7iPwNcLVS6nbz+yeBS5VSX0yQ9++AO4D3K6XGROT/AkVKqe+Z6XcDI0qpB2P327Bhg1q0aFFSHRMxPDxsyxXI7azr1peNy3XVdclv2Lzf/QUA/rs/m/Fxi/79KbTu43z1/Fo2V58a+KsejS7y64OdiELT9dUDfdzYPsLQ5e9EX5ps/JUaqfaBG54ztDz+Ab9t+70ddUFm2hobG7csXbr04vjtqUzFtAKnx3yfB7TFZxKRK4G7gBuVUmPp7JsJ3d3d2Swuazi6TiBdfWjdxxl262ytTLy26bk29RcvNF2vmNMxoeYDeZmOcfp9+lihLRXDvglYKCJniogHuBV4JjaDiFwE/BzDqMc+kl8DfFhEqs2Hph82t2WNysrKbBaXNRxdJ4g+NH2zroSwlth90GdTf/FC07W1qojjLo2q/hGksy/Hqpx+PxWs0DapYVdKhTCmV9YAu4EnlVLNInKfiNxoZvsxUAY8JSLbROQZc99e4LsYPw6bgPvMbVkjELCn14Kj6wT67sMArEngDROlzKb+4oWmKyzCq3XGVFc4D7FjnH6fPlZoS2mtXqXUamB13LZ7Yv6/Msm+jwCPTFXgZPj9fquKzghHl4H0DaB19DDq0thcNfGiGrWigw1Hx4Wo66U6Lze2jxBuOoD2oUtPCt1gNU6/Tx8rtBX8m6d29U91dBnou1oA2FxbQnCCaRgoIH9xm5BM1/bKIvrcGhUDI0hHVm+QJ8Xp9+mTLz92W2NX/1RHl4HebBj21TXJ/cELyV/cDiTTFRYZDzEQznHsGKffp48V2gresHs89rzwHF0g3cfROnrwuzTeTODiGMuQDV0KoXB1vWSG8o007c+pd4zT79PHCm0Fb9jLy8vzLSEhjq4T0zAb60uTTsMAtEXsGaSpUHVtr/TQ69YoH/Qj7T05UuX0+6lghbaCN+w9PbnrtOkw43Uphd5kxCx5bpJpGIBzbOovXqi6IiK8Uhedjsmdd8yM7/dTwAptBW/Yq6uzGlMsa8x0XdLZi9ZznCGPTuMELyXF0mJDzxMobF0npmNy97LSTO/3U8EKbQVv2O3qxjTTdUUfmr5eXzrhS0mx1Ni0Kxayrp0VHnrcGuVDo8ix3Lx5OdP7/VRw3B0TMDo6mm8JCZnRupRC32VOw1Sn9mCoSuzZFQtZV+x0TGhnbqZjZnS/nyJWaLNnr00Du/qnzmRdcqwbrW+Q/iIXOypTM+yF6C+eT1LVFZ2OUTmKHTOT+/1UcfzYE2BX/9SZrEs3F3p4rb6USIpvPRaiv3g+SVXXzgoPnR6N8uExtFbrV1aayf1+qjh+7AkoLk7uH50vZqwupcbdHFfXpO5RclzZ01+80HUpEV6sLwEgsG23lZKAGdzvM8AKbQVv2L1e7+SZ8sBM1aUd6UAbGKbH66a5PPXRbq9NXwSaDrrWmobdtesQWLyo80zt95lghbaCN+x9fbkPTZoKM1VXdBrmlbqStIJPLZCU4tHlnOmg62Cpi5YSF95ACO1gq4WqZm6/zwQrtBW8Ya+trc23hITMSF2RCPpuI/b687XpzU3viwStUJQx00KXyPiofWyrtdMxM7LfZ4gV2gresMeu4WknZqIu7WArMuynvdTDvtL03ticq+kWqcqM6aLrhXrjdt97oBXGrPP0mYn9PlOs0JaSYReRa0Rkr4gcEJFvJki/QkQaRSQkIrfEpYXNxTfGF+DIJnYNoD8Tdek7jUiCf56V3jQMQJlNxxjTRVdnsYutFR7cEYVm3lVZwUzs95lihbZJe4eI6MDDwLXAecBtInJeXLa3gE8Dv01QhF8pdaH5uTFBekbY1T91xukaC6DvPQzAc7WThxCIp9D9xXPNVHS9MMuYjhm1cDpmxvX7LJAvP/ZLgQNKqRalVAB4HLgpNoNS6rBSagfk3oXArv6pM02XvvsQEgqzp9pLe3H6DxwL3V8810xF18t1XgIC5a1dMDhsgaqZ1++zQb782E8DjsR8bzW3pUqxiGwWkY0icnNa6lLArm5MM02Xbr6yvrpuaj65Pcqe4XGnk64hl8aGmmI0QHZaswDHTOv32cAKbakMrRJNlqbzbvIZSqk2EVkArBORnUqpg7EZOjs7Wb58OS6Xi3A4zLJly1ixYgXt7e2Ulpai6zoDAwPU19fT29uLUor6+no6OjoQEXp6ehgaGmL27Nl0dXUhItTU1NDV1UVFRQXhcJjh4WEaGhpob2/H7XZTWVlJd3c3lZWVBAIB/H7/eLrH46G8vJyenh6qq6vx+/2Mjo6OpxcXF+P1eunr66O2tpbBwUECgcB4utfrJRAI4PP5qKuro7+/n2AwOJ4+WZ3KysoALKmT4VpVB4DP50tapwVm+/h8PrxeLx6Ph/7+/lPq1H3wMGcdPkZIE/z1FdQQYaHmwgU0RYJcoHk4ZhqiOaKzPRJgseYmBOyPhFisuQmgWISLWaLTGAnwLs3DCBF8kTDnam58KkQZQm1M+hAR2iJhztHctKgQNWhUiTaeflxF6CXCAnGxLxJkrqZTxon0HhVmCMV8cbE7EmS+plMSk96pwpSLxhVSRFMkmHadWlUID2JNnTQXPeFI2nXaNauc9/eMEtjcRPDChZP2vajJGRkZSdr3wJhOCAQCdHZ2Zv16mqjvpXo9jY6O4vP5bGUjonUqKiqira1tSjZiIkRNEj9CRC4D7lVKXW1+vxNAKfVAgryPAs8qpVZOUFbC9A0bNqhFixYl1TERPp+P+fPnT2lfK7GzrjuaTgdg1XXJb9i83/0FAP67P5s0n+sv23Cv28TGWWV845zKKem6Qivi1cjYlPa1kummyx1R/P6NY5SHFaOf/yhqVk3S/Kn2gRueM2Zhf7r4iG37vR11QWbaGhsbtyxduvTi+O2pTMVsAhaKyJki4gFuBVLybhGRahEpMv+vA94D7Epd9uTU1dVls7isMWN0KYW+w5iG+WOavuux7Lapv/h00xXUhJejC3Bs35tNScAM6vdZxAptkxp2pVQIuANYA+wGnlRKNYvIfSJyI4CIXCIircDfAD8XkWZz93OBzSKyHXgJ+IFSKquGvb+/P5vFZY2ZokvautC6jzPo0Sdd1zQZ823qLz4ddf3Z9I5h+z6IZNffYab0+2xihbaU3BeUUquB1XHb7on5fxMwL8F+rwPvyFBjUoJBe46oZoou1zZj1LdudimhFBbUmIgSm/qLT0ddOyo8HCnWOd0fYOxgK5GFZ2RN10zp99nECm327LVpYFf/1BmhKxgajw3zuyl6w0SZTv7iuSAjXSKsnl0KwNjm5kkyp8eM6PdZxonHngC7+qfOBF36rhZkLMjBymIOpxlCIJ7p5C+eCzLV9afZJYSA0oNHYTh7S7PNhH6fbZx47AkoLS3Nt4SEzARd+vZ9AKyalbkfbqdN/cWnq65e85mIrhSyY1+WVM2Mfp9trNBW8IZd1+35cGu665LefnTfMQK6xprazBcKCKT1akTumM66nmswF+DYvCtry+ZN935vBVZoK3jDPknTQDcAAB5kSURBVDAwkG8JCZnuuvRtxijv9VmljLgy70bzbBr3fDrr2lBdTK9bo+L4ENLWlQVV07/fW4EV2gresNfX1+dbQkKmta5IBJd5+/50XfoBvxLRZFN/8emsK6wJa0zXx+CmpozLg2ne7y3CCm0Fb9h7e3vzLSEh01mXts+HDI7QXuphR0V2Hi4u1Ow5Mp7uup6bbRj2ol2HYDRzD6Dp3O+twgptBW/YJwuJkC+msy7XFiPs6x8b0o+7PmGZWSkl+0x3XUdK3DRWevCEI2jbM3+IOp37vVVYoa3gDbtdb7Gmqy7p6UdvOUpAF56pz15Uuuk85WEF2dT1hzlGQKnAmzszfog6Xfu9lThTMQno6OjIt4SETFddeqMxWl8/u4yhLDw0jXKBTf3FZ4Ku9TXFdHs0yo8Pob2VmU/1dO33VmKFtoI37JOFr8wX01JXMITL9IZ5vD5zF8dYjtnUX3wm6AprwnPmm6jBjTsyKmta9nuLsUJbwRt2h9yh72pBRsc4VFHM3nJ7jmQdpsaqhlJCgHf/ERgaybcchwwpeMM+NDSUbwkJmY66XJuNwJxPN2R/xZc5Ys8XSGaKrq4inQ01xpuobJn6mqjTsd9bjRXaCt6wz549O98SEjLddElrB1pbFyNunT9n4U3TeLbbNNjWTNL1hznGdIza1AzhqU31TLd+nwus0Fbwhr2rKztvzGWb6abL9YbxAsvzc8oY07PfbRZrmQURs4qZpGtzVRGHSlyU+MfQmlumVMZ06/e5wAptBW/YJUt+1NlmOumS44Pouw8RFuGx2dYsChyypNTMmVG6RHhqrvEgL/j61im5Pk6nfp8rrNCWkmEXkWtEZK+IHBCRbyZIv0JEGkUkJCK3xKV9SkT2m59PZUt4lJqa5Gs25ovppEvf1IwoxYbZZXQVWTPnvD9iTxM603StrS+hz61R1tU/JdfH6dTvc4UV2iY17CKiAw8D1wLnAbeJyHlx2d4CPg38Nm7fGuA7wF8BlwLfEZHqzGWfwK63WNNJl6txDwC/asj+3HqUmTTlkQ2s0hXQhT80mK6Pf9ma9v7Tqd/ninxNxVwKHFBKtSilAsDjwE2xGZRSh5VSO4D4BRSvBtYqpXqVUn3AWuCaLOgep6KiIpvFZY3ppEsCQfZUe9lXZp2LY6uy58h4Jur645xSxgRKDh5Ne9/p1O9zhRXaUjHspwFHYr63mttSIZN9UyI8xaf3VjPddP16TkmWlZyMB3vOgc5EXX0enbWzpraq6nTr97nACm2pxBJK1INSfaqS0r6dnZ0sX74cl8tFOBxm2bJlrFixgvb2dkpLS9F1nYGBAerr6+nt7UUpRX19PR0dHfj9fkSEoaEhZs+eTVdXFyJCTU0NXV1dVFRUEA6HGR4epqGhgfb2dtxuN5WVlXR3d1NZWUkgEMDv94+nezweysvL6enpobq6Gr/fz+jo6Hh6cXExXq+Xvr4+amtrGRwcJBAIjKdH04aHh6mrq6O/v59gMDiePlmdom+iWVGntrY2wJjT8/l8Seu0wGyfvhIPfTWlzBeYLy52R4LM13RK0GiMBHiX5qFThQmgmCcumiJBFmouXBgxTS7QPONvSs4Rne2RAIs1NyGMueLFmptqs6vMEn28zBEi+CJhztXc+FSIMoTamPQhIrRFwpyjuWlRIWrQqJITmo6rCL1EWCAu9kWCzNV0ymI096gwQ6ikdXq7uJgl+pTq1KpCeBBL6nSJ5uF4ODKlOqXSTofm1UDHiReVRkZGkvY9MNbt7OnpIRKJZP168ng89Pf3T/l66u7uZnh42FY2IlqnQCBAIBCYko2Y0PBOFllMRC4D7lVKXW1+vxNAKfVAgryPAs8qpVaa328DPqCU+rz5/efAy0qpx2L327Bhg1q0aFFSHRMxNjZGUVF2YoJnEzvruuUFY3521XVJxmRK4f3eLwF48JwansnC8nfJKEMYsuFqRTNZ1/d29fC+3lEA/Hd/NmneG54zZmFXXhm0bb+3oy7ITFtjY+OWpUuXXhy/PZW7rU3AQhE5U0Q8wK3AMykedw3wYRGpNh+aftjcljXsukhtoevSdh8a/391nXUPTaNM10WjrSIXun59evmJLyOjKe1T6P0+H+RlMWulVAi4A8Mg7waeVEo1i8h9InIjgIhcIiKtwN8APxeRZnPfXuC7GD8Om4D7zG1Zw7gNtB8FrUsp3OtPeESENOvnmUdOee5uD2ayrj0x8YC0N3amtE9B9/s8YYW2lOL1K6VWA6vjtt0T8/8mYN4E+z4CPJKBxqRUVlZaVXRGFLIubf9baB25XXHGF7Hnwy1Hl4G80QSXXQDFye8UCrnf5wsrtBX8m6fd3d35lpCQgtWlFO7123IjJoZzbeov7ugy8ARDaCmsi1qw/T6PWKGt4A27XX+JC1WXdrAV7WgnQ57cRjX02dRf3NEVw+vbJ10XtVD7fT5xRuwJCATsGX2vIHUphXvdmwA8MS+3L3SU2dRf3NFlsK3CQ1EghPb69qT5CrLf5xkrtBW8Yff7/fmWkJBC1KU3HUTr6KWv2M3jFgX7moham8Y9d3QZ/OfbjB96beOOpAtxFGK/zzdWaCt4w97Q0JBvCQkpOF3hMK5XNgPw6BnlBPTcjggbbRr33NFl0FxRxGs1xbjDEeTVxgnzFVy/twFWaCt4w25X/9RC06Vv3YvWN8ixUg+rsryeaSrMZH/xqZAPXb+cX0EYcDfu+X/tnXmQHNV5wH9f9xx7arU7u+hghSR0CxkBdmGwE0IsbOPEgYrKLuNKXKSiuMqJ7DghODbGRRxiYlxOjIPtxOWAr4AtDoEsQCDEJcAsCBBISEgIXSt07a52V3tor5npL39M72q1mj1amtl+s7xf1dZOd7/p+fV09zevX7/+HtLanrVMoR33JhBKP3bTicXMPPEKyqsvSdSvhf1sZjnpEHJXdxraX9x6nWR/aZQnzynBVUX9ezFDKajj3hDy4Vbwgb28vHz0QiFQSF6RF99ETnSzp6KIZyrDeez6sKH9xa3XqfzivHJ6BYrf3odzsOG05YV03JtCPtwKPrA3NzeHrZCVQvGSljYiL28F4D9nlkNII83MN7S/uPU6lYaiCPfVZgKRPvYCeKdeORTKcW8S+XAr+MBeWZnTcTtyRqF4RZ+sQ9IeG6eVs31SeJerew3tL269Tufe2jIa4i5Fja24b75zyrJCOe5NIh9uBR/YTe3GVAhezrsHcN99j56Iww9nlIZoBVWGHorW63R6XIf/8bs/8tQr0H0yQVghHPemYbs7ZqGnZ2xZ58abQvCKPlkHwD0zK2gZ5ydNhzJZzDwUrVd2nq0uZnNFjFhvEueZVwfmF8Jxbxr5cDPzqA2Aqf1TC8HLaWnncFmc34zzw0jZsP3FgxG6lwh3nj+ZFBB9Y+fA7EI47k3D9mPPgqn9UwvBS4HbZ5eTHoe0vKNh+4sHwwSvfaVR7qstwxk03kchHPemYfuxZ6GoaPwfphkLxnpFTwaEdbUVbKkwY1SZ42pmf3HrNTK/PG8S9cUns38be9wb6gX5cSv4wF5cHH4zQjZM9arevn/g9Z0z8jtAdRBaDH0QyHqNTJ8jfG/e5IHp8rbh88iEiannI+THbUyBXUSuFpF3RGS3iHwjy/K4iNznL39FRGb582eJSLeIvOn//TS3+tDa2prrVeYEE73kaDOlm7YPTPe45vyuny9jGvNl3LFeo7N90smrvuInXoKkeV1ETTwf+8mH26hntoi4wE+ATwGLgc+LyOIhxVYArao6F7gD+N6gZXtU9SL/70s58h4gkUjkepU5wTivviSx1U8jnnkDMwPs8pJhK2TFegWjpK2LiN/byiSMOx8HkQ+3sVTZLgV2q+peVe0DVgHXDilzLfAr//WDwDKR8XmEsaOjYzw+JjBGeakSXfciTksbjWVmtKkPZbpjZnpc6xWMlGSShLnb94StcgpGnY9DyIfbWAL7ucB7g6YP+vOylvEHv24D+n+GZovIGyKyUUT+8Cx9T8PUBPomeblbdhF5azd9rsOqhTVh62SlzNDbPdYrGI/PyZz2ziPPIy3ZM0CGgUnn41Dy4TaWhrpsNe+h1/PDlTkCnKeqzSLyQWCNiFygqqfs8cbGRlasWEEkEiGdTrN8+XJWrlzJ0aNHKS0txXVd2tvbqampoaWlBVWlpqaGhoYGioqKaG5uprOzkylTptDU1ISIUFVVRVNTE5MmTSKdTnPixAmmTp3K0aNHiUajVFRUcOzYMSoqKujr66O7u3tgeSwWo7y8nObmZiorK+nu7qanp2dgeVFREcXFxbS2tpJIJOjo6KCvr29geXFxMSUlJdTX11NdXU1bWxvJZHJg+WjbVFZWBpCTbZruxomvexGAX8+toq0kBp2Z7/0KJ85mr49LnBjH1aMFj/Mlwi4veUqN8AonTrOm6USZKRF2eElmOi4lOAPvb9Q0fSi1EmGbl2SeEyECbPOSLHViHNFM0qpp4rLF62OJEyUFvOulWOJEadY0CyXCOeIOrLMLj3ovzSInSr2mKENIDFreicdhL818J8peTVGFw2RxRtymskHOY9mmZk1zhRM/o206qCliSF62KQZMxTmjbQqyn/qpwhlxm/rHVXp0aim1rd1c3NxF+oEnqb/qEqIlxWd9PsViMdra2s74fCoqKqK+vt6oGNG/TZMmTeLw4cNnFCOGQ1RHbnMVkcuBb6vqJ/3pmwBU9buDyqz3y9SJSAQ4CtTokJWLyHPAjar62uD5dXV1unDhwhE9hqO+vp6ZM2ee0XvziRFenV3Ef/47nLZONk4t55a5k7jCiVPX+ccAJEueGPHtG188BMAf/cHQC7Tcc4UT53mvN++fE5T3u9dYj4Fo19UAXF72LJv7urnrjUam9aZJLp5Navmy0JLL9WPE+TgMZ+O2efPm15ctW/ahofPHcj33KjBPRGaLSAy4Dlg7pMxa4Hr/9WeAZ1RVRaTGv/mKiJwPzAP2ntEWDIOp3ZhC90qliN+/Aaetkz0VRXxndiYjX7OamYbWegXDZK/OiMM3FyfocoXo2/uIjDDi0ngR+vk4AqF0d/TbzL8MrAd2APer6nYRuVVErvGL3Q0kRGQ3cAPQ3yXyCmCriGwhc1P1S6rakssNMDWBfqheqkTXPo9zqJGW4ig3Lpg8MNRd52mtaGZgvYJhutfe0ijfXlCVGXHp+c24b+0O1cvUOAEhDrShqutUdb6qzlHV2/x5t6jqWv91j6p+VlXnquqlqrrXn79aVS9Q1aWqeomqPpLrDWhra8v1KnNCaF6qRJ7eRGT7HnojDl9bWHlKgq+ZBvV/Hoz1CkYheL1SVcRPzq8AwF27EWfvobC0jI0TkB83M2+tB6C6ujpshayE5RXZuJlo3VbSIty6sIrdpacOyLDD0P7P1isYheK1elopq6eV4noekfvW49QfCcXL1DgB+XEr+MBu6i9xGF6RF98g+sJmPIHbFyV4cfLpfdZnGtr/2XoFo2C8RPjR+RU8NqUEN5Um8tsnsg6pl29MjRNga+xZSSbNrLmMq5cqkRfeIPrsayjwgwXVPFmV/UGkEkN3ufUKRiF5qQj/MXcyG2qKcZMpIvc+jnNgfGvupsYJyI+bmUdHAEzNszxuXp5H9ImXiD6XCep3LkjwSPXwT5eGnsd7GKxXMArNyxPhu/Mreba6GLcvSfSedTg79o2bl6lxAmw+9qyYmmd5XLxSKWKrnyby2tukHOHfF1fzUM3IKUBNyOOdDesVjEL0Sotw64JK1kwtxUl7xB58Cve1t8fFy9Q4AflxM/PWegBKS8Mdq3M48u0lxzuIPfgUzpFjdEddvrWoitfGMBh1o6H9n61XMArVyxPhjjkVNMVdvljfTuzx35NqbCH5icshkr/7BqbGCciPW8EHdtc18yZSPr2cdw8QW/Mc0tPLseIoX1tYyd4hvV+Go8/Q/s/WKxgF7SXCPTPKORZzuHH3caKv70AON5H8zFXo5PK8eJkaJyA/bgXfFNPebk6iocHkxSuZIrLhZeKr1iM9vWyuLuX6pdVjDuoAtYb2f7ZewZgIXk9MKeXvltZwJO7iHjlG7H8fwt22B0ZJc3ImmBonID9uBR/Ya2rMzFaYay/nwBHiP1tN9OW38AT+b3YlNyyooDMSbBduM7T/s/UKxkTx2lUW44sXncNLlXGcnj5iDz9D7P4N0JHbkZhMjROQH7eCD+wtLTnNUJAzcubV2UX00ReI/+pRnJZ2DpXFWXlhDXedW4KeQWKleY6ZNT3rFYyJ5NURdbhpcYLvz51Mlyu4u+qJ//QB3E3bIJ2bewmmxgnIj5uZR0cARstOGRZn7ZVMEanbSuSlLUgyRVqE1TMr+Nn0EpLOmWfKM3WHW69gTDgvER6dWsorlXFu3H2cy1p7ia2vw9u0neRVl+ItmHVWGSJNjROQHzdTj48xY+ol1hl7dfcSeW07kU3bka4eAF6vKeOHM0o4UDL2tvThmCiX8OOF9QrG2Xo1xSN8fXGCj7T08Lf72zmvtZ34A0/hTUmQ+shS0otngxO8ocHUOAG2KSYrDQ3j/3jyWAjqJQ3NRJ/4PUV3/pboc68jXT3sqSjiH5dUc8OCipwEdYClhvZ/tl7BmNBeIryUKOavLj6HO+ZU0BJzcRqaiT38DPH/vp9I3VY40R1olabGCciPW8HX2EcbSSQsxuTVcQJ3534iW9/FOdw0MPutRAm/mFbC6xWxnA9QcMTQ/s/WKxjvB6+0I6yZVsa6KaV8srGLzx/q5NzWDpynXiHyzCa8+bNILZmDN3cGREcOZabGCciPW8EH9oLC85CGZty9h3B21eMebBxY1BV12TillAeqi9hTlpvaucUyEehzhEemlvLYlBIua+nhzxq6+HBrD+7Ofbg796HRCN6cGaTnzcCbfS5aYW4QHy8KPrB3dnaSSCRGLzjOdHZ2kigpwznShHPY/ztwFOk+OZxZ0hHeTJSwvjLOxkTRwGAY+WSauOzP+6cEZ5q4vKupsDVOw3oFI59ent9E81KimERvmo83dXFlcw+LOvoGgjyAVzUJb8ZUvOk1eNNr0HOqjI0TkJ8YNqbALiJXA/8FuMBdqnr7kOVx4NfAB4Fm4HOqut9fdhOwAkgDf6+q63NmD0yZMiWXqwuO58GJbpyWdqSlDWluw2luY15jC+7xjtOKNxdHeaOyiBcmRXmlMk63O763ObYYmjzKegXj/e7VHHdZVVvOqtpyanpTfLS5h0vb+ri4rZeSlnaclnbYsgsAdRzm1kyGmj1oVQWamIyXqEAnl0FRPPTxWPMRw0YN7P6YpT8BPg4cBF4VkbWqOjh7zwqgVVXnish1wPeAz4nIYjJjpF4ATAeeEpH5qrlriGtqamLGjBlntxJVSHuQTEEqhSRTkEwhvUno7kV6epHuHujyX5/oRjq6oOME0tGFDNNdKekI9ZOK2FkWZUuJy/byGIeK3FAPpCVOlE2hffrwLHGi/N7AYGW9ghGGV1M8wprpZayZDq4q8zuTLOroY3FnkkWdSaZ3JYk0tEDD6f3F1XXR8pLMX1kJlJegJcVoUQyK4qf+j0XRaCST0ybinlHvnKz+uYhhQxhLjf1SYHf/cHcisgq4Fhgc2K8Fvu2/fhD4sYiIP3+VqvYC+/wxUS8F6nIiv+Flph1tIlb0TqbmrB546v95mYDtecjgef3z02kkmc4E82Rq2OA8FjpiLg3FUQ4WR9gfd9lX7FJRUsyjRUr6LPqc5wPzLt4zWK9gWK/spEXYUR5jR3mMh/x5JSmPT3dDR1c3td0pZvd4zOhOUd2boiSVRo53QJar69FQx8kE+GgE7Q/2ERfEAUdQRzKVOCczjeNkpgfNU8ehvLoMQgjs5wLvDZo+CHx4uDKqmhKRNiDhz395yHvPHfoBjY2NrFixgkgkQjqdZvny5axcuZKjR49SWlqK67q0t7dTU1NDS0sLqkpNTQ0lO/ZS3nYiwOYOT1qgx3XwHKHPFbocAdelOSKkoi6pqMs+FyZFoxyLOhyOO5TE42yNQcKJUCbCVk1xoURoV48O4EpxeFdT1OJSPGj5MfXoQ5kuLjs1xfm4uMBO0lwgERrUA2CKOGzXFAtxSQN7SbNQIhzWNDGEanEG1tmtykHSzJMI72maMoTKQcs7VWkgTbWcTDi0LFI8sLxNPVpRZonLHk0xhVPLtapHJ8oMvw0119vUrh5L3fgZbdMcibBf01QiVAxanm2bBu+nsWyT52//eO+n0bapWlxmSfSMtinIfurnXDc64ja96pdLiXCJGx/3/TTiNrkeB+JQWlnKI0P20wfSDt29fST6UtT2eXT09DEnBdFUGk151KQVSaYpS3sUpzKVxCJPcT3F8Tzo86AvydlU32JL53L48GGSySRTp04dNe41NDSM2pNGRnvqSUQ+C3xSVf/Gn/4CcKmqfmVQme1+mYP+9B4yNfNbgTpVvceffzewTlVXD/6Muro6XbhwYcCvA5xd9Rw+eIi+stKBX0TxfwnFcXFcB/Ff40rmEXwn818jLl4kgkZdvBxeVvXTduQoFdPMS+7fduQo33pnDgA/unLkH8VFP1gFwI4brhsXL1O/r/ez11iPga88l0k9+50Fe94f35cqkvaQdBpJpXFSmf+S9lsIVBG/hcDzPLy032qgadRTNH2yhcGNRZh+yQfOSGPz5s2vL1u27END54+lxn4QGHydUAscHqbMQRGJABVAyxjfe8Z482cSr5nE1MrKXK0yZ7QmUlSW5icF6dnQmjhZC5s1Rr+xljsbTP6+rNdYjoFMTf+8RLX9vgLS2tqa83WOpZr6KjBPRGaLSIzMzdC1Q8qsBa73X38GeEYzlwJrgetEJC4is4F5kNt7d+kcJQnKNdYrGNYrGNYrGKZ6QX7cRg3sqpoCvgysB3YA96vqdhG5VUSu8YvdDST8m6M3AN/w37sduJ/MjdYngJW57BEDcOJEbtrYc431Cob1Cob1CoapXpAftzH1Y1fVdcC6IfNuGfS6B/jsMO+9DbjtLBxHxNRBao322ha2xekY/X0ZiPUKhqleYAezzoqpg9Rar2BYr2BYr2CY6gX5cSv4wL5mzZqwFbJivYJhvYJhvYJhqhfkx63gA/tDDz00eqEQsF7BsF7BsF7BMNUL8uNW8IE9lQr7WbfsWK9gWK9gWK9gmOoF+XEb9QGl8eDpp59uAurP5L0tLS3VVVVVx3KsdNZYr2BYr2BYr2CY6gVn7TZz2bJlpw3BZERgt1gsFkvuKPimGIvFYrGcig3sFovFMsGYUIFdRG4UERWR6rBdAETk30Rkq4i8KSJPisj0sJ0AROT7IrLTd3tYRCaH7QSZhHMisl1EPBE5LbFRCD5Xi8g7IrJbRL4Rtg+AiPxcRBpFxKjHzERkhog8KyI7/H341bCdAESkSEQ2icgW3+tfw3YajIi4IvKGiDyay/VOmMAuIjPIDAZyIGyXQXxfVS9U1YuAR4FbRnvDOLEBWKKqFwK7gJtC9ulnG7AceD5skUEDzHwKWAx83h84Jmx+CVwdtkQWUsA/qeoi4DJgpSHfVy/wMVVdClwEXC0il4XsNJivkknVklMmTGAH7gD+GTDmbrCqtg+aLMUQN1V90s8BBJl8+bVh+vSjqjtU9Z2wPXwGBphR1T6gf4CZUFHV58lkTjUKVT2iqpv91x1kgtVpYy+MN5qh05+M+n9GnIciUgv8KXBXrtc9IQK7n4zskKpuCdtlKCJym4i8B/wF5tTYB/PXwONhSxhItgFmQg9UhYCIzAIuBl4J1ySD39zxJtAIbFBVI7yAH5KpjHq5XvGYkoCZgIg8BWTLlnMz8E3gE+NrlGEkL1X9nareDNzsD+r9ZeBfTPDyy9xM5hL63vFwGquXIWQbFMeImp7JiEgZsBr4hyFXrKHhZ5S9yL+X9LCILFHVUO9RiMingUZVfV1Ersz1+gsmsKvqVdnmi8gHgNnAlswwq9QCm0XkUlXNe+af4byy8BvgMcYpsI/mJSLXA58Gluk4PswQ4PsKm7wOEjMREZEomaB+r6oa9wy/qh4XkefI3KMI++bzR4FrRORPgCJgkojco6p/mYuVF3xTjKq+parnqOosVZ1F5oS8ZDyC+miIyLxBk9cAO8NyGYyIXA18HbhGVbvC9jGUsQwwY/HxB6+/G9ihqj8I26cfEanp7/UlIsXAVRhwHqrqTapa68es68gMTpSToA4TILAbzu0isk1EtpJpKjKiCxjwY6Ac2OB3xfxp2EIAIvLnInIQuBx4TETWh+Uy3AAzYfn0IyK/BeqABSJyUERWhO3k81HgC8DH/GPqTb82GjbTgGf9c/BVMm3sOe1aaCI2pYDFYrFMMGyN3WKxWCYYNrBbLBbLBMMGdovFYplg2MBusVgsEwwb2C0Wi2WCYQO7xWKxTDBsYLdYLJYJhg3sFovFMsH4f2c6JoKIAP8XAAAAAElFTkSuQmCC\n", 263 | "text/plain": [ 264 | "
" 265 | ] 266 | }, 267 | "metadata": { 268 | "needs_background": "light" 269 | }, 270 | "output_type": "display_data" 271 | } 272 | ], 273 | "source": [ 274 | "sns.set_palette('husl')\n", 275 | "pal = sns.color_palette()\n", 276 | "\n", 277 | "plt.title('Test Statistic on the Standard Normal')\n", 278 | "ax = sns.lineplot(domain, snorm)\n", 279 | "plt.axvline(tstat, c=pal[0], label='test stat')\n", 280 | "plt.axvline(-tstat, c=pal[0])\n", 281 | "plt.axvline(1.96, c=pal[4], label='level 0.05')\n", 282 | "plt.axvline(-1.96, c=pal[4])\n", 283 | "ax.fill_between(domain, snorm, color=pal[3])\n", 284 | "plt.legend(facecolor='white')\n", 285 | "plt.show()" 286 | ] 287 | }, 288 | { 289 | "cell_type": "code", 290 | "execution_count": 5, 291 | "metadata": {}, 292 | "outputs": [ 293 | { 294 | "name": "stdout", 295 | "output_type": "stream", 296 | "text": [ 297 | "This test has level 0.05, so we only reject if T >= q(0.025)\n", 298 | "T = 1.7, q(0.025) = 1.96, so the test fails to reject at level 0.05!\n", 299 | "\n", 300 | "We can do the same test by inverting the test statistic function to change the quantile into some constant c, \n", 301 | "then compare that to theta hat\n", 302 | "theta hat = 0.56, (+) c = 0.569, so the test fails to reject at level 0.05!\n" 303 | ] 304 | }, 305 | { 306 | "data": { 307 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEGCAYAAAB8Ys7jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztnXl4G+W1uN8jWYodb7Fjk4WwliWl0JYSSEmTEOLQmJSlhF4KlK03BUqhy8NyG265baDtpZeu0JZut0DbS1jaUi78CGFLWBuWEihbLiVQAglx4i1eZUuWvt8fM7YVeSzJlsZzhOZ9Hj+WNN/MvHM052g08+kbMcbg4+Pj41N4BLwW8PHx8fEZH34B9/Hx8SlQ/ALu4+PjU6D4BdzHx8enQPELuI+Pj0+B4hdwHx8fnwKl4Au4iKwSkc0pr/2LiLwpInERucV+bZGIvCIiMRF51AvXYkJE9hURIyLzvXbJB25uj4icJyIDSc8X2euale91jQcRKRGRm0Sk1fZa5LWT22R6T7Ts3yoLuIjcYgfHiMiAiLSJyAYR+ZaI1KY0/wHw8aR5g8BNwJ3A3sBX7Um/ADYC+wPL3d+KzIjIVSLydhbt9k2KR+rf5ROg6goiUiki14nIGyLSKyKvi8hZWcw3TUR+KiJvi0hURJpF5E8i8tE8eT08+MHvEX8FZgDveeiQzKnAmcCJWF5/dWqUsl/GRWS7iPyPiMycSNkJ4l2sWDzjpYTKAm7zBFaA9gYWAL8BzgBeFZGDBhsZY7qNMS1J880AKoA1xphtxpgO+/UDgYeMMe8aY9rGIyQWofHMmydOxtq+5L8bPfTJlX2AacAXgEOxPnR/JyIfGm0GEdkL+BswD7gIOAD4FBADnhaRRrel3cYYEzXGNBljEl672BwIbDPG/NX2iqZpewnDeftZ4CPAH3JZuYK8G4ExJm7HIua1iLo/4BbgYYfXq4A3gXVJr60CNtuPzwNMyt8ih9fOs9sfAPwZ2AW0Aw8ChyUt+zxgADgWeAGIAifY044DngIiwDbgZmBq6jYAFwBbgE7gf4H6NK6rRonHvvb0+WliNridxwGPA73Aa8DSlHYHA/cB3fbfvcABqducMs+swVgmvbYEeBnoA14CjrHbnJXifJq9jl7gLeDsNNsw1Z5naZo29wBNQJXDtDX2tLLkfQPrg+//gB5gPfCBDPue0z6U1fZgHTxcb+8TvfZ+szzD/r5bzJPey1ljfG+n2f7NQBfW/rkww7oFuNzelihWfn0tafqjKbF4O82yht7/pNe+DHSkvDauvMv2/QSWAc8D/cBOrIOc8nT1BTgLMGN4Twb3h3Q5Oeh7ph3fPqyasF+2tTDT34QW5qylRing9rTLgQTDhXAVwwW8DDjSDuxJwHQgbP83wMX24zJ7Z2/COrVyGFZh+ynQyu5FNgE8ByzGOv1Sbz/utXfOA+11rsdKLknahg7gNqyjy3lYhfx3Sa7fw/oqNt3+qxhlm7PZWQZ3sL8DjbbX77GSZErSOrcAjwBH2H/r7Z0s7LTj2q/tVsCBPe3t/2/gEKABK2GcCvhbWEXvAHt7B4ADHfxDwB3AK9gF2KFNDRAHrhpl+oLB9z5p3+gB1trb+hGsgvBYmjhW2+/jHUnvSzib7cEqhuuxit58e3+5AKsANaRZ524xZ/QCnum9fQ2rMM6x/b6BVcQ+mGbdF2MdhFxgL/eLWIVmhT29Fus05T/tWNSnWdZuBRzrSHw9cH/Sa7nkXcb3E/iw/Z78GPggcDzwDvCHdPUF9wp4D/AkVo04EuuUy9+x60TOtTIfC8n3n1OAk6Y12oE7KilIm5OmOwbWYedaBTyd0kZIOgJh+Ch5QUq7R4Hvpby2t932o0nb0AxMSmqzEtie9Pwq0hzROGxTL8NHzoN/R6fsYMuT5hv84FpqP19hL6MuJaEiwDlOO679WmoB/y7wNhB0eF9SC/ilSW1KbOcLU5YfwkrKF4FpaeJwlL3MU0aZXmtPvyLpPR4gqegAp2MVh9I063kYuGWU92DU7bHfgz6gOmXem4C706xvt5gzegFP996eB2wFSlKWvQ74SZp1vwtcl/Laj4G3UnJl82jLSMmxPjsmvQx/6MzIU95lfD+xTtc8mzLfyXabfZJyc6IKuGH3b7gH2a8tyRTPbP5KKDzE/m9yXM6RwBEi0p3yehnWkUgyzznM+3ERucRhuQdiFSKATcaY/qRp27AK5nj5PNaRbjJbU54PrhtjTJOIxJPW+SHgNZN0zcAYs0NEXrenZcshwHPGmHjSaxtGaZvsMyAiOxgZg+XAQqwEa06zXkkzDZz3ifdSlrnNXs4eWEdmYyXd9hyJdbS+TWQ31TDwxjjWlW7dqe/tkVhFfVfKuidhfUCPQESqsD6cH0+Z9BjwVRGZbIzpHaPjN7BOFQrWEfiVwMMicpQxpofc8g4yv58fwvrQSt0ewdpvt4xxe3Kl2Rgz1EvOGPMPEWmxXR7OdeGFWMAPZfirbC4EsE4lOBXhjqTHcWNMn8O8/4XzxZmmpMepF3sMmYtQOrYl7wyj4HSBKflitVORk6TXnS6cOV1ASl3OaB+oTjFIvXj+AWBnhuINVhFMYO0Df3GYfqj9//UM68fBIVvSbU8Aa985Mov58rHuwXUO/t8EnOLQJlMRTn3vctlHdyTto2+IyBtYvWk+i/VNJJe8g+zez9H2xeR9PHUbJ/IiaS7x3Y2CKuD2EcNFwCPGmNYcF/c3rK9J24wxjkcoGeb9UBbFNBNRIJjjMsbCq8AXRaRu8ChcRKZhfa37gd1mJxAUkWnGmB32ax9LWc5rwJkiEkw6Cj86B6+bsS4MpsUY0yYi9wMXi8j1xpjOlCb/DuwAHsrBBcb/vvwNmIL1df6VHB3Gs+5zgE5jzM5sZjDGdIrIVqwL0PclTVoI/HMcR99ODPalnpzkeR7jy7tseBVre5I5Bqt4v2Y/38nI/TV1H88X9SLyAWPMmwB2D7qpWB+2OaO5G2FYRKaLyAwROURE/hV4Fusr4UV5WP7PsJL0bhFZYPe1ni8i3xWReRnm/SZwsoj8WEQ+KiIfEJFGEfmtiJSNweGfwHQROVpE6kRkcob2tXZMkv8qx7C+1Vjn5e8QkY+JyBHA7VhfQ++w2zyL1YPheyJyoN0t75spy7kR66v7L0TkgyJyLNZ5cRjfqa1TgZ9k2fZirAuZ6+yY7yUiR4rIaqxeC+floTD8E+tr/gfs9yXbo7N1WF+L7xKRU0RkfxE5QkS+LCLn5+iUiVuxvO8TkU/a+/NcEblSRD6dZr5rgS+LyPn2+30hVn795zg9qpPy9nDgV1jfAB6wp+eSd9nwfeBjIvIjEZlt778/BW41xgyeMnsYmC0il9jv8flYF6bdoBe42d4P5gC/w+q9lfPpE9BdwBcA27EusjyFdZV8NXBoHo58sY8ujwZagLuwvnbfitU3eXuGeddjXR0/DKu/+ktYF366sPojZ8vdwB+xjn6agX/L0P5/bbfkv2uzXZld2D6J1TPhcaxzgz1Ao7H79hqrj/wZWD+Oegn4j1QvY8w2rF4+87DOy16PdUEWrItYY6UO6zRKNtuwBauXxTNYxeFN4H6sD/ajjTFrx7H+VH6ItV/8Het9+USWbgYrLncBP8Lq6nYfVj/1N/PglW7dfVhHmn/D+kbzD9vjKNKf9/0F1gf0v2MdoX4dWGmM+e04VX6GtV++h9U9sBLrQusbtue48y4bjDEvYb0Hx2C9f3/Aeg++mNTmYaz99Uq7zWLgmlzXPQrbgV9j9Q4a7HZ8ir2v5IzkaTk+RY6ILMT6QPiwMeZlr318fLxGRFZh9co6wK11FNQ5cB89iMhFWEcv72FdUf8x8IxfvH18Jo6Mp1DEGsRmp4g4XpSxf+Z6g4hsFpGXRMStiwE+utgH6/z561hfw5/AOlXg4+MzQWQ8hWJ/Ne4Gfm+MOdRh+jKsXyQuA+YC1xtj5rrg6uPj4+OTRMYjcGPM40C6wZ9OxiruxhjzNDBFRGbkS9DHx8fHx5l89ELZE6unyCBb7dd8fHx8fFwkHxcxnX5V5HheZu3atWb79u2ICMYYampqqK+vJxaLEQxav5uIx+OEQiEGBqz+/yUlJeOaHovFEBGCwSADAwMEg0GMMSQSiaHpgUCAQCDAwMAAJSUlJBKJMU8XEeLxOCUlJcTjcYwxQ9ODwSBv9fYAMLO0DBIJwCDBICYeB7E/P01i+DUEAgFIZDE9EABjwJjsp4tYf4kEBIIOTpmmDztlml4o2yS91vGHmbzX+2abtL5PLVGrR2PdpP3GtE3bIlbX/v0nl7/vakQm52g02tLQ0FDvVFPzUcC3AnslPZ/FKAPRV1dXM3fu+E6Pb9myhX322Wdc87pJJq/L168BYNX+R0yUEgDx5laC9VMndJ3ZoNErtO4KAAYW3+qxyUg0xgvG73XnpssAuGC/1WOa78aXrSGA1h6Vvn4Uap1Ix8aNG0ftx5+PUyj3AOfYvVE+jjX2b84d8lMJhVSN5z6EVi8JTuQv9LNHq5dWtMZLq5fWfHTLK+MRuIjchjWUYp09bsK3sAd+Mcb8EmsQ/WVYY0r3Yo2Yl3eqq6vdWGzOaPWSyWP5Rf/EodVLK1rjpdVLaz665ZWxgBtjzsgw3WCNT+EqLS0tlJeXu72aMaPVK9HVTbB0ktcaI9DqpRWt8dLqpTUf3fLy/JeYxhi6u7vJ1B+9vLyczs7Uwee8J5PXhXvuB8AeibyNIJkVZsoUZILXmQ2pXgZDXyJBV9C+gOWzG1qPdLV6+UfgE0x3dzeTJk0iHA6nbReLxVSe38rk9SG7Q84eZZkGGswz8ThoPE/p4BWLRXmvs5OuEr+Aj2BgIHMbL1DqFY3mY9j1/OOWl+ejERpjMhZvgERCyw26d0erF1oHKXPwCoXClAY83xVVYqLe3vR8NLR6RSJuDDGeO255FUzWaDz6Br1eKo++YVQvyd9NSt5XBKboPCWg1Wv69OleKzjillfBFPBYzJ1P/I6ODn772+Ghj5988klOP/30rOePxWKsXr2a7dvH1nOyv7+fL51/PgvmzuWkxkbefWc8t2dMQzyeuY0XaPVSSmJXR+ZGHqDVq6mpKXMjD3DLq2AKuLh0gSu1gI8VEeG2224b8xt0x+rVVE+ZwhPPPMMXLryQa7/97XE7jCKW3+XlC61eSpESzy9TOaLVK5vTsV7gllfBFPCgS6cErr76at5++20WLlzIN79p3Tmsp6eHc889l7lz53LBBRcM9ZB58cUXOeGEEzj22GM59dRTaWpq4r777uPFF1/kwgsvZOHChUQiEa677joaGhqYN28e1155pWMPmwfXruUzp1l3cVp24ok89eSTGXvijImA0kKp1UspUqavqx7o9aqsHMsdBicOt7xUfYw+f+cSV5Z7xGmj337uW9/6Fps2beLxxx8HrFMoL730En/961+ZMWMGjY2NPPPMMxxxxBF8/etf59Zbb6Wuro677rqL73znO/zwhz/kpptu4pprruHwww8H4Pzzz+ff/s26C9mZK/6VJx95hM+eeNJu623avp2Ze1pjfpWUlFBZWUl7Wxu1U/P0s+l4AkoUfj5r9VJKoquHYGmp1xoj0OrV2tpKRUWF1xojcMtLVQHXwsc+9jH2tIvrYYcdxjvvvENVVRWbNm1i+fLlgDX4zLRp0yhx+Cr5xBNPcMMNNxCJRGhpa2P/Aw8a0cbpWDuvp4m09urQ6qWUQPkEdz/NEq1eNTU1Xis44paXqgKe7kh5IvuBT5o0/PVwcKQygNmzZ/Pggw+O8Eqmr6+PK664gkceeYRZs2ZxxdVX09/fP2IdM2bM4L1t25gxcyYDAwN0dXUxJZ9vcgF1I/QZHRONqfzRjFavSCRCVVWV1xojcMurYA6H3OpvXVFRQXd3d8Z2BxxwAK2trTz77LOAVbg3bdpEIpHYbRmDxXrq1Kl0d3ez/v77HZd33NKl/OnOOwFYc++9zJs/P79H4FoLpVYvpRiXel/lilavvr4+rxUccctL1RF4Otw6+q6trWXu3LnMmzePJUuW8MlPftKxXTgc5pZbbmHlypV0dnYyMDDAF7/4RQ4++GDOOOMMLrvsMkpLS3nggQc455xzmD9/PnvvvTcf/PCHHZf32TPP5GuXXMKCuXOZMmUKP/vVr/K7YQXWD9zHGa39rbV6FVs/8Iz3xMwnGzZsMLNnz97ttc7Ozqy+WvT39+92akMLmbz+0Wn1l5050T+lHxgAjV29RvHa2d7OzoA3R+ehdZ8DIOaPB5414/X69aYzAbjgg2MbD3zV4Hjgxy5L2+59Oh748w0NDXOcphXMKZSA0otfWr3U9rfW6qUUUfpLX61epQp7xoB7Xkqrz0i0FkqtXmoLpVYvpUhYZ6HU6lVWpu/CKrjnpbT6jGRA6ehnWr3QOsiWVi+lJHp6vVZwRKtXe3u71wqOuOVVMAXcqb+1BrR6EVT61mr1UkqgUt/NCUCv19R8/RAuz7jlVTDZFFc6CJJWLxJKu+tp9VKKiYz8DYEGtHp1dXV5reCIW14FU8AnsrfMWNDqpba/tVYvpRilp+i0evk3dFCKW/3Acx1ONhQKjWs42Uu/8hXuu/feMXn+/uabs1+B1v7WWr2UorW/tVavYusHXjAFfKLGAx8rsVhsXMPJjpXOsRZwrad2tHopReu421q9/PHAleJWdz2vhpMFeGbDBk751Kf4xJFHDh2N9/T0cPqpp7JsyRKOO+YYHrR/iv+973yHLVu20Lh4Md+9+urMG6a1u55WL6Vo7a6n1avYuhGq6kLxr080uLLcmxY8Muq0XIeT/clPfsLNN9885uFkAXbu3Mmf772XzW+8wYpzzuFTJ57IpEmT+M0tt1BZWUlbaysnL1vGcY2NrLzqKl7/v/9j7bp12W201kKp1UsrWns5KfUqths66HwXPGYsw8k69ULJZjhZgKWNjQQCAQ46+GBampsB66Lodf/5nzyzYQOBQICmpiaa7WljIpHQOXSrVi+lmN4IKBy6VatXR0cHU6ZM8VpjBG55qSrg6Y6U4/G4a3flSWUsw8mmFvBsh5MFCCetZ/A0y91//jOtLS3c99BDhEIh5s2ZQ/94RjLT2t9aq5dSApX6bk4Aer3q6uq8VnDELa+CySa3+lvnOpxsPB4f13Cyo9HZ2UldXR2hUIi/PvkkW999d8izp6cn+wVp7W+t1UsppjfitYIjWr06OnReXHXLq2AKuFv9rZOHkx28iOnE4HCyV199NQsWLGDhwoU8++yzGGOGhpNduHAh4XB4aDjZs88+e9ThZEfjlFNP5aW//51PffKT3P3nP3PAgQcCUFNby5wjj2TJwoXZXcTU2t9aq5dSjNJeO1q93OqtlitueRXMcLKJRELlwFGZvDwbTtYYnRcMR/Hyh5N1xsQGkJCqM53A+L3cHk62UIedTsf7YjjZYvtkzRmlR0hqvZSitb+1Vi+/H7hSNB59g14vAgqPvkGvl1Jkks5ucVq9yst1DrLllpfS6jOSvN4vMo9o9QLf632B2gMEnV4T1VNtrLjlpfNdcEDrqH9avdSOu63VSykmovMmvVq9Ojs7vVZwxC2vgingWsfd1uqldtAorV5KCVRVeq3giFav+vp6rxUcccurYAq41iNdrV4kfK/3A6Z7DH3/JxCtXm1tbV4rOOKWV8EUcLdwGo3wtttuY86cOcyZM4fbbrst53U8v+Fpzvvc59K2+dH3v8+vbrxxxOv9/f186fzzWTB3Lic1NvLuO+84zv/ounUsmjePBXPn8vMbbgC7R96lX/kKn5gzh8bFi2lcvJhXX3llxLx/vP12/uPKK8e0TZvfeIPGxYs5vqGBt99+m5t+8xsWz5/PVy66KP2MfjfwsaG137xSL63j87vllVUBF5FGEXldRDaLyEqH6XuLyHoReUFEXhKR9J01x4FbpypSC3h7ezvXXXcdDz30EA8//DDXXXcdu3btSuv1kY98xBU3gDtWr6Z6yhSeeOYZvnDhhVz77W+PaBOPx7lq5Up+t3o1jzzxBPf85S/8Y/Pmoen//q1vsXbdOtauW8eHDj00L14P3H8/xzU2cv8jj7Dvvvvyh1tu4ZZbb+WGX/wi/Yz2KRS19xJVhig9VaHVyz+FkoKIBIGfA8cDhwBniMghKc2uAu40xhwOnA6MPJTMEbf6W6cOJ7tu3ToWLVpETU0NU6ZMYdGiRTzyyOhjtIzFK5FIMG/OnN1+Vrtg7lyad+4cdZ4H167lM6edBsCyE0/kqSefHPFp/uLGjey7337ss+++hMNhTvz0p4eGoc2W97Zt4+zTT2fRvHn8+Ac/AODdd95hycKFQ21+deON/Oj732fdww9z069/ze233spnTzmFK6+4gne2bGHFOefw37/8Jb09PVz+1a9ywtKlHN/QMOTyx9tv54srVvD5s87iLHubfNKT6NB5UU6r144dO7xWcMQtr2wOa48CNhtj3gIQkduBk4HXktoYYPDnlNXAe+ORaVy/ZjyzZSTdr7dSh5P96U9/OjQSIcDMmTN5773RN2cs3YMCgQDHLV3KA2vWcNoZZ/DC888za6+9qN9jj1Hnadq+nZm2T0lJCZWVlbS3tVGbdJPUpqYmZs6cOfR8xsyZvPj834aef//aa7n+hz/kEwsWsPKqqxx/EfbiCy/w8GOPUVZWxglLl7J4yRJqa2sdnRYvWcLnzj2X8vJyLvzSlwB4bP167rjrLmqnTuW/vvtd5i1YwA+uv56Ojg5Oamxkvv1BsHHjRh5cv54pNTVZx62YkVJ9vyoEvV4VFToH2XLLK5sCvifwbtLzrcDclDargAdF5MtAObDEaUE7d+5kxYoVlJSUEI/HWb58OZ///Ofp7+939Qcx/f39BINBjDEkEglCoRCxWAwRIR6PY4whHo+TSCQYGBjAGDPkNDhPPB4fahsKhbj88st57rnnAKuALliwAIATTjiBK664glgsNnKb4gOceMIJXH/99Zz2L//CPX/5CyeedBIMDFjnFBMJiA9AIGj9YtFe/9B0+4KpYL8WCIAxw/cnHBiwfqZuEtZHaiLB11euZI/6eqLxOCsvu4xf3HADX7vsMmtdwYA1uFQ8zoKFC6mpqgIRjl+2jOc2bGDpsmXDyw0GrXlMwvqfSNhOA9a6Bv0ScR5/9FEeeuABfn3jjWAM/X19bNu61VrPggVMqa4aXmY8DiKYRJx4azsyuQwGBjDRGIEp1SR2dSAlJUjZJBJdPQTKJ2OiMUwsaXoohIRDJHp6CVSWYyL9mIGB4enhEJSUYHojBCorML0RTDw+NH3w1gTx5lYCVZXWBTpjkKpKEh2dQ8XK9PUTqK7CdHaBCFJRTqKzCykrhUQC0x8dXmcwiEwuI9HVnds2dXYhoZIxb5NMClv7T6TPlW1CrG+UY92mQUxf35i2aZD29nY6Ozupr6+nra0NYwz19fXs2LGDiooKurq66O7uZtq0aTQ3NyMi1NbW0tzcTFVVFfF4nJ6eHqZPn05TUxOhUIjq6mpaWlqorq4mGo0SiUSGpofDYSorK2ltbaWmpoZIJEJfX9/Q9NLSUsrKymhvb2fq1Kl0dXURjUaHppeVlREOh2lubqa0tJSOjg5isdjQ9PLycoLBYNptSkc2BdzplxepZ+TPAG4xxvxQRI4G/iAihxpjduv0u8cee/DUU0/tNmNnZ+fQEWG6I+V8j3EwuKxgMIiIEAwGCQaD7L333jz55JND07dv3878+fOHpg/yox/9aMjrqKOO4oknnnBcPsmDtgVLOOLjH+ftr32N1l27eHDtWr5y6aXW4PgiVkEO2m+Jfc5/xsyZvLdjBzP23psBrLtbT6mdutt4IjNmzbK+JdjzbN/exLTp0yEQYJp9ZD4pFOK0M8+0LpQGAsM/xAgCwSASCAwP0i+CBIOUhEIkjBl6vT8aBQkMzy8y7CtiFeRAEAP86qab+MABB+wWkxdfeIHJkydbH1CDn232siUQJFg/lVSSXwuWllptJ5c5T7dfF7td6vTB8auTjx53m99+nPwrw93mt4dQlbpa5+mOziP32bFsk/T0IqWl494mKspd2aZ4cyuB6qpRpqfZJvvWsWPdpkFqamqosb+9TZ48PL7QPvvsA0B3d/fQ47322mvEdBge2jX5NadfSiZPHyykyeM2JU8ffD254CZP7+jooLy8fLf1JE9Pt01btmwZ4TZINoe9W4G9kp7PYuQpkhXAnQDGmA1AKZDXAXDduqlx6nCyixcvZv369ezatYtdu3axfv16Fi9enDcvEaFx2TKu+da3OOCgg6gZ5TTFIMctXcqf7rwTgDX33su8+fNH/PrzI4cfzj/feot3tmwhGo1y7913c1xjIzB87s0YwwP338/BKYOJDfLEY4+xq72dvkiEB+6/nzlHHUVdfT2tLS20t7XR39/PIw89lNU2HrNoEbf89rdD5+pfefnl5ABktQwfi+QiqQmtXtOmTfNawRG3vLIp4M8BB4rIfiISxrpIeU9Km3eABgAR+SBWAR/HbWRGx61eC6nDydbU1HD55ZfT0NBAQ0MDV1xxxdCnY768Tjj5ZP7ypz9xwsknZ2z72TPPpL29nQVz5/KbX/6Sld/4BmCdtjn3TGtkt5KSEr597bWcffrpLJ4/nxNOOomD7WFov3rRRRx3zDEcd8wxtLe1WUf8Dhw5dy5fu+QSGhsaOP6EE/jIRz9KKBTiq5deyknHH8/nzzqLD9jLzMRXL72UWCzGJxctYsnChfzge98bnqi0m5dWTGeX1wqOaPUa192rJgC3vLIaTtbuFvgTrC/cNxljvisi1wB/M8bcY/dK+Q1QgXV65d+MMQ+mLieX4WSj0ajK+91l8vJsONn4wPDpDU2M4uUPJ+tMorWdwFR9F3zH6+X2cLJbt25l1qxZY/Zym1y80g0nm1WGG2PWAGtSXvtm0uPXgE+Myy5Lim2QmpwJ+F7vB6RC5+h6Wr1G6znlNW55FcwvMbX+8EOrl9pxt7V6KSWh9FSFVq9iO4VSMAVc65GuVi+tw32q9VKKlJVmbuQBWr2yOR3rBW55FUw2FdsYB7lTWF5Gra/HaB1+V6mX1sHl3PLyvICLCNFoNGO7hNIdRquX2ru/O3jFYlH6tMbRY0x/5tzwAq1ePT06R0l0y8tolZlZAAAgAElEQVTzbgqD/bD7+tIPEB+Px+nv758gq+zJ5PVqk9VlvmTKyB9HuIlJJKwf5ygj1ctg6Esk6Ar6/cOdSP4Voia0ek2fPt1rBUfc8vK8gIsIlZWZRzbbsmXLbr9c0kImr189/yQAqyb46ni8tc3xF3Ve4+il8INGC4ldHSrfR61eTU1NKuuEW14Fkzlu/RIzV7R6idKLq1q9tKI1Xlq9tOajW14FU8Crq3V+ZdPqlTwOhSa0emlFa7y0emnNR7e8CqaAt7S0eK3giFavRFd35kYeoNVLK1rjpdVLaz665VUwBbzYPllzResRklYvrWiNl1YvrflY9Efg2XQ19AKtXmj9hahWL61ojZdSL6356JZXwRTwSCSSuZEHaPUyUXduQZcrWr20ojVeWr205qNbXgVTwIutf2euaO2nq9VLK1rjpdVLaz665VUwBbypqclrBUe0eiV2dWRu5AFavbSiNV5avbTmo1teBVPANY4FDnq9pMTz32g5otVLK1rjpdVLaz665VUwBTybX2t6gVYvKdN513CtXlrRGi+tXlrz0S2vgingra2tXis4otUr0aVzUB+tXlrRGi+tXlrz0S2vging6e5L6SVavQLlE3wLtyzR6qUVrfHS6qU1H93yKpgCXmzdg3JFazcvrV5a0RovrV5a87HouxFmGm7WK7R6mZjOBNPqpRWt8dLqpTUf3fIqmAJebP07c0VrP12tXlrRGi+tXlrz0e8HXmT9O3NFaz9drV5a0RovrV5a87Ho+4GXluq8iapWL1E6LrJWL61ojZdWL6356JZXwRTwsjKdo59p9ZKwzgTT6qUVrfHS6qU1H93yKpgC3t7e7rWCI1q9Ej29Xis4otVLK1rjpdVLaz665VUwBXzqVH333wO9XoHKcq8VHNHqpRWt8dLqpTUf3fIqmALe1dXltYIjWr1MpN9rBUe0emlFa7y0emnNR7e8CqaAF9tA7blilA64r9VLK1rjpdVLaz4W/Q0diq1/Z65o7aer1UsrWuOl1UtrPvr9wIusf2euaO2nq9VLK1rjpdVLaz4WfT/wYuselCtau3lp9dKK1nhp9dKaj0XfjbDYBmrPGaUD7qv10orWeCn10pqPRX9Dh44OnV/ZtHqZXp2jsmn10orWeGn10pqPbnkVTAGvq6vzWsERrV6BygqvFRzR6qUVrfHS6qU1H93yyqqAi0ijiLwuIptFZOUobU4TkddE5FURWZ1fzeL7ZM0VrUdIWr20ojVeWr205qNbXhlPZIlIEPg5cBywFXhORO4xxryW1OZA4ErgE8aYdhHZI9+iMaXjD2v1MvG41wqOaPXSitZ4afXSmo9ueWVzBH4UsNkY85YxJgrcDpyc0uZ84OfGmHYAY8zO/GoWX//OXNHaT1erl1a0xkurl9Z89LIf+J7Au0nPt9qvJXMQcJCIPCUiT4tIY74EBym2/p25orWfrlYvrWiNl1Yvrfnollc2fYHE4TXjsJwDgUXALOAJETnUGLMrudHOnTtZsWIFJSUlxONxli9fzsUXX0xTUxPl5eUEg0E6Ozupr6+nra0NYwz19fXs2LGDRCJBa2sr3d3dTJs2jebmZkSE2tpampubqaqqIh6P09PTw/Tp02lqaiIUClFdXU1LSwvV1dVEo1EikcjQ9HA4TGVlJa2trdTU1BCJROjr6xuaXlpaSllZGe3t7UydOpWuri6i0ejQ9MG+nVu2bKGuro6Ojg5isdjQ9PLy4QF/TH8U090DxiBVlSQ6OpHSSda0vn4C1VWYzi4QQSrKSXR2IWWlkEhg+qMEplST2NWBBIPI5DISXd3I5DIYGMBEY8PTS0qQskmY/n5MbwQTjWFiSdNDISQcItHTS6CyHBPpxwwMDE8Ph6CkBNMbIVBZYS0jHh+ePikMgQAm0kegqnLM24QxJDq7xrVNia4eAuWT875Ngz2a482t49qmXN6nTNtk+vsxfX0T/j5l2iYwJDo6x7xNQ/kwxm0apL29fdQaUVFRQSKRYMuWLapqRDgcpre3l56eHscaka7uVVSkv1gsxqTW4pQGIkcDq4wxS+3nVwIYY65NavNL4GljzC3280eAlcaY55KXtWHDBjN79uy06xuN9vZ2lXeczuTVuH4NAKsOO2KilABIdPcQqNA3YpxGr9C6zwEQW3yrxyYj0RgvGL/XrzedCcAFHxxbP4dVLz8PwNpjl6VtV6h1Ih0bN258vqGhYY7TtGxOoTwHHCgi+4lIGDgduCelzd3AsQAiUod1SuWtcdmOQmdnZz4Xlze0epmIzpu7avXSitZ4afXSmo9ueWUs4MaYAeAS4AFgE3CnMeZVEblGRE6ymz0AtIrIa8B64ApjTGs+Revr6/O5uLyh1StQVem1giNavbSiNV5avbTmo1teWf0e1hizBliT8to3kx4b4FL7zxXa2tqYPHmyW4sfN1q9THePdR5UGVq9tKI1Xlq9tOajW14F80vMTOfqvUKrF77X+wOt8VLqpTUf3fIqmAJebF+NckWUfsXV6qUVrfHS6qU1H93yKpgCvmPHDq8VHNHqlejQeTFHq5dWtMZLq5fWfHTLq2AKeKb+kF6h1Wuwn682tHppRWu8tHppzUe3vAqmgPv4+Pj47E7BFPDu7m6vFRzR6mX6dN41XKuXVrTGS6uX1nx0y6tgCvi0adO8VnBEq1egusprBUe0emlFa7y0emnNR7e8CqaANzc3e63giFYv09nltYIjWr20ojVeWr205qNbXgVTwEWcxtTyHq1e+F7vD7TGS6mX1nx0y6tgCnhtba3XCo5o9RKFAyCBXi+taI2XVi+t+eiWV8EU8GL7apQrCaVfcbV6aUVrvLR6ac3Hoj+FUlWl86KJVi8pK/VawRGtXlrRGi+tXlrz0S2vgingcaX34NPqRSLhtYEzWr20ojVeSr205qNbXgVTwHt6erxWcESrl+mPeq3giFYvrWiNl1YvrfnollfBFPBiu1lprmi96axWL61ojZdWL6356OVNjVVQbDcrzRWtN53V6qUVrfHS6qU1H93yKpgCHkq6IaomtHpJMOi1giNavbSiNV5avbTmo1teBVPAq6t1fmXT6iWTy7xWcESrl1a0xkurl9Z8dMurYAp4S0uL1wqOaPVKdOkc1Eerl1a0xkurl9Z8dMurYAp4sX2y5orWIyStXlrRGi+tXlrzseiPwKNRnd2WtHoxMOC1gTNavbSiNV5KvbTmo1teBVPAI5GI1wqOaPUy0ZjXCo5o9dKK1nhp9dKaj255FUwBL7b+nbmitZ+uVi+taI2XVi+t+ej3Ay+y/p25orWfrlYvrWiNl1YvrflY9P3Aw+Gw1wqOaPWSkhKvFRzR6qUVrfHS6qU1H93yKpgCXllZ6bWCI1q9pEznXcO1emlFa7y0emnNR7e8CqaAt7a2eq3giFavRJfOQX20emlFa7y0emnNR7e8CqaA19TUeK3giFavQPlkrxUc0eqlFa3x0uqlNR/d8iqYAl5s3YNyRWs3L61eWtEaL61eWvOx6LsR9vX1ea3giFYvE9OZYFq9tKI1Xlq9tOajW14FU8CLrX9nrmjtp6vVSyta46XVS2s++v3Ai6x/Z65o7aer1UsrWuOl1UtrPhZ9P/DSUp03UdXqJUrHRdbqpRWt8dLqpTUf3fIqmAJeVqZz9DOtXhLWmWBavbSiNV5avbTmo1teBVPA29vbvVZwRKtXoqfXawVHtHppRWu8tHppzUe3vLIq4CLSKCKvi8hmEVmZpt1nRMSIyJz8KVpMnTo134vMC1q9ApXlXis4otVLK1rjpdVLaz665ZWxgItIEPg5cDxwCHCGiBzi0K4S+ArwTL4lAbq6utxYbM5o9TKRfq8VHNHqpRWt8dLqpTUf3fLK5gj8KGCzMeYtY0wUuB042aHdt4HrAFc6PBbbQO25YpQOuK/VSyta46XVS2s+uuWVzZBiewLvJj3fCsxNbiAihwN7GWP+n4hcPtqCdu7cyYoVKygpKSEej7N8+XIuvvhimpqaKC8vJxgM0tnZSX19PW1tbRhjqK+vZ8eOHZSWltLa2kp3dzfTpk2jubkZEaG2tpbm5maqqqqIx+P09PQwffp0mpqaCIVCVFdX09LSQnV1NdFolEgkMjQ9HA5TWVlJa2srNTU1RCIR+vr6hqaXlpZSVlZGe3s7U6dOpauri2g0OjS9rKyMyZMns2XLFurq6ujo6CAWiw1NLy8f/ppp+qOY7h4wBqmqJNHRiZRaAwKZvn4C1VWYzi4QQSrKSXR2IWWlkEhg+qMEplST2NWBBIPI5DISXd3Wba0GBjDR2PD0khJroKFEAtMbwURjmFjS9FAICYdI9PQSqCzHRPoxAwPD08MhKCnB9EYIVFZYy4jHh6dPCkMggIn0EaiqHPs2hUMkOrvGtU2Jrh4C5ZPzvk2Dl+Piza3j26Yc3qdM20Qigenrm/j3KdM2TQqT6Ogc8zYN5cMYt2mQ9vb2UWtERUUFpaWlbNmyRVWNCIfDQ+t1qhHp6l5FRUXa4izGmPQNRP4FWGqM+YL9/GzgKGPMl+3nAWAdcJ4x5m0ReRS43Bjzt9RlbdiwwcyePTvt+kZjy5Yt7LPPPuOa100yeTWuXwPAqsOOmCglwCpGwXp95wM1eoXWfQ6A2OJbPTYZicZ4wfi9fr3pTAAu+ODqMc236uXnAVh77LK07Qq1TqRj48aNzzc0NDheV8zmFMpWYK+k57OA95KeVwKHAo+KyNvAx4F78n0hs9i6B+WK1m5eWr20ojVeWr205qOX3QifAw4Ukf1EJAycDtwzONEY02GMqTPG7GuM2Rd4GjjJ6Qg8F4ptoPacUTrgvlovrWiNl1Ivrfno2Q0djDEDwCXAA8Am4E5jzKsico2InOSKlQMdHTp/uqvVy/TqHJVNq5dWtMZLq5fWfHTLK6uPUWPMGmBNymvfHKXtoty1RlJXV+fGYnNGq1egMv3FD6/Q6qUVrfHS6qU1H93yKphfYhbbJ2uuaD1C0uqlFa3x0uqlNR/d8iqYAh5TOv6wVi8Tj3ut4IhWL61ojZdWL6356JZXwRTwYhvnN1e0jtes1UsrWuOl1UtrPvrjgRfZOL+5onW8Zq1eWtEaL61eWvOx6McDT/5Voya0eskknd2ptHppRWu8tHppzUe3vAqmgAeDQa8VHNHqRUDpW6vVSyta46XUS2s+uuWl811woLOz02sFR7R6mYjOm7tq9dKK1nhp9dKaj255FUwBr6+v91rBEa1egapKrxUc0eqlFa3x0uqlNR/d8iqYAt7W1ua1giNavUx3j9cKjmj10orWeGn10pqPbnkVTAHPNGqiV2j1wvd6f6A1Xkq9tOajW14FU8CL7atRrojSr7havbSiNV5avbTmY9GfQtmxY4fXCo5o9Up06LyYo9VLK1rjpdVLaz665VUwBTzTnSm8QqvX4B1XtKHVSyta46XVS2s+uuVVMAXcx8fHx2d3CqaAd3d3e63giFYv06fzruFavbSiNV5avbTmo1teBVPAp02b5rWCI1q9AtVVXis4otVLK1rjpdVLaz665VUwBby5udlrBUe0epnOLq8VHNHqpRWt8dLqpTUf3fIqmAIuIl4rOKLVC9/r/YHWeCn10pqPbnkVTAGvra31WsERrV5SoXNUNq1eWtEaL61eWvPRLa+CKeDF9tUoVxJKv+Jq9dKK1nhp9dKaj0V/CqWqSudFE61eUlbqtYIjWr20ojVeWr205qNbXgVTwONK78Gn1YtEwmsDZ7R6aUVrvJR6ac1Ht7wKpoD39Ogc/Uyrl+mPeq3giFYvrWiNl1YvrfnollfBFPBiu1lprmi96axWL61ojZdWL6356N/UuMhuVporWm86q9VLK1rjpdVLaz4W/U2NQ6GQ1wqOaPUSpfcG1OqlFa3x0uqlNR/d8iqYAl5drfMrm1YvmVzmtYIjWr20ojVeWr205qNbXgVTwFtaWrxWcESrV6JL56A+Wr20ojVeWr205qNbXgVTwIvtkzVXtB4hafXSitZ4afXSmo9FfwQejerstqTVi4EBrw2c0eqlFa3xUuqlNR/d8iqYAh6JRLxWcESrl4nGvFZwRKuXVrTGS6uX1nx0y6tgCnix9e/MFa39dLV6aUVrvLR6ac1Hvx94kfXvzBWt/XS1emlFa7y0emnNx6LvBx4Oh71WcESrl5SUeK3giFYvrWiNl1YvrfnolldWBVxEGkXkdRHZLCIrHaZfKiKvichLIvKIiOyTb9HKysp8LzIvaPWSMp13DdfqpRWt8dLqpTUf3fLKWMBFJAj8HDgeOAQ4Q0QOSWn2AjDHGPNh4E/AdfkWbW1tzfci84JWr0SXzkF9tHppRWu8tHppzUe3vLI5Aj8K2GyMecsYEwVuB05ObmCMWW+M6bWfPg3Myq8m1NTU5HuReUGrV6B8stcKjmj10orWeGn10pqPbnllU8D3BN5Ner7Vfm00VgD35yLlRLF1D8oVrd28tHppRWu8tHppzUe3vLK5EuF0N07j2FDkLGAOcIzT9J07d7JixQpKSkqIx+MsX76ciy++mKamJsrLywkGg3R2dlJfX09bWxvGGOrr69mxYweRSISSkhK6u7uZNm0azc3NiAi1tbU0NzdTVVVFPB6np6eH6dOn09TURCgUorq6mpaWFqqrq4lGo0QikaHp4XCYyspKWltbqampIRKJ0NfXNzS9tLSUsrIy2tvbmTp1Kl1dXUSj0aHpZWVl7Nq1i76+Purq6ujo6CAWiw1NLy8fvm+g6Y9iunvAGKSqkkRHJ1JqnUc0ff0EqqusO32LIBXlJDq7rLueJBKY/iiBKdUkdnUgwSAyuYxEV7f1a7iBAUw0Njy9pAQpm0SiowMJhzDRGCaWND0UQsIhEj29BCrLMZF+zMDA8PRwCEpKML0RApUVmN4IJh4fnj4pDIEAJtJHoKpyzNtkolEIyPi2qauHQPnkvG/T4DBD8ebWcW1TTu9Tpm3q6EBKwxP+PmXaJhOLgTDmbRrKh76+MW3TIO3t7aPWiIqKCtrb2+nr61NVI8Lh8JCfU41IV/cqKirSF2djHGtxclE+GlhljFlqP78SwBhzbUq7JcBPgWOMMTudlrVhwwYze/bstOsbjf7+fiZN0nfhJJNX4/o1AKw67IiJUgLAxAaQkL6eAhq9Qus+B0Bs8a0em4xEY7xg/F6/3nQmABd8cPWY5lv18vMArD12Wdp2hVon0rFx48bnGxoa5jhNy+YUynPAgSKyn4iEgdOBe5IbiMjhwK+Ak0Yr3rlSbP07c0VrP12tXlrRGi+tXlrz0bN+4MaYAeAS4AFgE3CnMeZVEblGRE6ym30fqAD+KCIvisg9oyxu3JSW6ryJqlYvUTouslYvrWiNl1YvrfnolldW34GMMWuANSmvfTPp8ZI8e42grEzn6GdavSSsM8G0emlFa7y0emnNR7e8CuaXmO3t7V4rOKLVK9HTm7mRB2j10orWeGn10pqPbnkVTAGfOnWq1wqOaPUKVJZnbuQBWr20ojVeWr205qNbXgVTwLu6urxWcESrl4n0e63giFYvrWiNl1YvrfnollfBFPBiG6g9V4zSAfe1emlFa7y0emnNx6K/oUOxjfObK1rHa9bqpRWt8dLqpTUf/fHAi6x/Z65o7aer1UsrWuOl1UtrPhb9eODF1j0oV7R289LqpRWt8dLqpTUfi74bYbEN1J4zSgfcV+ulFa3xUuqlNR89vaGDBjo6dH5l0+plenWOyqbVSyta46XVS2s+uuVVMAW8rq7OawVHtHoFKtOPYuYVWr20ojVeWr205qNbXgVTwIvtkzVXtB4hafXSitZ4afXSmo9FfwQei+kcQF6rl4nHvVZwRKuXVrTGS6uX1nx0y6tgCnix9e/MFa39dLV6aUVrvLR6ac1Hvx94kfXvzBWt/XS1emlFa7y0emnNx6LvB558ezJNaPWSSTq7U2n10orWeGn10pqPbnkVTAEPBoNeKzii1YuA0rdWq5dWtMZLqZfWfHTLS+e74EBnZ6fXCo5o9TKRPq8VHNHqpRWt8dLqpTUf3fIqmAJeX1/vtYIjWr0CVZVeKzii1UsrWuOl1UtrPrrlVTAFvK2tzWsFR7R6me4erxUc0eqlFa3x0uqlNR/d8iqYAm6M8VrBEa1e+F7vD7TGS6mX1nx0y6tgCnixfTXKFVH6FVerl1a0xkurl9Z8LPpTKDt27PBawRGtXokOnRdztHppRWu8tHppzUe3vAqmgFdU6Bw8R6uXlE7yWsERrV5a0RovrV5a89Etr4Ip4D4+Pj4+u1MwBby7u9trBUe0epk+nXcN1+qlFa3x0uqlNR/d8iqYAj5t2jSvFRzR6hWorvJawRGtXlrRGi+tXlrz0S2vgingzc3NXis4otXLdHZ5reCIVi+taI2XVi+t+eiWV8EUcBHxWsERrV74Xu8PtMZLqZfWfHTLq2AKeG1trdcKjmj1kgqdo7Jp9dKK1nhp9dKaj255FUwBL7avRrmSUPoVV6uXVrTGS6uX1nws+lMoVVU6L5po9ZKyUq8VHNHqpRWt8dLqpTUf3fIqmAIeV3oPPq1eJBJeGzij1UsrWuOl1EtrPrrlVTAFvKdH5+hnWr1Mf9RrBUe0emlFa7y0emnNR7e8CqaAF9vNSnNF601ntXppRWu8tHppzUf/psZFdrPSXNF601mtXlrRGi+tXlrz0dObGotIo4i8LiKbRWSlw/RJInKHPf0ZEdk336J33313vheZF7R6PXDvvV4rOKLVSyta46XVS2s+uuWVsYCLSBD4OXA8cAhwhogcktJsBdBujDkA+DHwX/kWveuuu/K9yLyg1WvtPToTTKuXVrTGS6uX1nx0y0sy3SlCRI4GVhljltrPrwQwxlyb1OYBu80GESkBmoB6k7LwDRs2mNmzZ49ZsuzbvxnzPFo4Zv6eADz25DaPTXxG48mD7wBg/uuf9djk/c8Zi24H4LZHTx/TfO+HPIr8x/njmm/jxo3PNzQ0zHGalk0B/wzQaIz5gv38bGCuMeaSpDav2G222s/ftNu0JC9rzZo1Xdu3bx866q+qqmqura3drc1otLW11WXbdiLJxqvfJCYnDCUT5QTQtWtXdeWUKepOVPpeY8P3GiYgDEySQG+6NoVcJ9KwT0NDg+MtfbIpKk4/4k+t+tm0YdmyZTrvw+Tj4+NTgGRzEXMrsFfS81nAe6O1sU+hVAM6bw/t4+Pj8z4hmwL+HHCgiOwnImHgdOCelDb3AOfajz8DrEs9/+3j4+Pjk2eMMRn/gGXAP4A3gW/Yr10DnGQ/LgX+CGwGngX2z7C8RuB1u/3KNO0+g3UqZk7Sa1fa870OLB3rMt3wAo4Dngdetv8vTmr7qL3MF+2/PSbQa18gkrTuXya1PcL23QzcgH09ZIK8Ppfk9CKQAD46UfECzgOak9bxhaRp5wJv2H/nTmS8RvMCPgpsAF4FXgI+mzTPLcA/k+b56ATHK570+j1Jr+8HPGPH8Q4gPIHxOjZl/+oDPp2veGW77wOnAa/Z79tqN/axMYvn+gcEsT4I9gfCwN+BQxzaVQKPA08znPiH2O0n2TvIm/byslqmi16HAzPtx4cC25LaP0rSB9AEx2tf4JVRlvsscDTW9Yv7geMnyitl+mHAWxMZL6zE/5nDvLXAW/b/GvtxzUTFK43XQcCB9uOZwHZgiv38FuAzXsTLntY9yut3Aqfbj38JXDSRXinvaRswOR/xGoPbgcALSfvPHm7sY178EvMoYLMx5i1jTBS4HTjZod23geuwPj0HORm43RjTb4z5J9Yn1VFjWKYrXsaYF4wxg9cFXgVKRSRft+3OJV6OiMgMoMoYs8FYe87vgU975HUGcNsY150PLyeWAg8ZY9qMMe3AQ0DjBMdrBMaYfxhj3rAfvwfsBBx7JYyDfOTOboh194LFwJ/sl37HBMYrhc8A9xtj0vZeccHtfODn9n6EMWan/Xpe9zEvCviewLtJz7farw0hIocDexlj/l+W82ZcpsteyZwKvGCMSb7r680i8qKI/IeM/dYcuXrtJyIviMhjIrIgaZlb0y1zArwG+SwjC7ir8bI5VUReEpE/icjgRfp0+5fr8UrjNYSIHIV11Pdm0svftef58TgOHHL1KhWRv4nI0yIyWHCmAruMMQMZlumm1yCnM3L/yiVe2bodBBwkIk/ZsWnMMO+49jEvCnjaLociEsD6NedlY5g3q26MLnoNtvkQ1q9QL0x6+XPGmMOABfbf2RPotR3Y2xhzOHApsFpEqjItcwK8BtvMBXqNMa8kvexqvGzuBfY1xnwYeBjrCDHdvK7HK4OXtQDrKO0PwOeNMYPjuV4JzAaOxPpa/vUJ9trbGDMHOBP4iYh8IMtluu01GK/DgAeSXs41Xtm6lWCdRlmE9S3zv0VkSpp5xxUzLwp4pm6JlVjnkR8VkbeBjwP3iMicNPNm09XRTS9EZBbwF+AcY8zQ0ZExZpv9vwtYjfX1a0K87FNNrfb6n8c6ajvIXuasNMt01SupzYijowmIF8aY1qRvSL/BuniUbt6JiFc6L+wP3vuAq4wxTyfNs91Y9AM3M7HxGjylgzHmLazrF4cDLcAUu0ux4zLd9rI5DfiLMSaWNE+u8crKzW7zv8aYmH2693Wsgp7ffSyXk/nj+cP6ZHoL6yLk4AWAD6Vp/yjDF+U+xO4XMd/CuqAwpmW64DXFbn+qwzLr7MchrHOCX5xAr3ogaD/eH9gG1NrPn8MqqoMXTJZNlJf9PGDvtPunLNP1eAEzkh6fAjxthi8w/RPr4lKN/XjC4pXGKww8AnzNYbkz7P8C/AT43gR61QCT7Md1WL0qDrGf/5HdL2J+aaK8kl57Gjg2n/Eag1sj8Luk2LyLdWopr/vYmMTz9UeGbokpbR9l98T/hj3f6yRdpXVa5kR5AVcBPezedWkPoByrW+FLWBc3r8cuqBPkdaq93r8DG4ETk9rNAV6xl/kzxtctLpf3cS5AsfoAAAC2SURBVJFDwk1IvIBrk+KyHpidNO+/Yl0c34x1qmLC4jWaF3AWEEvZvwa7Xa7D6nr2CvA/QMUEes2z1/13+/+KpGXuj9WrYjNWMZ80we/jvlgHLIGUZeYcryzdBPgRVjfCl7E/zPK9j2UcC8XHx8fHRycFc0MHHx8fH5/d8Qu4j4+PT4HiF3AfHx+fAsUv4D4+Pj4Fil/AfXx8fAoUv4D7+Pj4FCh+Affx8fEpUPwC7uPj41Og/H+ZCph0KUK7rgAAAABJRU5ErkJggg==\n", 308 | "text/plain": [ 309 | "
" 310 | ] 311 | }, 312 | "metadata": { 313 | "needs_background": "light" 314 | }, 315 | "output_type": "display_data" 316 | } 317 | ], 318 | "source": [ 319 | "print('This test has level 0.05, so we only reject if T >= q(0.025)') \n", 320 | "print(f'T = {round(tstat,2)}, q(0.025) = 1.96, so the test {\"rejects\" if tstat > 1.96 else \"fails to reject\"} at level 0.05!')\n", 321 | "\n", 322 | "print('\\nWe can do the same test by inverting the test statistic function to change the quantile into some constant c, \\nthen compare that to theta hat')\n", 323 | "theta_0_plus_buffer = theta_0 + ((1.96 * np.sqrt(theta_0 * (1 - theta_0))) / np.sqrt(n))\n", 324 | "print(f'theta hat = {round(theta_hat,3)}, (+) c = {round(theta_0_plus_buffer, 3)}, so the test {\"rejects\" if theta_hat > theta_0_plus_buffer else \"fails to reject\"} at level 0.05!')\n", 325 | "\n", 326 | "x = np.linspace(0.4, 0.6, 100)\n", 327 | "plt.title('Different Enough? On the line of Bernoulli p')\n", 328 | "ax = sns.lineplot(x, [0 for i in x])\n", 329 | "plt.ylim(0, 1)\n", 330 | "plt.axvline(theta_0, c=pal[1], label='theta 0')\n", 331 | "plt.axvline(theta_hat, c=pal[2], label='theta hat')\n", 332 | "plt.axvline(theta_0_plus_buffer, c=pal[3], label='t0 + lvl 0.05 buffer')\n", 333 | "plt.axvline(1 - theta_0_plus_buffer, c=pal[3])\n", 334 | "ax.fill_between([1 - theta_0_plus_buffer, theta_0_plus_buffer], y1=[1, 1], color=pal[3], alpha=0.3)\n", 335 | "plt.legend()\n", 336 | "plt.show()" 337 | ] 338 | }, 339 | { 340 | "cell_type": "code", 341 | "execution_count": 6, 342 | "metadata": {}, 343 | "outputs": [ 344 | { 345 | "name": "stdout", 346 | "output_type": "stream", 347 | "text": [ 348 | "To get the p-value, which gives us our minimum level alpha for T, we calculate P(Z > T)\n", 349 | "Or, in this case, the two-sided hypothesis version P(|Z| > T)\n" 350 | ] 351 | }, 352 | { 353 | "data": { 354 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEGCAYAAABxfL6kAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsvXl0HNd9qPn9qrob3ViJjQApypQYU6ZkOrIVWbFeJrJf6Fjy5ER6w9jvyU4yToaeiTPUS16WmdgnebZHceLEWSeJEjvz4tiJYyuy5EWWKFErtUISKZAU9w0kSBDEDjbQQAO91J0/uhpqNhtAV3dX122gvnNwDqpuVfVXuLd/qLr1q3tFKYWPj4+Pz8rB8FrAx8fHx6ey+IHdx8fHZ4XhB3YfHx+fFYYf2H18fHxWGH5g9/Hx8Vlh+IHdx8fHZ4XhB3YNEZF3iogSkQ947eImIjIgIp91uM+3RORJt5zcQEQCdn3e68Fnl/33KqU9ut2GReS/ichFEbFE5A/c+Ixaxg/sBRCRb9iNUolISkT6ReSrItK+xD4HROShRcq6RCQpIp92z7omeR/wtw732Ql8wgUXnxpBRK4F/hL4Q+Aa4K8qeOwv5Xz3F/v5pUp9nlsEvBbQmJeA/0zmb/QTwP8ArgV+bpHt/xH4KxHpVEqN5pX9KhAHHnTJtSYp8HcqZp+oGy4+NcWPAQL8UCl1qdSDiEhIKZXIW/0nwN/lLH8POAv8Ts66y6V+ZrXwr9gXJ6GUGlJKDSilfgj8NXCXiEQW2f5bQBL4VO5KERFgB/BtpVTMXvdbInJQRGIicklEvi0i3YuJLHZbKyLncm9DRaRJRP5WRAZFZFZE3hSRe0o6ewfYXSpfFJGviUhURIZF5NdFpE5EHhCRy/Y2v15gv8/mLX/ePodJ+zhfEREjZ5sruhayy/at+YD9N/2a3f2xU0TO28f6BxEJ5uz3soh8Nc/niyJyutxjF/k3W7KuROR1Efn7AvudFJEv5Sz/ot2W5kTkrIj8uYjUO3QptT1+UkSeF5G4iJyRwl1NG0Tkcfscz4jIJ8v87C8Bz9uLg7bHBrvsV0XkuIgkROSCiNwvImbOvi+LyD+KyB+LyCWgP//4SqmY/b0fUkoNAQkgnrtOKTW31N9TB/zAXjxxMn+vgnc5SqkpMlfk/3te0X8E3knmij6X3wbeA/wCsAn4t3Lk7H8gjwPvBj4ObAX+P+C7IvLBJfb7kP2lWurnR0Uo/CZwlMzdzT8ADwDfB04B7we+BvydiLxrmeP8FnAeuM3+/beBX1xmn9uBHwd+FvglMndIjwK3AneS+Wf7q8CvFHEerh+7yLr6F+BeEQnl7PcBYLNdhmS69v4G+DPgJtvhLjJ/e6eU0h7/zPZ+L/AQ8G8i8t68bf4U+Gcyf8PvAd8UkU1lfPafkLmTxj7mOjIB/h4yd9VfJ/P3/L+B3wDy+98/AbQA24CPFHGOtYlSyv/J+wG+ATyTs3wTcAZ4bZn9bgMUcEfOuu8A+5bZ7/32fl328jvt5Q8UWs7Z7xzwB/bvHybzz6cpb5t/AR5e4rMj9vGX+lm/jP9A7mcAJjADfD9nnQFMAZ/J2++zecvfyzv2M8C/5ix/C3gyb/kSEMxZtxsYBkI56x4HHsxZfhn4at5nfRE4Xe6xC/x9Anb93VtsXQFtwDzwCznlfw/05P29Pp13jJ8BrOyx8/9eRbb/YtvjF/L2ewP4Rt42v5FTHgRmgR3FfvYi23zY3qY7Z10Pmbvi3O1+x26HgZw6PwaIg7/FHuB/OPn76fDj97EvzodEJEYmSNUBzwK/BmDfwuc+QLlJKXVeKfWGiOwnc9X+omQetv4vZK4cFhCRnwE+C9wIrOHtO6eNZIJGKbzf9ryUuSBcIESmMRdEKRUHTi9W7oCDOcdMi8gY8FbOOktERoG1yxznQN7yRTJXZUtxVCmVzFkeAo6rK/tPh4DrlzlOtY69bF0ppSZE5HHgfwUesa/c/wvw+wAiso7Mg8O/EZG/zjmG2D/vBPYXI1NGe+zJW34F+Km8dQv1qZRK2m2gqwKfnc9NwDfz1r0A1JOpm1P2un3KjtgrGT+wL87rZG6zU8AlpdR8TtnngT/PWR7M+T37EPU37P2TZK7aARCR68lc4X0D+H+AMTKNeDeZL3YhrOzueetz+3UNYJxM10E++Q+IFhCRDwGPLVZu87xS6ueX2SaZt6wWWbdc91++azH7lPLZFkv/Pcs59nIUW1ffJNM90wH8NNAA/HvOMQDuA14scJwLxYiU2B4XPVyBdYvWZ4U/O3vsQj6562dKOG7N4Qf2xYkrpQpeySqlRoCRRfb7NzJ9j78MfBr4jlJqOqf8NjJXa7+ZveoTkZ9cxiX7WeuzK+wHTLlXsvuADjK3nceXOV4ur5PpI12KWQfHqxVGyPl72txSpc8utq52AVEy/cL/EfiRUmrSLhsk0010g1Lq62W4lNIes3wAeCpn+XaWuDus8GfncxT4IJlnOVnuINN2z5V4zJrFD+wVRik1LSLfIdNf20rmVjqXk2SuJH5HRB4kE1SXfMFCKRUTkdeB3xORU2SuZv4YyH06/xSZ/sAfisjvkekaaSNzaxxb7Mtfwa6YWuMZ4P8VkY+R+Vt9nExgGqvCZxdVV3bXxXeAXyeT4vex7AGUUkpEfh/4qohMAT8kc3d5E/ARpdQVGUhL4Lg95vB/iMgJMl0+nyLTxVTs55b72fl8Gfi+iBwg89D+J4D/DnxFKZUq8Zg1i58V4w5fIxPUDyil9uUWKKX2k8kg2UnmKuO3gP9WxDF/hczDtB7g22QyHxbuGux+w58j8wX/a+AEmdvcjwJ9ZZ3NyuSfyHSb/T2Zh37dlJZN4hiHdfVNMv3PUeCJvOP8M5mr+bvJ3AXsJdNNeNGBS6ntEeD3gP+TzLOUTwK/pJTqrdJn5x/rUTLPtv434AiZrtK/Ab601H4rFVkFzxF8fHwqiIi8k8zDyNuVUq957eNzNf4Vu4+Pj88Kww/sPj4+PisMvyvGx8fHZ4XhX7H7+Pj4rDD8wO7j4+OzwtAij33Pnj2qrq7Oaw0fHx+fmmJ2dnZs27ZtnfnrtQjsdXV1bNmypaR9+/v72bhxY4WNysf3cobv5Qzfyxm6ekF5br29vVcNPQwroCsmGHQ0DHbV8L2c4Xs5w/dyhq5e4I5bzQf2lpYWrxUK4ns5w/dyhu/lDF29wB23mg/sY2PVGNrDOb6XM3wvZ/heztDVC9xxq/nArut/Yt/LGb6XM3wvZ+jqBR5esYvIXSJyQkROS84clQW2+5g9B+GtOes+Z+93QkTurIR0LonEokONe4rv5Qzfyxm+lzN09QJ33JbNirEng32AzJyPA8BeEXlUKXU0b7smMjMFvZ6z7ibgXjJzO64HnhGRG5RS6UqdQDwer9ShKorv5Qzfyxm+lzN09QJ33Iq5Yr+NzDyQffZg+A8C9xTY7g+Br3DlGOH3kJkLcl4pdZbMuN+3lel8Bd3di05o7im+lzO085qbxzg/xIZ5hVwag3TFrkUqgnZ/LxvfyzluuBWTx34NV06zNQBcMcuJiLwPuFYp9ZiI/G7evq/l7XtN/geMjIywY8cOAoEA6XSa7du3s3PnToaGhmhoaMA0Taampujs7GRiYgKlFJ2dnQwPDxOPx+ns7CQWi9HV1cXo6CgiQltbG6OjozQ3N5NOp5mZmaG7u5uhoSGCwSAtLS2MjY3R0tJCIpEgHo8vlIdCIZqamhgfH6e1tZV4PM7c3NxCeTgcJhKJMDk5SXt7O9PT0yQSiYXybFk4HKajo4NoNEoymVwoX+6cGhsbAVw5p76+Pq677rqSzikUChGNRl05p3g8TkdHR9Xr6Ypzmpyk/sR5Os8OE7iUeaCVfW1OmQaxa9cyeeNGmrbe4Ho9LXdO586dY9OmTVWvp+XOaW5ubsHbtXoq4ZxGR0eJRCJaxYjsOSUSCRoaGkqqp8VYdhAwEfk4cKdS6tP28i8Dtyml/qu9bADPAb+ilDonInuA31VK7RORB8jMqv4te9t/AnYppR7J/Yyenh5V6gtKly5dYt265eY6rj6+lzO89jIGhgk+9hLGaGbmuaQhXGisA0MIJ1Ksn317mtPUTZtI3nk7NNZ7pev532sxfC/nlOPW29v75rZt227NX1/MFfsAcG3O8gaunLy5CdgK7LFnXO8GHhWRu4vYt2yampoqebiK4Xs5w0svc99Rgk++iijFSH2Ib25o5Kn2MAlT6MZgCIuO+TR3D83wXy7GCB/tQy4MkfzPH0Gtv+pt7qrg16MzdPUCd9yK6WPfC2wWketFJETmYeij2UKlVFQp1aGUuk4pdR2Zrpe77SnhHgXuFZE6e0byzWSmIasY4+PjlTxcxfC9nOGJl1IEnn2D0BOvIErx6LUt/OLNHTy2NkLCzExwf4OReStwrM7k6xub+dQta3mrOYQ5PUvoGz/C6Ct6FrqK4tejM3T1Anfclg3s9kSw9wG7ycxA/pBS6oiI3G9flS+17xHgITLzGT4J7KxkRgxAa2trJQ9XMXwvZ3jhFXjhTYKvHiQtwp9taecvNjYuBPQsfXnzIA+FA/zW1g6eWFuPkU4T+PfdGP2XqqkN+PXoFF29wB23ovLYlVK7lFI3KKV+TCn1R/a6z9sTyOZv+6HcCZyVUn9k7/cupdQT+duXi65pTL6XM6rtZR44QfCl/VgCf3JjO491hAtu11bgK5IyhD/dvIbHu+oxU2nMf9+NjEfdVr4Cvx6doasXeJfuqDVzc3PLb+QBvpczqullDAwTfPxlAP5ucxtPtS0+ZPQaKfwVUSL8+TvX8FJbmMB8EvPBJ2G+ei/B+PXoDF29wB23mg/suuan+l7OqJrX3DzB7z2HWBa7NjTzyNrIkpv3WosHa0uEP7qhlb76AMGJKcxdL1fadlFWfT06RFcvcMet5gP70NCQ1woF8b2cUS2v4K5XMKIxzjaH+fN3LJ0LDHCLEVqyPB4w+MKWNuYMIXT4DObh05VSXZLVXo9O0dUL3HGr+cAeDhfuG/Ua38sZ1fAyjp8jcOQMCdPg9ze3kDZk2X0uK2vZbc7XB/m76+2BnHa9DDPu9+eu5nosBV29wB23mg/skcjSt9Je4Xs5w3Wv+QShJ18F4OvXt3AxUtzkYRMsH9gBftRdz941dYTmkxi7e0rWLJZVW48loqsXuONW84F9cnLSa4WC+F7OcNsruOdNZHqGMy1h/r2r+C/SJily9kgR/vLH1jBvQN2RMxhn3c1vX631WCq6eoE7bjUf2Nvb271WKIjv5Qw3vWTsMua+I1gCf3x9E5Ys3wWT5aSVXH4jm8FIgH+5thkA64lXwCruar8UVmM9loOuXuCOW80H9unpaa8VCuJ7OcNNr+AzryGW4tl1zZxuXPphaD7rDdPR9g9d08ilOpPIeBRj/wlH+zphNdZjOejqBe641Xxg13UAfd/LGW55GecGMU9dYC5g8MAG54N2NTr8iiQM4WvXZa7a1fNvQDK1zB6lsdrqsVx09QJ33Go+sOuan+p7OcMVL6UIPL8XgO9uaGYy5OzqG5bOY1+M5zsiHG8MEo4nMN48uvwOJbCq6rEC6OoFfh57QXTNT/W9nOFKLu/pC5gDI8RCJv/WXVrmwXJ57AUR4RvvyIzYl355vytX7aupHiuBrl7g57EXRNc0Jt/LGRX3Uorgi70AfGdDM/FAaU19vMQx63pawxxvDFIfT4ALV+2rph4rhK5e4Kc7FiQUKuGKqgr4Xs6otJfRfwljcJRYyOS7XaW/ABJj6YloFkWEb16buWpXLly1r5Z6rBS6eoE7bjUf2KPR6o6qVyy+lzMq7RV49SAA31/fxLxZejPfWGweewFebQtzoiFz1V7pvvbVUo+VQlcvcMet5gN7R0eH1woF8b2cUUkvGRrHPDPAvGnwUBlX6wDHHOSxXy2S19dewQmxV0M9VhJdvcAdt6ICu4jcJSInROS0iHy2QPlnROSQiBwQkZdF5CZ7/XUiErfXHxCRr1b6BHT9T+x7OaOSXoGetwDYva6BqaDzTJhcNjrMY8/n1bYwffWBzFX7kb6yjpXLaqjHSqKrF3h0xS4iJvAA8FHgJuAT2cCdw7eVUu9RSr0X+ArwlzllZ5RS77V/PlMp8SzJZBlXVC7iezmjUl5yeRrzyBnSAt/qLn+y6fpyb2pFeHh9ZhTJuVf3wzKTxxfLSq/HSqOrF7jjVkyrvQ04rZTqU0olgAeBe3I3UEpN5Sw2QKlPnJyja36q7+WMSnkFXj+EKMUrXU0Mh0vvH89SSh57Pk931nM5YNAyGsUYGCn7eLDy67HS6OoF3uWxXwNcyFkesNddgYjsFJEzZK7YfyOn6HoR2S8iL4jIT5dlWwBd81N9L2dUxGs+gXngJAD/vK4yKWQl5bHnkTCFH9l3D3Ov7C/7eLDC69EFdPUCd9yKuaQpNGLSVVfkSqkHgAdE5JPAHwCfAi4B71BKjYvITwA/EJF3513hMzIywo4dOwgEAqTTabZv387OnTsZGhqioaEB0zSZmpqis7OTiYkJlFJ0dnYyPDyMZVmMj48Ti8Xo6upidHQUEaGtrY3R0VGam5tJp9PMzMzQ3d3N0NAQwWCQlpYWxsbGaGlpIZFIEI/HF8pDoRBNTU2Mj4/T2tpKPB5nbm5uoTwcDhOJRJicnKS9vZ3p6WkSicRCeTYvtb+/n46ODqLRKMlkcqF8uXNqbMzcurtxTrOzs0xNTZV0TqFQiGg06so5KaUYGxsrq54Cbx4jkkhyak2EDU2NNCqLCSw2SYCTVpL1hkkjBr1WgluMEOMqTQzFRglwzEqy0TCpzykfUWkExR1GHYetJJuNAAHgsJXkZiPEJTvHfZ2YHLQSbDWCpIBTVoqtRpABlSKEsFZMnl7XxCcvxmg6fYHLQyOMzMfLanuzs7PEYrGq19NybS/7fa7096ncc7Isi/7+fq1iRPacTNNkcHCwpHpaNGirZfr8ROR24ItKqTvt5c/ZgfzLi2xvAJNKqZYCZXuA382d7Bqgp6dHbdmyZUmPxZicnNRyBnLfyxlleylF3T9+D2Nkgi+9q42nOytzxb5JTPpKfEkpn88fn2DbWJyZD2zF+NnbyzrWiq1Hl9DVC8pz6+3tfXPbtm235q8vpitmL7BZRK4XkRBwL/Bo7gYisjln8eeAU/b6TvvhKyKyCdgMVC41AJiamlp+Iw/wvZxRrpcxMIIxMsF0yGRPe+VmpNlQRh57Pt+9JnOVZfYeL/uFpZVaj26hqxe447Zsq1VKpUTkPmA3YAJfV0odEZH7gX1KqUeB+0Tkw0ASmCTTDQNwB3C/iKSANPAZpdREJU+gs7OzkoerGL6XM8r1Mu0XgJ7sbiBZxJR3xXK4nDz2PI41hTjeGGRLLMn88bNY79m8/E6LsFLr0S109QJ33Iq6HFFK7QJ25a37fM7vv7nIfo8Aj5QjuBwTExPU15ef1lZpfC9nlOU1E8c82ocFPLy2suNubDYCvF6BzJgsj3U1sCV2mfgbh6grI7CvyHp0EV29wB23mn/zdLlnBF7hezmjHK/AwZNI2uJgRz1DFUhxvOLYFT0aPNsZIW4IawbHkfHSX0xZifXoJrp6gTtuNR/Ydb3F8r2cUbKXUpj7jwPw0NrKz/Zeya4YgNmAwZ6OzF1Fqrf08WNWXD26jK5e4I5bzQf24eFhrxUK4ns5o1Qv48IQxsQUl+sCvNZa+cB+cwXy2PN5zM5plwMnIF3avKgrrR7dRlcvcMet5gP7cvmcXuF7OaNUL/Ng5oWkZ7oaHE1SXSyXKpTqmMvhphDnIgEa5pIYp86XdIyVVo9uo6sXuONW84HdZxWTSGIePQvADzrqPJZxgAiP21ft8b2HPJbxWYnUfGCPxWJeKxTE93JGKV7m8XNIIsmpljAX6oMuWGXeKnWD3WvrSQo0nxuC6RnH+6+keqwGunqBO241H9i7urq8ViiI7+WMUryy3TCPdVa+bz3LwQqmOuYSDZr0tIUzX8BDpxzvv5LqsRro6gXuuNV8YB8dHfVaoSC+lzOcesnlacxzgyQN4ZkO9+az3Gq4cycA8FSnPTCYndXjhJVSj9VCVy9wx63mA7u48MCsEvheznDqZb6Vucp9vbOBWIkTVRdDZWcqvZLX2sJMm0LrxDQy4uyF7JVSj9VCVy9wx63mA3tbW5vXCgXxvZzhyEuphcD+fZcfmp6y3AvtSUN43h6sLHnA2VX7iqjHKqKrF7jjVvOBXddbLN/LGU68ZHAUY3KKaF2A3jXuBnY3u2Lg7e4YDp1yNLvSSqjHaqKrF/hdMQVpbm72WqEgvpcznHgFjpwB4MXOeldy13MZUG52xsCh5hCX6kyaZhMY/ZeK3m8l1GM10dUL3HGr+cCeruDM75XE93JG0V6WhXk0M/Lzrjb3c9dDBeeZqSAiPLXWfojqYIiBmq/HKqOrF7jjVvOBfWbGeQ5wNfC9nFGsl3FhGJmeZSwS5GiTu90kAGtdymPPJTspSPhEf9HjtNd6PVYbXb3AHbeaD+y6TlLrezmjWC/z8GkAnuushypkOlRiMuvluFAf5HhjkLqUhXG6uCEGar0eq42uXuDdZNZao+sktb6XM4rySluYxzJDCDzRXp0hBCoxmXUxPGfn4s8cOFHU9jVdjx6gqxe441ZUYBeRu0TkhIicFpHPFij/jIgcEpEDIvKyiNyUU/Y5e78TInJnJeUBgkH3b8dLwfdyRjFextmLSHyei40h+uorPVJ6YWYpbfRFp2SH8m08OwiJ5YcKruV69AJdvcAdt2UDuz1n6QPAR4GbgE/kBm6bbyul3qOUei/wFeAv7X1vIjNH6ruBu4C/z86BWilaWq6aM1sLfC9nFONl2tkwz3ZEqtINA9BvVeeh23A4wJGmIKG0hRQx4mMt16MX6OoF7rgVc8V+G3BaKdWnlEoADwL35G6glMqdjbUByCbk3gM8qJSaV0qdBU7bx6sYY2NjlTxcxfC9nLGsVzKFefwcAE9WcLLq5bjR5Tz2XJ7ryGTHxIp4Walm69EjdPUCd9yKCezXABdylgfsdVcgIjtF5AyZK/bfcLJvOej6n9j3csZyXsbpC0giydnmOi5GqtMNA9Dvch57Li/Y3TEt5y7B/NIPbWu1Hr1CVy9wx62Yb0ihe96rXpFTSj0APCAinwT+APhUsfuOjIywY8cOAoEA6XSa7du3s3PnToaGhmhoaMA0Taampujs7GRiYgKlFJ2dnQwPD5NOp0mn08RiMbq6uhgdHUVEaGtrY3R0lObmZtLpNDMzM3R3dzM0NEQwGKSlpYWxsTFaWlpIJBLE4/GF8lAoRFNTE+Pj47S2thKPx5mbm1soD4fDRCIRJicnaW9vZ3p6mkQisVAeiUSYnZ0lGo3S0dFBNBolmUwulC93TtmB9904p8HBQQzDKOmcQqGQa+dkWRapVGrRc9poX8We7GikEeEWI0QMi0ErzQ1GkD6Vog2DNWLQayW4xQhxWVlMYLFJApy0kqw3TBp5u3xcpYmh2CgBjllJNhom9TnlIyrNOjHYKAEOW0k2GwECZKbLu9kILUzCsU5MDloJthpBUmSGIdhqBBlQKUIIa8VcOOYsFv1WmhuNIP0qRSNCu11+Y6SevuY6Nk3NM/LqmwR+4t2L1tPg4CCBQKDq9bRc21NKufJ9KvecotEo0WhUqxiRPSfDMJidnS2pnhYN2stNpCoitwNfVErdaS9/zg7kX15kewOYVEq15G8rIrvtY/Xk7tPT06O2bNmypMdi9Pf3s3HjxpL2dRPfyxlLes0nCP/lt5BUmo+9v5vROvdzy7PcYdTxojVftc/bPhjjN/uijF3XTcMv//yi29VkPXqIrl5Qnltvb++b27ZtuzV/fTFdMXuBzSJyvYiEyDwMfTR3AxHZnLP4c0B2gOlHgXtFpE5Ergc2A2+UcgKLoWt+qu/ljKW8zJPnkVSa462RqgZ1qE4eey4vtEewgDXnh5fsjqnFevQSXb3Aozx2pVQKuA/YDRwDHlJKHRGR+0Xkbnuz+0TkiIgcAH6bTDcMSqkjwEPAUeBJYKdSlZ1EUtf8VN/LGUt5mccyQwg8XaXc9VyqlceeZbzO5GBziIClEPthcSFqsR69RFcvcMetqKdQSqldwK68dZ/P+f03l9j3j4A/KlVwOUKh6n7xisX3csaiXokkxpkBAJ5vrV42TJZYlfLYc9nTGeF9UwliB4/TcPMNBbepuXr0GF29wB23mn/ztKmpyWuFgvhezljMyzh9AUmlOdUSZrzK3TAAg1XKY8/lhfYIaWDN+RGYK9y/X2v16DW6eoE7bjUf2MfHx71WKIjv5YzFvLK568950A0DcEMV89izTIZMDrTUEVAKsYdQyKfW6tFrdPUCd9xqPrC3trZ6rVAQ38sZBb1SaUz7Lczn26rfDQPQV8U89lyyMyvNvFV47JiaqkcN0NUL3HGr+cAej8e9ViiI7+WMQl7G2YtIIkl/Ux2XwtV7KSmXNo++Ii+1hUkDLRdGYe7q7Jhaqkcd0NUL3HGr+cA+NzfntUJBfC9nFPIyj2e6IfZUYUKNxVgj3nxFLodM3moOZbpjTvVfVV5L9agDunqBO241H9h1zU/1vZxxlZdlYZ7IBLRnqzg2TD7VzmPP5aXsUL6HTl5VVjP1qAm6eoE/HntBdM1P9b2cke9l9F9C4vNcagjRX8WxYfKpdh57Li/a/9Cazg1dNbNSrdSjLujqBR6Ox64z4bB3V3NL4Xs5I98rmw3zQlu4akP0FuKyqn4ee5bRugBHG+2hfM9cuKKsVupRF3T1Anfcaj6wRyIRrxUK4ns54wovpTBPnAPgWY/SHLNMePCCUi4vZrtj3rqyO6Ym6lEjdPUCd9xqPrBPTk56rVAQ38sZuV5ycQSZnmU8HOBkg7cz32wS77qBAF6yu2Ma+i5Czmz2tVCPOqGrF7jjVvOBvb293WuFgvhezsj1ynbDvNRRnQmrl+Kktfw0dW4yEAnSVx8gnEwjZwcX1tdCPeqErl7gjlvNB/bp6WmvFQriezljwUuphTTHZ1u9H99jvVH9YQzyeaE9c6s+m5Mdo309aoauXuCOW80H9kTCu3S0pfC9nJH1kuEJjMlpoqEAh5u9D+yNGnxFsv3sdacugJVp/T/AAAAgAElEQVTp89e9HnVDVy9wx837Vlsmuuan+l7OyHplr9Zf7QhjedwNA97msWfpqw8wEDZpmE9iXBgG9K9H3dDVC/w89oLomp/qezkj65XtX3+m1dtsmCxe5rEvIMJLdndM/HCmO0b3etQNXb3Az2MviK5pTL6XMyKRCDJ2GWN0ktmAwYEWPQL7eGXnhSmZF+3Abh4/B0ppXY86oqsXeJjuKCJ3icgJETktIp8tUP7bInJURN4SkWdFZGNOWVpEDtg/j+bvWy66DqDvezkjFAot5K6/3lFPyvC+GwYgdvXc655wrCnISMigaTaBXBrTuh51RFcv8GiiDRExgQeAjwI3AZ8QkZvyNtsP3KqU+nHgYeArOWVxpdR77Z+7qTDRaLTSh6wIvpczotEopj32+NMaZMNk2ehxHnsWJcLL9lV74shpretRR3T1Anfcirlivw04rZTqU0olgAeBe3I3UEo9r5SatRdfAzZUVnNxOjo6qvVRjvC9nLE2FMG4NMa8abB3jT6vfx/zOI89l2x3jHXkDB2a5mXr2r509QJ33IoJ7NcAuQNVDNjrFmMH8ETOclhE9onIayLyn0pwXBJd/xP7Xs6wDp0C4M32CAlTj24YgI0a5LFneaslxOWAwZrpOLP9F73WKYiu7UtXL3DHrZj7zELfsoIdjyLyS8CtwAdzVr9DKTUoIpuA50TkkFLqTO5+IyMj7Nixg0AgQDqdZvv27ezcuZOhoSEaGhowTZOpqSk6OzuZmJhAKUVnZyfDw8PE43Hq6uqIxWJ0dXUxOjqKiNDW1sbo6CjNzc2k02lmZmbo7u5maGiIYDBIS0sLY2NjtLS0kEgkiMfjC+WhUIimpibGx8dpbW0lHo8zNze3UB4Oh4lEIkxOTtLe3s709DSJRGKhPBKJMD09TX9/Px0dHUSjUZLJ5EL5cufU2NgI4Mo5jY2N0djYWNI5hUIhotGoK+e0wZ6weqCtgUaEW4wQs1j0W2luNIL0qxSNCO1i0msluMUIEcNi0EpzgxGkT6Vow2CNGAvll5XFBBabJMBJK8l6w6SRt8vHVZoYio0S4JiVZKNhUp9TPqLSXINJvWFw2Eqy2QgQAA5bSW42QlyyH6yuE5ODVoKtRpAUcMpKsdUIMqBShBDW5jiXdU5mHcfa67l9OAZHzhDbeE3V62m5tjc3N8fIyEjFv0/lntPU1BTJZFKrGJE9p0QiweDgYEn1tGjQVmrph0MicjvwRaXUnfby5wCUUl/O2+7DwN8CH1RKjSxyrG8AjymlHs5d39PTo7Zs2bKkx2LMz89TV6dHBkUuvpcDYrOE/+rfSBvCz9/WzWxAn2StRkSbB6gAPzkxx1eOjjPZ2kj4vk94rXMVWrYv9PWC8tx6e3vf3LZt263564v5Bu0FNovI9SISAu4FrshuEZH3AV8D7s4N6iLSKiJ19u8dwE8BR0s6g0XQNT/V9yoe82Q/ArzVFtEqqIMmeew59K6pY8YUWidjyOSU1zpXoWP7An29wKM8dqVUCrgP2A0cAx5SSh0RkftFJJvl8mdAI/DdvLTGG4F9InIQeB74E6VURQN7Q0NDJQ9XMXyv4sm+lKRTNkyWEU3y2LMkDaHHntjbOtbnsc3V6Ni+QF8vcMetqFwupdQuYFfeus/n/P7hRfZ7FXhPOYLLYZr6PNzKxfcqkvg8xtmLWAKvtOmTDZMloVE3TJYX2yN8eDTO3KFTRP7De73WuQLt2peNrl7gjpte970lMDWl3+0o+F7FYp46j1iKC2vqiQb1+/Jt0CSPPZc3WutIGkLbyGWYnl1+hyqiW/vKoqsXuONW84G9s7PTa4WC+F7FkR3065k2PR9sHdYojz1L3DTotXP9xf776YJu7SuLrl7gjlvNB/aJiQmvFQriexVBIolhpzle6lg6fcsrNhv6XbEDnO3M9MvOHj65zJbVRav2lYOuXuCOW80H9uXSNb3C91oe4/QFJJXm1Jows3V6BlA9reBUWz0pgZaLYxCf81pnAZ3aVy66eoE7bjUf2HW9xfK9liebDbOnNaxllwfo2RUDsNe02N9Sh6nAONHvtc4COrWvXHT1Ar8rpiDDw8NeKxTE91qGVBrz1HkAnm2v42bN8sWz6OyVHTtm9vApj23eRpv2lYeuXuCOW80H9uVerfUK32tpjL4BJJHkfFMdl8KBhdfzdUNnr5fbw1hAU/8QJPS4s9ClfeWjqxe441bzgd2nNsl2w7zQrl/ueq0wETI53BwiYCkM++7HxwdWQGCPxWJeKxTE91qCtIVp9ws/bac5rhP9cthBf6+X7H+McU26Y7RoXwXQ1Qvccav5wN7V1eW1QkF8r8Ux+geRuXkuNYTorw8CcFCDSaMLobtXtp890ncRUikvlQA92lchdPUCd9xqPrCPjo56rVAQ32txsjMlvdjx9lyPW42gVzpLorvXUDjAyYYgdSkLo8/7Mdp1aF+F0NUL3HGr+cAuos+kDLn4Xotg5XTD5Az65f21ZmFqwetFuztmXoPuGM/b1yLo6gXuuNV8YG9ra/NaoSC+V2GMC8PITJzR+iCnGt6+Gj5l6RlCa8Ere+cTPHUBLMsrJcD79rUYunqBO241H9h1vcXyvQqTHRvm5Y56yLlS0b3LQzdyvfojAfojASKJFEb/JQ+tvG9fi6GrF/hdMQVpbm72WqEgvlcBlMKw0xx35429PqD0vDKuCS+RheyY+cOnPTLK4Ld757jhVvOBPZ3W8wUS3+tqZHAUY2qGyXCQ441XXgmHCk6t6z214pXNjjFOnAMPx0Xx271z3HArKrCLyF0ickJETovIZwuU/7aIHBWRt0TkWRHZmFP2KRE5Zf98qpLyADMzM5U+ZEXwva4mmw3zakc9Ku+B0VpN88VrxetEY5ChOpPGeAK5WHDK4argt3vnuOG2bGAXERN4APgocBPwCRG5KW+z/cCtSqkfBx4GvmLv2wZ8AfhJ4DbgCyLSWjl96O7uruThKobvlYdSC/3rT7Ze3W/dq2m+eM14iSxkx6SOnPHAKIPf7p3jhlsxV+y3AaeVUn1KqQTwIHBP7gZKqeeVUtmpXF4DNti/3wk8rZSaUEpNAk8Dd1VGPYOuk9T6XlciwxMYk9NMhwIcbr56YC3dJo3OUkteL9ndMdaxM551x/jt3jluuBUz3PQ1wIWc5QEyV+CLsQN4Yol9r8nfYWRkhB07dhAIBEin02zfvp2dO3cyNDREQ0MDpmkyNTVFZ2cnExMTKKXo7OxkeHiYRCLB+Pg4sViMrq4uRkdHERHa2toYHR2lubmZdDrNzMwM3d3dDA0NEQwGaWlpYWxsjJaWFhKJBPF4fKE8FArR1NTE+Pg4ra2txONx5ubmFsrD4TCRSITJyUna29uZnp4mkUgslEciEVKpFP39/XR0dBCNRkkmkwvly51TdlAgN85penqaqampks4pFAoRjUZLOqf1R/oJA6c7GmgSg61GkBSZtL2tRpB6YIsEWCsmvVaCW4wQs1j0W2luNIL0qxSNCO055TEsBq00NxhB+lSKNgzWiLFQfllZTGCxSQKctJKsN0waebt8XKWJodgoAY5ZSTYaJvU55SMqTT1wh1HHYSvJZiNAgMxQvjcboYUBwtaJyUErcdU5DagUIcSVc1ovBt0YV5xT65ompoITNE/PcfbgEZo2X1/1tpdKpRgZGan496mcttfY2EgikaC/v1+rGJE9J6UUg4ODJdXTYshyg7yLyMeBO5VSn7aXfxm4TSn1Xwts+0vAfcAHlVLzIvJ/AXVKqS/Z5f8dmFVK/UXufj09PWrLli1LeizGzMyMljOQ+15XUvcP38UYu8zvvLudfa1XD/zVicEo3uZgF6LWvH7n9CR3D80S+w8/jrltqesvd/DbvXPKcevt7X1z27Ztt+avL6YrZgC4Nmd5AzCYv5GIfBj4feBupdS8k33LYWxsrJKHqxi+19vI6CTG2GVmgib7WwrPbXqjpvniteb1gt0dkzpy2pPuGL/dO8cNt2IC+15gs4hcLyIh4F7g0dwNROR9wNfIBPXcR/K7gY+ISKv90PQj9rqK0dLSUsnDVQzf622yD03f6KgnbRROH+zXNF+81rz2r6njcsBgTXQWGZmsspXf7kvBDbdlA7tSKkWme2U3cAx4SCl1RETuF5G77c3+DGgEvisiB0TkUXvfCeAPyfxz2Avcb6+rGImEnlkLvtfbmMfOAbC7QDZMlkZN88VrzSstwosdma6utAdjx/jt3jluuBU1V69SahewK2/d53N+//AS+34d+HqpgssRj8fdOnRZ+F4ZZHIKY3icuYDBvjWLT6rRLiZoeHVci17Pd0S4e2iW9OHTGD9z2xVDN7iN3+6d44Zbzb95qmt+qu+VwTzaB8C+9nqSi3TDQA3li2vCUl4HW+qYDBo0T80iwxW9QV4Wv907x6s8dq3RNT/V98pgHskE9l1tS+eD11K+uA4s5ZUWWRhiIF3lsWP8du8cN9xqPrCHQnp+8XwvkLHLGMPjxAMGbxRIccwlpmFKIdSu1/P2UL7W4VNVzY7x271z3HCr+cDe1NTktUJBfK+3u2Fe62xYshsGYNDSc5CmWvU62BJiImjQNB1HhsarZOW3+1Jww63mA/v4ePUarRNWvZdSmIczY5Y8vkw3DMANmuaL16qXJcILHdnumOplx6z6dl8CbrjVfGBvba3omGIVY7V7ycgExvhlYiGT3kVeSsqlT8PME6htr7e7Y6r3stJqb/el4IZbzQd2XdOYVrtX9qHpq50Ni76UlEubpk2xlr0ONYcYDxo0xeaQS9V583K1t/tS8NMdCzA3N+e1QkFWtZdSmEftbpjW4h4MrRE9m2Ite+V2x6QOVac7ZlW3+xJxw03PVusAXfNTV7OXXBrDmJwmWhfgrZbiAnst5ot7SbFe2e4YVaWxY1Zzuy8VP4+9ALrmp65mL9Oe6OGlzgasIt96rMV8cS8p1utQc4iRkEHTzDzGgPszK63mdl8qfh57AcLhpfOjvWLVeim1kOa4q634jJLLSs988Vr3UiI821kPQOLAMTeVgFXc7svADbeaD+yRSMRrhYKsVi/jwjDG1AzjkSBHmoq/2p3Q9EWgleD1tB3YA0fPgsuTOq/Wdl8ObrjVfGCfnKz+0KTFsFq9st0wL3TUOxp8apMUNR5d1VkJXmcaAvTVB4gkUhhnBly0Wr3tvhzccKv5wN7e3u61QkFWpZdlYR7LjL3+RLuzvumTVtINo7JZEV4iC1ft8/vd7Y5Zle2+TNxwq/nAPj097bVCQVajl3FmAJmJM9QQ4mSDszc21xumS1blsVK8nunM3O5HTg/AvHuZPqux3ZeLG25FBXYRuUtETojIaRH5bIHyO0SkV0RSIvKxvLK0PfnGwgQclUTXAfRXo5d5KDOS4FNrnXXDADRqeo2xUrxGwgH2N4cIWgrDvqtyg9XY7svFDbdlW4eImMADwEeBm4BPiMhNeZudB34F+HaBQ8SVUu+1f+4uUF4Wuuanrjqv+QTmiXMAPN6+/BAC+dR6vni1KcXrmbWZ7pg5F7tjVl27rwBe5bHfBpxWSvUppRLAg8A9uRsopc4ppd6C6qcQ6Jqfutq8zGNnkVSa460RhsLOHzjWer54tSnFa09HhIRA08AoTM+4YLX62n0l8CqP/RrgQs7ygL2uWMIisk9EXhOR/+TIrgh0TWNabV6m/cr6ro7ScnLHlZ7D464kr1jAoKctjAHIIXcm4Fht7b4SuOFWzKVVoc5SJ+8mv0MpNSgim4DnROSQUupM7gYjIyPs2LGDQCBAOp1m+/bt7Ny5k6GhIRoaGjBNk6mpKTo7O5mYmEApRWdnJ8PDw4gI4+PjxGIxurq6GB0dRURoa2tjdHSU5uZm0uk0MzMzdHd3MzQ0RDAYpKWlhbGxMVpaWkgkEsTj8YXyUChEU1MT4+PjtLa2Eo/HmZubWygPh8NEIhEmJydpb29nenqaRCKxUB6JREgkEvT399PR0UE0GiWZTC6UL3dOjY2NAK6c0+TkJOFwuKRzCoVCRKPRq85p7Mw5fuzcJVKGEO9spg2LzUaAAHDYSnKzEeKSHYjWiclBK8FWI0gKOGWl2GoESaDYQoC1YtJrJbjFCDGLRb+V5kYjSL9K0YjQnlMew2LQSnODEaRPpWjDYI0YC+WXlcUEFpskwEkryXrDpJG3y8dVmhiKjRLgmJVko2FSn1M+otI0icEdUsdhK+n4nAZUihDizjkZAcbTluNzOrq2iQ+Oz5HYd5jkezdXvO2ZpsnIyEjFv0+Ltb1iv09zc3P09/drFSOy51RXV8fg4GBJMWLRoK2WGT9CRG4HvqiUutNe/hyAUurLBbb9BvCYUurhRY5VsLynp0dt2bJlSY/F6O/vZ+PGjSXt6yarySvwygGCz+3ltbWN/N4NLSUd4w6jjhet+Yp6VYKV5hW0FN9//RJNacXcr/0Cam1bRb1WU7uvFOW49fb2vrlt27Zb89cX0xWzF9gsIteLSAi4Fygqu0VEWkWkzv69A/gp4Gjx2svT0dFRycNVjFXjpRTmW5lumB86zF3P5Zim+eIrzStpCHuyE3AcPFFJJWAVtfsK4obbsoFdKZUC7gN2A8eAh5RSR0TkfhG5G0BE3i8iA8DHga+JyBF79xuBfSJyEHge+BOlVEUDezQareThKsZq8ZLBUYyxy0yHzGXnNV2KjZrmi69Er6fs7BgOngSrsvkOq6XdVxI33IpKX1BK7QJ25a37fM7ve4ENBfZ7FXhPmY5LkkzqeUW1WrwCBzJXfc91NZAqYkKNxajXNF98JXq91RziQtjk2niC+TMDWJvfUTGv1dLuK4kbbnq2Wgfomp+6KrySqYWxYb5XYjZMlpWUL14NyvISYVdXAwDz+44ss7EzVkW7rzD+eOwF0DU/dTV4mUf7kPkkZ1rCnHM4hEA+KylfvBqU6/VkVz0poOHMRZip3NRsq6HdVxp/PPYCNDQ0eK1QkNXgZR48CcCP1pafhzuiab74SvWasJ+JmEohb52skNXqaPeVxg23mg/spqnnw62V7iUTUcz+SyRMg93t5U8UkHD0akT1WMlej3fbE3DsO1qxafNWert3Azfcaj6wT01Nea1QkJXuZR7IXOW9uraB2UD5zWiDpuOer2SvntYwE0GD5ssxZHC0AlYrv927gRtuNR/YOzs7vVYoyIr2siwC9u37Ix3OB/wqxGFN88VXslfaEHbbqY/JvYfLPh6s8HbvEm641Xxgn5iY8FqhICvZyzjZj0zPMtQQ4q3myjxc3GzoeWW80r0e78oE9rqjZ2Gu/Aygldzu3cINt5oP7MsNieAVK9kr8GZm2Ncfdjsfd33RY1bkKJVnpXtdqA/S2xIilLYwDpb/EHUlt3u3cMOt5gO7rrdYK9VLxqOYfRdJmMKjnZUblW4ld3m4QSW9frAuM6BU4o1DZT9EXant3k38rpgCDA8Pe61QkJXqZfZmrtZf7mokVoGHpllu1jRffDV4vdwWZixk0HQ5hnG+vJzqldru3cQNt5oP7MsNX+kVK9IrmSJgZ8M82Fl+imMulzTNF18NXmlDeNx+EzX52ltlHWtFtnuXccOt5gO7T/Uwj/Yhc/OcbQ5zoknPK1mf0vhRdwMpIHLqAsRmvdbxKZOaD+yxWMxrhYKsRK/AvszAnI90V37Gl3Wi5wskq8VrtM6kpy3zJipvlj4n6kps927jhlvNB/auri6vFQqy0rxkYBhjcJTZoMlTFXjTNJ+Dmg62tZq8frAu0x2j9h6BdGldPSut3VcDN9xqPrCPjlbmjblKs9K8Aq9nXmB5Yl0j82blm81Wo7xBxNxiNXntW1PH2foA9fF5jCN9JR1jpbX7auCGW80HdqlQHnWlWUlecnka89hZ0iJ8p8udSYFTrhy1fFaVlwjfXZ95kJd8dX9JqY8rqd1XCzfcigrsInKXiJwQkdMi8tkC5XeISK+IpETkY3llnxKRU/bPpyolnqWtrbJzNlaKleRl7j2CKEVPVyOjde70OZ+y9Ayhq83r6c56JoMGjaPRklIfV1K7rxZuuC0b2EXEBB4APgrcBHxCRG7K2+w88CvAt/P2bQO+APwkcBvwBRFpLV/7bXS9xVoxXvMJAr3HAfhmd+X71rOspi6PSuCWV8IUftBtpz6+st/x/ium3VcRr7pibgNOK6X6lFIJ4EHgntwNlFLnlFJvAfkTKN4JPK2UmlBKTQJPA3dVwHuB5ubmSh6uYqwUL3P/CSSR5HhrhJON7qU4Dig9r4xXo9cP1zUwL1B/5iIyftnRviul3VcTN9yKCezXABdylgfsdcVQzr5FkS7x6b3brAgvyyLwRuah6bfW1btklCGEnn2gq9FrMmTy9NrMrKpWj7MXllZEu68ybrgVM5ZQoRZU7FOVovYdGRlhx44dBAIB0uk027dvZ+fOnQwNDdHQ0IBpmkxNTdHZ2cnExARKKTo7OxkeHiYejyMixGIxurq6GB0dRURoa2tjdHSU5uZm0uk0MzMzdHd3MzQ0RDAYpKWlhbGxMVpaWkgkEsTj8YXyUChEU1MT4+PjtLa2Eo/HmZubWygPh8NEIhEmJydpb29nenqaRCKxUJ4tm5mZoaOjg2g0SjKZXChf7pyyb6K5cU6Dg4OEQqGizil8vJ/10RiT9SEm2xrYKLBRAhyzkmw0TOox6LUS3GKEGFFpEig2SIDDVpLNRoAAmTFNbjZCC29KrhOTg1aCrUaQFJm+4q1GkFa7qawVc+GYs1j0W2luNIL0qxSNCO055TEsBq00NxhB+lSKNgzWyNtOl5XFBBabJMBJK8l6w6Qxx3lcpYmhljynd0mAtWKWdE4DKkUIceWc3m+EuJy2SjqnYurp7IY2GJ4ldPAk5991DS3XrCuq7c3NzWFZVsW/T6FQiGg0WvL3aWxsjJmZGa1iRPacEokEiUSipBixaOBdbmQxEbkd+KJS6k57+XMASqkvF9j2G8BjSqmH7eVPAB9SSv2avfw1YI9S6ju5+/X09KgtW7Ys6bEY8/Pz1NVVZkzwSlLzXkpR99WHMcYu8xc3tPFoBaa/W4pGhJiGsxWtZq8vHR3npyfmmPvAe1A/+4Gi9qn5du8B5bj19va+uW3btlvz1xfTFbMX2Cwi14tICLgXeLTIz90NfEREWu2Hph+x11UMXSeprXUv49hZjLHLTESC7Opw76FplpU6abRbVMPrW9c2AWDuOwKzc0XtU+vt3gs8mcxaKZUC7iMTkI8BDymljojI/SJyN4CIvF9EBoCPA18TkSP2vhPAH5L557AXuN9eVzGCQT2zFmraSymCL2cyIh68ppGU4X4/8+xVz931YDV7HW8K8fqaOoIpC6PIvvaabvce4YZbUeP1K6V2Abvy1n0+5/e9wIZF9v068PUyHJekpaXFrUOXRS17GafOYwxPkBLYs6Y6V6z9lp4Pt1a7179e28RPXp7HfP0Q1k+9F8JLt4dabvde4YZbzb95OjY25rVCQWrWSymCLx8AIA0kq3C1DnCjpvniq93rUEsdl+pMAmkL4/VDy25fs+3eQ9xwq/nArut/4lr1Ms4MYFwcISUwY1Yv1a9f03xx3wv2t2Su0s1XDiw7L2qttnsv8a/YC5BI6Dn6Xk16KUXwuTcASAkVm8+0GBo1zRf3vWDQHkYikLYwlnkbtSbbvce44VbzgT0ej3utUJBa9DIPn8EYniBpCOEqPzNs13Tcc9+LK/7BB147tOREHLXY7r3GDbeaD+zd3d1eKxSk5rzSaQIv7Mv86sGM7r2ajnvue12JaSlkz75Fy2uu3WuAG241H9h1zU+tNS9z/wmMyWkShhD24H2c1ZwvXgpeetXtP4FMThUsq7V2rwOe5LHrTiik5xevprwSSYIv9mZ+9+BqHSCmab6473U1AqinegqW1VS71wQ33Go+sDc1NXmtUJBa8gq8fACZiTNvCCGP3p4f1DRf3PcqTOTkeYyB4avW11K71wU33Go+sI+Pj3utUJBa8ZKJKIHXMm8VBizvxkS5QdN8cd9rCX64B6wr7xxqpd3rhBtuNR/YW1srOm9HxagVr+BTPUjaIm4KXuZ/9GmaL+57LU7dxBTmgRNXrKuVdq8TbrjVfGDXNY2pFryMU+cxT10gLRBJezuCYZumTdH3WhrjqR6Ivz1AWC20e93w0x0LMDdX3Khz1UZ7r1SaoP0ALKHBRL9rRM+m6HstTSCZxnj2jYVl7du9hrjhpkfrKANd81N19wq8tB9jYoqEIUQ87FvP4ueLO0MXLwWE9p9A7Aepurd7HfHz2Auga36qzl5yaYzAKwdQgKFBUAc/X9wpuniJ/WM88iykUlq3e13x89gLEA67PwlEKWjrFQwR+tELiFLETSlu3OYqcFnpmS/uexVHaGoGc8+b+rZ7Tb3AHbeaD+yRiLtTtpWKrl4dR85hDE+QMIR6jx+Y5jKh6YtAvldxKCDY8xZN0cXHkfESXb+P4I5bUYFdRO4SkRMiclpEPlugvE5E/t0uf11ErrPXXycicRE5YP98tbL6MDk5WelDVgQdvWRonIY3jmQWNOmCybJJdLl3uBLfqziyXTL1u16BpPepmPno+H3M4obbsoFdREzgAeCjwE3AJ0TkprzNdgCTSql3An8F/GlO2Rml1Hvtn89UyHuB9vb2Sh+yImjnlUgSeuRZxFLEDUGPHtq3OWklvVYoiO/ljMh0nMDuV73WuArtvo85uOFWzBX7bcBppVSfUioBPAjck7fNPcA37d8fBraJVCeHbnp6uhof4xitvJQiuOtljIko8ZCpRRZMPusNPYfH9b2coYDg/hOYR854rXIFWn0f83DDrZjAfg1wIWd5wF5XcBt78usokP03dL2I7BeRF0Tkp8v0vQpdB9DXycs8eJLAodOkBYY79Bwzo1HTxz2+lzPG2hoAMH+4B5koPAKkF+j0fczHDbdiOuoKXXnnX/Itts0l4B1KqXER+QngByLybqXUFTU+MjLCjiO5kxkAABJdSURBVB07CAQCpNNptm/fzs6dOxkaGqKhoQHTNJmamqKzs5OJiQmUUnR2djI8PEw4HGZ8fJxYLEZXVxejo6OICG1tbYyOjtLc3Ew6nWZmZobu7m6GhoYIBoO0tLQwNjZGS0sLiUSCeDy+UB4KhWhqamJ8fJzW1lbi8Thzc3ML5eFwmEgkwuTkJO3t7UxPT5NIJBbKI5EI9fX19Pf309HRQTQaJZlMLpQvd06NjY0AFTmn9WYddbteBmA6HGQ+EuJyc4TZ+hDxcJBrBi9zcf0aInNJ6mcTjLc10DkWY6o5zOVIkHoRthp1jKs0MRQbJcAxK8lGw6Qeg14rwS1GiBGVJoFigwQ4bCXZbAQIAIetJDcbIS6pzKBV68TkoJVgqxEkBZyyUmw1goyrNFskwFoxF445i0W/leZGI0i/StGI0J5THsNi0EpzgxGkT6Vow2CNvO10WVlMYLFJApy0kqw3TBpznIs5p3GV5g6jrqRzGlApQogr5xQCujFKOien9dTU0shMZJ6hrmZapuKkTYNYQ91C2wkm07RenmWks4mG6TnidQEi8ylSDz5J/523EqyPlP19CoVCRKPRkr9P4XCY/v5+rWJE9pyam5sZHBwsKUYsGrTVMsO0isjtwBeVUnfay58DUEp9OWeb3fY2PSISAIaATpV3cBHZA/yuUuqKkfp7enrUli1blvRYjP7+fjZu3FjSvm6ihVdslrqv/xAjGiNuv4h05roOfuxccZPnTgQNdrxvLRMh92/77zDqeNGad/1znOJ7wS+fn+LT54vrLshvX8kbryf1C9uqOs1iIbT4Pi5COW69vb1vbtu27db89cXcz+0FNovI9SISAu4FHs3b5lHgU/bvHwOeU0opEem0H74iIpuAzUBfSWewCLqmMXnulUpR99DTGNEY8zlvl9bP6nlLOq70HB7X93JGbvtSQPDYWQLZsf49xPPv4xK44bZsV4xSKiUi9wG7ARP4ulLqiIjcD+xTSj0K/BPwryJyGpggE/wB7gDuF5EUkAY+o5SaqOQJ6DqAvqdeShF89EWMiyMkDaEu52FpXUK/VDSA2FW9e3rgezkjt30JdnB/sRfV1kL6Pe/0zEvXOAHuuBWVDKuU2gXsylv3+Zzf54CPF9jvEeCRMh2XJBqNsmbNGjc/oiQ881KKwLNvEDhyhrSA5GXATK6pp+2yfi+RbJQA/RpehfpezshvX9kOmMAP96AaIlib8vMuqoOucQLccdPz0boDOjo6vFYoiFdegRd6Cfa8hQJSXP2fe+2onmlfxzTNy/a9nLFY+zKUIvjgkxj9l6pslEHXOAHuuNV8YI9Go14rFMQLr8DL+wm+1IsC5gyoK3C3PrmmvupexbBR07xs38sZS7UvI20R/PYTBafUcxtd4wS441bzgT2Z1PPKpapeShF4aT/B5/dlgrpAZJGhRJJBPQNCvaZN0fdyxnLty0ilCXxrF8b56l656xonwB03PVuHA3QdZ7lqXpZF8MlXCe6xg7ohRJZ4rnbN4OXqeDlEl/HF8/G9nFFM+zKTKYL/ugvj2NkqGGXQNU6APx57QXQdZ7kqXqkUoUeeJbDvKJZkul+WGy7g4no9HyDpMr54Pr6XM4ptX4ZlEXr4Gcx9R102yqBrnAB33PQaIq4EGhoavFYoiNtecnma0MPPYFwaIy1CCrVo90sujTP6vWwDMKJhhgf4Xk5x0r4ECD3xCqmRCZIfuR0C7nUT6honwB23mg/spqlnn7GbXsap84R+sAeZmydpCIalqCvWK63XON5ZEprmZfteznDavhQQePMYMjhK8mMfRq1xZywjXeMEuONW810xU1P6DDSUiyteyRSBp1+j7sHdyNw8cVMIWgonzSLarOcbeBs0G188i+/lDKftK/sSk3lpjNA/PoJ5+AwsM8xJKegaJ8AdNz1bhwM6Ozu9VihIpb2M85cI/uhFjImptx+SljADUvewng38sKZ52b6XM0ppX9mXmIz5JKHvP0f6yBkS//P/BE2VS83VNU6AO241f8U+MVHREQoqRsW8YrMEH3uJum8+hjExRcIQ0iz/kHQxRjuWHhXOKzYbel5j+F7OKLd9KcA82U/dPzyE+cZhSFfmWYKucQLccdOzdThgudEpvaJsr2SKQM9bBF49iCRTKCBuCPVlTpKhDG9H2VsMXRui7+WMctvXFVfvu3uw3jhC8sO3Yb3rurJGiNQ1ToA7brq2j6LR9RarZK/4PIF9Rwi8cQSZncusMjPdLuUGdfC7Ypziezmjku3LAozJKeq++/+3d/Yxcpz1Hf98Z3bv/f28vrPjE3YSp26SJqGglJd/Kgyt26KgRkUEtQip7n+mpQIEhEh9AVWiQiovAtQ/SFUhIChqkzTiLeQFhIQMieLGbiw7gSQ2udj3ktvzvft2d+bXP2busj7fcTvn3Zu51fORVrc7Mzv32d2Z7z77zMzze4JwaJDKO24nuPkAeMk7GrKaE+C6YtZlfHz7L0+uhaReGp8i/6Of0/aVB8j/9Fm0eJllT1RgS33pG3FhuLdu66ont2f0vGznlYx6bl8r4RQC3vgULQ8/RevXHyR3/BQsLCVaV1ZzAhrjtuNb7JtVEkmLmrzmFvDPniN36ld4FyZXJy/5Ih/YFcPt1oueuct1X2c9uJjR87KdVzIasX1dEfDTc3hP/JLcU08T3rSfyq03EN44AvnfHmVZzQlojNuOD/YdRRii8Sn8l1/De/E8/ujE6qxAUFJUEKOeLXSHo1moDniFhn/2Ffyzr2D5HOENIwQHRwgPXIf1ZjfEt4sdH+zz8/MMDg5uvuA2Mz8/z2BHF97FSbwL8e03Y2jpjSvzQmA5bp3nDNq34QDPbHcbhan5hv+fpOyRz68se0VAnFcytmP7qu4/DojGnlkJeYBwoIdwZJhwb4FwbwHbPZDZnIDGZFhNwS7pCPBlogpK3zCzz6+Z3wp8E3gLMAV8wMzOxfPuBY4SfQZ/Z2aP1c0eGBoaqufqkhOGsLCEV5xFxRk0NYM3NcPBiSL+pavHpi5LVAStoeFR3/7zWtg7ls3hS09mdFAr55WM7d6+qi/OC4jOqvGKs3jFWTj5IgDmedxY6IPCS9hALzbYRzjYi/V1QVtr6vVYG5FhmwZ7XLP0a8B7gFHgGUmPmln16D1HgWkzu1HSPcC/Ah+QdDNRmbxbgL3AE5JuMqtfB+Hk5CQjIyPXthIzCEIoV6BSQeUKlCtouQxLy+jyMlq6DIvx/YUlNLcIcwtobhFt0NIOgbInQt4I8rwZ+RR7WsaGejhwfio9gQ241cvz8wyGlfNKRprb19orsAPABH4YkhsvwvjV54ub72PdHdGtqwO6O7COdqytBdpar/zbksfyuWhMm5y/pbNz1qMuGbaGWlrsdwK/NrOXASR9F3gfUB3s7wP+Kb7/X8BXJSme/l0zWwZeiWui3gkcr4v8479gz9gkLW0vRC1nCyG0+BZGgR2GUXm4lWkr04MAlYMozMuVDcO5FiqCQIouroiD+7WhHq4bn23IAdBrYW2pvKyQvU6FCOeVjCxtXz6wMqTOxd3d7JmIfkGXFbfsDbwgQJfmYJ1f15thnhcFfD6HrYR9zgd54Ck6p1+KvgC8+K+unGaeR/euLkgh2K8DXq16PAr8wUbLxMWvZ4DBePov1jz3qqKHExMTHD16lFwuRxAE3H333Rw7doyxsTE6OzvxfZ/Z2VkKhQLFYhEzo1Ao0HHmZbpnFhK83I1ZGboo8D0srhVazvt4YUg551PK5+hYXGa2tx0Fhh8ETPd10j2zwHxPB6WWHMMXLzG2p4/2xRImOHN9gcHJOWb7Oijn/dX5nQvLeEHIXE87hYlZigNdmCd2TcwyMdxLV3xmwXx3G7vHZnh9dw8KjYHiPJO7e+ieXSL0PRY6W1fXmS8H9FxaZKrQTe+lRUotOZY6Wlbnt5QqdM0tMdfewsX+Di63t3C5Lb86v+1ymbalEpf6OxmYmmO+u53LrTkGvBxvzrUxbSHzGCNx3+4+fNolTlmF25TjdQspYeyVz1mrcD0+PnCWgFuUY9yid3hIHqetwiF8AuBlAg4px6yF3O63skve6jqXzBgl4KByvGoBXYj+qvnzZowTcINynLOAfkRv1fwZC5nG2C+fl6zCED5dVc61vKYQOJxr39JrumABLaghr2mXfPYrv6XXlPRz6uszpqZKTAxtvu21LZU4N9y77rZXHOymb3qhpm1v7f7UUqow09ex5f2plM9x5vrCVfvT67u66ZtZREAl5zMwvcBiZyu5SkjrcplK3qelFHUwiHiQM4EMFIZQCqFU5lo6c1pv2c+FCxcol8sMDw9vmnvj4+Obnkmjza56kvR+4I/N7G/ixx8C7jSzv61a5nS8zGj8+CWilvlngeNm9q14+v3AD+Ii16scP37cDh06lPDtAO/F81w8dx4/ADwv+ob0PfA8FH+byvOQ7xP6b3xbmoT5HpbzCXM+Fj+nnhRnphno7a/rOutBIi9BaaBnW/ogZy6O0bsne8UQnBeoXCFfYwOqKbb7Woh7A7wgREGIKkH0NwiQRfNlBmGAVUIsCOJegwACe6OHITBKw/3sfcttW9I4ceLEs4cPH37r2um1tNhHgerfCfuACxssMyopB/QCxRqfu2XCm95ES6GH/v7sbUjt053OKwHTgxX6OxszZOu14Lxi+mrbZrK6fWXVC2B6erru66ylmfoMcFDSAUktRAdDH12zzKPAh+P7fwE8ZdFPgUeBeyS1SjoAHASero96RFCnQYLqjfNKhvNKhvNKRla9oDFumwa7mVWAjwCPAWeAB83stKTPSrorXux+YDA+OPox4NPxc08DDxIdaP0RcKyeZ8QALCzUp4+93jivZDivZDivZGTVCxrjtmkf+3aw1T52gOXlZVpba60ftH04r2Q4r2Q4r2Rk1QuuzW2jPvYdPwhYVovUOq9kOK9kOK9kZNULGuO244P9kUceSVthXZxXMpxXMpxXMrLqBY1x2/HB/tBDD6WtsC7OKxnOKxnOKxlZ9YLGuO34YK9UsnkNnvNKhvNKhvNKRla9oDFumTh4+uSTT04C57fy3GKxuGtgYOD1OitdM84rGc4rGc4rGVn1gmt2e9Phw4evKsGUiWB3OBwOR/3Y8V0xDofD4bgSF+wOh8PRZDRVsEv6hCSTtCttFwBJn5N0StJzkn4saW/aTgCSviDpbOz2sKS+tJ0gGnBO0mlJoaSrLrpIweeIpBck/VrSp9P2AZD0H5ImJD2ftks1kkYk/UTSmfgz/GjaTgCS2iQ9Lelk7PXPaTtVI8mX9L+SvlfP9TZNsEsaISoG8pu0Xar4gpndZmZ3AN8D/iFtoZjHgVvN7DbgReDelH1WeB64G/hZ2iJVBWb+BLgZ+GBcOCZt/hM4krbEOlSAj5vZ7wJvA45l5P1aBt5lZrcDdwBHJL0tZadqPko0VEtdaZpgB74IfJLVofXTx8xmqx52khE3M/txPAYQROPl70vTZwUzO2NmL6TtEbNaYMbMSsBKgZlUMbOfEY2cminM7KKZnYjvzxGF1VW1F7Ybi1gpwpqPb5nYDyXtA/4M+Ea9190UwR4PRvaamZ1M22Utkv5F0qvAX5KdFns1fw38MG2JDLJegZnUg2onIGk/8Gbgl+maRMTdHc8BE8DjZpYJL+BLRI3RcLMFk1JTMessIOkJYL3KAvcBnwH+aHuNIn6bl5n9j5ndB9wXF/X+CPCPWfCKl7mP6Cf0t7fDqVavjLBedZFMtPSyjKQu4L+Bv1/zizU14hFl74iPJT0s6VYzS/UYhaT3AhNm9qykP6z3+ndMsJvZu9ebLun3gAPAyajMKvuAE5LuNLOGj/yzkdc6fAf4PtsU7Jt5Sfow8F7gsG3jxQwJ3q+0aWiRmGZEUp4o1L9tZpm7ht/MLkn6KdExirQPPr8TuEvSnwJtQI+kb5nZX9Vj5Tu+K8bM/s/MdpvZfjPbT7RD/v52hPpmSDpY9fAu4GxaLtVIOgJ8CrjLzBbT9skotRSYccTExevvB86Y2b+l7bOCpMLKWV+S2oF3k4H90MzuNbN9cWbdQ1ScqC6hDk0Q7Bnn85Kel3SKqKsoE6eAAV8FuoHH41Mx/z1tIQBJfy5pFHg78H1Jj6XlslGBmbR8VpD0AHAc+B1Jo5KOpu0U807gQ8C74m3qubg1mjZ7gJ/E++AzRH3sdT21MIu4IQUcDoejyXAtdofD4WgyXLA7HA5Hk+GC3eFwOJoMF+wOh8PRZLhgdzgcjibDBbvD4XA0GS7YHQ6Ho8lwwe5wOBxNxv8D6bfETHc6d8kAAAAASUVORK5CYII=\n", 355 | "text/plain": [ 356 | "
" 357 | ] 358 | }, 359 | "metadata": { 360 | "needs_background": "light" 361 | }, 362 | "output_type": "display_data" 363 | } 364 | ], 365 | "source": [ 366 | "print('To get the p-value, which gives us our minimum level alpha for T, we calculate P(Z > T)')\n", 367 | "print('Or, in this case, the two-sided hypothesis version P(|Z| > T)')\n", 368 | "\n", 369 | "plt.title('P-Value = minimum level alpha for T')\n", 370 | "ax = sns.lineplot(domain, snorm)\n", 371 | "mask_r, mask_l = domain > tstat, domain < -tstat\n", 372 | "x_r, y_r, x_l, y_l = domain[mask_r], snorm[mask_r], domain[mask_l], snorm[mask_l]\n", 373 | "ax.fill_between(domain, snorm, color=pal[3])\n", 374 | "ax.fill_between(x_r, y1=y_r, color=pal[0])\n", 375 | "ax.fill_between(x_l, y1=y_l, color=pal[0])\n", 376 | "plt.show()" 377 | ] 378 | }, 379 | { 380 | "cell_type": "code", 381 | "execution_count": 7, 382 | "metadata": {}, 383 | "outputs": [ 384 | { 385 | "name": "stdout", 386 | "output_type": "stream", 387 | "text": [ 388 | "p-value = 0.09, which can be thought of as...\n", 389 | "- the minimum level alpha at which we can reject the null\n", 390 | "- the probability of Type I error for our specific test & test statistic.\n", 391 | "\n", 392 | " the last one isn't totally true, but it's close enough and can help you remember\n" 393 | ] 394 | } 395 | ], 396 | "source": [ 397 | "pval = round(2 * (1 - norm.cdf(tstat)), 3)\n", 398 | "print(f'p-value = {pval}, which can be thought of as...')\n", 399 | "print(f'- the minimum level alpha at which we can reject the null')\n", 400 | "print(f'- the probability of Type I error for our specific test & test statistic.')\n", 401 | "print('\\n the last one isn\\'t totally true, but it\\'s close enough and can help you remember')" 402 | ] 403 | } 404 | ], 405 | "metadata": { 406 | "kernelspec": { 407 | "display_name": "Python 3", 408 | "language": "python", 409 | "name": "python3" 410 | }, 411 | "language_info": { 412 | "codemirror_mode": { 413 | "name": "ipython", 414 | "version": 3 415 | }, 416 | "file_extension": ".py", 417 | "mimetype": "text/x-python", 418 | "name": "python", 419 | "nbconvert_exporter": "python", 420 | "pygments_lexer": "ipython3", 421 | "version": "3.7.0" 422 | } 423 | }, 424 | "nbformat": 4, 425 | "nbformat_minor": 2 426 | } 427 | -------------------------------------------------------------------------------- /U03 L08 - Distance Measures Between Distributions.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Unit 3 Lecture 8 - Distance Measures Between Distributions" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "## Total Variation (TV) Distance\n", 15 | "\n", 16 | "Gives us a notion of distance between some distributions.\n", 17 | "\n", 18 | "- $TV$ is symmetric, positive, definite, and it satisfies the triangle inequality.\n", 19 | "\n", 20 | "For two probability distributions $\\mathbb{P}_{\\theta}$, $\\mathbb{P}_{\\theta^{\\prime}}$ over the union of their supports $E$, $\\mathbb{P}_{\\theta}$ and $\\mathbb{P}_{\\theta^{\\prime}}$ are close by measure of TV if\n", 21 | "\n", 22 | "$|\\mathbb{P}_{\\theta}(A) - \\mathbb{P}_{\\theta^{\\prime}}(A)|$ is small $\\forall A \\subset E$\n", 23 | "\n", 24 | "So,\n", 25 | "\n", 26 | "- $TV(\\mathbb{P}_{\\theta}, \\mathbb{P}_{\\theta^{\\prime}}) = max_{A \\subset E} |\\mathbb{P}_{\\theta}(A) - \\mathbb{P}_{\\theta^{\\prime}}(A)|$\n", 27 | "\n", 28 | "with formulas\n", 29 | "\n", 30 | "- $TV(\\mathbb{P}_{\\theta}, \\mathbb{P}_{\\theta^{\\prime}}) = \\frac{1}{2} \\sum_{x \\in E} |p_{\\theta}(x) - p_{\\theta^{\\prime}}(x)|$ \n", 31 | "- $TV(\\mathbb{P}_{\\theta}, \\mathbb{P}_{\\theta^{\\prime}}) = \\frac{1}{2} \\int_{x \\in E} |f_{\\theta}(x) - f_{\\theta^{\\prime}}(x)| dx$ \n", 32 | "\n", 33 | "outputs a value $\\in [0, 1]$\n", 34 | "\n", 35 | "$TV$ doesn't provide a notion of distance between most distributions, since $TV($continuous, discrete$)$, $TV($continuous distributions with no overlap$)$, and discrete cases like $X \\sim Ber(p)$ and $X + \\epsilon$, $\\epsilon \\notin \\{-1, 0, 1\\}$ all have $TV=1$.\n" 36 | ] 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "metadata": {}, 41 | "source": [ 42 | "## Kullback-Leibler (KL) Divergence\n", 43 | "\n", 44 | "Gives us a notion of divergence of one distribution from another. Also known as 'relative entropy.'\n", 45 | "\n", 46 | "- $KL$ is not symmetric, positive, definite, and does not satisfy the triangle inequality.\n", 47 | "\n", 48 | "For a fixed, unknown $\\theta^*$, KL-Divergence has formulas\n", 49 | "\n", 50 | "- $KL(\\mathbb{P}_{\\theta^*}, \\mathbb{P}_{\\theta}) = \\sum_{x \\in E} p_{\\theta^*}(x) \\cdot ln(\\frac{p_{\\theta^*}(x)}{p_{\\theta}(x)})$\n", 51 | "\n", 52 | "- $KL(\\mathbb{P}_{\\theta^*}, \\mathbb{P}_{\\theta}) = \\int_{x \\in E} f_{\\theta^*}(x) \\cdot ln(\\frac{f_{\\theta^*}(x)}{f_{\\theta}(x)})$\n", 53 | "\n", 54 | "$= \\mathbb{E}_{\\theta^*} [ln(\\frac{p_{\\theta^*}(X)}{p_{\\theta}(X)})]$\n", 55 | "\n", 56 | "$= \\mathbb{E}_{\\theta^*}[ln(p_\\theta^*(X)] - \\mathbb{E}_{\\theta^*}[ln(p_{\\theta}(X)]$\n", 57 | "\n", 58 | "The first term in this difference is a constant, so minimizing this difference means minimizing\n", 59 | "\n", 60 | "- $\\hat{KL}(\\mathbb{P}_{\\theta^*}, \\mathbb{P}_{\\theta}) = c - \\frac{1}{n} \\sum ln(p_{\\theta}(X_i))$\n", 61 | "\n", 62 | "given $\\frac{1}{n} \\sum h(X_i) \\xrightarrow[]{D} \\mathbb{E}_{\\theta^*}[h(X)]$ for all $\\theta$.\n" 63 | ] 64 | }, 65 | { 66 | "cell_type": "markdown", 67 | "metadata": {}, 68 | "source": [ 69 | "## Maximum Likelihood Principle\n", 70 | "\n", 71 | "here $\\to$ means 'if $\\theta$ satisfies the previous, it equivalently satisfies this'\n", 72 | "\n", 73 | "$min_{\\theta \\in \\Theta} \\hat{KL}(\\mathbb{P}_{\\theta^*}, \\mathbb{P}_{\\theta})$\n", 74 | "\n", 75 | "$\\to min_{\\theta \\in \\Theta} - \\frac{1}{n} \\sum ln(p_{\\theta}(X_i))$\n", 76 | "\n", 77 | "$\\to max_{\\theta \\in \\Theta} \\frac{1}{n} \\sum ln(p_{\\theta}(X_i))$\n", 78 | "\n", 79 | "$\\to max_{\\theta \\in \\Theta} ln(\\prod p_{\\theta}(X_i))$\n", 80 | "\n", 81 | "$\\to max_{\\theta \\in \\Theta} \\prod p_{\\theta}(X_i)$\n", 82 | "\n", 83 | "This is the maximum of the likelihood function $L_n(\\theta | X_1, ..., X_n)$." 84 | ] 85 | } 86 | ], 87 | "metadata": { 88 | "kernelspec": { 89 | "display_name": "Python 3", 90 | "language": "python", 91 | "name": "python3" 92 | }, 93 | "language_info": { 94 | "codemirror_mode": { 95 | "name": "ipython", 96 | "version": 3 97 | }, 98 | "file_extension": ".py", 99 | "mimetype": "text/x-python", 100 | "name": "python", 101 | "nbconvert_exporter": "python", 102 | "pygments_lexer": "ipython3", 103 | "version": "3.7.0" 104 | } 105 | }, 106 | "nbformat": 4, 107 | "nbformat_minor": 2 108 | } 109 | -------------------------------------------------------------------------------- /U03 L09 - Introduction to Maximum Likelihood Estimation.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Unit 3 Lecture 9 - Introduction to Maximum Likelihood Estimation" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "## Method of Moments\n", 15 | "\n", 16 | "$E[X^k]$ is the $k$th moment of an R.V. $X$. \n", 17 | "\n", 18 | "### $\\frac{1}{n} \\sum_{i=1}^n X_i^k$ \n", 19 | "\n", 20 | "is the estimator for the $k$th moment. A combination of method of moments estimators can be used to estimate a parameter, such as in $\\sigma^2 = E[X^2] - (E[X])^2$. \n", 21 | "\n", 22 | "Provides consistent, but often biased estimators.\n", 23 | "\n", 24 | "## Maximum Likelihood Estimation\n", 25 | "\n", 26 | "- $L_n := L(\\theta | X_1, ..., X_n) = \\prod_{i=1}^n f(X_i | \\theta)$ \n", 27 | "\n", 28 | "takes a sample $\\{X_1,...,X_n\\}$ and a parameter $\\theta$ and gives the likelihood the sample was created from a distribution with PDF/PMF $f$. Geometrically, it finds the maximum, or peak, of the likelihood function.\n", 29 | "\n", 30 | "- Requires iid.\n", 31 | "\n", 32 | "- $\\hat{\\theta}_{MLE}$ is the $\\theta$ that maximizes $L_n$ $\\forall \\theta \\in \\Theta$.\n", 33 | "\n", 34 | "Examples:\n", 35 | "\n", 36 | "- Bernoulli: $p^{\\sum x_i}(1-p)^{n - \\sum x_i}$\n", 37 | "- Poisson: $\\dfrac{\\lambda^{\\sum x_i}}{\\prod x_i!}e^{-n \\lambda}$\n", 38 | "- Gaussian: $\\dfrac{1}{(\\sigma \\sqrt{2 \\pi})^n} exp(- \\dfrac{1}{2 \\sigma^2} \\sum (x_i - \\mu)^2)$\n", 39 | "- Exponential: $\\lambda^n exp(- \\lambda \\sum x_i)$\n", 40 | "\n", 41 | "### Estimator w/ Indicator \n", 42 | "\n", 43 | "An MLE estimator might have an indicator attatched to it, such as with\n", 44 | "\n", 45 | "$\\lambda^n exp(- \\lambda \\sum x_i) \\mathbb{1}(x_i > 0) = \\lambda^n exp(- \\lambda \\sum x_i) \\mathbb{1}(min(x_i) > 0)$\n", 46 | "\n", 47 | "However, since in this class the model is always well-specified, $x_i$ samples will always be $> 0$, making the indicator unnecessary. Only indicators that depend on the parameter are relevant.\n", 48 | "\n" 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "### Concave, Convex\n", 56 | "\n", 57 | "If the second derivative of $L_n$ is negative (resp. positive) over all of $\\theta$, the function is strictly concave (resp. strictly convex). If $L_n$ is strictly concave and the argmax is finite, setting $\\frac{d}{d\\theta}[L_n] = 0$ finds the maximum. Then, solving for $\\theta$ gives $\\hat{\\theta}_{MLE}$.\n", 58 | "\n", 59 | "\n", 60 | " \n", 61 | " \n", 62 | " \n", 63 | " \n", 64 | " \n", 65 | " \n", 66 | " \n", 67 | " \n", 68 | " \n", 69 | " \n", 70 | " \n", 71 | " \n", 72 | " \n", 73 | " \n", 74 | " \n", 75 | " \n", 76 | "
concave$g \\prime \\prime (x) \\leq 0$
strictly concave$g \\prime \\prime (x) < 0$
convex$g \\prime \\prime (x) \\geq 0$
strictly convex$g \\prime \\prime (x) > 0$
" 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "metadata": {}, 82 | "source": [ 83 | "### Log-likelihood\n", 84 | "\n", 85 | "Maximizing $ln(L_n)$ will also maximize $L_n$, since $ln$ is a monotonic function.\n", 86 | "\n", 87 | "If setting $\\frac{d}{d\\theta}[L_n] = 0$ finds that maximum (as it will for a strictly concave function, resp. min for convex), so does setting $\\frac{d}{d\\theta}[ln(L_n)] = 0$. And, since taking the derivative of a product is harder than taking the derivative of a sum, in this case using $ln$ is the preferred method of finding $\\hat{\\theta}_{MLE}$. Thinking geometrically might also lead to an intuitive answer." 88 | ] 89 | }, 90 | { 91 | "cell_type": "markdown", 92 | "metadata": {}, 93 | "source": [ 94 | "### Gradient\n", 95 | "\n", 96 | "The gradient of a function is the multi-variable version of a derivative, or vector of partial derivatives. Geometrically, you can think of it as being the partial contribution of each variable to the direction of the function, just as the derivative does with one variable.\n", 97 | "\n", 98 | "$f(\\theta) = c_1 \\theta_1^2 + c_2 \\theta_2^3 + c_3 \\theta_3^2$\n", 99 | "\n", 100 | "$\\nabla f = \\begin{pmatrix}\n", 101 | "\\frac{df}{d\\theta_1} \\\\\n", 102 | "\\frac{df}{d\\theta_2} \\\\\n", 103 | "\\frac{df}{d\\theta_3}\n", 104 | "\\end{pmatrix}$\n", 105 | "\n", 106 | "$= \\begin{pmatrix}\n", 107 | "2 c_1 \\theta_1 \\\\\n", 108 | "3 c_2 \\theta_2^2 \\\\\n", 109 | "2 c_3 \\theta_3\n", 110 | "\\end{pmatrix}$\n", 111 | "\n", 112 | "If a function is strictly concave and all elements of the gradient are zero at a point, that point represents the maximum of the function. The same goes for being strictly convex and finding the minimum. \n", 113 | "\n", 114 | "Functions are sometimes more complicated. The most popular algorithm to search for the global minimum error of a prediction function is called 'gradient descent'. The process is like rolling a ball downhill, looking for the deepest pit on a hilly surface, with a blindfold on.\n", 115 | "\n", 116 | "" 117 | ] 118 | }, 119 | { 120 | "cell_type": "markdown", 121 | "metadata": {}, 122 | "source": [ 123 | "### Hessian Matrix\n", 124 | "\n", 125 | "The hessian is a square matrix of second-order partial derivatives of a function. Where the gradient describes local contribution to direction, the hessian describes local curvature.\n", 126 | "\n", 127 | "$H_{i, j} = \\dfrac{d^2 f}{d \\theta_i d \\theta_j}$\n", 128 | "\n", 129 | "$\\nabla f = \\begin{pmatrix}\n", 130 | "2 c_1 \\theta_1 \\\\\n", 131 | "3 c_2 \\theta_2^2 \\\\\n", 132 | "2 c_3 \\theta_3\n", 133 | "\\end{pmatrix}$\n", 134 | "\n", 135 | "$H_f = \\begin{bmatrix}\n", 136 | "2 c_1 & 0 & 0 \\\\\n", 137 | "0 & 6 c_2 \\theta & 0 \\\\\n", 138 | "0 & 0 & 2 c_3\n", 139 | "\\end{bmatrix}$\n", 140 | "\n", 141 | "#### Positive & Negative // Definite & Semi-Definite // Concave & Convex\n", 142 | "\n", 143 | "A $d$ x $d$ matrix $H$ is positive semi-definite (resp. negative semi-definite) if...\n", 144 | "\n", 145 | "$x^THx \\geq (resp. \\leq)$ $0$ $\\forall x \\in \\mathbb{R}^d$\n", 146 | "\n", 147 | "... where $x$ isn't a vector of all zeroes. Another way to think of it is that if you added any of the elements together in any combination, the result must be positive (resp. negative) or zero.\n", 148 | "\n", 149 | "- $\\leq 0$ $\\to$ negative semi-definite $\\to$ concave\n", 150 | "- $ < 0$ $\\to$ negative definite $\\to$ strictly concave \n", 151 | "- $\\geq 0$ $\\to$ positive semi-definite $\\to$ convex\n", 152 | "- $ > 0$ $\\to$ positive definite $\\to$ strictly convex \n", 153 | "\n", 154 | "If a matrix is definite, it is also semi-definite by definition.\n", 155 | "\n", 156 | "#### Determinant and Trace\n", 157 | "\n", 158 | "Another method of finding convexity/concavity if the matrix is 2x2 and symmetric:\n", 159 | "\n", 160 | "$H = \\begin{bmatrix}\n", 161 | "a & b \\\\\n", 162 | "c & d\n", 163 | "\\end{bmatrix}$\n", 164 | "\n", 165 | "- $det(H) = ad - bc$\n", 166 | "- $tr(H) = a + d$\n", 167 | "\n", 168 | "... where the trace is generally the sum of elements on the main diagonal.\n", 169 | "\n", 170 | "- $tr(H) \\leq 0$\n", 171 | "- $det(H) \\geq 0$\n", 172 | "\n", 173 | "would mean a matrix is negative semi-definite. \n", 174 | "\n", 175 | "I personally don't know if this works for larger matrices, though I know finding the determinant is a more complicated process without these conditions." 176 | ] 177 | } 178 | ], 179 | "metadata": { 180 | "kernelspec": { 181 | "display_name": "Python 3", 182 | "language": "python", 183 | "name": "python3" 184 | }, 185 | "language_info": { 186 | "codemirror_mode": { 187 | "name": "ipython", 188 | "version": 3 189 | }, 190 | "file_extension": ".py", 191 | "mimetype": "text/x-python", 192 | "name": "python", 193 | "nbconvert_exporter": "python", 194 | "pygments_lexer": "ipython3", 195 | "version": "3.7.0" 196 | } 197 | }, 198 | "nbformat": 4, 199 | "nbformat_minor": 2 200 | } 201 | -------------------------------------------------------------------------------- /U03 L10 - Consistency of MLE, Covariance Matrices, and Multivariate Stats.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Unit 3 Lecture 10 - Constistency of MLE, Covariance Matrices, and Multivariate Statistics" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 3, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "import numpy as np\n", 17 | "import random\n", 18 | "import matplotlib.pyplot as plt\n", 19 | "import seaborn as sns\n", 20 | "import warnings\n", 21 | "warnings.filterwarnings('ignore')" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "## Log-Likelihood = 0 doesn't always give MLE\n", 29 | "\n", 30 | "Sometimes the maximum of a function isn't found with the derivative. In those cases, you have to employ other methods to find the MLE. There isn't necessarily one prescription for this process. Example:\n", 31 | "\n", 32 | "$X_1, \\cdots, X_n \\sim Unif[0, \\theta]$\n", 33 | "\n", 34 | "$L_n(\\theta | X_1, \\cdots, X_n) = \\prod_n f_X(x|\\theta) = \\dfrac{1}{\\theta^n}$\n", 35 | "\n", 36 | "This function finds its argmax in $max(X_i)$, where the likelihood function isn't continuous, and the derivative isn't zero.\n", 37 | "\n", 38 | "- (Scripted Example 1)" 39 | ] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "metadata": {}, 44 | "source": [ 45 | "## Consistency of MLE\n", 46 | "\n", 47 | "Under mild regularity conditions (continuity in $\\theta$ of the pdf $p_{\\theta}$ almost everywhere),\n", 48 | "\n", 49 | "$\\hat{\\theta_n}^{MLE} \\xrightarrow[n \\to \\infty]{P} \\theta$\n", 50 | "\n", 51 | "which holds true in the multivariate case, meaning when $\\theta$ is a vector of parameters." 52 | ] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": {}, 57 | "source": [ 58 | "## Random Vector\n", 59 | "\n", 60 | "$X = \\begin{pmatrix}\n", 61 | "X^{(1)} \\\\\n", 62 | "\\vdots \\\\\n", 63 | "X^{(d)}\n", 64 | "\\end{pmatrix}$\n", 65 | "\n", 66 | "is a $d$ x $1$ random vector, or a random vector $\\in \\mathbb{R}^d$, where each $X^{(k)}$ is an RV. \n", 67 | "\n", 68 | "- The pdf of $X$ is the joint pdf of the elements\n", 69 | "- The cdf of $X$ is $P(X^{(1)} \\leq x^{(1)}, \\cdots, X^{(d)} \\leq x^{(d)})$ (similar to pdf of max, but $x$ is a vector)\n", 70 | "- $X$ converges in probability when all $X^{(k)}$ individually converge in probability. If $X$ = $\\hat{\\theta}_n$, a vector of estimators, all $X^{(k)}$ must be consistent estimators for $\\hat{\\theta}_n$ to be consistent.\n", 71 | "\n", 72 | "e.g.\n", 73 | "\n", 74 | "$X \\sim \\mathcal{N}(\\mu, \\sigma^2)$\n", 75 | "\n", 76 | "$\\hat{\\theta}_n = \\begin{pmatrix}\n", 77 | "\\bar{X}_n \\\\\n", 78 | "\\dfrac{\\sum_n(X_i - \\bar{X}_n)^2}{n}\n", 79 | "\\end{pmatrix}$\n", 80 | "\n", 81 | "converges in probability to\n", 82 | "\n", 83 | "$\\theta = \\begin{pmatrix}\n", 84 | "\\mu \\\\\n", 85 | "\\sigma^2 \n", 86 | "\\end{pmatrix}$\n", 87 | "\n", 88 | "- A random vector has a gaussian distribution iff all linear combinations of its elements are of that distribution. This isn't always the case." 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "metadata": {}, 94 | "source": [ 95 | "## Covariance\n", 96 | "\n", 97 | "- $Cov(X, Y) = E[(X - E[X])(Y - E[Y])]$\n", 98 | "- $= E[(X)(Y - E[Y])]$\n", 99 | "- $= E[(X - E[X])(Y)]$\n", 100 | "- $= E[XY] - E[X]E[Y]$\n", 101 | "- $Cov(X, X) = Var(X)$" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "metadata": {}, 107 | "source": [ 108 | "### Covariance & Independence\n", 109 | "\n", 110 | "- $X, Y$ are independent $\\Rightarrow Cov(X, Y) = 0$\n", 111 | "- $Cov(X, Y) = 0 \\nRightarrow X, Y$ are independent" 112 | ] 113 | }, 114 | { 115 | "cell_type": "markdown", 116 | "metadata": {}, 117 | "source": [ 118 | "### Covariance Matrix\n", 119 | "\n", 120 | "Covariance matrix $\\Sigma$ is the total representation of covariance between all of the elements of a random vector. It has the variances on the main diagonal and covariances symmetrically above and below.\n", 121 | "\n", 122 | "- $\\Sigma_{i, j} = Cov(X^{(i)}, X^{(j)})$\n", 123 | "\n", 124 | "e.g.\n", 125 | "\n", 126 | "$ M = \\begin{pmatrix}\n", 127 | "X \\\\\n", 128 | "Y \\\\\n", 129 | "Z\n", 130 | "\\end{pmatrix}$\n", 131 | "\n", 132 | "$\\Sigma_M = E[(M - E[M])(M - E[M])^T]$\n", 133 | "\n", 134 | "$= E\\begin{pmatrix}\\begin{pmatrix}\n", 135 | "X - E[X] \\\\\n", 136 | "Y - E[Y] \\\\\n", 137 | "Z - E[Z] \n", 138 | "\\end{pmatrix}\n", 139 | "\\begin{pmatrix}\n", 140 | "X - E[X] & Y - E[Y] & Z - E[Z] \n", 141 | "\\end{pmatrix}\\end{pmatrix}$\n", 142 | "\n", 143 | "$= \\begin{pmatrix}\n", 144 | "Cov(X, X) & Cov(X, Y) & Cov(X, Z) \\\\\n", 145 | "Cov(Y, X) & Cov(Y, Y) & Cov(Y, Z) \\\\\n", 146 | "Cov(Z, X) & Cov(Z, Y) & Cov(Z, Z)\n", 147 | "\\end{pmatrix}$\n" 148 | ] 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "metadata": {}, 153 | "source": [ 154 | "### Affine Transformation of a Covariance Matrix\n", 155 | "\n", 156 | "Univariate case\n", 157 | "\n", 158 | "$Var(aX + b) = a^2 Var(X)$\n", 159 | "\n", 160 | "The multivariate case is similar, but matrices don't have the distributive property, so...\n", 161 | "\n", 162 | "- $Var(AX + B) = A \\Sigma_X A^T$" 163 | ] 164 | }, 165 | { 166 | "cell_type": "markdown", 167 | "metadata": {}, 168 | "source": [ 169 | "## Multivariate CLT\n", 170 | "\n", 171 | "- $\\sqrt{n}(\\bar{X}_n - \\mu) \\to \\mathcal{N}(0, \\Sigma)$\n", 172 | "\n", 173 | "Where $\\bar{X}_n$ is the average of a set of iid copies $\\{X_1, \\cdots, X_n\\}$ of random vector $X$ with $E[X] = \\mu$ and $Var(X) = \\Sigma$, which, put another way, makes $\\bar{X}_n$ a vector of averages. Each $\\bar{X_n}^{(k)}$ represents the mean of the elements of $X_k$.\n", 174 | "\n", 175 | "$X_k = \\begin{pmatrix}\n", 176 | "X^{(1)}_k \\\\\n", 177 | "\\vdots \\\\\n", 178 | "X^{(d)}_k\n", 179 | "\\end{pmatrix}$\n", 180 | "\n", 181 | "$\\bar{X}_n = \\begin{pmatrix}\n", 182 | "\\bar{X_n}^{(1)} \\\\\n", 183 | "\\vdots \\\\\n", 184 | "\\bar{X_n}^{(d)}\n", 185 | "\\end{pmatrix}$\n", 186 | "\n", 187 | "and \n", 188 | "\n", 189 | "- $\\sqrt{n} (\\Sigma^{-1/2}) (\\bar{X}_n - \\mu) \\to \\mathcal{N}(0, I_d)$\n", 190 | "\n", 191 | "Where $\\Sigma^{1/2}$ is the square root of $\\Sigma$, and it's the matrix s.t. $\\Sigma^{-1/2} \\Sigma^{-1/2} = \\Sigma^{-1}$. \n", 192 | "\n", 193 | "This tranformation corresponds to the single variable version\n", 194 | "\n", 195 | "$\\sqrt{n}(\\sigma^{-1})(\\bar{X}_n - \\mu) \\to \\mathcal{N}(0, 1)$" 196 | ] 197 | }, 198 | { 199 | "cell_type": "markdown", 200 | "metadata": {}, 201 | "source": [ 202 | "## Multivariate Delta Method\n", 203 | "\n", 204 | "Univariate with some RV $X$ with variance $\\sigma^2$ and parameter $\\theta$. If $\\hat{\\theta}_n$ is consistent,\n", 205 | "\n", 206 | "$\\sqrt{n}(\\hat{\\theta}_n - \\theta) \\to \\mathcal{N}(0, \\sigma^2)$\n", 207 | "\n", 208 | "$\\sqrt{n}(g(\\hat{\\theta}_n) - g(\\theta)) \\to \\mathcal{N}(0, g\\prime(E[X])^2\\sigma^2)$\n", 209 | "\n", 210 | "So, multivariate\n", 211 | "\n", 212 | "- $\\sqrt{n}(\\hat{\\theta}_n - \\theta) \\to \\mathcal{N}(0, \\Sigma)$\n", 213 | "\n", 214 | "- $\\sqrt{n}(g(\\hat{\\theta}_n) - g(\\theta)) \\to \\mathcal{N}(0, \\nabla g(\\theta)^T\\Sigma \\nabla g(\\theta))$\n", 215 | "\n", 216 | "Where $\\nabla g$ is the vector of first derivatives of $g$\n", 217 | "\n", 218 | "- NOTE: for the homework, pay very close attention to the variability of the language for this sort of problem. Here, we're talking about $g(\\hat{\\theta})$, where $g$ makes the transformation. But, think about what it means for $\\hat{\\theta}$ to instead be the transforming function $g$ of some estimator. You would rewrite the above above convergence formula to $g($estimator$)$... instead of $g(\\hat{\\theta})$. What are you then taking the derivative wrt? Also very important, what are we converging to, and what does that make $X$ in the above formula?" 219 | ] 220 | }, 221 | { 222 | "cell_type": "markdown", 223 | "metadata": {}, 224 | "source": [ 225 | "## Scripted Examples" 226 | ] 227 | }, 228 | { 229 | "cell_type": "markdown", 230 | "metadata": {}, 231 | "source": [ 232 | "### 1. Can't Always Find MLE with Derivative" 233 | ] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "execution_count": 52, 238 | "metadata": {}, 239 | "outputs": [ 240 | { 241 | "data": { 242 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmcXFWZ//HPU93p7qTTIUt3FhNCEtYAAgFkEWTAnwoCgiM6gCuOyk9h3Ab9jTouqDij4wyIg4K4ASK44DJssoyyCkFDIGEJS4AAbQLp7Hs6VfX8/ji3kptKVVd1dy3dt77v16tfXXXvqXufunXvU+eec+4tc3dERCRZUvUOQEREKk/JXUQkgZTcRUQSSMldRCSBlNxFRBJIyV1EJIGU3EVEEqjmyd3MzjEzN7O9+ihzlZktiT2fEb3mwxWKYYmZXVUgphl5Za6txPoGq1B8RcpdFZUr9Ndcm2gLxnWImV1oZuMLzHMzu7AOMb3NzB4zsy1RDGOrtJ6K7rt9rGeymd1oZqui9X2qCuvo89iN5l00iOXOiE1rN7OfmdnyaN53Bh55ZZnZYWa2ycym5k1/u5k9Eu1TL5rZF82sqYzl5d5//t+jeeU+bWYLzaysvF23A76ErwOX1nB9twBHA8tquM5q6QFOy5/o7uk6xJJzCPAV4FpgVd68o4HuWgYTfdH9HHgAOB/oBdbXMoYq+DLwd8A5hP14ST2D6adCx9/5wNnAPwLPMLSOzW8DP3H3v+UmmNmJwG+AHwP/DMwB/g3oAP6lzOW+i52PhY1586+IlvUB4KelFjYkk7u7P1fj9fUQkmIS9Lr73HoHUa46xTqVcND9yt3vHezCotqZ1fkLdDawwN1/V8cYBqTI8TcbWOru11RiHWbW6u5bK7CcQ4ETgI/nzfomcL+7nxs9v8vMRgNfNLNL3P2VMhb/qLsvLjbT3Teb2TXAZygjuQ/JNvf8ZpkiZTrN7CEzW2Rm02PT32Fmc6PTpjVm9uv4/CLLKtrsYWZnRevYaGbzzOzYAmXea2YLotOxFdHp5JS8MiPM7KKouac3+n+RmY3IKzfLzG6J4u8xs0uB1r7i749i29bM7jazu2PPj4+2yWlmdln0vnrM7Nr8JgwzazazfzGzJ6Nt0GNmt5nZfmZ2Djt2xGdjp5wzotfu0ixjZieZ2YNmttnM1prZ781s3wLx3m9mbzKz+dH2etzM3l7i/V/Ijlrtj6P13x3Ns+jU9+noM1oWvfcxectwM/uGmX3OzF4g1Pxf29d6gRYzuzhqZthkZjcX2t8KxNtnTBY1+wDHA2/I374FltdmZpdE22qDmb1iZjeZ2X6lYumvaF/rNrM5ZnZf9L6fNbOP5pXb6fiL3s85wO6x93N8NG9fM/tddGxvtnCsn5S3vAuj1xxoZreb2QbgV9G83H5zkpk9Gi3jETM7MtqP/y3axqui+Nvz3tZHgIXu/kRsfbsTzk7zm3F/BowA3jqY7ZjnF8D+Zvb6UgWHZHIvJdoJ/gw4cKy7vxRN/yjh1OhJ4J3A/wUOBO4xs44BrOoNwAXAl4AzgSbg5nhyM7NzCR/iIuAdwOeAE6N1jo4t6+po3jXAqYSE9y/R9NyyWoA7Cad05xN28JnAF/sTdLSTxv8G8zlfStjO7wa+BpzBrk1mvwC+AdwKvJ1wADwJTCGccufaYd9FOP0u2gQWHai3ABsI2/xjhM/wfstr4wT2jGK5mLDtlwE3WB/9OcCPojiI4joaOC96/o1oWXcCbwP+g/AZ3FJgG54DnEKoRZ0CLO1jnQCfB/YGPkj4bA8D7rC8L/cCSsW0LHoPC4FHKLF9CRWFDsJ7P4WwfduAuWY2uUQsAzEGuI6Q+E4H/gpcbmYn9PGao4HbgVfY8X7mm9lrgPuBg4F/Av4BWEPYFoUS6P8A9xCaKS+JTd+L0LTyTcK+0ArcCFxO2GfPIezr7yE0J8adBNyXN+2A6P/j8Ynu/gKwCdi/j/cad7+ZZaIvlyusQB8V8CiwLoqjb+5e0z/ChnNgrz7KXAUsiT2fEb3mw4QPdikhkYyKlRkNrCW0hZH32l7gU7FpS4CrCsQ0I6/MamBcbNrhUbl3R8+bgFeBu/LWeWxU7hPR8wOj5xfmlftiNP2g6PlHoudHxcqkgCfy4+tju3mBv4uKbdvY9LuBu2PPj49ee3VeucuALYRmCIA3xt9rfz/z/O0CzAOeBZpj02YC24CL8+LdBuwdmzYRyABfKLGd9orWe05s2vjofV2VV/a9UdnT8mJeCowsY3+fEZV/EkjFph8TTf9QH6/tT0z3xz+/fhyPTcAoQp/Dpwdz7BbZ1xw4ITatFVgBXFni+Ls2fz8F/hNIx9cfxf80MD827cJoeZ8ssp9vA2bFpp0Wlf/fvLK/BV6IPZ8UlftIXrl3R9P3K7C+buDHJbbricBXgZMJTT5fjD6Px4C2AuXvA+4o9dnWteZuZj+JTlMfL10agOMI38T/S9ixN8XmnU6oJRxnZk+Y2XkWOs66Ccn969Fp2KOUf8byoLuvjj1/LPqfa+bZl5BQfh5/kbvfD7xI6ODKxQ27nrblnufKHQ287LF2aHfPEp1Slmk58Lq8v+/34/X5bsl7/hjhAJ0UPX8LYcf+4SDWAYQREsChwC891n7toQb0Z3Zsp5xn3f3ZWLnlhPffZzNcEUcR3lf+Z/QLQkLJX/dt7r65H8u/Ifosc7H+mbBvHl3BmMpiZv9goUlzTbScjYTK0b59v3JANrn7XbknHtq9n2Vgn9FxwFyPtUu7ewa4Hjgkv/kMKNb/8Iy7Px97/lT0//a8ck8B08zMouevif7n9w/k5he6xa4VmLYTd7/d3b/i7re6+13ufhHhC/zA6H++nlgsRdW7WeYqyjm92OFkwk74A9+18yo35GhPwmnQ9wjf0NuAdsIHeoi7HwJkKc9OIzt8R4dMW/Q/d9pU6BT4ldj8YuVeyZs/hXAmkK/QtGK2ufu8vL9STQZ9yR/dkr8NJgCr+pnoihlHOBhKbc9iseXiayswvZSCn1G0n60ssO7+jt4o9rnmNzUNJqaSzOxtwC8JzYjvBo4kVAB6KL3dcsfcLsP7bMeQv/zjcnV+WQb3GRXbN4yw/8QV+4zyY+rtY3ozO95vLub8jtncfljo8xhL4f20lBsJX7qvKzBvMzCy1ALqmtw9jFTY6Y2b2Z7Am4EpUSdMvKPnS8BNwB/M7Ji8xS2P/p8DvIlw4LyVsHHmE4YlVVou9kJtlZMJB2Bf5XLPc+WWsaNGHFdo2kBtAVoKTJ8wwOWtAMabWcmdrQyrCbWfUtuzGgp+RtHZ34QC6+7vDyEU+1z/VmD6QGMqx1nAYnc/J6op/gVYQHlfFLljrFCtMTetPxWR/lpF8X3D2TWJVvrHKnLbO/9LJNe5ekB8YtQ3OIrQJDdQhd7DeMJx16d619wLuRKYS0h0n2HnJoVthE6UO4DbzOwNsXkPEDrh/oPwBfB1d7/N3ecR2q8ujJplvlTBWJ8m7MxnxSdGPdl7EJqQiP3fqRyhwwYgNxzvQcIIgaNiy0oR3nOlvAhMMrPO2Dr2ZOCn5HcQak19XaSTq+n0+QXg7huBh4F3xWqCmNkewOvZsR2rYS4hzvzP6ExC7W2w635nvFM2qpxMI3zmtYxpFLvWrt9Hgdp4AQ8Rao1nFJiXm3b3AGIq1z3AUfGRQNF+cibwiLtX+1qFJYTK0az4RA8DOhaw43jOeS8hZ/1hAOt6O6HF4aEC82YSck+f6jnO/SQzewXoAsaY2TsJO/LrCTvQaOAH7DwM8KtA7sq7LHC3mT3s7ke4+zoz+wyhOeY64JNmtpSwM3cTvjRuIoymyR/eNCDunjGzLwM/sHA167WE0+xvENoVfxqVe8LMrid8wTQTvoiOJpyJXO/uC6NF5kbU/NbMvkCoKX2U0JdQKb8mXCT2czO7GOgkjOQoWRMoxN3vMrPfABdHQ8L+RBj+dRxwi7vfzY6ay/lmdjVhh1/o7r0FFvklQjv/zWb2fcJ+8FVCZ/l/DSTGMt/Hqmh7fN7MNhI67GcTRpXcz659D/3VAfzezH5A2Of/nbCPFB3HXaWYbgPebmaXADcTRu18gjDqpE/uvtbMvgFcZGathOOpl9D2fwGh43fRAGIq1yWEM/M7zewrhFEj5wH7EEb+VJW795rZQ8ARBWZ/gbDP/oDQBzCH0DF6qcfGuNuOocEnRMcGZnYncBdhtM1mQmf7ZwhfGNfFVxKN1NuH0LlcMuCa/rGjZ7zQ35OEGvtVFBktE5vWFL3xjezcG39ytKF6CV8Wi4GfAPvH1r+O8kbLXFsg/kKjXt4bfRBbCaduPwOm5JUZQTgoXyQktxej5yPyys0iHMSbCO2glxKGdJY7Wqa7jM/g7bEdaQGhU/RuCo+WeVORzy++rZqBfyVcSdgbxX0rsG+szFcITRCZ+OuLbM+TCDXazYSk/j/xZfmOUQ/3F3hvS8gbXVKgzC6jZaLpBnyaUCvqjfbF7wFjCuwDF/W1jgL77nmEIY090Wd7CzCzjNeXG1NZo2UIZ+sXEUb7bCLUhueUs93y9oG/Rq/fQmiW+CzQVM7+WGBfK7RP7TJaJpq+L/D7aL/YQji7OSmvzIXR8pqLrPv+vGm5z+jDpZZDGDq6AWgvsOx3sCMPvES4ajh/m5wfLXN2bNp3CH0g66PP+DlC8t6twDreE73vCaU+p9xwtrqJTrFudvcDo+cPAJe4+6+jXuqD3H1BGcuZBqz0cBXXOMLpzBmEjTbW3VdEY4qvJwx5uqI670hEkioakdMNnOfu/b73lJldR8hHJw9w/X8AVrj7+0qVrfdQyOsJNbR9LVzJ9iHCN9OHzGwBoUZwepmLmw08FL3uHuA/3T03bO92M1tIuADgb1Rg2J6INB53Xwd8C/h/sSGS/XEcOy7q6xczO4QwDv6rZZWvd81dRGQ4iUaGfRb4kQ9umHF/13sS4aLK68sqr+QuIpI8dRst09nZ6TNmzKjX6kWK27Il/G8byHU2ItX18MMPr3D3rlLl6pbcZ8yYwbx58+q1epHinnkm/N9nn/rGIVKAmb1YTrmheBGTiIgMkpK7iEgCKbmLiCSQkruISAIpuYuIJJCSu4hIAim5i4gkkJK7iEgCKblLYr310vu44eHueochUhdK7pJI7s6iZetY2F3yNyhEEknJXRIpG90Pb9XGQj/2JJJ8Su6SSOlsFlByl8al5C6JlImq7kru0qiU3CWR0kru0uCU3CWRMpmQ3Fdv6kU/SCONSMldEikTJfRtGWf91nSdoxGpPSV3SaRcmzvAajXNSANScpdESseS+0old2lASu6SSLk2d1DNXRqTkrskUm6cO6jmLo1JyV0SKeuquUtjU3KXRIq3uWusuzQiJXdJpHRGyV0am5K7JFJGNXdpcErukki5ZhkzWLVJyV0aj5K7JFKuQ3VCe4tq7tKQlNwlkXJt7p2jW5XcpSEpuUsi5drcuzpaWb8lTW86W+IVIsmi5C6JlLuIqaujFYA1aneXBlMyuZvZ7mZ2l5ktMrMnzOyTBcocb2ZrzezR6O/L1QlXpDzxmjvoKlVpPM1llEkDF7j7fDPrAB42szvd/cm8cve5+6mVD1Gk/3LJfWJHG6CrVKXxlKy5u/syd58fPV4PLAKmVjswkcFQzV0aXb/a3M1sBjAHeKjA7KPNbIGZ/cHMDijy+nPNbJ6Zzevp6el3sCLlSm+vuYfkvlpt7tJgyk7uZjYa+A3wKXdflzd7PrCHux8M/Dfw+0LLcPcr3f1wdz+8q6troDGLlJSruXeObgF0lao0nrKSu5mNICT2n7v7b/Pnu/s6d98QPb4VGGFmnRWNVKQfcjX3lqYmxo4aoeQuDaec0TIG/BhY5O4XFykzOSqHmR0RLXdlJQMV6Y9slNybmozxo3SVqjSeckbLHAO8D3jMzB6Npn0BmA7g7lcA7wQ+ZmZpYDNwlusn56WOcjX35pQxXrcgkAZUMrm7+/2AlShzGXBZpYISGaxMdBFTU8oY197Cy6s21TkikdrSFaqSSLmae5OZbh4mDUnJXRIpE2tzH9fewupNvailUBqJkrskUibW5j6hvYVtGWf91nSdoxKpHSV3SaTtzTIpY9yoMNZdtyCQRqLkLom0o+aeYnx0IZNuQSCNRMldEilXc08ZjFfNXRqQkrskUiabpSllmIVx7qCauzQWJXdJpEw2tLcD25O7au7SSJTcJZEy2SzNUXIf1dJES3NKY92loSi5SyKls7695m5mdLa3sGKDkrs0DiV3SaRMLLkDdI1pY/n6LXWMSKS2lNwlkdJZ394sAzCpo5Xl67bWMSKR2lJyl0TK5tXcJ45pVc1dGoqSuyRSqLnv2L0ndrSxetM2etPZOkYlUjtK7pJI+W3uud9S7dmgphlpDErukkjpAs0yAK+uU9OMNAYld0mk3BWqORM72gDUqSoNQ8ldEimTN1omV3PvUaeqNAgld0mk/Db3Ce2tpAyWr1fNXRqDkrskUv4496aU0Tm6VW3u0jCU3CWRMlknldr5d90njWlTzV0ahpK7JFI6s3PNHcJwSHWoSqNQcpdEyvjObe6Qu0pVyV0ag5K7JFIm7wpVgK6ONlZu3Eo6o6tUJfmU3CWR8i9igtAs445u/SsNQcldEin/IiYIHaqAbiAmDUHJXRIpnSlccwddpSqNQcldEinrBUbL5O4vo5q7NICSyd3Mdjezu8xskZk9YWafLFDGzOy7ZrbYzBaa2aHVCVekPIXa3DtHt2Kmmrs0huYyyqSBC9x9vpl1AA+b2Z3u/mSszFuBvaO/I4HLo/8idZF/bxmAEU0pJrS3aDikNISSNXd3X+bu86PH64FFwNS8YqcD13gwFxhrZlMqHq1ImdKZXa9QhTAcUjcPk0bQrzZ3M5sBzAEeyps1FXg59rybXb8AMLNzzWyemc3r6enpX6Qi/VCo5g6hU/VVNctIAyg7uZvZaOA3wKfcfV3+7AIv8V0muF/p7oe7++FdXV39i1SkH8IVqrvu3hM79Fuq0hjKSu5mNoKQ2H/u7r8tUKQb2D32fBqwdPDhiQxM0Zr7mFZWbOglk92l7iGSKOWMljHgx8Aid7+4SLEbgfdHo2aOAta6+7IKxinSL+nMrhcxQbiQKZN1Vm3UVaqSbOWMljkGeB/wmJk9Gk37AjAdwN2vAG4FTgYWA5uAD1Y+VJHy5f9YR07uQqZX122hK3oskkQlk7u730/hNvV4GQfOr1RQIoOV/2MdOV3Rb6n2aDikJJyuUJVEyha45S/EbkGgTlVJOCV3SaRiNffcLQh0laoknZK7JE4267hTcChka3MTY0eN0P1lJPGU3CVx0tEwx6Yie/ekjjZeWauauySbkrskTmZ7ci+8e79mbBtL12yuZUgiNafkLomT8ZDcC7W5A0wbN4q/KblLwim5S+JkMrmae+HkPnXcSNZu3sb6LdtqGZZITSm5S+Kks+EHsJubiiT3sSMBVHuXRFNyl8TJtbmnrHjNHeBvq5XcJbmU3CVxcqNlira5q+YuDUDJXRJnx2iZwsm9c3QrLU0p1dwl0ZTcJXFyyb1Ym3sqZbxmbBvdqrlLgim5S+KkS4xzh9Durpq7JJmSuyTO9maZIh2qEEbMqM1dkkzJXRInNxSyWJs7wNSxo+hZv5Ut2zK1CkukppTcJXGi3F50tAzsGA65bK1uICbJpOQuibO95l6kQxViFzKp3V0SSsldEidTYpw7wLTchUxrNtUkJpFaU3KXxEmX0aE6ebc2UqaauySXkrskTqmLmABGNKWYPEZj3SW5lNwlcUpdxJSjse6SZErukjilfqwjR2PdJcmU3CVxSt04LGfquJG8snbL9i8DkSRRcpfEyURDIYvd8jdn6thRpLPOq+s01l2SR8ldEifdjzZ30K1/JZmU3CVxyhktAzsuZOperbHukjxK7pI45VzEBLpKVZKtZHI3s5+Y2XIze7zI/OPNbK2ZPRr9fbnyYYqUL11mzX1kSxMT2lvULCOJ1FxGmauAy4Br+ihzn7ufWpGIRAap3GYZCLch6FbNXRKoZM3d3e8FVtUgFpGKKLfmDrDHhHaWrNxY7ZBEaq5Sbe5Hm9kCM/uDmR1QoWWKDEh2e5t76d17Zmc73as3szWt+7pLslQiuc8H9nD3g4H/Bn5frKCZnWtm88xsXk9PTwVWLbKr/tTcZ3W14w4vrtSIGUmWQSd3d1/n7huix7cCI8yss0jZK939cHc/vKura7CrFikodxFTqdEyALM6RwPwfI+aZiRZBp3czWyyWbgU0MyOiJa5crDLFRmo/tTcZ3SOAuCFFUrukiwlR8uY2fXA8UCnmXUDXwFGALj7FcA7gY+ZWRrYDJzl7rpZh9RNJlN+cu9oG0FXRysvrNhQ7bBEaqpkcnf3s0vMv4wwVFJkSMh46R/riJvV2a5mGUkcXaEqiZPJOimDVBk1dwidqmqWkaRRcpfESWe9rGGQOTM721m5sZe1m7ZVMSqR2lJyl8TJZJ1+5HZm5kbMqN1dEkTJXRInnelfzX1WVzugETOSLErukjhZ97JGyuTsPm4UTSlTcpdEUXKXxElns2VdwJTT0pxi93EjeV7JXRJEyV0SJ5PtX80dQqeqhkNKkii5S+KkMwNJ7qNZsmLj9puOiQx3Su6SOAOpuc/qamfztgyvrtePZUsyKLlL4mTc+9XmDuEqVYAX1DQjCaHkLomTHkibezQc8jl1qkpCKLlL4mT6Oc4dYFJHGyNHNKnmLomh5C6Jk8562feVyUmljJmd7bo7pCSGkrskTqaf49xz9pw4mmeXK7lLMii5S+JkvLx7uefbb3IH3as3s65Xv6cqw5+SuyTOQGvus6d0APDUqt5KhyRSc0rukjgDuYgJYPaUMQAsWrW10iGJ1JySuyROJus0N/U/uU8e08bYUSOU3CURlNwlcdJZJ1XmT+zFmRmzJ49RcpdEUHKXxMkO4ArVnNlTxvD06l4yuseMDHNK7pI4oc19YLv27CkdbMk4L6zTT+7J8KbkLomTyQ6u5g7qVJXhT8ldEiedzdI0gA5VgL0njabZlNxl+FNyl8TJZJ2mAXSoArQ2N7Hn2BYldxn2lNwlcQZyy9+42eNbWaQLmWSYU3KXxMkM8CKmnNnjW3llU5rVG5XgZfhScpfESQ/wIqac2eNbAFi0bF2lQhKpOSV3SZyB/Mxe3OzxrQA8qeQuw1jJ5G5mPzGz5Wb2eJH5ZmbfNbPFZrbQzA6tfJgi5UsPokMVoGtkM50jm1i0bH0FoxKprXJq7lcBJ/Ux/63A3tHfucDlgw9LZOCy2YFfxJQze3yrmmVkWCt5BLj7vcCqPoqcDlzjwVxgrJlNqVSAIv012DZ3gNdOaOWZV9ezWfd2l2GqEm3uU4GXY8+7o2m7MLNzzWyemc3r6empwKpFdjXYNneAwya2kc46C7vXVCgqkdqqRHIvdBQVvOuSu1/p7oe7++FdXV0VWLXIrtID/LGOuDkTRwIw/yUldxmeKpHcu4HdY8+nAUsrsFyRfnN3ss6AbvkbN76tiVld7Tz84uoKRSZSW5VI7jcC749GzRwFrHX3ZRVYrki/5W7VO9iaO8Ch08cx/6XVuOv2vzL8lDMU8nrgQWBfM+s2sw+Z2UfN7KNRkVuB54HFwA+B86oWrUgJ6Si5D/TGYXGH7TGOVRt7WbJy06CXJVJrzaUKuPvZJeY7cH7FIhIZhErW3A/bYxwA819czczO9kEvT6SWdIWqJMr2mvsgx7kD7NU1mo62Zh5+Se3uMvwouUui5GruFWiVIZUy5kwfx3x1qsowpOQuibI9uTdVZtc+bPo4nn51Peu36Gf3ZHhRcpdEqWSbO4R2d3d49GWNd5fhRcldEiWdzQIM+grVnIN33w0zNN5dhh0ld0mUStfcO9pGsO+kDiV3GXaU3CVRdoyWqUxyh9A08+hLa0hnshVbpki1KblLomSrkNxfv2cn67emWaCbiMkwouQuiZKucLMMwLF7dZIyuOeZFRVbpki1KblLomQqeBFTzm6jRnDw7mO59xndplqGDyV3SZRq1NwBjtu7i4Xda1izqbeiyxWpFiV3SZRMNBQyVenkvk8XWYf7F6tpRoYHJXdJlNyAlkrX3A+ethtj2prVNCPDhpK7JEqlL2LKaW5Kcezendz7zArd312GBSV3SZRKX8QU94a9u3hl3RaeXb6h4ssWqTQld0mUalzElHPcPuF3f9U0I8OBkrskSiZTveQ+dexI9uxq595n1akqQ5+SuyRKxquX3CHU3h96fiWbetNVWb5IpSi5S6LsaHOvzq79lv0nszWd5U9PLa/K8kUqRcldEqWabe4AR8wcT1dHKzcvWFaV5YtUipK7JEruIqZqjJaB8KVx8oGTuevp5WzYqqYZGbqU3CVR0lXsUM059eDXsDWd5Y+LXq3aOkQGS8ldEiVb5Q5VCL+rOnlMGzepaUaGMCV3SZRq3TgsLpUyTjloCvc+08PazfrhbBmalNwlUTJV7lDNOeWgKfRmstz5pJpmZGhScpdEybW5V2soZM6c3ccydexIbl64tKrrERkoJXdJlFzNvcq5HTPj1IOmcP+zK1i5YWt1VyYyAErukii5K1SrXXMHOOOwaaSzzq8f7q76ukT6q6wjwMxOMrOnzWyxmX2uwPxzzKzHzB6N/j5c+VBFSqtVmzvAPpM6OHLmeK6d++L29YoMFSWTu5k1Ad8D3grsD5xtZvsXKPpLdz8k+vtRheMUKcuONvfqJ3eA9x89g+7Vm7nnGd2OQIaWcmruRwCL3f15d+8FfgGcXt2wRAYmk81iVvmf2SvmLQdMYmJHK9c8+GJN1idSrnKS+1Tg5djz7mhavjPMbKGZ3WBmuxdakJmda2bzzGxeT4/uiS2Vl846TVabxA4woinF2UdM555nenhx5caarVeklHKSe6EjJb+B8SZghrsfBPwvcHWhBbn7le5+uLsf3tXV1b9IRcqQca9Je3vc2UdMJ2XGzx96qabrFelLOcm9G4jXxKcBOw3udfeV7p4bD/ZD4LDKhCfSP5mM16y9PWfybm2ceMAkfjXvZbZsy9R03SLFlJPc/wrsbWYzzawFOAu4MV7AzKbEnp4GLKpciCLlS2drX3MH+MDRM1izaRvXqfYuQ0Tx50+RAAAMq0lEQVTJ5O7uaeCfgNsJSftX7v6EmX3NzE6Lin3CzJ4wswXAJ4BzqhWwSF8yWae5qfaXbxw5awJHz5rA9+9+js29qr1L/ZV1FLj7re6+j7vv6e7fiKZ92d1vjB5/3t0PcPeD3f0Ed3+qmkGLFJPOOqkadqjGffrN+7Biw1Z+NndJXdYvEqcrVCVRstnat7nnHDFzPG/Yu5Mr7nmejfohD6kzJXdJlHq1ued8+s37sGpjL1c/uKRuMYiAkrskTCabpbmpfsn90OnjOGHfLq6893nWbdG93qV+lNwlUepdcwe44C37snbzNi6585m6xiGNTcldEiVT4ytUCzlw6m6858jpXP3AEh7rXlvXWKRxKblLomSGQM0d4LMn7seE0a184XeP6Y6RUhdK7pIoYZx7/ZP7biNH8OVT9+exv63lmgeX1DscaUBK7pIooc19aOzWpx40heP26eI/b3+apWs21zscaTBD4ygQqZBMHce55zMzLjr9QBz4xPWPsC2TrXdI0kCU3CVR0tls3TtU46ZPGMW/v+O1zHtxNd++/el6hyMNRMldEiWbrc1P7PXH6YdM5b1HTefKe5/njideqXc40iCU3CVR0nW+iKmYL526PwdN240Lfr2AJSv0ox5SfUrukihDZShkvtbmJr737kNpShnv/8lfWL5uS71DkoRTcpdESQ+hDtV8u48fxU/PeR0rNmzlfT/+C2s29dY7JEkwJXdJlEwdb/lbjjnTx/HD9x/OCys28sGr/qq7R0rVKLlLogyVi5j6csxenXz37DkseHkN7/nRQ6zcsLX0i0T6ScldEiUzhC5i6stJB07m8vcexqJl6zjj8gfUySoVN/SPApF+GMpt7vlOPGAy133kKNZu3sY7Ln+AeUtW1TskSRAld0mUoTpappjD9hjHb887hjFtzZx55Vy++8dndaMxqQgld0mUoXaFajlmdrZz48eP5dSDpnDxnc9w9pVz6V69qd5hyTCn5C6JkslC0xDvUC1kTNsIvnPmIfzXuw7miaVrefPF9/K9uxazNZ2pd2gyTCm5S6Jkstlh0+aez8w447Bp3P7p4/i7fbr49u1P85ZL7uW2x5eRVVON9JOSuyTKUPiZvcGaNm4UV7zvMK75xyNoShkfvXY+J3/3Pm5asFTt8VI2JXdJlKF0y9/BOm6fLu741HFccubBbMtk+fj1j/DG/7qby+9+jp71GhsvfWuudwAilZTOOqmEJHeA5qYUfz9nGqcdPJXbn3iFqx9Ywrdue4qL73yaN+43kVMOeg1v3G8io1t1KMvOtEdIomQTVHOPa0oZJ792Cie/dgqLl2/g+r+8xE0LlnL7E6/S0pziDXt1ctw+XRy7dyezOtuxYTZiSCpPyV0Sw92H1M/sVcteE0fzpVP3519Pns3DL63mloXL+NNTy/njU8sBmDymjcP2GMec6WOZM30cs6d0MKpFh3qj0ScuiZHra0xizb2QVMp43YzxvG7GeC487QBeWrmJ+xb38OBzK3nkpTXc8tgyAMxg5oR29pvSwZ5do5nZ2c6srtHsPm4k49tbVMtPqLKSu5mdBFwKNAE/cvdv5s1vBa4BDgNWAme6+5LKhirSt3Q2/EbpcB8tM1DTJ4ziPRP24D1H7gHAq+u28OjLa1i0bB2Llq3jyaXruP2JV3cacTNyRBNTx41kym5tTBrTxqQxrXSObmXC6FY621sYO6qFsaNGMG5UC20jUvoiGEZKJnczawK+B7wZ6Ab+amY3uvuTsWIfAla7+15mdhbwLeDMagQsUkwuaTVqcs83aUwbJx4wmRMPmLx9Wm86y0urNvJ8z0a6V2+O/jbx6vqtPPvqCno2bC063LI5ZXS0NdPRNoLRrc20tzbR3trMqJYm2kY0hf/N4XFrc4rWESlamlK0NDfR0pxiRJPR0pSiuSlFc5MxIhX+N6eMppTRnEqRSkFzKkVTClIWpqfMSKWMlEGTGWbhccrCPAxSFq4TMML03HdQ/LGxowyEM5okf1mVU3M/Aljs7s8DmNkvgNOBeHI/HbgwenwDcJmZmbtXfFDubY+/wgW/erTSi5UEyO1sjdIsMxAtzSn2mtjBXhM7Cs7PZJ21m7excsNWVmzoZe3mXtZs2sbqTdtYt2UbG7akWb9lGxu2ptm4NcOqjb10r86wuTfD5m0ZtkR/w3E4fvQ9ET3esQ9ZXpkd03d6UrJ83EfeMIsL3rLvQEMtSznJfSrwcux5N3BksTLunjaztcAEYEW8kJmdC5wbPd1gZgP9OfjO/GUPEUM1Lhi6sVU8ro98Cz4y+MU0zPaqEMXVD5+Bzs8MPK49yilUTnIv9N2T/71cThnc/UrgyjLW2XdAZvPc/fDBLqfShmpcMHRjU1z9o7j6p5HjKmfMWDewe+z5NGBpsTJm1gzsBujm1CIidVJOcv8rsLeZzTSzFuAs4Ma8MjcCH4gevxP4UzXa20VEpDwlm2WiNvR/Am4nDIX8ibs/YWZfA+a5+43Aj4GfmdliQo39rGoGTQWadqpkqMYFQzc2xdU/iqt/GjYuUwVbRCR5kn2dtohIg1JyFxFJoCGZ3M2sycweMbObC8xrNbNfmtliM3vIzGbE5n0+mv60mZ1Y47j+2cyeNLOFZvZHM9sjNi9jZo9Gf/md0dWO6xwz64mt/8OxeR8ws2ejvw/kv7bKcV0Si+kZM1sTm1e17WVmS8zssWjZ8wrMNzP7brQfLTSzQ2Pzqra9yojrPVE8C83sATM7uNzXVjmu481sbezz+nJs3knRsbjYzD5X47g+G4vp8WifGl/OawcZ11gzu8HMnjKzRWZ2dN782u1f7j7k/oB/Bq4Dbi4w7zzgiujxWcAvo8f7AwuAVmAm8BzQVMO4TgBGRY8/losrer6hjtvrHOCyAtPHA89H/8dFj8fVKq68ch8ndNRXfXsBS4DOPuafDPyBcO3GUcBDtdheZcT1+tz6gLfm4irntVWO6/gi+11TdAzOAlqiY3P/WsWVV/ZthBF8tdheVwMfjh63AGPrtX8NuZq7mU0DTgF+VKTI6YQNCOFWB//HzCya/gt33+ruLwCLCbdOqElc7n6Xu+d+sn4u4XqAqitjexVzInCnu69y99XAncBJdYrrbOD6Sq17kE4HrvFgLjDWzKZQ5e1Virs/EK0Xarh/DcL225a4ey+Qu21JPdRk/zKzMcBxhNGDuHuvu6/JK1az/WvIJXfgO8D/A7JF5u90qwMgd6uDQrdJmFrDuOI+RPh2zmkzs3lmNtfM3l7BmMqN64zoFPAGM8tdkDYktlfUfDUT+FNscjW3lwN3mNnDFm6Hka/Ydqn29ioVV1z+/tWf11YjrqPNbIGZ/cHMDoimDYntZWajCEnyN/197QDMAnqAn0bNkT8ys/a8MjXbv4bU/dzN7FRgubs/bGbHFytWYJr3Mb1WceXKvhc4HPi72OTp7r7UzGYBfzKzx9z9uRrFdRNwvbtvNbOPEs563sgQ2V6EprUb3D0Tm1aV7RU5Jlr2ROBOM3vK3e+Nh1/gNVXdv8qMKwRndgIhuR/b39dWKa75wB7uvsHMTgZ+D+zNENlehCaZP7v7qgG8tr+agUOBj7v7Q2Z2KfA54EuxMjXbv4Zazf0Y4DQzW0I4jXujmV2bV6bYrQ7KuU1CNePCzN4E/Ctwmrtv/wVjd18a/X8euBuYU6u43H1lLJYfEu65D0Nge0XOIu+UuYrbK77s5cDv2LXprth2qeb2KicuzOwgQjPX6e6+sj+vrVZc7r7O3TdEj28FRphZJ0Nge0X62r8qvb26gW53fyh6fgMh2eeXqc3+VY1OhUr8Ubyj5nx27lD9VfT4AHbuUH2eCneolohrDqEDae+86eOA1uhxJ/AsFexYKiOuKbHHfw/M9R0dOC9E8Y2LHo+vVVzRvH0JnVtWi+0FtAMdsccPACfllTmFnTu8/lLt7VVmXNMJ/Uiv7+9rqxzX5NznR0iSL0Xbrjk6Bmeyo0P1gFrFFc3LVfzaa7G9omXeB+wbPb4Q+Ha99q8h1SxTjJVxqwMPt0T4FeE+82ngfN/5VL/acX0bGA38OvTv8pK7nwbMBn5gZlnCmdI3fecfOql2XJ8ws9MI22QVYfQM7r7KzL5OuHcQwNd851PXascFoaPrFx7t3ZFqbq9JwO+iz6cZuM7db4uaq3D3K4BbCSMaFgObgA9G86q5vcqJ68uEvqXvR+XSHu4qWPC1NYzrncDHzCwNbAbOij7PgrctqWFcECozd7j7xlKvrVBcEEZ+/dzCfbieBz5Yr/1Ltx8QEUmgodbmLiIiFaDkLiKSQEruIiIJpOQuIpJASu4iIgmk5C4ikkBK7iIiCfT/AfiHQPGAE2IuAAAAAElFTkSuQmCC\n", 243 | "text/plain": [ 244 | "
" 245 | ] 246 | }, 247 | "metadata": { 248 | "needs_background": "light" 249 | }, 250 | "output_type": "display_data" 251 | }, 252 | { 253 | "name": "stdout", 254 | "output_type": "stream", 255 | "text": [ 256 | "The red line shows the actual b=5. The peak of Ln will always be at an x value less than b\n" 257 | ] 258 | } 259 | ], 260 | "source": [ 261 | "n = 50\n", 262 | "Xn = np.random.uniform(0, 5, n)\n", 263 | "Xmax = max(Xn)\n", 264 | "x = np.linspace(4, 6, 100)\n", 265 | "Ln = np.array([0 if i < Xmax else 1/i**n for i in x])\n", 266 | "\n", 267 | "plt.title('Likelihood Function for b of a Uniform(0,5)', fontsize=16)\n", 268 | "plt.plot([5, 5], [0, 1], c='red', alpha=0.2)\n", 269 | "plt.plot(x, Ln)\n", 270 | "plt.ylim((0, max(Ln) + 1/20 * max(Ln)))\n", 271 | "plt.show()\n", 272 | "\n", 273 | "print('The red line shows the actual b=5. The peak of Ln will always be at an x value less than b')" 274 | ] 275 | } 276 | ], 277 | "metadata": { 278 | "kernelspec": { 279 | "display_name": "Python 3", 280 | "language": "python", 281 | "name": "python3" 282 | }, 283 | "language_info": { 284 | "codemirror_mode": { 285 | "name": "ipython", 286 | "version": 3 287 | }, 288 | "file_extension": ".py", 289 | "mimetype": "text/x-python", 290 | "name": "python", 291 | "nbconvert_exporter": "python", 292 | "pygments_lexer": "ipython3", 293 | "version": "3.7.0" 294 | } 295 | }, 296 | "nbformat": 4, 297 | "nbformat_minor": 2 298 | } 299 | -------------------------------------------------------------------------------- /U03 L11 - Fisher Info, Asym Normality of MLE, Method of Moments.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Unit 3 Lecture 11: Fisher Information, Asymptotic Normality of MLE, Method of Moments" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "## Fisher Information\n", 15 | "\n", 16 | "Fisher Information is a way of measuring the amount of information each i.i.d. variable $X_i$ carries about an unknown parameter $\\theta_j$ in the distribution of $X \\sim P(\\theta_0, \\cdots, \\theta_n)$. In other words, it's the amount of information each sample provides on a parameter of the sample's distribution. Geometrically, it tells you, on average, how curved the log-likelihood of $(x_1, \\theta)$ is.\n", 17 | "\n", 18 | "Fisher Information is calculated using likelihood function of the pdf of $X$ with one sample. The likelihood function with one sample is just the pdf.\n", 19 | "\n", 20 | "$X \\sim \\mathcal{N}(\\mu, \\sigma^2)$, $\\theta := \\sigma^2$\n", 21 | "\n", 22 | "$L(\\theta | X) = \\dfrac{1}{\\sqrt{2 \\pi \\theta}} \\exp(-\\frac{1}{2 \\theta} (X - \\mu)^2)$\n", 23 | "\n", 24 | "To get to the Fisher Information, we first take the derivative of the log-likelihood, just like in getting $\\hat{\\theta}_{MLE}$. \n", 25 | "\n", 26 | "$\\dfrac{\\partial \\mathcal{ln}(L)}{\\partial \\theta} = \\dfrac{-\\theta + X^2 - 2 \\mu X + \\mu^2}{2 \\theta^2}$\n", 27 | "\n", 28 | "From here, there are two ways to get the Fisher Information. We can get the variance of the first derivative, or we can just get the negative expectation of the second derivative.\n", 29 | "\n", 30 | "$\\dfrac{\\partial^2 \\mathcal{ln}(L)}{\\partial \\theta^2} = \\dfrac{\\theta - 2 X^2 + 4 \\mu X - 2 \\mu^2}{2 \\theta^3}$\n", 31 | "\n", 32 | "Now we get the negative expected value, where $\\mathbb{E}[X] = \\mu$ and $\\mathbb{E}[X^2] = \\mu^2 + \\sigma^2$, so...\n", 33 | "\n", 34 | "$I(\\theta) = -\\mathbb{E}[\\dfrac{\\partial^2 \\mathcal{log}(L)}{\\partial \\theta^2}] = -\\dfrac{\\theta - 2 (\\mu^2 + \\theta) + 4 (\\mu) (\\mu) - 2 (\\mu)^2}{2 \\theta^3} = \\dfrac{1}{2 \\theta^2} = \\dfrac{1}{2 \\sigma^4} = (Var(\\hat{\\sigma^2}_{MLE}))^{-1}$\n", 35 | "\n", 36 | "Note this works no matter what $\\mu$ is. Now, we know the asymptotic variance of the variance estimator is...\n", 37 | "\n", 38 | "$Var(\\hat{\\sigma^2}_{MLE}) = 2 \\sigma^4$\n", 39 | "\n", 40 | "And we know that of all estimators, the Maximum Likelihood Estimator has the smallest variance.\n", 41 | "\n", 42 | "The complete Fisher Information for the model is the covariance matrix:\n", 43 | "\n", 44 | "$I_L(\\mu, \\sigma^2) = \\begin{pmatrix}\n", 45 | "-\\mathbb{E}[\\dfrac{\\partial^2 \\mathcal{log}(L)}{\\partial \\mu^2}] & -\\mathbb{E}[\\dfrac{\\partial^2 \\mathcal{log}(L)}{\\partial \\mu \\partial \\sigma^2}] \\\\\n", 46 | "-\\mathbb{E}[\\dfrac{\\partial^2 \\mathcal{log}(L)}{\\partial \\sigma^2 \\mu}] & -\\mathbb{E}[\\dfrac{\\partial^2 \\mathcal{log}(L)}{\\partial (\\sigma^2)^2}]\n", 47 | "\\end{pmatrix} = \\begin {pmatrix}\n", 48 | "\\dfrac{1}{\\sigma^2} & 0 \\\\\n", 49 | "0 & \\dfrac{1}{2 \\sigma^4}\n", 50 | "\\end{pmatrix}$\n", 51 | "\n", 52 | "Some ambiguous, probably complicated conditions have to be met for the inverse of the Fisher Information to be equal to the variance of MLE. So far it's always true.\n", 53 | "\n", 54 | "### Parameter of a distribution depending on parameter of another distribution\n", 55 | "\n", 56 | "If variable $Y$ has a parameter that depends on a parameter of variable $X$...\n", 57 | "\n", 58 | "$X \\sim Exp(\\lambda)$\n", 59 | "\n", 60 | "$Y \\sim Ber(p(\\lambda))$\n", 61 | "\n", 62 | "where $z$ is some threshold on the exponential distribution such that everything $> z$ is recorded as a $1$ and everything $< z$ gets a $0$, it makes sense that we can get some information about $\\lambda$ from $p$. It's important, then, that we focus on $\\lambda$ in the process of looking for information about it.\n", 63 | "\n", 64 | "$L(\\lambda | Y) = e^{-\\lambda z Y}(1 - e^{-\\lambda z})^{1 - Y}$\n", 65 | "\n", 66 | "$\\mathscr{l} = \\mathcal{ln}(L(\\lambda | Y)) = -\\lambda z Y + (1 - Y) ln(1 - e^{-\\lambda z})$\n", 67 | "\n", 68 | "Since we want information about $\\lambda$, we take the derivative with respect to it, rather than taking the derivative wrt $p = e^{-\\lambda z}$. With a little more work, this eventually yields $I(\\lambda)$. It's also possible to use the delta method here, though that's definitely not the easiest way to get the asymptotic variance of $\\hat{\\lambda}$." 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "## Asymptotic Normality of MLE\n", 76 | "\n", 77 | "MLE estimators are asympotically normal *under some regularity conditions*. If an MLE estimator is asymptotically normal, that means (among many things) that we can apply the delta method to find the variance of $g(\\hat{\\theta}_{MLE})$.\n", 78 | "\n", 79 | "$\\sqrt{n} (\\hat{\\theta}_{MLE} - \\theta) \\to \\mathcal{N}(0, \\sigma^2)$\n", 80 | "\n", 81 | "means\n", 82 | "\n", 83 | "$\\sqrt{n} (g(\\hat{\\theta}_{MLE}) - g(\\theta)) \\to \\mathcal{N}(0, g\\prime(E[X])^2\\sigma^2)$\n", 84 | "\n", 85 | "and, multivariate\n", 86 | "\n", 87 | "$\\sqrt{n} (g(\\hat{\\theta}_{MLE}) - g(\\theta)) \\to \\mathcal{N}(0, \\nabla g(\\theta)^T\\Sigma_X \\nabla g(\\theta))$" 88 | ] 89 | }, 90 | { 91 | "cell_type": "markdown", 92 | "metadata": {}, 93 | "source": [ 94 | "## Method of Moments\n", 95 | "\n", 96 | "$E[X^k]$ is the $k$th moment of an R.V. $X$. \n", 97 | "\n", 98 | "### $\\frac{1}{n} \\sum_{i=1}^n X_i^k = \\overline{X^k}_n$ \n", 99 | "\n", 100 | "is the estimator for the $k$th moment. A simple combination of method of moments estimators can be used to estimate a parameter, such as in $\\overline{X^2}_n - \\overline{X_n}^2 \\to \\sigma^2$ of a normal.\n", 101 | "\n", 102 | "The formulas for the $k$th moment are, of course\n", 103 | "\n", 104 | "$\\int_x x^k f_X(x) dx = \\mathbb{E}[X^k]$\n", 105 | "\n", 106 | "$\\sum_x x^k f_X(x) = \\mathbb{E}[X^k]$\n", 107 | "\n", 108 | "\n", 109 | "### Recovering parameters from moments\n", 110 | "\n", 111 | "If $\\psi$ is a function that maps the parameters of $X$ to a list of moments of $X$ \n", 112 | "\n", 113 | "- $\\psi(\\theta^{(X)}_1, \\cdots, \\theta^{(X)}_n) = (m_1, m_2, \\cdots) = (\\mathbb{E}[X], \\mathbb{E}[X^2], \\cdots)$,\n", 114 | "\n", 115 | "And if $\\psi$ is one-to-one, meaning given the output of $\\psi$, there could only be one set of parameters that generated that output, then\n", 116 | "\n", 117 | "- $\\psi^{-1}(m_1, m_2, \\cdots) = (\\theta^{(X)}_1, \\cdots, \\theta^{(X)}_n)$\n", 118 | "\n", 119 | "e.g. $X \\sim \\mathcal{N}(\\mu, \\sigma^2)$\n", 120 | "\n", 121 | "$\\psi_{(1, 2)}(\\mu, \\sigma^2) = (\\mu, \\mu^2 + \\sigma^2)$\n", 122 | "\n", 123 | "$\\psi^{-1}(\\mu, \\mu^2 + \\sigma^2) = (\\mu, \\sigma^2)$" 124 | ] 125 | }, 126 | { 127 | "cell_type": "markdown", 128 | "metadata": {}, 129 | "source": [ 130 | "## MLE vs MoM\n", 131 | "\n", 132 | "- MoM provides consistent, but often biased estimators. Typically easier to compute than MLE (though in this class MLE more often than not has been MoM). Doesn't give a good result if the model isn't well specified since noisy data can throw a mean estimation far away from the true mean.\n", 133 | "- MLE is more accurate by measure of quadratic risk, and still gives good results if the model isn't well specified." 134 | ] 135 | } 136 | ], 137 | "metadata": { 138 | "kernelspec": { 139 | "display_name": "Python 3", 140 | "language": "python", 141 | "name": "python3" 142 | }, 143 | "language_info": { 144 | "codemirror_mode": { 145 | "name": "ipython", 146 | "version": 3 147 | }, 148 | "file_extension": ".py", 149 | "mimetype": "text/x-python", 150 | "name": "python", 151 | "nbconvert_exporter": "python", 152 | "pygments_lexer": "ipython3", 153 | "version": "3.7.0" 154 | } 155 | }, 156 | "nbformat": 4, 157 | "nbformat_minor": 2 158 | } 159 | --------------------------------------------------------------------------------