├── .gitignore
├── 03_covariance-and-correlation-coefficient.ipynb
├── 05_gaussian-process.ipynb
└── 02_maximum_likelihood.ipynb
/.gitignore:
--------------------------------------------------------------------------------
1 | # Project
2 | .ipynb_checkpoints
3 |
4 | # Python
5 | *.pyc
6 |
--------------------------------------------------------------------------------
/03_covariance-and-correlation-coefficient.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": 1,
6 | "metadata": {},
7 | "outputs": [
8 | {
9 | "name": "stdout",
10 | "output_type": "stream",
11 | "text": [
12 | "Populating the interactive namespace from numpy and matplotlib\n"
13 | ]
14 | }
15 | ],
16 | "source": [
17 | "%pylab inline\n",
18 | "import numpy as np\n",
19 | "import pandas as pd"
20 | ]
21 | },
22 | {
23 | "cell_type": "markdown",
24 | "metadata": {},
25 | "source": [
26 | "# Covariance\n",
27 | "\n",
28 | "서로 다른 populations으로 부터 가져온 samples을 비교할때, 가장 많이 사용되는 방법이 covariance와 correlation입니다.
\n",
29 | "Sample covariance의 공식은 다음과 같습니다.\n",
30 | "\n",
31 | "$$ S_{XY} = \\frac{\\sum^N_{i=1} (X_i - \\bar{X})(Y_i - \\bar{Y}) }{N-1} $$\n",
32 | "\n",
33 | "위의 공식은 2개의 vectors의 covariance를 구할때 사용하고, numpy.cov 의 경우는 variance-covariance matrix로 표현이 됩니다. (covariance matrix라고도 합니다.)\n",
34 | "\n",
35 | "\n",
36 | "## Example\n",
37 | "\n",
38 | "수학, 영어 그리고 물리의 A학급의 점수가 다음과 같이 있을때 각 과목마다의 covariance는?\n",
39 | "\n",
40 | "| Student | Math | English | Physics |\n",
41 | "|:--------|:-----|:--------|:--------|\n",
42 | "| A | 4.0 | 2.0 | 0.6 |\n",
43 | "| B | 4.2 | 2.1 | 0.59 |\n",
44 | "| C | 3.9 | 2.0 | 0.58 |\n",
45 | "| D | 4.3 | 2.1 | 0.62 |\n",
46 | "| E | 4.1 | 2.2 | 0.63 |\n",
47 | "\n",
48 | "\n",
49 | "\n"
50 | ]
51 | },
52 | {
53 | "cell_type": "code",
54 | "execution_count": 2,
55 | "metadata": {},
56 | "outputs": [
57 | {
58 | "name": "stdout",
59 | "output_type": "stream",
60 | "text": [
61 | "[Numpy Covariance]\n",
62 | "[[0.025 0.0075 0.00175]\n",
63 | " [0.0075 0.007 0.00135]\n",
64 | " [0.00175 0.00135 0.00043]]\n",
65 | "\n",
66 | "[My Covariance]\n",
67 | "[[0.025 0.0075 0.00175]\n",
68 | " [0.0075 0.007 0.00135]\n",
69 | " [0.00175 0.00135 0.00043]]\n"
70 | ]
71 | },
72 | {
73 | "data": {
74 | "text/plain": [
75 | ""
76 | ]
77 | },
78 | "execution_count": 2,
79 | "metadata": {},
80 | "output_type": "execute_result"
81 | },
82 | {
83 | "data": {
84 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUYAAAD8CAYAAAAL8Q1XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFY5JREFUeJzt3X+wHWV9x/H3p/mFVX6ZKNIEDTRpLdJ2lEyktdOhIjQyltARxjCOQAcbrWb605nGOqYtg1PpH+3owMhEQIFRg4Uqt20shYKltkOaK0ZDQPSSapMQxQQnkNIQL/30j/Nceny4956TnD3nniSf18yZu2f32X2+LMyH3bPP7so2ERHx/35ipguIiBg2CcaIiEqCMSKikmCMiKgkGCMiKgnGiIhKT8Eo6eWS7pH07fL35CnaPS9pS/mM9NJnRBy9JK2Q9JikMUlrJ1k+T9LtZfkmSYvL/PMlfVXS1vL3zW3rfLlscyKDXtmxjl7GMUr6S+Ap2x8t/xAn2/7jSdrtt/2yw+4oIo56kmYB3wLOB3YCm4HLbD/S1uZ9wC/Yfq+kVcBv2n6HpNcD37f9hKSzgLttLyzrfBn4gO3Rbmvp9VR6JXBLmb4FuLjH7UXEsWs5MGZ7u+2DwAZaGdOuPXPuAM6TJNtfs/1Emb8NeImkeYdbyOzDXbE4xfbuMv094JQp2h0naRQYBz5q+4uTNZK0GlgNMAfOXtBjcUez3a8+e6ZLGH7/9dRMV3AE+M89tl/RyxaWSH62y7a7W6F1oG3Wetvry/RCYEfbsp3AG6tNvNDG9rikfcB8YE9bm7cDD9l+rm3epyQ9D9wJXOMOp8odg1HSvcCrJln0ofYvti1pqs5eY3uXpDOA+yRttf143ajsoPUAPyV5dafijmF//qGuzwqOXe/ZMNMVHAEu+26vW3gWeE+Xbf8MDthe1mufU5H0OuBa4IK22e8s+XM8rWB8F3DrdNvpGIy23zJNEd+XdKrt3ZJOBZ6cYhu7yt/t5Xz/9cCLgjEijmm7gNPavi8q8yZrs1PSbOBEYC+ApEXAF4DL2w+82vLnGUmfpXXKPm0w9vob4whwRZm+ArirbiDp5IlzfUkLgDcBj9TtIuKYtxlYKul0SXOBVbQypl175lwC3FfOVk8C/gFYa/vfJhpLml1yB0lzgLcBD3cqpNdg/ChwvqRvA28p35G0TNKNpc3PAaOSvg7cT+s3xgRjRPwY2+PAGuBu4FHg87a3Sbpa0kWl2U3AfEljwB8CE0N61gBLgHXVsJx5wN2SvgFsoXXE+clOtfR08cX2XuC8SeaPAu8u0/8O/Hwv/UTEscH2RmBjNW9d2/QB4NJJ1rsGuGaKzR7ylcrc+RIRUUkwRkRUeh3HGBHHuOOAn53pIhqWI8aIiEqCMSKikmCMiKgkGCMiKgnGiIhKgjEiopLhOhHRk+OA1850EQ3LEWNERCXBGBFRSTBGRFQSjBERlQRjREQlwRgRUUkwRkRUMo4xInrykp+A1760y8bP9LWUxuSIMSKikmCMiKgkGCMiKgnGiIhKgjEiopJgjIioZLhORPRE8+C4xV023trPSpqTI8aIiEqCMSKi0kgwSloh6TFJY5LWTrJ8nqTby/JNkhY30W9ERD/0HIySZgHXA28FzgQuk3Rm1ewq4Ie2lwB/DVzba78REf3SxBHjcmDM9nbbB4ENwMqqzUrgljJ9B3CeJDXQd0RE45oIxoXAjrbvO8u8SdvYHgf2AfMb6DsionFDNVxH0mpgNcCJM1xLRHRpLnB6l22PoeE6u4DT2r4vKvMmbSNpNq3c21tvyPZ628tsL/vJBgqLiDgcTQTjZmCppNMlzQVWASNVmxHgijJ9CXCfbTfQd0RE43o+lbY9LmkNcDcwC7jZ9jZJVwOjtkeAm4DbJI0BT9EKz4iIodTIOEbbG23/jO2ftv2RMm9dCUVsH7B9qe0ltpfb3t5EvxFxdDncMdGSzpf0VUlby983t61zdpk/Junj3YyIyZ0vETEUehwTvQf4Dds/T+tnu9va1vkE8NvA0vJZ0amWBGNEDIvDHhNt+2u2nyjztwEvKUeXpwIn2H6wXNe4Fbi4UyFDNVwnIo5A84DFjWxpsjHRb5yqTbm+MTEmek9bm7cDD9l+TtLCsp32bdbjrF8kwRgRg7RA0mjb9/W21ze1cUmvo3V6fUEv20kwRsQg7bG9bIplhzImemc9JlrSIuALwOW2H29rv6jDNl8kvzFGxLA47DHRkk4C/gFYa/vfJhrb3g08LemccjX6cuCuToUkGCNiKJTnKEyMiX4U+PzEmGhJF5VmNwHzy5joPwQmhvSsAZYA6yRtKZ9XlmXvA24ExoDHgS91qiWn0hExNGxvBDZW89a1TR8ALp1kvWuAa6bY5ihw1qHUkSPGiIhKgjEiopJT6YjozaE8duwIkSPGiIhKgjEiopJgjIioJBgjIioJxoiISoIxIqKS4ToR0Zt5ZLhORMTRLsEYEVFJMEZEVBKMERGVBGNERCXBGBFRyXCdiOhNc28JHBo5YoyIqCQYIyIqCcaIiEojwShphaTHJI1JWjvJ8isl/aDt7V3vbqLfiIh+6Pnii6RZwPXA+cBOYLOkEduPVE1vt72m1/4iIvqtiSPG5cCY7e22DwIbgJUNbDciYkY0MVxnIbCj7ftO4I2TtHu7pF8FvgX8ge0ddQNJq4HVALz81fz5X3y3gfIiop88F8bzdJ3D8nfAYtu/ANwD3DJZI9vrbS+zvYyXvWJApUVE/LgmgnEXcFrb90Vl3gts77X9XPl6I3B2A/1GRPRFE8G4GVgq6XRJc4FVwEh7A0mntn29CHi0gX4jIvqi598YbY9LWgPcDcwCbra9TdLVwKjtEeB3JV0EjANPAVf22m9ERL80cq+07Y3AxmreurbpDwIfbKKviIh+y50vERGVBGNERCWPHYuInvxo1mx2nrCgy9bf62stTckRY0REJcEYEVFJMEZEVBKMERGVBGNEDI0unu06T9LtZfkmSYvL/PmS7pe0X9J11TpfLtuceB7sKzvVkavSETEUuny261XAD20vkbQKuBZ4B3AA+DBwVvnU3ml7tNtaEowR0ZODzGUHi7psPe1wnRee7QogaeLZru3BuBL4szJ9B3CdJNn+b+ArkpYcSu1Tyal0RAzSAkmjbZ/Vbcsme7brwmr9F9rYHgf2AfO76PdT5TT6w5LUqXGOGCNikPbYXjbgPt9pe5ek44E7gXcBt063Qo4YI2JYdHy2a3sbSbOBE4G9023U9q7y9xngs7RO2aeVYIyIYdHx2a7l+xVl+hLgPtueaoOSZktaUKbnAG8DHu5USE6lI2IodPls15uA2ySN0Xq266qJ9SV9BzgBmCvpYuAC4LvA3SUUZwH3Ap/sVEuCMSKGRhfPdj0AXDrFuoun2Owhv0olwRgRPTnIXHb+2E+D0+l6KOGMym+MERGVBGNERCXBGBFRSTBGRFQSjBERlQRjREQlw3Uioiet4TrdPl3nyJAjxoiISoIxIqKSYIyIqDQSjJJulvSkpEmfWqGWj5f3NHxD0hua6Dcioh+aOmL8NLBimuVvBZaWz2rgEw31GxHRuEaC0fYDtB4BNJWVwK1ueRA4SdKpTfQdEdG0Qf3G2M27HJC0euJdEOz/wYBKi4j4cUM1jtH2emA9gF6zbMqn8kbE8PgRc9jR9WPHjgyDOmLs5l0OERFDYVDBOAJcXq5OnwPss717QH1HRBySRk6lJX0OOJfWO2N3An8KzAGwfQOtR5VfCIwBzwK/1US/ERH90Egw2r6sw3ID72+ir4iIfsudLxERlQRjRERlqIbrRMSRJ48di4g4BiQYIyIqCcaIiEqCMSKikmCMiKgkGCMiKhmuExE9OchcdmS4TkTE0S3BGBFRSTBGRFQSjBERlQRjREQlwRgRQ0PSCkmPlXfQr51k+TxJt5flmyQtLvPnS7pf0n5J11XrnC1pa1nn45LUqY4M14mInrSertP7y7AkzQKuB86n9SbRzZJGbD/S1uwq4Ie2l0haBVwLvAM4AHwYOKt82n0C+G1gE623CawAvjRdLTlijIhhsRwYs73d9kFgA6130rdbCdxSpu8AzpMk2/9t+yu0AvIF5f31J9h+sLxJ4Fbg4k6FJBgjYpAWTLw7vnxWty3r5v3zL7SxPQ7sA+ZP09/Csp3ptvkiOZWOiEHaY3vZTBfRSY4YI2JYdPP++RfaSJoNnAjs7bDN9vsVu3qnfYIxIobFZmCppNMlzQVW0XonfbsR4IoyfQlwX/ntcFLl/fVPSzqnXI2+HLirUyE5lY6IoWB7XNIa4G5gFnCz7W2SrgZGbY8ANwG3SRoDnqIVngBI+g5wAjBX0sXABeWK9vuATwMvoXU1etor0pBgjIghYnsjrSE17fPWtU0fAC6dYt3FU8wf5cVDeKaVYIyInowfnMP3vpvHjkVEHNUSjBERlUaCUdLNkp6U9PAUy8+VtE/SlvJZN1m7iIhh0NRvjJ8GrqN1u81U/tX22xrqLyKibxo5YrT9AK1L5xERR7xBXpX+JUlfB54APmB7W92g3DdZ7p1cAO/ZMMDy4ujzzZkuII5QgwrGh4DX2N4v6ULgi8DSupHt9cB6AOmMKUezR8QQeU7w+NE18m8gV6VtP217f5neCMyRtGAQfUdEHKqBBKOkV008NVfS8tLvdDd+R0TMmEaOfyV9DjiX1rPWdgJ/CswBsH0DrZu9f0fSOPA/wKrpbvyOiJhJjQSj7cs6LL+O1nCeiIihlztfIiIqCcaIiMrRdY09IgbvOeDxmS6iWTlijIioJBgjIioJxoiISoIxIqKSYIyIqCQYIyIqGa4TEb3JcJ2IiKNfgjEiopJgjIioJBgjIioJxoiISoIxIqKSYIyIqGQcY0T0JuMYIyKOfgnGiIhKgjEiopJgjIioJBgjIioJxogYGpJWSHpM0piktZMsnyfp9rJ8k6TFbcs+WOY/JunX2+Z/R9JWSVskjXZTR4brRERvDgBjvW9G0izgeuB8YCewWdKI7Ufaml0F/ND2EkmrgGuBd0g6E1gFvA74KeBeST9j+/my3q/Z3tNtLTlijIhhsRwYs73d9kFgA7CyarMSuKVM3wGcJ0ll/gbbz9n+T1pRvfxwC0kwRsQgLZA02vZZ3bZsIbCj7fvOMo/J2tgeB/YB8zusa+CfJH216m9KPZ9KSzoNuBU4pRSw3vbHqjYCPgZcCDwLXGn7oV77jogjzh7bywbc56/Y3iXplcA9kr5p+4HpVmjiiHEc+CPbZwLnAO8v5/vt3gosLZ/VwCca6Dciji67gNPavi8q8yZtI2k2cCKwd7p1bU/8fRL4Al2cYvccjLZ3Txz92X4GeJQXH/6uBG51y4PASZJO7bXviDiqbAaWSjpd0lxaF1NGqjYjwBVl+hLgPtsu81eVq9an0zoI+w9JL5V0PICklwIXAA93KqTRq9Ll0vnrgU3VoqnO/3c32X9EHLlsj0taA9wNzAJutr1N0tXAqO0R4CbgNkljwFO0wpPS7vPAI7TOYt9v+3lJpwBfaP2ax2zgs7b/sVMtjQWjpJcBdwK/b/vpw9zGalqn2sCCpkqLiH5q8Ok6tjcCG6t569qmDwCXTrHuR4CPVPO2A794qHU0clVa0hxaofgZ2387SZNufjvA9nrby1o/zh7fRGkREYes52AsV5xvAh61/VdTNBsBLlfLOcA+2zmNjoih1MSp9JuAdwFbJW0p8/4EeDWA7RtoHRpfSGvQ5bPAbzXQb0REX/QcjLa/AqhDGwPv77WviIhByJ0vERGVBGNERCVP14mI3vyv4ZkDM11Fo3LEGBFRSTBGRFQSjBERlQRjREQlwRgRUUkwRkRUEowREZWMY4yIHv0P8M2ZLqJROWKMiKgkGCMiKgnGiIhKgjEiopJgjIioJBgjIioZrhMRPTpAhutERBzlEowREZUEY0REJcEYEVFJMEZEVBKMERGVDNeJiB4dAB6b6SIalSPGiIhKgjEiopJgjIio9ByMkk6TdL+kRyRtk/R7k7Q5V9I+SVvKZ12v/UZE9EsTF1/GgT+y/ZCk44GvSrrH9iNVu3+1/bYG+ouI6Kuejxht77b9UJl+BngUWNjrdiMiZopsN7cxaTHwAHCW7afb5p8L3AnsBJ4APmB72yTrrwZWl69nAQ83VlwzFgB7ZrqINqlnesNWDwxfTT9r+/heNiDpH2n9c3Vjj+0VvfQ3CI0Fo6SXAf8CfMT231bLTgD+1/Z+SRcCH7O9tMP2Rm0va6S4hgxbTalnesNWDwxfTcNWz7Bo5Kq0pDm0jgg/U4cigO2nbe8v0xuBOZK6/T9MRMRANXFVWsBNwKO2/2qKNq8q7ZC0vPS7t9e+IyL6oYmr0m8C3gVslbSlzPsT4NUAtm8ALgF+R9I4rbdzr3Lnc/j1DdTWtGGrKfVMb9jqgeGradjqGQqNXnyJiDga5M6XiIhKgjEiojI0wSjp5ZLukfTt8vfkKdo933Zr4Ugf6lgh6TFJY5LWTrJ8nqTby/JNZexmX3VR05WSftC2X97dx1pulvSkpEnHmKrl46XWb0h6Q79qOYSaBnZLape3yA50H+W23cNgeyg+wF8Ca8v0WuDaKdrt72MNs4DHgTOAucDXgTOrNu8DbijTq4Db+7xfuqnpSuC6Af17+lXgDcDDUyy/EPgSIOAcYNMQ1HQu8PcD2j+nAm8o08cD35rk39dA91GXNQ1sHx0Jn6E5YgRWAreU6VuAi2eghuXAmO3ttg8CG0pd7drrvAM4b2Io0gzWNDC2HwCemqbJSuBWtzwInCTp1BmuaWDc3S2yA91HXdYUbYYpGE+xvbtMfw84ZYp2x0kalfSgpKbDcyGwo+37Tl78H9ALbWyPA/uA+Q3Xcag1Aby9nJbdIem0PtbTSbf1DtovSfq6pC9Jet0gOiw/s7we2FQtmrF9NE1NMAP7aFgN9NUGku4FXjXJog+1f7FtSVONI3qN7V2SzgDuk7TV9uNN13qE+Tvgc7afk/QeWke0b57hmobJQ7T+u5m4JfWLwLS3pPaq3CJ7J/D7bntuwEzqUNPA99EwG+gRo+232D5rks9dwPcnTifK3yen2Mau8nc78GVa//dryi6g/WhrUZk3aRtJs4ET6e9dPB1rsr3X9nPl643A2X2sp5Nu9uFAecC3pHa6RZYZ2Ee5bffQDNOp9AhwRZm+ArirbiDpZEnzyvQCWnfd1M997MVmYKmk0yXNpXVxpb7y3V7nJcB9Lr9e90nHmqrfpy6i9RvSTBkBLi9XXs8B9rX9RDIjBnlLauln2ltkGfA+6qamQe6jI8JMX/2Z+ND6ne6fgW8D9wIvL/OXATeW6V8GttK6MrsVuKoPdVxI66rd48CHyryrgYvK9HHA3wBjwH8AZwxg33Sq6S+AbWW/3A+8to+1fA7YDfyI1m9jVwHvBd5blgu4vtS6FVg2gP3TqaY1bfvnQeCX+1jLrwAGvgFsKZ8LZ3IfdVnTwPbRkfDJLYEREZVhOpWOiBgKCcaIiEqCMSKikmCMiKgkGCMiKgnGiIhKgjEiovJ/9bJsN20e84wAAAAASUVORK5CYII=\n",
85 | "text/plain": [
86 | ""
87 | ]
88 | },
89 | "metadata": {},
90 | "output_type": "display_data"
91 | }
92 | ],
93 | "source": [
94 | "def covariance(data, data2=None, ddof=0):\n",
95 | " k = data.shape[0]\n",
96 | " N = data.shape[1]\n",
97 | " \n",
98 | " if data2 is not None:\n",
99 | " k += data2.shape[0]\n",
100 | " data = np.vstack((data, data2))\n",
101 | " \n",
102 | " cov = np.zeros((k, k))\n",
103 | " for i in range(k):\n",
104 | " for j in range(k):\n",
105 | " a = data[i]\n",
106 | " b = data[j]\n",
107 | " cov[i, j] = np.sum((a-np.mean(a)) * (b-np.mean(b)))/(N-ddof)\n",
108 | " return np.array(cov)\n",
109 | "\n",
110 | "\n",
111 | "data = np.array([[4.0, 4.2, 3.9, 4.3, 4.1], \n",
112 | " [2.0, 2.1, 2.0, 2.1, 2.2], \n",
113 | " [0.6, 0.59, 0.58, 0.62, 0.63]])\n",
114 | "\n",
115 | "print('[Numpy Covariance]')\n",
116 | "print(np.cov(data))\n",
117 | "print()\n",
118 | "print('[My Covariance]')\n",
119 | "print(covariance(data, ddof=1))\n",
120 | "\n",
121 | "cax = imshow(np.cov(data), interpolation='nearest', cmap=cm.get_cmap('jet', 30))\n",
122 | "colorbar(cax)"
123 | ]
124 | },
125 | {
126 | "cell_type": "markdown",
127 | "metadata": {},
128 | "source": [
129 | "아래는 2개의 matrix에 대해서 covariance matrix 를 구하는 코드입니다."
130 | ]
131 | },
132 | {
133 | "cell_type": "code",
134 | "execution_count": 3,
135 | "metadata": {},
136 | "outputs": [
137 | {
138 | "name": "stdout",
139 | "output_type": "stream",
140 | "text": [
141 | "[Numpy Covariance]\n",
142 | "[[ 1. 1.5 -2.5 3. ]\n",
143 | " [ 1.5 6.33333333 -3.16666667 6.83333333]\n",
144 | " [-2.5 -3.16666667 6.33333333 -7.16666667]\n",
145 | " [ 3. 6.83333333 -7.16666667 10.33333333]]\n",
146 | "\n",
147 | "[My Covariance]\n",
148 | "[[ 1. 1.5 -2.5 3. ]\n",
149 | " [ 1.5 6.33333333 -3.16666667 6.83333333]\n",
150 | " [-2.5 -3.16666667 6.33333333 -7.16666667]\n",
151 | " [ 3. 6.83333333 -7.16666667 10.33333333]]\n"
152 | ]
153 | },
154 | {
155 | "data": {
156 | "text/plain": [
157 | ""
158 | ]
159 | },
160 | "execution_count": 3,
161 | "metadata": {},
162 | "output_type": "execute_result"
163 | },
164 | {
165 | "data": {
166 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAATgAAAD8CAYAAADjcbh8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFFxJREFUeJzt3XusXVWBx/Hvz9ICAsqjFTptoRgqDsEHeqk4RKfyMIUhYDLogJEBB3MNEQUHg6AJKJlk8BEceURyBxhhJDwUhA5hxCowSEYql055tAWt+GhrsQXkUYHWq7/54+yrl9P7OLd733PO3ff3SU7ufqyz1jqAP/fea++9ZJuIiDp6Tac7EBExURJwEVFbCbiIqK0EXETUVgIuImorARcRtVUq4CTtKWmppJ8Vf/cYodwfJa0oPkvKtBkR0SqVuQ9O0peBZ21fLOk8YA/bnx2m3Gbbu5boZ0TEuJUNuCeARbY3SJoN3Gv7wGHKJeAiou3KBtxztncvlgX8bnC9qdwAsAIYAC62fdsI9fUCvQDTdtnxnbu9+a+2u2/dagZbO92FCbNx0z6d7sKEOHDW6k53YcI88dBLT9ueVaaOAyS/1GLZDXCX7cVl2huPHcYqIOkHwHD/5X5+6IptSxopLfezvV7SG4G7JT1q++fNhWz3AX0Ae/S80Yv6/2XMHzDZzGNdp7swYS7rO7fTXZgQ/957aKe7MGHeq/5fla3jJeDjLZb9Aswcbb+ka4DjgI22Dy627QncBMwHfgl8yPbvWmlvzEEG20fZPniYz+3Ab4tTU4q/G0eoY33x90ngXuCQVjoXEVPON4HmI7zzgB/aXgD8sFhvSdnbRJYApxbLpwK3NxeQtIekHYvlmcDhwKqS7UZEDdm+D3i2afMJwLXF8rXAB1qtr2zAXQwcLelnwFHFOpJ6JF1VlPlroF/Sw8A9NK7BJeAiolV7295QLD8F7N3qF8e8Bjca288ARw6zvR/4WLH8v8BbyrQTEbUxU1L/kPW+4tp7S8a41r+NUgEXETFOT9vuGed3fitp9pDb0Ya91j+cPKoVEd1uzGv9I8kRXESUshOwzd3920nSDcAiGqey64ALaVzbv1nS6cCvgA+1Wl8CLiK6hu2TR9i1zbX+VuQUNSJqKwEXEbWVgIuI2krARURtJeAiorYyihoRpewEvLnTnRhBjuAiorYScBFRWwm4iKitBFxE1FYCLiJqKwEXEbWVgIuI2sp9cBFRys6vgTfv0mLhFye0K9uo5AhO0mJJT0haU8xw37x/R0k3FfuXSZpfRbsREaMpHXCSpgFXAMcABwEnSzqoqdjpNCaFPgD4GvClsu1GRIyliiO4hcAa20/a3grcSGOar6GGTvv1HeBISaqg7YioEUmflrRS0mOSbpC0U5n6qgi4OcDaIevrim3DlrE9ADwP7FVB2xFRE5LmAJ8CeopZ7acBJ5Wps6tGUSX1SuqX1L9lU5uvRkZEN9gB2FnSDsBrgd+UqayKgFsPzBuyPrfYNmyZouOvB55prsh2n+0e2z07ztqtgq5FRJeZOXgQU3x6B3fYXg98Ffg1sAF43vb3yzRWxW0iDwILJO1PI8hOAj7cVGZw2q8fAycCd9tuefLWiOhe2hF2mt9i4UdHnhdV0h40rtfvDzwHfFvSR2x/a3v7VvoIrrimdiZwF7AauNn2SkkXSTq+KHY1sJekNcA/A9vcShIRU95RwC9sb7L9B+BW4G/KVFjJjb627wTubNp2wZDlV4APVtFWRNTWr4HDJL0WeJnGVIH9ZSrsqkGGiJi6bC+jcRvZcuBRGvnUV6bOPKoVEV3D9oU0ZrOvRI7gIqK2EnARUVs5RY2IcmbQuLGjFY9OZEe2lSO4iKitBFxE1FYCLiJqKwEXEbWVgIuI2krARURt5TaRiChnR2B+pzsxvBzBRURtJeAiorYScBFRWwm4iKitBFxE1FYCLiJqKwEXEV1D0u6SviPpcUmrJb27TH2V3AcnaTHwdRoTtV5l++Km/acBX+Ev0wlebvuqKtqOiA4bz+uSxvZ14Hu2T5Q0g8bcqNutdMBJmgZcARxNY1b7ByUtsb2qqehNts8s215E1JOk1wPvBU4DsL0V2FqmziqO4BYCa2w/CSDpRhpzGzYH3LjMYCvzWFdB97rLpWd9ttNdmDCXrT+3012YEO+5o9TETvFqMyUN/QfaZ3twYpn9gU3Af0h6G/AQcJbt329vY1Vcg5sDrB2yvq7Y1uzvJT1SnF/PG64iSb2DM16/vGm7f1NEdK+nbfcM+QydNWsH4B3AN2wfAvyeknMot2uQ4b+A+bbfCiwFrh2ukO2+wR++86xd2tS1iOgS64B1xfSB0JhC8B1lKqwi4NYDQ4/I5vKXwQQAbD9je0uxehXwzgrajYgasf0UsFbSgcWmIyl5qauKa3APAgsk7U8j2E4CPjy0gKTZtjcUq8cDqytoNyLq55PA9cUI6pPAR8tUVjrgbA9IOhO4i8ZtItfYXinpIqDf9hLgU5KOBwaAZylGSSKiBnaksttEbK8AeqqpraL74GzfCdzZtO2CIcvnA+dX0VZERKvyJENE1FYCLiJqKwEXEbWVgIuI2krARURtZVatiCgns2pFRLRfAi4iaisBFxG1lYCLiNpKwEVEbSXgIqK2cptIRJTiGTBQ3aQzlcoRXETUVgIuImorARcRXUXSNEn/J+mOsnUl4CKi25xFRdMaJOAiomtImgv8HY3JqUqrJOAkXSNpo6THRtgvSZdKWlPMjVpqKrCImLRmDs59XHx6m/b/G3Au8KcqGqvqNpFvApcD142w/xhgQfF5F/CN4m9ETC1P2x52UhlJxwEbbT8kaVEVjVU16cx9kuaPUuQE4DrbBh6QtHvTVIIRMUn9YdoOrHvdzBZLPzXazsOB4yUdC+wEvE7St2x/ZHv71q5rcHOAtUPW1xXbXkVS7+Ch68ubft+mrkVEN7B9vu25tufTmF/57jLhBl02yGC7z3aP7Z6dZ+3S6e5ExCTXrke11gPzhqzPLbZFRGzD9r3AvWXradcR3BLgH4vR1MOA53P9LSImWiVHcJJuABbRGAJeB1wITAewfSWNWe+PBdYALwEfraLdiIjRVDWKevIY+w18ooq2IiJaldclRUQpW5nBWua2WHrU20Qq11WjqBERVUrARURtJeAiorYScBFRWwm4iKitBFxE1FZuE4mIUrYyg3WvehJzNP0T2pdmOYKLiNpKwEVEbSXgIqK2EnARUVsJuIiorQRcRHQFSfMk3SNplaSVks4qW2duE4mIUhq3ibT6NpFRDQDn2F4uaTfgIUlLba/a3gpzBBcRXcH2BtvLi+UXacxuv83kVOORI7iIaKeZkobe7dtnu6+5UDEN6SHAsjKNJeAiop1GnPh5kKRdgVuAs22/UKaxSk5RJV0jaaOkx0bYv0jS85JWFJ8Lqmg3IupF0nQa4Xa97VvL1lfVEdw3gcuB60Yp8yPbx1XUXkTUjCQBVwOrbV9SRZ2VHMHZvg94toq6ImLKOhw4BThiyNnesWUqbOc1uHdLehj4DfAZ2yubC0jqBXoB2HNfLus7t43da4/L1tfvNw3yHHW6CxNCl17Y6S5MoC92ugN/Zvt+oNL/iNoVcMuB/WxvLhL5NmBBc6FiNKUPQPv1uE19i4gS/sB01rb8uqT2ast9cLZfsL25WL4TmC5pZjvajoipqy0BJ2mf4gIikhYW7T7TjrYjYuqq5BRV0g3AIho38a0DLgSmA9i+EjgROEPSAPAycFIx231ExISpJOBsnzzG/stp3EYSEdE2eRY1ImorARcRtZVnUSOilApfl1S5HMFFRG0l4CKithJwEVFbCbiIqK0EXETUVgIuImort4lERClbmcHa3CYSEdFeCbiI6BqSFkt6QtIaSeeVrS8BFxFdQdI04ArgGOAg4GRJB5WpMwEXEd1iIbDG9pO2twI3AieUqTABFxHtNFNS/5BP75B9c4C1Q9bXkZntI2ISGXPi5yol4CKilMbbRCqZdGY9vKqiucW27ZZT1IjoFg8CCyTtL2kGcBKwpEyFpQNO0jxJ90haJWmlpLOGKSNJlxZDv49IekfZdiOiXmwPAGcCdwGrgZuHmz95PKo4RR0AzrG9XNJuwEOSltpeNaTMMTTmQV0AvAv4RvE3IuLPimlF76yqvtJHcLY32F5eLL9II3mbRz5OAK5zwwPA7pJml207ImI0lV6DkzQfOARY1rSrpeFfSb2Dw8ds3lRl1yJiCqos4CTtCtwCnG37he2pw3af7R7bPew6q6quRcQUVUnASZpOI9yut33rMEUqH/6NiBhL6UEGSQKuBlbbvmSEYkuAMyXdSGNw4XnbG8q2HRGdN7B1Ok/9qjtfl1TFKOrhwCnAo5JWFNs+B+wLYPtKGqMixwJrgJeAj1bQbkTEqEoHnO37AY1RxsAnyrYVETEeeZIhImorARcRtZWAi4jaSsBFRG3ldUkRUc4Wwc+7M0pyBBcRtZWAi4jaSsBFRG0l4CJiUpD0FUmPFy/N/a6k3cf6TgIuIiaLpcDBtt8K/BQ4f6wvJOAiYlKw/f3iteYAD9B4K9GounNsNyImjy3Az9ve6j8BN41VKAEXEe00U1L/kPU+232DK5J+AOwzzPc+b/v2osznacwFc/1YjSXgIqKdRp342fZRo31Z0mnAccCRxVuKRpWAi4hJQdJi4Fzgb22/1Mp3MsgQEZPF5cBuwFJJKyRdOdYXcgQXEZOC7QPG+50cwUVEbVUx6cw84Dpgb8A0RkW+3lRmEXA78Iti0622LyrbdkR0gc7cJtKSKk5RB4BzbC+XtBvwkKSltlc1lfuR7eMqaC8ioiWlT1Ftb7C9vFh+EVjNMLPWR0S0W6WDDJLmA4cAy4bZ/W5JDwO/AT5je+Uw3+8FegH23ncG3+49tMrudYX33NE/dqFJSpde2OkuTIgL+WKnuzBh6vvLGiobZJC0K43Z7c+2/ULT7uXAfrbfBlwG3DZcHbb7bPfY7tl9VgZ4I6KcSgJO0nQa4Xa97Vub99t+wfbmYvlOYLqkmVW0HRExktIBJ0nA1cBq25eMUGafohySFhbtPlO27YiI0VRxHng4cArwqKQVxbbPAfsC2L4SOBE4Q9IA8DJwUivPkUVElFE64GzfD2iMMpfTeMwiIuqmi++Dy5MMEVFbCbiIqK0EXETUVgIuImorARcRtZWAi4jayvNQEVHOK8Ca9jUn6Rzgq8As20+PVjZHcBExaRTvn3w/8OtWyifgImIy+RqNiWdaehIqp6gR0U6jzos6GkknAOttP1w82j6mBFxEtNOo86KONvEzjWfc3z+exhJwEdE1Rpr4WdJbgP2BwaO3ucBySQttPzVSfQm4iOh6th8F3jC4LumXQM9Yo6gJuIgop4vfJpKAi4hJx/b8VsrlNpGIqK0EXETUVgIuImqriklndpL0E0kPS1opaZupFiXtKOkmSWskLSvmT42ImFBVHMFtAY4o5jx9O7BY0mFNZU4Hfmf7ABqPWnypgnYjIkZVxaQzBjYXq9OLT/NzYicAXyiWvwNcLkmZWSuiBv5kePGVTvdiWFVN/DytmDJwI7DU9rKmInOAtQC2B4Dngb2qaDsiYiSVBJztP9p+O43HJxZKOnh76pHUK6lfUv9zmwaq6FpETGGVjqLafg64B1jctGs9MA9A0g7A6xlmZnvbfbZ7bPfsPiv3IEdEOVWMos6StHuxvDNwNPB4U7ElwKnF8onA3bn+FhETrYrDpNnAtZKm0QjMm23fIekioN/2EuBq4D8lrQGeBU6qoN2IiFFVMYr6CHDIMNsvGLL8CvDBsm1FRIxHnmSIiNrKlfyIKOlltr3s3h1yBBcRtZWAi4jaSsBFxKQh6ZOSHi9e7PHlscrnGlxETAqS3kfjufa32d4i6Q1jfSdHcBExWZwBXGx7C4DtjWN9IQEXEe00c/B58+LTO47vvgl4T/FOyf+RdOhYX8gpakSU9ArjuE2kzMTPOwB7AocBhwI3S3rjaI99JuAiomuMNPEzgKQzgFuLQPuJpD8BM4FNI30np6gRMVncBrwPQNKbgBlAJn6OiFq4BrhG0mPAVuDUsd5KlICLiEnB9lbgI+P5Tk5RI6K2EnARUVs5RY2Ikl4Bnuh0J4aVI7iIqK0EXETUVgIuImqrilm1dpL0E0kPF68w+eIwZU6TtEnSiuLzsbLtRkSMpYpBhi3AEbY3S5oO3C/pv20/0FTuJttnVtBeRERLqphVy8DmYnV68cmcpxHRcapi/uViTtSHgAOAK2x/tmn/acC/0ngo9qfAp22vHaaeXmDw9SkH0t6x55mM8VzbJJXfNfm087ftZ3tWmQokfY9Gn1vxtO3FZdobj0oC7s+VNWa4/y7wSduPDdm+F7C5eAvnx4F/sH1EZQ1XQFL/aK9xmazyuyafOv+2dqt0FNX2c8A9wOKm7c8MvoUTuAp4Z5XtRkQMp4pR1FnFkRuSdgaOpuntd5JmD1k9Hlhdtt2IiLFUMYo6G7i2uA73GuBm23dIugjot70E+JSk44EB4FngtArarVpfpzswQfK7Jp86/7a2qvQaXEREN8mTDBFRWwm4iKitKR9wkhZLekLSGknndbo/VZF0jaSNxeuda0PSPEn3SFpVPBp4Vqf7VIVWHnmM8ZvS1+CKgZGf0hj5XQc8CJxse1VHO1YBSe+l8YTJdbYP7nR/qlKMyM+2vVzSbjRuMP/AZP93JknALkMfeQTOGuaRxxiHqX4EtxBYY/vJ4n3vNwIndLhPlbB9H40R61qxvcH28mL5RRq3HM3pbK/Kc0MeeazYVA+4OcDQR8bWUYP/sUwVkuYDhwDLOtuTakiaJmkFsBFYarsWv6uTpnrAxSQlaVfgFuBs2y90uj9VsP1H228H5gILJdXm0kKnTPWAWw/MG7I+t9gWXay4RnULcL3tWzvdn6qN9MhjjN9UD7gHgQWS9pc0AzgJWNLhPsUoiovxVwOrbV/S6f5UpZVHHmP8pnTA2R4AzgTuonGx+mbbKzvbq2pIugH4MXCgpHWSTu90nypyOHAKcMSQN0Qf2+lOVWA2cI+kR2j8H+9S23d0uE+T3pS+TSQi6m1KH8FFRL0l4CKithJwEVFbCbiIqK0EXETUVgIuImorARcRtfX/YwhWgukbA0QAAAAASUVORK5CYII=\n",
167 | "text/plain": [
168 | ""
169 | ]
170 | },
171 | "metadata": {},
172 | "output_type": "display_data"
173 | }
174 | ],
175 | "source": [
176 | "a = np.array([[1,2,3], [-1, 4, 2]])\n",
177 | "b = np.array([[3, 1, -2], [-1, 4, 5]])\n",
178 | "\n",
179 | "print('[Numpy Covariance]')\n",
180 | "print(np.cov(a, b))\n",
181 | "print()\n",
182 | "print('[My Covariance]')\n",
183 | "print(covariance(a, b, ddof=1))\n",
184 | "\n",
185 | "cax = imshow(np.cov(a, b), interpolation='nearest', cmap=cm.get_cmap('jet', 30))\n",
186 | "colorbar(cax)"
187 | ]
188 | },
189 | {
190 | "cell_type": "markdown",
191 | "metadata": {},
192 | "source": [
193 | "남자와 여자의 키의 분포를 갖고서 covariance를 구합니다."
194 | ]
195 | },
196 | {
197 | "cell_type": "code",
198 | "execution_count": 4,
199 | "metadata": {},
200 | "outputs": [
201 | {
202 | "name": "stdout",
203 | "output_type": "stream",
204 | "text": [
205 | "[Numpy Covariance]\n",
206 | "[[ 8.19884325 -0.25129107]\n",
207 | " [-0.25129107 7.26994749]]\n",
208 | "\n",
209 | "[My Covariance]\n",
210 | "[[ 8.19884325 -0.25129107]\n",
211 | " [-0.25129107 7.26994749]]\n"
212 | ]
213 | },
214 | {
215 | "data": {
216 | "text/plain": [
217 | ""
218 | ]
219 | },
220 | "execution_count": 4,
221 | "metadata": {},
222 | "output_type": "execute_result"
223 | },
224 | {
225 | "data": {
226 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAATYAAAD8CAYAAAD9uIjPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFnVJREFUeJzt3X+wX3V95/Hny/wUf0AwLYaEhbhmVNQabDbq4ijyy0idxJnaNmytwYGJ65Za7Xa3MM6A4joT7c7qdsTKHUgBtYClUlM3iJEfYzs2NFcMv4mEWJvEYIAIQgPBG177xznXfrl8v/d+b77n/vh+8nrMnLnnfM77nO/nkOE9n3M+n3M+sk1EREleNNUViIhoWhJbRBQniS0iipPEFhHFSWKLiOIksUVEcXpKbJKOlrRJ0oP133kd4g5K2lovG1rKF0u6XdJ2SddJmt1LfSIioPcW2wXAzbaXADfX2+08bXtpvaxsKf8s8HnbrwZ+DpzbY30iIlAvA3QlbQNOsb1H0gLgNtuvaRP3lO2XjigT8AjwSttDkt4GfNL2uw+5QhERwMwejz/G9p56/WHgmA5xcyUNAkPAOtt/B7wCeNz2UB2zC1jY6YckrQXWAsyC35zfY8Vjcu1hwVRXIcblcez96uUMr5a8v8vYPXCT7RW9/F6rMRObpO8Cr2yz6xOtG7YtqVPz73jbuyW9CrhF0t3AE+OpqO0BYADgWMlrx3NwTLlPkX+x/jLQ8xn2Ax/uMvaT0GhbZczEZvv0Tvsk/UzSgpZb0b0dzrG7/rtD0m3AScDfAkdJmlm32hYBuw/hGiLiMCDp48B5gIG7gQ/ZfqZdbK+dBxuANfX6GuCbbSozT9Kcen0+cDJwn6uHe7cC7x/t+IgISQuBjwLLbL8BmAGs7hTfa2JbB5wh6UHg9HobScskXV7HvA4YlHQnVSJbZ/u+et+fAX8iaTvVM7creqxPRJRrJvBiSTOBI4CfjhZ4yGw/BpzWpnyQqsmI7e8Db+xw/A5geS91iIgizK87GIcN1M/VgepxlqT/Dfwr8DTwHdvf6XSyXntFIyKa8KjtZZ121oP/VwGLgceBv5H0AdtfbRefV6oioh+cDvzY9iO2fwl8A/jPnYLTYouICTEXeMFo/UP3r8BbJR1BdSt6GjDYKTgttoiY9mzfDlwP3EE11ONFjDLYLi22iOgLti8GLu4mNi22iChOEltEFCeJLSKKk8QWEcVJ50FETIi5wGun6LfTYouI4iSxRURxktgiojhJbBFRnCS2iChOEltEFCeJLSKKk3FsETEhXvwieO1Lugx+stnf7qnFJuloSZskPVj/ndcmZqmkf5J0r6S7JP1ey74rJf1Y0tZ6WdpLfSIioPdb0QuAm20vAW6ut0faD3zQ9uuBFcAXJB3Vsv9/2F5aL1t7rE9ERM+JbRVwVb1+FfC+kQG2f2T7wXr9p1Rzj/5aj78bEdFRr4ntGNt76vWHgWNGC5a0HJgNPNRS/Jn6FvXzw/OPRkT0YszEJum7ku5ps6xqjasnQPYo51kAfIVq9ubn6uILqd6T/U/A0VTzjHY6fq2kQUmD+8e+rogoiKTXtDyL3yrpF5I+1il+zF5R26eP8mM/k7TA9p46ce3tEPdy4P8Bn7C9ueXcw629A5L+CvjTUeoxQP2N82Oljgk0IspjexuwFEDSDGA3cEOn+F6He2wA1lDNAL8G+ObIAEmz6wpcbfv6EfuGk6Kons/d02N9ImKa0ByYe0KXwXeP69SnAQ/Z/kmngF6fsa0DzpD0INW8f+sAJC2TdHkd87vAO4Bz2gzr+Jqku6kuaz7wv3qsT0T0p/nDj5rqZe0osauBa0Y7WU8tNtuPUWXPkeWDwHn1+leBtrM12z61l9+PiGKMOhP8sPoOcCXV8/mO8kpVRPST9wB32P7ZaEFJbBHRT85mjNtQSGKLiD4h6SXAGcA3xorNS/AR0Rds/xvwim5ik9giYmLMBhZ3GTu+4R5jyq1oRBQniS0iipPEFhHFSWKLiOIksUVEcZLYIqI4Ge4RERNjDnDC1Px0WmwRUZwktogoThJbRBQniS0iipPEFhHFSWKLiOIksUVEcRpJbJJWSNomabukC9rsnyPpunr/7ZJOaNl3YV2+TdK7m6hPREwDw58t6mZpWM+JrZ7j71Kqb5GfCJwt6cQRYecCP7f9auDzwGfrY0+kmnHm9cAK4Ev1+SIiDlkTLbblwHbbO2w/C1wLrBoRswq4ql6/Hjitnkt0FXCt7QO2fwxsr88XEfE8ko6SdL2kByTdL+ltnWKbSGwLgZ0t27vqsrYxtoeAJ6g+8dvNsQBIWjs85+D+BiodEX3n/wLftv1a4E3A/Z0C++ZdUdsDwADAsZKnuDoRMYkkHUk98TpAfXf4bKf4Jlpsu4HjWrYX1WVtYyTNBI4EHuvy2Igo31gzwS8GHgH+StIPJV1ez1rVVhOJbQuwRNLiepbm1cCGETEbgDX1+vuBW2y7Ll9d95ouBpYA/9xAnSKivzxqe1nLMjBi/0zgzcBf2j4J+DfgBSMwWoN7YntI0vnATcAMYL3teyVdAgza3gBcAXxF0nZgH1Xyo477OnAfMAT8oe2DvdYpIqaBOTQ5lGMXsMv27fX29UxkYgOwvRHYOKLsopb1Z4Df6XDsZ4DPNFGPiCiT7Ycl7ZT0GtvbgNOoGkRt9U3nQUQc9v4I+Fr9yGsH8KFOgUlsEdEXbG8FlnUTm3dFI6I4SWwRUZwktogoTp6xRcTEyCxVERHNSWKLiOIksUVEcZLYIqI4SWwRUZwktogoToZ7RMSE8GwYmoCJWrqRFltEFCeJLSKKk8QWEcVJYouI4iSxRURxGklsklZI2iZpu6QXfIdc0p9Iuk/SXZJulnR8y76DkrbWy8hJYCIixq3n4R6SZgCXAmdQTbiwRdIG263fI/8hsMz2fkkfAT4H/F6972nbS3utR0SUTdK/AE8CB4Eh2x2/ptvEOLblwHbbO+ofvxZYRctEC7ZvbYnfDHyggd+NiGnslzNmsuvl87uMfrjb077L9qNjBTVxK7oQ2Nmyvasu6+Rc4MaW7bn1BKmbJb2v00GS1g5Pprq/t/pGROEm9c0DSR+gmozhnS3Fx9veLelVwC2S7rb90Mhj6wlUBwCOlTwpFY6IyTJf0mDL9kCbSZMNfEfV//+Xtdn/K00ktt3AcS3bi+qy55F0OvAJ4J22D/yqpvbu+u8OSbcBJwEvSGwRUbRHR3tmVnt73Qj6dWCTpAdsf69dYBO3oluAJZIW1/P9rQae17sp6STgMmCl7b0t5fMkzanX5wMnM8okqBFx+GppBO0FbqB6vt9Wz4nN9hBwPnATcD/wddv3SrpE0so67M+BlwJ/M2JYx+uAQUl3ArcC60b0pkZEIOklkl42vA6cCdzTKb6RZ2y2NwIbR5Rd1LJ+eofjvg+8sYk6RETRjgFukARV3vpr29/uFJzPFkXEhHiW2exkUZfRow/3qIeTvanb384rVRFRnCS2iChOEltEFCeJLSKKk8QWEcVJYouI4mS4R0RMiGeZza7nvW05msGxQ8YhLbaIKE4SW0QUJ4ktIoqTxBYRxUlii4jiJLFFRHEy3CMiJkQ13KPbr3s0Ky22iChOEltEFCeJLSL6gqQZkn4o6VtjxTaS2CStkLRN0nZJF7TZf46kR+r5DrZKOq9l3xpJD9bLmibqExFF+mOqeVXG1HNikzQDuBR4D3AicLakE9uEXmd7ab1cXh97NHAx8BaqGWculjSv1zpFRFkkLQJ+C7i8m/gmWmzLge22d9h+FrgWWNXlse8GNtneZ/vnwCZgRQN1ioj+Ml/SYMuydsT+LwD/E3ium5M1MdxjIbCzZXsXVQtspN+W9A7gR8DHbe/scOzCdj9SX2h9sUfyKT7Wc8Vj8jz90k9NdRViHE7eP+k/2XHCZEnvBfba/oGkU7o52WSNY/t74BrbByR9GLgKOHU8J6insx8AkI5181WMiCb9klns7PqzRaM6GVgp6SxgLvBySV+1/YFOBzRxK7obnlf7RXXZr9h+zPaBevNy4De7PTYiDm+2L7S9yPYJwGrgltGSGjST2LYASyQtljS7/uENrQGSFrRsruTfezZuAs6UNK/uNDizLouIOGQ934raHpJ0PlVCmgGst32vpEuAQdsbgI9KWgkMAfuAc+pj90n6NFVyBLjE9r5e6xQRZbJ9G3DbWHGNPGOzvRHYOKLsopb1C4ELOxy7HljfRD0iIiBvHkREgZLYIqI4+WxRREyIfLYoIqJBSWwRUZwktogoThJbRBQniS0iipPEFhHFyXCPiJgQzzKbnRnuERHRjCS2iChOEltEFCeJLSKKk8QWEcVJYouI4mS4R0RMiOrrHo1M5oKkucD3gDlUeet62xd3ik9ii4h+cAA41fZTkmYB/yjpRtub2wU3cisqaYWkbZK2S7qgzf7PS9paLz+S9HjLvoMt+zaMPDYiwpWn6s1Z9dJxGs6eW2ySZgCXAmdQTXi8RdIG2/e1VOrjLfF/BJzUcoqnbS/ttR4R0dfmSxps2R6o5xL+lTrX/AB4NXCp7ds7nayJW9HlwHbbO+ofvxZYBdzXIf5soOO9cUQcljrOBD/M9kFgqaSjgBskvcH2Pe1im7gVXQjsbNneVZe9gKTjgcXALS3FcyUNStos6X2dfkTS2jpuEPY3UO2I6Ee2HwduBVZ0ipns4R6rqXozDraUHV9n6v8CfEHSf2x3oO0B28uq2CMmo64RMU1I+rW6pYakF1M9+nqgU3wTiW03PK9Pd1Fd1s5q4JrWAtu76787qCZCPemFh0XEYW4BcKuku6gmWN9k+1udgpt4xrYFWCJpMVVCW03V+noeSa8F5gH/1FI2D9hv+4Ck+cDJwOcaqFNETLGhZ2fx8E+a+WyR7bsYR6On58Rme0jS+cBNwAxgve17JV0CDNoeHsKxGrjWdmsX7euAyyQ9R9V6XNfamxoRcSgaGaBreyOwcUTZRSO2P9nmuO8Db2yiDhERw/KuaEQUJ4ktIoqTxBYRxUlii4ji5OseETExDggempoUkxZbRBQniS0iipPEFhHFSWKLiOIksUVEcZLYIqI4Ge4RERPjAPDQ1Px0WmwRUZwktogoThJbRBQniS0ipj1Jx0m6VdJ9ku6V9MejxafzICL6wRDw323fIellwA8kber0xe202CJi2rO9x/Yd9fqTwP10mOYTGkpsktZL2iup7eSlqvyFpO2S7pL05pZ9ayQ9WC9rmqhPREwDw8M9ulnqmeBblrWdTivpBKqJXSZ0JniAK4EvAld32P8eYEm9vAX4S+Atko6mmhV+GWCq5uUG2z9vqF4R0R/GnAkeQNJLgb8FPmb7F53iGmmx2f4esG+UkFXA1a5sBo6StAB4N9X8gPvqZLaJUWZ3jojDl6RZVEnta7a/MVrsZHUeLAR2tmzvqss6lb9A3TStm6dHTkQdI2KakiTgCuB+2/9nrPi+6TywPWB7WdVcPWKqqxMRk+tk4A+AUyVtrZezOgVPVottN3Bcy/aiumw3cMqI8tsmqU4R0Sds/yOgbuMnq8W2Afhg3Tv6VuAJ23uoZo8/U9I8SfOAM+uyiIhD1kiLTdI1VC2v+ZJ2UfV0zgKw/WWqWeLPArYD+4EP1fv2Sfo0sKU+1SW2R+uEiIgYUyOJzfbZY+w38Icd9q0H1jdRj4iYRvLZooiI5iSxRURxktgiojhJbBFRnCS2iChOEltEFCcfmoyIifEM1cjVKZAWW0QUJ4ktIoqTxBYRxUlii4jiJLFFRHGS2CKiOBnuERETo8Gve0haD7wX2Gv7DWPFp8UWEf3gSsYx0VMSW0RMe13MhPc8SWwRUZw8Y4uI6WC+pMGW7QHbA4d6sqbmPBj1wZ6k3wf+jGqWmSeBj9i+s973L3XZQWCom9mgI6I4Xc0E362mbkWvZPQHez8G3mn7jcCngZGZ+F22lyapRUQTmprM5XuSThhl//dbNjdTzR8aESV7zvDkM42cqt1MeLav6BQ/Fc/YzgVubNk28B1JBi7r5b46Iso01kx4I01qYpP0LqrE9vaW4rfb3i3p14FNkh6ou3ZHHrsWWFttHTkJtY2IfjVpwz0k/QZwObDK9mPD5bZ313/3AjcAy9sdb3vA9rLqOdwRk1HliOhTk5LYJP0H4BvAH9j+UUv5SyS9bHgdOBO4ZzLqFBHlamq4xwse7AGzAGx/GbgIeAXwJUnw78M6jgFuqMtmAn9t+9tN1CkiDl9N9YqO+mDP9nnAeW3KdwBvaqIOERHD8kpVRBQnr1RFxAR5GnhgSn45LbaIKE4SW0QUJ4ktIoqTxBYRxUlii4jiJLFFRHEy3CMiJsgzZLhHRERDktgiojhJbBFRnCS2iOgLklZI2iZpu6QLRotNYouIaU/SDOBS4D3AicDZkk7sFJ/EFhH9YDmw3fYO288C1wKrOgVnuEdETJBngG3dBo81YfJCYGfL9i7gLZ1OlsQWEdPBtJwwOSJiIu0GjmvZXlSXtZXEFhH9YAuwRNJiSbOB1cCGTsGNJDZJ6yXtldR2hilJp0h6QtLWermoZV/XXbgRcXiyPQScD9wE3A983fa9neKbesZ2JfBF4OpRYv7B9ntbC1q6cM+gehi4RdIG2/c1VK+IKITtjcDGbmIbabHVM7fvO4RDx9WFGxHRjcnsFX2bpDuBnwJ/Wjcju+7ClbQWWFtvHoBPlTix8nzg0amuxER48VPFXlup1/Wa3k+x5yb45Pwugxv9bzhZie0O4HjbT0k6C/g7YMl4TlCPaRkAkDTYZNfwdFHqdUG511bydfV6DtsrmqjLoZiUXlHbv7D9VL2+EZglaT7j7MKNiOjGpCQ2Sa+UpHp9ef27jzHOLtyIiG40cisq6RrgFKrXInYBFwOzAGx/GXg/8BFJQ1SzqK62bWBI0nAX7gxg/WhduC0Gxg7pS6VeF5R7bbmuaUhVfomIKEfePIiI4iSxRURx+iKxSTpa0iZJD9Z/53WIO9jy2ta07YQY6zUySXMkXVfvv13SCZNfy/Hr4rrOkfRIy7/ReVNRz/Hq4pVBSfqL+rrvkvTmya7joejlVchpz/a0X4DPARfU6xcAn+0Q99RU17WLa5kBPAS8CpgN3AmcOCLmvwFfrtdXA9dNdb0buq5zgC9OdV0P4dreAbwZuKfD/rOAGwEBbwVun+o6N3RdpwDfmup6HsrSFy02qtesrqrXrwLeN4V16VU3r5G1Xu/1wGnDw2WmsWJfj/PYrwyuAq52ZTNwlKQFk1O7Q9fFdfWtfklsx9jeU68/DBzTIW6upEFJmyVN1+TX7jWyhZ1iXH3V4AngFZNSu0PXzXUB/HZ9u3a9pOPa7O9H3V57P3qbpDsl3Sjp9VNdmW5Nmy/oSvou8Mo2uz7RumHbkjqNUTne9m5JrwJukXS37Yearmscsr8HrrF9QNKHqVqlp05xnaKznl+FnCrTJrHZPr3TPkk/k7TA9p66ib+3wzl21393SLoNOInquc900s1rZMMxuyTNBI6kelNjOhvzumy3XsPlVM9OS1Dkq4G2f9GyvlHSlyTNtz3tX/rvl1vRDcCaen0N8M2RAZLmSZpTr88HTgam43fdunmNrPV63w/c4vpp7jQ25nWNeO60kuqDgSXYAHyw7h19K/BEy6OTvjXKq5DT3rRpsY1hHfB1SecCPwF+F0DSMuC/2j4PeB1wmaTnqP4B1nkafrDSdtvXyCRdAgza3gBcAXxF0naqh7urp67G3enyuj4qaSUwRHVd50xZhcehi1cGN1L1jG4H9gMfmpqajk8Pr0JOe3mlKiKK0y+3ohERXUtii4jiJLFFRHGS2CKiOElsEVGcJLaIKE4SW0QU5/8D4+O/p76f/6IAAAAASUVORK5CYII=\n",
227 | "text/plain": [
228 | ""
229 | ]
230 | },
231 | "metadata": {},
232 | "output_type": "display_data"
233 | }
234 | ],
235 | "source": [
236 | "data = pd.read_csv('dataset/gender-height-weight.csv', usecols=(0, 1, 2))\n",
237 | "x = np.array([data[data['Gender'] == 'Male']['Height'].as_matrix()])\n",
238 | "y = np.array([data[data['Gender'] == 'Female']['Height'].as_matrix()])\n",
239 | "\n",
240 | "print('[Numpy Covariance]')\n",
241 | "print(np.cov(x, y))\n",
242 | "print()\n",
243 | "print('[My Covariance]')\n",
244 | "print(covariance(x, y, ddof=1))\n",
245 | "\n",
246 | "cax = imshow(np.cov(x, y), interpolation='nearest', cmap=cm.get_cmap('jet', 30))\n",
247 | "colorbar(cax)"
248 | ]
249 | },
250 | {
251 | "cell_type": "markdown",
252 | "metadata": {},
253 | "source": [
254 | "## Problems with Interpretation\n",
255 | "\n",
256 | "Covariance의 값이 크면 클수록 strong relationship을 나타냅니다.
\n",
257 | "하지만 서로 다른 수치를 갖은 값끼리 (원달러 vs 엔화, inches vs Km) 비교를 할 수는 없습니다.\n",
258 | "\n",
259 | "해석의 가장 큰 문제점은 결과값의 range자체가 다양하기 때문에 해석의 어려움이 있습니다.
\n",
260 | "예를 들어서 데이터가 달라짐에 따라서 결과값이 3이 될수도 있고, 500000이 될 수도 있습니다.
\n",
261 | "300이라는 값이 두 variables사이의 관련성이 있다는 것을 설명할수는 있지만 `correlation coefficient`와는 다르게 정확하게 얼마나 강한 관계를 갖고 있는지는 설명을 못합니다."
262 | ]
263 | },
264 | {
265 | "cell_type": "markdown",
266 | "metadata": {},
267 | "source": [
268 | "# Correlation Coefficient\n",
269 | "\n",
270 | "위에서 적었듯이 covariance는 해석상의 문제가 있습니다. (즉 관련성 있는건 알겠는데 정확하게 얼마나 관련성이 있는지 확신하기 어려움..) 이를 해결하기 위해서는 `Correlation coefficient`를 사용하면 되면 아래와 같은 공식을 사용하면 되며 **결과값의 range는 -1 그리고 1 사이의 값**을 갖습니다.\n",
271 | "\n",
272 | "$$ Cor(X, Y) = \\frac{Cov(X, Y)}{\\sigma_X \\sigma_Y} $$"
273 | ]
274 | },
275 | {
276 | "cell_type": "code",
277 | "execution_count": 5,
278 | "metadata": {},
279 | "outputs": [
280 | {
281 | "name": "stdout",
282 | "output_type": "stream",
283 | "text": [
284 | "[Numpy Correlation Coefficient]\n",
285 | "[[1. 0.56694671 0.533745 ]\n",
286 | " [0.56694671 1. 0.77812706]\n",
287 | " [0.533745 0.77812706 1. ]]\n",
288 | "\n",
289 | "[My Correlation Coefficient]\n",
290 | "[[1. 0.56694671 0.533745 ]\n",
291 | " [0.56694671 1. 0.77812706]\n",
292 | " [0.533745 0.77812706 1. ]]\n"
293 | ]
294 | }
295 | ],
296 | "source": [
297 | "def correlation_coefficient(data, data2=None, ddof=0):\n",
298 | " k = data.shape[0]\n",
299 | " N = data.shape[1]\n",
300 | " \n",
301 | " if data2 is not None:\n",
302 | " k += data2.shape[0]\n",
303 | " data = np.vstack((data, data2))\n",
304 | " \n",
305 | " cov = np.zeros((k, k))\n",
306 | " \n",
307 | " for i in range(k):\n",
308 | " for j in range(k):\n",
309 | " a = data[i]\n",
310 | " b = data[j]\n",
311 | " cov[i, j] = np.sum((a-np.mean(a)) * (b-np.mean(b)))/(N-ddof)\n",
312 | " cov[i, j] /= (np.std(a) * np.std(b))\n",
313 | " return np.array(cov)\n",
314 | "\n",
315 | "data = np.array([[4.0, 4.2, 3.9, 4.3, 4.1], \n",
316 | " [2.0, 2.1, 2.0, 2.1, 2.2], \n",
317 | " [0.6, 0.59, 0.58, 0.62, 0.63]])\n",
318 | "\n",
319 | "print('[Numpy Correlation Coefficient]')\n",
320 | "print(np.corrcoef(data))\n",
321 | "print()\n",
322 | "print('[My Correlation Coefficient]')\n",
323 | "print(correlation_coefficient(data))"
324 | ]
325 | }
326 | ],
327 | "metadata": {
328 | "kernelspec": {
329 | "display_name": "Python 3",
330 | "language": "python",
331 | "name": "python3"
332 | },
333 | "language_info": {
334 | "codemirror_mode": {
335 | "name": "ipython",
336 | "version": 3
337 | },
338 | "file_extension": ".py",
339 | "mimetype": "text/x-python",
340 | "name": "python",
341 | "nbconvert_exporter": "python",
342 | "pygments_lexer": "ipython3",
343 | "version": "3.6.4"
344 | }
345 | },
346 | "nbformat": 4,
347 | "nbformat_minor": 2
348 | }
349 |
--------------------------------------------------------------------------------
/05_gaussian-process.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": 2,
6 | "metadata": {},
7 | "outputs": [
8 | {
9 | "name": "stderr",
10 | "output_type": "stream",
11 | "text": [
12 | "UsageError: Line magic function `%pyalb` not found.\n"
13 | ]
14 | }
15 | ],
16 | "source": [
17 | "%pyalb inline"
18 | ]
19 | },
20 | {
21 | "cell_type": "markdown",
22 | "metadata": {},
23 | "source": [
24 | "# Gaussian Processes\n",
25 | "\n",
26 | "**Gaussian Processes(GP)** 는 regression 또는 probablistic classification 문제를 푸는데 디자인된 supervised learning 방법중의 하나 입니다.\n"
27 | ]
28 | },
29 | {
30 | "cell_type": "markdown",
31 | "metadata": {},
32 | "source": [
33 | "## Gaussian Process Regression (GPR)\n",
34 | "\n"
35 | ]
36 | },
37 | {
38 | "cell_type": "code",
39 | "execution_count": 3,
40 | "metadata": {},
41 | "outputs": [
42 | {
43 | "data": {
44 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEICAYAAABLdt/UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnXd0VMf5v59ZrXrvXQIkRBUdDBhXwA03cMUlcRInjhMndn7+OomdxHEcJ7bjNB/bMXaKewnGveFOr6IjISEkAeq9d+3O74/ZFStpd7UrLYiIec7RkXTv3Lmzd3ffO/fzlhFSSjQajUYzejCM9AA0Go1G41m0YddoNJpRhjbsGo1GM8rQhl2j0WhGGdqwazQazShDG3aNRqMZZWjD/j+CEOIhIcSrIz2OkUQIIYUQ6R7u82whRL4QokUIcbUn+x7ieMZYXqdxpMfiCYQQNwshPh/pcZxpaMN+mmAxLNYfsxCi3eb/m0d6fKOYh4GnpZRBUsr3TvXJhRBHhRBLTmL/wUKIv1rO0yqEOC6EWCOEOMumjbTsaxFClFrae3ni/FLK16SUF3miL43raMN+mmAxLEFSyiDgOHCFzbbX3OlrtMz2ThGpQLa9HULxP/sdEUL4Al8DmcDlQAgwCXgTuLRf8+mWz95i4Cbg+x44/5A/h//r136k0RfufwsfIcTLQohmIUS2EGKOdYdlRvYLIcR+oFUIYRRCJAgh3hZCVAshioQQP7VpbxBC/FIIUSCEqBVCrBZCRNg7qRAiSgjxkRCiQQhRJ4TYaP3S2fTRLITIEUIstznuNiHEZiHE3yzHFgohFlq2FwshqoQQ37Zp/6IQYpUQ4gtLf+uFEKkOxuQrhPizZQZaaTnOf7Dx9uujABgHfGiZrfoKIdYJIf4ghNgMtAHjLNfxA0tfR4QQ37fp4yEhxFtCiFctYz4ghMgQQtxveX3FQgi7M1YhxCtAis35f26z+2bLa6sRQvxqKO8bcCuQBFwtpTwopTRJKVullGuklA/ZO0BKmQtsBKY6GLMUQvzU8l7WCCGesPks2L7ftcBDlm2bbI5fKITYKYRotPxeaLNvwLV38Lo0gyGl1D+n2Q9wFFjSb9tDQAdwGeAFPAps63fMXiAZ8EfdtHcBDwI+qC9JIXCxpf3dwDbUF98XeA54w8F4HgVWAd6Wn3MAYdl3HZBgOd8NQCsQb9l3G9ADfMcy5kdQTyPPWM55EdAMBFnav2j5/1zL/ieBTTbjkEC65e+/AR8AEUAw8CHw6GDjHexaA+ssY5wCGC3HbwD+AfgBM4Bq4MJ+78vFlvYvA0XAryzHfh8ocvW9BsZYXuc/Le/jdKATmDSE9+1N4EUXPm+213UyUAF8z0nbbyzXPQU4DNze7/3+ieVa+Fu2bbLsjwDqUTccI7DS8n+ko2s/0t/F/9WfER+A/rHzpjg27F/a/D8ZaO93zHdt/j8LON6vj/uBFyx/HwIW2+yLB7oBo53xPAy8b/3yDzL2vcBVlr9vA/Jt9mVaDEOszbZaYIbl7xeBN232BQEmINnyvwTSAYG6gaTZtF1gNaBujrfPtbYYl4dt/k+2jCHYZtujVoNpeV++sNl3BdACeFn+D7aMO8zF84+xtE+y2bYDuHEI79uXwGM2/88AGoAmIM9mu7RsqwcKUDdgg4PxSuASm/9/BHxl8373/8zdxgnDfiuwo9/+rcBt9q69/hn6j5Zi/reosPm7DfDrp2MW2/ydCiRY5IgGIUQD8AAQa7P/XZt9h1AGLJaBPAEcAT63PIL/0rpDCPEtIcRem36mAlE2x1ba/N0OIKXsvy3I3muQUrYAdagnAluigQBgl81511q2Ox2vi9hexwSgTkrZbLPtGJBo83//11MjpTTZ/A99X6Mr9H+vrce7877Vogw/AFLKvVLKMGAFarZvyywpZbiUMk1K+WsppdnJ2GyvzzH6vj/FOCbB0t6W/tfS2fEaF9GGfXRhW6qzGDWDDbP5CZZSXmaz/9J++/2klKUDOpWyWUp5r5RyHHAl8P+EEIst+vc/gbtQj9NhwEHUjHqoJFv/EEIEoR7fy/q1qUEZzCk2Yw+VyvnncLxujMH2OpYBEUKIYJttKcCA6zRE3C2v6vL7BnwFXCSECBz+MPuQbPN3Cn3fH2evpwx1Y7Kl/7XU5WY9gDbso5cdQLNQDlV/IYSXEGKqEGKuZf8q4A9W56QQIloIcZW9joQQlwsh0oUQAmhEzRDNQCDqi1htafcdHDjd3OAyIcQiIYQP8HuUH6HPLM4ym/wn8DchRIzl3IlCiIsHGa/bWM69BXhUCOEnhJgGfA/wVE5BJe45CV1+31B6fzlqhj/V8hnwA+Y4aO8q9wkhwoUQySjN/78uHvcJkCGEuEko5/4NKEnxo2GOR9MPbdhHKRYp4HKUrlqEmuX+Cwi1NHkS5Xz8XAjRjHLInWWnK4DxKL22BaWJ/kNK+Y2UMgf4i2VbJUpD3zzMob8O/BYlwcwGbnHQ7hcouWWbEKLJMr4JzsY7jDGtRGnfZcC7wG+llF8Ooz9bHgV+bZFW/s+F9i6/b1LKDuACIAf4GIu2DswFrh/GmN9HOeb3Wvr9tysHSSlrUZ/Je1Ey0c+By6WUNcMYi8YO1sgGjWbEEUK8CJRIKX890mPR2EcIIYHxUsojIz0WjWP0jF2j0WhGGcM27BbdcYcQYp9QSTO/88TANBqNRjM0hi3FWBxUgVLKFiGEN7AJuFtKuc0TA9RoNBqNewy7pohUd4YWy7/WTD8t3Gs0Gs0I4ZFiUUJVgtuFygp8Rkq53U6bHwA/AAgMDJw9ceJET5xao9Fozhh27dpVI6WMHqydR6NihBBhqHCwn0gpDzpqN2fOHJmVleWx82o0Gs2ZgBBil5Ry0DwEj0bFSCkbUAWCLvFkvxqNRqNxHU9ExURbZuoIVTZ1KZA73H41Go1GMzQ8obHHAy9ZdHYDsFpKqVOENRqNZoTwRFTMfmCmB8ai0Wg0Gg+gM081Go1mlKENu0aj0YwytGHXaDSaUYY27BqNRjPK0IZdo9FoRhnasGs0Gs0oQxt2jUajGWVow67RaDSjDG3YNRqNZpShDbtGo9GMMrRh12g0mlGGNuwajUYzytCGXaPRaEYZ2rBrNBrNKEMbdo1GoxllaMOu0Wg0owxt2DUajWaUoQ27RqPRjDK0YddoNJpRhjbsGo1GM8rQhl2j0WhGGdqwazQazShDG3aNRqMZZWjDrtFoNKMMbdg1w0JKSbe5e6SHodFobNCGXTMsfrXpV9z40Y10mbpGeigajcaCNuyaIbOjfAcfFn7I4frDvHro1ZEejkajsTBswy6ESBZCfCOEyBFCZAsh7vbEwDSnNz3mHh7f+TgJgQmcnXg2z+17juq26pEelkajwTMz9h7gXinlZGA+8GMhxGQP9Ks5jXkn/x0O1x/m3jn3cv+8++kyd/Hk7idHelgajQYPGHYpZbmUcrfl72bgEJA43H41py9NXU08vedpZsfOZmnqUlJDUrl18q28X/A+B2sOjvTwNJozHo9q7EKIMcBMYLsn+9WcXqzat4qGzgZ+MfcXCCEA+EHmD4j0i+TRHY8ipRzhEWo0ZzYeM+xCiCDgbeAeKWWTnf0/EEJkCSGyqqu1Fvu/SlFjEW8ceoMV41cwKXJS7/YgnyDunnU3+6v383HRxyM4Qo1G4xHDLoTwRhn116SU79hrI6V8Xko5R0o5Jzo62hOn1YwAT+x8Aj+jH3fNvGvAvqvSr2JK5BT+lvU32rrbRmB0Go0GPBMVI4B/A4eklH8d/pA0pyubSjexsXQjd0y7gyj/qAH7DcLAL+f9kqr2Kv514F8jMMKRp6Wzhzd3HMds1nKUZuTwxIz9bOBW4EIhxF7Lz2Ue6FdzGtFt7uZPO/9ESnAKN0+62WG7GTEzWDZuGS9lv0RJc8kpHOHpwRNrc/nlOwfYXlQ30kPRnMF4Iipmk5RSSCmnSSlnWH4+8cTgNKcPq/NWU9RYxH1z78Pby9tp23tm3YOXwYu/7jqzHuAOVzbz6vbjAOwtbhjh0WjOZHTmqWZQ6jvqeWbvMyyIX8B5SecN2j4uMI7bM2/ni2NfsKN8xykY4cgjpeT3H+UQ6ONFXIgfe47Xj/SQNGcw2rBrBuWF7Bdo7W7l53N/3hveOBjfmvwt4gPjeW7/cyd5dKcH3+RVsTG/hnuWZDB/XAR7ixt02KdmxNCGXeOUtu421hxew5KUJaSHp7t8nJ/RjyvSriCrMova9tqTOMKRp6vHzCMfHWJcdCC3LkhlRnIYVc2dlDd2jPTQNGco2rBrnPJ+wfs0dzVz6+Rb3T72otSLMEszXxd/fRJGdvrwyrZjFNa08ptlk/H2MjAzJRzQOrtm5NCGXeMQszTz2qHXyIzKZHr0dLePzwjPICU4hS+PfXkSRnd6UNfaxZNfHubcjGjOn6DyMybFh+BjNGidXTNiaMOuccjGko0cazrGrZNvdVlbt0UIwdLUpWwv305Dx+icvf71izxau0z8Ztmk3mvkYzQwJSFEz9g1I4Y27BqHvHLoFWIDYlmSumTIfSxNXYpJmvim+BsPjuz0IK+imde3H+fW+amMjw3us29GchgHShvpNplHaHSaMxlt2DV2yavLY3v5dlZOXIm3wXncujMmR04mITCBL4+PLjnGGt4Y7OfN3YvHD9g/MyWcjm4zeRXNIzA6zZmONuwau7x26DX8jf5cm3HtsPqxyjFbyrbQ3DV6jNxXh6rYdKSGny0ZT3igz4D9M5PDANij5RjNCKANu2YAte21fFz4MVemXUmob+iw+1uSuoQecw/ritcNf3CnAd0mM3/45BDpMUHcPD/VbpukcH8iA33Ye1wbds2pRxt2zQBWH15Nl7nLaU0Yd5gWPY2YgJhREx2TV9FMUU0rP74gDW8v+18hIQQzksPYW6wjYzSnHm3YNX3oMnXx39z/ck7iOYwNHeuRPg3CwNLUpWwq3URrd6tH+hxJSurbARgfE+y03cyUMAqqW2ls7z4Vw9JoetGGXdOHT4s+pbajllsm3+LRfpekLKHL3MXGko0e7XckKGtQhj0xzN9puxnJKlFpf4mWYzSnFm3YNb1IKXkl5xXSw9JZEL/Ao33PjJlJpF8kXxz7wqP9jgSlDe34e3sRFuA8WmhacihCwB6ts2tOMdqwa3rJqswirz6PWybdMqSEJGd4GbxYkrqEjaUbae9p92jfp5qyhnYSw/0HvUYhft6kRwfpRCXNKUcbdk0vL+e8TLhvOMvGLTsp/S9NXUp7TzubSzeflP5PFaUN7SQMIsNYUQ5UXelRc2rRhl0DwNHGo6wvXs91E67Dz+h3Us4xO3Y2Yb5hfH7sc6ftthfW0nMaZ2yW1rcPqq9bmZESRl1rF8V1/9tPKZr/LbRh1yCl5LGdj+Fv9GflxJUn7TxGg5HFKYvZULKBTlOn3TY5ZU3c8Pw2Xtxy9KSNYzh0dJuobe0iMcy1m9+M3kSlYYQ9Fq6H8n1DP15zxqENu4bPjn3G5tLN/GTmT+wuUu1JlqYupbW7la1lW+3u31aoarevzio+LeWLUmtETLhrM/YJscH4e3sN3YHaVgdvrIT37xra8ZozEm3Yz3Cau5p5fMfjTIqYxLXjb6CxvZvKpg6O1rRyqLyJ3cfr2XKkhtyKJo+cb17cPIJ9gh1Gx+w8qhaBPlzZwv6SRo+c05NYQx0TQl0z7EYvA5lJoUN3oO54HrpboWI/1BYMrQ/NGYdxpAegGVme3P0kdR113DX5UTIf+oJuk/1Zso+Xgd0PLiXId3gfGW8vby5IvoBvir+h29TdZ2FsKSU7j9axZFIsm45UszqrmOkWKeN0obTevRk7qLoxL2w+SmePCV+jl+sn62yB7asgcQ6UZkH2O3Dufe4OWXMGomfsZzD7q/ezOm81N028ifxiZUB/vWwSf1g+lb9eP51Vt8zixe/M5VeXTaLLZCa71DMz6ItSL6K5q5lt5dv6bD9a20ZNSxcXTozh0qnxfLCvjI5uk0fO6SnKGtoxCIgLcd3BPCM5jC6TmZwyN596dr8E7fVwyaOQfBZkv+fmaDVnKtqwn2lYdOtucze/2/o7ogOiuWvmXWwpqGFmcji3nzOOm89KZcWsJC6ZGs/5E2K4amYCAAc8ZNgXJCwgwi+CB7c8SHZNdu/2nUVKhpk3NpzrZifR3NHDZ9kVHjmnpyhpaCcuxA+jgxox9hjSUnk9nbDlaUhdBMnzYMoKqDwI1YfdHbLmDEQb9jOJpnJ4LAXyv+C1nNc4XH+YB+Y9QE+3DwdLG1mQFmn3sJhgP2JDfMl2d8bpAB8vH/590b/xMfhw29rb+OrYVwDsOFpHRKAPadFBzB8XSVK4P29llXjknJ6itL7dLRkGIC7Uj7gQP/cM+/7/QnMZnPMz9f/kqwAB2e+6dW7NmYk27HY4XNnMpvwa6lu7RnoonqXgK+hsomznKv6x7x+cn3Q+F6ZcyPaiWswSFjow7ACZiaEem7EDpIen89qy18gIz+Bn637GiwdfZMfRWuakhiOEwGAQXDs7ic0FNZTUt3nsvMOlrNH15CRbrIlKLmE2waa/Q9w0SFustoXEQ+pCpbNrNIOgDXs/alo6uW7VVm7593Zm/v4Lzn7sa374yi6e+eYI6/KqqG2xH3/9P0HheiTwh8b9IOGBsx5ACMGWglr8vA3MSHHsqJyaGEpBdQutnT0eG06UfxT/vvjfLE1dyl92/YVK79eYlRrSu//a2UkAvL2r1GPnHA4ms6S8ocPl5CRbZqSEcay2zbXPT877UFcA59wLtmULpiyH6lyoOuT2+TVnFtqw9+ORj3Jo6+rhyRtncP+lE5mZEkZuRRNPfJbHbS/sZPYjX/LSaZo84xQpoWgDX8ZnsCHAjx9HzSU+KB6ArQW1zB0T4TRiIzMxFCkhp9wzcowVP6MfT5z3BOfH3oBP+A7WNz7Wu9JSUngAC9MiWbO7GLN55GPaq5s76THLIc3YrSsq7Rus0qOUsOmvEJkOk67ou2/SlSAMcFDP2jXO0Ybdho351by3t4w7z0vjqhmJ3HFeGk/fNIt1913Avt9exOvfP4u06EA+2l820kN1n5p8WlsreSzIyEST4OYyFRNd3dxJXmWzQ33dytREtZLSgZMQW24QBiK7lmOqvI7chj3c+smt1LarRKXr5yRTXNfOtqJaj5/XXUoblCTkrsYOkJkUipdBDL6i0pGvoOIAnH0PGPrdaINjIfVspbOfhslbmtMHjxh2IcR/hBBVQoiDnuhvJOjoNvHr9w4yNiqQH12QPmB/qL83C9OiuHhKHLuPN9Dc8T+2eELRel4PCaaqp4VfJV6C8fhWaDjem+m5MM15xmlsiB/Rwb4c9KDObsuOojpmRlzEs0ufpaCxgPcL3gfg4ilxBPsZWXMaOFFLGzoAVYfd3QqVAT5GMmKDB18DddNfISQRpt1gf//UFVCbryJkNBoHeGrG/iJwiYf6GhGe+jqfY7Vt/OHqqfh5O5Ykzhkfjcks2Vow8jNId2gp/JoXw8I4N+lcZsy9U208sIYtBbUE+xmZmhDivAOUHHOwzPOGvamjm0MVTcwdE8H8+Pmkh6WzrUzFuPt5e3Hl9AQ+OVhO0wjfTK3JSRjrWPTGIraUbnHr+EErPR7fBsc2w4K7wDhwgWzAIsd46egYjVM8YtillBuAOk/0NRIcrmzmufWFrJiVyMJ05zPXWalhBPh4sTG/5hSNzgOYTbxak0WTAX4040cQMRaS5sGBt9haUMNZYyNdisuemhjKkaoW2ro850AF2HWsHilh7pgIAObHz2d31e7eQmHXzUmmo9vMx/vLPXpedyltaCPU35td1VvoMnexu2q3W8dPSQihuaOH8sYO+w02/hX8I2D2tx13EhgFY89VOruWYzQOOGUauxDiB0KILCFEVnV19ak67aCYzZIH3jlAsJ+RX102adD2vjU5nDcmgI35p89rGIym4m28HODD+aEZTImcojZOux6qcvCry3Ua5mhLZmIoZgmHPOxA3VlUh9EgmGmJylmQsIBOUyd7qvYAMD0plIzYIFZnFXv0vO5SZomI2VKmZupHGo64dXx6TBAABdUtA3dWHIT8z+CsH4JPoPOOpiyH+iJd8VHjkFNm2KWUz0sp50gp50RHR5+q0w7Kf7OKyTpWzwOXTSIyyNd549xPYNU53N/2Z47WtnG89vSJr3bGK/uep9nLwI9m/+zExinLMQsvrvbazMJ01w07eN6BuvNoHVMSQwnwUXVoZsfOxiiMvXKMEILrZiez53gDR6qaPXpudyitbyc+zJudFTsBKGhwryhXWrTFsFfZMezbV4F3IMz7/uAdTboCDEYd065xyBkdFVPV3MGjnxzirLERvTHTDinbA29/D3xDSKlZz0LDQTYeOf1n7Y2djbxSv5cl3QYmJS86sSMwitzAuVxt3EpG9CAzRAuxIb5EBflwoNRzM/aObhP7ihuZNyb8xNC8A5kWPa1PLZmrZyZiNIgRzUQta2jHP7iM1u5W0sPSOd583GFdeXtEBfkQ4mfkiL0Z+7EtMO58CIgYvKOACNVWR8doHHBGG/ZHPjpER7eZP67IdL5+ZWMJvH4jBETBnZuQYSn8zvc1NuVVnrrBDpGXDrxAK2bujJ7fZ7uUkv92zCeOGgzF9muj90cIwdTEULI96EA9UNpIl8ncq69bmZ8wn5zaHBo71bmig325YGIMb+8upXsEVldqbO+mubOHdq8cDMLAyokrMUszRxuPutyHEIK0mCAKqlr77mirUwlJSXNcH9CUFdBwHErd0/k1ZwaeCnd8A9gKTBBClAghvueJfk8mGw5X88G+Mn50QVrvI7JdOprgteuhuw1uXg1hKYilDzNeHiO28K3Tegm3+o56Xst9lYtbWsnIuLzPvqO1baxumUa3lz/sX+1yn5mJoeRXtXis6uIOS+GvOf0M+4L4BUgk28u39267akYCNS2dHtf4XcFah72i+wBTIqcwM2YmAPkN+W71kx4dNFBjL8lSv5Pmut7RxMvA4K3lGI1dPBUVs1JKGS+l9JZSJkkp/+2Jfk8mr2w7RnyoH3een+a4kakH1nwHavLg+pchxuJcnXw1tZGz+bF8kwOFIx9f7YgXs1+k3dTJnQ1NKrHFhi0FNbTjR2fapZDznqom6AJTE0MxmaXHMlB3Hq0jPSaIiMC+4X1ToqYQ6B3YR44ZG6UkI6uRPZWU1reDoZ2StjwWJCxgTMgYjMLovs4eE0RVc2ff0M2SnSqjNGGm6x35h0P6YlXK13z6Ti40I8MZK8VklzY6T6OXEj69D458Ccv+CmkXnNgnBN6XPkq0aKJn/Z9PzYDdpLa9ljdy3+ASGUBa1OQB2u2WglriQ/0InLMSOhoh3/6KRv2xOlA9kahkMkt2Ha0fIMMAeBu8mRs7t49ht65aVNbgIFzwJFLa0I4xsACzNLEgfgHeXt6khqS6HRlj14FashNipoCvkydHe0xZDk0l6niNxoYz0rDXt3ZR1tjBZGdJOVufgaz/qNRuO3HFIeln8Y3vhUwveR3qj53E0Q6NF7NfpNPUyQ/Lj8HY8/rsM5sl2wpqWZAWiUi7UPkODrgmx8SH+hEZ6OORyJjciiaaO3uYNzbc7v75CfMpbi6mpFk9FYUFeOPnbRiRGXtZQzs+QQX4G/2ZHj0dgLSwtCFExqinjoJqi85uNkPpLvf0dSsTLgMvH/XEpdHYcEYadquMMMWRYT/0EXz+a1UDe/FvHfaTN+VnmKSg67PfnIxhDpma9hrezH2TZVGzGdfZPsCwH65qpra1S5UR8DKqNPW8tWrmPghCCKZ4qIRv1tF6ALszdlA6O9CrswshSAj1d5zgcxIpaWjHO+gI8+Lm9S7nlx6WTklziVvlBVIiAvD2Eid09prD0Nnknr5uxS9Evbe5H+voGE0fzkzDblkwYnK8A8P+1cMQOxWWPwcGx5do5tQprOq5Ap/c91U6+Ahjlmaq26p5Zu8zdJu7uUOEq3jnlL4RMVuOqHIIvYW/Mq8HUycc+tCl82QmhnjEgbrjaB0JoX4khQfY3T82dCwx/jFsLT8RtZMQ5k9Z46mfsR9rLMZsrGZBwoLebWlhaUgkhY2FLvdj9DIwJjLwhBRjlVGGYtgBJi6DhmNQlTO04zWjkjNyMevsskbiQvzsJyR1d6giS+f8H3g7r+I3MyWcH3tdyXe9NxC69pdw+9dObwSeZG/VXjaXbaa8pZyK1grKWsuoaK2g26yccsvTl5OavVEZjH7a7ZaCGsZEBpyoK540B8LHquiYmbcMeu5MiwM1t6KZGUNcbFpKyc6iOuaPc5wcJYRgfsJ8NpRswCzNGISB+FC/ESnnUNa5DwJOPEWAWiwEVKJSb0avC6RFB3HYmmhVshP8QlWZ3qEw4VL46B6VPBfr+hg0o5szc8Ze3uRYX6/JA2mG2MmD9uNjNDAjLZGnxE0qgclFnXq4rC1ay21rb+O5fc+xtXwrHaYOpkZO5ZbJt/Crs37FM4uf4Tczfgrle1VdERt6TGa2F9axwLaaoxCQeR0UbYDmwWPze0v4DkOOOV7XRlVzJ3PHOk/ImR8/n4bOBvLq8gCID/OnsrnjlMayd/aYaPPKJdArkrGhY3u3pwSn4G3wdt+BGhPI8do29RpKsiBxztAnBMFx6vjcj4Z2vGZUcsbN2Du6TRRUt3LxlDj7DSotj7Qxgxt2UNUeHzo0l/tSpuP75e9UuvdgtT6GwftH3ufBLQ8yM2YmT134FME+wfYb5n6sblD99PWDZcphOaA+zPiLYMOfoDRLPd47ITHMn/AAbw4Ow4FqjV+f50BftzI/XslI28q3MSlyEgmhfkgJlU0dDiUcT1Pa0Iox8Ajjgxf1SWQzGoyMCR3DkXr3a8b0mCXHy6tIq8oZuKCGu0xcBl/9DhpLITRxeH1pRgVn3Iw9r6IZk1k61tercsDLFyKcxLfbcM74KCQG1o+5Wy0+fPBtD462L6vzVvPrzb/mrLizeHbJs46NOqjZt9F/QLTFlgIlYwxYWCN2ioqlrjgw6DisGajDmbHvPFpHqL8342P6hfj1dMHhz9VvIDogmvSwdLaWKZ093iIfnUoH6tbifQivdmbHnDVgX3po+pBrxtQe3grIoevrVqw34rz1+OiTAAAgAElEQVRPhtePZtRwxhn2ExExofYbVOVAdIaKFnGBsVGBJIb583ZNCgRGw9HNnhpqH17NeZXfb/s95yWdx1OLn8LfOMgqPoXrldPU2NePsLWglolxwUT19y/4BCid1wXDDkpnP1zZTGfP0ByoWUfrmTsmHIPBMgOWEvI+hX/Mh9evgx3P9ba1LeObGOYHnNokpe0VyjF+Yb8kL1AO1LLWMtq6XS8IN85i2M3HLY7TxFnDG2BUhpqI5H48vH40o4YzzrBnlzUS7GskydHyZpU5LsswoGav52ZEsaWgDnPyAjju3uILrvCvA//i8Z2PszR1KX87/2/4eg1ShbKlCqoPwbi+Mkxnj4mdR+scL4MXlwkV+10a09TEUHrMkrwK96stVjV1UFjTeiLMsTIHXlkOb9yoloOLSIN9b/a2nx8/n05TJ3ur9hIfeupn7IcasjB1JDAhJn7APlsHqqsE+RqJC/EjqGYPRI53rfCXM4RQs/ajm1wKWdWMfs44w55T1sSkhJATM0Vb2uuVnOKGYQelszd39lASOkMVZmrwTN1wKSXP7H2GJ3c/yWVjL+NP5/6pN4baKUUb1O9+jtNN+TV0dJs5Z7yDxURip6rxtw+yfBs2JXyHIMdstSzHtyhRwEf/D1adrZzPlzwOd26B+Xeqpd8sTw9z4uaoMr7l2wj0NRLiZzxlM/a27jYqu/Lw7Z5oN0s5PUwZdrdrs0cHktSaM3wZxsrEZWDudjmDWDO6OaMMu8ksOVTe7ERfP6R+uxk2tjAtEoOADR0ZasNx16olDsZL2S+xat8qlqcv54+L/ojR4KKvu2g9+IZC/Iw+m9/ZU0p4gDeL0h3Uw4+bpn67sJ5mUrg/of7eQyotsK2wlm/5bWDyW+fBrhdh7u3w0z0w/4fg5a0qFxqMvbP23jK+lvrsCWH+p6ysQFZlFhIT0cZpdvcnBSXh6+XrtmGfHdpEmGxADiXj1O5A5iopUMsxGs4ww360tpX2bpPjjFNrkkfM4Csp2RIW4MO0pDDeKwsF3xBVW3uYdPR08O+D/2ZR4iIeWvgQXv1XrHdG0QYYs6jPKvdNHd18kVPJFdMT8DE6eNvjMtXvisENuxCCzCE6UPcdKeYhnkfETFYz9Mue6CtHBEaqKJ0Da8CsNPz58fPJrs2msbOR+FA/yk9RktKWsi0gvUkLtn+z9zJ4MTZ0rNsO1NlG1b4ufPqgbX++4edcvOZi7l13Ly9lv8Suyl0DNX2DF2RcomobWRzPmjOXM8qwZ1szTh0Z9socNdMNcT9k7NzxUewuaaY7cZ5HDPvao2tp6GzgO1O+g0G48TZ1NkP9UUia3WfzpwfK6eoxs3ymk9cWHAuBMS47UKcmhpJX4Z4DtbyxndCGbAyY4dz7IGai/YbTb4SWCihcB6i6MRLJjoodxIedurICW8u2Ym4bS3K4A2c7So5xW4rpPESb9CXP7HyBlx5zD18f/xqjwUh2bTZ/zvozt629jQVvLOCaD67hoS0PcbzpuGo8cZkqT3B0o1tj6aWjUdVH2vT3oR3vYRwu+q0ZlDPKsOeUNeHtJRgf4yBMsCpHzdadLbrhgHMyojFLKAqcrpKcWoeeHSml5PVDr5MWmsbcODc12Loi9btfuOY7u0sZGxU4eKaoGw7UzMRQuk2S/Eo7KwI5YGtBLdOEJQXfWZnajEtURqZFjpkaNVWV8S3bRmKYP3WtXR6rCe+IitYKChsL6WpJP5Gla4e0sDQq2ypp6nK9lHF04372y3EU1Dq/QR1rOqaKuU3/IWuvWcu669fx9IVP8/3M7xPtH837R97npeyXVONx54N3gHtyjNkMBd/A27fDnzPgo5/Bl79VvpYRZMuRGjIf+nxEau+PBs4ow55d1sj4mGD7UoSUyrC7kHFqj8zEUISA3VhknGHo7Puq93Go7hArJ650vrKTPeosRjPiRIZkSX0b24vqWD4zcfD+4jKhOtelx/mpierJxx05ZmtBLXO8C5FhqUpycYTRV2ntuR9BZ3OfMr7xoacm5NEaO29qHe/UsFsdqIUNLtaM6e7AWHWQgyKDI/bWP7XhUJ3y+0yImABApH8k5yWfx10z72LV0lXMjp3NwVqLdObtD2kXqrDRwWa79Ufhmz/Ck9Phlash/3NVTuLqVWr/Mc/4iYbKa9uP09LZwyMf5+iZ+xAYGcPeeeoXJJZSklPW5FhfbypTj6JuRsRY8fP2IjUigM3tyWD0G5Yc80buGwR5B3FF2hAyEustM/bwE4b9/b1lAM5lGCtxmWDqUlUHByElIoAQP6N7hr2wllnGIoQrsdvTb1QrV1mKk81PmM/x5uP4+KpZ3MmWY7aWbyXIGI65M46EQWbs4EZkTMV+hLmbqpDME+V7HZBXl4ePwadPKQNbMqMzOVx3mI4ey7WYuExFdpXtcdzp9ufgyRmw/k8QlQ7X/BvuPQzL/gLTrld+opMQtusqTR3dfHGokvhQPzYfqeWbvKoRG4vHkBK626Glms7qXFpq809qRc6RKSlQVwhHvlIrwJwiqpo7qW3tcqyvV7lXSsAeGbHB5FS2qAiFY0NLVKppr+HzY59z44QbCfAeQsp8XaGKjvBTr1NKyTu7S5g7JpzkCBf6s0bGVByAuKlOm1ozUPeXDB4eCVBc10Z7fQWRflWQOHvwA5LPgvAxSo6ZcROTItTTUJdB3ahO5oxdSrUsX6LvNMoRJDrKewASgxLxN/q77kC1VHTsjJ1FQfHgM/b08HS8DfbDXKdGTaVH9pBbl8uMmBlKwhIGJcfYu3luexbW/hImLINLH4ew5L77DV7quo9gtVKrP+iplTO5b81+/vDxIc4ZH4231/+QwLD7FXWtOxqhqxk6W0Ca6Aa+Fx/LMW8jT9e1MT0sHaInnviJmah8fEOQg20ZmStl9IU3Virjfoqwlup1mnEKbkfE2JIRG8zR2jZ6kucrw9jhvj645vAaesw93DDhhqENoq6oz2z9QGkjBdWtLJ/p3EnXS2SaKkXgogN10fgoDpY2cbTG+cwTVJjjNINVX3dhxi4ETLtRRfk0lpIYpJ442mU1cHJn7PWd9dR11OFrTiXIEjvvCIMwMC50nOvrn5bshNAUYhJTKW/soKWzx24zKSV5dXm9NzR7ZEapSKaDNRY5JiACUhbaLy9gNeqTroDrXxpo1K2kLlByXFuda6/Hw7y7p5RxUYHMTg3n/ksnUlDdyps7RlbzdxkpYcMT8MFdYPRRuSTTboRF98Di3/LsnBXs8/PFyyeY26NDWW80w+G18Pmv4LVr4G9T4L0fDXsYI2PYI9NVGvQbK1V41ikgu0zJBZPiHThOK3MgOH5YWYAZccGYzJLSkFmqAFfxDreO7zZ381beW5ydcDZjQscMbRB1RRAxrvffd3aX4uNlYFnmwKxJuxi8lJ/BRQfqiplJGAS8s3vwtV+3FtYy3/coUhggfvAwP0BJA0g4sJrogGh8DD5UtJURFeR7UkMerZEmne3hJIb5D+qbcGs1pZIsSJrTWzOmsP/i1hYq2ypp6Gzo1dftERMQQ0xADAdqbG7EEy9TE5U6G81/6z9OGPVrX1D5Ao5IWah+eygfwx1KG9rZVnjCH7R0cizzx0Xwty/zaWzvHryDkcRshrX3w9ePwLQb4HtfwPJn4bI/weIH2ZG+iH/V7mLF+BWsueZjxkVO5KeGWt6+6nG4rxBu+wRm3wb7XlcL3wyDkTHsBiN8+wNVk+WNm06Jcc8pbyI1MoBgPwcf6Cr3SgnYIyNWfVEPiPHqNbopx3x1/Cuq2qtYOXHl0AbQ3QFNpb2O026TmQ/3lbF4UgyhAS5krFqJy1Qzdhc0wLhQPxaNj+bt3aWYzY7bS6mW4zvb/zgiaoLr63tGpilpYN+bGBAkBCVQ2lJKQpgfpScxSam4WWUPNzaFkGCpT+OM9LB0atpraOwcxN/QVA6NxZA0l/QY6zJ59g17bl0ugNMZO6hZe++MHdSSeaBqtINa5vGz+2HSlYMbdVASjpevR8J23eX9vaUAXDVDPZ0JIfj1ssnUt3Xxj2/cCyk9pZi64b0fwvZnYf6PlBPa5jo3dDRw/6b7SQ1J5Rdzf0GUfxQvXPwCC+IX8NDWh3j2yGpk6kK49AmIngQf36vkmyEycqJVQAR8y8a4559c455d1uQ449TUA9V5Q46IsTI2KhAvgyC31qyyPt2c8bxx6A2SgpJYlLhoaANoOAbI3hn7xvxqalu7XHOa2hKXCR0N6ibhAtfOTqK0ob23VIA9jte1UdbYTnr3YfeLXk27QUkD5ftICk6ipLlEJSmdRI29pLkEgaCyPtCpvm7FZQdqaZb6nTSXlAj1eSmosi9jHao7hECQEZ7htMvMqEyONx+nocPi64gYqxbHzvvEYtQfUMs8XvufwY06KKk0cfYpn7FLKXl3dylzUsNJiTzhD5qaGMqKmUm8sPkoxXWuF1s7ZXS1wZs3wf7/wuIH4eI/9qmvL6XkwS0PUt9Rz5/O/VOv7yzAO4CnFj/FlWlX8o+9/+DhbQ/TYzDAFU+qRcq/+cOQhzSy3ohe4z5BXZiTZNybO7o5Vtvm2LDXFaql4YY5Y/c1ejE2KpC8ymZIXagWKe52zfjk1eWxu2o3N0680b0sU1vq+kbEvLNblRA4f0KMe/3YOlBd4KLJsQT7GVmzy7Ecs7WgliRRg193vfP4dXtMWa4Wbd7/XxKDEilpKSH+JK99WtxcTLR/DE1tOI2IsTI+bDzA4LXZS3aq1xI/DR+jgdTIAIchj3l1eaSGpA7qRO/V2WttZu0TL1NPjJ89AJOvVpEvrhh1KynzoXwfdA3uO/EUOeVN5Fe1sHzWwInIfRdPwGCAx9bmnrLxuERbnQoXPfKlMsjn3DvA8bk6bzXfFH/DPbPuYVJk36cvb4M3j5z9CLdn3s6aw2v42bqf0Ro/FeZ8F7avgtLdQxrWyLuZAyLgW++fMO5HN3n8FIfKVXjllMSTFxFjJSM2iHyrYTd1KePuAm/kvoGflx9Xp1899JP3xrCP6y0hcPk0JyUEHBEzGRAuG3Y/by+umJ7ApwfLae6wr4NuLaxlUYDFAeZKRIwtARGQcTEceIvkwASau5qJCO6hpbOHJgfnGy7FzcVE+yUAOI1htxIXGEegd+DgM/aSLHXjtJRTTosOcirFONPXrUyOnIxA9NXZJ10JCItR/5d7Rh3U59fcc2JN1lPAu7tL8fYSdv1BcaF+/ODcND7eX86uY/WnbExOaauDF5ep0NLrXlL6eD/y6/N5IusJzk48m1sm2192UgjB3bPu5v5597O+eD3zX5/PwqZtXJ6UyLc++y73fH03D299mKf3PO3y0EbesMMJ4x4UA+se83j3ORbH6eR4JxExwqBuLsMkIzaYY3VttMfNA4RLiR6NnY18XPgxy8YtI9TXcer6oNQVqhjkgAjWHqigs8dsd/YzKL5BStt20YEKSo7p6DbzyYHyAfuklGwtqGVJSKmarcY6D6O0y/SV0FpNYouK1PD1V7JDeUOHykHY86pahHz3y0obbqkaVpxwcXMxwcZYwDXDLoQgLTSNgkYnDlRTj5qB2VR0TIsO4mhtKz39lvpr6mqitKWUiREOSi7YEOQTxLjQcX119vhp8NPdduUXKeXgoaLJls/vKQp7NJkl7+8r44IJMYQF+Nhtc8e544gJ9uX3H50mSUt7X1O246bVMPnKAbs7ejr4+YafE+QdxCNnPzJoaZCbJt3Efy7+Dz+Z+ROuTL+KyTHT8Olq5VjlHr46/hX/PPBPl4d2+iyNFxABs76ldKX6oyp+2UNklzURGehDbIiDOuaV2UqXHmTxalfIiA1GSihoMTI1dorFgXqf02PeO/IeHaaOoTtNrdQXKX1VCN7ZU8LYqEBmDnGxaeIynSe59GNmchjjogNZs6uEG+am9NlXWNNKVXMnmcEFyqgb7X9xnZK+FPwjSLIsTuHXuJkHjAdJfOMhaLQmUwnA5gvvG6JuUJHpMP5imHadS6dq626jtqOWKUFKwnJFYwels68vWe+4QVU29LT3WdUqLTqQbpOkuL6dsVEnllS0rvE6mOPUytSoqWws3YiU8kQEj010lC1rD1Zw52u7efm78zg3w0GlT79QlcdwihyoWwpqqG7uZIWTiUigr5H/u2gCP397Px/uL+fK6QmnZGwOyf9cPd2mXWB391+y/sKRhiOsWrKKKH8HpbL7MSduDnPiLJ8PKVXkYOF6+NE2TKFJGL/tmsk+PWbsVqavBATsfcOj3VoXr3YYslZ1qI8M89bht3g3/90hncsaGZNX0QwpC1TIo8l+nDKAyWzizdw3mRUzy6XHbqfUFULEuN6QsatnuFBCwBGxU9UN1sWFG4QQXDs7iZ1H6wfEtG8tqMWAmejmnKGvFmT0gakrSDysch/MOf/g216f0WIMgyW/gx9ugt/UwN374Ja34dI/qcxV/3C1qtU7t8OWp1w6lTUiRnZHYjQIYoIHj4oBFRlT11FHbbsDJ7K1Tr7NjD3dsjRgf53dGhHj6mdiWvQ06jrqKG0Z3OH94X6V4PW7D7OdLwqeslBJMSYld53MWfK7u0sJ8TMO6g+6ZnYSk+JDePzTXNq6HH+vTjqdzeppfPxSu7s3lGzgzbw3+dbkb3F24sCVt1xCCFX5FAEf34uXG8UAPWLYhRCXCCHyhBBHhBC/HHJHYcmqkNHe11VMqAfo6jGTX9niOOO0q00ZRIthr+uo49Htj/Lglgd5Oftlt8+XGhmIj5eBw1UWnb27FSr2OWy/uWwzJS0lrJw0zNm6qUcVbgofy3t71Jfb7WgYW3prs2e7fIg1pv3tfjHtWwtrmRdci6G71bXEJEfMu4PghNmECm+KJ17K7O5/8mrG0yr5Iy5TLWcYPgbSl8BZd6gvxa3vwj0HlNb8+a9VlMgglDSr8be3hREX6oeXvUVZ7GCtGWM3nr2uCNY9rm72YSeeaKzL5PXX2XPrcon2j3Z5pjc1SslbfeQYO3R0m1iXV83EuGAKqlt5actRx41TF6iSDuVKkvtz1p+56r2rXK+J4yJtXT2sza5g2bR4/LydBw54GQQPXTGZ0oZ2nvgsz6PjcIvC9Wphk/SBht1kNvHnrD8zNnQsd8+6e3jnCUuGxb+BI19A9jsuHzZswy6E8AKeAS4FJgMrhRBD90LOvAUajw+99Gg/jlS10GUyO46IqckDZG+o43tH3qPb3M28uHk8kfUErx16za3zeXsZGBcdqCoeploSPZw8zq4tWkuobyiLU4ZZXqGxWDm7Isbx3p6BIWNu01ub3TUHKtjEtO8q6Y1pl1KyvbCWK6MqVCN3Hae2RGfA9z4jMWI8ZUYvQoJDKXMlScnLqByIk69SUSLbnnXa3Dpjb2oIJCl0kGUIbXAY8mjqgXe+r/w4K57vEzUR6u9NdLAvBXZm7O48wY0PH4+PwaevA9UOG/NraOsy8atlk7hgQjRPfplPdXOn/cYpC9Tv41swmU18UPABhY2F3PLJLb0F0jzBFzmVtHWZXM6OPmtcJLfOT+XFLUfJOjoy2bHkfw4+wSp6qB8fFX5EUWMRP5n5E3y8hiA79mfeD1Qk2ae/cPkQT8zY5wFHpJSFUsou4E3gqiH3NnGZqom+1z2D6ohBF6+utEbETMEszbyV9xazY2ezaukqFqcs5rEdj/Hf3P+6dc6M2GAlxQTHKZ3TgWE3mU1sLN3IOYnnOKwF4jKW4l9Vxnjyq1pYNs3FTFNHBMdBQJRbhh2UE7WssaM3pv1IVQs1LV3M9TkKPkEQNX5440KtWlTSUkJCmL9ynrqCl7cK+Zt0hcrA3P68w6bH6/MJNfjwQtV3eLPiMvhDAvx5Ajw1G547D168HFZ/a8C1iQmIIdg7eOCMff3jStK4/K99ZutW0qODOGIzY+8ydVHYUOiyvg4qbG5S5KRBZ+xrD1YQ4mdk/rhIfnP5ZDp6TDzxmYMQwuA4FTp7bCv7qvfR0NnAvbPvJTYwlju/vJO3Dr/l8vic8e6eUhLD/JmTGu7yMb+4dCIJof78fM3+k16+eQBSqiUI0y4Y4JjuNnXz7L5nmRQxiSUpSzxzPoOXCqV0o8SDJwx7ImC7yGeJZVsfhBA/EEJkCSGyKqsrHffm7Q+Z10DOBx5ZmDe7rBF/b68+jqk+VOWoaowRY9latpWSlhKuz7geb4M3T5z7BOcnnc8j2x9hzeE1Lp8zIzaI0oZ2VQMkdaFK9LAjLR2oOUBDZwPnJZ1npxc3sYQ6bmtUN7BF6a49wjtEiBMZqG7QP6bdauBTOnJV0tZQY/RtSAxOpKyljLhQX9dm7Fa8vFXm5cTL4dP7YEe/KIO2Ovj6EYoPvU1yezNbTZPYknS7CmPLuEjJU0ExalWnoo3wwrI+USNCCNLD+y26cXQzbPwzzLgZMq+1O6y0mEAKqlp6NewjDUfokT1u+1wyozLJqc2hx2xfe+42mfkqt5Ilk2ItT5ZBfHfRWFZnlbC32EEhN8vnd13xNxiFkWszruWVS19hfsJ8Ht76ME/sfAKTeeiGtbq5k435NVw9M8H+OsQOCPI18tg1mRTWtPL3Lwev0dPVY+ahD7J5bn3B8H0Fldmqgub4iwbsejv/bUpbSvnprJ8O3b9lj/jpcN2LLjc/Zc5TKeXzUso5Uso5XX6D1PqecYuKHjjouqbkiJyyJibGBzvWSatyVJijwYvVeauJ8ItgSaq603p7efOX8//CosRFPLz1YZcdqhmxqh6Nimc/Wy2SXT1wVrS+ZD1ewouFiQuH9uJsqSsCox9fFRuICfbtdcoNi7hM5Vg2uR4r7uftxZU2Me1bC2pJDTXiW5MNiW4mJjkgKSiJbnM3YcFtlDd2uPdFtRr3CZfBJ/8HO/+lDPpXD8Pfp8GGJyj2CyQ68Vzu6P5/HM28Gy75I1z5FFz3Atz8Fnz3U7hjAwRFw8tX90msSwtL40jDETWm9nolwYSPUZUUHZAWHURTRw81Lep74Wopgf5kRmXSYepwGEu/o6iOhrZuLpoS17vtJxeOJzrYl4c+yLZfEiJlAbTXsf7Yl8yOm02QTxBBPkE8feHTrJy4kpdzXuaedfcMXKrPRT7cV4bJLIfkDzpnfDQ3zEnm+Q0F7HN0Y0Jp+Le/nMWLW47y6Ke5/OHjQ8Mz7kcsC4an952Rt/e08/z+55kVM4uzE4boMHWGnZBKR3jCsJcCtmXikizbHNLW3YZZOnGOJs5SJSyHKcdIKVVEjCN9HZQUEzOZitYK1pWs4+r0q/voYj5ePvz9gr8zP34+v93yWz4s+HDQ854w7C0ndEo7dWPWl6xnVuwsQnycjM9V6oqQ4WPZXFjPwrRIz8wW4qapjNwaF6sWWrDGtH+0v5xthbUsT2xQyVrDcZzakBSktFg//wa6eszUtrq5xqfRRyWUZFyqanL8bSps/CukL6b7jg1UYCIwQKXxxzuqExOWDN9Zq6SlN26Eg28DyoHa1NVETVs1fHg3tFQqCcjXQfE56C0GZnWg5tblEugdSFKwixU5LVgzUB3p7J9lV+DnbeA8mxDHIF8j9186kb3FDbyzx87XNnUhxUYjhS0lnJ90fu9mo8HIA2c9wP3z7mdDyQa+vfbbVLRWuDVegPf2lpKZGEq6o1XNBuGBZZOIDvbl52v209Uz0KY0tHVxy7+2sym/msdWZHLbwjH8a1MRv3n/oNPaRk7J/0JNekL6yp3/zf0v1e3Vnp+tDwFPGPadwHghxFghhA9wI/CBswN6ZI/zSnhCKCdqyU5Vw2WIlNS309zR41hfb6tT62rGTOad/HeQUnJtxsDHZV8vX5688Enmxs3l15t/zYaSDU7PmxwRgK/RoEoLhI+B4IQBdTfKWsrIr8/3jAwDUF9ES2AyNS2dLEwbpgxjZQgOVIAZyWGkRQfy5Jf51Ld1c26gRakbaqhjP3oNnrfSHF3W2W0x+qjStdNXqvT7H22F61+iPDAckzThI5XxSwh1EsMeFA23faTCF9d8D7Je6C0tcDjrWch5Hy78zaCvu3/IY25dLhPCJ7i31i3quoT6htrV2c1myWfZFZyfEYO/T1857OoZicxKCeOxT3MHZg5HjGNdmLoW5yUP/KzeNOkmnln8DMXNxTy4+UG3xnustpX9JY1cNWPo8eih/t78cXkmeZXNPN2vSFhlUwc3PLeNg6VN/OPm2dw4L4XfXjGZO84bx6vbjvPzt/djcte4tzco+a2fDNPS1cK/Dv6LsxPOZnbsMAIEPMSwDbuUsge4C/gMOASsllIOGiOXVZnlvMG0G0B4DWvWbi3VO9jiGj0xE3j78NssTFxIcrD9GtX+Rn+euvAp4gLieCPXeZy9l0EwPjaIw5XN6iaVulA5UG0e/6w3h3OTznX3ZQ1ESqgr4phZZUouTHey5Jw7RKarKn9uZKCCNaY9mYomZXAnmPIhIBLCUj0yrPjAeASCbqH0e7d0dluMvrB8lYqYsdTht41hBwav7OgXquLmxy+Fj+5hwuGvAcjd95Kqxb3wp4MOIy7EjwAfLwqqWzBLM3l1eUPKaRBCMDVqqt0Z+76SBiqbOrl4auyAfQaD4KErp1Db2slTX/eTcYRgfWg4aT04/G4sSlzEbVNuY2v51t7r5wpbCtT7d8FEN2sZ9WPxpFiWz0zkH98c6V134WhNK9c8u4WS+jZe/M5cLpkaZ3k5gl9eMpF7loxnza4S7n5zj/NY/v4UrgNpGmDYX8l5hcbORn4y8yfDei2ewiMau5TyEyllhpQyTUo5aEkyb4M3OysGqUERFKPqg+x702mCjzNyypsxCJgY56QGO7De1ERVexXXZ1zvtL8A7wAWpy5me/n2QTXFjJhgZdhBxQM3l59Ytg4lw6QEpzAmZIzLr8chzRXQ087e1jBSIwNICh9GmKMtXkZLbXb3ZuygYugNQi2fF1izX8kwHno89fbyJi4wjmaTevT35EpKVsPU0RZGsK/RcZlnW3wC4IbXYOo1hH7zKAkmM3k+PqPc8swAACAASURBVLD8uT5V/hxhMAjGRQdSUN1KcXMxbT1tbuvrVjKjMiloKBjw+VybXYHRILhw4kDDDjAtKYzrZyfzn01FfZKlmrua2WVu47yWRmh0XOTt6vSrMQiDW4l92wtriQryZZyjwAY3ePDyyYQFeHPfmn3sL2ng2lVbaesy8cYP5rOwXyCBEIJ7lmRw/6UT+Wh/OT96bTedPS46gPO/UDfzxBPZww0dDbyU8xJLUpYwJWrKsF+LJxiRzNNA70B2Ve4a3IEx42alURYMbaWl3PImxkQFOk56qMoBvzBWH/+S2IBYl2bPFyRfQLe5my1lzlOtM+KCqWzqVIsDWGO3y1WiUlt3GzvKd3Bu0rme0eIsETEbakI8J8NYcaM2e5/DQv34/jnj+P78WOU49pAMYyUxKJGa9gp8jQaPVnksbi7Gz8uPuiZ/x/q6PYw+sOKfMOd7TOjoIDc8AUJclxjSo4M4UtnsdsZpfzKjMjFLM9m1Jx6apZR8drCCBWmRhPo7vlHdd8kE/L29+P1HOb3bNpdupgcz57e1O617FBcYx6LERbx35D2HUTm2SCnZXlTHWeMiPPIdCA/04fdXTSW7rImrn9mMj5dg9R0LmJbkuKTGHeel8bsrp/BFTiXff3kX7V2DGHezWTlO0xarSY+F/xz8D23dbfx4xo+H/To8xYgZ9rqOOooai5w3zLhYxVLveXVI58mrbHY8WweoyqE4JoMt5Vu4JuMajIbB6zDMiJlBiE8I3xR/47SdtbRAfmWzKpwvvKBCaZ/by7fTZe6yq1kOCcuTwKGuKM72lAxjJW4atNepJw43uf+ySdya2qBWk/KQ49RKUnCSpXyvn8dn7EnBSVQ0dbhUrrcPBi9Y9hcmzrmTox01bkWKTIgLoayxg32V2RiFsTeL1V3sZaAermzhaG1brxzhiKggX+44bxzrD1f3XtN1JesI9w1nmvQddIHrFeNXUN1ezabSwSu0ltS3U97YwVljh75iWX8uzYzn2tlJTIgL4a07F7oUGfbthWN4/JpMNuZX88C7gzyZVuxXE00bGaaqrYrXc1/n8nGXkx4+tPfsZDAihj3AqKSCQeUYL2+lted9Cq2OF3GwR1tXD8fr2pgQ60BflxKqDvFWkB9ewosV6Stc6tfb4M05SeewoWSD0/hda2RMXmUzePup6IlK9WVbX7KeQO9AZsd4yMlSV4hJeFEmo1gwztOGfWgO1F6sZYtPwoy9ur2a2FAvj8/Yk4KTKGvoIN6Z49QRQjAhbiYS6foaqJyQC/dW5jAubNyQMxYj/CJIDErso7OvPViBELB0sn0ZxharVLOloJYecw8bSzZyTtI5eCXPG7TS47lJ5xLlH8Xbh98e9DzbCmsByfm+h1VZDw/xxLXT+OSni1yqyGnlhrkp/OSCdN7dU8pn2U4ie3rDHE9kiT+//3lMZhN3zrhzqEN2CbNZ8tAHrpf3GBHD7uPlQ0xAzOAOVICZN6uaDAdWu3WOw5UtSAkTHM3YG0vo6mzi3a4KLki+gNjAwT/0Vs5PPp+Gzgb2VTuuAZMY5k+gj5cKeQRVVKviIFJKNpRsYGHCQrzdrZHtiLoiqg2xZMSHExnkegq8S8RaNEM3Hai9lO6G0GTlM/Eg1siY0JBmj62kJKWkpLmEhIAk6lq7SAh1Q4qxwVpq11qh0aVjLGvxFjYddqlUrzP6L5X3WXYFs1PCXSpmNjEumIhAH7YU1LCveh9NXU0qcit1gZIunWQ/ehu8uSrtKjaUbqCy1UkSIrC9qI7v+68j5YPr4J8XDiv6zRYhxJCknbsuHM/k+BB+9e4BalsclFjI/0Kl9ls+y4frD/N2/tssH7/coWPZE0gp+d2H2bzorK5PP0asuuOc2DlkVWYNrrPHTlEXc4970TF5Fco77lCKqcrh88AAGkwdXDfBtXKuVs5OOBujwci64nUO2wghSLeWFgBVArWphENl26lur/ZcmCNgri3kcHc0Z6d5eLYOKv46YtzQZ+xlu91fMckFrLHsgQFNVDR1DKhnPhSq26vpMHUQbFSSRby7UoyFhMAEgn2Ce/VyV4gL8SM4sI1WU71HDHt5azk17TUU17WRU97ExVOcyzBWDAbBgnGRbDlSy7ridRgNRhYmLDyxwHXxdqfHrxi/ArM080GB04hnygv2cR//v73zDo/zLNP97x2NZtRGvXdb1bIsd7lFju3YSZw4yUlIIQ1YYMmhHRbOIUvIsiywC2RpYRf2QAjLoaQQSMNOSGIHlxTbsh3bclGzZKv3MuoaaeY7f7wzKtaMplojKd/vunwlGs2MXo9Hz7zf897Pff9OtugGO+CpbVIo4Sd0Wg0/vm8lxuExvvHq+Zl1aahbyq+tbZgDdQd4+PWHCdeF80jRI9d0bU8eqOa3R+v49HVLXH6M3wr7+sT1dA53cqXvivM7r3oQ2s5NHD66QkVrP8GBAaRHO1CItF3gT+FhpIelsjFpppHPbBh0BtYnrHfaZ89LCKO63VrYE2RL43D1qwiE57mmV6MoWLpquGyJZ4u3NgKO8MBaAJC/DD1XfN6Ggckdu9B1Y1Gg3ZGRlRvYFDETGnZ3Dk+nIIQgPzrfrR27EIK0RCnP9bqwx1kHlTrOTbQWXC3sIOWyrX0jHLhykPUJ6wnThUkBQIDOqT97eng6xYnFvFj9osMhxOYuI18b+hEWbTB89FlpuZy8Gl5+BF79vE9bM+6QnxjOP+zM5fVzrewtu+pMqeZvoFgwZ+/kyVNP8uVDXyY7Mps/7vmjW1f77vLf717mp29Xc++6VB6/1XWllF937OCCnh2g8COAgKo3XX7+ipZ+chMNDv0nqltO8kFQEPfk3ef2IAjIdsyVviuzHgDnJhjoHDDJS7tEeah1pK2UFXEriAn20e56uAftWD+NJFLsw4OoaSStksqbvmb3HtdszWv08cEpQExQDEEBQYxrOgFo8VTLPgVbYbeY5Os463CSE/Ki8qjqqXLLR8UQLtsXzsKrnZEfnU+ACOBc5zneON9KQVK4W06fm7NiEYGdNA7WTR7wBwbJf0c7E9RXc1fOXTQNNFHaWmr3+/1vfIcVmiu0Xf/vcnozPElmH2/9qrwyf/oG6Kiy+9hrzSNbl7IqLZJvvHKe9r4pZzfV++kNieaz5b/i1+d/zT259/Cbm39DYqjrH5ju8uKpRr697yI3L0/ku3eucKvF5LfCnhGeQWxwLCdbXSjsIdGyR+1iHqqiKFIRk+BYEfOS8SI6BHdke2ZEuS1tGwCHGxwn5tgOUKvaBiAsgc6wWM6PtPu0DWOTOmpilxKqv0aBWMutOaxn3QxAabImMCWv8u16kDvclLAUBs3tADR7Mn16FQ39DQSIAIaH5YF7ooc9dpDFdcQ8Ql1fncuPUQKbsZii6Rv07uwlWBtMTlQOH7SVcaq+x63dOkBmTAgxcXJQyfY+B6Q3StMp6J19CGlnxk7CdeG8VGXH6+nKe+RUP82L7CB185S5kQAt7PgnOew10G5tzbjnquoLtAEafnTvSkbGzDz20jnZkrFYuHjlbe5LjOZk2ym+tflb/POmf/aNJa8D3rrQyqMvlrElO4af3r8KbYB7pdpvhV0IwfqE9ZxsdaHPDpB5nUwjGnfuC9IxMEr3oMnxwalpiLPKMCv1cUQFuW4VOpXksGTyovJmbcdMeMa0ywnUd+KkZasvC/tQm1ReJC3xPojbIdFLZY/19DPu6dmbTkFMjhzouAakGFLoNvluSKmhv4HE0ETa+8aJCdU5DX2YDVs7xZ0+e8/4ZcwjSVTYzmW8oDC2kPOd51EUi1OZ49UIIQiJqgJTIkkhU7T4hVblmJPAB32AntuybuNA/QF6RqYET48Y4eVHaBEJHEj/sn1jvuwbrK2ZVfDyZzw/2/GCrLgwHr05n7cr2nnhZAOvnvpPPhalx6wN4ne7f8ddOa4p6Dzl/ZpOvvDcaQpTInjq4XXote6/D/0ajbcucR3tw+2ujSFnbpGOj7bL+1mwHVg6OjgdbymjKlBLfqR33uDb0rZxpuPM9DfvFBLC9YQHaSfWc1ivJWHcTG64/SxKT2iuvYBFESwrKPLZc9pl9YPQXeP08GyCwS45fp3po7MEO6SGpdIy2IRB7xvJY2N/I2mGNJp6PdCwX8XSiKUEagKp6HGtsA+NDdE23IhlNHni4N8bimKLGLUMkhY/NDFT4Sp9pj6MlkpG+/Ipn7qWmCzZaz/n3ML6rpy7GLOMsa923+SNr/0flL5mPj/yOVZlzeLmGJ4kTdoQUPmGW2v3FR/bmMayrMv86+lH+KeLT7Ny1MQfb/rtxJzAtaKssZe//+1JMqJD+H+fWO/xVbh/C7s7ffYMqw2mC8lKtkLqaMde33CEUY2G/OQNri3UAdvTtmNRLLzTZH9NQghyEwxUtw1gMpt4f6yL64eGEN32bVU9YaDlEm1EsWrJtTvAAWS0XGCo68Nip/5bfhBvuHaKgZSwFAbHBkmIMvtsx55mSKOld5gkL9owIG0PsiOzXT5AreqpQkEhWptJuQ927EsM8gouJ7PVbfnfe03vYcHM+MAyjtZcNT9SeLeUvjpx/MyNyqUotmjCXI9zf4ZzL1CV/1nOKNlscDZvERYnD1Sr33Jr7d5iMpv4c9WfuXPv/6BR90sUMcJnewL5qZKK0ZLIpfYBqtv6qWztp7ylj8rWfroHTV57vJstCk+/U8u9vzxKVKiO339qA1Ghnrd6rlFT1jWWRCwhOiiaE60nnF/ehERD/HIZXLD1q7PetaK1n9gwvUNNd4W1r5+X6p0P+rKYZcQHx3Oo4RC3Z9n3Ss5JMPD6uRZOtJ5g2DLG9UPDcgI13jMvkKsJNF6mJyiNJA8u19xCHyZ77Rdelt7iuln8PcZHZYhF1g0++3vaw6aMiQwfoMXoXbun39RP72ivLOzGEZ8ojPKi8zjSeARFUZwWV1vLJjcqf1Ii6wW9xijMo/F0G44B7uVuHmqQ06axofm8d6mTT5dMucJcfqeMFzz3Z9j+2KzPc1fOXfzL0X/h7OX9rNr3FUgt5lndPYToWil0ZMw3lZwbZfrUYBeEXgMp7xQGTAP8qepP/P7i7+kY7qAgpoAfXf8jeutiuPtvN/CT8Y/wnz9yfJ6mC9AQH64nITyIxPAg4sP1ZMWFsacoiciQ2Qv0pfZ+vvrnMk7X97IjP57v3rnCq/Md8HNhF0JM07M73VlkbpE7RvPYjEiqqVS2zm4lUGm8TKBWXi57g0ZouD7tevbV7mPUPIo+YOYHSV5CGM+VjvFm7UGCAvQUm8xSuol72nl7tPeNED/eQmf8Dq+fyyVWPSjdNi/+BVbNEr59/kU5en3nL67pclLC5OV8aKiRK83eKYJs7cCYoCQGRse93rGD7LO/cukVOoY7iA+ZfUDrYtdFIvWRFEWl815lLSNjZq96/OebjIwb11Cjf4P6vnrSw2dG8tlj3DLOu03vsi1tGyIojpc/aGLMbCHQdngXniTba+f+BNu+Nqux281LbuaJE0/w0pF/ZpVihrue4uhv61mbEeXaYWDOjXD4+1JqWOT974uNUfMo1T3VXOy6yMWui5R3l1PdU82YZYyNSRv5bsl32ZC4ASEEyth+hFAo3nYrP41bhRACjQCN9b/jFoWO/lFa+0Zo7xul1ThCeWsfhypHGDSZ+fa+i9y6Ion7i9NZnxk1rcaNmS08daSWnx6oJkQfwJP3reKOVck+8c7xa2EH2Wd/q+4tmgaanAcLZF4HpU9B82lIK7Z7F7NFoaqtn4c3OrCINY9ROWYkOyjeJ5Of29K28aeqP3Gi9YRdbbo8QFU40nSEDUkbCeoPmvCM8ZYTlfXcKoyYU72Tx7lMxmaZgXnmGceFXVHg/Z9BfAEs3X5Nl2N7vwToeugcSPGqGE5o2C1xQKvXPXaYfoA6W2E3W8wcaTxCcWIxy6IjMFsULrUPUJji+VVIWaORpIDN9PAme2v3umxQdbr9NH2mPralbWMsIpY/HKunrNHI2ql5pCvuliEiLWdmHT4LDQzllqhCXm87zqM7HsekT6Gq7SJ3rHIxLSl5tfSKurTfJ4X9txd+y96avdT01jCuSKOycF04BTEFPFTwEDdl3DTDnVFYrb1LtmyTXQM3uNjcx/Mn6nn5gyZePt1EdnwY9xen85E1KTT1DvPon8u40NzHLSsS+dbthcQZfDc17tceO8D6hPWAC74x4FKf/UrXIKPjFseKmI5KKgK15IW7PsU1GxuSNhCsDXY4hZqbaEATXE/XaIt0j0wsnPCM8ZaqCqkYiMu4hoqYqQghd+1X3pFRfPaoPQTtF2DT531m0+uI0MBQovRRmANkH7jViwNUt33YXcCmR3fWZ/+g/QO6Rrq4MfPGifett8qYc01GViVnsjFpI3tr9s6eWDaF12pfQ6fRsTl584Tv0PuXOqffadntoAl0fohqsXBXw0WGNRrejIyl9LK0I3DZ+EujkRLLSwdkzqwXHGo4xA9P/pAgbRB/V/h3/GTbT3jjI2/w7kff5Vc3/oqvrP2KfcvdtotgSHK7qIPMgfj2HYUcf/wG/v3uIgxBWr6z7yLF332bO372Hm19o/zioTX814NrfVrUYR4U9qzILKL0Ua4doIbGSqfEK46HJCYVMfZ7eJ0N79OlDSA/0Tdj7voAPZuTN3Ow4aDdA5QTHX8jNP3X6EUkN6TfIPX4A20w0OHVz1UUhc562ZcNiPHNh5RLrLofEI7Hv4/+HELjYYXvLp1nIyUshWGLVcvuxZBSQ38D0UHRdA/IDyOPDMCuwqAzkBqW6lTy+OaVNwnWBlOSUkJmjDV9ywtlTEf/KC3GEVakRHBb1m00DTTxQZtzNVnHUAd/qfkLd+bcKT80Q3UUJIVPBGJMEBItC+6Fl+2GtE9QsZcVLeUkBBo43naS45e70Gs1rEh140okZxcMdcmrdA/pGu7im+9/k7yoPP77pv/mf635X+zM2ElKWIrztkf7BXn16QUhOi33rkvj5c9t4a9fKuGB4nQe3pTBga9s5ebCJOdP4AF+L+xCCNYmrOVU2ynXHpC5RUruHAQsV7TKcI0cBxKviibpUJeb6ruw2W1p22gfaqe8u3zitjHLGE+UPsGjRx4lhHSSBh+T06bWCVTZZ/ec+u4hwoasMtGoOSzsEamwdBuceXbmL3V7hbxsLv57mU40B6QaUukZkxObHkXkWWnobyDdkE5L7wgBGkG8j3ZQ+dH5VPY43rGbLWYO1B2gJKWEkMAQtAEachLCvNqxn2+S1gRFqXIzEaINYW+t86ze35f/HrNi5uPLPz5x2+asGE7V9zAydtWOecXd0Nc0I/JxAosFDn4PEZPDyuSNnO04y/HabtakR7mny87aAULjsTpGURS+dfRb9Jv6+V7J99wbKjKPSXOyBN9dES9LCudfbl/ON29b7vRQ1Rv8XthB9tmbBppoHnBhZD1jC5gGHPrGVLb2kRnjOFyjwvpLlhfjO7XG1tStCMREO6ZzuJNPv/lp/lD+Bx5a9hC7or7J5dYAuaNPsNngeteOee9SFxmiFXNwDAT5IAzbHVY/BMb6mS2xY/8F2iBY96k5W4q0720FLF7ZCtikjs29wyQY9G5P+jkiLzqPur46BscG7X5/ahvGRn5iuFeFvazRiBCwPDmckMAQdmXs4s0rbzI87vj16TP18ULlC9yUedM0p8It2bGYxi2cqrtqViNvNwSGwHkH7ZgLL0FHOWz7GkVxK2kebKaio5ENS91saYREy0xZDwv7K5de4WDDQb605kvkRLk5t9JVI0PY4+dHKpI7zI/C7o6e3Tbw4sBeoLK133F/3WKhcqSTFE0Q4TrfFcPooGhWxa/iUMMhTref5t6991LeXc4TJU/wj8X/SF5iFAOj4zT1DkvZliHJ6z77ezWd5AR2oonJ8tHfwg3ybwV9xPQ82oEO2Z5Zef81l6ZNJdWQilkZJyp8iCYPd+wms4m2wTZZ2I3DHrs62sMWcVfVY9/75M0rbxIUEERJSsnEbfmJBjr6Rx3bxzrhXFMv2XFhE8Mtt2fdzuDYIAfrHU9Jv1D5AoNjg3yqcPqH8vol0Wg1gvdrruqz60Jlcb/wysyrZ4sZDn1ftk2X38XKuJUAaIIa2LDEg/dGzi7Zihlod+thDf0NfL/0+xQnFvNwwcPu/9x2q/+5D3fsc8W8KOw5UTlE6CNc840Ji4fYXLuFfcg0Tl33kOPC3nOZCq0gL9SJ+sYDtqVto7y7nE++8UmCtcH84ZY/cMvSWwBYnSbjuSaGPaze7J5itii8d6mTLG07InoO2zA2AoNhxUek7HFEXvZz8tdgHoWNn5vTpdgkj9ER/R7v2BsHGlFQSDWk0mIc8YnU0YYt4s5en32iDZMq2zA2bOdDnurZyxqN0/rY6xLXkRSa5NBKd2R8hN9f/D3XpVw3I5IvTK9lZVok712yE3RTeLdM16q56gPj3J+gq1rq3DUalsUsQ0MAgaENrE53HFXnEFti0SXXIzLNFjOPv/s4GqHhX7f8q0dGf7RdlMlnsZ7FFPqTeVHYNULD2vi1riljQO7a64/NCLmutoZrONKwDzWdpC5QS77V1tSX3JB+A1qNlutSruO5Pc9Nc+hbnhxOYngQB8qt4QOJhdBZKQd5PKCssZehoSGixtqlj4s/WPWQnCy98DKMjciBpJybIG6OpJdWbJLHsLA+j3vsjf0ypDk1TA4nuZO+44yEkAQi9ZF2lTG2NsxNmTdNu922MfFkArWtb4T2/lGKpkglNULDnqV7ONpylPahmbveVy+9SvdI94zduo3NWTGUNfbSN3LVzjx7JwRFTm/HmMflbj1hBeTfBkiBQaA5DUNEk2dy1MQiCEtwqx3zmwu/4XT7ab6+4eskhXl4QNl+EWKypbPlAsMvhd1iRz2yLnEdjQONtA7OEk1lI2MLmPqhdXqfvWIiXMN+m6W68V0UIcjz4cHpxJLCMzh4z0H+Y8d/zGjzCCHYWRDPkapOeQiVUAiWcY9TYw5VdpCuaUeg+K+wp6yBuHxpDHbuBRjqlBLHOSYxNBGN0BCo7/FYFWOTOoZqEjCNW3y6YxdCkBedZ3fHbq8NAxBn0BMbpvNIGVPWKK+grlae3JZ1GxbFwmu1r027fdwyzm8u/IaVcStZm2A/qnFTVgwWBUprr0pP0uqg4HaoeG3SQ/3sczKDd/vXpVwRGBgdZ8CYgimgzqWg6xkIAdm7ZKi92fnjy7vK+fmZn3Njxo3sWbrH/Z9no+3CgmzDgJ8Ke1PPzF9Az/rs02WPzsI1Kjtlz+xa7NgBIoMiHcqndhUkMjxmlr1KW46oh332w1UdbI+zRu7NpSJmKjZNe2MpHPye/Dst2TrnywjUBJIUmoSi7aZ/ZBzjkH211Gw09DcQog1heEQWdF/22EH22W2TjTYctWFs5CUaPDpAPdfYi0ZAQdL0wr4kYglFcUX8peYv02S5b12Rw4GfKvyUw/euVLJoZsoeQbZjTANQ/aZ0Xj3y73KwKG/3xF1O1fUwPpyOGZPDswan5OySbb/G2a/qR82jPPbOY0Tpo/jGxm94PsU52g+9dQvy4BT8VNiNw2MzDoZyo3IxBBpckz0aEuUl0lWm/5Wt/eQmhNkP11AUKoZaMKAlKfTaaEdnY+PSaEJ1Aey/2A7RWVI94kGfvXvQxNnGXq6LthZ2f+3YQQaNiwDob4ZNX7jmA0mOSAlLYRQ5F1DXbV99MhsT5l9G+Z70JmDDHnnReZgsJq4Yr0zcZk8NM5X8xHCq2voxW9wzlyprMpKbYCBYN7PlcfvS27nUe2lClqsoCr8+/2uyIrImAzXsEBQYwLrMqJkHqCA3WWGJcljpzB+gtx62Pz7tvXC8tgtGpKVBWYeH2blZ2+V7zUk75qmyp6gx1vCdLd8hMsiDfr6Ndqt0Wd2xu44C/PlU47TbAjQBFMYWcq7DRX13xhYZ0zVlIm1WRUx/K5XCTH5wgk+8GNxFrw3g+rw4DpS3YREB0hzLAy37O9UdKAoUBHeCPtyjiTifYUiQO7PwFFh+bT2qZyPVkIrRqmW/0uV+rFp9X/2E1BF8M3U6lfyomd7stjbM1hT7Vzl5iQZGxizUdbn+QaUoCucajaxwYEVw85KbCdQEsrdGatrfbXqXqp4qPrnik04PFzdnxVLR2k/n1UodTYA0Bqt+Cw7/QEoTs3dOu8vxy90sT8gkNjjW88IeFAHpm+ScxCzsr9vPluQtbEnxst3aZlXEeDmc5C/8UthDdVqeLa3HctVupCiuiOreaobGXPjlzCyB0b4JI/6O/lG6Bk3kOeivm5tPU6ULJC/GuzxJb9hVkEBH/yhlTcZJZYybdp+HKzuIDtURa2qG6CV+2yVPcOcv4DOHZb/VT6SEpdBr6gZhoq7TvR272WKmaaCJtPA0WozD6LUaor2wS7VHZkQmOo1u4gDVWRsGJgUA7ihjmo0jdA2aKHIw2Rmhj2Bb2jZev/w6Y5Yxfn3+1ySGJrJ7yW6795/KZmtQ+rFaO+2YFXdLvXd/s+ytT3lPNvYMcbahl01LYymKLeJsh+u5xTPI2Sl/3x1ENHYMdXDZeNntDGO7tF8EXRhEOvCcmuf4pbBHh+qo6xrivasu7YriirAoFi50XXD+JJk23xgpe3QWrlHX8B4jGg35KZs8X7iXbM+LJ0AjOHCxTfakh7uhv8X5A61YLApHqjsoyY5BtJ6Tsk9/ozdI72w/khomlTHx0YNcdmOHC9A+1M6YZcyqYZdSR19f0Wk1WnKiciZCN5y1YQBy4g1ohHvKmHONvQCsSHXcgrht6W10j3TzX2f+i1Ntp/jE8k8QqHFuhrciJQKDXmtf9piyViZlZVw3w/jtx/ur0GgEH9uUQVFcEfX99Q6DaZwyIXs8YPfbNlXd+qT1nj3/VNouyqtqzbwQDrqNV6sWQtwjhLgghLAIIda5+riI4ECiQ3U8c6x+2u1FsTIFyKXLtfBk2V+29tltihhHrZjK9jMAyY7fLAAAHg1JREFU5Mf7Pn/TVSJDdKzLiGL/xTa5Ywe3+uwXmvvoHDBxW0o/DLTKqxYVUgxSyx4XNUCdm60YmyJmMmDDt/11G/nR+VR2V6IoitM2DECwLoDMmFC3lDFljUa0GjGrZfV1qdcRpY/i6XNPE6mP5M7sO116bm2Aho1ZMey/2Ipx+KoDaiHgk2/A/c9N261fbO7j5dNN/N3mTJIjgycGlc51eminEV8g234O+uylraUYAg0TrS+PURSfeMT4E28/js4DdwFH3HmQEHDP2lT2l7fRNiUJPDIokozwDNf7cBlbZGG3mKm0hmvEOgrXGKhHi/Dag91bdhUkUNnWT4POug43+uyHKqUGeSPWD4OlPgzFXsDYduxhoX1u9aRhemFv9kEkniPyo/PpHe2lebDZaRtm4jFJBrdaMeeajOQlGmbVigdqAicG5x5Y9oDTNUzlizuy6R408f2/ls/8ZmjsDGuLf3+zAoNey+e2ZQNQEFNAgAjgjHWT5TZCSHVMzSG72ccnWk+wNmEtARovQ2f6W2G4BxIWpiIGvCzsiqKUK4rikRj7/uJ0zBaFP56YnndaFFtEWWeZ6wHXI0Zou0DFbOEawz1UWobJ1kX5xIPdG3YVyAi7/bUjEJHu1o79cFUHRakRhDW9B1GZ8o8K0UHRBGuD0ep76Bww0X/1IM0sNPQ3oNVoidXH094/4vODUxs2b/Zny5912oaxkZcQTl33EEMm59ptRVEoazQ67K9P5cFlD3Jz5s08kP+A84VPoSg1kk+XLOW50oaZkXlX8X5NJ4cqO/j89mwiQuTvXEhgCLlRuZR1eniACrIdY+qfkb3bOthKfX896xN90YZZ2AenMIc9diHEZ4QQJ4UQJzs6OsiMDaUkJ5bnS+unSbpWxK2gc7iTlkEXes9Wf3bL5XeoaptFEdNSRoVOR15kti/+Kl6RERNKTnyYbMckrnBZy24cGuOD+h625URJ860l6m7dhhCClLAUxjWy2LjTjmnobyAlLIXOwXEsim/seu2RG5WLQPB8xfNO2zA28pMMKApUtQ04vW9jzzDG4TFWpDiX+KUZ0vjB9T8gQu9+kMeXd+aSERPCYy+VzXR8tKIoCk/8tYLkiCA+vjlz2veK4oo433kes6f+6ku2Si/4q9oxtv56cZL9AB63mPCIWcQ7diHEASHEeTt/7nDnBymK8pSiKOsURVkXFycP2x7ckE6zcWSixQDyHx5w7VM9Mg0iMxiuPjJruEZn4zHpwe6LQxUfsKsggdIr3YzELIOuSzDmfGLy3UudWBTYHdMq1UBqG2YaqWGp9JttkkfX2zGX+y5Lj5hrJHW0ERIYQkZ4BiaLyaU2DEwKASpanPfZbROnruzYvSFYF8D37lzBla4hfnLA/rDR6+daOdto5Mu7cme0hVbGrWRwbJAaY41nC9AbZJJX9XTZY2lrKRH6iGlWHh7TdlFq8/0pJfYSp4VdUZSdiqIU2vnzqrc//IZlCcQb9DxzfPIQNTcqF32A3vU+e2YJgU1HEVgctmIqWqzh1Yn2R6bnmp0FCZgtCmXjaaBYpLTKCYcq2wkP0pI7aA1MUHfs00g1pNI50gIoLu/Y24faqe6pZl3COum8Cdesxw6ThmCutGEA0qJCCNEFuDSBWtbUiy5AY41ivLZszo7lvnVpPP3O5QnvdxtjZgs/eLOCvAQDd62ZabZnO0D1WM8O0qO9oxwGJ1V1J1pPsC5hnWdmX1fTfmFB79bBzyZggQEaPro+jYOV7TT2yF/GQE0gy2OWu1HYr0NnMrIr4ANy4h0Uduvu4GrnOn+xKjWS2DA9r7VbLUyd9NkVReFwVQcluXEEXD4sDZZCY+dgpQuH/Oh8hseHiYvp4LKLWvZ3GqWf/NbUrbRYY/V86RNzNcWJxUQHRbvUhgHQaAS5CYYJxddsnGs0sizJgE47N7/SX79lGdGhOh79cxlj5snAledL67nSNcQ/7s4jwM4EeJohjUh9pHeFPdUqwGuSm5ymgSaaBpp80183j0NH1YKdOLXhrdzxTiFEI7AJeE0I8aa7z3FfcToCeL508hC1KK6I8q5yTOaZJ98zKLiDel02Pw78JcF9tTO/bxqictxISkCoTz3YvUGjEexcFs9Ll7UourDJwxoHlLf0094/yo4sAzSUqm0YO2xN3YpGaAiNrnRZGXO48TBJoUnkRObQ0juMQa/FEHTtDtfvyb2HA/cccEuJssyqjJlNTGCxKJxrMroXOeclESGBfOeO5Vxs6eNX78jfu4HRcX76djXFS6LZnmc/vFsIQVGcl4NKSSsBAc2ysJe2lALyg9Nrumuk/fQC9Yix4a0q5mVFUVIVRdEripKgKMpNzh81nZTIYHbkx/P8iYaJT/6iuCJMFpPTEGAAdCE8qv1HFI0Wnrt/0h/cRtsFKgIDyQufXxNkO5cl0D9qoS881+kB6uEq6YOyI6RWvumWbrv2C1xgRAVFsTZhLaOBZ12yFRg1j3Ks5ZhMvxKCZuO1kzraEEK4NAw0lbwEAz1DY7T3O7Z4ruseon9knCIXDk59yc2FSdy8PJEnD1RT2zHA0+/U0jlg4rHd+bMOea2MW0mtsZY+k4e5rnoDxOVN7NhPtJ4gOiiabF+II9oWbrjGVObFWNWDGzLoHBiVShGmDCq5cIA6ZBrneE8obxY8Ad218NJnpmVxDjVbPdjjfRNe7Su2ZMcSFKihkgzZipklFPhQZTvLksKJan0fNFrpmaEygx1pOxhQGukcaWJwdHaJ4MnWkwyPD3N9qrz6ae4dJukaHZx6Q36SvMqcrc9eNjFxOnc7dhvfvmM5QVoNX3nhLL86UsvuwkRWp0fN+hibQOJ8hxcpYilroekUisVCaWsp6xLW+WZiuO3Cgg3XmMq8KOxbc+NIiQzmmeN1ACSEJhAfEu9SH67KGq4Rlr8Ndj8BVW/AwX+b+H510zHpwe4LGZQPCdYFUJITx6u9WTBqhKq/2r1f/8gYp+p62JYXB5cPS5Mlvf2g7g87O9J3AKA1XHCqjDnceJiggKCJvqxMTrq2O3ZPcEUZc67RiF6rISd+7t8X8eFBPH7rMs409DIybuGrNzkviIUxhQiEd+2Y5NUw1ElDcyltQ22+acOANVwja0GGa0xlXhT2AI3g/uI03rvURW2H1OyujFvpUmH/9buX0Wk1rEmPgvWfhtUPwzs/lFmMMNHOyfej+Zcjdi1L4PmBlZjCUuH9n9m9z3uXuhi3KOzI0EHzGbUNMwvJYcksMeQSaLg4qzJGURSONB5hY9JGgrRBjIyZ6R40kXwND049JTJER2J4EPsvtjFssq/9LmsyUpAc7rMAbne5d10a96xN5Su7clka5/zDJUwXRnZUNmc7vSjsKWsAKL20D/CRPwzIHfsCHkyyMS8KO8C969PQagQ/PygVLEWxRTQONNI17HjC7WhNF3vPNvM/r88iPjxIjhzf+iNILYZXPgvNZ6gY6cAgAv3iwe6M7fnxWEQAR+Pugfr3oWmmF/3hqg7C9FpWm88BiipzdMLOjBvQBNdzsb3R4X1qemtoGmhia5pUpzTPgdTRG760M4dT9T189KmjtPdPj/8zWxQuNBmnReHNNUIIfnDPSj6/3fUed1FsEec6zmFRHLcgZyWhEAJ0lLadIjY4liXhPgicsYVr2HycFjDzprDHG4J45PqlvPhBIwcutk304RwZBo2ZLXzzL+dJjQrmc9uyJr+h1cN9v5f+zb+7g8rAAPJDk/3iwe6MOIOeNelRPNm1EYvOMGPXrigKhyvb2ZIdg7buCASGyt6iikN2L92FEAqn2meGnds40iStjWyywwmp4zzssYO033jq4XVUtQ1w58/fn+Yfc7lzgEGTeVZHx/nIyriV9Jn6uNJ3xbMn0OpREpZzYriF9YnrffP73W71y1/gB6cwjwo7wJduyGVZUjhfe+kciUFZaIXWYTvmd0frqGob4Bt7CmaaHhkS4b5nMI8NSw/22Pn7CfyRNamcbjfzq6GtmC+8wi//cpDSy92Mmy1cah+g2TjCtrx4qD0kJ+786Hu+EMiOzCbQEsfl4eMO73O44TDLopeRECp9eyZ27POwx25jV0ECLzyyiTGzhbv/7/scsSql5mri1Nf4YlDpckI+ncJMcYLLxrKz077wPWJszKvCrtNq+PG9K+kbHuM7ey+RG51r9x++vX+EJ/dXcX1uHDdaTbVmkLqWut3flh7syRuu8co954EN6ez74nUEbfkcCgLtiae495dHWf2d/XzxudMAbE8ak9YDS7f5da0LASEEKbr1DGoq6DfNVJIYR42c6ThDSeqk5bFtx544D3vsU1mRGsErn99CSlQwf/f/TvDM8TrKGo0EBwaQ5UJvez6RGZGJQWfwqrCfDJV/5/U6H+UBtF2QV8ULNFxjKvOqsAMsSwrny7ty+ev5VsJYyvmumYZB33+9gtFxC/9y+/JZL8GqYjIByI+Z35/AhSkRfHz3dWhXfIRPBh/hV/fmcEthEr1DYxQviSaxy7r7VAeTXGJl9BYQZt6uOzzje+82vYtFsUzIHEHu2GPDdLPa3c4XkiOD+fNnN1OSE8vjL5/nhZMNFKaE253ynM9ohMbrRKVScx/x4+Ok99pPVHKbBR6uMZV5+Tf4zNalrM2IorTCwODYILXGyYnSE1e6eel0E58uWcKS2NBZn+dsx1n0AXq/e7C7zOYvIEwD7Bp+gyfuLuLoYzt44ZFNsg0TErPgp+HmiuLk1VjGw3i9dmY+5uHGw0QHRVM4pT3XPE+ljo4I02t5+mPr+NimDIZMZlYusP66jaK4Ii71XmJwzP0AckVRONFbSbHJjGg+7f1ibOEai6C/DvO0sAdoBD+6ZyXjQ2kAE5/q42YL//zqBZIjgvjCDucn8Mdbj7MqfpXfPdhdJmmlTEU6/gswj8mrEUWB2sNSDbMIdhJzwdLYcMb7C/ig/eg0W4pxyzjvNr1LSUrJNLMomZw0v9swV6MN0PCt25fz7Kc38MUdOf5ejkfYojA9SVSq6a2he6Sb4qCkCWsBr5gI15i/53HuMG8rRWZsKI/tvA5lPISXLx4F4NnSespb+vinPQWE6LSzPr5ruIvqnmrfBNvOJZu/CH1NEzp8OqtkDJ7ahnGZ9JgQxvuXM2oZ5ljLsYnbz3acpd/Uz/Vpk6+loig09w7PW6njbAgh2JwdOxFksdBYFbcKrdByrPmY8ztfRWmr9IdZn7BWBlzbSVRyi0V0cArzuLADPLwpkwhNFmfbz3KqrocfvlnJluwYdhcmOn2s7R9+Q+L8PTi1S/YuGVJ99D+tu/VD8nZVv+4y4UGBRLAMLUH8rf5vE7cfbjyMVqNlU9KkJUPfyDiDJvM182FXcUyYLozVCat5p+kdtx97ovUEyaHJpGaUgNnkcmCNQ9qs1tkL3K7Xxrwu7EII/kfBZoSunY8+/TeGTGa+5eTA1MbxluMYAg0UzPOD0xloNLDp89ByFq68K9swkRkQ7YMBjA8RS2IjCDEXcrDh4MTh+5GGI6xNWEuYblJB0mKUUseF1GNfTJSklFDVU0XrYKvLj7EoFk60nZB2EMlyAtXrdkz7wg/XmMq8LuwAW9LWgFCwBDbwyeuWkO3Ac/1qjrccZ13iOu+Dbf1B0UchJBbe+6ks7mobxm0yYkIYNRbQPdJNWWcZDf0N1BhrpqlhAFp6pdRR3bH7h5IUKTt9t8nxQNnVVPdUYxw1yhi8yHT5u9LkZWFvO79oDk5hARR2m3rhE9s1POqCwRBI4/3GgUY2JC2wNoyNwCAo/nu4tF8ahKltGLfJjAmls2MpgZpA3q57myONctr06sLebJzfdgKLnazILJJCkyZCT1zBdm5SnFgsbURS1nhX2MdHZbjGIumvwwIo7OG6cJZGLKXNVOWyydHxFqn7XnD99ams/zRorbtItbC7TWZsKFiCKIxex98a/saRxiNkhmeSHp4+7X7NvcMEaATxBnXH7g+EEJSklHCs5ZhrwTrAG5ffID86n8RQ61lb8hrorIRR56HfdmkolTkHGZs9e/w8ZN4XdpCyqLKOsllTZKZyrOUYscGxZEVmOb/zfCU0FjZ+DnJvhjAfTdZ9iMiMkSlFWaEbaOhv4Gjz0Rm7dZCtmASDfsEN+CwmSlJLGBof4oN257vuWmMt57vOs2fpnskbU9bK7OAWD4edag9KD/bM6zx7/DxkwRT2ntEeGvsdO/bZUBSF0pZSihOL56Xxl1vs/CY88Ed/r2JBkhEth9cilFUIBAoKW1NnZo02G4dJUtswfqU4sRidRudSO2ZfzT40QsOtS2+dvNFq4WvPHdUlag/JHNWgheW3MxsLo7BbE5WOtTrXu9b01tA10rXw9OsqPiUiJJCokEDae3Wsil9FWKCU1k3FODzG2QYjeYmuHcirXBtCAkNYn7jeqezRoljYV7uPTcmbiA2eEuYeGgsR6Z4pY4Z7oPn0ovNhWhCFPTcql+zIbF6ofMFpO+Z4q+yvF8+zxCSVuSczNpS6rkEe3/A4P9n+kxl5oy+eamR4zMwDxekOnkFlrihJLeGy8TIN/Q0O73Oq7RQtgy3ctvS2md9MWe3ZAerlI7KNs3S7+4+dxyyIwi6E4MFlD1LRXcGpttkvt461HCM1LJWUsJQ5Wp3KfCUzJpQrnUPkRefNuIKzWBT+cKyO1emRFPoxpEJF4orscW/NXkK0IRMRiNNIWStDMgY73fvBtYdAFyZbMYuIBVHYAW5deisR+gierXjW4X3GLeOcbD25cGWOKj4lIyaEZuMwI2MzI+XevdRJbecgH9+UOfcLU5lBeng6GeEZDvvsI+MjvFX3FrsydhGstXMmMjGo5KYhWM1BeWi6UPykXGTBFPZgbTAfyfkIb9e/TfOAfZvO8q5yBsYG1P66CiB37IoCjT0z809/d7SOmFAdu1c4t6dQmRtKUkoobS1lZHxkxvcONhxkcGyQ27LstGEAklcBwr12TM8V6Lm86NowsIAKO8BH8z6KQPB85fN2v6/211Wmkmm1db7SOb2wN3QP8XZFG/cXp6PXLsDJ5EVKSUoJo+ZRTrSemPG9vTV7SQhJkDYC9tAbIC7PvQNUmw9TllrY/UpSWBI70nfwYtWLDI3N3IUdazlGblQu0UGLw+9BxTtsWvYrXdP9vp85Xo9AplepzB/WJq4lWBs8Qx3TOdzJ+83vs2fpnml2yzNIXiMljy7Ou1BzEAxJ0nRvkbGgCjvAg8sepM/Ux77afdNuHzWPcqb9jBwzVlEBIkN0RAQHTivsI2Nm/niinhsLElUbgXmGPkDPhsQNvNP4zjT1218v/xWzYnbchrGRsgYGO8DofN4FiwUuH5ZtmIU+72KHBVfY18SvYVn0Mp4tf3baP/7Z9rOMmkfV/rrKNKTkcfLqbl9ZCz1DY3xs08LPtVyMlKSW0DjQyJW+KxO37a3Zy7LoZc4nyd1xemw9KzXsi7ANA14WdiHED4QQFUKIMiHEy0KIa57RJYTggWUPUGOsmRaicKzlGAEigLUJa6/1ElQWEJkxIdN27L87eoXs+DA2ZcX4b1EqDrkuRY7129Qxl3ouUd5d7ny3DpBYCJpA1yZQaw7K/y5SHyZvd+z7gUJFUYqAKuAx75fknN1LdhMdFM2z5ZPSx+OtxymMLZzmta2ikhETSlPPMKZxC2caeilrNPKxTRkL325ikZIclkx2ZDZHmqQb597avQSIAHYv2e38wVq9LO6NJ53ft/aQzBA2JHi34HmKV4VdUZS3FEUZt355DEj1fknO0QfouTv3bg43Hqahr4EB0wAXOi+o+nWVGWTGhGBRoKFniN+9f4UwvZa71szJ21TFQ0pSSjjVdop+Uz+v1b7G5uTN0y0EZiPvFqh7T06UOmJsGOqPLdo2DPi2x/5J4K8+fL5ZuS/vPgJEAM9WPMvJtpOYFfPCtulVuSbYJI8f1PWwr6yFu9akEKafPS9Xxb+UpJYwbhnnZ6d/RttQm2ttGBubvwhRmbDvK9Jn3R71R6VN79JtPljt/MRpYRdCHBBCnLfz544p93kcGAeemeV5PiOEOCmEONnR0eH1wuND4tmVuYtXLr3CwYaD6AP0rIxf6fXzqiwuMmNkYX/yQDUms0U9NF0ArIpfRWhgKM9VPEdYYBjb09zYWQcGwy0/hK5qeP8/7N+n5iAE6BaV//rVOC3siqLsVBSl0M6fVwGEEJ8A9gAPKrM4dCmK8pSiKOsURVkXF+cbf/GHlj3EwNgAL1e/zOr41egD9D55XpXFQ1RIIIYgLU29w2zOinE5WlHFfwRqAtmcvBkFhV0ZuwjSuhmCkrMLCu6AIz+E7sszv197ENI2gC7UNwueh3irirkZeBS4XVGUmRND15iiuCJWxK5AQVH76yp2EUKwxNqOUXfrCwdbKMrtWbd79gQ3fx80Wnj9q9MHlgY7ofXcos8R9rbH/jPAAOwXQpwRQvzCB2tyi48v/zgCMeEOp6JyNQVJ4aRFB7Nz2eJUQCxG9izdwzO3PMO6RA9dF8OTYfvjMje4/C+Tt9tsBJbacYhcRAhX4+Z8ybp165STJ12QJLlI+1A78SHxPns+lcXFsMnM6LiZyBCdv5eiMpeYx+FX22CwC75QKv1kXv08lO+FRy+DZuH5BAkhTimK4vTTbsFNntpDLeoqsxGsC1CL+oeRAC3seRL6W+Dg92RLpuYQLNm6IIu6OyyKwq6ioqJil9R1sPYTcPz/wsVXoK9xUdr0Xo1a2FVUVBY3O78JwdHw0iPy60U8mGRDLewqKiqLm+AouOm7cigpMh2ilvh7RdccdQRPRUVl8VN0L9S8DQnLF6VN79WohV1FRWXxIwTc9ZS/VzFnqK0YFRUVlUWGWthVVFRUFhlqYVdRUVFZZKiFXUVFRWWRoRZ2FRUVlUWGWthVVFRUFhlqYVdRUVFZZKiFXUVFRWWRoRZ2FRUVlUWGWthVVFRUFhlqYVdRUVFZZKiFXUVFRWWRoRZ2FRUVlUWGWthVVFRUFhlqYVdRUVFZZKiFXUVFRWWRoRZ2FRUVlUWGWthVVFRUFhlqYVdRUVFZZKiFXUVFRWWRoRZ2FRUVlUWGWthVVFRUFhleFXYhxHeEEGVCiDNCiLeEEMm+WpiKioqKimd4u2P/gaIoRYqirAL2Af/sgzWpqKioqHiBV4VdUZS+KV+GAop3y1FRUVFR8Ratt08ghPg34GOAEdg+y/0+A3zG+uWAEKLS25/tJbFAp5/XMF9QX4tJ1NdiEvW1mGS+vBYZrtxJKMrsm2whxAEg0c63HlcU5dUp93sMCFIU5ZvurNJfCCFOKoqyzt/rmA+or8Uk6msxifpaTLLQXgunO3ZFUXa6+FzPAK8DC6Kwq6ioqCxWvFXF5Ez58g6gwrvlqKioqKh4i7c99u8LIfIAC1AH/E/vlzRnPOXvBcwj1NdiEvW1mER9LSZZUK+F0x67ioqKisrCQp08VVFRUVlkqIVdRUVFZZGhFnZACPG/hRCKECLW32vxF0KIHwghKqwWES8LISL9vaa5RghxsxCiUghxSQjxNX+vx18IIdKEEAeFEBeFEBeEEF/y95r8jRAiQAhxWgixz99rcYUPfWEXQqQBNwL1/l6Ln9kPFCqKUgRUAY/5eT1zihAiAPg5sBsoAO4XQhT4d1V+Yxz434qiFAAbgc9/iF8LG18Cyv29CFf50Bd24CfAo3zI7RAURXlLUZRx65fHgFR/rscPFAOXFEWpVRTFBDyPlPB+6FAUpUVRlA+s/9+PLGgp/l2V/xBCpAK3Ak/7ey2u8qEu7EKIO4AmRVHO+nst84xPAn/19yLmmBSgYcrXjXyIi5kNIUQmsBo47t+V+JUnkZs/i78X4ipee8XMd2azRAC+jmzDfChwxR5CCPE48lL8mblcm8r8QwgRBrwI/MNVhn8fGoQQe4B2RVFOCSG2+Xs9rrLoC7sjSwQhxApgCXBWCAGy9fCBEKJYUZTWOVzinOHMHkII8QlgD3CD8uEbcGgC0qZ8nWq97UOJECIQWdSfURTlJX+vx49sAW4XQtwCBAHhQog/KIrykJ/XNSvqgJIVIcQVYJ2iKPPBwW3OEULcDPwYuF5RlA5/r2euEUJokYfGNyAL+gngAUVRLvh1YX5AyJ3Ob4FuRVH+wd/rmS9Yd+z/R1GUPf5eizM+1D12lWn8DDAA+62JWL/w94LmEuvB8ReAN5GHhS98GIu6lS3Aw8AO63vhjHXHqrJAUHfsKioqKosMdceuoqKisshQC7uKiorKIkMt7CoqKiqLDLWwq6ioqCwy1MKuoqKisshQC7uKiorKIkMt7CoqKiqLjP8PSG6xpVWYYxUAAAAASUVORK5CYII=\n",
45 | "text/plain": [
46 | ""
47 | ]
48 | },
49 | "metadata": {},
50 | "output_type": "display_data"
51 | }
52 | ],
53 | "source": [
54 | "import numpy as np\n",
55 | "import matplotlib.pyplot as pl\n",
56 | "\n",
57 | "# Test data\n",
58 | "n = 50\n",
59 | "Xtest = np.linspace(-5, 5, n).reshape(-1,1)\n",
60 | "\n",
61 | "# Define the kernel function\n",
62 | "def kernel(a, b, param):\n",
63 | " sqdist = np.sum(a**2,1).reshape(-1,1) + np.sum(b**2,1) - 2*np.dot(a, b.T)\n",
64 | " return np.exp(-.5 * (1/param) * sqdist)\n",
65 | "\n",
66 | "param = 0.1\n",
67 | "K_ss = kernel(Xtest, Xtest, param)\n",
68 | "\n",
69 | "# Get cholesky decomposition (square root) of the\n",
70 | "# covariance matrix\n",
71 | "L = np.linalg.cholesky(K_ss + 1e-15*np.eye(n))\n",
72 | "# Sample 3 sets of standard normals for our test points,\n",
73 | "# multiply them by the square root of the covariance matrix\n",
74 | "f_prior = np.dot(L, np.random.normal(size=(n,3)))\n",
75 | "\n",
76 | "# Now let's plot the 3 sampled functions.\n",
77 | "pl.plot(Xtest, f_prior)\n",
78 | "pl.axis([-5, 5, -3, 3])\n",
79 | "pl.title('Three samples from the GP prior')\n",
80 | "pl.show()"
81 | ]
82 | },
83 | {
84 | "cell_type": "markdown",
85 | "metadata": {},
86 | "source": [
87 | "\n",
88 | "https://www.robots.ox.ac.uk/~mebden/reports/GPtutorial.pdf"
89 | ]
90 | }
91 | ],
92 | "metadata": {
93 | "kernelspec": {
94 | "display_name": "Python 3",
95 | "language": "python",
96 | "name": "python3"
97 | },
98 | "language_info": {
99 | "codemirror_mode": {
100 | "name": "ipython",
101 | "version": 3
102 | },
103 | "file_extension": ".py",
104 | "mimetype": "text/x-python",
105 | "name": "python",
106 | "nbconvert_exporter": "python",
107 | "pygments_lexer": "ipython3",
108 | "version": "3.6.1"
109 | }
110 | },
111 | "nbformat": 4,
112 | "nbformat_minor": 2
113 | }
114 |
--------------------------------------------------------------------------------
/02_maximum_likelihood.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": 1,
6 | "metadata": {},
7 | "outputs": [
8 | {
9 | "name": "stdout",
10 | "output_type": "stream",
11 | "text": [
12 | "Populating the interactive namespace from numpy and matplotlib\n"
13 | ]
14 | }
15 | ],
16 | "source": [
17 | "%pylab inline\n",
18 | "import numpy as np\n",
19 | "import pandas as pd\n",
20 | "\n",
21 | "from IPython.display import display"
22 | ]
23 | },
24 | {
25 | "cell_type": "markdown",
26 | "metadata": {},
27 | "source": [
28 | "시작전 visualization에 사용될 코드 정의"
29 | ]
30 | },
31 | {
32 | "cell_type": "code",
33 | "execution_count": 2,
34 | "metadata": {},
35 | "outputs": [],
36 | "source": [
37 | "def disply_plot(theta, p, plot=None):\n",
38 | " if plot is None:\n",
39 | " fig, plot = plt.subplots()\n",
40 | " \n",
41 | " if len(theta) <= 30:\n",
42 | " plot.bar(theta, p, align='center', color='skyblue', width=0.05)\n",
43 | " else:\n",
44 | " plot.plot(theta, p)\n",
45 | " ticks = np.arange(0, 1.1, 0.1)\n",
46 | " plot.set_xticks(ticks)\n",
47 | " plot.set_xticklabels(ticks)\n",
48 | " plot.set_xlabel('theta')\n",
49 | " plot.set_ylabel('likelihood')\n",
50 | " plot.grid()"
51 | ]
52 | },
53 | {
54 | "cell_type": "markdown",
55 | "metadata": {},
56 | "source": [
57 | "# Likelihood\n",
58 | "\n",
59 | "Likelihood는 unknown parameters $ \\theta $ 를 추정한는데 사용되는 함수입니다.
\n",
60 | "$ X^n = (x_1, ... x_n) $ 는 joint density $ P(x^n;\\theta) = p(x_1, ..., x_n; \\theta) $ 를 따를때..
\n",
61 | "likelihood function 은 다음과 같이 정의 될 수 있습니다.\n",
62 | "\n",
63 | "$$ L(\\theta) = L(\\theta; x^n) = P(x^n; \\theta) $$\n",
64 | "\n",
65 | "여기서 $ x^n $ 은 parameter로 고정되었으며, $ \\theta $ 는 variable입니다.
\n",
66 | "Likelihood의 몇가지 속성은 다음과 같습니다.\n",
67 | "\n",
68 | "만약 데이터가 **iid** (identically independent distribution) 이라면 **likelihood function은 다음과 같이 정의** 됩니다.
\n",
69 | "\n",
70 | "\n",
71 | "$$ \\begin{align} \n",
72 | "L(\\theta\\ |\\ x) &= \\prod^n_{i=1} p(x_i ; \\theta) \\\\\n",
73 | "&= \\prod^n_{i=1} L(\\theta ; x_i)\n",
74 | "\\end{align}$$"
75 | ]
76 | },
77 | {
78 | "cell_type": "markdown",
79 | "metadata": {},
80 | "source": [
81 | "## Coin Flipping Example\n",
82 | "\n",
83 | "동전을 `independently` 10번 동졌을때 `앞면`이 나올 확률을 계산하려고 합니다.
\n",
84 | "10번을 던졌을때 HHTHHHTTHH 가 나왔을때 Binomial distribution을 적용했을때 다음과 같습니다.\n",
85 | "\n",
86 | "> Bernoulli distribution은 다음과 같으며, $ \\theta $ 는 성공확률이라고 보면 됩니다.
\n",
87 | "> $ f(k; \\theta) = \\theta^k (1-\\theta)^{1-k} \\qquad x \\in \\{0, 1\\} $ \n",
88 | "\n",
89 | "$$ \\begin{align} \n",
90 | "P(data\\ |\\ \\theta) &= P(HHTHHHTTHH\\ |\\ \\theta) \\\\\n",
91 | "&= \\theta^{7} (1-\\theta)^{10-7}\n",
92 | "\\end{align} $$\n",
93 | "\n",
94 | "이경우 variable data $ x $ 는 이미 수집되어 있으므로 `고정`되어 있다고 봅니다.
\n",
95 | "Parameters $ \\theta $ 는 원래는 고정된 값이지만 아직 모르는 값입니다.\n",
96 | "\n",
97 | "**Probability density function**와 **Likelihood function** 의 차이점은 고정되어 있는 parameters를 **서로 뒤바꿔** 준다는 것입니다.
\n",
98 | "예를 들어서 $ f(x\\ |\\ \\theta) $ 였다면 likelihood는 $ f(\\theta\\ |\\ x) $ 로 바꿔줍니다.
\n",
99 | "($ x_i $ 는 1이면 head이고 0이면 tail 입니다.)\n",
100 | "\n",
101 | "$$ \\begin{align} \n",
102 | "L(\\theta\\ |\\ data) &= P(\\theta\\ |\\ HHTHHHTTHH) \\\\\n",
103 | "&= \\prod^n_{i=1} f(x_i; \\theta) \\\\ \n",
104 | "&= \\theta^{x_1} (1-\\theta)^{1-x_1} \\cdot \\theta^{x_2} (1-\\theta)^{1-x_2} \\cdot ... \\cdot \\theta^{x_n} \\\\\n",
105 | "&= \\theta^{7} (1-\\theta)^{10-7}\n",
106 | "\\end{align} $$\n",
107 | "\n",
108 | "Probability distribution function $ P(data\\ |\\ \\theta) $ 와 Likelihoood function 의 결과는 동일합니다.
\n",
109 | "하지만 그 `의미`는 매우 다릅니다.\n",
110 | "\n",
111 | "$$ L(\\theta\\ |\\ data) = P(data\\ |\\ \\theta) $$\n",
112 | "\n",
113 | "### Code\n",
114 | "\n",
115 | "아래의 코드는 서로다른 $ \\theta $ 에 따른 likelihood를 계산한 것입니다."
116 | ]
117 | },
118 | {
119 | "cell_type": "code",
120 | "execution_count": 3,
121 | "metadata": {},
122 | "outputs": [
123 | {
124 | "data": {
125 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcUAAAF6CAYAAABoXUMkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzs3Xl8FeXZ//HPRSDsi7JvEhAUAyoKgtQNXCpqFWtRoWqxomgrLtUu2PZRax+ttj+rtWotiuLSGtFuqKh1AW1d2GQREDAsCsi+hz3h+v0xN32OaZJzIDmZk+T7fr3OK7PcM/M9J8uVmblnxtwdERERgVpxBxAREckUKooiIiKBiqKIiEigoigiIhKoKIqIiAQqiiIiIoGKotQYZnaKmS1MGF9mZmcexHruNLPnwvBhZlZgZllhfLKZXV1xqUvNcKWZ/fsglz3SzGaZ2TYzu7Gis6WY4adm9sRBLlvfzF42sy1m9mLxzyJ8P7pUXFqpSWrHHUCkopnZMuBqd38rcbq7/ws4siK35e5fAI0qcp2V4MfAJHfvVdJMM5sHdEqYVA94zd3Pr6gA7n5PORYfArQGmrt7oZldWWzdVe37IRlEe4oiNU8nYF5pM929h7s3CsWlMbAceLGywqWgE7DI3QvjDiLVj4qi1BhmNsDMVpQy7ygzW2pmw8J4OzP7i5mtC9NLPMxoZjlm5maWeNSlk5m9Hw5P/tPMWiS0v8DM5pnZ5nCo9ahiGSaHefPM7IKEec3NbIKZbTWzqcDhSd5ridsxs3eAgcDD4TDjEUk+tlOBFsBfytjW4HA4dquZLTazQWF6u5B5o5nlm9k1CcskHoLe/xkON7MvzGy9mf2slG39ArgduDTkH1FCGzezrmF4nJk9ZmZvhu/Hu2bWKcwzM3vAzNaG7J+YWc8kn4dUcyqKUuOZ2fHAG8AN7v68mdUCXgZmA+2BM4CbzezsFFf5beC7QCsgG/hh2M4RwPPAzUBLYCLwspllm1mdsM1/huVuAP5kZvsP9z4C7ALaAleFV2nvp9TtuPvpwL+AUWFvcFGS9zIc+Iu7by9lW32BZ4AfAc2IiuiyMDsPWAG0IzrkeY+ZnV7Gtk4mOrx9BnB74j8M+7n7HcA9wAsh/9gk+QEuA35JVNxnAX8K078e8h4BNAUuATaksD6pxlQUpaY7BZgAfMfdXwnTTgBauvtd7r7H3ZcAjwNDU1znU+6+yN13AuOB/efuLgVedfc33X0v8P+A+sDXgBOJzk3eG7b5DvAKMCx04vkWcLu7b3f3ucDTZWy/rO2kzMwaEBWzcWU0GwE8Gba1z91XuvsCM+sInAT8xN13ufss4AngO2Ws6xfuvtPdZxP9Q3LsgeQtw6vu/p677wZ+BvQP+fYSHR7uDpi7f+ruqypom1JFqShKTXcd8IG7T06Y1gloFw49bjazzcBPiTp3pGJ1wvAO/q8jTjvg8/0z3H0f0fm69mHe8jBtv8/DvJZEneKWF5tXmrK2cyAuAjYC75bRpiOwuJQMG919W8K0/e+nNKV9buX1n8/N3QuI3lO78I/Hw0R74WvNbIyZNamgbUoVpaIoNd11wGFm9kDCtOXAUndvlvBq7O7nlnNbX5LQq9PMjKiorAzzOoZDt/sdFuatAwpD28R5B7OdAzEceMbLfpTOcko+v/klcKiZNU6Ytv/9VLb/fG5m1gg4lCgf7v6Qu/cGcokOo/4ohnySQVQUpbqqY2b1El6lXX60DRgEnGpm94ZpU4FtZvaTcE1clpn1NLMTyplpPHCemZ0RziHeCuwGPgCmEO0d/djM6pjZAOB8IM/di4C/AneaWQMzyyUqWAeznZSYWQeiDjllHaYFGAt8N2yrlpm1N7Pu7r48bO9X4fM/huhQ63OpZqhA55rZyWaWTXRu8SN3X25mJ5hZv/AZbSc6Z7uvzDVJtaeiKNXVRGBnwuvO0hq6+2bgLOAcM/tlKELfIDoXuBRYT3Q+rGl5Arn7QuBy4PdhnecD54dziHvC+Dlh3qNE5zkXhMVHER1OXE10ju+pg9nOAcS9AvjQ3Us6NJq4ralEnYoeALYQHWrdv5c6DMgh2iv7G3BH8WtHK8mfgTuIDpv2JvpsAJoQnSveRHRodwPwmxjySQYxPWRYRKorMxsHrHD3n8edRaoG7SmKiIgEKooiIiKBDp+KiIgE2lMUEREJavRTMlq0aOE5OTlp3cb27dtp2LBhWrehDMqgDFVr+8pQ+RlmzJix3t1bJm3o7jX21bt3b0+3SZMmpX0byqAMylC1tq8MlZ8BmO4p1AUdPhUREQlUFEVERAIVRRERkUBFUUREJFBRFBERCWr0JRkiUj3dO3N9mfNzdhYmbTP6uBYVGUmqCO0pioiIBCqKIiIigYqiiIhIoKIoIiISqCiKiIgEKooiIiKBiqKIiEigoigiIhKoKIqIiAQqiiIiIkFai6KZDTKzhWaWb2ajS5hf18xeCPOnmFlOwrzbwvSFZnZ2mNbRzCaZ2Xwzm2dmNyW0P9TM3jSzz8LXQ9L53kREpPpJW1E0syzgEeAcIBcYZma5xZqNADa5e1fgAeC+sGwuMBToAQwCHg3rKwRudfdc4ETg+oR1jgbedvduwNthXEREJGXp3FPsC+S7+xJ33wPkAYOLtRkMPB2GXwLOMDML0/Pcfbe7LwXygb7uvsrdPwZw923Ap0D7Etb1NHBhmt6XiIhUU+bu6Vmx2RBgkLtfHcavAPq5+6iENnNDmxVhfDHQD7gT+MjdnwvTxwKvuftLCcvmAO8BPd19q5ltdvdmYZ4R7YE2KyHXSGAkQOvWrXvn5eVV9Fv/ioKCAho1apTWbSiDMijDV63eWVjm/Lp7d7K7Tv0y27Spn96HCNWE70MmZRg4cOAMd++TrF2VfHSUmTUC/gLc7O5bi893dzezEqu9u48BxgD06dPHBwwYkM6oTJ48mXRvQxmUQRm+Kumjo1bPYlmbXmW2GZrmR0fVhO9DVcmQKJ2HT1cCHRPGO4RpJbYxs9pAU2BDWcuaWR2igvgnd/9rQps1ZtY2tGkLrK2wdyIiIjVCOoviNKCbmXU2s2yijjMTirWZAAwPw0OAdzw6njsBGBp6p3YGugFTw2HRscCn7v7bMtY1HPhHhb8jERGp1tJ2+NTdC81sFPAGkAU86e7zzOwuYLq7TyAqcM+aWT6wkahwEtqNB+YT9Ti93t2LzOxk4ArgEzObFTb1U3efCNwLjDezEcDnwCXpem8iIlI9pfWcYihWE4tNuz1heBdwcSnL3g3cXWzavwErpf0G4IxyRhYRkRpMd7QREREJVBRFREQCFUUREZFARVFERCRQURQREQlUFEVERAIVRRERkaBK3vtURDJXsvuOAuTsLCyz3eg033dUpDTaUxQREQlUFEVERAIVRRERkUBFUUREJFBRFBERCVQURUREAl2SISKSBskuTUl2WQro0pQ4aE9RREQkUFEUEREJVBRFREQCFUUREZFARVFERCRQURQREQnSWhTNbJCZLTSzfDMbXcL8umb2Qpg/xcxyEubdFqYvNLOzE6Y/aWZrzWxusXXdaWYrzWxWeJ2bzvcmIiLVT9qKopllAY8A5wC5wDAzyy3WbASwyd27Ag8A94Vlc4GhQA9gEPBoWB/AuDCtJA+4e6/wmliR70dERKq/dO4p9gXy3X2Ju+8B8oDBxdoMBp4Owy8BZ5iZhel57r7b3ZcC+WF9uPt7wMY05hYRkRrK3L30mWbHl7Wwu39cxrJDgEHufnUYvwLo5+6jEtrMDW1WhPHFQD/gTuAjd38uTB8LvObuL4XxHOAVd++ZsK47gSuBrcB04FZ331RCrpHASIDWrVv3zsvLK+stlltBQQGNGjVK6zaUQRkyKcPqnYVJ29Tdu5PddeqXOr9N/fLdbCtZhmTbry4ZkqkJP4/7DRw4cIa790nWLtknfn/4Wg/oA8wGDDiGqPD0L0/ICvYH4JeAh6/3A1cVb+TuY4AxAH369PEBAwakNdTkyZNJ9zaUQRkyKUOyW5cB5KyexbI2vUqdP7SctzdLeou1JNuvLhmSqQk/jweqzMOn7j7Q3QcCq4Dj3b2Pu/cGjgNWJln3SqBjwniHEpb5Txszqw00BTakuGzxrGvcvcjd9wGPEw63ioiIpCrVc4pHuvsn+0fcfS5wVJJlpgHdzKyzmWUTdZyZUKzNBGB4GB4CvOPR8dwJwNDQO7Uz0A2YWtbGzKxtwug3gbmltRURESlJqges55jZE8BzYfwyYE5ZC7h7oZmNAt4AsoAn3X2emd0FTHf3CcBY4FkzyyfqPDM0LDvPzMYD84FC4Hp3LwIws+eBAUALM1sB3OHuY4Ffm1kvosOny4BrU3xvIiIiQOpF8bvA94Cbwvh7ROfwyhQui5hYbNrtCcO7gItLWfZu4O4Spg8rpf0VyfKIiIiUJaWi6O67zOwR4C2iPbGF7r43rclEREQqWUpF0cwGEF1PuIyo92lHMxserhkUERGpFlI9fHo/8HV3XwhgZkcAzwO90xVMRESksqXa+7TO/oII4O6LgDrpiSQiIhKPVPcUp5fQ+3R6eiKJiIjEI9Wi+D3geuDGMP4v4NG0JBIREYlJqr1Pd5vZw8CbqPepiIhUU+p9KiIiEqj3qYiISKDepyIiIoF6n4qIiATqfSoiIhKk3PsU+G14iYiIVEup9j49CbgT6JS4jLt3SU8sERGRypfq4dOxwA+AGUBR+uKIiIjEJ9WiuMXdX0trEhERkZiVWRTN7PgwOMnMfgP8Fdi9f767f5zGbCIiIpUq2Z7i/cXG+yQMO3B6xcYRERGJT5lF0d0HVlYQERGRuCU7fHq5uz9nZreUNN/ddYmGiIhUG8kOnzYMXxunO4iIiEjckh0+/WP4+ovKiSMiIhKfZIdPHyprvrvfWNZ8MxsE/A7IAp5w93uLza8LPEP0tI0NwKXuvizMuw0YQXRd5I3u/kaY/iTwDWCtu/dMWNehwAtADtEjri5x901l5RMREUmU7CkZM5K8SmVmWcAjwDlALjDMzHKLNRsBbHL3rsADwH1h2VxgKNADGAQ8GtYHMC5MK2408La7dwPeDuMiIiIpS3b49OnEcTNr4O47Ulx3XyDf3ZeEZfOAwcD8hDaDiW4fB/AS8LCZWZieF+65utTM8sP6PnT398wsp4TtDQYGhOGngcnAT1LMKiIigrl78kZm/Ylu9dbI3Q8zs2OBa939+2UsMwQY5O5Xh/ErgH7uPiqhzdzQZkUYXwz0IyqUH7n7c2H6WOA1d38pjOcArxQ7fLrZ3ZuFYSPaA21WQq6RwEiA1q1b987Ly0v6/sujoKCARo0apXUbyqAMmZRh9c7CpG3q7t3J7jr1S53fpn6qN9s6uAzJtl9dMiRTE34e9xs4cOAMd++TrF2qn/iDwNnABAB3n21mp5YjX1q5u5tZidXe3ccAYwD69OnjAwYMSGuWyZMnk+5tKIMyJLp35voy5+fsnMXcpj1LnT/6uBZp3T5AzupZLGvTq9T5Q9OcIdn2q0uGZGrK78SBSHZO8T/cfXmxScluDL4S6Jgw3iFMK7GNmdUGmhJ1uEll2eLWmFnbsK62wNok7UVERL4i1aK43My+BriZ1TGzHwKfJllmGtDNzDqbWTZRx5kJxdpMAIaH4SHAOx4dz50ADDWzumbWGegGTE2yvcR1DQf+kcobExER2S/VongdcD3QnmiPrVcYL5W7FwKjgDeICuh4d59nZneZ2QWh2VigeehIcwuhx6i7zwPGE3XKeR243t2LAMzseeBD4EgzW2FmI8K67gXOMrPPgDPDuIiISMpSPae4z90vS5wQ9uA2lLWQu08EJhabdnvC8C7g4lKWvRu4u4Tpw0ppvwE4o6w8IiIiZUl1T/FlM2uyf8TMjgJeTk8kERGReKRaFO8hKoyNzKw30TWFl6cvloiISOVL6fCpu79qZnWAfxLdHPyb7r4orclEREQqWbJ7n/6e6GHC+zUFFgOjzCzpvU9FRESqkmR7itOLjZd5v1MREZGq7IDufSoiIlKdJTt8Ot7dLzGzT/jqYVQA3P2YtCUTERGpZMkOn94Uvn4j3UFERETiluzw6arw9fPKiSMiIhKfZIdPt1HCYVPAiB5G0aSEeSIiIlVSsj3FxpUVREREJG4pPzpKRESkulNRFBERCVQURUREAhVFERGRQEVRREQkUFEUEREJVBRFREQCFUUREZFARVFERCRQURQREQnSWhTNbJCZLTSzfDMbXcL8umb2Qpg/xcxyEubdFqYvNLOzk63TzMaZ2VIzmxVevdL53kREpPpJ9uiog2ZmWcAjwFnACmCamU1w9/kJzUYAm9y9q5kNBe4DLjWzXGAo0ANoB7xlZkeEZcpa54/c/aV0vScREane0rmn2BfId/cl7r4HyAMGF2szGHg6DL8EnGFmFqbnuftud18K5If1pbJOERGRg5LOotgeWJ4wviJMK7GNuxcCW4DmZSybbJ13m9kcM3vAzOpWxJsQEZGaI22HT2NwG7AayAbGAD8B7ireyMxGAiMBWrduzeTJk9MaqqCgIO3bUAZlSJSzs7DM+XX37iRn9axS50+eXL4/C8m2nwkZkm2/umRIpqb8ThyIdH7iK4GOCeMdwrSS2qwws9pAU2BDkmVLnO7uq8K03Wb2FPDDkkK5+xiiokmfPn18wIABB/SmDtTkyZNJ9zaUQRkS3TtzfZnzc1bPYlmb0vuhDT2uRVq3nwkZkm2/xmTYOYu5TXuWOn90Obefikz4vUyUzsOn04BuZtbZzLKJOs5MKNZmAjA8DA8B3nF3D9OHht6pnYFuwNSy1mlmbcNXAy4E5qbxvYmISDWUtj1Fdy80s1HAG0AW8KS7zzOzu4Dp7j4BGAs8a2b5wEaiIkdoNx6YDxQC17t7EUBJ6wyb/JOZtQQMmAVcl673JiIi1VNaD1i7+0RgYrFptycM7wIuLmXZu4G7U1lnmH56efOKiEjNpjvaiIiIBCqKIiIigYqiiIhIoKIoIiISVKeL90Vil/y6sMIy21TGdWEiUjrtKYqIiAQqiiIiIoGKooiISKCiKCIiEqgoioiIBCqKIiIigYqiiIhIoKIoIiISqCiKiIgEKooiIiKBiqKIiEigoigiIhKoKIqIiAQqiiIiIoGKooiISKCiKCIiEughw1Jt6AG/IlJead1TNLNBZrbQzPLNbHQJ8+ua2Qth/hQzy0mYd1uYvtDMzk62TjPrHNaRH9aZnc73JiIi1U/aiqKZZQGPAOcAucAwM8st1mwEsMnduwIPAPeFZXOBoUAPYBDwqJllJVnnfcADYV2bwrpFRERSls49xb5Avrsvcfc9QB4wuFibwcDTYfgl4AwzszA9z913u/tSID+sr8R1hmVOD+sgrPPCNL43ERGphtJ5TrE9sDxhfAXQr7Q27l5oZluA5mH6R8WWbR+GS1pnc2CzuxeW0P4rzGwkMDKMFpjZwgN4TwejBVD2ya70U4YUMtymDDUpQ9KfR2XIjJ+FCtQplUY1rqONu48BxlTW9sxsurv3qaztKYMyKEPmb18ZMitDonQePl0JdEwY7xCmldjGzGoDTYENZSxb2vQNQLOwjtK2JSIiUqZ0FsVpQLfQKzSbqOPMhGJtJgDDw/AQ4B139zB9aOid2hnoBkwtbZ1hmUlhHYR1/iON701ERKqhtB0+DecIRwFvAFnAk+4+z8zuAqa7+wRgLPCsmeUDG4mKHKHdeGA+UAhc7+5FACWtM2zyJ0Cemf0vMDOsOxNU2qHaMihDRBkiyhD/9kEZ9suEDP9h0U6WiIiI6DZvIiIigYqiiIhIoKJYQcpzS7tKzHCqmX1sZoVmNqSkdVRChlvMbL6ZzTGzt80spWuHKjjDdWb2iZnNMrN/l3CnpbRnSGj3LTNzM6vQLukpfAZXmtm68BnMMrOrK3L7qWQIbS4JPw/zzOzPlZ3BzB5I+AwWmdnmGDIcZmaTzGxm+L04N4YMncLv4xwzm2xmHSp4+0+a2Vozm1vKfDOzh0K+OWZ2fEVu/4C4u17lfBF1+lkMdAGygdlAbrE23wceC8NDgRdiyJADHAM8AwyJ6XMYCDQIw9+L6XNokjB8AfB6ZWcI7RoD7xHdqKJPJX8GVwIPV/TPwAFm6EbUKe6QMN4qju9DQvsbiDrvVfbnMAb4XhjOBZbFkOFFYHgYPh14toIznAocD8wtZf65wGuAAScCU9L1s5nspT3FilGeW9pVWgZ3X+buc4B9FbjdA80wyd13hNGPiK4prewMWxNGGwIV3dsslZ8HgF8S3bN3V0zbT6dUMlwDPOLumwDcfW0MGRINA56PIYMDTcJwU+DLGDLkAu+E4UklzC8Xd3+P6AqD0gwGnvHIR0TXnbetyAypUlGsGCXd0q74bea+cks7YP8t7SozQ7odaIYRRP8dVnoGM7vezBYDvwZurOwM4fBQR3d/tYK3ndL2g2+FQ1UvmVnHEuanO8MRwBFm9r6ZfWRmg2LIAESHD4HO/F9hqMwMdwKXm9kKYCLRHmtlZ5gNXBSGvwk0NrOK/PuUTCb8/QJUFCUmZnY50Af4TRzbd/dH3P1woutbf16Z2zazWsBvgVsrc7vFvAzkuPsxwJv831GMylSb6BDqAKK9tMfNrFkMOSA6pfGSh+uhK9kwYJy7dyA6jPhs+BmpTD8ETjOzmcBpRHcEi+OziJ2KYsUozy3tKjNDuqWUwczOBH4GXODuu+PIkCCPin+iSrIMjYGewGQzW0Z0DmVCBXa2SfoZuPuGhM/+CaB3BW075QxEewMT3H2vR0/DWURUJCszw35DqfhDp6lmGAGMB3D3D4F6RDfJrrQM7v6lu1/k7scR/W7i7hXe6agMmfD3KxLXyczq9CL6j3cJ0eGX/SeyexRrcz1f7WgzvrIzJLQdR3o62qTyORxHdNK/W4zfi24Jw+cT3WEplu9FaD+Ziu1ok8pn0DZh+JvARzF8HwYBT4fhFkSHz5pX9vcB6A4sI9zMJIbP4TXgyjB8FNE5xQrLkmKGFkCtMHw3cFcaPoscSu9ocx5f7WgztaK3n3LOuDZc3V5Ehz0WhT/4PwvT7iLaG4Lov78XiZ4NORXoEkOGE4j+O99OtJc6L4YMbwFrgFnhNSGGDL8D5oXtTyrpD2W6MxRrO5kKLIopfga/Cp/B7PAZdI/h+2BEh5HnA58AQ+P4PhCd07u3ord9AJ9DLvB++F7MAr4eQ4YhwGehzRNA3Qre/vPAKmBv+Bs0ArgOuC7hZ+GRkO+Tiv59OJCXbvMmIiIS6JyiiIhIoKIoIiISqCiKiIgEKooiIiKBiqKIiEigoihShZlZMzP7fhgeYGavHODyV5pZu/SkE6l6VBRFqrZmRE9gOVhXAiqKIoGuUxSpwsxs/xMPFhJdGL0dWE90G7kZwOXu7mbWm+hC+UZh/pXASUR3N1oJ7AT6Az8iustPfeAD4FrXHwmpQVQURaqw8LDqV9y9p5kNAP4B9CC6Vdj7REVuCvAuMNjd15nZpcDZ7n6VmU0Gfuju08P6DnX3jWH4WaLbEb5cue9KJD614w4gIhVqqruvADCzWUT3m9xMtOf4ZniEZxbRLbdKMtDMfgw0AA4luhWciqLUGCqKItVL4lNHioh+x43oPrf9y1rQzOoBjxLdd3K5md1JdM9ekRpDHW1EqrZtRI+iKstCoKWZ9Qcwszpm1qOE5fcXwPVm1ojoJtEiNYr2FEWqMHffEJ5cP5eos8yaEtrsMbMhwENm1pTo9/5BokOj44DHzGx/R5vHgbnAamBa5bwLkcyhjjYiIiKBDp+KiIgENfrwaYsWLTwnJ6dc69i+fTsNGzasmEDllClZMiUHZE6WTMkBmZMlU3JA5mTJlByQOVkqKseMGTPWu3vLpA3jerpxJrx69+7t5TVp0qRyr6OiZEqWTMnhnjlZMiWHe+ZkyZQc7pmTJVNyuGdOlorKAUz3FOqCDp+KiIgEKooiIiKBiqKIiEhQozvaiEjNVLTPWbttF2u37mZP0T4WbSqiw9oCWjWpS5N6deKOJzFSURSRam9v0T7ez1/P5IXr+PiLTXy6ait7i756jfY9U94FoH2z+hzdvimnHtGSM3Nb0aqx7nRXk6goiki1tWjNNp56fxkTP1nFlp17qVenFr06NuOqkzpzWPMGtGlSj+zatZg9ew4du3Zn1ZZdzPtyKx9/vonX563mZ3+HAUe05MqTOnNK1xbUqmVxvyVJMxVFEal2Zn6xiQff+ox3F62jXp1aDOrRhm8c045TjmhB3dpZ/9W+aGUWA3q1/8+4u7NwzTZenbOK56cuZ/iTUzm6fVNGn9Odk7q2qMy3IpVMRVFEqo3lG3dw7+sLeHXOKlo0yuaHXz+Cb/frxKENsw9oPWZG9zZN6N6mCTec3o1/zFrJg299xmVPTOHrua353wt70qqJDqtWRyqKIlLl7dvnjPtgGb9+YwGGceMZ3Rh5ahca1S3/n7js2rW4uE9Hzj+2HU++v5TfvfUZZ/72Xf73m0dzwbHtKiC9ZBIVRRGp0j7fsJ1bx89m+uebGHhkS+656GjaNq1f4dupVyeL7w/oyqAebfjRS3O48fmZTF+2kZ+dd1SJh2SlalJRFJEq6/W5q/nRi7Mxg/svPpaLjm+PWXo7w3Rp2Yi8kSfy69cX8Pi/ljL/y608MbwPzRoc2CFayUy6eF9EqpzCon3cM/FTrntuBl1aNmTiTafwrd4d0l4Q96uTVYufnZfL74cdx5wVWxjy2Ies3LyzUrYt6aWiKCJVyvbdhVzzzHTGvLeE7/TvxPjr+tPhkAaxZDn/2HY8M6Iva7bu4qJH32fJuoJYckjFUVEUkSpjzdZdXPLHD3nvs/Xc882juWtwz9jP553YpTkvXtefwiLn249P4fMN22PNI+WT1qJoZoPMbKGZ5ZvZ6BLm1zWzF8L8KWaWkzDvtjB9oZmdHaZ1NLNJZjbfzOaZ2U0J7Q81szfN7LPw9ZB0vjcRqVz5awu46NEPWLp+O08M78O3+x0Wd6T/6N6mCX+6ph+7C4v49uNTWL5xR9yR5CClrSiaWRbwCHAOkAsMM7PcYs1GAJvcvSvwAHBfWDYXGAr0AAYBj4b1FQK3unsucCJwfcI6RwNvu3s34O0wLiLVwMLV2xg65kN2F+5j/LXrZlzvAAAgAElEQVT9GXhkq7gj/ZfubZrw7Ih+bNu1lyufmsqWHXvjjiQHIZ17in2BfHdf4u57gDxgcLE2g4Gnw/BLwBkWnSkfDOS5+253XwrkA33dfZW7fwzg7tuAT4H2JazraeDCNL0vEalE877cwtAxH5JVy3jh2hPp2b5p3JFK1bN9U8Z8pw9fbNzBtc9NZ0/hvrgjyQFKZ1FsDyxPGF/B/xWw/2rj7oXAFqB5KsuGQ63HAVPCpNbuvioMrwZal/cNiEi85q7cwrcfn0L9Olm8MLI/h7dsFHekpE7s0pxfDzmGj5ZsZPRf5xA99F2qiip5naKZNQL+Atzs7luLz3d3N7MSfxLNbCQwEqB169ZMnjy5XFkKCgrKvY6KkilZMiUHZE6WTMkBmZMlWY4vC/bxqyk7yc4yfnBsbZbNncaymLIcqEOAC7vW4a8fr6T+znWc1Sm1x1FlyvcGMidLpedw97S8gP7AGwnjtwG3FWvzBtA/DNcG1gNWvG2xdnXC+C3F1rUQaBuG2wILk2Xs3bu3l9ekSZPKvY6KkilZMiWHe+ZkyZQc7pmTpawcyzdu9xPvect7//JNX7KuINYsB6uoaJ+PGDfVD7/tVZ++bENsOQ5WpmSpqBzAdE+hdqXz8Ok0oJuZdTazbKKOMxOKtZkADA/DQ4B3QvgJwNDQO7Uz0A2YGs43jgU+dffflrGu4cA/KvwdiUjardu2myvGTmX77kKeuaovnVs0jDvSQalVy7j/kl60a1af7//pY9YX7I47kqQgbUXRo3OEo4j26j4Fxrv7PDO7y8wuCM3GAs3NLB+4hdBj1N3nAeOB+cDrwPXuXgScBFwBnG5ms8Lr3LCue4GzzOwz4MwwLiJVSMHuQoY/OZXVW3bx1HdPILddk7gjlUvT+nX4w+XHs3nHXm7Om8W+fTq/mOnSek7R3ScCE4tNuz1heBdwcSnL3g3cXWzav4kOr5bUfgNwRjkji0hMCov2ccOfP2bhmm2MHd6H3p0OjTtShejRril3nN+Dn/7tE576YBkjTu4cdyQpg+5oIyKxc3fuemU+kxau467BPRiQgdchlsewvh0586hW3Pf6Ahau3hZ3HCmDiqKIxO6p95fxzIefM/LULlzWr1PccSqcmXHvt46hSb3a3JQ3k92FRXFHklKoKIpIrN6cv4ZfvjqfQT3aMHpQ97jjpE2LRnX59ZBjWLB6Gw++9VnccaQUKooiEpvP1mzj5ryZHN2+KQ9c2otatSrn0U9xOb17ay7u3YEx7y1h/pf/dYm1ZAAVRRGJxfa9zshnZ1A/uzZjruhD/eya8fT6n513FIc0qMPov86hSL1RM46KoohUun37nDFzdrN84w4evex42jStF3ekStOsQTZ3nN+DOSu28NT7S+OOI8WoKIpIpXvwrUXMXlfEHefn0rdz9bj04kB845i2nN69Fff/c5EeM5VhVBRFpFK9MW81D72Tzynta3P5idWvp2kqzIxfXtiTWgZ3TpgXdxxJoKIoIpVm8boCbh0/m2M7NOWK3GyiOzfWTO2b1efGM7rx9oK1TFqwNu44Eqgoikil2LW3iOv/9DHZtWvxh8t7k51Vcwvift89qTNdWjbkFy/P07WLGUJFUUQqxS9enseC1dv47SXH0q5Z/bjjZITs2rW44/weLNuwg7H/VqebTKCiKCJp9/eZK3l+6nK+P+DwancLt/I67YiWnJXbmoffyWfTrn1xx6nxVBRFJK0Wryvgp3/7hBNyDuGWs46IO05G+p/zcinc54xfuCfuKDWeiqKIpM3OPdF5xHp1snho2HHUztKfnJIc1rwBI07uzIeripi7ckvccWo0/YSKSNoknkds21TnEcvyvQGH06gO3DPxU6JnrUscVBRFJC3+PnMledN0HjFVTerVYfDh2XyweAPvLloXd5waS0VRRCrc5xu28zOdRzxgAw+rTafmDfjVxAW6L2pMVBRFpEIVFu3j5hdmUauW8eBQnUc8ELVrGT8+uzsL12zjLx+viDtOjaSfVhGpUL9/J5+ZX2zmnm8eTXtdj3jAzj26Db06NuP+fy5k5x5d0F/ZVBRFpMLM+Hwjv3/nMy46vj3nH9su7jhVkplx2zndWbN1N8999HnccWocFUURqRDbdu3lprxZtD+kPr+4oEfccaq0fl2ac0q3Fvzh3cVs310Yd5waJa1F0cwGmdlCM8s3s9ElzK9rZi+E+VPMLCdh3m1h+kIzOzth+pNmttbM5hZb151mttLMZoXXuel8byLyVXf8Yx6rtuziwUuPo3G9OnHHqfJuOesINm7fw7gPlsUdpUZJW1E0syzgEeAcIBcYZma5xZqNADa5e1fgAeC+sGwuMBToAQwCHg3rAxgXppXkAXfvFV4TK/L9iEjp/jFrJX+duZJRA7vSu9MhccepFo477BDOPKoVf3x3MVt27o07To2Rzj3FvkC+uy9x9z1AHjC4WJvBwNNh+CXgDIueJTMYyHP33e6+FMgP68Pd3wM2pjG3iByAFZt28PO/z+W4w5pxw+ld445TrfzgrCPYuqtQNwuvRJauOyeY2RBgkLtfHcavAPq5+6iENnNDmxVhfDHQD7gT+MjdnwvTxwKvuftLYTwHeMXdeyas607gSmArMB241d03lZBrJDASoHXr1r3z8vLK9T4LCgpo1KhRudZRUTIlS6bkgMzJkik5oGKz7HPn3qm7+GLrPu46qT6tGqT+f3Z1/UwqOsfDM3cxd30RvzmtAY2zK+9xW5n8mRyMgQMHznD3PkkbunupL+D4sl5Jlh0CPJEwfgXwcLE2c4EOCeOLgRbAw8DlCdPHAkMSxnOAucXW1RrIItr7vRt4sqx87k7v3r29vCZNmlTudVSUTMmSKTncMydLpuRwr9gsD7/zmXf6ySv+0vTlseYor0zJUlKORau3es7oV/yeifNjzxKHisoBTPckNcHdqZ2kZt4fvtYD+gCzAQOOIdob61/GsiuBjgnjHcK0ktqsMLPaQFNgQ4rLfoW7r9k/bGaPA6+U1V5Eymf28s088OYivnFMWy46vn3ccaqtbq0bM/jYdjzzweeMPKULzRvVjTtStVbmsQ53H+juA4FVRHuGfdy9N3AcSYoUMA3oZmadzSybqOPMhGJtJgDDw/AQ4J1Q0ScAQ0Pv1M5AN2BqWRszs7YJo98k2gsVkTTYvruQm1+YRavGdbn7wqOJugJIuow6vRu7Cot0brESpHoC4Eh3/2T/iLvPBY4qawF3LwRGAW8AnwLj3X2emd1lZheEZmOB5maWD9wCjA7LzgPGA/OB14Hr3b0IwMyeBz4EjjSzFWY2Iqzr12b2iZnNAQYCP0jxvYnIAfrlK/NZtmE7v720F00b6PKLdOvaqhHnHt2WZz78nC071BM1nZIdPt1vjpk9ATwXxi8D5iRbyKPLIiYWm3Z7wvAu4OJSlr2b6Nxg8enDSml/RbI8IlJ+r89d9Z+nX5zYpXnccWqMUQO78uqcVTz1wVJuPlM3WU+XVPcUvwvMA24Kr/lhmojUIKu37GL0Xz/hmA5N9Ye5kh3Vtgln5bbmqfeXsW2X9hbTJaWiGPboHgFuB/6HqBfprnQGE5HMsm+fc+uLs9i9dx8PXtqL7Nq6S2Rlu+H0rmzZuZdndU/UtEnpp9rMBgCfEV0q8SiwyMxOTWMuEckwY/+9lPfzN3DH+bl0aRn/9Ws10TEdmnHaES154l9L2bFH90RNh1T/1bsf+Lq7n+bupwJnE92WTURqgHlfbuHXbyzg7B6tufSEjskXkLS58YyubNy+hz9P+SLuKNVSqkWxjrsv3D/i7osAdTkTqQF27iniprxZHNIgm19ddIwuv4hZ706H0r9Lc8a8t4Rde/W8xYqWalGcbmZPmNmA8Hqc6OJ9Eanm7pn4KflrC/jtJb04tGF23HEEuOGMrqzdtpsXpy+PO0q1k2pR/B5Rj9Mbw2t+mCYi1djbn67h2Y8+55pTOnNytxZxx5Ggf5fm9O50CI+9u4S9RfvijlOtpNr7dDdRJ5s7iHqgPhymiUg1tXbbLn780hyOatuEH559ZNxxJIGZ8f0Bh7Ny805emfNl3HGqFfU+FZH/4u786MU5FOwu5KGhvahbOyv5QlKpBh7ZiiNbN+axyUv2PxRBKoB6n4rIf3n6g2W8u2gdPz/vKLq1bhx3HClBrVrGtad1YeGabUxauDbuONWGep+KyFcsWrONe15bwOndW3H5iZ3ijiNlOP/YdrRvVp8/TF4cd5RqQ71PReQ/du0t4sbnZ9KkXm1+PUSXX2S6Olm1uOaUzkxbtolpyzbGHadaUO9TEfmP37yxkAWrt/GbIcfSQs/tqxIuOaEjhzSow2PaW6wQKT0lI/Q0/W14iUg19N6idYz991KG9+/EwO6t4o4jKWqQXZsrv9aZB95axMLV2ziyjc4Bl0eqvU9PMrM3zWyRmS3Z/0p3OBGpHBu37+HWF2fTrVUjbju3zEelSgb6Tv9ONMjO4o/vam+xvFI9fDqWaC/xZOCEhJeIVHHR5Rez2bJjL78behz16ujyi6rmkIbZDD3hMP4x+0tWbNoRd5wqLdWiuMXdX3P3te6+Yf8rrclEpFKM/fdS3l6wlp+e253cdk3ijiMH6epTOmPAE/9aGneUKq3Momhmx5vZ8cAkM/uNmfXfPy1MF5EqbM6Kzdz3+gLOym3N8K/lxB1HyqFds/oM7tWevGlfsHH7nrjjVFnJOtrcX2y8T8KwA6dXbBwRqSzbdu3lhudn0rJRXX6jyy+qhetO68JfPl7BuA+WcctZR8Qdp0oqsyi6+8DKCiIilcfd+enf5rJi005eGHkizRro6RfVQbfWjTkrtzVPf7CMa0/tQsO6KV1gIAmSHT69PHy9paRXspWb2SAzW2hm+WY2uoT5dc3shTB/ipnlJMy7LUxfaGZnJ0x/0szWmtncYus6NPSQ/Sx8PST52xepmcZPX87Ls7/klrOOoE/OoXHHkQp03WmHs2XnXvKm6bFSByNZR5uG4WvjUl6lMrMs4BHgHCAXGGZmucWajQA2uXtXonup3heWzQWGAj2AQcCjYX0A48K04kYDb7t7N+DtMC4ixazcto87JszjpK7Nue60w+OOIxWsd6dD6Nv5UJ741xL2FOqxUgcq2eHTP4avvziIdfcF8t19CYCZ5QGDie6Gs99g4M4w/BLwsEUnNgYDeeGmAUvNLD+s70N3fy9xj7LYugaE4aeBycBPDiK3SLW1c08Rj87eRaO6dXjg0l5k1dJ5xOroewMO57tPTePvs1ZySZ+OccepUqysR46Y2UNlLezuN5ax7BBgkLtfHcavAPq5+6iENnNDmxVhfDHQj6hQfuTuz4XpY4HX3P2lMJ4DvOLuPRPWtdndm4VhI9oDbVZCrpHASIDWrVv3zsvLK+stJlVQUECjRo3KtY6KkilZMiUHZE6WTMnx5NzdvLeikFt71+XolvGeb8qUzwQyJ0tF5XB3bv9gF3v3OfecXJ9aB9GJqrp9JgMHDpzh7n2StUv2WzGj3Eli4O5uZiVWe3cfA4wB6NOnjw8YMKBc25o8eTLlXUdFyZQsmZIDMidLJuQYP305762Ywze61OGGi8+MNQtkxmeyX6ZkqcgcPzr0S258fiZ7WnZnUM+2sWYpj8rOkezw6dOJ42bWwN1TvV3CSiBxv71DmFZSmxVmVhtoCmxIcdni1phZW3dfZWZtAT1gTCSY9+UW/ufvc/na4c355uE7444jleDcnm24v3kDHp28mLN7tNElNylK9d6n/c1sPrAgjB9rZo8mWWwa0M3MOptZNlHHmQnF2kwAhofhIcA7Hh3PnQAMDb1TOwPdgKlJtpe4ruHAP1J4ayLV3pade/n+nz6mWYM6PDTsOJ1HrCFqZ9Xi2lMPZ86KLbyfrxuQpSrV27w9CJxNtBeHu88GTi1rAXcvBEYBbwCfAuPdfZ6Z3WVmF4RmY4HmoSPNLYQeo+4+DxhP1CnndeB6dy8CMLPngQ+BI81shZmNCOu6FzjLzD4DzgzjIjXavn3OreNns3LTTh697Hg9DqqG+Vbv9rRqXJc/vJsfd5QqI+Uz7e6+vNjud1EKy0wEJhabdnvC8C7g4lKWvRu4u4Tpw0ppvwE4I1kmkZrkj+8t4a1P13D7N3Lp3UnXI9Y0dWtnMeLkzvzqtQXMXr6ZYzv+V99DKSbVPcXlZvY1wM2sjpn9kGjvT0Qy1AeL1/ObNxZw3jFt+e5JOXHHkZhcdmInmtSrzaOTtbeYilSL4nXA9UB7og4vvcK4iGSglZt3csOfZ9K5RUPu+5bua1qTNapbm+Ffy+GNeWvIX7st7jgZL9WiuM/dL3P31u7eyt0vB/SMGZEMtGNPIdc8PZ09hfv44xV9aKT7X9Z4V34th3p1avHYu3o2fDKpFsWXzew/RdDMjgJeTk8kETlY0QOD5/Dp6q08NOw4uraK/+JriV/zRnUZesJh/H3mSlZu1iU5ZUm1KN5DVBgbmVlvoluyXZ6+WCJyMB6dvJhXP1nFj8/uzsDureKOIxnkmlO7APD4e9pbLEtKRdHdXyW6Yfc/iW7I/U13n5XGXCJygN6av4b/98+FDO7VjutO6xJ3HMkw7fUQ4pQke3TU783soXAP1NOJ7jizFBiV7L6oIlJ5PluzjZtfmEXPdk3VsUZK9b0BXdhduI9x7y+NO0rGSnYGfnqx8Sp5L1SR6mzdtt18d9w06tXJYsx3elOvTlbyhaRG6tqqMV/Pbc24D5Yx8rTD1QmrBAd071MRySw79xRx9TPTWV+wm7yR/WnbtH7ckSTDfW9AV96Yt4Y/T/mckafqeZrFJTt8Oj58/cTM5hR/VU5EESlJ0T7n5hdmMmfFZn439Dh66W4lkoJeHZtxctcWjHlvCTv3JL0xWY2TrKPNTeHrN4DzS3iJSEzumfgpb8xbw/+cl8vZPdrEHUeqkJvO7Mb6gj38acrncUfJOGUWRXdfFb5+XtKrciKKSHHj3l/K2H8v5cqv5XDVyZ3jjiNVzAk5h3Jy1xY89u5i7S0Wk+zw6TYz21rCa5uZba2skCLyf177ZBV3vTKfs3Jb8z/fyI07jlRR2lssWbI9xcbu3qSEV2N3123eRCrZvz9bz015szjusEP43dBeejaiHDTtLZYs1TvaiEjMZi3fzMhnp9OlZUOeHH4CDbLVnV7KR3uL/01FUaQK+GzNNq58aiotGtXlmav60rRBnbgjSTWgvcX/pqIokuGWb9zBFWOnUierFs+N6EerJvXijiTViPYWv0pFUSSDLd+4g6FjPmLn3iKeHdGXw5o3iDuSVDOJe4s79hTGHSd2KooiGWrFph0Me/wjCnYX8qer+9G9jfq2SXr84Kxob/Gp95fFHSV2KooiGWjl5p0Me/wjtu7cy3Mj+tGzfdO4I0k11rvToZx5VGsem7yYTTX8CRppLYpmNsjMFppZvpmNLmF+XTN7IcyfYmY5CfNuC9MXmtnZydZpZuPMbKmZzQqvXul8byLpsnLzToaO+ZDNO/by3NX9OLqDCqKk348HHcn2PYU8Ojk/7iixSltRNLMs4BHgHCAXGGZmxa80HgFscveuRM9rvC8smwsMBXoAg4BHzSwrhXX+yN17hZee9yhVzuJ1BVz8hw+igjiiH8d00P1MpXIc0boxFx3fgac//JyVm3fGHSc26dxT7Avku/sSd98D5AGDi7UZDOx/EsdLwBkWPQhuMJDn7rvdfSmQH9aXyjpFqqS5K7dwyWMfsqdoH3kjT+RY3eBbKtkPzjoCgAffXBRzkviYu6dnxWZDgEHufnUYvwLo5+6jEtrMDW1WhPHFQD/gTuAjd38uTB8LvBYWK3GdZjYO6A/sBt4GRrv77hJyjQRGArRu3bp3Xl5eud5nQUEBjRo1Ktc6KkqmZMmUHJA5WZLlWLixiAc/3kWD2saPTqhHm4bp+3+1qnwmlSlTsmRCjucX7Oafywr56fFOt1bV5zMZOHDgDHfvk6xddbolxm3AaiAbGAP8BLireCN3HxPm06dPHx8wYEC5Njp58mTKu46KkilZMiUHZE6WsnK8/ekafvvWx3Q4pCHPXd0v7c9ErAqfSWXLlCyZkOPYE/bwwa8n8fIXzoRL4s0Clf+ZpPPw6UqgY8J4hzCtxDZmVhtoCmwoY9lS1+nuqzyyG3iK6FCrSEYb9/5SrnlmOke2acz4a/WQYInfIQ2zuenMbsxZX8SkBWvjjlPp0lkUpwHdzKyzmWUTdZyZUKzNBGB4GB4CvOPR8dwJwNDQO7Uz0A2YWtY6zaxt+GrAhcDcNL43kXIp2ufcOWEed748nzOOak3eyBNp3qhu3LFEAPhO/xzaNDB++cp89hTuiztOpUpbUXT3QmAU8AbwKTDe3eeZ2V1mdkFoNhZobmb5wC3A6LDsPGA8MB94Hbje3YtKW2dY15/M7BPgE6AF8L/pem8i5bF9dyHXPjudcR8sY8TJnXns8t66ubdklOzatRh2VDZL1m/nmQ+XxR2nUqX1N9HdJwITi027PWF4F3BxKcveDdydyjrD9NPLm1ck3ZasK+DaZ2eweF0Bdw3uwXf658QdSaREx7aszYAjm/C7tz7jwuPa06KGHMnQHW1EKsmb89cw+OH3WV+wm2eu6qeCKBnv5+flsnNvEfe9tiDuKJVGRVEkzYr2OX/9bA/XPDOdnBYNefmGkzm5W4u4Y4kk1bVVI0ac0pkXZ6zgw8Ub4o5TKVQURdLoy3AP0wmL93Jx7w68eF1/OhyiJ11I1XHzGUdw2KEN+NnfPmHX3ur/zEUVRZE0eX3uKs753b+Yt3IL1xydza+HHEO9OllxxxI5IPWzs7j7mz1Zsn47j06q/vdFVVEUqWDbdu1l9F/mcN1zH9OpeQNevfEUTmpfh+hqIZGq55RuLbnouPb84d3FLFy9Le44aaWiKFKBJi1Yy9cfeI/x05dz3WmH89J1XyOnRcO4Y4mU28+/kUuTenX4wQuz2F1YfQ+jqiiKVIDNO/Zwywuz+O64aTSqW5u/fO9rjD6nO9m19Ssm1cOhDbO571vHMH/VVn5bjW8YriuGRcqhaJ8zfvpyfvPGQrbu3MsNp3dl1OldqVtb5w6l+jkztzXf7ncYY95bwsAjW3Fil+ZxR6pw+jdW5CDN+HwTFz7yPrf99RO6tmzEhFEnc+vXj1RBlGrt5+cdRU7zhtzywiw279gTd5wKp6IocoCWrd/Ojc/P5Ft/+IC123bxu6G9eOHaE8lt1yTuaCJp1yC7Ng9e2ov1BXu4KW8WRfvS8/jBuOjwqUiKVm/ZxUPvfMb4acupnWV8f8DhXD+wKw3r6tdIapZjOzbjzgt68NO/fcKDby3i1q8fGXekCqPfZpEklm/cwRP/WkLetOXsc+fb/Q5j1MCutGpSL+5oIrEZ1rcjs5dv5vfv5NOjXVMG9WwTd6QKoaIoUor5X27lj+8t5pU5q6hlcGGv9tx4Rjc6Hqo70oiYGb8Y3IOFa7ZxU95M/nxNP3p3OjTuWOWmoiiSYE/hPv45fzV/nvIFHyzeQMPsLK46KYerTu6sBwCLFFOvThZPXnkCQ/7wAVeNm85L1/WnW+vGcccqFxVFEaLOM+OnL2f89OWsL9hDh0Pq8+NBR3JZ3040bVAn7ngiGevQhtk8fVVfLvrDB1wxdirPjzyRzlX4hhUqilJjrdm6i5dnf8nLs79k9oot1DI446joOqxTu7Ukq5ZuyyaSio6HNuCZq/py2RNTuOSPH/Lnq/tV2T1GFUWpMdydxesKePvTtbz96Vqmfb4Rd+jZvgk/O/cozj+2HW2aqvOMyME4qm0TXhh5It8OhfGxy3vTrwpe3K+iKNVawR7njXmr+XDxBt5ZsJYvNu4Aol/gG07vxuBe7Ti8ZaOYU4pUD91aN+bFa/tz1bhpXD52Cr8c3JOhfQ+LO9YBUVGUasPdWbFpJ3NWbGHq0g1MWbqRBat3ADOoW7sWJ3VtwchTu3B691a0a6ZOMyLpkNOiIX+7/iRueH4mo//6Cf/KX8//Du7JIQ2z446WEhVFqZJ27ili6frt5K8rYN7KLcz9cgtzV25ly869ANSvk0WfnEPIbbyLYWf04ZgOTXX7NZFK0rR+HZ4c3oc/vreEB99axJQlG/nxoCP51vEdMv5cfVqLopkNAn4HZAFPuPu9xebXBZ4BegMbgEvdfVmYdxswAigCbnT3N8pap5l1BvKA5sAM4Ap3r3435qsh9u1z1m/fzarNu1i1ZSdfbt7FFxt3sHhdAUvWbefLLTvxcHep7KxadG/bmHOPbsvR7ZvSs30TjmrbhDpZtZg8eTIn5FT9a6dEqpraWbW4fmBXBh7Zitv+9gk/fmkOT/57KSNP7cJ5x7TN2H9S01YUzSwLeAQ4C1gBTDOzCf+/vTOPsaq64/jnOzswLAKibDKiGAVrok6tVqJorRptxYUqNiaS0lqX2j+sXRKbxtiS1japrdHWoDUqiSLyh+LauDBatIC07FoQEBHc2ClQZv31j3OGeTO+gQfMffcO/D7Jy5x777lnPu+8+97v3nPPPcfM3svJNhnYamYnSpoI3AtcJ2k0MBEYAwwBXpN0UtynszLvBe4zs+mSHopl/zWp9+cUhplR39TC7oZmdjc0sbuhmZ31TWzb3cCWXY1s3dXAlt0NcTm8Ptuxh8+319PQ3NKurF4VpYw8upramqMYOXA4JwzqxciB1Zw4qNqnaHKcjDJ6SB+evfXrvLj0U+57dSV3zFjMlBff55JTj+WiUwZRW9OfPlXZeewpySvFs4BVZrYGQNJ0YDyQGxTHA3fH9EzgAYXpyccD082sHvhQ0qpYHvnKlPQ+cCHw3Zjn8VhuokFx7prNvPJhIx+UrAHACJcurVcwrcPkti23387e7fn329e+1iGDAWs/amBB/YoDcuFL27+8X3OL0djcQmNzC03NRkP829jc0i7d2GI0NrWwZcdueOd1dsUguL8Bg8tLRb+eFfTvWUG/nuWccdxRDFDEhQQAAAkvSURBVO7bg8F9qxjct4oh/UK6f68Kn73ecbohkvjWaUO4/CuDmbNqE0/NX8ezCzfw5Lx1ANQM6MmIAb04tk8V/asrqCwrobKslPJSsXptI2ObWygrLc6Jb5JBcSjwcc7yeuBrneUxsyZJ2wnNn0OBuR32HRrT+cocAGwzs6Y8+dsh6SbgJoBjjjmGurq6A3pTuUz/Tz2vrG2CFe8fdBldQVuYMFi9ita40TF8qENCBa4vEZSVQKkU/8blErWlBWUlokIwpEcL1VVNVJZCZWkZVaVQWaZ2f6vLRXWF6F0RlkOwawHq42s7NAAbYdNG2HSQdbNz585D+oy7iqx4QHZcsuIB2XHJigck63LtULjy2EpWbm1hzfZm1u2oZ91ne1j0kbGzwWjucB59Qd2bVJQW54T4iOtoY2ZTgakAtbW1Nm7cuIMu6+xzm5n95luMHTt27xXM3oCyN8Co3TL72d62f9sOXyqzk6uluro6DuX9dBVZ8YDsuGTFA7LjkhUPyI5LVjygOC4Xd7K+pSW0SDU0tzDnH3P45oXjitZKlGRQ3AAMz1keFtfly7NeUhnQl9DhZl/75lu/GegnqSxeLeb7X11OVXkpPcpE7wy1hzuO43R3SkpEVUkpVeWl9CxXUW+bJNlI+y4wStLxkioIHWdmdcgzC7gxpicAb1i4qTULmCipMvYqHQXM76zMuM/sWAaxzOcSfG+O4zjOYUhiV4rxHuGPgL8THp941MyWS7oHWGBms4C/AdNiR5othCBHzDeD0CmnCbjNzJoB8pUZ/+XPgemSfgMsjGU7juM4TsEkek/RzF4CXuqw7lc56T3AdzrZdwowpZAy4/o1tPVQdRzHcZwDxh/uchzHcZyIrONDc0cQkjYCHx1iMQM5+CcGupqsuGTFA7LjkhUPyI5LVjwgOy5Z8YDsuHSVxwgzO3p/mY7ooNgVSFpgZrVpe0B2XLLiAdlxyYoHZMclKx6QHZeseEB2XIrt4c2njuM4jhPxoOg4juM4EQ+Kh87UtAVyyIpLVjwgOy5Z8YDsuGTFA7LjkhUPyI5LUT38nqLjOI7jRPxK0XEcx3EiHhQdx3EcJ+JBsUAkXSpphaRVkn6RZ3ulpKfj9nmSalLyOE/SvyU1SZqQr4wiutwh6T1JSyS9LmlESh43S1oqaZGkOXES60TYn0tOvmskmaREupoXUCeTJG2MdbJI0veT8CjEJea5Nh4ryyU9mYaHpPty6mOlpG1JeBTocpyk2ZIWxu/PZSl5jIjf3SWS6iQNS8jjUUlfSFrWyXZJuj96LpF0RhIeQJhU1l/7fhHGWV0NjAQqgMXA6A55bgUeiumJwNMpedQApwFPABNSrpMLgJ4xfUuKddInJ30F8EpadRLz9QbeIswZWptSnUwCHkjq+DhAl1GE8YqPisuD0vpscvLfThhbOa06mQrcEtOjgbUpeTwD3BjTFwLTEqqT84AzgGWdbL8MeJkwk97ZwLykjlm/UiyMs4BVZrbGzBqA6cD4DnnGA4/H9EzgG+r6+U7262Fma81sCWHG3iQpxGW2me2Oi3MJU3ql4bEjZ7EXkFTvskKOE4BfA/cCe1L2KAaFuPwAeNDMtgKY2RcpeeRyPfBUAh6FuhjQJ6b7Ap+k5DEaeCOmZ+fZ3iWY2VuESSE6YzzwhAXmEqYKHJyEiwfFwhgKfJyzvD6uy5vHwpyO24EBKXgUiwN1mUw400vFQ9JtklYDvwd+nIBHQS6x2We4mb2YkENBHpFrYlPUTEnD82wvlstJwEmS3pY0V9KlKXkAockQOJ62YJCGy93ADZLWEyZAuD0lj8XA1TF9FdBbUlf/rhVC0X77PCg6iSPpBqAW+ENaDmb2oJmdQJhi7JdpOEgqAf4I/CSN/9+B54EaMzsNeJW2Vo40KCM0oY4jXKE9LKlfij4TgZkWp6tLieuBx8xsGKHpcFo8forNncD5khYC5xMmb0+zXhLHg2JhbAByz6SHxXV580gqIzR5bE7Bo1gU5CLpIuAu4Aozq0/LI4fpwJUJeBTi0hs4FaiTtJZwb2RWAp1t9lsnZrY55/N4BDizix0KdiGc9c8ys0Yz+xBYSQiSxfZoZSLJNZ0W6jIZmAFgZv8EqggDYxfVw8w+MbOrzex0wvcYM0usA9I+KN5vX1I3Kw+nF+FMdg2hSaX1hvSYDnluo31HmxlpeOTkfYxkO9oUUienE27kj0rZY1RO+tuESa5TcemQv45kOtoUUieDc9JXAXNT/HwuBR6P6YGEZrIBaXw2wMnAWuLAJinWycvApJg+hXBPsUudCvQYCJTE9BTgngTrpYbOO9pcTvuONvMT80iq4MPtRWjCWBl/5O+K6+4hXAFBOJN7BlgFzAdGpuTxVcKZ9y7CleryFOvkNeBzYFF8zUrJ48/A8ugwO9+PYbFcOuStI4GgWGCd/DbWyeJYJyeneJyI0Kz8HrAUmJjWZ0O4l/e7pOriAOpkNPB2/HwWARen5DEB+CDmeQSoTMjjKeBToDH+fk0GbgZuzjlGHoyeS5P63piZD/PmOI7jOK34PUXHcRzHiXhQdBzHcZyIB0XHcRzHiXhQdBzHcZyIB0XHcRzHiXhQdJxujKR+km6N6XGSXjjA/SdJGpKMneN0PzwoOk73ph9hhpaDZRLgQdFxIv6couN0YyS1zmywgvDg8y5gE2E4uX8BN5iZSTqT8IB8ddw+CTiXMPLRBuB/wDnATwmj/vQA3gF+aP4j4RxBeFB0nG5MnMz6BTM7VdI44DlgDGFYsLcJQW4e8CYw3sw2SroOuMTMviepDrjTzBbE8vqb2ZaYnkYYrvD54r4rx0mPsrQFHMfpUuab2XoASYsI40luI1w5vhqn+CwlDKmVjwsk/QzoCfQnDAXnQdE5YvCg6DiHF7kzkTQTvuMijIF7zr52lFQF/IUwruTHku4mjOnrOEcM3tHGcbo3/yVMSbUvVgBHSzoHQFK5pDF59m8NgJskVRMGg3acIwq/UnScboyZbY4z1i8jdJb5PE+eBkkTgPsl9SV87/9EaBp9DHhIUmtHm4eBZcBnwLvFeReOkx28o43jOI7jRLz51HEcx3EiHhQdx3EcJ+JB0XEcx3EiHhQdx3EcJ+JB0XEcx3EiHhQdx3EcJ+JB0XEcx3Ei/wfHy58h809J5AAAAABJRU5ErkJggg==\n",
126 | "text/plain": [
127 | ""
128 | ]
129 | },
130 | "metadata": {},
131 | "output_type": "display_data"
132 | }
133 | ],
134 | "source": [
135 | "def bernoulli_likelihood(theta):\n",
136 | " return theta**7 * (1-theta)**3\n",
137 | "\n",
138 | "# Display\n",
139 | "fig, subplots = plt.subplots(2)\n",
140 | "fig.set_figheight(5)\n",
141 | "fig.tight_layout()\n",
142 | "subplots[0].set_title('Likelihood of 7 coin flips')\n",
143 | "\n",
144 | "# Plot1\n",
145 | "theta = np.arange(0, 1.1, 0.1)\n",
146 | "p = bernoulli_likelihood(theta)\n",
147 | "disply_plot(theta, p, subplots[0])\n",
148 | "\n",
149 | "# Plot2\n",
150 | "theta = np.arange(0, 1.001, 0.001)\n",
151 | "p = bernoulli_likelihood(theta)\n",
152 | "disply_plot(theta, p, subplots[1])"
153 | ]
154 | },
155 | {
156 | "cell_type": "markdown",
157 | "metadata": {},
158 | "source": [
159 | "# Loglikehood\n",
160 | "\n",
161 | "대부분의 경우에서는 computation의 이유로 log-likelihood 를 사용합니다.\n",
162 | "\n",
163 | "$$ l(\\theta\\ |\\ x) = \\log L(\\theta\\ |\\ x) $$\n",
164 | "\n",
165 | "**예를 들어서 binomial loglikelihood** 는 다음과 같습니다.\n",
166 | "> [log rule](https://www.rapidtables.com/math/algebra/logarithm/Logarithm_Rules.html)을 참조합니다.
\n",
167 | "> power rule: $ \\log m^r = r( \\log m) $
\n",
168 | "> product rule: $ \\log_b(x \\cdot y) = \\log_b(x) + \\log_b(y) $\n",
169 | "\n",
170 | "$$ \\begin{align} \n",
171 | "l(\\theta\\ |\\ x) &= \\log \\left( \\theta^x (1-\\theta)^{n-x} \\right) \\\\\n",
172 | "&= \\log \\big(\\theta^x \\big) + \\log \\big((1-\\theta)^{n-x} \\big) \\\\\n",
173 | "&= x \\log \\theta + (n-x) \\log (1-\\theta)\n",
174 | "\\end{align}$$\n",
175 | "\n",
176 | "대부분의 문제에서, loglikelihood는 하나의 observation으로 연산하는 것이 아니라 sample로 부터 연산을 합니다.
\n",
177 | "Independent sample $ x_1, x_2, ..., x_n $ 의 likelihood의 곱은 overall likelihood와 동일합니다.\n",
178 | "\n",
179 | "> $ x $ 이나 $ x_i $ 냐를 잘 구분지어서 봐야 합니다.\n",
180 | "\n",
181 | "$$ \\begin{align} \n",
182 | "L(\\theta\\ |\\ x) &= \\prod^n_{i=1} f(x_i | \\theta) \\\\\n",
183 | "&= \\prod^n_{i=1} L(\\theta\\ |\\ x_i)\n",
184 | "\\end{align}$$\n",
185 | "\n",
186 | "**Log-likelihood의 정의**는 다음과 같습니다.\n",
187 | "\n",
188 | "$$ \\begin{align} \n",
189 | "l(\\theta\\ |\\ x) &= \\log \\prod^n_{i=1} f(x_i\\ |\\ \\theta) \\\\\n",
190 | "&= \\sum^n_{i=1} \\log f(x_i\\ |\\ \\theta) \\\\\n",
191 | "&= \\sum^n_{i=1} \\log f(\\theta\\ |\\ x_i)\n",
192 | "\\end{align} $$\n",
193 | "\n",
194 | "\n",
195 | "\n",
196 | "\n"
197 | ]
198 | },
199 | {
200 | "cell_type": "markdown",
201 | "metadata": {},
202 | "source": [
203 | "# Maximum Likelihood\n",
204 | "\n",
205 | "간단히 정의하면, 주어진 샘플 $ x $ 에 대해서 likelihood를 maximize하는 parameter $ \\theta $ 를 찾는 것입니다.
\n",
206 | "위의 동전 던지기 예제에서는 0.7 정도가 maximum likelihood로 볼 수 있습니다.
\n",
207 | "Likelihood function $ \\mathbf{L} = f(\\mathbf{x};\\theta) $ 일때 **Maximum likelihood estimator (MLE)의 공식**은 다음과 같습니다.\n",
208 | "\n",
209 | "$$ \\hat{\\theta}(x) = arg\\max_{\\theta} \\mathbf{L}(\\theta | \\mathbf{x}) $$ \n",
210 | "\n",
211 | "> hat을 씌워주는 이유는 estimate이라는 것을 표현하기 위해서 씌워 놓습니다."
212 | ]
213 | },
214 | {
215 | "cell_type": "markdown",
216 | "metadata": {},
217 | "source": [
218 | "## Fireman Test Example \n",
219 | "\n",
220 | "$ X^n = (x_i, ..., x_n) $ 샘플이 있으며 다음과 같은 값을 갖을 수 있습니다.\n",
221 | "\n",
222 | "* $ x_i = 0 $ : 소방관 시험에 합격한 사람\n",
223 | "* $ x_i = 1 $ : 소방관 시험에 불합격한 사람\n",
224 | "\n",
225 | "$ x_i $ 가 independent Bernoulli random variables이며 parameter $ \\theta $ 는 모르고 있을때,
\n",
226 | "각각의 $ x_i $ 에 대한 Probability mass function 은 다음과 같을 것입니다.\n",
227 | "\n",
228 | "$$ f(x_i; \\theta) = \\theta^{x_i} (1-\\theta)^{1-x_i} $$\n",
229 | "\n",
230 | "* $ \\theta $ 는 확률로서 0에서 1의 값을 갖습니다.\n",
231 | "\n",
232 | "따라서 likelihood function $ P(\\theta) $ 는 다음과 같을 것입니다.\n",
233 | "\n",
234 | "$$ \\begin{align} \n",
235 | "L(\\theta) &= \\prod^n_{i=1} f(x_i; \\theta) \\\\ \n",
236 | "&= \\theta^{x_1} (1-\\theta)^{1-x_1} * \\theta^{x_2} (1-\\theta)^{1-x_2} * ... * \\theta^{x_n} (1-\\theta)^{1-x_n} \\\\\n",
237 | "&= \\theta^{\\sum x_i} (1-\\theta)^{n-\\sum x_i}\n",
238 | "\\end{align} $$\n",
239 | "\n",
240 | "Likelihood를 알아냈지만, maximum likelihood의 연산량을 줄이기 위해서 log likelihood를 사용합니다. \n",
241 | "\n",
242 | "$$ \\begin{align} \n",
243 | "\\ln L(\\theta) &= \\left( \\sum^n_{i=1} x_i \\right) \\ln\\theta + \\left(n-\\sum^n_{i=1} x_i \\right) \\ln(1-\\theta) \\\\\n",
244 | "&= n\\big( \\bar{x} \\ln \\theta + (1-\\bar{x}) \\ln(1-\\theta) \\big)\n",
245 | "\\end{align} $$\n",
246 | "\n",
247 | "\n",
248 | "Likelihood를 알아냈으면 **maximum likelihood**를 partial derivative로 찾을수 있습니다.
\n",
249 | "**이때 derivative의 값은 0으로 놓습니다.**\n",
250 | "\n",
251 | "> Drivative rule 참고.
\n",
252 | "> $ \\frac{d}{dx} \\ln(x) = \\frac{1}{x} $
\n",
253 | "> $ \\frac{d}{dx} \\log_a(x) = \\frac{1}{x \\ln(a)} $
\n",
254 | "\n",
255 | "$$ \\begin{align}\n",
256 | "\\frac{\\partial}{\\partial \\theta} \\ln L(\\theta | \\mathbf{x}) &= \n",
257 | "n\\left( \\frac{\\bar{x}}{\\theta} - \\frac{1-\\bar{x}}{1-\\theta} \\right) \\\\\n",
258 | "&= n \\left( \\frac{\\bar{x}(1-\\theta)}{\\theta(1-\\theta)} - \\frac{\\theta(1-\\bar{x})}{\\theta(1-\\theta)} \\right) \\\\\n",
259 | "&= n \\frac{\\bar{x}-\\theta}{\\theta(1-\\theta)} = 0\n",
260 | "\\end{align} $$\n",
261 | "\n",
262 | "\n",
263 | "$ n \\frac{\\bar{x}-\\theta}{\\theta(1-\\theta)} $ 값이 0이 되려면 $ \\bar{x} = \\theta $ 일 경우에 0이 될 수 있습니다.
\n",
264 | "따라서 maximum likelihood estimate 은 다음과 같습니다.\n",
265 | "\n",
266 | "$$ \\hat{\\theta}(x) = \\bar{x} $$\n",
267 | "\n",
268 | "\n",
269 | "### Code\n",
270 | "\n",
271 | "아래의 코드는 7명이 합격하고, 3명이 실패했을때 입니다. (coin flip과 동일)
\n",
272 | "계산 방식을 각각의 probability mass function을 구한 다음에 곱으로 $ \\prod $ 계산한 것입니다."
273 | ]
274 | },
275 | {
276 | "cell_type": "code",
277 | "execution_count": 31,
278 | "metadata": {
279 | "scrolled": true
280 | },
281 | "outputs": [
282 | {
283 | "name": "stdout",
284 | "output_type": "stream",
285 | "text": [
286 | "Maximum likelihood estimate: 0.7\n"
287 | ]
288 | },
289 | {
290 | "data": {
291 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEKCAYAAAAvlUMdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xl8FeW5wPHfk5OFAGHLBoQlLAkQcIEgEhEEFcWlUpVWtFq1VGurtdeu2vZ6vd56r733tt7aaq3WBVek1FZUXIEAyo7sewgBAgQIS0L27bl/nKHGNJBDcuZMkvN8P5/zycycd+Z5huTwnJl35h1RVYwxxphgi/A6AWOMMe2TFRhjjDGusAJjjDHGFVZgjDHGuMIKjDHGGFdYgTHGGOMKKzDGGGNcYQXGGGOMK6zAGGOMcUWk1wl4KSEhQVNTU5u1bmlpKZ06dQpuQq08drjF9TK27XN4xG6r+7xmzZpCVU1ssqGqhu0rMzNTm2vhwoXNXrelvIodbnG9jG37HB6x2+o+A6s1gP9j7RSZMcYYV1iBMcYY4worMMYYY1xhBcYYY4wrrMAYY4xxhRUYY4wxrrACY4wxxhVhfaOlMcY7qsqh4kpyC0vIPVLK+twqKhIKGJzUiX49OhEdad9/2zorMMaYkKqqqeOd9Qf486e72Xqw+Evv/WXHGgDiYiK5Las/37p4AAmdY7xI0wSBFRhjTEioKrNX7+O3H+/gUHElaUmd+eU1wxjaswsDEzuxfvVyUoaOZNeREj7Zcpg/LtrF85/u5hsX9ucnVw4hNtrn9S6Ys2QFxhjjuqKyah762wbmbSxgdP/uPH7juUxMT0RE/tFmR6Rwbp9unNunG9eP7MOuIyU8k72LF5fuZsXuo/zptkz6dO/o4V6Ys2UnOY0xrlq37wRXP7mEjzYf4sGrhjL7O1lMGpL0peLSmEGJnfmfr53H87ePZu+xMq77w2cs3VUYoqxNMFiBMca4ZuXuY9zy3HJE4C/3ZHHPJYOIiDhzYWno0qHJvH3vOHp0iuabz69k0Y4jLmVrgs0KjDHGFctzj3LHiyvp1bUDb333Ikb2697sbQ1M7MzfvncR6clxfO/VNWzaXxTETI1brMAYY4Juee5R7nxxFb27xfLG3WNJ6tKhxduM6xDFi3deQLeO0dz50ir2HSsLQqbGTVZgjDFBtbuwlLteXk1K91jeuGssSXEtLy6nJHfpwEt3XkBldS13vLiSkxXVQdu2CT4rMMaYoDlZUc1dL68mMkJ48Y4LSIwL/j0saclx/Om20ewuLOVX724N+vZN8FiBMcYERV2d8sCb69ldWMpT3xhF3x7uXVKcNSieey4ZxJur9/HJlkOuxTEtYwXGGBMUTy7YySdbD/Gv1wzjokEJrsf7weVpDO0Zx4NvbeRYaZXr8czZswJjjGmxNXuO8+T8ndwwMoXbL0oNScyYSB9P3HQ+ReVV/OvfN+F/VLxpTazAGGNapKyqhh//ZT29usby71OHN3kDZTAN69WFf7k8nfc2HmT+1sMhi2sCYwXGGNMij7+/jd2Fpfzv184jrkNUyOPfPWEgAxM78Z/vb6W6ti7k8c3pWYExxjTbkp1HeHnZHmZcPICsQfGe5BDli+DnVw0j90gpb6zc60kOpnFWYIwxzVJWVcODf93I4KTO/OTKIZ7mctmwJLIGxvPExzsoKrd7Y1oLVwuMiEwRke0ikiMiDzbyfoyIvOm8v0JEUuu995CzfLuIXOks6ysiC0Vki4hsFpEf1GvfQ0Q+FpGdzs/mj0thjGnSUwtz2H+inP+8/hw6RHk7lL6I8ItrhnGivJqnF+Z4mov5gmsFRkR8wFPAVUAGcLOIZDRoNgM4rqqDgSeAXzvrZgDTgeHAFOBpZ3s1wI9UNQMYC9xbb5sPAvNVNQ2Y78wbY1yw60gJzy7O5YZRKYwZ0MPrdAAYkdKVG0b24cXP8mwYmVbCzSOYMUCOquaqahUwC5jaoM1UYKYzPQe4TPyXoEwFZqlqparuBnKAMap6UFU/B1DVk8BWIKWRbc0EvurSfhkT1lSVR+ZupkOUj4euGuZ1Ol/yoyvSUZRnF+d6nYrB3QKTAuyrN5/PF8Xgn9qoag1QBMQHsq5zOm0ksMJZlKyqB53pAiC5pTtgjPln8zYWsGRnIT++YogrQ8G0RO9usdw4qg9vrt7H4ZMVXqcT9sStm5NEZBowRVW/7czfBlyoqvfVa7PJaZPvzO8CLgQeAZar6qvO8ueB91V1jjPfGVgEPKaqbznLTqhqt3rbPq6q/9QPIyJ3A3cDJCcnZ86aNatZ+1dSUkLnzp2btW5LeRU73OJ6Gbu17nNVrfLgknI6RwmPXNSBiCDe8xKsfS4oreOhJeVcNSCKrw+JDmnss9Vaf89NmTRp0hpVHd1kQ1V15QVkAR/Wm38IeKhBmw+BLGc6EigEpGHbBu2inPkfNtjWdqCXM90L2N5UjpmZmdpcCxcubPa6LeVV7HCL62Xs1rrPf8zO0f4/e1eX5hSGNO7Zuu/1zzXjX9/XE6VVIY99Nlrr77kpwGoNoA64eYpsFZAmIgNEJBp/p/3cBm3mArc709OABU7yc4HpzlVmA4A0YKXTP/M8sFVVf3uGbd0OvB30PTImjB0vreKphTlcOjTJs3teAvW9iYMorapl5rI8r1MJa64VGPX3qdyH/2hjKzBbVTeLyKMicp3T7HkgXkRygB/iXPmlqpuB2cAW4APgXlWtBcYBtwGXisg653W1s63HgckishO43Jk3xgTJUwtzKK2s4WdThnqdSpOG9erCZUOTeOGz3ZRW1nidTtiKdHPjqjoPmNdg2cP1piuAr51m3ceAxxos+xT/KbTG2h8FLmthysaYRuw7VsbLy/YwLbMPQ3rGeZ1OQL43aRA3/nEZb32ez21ZqV6nE5bsTn5jTJN++/EOROCByelepxKwUf26MyKlC68s32MjLXvECowx5ox2HjrJ39ft585xA+jVNdbrdAImInxzbCo7DpWwcvcxr9MJS1ZgjDFn9Lv5O+kY5eM7EwZ6ncpZ+8p5vekaG8XLy/d4nUpYsgJjjDmtHYdO8t7Gg9wxLpXunQK7p6Q1iY328bXMPny4qYDDxXbjZahZgTHGnNbv5u+kU3Qk37647R29nHLr2P7U1ClvrNzXdGMTVFZgjDGN2l5wknkbD3LHRW3z6OWU1IROTEhP5PWVe+yBZCFmBcYY06gnTx29jB/gdSot9s2x/TlUXGmPVQ4xKzDGmH+Sc7iEeZv8Ry/dOrbdo5dTJg5JJCkuhjlr8r1OJaxYgTHG/JNnFu0iJjKCO8elep1KUET6Irh+ZArZ2w9TWFLpdTphwwqMMeZLjpbX8fe1+5l+QT/iO7eu4fhb4sbMPtTUKW+vO+B1KmHDCowx5ks+yPM/0/6uNnjfy5mkJ8dxbp+u/NVOk4WMFRhjzD8cK61iUX4NU89PIaVb27lrP1A3jurDloPFbDlQ7HUqYcEKjDHmH15amkdVLdxzSfs6ejnluvN6E+UT/vq5HcWEghUYYwwAZVU1zFyax8gkH2nJbWPE5LPVvVM0lw1N5u11++2emBCwAmOMAeAvq/MpKq/m6gFRXqfiqmmZfSgsqWLxjiNep9LuWYExxlBbpzz/6W5G9utGWnef1+m46pIhiXTrGMU76+1qMrdZgTHG8PGWAvYeK+Ou8e2z76W+KF8EU4b35OMth6iorvU6nXbNCowxhueW7KZP91iuyEj2OpWQuPbc3pRW1ZK93YaOcZMVGGPC3Od7j7Nmz3G+NW4Akb7w+C9h7MAexHeK5p0NB71OpV0Lj78mY8xpPb9kN3EdIvn6BX29TiVkIn0RXHVOTxZsPUxZVY3X6bRbVmCMCWP5x8t4f9NBbhnTj84xkV6nE1LXntub8upaG2HZRVZgjAljryzb4392/UWpXqcSchek9iAxLoZ3N9jVZG6xAmNMmCqrquGNlXu5cnhyuxwWpim+COGac3qxcPsRymvU63TaJSswxoSpv63dT3FFDXdc1PYfKNZc157bi6qaOtYetsuV3WAFxpgwpKq89Fkew3t34YLU7l6n45lR/bqT3CWGNYeso98NVmCMCUNLdx1l5+ES7rgoFRHxOh3PREQIkzOS2VhYazddusAKjDFh6MXPdhPfKZqvnNfb61Q8d+XwnlTVwpKdhV6n0u5YgTEmzOw9Wsb8bYe5eUw/OkS173HHAnHhgHhiI+HDzQVep9LuWIExJsy8umIPESLcOra/16m0CtGREZyf6GP+1kPU2BD+QWUFxpgwUl5Vy5ur9nHl8GR6du3gdTqtxqjkSI6XVbMq77jXqbQrVmCMCSPvrD9AUXk1t41N9TqVVuWcBB/RkRF2mizIrMAYEyZUlZeX55Ge3JmxA3t4nU6r0iFSmJCWwMdbDqFqN10GixUYY8LE2n0n2LS/mNuywvvS5NO5YnhP9p8oZ/OBYq9TaTeswBgTJl5emkdcTCQ3jEzxOpVW6fJhyUQIfGSnyYLGCowxYaCwpJJ5Gwu4MbMPncJs1ORA9egUTWb/7iywh5AFjRUYY8LA7NX7qKqt49ax/bxOpVW7dGgym/YXU1BU4XUq7YKrBUZEpojIdhHJEZEHG3k/RkTedN5fISKp9d57yFm+XUSurLf8BRE5LCKbGmzrERHZLyLrnNfVbu6bMW1FbZ3y2vK9ZA2MZ3BSnNfptGqXDUsCYKEdxQSFawVGRHzAU8BVQAZws4hkNGg2AziuqoOBJ4BfO+tmANOB4cAU4GlnewAvOcsa84Sqnu+85gVzf4xpqxbtOMz+E+XclmU3VjYlLakzfbrH2kPIgsTNI5gxQI6q5qpqFTALmNqgzVRgpjM9B7hM/Je3TAVmqWqlqu4GcpztoaqLgWMu5m1Mu/LKsj0kxcUwOSPZ61RaPRHhsqFJfJZTaINfBoGbBSYF2FdvPt9Z1mgbVa0BioD4ANdtzH0issE5jRa+Y5Ab49h3rIzsHUeYPqYfUT7rcg3EpcOSKa+uZVnuUa9TafPa0+UkfwT+A1Dn52+AbzVsJCJ3A3cDJCcnk52d3axgJSUlzV63pbyKHW5xvYwdrLizt1chQGptPtnZgT0auK3vc0tjV9UqMT54Zf5a5GBMyOKGWkhiq6orLyAL+LDe/EPAQw3afAhkOdORQCEgDdvWb+fMpwKbzhD7jO+femVmZmpzLVy4sNnrtpRXscMtrpexgxG3orpGRz76kd798qqQx26O1vR7vmvmKr3ov+ZrXV1dSOOGUktiA6s1gDrg5jHzKiBNRAaISDT+Tvu5DdrMBW53pqcBC5zk5wLTnavMBgBpwMozBRORXvVmrwc2na6tMeHgg00FHCutslGTm+GyYUnsP1HO9kMnvU6lTXOtwKi/T+U+/EcfW4HZqrpZRB4VkeucZs8D8SKSA/wQeNBZdzMwG9gCfADcq6q1ACLyBrAMGCIi+SIyw9nWf4vIRhHZAEwCHnBr34xpC15dvofU+I6MG5TgdSptzqQh/suV7WqylnG1D0b9lwrPa7Ds4XrTFcDXTrPuY8BjjSy/+TTtb2tRssa0I9sLTrIq7zi/uHoYERE27tjZSurSgREpXVi0/Qj3ThrsdTptll1WYkw79NqKPURHRjAts4/XqbRZE9OTWLP3OEXl1V6n0mZZgTGmnSmtrOGtz/dz7Tm96N4p2ut02qxLhiRSW6cszSn0OpU2ywqMMe3M3PUHKKms4Rs27liLjOzbjbgOkWRvP+J1Km2WFRhj2hFV5dXlexjaM45R/exe45aI9EUwPi2BRTuO2EPImskKjDHtyIb8IjYfKOYbF/azh4oFwcT0JAqKK9hWYJcrN4cVGGPakVeX76FjtI+v2kPFguKSIYkALNphp8mawwqMMe1EUVk172w4wFdHphDXIcrrdNqF5C4dGNozjmwbvr9ZzngfjIiMOtP7qvp5cNMxxjTXW2vzqaiu45Yx1rkfTBOHJPHnJbmcrKi2wn2WmrrR8jfOzw7AaGA9/rHCzgVW4x9vzBjjMVXltRV7Ob9vN0akdPU6nXblkvREnlm0i6W7jnLl8J5ep9OmnPEUmapOUtVJwEFglKqOVtVMYCSwPxQJGmOatmL3MXIOl/CNC+3oJdhGp3anc4xdrtwcgfbBDFHVjadmVHUTMMydlIwxZ+u1FXvp0iGSr5zX2+tU2p0oXwRZg+JZstMuVz5bgRaYDSLyZxGZ6LyeAza4mZgxJjCFJZV8sOkg0zL70iHK1/QK5qxNSEsg/3g5eUfLvE6lTQm0wNwJbAZ+4Ly2OMuMMR6bvXof1bXKLXZ6zDUT0v2XKy/ZaafJzkZAoymraoWIPAV8gv+JkdtV1UaAM8ZjtXXK6yv2kjUwnsFJnb1Op93qH9+Jfj06snjHEb6Zlep1Om1GQEcwIjIR2An8AXga2CEiE1zMyxgTgMU7jpB/vNzGHQuB8WkJLNt1lKqaOq9TaTMCPUX2G+AKVb1EVScAVwJPuJeWMSYQr63YQ0LnGK7IsMtn3TYhPZHSqlo+33vc61TajEALTJSqbj81o6o7ALvjyBgP7T9RzoJth5l+QV+iI21QDrdlDYrHFyHWD3MWAv2rXN3IVWSr3UzMGHNmb6zYC8DN1rkfEl06RDGqXzcW77DnwwQq0ALzXfxXjt3vvLY4y4wxHqiqqWPWqn1MGpJESrdYr9MJG+PTEtl0oIijJZVep9ImBFRgVLUSfwf/vwEPA39wlhljPPDRlgIKSyq5dWx/r1MJK+PTElCFT+0plwGxq8iMaYNeWbaHvj1i/3F/hgmNc/t0o2tsFEt2WoEJRED3wfDFVWTbAUQkHXgDyHQrMWNM43YcOsmK3cd48Kqh+CLsoWKh5IsQxg2O59OdhaiqPdStCXYVmTFtzKvL9xAdGcHXR/f1OpWwdPHgRAqKK9h1pMTrVFo9u4rMmDakpLKGtz7fz7Xn9KJHp2iv0wlL49MSAOw0WQDsKjJj2pC/r91PSWUNt2VZ575X+vboSP/4jnxqBaZJgY5FVgn81nkZYzygqry6fA8jUrpwft9uXqcT1sanJfDW5/upqqmzm1zPINCryMaJyMciskNEck+93E7OGPOFVXnH2VZwktvG9rfOZY9dPDiRsqpa1tqwMWcU6FVkzwMPAGuAWvfSMcaczsvL8uyhYq1E1qB4IsR/P8yFA+O9TqfVCvTYrkhV31fVw6p69NTL1cyMMf9wqLiCDzYVcNMFfekYHej3QuOWrrFRnNe3m3X0N+GMBUZERonIKGChiPyPiGSdWuYsN8aEwOsr9lKranfutyLjByewIf8ERWX2aKzTaeqr0G8azI+uN63ApcFNxxjTUFVNHa+v3MukIUn0j+/kdTrGcXFaIk8uyGFZbiFTRvTyOp1W6YwFRlUnhSoRY0zjPthcwJGTlXzTLk1uVUb260anaB+Ld1qBOZ0zFhgRuVVVXxWRHzb2vqraZcvGuOzlpXmkxndkQpqNO9aaRPkiyBoUz2c28OVpNdXJf+p4PO40L2OMizbtL2L1nuPclpVKhI071uqMG5zAnqNl7DtW5nUqrVJTp8j+5Pz899CkY4ypb+bSPGKjfEzL7ON1KqYR9YeNucUe/PZPmjpF9uSZ3lfV+5tYfwrwO8AH/FlVH2/wfgzwMv5RmY8CN6lqnvPeQ8AM/Pfd3K+qHzrLXwCuBQ6r6oh62+oBvAmkAnnA11XV7oIybdbRkkreXn+Ar4/uQ9dYG1u2NRqU2JmeXTrwac4RKzCNaOoU2ZomXqclIj7gKeAqIAO4WUQyGjSbARxX1cHAE8CvnXUzgOnAcGAK8LSzPYCXnGUNPQjMV9U0YL4zb0yb9cbKvVTV1HHHRalep2JOQ0S4OC2Bz3KOUlunXqfT6jR1imxm/XkR6aiqgZ5sHAPkqGqus+4sYCr+gTJPmQo84kzPAf4g/jEwpgKznDHQdotIjrO9Zaq6WERSG4k3FZjoTM8EsoGfBZirMa1KTZ3yyvI9jE9LYHCSdXe2ZuPTEpizJp9N+4s4z8aI+5JAxyLLEpEtwDZn/jwRebqJ1VKAffXm851ljbZR1RqgCIgPcN2GklX1oDNdACQ30d6YVmv1oVoOFVfyrXEDvE7FNGHcYH8/jD1G+Z8FOubE/wFXAnMBVHV9a35ksqqqiDR6vCoidwN3AyQnJ5Odnd2sGCUlJc1et6W8ih1ucb2M/UFuBckdI9CDm8ku2NL0CkFkv+ez1zcugndX7WS45Ic0bkuEJLaqNvkCVjg/19Zbtr6JdbKAD+vNPwQ81KDNh0CWMx0JFALSsG39ds58KrCpwba2A72c6V7A9qb2KzMzU5tr4cKFzV63pbyKHW5xvYq9du9x7f+zd/XFT3NDHlvVfs/N8dh7WzTt5/O0rLImpHFboiWxgdUaQO0IdLDLfSJyEaAiEiUiPwa2NrHOKiBNRAaISDT+Tvu5DdrMBW53pqcBC5zk5wLTRSRGRAYAacDKJuLV39btwNuB7Jgxrc0Ln+4mNhKm2SOR24xxgxOoqq1jxW4bA7i+QAvMPcC9+PtB9gPnO/Onpf4+lfvwH31sBWar6mYReVRErnOaPQ/EO534P8S58ktVNwOz8V8Q8AFwr6rWAojIG8AyYIiI5IvIDGdbjwOTRWQncLkzb0ybcuBEOe9tPMglfaLoHGOjJrcVY1J7EO2LsKdcNhDoX3Cdqn6j/gLnyOKM5VpV5wHzGix7uN50BfC106z7GPBYI8tvPk37o8BlZ8rHmNZu5tI8ACb3t+LSlsRG+7hgQHfr6G8g0COYd0Sky6kZERkGvONOSsaEp5LKGl5fuZerz+lFfKw9hretuXhwItsKTnK4uMLrVFqNQP+K/xN/keksIpn471m51b20jAk/f1m9j5MVNcy42C5NbotODRtjRzFfCKjAqOp7+O+0/wj/nfTXq+o6F/MyJqzU1ikvfLab0f27c77drNcmZfTqQnynaHvKZT1NjUX2e/wPFjulK7ALuE9EmhyLzBgTmI+3FLDvWDm/uHqY16mYZoqIEMYNTmDJzkJUFf+gJOGtqZ7E1Q3mzzj+mDHm7KkqzyzKpX98RyZn9PQ6HdMC49MSmLv+ANsKTjKsV5emV2jnzmosMmNM8K3cfYx1+07wq6+OwGfPfGnTxjsPhVuy84gVGJrogxGR2c7PjSKyoeErNCka0779aXEu8Z2i7Zkv7UDPrh1IS+ps/TCOpk6R/cD5ea3biRgTjrYXnGTBtsP8aHI6HaJ8Ta9gWr3xaYm8tmIPFdW1Yf87PeMRjDqjE6vqnsZeoUnRmPbr2cW5xEb5uC2rv9epmCAZn55AZU0dq/KOeZ2K55o6RXZSRIobeZ0UkeJQJWlMe3SwqJy31+1n+pi+dOsY7XU6JkguHGDDxpzS1BFMnKp2aeQVp6rWg2VMC/x5yW4U7MbKdqZjdCSZ/buzaMcRr1PxnI1HYYwHjpVW8fqKvUw9vzd9unf0Oh0TZBPS/cPGHArzYWOswBjjgRc+3U1FTS3fmzjY61SMCy5J91+uvDjMj2KswBgTYsUV1cxclsdVI3oyOKmz1+kYFwzrFUdiXAyLw7wfxgqMMSH2yrI9nKyosaOXdkxEmJCWyJKdR6ita/Tp7WHBCowxIVRWVcOfl+QyaUgiI1K6ep2OcdGE9AROlFWzIf+E16l4xgqMMSH0+oq9HC+r5r5L7eilvRuflogILN4RvqfJrMAYEyLlVbU8syiXiwbFk9m/h9fpGJf16BTNuSldWbTjsNepeMYKjDEh8tqKPRSWVPLA5HSvUzEhMiE9kXX7TlBUVu11Kp6wAmNMCJRV1fDMol1cPDiBC1Lt6CVcXJKeSJ2G71MurcAYEwKvLt9DYUkVD0xO8zoVE0Ln9+1GXIfIsD1NZgXGGJeVVdXwp0W5jE9LsL6XMBPpi+DiwQks2nEE1fC7XNkKjDEum7l0D0dLq6zvJUxNGpLEoeJKthwMv/GBrcAY46Ki8mqeWbSLiUMSGdWvu9fpGA9MHOofNiZ7e/gNG2MFxhgXPbt4F0Xl1fz0yqFep2I8khTXgXNSurJgW/j1w1iBMcYlh4sreOHTPK47rzcZve3pFuFs0pBE1u49zvHSKq9TCSkrMMa45PcLcqiureOH1vcS9iYNTaJOYfHO8DpNZgXGGBfsPVrGGyv3Mn1MX1ITOnmdjvHYeX26Ed8pOuxOk1mBMcYF//vRdiJ9wv2X2n0vBiIihEvSE1m0I7xGV7YCY0yQrd17nLnrD/DtiweS1KWD1+mYVmLS0CROlFWzbt9xr1MJGSswxgSRqvKr97aSGBfDdycO8jod04pMSEvEFyEs3BY+/TBWYIwJonkbC1iz5zg/mpxOp5hIr9MxrUjXjlFk9uvO/DDqh7ECY0yQVFTX8vgHWxnaM46vje7rdTqmFbo8I4mtB4vZd6zM61RCwgqMMUEyc2ke+46V88trMvBFiNfpmFZockZPAD7ZesjjTELDCowxQXCouIIn5+/k0qFJXJyW4HU6ppUakNCJtKTOfLTZCowxJkD/OW8r1bXKw9dmeJ2KaeUmZySzMu8YJ8ra/139rhYYEZkiIttFJEdEHmzk/RgRedN5f4WIpNZ77yFn+XYRubKpbYrISyKyW0TWOa/z3dw3Y05ZnnuUt9cd4DuXDLSbKk2Trhjek9o6DYubLl0rMCLiA54CrgIygJtFpOHXuxnAcVUdDDwB/NpZNwOYDgwHpgBPi4gvgG3+RFXPd17r3No3Y06prq3j397eTEq3WL43cbDX6Zg24NyUriTFxfDxlvZ/mszNI5gxQI6q5qpqFTALmNqgzVRgpjM9B7hMRMRZPktVK1V1N5DjbC+QbRoTMi8v28P2Qyf512sziI32eZ2OaQMiIoTJGcks2nGEqtr2fVe/mxfqpwD76s3nAxeero2q1ohIERDvLF/eYN0UZ/pM23xMRB4G5gMPqmplw6RE5G7gboDk5GSys7PPbq8cJSUlzV63pbyKHW5xm4p9tLyO//m0nHMSfMQc2Up29raQxHWb/Z7dl1xTQ1lVLZ/vLyW6He9ze7oT7CGgAIgGngV+BjzasJGqPuu8z+jRo3XixInNCpadnU1z120pr2KHW9wzxVZVZsxcjURU8fS3JtC3R8eQxA0F+z27L6sZD8cgAAASXElEQVSmlmc3fcLWYniwHe+zm6fI9gP17zbr4yxrtI2IRAJdgaNnWPe021TVg+pXCbyI/3SaMa54d8NBFmw7zI+uSA96cTHtX0ykj4lDEll7uIaa2jqv03GNmwVmFZAmIgNEJBp/p/3cBm3mArc709OABaqqzvLpzlVmA4A0YOWZtikivZyfAnwV2OTivpkwdqKsin9/ZzPn9enKneMGeJ2OaaOuOacXxVWwcvcxr1NxjWunyJw+lfuADwEf8IKqbhaRR4HVqjoXeB54RURygGP4CwZOu9nAFqAGuFdVawEa26YT8jURSQQEWAfc49a+mfD2H+9u5XhZNS9/60K7Y98028QhScT44N2NB7locPu8OdfVPhhVnQfMa7Ds4XrTFcDXTrPuY8BjgWzTWX5pS/M1pikfbS7gr5/nc9+kwfYYZNMisdE+zk/08cGmAh69bjiRvvZ333v72yNjXHK0pJKf/20jGb26cP9l9iAx03JjekVyrLSK5bnt8zSZFRhjAqCq/PxvGykur+GJm84nOtI+Oqblzknw0Snax7sbDnidiivsU2JMAP62dj8fbj7Ej69MZ0jPOK/TMe1EtM9/0+UHmwuobodXk1mBMaYJeYWlPPz2Zsak9mDGxQO9Tse0M9ec25sTZdUs3XXU61SCzgqMMWdQXafc+/rnRPqE/5t+vl01ZoJuQnoCcTGRvNcOT5NZgTHmDN7cVsXmA8X877Tz6N0t1ut0TDsUE+lj8vBk3t9UQEV1rdfpBJUVGGNO44NNBXyyt4YZFw/g8oxkr9Mx7diNo/pwsqKm3Y2wbAXGmEbsOlLCT/6yngFdIvjZlKFep2PauayB8aR0i2XOmnyvUwkqKzDGNFBcUc1dL68mKjKCe0fG2CXJxnUREcKNo1JYsvMIBUUVXqcTNPbJMaaeujrlgVnr2Hu0jKe/MYqEWPuImNC4MbMPdQpvrW0/RzH26TGmnic+2cH8bYd5+CsZjB0Y73U6Joz0j+/EmNQezFmTj3/M37bPCowxjtmr9/H7BTncNLovt43t73U6JgxNG92H3COlfL73hNepBIUVGGOARTuO8NBbGxmflsCvrh+B/6kPxoTW1ef0IjbK1246+63AmLC3aX8R33t1DenJcTz9jVFEtcNRbU3b0DkmkqvO6ck76w9QUlnjdTotZp8kE9Zyj5Rwx4ur6BobxUt3XkBchyivUzJh7rax/SmprOGv7eAoxgqMCVt7j5Zxy3MrUFVenjGG5C4dvE7JGEb26875fbsxc2kedXVtu7PfCowJS/nHy7j5ueVU1NTy6rcvZHCSjZBsWo87x6WSW1jKop1HvE6lRazAmLCz75j/yKW4oppXZ1zIsF72ZErTulw1ohdJcTG8+Fme16m0iBUYE1Z2HjrJtGeWUlRezSszLmRESlevUzLmn0RHRnDr2P4s3nGEnMMlXqfTbFZgTNhYv+8EX//TMuoU3vzOWM7v283rlIw5rVsu7Ee0L4KZS/O8TqXZrMCYsPDJlkPc8txyOneIZM49WQztaafFTOuW0DmGr5zXmzlr8iksqfQ6nWaxAmPaNVXlucW53PXKagYmdmbOPRfRP76T12kZE5DvThxEZU0tz2Tv8jqVZrECY9qtiupaHvzrRh6bt5WrRvRk9ney7FJk06YMTurM9SP78MryPW1ylGUrMKZdyiss5Yanl/Lm6n18/9LB/OHmUcRG+7xOy5iz9i+Xp1Fbpzy1MMfrVM6aFRjT7ry74QDX/v5TDhSV8/zto/nRFUOIiLCxxUzb1LdHR266oC+zVu1l37Eyr9M5K1ZgTLtxvLSKH8xay32vryU9uTPv3T+ey4bZo45N23ffpYMREX6/YKfXqZwVKzCmXfhocwGTn1jMexsO8i+Xp/Hmd7JI6RbrdVrGBEWvrrHcNrY/c9bksyG/7QzlbwXGtGl5haXMeGkVd7+yhqS4GObedzH/cnm6jYhs2p37L0sjMS6Gn87ZQHVtndfpBMQ+haZNKq6o5r8/2MYVTyxmee5Rfn71UP5+7zgyetv9LaZ96hobxa++eg7bCk62mcuWI71OwJizUVZVw0tL8/jTolyKyqu5YVQKD04ZSpJdfmzCwOSMZK49txe/X5DDlBE9SUtu3YO0WoExbcKx0ipeW76HmcvyKCyp4tKhSfxwcrqNJWbCziPXDefTnEJ++tcNzP5OVqs+HWwFxrRqWw4UM3NzJcvmz6eiuo5L0hO5/7I0Mvt39zo1YzyR0DmGR6eO4P431vLLv23i8RvPabWP+LYCY1qd46VVvLfxILNX72NDfhGRAjdk9uHb4weS3spPCRgTCted15ucQyd5ckEOfbrH8v3L0rxOqVFWYEyrcLi4ggXbDvPexoMs3XWU2jplaM84/u0rGSSW5nHtFed5naIxrcoDk9PJP17Obz7eQUr3WG4Y1cfrlP6JFRjjiZLKGj7fc5xluUdZtP0IWw4WA9A/viN3TxjINef0YnjvLogI2dl7PM7WmNZHRHj8xnM5WFTBT+ds4ERZNXeOS21Vp8tcLTAiMgX4HeAD/qyqjzd4PwZ4GcgEjgI3qWqe895DwAygFrhfVT880zZFZAAwC4gH1gC3qWqVm/tnAlNZU0vO4RI27S9i4/4i1u07wZYDxdQpREYImf2789MpQ5iYnsSwXnGt6gNiTGsWHRnBs9/M5IE31/Pou1tYs/c4v77xXDrHtI5jB9eyEBEf8BQwGcgHVonIXFXdUq/ZDOC4qg4WkenAr4GbRCQDmA4MB3oDn4hIurPO6bb5a+AJVZ0lIs842/6jW/tnvqysqoaDRRUcOFHO3mNl7D1axu7CUnIOl7DnWBm1dQpA55hIRqR04b5Jg7lgQA9G9uveaj4MxrRFcR2ieO6bmfxpcS7//cE2Nu8v4vuXpnHteb2IifR2gFc3P9ljgBxVzQUQkVnAVKB+gZkKPOJMzwH+IP6vr1OBWapaCewWkRxnezS2TRHZClwK3OK0mels1wrMGagqtXVKda1SVVtHZU0tVTV1VFTXUV5VS3l1LaVVNZRU1FBaWcPa3dWsqtzGibJqTpRVU1hSydHSKg4XV1BcUfOlbUf7IugX35H05DiuObcXaclxjOjdhdT4TjbwpDFBJiLcc8kgzu/bjV/+fRM/+st6/uv9bdw8pi+Z/bsztGcXkrvEhPzsgJsFJgXYV28+H7jwdG1UtUZEivCf4koBljdYN8WZbmyb8cAJVa1ppH3Q/X7+Tt5YVkanzxe1eFsaSBv9opUCZWVldFyd/Y91VRUFVEFR6pxRJOpUqVOlts4pJqrU1io1dU5hqatDA0mgHt/OXLrGRtGtYxQJnWNIT+7MRYPi6dm1A726dqBX11j6x3ckOa6DFRJjQmzswHg+fmACS3YW8uJnu/n9gi+G+I+LiaRTTCRRkUK0L4KvD6hlosv5hN25CRG5G7gbIDk5mezs7LPexrED1STH1OGT8iDlFECbetM1sXVERlYgDdb1z8s/lp/6GQFEiDMt4BMhQiKIjPDhE/BFQFSEEBUBkREQ7RNifBDjE6J9EBspdPBBXWUZCV071vsWVOm8Tvpni6CiCLbvhe0t/lf5QklJSbN+T205tu1zeMR2M+7tA+DGlI7kl9SRf7KOgtI6qupqqK5TqmtBq2rc32dVdeUFZAEf1pt/CHioQZsPgSxnOhIoxP//4pfanmp3um066xQCkY3FPt0rMzNTm2vhwoXNXrelvIodbnG9jG37HB6x2+o+A6s1gDrg5hgDq4A0ERkgItH4O+3nNmgzF7jdmZ4GLHCSnwtMF5EY5+qwNGDl6bbprLPQ2QbONt92cd+MMcY0wbVTZOrvU7kP/9GHD3hBVTeLyKP4q99c4HngFacT/xj+goHTbjb+CwJqgHtVtRagsW06IX8GzBKRXwFrnW0bY4zxiKt9MKo6D5jXYNnD9aYrgK+dZt3HgMcC2aazPJcvrjQzxhjjsdY7DKcxxpg2zQqMMcYYV1iBMcYY4worMMYYY1xhBcYYY4wrRM92rJB2RESOAM0dCz4B/82dXvAqdrjF9TK27XN4xG6r+9xfVRObahTWBaYlRGS1qo4Op9jhFtfL2LbP4RG7ve+znSIzxhjjCiswxhhjXGEFpvmeDcPY4RbXy9i2z+ERu13vs/XBGGOMcYUdwRhjjHGFFZgmiMgUEdkuIjki8mAj78eIyJvO+ytEJDVEcSeIyOciUiMi0xrbhouxfygiW0Rkg4jMF5H+IYp7j4hsFJF1IvKpiGQEI24gseu1u1FEVESCcvVNAPt8h4gccfZ5nYh8OxRxnTZfd37Pm0Xk9WDEDSS2iDxRb393iMiJEMXtJyILRWSt87d9dTDiBhi7v/NZ2iAi2SLSJ0hxXxCRwyKy6TTvi4g86eS1QURGBSPuPwTy0JhwfeF/JMAuYCAQDawHMhq0+R7wjDM9HXgzRHFTgXOBl4FpId7nSUBHZ/q7IdznLvWmrwM+CNU+O+3igMX4H+c9OkT7fAfwBw/+rtPwP/aiuzOfFMp/63rtv4//sRyh2Odnge860xlAXgj/vf8C3O5MXwq8EqTYE4BRwKbTvH818D7+hzaOBVYE82/NjmDObAyQo6q5qloFzAKmNmgzFZjpTM8BLhMJ5CHILYurqnmqugGoa2Gs5sReqKplzuxyIBjftgKJW1xvthMQrA7EQH7PAP8B/BqoCHHcYAsk7l3AU6p6HEBVD4cwdn03A2+EKK4CXZzprsCBIMQNNHYGsMCZXtjI+82iqovxP2vrdKYCL6vfcqCbiPQKRmywU2RNSQH21ZvPd5Y12kZVa4AiID4Ecd1ytrFn4P8GFJK4InKviOwC/hu4PwhxA4rtnDroq6rvBSlmQHEdNzqnL+aISN8QxU0H0kXkMxFZLiJTghA30NiA/7QRMIAv/uN1O+4jwK0iko//mVPfD0LcQGOvB25wpq8H4kSkpf+PBCu3ZrMCY5pNRG4FRgP/E6qYqvqUqg7C/wTTX4YipohEAL8FfhSKeA28A6Sq6rnAx3xxtOy2SPynySbiP4p4TkS6hSj2KdOBOeo8zTYEbgZeUtU++E8dveL87kPhx8AlIrIWuATYD4Rqv11jBebM9gP1vzH2cZY12kZEIvEfWh8NQVy3BBRbRC4HfgFcp6qVoYpbzyzgq0GIG0jsOGAEkC0iefjPVc8NQkd/k/usqkfr/fv+GchsYcyA4uL/JjtXVatVdTewA3/BCUXsU6YTnNNjgcadAcwGUNVlQAf843W5HltVD6jqDao6Ev/nClUNysUNLc2tRYLZodPeXvi/xeXiP0w/1Tk3vEGbe/lyJ//sUMSt1/YlgtvJH8g+j8TfaZkW4rhp9aa/AqwOVewG7bMJTid/IPvcq9709cDyEMWdAsx0phPwn0aJD9W/NTAUyMO5Vy9E+/w+cIczPQx/H0yL4wcYOwGIcKYfAx4Nxn4720vl9J381/DlTv6VwYqrqlZgAvjlXI3/29su4BfOskfxf3MH/7ecvwA5wEpgYIjiXoD/W2Yp/iOmzSHc50+AQ8A65zU3RHF/B2x2Yi5s7D8mt2I3aJtNEApMgPv8X84+r3f2eWiI4gr+04JbgI3A9FD+W+PvD3k8WDED3OcM4DPn33odcEUIY08Ddjpt/gzEBCnuG8BBoNr5/2IGcA9wT73f81NOXhuD9Xd96mV38htjjHGF9cEYY4xxhRUYY4wxrrACY4wxxhVWYIwxxrjCCowxxhhXWIExxmUi0k1EvudMTxSRd89y/TtEpLc72RnjHiswxrivG/5Rt5vrDsAKjGlz7D4YY1wmIqdGz92O/4a3UqAQ//Aza4BbVVVFJBP/zY2dnffvAMbhH61hP1AOZAE/wT+SQSywFPiO2gfZtEJWYIxxmfMQundVdYSITATeBobjH4rkM/wFYwWwCJiqqkdE5CbgSlX9lohkAz9W1dXO9nqo6jFn+hX8wxO9E9q9MqZpkV4nYEwYWqmq+QAisg7/WFEn8B/RfOw8TsiHf4iPxkwSkZ8CHYEe+IeSsQJjWh0rMMaEXv3Rp2vxfw4F/3hyWWdaUUQ6AE/jHzNqn4g8gn88PGNaHevkN8Z9J/EP+X8m24FEEckCEJEoERneyPqnikmhiHTGP0iiMa2SHcEY4zJVPeo8GXIT/o76Q420qRKRacCTItIV/2fz//Cf/noJeEZETnXyPwdsAgqAVaHZC2POnnXyG2OMcYWdIjPGGOMKKzDGGGNcYQXGGGOMK6zAGGOMcYUVGGOMMa6wAmOMMcYVVmCMMca4wgqMMcYYV/w/WnirO1EwuAoAAAAASUVORK5CYII=\n",
292 | "text/plain": [
293 | ""
294 | ]
295 | },
296 | "metadata": {},
297 | "output_type": "display_data"
298 | }
299 | ],
300 | "source": [
301 | "def bernoulli_likelihood2(x, theta):\n",
302 | " return theta**x * (1-theta)**(1-x)\n",
303 | "\n",
304 | "theta = np.arange(0, 1.01, 0.01)\n",
305 | "x = np.array([1, 1, 1, 1, 1, 1, 1, 0, 0, 0])\n",
306 | "\n",
307 | "p = [np.prod(bernoulli_likelihood2(x, t)) for t in theta]\n",
308 | "disply_plot(theta, p)\n",
309 | "\n",
310 | "# Estimate Maxmum Likelihood\n",
311 | "maximum_likelihood = float(np.mean(x))\n",
312 | "print('Maximum likelihood estimate: ', maximum_likelihood)"
313 | ]
314 | },
315 | {
316 | "cell_type": "markdown",
317 | "metadata": {},
318 | "source": [
319 | "## References\n",
320 | "\n",
321 | "* http://math.arizona.edu/~jwatkins/o-mle.pdf\n",
322 | "* https://onlinecourses.science.psu.edu/stat414/node/191"
323 | ]
324 | }
325 | ],
326 | "metadata": {
327 | "kernelspec": {
328 | "display_name": "Python 3",
329 | "language": "python",
330 | "name": "python3"
331 | },
332 | "language_info": {
333 | "codemirror_mode": {
334 | "name": "ipython",
335 | "version": 3
336 | },
337 | "file_extension": ".py",
338 | "mimetype": "text/x-python",
339 | "name": "python",
340 | "nbconvert_exporter": "python",
341 | "pygments_lexer": "ipython3",
342 | "version": "3.6.4"
343 | }
344 | },
345 | "nbformat": 4,
346 | "nbformat_minor": 2
347 | }
348 |
--------------------------------------------------------------------------------