├── .gitignore ├── custom.css ├── static ├── lda.png ├── blei-f1.jpg ├── buffalo.png ├── sheldon.gif └── graphical_model.png ├── .gitmodules ├── environment.yml ├── Makefile ├── README.md └── topic-models-with-simulation.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | *.html 2 | -------------------------------------------------------------------------------- /custom.css: -------------------------------------------------------------------------------- 1 | .reveal { 2 | font-size: 200%; 3 | } -------------------------------------------------------------------------------- /static/lda.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tdhopper/pydata-nyc-2015/HEAD/static/lda.png -------------------------------------------------------------------------------- /static/blei-f1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tdhopper/pydata-nyc-2015/HEAD/static/blei-f1.jpg -------------------------------------------------------------------------------- /static/buffalo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tdhopper/pydata-nyc-2015/HEAD/static/buffalo.png -------------------------------------------------------------------------------- /static/sheldon.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tdhopper/pydata-nyc-2015/HEAD/static/sheldon.gif -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "reveal.js"] 2 | path = reveal.js 3 | url = git@github.com:hakimel/reveal.js.git 4 | -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- 1 | name: understanding-lda 2 | dependencies: 3 | - python>=3.4 4 | - anaconda 5 | - seaborn -------------------------------------------------------------------------------- /static/graphical_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tdhopper/pydata-nyc-2015/HEAD/static/graphical_model.png -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | all: 2 | jupyter notebook 3 | 4 | make install: 5 | git submodule init 6 | git submodule update 7 | conda update conda 8 | conda env create 9 | 10 | slides: 11 | jupyter nbconvert topic-models-with-simulation.ipynb --to slides --post serve -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | By Tim Hopper: 6 | [tdhopper.com](http://www.tdhopper.com) 7 | 8 | [![alt text][1.1]][1] 9 | [![alt text][6.1]][6] 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | [1.1]: http://i.imgur.com/tXSoThF.png (twitter icon with padding) 18 | [2.1]: http://i.imgur.com/P3YfQoD.png (facebook icon with padding) 19 | [3.1]: http://i.imgur.com/yCsTjba.png (google plus icon with padding) 20 | [4.1]: http://i.imgur.com/YckIOms.png (tumblr icon with padding) 21 | [5.1]: http://i.imgur.com/1AGmwO3.png (dribbble icon with padding) 22 | [6.1]: http://i.imgur.com/0o48UoR.png (github icon with padding) 23 | 24 | 25 | 26 | [1.2]: http://i.imgur.com/wWzX9uB.png (twitter icon without padding) 27 | [2.2]: http://i.imgur.com/fep1WsG.png (facebook icon without padding) 28 | [3.2]: http://i.imgur.com/VlgBKQ9.png (google plus icon without padding) 29 | [4.2]: http://i.imgur.com/jDRp47c.png (tumblr icon without padding) 30 | [5.2]: http://i.imgur.com/Vvy3Kru.png (dribbble icon without padding) 31 | [6.2]: http://i.imgur.com/9I6NRUm.png (github icon without padding) 32 | 33 | 34 | 35 | 36 | 37 | [1]: http://www.twitter.com/tdhopper 38 | [6]: http://www.github.com/tdhopper 39 | 40 | 41 | 42 | # Understanding Probabilistic Topic Models By Simulation 43 | 44 | ## Overview 45 | 46 | ### Description 47 | 48 | Latent Dirichlet Allocation and related topic models are often presented in the form of complicated equations and confusing diagrams. I will present LDA as a generative model through probabilistic simulation in simple Python. Simulation will help data scientists to understand the model assumptions and limitations and more effectively use black box LDA implementations. 49 | 50 | ### Abstract 51 | 52 | Those without training in probabilistic graphical models and measure theory, data scientist may have a hard time understanding Latent Dirichlet Allocation and other probabilistic topic models. However, because LDA is a generative model, we can write Python code to generated data based on the model assumptions. 53 | 54 | The talk will progress as follows: 55 | 56 | * Introduction to mixture models 57 | * Simulation of mixture models 58 | * Introduction to grouped data 59 | * Simulation of latent Dirichlet allocation 60 | * ~~Fitting and visualizing LDA with Python~~ 61 | 62 | ## Setup Conda Environment and Launch Notebook 63 | 64 | With [Conda](http://conda.pydata.org/ "Conda") installed, run 65 | 66 | ``` 67 | $ git clone https://github.com/tdhopper/pydata-nyc-2015.git understanding-lda 68 | $ cd understanding-lda 69 | $ make install 70 | $ source activate understanding-lda 71 | ``` 72 | 73 | To view the notebook, run 74 | 75 | ``` 76 | $ make 77 | ``` 78 | 79 | To view the notebook as a slideshow, run 80 | 81 | ``` 82 | make slides 83 | ``` 84 | 85 | -------------------------------------------------------------------------------- /topic-models-with-simulation.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": false, 8 | "slideshow": { 9 | "slide_type": "skip" 10 | } 11 | }, 12 | "outputs": [], 13 | "source": [ 14 | "%matplotlib inline\n", 15 | "import scipy as sp\n", 16 | "import numpy as np\n", 17 | "import pandas as pd\n", 18 | "import matplotlib.pyplot as plt\n", 19 | "import warnings\n", 20 | "import seaborn as sns\n", 21 | "\n", 22 | "from collections import defaultdict\n", 23 | "from itertools import groupby\n", 24 | "from sklearn import datasets\n", 25 | "from numpy import random\n", 26 | "from scipy.stats import dirichlet, norm, poisson\n", 27 | "\n", 28 | "warnings.filterwarnings(\"ignore\")\n", 29 | "sns.set_style(\"whitegrid\")\n", 30 | "random.seed(1234)" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": { 36 | "slideshow": { 37 | "slide_type": "slide" 38 | } 39 | }, 40 | "source": [ 41 | "# Understanding Probabilistic Topic Models By Simulation\n", 42 | "\n", 43 | "* Tim Hopper\n", 44 | "* [@tdhopper](https://twitter.com/tdhopper)\n", 45 | "* [stiglerdiet.com](http://www.stiglerdiet.com)\n", 46 | "* Data Science at [Distil Networks](distilnetworks.com)\n", 47 | "\n", 48 | "Slides available at http://bit.ly/lda-talk." 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": { 54 | "slideshow": { 55 | "slide_type": "slide" 56 | } 57 | }, 58 | "source": [ 59 | "# What Is a Topic Model?" 60 | ] 61 | }, 62 | { 63 | "cell_type": "markdown", 64 | "metadata": { 65 | "slideshow": { 66 | "slide_type": "slide" 67 | } 68 | }, 69 | "source": [ 70 | "# What is Latent Dirichlet Allocation?\n", 71 | "\n", 72 | "> LDA is a three-level hierarchical Bayesian model, in which each\n", 73 | "item of a collection is modeled as a finite mixture over an underlying set of topics. \n", 74 | "\n", 75 | "> Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities.\n", 76 | "\n", 77 | "Duh.\n", 78 | "\n", 79 | "[[source](https://www.cs.princeton.edu/~blei/papers/BleiNgJordan2003.pdf)]" 80 | ] 81 | }, 82 | { 83 | "cell_type": "markdown", 84 | "metadata": { 85 | "slideshow": { 86 | "slide_type": "slide" 87 | } 88 | }, 89 | "source": [ 90 | "# What is Latent Dirichlet Allocation?\n", 91 | "\n", 92 | "![](./static/lda.png)\n", 93 | "\n", 94 | "Duh." 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": { 100 | "slideshow": { 101 | "slide_type": "slide" 102 | } 103 | }, 104 | "source": [ 105 | "# What is Latent Dirichlet Allocation?\n", 106 | "\n", 107 | "![](./static/graphical_model.png)\n", 108 | "\n", 109 | "Duh." 110 | ] 111 | }, 112 | { 113 | "cell_type": "markdown", 114 | "metadata": { 115 | "slideshow": { 116 | "slide_type": "slide" 117 | } 118 | }, 119 | "source": [ 120 | "# What is Latent Dirichlet Allocation?\n", 121 | "\n", 122 | "$\n", 123 | "p(D \\,|\\, \\alpha, \\beta) =\n", 124 | "\\prod_{d=1}^M \\int p(\\theta_d \\,|\\, \\alpha )\n", 125 | "\\left(\n", 126 | "\\prod_{n=1}^{N_d}\n", 127 | "\\sum_{z_{dn}}\n", 128 | "p(z_{dn} \\,|\\, \\theta_d)\n", 129 | "p(w_dn \\,|\\, z_dn, \\beta)\n", 130 | "\\right)\n", 131 | "d \\theta_d\n", 132 | "$\n", 133 | "\n", 134 | "Duh." 135 | ] 136 | }, 137 | { 138 | "cell_type": "markdown", 139 | "metadata": { 140 | "slideshow": { 141 | "slide_type": "-" 142 | } 143 | }, 144 | "source": [ 145 | "![](./static/sheldon.gif)" 146 | ] 147 | }, 148 | { 149 | "cell_type": "code", 150 | "execution_count": 2, 151 | "metadata": { 152 | "collapsed": false, 153 | "slideshow": { 154 | "slide_type": "skip" 155 | } 156 | }, 157 | "outputs": [], 158 | "source": [ 159 | "iris = datasets.load_iris()\n", 160 | "df = pd.DataFrame(iris['data'], columns=iris['feature_names'])\n", 161 | "df['class'] = iris['target']\n", 162 | "df = df[df['class'].isin([0,2])]\n", 163 | "gby = df.groupby(by='class')['sepal length (cm)']\n", 164 | "hist_data = [gby.get_group(cid).tolist() \n", 165 | " for cid in gby.groups.keys()]\n", 166 | "\n", 167 | "settings = {\"bins\": 10, \"histtype\": 'stepfilled', \"alpha\": .5}\n", 168 | "data = {\"x\": df['sepal length (cm)']}\n", 169 | "data.update(settings)\n", 170 | "data2 = {\"x\": hist_data }\n", 171 | "data2.update(settings)\n", 172 | "title = 'Sepal Length for Iris-Setosa and Iris-Virginica'" 173 | ] 174 | }, 175 | { 176 | "cell_type": "markdown", 177 | "metadata": { 178 | "slideshow": { 179 | "slide_type": "slide" 180 | } 181 | }, 182 | "source": [ 183 | "# Naive Clustering" 184 | ] 185 | }, 186 | { 187 | "cell_type": "markdown", 188 | "metadata": { 189 | "slideshow": { 190 | "slide_type": "notes" 191 | } 192 | }, 193 | "source": [ 194 | "Can we discriminate between Setosa and Virginica irises by modeling their sepal length as a mixture of Gaussians?" 195 | ] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "execution_count": 3, 200 | "metadata": { 201 | "collapsed": false 202 | }, 203 | "outputs": [ 204 | { 205 | "data": { 206 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAFeCAYAAABQCrzXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Wlc1WX+//E3m4AgOpjWaEya5W6mkOkQaqalZe7milJM\npqVTmZOipoUL2TjVI9SynDabh46ZptN/StPUGrcYQkvMPTVFTUmSTTjA9b/hzzOZgHA8yAW8nreG\ns3y/n4vrwKtzPHPwMMYYAQAAK3iW9wAAAOB/CDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzNCO\nHTs0YsQI9e7dWw8++KBGjRqlAwcOlMm5vv76az344IOXXX78+HG1adOmTM75W999952mT59e7DyF\nmT59urp27apXX33V5XPPmzdPM2fOLPS61157TatWrSrV8U6fPq2nn35aDz74oHr16qVBgwZp/fr1\nJbpvdHS00tLSSnW+8nD27Fk1bdq00OtiYmL0zjvvFHrd1KlTtXXr1hKfJyYmRtOmTbvs8jVr1qhP\nnz766aefNGTIkBIf76LHHntMBw8eLPY2ruw9Ki/v8h4A5Ss3N1ejR4/Wu+++6/zlt3r1ao0aNUrr\n16+Xh4fHNZvlWp1r//79OnXqVKnvt2zZMm3cuFHXX399GUwl/fnPfy71faZOnao//vGPeuWVVyRJ\nBw8e1NChQ7VkyRLdfPPNxd538+bNLs15rRljXHpsFPUfQEUZNmyYHn74YU2dOlXVqlVzXr5s2TIN\nGzZMdevW1ZIlS0o9x8KFC694G1f2HpUXYa7izp8/r4yMDGVkZDgv69Wrl2rUqKH8/Hx5e3trw4YN\nev3115WXlyc/Pz9NnDhRrVu31rx587R//36dOXNGZ86cUfPmzTVz5kwFBARow4YNWrhwofLy8vTz\nzz+rd+/eevLJJ12a0eFwaO7cuUpISFBBQYGaNWumqVOnKiAgQF26dFG/fv20detWnThxQj169NBf\n/vIXSdKbb76pjz76SAEBAQoLC9O6deu0ZMkSxcfHKyMjQ5MnT1afPn2UmZmp8ePH69ChQ8rNzdWM\nGTMUGhp6yQzDhg2TJD366KOaPn26atasqdjYWKWlpcnT01NRUVHq06ePvv76a82aNUv+/v46f/68\nPvzwQ/n4+BS6rpiYGKWlpenYsWPq3Lmzzpw5o8aNG+vhhx/Wa6+9pvXr18vHx0e1atXSiy++qOuu\nu+6yY5w+fVo5OTnOeDVq1EgLFixQzZo1JV0I9ezZs5WWlqaCggJFRkaqX79+iomJkSSNGDFCb731\nls6dO6cZM2Zctp6srCzFxMTo6NGj8vDwUMuWLRUbGytjjGbPnq1vv/1WmZmZMsZo5syZl73qUdzt\nYmJiFBAQoH379unkyZO6+eab9corr8jf319r167Vq6++Kn9/f7Vs2bJEj5PIyEjVqlVLP/zwg4YM\nGaLPPvtMkZGRuueeexQbG6ukpCT5+PgoJCREcXFx8vf3v+T+LVu2VMOGDfXZZ5+pV69ekqRjx44p\nOTlZ8+fP1/Hjx9WzZ08lJSVp3rx5SkpK0unTp9W0aVPFxsZq2rRp2rlzp2rWrKlGjRpJkuLi4tSl\nSxfFx8crMzNTr7zyikJCQrR//345HA5NmzZN7dq1U0xMjHPvd+7cqVmzZik7O1s+Pj569tln1b59\ney1fvlzLli1TXl6e0tLS9Oijj7r0DB4VgEGV984775jWrVubrl27mr/85S9m+fLlJjs72xhjzOHD\nh03Pnj1NWlqaMcaY/fv3m/DwcJOdnW3i4+NN586dTWpqqjHGmPHjx5s5c+YYY4wZMWKEOXLkiDHG\nmFOnTpnmzZubs2fPmu3bt5uePXteNsOxY8dMmzZtCp1v3rx55qWXXnJ+/fLLL5sXXnjBGGPM3Xff\n7TznyZMnzW233WaOHTtmvvzyS9OjRw+Tnp5ujDFm8uTJpkuXLsYYY1asWGEee+wxY4wx27dvNy1a\ntDDffvut83sRFRVV6BxNmjQxaWlpJi8vz3Tt2tV8/vnnzvV17NjR7Nixw2zfvt00b97cnDhxotBj\nxMfHmxkzZhhjjJk0aZJ5+OGHnddNmjTJvP322+bEiRMmNDTU5ObmOmdat25docfbtm2bueuuu0z7\n9u3NmDFjzKJFi8zJkyeNMcbk5eWZBx54wOzevdsYY0x6erq5//77zc6dO0u8no8//tj86U9/MsYY\nk5+fb5577jlz9OhRk5SUZJ588knnHAsXLjSjR4++bL7ibjdp0iQzZMgQ43A4jMPhMH379jUrVqww\nZ86cMWFhYebgwYPO+zRt2rTQ9V/8nhljzPDhw82UKVOc1w0fPtysWbPGJCQkmB49ejgvnzt3rklK\nSir0eCtXrjTDhw93fv3yyy+buLg4Y8ylj9H4+HjTo0cPU1BQYIwx5m9/+5t55plnnN/nXr16mUmT\nJhljLjxGd+3a5Xys7dmzxxhjzNtvv+0818V1OBwOEx4ebjZt2mSMMWbXrl3mwQcfNOnp6WbQoEHO\nn8MdO3YU+fOCio9nzFBUVJQeeughJSQkKCEhQW+99ZYWLVqkDz/8UJs3b9aZM2cUFRUl83+f3urt\n7a0jR45Ikrp3767g4GBJ0oABAxQXF6dnn31Wr7/+ujZu3KjVq1fr0KFDkqTs7GyX5tu4caPS09Od\nL73m5eWpdu3azuvvueceSdL111+v2rVr65dfftGXX36p7t27KzAwUNKFZ7zbtm0r9PghISFq1aqV\nJKlZs2ZasWJFkbMYY3T48GHl5uaqa9eukqS6devq3nvv1VdffaV27drphhtu0A033FCitbVt2/ay\ny66//no1a9ZMffv2VUREhDp27KgOHToUev8777xTmzZt0o4dO/Tf//5XGzZs0IIFC/Tee+/J399f\nR48e1eTJk517l5OTo927d+u2224r0Xr69u2rV199VZGRkQoPD9fIkSMVEhKikJAQPfnkk1qyZImO\nHj2qr7/+2vm9/rXbb7+92NtFRETI2/vCr6HGjRvrl19+UWJiopo0aeJ8KX7QoEHOl+qvJCws7LLL\nmjRpIi8vLw0cOFB33XWXunXr5lz/b91///166aWX9OOPP6pevXpauXKlPvjgg0Jv27p1a+dL7Js2\nbXK+ChEYGKi+fftq7969l92nXr16atKkiSSpefPmWrly5SXX79u3T97e3urYsaMkqUWLFlq9erUk\n6Y033tCGDRt05MgRff/99y7/PMF+hLmK++abb5SUlKTo6Gh16tRJnTp10vjx49WzZ09t2bJFBQUF\n6tChg15++WXnfU6ePKm6devq888/l5eXl/NyY4y8vLyUnZ2tPn366N5771VYWJgGDBigdevWOeNQ\nWvn5+ZoyZYoiIiIkXQh8Tk6O83o/P79Lbm+Mkbe39yXn8/Qs+n2OF8MgXfh37qLmvPhLuKCg4LLr\njDFyOBySpOrVq19pSU4BAQGFnmfx4sXatWuXtmzZori4ON15553q0KGDXnvtNXl4eKhu3bqKi4tT\nfHy8pk2bprZt26pt27YaNWqUpk6dqo8//lgPPfSQgoKCLvnln5qaqho1alxyvuLWU79+fa1Zs0YJ\nCQnatm2bRo4cqWnTpsnX11ezZs3SI488oq5du+rmm2/Wv/71r8uOs3HjRs2ePbvI2/167y5+7z08\nPC6Z6dePsSsp7Htfo0YNrVq1St988422bdump59+WpGRkfLz89PSpUudL9HPmDFD1apVU79+/bR8\n+XK1atVKTZo00R/+8IdCz/XrvfPy8irR483X1/ey9f5aYWvdv3+/goKCNGjQIA0aNEhhYWG67777\ntGnTpuK/GaiweFd2FRccHKw33nhD33zzjfOyU6dO6fz582rcuLHat2+vzZs3O5/1btq0Sb1791Zu\nbq4kaf369crIyFBBQYGWLVumLl266MiRI8rKytJTTz2lzp07a/v27XI4HMrPzy92lqKCGBERoX/8\n4x9yOBwqKCjQlClTLvkPhcJ06tRJa9eudf7b+fLly51h9fLyUl5eXsm+QYXM17BhQ/n4+GjdunWS\nLny/1qxZo/Dw8FIfszB79uxRz5491ahRI40aNUpRUVHau3evunTpoo8//lgrV67UwoULVbNmTW3Z\nskXvvfee877Z2dk6ceKEWrRooYYNG8rX19f5jOvEiRPq2bOnkpOTJf3v+1DYetauXavw8HAtWbJE\nMTExCg8P1zPPPKOIiAjt27dPmzdvVpcuXTR48GC1bNlS69evLzTwW7ZsKdHtfi00NFQHDx50PuMs\n7hWMkti4caNGjhypNm3aaOzYserTp4/27t2rwYMHO7+fM2bMcN5+0KBB+vTTT7Vy5Urnewuu5O67\n79aKFStkjFF2drY++eQTl96w1rBhQ3l6ejrfTZ6cnKyoqCglJSUpODhYY8aMUXh4uDZs2CCp6J8Z\nVGw8Y67iGjRooPnz5+vll1/WqVOn5Ovrq8DAQM2YMUMNGjSQJMXGxmr8+PGSLvwyf/31153PdK67\n7jqNGjVKZ8+e1R133KHHHntMPj4+6ty5s7p3766goCDddNNNuuWWW3T06NEi3wglXXgj2sWXdi8+\nc/rnP/+pxx9/XHPmzFHfvn2db/6aOHGipMvfyX3x6/bt22vgwIEaPHiw/Pz8dOuttzrf7NOmTRu9\n+uqrGjdunCIjI0v8vbp4bG9vb82fP18zZ87Ua6+9poKCAo0bN07t2rXT119/XeLjFaVp06bq0aOH\n+vXrp+rVq8vf319Tp0697HZeXl56++239dJLL2nx4sWqXr26PDw81K9fP/Xt21eStGDBAs2cOVOL\nFi1Sfn6+nn76aecbtLp166ahQ4dqwYIFl61n7NixateunVq1aqWEhATdf//98vf3V/369TVy5Eid\nPn1aEyZMUO/eveXl5aWwsDCtXbv2shkHDx5cotv9WnBwsObOnasJEybIx8dH7dq1K9H3rajHQqdO\nnfTVV1+pZ8+eql69umrVqnVJiH8rJCREDRs21IEDB9S5c+cSnXvUqFGKjY1Vr169FBgYqNq1azsf\nb6UJdLVq1RQfH69Zs2Zpzpw5qlatmubNm6dmzZpp1apVuu+++xQQEKBWrVopODhYR44ccf6covLw\nMPwnF1w0b948paWlFRqN8rZr1y4lJSU5w/vuu+/q22+/veIzbcAV//73vxUQEKBOnTrJGKNx48bp\nrrvu0uDBg8t7NFRAxT5jzsvL0+TJk3X8+HE5HA6NHj1at9xyiyZNmiRPT0/deuutzg9qAGzSoEED\nvfXWW1q2bJkkqX79+oqNjS3nqVBZ3XrrrZo2bZpeeeUVORwO5ys2gCuKfca8YsUK7d27VzExMTp3\n7px69+6tpk2bKjo6WmFhYZo+fboiIiKc7+YEAABXp9g3f/Xo0cP5oRD5+fny8vLS7t27nf+XhI4d\nO5bqI+8AAEDxig2zv7+/qlevroyMDD355JN6+umnL3kXYEBAgNLT08t8SAAAqoorviv7xIkTGjt2\nrIYPH64HHnhAf/3rX53XZWZmKigo6IonSUxMvLopAQCogH778b4lUWyYz5w5o+joaE2bNk3t27eX\ndOGTkRISEnTHHXfoyy+/dF5eFsNVFImJiVe9vtzcXL390WYFBf/eTVO5T0pKiurVq6cba2apY/vL\nP6mqonPH/tmsMq+vMq9NYn0VnatPSosN88KFC3Xu3Dnn/8/Rw8NDU6ZM0cyZM+VwONSoUSN1797d\npRMDAIDLFRvmKVOmaMqUKZddvnjx4jIbCACAqoyP5AQAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKE\nGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAI\nYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAs\nQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAA\nixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYA\nwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgB\nALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALBIicK8c+dORUZGSpK+\n//57dezYUSNGjNCIESP06aeflumAAABUJd5XusGiRYu0atUqBQQESJJ27dqlRx55RFFRUWU9GwAA\nVc4VnzHfdNNNmj9/vvPr5ORkbdy4UcOHD9eUKVOUlZVVpgMCAFCVXDHM3bp1k5eXl/Pr1q1b69ln\nn9UHH3ygkJAQxcfHl+mAAABUJVd8Kfu3unbtqho1aki6EO2ZM2e6fSgA196Px09o36GU8h6jSA6H\nQ53/eLv8/PzKexSgTJU6zNHR0XruuefUqlUrbd26VS1atCjR/RITE0s9XEVytetzOBw6eeKkMs4b\nN03kXikpKco+dUYBPnbOd7V4fErJew/pp5za12Aa12Sm/6JqylLNmjUvuZy9q9gq+/pcUeowP//8\n85oxY4Z8fHxUp04dxcbGluh+oaGhpR6uokhMTLzq9eXm5mrHD9kKCv69m6Zyn5SUFNWrV0831qyl\n0NC25T2O27lj/2xW0vXlyUfeZ6pdg4lck/6Lv26/vYmCg4Odl7F3FVtVWJ8rShTm+vXra+nSpZKk\n5s2ba8mSJS6dDAAAFI8PGAEAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAI\nYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAs\nQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAA\nixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYA\nwCKEGQAAi3iX9wCoOH48fkYb/vNNeY9RpDq1a6hls1vLewwAuCqEGSXmEfgHncgs7ymKlpWTSpgB\nVHi8lA0AgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHM\nAABYhDADAGARwgwAgEX461KoNHJzcpWamlrq+6Wlpbl0v9Ly8fFRUFBQmZ+nMktLS5Mx5pKvr8Xe\nlVRQUJB8fHzKewxUcIQZlcbPOQFavn5fqe938uQ5HUgt/f1Ky1fpinro3jI/T2UVUKOmPv/vKUmn\nnJddq70rqdBbAhXWtlV5j4EKjjCj0vCvHiApoNT3S8/MVo2av3P/QL/hmeMo83NUZp6enpft07Xa\nu5Jjj3H1+DdmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsUqIw\n79y5U5GRkZKko0ePaujQoRo+fLheeOGFMh0OAICq5ophXrRokaZOnSqH48JHzcXFxWn8+PH64IMP\nVFBQoHXr1pX5kAAAVBVXDPNNN92k+fPnO79OTk5WWFiYJKljx47aunVr2U0HAEAVc8Uwd+vWTV5e\nXs6vf/0n1wICApSenl42kwEAUAWV+q9LeXr+r+WZmZkl/vuyiYmJpT1VhXK163M4HDp54qQyzpsr\n37gcpKSklPcIZeparM9knSy3n4OSnHfv3kP6Kaf2NZjGvWx6bBb8nCoPk+vWY/K7s+opdZibN2+u\nhIQE3XHHHfryyy/Vvn37Et0vNDS01MNVFImJiVe9vtzcXO34IVtBwb9301Tuk5KSonr16pX3GGXm\nWq3PM8e7XH4OSvr4zJOPvM9UuwYTuY9tj83Gdeoo1I1/j9kdv1tsVhXW54pSh3nixIl67rnn5HA4\n1KhRI3Xv3t2lEwMAgMuVKMz169fX0qVLJUkNGjTQ4sWLy3QoAACqKj5gBAAAixBmAAAsQpgBALAI\nYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALBIqf+IRUX1y7l0JX23T56e\nHm4/9v79B5WZe3XHzcvLl4dnldkOAEARqkwJzqT+rB/OeMrXz9/txz6bf52Onat+1cepUauGG6YB\nAFRkvJQNAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYh\nzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBF\nCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFvMt7AKCqyHJ46p//+uqa\nn/fojz/qQErWFW+XlX1evjVDrsFEAIpDmIFrxC/wOuWXw3k9/B3Kr1bnirfzrXYNhgFwRbyUDQCA\nRQgzAAAWIcwAAFiEMAMAYBHCDACARQgzAAAWIcwAAFiEMAMAYBHCDACARQgzAAAWIcwAAFiEMAMA\nYBHCDACARfjrUgAAK+zdf0g/pf5S3mMUqbpvNYW2aVHm5yHMAAAr7Dv8k9ILapX3GEXKzzyl0DZl\nfx5eygYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAi7j8\nkZz9+vVTYGCgJOnGG2/U7Nmz3TYUAABVlUthzs3NlSS9//77bh0GAICqzqWXsvfs2aOsrCxFR0cr\nKipKO3fudPdcAABUSS49Y/bz81N0dLQGDhyow4cP69FHH9WaNWvk6ck/WQOArXJycpSfn1/eYzid\nP39eWVlZzq8L8vMlj3IcyBIexhhT2jvl5ubKGCNfX19J0sCBAzVv3jxdf/31hd4+MTHx6qZ0g+Mp\nJ7XruIeq+fqV9ygAKqkb/FLVrPHN5T1Gkf7fhm9U4B1U3mMUydvHT37Va5T3GEVypJ/QveFNS3Wf\n0NDQUp/HpWfMH330kfbt26fp06fr1KlTyszMVJ06ddw+nDvVCj6i1IKf5evn7/Zjp6SkqF69em4/\nri1YX8VWmddn29oa16mj0Lat3Ha8xMREt/7u3HssU/Kr67bjXS3b9u9K8jO9SrUfrj4pdSnMAwYM\nUExMjIYOHSpPT0/Nnj2bl7EBAHADl8Ls4+OjuXPnunsWAACqPJ7mAgBgEcIMAIBFCDMAABYhzAAA\nWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABZx6bOyAQCX23XghA6fPOe24x05\nfEQ/nDrvtuOdz/MUf/jWfoQZANykWs0/KNeNxzP+Ocr1qu224/kFuu1QKEO8lA0AgEUIMwAAFiHM\nAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUI\nMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGAR\nwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABY\nhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAA\nFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARb1fu\nZIzR888/r71796patWqaNWuWQkJC3D0bAABVjkvPmNetW6fc3FwtXbpUzzzzjOLi4tw9FwAAVZJL\nYU5MTFRERIQkqXXr1tq1a5dbhwIAoKpy6aXsjIwM1ahR438H8fZWQUGBPD3t/SdrL09PZaefUl62\nj9uPnXk2RZn+xu3HtQXrq9gq8/oq89ok1mebasq5JudxKcyBgYHKzMx0fl2SKCcmJrpyKre6s1md\nMjlu21trlclxbcH6KrbKvL7KvDaJ9dnoWrTMpTC3bdtWGzZsUPfu3bVjxw41bty42NuHhoa6NBwA\nAFWNhzGm1K8j/Ppd2ZIUFxenhg0bun04AACqGpfCDAAAyoa979YCAKAKIswAAFiEMAMAYBGX3pV9\nJampqerfv7/eeeedS94U9sUXX2jBggXy9vZW//79NXDgwLI4fZkqam3vvvuuli9fruDgYElSbGys\nGjRoUE5TuqZfv34KDAyUJN14442aPXu287rKsHfFra8y7N+bb76pL774Qg6HQ0OHDlX//v2d11WG\n/StufRV9/1auXKkVK1bIw8NDOTk52rNnjzZv3ux8vFb0/bvS+iry/uXl5WnixIk6fvy4vL29NWPG\njKvvnnEzh8NhnnjiCXPfffeZQ4cOXXJ5t27dTHp6usnNzTX9+/c3qamp7j59mSpqbcYYM2HCBJOc\nnFxOk129nJwc07dv30Kvqwx7V9z6jKn4+7d9+3YzevRoY4wxmZmZJj4+3nldZdi/4tZnTMXfv197\n4YUXzLJly5xfV4b9+7Xfrs+Yir1/69atM0899ZQxxpjNmzebcePGOa9zde/c/lL2nDlzNGTIENWt\nW/eSyw8ePKibbrpJgYGB8vHxUWhoqBISEtx9+jJV1NokKTk5WQsXLtTQoUP15ptvlsN0V2fPnj3K\nyspSdHS0oqKitHPnTud1lWHvilufVPH37z//+Y8aN26sxx9/XGPGjNHdd9/tvK4y7F9x65Mq/v5d\n9N133+nAgQOXPKuqDPt3UWHrkyr2/jVo0ED5+fkyxig9PV0+Pv/7dElX986tL2WvWLFCtWvXVnh4\nuN54441Lrvvtx3gGBAQoPT3dnacvU8WtTZIeeOABDRs2TIGBgXriiSe0adMmderUqRwmdY2fn5+i\no6M1cOBAHT58WI8++qjWrFkjT0/PCr93UvHrkyr+/p09e1YpKSlauHChfvzxR40ZM0afffaZpIr/\nsycVvz6XyBvgAAACgUlEQVSp4u/fRW+++abGjh17yWWVYf8uKmx9UsXev4CAAB07dkzdu3dXWlqa\nFi5c6LzO1b1z6zPmFStWaPPmzYqMjNSePXs0ceJEpaamSrrwMZ4ZGRnO22ZmZiooKMidpy9Txa1N\nkkaOHKlatWrJ29tbnTp10u7du8tx2tJr0KCBevXq5fzftWrV0unTpyVV/L2Til+fVPH3r1atWoqI\niJC3t7caNmwoX19f/fzzz5Iqx/4Vtz6p4u+fJKWnp+vw4cNq167dJZdXhv2Til6fVLH3791331VE\nRITWrFmj1atXa+LEicrNzZXk+t65NcwffPCBFi9erMWLF6tp06aaM2eOateuLUlq1KiRjhw5onPn\nzik3N1cJCQm6/fbb3Xn6MlXc2jIyMtSzZ09lZ2fLGKNt27apRYsW5Txx6Xz00Ud68cUXJUmnTp1S\nZmam6tS58NniFX3vpOLXVxn2LzQ0VF999ZWkC+s7f/68fve730mqHPtX3Poqw/5JUkJCgtq3b3/Z\n5ZVh/6Si11fR969mzZrON7HVqFFDeXl5KigokOT63pXZJ3+NGDFCL7zwgpKTk5Wdna2BAwdq48aN\nmjdvnowxGjBggIYMGVIWpy5zha1t9erVev/99+Xr66sOHToU+nKNzRwOh2JiYpSSkiJPT09NmDBB\nx44dqzR7d6X1VfT9k6S5c+dq27ZtMsZo/PjxOnv2bKXZP6n49VWG/fv73/8uHx8fjRgxQpL0ySef\nVKr9K259FXn/srKyNHnyZJ0+fVp5eXkaMWKEjDFXtXd8JCcAABbhA0YAALAIYQYAwCKEGQAAixBm\nAAAsQpgBALAIYQYAwCKEGQAAixBmAAAs8v8Bd6U8xfluLwYAAAAASUVORK5CYII=\n", 207 | "text/plain": [ 208 | "" 209 | ] 210 | }, 211 | "metadata": {}, 212 | "output_type": "display_data" 213 | } 214 | ], 215 | "source": [ 216 | "plt.hist(**data);_=plt.title(title)" 217 | ] 218 | }, 219 | { 220 | "cell_type": "markdown", 221 | "metadata": { 222 | "slideshow": { 223 | "slide_type": "slide" 224 | } 225 | }, 226 | "source": [ 227 | "# Naive Clustering" 228 | ] 229 | }, 230 | { 231 | "cell_type": "code", 232 | "execution_count": 4, 233 | "metadata": { 234 | "collapsed": false, 235 | "slideshow": { 236 | "slide_type": "-" 237 | } 238 | }, 239 | "outputs": [ 240 | { 241 | "data": { 242 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAFeCAYAAABQCrzXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X18z/Xi//HnrmyzmZ0JHdpByrXElnQ0JIqSq5KLjNUO\n6epUumCImoul48StoZTTlc5NRyJOv1NEqEO01igrFxFiI5ZlV7bPttfvD1+fk8wuPj4fe2173P/y\nuXi/36/X5/XZHvu89/GZlzHGCAAAWMG7sgcAAAD+hzADAGARwgwAgEUIMwAAFiHMAABYhDADAGAR\nwgxt375do0aN0oABA3THHXdo7Nix+uGHHzxyrC+//FJ33HHHedcfOXJEHTt29Mgxf+/bb7/VtGnT\nSh1PSaZNm6ZevXpp3rx5Lh97/vz5mjFjRom3vfTSS1q1alWF9nf8+HE9/vjjuuOOO9S/f38NHTpU\n69evL9e2sbGxyszMrNDxKsPJkyfVqlWrEm+Li4vTG2+8UeJtU6ZM0RdffFHu48TFxWnq1KnnXb9m\nzRoNHDhQP//8s4YPH17u/Z11//33a9++faXex5W1R/XlW9kDQOUqKCjQuHHj9Oabbzq/+a1evVpj\nx47V+vXr5eXldcnGcqmOtXfvXh07dqzC2y1btkwbN25Uw4YNPTAq6a9//WuFt5kyZYr+/Oc/a+7c\nuZKkffv2acSIEVq6dKmuvPLKUrfdvHmzS+O81IwxLj03LvQD0IXcc889uvfeezVlyhTVqlXLef2y\nZct0zz33qEGDBlq6dGmFx7Fo0aIy7+PK2qP6Isw13OnTp5Wdna3s7Gzndf3791edOnVUVFQkX19f\nbdiwQS+//LIKCwsVEBCgCRMmqEOHDpo/f7727t2rEydO6MSJE2rTpo1mzJihoKAgbdiwQYsWLVJh\nYaF++eUXDRgwQI8++qhLY3Q4HJozZ46SkpJUXFys1q1ba8qUKQoKClLPnj01ePBgffHFF0pPT1ff\nvn311FNPSZJeffVVvf/++woKClJkZKTWrVunpUuXKjExUdnZ2Zo0aZIGDhyonJwcjR8/Xvv371dB\nQYGmT5+uiIiIc8Zwzz33SJLGjBmjadOmqW7duoqPj1dmZqa8vb0VExOjgQMH6ssvv9TMmTMVGBio\n06dP67333pOfn1+J84qLi1NmZqYOHz6sHj166MSJE2rRooXuvfdevfTSS1q/fr38/PwUGhqq559/\nXpdddtl5+zh+/Ljy8/Od8WrevLkWLlyounXrSjoT6lmzZikzM1PFxcWKjo7W4MGDFRcXJ0kaNWqU\nXnvtNZ06dUrTp08/bz65ubmKi4vToUOH5OXlpXbt2ik+Pl7GGM2aNUvffPONcnJyZIzRjBkzzjvr\nUdr94uLiFBQUpD179ujo0aO68sorNXfuXAUGBmrt2rWaN2+eAgMD1a5du3I9T6KjoxUaGqoff/xR\nw4cP18cff6zo6GjdfPPNio+PV0pKivz8/BQeHq6EhAQFBgaes327du3UrFkzffzxx+rfv78k6fDh\nw0pNTdWCBQt05MgR9evXTykpKZo/f75SUlJ0/PhxtWrVSvHx8Zo6dap27NihunXrqnnz5pKkhIQE\n9ezZU4mJicrJydHcuXMVHh6uvXv3yuFwaOrUqercubPi4uKca79jxw7NnDlTeXl58vPz09NPP60u\nXbpo+fLlWrZsmQoLC5WZmakxY8a49AoeVYBBjffGG2+YDh06mF69epmnnnrKLF++3OTl5RljjDlw\n4IDp16+fyczMNMYYs3fvXtO1a1eTl5dnEhMTTY8ePUxGRoYxxpjx48eb2bNnG2OMGTVqlDl48KAx\nxphjx46ZNm3amJMnT5pt27aZfv36nTeGw4cPm44dO5Y4vvnz55sXXnjBefnFF180zz33nDHGmJtu\nusl5zKNHj5prrrnGHD582Hz22Wemb9++JisryxhjzKRJk0zPnj2NMcasWLHC3H///cYYY7Zt22ba\ntm1rvvnmG+djERMTU+I4WrZsaTIzM01hYaHp1auX+eSTT5zz69atm9m+fbvZtm2badOmjUlPTy9x\nH4mJiWb69OnGGGMmTpxo7r33XudtEydONK+//rpJT083ERERpqCgwDmmdevWlbi/rVu3mhtvvNF0\n6dLFPPDAA2bx4sXm6NGjxhhjCgsLze23326+++47Y4wxWVlZ5rbbbjM7duwo93w++OAD85e//MUY\nY0xRUZF55plnzKFDh0xKSop59NFHneNYtGiRGTdu3HnjK+1+EydONMOHDzcOh8M4HA4zaNAgs2LF\nCnPixAkTGRlp9u3b59ymVatWJc7/7GNmjDEjR440kydPdt42cuRIs2bNGpOUlGT69u3rvH7OnDkm\nJSWlxP2tXLnSjBw50nn5xRdfNAkJCcaYc5+jiYmJpm/fvqa4uNgYY8zf//5388QTTzgf5/79+5uJ\nEycaY848R3fu3Ol8ru3atcsYY8zrr7/uPNbZeTgcDtO1a1ezadMmY4wxO3fuNHfccYfJysoyQ4cO\ndX4dbt++/YJfL6j6eMUMxcTE6O6771ZSUpKSkpL02muvafHixXrvvfe0efNmnThxQjExMTL/9+mt\nvr6+OnjwoCSpT58+CgsLkyTdddddSkhI0NNPP62XX35ZGzdu1OrVq7V//35JUl5enkvj27hxo7Ky\nspynXgsLC1WvXj3n7TfffLMkqWHDhqpXr55+/fVXffbZZ+rTp4+Cg4MlnXnFu3Xr1hL3Hx4ervbt\n20uSWrdurRUrVlxwLMYYHThwQAUFBerVq5ckqUGDBrrlllv0+eefq3Pnzrr88st1+eWXl2tunTp1\nOu+6hg0bqnXr1ho0aJCioqLUrVs33XDDDSVuf/3112vTpk3avn27vvrqK23YsEELFy7UW2+9pcDA\nQB06dEiTJk1yrl1+fr6+++47XXPNNeWaz6BBgzRv3jxFR0era9euGj16tMLDwxUeHq5HH31US5cu\n1aFDh/Tll186H+vfuvbaa0u9X1RUlHx9z3wbatGihX799VclJyerZcuWzlPxQ4cOdZ6qL0tkZOR5\n17Vs2VI+Pj4aMmSIbrzxRvXu3ds5/9+77bbb9MILL+inn35So0aNtHLlSr3zzjsl3rdDhw7OU+yb\nNm1ynoUIDg7WoEGDtHv37vO2adSokVq2bClJatOmjVauXHnO7Xv27JGvr6+6desmSWrbtq1Wr14t\nSXrllVe0YcMGHTx4UN9//73LX0+wH2Gu4b7++mulpKQoNjZW3bt3V/fu3TV+/Hj169dPW7ZsUXFx\nsW644Qa9+OKLzm2OHj2qBg0a6JNPPpGPj4/zemOMfHx8lJeXp4EDB+qWW25RZGSk7rrrLq1bt84Z\nh4oqKirS5MmTFRUVJelM4PPz8523BwQEnHN/Y4x8fX3POZ6394Xf53g2DNKZ33NfaJxnvwkXFxef\nd5sxRg6HQ5JUu3btsqbkFBQUVOJxlixZop07d2rLli1KSEjQ9ddfrxtuuEEvvfSSvLy81KBBAyUk\nJCgxMVFTp05Vp06d1KlTJ40dO1ZTpkzRBx98oLvvvlshISHnfPPPyMhQnTp1zjleafNp3Lix1qxZ\no6SkJG3dulWjR4/W1KlT5e/vr5kzZ+q+++5Tr169dOWVV+rf//73efvZuHGjZs2adcH7/Xbtzj72\nXl5e54zpt8+xspT02NepU0erVq3S119/ra1bt+rxxx9XdHS0AgIC9O677zpP0U+fPl21atXS4MGD\ntXz5crVv314tW7bUn/70pxKP9du18/HxKdfzzd/f/7z5/lZJc927d69CQkI0dOhQDR06VJGRkbr1\n1lu1adOm0h8MVFm8K7uGCwsL0yuvvKKvv/7aed2xY8d0+vRptWjRQl26dNHmzZudr3o3bdqkAQMG\nqKCgQJK0fv16ZWdnq7i4WMuWLVPPnj118OBB5ebm6rHHHlOPHj20bds2ORwOFRUVlTqWCwUxKipK\n//znP+VwOFRcXKzJkyef84NCSbp37661a9c6f3e+fPlyZ1h9fHxUWFhYvgeohPE1a9ZMfn5+Wrdu\nnaQzj9eaNWvUtWvXCu+zJLt27VK/fv3UvHlzjR07VjExMdq9e7d69uypDz74QCtXrtSiRYtUt25d\nbdmyRW+99ZZz27y8PKWnp6tt27Zq1qyZ/P39na+40tPT1a9fP6Wmpkr63+NQ0nzWrl2rrl27aunS\npYqLi1PXrl31xBNPKCoqSnv27NHmzZvVs2dPDRs2TO3atdP69etLDPyWLVvKdb/fioiI0L59+5yv\nOEs7g1EeGzdu1OjRo9WxY0c9/PDDGjhwoHbv3q1hw4Y5H8/p06c77z906FB99NFHWrlypfO9BWW5\n6aabtGLFChljlJeXpw8//NClN6w1a9ZM3t7ezneTp6amKiYmRikpKQoLC9MDDzygrl27asOGDZIu\n/DWDqo1XzDVc06ZNtWDBAr344os6duyY/P39FRwcrOnTp6tp06aSpPj4eI0fP17SmW/mL7/8svOV\nzmWXXaaxY8fq5MmTuu6663T//ffLz89PPXr0UJ8+fRQSEqImTZroqquu0qFDhy74RijpzBvRzp7a\nPfvK6V//+pcefPBBzZ49W4MGDXK++WvChAmSzn8n99nLXbp00ZAhQzRs2DAFBATo6quvdr7Zp2PH\njpo3b54eeeQRRUdHl/uxOrtvX19fLViwQDNmzNBLL72k4uJiPfLII+rcubO+/PLLcu/vQlq1aqW+\nfftq8ODBql27tgIDAzVlypTz7ufj46PXX39dL7zwgpYsWaLatWvLy8tLgwcP1qBBgyRJCxcu1IwZ\nM7R48WIVFRXp8ccfd75Bq3fv3hoxYoQWLlx43nwefvhhde7cWe3bt1dSUpJuu+02BQYGqnHjxho9\nerSOHz+uJ598UgMGDJCPj48iIyO1du3a88Y4bNiwct3vt8LCwjRnzhw9+eST8vPzU+fOncv1uF3o\nudC9e3d9/vnn6tevn2rXrq3Q0NBzQvx74eHhatasmX744Qf16NGjXMceO3as4uPj1b9/fwUHB6te\nvXrO51tFAl2rVi0lJiZq5syZmj17tmrVqqX58+erdevWWrVqlW699VYFBQWpffv2CgsL08GDB51f\np6g+vAw/csFF8+fPV2ZmZonRqGw7d+5USkqKM7xvvvmmvvnmmzJfaQOu+M9//qOgoCB1795dxhg9\n8sgjuvHGGzVs2LDKHhqqoFJfMRcWFmrSpEk6cuSIHA6Hxo0bp6uuukoTJ06Ut7e3rr76aucHNQA2\nadq0qV577TUtW7ZMktS4cWPFx8dX8qhQXV199dWaOnWq5s6dK4fD4TxjA7ii1FfMK1as0O7duxUX\nF6dTp05pwIABatWqlWJjYxUZGalp06YpKirK+W5OAABwcUp981ffvn2dHwpRVFQkHx8ffffdd87/\nktCtW7cKfeQdAAAoXalhDgwMVO3atZWdna1HH31Ujz/++DnvAgwKClJWVpbHBwkAQE1R5ruy09PT\n9fDDD2vkyJG6/fbb9be//c15W05OjkJCQso8SHJy8sWNEgCAKuj3H+9bHqWG+cSJE4qNjdXUqVPV\npUsXSWc+GSkpKUnXXXedPvvsM+f1nhhcVZGcnHzR8ysoKNDr729WSNgf3TQq90lLS1OjRo10Rd1c\ndety/idVVXXuWD9X/f6/0njiP0lU5vw8rTrPTWJ+VZ2rL0pLDfOiRYt06tQp5/9z9PLy0uTJkzVj\nxgw5HA41b95cffr0cenAAADgfKWGefLkyZo8efJ51y9ZssRjAwIAoCbjIzkBALAIYQYAwCKEGQAA\nixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYA\nwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgB\nALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBm\nAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKE\nGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAI\nYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAs\nUq4w79ixQ9HR0ZKk77//Xt26ddOoUaM0atQoffTRRx4dIAAANYlvWXdYvHixVq1apaCgIEnSzp07\ndd999ykmJsbTYwMAoMYp8xVzkyZNtGDBAufl1NRUbdy4USNHjtTkyZOVm5vr0QECAFCTlBnm3r17\ny8fHx3m5Q4cOevrpp/XOO+8oPDxciYmJHh0gAAA1SZmnsn+vV69eqlOnjqQz0Z4xY4bbBwXg0vvp\nSLr27E+r7GFckMPhUI8/X6uAgIDKHgrgURUOc2xsrJ555hm1b99eX3zxhdq2bVuu7ZKTkys8uKrk\nYufncDh0NP2osk8bN43IvdLS0pR37ISC/Owc38Wy5fnpqXGUZ7+pu/fr5/x6Hjm+O+Rk/apaylXd\nunXPud6WtfMU5lfzVDjMzz77rKZPny4/Pz/Vr19f8fHx5douIiKiwoOrKpKTky96fgUFBdr+Y55C\nwv7oplG5T1pamho1aqQr6oYqIqJTZQ/H7dyxfu7iiXGUd36F8pPviVpuP767ZP0aqGuvbamwsDDn\ndTatnScwv6rN1R86yhXmxo0b691335UktWnTRkuXLnXpYAAAoHR8wAgAABYhzAAAWIQwAwBgEcIM\nAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQw\nAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYh\nzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBF\nCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWMS3sgeAquOnIye04b9fV/YwLqh+vTpq1/rq\nyh4GAFwUwoxy8wr+k9JzKnsUF5abn0GYAVR5nMoGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkA\nAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAi/HUpVBsF+QXKyMio8HaZmZku\nbVdRfn5+CgkJ8fhxqrPMzEwZY865fCnWrrxCQkLk5+dX2cNAFUeYUW38kh+k5ev3VHi7o0dP6YeM\nim9XUf7KUszdt3j8ONVVUJ26+uSrY5KOOa+7VGtXXhFXBSuyU/vKHgaqOMKMaiOwdpCkoApvl5WT\npzp1/+D+Af2Od77D48eozry9vc9bp0u1duXHGuPi8TtmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAs\nQpgBALAIYQYAwCKEGQAAixBmAAAsUq4w79ixQ9HR0ZKkQ4cOacSIERo5cqSee+45jw4OAICapsww\nL168WFOmTJHDceaj5hISEjR+/Hi98847Ki4u1rp16zw+SAAAaooyw9ykSRMtWLDAeTk1NVWRkZGS\npG7duumLL77w3OgAAKhhygxz79695ePj47z82z+5FhQUpKysLM+MDACAGqjCf13K2/t/Lc/JySn3\n35dNTk6u6KGqlIudn8Ph0NH0o8o+bcq+cyVIS0ur7CF41KWYn8k9WubzxFNfJ+XZ7+7d+/Vzfj2P\nHN+TbHpuFv+SIS9T4NZ98r2z5qlwmNu0aaOkpCRdd911+uyzz9SlS5dybRcREVHhwVUVycnJFz2/\ngoICbf8xTyFhf3TTqNwnLS1NjRo1quxheMylmp93vm+ZzxNPfJ2U9/lZKD/5nqjl9uN7km3PzRb1\n6yvCjX+P2R3fW2xWE+bnigqHecKECXrmmWfkcDjUvHlz9enTx6UDAwCA85UrzI0bN9a7774rSWra\ntKmWLFni0UEBAFBT8QEjAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHM\nAABYhDADAGARwgwAgEUq/EcsqqpfT2Up5ds98vb2cvu+9+7dp5yCi9tvYWGRvLxrzHIAAC6gxpTg\nRMYv+vGEt/wDAt2+75NFl+nwqdoXvZ86oXXcMBoAQFXGqWwAACxCmAEAsAhhBgDAIoQZAACLEGYA\nACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZ\nAACLEGYAACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhh\nBgDAIoQZAACLEGYAACziW9kDAGqKXIe3/vXvz0u9T1m3u+LQTz/ph7TcMu+Xm3da/nXD3X58ABVD\nmIFLJCD4MhWVcZ+iWvXdflyvQEe59utfy+2HBuACTmUDAGARwgwAgEUIMwAAFiHMAABYhDADAGAR\nwgwAgEUIMwAAFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGAR/roUAMAKu/fu188Zv1b2\nMC6otn8tRXRs6/HjEGYAgBX2HPhZWcWhlT2MCyrKOaaIjp4/DqeyAQCwCGEGAMAihBkAAIsQZgAA\nLEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAiLn8k5+DBgxUcHCxJuuKKKzRr1iy3DQoA\ngJrKpTAXFBRIkt5++223DgYAgJrOpVPZu3btUm5urmJjYxUTE6MdO3a4e1wAANRILr1iDggIUGxs\nrIYMGaIDBw5ozJgxWrNmjby9+ZU1ANgqPz9fRUVFlT0Mp9OnTys3N9d5ubioSPKqxAFZwqUwN23a\nVE2aNHH+OzQ0VMePH1fDhg0vuE1ycrJrI3STI2lHlZ7upVr+AR7Zf1pamkf2awvmd2l4ahy2zM8T\nbJpb8S8Z8jIFbt2nO793/r8NX6vYN8Rt+3OHT5IOO//t6xeggNr5lTia0jmy0i9Jy1wK8/vvv689\ne/Zo2rRpOnbsmHJyclS/fv1St4mIiHBpgO4SGnZQGcW/yD8g0O37TktLU6NGjdy+X1swv0vHE+Ow\naX7uZtvcWtSvr4hO7d22v+TkZLd+79x9OEcKaOC2/V0s29avLEU5PhVaD1cj7lKY77rrLsXFxWnE\niBHy9vbWrFmzOI0NAIAbuBRmPz8/zZkzx91jAQCgxuNlLgAAFiHMAABYhDADAGARwgwAgEUIMwAA\nFiHMAABYhDADAGARwgwAgEUIMwAAFiHMAABYhDADAGARlz4rGwBwvp0/pOvA0VNu29/BAwf147HT\nbtvf6UJveeYP38KdCDMAuEmtun+SO/8aswnMV4FPPbftLyDYbbuCB3EqGwAAixBmAAAsQpgBALAI\nYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAs\nQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAA\nixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYA\nwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgB\nALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCK+rmxkjNGz\nzz6r3bt3q1atWpo5c6bCw8PdPTYAAGocl14xr1u3TgUFBXr33Xf1xBNPKCEhwd3jAgCgRnIpzMnJ\nyYqKipIkdejQQTt37nTroAAAqKlcOpWdnZ2tOnXq/G8nvr4qLi6Wt7e9v7L28fZWXtYxFeb5uX3f\nOSfTlBNo3L5fWzC/Syfn5BEP7NOe+blbdZ6bxPxsU0v5l+Q4LoU5ODhYOTk5zsvliXJycrIrh3Kr\n61vX98h+O10d6pH92oL5ec5XX33l8WNU5/WrznOTmJ+NLkXLXApzp06dtGHDBvXp00fbt29XixYt\nSr1/RESES4MDAKCm8TLGVPg8wm/flS1JCQkJatasmdsHBwBATeNSmAEAgGfY+24tAABqIMIMAIBF\nCDMAABZx6V3ZZcnIyNCdd96pN95445w3hX366adauHChfH19deedd2rIkCGeOLxHXWhub775ppYv\nX66wsDBJUnx8vJo2bVpJo3TN4MGDFRwcLEm64oorNGvWLOdt1WHtSptfdVi/V199VZ9++qkcDodG\njBihO++803lbdVi/0uZX1ddv5cqVWrFihby8vJSfn69du3Zp8+bNzudrVV+/suZXldevsLBQEyZM\n0JEjR+Tr66vp06dffPeMmzkcDvPQQw+ZW2+91ezfv/+c63v37m2ysrJMQUGBufPOO01GRoa7D+9R\nF5qbMcY8+eSTJjU1tZJGdvHy8/PNoEGDSrytOqxdafMzpuqv37Zt28y4ceOMMcbk5OSYxMRE523V\nYf1Km58xVX/9fuu5554zy5Ytc16uDuv3W7+fnzFVe/3WrVtnHnvsMWOMMZs3bzaPPPKI8zZX187t\np7Jnz56t4cOHq0GDBudcv2/fPjVp0kTBwcHy8/NTRESEkpKS3H14j7rQ3CQpNTVVixYt0ogRI/Tq\nq69Wwuguzq5du5Sbm6vY2FjFxMRox44dztuqw9qVNj+p6q/ff//7X7Vo0UIPPvigHnjgAd10003O\n26rD+pU2P6nqr99Z3377rX744YdzXlVVh/U7q6T5SVV7/Zo2baqioiIZY5SVlSU/v/99uqSra+fW\nU9krVqxQvXr11LVrV73yyivn3Pb7j/EMCgpSVlaWOw/vUaXNTZJuv/123XPPPQoODtZDDz2kTZs2\nqXv37pXYDWZkAAACxklEQVQwUtcEBAQoNjZWQ4YM0YEDBzRmzBitWbNG3t7eVX7tpNLnJ1X99Tt5\n8qTS0tK0aNEi/fTTT3rggQf08ccfS6r6X3tS6fOTqv76nfXqq6/q4YcfPue66rB+Z5U0P6lqr19Q\nUJAOHz6sPn36KDMzU4sWLXLe5uraufUV84oVK7R582ZFR0dr165dmjBhgjIyMiSd+RjP7Oxs531z\ncnIUEhLizsN7VGlzk6TRo0crNDRUvr6+6t69u7777rtKHG3FNW3aVP3793f+OzQ0VMePH5dU9ddO\nKn1+UtVfv9DQUEVFRcnX11fNmjWTv7+/fvnlF0nVY/1Km59U9ddPkrKysnTgwAF17tz5nOurw/pJ\nF56fVLXX780331RUVJTWrFmj1atXa8KECSooKJDk+tq5NczvvPOOlixZoiVLlqhVq1aaPXu26tWr\nJ0lq3ry5Dh48qFOnTqmgoEBJSUm69tpr3Xl4jyptbtnZ2erXr5/y8vJkjNHWrVvVtm3bSh5xxbz/\n/vt6/vnnJUnHjh1TTk6O6tc/89niVX3tpNLnVx3WLyIiQp9//rmkM/M7ffq0/vCHP0iqHutX2vyq\nw/pJUlJSkrp06XLe9dVh/aQLz6+qr1/dunWdb2KrU6eOCgsLVVxcLMn1tfPYJ3+NGjVKzz33nFJT\nU5WXl6chQ4Zo48aNmj9/vowxuuuuuzR8+HBPHNrjSprb6tWr9fbbb8vf31833HBDiadrbOZwOBQX\nF6e0tDR5e3vrySef1OHDh6vN2pU1v6q+fpI0Z84cbd26VcYYjR8/XidPnqw26yeVPr/qsH7/+Mc/\n5Ofnp1GjRkmSPvzww2q1fqXNryqvX25uriZNmqTjx4+rsLBQo0aNkjHmotaOj+QEAMAifMAIAAAW\nIcwAAFiEMAMAYBHCDACARQgzAAAWIcwAAFiEMAMAYBHCDACARf4/xQhgSbK5q2EAAAAASUVORK5C\nYII=\n", 243 | "text/plain": [ 244 | "" 245 | ] 246 | }, 247 | "metadata": {}, 248 | "output_type": "display_data" 249 | } 250 | ], 251 | "source": [ 252 | "plt.hist(**data);plt.title(title);_=plt.axvline(5.8, color='black', linewidth=4)" 253 | ] 254 | }, 255 | { 256 | "cell_type": "markdown", 257 | "metadata": { 258 | "slideshow": { 259 | "slide_type": "slide" 260 | } 261 | }, 262 | "source": [ 263 | "# Gaussian Mixture Models" 264 | ] 265 | }, 266 | { 267 | "cell_type": "markdown", 268 | "metadata": { 269 | "slideshow": { 270 | "slide_type": "notes" 271 | } 272 | }, 273 | "source": [ 274 | "Here's what our actual underlying data looks like.\n", 275 | "\n", 276 | "Without knowing the _latent_ classes, we could _assume_ that each class is drawn from a normal distribution." 277 | ] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "execution_count": 5, 282 | "metadata": { 283 | "collapsed": false, 284 | "slideshow": { 285 | "slide_type": "-" 286 | } 287 | }, 288 | "outputs": [ 289 | { 290 | "data": { 291 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAFeCAYAAABQCrzXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X1c1eX9x/E3dwJyI2Fp01iay3szxZyNUDNdWuZdOW8S\npbGstlrLWoqaLrxhNlc9Qi3LdWd72JxhuT22bJo3zdQYoSXmTZaagkxREhDkANfvD3+eRdwfD57r\nwOv5V5y77+c618GX53g6x8cYYwQAAKzg6+kBAADA/xBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAs\nQpihXbt2afLkyRo5cqTuuusuTZ06VV9++WWDHOuTTz7RXXfdVen048ePq1evXg1yzO/7/PPPNXfu\n3BrnqcrcuXM1ePBgPf/88y4fe8mSJZo/f36V573wwgt677336nV7J0+e1GOPPaa77rpLI0aM0Lhx\n47Rx48Y6XTchIUF5eXn1Op4nnDlzRp07d67yvMTERL322mtVnjd79mxt3769zsdJTEzUnDlzKp2+\nfv16jRo1Sv/97381YcKEOt/eRQ888IAOHTpU42Vc2Xs0Xv6eHgCeVVJSogcffFCvv/668w+/devW\naerUqdq4caN8fHwu2yyX61gHDx5UTk5Ova+3evVqbd68Wa1bt26AqaRf//rX9b7O7Nmz9ZOf/ETP\nPfecJOnQoUOaOHGiVq1apeuuu67G627bts2lOS83Y4xLj43q/gJUnXvvvVf33XefZs+erWbNmjlP\nX716te699161atVKq1atqvccy5cvr/Uyruw9Gi/C3MQVFxeroKBABQUFztNGjBihsLAwlZWVyd/f\nX5s2bdKLL76o0tJSBQUFafr06erZs6eWLFmigwcP6tSpUzp16pS6du2q+fPnKyQkRJs2bdLy5ctV\nWlqq06dPa+TIkXr00UddmtHhcGjx4sVKS0tTeXm5unTpotmzZyskJESDBg3SmDFjtH37dmVnZ2vY\nsGH67W9/K0l6+eWX9c477ygkJER9+vTRhg0btGrVKqWkpKigoEAzZ87UqFGjVFhYqGnTpumrr75S\nSUmJ5s2bp+jo6Aoz3HvvvZKk+++/X3PnzlWLFi2UlJSkvLw8+fr6Kj4+XqNGjdInn3yiBQsWKDg4\nWMXFxfrrX/+qgICAKteVmJiovLw8HTt2TAMHDtSpU6fUsWNH3XfffXrhhRe0ceNGBQQEKCIiQr//\n/e915ZVXVrqNkydP6vz58854dejQQcuWLVOLFi0kXQj1woULlZeXp/LycsXFxWnMmDFKTEyUJE2e\nPFmvvPKKzp49q3nz5lVaz7lz55SYmKijR4/Kx8dH3bt3V1JSkowxWrhwoT777DMVFhbKGKP58+dX\netWjpsslJiYqJCREBw4c0IkTJ3TdddfpueeeU3BwsD744AM9//zzCg4OVvfu3ev0OImLi1NERIS+\n/vprTZgwQe+//77i4uJ02223KSkpSRkZGQoICFBUVJSSk5MVHBxc4frdu3dX+/bt9f7772vEiBGS\npGPHjikzM1NLly7V8ePHNXz4cGVkZGjJkiXKyMjQyZMn1blzZyUlJWnOnDnavXu3WrRooQ4dOkiS\nkpOTNWjQIKWkpKiwsFDPPfecoqKidPDgQTkcDs2ZM0d9+/ZVYmKic+93796tBQsWqKioSAEBAXry\nySfVr18/rVmzRqtXr1Zpaany8vJ0//33u/QMHl7AoMl77bXXTM+ePc3gwYPNb3/7W7NmzRpTVFRk\njDHm8OHDZvjw4SYvL88YY8zBgwdNTEyMKSoqMikpKWbgwIEmNzfXGGPMtGnTzKJFi4wxxkyePNkc\nOXLEGGNMTk6O6dq1qzlz5ozZuXOnGT58eKUZjh07Znr16lXlfEuWLDHPPPOM8+dnn33WPP3008YY\nY2699VbnMU+cOGFuuOEGc+zYMbN161YzbNgwk5+fb4wxZubMmWbQoEHGGGNSU1PNAw88YIwxZufO\nnaZbt27ms88+c94X8fHxVc7RqVMnk5eXZ0pLS83gwYPNv/71L+f6+vfvb3bt2mV27txpunbtarKz\ns6u8jZSUFDNv3jxjjDEzZsww9913n/O8GTNmmFdffdVkZ2eb6OhoU1JS4pxpw4YNVd7ejh07zC23\n3GL69etnHnroIbNixQpz4sQJY4wxpaWl5s477zR79+41xhiTn59v7rjjDrN79+46r+fdd981v/jF\nL4wxxpSVlZmnnnrKHD161GRkZJhHH33UOcfy5cvNgw8+WGm+mi43Y8YMM2HCBONwOIzD4TCjR482\nqamp5tSpU6ZPnz7m0KFDzut07ty5yvVfvM+MMWbSpElm1qxZzvMmTZpk1q9fb9LS0sywYcOcpy9e\nvNhkZGRUeXtr1641kyZNcv787LPPmuTkZGNMxcdoSkqKGTZsmCkvLzfGGPPHP/7RPP744877ecSI\nEWbGjBnGmAuP0T179jgfa/v27TPGGPPqq686j3VxHQ6Hw8TExJgtW7YYY4zZs2ePueuuu0x+fr4Z\nN26c8/dw165d1f6+wPvxjBmKj4/Xz372M6WlpSktLU2vvPKKVqxYob/+9a/atm2bTp06pfj4eJn/\n//RWf39/HTlyRJI0dOhQRUZGSpLuueceJScn68knn9SLL76ozZs3a926dfrqq68kSUVFRS7Nt3nz\nZuXn5ztfei0tLVXLli2d5992222SpNatW6tly5b69ttvtXXrVg0dOlShoaGSLjzj3bFjR5W3HxUV\npR49ekiSunTpotTU1GpnMcbo8OHDKikp0eDBgyVJrVq10k9/+lN99NFH6tu3r66++mpdffXVdVpb\n7969K53WunVrdenSRaNHj1ZsbKz69++vm2++ucrr//jHP9aWLVu0a9cu/ec//9GmTZu0bNkyvfHG\nGwoODtbRo0c1c+ZM596dP39ee/fu1Q033FCn9YwePVrPP/+84uLiFBMToylTpigqKkpRUVF69NFH\ntWrVKh09elSffPKJ877+rhtvvLHGy8XGxsrf/8IfQx07dtS3336r9PR0derUyflS/Lhx45wv1dem\nT58+lU7r1KmT/Pz8NHbsWN1yyy0aMmSIc/3fd8cdd+iZZ57RN998ozZt2mjt2rV66623qrxsz549\nnS+xb9myxfkqRGhoqEaPHq39+/dXuk6bNm3UqVMnSVLXrl21du3aCucfOHBA/v7+6t+/vySpW7du\nWrdunSTppZde0qZNm3TkyBF98cUXLv8+wX6EuYn79NNPlZGRoYSEBA0YMEADBgzQtGnTNHz4cH38\n8ccqLy/XzTffrGeffdZ5nRMnTqhVq1b617/+JT8/P+fpxhj5+fmpqKhIo0aN0k9/+lP16dNH99xz\njzZs2OCMQ32VlZVp1qxZio2NlXQh8OfPn3eeHxQUVOHyxhj5+/tXOJ6vb/Xvc7wYBunCv3NXN+fF\nP4TLy8srnWeMkcPhkCQ1b968tiU5hYSEVHmclStXas+ePfr444+VnJysH//4x7r55pv1wgsvyMfH\nR61atVJycrJSUlI0Z84c9e7dW71799bUqVM1e/Zsvfvuu/rZz36m8PDwCn/45+bmKiwsrMLxalpP\n27ZttX79eqWlpWnHjh2aMmWK5syZo8DAQC1YsEA///nPNXjwYF133XX629/+Vul2Nm/erIULF1Z7\nue/u3cX73sfHp8JM332M1aaq+z4sLEzvvfeePv30U+3YsUOPPfaY4uLiFBQUpLffftv5Ev28efPU\nrFkzjRkzRmvWrFGPHj3UqVMn/fCHP6zyWN/dOz8/vzo93gIDAyut97uqWuvBgwcVHh6ucePGady4\ncerTp49uv/12bdmypeY7A16Ld2U3cZGRkXrppZf06aefOk/LyclRcXGxOnbsqH79+mnbtm3OZ71b\ntmzRyJEjVVJSIknauHGjCgoKVF5ertWrV2vQoEE6cuSIzp07p9/85jcaOHCgdu7cKYfDobKyshpn\nqS6IsbGx+vOf/yyHw6Hy8nLNmjWrwl8UqjJgwAB98MEHzn87X7NmjTOsfn5+Ki0trdsdVMV87du3\nV0BAgDZs2CDpwv21fv16xcTE1Ps2q7Jv3z4NHz5cHTp00NSpUxUfH6/9+/dr0KBBevfdd7V27Vot\nX75cLVq00Mcff6w33njDed2ioiJlZ2erW7duat++vQIDA53PuLKzszV8+HBlZmZK+t/9UNV6Pvjg\nA8XExGjVqlVKTExUTEyMHn/8ccXGxurAgQPatm2bBg0apPHjx6t79+7auHFjlYH/+OOP63S574qO\njtahQ4eczzhregWjLjZv3qwpU6aoV69eevjhhzVq1Cjt379f48ePd96f8+bNc15+3Lhx+uc//6m1\na9c631tQm1tvvVWpqakyxqioqEh///vfXXrDWvv27eXr6+t8N3lmZqbi4+OVkZGhyMhIPfTQQ4qJ\nidGmTZskVf87A+/GM+Ymrl27dlq6dKmeffZZ5eTkKDAwUKGhoZo3b57atWsnSUpKStK0adMkXfjD\n/MUXX3Q+07nyyis1depUnTlzRjfddJMeeOABBQQEaODAgRo6dKjCw8N17bXX6kc/+pGOHj1a7Ruh\npAtvRLv40u7FZ05/+ctf9Mtf/lKLFi3S6NGjnW/+mj59uqTK7+S++HO/fv00duxYjR8/XkFBQbr+\n+uudb/bp1auXnn/+eT3yyCOKi4ur83118bb9/f21dOlSzZ8/Xy+88ILKy8v1yCOPqG/fvvrkk0/q\nfHvV6dy5s4YNG6YxY8aoefPmCg4O1uzZsytdzs/PT6+++qqeeeYZrVy5Us2bN5ePj4/GjBmj0aNH\nS5KWLVum+fPna8WKFSorK9Njjz3mfIPWkCFDNHHiRC1btqzSeh5++GH17dtXPXr0UFpamu644w4F\nBwerbdu2mjJlik6ePKknnnhCI0eOlJ+fn/r06aMPPvig0ozjx4+v0+W+KzIyUosXL9YTTzyhgIAA\n9e3bt073W3WPhQEDBuijjz7S8OHD1bx5c0VERFQI8fdFRUWpffv2+vLLLzVw4MA6HXvq1KlKSkrS\niBEjFBoaqpYtWzofb/UJdLNmzZSSkqIFCxZo0aJFatasmZYsWaIuXbrovffe0+23366QkBD16NFD\nkZGROnLkiPP3FI2Hj+GvXHDRkiVLlJeXV2U0PG3Pnj3KyMhwhvf111/XZ599VuszbcAV//jHPxQS\nEqIBAwbIGKNHHnlEt9xyi8aPH+/p0eCFanzGXFpaqpkzZ+r48eNyOBx68MEH9aMf/UgzZsyQr6+v\nrr/+eucHNQA2adeunV555RWtXr1aktS2bVslJSV5eCo0Vtdff73mzJmj5557Tg6Hw/mKDeCKGp8x\np6amav/+/UpMTNTZs2c1cuRIde7cWQkJCerTp4/mzp2r2NhY57s5AQDApanxzV/Dhg1zfihEWVmZ\n/Pz8tHfvXuf/ktC/f/96feQdAACoWY1hDg4OVvPmzVVQUKBHH31Ujz32WIV3AYaEhCg/P7/BhwQA\noKmo9V3Z2dnZevjhhzVp0iTdeeed+sMf/uA8r7CwUOHh4bUeJD09/dKmBADAC33/433rosYwnzp1\nSgkJCZozZ4769esn6cInI6Wlpemmm27S1q1bnac3xHDeIj09nfV5MdbnvRrz2iTW5+1cfVJaY5iX\nL1+us2fPOv8/Rx8fH82aNUvz58+Xw+FQhw4dNHToUJcODAAAKqsxzLNmzdKsWbMqnb5y5coGGwgA\ngKaMj+QEAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQ\nZgAALFLr1z7i8nA4HHrlz/9QcEiYp0ep5PjxLH321bfq0q6l+t3U09PjAECjRpgtYYyRb2C4AsN/\n4OlRKgkqkALD26ik7JynRwGARo+XsgEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZAACL\nEGYAACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDA\nIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEA\nsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZAACLEGYA\nACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZAACLEGYAACxCmAEAsAhhBgDAIoQZ\nAACLEGYAACxCmAEAsAhhBgDAIoQZAACL1CnMu3fvVlxcnCTpiy++UP/+/TV58mRNnjxZ//znPxt0\nQAAAmhL/2i6wYsUKvffeewoJCZEk7dmzRz//+c8VHx/f0LMBANDk1PqM+dprr9XSpUudP2dmZmrz\n5s2aNGmSZs2apXPnzjXogAAANCW1hnnIkCHy8/Nz/tyzZ089+eSTeuuttxQVFaWUlJQGHRAAgKak\n1peyv2/w4MEKCwuTdCHa8+fPd/tQAC6/Y9nHdeDYl54eo1qlDof6975FQUFBnh4FaFD1DnNCQoKe\neuop9ejRQ9u3b1e3bt3qdL309PR6D+dNLnV9DodDJ7JPqKDYuGki98rKylJRzimFBNg536Xi8Snt\nPbRPpyMKL8M0rin8tkD+xb5q0aJFhdPZO+/W2NfninqH+Xe/+53mzZungIAAXXXVVUpKSqrT9aKj\no+s9nLdIT0+/5PWVlJRo19dFCo/8gZumcp+srCy1adNG17SIUHR0b0+P43bu2D+b1XV9Zf5Gh/1O\nXIaJXJMffFY3drtRkZGRztPYO+/WFNbnijqFuW3btnr77bclSV27dtWqVatcOhgAAKgZHzACAIBF\nCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBg\nEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAA\nWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMA\nABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABbx9/QA8B7fHD+lTf/+\n1NNjVOuqlmHq3uV6T48BAJeEMKPOfEJ/qOxCT09RvXPncwkzAK/HS9kAAFiEMAMAYBHCDACARQgz\nAAAWIcwAAFiEMAMAYBHCDACARQgzAAAWIcwAAFiEMAMAYBHCDACARQgzAAAWIcwAAFiEb5cC4DXy\n8vJkjKnwc25urgcnqig8PFwBAQGeHgNejjAD8AohLUL14fHt0vH/nZZzOkdf7z3huaG+p1eLLoq+\nobenx4CXI8wAvIKvr6/Cr2hR4bTCosJKp3lUuacHQGPAvzEDAGARwgwAgEUIMwAAFiHMAABYhDAD\nAGARwgwAgEUIMwAAFiHMAABYhDADAGCROoV59+7diouLkyQdPXpUEydO1KRJk/T000836HAAADQ1\ntYZ5xYoVmj17thwOhyQpOTlZ06ZN01tvvaXy8nJt2LChwYcEAKCpqDXM1157rZYuXer8OTMzU336\n9JEk9e/fX9u3b2+46QAAaGJqDfOQIUPk5+fn/Pm7X7kWEhKi/Pz8hpkMAIAmqN7fLuXr+7+WFxYW\nKjw8vE7XS09Pr++hvMqlrs/hcOhE9gkVFJvaL+wBWVlZnh6hVqfLcpSe7tp34fL4lPYf2q/TEYWX\nYRr3yrbosen49qzkcO/vMI/NpqfeYe7atavS0tJ00003aevWrerXr1+drhcdHV3v4bxFenr6Ja+v\npKREu74uUnjkD9w0lftkZWWpTZs2nh6jVi38m7u0D+7YP5vVdX1l/kaH/ez5buO6yM7K0g8semx2\nuLqNW7+Pmcemd3P1Lx31DvP06dP11FNPyeFwqEOHDho6dKhLBwYAAJXVKcxt27bV22+/LUlq166d\nVq5c2aBDAQDQVPEBIwAAWIQwAwBgEcIMAIBFCDMAABYhzAAAWIQwAwBgEcIMAIBFCDMAABYhzAAA\nWIQwAwBgEcIMAIBF6v0lFt7q27P5yvj8gHx9fdx+2wcPHlJhyaXdbmlpmXx8m8x2NIjTeYXauv3T\nel/PHftXFyHBQYq+sWuDHweAd2syJTiVe1pfn/JVYFCw22/7TNmVOna2+SXfTlhEmBumabr8Qtvq\n2Nn6X89d+1cb35P/VfSNDX4YAF6Ol7IBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAA\nixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYA\nwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgB\nALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBmAAAsQpgBALAIYQYAwCKEGQAAixBm\nAAAsQpgBALAIYQYAwCKEGQAAixBmAAAs4u/pAYCmIvfMGf370+2X/bgHvzygIp+SWi+XlZMlRQVe\nhomAqu3/6oBO5uV6eoxqNW8WpN7dezX4cQgzcJmcOpurrGZ+l/24Z684r6xmdfjDjijDw7488bUK\nwx2eHqNapf8tVm81fJh5KRsAAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAi\nhBkAAIsQZgAALOLyR3KOGTNGoaGhkqRrrrlGCxcudNtQAAA0VS6FuaTkwgfiv/nmm24dBgCAps6l\nl7L37dunc+fOKSEhQfHx8dq9e7e75wIAoEly6RlzUFCQEhISNHbsWB0+fFj333+/1q9fL19f/ska\nAGx1/vx5lZWVeXoMp+LiYp07d875c1lZuQensYePMcbU90olJSUyxigw8MLXxI0dO1ZLlixR69at\nq7x8enr6pU3pBsezTmjPcR81Cwzy9Chooo4e/o86xER4egw0oCu/DVXn6zp5eoxqvf+fjTJh9j6B\n8g8MUHBYsKfHqNb5E+d0W7fYel0nOjq63sdx6RnzO++8owMHDmju3LnKyclRYWGhrrrqKrcP504R\nkUeUW35agUHu3/SsrCy1adPG7bdrC9bnHoUFV+oHbVo1+HG+LzsrSz9opPtn29o6XN1G0Tf0dtvt\npaenu/XPzoP5h+V7pT3fu23b/tWm1L+4Xvvh6pNSl8J8zz33KDExURMnTpSvr68WLlzIy9gAALiB\nS2EOCAjQ4sWL3T0LAABNHk9zAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQZgAA\nLEKYAQCwCGEGAMAihBkAAIu49FnZAIDKMrP260jecbfd3tGjR3WkMMttt1fiW6og2fPtUqgaYQYA\nNwm6Jkylbrw9n+IAlUb6uO32ghTitttCw+GlbAAALEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCw\nCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQZgAA\nLEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkA\nAIv4e3oAwF3Ky8tljHHpemVlZQ0wUUWuzAag6SHMaDS++PqAzvuU1Pt6+fn5+m/J6QaYqCJH6VlJ\nrRv8OAC8G2FGo+EX4KvAoKB6X6/EOBQYWv/r1Zevb8MfA4D349+YAQCwCGEGAMAihBkAAIsQZgAA\nLEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCN8uZYny8nJ9\n8fV++QX4eXqUSvLyvtXZ0gJFhlyhVi2v8vQ4XqusLEC7dnxz2Y+bl5ennKO1f990WWmxom+5/jJM\nBKAmhNkSxhgV+zgUFGRfmBXiq7Igqbik2NOTeLWgsFYeOa5/83D5BIbVerniwmOXYRoAteGlbAAA\nLEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwCGEGAMAihBkAAIsQZgAALEKYAQCwiEsfyWmM0e9+\n9zvt379fzZo104IFCxQVFeXu2QAAaHJcesa8YcMGlZSU6O2339bjjz+u5ORkd88FAECT5FKY09PT\nFRsbK0nq2bOn9uzZ49ahAABoqlx6KbugoEBhYf/7thp/f3+Vl5fL19fef7IuLyvVZ+kfyc/P/V+o\ndfr0aZ04GnlJt1FeXq4zJWcVHBrspqncJz8/X6VhYco5U6RTRw56epxqnS7IU1BE/e+/i+trrOq6\nvrO52fp36qHLMJH7nDl9WldEXtrvns1Yn12uCI2QBjX8cVyqVGhoqAoLC50/1yXK6enprhzKrSaN\njPH0CAAAL3Y5WuZSmHv37q1NmzZp6NCh2rVrlzp27Fjj5aOjo10aDgCApsbHGGPqe6XvvitbkpKT\nk9W+fXu3DwcAQFPjUpgBAEDDsPfdWgAANEGEGQAAixBmAAAs4v7/qVdSbm6u7r77br322msV3hT2\n4YcfatmyZfL399fdd9+tsWPHNsThG1R1a3v99de1Zs0aRf7//5OXlJSkdu3aeWhK14wZM0ahoaGS\npGuuuUYLFy50ntcY9q6m9TWG/Xv55Zf14YcfyuFwaOLEibr77rud5zWG/atpfd6+f2vXrlVqaqp8\nfHx0/vx57du3T9u2bXM+Xr19/2pbnzfvX2lpqaZPn67jx4/L399f8+bNu/TuGTdzOBzmV7/6lbn9\n9tvNV199VeH0IUOGmPz8fFNSUmLuvvtuk5ub6+7DN6jq1maMMU888YTJzMz00GSX7vz582b06NFV\nntcY9q6m9Rnj/fu3c+dO8+CDDxpjjCksLDQpKSnO8xrD/tW0PmO8f/++6+mnnzarV692/twY9u+7\nvr8+Y7x7/zZs2GB+85vfGGOM2bZtm3nkkUec57m6d25/KXvRokWaMGGCWrVqVeH0Q4cO6dprr1Vo\naKgCAgK9QJOqAAADjElEQVQUHR2ttLQ0dx++QVW3NknKzMzU8uXLNXHiRL388ssemO7S7Nu3T+fO\nnVNCQoLi4+O1e/du53mNYe9qWp/k/fv373//Wx07dtQvf/lLPfTQQ7r11lud5zWG/atpfZL3799F\nn3/+ub788ssKz6oaw/5dVNX6JO/ev3bt2qmsrEzGGOXn5ysgIMB5nqt759aXslNTU9WyZUvFxMTo\npZdeqnDe9z/GMyQkRPn5+e48fIOqaW2SdOedd+ree+9VaGiofvWrX2nLli0aMGCAByZ1TVBQkBIS\nEjR27FgdPnxY999/v9avXy9fX1+v3zup5vVJ3r9/Z86cUVZWlpYvX65vvvlGDz30kN5//31J3v+7\nJ9W8Psn79++il19+WQ8//HCF0xrD/l1U1fok796/kJAQHTt2TEOHDlVeXp6WL1/uPM/VvXPrM+bU\n1FRt27ZNcXFx2rdvn6ZPn67c3FxJFz7Gs6CgwHnZwsJChYeHu/PwDaqmtUnSlClTFBERIX9/fw0Y\nMEB79+714LT1165dO40YMcL53xERETp58qQk7987qeb1Sd6/fxEREYqNjZW/v7/at2+vwMBAnT59\nWlLj2L+a1id5//5JFz7T/PDhw+rbt2+F0xvD/knVr0/y7v17/fXXFRsbq/Xr12vdunWaPn26SkpK\nJLm+d24N81tvvaWVK1dq5cqV6ty5sxYtWqSWLVtKkjp06KAjR47o7NmzKikpUVpamm688UZ3Hr5B\n1bS2goICDR8+XEVFRTLGaMeOHerWrZuHJ66fd955R7///e8lSTk5OSosLNRVV10lyfv3Tqp5fY1h\n/6Kjo/XRRx9JurC+4uJiXXHFFZIax/7VtL7GsH+SlJaWpn79+lU6vTHsn1T9+rx9/1q0aOF8E1tY\nWJhKS0tVXl4uyfW9a7BP/po8ebKefvppZWZmqqioSGPHjtXmzZu1ZMkSGWN0zz33aMKECQ1x6AZX\n1drWrVunN998U4GBgbr55purfLnGZg6HQ4mJicrKypKvr6+eeOIJHTt2rNHsXW3r8/b9k6TFixdr\nx44dMsZo2rRpOnPmTKPZP6nm9TWG/fvTn/6kgIAATZ48WZL097//vVHtX03r8+b9O3funGbOnKmT\nJ0+qtLRUkydPljHmkvaOj+QEAMAifMAIAAAWIcwAAFiEMAMAYBHCDACARQgzAAAWIcwAAFiEMAMA\nYBHCDACARf4P3YqW1ZdHMO4AAAAASUVORK5CYII=\n", 292 | "text/plain": [ 293 | "" 294 | ] 295 | }, 296 | "metadata": {}, 297 | "output_type": "display_data" 298 | } 299 | ], 300 | "source": [ 301 | "plt.hist(**data2);_=plt.title(title)" 302 | ] 303 | }, 304 | { 305 | "cell_type": "markdown", 306 | "metadata": { 307 | "slideshow": { 308 | "slide_type": "slide" 309 | } 310 | }, 311 | "source": [ 312 | " ➡ Forward Sampling\n", 313 | "Let's generate some data" 314 | ] 315 | }, 316 | { 317 | "cell_type": "code", 318 | "execution_count": 6, 319 | "metadata": { 320 | "collapsed": false 321 | }, 322 | "outputs": [], 323 | "source": [ 324 | "# Fixed model parameters\n", 325 | "mixture_proportion = [0.50, 0.50]\n", 326 | "mean = [5.01, 6.59]\n", 327 | "std = [0.35, 0.64]\n", 328 | "\n", 329 | "data = []\n", 330 | "for _ in range(100):\n", 331 | " iris_type = random.choice(2, p=mixture_proportion)\n", 332 | " sepal_length = norm(loc=mean[iris_type], \n", 333 | " scale=std[iris_type]\n", 334 | " ).rvs()\n", 335 | " \n", 336 | " data.append((iris_type, sepal_length))" 337 | ] 338 | }, 339 | { 340 | "cell_type": "code", 341 | "execution_count": 7, 342 | "metadata": { 343 | "collapsed": false, 344 | "slideshow": { 345 | "slide_type": "slide" 346 | } 347 | }, 348 | "outputs": [ 349 | { 350 | "data": { 351 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAFeCAYAAABQCrzXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Hl8VNX9//H3JBMSSCJLWApogSJLEUSNaCymkSAKCChb\nWSSBbymyKKUCJSzFKBEKllb6BRTQVr7F70NUhIrWr6AgLiyCARRQEAQEkhAgEMieTHJ+f/BjZElC\nMsyQk/B6/pXMPeeez5lzZ965NzPXYYwxAgAAVvCr6AIAAMBPCGYAACxCMAMAYBGCGQAAixDMAABY\nhGAGAMAiBLOFdu7cqdjYWD366KPq2bOnnnjiCR04cECStHv3bo0bN85rY0VHR2vPnj2lttm1a5fi\n4+PLve+EhAQtWLCg2G0rV65Unz599Nhjj6lnz56aPn26MjMzyz1GWSxYsEDPP//8FY+vWrVKo0aN\n8smYl3v77bf1xhtvlFrP5TIzMzVw4ED17NlTH330kcdjx8TEaO3atcVuGzlypH744Ydy7a+047M0\nx44d0+9///tyjVVR1qxZo5iYmGK3lfaa6d27d7mO4y5duuiDDz644vGEhATNnDlT69ev18yZM8u8\nP0k6ceKEBg0adNV2nqw9rg9nRReAS+Xn52vUqFFaunSpWrduLUlavXq1nnjiCa1bt05t27bV3//+\n9+ta0/79+5Wamuq1/e3atUsvvfSSVq1apdDQUBlj9Oyzz+rZZ5/V3LlzvTaOTbZv366WLVuWq893\n332nM2fOaM2aNT6qSlq8eHG52l/t+HQ4HCX2TUpK0qFDh66p3uuptLmUZNWqVeVqP3jwYK1YsULd\nu3d3P5aXl6f3339fb731lpo0aaLo6Ohy7bN+/fruPwJLU961x/VDMFsmNzdXmZmZl/zV3atXL4WG\nhqqwsFDbt29XQkKC3nvvPU2ZMkWBgYHatWuX0tLS1LVrV9WpU0fr169XWlqann/+ed17772aMmWK\nWrZsqf/6r/+SpCt+lyRjjGbOnKldu3YpKytLxhg9//zzatiwoebPn6/MzExNnTpVs2bN0vr167Vo\n0SK5XC4FBQVp0qRJuuOOO5SZmak//elP2rdvn+rVqyd/f3+Fh4dfMceTJ0/KGKPs7GyFhobK4XBo\n3Lhxl5x1LVq0SGvXrpUxRo0bN1Z8fLzq1aunmJgY3Xrrrdq9e7fS09PVq1cvjR071t1n3bp1ys/P\nV05OjiZNmqQHH3zQo3VITU1VQkKCUlJS5HK59Mgjj+iJJ55QUlKShg0bpqioKH399dc6d+6cxo0b\np+7duys3N1fx8fHauXOnatasqebNm0uSOnfurPXr12vTpk0KDAyUJP3www+KjY3VyZMnVbduXb34\n4ouqW7eue/xDhw5p2rRpOnHihHr37q0333xTn332mRYuXKiioiKFhIQoLi5Ot99+uxYsWKAdO3bo\n5MmTat26tV544YUS5xUdHa327dvr+++/19NPP61Zs2Zp/vz5atasmaZMmaIjR47I4XCobdu2mjFj\nxhX9r3Z8Op1OffLJJ3r55Zfdx0dcXJzatWun6dOn68SJE/rd736nV199VR9//HGx8zl48KCmTZum\n/Px8GWPUr18/DR48WGlpaXrmmWeUlpamU6dOqVGjRpo3b57q1KlzSY2ltYuOjlafPn20efNmpaSk\nqFu3bvrjH/8oSfr73/+u999/X7Vr19bPf/7zMh0n7dq1U+fOnbVv3z795S9/Ub9+/bRlyxa5XC7F\nxcXpzJkzkqSoqKhir3T16dNH//3f/62UlBQ1bNhQkvTBBx+obdu2atKkiVatWqU1a9Zo0aJFiomJ\nUa1atXTo0CENGjRIkZGRmjp1qs6ePat69erJGKNHH31UHTp0UI8ePbRjxw4tWLBASUlJOnHihJKT\nk1WnTh3NmzdP9erVU3R0tObPn6/bbrtNK1as0NKlS+Xv76/atWtrzpw5ql+/frHvCXfeeWeZnhtc\nAwPrvPbaa6Z9+/bmwQcfNH/84x/NihUrTE5OjjHGmC+//NL06NHDGGPM5MmTzYABA0xhYaE5efKk\nadWqlXn99deNMcb8z//8j/ntb3/rbvfPf/7Tvf+Lf+/UqZPZvXu32bFjhxk3bpy7zeLFi82oUaOM\nMcasXLnSjBw50hhjzOHDh02PHj1Menq6McaY/fv3m44dO5qcnBwzc+ZMM3nyZGOMMWlpaSYqKsrM\nnz//ivkVFBSYCRMmmDZt2pjevXubGTNmmA0bNri3r1q1yjz99NOmsLDQGGPMm2++aUaMGGGMMWbI\nkCFm5MiRprCw0Jw7d8507drVbNiwwSQlJZmhQ4eavLw8Y4wx//nPf0zPnj2NMcbMnz/fJCQkXFHH\nxfO6XGxsrPnkk0+MMcbk5eWZ2NhY83//93/m2LFjplWrVu5616xZYzp16mSMMWbu3LlmwoQJxhhj\nMjIyTK9evdzPx8XP+fz5882DDz5ozpw5Y4wxZsyYMeall166ooaL1/rAgQOmY8eO5tixY8YYYzZv\n3mw6duxoMjMzzfz58023bt1MUVFRsXMZMmSIWbNmjTHm/HpfPFZ0dLTZvXu3+fe//21+97vfGWOM\nKSwsNNOnTzdHjhwpdn+lHZ+lHR9lnc/UqVPNkiVLjDHGnDx50owfP94Yc/6YfuWVV9x1jBgxwrz2\n2mtX1Fdau06dOpk5c+YYY4w5fvy4uf32282xY8fMRx99ZHr06GGys7NNYWGhGTlypImJiSl2/hde\nM8YY06pVK7N69Wr3ttatW5szZ86YhQsXmvj4eGOMMdnZ2Wb8+PEmIyOj2P1NmTLlktfJwIEDzfr1\n640xlx6jQ4YMMdOmTXO3GzBggFm+fLn7+bzjjjvMqlWrzLFjx8ydd95pjDl/rHXp0sVkZWUZY4wZ\nNWqUe6wL8/juu+9MRESEOX78uPv5i4+PL/U9Ab7FGbOFhg0bpt/85jfatm2btm3bpldeeUWvvvqq\n3n777SvadurUSX5+fqpbt66qV6+uyMhISdLPf/5znT17tsxj3nHHHRo3bpzeeOMNHTlyRFu3blVI\nSMgV7TZu3KhTp05p2LBhMv//bq5Op1OHDx/W5s2bNW3aNElSnTp1SjxbdTqdmjt3ruLi4vTll19q\n69atmjx5su677z797W9/04YNG7Rr1y716dNHklRUVKS8vDx3/wEDBsjPz0+hoaHq2rWrPv/8c0VF\nRWn27Nl69913deTIEe3cuVPZ2dllnv/FcnJytG3bNp07d07z5s1zP/bdd9+pXbt2CggIUFRUlCSp\nTZs27uf5s88+05QpUyRJISEh6t27t/bt21fsGL/61a9Uq1YtSVLr1q2VlpZWak1ffvml7rvvPjVu\n3FiSFBERobp167r/19m+ffsyX3q9++673T9fWMPw8HDNmzdPMTEx6tixo4YOHapbbrml2P6lHZ8l\nHR8//vhjmefz0EMPadKkSfrmm2903333uY+p2NhYffXVV1q6dKkOHz6sAwcOqH379lfUd7V2nTt3\nliQ1aNBAYWFhOnv2rLZs2aIuXbqoevXqkqS+fftq2bJlZXo+L74qdGHOkZGRGjlypJKTk/WrX/1K\nEyZMKPb1JEmDBg3SuHHj9NRTT2n//v06ceKEHnjggWLbXli7c+fO6ZtvvtH//u//SpKaN2+uiIiI\nYvvcc889qlGjhqTzx2t6evol27ds2aLIyEg1aNBA0vnn74KyvCfA+whmy2zfvl07duzQ8OHDFRUV\npaioKI0fP149evTQpk2b3G/mF1SrVu2S353O4pfUXHRL9Pz8/Cu2b9iwQbNmzdJvf/tbPfjgg/rF\nL36h995774p2RUVF7gC94Pjx46pfv74cDscl45RUyzvvvKPatWsrOjpaPXr0UI8ePTR69Gh17txZ\nzzzzjIqKijRixAgNHDhQklRQUKBz5865+/v7+19Sj7+/v7799luNGTNGw4YN0/33368OHTroueee\nK3b8qyksLJQkvfnmm+7n98yZMwoKCtLp06cVEBDgbnvxnP39/S+Zv59fyZ+tvHwfV2OKuaV9YWGh\nXC6XJCk4OPiq+7jgwpv0xW6++WatXbtWW7du1ZYtWzR06FBNnz5d+/bt0/r16+VwOBQdHa2OHTuW\nenyWdHw0aNBA27ZtK9N8oqKitHbtWm3cuFGbN2/WwoULtXz5ci1fvly7d+9W3759FRERIZfLVex+\n/vKXv5TaLigo6JL2F7Zd3ObiY8yT57Ndu3Zat26dNm3apC1btqhfv35auHCh3nnnHe3evVsOh0MD\nBw7UgAED1K5dO9WtW1ebNm3Shg0bNHDgwBKPiQtj+fn5XfF6K6nmi+db3H79/f0veTwvL09JSUk6\nevSoZs6cedX3BHgfn8q2TJ06dbRo0SJt377d/Vhqaqpyc3PL/eGhi/e5e/duSdLp06eVmJh4RZtN\nmzYpOjpaAwcOVNu2bbVu3ToVFRVJOv/CvRAAERER2rhxow4ePChJ+vTTT/Xoo48qPz9fkZGRWrFi\nhYwxOnv2rNatW1dsPX5+fvrrX/96yQfKDh06pMaNG6tmzZq6//779fbbb7v/jzlv3jxNmjTJ3Xb1\n6tXuMT788EN16tRJ27ZtU7t27TRs2DB16NBBH3/8sbv+8goJCVH79u31j3/8Q9L5s5NBgwa551Nc\nGEjSAw88oJUrV8oYo5ycHL3//vvuNzx/f38VFBR4VI/00/N+7NgxSdLmzZuVmpqq22+/3eN9XuyN\nN97Q5MmT1bFjR02YMEGRkZHav3+/fv/73+vf//63Vq1apbFjx171+Czp+MjLyyv2OCpuPhMmTNB/\n/vMfde/eXfHx8QoJCdHx48e1adMmDR06VL169VLt2rXdfwhcbuPGjWVqd7HIyEh9+OGHysjIUFFR\nkd59991rej7/+te/auHChercubOmTZumW2+9VT/++KOef/559/M5YMAAd/vBgwdr1apV+uijj9S/\nf/+r7j8kJER33XWX3nnnHUnS0aNHtXnzZvf2ko7R4tx7773atGmTTp06Jen8sTB37txS3xPgW5wx\nW6Zp06ZauHCh/va3vyk1NVWBgYEKCQlRQkKCmjZtqhMnTpTYt6S/smNiYjRx4kR169ZNjRs31r33\n3ntFn4EDB2rixIl69NFH5e/vr7vvvtv9FZs777xT8+bN09ixYzV//nzNmDFD48ePl3Q+cF5++WUF\nBQVp7Nixio+PV7du3RQWFqZWrVoVW0/v3r2Vm5urESNGqKCgQA6HQ02bNtWrr74qh8Oh/v3768SJ\nE+5L1g0bNtTs2bPd/fPy8tSvXz9lZ2fr8ccfV0REhFq0aKG1a9fqkUceUbVq1RQREaH09PSrXs7+\n4osvdNddd0k6/2ZWs2ZNbdiwQXPnzlVCQoJ69uwpl8ulnj17qkePHkpKSirxeR45cqRmzJihXr16\nKSQkRGFhYe5Lo7/+9a+VkJBQai2lad68ueLj4/XUU0+psLBQ1atX16JFi8p0afHiei+v/cLvjz32\nmLZu3aru3burevXqaty4sYYOHXrFvq52fEoq8fho0aKF/Pz89Jvf/EZvvfVWifMZM2aM/vSnP+mt\nt96Sn5+fHnroIXXo0EFjxozRnDlztHDhQjmdToWHh19xiVySnnzyyRLblTT/qKgo7d+/X3379lXN\nmjXVunVr9we3PHk+hw4dqri4OPXs2VPVqlVT69at9cgjjxS7P0nq3r27XnjhBUVFRV1xVayksWbP\nnq1p06bpjTfeUIMGDXTLLbe4j7eyXIW50KZly5aaNGmShg8fLofDoXr16mnWrFnKyMgo8T0BvuUw\n5fnTCqhgMTExiomJ0UMPPVTRpVzhgw8+UHBwsKKiomSM0dixY3X//fe7L8kD3rRo0SI9/PDDatas\nmTIzM9WrVy+98sor7m8DoPIq0xnz119/rblz52rZsmU6cuSIJk+eLD8/P7Vo0cKjG08AnvLku6XX\nS4sWLfTMM8/oxRdfVEFBgSIiIsp0WRLwRNOmTfWHP/xBfn5+Kiws1MiRIwnlKuKqZ8yvvvqq3n33\nXQUHB2v58uUaPXq0hg8frrvvvlvx8fGKjIz0+LuiAADgUlf98FeTJk20cOFC9+979uxxf2T/17/+\n9SUfOAAAANfmqsHcpUuXSz6Gf/EJdnBwsDIyMnxTGQAAN6Byfyr74u9mZmVl6aabbrpqn+K+ngMA\nQFVX3G2Jr6bcwdymTRtt27ZNHTp00GeffVbi3Wa8UVxlkZiYyPwqMeZXeVX2ub2xYYX86waVuD0l\nOVkNGzW65nFysrL1QKN71eSWJte8L2+q7Ot3NZ6elJY7mOPi4jR9+nQVFBSoefPm6tq1q0cDAwCA\nK5UpmBs3bqzly5dLOv8R/bLeQxYAAJQPt+QEAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGAR\nghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYA\nwCIEMwAAFiGYAQCwCMEMAIBFCGYAACxCMAMAYBGCGQAAixDMAABYhGAGAMAiBDMAABZxVnQBsNvW\n7XuU+PUBncoo8qh/w3o36fbbWnm5KsB3du3brePpJ67LWPlFBaquoOsyFioPghmlSko9qyxHPaXl\nhXrUvyg1Xbff5uWiAB9KOZOq9JCc6zJW9WDPXleo2riUDQCARQhmAAAsQjADAGARghkAAIsQzAAA\nWIRgBgDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGY\nAQCwCMEMAIBFCGYAACxCMAMAYBGCGQAAixDMAABYhGAGAMAiBDMAABYhmAEAsAjBDACARZyedHK5\nXIqLi1NSUpKcTqcSEhLUrFkzb9cGAMANx6Mz5k8//VRFRUVavny5xowZoxdffNHbdQEAcEPyKJib\nNm2qwsJCGWOUkZGhgIAAb9cFAMANyaNL2cHBwTp27Ji6du2q9PR0LV682Nt1AQBwQ3IYY0x5O82e\nPVuBgYF6+umnlZqaqtjYWL333nuqVq1ase0TExOvuVBUjM+3fS9XYAOP+9coTNW9d7b0YkW4mg07\nv5AroNDn4xQVFalN3Za6uWFjn491PW39frvyw8r9tmi13Oxc3RbYXA3r/6yiS7nhhIeHl7uPR2fM\nNWvWlNN5vmtoaKhcLpeKioq8XlxlkZiYWGXnd+REnn44nqtGjRp51L92QLD1z01VW7+DmUdlwn56\naackJ6uhh+tXGleBSy2CW6hNy196fd9l5Yu1SytIV3pIjlf36SlvrV1OVrbaNWqrJrc08UJV3lPV\nXnuX8/Sk1KNgHjp0qKZOnarHH39cLpdLEyZMUFBQkEcFAACAn3gUzDVq1NC8efO8XQsAADc8bjAC\nAIBFCGYAACxCMAMAYBGCGQAAixDMAABYhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGAR\nghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYA\nwCIEMwAAFiGYAQCwCMEMAIBFCGYAACzirOgCULr09LPKzs72uH+NGjVUq1ZNL1YEXH95eXlKO512\nxeMnT51UckqyV8fKzs6WQhxe3SdQHgSz5T7Z/I1OZQd63D+sRp76dIv0YkXA9bf7+z36JuP7Kx4/\nnnVcKYfSvTqWM9SpQAV5dZ9AeRDMlnM6AxQccpPn/f28+6YFVJTgm0KueKxGZo1iHwcqM/7HDACA\nRQhmAAAsQjADAGARghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZ\nAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBFCGYAACxCMAMAYBGCGQAAixDMAABYhGAGAMAi\nBDMAABYhmAEAsAjBDACARZyedlyyZInWr1+vgoICDR48WH379vVmXQAA3JA8CuatW7dqx44dWr58\nubKzs/XPf/7T23UBAHBD8iiYv/jiC7Vs2VJjxoxRVlaWJk2a5O26AAC4IXkUzGfOnFFycrIWL16s\no0ePavTo0frwww+9XRu8IP1clj75YrvH/c9m5V7T+KfOZF7T+I1+Vlutbm12TTUANzp/f399vX+X\nDqUe8flYDjkUdff9Ph+nKvMomGvVqqXmzZvL6XSqWbNmCgwM1OnTp1WnTp0S+yQmJnpcZGXgq/kd\nOnxI+c4G17AHh06cS7/mOpKTkz0eP/Uaxt/7/X5lnj3tcf+yqkrH55EjR+TMC7rksRSP169kha5C\nfZtfqJyMbK/v+3J7D36vUzUzit3mi7nZxFvzS5Ok075/rjJTzirEUb3M7avSa89bPArm8PBwLVu2\nTMOGDVNqaqpyc3NVu3btq/apqhITE302v+TTBcooquWTfZe5huRkNWrUqELGrhsU6vNjx5frVxEO\nZh6VCfvppZ2SnKyGPlg/V4FLbYJbqU3LX3p931eo5tAPjitDxVdzs0VlnF+WI6TMr6eq9tq7nKd/\ndHgUzA888IC++uor9evXT8YYxcfHy+FweFQAAAD4icdfl5o4caI36wAAAOIGIwAAWIVgBgDAIgQz\nAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBF\nCGYAACxCMAMAYBGCGQAAixDMAABYhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGARghkA\nAIsQzAAAWIRgBgDAIs6KLqCq27DxK6WkZXvcP7fAocAQLxYEeImfv5++OrxTu5K/8/lYObm5qnHz\nTT4fB7ABwexj2bmFMoH1Pe4fGOjFYgAv8vPzU1Cj0OsyVg1Vuy7jADbgUjYAABYhmAEAsAjBDACA\nRQhmAAAsQjADAGARghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZ\nAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBFCGYAACxCMAMAYBGCGQAAixDMAABYhGAGAMAi\nBDMAABa5pmBOS0vTAw88oEOHDnmrHgAAbmgeB7PL5VJ8fLyCgoK8WQ8AADc0j4N5zpw5GjRokOrX\nr+/NegAAuKE5Pem0cuVKhYWFqWPHjlq0aJG3awKui8LCQmVlZSkzM1Pnzp3z6VhOp1M1atTw6RgA\nqgaHMcaUt9OQIUPkcDgkSXv37lWzZs308ssvKywsrNj2iYmJ11ZlJbZ5x/fK9W9Q0WVUWsHmpO5p\nf6tP9n342I/anbFf/k5/n+z/Yn7njB4Oj/b5OJK04duNcjbgX0yoGJkpZ9W17fU51iuD8PDwcvfx\n6Iz59ddfd/8cExOjGTNmlBjKF3hSXGWRmJhY4vxS011Kd9W8zhV5V3Jysho1alQhY9cNCvXZsRN8\nU4hyMxw6deqkGvp6fmkF1+01cDDzqEzYTy/tlORk38+vglTluUmVc35ZjpAyH+ulvXdWBZ6elF7z\n16UunDkDAIBr59EZ88X+9a9/eaMOAAAgbjACAIBVCGYAACxCMAMAYBGCGQAAixDMAABYhGAGAMAi\nBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGARghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgBALAIwQwA\ngEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBFCGYAACzirOgCgNKc\nOJWuz7fs8Mm+jyQd1tGAFJ3NyFBBkaPYNjeFBqtWzZuueayc/Fx9sX3zNe+nTGMV5CpIIddlLOBy\nxlFU5mN9/4HvlePI93isOqG11KbFLz3ubyuCGVbzC7lFR8/6Zt8pmUE6WyNAOaaGzuYGFNumqCjL\nK8FcvWGokpV2zfspi6CfEcqoOCE/q1XmY/1c7TwlV/P8dXH6RHqVDGYuZQMAYBGCGQAAixDMAABY\nhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGARghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgB\nALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBFCGYAACxC\nMAMAYBGnJ51cLpemTp2qpKQkFRQUaNSoUYqOjvZ2bQAA3HA8CubVq1erdu3aeuGFF3T27Fk99thj\nBDMAAF7gUTB369ZNXbt2lSQVFRXJ6fRoNwAA4DIeJWr16tUlSZmZmRo3bpyefvpprxYFAMCNymGM\nMZ50TElJ0VNPPaUhQ4aod+/epbZNTEz0qDhv2LPvkE6m53vcv0Y1o3vubO1x/807vleufwOP+8N3\nTp46qqy6aXIGBJTYJsiRrYb1al/HqgCUVc6JTAUp8LqM1Tj0Z/rFzU3L3S88PLzcfTw6Yz516pSG\nDx+uZ555RhEREWXq40lx3nA608gZFuJx/2qFaVetPTExscQ2qekupbtqejy+DZKTk9WoUaOKLsP7\n/PLkVyNfOXm5Cg0JLbZJaLUgNWzU8DoX5l0pyclqWBXXT1V7bhLzu6rr+NQ0zq+r8LvKl2OenpR6\n9HWpxYsX69y5c3rppZcUExOj2NhY5ed7flYKAADO8+iMedq0aZo2bZq3awEA4IbHDUYAALAIwQwA\ngEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBFCGYAACxCMAMAYBGC\nGQAAixDMAABYhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGARghkAAIsQzAAAWIRgBgDA\nIgQzAAAWIZgBALAIwQwAgEWcFV2A9YxRQUFBqU1cLleJbYqKinxRFa4TY4pUVFjocX+Hn58cDocX\nKwJQ1RHMV3E2L0D/WPF5qW1SUo5rx8GsYrc5A6qrRqgvKsP1cDqzUGnfHfG4f1hoNTVr0tiLFQGo\n6gjmq6geXFPVg2uW2iYzV7qpTqPrVBGup2qBNa6pv8NR+tUWALgc/2MGAMAiBDMAABYhmAEAsAjB\nDACARQiPgz27AAAGrklEQVRmAAAsQjADAGARghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgBALAIwQwA\ngEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBFCGYAACxCMAMAYBGC\nGQAAixDMAABYhGAGAMAiTk86GWP07LPPat++fapWrZpmzpypW265xdu1AQBww/HojPnjjz9Wfn6+\nli9frgkTJujPf/6zt+sCAOCG5FEwJyYmKjIyUpLUvn177d6926tFAQBwo/LoUnZmZqZCQ0N/2onT\nqaKiIvn5Wfgva+NS1pkknw6RdSZZWdWNT8eoSFV1fvmZaUo/kaSsrCwV3HSTT8Zw1AxSprOGT/Zd\nVpkpZ5Wp4AqtwVeq8twk5mcTv3r1r9tYHgVzSEiIsrKy3L+XJZQTExM9Geqa1QkNUJ3QWj4d464W\nvt1/Rauy82tRS9IvK7oKn2sf1rqiS/CZqjw3iflZpej65ZhHwXzXXXfpk08+UdeuXbVz5061bNmy\n1Pbh4eEeFQcAwI3GYYwp9zXKiz+VLUl//vOf1axZM68XBwDAjcajYAYAAL5h4ae1AAC4cRHMAABY\nhGAGAMAiHn0q+2rS0tLUt29fvfbaa5d8KGz9+vV66aWX5HQ61bdvX/Xv398Xw/tUSXNbunSpVqxY\noTp16kiSZsyYoaZNm1ZQlZ7p06ePQkJCJEk333yzZs2a5d5WFdautPlVhfVbsmSJ1q9fr4KCAg0e\nPFh9+/Z1b6sK61fa/Cr7+q1atUorV66Uw+FQXl6e9u7dq40bN7qP18q+flebX2VeP5fLpbi4OCUl\nJcnpdCohIeHac894WUFBgXnyySfNww8/bA4ePHjJ4126dDEZGRkmPz/f9O3b16SlpXl7eJ8qaW7G\nGDNx4kSzZ8+eCqrs2uXl5ZnevXsXu60qrF1p8zOm8q/fl19+aUaNGmWMMSYrK8vMnz/fva0qrF9p\n8zOm8q/fxZ577jnz1ltvuX+vCut3scvnZ0zlXr+PP/7Y/OEPfzDGGLNx40YzduxY9zZP187rl7Ln\nzJmjQYMGqX79S++S8sMPP6hJkyYKCQlRQECAwsPDtW3bNm8P71MlzU2S9uzZo8WLF2vw4MFasmRJ\nBVR3bfbu3avs7GwNHz5cw4YN09dff+3eVhXWrrT5SZV//b744gu1bNlSY8aM0ejRo9WpUyf3tqqw\nfqXNT6r863fBrl27dODAgUvOqqrC+l1Q3Pykyr1+TZs2VWFhoYwxysjIUEBAgHubp2vn1UvZK1eu\nVFhYmDp27KhFixZdsu3y23gGBwcrIyPDm8P7VGlzk6RHHnlEjz/+uEJCQvTkk0/q008/VVRUVAVU\n6pmgoCANHz5c/fv31+HDhzVixAitWbNGfn5+lX7tpNLnJ1X+9Ttz5oySk5O1ePFiHT16VKNHj9aH\nH34oqfK/9qTS5ydV/vW7YMmSJXrqqacueawqrN8Fxc1PqtzrFxwcrGPHjqlr165KT0/X4sWL3ds8\nXTuvnjGvXLlSGzduVExMjPbu3au4uDilpaVJOn8bz8zMTHfbrKws3eSj+xP7Qmlzk6ShQ4eqVq1a\ncjqdioqK0rfffluB1ZZf06ZN1atXL/fPtWrV0smTJyVV/rWTSp+fVPnXr1atWoqMjJTT6VSzZs0U\nGBio06dPS6oa61fa/KTKv36SlJGRocOHD+uee+655PGqsH5SyfOTKvf6LV26VJGRkVqzZo1Wr16t\nuLg45efnS/J87bwazK+//rqWLVumZcuWqXXr1pozZ47CwsIkSc2bN9ePP/6oc+fOKT8/X9u2bdMd\nd9zhzeF9qrS5ZWZmqkePHsrJyZExRlu2bNFtt91WwRWXzzvvvKPZs2dLklJTU5WVlaV69epJqvxr\nJ5U+v6qwfuHh4fr8888lnZ9fbm6uateuLalqrF9p86sK6ydJ27ZtU0RExBWPV4X1k0qeX2Vfv5o1\na7o/xBYaGiqXy6WioiJJnq+dz+78FRsbq+eee0579uxRTk6O+vfvrw0bNmjBggUyxqhfv34aNGiQ\nL4b2ueLmtnr1av3rX/9SYGCg7rvvvmIv19isoKBAU6ZMUXJysvz8/DRx4kQdO3asyqzd1eZX2ddP\nkubOnastW7bIGKPx48frzJkzVWb9pNLnVxXW7x//+IcCAgIUGxsrSXr//fer1PqVNr/KvH7Z2dma\nOnWqTp48KZfLpdjYWBljrmntuCUnAAAW4QYjAABYhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAs\nQjADAGARghkAAIv8P0m47uoUB/PqAAAAAElFTkSuQmCC\n", 352 | "text/plain": [ 353 | "" 354 | ] 355 | }, 356 | "metadata": {}, 357 | "output_type": "display_data" 358 | } 359 | ], 360 | "source": [ 361 | "plt.hist([sepal_length for iris_type, sepal_length \n", 362 | " in data if iris_type==0], **settings)\n", 363 | "plt.hist([sepal_length for iris_type, sepal_length \n", 364 | " in data if iris_type==1], **settings)\n", 365 | "_=plt.title('Simulated Sepal Length for Iris-Setosa and Iris-Virginica')" 366 | ] 367 | }, 368 | { 369 | "cell_type": "markdown", 370 | "metadata": { 371 | "slideshow": { 372 | "slide_type": "slide" 373 | } 374 | }, 375 | "source": [ 376 | "Instead of having fixed parameters\n", 377 | "\n", 378 | "```\n", 379 | "mixture_proportion = [0.50, 0.50]\n", 380 | "mean = [5.01, 6.59]\n", 381 | "std = [0.35, 0.64]\n", 382 | "```\n", 383 | "\n", 384 | "we could also sample our parameters from probability distributions!\n", 385 | "\n", 386 | "```\n", 387 | "mixture_proportion = dirichlet(alpha=[1, 1]).rvs()[0]\n", 388 | "mean = norm(loc=6, scale=1).rvs(size=2)\n", 389 | "std = norm(loc=.5, scale=.2).rvs(size=2)\n", 390 | "```" 391 | ] 392 | }, 393 | { 394 | "cell_type": "markdown", 395 | "metadata": { 396 | "slideshow": { 397 | "slide_type": "slide" 398 | } 399 | }, 400 | "source": [ 401 | "## What is a Dirichlet Distribution? \n", 402 | "\n", 403 | "Continuous distribution over unit vectors. \n", 404 | "\n", 405 | "Each sample is often regarded as a probability distribution; i.e. this is a distribution of _distributions_. \n", 406 | "\n", 407 | "In our example problem, the sample from the Dirichlet distribution determines the relative number of data points for each flower type. " 408 | ] 409 | }, 410 | { 411 | "cell_type": "code", 412 | "execution_count": 8, 413 | "metadata": { 414 | "collapsed": false 415 | }, 416 | "outputs": [ 417 | { 418 | "data": { 419 | "text/plain": [ 420 | "array([[ 0.29914558, 0.24235484, 0.45849958],\n", 421 | " [ 0.26578658, 0.28070049, 0.45351293],\n", 422 | " [ 0.04966029, 0.69022865, 0.26011105],\n", 423 | " [ 0.3641554 , 0.47919556, 0.15664905],\n", 424 | " [ 0.00103759, 0.02029473, 0.97866768],\n", 425 | " [ 0.33356232, 0.05691374, 0.60952394],\n", 426 | " [ 0.0042538 , 0.35811682, 0.63762938],\n", 427 | " [ 0.02272919, 0.4869255 , 0.4903453 ],\n", 428 | " [ 0.0388615 , 0.30648445, 0.65465404],\n", 429 | " [ 0.15307055, 0.03631371, 0.81061574]])" 430 | ] 431 | }, 432 | "execution_count": 8, 433 | "metadata": {}, 434 | "output_type": "execute_result" 435 | } 436 | ], 437 | "source": [ 438 | "dirichlet(alpha=[1,1, 2]).rvs(10)" 439 | ] 440 | }, 441 | { 442 | "cell_type": "markdown", 443 | "metadata": { 444 | "slideshow": { 445 | "slide_type": "slide" 446 | } 447 | }, 448 | "source": [ 449 | " ➡ Forward Sampling \n", 450 | "Bayesian style!" 451 | ] 452 | }, 453 | { 454 | "cell_type": "code", 455 | "execution_count": 9, 456 | "metadata": { 457 | "collapsed": false, 458 | "slideshow": { 459 | "slide_type": "-" 460 | } 461 | }, 462 | "outputs": [], 463 | "source": [ 464 | "# Distribution on parameters\n", 465 | "mixture_proportion = dirichlet(alpha=[1, 1]).rvs()[0]\n", 466 | "mean = norm(loc=6, scale=1).rvs(size=2)\n", 467 | "std = norm(loc=.5, scale=.2).rvs(size=2)\n", 468 | "\n", 469 | "data = []\n", 470 | "for z in range(1, 101):\n", 471 | " iris_type = random.choice(2, p=mixture_proportion)\n", 472 | " sepal_length = norm(loc=mean[iris_type], \n", 473 | " scale=std[iris_type]\n", 474 | " ).rvs()\n", 475 | " data.append((iris_type, sepal_length))" 476 | ] 477 | }, 478 | { 479 | "cell_type": "code", 480 | "execution_count": 10, 481 | "metadata": { 482 | "collapsed": false, 483 | "slideshow": { 484 | "slide_type": "slide" 485 | } 486 | }, 487 | "outputs": [ 488 | { 489 | "data": { 490 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAFeCAYAAABU/2zqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VOW9x/HvJCEBkwBhfRGqQCnLVRE1oFiMkQAWEVAE\nyxrg1gWlUK5gCatREBBLKxajgFS9Qq9xg4LWKwiIVhbBsAgoFAQEMpBASMhGlsk89w8vI0iWyTAh\nD+Tz/ovJnPM7v/PMM+c752RycBhjjAAAQJUKqOoGAAAAgQwAgBUIZAAALEAgAwBgAQIZAAALEMgA\nAFiAQLbQjh07NGzYMN1///3q3bu3HnvsMR04cECStHv3bo0dO9Zv24qNjdWePXvKXGbXrl1KSEio\ncO0ZM2bo5ZdfLvG5ZcuW6cEHH9QDDzyg3r17a9q0acrJyanwNrzx8ssv67nnnrvo58uXL9fjjz9e\nKdv8uffee09vv/12mf38XE5OjgYOHKjevXvr008/9XnbcXFxWr16dYnPjRw5Ut9//32F6pU1P8ty\n7Ngx/eEPf6jQtqrKqlWrFBcXV+JzZb1n+vbtW6F53L17d3388ccX/XzGjBmaOXOm1q1bp5kzZ3pd\nT5LS0tI0aNCgcpfz5bVH5Qqq6gZwocLCQj3++ON688031bZtW0nSypUr9dhjj2nt2rW68cYb9dJL\nL13Wnvbv36/U1FS/1du1a5deeeUVLV++XOHh4TLG6JlnntEzzzyjuXPn+m07Ntm2bZtat25doXW+\n++47ZWRkaNWqVZXUlbRw4cIKLV/e/HQ4HKWum5KSokOHDl1Sv5dTWftSmuXLl1do+cGDB+v9999X\nz549PT8rKCjQRx99pHfffVfNmjVTbGxshWo2atTI8+GvLBV97VH5CGTL5OfnKycn54JP2X369FF4\neLiKi4u1bds2zZgxQx9++KEmTZqkkJAQ7dq1S+np6erRo4fq1aundevWKT09Xc8995xuv/12TZo0\nSa1bt9Z//ud/StJFjyXJGKOZM2dq165dys3NlTFGzz33nJo0aaL58+crJydHkydP1qxZs7Ru3Tot\nWLBALpdLNWvW1IQJE3TzzTcrJydHU6dO1b59+9SwYUMFBgYqKirqon08efKkjDHKy8tTeHi4HA6H\nxo4de8FZ1oIFC7R69WoZY9S0aVMlJCSoYcOGiouL069+9Svt3r1bmZmZ6tOnj8aMGeNZZ+3atSos\nLNTZs2c1YcIEdevWzafXITU1VTNmzNDx48flcrl033336bHHHlNKSopGjBihmJgY7dy5U1lZWRo7\ndqx69uyp/Px8JSQkaMeOHapTp45atmwpSeratavWrVunjRs3KiQkRJL0/fffa9iwYTp58qQaNGig\nF198UQ0aNPBs/9ChQ5oyZYrS0tLUt29fvfPOO/riiy+UmJgot9utsLAwxcfH66abbtLLL7+s7du3\n6+TJk2rbtq1eeOGFUvcrNjZW7du317///W89+eSTmjVrlubPn68WLVpo0qRJOnLkiBwOh2688UZN\nnz79ovXLm59BQUH67LPP9Oqrr3rmR3x8vNq1a6dp06YpLS1NjzzyiBYvXqw1a9aUuD8HDx7UlClT\nVFhYKGOM+vfvr8GDBys9PV1PP/200tPTderUKUVGRmrevHmqV6/eBT2WtVxsbKwefPBBbdq0SceP\nH9e9996rP/7xj5Kkl156SR999JEiIiJ03XXXeTVP2rVrp65du2rfvn3605/+pP79+2vz5s1yuVyK\nj49XRkaGJCkmJqbEK1sPPvig/vrXv+r48eNq0qSJJOnjjz/WjTfeqGbNmmn58uVatWqVFixYoLi4\nONWtW1eHDh3SoEGDFB0drcmTJ+vMmTNq2LChjDG6//771bFjR/Xq1Uvbt2/Xyy+/rJSUFKWlpcnp\ndKpevXqaN2+eGjZsqNjYWM2fP1833HCD3n//fb355psKDAxURESE5syZo0aNGpV4TLjlllu8Ghv4\nwMA6b7zxhmnfvr3p1q2b+eMf/2jef/99c/bsWWOMMV999ZXp1auXMcaYiRMnmgEDBpji4mJz8uRJ\n06ZNG7N06VJjjDH//d//bX73u995lnv99dc99c9/3KVLF7N7926zfft2M3bsWM8yCxcuNI8//rgx\nxphly5aZkSNHGmOMOXz4sOnVq5fJzMw0xhizf/9+07lzZ3P27Fkzc+ZMM3HiRGOMMenp6SYmJsbM\nnz//ov0rKioy48ePN9dff73p27evmT59ulm/fr3n+eXLl5snn3zSFBcXG2OMeeedd8yjjz5qjDFm\n6NChZuTIkaa4uNhkZWWZHj16mPXr15uUlBQzfPhwU1BQYIwx5p///Kfp3bu3McaY+fPnmxkzZlzU\nx/n79XPDhg0zn332mTHGmIKCAjNs2DDzv//7v+bYsWOmTZs2nn5XrVplunTpYowxZu7cuWb8+PHG\nGGOys7NNnz59PONx/pjPnz/fdOvWzWRkZBhjjBk1apR55ZVXLurh/Nf6wIEDpnPnzubYsWPGGGM2\nbdpkOnfubHJycsz8+fPNvffea9xud4n7MnToULNq1SpjzI+v9/nbio2NNbt37zb/+Mc/zCOPPGKM\nMaa4uNhMmzbNHDlypMR6Zc3PsuaHt/szefJks2jRImOMMSdPnjTjxo0zxvw4p1977TVPH48++qh5\n4403LuqvrOW6dOli5syZY4wx5sSJE+amm24yx44dM59++qnp1auXycvLM8XFxWbkyJEmLi6uxP0/\n954xxpg2bdqYlStXep5r27atycjIMImJiSYhIcEYY0xeXp4ZN26cyc7OLrHepEmTLnifDBw40Kxb\nt84Yc+EcHTp0qJkyZYpnuQEDBpikpCTPeN58881m+fLl5tixY+aWW24xxvw417p3725yc3ONMcY8\n/vjjnm2d24/vvvvOdOrUyZw4ccIzfgkJCWUeE1A5OEO20IgRI/Tb3/5WW7du1datW/Xaa69p8eLF\neu+99y5atkuXLgoICFCDBg1Uq1YtRUdHS5Kuu+46nTlzxutt3nzzzRo7dqzefvttHTlyRFu2bFFY\nWNhFy23YsEGnTp3SiBEjZP7/rqtBQUE6fPiwNm3apClTpkiS6tWrV+rZaVBQkObOnav4+Hh99dVX\n2rJliyZOnKg77rhDf/nLX7R+/Xrt2rVLDz74oCTJ7XaroKDAs/6AAQMUEBCg8PBw9ejRQ//6178U\nExOj559/XitWrNCRI0e0Y8cO5eXleb3/5zt79qy2bt2qrKwszZs3z/Oz7777Tu3atVONGjUUExMj\nSbr++us94/zFF19o0qRJkqSwsDD17dtX+/btK3Ebv/71r1W3bl1JUtu2bZWenl5mT1999ZXuuOMO\nNW3aVJLUqVMnNWjQwPO7zPbt23t9ibVDhw6ef597DaOiojRv3jzFxcWpc+fOGj58uK699toS1y9r\nfpY2P3744Qev9+eee+7RhAkT9M033+iOO+7wzKlhw4bp66+/1ptvvqnDhw/rwIEDat++/UX9lbdc\n165dJUmNGzdW/fr1debMGW3evFndu3dXrVq1JEn9+vXTkiVLvBrP868Cndvn6OhojRw5Uk6nU7/+\n9a81fvz4Et9PkjRo0CCNHTtWo0eP1v79+5WWlqa77767xGXPvXZZWVn65ptv9Pe//12S1LJlS3Xq\n1KnEdW677TZdc801kn6cr5mZmRc8v3nzZkVHR6tx48aSfhy/c7w5JsB/CGTLbNu2Tdu3b9fDDz+s\nmJgYxcTEaNy4cerVq5c2btzoOYifExwcfMHjoKCSX1Jz3i3LCwsLL3p+/fr1mjVrln73u9+pW7du\n+uUvf6kPP/zwouXcbrcnOM85ceKEGjVqJIfDccF2Suvlgw8+UEREhGJjY9WrVy/16tVLTzzxhLp2\n7aqnn35abrdbjz76qAYOHChJKioqUlZWlmf9wMDAC/oJDAzUt99+q1GjRmnEiBG688471bFjRz37\n7LMlbr88xcXFkqR33nnHM74ZGRmqWbOmTp8+rRo1aniWPX+fAwMDL9j/gIDSvzP58xrlMSXccr64\nuFgul0uSFBoaWm6Nc84dnM/3i1/8QqtXr9aWLVu0efNmDR8+XNOmTdO+ffu0bt06ORwOxcbGqnPn\nzmXOz9LmR+PGjbV161av9icmJkarV6/Whg0btGnTJiUmJiopKUlJSUnavXu3+vXrp06dOsnlcpVY\n509/+lOZy9WsWfOC5c89d/4y588xX8azXbt2Wrt2rTZu3KjNmzerf//+SkxM1AcffKDdu3fL4XBo\n4MCBGjBggNq1a6cGDRpo48aNWr9+vQYOHFjqnDi3rYCAgIveb6X1fP7+llQ3MDDwgp8XFBQoJSVF\nR48e1cyZM8s9JsB/+Ja1ZerVq6cFCxZo27Ztnp+lpqYqPz+/wl8KOr/m7t27JUmnT59WcnLyRcts\n3LhRsbGxGjhwoG688UatXbtWbrdb0o9v2HMH/k6dOmnDhg06ePCgJOnzzz/X/fffr8LCQkVHR+v9\n99+XMUZnzpzR2rVrS+wnICBAf/7zny/4otihQ4fUtGlT1alTR3feeafee+89z+8p582bpwkTJniW\nXblypWcbn3zyibp06aKtW7eqXbt2GjFihDp27Kg1a9Z4+q+osLAwtW/fXn/7298k/Xg2MmjQIM/+\nlBQCknT33Xdr2bJlMsbo7Nmz+uijjzwHusDAQBUVFfnUj/TTuB87dkyStGnTJqWmpuqmm27yueb5\n3n77bU2cOFGdO3fW+PHjFR0drf379+sPf/iD/vGPf2j58uUaM2ZMufOztPlRUFBQ4jwqaX/Gjx+v\nf/7zn+rZs6cSEhIUFhamEydOaOPGjRo+fLj69OmjiIgIzweAn9uwYYNXy50vOjpan3zyibKzs+V2\nu7VixYpLGs8///nPSkxMVNeuXTVlyhT96le/0g8//KDnnnvOM54DBgzwLD948GAtX75cn376qR56\n6KFy64eFhenWW2/VBx98IEk6evSoNm3a5Hm+tDlakttvv10bN27UqVOnJP04F+bOnVvmMQGVgzNk\nyzRv3lyJiYn6y1/+otTUVIWEhCgsLEwzZsxQ8+bNlZaWVuq6pX2qjouL01NPPaV7771XTZs21e23\n337ROgMHDtRTTz2l+++/X4GBgerQoYPnT2VuueUWzZs3T2PGjNH8+fM1ffp0jRs3TtKPQfPqq6+q\nZs2aGjNmjBISEnTvvfeqfv36atOmTYn99O3bV/n5+Xr00UdVVFQkh8Oh5s2ba/HixXI4HHrooYeU\nlpbmuTTdpEkTPf/88571CwoK1L9/f+Xl5WnIkCHq1KmTWrVqpdWrV+u+++5TcHCwOnXqpMzMzHIv\nW3/55Ze69dZbJf14EKtTp47Wr1+vuXPnasaMGerdu7dcLpd69+6tXr16KSUlpdRxHjlypKZPn64+\nffooLCxM9evX91wCveuuuzRjxowyeylLy5YtlZCQoNGjR6u4uFi1atXSggULvLqEeH6/P+/93OMH\nHnhAW7ZsUc+ePVWrVi01bdpUw4cPv6hWefNTUqnzo1WrVgoICNBvf/tbvfvuu6Xuz6hRozR16lS9\n++67CggI0D333KOOHTtq1KhRmjNnjhITExUUFKSoqKiLLoVL0u9///tSlytt/2NiYrR//37169dP\nderUUdu2bT1fyPJlPIcPH674+Hj17t1bwcHBatu2re67774S60lSz5499cILLygmJuaiq2Clbev5\n55/XlClT9Pbbb6tx48a69tprPfPNm6su55Zp3bq1JkyYoIcfflgOh0MNGzbUrFmzlJ2dXeoxAZXD\nYSryUQqoYnFxcYqLi9M999xT1a1c5OOPP1ZoaKhiYmJkjNGYMWN05513ei69A/60YMEC/eY3v1GL\nFi2Uk5OjPn366LXXXvN8ux9XHq8uWe/cufOiP5L/8MMPOdDgsvPlb0Mvl1atWmnBggV64IEH1KtX\nLzVu3Niry4+AL5o3b67/+q//Ut++fTV48GCNHDmSML7ClXuGvHjxYq1YsUKhoaFKSkqSJH377bd6\n4YUXlJ+f7/kZAADwXblnyM2aNVNiYqLncUZGhubNm+f5UwQAAHDpyg3k7t27e75O73a7NXXqVE2c\nOFG1atWq0Df5AABA6Sr0Les9e/boyJEjeuaZZ1RQUKDvv/9es2fP9twMoTQl/ZkNAABXu5JuH1wa\nrwPZGKN27dp5/jA8JSVF48ePLzeMfWmqOktOTr5qx2r7N3u0J8VR5g0zvOV0OhUZGSlJCnNkqs89\nJd+lCFf3nPInxsk7jJP3Knoy6vWR0eZvtwIAcKXzKpCbNm160bepS/oZAADwDbfOBADAAgQyAAAW\nIJABALAAgQwAgAUIZAAALEAgAwBgAQIZAAALEMgAAFiAQAYAwAIEMgAAFiCQAQCwAIEMAIAFCGQA\nACxAIAMAYAECGQAACxDIAABYgEAGAMACQVXdAOy0/ZtvVeRy+bXm0WNOqcZ1fq0JAFcLAhklSv7u\nuK6p29S/RUOac0kGAErB8REAAAsQyAAAWIBABgDAAgQyAAAWIJABALAAgQwAgAUIZAAALEAgAwBg\nAQIZAAALEMgAAFiAQAYAwAIEMgAAFiCQAQCwAIEMAIAFCGQAACxAIAMAYAECGQAAC3gVyDt37lRc\nXJwk6bvvvtOQIUM0bNgwPfLIIzp9+nSlNggAQHVQbiAvXrxYU6dOVVFRkSRp1qxZevrpp/XWW2+p\ne/fuWrRoUaU3CQDA1a7cQG7WrJkSExM9j1988UW1adNGkuRyuRQSElJ53QEAUE0ElbdA9+7dlZKS\n4nncoEEDSdK2bdv0P//zP1q6dGnldQd4ITMrR19u3u73um1bNVeD+hF+rwsAJSk3kEvy8ccfa+HC\nhVq0aJEiIrw7YCUnJ/uyqWrJhrFyOp26Js9R1W2Uyel0/v+/ApT2XYbf6x86dEDXt/6l3+tWBRvm\n1JWAcfIO41Q5KhzIK1as0LvvvqslS5aodu3aXq8XFRVV0U1VS8nJyVaM1fYDmbqmbmRVt1Eqp9Op\nyMjK7a9Vw4aKurVdpW7jcrBlTtmOcfIO4+S9in5wqVAgu91uzZo1S5GRkfr9738vh8Oh2267TaNH\nj67QRgEAwIW8CuSmTZsqKSlJkvTVV19VakMAAFRH3BgEAAALEMgAAFiAQAYAwAIEMgAAFiCQAQCw\nAIEMAIAFCGQAACxAIAMAYAECGQAACxDIAABYgEAGAMACBDIAABYgkAEAsACBDACABQhkAAAsQCAD\nAGABAhkAAAsQyAAAWIBABgDAAgQyAAAWIJABALAAgQwAgAUIZAAALEAgAwBgAQIZAAALEMgAAFiA\nQAYAwAIEMgAAFiCQAQCwAIEMAIAFCGQAACxAIAMAYAECGQAACxDIAABYgEAGAMACXgXyzp07FRcX\nJ0k6cuSIBg8erKFDh+rZZ5+t1OYAAKguyg3kxYsXa+rUqSoqKpIkzZ49W+PGjdPSpUvldru1Zs2a\nSm8SAICrXbmB3KxZMyUmJnoe79mzRx06dJAk3XXXXdq0aVPldQcAQDVRbiB3795dgYGBnsfGGM+/\nQ0NDlZ2dXTmdAQBQjQRVdIWAgJ8yPDc3V7Vr1/ZrQwBQGQ4c/l6HUn8o9fmDBw/qtOtMhWqGBAbr\nrg53XmprgCQfAvn666/X1q1b1bFjR33xxRfq1KmTV+slJydXuLnqyoaxcjqduibPUdVtlMnpdFZq\n/eLT6QowhZW6jcvFhjlV1Xbs36WcemW8nvWlb88cqFDNghN5CnXUusTOrjzMp8pR4UCOj4/XtGnT\nVFRUpJYtW6pHjx5erRcVFVXh5qqj5ORkK8Zq+4FMXVM3sqrbKJXT6VRkZOX216phQ0Xd2q5St3E5\n2DKnqlquzupESGapzx93OtWkgnPKFZRf7caW+eS9in5w8SqQmzZtqqSkJElS8+bNtWTJkop3BgAA\nSsWNQQAAsACBDACABQhkAAAsQCADAGABAhkAAAsQyAAAWIBABgDAAgQyAAAWIJABALAAgQwAgAUI\nZAAALEAgAwBgAQIZAAALEMgAAFiAQAYAwAIEMgAAFiCQAQCwAIEMAIAFCGQAACxAIAMAYAECGQAA\nCxDIAABYgEAGAMACBDIAABYgkAEAsACBDACABQhkAAAsQCADAGABAhkAAAsQyAAAWIBABgDAAgQy\nAAAWIJABALAAgQwAgAUIZAAALEAgAwBggSBfVnK5XIqPj1dKSoqCgoI0Y8YMtWjRwt+9AQBQbfh0\nhvz555/L7XYrKSlJo0aN0osvvujvvgAAqFZ8CuTmzZuruLhYxhhlZ2erRo0a/u4LAIBqxadL1qGh\noTp27Jh69OihzMxMLVy40N99AQBQrTiMMaaiKz3//PMKCQnRk08+qdTUVA0bNkwffvihgoODS1w+\nOTn5khvF5fXPL77VNXWbVnUbVapxzXRd3/qXVd0G/GTH/l3KqVfo15oFJ/LU9YZov9bE1SUqKsrr\nZX06Q65Tp46Cgn5cNTw8XC6XS263229NVWfJyclWjNX2A5m6pm5kVbdRKqfTqcjIyu2vVcOGirq1\nXaVu43KwZU5VtVyd1YmQzFKfP+50qkkF55QrKL/ajS3zyXsVPRn1KZCHDx+uyZMna8iQIXK5XBo/\nfrxq1qzpSykAACAfA/maa67RvHnz/N0LAADVFjcGAQDAAgQyAAAWIJABALAAgQwAgAUIZAAALEAg\nAwBgAQIZAAALEMgAAFiAQAYAwAIEMgAAFiCQAQCwAIEMAIAFCGQAACxAIAMAYAECGQAACxDIAABY\ngEAGAMACBDIAABYIquoGAODntu3Zof0nD/q1Zn5RgWo1CfdrTcCfCGQA1sktyJOjQbBfa9aSf+sB\n/sYlawAALEAgAwBgAQIZAAALEMgAAFiAQAYAwAIEMgAAFiCQAQCwAIEMAIAFCGQAACxAIAMAYAEC\nGQAACxDIAABYgEAGAMACBDIAABYgkAEAsACBDACABYJ8XXHRokVat26dioqKNHjwYPXr18+ffQEA\nUK34FMhbtmzR9u3blZSUpLy8PL3++uv+7gsAgGrFp0D+8ssv1bp1a40aNUq5ubmaMGGCv/sCAKBa\n8SmQMzIy5HQ6tXDhQh09elRPPPGEPvnkE3/3BgBAteFTINetW1ctW7ZUUFCQWrRooZCQEJ0+fVr1\n6tUrdZ3k5GSfm6xuKjJWW3fsVXqO8X8TNcIVkuf0f10/cjort7/i0+kKMIWVuo3L5dyc+vbgXqXk\npfq1tskv1j0dYv1a88CB/cqKKPBrTW8cr+CcKjiRVy2PbdVxny8HnwI5KipKS5Ys0YgRI5Samqr8\n/HxFRESUuw7Kl5ycXKGxSssqVu3C2pXYkZ2cTqciIyMrdRutGjZU1K3tKnUbl8P5c+qso0AhwXX8\nWr8wNc/v7+88R4GOB5/2a83yHHc61aSCc8oVlF/tjm0VPUZVZxX94OJTIN999936+uuv1b9/fxlj\nlJCQIIfD4UspAACgS/izp6eeesqffQAAUK1xYxAAACxAIAMAYAECGQAACxDIAABYgEAGAMACBDIA\nABYgkAEAsACBDACABQhkAAAsQCADAGABAhkAAAsQyAAAWIBABgDAAgQyAAAWIJABALAAgQwAgAUI\nZAAALEAgAwBggaCqbqC6OHDwsD7duFdBgTXKXC7luFPb/p3pfeGgYIXWvsTmAFjjX19v0L9PH/Jr\nzRqFgRrWa5Bfa8L/COTLpNhVrFp1mig4OKTM5cLyHQqtF3mZugJgG+MwCo+s69eaxSfz/VoPlYNL\n1gAAWIBABgDAAgQyAAAWIJABALAAgQwAgAUIZAAALEAgAwBgAQIZAAALEMgAAFiAQAYAwAIEMgAA\nFiCQAQCwAIEMAIAFCGQAACxAIAMAYAECGQAAC1xSIKenp+vuu+/WoUOH/NUPAADVks+B7HK5lJCQ\noJo1a/qzHwAAqiWfA3nOnDkaNGiQGjVq5M9+AAColoJ8WWnZsmWqX7++OnfurAULFvi7J8AKxcXF\nKioq8mtNh8OhoCDv3nZut1vFxcWXvE2Xy+XZD3/Uw0+MMX6fI26326/1cOVwGGNMRVcaOnSoHA6H\nJGnv3r1q0aKFXn31VdWvX7/E5ZOTky+ty6vAD0dStO9ksGrUCK7qVuCl7Mw0yfj34FiQm66Hev7a\nq2V3/nuXUlxpft1+QGCAwhrU9mvN/BM56nZDjF9rfnNgt7IiCvxaszIU5OWrIDvfrzUdQQEKr+/f\n16jgRJ663hDt15rwTlRUlNfL+nSGvHTpUs+/4+LiNH369FLD2JemrkZh4XWVtS9HwcEhZS7ndDoV\nGRl5mbq6cl2WcaqE+jmnU7x+L+QHFCmsRoNL3uZxp1NNKnGsCgPz/P7+znMU6Hjwab/WLE9lj1NV\nKq6R77fXKDk5udofz71V0ZPRS/6zp3NnygAAwHc+nSGf76233vJHHwAAVGvcGAQAAAsQyAAAWIBA\nBgDAAgQyAAAWIJABALAAgQwAgAUIZAAALEAgAwBgAQIZAAALEMgAAFiAQAYAwAIEMgAAFiCQAQCw\nAIEMAIAFCGQAACxAIAMAYAECGQAACxDIAABYgEAGAMACBDIAABYgkAEAsACBDACABQhkAAAsQCAD\nAGABAhkAAAsQyAAAWIBABgDAAgQyAAAWIJABALAAgQwAgAUIZAAALEAgAwBgAQIZAAALEMgAAFiA\nQAYAwAIEMgAAFgjyZSWXy6XJkycrJSVFRUVFevzxxxUbG+vv3gAAqDZ8CuSVK1cqIiJCL7zwgs6c\nOaMHHniAQAYA4BL4FMj33nuvevToIUlyu90KCvKpDAAA+H8+JWmtWrUkSTk5ORo7dqyefPJJvzbl\nrTNZ2TpzJsvvdevXi1Bo6DV+rwtcjYqLi3X02FG/1szKypIa+LVkteZyufz2GqWeTNXRY0cVEhKi\nRg0b+aUmfuQwxhhfVjx+/LhGjx6toUOHqm/fvmUum5yc7FNz5dm8fZ8yCsP9XrdJWJ5uvuFXfq35\nw5EU7TsZrBo1gv1aF1eWnIwU9Y653qtld32/R2fq5ldyR5eu2FWsooJCv9Z0BAYopGaIX2tWZ64i\nl1yFRX6g8+TwAAAH20lEQVStWXy6UL+5mV9VlicqKsrrZX06Qz516pQefvhhPf300+rUqZPfm/LW\n6Vyjk2fD/F73ujpnFRV1i19rhoXXVda+HAUHl32QcTqdioyM9Ou2r0ZX6jjl1DRevxfyA4qUUuPU\nJW/zuNOpJlfgWF1ujJN3zo2T+1RBpRzXryYVPRn16c+eFi5cqKysLL3yyiuKi4vTsGHDVFjo30/I\nAABUJz6dIU+ZMkVTpkzxdy8AAFRb3BgEAAALEMgAAFiAQAYAwAIEMgAAFiCQAQCwAIEMAIAFCGQA\nACxAIAMAYAECGQAACxDIAABYgEAGAMACBDIAABYgkAEAsACBDACABQhkAAAsQCADAGABAhkAAAsE\nVXUDNkpNO6Vv9uz1a02n84SkCL/WhG9yc7OUkZnm15o1gmqoceNm5S7nNsbruXXw8BFl1sm71NZ0\nOuOMAoKCfVq3XkQd1ahR45J7AFA+ArkEZ4MitfMHl5+rNlBwMMNtgxOnDiu/frZfa55NzfMqkMPr\nNfV6bh1MK1aR3JfamrILglV8puJ13G63pDNq3KjBJfcAoHwkRAkCAgOrugVUMn+/xg6Hw+vlAgO9\ne9sFBgTK5bj03yo5AgLk8KFOAL/QAi4r3nIAAFiAQAYAwAIEMgAAFiCQAQCwAIEMAIAFCGQAACxA\nIAMAYAECGQAACxDIAABYgEAGAMACBDIAABYgkAEAsACBDACABQhkAAAsQCADAGABAhkAAAt49z+l\n/4wxRs8884z27dun4OBgzZw5U9dee62/ewMAoNrw6Qx5zZo1KiwsVFJSksaPH6/Zs2f7uy8AAKoV\nnwI5OTlZ0dHRkqT27dtr9+7dfm0KAIDqxqdL1jk5OQoPD/+pSFCQ3G63AgIu76+kHe5i5WakXNZt\nVrbcDKdya5mqbsN6lzJOBdnpysg55dd+zjjTtfFgpl9rZuacUp1W9S+5TnZWlopq167wesa4FR4Z\noRxXjUvu4UqQc/yMchRa1W1Y79w4BbsCq7qVq45PgRwWFqbc3FzPY2/CODk52ZdNlSkiPEgR4XX9\nXrcq3drq6tqfynIp43Rrq9v82AmuFu3rt63qFq4I549TZRzXqzOfAvnWW2/VZ599ph49emjHjh1q\n3bp1mctHRUX51BwAANWFwxhT4et+53/LWpJmz56tFi1a+L05AACqC58CGQAA+Bc3BgEAwAIEMgAA\nFiCQAQCwgE/fsq6o9PR09evXT2+88QZf/irFgw8+qLCwMEnSL37xC82aNauKO7LXokWLtG7dOhUV\nFWnw4MHq169fVbdkneXLl2vZsmVyOBwqKCjQ3r17tWHDBs8cw09cLpfi4+OVkpKioKAgzZgxg+NU\nCQoLCzVp0iQdO3ZMYWFhSkhI0HXXXVfVbVll586dmjt3rpYsWaIjR45o4sSJCggIUKtWrZSQkFDu\n+pUeyC6XSwkJCapZs2Zlb+qKVVhYKEl66623qrgT+23ZskXbt29XUlKS8vLy9Prrr1d1S1bq27ev\n+vbtK0maPn26+vfvTxiX4vPPP5fb7VZSUpI2btyoF198UX/961+rui3rvPfeewoNDdU777yjQ4cO\n6dlnn9Xf/va3qm7LGosXL9aKFSsUGvrjzWVmz56tcePGqUOHDkpISNCaNWvUrVu3MmtU+iXrOXPm\naNCgQWrUqFFlb+qKtXfvXuXl5enhhx/WiBEjtHPnzqpuyVpffvmlWrdurVGjRumJJ55Qly5dqrol\nq+3atUsHDhzQQw89VNWtWKt58+YqLi6WMUbZ2dmqUaN63Jmsog4cOKC77rpLktSiRQsdPHiwijuy\nS7NmzZSYmOh5vGfPHnXo0EGSdNddd2nTpk3l1qjUM+Rly5apfv366ty5sxYsWFCZm7qi1axZUw8/\n/LAeeughHT58WI8++qhWrVp12W9FeiXIyMiQ0+nUwoULdfToUT3xxBP65JNPqrotay1atEijR4+u\n6jasFhoaqmPHjqlHjx7KzMzUwoULq7olK/3Hf/yH1q9fr27dumnHjh1KS0uTMUYOh6OqW7NC9+7d\nlZLy062cz/+L4tDQUGVnZ5dbo1KP+MuWLdOGDRsUFxenvXv3Kj4+Xunp6ZW5yStS8+bN1adPH8+/\n69atq5MnT1ZxV3aqW7euoqOjFRQUpBYtWigkJESnT5+u6raslJ2drcOHD+u227hVaFnefPNNRUdH\na9WqVVq5cqXi4+M9v0bCT/r166fQ0FANGTJEa9eu1Q033EAYl+H8E6rc3FzV9uJ+8pUayEuXLtWS\nJUu0ZMkStW3bVnPmzFH9+pd+s/yrzQcffKDnn39ekpSamqrc3Fw1bNiwiruyU1RUlP71r39J+nGs\n8vPzFRERUcVd2Wnr1q3q1KlTVbdhvTp16nh+vx4eHi6XyyW3213FXdln165duuOOO/T3v/9dv/nN\nb3TttddWdUtWu/7667V161ZJ0hdffOHVLaQvy7esJfFJqgz9+/fXpEmTNHjwYAUEBGjWrFlcri7F\n3Xffra+//lr9+/eXMUYJCQnMrVIcOnSIg6YXhg8frsmTJ2vIkCFyuVwaP348X0ItQbNmzfTSSy9p\nwYIFql27tmbOnFnVLVktPj5e06ZNU1FRkVq2bKkePXqUuw63zgQAwAKchgEAYAECGQAACxDIAABY\ngEAGAMACBDIAABYgkAEAsACBDACABQhkAAAs8H8RYG19oPJIKQAAAABJRU5ErkJggg==\n", 491 | "text/plain": [ 492 | "" 493 | ] 494 | }, 495 | "metadata": {}, 496 | "output_type": "display_data" 497 | } 498 | ], 499 | "source": [ 500 | "plt.hist([sepal_length for iris_type, sepal_length \n", 501 | " in data if iris_type==0], **settings)\n", 502 | "plt.hist([sepal_length for iris_type, sepal_length \n", 503 | " in data if iris_type==1], **settings)\n", 504 | "_=plt.title('Simulated Sepal Length for Iris-Setosa and Iris-Virginica')" 505 | ] 506 | }, 507 | { 508 | "cell_type": "markdown", 509 | "metadata": { 510 | "slideshow": { 511 | "slide_type": "slide" 512 | } 513 | }, 514 | "source": [ 515 | " ➡ Forward Sampling\n", 516 | " \n", 517 | "If we run sample again, our data looks completely different. These are samples from the distribution over this model." 518 | ] 519 | }, 520 | { 521 | "cell_type": "code", 522 | "execution_count": 11, 523 | "metadata": { 524 | "collapsed": false, 525 | "slideshow": { 526 | "slide_type": "-" 527 | } 528 | }, 529 | "outputs": [], 530 | "source": [ 531 | "# Distribution on parameters\n", 532 | "mixture_proportion = dirichlet(alpha=[1, 1]).rvs()[0]\n", 533 | "mean = norm(loc=6, scale=1).rvs(size=2)\n", 534 | "std = norm(loc=.5, scale=.2).rvs(size=2)\n", 535 | "\n", 536 | "data = []\n", 537 | "for z in range(1, 101):\n", 538 | " iris_type = random.choice(2, p=mixture_proportion)\n", 539 | " sepal_length = norm(loc=mean[iris_type], scale=std[iris_type]).rvs()\n", 540 | " data.append((iris_type, sepal_length))" 541 | ] 542 | }, 543 | { 544 | "cell_type": "code", 545 | "execution_count": 12, 546 | "metadata": { 547 | "collapsed": false, 548 | "slideshow": { 549 | "slide_type": "slide" 550 | } 551 | }, 552 | "outputs": [ 553 | { 554 | "data": { 555 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAFeCAYAAABQCrzXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlgU2W6x/Ff2tBFWii7tiogsrggSl2qiJUiDiqgCAyL\nFhg3EHEYwQEBBQVBUGZkLlYBueodnLEqgoh6xQXRkd2CCArIvrSsbelKt/S9f3CJYPeQktfm+/mL\nJOc953nynuTXc0hyHMYYIwAAYIUAXxcAAAB+RTADAGARghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgt\n9MMPP2jgwIG6++671b17dz3yyCPasWOHJGnz5s0aMWKE17YVFxenn376qdxlNm3apIkTJ1Z53ZMn\nT9Yrr7xS6mMLFy7Uvffeq3vuuUfdu3fXM888o+zs7CpvozJeeeUVPf/88yXuX7RokYYOHVot2/yt\n999/X++880659fxWdna2+vXrp+7du+uLL77weNvx8fH6/PPPS31syJAh2rlzZ5XWV97+WZ4DBw7o\nz3/+c5W25StLly5VfHx8qY+V95rp2bNnlfbjLl266NNPPy1x/+TJkzVlyhQtW7ZMU6ZMqfT6JOnI\nkSPq379/hct5Mvc4N5y+LgBnKigo0NChQ/XWW2+pTZs2kqSPPvpIjzzyiL766itdeeWV+sc//nFO\na9q+fbsOHz7stfVt2rRJr776qhYtWqTw8HAZY/Tss8/q2Wef1YwZM7y2HZusX79erVq1qtKYLVu2\nKD09XUuXLq2mqqQ5c+ZUafmK9k+Hw1Hm2OTkZO3evfus6j2XyuulLIsWLarS8gMGDNCCBQt05513\nuu/Lz8/Xxx9/rPfee09NmzZVXFxcldbZuHFj9x+B5anq3OPcIZgtk5eXp+zs7DP+6u7Ro4fCw8Pl\ncrm0fv16TZ48WUuWLNHYsWMVHBysTZs2KTU1VV27dlX9+vW1bNkypaam6vnnn9cNN9ygsWPHqlWr\nVvrTn/4kSSVuS5IxRlOmTNGmTZuUk5MjY4yef/55XXDBBZo1a5ays7M1btw4TZ06VcuWLdPs2bNV\nVFSkkJAQjR49WldffbWys7P19NNPa9u2bWrUqJECAwMVHR1dosejR4/KGKPc3FyFh4fL4XBoxIgR\nZxx1zZ49W59//rmMMYqKitLEiRPVqFEjxcfH69JLL9XmzZt1/Phx9ejRQ48//rh7zFdffaWCggKd\nOHFCo0eP1m233ebRPBw+fFiTJ0/WwYMHVVRUpLvuukuPPPKIkpOTNXjwYMXGxmrjxo3KzMzUiBEj\ndOeddyovL08TJ07UDz/8oLp166pFixaSpM6dO2vZsmVauXKlgoODJUk7d+7UwIEDdfToUTVs2FAv\nv/yyGjZs6N7+7t27NX78eB05ckQ9e/bUu+++q2+//VYJCQkqLi5WWFiYxowZo6uuukqvvPKKNmzY\noKNHj6pNmzZ68cUXy+wrLi5O7dq10y+//KInnnhCU6dO1axZs9S8eXONHTtW+/btk8Ph0JVXXqlJ\nkyaVGF/R/ul0OvX111/rtddec+8fY8aMUdu2bfXMM8/oyJEjeuihhzRv3jx9+eWXpfaza9cujR8/\nXgUFBTLGqHfv3howYIBSU1M1YcIEpaam6tixY4qMjNTMmTNVv379M2osb7m4uDjde++9WrVqlQ4e\nPKg77rhDf/3rXyVJ//jHP/Txxx+rXr16uvjiiyu1n7Rt21adO3fWtm3b9NJLL6l3795avXq1ioqK\nNGbMGKWnp0uSYmNjSz3Tde+99+q//uu/dPDgQV1wwQWSpE8//VRXXnmlmjZtqkWLFmnp0qWaPXu2\n4uPjFRERod27d6t///7q2LGjxo0bp4yMDDVq1EjGGN1999267rrr1K1bN23YsEGvvPKKkpOTdeTI\nEaWkpKh+/fqaOXOmGjVqpLi4OM2aNUtXXHGFFixYoLfeekuBgYGqV6+epk+frsaNG5f6nnDNNddU\n6rnBWTCwzptvvmnatWtnbrvtNvPXv/7VLFiwwJw4ccIYY8yaNWtMt27djDHGPPXUU6Zv377G5XKZ\no0ePmtatW5u3337bGGPM//zP/5gHHnjAvdwbb7zhXv/ptzt16mQ2b95sNmzYYEaMGOFeZs6cOWbo\n0KHGGGMWLlxohgwZYowxZs+ePaZbt27m+PHjxhhjtm/fbjp06GBOnDhhpkyZYp566iljjDGpqakm\nNjbWzJo1q0R/hYWFZtSoUebyyy83PXv2NJMmTTLLly93P75o0SLzxBNPGJfLZYwx5t133zUPP/yw\nMcaY+++/3wwZMsS4XC6TmZlpunbtapYvX26Sk5PNoEGDTH5+vjHGmE8++cR0797dGGPMrFmzzOTJ\nk0vUcXpfvzVw4EDz9ddfG2OMyc/PNwMHDjT/+7//aw4cOGBat27trnfp0qWmU6dOxhhjZsyYYUaN\nGmWMMSYrK8v06NHD/Xyc/pzPmjXL3HbbbSY9Pd0YY8ywYcPMq6++WqKG0+d6x44dpkOHDubAgQPG\nGGNWrVplOnToYLKzs82sWbPMHXfcYYqLi0vt5f777zdLly41xpyc79O3FRcXZzZv3mw+/PBD89BD\nDxljjHG5XOaZZ54x+/btK3V95e2f5e0fle1n3LhxZu7cucYYY44ePWpGjhxpjDm5T7/++uvuOh5+\n+GHz5ptvlqivvOU6depkpk+fbowx5tChQ+aqq64yBw4cMF988YXp1q2byc3NNS6XywwZMsTEx8eX\n2v+p14wxxrRu3dp89NFH7sfatGlj0tPTTUJCgpk4caIxxpjc3FwzcuRIk5WVVer6xo4de8brpF+/\nfmbZsmXGmDP30fvvv9+MHz/evVzfvn1NYmKi+/m8+uqrzaJFi8yBAwfMNddcY4w5ua916dLF5OTk\nGGOMGTp0qHtbp/rYsmWLiYmJMYcOHXI/fxMnTiz3PQHViyNmCw0ePFh//OMftW7dOq1bt06vv/66\n5s2bp/fff7/Esp06dVJAQIAaNmyo0NBQdezYUZJ08cUXKyMjo9LbvPrqqzVixAi988472rdvn9au\nXauwsLASy61YsULHjh3T4MGDZf7/11ydTqf27NmjVatWafz48ZKk+vXrl3m06nQ6NWPGDI0ZM0Zr\n1qzR2rVr9dRTT+nGG2/U3//+dy1fvlybNm3SvffeK0kqLi5Wfn6+e3zfvn0VEBCg8PBwde3aVf/5\nz38UGxuradOmafHixdq3b59++OEH5ebmVrr/0504cULr1q1TZmamZs6c6b5vy5Ytatu2rWrVqqXY\n2FhJ0uWXX+5+nr/99luNHTtWkhQWFqaePXtq27ZtpW7jpptuUkREhCSpTZs2Sk1NLbemNWvW6MYb\nb1RUVJQkKSYmRg0bNnT/X2e7du0qfer12muvdf/71BxGR0dr5syZio+PV4cOHTRo0CBddNFFpY4v\nb/8sa//Yu3dvpfu5/fbbNXr0aP3444+68cYb3fvUwIED9f333+utt97Snj17tGPHDrVr165EfRUt\n17lzZ0lSkyZN1KBBA2VkZGj16tXq0qWLQkNDJUm9evXS/PnzK/V8nn5W6FTPHTt21JAhQ5SSkqKb\nbrpJo0aNKvX1JEn9+/fXiBEjNHz4cG3fvl1HjhzRrbfeWuqyp+YuMzNTP/74o/71r39Jklq0aKGY\nmJhSx1x//fU677zzJJ3cX48fP37G46tXr1bHjh3VpEkTSSefv1Mq854A7yOYLbN+/Xpt2LBBDz74\noGJjYxUbG6uRI0eqW7duWrlypfvN/JSgoKAzbjudpU+pOe0n0QsKCko8vnz5ck2dOlUPPPCAbrvt\nNl1yySVasmRJieWKi4vdAXrKoUOH1LhxYzkcjjO2U1YtH3zwgerVq6e4uDh169ZN3bp106OPPqrO\nnTtrwoQJKi4u1sMPP6x+/fpJkgoLC5WZmekeHxgYeEY9gYGB+vnnnzVs2DANHjxYN998s6677jo9\n99xzpW6/Ii6XS5L07rvvup/f9PR0hYSEKC0tTbVq1XIve3rPgYGBZ/QfEFD2Zyt/u46KmFJ+0t7l\ncqmoqEiSVLt27QrXccqpN+nTXXjhhfr888+1du1arV69WoMGDdIzzzyjbdu2admyZXI4HIqLi1OH\nDh3K3T/L2j+aNGmidevWVaqf2NhYff7551qxYoVWrVqlhIQEJSYmKjExUZs3b1avXr0UExOjoqKi\nUtfz0ksvlbtcSEjIGcufeuz0ZU7fxzx5Ptu2bauvvvpKK1eu1OrVq9W7d28lJCTogw8+0ObNm+Vw\nONSvXz/17dtXbdu2VcOGDbVy5UotX75c/fr1K3OfOLWtgICAEq+3smo+vd/S1hsYGHjG/fn5+UpO\nTtb+/fs1ZcqUCt8T4H18Ktsy9evX1+zZs7V+/Xr3fYcPH1ZeXl6VPzx0+jo3b94sSUpLS1NSUlKJ\nZVauXKm4uDj169dPV155pb766isVFxdLOvnCPRUAMTExWrFihXbt2iVJ+uabb3T33XeroKBAHTt2\n1IIFC2SMUUZGhr766qtS6wkICNDf/va3Mz5Qtnv3bkVFRalu3bq6+eab9f7777v/H3PmzJkaPXq0\ne9mPPvrIvY3PPvtMnTp10rp169S2bVsNHjxY1113nb788kt3/VUVFhamdu3a6b//+78lnTw66d+/\nv7uf0sJAkm699VYtXLhQxhidOHFCH3/8sfsNLzAwUIWFhR7VI/36vB84cECStGrVKh0+fFhXXXWV\nx+s83TvvvKOnnnpKHTp00KhRo9SxY0dt375df/7zn/Xhhx9q0aJFevzxxyvcP8vaP/Lz80vdj0rr\nZ9SoUfrkk0905513auLEiQoLC9OhQ4e0cuVKDRo0SD169FC9evXcfwj81ooVKyq13Ok6duyozz77\nTFlZWSouLtbixYvP6vn829/+poSEBHXu3Fnjx4/XpZdeqr179+r55593P599+/Z1Lz9gwAAtWrRI\nX3zxhfr06VPh+sPCwtS+fXt98MEHkqT9+/dr1apV7sfL2kdLc8MNN2jlypU6duyYpJP7wowZM8p9\nT0D14ojZMs2aNVNCQoL+/ve/6/DhwwoODlZYWJgmT56sZs2a6ciRI2WOLeuv7Pj4eD355JO64447\nFBUVpRtuuKHEmH79+unJJ5/U3XffrcDAQF177bXur9hcc801mjlzph5//HHNmjVLkyZN0siRIyWd\nDJzXXntNISEhevzxxzVx4kTdcccdatCggVq3bl1qPT179lReXp4efvhhFRYWyuFwqFmzZpo3b54c\nDof69OmjI0eOuE9ZX3DBBZo2bZp7fH5+vnr37q3c3Fzdd999iomJUcuWLfX555/rrrvuUlBQkGJi\nYnT8+PEKT2d/9913at++vaSTb2Z169bV8uXLNWPGDE2ePFndu3dXUVGRunfvrm7duik5ObnM53nI\nkCGaNGmSevToobCwMDVo0MB9avSWW27R5MmTy62lPC1atNDEiRM1fPhwuVwuhYaGavbs2ZU6tXh6\nvb+t/dTte+65R2vXrtWdd96p0NBQRUVFadCgQSXWVdH+KanM/aNly5YKCAjQH//4R7333ntl9jNs\n2DA9/fTTeu+99xQQEKDbb79d1113nYYNG6bp06crISFBTqdT0dHRJU6RS9Jjjz1W5nJl9R8bG6vt\n27erV69eqlu3rtq0aeP+4JYnz+egQYM0ZswYde/eXUFBQWrTpo3uuuuuUtcnSXfeeadefPFFxcbG\nljgrVta2pk2bpvHjx+udd95RkyZNdNFFF7n3t8qchTm1TKtWrTR69Gg9+OCDcjgcatSokaZOnaqs\nrKwy3xNQvRymKn9aAT4WHx+v+Ph43X777b4upYRPP/1UtWvXVmxsrIwxevzxx3XzzTe7T8kD3jR7\n9mz94Q9/UPPmzZWdna0ePXro9ddfd38bAL9flTpi3rhxo2bMmKH58+dry5YtevbZZ+V0OtWsWbMq\nf/kdOBuefLf0XGnZsqUmTJigl19+WYWFhYqJianUaUnAE82aNdNf/vIXBQQEyOVyaciQIYRyDVHh\nEfO8efO0ePFi1a5dW4mJiRo+fLj69u2rjh076sknn1S3bt3K/AQhAAComgo//NW0aVMlJCS4b192\n2WVKT0+XMUY5OTllfvIWAABUXYXB3KVLlzM+hn/q9PVdd92ltLQ0XX/99dVaIAAA/qTKh7tTpkzR\nv//9b7Vo0UL/+te/NG3aNE2YMKHcMaV9PQcAgJqutJ8lrkiVgzkiIsL9FY0mTZpow4YNlRrnSXE1\nRVJSEv3Tv6/L8Bl/7t+fe5fo39OD0ioH8+TJk/WXv/xFTqdTQUFBZ/XdTAAAcKZKBXNUVJQSExMl\nnTzyrcwlxQAAQNXxk5wAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBFCGYAACxCMAMAYBGCGQAAixDM\nAABYhGAGAMAiBDMAABYhmAEAsEiVry4FwDuKi4u1/0Cyr8so0wXnN1FQUJCvywD8DsEM+EhWVpY+\n/HqrwurU93UpJRQU5Klz+0K1anmJr0sB/A7BDPhQSEioQs+r7esySggMDPR1CYDf4v+YAQCwCMEM\nAIBFCGYAACxCMAMAYBGCGQAAixDMAABYhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAsUqlg3rhx\no+Lj4yVJaWlpGjZsmOLj4zVgwADt37+/WgsEAMCfVHgRi3nz5mnx4sWqXfvkD+2/9NJL6tGjh7p2\n7ao1a9Zo165duuiii6q9UAAA/EGFR8xNmzZVQkKC+/b69et16NAh/elPf9LHH3+sG264oVoLBADA\nn1QYzF26dDnjEnDJycmKiIjQm2++qfPPP19z586t1gIBAPAnVb4ec0REhDp16iRJiouL08yZMys1\nLikpqaqbqlHon/5/Kzs7WwcPHVNWboEPKipfYUG+fjL7lZWZ7pX1+fP8+3PvEv17osrBHB0drW++\n+UY9evTQunXrdOmll1Z6nL9KSkqif/ovcX9GRoa2Htqk8IiGPqiqfAX5ebrisjpq1fKSs16XP8+/\nP/cu0b+nf5RU+etSY8aM0Ycffqj+/fvru+++09ChQz3aMAAAKKlSR8xRUVFKTEyUJEVGRuqNN96o\n1qIAAPBX/MAIAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGY\nAQCwCMEMAIBFCGYAACxCMAMAYBGCGQAAixDMAABYhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAs\nQjADAGARghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgBALBIpYJ548aNio+PP+O+JUuWqF+/ftVSFAAA\n/spZ0QLz5s3T4sWLVbt2bfd9P//8sz744INqLQwAAH9U4RFz06ZNlZCQ4L6dnp6umTNnavz48dVa\nGAAA/qjCI+YuXbooOTlZklRcXKynn35aTz31lIKCgmSMqfYCAeC3CgoKrH3/qVWrlgIC+PgOPOcw\nldi7k5OTNWrUKI0fP17jxo1TvXr1lJ+fr507d6pXr14aO3ZsueOTkpK8VjBQU2RnZ+vrH44prE49\nX5dSQmFBvtqcX6SLL4z0dSkluFwuLfhsrc4Lb+jrUkp16fmBuqzVJb4uA5aIjo6u8pgKj5hPMcao\nbdu2WrJkiaRfw7qiUD6b4mqKpKQk+qf/EvdnZGRo66FNCo+wL2AK8vN0xWV11Krl2QeMt+ff5XJp\n/Y4s1WkQ5bV1elPLRoWKbt9WEvs+/Xt2UFrp8y0Oh8OjDQAAgMqrVDBHRUUpMTGxwvsAAMDZ4RMK\nAABYhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGARghkAAIsQzAAAWIRgBgDAIgQzAAAW\nIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBFCGYA\nACxCMAMAYBGCGQAAixDMAABYhGAGAMAilQrmjRs3Kj4+XpK0ZcsW3XfffRo4cKAeeughpaWlVWuB\nAAD4kwqDed68eXr66adVWFgoSZo6daomTJigf/7zn+rSpYvmzp1b7UUCAOAvKgzmpk2bKiEhwX37\n5ZdfVuvWrSVJRUVFCg4Orr7qAADwMxUGc5cuXRQYGOi+3bBhQ0nS+vXr9e9//1uDBw+utuIAAPA3\nTk8Gffrpp5ozZ47mzp2revXqVWpMUlKSJ5uqMeif/n8rOztbBw8dU1ZugQ8qKl9hQb5+MvuVlZnu\nlfV5c/5dLpcOHjyk7HyH19bpTa60VAWYX+eUfd+/+/dElYN58eLFeu+99zR//nzVqVOn0uOio6Or\nuqkaIykpif7pv8T9GRkZ2npok8IjGvqgqvIV5OfpisvqqFXLS856Xd6ef5fLpfU7slSnQaTX1ulN\nLRs1UnT7tpLY9+nfsz9KqhTMxcXFmjp1qiIjI/XYY4/J4XDo+uuv1/Dhwz3aOAAAOFOlgjkqKkqJ\niYmSpDVr1lRrQQAA+DN+YAQAAIsQzAAAWIRgBgDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBg\nEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCwCMEMAIBFCGYAACxCMAMAYBGCGQAAixDMAABYhGAG\nAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGARghkAAIsQzAAAWKRSwbxx40bFx8dLkvbt26cB\nAwbo/vvv13PPPVetxQEA4G8qDOZ58+bp6aefVmFhoSTphRde0MiRI/X222+ruLhYX375ZbUXCQCA\nv6gwmJs2baqEhAT37Z9++knXXnutJOmWW27RqlWrqq86AAD8TIXB3KVLFwUGBrpvG2Pc/65du7ay\nsrKqpzIAAPyQs6oDAgJ+zfKcnBzVqVOnUuOSkpKquqkapSb3f/RYqtZuTpGzVq0yl1m6Yus5rOhX\nBfkn1On6lgoLC/PJ9k8pbf6zs7N18NAxZeUW+KCi8hUW5Osns19ZmeleWZ8393+Xy6WDBw8pO9/h\ntXV6kystVQHm1zmtya/9yvD3/j1R5WC+/PLLtW7dOl133XX69ttvFRMTU6lx0dHRVS6upkhKSqrR\n/e/Zu08pJxooJLR2qY+npKQoMjLyHFd1UtbxNLVt20b169f3yfalsuc/IyNDWw9tUnhEQx9UVb6C\n/DxdcVkdtWp5yVmvy9v7v8vl0vodWarTwDf7VEVaNmqk6PZtJdX8135F6N+zP0qqHMxjxozRM888\no8LCQrVo0UJdu3b1aMMAAKCkSgVzVFSUEhMTJUnNmjXT/Pnzq7UoAAD8FT8wAgCARQhmAAAsQjAD\nAGARghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFik\nypd9BH5PgkPP0yfLN8rpDPRZDfv37dcvyTkl7ne5ihUUUscHFQGwGcGMGi0oOEQKvkDGhzUE1C6S\nCW5c8n5Jwee+HACW41Q2AAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIE\nMwAAFiGYAQCwCMEMAIBFPPqt7KKiIo0ZM0bJyclyOp2aPHmymjdv7u3aAADwOx4dMX/zzTcqLi5W\nYmKihg0bppdfftnbdQEA4Jc8CuZmzZrJ5XLJGKOsrCzVqlXL23UBAOCXPDqVXbt2bR04cEBdu3bV\n8ePHNWfOHG/XBQCAX3IYY6p8qdpp06YpODhYTzzxhA4fPqyBAwdqyZIlCgoKKnX5pKSksy4U9ko5\neFg/7jcKDgn1dSnwksKCfLU5v0gXXxjp61JKcLlc+uQ/2xVe/wJfl1KqJiGpurzVJb4uA5aIjo6u\n8hiPjpjr1q0rp/Pk0PDwcBUVFam4uNjrxdUUSUlJNbr/Bnv36WjRUYWE1i718ZSUFEVG2vcGf678\nHvsvyM/TFZfVUauWZx8w3t7/XS6X1u/IUp0Gdj6nLRs1UnT7tpJq/mu/IvTv2UGpR8E8aNAgjRs3\nTvfdd5+Kioo0atQohYSEeFQAAAD4lUfBfN5552nmzJnergUAAL/HD4wAAGARghkAAIsQzAAAWIRg\nBgDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFvHo6lIA\naraAwECt3bRbm3ccOut17d2zV3uP5HuhqpOMMaoVUvq1v4GagGAGUILTWUtyRinPC+tyheQpL6C+\nF9b0q1ByGTUYp7IBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCw\nCMEMAIBFCGYAACzi8W9lz507V8uWLVNhYaEGDBigXr16ebMuAAD8kkfBvHbtWm3YsEGJiYnKzc3V\nG2+84e26AADwSx4F83fffadWrVpp2LBhysnJ0ejRo71dFwAAfsmjYE5PT1dKSormzJmj/fv369FH\nH9Vnn33m7doAAPA7HgVzRESEWrRoIafTqebNmys4OFhpaWmqX9+711yFnX7ctln705Ldt48dTdXW\nA5mqVSuk1OWPH09XesE+SVJRXr7atuqogAA+dwgApfEomKOjozV//nwNHjxYhw8fVl5enurVq1fu\nmKSkJI8KrClqUv8bftmonAZF7ttZRTk6EWZU4CwsdflatZ3KUdbJZbPSlZKSrICAwHNSqy1SUlJ8\nXYJP+VP/rrRUBZgC9+2a9Nr3hL/37wmPgvnWW2/V999/r969e8sYo4kTJ8rhcJQ7Jjo62qMCa4Kk\npKQa1X9mcY6Ohma6b4cez1BOcY6czqBSl8/KzlJ4WLgkyYQVKfKCSAUE+k8wp6SkKDIy0tdl+Iy/\n9d+yUSNFt28rqea99quK/j37o8Tjr0s9+eSTng4FAABl4D/6AACwCMEMAIBFCGYAACxCMAMAYBGC\nGQAAixDMAABYhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGARghkAAIt4fHUpwBO1Qmpp\n6751UgWXCS2NMcW6uEFrhdep79G2t+36Xq4Al0djz3OGqdmFV3g0FgCqgmDGORUaEfb//zJVHusq\nKFJObqb8yjIWAAALHUlEQVTHwZyrTIU0Oc+jsTmHMzwaBwBVxalsAAAsQjADAGARghkAAIsQzAAA\nWIRgBgDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACLnFUwp6am6tZbb9Xu3bu9VQ8A\nAH7N42AuKirSxIkTFRIS4s16AADwax4H8/Tp09W/f381btzYm/UAAODXPArmhQsXqkGDBurQoYOM\nqfrl+wAAQOk8uh7zwoUL5XA4tGLFCm3dulVjxozRa6+9pgYNGni7PlSTtZu+V2Zupkdjj2QekzPU\ns+sanw2Hw6GD6buVkXfUo/HFgS4vVwSU9MvuFKVn5kmSdu7aqdSsYh9X9KtWl0Sq2cVRvi4DFXCY\nszzkjY+P16RJk9S8efMyl0lKSjqbTaAafLdltdS4llfWlZWdo8OZRoFO76zPRgXJhbqk8dW+LgM4\nK42CjunKNi18XYZfiY6OrvIYj46YT+dwOCq1nCfF1RRJSUnW9b8/96AK6nlnXaHHM5RTnCOnM6jU\nx7OysxQeFu6djflIcUSxIiMjPRqbkpLi8diawJ/7t633S+o3UHR0u3O2PRvf+84lTw9KzzqY//nP\nf57tKgAAwP/jB0YAALAIwQwAgEUIZgAALEIwAwBgEYIZAACLEMwAAFiEYAYAwCIEMwAAFiGYAQCw\nCMEMAIBFCGYAACxCMAMAYBGCGQAAi5z11aVwbqSmpeuHzTulSl5msyI/b9+rwCbeuX5yfn6+HI5Q\nr6zLVnlFufpl73qPxqYdS7fq0n/wX7v2HVZOnmeXIvTEzp07dDy3csu2aHq+ml0cVb0F/U4QzL8T\nh48cVXJmkJy1Sr/mcVVlFQTLWeil6Q8IUqB31mSt86LC5FKBR2Nz0zO8XA3gobCLdbiSQekNmaaR\nDufWrtSyIQePEsz/j1PZAABYhGAGAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGARghkAAIsQ\nzAAAWIRgBgDAIgQzAAAWIZgBALCIR1cxKCoq0rhx45ScnKzCwkINHTpUcXFx3q4NAAC/41Ewf/TR\nR6pXr55efPFFZWRk6J577iGYAQDwAo+C+Y477lDXrl0lScXFxXI6uXokAADe4FGihoaGSpKys7M1\nYsQIPfHEE14tCpWzc89GFTuMR2PzXSfkVLiXK0KpAo227/3Bo6HG5dIlF18lp7OWl4sCYCuHMcaj\nd/aDBw9q+PDhuv/++9WzZ89yl01KSvKoOPxq99792pEaesYb9NYjKxUWVdeHVaG65aZn6eLAtgoJ\nrdzF5oHfqwa1UnXVZZf4ugyvi46OrvIYj46Yjx07pgcffFATJkxQTExMpcZ4UlxNkZSUdNb9h9YO\nV+6OPDlrBbnvSz4RrrAw+496s7KzFP47qLO6nE3/AYVG59e9QOfVDvNyVedOSkqKIiMjfV2GT/hz\n71LV+m8aUV/R0VdXc0XnlqcHpR59XWrOnDnKzMzUq6++qvj4eA0cOFAFBQUeFQAAAH7l0RHz+PHj\nNX78eG/XAgCA3+MHRgAAsAjBDACARQhmAAAsQjADAGARghkAAIsQzAAAWIRgBgDAIgQzAAAWIZgB\nALAIwQwAgEUIZgAALEIwAwBgEY8uYnE2jh5L1fZdB871ZivHFOumG66p0pAjx45o54Hd5S7zy85f\nVBBYVOL+4mKja6+4RsHBwVXaJgCg5jrnwbxrb7L2pNsZRJmpybrphqqN2XVgj/Y7j5a7zPGIE6Uu\nk3U8U62zLiWYAQBunMoGAMAiBDMAABYhmAEAsAjBDACARQhmAAAsQjADAGARghkAAIsQzAAAWIRg\nBgDAIgQzAAAWIZgBALAIwQwAgEU8uoiFMUbPPvustm3bpqCgIE2ZMkUXXXSRt2sDAMDveHTE/OWX\nX6qgoECJiYkaNWqUXnjhBW/XBQCAX/IomJOSktSxY0dJUrt27bR582avFgUAgL/y6FR2dna2wsPD\nf12J06ni4mIFBFSc87UCA5WTnuzJZqtdZmqylixZUqUxOw/s1sGiY+Uuk5aWqpT6B0rcfyInV1s+\nXK/Q0NAKt5OWlq7i4EYKDAx035ehVBXk1a9Svb6QlZmpwjp1fF2Gz5xN//m5J3TCcUimoOJ9xFY5\n6SnKCTW+LsMn/Ll3qWr9B9RrUM3V/H54FMxhYWHKyclx365MKCclJbn/3b5lhCebrX4e1BUZGVkN\nhQA1h7Wv93PAn3uXqtq/64yc8GceBXP79u319ddfq2vXrvrhhx/UqlWrcpePjo72qDgAAPyNwxhT\n5fMsp38qW5JeeOEFNW/e3OvFAQDgbzwKZgAAUD34gREAACxCMAMAYBGCGQAAi3j0qeyKpKamqlev\nXnrzzTfP+FDYsmXL9Oqrr8rpdKpXr17q06dPdWzep8rq/a233tKCBQtUv/7J7x1PmjRJzZo181GV\n1ePee+9VWFiYJOnCCy/U1KlT3Y/5w9yX178/zP/cuXO1bNkyFRYWasCAAerVq5f7MX+Y//L6r+nz\nv2jRIi1cuFAOh0P5+fnaunWrVqxY4X491OT5r6h3j+beeFlhYaF57LHHzB/+8Aeza9euM+7v0qWL\nycrKMgUFBaZXr14mNTXV25v3qbJ6N8aYJ5980vz0008+qqz65efnm549e5b6mD/MfXn9G1Pz53/N\nmjVm6NChxhhjcnJyzKxZs9yP+cP8l9e/MTV//k/33HPPmffee8992x/m/5Tf9m6MZ3Pv9VPZ06dP\nV//+/dW4ceMz7t+5c6eaNm2qsLAw1apVS9HR0Vq3bp23N+9TZfUuST/99JPmzJmjAQMGaO7cuT6o\nrnpt3bpVubm5evDBBzV48GBt3LjR/Zg/zH15/Us1f/6/++47tWrVSsOGDdOjjz6qTp06uR/zh/kv\nr3+p5s//KZs2bdKOHTvOOCL2h/mXSu9d8mzuvXoqe+HChWrQoIE6dOig2bNnn/HYb3/Gs3bt2srK\nyvLm5n2qvN4l6a677tJ9992nsLAwPfbYY/rmm28UGxvrg0qrR0hIiB588EH16dNHe/bs0cMPP6yl\nS5cqICCgxs+9VH7/Us2f//T0dKWkpGjOnDnav3+/Hn30UX322WeSav5rXyq/f6nmz/8pc+fO1fDh\nw8+4zx/mXyq9d8mzuffqEfPChQu1YsUKxcfHa+vWrRozZoxSU1MlnfwZz+zsbPeyOTk5qlODfj+5\nvN4ladCgQYqIiJDT6VRsbKx+/vlnH1brfc2aNVOPHj3c/46IiNDRo0cl1fy5l8rvX6r58x8REaGO\nHTvK6XSqefPmCg4OVlpamiT/mP/y+pdq/vxLUlZWlvbs2aPrr7/+jPv9Yf7L6l3ybO69Gsxvv/22\n5s+fr/nz56tNmzaaPn26GjQ4+cPkLVq00N69e5WZmamCggKtW7dOV199tTc371Pl9Z6dna1u3brp\nxIkTMsZo9erVuuKKK3xcsXd98MEHmjZtmiTp8OHDysnJUaNGjSTV/LmXyu/fH+Y/Ojpa//nPfySd\n7D8vL0/16tWT5B/zX17//jD/krRu3TrFxMSUuN8f5r+s3j2d+2r5VLYkORwOSdLHH3+sEydOqE+f\nPho7dqweeOABGWPUp0+fUv8vtiYorfeRI0cqPj5ewcHBuvHGG3XLLbf4uErv6t27t8aOHasBAwYo\nICBAU6dO1aeffuo3c19R/zV9/m+99VZ9//336t27t4wxmjBhgj755BO/mf+K+q/p8y9Ju3fv1kUX\nXeS+7U/v/eX17snc85OcAABYhB8YAQDAIgQzAAAWIZgBALAIwQwAgEUIZgAALEIwAwBgEYIZAACL\nEMwAAFjk/wBsnH3U5gRGxwAAAABJRU5ErkJggg==\n", 556 | "text/plain": [ 557 | "" 558 | ] 559 | }, 560 | "metadata": {}, 561 | "output_type": "display_data" 562 | } 563 | ], 564 | "source": [ 565 | "plt.hist([sepal_length for iris_type, sepal_length \n", 566 | " in data if iris_type==0], **settings)\n", 567 | "plt.hist([sepal_length for iris_type, sepal_length \n", 568 | " in data if iris_type==1], **settings)\n", 569 | "_=plt.title('Simulated Sepal Length for Iris-Setosa and Iris-Virginica')" 570 | ] 571 | }, 572 | { 573 | "cell_type": "markdown", 574 | "metadata": { 575 | "slideshow": { 576 | "slide_type": "slide" 577 | } 578 | }, 579 | "source": [ 580 | " \"\"\"\n", 581 | " ⬅ Reversing the Process\n", 582 | " \"\"\"\n", 583 | "\n", 584 | "Given data without cluster assignments (in no particular order)...\n", 585 | "\n", 586 | "...find the cluster for each point\n", 587 | "\n", 588 | "...find the parameters of each cluster (or distribution over parameters)\n", 589 | "\n", 590 | "...find the mixture proportion (or distribution proportions)\n", 591 | "\n", 592 | "----\n" 593 | ] 594 | }, 595 | { 596 | "cell_type": "markdown", 597 | "metadata": { 598 | "slideshow": { 599 | "slide_type": "fragment" 600 | } 601 | }, 602 | "source": [ 603 | "\n", 604 | "You can do this with MCMC as in [PyMC3](https://github.com/pymc-devs/pymc3/blob/5317dd4ba1e413003aa80e9746e4e99143654183/docs/source/notebooks/gaussian_mixture_model.ipynb). See also [this notebook](http://nbviewer.ipython.org/github/tdhopper/notes-on-dirichlet-processes/blob/master/2015-09-02-fitting-a-mixture-model.ipynb)." 605 | ] 606 | }, 607 | { 608 | "cell_type": "markdown", 609 | "metadata": { 610 | "slideshow": { 611 | "slide_type": "slide" 612 | } 613 | }, 614 | "source": [ 615 | "# Discrete Mixture Model\n", 616 | "\n", 617 | "There's nothing inheriently special about the normal distribution. (👈Not actually true.)" 618 | ] 619 | }, 620 | { 621 | "cell_type": "code", 622 | "execution_count": 13, 623 | "metadata": { 624 | "collapsed": false 625 | }, 626 | "outputs": [], 627 | "source": [ 628 | "mixture_proportion = [0.3, 0.7]\n", 629 | "cluster_distributions = [[0.33, 0.33, 0.34], \n", 630 | " [0.2, 0.8, 0.0]]\n", 631 | "\n", 632 | "docs = defaultdict(str)\n", 633 | "for _ in range(100):\n", 634 | " cluster = random.choice(len(mixture_proportion), \n", 635 | " p=mixture_proportion)\n", 636 | " docs[cluster] += random.choice(['🐍', '🐼', '📊'], \n", 637 | " p=cluster_distributions[cluster])\n", 638 | " docs[cluster] = ''.join(sorted(docs[cluster]))" 639 | ] 640 | }, 641 | { 642 | "cell_type": "code", 643 | "execution_count": 14, 644 | "metadata": { 645 | "collapsed": false, 646 | "slideshow": { 647 | "slide_type": "fragment" 648 | } 649 | }, 650 | "outputs": [ 651 | { 652 | "name": "stdout", 653 | "output_type": "stream", 654 | "text": [ 655 | "Cluster 1: 🐍🐍🐍🐍🐍🐍🐍🐍🐍🐼🐼🐼🐼📊📊📊📊📊📊📊📊📊📊\n", 656 | "Cluster 2: 🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼🐼\n" 657 | ] 658 | } 659 | ], 660 | "source": [ 661 | "for cluster, doc in docs.items():\n", 662 | " print(\"Cluster {}: \".format(cluster+1), doc)" 663 | ] 664 | }, 665 | { 666 | "cell_type": "markdown", 667 | "metadata": { 668 | "slideshow": { 669 | "slide_type": "slide" 670 | } 671 | }, 672 | "source": [ 673 | " ➡ Forward Sampling \n", 674 | "Bayesian style!" 675 | ] 676 | }, 677 | { 678 | "cell_type": "code", 679 | "execution_count": 15, 680 | "metadata": { 681 | "collapsed": false 682 | }, 683 | "outputs": [], 684 | "source": [ 685 | "num_clusters = 2\n", 686 | "mixture_proportion = dirichlet(alpha=num_clusters*[1]).rvs()[0]\n", 687 | "cluster_distributions = (dirichlet(alpha=[1, 1, 1])\n", 688 | " .rvs(size=num_clusters))\n", 689 | "\n", 690 | "docs = defaultdict(str)\n", 691 | "for _ in range(100):\n", 692 | " cluster = random.choice(num_clusters, \n", 693 | " p=mixture_proportion)\n", 694 | " docs[cluster] += random.choice(['🐍', '🐼', '📊'], \n", 695 | " p=cluster_distributions[cluster])\n", 696 | " docs[cluster] = ''.join(sorted(docs[cluster]))" 697 | ] 698 | }, 699 | { 700 | "cell_type": "code", 701 | "execution_count": 16, 702 | "metadata": { 703 | "collapsed": false, 704 | "slideshow": { 705 | "slide_type": "fragment" 706 | } 707 | }, 708 | "outputs": [ 709 | { 710 | "name": "stdout", 711 | "output_type": "stream", 712 | "text": [ 713 | "Cluster 1: 🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐼🐼🐼🐼🐼📊📊📊📊📊📊📊📊\n", 714 | "Cluster 2: 🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐼🐼🐼📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊📊\n" 715 | ] 716 | } 717 | ], 718 | "source": [ 719 | "for cluster, doc in docs.items():\n", 720 | " print(end=\"Cluster {}: \".format(cluster+1))\n", 721 | " print(doc)" 722 | ] 723 | }, 724 | { 725 | "cell_type": "markdown", 726 | "metadata": { 727 | "slideshow": { 728 | "slide_type": "slide" 729 | } 730 | }, 731 | "source": [ 732 | "## Mixture Model for Documents (Grouped Data)\n" 733 | ] 734 | }, 735 | { 736 | "cell_type": "code", 737 | "execution_count": 17, 738 | "metadata": { 739 | "collapsed": false 740 | }, 741 | "outputs": [], 742 | "source": [ 743 | "vocabulary = ['🐍', '🐼', '📊']; \n", 744 | "boring_vocabulary = [':snake:', ':panda_face:', ':bar_chart:']\n", 745 | "num_terms = len(vocabulary)\n", 746 | "\n", 747 | "# Topics are distributions over vocabulary words\n", 748 | "topic_labels = ['☀', '☁']\n", 749 | "boring_topics = [':sunny:', ':cloud:']\n", 750 | "topics = [[ 0.1, 0.3, 0.6],\n", 751 | " [ 0.8, 0.1, 0.1]]\n", 752 | "\n", 753 | "# Proportion of documents of each topic\n", 754 | "mixture_proportion = [0.3, 0.7]" 755 | ] 756 | }, 757 | { 758 | "cell_type": "code", 759 | "execution_count": 18, 760 | "metadata": { 761 | "collapsed": false, 762 | "slideshow": { 763 | "slide_type": "slide" 764 | } 765 | }, 766 | "outputs": [ 767 | { 768 | "data": { 769 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfsAAAFeCAYAAACCWraPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAF75JREFUeJzt3XtwVIXZx/HfxkBKuQgpF+lbBE2a0kJFjMNLQSjXUgtZ\ncwESdJbYIqgIdow3RECQFtTIUCVEI9Nak2JIIeXqYKngpQwF7SpDwkVSiKDAW0EgJiQSSM77h5Md\nbmZjYHPIk+/nr2x3z8nzbFK/2bMxehzHcQQAAMwKc3sAAAAQWsQeAADjiD0AAMYRewAAjCP2AAAY\nR+wBADCO2AMN4NChQ+revbt8Pt9F9z3xxBPq3r27Tp48qcLCQv32t7+t9VwFBQV66qmnQjWq3n//\nffXq1UsJCQlKTExUfHy8Ro8erbfffjtkn7M25+5bl+cHwMXC3R4AaCoiIiJUXFysI0eOqHPnzpKk\niooKffjhh/J4PJKknj176oUXXqj1PEVFRfrvf/8b0lmvv/56rVy5MnB7z549GjdunDZt2qR27dqF\n9HNf6Nx96/L8ALgYr+yBBhIWFqZf/epXWrNmTeB/27Bhg4YMGRK4/f777ysuLk6SdPfddys9PV2S\ntGXLFg0aNEhHjhzRokWL5Pf7NX369PMef+HxGRkZmjBhgrxerx577DFJ0ssvv6zExEQlJCRoypQp\nOnr0aJ1m7969u1q0aKHDhw9fdN6zZ89q7ty5GjlypLxer2bOnKny8nJJ0pAhQzR//nwlJSVpxIgR\nys3NDZwzLy9PcXFxio+P14QJE3TgwAFJX1/puP/++xUXF6cZM2Z8475lZWV69NFHFRcXJ6/Xq/T0\ndFVXV0uSbrrpJmVkZGjcuHEaNmyYXnvtNUnSsWPHNGHCBCUmJioxMZEfHNBkEHuggXg8HsXHx58X\n+1WrVikpKemSj09PT9fq1au1ceNGTZ8+XQsWLFDnzp314IMPKjY2VvPmzQv6OY8cOaLVq1frueee\n06pVq7R3716tWLFCK1eu1MCBA/Xkk0/WafYNGzYoLCxM0dHRF533pZde0tGjR7V27VqtWbNGVVVV\neu655wLHfvXVV8rPz1d2drZefPFFFRUVaevWrfrTn/6knJwcrVq1SqNGjdLkyZMDx5w+fVpr167V\n7373u2/cd+7cuWrXrp3Wrl2r/Px87dmzR3/84x8lSZWVlYqMjFRubq5eeOEFLViwQJWVlfrrX/+q\nLl266G9/+5uWLl2qgwcPqqysrE7PAdCYcRkfaEA/+clPFBYWpl27dikyMlLl5eWKjo7Wpf5qdYcO\nHTR37lxNnjw5ELxvq1evXoG3CN555x0VFBQoMTFRklRdXa3Tp09f8riDBw8qISFBjuOoqqpK1113\nnV566SVFRERcdN733ntPaWlpCgv7+rWDz+fTAw88EDjXXXfdJUnq1KmTBg4cqM2bN+vYsWO6/fbb\n1bZtW0lSQkKC5s2bp0OHDkmSbrnllqC7/fOf/9SyZcskSc2aNdO4ceP02muvaeLEiZKkoUOHSpJ6\n9OihM2fOqKKiQgMGDNC9996rw4cPq1+/fnr44YfVqlWruj6dQKNF7IEG5vV6tXr1akVGRsrr9db6\n2KKiIrVv3147duy45P01wa1x5syZ8263bNky8HF1dbUmTpyolJSUwGNLSkoued4L37O/0IXnPVdV\nVZXOnj0buH3NNdecd98111xz0TE156k57tzzf5MLz3Hu8ZICP5jUcBxHP/3pT7Vx40Zt2bJFW7du\n1ejRo5WZmambb7456OcDGjMu4wMNpObVu9fr1Ztvvqn169ef9377hXbs2KGcnBzl5+fryy+/VHZ2\ntqSv41kTtcjISB0+fFjHjx+X4zh66623vvF8t912m5YvXx64bP2HP/xBjz/++GXvddtttyk3N1dn\nz55VdXW1Xn/9dfXv3z9w/+rVqyVJhw8f1pYtWzRw4EANGDBA69ev1/HjxyVJ+fn5ateunbp27XrR\n+c/d98LPu3TpUklfX7bPy8s77/Oeq+a5X7BggRYvXqyhQ4fqySefVHR0tD755JPL2h9oDIg90EBq\nXoV36tRJ0dHR6tatm9q0aXPefTVOnTqlhx9+WLNmzVLHjh31zDPPKDMzU3v27FHv3r21f/9+TZ06\nVVFRURo7dqySkpKUkpKijh07fuPnHzNmjAYPHqzk5GTFxcWpqKhI8+fPv+y9Jk+erA4dOig+Pl4j\nR45UVVXVeb8L8NlnnykxMVGTJk3SjBkz1K1bN/Xr10+pqalKTU1VXFycVq9eraysrEue/9x9zzVj\nxgwdO3Ys8At6N954o+677z5JFz+fNbdTU1O1e/duxcXFKSkpSV26dNHIkSMv+zkArnYe/hO3AEJl\nyJAhWrRokXr06OH2KECTxit7ACFz4StsAO7glT0AAMbxyh4AAOOIPQAAxjXqf8/e7/e7PQIAAA2q\nPn9gq1HHXqrf0hb4/f4mu7vE/uzP/k11/6a8u1T/F7lcxgcAwDhiDwCAccQeAADjiD0AAMYRewAA\njCP2AAAYR+wBADCO2AMAYByxBwDAOGIPAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH7AEAMI7YAwBg\nHLEHAMA4Yg8AgHHEHgAA44g9AADGEXsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADj\niD0AAMYRewAAjCP2AAAYR+wBADCO2AMAYByxBwDAOGIPAIBxxB4AAOPC3R7gcu3du9ftEVxx4MAB\ntW7d2u0xXMP+7M/+TXP/prz75fA4juO4PUR9+f1+PbJ4m9tjAAAQcuUlnytzepxiY2O/9bGN/pV9\nq3b/4/YIAABc1XjPHgAA44g9AADGEXsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADj\niD0AAMYRewAAjCP2AAAYR+wBADCO2AMAYByxBwDAOGIPAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH\n7AEAMI7YAwBgHLEHAMA4Yg8AgHHEHgAA44g9AADGEXsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhi\nDwCAcVcs9k888YQ2b958WedITk7W4cOHr9BEAABA4pU9AADmfavYl5SU6MEHH9SBAwfk8/mUkpKi\nX//61zp+/HjgMWfPntWjjz6qlJQUJScna/369ZIkn8+n4uJiSdKyZcuUkZEhSVq4cKGSkpL0wAMP\n6OTJk5Kkffv26emnn74iCwIA0NSFf5sHX3vttXrxxRc1efJk3Xffferfv7/efvtt7d69O/CYvLw8\nfe9731N6erpOnTqlxMRE9e3b95LnKywslN/vV35+vsrKyvTLX/5SkhQVFaVZs2ZdxloAAKBGvS7j\nFxcXq1evXpKkwYMHq3///oH79u3bp1tvvVWS1LJlS0VFRenTTz8973jHcSRJn3zyiXr27ClJatWq\nlX74wx/WZxwAAFCLesU+OjpaBQUFkqS1a9fqL3/5S+C+qKgo/fvf/5YklZWVqaioSD/4wQ8UERGh\no0ePSpJ27doVOM+OHTskSeXl5frPf/5T/00AAMAl1es9+8cee0xZWVny+Xxat26dvF5v4DFjx47V\nyZMndeeddyo1NVVTpkxRZGSkfD6fZs+erXvuuUfV1dWSpO7du2vAgAFKSkpSWlqa2rdvL4n37AEA\nuJI8Ts019UbI7/dr9uufuT0GAAAhV3bikJ5/4H8VGxv7rY/lX70DAMA4Yg8AgHHEHgAA44g9AADG\nEXsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADjiD0AAMYRewAAjCP2AAAYR+wBADCO\n2AMAYByxBwDAOGIPAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH7AEAMI7YAwBgHLEHAMA4Yg8AgHHE\nHgAA44g9AADGEXsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAceFuD3C5yk4ccnsEAABCrrzk\n83of2+hjnzVtmNsjuKKwsFA9e/Z0ewzXsD/7s3/T3L8p7y5JpaWl9Tqu0cc+JibG7RFcUVpa2mR3\nl9if/dm/qe7flHeXJL/fX6/jeM8eAADjiD0AAMYRewAAjCP2AAAYR+wBADCO2AMAYByxBwDAOGIP\nAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH7AEAMI7YAwBgHLEHAMA4Yg8AgHHEHgAA44g9AADGEXsA\nAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADjiD0AAMYRewAAjCP2AAAYR+wBADCO2AMA\nYByxBwDAOGIPAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH7AEAMI7YAwBgHLEHAMA4Yg8AgHHEHgAA\n44g9AADGEXsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADjiD0AAMYRewAAjCP2AAAY\nR+wBADCO2AMAYByxBwDAOGIPAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH7AEAMI7YAwBgHLEHAMA4\nYg8AgHHEHgAA44g9AADGEXsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADjiD0AAMaF\nuz3A5dq7d6/bI7jiwIEDat26tdtjuKap719VVeX2CAAakUYf+3ufecvtEdyz7v/cnsBdTXT/8pLP\n9fhdN6lPnz5ujwKgkWj0sW/V7n/cHgEAgKsa79kDAGAcsQcAwDhiDwCAccQeAADjiD0AAMYRewAA\njCP2AAAYR+wBADCO2AMAYByxBwDAOGIPAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH7AEAMI7YAwBg\nHLEHAMA4Yg8AgHHEHgAA44g9AADGEXsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADj\niD0AAMYRewAAjCP2AAAYR+wBADCO2AMAYByxBwDAuPBgDygpKdGMGTPk9Xq1ZMkSNWvWTB07dtTC\nhQuVkZGhDh06KDk5Wfv379dTTz2lnJwceb1e9enTRx9//LE8Ho8yMzO1a9euwPGfffaZRo4cqUmT\nJmnEiBFasWKF2rRpo9zcXJWXl6tHjx768MMPNXny5IZ4DgAAMC3oK/trr71WixYt0htvvKF77rlH\nS5cu1aBBg1RaWnrRYz0ejySprKxMcXFxysnJUceOHfXee+9Jko4cOaLFixcrLy9PS5Yskcfjkdfr\n1RtvvCFJWrNmjRISEtS3b19CDwDAFRL0lX2NadOmKSsrSzk5OYqKitLQoUPPu99xnPNu//jHP5Yk\nde7cWZWVlZKkmJgYeTwetWjRQt/5znckSYmJiUpLS9Ott96qDh06KDIy8rIWApoKv9/v9giuYv+m\nu39T3r2+6hz7vLw8TZ06VZGRkZo1a5beeustRURE6PPPP5ck7dy5s14DfP/731fr1q318ssvKykp\nqV7nAJqi2NhYt0dwjd/vZ/8mun9T3l2q/w86QS/jl5SUaOrUqerVq5fuvfde3X333friiy80ePBg\n3X777Xr33Xc1fvx47d69O3BMzeX82j4+19ixY+X3+zVw4EBJ0tatW5WZmVmvhQAAwPmCvrKvec9e\nkgYNGnTRfStWrLjomI0bNwY+TktLC3zcp0+fwMebN28OfFxVVaWkpKTADwN9+/ZV375967gCAACo\nTZ0v44fKwoULtW3bNmVlZbk9CgAAJrke+4ceesjtEQAAMI0/qgMAgHHEHgAA44g9AADGEXsAAIwj\n9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADjiD0AAMYRewAAjCP2AAAYR+wBADCO2AMAYByx\nBwDAOGIPAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH7AEAMI7YAwBgHLEHAMA4Yg8AgHHEHgAA44g9\nAADGEXsAAIwj9gAAGEfsAQAwjtgDAGBcuNsDXK6yE4fcHgFoUOUln0u6zu0xADQijT72WdOGuT2C\nKwoLC9WzZ0+3x3BNU9//5MmTbo8AoBFp9LGPiYlxewRXlJaWNtndJfb3+/1ujwCgEeE9ewAAjCP2\nAAAYR+wBADCO2AMAYByxBwDAOGIPAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH7AEAMI7YAwBgHLEH\nAMA4Yg8AgHHEHgAA44g9AADGEXsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADjiD0A\nAMYRewAAjCP2AAAYR+wBADCO2AMAYByxBwDAOGIPAIBxxB4AAOOIPQAAxhF7AACMI/YAABhH7AEA\nMI7YAwBgHLEHAMA4j+M4jttD1Jff73d7BAAAGlRsbOy3PqZRxx4AAATHZXwAAIwj9gAAGEfsAQAw\njtgDAGAcsQcAwLhwtwcIxnEczZ49Wx9//LGaN2+u3//+9+rSpUvg/k2bNikzM1Ph4eFKSkrSmDFj\nXJz2ygu2vyRVVFToN7/5jebNm6cbbrjBpUlDI9j+69atU3Z2tsLDwxUTE6PZs2e7N+wVFmz3v//9\n71qyZInCwsI0atQojR8/3sVpr7y6fO9L0qxZs9S2bVulpaW5MGXoBNv/z3/+s1asWKHIyEhJ0tNP\nP61u3bq5NO2VF2z/HTt26Nlnn5UktW/fXunp6WrevLlb415Rte1+7NgxPfTQQ/J4PHIcR3v27NEj\njzyi5OTkoCe9qm3YsMGZNm2a4ziOs337duf+++8P3HfmzBln+PDhTmlpqVNZWekkJSU5X3zxhVuj\nhkRt+zuO4xQUFDiJiYlO//79nf3797sxYkjVtv9XX33lDB8+3Dl9+rTjOI6TlpbmbNq0yZU5Q6G2\n3auqqpxf/OIXTllZmVNVVeWMGDHCOXHihFujhkSw733HcZzc3FwnOTnZWbBgQUOPF3LB9n/kkUec\nnTt3ujFagwi2/x133OEcPHjQcRzHWb58uVNcXNzQI4ZMXb73HcdxPvroIyc1NdWprq4Oes6r/jK+\n3+/XgAEDJEm9evVSYWFh4L59+/apa9euatWqlZo1a6bY2Fh98MEHbo0aErXtL0lnzpxRZmambrzx\nRjfGC7na9m/evLmWLVsW+Gn+7NmzioiIcGXOUKht97CwMK1fv14tW7bUiRMn5DiOmjVr5taoIRHs\ne/+jjz5SQUGBUlJS3Bgv5ILtv3PnTmVlZenOO+/UK6+84saIIVXb/sXFxWrbtq1effVV+Xw+lZSU\nmLqqEexrX2Pu3LmaM2eOPB5P0HNe9bEvKytT69atA7fDw8NVXV19yftatmyp0tLSBp8xlGrbX5J6\n9+6tTp06yTH6t5Fq29/j8QQuYebk5KiiokL9+vVzZc5QCPa1DwsL0z/+8Q/dcccd6tOnj7773e+6\nMWbI1Lb/0aNHlZGRoVmzZjXJ731JGjlypObMmaPs7Gz5/X69++67bowZMrXtf+LECW3fvl0+n0+v\nvvqqtmzZom3btrk16hUX7Gsvff0WdkxMjLp27Vqnc171sW/VqpVOnToVuF1dXa2wsLDAfWVlZYH7\nTp06pTZt2jT4jKFU2/5NQbD9HcfRs88+q3/961/KyMhwY8SQqcvXfvjw4dq8ebMqKyu1atWqhh4x\npGrb/80339TJkyc1ceJEvfLKK1q3bl2T2l+SUlNT1bZtW4WHh+vnP/+5du3a5caYIVPb/m3bttX1\n11+vG264QeHh4RowYMA3vvptjOry//01a9Zo7NixdT7nVV+NW265JfAT6/bt2xUTExO4LyoqSgcO\nHNCXX36pyspKffDBB7r55pvdGjUkatu/KQi2/8yZMwNvZVj55Zwate1eVlYmn8+nyspKSVKLFi3q\ndCmvMaltf5/Pp/z8fGVnZ2vSpEkaNWqU4uPj3Ro1JIJ9/UeNGqWKigo5jqOtW7eqR48ebo0aErXt\n36VLF5WXl+vTTz+V9PVl7+joaFfmDIW6/HO/sLBQvXv3rvM5r/q/je+c81uJkjR//nzt3LlTFRUV\nGjNmjN555x1lZGTIcRyNHj1a48aNc3niKyvY/jXGjx+vOXPmmP5tfOn8/Xv06KHRo0cH/qMQHo9H\n48eP17Bhw9wc+YoJ9rVfvny5li9frmbNmulHP/qRZs6caSr4df3eX7lypYqLi03/Nr508f5r1qxR\ndna2IiIi9LOf/UxTpkxxeeIrK9j+27Zt0/PPPy/p67czp0+f7ua4V1Sw3Y8fP64JEyZo5cqVdT7n\nVR97AABwea76y/gAAODyEHsAAIwj9gAAGEfsAQAwjtgDAGAcsQcAwDhiDwCAccQeAADj/h+yntge\nNwyiCgAAAABJRU5ErkJggg==\n", 770 | "text/plain": [ 771 | "" 772 | ] 773 | }, 774 | "metadata": {}, 775 | "output_type": "display_data" 776 | } 777 | ], 778 | "source": [ 779 | "pd.Series(mixture_proportion, index=boring_topics).plot(kind='barh')\n", 780 | "_=plt.title(\"Mixture Proportions\")" 781 | ] 782 | }, 783 | { 784 | "cell_type": "code", 785 | "execution_count": 19, 786 | "metadata": { 787 | "collapsed": false, 788 | "slideshow": { 789 | "slide_type": "slide" 790 | } 791 | }, 792 | "outputs": [ 793 | { 794 | "data": { 795 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfsAAAFeCAYAAACCWraPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlclXXe//H3YZEQkaVE8zduoUQj5cLcZqOIpFg9NGVx\ny0LNu5xq0gprlFIzvV3QUZxEzWwxvUlNLQedu9HIbm1VhztH0dQyl1waTRZBUBSu+w9vz08TOSzG\nBd9ez7/gXJzrvD8c9H3OdZ1zvg7LsiwBAABjudkdAAAA/LIoewAADEfZAwBgOMoeAADDUfYAABiO\nsgcAwHCUPVCOxx9/XEuXLnV+f+jQIYWGhiolJcV5WXZ2tsLCwlRQUFDl29mwYYMSEhKuufzYsWP6\n7W9/q9jYWMXGxqpv376Kj4/X2rVrnT/z6quv6q9//Wu5+58/f742bdpU5rYrrx8aGqrc3NxKZd+1\na5defvllSVJWVpaeeeaZSl2/PFXJc9natWsVExOj2NhY3X333erWrZvz+8zMzCrtMy0tTW+99VaV\nrgvYycPuAEBt1q1bN23dulVDhw6VJH3yySe69957tWnTJj333HOSpK+++krh4eFq0KBBtW7L4XCU\neflNN92kDz74wPn98ePHNXz4cPn4+Cg6OlqjR492ue+vvvpKbdq0KXPblde/XobyfPvtt/rXv/4l\nSQoLC9Nf/vKXSu/jeqqS57KYmBjFxMRIkpKSkhQSEqJHH320Wnkefvjhal0fsAtlD5SjW7duSk1N\ndX6/adMmjRkzRomJiTp69Kh+85vf6Msvv1T37t0lXSq+KVOmKDc3V25ubho+fLhiYmK0bds2TZ06\nVd7e3jp37pxWrVqlBQsWaP369QoICFDz5s0rnKlp06YaPXq03nzzTUVHR19VZK+++qo+/vhjeXp6\nyt/fX9OnT9fGjRuVlZWlmTNnys3NTR9//LFyc3N19OhRde/eXT/99JPz+pZlac6cOcrKypJlWXrm\nmWfUvXt3ffDBB9qwYYNee+01SXJ+P2nSJM2bN08FBQV68cUXFRMToylTpmjdunUqKCjQK6+8or17\n98rhcCgiIkJjxoyRm5ub7rrrLo0cOVKff/65Tp06pYSEBA0bNuyaWX+e59lnn1VkZKRGjBihBx54\nQAMGDJAkvfbaa8rNzdW4ceMq/Hs8ceKEXnnlFR0/flySFBcXp+HDh+vIkSMaMWKEOnbsqP3798vh\ncOjll19W+/btNXfuXBUVFSkpKUkHDhzQpEmTlJOTI3d3dz311FO67777Knz7QE3iMD5QjhYtWsjf\n31979+7VmTNndOjQIbVv314RERH6+OOPJclZ9iUlJXrqqac0dOhQpaen6/XXX1dKSor++c9/SpK+\n++47zZ07V2vXrtXmzZuVkZGh9PR0rVixotKnAEJDQ7Vv376rLvvxxx+1dOlSrV69WqtXr1bXrl21\nc+dOPfzwwwoLC9PYsWPVs2dPSdL58+e1bt06jRkzpsyZ33//fc2cOVNjx45VTk7OdXM0adJEo0eP\nVnh4uKZNm3bVtilTpiggIEDr1q3TmjVrtHfvXr355puSpOLiYgUGBmr58uX6y1/+otmzZ6u4uLjM\n27gyz5/+9Cfl5ORoyJAheu+99yRdekCwatUqPfTQQxX/BUpKTExURESE0tPTlZaWpjVr1mjjxo2S\npKNHj6pHjx5au3atRo8erWeeeUY//7DRxMRE9e3bV+vXr9fChQs1e/ZsFRUVVSoDUFMoe8CFiIgI\nbdu2TVu2bFGXLl0kSVFRUfr888917NgxORwOtWrVSocOHVJxcbGzUIOCgtSrVy99+umnki4VY5Mm\nTSRdOqweHR0tb29vubm5KT4+vlKZHA6HvL29r7qscePGuuOOOxQbG6vk5GTdfvvt6tGjh3P7lWXV\nsWPH6+578ODBkqQ2bdqoTZs22rFjR6WyXfbpp5/qkUcekSR5enrqoYce0pYtW5zbL2dr27atLly4\ncN2ivDJP69attWPHDt177706ffq09u3bp08//VTNmjVTixYtKpytoKBAu3btcu7b19dXMTExznw3\n33yz81l6VFSUSktLtX//fuf1s7Oz9d133znvt6ZNm2rjxo3X3CdAbUHZAy5ERERo+/bt+uSTT5yH\n6zt37qw9e/boiy++UGRkpCSptLT0mutalqULFy5IkurXr3/Ntsvc3d0rlWnnzp0KCQm56jKHw6Fl\ny5ZpxowZCggI0PTp0695tn2Zj4/Pdfft5vb//1soLS2Vh4fHNefOL89Unp//PkpLS3Xx4kXn915e\nXldtL2uZDofDcVUey7Lk4eEhNzc3DR48WKtXr9aaNWucpV1RZd3WlfmuvM3L2668j9zd3eVwOK76\nvXz//ffXPToB2I2yB1zo3LmzvvnmG/3jH/9Q165dJV160Vzbtm2VlpbmfADQqlUreXp6KiMjQ5L0\nr3/9Sxs2bHAeDbhSRESE/v73vys/P1+lpaXlvpr+58V08OBBLVy4UCNGjLjq8r1796pPnz4KDg7W\nyJEjNXz4cO3du1eS5OHhcVXRluf999+XJO3evVtHjhxRu3btFBAQoP3796u4uFgXL1686pX97u7u\nZe67a9euSktLk3TpsP3KlSvL/F2UNeOVl5eVR5L69++vjIwM7dmzR9HR0RWa7TJfX1+1bdtWy5cv\nlySdOXNG6enpznynTp3Sl19+KUn66KOP5OPjo9atWzuv7+fnp5CQEOf9dvToUQ0ZMkRnz56tVA6g\npvACPcAFLy8vtWzZUiUlJVe94j4yMlKzZs1Sp06dJF0q1Pnz5+s//uM/9Oqrr6q0tFSjRo1Sp06d\ntG3btqv2GRkZqW+//Vbx8fHy8/NTaGjodc+NFxcXKzY2VtKlZ7peXl56/vnn1a1bt6t+LjQ0VA88\n8IDi4uJUv359eXt7a/z48ZIuHYpOTk52+czT4XDo6NGjio2NlcPhUEpKiho2bKiuXbuqU6dOuv/+\n+xUUFKS7777b+ZqBDh06aO7cuRo1atRVbx8cP368pkyZogcffFAXLlxQRESEnnjiCeft/Px2K5NH\nkgIDAxUWFqbg4OBKHxmRpNmzZ2vy5Ml67733dPHiRfXr108PPvigjhw5Im9vb61evVozZsyQt7e3\n5s+ff83158yZo8mTJ2vJkiVyOBxKTk5WQEBApXMANcHBErcA6qLs7GwNHDhQaWlpaty48Q3b75Ej\nRxQfH6/t27ffsH0CduMwPoA6Z9WqVerTp4+GDh16Q4v+suq8vx+ojXhmDwCA4XhmDwCA4Sh7AAAM\nV6dfjV/VxSwAAKirwsPDK32dOl32UtWGrk0yMzPr/AwSc9QmJswgmTGHCTNIzFGbVPVJLofxAQAw\nHGUPAIDhKHsAAAxH2QMAYDjKHgAAw1H2AAAYrs6/9Q4AUHNKSkp04MCBG7rPqq5ciIqj7AEAFXbg\nwAElJL2r+n5BN2R/hXkntWz6EIWEhNyQ/bmybds2rVixQnPmzKmR26stKHsAQKXU9wtSg4D/Z3eM\nKvs1rmpI2QMAar28vDyNHz9effv21eLFi+Xp6amgoCClpKQoNTVVR48e1enTp3XixAklJSWpS5cu\n2rBhg9LS0lRSUiKHw6HHH39c3t7ekqRz585p1KhR6tevn/r06aM5c+YoMzNTJSUlevTRR3Xfffdp\n/fr1Kioq0oABA2yevvooewBArefn56d58+bp2Wef1WOPPaZevXrpr3/9qwoKCiRJ9erV0+LFi/XF\nF1/o7bffVpcuXXTo0CEtXrxYXl5emjhxonbu3Km7775bZ8+e1RNPPKFhw4YpKipKW7Zs0bFjx5SW\nlqbi4mINHDhQXbp0UZ8+fWye+sah7AEAdca4ceO0aNEiLVu2TMHBwerRo4ck6be//a0kqUmTJjp/\n/rwkKTAwUGPHjpW3t7cOHjyogIAASZfO299+++0qLi6WJO3fv19ZWVkaOnSoLMtSSUmJjh49qtDQ\nUBsm/GVQ9gCAOmPlypUaNWqUAgMDNXHiRGVkZEi69jx8QUGB5s2bp82bN8uyLD366KPObVFRUXrp\npZf00EMPqUOHDrrtttt09913a/LkybIsSwsWLFDz5s1rdK5fGmUPAKiUwryTNb6vy+fs4+Pj9Yc/\n/EE+Pj7y8fFRVFSUli1bds3PN2jQQOHh4Ro4cKDc3d3l7++vnJwc5/bAwECNHj1aL774ot544w1t\n27ZNDz/8sIqKitSzZ0/Vr1/fqHP2DsuyLLtDVJUpyxXW9Rkk5qhNTJhBMmMOE2aQrp6jLr/P3oT7\no6oz8MweAFBh7u7uNfaeeNw4fFwuAACGo+wBADAcZQ8AgOEoewAADMcL9AAAFVaXX43/a0bZAwAq\n7MCBA3psSaJ8GvnekP2dPZWvN4bPse0V/rNnz1ZwcLBiYmIqdb3//M//1PLly/X000/rgQce+IXS\n3TiUPQCgUnwa+cq3qb/dMWz10Ucfae7cuWrTpo3dUSqEsgcA1HqXP0GvQYMGsixLJ06cUFFRkZKT\nk9WqVSvNmTNHu3fvVk5OjkJDQzVt2rRrVsPr37+/wsPDtWHDBr322msKDAxUcXGxgoODVVpaqokT\nJ+rHH3/UqVOnFBUVpWeffbbMLO+995727Nmjl156SSkpKVq5cuU1t52dna1x48bpzJkzkqSZM2cq\nMDBQL774ovLy8iRJ48ePV5s2bTRr1izdf//9uvPOO3+x3x9lDwCo9S6vepeUlKTmzZtrxowZ2rx5\ns2bOnKk///nP8vPz05tvvinLstS7d2+dPHnpY3ivXA1v7ty5euSRR5ScnKy1a9eqYcOGGjlypCTp\nxIkTat++vfr376/i4mJ169btumU/cOBArV+/XpMnT5afn1+Zt7148WL16NFDgwYN0o4dO7Rz507t\n3btXv//97zV48GAdPnxYSUlJevfdd/XCCy/84r8/yh4AUKd07txZktSxY0fNmDFDN910k3766SeN\nGTNG9evXV1FRkS5evCjp6tXwLly4oOzsbPn5+alhw4aSpA4dOki69GBi586d2rp1q3x8fHThwoVy\nM1iWJcuyrnvbBw8eVP/+/SVJ7du3V/v27ZWenq6tW7fqv/7rv2RZlvNZf02g7AEAdcru3bvVsWNH\nZWZmqk2bNtq8ebN+/PFHpaSkKDs7WxkZGbq87MvPV8O7+eablZ+fr5ycHAUEBGjXrl269dZb9cEH\nH8jPz0+TJ0/W4cOHtWrVqgpl2bJlS5m33bp1a+3cuVO33367tm/frs2bNys4OFhhYWHq3bu3srOz\ntXr16hv+u7keyh4AUClnT+XX+L6uPGe/ZcsWZWRkqLS0VDNmzJCXl5cWLlyohIQESVKzZs2ch/F/\nzt3dXRMmTNC///u/y9/fXx4el2rwnnvu0ZgxY7Rjxw55enqqZcuWOnnypIKCgsrcz+UHEXfddVeZ\ntz1y5Ei9+OKLSk9Pl5ubm6ZOnaoGDRropZde0ooVK3T27FmNGjVKkmrknD2r3tnMhBkk5qhNTJhB\nMmMOE2aQateqd0lJSerdu7e6du1a6dsx4f5g1TsAwC/u17Tq3aZNm/T22287n8VbliWHw6GhQ4eq\nZ8+eNqerHMoeAFBnTJ8+vcZu695779W9995bY7f3S+Kz8QEAMBxlDwCA4Sh7AAAMxzl7AECF2f1q\nfFRNnS/7/fv32x2hWkpKSuyOAAAVduDAAa0fNkK31q9/Q/Z3orBQfd55q8Kv8K/OW+8qKjU1VY0a\nNdKgQYMqfd2MjAy1a9dOjRo1+gWSVV2dL/vMJ5+2O0KVnSgsVMsXEtWpUye7owBAhd1av76aN7gx\nS9ya5p133tHkyZMp+xuNPzgAMN+Vn6CXlpamN954QyUlJZo2bZqaNWt23VXvvv76axUWFmrq1Kll\n7vfnq9MlJydLuvQM/cMPP1ReXp6eeeYZde/eXWlpadq4caPOnTungIAApaamat26dVqzZo0sy9LI\nkSO1d+9ejR07VmlpaRo/fryee+45NWnSpMZ+T9fDC/QAALXe5VXvpEsL4CxZskSPPfaYZs6cqbNn\nzzpXnluzZo127Njh/Ljc4OBgLV++XLfddluZ+124cKF69OihFStWaNy4cdq1a5ekSwvnLFmyRElJ\nSVq+fLkkKScnR++8845WrlypCxcuOH/Wz89PaWlpioyM1B133KGZM2fK09NTycnJtaLoJQOe2QMA\nfl3+7d/+TdKl0v/zn/9c7qp3rVq1KndfZa1Ol5qaqrZt20qSbrnlFhUVFUm6tFxuYmKivL29dfLk\nyTJv4/JqeLUNZQ8AqFN27typ9u3ba/v27S5XvXNzK/8Adlmr0910003XrJa3b98+ZWRk6L333tO5\nc+cUFxdX5m24ubmptLT0Bk9cfZQ9AKBSThQW1vi+rjxn/89//lPDhg2Tm5ubpk2bJk9Pzwqvevdz\nZa1Ot3bt2mt+rmXLlqpfv76GDBkiy7IUFBRU5m106NBBY8eO1VtvvaWpU6fWmnP2dX7Vu3OTp9kd\no8qOFOTL649PKC4uzu4o1WbCalKSGXOYMINkxhwmzCDVrlXvqsOE+4NV7wAAv7i6uurdqFGjdPTo\nUfn6XnoHl2VZatiwoebPn29zsppB2QMAjDdv3jwjntlXFW+9AwDAcJQ9AACGo+wBADAcZQ8AgOEo\newAADEfZAwBgOMoeAADDUfYAABiOsgcAwHCUPQAAhqPsAQAwHGUPAIDhKHsAAAxH2QMAYDjKHgAA\nw1H2AAAYzsPuANV1pCDf7ghVdqKwUM1LS+2OAQAwXJ0v+4WdPO2OUGWF2R4aa1l2xwAAGK7Ol32T\n9s3tjlBl+cdz5e7ubncMAIDhOGcPAIDhKHsAAAxH2QMAYDjKHgAAw1H2AAAYjrIHAMBwN6zsk5KS\n9Nlnn1VrH4MGDdLx48dvUCIAACDxzB4AAONVquzz8vI0evRoHT58WAkJCRo8eLAeffRRZWdnO3/m\n4sWLeuGFFzR48GANGjRIH374oSQpISFBBw8elCStWLFCqampkqSUlBTFx8frj3/8o3JzcyVJBw4c\n0OTJk2/IgAAA/NpV6hP0/Pz89Oqrr+qpp57SE088oS5duuiTTz7RN9984/yZlStX6uabb9asWbN0\n9uxZxcXFqXPnzmXuLysrS5mZmVqzZo0KCgp0//33S5KCg4M1ceLEaowFAAAuq9Jh/IMHD6pdu3aS\npKioKHXp0sW57cCBA/rd734nSfLx8VFwcLB++OGHq65v/d/nwR86dEhhYWGSpAYNGqhNmzZViQMA\nAMpRpc/Gb926tXbt2qV77rlH69atU15ennNbcHCw/vGPf6hnz54qKCjQt99+q9/85jfy8vLSqVOn\n1KpVK+3Zs0eNGzdW69at9e6770qSCgsL9d13392YqeqYzMxMuyPcEMxRe5gwg2TGHCbMIDFHXVep\nss/Ly9OECRP0pz/9SRMmTNCCBQtUv359zZo1S7t375YkDRw4UBMmTNCQIUN0/vx5Pf300woMDFRC\nQoImTZqkpk2bqnHjxpKk0NBQRUREKD4+Xo0aNdItt9wi6dLRgbS0tF/Nofzw8HC7I1RbZmYmc9QS\nJswgmTGHCTNIzFGbVPXBSpXO2UvSkiVLrto2ffp059czZsy45rqRkZGKjIy85vInn3xSTz755DWX\n/1qKHgCAXxpvvQMAwHCUPQAAhqPsAQAwHGUPAIDhKHsAAAxH2QMAYDjKHgAAw1H2AAAYrkofl1ub\n5B/PtTtClZ09la/jnse1f/9+u6NU2+HDh+Xr62t3jGozYQ4TZpDMmMOEGSRz5igpKbE7gm0c1uVV\naeqgzMxMPTUt3e4Y1eLt20gONw6wAMAvqTDvpMY+fJfi4uLsjlItVf3I3zr/zD6oZUe7IwAAUKvx\nlBIAAMNR9gAAGI6yBwDAcJQ9AACGo+wBADAcZQ8AgOEoewAADEfZAwBgOMoeAADDUfYAABiOsgcA\nwHCUPQAAhqPsAQAwHGUPAIDhKHsAAAxH2QMAYDjKHgAAw1H2AAAYjrIHAMBwlD0AAIaj7AEAMBxl\nDwCA4Sh7AAAM52F3gOoqyDlmdwQAQC1XmHdSUhO7Y9imzpf9onE97Y5QLVlZWQoLC7M7RrUxR+1h\nwgySGXOYMINkzhy5ubl2R7BNnS/7kJAQuyNUS35+fp2fQWKO2sSEGSQz5jBhBsmcOTIzM+2OYBvO\n2QMAYDjKHgAAw1H2AAAYjrIHAMBwlD0AAIaj7AEAMBxlDwCA4Sh7AAAMR9kDAGA4yh4AAMNR9gAA\nGI6yBwDAcJQ9AACGo+wBADAcZQ8AgOEoewAADEfZAwBgOMoeAADDUfYAABiOsgcAwHCUPQAAhqPs\nAQAwHGUPAIDhKHsAAAxH2QMAYDjKHgAAw1H2AAAYjrIHAMBwlD0AAIaj7AEAMBxlDwCA4Sh7AAAM\nR9kDAGA4yh4AAMNR9gAAGI6yBwDAcJQ9AACGo+wBADAcZQ8AgOEoewAADEfZAwBgOMoeAADDUfYA\nABiOsgcAwHCUPQAAhqPsAQAwHGUPAIDhKHsAAAxH2QMAYDjKHgAAw1H2AAAYjrIHAMBwlD0AAIaj\n7AEAMBxlDwCA4Sh7AAAMR9kDAGA4yh4AAMNR9gAAGI6yBwDAcJQ9AACGo+wBADAcZQ8AgOEoewAA\nDEfZAwBgOMoeAADDUfYAABiOsgcAwHCUPQAAhqPsAQAwHGUPAIDhKHsAAAxH2QMAYDjKHgAAw3nY\nHaC69u/fb3eEajl8+LB8fX3tjlFtzFF7mDCDZMYcJSUldkcAJBlQ9plPPm13hGrLtDvADcIctYcJ\nM0h1e44ThYVq+UKiOnXqZHcUoO6XffMGdfuRPwAAvzTO2QMAYDjKHgAAw1H2AAAYjrIHAMBwlD0A\nAIaj7AEAMBxlDwCA4Sh7AAAMR9kDAGA4yh4AAMNR9gAAGI6yBwDAcJQ9AACGo+wBADAcZQ8AgOEo\newAADEfZAwBgOMoeAADDUfYAABiOsgcAwHCUPQAAhqPsAQAwnIfdAarrSEG+3REA4BonCgvV0u4Q\nwP+p82W/sJOn3RFgiMLssxp+Z3/16tXL7ijVkpWVpbCwMLtjVJsJc+Tm5todAZBkQNk3ad/c7ggw\nRP7xXDVt2lQhISF2R6mW/Pz8Oj+DZMYcmZmZdkcAJHHOHgAA41H2AAAYjrIHAMBwlD0AAIaj7AEA\nMBxlDwCA4Sh7AAAMR9kDAGA4yh4AAMO5/AS9vLw8jR8/Xn379tXixYvl6empoKAgpaSkKDU1VY0a\nNdKgQYP0/fff6+WXX9ayZcvUt29fderUSfv27ZPD4dCCBQu0Z88e5/WPHj2q3r17a+TIkbrvvvu0\nevVqNWzYUMuXL1dhYaHatm2r//mf/9FTTz1VE78DAACM5vKZvZ+fn+bNm6e//e1veuyxx5SWlqbu\n3bsrP//aBWgcDockqaCgQA8++KCWLVumoKAgbdmyRZJ04sQJzZ8/XytXrtTixYvlcDjUt29f/e1v\nf5MkpaenKzY2Vp07d6boAQC4QSr82fjjxo3TokWLtGzZMgUHB6tHjx5Xbbcs66rv77jjDknSrbfe\nquLiYklSSEiIHA6HvL29ddNNN0mS4uLilJiYqN/97ndq1KiRAgMDqzUQUF0mfJ65CTNIZsxhwgwS\nc9R1FS77lStXatSoUQoMDNTEiROVkZEhLy8vnTx5UpK0e/fuKgVo2rSpfH199dprryk+Pr5K+wBu\npPDwcLsjVEtmZmadn0EyYw4TZpCYozap6oMVl4fx8/LyNGrUKLVr105/+MMfNHz4cJ0+fVpRUVF6\n4IEHtHnzZg0dOlTffPON8zqXD+eX9/WVBg4cqMzMTHXr1k2S9NVXX2nBggVVGggAAFzN5TP7y+fs\nJal79+7XbFu9evU11/n444+dXycmJjq/7tSpk/Przz77zPl1SUmJ4uPjnQ8GOnfurM6dO1dwBAAA\nUB7b17NPSUnR1q1btWjRIrujAABgJNvL/rnnnrM7AgAARuNDdQAAMBxlDwCA4Sh7AAAMR9kDAGA4\nyh4AAMNR9gAAGM72t95VV/7xXLsjwBBnT+VLLexOAQA3Xp0v++zPQuyOUGVF+dka2fd29erVy+4o\n1ZaVlaWwsDC7Y1Rbbi4PHgGYp86XfVDLjnZHqLKCnGNq2rSJQkLq7gOWy/Lz842Y49e6IhYAs3HO\nHgAAw1H2AAAYjrIHAMBwlD0AAIaj7AEAMBxlDwCA4Sh7AAAMR9kDAGA4yh4AAMNR9gAAGI6yBwDA\ncJQ9AACGo+wBADAcZQ8AgOEoewAADEfZAwBgOMoeAADDUfYAABiOsgcAwHCUPQAAhqPsAQAwHGUP\nAIDhPOwOUF0FOcfsjlBlhXknJTWxOwYAwHB1vuwXjetpd4Rqyc3NtTsCAMBwdb7sQ0JC7I5QLZmZ\nmXZHAAAYjnP2AAAYjrIHAMBwlD0AAIaj7AEAMBxlDwCA4Sh7AAAMR9kDAGA4yh4AAMNR9gAAGI6y\nBwDAcJQ9AACGo+wBADAcZQ8AgOEoewAADEfZAwBgOMoeAADDUfYAABiOsgcAwHCUPQAAhqPsAQAw\nHGUPAIDhKHsAAAxH2QMAYDjKHgAAw1H2AAAYjrIHAMBwlD0AAIaj7AEAMBxlDwCA4Sh7AAAMR9kD\nAGA4yh4AAMNR9gAAGI6yBwDAcJQ9AACGo+wBADAcZQ8AgOEclmVZdoeoqszMTLsjAABQo8LDwyt9\nnTpd9gAAwDUO4wMAYDjKHgAAw1H2AAAYjrIHAMBwlD0AAIbzsDuAK5ZladKkSdq3b5/q1aunqVOn\nqlmzZs7tmzZt0oIFC+Th4aH4+HgNGDDAxrTX52oOSSoqKtKIESM0bdo0tWrVyqak1+dqhvXr12vp\n0qXy8PBQSEiIJk2aZF/YcriaY8OGDVq8eLHc3NzUp08fDR061Ma0ZavI35MkTZw4Uf7+/kpMTLQh\npWuu5liyZIlWr16twMBASdLkyZPVsmVLm9Jen6s5du7cqeTkZEnSLbfcolmzZqlevXp2xS1TeTP8\n9NNPeu655+RwOGRZlvbu3avnn39egwYNsjn1tVzdF+np6VqyZInc3d0VFxenhx56yMa0ZXM1w9q1\na/XWW29kiew0AAAFyElEQVSpYcOGiomJUf/+/Su001pt48aN1rhx4yzLsqwdO3ZYTz75pHPbhQsX\nrOjoaCs/P98qLi624uPjrdOnT9sVtVzlzWFZlrVr1y4rLi7O6tKli/X999/bEdGl8mY4d+6cFR0d\nbZ0/f96yLMtKTEy0Nm3aZEtOV8qbo6SkxOrVq5dVUFBglZSUWPfdd5+Vk5NjV9TrcvX3ZFmWtXz5\ncmvQoEHW7Nmzazpehbma4/nnn7d2795tR7RKcTVHv379rCNHjliWZVmrVq2yDh48WNMRXarI35Rl\nWdbXX39tDRs2zCotLa3JeBXmao4uXbpYZ86csYqLi63o6GjrzJkzdsQsV3kzZGdnW1FRUdaZM2es\n0tJSa+jQodaxY8dc7rPWH8bPzMxURESEJKldu3bKyspybjtw4IBatGihBg0ayNPTU+Hh4dq+fbtd\nUctV3hySdOHCBS1YsEC33XabHfEqpLwZ6tWrpxUrVjifrVy8eFFeXl625HSlvDnc3Nz04YcfysfH\nRzk5ObIsS56ennZFvS5Xf09ff/21du3apcGDB9sRr8JczbF7924tWrRIQ4YM0euvv25HxAopb46D\nBw/K399fb7/9thISEpSXl1crj064ui8umzJlil555RU5HI6ajFdhruYIDQ1VXl6ezp8/L0m1co7y\nZvjhhx90xx13yNfXVw6HQ3feead27Njhcp+1vuwLCgrk6+vr/N7Dw0OlpaVlbvPx8VF+fn6NZ6yI\n8uaQpA4dOqhx48ayavFnHJU3g8PhcB5qXbZsmYqKivT73//elpyuuLov3Nzc9NFHH6lfv37q1KmT\n6tevb0fMcpU3w6lTp5SamqqJEyfW6r8nyfV90bt3b73yyitaunSpMjMztXnzZjtiulTeHDk5Odqx\nY4cSEhL09ttv64svvtDWrVvtinpdru4L6dJp05CQELVo0aKm41WYqznatGmj+Ph4Pfjgg+revbsa\nNGhgR8xylTdDy5Yt9d133yk7O1tFRUX68ssvVVRU5HKftb7sGzRooLNnzzq/Ly0tlZubm3NbQUGB\nc9vZs2fVsGHDGs9YEeXNUVe4msGyLCUnJ+vLL79UamqqHRErpCL3RXR0tD777DMVFxdr7dq1NR3R\npfJm+Pvf/67c3Fw9/vjjev3117V+/fpaOYPk+r4YNmyY/P395eHhocjISO3Zs8eOmC6VN4e/v7+a\nN2+uVq1aycPDQxEREdd91mynivy7SE9P18CBA2s6WqWUN8e+ffv03//939q0aZM2bdqk06dPa8OG\nDXZFva7yZmjYsKHGjRunUaNG6fnnn1fbtm0VEBDgcp+1vm06duzofDS/Y8cOhYSEOLcFBwfr8OHD\nOnPmjIqLi7V9+3a1b9/erqjlKm+OusLVDBMmTHCejqhtLz66UnlzFBQUKCEhQcXFxZIkb2/vWnmY\nr7wZEhIStGbNGi1dulQjR45Unz59FBMTY1fUcrm6L/r06aOioiJZlqWvvvpKbdu2tStqucqbo1mz\nZiosLNQPP/wg6dIh2tatW9uSszwV+T8qKytLHTp0qOlolVLeHL6+vvL29la9evWcRyPPnDljV9Tr\nKm+GkpIS7d69W2lpaUpJSdHBgwfVsWNHl/us9a/Gj46O1ueff+489zh9+nStX79eRUVFGjBggJKS\nkjRixAhZlqUBAwYoKCjI5sRlczXHZbWxWC4rb4a2bdvq/fffV3h4uBISEuRwODR06FD17NnT5tTX\ncnVf9O3bV4888og8PT11++23q1+/fjYnvlZF/55qO1dzJCYmKiEhQV5eXrrnnnvUrVs3mxOXzdUc\nU6dOdb4jokOHDoqMjLQzbplczZCdnX3VoeXaytUcAwcO1JAhQ1SvXj01b95csbGxNie+VkX+fcfG\nxsrLy0sjRoyQv7+/y32yEA4AAIar9YfxAQBA9VD2AAAYjrIHAMBwlD0AAIaj7AEAMBxlDwCA4Sh7\nAAAMR9kDAGC4/wUqodKhIyj4ZAAAAABJRU5ErkJggg==\n", 796 | "text/plain": [ 797 | "" 798 | ] 799 | }, 800 | "metadata": {}, 801 | "output_type": "display_data" 802 | } 803 | ], 804 | "source": [ 805 | "pd.DataFrame(topics, columns=boring_vocabulary, index=boring_topics).plot(kind='barh')\n", 806 | "_=plt.title(\"Word Distribution by Topic\")" 807 | ] 808 | }, 809 | { 810 | "cell_type": "markdown", 811 | "metadata": { 812 | "slideshow": { 813 | "slide_type": "slide" 814 | } 815 | }, 816 | "source": [ 817 | " ➡ Forward Sampling\n", 818 | "Let's generate some documents!" 819 | ] 820 | }, 821 | { 822 | "cell_type": "code", 823 | "execution_count": 20, 824 | "metadata": { 825 | "collapsed": false, 826 | "slideshow": { 827 | "slide_type": "-" 828 | } 829 | }, 830 | "outputs": [ 831 | { 832 | "name": "stdout", 833 | "output_type": "stream", 834 | "text": [ 835 | "Document 01 (topic ☀): 🐍🐼🐼🐼🐼🐼🐼📊📊📊📊📊📊📊📊\n", 836 | "Document 02 (topic ☀): 🐍🐼🐼📊📊📊📊📊📊📊📊\n", 837 | "Document 03 (topic ☁): 🐍🐍🐍🐍🐼\n", 838 | "Document 04 (topic ☁): 🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍\n", 839 | "Document 05 (topic ☁): 🐍🐍🐍🐍🐍🐍🐼🐼\n", 840 | "Document 06 (topic ☁): 🐍🐍🐍🐍🐍🐍🐍🐼🐼📊\n", 841 | "Document 07 (topic ☀): 🐍🐼🐼📊📊📊📊📊📊📊📊\n", 842 | "Document 08 (topic ☁): 🐍🐍🐍🐍🐍🐍🐍🐼🐼\n", 843 | "Document 09 (topic ☁): 🐍🐍🐍🐍🐍🐍🐍🐍🐼🐼🐼📊\n", 844 | "Document 10 (topic ☁): 🐍🐍🐍🐍🐍🐍🐍🐍📊\n" 845 | ] 846 | } 847 | ], 848 | "source": [ 849 | "for doc_index in range(1, 11):\n", 850 | " topic = random.choice(topic_labels, p=mixture_proportion)\n", 851 | " doc_len = poisson(mu=10).rvs(size=1)\n", 852 | " doc = sorted([random.choice(vocabulary, \n", 853 | " p=topics[topic_labels.index(topic)]) \n", 854 | " for _ in range(doc_len)])\n", 855 | " print(\"Document {:02d} (topic {}): {}\".format(doc_index, topic, ''.join(doc)))" 856 | ] 857 | }, 858 | { 859 | "cell_type": "markdown", 860 | "metadata": { 861 | "slideshow": { 862 | "slide_type": "slide" 863 | } 864 | }, 865 | "source": [ 866 | " \"\"\"\n", 867 | " ⬅ Reversing the Process\n", 868 | " \"\"\"\n", 869 | "\n", 870 | "Given a collection of documents...\n", 871 | "\n", 872 | "...find the topic for each document\n", 873 | "\n", 874 | "...find the distribution over topics\n", 875 | "\n", 876 | "...find the distribution over terms for each topic" 877 | ] 878 | }, 879 | { 880 | "cell_type": "markdown", 881 | "metadata": { 882 | "slideshow": { 883 | "slide_type": "slide" 884 | } 885 | }, 886 | "source": [ 887 | " \"\"\"\n", 888 | " ⬅ Limitations of this Model\n", 889 | " \"\"\"\n", 890 | " \n", 891 | "* __Single topic per document.__\n", 892 | "* Word order irrelevant. (Bag-of-words assumption!)\n", 893 | "* Assumes we know # of topics!" 894 | ] 895 | }, 896 | { 897 | "cell_type": "markdown", 898 | "metadata": { 899 | "slideshow": { 900 | "slide_type": "slide" 901 | } 902 | }, 903 | "source": [ 904 | "### Excursis: Bag of Words Assumption" 905 | ] 906 | }, 907 | { 908 | "cell_type": "code", 909 | "execution_count": 21, 910 | "metadata": { 911 | "collapsed": false 912 | }, 913 | "outputs": [ 914 | { 915 | "data": { 916 | "text/plain": [ 917 | "'a a although although although ambiguity and are arent at bad be be be beats beautiful better better better better better better better better break cases complex complex complicated counts dense do do dutch easy enough errors explain explain explicit explicitly face first flat good great guess hard honking idea idea idea if if implementation implementation implicit in is is is is is is is is is is it it its lets may may more namespaces nested never never never not now now obvious obvious of of often one one one only pass practicality preferably purity readability refuse right rules should should silenced silently simple sparse special special temptation than than than than than than than than that the the the the the there those to to to to to ugly unless unless way way youre'" 918 | ] 919 | }, 920 | "execution_count": 21, 921 | "metadata": {}, 922 | "output_type": "execute_result" 923 | } 924 | ], 925 | "source": [ 926 | "import collections\n", 927 | "this = 'beautiful is better than ugly explicit is better than implicit simple is better than complex complex is better than complicated flat is better than nested sparse is better than dense readability counts special cases arent special enough to break the rules although practicality beats purity errors should never pass silently unless explicitly silenced in the face of ambiguity refuse the temptation to guess there should be one and preferably only one obvious way to do it although that way may not be obvious at first unless youre dutch now is better than never although never is often better than right now if the implementation is hard to explain its a bad idea if the implementation is easy to explain it may be a good idea namespaces are one honking great idea lets do more of those'\n", 928 | "' '.join(sorted(this.split(' ')))" 929 | ] 930 | }, 931 | { 932 | "cell_type": "markdown", 933 | "metadata": { 934 | "slideshow": { 935 | "slide_type": "fragment" 936 | } 937 | }, 938 | "source": [ 939 | "```\n", 940 | "➡ What if we sample the topic for each word?\n", 941 | "```" 942 | ] 943 | }, 944 | { 945 | "cell_type": "code", 946 | "execution_count": 22, 947 | "metadata": { 948 | "collapsed": false, 949 | "slideshow": { 950 | "slide_type": "slide" 951 | } 952 | }, 953 | "outputs": [], 954 | "source": [ 955 | "# recycle our `topics` sampled from the Dirichlet distribution\n", 956 | "\n", 957 | "docs = defaultdict(lambda: defaultdict(str))\n", 958 | "for doc_index in range(10):\n", 959 | " doc_len = poisson(mu=10).rvs(size=1)\n", 960 | " mixture_proportion = dirichlet([0.5, 0.5]).rvs()[0]\n", 961 | " \n", 962 | " for word in range(doc_len):\n", 963 | " topic = random.choice(topic_labels, \n", 964 | " p=mixture_proportion)\n", 965 | " word_dist = topics[topic_labels.index(topic)]\n", 966 | " word = random.choice(vocabulary, \n", 967 | " p=word_dist)\n", 968 | " docs[doc_index+1][topic] += word\n", 969 | " \n", 970 | " docs[doc_index+1][topic] = ''.join(sorted(docs[doc_index+1][topic]))" 971 | ] 972 | }, 973 | { 974 | "cell_type": "markdown", 975 | "metadata": {}, 976 | "source": [ 977 | "```\n", 978 | "⬆ Generative model for latent dirichlet allocation\n", 979 | "```" 980 | ] 981 | }, 982 | { 983 | "cell_type": "code", 984 | "execution_count": 23, 985 | "metadata": { 986 | "collapsed": false, 987 | "slideshow": { 988 | "slide_type": "slide" 989 | } 990 | }, 991 | "outputs": [ 992 | { 993 | "name": "stdout", 994 | "output_type": "stream", 995 | "text": [ 996 | "Doc 01\n", 997 | " Words from topic ☀: 🐍🐍🐍🐼🐼🐼📊📊📊📊📊📊\n", 998 | " Words from topic ☁: 🐍\n", 999 | "Doc 02\n", 1000 | " Words from topic ☀: 🐍🐼🐼📊📊📊📊📊\n", 1001 | "Doc 03\n", 1002 | " Words from topic ☀: 🐍🐼🐼🐼📊📊📊📊📊\n", 1003 | " Words from topic ☁: 🐍🐍\n", 1004 | "Doc 04\n", 1005 | " Words from topic ☀: 🐍📊📊\n", 1006 | "Doc 05\n", 1007 | " Words from topic ☀: 🐍🐼📊\n", 1008 | " Words from topic ☁: 🐍🐍🐍🐍🐍🐍🐼\n", 1009 | "Doc 06\n", 1010 | " Words from topic ☀: 📊\n", 1011 | " Words from topic ☁: 🐍🐍🐍🐍🐍🐍🐍🐍📊\n", 1012 | "Doc 07\n", 1013 | " Words from topic ☀: 🐼\n", 1014 | " Words from topic ☁: 🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐍🐼🐼\n", 1015 | "Doc 08\n", 1016 | " Words from topic ☀: 🐍🐍🐼📊📊📊📊📊📊📊📊\n", 1017 | "Doc 09\n", 1018 | " Words from topic ☁: 🐍🐍🐍🐍🐍🐍🐍🐍🐍🐼🐼\n", 1019 | "Doc 10\n", 1020 | " Words from topic ☀: 🐍🐍🐼🐼📊📊📊📊📊📊📊\n", 1021 | " Words from topic ☁: 🐍🐍🐍🐍🐍🐍📊\n" 1022 | ] 1023 | } 1024 | ], 1025 | "source": [ 1026 | "for doc_id, doc in docs.items():\n", 1027 | " print(\"Doc {:02d}\".format(doc_id))\n", 1028 | " for topic, words in doc.items():\n", 1029 | " print(\" Words from topic {}: {}\".format(topic, words))" 1030 | ] 1031 | }, 1032 | { 1033 | "cell_type": "markdown", 1034 | "metadata": { 1035 | "slideshow": { 1036 | "slide_type": "slide" 1037 | } 1038 | }, 1039 | "source": [ 1040 | "### Excursis: Multi-topic documents" 1041 | ] 1042 | }, 1043 | { 1044 | "cell_type": "markdown", 1045 | "metadata": {}, 1046 | "source": [ 1047 | "![](static/buffalo.png)\n", 1048 | "\n", 1049 | "[source](https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffalo_buffalo_buffalo_Buffalo_buffalo)" 1050 | ] 1051 | }, 1052 | { 1053 | "cell_type": "markdown", 1054 | "metadata": { 1055 | "slideshow": { 1056 | "slide_type": "slide" 1057 | } 1058 | }, 1059 | "source": [ 1060 | " \"\"\"\n", 1061 | " ⬅ Reversing the Process\n", 1062 | " \"\"\"\n", 1063 | "\n", 1064 | "Given a collection of documents...\n", 1065 | "\n", 1066 | "Given each word in each document...\n", 1067 | "\n", 1068 | "...find the topic for each __word__\n", 1069 | "\n", 1070 | "...find the distribution over topics for each __document__\n", 1071 | "\n", 1072 | "...find the distribution over terms for each topic" 1073 | ] 1074 | }, 1075 | { 1076 | "cell_type": "markdown", 1077 | "metadata": { 1078 | "slideshow": { 1079 | "slide_type": "fragment" 1080 | } 1081 | }, 1082 | "source": [ 1083 | "------\n", 1084 | "You can do this with MCMC (Gibbs sampling), variational inference, etc. " 1085 | ] 1086 | }, 1087 | { 1088 | "cell_type": "markdown", 1089 | "metadata": { 1090 | "slideshow": { 1091 | "slide_type": "slide" 1092 | } 1093 | }, 1094 | "source": [ 1095 | " \"\"\"\n", 1096 | " ⬅ Limitations of this Model\n", 1097 | " \"\"\"\n", 1098 | "\n", 1099 | "* Single topic per document.\n", 1100 | "* Word order irrelevant. \n", 1101 | " * (See Hanna Wallach's work for possible ideas.)\n", 1102 | "* Assumes we know # of topics!\n", 1103 | " * (See hierarchical Dirichlet process models ideas.)" 1104 | ] 1105 | }, 1106 | { 1107 | "cell_type": "markdown", 1108 | "metadata": { 1109 | "slideshow": { 1110 | "slide_type": "slide" 1111 | } 1112 | }, 1113 | "source": [ 1114 | "### Excursis: Bag of Words Assumption" 1115 | ] 1116 | }, 1117 | { 1118 | "cell_type": "markdown", 1119 | "metadata": { 1120 | "slideshow": { 1121 | "slide_type": "-" 1122 | } 1123 | }, 1124 | "source": [ 1125 | "![](static/blei-f1.jpg)\n", 1126 | "\n", 1127 | "[source](http://cacm.acm.org/magazines/2012/4/147361-probabilistic-topic-models/fulltext)" 1128 | ] 1129 | }, 1130 | { 1131 | "cell_type": "markdown", 1132 | "metadata": { 1133 | "slideshow": { 1134 | "slide_type": "fragment" 1135 | } 1136 | }, 1137 | "source": [ 1138 | "(There are more.)" 1139 | ] 1140 | }, 1141 | { 1142 | "cell_type": "markdown", 1143 | "metadata": { 1144 | "collapsed": true, 1145 | "slideshow": { 1146 | "slide_type": "slide" 1147 | } 1148 | }, 1149 | "source": [ 1150 | "# Questions?\n", 1151 | "\n", 1152 | "\n", 1153 | "-------\n", 1154 | "\n", 1155 | "* Tim Hopper\n", 1156 | "* [@tdhopper](https://twitter.com/tdhopper)\n", 1157 | "* [stiglerdiet.com](http://www.stiglerdiet.com)\n", 1158 | "\n", 1159 | "------\n", 1160 | "\n", 1161 | "Read More:\n", 1162 | "\n", 1163 | "* [Probabilistic Topic Models](http://cacm.acm.org/magazines/2012/4/147361-probabilistic-topic-models/fulltext) by David Blei, 2012\n", 1164 | "* [Introduction to Latent Dirichlet Allocation](http://blog.echen.me/2011/08/22/introduction-to-latent-dirichlet-allocation/) by Edwin Chen\n", 1165 | "* [lda package](https://github.com/ariddell/lda) by Allen Riddell\n", 1166 | "* [pyLDAvis package](https://pyldavis.readthedocs.org/en/latest/) by Ben Mabey [(example notebook)](http://nbviewer.ipython.org/github/bmabey/pyLDAvis/blob/master/notebooks/pyLDAvis_overview.ipynb)\n", 1167 | "* [Finding Scientific Topics](http://psiexp.ss.uci.edu/research/papers/sciencetopics.pdf) by Thomas L. Griffiths and Mark Steyvers\n", 1168 | "\n", 1169 | "-----\n", 1170 | "\n", 1171 | "\n" 1172 | ] 1173 | } 1174 | ], 1175 | "metadata": { 1176 | "anaconda-cloud": {}, 1177 | "celltoolbar": "Slideshow", 1178 | "kernelspec": { 1179 | "display_name": "Python [understanding-lda]", 1180 | "language": "python", 1181 | "name": "Python [understanding-lda]" 1182 | }, 1183 | "language_info": { 1184 | "codemirror_mode": { 1185 | "name": "ipython", 1186 | "version": 3 1187 | }, 1188 | "file_extension": ".py", 1189 | "mimetype": "text/x-python", 1190 | "name": "python", 1191 | "nbconvert_exporter": "python", 1192 | "pygments_lexer": "ipython3", 1193 | "version": "3.5.2" 1194 | } 1195 | }, 1196 | "nbformat": 4, 1197 | "nbformat_minor": 0 1198 | } 1199 | --------------------------------------------------------------------------------