├── LICENSE
├── README.md
├── Chapter 6 - Multiple Random Variables.ipynb
├── Chapter 1 - Joint and Conditional Probability.ipynb
├── Chapter 3 - PMF, PDF and CDF.ipynb
└── Chapter 2 - Bayes' Theorem.ipynb


/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2020 WeijieChen-MacroAnalyst
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | [Last updated in Jan 2021]
 2 | # Probability Theory
 3 | A Quick Introduction to Probability Theory. This tutorial aims to refresh all most important concepts in probability theory with simple examples. Probability theory is the key foundation of statistics, econometrics and data science. Make you understand all the details in the example, use these examples as your references when encounter new problems.
 4 | 
 5 | ## Prerequiste
 6 | Since this is not an academic course, there is no sophisticated derivation, only practical examples, you need don't any prerequiste except freshman level mathematics. 
 7 | 
 8 | ## Content (updating)
 9 | [Lecture 1 - Joint and Conditional Probability](https://nbviewer.jupyter.org/github/WeijieChen-MacroAnalyst/Probability_Theory/blob/master/Chapter%201%20-%20Joint%20and%20Conditional%20Probability.ipynb)<br>
10 | [Lecture 2 - Bayes' Theorem](https://github.com/WeijieChen-MacroAnalyst/Probability_Theory/blob/master/Chapter%202%20-%20Bayes'%20Theorem.ipynb)<br>
11 | [Lecture 3 - PMF, PDF and CDF](https://nbviewer.jupyter.org/github/MacroAnalyst/Probability_Theory/blob/master/Chapter%203%20-%20PMF%2C%20PDF%20and%20CDF.ipynb)<br>
12 | [Lecture 4 - Important Distributions](https://nbviewer.jupyter.org/github/MacroAnalyst/Probability_Theory/blob/master/Chapter%204%20-%20Most%20Important%20Discrete%20and%20Continuous%20Distributions.ipynb)<br>
13 | [Lecture 5 - Joint Distributions](https://nbviewer.jupyter.org/github/MacroAnalyst/Probability_Theory/blob/master/Chapter%205%20-%20Joint%20Distributions.ipynb)<br>
14 | 


--------------------------------------------------------------------------------
/Chapter 6 - Multiple Random Variables.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "markdown",
  5 |    "metadata": {
  6 |     "toc": true
  7 |    },
  8 |    "source": [
  9 |     "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
 10 |     "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#-Joint-Distribution-and-Independence\" data-toc-modified-id=\"-Joint-Distribution-and-Independence-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Joint Distribution and Independence</font></a></span><ul class=\"toc-item\"><li><span><a href=\"#An-Example\" data-toc-modified-id=\"An-Example-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\">An Example</font></a></span></li><li><span><a href=\"#-Independence\" data-toc-modified-id=\"-Independence-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Independence</font></a></span></li><li><span><a href=\"#-Moment-Generating-Functions\" data-toc-modified-id=\"-Moment-Generating-Functions-1.3\"><span class=\"toc-item-num\">1.3&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Moment Generating Functions</font></a></span><ul class=\"toc-item\"><li><span><a href=\"#-How-It-Works?\" data-toc-modified-id=\"-How-It-Works?-1.3.1\"><span class=\"toc-item-num\">1.3.1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> How It Works?</font></a></span></li><li><span><a href=\"#-Example-1---Moments-of-Bernoulli\" data-toc-modified-id=\"-Example-1---Moments-of-Bernoulli-1.3.2\"><span class=\"toc-item-num\">1.3.2&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 1 - Moments of Bernoulli</font></a></span></li><li><span><a href=\"#-Example-2---Moments-of-Binomial-\" data-toc-modified-id=\"-Example-2---Moments-of-Binomial--1.3.3\"><span class=\"toc-item-num\">1.3.3&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 2 - Moments of Binomial </font></a></span></li></ul></li></ul></li></ul></div>"
 11 |    ]
 12 |   },
 13 |   {
 14 |    "cell_type": "markdown",
 15 |    "metadata": {},
 16 |    "source": [
 17 |     "Here we go through the methods for distributions which have more than two variables. Most of results are similar to us already. "
 18 |    ]
 19 |   },
 20 |   {
 21 |    "cell_type": "markdown",
 22 |    "metadata": {},
 23 |    "source": [
 24 |     "# <font face=\"gotham\" color=\"purple\"> Joint Distribution and Independence</font>"
 25 |    ]
 26 |   },
 27 |   {
 28 |    "cell_type": "markdown",
 29 |    "metadata": {},
 30 |    "source": [
 31 |     "Let $X_1, X_2,...X_n$ be discrete random variables, the PMF is defined as\n",
 32 |     "\n",
 33 |     "$$\n",
 34 |     "P_{X_{1}, X_{2}, \\ldots, X_{n}}\\left(x_{1}, x_{2}, \\ldots, x_{n}\\right)=P\\left(X_{1}=x_{1}, X_{2}=x_{2}, \\ldots, X_{n}=x_{n}\\right)\n",
 35 |     "$$"
 36 |    ]
 37 |   },
 38 |   {
 39 |    "cell_type": "markdown",
 40 |    "metadata": {},
 41 |    "source": [
 42 |     "The marginal PMF of $x_1$ is\n",
 43 |     "\n",
 44 |     "$$\n",
 45 |     "P_{X_1}=P(X=x_1)\n",
 46 |     "= \\sum_{x_2\\in R}\\sum_{x_3\\in R}...\\sum_{x_n\\in R}P\\left(X_{1}=x_{1}, X_{2}=x_{2}, \\ldots, X_{n}=x_{n}\\right)\n",
 47 |     "$$"
 48 |    ]
 49 |   },
 50 |   {
 51 |    "cell_type": "markdown",
 52 |    "metadata": {},
 53 |    "source": [
 54 |     "If $X_1, X_2,...X_n$ are continuous random variables, the PDF is defined as\n",
 55 |     "\n",
 56 |     "$$\n",
 57 |     "P\\left(\\left(X_{1}, X_{2}, \\cdots, X_{n}\\right) \\in A\\right)=\\int_A \\int_{A} \\cdots \\int_A f_{X_{1} X_{2} \\cdots X_{n}}\\left(x_{1}, x_{2}, \\cdots, x_{n}\\right) d x_{1} d x_{2} \\cdots d x_{n}\n",
 58 |     "$$"
 59 |    ]
 60 |   },
 61 |   {
 62 |    "cell_type": "markdown",
 63 |    "metadata": {},
 64 |    "source": [
 65 |     "The marginal PDF of $x_1$ is\n",
 66 |     "\n",
 67 |     "$$\n",
 68 |     "f_{X_{1}}\\left(x_{1}\\right)=\\int_{-\\infty}^{\\infty} \\cdots \\int_{-\\infty}^{\\infty} f_{X_{1} X_{2} \\cdots X_{n}}\\left(x_{1}, x_{2}, \\ldots, x_{n}\\right) d x_{2} \\cdots d x_{n}\n",
 69 |     "$$"
 70 |    ]
 71 |   },
 72 |   {
 73 |    "cell_type": "markdown",
 74 |    "metadata": {},
 75 |    "source": [
 76 |     "The joint CDF is \n",
 77 |     "\n",
 78 |     "$$\n",
 79 |     "F_{X_{1}, X_{2}, \\ldots, X_{n}}\\left(x_{1}, x_{2}, \\ldots, x_{n}\\right)=P\\left(X_{1} \\leq x_{1}, X_{2} \\leq x_{2}, \\ldots, X_{n} \\leq x_{n}\\right)\n",
 80 |     "$$"
 81 |    ]
 82 |   },
 83 |   {
 84 |    "cell_type": "markdown",
 85 |    "metadata": {},
 86 |    "source": [
 87 |     "## <font face=\"gotham\" color=\"purple\">An Example</font>"
 88 |    ]
 89 |   },
 90 |   {
 91 |    "cell_type": "markdown",
 92 |    "metadata": {},
 93 |    "source": [
 94 |     "Let $X, Y$ and $Z$ be three jointly continuous random variables with joint PDF\n",
 95 |     "\n",
 96 |     "$$\n",
 97 |     "f_{X Y Z}(x, y, z)=\\left\\{\\begin{array}{ll}\n",
 98 |     "cx+2 y+3 z & 0 \\leq x, y, z \\leq 1 \\\\\n",
 99 |     "0 & \\text { otherwise }\n",
100 |     "\\end{array}\\right.\n",
101 |     "$$"
102 |    ]
103 |   },
104 |   {
105 |    "cell_type": "markdown",
106 |    "metadata": {},
107 |    "source": [
108 |     "1. Find out $c$.\n",
109 |     "\n",
110 |     "Use the property we get \n",
111 |     "\n",
112 |     "\\begin{align}\n",
113 |     "\\int_0^1\\int_0^1\\int_0^1(cx+2y+3z)dxdydz &= 1\\\\\n",
114 |     "\\int_0^1\\int_0^1\\left[\\frac{1}{2}cx^2+2xy+3xz\\right]_0^1dydz&=1\\\\\n",
115 |     "\\int_0^1\\int_0^1\\left(\\frac{1}{2}+2y+3z\\right)dydz&=1\\\\\n",
116 |     "\\int_0^1\\left[\\frac{cy}{2}+y^2+3zy\\right]^1_0dz&=1\\\\\n",
117 |     "\\left[\\frac{cz}{2}+z+\\frac{3z^2}{2}\\right]^1_0&=1\\\\\n",
118 |     "c&=-3\n",
119 |     "\\end{align}"
120 |    ]
121 |   },
122 |   {
123 |    "cell_type": "markdown",
124 |    "metadata": {},
125 |    "source": [
126 |     "2. Find the marginal PDF of $X$."
127 |    ]
128 |   },
129 |   {
130 |    "cell_type": "markdown",
131 |    "metadata": {},
132 |    "source": [
133 |     "\\begin{align}\n",
134 |     "f_X(x)=\\int_{-\\infty}^\\infty\\int_{-\\infty}^\\infty f_{XYZ}(x,y,z)dydz & = \\int_{0}^1\\int_{0}^1 (-3x+2 y+3 z )dydz \\\\\n",
135 |     "& = \\int_0^1 \\left[-3xy+y^2+3yz\\right]_0^1dz =\\int_0^1(-3x+1+3z)dz\\\\\n",
136 |     "& = \\left[-3xz+z+3\\frac{z^2}{2}\\right]^1_0 = -3x+\\frac{5}{2}\n",
137 |     "\\end{align}"
138 |    ]
139 |   },
140 |   {
141 |    "cell_type": "markdown",
142 |    "metadata": {},
143 |    "source": [
144 |     "## <font face=\"gotham\" color=\"purple\"> Independence</font>"
145 |    ]
146 |   },
147 |   {
148 |    "cell_type": "markdown",
149 |    "metadata": {},
150 |    "source": [
151 |     "If $X_1, X_2,...X_n$ are independent discrete random variables \n",
152 |     "\n",
153 |     "$$\n",
154 |     "P_{X_{1}, X_{2}, \\ldots, X_{n}}\\left(x_{1}, x_{2}, \\ldots, x_{n}\\right)=P_{X_{1}}\\left(x_{1}\\right) P_{X_{2}}\\left(x_{2}\\right) \\cdots P_{X_{n}}\\left(x_{n}\\right)\n",
155 |     "$$"
156 |    ]
157 |   },
158 |   {
159 |    "cell_type": "markdown",
160 |    "metadata": {},
161 |    "source": [
162 |     "And if $X_1, X_2,...X_n$ are independent continuous random variables \n",
163 |     "$$\n",
164 |     "f_{X_{1}, X_{2}, \\ldots, X_{n}}\\left(x_{1}, x_{2}, \\ldots, x_{n}\\right)=f_{X_{1}}\\left(x_{1}\\right) f_{X_{2}}\\left(x_{2}\\right) \\cdots f_{X_{n}}\\left(x_{n}\\right)\n",
165 |     "$$"
166 |    ]
167 |   },
168 |   {
169 |    "cell_type": "markdown",
170 |    "metadata": {},
171 |    "source": [
172 |     "Their CDF and expections are \n",
173 |     "\n",
174 |     "$$\n",
175 |     "F_{X_{1}, X_{2}, \\ldots, X_{n}}\\left(x_{1}, x_{2}, \\ldots, x_{n}\\right)=F_{X_{1}}\\left(x_{1}\\right) F_{X_{2}}\\left(x_{2}\\right) \\cdots F_{X_{n}}\\left(x_{n}\\right)\n",
176 |     "$$\n",
177 |     "\n",
178 |     "$$\n",
179 |     "E\\left[X_{1} X_{2} \\cdots X_{n}\\right]=E\\left[X_{1}\\right] E\\left[X_{2}\\right] \\cdots E\\left[X_{n}\\right]\n",
180 |     "$$"
181 |    ]
182 |   },
183 |   {
184 |    "cell_type": "markdown",
185 |    "metadata": {},
186 |    "source": [
187 |     "One common setting for error term of regression model is <font face=\"gotham\" color=\"red\">independent identically distributed</font>. \n",
188 |     "\n",
189 |     "\\begin{aligned}\n",
190 |     "E\\left[X_{1} X_{2} \\cdots X_{n}\\right] &=E\\left[X_{1}\\right] E\\left[X_{2}\\right] \\cdots E\\left[X_{n}\\right] \\\\\n",
191 |     "&=E\\left[X_{1}\\right] E\\left[X_{1}\\right] \\cdots E\\left[X_{1}\\right] \\\\\n",
192 |     "&=E\\left[X_{1}\\right]^{n}=E\\left[X_{2}\\right]^{n}=\\cdots = E\\left[X_{n}\\right]^{n}\n",
193 |     "\\end{aligned}\n"
194 |    ]
195 |   },
196 |   {
197 |    "cell_type": "markdown",
198 |    "metadata": {},
199 |    "source": [
200 |     "## <font face=\"gotham\" color=\"purple\"> Moment Generating Functions</font>"
201 |    ]
202 |   },
203 |   {
204 |    "cell_type": "markdown",
205 |    "metadata": {},
206 |    "source": [
207 |     "Without formal definition, we call $E[X], E[X^2], E[X^k]$ the first moment, the second moment and the $k$th moment. \n",
208 |     "\n",
209 |     "Another important concept is central moment, for instance $E[(X - E[X])^k]$ is called the $k$th moment."
210 |    ]
211 |   },
212 |   {
213 |    "cell_type": "markdown",
214 |    "metadata": {},
215 |    "source": [
216 |     "The moment generating function (MGF) is a convenient way of generating all moments that we need. \n",
217 |     "\n",
218 |     "If $X$ is discrete random variable, the MGF is\n",
219 |     "\n",
220 |     "$$\n",
221 |     "M_X(t) = E[e^{tX}] = \\sum_0^\\infty e^{tk}P_X(k)\n",
222 |     "$$\n",
223 |     "\n",
224 |     "For continuous random variable $X$, the MGF is \n",
225 |     "\n",
226 |     "$$\n",
227 |     "M_X(t) = E[e^{tX}]  = \\int_{-\\infty}^\\infty e^{tX} f_X(x) dx\n",
228 |     "$$"
229 |    ]
230 |   },
231 |   {
232 |    "cell_type": "markdown",
233 |    "metadata": {},
234 |    "source": [
235 |     "We obtain moments by taking derivatives of MGFs\n",
236 |     "\n",
237 |     "$$\n",
238 |     "E(X^n) = \\frac{d^nM_x(t)}{dt^n}\\bigg|_{t=0}\n",
239 |     "$$"
240 |    ]
241 |   },
242 |   {
243 |    "cell_type": "markdown",
244 |    "metadata": {},
245 |    "source": [
246 |     "### <font face=\"gotham\" color=\"purple\"> How It Works?</font>"
247 |    ]
248 |   },
249 |   {
250 |    "cell_type": "markdown",
251 |    "metadata": {},
252 |    "source": [
253 |     "Perform Taylor expansion on $e^x$ \n",
254 |     "\n",
255 |     "$$\n",
256 |     " e^{x}=1+x+\\frac{x^{2}}{2 !}+\\frac{x^{3}}{3 !}+\\ldots=\\sum_{k=0}^{\\infty} \\frac{x^{k}}{k !}\n",
257 |     "$$"
258 |    ]
259 |   },
260 |   {
261 |    "cell_type": "markdown",
262 |    "metadata": {},
263 |    "source": [
264 |     "Suppose $X$ is a discrete random variable, therefore $e^{tX}$ is\n",
265 |     "\n",
266 |     "$$\n",
267 |     "e^{tX} = 1+tX+\\frac{(tX)^{2}}{2 !}+\\frac{(tX)^{3}}{3 !}+\\ldots=\\sum_{k=0}^{\\infty} \\frac{(tX)^{k}}{k !}\n",
268 |     "$$"
269 |    ]
270 |   },
271 |   {
272 |    "cell_type": "markdown",
273 |    "metadata": {},
274 |    "source": [
275 |     "Take expectation, we obtain MGF\n",
276 |     "\n",
277 |     "$$\n",
278 |     "E[e^{tX}] =1+tE[X]+\\frac{t^2E[X^{2}]}{2 !}+\\frac{t^3E[X^{3}]}{3 !}+\\ldots= \\sum_{k=0}^{\\infty} \\frac{t^kE[X^{k}]}{k !}\n",
279 |     "$$"
280 |    ]
281 |   },
282 |   {
283 |    "cell_type": "markdown",
284 |    "metadata": {},
285 |    "source": [
286 |     "Take the first order derivative and evaluate at $t=0$\n",
287 |     "\n",
288 |     "$$\n",
289 |     "\\frac{d E[e^{tX}]}{dt}= 0 + E[X] + tE[X^2] + \\frac{t^2E[X^3]}{2!}+\\cdots = 0+ E[X]+ 0 + 0 +\\cdots = E[X]\n",
290 |     "$$"
291 |    ]
292 |   },
293 |   {
294 |    "cell_type": "markdown",
295 |    "metadata": {},
296 |    "source": [
297 |     "Take the second order derivative and evaluate at $t=0$\n",
298 |     "\n",
299 |     "$$\n",
300 |     "\\frac{d^2 E[e^{tX}]}{dt^2} = 0 + 0 + E[X^2]+ tE[X^3]+ \\frac{t^2E[X^4]}{2!}+\\cdots=E[X^2]\n",
301 |     "$$"
302 |    ]
303 |   },
304 |   {
305 |    "cell_type": "markdown",
306 |    "metadata": {},
307 |    "source": [
308 |     "Take the third order derivative and evaluate at $t=0$\n",
309 |     "\n",
310 |     "$$\n",
311 |     "\\frac{d^3 E[e^{tX}]}{dt^3} = 0+0+0+E[X^3]+tE[X^4] = E[X^3]\n",
312 |     "$$"
313 |    ]
314 |   },
315 |   {
316 |    "cell_type": "markdown",
317 |    "metadata": {},
318 |    "source": [
319 |     "### <font face=\"gotham\" color=\"purple\"> Example 1 - Moments of Bernoulli</font>"
320 |    ]
321 |   },
322 |   {
323 |    "cell_type": "markdown",
324 |    "metadata": {},
325 |    "source": [
326 |     "Let's calculate the MGF of $X\\sim Ber(p)$, we know $P_X(1) = p, P_X(0)= 1-p$."
327 |    ]
328 |   },
329 |   {
330 |    "cell_type": "markdown",
331 |    "metadata": {},
332 |    "source": [
333 |     "$$\n",
334 |     " E[e^{tX}] = e^{0t}P_X(0)+e^{1t}P_X(1) = 1-p+pe^t\n",
335 |     "$$"
336 |    ]
337 |   },
338 |   {
339 |    "cell_type": "markdown",
340 |    "metadata": {},
341 |    "source": [
342 |     "If we want to calculate the variance with formula $E(X^2)-(E[X])^2$, we need to generate the first and second moment.\n",
343 |     "\n",
344 |     "$$\n",
345 |     "E[X]=\\frac{d E[e^{tX}]}{dt} \\bigg|_{t=0}= p\\\\\n",
346 |     "E[X^2]=\\frac{d^2 E[e^{tX}]}{dt^2} \\bigg|_{t=0}= p\n",
347 |     "$$"
348 |    ]
349 |   },
350 |   {
351 |    "cell_type": "markdown",
352 |    "metadata": {},
353 |    "source": [
354 |     "So the variance is $E(X^2)-(E[X])^2 = p - p^2$."
355 |    ]
356 |   },
357 |   {
358 |    "cell_type": "markdown",
359 |    "metadata": {},
360 |    "source": [
361 |     "### <font face=\"gotham\" color=\"purple\"> Example 2 - Moments of Binomial </font>"
362 |    ]
363 |   },
364 |   {
365 |    "cell_type": "markdown",
366 |    "metadata": {},
367 |    "source": [
368 |     "If $X_1, X_2, \\ldots ,X_n \\stackrel{i.i.d.}{\\sim} Ber(p)$, \n",
369 |     "\n",
370 |     "$$\n",
371 |     "\\sum^n_{i=1}X_i\\ \\sim\\ B(n,p)\n",
372 |     "$$\n",
373 |     "\n",
374 |     "where $B(n,p)$ denotes binomial distribution."
375 |    ]
376 |   },
377 |   {
378 |    "cell_type": "markdown",
379 |    "metadata": {},
380 |    "source": [
381 |     "The goal is to find the first moment $E(\\sum^n_{i=1}X_i)$ and the second central moment $\\text{Var}(\\sum^n_{i=1}X_i)$."
382 |    ]
383 |   },
384 |   {
385 |    "cell_type": "markdown",
386 |    "metadata": {},
387 |    "source": [
388 |     "Using MGF definition\n",
389 |     "\n",
390 |     "\\begin{align}\n",
391 |     "E[e^{t\\sum^n_{i=1}X_i}] &= E[e^{tX_1}e^{tX_2}\\ldots e^{tX_n}]\n",
392 |     "= E[e^{tX_1}] E[e^{tX_2}]\\ldots E[e^{tX_n}]  \\tag{$X_i$'s are i.i.d.}\\\\\n",
393 |     "& = (1-p+pe^t)(1-p+pe^t)\\ldots (1-p+pe^t)\\\\\n",
394 |     "& = (1-p+pe^t)^n\n",
395 |     "\\end{align}"
396 |    ]
397 |   },
398 |   {
399 |    "cell_type": "markdown",
400 |    "metadata": {},
401 |    "source": [
402 |     "The first moment is \n",
403 |     "\n",
404 |     "$$\n",
405 |     "E\\bigg(\\sum^n_{i=1}X_i\\bigg) = \\frac{d}{dt}(1-p+pe^t)^n\\bigg|_{t=0}=n(1-p+pe^t)^{n-1}pe^t\\Big|_{t=0}=np\n",
406 |     "$$"
407 |    ]
408 |   },
409 |   {
410 |    "cell_type": "markdown",
411 |    "metadata": {},
412 |    "source": [
413 |     "The second moment requires product rule of differentiation\n",
414 |     "\n",
415 |     "\\begin{align}\n",
416 |     "E\\bigg[\\bigg(\\sum^n_{i=1}X_i\\bigg)^2 \\bigg]=\\frac{d}{dt}n(1-p+pe^t)^{n-1}pe^t  \n",
417 |     "&= n(n-1)(1-p+pe^t)^{n-2}pe^tpe^t+n(1-p+pe^t)^{n-1}pe^t\\Big|_{t=0}\\\\\n",
418 |     "&=n(n-1)p^2+np=n^2p^2-np^2+np\n",
419 |     "\\end{align}"
420 |    ]
421 |   },
422 |   {
423 |    "cell_type": "markdown",
424 |    "metadata": {},
425 |    "source": [
426 |     "Thereby \n",
427 |     "\n",
428 |     "$$\n",
429 |     "\\text{Var}\\left(\\sum^n_{i=1}X_i\\right)= n^2p^2-np^2+np - n^2p^2 = np - np^2\n",
430 |     "$$"
431 |    ]
432 |   }
433 |  ],
434 |  "metadata": {
435 |   "kernelspec": {
436 |    "display_name": "Python 3",
437 |    "language": "python",
438 |    "name": "python3"
439 |   },
440 |   "language_info": {
441 |    "codemirror_mode": {
442 |     "name": "ipython",
443 |     "version": 3
444 |    },
445 |    "file_extension": ".py",
446 |    "mimetype": "text/x-python",
447 |    "name": "python",
448 |    "nbconvert_exporter": "python",
449 |    "pygments_lexer": "ipython3",
450 |    "version": "3.7.4"
451 |   },
452 |   "toc": {
453 |    "base_numbering": 1,
454 |    "nav_menu": {},
455 |    "number_sections": true,
456 |    "sideBar": true,
457 |    "skip_h1_title": false,
458 |    "title_cell": "Table of Contents",
459 |    "title_sidebar": "Contents",
460 |    "toc_cell": true,
461 |    "toc_position": {},
462 |    "toc_section_display": true,
463 |    "toc_window_display": false
464 |   }
465 |  },
466 |  "nbformat": 4,
467 |  "nbformat_minor": 2
468 | }
469 | 


--------------------------------------------------------------------------------
/Chapter 1 - Joint and Conditional Probability.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "markdown",
  5 |    "metadata": {
  6 |     "toc": true
  7 |    },
  8 |    "source": [
  9 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
 10 |     "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
 11 |     "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#-Joint-Probability\" data-toc-modified-id=\"-Joint-Probability-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Joint Probability</font></a></span><ul class=\"toc-item\"><li><span><a href=\"#-Example-1\" data-toc-modified-id=\"-Example-1-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 1</font></a></span></li><li><span><a href=\"#-Example-2\" data-toc-modified-id=\"-Example-2-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 2</font></a></span></li><li><span><a href=\"#-Example-3\" data-toc-modified-id=\"-Example-3-1.3\"><span class=\"toc-item-num\">1.3&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 3</font></a></span></li></ul></li><li><span><a href=\"#-Conditional-Probability\" data-toc-modified-id=\"-Conditional-Probability-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Conditional Probability</font></a></span><ul class=\"toc-item\"><li><span><a href=\"#-Example-1\" data-toc-modified-id=\"-Example-1-2.1\"><span class=\"toc-item-num\">2.1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 1</font></a></span></li><li><span><a href=\"#-Example-2\" data-toc-modified-id=\"-Example-2-2.2\"><span class=\"toc-item-num\">2.2&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 2</font></a></span></li><li><span><a href=\"#-Example-3\" data-toc-modified-id=\"-Example-3-2.3\"><span class=\"toc-item-num\">2.3&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 3</font></a></span></li></ul></li><li><span><a href=\"#-Independence\" data-toc-modified-id=\"-Independence-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Independence</font></a></span><ul class=\"toc-item\"><li><span><a href=\"#-Independence-vs-Disjoint\" data-toc-modified-id=\"-Independence-vs-Disjoint-3.1\"><span class=\"toc-item-num\">3.1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Independence vs Disjoint</font></a></span></li></ul></li><li><span><a href=\"#Law-of-Total-Probability\" data-toc-modified-id=\"Law-of-Total-Probability-4\"><span class=\"toc-item-num\">4&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\">Law of Total Probability</font></a></span><ul class=\"toc-item\"><li><span><a href=\"#-Example-1\" data-toc-modified-id=\"-Example-1-4.1\"><span class=\"toc-item-num\">4.1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 1</font></a></span></li><li><span><a href=\"#-Example-2\" data-toc-modified-id=\"-Example-2-4.2\"><span class=\"toc-item-num\">4.2&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 2</font></a></span></li><li><span><a href=\"#-Example-3\" data-toc-modified-id=\"-Example-3-4.3\"><span class=\"toc-item-num\">4.3&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 3</font></a></span></li></ul></li></ul></div>\n",
 12 |     "</div>"
 13 |    ]
 14 |   },
 15 |   {
 16 |    "cell_type": "markdown",
 17 |    "metadata": {},
 18 |    "source": [
 19 |     "# <font face=\"gotham\" color=\"purple\"> Joint Probability</font>"
 20 |    ]
 21 |   },
 22 |   {
 23 |    "cell_type": "markdown",
 24 |    "metadata": {},
 25 |    "source": [
 26 |     "**Joint probability** and **conditional probability** are closely intertwined, however elusive for beginners, sometimes even experienced data scientists are hard to tell the subtlety. \n",
 27 |     "\n",
 28 |     "We will tackle the joint probability first. "
 29 |    ]
 30 |   },
 31 |   {
 32 |    "cell_type": "markdown",
 33 |    "metadata": {},
 34 |    "source": [
 35 |     "The joint probability is defined as\n",
 36 |     "\n",
 37 |     "$$\n",
 38 |     "P(A\\cap B) = P(A)P(B)\n",
 39 |     "$$\n",
 40 |     "\n",
 41 |     "when we talking about joint probability, we implicitly assume that\n",
 42 |     "\n",
 43 |     "1. $A$ and $B$ are independent events.\n",
 44 |     "2. $A$ and $B$ happen in the same time.\n",
 45 |     "\n",
 46 |     "For instance, I throw a dice (event A) and you also throw a dice (event B) in the mean time. The joint probability of obtaining two $1$'s is $\\frac{1}{6}\\times\\frac{1}{6}$. \n",
 47 |     "\n",
 48 |     "If $A$ and $B$ are not independent, then it is pointless to talk about joint probability, because they are affection each other's results. In other words, if $A$ and $B$ are dependent, when $A$ happens, the probability of $B$ happening is no longer $P(B)$ anymore, hence $P(AB) = P(A)P(B)$ won't hold in this case."
 49 |    ]
 50 |   },
 51 |   {
 52 |    "cell_type": "markdown",
 53 |    "metadata": {},
 54 |    "source": [
 55 |     "## <font face=\"gotham\" color=\"purple\"> Example 1</font>"
 56 |    ]
 57 |   },
 58 |   {
 59 |    "cell_type": "markdown",
 60 |    "metadata": {},
 61 |    "source": [
 62 |     "What is the joint probability of rolling the number $6$ twice in a fair six-sided dice?"
 63 |    ]
 64 |   },
 65 |   {
 66 |    "cell_type": "markdown",
 67 |    "metadata": {},
 68 |    "source": [
 69 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
 70 |     "Throwing twice dice at a time are independent events, <br>\n",
 71 |     "\n",
 72 |     "$\\text{Event A}$ = $\\text{Obtaining number 6}$\n",
 73 |     "\n",
 74 |     "$$\n",
 75 |     "P(A\\cap A)=\\frac{1}{6}\\frac{1}{6}= \\frac{1}{36}\n",
 76 |     "$$\n",
 77 |     "</div> "
 78 |    ]
 79 |   },
 80 |   {
 81 |    "cell_type": "markdown",
 82 |    "metadata": {},
 83 |    "source": [
 84 |     "## <font face=\"gotham\" color=\"purple\"> Example 2</font>"
 85 |    ]
 86 |   },
 87 |   {
 88 |    "cell_type": "markdown",
 89 |    "metadata": {},
 90 |    "source": [
 91 |     "What is the joint probability of obtaining 3 times head followed by 3 times tail in a sequence of coin tosses?"
 92 |    ]
 93 |   },
 94 |   {
 95 |    "cell_type": "markdown",
 96 |    "metadata": {},
 97 |    "source": [
 98 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
 99 |     "$\\text{Event $A$}$ = $\\text{Head}$<br>\n",
100 |     "$\\text{Event $B$}$ =$\\text{Tail}$\n",
101 |     "\n",
102 |     "$$\n",
103 |     "P(BBBAAA)= .5^6\n",
104 |     "$$\n",
105 |     "</div>"
106 |    ]
107 |   },
108 |   {
109 |    "cell_type": "code",
110 |    "execution_count": 1,
111 |    "metadata": {},
112 |    "outputs": [
113 |     {
114 |      "data": {
115 |       "text/plain": [
116 |        "0.015625"
117 |       ]
118 |      },
119 |      "execution_count": 1,
120 |      "metadata": {},
121 |      "output_type": "execute_result"
122 |     }
123 |    ],
124 |    "source": [
125 |     ".5**6"
126 |    ]
127 |   },
128 |   {
129 |    "cell_type": "markdown",
130 |    "metadata": {},
131 |    "source": [
132 |     "## <font face=\"gotham\" color=\"purple\"> Example 3</font>"
133 |    ]
134 |   },
135 |   {
136 |    "cell_type": "markdown",
137 |    "metadata": {},
138 |    "source": [
139 |     "What is the joint probability that you pick a card which is  <b>larger</b> than 4 and <b>less</b> than $10$ and in the meanwhile a $diamond$?"
140 |    ]
141 |   },
142 |   {
143 |    "cell_type": "markdown",
144 |    "metadata": {},
145 |    "source": [
146 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
147 |     "$\\text{Event A}$ = $\\text{Getting card in (4<x<10)}$<br>\n",
148 |     "$\\text{Event B}$ = $\\text{Getting a diamond}$.\n",
149 |     "\n",
150 |     "There are five cards in range $(4, 10)$, probability is $\\frac{5}{52}$, considering four suits $\\frac{5\\times 4}{52}$, and thirteen cards for each suit.\n",
151 |     "\n",
152 |     "$P(A\\cap B) = P(A)P(B) = \\frac{5\\times 4}{52}\\frac{13}{52}$\n",
153 |     "</div>"
154 |    ]
155 |   },
156 |   {
157 |    "cell_type": "code",
158 |    "execution_count": 2,
159 |    "metadata": {},
160 |    "outputs": [
161 |     {
162 |      "data": {
163 |       "text/plain": [
164 |        "0.09615384615384616"
165 |       ]
166 |      },
167 |      "execution_count": 2,
168 |      "metadata": {},
169 |      "output_type": "execute_result"
170 |     }
171 |    ],
172 |    "source": [
173 |     "(5*4/52)*(13/52)"
174 |    ]
175 |   },
176 |   {
177 |    "cell_type": "markdown",
178 |    "metadata": {},
179 |    "source": [
180 |     "# <font face=\"gotham\" color=\"purple\"> Conditional Probability</font>"
181 |    ]
182 |   },
183 |   {
184 |    "cell_type": "markdown",
185 |    "metadata": {},
186 |    "source": [
187 |     "Conditional probability is more complicated, denoted as\n",
188 |     "\n",
189 |     "$$\n",
190 |     "P(B|A)= \\frac{P(A\\cap B)}{P(A)}=\\frac{P(A\\cap B)}{P(A\\cap B)+P(A\\cap B^C)}\n",
191 |     "$$"
192 |    ]
193 |   },
194 |   {
195 |    "cell_type": "markdown",
196 |    "metadata": {},
197 |    "source": [
198 |     "which reads, the probability of $B$ given $A$. Here we treat $A$ as a fact, it either happens or assumed to happen. $B^C$ is the complement event of $A$, such that $P(B^C)+P(B) =1 $"
199 |    ]
200 |   },
201 |   {
202 |    "cell_type": "markdown",
203 |    "metadata": {},
204 |    "source": [
205 |     "## <font face=\"gotham\" color=\"purple\"> Example 1</font>"
206 |    ]
207 |   },
208 |   {
209 |    "cell_type": "markdown",
210 |    "metadata": {},
211 |    "source": [
212 |     "What is the probability of drawing a card, given which is a $\\text{diamond}$, that is also <b>larger</b> than 4 and <b>less</b> than $10$?"
213 |    ]
214 |   },
215 |   {
216 |    "cell_type": "markdown",
217 |    "metadata": {},
218 |    "source": [
219 |     "Note how we rephrase the question than Example 3 in last section. When we are talking about conditional probability, the sample space is usually different from joint probability, here the sample space changes from all deck to $\\text{diamonds}$."
220 |    ]
221 |   },
222 |   {
223 |    "cell_type": "markdown",
224 |    "metadata": {},
225 |    "source": [
226 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
227 |     "$\\text{Event A}$ = $\\text{Getting card in (4<x<10)}$<br>\n",
228 |     "$\\text{Event B}$ = $\\text{Getting a diamond}$.<br><br>\n",
229 |     "    \n",
230 |     "There are $13$ cards of diamonds, this is the new sample space. And in this new sample space, we count how many numbers there are in $(4,\\ 10)$. There are 5 numbers. Thus\n",
231 |     "\n",
232 |     "$P(A|B) = \\frac{5}{13}$\n",
233 |     "</div>"
234 |    ]
235 |   },
236 |   {
237 |    "cell_type": "code",
238 |    "execution_count": 3,
239 |    "metadata": {},
240 |    "outputs": [
241 |     {
242 |      "data": {
243 |       "text/plain": [
244 |        "0.38461538461538464"
245 |       ]
246 |      },
247 |      "execution_count": 3,
248 |      "metadata": {},
249 |      "output_type": "execute_result"
250 |     }
251 |    ],
252 |    "source": [
253 |     "5/13"
254 |    ]
255 |   },
256 |   {
257 |    "cell_type": "markdown",
258 |    "metadata": {},
259 |    "source": [
260 |     "Then what is the joint probability of $P(A\\cap B)$? Using multiplication rule $P(A\\cap B) = P(A|B)P(B)$."
261 |    ]
262 |   },
263 |   {
264 |    "cell_type": "code",
265 |    "execution_count": 4,
266 |    "metadata": {},
267 |    "outputs": [
268 |     {
269 |      "data": {
270 |       "text/plain": [
271 |        "0.09615384615384616"
272 |       ]
273 |      },
274 |      "execution_count": 4,
275 |      "metadata": {},
276 |      "output_type": "execute_result"
277 |     }
278 |    ],
279 |    "source": [
280 |     "5/13*1/4"
281 |    ]
282 |   },
283 |   {
284 |    "cell_type": "markdown",
285 |    "metadata": {},
286 |    "source": [
287 |     "The joint probability only makes sense is because drawing a $\\text{diamond}$ and in $(4 <x< 10)$ are independent events."
288 |    ]
289 |   },
290 |   {
291 |    "cell_type": "markdown",
292 |    "metadata": {},
293 |    "source": [
294 |     "Note that we have reproduced the same result of the joint probability in Example 3 of last section. Note that, this is also a common way of calculating joint probability"
295 |    ]
296 |   },
297 |   {
298 |    "cell_type": "markdown",
299 |    "metadata": {},
300 |    "source": [
301 |     "## <font face=\"gotham\" color=\"purple\"> Example 2</font>"
302 |    ]
303 |   },
304 |   {
305 |    "cell_type": "markdown",
306 |    "metadata": {},
307 |    "source": [
308 |     "We roll a fair die twice and obtain two numbers $X_1= a$ and $X_2= b$. Given $a+b=7$, what is the probability that $a=4$ or $b=4$?"
309 |    ]
310 |   },
311 |   {
312 |    "cell_type": "markdown",
313 |    "metadata": {},
314 |    "source": [
315 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
316 |     "Our goal is to find out $P(a=4|a+b=7)$ and $P(b=4|a+b=7)$.\n",
317 |     "\n",
318 |     "$\\text{Event A}$ : $a = 4$ or $b = 4$<br>\n",
319 |     "$\\text{Event B}$ : $a+b = 7$\n",
320 |     "\n",
321 |     "There are only two possibilities of obtaining a $7$ if either $a$ or $b$ equals $4$, that are $(3, 4)$ and $(4, 3)$. And two dice have $6^2 = 36$ combinations, thus $P(A\\cap B) = \\frac{2}{36}$.\n",
322 |     "\n",
323 |     "To obtain $7$ there are $6$ combination: $(1, 6), (2, 5), (3,4), (4,3), (5,2),(6, 1)$, this is actually the sample space, so we get\n",
324 |     "\n",
325 |     "$$\\frac{P(A\\cap B)}{P(B)}= \\frac{2}{36}\\Big/\\frac{6}{36}=\\frac{1}{3}$$\n",
326 |     "</div>"
327 |    ]
328 |   },
329 |   {
330 |    "cell_type": "markdown",
331 |    "metadata": {},
332 |    "source": [
333 |     "## <font face=\"gotham\" color=\"purple\"> Example 3</font>"
334 |    ]
335 |   },
336 |   {
337 |    "cell_type": "markdown",
338 |    "metadata": {},
339 |    "source": [
340 |     "Consider a family that has two children. We are interested in the children's genders. The sample space of gender is $S=\\{(G,G),(G,B),(B,G),(B,B)\\}$. Also assume that all four possible outcomes are equally likely.\n",
341 |     "\n",
342 |     "1. What is the probability that both children are girls given that the first child is a girl?\n",
343 |     "2. We ask the father: \"Do you have at least one daughter?\" He responds \"Yes!\". Given this extra information, what is the probability that both children are girls? In other words, what is the probability that both children are girls given that we know at least one of them is a girl?\n"
344 |    ]
345 |   },
346 |   {
347 |    "cell_type": "markdown",
348 |    "metadata": {},
349 |    "source": [
350 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
351 |     "Since we have sample space explicitly presented, the conditional probability is simply counting events\n",
352 |     "\n",
353 |     "The first question's sample space is $\\{(G,G),(G,B)\\}$, the answer is $1/2$.\n",
354 |     "\n",
355 |     "The second question's sample space is $\\{(G,G),(G,B),(B,G)\\}$, the answer is $1/3$.\n",
356 |     "</div>"
357 |    ]
358 |   },
359 |   {
360 |    "cell_type": "markdown",
361 |    "metadata": {},
362 |    "source": [
363 |     "# <font face=\"gotham\" color=\"purple\"> Independence</font>"
364 |    ]
365 |   },
366 |   {
367 |    "cell_type": "markdown",
368 |    "metadata": {},
369 |    "source": [
370 |     "Let me define two events:\n",
371 |     "\n",
372 |     "$\\text{Event A}$: $\\text{Obtaining head when tossing a coin}$<br>\n",
373 |     "$\\text{Event B}$: $\\text{Raining today}$<br>\n",
374 |     "\n",
375 |     "What is $P(A|B)$? Unless you believe weather will affect your winning odds of gambling, the sober answer is that it doesn't have a conditional probability."
376 |    ]
377 |   },
378 |   {
379 |    "cell_type": "markdown",
380 |    "metadata": {},
381 |    "source": [
382 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
383 |     "Apparently, for independent events, conditional probability reduces to $P(A|B) = P(A)$. Formally derived from\n",
384 |     "\n",
385 |     "$$\n",
386 |     "\\begin{aligned}\n",
387 |     "P(A | B) &=\\frac{P(A \\cap B)}{P(B)} \\\\\n",
388 |     "&=\\frac{P(A) P(B)}{P(B)} \\\\\n",
389 |     "&=P(A)\n",
390 |     "\\end{aligned}\n",
391 |     "$$\n",
392 |     "That's exactly why we mentioned in the beginning that joint probability only makes sense in the context of probabilistic independence.\n",
393 |     "</div>"
394 |    ]
395 |   },
396 |   {
397 |    "cell_type": "markdown",
398 |    "metadata": {},
399 |    "source": [
400 |     "## <font face=\"gotham\" color=\"purple\"> Independence vs Disjoint</font>"
401 |    ]
402 |   },
403 |   {
404 |    "cell_type": "markdown",
405 |    "metadata": {},
406 |    "source": [
407 |     "Independence and disjoint are entirely disparate concepts. **Disjoint events** mean if $A$ happens, then $B$ certainly can't happen, which exactly assures they are dependent."
408 |    ]
409 |   },
410 |   {
411 |    "cell_type": "markdown",
412 |    "metadata": {},
413 |    "source": [
414 |     "Formally, consider two events $A$ and $B$, with $P(A) \\neq 0$ and $P(B) \\neq 0$. \n",
415 |     "\n",
416 |     "We can show that if $A$ and $B$ are disjoint, we have\n",
417 |     "\n",
418 |     "$$\n",
419 |     "P(A \\cap B)=\\varnothing  \\neq P(A) P(B)\n",
420 |     "$$\n",
421 |     "\n",
422 |     "Thus, $A$ and $B$ are not independent."
423 |    ]
424 |   },
425 |   {
426 |    "cell_type": "markdown",
427 |    "metadata": {},
428 |    "source": [
429 |     "# <font face=\"gotham\" color=\"purple\">Law of Total Probability</font>"
430 |    ]
431 |   },
432 |   {
433 |    "cell_type": "markdown",
434 |    "metadata": {},
435 |    "source": [
436 |     "If $B_{1}, B_{2}, B_{3}, \\cdots B_k$ are partitions of the sample space $S$, then for any event $A$ we have\n",
437 |     "\n",
438 |     "$$\n",
439 |     "P(A)=\\sum_{i=1}^k P\\left(A \\cap B_{i}\\right)=\\sum_{i=1}^k  P\\left(A | B_{i}\\right) P\\left(B_{i}\\right)\n",
440 |     "$$\n",
441 |     "\n",
442 |     "This is called **law of total probability**."
443 |    ]
444 |   },
445 |   {
446 |    "cell_type": "markdown",
447 |    "metadata": {},
448 |    "source": [
449 |     "## <font face=\"gotham\" color=\"purple\"> Example 1</font>"
450 |    ]
451 |   },
452 |   {
453 |    "cell_type": "markdown",
454 |    "metadata": {},
455 |    "source": [
456 |     "A person has undertaken a mining job. The probabilities of completion of job on time with and without rain are $0.42$ and $0.90$ respectively. If the probability that it will rain is $0.45$, then determine the probability that the mining job will be completed on time."
457 |    ]
458 |   },
459 |   {
460 |    "cell_type": "markdown",
461 |    "metadata": {},
462 |    "source": [
463 |     "Total probability to some extent means that you have to consider all scenarios, in this case raining $R$ and not raining $R^C$."
464 |    ]
465 |   },
466 |   {
467 |    "cell_type": "markdown",
468 |    "metadata": {},
469 |    "source": [
470 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
471 |     "$\\text{Event R}$ : $\\text{raining}$<br>\n",
472 |     "\n",
473 |     "$\\text{Event C}$ : $\\text{complete the job on time}$\n",
474 |     "$$p(C) = p(C|R)p(R) + p(C|R^c)p(R^c) = .42\\times .45 + .9\\times .55$$\n",
475 |     "</div>"
476 |    ]
477 |   },
478 |   {
479 |    "cell_type": "code",
480 |    "execution_count": 8,
481 |    "metadata": {},
482 |    "outputs": [
483 |     {
484 |      "name": "stdout",
485 |      "output_type": "stream",
486 |      "text": [
487 |       "The probability of completing mining job on time is 68.400%.\n"
488 |      ]
489 |     }
490 |    ],
491 |    "source": [
492 |     "P_C = .42*.45+.9*.55\n",
493 |     "print('The probability of completing mining job on time is %.3f%%.'%(P_C*100))"
494 |    ]
495 |   },
496 |   {
497 |    "cell_type": "markdown",
498 |    "metadata": {},
499 |    "source": [
500 |     "## <font face=\"gotham\" color=\"purple\"> Example 2</font>"
501 |    ]
502 |   },
503 |   {
504 |    "cell_type": "markdown",
505 |    "metadata": {},
506 |    "source": [
507 |     "We draw two cards from a well-shuffled deck without replacement. Find the probability of getting a queen in the second draw. So we calculate probabilities of two scenarios (first card queen and first card not queen) then sum up."
508 |    ]
509 |   },
510 |   {
511 |    "cell_type": "markdown",
512 |    "metadata": {},
513 |    "source": [
514 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
515 |     "$\\text{Event Q1}$ : $\\text{Drawing a queen in the first card}$<br>\n",
516 |     "\n",
517 |     "$\\text{Event Q2}$ : $\\text{Drawing a queen in the second card}$\n",
518 |     "\n",
519 |     "\n",
520 |     "$$\n",
521 |     "P(Q2) = P(Q2|Q1)P(Q1) + P(Q2|Q1^c)P(Q1^c) = \\frac{3}{51}\\frac{4}{52}+\\frac{4}{51}\\frac{48}{52}\n",
522 |     "$$\n",
523 |     "</div>"
524 |    ]
525 |   },
526 |   {
527 |    "cell_type": "code",
528 |    "execution_count": 6,
529 |    "metadata": {},
530 |    "outputs": [
531 |     {
532 |      "name": "stdout",
533 |      "output_type": "stream",
534 |      "text": [
535 |       "The probability of drawing a queen in the second card is 7.692%.\n"
536 |      ]
537 |     }
538 |    ],
539 |    "source": [
540 |     "PQ2 = 3/51*4/52+4/51*48/52\n",
541 |     "print('The probability of drawing a queen in the second card is %.3f%%.'%(PQ2*100))"
542 |    ]
543 |   },
544 |   {
545 |    "cell_type": "markdown",
546 |    "metadata": {},
547 |    "source": [
548 |     "## <font face=\"gotham\" color=\"purple\"> Example 3</font>"
549 |    ]
550 |   },
551 |   {
552 |    "cell_type": "markdown",
553 |    "metadata": {},
554 |    "source": [
555 |     "In a certain county<br>\n",
556 |     "1. $60 \\%$ of registered voters are Republicans \n",
557 |     "2. $30 \\%$ are Democrats\n",
558 |     "3. $10 \\%$ are Independents\n",
559 |     "\n",
560 |     "When those voters were asked opinions on increasing military spending\n",
561 |     "1. $40 \\%$ of Republicans opposed it \n",
562 |     "2. $65 \\%$ of the Democrats opposed it \n",
563 |     "3. $55 \\%$ of the Independents opposed it.\n",
564 |     "\n",
565 |     "What is the probability that a randomly selected voter in this county opposes increasing military spending? \n",
566 |     "\n",
567 |     "Three scenarios to be taken into consideration, being a Republican, a Democrat or an Independent."
568 |    ]
569 |   },
570 |   {
571 |    "cell_type": "markdown",
572 |    "metadata": {},
573 |    "source": [
574 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
575 |     "$\\text{Event A}$: $\\text{opposes increased military spending}$<br>\n",
576 |     "$\\text{Event B}$: $\\text{Being Republican}$<br>\n",
577 |     "$\\text{Event C}$: $\\text{Being Democrats}$<br>\n",
578 |     "$\\text{Event D}$: $\\text{Being Independents}$\n",
579 |     "\n",
580 |     "\n",
581 |     "$$P(A) = P(A|B)P(B)+P(A|C)P(C)+P(A|D)P(D) = .4\\times.6+ .65\\times.3+ .55\\times.1$$\n",
582 |     "</div>"
583 |    ]
584 |   },
585 |   {
586 |    "cell_type": "code",
587 |    "execution_count": 9,
588 |    "metadata": {},
589 |    "outputs": [
590 |     {
591 |      "name": "stdout",
592 |      "output_type": "stream",
593 |      "text": [
594 |       "The probability that a randomly selected voter in this county opposes increased military spending is 49.000%.\n"
595 |      ]
596 |     }
597 |    ],
598 |    "source": [
599 |     "PA = .4*.6+.65*.3+.55*.1\n",
600 |     "print('The probability that a randomly selected voter in this county opposes increased military spending is %.3f%%.'%(PA*100))"
601 |    ]
602 |   }
603 |  ],
604 |  "metadata": {
605 |   "kernelspec": {
606 |    "display_name": "Python 3",
607 |    "language": "python",
608 |    "name": "python3"
609 |   },
610 |   "language_info": {
611 |    "codemirror_mode": {
612 |     "name": "ipython",
613 |     "version": 3
614 |    },
615 |    "file_extension": ".py",
616 |    "mimetype": "text/x-python",
617 |    "name": "python",
618 |    "nbconvert_exporter": "python",
619 |    "pygments_lexer": "ipython3",
620 |    "version": "3.8.8"
621 |   },
622 |   "toc": {
623 |    "base_numbering": 1,
624 |    "nav_menu": {},
625 |    "number_sections": true,
626 |    "sideBar": true,
627 |    "skip_h1_title": false,
628 |    "title_cell": "Table of Contents",
629 |    "title_sidebar": "Contents",
630 |    "toc_cell": true,
631 |    "toc_position": {},
632 |    "toc_section_display": true,
633 |    "toc_window_display": true
634 |   }
635 |  },
636 |  "nbformat": 4,
637 |  "nbformat_minor": 4
638 | }
639 | 


--------------------------------------------------------------------------------
/Chapter 3 - PMF, PDF and CDF.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "markdown",
  5 |    "metadata": {
  6 |     "toc": true
  7 |    },
  8 |    "source": [
  9 |     "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
 10 |     "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#-Probability-Mass-Function\" data-toc-modified-id=\"-Probability-Mass-Function-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Probability Mass Function</font></a></span><ul class=\"toc-item\"><li><span><a href=\"#-Cumulative-Distribution-Function\" data-toc-modified-id=\"-Cumulative-Distribution-Function-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Cumulative Distribution Function</font></a></span></li></ul></li><li><span><a href=\"#-Probability-Density-Function\" data-toc-modified-id=\"-Probability-Density-Function-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Probability Density Function</font></a></span></li><li><span><a href=\"#Independent-Random-Variables\" data-toc-modified-id=\"Independent-Random-Variables-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\">Independent Random Variables</font></a></span></li><li><span><a href=\"#-Expected-Value-and-Variances\" data-toc-modified-id=\"-Expected-Value-and-Variances-4\"><span class=\"toc-item-num\">4&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Expected Value and Variances</font></a></span></li></ul></div>"
 11 |    ]
 12 |   },
 13 |   {
 14 |    "cell_type": "code",
 15 |    "execution_count": 9,
 16 |    "metadata": {},
 17 |    "outputs": [],
 18 |    "source": [
 19 |     "import numpy as np\n",
 20 |     "import matplotlib.pyplot as plt\n",
 21 |     "plt.style.use('fivethirtyeight')"
 22 |    ]
 23 |   },
 24 |   {
 25 |    "cell_type": "markdown",
 26 |    "metadata": {},
 27 |    "source": [
 28 |     "# <font face=\"gotham\" color=\"purple\"> Probability Mass Function"
 29 |    ]
 30 |   },
 31 |   {
 32 |    "cell_type": "markdown",
 33 |    "metadata": {},
 34 |    "source": [
 35 |     "If $X$ is a discrete variable with a finite range $R_X = \\{x_1,x_2,x_3...\\}$, the **probability mass function** (PMF) of $X$ is \n",
 36 |     "\n",
 37 |     "$$\n",
 38 |     "P_X(x_k)=P(X=x_k),\\ \\text{for } k=1,2,3\\ldots\n",
 39 |     "$$\n",
 40 |     "\n",
 41 |     "which is a function which maps the possible values to the corresponding probabilities. "
 42 |    ]
 43 |   },
 44 |   {
 45 |    "cell_type": "markdown",
 46 |    "metadata": {},
 47 |    "source": [
 48 |     "For instance, here is a discrete uniform distribution of a fair die"
 49 |    ]
 50 |   },
 51 |   {
 52 |    "cell_type": "code",
 53 |    "execution_count": 10,
 54 |    "metadata": {},
 55 |    "outputs": [
 56 |     {
 57 |      "data": {
 58 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa0AAAEeCAYAAAA5CErsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAApK0lEQVR4nO3df1xUdb4/8BcwiPFrRgll0BVhHDWMdjdCDRXQBEm0VveW7FBmtrdd9z6Wx8PNH2W7Fmq77E3qyrfr3m4ZqxQCFmVCBVg4F0WLMsoVxRENVH4I5gxMijLMfP/wwWzjoDMMuMPn+Ho+Hvwxn/M553zeSbw453w+Bw+9Xm8BERGRADzdPQAiIiJnMbSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLaIBeP311zF9+nQolUooFAr85S9/cfeQBsU777wDhUKBd955x91DAQD85S9/gUKhQGVlpbuHQm7G0KJBo1AobL5GjhyJsLAwJCcnIycnBz09PTb9U1JSrH23b99+w+Nu3rzZ2m/58uU223p/uN7sS6/X34py8d5772Ht2rXo7u7G008/jbVr12LmzJlO79/d3Q21Wg2FQoFHHnnEpTG4s/7+un5cwcHBiIiIwMyZM7FixQoUFxeju7vb3cOkIU7m7gGQ9KxduxYA0NPTg9OnT6O4uBiHDh3Cvn37+gwnmUyG7du344knnrDbZrFYkJubC5lMBpPJdMNz3n333UhJSelz2/Dhw12s5OZKS0sBAP/zP/+DmJiYfu9fXFyMtrY2eHh44NNPP0VjYyPGjRvn0lgGu/4FCxYgJiYGo0ePdmk8N/Pj74+Ojg7odDp88MEH2LlzJyZNmoTXX38dP/vZz2z2efrpp/HLX/4SY8eOHfTxkFgYWjTonnvuOZvPR48exdy5c7F7925UVVUhNjbWZvu8efNQUlKCI0eOICoqymbbvn370NDQgAULFqC4uPiG54yKirI7763W3NwMABg1apRL++fk5AAAnnnmGWzevBk7duzAH//4R5eONdj1y+VyyOXyQTvej/U1Tr1ej40bN2Lbtm1YtGgR9u7dC5VKZd0eFBSEoKCgWzIeEgtvD9ItN2XKFMyYMQMA8NVXX9ltX7p0KTw8PLBjxw67bdu3b8fw4cPx6KOP3vJxdnR0YOPGjdYrjHHjxmHBggXYs2ePTb/rn6/89Kc/td7yctapU6dQWVmJ6dOn4w9/+AMCAgLw9ttv3/RqciBqamqwZs0axMbGIiwsDKNHj8a9996LdevW4eLFi3b9b/RMKyoqCgqFAl1dXdi0aRN+/vOfIzg4GM8+++yAxqdQKJCVlYVHH30UFy9exIsvvmiz/WbPtE6dOoXf//73uPvuuzFq1CioVCqkpaWhpqZmQGOioYlXWuR24eHhmDVrFgoLC7FhwwbccccdAID29nZ89NFH+MUvfoERI0bc0jHo9XokJyfj+PHjuOeee/Db3/4WBoMBH3zwAR5//HGsWbMG69atAwDrc6u8vDycOXMGv/3tb/t9VfL3v/8dFosFGo0Gvr6++MUvfoHc3Fx8/PHHWLhw4aDXt337dhQXF2PGjBmYPXs2enp6UFNTg61bt6K8vByfffYZAgICnD7e0qVL8e233+KBBx7AiBEjMH78+EEZ57PPPovCwkJ8/PHH6OzsdDgmrVaLtLQ0dHV1Yd68eVCpVGhubsaePXuwd+9e5OXl4YEHHhiUsdHQwNCiW+7YsWM4cOAAAODee+/ts88TTzyBp556Crt370ZqaioAYOfOnbh69SqWLl0Ki+Xm73U+cuRInzP35s6d69TzphdffBHHjx9HWloaXnvtNXh4eAAAVq9ejTlz5uDll1/GvHnzEB0djVmzZmHWrFnYv38/zpw5gxUrViAsLMzhOXpdvXoVeXl58PX1xaJFiwAAaWlpyM3Nxfbt210KLUf1r1y5Eps3b4aXl5fN9pycHKxcuRJvvvkmVq5c6fT5zp49iwMHDgz6LbuIiAiMGTMG586dQ01NDWbNmnXDvgaDAU8++SS8vb2xd+9eTJ482bqtrq4ODzzwAP7jP/4D33zzDXx8fAZ1nOQ+DC0adL0/PHt6etDQ0IA9e/agq6sLCxYssN4mvN6CBQswcuRIbN++3RpaO3bswIQJEzBz5kyHU53/8Y9/4B//+Iddu1wudxha3d3dKCwshK+vLzIyMqyBBQBjxozBH/7wB6xduxY7duxAdHT0TY/ljOLiYrS3tyM1NdV6JTF9+nRMmDABn332GRoaGvoVgoDj+m80wWPZsmV44YUX8Nlnn/UrtNatW3fLnjGFhITg3LlzaG9vv2m//Px8fP/998jMzLQJLACYNGkSli5diq1bt2Lfvn2YN2/eLRkr/esxtGjQ/fWvfwUAeHh4ICAgAD/96U/xyCOPYNmyZTfcx8fHB6mpqdi6dSt0Oh1aW1uh0+mwYcMGp875q1/9Cn/7299cGu+JEydw6dIl3HfffbjzzjvttickJAAAvvnmG5eOf73eCRhpaWk27RqNBhs2bEBubm6/J2Q4qr+7uxs5OTkoKirCsWPH0NnZCbPZbN3eO6nEWffdd1+/+rvix7889OXzzz8HcG2iT19XmSdPngRw7d+XoSUdDC0adK6uC3riiSewdetW7NixA62trfD29savfvWrwR1cHzo6OgDceBZg77Tv3n4DUV9fj/3792PcuHF2a7pSU1OxadMmvP3223j22Wchkw3e/55PPvkkiouLMX78eKSkpGD06NEYNmwYAOBvf/sbrly50q/j3Yqp8L1aWloAwOGV3Pfffw8AyM3NvWm/H374YXAGRkMCQ4uGjEmTJuH+++9HXl4efvjhB6SkpCA4OPiWnzcwMBAAcP78+T63t7a22vQbiN4JGI2NjTecXNLS0oKPPvoIDz300IDPBwBff/01iouLER8fj3fffRfe3t7WbWazGdnZ2f0+pqOrIFedOnUK586dg0wms1urdb3ef499+/Y57EvSwdCiIWXp0qVYsWIFAPS52PhWmDhxInx9fVFbW4sLFy7Y/Yav1WoBYMA/GK9evYqdO3fCw8MDGo0Gnp72K04uXLiAjz76CNu3bx+00Dp16hQAYP78+TaBBVxbgnD58uVBOc9gyMzMBHBtrI5mDsbExODDDz/EwYMHGVq3EYYWDSmLFi1CQEAAPDw8rM+SbjVvb28sWbIEOTk5yMjIwJYtW6xXEs3NzXj11Vfh4eGBxx57bEDn2bNnD9rb2xEXF4f//u//7rNPd3c3pkyZ4vKEjL70TsLYv38/fvOb31jb29rasGrVqgEffzD0Li4uLCyEQqGwW6fVl8ceewxZWVl4+eWXER0djalTp9pst1gsOHToEKKjo623Qkl8DC0aUoYPH44FCxb8y8/7wgsv4ODBg9ixYwe+/fZbJCQkWNdpXbx4EWvWrBnw5IO///3vAHDTCSne3t7QaDT4r//6L+zYsQN/+tOfBnRO4Noyg+nTp2PPnj1ISkrC9OnTcf78eezduxdqtRpKpXLA5+iP3kkTZrPZ+hqngwcP4vLly9bXOEVERDg8zogRI7Bjxw489thjSEpKQlxcHCZPngxvb2+cO3cOX375Jc6ePYvvvvuOoSUhDC0iXHsjQ2lpKbZs2YIPP/wQW7duhY+PD+655x785je/GfCtut4JGEFBQQ5DeenSpdiyZQvefvttPPfccwOekOHl5YWdO3di06ZNKCsrw+uvvw6lUomlS5di1apVmDZt2oCO31+9s0u9vb3h7++P0NBQPPzww0hJSUFycrLdLcybiYuLw4EDB/Daa6/h008/xRdffAGZTIbRo0cjJiYGL7zwwqA8i6Shw0Ov19981SYREdEQwXcPEhGRMJy+71BRUYHS0lIYDAaEhoZiyZIlmDhxYp99m5qakJeXh6amJly+fBkKhQIxMTF46KGHbG511NXVobCwEE1NTVAoFJg3b96/7OE7ERGJx6nQqq6uRkFBATQaDdRqNSoqKpCdnY2MjIw+FwDKZDLcf//9GDduHHx9fXHmzBns2LEDZrMZ//Zv/wbg2syl7OxszJgxA7/+9a+h0+mQl5eHgICAQXlVDhERSY9ToVVeXo7Y2FjExcUBuPa6maNHj0Kr1WLx4sV2/UeNGmXzdoGgoCDU1dVBp9NZ27RaLRQKBTQaDQBAqVTi9OnTKCsrY2gREVGfHD7TMplMaGhoQGRkpE17ZGQk6uvrnTrJ+fPncfToUZvbiadOnbI75pQpU9DQ0HDL/qYQERGJzeGVltFohNlstps2GhgYiGPHjt1038zMTGsIzZo1y/pnGIBrf1bgrrvusjtmT08PjEZjv/6gHhER3R5cXgBisVgcvn/s6aefRldXF86cOYN3330Xn3zyCebPn2/dfv3+vX8z6Va914yIiMTm8Pagv78/PD097d5w3dnZ6XDR3siRIxEaGopp06bhl7/8Jfbs2YOenh4A1/7Oj8FgsDuml5cX/Pz8+luHsH78nE8qpFgTwLpEIsWaAOnW1R8OQ0smkyEsLAy1tbU27bW1tVCpVE6fyGw2W7+Aa3+h9Prbi7W1tQgLCxvUP8lARETS4dTi4sTERFRVVaGyshLNzc3Iz8+HwWBAfHw8AKCoqAhZWVnW/gcPHsSXX36J5uZmtLW1obq6Gu+//z6io6Otr2iJj4/HxYsXkZ+fj+bmZlRWVqKqqgpJSUm3oEwiIpICpy5pYmJiYDQaUVJSYl1cnJ6ebl2jpdfr0dbWZu3v6emJjz/+2Pp3iEaOHImEhAQkJiZa+wQHByM9PR2FhYXQarWQy+VITU3ldHciIrohvnvQzXQ6HdRqtbuHMaikWBPAukQixZoA6dbVH3z3IBERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDJmzHSsqKlBaWgqDwYDQ0FAsWbIEEydO7LNvXV0dysvL8d133+Hy5csIDg7G3LlzMXPmTJs+mzdvttt3w4YNUCqVLpRCRERS51RoVVdXo6CgABqNBmq1GhUVFcjOzkZGRgaCgoLs+p88eRJjxoxBcnIy5HI5jh49itzcXHh7e2PatGk2fTMyMuDn52f9HBAQMMCSiIhIqpwKrfLycsTGxiIuLg4AoNFocPToUWi1WixevNiuf0pKis3nhIQEHD9+HIcPH7YLrYCAAAYVERE5xWFomUwmNDQ0ICkpyaY9MjIS9fX1Tp+oq6sLI0aMsGt/6aWXYDKZoFQqkZKSgsmTJzt9TCIiur04DC2j0Qiz2YzAwECb9sDAQBw7dsypk3zzzTc4fvw41q5da22Ty+VIS0tDeHg4TCYTDh06hFdeeQWrVq264bMyANDpdE6dUySsSRysSxxSrAmQXl1qtbpf/Z2eiHE9i8UCDw8Ph/1OnjyJN998E6mpqQgPD7e2h4SEICQkxPpZpVKhvb0dZWVlNw2t/hY41Ol0OtYkCNYlDinWBEi3rv5wOOXd398fnp6e6OjosGnv7Oy0u/q6nk6nw5YtW/Dwww8jISHB4WDCw8PR2trqsB8REd2eHIaWTCZDWFgYamtrbdpra2uhUqluuN+JEyewZcsWLFy4EHPnznVqMGfOnIFcLneqLxER3X6cuj2YmJiIbdu2ITw8HBMmTIBWq4XBYEB8fDwAoKioCKdPn8YzzzwD4NoarOzsbCQkJGDatGkwGAwAAE9PT+tMwb179yIoKAihoaEwmUz4/PPPUVNTgxUrVtyKOomISAKcCq2YmBgYjUaUlJRYFxenp6db12jp9Xq0tbVZ+x84cABXr15FWVkZysrKrO1BQUHIzMwEcG1W4q5du6DX6+Ht7W09ZlRU1GDWR0REEuKh1+st7h7E7UyKD1alWBPAukQixZoA6dbVH3z3IBERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMGTOdqyoqEBpaSkMBgNCQ0OxZMkSTJw4sc++dXV1KC8vx3fffYfLly8jODgYc+fOxcyZM+36FRYWoqmpCQqFAvPmzUNCQsKACiIiIulyKrSqq6tRUFAAjUYDtVqNiooKZGdnIyMjA0FBQXb9T548iTFjxiA5ORlyuRxHjx5Fbm4uvL29MW3aNABAW1sbsrOzMWPGDPz617+GTqdDXl4eAgICEB0dPbhVEhGRJDgVWuXl5YiNjUVcXBwAQKPR4OjRo9BqtVi8eLFd/5SUFJvPCQkJOH78OA4fPmwNLa1WC4VCAY1GAwBQKpU4ffo0ysrKGFpERNQnh8+0TCYTGhoaEBkZadMeGRmJ+vp6p0/U1dUFX19f6+dTp07ZHXPKlCloaGiAyWRy+rhERHT7cBhaRqMRZrMZgYGBNu2BgYEwGAxOneSbb77B8ePHrVdqAGAwGPo8Zk9PD4xGo1PHJSKi24vTEzGuZ7FY4OHh4bDfyZMn8eabbyI1NRXh4eE2267f32Kx9Nn+YzqdzoXRDm2sSRysSxxSrAmQXl1qtbpf/R2Glr+/Pzw9PdHR0WHT3tnZaXeldD2dTofs7Gw8/PDDdrMC5XK53ZVaZ2cnvLy84Ofnd8Nj9rfAoU6n07EmQbAucUixJkC6dfWHw9uDMpkMYWFhqK2ttWmvra2FSqW64X4nTpzAli1bsHDhQsydO9due0REBI4dO2Z3zLCwMMhkLl8AEhGRhDm1uDgxMRFVVVWorKxEc3Mz8vPzYTAYEB8fDwAoKipCVlaWtX9dXR22bNmC+Ph4TJs2DQaDAQaDAZ2dndY+8fHxuHjxIvLz89Hc3IzKykpUVVUhKSlpkEskIiKpcOqSJiYmBkajESUlJdbFxenp6dY1Wnq9Hm1tbdb+Bw4cwNWrV1FWVoaysjJre1BQEDIzMwEAwcHBSE9PR2FhIbRaLeRyOVJTUzndnYiIbshDr9db3D2I25kU71FLsSaAdYlEijUB0q2rP/juQSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkPLTXbV/4CowhZM3X8HogpbsKv+B3cPacCkWBPAukQixZoA6dblCg+9Xm9x9yBuN7vqf0D6AQMu9/zzP/0dXh7IniHHIyo/N47MdVKsCWBdIpFiTYB063IVQ8sNogpbcOaHHrv2n/h54cijIW4Y0cBJsSaAdYlEijUB0q3LVbw96AZn+/gGvFm7CKRYE8C6RCLFmgDp1uUqhpYbjPXz6le7CKRYE8C6RCLFmgDp1uUqhpYbrI8OsGu7w8ujz3ZRSLEmgHWJRIo1AdKty1UMLTfo6+Gp6A9VpVgTwLpEIsWaAOnW5SqG1hAhxW9AKdYEsC6RSLEmQLp1OYOhRUREwmBoERGRMBhaREQkDJmzHSsqKlBaWgqDwYDQ0FAsWbIEEydO7LNvd3c3cnNz0djYiJaWFqhUKqxevdqmT11dHTZv3my374YNG6BUKvtZBhER3Q6cCq3q6moUFBRAo9FArVajoqIC2dnZyMjIQFBQkF1/s9kMb29vzJkzB0eOHMGlS5dueOyMjAz4+f3zoWJAwO05jZOIiBxz6vZgeXk5YmNjERcXB6VSCY1GA7lcDq1W22d/Hx8fPP7444iLi8OIESNueuyAgADI5XLrl6cn71gSEVHfHF5pmUwmNDQ0ICkpyaY9MjIS9fX1Ax7ASy+9BJPJBKVSiZSUFEyePHnAxyQiImlyeFljNBphNpsRGBho0x4YGAiDweDyieVyOdLS0rBixQqsWLECISEheOWVV3DixAmXj0lERNLm9ESM61ksFnh4eLh84pCQEISE/PMNxSqVCu3t7SgrK7vhBA8A0Ol0Lp9zaPG1+SSNuqRYE8C6RCLFmgDp1gWo1ep+9XcYWv7+/vD09ERHR4dNe2dnp93V10CFh4ejurr6pn36W+CQtf+czUdJ1CXFmgDWJRIp1gRIty4XOLw9KJPJEBYWhtraWpv22tpaqFSqQR3MmTNnIJfLB/WYREQkHU7dHkxMTMS2bdsQHh6OCRMmQKvVwmAwID4+HgBQVFSE06dP45lnnrHu09TUBJPJBKPRiCtXrqCxsREAMG7cOADA3r17ERQUhNDQUJhMJnz++eeoqanBihUrBrtGIiKSCKdCKyYmBkajESUlJdbFxenp6dY1Wnq9Hm1tbTb7ZGdn48KFC9bPGzduBAC88cYbAK7NSty1axf0ej28vb2tx4yKihqUwoiISHqcnogxe/ZszJ49u89ty5cvt2vLzMy86fGSk5ORnJzs7OmJiIj47kEiIhIHQ4uIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgyZztWVFSgtLQUBoMBoaGhWLJkCSZOnNhn3+7ubuTm5qKxsREtLS1QqVRYvXq1Xb+6ujoUFhaiqakJCoUC8+bNQ0JCgsvFEBGRtDl1pVVdXY2CggLMnz8f69evh0qlQnZ2Ni5cuNBnf7PZDG9vb8yZMwdRUVF99mlra0N2djZUKhXWr1+PBx98EPn5+fjqq69cr4aIiCTNqdAqLy9HbGws4uLioFQqodFoIJfLodVq++zv4+ODxx9/HHFxcRgxYkSffbRaLRQKBTQaDZRKJeLi4nD//fejrKzM9WqIiEjSHIaWyWRCQ0MDIiMjbdojIyNRX1/v8olPnTpld8wpU6agoaEBJpPJ5eMSEZF0OQwto9EIs9mMwMBAm/bAwEAYDAaXT2wwGPo8Zk9PD4xGo8vHJSIi6XJ6Isb1LBYLPDw8BnTy6/e3WCx9tv+YTqcb0DmHDl+bT9KoS4o1AaxLJFKsCZBuXYBare5Xf4eh5e/vD09PT3R0dNi0d3Z22l0p9YdcLre7Uuvs7ISXlxf8/PxuuF9/Cxyy9p+z+SiJuqRYE8C6RCLFmgDp1uUCh7cHZTIZwsLCUFtba9NeW1sLlUrl8okjIiJw7Ngxu2OGhYVBJnP5ApCIiCTMqdmDiYmJqKqqQmVlJZqbm5Gfnw+DwYD4+HgAQFFREbKysmz2aWpqQmNjI4xGI65cuYLGxkY0NjZat8fHx+PixYvIz89Hc3MzKisrUVVVhaSkpEEsj4iIpMSpS5qYmBgYjUaUlJRYFxenp6cjKCgIAKDX69HW1mazz/XruDZu3AgAeOONNwAAwcHBSE9PR2FhIbRaLeRyOVJTUxEdHT0ohRERkfQ4fR9u9uzZmD17dp/bli9fbteWmZnp8JiTJk3Cn/70J2eHQEREtzm+e5CIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLSIiEobM2Y4VFRUoLS2FwWBAaGgolixZgokTJ96w/9mzZ7Fz506cPn0afn5+iIuLw4IFC+Dh4QEAqKurw+bNm+3227BhA5RKpQulEBGR1DkVWtXV1SgoKIBGo4FarUZFRQWys7ORkZGBoKAgu/6XL1/Gq6++CrVajeeffx4tLS3IycmBj48PkpKSbPpmZGTAz8/P+jkgIGCAJRERkVQ5dXuwvLwcsbGxiIuLg1KphEajgVwuh1ar7bP/559/jqtXr2L58uUYM2YMoqOjkZycjPLyclgsFpu+AQEBkMvl1i9PT96xJCKivjm80jKZTGhoaLC7QoqMjER9fX2f+9TX10OtVmPYsGHWtilTpmD37t1ob29HcHCwtf2ll16CyWSCUqlESkoKJk+e7GotREQkcQ5Dy2g0wmw2IzAw0KY9MDAQx44d63Ofjo4OjBgxwq5/77bg4GDI5XKkpaUhPDwcJpMJhw4dwiuvvIJVq1bd9FmZTqdzWJQYfG0+SaMuKdYEsC6RSLEmQLp1AWq1ul/9nZ6IcT2LxWKdVOGKkJAQhISEWD+rVCq0t7ejrKzspqHV3wKHrP3nbD5Koi4p1gSwLpFIsSZAunW5wOEDJH9/f3h6eqKjo8OmvbOz0+7qq1dgYKBd/97PN9oHAMLDw9Ha2upw0EREdHtyGFoymQxhYWGora21aa+trYVKpepzH5VKBZ1Oh+7ubpv+CoUCd9555w3PdebMGcjlcmfHTkREtxmnpuolJiaiqqoKlZWVaG5uRn5+PgwGA+Lj4wEARUVFyMrKsvafOnUqhg0bhpycHJw7dw6HDx/GJ598gsTEROstxb179+Lrr79Ga2srzp07h6KiItTU1GDOnDm3oEwiIpICp55pxcTEwGg0oqSkxLq4OD093bpGS6/Xo62tzdrf19cXK1euRF5eHjZt2gQ/Pz8kJiYiMTHR2sdkMmHXrl3Q6/Xw9va2HjMqKmqQSyQiIqlweiLG7NmzMXv27D63LV++3K5t7NixWLNmzQ2Pl5ycjOTkZGdPT0RExHcPEhGROBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCkDnbsaKiAqWlpTAYDAgNDcWSJUswceLEG/Y/e/Ysdu7cidOnT8PPzw9xcXFYsGABPDw8rH3q6upQWFiIpqYmKBQKzJs3DwkJCQMqiIiIpMupK63q6moUFBRg/vz5WL9+PVQqFbKzs3HhwoU++1++fBmvvvoqAgIC8PzzzyM1NRWlpaUoLy+39mlra0N2djZUKhXWr1+PBx98EPn5+fjqq68GpzIiIpIcD71eb3HU6c9//jPGjh2LpUuXWtuef/55REdHY/HixXb99+3bh/feew9ZWVkYNmwYAKC4uBharRb/+Z//CQ8PD7z77rv4+uuv8dJLL1n32759O5qamvDcc88NRm1DmiLnHADg9x7r3DwSIhLF/7P8GQCgf3KMm0fiPg5vD5pMJjQ0NCApKcmmPTIyEvX19X3uU19fD7VabQ0sAJgyZQp2796N9vZ2BAcH49SpU4iMjLTZb8qUKTh48CBMJhNkMqfvXArpn9902906DiISx0Z3D2AIcHh70Gg0wmw2IzAw0KY9MDAQBoOhz306Ojr67N+7DQAMBkOffXp6emA0Gp2vgIiIbhsuzx60WCw2kypccf3+Foulz3YiIiLAidDy9/eHp6en9QqpV2dnp92VUq/AwEC7/r2fe/eRy+V2V2qdnZ3w8vKCn5+f8xUQEdFtw2FoyWQyhIWFoba21qa9trYWKpWqz31UKhV0Oh26u7tt+isUCtx5550AgIiICBw7dszumGFhYZJ/nkVERK5x6vZgYmIiqqqqUFlZiebmZuTn58NgMCA+Ph4AUFRUhKysLGv/qVOnYtiwYcjJycG5c+dw+PBhfPLJJ0hMTLTe+ouPj8fFixeRn5+P5uZmVFZWoqqqym7CBxERUS+nprwDN19c/NZbb+HEiRPIzMy09j979izy8vJsFhcvXLjwhouL5XI5kpOT7RYX93dRswhOnDiBsrIyNDQ0QK/XY9myZZgxY4a7hzUgH330EQ4fPozW1lbIZDJERERg8eLFGDNG7Km5FRUV0Gq11jWJoaGhSElJwT333OPmkQ2ekpISfPDBB5g9ezY0Go27h+OyDz/8EHv27LFpCwwMtPmFWlR6vR5FRUU4cuQIurq6EBwcjLS0NEyaNMndQ3PJs88+2+c636ioKKSnp990X6dDyx2qq6uxbds2aDQaqNVqVFRUoKqqChkZGQgKCnL38Fx25MgR6HQ6hIWF4a233oJGoxE+tF599VVMnToV48ePh8Viwe7du3Hq1Cls2LBB6GeUNTU18PLywujRo2E2m3Hw4EGUlpbij3/8I8aOHevu4Q1YfX093njjDdxxxx1Qq9XCh1Z1dTVWrVplbfP09ERAQIAbRzVwly5dwsaNGzFhwgTMmTMH/v7+aG9vh0KhgFKpdPfwXNLZ2Qmz2Wz9bDAYsGnTJixbtgyxsbE33XdIv3uwvLwcsbGxiIuLg1KphEajgVwuh1ardffQBiQqKgqLFy9GdHS0ZGZKrly5EjNmzMCYMWMwduxYPPXUU+js7MTJkyfdPbQB+dnPfoaoqCiMGjUKISEhWLRoEXx8fG64RlEkly5dwrZt2/DEE0/A19fX3cMZFJ6enpDL5dYv0QMLAD755BPI5XI89dRTCA8PR3BwMO666y5hAwsAAgICbP6djhw5guHDhyM6OtrhvkN2xoMri5pp6Ojq6oLFYpHMD0MAMJvN+PLLL3HlypUbTkISSW5uLu69917cddddKC4udvdwBkV7eztWr14NLy8vREREYNGiRQgODnb3sAakpqYGU6ZMweuvv466ujrI5XLMmjULs2fPlsQvvRaLBfv378f06dPh4+PjsP+QDa2bLWq+ftYhDT35+fn4yU9+Iokf7mfPnkVmZia6u7vh4+OD3/3ud8LfGvy///s/nD9/Hk899ZS7hzJowsPDsWzZMiiVSnR0dKCkpASZmZnIyMiAv7+/u4fnsra2Nuzbtw+JiYl48MEHcebMGezcuRMAMGfOHDePbuBqa2vR3t6OmTNnOtV/yIbWjQzGoma6tQoKCnDy5EmsXbsWnp5D+g60U0JCQrB+/XpcunQJhw8fRk5ODlatWiXsJJOWlha8//77WLNmjaSWl0RFRdl8joiIwLp164SflWyxWDB+/Hjre17HjRuH1tZWVFRUSCK0KisrMX78eIwbN86p/kP2O9aVRc3kfgUFBaiursYzzzwj/G2ZXjKZDKNGjQIAjB8/Ht999x3Ky8uxbNky9w7MRfX19TAajXjxxRetbWazGTqdDlqtFq+99hq8vb3dN8BBMnz4cISGhuL8+fPuHsqAyOVyu+dXSqUSn376qZtGNHg6OjpQU1PTrwlAQza0fryo+b777rO219bWOvWwjv718vPz8cUXX2D16tVCPyR2xGw2w2QyuXsYLvv5z3+O8ePH27Tl5ORg9OjRmD9/vmSuvrq7u9HS0iLstPBeEyZMQEtLi01ba2ur0DOoex04cAAymQxTp051ep8h/d2ZmJiIbdu2ITw8HBMmTIBWq7VZ1Cyqrq4u629/FosF33//PRobG+Hn5yfsN+I777yDQ4cO4Xe/+x18fX2tr+jy8fHB8OHD3Tw617333nuIiorCyJEj0dXVhS+++AInTpzA73//e3cPzWW+vr52E2R8fHzg5+cn7C1PANi1axfuuecejBw5Ep2dnSguLsaVK1ccTqEe6ubOnYu//vWvKCkpQUxMDBobG/HZZ59h0aJF7h7agPROwJg6dWq/fkYM6XVagDQXF9fV1WHz5s127ffffz+WL1/uhhEN3L//+7/32b5w4UI89NBD/+LRDJ633noLdXV16OjowB133IGxY8ciKSkJd999t7uHNqhefvlljBkzRuh1Wv/7v/+LEydOwGg0IiAgABEREXj44YcRGhrq7qEN2Lfffov3338fLS0tGDlyJObMmYM5c+YI/Xz/+PHjyMrKwrp16xAeHu70fkM+tIiIiHqJP7WLiIhuGwwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBj/H+2JVEuNNiVvAAAAAElFTkSuQmCC\n",
 59 |       "text/plain": [
 60 |        "<Figure size 432x288 with 1 Axes>"
 61 |       ]
 62 |      },
 63 |      "metadata": {},
 64 |      "output_type": "display_data"
 65 |     }
 66 |    ],
 67 |    "source": [
 68 |     "pmf = 1/6*np.ones(6)\n",
 69 |     "diceN = np.arange(1,7)\n",
 70 |     "plt.stem(diceN, pmf,use_line_collection = True)\n",
 71 |     "plt.axis([0, 7, 0, .3])\n",
 72 |     "plt.title('PMF of A Fair Die')\n",
 73 |     "plt.show()"
 74 |    ]
 75 |   },
 76 |   {
 77 |    "cell_type": "markdown",
 78 |    "metadata": {},
 79 |    "source": [
 80 |     "## <font face=\"gotham\" color=\"purple\"> Cumulative Distribution Function"
 81 |    ]
 82 |   },
 83 |   {
 84 |    "cell_type": "markdown",
 85 |    "metadata": {},
 86 |    "source": [
 87 |     "The **cumulative distribution function** (CDF) of random variable $X$ is defined as\n",
 88 |     "$$\n",
 89 |     "F_{X}(x)=P(X \\leq x), \\text { for all } x \\in \\mathbb{R}\n",
 90 |     "$$"
 91 |    ]
 92 |   },
 93 |   {
 94 |    "cell_type": "markdown",
 95 |    "metadata": {},
 96 |    "source": [
 97 |     "The CDF of the die example is "
 98 |    ]
 99 |   },
100 |   {
101 |    "cell_type": "code",
102 |    "execution_count": 11,
103 |    "metadata": {},
104 |    "outputs": [
105 |     {
106 |      "data": {
107 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaQAAAEeCAYAAADFHWEmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAkb0lEQVR4nO3de1iUZf4/8DcHRU4D64mjIiCa5JgbQgkJYk7lqTy1KF6bLtZWtuK6lpoWm5alrawb1uVaKqGpgIkrhldARYhgSfrziMqICcqKgn6Z4Wgc5veHMTUxODMw7NwzvF/X5R9zP/czz+fTmG+e57mfwaq6uloFIiIiE7M2dQFEREQAA4mIiATBQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIurA/v37ERERgUGDBsHV1RUvv/yyqUsyiry8PLi6uuK9994zdSkAgN27d8PV1RW7d+82dSlkYramLoDMy+XLl7Ft2zbk5eXh2rVraGhogKurK0aOHIlJkyZhzpw5kEgk6vlSqRTXrl1Tv7a1tYWTkxPc3d0xcuRIyGQyPP3007C3t293rLy8PEybNu2+9Rw5cgSjRo0yXoM/O378OJ5//nm4ubnhueeeg5OTE6RSqUHvMW7cOJw9exbDhw/H999/b3ANpuzfUF35nInaMJBIb/Hx8Vi3bh1aW1sRFBSEqKgoSCQSVFVV4dixY1i+fDnWr1+PK1eutNv3pZdegouLC1QqFWpqanDlyhVkZWXh888/x9q1a/HRRx9h/PjxWo87aNAgREdHa93m5uZmzBbVsrKyoFKpsG7dOsyaNcvg/U+cOIGzZ8/CysoKly5dQkFBAUJDQztVi7H7DwoKwvHjx9GvX79O1XM/nfmcp06diuDg4G77LMl8MJBIL//617/w9ttvw8vLCzt27MAjjzzSbk5+fj6WL1+udf+XX34ZPj4+GmP19fVISEjAhg0bEBUVhfT0dK3vO3jwYLz++uvGaURPN27cAAAMHDiwU/snJiYCAJYtW4aNGzfi008/7XQgGbt/BwcHDBs2zGjv92ud+ZxdXFzg4uLSLfWQeeE9JNKprKwM69atQ69evZCcnKw1NAAgLCwM33zzjd7v6+DggJUrV2LZsmW4e/duh2FmDHfv3sUHH3yAsLAweHh4wNvbGxMnTsTOnTuhUv3y7Vm/vZ8xbdo0uLq6wtXVFaWlpXodS6lU4sCBA/D29sbrr78OX19fpKen4//+7/+6pbfLly/jrbfewvjx4+Hv74+BAwdi5MiRiI2N1biM1qaje0hTpkyBq6srrl69is2bN+PRRx+Fm5tbh2dn+tL1Od/vHtLNmzexcuVKPPzww3Bzc4OPjw9mzJiB3NzcLtVEYmIgkU67d+9GU1MTpk6dqvM+ip2dncHvv2TJEtjb2+P06dO4dOlSZ8vsUFNTE2bNmoW///3vuHv3LmJiYjB37lyUl5cjNjZWY7GCVCrFihUrMHLkSADA3LlzsWLFCqxYsULvn+JTU1NRV1eHOXPmwMbGBtHR0WhsbMSePXuM3hsAHDp0CDt27ICXlxdmzZqFP//5zxg+fDh27dqFCRMmoLy83KD3W758OTZu3IhRo0bhxRdfxOjRo41Sp6Gf8/nz5zFu3Dhs3boVvr6+eOGFFzBlyhScPHkS06dPx65du4xSF4mDl+xIp2PHjgEAIiMju+X9nZ2dMXr0aBw7dgw//PADhg8frrG9rKxM64qw4OBgTJw4Uef7b968GUePHsWECROQnJyM3r17AwDeeOMNPPXUU0hOTsZTTz2F6dOnY9SoURg1ahTKyspw7tw5REdHY9y4cQb18+mnnwIA5s2bB+BeqL333ntISkrCK6+8YtB7Abr7j4qKwqJFi9r9MJCdnY2oqChs3LgRmzZt0vt4Z8+exZEjR9pdeusqXZ/zr7W0tGD+/PlQKBQ4dOgQHnvsMfW2iooKPP7443jttdfw5JNPdvqyKomHgUQ63bx5EwDg6enZbcdwd3cHAFRVVbXbdu3aNWzYsKHd+EsvvaRXIH322WcAgHXr1qnDCLh37yIuLg5z585FUlISpk+f3snqf/HDDz/g3LlzCA0Nha+vLwDA29sbERERyMnJQX5+PsLCwgx6T139d/S5yGQyPPDAAwZdRgWAxYsXGz2M2tzvc/61rKwsXL58GYsWLdIIo7b3WLx4MVauXImDBw/ihRde6JZa6X+PgUQ6td1jsbKy6vZjaTtGWFgYMjIyOvV+bSu9Bg4ciBEjRrTbHhERAQA4ffp0p97/t9oWM7SdHbWZN28ecnJykJSUZHAg6epfpVIhNTUVe/bswblz51BdXY2Wlhb19l+HsD7GjBlj0PzO0PV3qW2Z/PXr17WeHbat5CwuLjZ+cWQyDCTSyd3dHcXFxQbfizBERUUFABh9KbJSqQTQ8Wo5BwcHSCQS9byuHuvAgQNwdHTEM888o7Ft6tSpcHFxwcGDB7F+/Xr07du3y8drs2rVKmzZsgXu7u54/PHH4eHhgT59+gAA9uzZo3Vhw/105yUwfT/nO3fuAADS09ORnp7e4by6ujrjFUcmx0AincaOHYsjR44gNzcXzz33nNHfX6lU4tSpUwDu3RcxpraHdG/duqV1e319PZRKpVECIiUlBfX19QDuXabryJ49e/CXv/yly8cDgMrKSmzduhWBgYHIzMyEs7Ozxvb9+/cb/J7ddSZsyOfc9rnt3LkTTz/9dLfUQ+JhIJFO8+bNwz//+U+kp6ejqKgIgYGBHc69e/euwSvtEhIS0NDQgNGjRxv9+RhnZ2f4+fnhypUruHjxIh544AGN7UeOHAEAo6wkS0pKAgBMnz69XTAA9/7bpKamYufOnUYLpKtXr6K1tRWRkZHtjlleXo6rV68a5TjGYMjn3BZYx44dYyD1IFz2TToNHjwYq1evRlNTE/7whz+gsLBQ67zvvvtOr0UGberr67FhwwbEx8fDzs5O6417Y/jjH/8I4N6quqamJvW4UqnE2rVrAaDLZ36FhYU4d+4cfH19kZiYiM2bN7f78/HHH+Ohhx5CcXExjh492qXjtRk8eDCAe//tf33fqLa2FkuWLEFzc7NRjtMVnfmcJ0+eDD8/PyQmJuLw4cNa55w+fVp9aY8sA8+QSC9//etf0dzcjHfffRcymQxjxozBww8/DGdnZ9y+fRvHjx9HUVFRh/cGtmzZov5KmdraWly5cgUFBQVQKpXw9PTERx991OEDt131yiuv4KuvvsJXX32F0NBQPPnkk2hqasKhQ4fw3//+F3PmzOnyCru2pd7z58+/7yWv+fPn429/+xuSkpLarR7rDDc3N8yaNQv79+/HuHHjEBkZCaVSiZycHPTp0wdSqRRnz57t8nH0ZazPuVevXvjss88wc+ZMREdHY8yYMXjooYfg6OiI8vJynDlzBnK5HEeOHDHq/TgyLQYS6e3VV1/F9OnTsW3bNhw5cgTJycmor6+Hq6srAgMDsWHDhg6f6v/3v/8NALCxsVF/6aZMJlN/6aaDg0O31d27d2+kpaVhy5YtSE1NxbZt22BtbY0RI0Zg5cqV6jOozlIoFDhw4ABsbW11fqvB7Nmz8eabbyI9PR0bNmwwyj+mmzdvxpAhQ5CWloZt27ahf//+mDRpElatWtXl3gxlzM85MDAQ+fn52LJlCw4fPoy9e/dCpVLBzc0NDzzwABYvXoyAgIDuaoVMwKq6ulqlexoREVH34j0kIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkLqRXC43dQlGZ4k9AezLnFhiT4Dl9mUIBhIREQmBgUREREJgIBERkRAYSEREJAQGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREKw1WdScXExsrKyUFpaiurqaixYsABhYWH33ef69evYu3cvfvzxRzg6OiI8PBxTp06FlZWVUQonIiLLotcZ0t27d+Hp6Yk5c+agd+/eOuc3NDRg06ZNcHZ2xurVqzFnzhxkZmYiOzu7ywUTEVmSfSV1kKZWIOSoPaSpFdhXUmfqkkxGrzMkqVQKqVQKAEhMTNQ5//vvv8dPP/2EmJgY9O7dG15eXrhx4ways7Mhk8l4lkREhHthFJuvQEOLCoAVrtW1IDZfAQB41t/RtMWZQLfcQyopKUFAQIDG2dSDDz6I6upqVFVVdcchiYjMztoTNT+H0S8aWlRYe6LGRBWZll5nSIZSKpX43e9+pzEmkUjU2wYMGKB1P0v8Fb7syXywL/NhKT1dr7MH0P6K0fW6ZovoMSAgwKD53RJInWVo8aKTy+XsyUywL/NhST15/78KXKtraT/uaGsxPRqiWy7ZSSQSKJVKjbG2121nSkREPV1ckHO7MXsbK63jPUG3BJK/vz/kcjmamprUY0VFRXB1dUX//v2745BERGZH28KFhDCXHrmgAdAzkBobG1FWVoaysjKoVCrcuXMHZWVluH37NgAgLS0N8fHx6vkhISHo3bs3EhMTUV5ejpMnT+LLL7/kCjsiIh16ahgBet5DKi0txcaNG9Wv09PTkZ6ejrFjxyImJgbV1dWorKxUb3dwcMDSpUuxZ88evPPOO3B0dIRMJoNMJjN+B0REZBH0CqThw4fjk08+6XB7TExMuzFvb28sX76885UREVGPwu+yIyIiITCQiIhICAwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIjIL+0rqIE2tQMhRe0hTK7CvpM7UJZGR2Zq6ACIiXfaV1CE2X4GGFhUAK1yra0FsvgIA8Ky/o2mLI6PhGRIRCW/tiZqfw+gXDS0qrD1RY6KKqDswkIhIeNfrWgwaJ/PEQCIi4Xk72hg0TuZJ73tIOTk5yMzMhEKhgKenJ6KiojBs2LAO5587dw6HDh1CeXk5evXqBX9/f8yePRvu7u5GKZyIeo64IGe8cKRaY8zexgpxQc6mKYi6hV5nSIWFhUhJScHkyZMRFxcHf39/JCQk4Pbt21rnV1ZW4qOPPkJAQADi4uKwdOlSNDU1ISEhwajFE1HPoG3hQkKYCxc0WBi9Aik7OxuhoaEIDw+Hh4cHoqOj4eLigtzcXK3zy8rK0NLSgpkzZ2LgwIEYPHgwJk2ahMrKStTU8CYkEXUdw8jy6Ayk5uZmlJaWIjAwUGM8MDAQJSUlWvfx8fGBjY0N8vLy0NraisbGRhQUFGDIkCFwduYpNhERtafzHlJtbS1aW1shkUg0xiUSCS5cuKB1n/79+2Pp0qXYunUr9uzZA5VKhUGDBmHJkiXGqZqIiCxOpx+MValUsLKy0rpNoVAgKSkJY8eORUhICBobG3Hw4EFs3boVy5Ytg7W19hMzuVze2XKExZ7MB/sSnYPGK/YlvoCAAIPm6wwkJycnWFtbQ6lUaozX1NS0O2tqk5OTAzs7O8yePVs9tnDhQqxYsQIlJSUdFmlo8aKTy+XsyUywLzNwtFzjJfuyPDrvIdna2sLHxwdFRUUa40VFRfD399e6z08//dTuLKjttUql0rYLERH1cHqtspPJZCgoKEBeXh5u3LiB5ORkKBQKREREAADS0tIQHx+vni+VSlFWVob09HTcvHkTpaWl+PTTT9G3b1/4+Ph0TydERGTW9LqHFBwcjNraWmRkZKgfjI2NjUW/fv0AANXV1aisrFTPHzFiBJ5//nlkZmYiKysLvXr1gp+fH5YsWQI7O7vu6YSIiMya3osaIiMjERkZqXVbTExMu7GQkBCEhIR0vjIiIupR+F12REQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREJgIBERkRAYSEREJAQGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREJgIBFZoH0ldZCmViDkqD2kqRXYV1Jn6pKIdLI1dQFEZFz7SuoQm69AQ4sKgBWu1bUgNl8BAHjW39G0xRHdB8+QiCzM2hM1P4fRLxpaVFh7osZEFRHph4FEZGGu17UYNE4kCgYSkYXxdrQxaJxIFAwkIgsTF+TcbszexkrrOJFIGEhEFkbbwoWEMBcuaCDhMZCIegCGEZkDBhIREQmBgUREREJgIBERkRAYSEREJAQGEhERCUHv77LLyclBZmYmFAoFPD09ERUVhWHDhnU4X6VS4euvv0Zubi6qqqrg6OiIsWPHYtasWUYpnIiILItegVRYWIiUlBRER0cjICAAOTk5SEhIwJo1a9CvXz+t+6SmpuLMmTN49tln4eXlhYaGBigUCqMWT0RElkOvQMrOzkZoaCjCw8MBANHR0Th//jxyc3Mxc+bMdvMrKiqQk5ODv//97/Dw8DBuxUREZJF0BlJzczNKS0vxxBNPaIwHBgaipKRE6z6nTp1C//79ce7cOSQkJEClUmHYsGGYPXs2JBKJcSonIiKLojOQamtr0dra2i5IJBIJLly4oHWfyspK3L59G4WFhfjTn/4EAPj888/x4YcfYuXKlbC21r6WQi6XG1q/8NiT+bCsvhw0XllGb5bYE2C5fQEBAQEGze/0L+hTqVSwsrLqcFtzczNiYmLg7u4OAIiJicGbb76Jq1evws/PT+t+hhYvOrlczp7MhMX1dbRc46VF9GaJPQGW21cn6Fz27eTkBGtrayiVSo3xmpqaDi+/ubi4wMbGRh1GAODm5gYbGxvcuXOniyUTEZEl0hlItra28PHxQVFRkcZ4UVER/P39te4zdOhQtLS04NatW+qxyspKtLS0dLgqj4iIeja9HoyVyWQoKChAXl4ebty4geTkZCgUCkRERAAA0tLSEB8fr54/YsQIDB48GElJSSgrK0NZWRmSkpLg6+sLHx+f7umEiIjMml73kIKDg1FbW4uMjAz1g7GxsbHqs53q6mpUVlaq51tbW2Px4sVITk7G+++/j969e2PEiBGIiorqcEEDERH1bHovaoiMjERkZKTWbTExMe3GXF1d8dJLL3W+MiIi6lF4ukJEREJgIBERkRAYSEREJAQGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhL1aPtK6iBNrUDIUXtIUyuwr6TO1CUR9Vid/o2xROZuX0kdYvMVaGhRAbDCtboWxOYrAADP+juatjiiHohnSNRjrT1R83MY/aKhRYW1J2pMVBFRz8ZAoh7rel2LQeNE1L0YSNRjeTvaGDRORN2LgUQ9VlyQc7sxexsrreNE1P0YSNRjaVu4kBDmwgUNRCbCQCL6FYYRkekwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiHo/SvMc3JykJmZCYVCAU9PT0RFRWHYsGE697t58ybefvttAMCHH37Y+UqJiMii6XWGVFhYiJSUFEyePBlxcXHw9/dHQkICbt++fd/9mpub8fHHH+sVXERE1LPpFUjZ2dkIDQ1FeHg4PDw8EB0dDRcXF+Tm5t53v/3798Pb2xtBQUFGKZaIiCyXzkBqbm5GaWkpAgMDNcYDAwNRUlLS4X5nzpzBmTNnMHfu3K5XSUREFk/nPaTa2lq0trZCIpFojEskEly4cEHrPtXV1di5cydefvll9OnTR+9i5HK53nPNBXsSnYPGK8vpzRL7ssSeAMvtCwgICDBovt6LGn5LpVLByspK67bt27dj/Pjx8Pf3N+g9DS1edHK5nD2J7mi5xkuL6c0S+7LEngDL7asTdAaSk5MTrK2toVQqNcZramranTW1uXjxIoqLi3Ho0CEA98JLpVLhxRdfxLx58xAeHm6E0omIyJLoDCRbW1v4+PigqKgIY8aMUY8XFRV1uFjhrbfe0nh96tQpHD58GKtWrYKrq2uXCiYiIsuk1yU7mUyG7du3w9fXF0OHDkVubi4UCgUiIiIAAGlpafjxxx+xbNkyAICXl5fG/levXoWVlVW7cSIiojZ6BVJwcDBqa2uRkZGhfjA2NjYW/fr1A3BvEUNlZWW3FkpERJZN70UNkZGRiIyM1LotJibmvvuGhYUhLCzMsMqIiKhH4XfZERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEikl30ldZCmViDkqD2kqRXYV1Jn6pKIyMJ0+rvsqOfYV1KH2HwFGlpUAKxwra4FsfkKAMCz/o6mLY6ILAbPkEintSdqfg6jXzS0qLD2RI2JKiIiS8RAIp2u17UYNE5E1BkMJNLJ29HGoHEios5gIJFOcUHO7cbsbay0jhMRdRYDiXTStnAhIcyFCxqIyKgYSNQpDCMiMjYGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREJgIBERkRAYSEREJAQGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQbPWdmJOTg8zMTCgUCnh6eiIqKgrDhg3TOvfSpUvIzs7G1atX0dDQgAEDBmDixIl47LHHjFY4ERFZFr0CqbCwECkpKYiOjkZAQABycnKQkJCANWvWoF+/fu3mX758GV5eXnjqqafg4uKC8+fPY9euXejVqxceeeQRozdBRETmT69Ays7ORmhoKMLDwwEA0dHROH/+PHJzczFz5sx286dMmaLxevz48bh48SJOnjzJQCIiIq103kNqbm5GaWkpAgMDNcYDAwNRUlKi94EaGxvh4OBgeIVERNQj6DxDqq2tRWtrKyQSica4RCLBhQsX9DrI6dOncfHiRaxYseK+8+RyuV7vZ04spyfNHybYl+gssS9L7Amw3L6AgIAAg+brvajht1QqFaysrHTOu3z5MrZt24Y5c+bA19f3vnMNLV50crnccno6Wq7xkn0JzhL7ssSeAMvtqxN0BpKTkxOsra2hVCo1xmtqatqdNf2WXC5HQkICnnnmGYwfP75LhRIRkWXTeQ/J1tYWPj4+KCoq0hgvKiqCv79/h/sVFxfjgw8+wLRp0zBx4sSuV0pERBZNrwdjZTIZCgoKkJeXhxs3biA5ORkKhQIREREAgLS0NMTHx6vnX7p0CR988AEiIiLwyCOPQKFQQKFQoKampnu6EMy+kjpIUysQctQe0tQK7CupM3VJRETC0+seUnBwMGpra5GRkaF+MDY2Nlb9DFJ1dTUqKyvV8/Pz8/HTTz8hKysLWVlZ6vF+/fph/fr1Rm5BLPtK6hCbr0BDiwqAFa7VtSA2XwEAeNbf0bTFEREJTO9FDZGRkYiMjNS6LSYmpt3r3471FGtP1PwcRr9oaFFh7YkaBhIR0X3wu+yM7Hpdi0HjRER0DwPJyLwdbQwaJyKiexhIRhYX5NxuzN7GSus4ERH9goFkZNruEyWEufD+ERGRDgyk/wGGERGRbgwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIiISAgOJiIiEYKvvxJycHGRmZkKhUMDT0xNRUVEYNmxYh/OvX7+OvXv34scff4SjoyPCw8MxdepUWFlZGaVwIiKyLHqdIRUWFiIlJQWTJ09GXFwc/P39kZCQgNu3b2ud39DQgE2bNsHZ2RmrV6/GnDlzkJmZiezsbKMWT0RElsOqurpapWvSu+++C29vbzz33HPqsdWrVyMoKAgzZ85sN//bb7/F/v37ER8fj969ewMAvvjiC+Tm5uL999+3+LMk18RyAMBiq1UmroSIzMVm1bsAgOo/eZm4EtPRecmuubkZpaWleOKJJzTGAwMDUVJSonWfkpISBAQEqMMIAB588EEcPHgQVVVVGDBgQBfLFtsvf6GSTFoHEZmPt01dgAB0XrKrra1Fa2srJBKJxrhEIoFCodC6j1Kp1Dq/bRsREdFvdXqVnUqlsvhLb0RE9L+jM5CcnJxgbW3d7sympqam3VlQG4lE0m5+2+uO9iEiop5NZyDZ2trCx8cHRUVFGuNFRUXw9/fXuo+/vz/kcjmampo05ru6uqJ///5dLJmIiCyRXpfsZDIZCgoKkJeXhxs3biA5ORkKhQIREREAgLS0NMTHx6vnh4SEoHfv3khMTER5eTlOnjyJL7/8EjKZjJf5iIhIK70ejA0ODkZtbS0yMjLUD8bGxsaiX79+AIDq6mpUVlaq5zs4OGDp0qXYs2cP3nnnHTg6OkImk0Emk2m8r6EP25qD4uJiZGVlobS0FNXV1ViwYAHCwsJMXVaXHD58GCdPnsTNmzdha2sLPz8/zJw5E15e5r08NScnB7m5uern6Tw9PTFlyhSMGjXKxJUZT0ZGBv7zn/8gMjIS0dHRpi6n09LT03Ho0CGNMYlEovGDsLmqrq5GWloazp49i8bGRgwYMADz5s3D8OHDTV1ap6xcuVLrM6pSqRSxsbH33Vev55C6Q2FhIbZv347o6GgEBAQgJycHBQUFWLNmjTrozNHZs2chl8vh4+ODHTt2IDo62uwDadOmTQgJCcGQIUOgUqlw8OBBXLlyBWvXroWjo6Opy+u0U6dOwcbGBm5ubmhtbcWxY8eQmZmJN954A97e3qYur8tKSkrwySefwN7eHgEBAWYfSIWFhXj11VfVY9bW1nB2djZhVV1XX1+Pt99+G0OHDsWECRPg5OSEqqoquLq6wsPDw9TldUpNTQ1aW1vVrxUKBd555x0sWLAAoaGh993XZN9ll52djdDQUISHh8PDwwPR0dFwcXFBbm6uqUoyCqlUipkzZyIoKMhiLk8uXboUYWFh8PLygre3NxYuXIiamhpcvnzZ1KV1yejRoyGVSjFw4EC4u7tjxowZsLOz6/D5OnNSX1+P7du3Y/78+XBwcDB1OUZhbW0NFxcX9R9zDyMA+PLLL+Hi4oKFCxfC19cXAwYMwIgRI8w2jADA2dlZ43M6e/Ys+vTpg6CgIJ376v1ddsbUmYdtSRyNjY1QqVQW8w8dALS2tuKHH37A3bt3O1ysY0527dqFhx9+GCNGjMAXX3xh6nKMoqqqCq+99hpsbGzg5+eHGTNmmP1D9qdOncKDDz6IrVu34tKlS3BxccG4ceMQGRlpET/QqlQqHD16FI8++ijs7Ox0zjdJIN3vYdsLFy6YoiQyQHJyMgYNGmQR/3Bfv34d69evR1NTE+zs7LBo0SKzv1x35MgR3Lp1CwsXLjR1KUbj6+uLBQsWwMPDA0qlEhkZGVi/fj3WrFkDJycnU5fXaZWVlfj2228hk8kwadIkXLt2DXv37gUATJgwwcTVdV1RURGqqqrw2GOP6TXfJIHUET5sK76UlBRcvnwZK1asgLW1+f/2End3d8TFxaG+vh4nT55EYmIiXn31VbNdsFFRUYEDBw5g+fLlsLUV6n/vLpFKpRqv/fz8sGrVKhQUFLS70mJOVCoVhgwZov5O0MGDB+PmzZvIycmxiEDKy8vDkCFDMHjwYL3mm+RvbGcetiXTS0lJQWFhIZYtW2b2l0ra2NraYuDAgQCAIUOG4OrVq8jOzsaCBQtMW1gnlZSUoLa2Fm+99ZZ6rLW1FXK5HLm5ufjwww/Rq1cv0xVoJH369IGnpydu3bpl6lK6xMXFpd39Ig8PD3z99dcmqsh4lEolTp06ZdBiGpME0q8fth0zZox6vKioSK8bX/S/l5ycjOPHj+O1114z6xuuurS2tqK5udnUZXTa73//ewwZMkRjLDExEW5ubpg8ebLFnDU1NTWhoqLCbJdGtxk6dCgqKio0xm7evGnWK43b5Ofnw9bWFiEhIXrvY7K/nTKZDNu3b4evry+GDh2K3NxcjYdtzVVjY6P6pzaVSoU7d+6grKwMjo6OZvuXbPfu3fjuu++waNEiODg4qL9U187ODn369DFxdZ23f/9+SKVS9O3bF42NjTh+/DiKi4uxePFiU5fWaQ4ODu0Wm9jZ2cHR0dFsL0MCwL59+zBq1Cj07dsXNTU1+OKLL3D37l2dy4hFN3HiRGzYsAEZGRkIDg5GWVkZvvnmG8yYMcPUpXVJ22KGkJAQg/6NMNlzSIBlPhh76dIlbNy4sd342LFjERMTY4KKuu6FF17QOj5t2jQ8/fTT/+NqjGfHjh24dOkSlEol7O3t4e3tjSeeeAIjR440dWlG9Y9//ANeXl5m/RzSxx9/jOLiYtTW1sLZ2Rl+fn545pln4OnpaerSuuzMmTM4cOAAKioq0LdvX0yYMAETJkww6/vpFy9eRHx8PFatWgVfX1+99zNpIBEREbUx/2VSRERkERhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREJgIBERkRD+P9uF/rY40O6aAAAAAElFTkSuQmCC\n",
108 |       "text/plain": [
109 |        "<Figure size 432x288 with 1 Axes>"
110 |       ]
111 |      },
112 |      "metadata": {},
113 |      "output_type": "display_data"
114 |     }
115 |    ],
116 |    "source": [
117 |     "pmf = 1/6*np.ones(6)\n",
118 |     "cdf = np.cumsum(pmf)\n",
119 |     "plt.stem(diceN, cdf ,use_line_collection = True)\n",
120 |     "plt.axis([0, 7, 0, 1.1])\n",
121 |     "plt.title('CDF of A Fair Die')\n",
122 |     "plt.show()"
123 |    ]
124 |   },
125 |   {
126 |    "cell_type": "markdown",
127 |    "metadata": {},
128 |    "source": [
129 |     "One particular useful formula for calculating probability within an interval is\n",
130 |     "\n",
131 |     "$$\n",
132 |     "P(a<X\\leq b) = F_X(b)-F_X(a)\n",
133 |     "$$\n",
134 |     "\n",
135 |     "It is commonly used for computing the probability of a range, for instance, what is the probability $P(2<X\\leq 5)$ in this case?"
136 |    ]
137 |   },
138 |   {
139 |    "cell_type": "markdown",
140 |    "metadata": {},
141 |    "source": [
142 |     "Using the formula, \n",
143 |     "\n",
144 |     "$$\n",
145 |     "P(2<X\\leq5) = F_X(5) - F_X(2) = 5/6 - 2/6 = 1/2\n",
146 |     "$$"
147 |    ]
148 |   },
149 |   {
150 |    "cell_type": "markdown",
151 |    "metadata": {},
152 |    "source": [
153 |     "# <font face=\"gotham\" color=\"purple\"> Probability Density Function"
154 |    ]
155 |   },
156 |   {
157 |    "cell_type": "markdown",
158 |    "metadata": {},
159 |    "source": [
160 |     "The **Probability Density Function** (PDF) is concept used for continuous distribution. For instance the temperature of a room follows a continuous distribution.\n",
161 |     "\n",
162 |     "We denote PDF as $f_X(x)$, however any realization of $x$ must be $0$ probability, using the room temperature example, the probability that a room reaches exactly $20  ^\\circ \\text{C}$ is theoretically $0$. $f_X(x = 25)= 0$. Therefore to obtain positive probability, we must delimit range of the arguments $(x,\\ \\Delta x)$\n",
163 |     "\n",
164 |     "$$\n",
165 |     "f_{X}(x)=\\lim _{\\Delta \\rightarrow 0^{+}} \\frac{P(x<X \\leq x+\\Delta)}{\\Delta}\n",
166 |     "$$"
167 |    ]
168 |   },
169 |   {
170 |    "cell_type": "markdown",
171 |    "metadata": {},
172 |    "source": [
173 |     "Furthermore, use CDF formula and definition of derivative, the relation of CDF and PDF\n",
174 |     "\n",
175 |     "$$\n",
176 |     "f_{X}(x)=\\lim _{\\Delta \\rightarrow 0} \\frac{F_{X}(x+\\Delta)-F_{X}(x)}{\\Delta}=\\frac{d F_{X}(x)}{d x}=F_{X}^{\\prime}(x)\n",
177 |     "$$\n",
178 |     "\n",
179 |     "We will see examples in next chapter."
180 |    ]
181 |   },
182 |   {
183 |    "cell_type": "markdown",
184 |    "metadata": {},
185 |    "source": [
186 |     "# <font face=\"gotham\" color=\"purple\">Independent Random Variables"
187 |    ]
188 |   },
189 |   {
190 |    "cell_type": "markdown",
191 |    "metadata": {},
192 |    "source": [
193 |     "Independent R.V.s are similar to independent events, recall that independent events have the property\n",
194 |     "\n",
195 |     "$$\n",
196 |     "p(A\\cap B) =p (A)p(B)\n",
197 |     "$$"
198 |    ]
199 |   },
200 |   {
201 |    "cell_type": "markdown",
202 |    "metadata": {},
203 |    "source": [
204 |     "Now consider two random variable $A$ and $B$, they are independent as long as\n",
205 |     "\n",
206 |     "$$\n",
207 |     "p(X=x,Y=y)=p(X=x)p(X=y)\n",
208 |     "$$"
209 |    ]
210 |   },
211 |   {
212 |    "cell_type": "markdown",
213 |    "metadata": {},
214 |    "source": [
215 |     "In general, independent variables have the property:\n",
216 |     "\n",
217 |     "$$\n",
218 |     "p(X_1=x_1,X_2=x_2, ..., X_n = x_n)=p(X_1=x_1)p(X_2=x_2)...p(X_n=x_n)=\\prod_{i=1}^np(X_i=x_i)\n",
219 |     "$$"
220 |    ]
221 |   },
222 |   {
223 |    "cell_type": "markdown",
224 |    "metadata": {},
225 |    "source": [
226 |     "# <font face=\"gotham\" color=\"purple\"> Expected Value and Variances"
227 |    ]
228 |   },
229 |   {
230 |    "cell_type": "markdown",
231 |    "metadata": {},
232 |    "source": [
233 |     "The **expected value** of discrete and continuous random variables are\n",
234 |     "\n",
235 |     "$$\n",
236 |     "\\text{Discrete:}\\qquad E(X)=\\sum_{i=1}^k x_ip_X(x_i)=\\sum_{i=1}^k x_i p(X_i = x_i)\n",
237 |     "$$\n",
238 |     "\n",
239 |     "$$\\text{Continuous:}\\qquad E(X) = \\int_{-\\infty}^{\\infty}xf_X(x)dx$$"
240 |    ]
241 |   },
242 |   {
243 |    "cell_type": "markdown",
244 |    "metadata": {},
245 |    "source": [
246 |     "They are expressing the same idea that weighting each possibilities equally, then sum up."
247 |    ]
248 |   },
249 |   {
250 |    "cell_type": "markdown",
251 |    "metadata": {},
252 |    "source": [
253 |     "The variance of discrete and continuous random variables are similar"
254 |    ]
255 |   },
256 |   {
257 |    "cell_type": "markdown",
258 |    "metadata": {},
259 |    "source": [
260 |     "$$\n",
261 |     "\\text{Discrete:}\\qquad\\operatorname{Var}(X)=E\\left[\\left(X-\\mu_{X}\\right)^{2}\\right]=E (X^{2})-[E (X)]^{2}\n",
262 |     "$$\n",
263 |     "\n",
264 |     "\n",
265 |     "\\begin{aligned}\n",
266 |     "\\text{Continuous:}\\qquad \n",
267 |     "\\operatorname{Var}(X)&= E\\left[\\left(X-\\mu_{X}\\right)^{2}\\right]=\\int_{-\\infty}^{\\infty}\\left(x-\\mu_{X}\\right)^{2} f_{X}(x) d x \\\\\n",
268 |     "&=E (X^{2})-[E (X)]^{2}=\\int_{-\\infty}^{\\infty} x^{2} f_{X}(x) d x-\\mu_{X}^{2}\n",
269 |     "\\end{aligned}"
270 |    ]
271 |   },
272 |   {
273 |    "cell_type": "markdown",
274 |    "metadata": {},
275 |    "source": [
276 |     "And a common method for manual calculation of variance"
277 |    ]
278 |   },
279 |   {
280 |    "cell_type": "markdown",
281 |    "metadata": {},
282 |    "source": [
283 |     "$$\n",
284 |     "\\begin{aligned}\n",
285 |     "\\operatorname{Var}(X) &=E\\left[\\left(X-\\mu_{X}\\right)^{2}\\right] \\\\\n",
286 |     "&=E\\left[X^{2}-2 \\mu_{X} X+\\mu_{X}^{2}\\right] \\\\\n",
287 |     "&=E\\left[X^{2}\\right]-2 E\\left[\\mu_{X} X\\right]+E\\left[\\mu_{X}^{2}\\right]\\\\\n",
288 |     "&=E\\left[X^{2}\\right]-2 \\mu_{X}^{2}+\\mu_{X}^{2}\\\\\n",
289 |     "&=E\\left[X^{2}\\right]-\\mu_{X}^{2}\n",
290 |     "\\end{aligned}\n",
291 |     "$$"
292 |    ]
293 |   }
294 |  ],
295 |  "metadata": {
296 |   "kernelspec": {
297 |    "display_name": "Python 3",
298 |    "language": "python",
299 |    "name": "python3"
300 |   },
301 |   "language_info": {
302 |    "codemirror_mode": {
303 |     "name": "ipython",
304 |     "version": 3
305 |    },
306 |    "file_extension": ".py",
307 |    "mimetype": "text/x-python",
308 |    "name": "python",
309 |    "nbconvert_exporter": "python",
310 |    "pygments_lexer": "ipython3",
311 |    "version": "3.8.8"
312 |   },
313 |   "toc": {
314 |    "base_numbering": 1,
315 |    "nav_menu": {},
316 |    "number_sections": true,
317 |    "sideBar": true,
318 |    "skip_h1_title": false,
319 |    "title_cell": "Table of Contents",
320 |    "title_sidebar": "Contents",
321 |    "toc_cell": true,
322 |    "toc_position": {},
323 |    "toc_section_display": true,
324 |    "toc_window_display": false
325 |   }
326 |  },
327 |  "nbformat": 4,
328 |  "nbformat_minor": 4
329 | }
330 | 


--------------------------------------------------------------------------------
/Chapter 2 - Bayes' Theorem.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "markdown",
  5 |    "metadata": {
  6 |     "toc": true
  7 |    },
  8 |    "source": [
  9 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
 10 |     "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
 11 |     "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#-Concept-of-Bayes'-Theorem\" data-toc-modified-id=\"-Concept-of-Bayes'-Theorem-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Concept of Bayes' Theorem</font></a></span><ul class=\"toc-item\"><li><span><a href=\"#-Example-1\" data-toc-modified-id=\"-Example-1-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 1</font></a></span></li><li><span><a href=\"#-Example-2\" data-toc-modified-id=\"-Example-2-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 2</font></a></span></li><li><span><a href=\"#-Example-3\" data-toc-modified-id=\"-Example-3-1.3\"><span class=\"toc-item-num\">1.3&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 3</font></a></span></li><li><span><a href=\"#-Example-4\" data-toc-modified-id=\"-Example-4-1.4\"><span class=\"toc-item-num\">1.4&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 4</font></a></span></li><li><span><a href=\"#-Example-5\" data-toc-modified-id=\"-Example-5-1.5\"><span class=\"toc-item-num\">1.5&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 5</font></a></span></li><li><span><a href=\"#-Example-6\" data-toc-modified-id=\"-Example-6-1.6\"><span class=\"toc-item-num\">1.6&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Example 6</font></a></span></li></ul></li></ul></div>\n",
 12 |     "</div>"
 13 |    ]
 14 |   },
 15 |   {
 16 |    "cell_type": "code",
 17 |    "execution_count": 13,
 18 |    "metadata": {},
 19 |    "outputs": [],
 20 |    "source": [
 21 |     "import numpy as np\n",
 22 |     "import matplotlib.pyplot as plt\n",
 23 |     "plt.style.use('ggplot')"
 24 |    ]
 25 |   },
 26 |   {
 27 |    "cell_type": "markdown",
 28 |    "metadata": {},
 29 |    "source": [
 30 |     "# <font face=\"gotham\" color=\"purple\"> Concept of Bayes' Theorem</font>"
 31 |    ]
 32 |   },
 33 |   {
 34 |    "cell_type": "markdown",
 35 |    "metadata": {},
 36 |    "source": [
 37 |     "The **Bayes' Theorem** is occasionally called **inverse probability theorem**. As the name suggests, once we have known $p(A|B)$, the Bayes' Theorem can tell us something about $P(B|A)$. "
 38 |    ]
 39 |   },
 40 |   {
 41 |    "cell_type": "markdown",
 42 |    "metadata": {},
 43 |    "source": [
 44 |     "To see how Bayes' Theorem is derived, start from conditional probability\n",
 45 |     "\n",
 46 |     "$$\n",
 47 |     "P(A|B)P(B)=P(A\\cap B)\\\\\n",
 48 |     "P(B|A)P(A)=P(A\\cap B)\n",
 49 |     "$$"
 50 |    ]
 51 |   },
 52 |   {
 53 |    "cell_type": "markdown",
 54 |    "metadata": {},
 55 |    "source": [
 56 |     "Equate them and arrange, we obtain the famous Bayes' Theorem:\n",
 57 |     "\n",
 58 |     "$$\n",
 59 |     "P(B|A) = \\frac{P(A|B)P(B)}{P(A)}\n",
 60 |     "$$"
 61 |    ]
 62 |   },
 63 |   {
 64 |    "cell_type": "markdown",
 65 |    "metadata": {},
 66 |    "source": [
 67 |     "If there are $k$ partitions of $B$, $P(A)$ can bee explicitly written in total probability form\n",
 68 |     "\n",
 69 |     "$$\n",
 70 |     "P(B_i|A)=\\frac{P(A|B_i)P(B_i)}{\\sum_{i=0}^k P(A|B_i)P(B_i)}\n",
 71 |     "$$"
 72 |    ]
 73 |   },
 74 |   {
 75 |    "cell_type": "markdown",
 76 |    "metadata": {},
 77 |    "source": [
 78 |     "Since Bayes' Theorem can be somewhat counter-intuitive, it might take some time to grasp the tricky logic. We will go through some examples, strongly advise to study them thoroughly."
 79 |    ]
 80 |   },
 81 |   {
 82 |    "cell_type": "markdown",
 83 |    "metadata": {},
 84 |    "source": [
 85 |     "## <font face=\"gotham\" color=\"purple\"> Example 1</font>"
 86 |    ]
 87 |   },
 88 |   {
 89 |    "cell_type": "markdown",
 90 |    "metadata": {},
 91 |    "source": [
 92 |     "A certain disease affects about 1 out of 10,000 people. There is a test to check whether the person has the disease. The test is quite accurate. In particular, we know that\n",
 93 |     "\n",
 94 |     "1. the probability that the test result is positive (suggesting the person has the disease), given that the person does not have the disease, is only 2 percent;\n",
 95 |     "\n",
 96 |     "2. the probability that the test result is negative (suggesting the person does not have the disease), given that the person has the disease, is only 1 percent.\n",
 97 |     "\n",
 98 |     "A random person gets tested for the disease and the result comes back positive. What is the probability that the person has the disease?"
 99 |    ]
100 |   },
101 |   {
102 |    "cell_type": "markdown",
103 |    "metadata": {},
104 |    "source": [
105 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
106 |     "$\\text{Event A}$: $\\text{the test result is positive}$<br>\n",
107 |     "$\\text{Event B}$: $\\text{the person has the disease}$\n",
108 |     "\n",
109 |     "\n",
110 |     "\n",
111 |     "$$P(B|A) = \\frac{P(A|B)P(B)}{P(A|B)P(B)+P(A|B^c)P(B^c)}= \\frac{(1- .01)\\times.0001}{(1- .01)\\times.0001+.02\\times(1-.0001)}$$\n",
112 |     "</div>"
113 |    ]
114 |   },
115 |   {
116 |    "cell_type": "code",
117 |    "execution_count": 1,
118 |    "metadata": {},
119 |    "outputs": [
120 |     {
121 |      "name": "stdout",
122 |      "output_type": "stream",
123 |      "text": [
124 |       "The probability that a person have positive results and having the disease is 0.493%.\n"
125 |      ]
126 |     }
127 |    ],
128 |    "source": [
129 |     "pB_A =( .99*.0001)/(.99*.0001+.02*.999)\n",
130 |     "print('The probability that a person have positive results and having the disease is %.3f%%.'%(pB_A*100))"
131 |    ]
132 |   },
133 |   {
134 |    "cell_type": "markdown",
135 |    "metadata": {},
136 |    "source": [
137 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
138 |     "The result comes with a surprise. But if you dwell on it a while, you will notice an obvious fact that there are much more 'healthy people'(people who do not have this type of disease) i.e.  $9999:1$. Even if you got to guess, you should consider this fact, which we call **prior**. It means even if a person has a positive result, yet he is still likely to be from healthy group.<br><br>\n",
139 |     "</div>"
140 |    ]
141 |   },
142 |   {
143 |    "cell_type": "markdown",
144 |    "metadata": {},
145 |    "source": [
146 |     "## <font face=\"gotham\" color=\"purple\"> Example 2</font>"
147 |    ]
148 |   },
149 |   {
150 |    "cell_type": "markdown",
151 |    "metadata": {},
152 |    "source": [
153 |     "Suppose a patient has an unidentified lesion in the liver, that internist recommend abdominal MRI for firmer evidences to exclude critical diseases. The sensitivity (true positive) rate of MRI is $92\\%$, the prevalence of critical liver disease is $.06\\%$. \n",
154 |     "\n",
155 |     "The question is if the result is positive, how likely this patient has critical liver disease?"
156 |    ]
157 |   },
158 |   {
159 |    "cell_type": "markdown",
160 |    "metadata": {},
161 |    "source": [
162 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
163 |     "$\\text{Event A}$: $\\text{MRI positive results}$<br>\n",
164 |     "$\\text{Event B}$: $\\text{the person has critical disease}$\n",
165 |     "\n",
166 |     "$$p(B|A) = \\frac{P(A|B)P(B)}{P(A|B)P(B)+P(A|B^C)P(B^C)} = \\frac{.92\\times .0006}{.92\\times .0006+.08\\times .9994}$$\n",
167 |     "</div>"
168 |    ]
169 |   },
170 |   {
171 |    "cell_type": "code",
172 |    "execution_count": 3,
173 |    "metadata": {},
174 |    "outputs": [
175 |     {
176 |      "name": "stdout",
177 |      "output_type": "stream",
178 |      "text": [
179 |       "The probability that a person who has symptoms truly have critical liver disease is 0.686%.\n"
180 |      ]
181 |     }
182 |    ],
183 |    "source": [
184 |     "pB_A = (0.92*.0006)/(.92*.0006+.08*.9994)\n",
185 |     "print('The probability that a person who has symptoms truly have critical liver disease is %.3f%%.'%(pB_A*100))"
186 |    ]
187 |   },
188 |   {
189 |    "cell_type": "markdown",
190 |    "metadata": {},
191 |    "source": [
192 |     "## <font face=\"gotham\" color=\"purple\"> Example 3</font>"
193 |    ]
194 |   },
195 |   {
196 |    "cell_type": "markdown",
197 |    "metadata": {},
198 |    "source": [
199 |     "A factory produces an item using three different brands of machines—$\\text{A}$, $\\text{B}$, and $\\text{C}$—which account for $20\\%$, $30\\%$, and $50\\%$ of output respectively. Of the items produced by machine $\\text{A}$, $5\\%$ are defective; similarly, $3\\%$ of machine $\\text{B}$'s items and $1\\%$ of machine $\\text{C}$'s are defective. If a randomly selected item is defective, what is the probability it was produced by machine $\\text{C}$?"
200 |    ]
201 |   },
202 |   {
203 |    "cell_type": "markdown",
204 |    "metadata": {},
205 |    "source": [
206 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
207 |     "$\\text{Event A}$: $\\text{Produced by machine A}$<br>\n",
208 |     "$\\text{Event B}$: $\\text{Produced by machine B}$<br>\n",
209 |     "$\\text{Event C}$: $\\text{Produced by machine C}$<br>  \n",
210 |     "$\\text{Event D}$: $\\text{Defective product}$  \n",
211 |     "$$\n",
212 |     "P(C|D) = \\frac{P(D|C)P(C)}{P(D|A)P(A)+P(D|B)P(B)+P(D|C)P(C)}\n",
213 |     "$$\n",
214 |     "</div>"
215 |    ]
216 |   },
217 |   {
218 |    "cell_type": "code",
219 |    "execution_count": 5,
220 |    "metadata": {},
221 |    "outputs": [
222 |     {
223 |      "name": "stdout",
224 |      "output_type": "stream",
225 |      "text": [
226 |       "The probability that a defective product made by machine C is 20.833%.\n"
227 |      ]
228 |     }
229 |    ],
230 |    "source": [
231 |     "pB_A = .5*.01/(.2*.05+.3*.03+.5*.01)\n",
232 |     "print('The probability that a defective product made by machine C is %.3f%%.'%(pB_A*100))"
233 |    ]
234 |   },
235 |   {
236 |    "cell_type": "markdown",
237 |    "metadata": {},
238 |    "source": [
239 |     "## <font face=\"gotham\" color=\"purple\"> Example 4</font>"
240 |    ]
241 |   },
242 |   {
243 |    "cell_type": "markdown",
244 |    "metadata": {},
245 |    "source": [
246 |     "Bag I contains 4 white and 6 black balls while Bag II contains 4 white and 3 black balls. One ball is drawn randomly from one of the bags and it is found to be black. Find the probability that it was drawn from Bag I."
247 |    ]
248 |   },
249 |   {
250 |    "cell_type": "markdown",
251 |    "metadata": {},
252 |    "source": [
253 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
254 |     "$\\text{Event A}$: $\\text{black ball}$<br>\n",
255 |     "$\\text{Event B}$: $\\text{from bag I}$<br>\n",
256 |     "\n",
257 |     "$$p(B|A) = \\frac{p(A|B)p(B)}{p(A)}= \\frac{\\frac{6}{10}\\frac{1}{2}}{\\frac{6}{10}\\frac{1}{2}+\\frac{3}{7}\\frac{1}{2}}$$\n",
258 |     "</div>"
259 |    ]
260 |   },
261 |   {
262 |    "cell_type": "code",
263 |    "execution_count": 6,
264 |    "metadata": {},
265 |    "outputs": [
266 |     {
267 |      "name": "stdout",
268 |      "output_type": "stream",
269 |      "text": [
270 |       "The probability that the ball is from Bag I given it is black is 58.33333%.\n"
271 |      ]
272 |     }
273 |    ],
274 |    "source": [
275 |     "pB_A = 6/10*.5/(6/10*.5+3/7*.5)\n",
276 |     "print('The probability that the ball is from Bag I given it is black is %.5f%%.'%(pB_A*100))"
277 |    ]
278 |   },
279 |   {
280 |    "cell_type": "markdown",
281 |    "metadata": {},
282 |    "source": [
283 |     "## <font face=\"gotham\" color=\"purple\"> Example 5</font>"
284 |    ]
285 |   },
286 |   {
287 |    "cell_type": "markdown",
288 |    "metadata": {},
289 |    "source": [
290 |     "A man is known to speak truth 2 out of 3 times. He throws a die and reports that number obtained is a four. Find the probability that the number obtained is actually a four."
291 |    ]
292 |   },
293 |   {
294 |    "cell_type": "markdown",
295 |    "metadata": {},
296 |    "source": [
297 |     "<div style=\"background-color:Lavender; color:DarkBlue; padding:30px;\">\n",
298 |     "$\\text{Event A}$: $\\text{reports number 4}$<br>\n",
299 |     "$\\text{Event B}$: $\\text{speak truth}$<br>\n",
300 |     "$$   \n",
301 |     "P(A|B)=\\frac{P(B|A)P(A)}{P(B|A)P(A)+P(B^C|A)P(A)}=\\frac{\\frac{2}{3}\\frac{1}{6}}{\\frac{2}{3}\\frac{1}{6}+\\frac{1}{3}\\frac{5}{6}}\n",
302 |     "$$\n",
303 |     "</div>"
304 |    ]
305 |   },
306 |   {
307 |    "cell_type": "code",
308 |    "execution_count": 8,
309 |    "metadata": {},
310 |    "outputs": [
311 |     {
312 |      "name": "stdout",
313 |      "output_type": "stream",
314 |      "text": [
315 |       "The probability that he truly obtains 4 given he speaks the truth is 28.571%.\n"
316 |      ]
317 |     }
318 |    ],
319 |    "source": [
320 |     "pB_A = (1/6*2/3)/(1/6*2/3+5/6*1/3)\n",
321 |     "print('The probability that he truly obtains 4 given he speaks the truth is %.3f%%.'%(pB_A*100))"
322 |    ]
323 |   },
324 |   {
325 |    "cell_type": "markdown",
326 |    "metadata": {},
327 |    "source": [
328 |     "# <font face=\"gotham\" color=\"purple\"> Bayesian Inference </font>"
329 |    ]
330 |   },
331 |   {
332 |    "cell_type": "markdown",
333 |    "metadata": {},
334 |    "source": [
335 |     "Bayes' Theorem won't be much helpful in scientific researches if it is only a tool for calculate the inverse probability. Without changing the theory, just reframe the definition, we will have a completely revolutionary view. "
336 |    ]
337 |   },
338 |   {
339 |    "cell_type": "markdown",
340 |    "metadata": {},
341 |    "source": [
342 |     "$\\text{Event A}$: $\\text{Newly collected data}$<br>\n",
343 |     "$\\text{Event B}$: $\\text{Hypothesis}$<br>\n",
344 |     "$$\n",
345 |     "P(B|A) = \\frac{P(A|B)P(B)}{P(A)} = \\frac{P(A|B)P(B)}{P(A|B)P(B)+P(A|B^c)P(B^c)}\n",
346 |     "$$"
347 |    ]
348 |   },
349 |   {
350 |    "cell_type": "markdown",
351 |    "metadata": {},
352 |    "source": [
353 |     "Hypothesis $\\text{B}$ has a probability $P(B)$ which is called **prior** in **Bayesian Inference**, $P(A|B)$ is the **likelihood** (exactly the same meaning as in maximum likelihood), $P(B|A)$ is the **posterior** meaning which is also your new prior before next data collection.\n",
354 |     "\n",
355 |     "The reason why $P(A|B)$ is called likelihood is actually quite intuitive, it measures how likely data $A$ will present given $B$ hypothesis is true. $P(A)$ is calculated by law of total probability based on $B$ and $B^C$ hypothesis."
356 |    ]
357 |   },
358 |   {
359 |    "cell_type": "markdown",
360 |    "metadata": {},
361 |    "source": [
362 |     "If $B$ is called a hypothesis, then it expresses a subjective belief. Broadly speaking, you start with a scientific question and set forth two or more alternative hypotheses. You then assign a prior probability that each alternative hypothesis is true. Next, you collect data. Finally, you use Bayes’ Theorem to update the probability for each hypothesis considered."
363 |    ]
364 |   },
365 |   {
366 |    "cell_type": "markdown",
367 |    "metadata": {},
368 |    "source": [
369 |     "What if we change the prior gradually in the first example above, from $1$ to $1000$ people out of $10000$, and rest parameters are the same, let's see how the posteriors change."
370 |    ]
371 |   },
372 |   {
373 |    "cell_type": "code",
374 |    "execution_count": 18,
375 |    "metadata": {},
376 |    "outputs": [
377 |     {
378 |      "data": {
379 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtMAAAGHCAYAAAByA95NAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA+5ElEQVR4nO3deXxU1eH+8edMAoQQEpIZkkASFgERwQWIoCAgkOJuca17kS9Vi1ttbd2tXbDYuv1UKLYgUu2iVutW1wiKEheURXYIArIkZGdJSEhyz++PC5E0LCHL3Ezm83698kpm5k7muRy1Ty/nnmOstVYAAAAAjprP6wAAAABAqKJMAwAAAA1EmQYAAAAaiDINAAAANBBlGgAAAGggyjQAAADQQJRpAMBhPfjgg+rdu7fXMQCgRaJMA0AzmjBhgowxdb5iYmK8jhZ0f/nLXzR27FglJCTIGKNPP/3U60gA0GiUaQBoZiNGjFBOTk6tr2+//dbrWEFXVlamMWPG6E9/+pPXUQCgyVCmAaCZtW3bVsnJybW+EhMTJUmFhYVKS0vTbbfdVnN8Xl6eunTpojvvvFOSZK3VT37yE/Xq1Uvt27fXMccco3vuuUcVFRU179k/FeOll15Snz59FB0drfHjx2vnzp169dVX1bdvX3Xs2FGXXHKJduzYUfO+CRMmKCMjQ4899phSUlIUHR2tiy++WAUFBYc9pw8++EDDhw9X+/btlZKSouuuu06FhYWHfc/PfvYz3XvvvRo7duxR/xkCQEtFmQYAD/n9fv3973/X9OnT9eabb8paq2uuuUbdu3fX73//e0lumU5KStI//vEPrVq1Sk888YRmz56thx56qNbvysnJ0Zw5c/TKK6/onXfe0YIFC3TJJZdo5syZeumll/T222/rk08+qfO+L7/8Uh999JHeffddvf322/rmm280ceLEQ2aeO3eufvjDH+ryyy/XN998o9dee00bN27UhRdeKGtt0/8hAUALFul1AABo7T766KM6c6RHjx6tN998U5I0cuRI3Xfffbruuuv04x//WF988YUWL16sNm3aSJJ8Pl9NsZakHj16aP369Zo+fbp+85vf1DxfUVGhOXPmKBAISJIuu+wyzZgxQ7m5uercubMk6fLLL9eHH35YK4vjOHr++ecVFxcnSZo2bZrOPPNMrVu3Tn369KlzPr/97W9166236pZbbql5bs6cOerevbuWLl2qk08+uaF/VAAQcijTANDMhg4dqjlz5tR6Ljo6utbj+++/X++9954ee+wx/etf/1LPnj1rvf7Xv/5VM2fO1MaNG1VaWqqqqio5jlPrmJSUlJoiLalmSsn+Ir3/uby8vFrvO/7442uKtCQNHz5ckrRq1aqDlumFCxfq888/19NPP13ntXXr1lGmAYQVyjQANLP27dsfcWm5nJwcrV27VhEREVq7dm2t115++WXddNNNmjp1qkaNGqXY2Fi9/PLLuvfee2sdt/9K9n7GmIM+978l/Gg5jqM777xT11xzTZ3XkpOTG/W7ASDUUKYBwGOO4+jqq69W//79dfvtt+vSSy/V6NGjdfrpp0uS5s+fr4EDB+rnP/95zXs2btzYZJ+/atUq7dy5U7GxsZKkrKwsSVK/fv0Oenx6erpWrFjB2tMAIMo0ADS7vXv3Kjc3t87zSUlJMsZoypQpWrZsmZYsWaLU1FTdeOONuuqqq7RkyRLFx8erb9++mjVrll5//XUNGDBAb731ll599dUmy2eM0bXXXqvf//73Kioq0k033aRzzz33oFM8JHfO9Lhx43T77bfrxz/+sTp27Kh169bp5Zdf1tNPP6327dsf9H25ubnKzc3Vtm3bJEnZ2dmKiYmpmY4CAKGI1TwAoJl98skn6tKlS52vwsJCZWVl6be//a2effZZpaamSpIeeeQRderUSZMmTZIk3XDDDbrmmmt03XXXaeDAgfriiy/04IMPNlm+IUOG6PTTT9cPfvADnXnmmerfv79mz559yONHjx6tuXPnatmyZRoxYoROPPFE3X777erYsWOdaSUHmjFjhgYOHKhzzz1XkmrOZ8aMGU12LgAQbMayjhEAhK0JEyZoy5YtyszM9DoKAIQkrkwDAAAADUSZBgAAABqIaR4AAABAA3FlGgAAAGggyjQAAADQQJRpAAAAoIFCftOW/Yv/B1MgEFBBQUHQPxfBxTiHB8Y5PDDOrR9jHB68HOeuXbse9HmuTAMAAAANRJkGAAAAGogyDQAAADQQZRoAAABoIMo0AAAA0ECUaQAAAKCBKNMAAABAA1GmAQAAgAaiTAMAAAANRJkGAAAAGogyDQAAADRQpNcBAAAAgIOx1koV5dKuHdKuHarYsFo2rZdMZBuvo9WgTAMAACBobFWltGuntKtE2rlDdteOmp+1a9/jnSVugd69Q9q7t+a9JZJ8U2dJ/s7ehD8IyjQAAAAaxZbvcQvwvi+7vwzX/Pz9Y5WVHvyXRLaRYuOkjp2kjrEyXbtJHeP2PRcn0zFOndK6q6RDp+CcVD1RpgEAAFCHrSiXdhTvK8jFbineccDPB37trTj4L+nQcV8h7iST0kPq18ktx7GdZDp2qnlNsXFSu/Yyxhw2U5tAQKagoKlOsUlQpgEAAMKEra52rxLvKJZ2FMvuL8s7imV3FteUZe0okSr21P0FxrgFOS7eLcS9jttXht0vs//n/VeXW9Dc5uZCmQYAAAhxdm+FW5BLiqQdRbI7SqQdRfsKc1FNedbunZK1dX9BdAcpNl6Ki5fp3ntfWY6X4vYX5Ph9JTlOJiIiyGfXslGmAQAAWqgDS7ItKZJ2FLqFuebxvpK85yDzkCMiagqy/Ikyx/StKclm35VldUpwryi3aRv0c2stKNMAAABBZqur9xXhIqm4ULbkgJK8o6jmZ5XtrvvmyDZuKe6UIHXtJtPvJPfnuAS3JHeKl+ISpA4dZXxsKdLcKNMAAABNyJaXScVFUkmhbHGBVOwWZVtc8H1J3lkiWaf2GyMi3JIclyAldZXpO8D9uZNfplOCW5g7JUjRMUe8UQ/BQ5kGAACoB2utu6xbcb57NXl/US4ukC0ukooLpJJCaU9Z3Td36OgW4Xi/TFrPfcXYL9PJv+/5BCkmjivJIYgyDQAAIMnuKZOK8qWiAtli93tNaS4qcMvy/y4BZ3xSXCcpPiB1SZU5/mQp3v99UY7fV5rbtvPilBAElGkAANDq2apK9ypyYZ7svsKsovzaRfl/rygbnzvtIiEgpXaXOSHdvbKcEHDLc7zfvZkvkjoVzhh9AAAQ0qy1cnbtlP3uW6koT7awQCrKkwrzvy/OO4vrLgnXMU5K6CwldpU57kS3NCd0lonfV5Y7JbAMHI6IMg0AAFo0a617w17BdrccF+a5V5gL9/+cr/z/3WCkTVu3KPs7y5ww2P05ISCT0NktygkBpl6gSQStTC9ZskSzZ8+W4zgaO3asxo8fX+v1srIyPfnkkyosLFR1dbXOP/98jR49OljxAACAR2qV5YLtNQXZFn7/syr31n5TdIzk33dVud9J6tCtp8raRbvPJXR2NxdhxQsEQVDKtOM4mjVrlu677z75/X7dfffdSk9PV2pqas0x7777rlJTU3XXXXdp586duu222zRixAhFMg8JAICQZ0t3SwXbvy/MB3xXYV7dshwTK/kTpZTuMicOkQKJMgmJUiDRvdocFV3r8A6BgPYUFATxjABXUJpqdna2kpOTlZSUJEkaNmyYFi5cWKtMG2NUXl4ua63Ky8sVExMjH8vDAAAQEmxVpXsFOT93X0nOlc13vyt/e90d+qJjpECSu+nIienuDn2BJMmftK8st/fmRICjFJQyXVRUJL/fX/PY7/dr3bp1tY4566yz9Mc//lE33HCD9uzZo9tvv50yDQBAC2LLdrtlOS9Xys9xry7n5Uj5ue5KGQduQhLZxi3LgSSZXse53wPJ3z8X3cG7EwGaUFDKtP3fu2elOvOYli5dqu7du+uBBx7Q9u3b9bvf/U7HHXecoqNr/zVOZmamMjMzJUlTp05VIBBovuCHEBkZ6cnnIrgY5/DAOIcHxrl+rLVydhSrOmeL+5W7VdW5W1SVu1XVOVtkd++sdbwvLl6RySmKGDBQEUkpikhOUURyV0UkpcgX7w/qBiSMcXhoieMclDLt9/tVWFhY87iwsFDx8fG1jpk3b57Gjx8vY4ySk5OVmJiobdu2qXfv3rWOy8jIUEZGRs3jAg/mRwUCAU8+F8HFOIcHxjk8MM7fq7nZb/s22bxtUl6OlJcjm+9+V/kBq2IYn3tDX+dkmcHDZDp3kemcLCUmS4Fkmaj2ciQ5kiprfYikoqJgnhZjHCa8HOeuXbse9PmglOlevXopJydHeXl5SkhIUFZWlm699dZaxwQCAS1btkz9+vVTSUmJtm3bpsTExGDEAwCg1bG7d7qFefs2afs2aftWtzBvz5EOXEYuIkIKJEuJXWT69Jc6d5FJ7CJ1TnZv+ots491JACEgKGU6IiJCEydO1JQpU+Q4jkaPHq20tDS9//77kqRx48bp4osv1vTp0/WLX/xCknTVVVcpNjY2GPEAAAhJdm+FlOeWZZu7Vcrd6l5t3r5NKt31/YE+nztXObHr94U5qYuU2NW98Y+NSYAGM/ZgE5pDyLZt24L+mfxVUnhgnMMD4xweQnmcrbVSSZGUu2VfYd73fftWqSi/9q5+8QH3CnNyiluck1KkpK7uDX+tfKnZUB5j1F/YTvMAAACHZysr3TnLuVtkczYfUJ631p6W0a69lJwi07uflJTh/pzU1S3PLCcHBB1lGgCAILLlZVLOvsJ8wHcV5ErOAUvLJXSWklNlTt9fmFOkLqlSXAI7+wEtCGUaAIBmYMt2u2V523fSts3u95zNUvEBf0UdEelOw0jrITNkhFuek1Pd8twuyrvwAOqNMg0AQCPYPWXStu/2lebvv6vkgKXh2raVktNk+g6QuqTJdElzrzJ37sLNf0CIo0wDAFAPtnKve6V56yZp6yb3+7ZNUtEBV5r3l+bjTnK3ye7aTeqa5q6Ywa6+QKtEmQYA4ADWcaTCPGnLRtmtG6Ut+4pz3rbv5zRHREpdUmV6H++W5pRuUtfu7rrMPq40A+GEMg0ACFu2rNQtzVs27Pu+0Z2iUVHuHmCMuz5zSg+ZwcPc7ynd3JUzWvlScwDqh/8SAABaPes4snk50pYNspvdL23Z6F6B3q9DRym1h8zwDPd7Snf3qjPLzQE4DMo0AKBVsZWV0rZNst99K+0rzvlbN7o3CkqS8bmrZRzTVxp5pkxaTymlhxTvZ8k5AEeNMg0ACFl2T5m0+Vu3OH/3rezmb93l56qr3QOi2kupPRR1xtkq79xFJrWnlNJNpm07b4MDaDUo0wCAkGBLd7mF+bv10qb1spvWuzcF7hcXL6UdI3PiKTLdjpHSekqBZBmfT7GBgPay1TSAZkCZBgC0OLZ0t/TdetmN2dKmbNlN2VLB9u8P8CdK3XvJDBvjFuduvWTi4r0LDCBsUaYBAJ6y5Xv2Fed10sZs93t+7vcHBJJkuveWRp4l072X1O0YmZhY7wIDwAEo0wCAoLFVle6GJ9+ulTauc4tzzhbJ7lu/OaGz1KO3zIhxbnHu3lumQ0dvQwPAYVCmAQDNwlor5efIblgnbVgru2Gt9N23UlWle0DHOKlHH5nBw2R69HF/ju3kaWYAOFqUaQBAk7Blpe7V5m/XyH67RtqwRtq9y32xbTt3jvOYc6Uex8r07ONusc1SdABCHGUaAHDUrONI27fKrl8trV/tlueczZK17gFd0mROGiL17Ouu59y1m0wE22wDaH0o0wCAI7IV5dK3a2TXr3YL9LdrpLLd7ovRMdIxfWVOOd0tzj36yETHeBsYAIKEMg0AqMMWF8pmr5LWr3K/b/5WchzJGPeq8+BhUq/jZI45TkrqKuPzeR0ZADxBmQaAMGetlXI2y2avlNatlF23UirMc19s29adqnH2JTK9+7lXoLnqDAA1KNMAEGZsdbW7k+C65W5xzl75/Y2CHeOkPv1lMs6X6XW8lNZTJpL/qQCAQ+G/kADQytmqSneVjbUrZNcul7JXSxV73BcTu7g3CvY+XqZPf/cxK2wAQL1RpgGglakpz2uWy65ZJq1fJe3d677YtZvMaWdIxw6Q6dNfplOCp1kBINRRpgEgxNnqarc8r/6mbnlO7Skz4kyZY/u70zc6xnkbFgBaGco0AIQY6zjSlo1ueV79jbRuhVS+b9pGao995XmAdGx/mZhYb8MCQCtHmQaAEGDzc2VXLZFWLpVd8833NwwmpcgMHSXT7yTp2BNkOlKeASCYKNMA0ALZ3Tul1d/Irloqu2qplJ/rvtDJL3NCutTvZJm+J8gkBLwNCgBhjjINAC2ArapydxhcuVh25RJp4zp3a+720VLfE2QyLpDpd7KUnMJqGwDQglCmAcAjNj9XdsUi2eWLpDXL3HnPPp/U81iZ8y6XOf5k9+eICK+jAgAOgTINAEFi91ZIa1fILv9adsUiKXer+4I/UWbIKJn+A6XjTmCHQQAIIZRpAGhGtmC77LKvZJd9La35xl2yrk1bqe8AmVFnywwYLCV1ZeoGAIQoyjQANCFbVSWtXyX7zVeyy76Scja7L3ROdpesGzDI3TClbTtvgwIAmgRlGgAayZbucovz0oXu9I09ZVJEpLvO88hxMgPSZZJTvI4JAGgGlGkAaACbu1V26Zey33wprVslWUeK7SQzeLjMiadI/U6UiYr2OiYAoJlRpgGgHqzjSBvWyi75QnbJF1LuFveF1B4yZ18ic9IpUo8+Mj6ft0EBAEFFmQaAQ7CVle7GKUs+l136pbSj2F26ru8JMmecI3PyEBl/otcxAQAeokwDwAFsRbm0fJHsos9kly105z+3a+/eOHjyUJkT0mU6sHQdAMBFmQYQ9mxZqew3X8ou+kxaschdvi6mo8ygYTKDTpP6nSTTpq3XMQEALRBlGkBYsmW7ZZd8qeJlC+Us+UKqqpI6JcgM/4FboPv0Z+dBAMARUaYBhA23QH8h+9UCaeUSqbpKVYEkmTPOlUkf7m7dzQ2EAICjQJkG0KrZinJ3CbuFn0jLv3avQCd0lhl7nsygYQqcMkyFRUVexwQAhCjKNIBWx1bulZZ9LbvwE3cd6L173SkcZ5wrc8rp7hXofdt3cyUaANAYlGkArYJ1qqU1y2W/+Ni9kXBPqRQTKzNsrMwpI6Tex1OcAQBNjjINIGRZa6XN37oF+sv5UkmRu4zdoFNlhoxyV+HgJkIAQDOiTAMIOba4UPaLj2Q/mydt+06KiJQGDJK5bJTMiUNk2rXzOiIAIExQpgGEBFtRLrv4M7dAr1oqWSv1Ok7mqp/KpA+XiYn1OiIAIAxRpgG0WNZaad0K2QUfyn69QKool/yJMuf+SOa0M2QSu3odEQAQ5ijTAFocW5Qv+9k82QWZUn6uFNVe5pQRMqeNkXr340ZCAECLQZkG0CLYykp3Q5UFH7gbqlgr9T1B5vwrZAadJtMuyuuIAADUQZkG4Cmbs0X2k/fcudC7d7obqpx7mbukXedkr+MBAHBYlGkAQWf3Vsh+tUD2k/el7JVSRIR00lD5RoyTjj9JxsdydgCA0ECZBhA0NmeL7MfvyH42VyorlRK7yFz8Y5lhY2Ri472OBwDAUaNMA2hWtqpKWvK5nI/ekdYskyIi3TnQo86Sjh1Qs603AAChiDINoFnYogLZ+e/KfvqBtKPYXdLuwmtkTs/gKjQAoNWgTANoMvvXhXbmviUt/txdkeOEdPnOOFvqP5C50ACAVocyDaDRbEWFu733vP9KWzZK0TEyP/ihzBnnyASSvI4HAECzoUwDaDBbVCA777+y89+TynZLqT1krr1ZZsgomXbtvI4HAECzo0wDOGp2w1rZD153t/i2kgadKt+Y86U+x3NDIQAgrFCmAdSLdaqlxV/IyXxdyl4ltY+WGXu+zJjzmMoBAAhblGkAh2UrKmSzPpT94DUpP1cKJMn8aJK7KkdUtNfxAADwFGUawEHZXTvd+dDz/utu831MX/kumSCdPJRVOQAA2IcyDaAWm58r+8Frsgsypb17pZOGyHfmRVLvfsyHBgDgfwStTC9ZskSzZ8+W4zgaO3asxo8fX+eYFStW6LnnnlN1dbU6duyo3/zmN8GKB4Q9u/U72Xf/LfvlfMn4ZE49Q2bceJmu3byOBgBAixWUMu04jmbNmqX77rtPfr9fd999t9LT05WamlpzTGlpqWbOnKl7771XgUBAO3bsCEY0IOzZDevkvP2ytORzqW0796bCH4yXifd7HQ0AgBYvKGU6OztbycnJSkpy7/gfNmyYFi5cWKtMf/rppxo6dKgCgYAkKS4uLhjRgLBl1yyX8/ZL0solUnQHmfMulxl7nkxMrNfRAAAIGUEp00VFRfL7v7/K5ff7tW7dulrH5OTkqKqqSg8++KD27Nmjc845R6NGjarzuzIzM5WZmSlJmjp1ak35DqbIyEhPPhfB1RrH2VqryuWLtPvFZ1W5YrF8nRIUfe1Nan/WePnad/A6nida4zijLsa59WOMw0NLHOeglGlrbZ3n/vdGpurqam3YsEH333+/9u7dq/vuu099+vRR165dax2XkZGhjIyMmscFBQXNE/owAoGAJ5+L4GpN42ytlVZ/I+fNf0rrVkpxCTKX/0QaMU572rbTntI9Uuker2N6ojWNMw6NcW79GOPw4OU4/28n3S8oZdrv96uwsLDmcWFhoeLj4+sc07FjR0VFRSkqKkr9+vXTpk2bDhkcwJHVlOg3/illr5Q6Jchccb3MiHEybdp6HQ8AgJDnC8aH9OrVSzk5OcrLy1NVVZWysrKUnp5e65j09HStXr1a1dXVqqioUHZ2tlJSUoIRD2iVbPZKOY/cK+ex+6WC7TJX3iDfQ3+Rb8x5FGkAAJpIUK5MR0REaOLEiZoyZYocx9Ho0aOVlpam999/X5I0btw4paam6uSTT9Ydd9whn8+nMWPGqFs3luQCjpbdtF7Oay9Iy7+WYjvJXP4TmZFnUqABAGgGxh5sQnMI2bZtW9A/k3lZ4SHUxtlu+07O6/+QFmVJ0TEyZ10sM+ZcmXZRXkdr0UJtnNEwjHPrxxiHh7CdMw2g+diifNk3/iGbNU9q107m/MtlMn4oEx2eq3MAABBMlGkgRNnS3bLvvCz74VuSrMwPLpA56xKZjqwTDQBAsFCmgRBjK/fKzn1L9u2XpT1l7rbfP7xKxp/odTQAAMIOZRoIEdZxZL/8WPY/z0tFBdKAwfJdfK1Mak+vowEAELYo00AIsOtWynlxprQpW+reW77rfiZz3IlexwIAIOxRpoEWzObnynnlOenrLKmTX2bi7TJDR8n4grJEPAAAOALKNNAC2bJS2bdfkv3wTckXIXPBlTLjxrPMHQAALQxlGmhBrOPIfjZP9tU50q4dMqeNkRl/tUy83+toAADgICjTQAthN66T88+/SN+ukXoeK98t98v06ON1LAAAcBiUacBjdtcO2f88L/vpB1LHOJnrbpM5dTTzogEACAGUacAj1qmW/fg92deelyrKZTIukDnvcnYuBAAghFCmAQ/YTevlvDBd2rhO6neSfFdcL9MlzetYAADgKFGmgSCye8pkX/+77Nz/Sh1jZSb9QmbISBljvI4GAAAagDINBIG1Vvp6gZx/zZR2Fsuccba7Skd0jNfRAABAI1CmgWZmC/Pl/P3P0rKvpG695LvpXpmerNIBAEBrQJkGmol1HNmP3pZ99XlJVuZHk2TGnCvji/A6GgAAaCKUaaAZ2JzNcuY8Ja1fLfUfKN81N8n4E72OBQAAmhhlGmhCtqpS9t1XZf/7otQ2Sua6n8mcNpobDAEAaKUo00ATsZs3yHn2cWnLRpn002Wu+IlMbLzXsQAAQDOiTAONZKurZd/5t+xbL0odYuSbfI/MwFO9jgUAAIKAMg00gt32nZxnn5A2ZcucMkLmyhtkYmK9jgUAAIKEMg00gHWqZT94Xfa1v0tR7eW74Vcy6ad7HQsAAAQZZRo4SjY/150bnb1KGniqfFf/lLnRAACEKco0UE/WWtnPP5L9xwzJGJn/u11m6Bms1AEAQBijTAP1YMt2y77wZ9mFn0h9jpfv/37OutEAAIAyDRyJXbtczqzHpR1FMuOvljn7YnYxBAAAkijTwCHZqio5/3le9p1/S52T5bvzYZmex3odCwAAtCCUaeAgbGG+ih+9V3b1MpnhGTKX/0Qmqr3XsQAAQAtDmQb+h136pZxnn5C1jsxP7pBvyEivIwEAgBaKMg3sY6sqZV/9m+wHr0vdjlHCnX9QSVuuRgMAgEOjTAOSbMF2OX/5k7Rhrczoc2QunajILl2lggKvowEAgBaMMo2w507reFyyVr4b75QZPNzrSAAAIETUq0xba5WXl6fOnTvL5/M1dyYgKKxTLfvGP2X/+5LUrZdbpDsnex0LAACEkHo1Y2OM7rjjjubOAgSNLd0l56nfyf73JZnhY+W7cypFGgAAHLV6T/Po0aOHcnJylJKS0px5gGZnv1sv589TpeJCmasny4w8ky3BAQBAg9S7TPfv318PPfSQRo0apUAgUOu1MWPGNHkwoDk4WXNlX5gudego36/+IHNMX68jAQCAEFbvMr1mzRolJiZq1apVdV6jTKOls9XVsi8/K/vhm1LfE+S7/pcysZ28jgUAAEJcvcv0r3/96+bMATQbW7pbzl/+KK1cIpNxgcwl18lERHgdCwAAtAJHtTTe7t279fXXX6uoqEgJCQkaPHiwYmJimisb0Gg2d4ucp34vFebJ/PgW+U7/gdeRAABAK1Lvde7Wrl2rW265RR988IE2bdqkzMxM3XLLLVq7dm1z5gMazC5fJOehX0p7SuX7xe8p0gAAoMnV+8r0c889p0mTJmn48O83tMjKytLs2bP1hz/8oVnCAQ1hrZX98E3Zl56VUrrJd/N9Mv5Er2MBAIBWqN5XpnNycnTaaafVeu7UU09Vbm5uk4cCGspWV8u+MF32xZnSSUPku/NhijQAAGg29S7TycnJysrKqvXcZ599pqSkpCYPBTSELS+T8/TvZOe/J3P2JfL99C6ZqPZexwIAAK1Yvad5TJgwQVOnTtU777yjQCCg/Px85eTk6K677mrOfEC92JJCOU/+Vtq6Seaam+QbeabXkQAAQBiod5nu27evnnrqKS1atEjFxcUaPHiwBg0axGoe8JzdstEt0mWl8t18v8wJg72OBAAAwsRRLY0XExOjkSNHNlcW4KjZVUvl/PkPUrsod0fDbsd4HQkAAISRw5bpKVOm6N5775UkPfDAAzLGHPS43/zmN02fDDgCJ2uu7N+ekpJT5bv1AZmEzl5HAgAAYeawZXrUqFE1P7NlOFoS5/3/yL48W+p3knw33iUT3cHrSAAAIAwdtkyffvrpkiTHcbR9+3ZddNFFatOmTVCCAQdjrZX9z99k33lFZvBwmf/7uQz/TAIAAI/Ua2k8n8+n9957TxEREc2dBzgk61TLPj/NLdIjz5K5/g6KNAAA8FS915keNWqUPvjgg+bMAhySrdwr55k/yn7yvsy5l8lc/VMZH//nDgAAeKveq3lkZ2fr3Xff1RtvvCG/31/rZkRuQERzsuVlcqY9JK3+RuZH/ydfxg+9jgQAACDpKMr02LFjNXbs2ObMAtRhS3fJeeJB6bv1MhNvl++00V5HAgAAqFHvMn3GGWc0YwygLrtrp5zH75dyNss3+R6Zk4Z4HQkAAKCWepdpa60+/PBDLViwQLt27dIjjzyilStXqqSkRMOGDWvOjAhDdmexnEfvl/Jz5bvpPpkBg7yOBAAAUEe9b0B88cUXNW/ePGVkZKigoECS5Pf79frrrzdbOIQnW1Io50/3SgXb5bvlfoo0AABosepdpj/++GPdeeedGj58eM3Nh4mJicrLy2u2cAg/tihfzp/ukYoL5bvtQZl+J3kdCQAA4JDqPc3DcRxFRUXVeq68vLzOc0BD2YLtch65VyrbLd/tv5HpdZzXkQAAAA6r3lemBw4cqL/97W+qrKyU5M6hfvHFFzV48OBmC4fwYQu2y/nT3dKeUvlu/x1FGgAAhIR6l+lrr71WRUVFmjBhgsrKynTttdcqPz9fV155ZXPmQxiwRflyHr1PKt8j3y9+L9Ozj9eRAAAA6qXe0zyio6P1q1/9Sjt27FB+fr4CgYA6derUjNEQDmxJkbtqx+6d8v389zLdenkdCQAAoN7qfWX6V7/6lSQpLi5OvXv3rinSd911V73ev2TJEt1222265ZZb9Nprrx3yuOzsbP3oRz/S559/Xt9oCFF21w45j90v7SiS77Zfc0UaAACEnHqX6dzc3DrPWWu1ffv2I77XcRzNmjVL99xzjx5//HEtWLBAW7ZsOehxf//733XyySfXNxZClC3dJeexB75f/q738V5HAgAAOGpHnObx9NNPS5Kqqqpqft4vPz9faWlpR/yQ7OxsJScnKykpSZI0bNgwLVy4UKmpqbWOe+eddzR06FCtX7++3ieA0GPLSuU8/mspd7N8N98v0/cEryMBAAA0yBGvTCclJdWU4P0/JyUlKTk5WaeffnrN9I/DKSoqkt/vr3ns9/tVVFRU55gvv/xS48aNO9pzQAix5XvkPPkbacsG+W68W6b/QK8jAQAANNgRr0xfeumlkqQ+ffo0ePqFtbbOc/s3ftnvueee01VXXSWf7/D9PjMzU5mZmZKkqVOnKhAINChTY0RGRnryuaHOVlaqZPoU7d2wVnF3/E5Rp432OtJhMc7hgXEOD4xz68cYh4eWOM71Xs0jMjJSeXl5SkxMVElJiV544QX5fD5deeWVR1zVw+/3q7CwsOZxYWGh4uPjax2zfv16/b//9/8kSTt37tTixYvl8/k0ZMiQWsdlZGQoIyOj5vH+rc2DKRAIePK5ocw6juysx2QXfyFz7c3a3ecE7W7hf4aMc3hgnMMD49z6Mcbhwctx7tq160Gfr/cNiLNmzaq5ajxnzhxVV1fLGKNnnnnmiO/t1auXcnJylJeXp6qqKmVlZSk9Pb3WMdOmTav5OvXUUzVp0qQ6RRqhyVor+9Is2S/ny1x0rXwjmMoDAABah3pfmS4qKlIgEFB1dbWWLl2q6dOnKzIyUjfccMMR3xsREaGJEydqypQpchxHo0ePVlpamt5//31JYp50K2fffln2wzdlMn4oc9bFXscBAABoMvUu0+3bt1dJSYk2b96s1NRURUVFqaqqSlVVVfV6/6BBgzRo0KBazx2qRN900031jYUWzpn/nuxrL8iceobMpdfVmSsPAAAQyupdps866yzdfffdqqqq0oQJEyRJq1evVkpKSnNlQ4iziz6TfeHP0oDBMj++VeYIN5cCAACEmnqX6fHjx2vIkCHy+XxKTk6WJCUkJOjGG29stnAIXXbtCjl/fUTq2Ue+G++Uiaz3P2oAAAAh46gaTlJSktasWaPs7GwlJCSob9++ioiIaK5sCFF2+zY50x+SAonu7obtoryOBAAA0CzqXaa3bt2qhx9+WHv37q1Z6q5Nmza688476+xkiPBld+90N2UxRr5bHpCJifU6EgAAQLOpd5meOXOmMjIydP7559fcRPbGG29o1qxZ+vWvf91sARE6bGWlnGkPSUUF8v3i9zKJXbyOBAAA0KzqfUfYxo0bdd5559VajeHcc8/Vxo0bmyMXQoy1VnbOk1L2SpmJP5Pp3c/rSAAAAM2u3mU6ISFBK1eurPXcqlWr6uxkiPBk3/yn7Bcfy4y/Wr5TRngdBwAAICjqPc3jiiuu0MMPP6zBgwfXbOW4aNEi3XLLLc2ZDyHA+Wye7Jv/khk+VuacS72OAwAAEDRHLNMVFRV65ZVXtHnzZp1yyilKSUlRcXGx0tLSdNlllx1yn3KEB7t2heycp6TjTpS5ejKbsgAAgLByxDI9a9YsrV+/XgMHDtTixYvVoUMHTZo0KRjZ0MLZwnw5M6ZKnZPku/Eumcg2XkcCAAAIqiPOmV6yZInuu+8+XX311br77ru1aNGiYORCC2f3VrhrSVdVynfTvTIdYryOBAAAEHRHLNMVFRU1NxkGAgGVlZU1eyi0bNZa2eenSZu/lW/SL2SSWWccAACEpyNO86iurtby5ctrHjuOU+uxJA0YMKDpk6HFsh+8Lvv5RzLjr5Y58RSv4wAAAHjmiGU6Li5Of/7zn2sex8TE1HpsjNHTTz/dPOnQ4tiVS2T//Zw0aBgrdwAAgLB3xDI9bdq0YORACLD5uXL+8iepa5p8193Gyh0AACDs1XvTFoQ3W1EuZ9oUyTryTb5HJqq915EAAAA8R5nGEblbhT8lbdss309+KZPYxetIAAAALQJlGkdkP35HduEnMj+8UmbAIK/jAAAAtBiUaRyW3bRe9sWZ0oDBMmdf4nUcAACAFoUyjUOyZaVynnlY6thJvom3y/j4xwUAAOBAtCMclLVWzpynpMI8+a6/Q6ZjrNeRAAAAWhzKNA7Kzv2vtChL5qJrZXof73UcAACAFokyjTrshnWyLz8rnXiKzA/Gex0HAACgxaJMoxZbutudJx0XL9/EnzFPGgAA4DBoSqhhrZXzt6ekkkL5rv+lTIeOXkcCAABo0SjTqGE//UBa9JnMhdfK9DrO6zgAAAAtHmUakiSbt81dT/q4E2V+8EOv4wAAAIQEyjRkq6rkzHxMioiU7zrmSQMAANQXrQmyb78kbVgr3zWTZRICXscBAAAIGZTpMGfXr5Z96yWZ00bLpJ/udRwAAICQQpkOY7a8TM6sx6SEgMwVN3gdBwAAIORQpsOY/ddfpYI8+Sb9XKZ9tNdxAAAAQg5lOkzZr7NkF3woc84lbBcOAADQQJTpMGR3Fst5YZrUo4/MeZd7HQcAACBkUabDkPOPZ6Tycne78MhIr+MAAACELMp0mLFfL5C+zpK54AqZLmlexwEAAAhplOkwYnftlPP3GVL33jLjLvQ6DgAAQMijTIcR++JfpbLd8k24RSYiwus4AAAAIY8yHSbs0i9lv/hY5pxLZVJ7eh0HAACgVaBMhwFbtlvOC9OllO4y51zqdRwAAIBWgzIdBuxLz0o7S+SbcKtMZBuv4wAAALQalOlWzq5YLLsgU+bMC2V69PE6DgAAQKtCmW7FbPkeOX97WkpOkTn/Cq/jAAAAtDqU6VbMvvUvqShfvmtvkWnT1us4AAAArQ5lupWyWzfJZr4hMzxDps/xXscBAABolSjTrZC1Vs4Lf5aiomUunuB1HAAAgFaLMt0K2c/mStkrZS7+sUzHWK/jAAAAtFqU6VbGlu6SfXm21Os4meEZXscBAABo1SjTrYx99Xl3y/CrfirjY3gBAACaE22rFbHfrpH95D2ZMefLpLFlOAAAQHOjTLcStrpazt//LMXFy/yQNaUBAACCgTLdStiP3pa++1a+H02SiYr2Og4AAEBYoEy3AnZHsexrL0jHD5QGD/c6DgAAQNigTLcC9j/PS5WV8l15g4wxXscBAAAIG5TpEGe/Wy+b9aHMmHNlkrp6HQcAACCsUKZDmLVWzkvPSh1iZM77kddxAAAAwg5lOpQt+UJas0zmgqtkomO8TgMAABB2KNMhylZVyvn3bKlLmszIM72OAwAAEJYo0yHKzv2vlJcj32UTZSIivI4DAAAQlijTIcju2in71ovSgEEyAwZ7HQcAACBsUaZDkH3zH1LFHvkuneh1FAAAgLBGmQ4xdtt3sh+/KzPyLJmu3byOAwAAENYig/VBS5Ys0ezZs+U4jsaOHavx48fXev2TTz7R66+/LkmKiorSpEmT1KNHj2DFCxnOy7Oldu1lLrjS6ygAAABhLyhXph3H0axZs3TPPffo8ccf14IFC7Rly5ZaxyQmJurBBx/UI488oosvvlh/+ctfghEtpNgVi6XlX8ucd5lMx1iv4wAAAIS9oJTp7OxsJScnKykpSZGRkRo2bJgWLlxY65i+ffsqJsZdK7lPnz4qLCwMRrSQYa2V8+rfJH+izOjzvI4DAAAABWmaR1FRkfx+f81jv9+vdevWHfL4uXPnauDAgQd9LTMzU5mZmZKkqVOnKhAING3YeoiMjAz655Z/Nk87vluv2FvvV/suXYL62eHKi3FG8DHO4YFxbv0Y4/DQEsc5KGXaWlvnOWPMQY9dvny55s2bp9/+9rcHfT0jI0MZGRk1jwsKCpom5FEIBAJB/VzrVMv525+lLmna3X+QSj0453AU7HGGNxjn8MA4t36McXjwcpy7du160OeDMs3D7/fXmrZRWFio+Pj4Osdt2rRJzzzzjH75y1+qY8eOwYgWEuznH0m5W+Qbf5WMjw1aAAAAWoqglOlevXopJydHeXl5qqqqUlZWltLT02sdU1BQoEceeUQ333zzIZt/OLKVlbJv/FPq3lsaeJrXcQAAAHCAoEzziIiI0MSJEzVlyhQ5jqPRo0crLS1N77//viRp3Lhx+ve//63du3dr5syZNe+ZOnVqMOK1aPbT96XCPPmuuemQU2MAAADgDWMPNqE5hGzbti3onxms+Tq2olzOPddLyany3TGFMh1kzL8LD4xzeGCcWz/GODyE7ZxpNIyd+5a0s0S+C6+mSAMAALRAlOkWypbtln33FemEdJnex3sdBwAAAAdBmW6h7HuvSWWl8l14jddRAAAAcAiU6RbI7iyW/fANmVNGyKT19DoOAAAADoEy3QLZd1+VKvfKXHCl11EAAABwGJTpFsbu2iH78bsyQ0fJJKd4HQcAAACHQZluYWzmG+5V6bMv9ToKAAAAjoAy3YLYst2y8/4rM2iYTJdUr+MAAADgCCjTLYid+19pT5nMuZd5HQUAAAD1QJluIWz5HneKx0lDWMEDAAAgRFCmWwj78btS6S75zmGuNAAAQKigTLcAdm+F7AevSf1Okjmmr9dxAAAAUE+U6RbALsiUdhTLd+6PvI4CAACAo0CZ9pitqnQ3aendTzq2v9dxAAAAcBQo0x6zn38kFeXLd+5lMsZ4HQcAAABHgTLtIVtdLfvOv6XuvaX+g7yOAwAAgKNEmfaQ/epTKS9HvnMu5ao0AABACKJMe8Ra686V7pImnTzU6zgAAABoAMq0V1Z/I23ZIDNuvIyPYQAAAAhFtDiPOJlvSB3jZIaO8joKAAAAGogy7QGbu0X6ZqHMGefItGnrdRwAAAA0EGXaAzbzDSmyjcwZZ3sdBQAAAI1AmQ4yu3un7GdzZU49Qya2k9dxAAAA0AiU6SCzH78r7d0rk3GB11EAAADQSJTpILJVlbLz3paOHyiT0t3rOAAAAGgkynQQ2YWfSjuK5PsBV6UBAABaA8p0kFhrZT94zd2kha3DAQAAWgXKdLCsXS5t3iCTcQFbhwMAALQSlOkgcT54XYqJlTn1DK+jAAAAoIlQpoPA5m7dt0nL2TJt23kdBwAAAE2EMh0E9sM3pYgImTPO8ToKAAAAmhBlupnZPWXuJi2njJCJi/c6DgAAAJoQZbqZ2S8+lirKxVVpAACA1ocy3Yyste6Oh2k9pZ7Heh0HAAAATYwy3Zy+XSNt2SAz6myWwwMAAGiFKNPNyH78rtSuvczQkV5HAQAAQDOgTDcTW7pL9qtPZU4dJRMV7XUcAAAANAPKdDOxn82VKvfKjDrb6ygAAABoJpTpZlBz4+ExfWXSenodBwAAAM2EMt0c1q6QcrfKjDrL6yQAAABoRpTpZmA/fkeK7iCTfrrXUQAAANCMKNNNzO4skV30mcywsTJt23kdBwAAAM2IMt3E7IIPpeoqmZFM8QAAAGjtKNNNyDqO7Px3pWMHyHRJ9ToOAAAAmhlluimtXCIVbOfGQwAAgDBBmW5CzsfvSh3jZAae5nUUAAAABAFluonYncXSN1+6Nx62aeN1HAAAAAQBZbqJ2C/mS44jM3ys11EAAAAQJJTpJmKz5ko9+sh0SfM6CgAAAIKEMt0E7OYN0pYNMsPGeB0FAAAAQUSZbgI2a64UESlzygivowAAACCIKNONZKurZb/4SDoxXSYm1us4AAAACCLKdGOtWCTt2iEfUzwAAADCDmW6kWzWXCkmVhow2OsoAAAACDLKdCPY0t2yS7+UGTpKJpK1pQEAAMINZboR7FefSlWVMqeN9joKAAAAPECZbgT72VypazepWy+vowAAAMADlOkGstu3SetXywwbI2OM13EAAADgAcp0A9nP5krGJzN0lNdRAAAA4BHKdANYx5H9bJ50/EkynfxexwEAAIBHKNMNULlisVSUL3Maa0sDAACEs8hgfdCSJUs0e/ZsOY6jsWPHavz48bVet9Zq9uzZWrx4sdq1a6fJkyfrmGOOCVa8o7Lno3ekqPYyJ5/qdRQAAAB4KChXph3H0axZs3TPPffo8ccf14IFC7Rly5ZaxyxevFi5ubl68skndf3112vmzJnBiHbUbEW5KrI+kkk/XaZdO6/jAAAAwENBKdPZ2dlKTk5WUlKSIiMjNWzYMC1cuLDWMV999ZVGjhwpY4yOPfZYlZaWqri4OBjxjs7OErU5boDMsLFeJwEAAIDHglKmi4qK5Pd/f6Oe3+9XUVFRnWMCgcBhj2kJTOdkxf/6CZk+x3sdBQAAAB4Lypxpa22d5/53beb6HCNJmZmZyszMlCRNnTq1VgEPlsjISE8+F8HFOIcHxjk8MM6tH2McHlriOAelTPv9fhUWFtY8LiwsVHx8fJ1jCgoKDnuMJGVkZCgjI6Pm8YHvCZZAIODJ5yK4GOfwwDiHB8a59WOMw4OX49y1a9eDPh+UaR69evVSTk6O8vLyVFVVpaysLKWnp9c6Jj09XfPnz5e1VmvXrlV0dPRByzQAAADQUgTlynRERIQmTpyoKVOmyHEcjR49WmlpaXr//fclSePGjdPAgQO1aNEi3XrrrWrbtq0mT54cjGgAAABAgwVtnelBgwZp0KBBtZ4bN25czc/GGE2aNClYcQAAAIBGYwdEAAAAoIEo0wAAAEADUaYBAACABqJMAwAAAA1EmQYAAAAaiDINAAAANBBlGgAAAGggyjQAAADQQMZaa70OAQAAAIQirkw3wF133eV1BAQB4xweGOfwwDi3foxxeGiJ40yZBgAAABqIMg0AAAA0EGW6ATIyMryOgCBgnMMD4xweGOfWjzEODy1xnLkBEQAAAGggrkwDAAAADRTpdYCWZMmSJZo9e7Ycx9HYsWM1fvz4Wq9bazV79mwtXrxY7dq10+TJk3XMMcfU671oORo6zgUFBZo2bZpKSkpkjFFGRobOOeccb04CR9SYf58lyXEc3XXXXUpISGiRd4/D1ZhxLi0t1YwZM7R582YZY/TTn/5Uxx57rAdngSNpzDi/9dZbmjt3rowxSktL0+TJk9W2bVsPzgJHcqRx3rp1q6ZPn64NGzbo8ssv1wUXXFDv9zYrC2uttdXV1fbmm2+2ubm5trKy0t5xxx128+bNtY75+uuv7ZQpU6zjOHbNmjX27rvvrvd70TI0ZpyLiors+vXrrbXWlpWV2VtvvZVxbqEaM877vfnmm/aJJ56wf/jDH4IZHUehseP81FNP2czMTGuttZWVlXb37t1BzY/6acw4FxYW2smTJ9uKigprrbWPPvqonTdvXrBPAfVQn3EuKSmx69ats//4xz/s66+/flTvbU5M89gnOztbycnJSkpKUmRkpIYNG6aFCxfWOuarr77SyJEjZYzRscceq9LSUhUXF9frvWgZGjPO8fHxNVc62rdvr5SUFBUVFXlxGjiCxoyzJBUWFmrRokUaO3asF/FRT40Z57KyMq1atUpjxoyRJEVGRqpDhw5enAaOoLH/PjuOo71796q6ulp79+5VfHy8F6eBI6jPOMfFxal3796KiIg46vc2J8r0PkVFRfL7/TWP/X5/naJUVFSkQCBQ55j6vBctQ2PG+UB5eXnasGGDevfu3byB0SCNHefnnntOV199tYwxwQmMBmnMOOfl5Sk2NlbTp0/Xr371K82YMUPl5eVBy476a8w4JyQk6Pzzz9dPf/pTXX/99YqOjtZJJ50UtOyov8Z0Ka97GGV6H3uQRU3+939ID3VMfd6LlqEx47xfeXm5Hn30UU2YMEHR0dFNHxKN1phx/vrrrxUXF1dr/jRapsaMc3V1tTZs2KBx48bpj3/8o9q1a6fXXnutuaKiERozzrt379bChQs1bdo0PfPMMyovL9f8+fObLSsarjFdyuseRpnex+/3q7CwsOZxYWFhnb8K8vv9KigoqHNMfd6LlqEx4yxJVVVVevTRRzVixAgNHTo0OKFx1BozzmvWrNFXX32lm266SU888YSWL1+uJ598MmjZUX+N/e+23+9Xnz59JEmnnnqqNmzYEJzgOCqNGedly5YpMTFRsbGxioyM1NChQ7V27dqgZUf9NaZLed3DKNP79OrVSzk5OcrLy1NVVZWysrKUnp5e65j09HTNnz9f1lqtXbtW0dHRio+Pr9d70TI0ZpyttZoxY4ZSUlJ03nnneXQGqI/GjPOVV16pGTNmaNq0afrZz36mAQMG6NZbb/XoTHA4jRnnTp06ye/3a9u2bZKkZcuWKTU11YvTwBE0ZpwDgYDWrVuniooKWWu1bNkypaSkeHQmOJzGdCmvexibthxg0aJFmjNnjhzH0ejRo3XRRRfp/ffflySNGzdO1lrNmjVLS5cuVdu2bTV58mT16tXrkO9Fy9TQcV69erUeeOABdevWreavj6644goNGjTIy9PBITTm3+f9VqxYoTfffJOl8Vqwxozzxo0bNWPGDFVVVSkxMVGTJ09WTEyMl6eDQ2jMOL/00kvKyspSRESEevTooRtvvFFt2rTx8nRwCEca55KSEt11113as2ePjDGKiorSY489pujoaE97GGUaAAAAaCCmeQAAAAANRJkGAAAAGogyDQAAADQQZRoAAABoIMo0AAAA0ECUaQAIE9dcc422b9/udQwAaFVYGg8AQthNN92kkpIS+Xw+RUVFaeDAgZo4caKioqK8jgYAYYEr0wAQ4u688049//zzevjhh7V+/Xq98sortV6vrq5u1O9v7PsBoDWL9DoAAKBpJCQk6OSTT9bmzZt12WWXaeLEiXr77bdVXV2tadOm6bLLLtOTTz6p5ORklZWV6dlnn9XixYvVrl07jR07VhdeeKF8Pp8++ugjffjhh+rVq5c+/vhjnXnmmbr88su9Pj0AaJEo0wDQShQUFGjx4sUaMmSIFi1apIULF+qhhx5S27Zt6xz77LPPqqysTE8//bR27dqlKVOmKD4+XmPGjJEkrVu3TsOGDdPMmTO5Mg0Ah0GZBoAQ96c//UkRERGKjo7WoEGDdNFFF+nVV1/VhRdeqJiYmDrHO46jrKws/fGPf1T79u3Vvn17nXfeeZo/f35NmY6Pj9fZZ58tSYqIiAjq+QBAKKFMA0CI++Uvf6kTTzyxzvN+v/+gx+/cuVNVVVUKBAI1z3Xu3FlFRUU1jw98DQBwaNyACACtlDHmoM/HxsYqIiJCBQUFNc8VFBQoISEhWNEAoNWgTANAmPH5fDrttNP0z3/+U3v27FF+fr7eeustjRgxwutoABBymOYBAGFo4sSJevbZZ3XzzTerbdu2Gjt2rEaPHu11LAAIOWzaAgAAADQQ0zwAAACABqJMAwAAAA1EmQYAAAAaiDINAAAANBBlGgAAAGggyjQAAADQQJRpAAAAoIEo0wAAAEADUaYBAACABvr/GbLcPy84RgAAAAAASUVORK5CYII=\n",
380 |       "text/plain": [
381 |        "<Figure size 864x432 with 1 Axes>"
382 |       ]
383 |      },
384 |      "metadata": {},
385 |      "output_type": "display_data"
386 |     }
387 |    ],
388 |    "source": [
389 |     "p_B = np.linspace(.0001, .1, 100)\n",
390 |     "pB_A =( .99*p_B)/(.99*p_B+.02*(1-p_B))\n",
391 |     "fig, ax = plt.subplots(figsize=(12, 6))\n",
392 |     "\n",
393 |     "ax.plot(p_B, pB_A)\n",
394 |     "ax.set_xlabel('Prior')\n",
395 |     "ax.set_ylabel('Posterior')\n",
396 |     "ax.set_title('Example 1')\n",
397 |     "plt.show()"
398 |    ]
399 |   },
400 |   {
401 |    "cell_type": "markdown",
402 |    "metadata": {},
403 |    "source": [
404 |     "## <font face=\"gotham\" color=\"purple\"> Probability Density Form of Bayesian Inference</font>"
405 |    ]
406 |   },
407 |   {
408 |    "cell_type": "markdown",
409 |    "metadata": {},
410 |    "source": [
411 |     "To handle complicated issues, the integral version of Bayes' Theorem is our guideline, every Bayesian computation requires solving an integral numerically. \n",
412 |     "\n",
413 |     "$$\n",
414 |     "p(\\theta \\mid D)=\\frac{ p(D \\mid \\theta)p(\\theta)}{p(D)}=\\frac{p(D \\mid \\theta)p(\\theta) }{\\int p(D \\mid \\theta) p(\\theta) d \\theta}\n",
415 |     "$$"
416 |    ]
417 |   },
418 |   {
419 |    "cell_type": "markdown",
420 |    "metadata": {},
421 |    "source": [
422 |     "where $\\theta$ is the parameter that we hypothesized which characterizes the distribution $p(\\theta)$, note that we change the notation from $P$ to $p$, the former represents probability mass function and latter represents probability density distribution, which will be illustrated in details in next chapter."
423 |    ]
424 |   },
425 |   {
426 |    "cell_type": "markdown",
427 |    "metadata": {},
428 |    "source": [
429 |     "Some important differences has to be pointed out, density form has an _infinite number_ of hypotheses though bounds could be specified. And also the integral in the denominator won't always have a closed-form solution, which is why **Markov Chain Monte Carlo** is usually used for numerical computation of the integral.\n",
430 |     "\n",
431 |     "In some special cases you can derive posterior in closed-form, usually both prior and posterior share the same distribution but different parameters, such prior is called **conjugate prior**. "
432 |    ]
433 |   },
434 |   {
435 |    "cell_type": "code",
436 |    "execution_count": null,
437 |    "metadata": {},
438 |    "outputs": [],
439 |    "source": []
440 |   }
441 |  ],
442 |  "metadata": {
443 |   "kernelspec": {
444 |    "display_name": "Python 3",
445 |    "language": "python",
446 |    "name": "python3"
447 |   },
448 |   "language_info": {
449 |    "codemirror_mode": {
450 |     "name": "ipython",
451 |     "version": 3
452 |    },
453 |    "file_extension": ".py",
454 |    "mimetype": "text/x-python",
455 |    "name": "python",
456 |    "nbconvert_exporter": "python",
457 |    "pygments_lexer": "ipython3",
458 |    "version": "3.8.8"
459 |   },
460 |   "toc": {
461 |    "base_numbering": 1,
462 |    "nav_menu": {},
463 |    "number_sections": true,
464 |    "sideBar": true,
465 |    "skip_h1_title": false,
466 |    "title_cell": "Table of Contents",
467 |    "title_sidebar": "Contents",
468 |    "toc_cell": true,
469 |    "toc_position": {},
470 |    "toc_section_display": true,
471 |    "toc_window_display": false
472 |   }
473 |  },
474 |  "nbformat": 4,
475 |  "nbformat_minor": 4
476 | }
477 | 


--------------------------------------------------------------------------------