├── AUC_Derivation.ipynb ├── Adaptive_Forgetting.ipynb ├── BanachMatchbox.ipynb ├── Buffons_Needle_Sim.ipynb ├── Compressive_Sampling.ipynb ├── Conditional_Expectation_Gaussian.ipynb ├── Conditional_Expectation_Projection.ipynb ├── Conditional_expectation_MSE.ipynb ├── Conditional_expectation_MSE_Ex.ipynb ├── Confidence_Intervals.ipynb ├── Example_CSVs.ipynb ├── Expectation_Maximization.ipynb ├── Exponential_splines.ipynb ├── Filtering.ipynb ├── Filtering_Part2.ipynb ├── Filtering_Part3.ipynb ├── Fourier_Transform.ipynb ├── Frequency_Resolution.ipynb ├── Gauss_Markov.ipynb ├── Hypothesis_Testing.ipynb ├── Inverse_Projection_Constrained_Optimization.ipynb ├── LICENSE.html ├── Lighthouse_problem.ipynb ├── Lighthouse_schematic.jpg ├── MAP_Estimation.ipynb ├── Markov_chains.ipynb ├── Maximum_likelihood.ipynb ├── More_Fourier_Transform.ipynb ├── Projection.ipynb ├── Projection_Ex.ipynb ├── Projection_mdim.ipynb ├── README.md ├── Random_Sum.ipynb ├── Rectangle_Wedge_Tail_Decomposition.ipynb ├── Regression.ipynb ├── Robust_Statistics.ipynb ├── Robust_Statistics_MMA.ipynb ├── Sampling_Monte_Carlo.ipynb ├── Sampling_Theorem.ipynb ├── Sampling_Theorem_Part_2.ipynb ├── Sampling_Theorem_Part_2_v2.ipynb ├── Windowing.ipynb ├── Windowing_Part2.ipynb ├── Windowing_Part3.ipynb ├── ex214.jpg ├── ex23.jpg ├── ex24.jpg ├── ex26.jpg ├── ls_slider.py └── random_walk_random_stumble.ipynb /Conditional_Expectation_Gaussian.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "" 4 | }, 5 | "nbformat": 3, 6 | "nbformat_minor": 0, 7 | "worksheets": [ 8 | { 9 | "cells": [ 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Introduction\n", 15 | "-----------------\n", 16 | "\n", 17 | "By this point, we have developed many tools to deal with computing the conditional expectation. In this section, we discuss a bizarre and amazing coincidence regarding Gaussian random variables and linear projection, a coincidence that is the basis for most of statistical signal processing." 18 | ] 19 | }, 20 | { 21 | "cell_type": "heading", 22 | "level": 3, 23 | "metadata": {}, 24 | "source": [ 25 | "Conditional Expectation by Optimization" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "Now, let's consider the important case of the zero-mean bivariate Gaussian and try to find a function $h$ that minimizes the mean squared error (MSE). Again, trying to solve for the conditional expectation by minimizing the error over all possible functions $h$ is generally very, very hard. One alternative is to use parameters for the $h$ function and then just optimize over those. For example, we could assume that $h(Y)= \\alpha Y$ and then use calculus to find the $\\alpha$ parameter.\n", 33 | "\n", 34 | "Let's try this with the zero-mean bivariate Gaussian density,\n", 35 | "\n", 36 | "$$\\mathbb{E}((X-\\alpha Y )^2) = \\mathbb{E}(\\alpha^2 Y^2 - 2 \\alpha X Y + X^2 )$$\n", 37 | "\n", 38 | "and then differentiate this with respect to $\\alpha$ to obtain\n", 39 | "\n", 40 | "$$\\mathbb{E}(2 \\alpha Y^2 - 2 X Y ) = 2 \\alpha \\sigma_y^2 - 2 \\mathbb{E}(XY) = 0$$\n", 41 | "\n", 42 | "Then, solving for $\\alpha$ gives\n", 43 | "\n", 44 | "$$ \\alpha = \\frac{ \\mathbb{E}(X Y)}{ \\sigma_y^2 } $$\n", 45 | "\n", 46 | "which means we that\n", 47 | "\n", 48 | "\\begin{equation}\n", 49 | "\\mathbb{ E}(X|Y) \\approx \\alpha Y = \\frac{ \\mathbb{E}(X Y )}{ \\sigma_Y^2 } Y =\\frac{\\sigma_{X Y}}{ \\sigma_Y^2 } Y \n", 50 | "\\end{equation}\n", 51 | "\n", 52 | "where that last equality is just notation. Remember here we assumed a special linear form for $h=\\alpha Y$, but we did that for convenience. We still don't know whether or not this is the one true $h_{opt}$ that minimizes the MSE for all such functions.\n" 53 | ] 54 | }, 55 | { 56 | "cell_type": "heading", 57 | "level": 3, 58 | "metadata": {}, 59 | "source": [ 60 | "Conditional Expectation Using Direct Integration" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "Now, let's try this again by computing $ \\mathbb{E}(X|Y)$ in the case of the bivariate Gaussian distribution straight from the definition.\n", 68 | "\n", 69 | "\\begin{equation}\n", 70 | "\\mathbb{E}(X|Y) = \\int_{\\mathbb{ R}} x \\frac{f_{X,Y}(x,y)}{f_Y(y)} dx\n", 71 | "\\end{equation}\n", 72 | "\n", 73 | "where \n", 74 | "\n", 75 | "$$ f_{X,Y}(x,y) = \\frac{1}{2\\pi |\\mathbf{R}|^{\\frac{1}{2}}} e^{-\\frac{1}{2} \\mathbf{v}^T \\mathbf{R}^{-1} \\mathbf{v} } $$ \n", 76 | "\n", 77 | "and where\n", 78 | "\n", 79 | "$$ \\mathbf{v}= \\left[ x,y \\right]^T$$ \n", 80 | "\n", 81 | "$$ \\mathbf{R} = \\left[ \\begin{array}{cc}\n", 82 | "\\sigma_{x}^2 & \\sigma_{xy} \\\\\\\\\n", 83 | "\\sigma_{xy} & \\sigma_{y}^2 \\\\\\\\\n", 84 | "\\end{array} \\right] $$ \n", 85 | "\n", 86 | "and with\n", 87 | "\n", 88 | "\\begin{eqnarray}\n", 89 | " \\sigma_{xy} &=& \\mathbb{E}(xy) \\nonumber \\\\\\\\\n", 90 | " \\sigma_{x}^2 &=& \\mathbb{E}(x^2) \\nonumber \\\\\\\\ \n", 91 | " \\sigma_{y}^2 &=& \\mathbb{E}(y^2) \\nonumber \n", 92 | "\\end{eqnarray}\n", 93 | "\n", 94 | "This conditional expectation (Eq. 4 above) is a tough integral to evaluate, so we'll do it with `sympy`.\n" 95 | ] 96 | }, 97 | { 98 | "cell_type": "code", 99 | "collapsed": false, 100 | "input": [ 101 | "from sympy import Matrix, Symbol, exp, pi, simplify, integrate \n", 102 | "from sympy import stats, sqrt, oo, use\n", 103 | "from sympy.abc import y,x\n", 104 | "\n", 105 | "sigma_x = Symbol('sigma_x',positive=True)\n", 106 | "sigma_y = Symbol('sigma_y',positive=True)\n", 107 | "sigma_xy = Symbol('sigma_xy',real=True)\n", 108 | "fyy = stats.density(stats.Normal('y',0,sigma_y))(y)\n", 109 | " \n", 110 | "R = Matrix([[sigma_x**2, sigma_xy],\n", 111 | " [sigma_xy,sigma_y**2]])\n", 112 | "fxy = 1/(2*pi)/sqrt(R.det()) * exp((-Matrix([[x,y]])*R.inv()* Matrix([[x],[y]]))[0,0]/2 )\n", 113 | "\n", 114 | "fcond = simplify(fxy/fyy)" 115 | ], 116 | "language": "python", 117 | "metadata": {}, 118 | "outputs": [], 119 | "prompt_number": 13 120 | }, 121 | { 122 | "cell_type": "markdown", 123 | "metadata": {}, 124 | "source": [ 125 | "Unfortunately, `sympy` cannot immediately integrate this without some hints. So, we need to define a positive variable ($u$) and substitute it into the integration" 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "collapsed": false, 131 | "input": [ 132 | "u=Symbol('u',positive=True) # define positive variable\n", 133 | "\n", 134 | "fcond2=fcond.subs(sigma_x**2*sigma_y**2-sigma_xy**2,u) # substitute as hint to integrate\n", 135 | "g=simplify(integrate(fcond2*x,(x,-oo,oo))) # evaluate integral\n", 136 | "gg=g.subs( u,sigma_x**2 *sigma_y**2 - sigma_xy**2 ) # substitute back in\n", 137 | "use( gg, simplify,level=2) # simplify exponent term" 138 | ], 139 | "language": "python", 140 | "metadata": {}, 141 | "outputs": [ 142 | { 143 | "metadata": {}, 144 | "output_type": "pyout", 145 | "prompt_number": 14, 146 | "text": [ 147 | "sigma_xy*y/sigma_y**2" 148 | ] 149 | } 150 | ], 151 | "prompt_number": 14 152 | }, 153 | { 154 | "cell_type": "markdown", 155 | "metadata": {}, 156 | "source": [ 157 | "Thus, by direct integration using `sympy`, we found\n", 158 | "\n", 159 | "$$ \\mathbb{ E}(X|Y) = Y \\frac{\\sigma_{xy}}{\\sigma_{y}^{2}} $$ \n", 160 | "\n", 161 | "and this matches the prior result we obtained by direct minimization by assuming that $\\mathbb{E}(X|Y) = \\alpha Y$ and then solving for the optimal $\\alpha$!\n", 162 | "\n", 163 | "The importance of this result cannot be understated: the one true and optimal $h_{opt}$ *is a linear function* of $Y$. \n", 164 | "\n", 165 | "In other words, assuming a linear function, which made the direct search for an optimal $h(Y)$ merely convenient yields the optimal result! This is a general result that extends for *all* Gaussian problems. The link between linear functions and optimal estimation of Gaussian random variables is the most fundamental result in statistical signal processing! This fact is exploited in everything from optimal filter design to adaptive signal processing.\n", 166 | "\n", 167 | "We can easily extend this result to non-zero mean problems by inserting the means in the right places as follows:\n", 168 | "\n", 169 | "$$ \\mathbb{ E}(X|Y) = \\bar{X} + (Y-\\bar{Y}) \\frac{\\sigma_{xy}}{\\sigma_{y}^{2}} $$\n", 170 | "\n", 171 | "where $\\bar{X}$ is the mean of $X$ (same for $Y$)." 172 | ] 173 | }, 174 | { 175 | "cell_type": "markdown", 176 | "metadata": {}, 177 | "source": [ 178 | "Summary\n", 179 | "-------------\n", 180 | "\n", 181 | "In this section, we showed that the conditional expectation for Gaussian random variables is a linear function, which, by a bizarre coincidence, is also the easiest one to work with. This result is fundamental to all optimal linear filtering problems (e.g. Kalman filter) and is the basis of most of the theory of stochastic processes used in signal processing. Up to this point, we have worked hard to illustrate all of the concepts we will need to unify our understanding of this entire field and figured out multiple approaches to these kinds of problems, most of which are far more difficult to compute. Thus, it is indeed just plain lucky that the most powerful distribution is the easiest to compute as a conditional expectation because it is a linear function. We will come back to this same result again and again as we work our way through these greater concepts." 182 | ] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": {}, 187 | "source": [ 188 | "### References \n", 189 | "\n", 190 | "This post was created using the [nbconvert](https://github.com/ipython/nbconvert) utility from the source [IPython Notebook](www.ipython.org) which is available for [download](https://github.com/unpingco/Python-for-Signal-Processing/blob/master/Conditional_Expectation_Gaussian.ipynb) from the main github [site](https://github.com/unpingco/Python-for-Signal-Processing) for this blog. " 191 | ] 192 | } 193 | ], 194 | "metadata": {} 195 | } 196 | ] 197 | } -------------------------------------------------------------------------------- /Conditional_Expectation_Projection.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "Conditional_Expectation_Projection" 4 | }, 5 | "nbformat": 3, 6 | "nbformat_minor": 0, 7 | "worksheets": [ 8 | { 9 | "cells": [ 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Introduction\n", 15 | "--------------\n", 16 | "\n", 17 | "In these pages, I have tried to distill and illustrate the keys concepts needed in statistical signal processing, and in this section, we will cover the most fundamental statistical result that underpins statistical signal processing. The last sections on [conditional expectation](http://python-for-signal-processing.blogspot.com/2012/11/conditional-expectation-and-mean.html) and [projection](http://python-for-signal-processing.blogspot.com/2012/11/the-projection-concept.html) are prerequisites for what follows. Please review those before continuing." 18 | ] 19 | }, 20 | { 21 | "cell_type": "heading", 22 | "level": 2, 23 | "metadata": {}, 24 | "source": [ 25 | "Inner Product for Random Variables" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "From our previous work on projection for vectors in $\\mathbb{R}^n$, we have a good geometric grasp on how projection is related to minimum mean squared error (MMSE). It turns out by one abstract step, we can carry all of our geometric interpretations to the space of random variables.\n", 33 | "\n", 34 | "For example, we previously noted that at the point of projection, we had\n", 35 | "\n", 36 | "$$ ( \\mathbf{y} - \\mathbf{v}_{opt} )^T \\mathbf{v} = 0$$\n", 37 | "\n", 38 | "which by noting the inner product slightly more abstractly as $\\langle\\mathbf{x},\\mathbf{y} \\rangle = \\mathbf{x}^T \\mathbf{y}$, we can express as\n", 39 | "\n", 40 | "$$\\langle \\mathbf{y} - \\mathbf{v}_{opt},\\mathbf{v} \\rangle = 0 $$ \n", 41 | "\n", 42 | "and, in fact, by defining the inner product for the random variables $X$ and $Y$ as \n", 43 | "\n", 44 | "$$ \\langle X,Y \\rangle = \\mathbb{E}(X Y)$$ \n", 45 | "\n", 46 | "we remarkably have the same relationship:\n", 47 | "\n", 48 | "$$\\langle X-h_{opt}(Y),Y \\rangle = 0 $$ \n", 49 | "\n", 50 | "which holds not for vectors in $\\mathbb{R}^n$, but for random variables $X$ and $Y$ and functions of those random variables. Exactly why this is true is technical, but it turns out that one can build up the **entire theory of probability** this way (see Nelson,1987), by using the expectation as an inner product.\n", 51 | "\n", 52 | "Furthermore, by abstracting out the inner product concept, we have drawn a clean line between MMSE optimization problems, geometry, and random variables. That's a lot of mileage to get a out of an abstraction and it is key to everything we pursue in statistical signal processing because now we can shift between these interpretations to address real problems. Soon, we'll see how to do this with some examples, but first we will collect one staggering result that flows naturally from this abstraction.\n", 53 | "\n" 54 | ] 55 | }, 56 | { 57 | "cell_type": "heading", 58 | "level": 2, 59 | "metadata": {}, 60 | "source": [ 61 | "Conditional Expectation as Projection" 62 | ] 63 | }, 64 | { 65 | "cell_type": "markdown", 66 | "metadata": {}, 67 | "source": [ 68 | "[Previously](http://python-for-signal-processing.blogspot.com/2012/11/conditional-expectation-and-mean.html), we noted that the conditional expectation is the minimum mean squared error (MMSE) solution to the following problem:\n", 69 | "\n", 70 | "$$ \\min_h \\int_{\\mathbb{R}} (x - h(y) )^2 dx $$ \n", 71 | "\n", 72 | "with the minimizing $h_{opt}(Y) $ as \n", 73 | "\n", 74 | "$$ h_{opt}(Y) = \\mathbb{E}(X|Y) $$\n", 75 | "\n", 76 | "which is another way of saying that among all possible functions $h(Y)$, the one that minimizes the MSE is $ \\mathbb{E}(X|Y)$ (see appendix for a quick proof). From our discussion on [projection](http://python-for-signal-processing.blogspot.com/2012/11/the-projection-concept.html), we noted that these MMSE solutions can be thought of as projections onto a subspace that characterizes $Y$. For example, we previously noted that at the point of projection, we have\n", 77 | "\n", 78 | "$$\\langle X-h_{opt}(Y),Y \\rangle = 0 $$\n", 79 | "\n", 80 | "but since we know that the MMSE solution\n", 81 | "\n", 82 | "$$ h_{opt}(Y) = \\mathbb{E}(X|Y) $$\n", 83 | "\n", 84 | "we have by direct substitution,\n", 85 | "\n", 86 | "$$ \\mathbb{E}( X-\\mathbb{E}(X|Y), Y) = 0$$ \n", 87 | "\n", 88 | "That last step seems pretty innocuous, but it is the step that ties MMSE to conditional expectation to the inner project abstraction, and in so doing, reveals the conditional expectation to be a projection operator for random variables. Before we develop this further, let's grab some quick dividends:\n", 89 | "\n", 90 | "From the previous equation, by linearity of the expectation, we may obtain,\n", 91 | "\n", 92 | "$$ \\mathbb{E}(X Y) = \\mathbb{E}(Y \\mathbb{E}(X|Y))$$ \n", 93 | "\n", 94 | "which we could have found by using the formal definition of conditional expectation,\n", 95 | "\n", 96 | "\\begin{equation}\n", 97 | "\\mathbb{E}(X|Y) = \\int_{\\mathbb{R}^2} x \\frac{f_{X,Y}(x,y)}{f_Y(y)} dx dy\n", 98 | "\\end{equation}\n", 99 | "\n", 100 | "and direct integration,\n", 101 | "\n", 102 | "$$ \\mathbb{E}(Y \\mathbb{E}(X|Y))= \\int_{\\mathbb{R}} y \\int_{\\mathbb{R}} x \\frac{f_{X,Y}(x,y)}{f_Y(y)} f_Y(y) dx dy =\\int_{\\mathbb{R}^2} x y f_{X,Y}(x,y) dx dy =\\mathbb{E}( X Y) $$\n", 103 | "\n", 104 | "which is good to know, but not very geometrically intuitive. And this lack of geometric intuition makes it hard to apply these concepts and keep track of these relationships. \n", 105 | "\n", 106 | "We can keep pursuing this analogy and obtain the length of the error term as\n", 107 | "\n", 108 | "$$ \\langle X-h_{opt}(Y),X-h_{opt}(Y) \\rangle = \\langle X,X \\rangle - \\langle h_{opt}(Y),h_{opt}(Y) \\rangle $$\n", 109 | "\n", 110 | "and then by substituting all the notation we obtain\n", 111 | "\n", 112 | "\\begin{equation}\n", 113 | "\\mathbb{E}(X- \\mathbb{E}(X|Y))^2 = \\mathbb{E}(X)^2 - \\mathbb{E}(\\mathbb{E}(X|Y) )^2 \n", 114 | "\\end{equation}\n", 115 | "\n", 116 | "which would be tough to compute by direct integration.\n", 117 | "\n", 118 | "We recognize that $\\mathbb{E}(X|Y)$ *is* in fact **a projection operator**. Recall previously that we noted that the projection operator is idempotent, which means that once we project something onto a subspace, further projections essentially do nothing. Well, in the space of random variables, $\\mathbb{E}(X|\\cdot$) is the idempotent projection as we can show by noting that\n", 119 | "\n", 120 | "$$ h_{opt} = \\mathbb{E}(X|Y)$$\n", 121 | "\n", 122 | "is purely a function of $Y$, so that\n", 123 | "\n", 124 | "$$ \\mathbb{E}(h_{opt}(Y)|Y) = h_{opt}(Y) $$\n", 125 | "\n", 126 | "since $Y$ is fixed and this is the statement of idempotency. Thus, conditional expectation is the corresponding projection operator in this space of random variables. With this happy fact, we can continue to carry over our geometric interpretations of projections for vectors ($\\mathbf{v}$) into random variables ( $X$ ). \n", 127 | "\n", 128 | "Now that we have just stuffed our toolbox, let's consider some example conditional expectations obtained by using brute force to find the optimal MMSE function $h_{opt}$ as well as by using the definition of the conditional expectation." 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "metadata": {}, 134 | "source": [ 135 | "Example\n", 136 | "---------\n", 137 | "\n", 138 | "Suppose we have a random variable, $X$, then what constant is closest to $X$ in the mean-squared-sense (MSE)? In other words, which $c$ minimizes the following:\n", 139 | "\n", 140 | "$$ J = \\mathbb{E}( X - c )^2 $$ \n", 141 | "\n", 142 | "we can work this out as\n", 143 | "\n", 144 | "$$ \\mathbb{E}( X - c )^2 = \\mathbb{E}(c^2 - 2 c X + X^2) = c^2-2 c \\mathbb{E}(X) + \\mathbb{E}(X^2) $$ \n", 145 | "\n", 146 | "and then take the first derivative with respect to $c$ and solve:\n", 147 | "\n", 148 | "$$ c_{opt}=\\mathbb{E}(X) $$ \n", 149 | "\n", 150 | "Remember that $X$ can take on all kinds of values, but this says that the closest number to $X$ in the MSE sense is $\\mathbb{E}(X)$.\n", 151 | "\n", 152 | "Coming at this same problem using our inner product, we know that at the point of projection\n", 153 | "\n", 154 | "$$ \\mathbb{E}((X-c_{opt}) 1) = 0$$\n", 155 | "\n", 156 | "where the $1$ represents the space of constants (i.e. $c \\cdot 1 $) we are projecting on. This, by linearity of the expectation, gives\n", 157 | "\n", 158 | "$$ c_{opt}=\\mathbb{E}(X) $$ \n", 159 | "\n", 160 | "Because $\\mathbb{E}(X|Y)$ is the projection operator, with $Y=\\Omega$ (the entire underlying probability space), we have, using the definintion of conditional expectation:\n", 161 | "\n", 162 | "$$ \\mathbb{E}(X|Y=\\Omega) = \\mathbb{E}(X) $$\n", 163 | "\n", 164 | "Thus, we just worked the same problem three ways (optimization, inner product, projection)." 165 | ] 166 | }, 167 | { 168 | "cell_type": "markdown", 169 | "metadata": {}, 170 | "source": [ 171 | "Example\n", 172 | "-----------\n", 173 | "\n", 174 | "Let's consider the following example with probability density $f_{X,Y}= x + y $ where $(x,y) \\in [0,1]^2$ and compute the conditional expectation straight from the definition:\n", 175 | "\n", 176 | "$$ \\mathbb{ E}(X|Y) = \\int_0^1 x \\frac{f_{X,Y}(x,y)}{f_Y(y)} dx= \\int_0^1 x \\frac{x+y}{y+1/2} dx =\\frac{3 y + 2}{6 y + 3} $$\n", 177 | "\n", 178 | "That was pretty easy because the density function was so simple. Now, let's do it the hard way by going directly for the MMSE solution $h(Y)$. Then,\n", 179 | "\n", 180 | "$$ \\min_h \\int_0^1\\int_0^1 (x - h(y) )^2 f_{X,Y}(x,y)dx dy = \\min_h \\int_0^1 y h^2 {\\left (y \\right )} - y h{\\left (y \\right )} + \\frac{1}{3} y + \\frac{1}{2} h^{2}{\\left (y \\right )} - \\frac{2}{3} h{\\left (y \\right )} + \\frac{1}{4} dy $$ \n", 181 | "\n", 182 | "Now we have to find a function $h$ that is going to minimize this. Solving for a function, as opposed to solving for a number, is generally very, very hard, but because we are integrating over a finite interval, we can use the Euler-Lagrange method from variational calculus to take the derivative of the integrand with respect to the function $h(y)$ and set it to zero. Euler-Lagrange methods will be the topic of a later section, but for now we just want the result, namely,\n", 183 | "\n", 184 | "$$ 2 y h{\\left (y \\right )} - y + h{\\left (y \\right )} - \\frac{2}{3} =0 $$\n", 185 | "\n", 186 | "Solving this gives\n", 187 | "\n", 188 | "$$ h_{opt}(y)= \\frac{3 y + 2}{6 y + 3} $$\n", 189 | "\n", 190 | "Finally, we can try solving this using our inner product as\n", 191 | "\n", 192 | "$$ \\mathbb{E}( (X - h(Y)) Y ) = 0$$\n", 193 | "\n", 194 | "Writing this out gives,\n", 195 | "\n", 196 | "$$ \\int_0^1 \\int_0^1 (x-h(y))(x+y) dx dy = \\int_0^1 \\frac{y\\,\\left(2 + 3\\,y - 3\\,\\left( 1 + 2\\,y \\right) \\,h(y)\\right) }{6} dy = 0$$\n", 197 | "\n", 198 | "and if this is zero everywhere, then the integrand must be zero,\n", 199 | "\n", 200 | "$$ 2 y + 3 y^2 - 3 y h(y) - 6 y^2 h(y)=0 $$\n", 201 | "\n", 202 | "and solving this for $h(y)$ gives the same solution:\n", 203 | "\n", 204 | "$$ h_{opt}(y)= \\frac{3 y + 2}{6 y + 3} $$\n", 205 | "\n", 206 | "Thus, doing it by the definition, optimization, or inner product gives us the same answer; but, in general, no method is necessarily easier because they both involve potentially difficult or impossible integration, optimization, or functional equation solving. The point is that now that we have a deep toolbox, we can pick and choose which tools we want to apply for different problems.\n", 207 | "\n", 208 | "Before we leave this example, let's use `sympy` to verify the length of the error function we found earlier:\n", 209 | "\n", 210 | "\\begin{equation}\n", 211 | "\\mathbb{E}(X- \\mathbb{E}(X|Y))^2 = \\mathbb{E}(X)^2 - \\mathbb{E}(\\mathbb{E}(X|Y) )^2 \n", 212 | "\\end{equation}\n", 213 | "\n", 214 | "that is based on the Pythagorean theorem." 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "collapsed": false, 220 | "input": [ 221 | "from sympy.abc import y,x\n", 222 | "from sympy import integrate, simplify\n", 223 | "\n", 224 | "fxy = x + y # joint density\n", 225 | "fy = integrate(fxy,(x,0,1)) # marginal density\n", 226 | "fx = integrate(fxy,(y,0,1)) # marginal density\n", 227 | "\n", 228 | "h = (3*y+2)/(6*y+3) # conditional expectation\n", 229 | "LHS=integrate((x - h)**2 *fxy, (x,0,1),(y,0,1)) # from the definition\n", 230 | "RHS=integrate( (x)**2 *fx, (x,0,1)) - integrate( (h)**2 *fy, (y,0,1)) # using Pythagorean theorem\n", 231 | "print simplify(LHS-RHS)==0" 232 | ], 233 | "language": "python", 234 | "metadata": {}, 235 | "outputs": [ 236 | { 237 | "output_type": "stream", 238 | "stream": "stdout", 239 | "text": [ 240 | "True\n" 241 | ] 242 | } 243 | ], 244 | "prompt_number": 1 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "Summary\n", 251 | "--------\n", 252 | "\n", 253 | "In this section, we have pulled together all the projection and least-squares optimization ideas from the previous posts to draw a clean line between our geometric notions of projection from vectors in $\\mathbb{R}^n$ to general random variables. This resulted in the remarkable realization that the conditional expectation is in fact a projection operator for random variables. The key idea is that because we have these relationships, we can approach difficult problems in multiple ways, depending on which way is more intuitive or tractable in a particular situation. In these pages, we will again and again come back to these intuitions because they form the backbone of statistical signal processing. \n", 254 | "\n", 255 | "\n", 256 | "In the next section, we will have a lot of fun with these ideas working out some examples that are usually solved using more general tools from measure theory. \n", 257 | "\n", 258 | "Note that the book by Mikosch (1998) has some excellent sections covering much of this material in more detail. Mikosch has a very geometric view of the material as well." 259 | ] 260 | }, 261 | { 262 | "cell_type": "markdown", 263 | "metadata": {}, 264 | "source": [ 265 | "Appendix\n", 266 | "---------\n", 267 | "\n", 268 | "#### Proof of MMSE of Conditional expectation by Optimization\n", 269 | "\n", 270 | "$$ \\min_h \\int_\\mathbb{R^2} | X - h(Y) |^2 f_{x,y}(X,Y) dx dy $$\n", 271 | "\n", 272 | "$$ \\min_h \\int_\\mathbb{R^2} | X |^2 f_{x,y}(X,Y) dx dy + \\int_\\mathbb{R^2} | h(Y) |^2 f_{x,y}(X,Y) dx dy - \\int_\\mathbb{R^2} 2 X h(Y) f_{x,y}(X,Y) dx dy $$\n", 273 | "\n", 274 | "Now, we want to maximize the following:\n", 275 | "\n", 276 | "$$ \\max_h \\int_\\mathbb{R^2} X h(Y) f_{x,y}(X,Y) dx dy $$ \n", 277 | "\n", 278 | "Breaking up the integral using the definition of conditional expectation\n", 279 | "\n", 280 | "$$ \\max_h \\int_\\mathbb{R} \\left(\\int_\\mathbb{R} X f_{x|y}(X|Y) dx \\right)h(Y) f_Y(Y) dy $$ \n", 281 | "\n", 282 | "$$ \\max_h \\int_\\mathbb{R} \\mathbb{E}(X|Y) h(Y)f_Y(Y) dy $$ \n", 283 | "\n", 284 | "From properties of the Cauchy-Schwarz inequality, we know that the maximum happens when $h_{opt}(Y) = \\mathbb{E}(X|Y)$, so we have found the optimal $h(Y)$ function as :\n", 285 | "\n", 286 | "$$ h_{opt}(Y) = \\mathbb{E}(X|Y)$$ \n", 287 | "\n", 288 | "which shows that the optimal function is the conditional expectation." 289 | ] 290 | }, 291 | { 292 | "cell_type": "markdown", 293 | "metadata": {}, 294 | "source": [ 295 | "### References \n", 296 | "\n", 297 | "This post was created using the [nbconvert](https://github.com/ipython/nbconvert) utility from the source [IPython Notebook](www.ipython.org) which is available for [download](https://github.com/unpingco/Python-for-Signal-Processing/blob/master/Conditional_Expectation_Projection.ipynb) from the main github [site](https://github.com/unpingco/Python-for-Signal-Processing) for this blog. " 298 | ] 299 | }, 300 | { 301 | "cell_type": "markdown", 302 | "metadata": {}, 303 | "source": [ 304 | "### Bibliography\n", 305 | "\n", 306 | "* Nelson, Edward. Radically Elementary Probability Theory.(AM-117). Vol. 117. Princeton University Press, 1987.\n", 307 | "\n", 308 | "* Mikosch, Thomas. Elementary stochastic calculus with finance in view. Vol. 6. World Scientific Publishing Company Incorporated, 1998.\n" 309 | ] 310 | } 311 | ], 312 | "metadata": {} 313 | } 314 | ] 315 | } -------------------------------------------------------------------------------- /Example_CSVs.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "" 4 | }, 5 | "nbformat": 3, 6 | "nbformat_minor": 0, 7 | "worksheets": [ 8 | { 9 | "cells": [ 10 | { 11 | "cell_type": "heading", 12 | "level": 2, 13 | "metadata": {}, 14 | "source": [ 15 | "Examples using CSV files" 16 | ] 17 | }, 18 | { 19 | "cell_type": "markdown", 20 | "metadata": {}, 21 | "source": [ 22 | "Parsing comma-separtated-values (CSV) files is a common task. There are many tools available in Python to deal with this. Let's start by using the built-in `csv` module." 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "collapsed": false, 28 | "input": [ 29 | "import csv # using Python module" 30 | ], 31 | "language": "python", 32 | "metadata": {}, 33 | "outputs": [], 34 | "prompt_number": 147 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "Now, we want to open an example file, and read the contents:" 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "collapsed": false, 46 | "input": [ 47 | "f = open('example_1.csv','rb') # use binary mode if on MS windows\n", 48 | "d = [i for i in csv.reader(f) ] # use list comprehension to read from file\n", 49 | "f.close() # close file\n", 50 | "print d" 51 | ], 52 | "language": "python", 53 | "metadata": {}, 54 | "outputs": [ 55 | { 56 | "output_type": "stream", 57 | "stream": "stdout", 58 | "text": [ 59 | "[['Name', 'DOB', 'Years', 'Degree'], ['Alice Jones', '12/1/1980', '3', 'MS'], ['Bob Smith', '1/1/1969', '4', 'BS'], ['John Book', '5/3/1980', '11', 'BA'], ['Billy Blanks', '6/9/2000', '8', 'AA']]\n" 60 | ] 61 | } 62 | ], 63 | "prompt_number": 148 64 | }, 65 | { 66 | "cell_type": "heading", 67 | "level": 3, 68 | "metadata": {}, 69 | "source": [ 70 | "Adding new fields as columns" 71 | ] 72 | }, 73 | { 74 | "cell_type": "markdown", 75 | "metadata": {}, 76 | "source": [ 77 | "This reads the rows of the CSV file into a list-of-lists.\n", 78 | "\n", 79 | "Now, let's suppose we want to create columns for `last name` and `first name` instead of having just one `name` field. The first element in the list `d` is the header, so we had the additional fields there:" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "collapsed": false, 85 | "input": [ 86 | "d[0].append('Last Name')\n", 87 | "d[0].append('First Name')\n", 88 | "print d[0]" 89 | ], 90 | "language": "python", 91 | "metadata": {}, 92 | "outputs": [ 93 | { 94 | "output_type": "stream", 95 | "stream": "stdout", 96 | "text": [ 97 | "['Name', 'DOB', 'Years', 'Degree', 'Last Name', 'First Name']\n" 98 | ] 99 | } 100 | ], 101 | "prompt_number": 149 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "Now, we want to split the original `Name` field into first and last names and put these at the ends of their respective rows." 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "collapsed": false, 113 | "input": [ 114 | "for row in d[1:]: # start at 1st, not 0th column. Each row is a list\n", 115 | " first,last= row[0].split() # split on white-space\n", 116 | " row.append(last) # append to each row\n", 117 | " row.append(first)\n", 118 | " \n", 119 | "print d" 120 | ], 121 | "language": "python", 122 | "metadata": {}, 123 | "outputs": [ 124 | { 125 | "output_type": "stream", 126 | "stream": "stdout", 127 | "text": [ 128 | "[['Name', 'DOB', 'Years', 'Degree', 'Last Name', 'First Name'], ['Alice Jones', '12/1/1980', '3', 'MS', 'Jones', 'Alice'], ['Bob Smith', '1/1/1969', '4', 'BS', 'Smith', 'Bob'], ['John Book', '5/3/1980', '11', 'BA', 'Book', 'John'], ['Billy Blanks', '6/9/2000', '8', 'AA', 'Blanks', 'Billy']]\n" 129 | ] 130 | } 131 | ], 132 | "prompt_number": 150 133 | }, 134 | { 135 | "cell_type": "code", 136 | "collapsed": false, 137 | "input": [ 138 | "#%qtconsole" 139 | ], 140 | "language": "python", 141 | "metadata": {}, 142 | "outputs": [], 143 | "prompt_number": 151 144 | }, 145 | { 146 | "cell_type": "heading", 147 | "level": 3, 148 | "metadata": {}, 149 | "source": [ 150 | "Writing updated CSV file" 151 | ] 152 | }, 153 | { 154 | "cell_type": "markdown", 155 | "metadata": {}, 156 | "source": [ 157 | "Now, we want to write out our new data in CSV format" 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "collapsed": false, 163 | "input": [ 164 | "f = open('example_1_out.csv','wb') # write mode binary\n", 165 | "fw = csv.writer(f) # create csv writer\n", 166 | "fw.writerows(d) \n", 167 | "f.close() # close file " 168 | ], 169 | "language": "python", 170 | "metadata": {}, 171 | "outputs": [], 172 | "prompt_number": 152 173 | }, 174 | { 175 | "cell_type": "markdown", 176 | "metadata": {}, 177 | "source": [ 178 | "Now, opening the file `example_1_out.csv` using excel (or another reader) should show the new column. That covers the most direct and pure-Python way to dealing with CSV files. However, there are many other tools available. For example, `numpy` provides power methods to access these files." 179 | ] 180 | }, 181 | { 182 | "cell_type": "heading", 183 | "level": 2, 184 | "metadata": {}, 185 | "source": [ 186 | "Using Numpy to parse CSV files" 187 | ] 188 | }, 189 | { 190 | "cell_type": "markdown", 191 | "metadata": {}, 192 | "source": [ 193 | "Let's see how to accomplish the same work as above by using Numpy." 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "collapsed": false, 199 | "input": [ 200 | "import numpy as np \n", 201 | "\n", 202 | "d = np.loadtxt('example_1.csv',delimiter=',',dtype=str)\n", 203 | "\n", 204 | "print d\n", 205 | "print d.dtype" 206 | ], 207 | "language": "python", 208 | "metadata": {}, 209 | "outputs": [ 210 | { 211 | "output_type": "stream", 212 | "stream": "stdout", 213 | "text": [ 214 | "[['Name' 'DOB' 'Years' 'Degree']\n", 215 | " ['Alice Jones' '12/1/1980' '3' 'MS']\n", 216 | " ['Bob Smith' '1/1/1969' '4' 'BS']\n", 217 | " ['John Book' '5/3/1980' '11' 'BA']\n", 218 | " ['Billy Blanks' '6/9/2000' '8' 'AA']]\n", 219 | "|S12\n" 220 | ] 221 | } 222 | ], 223 | "prompt_number": 153 224 | }, 225 | { 226 | "cell_type": "markdown", 227 | "metadata": {}, 228 | "source": [ 229 | "Notice that we did not have to use `open` to get at the contents of the file. By default, the `delimiter` is any whitespace, so we had to change this to the comma character. The `dtype` specifies we want everything to be read in as a string. Numpy can figure out how long that string needs to be as it goes through the file so we don't have to specify that ahead of time (we probably don't know it anyway). In this case, the `dtype` turns out to be a twelve character string `S12`. Note that this is the maximum length is used for *all* strings so there is obviously a lot of extra space if most of the strings are short and just a few are long.\n", 230 | "\n", 231 | "Numpy provides many more ways of reading data via the `dtype`. For example," 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "collapsed": false, 237 | "input": [ 238 | "dt = [('name', 'S64'), # The first element of the tuple is our name for each respective column\n", 239 | " ('dob', 'S64'), # and the second element is the numpy dtype we want for that column.\n", 240 | " ('years', 'int'), # Here we want years as an integer, not a string\n", 241 | " ('degree', 'S64'), ]\n", 242 | "\n", 243 | "d = np.loadtxt('example_1.csv',delimiter=',',dtype=dt,skiprows=1) # skip the header row\n", 244 | "print d" 245 | ], 246 | "language": "python", 247 | "metadata": {}, 248 | "outputs": [ 249 | { 250 | "output_type": "stream", 251 | "stream": "stdout", 252 | "text": [ 253 | "[('Alice Jones', '12/1/1980', 3, 'MS') ('Bob Smith', '1/1/1969', 4, 'BS')\n", 254 | " ('John Book', '5/3/1980', 11, 'BA') ('Billy Blanks', '6/9/2000', 8, 'AA')]\n" 255 | ] 256 | } 257 | ], 258 | "prompt_number": 154 259 | }, 260 | { 261 | "cell_type": "markdown", 262 | "metadata": {}, 263 | "source": [ 264 | "The advantage of doing it this way is that now we can compute the `years` column using `numpy` tools. For example, here is the mean of the years." 265 | ] 266 | }, 267 | { 268 | "cell_type": "code", 269 | "collapsed": false, 270 | "input": [ 271 | "print d['years'].mean() # using numpy arrays" 272 | ], 273 | "language": "python", 274 | "metadata": {}, 275 | "outputs": [ 276 | { 277 | "output_type": "stream", 278 | "stream": "stdout", 279 | "text": [ 280 | "6.5\n" 281 | ] 282 | } 283 | ], 284 | "prompt_number": 155 285 | }, 286 | { 287 | "cell_type": "markdown", 288 | "metadata": {}, 289 | "source": [ 290 | "Now to get back to the main task at hand: splitting the name field into first and last name." 291 | ] 292 | }, 293 | { 294 | "cell_type": "code", 295 | "collapsed": false, 296 | "input": [ 297 | "import string\n", 298 | "\n", 299 | "n=map(string.split,d['name'])\n", 300 | "w=array([tuple(i)+tuple(j) for i,j in zip(d,n)], # list comprehension glues tuple-ized rows together\n", 301 | " dtype=dt+[('first','S64'),('last','S64')]) # append new dtypes to existing list of dtypes" 302 | ], 303 | "language": "python", 304 | "metadata": {}, 305 | "outputs": [], 306 | "prompt_number": 156 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "metadata": {}, 311 | "source": [ 312 | "That was kind of non-simple, but now we can write this to a CSV using `savetxt`.\n" 313 | ] 314 | }, 315 | { 316 | "cell_type": "code", 317 | "collapsed": false, 318 | "input": [ 319 | "# the comments are set to '' to avoid hash marks on the first line.\n", 320 | "np.savetxt('np_output.csv',w,delimiter=',',fmt='%s',header='name,dob,years,degree,first,last',comments='')" 321 | ], 322 | "language": "python", 323 | "metadata": {}, 324 | "outputs": [], 325 | "prompt_number": 157 326 | }, 327 | { 328 | "cell_type": "markdown", 329 | "metadata": {}, 330 | "source": [ 331 | "Now, you can inspect the so-generated file and verify it is a CSV." 332 | ] 333 | }, 334 | { 335 | "cell_type": "heading", 336 | "level": 2, 337 | "metadata": {}, 338 | "source": [ 339 | "Using pandas to parse CSV files" 340 | ] 341 | }, 342 | { 343 | "cell_type": "markdown", 344 | "metadata": {}, 345 | "source": [ 346 | "`pandas` is the real power tool for this job." 347 | ] 348 | }, 349 | { 350 | "cell_type": "code", 351 | "collapsed": false, 352 | "input": [ 353 | "import pandas as pd\n", 354 | "\n", 355 | "d = pd.read_csv('example_1.csv')\n", 356 | "print d\n", 357 | "print type(d)" 358 | ], 359 | "language": "python", 360 | "metadata": {}, 361 | "outputs": [ 362 | { 363 | "output_type": "stream", 364 | "stream": "stdout", 365 | "text": [ 366 | " Name DOB Years Degree\n", 367 | "0 Alice Jones 12/1/1980 3 MS\n", 368 | "1 Bob Smith 1/1/1969 4 BS\n", 369 | "2 John Book 5/3/1980 11 BA\n", 370 | "3 Billy Blanks 6/9/2000 8 AA\n", 371 | "\n" 372 | ] 373 | } 374 | ], 375 | "prompt_number": 158 376 | }, 377 | { 378 | "cell_type": "markdown", 379 | "metadata": {}, 380 | "source": [ 381 | "Now, we have read the CSV file as a `pandas` DataFrame which is a super-structure that sits on top of `numpy`. Let's examine the columns of this DataFrame.\n" 382 | ] 383 | }, 384 | { 385 | "cell_type": "code", 386 | "collapsed": false, 387 | "input": [ 388 | "print d.columns" 389 | ], 390 | "language": "python", 391 | "metadata": {}, 392 | "outputs": [ 393 | { 394 | "output_type": "stream", 395 | "stream": "stdout", 396 | "text": [ 397 | "Index([Name, DOB, Years, Degree], dtype=object)\n" 398 | ] 399 | } 400 | ], 401 | "prompt_number": 159 402 | }, 403 | { 404 | "cell_type": "markdown", 405 | "metadata": {}, 406 | "source": [ 407 | "Notice that there is an extra space after the `Name `. This potentially makes it hard to access the columns using pandas slicing. For example," 408 | ] 409 | }, 410 | { 411 | "cell_type": "code", 412 | "collapsed": false, 413 | "input": [ 414 | "print d.DOB # this works great when the column header name has no spaces in it.\n", 415 | "print d['Name'] # you can also refer to columns using this syntax" 416 | ], 417 | "language": "python", 418 | "metadata": {}, 419 | "outputs": [ 420 | { 421 | "output_type": "stream", 422 | "stream": "stdout", 423 | "text": [ 424 | "0 12/1/1980\n", 425 | "1 1/1/1969\n", 426 | "2 5/3/1980\n", 427 | "3 6/9/2000\n", 428 | "Name: DOB, dtype: object\n", 429 | "0 Alice Jones\n", 430 | "1 Bob Smith\n", 431 | "2 John Book\n", 432 | "3 Billy Blanks\n", 433 | "Name: Name, dtype: object\n" 434 | ] 435 | } 436 | ], 437 | "prompt_number": 161 438 | }, 439 | { 440 | "cell_type": "markdown", 441 | "metadata": {}, 442 | "source": [ 443 | "Luckily, this is not hard to fix. We just need to create another column that is free of these trailing spaces:" 444 | ] 445 | }, 446 | { 447 | "cell_type": "code", 448 | "collapsed": false, 449 | "input": [ 450 | "d['name']=d['Name '] # easily create extra column\n", 451 | "print d.name # now you can access this column using this syntax" 452 | ], 453 | "language": "python", 454 | "metadata": {}, 455 | "outputs": [] 456 | }, 457 | { 458 | "cell_type": "markdown", 459 | "metadata": {}, 460 | "source": [ 461 | "Pandas is a lot more powerful than this! We can parse the columns by types individually by providing a `dtype` for each column as a dictionary." 462 | ] 463 | }, 464 | { 465 | "cell_type": "code", 466 | "collapsed": false, 467 | "input": [ 468 | "d = pd.read_csv('example_1.csv',dtype={'Name ':'S64','DOB':'S64','Years':int,'Degree':'S64'})\n", 469 | "print d" 470 | ], 471 | "language": "python", 472 | "metadata": {}, 473 | "outputs": [] 474 | }, 475 | { 476 | "cell_type": "markdown", 477 | "metadata": {}, 478 | "source": [ 479 | "Now, we can compute along the columns as we did before with `numpy`." 480 | ] 481 | }, 482 | { 483 | "cell_type": "code", 484 | "collapsed": false, 485 | "input": [ 486 | "print d.Years.mean()" 487 | ], 488 | "language": "python", 489 | "metadata": {}, 490 | "outputs": [] 491 | }, 492 | { 493 | "cell_type": "markdown", 494 | "metadata": {}, 495 | "source": [ 496 | "You can also parse the `DOB` field to get a true timestamp instead of a string using the `parse_dates` keyword." 497 | ] 498 | }, 499 | { 500 | "cell_type": "code", 501 | "collapsed": false, 502 | "input": [ 503 | "d = pd.read_csv('example_1.csv',dtype={'Name':'S64',\n", 504 | " 'DOB':'S64',\n", 505 | " 'Years':int,\n", 506 | " 'Degree':'S64'},parse_dates=[1])" 507 | ], 508 | "language": "python", 509 | "metadata": {}, 510 | "outputs": [], 511 | "prompt_number": 171 512 | }, 513 | { 514 | "cell_type": "markdown", 515 | "metadata": {}, 516 | "source": [ 517 | "Now, we can compute with these `datetime` objects as in the following." 518 | ] 519 | }, 520 | { 521 | "cell_type": "code", 522 | "collapsed": false, 523 | "input": [ 524 | "# difference in birthdays between Alice Jones and John Book\n", 525 | "print d.DOB[0] - d.DOB[2]" 526 | ], 527 | "language": "python", 528 | "metadata": {}, 529 | "outputs": [ 530 | { 531 | "output_type": "stream", 532 | "stream": "stdout", 533 | "text": [ 534 | "212 days, 0:00:00\n" 535 | ] 536 | } 537 | ], 538 | "prompt_number": 164 539 | }, 540 | { 541 | "cell_type": "markdown", 542 | "metadata": {}, 543 | "source": [ 544 | "Now we now how many days are between the respective birthdays of Alice Jones and John Book." 545 | ] 546 | }, 547 | { 548 | "cell_type": "code", 549 | "collapsed": false, 550 | "input": [ 551 | "%qtconsole" 552 | ], 553 | "language": "python", 554 | "metadata": {}, 555 | "outputs": [] 556 | }, 557 | { 558 | "cell_type": "code", 559 | "collapsed": false, 560 | "input": [ 561 | "d['first']=map(lambda x:string.split(x)[0],d['Name'])\n", 562 | "d['last']=map(lambda x:string.split(x)[1],d['Name'])\n", 563 | "print d" 564 | ], 565 | "language": "python", 566 | "metadata": {}, 567 | "outputs": [ 568 | { 569 | "output_type": "stream", 570 | "stream": "stdout", 571 | "text": [ 572 | " Name DOB Years Degree first last\n", 573 | "0 Alice Jones 1980-12-01 00:00:00 3 MS Alice Jones\n", 574 | "1 Bob Smith 1969-01-01 00:00:00 4 BS Bob Smith\n", 575 | "2 John Book 1980-05-03 00:00:00 11 BA John Book\n", 576 | "3 Billy Blanks 2000-06-09 00:00:00 8 AA Billy Blanks\n" 577 | ] 578 | } 579 | ], 580 | "prompt_number": 166 581 | }, 582 | { 583 | "cell_type": "code", 584 | "collapsed": false, 585 | "input": [ 586 | "print d" 587 | ], 588 | "language": "python", 589 | "metadata": {}, 590 | "outputs": [ 591 | { 592 | "output_type": "stream", 593 | "stream": "stdout", 594 | "text": [ 595 | " Name DOB Years Degree first last\n", 596 | "0 Alice Jones 1980-12-01 00:00:00 3 MS Alice Jones\n", 597 | "1 Bob Smith 1969-01-01 00:00:00 4 BS Bob Smith\n", 598 | "2 John Book 1980-05-03 00:00:00 11 BA John Book\n", 599 | "3 Billy Blanks 2000-06-09 00:00:00 8 AA Billy Blanks\n" 600 | ] 601 | } 602 | ], 603 | "prompt_number": 167 604 | }, 605 | { 606 | "cell_type": "heading", 607 | "level": 2, 608 | "metadata": {}, 609 | "source": [ 610 | "Inject into a sqlite database" 611 | ] 612 | }, 613 | { 614 | "cell_type": "code", 615 | "collapsed": false, 616 | "input": [ 617 | "import pandas.io.sql as pd_sql\n", 618 | "import sqlite3 as sql # sqlite3 is built into Python\n", 619 | "\n", 620 | "con = sql.connect(\"example_1.db\")\n", 621 | "pd_sql.write_frame(d,'data',con) # write to DB as table named \"data\"\n", 622 | "con.close()" 623 | ], 624 | "language": "python", 625 | "metadata": {}, 626 | "outputs": [], 627 | "prompt_number": 170 628 | }, 629 | { 630 | "cell_type": "code", 631 | "collapsed": false, 632 | "input": [], 633 | "language": "python", 634 | "metadata": {}, 635 | "outputs": [] 636 | } 637 | ], 638 | "metadata": {} 639 | } 640 | ] 641 | } -------------------------------------------------------------------------------- /Exponential_splines.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "" 4 | }, 5 | "nbformat": 3, 6 | "nbformat_minor": 0, 7 | "worksheets": [ 8 | { 9 | "cells": [ 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Start notebook with --profile=sympy flag.\n", 15 | "\n", 16 | "Following the 1968 paper by Spath. Consider special case with equally spaced abscissae" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "collapsed": false, 22 | "input": [ 23 | "%qtconsole" 24 | ], 25 | "language": "python", 26 | "metadata": {}, 27 | "outputs": [], 28 | "prompt_number": 1 29 | }, 30 | { 31 | "cell_type": "code", 32 | "collapsed": false, 33 | "input": [ 34 | "(A, B, C, D)=symbols('A:D') # coefficients\n", 35 | "p = symbols('p') # tension parameter\n", 36 | "y1p = symbols('y1p') # derivative a left endpoint\n", 37 | "ynp = symbols('ynp') # derivative a right endpoint\n", 38 | "xcp = symbols('xcp') # x control point\n", 39 | "ycp = symbols('ycp') # y control point" 40 | ], 41 | "language": "python", 42 | "metadata": {}, 43 | "outputs": [], 44 | "prompt_number": 2 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "metadata": {}, 49 | "source": [ 50 | "This is the $k^{th}$ piece of the $n$-piece interpolant, \n", 51 | "\n", 52 | "$$f_k(x) = A_k+B_k (x-x_k) + C_k \\exp(p_k (x-x_k)) +D_k \\exp(-p_k (x-x_k))$$\n", 53 | "\n", 54 | "given the derivative of the target function at $x(0)$ and at the other end $x(n-1)$.\n" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "collapsed": false, 60 | "input": [ 61 | "f = A(k)+B(k)*(x-x(k)) + C(k)*exp(p(k)*(x-x(k))) +D(k)*exp(-p(k)*(x-x(k)))" 62 | ], 63 | "language": "python", 64 | "metadata": {}, 65 | "outputs": [], 66 | "prompt_number": 3 67 | }, 68 | { 69 | "cell_type": "markdown", 70 | "metadata": {}, 71 | "source": [ 72 | "Sample data to interpolate" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "collapsed": false, 78 | "input": [ 79 | "X =[0,xcp,1]\n", 80 | "Y =[0,ycp,1]" 81 | ], 82 | "language": "python", 83 | "metadata": {}, 84 | "outputs": [], 85 | "prompt_number": 4 86 | }, 87 | { 88 | "cell_type": "markdown", 89 | "metadata": {}, 90 | "source": [ 91 | "Set up each piece of interpolant" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "collapsed": false, 97 | "input": [ 98 | "c=[f.subs(x(k),X[i]).subs(k,i) for i in range(2)]" 99 | ], 100 | "language": "python", 101 | "metadata": {}, 102 | "outputs": [], 103 | "prompt_number": 5 104 | }, 105 | { 106 | "cell_type": "markdown", 107 | "metadata": {}, 108 | "source": [ 109 | "Left-side Interpolation conditions" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "collapsed": false, 115 | "input": [ 116 | "cond_i=[(Y[i]-f.subs(x,x(k)).subs(k,i)) for i in range(2)] # conditions for interpolation\n", 117 | "cond_i+= [ Y[2]- c[1].subs(x,X[2])]" 118 | ], 119 | "language": "python", 120 | "metadata": {}, 121 | "outputs": [], 122 | "prompt_number": 6 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": {}, 127 | "source": [ 128 | "Match end-point 1st derivatives from givens" 129 | ] 130 | }, 131 | { 132 | "cell_type": "code", 133 | "collapsed": false, 134 | "input": [ 135 | "cond_end=[ diff(f,x).subs(k,0).subs(x(0),X[0]).subs(x,X[0]) - y1p,\n", 136 | " diff(f,x).subs(k,1).subs(x(1),X[1]).subs(x,X[2]) - ynp,\n", 137 | " ]\n", 138 | "cond_end" 139 | ], 140 | "language": "python", 141 | "metadata": {}, 142 | "outputs": [ 143 | { 144 | "metadata": {}, 145 | "output_type": "pyout", 146 | "prompt_number": 7, 147 | "text": [ 148 | "[-y1p + B(0) + C(0)*p(0) - D(0)*p(0),\n", 149 | " -ynp + B(1) + C(1)*p(1)*exp((-xcp + 1)*p(1)) - D(1)*p(1)*exp(-(-xcp + 1)*p(1))]" 150 | ] 151 | } 152 | ], 153 | "prompt_number": 7 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "metadata": {}, 158 | "source": [ 159 | "Inner continuity conditions" 160 | ] 161 | }, 162 | { 163 | "cell_type": "code", 164 | "collapsed": false, 165 | "input": [ 166 | "cond_cont=[] # continuity conditions\n", 167 | "for i,j,v in zip(c[:-2],c[2:],range(1,3)):\n", 168 | " cond_cont.append( (i-j).subs(x,v) )\n", 169 | "cond_cont\n", 170 | "\n", 171 | "cond_cont=[(c[0]-c[1]).subs(x,X[1])]\n" 172 | ], 173 | "language": "python", 174 | "metadata": {}, 175 | "outputs": [], 176 | "prompt_number": 8 177 | }, 178 | { 179 | "cell_type": "markdown", 180 | "metadata": {}, 181 | "source": [ 182 | "Inner second derivatives must match" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "collapsed": false, 188 | "input": [ 189 | "d2=[i.diff(x,2) for i in c]\n", 190 | "cond2_cont=[] # 2nd derivative continuity conditions\n", 191 | "for i,j,v in zip(d2[:-2],d2[2:],range(3)):\n", 192 | " cond2_cont.append( (i-j).subs(x,v) )\n", 193 | " \n", 194 | "cond2_cont= [diff(c[0]-c[1],x,2).subs(x,X[1])]" 195 | ], 196 | "language": "python", 197 | "metadata": {}, 198 | "outputs": [], 199 | "prompt_number": 9 200 | }, 201 | { 202 | "cell_type": "markdown", 203 | "metadata": {}, 204 | "source": [ 205 | "Inner first derivatives must match" 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "collapsed": false, 211 | "input": [ 212 | "d=[i.diff(x) for i in c]\n", 213 | "cond1_cont=[] # 2nd derivative continuity conditions\n", 214 | "for i,j,v in zip(d[:-2],d[2:],range(3)):\n", 215 | " cond1_cont.append( (i-j).subs(x,v) )\n", 216 | " \n", 217 | "cond1_cont=[diff(c[0]-c[1],x).subs(x,X[1])]" 218 | ], 219 | "language": "python", 220 | "metadata": {}, 221 | "outputs": [], 222 | "prompt_number": 10 223 | }, 224 | { 225 | "cell_type": "code", 226 | "collapsed": false, 227 | "input": [ 228 | "len(cond_i)+len(cond_cont)+len(cond_end)+len(cond2_cont)+len(cond1_cont)" 229 | ], 230 | "language": "python", 231 | "metadata": {}, 232 | "outputs": [ 233 | { 234 | "metadata": {}, 235 | "output_type": "pyout", 236 | "prompt_number": 11, 237 | "text": [ 238 | "8" 239 | ] 240 | } 241 | ], 242 | "prompt_number": 11 243 | }, 244 | { 245 | "cell_type": "code", 246 | "collapsed": false, 247 | "input": [ 248 | "for i in (cond_i+cond_cont+cond_end+cond2_cont):\n", 249 | " print i.subs(p(k),0)" 250 | ], 251 | "language": "python", 252 | "metadata": {}, 253 | "outputs": [ 254 | { 255 | "output_type": "stream", 256 | "stream": "stdout", 257 | "text": [ 258 | "-A(0) - C(0) - D(0)\n", 259 | "ycp - A(1) - C(1) - D(1)\n", 260 | "-(-xcp + 1)*B(1) - A(1) - C(1)*exp((-xcp + 1)*p(1)) - D(1)*exp(-(-xcp + 1)*p(1)) + 1\n", 261 | "xcp*B(0) + A(0) - A(1) + C(0)*exp(xcp*p(0)) - C(1) + D(0)*exp(-xcp*p(0)) - D(1)\n", 262 | "-y1p + B(0) + C(0)*p(0) - D(0)*p(0)\n", 263 | "-ynp + B(1) + C(1)*p(1)*exp((-xcp + 1)*p(1)) - D(1)*p(1)*exp(-(-xcp + 1)*p(1))\n", 264 | "C(0)*p(0)**2*exp(xcp*p(0)) - C(1)*p(1)**2 + D(0)*p(0)**2*exp(-xcp*p(0)) - D(1)*p(1)**2\n" 265 | ] 266 | } 267 | ], 268 | "prompt_number": 12 269 | }, 270 | { 271 | "cell_type": "code", 272 | "collapsed": false, 273 | "input": [ 274 | "linsys=(cond_i+cond_cont+cond_end+cond2_cont+cond1_cont)\n", 275 | "M=Matrix([[ l.coeff(i) for i in flatten([[A(j),B(j),C(j),D(j)] for j in range(2)]) ] for l in linsys])" 276 | ], 277 | "language": "python", 278 | "metadata": {}, 279 | "outputs": [], 280 | "prompt_number": 13 281 | }, 282 | { 283 | "cell_type": "code", 284 | "collapsed": false, 285 | "input": [ 286 | "sum([abs(diff(i,x,2)) for i in c]) # curvature metric" 287 | ], 288 | "language": "python", 289 | "metadata": {}, 290 | "outputs": [ 291 | { 292 | "metadata": {}, 293 | "output_type": "pyout", 294 | "prompt_number": 14, 295 | "text": [ 296 | "Abs(C(0)*p(0)**2*exp(x*p(0)) + D(0)*p(0)**2*exp(-x*p(0))) + Abs(C(1)*p(1)**2*exp((x - xcp)*p(1)) + D(1)*p(1)**2*exp(-(x - xcp)*p(1)))" 297 | ] 298 | } 299 | ], 300 | "prompt_number": 14 301 | }, 302 | { 303 | "cell_type": "code", 304 | "collapsed": false, 305 | "input": [ 306 | "print c[0]\n", 307 | "print c[1]" 308 | ], 309 | "language": "python", 310 | "metadata": {}, 311 | "outputs": [ 312 | { 313 | "output_type": "stream", 314 | "stream": "stdout", 315 | "text": [ 316 | "x*B(0) + A(0) + C(0)*exp(x*p(0)) + D(0)*exp(-x*p(0))\n", 317 | "(x - xcp)*B(1) + A(1) + C(1)*exp((x - xcp)*p(1)) + D(1)*exp(-(x - xcp)*p(1))\n" 318 | ] 319 | } 320 | ], 321 | "prompt_number": 15 322 | }, 323 | { 324 | "cell_type": "code", 325 | "collapsed": false, 326 | "input": [], 327 | "language": "python", 328 | "metadata": {}, 329 | "outputs": [], 330 | "prompt_number": 15 331 | } 332 | ], 333 | "metadata": {} 334 | } 335 | ] 336 | } -------------------------------------------------------------------------------- /Hypothesis_Testing.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "", 4 | "signature": "sha256:4220cb9e8dee84a340296374931ce0e8a5a6e3a8b7ef2ec1adccfd45a38d435f" 5 | }, 6 | "nbformat": 3, 7 | "nbformat_minor": 0, 8 | "worksheets": [ 9 | { 10 | "cells": [ 11 | { 12 | "cell_type": "code", 13 | "collapsed": false, 14 | "input": [ 15 | "%matplotlib inline" 16 | ], 17 | "language": "python", 18 | "metadata": {}, 19 | "outputs": [], 20 | "prompt_number": 1 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "metadata": {}, 25 | "source": [ 26 | "Given the coin-tossing experiment with unknown parameter $p$. The individual coin-flips are Bernoulli distributed. As we discussed previously, the estimator for $p$ is the following:\n", 27 | "\n", 28 | "$ \\hat{p} = \\frac{1}{n} \\sum_{i=1}^n X_i$\n", 29 | "\n", 30 | "We have to separate hypotheses: $H_0$ is the so-called null hypothesis. In our case this can be \n", 31 | "\n", 32 | "$$ H_0 : p \\lt \\frac{1}{2}$$\n", 33 | "\n", 34 | "and the alternative hypothesis is then\n", 35 | "\n", 36 | "$$ H_1 : p \\ge \\frac{1}{2}$$\n", 37 | "\n", 38 | "To choose between these, we need a statistical test that is a function, $G$, of the sample set\n", 39 | "$\\mathbf{X}_n=\\left\\{ X_i \\right\\}_n $. This test will have a threshold, something like\n", 40 | "\n", 41 | "$$ G(\\mathbf{X}_n) < c \\implies H_0 $$ \n", 42 | "\n", 43 | "and otherwise choose $H_1$. To sum up at this point, we have the observed data $\\mathbf{X}_n$ and a function $G$ that will somehow map that data onto the real line. Then, using the constant $c$ as a threshold, the inequality effectively divides the real line into two parts, one corresponding to each of the hypotheses.\n", 44 | "\n", 45 | "Whatever this test $G$ is, it will make mistakes of two types -- false negatives and false positives. The false positives arise from the case where we declare $H_0$ when the test says we should declare $H_1$. Here are the false positives (i.e. false alarm):\n", 46 | "\n", 47 | "$$ \\mathbb{P}_{FA} = \\mathbb{P}( G(\\mathbf{X}_n) \\gt c \\vert p \\le \\frac{1}{2})$$\n", 48 | "\n", 49 | "Or, alternatively,\n", 50 | "\n", 51 | "$$ \\mathbb{P}_{FA} = \\mathbb{P}( G(\\mathbf{X}_n) \\gt c \\vert H_0)$$\n", 52 | "\n", 53 | "Likewise, the other error is a false negative, which we can write analogously as\n", 54 | "\n", 55 | "$$ \\mathbb{P}_{FN} = \\mathbb{P}( G(\\mathbf{X}_n) \\lt c \\vert H_1)$$\n", 56 | "\n", 57 | "Thus, by choosing some acceptable values for either of these errors, we can solve for the other one. Later we will see that this is because these two quantities are co-dependent. The practice is usually to pick a value of $ \\mathbb{P}_{FA}$ and then find the corresponding value of $ \\mathbb{P}_{FN}$. Note that it is traditional to speak about *detection probability*, which is defined as\n", 58 | "\n", 59 | "$$ \\mathbb{P}_{D} = 1- \\mathbb{P}_{FN} = \\mathbb{P}( G(\\mathbf{X}_n) \\gt c \\vert H_1)$$ \n", 60 | "\n", 61 | "In other words, this is the probability of declaring $H_1$ when the test exceeds the threshold. This is Otherwise known as the *probability of a true detection*." 62 | ] 63 | }, 64 | { 65 | "cell_type": "heading", 66 | "level": 2, 67 | "metadata": {}, 68 | "source": [ 69 | "Back to the coin flipping example" 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "In our previous problem, we wanted to derive an estimator for the underlying probability for the coin flipping experiment. For this example, we want to ask a softer question: is the underlying probability greater or less than 1/2. So, this leads to the two following hypotheses:\n", 77 | "\n", 78 | "$$ H_0 : p \\lt \\frac{1}{2}$$\n", 79 | "\n", 80 | "versus,\n", 81 | "\n", 82 | "$$ H_1 : p \\ge \\frac{1}{2}$$\n", 83 | "5\n", 84 | "Let's suppose we want to determine is based upon five observations. Now the only ingredient we're missing is the $G$ function and a way to pick between the two hypotheses. Out of the clear blue sky, let's pick the number of heads observed in the sequence of five observations. Thus, we have\n", 85 | "\n", 86 | "$$ G (\\mathbf{X}_5) := \\sum_{i=1}^5 X_i$$\n", 87 | "\n", 88 | "and, suppose further that we pick $H_1$ only if exactly five out of five observations are heads. We'll call this the *all-heads* test.\n", 89 | "\n", 90 | "Now, because all of the $X_i$ are random variables, so is $G$. Now, to find the corresponding probability mass function for this. As usual, assuming independence, the probability of five heads is $p^5$. This means that the probability of rejecting the $H_0$ hypothesis (and choosing $H_1$, because there are only two choices here) based on the unknown underlying probability is $p^5$. In the parlance, this is known and the *power function* as in denoted by $\\beta$ as in\n", 91 | "\n", 92 | "$$ \\beta(\\theta) = \\theta^5 $$\n", 93 | "\n", 94 | "where I'm using the standard $\\theta$ symbol to represent the underlying parameter ($p$ in this case). Let's get a quick plot this below." 95 | ] 96 | }, 97 | { 98 | "cell_type": "code", 99 | "collapsed": false, 100 | "input": [ 101 | "px = linspace(0,1,50)\n", 102 | "plot( px, px**5)\n", 103 | "xlabel(r'$\\theta$',fontsize=18)\n", 104 | "ylabel(r'$\\beta$',fontsize=18)" 105 | ], 106 | "language": "python", 107 | "metadata": {}, 108 | "outputs": [ 109 | { 110 | "metadata": {}, 111 | "output_type": "pyout", 112 | "prompt_number": 2, 113 | "text": [ 114 | "" 115 | ] 116 | }, 117 | { 118 | "metadata": {}, 119 | "output_type": "display_data", 120 | "png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEXCAYAAACDChKsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHRJJREFUeJzt3Xm0VNWZ9/HvI6CioqgomAuCARU14gRoNMZKQxuc2iEq\n7YDt0AndrzjE2CEmJpIVW5vuNwtU3ggNBLEdIDQqqAQ0YokhiqIiDqCAYJjECYUIKvfyvH/sCxSX\nC9St4QxVv89aZ9W0b9Vzz7rUj7332eeYuyMiItJUu8RdgIiIpJMCRERECqIAERGRgihARESkIAoQ\nEREpiAJEREQKkogAMbPfm9kqM3tjB23uNrMFZva6mR0XZX0iIrKtRAQIMAbos70XzexMoIu7Hwr8\nCLg3qsJERKRxiQgQd38eWL2DJv8AjK1vOwtobWZto6hNREQal4gAyUMNsDTn8TKgfUy1iIgI6QkQ\nAGvwWOdgERGJUfO4C8jTcqBDzuP29c9tpUuXLr5o0aLIihIRqRCL3L1LU38oLT2QycAVAGZ2EvCZ\nu69q2GjRokW4uzZ3brvttthrSMqmfaF9oX3R+Hbffc755ztA50K+mBMRIGb2MPAX4HAzW2pmV5tZ\nfzPrD+DuU4D3zGwhMAL4PzGWKyJSER57DM47r/CfT8QQlrtfkkebAVHUIiJSDdatg2eegVGjCn+P\nRPRApPQymUzcJSSG9sUW2hdbVPu+ePpp6N4d9t+/8Pcw98o5mMnMvJJ+HxGRcrnqKjjuOLj+ejAz\n3L3hka47pQAREakytbXQrh288gp07Fh4gGgIS0SkysycCQcfHMKjGAoQEZEqU+zRV5sk4igsERGJ\nhnsIkEmTin8v9UBERKrI3LlgBkcfXfx7KUBERKrIpuEra/KU+bYUICIiVaRU8x+gABERqRpLlsCy\nZXDyyaV5PwWIiEiVmDQJzjkHmpfo8CkFiIhIlSjl8BVoJbqISFX45BM45BBYtQpattz6Na1EFxGR\n7XriCejde9vwKIYCRESkCpR6+Ao0hCUiUvHWrQsnT1y8uPHTt2sIS0REGlWKa380RgEiIlLhHnmk\n9MNXoCEsEZGK9uWXcNBB8OabUFPTeBsNYYmIyDaeeiqcOHF74VEMBYiISAUbPx769i3Pe2sIS0Sk\nQq1fH4av3nkH2rbdfjsNYYmIyFamTIETTthxeBRDASIiUqHKOXwFGsISEalIf/tbmDhftAjatNlx\nWw1hiYjIZk88Ad/+9s7DoxgKEBGRClTu4SvQEJaISMVZswbat4f334d99915ew1hiYgIEK48eNpp\n+YVHMRQgIiIVJorhK9AQlohIRVm9Gjp1gqVLYe+98/sZDWGJiAiPPgq9euUfHsVQgIiIVJCohq9A\nQ1giIhXjo4+gSxdYsQL23DP/n9MQlohIlXvkEejTp2nhUQwFiIhIhYhy+AoSEiBm1sfM5pvZAjMb\n2MjrbcxsqpnNMbM3zezKGMoUEUmsDz6AV1+FM86I7jNjDxAzawYMA/oARwKXmNkRDZoNAF5z92OB\nDPBbM2seaaEiIgk2YQKcfTa0bBndZ8YeIEBPYKG7L3H3DcA44NwGbVYCmw5K2xv4xN1rI6xRRCTR\n7r8f+vWL9jOT8L/4GmBpzuNlwIkN2owEppvZCqAVcHFEtYmIJN7bb4cjr3r3jvZzkxAg+Rx3+3Ng\njrtnzKwz8LSZHePuaxs2HDRo0Ob7mUyGTCZTqjpFRBLp/vvhssugWbP82mezWbLZbNGfG/s6EDM7\nCRjk7n3qH98CbHT3wTltpgD/7u4z6x8/Awx099kN3kvrQESkqtTVQceOMG0aHHVUYe+R5nUgs4FD\nzayTme0K9AUmN2gzH+gNYGZtgcOB9yKtUkQkgaZPh3btCg+PYsQ+hOXutWY2AJgGNANGu/s8M+tf\n//oI4A5gjJm9Tgi9n7r7p7EVLSKSEGPHwhVXxPPZsQ9hlZKGsESkmqxZAwcfDAsWwAEHFP4+aR7C\nEhGRAkycCJlMceFRDAWIiEhKxTl8BRrCEhFJpcWLoWdPWLYMdtutuPfSEJaISBX5n/8JJ04sNjyK\nEftRWCIi0jTuYfHgQw/FW4d6ICIiKfOXv0CLFtCjR7x1KEBERFJm7Fj4p38Ca/KsRWlpEl1EJEXW\nr4eaGpg7F9q3L817ahJdRKQKTJ4MJ5xQuvAohgJERCRFNg1fJYGGsEREUuKDD+CII8Lajz33LN37\naghLRKTC3XcfXHBBacOjGOqBiIikwMaN0KULjB9f+sN31QMREalgTz8NrVtD9+5xV7KFAkREJAVG\njID+/eNf+5FLQ1giIgm3YkW44uBf/wqtWpX+/TWEJSJSoUaPDidOLEd4FEM9EBGRBKurg0MOgUmT\n4LjjyvMZ6oGIiFSgqVOhXbvyhUcxFCAiIgk2fDj8y7/EXUXjNIQlIpJQS5fCsceGyfNyLh7UEJaI\nSIUZNQouuSQ5K88bUg9ERCSBamuhUyf44x/h6KPL+1nqgYiIVJAnnoCOHcsfHsVQgIiIJNCmledJ\npiEsEZGEWbw4nDBx6VJo2bL8n6chLBGRCjFyJPTrF014FEM9EBGRBPn66zD3MX16uHhUFNQDERGp\nABMmhOCIKjyKoQAREUkIdxgyBH7847gryY8CREQkIf78Z1izBs46K+5K8qMAERFJiKFD4YYbYJeU\nfDNrEl1EJAE2Hbq7ZAnstVe0n61JdBGRFLv7brjmmujDoxjqgYiIxGzNmnDRqDlzoEOH6D8/1T0Q\nM+tjZvPNbIGZDdxOm4yZvWZmb5pZNuISRUTKZvRoOP30eMKjGLH3QMysGfAO0BtYDrwMXOLu83La\ntAZmAt9392Vm1sbdP27kvdQDEZFUqauDLl1g3Dg48cR4akhzD6QnsNDdl7j7BmAccG6DNpcCE919\nGUBj4SEikkaPPQYHHRRfeBQjCQFSAyzNebys/rlchwL7mdmzZjbbzPpFVp2ISBkNHZqehYMNNY+7\nACCfMacWwPFAL2AP4AUze9HdF5S1MhGRMpo9O5xx9/zz466kMEkIkOVA7tRRB0IvJNdS4GN3Xw+s\nN7MZwDHANgEyaNCgzfczmQyZTKbE5YqIlMaQIXDdddA84m/ibDZLNpst+n2SMInenDCJ3gtYAbzE\ntpPoXYFhwPeB3YBZQF93f7vBe2kSXURSYfnycLXBxYthn33iraXQSfTYeyDuXmtmA4BpQDNgtLvP\nM7P+9a+PcPf5ZjYVmAtsBEY2DA8RkTQZNixc8yPu8ChG7D2QUlIPRETSYO1a+OY34cUXoXPnuKtJ\n92G8IiJVZfhw6N07GeFRDPVAREQitH596H089VSYA0kC9UBERFJg1KiwaDAp4VEM9UBERCLy9dfh\ntCUTJ4ZTtyeFeiAiIgl3//1w5JHJCo9iqAciIhKB2lro2hXGjIFTT427mq2pByIikmDjx0NNTfLC\noxixLyQUEal0GzfCHXeEU5dUEvVARETK7LHHYM894e//Pu5KSksBIiJSRu5w++1w661gTZ5lSDYF\niIhIGU2dGibQzz477kpKTwEiIlIm7vCb38AvfgG7VOC3bQX+SiIiyZDNwiefwIUXxl1JeShARETK\nYFPv42c/g2bN4q6mPHZ6GK+ZtQCuBr4AJrj7V2WvSkQk5f70p3DRqH794q6kfHa6Et3MhhAuM3s4\nUEu4cuANwPmAAVng/7r7+2WtNA9aiS4iSeAeTlcycCBcdFHc1excOa9I+Lm7/7j+Q/oSrgq4O3A/\nsA/wd8DVZna5uz/a1AJERCrNxIkhRH7wg7grKa98AuTrTXfcfbyZXQvMcfebNj1vZhlgpJktdvc5\npS9TRCQdamvDUVf33FOZR17lyuvXM7PcKaCXCPMhm7l7FsgA15aqMBGRNLrvvnDOq0pbdd6YfAJk\nFPCTnMfvAH9p2MjdlwMflKguEZHUWb8efv3rcN6rSlt13pidDmG5+4dmNsHMrgcecPeRO2heBbtM\nRKRxv/sddO8OJ50UdyXRyOtsvO6+2MzuBfqa2f7AcuAZd1+9qY2ZtQV0CJSIVKXPP4fBg+HZZ+Ou\nJDoFXVDKzNoDpwKtgBbAN4GWwM3uvq6kFTatLh3GKyKx+OUvYenSMAeSNoUexluSKxKaWQfgFMJh\nvbsR5kJecPelRb950+pQgIhI5FatCpeqfeUV6NQp7mqaLtYAaaSYDoSjsjq6++0l/4Dtf64CREQi\nd/31YdL8rrvirqQwiQqQuChARCRqixeHifN58+DAA+OupjC6JrqISAxuvRWuvTa94VEM9UBERAo0\ncyb07Qvz58Nee8VdTeHUAxERiVBdXZj7GDw43eFRDAWIiEgBxoyB3XeHSy+Nu5L4aAhLRKSJPvsM\nunaFJ5+EE06Iu5ri6SgsFCAiEo2bboK1a2Hkjk7slCIKEBQgIlJ+8+bBd78Lb71VOUdeaRJdRKTM\n3OHGG8P1PiolPIqhABERydPjj4fzXV2rKx8BeZ6NV0Sk2n31VZj7+N3voEWLuKtJhkT0QMysj5nN\nN7MFZjZwB+16mFmtmV0QZX0iIkOGwFFHwemnx11JcsQ+iV5/udx3gN6E64y8DFzi7vMaafc0sA4Y\n4+4TG3kvTaKLSMmtWAHdusGsWdC5c9zVlF6aJ9F7AgvdfYm7bwDGAec20u464H+Bj6IsTkTkxhuh\nf//KDI9iJGEOpAbIvW7IMuDE3AZmVkMIlb8DeqArH4pIRCZNgjlzYOzYuCtJniQESD5hMBT4mbu7\nmRm69rqIRODzz2HAAHjgAWjZMu5qkicJAbIc6JDzuAOhF5LrBGBcyA7aAGeY2QZ3n9zwzQYNGrT5\nfiaTIZPJlLhcEakWAwfCmWfCaafFXUlpZbNZstls0e+ThEn05oRJ9F7ACuAlGplEz2k/Bnjc3R9p\n5DVNootIScyYEU6U+NZbsM8+cVdTXoVOosfeA3H3WjMbAEwDmgGj3X2emfWvf31ErAWKSNX58kv4\n4Q9h2LDKD49ixN4DKSX1QESkFH7xC3j3XZgwIe5KopHaHoiISJK8/no4y+7cuXFXknxJWAciIpII\ntbVwzTVw553Qrl3c1SSfAkREpN5dd4U5j6uvjruSdNAciIgIsHAhnHQSvPgidOkSdzXRSvOpTERE\nYrVhA1x+Odx6a/WFRzEUICJS9W6/HVq3huuvj7uSdNFRWCJS1WbOhBEj4LXXYBf9l7pJtLtEpGp9\n/nkYuhoxAg46KO5q0keT6CJSta64AvbYA4YPj7uSeGkhoYhIEzz8MLz0ErzyStyVpJd6ICJSdd5/\nH3r0gKlT4fjj464mfjqMV0QkD3V10K8f3HyzwqNYChARqSr/+Z/QrBn85CdxV5J+mgMRkaoxcyYM\nHQqzZ4cQkeKoByIiVWHlSujbF8aMgQ4ddt5edk4BIiIVb8MGuPhi+NGPwiVqpTR0FJaIVLwbboD3\n3oNJk7TavDFaByIi0ogHHoApU+DllxUepaYeiIhUrNdfh969Yfp0OProuKtJLq0DERHJsXo1XHAB\n3H23wqNc1AMRkYqzcSOcfTYcfjgMGRJ3NcmnHoiISL1f/xq++CIsGpTy0SS6iFSUhx6C++6DWbOg\nRYu4q6lsChARqRjPPQc33hgmzdu1i7uayqchLBGpCG+/HRYLPvwwfOtbcVdTHRQgIpJ6K1eGFeb/\n9V/Qq1fc1VQPBYiIpNrf/haOuLrmmnCFQYmODuMVkdSqrYXzzoO2bWHUKLAmH4gqoMN4RaTKuMN1\n14UTJQ4frvCIg47CEpFU+o//gBdegBkzdLhuXBQgIpI699wThqxmzIC99467muqlABGRVBkxAn77\nW8hmoaYm7mqqmwJERFJjzBi4/fYQHp06xV2NKEBEJBUefBBuvTWsMu/cOe5qBBQgIpICEybAzTfD\nn/4UzrAryaAAEZFEe+yxcLjutGlw1FFxVyO5ErMOxMz6mNl8M1tgZgMbef0yM3vdzOaa2Uwz6xZH\nnSISnSeegP79wyVpjzkm7mqkoUQEiJk1A4YBfYAjgUvM7IgGzd4Dvuvu3YDfAP8dbZUiEqUHH4R/\n/md4/HE4/vi4q5HGJGUIqyew0N2XAJjZOOBcYN6mBu7+Qk77WUD7KAsUkejcdVc4VHf6dDjyyLir\nke1JSoDUAEtzHi8DTtxB+2uAKWWtSEQi5w6/+hX84Q/w/PPQsWPcFcmOJCVA8j4Dopl9D7gaOKWx\n1wcNGrT5fiaTIZPJFFmaiEShrg6uvRZmz4Y//xkOOCDuiipXNpslm80W/T6JOBuvmZ0EDHL3PvWP\nbwE2uvvgBu26AY8Afdx9YSPvo7PxiqTQV1/B5ZfDp5+Go65atYq7ouqS9rPxzgYONbNOZrYr0BeY\nnNvAzA4mhMfljYWHiKTT2rVw1lmwcSM8+aTCI00SESDuXgsMAKYBbwPj3X2emfU3s/71zX4F7Avc\na2avmdlLMZUrIiWyZAmccgp06RLmPXbfPe6KpCkSMYRVKhrCEkmP556Df/xH+PnPYcAAXc8jToUO\nYSVlEl1Eqsjw4XDbbWGtR+/ecVcjhVKAiEhkNmyAG24IZ9OdOTMMXUl6KUBEJBIffwwXXhgmyV98\nUReCqgSJmEQXkcr26qvQsyecfHI4TFfhURnUAxGRsnGHoUPhzjth2DC4+OK4K5JSUoCISFl8+CFc\ndRV88gnMmgWHHBJ3RVJqGsISkZJ75hk47jjo1i2c00rhUZnUAxGRktmwIZwM8f77YexYHaJb6RQg\nIlIS8+fDlVfC/vvDnDk6GWI10BCWiBRlwwa44w74znfCCREff1zhUS3UAxGRgs2eHa4aeNBB8Mor\nun5HtVEPRESabN06+Ld/C2fRvfnmcM1yhUf1UYCISJM8+2w4umrZMnjjjTBspRMhVicNYYlIXhYv\nhp/+FF56KSwKPOecuCuSuKkHIiI7tGYN3HILdO8OxxwTjrZSeAgoQERkO+rqYNQo6NoVVq4Mw1W3\n3gotW8ZdmSSFhrBEZBvTp8NNN8Fee8HkyaH3IdKQAkRENstmYdCgMEF+xx1w0UWaIJftU4CIyFbB\n8ctfwmWXQXN9O8hO6E9EpIopOKQY+lMRqTK1tTBpEgwZAh98oOCQwulPRqRKfPYZjB4N99wD7dvD\njTfCeecpOKRw+tMRqXALFsDdd8ODD8IZZ8Af/hAuLytSLAWISAX66qswTPX734frkf/wh2EdR01N\n3JVJJVGAiFSQ114LofHww3DssXD11fDoo1r8J+WhABFJuQ8/hPHjQ3CsXh0u6jR7NnTqFHdlUunM\n3eOuoWTMzCvp9xHZng8+gEcegQkTQq/j7LPhqqvge9+DXXSCImkiM8Pdm7xkVAEikhIrV8LEiSE0\n5s4N1+K48EL4/vc1RCXFUYCgAJHKUlcXTp0+ZQr88Y+waFHoaVx0EZx+Ouy+e9wVSqVQgKAAkfRb\ntQqmTQuB8dRT4aipM84I28knw667xl2hVCIFCAoQSZ8VK+C552DGjHC7YgX06hUCo0+fsOBPpNwU\nIChAJNnq6uCdd8Kw1PPPh9BYvRpOPRVOOy1s3bpBs2ZxVyrVRgGCAkSSwx3efx9efjkExssvhwV9\nbdtCjx5wyikhMI48UkdNSfwUIChAJB5r18Kbb4aV3rnbrruGU4b06BFuu3eH/faLu1qRbSlAUIBI\n+WzcCMuXw7vvbr29/XZYyHfEEXD00VtvbdvqYkySDqkOEDPrAwwFmgGj3H1wI23uBs4A1gFXuvtr\njbRRgEhB3MPZapcsCdv772+5v3gxLFwI++wDhx229da1K3TurHkLSbdCAyT2U5mYWTNgGNAbWA68\nbGaT3X1eTpszgS7ufqiZnQjcC5wUS8Epkc1myWQycZcRO3d48sksXbtm+PDDsBhv+fJwtNOmbfny\nsEE4/UenTtCxY7g99dRwe9hh0KpVfL9HqejvYgvti+LFHiBAT2Chuy8BMLNxwLnAvJw2/wCMBXD3\nWWbW2szauvuqqItNi0r7x1FXF+Ya1qwJ22efwaefhqOYVq/ecv/TT+Hjj8Ow0kcfhc09S4cOGQ44\nAL7xjS1b165b7tfUQOvWlT/kVGl/F8XQviheEgKkBlia83gZcGIebdoDCpCI1dXBhg2Nb19/Hbav\nvgpb7v0vvwzb+vVb365bB198sf1tU2CsXx96AHvvHW5btw4T0vvuu2U79NBw26YNHHggHHBA2AYP\nDpdtFZHSSkKA5Dtp0fD/ho3+3DnnFFdMvnY01bK91xo+n/u4sfu5tzu6n7tt3Bhu//pXmDo1PM7d\nNrWpq9ty2/D+pq22dttbgBYttt2aN4fddgvbrrtue9uyZdh2333r2zZtwnDRXnvBnntuve21V5h3\naNUqPK703oFI2sQ+iW5mJwGD3L1P/eNbgI25E+lmNhzIuvu4+sfzgdMaDmGZ2UKgc2TFi4hUhkXu\n3qWpP5SEHshs4FAz6wSsAPoClzRoMxkYAIyrD5zPGpv/KGQHiIhIYWIPEHevNbMBwDTCYbyj3X2e\nmfWvf32Eu08xszPrexhfAFfFWLKIiJCAISwREUmnVJ6Fx8z6mNl8M1tgZgO30+bu+tdfN7Pjoq4x\nKjvbF2Z2Wf0+mGtmM82sWxx1RiGfv4v6dj3MrNbMLoiyvqjk+e8jY2avmdmbZpaNuMTI5PHvo42Z\nTTWzOfX74soYyoyEmf3ezFaZ2Rs7aNO07013T9VGGOZaCHQCWgBzgCMatDkTmFJ//0TgxbjrjnFf\nfBvYp/5+n2reFzntpgNPAD+Iu+6Y/iZaA28B7esft4m77hj3xSDgzk37AfgEaB537WXaH6cCxwFv\nbOf1Jn9vprEHsnnhobtvADYtPMy11cJDoLWZtY22zEjsdF+4+wvu/nn9w1mE9TOVKJ+/C4DrgP8F\nPoqyuAjlsx8uBSa6+zIAd/844hqjks++WAnsXX9/b+ATd6+NsMbIuPvzwOodNGny92YaA6SxRYU1\nebSpxC/OfPZFrmuAKWWtKD473RdmVkP4Arm3/qlKnADM52/iUGA/M3vWzGabWb/IqotWPvtiJHCU\nma0AXgduiKi2JGry92bsR2EVoKQLD1Mu79/JzL4HXA2cUr5yYpXPvhgK/Mzd3cyMbf9GKkE++6EF\ncDzQC9gDeMHMXnT3BWWtLHr57IufA3PcPWNmnYGnzewYd19b5tqSqknfm2kMkOVAh5zHHQhJuaM2\n7eufqzT57AvqJ85HAn3cfUdd2DTLZ1+cQFhLBGG8+wwz2+Duk6MpMRL57IelwMfuvh5Yb2YzgGOA\nSguQfPbFycC/A7j7IjNbDBxOWJ9WbZr8vZnGIazNCw/NbFfCwsOGXwCTgStg80r3RhceVoCd7gsz\nOxh4BLjc3RfGUGNUdrov3P2b7n6Iux9CmAf51woLD8jv38ck4Dtm1szM9iBMmL4dcZ1RyGdfzCec\nCZz68f7DgfcirTI5mvy9mboeiGvh4Wb57AvgV8C+wL31//Pe4O4946q5XPLcFxUvz38f881sKjAX\n2AiMdPeKC5A8/ybuAMaY2euE/1D/1N0/ja3oMjKzh4HTgDZmthS4jTCcWfD3phYSiohIQdI4hCUi\nIgmgABERkYIoQEREpCAKEBERKYgCRERECqIAERGRgihARESkIAoQEREpSOpWooukkZldA3QEvgE8\n5O7TYy5JpGhaiS5SZmZ2O7DS3f+fmbUBXnX3g+OuS6RY6oGIlJGZnQuc5+7fqn9qLdDezA5z93dj\nLE2kaJoDESkTM2sB3AMMznn6kPrb/aKvSKS0FCAi5XMxsA8wIee5U+tvP4u+HJHS0hyISJmY2RTg\nCCCb83QGOBDY293rYihLpGQ0ByJSBma2G+HaCwPdfVj9cwZ8ADyn8JBKoCEskfLoCLQEZuU8dwpw\nAPBALBWJlJgCRKQ89q+/XZzz3FXAEmB85NWIlIGGsETK48P6208BzGx/4ELgUg1fSaXQJLpIGZhZ\nM2Ap0NPdl5nZKGC5u98Wc2kiJaMAESkTMzsLuIRwyO4cdx8Vc0kiJaUAERGRgmgSXURECqIAERGR\ngihARESkIAoQEREpiAJEREQKogAREZGCKEBERKQgChARESmIAkRERAqiABERkYL8fwbTt3BE2f/e\nAAAAAElFTkSuQmCC\n", 121 | "text": [ 122 | "" 123 | ] 124 | } 125 | ], 126 | "prompt_number": 2 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "metadata": {}, 131 | "source": [ 132 | "Now, to calculate the \n", 133 | "\n", 134 | "$$ \\mathbb{P}_{FA} = \\mathbb{P}( G(\\mathbf{X}_n)= 5 \\vert H_0)$$\n", 135 | "\n", 136 | "or, in other words,\n", 137 | "\n", 138 | "$$ \\mathbb{P}_{FA}(\\theta) = \\mathbb{P}( \\theta^5 \\vert H_0)$$\n", 139 | "\n", 140 | "Notice that this is a function of $\\theta$, which means there are many false alarm probability values that correspond to this test. To be on the conservative side, we'll pick the maximum (or, supremum if there are limits involved) of this function, which is known as the *size* of the test, traditionally denoted by $\\alpha$.\n", 141 | "\n", 142 | "$$ \\alpha = \\sup_{\\theta \\in \\Theta_0} \\beta(\\theta) $$\n", 143 | "\n", 144 | "which in our case is\n", 145 | "\n", 146 | "$$ \\alpha = \\sup_{\\theta < \\frac{1}{2}} \\theta^5 = (\\frac{1}{2})^5 = 0.03125$$\n", 147 | "\n", 148 | "which is a nice, low value. Likewise, for the detection probability, \n", 149 | "\n", 150 | "$$ \\mathbb{P}_{D}(\\theta) = \\mathbb{P}( \\theta^5 \\vert H_1)$$\n", 151 | "\n", 152 | "which is again a function of the parameter $\\theta$. The problem with this test is that the $\\mathbb{P}_{D}$ is pretty low for most of the domain of $\\theta$. Useful values for $\\mathbb{P}_{D}$ are usually in the nineties, which only happens here when $\\theta \\gt 0.98$. Ideally, we want a test that is zero for the domain corresponding to $H_0$ (i.e. $\\Theta_0$) and equal to one otherwise. Unfortunately, even if we increase the length of the observed sequence, we cannot escape this effect with this test. You can try plotting $\\theta^n$ for larger and larger values of $n$ to see this." 153 | ] 154 | }, 155 | { 156 | "cell_type": "heading", 157 | "level": 3, 158 | "metadata": {}, 159 | "source": [ 160 | "Majority Vote Test" 161 | ] 162 | }, 163 | { 164 | "cell_type": "markdown", 165 | "metadata": {}, 166 | "source": [ 167 | "Due to the problems with the detection probability we uncovered in the last example, maybe we can think of another test that will have the performance we want. Suppose we reject $H_0$ if the majority (i.e. more than half) of the observations are heads. Then, using the same reasoning as above, we have\n", 168 | "\n", 169 | "$$ \\beta(\\theta) = \\sum_{k=3}^5 \\binom{5}{k} \\theta^k(1-\\theta)^{5-k} $$\n", 170 | "\n", 171 | "Using some tools from `sympy`, we can plot this out and compare it to the previous case as in the cell below.\n" 172 | ] 173 | }, 174 | { 175 | "cell_type": "code", 176 | "collapsed": false, 177 | "input": [ 178 | "from sympy.abc import p,k # get some variable symbols\n", 179 | "import sympy as S\n", 180 | "\n", 181 | "expr=S.Sum(S.binomial(5,k)*p**(k)*(1-p)**(5-k),(k,3,5)).doit()\n", 182 | "p0=S.plot(p**5,(p,0,1),xlabel='p',show=False,line_color='b')\n", 183 | "p1=S.plot(expr,(p,0,1),xlabel='p',show=False,line_color='r',legend=True,ylim=(0,1.5))\n", 184 | "p1.append(p0[0])\n", 185 | "p1.show()" 186 | ], 187 | "language": "python", 188 | "metadata": {}, 189 | "outputs": [ 190 | { 191 | "metadata": {}, 192 | "output_type": "display_data", 193 | "png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl0FFX2wPHvI4IIAQLiAmEJm2wujAoOihAUHQS3EUFR\nRoILiKLM+BvcQAhKQFCPjoMoIAQUBQUXcAAR2UEE2dco+2IA2UkI2e/vj5e0ScjSaTpdne77OadO\n0p3qqtvV6duvb716z4gISimlAksZpwNQSinlfZrclVIqAGlyV0qpAKTJXSmlApAmd6WUCkCa3JVS\nKgBd5MN9aZ9LpZQqPuPJg7TlrpRSAUiTu1JKBSBN7kopFYA0uSulVADS5K6UUgHIl71l3FatWjVO\nnjzpdBhKKeUzVatW5cSJE17bnvHhqJBu78gYg45WqZQKJoXkPe0KqZRSytLkrpRSAUiTu1JKBSBN\n7srvjR07ln/9619Oh1GoBx98kO+//97pMEqVo0eP0rRpU1JSUpwOpUDfffcdDz/8sNNheESTewnp\n1auXW/dPmjSJkJAQKlWq5FqWLl3q9XjKlClDaGioax+9e/d2O+bCDB06lH379hVrW6+//jplypRh\n4cKFue7ft28fQ4cOzXVfamoqMTExvPjii8WOzZsOHz7MvffeS3h4OGXKlGH//v25/v7SSy8xaNAg\nn+83JSWFxx9/nCpVqlCjRg3efffd87bhyeua32sBsGTJEiZPnpzrvh49elCjRg0qV65M/fr1iYmJ\ncSuGN998k169enHxxRcXOz5veu2117jmmmsoW7bsec/5nnvuYevWrWzevDnX/fkdB4DJkyezZMkS\n1+2jR4/SvXt3wsPDCQsLo02bNqxevbpknkgemty9rG/fvq434PHjx+nTpw/nzp3j6aefdnVz2r9/\nP3379nU95pZbbiEhIcG1tG3b1q19RUVF5fsPVpDNmze79jFu3DgADhw4wAsvvEBSUhIAW7ZscSuR\njhgxguXLlwOQnp5OTEwMq1at4sUXX2TLli0AJCUl8cILL3DgwAHX43bt2sWMGTOoWbOm675Vq1Yx\nfPhw0tPTAVi2bBkjRowAYObMmTRt2pQaNWq4/TyLIyIi4ryEmZ8yZcrQqVMnvvrqq3z/3rJlS86c\nOcPatWt9ut/o6Gh27drF/v37WbRoEaNGjWLevHkARb4W+SnotZg5cybjx493rffNN9+4/odeeeUV\n9uzZw5kzZ5g7dy7//e9/XTHk935ITk4mJSWFTz75hB49ehR5DDwRHR2d74dTfho1asRbb71F586d\nMeb8jindu3d3PdeCjsO4ceP49ttvARARxo0bx8yZM0lMTKRVq1asW7eOkydP0rNnTzp37szZs2e9\n8CyLICK+Wtxmw/JfdevWlREjRkizZs2katWq0qtXL0lOThYRkX379knPnj2lQYMG8tBDD8nWrVtF\nRGTLli3SrVs3adCggURFRcn+/ftFRCQ2NlbatGnjURxRUVEyadIkt9Y1xsjOnTvz/dvy5cvlgQce\nkLp160q/fv3k+PHjIiLSs2dP6dOnj9xxxx1SqVIladeunezbt09ERM6ePSuvvvqqNGnSRO644w6Z\nNWuWiIgcO3ZMnn32Walbt6506dJFVqxYkWtfHTt2lDlz5khERIQsWLDAdf/MmTOlQ4cO0qRJExk4\ncKAkJSWJiEivXr0kJiam0OfWrl07efnll6VVq1ZSuXJlue++++TEiRNuHZeIiAjXc3JHWlqaGGPy\nfcxTTz0lQ4cO9el+a9asKfPnz3fdHjx4sDz88MMiUvBrsWjRIgkPD5fhw4dL9erVJSIiQj777DPX\nNgp6LSZOnCitW7eWFi1ayNtvvy3p6ennxRkXFyfh4eGydu1aESn4/bBkyRJp2LBhoc85NjZWbr75\nZunXr59UqVJFmjRpkut/pjDR0dESHR3t1rrZevToke9jVqxYIfXq1XPdzu84pKWlyVtvvSUtWrSQ\n1q1bS2xsbIH7qVy5sqxbt+68+wvJex7lXG25e+jzzz/nhx9+YNeuXfz2228MGzYMsB+WZcqUQURy\n9VvNeTv772D7tq5fv57LLruMxo0bM2zYMDIyMtyOI7+WRkHatm1LjRo16NKlS65SSnZsQK7Ysp/n\n4MGDOXbsGC1atODRRx/N9TzzxpD3/pzbmj59OuXLl+euu+4q9LnkjGHLli00bty4yOf26aefEhsb\ny6FDh7jooot4/vnniz4gXta0aVM2btzos/2dPHmSQ4cOcd1117nuu/baa9m6dStQ+Gtx5MgRjh8/\nTnx8PJMnT6Z379789ttvrr/nfS1y/r8aY877P3nmmWeoWLEizZs3Z9CgQVx//fW5Ysj7fti8ebNb\nr+vq1atp2LAhx48fZ+jQoTzwwAM+v8CxSZMm7N27l8TExAKPQ85jnf0zPxs2bCA1NZWGDRuWeNyl\nN7kbc+GLx7s29OvXj/DwcKpWrcrAgQOZOnUqYOuI0dHRtG3bltGjR/P+++9z7tw5/vvf/zJmzBja\ntm3LkCFDXGWHtm3bsnXrVo4ePcpXX33F1KlTeeutt9yKI+ebrihLly5l3759xMXFUbNmTe6++24y\nMjLYv38/X3/9NZ9++int27fnySefZOTIka7H3X333bRp04Zy5coRExPDypUrOXjwIO+//z533nkn\nDz/8MGPGjGHjxo2sWrWKUaNG0bt3b9q3b8/kyZOZMWMGBw8eJCEhgYEDB/Kf//znvNhWrVrF5s2b\n+fDDD3n44Yfp0KGDa71Tp05RqVKlIl+Pxx57jGbNmlGhQgXeeOMNvvzyS7ePjbvrFSU0NJRTp065\nvf6F7jcxMRGAKlWquO6rXLkyCQkJAPm+FjnLMm+88QZly5albdu2dO7cmS+//LLA12LmzJmkpaUR\nExND//79iYiIYOLEia5tjRkzhsTERH788UcGDRrkqisX9H5w53UFuPzyy+nfvz8hISF069aNxo0b\nM3v27CIf563XFHDFeerUqQKPQ2xsLPXq1eOf//wnw4YNIyUlxVWmyXbmzBn+8Y9/EB0d7dZzv2Ce\nNvk9WNyGn5dlIiIiZM6cOa7bW7ZskUsuuSTXOlFRUfk+tqD7s02bNk1uuOGGAv9+zTXXSFhYmISF\nhUm5cuWkQoUKrtvPPvusW/FnZGRIaGiobNmypdDYoqKiZMCAAbnuu+yyy2T16tWu29HR0bJ3797z\n9pF3Wy+88IK8/vrrrtsRERHy448/5lpn7969530tbtWqlUyfPt11u0+fPhIaGiqhoaEyYsQIERGJ\njIyUMWPGuNZJTEwUY4z88ccf58W1b98+1/EKCwuTMmXKSOXKlV23p06det5jciqsLPPOO+/IAw88\nkO/jSmK/J06cEGOMHD161HXf9OnT5Zprrsn12LyvxaJFi+Syyy7Ldd+AAQOkb9++rtv5vRYiIosX\nLy6yFPj000/LP//5z0JjGDNmjHTu3Nl1e+nSpa7X9eqrrxYRW5Zp2bJlrsd17dpVRo4cme9+O3fu\n7Dqe5cuXl/Lly7tu33PPPYXGLFJwWeb48eNijJGEhATXfQUdh0mTJsnixYvPuz8pKUnatm0rvXv3\nLnD/heQ9j3KuJncPREREyEcffeS6PWfOHGnQoIFXtj1t2jS5/vrr3Vo3KipKJk+eXOx9pKenS2ho\nqGzevLnQ9Xr27Omq34qIJCQkSEhIiBw8eLDY+2zRooVUr15drrzySrnyyislJCREqlWrJqNGjSr0\ncU8++WSRNffIyEh5+eWXXbe3bdsm5cqVk8zMzCLj8mbN/cknn8z1AeaL/eatuQ8aNEi6d+9e6LYW\nLVokF110kZw9e9Z1X7du3WTYsGFux1OYJ554QgYOHFjoOsuXL5dGjRoVuk5sbKzUrFkz132tWrWS\nKVOmFBlDdHS02+c/svXo0SPfxyxfvjxXzb24kpOT5c4775QePXoUup63k3vpLcs4SEQYM2YMv//+\nOydOnCAmJsbjvrBz587lyJEjAMTFxTFs2DDuv//+YsVSlG3btrFhwwYyMjJITEzkhRdeoFatWjRt\n2rTIx86ZM4cVK1aQmprKa6+9RuvWrQkPD3c7vmwLFixg69atbNy4kQ0bNlCzZk3GjRvHM888U+jj\nOnXqlKtrWX5EhClTprB9+3aSkpIYPHgwXbt2Ldb5CHckJyeTnJx83u/Zli5dWuD5hJLa72OPPcaw\nYcM4deoU27dv5+OPPyYqKsqt7Q4ZMoS0tDSWLVvG7Nmz6dq1a7FjO3r0KNOmTePs2bNkZGQwb948\npk+fzn333Vfo41q2bMmpU6eIj48vdL0//viD999/n7S0NKZPn86vv/5Kp06diowrO8G5Iz09neTk\nZDIyMkhLSyM5OZnMzEzX35csWeLWPvOTlpbGgw8+SIUKFZg0aZJH2/CYp58KHixuoxS03N98801p\n1qyZhIWFSVRUlJw7d86jbf373/+WK664QipWrCj169eXIUOG5NsLIT/uttwXLlwojRs3looVK8rl\nl18uf//73wvsOZN3+08//bTccccdEhoaKu3atcu3BOOJvL1lCpKamip16tSR+Pj4AteJjIyUV155\nxdVb5t5773X1+HEnDndb0MYYMcZImTJlXD+zrV69utByWkntNyUlRR5//HGpXLmyXHHFFfLuu+8W\nub1FixZJrVq1JCYmRqpXry5169Z1qzWcn6NHj0q7du0kLCxMqlSpIi1btpSZM2e69dgBAwYUWGIR\nsS33W265xdVbpnHjxrm+pRSmOC33nj17uo5x9pLzfXXNNdfIpk2b3NpWXosXLxZjjFSsWNFVdgoN\nDZXly5eft24heU/LMr7ibmIq7aKiomTQoEFOhyHjxo07r4abU2RkpEyYMMGHEZ2vS5cuMnfuXEdj\ncFd2cnfa0aNHpUmTJq5uxHldSDdhb5k1a5Y89NBDPtmXt5O7X47nrvyDuPm1tqQ99dRTRa7jdKwz\nZsxwdP+lUfXq1dm+fbvTYRTqnnvu4Z577nE6DI9ozV0VKLsvb2lQWuL0F6XheJWm/z9/pJN1KKWU\nH9DJOpRSShVJk7tSSgUgTe5KKRWANLkrpVQA0uSulFIBSJO7UkoFIE3uXuZv0+sppYKTJncv8eX0\nekopVRRN7sUUERHBm2++SfPmzalWrRqPP/44KSkpvPLKKwwePJhly5bx7LPP0r9/fy655BKee+45\n+vbty7JlyxgyZAivvvqqa1t6oZZSqqRocvdAflPsifh2ej2llCqUpyOOebB4Y3S0HOtc+OKJiIgI\nGTt2rOt29kQdffv2lX379klUVJQcO3ZMevfuLUlJSdK7d285fvy4REVFyb59++Tpp58WEZHdu3e7\nhs/dvHmzNGvWzDWzkFIq+BSS9zzKuaW25e6N9O6p2rVru36vU6cO8fHxjBkzhjp16gBw6aWXMnbs\nWC655BLGjh1LtWrVXOt++OGHANSrV4+6desCcPXVVzN48GAdWVAp5TWlNrk7KfvEafbvNWvWdN2O\njY3N9zEF3Z+TaA1eKeUlmtyLScQ7U+xd6PR6SilVGE3uxWSM4ZFHHuHOO++kQYMGNGrUiEGDBhV7\nOwsXLuS6664jNDSUzp0706VLl1w9aZRS6kLoeO7FVK9ePSZMmMBtt93mdChKqQCi47krpZQqkiZ3\npZQKQFqWUUopP6BlGaWUUkXS5K6UUgFIk7tSSgWgi5wOID9Vq1bFGI/KTEopVSpVrVrVq9vzyxOq\nSimlXPSEqlJKKUuTu1JKBSBN7kopFYA0uSulVADS5K6UUgFIk7tSSgUgTe5KKRWANLkrpVQA0uSu\nlFIBSJO7UkoFIE3uSikVgDS5K6VUANLkrpRSAUiTu1JKBSBN7kopFYA0uSulVADS5K6UUgFIk7tS\nSgUgTe5KKRWANLkrpVQA0uSulFIBqMjkboyZaIw5YozZXMg67xtjdhhjNhpj/uLdEJVSShWXOy33\nWKBjQX80xnQCGopII6A38KGXYlNKKeWhIpO7iCwDThayyr3A5Kx1VwFhxpgrvBOeUkopT1zkhW2E\nAwdy3D4I1AKOeGHbSinlv0QgKQnOnLFLQgKcO/fnkplp70tLg9RUCAmBxETIyLBLxYpw+rTdjohd\nP4fTlzeiyjOPehSaN5I7gMlzW/KuEBUVRUREhOt2ZGQkkZGRXtq9Ukp5SXIy/P47xMf/+TM5GX79\nFY4dg+PH7c8GDWDBAihXDipXtkv16lCmDFxyiV0aNLDrly1r17v0Ujh71ib5kBC77rlzYIxdwN4P\nHD1bgWYvdeHoM549DW8k99+B2jlu18q6L5fJkycjcl7OV0op30tMhLg42L4dfvsN9u6F3bvt0qSJ\nvV2zJoSH258NG0L79jZ5V69uk3TVqlClik3cJWD0EHjgEc8f743kPgvoB0wzxvwVOCUiWpJRSjkv\nMxN27YL16+1y+LBtbR87BlddZRN506bQoQPUqwf169tkXsbZXuKJiTBmDPz0k+fbMEW1po0xU4F2\nQHVsHX0IUBZARMZmrTMa26PmLNBLRNblsx3RlrtSqkT98QesXGmX+Hj49luoVg1atIC//MUuV18N\ndeu6yh/+6N137VP48kvg/LK3W4pM7t6iyV0p5XXx8bBwIfz8M8ybZ1vkN90EN99sl+uvt8m9FElN\ntaX6b7+FG24APEzu3jqhqpRSJS8pyZZVFi6E77+3LfX27e3Spw80b+54SeVCff65rRZlJXaPactd\nKeXfDhyA2bPhu+9g6VK48UZ44AG45RZbbinlyTynzEz7+TR6NNx+u+tubbkrpQLEnj224LxqlU3o\nd90Fjz0Gn30GYWFOR1diZs2C0FC47bYL35a23JVS/uHAAZvQv/jCJvcuXaBrV2jXDi4K/HaoCLRu\nDf/+Nzz4YK4/actdKVXKnDsH33wDEybY8krdujBsmG26BkFCz2npUjhxAv7+d+9sL7iOnlLKeSKw\nbh1MnAjTpkHLlvD003DvvXDxxU5H55g334QXX/ReD01N7kop30hIgBkz4D//seOpPP64vbCoTh2n\nI3Pcxo2waZPt/ugtmtyVUiVrxw7b/ePTT+2J0XfesV0XA6iXy4UaORL++U/vfnHR5K6U8j4R2xf9\nnXdgzRp48knbPK1du+jHBpmdO+HQIfjoI+9uV5O7Usp70tNt6WXUKEhJsUXkr76yIySqfMXE2A5B\nlSt7d7vaFVIpdeGSk2HqVHjjDTuS4ksvQadOWnopws6dtvvjjh2Fdt/3qCukHnmllOfOnYP337eD\noaxcCVOmwLJlcPfdmtjd8MYb8NxzJXNdlpZllFLFl5QE48fbM4EtW9pLKy90MJQg8+uvMGeObb2X\nBE3uSin3paVBbCy89ZZN5rNn22F0VbG98Qb072/n+ygJWnNXShUtM9OeGB00CGrVghEjoFUrp6Mq\nteLioG1b22p340SqDj+glCoBixbBgAG2e+Po0XbWIuNRvlFZXn/d9mv3dg+ZnLTlrpTKX1ycHcUq\nNdX2U3/wQT1J6gXbttlruHbuhEqV3HqI9pZRSnnBiRO2WXnrrRAZacdR79ZNE7uXvP46vPCC24nd\nY/pqKaWs9HT4+GM7YXRqqm1i/vvfQT2Yl7dt3QqLF8Ozz5b8vrTmrpSCn36CZ56xiX3BAjuJtPK6\noUPh//7PTshR0opsuRtjOhpj4owxO4wxL+Xz9+rGmO+NMRuMMVuMMVElEqlSyvuOHrWjM3brBi+/\nbCfw1MReIjZsgFOn7GeoLxSa3I0xIcBooCPQDOhujGmaZ7V+wHoRaQFEAu8YY/QbgVL+LD0dPvzQ\nTthZtaotwTz8sPaCKUGvvgr33AMVK/pmf0Ul4VbAThHZC2CMmQbcB2zPsc4h4Nqs3ysDx0Uk3ctx\nKqW8ZdMmOzlGeLgtwVxzjdMRBbxFi+wVqd4cr70oRZVlwoEDOW4fzLovp/FAc2NMPLAR6O+98JRS\nXpOcDAMH2n7qvXrZuUo1sZc4ETs4ZkwMlCvnu/0Wldzd6Zj+KrBBRGoCLYAPjDEl3MlHKVUsS5bA\ntdfa5uPGjfDUU9q10UemT7cJvls33+63qLLM70DO0fVrY1vvOd0MxACIyC5jzB6gMbAm78aio6Nd\nv0dGRhIZGVnsgJVSxXDmjB0HZtIke3Xpffc5HVFQSUuztfaxY33/WVpUcl8DNDLGRADxwENA9zzr\nxAEdgBXGmCuwiX13fhvLmdyVUiVs/nx7ZWmnTrBlS8mNUKUKNH68HQ359tt9v+8ihx8wxtwFvAeE\nABNEZIQxpg+AiIw1xlQHYoE62DLPCBH5PJ/t6PADSvlCQoK9+Oj77212ufNOpyMKSgkJcNVVdljf\nCxw406MuTDq2jFKBZPFiW0+PjIS339bWuoOGDrUzLE2ZcsGb0uSuVNBKTrbF3S+/tPX1Dh2cjiio\nHTkCzZrZucHr1bvgzenAYUoFpfXr7cQZBw/anjCa2B03eLCdPs8Lid1jeiWpUqVVRgaMGgXvvmuX\nRx7RK0z9wIYN9mKluDhn49DkrlRpdOAA9Ohhu2KsWQN16jgdkcL2Z+/f3w7rW7Wqs7FoWUap0ubr\nr+HGG6FjR9sbRhO735gxww4O9uSTTkeiLXelSo+kJDuJxoIFMGsW3HST0xGpHM6ds7MRTpoEISFO\nR6Mtd6VKh40b7UnTpCR7AlUTu995+21o2dL2QvUH2hVSKX8mAh98AFOn2pEc//EPpyNS+ThwAFq0\ngLVrISLC65vXfu5KBZTTp+GJJ2DXLjuC41VXOR2RKsCjj0L9+vDGGyWyee3nrlTAWLMGrr8errwS\nVq7UxO7HVqyApUvtRFb+RJO7Uv4kuwzTqROMHGlHcixf3umoVAEyMuzLNWqU72ZYcpf2llHKX5w5\nY8swmZm2td6ggdMRqSJ88AEcOmRnKPQ3WnNXyh9s3gxdutixYd99V1vrpcDBg3a0x2XLoEmTEt2V\n1tyVKpUmT4bbboPXXrOTVmtiLxX694dnninxxO4xLcso5ZSUFBg0yI67vmgRXH210xEpN333nf2y\n9dlnTkdSME3uSjnhwAFbhqlTx3a3qFzZ6YiUmxIToV8/iI317y9ZWpZRytcWLIBWraBrVzt7sib2\nUiU6Gtq1s5U0f6Ytd6V8RcT2mXvvPft93t+zgzrPhg3wySd2Slp/p8ldKV9ISLDf5ePiYPVqqF3b\n6YhUMWVkQJ8+MGIEXH6509EUTcsySpW0336zA31deaW9lFETe6n00Udw8cXQq5fTkbhHk7tSJWn2\nbGjTxg7VO3KkzQ6q1Dl4ECZOtAm+TCnJmkWGaYzpaIyJM8bsMMa8VMA6kcaY9caYLcaYxV6PUqnS\nJjPTjiLVp4+dc613b6cjUh4SsZNv3H+/nfS6tCi05m6MCQFGAx2A34FfjDGzRGR7jnXCgA+Av4nI\nQWNM9ZIMWCm/l5AAr7xix11fvRpq1nQ6InUBxo2D48f9b2CwohTVcm8F7BSRvSKSBkwD7suzziPA\nVyJyEEBEjnk/TKVKiV27oHVre4HSwoWa2Eu53bvtdWaffAJlyzodTfEUldzDgQM5bh/Mui+nRkA1\nY8wiY8waY4zOJqCC048/ws0322vSx43T+nopl5EBUVH2S1jTpk5HU3xFdYV0Z6SvssD1wO1ABWCl\nMeZnEdlxocEpVSqI2KF5hw+3k2r4yzxr6oK895792b+/s3F4qqjk/juQs99WbWzrPacDwDEROQec\nM8YsBa4Dzkvu0dHRrt8jIyOJ1DeBKu1SUuz0dydPws8/Q926TkekvGDbNtufffVq/5js2hOFDvlr\njLkI+BXbKo8HVgPd85xQbYI96fo34GJgFfCQiGzLsy0d8lcFlsOH4YEHbF190iQIDXU6IuUFaWn2\ntMlTT9nOTn7A+0P+ikg60A+YB2wDvhCR7caYPsaYPlnrxAHfA5uwiX183sSuVMBZu9aOD/O3v8GX\nX2piDyAjRkD16qW/96pO1qFUcX39tW3SjR1rW+4qYKxdC3fdZXuxhuftOuIcj1ruOraMUu7KzIQh\nQ2xLff58aNHC6YiUFyUnQ8+ediIsP0rsHtPkrpQ7EhPhscfg6FE7r1ppGDlKFcvw4XDLLfDII05H\n4h2a3JUqyr59cO+9cMMNMHWq9l8PQLNm2dkO168H41ERxP+UkiFwlHLIihW260RUFEyYoIk9AO3b\nZ3vGTJ0K1ao5HY33aMtdqYJMmmQvSpo4ETp2dDoaVQLS0uDhh+H//s9eXBxItLeMUnllZNhRor75\nxs6EXBqvPVduefFFO6vS//7n10P5am8ZpS7YmTPw6KNw9iysWgWXXup0RKqEzJ4N06bBunV+ndg9\nFoBPSSkP7dljv5vXqgXz5mliD2AHDsATT8Dnn9sLlgKRJnelwJ44vflmO07MmDGlb3xX5ba0NOje\n3Q4I1qaN09GUHC3LKDVxoq2xT50Kt9/udDSqhL39th0t4qV855ULHJrcVfDKyLBn1GbNshNXN2ni\ndESqhE2daofa/+WXwKyz56TJXQWnM2fsd/Nz5+yJ00Dq4KzytXYtPP88LFgQuHX2nAL8s0upfOze\nbevrderYE6ea2APe4cPw97/bsd6uvdbpaHxDk7sKLkuXwuOP64nTIJKSYgfvfOKJ4BrEUy9iUsFj\nwgQ7Ieann9px2FXAE7FJ/cwZO5hnKa2z60VMSuUrPd2eOP3f/+yIjo0bOx2R8pH337e19hUrSm1i\n95gmdxXYTp+2g4ekp9sTp1WrOh2R8pH58+HNN2HlyuCcKCvIPstUUNm5E/76V2jYEObM0cQeRHbs\ngB497LhvERFOR+MMTe4qMC1damde6N8f/vtfPXEaRE6csBcovf46tG3rdDTO0eSuAs+YMbYU88UX\ntleMChrnztl5VerVs9PcBjPtLaMCR2qqbakvXWqvOm3QwOmIlA9lZMCDD0KFCrZDVACdQPWot0yR\nT98Y09EYE2eM2WGMKXA0BmNMS2NMujEmiHqSKr9x9CjceSf8/rs9g6aJPaiIQL9+kJAAsbEBldg9\nVughMMaEAKOBjkAzoLsx5ryZC7LWGwl8j4efMkp5bNMmaNXKXnX6zTdQubLTESkfi4mBn3+Gr7+G\ncuWcjsYsHoFJAAASSklEQVQ/FPX51grYKSJ7RSQNmAbcl896zwEzgKNejk+pwn37rR3JMSbGTl8f\nEuJ0RMrHJkywA3vOnauf6zkV1c89HDiQ4/ZB4KacKxhjwrEJ/zagJaCFdVXyMjPhnXdg/HjbzbFl\nS6cjUg74/nsYNAiWLIErr3Q6Gv9SVHJ3J1G/B7wsImKMMRRSlomOjnb9HhkZSWRkpBubVyqPxETo\n2RPi42HxYqhZ0+mIlAN+/BEee8y22K+6yulo/E+hvWWMMX8FokWkY9btV4BMERmZY53d/JnQqwNJ\nwFMiMivPtrS3jLpwu3fDfffBTTfBBx/AxRc7HZFywJIl0LUrfPUV3Hqr09GUuBLpLbMGaGSMiTDG\nlAMeAnIlbRGpLyL1RKQetu7eN29iV8orfvwRWre2HZjHj9fEHqR++sl2eZw2LSgSu8cKLcuISLox\nph8wDwgBJojIdmNMn6y/j/VBjCrYicB778GoUfbCJC3nBa1ffoH777f92G+7zelo/JtexKT8W1IS\n9O5tT6AOHx68A4UoNmywIzWPG2crc0GkZC5iUsoxe/bY8WGMgY8/1sQexNats71iRo8OusTuMU3u\nyj/98IMd0bFXL/jkE3tNuQpKK1ZAx4520o2uXZ2OpvTQ8dyVfxGxtfX//MdOndOundMRKQfNnw+P\nPAKffWZHl1Du0+Su/MeZMzB4sO0OsWoV1K7tdETKQTNnwlNP2REl2rRxOprSR8syyj9s3WrHh0lK\nsqM6amIPap99ZkdrnjtXE7unNLkr502bZrs3vvyy7QpRvrzTESkHjR1rJ9tYsABuuMHpaEovLcso\n56SmwoABduLq+fOhRQunI1IOErHjvy1fbq9A1VGbL4wmd+WMgwfhoYfg0kvt9PRhYU5HpByUmmov\nPN68Gb77DmrUcDqi0k/LMsr35s2zozh2726H7NXEHtROnYK77rJzny5ZoondWzS5K99JT7dXojzx\nhK2z9+unU+YEub177XVq11xjJ9qoWNHpiAKHlmWUbxw6ZFvqZcvaMswVVzgdkXJY9jgxL70Ezz/v\ndDSBR5tNquRld3u47TY7u4Im9qA3bRq88gqMGaOJvaRoy12VnLQ0iI62c5xOmaLD+CnS021L/Ztv\nbBlGO0iVHE3uqmTs2WOvGw8Lg8mT4fLLnY5IOeyPP2wHqYsvhjVroFo1pyMKbFqWUd73xRf2atOu\nXWH2bE3sitWr4cYb7cnT2bM1sfuCttyV95w9awuoS5fa2rpeXhj0RCA29s+Lj++/3+mIgocmd+Ud\na9fCs89CkyZ28O1KlZyOSDns1Cno2xeSk2HZMmjc2OmIgouWZdSFSU+HYcPsVSgDBsCkSZrYFT/9\nBH/5iy2/fP65JnYnaMtdeW7HDnjsMXvlybp1UKuW0xEph2Vk2PFhPvggKKfD8yvaclfFJ2KH7rv5\nZnth0g8/aGJX7N8P7dvbIQTWrdPE7jRtuaviOXQInnwSjhyxJ06bNnU6IuWw7JOmn3wCnTrZ6lxI\niNNRKbda7saYjsaYOGPMDmPMS/n8/VFjzEZjzCZjzApjzLXeD1U5SgS++soWUm+8EVau1MSu2L/f\nzm86erSdGfHllzWx+4sik7sxJgQYDXQEmgHdjTF539W7gbYici3wBjDO24EqBx06BF26wGuv2fFY\nhw61Y8SooJWZCR99BNdfD23b2lkRr7vO6ahUTu603FsBO0Vkr4ikAdOAXNU0EVkpIqezbq4CtAAb\nCERs75frroNmzWD9ejtUrwpqv/0GvXrBxIm2vj5woH7W+yN3au7hwIEctw8CNxWy/hPAnAsJSvmB\n/fuhd29bW583z5ZjVFA7e9b2hBk3Dl5/HSZMgIv0rJ3fcuelEXc3ZoxpDzwO3JLf36Ojo12/R0ZG\nEhkZ6e6mla9kZtqzYy+9BP/6F7z4ojbLgpyIHejrX/+ywwds2gQ1azodlSqKO8n9dyDnVPS1sa33\nXLJOoo4HOorIyfw2lDO5Kz+0YYOdcr5mTdsTplkzpyNSDtuxA557zn6RmzTJdnVUpYM7yX0N0MgY\nEwHEAw8B3XOuYIypA3wN9BCRnV6OUZW0hAQYPNheSjh8uC2o6gxJQe3ECVuC2b4dbr8d+veHcuWc\njkoVR5HvYBFJB/oB84BtwBcist0Y08cY0ydrtcFAVeBDY8x6Y8zqEotYeY8ITJ9uuzSePg1bt9op\n8DSxB63kZHjnHTtcwNmz9qTpgAGa2EsjI+J2Sf3CdmSM+Gpfyg2//Wa7OWzfbvu0tWnjdETKQRkZ\ndvKMF1+085mOHKmXMfgR48mDtIkWbE6ehBdesEMH3Hab7d6oiT1oZWbCjBlw7bW2ph4bC7NmaWIP\nBNqRKVikp9s+bEOH2kG1t27VuUyDWGYmzJkDgwbZzlBvv22vNDUetRGVP9LkHgzmz7f92C67zA7y\npZcSBq20NJg61ZZdLr/cftbfe68m9UCkyT2Qbdlip5f/4QfbNLvvPn0XB6mkJHvR0dtvQ4MG8N57\n0KGD/jsEMk3ugWjvXhgyxE51N3AgvPuunZVYBZ34ePjwQ/jlF7jkEvjyS7ipsOvLVcDQE6qB5MgR\n2yH5hhsgIsJegfL885rYg9CqVfDII3D11fYc+nvv2atMNbEHD225B4IjR+z37V9+sfX07dttQVUF\nlYQEW0+fORPi4qBfP1uVCwtzOjLlBE3upVl8PIwaZWdJ6NEDpkzRGZGCjAisXg3jx9vh9tu3t0n9\nzjt1XPVgp8m9NNq/3yb1zz+HqCjbrbFGDaejUj508KBtpa9ZA2vX2smxtm+HK690OjLlLzS5lyYb\nNthrwzdutJ2S4+K0/BJETp2ydfMpU+y1Z1262Fb6LbfoiBHqfDr8gL8TseOpv/OObZo9/7wdZ10L\nqUHh1Cl7xej06bb8cuutdk7yzp2hfHmno1M+4lGHVU3u/ur0aVtL//hjqFIFnnoKHnpIR3AKAvHx\n8L//2WQ+fbodJeLBB+Huu+2/ggo6mtwDwqZNtovDF1/A3/4Gzzxjm2t6tUnAysiwdfNly2DaNNi1\ny1bduna1FxpVquR0hMphHr35tebuD06etO/qSZPsd+0OHfTsWID7/Xd74fC8efDjj/al7t7dnidv\n00Ynv1IXTlvuTsnIsO/q2FiYO9e20nv1gjvu0IkpA9Dhw3Yy6cWL7VKtmu21+re/2W6L2oNVFULL\nMn4vIwOWL7eF1F9+sbd79bJNtmrVnI5OeYkI/PorrFgBP/0E+/bZssutt0JkpO2Lfu212g9duU2T\nu1/KyLDF1OnT7WwIV1wB3brZM2RXXeV0dMoL4uNtf/M1a+wcKPPn2xOfN99suym2bm0nwNBkrjyk\nyd1vnDxpC6pz59qhAQ4ftmfHunaFRo2cjk55KDPTnuzctMku8fF2TPSUFLjxRrvccIMdv6VmTaej\nVQFEk7tj0tPthUXff2/f7Zs3Q9u2cNdddqlf3+kIVTFkZMCePfYase3b7RIXZxN69eq2pHLdddCy\npf29bl3tzKRKlCZ3n0lJsTXzpUttyeWnn+x37yZNbDJv106vMPFzGRn2Ev5du/5cTp+2dfKdO+2F\nv02a2OnmmjSB5s1taUX7mSsHlExyN8Z0BN4DQoCPRWRkPuu8D9wFJAFRIrI+n3VKZ3IXse/8tWvt\ncuSIHaGpaVN7hqxtW9t3rXp1pyNVOSQl2e6Ghw/D7t12OJ59++xSsaL9klW9up24Inu56ir7s3Fj\nu45SfsL7yd0YEwL8CnQAfgd+AbqLyPYc63QC+olIJ2PMTcB/ROSv+WzL/5P7yZN2EK6tW2HbNjh7\n1ibySpVsMfXGG20L/cYboXJlj3ezePFiIiMjvRd3KVacY5GaCseOwdGjcOKErXkfPvznaY3Dh+2w\nt3FxcO6crXtHRtqp5erUseWTOnXsUPd169rJK/yJ/l/8SY/Fn4wx7UVkcXEfV1SH6lbAThHZm7WT\nacB9wPYc69wLTAYQkVXGmDBjzBUicqS4wZS4jAw4dMjOVLR7t1327LE/K1Sw5ZVmzex38ObNoUUL\nGD7c64NzBes/roj9vDxzxibh06dhwoTFHDsWSUqKTc4nT9rxVFJTbWv7+HGbyBs2hIUL4dJL7VSw\n119vk/aVV9qleXP7s0YNm9QvvbT01cGD9f8iP3oscokEFhf3QUUl93DgQI7bB4G8c7nkt04toOSS\nu4h99yck2CUx0S7HjtlMcPy4rYvv2mWTefZy7Jhtyp09C/Xq2ROd7drZvub160Pt2qUvI7hJxH62\npafbpJj9M+eSmnr+kpLy55KaalvEycm5f1asCAcO2FJI9iJiW9iJifZwN2sGixbZUxGVKtkvQAcP\n2pfk7FnbiSgtzY6H1rix/TytXNl2/7/0UvszLExHP1TKXUUld3frKHkzYv6Pu/vuP3+vUcMWRTMz\nbSbIzLTTwSUm5s5A9evDpk384+AITqWH2nXT06jGSU5cdJm9mjPkIqhShTohhv2Z10LZclD+Yrio\nLOE1hYMVwqD5xXb7pgzh9e2uWZ+1YL+m792bO9yyZW1Cy6l2bduizFlhyn5s3vv27Mk6GGKX0FDb\nat2zx7ZCs+8XsU9z587ch6NOHbtuZuafS61adl8ZGfZ29s/mze0wsBkZuZfmzWHdOvt7SIg9XHXq\nwB9/2OeXvbRoYfdfrtyfy+WX20R9cdahCw21jy9f3pY0ypeHqlVt7bpGDfvlp0IF+7eKFe0SGvrn\nz9DQ8/t6R0fbRSnlXUXV3P8KRItIx6zbrwCZOU+qGmM+AhaLyLSs23FAu7xlGWPMTqCB95+CUkoF\ntMkiElXcBxXVcl8DNDLGRADxwENA9zzrzAL6AdOyPgxO5VdvF5GGxQ1OKaWUZwpN7iKSbozpB8zD\ndoWcICLbjTF9sv4+VkTmGGM6ZbXMzwK9SjxqpZRShfLZRUxKKaV8x+t9D4wxHY0xccaYHcaYlwpY\n5/2sv280xvzF2zH4i6KOhTHm0axjsMkYs8IYc60TcfqCO/8XWeu1NMakG2Me8GV8vuLm+yPSGLPe\nGLPFGLPYxyH6jBvvj+rGmO+NMRuyjkWUA2H6hDFmojHmiDFmcyHrFC9viojXFmzpZicQAZQFNgBN\n86zTCZiT9ftNwM/ejMFfFjePRWugStbvHYP5WORYbyHwP6CL03E79D8RBmwFamXdru503A4ei2hg\nRPZxAI4DFzkdewkdj1uBvwCbC/h7sfOmt1vuroueRCQNyL7oKadcFz0BYcaYK7wchz8o8liIyEoR\nOZ11cxX2+oBA5M7/BcBzwAzgqC+D8yF3jsMjwFcichBARI75OEZfcedYHAKyLwWvDBwXkXQfxugz\nIrIMOFnIKsXOm95O7vld0BTuxjqBmNTcORY5PQHMKdGInFPksTDGhGPf3B9m3RWIJ4Pc+Z9oBFQz\nxiwyxqwxxvzDZ9H5ljvHYjzQ3BgTD2wE+vsoNn9U7Lzp7fncvHvRU+nm9nMyxrQHHgduKblwHOXO\nsXgPeFlExBhj8HCwJD/nznEoC1wP3A5UAFYaY34WkR0lGpnvuXMsXgU2iEikMaYBMN8Yc52IJJRw\nbP6qWHnT28n9d6B2jtu1sZ8wha1TK+u+QOPOsSDrJOp4oKOIFPa1rDRz51jcgL1WAmx99S5jTJqI\nzPJNiD7hznE4ABwTkXPAOWPMUuA6INCSuzvH4mYgBkBEdhlj9gCNsdffBJti501vl2VcFz0ZY8ph\nL3rK++acBTwGritg873oKQAUeSyMMXWAr4EeIrLTgRh9pchjISL1RaSeiNTD1t37BlhiB/feHzOB\nNsaYEGNMBezJs20+jtMX3DkWcdgRacmqLzcGdvs0Sv9R7Lzp1Za76EVPLu4cC2AwUBX4MKvFmiYi\nrZyKuaS4eSwCnpvvjzhjzPfAJiATGC8iAZfc3fyfGA7EGmM2YhuiL4rICceCLkHGmKlAO6C6MeYA\nMARbovM4b+pFTEopFYB0AFWllApAmtyVUioAaXJXSqkApMldKaUCkCZ3pZQKQJrclVIqAGlyV0qp\nAKTJXSmlApAmd6WU8jNZwzLEGWOmGGO2GWOmG2MuKc42NLkrpZR/ugr4QESaAWeAZ4rzYE3uSinl\nnw6IyMqs36cAbYrzYE3uSinln3IO/GUo5rwXmtyVUso/1cka3hfs9IvLivNgTe5KKeWffgWeNcZs\nA6rw5xSUbvH2TExKKaW8I11EPJ5DV1vuSinlny5osg2drEMppQKQttyVUioAaXJXSqkApMldKaUC\nkCZ3pZQKQJrclVIqAGlyV0qpAPT/CHA2cT3m8f4AAAAASUVORK5CYII=\n", 194 | "text": [ 195 | "" 196 | ] 197 | } 198 | ], 199 | "prompt_number": 3 200 | }, 201 | { 202 | "cell_type": "markdown", 203 | "metadata": {}, 204 | "source": [ 205 | "In this case, the new test has *size*\n", 206 | "\n", 207 | "$$ \\alpha = \\sup_{\\theta < \\frac{1}{2}} \\theta^{5} + 5 \\theta^{4} \\left(- \\theta + 1\\right) + 10 \\theta^{3} \\left(- \\theta + 1\\right)^{2}\n", 208 | " = (\\frac{1}{2})^5 = 0.5 $$ \n", 209 | " \n", 210 | "which is a pretty bad false alarm probability. Values less that one percent are usually considered acceptable. As before we only get to upwards of 90% for detection probability only when the underlying parameter $\\theta > 0.75$. As before, this is not so good. Let's run some simulations to see how this plays out." 211 | ] 212 | }, 213 | { 214 | "cell_type": "heading", 215 | "level": 3, 216 | "metadata": {}, 217 | "source": [ 218 | "All-heads Test" 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "collapsed": false, 224 | "input": [ 225 | "# implement 1st test where all five must be heads to declare underlying parameter > 0.5\n", 226 | "from scipy import stats\n", 227 | "b = stats.bernoulli(0.8) # true parameter is 0.8 (i.e. hypothesis H_1)\n", 228 | "samples = b.rvs(1000).reshape(-1,5) # -1 means let numpy figure out the other dimension i.e. (200,5)\n", 229 | "print 'prob of detection = %0.3f'%mean(samples.sum(axis=1)==5) # approx 0.8**3" 230 | ], 231 | "language": "python", 232 | "metadata": {}, 233 | "outputs": [ 234 | { 235 | "output_type": "stream", 236 | "stream": "stdout", 237 | "text": [ 238 | "prob of detection = 0.405\n" 239 | ] 240 | } 241 | ], 242 | "prompt_number": 4 243 | }, 244 | { 245 | "cell_type": "code", 246 | "collapsed": false, 247 | "input": [ 248 | "# here's the false alarm case\n", 249 | "b = stats.bernoulli(0.3) # true parameter is 0.3 (i.e. hypothesis H_0)\n", 250 | "samples = b.rvs(1000).reshape(-1,5)\n", 251 | "print 'prob of false alarm = %0.3f'%mean(samples.sum(axis=1)==5)" 252 | ], 253 | "language": "python", 254 | "metadata": {}, 255 | "outputs": [ 256 | { 257 | "output_type": "stream", 258 | "stream": "stdout", 259 | "text": [ 260 | "prob of false alarm = 0.000\n" 261 | ] 262 | } 263 | ], 264 | "prompt_number": 5 265 | }, 266 | { 267 | "cell_type": "markdown", 268 | "metadata": {}, 269 | "source": [ 270 | "The above two cells shows that the false alarm probability is great, but the detection probability is poor. Let's try the same simulation for the majority vote test." 271 | ] 272 | }, 273 | { 274 | "cell_type": "heading", 275 | "level": 3, 276 | "metadata": {}, 277 | "source": [ 278 | "Majority Vote Test" 279 | ] 280 | }, 281 | { 282 | "cell_type": "code", 283 | "collapsed": false, 284 | "input": [ 285 | "# implement majority vote test where three of five must be heads to declare underlying parameter > 0.5\n", 286 | "b = stats.bernoulli(0.8) # true parameter is 0.8 (i.e. hypothesis H_1)\n", 287 | "samples = b.rvs(1000).reshape(-1,5)\n", 288 | "print 'prob of detection = %0.3f'%mean(samples.sum(axis=1)>=3) " 289 | ], 290 | "language": "python", 291 | "metadata": {}, 292 | "outputs": [ 293 | { 294 | "output_type": "stream", 295 | "stream": "stdout", 296 | "text": [ 297 | "prob of detection = 0.935\n" 298 | ] 299 | } 300 | ], 301 | "prompt_number": 6 302 | }, 303 | { 304 | "cell_type": "code", 305 | "collapsed": false, 306 | "input": [ 307 | "# here's the false alarm case\n", 308 | "b = stats.bernoulli(0.3) # true parameter is 0.3 which means it's hypothesis H_0\n", 309 | "samples = b.rvs(1000).reshape(-1,5)\n", 310 | "print 'prob of false alarm = %0.3f'%mean(samples.sum(axis=1)>=3)" 311 | ], 312 | "language": "python", 313 | "metadata": {}, 314 | "outputs": [ 315 | { 316 | "output_type": "stream", 317 | "stream": "stdout", 318 | "text": [ 319 | "prob of false alarm = 0.165\n" 320 | ] 321 | } 322 | ], 323 | "prompt_number": 7 324 | }, 325 | { 326 | "cell_type": "markdown", 327 | "metadata": {}, 328 | "source": [ 329 | "Both of the simulation results follow our earlier analysis. Try tweaking the underlying parameter (i.e. 0.8, 0.3) here and see how the simulation reacts. Not surprisingly, the majority vote test does better when the underlying parameter is much greater than 0.5" 330 | ] 331 | }, 332 | { 333 | "cell_type": "heading", 334 | "level": 2, 335 | "metadata": {}, 336 | "source": [ 337 | "P-Values" 338 | ] 339 | }, 340 | { 341 | "cell_type": "markdown", 342 | "metadata": {}, 343 | "source": [ 344 | " As we have seen there are a lot of moving parts in hypothesis testing. What we need is a simple way to statistically report the findings. The idea is that we want to find the minimum level at which the test rejects $H_0$. Thus, the p-value is the probability, under $H_0$, that the test-statistic is at least as extreme as what was actually observed. Informally, this means that smaller values imply that $H_0$ should be rejected, although this doesn't mean that large values imply that $H_0$ should be retained. This is because a large p-value can arise from either $H_0$ being true or the test having low statistical power.\n", 345 | " \n", 346 | "If $H_0$ happens to be true, the p-value is like a uniformly random draw from the interval $\n", 347 | "(0,1)$. If $H_1$ is true, the distribution of the p-value will concentrate closer to zero. For continuous distributions, this can be proven rigorously and implies that if we reject $H_0$ when the corresponding p-value is less than $\\alpha$, then the probability of a false alarm (a.k.a. type I error) is $\\alpha$. Perhaps it helps to formalize this a bit before we get to computing it. Suppose $\\tau(X)$ is a test statistic that rejects $H_0$ as it gets bigger. Then, for each sample $x$, corresponding to the data we actually have on-hand, we define\n", 348 | "\n", 349 | "$$ p(x) = \\sup_{\\theta \\in \\Theta_0} \\mathbb{P}_{\\theta}(\\tau(X) \\ge \\tau(x))$$ \n", 350 | "\n", 351 | "Here's one way to think about this. Suppose you developed a really controversial study and you are ready to report your results, and someone says that you just got *lucky* and somehow just drew data that happened to correspond to a rejection of $H_0$. What do you do? In a perfect world, someone else would replicate your study (thereby obtaining a different draw) and hopefully also reject $H_0$ just as you did. What p-values provide is a way to addressing this by capturing the odds of just a favorable data-draw. Thus, suppose that your p-value is 0.05. Then, what you are showing is that the odds of just drawing that data sample, given $H_0$ is in force, is just 5%. This means that there's a 5% chance that you somehow lucked out and got a favorable draw of data.\n", 352 | "\n", 353 | "Let's make this concrete with an example. Given, the majority-vote rule above, suppose we actually do observe three of five heads. Given the $H_0$, the probability of observing this event is the following:\n", 354 | "\n", 355 | "$$ p(x) =\\sup_{\\theta \\in \\Theta_0} \\sum_{k=3}^5\\binom{5}{k} \\theta^k(1-\\theta)^{5-k} = \\frac{1}{2}$$\n", 356 | "\n", 357 | "\n", 358 | "For the all-heads test, the corresponding computation is the following:\n", 359 | "\n", 360 | "$$ p(x) =\\sup_{\\theta \\in \\Theta_0} \\theta^5 = \\frac{1}{2^5} = 0.03125$$\n", 361 | "\n", 362 | "From just looking at these p-values, you might get the feeling that the second test is better, but we still have the same detection probability issues we discussed above; so, p-values help in summarizing and addressing aspects of our hypothesis testing, but they do *not* summarize all the salient aspects of the *entire* situation.\n" 363 | ] 364 | }, 365 | { 366 | "cell_type": "heading", 367 | "level": 2, 368 | "metadata": {}, 369 | "source": [ 370 | "Summary" 371 | ] 372 | }, 373 | { 374 | "cell_type": "markdown", 375 | "metadata": {}, 376 | "source": [ 377 | "In this section, we discussed the structure of statistical hypothesis testing and defined the various terms that are commonly used for this process, along with the illustrations of what they mean in our running coin-flipping example. From an engineering standpoint, hypothesis testing is not as common as confidence-intervals and point estimates, which we will discuss shortly. On the other hand, hypothesis testing is very common in social and medical science, where one must deal with practical constraints that may limit the sample size or other aspects of the hypothesis testing rubric. In engineering, we can usually have much more control over the samples and models we employ because they are typically inanimate objects that can be measured repeatedly and consistently. This is obviously not so with human studies, which generally have many more confounding factors. Nevertheless, as we will see, hypothesis testing can provide another useful tool for examining our own model assumptions within an engineering context." 378 | ] 379 | } 380 | ], 381 | "metadata": {} 382 | } 383 | ] 384 | } -------------------------------------------------------------------------------- /LICENSE.html: -------------------------------------------------------------------------------- 1 | Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License. 2 | -------------------------------------------------------------------------------- /Lighthouse_schematic.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unpingco/Python-for-Signal-Processing/0183e2f799556b20eb4519c9cc6ff9ec0afd3089/Lighthouse_schematic.jpg -------------------------------------------------------------------------------- /Markov_chains.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "Markov_chains" 4 | }, 5 | "nbformat": 3, 6 | "nbformat_minor": 0, 7 | "worksheets": [ 8 | { 9 | "cells": [ 10 | { 11 | "cell_type": "heading", 12 | "level": 1, 13 | "metadata": {}, 14 | "source": [ 15 | "Basic Definitions" 16 | ] 17 | }, 18 | { 19 | "cell_type": "heading", 20 | "level": 1, 21 | "metadata": {}, 22 | "source": [ 23 | "Multistep Transitions" 24 | ] 25 | }, 26 | { 27 | "cell_type": "heading", 28 | "level": 1, 29 | "metadata": {}, 30 | "source": [ 31 | "Classification of States" 32 | ] 33 | }, 34 | { 35 | "cell_type": "heading", 36 | "level": 1, 37 | "metadata": {}, 38 | "source": [ 39 | "Steady-State Behavior" 40 | ] 41 | }, 42 | { 43 | "cell_type": "heading", 44 | "level": 1, 45 | "metadata": {}, 46 | "source": [ 47 | "Examples" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "collapsed": false, 53 | "input": [], 54 | "language": "python", 55 | "metadata": {}, 56 | "outputs": [] 57 | } 58 | ], 59 | "metadata": {} 60 | } 61 | ] 62 | } -------------------------------------------------------------------------------- /Projection_Ex.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "Projection_Ex" 4 | }, 5 | "nbformat": 3, 6 | "nbformat_minor": 0, 7 | "worksheets": [ 8 | { 9 | "cells": [ 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "As before, we can easily extend this projection operator to cases where the measure of distance between $\\mathbf{y}$ and the subspace $\\mathbf{v}$ is weighted (i.e. non-uniform). We can accomodate these weighted distances by re-writing the projection operator as\n", 15 | "\n", 16 | "$$ \\mathbf{P}_{V} = \\mathbf{V} ( \\mathbf{V}^T \\mathbf{Q V})^{-1} \\mathbf{V}^T \\mathbf{Q} $$\n", 17 | "\n", 18 | "where $\\mathbf{Q}$ is positive definite matrix. Earlier, we started with a point $\\mathbf{y}$ and inflated a sphere centered at $\\mathbf{y}$ until it just touched the plane defined by $\\mathbf{v}_i$ and this point was closest point on the subspace to $\\mathbf{y}$. In the general case with a weighted distance except now we inflate an ellipsoid, not a sphere, until the ellipsoid touches the line.\n", 19 | "\n", 20 | "The code and figure below illustrate what happens using the weighted $ \\mathbf{P}_v $. It is basically the same code we used above. You can download the IPython notebook corresponding to this post and try different values on the diagonal of $\\mathbf{Q}$." 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "collapsed": false, 26 | "input": [ 27 | "x = np.linspace(0,2,50)\n", 28 | "y = x + x**2 + np.random.randn(len(x))\n", 29 | "\n", 30 | "V = matrix(np.vstack([ones(x.shape),x,x**2]).T)\n", 31 | "Q = np.matrix(np.eye(V.shape[0]))\n", 32 | "i,j =np.diag_indices_from(Q)\n", 33 | "Q[i[:20],j[:20]]=100\n", 34 | "R = np.matrix(np.diag([1,1,13]))*0\n", 35 | "\n", 36 | "Pv = V*inv(V.T*Q*V+R)*V.T*Q\n", 37 | "\n", 38 | "p=np.polyfit(x,y,2)\n", 39 | "\n", 40 | "fig, ax = subplots()\n", 41 | "fig.set_size_inches(5,5)\n", 42 | "\n", 43 | "ax.plot(x,y,'o',label='data',alpha=0.3)\n", 44 | "ax.plot(x,np.dot(Pv,y).flat,label='projection')\n", 45 | "ax.plot(x,np.polyval(p,x),'-',label='polyfit',alpha=0.3)\n", 46 | "ax.grid()\n", 47 | "ax.legend(loc=0)" 48 | ], 49 | "language": "python", 50 | "metadata": {}, 51 | "outputs": [ 52 | { 53 | "output_type": "pyout", 54 | "prompt_number": 69, 55 | "text": [ 56 | "" 57 | ] 58 | }, 59 | { 60 | "output_type": "display_data", 61 | "png": "iVBORw0KGgoAAAANSUhEUgAAAT0AAAE1CAYAAACVyDfwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVOX+B/DPsCOCgMKggKIooggBIuCKWmZ1pTK19GqK\nW3nTXG51+12vpd3KNstsUzO9pqaZprlUkqgsWqgoIiKbC4ssgzgsIsM2PL8/JiZxhmVmzsw5M/N9\nv17zqpk5c84zzxy/POd7nkXEGGMghBAzYcF3AQghxJAo6BFCzAoFPUKIWaGgRwgxKxT0CCFmhYIe\nIcSs6BT0tmzZghEjRmDo0KFYvnw5V2UihBC90TroSaVSrF27FsePH8f58+eRk5OD2NhYLstGCCGc\ns9L2g/b29mCMoaqqCgBQW1sLFxcXzgpGCCH6oHVLz97eHhs3boSPjw88PDwwcuRIhIeHc1k2Qgjh\nHtNSWVkZ69OnD8vNzWXl5eVs3Lhx7OjRoyrbAaAHPehBD708tKF1S+/cuXOIjIxE//790b17d0yb\nNg2JiYlqt2WM0eOBx+rVq3kvg9AeVCfmVy+lpXewd28mDh9mysfevZkoLb3T4We1pXXQGz16NFJS\nUiCVSlFfX49ff/0Vjz76qNYFMTd5eXl8F0FwqE7UM+V6SU8vg729f6vX7O39kZFxW2/H1PpGhpOT\nE1atWoXJkyejtrYWjz32GMaNG8dl2QghJk4uV9/uamwU6e2YWgc9AIiJiUFMTAxHRTEvVG+qqE7U\nM+V6sbRsVvu6tbX2l68dETFdLo47cwCRSKfrb0KI6ZJIpEhIaH2JK5NlISrKHWKxa7uf1Ta28Bb0\nXF1dUVFRoc9DEw25uLhAKpXydvz4+HiMHTuWt+MLlanXi0QiRUbGbTQ2imBtzRAQ4NZhwAO0D3o6\nXd7qoqKiglqAAiMS6S+PQkhbxGLXTgU5rvDW0qPLXuGh34QYE23PV5plhRBiVijodUJMTAzeeOMN\nvoth8uLj4/kugiBRvXCLgl4niESiTuW7xo4di61btxqgRIQQbVHQ66TO5A7oRoBuTPkOpS6oXrgl\nuKAnkUgRF5eF2NgcxMVlQSLRvAuFrvtITU1FaGgoxGIxFi5ciKamJgBAZWUlJk2aBHd3d7i4uCA6\nOhpFRUUAgP/85z9ISkrCkiVL4OjoiKVLlwIAli1bht69e8PJyQlhYWE4ffq0xt+HEMIdQQW9lo6K\nMpk/Ghr8IJP5IyGhTKOgpes+Ghoa8PTTT2PmzJkoLi7GhAkT8MMPP0AkEqG5uRnz589HQUEBUlJS\n0NjYiCVLlgAA3n33XYwePRpffvkl7t69i88++wwAEB4ejrS0NBQUFGD8+PGYNm0a6uvrNa8cM0C5\nK/WoXrglqKDHxeBjXfeRnJwMiUSCxYsXw9LSEs8++yzEYjEARYfqyZMnw87ODr6+vnj11VeRkJDQ\n6vMPXgbPnDkTLi4ucHZ2xhtvvIGqqirk5OR0+vsQQrglqKDHxeBjXfdRXFyMAQMGwM7OTvlaaGgo\nAEAmk+HFF1+Ej48PunXrhilTpqCqqqpVoHswr7du3ToMHjwYzs7O6N27N+rr61FeXt7p72NOKHel\nHtULtwQV9LgYfKzrPnr27Inc3FzIZDLlaxcvXgRjDOvWrUNycjLOnj2Lqqoq/Pjjj63m9nqws2RS\nUhJWr16NXbt2obKyEgUFBcpp9gkh/BBU0AsMdIdMltXqNZksCwEBbgbbx4gRIyAWi7Fx40Y0NjZi\n3759kEgkAICuXbvC2dkZtra2uHr1Kj744INWnxWLxcoACQBdunSBjY0NunXrBolEgpUrV1I+rx2U\nu1KP6oVbggp6YrEroqLc4eCQDRubHDg4ZHdqtgUu92FtbY0DBw5g165d8PT0xG+//YbnnnsOIpEI\n8+bNg6enJ/z8/PD8889j3rx5rS5nly1bhri4ODg7O2P58uUYOnQoXnrpJYwfPx5jxozBkCFD4O3t\nrXG9EEK4Q2NviRL9JsSY0NhbQgjpBAp6RDAod6Ue1Qu3KOgRQozOt5e+1fqzlNMjSvSbEGOQWpKK\nR3c9ivJ/lVNOjxBi2mobazHzwEx8OvFTrfdBQY8IBuWu1KN6+ctrx19DsEcwZgbN1HofvK2RQQgh\nmjiacxRHc44ibVGaTvuhnB5Rot+ECJWkRoLgzcHYO3UvxvQZA4D66QlOUlIS/P39O95QIPslRKgY\nY5h3eB7mBs9VBjxdUNDTk9GjRyMrK6vjDTtgYWGBGzducL5fIaLclXrmXi9fnf8KZffKsGbsGk72\nRzk9LTU1NcHKyjDVR5ecxFxdvX0Vq+NX4/f5v8PG0oaTfVJL7wE+Pj744osvEBkZiR49emDevHmo\nr69HfHw8vLy8sGnTJvj5+WH+/PloaGjA8uXL4enpCU9PT6xYsQINDQ0AFH+d759coLi4GFOmTIGb\nmxv69u2Lzz//XPlec3Mz1q5di/79+8PJyQnDhg3DrVu3MGaMoin/0EMPwdHREfv27VPZb2ZmJsaO\nHQsXFxcMGTIER44cUb4XExODf/7zn5g+fTpcXV0RGRnZqtUoNDRvnHrmWi91TXWYvn863nv4Pfh1\n9+Nux0zP2jqEAQ6tlT59+rCAgAB269YtJpVKWWRkJFu1ahWLj49nVlZWbN68eaykpITJZDL2xhtv\nsLCwMHb79m12+/ZtFhERwd544w3GGGOnTp1iXl5ejDHG5HI5Cw0NZUuXLmWlpaUsMTGR9erVi8XG\nxjLGGPvwww+Zv78/y8nJYYwxlpaWxu7cucMYY0wkErHr168ry3f/fhsaGpivry975513WGNjIzt5\n8iSzt7dn2dnZjDHG5syZw5ycnNiBAwfY7du32aRJk9j06dPb/O5C/U2IeXr5l5fZlL1TWHNzs9r3\ntT1fKeg9wMfHRxm4GGNs8+bNzNfXl8XHxzORSMQKCgqU7/Xr149t3rxZ+XzLli3Mx8eHMdY6OCUn\nJzNnZ2dWW1ur3HbZsmVs7ty5jDHG/Pz82Pr169WWp72gl5iYyKytrVlNTY3y/ZEjR7I1a9YwxhRB\nLzo6Wvnenj17mL+/f5vfne/f5NSpU7weX6jMsV6OZB9hvdf3ZtJaaZvbaHu+CjanJ3qLm+UU2WrN\n82HBwcHK/w8JCUFxcTEAxSSh919alpSUYOjQocrnQ4cOVW57v/z8fNTU1KBXr17K1+RyufLy9dat\nWxg5cqTG5SwuLoafnx8cHByUr4WFhSnLIBKJWn0XDw8P1NTUaHwcQgyp5G4JFhxegP3P7oeLvQvn\n+xds0NMmWHElNTUVzzzzDADFVPGenp4AoHLjolevXkhJSVEGvpSUFOW29/P29oajoyNKS0thY6Oa\njPX29sbp06cxbNgwteVhbdzI6NWrF3JycnDv3j1l4Dt//jwmTpzY4WeFyFxzVx0xp3ppZs2Y/dNs\nLApbhFG9R+nlGHQj4wGMMfz0008oKiqCVCrF9u3b8dxzz6nddsaMGdi2bRvKy8tRXl6Obdu2Ydas\nWSrbhYeHo2/fvli5ciXy8vIgl8tx5coVpKSkAAAWLFiAzZs349dff0VTUxMuX74MqVSxZKVYLFZu\n96CIiAh4e3tjw4YNaGxsRHx8PFJTUzF9+nTldyHEmKz7fR1kjTKsGrNKb8egoPcAkUiERYsW4Zln\nnsGAAQPg7++PVatWgTGmstLZqlWrEBkZiaCgIAQFBSEiIgKrVqn+WJaWljh69CiKi4sRGRkJNzc3\nvPDCC6iurgYA/POf/8Tzzz+PxYsXw8XFBQsXLkRdXR0AYM2aNVi2bBmcnZ2xf/9+iEQiZTlsbGxw\n5MgR/Pbbb3Bzc8OSJUuwZ88e+Pn5Kb/Lg2V+8LmQmHt/tLaYS72cLzqPj//4GN898x2sLDq4CC0t\n1f5AWmUCNdDWIQxwaK34+PiwEydO6LyfEydOsH79+nFQIsPh+zcxx4R9Z5hDvVTXVbP+n/Vn+zL2\ndbyxVMpYbKzp3cgwZowxnD9/Hv369eO7KEbFnHJXmtBXvUgkUqSnl0Eut4ClZTMCAzVbhIsrjDEs\n+nkRxvmMw9TBU9svn5M9kJIC3HeDTlMU9PRg2bJlyMzMxNq1a/kuCiFqSSRSJCSUwd7+r3HcCQlZ\niIqCwQPf/y79D2mlaTi38Fy75Us8lYGH7fPgOsQfcHfX+ng0ywpR4vs3iY+Pp9aeGvqol7i4LMhk\nqhNXODhkY/z4gZweqz1Xb19F1PYoJMQkYLDb4HbL1+36RdjZ5yP8xckAaJYVQogG5HL1//QbGw13\no0vWKMNz+5/DB4980CrgAarlcyjOhWV9LSp6D9L5uBT0iGBQK089fdSLpWWz2tetrQ3X0l8euxxB\n4iDMDZ6r8t795bOtKIV9WT4q+4fB2lb3oExBjxAzFBjoDpms9RRlMlkWAgLcDHL8vVf24uTNk9j0\nt01qu1G1lM+y9i6c8i6jsn8Y7snzOCkfBT0iGObSH01T+qgXsdgVUVHucHDIho1NDhwcshEVZZi7\nt9el1/Hyry9j79S9cLR1bLt8w53Rs/BH1Pnaw1Ys4ax8dPeWEDMlFrsa/E5tfVM9pv84HavGrEJo\nz9C2N2xuhrjgOsTRw4GB3N5YoZYeRx6c4bg9q1atQt++fREZGYnTp0/T9O9/opyeeqZUL68efxXe\nTt54Ofzl9jdMSwPs7DgPeAC19AwuKSkJmzZtQnZ2Nrp37w4AraZ/9/HxwbZt2zB+/Hi+ikiIXuzL\n2Idfcn/BhRcutD8c8to1oKYGGDFCL+Wglp6B5efnY+DAgcqA9yC++8rxiXJ66gmlXiQSKeLishAb\nm4O4uCxIJNJOf/aa9BoW/7IYP0z9Ac52zm1vWFoK5OUBw4YBlpa6F1oNnYLevXv3MGfOHISEhGDw\n4MFITk7mqly8aWu6eADYsmULBgwYgO7du+Opp55CSUmJyufPnz8PDw+PVoHrwIEDCA4OxrZt27Bw\n4UKcPXsWjo6OeOutt1pN//7888+joKAA0dHRcHR0xLp16wzzpQnpQMsICZnMHw0NfpDJ/JGQUNap\nwFfXVIdp+6ZhddRqDO01tO0Nq6qAy5cVAc/OjsPSP0CrEbt/mj17Ntu6dStjjLHGxkZWWVmpsk1b\nh9Dx0HrT1nTxJ06cYM7Oziw1NZXV19ezl156iY0ZM0b5uftnOB48eDD79ddfle89/fTT7JNPPmGM\nMbZ9+3Y2atQo5Xv3z4TMGHcTHmhDqL8J4d/x45ns8GGm8jhxIqvDzy46uog9u+/ZNqd9Z4wxVlfH\n2PHjjBUXd7pM2p6vWrf0qqqqkJSUhHnz5gFQTLDZrVs3TgIxn0QiEZ555hl4enrCxcUFc+fOxZ49\ne7B79248/vjjCA4Oho2NDVasWIGkpCQUFBSo7GP27NnYtWsXAEAqleK3337D3//+dwA0xx0xTtqO\n4NiTvgcnbpzAlugtbefx5HLg3DmgTx+gZ09di9ohrW9k3Lx5E25uboiJiUFKSgqGDx+Ozz77DPb2\n9tyU7L5VvXQSHa3xR9RNF19cXNzq5kLLymVFRUXo3bt3q8/PnDkTAQEBqK2txQ8//IAxY8ZALBZr\n/x3MBI29VU8I9aLNCI7s8mwsPbYUx58/Didbp7Z3npoKODoCAwboWsxO0TroNTU14fz581i1ahU2\nbtyIF198Efv27cPs2bO5KZkWwYorD04X36tXL/Tq1QsXLlxQbpObm4vq6mq108N7eXkhMjISBw4c\nwK5du/DSSy91+tjmfCODCFdgoDsSErJazXoik2UhPFz9bCf3Gu5hyg9TsHb8WgR7tDMN1NWrQEMD\nENpOnz2OaR30vLy80L17d0T/GZxmzJiBHTt2qA16MTEx8PHxAQA4Ozu3akkJDftzuvhFixbB3t4e\n27dvx/Tp0zFu3DhMmzYNaWlp8Pf3x4YNGzBq1CiVVl6L2bNn4/3330dhYaEygHaGWCzGhQsX8PDD\nD3P1lTTScqewpWVhyOdjx47l9fhCft6Cz/JERQHffbcTcjkwdGg4wsPdkZl5GZmZrbdnjGFb5TaE\n9QpD/+r+rVqqrfZfUID42FhgyBCMtbDo8Pjx8fHYvn07ACjjiVa0ygT+KTIykiUnJzO5XM4WL17M\nvvnmm04nG3U8tN74+PiwL774goWHhzNXV1cWExPDZDIZY4yxTZs2MV9fX+bq6sqio6NZUVGR8nMW\nFhatlmqsra1lTk5OLCYmptX+t2/fzkaPHq18furUKebt7a18fujQIda7d2/m5OTEPv74Y319TbWE\n+psQ4/LVua9Y0MYgdq/hXtsblZUxFhvL2H3Ll2pK2/NVp7M8OzubRUREMF9fX/b000+3Wn+1o4IJ\n9R8YV3dPm5ubWf/+/Xm7E6sNvn8Tc5gWXRvGVC/nbp1jbh+6sZzynLY3untXEfD+XNBeW9qerzqN\nyPDz8zOJvnn6sHv3bohEIhpZQczGndo7mLZvGjZP2owB3du4KVFfD5w9CwweDLgafmp6gIah6cXY\nsWNRWlqKr776iu+iGBW+71AKlTHUSzNrxqyDszAtYBomD5qsfqOWrine3oCXl2ELeB+aLp4o0W9C\ntPXfhP/ixM0TODH7hPrlGxlTLOhjba3Toj73o+niidETyhhToRF6vRy7dgybL2zG91O+b3u92owM\noKkJCAoybOHUoMtbQojWblTcwJyf5mD/tP3o6djGaIobN4DycmDkSMBC93ZWy9KQ2qLLW6JEvwnR\nRG1jLUZsHYH5IfPxckQb8+OVlABXrgCjRgEcjNa6f2nIJ5+ky1tCiIEwxvDCkRcQKA7EkvAl6jeq\nqADS04HwcE4CHgCkp7deC1cbvAU9FxcXiEQiegjo4eLiwtfpAED4uSu+CLFevjj3Ba6UXcHmSZvV\nTyRw7x5w/rzipgWHE5G0NfGBJnjL6UmlnZ+A0BQJYRA5IdpIyk/CO0nv4I/5f6CLdRfVDVr64vn7\nA+7qx+Zqq62JDzTBW06PEGJ8iu8WY9iWYfgm+hs8PuBx1Q3kcuD33xXBTg/rW1BOjxBiMPVN9Zj6\nw1T8I+wf6gNeS188Jye9BDyg9dKV2qKgxxMh5mn4xled6LL2gyEI5VxZemwpxF3FWDl6pfoNLl9W\n/FfPffHEYleMH699UKV+esSs3X+51CIhIQtRUTD4mrBCtillE04XnEby/GRYiNS0lXJygOpqxQpm\n7a10JgCU0yNmLS4uCzKZahcIB4dsnVoTQtPSoVcut4ClZTMCA907HdST8pMw5Ycp+H3+7+jv2l91\ng4ICxbKNI0cCtrYcl7xtNAyNEC1ou/aDMdFlJbPCqkI8t/857Ji8Q33AKy0FsrOBiAiDBjxdUNDj\niVDyNELCR51os/aDoelaL+o69Nrb+yMj43a7n5M1yjB572Qsj1yOx/o/prqBVKrI44WHAw4OOpXR\nkCjoEbMWGOgOmSyr1WsyWRYCAtx4KhH3tGnNMsaw6OdFGNB9AF4b8ZrqBnfvKu7UhoRw2vnYECin\nR8yeRCJFRsZtNDaKYG3NEBDgZlI3MbTJW36a/Cm+TfsWZ+adUe2ALJMBZ84AgwYBahbGMhRtYwsF\nPUJMnLo71DJZFqKi1N/MiL0Wi5hDMUien4w+zn1av9nQoOh83KcP0LevvoveLrqRYWQop6eK6kQ9\nXevl/g69NjY5cHDIbjPgZZdn4/mDz2PftH2qAa9l5mOxuMOAJ+S+j9RPjxAzIBa7dnjJXiGrQPSe\naLz/yPsY1XtU6zebmxU5PEdHxWXtn9R1hQEg6L6PdHlLCEFTcxMe/+5xBLoH4pOJn7R+kzHg4kXF\nf4cOVXY+buuyWSSqhp1duMoxuO77SJe3hBCtvfLbK7AUWeLDCR+qvpmersjlhYa2Gm3RVleYGzfu\nqj2GUPo+UtDjCeWvVFGdqKfvetlyYQtir8Xi+6lq1rjIygKqqoBhw1Smem+rKwxjwu77SEGPEDOW\nkJeAVadW4ciMI3C2c2795o0biuneIyIAK9X0f1sdu/v3dxJ030fK6RFipnLv5GLU/0Zh9zO78XC/\nh1u/eeuWYnjZiBFtTvXeXlcYAHrv+0j99AghnSaVSTF863C8MvwVvDD0hdZvtizmM3w40LVru/vh\ns2M3BT0jQ9PFq6I6UY/remmQN+CxXY8hpGcIPn7049Zv3r4NpKYqLmkFPryM7t4SQjrEGMNLP78E\nBxsHfPjIA3dqpVJFwBs2TPABTxfU0iPEjKz7fR12Xt6J03NPw9HW8a83qqoUi/mEhABuwrjh0BFt\nYwuNyCDETBzKOoT1yeuRPD+5dcCrqVEMLwsKMpqApwu6vOUJ9UlTRXWiHhf1cqH4AhYcWYCDzx2E\ndzfvv96orQWSkxVDyzw8dD6OMaCgR4iJy6/Mx5PfP4nNkzYj3PO+4WEyGfDHH8CAAYCXF38FNDDK\n6RFiwqrqqjBy20jMD5mPFcNX/PVGff1fU0T162ew8uiyVseDqMsKIaSVBnkDnvjuCQxyG4TPHvsM\nopZxsy1z4nl6Klp5BqLpvH4doS4rRobyV6qoTtTTpl4YY1h0dBG6WHfBpxM//SvgNTYqcngeHgYN\neID2a3Vwje7eEmKC3k16F5cll5EQkwBLC0vFi01Nim4p3bsD/qrTx+ubUFaeo8tbQkzMd5e/w39O\n/gd/zP8DPR17Kl6UyxUBz9ERCAzkpVxcrzFMl7eEEJy4cQIrYlfg6N+PqgY8BwfeAh4gnJXnKOjx\nhPJXqqhO1OtsvaSVpmHGjzOwb9o+DHEfonixZV2LLl0UnY95pMlaHfpEOT1CTEB+ZT4m7ZmEL574\nAlE+UYoXm5uB8+cBOzvgoYdazXrMl86s1aFvlNMjxMhJZVKM2jYKLwx9AcsjlytebAl41taK8bQC\nCHhco356hJihuqY6TNg5ARGeEVj36DrFiy0rl1laqqxrYUroRoaRofyVKqoT9dqqF3mzHLMOzIKX\nk9dfC/o0NwMXLijWszDhgKcLyukRYoQYY1h2bBnuyO7g2MxjsBBZ/NXCo4DXLrq8JcQIvZ3wNg5k\nHUD8nHh0s+vWOuDdtzatKaP59AgxE5tSNuHbtG9xet5pvQQ8LicFECLK6fGE8leqqE7Uu79e9l/d\nj7cT30bsrFh4dPX46y6tpSVnAS8hoQwymT8aGvwgk/kjIaEMEolUx28hHDoHPblcjpCQEERHR3NR\nHkJIG07ePImXfn4JR2ccha+r718Bz8qKsxyeUCYF0CedL283bNiAwYMH4+7du1yUx2zQql+qqE7U\nGzt2LFJLUjF9/3T8MO0HhPQMgaT4NvJ/PIEmC1vU+vkjsKyCk0tQoUwKoE86tfRu3bqFX375BQsW\nLKCbFYR0kkQiRVxcFmJjcxAXl9XupaNEIsX2I8fwyP8ew0u938Ag+yBIim/jytY4yJr9ccf7acjq\nBnF2CWpp2az2dWtr0/n3rVPQW7FiBT766CNYWFBqUFOUv1JlDnWiSc5MIpHix7g0rDgcg5k912Jo\nl5eReLIYGVt/gqVzAKr7BSsvabm6BBXKpAD6pPXl7dGjR+Hu7o6QkJAOT9aYmBj4+PgAAJydnREc\nHKy8lGn5rLk9byGU8tBzwzzfteso6ut7IzBQkTdLT1e836OHCGKxa6vtEy9kYnXCHPjdjcCEHvMh\namqE5NRJnK8oxOjZC1t9PjBwLBobRTqXLzPzMqytq+HgIEJjowgZGWfRt283iMX+vNdffHw8tm/f\nDgDKeKINrfvprVy5Ejt37oSVlRXq6upQXV2NKVOmYMeOHa0PQP30CFGKjc1BQ4Ofyus2NjmYOPGv\n16vqqjD08xEY6vQMZvV6G6LGBrhkJ6PBqQfO3i1CQMAElX1oOy+dseJ17G1CQgLWrVuHI0eOcFYw\nQkxRZybSrG2sxcRdE9G9sQ/meeyEZVMDXLL+QL2LB2q8/FFffxbNzd04W2vCWPE+9lZkBj3AuWQO\n+StNmUOddJQza5A3YMoPU9DXuS82PvkpGqvTcPOXL1HX3RM1Xv6QybIwevQAQcxLZ6w4GZERFRWF\nqKgoLnZFiElTTKQJZGRko7FRBGtrhvBwRcBqam7CrAOzYGdlh21PbYOVrB7j7W7iOw+g0YfBwTpb\nuW3LvojmaOwtIQLQzJoR81MMSmtKcXjGYdjVNiimePf3B7y9+S6eINHYW0KMFGMM//j5H8ivysev\nM3+FXU2dYor3IUOAXr002pepj5vlAnWw44k55K80ZWp10plOyIwxrIhdgbTSNBydcRRdqmWKgPfQ\nQ8qA19l6MYdxs1ygoEeIHnQmADHGsPLkSiTmJ+LYrGNwrKxVTAAaGgqIxRof0xzGzXKBgh5PaJyp\nKlOqk84EoHeT3sWR7CP47fnf4FxeA6SlAeHhQI8erT7X2Xoxh3GzXKCcHiF60FEA+ujMR9iRtgOJ\ncxPRQ3IXuH4dGDEC6NpV62Oaw7hZLlBLjyemlr/iginVSXsB6JM/PsHmC5txcs5JeBRVAXl5wMiR\nbQa8ztaLOYyb5QIFPUL0oK0AdEb+I748/yVOzTkFr4JKQCJRtPDs7XU+plAW0xY66qdHiJ5IJFJk\nZNxWdkJObv4J32RsRvzsk+h9UwrU1SlyeFaUZdIGrXtLiIB9df4rfHjmQ8TPioPPtXJFoAsJUUzz\n/ifqY6cZ3sfeEs2YUv6KK6ZaJ5tSNuGDMx/g1IxY+GSWAF26KNazeCDgtdXFxVTrhS8U9AjRo43n\nN2Jt0lqcmvoz+mYUAe7uQFCQynoW1MfOcOjylhA9+fzs5/j4j49xavIh9M0pA/z8gD591G7b2Xn2\nyF9o7C0hArL+j/X4/NznSIw+gN7ZEiAwEOjZs83tqY+d4dDlLU8oT6PKVOrkwzMf4quUr3B6wvfo\nfb0cCAtrN+AB7fexM5V6EQpq6RHCoXcT38W3ad8iadT/4FFyt9OjLNqbZy8z0wAFNyOU0yOEA4wx\n/Dfhv/j+yh4kDv0Sbo3WQEQEYGvLd9FMFuX0COEJYwz/d+L/cCz7Z5we8im6W3QFRgylTscCRTk9\nnlCeRpUx1kkza8aSX5cgKScOiX7vo7tLT85HWRhjvQgZ/SkiREtNzU1YcHgBiouzcbzfajj4DAQG\nDOC7WKTbGRcLAAAbKklEQVQDlNMjRAsN8gbMOjALrKwMO33+CbvgoYCnJ9/FMiuU0yNGxZjHmdY1\n1WHavmlwv1OHjb6vwiZiBOBqHGUnlNPjjTnnadoaZ3rw4GG+i9ah6vpqPPHdE/Ark2PzoH/BZsw4\nvQc8cz5X9IFaesTg2hpnmpt7nqcSqVLXErXoKsffdj6GyXU++FfYcliGRwA2NnwXlWiIcnrE4IQ+\nzrSlJXp/YC6sPoUvSxZihdUIzI9aAdFDDwEWdKHEJ8rpEaMh9HGmD7ZEb9Vl4ZPs2Xi5fhQWvLAa\n8PXlsXREV/SniifmnKdpa5xpVVU2TyVq7f5FfXLvpeDzlDF4pXY6gkJe5iXgmfO5og/U0iMG19Y4\n08zMUr6LBuCvlmha9QkcTJmGV21XoOewZWBuJTyXjHCBcnrE5OjaHUYikeK9n7biUtpazHd7G64P\nLcS9ppu0yI7AUE6PGIyQ+9ipuwmRkJCFqCh0uox7M7fgVto6LPV/Cw4DHoG1bR7CAoTzHYluKKfH\nE2PN07S3loOuuKgTXaZdb2bNeHP/Ylze/yU+W/QTnlm6FBMf98f48QN5DXjGeq4IFbX0iEbaDirZ\ngmgJ3X8T4n6NjSK1r7dokDfgtS3TYJF7DR+9fgIu3jSG1lRR0OPJ2LFj+S6CVrQNKp3BRZ1o0x2m\nqrYCr61/HO4N1vjPm4mw79Zd53JwyVjPFaGiy1uiEaH3sWtv2nV1CiQ5ePmNoejftTfe+k+c4AIe\n4R4FPZ4Ya55G06CiCS7qRNEdxh0ODtmwscmBg0N2m3dd066ewutvjsDYyBl4beleWNoIc5ZjYz1X\nhIoub4lG2lvLQSjEYtcOy3Py5FZs3PNPzJ6zHtGj5hmoZEQIqJ8eMS9yOX7Yswq7//gaK5f/iPAB\nY/kuEdES9dMjpAPymrvYtHkhjpf9gY/X/AFfN/4nNyCGRzk9nlCeRpU+66Qm/xrWvDsBScjH1tUX\njSrg0bnCLWrpEUHhfLQHYyg6dxKrdy9El8hR2PHsN7CxpDnwzJkgcnpCHtZEDEfdEDKZLEv7Ma/1\n9Uj/5Vv836mVeHTq61g6+lWIRLr3JyTCoG1Oj/egx/mJToxWXFwWZDJ/ldcdHLIxfvxAzXZ25w5O\n7P8IK69/jTcX7MTf/P7GUSmJUGgb9HjP6ekyVtKYUZ5GVWqq+uniNRrtwRjkWZnYtnUJ/lW+G9+8\nkmD0AY/OFW7xntPT57AmYlwsLNT/1e70aI/6etQkJ2JN/Bqk9bLAsb9fgJuD7p2miWnhvaUn9GFN\n+kLjKVXNmjVJ+9Ee5eUoPPIdpsa9iLqwYPwy76TJBDw6V7hFOT0iKBKJFBkZt5WjPQIC3No/DxgD\ncnJw7o/9mJ23Hq8++T4WhC4wXIEJb4z2RgagxYluAuLj4+kv+AM0rhOZDOziRey6shtvVh3EdzP2\nY4T3CL2Vjy90rqjHy4iMwsJCzJ49G2VlZXBzc0NMTAxiYmI03k9nxkoS0kpJCe5dPIt/Xd+I8/YV\nSFp0Dl5OXnyXihgBnVp6paWlKC0tRXBwMMrLyzFkyBCcOnUKgwYN+usANPaWcEkuBzIykH/tAqZd\nfw+hg8Zjw2MbYGvF7wwp1NfU8Hhp6Xl4eMDDwwMA0KNHDwwbNgzFxcWtgh7hj8n9Q6yuBi5exG/l\nZzH72kd4b+JHmBsyl+9ScbIuBzEczu7eXrt2DRkZGYiMjORqlyZN332v9LmWhb60Wyc3bkD++xn8\nt2g3FhVtxi9zfhNEwAP039eU+ulxi5N+ejU1NZg+fTrWr18PBwcHlfdjYmLg4+MDAHB2dkZwcLAy\nMdvyg5rb8xb62n9Tkwfs7f2Rnq54Hhg4Fvb2/vjuu50IDfXm/ft3+vlvvwG5uQjw88LM/I9QVdiI\nT0d/itCeocIoX3w8UlML4e+vCHr313djo4iT/V+6dEk4vwePz+Pj47F9+3YAUMYTbeh897axsRGT\nJk3C448/juXLl6segHJ6vIiNzUFDg+pMIjY2OZg40UhmGCkuBq5cwWnLIkxL/TcWhy/BytErYSHi\nvXtpK5wOnyOdxktOjzGG+fPnIyAgQG3AI/wx6k7fTU1AejrkUik+qI/Dl7nfYfeUPRjXdxzfJVMr\nMNAdCQlZKn1Nw8PdeSwVaYtOfzLPnDmDXbt24eTJkwgJCUFISAiOHTvGVdlM2oOXuVzT51oW+hIf\nHw+UlwPx8bhdX4FHC97GiYoLuPjCRcEGPECzdTm0oe9zxdzo1NIbNWoUmpvVtygIv7hay8Jgd4Dl\ncuDGDaCpCQndKjEjcSleGPoC3hjzBiwtLLk/Hseor6nxEMSIDCJMBhsiKJUCly6h0akr3iz/ATuv\nfo9vn/4WD/d7mLtjEJNDa2SYMX21xtruipHNTdCTy4GcHODWLeR5OeLZ00vg7uCO1BdTTWayACI8\nFPR4Es/ReEp9dozlatovtUHZRgRcugQ4OWG3axGWHX0NM7rOwIYZG9TObmxyHa01wNW5QhQo6Bk5\nfbbGuLgDrBKU5XKk7voFEd6NsAofgsWp7+B88XnEPR+HiqyKNgMejXggXBFWhyczwtVfbn1OwsrF\nHeD7g7J19R10z0iAvaUXdqIaQYcmwt7aHikLU/CQx0Nt1om5zq7dglp53KKWnpHTZ388Lu4Ay+UW\nEMmb0LUwE7aVEkh7+2Gb7EvEX9uOb6duxSS/SZ3ahzo0uzbRBgU9nnCVp9F3x1hdu2LYV0nglF2A\nemd3XPR1xbpb0RDb9MWW0J8wyW94q23bqpP2Aruhc3185BYpp8cturw1cvruGKu1ujogJQVDrKQo\n6WmPnfbHsfLmRES7LcOKnu9hZEjnh2e1dZnt7m5l0EkVjHESB6KK+ukR7uXnA9nZQJ8+yHFlmPXj\nHNTXMazo+x56O3ppNTO2utm109PLDDrmlcbYCgv10yP8q64G0tMVyzBGhGPD1W1Ye2gt3ox6E0vC\nl+g0UYC6y+xLl8rVbquvXB/lFk0DXd7yxKTGUzY1ARkZQHIy4O2NnEHuiPpxEn7K+gnJC5KxNGJp\npwKepnVi6EkV+JrEwaTOFQGgoEd0U1ICxMcDjY1oGjMKH9/ahxFbR+DZgGcRHxOP/q799XZoQ0+q\nYIyTOBBVlNMj2rl3D7hyBZDJgKAgXGoowILDC+Bk64Svo7/Wa7C7n6FX0jPHlfuEyqiXgCRGRC4H\ncnMVNyt8fSHz7om3kt7GttRt+OCRDxATHKN2VAUhXNM2ttDlLU+MMk9TXAycOqVo3UVF4ZRlIYK+\nDkZeZR7S/5GOuSFzdQp4RlknBkD1wi26e0s6dveu4lK2sREIDUWZbRNeO/YiTt08hS+f+BLRA6P5\nLiEhnUaXt6RtDQ2K/nYlJYCfH+TeXvj64hasjl+NmOAYvBn1JrradOW7lMRMUT89wp3mZuDmTeD6\ndcDTExg3DhduX8Y/tj0LWytbnJxzEkPch/BdSkK0Qjk9ngg2T1NaquiCcucOMGIEpP16YsnxFfjb\n7r9h8bDFSIxJ1FvAE2yd8IzqhVvU0iMKlZXA1auKS9rAQMi7u+LrC19jTcIaTB08FVcXX4WrPXXN\nIMaPcnrmrrYWyMxUrFMxcCDg7Y2E/EQsPbYUrvau2PDYBgSJg/guJSEqKKdHNNPQoFifoqgI6NcP\nCA5G/t1beG3/czhXdA7rHl2HKYOmUJ87YnIop8cT3vI0TU2KzsWnTimejxuHSm83vH5qJYZ+PRRD\n3Icgc3Empg6eavCAR7kr9aheuEUtPXPR3Azk5QHXrgFubsCoUWiws8amlE14N+ldPDnwSaT/Ix09\nHXvyXVJC9IpyeqaOMaCwUHEp260b4O8P1rUrDmYdxOtxr2OA6wB8OOFD6oJCjA6NvdWQyS8pyJgi\nX5ebC9jZAf7+gIsLTtw4gZUnV6K+qR4fTfgIE3wn8F1SQrRCY281IIRpv/WWp2kJdvHxQEEBEBgI\nDB+Os/dy8MiOR7Do50VYEbkCF1+8KLiAR7kr9aheuGWWOT19rhXLG8YUEwLk5AC2topg16MHrpRd\nwRt7FyKlOAVvjnkTMcExsLa05ru0hPDGIJe3x49nCuryMTY2Bw0Nfiqv29jkYOJE1dcFrblZ0bK7\ndg2wsVH0tfsz2P034b9IyE/A6yNfx0vDXoKdlR3fpSWEM4Lup6e4fBTOivR8Tfutjta5Rblccfl6\n/TrQtSsQFAR0747Lksv47w+LcLrgNF4Z/gq2PbWNJgX4k8nncUmnGKSld/iw4hBCWTWqJaf34Fqx\nbS2dyMU/lgf3cfduDkaMGKVxOa5cKoZ1cQm6SgrQJ7gPukeGAc7OuFR6CW8nvo3fC3/Hq8NfxaKw\nRXCwcdCojHzT5/qumv7mQkLr3qon6JZeC6GsGqVYKxbIyMhWTvsdHt52oHnwH4umrVZ1+7h06Rya\nmq7B3j681bZt5RYl+cVI/fEinO9ao76bO0r7RiK36hZsbiZic8ZXSC9LxyvDX8HOyTvRxbqLJtVh\nFkwyj0u0YtCgx8flY1vULSmoDhf/WNTtIyxsNq5ePYHBg1W3b/XH4e5d4Pp1FB1Lga3jcNzp3Q9N\nNrY4W3UIP5a+j9r8Mqx5ZBUOTT8EWyvbTpVHqPTZmjHm5RuplcctgwU9mSwL4eHuhjocZ7j4x9LW\nPhhrJ7d4+zZw44ZiLdm+fVEeOBzVTf0QL92FQ2WfwM6iK6aK/w+j3QfjidBBnS6LuRJSHpfwyyD9\n9Bwcso0id6IOF/9Y1O0jPT0e/fs7tV5SUC6HqOAEgquyFdM89eoFPPwwisVdsL30cyy40gfnqg7j\nRa8v8PHAcxjhMgV2NqbT1VKf/dGMeflG6qfHLYO09IRw80JbgYHuSEjIUkmAa9JqVbeP+voCjB49\nCQCQeTENVreK0VVaDO9gH7iGjQS6d0dKcQo+/enf+Dn3Z0z2nYI1Njvg6/yY1uVoj6nf2dQkj0tM\nm9kOQ9MEF2udqt2HqEmxlKJUCnh5AX37QmYtwt6MvdiYshGlNaVYMmwJFoQugIu9i97WXDXmO5vE\nfNHYW2PR0KCYACA/H7C2Bvr0ATw9kVN5HZtSNmFH2g6Ee4bjH2H/wBMDnoClhaXeixQXlwWZzF/l\ndaF0MSJEHaPosmLWyssVwU4iATw8EF9bi4iJD+Ng1kFsjd+KdEk65oXMw7mF59DPpZ/eiqHuMlYo\ndzapP5p6VC/coqCnT3V1wK1bipETlpZA795gAQFIuZ2G9ce2ICljNsI9w/FC6At42v9pvXc5aavP\noUhUDTs1I9ToziYxRXR5y7XmZkVrrrAQqKgAevYE+vRBIarx/ZXvsePyDsgaZZgbPBezH5oN727e\nBitaW5ex9fVn0dzcjXJ6xKjQ5S3fKioUrbriYsDJCfD2hnRwX+zPPojdB5civSwdUwZNwRePf4Ex\nfcbwsvZEW5exjo4uCA7uQXc2iVmgoKeLe/cUM5zcugWIRIC3NyrDg3CkIA77T7+P+Lx4TPSdiBWR\nK/BY/8daXb7ykadpr89hZ0eo6BPlrtSjeuEWBT1NyWSK1lxREVBfD/TsiTv+ffBTaTx+PPsRThec\nxlifsZgyaAp2Tt4JJ1snvkusxEWfQ0KMncnn9DjpdCuTAaWlimBXUwP07IlrdjLszjuOg1d/Rk7N\nFYQ5j8TfQ6bi72HPwdHWUT9fhgP66utHiKFRPz01dOp0e+8eUFKieNTWorGHK/5ozseh8jM4cu1n\n3K27iwCb0Yh0nYlgpwmwtehCyX9CDIiCnhoad7qtrFTceS0tBauvxzXbGsTVZuCo9A8kFZyGfw9/\nRPtFY5LfJJRfsUNdnepA/8526KU8jSqqE/WoXtTj5e5tYmIili9fjqamJixcuBAvv/yyLrvjXIed\nbuVyxWwmEglYaSkK68pwrikPcffScagsEfbWXTDBdwJigudix+Sd6N6lu3IfsZdz2t83IUSQtG7p\nyeVyDBw4EHFxcfD09MSwYcOwZ88eDBrUuvUjtJaeVW01nBvPIqyvHW7eTMVF2Q0k1Gfh1+qLYF3s\nMbr3aIzpMwaP9Huk3ZERNHSLEH4ZvKV37tw59O/fHz4+PgCA6dOn49ChQypBj0+Bge5IirsMpwZX\nVJdfxJ2y33FTdhE3XYpwLuMmnHr1RWTfURjT+3n8u8/X6N2tt0b71uedUFOf9YQQvmgd9IqKiuDt\n/ddoAi8vL5w9e5aTQuniXk0FsrLP4EbuORTlXcaN4iykNhag1rErevQMQkTAaLw0cCy+6Rmq011W\nXacqai9Pw8UU9caIclfqUb1wS+ugp8mIgpiYGGWL0NnZGcHBwcofsWWCRE2eNzc3Y8DQAci5k4Nf\njh7E7dLr6NK1CpXFN1CYewcOPXoieEwEfMIi0LdsDB529cVTjz2l/DzLY3D0cdT6+C3PxWJXZGZe\nhq2t5p9voe79CxcK4Oc3G4BislEACAwci4yMbGRmXta6vPTcOJ9funRJUOXh63l8fDy2b98OAMp4\nog2tc3rJyclYs2YNjh07BgB47733YGFhgddff731ATS87m5mzbhTewclNSUouVuC0ppSFFYXIq8y\nDwXSm5CW3MC928Xo0+yIAGtPeDt5oYeXH3r1GYK+/YbC2zsAVka+mLVJrctLiJ4YPKcXFhaG3Nxc\n5OXloVevXti7dy/27NmjdtsDmQdQ21irfNxruIfK+kpIZVJUyCpQUVeBClkFymvLUXavDE62TujV\nRQxfKzf0EbnA16I7nmIu6GXVF25BCyD2GgjbHmLA1RWwt9f2KwgWredAiP5oHfSsrKywbds2TJ48\nWdllpa2bGDsvK5YlvP/h1sUNfq5+cLF3QQ90QXe5Lbo3WaNHow1s7skUoyAcHRWD97t1A1xcFM8t\nTGNNiPbyNMY6XEzXmy/t1Yk5o3rhlk799KKiopCamtrhdgefO6joE1dbqxjpUFOjeNy9C5TVKGYQ\nduyqCGpufwa5rl0Vg/jNkDGu52CuN1+I8THMiIzjxxXTpHfpAjg4KB5d/wxyjo6AlenMe2CuXU2o\n3yIxNGHPpzdihCL3ZuItN3Nu7QhlynlCOmKYBFmXLiYf8AAgPb11wAMAe3t/ZGTcVtn2wa4rxo6L\nmy+mVidcoXrhlmncFRAIc27tGPNi2sS8mPQsK4Zm7nktmquPGBJNLSUAtGg2IYajbWyhy1sOKbqa\nuMPBIRs2NjlwcMhuM+BRnkYV1Yl6VC/cMp2+IhzgoruJEBbYIYS0jS5v/0SXpoQYF7q81ZEm3U0I\nIcaLgt6fDN3dhPI0qqhO1KN64ZZR5vT0MdSLZjYhxDwYXU5PX7k3yukRYlzMpp+ePjsAU+daQoyH\n2dzI0GfuTSx2xfjxAzFxoh/Gjx+o14BHeRpVVCfqUb1wy+iCHuXeCCG6MLrLW8q9EUIAM8rpAZR7\nI4SYWdAzBbTugSqqE/WoXtQzmxsZhBCiC0G39Mx1vQlCSMdM7vKWblgQQtpjcpe3pj4BAPW9UkV1\noh7VC7cEG/TMeb0JQoj+GPzytrN5OnNfb4IQ0j6juLxtydPJZP5oaPCDTOaPhIQySCRSlW1pdS1C\niD4YNOhpkqfTZL0JY0R5GlVUJ+pRvXDLoPPpaZqno/UmCCFcM2hOj/J0hBCuGEVOj/J0hBC+GTTo\nmXqeThOUp1FFdaIe1Qu3DL5GBuXpCCF8EuwwNEIIaY9R5PQIIYRvFPR4QnkaVVQn6lG9cIuCHiHE\nrFBOjxBilCinRwghnUBBjyeUp1FFdaIe1Qu3KOgRQsyKyeT0aD0NQsyLya2RoQlaT4MQ82PWNzKM\ncT0NytOoojpRj+qFWyYR9Gg9DUJIZ5nE5S3N00eI+THry1uap48Q0lkmEfSMcZ4+ytOoojpRj+qF\nW1oHvddeew2DBg1CaGgoli9fjqqqKi7LpTGx2BXjxw/ExIl+GD9+oKADHgBcunSJ7yIIDtWJelQv\n3NI66D366KPIyMhASkoK7t27h/fee4/Lcpm8yspKvosgOFQn6lG9cEvroDdhwgRYWFjAwsICEydO\nxK1bt7gsFyGE6AUnOb0tW7bgqaee4mJXZiMvL4/vIggO1Yl6VC/carfLyoQJE1BaWqry+tq1axEd\nHQ0AePfdd5Gamor9+/erP4CI+soRQvTD4MPQtm/fji1btuDEiROws7PTdjeEEGIwWq+GduzYMXz0\n0UdITEykgEcIMRpat/QGDBiAhoYGuLoquoYMHz4cX331FaeFI4QQrml9IyM3Nxf5+flITU3Fhg0b\nkJycjKCgIHz++edqt//3v/+NoKAgREZGIisrS+02piYxMRGhoaFt1kt8fDy6deuGkJAQhISE4J13\n3uGhlIY1b948iMViBAYGtrmNOZ4rHdWLOZ4rhYWFGDduHAICAjB27Fhs375d7XYany9MR01NTczX\n15fdvHmTNTQ0sIceeohdvXq11TY///wze/zxxxljjCUnJ7OIiAhdDyt4namXU6dOsejoaJ5KyI/E\nxER28eJFNmTIELXvm+O5wljH9WKO50pJSQlLTU1ljDF2+/ZtJhaLOYktOndZOXfuHPr37w8fHx9Y\nW1tj+vTpOHToUKttDh8+jDlz5gAAIiIiUFlZCYlEouuhBa0z9QJod/fJmI0ePRouLi5tvm+O5wrQ\ncb0A5neueHh4IDg4GADQo0cPDBs2DMXFxa220eZ80TnoFRUVwdvbW/ncy8sLRUVFHW5j6p2ZO1Mv\nIpEIv//+OwICAvDEE0/g6tWrhi6m4JjjudIZ5n6uXLt2DRkZGYiMjGz1ujbni85Br7P98B78K2Xq\n/fc68/1CQ0NRWFiIS5cu4ZlnnsGTTz5pgJIJn7mdK51hzudKTU0Npk+fjvXr18PBwUHlfU3PF52D\nnqenJwoLC5XPCwsL4eXl1e42t27dgqenp66HFrTO1IujoyO6dOkCa2trzJ8/HxUVFZBKpYYuqqCY\n47nSGeZ6rjQ2NmLKlCmYNWuW2lFf2pwvOge9sLAw5ObmIi8vDw0NDdi7d6/KX6Enn3wSO3bsAAAk\nJyfD2dkZYrFY10MLWmfqRSKRKP9KHTlyBPb29souQObKHM+VzjDHc4Uxhvnz5yMgIADLly9Xu402\n54vWnZOVO7CywrZt2zB58mQ0NTVh4cKFGDRoEDZv3gwAePHFF/HEE08gMTERgYGBcHBwwP/+9z9d\nDyt4namX/fv3Y+PGjbCyskJQUJDaGx2mZsaMGUhISEB5eTm8vb3x1ltvobGxEYD5nitAx/VijufK\nmTNnsGvXLgQFBSEkJASAYghsQUEBAO3PF71PF08IIUJiEjMnE0JIZ1HQI4SYFQp6hBCzQkGPEGJW\nKOgRQswKBT1CiFn5fwnto9Nr1Ln9AAAAAElFTkSuQmCC\n", 62 | "text": [ 63 | "" 64 | ] 65 | } 66 | ], 67 | "prompt_number": 69 68 | }, 69 | { 70 | "cell_type": "code", 71 | "collapsed": false, 72 | "input": [ 73 | "x = np.random.rand(50)*2\n", 74 | "x.sort() # sort for plotting\n", 75 | " \n", 76 | "y = x + x**2 + np.random.randn(len(x))\n", 77 | "\n", 78 | "V = matrix(np.vstack([ones(x.shape),x,x**2]).T)\n", 79 | "Pv = V*inv(V.T*V+eye(3)*4)*V.T\n", 80 | "\n", 81 | "p=np.polyfit(x,y,2)\n", 82 | "\n", 83 | "fig, ax = subplots()\n", 84 | "fig.set_size_inches(5,5)\n", 85 | "\n", 86 | "ax.plot(x,y,'o',label='data',alpha=0.3)\n", 87 | "ax.plot(x,np.dot(Pv,y).flat,'-s',label='projection')\n", 88 | "#ax.plot(x,np.polyval(p,x),'s',label='polyfit')\n", 89 | "ax.grid()\n", 90 | "ax.legend(loc=0);" 91 | ], 92 | "language": "python", 93 | "metadata": {}, 94 | "outputs": [ 95 | { 96 | "output_type": "display_data", 97 | "png": "iVBORw0KGgoAAAANSUhEUgAAAT0AAAE1CAYAAACVyDfwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtYVPW+P/D3AIPAaAymDIoo3hDFQUAUM2mQTNPELbrz\n0kXRQjvWTrJ++Ttlp3bbardPlyfrZLWrTZnbsjpR6jaSvZshPAfBFEUERhMQQUYUQS7DfZ0/cEaG\nWcBc1sxaM+vzeh6eWmsWa335sPzwXd/bkjAMw4AQQkTCg+8CEEKIM1HSI4SICiU9QoioUNIjhIgK\nJT1CiKhQ0iOEiIpdSe+vf/0r5s6di5kzZyItLY2rMhFCiMPYnPTq6urw6quv4siRI8jPz4dWq0Vm\nZiaXZSOEEM552fqNvr6+YBgGDQ0NAICWlhYEBARwVjBCCHEEm2t6vr6+2L17N0JDQxEUFIQ777wT\ns2fP5rJshBDCPcZGV65cYcaNG8ecO3eOuXr1KjN//nzm4MGDZscBoC/6oi/6csiXLWyu6eXl5WHO\nnDmYNGkSbr/9dtx///3Izs5mPZZhGPrq8/Xiiy/yXgahfVFMKC7WfNnK5qQXHx+P48ePo66uDm1t\nbTh8+DAWLlxoc0HEpry8nO8iCA7FhB3FhVs2d2Tcdttt2LFjB5KTk9HS0oJ7770X8+fP57JshBDC\nOQljTz3RkgtIJHZVRd2VWq1GQkIC38UQFIoJO4oLO1tzCyU9QohLsjW38DYNbfjw4ZBIJPQloK/h\nw4fzdTsA6KnREHMUF27Z3KZnr+vXr1MNUGAkEgnfRSDE4Xh7vKXHXuGh3wlxJS73eEsIIXygpGeB\nlJQUvPDCC3wXw+1R2xU7igu3KOlZwNDQP5iEhAR88sknTigRIcRWlPQsZEnbAXUE2IfGorGjuHBL\ncElPp6tDVlYJMjO1yMoqgU5X5/RznDx5EjExMVAoFEhNTUVnZycAoL6+HkuXLkVgYCACAgKQlJSE\nqqoqAMDzzz+PX375BU888QSGDRuGJ598EgCwdetWjB07FrfddhtiY2ORk5Nj9c9DCOGOoJKeTlcH\njeYK9PpwtLeHQa8Ph0ZzxaqkZe852tvbsXz5cjz44IOorq7GPffcg/3790MikaC7uxuPPPIILl68\niOPHj6OjowNPPPEEAOCVV15BfHw8/uu//guNjY3YtWsXAGD27Nk4deoULl68iMTERNx///1oa2uz\nPjgiQG1X7IQcFy4qKbZez1aCSnqFhVfg6xtuss/XNxxFRbVOO0dubi50Oh0ef/xxeHp6YtWqVVAo\nFAB6BlQnJyfDx8cHEydOxDPPPAONRmPy/X0fgx988EEEBARALpfjhRdeQENDA7Ra239hhAgFF5UU\ne65nK0Elva4u9uJ0dFjeVmbvOaqrqzF58mT4+PgY98XExAAA9Ho9Nm/ejNDQUPj7+2PlypVoaGgw\nSXR92/XeeOMNTJs2DXK5HGPHjkVbWxuuXr1q8c8jJtR2xU6oceGikmLv9WwhqKTn6dnNul8qtXwA\nor3nGDVqFM6dOwe9Xm/cd+LECTAMgzfeeAO5ubk4duwYGhoa8O2335qs7dV3sOQvv/yCF198EV98\n8QXq6+tx8eJF4zL7hLg6LiopXFzPWoJKekplIPT6EpN9en0JIiJGOu0cc+fOhUKhwO7du9HR0YGv\nv/4aOp0OADB06FDI5XIMGTIEZ8+exeuvv27yvQqFwpggAcDPzw/e3t7w9/eHTqfDc889R+15AxBy\n2xWfhBoXLiopXFzPWoJKegrFcKhUgZDJSuHtrYVMVgqVKhAKheUT4e09h1QqxX//93/jiy++QHBw\nMH766SesXr0aEokEGzduRHBwMMLCwvDwww9j48aNJo+zW7duRVZWFuRyOdLS0jBz5kxs2bIFiYmJ\nuOuuuzB9+nSEhIRYHRdChIiLSoq917MFzb0lRvQ7IdbS6epQVFSLjg4JpFIGEREjraqk2HO9e++d\n4lrr6dE/MOGh3wlxJbTgAHF5Qm274hvFhVuU9AghokKPt8SIfifEldDjLSGEWICSHhEMartiR3Hh\nll1Jr7S0FNHR0cYvf39/40R7QggRIs7a9Lq7uxEcHIy8vDyTAbjUpuc66HdCXAnvbXpZWVmYOHEi\nzTi46ZdffkF4uP2To511XkLEgrOk9+WXX+KBBx7g6nQuLz4+HiUl9k+Z8fDwwIULFzg/rxBR2xU7\nigu3OHnvbXt7Ow4cOGA2Ad8gJSUFoaGhAAC5XI6oqCguLsurzs5OeHk557XBznzkNPwDMyxnRNv8\nbxcUFAiqPHxtq9VqpKenA4Axn9iE4UBGRgazaNEi1s/6uwTb/tRnUxnVepXZV+qzqRaXxd5zjBs3\njnn33XeZuLg45vbbb2c2bNjAtLa2Mj///DMTHBzM7N69m5k8eTKzbt06pq2tjdm6dSszevRoZvTo\n0UxaWhrT1tbGMAzD/Pzzz8yYMWOM562qqmJWrFjBjBgxggkNDWV27dpl/Kyrq4t55ZVXmIkTJzLD\nhg1jYmNjmcrKSiY+Pp6RSCSMTCZjhg4dyuzfv9/svGfPnmVUKhUjl8uZiIgI5ocffjB+tn79euap\np55iVq9ezQQEBDBxcXHMb7/91u/PztHtQIjNamquMUeOFDM//ljKHDlSzNTUXOv3WFvvV06qKvv2\n7cPatWvtPo9Wp4VmvMb8gzLnnuODDz5AZmYm/Pz8sGTJEuzcuRMLFiyATqdDfn4+srOzIZfLsXPn\nThw9ehSnTp0CACxduhQ7d+7Eyy+/bHK+7u5uJCUlYd68eThz5gy0Wi3WrFmDKVOmYOHChXjzzTex\nZ88eHD58GJMnT8bp06fh5+eH7OxseHh44PTp05gwYQIA00edjo4OJCUlYcOGDcjKysIvv/yC++67\nDwUFBQgL61lZ9pNPPkF6ejree+89bNiwAc8//zz27dtneTAIcRLDysi9FwrVaEqgUoHTRQzsbtNr\nbm5GVlYWVqxYwUV5eCeRSLBixQoEBwcjICAAGzZsMCaJrq4uvPTSSwgKCoKPjw/27t2L1NRUjBgx\nAiNGjMCjjz6KPXv2mJ0zPz8fFy5cwJ///GcoFArEx8fj/vvvx5dffgkA+Pjjj7F582ZMnjwZABAZ\nGYnhwwf/Jefm5uLixYtIS0uDl5cX5s+fj5iYGJOkplKpkJycjBEjRuDBBx9EQUEBF2FyCGq7YieW\nuDhrJWa7a3oymczhy59rKjSQ/NHC1VgrAIy373q92xyjo6NRXV0NoGeR0N6905cvX8bMmTON2zNn\nzjQea1Kkigo0NTVh9OjRxn1dXV246667AACXLl3CnXfeaXU5q6urERYWBplMZtwXGxtrLINEIjH5\nWYKCgtDU1GT1dQhxBmetxOyclng7qcapoH5RbdGxCWUJ0IDl8dYKJ0+eNNZcT5w4geDgYAAw67gY\nPXo0jh8/bkx8x48fNx7bW0hICIYNG4aamhp4e3uzfp6Tk4NZs2axlofppyNj9OjR0Gq1aG5uNia+\n/Px8LFq0aNDvFSKhvguCb2KJi7NWYqZpaH0wDIOMjAxUVVWhrq4O6enpWL16Neuxa9euxaeffoqr\nV6/i6tWr+PTTT/HQQw+ZHTd79myMHz8ezz33HMrLy9HV1YUzZ87g+PHjAIBHH30UH374IQ4fPozO\nzk6cPn0adXU9b5RSKBTG4/qKi4tDSEgI3nnnHXR0dECtVuPkyZNYs2aN8WchxFU4ayVmQdX0whRh\nrB0OYQrLX/dm7zkkEgkee+wxrFixAufPn8eyZcuwY8cO5Obmmr3pbMeOHbhx4wYiIyMBAKtWrcKO\nHTvMzunp6YmDBw/i6aefxpw5c9De3o7w8HDs3LkTALBt2za0tbXh8ccfR21tLaZNm4bvvvsOAPDS\nSy9h69at2Lx5Mz7++GOMHDnSWA5vb28cOHAAW7ZswV/+8heMGTMG+/btM3ZiSCQSszL33RYStVot\nmlqNNcQSl55XPQBFRaXGlZhnz7budRGWoKWl+hg/fjw++eQTJCYm2nWef/3rX0hNTcVvv/3GUckc\nj+/fiVj+cVuL4sLO1vtVUDU9d8EwDPLz843DTIhl6B82O3eOi05Xh8LCK+jq8oCnZzeUSu5rdn1R\n0nOArVu3ori4GK+++irfRSFEsJw1Lq8verwlRnz/Tugxjp27xiUrqwR6vfniGTJZKRITpwz6/byv\nskIIIdZw1ri8vijpEcFwx9oMF9w1Ls4al9cXJT1CCC+cNS6vL0p6RDDEMsfUWu4al55xeYGQyUrh\n7a2FTFYKlcqNe28DAgIEPVBWjAICAvguAhEZhWK4w5NcX7z13hJCiD2o95YQQixASY8n7tpOYw+K\nCTuKC7co6RFCRIXa9AghLona9AghxAKU9HhC7TTmKCbsKC7coqRHCBEVatMjhLgkatMjhBAL2JX0\nmpubsX79ekRHR2PatGnIzc3lqlxuj9ppzFFM2FFcuGXX3NstW7ZApVLhs88+Q2dnJ5qbm7kqFyGE\nOITNbXoNDQ2Ijo7GhQsXBr4AtekRQhzA6W16ZWVlGDlyJFJSUjB9+nSkpqZCr9fbejpCCHEKmx9v\nOzs7kZ+fjx07dmD37t3YvHkzvv76a6xbt87s2JSUFISGhgIA5HI5oqKijKvBGtorxLZt2CeU8ghh\nu29s+C6PULYLCgqQlpYmmPLwta1Wq5Geng4AxnxiC5sfb2tqaqBUKlFbWwsAOHz4MD7//HPs27fP\n9AL0eMtK7aYve7EHxYQdxaXHpu2boNVpjduazzTOfe9tUFAQJk2ahGPHjmHWrFk4dOgQFixYYOvp\nRIduYnMUE3YUlx5anRaa8Rq7z2PXkJXPPvsMW7duRVhYGKqqqrBmzRq7C0QIIY5EMzJ4Qo8s5igm\n7MQcl96PtAUlBWhY3HDrw5fg3MdbQghxpE3bN2G/Zv+tRFfBzXmppkcIEaSElARoKjRAws0datz6\nf4BqeoQQN+eFnsQHwF/vjwY0DHR0v2jBAZ70HpNGelBM2FFcbpqHnppeAhAVHmXzaSjpEUJEhdr0\nCCGC0Hfwcf7ZfLT4twCdpsf56/2xSrUKf/3LX23KLZT0CCGCkJCSYDr4OAdAZ0+S6/04G6YIw0ev\nf2RzbqGODJ6IeexVfygm7EQbl3k9/4kqi4I6Xc3ZaalNjxAiKpT0eCLKv9yDoJiwE0tc2jrbnHId\nerwlhPCid8dFR3cH8s7mAZMdf11KejwRbTvNACgm7Nw1LmarplwGoGbvuOASJT1CiDA4qOOiL2rT\n44k7/uW2F8WEnbvGhQE/Q9ko6RFCnK6b6Ubp1VJerk2Ptzxx13Yae1BM2AktLjpdHQoLr6CrywOe\nnt1QKgOhUAzv9/i+My0A4FzdOdTV1HF+LUtQ0iOEWEynq4NGcwW+vuHGfRpNCVQq9JuMWJd5Hw8o\nahQILws3O97QcWHLtSxB09AIIRbLyiqBXm+eqGSyUiQmTjHbr9PVYeGme3E6Jt/sM1WZasAOi8Gu\nRdPQCCEO19XF3g3Q0SEx2/fwk+tx4rdiVOi0QIxjr2UN6sjgCa2RZo5iwk5IcfH07GbdL5Wa17hO\nVxTj7Ox8NA+zbbFPa65lDUp6hBCLKZWB0OtLTPbp9SWIiBjJcrR9NTLrrmU5atMjhFhFp6tDUVEt\nOjokkEoZRESMZO1YmPG72T1teTeXiDKQNQ5D7PQY4xJRtl7L1txid9ILDQ3FbbfdBk9PT0ilUuTl\n5ZlegJIeIaI0/ffTUaQsMts/V3snju7Nsfv8tuYWux9vJRIJ1Go1Tp48aZbwSP+E1E4jFBQTdq4Y\nl5yLOdBe17J+JpXy23/KydWpJkcIMcivyseKr1ZgQdgCtJS1mH3O9QIC1rL78XbChAkYNmwYPDw8\nsGXLFqSmpppegB5vCRGNUzWnsPCLhfg46WMkTUly6LV4G6d39OhRjBo1CsXFxViyZAnCw8MRHx9v\nckxKSgpCQ0MBAHK5HFFRUcZpNYaqO23TNm271vam7ZuQd6qnSUseJEdLRwtOlpzEjFEzkPRMEufX\nU6vVSE9PBwBjPrEFp72327ZtQ3BwMJ5++ulbF6CaHiu1wOZTCgHFhJ0Q47Jp+ybs1+xHw2LzMXiD\nzbSwVn/zb3npyGhpaUFjYyMAoLa2Fv/4xz+gVCrtOSUhxAVodVo0+No26Ngahvm3en042tvDoNeH\nQ6O5Ap1u8MUK+mPX461Op0NycjIA4Pbbb8dTTz2FhQsX2nNK0RDaX24hoJiwE1JcDCumFJQUAL6O\nv15hoemCAwDg6xuOoiLbl6WyK+mNHz8eBQUF9pyCEJs4YskhMjjjiikVzrmeI+bf0jQ0nhgaaMkt\nlsbEEY88Qibme8UR828p6RGX0/8jTy1PJRIpLwBq0y//w/6cjsNzxPxbWlqKJ0JqpxEKS2PiqCWH\nhEpI90p7V3vP/3jBZD6t4Q1mlsyntYZCMRwqFVBUVGqcfzt7tn1NGZT0iMtx1JJDxFTfZd7bu9px\nrOgYMAnGN5cZOPINZgrFcE7baynp8USIY6/4ZmlMlMpAaDQlJo+4en0JZs8OdGDp+MPXvcK6zHtN\nzyNs7/fSAvxPLbMGJT3ichzxyEMsNM/x76V1NEp6PKFanjlrYsL1I49QsA3FoXuFW9R7S4hACG0o\nTltnGy/XdTRKejwR89ir/og9Jv0Nxdm795DTy3Lh+gWcrDnp9Os6Az3eEiIQ/Q3F6epybjm017RY\n8PkCzBo7C55lnmafu1KnBRt6RwYhAmHtO2Ud4WztWdyz5x78af6fsDF6o1OuaSvelosnhHDDUW//\nstRp3Wks+HwBXl/wuuATnj0o6fFE7O1XbMQek56hOIGQyUrh7a2FTFYKlSoQxcWnHX7tE5dPYOGe\nhXh70dt4KPIhh1+PT9SmR4iAsA3FKS527DXzqvKQtC8JH9z3AZKnJjv2YgJAbXqEiEjfqWU32m7g\nzJUzSJyciB8/+JHHklmPt3dkEEJcB+vUsnCgtayVnwLxgNr0eCL29is2YouJTleHrKwSZGZqkZVV\n0u8gZLHFxdGopkcIDwyzL3oPRtZoSqBSgfPpdb0faQtKCoDxnJ7eIkJa6ZqSHk9oPqU5McVkoHc/\n9E0G9sbF5JHWScu89+bMBG8JerwlhAdiWghVaCtdU9LjCbXTmBNTTKxZCJXTuPRa4t3/sD9UZSqo\nylQOnVomtARPj7fE5QipfchWzloIlWEYXLh+4VY7Xq8Vj8edmIIdD33g8PgJbaVru2t6XV1diI6O\nRlJSEhflEQ0xtV9ZypKYCG35JVv1N/uCLfnYeq90dnfi0QOPor61nvXz7m5fp8SP7+l1fdld03vn\nnXcwbdo0NDY2clEeQgZkTQeA0DlyIdTWzlas/XYtWjpasFK5EmVlZQCA69db0N3tBwAYLet5pHV0\n/IS20rVdSe/SpUv4xz/+geeffx5vvfUWV2USBXpHhjlLYiK09iFnsPZeudF2A8u/XI6RspH46vdf\nwfshb+NnmZlatLebt9/ZEj9rmhmEtNK1XY+3Tz31FP7zP/8THh7UH0KcQ2jtQ0JzpfkK5n82H1NG\nTMHfV/wd3p7eJp9zFT9XbmawuaZ38OBBBAYGIjo6etDepZSUFISGhgIA5HI5oqKijH+5DN9L27Sd\nkJAw6PGNjVoUFOQhNnYdAKCwUI22tovYsmUp7+Vn2/7uux9w4UI9pk+fA0/PbjQ2ahEQcJvV5zMY\n6PiK+grMe3Ee5ofOx/tL3odEInFY/Do7g+DrG47Cwp5tpTLh5irPexATE+KQeKrVaqSnpwOAMZ/Y\nwuYFB5577jns2bMHXl5eaG1txY0bN7By5Up8/vnnphegBQcIx3S6OhQV1RrbhyIiRgrm0ak3tkG5\nen1Jvx0Wluq7aAAAtHS04Oz1s9j5HzuRNidt0HLZG7/+HpO9vbVYtMg5Kyvbmls4WWVFo9HgjTfe\nwIEDBzgrmLujNj1z7hYTrlZC7huXhJQE80UDAISfCUfx1w5eh+omIazyzPvKyRKJ+zYkE2ILZ3e6\nKGQKh5yXjdCGoViDk8HJKpUKKpWKi1OJhjvVaLjibjHhqtNAiHER2jAUa9CMDEIchMtZF0JYKaUv\nIQ1DsQYlPZ64W/sVF9wtJlzVhtRqNW8rpbjDlL++KOkR4kAOqQ0ZFg0A4K/3R1R4FADu30crtCWh\nuELvyCDEBcSvi0fOxByz/aoyFdTpaodcUwg9tAOhd2QQ4ib6jsNr62pDXlEeMNG55XDXKX+U9Hji\nbu1XXKCY9DB7eU85AH/nl8Ndp/xR0iNEIAw1PNbeWa+eRT8N7XcGjlz801lr/jkbtekRIhDGmRZq\nAAnmnzuy/a4/Qp7yR216hBDOuepYvIHQmlA86bt6BqGY9FtrKXdqMdwe1fQI4UF4QjhqWmuM2wwY\n3Gi+AUyAyTg8AJDpZIhlYh3aficm1KZHCA/kc+RoWNxgulMNwbTluQJq0yPE1XkBnt96Yl7MPJPd\nVMPjFiU9ntCYNHOij8k8YOjhoWa1OrG3dXKNOjIIcbKu7i60drbyXQzRojY9QpzoRtsNrP12LX56\n8yd0Lus0+9z/sD/qc9nfU0tMUZseIQLTdw5ta2crCq8UYtzt4zBx2ETUHK4x+54gnyCLzu2OSz45\nCyU9noi+/YqFu8XEbA4tAEwGAssCrWqn6xsXd13yyVmoTY8QF1NYaJrwAMDXNxxFRbU8lci1UNLj\niTvVaLjibjHpZthXKbFW37i465JPzkJJjxAHqKivQEFNgUPO7a5LPjkLJT2e0Ngrc+4Sk6wLWYj7\nOA6BMm6WYOobF1d+/aIQ2NWR0draCpVKhba2Nvj4+GD16tV46qmnuCobIS6FYRi8fvR17Dq2C/tW\n7sO+2n0YUzbG7Dh7Z1i48usXhcDucXotLS3w8/NDW1sbZs6ciYyMDEyaNOnWBWicHhGBG203sD5j\nPS43XsY3q77BmNvMkx3hFm/j9Pz8/AAATU1N6OzsxJAhQ+w9JSE242P82tnas0j+Khl3j78bX678\nEkO8hPdvgMb13WJ3m153dzdmzJgBhUKBJ554AiEhIVyUy+25S/sVl+yNiWH8ml4fjvb2MOj14dBo\nrkCnq+OmgCz2F+1HQnoCnpv3HN6/732HJDxXjIuQ2V3T8/DwwKlTp1BeXo4lS5bgzjvvRHR0tMkx\nKSkpCA0NBQDI5XJERUUZu+ENv1CxbRsIpTzusF1YeAXnz9cAqIFS2fP5+fM1uHQpH9u2Pczp9ebd\nNQ/bs7bj7wf+jlcSXsH6qPUO+/kKCgrs+v5ff72IsLB1AIDCwp7PlcoEFBWVorj4NOflddS2Wq1G\neno6ABjziS04nXv7zDPPYMyYMUhLS7t1AWrTI06SmalFe7t5J4G3txaLFnG3PJOuSYfV36yGr9QX\ne1fsxXBfYT8mOisuzmZrbrHr8fbq1auor++ZHH3t2jUcPnwYSqXSnlMSF6fT1SErqwSZmVpkZZU4\n9RHKGePXci/lIvavsbhr3F04uPag4BMeQOP6+rIr6V2+fBmJiYmYMWMGHnjgAWzbtg133303V2Vz\na30fc92BvW1H9sbEkePXGIbB+/nv43df/g6779uNl+e/DE8PT7vPawkhx8UV2dWmp1QqceLECa7K\nQlxc/3NCS53SU+io8Wv6Dj0eO/QYTl4+iaMbj2LS8EmDf5OA0Lg+U7SeHuGMO7YdXbh+ASv3r8S0\nkdPw0dKPIPOW8V0kchMvbXqE9OZubUeHzx3GHZ/cgY1RG/FF8heU8NwErafHE7WbrR0H9LQdaTQl\nJo+4en0JZs+2bA6qs2NiGLD71t9eRFVTGYYO9YZU2vNPoqKhAldaryDzw0zMGztvkDM5ljveK3yi\npEc440ptR70X4qy8ocOZ2HyzY+44fwfvCY9wj9r0iCjd99AqXGy4AgC4UH0SLctumB0zV3snju7N\ncXbRiIXoHRmE9DLYXNNLjRU4E5vXs6FmP0dzc4fjC0qcjjoyeOKO4/TsxVVMLBsvOHgNQSgPKHSv\ncIuSHnE7lrxDYuhQ70HPI6HV190SPd7yhHrjzHEVk8HeIdHR1YGq5ku3PvCC8RFX1uiP8aOi0N3d\nAuXEcLNz8IHuFW5R0iNuZ6DxgheuX8AD3z6AxvbGWx/06qCdeGIK3t76ISIiRgqy15nYjx5veULt\nNOa4ikl/c02LpT8j7uM4rJm+Biumr4CqTGX2FRc+A4mJUwSV8Ohe4RbV9IjbeeGt/4+iS2fR3NwB\nhgG60YWajovo8GiH+mM1ooKigDl8l5LwhcbpEbeg09Vh49OP4VJjBX6rKkZzUqPZMfEX4pH9WTYP\npSOOQOP0iGg9/OR6nPitGBU6LZqTGgDzccYAAA8JteYQatPjDbXTmLM1JqcrinF2dj6ahzVwWyCB\noHuFW5T0iBugAXXEcvR4yxMae2XO0phs2r4JWp3WuP1bVTEQ46BCCQDdK9yipEdcyqbtm7Bfsx8N\ni3s9ylb0OajXYGMA8Nf7Iyo8CmEK11zI1IDeXcsNerzlCbXTmLMkJlqdFg2+/bTdGZJdZ8+mv94f\nqnEqrFKtgjpdjY9e/4ijkjqXWq2md9dyiGp6xPXdTHaGGp1BmCLMZRNdX3y/f8SdUNLjCbXTmLMk\nJm2dbeY7b04jiyqLgjpdzWmZhCAhIQGZmVrWzwzziYnlKOkRl9DV3YX38t7D8cvH+S4KL9zt/SN8\nsivpVVZWYt26dbhy5QpGjhyJlJQUpKSkcFQ092brew/cuTG7v5gU6grx6IFH4evli5hRMcjT5Zkt\n/Omv90eYyrU6Kiz9XarVaiiVkXa9f4TcYlfSk0qlePvttxEVFYWrV69i+vTpiIuLw9SpU7kqH+ml\n93sdDDSaEqhUEGzisydJt3a2Ymf2Tnz464d4NfFVPBLzCB4rfgy+Xr5mx7pa+521v0tXev+I0HE6\n9zYpKQlpaWm4++67b12A5t5yJiurBHq9+RpvMlkpEhOn8FCigbH9w9brS6BSDf6PNbsiG6kHUqEM\nVOLdxe9i1LBRji6uU7na71KIeJ97e/78eRQVFWHOHFq+wlEGWxxTaGzpcaxvrcf2rO04pD2E95a8\nh+Xhy506CQZqAAASa0lEQVRRVKdztd+lO+Ek6TU1NWHNmjV4++23IZOZvxA5JSUFoaGhAAC5XI6o\nqChj241hbJbYtg37rPl+T89uFBb2bCuVPZ8XFqrh41OJRYvCBPXzJSQkoKvLg7W8UmmlsTbT+/g/\nffYn7Dq2C3PHzkVRWhH8ffwF9fNwue3pGWSMR+/4FBUdw5Ah1SbHFxQUIC0tTVDl52NbrVYjPT0d\nAIz5xBZ2P952dHRg6dKlWLx4sfEXY3IBerxl1V+j/UDseVy09PxcdpJY+ghX3ViNPxz+A/KP5mPv\ntr2IHxdv8zVdhTW/S1vuFTGwNbfYlfQYhsH69esxYsQIvPXWW5wWjLDT6epQVFRrbMzmallzRyTU\nvud8b+8mXGosgFzuDam05yHjctNlVDRV4Nntz+K5+Ofg4+Vj98/iKhz1uxQLXpJeTk4O7rrrLkRG\nRkJy89VRr732Gu699167C0acy1EN673/YW/f/TBOReeZHROrjUX+3nybr+GO3HloEld46ciYN28e\nurvZB02SgQntkcURDet9V0MpryllPU4m7WkHFlpM+NK3hlxYqEZdHQQ9NMmV0IwMAsAxI/61Oi00\n4zW3dvRdDYWwonm2jkWrrPBEaDWa/t4gFhEx0uJzbNq+CaPiRkE+Rw75HDlyTuRYVQahxYQvfWvd\nhp5dGs7CDarpEQDcjPjX6rSo8asBEm7uUDuipO6P5tk6FiU9ngix/UqhGO7Yx6cBloAChBkTPiiV\ngSbzbAsL1Zg0KYjm2XKEkh6x2abtm3BAfQB6iR4A0NTSBNw+wDe4+RJQXOlb6/bxqYRKFUnteRyh\n994SmyWkJEBToTF/nDVs58C4irGscRhip/e8yMLVFgcgwsT73FtCzMy79b8TT0yl2h0RBOq95Unv\nObiuqJvphq5ZZ7rTC8AVABmAxzeekB3wh+yAPwIOjcKYYaGDntPVY+IoFBduUU2PWO2n337Cs0ee\nRdWNKtMPetXsxmlm4Z2n/hfArelshAgBtemJnDXTnU5ePolns55FRX0FXrv7Nbz7l3dN2/R6mZAd\nifef/ZrmlBKHcdk2PWfMMaR5jOwsXb23vL4cO/61A/8s+yf+467/wKMxj0LqKUWmIhOFBUVoO9Bh\ncl5vDz+Eh4Ubl7siREh4rek5eqkkZ13DFkIYk9Z7kYH39m5CdXPPPFlPTz3kcl90dneioasB1TOq\n8YfZf8DTdzyNYUOGmZyDy/gKISZCRHFh55I1PWfMMaR5jP17628vIrcoG+1eeujbmtD9+y6zY0b/\nOhpFW4oQNDSI9Rz07gbianhNes5YMluoy3IL4S93VVMZrgfcnDamZj9m8vDJ/SY8A65mcgghJkJE\nceEWr0NWnDHHkOYx9m/oUG++i0CI0/Ga9LhY2UMI17CFs8debdq+CQkpCSZfRb+dcWoZBkPj0dhR\nXLjF6+OtM9qDqM2ph9nadgCtb0dEifchKw5f2cNJ17CW4Nppbq6AYmBYCcWwAoozCC4mAkFx4Rbv\nSY84XmtnK6obq80/8AKkF6XwO+xnstvP2w9LVUtpUQDilmjuLU+c0U5zo+0GXs95HePfGY9r+mvm\nB8wD5t4xF/W59SZf1dnVvCQ8artiR3HhFtX0XFTfl+4YhCnC8KcX/oR3jr2Dj379CPdOuheZD2Xi\nycInoYGG5Uz2odkuxNXQ3FsXlZCSYN4xgZ7BxPo79VirXIun73gaEwImABg4SdpaqxPqbBciDry8\n93bjxo04dOgQAgMDUVhYyGnB7OWONZDeiaugpAANixvMjhl7cizyvsiDYqjC4eVx1LtyCbGErbnF\nrja9DRs24Mcff7TnFA5hqIHo9eFobw+DXh8OjeYKdLo6votmZEs7jWHYiWa8Bg2+5gkPAMbIQvpN\neDpdHbKySpCZqUVWVond8eB6tgu1XbGjuHDLrqQXHx+PgIAArsrCmf7n29byVCLnaW7uYN3viD8E\nNNuFuCK37L0V6nzb3qwde1VcW4zzdecHPa6/2r4j/hBwPduFxqOxo7hwyym9tykpKQgNDQUAyOVy\nREVFGX+Rhqo7l9tnzlxEWFjPoNrCwp7PlcoESKWMQ67nqO32rna8sucVfF/yPXQjdRjiMQQoRw/D\nYOJ6wKdFhklhsQAA31aJyVJEhvN1dY02iwcA/PprHjw8LttUPoViOKTSHJw7l4+IiDhIpQw6O7Uo\nLq6BQsF//GjbvbbVajXS09MBwJhPbGF37215eTmSkpIE1ZHhCr2KvRNTXxcbLuKjXz/CJyc/QfiI\ncPxb7L9hefhyLHxkIWuP7fTjKry6ST3gz+gKnQ4DxUTMKC7sXHI9PUcR8nxbQw9sfU095Oly4/7J\ngZOxctNKvJ//Po5WHsWDygfxz3X/xLSR04zHhCnCgLJb5+ro6ERzcwdG36aATFY64M/Y9wXSQM8f\nAnqBNBEbu2p6a9euhUajwbVr1xAYGIiXX34ZGzZsML0AjdMz0d/4Op8cH0z9/VRsmbUFa6evhcxb\nxvm1dbo6FBXVGv8Q0LsriCvjZZyeRRcQadLrbzCw9oIWl+++bLY/piQGx/9+HBKJcDpbCBEyerwV\nGNalnAD4FPn0/E85gNBb+4cNGSb6hEdtV+woLtxyyyErQtbNsI9tI4Q4ByU9B7jWcg26Zh3rZ75e\nvj3/E+q88rgKqs2wo7hwy+0eb/mac1tRX4HvS79HRkkGfr38K7xb2N8/4cv4Iqosymy/MxfrJETM\n3Kojw5nj8xiGwWndaWSUZCCjNAOXblxCUlgSlocvx4IJC7AkdQlrm56qTAV1upraaVhQTNhRXNhR\nRwYc/47bzu5OHL14FBmlGcgoyYAEEiRPTcaue3dhbshceHp4Go/tO6bOZD8hhDduVdPLzNSivd08\nqXh7a7FokW3JpqWjBUd+O4KM0gwc1B7EWP+xWD5lOZaHL8f0wOmi73ElhC9U0wN3q35ca7mGg9qD\nyCjNwL/K/oXY0bFYPmU5/pjwR4z1H8tFUQkhPHGrml5/bXoHcl5D5Q3z9x32XjW4vL4c35d8j4zS\nDJy4fAILJizA8inLcV/YfRjuy31HCLXTmKOYsKO4sKOaHm7Nud349GpcaiyHRAJcrqrAtbar6FrZ\nZXZ8U2kTXlK/hIySDFQ3ViNpShK2zdmGBRMWwFfqy8NPQAhxNF5relwOL9m0fRMOqA9AL9GjqaXp\nVpJT3zwgwfx7fHJ88Pj/exzLw5fjjjF3mHREEEKEzeVqemyPohpNCVQq2JT4tDotavxqepKb2rLv\niQuOwxsL37D6WpZyx/d0EOLqeJuR4e5Lug+2PLthcURyC8WEHcWFW7zV9NiWdH9v7ybU6E8h4HPT\n9jS21xS2draioKYAeVV5yK/OR15VnkPLay1HjxkkhNiGt6S3a88O/O+ZX9DupUfbjRZ0D+lGN9MF\n3M9ycBlQqCs0Jrf86nwU1xYjfEQ4ZgXPQsK4BJQEluB49XH2ixmWVu/FX++PMJXjBgoP9p4O6o0z\nRzFhR3HhFm9Jr56pxvWAXm1whv+yyLmYg99//XvMGj0Ls4NnY/2M9YgKijLpYd0j3XPrG3onuSuA\np6cnhvoNhZ+3H8Im9CQ6e15ybQl6UxghwsRb0pNKLb/0nDFzkPNEzoDHhCnCUFpcCv1hvcl+v9v9\nsPSOpQ5NcGwGW55drGOvBurcEWtMBkNx4ZZLjNPz8hi8mM5OaoMR8ns6+MJ1jz0htnCJpOeqFIrh\n/f5jFtJfbmcNrRmsc0dIMRESigu3hJX0+nQ4+Ov9ERUeRSuTOJAza1+u8BJ24v54G6cXpghDUEsQ\n/A/7Q1othee3nvDUecL7ujdGdY2CapwKq1SroE5XC+7RlQtCGXvlzPGSg3XuCCUmQkNx4RZvNT13\nTGSuyJm1L3r3LhECu2p62dnZiImJQWRkJN59912uyiQKQmmncebQmp7OnUDIZKXw9tZCJis1WdVa\nKDERGooLt2xecKCrqwtTpkxBVlYWgoODMWvWLOzbtw9Tp041vYBI33vrKpy5xD4hXLI1t9hc08vL\ny8OkSZMQGhoKqVSKNWvW4Pvvv7f1dKIjlHaawWpfziSUmAgNxYVbNrfpVVVVISQkxLg9ZswYHDt2\nzOaC0Iok/BloaA0h7sbmpGfNuyFSUlIQGhoKAJDL5YiKijK2U6jValy/fgMdHWHw9Q1HYaEaAFBX\nB6hUQHHxaQAwOZ623XM7ISFBUOUR0raBUMrDx7ZarUZ6ejoAGPOJLWxu08vNzcVLL72EH3/8EQDw\n2muvwcPDA9u3bze9gAXP3VlZJdDrw832y2SlSEycYkvxCCFuzulterGxsTh37hzKy8vR3t6Or776\nCsuWLbPpXGIctNr3LzihmPSH4sItmx9vvby88OmnnyI5ORmdnZ1ITU0167m1FK1IQghxFkG8DY2G\nTRBCrGXr460gkh7Qk/iKimqNK5JERIykhEcI6ZfLJz2xUdMaaWYoJuwoLuxc7m1ozkbjAAkhgEhq\netRmSIj7cfqQFVfi7q+bJIRYzq2Snk5Xh6ysEmRmapGVVWJ8x6wQxwHS2CtzFBN2FBduuU2b3kAr\nANM4QEKIgdu06Q00lS0iYiS16RHiZkTfezvQIyy9mYwQYuA2bXqDPcIqFMORmDgFixaFITFxCu8J\nj9ppzFFM2FFcuOU2SU+pDIReX2KyT68vQUTESJ5KRAgRIrdp0wNoKhshYkLT0AghokKDk10MtdOY\no5iwo7hwS3C9tzRHlhDiSIJ6vKU5soQQS7nFOL3+58iWulXSo9osIfwRVJueEOfIcs1Qm83Lq0F7\nexj0+nBoNFeM84TFjNqu2FFcuCWopCeGObK04gsh/BJU0hPDAGNDbVapTDDZ7061WVvR6sDsKC7c\nElSbnhjmyIqhNkuIkAmq91YMDG1658/XGGt71EPdg94FwY7iws7pg5O//vprREREwNPTEydOnLD1\nNKLTU5sNxKVLP8HbWwuZrJQS3k0FBQV8F0GQKC7csvnxVqlU4rvvvsPmzZu5LI8oKBTDERjojUWL\nwvguiqDU19fzXQRBorhwy+akFx5uvmAnIYQInaB6b8WkvLyc7yIIDsWEHcWFWwN2ZNxzzz2oqakx\n2//qq68iKSkJADB//ny8+eabiImJYb+AhIZiEEIcg/NpaEeOHLG5MAbUc0sIERJOHm8psRFCXIXN\nSe+7775DSEgIcnNzcd9992Hx4sVclosQQhzC5qSXnJyMyspK6PV67N+/HzqdDpGRkXj33XdZj//3\nf/93REZGYs6cOSgpKWE9xt1kZ2cjJiam37io1Wr4+/sjOjoa0dHR2LlzJw+ldK6NGzdCoVBAqVT2\ne4wY75XB4iLGe6WyshLz589HREQEEhISkJ6eznqc1fcLY6fOzk5m4sSJTFlZGdPe3s7MmDGDOXv2\nrMkxhw4dYhYvXswwDMPk5uYycXFx9l5W8CyJy88//8wkJSXxVEJ+ZGdnMydOnGCmT5/O+rkY7xWG\nGTwuYrxXLl++zJw8eZJhGIapra1lFAoFJ7nF7ja9vLw8TJo0CaGhoZBKpVizZg2+//57k2N++OEH\nrF+/HgAQFxeH+vp66HQ6ey8taJbEBRBfe2h8fDwCAgL6/VyM9woweFwA8d0rQUFBiIqKAgCMGDEC\ns2bNQnV1tckxttwvdie9qqoqhISEGLfHjBmDqqqqQY+5dOmSvZcWNEviIpFI8D//8z+IiIjAkiVL\ncPbsWWcXU3DEeK9YQuz3yvnz51FUVIQ5c+aY7LflfrE76Vk6Dq/vXyl3H79nyc8XExODyspKFBQU\nYMWKFVi2bJkTSiZ8YrtXLCHme6WpqQlr1qzB22+/DZlMZva5tfeL3UkvODgYlZWVxu3KykqMGTNm\nwGMuXbqE4OBgey8taJbEZdiwYfDz84NUKsUjjzyC69evo65O3Csoi/FesYRY75WOjg6sXLkSDz30\nEH73u9+ZfW7L/WJ30ouNjcW5c+dQXl6O9vZ2fPXVV2Z/hZYtW4bPP/8cAJCbmwu5XA6FQmHvpQXN\nkrjodDrjX6kDBw7A19cXw4eLe7UVMd4rlhDjvcIwDB555BFEREQgLS2N9Rhb7he7FxH18vLCp59+\niuTkZHR2diI1NRVTp07Fhx9+CADYvHkzlixZguzsbCiVSshkMvztb3+z97KCZ0lcvvnmG+zevRte\nXl6IjIxk7ehwN2vXroVGo8HVq1cREhKCP/7xj+jo6AAg3nsFGDwuYrxXjh49ii+++AKRkZGIjo4G\n0DMF9uLFiwBsv18cvogoIYQICa2yQggRFUp6hBBRoaRHCBEVSnqEEFGhpEcIERVKeoQQUfk/GLt/\nPsiaGrEAAAAASUVORK5CYII=\n", 98 | "text": [ 99 | "" 100 | ] 101 | } 102 | ], 103 | "prompt_number": 2 104 | }, 105 | { 106 | "cell_type": "code", 107 | "collapsed": false, 108 | "input": [ 109 | "import numpy as np\n", 110 | "import matplotlib.pyplot as plt\n", 111 | "from matplotlib.widgets import Slider \n", 112 | "\n", 113 | "fig, ax = plt.subplots()\n", 114 | "fig.set_size_inches(5,5)\n", 115 | "\n", 116 | "plt.subplots_adjust(left=0.25, bottom=0.30,right=1)\n", 117 | "\n", 118 | "x = np.linspace(0,2,50)\n", 119 | "y = x + x**2 + np.sin(2*np.pi*x) + np.random.randn(len(x))\n", 120 | "\n", 121 | "V = np.matrix(np.vstack([np.ones(x.shape),x,x**2]).T)\n", 122 | "Q = np.matrix(np.eye(V.shape[0]))\n", 123 | "i,j =np.diag_indices_from(Q)\n", 124 | "#Q[i[:20],j[:20]]=100\n", 125 | "R = np.matrix(np.diag([1,1,1]))*0\n", 126 | "\n", 127 | "Pv = V*np.linalg.inv(V.T*Q*V+R)*V.T*Q\n", 128 | "p=np.polyfit(x,y,2)\n", 129 | "\n", 130 | "ax.plot(x,y,'o',label='data',alpha=0.3)\n", 131 | "ax.set_title('err^2=%3.2f'%(np.linalg.norm(y)**2 - np.linalg.norm(np.dot(Pv,y))**2 ))\n", 132 | "ls_line,=ax.plot(x,np.dot(Pv,y).flat,label='projection')\n", 133 | "ax.plot(x,np.polyval(p,x),'-',label='polyfit',alpha=0.3)\n", 134 | "ax.legend(loc=0)\n", 135 | "ax.grid()\n", 136 | "\n", 137 | "axw0 = plt.axes([0.25, 0.2, 0.65, 0.03] )\n", 138 | "sw0 = Slider(axw0, 'w0', 0.1, 30.0, valinit = 1)\n", 139 | "axw1 = plt.axes([0.25, 0.15, 0.65, 0.03] )\n", 140 | "sw1 = Slider(axw1, 'w1', 0.1, 30.0, valinit = 1)\n", 141 | "axw2 = plt.axes([0.25, 0.10, 0.65, 0.03] )\n", 142 | "sw2 = Slider(axw2, 'w2', 0.1, 30.0, valinit = 1)\n", 143 | "\n", 144 | "def update_w0(val):\n", 145 | " w = sw0.val\n", 146 | " R[0,0]=w\n", 147 | " Pv = V*np.linalg.inv(V.T*Q*V+R)*V.T*Q\n", 148 | " ls_line.set_ydata(np.dot(Pv,y).flat)\n", 149 | " ax.set_title('err=%3.2f'%(np.linalg.norm(y)**2 - np.linalg.norm(np.dot(Pv,y))**2 ))\n", 150 | " plt.draw()\n", 151 | "sw0.on_changed(update_w0)\n", 152 | "\n", 153 | "def update_w1(val):\n", 154 | " w = sw1.val\n", 155 | " R[1,1]=w\n", 156 | " Pv = V*np.linalg.inv(V.T*Q*V+R)*V.T*Q\n", 157 | " ls_line.set_ydata(np.dot(Pv,y).flat)\n", 158 | " ax.set_title('err=%3.2f'%(np.linalg.norm(y)**2 - np.linalg.norm(np.dot(Pv,y))**2 ))\n", 159 | " plt.draw()\n", 160 | "sw1.on_changed(update_w1)\n", 161 | "\n", 162 | "def update_w2(val):\n", 163 | " w = sw2.val\n", 164 | " R[2,2]=w\n", 165 | " Pv = V*np.linalg.inv(V.T*Q*V+R)*V.T*Q\n", 166 | " ls_line.set_ydata(np.dot(Pv,y).flat)\n", 167 | " ax.set_title('err=%3.2f'%(np.linalg.norm(y)**2 - np.linalg.norm(np.dot(Pv,y))**2 ))\n", 168 | " plt.draw()\n", 169 | "sw2.on_changed(update_w2)\n", 170 | "\n", 171 | "plt.show()\n" 172 | ], 173 | "language": "python", 174 | "metadata": {}, 175 | "outputs": [ 176 | { 177 | "output_type": "display_data", 178 | "png": "iVBORw0KGgoAAAANSUhEUgAAATQAAAE8CAYAAABO0k3yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlYVPX+B/A3CFwJUXAbZFFURBFBRAVcWNRcczcLc0NM\n7WeLaHYrs6vdm0vl1bSby7XIraxcKpdccmExQ0VR2ReNXUACF3SEGfj+/uAyODDILGfOnDnzeT3P\nPI+znfPlw/HDdz9mjDEGQggRAXNDF4AQQrhCCY0QIhqU0AghokEJjRAiGpTQCCGiQQmNECIalNCI\n2u7fv4++ffuiVatWiIuLa/T+8uXL4e7uDltbW3h4eGDv3r1ancfc3BytWrWCra0tbG1tsXDhQqX3\ni4uLMX36dHTq1AkdOnTAu+++q/I4sbGximPUPczNzfHTTz9pfCxiJBgh/yOXy5We19TUsJqaGsYY\nY1KplAUFBbE5c+aw3bt3M4lEwlJSUpQ+v2rVKpaens5kMhk7duwYs7KyYhcvXtS4HGZmZuz27dsq\n36usrGRubm5s06ZN7PHjx6yyspLdvHlTreNGRUUxW1tb9vjxY52PRYSJEpoJKCgoYFOnTmXt27dn\nrq6ubMuWLYyx2gQUGhrKXnvtNebg4MC++uorFhwczNasWcNGjhzJbG1t2a1bt5hcLmeTJ09mixYt\nUhzz4MGDrHv37iwvL6/J8z7//PNs48aNGpfXzMyMZWVlqXxvx44dLCgoSONjMsZYWFgYCw8P5+RY\nRJioySlyNTU1mDBhApydnZGUlIQ9e/Zg/fr1OH36NADg0KFD6N27N/7880/MnDkTAPDll19iyZIl\nKCsrQ+fOnZGYmIjRo0dj+/btiuNOmzYN//3vf3Hx4kWV562oqEBycjK8vb0Vr9nZ2cHe3l7l49NP\nP1X6flBQEDp16oRp06YhJydH8XpcXBy6dOmCyZMnQyKRYNiwYUhKSmo2Do8ePcKhQ4cwd+5cnY9F\nBMzQGZXoV1xcHLOzs1M0sxhjbMmSJWzevHls9erVrFu3bkqfDwkJUarFaGvmzJls4sSJWn03NjaW\nyWQylpKSwl555RXWp08fVl1dzRhjbPTo0czMzIxFRkay0tJS9vbbb7Nu3bqxqqqqZx5zz549jX5W\nbY9FhItqaCKXk5ODiooKODo6KmpDkZGRKCkpAQD4+fk1+o6/v79O53znnXeQnp6Ob7/9VqvvDx06\nFBYWFvDw8MDGjRuRnp6O1NRUAEDbtm3h6emJefPmoV27dli1ahUKCgqQlpb2zGPu3r0bc+bMUXpN\n22MR4aKEJnIuLi6wtbVFcXExysvLUV5ejgcPHuDYsWMAAAsLi0bfUfWaulatWoVTp07h9OnTaNWq\nldJ7T49cNnysX7++yWOamZmB/W8PBQ8PD5ib11+2TI29FfLy8hAdHd0ooWlzLCJslNBEzt/fH127\ndsWKFSuQnZ2N6upqJCUlIT4+vsnvaPsfe/369di/fz9+++032NvbN3q/oqICDx8+VPl47733AAAp\nKSm4fv06qqurkZGRgeXLl6Nbt27w8PAAAMybNw/p6enYu3cvysvLsWbNGri4uCjeV2Xv3r0YMmQI\nunbtqvS6NsciwkYJTeTMzc1x7NgxFBYWIiAgAB06dMDChQtx//59ALW1n4ZUvaaOFStWIC8vD25u\nbmrVvFQpLi5GaGgo2rRpg6FDh+LRo0c4duwYWrRoAQBwdnbG8ePHsXHjRri6uuLKlSs4cuSIolY5\nbty4Rufcu3ev0mBAneaORYyPGaN6NiFEJHSqoe3cuRODBw9G//79ERERwVWZCCFEK1ontLKyMqxd\nuxa//fYbrly5goyMDJw6dYrLshFCiEa07iywtrYGY0zRF/P48WOVHcGEEMIXrWto1tbW2LZtG1xd\nXeHg4IAhQ4aonNNECCG80XZGbklJCevSpQvLzMxkpaWlbNiwYezYsWONPgeAHvSgBz00fvC6UuDy\n5csICAiAm5sb2rVrh+nTpyMmJkblZ1ntIniTf6xatcrgZRDSg+JBsWjqoS2tE1pgYCDi4+NRVlaG\nyspKnDhxAqNGjdK6IIQQoiutBwVat26NlStXYsqUKXj8+DHGjBmDYcOGcVk20cnOzjZ0EQSF4lGP\nYsENnaZEh4WFISwsjKOiiJ+Pj4+hiyAoFI96FAtu6H2lwNMLiwkhRB3a5g2DLVpr27YtysvLDXV6\nooK9vT3KysoMXQxCtGawxenl5eUGH0mhh/KD7z8wUVFRvJ5PyCgW3KDdNgghomGwPjTqWxMe+p0Q\nodD2WqQamhrCwsLw4YcfGroYhJBmUEJTg5mZmVqbHoaEhODrr7/moUTiQP1G9SgW3KCEpiZ1qr/a\n7vRKCOGG4BJacXEZzpxJw6lTGThzJg3FxZpPI9D1GAkJCfD19YVEIsGCBQsgl8sBAPfu3cP48ePR\nsWNH2NvbY8KECSgoKAAAfPDBB4iNjcUbb7wBW1tbvPXWWwCAJUuWoHPnzmjdujUGDBiACxcuaPzz\niFVISIihiyAYFAtuCCqhFReXITq6BFJpL1RVuUMq7YXo6BKNEpKux6iqqsLkyZMxc+ZMFBYWYuTI\nkfjxxx9hZmaGmpoazJ8/H7m5uYiPj4dMJsMbb7wBAFizZg0CAwPx5Zdf4uHDh9iyZQuA2tvE3bhx\nA7m5uRg+fDimT5+OyspKzYNDCGmWoBJaYmIJrK17Kb1mbd0Lycl3eTtGXFwciouL8frrr6NFixZ4\n6aWXIJFIANROBp4yZQpatmyJ7t27Y/ny5YiOjlb6fsOm6cyZM2Fvbw87Ozt8+OGHuH//PjIyMtT+\necSM+o3qiTkWXLS61CWohFZdrbo4Mpn6fVO6HqOwsBA9evRAy5YtFa/5+voCAKRSKRYtWgRXV1e0\nadMG06ZNw/3795WSWMN+tA0bNqB3796ws7ND586dUVlZidLSUrV/HkKMGRetLk0IKqG1aFGj8nVL\nS/Xno+h6jE6dOiEzMxNSqVTx2rVr18AYw4YNGxAXF4dLly7h/v37OHTokNL+TQ3nzsTGxmLVqlXY\nt28f7t27h9zcXMXW5YT6jZ4m1lhw0erShKASmpdXR0ilaUqvSaVp8PTswNsxBg8eDIlEgm3btkEm\nk+HAgQMoLi4GUHvnbzs7O/ztb39DSkoKPvnkE6XvSiQSRfIDgOeeew5WVlZo06YNiouLsWLFCuo/\nIyaFi1aXJgSV0CSStggO7ggbm3RYWWXAxiYdwcEdIZG05e0YlpaWOHz4MPbt2wcnJyecPn0aL7/8\nMszMzBAeHg4nJye4u7tj9uzZCA8PV2piLlmyBGfOnIGdnR0iIiLQv39/LF68GMOHD0dQUBD69OkD\nFxcXjeMiVmLuN9KUWGPBRatLE7T0iSjw/TuJiooSbVNLU2KNRV0f2tPNTqk0rdlKhrbXIiU0okC/\nE6IPxcVlSE6+C5nMDJaWDJ6eHZptMVFCIzqj3wkRClqcToyOWPuNtEGx4AYlNEKIaFCTkyjQ74QI\nBTU5CSEmjxIaMRjqN6pHseAGJTRCiGhQQtOT2NhY9OrVq/kPCuS4hiDGiaTaolhwgwYFBM7c3BxZ\nWVno1q2b3s9FvxNiaMXFZUhMLMHIkR40KMCnul1s+SDWJEP9RvUoFspbDWmLEloDrq6u+M9//oOA\ngAC0b98e4eHhqKysRFRUFJydnbF9+3a4u7tj/vz5qKqqQkREBJycnODk5ISlS5eiqqoKQO0F+vRC\n9MLCQkybNg0dOnRA165d8cUXXyjeq6mpwdq1a+Hm5obWrVtj4MCByM/PR1BQEACgb9++sLW1xYED\nBxodNzU1FSEhIbC3t0efPn1w9OhRxXthYWFYtmwZQkND0bZtWwQEBOD27dv6DiEhWlG11ZDGmJ41\ndQoeTq2VLl26ME9PT5afn8/KyspYQEAAW7lyJYuKimIWFhYsPDyc3blzh0mlUvbhhx+yAQMGsLt3\n77K7d+8yf39/9uGHHzLGGDt//jxzdnZmjDFWXV3NfH192VtvvcWKiopYTEwMc3R0ZKdOnWKMMfbp\np5+yXr16sYyMDMYYYzdu3GB//fUXY4wxMzMzduvWLUX5nj5uVVUV6969O/v444+ZTCZj586dY9bW\n1iw9PZ0xxtjcuXNZ69at2eHDh9ndu3fZ+PHjWWhoaJM/u1B/J8Q0nDyZzo4cYezIEe2vRaqhNWBm\nZoapU6fCyckJ9vb2mDdvHvbv3w8AqK6uxurVq+Hg4ICWLVvi22+/xYIFC9C+fXu0b98er776Kvbu\n3dvomFeuXMHt27exfv16SCQSBAYGYvr06fj+++8BAF999RUWLVqEHj16AAC8vb3Rtm3z2x3FxcUh\nNzcXERERsLCwwLBhw+Dr66soLwAEBwdjypQpaN++PWbOnInr169zESZCONfUVkOasOCgHHph9hE3\nG8CxVZr3P/n4+Cj+3a9fPxQWFgKo3cDx6ebenTt30L9/f8Xz/v37Kz77tJycHFRUVMDR0VHxWnV1\ntaJJmZ+fjyFDhmhczsLCQri7u8PGxkbx2oABAxRlMDMzU/pZHBwcUFFRofF59EWsW+Zog2JRuzlr\ndHSaTs1OwSY0bRIRVxISEjB16lQAtdtvOzk5AQAsLJTD5ejoiPj4eEVSi4+PV3z2aS4uLrC1tUVR\nURGsrKxUvn/hwgUMHDhQZXlYE4MCjo6OyMjIwKNHjxRJ7cqVKxg9enSz3yVEaGo3ZwWSk9O1PgY1\nORtgjOHnn39GQUEBysrKsGvXLrz88ssqPztjxgxERkaitLQUpaWliIyMxKxZsxp9zs/PD127dsWK\nFSuQnZ2N6upqJCUlIT4+HgDw6quvYseOHThx4gTkcjlu3ryJsrLam0hIJBLF5xry9/eHi4sLNm/e\nDJlMhqioKCQkJCA0NFTxswiZqddInkaxqCWRtMXw4T21/j4ltAbMzMzw2muvYerUqejRowd69eqF\nlStXgjHW6I5OK1euREBAALy9veHt7Q1/f3+sXLmy0TFbtGiBY8eOobCwEAEBAejQoQMWLlyIBw8e\nAACWLVuG2bNn4/XXX4e9vT0WLFiAJ0+eAABWr16NJUuWwM7ODgcPHoSZmZmiHFZWVjh69ChOnz6N\nDh064I033sD+/fvh7u6u+Fkalpnu7k5ETZdRiYqKCjZnzhzm4+PDPDw82B9//NHoM02dQsdT642r\nqys7e/aszsc5e/Ys69atGwcl4g/fv5Pz58/zej4ho1go0/Za1KkPbfHixQgODsbu3bshl8vx6NEj\nLnKs0WOM4cqVK7zM7ieE1NM6od2/fx+xsbHYvXt37YEsLNCmTRvOCmbMlixZgtTUVKxdu9bQRRE0\n6jeqR7HghtZrOa9fv45FixbBw8MD8fHxGDRoELZs2QJra2vlE9BaTqNBvxMiFNpei1rX0ORyOa5c\nuYKVK1di27ZtWLRoEQ4cOIA5c+Y0+mxYWBhcXV0BAHZ2dkpzo4iw1K0prKsx6PP50+sX+TifkJ/X\nvSaU8hji54+KikJ2djZ0oXUNraioCF5eXrh7t/aW7idOnMCePXuUZqkDVEMzJnRfTsOhWCjjfQtu\nBwcHuLm54dKlS6ipqcHx48fx/PPPa3s4YoLoP3A9igU3dNoPLSMjA3PmzEFpaSm8vLywb98+pWU4\nANXQjAn9TohQ0I2Gic6oyWk4FAtldNcnAzM3N1d7r7GVK1eia9euCAgIwIULF0SzpTYhhkY1NI6o\nu1V2bGwspkyZgvT0dLRr167R+66uroiMjMTw4cP1VdQmie13QowX1dCMRE5ODnr27KkymQGUVAjR\nBSW0BpraghsAdu7ciR49eqBdu3aYNGkS7ty50+j7V65cgYODg1JSOnz4MHx8fBAZGYkFCxbg0qVL\nsLW1xUcffaS0pfbs2bORm5uLCRMmwNbWFhs2bODnhzYQ2ke/HsWCI1qtANVAU6fg4dRaaWoL7rNn\nzzI7OzuWkJDAKisr2eLFi1lQUJDie09vld27d2924sQJxXuTJ09mGzduZIwxtmvXLjZ06FDFe09v\nqc0Yd4vjtcH374QWZNejWCjT9lqkGloDTW3B/d1332Hs2LHw8fGBlZUVli5ditjYWOTm5jY6xpw5\nc7Bv3z4AQFlZGU6fPo1XXnkFgPD3KOMTjerVo1hwQ7A71uKpuxfpZMIEjb+iagvuwsJCpY76ujs0\nFRQUoHPnzkrfnzlzJjw9PfH48WP8+OOPCAoKgkQi0f5nIISoRbgJTYtExJWGW3A7OjrC0dERV69e\nVXwmMzMTDx48ULnltrOzMwICAnD48GHs27cPixcvVvvcpjQoQHOv6lEsuEFNzgaYii24Q0NDMWPG\nDJw6dQo3btxAZWUlNm/ejKFDhzaqndWZM2cOPvnkEyQlJSmSozokEolS4iTElMjkVdj63VKtv08J\nrYGmtuAeMWIE1q1bh2nTpsHR0RG5ubmK29DVfe9pU6dORW5uLqZMmYKWLVsqfe5Z22K///77+PLL\nL9GmTRts3LhRTz+lMFCNpB7FAki5fQn/974X0tJ/1/oYNLG2ga5du+Lrr7/WeWIrYwzu7u7YsWOH\nQSbJakOovxMibvJqGb768X2cPbcTUya/jxnj3oW5uTlNrBWS7777DmZmZkaTzAyB5l7VM9VYpGbH\nY/H73khOjca//3kZr7zwnk438hHuoIARCwkJQVFREbZu3WroohAiSPJqGb46sAJnz/4Xkyb+HTPH\nr+DkjmTU5CQK9DshfEi+FYdN2+fC+rk2WP7aHnTp1HhzBt634CaEEE3I5FXY8f1yREfvxtTJKxA6\n7u+c3yeW+tCIwZhqv5EqYo/FjfQYLHq3N27disemf8Vjxgvv6uWm11RDI4TozZOqx9j67VL8cfEH\nvPTiKrw4KkIviawO9aERBfqdEC7F3fwVW796DZ2c3LH01a/h0K6L2t81uj40e3t7vWZqojl7e3tD\nF4GIwMNH5diyezESb5zG7Jmf4oWg+byd22B9aGVlZWCMmdTj/PnzBi/Dsx5lZWW8XgNi7zfShFhi\ncSbuOyx8pxeqZE+w7bNUXpMZQH1ohBAOFP+Vi88jFyA3JxGLwv6DEL/pBimHwfrQCCHGjzGGH09u\nxMHD/4J/wDQsnvk5nmtpq/Nxja4PjRChKC4uQ2JiCaqrzdGiRQ28vDpCImlr6GIJ3q28m/h8Zzge\nSx9i5fIj6NszSO3v6ivmNA+NR2LpJ+GKEOJRXFyG6OgSSKW9UFXlDqm0F6KjS1BcTP2JTamSPcG2\n/W/jvY+GwtvreexYn6RxMtNXzKmGRkxaYmIJrK2Vl95YW/dCcnK6aGtputSOLt08ge2Ri2Fn3wkb\n/nVZ5bKl5ugz5pTQeER7XikTQjyqq1U3UmQyfqcU6SsWDZOXg4MlUlJkSgklOjoNwcF4ZjK59/Au\nvti1GMlJ5zBzxjqMD16g9bQrfcacmpzEpLVoUaPydUtL4x/IUtW0++GHW6isVN42vrZ2dFflMRhj\n+PncNry2vBfMzM2xfUMaJoQs1GkOqT5jTgmNR8bUT8IHIcTDy6sjpNI0pdek0jR4enbgtRz6iIWq\npp2FhStyc+83+qyq2lFW7g28tdofR45twLI3vsPK13+Ana3ucdFnzKnJSUyaRNIWwcFAcnI6ZDIz\nWFoy+PmJY5RTVdPO3LwGcnnj5PV07ehJ1WPs/OE9xETvwpgxb2LulNWwaGHJWbn0GXOah0aISJ05\nkwapVLmGVl5ehqys6xg4sH4nZak0DcHBtQkl6vIBfL03Ap0ce+KteTvg7NCD72ID0D5vUEIjREd8\nz2NT93x1fWhPNzul0jT07m2JkhK5onbk6dkBMrMyfPHN/6EgNwlzZ23AyEEz9VZ+dVBCMwJ070Vl\nYohHU0mjrsajLnVjoen5iovLkJx8Vyl5Pf05mbwK3xz+B347vRWBQbOx4KVPYN2yldrl1hdaKUCI\nAfA9j03T80kkbZssx4WEX/DV7iWws++E9asuoJWVM36/kG/UKyYoofHI2GsjXBNDPLiaU6VuLLg4\nX0HJLXy55w1k37qKmTPWYVxgOEpKyhvV/J41P02oy8V0TmjV1dUYMGAAnJ2dcfToUS7KRIjO+PoP\nx/c8Nl3OVyV7gl0/rcJvp7dh8NAZ+OD1H2Bj3RpA0zW/2NhLsLNroxRHABolPz7pPA9t8+bN6N27\nN23WqAYhzLsSEn3Fg8/1mVzNqVI3Ftqe7/zlH/Hq2+5ITY3Fun/EYmnYDkUyA1TX/MrLy5CQ8KhR\nHGNisppo9qqenMsnnRJafn4+fv31V7z66qvU8U8Eo+l+Ju7/w9XOqeoIG5t0WFllwMYmXeMBAX2e\nL7sgBe+sHYavdy/BzJfXYOM/fodb576NPqeq5peTU4LnnnNXes3auhdu336o8lx8LxdTRacm59Kl\nS/HZZ5/hwYMHXJVH1MTQZ8QlfcWD7/WZz+p4V5cmsVDnfI+kD7DzwHu4ELMPw4bPx7+WH0dLq+ea\n/LyXV0dER6c1GD3Nh4eHf6PPMibc5WJaJ7Rjx46hY8eO6NevX7PV5bCwMLi6ugIA7Ozs4OPjo/gF\n1n2XntNzrp4nJeXC3b22ZpGYWPu+l1cILC2ZIMqnz+fnz5/H7wlHkJL5LVy7+eLlF7ahg72TIpk9\n6/vBwcC33+5FdTXQv78f+vVrjczMq8jLq40fUBtPmSwLUqkLrK17KeLr5uYAP7+OWpe/7t/Z2dnQ\nhdbz0FasWIG9e/fCwsICT548wYMHDzBt2jTs2bNH+QQ0D01B3blGpkJf8eBqbhifuIjFjfQYbN/z\nFqoqH2PBnM8R4D1Op+M9K44Anjm/TVcGnVgbHR2NDRs2qBzlpIRWjxKaMn3Go7kJpUKjSyyK/srB\ntn0RSEuJxqSJf8fLY5ajRQtuZmQZKo4Gn1hLo5zNo2SmTNN4aDIVg4t+LT5pc208qXqMXYdX4eyZ\nHfAfNB3//Xcm2rRqx2m5jC2OtPSJGAVjbEbqC2MMv8Z8jf0/fggHp574v9lb0N3F29DF4pS2eYP2\nQ+PR0x2gRLN48DkVwxDUjcXVlLNYvNIHh35Zh0XhW7FhRZTokpkuaOkTMQp8T8UQ2tKevKIMbNu3\nBLcyL2Pq5Pfx4qgIzvrJxISanMQoqNrbCwBsbNIxfHhPtY6h67Y7hmjePnxUjq8OvIffL+xHYNAs\nzJ++Dq2s2/BaBkMw+KAAIfqkeuJnGvz8Oqr1fVVJqqn1h1ztoKFLLU8mr8L3xz/BsWMb4d57KD5f\new3OHd3UPrepoj40HlEfmjJN4qHrEiNN+uC4aN5qup60Lha1Hf6RmL+sOy5fO4oVy3/Bv5YepWSm\nJqqhEaOhyxQCTZIUFztoaFPLu5J0GpH738ET6UPMnfUZhg98maZDaYgSGo9oHpoyPuOhSZLStXkL\naJZAM7Kv4dc//oW83JuYPPE9vDhqCXX4a4miRkyCJkmKi7sSqZNA75Rm46v97+DmzdMYOfI1fLT0\nF0Fsf23MaJSTR7T0SRkX8dCk453PZTzPGiltacPwzcEPcOHCd/APmIb5L63HzWupdG08hUY5icnR\nZOQS4HcZj6panlff5/BL9Kf47bft8PAMxqaP4+HiULffWCov5RI7qqERg9JlagMXc9P4IJNX4cdT\nG3H06Aa4dPZC+CufwcN1gKGLJWhUQyNGR9MaVkN8rx7QVA2rwZHzO3Dop49hZ++Id976Af17jzB0\nsfRGCKsraB4aj2gemrJ9+47ptD6T7xuUqIsxhpOxuzD/bTf8enILXp27BV/888ozk5mxXxt83sfh\nWaiGRgympkZ1TUrdGhYX0yu4xBhD9JWD+O7QKshllQidthqjBs82iblkfN+ftCmU0HhEo1jK+vUb\nCKm08evq1rC4mF7BlT9uHMfeHz9AxcMyvDhlBcaHLIS5mfoNIGO/NoTS/KeERgyGixqWoTcgvHTz\nBPYd/AfKygowafzbmDryLVi0sDRYeQxFKM1/6kPjkbH3k3AtNfUmr7eA49KVpNN4a7U/tuyYh6GD\nXsauz7Px0pi3tU5mxn5tcHV/Ul1RDY0YlKFrWJqKTz6Dbw/+A8VFWXhh/DJMHxUBK8uWan9fCCOB\n+iCU5j/NQyNEDZcTT+HbQ6twt/g2xo59Cy+NeRt/s7LW6BhC2mdN6GgeGiF6EHfzV3x7aDVK7+Zg\n/NgleHH0Uo0TWR2hjASKGfWh8cjY+0m4JtR4MMYQe+1nvPmPAfhiRzgG+U3F7s05mDlhhdbJDHj2\nSKBQY2FsqIZGyP8wxnDu0vc48MtaPH50DxNeWIbJI16HpYUVJ8cXykigmFEfGjF5NawGJ2N34dDR\nT1BdLcOU8e9gfPACzvckoz409Rn0zunPPAElNCJQMnkVfj77JY6d+BxWVtaYOuFdjB4695kTYnUd\npTS2O7obCiU0I0D7oSkzVDykTyrw48l/49TprWjX3gUvTnwfQf2nNrtESZ81LLo2lNEoJyHNKH9Q\ngu+Pf4Koc5FwcfXGste/xQDP59X+Pt+jlGKds6ZPVEMjopdflInvjq7B5bhD6N1nGEInfYDe3fw1\nPs6pUxmoqnJv9LqVVQZGj278ui5Mvb+NamiENJB6+zL2//IxUpLOwy9gGj5fcw3ODj20Pp4+Rykb\n1sbKyx/A2tpP6TM0Z615NA+NR4aaa1RcXIYzZ9Jw6lQGzpxJ432PqqboIx6MMcTEH8bSjwZj9brR\n6OTQAzs33cLfF+zSKZkB+luvWFxchq1bjyntJZaQ8AD37j1s9FmhbF4pVFRDEzldd4U1FjJ5FY6e\n34Hjp7ZALq/C2JGvY/L7Z9DS6jnOzqGv9YqJiSX42986K71mbe2M3Nz7sLOzVXqd5qw9G/WhacGY\nOmuNZd99bd17eBc/nvg3os5Hom17F0watwwjAmZotBeZoanqmysvL0N6+nUEBAxXvEZ9aM2jGpqG\njK3GI5SN97h2K+8mfji2Htfij6KnRyDeWXIA/XoFG7pYWlHVN2dv3xb9+tnAxsbwm1caE0poGtJl\n6N4Qc42EvNxG03gwxvD79SP46dd/Izf7JgYPeRmb112HU8fu+iskD7y8OmLr1j0YMGCO4rXa2lgP\nSmAaooQc7pxxAAAWw0lEQVSmIWOr8Qht331tSJ9U4JezW3H67A5U18gxasQi/Ovt43iupW3zXzYC\nEklb+PjYUW2MAzr1oeXl5WHOnDkoKSlBhw4dEBYWhrCwMOUTiKwPzRj7pIx1uU1eUQYOnNiAPy7+\nCJfOfTBp7FK1ZvQT42eQpU9FRUUoKiqCj48PSktL0adPH5w/fx4eHh46F0yoTH3Co74xxnAh4Rcc\nPbUFt2/FY6DfZLw4djm6u3gbumiER4JYyzlhwgRERERgxIj6+w9yndCEMMKobY2H1uspezoeDx+V\n46ffvsC5qEiYwQzDh8/H5BGvw9bG3rCF5AldG8oMPsqZlZWF5ORkBAQEcHXIRoQywmhs++ALWcrt\nS/jp5OdIuHoM3d398erczRjiM5GalUQrnCS0iooKhIaGYtOmTbCxsWn0flhYGFxdXQEAdnZ28PHx\nUfw1qpstrs7zxMQSZGUVASiCl1ft+1lZRcjPv4Jly2ZrfDy+n4eEhAiqPIZ6LpNVosIsE6fP70T6\nzT/h2We4YrQyKioK0dHRgiovPdf/87p/Z2dnQxc6NzllMhnGjx+PsWPHIiIiovEJOGxy8rk4mCtC\naCILxe38JBw+/Tku/3EQDo7uGD1iEUYNns3ZjrCmTGzXmbZ5Q6fp1IwxzJ8/H56eniqTGdeEPKdK\nlbomct0avcuXixAdXSKYtZR8qKyS4sj57XjzHwPx7kdDUVNdjbUfRmPLR5dhU9Odktn/PF1T0VTD\n60wq7WVy11kdnZqcv//+O/bt2wdvb2/069cPALBu3TqMGTOGk8I1ZGxzqkz5Lj+Zudfx8+nNuHzp\nJzh0csPzIfMxZmiYTjcZIaqZ8nXWkNGt5TSmOVXG2ETWxSPpA/wa8zXOxexGWUkO/AZNw+SRb9GU\nCz0T43Vm8FFOvhjTCKOxNZG1wRjDtdRzOH52G25ePwUX176YMPotjBw8i5qTPDGF60xdxrMlgRFq\nuH9WYmIUJ/tnCUHRXznYeeA9hC/thk1bZ6ND+y7YvO46Nn14AeOCwtVKZrr0G4mNLrHQ1z5txsjo\namjGpOH+WS1b5iE42NtoapgNVVZJ8dsf+3A2Zjfy/rwOz74jsXDefxDgPU6tG4w0HIUj3NDXPm3G\nyOj60MRMiEPvjDFcSTqN0zGRuHH9JBw69UDI0NkYEzgPNtat1ToGLRcjmhLE0ieVJ6CEphah/af/\nsyAZx8/vwKVLh2Bm1gIBAdPwwrBF6NKp8cL85hjjgn5iWCYzKGDMop6xXk8IQ++l9wpxIvprXIw7\ngNLSHPTzfQER/7cHvh7DVTYp1a1RNrXl0tWrlymh/c+zrg2iPkpoAmGofdYqpPdx+sIexMZ9j9w/\nr6OnRyCmTHgHIQOnw8qyZZPf02RdbVOjcC1acPMzEFKHmpwCwWez7EnVY5y/9AOi//gemWkX0LlL\nXwQOnoFRQ+eglXUbzssrtOY0ET5qcho5fa+CkMmrEBN/CFF/fIfUxPOQOLljsN+LeHtBJDrYO2l8\nPE1qlDQKR/giioQmxNFBVZ7VT6KP//QyeRVirh1GzMXvkZx0Du3auyDAbxr+b9ZmOHbopvVxAc0n\nc6qaEE39RvUoFtwQbEJTN0kJZY80LnCxCqJK9gSxV39CTNwPtUmsQ2f495+MV0M/hYuD8jIYXf4Q\nGNu6WmIaBNmHpkmfC00JqF1DGXXlAC5ePoT0lBi0l3SF38DJGDV4TpN3C+eiX8uY1tUS4yKqPjRN\npjAY212YuFL2oBjn475HXPzPuJV1GU4unvAfMAlvhG1Fp/auzX6fi2kixrSulpgGQSY0TZKUPhfm\nct03p2s/Sc6dNJz74zvEJxxDUV4auvYYCL/+E/H+G9+jbWuJRscSwh8C6jeqR7HghiATmiZJSl99\nOULom6thNbiWcg4Xrh7GzRun8fBBKXp7hmDimAi4OwfjVqYU1dXmuHa5HF5elhqVi3ZoIGJk9H1o\ndZ/nui/HUH1zFdL7iLlyCJeuHUFqchSes7FDH+/nMaT/FPj1GY0WLSw46//SNMbGMJJMxEF0azkN\n3eHM16Z5jDFk5FzDhfjDuH7zNApyk+DUxQv9fcYiaOB0dHPu0+g7XCVbdWNME2MJ30Q1KAAYvsNZ\nH02yun6SCul9/H7tCOJv/IrU5ChU18jh0TsYL4x8HUN8JzV7L0qu+r/UjbG+1plSv1E9igU3BJvQ\nDI3LvrkaVoPEjAs4cm47Dpxdhjt5qXDq7Ik+vYfh/WU/oXc3f43uQ8l3/5cQBhAIUYdgm5xCoEuz\nN68oAxcTjuB60m/IyrgEa2tbeHgEYYDPOAz2maD2XmJNlYvPJiDN9SN8E10fmrEpvVeIuBvHcT3p\nDNLTfseTJxXo4R6Avn2exyCfCejcidv/+Hz2MVIfGuGbUSQ0MY2U3Xt4F5dunkBC0m/ISL+Ie/eK\n0KVrP3j2CoK/zwvwdBsEczPlppox95PoI4Eaczy4RrFQJvhBAUPM6+Iygd4tL8DlxJO4mXoeWZmX\nUF6aD6cuXvDsFYTF83egk70XUlPKUV1tjuKcGnRsfY/3ZK3PPxiGHqQhRB281dD47ofRpZnEGENO\nYSquJv+G5PRY3L4VjwcP7qJLl77o1XMIfPuMRN+eQYoNEIXQJBNCGQjhiuBraHyPlGky1aBK9gSJ\nmb/jZlo00jL/QM7ta2AAXLv5wsN9MCaMfBPePYagRQvV4RLC9tlCKAMhhsZbQhPSVIOCklu4nhaF\nlIwLuH3rKoruZMC+rRO6dvXFAJ9xeG3W53B17K32VAp1k7U++0mMcWpFU/EQU1+ruqgPjRu8JTS+\n98+qS6DSqjLkl0bjTulF/FUWj/vl12F+sAbOnfvArbsfZkxbjX4ew5qdzKrOuRric12kEMrABSGs\noSXGi/dRTn1ONXjwqAyJGReQdusSUtMvIfvPJMgq76FVa3e0bdsP9m1cMWnUaPh6DdJoImtzhNB/\nxVUZDF07ojlvBDCSaRtcKvorB8m3/kDmrXhk595AQX4qHj4shcShO7p06Qu3rv3h2N4b0nudUFNt\nxctcLUNvdqhrGYSQmPlaQ0uETfCDAtqqrJIiPecqMv6Mx+2cG8jPT0ZxYSaqa+To5NQTLi5eGOg7\nAXOnr0GPLv1g0cLSIOVUZ1qDvvtJdJ1awffAgqp4iKXprCnqQ+OGYBJalewJbuXdRGbONWTnJSK/\nIA1FxVl4UHYH9m2d0MmpJzq7eGHi2Aj0ch0IJ4kbp81GIoyBBbpXAdEF703O8gcluJV3E7fzbiCv\nMBV37mSipPg27t8rQhs7B0gc3ODk7AFXZy/06NIPbi598Tcra30WEYDh+46EQCj9V0JovhPDEnQf\n2sqN41FS8ifK7uaiRi6HfQcXSCTd4eTYE12cPNHNxRvdnL14SVyqCKHvSAgoDkQoBJ3Qvv/1M3R2\n9EA3Zy90bOsiuKYiXzUTY+gn4bN2ZAzx4AvFQplBBgViYmIQEREBuVyOBQsW4M0331T5uZfHLtfl\nNHonhL4joaA1m8SYaV1Dq66uRs+ePXHmzBk4OTlh4MCB2L9/Pzw8PJRPYATbBwml70gsqD+S6Erb\nvKG6aqKGy5cvw83NDa6urrC0tERoaCh++eUXbQ9nUF5eHSGVpim9JpWmwdOzg4FKZLzq+uGk0l6o\nqnKHVNoL0dElKC4uM3TRiAnQOqEVFBTAxcVF8dzZ2RkFBQWcFIpvEklbBAd3hI1NOqysMmBjk66X\njvCoqCiNv1NcXIYzZ9Jw6lQGzpxJE3xiaHou291Gn9UmHmJFseCG1n1oQuvY15UQ+46McV0j9UcS\nQ9I6oTk5OSEvL0/xPC8vD87Ozio/GxYWBldXVwCAnZ0dfHx8FCM6dX+ZTOF5SEiIRp9PTCxBVlYR\ngCJ4edW+n5VVhPz8K1i2bLbBfx5Vz5OS4lBZWagob2Ji7fsBAZ10jgc9F+/zun9nZ2dDF1oPCsjl\ncvTs2RNnz56Fo6Mj/Pz8jHZQQKiMcV0jzWUjXOB9UMDCwgKRkZGYMmUK+vfvj/Dw8EbJjCjTtJ/E\nGNc1atIfqWk8xIxiwQ2d5qEFBwcjISFB50LQML9qxrquUYj9kcQ0GHz7IGqiPButaySmSNBLn551\nCprUSghpiPc+NK6Y0jA/9ZMoo3jUo1hww+AJzRg7vgkhwmTwJif1oRFCGjLaPjSAOr4JIcqMOqGZ\niija80oJxaMexUKZ0Q4KEEIIV0yuhkaTeAkRPmpyqoEGIAgxDtTkVIMme3XpA801UkbxqEex4IZJ\nJTRTmsRLiCkyqSYnLbMixDgIuskplK2j6d4BhIgbLwlNKDfK4OveAU2hfhJlFI96FAtu6LQfmiZq\nO9/TDT6aKIS9umjqCCH6wUsf2pEjtacQ8tbRfKGpI4Q0T9B9aHVoBw3DTx0hRMx4S2jU+V7bT0JT\nR+pRv1E9igU3eOlDs7FJh58fNakA2v+NEH0yqXloQkB9aIQ0j9ZyGhHa/42QZ6OEZgRozytlFI96\nFAtlRjHKSQgh+kQ1NEKI4FANjRBi8iih8YjmGimjeNSjWHCDEhohRDSoD40QIjjUh0YIMXmU0HhE\n/STKKB71KBbcoIRGCBEN6kMjhAgO9aERQkweJTQeUT+JMopHPYoFN7ROaO+88w48PDzg6+uLiIgI\n3L9/n8tyidL169cNXQRBoXjUo1hwQ+uENmrUKCQnJyM+Ph6PHj3CunXruCyXKN27d8/QRRAUikc9\nigU3tE5oI0eOhLm5OczNzTF69Gjk5+dzWS5CCNEYJ31oO3fuxKRJk7g4lKhlZ2cbugiCQvGoR7Hg\nxjOnbYwcORJFRUWNXl+7di0mTJgAAFizZg0SEhJw8OBB1ScwM72bfxBCdMf7jrW7du3Czp07cfbs\nWbRs2VLbwxBCCCe0vuvTyZMn8dlnnyEmJoaSGSFEELSuofXo0QNVVVVo27b25h6DBg3C1q1bOS0c\nIYRoQutBgczMTOTk5CAhIQGbN29GXFwcvL298cUXX6j8/Pvvvw9vb28EBAQgLS1N6wIbg5iYGPj6\n+jYZj6ioKLRp0wb9+vVDv3798PHHHxuglPoXHh4OiUQCLy+vJj9jStdFc/EwlesCAPLy8jBs2DB4\nenoiJCQEu3btUvk5ja8PpiO5XM66d+/O/vzzT1ZVVcX69u3LUlJSlD5z/PhxNnbsWMYYY3Fxcczf\n31/X0wqWOvE4f/48mzBhgoFKyJ+YmBh27do11qdPH5Xvm9J1wVjz8TCV64Ixxu7cucMSEhIYY4zd\nvXuXSSQSTvKGztM2Ll++DDc3N7i6usLS0hKhoaH45ZdflD5z5MgRzJ07FwDg7++Pe/fuobi4WNdT\nC5I68QC0G8ExNoGBgbC3t2/yfVO6LoDm4wGYxnUBAA4ODvDx8QEAtG/fHgMHDkRhYaHSZ7S5PnRO\naAUFBXBxcVE8d3Z2RkFBQbOfEetEXHXiYWZmhosXL8LT0xPjxo1DSkoK38UUBFO6LtRhqtdFVlYW\nkpOTERAQoPS6NteHzglN3XlmDf/yiHV+mjo/l6+vL/Ly8nD9+nVMnToVEydO5KFkwmQq14U6TPG6\nqKioQGhoKDZt2gQbG5tG72t6feic0JycnJCXl6d4npeXB2dn52d+Jj8/H05OTrqeWpDUiYetrS2e\ne+45WFpaYv78+SgvL0dZWRnfRTU4U7ou1GFq14VMJsO0adMwa9YslSuNtLk+dE5oAwYMQGZmJrKz\ns1FVVYUffvih0V+WiRMnYs+ePQCAuLg42NnZQSKR6HpqQVInHsXFxYq/PEePHoW1tbVi+ospMaXr\nQh2mdF0wxjB//nx4enoiIiJC5We0uT60nlirOICFBSIjIzFlyhTI5XIsWLAAHh4e2LFjBwBg0aJF\nGDduHGJiYuDl5QUbGxt88803up5WsNSJx8GDB7Ft2zZYWFjA29tb5aCBGMyYMQPR0dEoLS2Fi4sL\nPvroI8hkMgCmd10AzcfDVK4LAPj999+xb98+eHt7o1+/fgBql1Tm5uYC0P760PsW3IQQwhfasZYQ\nIhqU0AghokEJjRAiGpTQCCGiQQmNECIalNAIIaJBCY0QIhqU0AghokEJjRAiGpTQCCGiQQmNECIa\nlNAIIaKh824bzTHlDfsIMSZi2KdCqxpaamoq/P394e3tjQ8++ECNbzCVD4baINKDHvQw7KMhXe7Y\n1dxdz/SKaWHgwIHs0qVLjDHGxo4dy06cONHkZwEwgKl8MN1vOkUI4UDDVKDtHbvUueuZPqmsoX32\n2WeKzLp06VKMGDECAHDu3DmMGDECFRUV8PPzAwDMmTMHP//8My/JlxDCD23u2FVUVKT2Xc/0RWVC\nCwoKQmxsLAAgPj4ejx49glwuR2xsLEaPHq20r7eTk1OjuxoRQsStqbubFRYWNnvXM31SmdB8fX1x\n9epVPHz4EC1btsSgQYMQHx+PCxcuYNCgQbwVjhAiXExF35uhqRzltLS0RNeuXbFr1y4MHjwY3t7e\nOHfuHLKystCjRw+le+OZ+p16CDFFqu7I5OzsDJlM1uxdz/SpyVHOwMBAbNiwAcHBwQgMDMT27dvh\n6+sLBwcHtG7dGpcuXQJjDHv37sXkyZN5KzAhxPCauiOTOnc906cm56EFBgZi7dq1GDRoEKytrWFt\nbY3AwEAAwDfffIN58+bh8ePHGD9+PMaMGcNbgQkh+qftHbuauusZX/R+16faibWqT8FgVjuDgxBi\nUGZmZoLsE9MULX0ihIiG3pc+1VK9/MkMAGhpFCGEI3pPaGKoxhJCjAM1OQkhokEJjRAiGpTQCCGi\nQQmNECIalNAIIaJBO9YSQgCIY0aC1jW0Dz74AJ07d4atra0an6Yda+lBDyE/GjLWHWu1TmiTJk3C\n5cuXuSwLIUQg5s2bh5MnTzb5/q+//oobN27g5s2b2Lx5M8LCwgAA1dXVCA8Px+HDh3H16lV8/fXX\nSE1N5anUz0hoz9q1dtasWfDz84ODgwM/pSSE8EpUO9YCz961Njg4mLcCEkKEx6h2rAWevWtt3TZC\nhBDTparvzdCaHOV81q61vXr14rOMhBCBMboda4Gmd60lhJg2oe5Y22xCKyoqwqBBg9CxY0elXWv/\n/ve/w8XFBVKpFC4uLvjnP//JS4EJIfo3Y8YMDB48GOnp6XBxcUFkZCR27NiBHTt2AADGjRsHb29v\neHl5ISIiQuWOtf3790d4eDjtWEsI4RftWEsIIQJDO9YSQkSDdqwlhIgGNTkJIaJBCY0QIhqU0Agh\nokEJjRAiGpTQCCGiQTvWEkIAiGNGglY1NKlUihdeeAEeHh4YMmQINmzY0Mw3aMdaetBDyI+GjHXH\nWjAtPHr0iEVFRTHGGHv48CHz9vZmJ06cUPlZAKx2fVPjB9Pu9IQQjjVMBTExMezatWusT58+Kj9/\n/PhxNnbsWMYYY3Fxcczf358xxphcLmfdu3dnf/75J6uqqmJ9+/ZlKSkp+i38U7TasXbhwoWKTR5b\ntWqFoKAgXjdxI4Tol8nuWPvXX3/h+PHjGDVqlP5LSwgRBFHuWCuXyzFr1iwsXbpU6YcghIgfU9H3\nZmg67Vi7cOFC9OzZE2+++SZvBSaEGJ7odqxduXIlHjx4gE2bNvFSUEKIcAh1x9pnzkMLDAzE2rVr\nMWjQIFhbWyt2rC0oKMDatWvh4eGhSHBvvvkmwsPDeSk0IUS/ZsyYgejoaJSWlsLFxQUfffQRZDIZ\nAGDRokUYN24cYmJi4OXlBRsbG5U71srlcixYsIB2rCWE8It2rCWEEIGhHWsJIaJBO9YSQkSDmpyE\nENGghEYIEQ1KaIQQ0aCERggRDUpohBDR+H9/bKcQLUG+MwAAAABJRU5ErkJggg==\n", 179 | "text": [ 180 | "" 181 | ] 182 | } 183 | ], 184 | "prompt_number": 77 185 | }, 186 | { 187 | "cell_type": "heading", 188 | "level": 3, 189 | "metadata": {}, 190 | "source": [ 191 | "References" 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "metadata": {}, 197 | "source": [ 198 | "This post was created using the [nbconvert](https://github.com/ipython/nbconvert) utility from the source [IPython Notebook](www.ipython.org) which is available for [download](https://github.com/unpingco/Python-for-Signal-Processing/blob/master/Projection_mdim.ipynb) from the main github [site](https://github.com/unpingco/Python-for-Signal-Processing) for this blog. The projection concept is masterfully discussed in the classic Strang, G. (2003). *Introduction to linear algebra*. Wellesley Cambridge Pr. Also, some of Dr. Strang's excellent lectures are available on [MIT Courseware](http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/). I highly recommend these as well as the book." 199 | ] 200 | } 201 | ], 202 | "metadata": {} 203 | } 204 | ] 205 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Jupyter notebooks for Python 2.7 for Signal Processing Book. 2 | 3 | This book is [available as a 4 | blog](http://python-for-signal-processing.blogspot.com) where you can read the 5 | formatted notebooks and comment further. The following are the **draft** 6 | Jupyter notebooks. A subset of the blog and the content here is available 7 | in printed form on [Amazon](http://www.amazon.com/Python-Signal-Processing-Featuring-Notebooks/dp/3319013416). 8 | 9 | Notebook Viewer Static Page Views 10 | ----------------------------------- 11 | 12 | **Signal Processing Reading List** 13 | 14 | - [Sampling Theorem](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Sampling_Theorem.ipynb) 15 | - [Sampling Theorem Part 2 v2](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Sampling_Theorem_Part_2_v2.ipynb) 16 | - [Fourier Transform](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Fourier_Transform.ipynb) 17 | - [Frequency Resolution](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Frequency_Resolution.ipynb) 18 | - [More Fourier Transform](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/More_Fourier_Transform.ipynb) 19 | - [Windowing](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Windowing.ipynb) 20 | - [Windowing Part2](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Windowing_Part2.ipynb) 21 | - [Windowing Part3](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Windowing_Part3.ipynb) 22 | - [Filtering](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Filtering.ipynb) 23 | - [Filtering Part2](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Filtering_Part2.ipynb) 24 | - [Filtering Part3](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Filtering_Part3.ipynb) 25 | - [Compressive Sampling](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Compressive_Sampling.ipynb) 26 | 27 | **Stochastic Processes Reading List** 28 | 29 | - [Conditional Expectation Gaussian](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Conditional_Expectation_Gaussian.ipynb) 30 | - [Conditional expectation MSE](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Conditional_expectation_MSE.ipynb) 31 | - [Conditional expectation MSE Ex](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Conditional_expectation_MSE_Ex.ipynb) 32 | - [Conditional Expectation Projection](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Conditional_Expectation_Projection.ipynb) 33 | - [Projection](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Projection.ipynb) 34 | - [Projection Ex](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Projection_Ex.ipynb) 35 | - [Projection mdim](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Projection_mdim.ipynb) 36 | - [Inverse Projection Constrained Optimization](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Inverse_Projection_Constrained_Optimization.ipynb) 37 | - [Gauss Markov](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Gauss_Markov.ipynb) 38 | - [Maximum likelihood](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Maximum_likelihood.ipynb) 39 | - [Expectation Maximization](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Expectation_Maximization.ipynb) 40 | - [Markov chains](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Markov_chains.ipynb) 41 | - [Buffons Needle Sim](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Buffons_Needle_Sim.ipynb) 42 | - [Sampling Monte Carlo](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Sampling_Monte_Carlo.ipynb) 43 | - [Rectangle Wedge Tail Decomposition](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Rectangle_Wedge_Tail_Decomposition.ipynb) 44 | 45 | **Misc.** 46 | 47 | - [Example CSVs](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Example_CSVs.ipynb) 48 | -------------------------------------------------------------------------------- /Rectangle_Wedge_Tail_Decomposition.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "Rectangle_Wedge_Tail_Decomposition" 4 | }, 5 | "nbformat": 3, 6 | "nbformat_minor": 0, 7 | "worksheets": [ 8 | { 9 | "cells": [ 10 | { 11 | "cell_type": "code", 12 | "collapsed": false, 13 | "input": [ 14 | "from __future__ import division\n", 15 | "%qtconsole" 16 | ], 17 | "language": "python", 18 | "metadata": {}, 19 | "outputs": [], 20 | "prompt_number": 1 21 | }, 22 | { 23 | "cell_type": "code", 24 | "collapsed": false, 25 | "input": [ 26 | "import sympy as S\n", 27 | "import sympy.stats as st" 28 | ], 29 | "language": "python", 30 | "metadata": {}, 31 | "outputs": [], 32 | "prompt_number": 2 33 | }, 34 | { 35 | "cell_type": "code", 36 | "collapsed": false, 37 | "input": [ 38 | "u=S.symbols('u')\n", 39 | "p=st.LogNormal('u',0,0.5)\n", 40 | "pu=st.density(p)(u)\n", 41 | "fx=S.lambdify(u,pu,'numpy')\n", 42 | "x = linspace(0.001,4,100)\n", 43 | "hist([st.sample(p) for i in range(1000)],bins=50,normed=1);\n", 44 | "plot(x,fx(x),'r-',lw=3.)" 45 | ], 46 | "language": "python", 47 | "metadata": {}, 48 | "outputs": [ 49 | { 50 | "output_type": "pyout", 51 | "prompt_number": 20, 52 | "text": [ 53 | "[]" 54 | ] 55 | }, 56 | { 57 | "output_type": "display_data", 58 | "png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD9CAYAAAChtfywAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl0VfW99/H3DgkyhnnMCY2SCGHGgsGqNQ6MloiIVyiI\nF5RSh4va1sdn9barwNMr0tbl6gPam95HtIpVWkUZhMigUQFJVFCsAwQNkgSMBAlEBjPt54+dnIEk\nJwk5J3vvcz6vtfZy//b5nX2+Afyw+Z3fb2/DNE0TERFxrRi7CxARkZZRkIuIuJyCXETE5RTkIiIu\npyAXEXE5BbmIiMsFDfL58+fTp08fhg8f3mCfRYsWkZKSwsiRI9m7d2/ICxQRkeCCBvm8efPIyspq\n8PVNmzZx8OBB8vLy+Otf/8rdd98d8gJFRCS4oEF+9dVX061btwZfX79+PXfccQcAaWlplJaWUlxc\nHNoKRUQkqNiWvLmoqIjExERv2+PxUFhYSJ8+fQL6GYbRko8REYlaTVl83+IvO8//kIZC2zTNiN1+\n97vf2V6Dfj79bPr5Im9rqhYFeUJCAgUFBd52YWEhCQkJLTmliIg0U4uCPCMjg2effRaA3bt307Vr\n1zrDKiIiEl5Bx8hnzZrFW2+9RUlJCYmJiSxZsoSKigoAFi5cyJQpU9i0aRPJycl07NiRp59+ulWK\ndpr09HS7SwirSP75IvlnA/180cIwmzMQc6EfYhjNGu8REZGmZ6dWdoqIuJyCXETE5RTkIiIupyAX\nEXE5BbmIiMspyEVEXE5BLiLicgpyERGXU5CLiLicglxExOUU5CIiLqcgFxFxOQW5iIjLKchFRFyu\nRc/sjDjl5bBuHcTFwbhx0Lev3RWJiDRKQV6rsBBuvRV27/YdGz0ann4aRo60ry4RkUbowRIAu3bB\ntGlw7Fjd13r1gh074NJLW78uEYlqerBEU50+DdOn+0K8TRtIS4O2ba32sWMwfjz4PWRaRMRJoibI\n4+O7YxhGnW1x955QXGx16tEDtm+3hlfefBM6dLCOHz4MM2eCk/9VISJRK2qCvKzsBGAGbF04wf3l\n53ydli2Da66x9n/0I1i7FmJrvkbYtQv++c/WLVpEpAmiJsjr80seo1ttY+BA+Pd/D+wwcSLcf7+v\n/fDDcO4cIiJOErVB3oVSHuRx34GlS61ph+f7zW+sIReAQ4dgxYpWqU9EpKmiNsinsoFOnAbgEyBm\n9uyAsfP4+O5Wx65dYfFi3xt//3s4darV6xURaUjUBvk0XvXur+YRzPPGz60x9RoLF0JKirV/6hT8\n7W+tWaqISFBRGeTtOMsksrztV7g5+Bvi4uDBB33tlSuhujpM1YmINE9UBvkNbKMjZwD4HNjP4Mbf\ndPvtEB9v7R84AFu3hq9AEZFmiMog9x9WeTVIvwCdOsG8eb62vvQUEYeImiX6hmEAJm2o5Cj96EUJ\nAOOAHOqrrZ6a8/J8S/UNw2oPHBjOskUkimmJfgN+xC5viB+hH7nNeXNKCkyebO2bJjz7bMjrExFp\nrqgL8uvZ7t3fwNR6r8UbEh/fnRmbN3vbXyxdGjhVUUTEBlEX5OPw3aY2m/Rmvbes7AQbOctJrC89\nBwJpvBs4VVFEpJVFVZAbVJNGjre9m3HNPsf3tOMlZnjbc1gdktpERC5UVAX5YD6nKycBKKY3h0i6\noPOsZo53/zbW6OkcImKrqApy/2EV62rcuKDzvMU1FOABoBclTAhFcSIiFyhqg/xdrrjg85jE8AKz\nvO1ZQfqKiIRbo0GelZXF4MGDSUlJYfny5XVeLykpYdKkSYwaNYphw4bxzDPPhKPOkKh7RX7hXmSm\nd/8nYD24WUTEBkEXBFVVVTFo0CC2bdtGQkICY8eO5YUXXiA1NdXbZ/HixXz//fcsW7aMkpISBg0a\nRHFxMbGxvpFjJywI6mwYnMQgBpMqYujCSU7TCWt4pWkLgmoXFVlM8rmYJL6ymllZ1v3LRURCJCQL\ngnJzc0lOTiYpKYm4uDhmzpzJunXrAvr069ePUzW3dT116hQ9evQICHGnuByIqQnhjxleE+ItYbCW\n6b7m2rUtPJ+IyIUJmrhFRUUkJiZ62x6Ph5ycnIA+CxYs4LrrrqN///6UlZXxj3/8o95zLfa7p3d6\nejrp6ekXXvUF8B9IadqwSmzNFXjD1jKdX9Q+nOLVV+HJJ62HN4uIXIDs7Gyys7Ob/b6gQd5YkAE8\n8sgjjBo1iuzsbL744gvGjx/PRx99ROfOnQP6+Qe5Hcb67TctyCupO+QS+OvxLlfwNX3oSzF88431\nXM+rr25hpSISrc6/yF2yZEmT3hd0aCUhIYGCggJvu6CgAI/HE9Bn165d3HrrrQAMHDiQiy++mP37\n9ze17lYzzG//Q0aF5JzVtOFVpvkOvPJKSM4rItIcQYN8zJgx5OXlcejQIcrLy1mzZg0ZGRkBfQYP\nHsy2bdsAKC4uZv/+/VxyySXhq/hCnDlDbUVVxPB5U+4/3kQB4+SvvGLdTEtEpBUFHVqJjY1l5cqV\nTJw4kaqqKu68805SU1PJzMwEYOHChfz6179m3rx5jBw5kurqav7whz/QvbvDbiL1+efev7EOksz3\ntAvZqbNJ5yTQBayHM3/6KQwdGrLzi4g0JjruR/7cczB3LgBruZlb8J9h0vD0w/rHyOv2XYPBv9U2\nHn0UHn64pRWLiOh+5AE++cS3S+ivljf6NzZsCPn5RUSCUZCHwGawnhgE8O67UFIS8s8QEWmIgjwE\nSgDG1UxprK62VnmKiLSSyA/y776D/HwAKmnDAS4Nz+f85Ce+/Y0bG+4nIhJikR/kn33m3c0jhXIu\nCs/n+Ad5VhZUVITnc0REzhP5QR7mYRWv4cNhwABr/+RJa5WniEgrUJCHimHA5Mm+tt9DmkVEwklB\nHkqTJvn2FeQi0kqiKsj/FXDHlTC4/nqIi7P29+2DoqLwfp6ICJEe5OfOQc1Nv6qwlueHVefOcNVV\nvramIYpIK4jsIM/P997E6iuggrbh/0z/cXIFuYi0gsgO8i++8O221mf6B/nWrVBZ2VqfLCJRKrKD\n/OBB726rBfnQoVB7z/aTJ60l+yIiYRTZQW7HFfn50xA1vCIiYaYgD4eJE7277z/yCIZheLf4eIfd\nq11EXE9BHg7XX+99CPNlGPTkG6z7mJuUlZ1ozUpEJApEbpBXVXlvlgXwZWt+dteu3rshxmAynq2t\n+ekiEmUiN8gLC303rurdm+9a+/P9hlcmoXFyEQmfyA1yv2EVBg5s/c/3W64/gS0YVLd+DSISFRTk\n4XLZZdQ+J6gvxYxgX+vXICJRQUEeErEBM1MMw8CIjQ0YGZ/I62GuQUSilYI8JCqpnZXiv/lHt4Jc\nRMJFQR5GW/z2r2IHHVv/K1cRiQIRGeTxnbtxcu9eb7vPlVfaUsdRYB/DAWhLBelk21KHiES2iAzy\ntt+V0qVm/zs68o2NM0ZexzcNUcMrIhIOERnkl/jtf8klgGFXKQFBPiFgsEVEJDQiMsgH+O0fIsmu\nMgDYwVWcoT0AgzjAD2ytRkQiUcQH+Vc2R+f3tOMtrvG2JwbpKyJyISI+yA8HtOwROE4uIhJaERnk\n/tfgTgvy64G48xcP6fa2ItICERnkTrsi/5zBHCYRgC5AGu9w/uIh3d5WRC6Uq4M8Pr573aXxhuG4\nIAdD0xBFJGxcHeTWVWzglW17TtOr5vVy4jhKP9vq8+cf5LqtrYiEkquDvD6JFHj3C/FgOuRH3MYN\nVGI9NeiHfEAP770RRURaxhkpF0IDOOzdd8awiuUkXckhDdBTg0QktCIuyH/AV959JwU5aLm+iIRH\no0GelZXF4MGDSUlJYfny5fX2yc7OZvTo0QwbNoz09PRQ19gsTr0ih/qC3LSvGBGJGLHBXqyqquK+\n++5j27ZtJCQkMHbsWDIyMkhNTfX2KS0t5d577+X111/H4/FQUmLv2K9/kNu9qvN87zOG40APoB9f\nM5KP+IhRdpclIi4X9Io8NzeX5ORkkpKSiIuLY+bMmaxbty6gz9///nduueUWPB4PAD179gxftU3g\n5CvyatoE3DZLs1dEJBSCXpEXFRWRmJjobXs8HnJycgL65OXlUVFRwbXXXktZWRn3338/t99+e51z\nLV682Lufnp4etiEYJwc5QBYwq2Z/Elks53/bWY6IOEh2djbZ2dnNfl/QIDeMxm//WlFRwZ49e9i+\nfTtnzpzhiiuuYNy4caSkpAT08w/ycDGoDph+WEBikN728P+K80p20plTlBFvWz0i4hznX+QuWbKk\nSe8LOrSSkJBAQYFfMBYUeIdQaiUmJjJhwgTat29Pjx49+PGPf8xHH33UjNJDpw/FXEQ5AMfpzmk6\n2VJHMMXAHkYDEEcl17Pd3oJExPWCBvmYMWPIy8vj0KFDlJeXs2bNGjIyMgL63HTTTezYsYOqqirO\nnDlDTk4OQ4YMCWvRDXH6sEqtzUz27mucXERaKmiQx8bGsnLlSiZOnMiQIUO47bbbSE1NJTMzk8zM\nTAAGDx7MpEmTGDFiBGlpaSxYsMARQe60GSv+spjk3Z/MZjQNUURawjBNM+wpYhgG4fgYawzfd95f\n8BiP8SsAVnAfi1jh35v6A7M5x0NzjjZUUEJPunISgCF8wmcMDcuvkYi4V1OzM6JWdiZQ5N134hed\ntaqIZRs3eNtT2GRjNSLidhEV5B4KvftFJNhYSeM2McW7ryAXkZaIqCD3vyJ3epD7f+F5Ne/Q2cZa\nRMTdFOQ2+Zp+fMBlgDUN8YZG+ouINCRigtygmv4c8badHuQQOLxyo411iIi7RUyQ9+IYbakA4Fvg\nLB3sLagJAsfJAc1aEZELEDFBHjis4g65XE4JPQCsB9J9+KGt9YiIOynIbVRNm4DFQbz2mn3FiIhr\nKcht5j+8wsaN9hUiIq6lILdZFpO8D2UmNxeKi+0tSERcJ2KC3H8xUGGQfk5zgu7s5EqrYZoaXhGR\nZouYIHfrFTnABqb6GhpeEZFmUpA7QECQb9kC587ZV4yIuI6C3AEOMIgDtY3Tp5nUvj2GYWAYBvHx\n3e0sTURcICKCvCPfeW8J+z1tKbG5nguxwW9/Kvdg3f7WpKzshE0ViYhbRESQu+keKw3xHxmfygb0\nsAkRaSoFuUPsAL6lGwADKGA0e+0tSERcQ0HuEJXAa363zprGq/YVIyKuoiB3kFeZ5t1XkItIU0VE\nkAcuBvLYWEnLvM5EznERACP4mEv4wuaKRMQNIiLII+WK/DSd2Mp4b/sm1tlYjYi4RUQEudseKBGM\nhldEpLkiLsiPWnf2dq0NTKUaA4Cr2EEvm+sREedzfZAbVNOXr71ttwf5MXp7b6IVg+l3fS4iUj/X\nB3lPSoijErDmYZ+jvc0VtdzL3OLdvyVIPxERiIAg9x9WOUJ/GysJnbVM9+5fB/Dtt7bVIiLOpyB3\noAIGsJs0AOIA1q+3tR4RcTYFuUP5D6/w8sv2FSIijqcgd6iAIN+yBU6dsq8YEXE0BblD5XMJexht\nNcrLYcOG4G8QkailIHewf3Krr7FmjX2FiIijKcgdbA23+RpZWXBCD5kQkboU5A6WzyXk1jYqKuCV\nV+wsR0QcytVB3gboQ7G3/TV97SsmTF70b2h4RUTqYZimGfZnihmGQTg+pr9heK/Hv6EXffim9hOp\n+6i0+o4193jrnyMBw3eT3jZt4MgR6N27nveLSKRpana6+orcfyAl0oZVahUBXHWV1aiqgpdesrMc\nEXGgRoM8KyuLwYMHk5KSwvLlyxvs99577xEbG8vatWtDWmAw/rfHcvvNsoKaOdO3/8IL9tUhIo4U\nNMirqqq47777yMrK4tNPP+WFF17gs88+q7ffww8/zKRJk8IyhNKQaLgiB+DWW61hFYAdOyA/3956\nRMRRggZ5bm4uycnJJCUlERcXx8yZM1m3ru5Ta1asWMGMGTPo1at1754dNUHeuzdMmuRrP/+8fbWI\niOPEBnuxqKiIxMREb9vj8ZCTk1Onz7p163jjjTd47733MAyj3nMtXrzYu5+enk56evqFV10jaoIc\nYM4ceO01a/+55+A//xMa+LUWEXfKzs4mOzu72e8LGuQNhbK/Bx54gEcffdT77WpDQyv+QR4qURXk\nGRnQuTOUlcGBA/Dee3D55XZXJSIhdP5F7pIlS5r0vqBBnpCQQEFBgbddUFCAxxP4lPoPPviAmTVf\nxpWUlLB582bi4uLIyMhoau0XLKqCvEMHuOUWeOYZq716tYJcRIBG5pFXVlYyaNAgtm/fTv/+/bn8\n8st54YUXSE1Nrbf/vHnzmDp1KtOnTw84Hq555MWGQZ+a/QQKOeJ98LIz5oCH6hzeX7vt2+GGG6z9\nnj2hqAjatq3nPSISCUIyjzw2NpaVK1cyceJEhgwZwm233UZqaiqZmZlkZmaGrNgLUlHhDfFqDIq9\nrQiWng61/yIqKYGNG20tR0Scwb0rOwsKYMAAAI7Sl/4c9f9EnHI13fJzxEHNM0kBlgK/rW3ceKPC\nXCSCRf7KzqIi725kj49XYgW8tT3DQd9LmzdbS/ZFJKq5N8j9AqzIOzYe+b5kIG/WNqqr4dln7SxH\nRBwgIoI8sq/I61oV0FgFrbiaVkScR0HuQi+DNaccIC8P3n7bznJExGYKchc6CzB7tu+A3TOIRMRW\nCnK3+vnPffsvvQTHjtlXi4jYyr1BHjWzVhowciSkpVn7FRW+FZ8iEnXcG+RROmslgP9VeWamNYtF\nRKKOOxcEnTkDHTsCUE4c7TiHGfB3klMW84TrHNYiofZYTxDqVnN0WvtOvHqmrJ5ziIgbRfaCoKO+\nVZxH6XdeiEcDa5HQWUz+xv3eo/PPfmdfSSJiG3cmYLR/0ennSe7x7v8E4IsvbKtFROzhziCP9i86\n/eRxKZuYDNT8Zq5caWs9ItL63BnkuiIP8Ge/4RVWrbIePiEiUcP1QR61M1b8bGU8nzPIapw6BX/7\nm70FiUircn2Q64ocTGJYwX/4Djz+OFRV2VeQiLQqBXmE+Bt38G1t48svYe1aO8sRkVbkziDXl511\nnKYTT/of+OMfdVdEkSjhvgVBpgmdOlmLgoCunOAkXc//RJy7mCd85+hNLF9RRbuadjqwp3M3Tp36\ntk5fEXG+yF0QdOqUN8TPACfpYm89DvINVTzDQm/7fzGZsrITNlYkIq3BfUEeMGMFrKtTqfUYv6S6\n5tdkCpu5zOZ6RCT83BfkhYXeXT2tsq6DpPBPbvW2f2NjLSLSOlwd5AU2luFkv/eL75sBPvrItlpE\nJPzcF+QFvvguDNItmv2L4bzMdN+B3//evmJEJOzcF+R+V+QK8ob5X5Xz8suwb599xYhIWLk6yDW0\n0rAPGc06MqyGacJvf2tvQSISNu4Lcg2tNNlv+T++xvr1sHt3o++Jj++OYRgBW3x89zBWKSIt5b4g\n1xV5k33MCP7uf+DXv250tac179wM2DQXXcTZ3BXk330HpaXWftu2lNhbjSv8DqBNG6vx5puwdaud\n5YhIGLgryP2uxklIqHdBuwQ6CHDnnb4Dv/qV7owoEmHcG+SJifbV4Ta/+533YdV8/DE8/bS99YhI\nSLkiyGu/gJs3frz32PNvv21jRS7Tvz88/LCv/Zvf6ClCIhHEFUFe+wWch6XeYwU83PAbpK5f/hIS\nap6mVFwMy5bZW4+IhIwrgrxWIv5TDz02VuJCHTrAI4/42n/6Exw4YF89IhIyrgpyD/6rOhXkzTZn\nDowbZ+1XVMB//IcePiESAVwb5AXoy85mi4mBJ5+0/guwZQszYmICFv+IiPs0GuRZWVkMHjyYlJQU\nli9fXuf1559/npEjRzJixAiuvPJK9oXxnh4aWgmB0aPhnnu8zT/Tn3hK8S0AEhHXMYOorKw0Bw4c\naObn55vl5eXmyJEjzU8//TSgz65du8zS0lLTNE1z8+bNZlpaWp3zNPIxjQLMjpSZpjUQYJ6jrWlQ\nVZM8Zj1bfceb0zeSzlHPr/2JE+bXfp3+wsLmn0NEwq6p/+8FvSLPzc0lOTmZpKQk4uLimDlzJuvW\nrQvoc8UVV9Cli/W4tbS0NAoLw3MHFP9hlSISMN01KuQsXbtyn1/z52TyY96yrRwRaZnYYC8WFRWR\n6LfwxuPxkJOT02D/p556iilTptT72uLFi7376enppKenN6tQfdEZWi8BrzCNm3kVgP/HXYzkI87a\nW5ZIVMvOziY7O7vZ7wsa5M358uvNN99k1apV7Ny5s97X/YP8QviPj+uLztC4lye4ljfpyklSOMgf\neSjgSl1EWtf5F7lLlixp0vuCjk8kJCRQ4Hfb2IKCAjyeulfD+/btY8GCBaxfv55u3bo1seTmGcBh\n776uyEPjKP15kMe97Xt5kkk21iMiFyZokI8ZM4a8vDwOHTpEeXk5a9asISMjI6DP4cOHmT59OqtX\nryY5OTlshV5Mvnf/Sy4J2+dEm2f4d15hmre9Cuih+0qKuErQII+NjWXlypVMnDiRIUOGcNttt5Ga\nmkpmZiaZmZkALF26lBMnTnD33XczevRoLr/88rAUeglfevfzuTgsnxGdDH7GXymmNwD9sMLdoNre\nskSkyYyaKS7h/RDDoCUfYxgGh/GQWPOFZzJ5fEEyYFD/3Of6jjenbySdIw6orKcvAX0ns4lN3Oht\nP8Qf+BMPec/bCn9MROQ8Tc1OVwR5O8PgDAYxmFQRQ3vOUkFbnB2gTjlH0/v+gYd4iD8BUEkbruEt\ndnElCnIRe0RUkA8yDPbX7H/FAJL4qvbMODdAnXKOpveNpYK3acsVNe2j9OWHfMBREhTkIjZoana6\nYlWN/4i4vugMn0rimAkcx3rYcj++Zi3TucjeskSkEa4Icv/oVpCH12Hg3/gHlVjP+RxHDn8BdJdE\nEedyXZBrxkr4vcH1/KpmrBxgHsB//Zdt9YhIcK4Icg2ttL4/cz+rrAi3/Pa3sHq1fQWJSINcEeQa\nWrGDwc/5b7Zxve/Q/PmQlWVfSSJSL+fPWjFNTsbE0KWm2ZtijtUsXnH2bBGnnKNln9eFUnbQjWG1\nB9q3hy1b4Kqr6nmviIRS5MxaOXHCG+Kn6cAxetlaTrQ5SVcmAwwYYB04exZuvBHef9/OskTEj/OD\n/Evf0nxrWEWPI2tthQDbtkGfPtaBU6fghhsgN9fOskSkhguDXGyRkmINqXS35phz8iSMHw/vvmtv\nXSLigiDP9931UFMPbTZiBLzxBvToYbVrr8w3b7a3LpEo5/wg1xW5A8RiGIa1jRrF8OPH+ab2pTNn\nICMDnnvOzgJFoprzg/zgQe+ugtwulVgzWaztX5hcxX4OeV+uhLlzYfFirQAVsYHzg/zjj727nzLE\nxkLEXx6XciXA0KG+g0uWwE9/al2li0ircXaQFxfDsWMAfEdHDpFkbz0S4AjAjh3Wl561XnyRfZ3j\nuaR2KKZmi4/vbleZIhHP2UHudzX+L4ZhOrzcqNS1K7z2Gvz8595DI6qr+ICuTGMttcMxZWUnbCtR\nJNI5Oxn9gvxjhttYiAQVFwd/+Qv89a/Qti0A3SjlFabzBPfQjrM2FygS2RTkEjoLFsCOHd7HfgDc\nw1/4kFGk2VaUSORzdpDv2+fdVZC7xNixjAJe4hbvoUEcYCfAr34Fp0/bVZlIxHLuTbOqqqBTJzh3\nDoCeHOM4Pc8/M869WZVTzhHez6vv99UwDKCa+azicR4knjLfiwMGwIoVMHUqGLrdgkgw7r9p1hdf\neEP8KNQT4mK/2ICZKbWbxWAVdzKcj9nKDb63HD4MN90EkyfDZ5/ZUrVIpHFukAeMj4szBS4U8m0+\nh/kBE9jCXPAt7Qd4/XUYPhwWLoQjR1qrYJGIpCCXVmDwHMD+/dY0xZiaP3ZVVdZMl+Rk+MUv4Ouv\n7SxSxLUU5NJ6evSwpinu2QPXXec7fvYsPP44XHwx3HdfwP11RKRxCnJpfSNHWvc337IFLrvMd/zc\nOXjiCeuWuTNmwFtv6d4tIk3guCBftOghEi/qCHl5AFQAn9pbkoSDYVhL+99/H9avhzFjfK9VV8PL\nL0N6Oh/HxLDIMOhuGBhG23q/XNXyf4l2jgvyQ4eOcE35LG97F1dyzsZ6JMwMw5qKmJsLW7fChAkB\nLw8H/i9whIv4BxVMZR1xfI//l6ta/i/RznFBDjAJ37S0LG60sRIJt/j47taVdUwMxvjxGFu2MAz4\nC9YzWmu143tuBdZzE8X04SnmM4XXuEh/zYs4L8gN02QC//K2X2eijdVIuFlX04HTFz/B5B6gP0dY\nyH/zPj8MeE83SpnP07zGTyihJy8BrFqlaYwStRy3svPBH0/m8XeyACimN/04ikkb3Lei0inncErN\ncVjzzuvT+DmG8AlzGMYsfkBSwN1czjN0qPX4uWuvhauv9j1jVMSFmpqdjgvyZ1NHMffzjwB4jjnM\n5TmcE0ZuPIcbaw52jmrG8h4zeIlpvMql5NXz3truBgwbBj/6kbVdfjlceqlvHruIw7k2yP/VozfD\nvrUeJjGb1fyd2TgrSNx2DjfW3NRzmAzmc6YwhMeuvdZ6yEVFRT3n8hMfD6NHW9MeR42yHiidmgoX\nXRT8fSI2cGeQl5ZS1b07bUyTagz6UEwJvXBukLjhHG6sufnnME3TurPijh2wfbs1B/2DD6zVo41p\n08ZaXTpkCAwebG0pKdaxnj11cy+xTVOzM7YVamm65ctpU1P0B/ywJsRFGhPrd7MuS+fO3ThVWmpN\na9y50/pvTo730YEBqqqs2wfs31/npZPAl0A+cAj4Cihp15Hn334TPB7o3dv6i0AcKT6+e73TUzt3\n7sapU9/aUFF4OOeKPD/fuhIqLwfgdp5lNbfXngGnXxE69xxurDkU56jnz5xpkhgTw2W8ymXsYTgf\nc47dzORrYuo9bxPExkKfPtCvn7X16WNtvXtbW8+e0KuXdXuCHj2gffsL+5wLlJ2dTXp6eqt+Zmtq\n7Oez/oIP8q84hwvZFXlWVhYPPPAAVVVV3HXXXTz88MN1+ixatIjNmzfToUMHnnnmGUaPHt38ih96\nyBviOQzkeWY3/xwiwRgGhUAhN7Gem2oOLmYBDzGYTqTyHIPYzyD2k0IeKXxIp8bOWVkJRUXW1hTt\n2lkzabqiqH5HAAAGVklEQVR1s5532rUrdOni2+Ljra1zZ2vr1Mm3dexobR06WH8hxDb+D+poD/Jo\nEfRPQlVVFffddx/btm0jISGBsWPHkpGRQWpqqrfPpk2bOHjwIHl5eeTk5HD33Xeze/fupldw4gQs\nXWotya5xP7ejBy1Ly9QdbmnIGTqyB9jDnPNeMehFMQP5gh/wFReTzwAOk0gmAzBJAHrUc76gzp2z\n5ruHYs5727ZWoLdvb4V7u3bWfrt21nbRRda/dA8c4KG1r/Kn7+s+O7W+IYbmDkfU1z/Shi6cLmiQ\n5+bmkpycTFJSEgAzZ85k3bp1AUG+fv167rjjDgDS0tIoLS2luLiYPn36BJ5s6lTrv6ZpbZWV1njl\nwYNQ5nuCTHZCEjlFySH40SS61d4r/XzN++LyGL05Rm92c4Xf0f/2nrsdZ+nL1/TjKH25krVPPAHf\nfOPdsl9+mZ4MpQfH6cFx2tLIrJrmKC+3tpMng/f77DOsO9nU/fUoK6v76+FbpNV434b6N9RXwiNo\nkBcVFZGYmOhtezwecnJyGu1TWFhYJ8iNjRubVlHRIWBOzRZwhgbe0Jzj0XoON9YcinM0te+SCz7H\nOawvQQ/VHr333nr6ftLAOVrHEu9e/T9L/f9yaU7f+vs39V9ELbVkyZJGejT3Z3GfoEHe1B/0/MH4\n89/nhi8VRETcKuhAdEJCAgUFBd52QUEBHo8naJ/CwkISEhJCXKaIiDQkaJCPGTOGvLw8Dh06RHl5\nOWvWrCEjIyOgT0ZGBs8++ywAu3fvpmvXrnXHx0VEJGyCDq3ExsaycuVKJk6cSFVVFXfeeSepqalk\nZmYCsHDhQqZMmcKmTZtITk6mY8eOPP30061SuIiIWMK+IKgp89Ddav78+bz22mv07t2bjz+OvAfS\nFRQUMHfuXL755hsMw+BnP/sZixYtsruskDh37hzXXHMN33//PeXl5dx0000sW7bM7rJCrqqqijFj\nxuDxeNiwYYPd5YRUUlIS8fHxtGnThri4OHJzc+0uKaRKS0u56667+OSTTzAMg1WrVjFu3Lj6O5th\nVFlZaQ4cONDMz883y8vLzZEjR5qffvppOD+yVb399tvmnj17zGHDhtldSlgcPXrU3Lt3r2mapllW\nVmZeeumlEfX7d/r0adM0TbOiosJMS0sz33nnHZsrCr3HHnvM/OlPf2pOnTrV7lJCLikpyTx+/Ljd\nZYTN3Llzzaeeeso0TevPaGlpaYN9w7rqxn8eelxcnHceeqS4+uqr6datm91lhE3fvn0ZNWoUAJ06\ndSI1NZUjEfTwhg4drCcQlZeXU1VVRfcIu3d5YWEhmzZt4q677orYmWOR+nOdPHmSd955h/nz5wPW\nMHeXLl0a7B/WIK9vjnlRU5cyi6McOnSIvXv3kpaWZncpIVNdXc2oUaPo06cP1157LUOGDLG7pJB6\n8MEH+eMf/0hMhN5/3TAMbrjhBsaMGcP//M//2F1OSOXn59OrVy/mzZvHZZddxoIFCzhz5kyD/cP6\nOxxJE+6j2XfffceMGTP485//TKdOjd59xDViYmL48MMPKSws5O233yY7O9vukkJm48aN9O7dm9Gj\nR0fsVevOnTvZu3cvmzdv5oknnuCdd96xu6SQqaysZM+ePdxzzz3s2bOHjh078uijjzbYP6xB3pR5\n6OJsFRUV3HLLLcyZM4dp06bZXU5YdOnShRtvvJH333/f7lJCZteuXaxfv56LL76YWbNm8cYbbzB3\n7ly7ywqpfv36AdCrVy9uvvnmiPqy0+Px4PF4GDt2LAAzZsxgz549DfYPa5A3ZR66OJdpmtx5550M\nGTKEBx54wO5yQqqkpITS0lIAzp49y9atWy/srp0O9cgjj1BQUEB+fj4vvvgi1113nXe9RyQ4c+YM\nZTX3aDp9+jRbtmxh+PDhNlcVOn379iUxMZEDBw4AsG3bNoYOHdpg/7A+WKKheeiRYtasWbz11lsc\nP36cxMREli5dyrx58+wuK2R27tzJ6tWrGTFihDfkli1bxqRJk2yurOWOHj3KHXfcQXV1NdXV1dx+\n++1cf/31dpcVNpE2zFlcXMzNN98MWMMQs2fPZsKECTZXFVorVqxg9uzZlJeXM3DgwKBrdFrlwRIi\nIhI+kfl1tohIFFGQi4i4nIJcRMTlFOQiIi6nIBcRcTkFuYiIy/1/SGC1FjhVh6MAAAAASUVORK5C\nYII=\n", 59 | "text": [ 60 | "" 61 | ] 62 | } 63 | ], 64 | "prompt_number": 20 65 | }, 66 | { 67 | "cell_type": "code", 68 | "collapsed": false, 69 | "input": [ 70 | "pw=[i*(S.Heaviside(u-j/2)-S.Heaviside(u-j/2-1/2)) for i,j in zip(S.symbols('a:8'),range(8))]\n", 71 | "pwf = lambda i: sum(pw).subs(u,i)\n" 72 | ], 73 | "language": "python", 74 | "metadata": {}, 75 | "outputs": [], 76 | "prompt_number": 162 77 | }, 78 | { 79 | "cell_type": "heading", 80 | "level": 2, 81 | "metadata": {}, 82 | "source": [ 83 | "Rectangle Wedge Tail Decomposition" 84 | ] 85 | }, 86 | { 87 | "cell_type": "markdown", 88 | "metadata": {}, 89 | "source": [ 90 | "\n", 91 | "$$ f(x) = \\exp\\left(-\\frac{(x-1)^2}{2x} \\right)(x+1)/12 $$ \n", 92 | "\n", 93 | "where $x>0$" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "collapsed": false, 99 | "input": [ 100 | "x = linspace(0.001,10,100)\n", 101 | "f= lambda x: (sqrt(1/x) + sqrt(x))/ (2.*pow(exp(1),pow(-sqrt(1/x) + sqrt(x),2)/ 2.)*sqrt(2*pi)*x)\n", 102 | "fx = f(x)\n", 103 | "plot(x,fx)" 104 | ], 105 | "language": "python", 106 | "metadata": {}, 107 | "outputs": [ 108 | { 109 | "output_type": "pyout", 110 | "prompt_number": 13, 111 | "text": [ 112 | "[]" 113 | ] 114 | }, 115 | { 116 | "output_type": "display_data", 117 | "png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD9CAYAAABDaefJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt01OWdx/H3wAzlVkGuwkzYWBLJRC6JJiLgZRQ1lGq8\nYNfYY3UpIFXZFbeuvZz1NOy6SLxUWaPb1GNXrTakFjVUZbQRR6lIolKLCkJgTR2iUMMdI4QMs388\nzQ3CZJLMzG8un9c5z5n5ZZ78ni85+s2T7/P8fj9bMBgMIiIiSaGP1QGIiEjkKKmLiCQRJXURkSSi\npC4ikkSU1EVEkoiSuohIEukyqXu9XrKyssjMzKSkpOSEzxsaGpg1axY5OTlMnDiRJ598MhpxiohI\nGGyh9qkHAgEmTJhAVVUVTqeT/Px8ysvLcbvdrX2Ki4s5cuQI9957Lw0NDUyYMIFdu3Zht9tj8g8Q\nEZE2IWfqNTU1ZGRkkJ6ejsPhoKioiMrKyg59xowZw4EDBwA4cOAAw4cPV0IXEbFIyOxbX19PWlpa\n67HL5aK6urpDnwULFnDxxRczduxYDh48yO9+97sTzmOz2SIUrohIaunuRf8hZ+rhJOOlS5eSk5PD\n559/zgcffMBtt93GwYMHOw1MLcjPf/5zy2OIl6afhX4W+lmEbj0RMqk7nU78fn/rsd/vx+Vydeiz\nbt06vvvd7wIwfvx4Tj/9dLZs2dKjYEREpHdCJvW8vDxqa2upq6ujqamJiooKCgsLO/TJysqiqqoK\ngF27drFlyxa+9a1vRS9iERE5qZA1dbvdTmlpKQUFBQQCAebNm4fb7aasrAyAhQsX8rOf/Yy5c+cy\nZcoUjh07xn333cewYcNiEnwi8ng8VocQN/SzaKOfRRv9LHon5JbGiA1is/W4PiQikqp6kjt1RamI\nSBJRUhcRSSJK6iIiSURJXUQkiSipi4gkESV1EZEkoqQuIpJElNRFRJKIkrqISBJRUhcRSSJK6iIi\nSURJXUQkiSipi4gkESV1EZEkoqQuIpJElNRFRJKIkrqISBJRUhcRSSJK6iIiSaTLpO71esnKyiIz\nM5OSkpITPn/ggQfIzc0lNzeXSZMmYbfb2bdvX1SCFRGR0EI+eDoQCDBhwgSqqqpwOp3k5+dTXl6O\n2+3utP9LL73Eww8/TFVVVcdB9OBpEZFui/iDp2tqasjIyCA9PR2Hw0FRURGVlZUn7f/b3/6W66+/\nvlsBHK+pCWpq4KGH4PbbIRDo1elERFKKPdSH9fX1pKWltR67XC6qq6s77dvY2Mirr77KY4891unn\nxcXFre89Hg8ej+eEPm++Cd/5DowfDzNmwMsvw003wVlnhfEvERFJcD6fD5/P16tzhEzqNpst7BP9\n4Q9/4LzzzmPo0KGdft4+qZ/Me+/B/Pnw8MPmeNEieP11JXURSQ3HT3iXLFnS7XOELL84nU78fn/r\nsd/vx+Vyddp3xYoVvS69NDTAqFFtxxdfDGvW9OqUIiIpJWRSz8vLo7a2lrq6OpqamqioqKCwsPCE\nfvv37+ett97iyiuv7FUwX34JI0a0HXs88Pbbps4uIiJdC5nU7XY7paWlFBQUkJ2dzXXXXYfb7aas\nrIyysrLWfi+++CIFBQUMGDCgV8E0NMDIkW3Hw4bBGWfAScr4IiJynJBbGiM2SJjbcmbMgGXL4Pzz\n2752110waBD8/OdRDFBEJA5FfEtjrH35ZceZOsDMmWaxVEREuhZXM/Vhw2Dr1o519a++gtGjYdcu\nM2MXEUkVCT1TP3oUDhwwib29QYPMlsY//cmauEREEkncJPU9e0xC79NJRBdfrBKMiEg44iapH7+d\nsb2ZM7VfXUQkHHGV1I9fJG0xdaqpte/ZE9uYREQSTdwk9YaGk8/U+/WD6dOhl7dEEBFJenGT1EPN\n1MHsYV+/PnbxiIgkorhJ6sdfTXq8/Hxzwy8RETm5uEnqoRZKAfLy4P334dix2MUkIpJo4iapdzVT\nHzHCbHmsrY1dTCIiiSZuknpXM3UwJZh3341NPCIiiSiuknqomTqYEoySuojIycVNUg+1pbGFFktF\nREKLixt6BYPQvz/s329eT+bAARg7FvbtA3vIB/GJiCS+hL2h18GD5gKjUAkd4JRTwOWCjz+OTVwi\nIokmLpJ6OIukLVSCERE5ubhJ6l0tkrbQYqmIyMnFRVIPZ5G0hbY1ioicXJdJ3ev1kpWVRWZmJiUl\nJZ328fl85ObmMnHiRDweT7eD6M5MPScHNm+Gw4e7PYyISNILuYckEAiwaNEiqqqqcDqd5OfnU1hY\niNvtbu2zb98+brvtNl599VVcLhcNDQ3dDqKrq0nbGzgQMjNh40Y455xuDyUiktRCJvWamhoyMjJI\nT08HoKioiMrKyg5J/be//S1z5szB5XIBMOIkdZTi4uLW9x6Pp8OMvjsLpdC2WKqkLiLJxOfz4evl\nPcZDJvX6+nrS0tJaj10uF9XV1R361NbWcvToUS666CIOHjzI7bffzve///0TztU+qR+voQEmTAg/\n6Px83YZXRJLP8RPeJUuWdPscIZO6zWbr8gRHjx5lw4YNvP766zQ2NjJt2jTOPfdcMjMzww6iuzP1\nvDx45JHw+4uIpIqQSd3pdOL3+1uP/X5/a5mlRVpaGiNGjGDAgAEMGDCACy64gL/85S/dTurh1tQB\nJk2CTz+FQ4dg8ODwv09EJNmF3P2Sl5dHbW0tdXV1NDU1UVFRQWFhYYc+V155JX/6058IBAI0NjZS\nXV1NdnZ2t4LozkIpmKtPJ02CDRu6NYyISNILOVO32+2UlpZSUFBAIBBg3rx5uN1uysrKAFi4cCFZ\nWVnMmjWLyZMn06dPHxYsWNDtpN7d8gu07Ve/4ILufZ+ISDKz/IZeTU0waJB5DaOE3+rpp+Hll6Gi\nIkJBiojEmYS8oVdDAwwf3r2EDrqyVESkM5Yn9e4ukraYMMH8QujBtU4iIknL8qTenfu+tNenD5x9\ntu7YKCLSnuVJvaczdTBXlKoEIyLSxvKk3t3tjO2pri4i0pHlSb0n2xlb5OdDTY15HJ6IiMRBUu/N\nTH3cODh2DHbsiGxMIiKJyvKk3puZus2mEoyISHuWJ/We7n5poaQuItLG8qTe2GiuKO0p7YAREWlj\neVI/cgS+8Y2ef3/LAzOOHYtcTCIiicrypH74MPTv3/PvHzkSTj0VamsjF5OISKKyPKn3dqYOcO65\n8M47kYlHRCSRWZ7UeztTB5g+Hdati0w8IiKJzPKkHomZupK6iIiRFEl98mT4619h377IxCQikqgs\nT+qRKL84HOZh1OvXRyYmEZFEZWlSb242WxHtIR+qFx6VYEREwkjqXq+XrKwsMjMzKSkpOeFzn8/H\nkCFDyM3NJTc3l3vuuSfswY8cMbP07j71qDPTp2sHjIhIyDlyIBBg0aJFVFVV4XQ6yc/Pp7CwELfb\n3aHfhRdeyKpVq7o9eCTq6S3OPReqqyEQgL59I3NOEZFEE3KmXlNTQ0ZGBunp6TgcDoqKiqisrDyh\nX0+fXd0yU4+E4cPB6YSPPorM+UREElHImXp9fT1paWmtxy6Xi+rq6g59bDYb69atY8qUKTidTh54\n4AGys7NPOFdxcXHre4/Hg8fj4fDhyM3Uoa2uPmVK5M4pIhIrPp8Pn8/Xq3OETOq2MIrdZ511Fn6/\nn4EDB7J69Wquuuoqtm7dekK/9km9RSTLLwDTpsGbb8Itt0TunCIisdIy4W2xZMmSbp8jZPnF6XTi\n9/tbj/1+Py6Xq0Ofb37zmwwcOBCAb3/72xw9epQ9e/aENXgktjO2px0wIpLqQib1vLw8amtrqaur\no6mpiYqKCgoLCzv02bVrV2tNvaamhmAwyLBhw8IaPNIz9aws2LMHdu6M3DlFRBJJyPKL3W6ntLSU\ngoICAoEA8+bNw+12U1ZWBsDChQv5/e9/z//8z/9gt9sZOHAgK1asCHvwSC6UAvTpY0ow77wDV18d\nufOKiCQKW7CnW1e6M4jN1ukOmddeg/vvhz/+MXJj3XMP7N0LDz4YuXOKiFjhZLkzFEuvKI10+QXA\n4zGLpSIiqcjSpB7phVIwT0LaskU39xKR1JR0M/VvfMNcXfrWW5E9r4hIIrA8qUd6pg5w0UXQy/37\nIiIJyfLyS6Rn6mDq6m+8EfnziojEO8tn6tFI6vn5sH272bMuIpJKLJ+pR6P84nCYq0u1C0ZEUk1S\nztRBdXURSU2WJ/VozNRBdXURSU2Wl1+iNVM/+2zzMOovv4zO+UVE4pHlM/VoJXW7Hc47T/vVRSS1\nWD5Tj1b5BUxdXSUYEUklSTtTB9XVRST1WJ7UozlTz82FXbug3XM+RESSmuXll2jO1Pv2hcsuA683\nemOIiMQTy2fq0UzqALNnw+rV0R1DRCReWJ7Uo1l+ASgogDVroKkpuuOIiMSDpC6/AIwcCWecAW+/\nHd1xRETiQdLP1AG+/W2VYEQkNXSZ1L1eL1lZWWRmZlJSUnLSfu+++y52u53nn38+7MFjMVMH1dVF\nJHWETOqBQIBFixbh9XrZtGkT5eXlbN68udN+P/7xj5k1a1a3HpIai4VSgLw82LlTWxtFJPmFTOo1\nNTVkZGSQnp6Ow+GgqKiIysrKE/o98sgjXHvttYwcObJbg8eq/NKytVGzdRFJdvZQH9bX15OWltZ6\n7HK5qK6uPqFPZWUla9as4d1338Vms3V6ruLi4tb3Ho8Hj8cTs/ILmBLM738PN98cm/FERLrL5/Ph\n6+U9w0Mm9ZMl6PYWL17MsmXLsNlsBIPBk5Zf2if1FrGaqYPZ2njrrWZrY79+sRlTRKQ7Wia8LZYs\nWdLtc4RM6k6nE3+7QrTf78flcnXo8/7771NUVARAQ0MDq1evxuFwUFhYGHLgYDB2NXWAESMgKwvW\nroWZM2MzpohIrNmCIVY2m5ubmTBhAq+//jpjx47lnHPOoby8HLfb3Wn/uXPncsUVV3DNNdd0HOTv\ns/j2mppg0CA4ejQC/4ow3Xsv7NgBjz4auzFFRHqqs9zZlZALpXa7ndLSUgoKCsjOzua6667D7XZT\nVlZGWVlZr4KNZemlxZw58MILcOxYbMcVEYmVkDP1iA3SyW+bL78EtxsaGqI9ekeTJsEvfwkzZsR2\nXBGR7or4TD2arJipg5mtr1wZ+3FFRGLBsqQey+2M7c2ZA88/bxZqRUSSjaUzdSuS+sSJZkvj++/H\nfmwRkWhLufKLzaYSjIgkr5Qrv0BbUlcJRkSSTcrN1AHOPtvsk//oI2vGFxGJlpScqdtscM01KsGI\nSPJJuYXSFv/4j1BerhKMiCSXlCy/AEydahJ6TY11MYiIRFpKll/AlGBuvBGeftq6GEREIi1lZ+oA\nN9wAFRUmFhGRZJCyM3WA9HRzMdLLL1sbh4hIpKTsQmmLm25SCUZEkkdKl1/AXIjk88X+bpEiItGQ\n0uUXgFNOge98B1assDoSEZHeS/mZOphdME89ZXUUIiK9l/IzdYBLLoEvvoCNG62ORESkd1J+oRSg\nb19YsAAee8zqSEREekfll79bsMDsWd+/3+pIRER6rsuk7vV6ycrKIjMzk5KSkhM+r6ysZMqUKeTm\n5nL22WezZs2asAaOp/ILwNixcOml8JvfWB2JiEjPhXzwdCAQYMKECVRVVeF0OsnPz6e8vBy3293a\n56uvvmLQoEEAfPjhh1x99dVs27at4yCdPDz1qqvgn/7JvMaLN9+EW26Bjz82txEQEbFSxB88XVNT\nQ0ZGBunp6TgcDoqKiqisrOzQpyWhAxw6dIgRI0aENXC8zdQBLrgA+vQx+9ZFRBKRPdSH9fX1pKWl\ntR67XC6qq6tP6Pfiiy/y05/+lC+++ILXXnut03MVFxe3vvd4PBw54om7pG6zwa23wqOPwkUXWR2N\niKQan8+Hr5ezypDll5UrV+L1enn88ccBeOaZZ6iuruaRRx7ptP/atWuZP38+W7Zs6ThIJ39CTJ8O\nDzxgXuPJgQPmnjAffghOp9XRiEgqi3j5xel04vf7W4/9fj8ul+uk/c8//3yam5vZvXt3lwPHY/kF\nzBWm3/uetjeKSGIKmdTz8vKora2lrq6OpqYmKioqKCws7NBn+/btrb9JNmzYAMDw4cO7HDjetjS2\nd8cdUFZmZu0iIokkZE3dbrdTWlpKQUEBgUCAefPm4Xa7KSsrA2DhwoWsXLmSp59+GofDweDBg1kR\n5k1U4unio+ONH2+2N/7qV3DnnVZHIyISvpA19YgN0kldyOWCd96BduuwceWDD8yNvv7v/+L3l4+I\nJLeI19SjKZ7LLwA5OTB5si5GEpHEoht6hfCTn8B990EgYHUkIiLh0Uw9hAsugOHD4YUXrI5ERCQ8\nliT1Y8eguRkcDitGD5/NZmbr//VfJmYRkXhnSVJv2fmSCPdXueIKc+uA55+3OhIRka5ZmtQTQZ8+\nsHQp/Pu/m78uRETimSVJPREWSdu77DIYPRqeftrqSEREQrNsph7vi6Tt2Wxw771QXGx+IYmIxCuV\nX8I0fTpMmWJuHyAiEq8sK78k0ky9xT33mBn7wYNWRyIi0jnN1LthyhRTX1+61OpIREQ6p4XSblq2\nDB5/HGprrY5EROREWijtprFj4a67zO15RUTijcovPbB4sZmpv/yy1ZGIiHSkhdIe6NcPli83yf3I\nEaujERFpo5l6D82aBdnZ8OCDVkciItJGC6W9sHw5/OIX8MknVkciImJoobQX0tNhyRL4wQ90z3UR\niQ8qv/TSLbeA3Q6lpVZHIiISRlL3er1kZWWRmZlJSUnJCZ8/++yzTJkyhcmTJzNjxgw2btzY5aCJ\nvlDaXp8+8MQT8J//Cdu3Wx2NiKS6kEk9EAiwaNEivF4vmzZtory8nM2bN3fo861vfYu33nqLjRs3\ncvfdd3PzzTd3OWgyzdQBMjPhpz+F+fP1MA0RsVbIpF5TU0NGRgbp6ek4HA6KioqorKzs0GfatGkM\nGTIEgKlTp7Jjx44uB02WhdL2Fi8291u//36rIxGRVGYP9WF9fT1paWmtxy6Xi+rq6pP2f+KJJ5g9\ne3annxUXF7e+37bNw7Rpnu5FGuf69oVnn4X8fPB4YOpUqyMSkUTj8/nw+Xy9OkfIpG7rxvPm3njj\nDX7961/z9ttvd/p5+6R+223JN1MHGDcOfvlL+N73YMMG+PsfMCIiYfF4PHg8ntbjJUuWdPscIcsv\nTqcTv9/feuz3+3G5XCf027hxIwsWLGDVqlWceuqpXQ6aTAulx7v6aigogB/+EIJBq6MRkVQTMqnn\n5eVRW1tLXV0dTU1NVFRUUFhY2KHPZ599xjXXXMMzzzxDRkZGWIMm20Lp8R58ED76CH71K6sjEZFU\nE7L8YrfbKS0tpaCggEAgwLx583C73ZT9/fE/Cxcu5D/+4z/Yu3cvt9xyCwAOh4OampqQgybjQml7\nAwbAypVw3nkwcSLMmGF1RCKSKmzBYPSLBDabjfbDXHEF3HyzeU1mXq+52rSmBjqpWomIhHR87gyH\nriiNolmz4PbbTZ3966+tjkZEUoFuvRtld90FGRmwYIEWTkUk+jRTjzKbzdxGYPt2+NnPrI5GRJKd\nbr0bAwMHwh/+AM8/rxt/iUh0hdz9Ei3Jcuvd7hgxAl591eyIGTMG5syxOiIRSUaWJfVUmqm3SE+H\nl16Cyy4zV5teconVEYlIstFCaYzl5JgyzPe+B2vWWB2NiCQbLZRa4Lzz4LnnoKgI3nzT6mhEJJko\nqVvkwgthxQr47nfhrbesjkZEkoV2v1jo4ouhvByuvRZeftnqaEQkGcQ8qTc3m73bdkuWaOPPzJlm\nu+O8eeZ+7CIivRHz1JrKi6QnM3UqvP66ua3Anj3wz/9sdUQikqhiPlNXPb1zZ54Ja9fCY4+Z+8U0\nN1sdkYgkIiX1OJKeDu+8A5s3Q2EhHDhgdUQikmhintS//lrll1CGDjWLpunpMH26uWeMiEi4Yp7U\nGxpg5MhYj5pYHA549FG49VaT2FetsjoiEUkUMU/qO3fCaafFetTEY7OZpF5ZCYsWwU9+ojq7iHRN\nST3OnXsuvP8+bNgAF10EdXVWRyQi8UxJPQGMHAmrV5vF0/x8eOYZPXBDRDrXZVL3er1kZWWRmZlJ\nSUnJCZ9/8sknTJs2jf79+/Pggw92OaCSes/07Qv/9m/wxz/CvffC9deb9QkRkfZCJvVAIMCiRYvw\ner1s2rSJ8vJyNm/e3KHP8OHDeeSRR7jzzjvDGlBJvXdycuC992DsWJg0ydxmQLN2EWkRMqnX1NSQ\nkZFBeno6DoeDoqIiKisrO/QZOXIkeXl5OByOsAZUUu+9AQPgF78wi6hLl5qyzGefWR2ViMSDkLcJ\nqK+vJy0trfXY5XJRXV3do4GKi4sB+OQTqKvzMG2ap0fnkTbnnGMWUUtK4Kyz4I474Ec/0nUAIonK\n5/Ph8/l6dY6QSd1ms/Xq5O0VFxcTDJoEdOWVETttyuvXD+6+G264Af71X2HiRHjoIbj8crMtUkQS\nh8fjwePxtB4vWbKk2+cIWX5xOp34/f7WY7/fj8vl6vYgLQ4cMElo4MAen0JO4vTT4YUXzIOt77rL\n3Nb33XetjkpEYi1kUs/Ly6O2tpa6ujqampqoqKigsLCw077BMFbrVE+Pvlmz4MMPzePyrrrKPF1p\n61aroxKRWAmZ1O12O6WlpRQUFJCdnc11112H2+2mrKyMsrIyAHbu3ElaWhoPPfQQ99xzD+PGjePQ\noUOdnm/nThg9OvL/COnIbocFC0wynzQJZsyAG29UchdJBbZgOFPs3g5isxEMBqmogJUr4Xe/i/aI\n0t7+/fDf/21aQYEpz0yebHVUItKVltzZHTG9olTlF2sMGWIWU7dtMwupBQUwezb4fNrjLpJslNRT\nyJAh5sZgn34KV18NP/wh5ObCE0+YWyKLSOJTUk9B/fubmvumTWaL6QsvwLhxcOedsGWL1dGJSG8o\nqaewPn1MKeall8wTl+x2uPBCuOACePppOMl6t4jEMSV1ASAjA5YtA78fFi82i9lpaWbXTFUVBAJW\nRygi4Yjp7pcxY8xl7WPHRntEiYRdu2DFCjNrr6+HOXPguuvgvPPMLF9Eoqsnu19iltSbm4P07w+N\njeZxbZJYamvhueegogL+9jdzq4errjJXrvbrZ3V0IskprpP6zp1BJk0yCUES27Zt8OKLZoH1449h\n5kz4znfMNkmV10QiJ66T+gcfBPn+92HjxmiPJrH0t7+B1wsvv2we4DFuHFx6KVx2mSnTDBhgdYQi\niSuuk7rXG+TBB+G116I9mliluRlqakxyf+01+MtfIC8PPB7zfNVzzlGSF+mOuE7qTz4Z5PXXzaKb\npIaDB+Htt+GNN8zVqx99BFOmmBn89Onmodoq14icXFwn9WXLguzeDffdF+3RJF599ZWZya9dC+vX\nm3bKKTB1qnmgdl6eedjHKadYHalIfOhJUg/5kIxI2rnT7HuW1DVokCnDXHSROQ4Gza6a6mrz3NUX\nXjAlmzFjzO0LcnLMzH7yZHC59NAPkXDEbKZeVBTkiivMfb5FTqa52dwi+M9/Nm3jRtOOHDE3Izvz\nTMjONq9ZWeaaByV7SVZxXX7xeILcfbfZ1yzSXV9+aR7+sWmTaR9/bJ5329gIEybAGWdAZmZbGz8e\nhg9XwpfEFtdJPSsryMqVZpYlEil795qbkG3ZYko527aZ1+3bTXln/HjzqL/TT4f0dPiHfzBt3Dhz\n10olfYlncZ3Uhw4Nsn07DBsW7dFETELfu9ck908/hbq6tvbZZ/DXv5qEnpZm6vUtbexYcDrN65gx\nMGoU9O1r8T9GUlZcJ/V+/YIcPqyZkcSHYNA8Ecrvhx072lp9PXz+uXnduRP27IERI8zWy9Gj29qo\nUaaNHNnWRowwD1XXf+MSKXGd1NPSgnz2WbRHin8+nw+Px2N1GHEhEX4WR4+aq2Z37WprO3eaGv/f\n/mZaQ4M5/vJL88ti+PCObdgw0049FYYONa8t71vaBx/4uPRSj9X/3LiQCP9dxEpUtjR6vV4WL15M\nIBBg/vz5/PjHPz6hz7/8y7+wevVqBg4cyJNPPklubu4JfXSRiaH/YNskws/C4TDlGKczvP6NjbB7\nt2kNDaYEtGdP29e2bTNf27vX/KWwb59pe/f6cDg8DBliav2nnGLaN79pWvv3gwef2AYNMm3gwLb3\n/fsn5t00E+G/i3gWMqkHAgEWLVpEVVUVTqeT/Px8CgsLcbvdrX1eeeUVtm3bRm1tLdXV1dxyyy2s\nX7/+hHMpqUsqGDjQtO5ek1FcbB4Ivn8/HDhg2sGD5vjgwbZ26JD5ZdHy/quv2l7bt8ZGOHzYJPZB\ng8ztGQYONK+dtf79215b2je+0fH9yVq/fm2vxzeHQ+WoWAuZ1GtqasjIyCA9PR2AoqIiKisrOyT1\nVatWcdNNNwEwdepU9u3bx65duxg9enSHcx13KCLHafmFMGZMZM537Jh59uxXX5nXxkbTvv7aJPyv\nvz7x/ZEj5nj//rb37V87a01Nbe3IEVOyannf3GyeqNWS4B2Oju/bN7vdvO7YAevWdfya3d6x9e3b\n+ddaWvs+7b9+stanT3jv27fOvhZOs9lOfG+zdXzf8nmPBEN47rnngvPnz289/s1vfhNctGhRhz6X\nX3558O233249njlzZvC9997r0AdQU1NTU+tB666QM3VbmH83HV/IP/77YrAWKyIidPGMUqfTid/v\nbz32+/24XK6QfXbs2IEz3FUlERGJqJBJPS8vj9raWurq6mhqaqKiooLCwsIOfQoLC3n67/fTXb9+\nPUOHDj2hni4iIrERsvxit9spLS2loKCAQCDAvHnzcLvdlJWVAbBw4UJmz57NK6+8QkZGBoMGDeJ/\n//d/YxK4iIh0ottV+G5avXp1cMKECcGMjIzgsmXLoj1c3Prss8+CHo8nmJ2dHTzzzDODy5cvtzok\nyzU3NwdzcnKCl19+udWhWGrv3r3BOXPmBLOysoJutzv4zjvvWB2SZZYuXRrMzs4OTpw4MXj99dcH\nDx8+bHVIMTN37tzgqFGjghMnTmz92u7du4OXXHJJMDMzM3jppZcG9+7d2+V5onppQss+d6/Xy6ZN\nmygvL2d1bUHsAAADWUlEQVTz5s3RHDJuORwOHnroIT7++GPWr1/Po48+mrI/ixbLly8nOzs77AX5\nZHX77bcze/ZsNm/ezMaNGztsGU4ldXV1PP7442zYsIEPP/yQQCDAihUrrA4rZubOnYvX6+3wtWXL\nlnHppZeydetWZs6cybJly7o8T1STevt97g6Ho3Wfeyo67bTTyMnJAWDw4MG43W4+//xzi6Oyzo4d\nO3jllVeYP39+Su+O2r9/P2vXruUHP/gBYEqeQ4YMsTgqa5xyyik4HA4aGxtpbm6msbExpTZdnH/+\n+Zx66qkdvtb+OqCbbrqJF198scvzRDWp19fXk9bu0jqXy0V9fX00h0wIdXV1/PnPf2bq1KlWh2KZ\nO+64g/vvv58+iXgdewR9+umnjBw5krlz53LWWWexYMECGhsbrQ7LEsOGDeNHP/oR48aNY+zYsQwd\nOpRLLrnE6rAs1f5CztGjR7Nr164uvyeq/0el+p/VnTl06BDXXnsty5cvZ/DgwVaHY4mXXnqJUaNG\nkZubm9KzdIDm5mY2bNjArbfeyoYNGxg0aFBYf2Ino+3bt/Pwww9TV1fH559/zqFDh3j22WetDitu\n2Gy2sHJqVJN6OPvcU8nRo0eZM2cON9xwA1dddZXV4Vhm3bp1rFq1itNPP53rr7+eNWvWcOONN1od\nliVcLhcul4v8/HwArr32WjZs2GBxVNZ47733mD59OsOHD8dut3PNNdewbt06q8Oy1OjRo9m5cycA\nX3zxBaNGjerye6Ka1MPZ554qgsEg8+bNIzs7m8WLF1sdjqWWLl2K3+/n008/ZcWKFVx88cWt1zqk\nmtNOO420tDS2bt0KQFVVFWeeeabFUVkjKyuL9evX8/XXXxMMBqmqqiI7xR+VVlhYyFNPPQXAU089\nFd5kMFrbc1q88sorwTPOOCM4fvz44NKlS6M9XNxau3Zt0GazBadMmRLMyckJ5uTkBFevXm11WJbz\n+XzBK664wuowLPXBBx8E8/LygpMnTw5effXVwX379lkdkmVKSkpatzTeeOONwaamJqtDipmioqLg\nmDFjgg6HI+hyuYK//vWvg7t37w7OnDmzW1saY/KQDBERiY3U3nogIpJklNRFRJKIkrqISBJRUhcR\nSSJK6iIiSURJXUQkifw/H7853mKQivAAAAAASUVORK5CYII=\n", 118 | "text": [ 119 | "" 120 | ] 121 | } 122 | ], 123 | "prompt_number": 13 124 | }, 125 | { 126 | "cell_type": "code", 127 | "collapsed": false, 128 | "input": [ 129 | "u=S.symbols('u')\n", 130 | "p=S.Piecewise(*[(f(j),(i,\n", 141 | " ]" 142 | ] 143 | }, 144 | { 145 | "output_type": "display_data", 146 | "png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD9CAYAAABDaefJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtclHW+B/DPM8xwVUAQUWYwVFAGL4hCXsvximtFlrZS\np+yYqVuxm53d41Zney3u6Zh02S5SLdurtmwLqUyx1KnIpryCSkol6mhOjSgI3hFlYHjOH+MgCA4z\nMDPPMPN5v17zmnlmfvP7fSX6+vP7/J7fI4iiKIKIiLyCTOoAiIjIeZjUiYi8CJM6EZEXYVInIvIi\nTOpERF6ESZ2IyIt0mNS1Wi0SExORkJCAnJycNp/X1NRg5syZGDlyJIYNG4Z3333XFXESEZEdBFvr\n1M1mM4YMGYKioiIolUqkpaUhPz8farW6uU12djbq6+vx3HPPoaamBkOGDEFVVRXkcrlb/gBERHSN\nzZl6SUkJ4uPjERcXB4VCgczMTBQWFrZq069fP1y4cAEAcOHCBURGRjKhExFJxGb2raioQGxsbPOx\nSqVCcXFxqzaLFi3ClClTEBMTg4sXL+Kjjz5q048gCE4Kl4jItzh60b/Nmbo9yXjFihUYOXIkTpw4\ngX379uGxxx7DxYsX2w2MDxF//etfJY/BUx78WfBnwZ+F7Udn2EzqSqUSRqOx+dhoNEKlUrVqs2PH\nDtxzzz0AgEGDBmHAgAE4dOhQp4IhIqKusZnUU1NTodfrYTAYYDKZUFBQgIyMjFZtEhMTUVRUBACo\nqqrCoUOHMHDgQNdFTEREN2Szpi6Xy5Gbm4v09HSYzWYsXLgQarUaeXl5AIAlS5bg6aefxoIFC5Cc\nnIympiY8//zziIiIcEvw3ZFGo5E6BI/Bn8U1/Flcw59F19hc0ui0QQSh0/UhIiJf1ZncyStKiYi8\nCJM6EZEXYVInIvIiTOpERF6ESZ2IyIswqRMReREmdSIiL8KkTkTkRZjUiYi8CJM6EZEXYVInIvIi\nTOpERF6ESZ2IyIswqRMReREmdSIiL8KkTkTkRZjUiYi8CJM6EZEXYVInIvIiHSZ1rVaLxMREJCQk\nICcnp83nL774IlJSUpCSkoLhw4dDLpfj3LlzLgmWiIhss3njabPZjCFDhqCoqAhKpRJpaWnIz8+H\nWq1ut/3nn3+OV155BUVFRa0H4Y2niYgc5vQbT5eUlCA+Ph5xcXFQKBTIzMxEYWHhDdt/+OGHuPfe\nex0K4HomswklFSV4eefLeFz7OMxN5i71R0TkS+S2PqyoqEBsbGzzsUqlQnFxcbtt6+rq8MUXX+CN\nN95o9/Ps7Ozm1xqNBhqNpk2bbw3f4rYPb8OgiEGYEDsBG/Ub8WDygxjVb5QdfxQiou5Np9NBp9N1\nqQ+bSV0QBLs7+uyzzzBx4kSEh4e3+3nLpH4je07swcOjHsYrM18BAGRtysLXP3/NpE5EPuH6Ce/y\n5csd7sNm+UWpVMJoNDYfG41GqFSqdtuuWbOmy6WXmss16BPSp/l4yoAp2GLY0qU+iYh8ic2knpqa\nCr1eD4PBAJPJhIKCAmRkZLRpd/78eXz33Xe48847uxRM9aVq9A7u3XysidNg+6/bYTKbutQvEZGv\nsJnU5XI5cnNzkZ6ejqSkJMybNw9qtRp5eXnIy8trbrd+/Xqkp6cjKCioS8HU1NUgKjiq+TgiKAKD\nIwej+Hj7dXwiImrN5pJGpw1i57KcCe9MwMqpK3HLTbc0v7fsq2UIUYTgr5q/ujJEIiKP4/Qlje5W\nfakaUSFRrd6bOmAqvj72tUQRERF1Lx6V1GvqalrV1AFgYv+JKD1ZikumSxJFRUTUfXhMUm8wN+BC\n/QVEBEW0ej/EPwSj+o3Ctl+3SRQZEVH34TFJ/czlM4gIioBMaBvSlAFTWIIhIrKDxyT16rrqNqUX\nq6kDpmLLMa5XJyLqiOck9XZOklqNUY3B4dOHcebyGTdHRUTUvXhMUm/vJKmVv58/xseOh86gc29Q\nRETdjMck9eq66lYXHl1vQuwE7Dq+y40RERF1Px6T1Gvqam5YfgGANGUa9pzY48aIiIi6H49J6tV1\n1egd1H75BQBSY1Kx9+ReNIlNboyKiKh78Zik3tFMvXdwb0QERUB/Wu/GqIiIuhePSerX79DYnrSY\nNOw+sdtNERERdT+ek9Q7OFEKWEowTOpERDfmMUnd1pJGq7QYniwlIrLFI5K6KIod1tQBYHTMaOyv\n3I/GpkY3RUZE1L14RFK/aLoIfz9/BMoDbbYLDQiFKlSFn0795KbIiIi6F49I6vacJLXienUiohvz\njKRux0lSq9R+PFlKRHQjHpHU7TlJapWm5LJGIqIb6TCpa7VaJCYmIiEhATk5Oe220el0SElJwbBh\nw6DRaBwOwtYOjdcb2XckyqvLcaXxisPjEBF5O7mtD81mM7KyslBUVASlUom0tDRkZGRArVY3tzl3\n7hwee+wxfPHFF1CpVKipqXE4iJq6GrvLL8GKYCREJqCsqgw3K292eCwiIm9mM6mXlJQgPj4ecXFx\nAIDMzEwUFha2Suoffvgh5syZA5VKBQDo3bv9Mkp2dnbza41G02pGb+sGGe2xrldnUicib6LT6aDT\n6brUh82kXlFRgdjY2OZjlUqF4uLiVm30ej0aGhowefJkXLx4EY8//jgeeOCBNn21TOrXq6mrwZDI\nIXYHnRaThl0V3IaXiLzL9RPe5cuXO9yHzaQuCEKHHTQ0NKC0tBRff/016urqMG7cOIwdOxYJCQl2\nB+HoTD01JhWrSlbZ3Z6IyFfYTOpKpRJGo7H52Gg0NpdZrGJjY9G7d28EBQUhKCgIt956K/bv3+9Y\nUnfgRCkADI8ejmPnjqHWVIse/j3s/h4RkbezufolNTUVer0eBoMBJpMJBQUFyMjIaNXmzjvvxLZt\n22A2m1FXV4fi4mIkJSU5FIQjJ0oBy+3thvcZjtKTpQ6NQ0Tk7WzO1OVyOXJzc5Geng6z2YyFCxdC\nrVYjLy8PALBkyRIkJiZi5syZGDFiBGQyGRYtWuRwUne0/AJcXa9esRu33nSrQ98jIvJmgiiKossH\nEQTcaBiT2YSQFSEw/cVkVw3favX+1dio34iCuQXOCpOIyKPYyp03IvkVpTV1NYgMinQooQNXb5hR\nwStLiYhakjypO3qS1GpI7yGoqatBTZ3jFzsREXkryZO6I/u+tCQTZBgdM5o7NhIRtSB5Undkh8br\n3ay8mSUYIqIWJE/q9tzx6EZ4I2oiotYkT+qdWc5olRaThpKKEofPDhMReSvJk7qjFx611D+sP5rE\nJhy/cNzJURERdU+SJ3VHbmV3PUEQeNMMIqIWJE/qnV39YsW6OhHRNZIn9bqGOoQoQjr9fa6AISK6\nRvKkXm+uR4A8oNPft94wo0lscmJURETdk+RJ/UrjFQTKAzv9/aiQKPQK6gX9ab0ToyIi6p4kT+r1\njfUI8Ov8TB0AxqrGYufxnU6KiIio+5I8qXd1pg4A41XjscO4w0kRERF1X5In9a7W1AFgfCyTOhER\n4AlJ3QnllxHRI/DL+V9w7so5J0VFRNQ9SZ7UnVF+UfgpkBqTil3HdzkpKiKi7knSpN7Y1IgmsQly\nmc276tmFJRgiIjuSularRWJiIhISEpCTk9Pmc51Oh7CwMKSkpCAlJQXPPvus3YPXN9YjUB7o8F2P\n2jNeNZ4rYIjI59mcIpvNZmRlZaGoqAhKpRJpaWnIyMiAWq1u1W7SpEnYsGGDw4M74ySp1VjVWBSv\nLYa5yQw/mZ9T+iQi6m5sztRLSkoQHx+PuLg4KBQKZGZmorCwsE27zm59a52pO0NkcCSUoUr8eOpH\np/RHRNQd2ZypV1RUIDY2tvlYpVKhuLi4VRtBELBjxw4kJydDqVTixRdfRFJSUpu+srOzm19rNBpo\nNBpcabzS5ZUvLVnr6sl9k53WJxGRu+h0Ouh0ui71YTOp21PrHjVqFIxGI4KDg7F582bMnj0bhw8f\nbtOuZVK3cmb5BQDGqcbh21++xSNpjzitTyIid7FOeK2WL1/ucB82yy9KpRJGo7H52Gg0QqVStWrT\ns2dPBAcHAwB+85vfoKGhAWfOnLFrcGcsZ2yJK2CIyNfZTOqpqanQ6/UwGAwwmUwoKChARkZGqzZV\nVVXNNfWSEsut5SIiIuwa3BkXHrWU2DsRZy6fQWVtpdP6JCLqTmyWX+RyOXJzc5Geng6z2YyFCxdC\nrVYjLy8PALBkyRJ88sknePPNNyGXyxEcHIw1a9bYPXi92XknSgFAJsgwTjUOO407cZf6Lqf1S0TU\nXQiiG+7aLAhCuytkvjz6JV7Y8QK+euArp4317HfP4uyVs3hpxktO65OISAo3yp22SHpFqbPLLwCg\nidPgW8O3Tu2TiKi7kDSpO/tEKWC5E9Kh04e4uRcR+SRpZ+pOXtIIAAHyAIxVjcV3v3zn1H6JiLoD\nycsvzp6pA8DkuMnQGXRO75eIyNNJXn5xdk0dsNTVvzF84/R+iYg8ndeVXwBLXf3omaM4c9m+i6CI\niLyF5DN1V5RfFH4KjI8dz1UwRORzpJ+pu6D8Alytq/+ic0nfRESeyitPlAJX6+rHWFcnIt8iefnF\nVTP10TGj8cv5X1B9qdol/RMReSLpyy8uOFEKAHKZHBP7T+R6dSLyKZLP1F1VfgEsdXUubSQiXyJ5\nTd1V5ReA69WJyPdIXn5x5Uw9pW8KqmqrYDxv7LgxEZEXkLz84qqaOgD4yfwwY9AMaI9oXTYGEZEn\n8eryCwDMSpiFzUc2u3QMIiJP4dXlFwBIH5SOLce2wGQ2uXQcIiJP4NXlFwCIConC4MjB2P7rdpeO\nQ0TkCSQvv7h6pg4Av0n4DUswROQTOkzqWq0WiYmJSEhIQE5Ozg3b7d69G3K5HJ9++qndg7vyitKW\nZsWzrk5EvsFmUjebzcjKyoJWq8WBAweQn5+P8vLydtv9+c9/xsyZMx26SaorryhtKTUmFZW1lVza\nSERez2ZSLykpQXx8POLi4qBQKJCZmYnCwsI27VatWoW5c+ciKirKocHdVX6xLm3kbJ2IvJ3c1ocV\nFRWIjY1tPlapVCguLm7TprCwEFu2bMHu3bshCEK7fWVnZze/njRJA41GgyuNV+AvC4ADk/tOEQRL\nCeaT8k+wePRi1w5GRNRJOp0OOp2uS33YTOo3StAtLV26FCtXroQgCBBF8Ybll+XLs9u++T/1iAwL\nBBrtirVTBg0CjhwB0uPT8eimR2Eym+Dv5++6AYmIOkmjsUx4rZYvX+5wHzaTulKphNF4rQ5tNBqh\nUqlatdm7dy8yMzMBADU1Ndi8eTMUCgUyMjJatbs+14uiCL+/1aPRFABZx393dMrly0B4uOV17+De\nSOydiK2/bMXUgVNdMyARkcRs1tRTU1Oh1+thMBhgMplQUFDQJln//PPPOHbsGI4dO4a5c+fizTff\nbNOmPQ1NDfCT+UEmuG5VZWAgYDYDpqvXHc0eMhufHrR/dQ4RUXdjM6PK5XLk5uYiPT0dSUlJmDdv\nHtRqNfLy8pCXl9elgd1xklQQgJ49gYsXLcdzkuZgXfk6NIlNLh2XiEgqgujIGsTODnK13t5S9aVq\nqF9Xo2ZZjUvHvukm4Ntvgbg4y/HwN4fjH7f9AxP6T3DpuEREXdVe7uyIZFeUumPfF8AyU79w4drx\nHPUcrC1f6/JxiYikIFlSd8e+L0Dr8gtgSeqfln/q8N9+RETdgXQzdTdsuwu0TerD+gyDv58/9p7c\n6/KxiYjczevLL6GhrZO6IAiYk8QSDBF5J58rvwBX6+oH1rIEQ0ReR9LyixQnSgFgdL/RMJlN+PHU\njy4fn4jInaSdqUtQUwcsJZi71XezBENEXkfSmro7yi/X19Stfjv0t8j/MZ8lGCLyKj5RfmkvqY9R\njoEoiiipKHF5DERE7uIT5Zfra+qApQQzP3k+VpetdnkMRETu4vVLGm80UweA+0fcj4IfC1DfWO/y\nOIiI3MHrlzTeqKYOAHHhcRjWZxg26je6PA4iInfwuStKr/dg8oNYvZ8lGCLyDj5Rfmmvpm41J2kO\ndAYdaupcu1skEZE7+MSJUlsz9dCAUNw2+Das+XGNy2MhInI1r5+p26qpW80fMR/v7X/P5bEQEbma\n158oDQmx3KvUbL5xm2kDp+HkxZMoqypzeTxERK7k9SdKZTIgOBiorb1xGz+ZHxaNWoQ3dr/h8niI\niFzJ68svgH0lmEWjF6HgpwKcv3LeLTEREblCh0ldq9UiMTERCQkJyMnJafN5YWEhkpOTkZKSgtGj\nR2PLli12Deyu8gvQ8clSAIjpGYPpA6fj/bL33RITEZEr2EzqZrMZWVlZ0Gq1OHDgAPLz81FeXt6q\nzbRp07B//358//33ePfdd7F48WK7BnbX3i+AfUkdAB5Lewxv7H6Dm3wRUbdlM6mXlJQgPj4ecXFx\nUCgUyMzMRGFhYas2ISEhza9ra2vRu3dvuwZ215JGoOO16la33nQrZIIMOoPO5TEREbmC3NaHFRUV\niI2NbT5WqVQoLi5u0279+vV46qmncPLkSXz55Zft9pWdnd38WqPRuG3rXcC+mjpg2eTr0bRH8fru\n1zF5wGTXB0ZE1IJOp4NOp+tSHzaTuiAIdnUye/ZszJ49G1u3bsUDDzyAQ4cOtWnTMqkDwNNvP+1x\n5RfAssnXX7b8BRUXKqAMVbo2MCKiFjQaDTQaTfPx8uXLHe7DZvlFqVTCaDQ2HxuNRqhUqhu2v+WW\nW9DY2IjTp093OLC7yy/2JvXQgFDcN/w+vLGHyxuJqPuxmdRTU1Oh1+thMBhgMplQUFCAjIyMVm2O\nHj3afGKxtLQUABAZGdnhwO5c0mhvTd3qibFPIG9PHi7UO/AlIiIPYLP8IpfLkZubi/T0dJjNZixc\nuBBqtRp5eXkAgCVLlmDt2rVYvXo1FAoFevTogTVr7NtDpb7R82rqVoMiBmH6oOn4595/4k/j/+S6\nwIiInEwQ3bB+TxCENssEVX9XYefCnYgNi73Bt5zntdcAvR5Ytcr+7+yr3IfbPrwNP//hZ7f95UNE\n1FJ7ubMjPnFFqSM1dauRfUdiRPQIXoxERN2K12/oBTheU7d6csKTeH778zA32dgNjIjIg0i6oZcn\n7f3SnltvuhWRwZFYd3Cd84MiInIBSZJ6k9iExqZGKGQKt4zXmfILYKlnPTnhSfzf1v9Dk9jk/MCI\niJxMkqRuXfli78VNXdXZpA4Adwy5AzJBhk/LP3VuUERELiBNUje7Zy91q87W1AFAJsiwYsoK/GXL\nX9DY1OjcwIiInEySpO7Ok6RA52vqVjMGzUB0j2is3r/aeUEREbmAZOUXd50kBSwz9dpaoLMr8gVB\nwHNTn0O2LhtXGq84NzgiIifyifKLXA4oFJZ7lXbW+NjxSO6bjLw9ec4LjIjIySQrv7hzpg50ra5u\n9ezkZ/Hctudwsb4LtRwiIheSdPWLO3W1rg4AyX2TMWPQDKzYtsI5QREROZl0J0rdWH4BurassaWV\n01birb1vQX9a3/XOiIicTLKauhTlF2ck9ZieMVg2YRme+OKJrndGRORkPlN+cUZN3Wrp2KXQn9Fj\n4+GNzumQiMhJfOZEqTNq6lb+fv54dearWPrFUtQ31junUyIiJ/CJJY2A88ovVjPjZyIpKgkv7XzJ\neZ0SEXWRT1xRCjg/qQPAqzNfxd93/h0Haw46t2Miok7yiStKAeeWX6ziwuOwXLMcDxU+xD3Xicgj\n+FT5xVknSlt6JO0RyGVy5JbkOr9zIiIHdZjUtVotEhMTkZCQgJycnDaff/DBB0hOTsaIESMwYcIE\nlJWVdTioVFeUOnumDlh2cXw7423873f/i6Nnjjp/ACIiB9hM6mazGVlZWdBqtThw4ADy8/NRXl7e\nqs3AgQPx3XffoaysDM888wwWL17c4aBSLWl0RVIHgITIBDw18Sk8/NnDvJkGEUnKZlIvKSlBfHw8\n4uLioFAokJmZicLCwlZtxo0bh7CwMADAmDFjcPz48Q4HvWJ2/xWlrqipt7R07FI0NjXihe0vuG4Q\nIqIOyG19WFFRgdjY2OZjlUqF4uLiG7Z/++23MWvWrHY/y87Obn59JOQIxk0c52CoXeOqmrqVn8wP\nH9z9AdLeSoMmToMxqjGuG4yIvJJOp4NOp+tSHzaTuiO3m/vmm2/wzjvvYPv27e1+3jKpP7bpsW6/\nTr09/cP64x+3/QP3fXofSheXIiwwzLUDEpFX0Wg00Gg0zcfLly93uA+b5RelUgmj0dh8bDQaoVKp\n2rQrKyvDokWLsGHDBvTq1avDQb3pROn17lLfhfRB6fjdxt9B7OxdOYiIOslmUk9NTYVer4fBYIDJ\nZEJBQQEyMjJatfn1119x991349///jfi4+PtGrS7br1rr5dmvIQfT/2If+79p3sGJCK6ymb5RS6X\nIzc3F+np6TCbzVi4cCHUajXy8ix3/1myZAn+9re/4ezZs3jkkUcAAAqFAiUlJTYHlWrrXVfW1FsK\nUgRh7W/XYuI7EzGszzBM6D/BPQMTkc8TRDfUCARBaFWKuCP/DiwetRh3DLnD1UM3E0XA3x+4dMny\n7A7aI1o8VPgQShaVQBXatmxFRGTL9bnTHj6z9a4guK+ubjUzfiYeH/M47iq4C5cbunCDVCIiO/nM\n1ruAe+vqVssmLEN8RDwWfbaIJ06JyOV8Zu8XwL11dStBEPB2xts4evYont7ytHsHJyKf4zNb7wLu\nL79YBSuC8dm9n+HT8k+58RcRuZTN1S+uIsXWuwAQFgY8+SQQHe32oTFpUm988cAXmPjORPTr0Q9z\nkua4Pwgi8nrSJHWJyi8vvghctx+ZW1RUAHl5wO9/H4fP7/scM96fgbDAMEwbOM39wRCRV5NkSWO/\nl/qhdHEp+vXs5+qhPcKlS0BUFFBbC8hkwLZft+HugruxZu4aTBkwRerwiMhDcUmjhwoJAcLDLTN2\nAJjYfyI+vudjZH6SiW8N30obHBF5FZ9a/SKl+HjgyJFrx5PiJmHN3DW45+N78N0v30kXGBF5FZ9a\n/SKl65M6AEwZMAX5c/Ix96O52Hh4ozSBEZFXcXtSb2xqhAABcpkk52gl015SB4CpA6fis3s/w8IN\nC/FB2QfuD4yIvIrbk7pUV5NK7UZJHQDGqMbg6/lf48mvn8Sq4lXuDYyIvIrbk7qvnSS1spXUAWBo\nn6HYumAr3tjzBh7XPo7Gpkb3BUdEXsP9Sd0HT5IClqR+9Khlt8gbiQuPw86FO1FeXY6M/AxcqHfz\nngZE1O25Palfbrjsk+WX0FDL0sbKStvtwgPDsfG+jYgLj8P4t8fj6Jmj7gmQiLyC25N6TV0NokKi\n3D2sR+ioBGOl8FPg9Vmv49G0RzH+nfHYcGiD64MjIq/g9qReWVuJvj36untYj2BvUgcsV5I9mvYo\nCjMLkbUpC08WPck6OxF1iEndjeLjAb3ese+MVY3F3sV7UXqyFJPfmwzDOYNLYiMi78Ck7kaOzNRb\nigqJwub/2IyMwRlIeysN/y77N2+4QUTt6jCpa7VaJCYmIiEhATk5OW0+P3jwIMaNG4fAwEC89NJL\nHQ5YeakSfUOY1B3lJ/PDf0/4b3z1wFd4bttzuHftvaipq3FugETU7dlM6mazGVlZWdBqtThw4ADy\n8/NRft3etZGRkVi1ahX+9Kc/2TUgZ+q2lzV2ZGTfkdizaA9iesZg+JvDkf9DPmftRNTMZlIvKSlB\nfHw84uLioFAokJmZicLCwlZtoqKikJqaCoVCYdeAvpzUe/UCFAqgurpr/QQpgvD39L+jMLMQK7at\nQMaaDPx6/lfnBElE3ZrNDVgqKioQGxvbfKxSqVBcXNypgbKzswEAB4sPwtDHgHGx4zrVT3dnna33\n6dP1vm5W3oy9i/ciZ1sORuWNwhNjn8Afx//RJ68DIPIGOp0OOp2uS33YTOqCIHSp85ays7MhiiJy\nVuTgzpl3Oq3f7saa1MePd05//n7+eGbSM7h/xP34ry//C8PeGIaX01/G7YNvd+p/PyJyPY1GA41G\n03y8fPlyh/uwWX5RKpUwGo3Nx0ajESqVyuFBrC7UX4C/nz+CFcGd7qO7S0jo/MlSWwb0GoB189Yh\nd1YulhUtw5TVU7C7YrfzByIij2YzqaempkKv18NgMMBkMqGgoAAZGRnttrXnZJ0v19OturICxh4z\n42fih0d+wH3D7sPsgtnI/CQTh08fdt2ARORRbCZ1uVyO3NxcpKenIykpCfPmzYNarUZeXh7y8vIA\nAJWVlYiNjcXLL7+MZ599Fv3790dtbW27/VXWViI6JNr5f4puxNVJHQDkMjkWjV6Ew1mHMbzPcEx4\nZwLmr5vP5E7kA9x64+mCHwuwtnwtPrrnI1cP6bFqaiwlmDNnAHeVvM9fOY/Xil/DayWvIX1QOpZN\nWIYR0SPcMzgRdVpnbjzt1tsPsfwCREZanu+4A5C57XreMADPIFX2B/iFv4n0f6cjpW8Klk1Yhkk3\nTeIJVSIv4t6kfolJXRCAL77oeAteVzh8OAwfvfUkju1Yivf3v4/fff47BMoD8fubf4/7ht+HIEWQ\n+4MiIqdya/llQeEC3NL/FjyU8pCrh6R2NDQAffsC+/cDKhXQJDbhq6NfYVXJKhRXFOPB5AexaNQi\nDOk9ROpQiQidK7+4dUMvll+kpVAAt98OrF9vOZYJMqTHp+Pz+z7HzoU7IZfJMendSbj1X7di9f7V\nqDW1f8KbiDwXk7qPmT0bWLeu7fvxEfFYOW0ljE8YsXTsUnz000eIfTkW89fNR9HPRTA3md0fLBE5\nzK3ll34v9cPexXsR0zPG1UPSDdTVWUowx45dO2l7I1W1VVjz4xqsLluNigsVmJM0B/OGzsPE/hMh\nE9y+azORz+lM+cVtSb3R3IjA/wtE3dN1UPjZt/kXucZdd1lm7A8+aP939Kf1+PjAxyj4qQCnLp3C\nnUPuxOzE2ZgyYAr8/fxdFyyRD/PopF55sRLD3xyOU/99ytXDUQdWr7aUYNorw9jjyJkjWH9wPdYd\nXIefTv2EqQOn4raE2zArYRbLa0RO5NFJfd/JfXhg3QMoe6TM1cNRB86cAeLiLMsqg7u4Dc+pS6eg\nPaLFRv3FMcE8AAAK2klEQVRGfHX0K/QP64/pg6ZjxsAZmNh/IpdJEnWBRyd1rV6Ll3a+hC8f+NLV\nw5Edpk4FsrIspRhnaWxqRElFCb46+hW+/PlL7K/cj9SYVGjiNJgcNxk3K29mkidygEcn9Xe/fxdf\nH/saq+9a7erhyA65uUBJiaUU4yoX6y9iu3E7vjF8A51Bhx9P/Yjk6GRM7D8R42PHY6xqLMs1RDZ4\ndFJfuXUlTl8+jeenP+/q4cgOFRXA0KHAhQvuG1NUXAKUJRBu2gr19F046bcLoQGhGKMag7SYNKTG\npGJUv1EIDQh1X1BEHsyj936pvFSJ2NDYjhuSWyiVltq6e4UAmIyff56MsWOB3cUizOF6FB8vxp6T\ne7Du4Drsr9yPfj37IaVvCkb2HYnk6GSMiB4BVaiKe9QQ2cFtM/XMTzJxx+A7cN/w+1w9HHUDL75o\n2QPnyy9b71bZ2NSIw6cP4/uT3+P7yu9RVlWGsqoy1JvrMazPMAyNGoqkqCQMjRqKxN6JiOkZw2RP\nXsujyy+adzV45tZnMGXAFFcPR91AYyMwZozlZO2CBR23r75UjR9O/YAD1QdwoPoAfqr+CQdrDqKu\noQ5DIodgcORgJEQmICHC8hgUMQiRQZFM+NSteXRST8xNxNrfrkVSVJKrh6NuYt8+YMYMoKzMcpVr\nZ5y9fBaHTh/CoZpD0J/R48iZI9Cf0ePomaMQIWJQr0EY0GsABoQPQFx4HG4Kuwk3hd+E/mH9ERYQ\nxqRPHs2jk3r4ynAc/cNRRARFuHo46kaeegr49FOgf3/n9itCRKP8LOoCj+JK4DFcDjDgSqDB8hzw\nK64E/AJAQGxYLAb3VUEVannE9IiBMlSJmJ4x6NejH/qE9IGfzM+5wRHZyaOTuv//+uPK/1zhzIha\naWgAtm2zlGPcSRRFHK04j788b8R/PHIcwyccx/GLx1FxoQInLp5AxcUKVNZW4szlM+gd3Bt9e/RF\ndEg0ontEIzokGn1C+qBPSB9EBUchKiQKUcFR6B3cG8GKYP6Ok9N49OqX6JBo/rID0Ol00Gg0Uofh\nEaw/i8mTpRhdABCOaRPDceedw2E6ALz2GuB/3TY2DeYGnLp0ClWXqlBVW4WqS1WorK3EydqT2F+1\nH6cunUJNXQ2qL1Wjuq4aoigiMjgSkUGRzc8RQRGICIpAr8BeCA8MR6+gXs2vrY99xfswfcp0KX4Q\nHof/j3RNh0ldq9Vi6dKlMJvNePjhh/HnP/+5TZs//OEP2Lx5M4KDg/Huu+8iJSWlTRteZGLBX9hr\nPOFnkZAA7NoF/Od/AgEB7bVQAFBefdhBUYcTQadxIvg0EFwDBJ4Fgs4AwaeBwNNA4BEg6Kzl/cDz\nQOA5y6P4LPy3KxAeFIawgDCEBoQiNCAUPQN6oqd/z1ave/j3aPMIUYQgxD8EwYrg5teB8sBuuZum\nJ/xedGc2k7rZbEZWVhaKioqgVCqRlpaGjIwMqNXq5jabNm3CkSNHoNfrUVxcjEceeQS7du1q0xeT\nOnmq0FBLXd85gq8+7L8mo7oamDcvG+VvLUNc4nkMTbmAeuECTLiIeuE8zgoXUYWLMAkXYUItGoQa\nmHARJqEWDbiEhubnSy2e69CIK5AjEAqEQC4GQYFgyMUgyHH1IbZ8DkSQIgg3xQRiYP9AhAYHIkAe\ngEB5IALlgQjwC0CAPKDdZ38/fwTILc/XPxQyBf+F7mY2k3pJSQni4+MRFxcHAMjMzERhYWGrpL5h\nwwY8eHUP1zFjxuDcuXOoqqpCdHR0q76ie7Q+JiKLqCjg1lsBrTYY69cHw2Do55R+m8QmNOIyTLiE\nBvEyGlAHE+rQIF5GI66gAZdbvb5Qdxkl39VjXdUV9Io6j4CQejQJV9Akq0eT7OqzUA9RVt/82vJs\ngigzXX2uR5PQcPW4HqKsEUKTHILoD5mogNCkgEz0hyAqLI8mheV9UQFBlEMQFWjYdRz/atyBHkEK\n9AiWQ+GngJ8gh58ghwxXnwU/y3tXj63vyeDX/JkMftfeb/GZ5f3W7wmQXf3etddBAX4IC5UhPMwP\noT384CeTQSZce1j6aP2ecN1x8wMt2wjNrwHLawFC8/sCWn7eCaINH3/8sfjwww83H7///vtiVlZW\nqza33367uH379ubjqVOninv27GnVBgAffPDBBx+deDjK5kzd3n82XX929vrvuWGBDRERoYN7lCqV\nShiNxuZjo9EIlUpls83x48ehVNp5UomIiJzKZlJPTU2FXq+HwWCAyWRCQUEBMjIyWrXJyMjA6qv7\nt+7atQvh4eFt6ulEROQeNssvcrkcubm5SE9Ph9lsxsKFC6FWq5GXlwcAWLJkCWbNmoVNmzYhPj4e\nISEh+Ne//uWWwImIqB0OV+EdtHnzZnHIkCFifHy8uHLlSlcP57F+/fVXUaPRiElJSeLQoUPFV199\nVeqQJNfY2CiOHDlSvP3226UORVJnz54V58yZIyYmJopqtVrcuXOn1CFJZsWKFWJSUpI4bNgw8d57\n7xWvXLkidUhus2DBArFPnz7isGHDmt87ffq0OG3aNDEhIUGcPn26ePbs2Q77cemVCdZ17lqtFgcO\nHEB+fj7Ky8tdOaTHUigUePnll/HTTz9h165deP311332Z2H16quvIikpyefXMT/++OOYNWsWysvL\nUVZW1mrJsC8xGAx46623UFpaih9++AFmsxlr1qyROiy3WbBgAbRabav3Vq5cienTp+Pw4cOYOnUq\nVq5c2WE/Lk3qLde5KxSK5nXuvqhv374YOXIkAKBHjx5Qq9U4ceKExFFJ5/jx49i0aRMefvhhn14d\ndf78eWzduhUPPfQQAEvJMywsTOKopBEaGgqFQoG6ujo0Njairq7OpxZd3HLLLejVq1er91peB/Tg\ngw9i/fr1Hfbj0qReUVGB2NhrV9apVCpUVFS4cshuwWAw4Pvvv8eYMWOkDkUyTzzxBF544QXIZN3v\nMnZnOnbsGKKiorBgwQKMGjUKixYtQl1dndRhSSIiIgJ//OMf0b9/f8TExCA8PBzTpk2TOixJtbyQ\nMzo6GlVVVR1+x6X/R/n6P6vbU1tbi7lz5+LVV19Fjx49pA5HEp9//jn69OmDlJQUn56lA0BjYyNK\nS0vx6KOPorS0FCEhIXb9E9sbHT16FK+88goMBgNOnDiB2tpafPDBB1KH5TEEQbArp7o0qduzzt2X\nNDQ0YM6cObj//vsxe/ZsqcORzI4dO7BhwwYMGDAA9957L7Zs2YL58+dLHZYkVCoVVCoV0tLSAABz\n585FaWmpxFFJY8+ePRg/fjwiIyMhl8tx9913Y8eOHVKHJano6GhUVlYCAE6ePIk+ffp0+B2XJnV7\n1rn7ClEUsXDhQiQlJWHp0qVShyOpFStWwGg04tixY1izZg2mTJnSfK2Dr+nbty9iY2Nx+PBhAEBR\nURGGDh0qcVTSSExMxK5du3D58mWIooiioiIkJfn2ndIyMjLw3nvvAQDee+89+yaDrlqeY7Vp0yZx\n8ODB4qBBg8QVK1a4ejiPtXXrVlEQBDE5OVkcOXKkOHLkSHHz5s1ShyU5nU4n3nHHHVKHIal9+/aJ\nqamp4ogRI8S77rpLPHfunNQhSSYnJ6d5SeP8+fNFk8kkdUhuk5mZKfbr109UKBSiSqUS33nnHfH0\n6dPi1KlTHVrS6JY7HxERkXv49tIDIiIvw6RORORFmNSJiLwIkzoRkRdhUici8iJM6kREXuT/AU6y\nY0VuuYqwAAAAAElFTkSuQmCC\n", 147 | "text": [ 148 | "" 149 | ] 150 | } 151 | ], 152 | "prompt_number": 14 153 | }, 154 | { 155 | "cell_type": "code", 156 | "collapsed": false, 157 | "input": [], 158 | "language": "python", 159 | "metadata": {}, 160 | "outputs": [], 161 | "prompt_number": 14 162 | }, 163 | { 164 | "cell_type": "code", 165 | "collapsed": false, 166 | "input": [], 167 | "language": "python", 168 | "metadata": {}, 169 | "outputs": [] 170 | } 171 | ], 172 | "metadata": {} 173 | } 174 | ] 175 | } -------------------------------------------------------------------------------- /ex214.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unpingco/Python-for-Signal-Processing/0183e2f799556b20eb4519c9cc6ff9ec0afd3089/ex214.jpg -------------------------------------------------------------------------------- /ex23.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unpingco/Python-for-Signal-Processing/0183e2f799556b20eb4519c9cc6ff9ec0afd3089/ex23.jpg -------------------------------------------------------------------------------- /ex24.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unpingco/Python-for-Signal-Processing/0183e2f799556b20eb4519c9cc6ff9ec0afd3089/ex24.jpg -------------------------------------------------------------------------------- /ex26.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unpingco/Python-for-Signal-Processing/0183e2f799556b20eb4519c9cc6ff9ec0afd3089/ex26.jpg -------------------------------------------------------------------------------- /ls_slider.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import matplotlib.pyplot as plt 3 | from matplotlib.widgets import Slider 4 | 5 | fig, ax = plt.subplots() 6 | #fig.set_size_inches(5,5) 7 | 8 | plt.subplots_adjust(left=0.25, 9 | bottom=0.30, 10 | # right=1, 11 | ) 12 | 13 | x = np.linspace(0,2,50) 14 | y = 1+x + 2*x**2 + 0*np.sin(2*np.pi*x) + np.random.randn(len(x)) 15 | 16 | V = np.matrix(np.vstack([np.ones(x.shape),x,x**2]).T) 17 | Q = np.matrix(np.eye(V.shape[0])) 18 | i,j =np.diag_indices_from(Q) 19 | #Q[i[:20],j[:20]]=100 20 | R = np.matrix(np.diag([1,1,1]))*.01 21 | 22 | Pv = V*np.linalg.inv(V.T*Q*V+R)*V.T*Q 23 | p=np.polyfit(x,y,2) 24 | 25 | #d_line,=ax.plot(x,y,'o',label='data',alpha=0.3) 26 | d=[ (ax.plot(i,j,'ob',alpha=0.8))[0] for i,j in zip(x,y) ] 27 | 28 | ax.set_title('err^2=%3.2f'%(np.linalg.norm(y)**2 - np.linalg.norm(np.dot(Pv,y))**2 )) 29 | ls_line,=ax.plot(x,np.dot(Pv,y).flat,label='projection') 30 | ax.plot(x,np.polyval(p,x),'-',label='polyfit',alpha=0.3) 31 | ax.legend(loc=0) 32 | ax.grid() 33 | 34 | axw0 = plt.axes([0.25, 0.2, 0.65, 0.03] ) 35 | sw0 = Slider(axw0, 'w0', 0.1, 30.0, valinit = 1) 36 | axw1 = plt.axes([0.25, 0.15, 0.65, 0.03] ) 37 | sw1 = Slider(axw1, 'w1', 0.1, 30.0, valinit = 1) 38 | axw2 = plt.axes([0.25, 0.10, 0.65, 0.03] ) 39 | sw2 = Slider(axw2, 'w2', 0.1, 30.0, valinit = 1) 40 | scale_factor = 1. 41 | ##DEBUG scale_factor = np.mean(np.array(np.dot(Pv,y)).flatten()**2) 42 | print scale_factor 43 | 44 | def update_w0(val): 45 | w = sw0.val 46 | R[0,0]=w 47 | Pv = V*np.linalg.inv(V.T*Q*V+R)*V.T*Q 48 | err = np.array(np.dot(Pv,y)-y).flatten()**2 49 | err_scaled=err/scale_factor 50 | for i,j in zip(d,err_scaled) : 51 | i.set_color(plt.cm.gray(j)) 52 | i=np.argsort(np.array(np.dot(Pv,y)-y).flatten()**2) 53 | ls_line.set_ydata(np.dot(Pv,y).flat) 54 | ax.set_title('err=%3.2f'%(err_scaled.mean())) 55 | print np.linalg.inv(V.T*Q*V+R)*V.T*Q 56 | plt.draw() 57 | sw0.on_changed(update_w0) 58 | 59 | def update_w1(val): 60 | w = sw1.val 61 | R[1,1]=w 62 | Pv = V*np.linalg.inv(V.T*Q*V+R)*V.T*Q 63 | err = np.array(np.dot(Pv,y)-y).flatten()**2 64 | err_scaled=err/scale_factor 65 | for i,j in zip(d,err_scaled) : 66 | i.set_color(plt.cm.gray(j)) 67 | ls_line.set_ydata(np.dot(Pv,y).flat) 68 | ax.set_title('err=%3.2f'%(err_scaled.mean())) 69 | plt.draw() 70 | sw1.on_changed(update_w1) 71 | 72 | def update_w2(val): 73 | w = sw2.val 74 | R[2,2]=w 75 | Pv = V*np.linalg.inv(V.T*Q*V+R)*V.T*Q 76 | err = np.array(np.dot(Pv,y)-y).flatten()**2 77 | err_scaled=err/scale_factor 78 | for i,j in zip(d,err_scaled) : 79 | i.set_color(plt.cm.gray(j)) 80 | ls_line.set_ydata(np.dot(Pv,y).flat) 81 | ax.set_title('err=%3.2f'%(err_scaled.mean())) 82 | plt.draw() 83 | sw2.on_changed(update_w2) 84 | 85 | plt.show() 86 | 87 | --------------------------------------------------------------------------------