"
24 | ]
25 | },
26 | {
27 | "attachments": {},
28 | "cell_type": "markdown",
29 | "metadata": {},
30 | "source": [
31 | "# Multiple Linear Regression\n",
32 | "\n",
33 | "By the end of this lesson, you should be able to:\n",
34 | "- Transform data for **higher** order features\n",
35 | "\n",
36 | "Important words:\n",
37 | "- hypothesis\n",
38 | "- polynomial\n",
39 | "- update function\n",
40 | "- matrix form"
41 | ]
42 | },
43 | {
44 | "attachments": {},
45 | "cell_type": "markdown",
46 | "metadata": {},
47 | "source": [
48 | "## Introduction\n",
49 | "\n",
50 | "In the previous notes, we only have one independent variable or one feature. In most cases of machine learning, we want to include more than one feature or we want to have a hypothesis that is not simply a straight line. For the first example, we may want to consider not only the floor area but also the storey level to predict the resale price of HDB houses. For the second example, we may want to model the relationship not as a straight line but rather as quadratic. Can we still use linear regression to do these?\n",
51 | "\n",
52 | "This section discusses how we can include more than one feature and how to model our equation beyond a simple straight line using multiple linear regression."
53 | ]
54 | },
55 | {
56 | "attachments": {},
57 | "cell_type": "markdown",
58 | "metadata": {},
59 | "source": [
60 | "## Hypothesis\n",
61 | "\n",
62 | "Recall that in linear regression, our hypothesis is written as follows.\n",
63 | "\n",
64 | "$$\\hat{y}(x) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x$$\n",
65 | "\n",
66 | "where $x$ is the only independent variable or feature. In multiple linear regression, we have more than one feature. We will write our hypothesis as follows.\n",
67 | "\n",
68 | "$$\\hat{y}(x) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x_1 + \\hat{\\beta}_2 x_2 + \\ldots + \\hat{\\beta}_n x_n$$\n",
69 | "\n",
70 | "In the above hypothesis, we have $n$ features. Note also that we can assume to have $x_0 = 1$ with $\\hat{\\beta}_0$ as its coefficient.\n",
71 | "\n",
72 | "We can write this in terms of a row vector, where the features are written as\n",
73 | "\n",
74 | "$$\\mathbf{X} = \\begin{bmatrix}\n",
75 | "x_0 & x_1 & \\ldots & x_n\n",
76 | "\\end{bmatrix} \\in {\\rm I\\!R}^{n+1}$$\n",
77 | "\n",
78 | "Note that the dimension of the feature is $n+1$ because we have $x_0 = 1$ which is a constant of 1. \n",
79 | "\n",
80 | "The parameters can be written as follows.\n",
81 | "\n",
82 | "$$\\mathbf{\\hat{b}} = \\begin{bmatrix}\n",
83 | "\\hat{\\beta}_0 \\\\\n",
84 | "\\hat{\\beta}_1 \\\\\n",
85 | "\\ldots \\\\\n",
86 | "\\hat{\\beta}_n\n",
87 | "\\end{bmatrix} \\in {\\rm I\\!R}^{n+1}$$\n",
88 | "\n",
89 | "Our system equations for all the data points can now be written as follows.\n",
90 | "\n",
91 | "$$\\hat{y}(x^1) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x_1^1 + \\hat{\\beta}_2 x_2^1 + \\ldots + \\hat{\\beta}_n x_n^1$$\n",
92 | "$$\\hat{y}(x^2) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x_1^2 + \\hat{\\beta}_2 x_2^2 + \\ldots + \\hat{\\beta}_n x_n^2$$\n",
93 | "$$\\ldots$$\n",
94 | "$$\\hat{y}(x^m) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x_1^m + \\hat{\\beta}_2 x_2^m + \\ldots + \\hat{\\beta}_n x_n^m$$\n",
95 | "\n",
96 | "In the above equations, the superscript indicate the index for the data points from 1 to $m$, assuming there are $m$ data points.\n",
97 | "\n",
98 | "To write the hypothesis as a matrix equation we first need to write the features as a matrix for all the data points.\n",
99 | "\n",
100 | "$$\\mathbf{X} = \\begin{bmatrix}\n",
101 | "1 & x_1^1 & \\ldots & x_n^1 \\\\\n",
102 | "1 & x_1^2 & \\ldots & x_n^2 \\\\\n",
103 | "\\ldots & \\ldots & \\ldots & \\ldots \\\\\n",
104 | "1 & x_1^m & \\ldots & x_n^m\n",
105 | "\\end{bmatrix} \\in {\\rm I\\!R}^{m \\times (n+1)}$$\n",
106 | "\n",
107 | "with this, we can now write the hypothesis as a matrix multiplication.\n",
108 | "\n",
109 | "$$\\mathbf{\\hat{y}} = \\mathbf{X} \\times \\mathbf{\\hat{b}}$$\n",
110 | "\n",
111 | "Notice that this is the same matrix equation as a simple linear regression. What differs is that $\\mathbf{\\hat{b}}$ contains more than two parameters. Similarly, the matrix $\\mathbf{X}$ is now of dimension $m\\times(n+1)$ where $m$ is the number of data points and $n+1$ is the number of parameters. Next, let's see how we can calculate the cost function."
112 | ]
113 | },
114 | {
115 | "attachments": {},
116 | "cell_type": "markdown",
117 | "metadata": {},
118 | "source": [
119 | "## Cost Function\n",
120 | "\n",
121 | "Recall that the cost function is written as follows.\n",
122 | "\n",
123 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}\\Sigma^m_{i=1}\\left(\\hat{y}(x^i)-y^i\\right)^2$$\n",
124 | "\n",
125 | "We can rewrite the square as a multiplication instead and make use of matrix multplication to express it.\n",
126 | "\n",
127 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}\\Sigma^m_{i=1}\\left(\\hat{y}(x^i)-y^i\\right)\\times \\left(\\hat{y}(x^i)-y^i\\right)$$\n",
128 | "\n",
129 | "Writing it as matrix multiplication gives us the following.\n",
130 | "\n",
131 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}(\\mathbf{\\hat{y}}-\\mathbf{y})^T\\times (\\mathbf{\\hat{y}}-\\mathbf{y})$$\n",
132 | "\n",
133 | "This equation is exactly the same as the simple linear regression. "
134 | ]
135 | },
136 | {
137 | "attachments": {},
138 | "cell_type": "markdown",
139 | "metadata": {},
140 | "source": [
141 | "## Gradient Descent\n",
142 | "\n",
143 | "Recall that the update function for gradient descent algorithm for a linear regression is given as follows.\n",
144 | "\n",
145 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)$$\n",
146 | "\n",
147 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)x^i$$\n",
148 | "\n",
149 | "In the case of multiple linear regression, we have more than one feature and so we need to differentiate for each $\\theta_j$. Doing this will result in a system of equation as follows.\n",
150 | "\n",
151 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)x_0^i$$\n",
152 | "\n",
153 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)x_1^i$$\n",
154 | "\n",
155 | "$$\\hat{\\beta}_2 = \\hat{\\beta}_2 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)x_2^i$$\n",
156 | "\n",
157 | "$$\\ldots$$\n",
158 | "\n",
159 | "$$\\hat{\\beta}_n = \\hat{\\beta}_n - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)x_n^i$$\n",
160 | "\n",
161 | "Note that $x_0 = 1$ for all $i$.\n",
162 | "\n",
163 | "We can now write the gradient descent update function using matrix operations.\n",
164 | "\n",
165 | "$$\\mathbf{\\hat{b}} = \\mathbf{\\hat{b}} - \\alpha\\frac{1}{m} \\mathbf{X}^T \\times (\\mathbf{\\hat{y}} - \\mathbf{y})$$\n",
166 | "\n",
167 | "Substituting the equation for $\\mathbf{\\hat{y}}$ gives us the following.\n",
168 | "\n",
169 | "$$\\mathbf{\\hat{b}} = \\mathbf{\\hat{b}} - \\alpha\\frac{1}{m} \\mathbf{X}^T \\times (\\mathbf{X}\\times \\mathbf{\\hat{b}} - \\mathbf{y})$$\n",
170 | "\n",
171 | "Again, this is exactly the same as the simple linear regression. \n",
172 | "\n",
173 | "This means that all our equations have not changed and what we need to do is create the right parameter vector $\\mathbf{\\hat{b}}$ and the matrix $\\mathbf{X}$. Once we constructed these vector and matrix, all the other equations remain the same. "
174 | ]
175 | },
176 | {
177 | "attachments": {},
178 | "cell_type": "markdown",
179 | "metadata": {},
180 | "source": [
181 | "## Polynomial Model\n",
182 | "\n",
183 | "There are time that even when there is only one feature we may want to have a hypothesis that is not a straight line. An example of would be if our model is a quadratic equation. We can use multiple linear regression to create hypothesis beyond a straight line. \n",
184 | "\n",
185 | "Recall that in multiple linear regression, the hypothesis is writen as follows.\n",
186 | "\n",
187 | "$$\\hat{y}(x) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x_1 + \\hat{\\beta}_2 x_2 + \\ldots + \\hat{\\beta}_n x_n$$\n",
188 | "\n",
189 | "To have a quadratic hypothesis, we can set the following:\n",
190 | "\n",
191 | "$$x_1 = x$$\n",
192 | "$$x_2 = x^2$$\n",
193 | "\n",
194 | "And so, the whole equation can be written as\n",
195 | "\n",
196 | "$$\\hat{y}(x) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x + \\hat{\\beta}_2 x^2 $$\n",
197 | "\n",
198 | "In this case, the matrix for the features becomes as follows.\n",
199 | "\n",
200 | "\n",
201 | "$$\\mathbf{X} = \\begin{bmatrix}\n",
202 | "1 & x^{(1)} & (x^2)^{(1)} \\\\\n",
203 | "1 & x^{(2)} & (x^2)^{(2)} \\\\\n",
204 | "\\ldots & \\ldots \\\\\n",
205 | "1 & x^{(m)} & (x^2)^{(m)}\n",
206 | "\\end{bmatrix} \\in {\\rm I\\!R}^{m \\times 3}$$\n",
207 | "\n",
208 | "In the notation above, we have put the index for the data point inside a bracket to avoid confusion with the power.\n",
209 | "\n",
210 | "We can genearalize this to any power of polynomial where each power is treated as each feature in the matrix. This means that if we want to to model the data using any other polynomial equation, what we need to do is to transform the $\\mathbf{X}$ matrix in such a way that each column in $\\mathbf{X}$ represents the right degree of polynomial. Column zero is for $x^0$, column one is for $x^1$, column two is for $x^2$, and similarly all the other columns until we have column n for $x^n$.\n",
211 | "\n",
212 | "The parameters can be found using the same gradient descent that minimizes the cost function."
213 | ]
214 | }
215 | ],
216 | "metadata": {
217 | "kernelspec": {
218 | "display_name": "Python 3 (ipykernel)",
219 | "language": "python",
220 | "name": "python3"
221 | },
222 | "language_info": {
223 | "codemirror_mode": {
224 | "name": "ipython",
225 | "version": 3
226 | },
227 | "file_extension": ".py",
228 | "mimetype": "text/x-python",
229 | "name": "python",
230 | "nbconvert_exporter": "python",
231 | "pygments_lexer": "ipython3",
232 | "version": "3.8.5"
233 | }
234 | },
235 | "nbformat": 4,
236 | "nbformat_minor": 4
237 | }
238 |
--------------------------------------------------------------------------------
/Confusion_Matrix_Metrics.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "attachments": {},
5 | "cell_type": "markdown",
6 | "metadata": {},
7 | "source": [
8 | "
"
24 | ]
25 | },
26 | {
27 | "attachments": {},
28 | "cell_type": "markdown",
29 | "metadata": {},
30 | "source": [
31 | "# Confusion Matrix and Metrics\n",
32 | "\n",
33 | "By the end of this lesson, you should be able to:\n",
34 | "- Calculate confusion matrix, precision, and recall\n",
35 | "\n",
36 | "Important words:\n",
37 | "- metrics\n",
38 | "- confusion matrix\n",
39 | "- accuracy\n",
40 | "- precision\n",
41 | "- sensitivity\n",
42 | "- specificity\n",
43 | "- positive case\n",
44 | "- negative case\n",
45 | "\n"
46 | ]
47 | },
48 | {
49 | "attachments": {},
50 | "cell_type": "markdown",
51 | "metadata": {},
52 | "source": [
53 | "## Introduction\n",
54 | "\n",
55 | "In Linear Regression, we use the correlation coefficient and some mean square errors as metrics to see if our model fits the data well. What kind of metrics we can use in the case of classification problems? In this lesson we will use confusion matrix and a few matrix to evaluate our classification model.\n"
56 | ]
57 | },
58 | {
59 | "attachments": {},
60 | "cell_type": "markdown",
61 | "metadata": {},
62 | "source": [
63 | "\n",
64 | "## Confusion Matrix\n",
65 | "\n",
66 | "Recall in the first lesson on machine learning, we give an example of classifying image as cat or not a cat. Now, let's say we have a dataset of images of various animals including some cats picture inside. First, we need to separate the dataset into training set and test set. The training set is used to build the model or to train the model. Our model for classification which we discussed in the previous lessson was called Logistic Regression. After we train the model, we would like to measure how good the model is using the **test set**. We can write a table with the result of how many data is predicted correctly and not correctly as shown below.\n",
67 | "\n",
68 | "| actual\\predicted | a cat | not a cat |\n",
69 | "|------------------|-------|-----------|\n",
70 | "| a cat | 11 | 3 |\n",
71 | "| not a cat | 2 | 9 |\n",
72 | "\n",
73 | "The above table gives some example of what is called as a **confusion matrix**. The vertical rows are the labels for the **actual** data while the horizontal columns are the labels for the **predicted** data. We can read this table as follows.\n",
74 | "- Out of all the *actual* image which is *a cat*, 11 images are *predicted* as *a cat* and 3 images are *predicted* as *not a cat*. This means that 11 of them are accurate and 3 of them is not. \n",
75 | "- Out of all the *actual* image which is *not a cat*, 2 images are *predicted* as *a cat* and 9 images are *predicted* as *not a cat*. This means that 2 of them are not accurate and 8 is.\n",
76 | "\n",
77 | "\n",
78 | "We can see that there are a total of $11+3=14$ images of the category *a cat*. On the other hand, there are $2+9=11$ images of the category *a cat*. So the total number of images are $11 + 3 + 2 + 9 = 25$ images. \n",
79 | "\n",
80 | "The confusion matrix in general is written as follows.\n",
81 | "\n",
82 | "| actual\\predicted | Positive Case | Negative Case |\n",
83 | "|------------------|-----------------|-----------------|\n",
84 | "| Positive Case | True Positives | False Negatives |\n",
85 | "| Negative Case | False Positives | True Negatives |\n",
86 | "\n",
87 | "We use the term **positive** case here to refer to the category of point of interest. In this case, we would like to detect a cat and so category *a cat* is a positive case. On the other hand, the category *not a cat* is a negative case. This means that:\n",
88 | "- There are 11 True Positive cases where the actual cat images are predicted as a cat.\n",
89 | "- There are 9 True Negative cases where the actual not a cat images are predicted as not a cat.\n",
90 | "- There are 3 False Negative cases where the actual cat images are predicted as *not a cat* (negative case). \n",
91 | "- There are 2 False Positive cases where the actual not. acat images are predicated as *a cat* (positive case). \n"
92 | ]
93 | },
94 | {
95 | "attachments": {},
96 | "cell_type": "markdown",
97 | "metadata": {},
98 | "source": [
99 | "## Metrics\n",
100 | "\n",
101 | "Knowing the confusion matrix allows us to compute several other metrics that is useful for us to evaluate our model. \n",
102 | "\n",
103 | "### Accuracy\n",
104 | "\n",
105 | "In this metrics, we are interested in how many data is predicted correctly. In the above examples, we have 11 images predicted correctly as cats and 9 images predicted correctly as not a cat. Therefore, we have the accuracy of:\n",
106 | "\n",
107 | "$$\\text{accuracy} = \\frac{11 + 9}{25} = \\frac{20}{25} = 0.8$$\n",
108 | "\n",
109 | "This means our model have 80% accuracy. In general, the accuracy formula can be written as:\n",
110 | "\n",
111 | "$$\\text{accuracy} = \\frac{\\text{TP} + \\text{TN}}{\\text{Total Cases}}$$\n",
112 | "\n",
113 | "where TP is the number of True Positive cases, TN is the number of True Negative cases. You can also see accuracy as a fraction of the green circle over the blue circle in the image below.\n",
114 | "\n",
115 | "\n",
116 | "\n",
117 | "Given the accuracy, we can also calculate another metric called **error rate**. In fact, the error rate is just given by the following:\n",
118 | "\n",
119 | "$$\\text{error rate} = 1 - \\text{accuracy}$$\n",
120 | "\n",
121 | "So in our example, the error rate is $1 - 0.8 = 0.2$ which is 20%."
122 | ]
123 | },
124 | {
125 | "attachments": {},
126 | "cell_type": "markdown",
127 | "metadata": {},
128 | "source": [
129 | "### Precision\n",
130 | "\n",
131 | "In precision, we put more attention into the positive cases. We are interested in how many of the positive cases are detected accurately.\n",
132 | "\n",
133 | "| actual\\predicted | a cat | not a cat |\n",
134 | "|------------------|-------|-----------|\n",
135 | "| a cat | 11 | 3 |\n",
136 | "| not a cat | 2 | 9 |\n",
137 | "\n",
138 | "$$\\text{precision} = \\frac{11}{11 + 2} = \\frac{11}{13} = 0.846$$\n",
139 | "\n",
140 | "This means that we have a precision of about 85\\%. Out of all the total 13 cases detected positive, 11 of them is correct. Obviously we want our precision to be as high as possible. We can write precision formula as follows.\n",
141 | "\n",
142 | "$$\\text{precision} = \\frac{\\text{TP}}{\\text{Total Predicted Positives}} = \\frac{\\text{TP}}{\\text{TP} + \\text{FP}}$$\n",
143 | "\n",
144 | "The calculation of precision can be seen as a fraction between the green circle and the blue circle in the image below.\n",
145 | "\n",
146 | ""
147 | ]
148 | },
149 | {
150 | "attachments": {},
151 | "cell_type": "markdown",
152 | "metadata": {},
153 | "source": [
154 | "### Sensitivity\n",
155 | "\n",
156 | "This is also often called as Recall. In this metric, we are interested to know the fraction of the number of positive cases predicted accurately out of all the *actual* positive cases. The difference with precision is that precision is calcalculated as a fraction out of all the total predicted positive cases. In the above example, we have 11 true positives and the total number of all actual positive cases are $11 + 3 = 14$.\n",
157 | "\n",
158 | "| actual\\predicted | a cat | not a cat |\n",
159 | "|------------------|-------|-----------|\n",
160 | "| a cat | 11 | 3 |\n",
161 | "| not a cat | 2 | 9 |\n",
162 | "\n",
163 | "$$\\text{sensitivity} = \\frac{11}{11 + 3} = \\frac{11}{14} = 0.786$$\n",
164 | "\n",
165 | "This means that the sensitivity is about 79\\%. Notice that the sensitivy value is different from precision which is about 85\\%. We can write the formula for sensitivity as follows.\n",
166 | "\n",
167 | "$$\\text{sensitivity} = \\frac{\\text{TP}}{\\text{Total Actual Positives}} = \\frac{\\text{TP}}{\\text{TP} + \\text{FN}}$$\n",
168 | "\n",
169 | "We can see sensitivty as a fraction between the green circle and the blue circle in the image below.\n",
170 | "\n",
171 | "\n"
172 | ]
173 | },
174 | {
175 | "attachments": {},
176 | "cell_type": "markdown",
177 | "metadata": {},
178 | "source": [
179 | "### Specificity\n",
180 | "\n",
181 | "Another common matrix is called specificity. This is the metrix we normally use when we are interested in the negative cases. It is also called as *True Negative Rate*. Specificity is calculated as the number of True Negative cases divided over all actual Negative cases.\n",
182 | "\n",
183 | "| actual\\predicted | a cat | not a cat |\n",
184 | "|------------------|-------|-----------|\n",
185 | "| a cat | 11 | 3 |\n",
186 | "| not a cat | 2 | 9 |\n",
187 | "\n",
188 | "$$\\text{specificity} = \\frac{9}{9 + 2} = \\frac{9}{11} = 0.818$$\n",
189 | "\n",
190 | "This means that the true negative rate is about 82\\%. You can see specificity as the sibling of sensitivity. In sensitivity, you are interested in the positive cases while in the specificity you are interested in the negative cases. The formula for specificity is given as follows.\n",
191 | "\n",
192 | "\n",
193 | "$$\\text{specificity} = \\frac{\\text{TN}}{\\text{Total Actual Negatives}} = \\frac{\\text{TN}}{\\text{FP} + \\text{TN}}$$\n",
194 | "\n",
195 | "We can see specificity as a fraction between the green circle and the blue circle in the image below.\n",
196 | "\n",
197 | ""
198 | ]
199 | },
200 | {
201 | "attachments": {},
202 | "cell_type": "markdown",
203 | "metadata": {},
204 | "source": [
205 | "## Confusion Matrix for Multiple Classes\n",
206 | "\n",
207 | "What if we have more than two categories. What would the confusion matrix look like? Let's say we are classifying images of cat, dog, and a fish. In this case, we have three categories. We can write our confusion matrix in this manner.\n",
208 | "\n",
209 | "| actual\\predicted | cat | dog | fish |\n",
210 | "|------------------|-----|-----|------|\n",
211 | "| cat | 11 | 1 | 2 |\n",
212 | "| dog | 2 | 9 | 3 |\n",
213 | "| fish | 1 | 1 | 8 |\n",
214 | "\n",
215 | "The diagonal element again gives us the accuracy of the model.\n",
216 | "\n",
217 | "$$\\text{accuracy} = \\frac{11 + 9 + 8}{38} = \\frac{28}{38} = 0.737$$\n",
218 | "\n",
219 | "which is about 73\\%. We can write the formula for accuracy as follows.\n",
220 | "\n",
221 | "$$\\text{accuracy} = \\frac{\\sum_i M_{ii}}{\\sum_i\\sum_j{M_{ij}}} $$\n",
222 | "\n",
223 | "The formula simply sums up all the diagonal elements and divide it with the sum of all.\n",
224 | "\n",
225 | "We can calculate the other metrices by defining our positive case in a *one-versus-all* manner. This means that if we define cat as our positive case, we define both dog and fish as the negative cases. \n",
226 | "\n",
227 | "For example, to calculate the sensitivity for class *i*, we use:\n",
228 | "\n",
229 | "$$\\text{sensitivity}_i = \\frac{M_{ii}}{\\sum_j{M_{ij}}}$$\n",
230 | "\n",
231 | "where $i$ is the class we are interested. The summation over $j$ means we sum **over all the columns** in the confusion matrix.\n",
232 | "\n",
233 | "Similarly, we can calculate the precision for class *i* using:\n",
234 | "\n",
235 | "$$\\text{precision}_i = \\frac{M_{ii}}{\\sum_j{M_{ji}}}$$\n",
236 | "\n",
237 | "**Notice that the indices are swapped for the denominator**. For precision, instead of summing over all the columns, we **sum over all the rows** in column *i* which is the total cases when class *i* is *predicted*.\n",
238 | "\n",
239 | "\n"
240 | ]
241 | }
242 | ],
243 | "metadata": {
244 | "kernelspec": {
245 | "display_name": "Python 3 (ipykernel)",
246 | "language": "python",
247 | "name": "python3"
248 | },
249 | "language_info": {
250 | "codemirror_mode": {
251 | "name": "ipython",
252 | "version": 3
253 | },
254 | "file_extension": ".py",
255 | "mimetype": "text/x-python",
256 | "name": "python",
257 | "nbconvert_exporter": "python",
258 | "pygments_lexer": "ipython3",
259 | "version": "3.8.5"
260 | }
261 | },
262 | "nbformat": 4,
263 | "nbformat_minor": 4
264 | }
265 |
--------------------------------------------------------------------------------
/Logistic_Regression.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "attachments": {},
5 | "cell_type": "markdown",
6 | "metadata": {},
7 | "source": [
8 | "
"
24 | ]
25 | },
26 | {
27 | "attachments": {},
28 | "cell_type": "markdown",
29 | "metadata": {},
30 | "source": [
31 | "# Logistic Regression for Classification\n",
32 | "\n",
33 | "By the end of this lesson, you should be able to:\n",
34 | "- Write objective function of **logistic** regression\n",
35 | "- Use logistic regression to **calculate** probabilities of binary classification\n",
36 | "- Train logistic **regression** model\n",
37 | "- Split data into **training**, **validation**, and **testing** set\n",
38 | "\n",
39 | "Important words:\n",
40 | "- classification\n",
41 | "- hypothesis\n",
42 | "- probability\n",
43 | "- logistic function\n",
44 | "- coefficients\n",
45 | "- binary classification\n",
46 | "- cost function\n",
47 | "- gradient descent\n",
48 | "- update function\n",
49 | "- matrix notation"
50 | ]
51 | },
52 | {
53 | "attachments": {},
54 | "cell_type": "markdown",
55 | "metadata": {},
56 | "source": [
57 | "## Introduction\n",
58 | "\n",
59 | "The problem of classification deals with *categorical* data. In this problem, we wish to identify a set of data whether they belong to a particular class of category. For example, a given text message from an email, we would like to classify if it is a spam or not a spam. Another example would be given some measurement of cancer cells we wish to classify if it is benign or malignant. In this section we will learn logistic regression to solve this classification problem."
60 | ]
61 | },
62 | {
63 | "attachments": {},
64 | "cell_type": "markdown",
65 | "metadata": {},
66 | "source": [
67 | "## Hypothesis Function\n",
68 | "\n",
69 | "Let's take an example of breast cancer classification problem. Let's say depending on the cell size, an expert can identify if the cell is benign or malignant. We can plot something like the following figure.\n",
70 | "\n",
71 | "\n",
72 | "\n",
73 | "In the y-axis, value of 1 means it is a malignant cell while value of 0 means it is benign. The x-axis can be considered as a normalized size of the cell with mean 0 and standard deviation of 1 (recall z-normalization).\n",
74 | "\n",
75 | "If we can model this plot as a function $p(x)$, we can set the following criteria to classify the cells. For example, we will predict it is malignant if $p(x) \\geq 0.5$, otherwise, it is benign. This means we need a function where we can model the data in a step wise manners and fulfills the following:\n",
76 | "\n",
77 | "$$0 \\leq p(x) \\leq 1$$\n",
78 | "\n",
79 | "where $p(x)$ is the probability that a cell with feature $x$ is a malignant cell. \n",
80 | "\n",
81 | "One of the function that we can use that have this step-wize shape and the above properties is a logistic function. A logistic function can be written as.\n",
82 | "\n",
83 | "$$y = \\frac{1}{1 + e^{-z}}$$\n",
84 | "\n",
85 | "The plot of a logistic function looks like the following."
86 | ]
87 | },
88 | {
89 | "cell_type": "code",
90 | "execution_count": 2,
91 | "metadata": {},
92 | "outputs": [
93 | {
94 | "data": {
95 | "text/plain": [
96 | "[]"
97 | ]
98 | },
99 | "execution_count": 2,
100 | "metadata": {},
101 | "output_type": "execute_result"
102 | },
103 | {
104 | "data": {
105 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAczElEQVR4nO3deZhU9Z3v8fe3V6Bp1m52sBsEFIkbLZjERCZuYMyQ5cZoFk3Ux5CEOzFm7qOTuZM7z808904mT8xkMRJGuRqT0ZjFSAxGHVQyE6PSGFYFZBOaRmhAWRro7qr63j+qGoqymq5uqvpUnfq8nqepOuf8qurbp6o/nP72WczdERGRwlcSdAEiIpIdCnQRkZBQoIuIhIQCXUQkJBToIiIhURbUC9fU1HhdXV1QLy8iUpBWrly5z91r0y0LLNDr6upobGwM6uVFRAqSmb3Z1TK1XEREQkKBLiISEgp0EZGQUKCLiISEAl1EJCS6DXQzW2xme81sXRfLzcx+YGabzWyNmV2c/TJFRKQ7mWyhPwjMOc3yucDkxNftwH1nXpaIiPRUt/uhu/sfzazuNEPmAT/1+Hl4XzKzIWY22t13Z6tIEQmnaMxpi0TpiDqRaIyOqNMRjRGJpU7HEmOcjliMjkh8TDTmxNxxh5g7scStn3KfU6ZjJ6bjywAc4uOIz0g+q7h3PseJ6ZPjSJp3ynTqN5oyoKFuGB+ckvbYoDOSjQOLxgI7k6abEvPeFehmdjvxrXgmTJiQhZcWkSAd74iy48BR3jnaweHjHRxpi3D4eIQjbRGOJG7j0x0n5h1OWna0PRr0t9BnzE7en3/5pLwNdEszL+1VM9x9EbAIoKGhQVfWECkAHdEYOw8cZdu+1hNf2/e3sq2lleaDx7t8XInBwMoyqvuVM7CyjIH9yhgyoIJxwwZQXVnGwMoyqirL6F9RSlmJUV5aQnlpCWWlRnmpUVZSQnmpJeaVUF5ilCWWVyRuS80oKTFKzCgxKDHDurjtHGNJt8bJoDXslNA1i887eT/+mPjYUwOapGVBykagNwHjk6bHAc1ZeF4R6SPRmNP8zrF4UCcH975Wdr59jGjs5PbX4P7l1NdUMWvicOprqjhr+ABqBlaeCO3qxG3/8tK8CLliko1AXwIsMLNHgVnAQfXPRfJfLOb8eet+HmvcyTPr93Cs42T7o395KfU1VZw3ZjDXnT+G+poq6mqqmFhTxdCqigCrltPpNtDN7BFgNlBjZk3A/wLKAdx9IbAUuBbYDBwFvpCrYkXkzDW9fZRfrWzil41N7HrnGIP6lfHRi8Zy/rjB1A2vYmJtFSOqK7V1XYAy2cvlxm6WO/CVrFUkIll3vCPK0+vf4peNTfxpyz4ALju7hrvmnsPV00bSr7w04AolGwI7fa6I5Ja7s27XIR5r3MkTq3Zx6HiEcUP7c8cVU/jEjLGMGzog6BIlyxToIiFzoLWd3/5lF4817mTDW4epLCth7vRRXN8wnksnDqekRK2UsFKgi4RALOYs39TCY407+Y/X99ARdS4YP4R/+uh0PnLBGAb3Lw+6ROkDCnSRAne8I8pXfv4qyzbsZXhVBTe/t45PNoxn6qjqoEuTPqZAFylgh493cNtDjbyy/QD/88PnctN766go00lUi5UCXaRA7T/Sxs3/7xU27D7Mv37qQuZdODbokiRgCnSRAtT8zjE+98DLNL19jEU3zeBD54wMuiTJAwp0kQKzbV8rn73/ZQ4d6+Cnt8xk1sThQZckeUKBLlJA1jcf5ObFr+AOj9x+KdPHDg66JMkjCnSRArFi+wFueXAF1ZVlPHzbLCbVDgy6JMkzCnSRAvDCxr3M/9lKxgzuz8O3zWLskP5BlyR5SIEukud+t7qZOx9bxeQR1fz01pnUDKwMuiTJUwp0kTz2yCs7+Mbja2k4aygPfP4SBvXTEZ/SNQW6SJ5auHwL//zUBmZPreW+z8ygf4XOiCinp0AXyTPuzrf/sJGFy7dw3fmjuef6C3X0p2REgS6SR6Ix5x+eWMe/v7yDz8yawP+eN51SnR1RMqRAF8kT7ZEYdz62iifX7ObLsyfxP66ZqqsGSY8o0EXyxDceX8uTa3Zz99xzmH/5pKDLkQKkxpxIHnh99yF+tbKJL35wosJcek2BLpIHvvfsJqr7lfHl2WcHXYoUMAW6SMDWNh3kmdf2cNtlExk8QPuZS+8p0EUC9t1nNzJkQDm3XFYXdClS4BToIgFa+eYBXtjYwvzLJ1Gto0DlDCnQRQL03Wc2UTOwgpvee1bQpUgIKNBFAvLiln28uGU/X559NgMqtAexnDkFukgA3J17ntnEqEH9+PSsCUGXIyGhQBcJwPJNLTS++TYLPnQ2/cp10i3JDgW6SB9zd+55dhPjhvbn+obxQZcjIaJAF+ljz762hzVNB/mbKybrLIqSVfo0ifShWCy+dV5fU8XHLxobdDkSMgp0kT60dN1uNrx1mDuunExZqX78JLsy+kSZ2Rwz22hmm83s7jTLB5vZ78xstZmtN7MvZL9UkcIWica459lNTBk5kOvOHxN0ORJC3Qa6mZUC9wJzgWnAjWY2LWXYV4DX3P0CYDbwXTOryHKtIgXtiVXNbG1p5c6rpuiiFZITmWyhzwQ2u/tWd28HHgXmpYxxoNriZ+MfCBwAIlmtVKSAdURjfH/ZG5w3ZhDXnDcq6HIkpDIJ9LHAzqTppsS8ZD8CzgWagbXAV909lvpEZna7mTWaWWNLS0svSxYpPL9a2cSOA0f5+tVTdBUiyZlMAj3dp89Tpq8BVgFjgAuBH5nZoHc9yH2Ruze4e0NtbW0PSxUpTG2RKD9c9gYXjh/CX00dEXQ5EmKZBHoTkHz0wzjiW+LJvgD8xuM2A9uAc7JTokhhe/SVnTQfPM7fXq1rhEpuZRLoK4DJZlaf+EPnDcCSlDE7gCsAzGwkMBXYms1CRQrRsfYoP3p+M7Pqh/H+s4cHXY6EXLeneHP3iJktAJ4GSoHF7r7ezOYnli8EvgU8aGZribdo7nL3fTmsW6Qg/OylN2k53Ma9n75YW+eScxmds9PdlwJLU+YtTLrfDFyd3dJECtuRtgj3Ld/CBybXMLN+WNDlSBHQoWoiOfLQi9s50NrO16+eGnQpUiQU6CI5cPBYBz9ZvoUrzx3BheOHBF2OFAkFukgOPPCfWzl0PMLXrpoSdClSRBToIll2oLWdxX/azrXvGcV5YwYHXY4UEQW6SJb95I9baG2P8LUrtXUufUuBLpJFew8f56EXtzPvgjFMHlkddDlSZBToIll03wtb6Ig6X9XWuQRAgS6SJa1tER55ZQcfu2gs9TVVQZcjRUiBLpIlyzbs5XhHjE/OGBd0KVKkFOgiWbJ0zW5qqytpqNNRoRIMBbpIFrS2RXh+417mTh+lqxFJYBToIlmwbMNe2iIxPvye0UGXIkVMgS6SBWq3SD5QoIucIbVbJF8o0EXOkNotki8U6CJnSO0WyRcKdJEzoHaL5BMFusgZeC7RbrlW7RbJAwp0kTPw+0S75RK1WyQPKNBFekntFsk3CnSRXlK7RfKNAl2kl9RukXyjQBfpBbVbJB8p0EV6Qe0WyUcKdJFeWLp2NzUD1W6R/KJAF+mh1rYIz23Yy7XvUbtF8osCXaSH1G6RfKVAF+khtVskXynQRXpA7RbJZwp0kR5Qu0XyWUaBbmZzzGyjmW02s7u7GDPbzFaZ2XozW57dMkXyg9otks/KuhtgZqXAvcBVQBOwwsyWuPtrSWOGAD8G5rj7DjMbkaN6RQLTeTDRJ2eMV7tF8lImW+gzgc3uvtXd24FHgXkpYz4N/MbddwC4+97slikSvOc27OV4R4wPn692i+SnTAJ9LLAzabopMS/ZFGComb1gZivN7KZ0T2Rmt5tZo5k1trS09K5ikYCo3SL5LpNAT/e7padMlwEzgA8D1wD/YGZT3vUg90Xu3uDuDbW1tT0uViQoOneLFIJue+jEt8jHJ02PA5rTjNnn7q1Aq5n9EbgA2JSVKkUCpnaLFIJMttBXAJPNrN7MKoAbgCUpY54APmBmZWY2AJgFvJ7dUkWCo3aLFIJut9DdPWJmC4CngVJgsbuvN7P5ieUL3f11M/sDsAaIAfe7+7pcFi7SV462a+8WKQyZtFxw96XA0pR5C1OmvwN8J3ulieSHznaLDiaSfKcjRUW68fs18XbLzHq1WyS/KdBFTqOz3aK9W6QQKNBFTkPtFikkCnSR01C7RQqJAl2kC2q3SKFRoIt0Qe0WKTQKdJEuqN0ihUaBLpKG2i1SiBToImmo3SKFSIEukkbnuVvUbpFCokAXSXG0PX4haLVbpNAo0EVSqN0ihUqBLpJC7RYpVAp0kSRqt0ghU6CLJFG7RQqZAl0kidotUsgU6CIJne2WOdNHqt0iBUmBLpJw4kLQ7xkTdCkivaJAF0lQu0UKnQJdBLVbJBwU6CKo3SLhoEAXQe0WCQcFuhQ9tVskLBToUvSe39Cig4kkFBToUvR+v7aZmoEVzKofHnQpImdEgS5F7WS7RedukcKnQJeipnaLhIkCXYqa2i0SJgp0KVpqt0jYKNClaKndImGjQJeiFT+YSO0WCY+MAt3M5pjZRjPbbGZ3n2bcJWYWNbP/lr0SRbLvaHuEZRv2qN0iodJtoJtZKXAvMBeYBtxoZtO6GPdt4OlsFymSbWq3SBhlsoU+E9js7lvdvR14FJiXZtx/B34N7M1ifSI5oXaLhFEmgT4W2Jk03ZSYd4KZjQU+Biw83ROZ2e1m1mhmjS0tLT2tVSQr1G6RsMok0NN94j1l+l+Bu9w9eroncvdF7t7g7g21tbUZliiSXWq3SFiVZTCmCRifND0OaE4Z0wA8amYANcC1ZhZx999mo0iRbFK7RcIqk0BfAUw2s3pgF3AD8OnkAe5e33nfzB4EnlSYSz7qPJjo4xePVbtFQqfbQHf3iJktIL73Simw2N3Xm9n8xPLT9s1F8snzG1o41hHlw+er3SLhk8kWOu6+FFiaMi9tkLv758+8LJHcULtFwkxHikrR6Gy3XHOe9m6RcFKgS9FQu0XCToEuRUPtFgk7BboUBbVbpBgo0KUonGi36GAiCTEFuhSFznbLzPphQZcikjMKdAm9Y+3RE+2WslJ95CW89OmW0Ht+4161W6QoKNAl9H6/Ru0WKQ4KdAk1tVukmOgTLqGmdosUEwW6hJraLVJMFOgSWmq3SLHRp1xCS+0WKTYKdAmt36/dzfAqtVukeCjQJZSOtUd57vW9zJmudosUD33SJZTUbpFipECXUFK7RYqRAl1CR+0WKVb6tEvoPPPaW2q3SFFSoEuoRGPOD5a9wdkjBjJroq5MJMVFgS6h8sSqXWxpaeVrV07RlYmk6CjQJTQ6ojG+v+wNzh09iLnTRwVdjkifU6BLaPx6ZRNv7j/K16+aQom2zqUIKdAlFNoiUX743GYuGD+EK84dEXQ5IoFQoEso/GLFTna9c4yvXzUFM22dS3FSoEvBO94R5UfPbeaSuqF8YHJN0OWIBEaBLgXvZy+9yd7DbXz96qnaOpeipkCXgtbaFuHHL2zhsrNruFT7nUuRU6BLQXvwxe0caG3nzqunBF2KSOAyCnQzm2NmG81ss5ndnWb5Z8xsTeLrRTO7IPulipzq4LEOfrJ8Cx86ZwQXTxgadDkiges20M2sFLgXmAtMA240s2kpw7YBl7v7+cC3gEXZLlQk1QP/tY1DxyPceZW2zkUgsy30mcBmd9/q7u3Ao8C85AHu/qK7v52YfAkYl90yRU71dms7i/9rG3POG8X0sYODLkckL2QS6GOBnUnTTYl5XbkVeCrdAjO73cwazayxpaUl8ypFUvzkj1tpbY/wNW2di5yQSaCn2w/M0w40+yvigX5XuuXuvsjdG9y9oba2NvMqRZK0HG7joRe385HzxzB1VHXQ5YjkjbIMxjQB45OmxwHNqYPM7HzgfmCuu+/PTnki73bfC1toi0S548rJQZciklcy2UJfAUw2s3ozqwBuAJYkDzCzCcBvgM+5+6bslykSt/vgMX728pt84uJxTKwdGHQ5Inml2y10d4+Y2QLgaaAUWOzu681sfmL5QuCbwHDgx4kj9SLu3pC7sqVY3fv8ZmIx52+u0Na5SKpMWi64+1Jgacq8hUn3bwNuy25pIqfaeeAov1ixk09dMp7xwwYEXY5I3tGRolIwfvjcG5gZCz50dtCliOQlBboUhK0tR/j1q7v4zKwJjB7cP+hyRPKSAl0KwveXvUFFaQlfmj0p6FJE8pYCXfLepj2HWbK6mZvedxYjqvsFXY5I3lKgS9773rObqKooY/4HtXUucjoKdMlr63Yd5Kl1b3HL++sYWlURdDkieU2BLnnte89uYlC/Mm79wMSgSxHJewp0yVuv7nibZRv28sXLJzG4f3nQ5YjkPQW65K3vPbuJYVUVfP59dUGXIlIQFOiSl17aup//fGMfX7p8ElWVGR3QLFL0FOiSd15rPsSCf/8Lowb147OXnhV0OSIFQ4EueaVx+wE+tejPlJcaP7ttFv0rSoMuSaRg6HdZyRvLN7XwxYcbGT24Pw/fOpNxQ3UCLpGeUKBLXnhyTTNf+8UqJo+o5qFbZlJbXRl0SSIFR4EugXvklR184/G1NJw1lPtvvkS7KIr0kgJdAvWT5Vv4v09t4PIptSz87Az1zEXOgAJdAuHu/MvTG7nvhS1cd/5o7rn+QirK9Dd6kTOhQJc+F40533xiHT9/eQc3zpzAP310OqUlFnRZIgVPgS59qiMa487HVvO71c3Mv3wSd82ZSuI6tCJyhhTo0meOtUf58s9X8vzGFu6ac44uViGSZQp06ROHjndw24ONrHjzAP/nY+/h07MmBF2SSOgo0CXn9h1p4+bFr7Bpz2F+cMNFfOSCMUGXJBJKCnTJqV3vHONz979M88Fj/NtNDcyeOiLokkRCS4EuObN572FueuAVDrdFePjWWVxSNyzokkRCTYEuWdUeifHchj081tjECxv3Mqyqgkdvv5TzxgwOujSR0FOgS1ZseOsQv2xs4vG/7OJAazsjB1XypdmT+OylZzF6cP+gyxMpCgp06bWDxzpYsrqZXzbuZE3TQcpLjaumjeSTDeP54ORaHSwk0scU6NIjsZjz5637eaxxJ39Y9xZtkRjnjKrmm9dN46MXjWVYVUXQJYoULQW6ZKTp7aP8amUTv2xsYtc7xxjUr4zrG8ZzfcN4po8dpKM9RfKAAl3SOni0g237W9n01mGWrG7mT1v2AXDZ2TXcNfccrp42kn7lOjOiSD5RoBexo+0Rtu87yrZ9rWzf38rWlvjttn2tHGhtPzFu3ND+3HHFFD4xY6yuIiSSxzIKdDObA3wfKAXud/d/TlluieXXAkeBz7v7q1muVXrA3TneEeNwWwcHj3awff9Rtu9rZeu+Vrbvi4f2W4eOn/KYkYMqqa+p4przRlJfU0Xd8Crqa6qYVDuQEv2BUyTvdRvoZlYK3AtcBTQBK8xsibu/ljRsLjA58TULuC9xKwnuTjTmRGJOezRGJOpEojE6YonbaIyOqBOJOh2x+PKOxPxINP6YI8cjHG6LcOR4hCNtHRxpi3D4eIQjJ+YlTbdFiMb8XXUMHVBOfU0V7z+7hvqaAdTXDKSuZgB1w6uoqtQvbCKFLJOf4JnAZnffCmBmjwLzgORAnwf81N0deMnMhpjZaHffne2Cl29q4VtPnnzp+Eue9K4I8/ST7o4DnQ93HPeT08nP3Tmuc0zM48tiieeIxTrnO7HE7cnp+Lxs619eysB+ZVRXljGwXxkDK8uYUDWA6n7lVCemO+dX9ytjwrAB1NdUMWSA9kIRCatMAn0ssDNpuol3b32nGzMWOCXQzex24HaACRN6d7a9gZVlTB1ZfepMO+3ku/bAsBPz4/c7l1viH0uM6Fx+8r5RUhIfb0CJGSUWn+68X1JimJ1cVpIYa2aUlhhlpUZFaQllJUZZaQnlpUZZSQnlZSWUJ+aVlRrlJYnbxJiKspJ4OFeWU1VZSlmpru4jIqfKJNDTNU9TtzkzGYO7LwIWATQ0NPRqu3XGWUOZcdbQ3jxURCTUMtnMawLGJ02PA5p7MUZERHIok0BfAUw2s3ozqwBuAJakjFkC3GRxlwIHc9E/FxGRrnXbcnH3iJktAJ4mvtviYndfb2bzE8sXAkuJ77K4mfhui1/IXckiIpJORvupuftS4qGdPG9h0n0HvpLd0kREpCe0q4SISEgo0EVEQkKBLiISEgp0EZGQsNRD5/vshc1agDd7+fAaYF8Wy8mWfK0L8rc21dUzqqtnwljXWe5em25BYIF+Jsys0d0bgq4jVb7WBflbm+rqGdXVM8VWl1ouIiIhoUAXEQmJQg30RUEX0IV8rQvytzbV1TOqq2eKqq6C7KGLiMi7FeoWuoiIpFCgi4iERN4Gupl90szWm1nMzBpSlv2dmW02s41mdk0Xjx9mZs+a2RuJ26xfFcPMfmFmqxJf281sVRfjtpvZ2sS4xmzXkeb1/tHMdiXVdm0X4+Yk1uFmM7u7D+r6jpltMLM1Zva4mQ3pYlyfrK/uvv/E6aB/kFi+xswuzlUtSa853syeN7PXE5//r6YZM9vMDia9v9/MdV1Jr33a9yagdTY1aV2sMrNDZnZHypg+WWdmttjM9prZuqR5GWVRVn4e3T0vv4BzganAC0BD0vxpwGqgEqgHtgClaR7/L8Ddift3A9/Ocb3fBb7ZxbLtQE0frrt/BP62mzGliXU3EahIrNNpOa7raqAscf/bXb0nfbG+Mvn+iZ8S+iniV+S6FHi5D9670cDFifvVwKY0dc0Gnuyrz1NP3psg1lma9/Ut4gff9Pk6Az4IXAysS5rXbRZl6+cxb7fQ3f11d9+YZtE84FF3b3P3bcTPwT6zi3EPJe4/BHw0J4US3yoBrgceydVr5MCJi3+7ezvQefHvnHH3Z9w9kph8ifiVrYKSyfd/4uLn7v4SMMTMRueyKHff7e6vJu4fBl4nfn3eQtHn6yzFFcAWd+/tUehnxN3/CBxImZ1JFmXl5zFvA/00urogdaqRnrhqUuJ2RA5r+gCwx93f6GK5A8+Y2UqLXyi7LyxI/Mq7uItf8TJdj7lyC/EtuXT6Yn1l8v0Huo7MrA64CHg5zeL3mtlqM3vKzM7rq5ro/r0J+nN1A11vWAW1zjLJoqyst4wucJErZvYfwKg0i/7e3Z/o6mFp5uVs38sMa7yR02+dv9/dm81sBPCsmW1I/E+ek7qA+4BvEV8v3yLeDrol9SnSPPaM12Mm68vM/h6IAD/v4mmyvr7SlZpmXq8ufp4LZjYQ+DVwh7sfSln8KvGWwpHE30d+C0zui7ro/r0Jcp1VAH8N/F2axUGus0xkZb0FGujufmUvHpbpBan3mNlod9+d+JVvby5qNLMy4OPAjNM8R3Pidq+ZPU7816szCqhM152Z/RvwZJpFObmwdwbr62bgOuAKTzQP0zxH1tdXGnl78XMzKyce5j9399+kLk8OeHdfamY/NrMad8/5SagyeG+CvGD8XOBVd9+TuiDIdUZmWZSV9VaILZclwA1mVmlm9cT/l32li3E3J+7fDHS1xX+mrgQ2uHtTuoVmVmVm1Z33if9hcF26sdmS0rP8WBevl8nFv7Nd1xzgLuCv3f1oF2P6an3l5cXPE3+PeQB43d3v6WLMqMQ4zGwm8Z/j/bmsK/Fambw3QV4wvsvflINaZwmZZFF2fh5z/Vff3n4RD6ImoA3YAzydtOzvif9FeCMwN2n+/ST2iAGGA8uANxK3w3JU54PA/JR5Y4ClifsTif/FejWwnnjrIdfr7mFgLbAm8aEYnVpXYvpa4ntRbOmjujYT7xOuSnwtDHJ9pfv+gfmd7yfxX4PvTSxfS9LeVjms6TLiv2qvSVpP16bUtSCxblYT/+Py+3Jd1+nem6DXWeJ1BxAP6MFJ8/p8nRH/D2U30JHIr1u7yqJc/Dzq0H8RkZAoxJaLiIikoUAXEQkJBbqISEgo0EVEQkKBLiISEgp0EZGQUKCLiITE/weHRU8GHbId0gAAAABJRU5ErkJggg==",
106 | "text/plain": [
107 | ""
108 | ]
109 | },
110 | "metadata": {
111 | "needs_background": "light"
112 | },
113 | "output_type": "display_data"
114 | }
115 | ],
116 | "source": [
117 | "import numpy as np\n",
118 | "import matplotlib.pyplot as plt\n",
119 | "\n",
120 | "z = np.array(range(-10,11))\n",
121 | "y = 1/(1+np.exp(-z))\n",
122 | "plt.plot(z,y)"
123 | ]
124 | },
125 | {
126 | "attachments": {},
127 | "cell_type": "markdown",
128 | "metadata": {},
129 | "source": [
130 | "We can write our hypothesis as follows.\n",
131 | "\n",
132 | "$$p(x) = \\frac{1}{1 + e^{-z(x)}}$$\n",
133 | "\n",
134 | "where $z$ is a function of $x$. What should be this $z$ function. We can then use our linear model of a straight line and transform it into a logistic function if we use the following transformation.\n",
135 | "\n",
136 | "$$z(x) = \\beta_0 x_0 + \\beta_1 x_1$$\n",
137 | "\n",
138 | "when $x_0 = 1$, the above equation is simply the straight line equation of linear regression.\n",
139 | "\n",
140 | "$$\\beta_0 + \\beta_1 x_1$$\n",
141 | "\n",
142 | "This is the case when we only have one feature $x_1$. If we have more than one feature, we should write as follows.\n",
143 | "\n",
144 | "$$z(x) = \\beta_0 x_0 + \\beta_1 x_1 + \\ldots + \\beta_n x_n$$\n",
145 | "\n",
146 | "Note that in this notes we tend to omit the *hat* symbol to indicate it is the estimated parameters as in the previous notes. We will just indicate the estimated parameters as $\\beta$ instead of $\\hat{\\beta}$. \n",
147 | "\n",
148 | "The above relationship shows that we can map the value of linear regression into a new function with a value from 0 to 1. This new function $p(x)$ can be considered as *an estimated probability* that $y = 1$ on input $x$. For example, if $p(x) = 0.7$ this means that 70% chance it is malignant. We can then use the following boundary conditions:\n",
149 | "- y = 1 (malignant) if $p(x) \\geq 0.5$\n",
150 | "- y = 0 (benign) if $p(x) < 0.5$\n",
151 | "\n",
152 | "The above conditions also means that we can classify $y=1$ when $\\beta^T x \\geq 0$ and $y = 0$ when $\\beta^T x < 0$. We can draw these boundary conditions.\n",
153 | "\n",
154 | "\n",
155 | "\n",
156 | "In the figure above, we indicated the predicted label $y$ with the orange colour. We see that when $p(x)\\geq 0.5$, the data is marked as $y=1$ (orange). On the other hand, when $p(x) \\leq 0.5$, the data is marked as $y=0$ (orange). The thick black line shows the decision boundary for this particular example. \n",
157 | "\n",
158 | "How do we get this boundary decision. Once we found the estimated values for $\\beta$, we can find the value of $x$ which gives $\\beta^Tx = 0$. You will work on computing the parameters $\\beta$ in the problem set. For now, let's assume that you manage to find the value of $\\beta_0 = -0.56$ and $\\beta_1 = 1.94$. The equation $\\beta^T x = 0 $ can be written as follows.\n",
159 | "\n",
160 | "$$\\beta_0 + \\beta_1 x = 0$$\n",
161 | "\n",
162 | "We can then substitute the values for $\\beta$ into the equation.\n",
163 | "\n",
164 | "$$-0.56 + 1.94 x = 0$$\n",
165 | "$$x = 0.29 \\approx 0.3$$\n",
166 | "\n",
167 | "From the figure above, this fits where the thick line is, which is at around 0.3."
168 | ]
169 | },
170 | {
171 | "attachments": {},
172 | "cell_type": "markdown",
173 | "metadata": {},
174 | "source": [
175 | "## Cost Function\n",
176 | "\n",
177 | "Similar to linear regression, our purpose here is to find the parameters $\\beta$. To do so, we will have to minimize some cost function using optimization algorithm. \n",
178 | "\n",
179 | "For logistic regression, we will choose the following cost function.\n",
180 | "\n",
181 | "$$J(\\beta) = \\frac{1}{m} \\Sigma_{i=1}^m \\left\\{ \\begin{matrix}\n",
182 | "-\\log(p(x)) & \\text{ if } y = 1\\\\\n",
183 | "-\\log(1 - p(x)) & \\text{ if } y = 0\n",
184 | "\\end{matrix}\\right.$$\n",
185 | "\n",
186 | "We can try to understand the term inside the bracket intuitively. Let's see the case when $y=1$. In this case, the cost term is given by:\n",
187 | "\n",
188 | "$$-\\log(p(x))$$\n",
189 | "\n",
190 | "The cost is 0 if $y = 1$ and $p(x) = 1$ because $-\\log(z)$ is 0 when $z=1$. Moreover, as $p(x) \\rightarrow 0$, the cost will reach $\\infty$. [See plot by wolfram alpha](https://www.wolframalpha.com/input/?i=-log%28x%29+from+0+to+1).\n",
191 | "\n",
192 | "On the other hand, when $ y = 0$, the cost term is given by:\n",
193 | "\n",
194 | "$$-\\log(1-p(x))$$\n",
195 | "\n",
196 | "In this case, the cost is 0 when $p(x) = 0$ but it reaches $\\infty$ when $p(x) \\rightarrow 1$. [See plot by wolfram alpha](https://www.wolframalpha.com/input/?i=-log%281-x%29+from+0+to+1). \n",
197 | "\n",
198 | "We can write the overall cost function for all the data points from $i=1$ to $m$ as follows.\n",
199 | "\n",
200 | "$$J(\\beta) = -\\frac{1}{m}\\left[\\Sigma_{i=1}^m y^i \\log(p(x^i)) + (1 - y^i) \\log(1 - p(x^i))\\right]$$\n",
201 | "\n",
202 | "Notice that when $y^i = 1$, the function reduces to\n",
203 | "\n",
204 | "$$J(\\beta) = -\\frac{1}{m}\\left[\\Sigma_{i=1}^m \\log(p(x^i)) \\right]$$\n",
205 | "\n",
206 | "and when $y^i = 0$, the function reduces to\n",
207 | "\n",
208 | "$$J(\\beta) = -\\frac{1}{m}\\left[\\Sigma_{i=1}^m \\log(1 - p(x^i))\\right]$$"
209 | ]
210 | },
211 | {
212 | "attachments": {},
213 | "cell_type": "markdown",
214 | "metadata": {},
215 | "source": [
216 | "## Gradient Descent\n",
217 | "\n",
218 | "We can find the parameters $\\beta$ again by using the gradient descent algorithm to perform:\n",
219 | "\n",
220 | "$$\\begin{matrix}\n",
221 | "min & J(\\beta)\\\\\n",
222 | "\\beta & \\end{matrix}$$\n",
223 | "\n",
224 | "The update functions for the parameters are given by\n",
225 | "\n",
226 | "$$\\beta_j = \\beta_j - \\alpha \\frac{\\partial}{\\partial \\beta_j} J(\\beta)$$\n",
227 | "\n",
228 | "The derivative of the cost function is given by\n",
229 | "\n",
230 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = \\frac{1}{m}\\Sigma_{i=1}^m \\left(p(x)-y^i \\right)x_j^i$$\n",
231 | "\n",
232 | "See the appendix for the derivation. We can substitute this in to get the following update function.\n",
233 | "\n",
234 | "$$\\beta_j = \\beta_j - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m \\left(p(x)-y^i \\right)x_j^i$$\n"
235 | ]
236 | },
237 | {
238 | "attachments": {},
239 | "cell_type": "markdown",
240 | "metadata": {},
241 | "source": [
242 | "## Matrix Notation\n",
243 | "\n",
244 | "The above equations can be written in matrix notation so that we can perform a vectorized computation. \n",
245 | "\n"
246 | ]
247 | },
248 | {
249 | "attachments": {},
250 | "cell_type": "markdown",
251 | "metadata": {},
252 | "source": [
253 | "### Hypothesis Function\n",
254 | "\n",
255 | "Recall that our hypothesis can be written as:\n",
256 | "\n",
257 | "$$p(x) = \\frac{1}{1 + e^{-z(x)}}$$\n",
258 | "\n",
259 | "where\n",
260 | "\n",
261 | "$$z(x) = \\beta_0 x_0 + \\beta_1 x_1 + \\ldots + \\beta_n x_n$$\n",
262 | "\n",
263 | "We can write this equation as vector multiplication as follows.\n",
264 | "\n",
265 | "$$z = \\mathbf{b}^T \\mathbf{x}$$\n",
266 | "\n",
267 | "and\n",
268 | "\n",
269 | "$$p(x) = \\frac{1}{1 + e^{-\\mathbf{b}^T \\mathbf{x}}}$$\n",
270 | "\n",
271 | "where\n",
272 | "\n",
273 | "$$\\mathbf{b} = \\begin{bmatrix}\n",
274 | "\\hat{\\beta}_0\\\\\n",
275 | "\\hat{\\beta}_1 \\\\\n",
276 | "\\ldots\\\\\n",
277 | "\\hat{\\beta}_n\n",
278 | "\\end{bmatrix}$$\n",
279 | "\n",
280 | "and\n",
281 | "$$\\mathbf{x} = \\begin{bmatrix}\n",
282 | "1 \\\\\n",
283 | "x_1 \\\\\n",
284 | "x_2 \\\\\n",
285 | "\\ldots \\\\\n",
286 | "x_n \\\\\n",
287 | "\\end{bmatrix}$$\n",
288 | "\n",
289 | "Recall that this is for a single data with $n$ features. The result of this vector multiplication $z$ is a single number for that one single data with $n$ features. \n",
290 | "\n"
291 | ]
292 | },
293 | {
294 | "attachments": {},
295 | "cell_type": "markdown",
296 | "metadata": {},
297 | "source": [
298 | "### Cost Function\n",
299 | "\n",
300 | "Recall that the cost function for all the data points from $i=1$ to $m$ as follows.\n",
301 | "\n",
302 | "$$J(\\beta) = -\\frac{1}{m}\\left[\\Sigma_{i=1}^m y^i \\log(p(x^i)) + (1 - y^i) \\log(1 - p(x^i))\\right]$$\n",
303 | "\n",
304 | "Notice that when $y^i = 1$, the function reduces to\n",
305 | "\n",
306 | "$$J(\\beta) = -\\frac{1}{m}\\left[\\Sigma_{i=1}^m \\log(p(x^i)) \\right]$$\n",
307 | "\n",
308 | "and when $y^i = 0$, the function reduces to\n",
309 | "\n",
310 | "$$J(\\beta) = -\\frac{1}{m}\\left[\\Sigma_{i=1}^m \\log(1 - p(x^i))\\right]$$\n",
311 | "\n",
312 | "How can we vectorize this computation in Python? Numpy provides the function `np.where()` which we can use if we have more than one computation depending on certain conditions. \n",
313 | "\n",
314 | "For example, if we have an input `x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]`, we can compute the value of y on whether $x$ is even or odd. Let's say, we will square the value if the value is even. On the other hand, we will leave the value as it is if it is odd. Below cell shows how the code can be written."
315 | ]
316 | },
317 | {
318 | "cell_type": "code",
319 | "execution_count": 3,
320 | "metadata": {},
321 | "outputs": [
322 | {
323 | "name": "stdout",
324 | "output_type": "stream",
325 | "text": [
326 | "[ 0 1 4 3 16 5 36 7 64 9]\n"
327 | ]
328 | }
329 | ],
330 | "source": [
331 | "# create a list from 0 to 9\n",
332 | "x = list(range(10))\n",
333 | "\n",
334 | "# using np.where()\n",
335 | "y = np.where(np.mod(x,2) == 0, np.power(x,2), x)\n",
336 | "print(y)"
337 | ]
338 | },
339 | {
340 | "attachments": {},
341 | "cell_type": "markdown",
342 | "metadata": {},
343 | "source": [
344 | "We can, thus, use `np.where()` to calculate the cost function depending on whether $y^i$ is 1 or zero using the two equations above. The summation in the above equation can be computed using `np.sum()`.\n",
345 | "\n",
346 | "An example of using `np.sum()` can be seen in the below cell."
347 | ]
348 | },
349 | {
350 | "cell_type": "code",
351 | "execution_count": 4,
352 | "metadata": {},
353 | "outputs": [
354 | {
355 | "name": "stdout",
356 | "output_type": "stream",
357 | "text": [
358 | "45\n"
359 | ]
360 | }
361 | ],
362 | "source": [
363 | "# create a list from 0 to 9\n",
364 | "x = list(range(10))\n",
365 | "\n",
366 | "# using np.sum() to sum up all the numbers in the vectors\n",
367 | "y = np.sum(x)\n",
368 | "print(y)"
369 | ]
370 | },
371 | {
372 | "attachments": {},
373 | "cell_type": "markdown",
374 | "metadata": {},
375 | "source": [
376 | "If you are dealing with a matrix, you can specify the axis of `np.sum()` whether you want to sum over the rows or the columns. By default is over the rows or `axis=0` in Numpy."
377 | ]
378 | },
379 | {
380 | "cell_type": "code",
381 | "execution_count": 5,
382 | "metadata": {},
383 | "outputs": [
384 | {
385 | "name": "stdout",
386 | "output_type": "stream",
387 | "text": [
388 | "[5 7 9]\n"
389 | ]
390 | }
391 | ],
392 | "source": [
393 | "x = [[1, 2, 3], [4, 5, 6]]\n",
394 | "print(np.sum(x, axis=0))\n"
395 | ]
396 | },
397 | {
398 | "attachments": {},
399 | "cell_type": "markdown",
400 | "metadata": {},
401 | "source": [
402 | "In the above code we sum over the rows and so we have three values for each column. If we wish to sum over all the columns, we should do as the one below."
403 | ]
404 | },
405 | {
406 | "cell_type": "code",
407 | "execution_count": 6,
408 | "metadata": {},
409 | "outputs": [
410 | {
411 | "name": "stdout",
412 | "output_type": "stream",
413 | "text": [
414 | "[ 6 15]\n"
415 | ]
416 | }
417 | ],
418 | "source": [
419 | "x = [[1, 2, 3], [4, 5, 6]]\n",
420 | "print(np.sum(x, axis=1))"
421 | ]
422 | },
423 | {
424 | "attachments": {},
425 | "cell_type": "markdown",
426 | "metadata": {},
427 | "source": [
428 | "In the above output, we see that 6 is the sum of `[1, 2, 3]` and 15 is the sum of `[4, 5, 6]`. "
429 | ]
430 | },
431 | {
432 | "attachments": {},
433 | "cell_type": "markdown",
434 | "metadata": {},
435 | "source": [
436 | "### Gradient Descent Update\n",
437 | "\n",
438 | "Recall that the update function in our gradient descent calculation was the following.\n",
439 | "\n",
440 | "$$\\beta_j = \\beta_j - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m \\left(p(x)-y^i \\right)x_j^i$$\n",
441 | "\n",
442 | "We can write this a vectorized calculation particularly because we have the summation of some multiplication terms. This sounds like a good candidate for a matrix multiplication. Recall that our hypothesis for $m$ data points is a column vector.\n",
443 | "\n",
444 | "$$\\mathbf{p}(x) = \\frac{1}{1 + e^{-\\mathbf{X}\\mathbf{b}}}$$\n",
445 | "\n",
446 | "Similarly, $y$ which is the actual target value from the training set can be written as a column vector of size $m\\times 1$. Therefore, we can do the calculation element-wise for the following term.\n",
447 | "\n",
448 | "$$\\mathbf{p} - \\mathbf{y}$$\n",
449 | "\n",
450 | "The result is a column vector too.\n",
451 | "\n",
452 | "The features $x_j^i$ can be arranged as a matrix as shown below.\n",
453 | "\n",
454 | "$$\\mathbf{X} = \\begin{bmatrix}\n",
455 | "1 & x_1^1 & x_2^1 & \\ldots & x_n^1 \\\\\n",
456 | "1 & x_1^2 & x_2^2 & \\ldots & x_n^2 \\\\\n",
457 | "\\ldots & \\ldots & \\ldots & \\ldots & \\ldots \\\\\n",
458 | "1 &x_1^m & x_2^2 & \\ldots & x_n^m \n",
459 | "\\end{bmatrix}$$\n",
460 | "\n",
461 | "We can do the multiplication and the summation as a matrix multiplication of the following equation.\n",
462 | "\n",
463 | "$$\\mathbf{X}^T(\\mathbf{p} - \\mathbf{y})$$\n",
464 | "\n",
465 | "Note that we transpose the matrix $\\mathbf{X}$ so that it has the shape of $(1+n) \\times m$. In this way, we can do matrix multiplication with $(\\mathbf{p} - \\mathbf{y})$ which has the shape of $m \\times 1$. \n",
466 | "\n",
467 | "The rest of the computation is just a multiplication of some constants. So we can write our update function as follows.\n",
468 | "\n",
469 | "$$\\mathbf{b} = \\mathbf{b} - \\alpha\\frac{1}{m}\\mathbf{X}^T(\\mathbf{p} - \\mathbf{y}) $$"
470 | ]
471 | },
472 | {
473 | "attachments": {},
474 | "cell_type": "markdown",
475 | "metadata": {},
476 | "source": [
477 | "## Multi-Class\n",
478 | "\n",
479 | "Since Logistic function's output range only from 0 to 1, does it mean that it can only predict binary classification, i.e. classification problem involving only two classes? The answer is no. We can extend the technique to apply to multi-class classification by using a technique called one-versus-all. \n",
480 | "\n",
481 | "The idea of one-versus-all technique is to reduce the multi-class classification problem to binary classification problem. Let's say we have three class and we would like to predict between cat, dog, and fish images. We can treat this problem as binary classification by predicting if an image is a cat or no cat. In this first instance, we treat both dog and fish images as a no-cat image. We then repeat the same procedures and try to predict if an image is a dog or a no-dog image. Similarly, we do the same with the fish and no-fish image. \n",
482 | "\n",
483 | "To facilitate this kind of prediction, instead of having **one** target column in the **training set** , we will be preparing **three** target columns, each column for each class. We need to prepare something like the following data set.\n",
484 | "\n",
485 | "| feature_1 | feature_2 | cat | dog | fish |\n",
486 | "|-----------|-----------|-----|-----|------|\n",
487 | "| x | x | 1 | 0 | 0 |\n",
488 | "| x | x | 1 | 0 | 0 |\n",
489 | "| x | x | 0 | 1 | 0 |\n",
490 | "| x | x | 0 | 0 | 1 |\n",
491 | "| x | x | 0 | 1 | 0 |\n",
492 | "\n",
493 | "We can then train the model **three times** and obtain the coefficients for **each class**. In this example, we would have **three sets** of beta coefficients, one for the cat versus no-cat, another one for dog versus no-dog, and the last one for fish versus no-fish. We can then use these coefficients to calculate the probability for each class and produce the probability.\n",
494 | "\n",
495 | "Recall that our hypothesis function returns a probability between 0 to 1.\n",
496 | "\n",
497 | "$$\\mathbf{p}(x) = \\frac{1}{1 + e^{-\\mathbf{Xb}}}$$\n",
498 | "\n",
499 | "We can then construct three columns where each column contains the probability for the particular binary classification relevant to the column target. For example, we can have something like the following table.\n",
500 | "\n",
501 | "| feature_1 | feature_2 | cat | dog | fish | predicted class |\n",
502 | "|-----------|-----------|-----|-----|------|-------|\n",
503 | "| x | x | **0.8** | 0.2 | 0.3 | cat |\n",
504 | "| x | x | **0.9** | 0.1 | 0.2 | cat |\n",
505 | "| x | x | 0.5 | **0.9** | 0.4 | dog |\n",
506 | "| x | x | 0.3 | 0.2 | **0.8** | fish |\n",
507 | "| x | x | 0.1 | **0.7** | 0.5 | dog |\n",
508 | "\n",
509 | "In the above example, the first two rows have cat class as their highest probability. Therefore, we set \"cat\" as the predicted class in the last column. On the other hand, the third and the last row have \"dog\" as their highest probability and therefore, they are predicted as \"dog\". Similarly, with \"fish\" in the second last row."
510 | ]
511 | },
512 | {
513 | "attachments": {},
514 | "cell_type": "markdown",
515 | "metadata": {},
516 | "source": [
517 | "# Appendix\n",
518 | "\n",
519 | "## Derivation of Logistic Regression Derivative \n",
520 | "\n",
521 | "We want to find $\\frac{\\partial}{\\partial \\beta_j}J(\\beta)$, where\n",
522 | "\n",
523 | "$$J(\\beta) = -\\frac{1}{m}\\left[\\Sigma_{i=1}^m y^i \\log(p(x^i)) + (1 - y^i) \\log(1 - p(x^i))\\right]$$\n",
524 | "\n",
525 | "To simplify our derivation, we will consider each case when $y=1$ and when $y=0$. When $y=1$, the cost function is given by\n",
526 | "\n",
527 | "$$J(\\beta) = -\\frac{1}{m}\\left[\\Sigma_{i=1}^m \\log(p(x^i)) \\right]$$\n",
528 | "\n",
529 | "Derivating this with respect to $\\theta$ is\n",
530 | "\n",
531 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = -\\frac{1}{m}\\Sigma \\frac{1}{p(x)}\\frac{\\partial}{\\partial \\beta}p(x)$$\n",
532 | "\n",
533 | "Recall that the expression for the hypothesis is\n",
534 | "\n",
535 | "$$p(x) = \\frac{1}{1 + e^{-\\beta^T x}}$$\n",
536 | "\n",
537 | "The derivative of this is given by\n",
538 | "\n",
539 | "$$\\frac{\\partial}{\\partial \\beta_j} p(x) = - \\frac{1}{(1 + e^{-\\beta^T x})^2} \\times -x_j \\times e^{-\\beta^T x}$$\n",
540 | "\n",
541 | "or\n",
542 | "\n",
543 | "$$\\frac{\\partial}{\\partial \\beta_j} p(x) = \\frac{x_j e^{-\\beta^T x}}{(1 + e^{-\\beta^T x})^2} $$\n",
544 | "\n",
545 | "We can then now substitute this back \n",
546 | "\n",
547 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = -\\frac{1}{m}\\Sigma (1 + e^{-\\beta^T x}) \\frac{x_j e^{-\\beta^T x}}{(1 + e^{-\\beta^T x})^2}$$\n",
548 | "\n",
549 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = -\\frac{1}{m}\\Sigma \\frac{x_j e^{-\\beta^T x}}{(1 + e^{-\\beta^T x})}$$\n",
550 | "\n",
551 | "This can be written as\n",
552 | "\n",
553 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = -\\frac{1}{m}\\Sigma p(x) x_j e^{-\\beta^T x}$$\n",
554 | "This is for the case of $y = 1$.\n",
555 | "\n",
556 | "Now let's do the same for $y = 0$, the cost function is given by\n",
557 | "\n",
558 | "$$J(\\beta) = -\\frac{1}{m}\\left[\\Sigma_{i=1}^m \\log(1 - p(x^i))\\right]$$\n",
559 | "\n",
560 | "Derivating this with respect to $\\theta$ gives\n",
561 | "\n",
562 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = \\frac{1}{m}\\Sigma \\frac{1}{1 - p(x)}\\frac{\\partial}{\\partial \\beta}p(x)$$\n",
563 | "\n",
564 | "Substituting expression for the hypothesis function and its derivative gives us\n",
565 | "\n",
566 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = \\frac{1}{m}\\Sigma \\frac{1}{1 - \\frac{1}{1 + e^{-\\beta^T x}}} \\frac{x_j e^{-\\beta^T x}}{(1 + e^{-\\beta^T x})^2} $$\n",
567 | "\n",
568 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = \\frac{1}{m}\\Sigma \\frac{1 + e^{-\\beta^T x}}{e^{-\\beta^T x}} \\frac{x_j e^{-\\beta^T x}}{(1 + e^{-\\beta^T x})^2} $$\n",
569 | "\n",
570 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = \\frac{1}{m}\\Sigma \\frac{x_j}{(1+e^{\\beta^T x})} $$\n",
571 | "\n",
572 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = \\frac{1}{m}\\Sigma p(x) x_j$$\n",
573 | "This is for $y = 0$.\n",
574 | "\n",
575 | "Combining for both cases $y=0$ and $y=1$, we have\n",
576 | "\n",
577 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = -\\frac{1}{m}\\Sigma_{i=1}^m y^i p(x) x_j e^{-\\beta^T x} + (y^i - 1) p(x) x_j^i$$\n",
578 | "\n",
579 | "\n",
580 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = -\\frac{1}{m}\\Sigma_{i=1}^m y^i p(x) x_j e^{-\\beta^T x} + y^i p(x) x_j - p(x) x_j^i$$\n",
581 | "\n",
582 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = -\\frac{1}{m}\\Sigma_{i=1}^m \\left(y^i p(x)(1 + e^{-\\beta^T x}) - p(x) \\right)x_j^i$$\n",
583 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = -\\frac{1}{m}\\Sigma_{i=1}^m \\left(y^i - p(x) \\right)x_j^i$$\n",
584 | "$$\\frac{\\partial}{\\partial \\beta_j}J(\\beta) = \\frac{1}{m}\\Sigma_{i=1}^m \\left(p(x)-y^i \\right)x_j^i$$\n",
585 | "\n"
586 | ]
587 | }
588 | ],
589 | "metadata": {
590 | "kernelspec": {
591 | "display_name": "Python 3.8.5 ('base')",
592 | "language": "python",
593 | "name": "python3"
594 | },
595 | "language_info": {
596 | "codemirror_mode": {
597 | "name": "ipython",
598 | "version": 3
599 | },
600 | "file_extension": ".py",
601 | "mimetype": "text/x-python",
602 | "name": "python",
603 | "nbconvert_exporter": "python",
604 | "pygments_lexer": "ipython3",
605 | "version": "3.8.5"
606 | },
607 | "vscode": {
608 | "interpreter": {
609 | "hash": "ac37b77c3c0f43e60bec193f0626743b91dd65d8d4aeca5713e457ab7e777e4c"
610 | }
611 | }
612 | },
613 | "nbformat": 4,
614 | "nbformat_minor": 4
615 | }
616 |
--------------------------------------------------------------------------------
/LinearRegression.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "attachments": {},
5 | "cell_type": "markdown",
6 | "metadata": {},
7 | "source": [
8 | "
"
24 | ]
25 | },
26 | {
27 | "attachments": {},
28 | "cell_type": "markdown",
29 | "metadata": {},
30 | "source": [
31 | "# Linear Regression"
32 | ]
33 | },
34 | {
35 | "attachments": {},
36 | "cell_type": "markdown",
37 | "metadata": {},
38 | "source": [
39 | "By the end of this lesson, you should be able to:\n",
40 | "- Write **objective** function of linear regression\n",
41 | "- Implement **Gradient Descent algorithm** for optimisation\n",
42 | "- Train **linear regression model** using gradient descent\n",
43 | "- Evaluate linear regression model using `r^2` and mean-squared-error\n",
44 | "- Plot **cost** function over iteration time\n",
45 | "- Plot **linear** regression\n",
46 | "\n",
47 | "Important words:\n",
48 | "- linear regression\n",
49 | "- hypothesis\n",
50 | "- coefficients\n",
51 | "- update function\n",
52 | "- gradient descent\n",
53 | "- matrix form\n",
54 | "- metrics\n",
55 | "- mean squared error\n",
56 | "- coefficient of determination\n"
57 | ]
58 | },
59 | {
60 | "attachments": {},
61 | "cell_type": "markdown",
62 | "metadata": {},
63 | "source": [
64 | "## Introduction\n",
65 | "\n",
66 | "Linear regression is a machine learning algorithm dealing with a continuous data and is considered a supervised machine learning algorithm. Linear regression is a useful tool for predicting a quantitative response. Though it may look just like another statistical methods, linear reguression is a good jumping point for newer approaches in machine learning. \n",
67 | "\n",
68 | "\n",
69 | "In linear regression we are interested if we can model the relationship between two variables. For example, in a HDB Resale Price dataset, we may be interested to ask if we can predict the resale price if we know the floor size. In the simplest case, we have an independent variable $x$ and a dependent variable $y$ in our data collection. The linear regression algorithm will try to model the relationship between these two variables as a straight line equation.\n",
70 | "\n",
71 | "$$ y = m x + c$$\n",
72 | "\n",
73 | "In this sense, the model consists of the two coefficients $m$ and $c$. Once we know these two coefficients, we will be able to predict the value of $y$ for any $x$."
74 | ]
75 | },
76 | {
77 | "attachments": {},
78 | "cell_type": "markdown",
79 | "metadata": {},
80 | "source": [
81 | "## Hypothesis\n",
82 | "\n",
83 | "We can make our straight line equation as our hypothesis. This simply means we make a hypothesis that the relationship between the independent variable and the dependent variable is a straight line. To generalize it, we will write down our hypothesis as follows.\n",
84 | "\n",
85 | "$$y = \\beta_0 + \\beta_1 x$$\n",
86 | "\n",
87 | "where we can see that $\\beta_0$ is the constant $c$ and $\\beta_1$ is the gradient $m$. The purpose of our learning algorithm is to find an estimate for $\\beta_0$ and $\\beta_1$ given the values of $x$ and $y$. Let's see what this means on our actual data. Recall that we have previously work with HDB Resale price dataset. We will continue to use this as our example. In the codes below, we read the dataset, and choose resale price from TAMPINES and plot the relationship between the resale price and the floor area.\n"
88 | ]
89 | },
90 | {
91 | "cell_type": "code",
92 | "execution_count": 1,
93 | "metadata": {},
94 | "outputs": [],
95 | "source": [
96 | "import pandas as pd\n",
97 | "import matplotlib.pyplot as plt\n",
98 | "import seaborn as sns\n",
99 | "import numpy as np"
100 | ]
101 | },
102 | {
103 | "cell_type": "code",
104 | "execution_count": 2,
105 | "metadata": {},
106 | "outputs": [
107 | {
108 | "data": {
109 | "text/plain": [
110 | ""
111 | ]
112 | },
113 | "execution_count": 2,
114 | "metadata": {},
115 | "output_type": "execute_result"
116 | },
117 | {
118 | "data": {
119 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAESCAYAAAAfXrn0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAACUx0lEQVR4nOydeXhU5dn/P89smZnse0IggZCEJYEABtS+QBXUYsvigmuLVbG8tiJUq7ValRewti7VgkvRimtbxUpdf9aqaEUqLqCybyGQkLBk32bJbOf3x8w5zMmcAYIJJHA+1+UlmTlzznPOJM/9PPfyvYUkSejo6OjonN4YTvYAdHR0dHROProx0NHR0dHRjYGOjo6Ojm4MdHR0dHTQjYGOjo6ODrox0NHR0dGhDxsDIcSzQohaIcTmYzz+ciHEViHEFiHE33t6fDo6Ojp9CdFX6wyEEBOBduBFSZJKjnJsIfAqMEmSpCYhRIYkSbUnYpw6Ojo6fYE+uzOQJGk10Bj+mhBisBDiPSHEeiHEp0KIoaG3fgY8IUlSU+izuiHQ0dHRCaPPGoMoPA3cLEnSGcBtwJOh14uAIiHEf4UQnwshppy0Eero6Oj0QkwnewDdhRAiDvge8A8hhPxyTOj/JqAQOAfoD3wqhCiRJKn5BA9TR0dHp1dyyhgDgrucZkmSRmm8Vw18LkmSF9gjhNhB0Dh8dQLHp6Ojo9NrOWXcRJIktRKc6C8DEEFKQ2+/AZwbej2NoNuo4mSMU0dHR6c30meNgRDiZWAtMEQIUS2EmA38GJgthNgAbAFmhA7/N9AghNgKfAzcLklSw8kYt46Ojk5vpM+mluro6OjodB99dmego6Ojo9N99MkAclpamjRw4MCTPQwdHR2dPsX69evrJUlK13qvTxqDgQMHsm7dupM9DB0dHZ0+hRCiMtp7uptIR0dHR0c3Bjo6Ojo6PWwMjqYsGqoFWCqEKBdCbBRCjOnJ8ejo6OjoaNPTO4PngSPpAF1IsBK4EJgD/LmHx6Ojo6Ojo0GPGgMtZdFOzCAoQS1JkvQ5kCSEyO7JMeno6OjoRHKys4lygH1hP1eHXjtwcoajo6Ojc/IJBCT2Njg41OomM8HKwNRYDAZx9A9+B062MdC6O82SaCHEHIKuJHJzc3tyTDo6OjonjUBA4r0tB7n11W9xewNYzQYeuXwUU4qzetQgnOxsompgQNjP/YH9WgdKkvS0JEllkiSVpadr1kzo6Ojo9Hn2NjgUQwDg9ga49dVv2dvg6NHrnmxj8BZwTSir6CygRZIk3UWko6Nz2nKo1a0YAhm3N0Btm7tHr9ujbqKQsug5QJoQohpYAJgBJElaBrwL/BAoB5zAdT05Hh0dHZ3eTmaCFavZoDIIVrOBjHhrj163R42BJElXHeV9CbipJ8ego6Oj05cYmBrLI5ePiogZDEyN7dHrnuwAso6Ojo5OGAaDYEpxFkPnTaC2zU1G/OmRTaSjo6Oj0wmDQZCfHkd+etyJu+YJu5KOjo6OTq9FNwY6Ojo6Orox0NHR0dHRjYGOjo6ODrox0NHR0dFBNwY6Ojo6OujGQEdHR0cH3Rjo6Ojo6KAbAx0dHR0ddGOgo6Ojo4NuDHR0dHR00I2Bjo6Ojg66MdDR0dHRQTcGOjo6OjroxkBHR0dHB90Y6Ojo6OhwAoyBEGKKEGKHEKJcCPEbjfeThRCvCyE2CiG+FEKU9PSYdHR0dHTU9KgxEEIYgSeAC4HhwFVCiOGdDrsL+FaSpJHANcCSnhyTjs6JIBCQqKhrZ+3ueirq2gkEpJM9JB2dI9LTbS/HAeWSJFUACCFeAWYAW8OOGQ78HkCSpO1CiIFCiExJkg718Nh0dHqEQEDivS0HIxqaTynO6vE+tjo6x0tPu4lygH1hP1eHXgtnA3AJgBBiHJAH9O98IiHEHCHEOiHEurq6uh4aro7Od2dvg0MxBABub4BbX/2WvQ2OkzwyHRl95xZJT+8MtJZBnZ/6H4AlQohvgU3AN4Av4kOS9DTwNEBZWZn+zen0Wg61uhVDIOP2BjjU6j6hDc51tOnJnVsgILG3wcGhVjeZCVYGpsb2md1gTxuDamBA2M/9gf3hB0iS1ApcByCEEMCe0H86On0Su8WE1WxQGQSr2YDdYjyJo9KRibZzGzpvwncy1n3dPdjTbqKvgEIhxCAhhAW4Engr/AAhRFLoPYAbgNUhA6Gj0yfx+P3Mm1SI1Rz887KaDcybVIjXHzjKJ3VOBNF2brVt7u903r7uHuzRnYEkST4hxFzg34AReFaSpC1CiBtD7y8DhgEvCiH8BAPLs3tyTDo6PU1qbAwr1lUxe3w+QoAkwYp1VUwpyTrZQ9MBMhOsmju3jHjrdzrvkYxMX3AP9rSbCEmS3gXe7fTasrB/rwUKe3ocOjonioGpsdwxZViEu2BgauzJHpoOwe/nkctHdfv301NG5kQhJKnvxWLLysqkdevWnexh6OhERQ4k1ra5yYjvW4HE04Ge+H66M2bQU4FoIcR6SZLKNN/TjYGOTs/SlzNMdLpGdxiZngxEH8kY9LibSEfndKavZ5j0FbQMLnDCjbDBIMhPj/tOMYK9DQ4eeG+bEnMCeOC9bQzNiu/R2INuDHR0epCeSmPUOUw0gxsbY2B9ZTMBCYwCRvRPZNKQzF5vhBscHVxRlsvSj3Yp9zNvUiGNjo4e/Z3RVUt1dHqQnkpj1DmMlsF94L1tVDe5eXp1BY9/VM5TqyvYdaidqsbINM/eVo1sMRoUQwDB+1n60S7Mxp6drvWdgY5OD9LXM0z6AloGd+rIHBa/s1U1oS5ZtYsxuckEpODq22I04Ojw4w9I3P3mJiobXL3Cjdfu9mkuIBzuCGGGbkXfGejo9ADyarPB0cEDl45UFaDpaabdi2xwwzEa0JxQGx0ernv+S77a08QVT3/O1c98wc9eWscVZblkJ1p7RaGYxWSIuB+r2YDZpO8MdHT6FJ192HmpNp6eVYbZKLBbjHj8AfY2OPSsom5Cq25geHaC5o5sb4ODqSNzNN0ws8fn88TH5UctFOvp7LAml5dbzivi0Q93Kvdzy3lFNLu83XYNLXRjoKPTzXT2YVc2uJjz0jqenlXGFU9/rmcVdTMGg2BKcRap143j0/J6JAme/Lg8YkJdPKOEl9buZeKQDM1dg5y5cyQ33onIDsuMi+Fgs4s5E/MJSGAQYDMbSI+L6ZbzR0M3Bjo63cyhVjdnD0rh2vGDaHJ4SYk189yaPayrbNSzirqBaGmkVrMBg4AAUNfu4fnP9jJnYj45iTaqmlw8/vEuZozKISChuWuQpEg3XudrSRI9nh1mjzFx/7+2R4zv7bnju+X80dCNgY5ON9MvycqUEdn870vrldXjwunFWDuJlvYl3ZregtbK/PGrR9PhlfjVP75VuVX+/mUlVpORP63axYGWYPbW4PQ4/t/GahZOL2bBW1uU4++7aAR5qTYuHZOjuH20rvXwzNIe1x+qb+/QvEaDo4NC4rvlGlroAWQdnW6mtrVDmWgg+Ie84K0tpMTZmDupgOzEoAuiszuit6U49ka00kh3HWpXDIH82qMf7uR3F43gvc0HFENgNRvY3+zix2cN4sn/lDN7fD5zJxUwe3w+j320E4vRQH56nOLu0bxWbZtmcLc7s8My4iMD4lazgfS4ns1A042Bjk43EghIHGzVXtl9sbeRZz6tYNZZeeSl2iLcEe9tOcgPl37KVX/5gh8u/ZT3thzUDUII2VDuPNQW8WxT7BbN593k9DLzjP5kJwYn1/svHsG/Nh3A75e4dMwAhmbFExdjRAjw+CQOtqhrPw61ukm2W7jp3ALmTgr+9/H2Wu6/eESPZocZDTB/sloCff7kQnq4zEB3E+nodBfyhJ4WZ4nqk5YzV1bMOYsROUlHXIXe+uq35HQ67nQk3F1zw4T8iGcba9VuJhRrMdLkgD9dUUp6vBWjAQ61ZvOLv3+tqux9e0MN15ydR06yTXXdeKuJa87OY8mqw5XA8ycXMiw7jnfnTThm/aGuZh8daHHz4tpKlQT6i2srGZ2bxMA0vQJZR6fXs6c+OKH/46sqFk0vjmhu88+vq4HgRO/y+lUTQrRK5VXba0/7HUK4oVy5vjqicVBarEVzJV1e24bD41cqdw+2dCgTOxw2zFNH5rBk1S6sJnVQx+nxRxy/ZNUu2t1+8tPjOCs/TeVW0uJ4dnyZCVaanB6e+Licxz8q54mPy2lyenq8UFHfGejodBOVjQ6S7Rayk2MBiReuG0ez08OWA6289HklADedW4DRAGajAZ8vgClUSBStUtkf4LTPOpLdNZeM6Y8QYDDA3HMLKMyIozAzHqMBdh4yMmdiPiaDgfy0WAJI7Gt08uq6fQzvl8AnO+swGgyaBleI4P/L69oxGISycm/v8KmuC7ByfTXtHcdeCXw82lQ91W/haOjGQEenm0i0miPcCrddMIRBabFYTEIlPvb06gruu6iEi0pzMJkMmhPAvEmFvPR5JW5vgEOtp2/WUXailZ9/P596h4eABC6Pn9RYC0Oy4hmYFsfa3fX8+ZMKHpw5gqpGF7e9tkH1DJsdHp5aXcGjl486YkppdZOTX674VqkbGJQaq+km6sqkfDzdz+S6iaFdcEV1Bz1uDIQQU4AlBNtePiNJ0h86vZ8I/BXIDY3nYUmSnuvpcenodDdmo4hwKzz8/g7mTy7kVxcM5dehSUp+7+43NpOdaCU70cbA1FimFGeRMftMVu+qwx+Alz6v5ECLG6vZgN1iPNKlT2kCAQmHx8/TqytUk7LsapHdKs6OQIQe0dKPdrHsJ2fw0MxSXv+minumDmfxO1s5e1AK140fRLPLS7LdwpBMOzFms2rlLklouokuGH7s7UuPV5uqO6Swu0qPxgyEEEbgCeBCYDhwlRBieKfDbgK2SpJUCpwD/FEIYenJceno9AQOj19bYMzj18yCcXsDrK1oVPzIABISVpOR5WsqFEMwb1IhHr/6s6cTB1rdmpPygdZg9o+8q3J5tQXe1lU2cftrGzh3SDZmI9w3YzgXjujHnJfWM+/lb/nZi+tweSXMxsOfqW1zU9umvaqvaz92xVl5bH1Bm6qndwbjgHJJkioAhBCvADMINr6XkYB4IYQA4oBGoGfl+XR0egA5P1wrq8Xh8R8xw0hejVqMRlasq1JlkqxYV8X/FKSejFvqFbRFUfFsd/uBw26Vz/fUH/EZL3xnC0/POgOLyci1z30ZUQfy/HXjlM/IK/fvqjh7slw+x0NPZxPlAPvCfq4OvRbO48AwYD+wCZgvSVLEMkgIMUcIsU4Isa6urq6nxqujc9yE54dnJ1qZN7mA+y4qYWT/RD7fXcct5xWpVoj/N62YT3fWAodXox6/nyvKclm+JqjDv3xNBVeU5eI9jXcG/ZJsmkVY2YmHtXoMBoHZaIjIKuqcxdXs9FLXpl0HUtfmZv7kAh6/ejQDU2MZmBrLHy9Tr+r/eFnXV/Wyy+dYso9OJj29M9C66845VT8AvgUmAYOBD4QQn0qS1Kr6kCQ9DTwNwR7I3T9UHZ3vhpwfPn9yIRkJMVQ2ONnb4GRfo5NZZw/EajKoxMc8Pj8XjshmY02rUmHa3uHFZICHZpayp95Bhy/AinVVTCk5dj/1qcbwrAR+d/EIfvv6JiVm8LuLRzA8O1E5JhCQ8PgCeP0SD80sJSAF2F3nUOIuEJzME+1mbGaj5oo/wWbmqdUVPHL5KOWcASmg+s4CUoBAQOq1E/p3oaeNQTUwIOzn/gR3AOFcB/xBkiQJKBdC7AGGAl/28Nh0dLoVu8VEk9MDBCUpOgc8jQKWripXjreaDTw0s1TxI+9paGfu379RZcKs3FjDHVOG9Uof84nCYBAk2kyqSTnRZlImZC0NoVvOKyLVblG+D6vZwIKpxTy2aid3/nAIi6YXc2+YNtGi6cWs31Ovctm1ub3c/trGCKORm2KndEDySXkWPUlPG4OvgEIhxCCgBrgSuLrTMVXAZOBTIUQmMASo6OFx6eh0Ox6/n3mTChmQbFfSG+FwwPOhmaWq493eACaD4G+zzyQ1zsKUJZ9GZMJ0rlTurfSkxv/eBodiJGWsZgPvhnL1tXL5H/1wJ/MnF/LnH4+h0eHBbDLyzOrdbKxpxdEh8eG2Azw16wyanV6S7Gb+9vkefnzWIPh4T6gJTgcub4AbJuQDwfqCAy3BgPLBFjelAzSHelTCn5PdYsLj95MaG9Mr4gg9agwkSfIJIeYC/yaYWvqsJElbhBA3ht5fBiwGnhdCbCLoVrpDkqT6nhyXjk5PIAd/b7tgqKZP2uVR50VYzQZ217Vzy6vfsnhGCcl2i+LSUD7TqVK5N9LTGv8HW7SzeuTai2i5/LkpdkxGA3sanEBQ1hqgyenl/a31vL9VPc38aGR/APJSbdQ0u7lj5caImo8mp4eshOOrBNZ6TvMmFbJiXRV3TBl20ntb9HidgSRJ7wLvdnptWdi/9wMX9PQ4dHR6mmZnB1eU5VLT5NT0SSfbD2sWya6M5z/bi9sb4J43NzNnYn6EG6kv9Eo+nirbrhBjMpCXamPqyBylEvjtDTWKzER4Ln92opVLxvTHaIAku5m7Xj/c21ie0DMTYqJkCcWEmuCMYM5L6yJ2aXMm5pOTZCM5znxc96H1nOQOa72hylzXJtLR6SYMBgMr1lUhAQumqbWJFkwrBgHPXTuWJ64ezezx+Tz/2V5lJ+D2BihIj+sT+eidOVKVbXfg9Hr55eQiRbXTKOCXk4twe4NtIHOT7dx3UQl5qTZmnZXH8jUVLF1VzuwX1nH1uDylt/HSj3ZxWVl/TAah+f1YjIJ3503AbBSa9zMsK4FV2w5woLnjuO4j2nOS5TC663kdL7ochY5ON5FsN/PjM/N45IOdJNstzJmYT16KncwEKwdaXDS0e7j55W+4YUI+y9dURKxM0+NjuqSG2Vs43irbYyU+xsLmmraIgPygtKChrGpy8thHwZjMNc+q6wce/XAnc88t4OH3g+0vcxJt1LV5ePmLSh6cWYrL48NmMfHM6t38/JwCzhiYqoy/8/1sO9jKqAGpuL3HVwYV7TnJchgnexeo7wx0dLoJsxEyE2KYMzGfS8/oH3pN0Ohwk5VgY3+Lixsm5LN6R22E8ub8yYVK0xupjyVO93SVrdsX0KxA7vAFfz7U6qayIWhstVbecu9gq9lAW4eX1DgLLW4vOw62sa/Jxc5DbbS4vaTFWaLez7xJhfxjXTVLP9qFzXJ8a+ho531nY02v2AXqOwMdnW6its3LHSs3qSakvFQbcyYO5tcr16mChu9tPsDs8fnkptioaXYxvF88Ww+09Wij9Z6ip6tsnR7tCmSv309FXTsur5/5kwtIiTNHrR+QRQP7JVrZfqCVm84p5N63NivPeuH0YrwBdUVz/KwyvtjbiCShqldodHiO6z7Cn1Mwm8iI1x9gSklWr9gF6sZAR6ebcHsPaxONzEnghomDCQQk4mKMPH7VaBraPdhjTLzwWQUTijJYvqaC5T8tY+zAFCQJfvSYOrW0LzW36UlhtbyUWMryErnme/m4OnzYY0y8vWEfDQ4vN7z4qTKhD0yN5a4LhyrN5OUdV1ZiDM9dO5Y2t5eAFNSQWv7fPYrkB8CT/ynn4U6pvzaLMVhoFvaa1Wwg+zizieDkCNAdK7ox0NHpJmSfcFFGHFeNy1NUSuVJ6cW1wdTEBVOL8Qf8LJ5RoqiRRhNFW7W9lppmd5/YIfQUOQlWLi9TP88nrx6jdCyD4LO68/VNzD23QKXr9K9NB0iPj+HuNw7vAh6cOVIlJ35YDDC4MzhSCujNkwop7pd4pOH2WfSYgY5ON+H1BVg4bTi3/WAIC9/ZEuHjvmRMf0UwrSgzgRVfVXLxk2v54dJP8fklTf0dubnN3gbHybilXsHmg62KSweCz/Pb6mZN4+nxB1Qdws4ZmqEYAvkYA0IxBPJrSz/ahdkYNMzRUkAfnlnK9BH9lIZEpxqn5l3p6JwE/JIfg8GgVKqGEyyCsjF3UkGwuKzVTU0oRTFoLHawaEaJppBdb0g7PJlopWQGQhk44VjNBoZmJaie4YBke8RnK+odmt+PLF0RLQV0T72DT8rrTtkWpLqbSEen2zBwz5ubFb2hzoHMqkYXy9dUMH9yITazkV9dUITZIDAYDDg9Xp74eJfKxfHnT8qZdVYeO2vbSY/r/cVnPUV6fGSR2Nsbarj/4hHcFSZet2DqcJ5ds1v1DBvaOyI+6w8EokiNB6fDaEVs9hgTB5pd7K1vJz8jXnOsPSnL0dPoOwMdnW6itjUojfyX1btZMFVd1CRLKcsuo6pGB3e/sZl9TS5uf20D8TFmPL7DK04hwOOTyEywBUXuTuO/VIfHF/E8b5xYQGaChXfnTWDZT8Ywe3w+Xn+AycOyVPLfNrMxQjq8MDM+ouhs/uRCzKbgpN0/0caiGZFFbLf9YwMOj5+6du2iMznW8MOln3LVX75Qmhb1lZ2EvjPQ0ekmshODK8qNNa2kbgkKobW6vIDgL6t3q6qNnZ4As8fn0z/Zzg0T8rGYDZr9dtPizNz/biWjc5MYmNb7MlBOBHaLiZVf71IVib34WQW3/2CYkpXzyxXfcsOEfFaur2b2+HxiTAYKMuJ44L1teHwScybmMygtlmS7mfgYE43tHSoV1FiLURHX33aolSc+3qXZqnTJql385ZoyzXH2tCxHT6MbAx2dbmJYZhyLZ5Tw+Me7GDsolf99ab0qG6Wu3aO0sizMiOOWsGyVZ39apllY9dfZZ9Lk9JzWbqIkm5FZZw+kvLaNgBSUo5h19kCS7MGAr1zMteNgK5bQ6t7jD7Cnvp3Z/zOIRLuF3XXtPPjeDpqcHp6/tox4m5n6sHqBeJsZQ2j3daAlWMQWrVWpy+PXHOeRZDl0Y6Cjcxqxq96BEBIPXlrKr1duUOWxr1hXxSVj+rN8TQX3TB3OH97bppr4v67Szo5pcnpOezfRoJR4dh5SZ1MZhGBQStBvLxdz5abYGJBiV6WR/v7iERxqcSF7atzeAN/sa+GVr6oU4Tt/AP704U7+GKozyE60kZdqY0hmvGZsQZbB6ExPy3L0NLox0Dlh9OXg2rHg8vppdHhpbG/SzGPPT4/loZmlNDk7qGxwqT7r9kUJasaYeHHt6e0m2t/qorrJFaFNtL/VpTwTg0Hg9voj0kjvfH0Tt11QxDOfViiqpQ6Pn8oGF098XK66TpMzKHw3JD2Om84p5OH3tzNvUqHqe7z/4hEMiiIbIe9QOleRn2yZiWNFNwY6J4Se1rzvDfgDEo98sJOHZpZyeydf89KPgj7vX7+2gXunDicv1aYyCG9vqOGeqcNZ/M5WdSGUz0+T09NnVpfdTSAgUdPs4pWvqlQ7rVe+qmJMbrLKQDY4tLWJshJtKrloo4jS6D4hhoq6durbOnjiP7uYOjIHQ6gFaWWDg4GpsTQ6Oqhudmoa5p6W5ehpdGOgc0Lo68G1Y6G9w6fko2tNSntDry96Zyt/umIUv1xx2DD+cnIR/9pcowRJ7ZagbMWZg1L61Oryu9B555ibbOf9bYfw+f2aOy2v/7Dv3ucLkGAzaU7yaXEWbjq3ACFgaFY8zg4vt5xXxKMf7lTOd+v5Rbh9fq5Y+jkPRalQrm528qcPdzEkK+Gou7S+JjYIXTAGQgg78CsgV5KknwkhCoEhkiS902Oj0zll6OvBtWMhzhKcjDx+bZePrLLp9gbY2+AMZrikxlJe56BfsoVJQ7NVkgv/N60Yt8/3nXdPvdE9F23iD985PnDpSB75YAd3TBmqWTH8t9lnAkFD8MaGGgrT41gwrZiFb29RPcM2t0+RDLeaDfzu4hEU51h57tqx1DS7sJqMODu8bAjFbdLiYlS9j8N3dsFMMG0J676+++1KWOo5oAM4O/RzNXDf0T4khJgihNghhCgXQvxG4/3bhRDfhv7bLITwCyFSujAunT6AHFwLpy8F144Fu8XI/MmFvL2hJkKiWq4zkH/ul2Rj6apydtc7WL6mAiEZ+fMn5cwen8/cSQXcMCGfP39Sjtlg/M6GoLflvmuN6bOKhoid4x0rNzJ1ZA7ltdo7rYZQNtCWAy3c/cZm2jp8LAt7hrPHB5/hppoW1Xl/+/om2lwBrnv+K277x0Zue20DDU4veaHV/v5m7YXL3npHUAXVqt3pLNrut69IiXTFTTRYkqQrhBBXAUiS5BJCHPG3VAhhBJ4AzidoPL4SQrwlSdJW+RhJkh4CHgodPw24RZKkxi7eh04vp68H146FJpcHu9nIjFFBX/PDM0sRBhAIHnhvm5JWOm9SIfubg60xxw5MZnzBONo9Xk3XhMPj/U5j6o3uOa0xrats1JyAjQai7rTsFiMVde3sawr2iXB4fJqBYbvFqLiJINjcvs3tVeoSDrS4WbJqF49cFswmspgMmtfLT4vlzilDaXVrS1j39d1vV4yBRwhhI1SaIYQYTHCncCTGAeWSJFWEPvMKMAPYGuX4q4CXuzAmnT5CXw+uHQsWo5FnP9vD1JE5tHf42XawjfgYIy+HpTFKUjDNdMaoHBbPKKHZ4cFoNJAWa9F0hTx37djvNKYTNUF1xRUVTWtIq89xWV4K97y5SdPHbzTAD5celrB+7tqyiEk8L9VGos3Cnz487Dq6Z+pwYkwGjAJu+0ERdW0dvPBZJUIIrGZDsIK8k7tp3qRCHnp/Oz8+M4/0KLvZ0ym1dAHwHjBACPE34H+Aa4/ymRxgX9jP1cCZWgeGYhJTgLldGJNOL0drkugLq6TjwW4xcuXYXJas2kWy3cJlZf3JSLCycHoJC97arDRmv++iErITrWypaaG1w49RQExOouak3d5xfC0WZU7EBNVVX7nWmD7fXcfNkwpVNQL3XVTCWQNTePLqMeyud6gqhjMTYvjj+zuUcyTbLcSYjBEZWb+ZMkwp7oPgM138zlZmj89Xaj7y0+P4+ffzyUmy8u68CVQ2ODAaYPlPy/hiTyP+wOHmNo98sJNJQ8ZrPoe+vvs9ZmMgSdIHQoivgbMAAcyXJKn+KB/TWhpEc1ZOA/4bzUUkhJgDzAHIzc09tkHrnFT6ekCtq6TEWkiymbj1vEJirWbVpHTfRSW0OL00Or0MTo/l84pGlfTE89eN1VwZx1uP/id6pFX5iZiguuqKkhvYh0/8888bws9eXKc6x91vbGZU/ySsZhO/DgvowmE9oTPz04m3Gom3mrn6mS+U3tO5yXYOtrrZVduuaWTlJvSL39nKnIn52MxGXD4/I9OTqWvr4KfPfckvzilg6aryiM/WtXdQkBkpVNfXd79dySa6GPhIkqT/F/o5SQhxkSRJbxzhY9XAgLCf+wP7oxx7JUdwEUmS9DTwNEBZWVkfTNw6/dhTrz1JDLl5AoMzTr3dQf8kOylxMeQk25kTkqKAwxPbgzNLeej9HZw1KCVCeqLF5eXG7xeoXBMLphUTCASOdMmjGtwTMUF11RVV1eTklS9DDelDnctaXV7Nc+ypd2A0ioj3ku0WEm1mlqwK1g786cPg8zzQ4mbpqnKsZgOzx+djMmjXFEhhFckBCZas2sUzIc0hR6jN5qC02Kiximj05k5mR6NLbiJJkl6Xf5AkqVkIsQB44wif+QooFEIMAmoITvhXdz5ICJEIfB/4SRfGo9PLqWx0kGy3cMmY/qrgXVWjo8eMwclMo9zX5KSizoErrP2ljNsbQAAr5pylma1iMxtZ+LbacC58ewsvXj/uiNfc2+Dggfe2qQqyHnhvG0Oz4pUJSWuC6s7n1FVXVIOjg/OHq9Nol/3kDM1zWMwGtu1vjXjvsrL+LArtvORVfjhubwCb2cCw7PiIXYhciSxfQ5KCx7u9wbqFvJRY8lKD4dHOFcjBjmhHNtA9RU//bnfFGGiloR7x85Ik+YQQc4F/A0bgWUmStgghbgy9vyx06MXA+5Ik9Y0cLJ1jItFqVilx5qXauHfqcDp8AdbvbWREv0QsR1hldZWT7Zba3+Jiyapd3DAhP8rkGMMVT3/OY1eOjni/tq1D+VnW0A9Ocn4CASnq+BscHZpZSI2Ojqir0+5+Tl11RcUYDUowGIIT8c6DrcyfXBih2trhDZASa2bh9GIWvHV41zQ4PS7i+Xb+uTgngW+qmrFbjDw8sxQJiQSrmXvf2qxkdi2cXsx7mw4EU0ZtwZTRvBQ7N08qRAjBinVVqv4IK9ZV8T+DU7v8jL4rJ+J3W0jHWConhHgWaCaYKioBNwPJkiRd2y0j6QJlZWXSunXrTvRldbrIxn1NXP7057i9wSYhs87KU01ai2aUcNHIft1mECrq2rnu+S8j/O7PXTvuhGzb39t8gBv/+rXmvS6cXozdIth+0EmK3UxGQgy7atsVFc7xhWnMWv4lyXZLxGf/eNkoLizR/qPfsK+JK0LPWMZqNrBizlmUDkjWHGdFXbuShRP+mXe/Q7qpzxdgy4EWDrS4yU60UpydGLU95EfbDnH9C+q/3+xEK3f+cCjloWdiEFCQHsszn+7hVz8Ywr1vbmbqyBxFmrqqwcGfQoZD63kvmFaMPxDgvv+3TZWBlGQzYbWY8fj8pMfHUFHbzuCMOPbUORicEcv4wgx217bzo8c+5ZfnFRIIEGFoR/ZPYHxhxnE9p+Olu74zIcR6SZI0Nbi7sjO4GbgHWEEwMPw+cFMXPq9zmtHuPuwuuWRM/4jUyXvf3Ex+WixlA7unxvB4VsndSb8kG1ZzsO3lS59XBnVwDDC+IA2jEFQ1OTEZ4M1va5ha2k8lvDZpaAYLpxdT0+yKeE6/+se3FGWOp1AjaOn0+DVdcc4oMsvQ/emmgYAUUT18pFVrbEykbEST00N2opWsBCsHWtwk2c0YhMTO2naaHF6lfuCmcwu49dVvSbZbFBfOgRY3K9ZVsewnY2ho91DZ6CQj3sJNf/9G9Rz/9kUlN08qVLmn5k0q5J43N/PjM/PITrRRUdfOjkOtoUwuP29vqInYGZyRV9rlZ/RdOREpwl3JJnIAERXEOjrRiDEfLt6J5tc91Np9vX0tRoNmrv6KOWd12zWOxPCsBMU/faDFzfI1FTw0cyR7G5zc++Zm1UTZObDe3uHjyf+U84vvF0QNpGoZg+xEq2ZTnOzE6Kmj3Z1u2tVsosyEGE2XUKvLq0zgQfnpEm49P9ilrPPvUbjBFQLGDEiiNmzC9PqliOc4dWROhKqpLF73yAc7GTUgiWue/VJx861cXx2x45g/uZDMhJjjek7fhRORInxUYyCE+JMkSb8UQryNRlqoJEnTu200OqcUTS4Pd04ZSoPTQ06ijfmTC3h1XbXS8ctqNpCZ0H2/zE6PduD2SKvk7sRkMjB9RD9yk+0caHWTHh+DJEnMfkGdMrn9YGvEOBtDq999zS7NP/rOUh4y/gCaTXEuGJ4VdZzdnW7a1VVrbkoshZlxqrqBrEQrf/5PuSoQ/qdVu7jvomJiTCb+cMlI9tS3Y7MYNVNwDQIGp8fx188ruGHiYCSJiN83o0F7QSIbmEOtHdwwIZ/VO2qV31tfQOLhy0qRJIlkuwW/FCA35cTXDZyIFOFj2Rm8FPr/w912VZ3TglS7hYo6R4QO/YtrK5WmLQGp+zIzoq2eutPgHIlAQOLDHbURYmudJ6CAFBnwTLabldVo5wyW+ZMLo6Yz1rZpT8R17e6oGVsGg+CCYZmsmHNWyMdvozg74YRlE2lhNgrOH56tqjK+c8pQ6tu83Pn618pr911Uwq/OH8KvV248HHuaXgxCosPrY2ppP5UbKPz3bVh2QtQ0U6vZwPaDbSxfU8GdU4biDUg8vbpCKR7MT4tjV20bA1Lsx/WMOtPVzKATkSJ8VGMgSdL6kMbQzyRJ0lM/dY4ZCXjkg50Rq9aHZpay/WAbL66tjNo16niQG5mHu2QWzSihf6Kt265xJLTcJUaDiJiAtHoXePx+FkwtZuE7W3jp80rmTMwnPy2OuBgjTo+PjHiL5jWPZyLuqo//aHR11bq3wcHcMH8+wLzJBcqiAUJCdE4Pv39vu+q1u9/YzJyJ+erY01tbeHhmKXFWE/e+tV7z921XbRsCidsuGMLDocplOWawYl2Vkm4qX1c2BFpuooL0uO/UaOh4M4N6uobhmGIGkiT5hRDpQgiLJEnaKk06Op0IT5eUcXsD7DjUxhMfBwuDUmO1J7njQW5kHh7we+LjXQzJjIuaWfNd6Ly6a3B0RAZz1+1j8YwS7gkzUHPPLaSkXzzvzB1PvaODjHgrjY4Onvy4PKLp+60XDOXVdZXkJNnJ1choPB73wdF8/OH3lRFvxWgI9gWOtoLt6qo1mjbRsbzm9gaUFpbhr/klifp27d83gDPyknF1+AgEAsw9t4CABAPTYjnY7GTRjBJ+s3ITADedW0BOoo0bJuQTazFquuBKchKDnz/OlXm05z/k5gkIwUmTGu9KNtFe4L9CiLcApR5AkqRHuntQOqcGWYm2I27LF88owRvoPn++3Mi8s2rlgRY3pQOifOg40VrdPXjpSG46d7AqnfHuHw3DYhLMn1xIelwMVU1OHvlgJ01Oj2o1uK/REVGINW9SIXf+cyNXj8ujyaW9Bjse90E0H/+hVjcDU2Mj7ivc1RJtBSuvWgemxrK3wcEXexo0J7RAQNLMJtLqPhatI1nnW7OaDRiFIDEuRvP4OKuJ/w1VhMspps/9dy9NTg8Lpxfz2KqdABG7gHumDifZblFiDvJz+mJPI898WnHcu6loz3/bwVZu+8cGlVE/kdItXelnsB94J/SZ+LD/dHQ0MQpYMK1YCX7KBqAoI9gLeMVXlZhE9xWdpYcmg3CsZgNpcd2f/aG1uvv1yo2q3VAwiNrBHSs30d7h5+43N7N0VTkHWtzKalDWuk+wmrGZDfxlVhnzJge1+F/6vJLKBhePfriTxCga+nB4Ij4rP4389LiIySMQkKioa2ft7noq6tqJt5o0n5PdYtS8ryWrdnHJmP4RY+5MICDx0Y5DvPFtDf/d3cCb39bw0Y5DSu8E2YB+vrue+ZPV/R5SYy08cOlI1WtFmfFKNpH82uIZJRT3S2De5ALmTipg/uQCfn/xCFau34fVZOCuC4eq3rvrwqHsCgvYu70BHvlgJ3+8bCRPzzqDL3bXM3lYFpeVRaY+L35nK5eV9Y94TnLF8vH2KojW22PnobaI3cKJ7IXQldTShQBCiITgj1Jbj41K55SgstHJy18ENWgMAgwiqOsvq3fOm1RI+3fU6w/HbBQR0sMLphVjMXb/yira6q6zC0N2dURLrZUzblxeP/f/azuLZ5RoiqPJjVy0OFIwUmsHc//FI1g0bTjVLW6l6C3FbsHrD0S9L9ntFT7mztcVwK5D7REJA4PT4pAIGtAdB1uxWYy8uDaYFhpjMjAwLZb9zU6Kk228cN1Y9jW6qGpy4ujwEWM0qHZVL63dy+Vj1ds8j8/POUMyKK9tw2Iyqq6/eEYJH2+vjehn0OzycuurG1g8o4Rh2fFUN7k07ztcn6izlMWRMqaOhJZr7/6LR/DQv3dEXP9E9kLoilBdGcFuZ/Ghn1uA6yVJWt9DY9Pp42QnWGlxe9lxsI2zBqVw95ubVCmBK9ZV8fDM7ivgOdDiVoyP7Hd/ZvVu5p5byMhudhNFC9x23tHLrg75/WiB3la3LzQxGCnLS+Sa7+UrIm4vfFZBXIz2n+rRgpHaK/2d/OKcAtWkefsPhpAWF6OZ6SSvhsPHLO8CNla3KAZl7MBIAb4lq3YxIieRL/c2Kselx8eQkxSDLZTptbu2jVfXVfOnD3dx/8UjeOWrSs7MTyfBZuaJ/5TzqwuGKu6zO6YMoc3tizA4RgHZSXZuDx0HQTG7+vYOpo/OYW+9g1fXVStZbPFWE25vgHve3Mzz140lJdaied81TS6leLAoI57fvbtNlRp9PHn+Wq49gwgW3oXT3XUER6MrMYNngV9IkvQpgBBiPEHjMLInBqbT9ynOSmDuucEKz3irsUc6ecFhKYSABOcOzeD3nf5grZaueEOPDa3V3e8vHkGb26taSabGWlgwrZhln5RHpIyGB3rjQ350l9fL5WW5qtjBounFpMR2rdWiHAzWWulfdsYARedH/sxD/97B9wvTyU+Pi7ive6YOp83tZf7kAkb0T2RgaixVjY6IXcCgtLiIa7m9ARodHtVxd104lCvH5WmKx931+iaeuHo0G6pbaO/w8ZsLh7Gvwamct3+ynYff366qR3jlqypuu2Aoe+oPt8fMTrRy48R8Gpwedtc5MAq4cWI+y1ZXsGRVsGnQY1eN5i+rd9PQ7sFmMWim9Mqxkj9eNoqAFFAmbDnNNTf5+FJNO2cGBQLSSe+F0BVj0CYbAgBJktYIIXRXkU5UalrdShZNQXocc19WywMs/WjXUVU5j4bcDF2ljR/2R3zr+UVRe9Z+F7RWd0YDfLKjTlVMFWs1MTw7nmd/Oo4mZwcr5pyF0+OPcOfEmA0smj6c3JRYrn3uq4jUSbn5e2eOVvCVER+5g0mPj9H8zL4mJwWZ8cp9HWp14/VL3PPmJsW198jlo0LX7YjYBeypb4+yWxKq4+odHp7+1/aI34XZ4/P559fVNDq9KuPx3LVjlfMGAgHNRYUkBVTtMa85Ow+n1x+xg7jm7DweeG8HW/e38tD7O1gwrZicJCsf76jjv+V1iqx2vM1EY3sHC6cXU5ARh0HAT5/7UpWp9thHuxiTm9wtbpze0AuhK8bgSyHEUwR7DkjAFcB/hBBjACRJ+roHxqfTh6lsPLxS8wUi5QHc3gAtzu+2M5CboXd2Tci55ZkJMSRHWVV/V7RWd/2S26kP8+8n2swMyUwM/VFHnzRiLUZAcLBFOz3yUJu2bMfR6gyMBrjrwqHUOzyKmyY7Ufszdot6OnB5/VQ2OLjrh8Npdniwx5h4ds1uhmbFK5r/4by6rjqifmLepMIIyZFoKaNCBKWp7+n0fbZ1+JRVu9mkLTmy/KdlvP3BTuW4/slql1H474XVbKAoKx63NygT/vx1Y/l4ey1XjcvrtCMrYeLgNKxWE2t312tmqnWnT/9k90LoijEYFfr/gk6vf4+gcZjUHQPSOXWItRxOIbSZjZoTkCWKzMKxImfmhOP2BnB6fASkYNHbg5eWkpfa839gBoNg0pBM8tPiury6a3H7eOI/5fz+4hGazyklSj3GwNRYHr96tMp3P6J/IrnJdirq2tnb0I4Q6uu3u73cen6RUhAor5ozE2I0YxDhO60FU4tpdXnIS4ls/NLk9OBweyOE3W6/YKhyTHailaFZ8VHjLYUZka4mq8mgSElHkxxpdXt5+LJS6ts8PDyzFIMhsiGO2xvA5fExf3IhchDE7Q1Q19bBhKIMFr6jdp3d+9ZmygYmk2+N0zS6eak2bGYja3fXn5S6gO7mmP8SJUk69wj/TRJC/LQnB6rT98iIj1FSCOVc+/Bg6vzJhcRHCYweK7JSaDhWs4G6tg6e+bSCK8py8fpPjDYRHD3NMxptbh9XlOVSUdcekY67YFoxNlP0FFyPLyid8PhH5Ty1ugKA97cd4odLP8Xrh/YOn+r9ykYXJTkJzJmYz9xJBcE2kSGZBa3udEtW7eKuHw7jhgn5LFtdji8Ag9KCMZPwcf7u4hFkJcQwLCueAUk2hmXHc9P3B2MyBquwZVG95/9bwYKp6ntcOL2YjDiLsoAIJy7GyI3fL2D5mgr2Njg1v++MOCtOj59mZwcpcRbS4yyax8Vbzby4tpIWt095LT0+5ojZXnA4RiSfMy/Vxs2TCrni6c+56i9f8MOln/LeloNKGm1f5Lv9JaqZD7zQjefT6eMIEXR/zJmYj18KkJNkU/nTc5KspMUdf51BICAhSZIiYiZniyyYVozX51dcCK/87MSoln4XEm1mln60i2S7hVsmF/DwzFIcHh+xFhNOj5dE+7EHkDdWt/Dmt0HpZbNRaGb4LLliNDazkZwkG+V17dz3/7bR5AyuqrUmxR2H2njm0wrmTSqk1e3R9HG3uTzsbXSys7ZN2aXkp8eSFhfsS1yWl8ycl9Yze3zQqMwen0+81Ui/JDtVDQ7yUoMppp13Lf4ArFxfxYMzS7EYRERF9/zJhbh9fv73pfWKltCY3CTuv3gEd72+STnutguGsOidrTQ5PdjMBsUI2cyGqAVusrut8/3azEZVH4nOgfu+SHcag767P9LpEQ60uHltfTU3TBxMk8OPPxBgYmE6B1pcpMdb+femGqxmE7mpiV0+t5Y7Y9H0YhLtFv6yupzZ4wcDh90AvR1ZSuFAi5tHV5UrkhbjBiZjtRiiKmVqBZDtlsOZW0Lka07ufimo1y8LvslUNzmjppbKxvWvoWB2Zx/3t5WN7G92RwRtc5PtXDQqRxmrEFDZ4OKfXwclosP99AumFZNkNKgWDc0ur6o6Oy/VxiOXj6K8tp0OX4AX11ZyWVl/ku0Wrv3eQEXsLi/VxqOXj2JXbTu+QACzQWAxCW45rwhfAGaPz+fJ/5SzeEYJF43KIS81VmU8OmfzhN/v2t31Rwzc90W60xj03f2RTo8QbzVx4YhIiYWXPq9U2g6eOyy61PKR0FoR3/vWFuZMzOf84dmKe8ZqNhDzHeMSJ4L0MCmFAy1uRbtpYuE4xhekR3U3afmy+yfZuS0seKo1uafGxmA0tEVMaBKoVuZ5qTbumDKM8tp25k4qYOX6arxRegC3dfi1tXyuKWN0Xpxybfn/Wg2PFr4d/A7lwrvsRCvjBqao2mRWNri49dVvmT0+X3lO/gD8+MzciONu6XTcU7PO4N43N3PbBUOVYHCHL8Cg9DjyUmMZNSDpmOI9J6K/wImmO/9KNJ+aEGKKEGKHEKJcCKHZHEcIcY4Q4lshxBYhxCfdOCadk4jTEzk5LP0oKG0grxqPN+3zSBXAj364k1iLUblGoq1nsom6k+KseBZNL1FNlouml1CSFR91QgoEJAIBiQcuHcn8yQVKlpDE4cwtWRI7/Ly3nFeEw+NlTG5yhF/dKCDJauLhmaU8NesM5kwczK2vfssjH+zkmU8ruObsPDLiI+U9fL4AJqNg4bRiHrtqNCNzEoBQ4x63jw37mumfGFzRv72hhnmTCrF1mkzl402G4JjkWoFmp1fzuHirkXmTC/j9JSOIjzGSlWiNmqUk//tQi5srx+ZS3eRUnoe8+teK93SW8pBjAp1jCCejLqC76c6dwX87vxCSvn4COB+oBr4SQrwlSdLWsGOSgCeBKZIkVQkhTmxzUZ0eo70jMv3Q7Q1QlBHHw5eVYjYIhhznljojiiiZ7M5odgWzS5wd3j6xZ91V7+CJ/3RSXP3PLoZkxVE6IDKTSMtNtmBaMc1OD/XtHapdhiyJXZgRlBJ74bMKivsNwS9J/O7iEVQ1HK7OHTkgiW+qmrn37a3MPbeAxz8uj1jpj+3UplSr1mPB1GL4spKdte3srmvnlle/5b6LSkiPt3Dl2Fz6J9tIi/IdFob6MMi1Aqka/vy8VBtpcVbufuOwW+eJq8cctXo6aDD8/PmTCqXy2qwhVxIISOypd7DtQCu7QhXSnYX6TnZdQHdzzDsDIUSmEGK5EOJfoZ+HCyFmy+9LkjRX42PjgHJJkipC0tevADM6HXM18E9JkqpC56nt6k3o9E7k9MNwrGYDO2vbue0fGzjQ4mZ34/EJcbm8fiU7KTvRyrzJBSyeUYLdYiAv1ca2A23c9o8NNDl91Dt7JmYQbdV4PIQrrj7+UTlPfFxOZYOLgy3a9QVaWT8L395Cflocw/vFs3D64WydYMDUyP3vbuP21zZweVkuHaGA669e3cBTqyuYO6mAW88rxOMLKLu5aPUAtZ1qHrRqPRa+s4U5Ewdzy3lF/O2LKtzeYC+C9ZXNtHf4uf21jdS2uiN2LfMmFWI2BbOP+ifbWbJqF9VNzohMtN9MGaYYAvmai97Zwj1Th0dkrP3z62plp2U2Qv9kG3f/aBgPzSzlzW9qONi5DiJkaH/02KfMffkbnlpdwayz8ki2W1TiccebOdZb6crO4HmC8hO/Df28E1gBLD/CZ3KAfWE/VwOdSymLALMQ4j8EdY+WSJL0YucTCSHmAHMAcnNzuzBsnZOFnH4YvnoNbyLyh/e285dryo7r3FVNTl5cW8mKn53Fztp2VXbJounFfB4K8D364U6eu3ZsN9/Z8TcoiUZ2FLnvrCi9jMML+mTc3qAsR15qLP8I5eUPz45nx6FgIyFZouPet7aw5IrRqol08TtbWTHnLOrCVFfDRdrCx5Qaq3YTRav18Acknv9sr3Jd2QUUkAK4vQEMhsP1A+F1CXf8YKjymtsb4IXPKrlxYr4qqOwLBCKuWdngIsFqUo5LsZspzkmgf7Kd1DgLH2zeT327PbKncSc/v1Y8Sq6QfuLj8j4dJD4SXYkZpEmS9CoQAJAkyQccLYFb66+i8/LJBJwB/Aj4AXCPEKIo4kOS9LQkSWWSJJWlp6d3Ydg6Jwt5K/3uvAk8cfVoRZY5fHJwe4+vBiAtLoYmpwe3L6AYAvmc9761hRlj+is/t4dyyruTaJpAxys5XJydwH0XqWMG911UQnG2dqZVeD5+dqKVm84tYN7kAsryknn5i71MDgXmJQlFNlvG7Q3gDQTIDjM0bm8Al9dPos2snLem2am5cnd71c9TNmThWM0GjAahuq7VbCA/1NnOajZQ0+zkyrG5LF8TrIFYvqaCK8fmUtno5ImPy5UucQda3CxbXYE/EDQE4wenKUWMna9Z3eTCHwCL0cCAZDsbqpqpaXbS2O5hVF4qBhEUsJPvecmqXbS51VXwR1Ju7etB4iPRlZ2BQwiRSmgyF0KcBbQc5TPVQLheZH+CfRE6H1MvSZIjdI3VQCnBnYdOH0feSvsDEr/6x4aIVebxBtwyE2L4wyUjona3anV5mTupgLc31JAepWXkd+FQqzuyq9n66uNeNZpMBi4qzaEwI46DLW6yEq0UZydiMmmv1zITggV9r3xVFaHVs2BaMalxZjbXtGI0CH4zZQgS4PVLDEqL5UCzk4z4GB65vJR9TS5sZiMvfFZBamwM+5ud3HJeEY9+uJP2Dj87Dzbz1KwzaHZ4SYo187fP9zCiv9pAyYYsPGaweEYJQkjKzkI2JDEWgxJAfvnLKm76/uBgTUWHj1irCZAUKee6NrciSQ5BaY38tDhcXh8GIbjvohL2NToVf/7/TSumwxtMWpg/uZDyunbl+dwWJaNNjquEq9oeSZG2rweJj0RXjMGtwFvAYCHEf4F0YOZRPvMVUCiEGATUAFcSjBGE8ybwuBDCBFgIupEe7cK4dPoAZmOwAln2R8tbdK3g3bGQGWvFF5DITNCWHt5+sJ3laypYOL0YYw/4cuVq2s73k5Vw/KtGk8lA6YDkY+rKlpsSS2FmHA9eWspPn/tStUNZ9kk5c88tjMj3f/nLKpqcHuZNKuT21zbw4zPzVB2/Orx+KuodfLD1AA/OLCXeaiA3xa7qErZoejFZCcHsLLmfwcEWN1kh4+Tw+DGI4Pf95rfVPHr5KAwCNta0smJdFY9dOZrnrh1Hq8vDmfkp1DS5VAHaRdOLufX8Iu5YuYlkewwPv789mHVmM7P4na0k2y0Rz/2eqcMZlGZnc3ULf/sy6HYakhnPL/7+NbPH52tqGYWnm3Z2xUXrNzAmN4nclL4dJD4SXWlu87UQ4vvAEILunx2SJB1RZUySJJ8QYi7wb8AIPCtJ0hYhxI2h95dJkrRNCPEesJGgC+oZSZI2H+f96PRSDrS4lYYmsn/4xbWVjM5NOq7m4psPtnL3G5u568Jh3DllKA1Oj6pRy7LQRLjgrS08+9Pji0scCX8AzZz6C4YfX93E8VKnsTOaOjInwnW2ZNXhCVCeDB/5YCdzzy3g4fd3suCtLTw96wziYkxcc/YgymvbOCMvmXs1XHAvXT9OM2Yyb1IhK9dXKzUkD84s5ZZXv+XpWWewfE0F8ycX4vNLodaabZqxpHvf2sL8yYXMHp9Ph8+PxyeRk2xXalUuGdM/4rkvfmcrT149htT4GK4el8ejH+4kd1qx4to5kstHyxV3KmYKHQtHNQZCiEuivFUkhECSpH8e6fOSJL0LvNvptWWdfn4IeOhoY9Hpu2QmWGlyelSqj9/F/3ow5Nd1en2YjQbVKviW8w6HnIIxg+7XJqpt0/Yr17W7GZzR88HFvQ0O5v79G168fhx5qTZV0yCrSTt/PzfFphSOyZPkgGS78lqTw4vL40eSJIoy4mmJkt9/qLXjqEFWtzcoChf8f4AHZ5ayt95Bu8dHRV275mfnnluAyxsgO9GG0SBwdni45uw8ymsPF8dFm9ydXj91bR18uPUgD84sJcV+OPahtXMcNzCZ7xeOw2QUVDU5Iyb7k60gejI4lgDytCP8N7XnhqZzKtHdRTpygZXJIHjo3ztUE8ujH+7kklAA2Wo2kGDrznKaINH62H6X4GJXUlXlIKfH7+MX5xQoQdhnPq1geL8EzbFVNbqUwjG5KG9nbbvyWmZiDEOy4/FLgtte28DO2vYo9xhz1PaYVrMBWyjInRpnpry2DY8/wPrKJsrr2jU/m5VgZfmaCn654ltuf20DRqORV76qUrqvhY+h85gOtrj4cOtBrhgblLdwePzcen6REp/onG56x8pNzHr2S9bubuS657/s8yJz3cFRjYEkSdcd4b/rT8Qgdfo+BoPggmGZrJhzFst+MoYVc87igmGZx731ljuIpcRqN2qR3QDzJxdiiRKE/S50t3GT3S4/XPrpMalgysbIYjBFdC1b+PaWiMykeZOC+fayywiIeM3Z4SMQQOkhrVW9vGBqMQ/+exs+v6Q5KUvS4eNe/KyCuy4cSlWjS6Wa2ujwkJdqi/hsVZNTdR93v7GZqSNzVONYub5aU/02EJCYN3mI4h575YvKUJLBSEwGeHBmKQ9fNpI5E/OVNFt5RzJ1ZM4Jbz7fG+nSkkkI8SOgGFCWP5IkLeruQemcegQCEu9vO9Rteflyv+O7fjRM0w0wJDOe2eODf/iyPHN30t1+5aO1r+yMbIwanJExg8oGFzGmoNjbgGQ7exucESm9eal2Fr+zTfXat9UtFGXGK+eTq5dnj89naFY8EvDM6t1srGnl7jc38cClI7kjJHQnB1lNRsFTs86gssHBmfnptLp93N+pq9mid7by5I/H8Iu/fa3KPnrkA3UCodsbwGhQj8NogPz0OOaeW4DbF1BiT/dfXILZeLiHwfCcJB75YCe/On8Iz31WySVj+pObbFM0j8KvIbueTtX6gWPlmI2BEGIZYAfOBZ4hmEn0ZQ+NS+cUo6uT3dHITLDS4vbi9PgietfOm1TI/aE+yLJboyfoTr/y0dpXal17SnEW6yobNY3hrtp2lq4q57GrRrN8TUXE+53TYt/eUIM/gFJnEG4Qlq+p4PnrxvLprgYmDslg4pAMVq6vJjXWzIo5Z3GopYPNB1rITbHxk+Vfqq41b3KB5n21OL3BFpMeH3aLidgYg2ZD+JH9k5Rag+VrKrh36nDuf3crlQ0u1XF5qYfrF+QJvrLBRXr84eyjGybkR5WrOJXrB46VruyfvydJ0jVAkyRJC4GzUdcQ6OhE5UiT3fGQm2zn5kmF7DzYrlSx3np+EY9cPooV66oUQ7B4RgkjohRudYXulJ7QOm9QXqNAVQh2tAnKYBB4/L6IRjGLppfwj3XVADQ5OyLeXzC1GLfPx/I1FazeUcvQzHh+df4QSvsn0uR0RzTXue+iElpcPkyGYC3FM59WcN3/DMRsMtDi8tLh91OYEY+jw8ui6erPjgpN5uFYzcEWm79+bQN3rNzE7a9t4GCLm0Uz1K6thdOL+dvavcweH2zCM3t8Pq98WcXNnVxXsntOy3VXfsihZB9pub3mTSrknY01p3T9wLHSFTeRbIqdQoh+QAMwqPuHpHMq0t2Sv1VNTu5+YzNFGXH84pwCxW9elpfI/ReP4GBLBzFmw3HXMYTT3dITRzpveIvJY5mgjMKoNIqRU3abnR3KKlvO1Q9/f9nqoIZ/UUYc148fFNGMZkCKJdjkxucnyW5h8TtbqGxwqVJAH/lgJ3+5pow5YTUIf7thHC5vQNWYByHx+4tHcGdYn4DfXTyC+/+1tZPraBsvXj+WRy4rpa3Dh81iotXZwdo9jXy8s165X6vZwNCseN6N4p6TXXeNjg4KM+LYHRasDnc3Dc+OJzPBioTElJKs0yJ19Gh0xRi8E1IYfQj4mmAl8jM9MSidUw+tQp7vshqTdxoXjsimw+tnzsR87BYj8VYzs19Yp5pcM+JiGJ2XcvSThpCLqQ61uslMsGIQdKuLSyZai8kXrhtHenzMMU1Qjg5fRKP2c4vSWDS9mHvf2sKeeodmI/etB9qYf14hOw+1RxSnDUi204yfFLuFX/z966jpoweaXar3nJ4Av1wRWWX+1E/GqIyRyYDKzSN/vq7No1SpW80GHr96dERl830XlTA0MwGTyRDVfSa77sYEJL6palLuDw67vR6cWcpPln/BI5eP4ozclNPeEEDXis4Wh/65UgjxDmCVJOlochQ6OkD3B1ztobTF/sl2bg8VJN10bgF/+jCyEKx41hlRz9N54s9NtkcEuu+/eATJdkuEvs++RgeBgERdewd2iwmP309q7LFN4hBdbK7B0UH6McQ5AgGJBJtZVWcQF2OkpF8i3+5r5qGZpVjNhih+cgmT0aBZOPfUrDOY+/I3zJ2k7e+XM7WyE608dtVo9tQ7MBqCVce/OKeAQWmx1DQ7ae/ws3J9NU5PQGWMHrtqtOaY7BYjc88toCAjjqLMeAamxhIISMcs0aH1fNo7vJoxpf3Nzm4z6qcKXQkgXwa8J0lSG3A7MEYIsViSpG96bHQ6pxTdGXANSAEWzyhRCpsAYqIUW3V+TTmHhpvm6VllEav1u17fpOq+BUE9/do2D//7169Vk8yKdVXcMWXYMbmQZLG5zpPilv1t3PrqhiO6ouSxN7a7uXFiAQvf2aJINdzw4uGd0YJpxdx14VAlo0de/Rf3S6Dd7dPUV3J0+Ljp3AKGZMZH1ei568KhVDe7lTRULcmLtzfUcM3ZeSTa1dPMgWanpjTJrkNtGA2CoVnxSlW6wSCOWaIjHLnHwr5GJ2+E+kGHK6NOHZmjfL+nexaRTFcCyPdIktQmhBhPUF30BWDZUT6jo9Mj2C0G4q1G8tMP90yQJZfDCWbOqDudyUHb/+ysZcfBVpWK5brKRk2DUhSaGOVz3jFlmJLTnp1oZfb4fNw+Pw9cMgKLUfDelgNs2NeMz6dtiOCw2Fy0eoAj5b7L2VmJ9hgWvhOckLWkGha+vQWPP1gB/MClI3hoZin/2nQAtzdAWkLQeIQXrF1zdh5pcRaWr6ng/ne3RYzvnqnDibMYyU60KYZAvtaSVcEuduH5+0tW7UKS4PGrg93P8lJt5KfHUZgZx3PXjmXplaN4etYZ5CTG8NxnlSxZtQuf//iD8/J3+/HOWvY1Ovl4ey1XlKmVUa8oy+WfX1cr93S6ZxHJdCVmINf0/wj4syRJbwoh/q/7h6Sjc3Qc7gC1rR0k2c2KX1mWXO7sEmgJkyiOpqkj5+HL1a6dV8PDshJ47rqxrN3dgD8A5bXtiiGYdVYeSz/aRbLdgs1sVK1477uohItKczRdG7LY3JyJ+eQk2ahqdEXUA0Rbtcoxkz31h11N0aQaUmNjKA8LEs88oz/p8RYcHX5cXj83TMhXNIWWrNrF/00dpqykA5LErecVkpcai9cvYTYJ9je72XYwsn9yeAVyeP7+2opGnvm0ggcuHYHbG2Dx/9saobS6cHoxPz8nnz//p4J9TU4KMuOV7yvcjXckF1y07/a9zQeU+xmbl8y9b21Wss30LKLDdMUY1AghngLOAx4QQsTQvT2UdXSOmQ6/H4fHzx0rNykyyBJQE+qK5fD4FZfAQ5eWKp87mqbO2xtqIrJfHrl8FIPSYimva1NcRXMnFWA1q5u6a63M735jMwNTYxmTmxwxiRkMgklDMslPi6OuvUNVSQxHXrXK2Vkef0BlvLQMmdVsVAWJ77pwKHvqndwVdo+3nFfE85/tBcCPUGoT5AnVbBL871+/Zv7koPRFtJx9q8nATecWYDRAYUY8eak2pRXp7joHT6+u0FQSXfDWFh6eWco1Z+dhsxipqGvXjN8cyXV2tO/WajZwwbCzeO7acaeVAN2x0pXJ/HKC6qNTJElqBlIIxg50dE4YPl+ADfua8AXgla+quOuHw2lwernttQ3c/PI3/CkktbByfTXL11Rw48QCXGHNWI7WuOSKslySY83MmRjMbZ8zMR+LKThZxFtN5KXauOncAixGA49fNZpBafajrswPtbr5Ym8D722OdB3JcZSxeSldkreQs7M+312n1BFoSTXcd9EIlq/ZreTq3zAhH4fHrxgCeYyPfriTH5+Zy2Vl/Vn8ztaICdVqNjF3UgEfb69l8YwSdh5sVrXWlOMTGSEX09JV5dz+2gZ+/v0CttY0c9O5BeQk2rhhQn7U2I7D4ws2m3H5+OHST/msoqFLDYSO9t3ed1EJw7MTT6lWld1JV7KJnEKIWmA8sAvwhf6vo3NCCG+8vuwno7miLBdvWM9eUGfEtLh8vPhZBb+5cLhyjmj1DsOy4nlwZikHQ1kw/1hXrbhrrGYD782fQKzFxNxzC1UtNh+9YtRRV+Y2s5HrnvvqqK4ji0moWjvKRkgLOTsrLc7C7a9tUNwggNJXISXOgscXYPb4wfzhvW1KrcDiGSWak2ZGfAwGg9AMKm+sbuGZT4My1B1eHz8+axD3vrlZXb/wSTkzRuWovos/f1LO/04czKKQgbGaDTx6+Sjt52Qx4fYGcPn8zB6fz4EWl8qFJZ8zmuss2nd75qAUzh+W0aVMpNORrmQTLQDKCPYzeA4wA38F/qdnhqajoya88brdYmbpR7tYGNKtD8ftDeDs8PPr1zawYGox4X//WvUOt55fRH1bB79/73DGTXgcIdlu4euqZg61uiMMzx/+tY17pg5n8TtblZV5eMzgdxePUAK88mfufmMzhRlxlA5IVsYlS1J3nsjePULao8EgqG3tiKgjyE60cuv5RaqmNOH3U93k1Jw00+NjCCBpNu0ZlBarGNq55xZgbnZr1i90LsyeOjJHMQTKM3vv8DNTsp6mFvPM6t3kpdro8AYi3FTy2I/kOotWy/I/g9P0HcAx0JWYwcXAaIIFZ0iStF8IEd8jozqN8Xj8bNzfwsFWN9kJVkb0S8RiMZ7sYfUKwhuvy43b6x0dmhNbSqyZB2eW8uJnFTw4c5TyXud6h/Q4Ky6vj5nL1irnKMqIIzfFzh1ThmI0COra3CxZtZPbLhgaYXgqG1y0ub3KClkWUctLjSXBZsLl8WsWWB1scavSJbuqTSSTkRATcf+XlfWPaG4T7jt/dV01904drlqtz59cyPaDbQxMjdXcaf1lVhmPXTWav6zejccfwB6jnRbbec41GiJdZ5UNLlJjLTx37Vga2oOV0k+v3s3O2nYevXwUt0Tx+y9fU3FE19np2pSmu+iKMfBIkiQJIeQeyHoIvhsJBCR217Wz/WArQgiqm5zsOtRGTYuLKcOydIPA4cbrskvDajbw18+ruHfqMA62dijZMgUZcSz7z27W7mnUnDw61zus3V2vTD4jcxK4alyeqmfukitHcUVZLjVNzohGMm9vqMHnl5QApTxp/fknZ/B1ZRNDs7Rz9Tu3WsyI13ZxpMcdOe3RYhRBTaYPdirjHZwWd8RMnyanh9xUu8ollZkQw5JVu/jF97ULzb7YG8wIWjCtmEGpNspr21k4vVgJesvFee1ur3IfVrOBYdkJmve1qaaV5WsquOW8IgKSxDlDM/j5OQV4/ZF1IW5vgJE5Cbw7b8JRJ/fTsSlNd3FMxkAIIQjKUTwFJAkhfgZcD/zlGD47BVhCsO3lM5Ik/aHT++cQ7IO8J/TSP083WexAQOKjHYfYWN2CyWAgPz0ouhUXY6LR4WbboVaVS+F0JbzxuhCwYFoxK9dXYRDqTmcLpxdz06QC7jnGauBwX/MNEwfzxzAtHwhWOy/9aBd3XThUpYMkX6vD61dcHR/vOMCjl49iU3WwON8bCChN3eXPLJpegsUk8PkCig/baECzEMt4FBd3s8tDktXEwzNLcXn99EuyYjKIKFXHh/X/bZ3rMUxGrvveIJqcnqifdXuDdQvPXzeW+97dTlFGXCiLS8JuMbHiq71cWJKjMjIGiCh6k90+cuB6zsR8bGYjiXYTG/e1aF6/MDNen+B7mGOKpkiSJAEXAa8BKwnGDe6VJOmxI31OCGEEngAuBIYDVwkhhmsc+qkkSaNC/51WhgCgssFBRSjt7pEPdnLbPzZQUefgt29swuWVCD5+HUDR6a9v9/DyF5XcesFQ/q9T8dOCt7bg9UvHnC0SrnYpkLj+e4OUSdgooMUVbP+YHm+NaCSz4K0t5KXFMnt8Piu/ruIHJTnc8uq3PPLBTp5aXUFNkxu7CZZcMYrHrhrFwzNL+evne7n4ybW8saFGySwK7xEtK3S+uLaSg61HVnWNjTEr2VSPfLCTjdUtPPTv7RFKpfddNILMhBjlvHVtHiyhm/QH4IF/b6d/io24GGNElpBcCCffc327h2S7hY01rcx9+Ruqm1zc9PevKcxM4k+rduIPzeP+ANz/r23YzEZmj8/ngUtHMHt8fkQtRU6ijRfXVrKvwcULayv53cUjlOvnpdp4elYZh1rd3aoWqxNJV9xEa4FmSZK6kk46DiiXJKkCQAjxCjAD2NqFc5zyHGx1K9t8UPtJF7y1hReuG3eSR9g72HKghdtfCzZTeWXOOHbWtrOvwanpVmh0eKKcJZJwX3Ozw0N1k0u103ji6jFYzQb8AUnzWi5PsB7zp9/LV3SS5PeWrNrFC9ePxeOV2FLZSECCunZPRCA5M8Gqyh4SoWyio1XHdngPZ1PJdQ6zx+erlEytJgON7W4SbRblvFazkfz0OMpr2/H4A3h8EhurW/AH4J2NQfmG3BQbNc3qQjir2UBqrIVZZ+Upr6fHxyhuKK2gcpzVzBMfb2LupALN3gpVTS6anB7q2juwmASxFqNKeHDOS4flNbpDLVZHm64Yg3OB/xVCVAJKoq8kSSOP8JkcYF/Yz9XAmRrHnS2E2ADsB26TJGlL5wOEEHOAOQC5ubldGHbvR155hiP/cbm9AeraOk7SyHoX4QFkgeA3U4YSa9UOZKbEWY7rGofaOhCgFK4B/Pk/u1j+0zMwG4z86YpRpMSaeW7NHj7eWR9KiTQyLDueRkdk17Fku4XyWocqcyY8O0YOJMv9GTordPZPtFFR1x61AtcZps0k/74IAR5fcAUdFxOcUFWZO9OKibea+LqqSYmz/Pz7+QDEmIzKhC5XV8ty2LIrzO3zqwLSGfEx5KXaGJIZT1leItd8Lx9Xhw97jIkXPqvAHmNU1UGEu8JkPadbzivi719WsnjGCGXy1xIe1IXleo6uGIMLj+P8Wua78z7vayBPkqR2IcQPgTeAwogPSdLTwNMAZWVlp9ReMS7GpBmYlH282Ym6dgqoA8ixFhMpsWYyEywRgcwF04qJsxzbr3YgIFHV6ODrqmZVRe78yYWsXF9Nk9PDn68eRU1TB/e+dXiiXji9GLMJRuem8puVm2hyelg4vZgLhqdRmJmkfI+xFqNmEZccaJYDyXJ/hs4pqBnx1iOujOM7GUOr2UBcjFFJD509Pj9iQl349hae+WlZRFVyjNnI/maXcj5Z/3/OxHwKM4JtL1/8rIJfnFOoGJ1gPEHixokFvL1hH5eV5fLrsOD7wunFJNvNqhjM3HMLyEywEmM2Utfq5lcXDOVgs5OlV47G6fFHGLdw3N5gEZ9uDLqfrhSdVR7H+atRd0PrT3D1H37e1rB/vyuEeFIIkSZJUj2nCWnxZm78fgEL3w4qT15W1p9bzhtCk7ODuy4cGiG0droSHkB2ev28s7GG6/9nMEl2s6qhitkoGJJx9KxnWctm+8FWlea97N6RV772GAv3/l2d7rjgrS28eN04/vyfcsWFsuCtLUq6pKPDR6zVhM2sXW1rNMB9F5VQHOrCFi21dGN18xFXxgk2s5JNJK+8JQll9R1tQm1zhdpOhlbwTY6gHEay3aLSd2pyerCZjdz/7jaanB7mTy6kssGB1Xy4z3SHT2LhO1t4cGapYgjCn9ML141TuY6sZgPP/LSML/cE3Watbi8j+idSnJ3IlgMtzJtcQEACm1lbftuuZ9b1CF3ZGRwPXwGFQohBQA1wJXB1+AFCiCzgUChtdRzBoHZDD4+rV2HEqBgCWfRMXlndcl4RDU4P+Sd7kL0Ag0GQZA9KRXj9fiYPy+ba579SDOjg9DgyEmJweLzH5FOWtWxumJCvOWHmptiYO6mA2rZI94/bG+BAq5sLirNpcHjYWNOK2xugweFRpaUunF5MWV4i6yoPt/6wmg1MLExnVP8kJZvIHkXOOrdTWmzn2oMObwBrKKg+LCuBfY0O+iXZlSriAUk25k8u4NVOFdUxZiO3hDWkXzS9ROnZIHcDEwLOyEtm18FWLj2jPwYR3MU+s6ZC1Wf61vOLcHsDuL1+zedU13a4FkTe3bi9ftXO5PGrR0foEN05ZWhE2uy8SYV4/epr6HQPPWoMJEnyCSHmEtQ0MgLPSpK0RQhxY+j9ZcBM4OdCCB/B1ppXSqdZ+szeUJOTcNEzOKwZ89L1egAZgpP34ne2MnVkDmaDUXENHWhxs3RVMM//2WvH8vO/fsOKOWcdNR03fDWuNRFXNbpCzeDHab6fZDdzx8qNPDizlHkvf4PVbCDBaopcGV8/lv+WNyj++RH9EyOE6zx+v6biqsvtYelVo1U++PCgcoPDw58/qeCSMf3JiI9hb4OD1DizZhWx3E7zvotKIuSn731rs9Kz4UCLW6mbsJoKcHkDDMuOJ8Fqwh8IMGNUjiqobDEGU1mzNArgrGYDmQkxqjaVBgFTlnyquv7G6paI3dnv39vO0itHM/fcAty+gCI8OKUkq+u/PDpHpad3BkiS9C7wbqfXloX9+3Hg8Z4eR2/Gag4G2KIJeHUlM6Y30xU5Yi0aHB3c9P3B2GPMHGrr4OZJBRiEUAK9K9dXK5XJ4RW+na/bL97K5oOteEOKn6t31LJgarEiGyEHSt/fEpQ+3ra/hSevHsPCd7bg8UnBXUhaHP6ARFFGHC6PT5m8dx1qV4052W6hqiGYnSTvYNyeoPT0oLTD958aG8OKdVUqrZ+Pth/kynEDVT74RTOCQWWZBKuZJqeHJz4u54Lh6eQk2RBCaFYRL7liNFsOtBAfY9Ksih6cHqdawctKpk1OD3+8rJQ5L63ndxeXEGsxqYLKBZlx3P2jYTg9Ps3n6A8Eohb5Kb8bkrY7y+MP8PjH5apdhS453TP0uDHQOTrJdrOi/6K1skqNO3oLxN5OdzSVt5qM+BEqN0x4oDco0BajqvDtfN28VBs3nVPIvW9tpigjjgXTijnQ4opoKr9sdTl3TBmmGu8fLwsWdoVn/CyYVszw7HiemnUGta0dZCbEMDIngY01wVCYLA2h5QIMv/+BqbER11v2kzNY8NZmVfD1iY93kZ92WBJbbpCzZNUuGto9/PGDHdx54TDNiTUQ2nAHJEnz9ywtzqIqGLNbjFhMwQrnqsZgCu9vX9/MrecVqp5VQ6sbk8HAtgNt/Le8LhiL8ASb2r/4WQW3nj9ElRGVnRhZbW0U2ruzoZnxql2FLi/Rc+jGoBcwNDOBPfUOkmxmzSpUX6Dv+0i1tOa7mibo8Pg0u2vJgd4lq3bx3LVlqsBs5+tOHZmjZAVNKMpg2Sfl/OL7BZr58dsPtqquteNQW4QrY9kn5dw8qYi739ikMhB8UcnO2nYGJNujugBvffVbUq8bR3p8sFK6s65OVaMjognMvEmF1La5eW/LQaYUZ6ka5HT4JSobXNgtxihVxBJLV5UzMichYgW/cHoxj36wIyK28cw1ZdS1dfDcf/co485MtPHIhxuVz75w3Th++tyXisH7dSdjHQB+uPRTlRF8/OrRijCfLFvROT4wf3IhJqNgYJouL3Ei0I1BL8BkMnBhcTYfbDukVKHKq64X11YyMK3vb4uPV4gtnIZQsZZMdqKVS8b0Jzc5GOhdub6ahnYv00f0UwKzh1rdKknmIZnxSqBUzsePt2kHbzvHKbVcGVNH5iiGQL6nhW9v4S+zymhyeenw+pk/uQCbxagpDb23wcFnu+sZ0T8x2OgmzJ1S396hGAL5Xt0+P2lxMdz+2gaGZgUlGs4pzCA9LoYWV1AXyGwUmouKykYnQHDX8mUlD84sRSCRYDPj9vpVhkC+lyanlztf38Ts8flsrGnFajYwKM3OijlnKU3qq5tcqlRUpavYwGTa3F5+26l3wq2vfsv/u3mCasXf4Ohg4dtbI373R+cmKf2QdXoW3Rj0EkwmAxkJMYr/VyYoVtb33UTRtOaPVGHb2dcf7l4IbzcZPuGlxpn5cEet4n7JTrRGDabGxRj5+ffzqWxwct9FJexrdPLqumolyPrYR4fbdWQnWinpFym6pqXK6fYGaHZ5+PVrh1fPC6YV8/Pv50c0pm90eHhqdbBPQEF6nGrik+Mfne/16dVBgbdGRwcDU2OVLJxfnlfI/MmF1DQ56ZdkVbl8+iVZeWntXuXcG2ta+XWoD8LyNRX8+cdjNL+f+FBAPDclmJVUkBHH8KxgXwA5JuP0+FW1CXLw2SDyyU60KQVw4c+nrt2tNJiR0frd1/sTnzj0Tg+9hEBAUqpOw3VhFs8oISPhyNW0chPwtbvre61+S26yPeLe7ruohNxku+bxsq//h0s/5aq/fMEPl35KZYOTBy4N6taEu13khvQurx9Hh58H3tvGnvpgkbw/gGYw9bKy/sQYDTg8fpas2sVt/9jIU6sruOW8QpZeORqLAa4el6cU/V1zdh5VDY6ITmKyKmc4Qf97TMRuod7hiRiH2SiUsde2dai+u6yQAY2WZWYxGdhU06y4wV74rBK72UhcjIVHPlBrBD3ywU6uHz9YU3PI7Q2w61Abd104lHmTC5g7qYD5kwuChWgmoWRWPbW6Aq9forrZqRpnrMXIgmmRekb/WFfNwre38NsfDmPupAKleFJrkg/Xh5KP0YPFJxZ9Z9ALCA9yJtuDQbxBabFYTUaWr9lNboqd3BTtrXK44ml46uKkIZm9KtBW1eTklZBrwuXxYbcE0yTH5CZruom0Ygx3vr6JZT85I9hAPtEWdYcwb1IhB1qcDM6Io7ZN2z3VP8lGVqKVOaEGMPLr9761RVHRjLUEBdbOHJTMnJfWK123wl0Zf/64nMUzSlTdzxZMLabV7Y24Zmcb7fYGSLJZePj9ncqKPzyonBRKLHBFyd+vbHCyq7Zdee9Ai5tlqyv45eRCzRhIeW07cybmMzg9jro2N8vX7FXSQ9/ZeICfnD2Qp8N2Lv83rZhWl5cFU4t5+cugyuhdr29izsR8hmYlKOMsSo/nQIubJ68ew9f7mpEkVKmn2w+18cynFYr0xB1ThmnKiuu9CE4u+s6gFxA+8ck583f+cxOb97eyrrIFp8cX9bNVjYcVTx//qJynVldQUeegqlG7T+zJosHRwaShWfz6tQ3csXITt7+2gUlDs2h0aOsuRYsxfF3VxNJV5ewLySZorZqXfrQLizFYpSoXc4VjNRuobHTh6NCeZAOhCl45iytcIkF2ZTz+UTlPfFzOztp2UkONdB64dAQPzixl5ddV2MzGiGt2ntesZgNVTU7V2MN7/Na2dfDi2koKM+I172FXbTuBkGSJzIEWN/tbXJrHlw5IpCgjnn5JMTz8/k5lsgY4Z2gG93ZqiPN/b28h3mZh2epyJhRlqJ5P+DitVhPfL0gnzmrimU8reOLjclWBmyx/vfSjXSy9cvRRM8hOryqj3oNuDHoBR2vkPSCKKwWCE8bfvjgsfXzDhHz+9kUlTQ7PSXUddXZd2cxG9ta38uy1Y1l65Sieu3Yse+tbMUcR7JdjDOGEB3VXrq9m3qTCqD57R8iAysVcWu6RzIQYpcH93EnB//JSbVhD9R4mY7A3cqzlsNBa53MtCvUTlo3cr1/bwKVjcvEFAspxeak2nrh6DIUZccyfXKDEPu6ZOpx/rKuOGHttW3AitYfy+e9/dxu3nFekuu69oc9qjWlwRhy3XTBE9dot5xVRUdvO3Je/YeO+1ojPyFlPnceysbqFygaXEvQOn9zlcQJYLEZcHl/UZy2fz+X1axoCLbfge1sO9kqX56mK7ibqBUQLrhoESnpdNFweH9d/bxANTo/iJrrp+4OpqHfy2ze+OO6c/u+Cdk1BKWMHpXP984cbwy+aXhz13von2lg0o0RZrVrNBn530QiWfrQTQMlc+e2Phmk+u9gYExV17aTHqYu5zhyYzN1vbuZAixuTQSiaUOGBXq/PT16qjfr2Dha/s5Vku0XJzpGF2wanx9Ev0UqLq4P3txzk2WvHUt/WQXp8DK9/XcXMsjwenllKo9OD3WLkpr8fln5YPKMEg5BItFuU4q3wscvdzTx+P3dOGYrT6ycjIYaHLyul0dFBij0Go0FweVl/Xl1XrWTwGA0wLCuBfklWmh0eVQDZZjaQGSpWa3J5eXtDjcrd1d7h1XyOvlBhniyaKBeihfv95e+7qsGhPOto8tfRAsLdkXqs890QfVH5oaysTFq3bt3JHka3oTV53jN1OK0uLy+ureTRK0o5Kz9N87NfVzbyxZ5GVbbMPVOH8/Tq3aoqU6v5yM3Vu5OKunYlr1xm3uQC3vy2JkKZ9aGZpYwblBpxjg37mpj3yjfK8ZIEX1TUceW4PFXR151ThuL2BQOq4Tnzf/s8mOf/yOWjsJiEktN+34zhGI1GJf3zZyFF0PDn9PDMUmxhEzgEs4kuK+vPyP6JuD0BKuod+AMBhmbF4+jwc3eY0Vo0vZjqxnaKspKIt5mUxvTh13j+urFs3NeMP0xUTs4wurAki4Fpceytb+f9rYeU3Pu8VBs3TixQ1QeEy0zcdsEQUmMtpMXH8LMXI+/rT5eP4sa/fa0ZZ3n86tHsOtQe2WlNQFJsDAlWIz4/ODu8PPHJbu6YMkxZXMjf9/zJQbHhJat2kWy3RGRxHWlBsnZ3PVf95YuI11+Zc2bU332driOEWC9JUpnWe/rOoBcgB89y5pzFqu21+APw+EdBv+vR0uucHj+vfFWlqlJ9evVupo7MUQUQu5rT/13QcnslWs1cPS5PNWnfcl4RLS6v5jkOtLg1g6A3nmMOKoM6PGw70May1RUAqtz2e9/czNSROWysaY3IaXd6/Pzpg508OLOU9g5fxDjd3gASsO1Aa0RdwH+215KZYFX1BpANb/iK9tV1VVw+NthHOZoIXl1bBwNSYln0zlZVExqA7QfbCEhB/ffwpkdTR+YohkA+z5JVu3hoZinbD7bh9Qf406qdUSuQ/QQz1g60uFmxrorHrx5Dm8tLapyFVrdPs8blVxcU8dhHu5g6Mofla4IpsE9cPYbh2YnKpC5/3w6PXxVgh6BcdWFGHIWZ8UcMCB9P6rFO96Ibg16CwSAYkZNETbM7QrLhSOl1Hr9fs0rV1MkVfyL/sLT+sIdkxXND2GpVTo+MJsLXL6x3gYzVbMBqNtLQ7iHealK6ZskpiwYBFqOBRKtZmYw657RX1LWzs7adeS9/w2s3nqV5jcoGByajUK1s81Jt3DdjhGon4fYGWByazMON1jXfy1dJOWtdIzPeytdVTUpAWl6th6+k7794hFIgB9H1/XfVtvHEx+XMnRSspE6Ls2he80CzS5mox+QmUX6ojUSbhXWVTZw5MEUzz98SanYjX3vJql0RInvh8R2tcxzLjlROLe3K775O96IHkHsR8g7h3XkTeGXOmbw7b8JR/fzxMWbNbJqSnKRjytnuiRoFrZzxNrf2KrxBQ4QvEJDwBQIROf23/2AIDe0d3PbaBn7zz03cen4Reak2Zp2Vx/I1FSxdVc51z3/FVWfmMSAxRvlcdqKVDfuaeG/zAdrcPlb87zhe/tmZwcYzM9S1D7ecV8TfvqhS9QTITrRyRVkuG8J6C4TfQ+fm8q6w7mNaAd6F04txeLzYzEYlMKyVFXXX65u4rKy/6txaQfWyvBTV+WtbXRE1HbddMITla/byxMflPPNpBc4OP74A/GnVLpauKmfL/hbN4O/+ZqcSM5DH1Tm7Tf6+395QE3GOY53Qj+d3X6d70XcGvQyDQagkCY5Gq8urKXPQ3uFVNXyJjYm0+90hHhftHjrnjB9qc2muVhNt6l9BeUxOT6TbIi/FzuL/F1yJx5gMDEqN5bc/HM68V76JKPB65poyrGYDT80aw8bqFiUNc9uBVganx/Hi2j3MP68ISQrw+FWjcXj87K5rV1Q6+yXZlHPKE/VDM0s1O9KNzk1S7s1qNtA/+fCuJrxb2PDsBGwWI3/7fA+XleXR4PQwMDWWp2adEbX1abiS6NsbaiK6uj1y+Si+l5/Ku/Mm0OjooDAjjuRYC/WH2lUBZEsoUC/HAqxmgyq429rhjwgqr1hXxYxROUqbTvnznbPblO87K9j6c8Wcs3B6/F1Wpu3q775O96Ibgz5OcmxkoG7+5EKSYy3MWv6l6rW8lFiV3EFPZnB0/sNuc3tZMK04InPH3qk9pTymp2edEeFyeGrWmAiX2D1Th6tcKfJ9NDu9vDtvAs4OH1v3t6kaqcyfXMjPJga7ad37VtD/PzIngRsmDuaW8wrJTY3FHNLol1N83d4ATc6OiADugqnF+AIBHpxZiiRJGIVg5bp9KiG4JqeH7EQbdouBOS+t564Lh9Li8qrG9GhoJ9XZWO4PuXbirSaGZcfj9vh46fpx+AKSarKVn/WYgMSa8jpF9iL8XA/NLGVXbRtZiVaWfVKuWkB8vruOW88fwh0rD0to3HdRCRnxMdwTyr6ymg3cen6RZgaYPpH3fXRj0MdxhuQUOgcVn/rJGRGvjclNVhmD7hCPO1ZaXF5Wrq+KqEDOmVykOk4e0956J4umDcceY1ZaSGbExTD/FbXxWvzOVqUpi0wwPhJDfnocn+6q03w+T149hvq2w/IQG2tamffyNwA8NHMkA5KtivGCYK1AXmqsKjPI7Q2w8J1gq8tfv/a1kkHz7pZDlNc5eHhmKX5JIjvBysHWoEjcnIn59EuyMfflbxQX1CVj+rOv0RFRyXzP1OFKIkE4r8w5k7MHa2fYGAyCDl9A83uFoDTFW9/UcOmYXJVRu++iEi4cnsWInESVeNy8l79VjIYkwXP/3cvI/om6eNwpiG4M+jjRMmLa3L6I1zr7ek9kBodfCjBtZA7ltW1KPcS0kTmKxn7nMQ1ItXGopUPVu2DxjBLNXUBusl3lplk0vRhEgIq6dtxev6YbzeHxc7BV23V1qNWN1Wzk5S+C8hlJNiMDkm1sqm7RfNZNTg+zx+djMxsY3i+B+ZMLcXj87KxtI9Vu4ZEPdjDrrIHExpgpy0umJeTau+bsPBJsZiU7KS/VxiOXj8Lp8bGn3onD7dWsQzja95Ofqt0X41BrMDvrpnMLIrKS7n5jsyINoovHnZ70eABZCDFFCLFDCFEuhPjNEY4bK4TwCyFm9vSYTiXS42I0g4qxMcaIytrUOLXg3YkUB0uyWnCG+t7KshlOr59EqzliTI9dNRqb2cj+Fhc3TMgnO9GK2xvgnjc3c9338iLu62Crm9nj85k3uYAXrxvHh9sO4PEFNfQT7SauOTsYYH78o2Dw9JbzCkiJNZMeb+Uv15RRlpeo3L8cQK5vdytZRyaDIein9wU0n3W81czyNRWkx8fg7PAxINmOQQRX4ctWV1DZ4OLh93dgMRlZX9lEenzQELi8fsUQAFQ2uLj11W+JjwlmSuVnxB3X9zMoPY4/Xqb+3O8vGUEgIDF3UgHxVmPUHWHn70IXjzt96NGdgRDCCDwBnA9UA18JId6SJGmrxnEPEOyVrNMFml3BYqOH39+hrIzvmDKUtg6fknop++elTvr8BoPggmGZrJhzFgda3GQn2ijOTuiRDA5HFHfWiGvU9S8+X9Dff/PLhxufhAcwk2NjeORDtSjcy19WKp3FLEYDowak0uwMuoA2V7eqrluUEYdfEsx+Yd3hncSMEn420cSW/a1KANmA4HcXj+C3r2+i0RkM7sqZQeExi/+bVkwgFDN4ZvVuAH5+ToHKbSXf7/6moPJnVqKNV76qYlppjuauxSAE786boEy6XRVvMxgEF5ZkMSx7Aoda3Xj9Eve8uYnKBpfifspLtUUUJXZe8evicacXPe0mGgeUS5JUASCEeAWYAWztdNzNwEpgbA+P55QjxmTkpc/3qrJAvD6/oqUPhzNsnp51huqzgYCkaOHL/Xn3NToZlp2g6s97PHTuReDwaLuznKH+xTIb97cofnP5mKUf7VLkFjq/t/CdLcw9t4AJRRkYDTCyfyJLV+3k5lAsItZiUl33homDVTUAbm+Ae9/czKOXj8IfgEvP6I8kwbOf7eEPl4xg9vh8Eq1mpVgrXPrhrPxUHnl/O/PPG8KvXzssN2GL0mnMHmNSvovZ4/OJizFqBv8T7SaVq+Z4ArNyQBdQVYPLcZZlPzmDG/+6/qg5/Xpg+PShp41BDrAv7Odq4MzwA4QQOcDFwCSOYAyEEHOAOQC5ubndPtC+SoxZcOXYXNWEct9FJZoTb+fX9tQ7eOC9bcw9t4DMBCvVTU5+9+42mpyeqCmmx9LUXitlVU71jJggLWp1z2iS00YD5KfFab6XlWBVyUEsmFaM2xs0MrFWdRczV5QYiy8QUO2kFk0v5ulPKvh4Zz3jBpYp2kQHWtxKJW75oTYmD8vCbBSK7LjL48NqMmh2Gqtucqrux+eXlGbv8utLVu2KWoh3PERLEogxGfTewjoqetoYaP12da5q+hNwhyRJfiGi/zJKkvQ08DQEtYm6a4B9HavJGNHVSo4jdJ54k+xq//z+Fqdm9fJLn1dqpph27rtwWVl/ijLiI3YSWimr8TGmiB63t55fRFyM+lcw2tjPzk/FGxJN6/xeZxno8DqDtFiLamLubBzkc9jMRp6adQbNTi9JdjOuDg9r9zQCsO1gG/FWk+oZZyVaqWt1E2sxYgkpr2YlWNjbEOCOf27k+u8NUjeXNxsV6Qyr2cCwrAS2hfVYlgn67rVlveXv4GjGOJxoSQKZCVZ9xa+joqcDyNXAgLCf+wP7Ox1TBrwihNgLzASeFEJc1MPjOmVoc/t4ae1eCjLiGZBkozAjnr99sSeiAnXepEJcnbKJYoxGzerlS8b0x+0NcKhVHVCUJ3m58fnTqyuY+/I3/Ogxtdyw1mrUL0lYTQbmTAxKbc+ZmI/VZIjIJpJAsxK2ttXNc2v2sGh6ZCc4LRnoFlewzqAsL0VpGD93UgFZCTERXbkWTi+msb2D/31pPfNf+Zb/fWk9CfYYbj0/WB38wmeVBAISZw5KIT8tlqKMeA61umn3+JGkYNwGoNXlZ+HbW6hscLFsdQX+QNAQ/M/gNJ79bI+Sq3/LeUUsX7ObMwcerhyWkdNitTgemWc9CKxzrPT0zuAroFAIMQioAa4Erg4/QJKkQfK/hRDPA+9IkvRGD4/rlMHt83P+8GzFDy5PNvExxohq0gcuHan6bLgfX853l5vG56XaInoNyJP8JWP6q2ShAR54b5vSoF1rNbq/xc2fP6lQruEPwJ8/qeDuHw1TXSMtTHI6xmRgYFosB5udGAwGKhocDEiJ4dlrx9LQ3kFaXAwmo9BMv0yNsyir3klDMslPC3Y9c3n9GJFU1dmpcWZ+889NqufV4vTx3H/3qu4xKFGxU6Wk+uxne1g8o4Sz81NpdR+uIpZ7AUPwed53UQmODj8ev4RA4uZJhRgNgnumDlcJ382fXIgpykr/eIoE9SCwzrHSo8ZAkiSfEGIuwSwhI/CsJElbhBA3ht5f1pPXPx2IizErSqBwWADu+evGqnzg8yYVRtQZJNuDgmbySj/cXbR4RgnegI+KunZl8pAn+XirUdO91OoKNtQ51OrmL7PKuDssgyU11qKZs54Sq053zU22M29yEUtX7eSKslyVkfu/acW0uHxsPRCsVSivbWfSkLQIiYaF04uJC6tsDg+CVtS1c/cbm1UTenaCJSLu8pewCmhZRG7r/paI4+ZPLmTL/mDG0u8uHqGZpZMeF8OD/97GL84tZFNNM6MHJLL1QJsi9TxnYj65yXYOtrp5cW0luSl2Rmt818dbJKgHgXWOBb2fQR/n/S0HmPPS1xGvP3H1aKxmI60uL+V1Dt7ZWMPvLhrB9wanKavCL/bUsbfeRU2zS5FGkLGaDbx0/ThmPfulEkwG+LS8FovRyNqKBgJSMBXyQIubvFQbv5xcxJ2vb1ImyvsvHkFKrJlYiwmP30d1U4dq0l4wrZjh2fGUDkhWrlvZ0M7XVc2YDQal4EwmL9XGnImDVSvp3108gn9vrmFq6QBcHh82i4kXP6vg3qnFjAw7r4xWcPuhmSORkEiyWWhyeEmONRNvNfF5RbBPxOzx+SxfU0Gy3cKNE/OVRkIGAal2C8tWVyguoCevHsMvwhrZLJhajD/gp9XtZ2h2HN/ua2FU/yTlmPDnLV/nr7PPpGxgSsTYtfpEWM0nrk+FTt9H72dwCmMzR0ljtBj5+d++ZskVo1m+poKF04sxG+G9LQeVLCGBgSf/U85vpmjr39c7PCpXRG6ynbo2j6q5zD1Th9Pm9uLzS4ohkD+/ZNVO1eRdlpfIX2aVcbDVjc1sxGAAf0B93UMtHdz5z02afQCmjsxRFWm5vQF++3rQvSNLSci0d6hTVmW03CZGIfFZRRO/fu1wquXyn5YRazEGV+0pwZaQcsP5S8b0JzfFRlWjSzEE8nhaXF5FckM2TGfmpweL3q4ew9OrK6L2ODAaYNH0YjITLVpD12WedXoUXcK6jxMbY4qQep4/uZCYkIGQkJgzMZ8Or5/2Dr+qkXldWweJVjNxMdpN4+NiTDx21WiKMuI41Opmy4EWxRDA4Zx1p8dPVqhKOJzOk/e6yhZ+9tI69jY4mfvyN9z66gacHvVn6h0dyvGdxxSt33HnNspWswFL54YOYchuE7nHQbPTpwjoyeesaXLx50+CQeCM+MNV3nIsYH+zi+VrKlTSGFazgaxEK+W1bexrcrG7to1pI3P459fVuL0BNoZJYGs974KMeJ74Tzn7m7SziXSZZ52eRN8Z9AGOlE7o8HjJSlCnluYk2fD4glo3MSYDS1eVK8qYbm+ABkdwskmwGfnxWXnsqW9n0fTh2C2HReGcHV4CgQC3v7aBBdOKSbab2Neo7bMOSLCv0anaoWQnWslNsWkeH954pr1DHcfISbKTl2oj1mKMCK6OHpCkuQsampWg0iaaN6kQR4d2BzUtDmj44uvaO5SYQf+kYpUKqdVsIC/Vzn0Xlah2SQunF9Po8KjUSG85r0gZpz90Ca1K5nmTCvn9u9s40OKOiO2Eo/v/dXoK3Rj0crR83A9cOpJ+SVZSY2MwGYz87YtdXP8/+XgDEkaD4OlPdrOztp0F04qVSo9gYZVEXqqN6kYXs5Z/yYMzS1nw1hZ+WJzJWYPTVaJwi6aXkBZvUfL2X7p+HMmxZs3JWJLg1fXVyuQoi7Dtb9YWgpPDVHIdQDjDMuO56dxC7n1zsxJcHZweR6vTS6urg9t/MISH/n1YemPBtGKeXbM7InPq4Zmlx/yMszW6qr26bp+iWtov2c69b26OyHAanZfEsz8dS317BwYhSI0zc93zkd3c5kzMp1+ijT9/Egyeh/c46J9ko7LxcON4q9lAboru9tE58ejGoJejlU54x8qNSrDxqZ+MYdLQLG79xwbVKrPu80oWvr2F2y44vDI1GwR3TBmmnM/V4SPZbmHm2Fyuf/4r1TXufWszL1w3Tvn5UFsHcTEGzRXtS58HG7IPSLbx7rwJ1DQ5+dlL60m2WyKOlxu4yxlLnbXxq5qc3BuqJj7Q4lZ2Nb/+wRAyEmIJ0KHaBfkDQTXU37+3vdPOIPrqujPF2QkRq/ybJxUqhWYtTq9mP+Zbzy/iyf+Us/ynZXyxp5FCtCukB6XFkmgzqQLsTU4PcTEmXB6/KuvrkctHMShNNwY6Jx7dGPRyDrW6NcXM5IYrdotJs3Bs7rkFuLwB4mPMzJ9cQH56LCnxZr6tPCzDnBRqjPN5RUPUSti5kwp4e0MNqbEWfAGJj7Yf5Llrx9Lk8BBnNVHb1sFvfzQMZ4dXKS13ePzKZC5r+QgBJdkJ7GlwKPo/j3+8i/suGqG67p4Gh+ZY+iXZaHH6uPOfmyJ2Gn+6fBQPzixlb72DDl+AFeuqWDyj5Jifsclk4KLSHAoz4jjY4iYr0UpxdlDJNCvBitPj19zhyH0Dmp1e/AGo6uQqk48blBrLyP5JAJQOSKK2zU16nBWjIeiO+uvsM2l0eMhMiKE4rNG8js6JRDcGJ5FjkRbISrBqipmN7J/IkMzRNDk9mpNnVqJVtdK9/QdD8PkDFGYebqNY2+Jmyapd3DAhX1u+wmbmmU8rWDyjhKwEE62uALPOHojb68cgBK0uH/ub3fgDAQanxxJjMvDDpZ/y/HVjlfPJAVc5dbLz6trt8Su1CZkJVmJjtLOjLEZD1KYtLp9fMRLy80mxa2fkRMNkMlA6IJnSAerXSwcks6+pnUXTi7k3LC1W3hFZzQYOtLiUeoTOO6E/XjaKkf2TlO813N8fCEhsPdDW7W1HdXSOB90YnCSOtf9wm9urkmBOtltwef20uLwIEazY1dTraVTr9Tz07x0s/2kZj36wgwcuHckdKzditQR17bUCmkGJBhdub7CPwEvXjyOAxP5mN498tTOi6Gz+5EIyE4IZRWaDiBBqWzCtmGWfqA2B1Wwg0W5ScuetZgOPXDYqQsNo/uRCUmItxMZo6wplxMeoXEf9k20MzUrotu/K44VX11Xx6OWj6PD5qah3KK6xW88vUuQgwmMBowckkZcae8Rq355sO6qj01V0Y3CSONaJYH/L4UwXuRI2fBL+2+xxERW4i6aXsG1/E0uvGo2rw4c9xsRfVu+m3e1j0tAsEmwmHp5ZqgSEO0szn5GXzGOrdnLd/+QrY2tyeokxG5QirM6uqSWrdrHsJ0GJ7AMtHRHN7F/+opKff7+A/wvrgXz7D4YghFA/g398yzPXnBExuZf0S6S62ampBpqdYGXy0AyVi8d0hNTSrlLb5mZdZQvrKr9WZDsuPaM/QzLjuf/dbfz2R8MUI9Xk9DA0K4HvF2UcdXV/ItuO6ugcDd0YnCSOdSLol2SjLC+Ra76XTyAgYTIKijLi2FgTVLx0egI8+Z9y1cT74bb9nDesn0rKYcG0YlLizMxfsYslV45mZ20bEwrTuOvCodQ7PEoryhS7hd+s3MSBFjc//V5wDHIRm9xiU45XdB67LBudGqctPSG3h5TH+cyne7jrh0MjzmM0GJg8NIMDLW7S4mIwGoKB5UMhuYbwc7y4tpLRuUmclZ8W4eLpLsK1ljq7vZqcHoZlJRyXHPSJbDuqo3M0dGNwkshMsJKXalM0cgDe3lATMREkWo3cMCEfnx9cXj8mo4kbvz+YZZ/sZmNNK82uyEyXpVeNjmjgsvDtLbx4/TiS7RYlF37UgEQ8filqXrzJIFRFbDazRVXc1nkSi4sxMX9yAZIUUNIy5fPed1EJj320K0K3x2SILL6S4yc1zQf5yfIvlHP8ZVbZSenJq1X5O29SISvWVSnZP+HNZL7LefWKYp2ThW4MThL9E21KPr3i3plRQv9Em+q4JqeXFpcvQojtlguKuO65daRq5P67onQVq23r4LKy/soknWA1K+0y5WPkvPicJBup8WbmTMzHbjbS7vaRYDUpvv/OMYZbzy9iU00LT62uoCBjJCmxZpUyaJzVyK3nF7G7zqHsQgalxWI2ClXBmDwZarnRlqzaweIZJUq3M9nI5Cbbe/S7CpewONTqxm4x4vUHmFKS9Z0UQHVFUZ3ehG4MThLbDrXyxMe7VBLJT3y8iyGZcSrhtg5fQDEEEJwUF7y1heeuHau4b+RVuNxwJjvxsPtB9nEbDcHGMcl2M7L8fX27diZSTqKNJ/9TzkMzS/EHDreAdHkDrFxfxW+mDEMKyUBX1DvwBQLYzEae/M9u3N4At7+2kSevHo1fBC8UAA41u2jzBFS7kNsuGEJeql3TxaLlRjszP53Hw56ZJMFjH+1iTG5yj/vYe6ryV68o1ukt6MbgJNHg6NCUgW50qHVpWl3aq/x2t4+HZ5ZS0+zm5S8qWXLlaFqcHu59awv/sAe7e73yVZXqGk+vDrZrXLm+mianJ2oryqomF5UNLuraOli+poLbfzAEq8lIs9vDpKFZ3BLW6Sw32U56Qgx76xyq8X1b3YLNbOTFtcGsm7/MKuNnL6mrcx9+fwcr5pylORlq+dONBjSLv/SAq47Od0cXqjtJJFgtShOXuZMKuGFCPivWVRFvVefHJ9rNmqJmiTYzc1/+hrT4GHbWtrOxukXJgz/QEgy03n7BUM2sH7mTWUwoHhAeB5g3qZB/fl2tStmMsxiRgFizWTmfXB1895ubsZqMvL2xhkvG9FfO4w/AklW7uP/iEuZMzGfLfu0Wj06PtrqoVoeusXnROoPpAVcdne+KvjM4SQQkSXNnIHVqEd3k9GpKQDS7gkJsPr+P3188Ao9fXZB1oMXN9kNtmhOw7JZydHixm43Mn1xIelwMVU1OJX9+wbRiGp1eijLjMQBur48mp/YupbrJyaVjcvEG/KqCLLc3QH27h8KMeFVsQEYOFmuh5U/PTbbrAVcdnR5CNwYniRijIaJ15Ip1VfzP4FTVcUl2s+o4WYjtwUtLmTupAKvJTIvLSZPTE+lWEdpZP7JQXKzFzLOfbQ1lNHVQkBHPj8/MpWxgMnes3Kh0KZs/uZCsxBSMBm1ZBrPJyG/f2MAL141j9vh8leja3gYny9dUcOeUoRHFZEebyLX86XrAVUenZ+hxYyCEmAIsIdj28hlJkv7Q6f0ZwGKCcUYf8EtJktb09LhONu0er+bOoN2jll6ONRv5xTkFEdlEsRYjz3xawfiCVP7w3nZNUbgUu4VbzitS2mJ2FoprdGqPYeO+ZiUFVHYtPT3rDFZ8WcWi6SXc+9bhbJ4FU4t5ZnUwcFzX3hHRalPeIfz+ve28duPZTB46gbr245/I9YCrjk7P0KPGQAhhBJ4Azgeqga+EEG9JkrQ17LBVwFuSJElCiJHAq8DQyLOdeI5FO+h4MRuNmgJzL14/TnVcbbsbm0nw9KwzaHJ6SbabaXZ0cKgtmG3T4vQqPvxwUTi5OhZQXjtzUAoGgSIUl2I3c7/GrmPqyBzVGNzeAC5PgBEDkvhw235evH4ce+sdWC0mnlkdrHeQYwzvzpvAzkNtbKppVXYI8jnaO3yU5CQxOEOfyHV0ehs9vTMYB5RLklQBIIR4BZgBKMZAkqT2sONjoZPT/CRxrNpBx0t9e4em/72+XZ1NlGA1s/DtraoG7nI/YwBrWNvL8OrYORPzlYlYfu3s/BS2H2jjmU+Dq/cfjciM2HUsmlHCq19VqsZgNRtIjTMzNCseq8nIg+9t49Ix6mb1i6YXk5lgIS81ONH/csW3Ee4kPdCro9N76WljkAPsC/u5Gjiz80FCiIuB3wMZwI96eEzHRE+LiEUTmEuLjVEdFx9j5OZJhSoF0vsuKqHREZzoLaZIUbhbzy+iX9Lhhi2ye8hoELz4+V6lg1ijwxshZfHEx7tYMLVY1dT9votKiDEGq4Xl3YzHV8mDM0txe3zkpcbS7HQTCASNpF5Zq6PT9+hpY6C1hI5Y+UuS9DrwuhBiIsH4wXkRJxJiDjAHIDc39zsNSnb/tLg8+APBVXp2oo3i7ARF4KznRcQCEbLIi6YXg0F9zSanD6/PrxJu8/r8xFuDefitbj92s1H1fozRQKOjI0LDJy/VzvzJRTz/3z3MHp+P0+PTzNt3ef0s+8kZNDk87GlwYDYKHF4f3rCMpY01rUoT+gcuGcGfQmmkg9Lj9MpaHZ0+SE8bg2ogXD6sP7A/2sGSJK0WQgwWQqRJklTf6b2ngacBysrKjtuVJLt/nl2zm2vOHkh5SB5h24FW9jU5mTI8C5PJ0OMiYgIDr66r4sGZpbg8PmwWEy9+VsEdU4apjvP6JRa+sy1iHMt/WsZfZ59JXVsHHT4/FqMBty+gVAzPGJUToeGTFhdDRnwMd73ezsaaVl6ZM07zHhNtZvY3OXl0VbmSFfTMNWWqXsPhx1c1uWhyerBbDv866YFeHZ2+RU8bg6+AQiHEIKAGuBK4OvwAIUQBsDsUQB4DWICGnhqQ7P556idj2HqgTSWPMH9yITtqWynul9Tjro76dk9IFvmbiNfDcXm1c/sb2j38euVG1djlyuJbzivCZjao3ES3nFeE0+MjNyVFuS+b2Rih9bN4Rgmbalp4ca06+Ovy+BmUFl2wLdjPQO3i0tHR6Tv0qDGQJMknhJgL/JtgaumzkiRtEULcGHp/GXApcI0Qwgu4gCskSer2ILLsGtp5sI0bJuRjMBhUTWPkFMoROcF2hz3t6oi3ajdqibeqv5LUWO3YQnlde8TYH55ZyraDbTz/2V4A5kzMZ1hWApv3t/L3Lyt58NJS1X3Vt3fw+Me7eGrWGayvbMIfCMpkhD8X+XopsRbls0NunsD2Q61IAahudjJjVA6FmXF6I3cdnT5Mj9cZSJL0LvBup9eWhf37AeCBnhyDVmZQcXaCtuZPWCP1nnR1xMeYNBu1xMWov5IWt4fbLhiiqIvKvQmWrtoVMfbth9rUUtarynn4spEsX1PBgmnFZCXGqO5ry/5WKhtclB9qx2oKpromh3SNwsc1b1IhLq9P+ezgjDgGpQWVRdPiLXpMQEfnFOC0qEDunBmUbLcQF6WFYmps13rnHi/1jg7iYkyqwG9cjIkGpzq1NNZi5qXP96qCwW0uD01OtTvJajbQeS62mg3EWkzMHp/Psk/KGTdQXcOQkRDcdWQkWHn4/e2qauj5kwvpn2xn+8E2Vqyr4oy8UtVn9ZiAjs6pxWkhVBeeGSS3jtx2oIV5kyJF2pwe35FO1W3YLUaeWVOBP2SL/AF4Zk0FNpNRdZw/4OfGiQUsX1PB4x+Vs3xNBenxVu6/eIRq7AumFTMoLVb12i3nFbHona088XE5lQ0uatvcqnMn2YwsnF7MgWYnV5TlKtdYsmoXkgQ1zUEpiSvH5pIRr8cDdHROZU6LnUFGWE7/JWP6s/SjXdwwIZ+3N9REVN8uvXL0CRmT3WLil+cVsac+mM1kMsAvzysitpObyGQ0smy1uhbgT6t28ujlo3h33gQqGxxsP9hGs9ODEPDQzFI8Pj+VjU6e/2yvEgSWex+EMzgtgf0tbvJS7dyxcmPEs7jtgqHMmZhPrMWo7Bh0dHROTU4LY2Aw+JWsGbl/78r11RHN5RfNKGFYZsIJGZPT46OurUOVzXTr+UVkdcrIaWj3RNHw72BMXgq5yXYaHB5VUdriGSWkx8UoriR51+P1q2MkJpOBCYMz2H6oNaKw7d6pw6lpduIPwJ8/qaAoK56BabpLSEfnVOW0MAZNjgAuj485E/MZnh3MlQ/X8jEaYGxeCne/uYmyvJ7vmgXgC6AoeELQQD3ywU6euaZMdVx4JbGM1WwgOyT9XNXkVCZx+Tz3vLmZ+ZMLI1b6U0qyIsZhMhkoyUlieHYiY3KTqWxw8M2+Zh77qFy1q9ClJHR0Tm1Oi5hBm8fH/f/aztJV5eypdygNXQ60uFm+pgKb2YjT69f0q/cULo//mJq9FGcncN9FJapYwH0XlVDcL5gCG61SekCKXRVnuGPKsGOSi/5+UQZDsxJUuwpdSuLYCQQkKuraWbu7noq6dgKBXiG1paNzVE6LnUGH9/DE6/D4Wbm+OkKq4Ybxg07oCjgtzqKdzRSnzmYymQxcVJpDYUYcB1vcZCVaKc5OVGQzolVKD8tK0OwtfDR0KYnjp6fFDXV0epLTwhikx6sLt5qcngiphqLM+BO6AvZLUkSvgVvOKyKgUW9nMhkoHZBM6YDI80SrlB6UFqus9ruKnjZ6fPS0uKGOTk9yWhgDWaLh0Q93snJ9NXddOJR6h4eAFOwGlmK34Pb5T+gKLjU2hr9/Wanaofz9y0rOH57ZpfPoK/neQ8+LG+ro9BynhTFIjbWy8MutzB6fz8j+iTQ6PBFZPCl2ywmdQAemxnLHlGHdon2kr+R7Bz0tbqij05OIHpAB6nHKysqkdevWHfPxPl+ANzbUcPcbm7ntgiIefn9nxB/sc9eWcfbg9J4YblRkvSR9RX9qoMcMdHo7Qoj1kiSVab13WuwMwoOw1U2uKJpE/iif7jn0Ff2phe6y0+nLnBbGAA4HYeMs2ppEg/TUSZ1uQDfwOn2V06LOIJxB6XH88bJRqrz9P142Sv/j1dHROa05bXYGMgaD4MKSLIZl61t5HR0dHZnTzhiAvpXX0dHR6cxp5ybS0dHR0YlENwY6Ojo6Orox0NHR0dHRjYGOjo6ODrox0NHR0dGhj8pRCCHqgMrQj2lA/UkcTndwKtwDnBr3cSrcA5wa93Eq3AP0rvvIkyRJU3enTxqDcIQQ66JpbfQVToV7gFPjPk6Fe4BT4z5OhXuAvnMfuptIR0dHR0c3Bjo6Ojo6p4YxePpkD6AbOBXuAU6N+zgV7gFOjfs4Fe4B+sh99PmYgY6Ojo7Od+dU2Bno6Ojo6HxHdGOgo6Ojo9P3jIEQwiiE+EYI8U7o5xQhxAdCiF2h/yef7DEeDSFEkhDiNSHEdiHENiHE2X3tPoQQtwghtgghNgshXhZCWPvCPQghnhVC1AohNoe9FnXcQog7hRDlQogdQogfnJxRq4lyDw+Ffp82CiFeF0Ikhb3X6+4BtO8j7L3bhBCSECIt7LVedx/R7kEIcXNonFuEEA+Gvd7r7kGmzxkDYD6wLezn3wCrJEkqBFaFfu7tLAHekyRpKFBK8H76zH0IIXKAeUCZJEklgBG4kr5xD88DUzq9pjluIcRwgvdVHPrMk0II44kbalSeJ/IePgBKJEkaCewE7oRefQ+gfR8IIQYA5wNVYa/11vt4nk73IIQ4F5gBjJQkqRh4OPR6b70HoI8ZAyFEf+BHwDNhL88AXgj9+wXgohM8rC4hhEgAJgLLASRJ8kiS1Ewfuw+CvTBsQggTYAf20wfuQZKk1UBjp5ejjXsG8IokSR2SJO0ByoFxJ2KcR0LrHiRJel+SJF/ox8+B/qF/98p7gKjfBcCjwK+B8OyWXnkfUe7h58AfJEnqCB1TG3q9V96DTJ8yBsCfCP6ShHe0z5Qk6QBA6P8ZJ2FcXSEfqAOeC7m7nhFCxNKH7kOSpBqCq50q4ADQIknS+/She+hEtHHnAPvCjqsOvdbbuR74V+jffeoehBDTgRpJkjZ0eqsv3UcRMEEI8YUQ4hMhxNjQ6736HvqMMRBCTAVqJUlaf7LH8h0xAWOAP0uSNBpw0DvdKVEJ+dRnAIOAfkCsEOInJ3dUPYJWL9RenYsthPgt4AP+Jr+kcVivvAchhB34LXCv1tsar/XK+yD4N54MnAXcDrwqhBD08nvoM8YA+B9guhBiL/AKMEkI8VfgkBAiGyD0/9rop+gVVAPVkiR9Efr5NYLGoS/dx3nAHkmS6iRJ8gL/BL5H37qHcKKNuxoYEHZcf4LusF6JEOKnwFTgx9LhAqK+dA+DCS4wNoT+zvsDXwshsuhb91EN/FMK8iVBT0Yavfwe+owxkCTpTkmS+kuSNJBgEOYjSZJ+ArwF/DR02E+BN0/SEI8JSZIOAvuEEENCL00GttK37qMKOEsIYQ+teCYTDIL3pXsIJ9q43wKuFELECCEGAYXAlydhfEdFCDEFuAOYLkmSM+ytPnMPkiRtkiQpQ5KkgaG/82pgTOhvps/cB/AGMAlACFEEWAiqlvbue5Akqc/9B5wDvBP6dyrBDJBdof+nnOzxHcP4RwHrgI0Ef3GS+9p9AAuB7cBm4CUgpi/cA/AywTiHl+BkM/tI4ybottgN7AAuPNnjP8I9lBP0R38b+m9Zb76HaPfR6f29QFpvvo8o34UF+Gvob+NrYFJvvgf5P12OQkdHR0en77iJdHR0dHR6Dt0Y6Ojo6OjoxkBHR0dHRzcGOjo6OjroxkBHR0dHB90Y6Ojo6OigGwOdPo4QYl5IBrxGCPH4yR6Pjk5fRTcGOn2dXwA/JFjM850JqbCe8M/q6JxsdGOg02cRQiwjqAL7FsEqbvn1PCHEqlCjl1VCiNyjvP68EOIRIcTHwANRrjVOCPFZSGn2M1lORAhxrRDiH0KIt4H3hRCxoYYnX4WOnRE6buD/b+9uQqKKwjCO/59MEKKMooWbcuNSsAiioBKK9iWtKhzb9rWoXRshKLBVILUQwnAVEQS2iQLFsg/6Ei2iFu0ipF0Q5ELfFucIl2rmGhrW+Pzg4LnHufeemcE5c64zz5H0UNKrXHbVuF8tksYkTSgtHrQ7t/dI+pCTMAfmZ0K5/9ckjUj6KGlv7sM7SYOLf6RtRVjur0C7uCymkCMLgArQn9uGge5cPw7cKWkfBO4CDTXOsw5Ynev7gdu5XiHFEGzI2xeBo7m+nrTQzBrSmg9Nub0NeFHjXGeB87neAKwFWkiZUJtIcQfjhfs7SApvFClN9ivQTnqz9xLoWO7nyeXfL57WWj3aCRzK9SGgr6Qd4FZEzNY4ZjNwQ1IbKXa4sfC7+xExv8DJAVK67rm83QRsJqVT9kvqAGZJmffVPAeuS2okDVgTkvYBoxHxBUDSzZ+OMRwRIWkKmI6IqXy7t0ArKa/IrCoPBrYSVAvgKrZ/KznGBWAkIg5KagVGq+wroCsi3hd3ltQLTJOWOV0FfK/a2YgxSXtIq/oNSbpMerdfK0hsJv+cK9Tnt/13bqX8PwOrR49JMecAR4BHJe0L0Qx8yvVKjdvdA07laG8kbS3s/zki5oBjpMs/vyVpC2khpwHS8qjbgGdAp6SNecZw+A/6blbKg4HVo9NAj6RJ0gvvmZL2hegDLkkap8YLOWkG0QhMSnqTtwGuAt2SnpIu79SaiXQCE5JeA13AlUjLcfYCT4AHpGhksyXjCGuz/5CkCrA9Ik4ud1+sPnhmYGZmnhmYFUnq4dfLR+MRceIvnKud9KmmopmI2LHU5zIr48HAzMx8mcjMzDwYmJkZHgzMzAwPBmZmBvwA32ibYwo7suoAAAAASUVORK5CYII=",
120 | "text/plain": [
121 | ""
122 | ]
123 | },
124 | "metadata": {
125 | "needs_background": "light"
126 | },
127 | "output_type": "display_data"
128 | }
129 | ],
130 | "source": [
131 | "file_url = 'https://www.dropbox.com/s/jz8ck0obu9u1rng/resale-flat-prices-based-on-registration-date-from-jan-2017-onwards.csv?raw=1'\n",
132 | "df = pd.read_csv(file_url)\n",
133 | "df_tampines = df.loc[df['town'] == 'TAMPINES',:]\n",
134 | "sns.scatterplot(y='resale_price', x='floor_area_sqm', data=df_tampines)"
135 | ]
136 | },
137 | {
138 | "attachments": {},
139 | "cell_type": "markdown",
140 | "metadata": {},
141 | "source": [
142 | "Notice that the resale price increases as the floor area increases. So we can make a hypothesis by creating a straight line equation that predicts the resale price given the floor area data. The figure below shows the plot of a straight line and the existing data together."
143 | ]
144 | },
145 | {
146 | "cell_type": "code",
147 | "execution_count": 3,
148 | "metadata": {},
149 | "outputs": [
150 | {
151 | "data": {
152 | "text/plain": [
153 | ""
154 | ]
155 | },
156 | "execution_count": 3,
157 | "metadata": {},
158 | "output_type": "execute_result"
159 | },
160 | {
161 | "data": {
162 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAESCAYAAAAfXrn0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAACdTUlEQVR4nOydeXhU1fnHP2f2mewrCQkJhCQsAcISEC2gggv6Y1EBtyouWGoVodqqra1S0Fq3ouJSRBHFtoqKVqXUqqCiFZegguwJgYSwZCPr7Mv9/TGZy9zMDRBIWPR+nsdHMnPn3nPvJOc9512+r5AkCQ0NDQ2Nnza6kz0ADQ0NDY2Tj2YMNDQ0NDQ0Y6ChoaGhoRkDDQ0NDQ00Y6ChoaGhgWYMNDQ0NDQ4jY2BEOJFIUS1EGLTUR5/uRBiixBisxDin109Pg0NDY3TCXG61hkIIcYALcAySZIGHOHYPOB1YKwkSfVCiFRJkqpPxDg1NDQ0TgdO252BJElrgYPhrwkhegsh3hdCrBdCfCaE6Nv61i+AZyRJqm/9rGYINDQ0NMI4bY1BOywGbpMkaRjwW+DZ1tfzgXwhxP+EEF8KIcaftBFqaGhonIIYTvYAOgshRDRwFvCGECL0srn1/wYgDzgHyAQ+E0IMkCSp4QQPU0NDQ+OU5EdjDAjuchokSRqs8l4l8KUkSV5glxBiO0Hj8M0JHJ+GhobGKcuPxk0kSVITwYl+GoAIUtj69r+Ac1tfTyboNio7GePU0NDQOBU5bY2BEOJVYB3QRwhRKYSYAfwcmCGE2ABsBia3Hv5foE4IsQX4GLhTkqS6kzFuDQ0NjVOR0za1VENDQ0Oj8zhtdwYaGhoaGp3HaRlATk5Olnr27Hmyh6GhoaFxWrF+/fpaSZJS1N47LY1Bz549KS4uPtnD0NDQ0DitEEKUt/ee5ibS0NDQ0NCMgYaGhoZGFxuDIymLttYCLBRClAohNgohhnbleDQ0NDQ01OnqncFLwOF0gC4iWAmcB8wE/tbF49HQ0NDQUKFLjYGasmgbJhOUoJYkSfoSiBdCpHflmDQ0NDQ0IjnZ2UQZwJ6wnytbX9t/coajoaGhcfIJBCR219mpanLRLdZCz6QodDpx5A8eByfbGKjdnWpJtBBiJkFXEllZWV05Jg0NDY2TRiAg8f7mA9zx+ve4vAEsRh0LLh/M+IK0LjUIJzubqBLoEfZzJrBP7UBJkhZLklQkSVJRSopqzYSGhobGac/uOrtsCABc3gB3vP49u+vsXXrdk20M3gWmt2YVjQQaJUnSXEQaGho/WaqaXLIhCOHyBqhudnXpdbvUTdSqLHoOkCyEqATmAkYASZIWAauAi4FSwAHc0JXj0dDQ0DjV6RZrwWLUKQyCxagjNcbSpdftUmMgSdJVR3hfAm7tyjFoaGhonE70TIpiweWDI2IGPZOiuvS6JzuArKGhoaERhk4nGF+QRt/Zo6ludpEa89PIJtLQ0NDQaINOJ8hJiSYnJfrEXfOEXUlDQ0ND45RFMwYaGhoaGpox0NDQ0NDQjIGGhoaGBpox0NDQ0NBAMwYaGhoaGmjGQENDQ0MDzRhoaGhoaKAZAw0NDQ0NNGOgoaGhoYFmDDQ0NDQ00IyBhoaGhgaaMdDQ0NDQQDMGGhoaGhpoxkBDQ0NDA80YaGhoaGhwAoyBEGK8EGK7EKJUCPE7lfcThBBvCyE2CiG+FkIM6OoxaWhoaGgo6VJjIITQA88AFwH9gauEEP3bHHYP8L0kSYOA6cCTXTkmDY0TQSAgUVbTwrqdtZTVtBAISCd7SBoah6Wr216OAEolSSoDEEK8BkwGtoQd0x/4C4AkSduEED2FEN0kSarq4rFpaHQJgYDE+5sPRDQ0H1+Q1uV9bDWOjkBAYnednaomF91iT0yP4VOdrnYTZQB7wn6ubH0tnA3AZQBCiBFANpDZ9kRCiJlCiGIhRHFNTU0XDVdD4/jZXWeXDQGAyxvgjte/Z3ed/SSPTAMOGeuLF37GVc9/xcULP+P9zQd+8ru3rjYGaqa27RN/CEgQQnwP3AZ8B/giPiRJiyVJKpIkqSglJaXTB6qh0VlUNblkQxDC5Q1Q1eQ6SSPSCKcrjfXp7B7sajdRJdAj7OdMYF/4AZIkNQE3AAghBLCr9T8NjdMSm8mAxahTGASLUYfNpD+Jo9II0Z6xrm52kZMSfcznPd3dg129M/gGyBNC9BJCmIArgXfDDxBCxLe+B3ATsLbVQGhonJZ4/H5mj83DYgz+eVmMOmaPzcPrDxzhkxongm6xFvm7CWEx6kiNsRzXeU9392CX7gwkSfIJIWYB/wX0wIuSJG0WQtzc+v4ioB+wTAjhJxhYntGVY9LQ6GqSoswsL65gxqgchABJguXFFYwfkHayh6YB9EyKYsHlgyNW8D2Too7rvF214zhRdLWbCEmSVgGr2ry2KOzf64C8rh6HhsaJomdSFHeP79fpk41G56DTCcYXpNF39miqm12kxnRONlFox9HWPXi8O44ThZCk0yfAEaKoqEgqLi4+2cPQ0GiXUOpidbOLtFgL/gBUN2tpjD9mOjNm0FWpr0KI9ZIkFam91+U7Aw2NnyI6nSAnJZqeSVGndVDxdOZE1xJ01o4jEJBYs72KjZWNBCTQCxiYGcfYPt26dPyaMdDQ6ELaCyr2nT36tPAjny60nfizEmx8sLXqhBvh0CLgeL7bioN2SqpaWLy2TB77nHF55KZE0zO5635nNGOgodGFnO5BxdMBNffMw1MG8cq6XXIQH+Dh97fSNy3mlH/uVU1unlxdolhAPLm6hKFZCZox0NA4XTndg4qnA2q7rwUfbmfmmN7cv3KLbCBmj83joN0dYQxONWkKu8enuoBweCJqcTsVzRhoaHQhXZXGqHEItd3XhEEZsiGA4GS6cE0Jy38xkrKaFursbkx6HXa3H39A4o/v/EB5nfOUiOn0iLepLiAy421del3NGGhodAHhq83+6TH8+7bR1LR0XhqjxiHUdl96Haqr68pGJ48s/44rirJYuKZEsWt45cty9je6TnpMx+7xccf5+Sz4cIc8vjvOz8fexTsDrbmNhkYn01YIbfyTn7G9qpmirEQAvtpVd9rp1pzKhHZf4RXf/dJiVauMtx9oZsKgDNkQwKFdw2VDM+Wfq5vb15Hqav2hygYnKzfs45GphTw8ZSCPTi1k5YZ97G1wdup12qLtDDQ0Opn2MogWX1vEzFeKtRTTTiaU0pl0wwg+K61FkuBvn5Qye2yeYvU/f/IA/r5uN2P6pKruGkKB5sPFdE6E/lCPeCsXDUznrjc3KLKJMuKtnXL+9tCMgYZGJ1PV5OLMXolcP6oX9XYviVFGln6+i+Lyg1qKaSegFvCF4CSuExAAalo8vPJlOTPH5JARZ6Wi3skzH5cweXAGAQlVn7zU+np4TKfttSSJLk8VtpkNqtlE5/Xr1innbw/NGGhodDLd4y2MH5jOL19ZL6/sHrhkIFEmHbPG5rJifSX7G10RKaanWlbLqYjayvzpq4fg9kr85o1Dr82dWECz04PZoOeJ1SXsbwy6fXqnRPPvjZXMm1TA3Hc3K76f7CQrU4ZmyM9d7VqPTS3s8lTh2ha36jXq7G7yiOmUa6ihxQw0NDqZ6ia3PNFA8A/5j//6gU37mnnhszKuHZlNepxF4Y7QGq4cHWouuJKqFtkQhF6b995m/BK4fMrV/74GJz8f2YtnPyllxqgcZo3NZcaoHJ5aswOTXkdOSrRsgFWvVd3cJYqn4aTGqKuqpkR3bTqyZgw0NDqRQEDiQJP6yk6IQ8HKaUWZCnfE6S5/3NWEgrY7qpojnm2izaT6vDMTbPzz63IuG5qJxajjwUsH8p8f9uP3S0wZ2oO+aTFEm/UIAR6fxIFGZdC4qslFgs3ErefmMmts8L+Pt1Xz4KUDFcHqzk4V1utgzjilBPqccXnou3i21txEGhqdRGh1nxxtatcnDcGJakiPeM7OT5VXoe1VKpfX2X/y7qJwd81No3Minm2URb2ZUJRJz41n9aJ7gpVVs0ej10FVUzq3/PNbRUrpexv2Mv3MbDISlAHaGIuB6Wdmy/770KTcLz2aVR3QH+qo+29/o4tl68oVEujL1pUzJCu+SyuQtZ2BhkYnsavWzsPvb6XyoIP7Jw+IaG7z2Y5qbj03l9njckmMMis+217Dle/2NPzk3UXhu6YV6ysjGgcl2oyqK+nS6mYcXj/psUH3yoHGSJmHhWtKmDAogydXl9Di9itSRR0ev2ogt8XlJyclmpE5yQq3khrH4v7rFmuh3uHhmY9LeXpNKc98XEq9w9PlVevazkBDo5PY1+jgiqIs7n13Mwk2EzPH5JCbEk2czchTq3cwfkC6nOq4eG2ZIiVRrVI5VAhV7/D8pLOOQu6ay4ZmIgTodDDr3FzyUqPJ6xZDo9ODzahn5pgcDDodOclRBJDYc9DB68V76N89lg+3VqHX6Q7rvvt610Ge/aRU/l5a3D7FdQFWrK+kxX30xV/HIlR4sqrWNWOgodFJmPV6ebLf3+hi4epSeZU6/awcOW8cDk0KSTeMICXGTM+kqIhc+VBFLAQnxJ+qMUiPs/Crs3OotXsISOD0+EmKMtEnLYaeydGU1bTw4he7eGDyACrqnfw2LD9/9tg8GuwenltbxuOthWntpZTmpkQpJuteSVGqbqKOTMrHIlTYVc13jkSXGwMhxHjgSYJtL1+QJOmhNu/HAX8HslrH85gkSUu7elwaGp1NewJjdo9fNfDp8gb4rLSWFz47tEuwGPW88FlZxIRlM+lPyD2cigQCEnaPP0LSOeRqCXWWc3oDqnpEi64ZxqNTC3n7uwrundCf+1du4cxeidwwqhcNTi8JNhN9Um1yj+rQZJ0SbVF1E13Q/+jblx6rUGFnSGF3lC6NGQgh9MAzwEVAf+AqIUT/NofdCmyRJKkQOAf4qxDC1JXj0tDoCrISo1T9/rqwyta270mSMnPI4/dH+MRnj83D41cakp8S+5tcqpPy/qbgrim0ktYJdT2i4vJ67nxzA+f2Sceohwcm9+eigd2Z+cp6Zr/6Pb9YVozTJ5EUE5x2QpN1dbP6qr6mpX2piraoSWWcqkKFXR1AHgGUSpJUJkmSB3gNmNzmGAmIEUIIIBo4CHStIpOGRheQnWjjgUuUgeO5EwvIjLPw3oa9EZP83AkFfLajGji0GjXp9SwvrlDkwC8vrsDU1XmFpzDNLvUdV4vLL/+s0wmizPrDGtx5KzeTFmslMzGa+97dpDAuc9/djF7oFJN1e0H9jgRyQ4Zq1ezRvDbzDFbNHn3KSpB0tZsoA9gT9nMlcEabY54G3gX2ATHAFZIkRSyDhBAzgZkAWVlZXTJYDY3joaLewVNrSpgxKocYi57u8TYq6uz0To3m3gn9MeoEL98wgn0NTgx6HS9/Ucbo/FQ27m2SJ5k6u1tVUdP7E94ZdI+3qrpa0uOUGVkGfdB9FO7jDwXhITjpNzi8CKFuXGpbPCyfeSYF6bFyUP+v0wYrKpv/Oq3jq/qT4fI5FrraGKiZv7Y5VRcC3wNjgd7Ah0KIzyRJalJ8SJIWA4sBioqKfrp5dhqnLFVNLsrrnLz1bSU3j8mhtLqZgATb9jeRFmvGZNTzh7eVOe463SHXQVaCjWaXF4MOHp1ayK5aO25fgOXFFYwfcPR+6h8b/dNi+fOlA/nD2z/Iz+7Plw6kf3qcfEwgIOHxBfD6JR6dWkhACrCzxq4IwluMOuJsRqxGvapxibUauGLxOjl+EwhIBKQAM8fkEJAI6h5JAQIB6ZRc2R8vXW0MKoEeYT9nEtwBhHMD8JAkSRJQKoTYBfQFvu7isWlodCo2U7D4afqZ2Ti8kQFPfZi7IxTcXHr9cP4x4wwK0mIjevbOHpvHio17uXt8v1PSx3yi0OkEsRaDYlKOtRjkCVlNQ+j28/JJspmod3iAQ265p1bv4PcX92H+pALuC9Mmmj+pgPW7auX4Tf85o6lt8VBS3UJAQtaTshh1ZCXaKOyRcEz3El6AZjMZ8Pj9JEWZT4nCwq42Bt8AeUKIXsBe4Erg6jbHVADjgM+EEN2APkBZF49LQ6PTaXJ5mD02jx4JNjm9EQ4FPB+dWqg43uUN8M3uep79pJQ/XzqQhat3RHbmmjmSgRnxJ32iOBJdKbJXVtPCba9+F7GSXzlrFLndYlRz+R//aAdzxuXxt2uGcbDFjdGg54W1O9m4twmHGz7aup/nrh1Gg8NLvM3IP77cxTUje8HHu0iwmfi2ooF7wnYi4c1vDjS6KOzR3mgP/4zaGq3ZY/NYXlzB3eP7nfRYQpcaA0mSfEKIWcB/CaaWvihJ0mYhxM2t7y8C7gdeEkL8QNCtdLckSbVdOS4Nja7A2Br8/e0FfVV90s42naosRh1uXwCXN8Af3v6BGaNyeObjUuVnvP7TwhB0pcb/rjq7avHXrjo7ud1i2s3lz0q04fcH2FXnAIKy1gD1Dg8fbKnlgy3KaWZiYbC5zbSiTNkQhM61sDUWtOTzMtJij60SWM1ohc57KsiZd3mdgSRJq4BVbV5bFPbvfcAFXT0ODY2uxmrQceXwLPbWO1R90gm2Q5pFasHNtglDna2G2VUcS5VtR2hPIyjGEpy+wnP50+MsXDY0E70OkqJNPPbfbRSXNyqed2qsWfX7SY4Ovp6fGqNqXPQ6mDepgPgo4zHdR3tGK1QB3Zky2MfCTzdfTUOjk6lzuLEZ9egEzJ1YoEgjfeCSAbS4PSy9fjjPXTOUmWNyIoKbQ7MSTot89LYcrsq2MzDqdbz2zaF025tG5/DaNxUYW61nVkIwpTc7ycq1I7NZ8nkZC1eXcv3Sb5gyNItBGbEKtViLQRfx/cydWIDNqGfV7NH0S1dvmTkyJ4nVW/dTWX9s7SfbS1UNVUCfbMOvyVFoaHQSSVEWnv24lOln5SBJAZ67dhjbDzQzMCOON76pYHivZG546RsSbCamn5mtCG7OHptHYpSR5TNHsr/RRXqcVU5xPNU51irbo8UfkFTTbQOtMrChlN5HpxQyfenXyr4GKzfzyNRCZrfGHLISbRxocvPqV+U8MrUQp8eH1WTghbU7ufmc3lyclUAgIKnqRP3+rY1cUZR1zGm+7elPLS+uOCUMv2YMNDQ6CYsRphVlKXrXzp9UwBvfVDB5aCbry+u5aXQOK9ZXsmydsiXjmm0HyEiwcveKjV3id+9KulpYzWzQqTawXz5zJHAopbeqWb2PRChWE5S1NrB1fxONLi/bDzTLMYhGV1CWAg4ViiXfOILPS2vxBw7pRC1cU8IrN444pvsI1xwKZhPp8foDjB+Q9pPIJtLQ+MlQ3eyN6HD2zCelzB6Xx/ryegIS6AXcPCaHRWuDroxZY3NZ8nkZi68tYuYrxV3md+9KulpYzeHxq07yHn+AspoWnF4/c8blkhmvvkOJtRixGHXcN6E/Sz7fyf8N7M6t5+TJVcgWo455kwqwmQ65cHQ6QYPDw8LVpRHXrXd4j/leTuUCNM0YaGh0Ei7voUlrUEYsN43pDZJElNnAwu9LKK9zYjHquOP8fDkgemZOIpcNyTjtm9t05STXLdZCUXYc08/Kwen2YTMbeG/DHvY1uLh2ydfyhN4zKYp7LurLg//Zpgg0p8aaWHr9cOodHm4a3ZtdtXZeWlciN48BgtLV0wbL1wwEJBKjzMwelxtRZ9At1qw+0NMczRhoaHQS3WItZCdZmfGzXpiNeoW7KDxPfcGHO3ji8sHcP3mArEYa6nvbdlX73Z4GnN7AaeEu6iq6x1i4vChb8TyfvXqo3LEMgobz92//wKxzcxUdwv7zw35SYsz88V+HdgGPTB2kGoNw+YLupMPVA9w2No+CsMrnHxNaNpGGRichBSR+dXawi1lbd9HCNSVcNjRT/lkIwfJvyrn02XVcvPAzKhvszG/THW3uhAI+2Vb9k++FvPlAU4Sw3PeVDe26jsI7hJ3TN1U2BKFjdAjVGIRJHzTM7dUD3D95ACkxph+tUdaMgYZGJ+H0+fnTe5uxu9WF0LISrcwam0t2khWTQbC3wS2/t768gWc+LlGolS5aW8ro/FRc3gBVTZ2Tpnk6ckDFhRaQ1CXB+6bFKgxqjwRbxGfLau2q308oFtCey664vJ4t+5qpONi+YQ4EJMpqWli3s1bRQvN0QHMTaWh0Eo1OLy5vAJtZvUF7xUEnSz4vY+7EAvbWO/jNBfkYdQKdTke93U15nVNRgQzBgqufenObUDFY+PN8b8NeHrx0oEIyYu6E/rz4+U6Fm6iuxR3xWX8goPr9hIxIatj1wovYBnSP5c+rtjI0K0G1MX1XV2J3NdrOQEOjk7C1qmE+v3Ynf2pT1HT7efm89W1lMPf9vc24fAEqDjrw+CVKq5vJ7RZDdpKVW8/NZdbYXHkH0T895icvYe0L+Jg7Qfk8bx6TS0aCmVWzR7P0+iJmjsnB6w8wrl8aSz4v4+k1pSz5vAyrUc/t5+UrPpvfLSai6GzOuDxizMG1sdPnZ864vIgittmvfc8VRVk4vertVtqrxD5dXHzazkBDo5NIjDIxZ1wer31TgdevlD62GA6tu1zeAPE2E499sENeQQ7MjOPmMbnMW7k5bKVbgEEnsby4ggsLfroS1jqdjhXfViiKxJZ9UcZvLuhDTko0PZOicHoDbDvQxBvFlcwYlYPZoCM3NZqH39+Kxycxc0wOWQk2mlxeUmKN+ALK76d7vAW3PzjJV9Y7WbaunN9f3C+ib/XCNSW8dMNw1XEeS7/jUwnNGGhodBYCkqKMPHTZIG546ZsIN0RIiC7oMnIoJhmXJyAbgtBr81ZuZtkNI7hyeFaEbtFPiRizkSlDM+X+EHoBU4ZmEmMJagTJRWLRJt75fi8AHn+AXbUt/OHi/mza14g/AH/9cAf7G108fdUQXlm3O5iqqjAufQFIjwvGdJztxH5aXOo7g66uxCbghT1vQ/IZEJXdOecMQzMGGieMrpQ5PhWobXYjhJCbrLdV2RTikA7OwtUlis+2F9SstbtZtq6cIVnxqn7qnwL5KTGU1dqhyS2/FmUxkp8SI/+s0wn8AT+zx+UrmuD85dKB2Ex6WtyHWmSW1dopLm+kuPw7xXWaWyf5ft1iuPXcvHYFB9urrO6ySmxXNZQuhpJF4NwLA+fDwHuP75wqaMZA44RwugfXjoZ4m4l9jW721NlVVTZzUqKZOSYHs0FgMijvub2gZnK0mXqH56SLmJ0sAgGJHTXNNNg9itcb7B72NTkVBtJiMPCHt9dH1B784aJ+PPFRiVzr0d6zTmstJtvT4OSZj0u4cngW907oz/0rt8jf4/2TB5CVYFMda6dXYtd9A9ufgorlEPBA+oUw4jnoftGxne8IaMZA44TQ1TLHpwIOj595721m1rm5PP1xqeJen1xdwqxzc1m4OugmeuKKwfx6+SHDmJsaze3n5fP4R4fiCLPH5uH0+E4JEbOTQWgBYdAJ7J7IznF1LR56Jh86vtbuUd1ddYuzyP7+mWNySIoy8dsL+vDYB9vl891xfj5RZgNlNS3sqbdzRVEWT64uIcFmkuMNUWYDD72/lfxu0e12OjvuSmy/B/a8CdsXQt1XYIiG3JmQPwti+xzbOY8SzRhonBBO9+Da0XCwdTJytTasCSf0eujfO6pamDkmh/7psfywt4nUGDMvf7FLDpLaTAZe/qKMQZlxjC9I/dHsnkKouQwBxWs6AXe8/j3PXj1U3mXBIeP6wvQi+Xw+X0DWIGq74o+3Gbn13FyEgKLsBCrq7KREm3hsaiEBScKo1/G3T0pJj7Nw55sbWXr9cLkobX+jSzbgj0wtpLzOedhOZ8fsCnXuD7qBSp8DVxXE5MOwhZBzHRhjj/1Bd4CjNgZCCBvwGyBLkqRfCCHygD6SJK3sstFp/Gjo8uDaKUC31qYpgOq9tiouYzHq0AlYuLqUe/+vH898XMrYvsn8fGRPRZD05yN7YjXpjtsQnGqxGjWX4dNXD8HjkxSvPXjpQBJsJuraWfE7PME4gM8X4F8b9pIcbWLuxIJg6m7rOf40sYB6h4clnx/aVcyfPIBYmwG7K8DuOgf+QICpwzLZ3+DE5Q1Q04766e5ae9CdFKf+O9thV6gkQe062PEUVLwJkh+6Xwz5t0H6+SBObNZAR3YGS4H1wJmtP1cCbwCHNQZCiPHAkwTbXr4gSdJDbd6/E/h52Hj6ASmSJB3swNg0TnG6Wub4VMBs0DN3YgGLPi1l9tg8hfbNnHF5LFtXLv8bgkbhoMOLxahDL3QcaHRFuEJ6HefzORVjNWouw42VjfK9h1675+0fmDkmh+pml6pxTYwyUVbTws6aFvYcdNAz0caiTzcris7+9mkpPx+RpRCle+bjEuZNGkBJdSOvF1dS7/AwZ1yenP5r1AvV6/kCAeaMy8PQznM7aleo3wXlrwXjAfXfgjEuaADyb4GY3E57zh2lI8agtyRJVwghrgKQJMkphDjsb5MQQg88A5xP0Hh8I4R4V5KkLaFjJEl6FHi09fiJwO2aIfjx0dUyx6cClQ1OXv2qvDVFUeLF64dT1+KmvLUH75RhmUgSLFtXzrSiTB68dCBRZj1Lrx9Os8un6goZkHF8omgnKlbTkd2HmsswIKGagZXfLYZH/7stwsf/2wv6cNDu4ZolX8mvFXSPjajiTo+zEGUxsuCjQ4b53gn9aWxtLPTbC/OpaXbz6tcV3NmaWvq3T3ZG7DDmTyogzmbimTUl5CRHUZARf1T3pXCF2vdAyd9g5/PgroW4/jD8b9DzGjCefFdpR4yBRwhhBSQAIURvwH34jzACKJUkqaz1M68Bk4Et7Rx/FfBqB8akcRpxKmu5dwbRZgM7qluY/eqhlMU543J5LmzFC8FV5vCeifzxXz/IstaPXz5YdSKxu9Vz2o+WExGr6ejuQ81lGGvWq2ZgFaTH8uQVQ9hZ26IoEjPpBbtrWxRGLtoSKQMyrShTzgYKHXf/yi1yc/s54/LQC7iiKAuLKShJsXFvE6b1FSy7cQQVBx1UHHTw+Ecl8g4iI8Gq+hzUXaGCLG8xfLYEKt8GJMiYFNwJdDsXDr+ePqF0xBjMBd4Heggh/gH8DLj+CJ/JAPaE/VwJnKF2YGtMYjwwq533ZwIzAbKysjowbI2Tic8XYPP+RkUrR4Phx1lBlWgzMmdcnpyFMq0ok+ykKBZdM4y5726SJ/4/XzKQJz/azoRBGfJckBxjIjvJqnjtvQ17SYoyHfG6h1uVn4hYTUd3H2ouw/4Zcdz0cnHEzuiC/mlEmw384e1NEfdw94V95MBwtFmPRa+PSAXNSowUqnN5DzWhf3J1Cc9dO4z73tnEI1MKWTV7NCVVLWze38iu2hbmvrtF8fknV5dwXr9uqs8h/L7wOZiWvJY7e31E7PotYEqEvr+BvF9BdM/jfOJdw1EbA0mSPhRCfAuMBAQwR5Kk2iN8TM3stSfjNxH4X3suIkmSFgOLAYqKik4fKcCfMKHAXriW/AOXDOCSwowfpUHIS4mhssHJwiuH0ODwcN+7mxX33ejw0ujykhZnYmzfNEVM4We5CdxyTq4sfR3qviVxeE2iI63KT0Ss5lh2H1FmHY9NLcTu9hFlMbSr9FrV5MLli+x0lmAzYTLqWfL5drmn9NVLvlKkgh5oclHVpB5vCAXzXd4A68vruaIoC5fXR05KErvr7CxcXcrDlw1UHVNlvYO8bjG0RacTjM92ctb494muXIbB34BkKYTCFyD7KjCo1yecKnQkm+hSYI0kSf9u/TleCHGJJEn/OszHKoHwJKxMYF87x16J5iL6UbF5f2OElvwf/7WJvNT287SPl5OZObOvyUl5nYMWty8iGPrHf21i6fXDsZn0cs57+Ps+v4jogTD33c1H7Le7u87Ow+9vVQRIH35/K33TYshJiT4hsZqO7j4qDtrZfqCFBR8eqqlYcl2R6jl0OsGGPQ2Hdf9cNjRTdi+Fp4LOGJXDyo2R6qah4rPQNfwBWLimhL/PCDotshOjyE6ykhobqZYaVJBtM21KElStge0L0e19j3ihgx6XQf5tiJRRneYK6urf7Q65iSRJejv0gyRJDUKIucC/DvOZb4A8IUQvYC/BCf/qtgcJIeKAs4FrOjAejVOc/Y0u1aDg4fK0j4eTnTlT1eRmwYc7uGl0jrq0RIuHO9/cwCNTBkW8397quqbl8GG5OrtbtWvXQbtbXpWrxWo6c2Lp6O6jutktG4LQfW7e2xiRgTV7bB7VTU721NmZP6lAsdPqnRwtfz7k8mn77KxGHXeP70cgIPHY1EKEAKtJz7z3NsstLP80sYB/flWOyxugvrXKOTvRxm1j8/AGJNntFx7HiArJiXtbYPcrsONpaNwC5hQouAfybgZb5jE9y/Y4Eb/bHTEGavv6w35ekiSfEGIW8F+CqaUvSpK0WQhxc+v7i1oPvRT4QJKk00PrVeOoyEywKoKC2UlW7pvQHwlYv/sgA7vHYepEnf6TXeXc5PLK11aXljDh8gbaSV3Uq2roJ0eb8fkC7brVTHqdateu5TNHtjvOzp5YOrr7CPV9CKfJ7ee9DXsVaaHLiyv4zQV9mTw0k/ve2SS/ZzHoMLf2Hzjc8y7IiOW7iga5bqNnchT2Zi+TB2fIgWiPz89FA9PZUd1CQmt8pqLewR//tYlfn5eHzahXBK5tRj2u+m1Q9RqULQVvEyQOg5EvQ/bloO+aupkT8bvdEWNQLIRYQDBVVAJuI1h3cFgkSVoFrGrz2qI2P78EvNSBsWicBpgNetkQpMdZuKIoi9vDJqD5kwdwyaDunWYQqprUdyInqso5zhqsgF2xvjJilTt/0gA+2LSPW8/NZV+Di0enDqKkukWeqFJjjDxwyQCeWlOiWOkvXlvG/ZMHcOlg9TiLwxPpTw8vyFKjKycW6SiieWqVwu9t2MttY/P5479+UDyzZV+Ucc3IXnLKaHqchWtHZvOX/2yVn/GK9ZURK/hgAyGnom7jjvPzsbW6hcwGHb2So9jf4KCgeyy/H98XqTWcGdqltbj9vPp1RTCoT4Bs18f02vM8Qy3fgzBA1rRgVlDyyC7PCjoRWWEdMQa3AfcCywkGhj8Abu2UUWj8KNkTJtN82dDMiBXsfe9sIic5iqKeiZ1yvfQ4i2p6YlrsialyTo0xc8f5+Sz4cAevfFnOzDE59E6JJjPeil4HRoNgd62dd77fy4TC7oqJ6uz8FPQ6wZ0X9OW3bTT0730nGGcZnBUZZ+kWa1HNQup2mHvu7Iml46ml5ojJ+8rhWQzMiGbZjSOoanKRFG2myeFiXL80EqMOGY/w36NXvixnxqgc9Doo7BHHs1cPYcv+ZrKSokiONnLjS8rspH98Vc5tY/NY8vk2hSvq3nc28fMzsrEa9ZTVtCAhyUZ9xohEGjf+hSsTVpFhqsFpTaA+5x4SBs0Ca3qHn9WxciKywjqSTWQHftdpV9b40WMJ28q359etbu683r7+AKqFWxf0PzGNYbISo8hJiZLdCgBCCrCjukVRwBTuXw+N0+UN8Pu3fmg33rC3walqDLISgv7tthlb7SlrQudPLB3daWQlRpHXLVrhfomzGtiwp5HKBldwt1TdQq/kKFJiTNhMOjlmEP57tL/RFVZglsvw7AT0OsFdb27giSsi6zYmDMqISGhYuCbYd3rBhzsozBzO1EXrSLCZeGicgby6p8iv+i/Gbi6azH3ZmHIPNUmXcW6/HsFBn0BORFbYEY2BEOIJSZJ+LYR4D5W0UEmSJnXaaDR+VBgMgnsu6kut3UNGnJU543J5vbiS/Y1BAxCSaO4sqpvbC8K66J3a9W4inU5wTl4qsWYjFfUOUmLM6ITgrmXKFeq2A00R4zwYpr/TnvSCGiH/dtvMpaFZCe2u8jt7YumMnUas1ci+hkg5jnP6JNPg8GF3+1h6/XAkSeK9DZE7obzUGLbub+I/P+znkamFSBIRv296nfqCJGRgqpscPDyihD4Hn6Pvvi14JT3/c45kb/oczGkj6ZkUxejucSelav5EZIUdzc7gldb/P9ZpV9X4SWAzGLCaDCz+zzbFH/iydeXUOzzMHptHk8vbaddLjVFf8aZEnxg3USAg8cHWKsUk+7BK5lBAipzw42ztxxvC+/W25VgmYp1OcEG/biyfOVJRDHisE0tHdxq76+zM+ud3iuNnj8uNSMd97ZsKUmMtimY191zUlzvOz+fuFYdemzepgPc27OGiAd2ZPCRDblXZ9vetX3qs6jhN/kbuSHuNc7beSIxUjduSzKKqy3nXOY0JIwa3fhcbO7VO5liyubq6gv+IxkCSpPWtGkO/kCRJS/3UOGrcvqC/u63b5tGphWw70Mzy4goeumxQp11PJ1BNBeyqhVzbP2idICLn32zQRUxA723YG1EpazIIWQ8nFG/ISY6me7yFA40OjDr1yedYXD5qRivcxx9+X6kxFvS6oEumvUmrozuN9rSJ1Nw6IUMAwfcf/M82Zo7JUbw2993NPHftMHZWt/Dof7er/r6VVDejQ2LepAK5nmNQ9G7+2v8/ZB38AHOql1JvPp/G34kj7VJaEnxMNOkj3I5//Ncm0uMspMdZj3ll3l6M5YJ+3aiod5w0ddmjihlIkuQXQqQIIUySJHmO/AkNDdqVAt5e1cySz8uYN6kAq7Hzftl31dlZtq5ckZ64bF05fdNi6NXJqym1P+gHLx3IjWf14i/vH9oJPXLZgIgc+ZljeiMFArxwXRF6IegWa8Hj96EXwXx4u8dHlMmAw+3lzjc3cPWIbOKs6vUGx+Ly2VWr7uPvc9toeiVHRdxX+OpaLTDcUReGmgHTi8jdUntunUAbZ7XLG6Cq1Vi15wbKT43hmY930uJy8eoFu8k48Cyp3k34XSaq4sbzcMVk+vQ7u3UXsE0WtEuwmWQ3U+h868oO8sJnZcecjttejGXxtUXMfKX4qILwXUFHsol2A/8TQrwLyPUAkiQt6OxBafw4SI4xqa5ai7ITGN5zGP/4chc5yZ0n2RtlMlDv8ChUK1UrRjsBtT/okORy+GultQ7e+X4vj0wtpLS6GX8Anl5TKhc9rWoNsr6/aT9//7KCP/5ffz7fGTzurW+D/u7HP9rBSzcMVx3HsfiSyw+q91uuOGhHtDaUabu6njEqh2c+Lm03MBxyYWQl2Ni8v5EPthxQ1aIKBCT0Opg/eQD3vXMo6N07NZpHpgzirhUb5deG9IhX/f2JMullTSII7rQsJgPWsFqN8OOR4N7XP+HG5He4IvUDUvc00CJSeLrmatJH3MXCz+uYMCgjItvt/pVbmDkmh4Wrlb9PknR86bjtufaKyw8edRC+K+jIX8m+1v90QKQwh4aGCm2lgOeMy+P3b/3QJTGDaIte1U0Ube68wrYQ7f1Bt121BiQor3Oyo6qZp9eURhwf8u0n2IxcNDCd7/c0KCaf0HHNziOrl7aX49/WnRVnMaqmo9pMhnbvK3Rc+Jjbnjczzsq7P+yLyGyaNLA7lY1Oqppc+PwSW/c38o+vK5gxKgezQUfP5CgONDg4KzeZf9x0Bp/uqMEfAJtJF/F9PnDJAGIsBjbva5JrNO44P59Pth7gsmGZcsJC6L08/TYG7ppBccFq9HjZbxjIH/bN4mdjZzL8DBvVzU5uG5tIRVgadPh990qOkg1MWymLjgbJQ7Tn2vO3kaE61vMfKx1JLZ0HIISIDf4oNXfZqDR+FNQ7vLz6VTmPTC3EpBfsqg1uKKcMC5bqLy+u4NGphZ12PbcnQJRJWTEaZdLjbvNH3hm09wfddkEecn+E3m/Pt68TQvZvq03UMVb1P9Uj5firvf/kFYO59Zxchetq3qQC0uLM+ANH7tKWGmMhEJBYs72KjZWN8sR7Vu9k1cymHgk2/rezVj4uK8lGnMWI1aijW6yFndXNvF5cyYKPSvjzpQNJspn4y/vbyIw3E2UKGviUGDMVBx043D5qmt0RWUcXF3Znd60ds1HPi2t3cF7UJ8xK+xe5pjJ8wsIO83ju3zWJbxszmDMuj4QoC9ct/Zp5kwoYmhVLzySbIoAdute99U65liE/NYY/r9qqyIY7lnRcNdfew1MGseDD7YrjOruO4Eh0RKiuiGC3s5jWnxuBGyVJOmIVssZPk5ykKEyG4Iym0wkkiQjtGafn+PT64ZBM9t5GF81uH1ajHrvHjz8Af/u0jD/+X7/jvkZb1P6g/3zpQFqcXsVKMtFmarf7Wbhv/6AjmFpa73Dzq7Nz+VOb1o2BgLpBO1KOv9r7ZbX2iMDo3Hc3s3LWKHJSoiPu694J/Wl2eZkzLpeBmXH0TIqi4qCdkqoWxaTcK0wvKITLG1QeDT/unov6ctUZ2YodY2jF/Ye3f+Cei/ryyNRCdAKsJh0BP9z2WjD76KmrhrDkf9sUQfrXvgk2pnG3HMCxdQGf5n1IkqERuz6Nz82/5jvL1fj0sVx4polda8tk2eqbRufw7CelPDKlEIshchcSHiv567TBBKQA9a1NcY6mnqM91Fx7WQk2jHrdSe0E2BE30YvALZIkfQYghBhF0Dh0XjqIxo+KHgk2Li/K5q43N7Dg8kJVDZ1lR1DlPBJqMtmzx+axYn2l7JdvL0f/eFD7g663e7jjje8VAewXv9jF49MG8+J1I6h3uFk+cyQOjz8iW8TW6u9OibbIkh0QfE5/em8zf2/nOR0ptVTt/USbSfUze+od5HaLke+rqsmF1y9x7zuHmvAsuHxw63XdEQZlV21LO7sloTiu1u6JSCMNFX+99W0lFpNBkR4a3vgnEAhECPP9+WfNDCu/gdSWNeiTfRTb+/F1yoPstpzHk2t24vJWYzHWMmdcHtPPzObh97fT4vKhF3DjWb1ocHjwtyYbhOsfAfzh4n707x6LTsB1S79WfLdPrSk5bD3H4VBLEz3ZnQA7YgyaQ4YAQJKkz4UQmqtIo122VjVx37vBSdrjC6hOQI2O44sZqMlkhyaWUMZSvK3zA8gQ+Qe9W7Rw5fCsiNVlYrSJnsnRQPuTRozFwPxJ/aGdSu0Gp/pzOlJqqc0U2f0rSqUjWHigPXRfADe89LXCZRWSx7Z7IvsPvF5cGZEyO3tsHlVNyipztTTSUFxiWlGmHFgOvR5lPhQYNhqCwnwBn5srEtYwI+Ud8g9W4BM2tpgncteW8Wy1Z/DUiCE82UbWI+SGsxh1GHSC59YGO52lRJv5tKRG3sUCuHwB3tuwl79OG0xOSjTrdtZGtNQEOtWnf7I7AXbkr+RrIcRzBHsOSMAVwCdCiKEAkiR92wXj0ziN2d94aFWaGqOuDR9tOb6JOvwaIVzeAFmJVmaOycFi1GE1do0xaIuazEJet2iyEo+81fcF/IAOsz6yLsFi1GFtR8yvZ1IUT189ROG7D7lyADx+P78f35c6x6Ggamw7gfZuscFq8FBguLS6mT9e3J9dtS00uf3ySvqg3U12YlTEOOsdHuwub4TyaKi3cAi1NNJQvCUvNdLVFF4/sn9/Gb9NXsyUhI9IMLSwx5PKg/uuZ/C5d/Pwmn1cMSqLXWtKcLTTLMfp8TF3QgGSJMkG4rlrh/HxtmpuHpPLvJXhfY8HMCAtFlA3utlJVqxGPet21p6UuoDOpiN/JYNb/z+3zetnETQOYztjQBo/HtLjrPIfULPbr6pX7zjOmEH4NUKEVrkBCRZ8uINHphS2rsy7Fp1OMLZPN3KSo49hq6/jmU9KeGDyANWJ2qBv/xwen6TwyS+4fLA8oesQ6Nt8dm+9k7xUpdHKSgz6vn2+QERB2r0T+hMAnB5/0IVi1NMrWT1mYhQS3eP1cgez7ITerZ8JfkfZSVZG5iSRnRSlaDgzd2IBzU4PUSo7GbNBR2/pW1YNeoaetf+D5ABf2wt4rmYKnzQPw2w0cH5iCo9OTaGuxcNjUwuJt0Uqo1qMOmIsRh56fysPTB4IHFJ4HZ2fKhuC0Ov3vbuJop4J5FiiI2JE2UlWbhubxxWLv1QN3J+OdCSb6NzDvS+EuE6SpJePf0gaPxbibQY5tVSSJJYXV0SsGv96nNlEeck25k8aILujQpkxb63fw7pdB5k9Ng+vv305587mWLf69Q4vVxRlsXV/k2pGlEGoVyCrBYgffn8rOgFb9jeRmxqNToiI7BuBixG9Emmwe9lR3cwD/95KvcPD4muLIs7XtoG82xtQjZm4vD42VDYx770Nikk+LdbMzDE5DMiIw+nx87u3NnLjWb0U9+jx+ln6RTk3nJUtG0PJ5+Sq5DUUfPdrbK5S3CKKivgr2BDzS+7+wK64H48/wIyXi+Xe02fnpcgKsqHjbj8vn/krt7C/0UWzO+h2C9UtHE5IUa1jnNWolw1B6NgTXRfQ2XTm/nkOoBkDDZl9DS4+2LSf564dht3t408TC/jTe5vlYOT9kwcQOEKP38MRCEhsOtBMg8PDc9cOo8HuQafT8fIXZUw/K4ePd9SycE0J/2htZ3gqE2sxsnBNCQk2EzePyYl4PylaPQiuFiC+cngWu2rtLF5bJk/ibX3nT14xhK93HYxIpwwvfAoR8ueHPrvkuiIg0vCtK6tl0aelikyfRZ+W8tBlg7hkcAaSBP/31GfMGJUjV2mHGvnUOTzcc3E/9jY4+Gbz96w6ayU9Gt/GGGjCJfXkRedtdBs6Gy9WDjQ5mXWuhMsXkKvMpxVl4vIGSIk2kZsaw4EmF4MyYll24wh219qxmgw8v3annFQQaiY0Z1weJr2uXddVeGpn+P2u21l7WONxOtKZxuD03BtpdBlpsRZG5CTxy1fWK1wJUSY9VpOef3y5i2tG9jqmc6vlz88em8fy4gquKMpCtArsurwBDtpPfQWVUJe0/Y0uFq0tkxv0FGUn4PEH2o07qPmyeyTY5J4I7a14JSTVQG5bLaX0OAvTijLJiLMya2wuK9ZX4vGpG3C726fagtPh8Skm0NCYQo1qgsf7OSfuB/7SdxW/TPiCwEGJL1oG8bbrai4ZPx1rg4ffvKVMRQ3PGPMHYFBGLFeNyG5XqG722DwaXV6uHJ6FUSeYMSqHZevKyUmO5tIhGRGuq8Oldp6I/gInms40Bqr1j0KI8cCTBNteviBJ0kMqx5wDPAEYgVpJks7uxHFpnCSaXd6I9MM/vP2DwuUQfYxSEWrukVAW0cI1Jbx4fVC+wWLUYenE1ppdRXaiTZ5cQjr9FqOOC/qNZEBGvKofOhCQCAQkHp4yiLKaFl4vrqTe4UHXRtNH3XduINoUKd9g1AnuvLAPj/53Owk2k2qzoFB8IRyfL0CsxUhStIlHpxby/NqdbNzbxMI1JSy9fjgb9jSQ1jqBhsZw2dBMnv/4B66MfZ/rkt+jl3k/zS02NkRdya3fnUfAlsXNY3IwGgzMW/ldxHc9Z1weTq+fXslRVDW6+NU5uRFpueFSGgvXlLD42mHsrrWzaV+T/IzT48z0TA4G+gf3iFfEewDKaloixONORH+BE02X7gxa1U6fAc4HKoFvhBDvSpK0JeyYeOBZYLwkSRVCiNROHJPGSWTfETJ90uIsJMUcWw3AgXbOfUib3iVPXrHmE5NNdDx4/AF1xVWdaNcQtN0Z3TuhP01OL/V2jzzJq0li33F+Pnoh6B5v4Xfj+/JQmLBednIUDrefmWNy6NMtht+8EZmeeUYvZWc6tVqPuRMK4OtyNu5t4pvd9Tz7SSl/vnQgj04dxGMfbOePo00Mts/lV3nvEq13stOVwR8rb+aN+vO5fXwhe73buPvMbBxev6pURILNRHK0WVbFtRh1/OWyge3+ToT+Xd+ayrxsXbnsquzbLZgx1NbtdaTq7pNdF9DZdOZfyf9UXhsBlEqSVAYghHgNmAxsCTvmauAtSZIqACRJqu7EMWmcRNrL9OkWayE9zoJfCpAeYz2mc6tJQ4dkE4KFZmYem1aIw+3F00717qlEZb1TVXE1JzmKARnxEcer7YxCgV5AXt3vb3SxvLiCp68eSoPDg8Wgx+v3EZAClNbY6d89lsemFVJZ78Dh8RNvNTHnta9xeQPMGpurOrmGq3iCeq3HvJWbeWRqIXe9uQF3a43JH97eyF9/to8VeUtIqv0KEKx1DuGZA1P52jEAEFiMOvqnB6XPMhNs3PnmBrk2IHws04oyI+TRd9fajyilkRZrodHhYe7E/nj8Esu+KGNYdmThWCAg8cPeBrYdaOKm0TmySyo8SHyy6wI6m47IUXQDHgS6S5J0kRCiP3CmJElLACRJmqXysQxgT9jPlUDbaF4+YBRCfEJQ6uJJSZKWqVx/JjATICsr62iHrXESKUiP5YFLBihWjOFCdXecn09KtEV1sjsSjS6PvOINZZD0SLBR1+Jm7oR+3L1iI/UOD7efl0/8cRa2tcexNChpj/Q4q6rialqcug+6vcrjnOQoEqIMNDp8zByTQ5zFSK+UKOa1CdzXOzyKDKP7Jw/gwv4JbD/QLJ/XalQ3uCkxyu507dV6uDy+oG//q638IvlfTE9eSY+D1bhELC/WTCLzzD+w057Axj0lQJhEidePxaiTawWeX7uTuRMKFDUAOSrSF68XV0ZkloViBqEss+Vfl5OeEEVStJkdVc3sbXBHBH3bi0e98mU5+xtdp3WQ+HB0ZGfwEkH5iT+0/rwDWA4sOcxn1P4y2sYWDMAwYBxgBdYJIb6UJGmH4kOStBhYDFBUVNSOPqPGqYTBoOOSwgzyUqPZ1+Bi64Emlq0rl1eWCz7cQb/0omM6d5zFxPLiCpb/YiQ7qlsU7oLg1j+aj3fU8vhHO45b8kKNjjaBPxJqhvOBSwZQkB6nenx4ZXEoI0evg5QYEy5fgH98tZszclLo1z2WG1/6RrGCvvedTbwwvSjitVWzR5MSe6g4UCeEes1DmyzX9naAvUyVWHfM572MT7Dp3ZS4s/g47gG+N0ziuS2VPGrOYPmn2yLSjX83vh8zRuWQFB2UQN+4twm+Dgoeujw+eiZH4Q8EVIveXF4fj0wtxO310yPRChJ0j7eSGmPG7vFx0cDuPLBqC8987DxUbNcm6Hu4eNSSz8tO6yDx4eiIMUiWJOl1IcTvASRJ8gkhjpTAXQn0CPs5k6AMdttjaiVJsgN2IcRaoJCgsdE4zTEYdBT2SKCm+YCqNLPbe2w1ADod3DwmV7Wb2r3vbApWle4IZq80tSPlcDx0tAn8kQg3nAcaXaTFWShIj2u3vaLH71dkT7Vtkznz7N5s2ttEbbOb28bmohMCuyf4rFesr6TB6SU9ziIbZpc3KChnM+rl2hC7x8+K9ZURrqvebe4v3JC5vT4uiv+GP+X/h9TNxXij9XzSNIwlBy/n7J9NZmh2PP96cyOzx+bx8hdlEVW/8yYV8LdPStm4t4n5k/rLY9m4t4m73tzA3IkF7DgQlK++/bx8Hv9oh+Kzbq+fu97cwJxxedS2uHntm+DzuTusT0L4Kv/J1SX8rHeS4n7a23XpdZz2QeLD0RFjYBdCJNG6shdCjAQaj/CZb4A8IUQvYC9wJcEYQTjvAE8LIQyAiaAb6fEOjEvjNCClnf7EydHmw3yqfeKsJnZWNxBrNaj+4TY5vTx11RBe/qKMWIvxuMauRlWTiwSbSU4BheAkezwuhJDhLOxx5GOToswsL67g4SmDuH6pcuW/6NNSZp2bF1FotmJ9MNvo3gn96R5nYcHlheypd2I16nn5izKMeh3VLW5ZdjzOYuDLnTX0SYvB6fZhMxvIiDdHSGMYDDom5Ns444L/kbx/KVbfPtzeRL6L/gUbom6g26CeDNvXJHedW3r9CA7a3fwsNwkh4JUbR3CgyU2s1UBVk5Md1S0AJESZeeyD4M4hxqKne7yNijo7w7Lj+X5PIykxZh6dWsj+Bgd90mPZur+Jl78Ixl36dIvhln9+K2eXqa3yn/m4VI6BDAp75u2ljY7rm8rAdjK7fgx0xBjcAbwL9BZC/A9IAaYe7gOtu4dZwH8Jppa+KEnSZiHEza3vL5IkaasQ4n1gIxAgmH666RjuReMUJrSSbZuD7jvG4G73GAv5aQkkRql3U9t2oEUWqtPrOt+rmB5nUU27TIs9MS6EnklR3D2+HzXNnghjOGFQhmrv6dAEGOrgFW02sPR/u6l3eJg3qQCv30dlvYNGl5ftB5oZ3COOy4cr8/bnTyogLTZoXAMBib27izGXPELSwZX0kFxscfbi5YN3MHT0r3jjuwOM7ZuCTiCnccZYDOSkBOUddtXa2bq/iZLWfgahcTw8ZSB3r/gBh9tHeZ2Tt76t5NqRwXEk2EzodULx3GePzaPB7sHp8ctxl4dbM4vaq7MIGXC1uEx7aaM/ZkMAIKT22iOpHRxcvfchGAvYLklS10TmjkBRUZFUXFx8Mi6tcYyU1bQoFDAlCVZu3MvS60cc00q6ePdBrlnyFfdc1A8hoKbFLQuxJdpMLFpbJhckvXLjCIb3SjrySTvAzuoW/u+pzyKM0L9vG03v1K4PLoaay8RbjfzmjQ0KZVGLQcdjH0R6WR+eMpA99U5WrK9kyrBMXvisjFnn5vLYBzuwGHUsvnYYDU4PMRYD+lb5i5mtBYPh9/jKDcMYpvuUxu/+QoLzOzwBAx+3jKCp129Y8G2c/NxD2UTLbhzB56W1ZCXa6JkUxeDM+Aj9o5Drpt7hYcG0QvxSMIB9/7+38JsL+soG6dZzcxUV1aExPX75YJxeH7UtHhZ8uIPbxuby1JpSbhqdwwufRR4f8v8/cMkALinMiHDHhZIDfixpoyGEEOslSVIN1B1xZyCEuKydt/KFEEiS9NZxjU7jJ0FoJdtZRTqyX1dImI16hUvk9vPy5eNc3gDVzerN5I+H6mZ1v3JNi+uYjUFHspN219mZ9c/veP2XI7nlnFzmhnUte/bnQ1V3SxUHnXKxX2i8PRJscmVxvcNLZryF0hoHc9/dzE2jcxTniNM3c13CewwsvgGdrwqfL56n6i5nae1kDvrjsOzVMWNUpux+cbbKXIcaGO2uc7DnoAOvP6AaoJ11bi5Ob4Bml499jU56J9v41dm5lFYfynBqb6Vv9/hY9sVubhuXx2NTCzEaRLtNhULNemaOycFq1FPZ4CArUfmsf2xpo0fD0biJJh7mPQnQjIHGEensIp2QJHZGvJVb/6msTn38ox2yS8Ri1NEt5tjiEoejs+UIOpqdFDKG9Q6vbAigNcf/vc3cN6E/89v0FXjly3LZZTTr3FwsRh07qltkA2E16vEFhOJ8FqOOXvoSbkl9k/Njv8Si89JgGMSWbvcx7T+Z+DkUP2jrfrGaDGQnWamzexXGOjXWQoLNpKhXcHkDpMVa+GNYVti9E/qzeO1OJhZmKJ612nM/0Oii0eXF6fFz14qNPDq1kEWflvKXywaxqbKBR6YW4vEFC9ieXlOqaF05c0wOfdNiT2vF0c5APVUhDEmSbjjMfzeeiEFq/LjogGeyXQx6wdwJBbi96k1zQrLJc8blEVBXSjkuQn7lcHmF49nptJedtLvOrnp8yBg5Pf6I+y+vcxJvMzJzTA4PTxnIjFE5cvZM6Nwef4DZY/N469tK2UDodVDd7A5mzuCneevLrB14J//Jn8P5sV+yuvlM/pOxiul7n+ZTzziMRmVgPrzob+6EApZ9Ucbd4/vJej+ha9+/cgvTijIjPltR74g4bsKgDLmK2mLUsWJ9JXPG5Smee1BsTnD/5IHc1Zo19Nb6PdxyTi7VTW4e/M92Zr/6HbvrHCxcXRphhAISh33WPxU6VIEshPg/oACQlz+SJM3v7EFp/Pjo7Lz8OKuJj7fv59ozc1RXin26xchCZL2SOz8VsLN3OkdqX9mWkDFKiW4vgN7MwtWl3HlhvqqPfWSvRDZUNjJlWHBSXrG+kg2VjZydBb/r/gpT4v5LirGBGlcSX8TeiqPnLUTbkimvszOmj4ePt1Xz4KUDFcJuD146EINe8Ny1wyivs3NGTgplNS2q95WbEi2PO1QbsuDDHRHH6XXBorZXviyXG9PnpEQz69xchWrpg5cOwKg/1F6zf0Y8z34SVEzNTrIyYVAGfbrFtFuhfLhn/VOhIxXIiwAbcC7wAsFMoq+7aFwaPzI6Oy8/K8HGBQXd2VXTElEYNXtsHg+u2ioHMtPbqeLtCO3584/Xrxw6r9PrZ864XF4vrlS4MNpzO4WM0aa9DXIu/qGMnwE8/lFwYo0xGyLenzuxALNRx5OrS8hPjWbmmN785WwXAxruIPHbDxmW7OV7Rx/u23sznzp/xvxLBhNjNPBDeb2c9XPH+fnkpkSx9Prh1LZ4iLUa0AuJfQ1u7nrzUE7/s1erxy+izMqeDSkxJrnZfPhx/dNjsRh17G90seTzMu6b0J8HV22hvM6pOC67dUcWupYQwR3SG99UcOs5udz37mYSbCbV35VXviw/Lhffj4WO7AzOkiRpkBBioyRJ84QQf0WLF2gcJR1d+R6JinoHf/zXJvJTo7lhVLBRikGnIy81mofeP2QI5k8aQN5x7gw6e1dzuPOGSy4fye2k0wkanF5FDwFJggaHW55Yw3P1Q+8v+rSUuRMK6J9q5s6+X5Ow/jr6WnbhDJgoj51Ac69f02Tuy8VOL9PMBoWURWjyXPDhDp6fXsQvlhXLY39lxnDibEYem1qI3eMjymRA6CT+fOlA/hC2g/jzpQOZv1I5oWcnWSOOmzuxgGc/Vt7ba19XcNvYPEWldvhzCqWEQtAwpCdEcV9rDGR/o4tl68qZOSaH/umxbNnfdCiD6UdcTHa0dMQYhL45hxCiO1AHHJsYvcZPjs4OuIaMy+j8VP7x5W6mn5WD0+PDYtJxz0X9cXr96HWCl78oo1eyrUOppW13ATpBp+5qQuyqjdwtPbm6hJdvGEFKjPmo3E5ubyCiUfugjFjmTx7Afe9sknP1w99PMdSTVfEGr6Yux1xTT5MtjbWWX/O95ees+KGJB/v3QwpIxFiM3Pz39YrxhRds7W9wKt5zeyXmvPZ9xHf87NVDFBO6QYfCEEBrnMNqUOwWshOt7KhuCcpRhJ2vb1oMq9pxz4VcdwftbvJSo9nZxk21v9HFwtWlLL52GOP6pnJmTtKPon9xZ9ARY7CyVW76UeBbgplEL3TFoDR+fHS2/ntqa0VzjEXP2L5pckFS20Kw2WPzqD+MHEXbiT8rwRaRA//gpQNVs1921rSQlWBrVzLiSJQftKvuloLNYI5svAIBibg2vX7T4yzcck4uDq+fx6YVEmMO9SzwM9S2lVtS3uTs2PUY6/zUWYez2nIjv/miB04vWIw1rc1o/Nz26ncRqaWh8YWC83E2E7eem4sQwZoAg05w0+igampI5dPlDeD0BhTG6KmrhqguDPQ6HSa9jtzUaPK7xZAZZ1XVa+rbLRaDQadqiMNdd4M8fr6trI/o5mYx6nD7Alyx+EsWXD6YM3ol/eQNAXSsB/L9rf9cIYRYCVgkSTqSHIWGBtD5AVe9Du65qC+JUWa5q9dlQzMjmuksXFPCshvUherU3DRqPYDvefsHZo7JUWgrhQKP/yurxSh0RFkMePx+kqKObkUPqDZ/txh1GPQ6ympaDnue0NgrD9r5/fi+1Dk82Ex6Yi1Gfh12P/Mm5LJ09AZS9j5LrnkPjoCZnVGX0NTrDvzR+Tz81kZuHHWoYG15cQUPXTZIlnRQG59OBJ99s8srB6fVJC+aXV5e/bqCeJsy62h/g0NVAG/HgSb0OkHftBh6Jgcn+o7oNYXj8wV494d9HGxxq16rsjVz6XTvW9yZdCSAPA14X5KkZuBOYKgQ4n5Jkr7rstFp/KjozEIeu8dHvM2IISyDxGzQqa5km1zKnUFoN7C7zs72A03yqt/lDbTbAzg/bGK0GHX8fnxfdtfZFQ3X753QH7urnpzUaMb26XZEg9At1qw6Ud315kbZj91eXCIUkP/1eXm4/QG53/ETHwXPlWas5RfJb3HhjjXE61tosGbwieVOvrdcwfLvGvhjTk8MLq9qm8pml48ln5epBlyD9+glPc7Kba8p6zvUJC9uOScXnZAUz85i1JOZYA3GFtw+oiwGHC4vj68upd7h4fx+3eT77IheUzihHguhntLh7iebUc+itWXyuH/qWUQhOuImuleSpDeEEKOAC4HHgEVE9ifQ0Ohy7G4/y9bt5s4L+8kTTa/kKNWVbLTl0K/5kbTqA5J6UVO/tFiWXj+cdWV1+APQ7PbxdGulbUhCuqrJxYCMOPQ6wX83H6B7vJWC9Nh2V7JZiVHkdYtm5pgcMuKs7G10KiS+D7dqDcVMfH6Jpz8OTtYCiULj9/wq401Gx3yPQOIr+wDqs+fg6zYep8dPb7OBjF1OEqOCHebURNyWXl8k+/gDksQd5+XRMzmaeKuRFo+X75tcbA3rexAivOgslL8/993NPD+9iKXXD6fR6UEIHdWNLrz+YO1HVbMbf6OTnslRXHdWNi9/Uc6eege53WLk76sjPSNCx++pd8pNacJ7SuenxvDn1kyz0Hf7U88iCtERYxDSGv4/4G+SJL0jhPhT5w9JQ+PI2D0+pgzN4tH/buUvlw5kV50dt8/P45cP5qH3tyqyXxytcghweK36Zz4u5b0Ne3n4skHc/dah9MgFlw+mV3IUZTUtsqso1AUsvKl7gs2E1ahXrKTnTx7AGb0S6JEQOYnpdIKxfbqRkxzNjqpm7n7rB8X7h1u1ygF5XwDJ5+TapA+5seF9fpu7mxa/ldcOXsCiminUSBk8O3oot/zz20Ouo0kFmAzgcEvMm1iAzWyQexYn2EzsrnMq3D+zx+ah18H0pV/z8JRBLFxdyqyxuapG02LQceu5ueh1kJcaQ4LNxFe7DvLCZ2X85dKBHHS4WLZud8SOZM64PPQCpp+ZjdWkp6w1HtM2fnO43dLhDH2oGn3RNcPkTKvjjVv92OiIMdgrhHgOOA94WAhh5igqmDU0OhOfL8Dm/Y1EmQwsWruZey7uT3mdXSF3EPJXN7v8LC+u4K9TC+XPt5fiGgqKXjk8i+4JFoVbwWQITjzxbYK12UlWhYiaWszivtZGMnvqneiFiFjdhlxnoL4jaW/V2jMpimcvSSZ1z3x+2f9fxOgd7HF1Z3PiH5j9zTDqPSY5+P3sJyXySh/gjeIKrhzRkz/+KyyNs7Vn8Tl9U+WahNA9hJra3zQ6h6rGoJHdXNnAvEkFCk2kuRML0CNx33tbFJN8lEnPjFE5ePxBafFpw3pE7EieXF3Co1MLufPNDTx15RCuX/qZavzmcLulIzWleeCSAZzVK6ndTKSfOh0xBpcD44HHJElqEEKkE4wdaGicEMIbrz999RCuKMrC6wvIfntQ9gJe8nkZt5+XT0NYzKC9FNfwimWn1x8RLF41ezQOj08O1sZZjMydUMCW/U3yudoTUTto91Ba0yKrqg7MjIuIKWQl2FQzZ7ISbMqHIElw4CN0Wx7h3Ko1AFRaz+S2kv/jk8aBZCfZeHBaP+weH7tqHSTYDIztm6ZYhYdndYXGOG/lZh6bWojLFylv4fIG2FDZyAufBZ/nI1MHoUPwqEr9wuTBGYrzvvZNBb8c01ux07h/8gD1LKrWNpcNTu9h4zft7ZbaM/T902NYPnOkHHz+qQnQHS0dySZyCCGqgVFACeBr/b+GxgkhvPF6rMXIwjUlzJtYoDoB9EsPTu7//LqcR6cc2hmopbjOHpvH82t3Mjo/lWlFmeSlxkR0ATtodxNjMWIx6Vn8/qGJ7fFWfaLQGNQMjVGvi2g0k5sSLWfMQLCI7qk1JYrJ9ak1JQzNam3W7rND6WLY8TS0lIExDpF3Cx8xnXs/auKyoZkMaP3c/JVbuPXc3ozolYDT48ft8yuC5NsONKk+MwmIMhtk+YbQTuK9DXvpHm/F5Q2KAD5x+WBcvsj6BoBAGxmoCYMyZMG80HUq6x2qz6mmxY3FqCPWauTWc3PJiLN2qCq7PUPfLz1Wm/yPgo5kE80Figj2M1gKGIG/Az/rmqH9NAm5QfY3ukiPO3wA8qdGeOP1mlZBtVq7W3UCMOt1LPm8jLkTC+gWd0i1NDzFtaopGMhcuHo74weky6vnouw4Hp4yiANNLqxGPe9t2ENdi0cOiIZPbA+9v5V7J/Tn/pVbZBG18JjBny8ZyEPvb41wiQzNSlAYg6oml+rk2li9Bcqeh/LXwNcCsX2g6FnIuR4MVuJ311Hv+FrxuewkKxajgV+29iI42iC5XkBNsyuiFeX8SQOIMul46qohPL92JzuqW9oN1rf1uOh1kbul14sr5WcWusbt5+Xzz6/LueeivtTbPREpq0dTld3ZtSw/NTriJroUGEKw4AxJkvYJIWK6ZFQ/QQIBibKaFrYeaEIIQWW9g637m6g4aOeignTNIKBsvB6SsP77lxXcN6EfB5oONbfJTY0hgMRjUwuJserpkaCcDMJTXAMBiTirgSsWf4nLG2BQRixThmYx85VDMgsvXl/Ehj2NOL1+1VaXzS6vQkRtzrg8eiZFEWXW0+T0RVTburwBRVAbDhXRBbOCApwTU8wtqW8xeMNmEDpIvxD63QWpY5AvTrDL1B3n5ytSXOdOLOCWf3x72CD5A5cMVMQM5k0q4MX/lXHdWTnc2RoDCX32vnc3yW63uRMLsBqg3u6OiBk8cMkA7C6fIo20X1pshNGod3hIjjbz2NRCvP4AFqOePQft/G58P6wmfUTV89FWZXd2LctPjY4YA48kSZIQItQD+ajMrRBiPPAkwbaXL0iS9FCb988h2Ad5V+tLb/3UlFBDXas2VjZi0OnISYkiK9GGTgianB62VzVRkBF/sod50glvvC4EzJ1YwIr1FeiE0g0zb1IB6XFRJB5FAZhOJ3CEyUDfNKY3fw3zhUNQ8uHJ1SX8bnwfbvhZT8XEe8f5+Rh1gic+L2HuhAJWrC/n0iFZlFS34A8EGNwjXnUFHWs14vMFZCOv18Fvz03n4IanuCphJT1M1bhEHE09biVu6O/B1l11/I0uL/EWA49NLcTp9dM93kKT03fEIHleqk2hIWQx6pgwKIPyOvWq6FA8ZN57m3n5hhH8bunX5KdG89jUQmjN3a+sd/DiF7sUrq4ln++MiIXMHpvHn97dLOtHhZrMxNkMfFve0I4LSzoqV89PsSlNZ3FUxkAIIQjKUTwHxAshfgHcCDx/hM/pgWeA84FK4BshxLuSJG1pc+hnkiRN6PDofyRUHLRTUtWimNBmj81jeXEFt5yTi8vrP/JJfiKYDcHJo7bFw6tflfO7i/tx40vKhvBz393M0uuHH/WEEO5rFkgRaY/9Ly/E5Q2QEW+LKLRa8OEOlt0wnBmjcljxbQVXndGT28PcFPdc1Jf5kwpksbSQS+SO17/ntrF5wZaL9hJs3/6Jn1f/C2s3F7X6XN633MaDW4bxyBnDGWlLbnfsMRYj2w+0cN97W2Q5DoG6G2hYVjxSmKz31gPNihak907oj88vqX421IPC5Q1Q0+ImwWZi494mZr36HXeP78OTq0uYMy6PK4dnRRTRRZuDbSazEq1UHHRG9FbIiLPyxOoSfm3Lw+ULRMhrTCvKxOHxH7EqW+P4OCrfgxRslHwJ8CawgmDc4D5Jkp46wkdHAKWSJJVJkuQBXgMmH/twf5wcaHKryihMGJTB3Hc34zu2nvE/Ojbvb+TONzeycHUpKdFmdlS3sKfOobqSPGj3tHOWSMIb1SRHm1leXMGMUTnMGpvLTaNzMOl1ck6/2rVq7cEm7GfkpMiqm6H3HvzPNnqnBqWeH5kykEenFvLvjfuoqLPz8UeLcf9nJPy7Lyl1b/K/lsFcUvIoRd89wc3rhlDt4IgFUaFdS3hqqwSqDWB219p55uNglW+U2YC19X0hwOOT2FjZyK5ae8RnQ01wQj8nRZm4dmS2LA2eaDPh8gawe/wsW1cuP7tQdpYk6Xjm41L21AfrF8I1nixGHRX1TuodHmpa3Hy5s4b7Jw/AYgxKj08/M5vFa8u48aViLl74Ge9vPkCgbZRao1PoiJtoHdAgSVJH0kkzgD1hP1eiXrF8phBiA7AP+K0kSZvbHiCEmAnMBMjKyurAEE59mltT6cIJ35ofdBz9xPZjJjyAXOfwcPf4vkRZ1PV9EqNNR33ecF/zzpoWbjyrF3UOjxyDqHd4+Ns1Q7AZjTxxxWASo4ws/XwXH++oDVY4mw08ddUQAlKksUiwmdh2oEUOlqaYHTw1bB294v5ON0MNHkcS9Lsbqc+voRzGVDYyKiwFNSvBRllNS7sVuA7PIZdQ6PfF7vGzdns1j0wtxOnxEWsxcqDRic1kYM64XJKiTJgMQq6gDq8HMBv0PNEqK2E2BEXjHg6TBJ87oYAml1cRg4izGclOstKnW4xckxEaj8kgsLWK5akF2EM74FAA+f7JA7n3nR+YMSqHvmkxEfELTUuo6+iIMTgX+KUQohyQ+8NJkjToMJ9R28+1NevfAtmSJLUIIS4G/gXkRXxIkhYDiwGKiop+VEuDGKt6Ol+ohWCMuUMN6X60KALI0SYONDjpFmtSLX6KNh3dMwvJF9TZ3YjWX9fUOAsvfrFLrmJ+8opB1DR5ue/d7xRxCaMBhmQlcfeKH6h3eJg7sYCi7DjOyEmRv8cok577V26hh66MW3q8yUVxX2Bp8VBl7stdFdP5+eV3UZidCgEJt/eAwlX412mD+aSkmln/PHTdthW4SVHmiNTWaLOeiwamt6vk+qeJBTQ7fYq4yOqtB5gyLIt9jcFVeig7KeSmyUuNQQKWfVHGbePyFTGIlBgTN5+dy8tflPHrcfnsqrPLhvTX5+UTazEorjVnXB4pMWZMBj01TS5+c0FfDjQ4WHjlEBwev5xVFaryDudwdQYax0dHZpmLjuH8lUC4xFQmwdW/jCRJTWH/XiWEeFYIkSxJUu0xXO+0JBAIRKTzzZ1QwIpvK5g7sYDMeE07BZQBZK9f4sH/bCM/NZqbz+6tCIYaDYI+qUdOdAvJFzz8/lZVwbaQbzvWambOcvW4xB2vb5DdHvPe28yia4bJ2TA2o8RL4yp5LftxBtt24AkY+LDpDJ6tnsq4n13EiFwbBRnBeMCuWju/eUNZCPabN75nzri8w66MnV4ft5+Xz+Mf7ZBX3pJEhOso/Bx/ei849vD0zVBKZoLNxOyxefKzqHd4sBr1PLhqK/UOD7PH5lFa1aIo1Ktp8jDvvc3MGZfHgSZXRE1FZoI1ImX2mZ8PYfuBZgISNLm8DMyMY2BGPLvr7Ees20iJ1v4euoKOFJ2VH8P5vwHyhBC9gL3AlcDV4QcIIdKAqtZMpREE4xh1x3Ct05Yok5F5K4MTSEj0bH+Tk99c0Je/frCNET0TT/YQTwl0OiE3eq9qdnNmr0SuH9ULp8dPks2ErzFAUrSJwd3jMJn0RzxfSL5gxqgcVcG2R6cWsr2qmdoWj3qsoMVDSrSpTXGah8cv6UGP6hfIbX4Vy75qakzxLKy6gqW1k6j3xwV3FnkpDM6Ml7OJ2uttkGgzRbwWvjK2GPX88+ty2a1SWe8gM8Emp8BmJVhlwbbwcVY3u4JuJLcPm9lAVWOwUU14v2EhoH9aDOUHHUwZlokkBSWuJw/OULQWffiygbi8ATITbBFunSdXl7DommGKewjudo0Ko7Hg8sEEAhKSBI9NLaSkupmPt1WrqrrqtSzrLqFL/Q+SJPmEELOA/xJMLX1RkqTNQoibW99fRLCX8q+EED6C3dSubA1Y/2Swu/2yIbj+rJ48/lEwdXHx2jJ+e0EfDjrc9EbbFu+us3P/yi1MGJRBVqKV8QPTFYVV8yYVsLfezt4GZzBL5wi1GSH5gvZkJLZXNfPCZ2W8dMNw1RVqnNXAVSOy4etyNu5tYkh0GaOrniO27n3MOi8/OPNo6nkvjrTJ+PY5+HlYLGBoVoLC999eb4OoNi7CthW4Bp1OzuBZev1w7nxzA89cPUS1yU9opxOajMPF6x64ZADZSVbK65zsb3TJwm4LLh+sOM8DlwzgYIubpV8cyghyeEO1BepSFp6wDCGLUcfDUwZx7zvKQPvD72/F6w9w94qNims53D5mnZsbFOSTYNm6coZkxSsK9jQ6hy53RkuStApY1ea1RWH/fhp4uqvHcSpjNgazVX5+RpZsCCD4R/LYB9v5x4wfh0p4R+WI21Jnd8vunNF5STz7SanCF/3sJ6XcP3kAM19ZT15qNIU9ElSv2z3GwqYDTa0FTzqsrc+/7UQcbw36urfua+LZq4cyb+WhXsDzJxVwoMnJ82u38+iI7VhsT1Jg3Ym/wczKxjP5W81UtrlyyD5oZc44weK1wf4A04oycXkC7Kq10yv50P2319sgPd7M01cNkXX/Yyx6RUXtgSaXnMETZdYzb1IBOiFUs9NChWN/vnSg7JIMvf/Hf23ib9cM41d/P2Rcbz8vn0WflDJzTA790mLZeqApmGUkhEL5MyvRxtwJ/UiPU5eDSIoyKcTh6uzuiEK8CYMyZEMQPia1pkKa5HTXoEUmTwEaXUFfbGqs+bjTJE9VOqOpvEDw/Z46nrt2GLUtHn43vl+EXLWztYDsQKOLwh6R181OsnLrOXnc9+4m8lOjmTuxgGanR3Ui7pkcxWNhwdsHLx1IQpQBi8FA2Z5d5NW/wL97rMC6tw5nTBrrzHNoyprJQ+/vZ78ruGqeMCiDe97+gQSbSZa6Vrv/8N4GIbXU/t1j2NfgoqT6kMhd79Rg1XTomaXHWeWA78heCbxRXMGMUb1Vf4/6pcfw2NRCAkiqVdF2t09xfYtBR6PLi8WgZ1+Dk4WrS+VnE15YVt3kQggdW/Y1yvGLcINS7/QwrGeiIujb1mioyVa4vJFNhTR5ia5DMwanAElRFpYXb+GhywaprqzCm7OcrqjJC3c0TdAv+RjXL71dzZ2Fa4KyBRajjrTWHPi2150wKIP73g1Ww47OT2XRp6XccnaunE4ZmuCWrSsPruIV/u8d3DOiGd/Wh5kW/Q0mnY8a8xA2JD3Ijau74/AKLMXl3H5ePi99sZv9jS55krtsaGZEXOKO178nKUxmIdTbILSCdnh8bNnXHBGQ3byvkYGZ8eh0QhFUP9Dkpri8kTnnqbucbEY9N75c3G4PYiQiVuEvXj+cBR9s44ycFHncTq9fERB+5cYR/OKVYm4ancN7G/YqnuM/vy7n0amFivTYrARbhIbQsKwE1TFlJVg1yekTxOk/y/wIKEiP5baxedjdPkUmR2iyc3h9Rz7JKU578sIdSRPUoVcIxSXYTLh8fn49Lo89DU5WrK+ktsXNA5cMoCA9Tr5uuJ5Qn24xsoJnqNgqxmpQpFNCcCLytw7XKLxMjvuEu7P+Q8reHTiizbzXMIZna6byf6PGsnh1mWKSf/yjHfz2gnyaXD7yU2OYMy4Xq0mvqmu0u87OFztrZVnrcCmFz3bUyLuVUGKB0+vH4fWzZnsVY/t0w2DQMWlgd3omReHwBH33Rh2qO53yumBG+PNrdzJ3QoEie23epAIWr90Z8f3sb3AyZWgWr35dLj+XIWESGxZjcPfg8gZYsb4yYvczd2IB9Q4v01/8WrG6v6BfN8Uk3+j0qP7ue/wBTV7iBKEZg1MAg0HHJYUZfF9ZL1e/hlZWy4sr+Ou0wSd7iMdNe/LCh/P/tvX1N4QV54V3GAuf8FJjzTQ6vWFuFEtEMDWkghlt1vOrs3Mor3PwwCUD2HPQwevFwYbuD1wygH988iX3pL/K1ITVJBqasIvuPLx/On+v+z+aA0FXRUBSd29kxNt4LGzlO3diAb86O4cH/7NNMY6Ddg/PrS1TlbW2e32yIQi/18Vry7jj/HxyU+xkJUbx0fZquR/ynHFBSYf0OGWDnu7xFl5ZtxuAjXub4OtyHptaiD8godcJ0uKCFd3hWIw6usVauGlZMY9OLaS0upnc1GjO7JnE8pkj5Sb1Dq8fi1GnyETS62BU72QqDrYwp42Exx2vf8+q1h1haJIvq2lR/d0fPyDtWH7dNI4BLUnrFMFg0JEaa2bWuXks+byMp9eUsuTzMmadm0dq7NFX056qhJq3hMscqDZvaSXk67944Wdc9fxXXLzwM5pdPrKTrACqbpcnV5ewbV8zs/75HbtbV8H+ABHB1CdXlzCtKBOzXofd4+fJ1SX89o2NPLe2jFnn9ubh0Y2cted6VqRfxU3J/6Lcl82qhEV83OcrljZcKRsCCPrxQ/cUwmLUYTPrFdec995mau2eiHH4A5L876omt+I88dZgZzW1e13w4Q5qmt0KN9jLX5RjM+rx+SUe/2iHvLPxB2DBhzu4cVRveaw7qluoOOjgofe3MevV7/i2vJ7ZYyNlKEIifturmnlubRlev8SBFhcDM+K5cEA6hT0SMOkEcycWyAZhyedlWAx67lyxgYCkI6Gd9NhweiZFcff4forf/bvH99PiAycQbWdwihAISBxocNPi8vLctcOoaXZjMuhZ9kUZWYk2shLb3yafDj0Qjti8pQ1qMYZ73v6BZ68O9vMNTwcNuVCEgPgoEwk2E1VNQfdTdbO6e2pA92C+/8zW+INZeLg0ZjVnbP0lueY9OAIWtkVfxv+MM8jPG8Idr6zn1+e5IlwZiTYTcycWyK0iQyv+tk/f5Q1ENH4JvXbrubkIAT5/QBEcthr03HF+Pi1udRXSRqeX5rD39je6WLS2jF+Py1PtjVBa3SKv2vt2i+H+fx9qDN/k9vPlzhpF7cHLX5TRNy1PFqoLfQczx+TQNy1WDn4n2kzoRQuLrx1GcXk9/gByHOe+dzfx2NRCtlU1y7UOajtCTX765KMZg1OAwzXy3t/oitC+Dye8FWR4fvbR5NmfSKqaXHh8h2bDkL++oy0M7Z5gxkteajDLRC1LZ864PGJag+7hfQJCWIw6Nu1rpF9aLPFUMzN9BVMS1hBnsFPh7sYnUXezI/oq9KYYGhxeOUOpxe2PCJC++MUu5k8eoHDJRJn0ePzKsVuMkY1fLMag9k/oe3+hTYaR3e1n6f92c8/F/VTvQa/X8X1FveK9/Y0u9rX2KW57vNsXkHsBP/zfbQrBuC931jCtKEvu5xxybel0MHdCgRwzCBmw8OB/dnI0pbX2YBB/tdIAubwBtrXWa4R0iI604v9pVRmdOpw6s8VPmPYaeV82NBOLUUePdlwpAFv2N8or7pDK5lNrSthe1URZTQvrdtZSVtNy0pUe0+Ms/P6ifM7olUBuSjQjeibw+4vySYs9fAvDcCxGHSXVLSxcXcqDq7Yye2we04rU3UVOT1D2W69TUfAcm0vJD6s4a881/K/vjdyQspJGywDeiH2ea2v/waaYGSz4ZB/9u8fQLy2GaLNBFlq7oihL4cq49dw8vG0m/hirkehWcTYIdh579uqh5KVGM2dcrpyPf++E/jzcpgvaHa9/L7u4oizBwPaDq7Zy+3n5inuYO7GAP727ideLKyPcO/ndYrjzwj4Rx/dNi2bGqBz8Pj9XDs9SvD9jVO+ILm7z3tuM1yexaG0po/NT5WNDu4SQq0enE5ybnyp3P2v7nYWOX7imhIVXDlFNJ1ZzC2oKpScWbWdwCtDeKjg0kRn07W+Va+3uCJXNW8/uzdYDLYpOVh3N6T9e2gZ/DXpw+5DdMhZjsDG6aGc4mXFW5k8ewH3vbFIcv+DDHQBysPLXYdo9IYK1GV7Kalqos7vloiwjTi6MWoO+9DZ+lbYTf3MMZXFXc/O349jpSGmdNPPwtvYMLqt1cP/KYJ+AUHbOK1+WM3NMDr1TokmPs+Dz+3mzuJKpw7OobXaTEmPmjW8qOK8gjcemFnLQ4cFm0iuqfe+fPACv30+0xaCa7x9ycXn8fn4/vi8Or5/UWDOPTSvkoN1Noi3YxnNiYQYr1lcq5CP6pcUQZ9VT2ywUuxWP10/AqJMF4Fasr1TscMrbkQLfeqCZ8jqnLEoXSpsNd/UEAhIfbK2iqtGpmhH0ypeHdhVOr1/1d7AzUo81jg/NGJwCtJdpk5saw19WbT1s+X2cxcR2r7Ixzr0T+rN4zY6T9oel5vZadM0wnv64RFEx/PTHJfw1qZDspMgxba1q4pmPlTGGertbrnyFw7tE9tTbmbP8Ox6eMoge5moSty3ksoSPSTA0sy3Qi7srb2PypN9w4z+2RKyGH51ayLSiTFl2en9jsMo36CuPYduBZh76zzbqHR4enjKQkb2T5QY7oVV4UrSRmiYv2Uk2uS4idI1739nE45cPxhdQbyRja9VVSok24/Yf6leQnWTl5jG5/DbMlROabEPyETPH5HBmThJ/eX9bxHkXX3tII6htKu3TV6vXHvRsXe0XZSfQt1shDrcXk0Eoir9CE/mjUwt5rLVLXFailb0NykY2h8se64zUY43jQ3MTnQKEN1eBQ5kcf2lVijxc+qXT6+e1b5TNWBav3cmEQRmK49QyOLoKtVXerpoWrh6RLbtYXvisjKtHZNPs9qqeY3/joQbxT68p5ZmPS1n6Rbnc+ASQA7htXSh3nJ+P1xdguPkbUr/5P5anXcP1ySv5omUQH6S9xnf9P+ad5oto8plUJyCnx0dWYlDs7dZzc5k1NpcpwzJ5o7iSyoMO/AGYMiyTm0bnUNXokruYhT6/Yn0F5XUufvvmBr7ZXa96Db8ksb/BEeHiuXdCf+rsbspqWpAk5BabECyYaysjEe5OnDuxgOQoE80u9YBzs9snu7vaus4CkqSaTVTX7GLOuDy27m/it29uoM7h5emrhip2maGJfG+DQ3ajPfFRCVajXiFbcbjq4fbcgpr0xIlD2xmcAoQyKfrcNpqtB5rYUdXMK1+WU+/wHLH83uP3q8ovt40dn8g/LLVVXm63GH6xrDiiOOuVG0eonqN7WO+CEPUOD1mJFpZeP5wGh5etB5pYtLaMlGgTj0wtxOXx0SteovSLBVxkfotf5VTS6IviQLcZePLvIMmbjNVoYPHKTTwytZCYdhrjxFiMVDU5FfUJ2UlWHrikgIN2Lws+2qTYBYSK2EJMPytHDsSGztn2GklRJvQ6wUP/2RrRSCYkr/HgpQMV525PUC870cqMUTk0Oj28/EV5u6t8k17HI1ML2V1rp7BHHC9ML6LR6SUl2ozHH+CvH2yPyPOfP3kAv1vxA1OGZcrxmLYie6GJvG2AHYJuzr5pMWQnRR02Oyi0IGorV6Kllp44tJ3BKYJOJ+idGs3FA9K5ZHAGj19RyKrZo4/o548xG1XllwdkxCtWee39YQUCUqcHmtVWeY0O9W5ude3oLvkDgYjV650X9uFAk5sbXvqG+Su3EG02YDIIxg9I56l3/oP/y1/Q/8t+XGVciNBb+H3lrYwu/QeewQtooBsNjuAu5N6J/UiJNhFtMnDH+cpdxe3n5TN/5Rac3oCi+veKoix2VLVwb2sMIzT+ee9tZvqZ2YqxO8O6j61YHxngnTuxAIFEdaNL3i25fUFXXiiGEErjnFaUqTi32uo5Lc7Kks/LaHH7qXd4qGp0RNR0/PaCPtz3zmZmv/odz35SysEWLz/sbeSBf29l+tKvqXd4IoLjVxRlsXV/E/UOj6IHctvsttBE/t6GvYpzPLm6hOykKM7OTyUnJfqwv8ehBdGq2aN5beYZR/W7r9G5aDuDUwydTnSo/L7J5VWVOWhxexUNX6LMkXa/M8Tj1FBb5SVEGdtZhUf+CgYCEvvC1DhDK9XsRBv3/3uLvJLulWjl8Z/txb95Djf33kwAwadNw3ih/nJunTqdtzeu57lrh7KxslEWe9u6v4neKdEsW7eLuy7qS0qMmbsu7ENilJmy2hZe+mI39Q4P3eOt8lhDRV/zJhaor8yTbPK9WYw6MhMO7WpCge6ZY3Lonx6Lzy/h8Hhx+wLUOTz0TIriuWuH0dhO69PeKdHyud7bsDeiq9uCywdzVk4Sq2aP5qDdTV5qNAlRJmqrWhQBZFNrEkIo9dZq0rFs3SF/fml1C//6fm/EziDUuyAUBFbLbpNrBNJiOGh3s3zmSBwef4eVaTv6u6/RuWjG4DRHra3hnHF5JESZuHbJ14rXshOjFIHorsrgUCsgsrt9EcVZcycWEGsxKj4bMlAx5ki9oL/9fGhw5fnJBqbFvkf/pP+QYaqm0RLLC7WX8HzNpdT4go2Arnb6WDV7NA63utjbL8bk4vQEeOKjHUwYlIFBLxiWnUCPBBtZiTYMeiFPwiH3jM2s3p402mzgkamFSJKEXghWFO9RaP/UOzykx1lJjDbyeUkdmfEWalo8ijE93hozamss9zU4mTEqhxiLgX7pMbg8Pl65cQS+gKSYbEOT6NCAxOelNbLsRfi5Hm1tGpMWZ+HJj0oUC4hgI5l87nn7UAbaA5cMIDXGzL3vbJKLxe44P181u02byE9/NGNwmuNolVNom2f/3DXDIl4bmpWgMAZdmcHRdnL4vKSGRZ+WKlaeiz4t5c+XDFR8LmSg7rmoH78f31eRMttDX4576x/4LO9/2HRutjmzuWffHFIHzeCJzRXyOSxGHd1izeSkRPNZSY3q83n26qHUNntUK3UfmzaIrESrbLwgWCsQbdFz89m5EQbNatIx85VD/YZXba6itMYe1P6RJNJjLdg9XmqbgqvwxCgzs179TnZBXTY0kz0H7dw/eYDshgoFk59eU6qIRwC8NvMMzuyd3O5zd/sCqt8rBKUpml1eLhqYHrGAGJYdH9F3YPar38tGQ5Jg6f92MygzTmsu8yNEMwanOQ5PO5kjLl/Ea219vcciHnestLh9xFmM9EmLkeUOPtthpMWtHFPIQHWLs1DV6OT5taWMtn7Br1LfYkDNNvrEGVjTVMTfqqfyvbMvAI8lxmMxVsoT2/xJBcSY9ZTVtODy+lXdaHaPnwNN6mmpBxpdmA16Xv2qnEemFhJv1dMjwcq2/c0RhmXee5tZcl0RM0blYDXq6N89ljnj8rB7/OyobibJZmLBh9uZfmZPBmXGEW0xyS6htgJ02UlWFlw+GIfHx65aB3aXV5FKe7TfT05SlOp9VTUFDd9vL8jn6Y9LIwzkBf3T6J2qXN2rqblqGT4/Tro8gCyEGC+E2C6EKBVC/O4wxw0XQviFEFO7ekw/JpKiTKpBRbXXMuKtitfUUlq7KoMjI97CVWdkc9ebG7j7rR+4880NXHVGNt3jlBNLyEBFiSYOfj2XNb2vY3HPB+lh3M9ztdN4IflTfln+R9kQWIw6DjS5mDEqh9njcnnx+uG8XlxBdYuHixd+RpzNwPQzlSmt08/Mxu7y8PcvK1QDyP/4qoLqJhc7qluY/ep3GHQ65r67GbtHva3jgUY3Sz4vIyXGjMPto0eCDZ0IrsIXrS2jvM7Jo//djsPjZ+Yr64lqdTf9/uJ+iuB/eZ2TO17/nhizgSWfl5GTGn1M30+vlGj+Ok35uUemDDpU/NXOzqGmJVI87kT9fmicfLp0ZyCE0APPAOcDlcA3Qoh3JUnaonLcwwR7JWt0gAanl99e0IfHPtgur4x/e0Ef9DqYPS5XdrGkxJixt1mF63SCC/p1Y/nMkQqRu67I4HB4/LJ7BQ6tql++YbjiuJ76XXw06lXSN7zBqFQXVfp+/KnqF/x930h8GHj23J7MHpck31eizcSitWWyKyUvNYa9DW4crX2l1+9uiFjNr956gLvG9+Oei/uREmPin784g0+21+APIDelWfblbh68dCD3vP0DB8MyodRW3JkJVh6ZWsgLrf0AfnVOrqpGj8Pt56bRObz29W5uOSeX0upm1UlZJwSrZo+WJ92OirfpdIKLBqTRLz34uZRoCzoBd17Ql5LqZqwm/VHtCDXxuJ8WXe0mGgGUSpJUBiCEeA2YDGxpc9xtwApgOBodwmTQ88qXuxW++Pc37eOqM3oqApS/vaAPNW3SOEMyAne8/r3cn3fPQQf90mMV/XmPhbZyFLUtHtWJr7bFAwEf7H0Xtj+FrvoT0oWZd+pHsaR6Ajv9fZg9No8Uezkmg8Dh8Svu6/bz8kmJDrqB9Lqg7PPt5+ViMgTH7vVLiusOyohlytAsRbOV+ycPIDPewn3vbpFfu3J4Fr2SbTw2tZA4i1Eu1mortzB3QgFOr08h8Bb6r+1k6wtIvPBZGfdO6M+zn5QysTBDNSBtNekVrppjCcyGYjY9k6IiMsYevHQgT1wxmF8vP3JOvxYY/unQ1cYgA9gT9nMloOjuLoTIAC4FxqIZgw4TY9Zz5fAsRTDw8csHc3ubLKHHPtgeUeC1q9bOw+9vZda5uXSLtVBZ7+DPrVXP7aWYHk1Te7WU1WU3joiYINMszQxqXATvvgyOCrBlUZl9H1M/7M8BV2jyCdZNhJqyt72vxz/aEZHGOndiAcbWjJecFKX//KYxvRUFYS5vUB7ixeuGK1IxM+Kt/O2TEj7YUsuyG4sitImyEm2kx1l4/MPt3HVhP2aOyaFXchSBQIBoi0G101hlfVD/5/6VwfTYtduruXlMrqLj2PxJBcRbO+/Psj0p8H/fNlprJ6mhoKuNgdpvV9uqpieAuyVJ8ov2VMsAIcRMYCZAVlZWZ43vtMfh8UV0tfL61X3CbQu89jU6VKuXX/myXDXF9GjrEtQmIJfXJ6db5uhLuTF1JZckfIphjxu6nQvDnoCMSfywuZoDrm8jxp4Zb8UbUL+vbQeaItxPz08vkqtuw1fzznYC7vsbXQzLTqDB4SXeZqTR4eGDLbUAbNnfTLTZID9jAKfHzz1v/8DNY3LxBoIKqfE2AwcaPPz2jQ3ceFYvxXdiM+pZtLZMvp5eB6PzUyPkJe57dzPL2qnKPhbayxiraXExMidZW/FryHR1ALkS6BH2cyawr80xRcBrQojdwFTgWSHEJW1PJEnSYkmSiiRJKkpJSemi4Z5+CKGL6Gq1r8GpGkCOMittv1mvV61evmxoUHqgqkkZUAyf5NPjLMwYlcO2A038sLdRUbmsNgFZ9RLVPzzPl0PuYlX+bC5J+Iw1rvH8MOgzGLcGelwKOj2xrRIRbceekWDFZtSrvtdGQRqXN0CT08uq2aPpnRItt1OcNTa3XZlls0HHdS9+w5zXvue6F79BpzsUhG9x+3nh8zLOzEki1mJgWHYCsVYDv7mgLyu+rSAgCfwB0Asd81ZuprzOyaK1ZfgDQUPws97JiriGxaijX1osep26vERNs7LjWTgdrRjXNH80jpauNgbfAHlCiF5CCBNwJfBu+AGSJPWSJKmnJEk9gTeBWyRJ+lcXj+tHQ6PLK6ds9oi30jcthq/K6pjfRtBtzrg8olvVMEPYw1bJ6XEWbj03KHQX1JKxEtXm+NAknx5n4eYxwY5ZAQnWbKtizfYqeWIKn4CSDfXcnbaUwuIBzLE9gKt5P3/Zfz1nbXuJ0p6PcUCXq7hG9zhLhAzFvRP60+L283pxOfMnFSjee/DSgazcuFdxDotRR1K0iZyUaHolK9sp7q+3M6/NOeZPHoDD41W8ZmzdUYRiBTN+1qs1Q0jil6+sZ85rG7jrzQ2M65eGQQcGHYoezfsbXTzzcSkLV5dS0+JWCLbdP3kAyTFGRvRMbGeiNqt+18ei+a9lBGkcLV3qJpIkySeEmEUwS0gPvChJ0mYhxM2t7y/qyuv/FMiMs8opm+E+894ptojuW22njARbMC1VrVvY/ZMHgAiuQkP+5NAkP/3MbBxef0RVb15qNP5A0Gi8eqmg5bsHGWn+CqPOT7N1OPPKbyMhbxrGdD3TWuUOHrpskGJMPRKj6JFgY864PFKizVTUO3h6TSn1Dg/zJhWQHm9m8bXDqLd7SYgykhhl5JZzchUSDfMmFRBtCv5qt82IsRoNPPPxdp67dpjsEmpyuqm3BxTPK95mlHcUMRY9RoOeDXsaiDLpI57r93saeW5tGQ9eOpDsJKuiR4HFqKNbjJmXbxhOdbMbCUGcRc93FY3846vyiID0nHF5BNpp9XUsFeNaRpDG0SKk07DHXFFRkVRcXHyyh3FK8HVZHdOXfh2RufLC9CLcvqC7pLTGzlvfVvKHi/tx8cB0eSL4elcNu2qd7G1wyhN7+DmW3TCC6Uu/luMCAJ+VVmPS61lXVkdAQu5rm51k5ddje/K/NX/jpsQ36Gstxyes1CRMpLbH7TSZe1FZ71ZM2nMnFtA/PYbCHgnydcvrWvi2ogGjTifr9ofITrIyc0xvuc+Axajjz5cO5L+b9jKhsAdOjw+rycCyL8q4b0IBg8LOG0It7vHo1EFISMRbTbKRCUgBtuxrYWFrF7kln5eRYDNx85gcuSpaJyApLLXVYtTJPZrD79Hv99Ps9jMkK44vdh5kcGa8fEyoAlmvQ+5fMX9SAee3Pu9w1u2s5arnv4p4/bWZZzAyR70iWUMjHCHEekmSitTe0yqQT3Oqm93tBovvXrGRJ68YIjc+6RZr5v3NB8ICvjqe/aSU313UT/0cDo9i9ZmVYKOm2aPot3zvhP747ZX0bX6VXpv/yaUZ9VR6UvnLvutZy2VckzOI+1/bgstbR1F2HM9fW8SBJhdWox6dLqhOGk5Vo5vfv/UDN43OiRjThEEZsiEIjfEPb/8QLDh79TvFsS1uv+rzUlsp+wI+1pc3cdebh7qwvXj9cHln0CvZJgeZF60t47KhmWQlWqk46FTEAlzeYJP6R6YWKgzTGTkpQTfV1UNZvLZMcW8hdxLArLG51Ds8dItTdxOdyIpxjZ8emoT1aU5qrFnV75wSY8blDRCQJHmFKqHssVvT7CbOYiTapB60jTYbeOqqIeSnRlPV5GLz/kbZEIDEAONGEr69lp8fOIfh9mfZ6Mhletk8Rm97gedqpzJ2YL5i8i4ub+QXrxSzu87BrFe/447XN+DwKCf8Wvsh49Z2TO0FXPUqvRtMbRs6hBHKnQ9l0zjcUkRBXG2zS5ZjNhsOBa5Dk/e+BidLPi9T6AZZjDrS4iyUVjezp97JzupmJg7K4K1vg1IZGysb2r03i1GHTgT1/70+9d265v/X6Eq0ncFpTkDyR8gaz580AEkKat3YzHpmjMph0ael/G58v9ZdQzBbJdaq5+cjs9lV28L8Sf2xmYzY3T6iLAYcbi+BQIA739zQ2sDFQMVBJ5LPxbSET5iR/A59reU0+218Z/k5G20/55GtyrTN9ibvUAaxyxuI0CbKiLfJwet7J/RXuISG9IhXXRn3TYuVXw+lx9rb6aCmxgGV7Kc99U5eL97DjFE5mAxCoUJqMerITrLxwCUDFLuk+ZMKOGj3RBTFhcYZynpSK167d0J/mpxelq0rp29ajOo4Nf+/RleiGYPTgPBCr9QYC3pdcIXaLdaCJOlYvXU/i68dRoPTS7zVyNLPd7Fu10HmTiwgIEmyG8IXkMhOslJ50Mm1S77mkamFzH13MxcXdGNk7xRFb935kwaQEGWU8/aX/zyDQXVLWdfvRRINTZS5unPf3l+ysvkCrvpZAW99W6mYHLOTrAzPTlSdvENhKotRR3KUSXGv/brFcOu5edz3ziYSbCa5+XyTw0uT082dF/bh0f8ekt6YO7GAFz/fGaHD/9jUwqN+vukqXdVeL94jK5Q+d+0wFq091JGsZ3IUBxocDMmO58XrhlPb4kYnBInRRm58KbKb28wxOWTEW3n2k+D3EN7jIDPeSvlBp6xOajHqyEpsf6WvVQRrdBWaMTjFUQt4zhmXx7J1wbaYz10zlME9kpj5ynrFynhbVQvz3tvMby84tDI16gR3j+8nn8vp9pFgMzF1eJbc0B1CxU+bePn64ZwZtYHpSSsZ+O3XCCR2RY/j9u3n8mnjQCxGvVykVu/w0CPByqrZo9lz0E5Ni4c/vvODarbMsnXlcsaS0aBc1VbUO7ivVcZ5f6OLhauD8Y67LuxDamwUAdyKbB5/IMDEQRlyA/hDOwNfxLNsj4L02IhV/m1j84ixBAvNGh1eVanrO87P59lPSllyXRH1di8H7eoNanolRxFnNfDrcfn8vrVfQL3DQ7TZgNPjZ8nnh3YSCy4fTK9kze2jceLRjMEpzu66oGREeF/Z176p4LKhmTzzcSk2k0G1cGzWubk4vQFizEbmjMslJyWKxBgj35c3ysfGRwX1978sq1Ou3oWLK2M/oOC7W3i1926aA9HsT7+F7THX8MQ6N7+/vB9X2D1EWwxUN7v5w//1w+H2yqmrOqGTJ9ZXvgx2K9ProH9aLLvq7EwZlokkwdMfl/BAm34Gu+rsqhNq93grjQ4fv3/rh4idxhOXD5Z7+7p9AZYXVwRTY48Sg0HHJYUZ5KVGc6DRRVqchYL0OADSYi04PH7VHU6ob0CwH3MzNpO6JlGvpCgGZcYDUNgjXhaP0+ugpsXN32ecwUG7h26xZgrS4zS3j8ZJQTMGJ5Gj0fmps7tVJSN6p0bx1FVDqHeoC8ClxVkUK907L+yDzx8gr9uhNorVjS6eXF3CTaNzgi4b9nFz6ptMjl9LjN6BQ+Qw/8CvGDDq1wzplUqc3c/UYU24vH68fon15fW8XlxJvcPD/EkFmA06Ll74GQuvHNxutszTa5Sra6fHT1lNi/wMoszqipomva7dpi1On182EqHdR6JN6X46EgaDjsIeCRT2UL5e2COBPfUtzJ80gPve3aT4Dl75MrjD2d8Y3DWkx1kidkJ/nTaYQZnx8vca7uIJBCS27G/u9LajGhrHgmYMThJHq/MjEIqVf4LNhMvnxx8AISA52qw6eVYcdCh2C4/+dztLrivi8Q+38/CUQdy9YiMWkx6X10/l5rf4qPBduru/QkLwP/sQfH3upDl6JC9+swnLezt55cYk9DqB3ePnV//4VlY5veP8fKqbXTzzSbBrmcsbIL61mK3tmNrOb8GCNwMXL/zs0DOYNpg7zs9nwYc7lJN7lIkos6Gd1EqzwnWUmWClb1psp31XHi+8XlzO45cPxu3zU1Zrl11jd5yfL1cAh8cChvSIJzsp6rAB3q5qO6qhcSxoxuAkcbQTwcGwVMu2nbEsRh3/mDFCNZto6756Fl41RO4q9vzanbS4fIztm0as1cDjl+bS1/4in/X9Gz1MVTQ5o/k26hq+tc2gz/B+PLt6Bzf8zCiPrd7hxWzQ8eTqEtWK5dlj82hyBjN4GhzeiBXy7eflkx53yHBZjDp+N74vCKF8Bm98zwvTh0VM7gO6x1HZ4FBVA02PtTCub6rCxWM4TGppR6ludlFc3khx+bdykdiUYZn06RbDg6u28of/6yffV73DQ9+0WM7OTz3i6r4r245qaHQUzRicJI52IkiLs1KUHcf0s3IIBCQMekF+ajQb9waVOh2eAM9+ouwt/NHWfZzXr3uEREVitJEFb33MpOFrSK1/FxMOGqJy+cB8K5vNk0FnJtFm4ncrfmB/o4vrzgqOwWLUYTPpaXEHU0cvG5qpGqd4+Yag2maMxSAXbIXG9M+vy7lyeJbitefWlnHPxX0jnoFep2Nc31T2N7pIjjaj1wUDy1VNLpatK1ecY9m6coZkxTMyJznCxdNZhBd7hdxeFqOOGaNyqHd46JcWe0xy0FoRmcaphGYMThLdYi2qjU3aTgRxFj03jc7B5wen149Bb+Dms3uz6NOdbNzbRIMzMtNl4VVDFJr9bq+PtWuWMKHgv/w39ysC9QY+aR5K7LA/8L2jgMc+3IHLuyciL96gE/Lq22zUYTUG3T9CqNcP1La4mTMuF0kKRDSOf+CSATy1piRCt8egiyy+CsVP9jYc4JolX8nneP7aopPSkzdU7BXu0ps9No/lxRVy9k8o5fN4z6sVkWmcLDRjcJLIjLPK+fSye2fyADLjlH2K6x1eGp2+CCG22y/I54alxSRFGSNWlyHN/hidnWuS/s01Sf8hw1SDx5FAsfUGbt84jj3OeN68cCSPLflascIPz4tPijEyc0wONqOeFpePWIuBuRMLONCo3kje37raz00dRGKUkcemFmL3+IgyGYi26Lnj/Hx21tjllpW9kqMw6oXCdRSaDNXcaE+u3s79kwdwb9gze+CSAWQl2Lr0uwov9qpqcmEz6fH6A4wfkHZcRV9aEZnGqYQmR3GS2FrVJBsCaM3tf2cTW6uaFMe5fQHZEISOm/vuZiytEgk2k565E5WSzHnmCp7IWsBX/adzd/oymvxR/G7v7Xw7dBN7es5njzMeoN1WlBlxrQVSrTr9L36xC4tRh9MbYNGnpcRbjcyfpJTInjuxgBfW7sTlDXDnmxsx6oQcMA4AVQ1OapqD1blPrynlubVl1LZ4SIo2sWr2aF6beQarZo+WA+hqbrQzclJ4+uMSuTfBjFE5PLWmhIp6R2d+NaqEVv5n9k6msEcCRT2TyEmJPu6Ju600hmYINE4W2s7gJLG/0UWCLdi7N+QmWrG+kgONLoXvu96hXsjU5PTy4vXD2dvg4tWvynlkygDSGlfRp+F54jatZ3C8gY+aRvCa/WoGDDqPkUOjMRotZCYEmD0ul4AE0e1k51TUOymvc1LV5GLJ52XceWEfLAY9Vc1uPD6JWrsXvV7w3LXDqG5y0z3ewt6DDmpaPPL4vq9sxGrUy8Vxz19bxC9eUVbnPvbBdpbPHKlaUavmT9frUC3+0gKuGhrHj7YzOElkJlj51dk5ssiaXsCvzs4hI0HpJkqONqmKmiVGmbjxpW9Itzg4z7+YMzcOYcTem/E27uTpqsu5ZP9yzGPf5qKzJ7F4bRm/Xv49P1/yFcW7G3ijuJIXPivDqBcRjWRmj83jrW8rsRh1cjvN6NZeCEnRRqafmc2Sz8u4750t/PKV9dS2uDHoBa+v38NlQzPl8/gD8OTqEh68dAAzx+SweV+TqlFzeNTVRdVE2ULyFm2fhRZw1dA4frSdwUnCZtRj90Q2iLEZld3FJFBNp7TaN/FI2kMMXv8VQ1PdNFgG8tudN/B23Wj8BM+xvrxe0acg6HcPulme+bgUk15HlEmvaCQTyp+fO7GAZreX/G4x6ACX14dBr5fHEX6+Jy4fzJShWXgDfkVBVjCo7CEvNUYRGwgRCharoeZPz0qwaQFXDY0uQjMGJ4maFg+vfVMRITMxNCuBnNRDx1U1uuV0Sh0+8t3/JXfXHPrtK6FvgomauPGss/yCMn8eb9Yp3ScB6fCqodUtbv72aVmrq8pNbmoMV43IYnCPOO59ZxPldU7Z+KTFJrKzVl0qotHlZd7Kzbx8wwhmjMrhlS/LZdG13XUOlnxexu/H940oJjvSRK4myqYFXDU0uoYuNwZCiPHAkwTbXr4gSdJDbd6fDNxPMM7oA34tSdLnXT2uk43L61OVmXB5lQJrqbFmJHcN5i3Pc2Xif0k1NlCtS6Qy8y6mrB7GU9eN43dLv+am0UT62EXka+GqoVajXjVV87GphXIKaGj1v/jaYUS3IxVhNRnk1NJw0bXwHcJf3t/Gmzefybi+o6lpOfaJXFPt1NDoGro0ZiCE0APPABcB/YGrhBD92xy2GiiUJGkwcCPwQleOqSMEAsEewOt21lJW03LYxuMdpT2BOaspzD7XFdN7+3X8L386s7stZ68nldmV9/BJv2IqM39DlTuKBmcwIyikkR/uY0+0mbj9vHzFa3PGHYoJxJgNETGDOePy2NMmO8flDeD0BKhrcUdkLs2dEMwishiDDXVWzR7NomuGKnYIoXO0uH30TtUyZzQ0TkW6emcwAiiVJKkMQAjxGjAZ2BI6QJKklrDjoyCib/tJ4Wi1g46VmpZ22lU2t0DZSti2ABo2kqCz8HVgLPoB97CP3kyxGfnHl7uYfmYOgNyFK6SLE1IIzU+N4c+rtgLIr/2sdzIur49pRZn4A8HJvHu8RSH9kJFgZdkXuxTjshh1JEUb8UsBWpxenrxiCG6/Hx2CxWt3sqM6KOTWLdZEdlJwxf7r5d9H7CC0QK+GxqlLV2cTZQB7wn6ubH1NgRDiUiHENuDfBHcHJ532tINCLSOPl5DAnPyzoZ4/dH+J87YNgy+vB89BKPwzO8/aRmW/57l2hZ05r33PL19ZzwUF3TloD664TYZDGUH7G4OpoNFmAwiod3jk16xGPSBx37ub6RZrYcnnZdTZvSz4cIfcgcsfgL9+sJ1bzlHuFh64ZABmfbBa+L73tvDLv6/nhbVlBIBrRmbz8g0jiLPqCASCRlJrz6ihcfrR1TsDtSV0xMpfkqS3gbeFEGMIxg/OiziREDOBmQBZWVnHNaiQdHSj04M/ALUtbtLjrBSkx8oCZ10vIhZg/qQC3vjgLW5KXM7Y2G8wCj/N1hEYBy+BzIkgdNTtrMXra1Gs3r0+PzGWYB5+k8uPzahXvG/W6zhod0do+GQmWJkzLp+X/reLGaNycHh8qnn7Tq+fRdcMo97uYVedHaNeYPf68PoPSUhv3NskN6F/+LKBPNGaRtqr1f2jBXo1NE4vutoYVALh8mGZwL72DpYkaa0QorcQIlmSpNo27y0GFgMUFRUdsysp5P558fOdTD+zJ6Wt8ghb9zexp97B+P5pGAy6rhUR87tJrnmThG1PcXmvrfiElX1xU3l0zySmn3khw3skyYd6/RLzVm6NGMeS64r4+4wzqGl24/b5Mel1uHwBuWJ48uAMVQ2fbrFm7nm7hY17m3ht5gjVe4yzGtlX7+Dx1YdaMb4wvUjRazj8+Ip6J/UOD7aweIcW6NXQOL3oajfRN0CeEKKXEMIEXAm8G36AECJXiGCyoxBiKGAC6rpqQCH3z21j89jb4FLII+w56GB7dVAOoktcHY59sPE+eCeLnNJbEL5m7t17M4U/vMyYtdfy3q44alureEM4vT712EKLh2uWfMWv/vEtCz4qQa8TrFhfyZLPy7h6RDbJUaaIQK8v4Ccr8dB9WY167p+slJW4f/IAftjbKBuC0PWcHj+9kiOfyeyxeazcuJc54/LoFms+9mejoaFxUunSnYEkST4hxCzgvwRTS1+UJGmzEOLm1vcXAVOA6UIIL+AErpAkqdODyCHX0I6qZm4anYNOp1MtoBqYEWx32GmuDkmCmv/9f3tnHiVVeabx39MbDQ3N0m0DgmzajSwCKmI0gg6gYnTEuGTIUYNGj5NkAmhGY5w4SYweMy6jUVGJosHBiTpKBpch44Ioo8QFVBBlFQRBQEDWbqC3d/64t5qq7qpuECpdVf3+zrmn7v3uUt/T1VXvvd/yvLB8MnwxA6wGup3H4oLLufj5AvZW7Zean5tFu/zYj6SoIH7ympWbdzeo+90XD2bJxl1Mm/c5ANeM6EO/LoVU1Rr/MW8VN5zdL0bXlt37mDxnBX+4/EQWrNlGTW2QWS367xJ5v04FeXXn9p0wnKWbdmK1sG57BWOHdKO0c9tGE7k7jpPaJH2egZnNAmbVK5sStX4HcEcy6xBvZNCAroVx77h3RyVSP6Smjuo9sOYpWP4AbPsIcjtA30lQ9hNo24eatduYNGprg5nFbVvFfiQ79lZy/Vl9ufuVZTG5Ce6fvaJB3Zdu2hVrZT17JXdfMoibZy7m138/gC7tW8Xo+uTLnazZuoeVm3aTn5PN/a8HiWvqz3ieOLKUPeH8h6wscXRJW3oXB86ixe3yvE/AcTKAFjEDuf7IoI5t8hKatBUVHFzu3AaUr4UVD8FnU2HfVmg/EIb9AXpdCjn775y3lO+jbaucmI7ftq1y2FqxL+ZyBXm5TH/n85jO4F17KtlWEduclCitZEFeDled1ocpb65kWK9hMftLCoOnjpLCfO5+ZWnMbOhJo0rp3rENSzfu4pn5azmx5+CYc71PwHEyixZhVBc9MiiSOnLJhh0NJmlNHFlKRWV1Y5eKjxlsmgNzL4QXesOSu6DkdBg1B76zCI65JiYQALTJy2bqW6tihnVOfWsVrXNivYlqamv40YhjeOytoG/jsbdWcUS7fG7/7nENLKR7FxfElF03uozfvvQpD85ZyZqte/hq196Ya3donc0t5w9gw/YK/mFoj7r3uG/2Csxg/fbASmLcST0oaef9AY6TybSIJ4OSqKTxkZSNVw/vw4sL18fccT8zfy33jzv+wC9cXQ6rnwz6A3YshlZF0O/nUPpjKGh8+GubvByuHV3G6i3BaKacLLh2dBkF9ZqJcrKzmTI3Nq3l72cv597vDWHWxOGs2VrO0o272F5RiQR3XTyYyuoa1nxdwbR5n9d1AkdyH0RzdHEhX+7YS8+iNtw4Y1GDv8X1Zx3LNSP6UJCXXffE4DhOZtIigkFWVk1dhqxIysYZC9Y1SOr+27ED6de5sOkL7l4Fyx+Ezx6Hqu3QcQic/Bj0/D7ktG7qbAAqKqvZvGtfjGvpz84so0u9ETlbd1cm8PDfxwk9O9GjYxu2lldy88z92b9uHTuQI9q2qmtKijz1VNXE9pHk5GQx/OgSlm7ayYSRpTHX+NV5/Vm/vYKaWnj4zVWUdWlHr2JvEnKcTKVFBINt5bXsqazmmhF96N+1MK59w0k9O3Hz8x8ztGfH+O3gZrDxVVj2AHz5P6BsOOoi6DsBik/lYG+dq2upc/CEIEDd8+pypv5gaMxxR3ZoHbdvo2to/bx2W0Xdj3jkOv/6/GImjSptcKc/ZmCXBvXIycliYLcO9O/anhN6dGTN1nI+/GI7D7y+Muapwq0kHCezaRHBYFdlNbf/ZSl7q2q5cUzfutEyEauGSaNKqaiqqWtXjwkGVbtg1ROwYjLsXAb5JTDwZjjmH6FNA2eNA2ZPZc0BJXsZ0LWQ2y4YGHPXftsFAxlwZDAENtFM6aM6teH6ZxcetF10r6IC9lTVxjxVuJXEgRMZwrxp5146F/ooKyd9aBHBYF/V/h/e8soaZixY18Cq4erTesfeAe9cHvQFrJoG1bugaBicMh16XALZh96ZGslg1mA0U9vY0Uw5OVlcMLgbpSVt2bhjL13a5zOga/s624xEM6X7dSlk1jeYI+FWEt+cZJsbOk4yaRHB4Ih2sRO34nn4l3Vuxz2XDKLXvjdgzmTY8DJk5UKP70HZBCg++bDWqcaM60aXce9r+5O9XDe6jNo48+1ycrIYfFTHmNzIESIzpev/APUuLqi72z9YfNjoNyORueGxE4f739JJeVpEMNhWUVn3wztjwTr+5Zxj2VJeSa0FCWA65++l5MuHOG3ndLT4M2jdFY67JRgS2rphO/vhoKigFX96b03ME8qf3lvDmf07H9R1/E4+dUi+uaHjJI8WEQyKCvK55b1Pueq0Pgzq3p6vyyt5ZO4qumetZmKX5zinw1/Jtb1BR/Cg2+CoCyH7ECefNUGvogJuHNPvsOTz9Tv51CCp5oaOk2SUBBugpDN06FCbP3/+AR9fXV3LzIXruXnmYm4482gWvjONKzvO4PiC5VTW5vDa7lPofuqvGTRoVBJr3ZBIZ6Pf0WcG3mfgpDqSFpjZ0Lj7WkIwgCAgfLFoOl1X3kh+9SY2V3Xg6a/P4o9bxvJ1TXse/cGJnNk/OU1CTsvBA7yTyjQWDFpEMxEEnbC9Oxezd303frZ0PM9v/TY1BDNy83Oz6O1DJ53DgDfZOelKi/AmqqPbueSdO59RZ19Lbm4uEASCf79kiH95Hcdp0bSYJ4MIWVninIFd6NfVR984juNEaHHBAPxR3nEcpz4tq5nIcRzHiYsHA8dxHCf5wUDSGEnLJK2U9Is4+y+VtChc5kkaHO86juM4TvJIajCQlA08CJwD9Ae+L6l/vcNWA6eb2SDgVuCRZNbJcRzHaUiynwyGASvNbJWZVQJPA2OjDzCzeWa2Ldx8B+ie5Do5juM49Uh2MOgGfBG1vS4sS8RVwF/i7ZB0jaT5kuZv3rz5MFbRcRzHSfbQ0niD9+P6X0j6O4JgcFq8/Wb2CGETkqTNktaEu4qBLYde1WYlEzRAZujIBA2QGToyQQOklo6eiXYkOxisA6Jd+LsDX9Y/SNIgYCpwjpltbeqiZnZE1LnzE3ltpAuZoAEyQ0cmaIDM0JEJGiB9dCS7meh9oFRSb0l5wDjghegDJPUA/gxcbmbLk1wfx3EcJw5JfTIws2pJPwVeBrKBx83sE0k/CvdPAX4FFAEPKUgqX50OUdRxHCeTSLodhZnNAmbVK5sStX41cPUhvEUmDEXNBA2QGToyQQNkho5M0ABpoiMt8xk4juM4hxe3o3Acx3E8GDiO4zhpGAwkZUv6UNJL4XYnSa9KWhG+dmzuOjaFpA6SnpO0VNISSaekmw5J10n6RNJiSU9Jyk8HDZIel/SVpMVRZQnrLemm0FdrmaSzm6fWsSTQcFf4/7RI0n9L6hC1L+U0QHwdUfuul2SSiqPKUk5HIg2SJoT1/ETSnVHlKachQtoFA2ASsCRq+xfAbDMrBWaH26nOfcD/mtmxwGACPWmjQ1I3YCIw1MwGEowUG0d6aJgGjKlXFrfeoY/WOGBAeM5Dod9WczONhhpeBQaGHl/LgZsgpTVAfB1IOgo4E1gbVZaqOqZRT0M4gXYsMMjMBgB3h+WpqgFIs2AgqTtwLsEEtQhjgSfC9SeAC/7G1TooJBUCI4DHAMys0sy2k2Y6CEaitZaUA7QhmEyY8hrMbC7wdb3iRPUeCzxtZvvMbDWwksBvq1mJp8HMXjGz6nAz2uMrJTVAws8C4F7g58S6FaSkjgQafgz8m5ntC4/5KixPSQ0R0ioYAL8n+CepjSrrbGYbAMLXkmao18HQB9gM/DFs7poqqYA00mFm6wnudtYCG4AdZvYKaaShHonqfbDeWqnCD9nv8ZVWGiSdD6w3s4X1dqWTjjJguKR3Jb0p6aSwPKU1pE0wkHQe8JWZLWjuuhwiOcAJwMNmdjxQTmo2pyQkbFMfC/QGjgQKJF3WvLVKCgfsrZUqSPolUA38Z6QozmEpqUFSG+CXBBNRG+yOU5aSOgi+4x2BbwE3AP+lYEZtSmtIm2AAfBs4X9LnBFbYIyU9CWyS1BUgfP0q8SVSgnXAOjN7N9x+jiA4pJOO0cBqM9tsZlUEdiKnkl4aoklU7wPy1koVJI0HzgMutf0TiNJJw9EENxgLw+95d+ADSV1ILx3rgD9bwHsELRnFpLiGtAkGZnaTmXU3s14EnTCvm9llBF5H48PDxgPPN1MVDwgz2wh8IalvWDQK+JT00rEW+JakNuEdzyiCTvB00hBNonq/AIyT1EpSb6AUeK8Z6tckksYANwLnm1lF1K600WBmH5tZiZn1Cr/n64ATwu9M2ugAZgIjASSVAXkErqWprcHM0m4BzgBeCteLCEaArAhfOzV3/Q6g/kOA+cAign+cjummA7gFWAosBqYDrdJBA/AUQT9HFcGPzVWN1Zug2eIzYBmBq26qalhJ0B79UbhMSWUNiXTU2/85UJzKOhJ8FnnAk+F34wNgZCpriCxuR+E4juOkTzOR4ziOkzw8GDiO4zgeDBzHcRwPBo7jOA4eDBzHcRw8GDiO4zh4MHDSHEkTQxvw9ZImN3d9HCdd8WDgpDs/Ab5DMJnnkAldWP/m5zpOc+PBwElbJE0hcIF9gWAWd6S8p6TZYaKX2ZJ6NFE+TdI9kuYAdyR4r2GS5oVOs/MidiKSrpD0rKQXgVckFYQJT94Pjx0bHtdL0v9J+iBcTm1EV1dJcyV9pCB50PCw/EpJy0MnzEcjT0Jh/R+WNEfSKkmnh3VYImnaof+lnRZBc0+B9sWXQ1kILQuAK4DJYdmLwPhw/YfAzCbKpwEvAdmNvE8hkBOujwZmhOtXENgQdAq3bwcuC9c7ECSaKSDI+ZAflpcC8xt5r38GfhmuZwPtgK4EnlBHENgdvB2ldxqBeaMI3GR3AscR3OwtAIY09+fkS+ov/ljrZCKnABeG69OBO5soB3jWzGoauWZ74AlJpQS2w7lR+141s0iCk7MI3HWvD7fzgR4E7pSTJQ0Bagg87xPxPvC4pFyCgPWRpFHAG2a2GUDSM/Wu8aKZmaSPgU1m9nF43CdALwK/IsdJiAcDpyWQyIArury8iWvcCswxs+9K6gW8keBcAReZ2bLokyX9BthEkOY0C9ibsLJmcyWNIMjqN13SXQR3+40Zie0LX2uj1iPb/j13msT7DJxMZB6BzTnApcBbTZQfCO2B9eH6FY0c9zIwIbT2RtLxUedvMLNa4HKC5p+4SOpJkMjpUYL0qCcA7wJnSCoKnxguOYi6O06TeDBwMpGJwJWSFhH88E5qovxAuBP4naS3aeSHnOAJIhdYJGlxuA3wEDBe0jsEzTuNPYmcAXwk6UPgIuA+C9Jx/gb4K/AagTWy4xw23MLacdIQSVcAQ83sp81dFycz8CcDx3Ecx58MHCcaSVfSsPnobTP7pyS813EEo5qi2WdmJx/u93KcpvBg4DiO43gzkeM4juPBwHEcx8GDgeM4joMHA8dxHAf4f8TAQMIVDZGKAAAAAElFTkSuQmCC",
163 | "text/plain": [
164 | ""
165 | ]
166 | },
167 | "metadata": {
168 | "needs_background": "light"
169 | },
170 | "output_type": "display_data"
171 | }
172 | ],
173 | "source": [
174 | "y = 52643 + 4030 * df['floor_area_sqm']\n",
175 | "sns.scatterplot(y='resale_price', x='floor_area_sqm', data=df_tampines)\n",
176 | "sns.lineplot(y=y, x='floor_area_sqm', data=df_tampines, color='orange')"
177 | ]
178 | },
179 | {
180 | "attachments": {},
181 | "cell_type": "markdown",
182 | "metadata": {},
183 | "source": [
184 | "Note that in the above code, we created a straight line equation with the following coefficients:\n",
185 | "$$\\beta_0 = 52643$$\n",
186 | "and\n",
187 | "$$\\beta_1 = 4030$$\n",
188 | "\n",
189 | "\n",
190 | "In machine learning, we call $\\beta_0$ and $\\beta_1$ as the model *coefficents* or *parameters*. What we want is to use our training data set to produce estimates $\\hat{\\beta}_0$ and $\\hat{\\beta}_1$ for the model coefficients. In this way, we can **predict** future resale prices by computing\n",
191 | "\n",
192 | "$$\\hat{y} = \\hat{\\beta}_0 + \\hat{\\beta}_1 x$$\n",
193 | "\n",
194 | "where $\\hat{y}$ indicates a prediction of $Y$. Note that we use the *hat* symbol to denote the estimated value for an unknown parameter or coefficient or to denote the predicted value. The predicted value is also called a hypothesis."
195 | ]
196 | },
197 | {
198 | "attachments": {},
199 | "cell_type": "markdown",
200 | "metadata": {},
201 | "source": [
202 | "## Cost Function\n",
203 | "\n",
204 | "In order to find the values of $\\hat{\\beta}_0$ and $\\hat{\\beta}_1$, we will apply optimization algorithm that minimizes the error. The error caused by the difference between our predicted value $\\hat{y}$ and the actual data $y$ is captured in a *cost function*. Let's find our cost function for this linear regression model.\n",
205 | "\n",
206 | "We can get the error by taking the difference between the actual value and our hypothesis and square them. The square is to avoid cancellation due to positive and negative differences. This is to get our absolute errors. For one particular data point $i$, we can get the error square as follows.\n",
207 | "\n",
208 | "$$e^i = \\left(\\hat{y}(x^i) - y^i\\right)^2$$\n",
209 | "\n",
210 | "Assume we have $m$ data points, we can then sum over all data points to get the Residual Sum Square (RSS) of the errors.\n",
211 | "\n",
212 | "$$RSS = \\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)^2$$\n",
213 | "\n",
214 | "We can then choose the following equation as our cost function.\n",
215 | "\n",
216 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)^2$$\n",
217 | "\n",
218 | "The division by $m$ is to get an average over all data points. The constant 2 in the denominator is make the derivative easier to calculate.\n",
219 | "\n",
220 | "The learning algorithm will then try to obtain the constant $\\hat{\\beta}_0$ and $\\hat{\\beta}_1$ that minimizes the cost function.\n",
221 | "\n",
222 | "$$\\begin{matrix}\\text{minimize} & J(\\hat{\\beta}_0, \\hat{\\beta}_1)\\\\\n",
223 | "\\hat{\\beta}_0, \\hat{\\beta}_1\\\\ \\end{matrix}$$"
224 | ]
225 | },
226 | {
227 | "attachments": {},
228 | "cell_type": "markdown",
229 | "metadata": {},
230 | "source": [
231 | "## Gradient Descent\n",
232 | "\n",
233 | "One of the algorithm that we can use to find the constants by minimizing the cost function is called *gradient descent*. The algorithm starts by some initial guess of the constants and use the gradient of the cost function to make a prediction where to go next to reach the bottom or the minimum of the function. In this way, some initial value of $\\hat{\\beta}_0$ and $\\hat{\\beta}_1$, we can calculate the its next values using the following equation.\n",
234 | "\n",
235 | "$$\\hat{\\beta}_j = \\hat{\\beta}_j - \\alpha \\frac{\\partial}{\\partial \\hat{\\beta}_j} J(\\hat{\\beta}_0, \\hat{\\beta}_1)$$\n",
236 | "\n",
237 | "In order to understand the above equation, let's take a look at a two countour plot below.\n",
238 | "\n",
239 | "\n",
240 | "\n",
241 | "The contour plot shows the minimum somewhere in the centre. The idea of gradient descent is that we move the fastest to the minimum if we choose to move in the direction with the steepest slope. The steepest slope can be found from the gradient of the function. Let's look at point $x_0$ in the figure. The gradient in the direction of $\\beta_0$ is non zero as can be seen from the contour since it is perpendicular to the contour lines. On the other hand, the gradient in the direction of $\\beta_1$ is zero as it is parallel with the contour line at $x_0$. Recall that contour lines show the points with the same value. When the points have the same values, the gradient is zero. We can then substitute this into the above equation.\n",
242 | "\n",
243 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\frac{\\partial}{\\partial \\hat{\\beta}_0} J$$\n",
244 | "\n",
245 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 - \\alpha \\frac{\\partial}{\\partial \\hat{\\beta}_1} J$$\n",
246 | "\n",
247 | "The partial derivative with respect to $\\beta_0$ is non-zero while the derivative with respect to $\\beta_1$ is zero. So we have the following:\n",
248 | "\n",
249 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\times m $$\n",
250 | "\n",
251 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 $$\n",
252 | "\n",
253 | "where $m$ is the gradient in $\\beta_0$ direction which is the partial derivative $\\partial J/\\partial\\hat{\\beta}_0$. If the optimum is a minima, then $m < 0$, a negative value. Now, we can see how the next point increases in $\\beta_0$ direction but not in $\\beta_1$ direction at $x_0$.\n",
254 | "\n",
255 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 + \\delta $$\n",
256 | "\n",
257 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 $$\n",
258 | "\n",
259 | "When the gradient in the $\\beta_1$ direction is no longer zero as in the subsequent steps, both $\\beta_0$ and $\\beta_1$ increases by some amount.\n",
260 | "\n",
261 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 + \\delta_0 $$\n",
262 | "\n",
263 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 + \\delta_1 $$\n",
264 | "\n",
265 | "We can actually calculate the derivative of the cost function analytically.\n",
266 | "\n",
267 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)^2$$\n",
268 | "\n",
269 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_j} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{\\partial}{\\partial \\hat{\\beta}_j} \\frac{1}{2m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)^2$$\n",
270 | "\n",
271 | "We can substitute our straight line equation into $\\hat{y}$ to give the following.\n",
272 | "\n",
273 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_j} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{\\partial}{\\partial \\hat{\\beta}_j} \\frac{1}{2m}\\Sigma_{i=1}^m\\left(\\hat{\\beta}_0 + \\hat{\\beta}_1 x^i - y^i\\right)^2$$\n",
274 | "\n",
275 | "Now we will differentiate the above equation with respect to $\\hat{\\beta}_0$ and $\\hat{\\beta}_1$. \n",
276 | "\n",
277 | "Let's first do it for $\\hat{\\beta}_0$. \n",
278 | "\n",
279 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_0} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{\\beta}_0 + \\hat{\\beta}_1 x^i - y^i\\right)$$\n",
280 | "\n",
281 | "or\n",
282 | "\n",
283 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_0} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)$$\n",
284 | "\n",
285 | "Now, we need to do the same by differentiating it with respect to $\\hat{\\beta}_1$.\n",
286 | "\n",
287 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_1} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{\\beta}_0 + \\hat{\\beta}_1 x^i - y^i\\right) x^i$$\n",
288 | "\n",
289 | "or\n",
290 | "\n",
291 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_1} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right) x^i$$\n",
292 | "\n",
293 | "Now we have the equation to calculate the next values of $\\hat{\\beta}_0$ and $\\hat{\\beta}_1$ using gradient descent.\n",
294 | "\n",
295 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)$$\n",
296 | "\n",
297 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)x^i$$"
298 | ]
299 | },
300 | {
301 | "attachments": {},
302 | "cell_type": "markdown",
303 | "metadata": {},
304 | "source": [
305 | "## Matrix Operations\n",
306 | "\n",
307 | "We can calculate these operations using matrix calculations. \n",
308 | "\n",
309 | "### Hypothesis\n",
310 | "\n",
311 | "Recall that our hypothesis (predicted value) for one data point was written as follows.\n",
312 | "\n",
313 | "$$\\hat{y}(x^i) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x^i$$\n",
314 | "\n",
315 | "If we have $m$ data points, we will then have a set of equations.\n",
316 | "\n",
317 | "$$\\hat{y}(x^1) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x^1$$\n",
318 | "$$\\hat{y}(x^2) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x^2$$\n",
319 | "$$\\ldots$$\n",
320 | "$$\\hat{y}(x^m) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x^m$$\n",
321 | "\n",
322 | "We can rewrite this in terms of matrix multiplication. First, we write our independent variable $x$ as a column vector.\n",
323 | "\n",
324 | "$$\\begin{bmatrix}\n",
325 | "x^1\\\\\n",
326 | "x^2\\\\\n",
327 | "\\ldots\\\\\n",
328 | "x^m\n",
329 | "\\end{bmatrix}$$\n",
330 | "\n",
331 | "To write the system equations, we need to add a column of constant 1s into our independent column vector.\n",
332 | "\n",
333 | "$$\\mathbf{X} = \\begin{bmatrix}\n",
334 | "1 & x^1\\\\\n",
335 | "1 & x^2\\\\\n",
336 | "\\ldots & \\ldots\\\\\n",
337 | "1 &x^m\n",
338 | "\\end{bmatrix}$$\n",
339 | "\n",
340 | "and our constants as a column vector too.\n",
341 | "\n",
342 | "$$\\mathbf{\\hat{b}} = \\begin{bmatrix}\n",
343 | "\\hat{\\beta}_0\\\\\n",
344 | "\\hat{\\beta}_1\n",
345 | "\\end{bmatrix}$$\n",
346 | "\n",
347 | "Our system equations can then be written as\n",
348 | "\n",
349 | "$$\\mathbf{\\hat{y}} = \\mathbf{X} \\times \\mathbf{\\hat{b}}$$\n",
350 | "\n",
351 | "The result of this matrix multiplication is a column vector of $m\\times 1$. Note that in the above matrix equation, we use $\\mathbf{\\hat{y}}$ to denote the column vector of *predicted* value or our hypothesis. "
352 | ]
353 | },
354 | {
355 | "attachments": {},
356 | "cell_type": "markdown",
357 | "metadata": {},
358 | "source": [
359 | "### Cost Function\n",
360 | "\n",
361 | "Recall that the cost function is written as follows.\n",
362 | "\n",
363 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}\\Sigma^m_{i=1}\\left(\\hat{y}(x^i)-y^i\\right)^2$$\n",
364 | "\n",
365 | "We can rewrite the square as a multiplication instead and make use of matrix multplication to express it.\n",
366 | "\n",
367 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}\\Sigma^m_{i=1}\\left(\\hat{y}(x^i)-y^i\\right)\\times \\left(\\hat{y}(x^i)-y^i\\right)$$\n",
368 | "\n",
369 | "Writing it as matrix multiplication gives us the following.\n",
370 | "\n",
371 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}(\\mathbf{\\hat{y}}-\\mathbf{y})^T\\times (\\mathbf{\\hat{y}}-\\mathbf{y})$$"
372 | ]
373 | },
374 | {
375 | "attachments": {},
376 | "cell_type": "markdown",
377 | "metadata": {},
378 | "source": [
379 | "### Gradient Descent\n",
380 | "\n",
381 | "Recall that our gradient descent equations update functions were written as follows. \n",
382 | "\n",
383 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)$$\n",
384 | "\n",
385 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)x^i$$\n",
386 | "\n",
387 | "And recall that our independent variable is a column vector with constant 1 appended into the first column.\n",
388 | "\n",
389 | "$$\\mathbf{X} = \\begin{bmatrix}\n",
390 | "1 & x^1\\\\\n",
391 | "1 & x^2\\\\\n",
392 | "\\ldots & \\ldots\\\\\n",
393 | "1 &x^m\n",
394 | "\\end{bmatrix}$$\n",
395 | "\n",
396 | "Transposing this column vector results in\n",
397 | "\n",
398 | "$$\\mathbf{X}^T = \\begin{bmatrix}\n",
399 | "1 & 1 & \\ldots & 1\\\\\n",
400 | "x^1 & x^2 & \\ldots & x^m\\\\\n",
401 | "\\end{bmatrix}$$\n",
402 | "\n",
403 | "Note that we can write the update function summation also as a matrix operations.\n",
404 | "\n",
405 | "$$\\mathbf{\\hat{b}} = \\mathbf{\\hat{b}} - \\alpha \\frac{1}{m}\\mathbf{X}^T \\times (\\mathbf{\\hat{y}} - \\mathbf{y})$$\n",
406 | "\n",
407 | "Substituting the equation for $\\mathbf{\\hat{y}}$, we get the following equation.\n",
408 | "\n",
409 | "$$\\mathbf{\\hat{b}} = \\mathbf{\\hat{b}} - \\alpha \\frac{1}{m}\\mathbf{X}^T \\times (\\mathbf{X} \\times \\mathbf{\\hat{b}} - \\mathbf{y})$$\n",
410 | "\n",
411 | "In this notation, the capital letter notation indicates matrices and small letter notation indicates vector. Those without bold notation are constants."
412 | ]
413 | },
414 | {
415 | "attachments": {},
416 | "cell_type": "markdown",
417 | "metadata": {},
418 | "source": [
419 | "## Metrics\n",
420 | "\n",
421 | "After we build our model, we usually want to evaluate how good our model is. We use metrics to evaluate our model or hypothesis. To do this, we should split the data into two:\n",
422 | "- training data set\n",
423 | "- test data set\n",
424 | "\n",
425 | "The training data set is used to build the model or the hypothesis. The test data set, on the other hand, is used to evaluate the model by computing some metrics.\n"
426 | ]
427 | },
428 | {
429 | "attachments": {},
430 | "cell_type": "markdown",
431 | "metadata": {},
432 | "source": [
433 | "### Mean Squared Error\n",
434 | "\n",
435 | "One metric we can use here is called the mean squared error. The mean squred error is computed as follows.\n",
436 | "\n",
437 | "$$MSE = \\frac{1}{n}\\Sigma_{i=1}^n(y^i - \\hat{y}^i)^2$$\n",
438 | "\n",
439 | "where $n$ is the number of predicted data points in the *test* data set, $y^i$ is the actual value in the *test* data set, and $\\hat{y}^i$ is the predicted value obtained using the hypothesis and the independent variable $x^i$ in the *test* data set."
440 | ]
441 | },
442 | {
443 | "attachments": {},
444 | "cell_type": "markdown",
445 | "metadata": {},
446 | "source": [
447 | "### R2 Coefficient of Determination\n",
448 | "\n",
449 | "Another metric is called the $r^2$ coefficient or the coefficient of determination. This is computed as follows.\n",
450 | "\n",
451 | "$$r^2 = 1 - \\frac{SS_{res}}{SS_{tot}}$$\n",
452 | "\n",
453 | "where\n",
454 | "\n",
455 | "$$SS_{res} = \\Sigma_{i=1}^n (y_i - \\hat{y}_i)^2$$ \n",
456 | "\n",
457 | "where $y_i$ is the actual target value and $\\hat{y}_i$ is the predicted target value.\n",
458 | "\n",
459 | "$$SS_{tot} = \\Sigma_{i=1}^n (y_i - \\overline{y})^2$$\n",
460 | "\n",
461 | "where \n",
462 | "$$ \\overline{y} = \\frac{1}{n} \\Sigma_{i=1}^n y_i$$\n",
463 | "and $n$ is the number of target values.\n",
464 | "\n",
465 | "This coefficient gives you how close the data is to a straight line. The closer it is to a straight line, the value will be close to 1.0. This means there is a correlation between the independent variable and the dependent variable. When there is no correlation, the value will be close to 0.\n",
466 | "\n",
467 | "In your problem sets, you will work on writing a code to do all these tasks."
468 | ]
469 | }
470 | ],
471 | "metadata": {
472 | "kernelspec": {
473 | "display_name": "Python 3.8.5 ('base')",
474 | "language": "python",
475 | "name": "python3"
476 | },
477 | "language_info": {
478 | "codemirror_mode": {
479 | "name": "ipython",
480 | "version": 3
481 | },
482 | "file_extension": ".py",
483 | "mimetype": "text/x-python",
484 | "name": "python",
485 | "nbconvert_exporter": "python",
486 | "pygments_lexer": "ipython3",
487 | "version": "3.8.5"
488 | },
489 | "vscode": {
490 | "interpreter": {
491 | "hash": "ac37b77c3c0f43e60bec193f0626743b91dd65d8d4aeca5713e457ab7e777e4c"
492 | }
493 | }
494 | },
495 | "nbformat": 4,
496 | "nbformat_minor": 4
497 | }
498 |
--------------------------------------------------------------------------------
![\"drawing\"](\"https://data-driven-world.github.io/assets/images/week10/confusion_matrix_accuracy.png\")
\n",
116 | "\n",
117 | "Given the accuracy, we can also calculate another metric called **error rate**. In fact, the error rate is just given by the following:\n",
118 | "\n",
119 | "$$\\text{error rate} = 1 - \\text{accuracy}$$\n",
120 | "\n",
121 | "So in our example, the error rate is $1 - 0.8 = 0.2$ which is 20%."
122 | ]
123 | },
124 | {
125 | "attachments": {},
126 | "cell_type": "markdown",
127 | "metadata": {},
128 | "source": [
129 | "### Precision\n",
130 | "\n",
131 | "In precision, we put more attention into the positive cases. We are interested in how many of the positive cases are detected accurately.\n",
132 | "\n",
133 | "| actual\\predicted | a cat | not a cat |\n",
134 | "|------------------|-------|-----------|\n",
135 | "| a cat | 11 | 3 |\n",
136 | "| not a cat | 2 | 9 |\n",
137 | "\n",
138 | "$$\\text{precision} = \\frac{11}{11 + 2} = \\frac{11}{13} = 0.846$$\n",
139 | "\n",
140 | "This means that we have a precision of about 85\\%. Out of all the total 13 cases detected positive, 11 of them is correct. Obviously we want our precision to be as high as possible. We can write precision formula as follows.\n",
141 | "\n",
142 | "$$\\text{precision} = \\frac{\\text{TP}}{\\text{Total Predicted Positives}} = \\frac{\\text{TP}}{\\text{TP} + \\text{FP}}$$\n",
143 | "\n",
144 | "The calculation of precision can be seen as a fraction between the green circle and the blue circle in the image below.\n",
145 | "\n",
146 | "![\"drawing\"](\"https://data-driven-world.github.io/assets/images/week10/confusion_matrix_precision.png\")
"
147 | ]
148 | },
149 | {
150 | "attachments": {},
151 | "cell_type": "markdown",
152 | "metadata": {},
153 | "source": [
154 | "### Sensitivity\n",
155 | "\n",
156 | "This is also often called as Recall. In this metric, we are interested to know the fraction of the number of positive cases predicted accurately out of all the *actual* positive cases. The difference with precision is that precision is calcalculated as a fraction out of all the total predicted positive cases. In the above example, we have 11 true positives and the total number of all actual positive cases are $11 + 3 = 14$.\n",
157 | "\n",
158 | "| actual\\predicted | a cat | not a cat |\n",
159 | "|------------------|-------|-----------|\n",
160 | "| a cat | 11 | 3 |\n",
161 | "| not a cat | 2 | 9 |\n",
162 | "\n",
163 | "$$\\text{sensitivity} = \\frac{11}{11 + 3} = \\frac{11}{14} = 0.786$$\n",
164 | "\n",
165 | "This means that the sensitivity is about 79\\%. Notice that the sensitivy value is different from precision which is about 85\\%. We can write the formula for sensitivity as follows.\n",
166 | "\n",
167 | "$$\\text{sensitivity} = \\frac{\\text{TP}}{\\text{Total Actual Positives}} = \\frac{\\text{TP}}{\\text{TP} + \\text{FN}}$$\n",
168 | "\n",
169 | "We can see sensitivty as a fraction between the green circle and the blue circle in the image below.\n",
170 | "\n",
171 | "![\"drawing\"](\"https://data-driven-world.github.io/assets/images/week10/confusion_matrix_sensitivity.png\")
\n"
172 | ]
173 | },
174 | {
175 | "attachments": {},
176 | "cell_type": "markdown",
177 | "metadata": {},
178 | "source": [
179 | "### Specificity\n",
180 | "\n",
181 | "Another common matrix is called specificity. This is the metrix we normally use when we are interested in the negative cases. It is also called as *True Negative Rate*. Specificity is calculated as the number of True Negative cases divided over all actual Negative cases.\n",
182 | "\n",
183 | "| actual\\predicted | a cat | not a cat |\n",
184 | "|------------------|-------|-----------|\n",
185 | "| a cat | 11 | 3 |\n",
186 | "| not a cat | 2 | 9 |\n",
187 | "\n",
188 | "$$\\text{specificity} = \\frac{9}{9 + 2} = \\frac{9}{11} = 0.818$$\n",
189 | "\n",
190 | "This means that the true negative rate is about 82\\%. You can see specificity as the sibling of sensitivity. In sensitivity, you are interested in the positive cases while in the specificity you are interested in the negative cases. The formula for specificity is given as follows.\n",
191 | "\n",
192 | "\n",
193 | "$$\\text{specificity} = \\frac{\\text{TN}}{\\text{Total Actual Negatives}} = \\frac{\\text{TN}}{\\text{FP} + \\text{TN}}$$\n",
194 | "\n",
195 | "We can see specificity as a fraction between the green circle and the blue circle in the image below.\n",
196 | "\n",
197 | "![\"drawing\"](\"https://data-driven-world.github.io/assets/images/week10/confusion_matrix_specificity.png\")
"
198 | ]
199 | },
200 | {
201 | "attachments": {},
202 | "cell_type": "markdown",
203 | "metadata": {},
204 | "source": [
205 | "## Confusion Matrix for Multiple Classes\n",
206 | "\n",
207 | "What if we have more than two categories. What would the confusion matrix look like? Let's say we are classifying images of cat, dog, and a fish. In this case, we have three categories. We can write our confusion matrix in this manner.\n",
208 | "\n",
209 | "| actual\\predicted | cat | dog | fish |\n",
210 | "|------------------|-----|-----|------|\n",
211 | "| cat | 11 | 1 | 2 |\n",
212 | "| dog | 2 | 9 | 3 |\n",
213 | "| fish | 1 | 1 | 8 |\n",
214 | "\n",
215 | "The diagonal element again gives us the accuracy of the model.\n",
216 | "\n",
217 | "$$\\text{accuracy} = \\frac{11 + 9 + 8}{38} = \\frac{28}{38} = 0.737$$\n",
218 | "\n",
219 | "which is about 73\\%. We can write the formula for accuracy as follows.\n",
220 | "\n",
221 | "$$\\text{accuracy} = \\frac{\\sum_i M_{ii}}{\\sum_i\\sum_j{M_{ij}}} $$\n",
222 | "\n",
223 | "The formula simply sums up all the diagonal elements and divide it with the sum of all.\n",
224 | "\n",
225 | "We can calculate the other metrices by defining our positive case in a *one-versus-all* manner. This means that if we define cat as our positive case, we define both dog and fish as the negative cases. \n",
226 | "\n",
227 | "For example, to calculate the sensitivity for class *i*, we use:\n",
228 | "\n",
229 | "$$\\text{sensitivity}_i = \\frac{M_{ii}}{\\sum_j{M_{ij}}}$$\n",
230 | "\n",
231 | "where $i$ is the class we are interested. The summation over $j$ means we sum **over all the columns** in the confusion matrix.\n",
232 | "\n",
233 | "Similarly, we can calculate the precision for class *i* using:\n",
234 | "\n",
235 | "$$\\text{precision}_i = \\frac{M_{ii}}{\\sum_j{M_{ji}}}$$\n",
236 | "\n",
237 | "**Notice that the indices are swapped for the denominator**. For precision, instead of summing over all the columns, we **sum over all the rows** in column *i* which is the total cases when class *i* is *predicted*.\n",
238 | "\n",
239 | "\n"
240 | ]
241 | }
242 | ],
243 | "metadata": {
244 | "kernelspec": {
245 | "display_name": "Python 3 (ipykernel)",
246 | "language": "python",
247 | "name": "python3"
248 | },
249 | "language_info": {
250 | "codemirror_mode": {
251 | "name": "ipython",
252 | "version": 3
253 | },
254 | "file_extension": ".py",
255 | "mimetype": "text/x-python",
256 | "name": "python",
257 | "nbconvert_exporter": "python",
258 | "pygments_lexer": "ipython3",
259 | "version": "3.8.5"
260 | }
261 | },
262 | "nbformat": 4,
263 | "nbformat_minor": 4
264 | }
265 |
--------------------------------------------------------------------------------
/Logistic_Regression.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "attachments": {},
5 | "cell_type": "markdown",
6 | "metadata": {},
7 | "source": [
8 | "![](\"https://data-driven-world.github.io/assets/images/week9/gradient_descent.jpeg\")
\n",
240 | "\n",
241 | "The contour plot shows the minimum somewhere in the centre. The idea of gradient descent is that we move the fastest to the minimum if we choose to move in the direction with the steepest slope. The steepest slope can be found from the gradient of the function. Let's look at point $x_0$ in the figure. The gradient in the direction of $\\beta_0$ is non zero as can be seen from the contour since it is perpendicular to the contour lines. On the other hand, the gradient in the direction of $\\beta_1$ is zero as it is parallel with the contour line at $x_0$. Recall that contour lines show the points with the same value. When the points have the same values, the gradient is zero. We can then substitute this into the above equation.\n",
242 | "\n",
243 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\frac{\\partial}{\\partial \\hat{\\beta}_0} J$$\n",
244 | "\n",
245 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 - \\alpha \\frac{\\partial}{\\partial \\hat{\\beta}_1} J$$\n",
246 | "\n",
247 | "The partial derivative with respect to $\\beta_0$ is non-zero while the derivative with respect to $\\beta_1$ is zero. So we have the following:\n",
248 | "\n",
249 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\times m $$\n",
250 | "\n",
251 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 $$\n",
252 | "\n",
253 | "where $m$ is the gradient in $\\beta_0$ direction which is the partial derivative $\\partial J/\\partial\\hat{\\beta}_0$. If the optimum is a minima, then $m < 0$, a negative value. Now, we can see how the next point increases in $\\beta_0$ direction but not in $\\beta_1$ direction at $x_0$.\n",
254 | "\n",
255 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 + \\delta $$\n",
256 | "\n",
257 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 $$\n",
258 | "\n",
259 | "When the gradient in the $\\beta_1$ direction is no longer zero as in the subsequent steps, both $\\beta_0$ and $\\beta_1$ increases by some amount.\n",
260 | "\n",
261 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 + \\delta_0 $$\n",
262 | "\n",
263 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 + \\delta_1 $$\n",
264 | "\n",
265 | "We can actually calculate the derivative of the cost function analytically.\n",
266 | "\n",
267 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)^2$$\n",
268 | "\n",
269 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_j} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{\\partial}{\\partial \\hat{\\beta}_j} \\frac{1}{2m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)^2$$\n",
270 | "\n",
271 | "We can substitute our straight line equation into $\\hat{y}$ to give the following.\n",
272 | "\n",
273 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_j} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{\\partial}{\\partial \\hat{\\beta}_j} \\frac{1}{2m}\\Sigma_{i=1}^m\\left(\\hat{\\beta}_0 + \\hat{\\beta}_1 x^i - y^i\\right)^2$$\n",
274 | "\n",
275 | "Now we will differentiate the above equation with respect to $\\hat{\\beta}_0$ and $\\hat{\\beta}_1$. \n",
276 | "\n",
277 | "Let's first do it for $\\hat{\\beta}_0$. \n",
278 | "\n",
279 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_0} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{\\beta}_0 + \\hat{\\beta}_1 x^i - y^i\\right)$$\n",
280 | "\n",
281 | "or\n",
282 | "\n",
283 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_0} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)$$\n",
284 | "\n",
285 | "Now, we need to do the same by differentiating it with respect to $\\hat{\\beta}_1$.\n",
286 | "\n",
287 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_1} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{\\beta}_0 + \\hat{\\beta}_1 x^i - y^i\\right) x^i$$\n",
288 | "\n",
289 | "or\n",
290 | "\n",
291 | "$$\\frac{\\partial}{\\partial \\hat{\\beta}_1} J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right) x^i$$\n",
292 | "\n",
293 | "Now we have the equation to calculate the next values of $\\hat{\\beta}_0$ and $\\hat{\\beta}_1$ using gradient descent.\n",
294 | "\n",
295 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)$$\n",
296 | "\n",
297 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)x^i$$"
298 | ]
299 | },
300 | {
301 | "attachments": {},
302 | "cell_type": "markdown",
303 | "metadata": {},
304 | "source": [
305 | "## Matrix Operations\n",
306 | "\n",
307 | "We can calculate these operations using matrix calculations. \n",
308 | "\n",
309 | "### Hypothesis\n",
310 | "\n",
311 | "Recall that our hypothesis (predicted value) for one data point was written as follows.\n",
312 | "\n",
313 | "$$\\hat{y}(x^i) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x^i$$\n",
314 | "\n",
315 | "If we have $m$ data points, we will then have a set of equations.\n",
316 | "\n",
317 | "$$\\hat{y}(x^1) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x^1$$\n",
318 | "$$\\hat{y}(x^2) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x^2$$\n",
319 | "$$\\ldots$$\n",
320 | "$$\\hat{y}(x^m) = \\hat{\\beta}_0 + \\hat{\\beta}_1 x^m$$\n",
321 | "\n",
322 | "We can rewrite this in terms of matrix multiplication. First, we write our independent variable $x$ as a column vector.\n",
323 | "\n",
324 | "$$\\begin{bmatrix}\n",
325 | "x^1\\\\\n",
326 | "x^2\\\\\n",
327 | "\\ldots\\\\\n",
328 | "x^m\n",
329 | "\\end{bmatrix}$$\n",
330 | "\n",
331 | "To write the system equations, we need to add a column of constant 1s into our independent column vector.\n",
332 | "\n",
333 | "$$\\mathbf{X} = \\begin{bmatrix}\n",
334 | "1 & x^1\\\\\n",
335 | "1 & x^2\\\\\n",
336 | "\\ldots & \\ldots\\\\\n",
337 | "1 &x^m\n",
338 | "\\end{bmatrix}$$\n",
339 | "\n",
340 | "and our constants as a column vector too.\n",
341 | "\n",
342 | "$$\\mathbf{\\hat{b}} = \\begin{bmatrix}\n",
343 | "\\hat{\\beta}_0\\\\\n",
344 | "\\hat{\\beta}_1\n",
345 | "\\end{bmatrix}$$\n",
346 | "\n",
347 | "Our system equations can then be written as\n",
348 | "\n",
349 | "$$\\mathbf{\\hat{y}} = \\mathbf{X} \\times \\mathbf{\\hat{b}}$$\n",
350 | "\n",
351 | "The result of this matrix multiplication is a column vector of $m\\times 1$. Note that in the above matrix equation, we use $\\mathbf{\\hat{y}}$ to denote the column vector of *predicted* value or our hypothesis. "
352 | ]
353 | },
354 | {
355 | "attachments": {},
356 | "cell_type": "markdown",
357 | "metadata": {},
358 | "source": [
359 | "### Cost Function\n",
360 | "\n",
361 | "Recall that the cost function is written as follows.\n",
362 | "\n",
363 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}\\Sigma^m_{i=1}\\left(\\hat{y}(x^i)-y^i\\right)^2$$\n",
364 | "\n",
365 | "We can rewrite the square as a multiplication instead and make use of matrix multplication to express it.\n",
366 | "\n",
367 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}\\Sigma^m_{i=1}\\left(\\hat{y}(x^i)-y^i\\right)\\times \\left(\\hat{y}(x^i)-y^i\\right)$$\n",
368 | "\n",
369 | "Writing it as matrix multiplication gives us the following.\n",
370 | "\n",
371 | "$$J(\\hat{\\beta}_0, \\hat{\\beta}_1) = \\frac{1}{2m}(\\mathbf{\\hat{y}}-\\mathbf{y})^T\\times (\\mathbf{\\hat{y}}-\\mathbf{y})$$"
372 | ]
373 | },
374 | {
375 | "attachments": {},
376 | "cell_type": "markdown",
377 | "metadata": {},
378 | "source": [
379 | "### Gradient Descent\n",
380 | "\n",
381 | "Recall that our gradient descent equations update functions were written as follows. \n",
382 | "\n",
383 | "$$\\hat{\\beta}_0 = \\hat{\\beta}_0 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)$$\n",
384 | "\n",
385 | "$$\\hat{\\beta}_1 = \\hat{\\beta}_1 - \\alpha \\frac{1}{m}\\Sigma_{i=1}^m\\left(\\hat{y}(x^i) - y^i\\right)x^i$$\n",
386 | "\n",
387 | "And recall that our independent variable is a column vector with constant 1 appended into the first column.\n",
388 | "\n",
389 | "$$\\mathbf{X} = \\begin{bmatrix}\n",
390 | "1 & x^1\\\\\n",
391 | "1 & x^2\\\\\n",
392 | "\\ldots & \\ldots\\\\\n",
393 | "1 &x^m\n",
394 | "\\end{bmatrix}$$\n",
395 | "\n",
396 | "Transposing this column vector results in\n",
397 | "\n",
398 | "$$\\mathbf{X}^T = \\begin{bmatrix}\n",
399 | "1 & 1 & \\ldots & 1\\\\\n",
400 | "x^1 & x^2 & \\ldots & x^m\\\\\n",
401 | "\\end{bmatrix}$$\n",
402 | "\n",
403 | "Note that we can write the update function summation also as a matrix operations.\n",
404 | "\n",
405 | "$$\\mathbf{\\hat{b}} = \\mathbf{\\hat{b}} - \\alpha \\frac{1}{m}\\mathbf{X}^T \\times (\\mathbf{\\hat{y}} - \\mathbf{y})$$\n",
406 | "\n",
407 | "Substituting the equation for $\\mathbf{\\hat{y}}$, we get the following equation.\n",
408 | "\n",
409 | "$$\\mathbf{\\hat{b}} = \\mathbf{\\hat{b}} - \\alpha \\frac{1}{m}\\mathbf{X}^T \\times (\\mathbf{X} \\times \\mathbf{\\hat{b}} - \\mathbf{y})$$\n",
410 | "\n",
411 | "In this notation, the capital letter notation indicates matrices and small letter notation indicates vector. Those without bold notation are constants."
412 | ]
413 | },
414 | {
415 | "attachments": {},
416 | "cell_type": "markdown",
417 | "metadata": {},
418 | "source": [
419 | "## Metrics\n",
420 | "\n",
421 | "After we build our model, we usually want to evaluate how good our model is. We use metrics to evaluate our model or hypothesis. To do this, we should split the data into two:\n",
422 | "- training data set\n",
423 | "- test data set\n",
424 | "\n",
425 | "The training data set is used to build the model or the hypothesis. The test data set, on the other hand, is used to evaluate the model by computing some metrics.\n"
426 | ]
427 | },
428 | {
429 | "attachments": {},
430 | "cell_type": "markdown",
431 | "metadata": {},
432 | "source": [
433 | "### Mean Squared Error\n",
434 | "\n",
435 | "One metric we can use here is called the mean squared error. The mean squred error is computed as follows.\n",
436 | "\n",
437 | "$$MSE = \\frac{1}{n}\\Sigma_{i=1}^n(y^i - \\hat{y}^i)^2$$\n",
438 | "\n",
439 | "where $n$ is the number of predicted data points in the *test* data set, $y^i$ is the actual value in the *test* data set, and $\\hat{y}^i$ is the predicted value obtained using the hypothesis and the independent variable $x^i$ in the *test* data set."
440 | ]
441 | },
442 | {
443 | "attachments": {},
444 | "cell_type": "markdown",
445 | "metadata": {},
446 | "source": [
447 | "### R2 Coefficient of Determination\n",
448 | "\n",
449 | "Another metric is called the $r^2$ coefficient or the coefficient of determination. This is computed as follows.\n",
450 | "\n",
451 | "$$r^2 = 1 - \\frac{SS_{res}}{SS_{tot}}$$\n",
452 | "\n",
453 | "where\n",
454 | "\n",
455 | "$$SS_{res} = \\Sigma_{i=1}^n (y_i - \\hat{y}_i)^2$$ \n",
456 | "\n",
457 | "where $y_i$ is the actual target value and $\\hat{y}_i$ is the predicted target value.\n",
458 | "\n",
459 | "$$SS_{tot} = \\Sigma_{i=1}^n (y_i - \\overline{y})^2$$\n",
460 | "\n",
461 | "where \n",
462 | "$$ \\overline{y} = \\frac{1}{n} \\Sigma_{i=1}^n y_i$$\n",
463 | "and $n$ is the number of target values.\n",
464 | "\n",
465 | "This coefficient gives you how close the data is to a straight line. The closer it is to a straight line, the value will be close to 1.0. This means there is a correlation between the independent variable and the dependent variable. When there is no correlation, the value will be close to 0.\n",
466 | "\n",
467 | "In your problem sets, you will work on writing a code to do all these tasks."
468 | ]
469 | }
470 | ],
471 | "metadata": {
472 | "kernelspec": {
473 | "display_name": "Python 3.8.5 ('base')",
474 | "language": "python",
475 | "name": "python3"
476 | },
477 | "language_info": {
478 | "codemirror_mode": {
479 | "name": "ipython",
480 | "version": 3
481 | },
482 | "file_extension": ".py",
483 | "mimetype": "text/x-python",
484 | "name": "python",
485 | "nbconvert_exporter": "python",
486 | "pygments_lexer": "ipython3",
487 | "version": "3.8.5"
488 | },
489 | "vscode": {
490 | "interpreter": {
491 | "hash": "ac37b77c3c0f43e60bec193f0626743b91dd65d8d4aeca5713e457ab7e777e4c"
492 | }
493 | }
494 | },
495 | "nbformat": 4,
496 | "nbformat_minor": 4
497 | }
498 |
--------------------------------------------------------------------------------