├── .gitignore ├── README.md ├── image ├── 1 ├── CodeCogsEqn (1).gif ├── adagrad1.png ├── adaptivelr.png ├── derivative.png ├── derivative1.png ├── derivative2.png ├── imbd.png ├── lr_large.png ├── lr_small.png ├── mnist.png ├── pseudocode1.png └── pseudocode2.png └── mashroom ├── cs677-project.pptx ├── cs677project.ipynb └── read.me /.gitignore: -------------------------------------------------------------------------------- 1 | Mercury/ 2 | Mercury.modules 3 | *.mh 4 | *.err 5 | *.init 6 | *.dll 7 | *.exe 8 | *.a 9 | *.so 10 | *.dylib 11 | *.beams 12 | *.d 13 | *.c_date 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Machine Learning 2 | 3 | ***Explained machine learning models and projects based on ML*** 4 | 5 | Basic machine learning algorithms: 6 | 7 | 1.**Linear regression** 8 | 9 | 2.**Logistic regression** 10 | 11 | 3.**Support Vector Machines** 12 | 13 | 4.**Naive Bayes** 14 | 15 | *(below coming soon...)* 16 | 17 | 5.Decision trees 18 | 19 | 6.Random Forest 20 | 21 | 7.k- Means clustering 22 | 23 | 8.k-Nearest neighbors 24 | 25 | 26 | 27 | ## Linear regression 28 | 29 | Linear regression is very simple and basic.First, linear regression is supervised model, which means data should be labelled.Linear regression will find the relationships between features(x1,x2,x3....), which represent as coefficients of these variables. 30 | 31 | 32 | ### simple linear regression 33 | 34 | 35 | Let's look at a simple linear regression equation: 36 | 37 | ![equation](https://latex.codecogs.com/gif.latex?y%20%3D%20%5CTheta%20_1x+%5CTheta%20_0) 38 | 39 | ▷ θ1 is the coefficient of the independent variable (slope) 40 | 41 | ▷ θ0 is the constant term or the y intercept. 42 | 43 | 44 | Then consider this dataset as tuples of (1, 18), (2, 22), (3, 45), (4, 49), (5, 86) 45 | 46 | • We might want to fit a straight line to the given data 47 | 48 | • Assume to fit a line with the equation Y = θ1X + θ0 49 | 50 | • Our goal is to minimize errors 51 | 52 | To minimize the amount of distance(errors), we need to find proper θ1 and θ0.We build a function ,which often referred to as a ***lost function***,. 53 | 54 | lost function has three common formula: 55 | (1)MSE(Mean Squared Error) 56 | (2)RMSE(Root Mean Squared Error) 57 | (3)Logloss(Cross Entorpy loss) 58 | 59 | 60 | In this case, we choose **Mean Squared Error**. 61 | 62 | ![equation](https://latex.codecogs.com/gif.latex?l%28%5CTheta%20_0%2C%5CTheta%20_1%29%20%3D%20%5Cfrac%7B1%7D%7B2n%7D%5Csum_%7Bi%7D%20%28f%28x_%7Bi%7D%29-y_%7Bi%7D%29%5E%7B2%7D) 63 | 64 | So when we want to fit a line to given data, we need to minimize the lost function. 65 | 66 | then, computing lost function: 67 | 68 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20l%28%5CTheta%20_0%2C%5CTheta%20_1%29%20%26%20%3D%20%5Cfrac%7B1%7D%7B2n%7D%20%5Csum_%7Bi%7D%20%28f%28x_%7Bi%7D%29-y_%7Bi%7D%29%5E%7B2%7D%20%5Cnonumber%20%5C%5C%20%26%3D%20%5Cfrac%7B1%7D%7B2n%7D%20%5Csum_%7Bi%7D%28%5CTheta%20_1%20x_i%20+%20%5CTheta%20_1%20-%20y_%7Bi%7D%29%5E%7B2%7D%5Cnonumber%20%5Cend%7Balign%7D) 69 | 70 | 71 | 72 | 73 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20l%7B%7D%27%28%5CTheta%20_0%2C%5CTheta%20_1%29%20%26%20%3D%20%5Cfrac%7B1%7D%7Bn%7D%20%5Csum_%7Bi%7D%20%28%5CTheta%20_1%20x_%7Bi%7D%20%5E%7B2%7D%20-x_%7Bi%7Dy_%7Bi%7D%29%20%5Cnonumber%20%5C%5C%20%26%3D%20%5Cfrac%7B1%7D%7Bn%7D%20%5Csum_%7Bi%7D%282%5CTheta%20_1%281%20+%204%20+%209%20+%2016%20+%2025%29%20-%202%2818%20+%2044%20+%20135%20+%20196%20+%20430%29%29%5Cnonumber%20%5C%5C%20%26%3D%20%5Cfrac%7B1%7D%7B5%7D%28110m%20-%201646%29%5Cnonumber%20%5Cend%7Balign%7D) 74 | 75 | Since cost function is a ”Convex” function, when its derivative is 0, the cost function hits bottom. 76 | So loss minimized at θ = 14.96. 77 | 78 | Now we have a polynomial linear regression, suppose each entity x has d dimensions: 79 | 80 | ![equation](https://latex.codecogs.com/gif.latex?y%20%3D%20%5CTheta_%7B0%7D%20+%20%5CTheta_%7B1%7Dx_1%20+%20...%20+%20%5CTheta_%7Bd%7Dx_d) 81 | 82 | Similarly, we get the lost function : 83 | 84 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20l%28%5CTheta%20_0%2C%5CTheta%20_1...%5CTheta%20_d%29%20%26%20%3D%20%5Cfrac%7B1%7D%7B2n%7D%20%5Csum_%7Bi%7D%20%28f%28x_%7Bi%7D%29%20-%20y_%7Bi%7D%29%5E2%20%5Cnonumber%20%5Cend%7Balign%7D) 85 | 86 | So in order to minimize the cost function, we need to choose each θi to minimize l(θ0,θ1...),this is what we called ***Gradient Descent***. 87 | 88 | Gradient Descent is an iterative algorithm,Start from an initial guess and try to incrementally improve current solution,and at iteration step θ(iter) is the current guess for θi. 89 | 90 | 91 | #### How to calculate gradient 92 | 93 | Suppose ▽l(θ) is a vector whose ith entry is ith partial derivative evaluated at θi 94 | 95 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20%5CDelta%20l%28%5Ctheta%29%20%3D%20%5Cbegin%7Bbmatrix%7D%5Cnonumber%20%5Cfrac%7B%5Cpartial%20l%28%5Ctheta%29%7D%7B%5Cpartial%20%5Ctheta%20_0%7D%5C%5C%20%5Cfrac%7B%5Cpartial%20l%28%5Ctheta%29%7D%7B%5Cpartial%20%5Ctheta%20_1%7D%5C%5C%20.%5C%5C%20.%5C%5C%20%5Cfrac%7B%5Cpartial%20l%28%5Ctheta%29%7D%7B%5Cpartial%20%5Ctheta%20_d%7D%5C%5C%20%5Cend%7Bbmatrix%7D%20%5Cend%7Balign%7D) 96 | 97 | 98 | In privious sessions, we got the loss function, which is 99 | 100 | 101 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20l%28%5CTheta%20_0%2C%5CTheta%20_1...%5CTheta%20_d%29%20%26%20%3D%20%5Cfrac%7B1%7D%7B2n%7D%20%5Csum_%7Bi%7D%20%28f%28x_%7Bi%7D%29%20-%20y_%7Bi%7D%29%5E2%20%5Cnonumber%20%5Cend%7Balign%7D) 102 | 103 | then do expansion: 104 | 105 | 106 | ![equation](https://latex.codecogs.com/gif.latex?l%28%5Ctheta%20_0%2C%5Ctheta%20_1%2C...%2C%5Ctheta%20_d%29%3D%20%5Cfrac%7B1%7D%7B2n%7D%5Csum_%7Bi%7D%5E%7B%7D%28y%5E%7B%28i%29%7D-%20%28%5Ctheta%20_0+%20%5Ctheta%20_1x_1%5E%7B%28i%29%7D+...%5Ctheta%20_dx_d%5E%7B%28i%29%7D%29%29) 107 | 108 | Since it has mutiple dimensions,we compute partial derivatives: 109 | 110 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20%5Cfrac%7B%5Cpartial%20l%28%5Ctheta%29%7D%7B%5Cpartial%20%5Ctheta%20_1%7D%20%3D%20-%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7B1%7D%5E%7Bn%7D%20x_%7B1%7D%5E%7B%28i%29%7D%28y%5E%7B%28i%29%7D%20-%20%28%5Ctheta%20_0+%5Ctheta%20_1x_1%5E%7B%28i%29%7D%20+%20...+%20%5Ctheta%20_dx_d%5E%7B%28i%29%7D%20%29%5Cnonumber%5C%5C%20%5Cfrac%7B%5Cpartial%20l%28%5Ctheta%29%7D%7B%5Cpartial%20%5Ctheta%20_2%7D%20%3D%20-%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7B1%7D%5E%7Bn%7D%20x_%7B2%7D%5E%7B%28i%29%7D%28y%5E%7B%28i%29%7D%20-%20%28%5Ctheta%20_0+%5Ctheta%20_1x_1%5E%7B%28i%29%7D%20+%20...+%20%5Ctheta%20_dx_d%5E%7B%28i%29%7D%20%29%5Cnonumber%5C%5C%20...%5Cnonumber%5C%5C%20%5Cfrac%7B%5Cpartial%20l%28%5Ctheta%29%7D%7B%5Cpartial%20%5Ctheta%20_d%7D%20%3D%20-%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7B1%7D%5E%7Bn%7D%20x_%7Bd%7D%5E%7B%28i%29%7D%28y%5E%7B%28i%29%7D%20-%20%28%5Ctheta%20_0+%5Ctheta%20_1x_1%5E%7B%28i%29%7D%20+%20...+%20%5Ctheta%20_dx_d%5E%7B%28i%29%7D%20%29%5Cnonumber%20%5Cend%7Balign%7D) 111 | 112 | Now we can compute components of the gradients and then sum them up and update weights in the next iteration. 113 | 114 | #### Gradient Descent pseudocode** 115 | 116 | ![pseudocode](https://github.com/gnayoaixgnaw/machine_learning_project/blob/main/image/pseudocode1.png) 117 | 118 | • Here λ is the ”learning rate” and controls speed of convergence 119 | • ▽l(θ iter) is the gradient of L evaluated at iteration ”iter” with parameter of qiter 120 | • Stop conditions can be different 121 | 122 | **When to stop** 123 | 124 | Stop condition can be different, for example: 125 | • Maximum number of iteration is reached (iter < MaxIteration) 126 | • Gradient ▽l(θ iter ) or parameters are not changing (||θ(iter+1) - θ(iter)|| < precisionValue) 127 | • Cost is not decreasing (||l(θ(iter+1)) - L(θ(iter))|| < precisionValue) 128 | • Combination of the above 129 | 130 | more detailed pseudocode to compute gradient: 131 | 132 | // initialize parameters 133 | iteration = 0 134 | learning Rate = 0.01 135 | numIteration = X 136 | theta = np.random.normal(0, 0.1, d) 137 | 138 | while iteration < maxNumIteration: 139 | 140 | calculate gradients 141 | //update parameters 142 | theta -= learning Rate*gradients 143 | iteration+=1 144 | 145 | 146 | 147 | #### Implement code via Pyspark 148 | 149 | ***Check [here](https://github.com/gnayoaixgnaw/Big_Data_Analytics/tree/main/assignment3)*** 150 | 151 | 152 | 153 | ## Regulation in lost function 154 | 155 | We will always face **over-fitting issue** in real problem. **over-fitting issue** is that the parameters of model are large and model's rebustness is poor, which means a little change of test data may cause a huge difference in result.So in order to aviod over-fitting, 156 | 157 | ### l1 norm 158 | 159 | We need to remove parameters which have little contribution and generate sparse matrix, that is, the l1 norm( mean absolute error): 160 | 161 | ![equation](https://latex.codecogs.com/gif.latex?l_1%20%3D%20l+%5Clambda%20%5Csum_%7Bi%3D1%7D%5E%7Bd%7D%5Cleft%20%7C%20%5Ctheta%20_i%20%5Cright%20%7C) 162 | 163 | where l is lost function, ∑ i|θi| is l1 regularizers, λ is regularization coefficient, θi is parameters. 164 | we can visualize l1 lost function： 165 | 166 | ![l1](https://i.loli.net/2018/11/28/5bfe89e366bba.jpg) 167 | 168 | The contour line in the figure is that of l, and the black square is the graph of L1 function. The place where the contour line of l intersects the graph of L1 for the first time is the optimal solution. It is easy to find that the black square must intersect the contour line at the vertex of the square first. l is much more likely to contact those angles than it is to contact any other part. Some dimensions of these points are 0 which will make some features equal to 0 and generate a sparse matrix, which can then be used for feature selection. 169 | 170 | ### l2 norm 171 | 172 | We can make parameters as little as possible by implement l2 norm: 173 | 174 | ![equation](https://latex.codecogs.com/gif.latex?l_1%20%3D%20l+%5Clambda%20%5Csum_%7Bi%3D1%7D%5E%7Bd%7D%5Cleft%20%7C%20%5Ctheta%20_i%20%5Cright%20%7C%5E%7B2%7D) 175 | 176 | 177 | where l is lost function, ∑ i|θi|² is l2 regularizers, λ is regularization coefficient, θi is parameters. 178 | we can visualize l2 lost function： 179 | 180 | ![l2](https://i.loli.net/2018/11/28/5bfe89e366bba.jpg) 181 | 182 | In comparison with the iterative formula without adding L2 regularization, parameters are multiplied by a factor less than 1 in each iteration, which makes parameters decrease continuously. Therefore, in general, parameters decreasing continuously. 183 | 184 | 185 | 186 | 187 | ## Logistic regression 188 | 189 | Logistic regression is supervised model especially for prediction problem.It has binary-class lr and multi-class lr. 190 | 191 | 192 | ### Binary-class logistic regression 193 | 194 | 195 | Suppose we have a prediction problem.It is natural to assume that output y (0/1) given the independent variable(s) X ,which has d dimensions and model parameter θ is sampled from the exponential family. 196 | 197 | It makes sense to assume that the x is sampled from a Bernoulli and here is the log-likelihood: 198 | 199 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20L%28p%7Cx_1%2Cx_2...%2Cx_n%29%20%26%3D%20%5Cprod_%7Bi%20%3D%201%7D%5E%7Bn%7Dp%5E%7Bx_i%7D%281-p%29%5E%7B%281-x_i%29%7D%5Cnonumber%20%5C%5C%20%26%3D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5Bx_i%5Clog%20%28p%29+%281-x_i%29%5Clog%20%281-p%29%5D%20%5Cnonumber%20%5Cend%7Balign%7D) 200 | 201 | Given a bunch of data for example,suppose output Y has (0/1): 202 | 203 | (92, 12), (23, 67), (67, 92), (98, 78), (18, 45), (6, 100) 204 | 205 | Final Result in class: 0, 0, 1, 1, 0, 0 206 | 207 | • If coefs are (-1, -1), LLH is -698 208 | • If coefs are (1, -1), LLH is -133 209 | • If coefs are (-1, 1), LLH is 7.4 210 | • If coefs are (1, 1), LLH is 394 211 | 212 | 213 | However this is not enough to get the loss function, logistic regreesion needs a ***sigmoid*** function to show the probability of y = 0/1,which is : 214 | 215 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20P%28x_i%29%20%26%3D%20%5Cfrac%7B1%7D%7B1-e%5E%7B-y_i%7D%7D%5Cnonumber%20%5C%5C%20%26%3D%5Cfrac%7Be%5E%7B%5Ctheta%20_0+%5Ctheta_1%20x_1+...+%5Ctheta_d%20x_d%7D%7D%7B1+e%5E%7B%5Ctheta%20_0+%5Ctheta_1%20x_1+...+%5Ctheta_d%20x_d%7D%7D%20%5Cnonumber%20%5Cend%7Balign%7D) 216 | 217 | The parameter ω is related to X that is, assuming X is vector-valued and ω can be represent as : 218 | 219 | ![equation](https://latex.codecogs.com/gif.latex?%5Comega%20_i%20%3D%20%5Csum_%7Bj%20%3D%201%7D%5E%7Bd%7Dx_j%5E%7B%28i%29%7D%20%5Ctheta%20_j) 220 | 221 | where θ is regression coefficent and j is entity's jth dimension . 222 | 223 | Now its time to implement Log-likelihood in logistic regression, written as: 224 | 225 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20L%28p%7Cx_1%2Cx_2...%2Cx_n%2C%20y_1%2Cy_2...%2Cy_n%29%20%26%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5By_i%5Clog%20%5Cfrac%7Be%5E%7B%5Comega%20_i%7D%7D%7B1+e%5E%7B%5Comega_i%7D%7D+%281-y_i%29%5Clog%20%281-%5Cfrac%7Be%5E%7B%5Comega_i%7D%7D%7B1+e%5E%7B%5Comega_i%7D%7D%29%5D%5Cnonumber%20%5C%5C%20%26%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5By_i%28%5Clog%20e%5E%7B%5Comega_i%7D%29-%20%5Clog%20%281+e%5E%7B%5Comega_i%7D%29%5D%5Cnonumber%20%5C%5C%20%26%3D%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5By_i%5Comega_i-%5Clog%20%281+e%5E%7B%5Comega_i%7D%29%5D%5Cnonumber%20%5Cend%7Balign%7D) 226 | 227 | 228 | Now calculate loss function.As gradient descent need to minimize loss function,the loss function should be negative LLH: 229 | 230 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20loss%20function%20%26%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5By_i%5Comega_i-%5Clog%20%281+e%5E%7B%5Comega_i%7D%29%5D%5Cnonumber%20%5C%5C%20%26%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5B-%20y_i%5Comega_i%20+%20%5Clog%20%281+e%5E%7B%5Comega_i%7D%29%5D%5Cnonumber%20%5Cend%7Balign%7D) 231 | 232 | Appling regularization (l2 norm): 233 | 234 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20loss%20function%20%26%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5B-%20y_i%5Comega_i%20+%20%5Clog%20%281+e%5E%7B%5Comega_i%7D%29%5D%5Cnonumber%20%5C%5C%20%26%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5B-%20y_i%5Comega_i%20+%20%5Clog%20%281+e%5E%7B%5Comega_i%7D%29%5D%20+%20%5Clambda%20%5Csum_%7Bi%3D1%7D%5E%7Bj%7D%5Ctheta%20_i%20%5E%7B2%7D%5Cnonumber%20%5Cend%7Balign%7D) 235 | 236 | where j is entity's jth dimension. 237 | 238 | 239 | #### How to calculate gradient 240 | 241 | Suppose θj is jth partial derivative : 242 | 243 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20%5CDelta%20l%28%5Ctheta%29%20%3D%20%5Cbegin%7Bbmatrix%7D%5Cnonumber%20%5Cfrac%7B%5Cpartial%20l%28%5Ctheta%29%7D%7B%5Cpartial%20%5Ctheta%20_0%7D%5C%5C%20%5Cfrac%7B%5Cpartial%20l%28%5Ctheta%29%7D%7B%5Cpartial%20%5Ctheta%20_1%7D%5C%5C%20.%5C%5C%20.%5C%5C%20%5Cfrac%7B%5Cpartial%20l%28%5Ctheta%29%7D%7B%5Cpartial%20%5Ctheta%20_d%7D%5C%5C%20%5Cend%7Bbmatrix%7D%20%5Cend%7Balign%7D) 244 | 245 | Since it has mutiple dimensions,we compute partial derivatives: 246 | 247 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_1%7D%20%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5By_ix_1%5E%7B%28i%29%7D+x_1%5E%7B%28i%29%7D%5Cfrac%7Be%5E%7B%5Comega_i%7D%7D%7B1+e%5E%7B%5Comega_i%7D%7D%5D+2%5Clambda%20%5Comega%20_1%20%5Cnonumber%20%5C%5C%20%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_2%7D%20%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5By_ix_2%5E%7B%28i%29%7D+x_2%5E%7B%28i%29%7D%5Cfrac%7Be%5E%7B%5Comega_i%7D%7D%7B1+e%5E%7B%5Comega_i%7D%7D%5D+2%5Clambda%20%5Comega%20_2%20%5Cnonumber%20%5C%5C%20...%20%5Cnonumber%20%5C%5C%20%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_d%7D%20%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5By_ix_d%5E%7B%28i%29%7D+x_d%5E%7B%28i%29%7D%5Cfrac%7Be%5E%7B%5Comega_i%7D%7D%7B1+e%5E%7B%5Comega_i%7D%7D%5D+2%5Clambda%20%5Comega%20_d%20%5Cnonumber%20%5Cend%7Balign%7D) 248 | 249 | 250 | #### Gradient Descent pseudocode in Pyspark** 251 | 252 | ![pseudocode1](https://github.com/gnayoaixgnaw/machine_learning_project/blob/main/image/pseudocode2.png) 253 | 254 | 255 | #### Implement code via Pyspark 256 | 257 | ***Check [here](https://github.com/gnayoaixgnaw/Big_Data_Analytics/tree/main/assignment4)*** 258 | 259 | 260 | ### multi-class logistic regression 261 | 262 | In binary-class lr model, we use sigmoid function to map samples to (0,1),but in more cases, we need multi-class classfication, so we use ***softmax*** function to map samples to multiple (0,1). 263 | 264 | Softmax can be written as a hypothesis function : 265 | 266 | ![equation](https://latex.codecogs.com/gif.latex?h_%5Ctheta%20%28x%5E%7B%28i%29%7D%29%20%3D%20%5Cbegin%7Bbmatrix%7D%20p%28y%5E%7B%28i%29%7D%20%3D%201%7Cx%5E%7B%28i%29%7D%3B%5Ctheta%20%29%5C%5C%20p%28y%5E%7B%28i%29%7D%20%3D%202%7Cx%5E%7B%28i%29%7D%3B%5Ctheta%20%29%5C%5C%20...%5C%5C%20p%28y%5E%7B%28i%29%7D%20%3D%20k%7Cx%5E%7B%28i%29%7D%3B%5Ctheta%20%29%20%5Cend%7Bbmatrix%7D%20%3D%20%5Cfrac%7B1%7D%7B%5Csum_%7Bj%3D1%7D%5E%7Bk%7De%5E%7B%5Ctheta_j%20%5E%7BT%7D%20x%5E%7B%28i%29%7D%7D%7D%5Cbegin%7Bbmatrix%7D%20e%5E%7B%5Ctheta_1%20%5E%7BT%7D%20x%5E%7B%28i%29%7D%7D%5C%5C%20e%5E%7B%5Ctheta_2%20%5E%7BT%7D%20x%5E%7B%28i%29%7D%7D%5C%5C%20...%5C%5C%20e%5E%7B%5Ctheta_k%20%5E%7BT%7D%20x%5E%7B%28i%29%7D%7D%20%5Cend%7Bbmatrix%7D) 267 | 268 | where k is the total number of classes, i is ith entity. 269 | 270 | then we can get the loss function,which is also be called log-likelihood cost: 271 | 272 | ![equation](https://latex.codecogs.com/gif.latex?J%28%5Ctheta%20%29%20%3D%20-%5Cfrac%7B1%7D%7Bn%7D%5B%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5Csum_%7Bj%3D1%7D%5E%7Bk%7D1%5Cleft%20%5C%7B%20y%5E%7B%28i%29%7D%20%3D%20j%20%5Cright%20%5C%7D%5Clog%20%5Cfrac%7Be%5E%7B%5Ctheta%20_j%5ET%7Dx%5E%7B%28i%29%7D%7D%7B%5Csum_%7Bl%3D1%7D%5E%7Bk%7De%5E%7B%5Ctheta%20_l%5ET%7Dx%5E%7B%28i%29%7D%7D%5D) 273 | 274 | where 1{expression} is a function that if expression in {} is true then 1{expression} = 1 ,else 0. 275 | 276 | then rearrange it, and add l2 norm : 277 | 278 | ![equation](https://latex.codecogs.com/gif.latex?J%28%5Ctheta%20%29%20%3D%20-%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5Csum_%7Bj%3D1%7D%5E%7Bk%7D1%5Cleft%20%5C%7B%20y%5E%7B%28i%29%7D%20%3D%20j%20%5Cright%20%5C%7D%5B%5Clog%20e%5E%7B%5Ctheta%20_j%5ET%7Dx%5E%7B%28i%29%7D-%20%5Clog%20%5Csum_%7Bl%3D1%7D%5E%7Bk%7De%5E%7B%5Ctheta%20_l%5ET%7Dx%5E%7B%28i%29%7D%5D%5Clambda%20%5Csum_%7Bi%3D1%7D%5E%7Bk%7D%5Ctheta%20_i%5E%7B2%7D) 279 | 280 | #### How to calculate gradient 281 | 282 | Suppose θj is jth partial derivative : 283 | 284 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20%5Cfrac%7B%5Cpartial%20J%28%5Ctheta%20%29%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%20%26%20%3D%20-%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%20%5Bx%5E%7B%28i%29%7D%20%281%5Cleft%20%5C%7B%20y%5E%7B%28i%29%7D%20%3D%20j%20%5Cright%20%5C%7D%20-%20%5Cfrac%7Be%5E%7B%5Ctheta%20_j%5ET%7Dx%5E%7B%28i%29%7D%7D%7B%5Csum_%7Bl%3D1%7D%5E%7Bk%7De%5E%7B%5Ctheta%20_l%5ET%7Dx%5E%7B%28i%29%7D%7D%29%5D%20+%202%5Clambda%20%5Ctheta%20_j%5Cnonumber%5C%5C%20%26%3D%20-%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5B%20x%5E%7B%28i%29%7D%20%281%5Cleft%20%5C%7B%20y%5E%7B%28i%29%7D%20%3D%20j%20%5Cright%20%5C%7D%20-%20p%28y%5E%7B%28i%29%7D%3Dj%7Cx%5E%7B%28i%29%7D%3B%5Ctheta%20%29%29%5D%20+%202%5Clambda%20%5Ctheta%20_j%20%5Cnonumber%20%5Cend%7Balign%7D) 285 | 286 | 287 | #### Gradient Descent pseudocode in Pyspark** 288 | 289 | ![pseudocode1](https://github.com/gnayoaixgnaw/machine_learning_project/blob/main/image/pseudocode2.png) 290 | 291 | 292 | #### Implement code via Pyspark 293 | 294 | ***Check [here]()*** 295 | 296 | 297 | 298 | ## Support victor machine 299 | 300 | Traditional svm is binary-class svm, and there is also multi-class svm. 301 | 302 | 303 | ### binary class svm 304 | 305 | Suppose there is a dataset that is linearly separable, it is possible to put a strip between two classes.The points that keep strip from expending are 'support vector'. 306 | 307 | So basiclly, all points x in any line or plane or hyperplane can be discribed as a vevtor with distance b: 308 | 309 | ![equation](https://latex.codecogs.com/gif.latex?%5Cvec%7Bw%7D%5Ccdot%20x_0%20+%20b%20%3D%200) 310 | 311 | Now here is a point x and we need to caluculate the distance (which can be discribed as y) between this point and plane: 312 | 313 | ![equation](https://latex.codecogs.com/gif.latex?y%20%3D%20%5Cvec%7Bw%7D%5Ccdot%28x-x_0%29%20%3D%20%5Cvec%7Bw%7D%5Ccdot%20x%20-%20%5Cvec%7Bw%7D%5Ccdot%20x_0%20%3D%20%5Cvec%7Bw%7D%5Ccdot%20x+b) 314 | 315 | Notice y should be -1 or 1 to determine which of the sides the point x is. 316 | 317 | This is because in basic svm, it choose two planes: 318 | 319 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20%5Cvec%7Bw%7D%5Ccdot%20x+b%20%3D%201%20%5Cnonumber%20%5C%5C%20%5Cvec%7Bw%7D%5Ccdot%20x+b%20%3D%20-1%5Cnonumber%20%5Cend%7Balign%7D) 320 | 321 | where y > 1 then x is '+'(or x is positive sample) 322 | where y < -1 then x is '-'(or x is negative sample) 323 | 324 | ![image](https://pic4.zhimg.com/v2-197913c461c1953c30b804b4a7eddfcc_1440w.jpg?source=172ae18b) 325 | 326 | The distance between two planes are ![equation](https://latex.codecogs.com/gif.latex?%5Cfrac%7B2%7D%7B%5Cleft%20%5C%7C%20%5Cvec%7Bw%7D%20%5Cright%20%5C%7C%7D),so we need to maximize this distance to get optimal solution. 327 | 328 | #### loss function 329 | 330 | First, define a normal loss function: 331 | 332 | ![equation](https://latex.codecogs.com/gif.latex?l%20%3D%20%5Csum_%7Bi%20%3D%201%7D%5E%7Bn%7Dl%28y_i%20%5E%7Bpred%7D%2C%20y_i%5E%7Btrue%7D%29) 333 | 334 | when this loss function is hinge loss, it is exactly svm's loss function: 335 | 336 | ![equation](https://latex.codecogs.com/gif.latex?l%20%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7Dmax%280%2C%20y_i%5E%7Bpred%7D*y_i%5E%7Btrue%7D%29%20%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7Dmax%280%2C%201%20-%5Cvec%7Bw%7D%5Ccdot%20x_i*y_i%29) 337 | 338 | then add l2 norm, the final loss function is : 339 | 340 | ![equation](https://latex.codecogs.com/gif.latex?l%20%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7Dmax%280%2C%201%20-%5Cvec%7Bw%7D%5Ccdot%20x_i*y_i%29%20+%20%5Clambda%20%5Cleft%20%5C%7C%20%5Cvec%7Bw%7D%20%5Cright%20%5C%7C%5E%7B2%7D) 341 | 342 | #### How to calculate gradient 343 | 344 | We can use Chain rule to compute the derivative: 345 | 346 | ![equation](https://latex.codecogs.com/gif.latex?%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Cvec%7Bw%7D%7D%20%3D%20%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%28%5Cvec%7Bw%7D%5Ccdot%20x%29%7D%5Cfrac%7B%5Cpartial%20%28%5Cvec%7Bw%7D%5Ccdot%20x%29%7D%7B%5Cpartial%20%28%5Cvec%7Bw%7D%29%7D%20+%202%5Clambda%20%5Cvec%7Bw%7D) 347 | 348 | then calculate derivatives for two parts: 349 | 350 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%28%5Cvec%7Bw%7D%5Ccdot%20x%29%7D%20%3D%20%5Cfrac%7B%5Cpartial%20%5Csum_%7Bi%3D1%7D%5E%7Bn%7Dmax%280%2C%201-%20%5Cvec%7Bw%7D%5Ccdot%20x_i*y_i%20%5E%7Btrue%7D%29%7D%7B%5Cpartial%20%28%5Cvec%7Bw%7D%5Ccdot%20x_i%29%7D%20%5C%5C%20%5Cfrac%7B%5Cpartial%20%28%5Cvec%7Bw%7D%5Ccdot%20x%29%7D%7B%5Cpartial%20%5Cvec%7Bw%7D%7D%20%3D%20%5Csum_%7Bi%3D1%7D%5E%7Bn%7Dx_i%20%5Cend%7Balign%7D) 351 | 352 | In conclusion, the final gradient is : 353 | 354 | (1)if ![equation](https://latex.codecogs.com/gif.latex?%281-%20%5Cvec%7Bw%7D%5Ccdot%20x_i*y_i%20%5E%7Btrue%7D%29%20%3C%200): 355 | 356 | below fomula(1) = 0, which means the derivative = ![equation](https://latex.codecogs.com/gif.latex?0+%202%5Clambda%20%5Cvec%7Bw%7D) 357 | 358 | (2)if ![equation](https://latex.codecogs.com/gif.latex?%281-%20%5Cvec%7Bw%7D%5Ccdot%20x_i*y_i%20%5E%7Btrue%7D%29%20%3E%20%3D%200): 359 | 360 | below fomula(1) = ![equation](https://latex.codecogs.com/gif.latex?-%20y_i%20%5E%7Btrue%7D), which means the derivative = ![equation](https://latex.codecogs.com/gif.latex?%5Csum_%7Bi%20%3D%200%7D%5E%7Bn%7D-%20y_i%20%5E%7Btrue%7Dx_i+2%5Clambda%20%5Cvec%7Bw%7D) 361 | 362 | then the final derivatives for ***batch of data*** can be written as: 363 | 364 | ![equation](https://latex.codecogs.com/gif.latex?%5Cfrac%7B1%7D%7Bn%7D%20%5Csum_%7Bi%20%3D%200%7D%5E%7Bn%7D%20%5Cleft%5C%7B%5Cbegin%7Bmatrix%7D%200%26%20%2Cif%20%281-%20y_i%20%5E%7Btrue%7D*%5Cvec%7Bw%7D%5Ccdot%20x_i%29%20%3C0%5C%5C%20-%20y_i%20%5E%7Btrue%7Dx_i%26%20%2Cif%20%281-%20y_i%20%5E%7Btrue%7D*%5Cvec%7Bw%7D%5Ccdot%20x_i%29%20%3E%3D0%20%5Cend%7Bmatrix%7D%5Cright.+%202%5Clambda%20%5Cvec%7Bw%7D) 365 | 366 | 367 | #### Implement code via Pyspark 368 | 369 | ***Check [here]()*** 370 | 371 | 372 | 373 | ### multiple class svm 374 | 375 | Different from binary-class svm, multi-class svm's loss function requires the score on the correct class always be ***Δ(a boundary value)*** higher than scores on incorrect classes. 376 | 377 | Suppose there is a dataset, the ***ith entity xi*** contains its features(has ***d*** features represent as vector) and class ***yi***, then given svm model f(xi,w) to calculate the scores(as vector) in all classes,here we use **si** to represent this vector. 378 | 379 | #### loss function 380 | 381 | According to the definition of multi-class svm, we can get ith entity xi's loss fucntion: 382 | 383 | ![equation](https://latex.codecogs.com/gif.latex?l_i%20%3D%20%5Csum_%7Bi%5Cneq%20y_i%7D%5E%7Bd%7Dmax%280%2Cs_j%20-%20s_y_i+%5CDelta%20%29) 384 | 385 | For example: 386 | 387 | Suppose there are 3 classes, for the ith sample xi, we get scores = [12,-7,10],where the first class(yi) is correct.Then we make Δ=9, applying the below loss function, we can calculate the loss of xi: 388 | 389 | ![equation](https://latex.codecogs.com/gif.latex?l_i%20%3D%20max%280%2C%20-7-12+9%29+max%280%2C%2010-12+9%29) 390 | 391 | As ***w*** is a vector in loss function, we expend loss function: 392 | 393 | ![equation](https://latex.codecogs.com/gif.latex?l_i%20%3D%20%5Csum_%7Bi%5Cneq%20y_i%7D%5E%7Bd%7Dmax%280%2C%5Cvec%7Bw_j%7D%5E%7BT%7Dx_i%20-%20%5Cvec%7Bw_y_i%7D%5E%7BT%7Dx_i%20+%5CDelta%20%29) 394 | 395 | Then add l2 norm: 396 | 397 | ![equation](https://latex.codecogs.com/gif.latex?l_i%20%3D%20%5Csum_%7Bi%5Cneq%20y_i%7D%5E%7Bd%7Dmax%280%2C%5Cvec%7Bw_j%7D%5E%7BT%7Dx_i%20-%20%5Cvec%7Bw_y_i%7D%5E%7BT%7Dx_i%20+%5CDelta%29%20+%20%5Clambda%20%5Cleft%20%5C%7C%20%5Cvec%7Bw%7D%20%5Cright%20%5C%7C%5E2) 398 | 399 | #### How to calculate gradient 400 | 401 | As ***yi*** and ***j*** are different, we calculate ***wyi***'s derivative for ith entity xi first: 402 | 403 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20%5Cfrac%7B%5Cpartial%20l_i%7D%7B%5Cpartial%20%5Cvec%7Bw_y_i%7D%7D%20%26%3D%20%5Cfrac%7B%5Cpartial%20%5Csum_%7Bi%5Cneq%20y_i%7D%5E%7Bd%7Dmax%280%2C%5Cvec%7Bw_j%7D%5E%7BT%7Dx_i%20-%20%5Cvec%7Bw_y_i%7D%5E%7BT%7Dx_i%20+%5CDelta%29%20+%20%5Clambda%20%5Cleft%20%5C%7C%20%5Cvec%7Bw%7D%20%5Cright%20%5C%7C%5E2%7D%7B%5Cpartial%20%5Cvec%7Bw_y_i%7D%7D%20%5Cnonumber%5C%5C%20%26%3D%20%5Cleft%5C%7B%5Cbegin%7Bmatrix%7D%20-x_i+%202%5Clambda%20%5Cvec%7Bw_y_i%7D%26%2C%20if%20%28%5Cvec%7Bw_j%7D%5E%7BT%7Dx_i%20-%20%5Cvec%7Bw_y_i%7D%5E%7BT%7Dx_i%20+%5CDelta%29%20%3E%3D0%5C%5C%200+2%5Clambda%20%5Cvec%7Bw_y_i%7D%26%20%2Cif%20%28%5Cvec%7Bw_j%7D%5E%7BT%7Dx_i%20-%20%5Cvec%7Bw_y_i%7D%5E%7BT%7Dx_i%20+%5CDelta%29%20%3C0%20%5Cend%7Bmatrix%7D%5Cright.%20%5Cnonumber%20%5Cend%7Balign%7D) 404 | 405 | then calculate ***wj***'s derivative for ith entity xi: 406 | 407 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20%5Cfrac%7B%5Cpartial%20l_i%7D%7B%5Cpartial%20%5Cvec%7Bw_j%7D%7D%20%26%3D%20%5Cfrac%7B%5Cpartial%20%5Csum_%7Bi%5Cneq%20y_i%7D%5E%7Bd%7Dmax%280%2C%5Cvec%7Bw_j%7D%5E%7BT%7Dx_i%20-%20%5Cvec%7Bw_y_i%7D%5E%7BT%7Dx_i%20+%5CDelta%29%20+%20%5Clambda%20%5Cleft%20%5C%7C%20%5Cvec%7Bw%7D%20%5Cright%20%5C%7C%5E2%7D%7B%5Cpartial%20%5Cvec%7Bw_j%7D%7D%20%5Cnonumber%5C%5C%20%26%3D%20%5Cleft%5C%7B%5Cbegin%7Bmatrix%7D%20x_i+%202%5Clambda%20%5Cvec%7Bw_j%7D%26%2C%20if%20%28%5Cvec%7Bw_j%7D%5E%7BT%7Dx_i%20-%20%5Cvec%7Bw_y_i%7D%5E%7BT%7Dx_i%20+%5CDelta%29%20%3E%3D0%5C%5C%200+2%5Clambda%20%5Cvec%7Bw_j%7D%26%20%2Cif%20%28%5Cvec%7Bw_j%7D%5E%7BT%7Dx_i%20-%20%5Cvec%7Bw_y_i%7D%5E%7BT%7Dx_i%20+%5CDelta%29%20%3C0%20%5Cend%7Bmatrix%7D%5Cright.%20%5Cnonumber%20%5Cend%7Balign%7D) 408 | 409 | The gradient of vector ***wi*** contians ***wj***(from 1 to d) and each ***wj***'s value depends on the value of ![equation](https://latex.codecogs.com/gif.latex?%28%5Cvec%7Bw_j%7D%5E%7BT%7Dx_i%20-%20%5Cvec%7Bw_y_i%7D%5E%7BT%7Dx_i%20+%5CDelta%29): 410 | 411 | ![equation](https://latex.codecogs.com/gif.latex?%5CDelta%20_i%20%3D%20%5Bw_1%5E%7Bi%7D%20%3D%20%5Cleft%5C%7B%5Cbegin%7Bmatrix%7D%20x_i%5C%5C%20-x_i%5C%5C%200%20%5Cend%7Bmatrix%7D%5Cright....w_d%5E%7Bi%7D%20%3D%20%5Cleft%5C%7B%5Cbegin%7Bmatrix%7D%20x_i%5C%5C%20-x_i%5C%5C%200%20%5Cend%7Bmatrix%7D%5Cright.%5D) 412 | 413 | So the gradient for ***batch of data*** can be written as: 414 | 415 | ![equation](https://latex.codecogs.com/gif.latex?%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Cvec%7Bw%7D%7D%20%3D%20%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi%3D1%7D%5E%7Bn%7D%5CDelta%20%5Cvec%7Bw_i%7D%20+%202%5Clambda%20%5Cvec%7Bw%7D) 416 | 417 | #### Implement code via Pyspark 418 | 419 | ***Check [here]()*** 420 | 421 | 422 | ## Naive Bayes 423 | 424 | Naive Bayes is based on Bayes' theorem, which is : 425 | 426 | ![equation](https://latex.codecogs.com/gif.latex?P%28Y_k%7CX%29%20%3D%20%5Cfrac%7BP%28X%7CY_k%29P%28Y_k%29%7D%7B%5Csum_%7Bk%7D%5E%7B%7DP%28X%7CY%20%3D%20Y_k%29P%28Y_k%29%7D) 427 | 428 | we suppose dataset has m entities, each entitiy has n dimensions.There are k classes, define as : 429 | 430 | 431 | ![equation](https://latex.codecogs.com/gif.latex?%28x_1%5E%7B1%7D%2Cx_2%5E%7B1%7D%2C...x_n%5E%7B1%7D%2C%20y_1%29%2C%28x_1%5E%7B2%7D%2Cx_2%5E%7B2%7D%2C...x_n%5E%7B2%7D%2C%20y_2%29%2C...%2C%28x_1%5E%7Bm%7D%2Cx_2%5E%7Bm%7D%2C...x_n%5E%7Bm%7D%20%2C%20y_m%29) 432 | 433 | We can get p(X,Y)'s joint probability via Bayes' theorem: 434 | 435 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20P%28X%2CY%3DC_k%29%20%26%3D%20P%28Y%20%3D%20C_k%29P%28X%3Dx%7CY%20%3D%20C_k%29%5Cnonumber%20%5C%5C%20%26%20%3D%20P%28Y%20%3D%20C_k%29P%28X_1%3Dx_1%2CX_2%3Dx_2%2C...X_n%3Dx_n%7CY%20%3D%20C_k%29%5Cnonumber%20%5Cend%7Balign%7D) 436 | 437 | Suppose n dimensions in entity are independent of each other : 438 | 439 | ![equation](https://latex.codecogs.com/gif.latex?P%28X_1%3Dx_1%2CX_2%3Dx_2%2C...X_n%3Dx_n%7CY%20%3D%20C_k%29%5C%5C%20%3D%20P%28X_1%3Dx_1%7CY%20%3D%20C_k%29P%28X_2%3Dx_2%7CY%20%3D%20C_k%29...P%28X_n%3Dx_n%7CY%20%3D%20C_k%29) 440 | 441 | Notice some dimentions are discrete type，while others are continuous type, 442 | 443 | suppose ***Sck*** is subset where item's class is Ck, ***S k,xi*** is subset of Sk where item's i dimension is xi. 444 | 445 | ***discrete type*** 446 | 447 | ![eqution](https://latex.codecogs.com/gif.latex?P%28x_i%7CY%20%3D%20C_k%29%20%3D%20%5Cfrac%7B%7CS_%7Bk%2Cx_i%7D%7C%7D%7B%7CS_k%7C%7D) 448 | 449 | ***continuous type*** 450 | 451 | Suppose Sk,xi are subject to Gaussian distribution： 452 | 453 | ![equation](https://latex.codecogs.com/gif.latex?P%28x_i%7CY%20%3D%20C_k%29%20%5Csim%20N%28%5Cmu%20_%7BC_k%2Ci%7D%2C%5Csigma_%7BC_k%2Ci%7D%5E%7B2%7D%29) 454 | 455 | then we can get: 456 | 457 | ![equation](https://latex.codecogs.com/gif.latex?P%28x_i%7CY%20%3D%20C_k%29%20%3D%20%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%20%5Csigma%20_%7BC_k%2Ci%7D%7D%7De%5E%7B-%5Cfrac%7B%28x-%5Cmu%20_%7BC_k%2Ci%7D%29%5E2%7D%7B2%5Csigma%20_%7BC_k%2Ci%7D%5E2%7D%7D) 458 | 459 | After comparing p(X,Y) in all classes, we can get the result class which has the highest p(X,Y). 460 | 461 | 462 | ### Laplacian correction 463 | 464 | For example, if there is no sample(which belongs to Ck and ith dimension is Xa), then ![equation](https://latex.codecogs.com/gif.latex?P%28X%20%3D%20X_a%7CY%20%3D%20C_k%29%20%3D%200). 465 | 466 | Becasue of the continued product, ![equation](https://latex.codecogs.com/gif.latex?P%28X%2CY%20%3D%20C_k%29%20%3D%200) 467 | 468 | use laplacian correction: 469 | 470 | ![equation](https://latex.codecogs.com/gif.latex?P%28Y%20%3D%20C_k%29%20%3D%20%5Cfrac%7B%7CS_%7BC_k%7D%7C+1%7D%7B%7CS%7C+N%7D) 471 | 472 | where |Sck| is the number of subset Sck(where items are class ck) , |S| is number of whole dataset, N is the number of classes 473 | 474 | ![equation](https://latex.codecogs.com/gif.latex?P%28x_i%7CY%20%3D%20C_k%29%20%3D%20%5Cfrac%7B%7CS_%7BC_%7Bk%2Cx_i%7D%7D%7C+1%7D%7B%7CS_%7BC_k%7D%7C+N_i%7D) 475 | 476 | where |S k,xi| is the number of the subset of Sck where item's i dimension is xi, Ni is the number of possible value of ith attribute. 477 | 478 | 479 | ### GaussianNB，MultinomialNB, BernoulliNB 480 | 481 | MultinomialNB is naive Bayes with a priori Gaussian distribution, multinomialNB is naive Bayes with a priori multinomial distribution and BernoullinB is naive Bayes with a priori Bernoulli distribution. 482 | 483 | These three classes are applicable to different classification scenarios. In general, if the distribution of sample features is mostly continuous values, Gaussiannb is better. MultinomialNb is appropriate if the most of the sample features are multivariate discrete values. If the sample features are binary discrete values or very sparse multivariate discrete values, Bernoullinb should be used. 484 | 485 | 486 | 487 | 488 | 489 | ## Optimizer 490 | 491 | 492 | Variations of Gradient Descent depends on size of data that be used in each iteration: 493 | 494 | • Full Batch Gradient Descent (Using the whole data set (size n)) 495 | • Stochastic Gradient Descent (SGD) (Using one sample per iteration (size 1)) 496 | • Mini Batch Gradient Descent (Using a mini batch of data (size m < n)) 497 | 498 | **BGD** has a disadvantage: In each iteration, as it calculates gradients from whole dataset, this process will take lots of time.BGD can't overcome local minimum problem, because we can not add new data to trainning dataset.In other word, when function comes to local minimum point, full batch gradient will be 0, which means optimization process will stop. 499 | 500 | **SGD** is always used in online situation for its speed.But since SGD uses only one sample in each iteration, the gradient can be affacted by noisy point, causeing a fact that function may not converge to optimal solution. 501 | 502 | **MSGD** finds a trade-off between SGD and BGD.Different from BGD and SGD, MSGD only pick a small batch-size of data in each iteration, which not only minimized the impact from noisy point, but also reduced trainning time and increased the accuracy. 503 | 504 | ### Learning rate 505 | 506 | The learning rate is a vital parameter in gradient descent as learning rate is responsible for convergence, if lr is small, convergent speed will be slow, on the contrary,when lr is large, function will converge very fast. 507 | 508 | compare different learning rate: 509 | 510 | ![image](https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fimg2018.cnblogs.com%2Fblog%2F1217276%2F201810%2F1217276-20181007182807634-196732269.png&refer=http%3A%2F%2Fimg2018.cnblogs.com&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1617069776&t=b25621a89b513f8b765ac8f116bee051) 511 | 512 | 513 | So how to find a proper learning rate? If we set lr a large value, the function will converge very fast at beginning but may miss the optimal solution,but if we set a small value, it will cost too much time to converge. As iterations going on, we hope that learning rate becomes smaller.Because when function close to optimal solution, the changing step should be small to find best solution.So we need to gradually change learning rate. 514 | 515 | 516 | >Here is a very simple method, which name is ***Bold Driver*** to change learning rate dynamicly: 517 | > 518 | >At each iteration, compute the cost l(θ0,θ1...) 519 | > 520 | >Better than last time? 521 | > 522 | >If cost decreases, increase learning rate 523 | >l = 1.05 * l 524 | > 525 | >Worse than last time? 526 | > 527 | >l = 0.5 * l 528 | >If cost increases, decrease rate 529 | 530 | A better method is ***Time-Based Decay*** .The mathematical form of time-based decay is lr = lr0/(1+kt) 531 | 532 | where lr, k are hyperparameters and t is the iteration number. 533 | 534 | Those graphs illustrate the advantages of Time-Based Decay lr vs constant lr: 535 | 536 | ***constant lr*** 537 | 538 | ![image](https://miro.medium.com/max/864/1*Lv7-jMtHOoucryv9mUtFGg.jpeg) 539 | 540 | 541 | ***Time-Based Decay lr*** 542 | 543 | ![image](https://miro.medium.com/max/864/1*YpzU0MkpNaZ8f6cGvqex7g.jpeg) 544 | 545 | Also, we know that the weights for each coefficent is different, which means gradients of some coefficents are large while some are little.So in traditional SGD, changes of coefficents are not synchronous.So we need to balance the coefficents when doing gradient descent. 546 | 547 | To deal with this issue, we can use **SGDM**, **Adagrad(adaptive gradient algorithm)**, **RMSProp**, **Adam** 548 | 549 | ### SGDM 550 | 551 | SGDM is SGD with momentum.It implement momentum to gradient: 552 | 553 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20m_j%20%5Cleftarrow%20%5Clambda%20m_%7Bj%7D%20+%20%5Ceta%20%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%5Cnonumber%20%5C%5C%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20+%20m_j%5Cnonumber%20%5Cend%7Balign%7D) 554 | 555 | where m0 = 0, λ is momentum's coefficent, η is learing rate. 556 | 557 | 558 | 559 | visualization: 560 | 561 | ![image](https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbnRyYW5ldHByb3h5LmFsaXBheS5jb20vc2t5bGFyay9sYXJrLzAvMjAyMC9wbmcvOTMwNC8xNTk4NTIxNDQ4NTQwLWViMjEwNTQ5LWNiOTMtNDIxMC05NDJmLTg2Mzk0Y2Y4Njk5ZC5wbmc?x-oss-process=image/format,png#align=left&display=inline&height=306&margin=%5Bobject%20Object%5D&name=image.png&originHeight=682&originWidth=1080&size=488854&status=done&style=none&width=484wZw#pic_center) 562 | 563 | 564 | ### Adagrad 565 | 566 | Adagrad will give each coefficent a proper learning rate: 567 | 568 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20h_j%20%5Cleftarrow%20h_j%20+%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5E%7B2%7D%20%5C%5C%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20-%20%5Ceta%20%5Cfrac%7B1%7D%7B%5Csqrt%7Bh_j%7D%7D%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%20%5Cend%7Balign%7D) 569 | 570 | θ : coefficents 571 | 572 | ∂l/∂θ : gradient 573 | 574 | η: learning rate 575 | 576 | hj: sum of squares of all the previous θj's gradients 577 | 578 | when updating coefficent, we can adjust the scale by mutiplying 1/√h. 579 | 580 | But as iteration going on, h will be very large, making updating step becomes very small. 581 | 582 | ### RMSProp 583 | 584 | **RMSProp** can optimize this problem.RMSProp uses an exponential weighted average to eliminate swings in gradient descent: a larger derivative of a dimension means a larger exponential weighted average, and a smaller derivative means a smaller exponential weighted average. This ensures that the derivatives of each dimension are of the same order of magnitude, thus reducing swings: 585 | 586 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20h_j%20%5Cleftarrow%20%5Cbeta%20h_j%20+%281-%5Cbeta%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5E%7B2%7D%20%5C%5C%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20-%20%5Ceta%20%5Cfrac%7B1%7D%7B%5Csqrt%7Bh_j+c%7D%7D%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%20%5Cend%7Balign%7D) 587 | 588 | √hj can be 0 some times, so we add a small value c to √hj 589 | 590 | ***RMSProp code here*** 591 | 592 | ``` 593 | def RMSprop(x, y, lr=0.01, iter_count=500, batch_size=4, beta=0.9): 594 | length, features = x.shape 595 | data = np.column_stack((x, np.ones((length, 1)))) 596 | w = np.zeros((features + 1, 1)) 597 | h, eta = 0, 10e-7 598 | start, end = 0, batch_size 599 | for i in range(iter_count): 600 | # calculate gradient 601 | dw = np.sum((np.dot(data[start:end], w) - y[start:end]) * data[start:end], axis=0) / length 602 | # calculate sum of square of gradients 603 | h = beta * h + (1 - beta) * np.dot(dw, dw) 604 | # update w 605 | w = w - (lr / np.sqrt(eta + h)) * dw.reshape((features + 1, 1)) 606 | 607 | start = (start + batch_size) % length 608 | if start > length: 609 | start -= length 610 | end = (end + batch_size) % length 611 | if end > length: 612 | end -= length 613 | return w 614 | ``` 615 | ### Adam 616 | 617 | ***Adam*** is another powerful optimizer.It not only saved the sum of square of history gradients(h) but also save sum of history gradients(m, known as momentum): 618 | 619 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20m_j%20%5Cleftarrow%20%5Cbeta_1%20m_j%20+%281-%5Cbeta%20_1%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5Cnonumber%5C%5C%20h_j%20%5Cleftarrow%20%5Cbeta_2%20h_j%20+%281-%5Cbeta%20_2%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5E%7B2%7D%5Cnonumber%20%5Cend%7Balign%7D) 620 | 621 | If m and h are initialized to the 0 vectors, they are biased to 0, so bias correction is done to offset these biases by calculating the bias corrected m and h: 622 | 623 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20m_j%20%5Cleftarrow%20%5Cfrac%7Bm_j%7D%7B1-%5Cbeta%20_1%5E%7Bt%7D%7D%5Cnonumber%5C%5C%20h_j%20%5Cleftarrow%20%5Cfrac%7Bh_j%7D%7B1-%5Cbeta%20_2%5E%7Bt%7D%7D%5Cnonumber%20%5Cend%7Balign%7D) 624 | 625 | t means t th iteration. 626 | 627 | ![equation](https://latex.codecogs.com/gif.latex?%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20-%20%5Ceta%20%5Cfrac%7B1%7D%7B%5Csqrt%7Bh_j+c%7D%7Dm_j) 628 | 629 | 630 | ***Adam code here*** 631 | 632 | ``` 633 | def Adam(x, y, lr=0.01, iter_count=500, batch_size=4, beta1=0.9,beta2 = 0.999): 634 | length, features = x.shape 635 | data = np.column_stack((x, np.ones((length, 1)))) 636 | w = np.zeros((features + 1, 1)) 637 | m, h,eta = 0, 0,10e-7 638 | start, end = 0, batch_size 639 | for i in range(iter_count): 640 | # calculate gradient 641 | dw = np.sum((np.dot(data[start:end], w) - y[start:end]) * data[start:end], axis=0) / length 642 | # calculate sums 643 | m = beta1 * m + (1 - beta1) * dw 644 | h = beta2 * h + (1 - beta2) * np.dot(dw, dw) 645 | # bias correction 646 | m = m/(1- beta1) 647 | h = h/(1- beta2) 648 | # update w 649 | w = w - (lr / np.sqrt(eta + h)) * m.reshape((features + 1, 1)) 650 | 651 | start = (start + batch_size) % length 652 | if start > length: 653 | start -= length 654 | end = (end + batch_size) % length 655 | if end > length: 656 | end -= length 657 | return w 658 | ``` 659 | 660 | ### how to choose optimizer 661 | 662 | By far, the most popular models are SGDM and Adam. 663 | 664 | ![image](https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbnRyYW5ldHByb3h5LmFsaXBheS5jb20vc2t5bGFyay9sYXJrLzAvMjAyMC9wbmcvOTMwNC8xNTk4NTIzMDIxNTA5LTMyNTI1OGIwLTI5NzItNGNiNy04MDhkLTg4OTQ0Mzk0MWE3ZC5wbmc?x-oss-process=image/format,png#align=left&display=inline&height=302&margin=%5Bobject%20Object%5D&name=image.png&originHeight=604&originWidth=1074&size=76636&status=done&style=none&width=537wZw#pic_center) 665 | 666 | This graph illustrates that SGDM is always used in computer vision whereas Adam are popular in NLP. 667 | 668 | 669 | ### optimize Adam and SGDM 670 | 671 | For Adam, there are ***SWATS***,***AMSGrad***,***AdaBound***,and ***AdamW***. 672 | 673 | For SGDM, there are ***SGDMW***,***SGDNM*** 674 | 675 | #### SWATS 676 | 677 | combine Adam and SGDM: 678 | 679 | ![image](https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbnRyYW5ldHByb3h5LmFsaXBheS5jb20vc2t5bGFyay9sYXJrLzAvMjAyMC9wbmcvOTMwNC8xNTk4NTI0MjE0NzU0LTkwY2VlMmE0LTFiZWYtNGRhNC1hY2M5LTljYjVhMjE2ZTBmMS5wbmc?x-oss-process=image/format,png#align=left&display=inline&height=146&margin=%5Bobject%20Object%5D&name=image.png&originHeight=292&originWidth=1066&size=43146&status=done&style=none&width=533wZw#pic_center) 680 | 681 | #### AMSGrad 682 | 683 | optimize Adam in changing the way to update ***hj***: 684 | 685 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20m_j%20%5E%7Bi%7D%5Cleftarrow%20%5Cbeta_1%20m_j%20%5E%7Bi-1%7D+%281-%5Cbeta%20_1%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5Cnonumber%5C%5C%20h_j%5E%7Bi%7D%20%5Cleftarrow%20%5Cbeta_2%20h_%7Bj%7D%5E%7Bi-1%7D+%281-%5Cbeta%20_2%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5E%7B2%7D%5Cnonumber%5C%5C%20AMSGrad%20%3A%20h_j%5E%7Bi%7D%3D%20max%28h_j%5E%7Bi%7D%2Ch_%7Bj%7D%5E%7Bi-1%7D%29%5Cnonumber%20%5Cend%7Balign%7D) 686 | 687 | AMSGrad makes learning rate monotone decrease and waives small gradients. 688 | 689 | #### AdaBound 690 | 691 | AdaBound limits learning rate in a certain scale: 692 | 693 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20m_j%20%5E%7Bi%7D%5Cleftarrow%20%5Cbeta_1%20m_j%20%5E%7Bi-1%7D+%281-%5Cbeta%20_1%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5Cnonumber%5C%5C%20h_j%5E%7Bi%7D%20%5Cleftarrow%20%5Cbeta_2%20h_%7Bj%7D%5E%7Bi-1%7D+%281-%5Cbeta%20_2%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5E%7B2%7D%5Cnonumber%5C%5C%20AMSBound%20%3A%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20-%20clip%28%5Ceta%20%5Cfrac%7B1%7D%7B%5Csqrt%7Bh_j+c%7D%7D%29m_j%20%5Cnonumber%5C%5C%20where%20%3A%20clip%28x%29%3D%20np.clip%28x%2C0.1-%5Cfrac%7B0.1%7D%7B%281-%5Cbeta%20_2%29t+1%7D%2C0.1+%5Cfrac%7B0.1%7D%7B%281-%5Cbeta%20_2%29t%7D%29%5Cnonumber%20%5Cend%7Balign%7D) 694 | 695 | 696 | 697 | #### AdamW 698 | 699 | Add weight decay to Adam: 700 | 701 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20m_j%20%5E%7Bi%7D%5Cleftarrow%20%5Cbeta_1%20m_j%20%5E%7Bi-1%7D+%281-%5Cbeta%20_1%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5Cnonumber%5C%5C%20h_j%5E%7Bi%7D%20%5Cleftarrow%20%5Cbeta_2%20h_%7Bj%7D%5E%7Bi-1%7D+%281-%5Cbeta%20_2%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5E%7B2%7D%5Cnonumber%5C%5C%20AdamW%20%3A%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20-%20%5Ceta%20%28%5Cfrac%7B1%7D%7B%5Csqrt%7Bh_j+c%7D%7Dm_j+%5Cgamma%20%5Ctheta%20_j%20%29%5Cnonumber%20%5Cend%7Balign%7D) 702 | 703 | 704 | 705 | #### SGDMW 706 | 707 | Add weight decay to SGDM: 708 | 709 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20m_j%20%5Cleftarrow%20%5Clambda%20m_%7Bj%7D%20+%20%5Ceta%20%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%5Cnonumber%20%5C%5C%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20+%20m_j+%20%5Cgamma%20%5Ctheta%20_j%5Cnonumber%20%5Cend%7Balign%7D) 710 | 711 | where m0 = 0, λ is momentum's coefficent, η is learing rate, γ is weight decay coefficent. 712 | 713 | 714 | #### SGDMN 715 | 716 | SGDMN(SGDM with Nesterov) is aimed to solve local optimal problem.When local optimal problem happend, SGDMN will do an additional calculation to determine whether to stop iteration: 717 | 718 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20m_j%20%5Cleftarrow%20%5Clambda%20m_%7Bj%7D%20%5Cnonumber%20%5C%5C%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20+%20m_j%20%5Cnonumber%5C%5C%20m_j%20%5Cleftarrow%20%5Clambda%20m_%7Bj%7D%20+%20%5Ceta%20%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%5Cnonumber%20%5C%5C%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20+%20m_j%20%5Cnonumber%20%5Cend%7Balign%7D) 719 | 720 | 721 | #### NAdam 722 | 723 | Nadam(Adam with Nesterov) is aimed to solve local optimal problem.When local optimal problem happend, NAdam will do an additional calculation to determine whether to stop iteration: 724 | 725 | ![equation](https://latex.codecogs.com/gif.latex?%5Cbegin%7Balign%7D%20m_j%20%5Cleftarrow%20%5Cbeta_1%20m_j%5Cnonumber%5C%5C%20h_j%20%5Cleftarrow%20%5Cbeta_2%20h_j%20%5Cnonumber%5C%5C%20m_j%20%5Cleftarrow%20%5Cfrac%7Bm_j%7D%7B1-%5Cbeta%20_1%5E%7Bt%7D%7D%5Cnonumber%5C%5C%20h_j%20%5Cleftarrow%20%5Cfrac%7Bh_j%7D%7B1-%5Cbeta%20_2%5E%7Bt%7D%7D%5Cnonumber%5C%5C%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20-%20%5Ceta%20%5Cfrac%7B1%7D%7B%5Csqrt%7Bh_j+c%7D%7Dm_j%20%5Cnonumber%5C%5C%20m_j%20%5Cleftarrow%20%5Cbeta_1%20m_j%20+%281-%5Cbeta%20_1%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5Cnonumber%5C%5C%20h_j%20%5Cleftarrow%20%5Cbeta_2%20h_j%20+%281-%5Cbeta%20_2%29%28%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%29%5E%7B2%7D%5Cnonumber%5C%5C%20m_j%20%5Cleftarrow%20%5Clambda%20m_%7Bj%7D%20+%20%5Ceta%20%5Cfrac%7B%5Cpartial%20l%7D%7B%5Cpartial%20%5Ctheta%20_j%7D%5Cnonumber%20%5C%5C%20%5Ctheta%20_j%20%5Cleftarrow%20%5Ctheta%20_j%20-%20%5Ceta%20%5Cfrac%7B1%7D%7B%5Csqrt%7Bh_j+c%7D%7Dm_j%20%5Cnonumber%20%5Cend%7Balign%7D) 726 | 727 | 728 | comparation between these optimizers ,lets see the differenes of those optimizers: 729 | 730 | ![image](https://miro.medium.com/max/892/1*63HMdMyw_XDcNkRCQ1nrpw.png) 731 | 732 | 733 | 734 | ### Now I will share a kaggle project based on ML: 735 | 736 | ***kaggel link :*** 737 | 738 | https://www.kaggle.com/uciml/mushroom-classification 739 | 740 | 741 | This project is mushroom classification. There are about 2 categories (edible or poisonous) and 8124 records (52% edible and 48% poisonous). 742 | For this project, I have tried three machine learning models: 743 | SVM , Random forest and logistic regression. 744 | All three models have good performances: 745 | Compare recall, precision, f1-score for both class 746 | F1_score > 95% 747 | Accuracy > 95% 748 | 749 | This dataset includes descriptions of hypothetical samples corresponding to 23 species of gilled mushrooms in the Agaricus 750 | and Lepiota Family Mushroom drawn from The Audubon Society Field Guide to North American Mushrooms (1981). 751 | Each species is identified as definitely edible, definitely poisonous, or of unknown edibility and not recommended. 752 | This latter class was combined with the poisonous one. The Guide clearly states that there is no simple rule for determining the edibility of a mushroom; 753 | no rule like "leaflets three, let it be'' for Poisonous Oak and Ivy. 754 | 755 | ***Code*** 756 | 757 | #### Check [here](https://github.com/gnayoaixgnaw/machine_learning_project/blob/main/mashroom/cs677project.ipynb) 758 | 759 | 760 | 761 | 762 | 763 | -------------------------------------------------------------------------------- /image/1: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /image/CodeCogsEqn (1).gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/CodeCogsEqn (1).gif -------------------------------------------------------------------------------- /image/adagrad1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/adagrad1.png -------------------------------------------------------------------------------- /image/adaptivelr.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/adaptivelr.png -------------------------------------------------------------------------------- /image/derivative.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/derivative.png -------------------------------------------------------------------------------- /image/derivative1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/derivative1.png -------------------------------------------------------------------------------- /image/derivative2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/derivative2.png -------------------------------------------------------------------------------- /image/imbd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/imbd.png -------------------------------------------------------------------------------- /image/lr_large.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/lr_large.png -------------------------------------------------------------------------------- /image/lr_small.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/lr_small.png -------------------------------------------------------------------------------- /image/mnist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/mnist.png -------------------------------------------------------------------------------- /image/pseudocode1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/pseudocode1.png -------------------------------------------------------------------------------- /image/pseudocode2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/image/pseudocode2.png -------------------------------------------------------------------------------- /mashroom/cs677-project.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gnayoaixgnaw/machine_learning_project/6b39513bea4b16288fb5e5ad06b4d7c81c1ecd72/mashroom/cs677-project.pptx -------------------------------------------------------------------------------- /mashroom/cs677project.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 2, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import pandas as pd\n", 10 | "import numpy as np\n", 11 | "\n", 12 | "from sklearn.decomposition import PCA\n", 13 | "from sklearn.pipeline import make_pipeline\n", 14 | "from sklearn.model_selection import train_test_split\n", 15 | "from sklearn.model_selection import GridSearchCV\n", 16 | "from sklearn.svm import SVC\n", 17 | "from sklearn.metrics import classification_report\n", 18 | "from sklearn.metrics import f1_score\n", 19 | "from sklearn.metrics import confusion_matrix\n", 20 | "from sklearn.metrics import classification_report\n", 21 | "from sklearn.neighbors import KNeighborsClassifier\n", 22 | "from sklearn.linear_model import LogisticRegression\n", 23 | "from sklearn.linear_model import SGDClassifier\n", 24 | "from sklearn.ensemble import RandomForestClassifier\n", 25 | "from sklearn.model_selection import cross_val_score\n", 26 | "\n", 27 | "\n", 28 | "\n", 29 | "import matplotlib.pyplot as plt\n", 30 | "\n" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": 3, 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "raw_data = pd.read_csv('mushrooms.csv')\n", 40 | "dropna_data = raw_data.dropna(axis=0, how='any')" 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": 4, 46 | "metadata": {}, 47 | "outputs": [ 48 | { 49 | "data": { 50 | "text/plain": [ 51 | "array(['class', 'cap-shape', 'cap-surface', 'cap-color', 'bruises',\n", 52 | " 'odor', 'gill-attachment', 'gill-spacing', 'gill-size',\n", 53 | " 'gill-color', 'stalk-shape', 'stalk-root',\n", 54 | " 'stalk-surface-above-ring', 'stalk-surface-below-ring',\n", 55 | " 'stalk-color-above-ring', 'stalk-color-below-ring', 'veil-type',\n", 56 | " 'veil-color', 'ring-number', 'ring-type', 'spore-print-color',\n", 57 | " 'population', 'habitat'], dtype=object)" 58 | ] 59 | }, 60 | "execution_count": 4, 61 | "metadata": {}, 62 | "output_type": "execute_result" 63 | } 64 | ], 65 | "source": [ 66 | "columns = dropna_data.columns.values\n", 67 | "columns\n", 68 | "# Attribute Information: (classes: edible=e, poisonous=p)\n", 69 | "\n", 70 | "# cap-shape: bell=b,conical=c,convex=x,flat=f, knobbed=k,sunken=s\n", 71 | "\n", 72 | "# cap-surface: fibrous=f,grooves=g,scaly=y,smooth=s\n", 73 | "\n", 74 | "# cap-color: brown=n,buff=b,cinnamon=c,gray=g,green=r,pink=p,purple=u,red=e,white=w,yellow=y\n", 75 | "\n", 76 | "# bruises: bruises=t,no=f\n", 77 | "\n", 78 | "# odor: almond=a,anise=l,creosote=c,fishy=y,foul=f,musty=m,none=n,pungent=p,spicy=s\n", 79 | "\n", 80 | "# gill-attachment: attached=a,descending=d,free=f,notched=n\n", 81 | "\n", 82 | "# gill-spacing: close=c,crowded=w,distant=d\n", 83 | "\n", 84 | "# gill-size: broad=b,narrow=n\n", 85 | "\n", 86 | "# gill-color: black=k,brown=n,buff=b,chocolate=h,gray=g, green=r,orange=o,pink=p,purple=u,red=e,white=w,yellow=y\n", 87 | "\n", 88 | "# stalk-shape: enlarging=e,tapering=t\n", 89 | "\n", 90 | "# stalk-root: bulbous=b,club=c,cup=u,equal=e,rhizomorphs=z,rooted=r,missing=?\n", 91 | "\n", 92 | "# stalk-surface-above-ring: fibrous=f,scaly=y,silky=k,smooth=s\n", 93 | "\n", 94 | "# stalk-surface-below-ring: fibrous=f,scaly=y,silky=k,smooth=s\n", 95 | "\n", 96 | "# stalk-color-above-ring: brown=n,buff=b,cinnamon=c,gray=g,orange=o,pink=p,red=e,white=w,yellow=y\n", 97 | "\n", 98 | "# stalk-color-below-ring: brown=n,buff=b,cinnamon=c,gray=g,orange=o,pink=p,red=e,white=w,yellow=y\n", 99 | "\n", 100 | "# veil-type: partial=p,universal=u\n", 101 | "\n", 102 | "# veil-color: brown=n,orange=o,white=w,yellow=y\n", 103 | "\n", 104 | "# ring-number: none=n,one=o,two=t\n", 105 | "\n", 106 | "# ring-type: cobwebby=c,evanescent=e,flaring=f,large=l,none=n,pendant=p,sheathing=s,zone=z\n", 107 | "\n", 108 | "# spore-print-color: black=k,brown=n,buff=b,chocolate=h,green=r,orange=o,purple=u,white=w,yellow=y\n", 109 | "\n", 110 | "# population: abundant=a,clustered=c,numerous=n,scattered=s,several=v,solitary=y\n", 111 | "\n", 112 | "# habitat: grasses=g,leaves=l,meadows=m,paths=p,urban=u,waste=w,woods=d" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": 5, 118 | "metadata": {}, 119 | "outputs": [ 120 | { 121 | "name": "stderr", 122 | "output_type": "stream", 123 | "text": [ 124 | ":9: MatplotlibDeprecationWarning: Using a string of single character colors as a color sequence is deprecated. Use an explicit list instead.\n", 125 | " autolabel(plt.bar(range(len(num_list)), num_list, color='rgb', tick_label=name_list))\n" 126 | ] 127 | }, 128 | { 129 | "data": { 130 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAbnklEQVR4nO3df3BV9bnv8fcTFCgG50ohNrDlRkpUSKARUpTxHIoynELLFa9WDqmnEKLllELRtAyDvb0TsGUUS496W8s0VafBo1LFI1CLWAakom3ARFMVlAYlB3bI8ENFEC0aeO4fWcSE7JAfJGsH1uc1sydrP3ut/Xx3Bj5Z+a4fMXdHRESiISXZAxARkfAo9EVEIkShLyISIQp9EZEIUeiLiESIQl9EJEIiG/pmdomZvWBmb5nZNjO7Paj/3MzeNrPXzewZM/sfQf18MysxszeCbe5M8J5rzOzNsD+LiEhrRTb0gVrgR+4+BLgamG1mQ4H1QLa7Dwf+DpwM95uBHu4+DBgJ/LuZZZx8MzO7EfgovOGLiLRdZEPf3Wvc/dVg+QjwFjDA3f/k7rXBaqVA7OQmwAVmdh7wBeBT4DCAmaUCPwR+FuJHEBFpM+vqV+T27dvXMzIyOrXHsWPH2LFjB1lZWXTr1q2+vnPnTi666CK++MUv4u7s2rWLI0eOcOLECWKxGP369QNgz549pKam0qtXL3bu3ElWVlanjldEpCXl5eUH3b1fkxfcvUs/Ro4c6Z3pyJEjPmLECH/66acb1X/2s5/5DTfc4CdOnHB395deesm//e1v+6effur79u3zyy67zN955x1/7bXXfNKkSe7uvmvXLs/KyurU8YqItAZQ5gky9bzwf/50HZ999hk33XQTt9xyCzfeeGN9vaSkhGeffZYNGzZgZgA8/vjjTJgwgfPPP5+0tDSuueYaysrKeO+99ygvLycjI4Pa2lr279/P2LFj2bRpU5I+lYhI86I8p8+tt97KkCFD+OEPf1hfX7duHUuWLGHNmjX06tWrvj5w4EA2btyIu3P06FFKS0u54oormDVrFnv37qWqqoqXXnqJyy67TIEvIl1WZEP/5Zdf5tFHH2Xjxo3k5OSQk5PD2rVrmTNnDkeOHGH8+PHk5OTwve99D4DZs2fz0UcfkZ2dzVe/+lVmzJjB8OHDk/wpRETapssfyM3NzfWysrJkD0NE5KxiZuXunntqPbJ7+iIiUaTQFxGJEIW+iEiEKPRFRCJEoS8iEiHn9sVZwYVVnaqLn/0kItKQ9vRFRCJEoS8iEiEKfRGRCFHoi4hEiEJfRCRCFPoiIhGi0BcRiRCFvohIhCj0RUQiRKEvIhIhCn0RkQhR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIQo9EVEIkShLyISIQp9EZEIUeiLiESIQl9EJEJaDH0zu8TMXjCzt8xsm5ndHtT7mNl6M6sMvl7UYJs7zWynme0ws683qI80szeC1/6fmVnnfCwREUmkNXv6tcCP3H0IcDUw28yGAguADe6eCWwInhO8NhXIAiYAvzazbsF7LQNmApnBY0IHfhYREWlBi6Hv7jXu/mqwfAR4CxgATAZKgtVKgBuC5cnACnc/5u67gJ3AKDNLBy5097+6uwPLG2wjIiIhaNOcvpllAFcCW4CL3b0G6n4wAGnBagOAPQ02iwe1AcHyqfVEfWaaWZmZlR04cKAtQxQRkdNodeibWSrwNHCHux8+3aoJan6aetOie7G757p7br9+/Vo7RBERaUGrQt/Mzqcu8B9z9/8KyvuCKRuCr/uDehy4pMHmMWBvUI8lqIuISEhac/aOAQ8Db7n7fzR4aQ0wPVieDqxuUJ9qZj3M7FLqDthuDaaAjpjZ1cF7TmuwTaQUFBSQlpZGdnZ2fa2iooKrr76anJwccnNz2bp1a6Ntdu/eTWpqKkuXLq2vlZeXM2zYMAYPHszcuXOpO1QiItK81uzpXwN8B7jOzCqCxzeAe4DxZlYJjA+e4+7bgCeB7cA6YLa7Hw/eaxbwEHUHd98BnuvID3O2yM/PZ926dY1q8+fPp6ioiIqKCu666y7mz5/f6PXCwkImTpzYqDZr1iyKi4uprKyksrKyyXuKiJzqvJZWcPeXSDwfDzCumW0WA4sT1MuA7KZbRMuYMWOoqqpqVDMzDh+uO1Ty4Ycf0r9///rXVq1axaBBg7jgggvqazU1NRw+fJjRo0cDMG3aNFatWtXkB4OISEMthr6E4/777+frX/868+bN48SJE/zlL38B4OjRoyxZsoT169c3mtqprq4mFvv8EEksFqO6ujr0cYvI2UW3Yegili1bxn333ceePXu47777uPXWWwEoKiqisLCQ1NTURusnmr/XBc4i0hLt6XcRJSUlPPDAAwDcfPPN3HbbbQBs2bKFlStXMn/+fA4dOkRKSgo9e/bkpptuIh7//LKHeDzeaEpIRCQRhX4X0b9/f/785z8zduxYNm7cSGZmJgCbN2+uX2fhwoWkpqYyZ84cAHr37k1paSlXXXUVy5cv5wc/+EFSxi4iZw+FfhLk5eWxadMmDh48SCwWY9GiRfz2t7/l9ttvp7a2lp49e1JcXNzi+yxbtoz8/Hw++eQTJk6cqIO4ItIi6+rndufm5npZWVn7Ng5jjruLf/9EJJrMrNzdc0+t60CuiEiEKPRFRCJEoS8iEiE6kNtJbFHnH0/wIh1PEJG20Z6+iEiEKPRFRCJEoS8iEiEKfRGRCFHoi4hEiEJfRCRCFPoiIhGi0BcRiRCFvohIhCj0RUQiRKEvIhIhCn0RkQhR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIQo9EVEIkShHzEFBQWkpaWRnZ1dX1u4cCEDBgwgJyeHnJwc1q5dW//a3XffzeDBg7n88st5/vnn6+vl5eUMGzaMwYMHM3fuXNz1pxtFzgYK/YjJz89n3bp1TeqFhYVUVFRQUVHBN77xDQC2b9/OihUr2LZtG+vWreP73/8+x48fB2DWrFkUFxdTWVlJZWVlwvcUka5HoR8xY8aMoU+fPq1ad/Xq1UydOpUePXpw6aWXMnjwYLZu3UpNTQ2HDx9m9OjRmBnTpk1j1apVnTxyEekICn0B4Fe/+hXDhw+noKCADz74AIDq6mouueSS+nVisRjV1dVUV1cTi8Wa1EWk61PoC7NmzeKdd96hoqKC9PR0fvSjHwEknKc3s2brItL1KfSFiy++mG7dupGSksJ3v/tdtm7dCtTtwe/Zs6d+vXg8Tv/+/YnFYsTj8SZ1Een6FPpCTU1N/fIzzzxTf2bP9ddfz4oVKzh27Bi7du2isrKSUaNGkZ6eTu/evSktLcXdWb58OZMnT07W8EWkDc5L9gAkXHl5eWzatImDBw8Si8VYtGgRmzZtoqKiAjMjIyOD3/zmNwBkZWUxZcoUhg4dynnnnceDDz5It27dAFi2bBn5+fl88sknTJw4kYkTJybzY4lIK1lL51eb2SPAJGC/u2cHtYXAd4EDwWo/dve1wWt3ArcCx4G57v58UB8J/A74ArAWuN1bcXJ3bm6ul5WVtfmDBYNv33Zt0cxHsEWd39uLdG68iCRmZuXunntqvTXTO78DJiSo3+fuOcHjZOAPBaYCWcE2vzazbsH6y4CZQGbwSPSeIiLSiVoMfXd/EXi/le83GVjh7sfcfRewExhlZunAhe7+12DvfjlwQ3sHLSIi7XMmB3LnmNnrZvaImV0U1AYAexqsEw9qA4LlU+siIhKi9ob+MuDLQA5QA/wiqCeayPbT1BMys5lmVmZmZQcOHGhuNWmGWec/ROTs1K7Qd/d97n7c3U8AvwVGBS/FgUsarBoD9gb1WIJ6c+9f7O657p7br1+/9gxRREQSaFfoB3P0J/1v4M1geQ0w1cx6mNml1B2w3eruNcARM7va6i7dnAasPoNxi4hIO7R4nr6ZPQGMBfqaWRwoAsaaWQ51UzRVwL8DuPs2M3sS2A7UArPd/XjwVrP4/JTN54KHiIiEqMXQd/e8BOWHT7P+YmBxgnoZkN10CxERCYtuwyAiEiEKfRGRCFHoi4hEiEJfRCRCFPoiIhGi0BcRiRCFvohIhCj0RUQiRKEvIhIhCn0RkQhR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIQo9EVEIkShLyISIQp9EZEIUeiLiESIQl9EJEIU+iIiEaLQFxGJEIW+iEiEKPQlNAUFBaSlpZGdnV1fe//99xk/fjyZmZmMHz+eDz74AIDHHnuMnJyc+kdKSgoVFRV8/PHHfPOb3+SKK64gKyuLBQsWJOvjiJyVFPoSmvz8fNatW9eods899zBu3DgqKysZN24c99xzDwC33HILFRUVVFRU8Oijj5KRkUFOTg4A8+bN4+233+a1117j5Zdf5rnnngv9s4icrRT6EpoxY8bQp0+fRrXVq1czffp0AKZPn86qVauabPfEE0+Ql5cHQK9evbj22msB6N69OyNGjCAej3fyyEXOHQp9Sap9+/aRnp4OQHp6Ovv372+yzu9///v60G/o0KFD/OEPf2DcuHEt9mnL1NJJu3fvJjU1laVLl9bXysvLGTZsGIMHD2bu3Lm4e6s/q0hXoNCXLm3Lli306tWrUVgD1NbWkpeXx9y5cxk0aFCL79OWqaWTCgsLmThxYqParFmzKC4uprKyksrKyibvKdLVKfQlqS6++GJqamoAqKmpIS0trdHrK1asSLiXP3PmTDIzM7njjjta1aetU0urVq1i0KBBZGVl1ddqamo4fPgwo0ePxsyYNm1awukoka5MoS9Jdf3111NSUgJASUkJkydPrn/txIkTPPXUU0ydOrXRNj/5yU/48MMPuf/++8+od3NTS0ePHmXJkiUUFRU1Wr+6uppYLFb/PBaLUV1dfUZjEAmbQl9Ck5eXx+jRo9mxYwexWIyHH36YBQsWsH79ejIzM1m/fn2jUzBffPFFYrFYo+mbeDzO4sWL2b59OyNGjCAnJ4eHHnqoQ8dZVFREYWEhqampjeqJ5u/NrEN7i3S285I9AImOJ554ImF9w4YNCetjx46ltLS0US0Wi3XYwdOTU0vp6emNppa2bNnCypUrmT9/PocOHSIlJYWePXty0003NTpTKB6P079//w4Zi0hYtKcvkdXc1NLmzZupqqqiqqqKO+64gx//+MfMmTOH9PR0evfuTWlpKe7O8uXLG01HiZwNFPoSCW2dWmrOsmXLuO222xg8eDBf/vKXm5zdI9LVWVc/zzg3N9fLysrat3EY863NfP9sUef39qJmeifvYye5uYicZGbl7p57al17+iIiEaLQFxGJkBbP3jGzR4BJwH53zw5qfYDfAxlAFTDF3T8IXrsTuBU4Dsx19+eD+kjgd8AXgLXA7d7V55bkrJLMKTWRs0Vr9vR/B0w4pbYA2ODumcCG4DlmNhSYCmQF2/zazLoF2ywDZgKZwePU9xQRkU7WYui7+4vA+6eUJwMlwXIJcEOD+gp3P+buu4CdwCgzSwcudPe/Bnv3yxtsIyIiIWnvnP7F7l4DEHw9ecOUAcCeBuvFg9qAYPnUekJmNtPMysys7MCBA+0cooiInKqjD+QmmlT109QTcvdid89199x+/fp12OBERKKuvaG/L5iyIfh68iboceCSBuvFgL1BPZagLiIiIWpv6K8BpgfL04HVDepTzayHmV1K3QHbrcEU0BEzu9rq7lA1rcE2IiISktacsvkEMBboa2ZxoAi4B3jSzG4FdgM3A7j7NjN7EtgO1AKz3f148Faz+PyUzeeCh4iIhKjF0Hf3pn/Bok7Cv1Hn7ouBxQnqZUB20y1ERCQsuiJXRCRCFPoiIhGi0BcRiRCFvohIhCj0RUQiRKEvIhIhCn0RkQhR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIQo9EVEIkShLyISIQp9EZEIUeiLiESIQl9EJEIU+iIiEaLQFxGJEIW+iEiEKPRFRCJEoS8iEiEKfRGRCFHoi4hEiEJfRCRCFPoiIhGi0BcRiRCFvohIhCj0RUQiRKEvIhIhCn0RkQhR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIScUeibWZWZvWFmFWZWFtT6mNl6M6sMvl7UYP07zWynme0ws6+f6eBFRKRtOmJP/1p3z3H33OD5AmCDu2cCG4LnmNlQYCqQBUwAfm1m3Tqgv4iItFJnTO9MBkqC5RLghgb1Fe5+zN13ATuBUZ3QX0REmnGmoe/An8ys3MxmBrWL3b0GIPiaFtQHAHsabBsPak2Y2UwzKzOzsgMHDpzhEEVE5KTzznD7a9x9r5mlAevN7O3TrGsJap5oRXcvBooBcnNzE64jIiJtd0Z7+u6+N/i6H3iGuumafWaWDhB83R+sHgcuabB5DNh7Jv1FRKRt2h36ZnaBmfU+uQz8C/AmsAaYHqw2HVgdLK8BpppZDzO7FMgEtra3v4iItN2ZTO9cDDxjZiff53F3X2dmrwBPmtmtwG7gZgB332ZmTwLbgVpgtrsfP6PRi4hIm7Q79N39XeArCervAeOa2WYxsLi9PUVE5MzoilwRkQhR6IuIRIhCX0QkQhT6IiIRotAXEYkQhb6ISIQo9EVEIkShLyISIQp9EZEIUeiLiESIQl9EJEIU+iIiEaLQFxGJEIW+iEiEKPRFRCJEoS8iEiEKfRGRCFHoi4SgoKCAtLQ0srOzG9V/+ctfcvnll5OVlcX8+fMB+Oyzz5g+fTrDhg1jyJAh3H333ckYspyjzuRv5IpIK+Xn5zNnzhymTZtWX3vhhRdYvXo1r7/+Oj169GD//v0APPXUUxw7dow33niDjz/+mKFDh5KXl0dGRkaSRi/nEu3pi4RgzJgx9OnTp1Ft2bJlLFiwgB49egCQlpYGgJlx9OhRamtr+eSTT+jevTsXXnhhu3s391sGwNKlSzEzDh482Ki+e/duUlNTWbp0abv7Stek0BdJkr///e9s3ryZq666iq997Wu88sorAHzrW9/iggsuID09nYEDBzJv3rwmPzDaIj8/n3Xr1jWp79mzh/Xr1zNw4MAmrxUWFjJx4sR295SuS6EvkiS1tbV88MEHlJaW8vOf/5wpU6bg7mzdupVu3bqxd+9edu3axS9+8QvefffddvdJ9FsG1AX7vffei5k1qq9atYpBgwaRlZXV7p7SdSn0RZIkFotx4403YmaMGjWKlJQUDh48yOOPP86ECRM4//zzSUtL45prrqGsrKxDe69Zs4YBAwbwla98pVH96NGjLFmyhKKiog7tJ12HQl8kSW644QY2btwI1E31fPrpp/Tt25eBAweyceNG3J2jR49SWlrKFVdc0WF9P/74YxYvXsxdd93V5LWioiIKCwtJTU3tsH7StejsHZEQ5OXlsWnTJg4ePEgsFmPRokUUFBRQUFBAdnY23bt3p6SkBDNj9uzZzJgxg+zsbNydGTNmMHz48A4byzvvvMOuXbvq9/Lj8TgjRoxg69atbNmyhZUrVzJ//nwOHTpESkoKPXv2ZM6cOR3WX5LL3D3ZYzit3Nxcb/evtqfMVXaKZr5/tqjze3tRM72T97H1Pe/M3mfwX7WqqopJkybx5ptvNnktIyODsrIy+vbt26i+cOFCUlNTmTdvXvsbS9KYWbm7555a1/SOyDkuLy+P0aNHs2PHDmKxGA8//HCyhyRJpD39M6W9zi7TXN/zZDSXrkp7+iKSFMePH+fKK69k0qRJjerNXRgmnUuhLyKd6oEHHmDIkCGNaqe7MEw6l0JfRDpNPB7nj3/8I7fddlujenMXhknnU+iLSKe54447uPfee0lJ+TxqmrswrKO15c6m51Lvlij0RaRTPPvss6SlpTFy5Mj62ukuDOtoie451PDOptu2beu001GT2bslujhLRDrFyy+/zJo1a1i7di3/+Mc/OHz4MN/5zneavTDsS1/6Uof2HzNmDFVVVY1qzd3ZtKMls3dLtKcvIp3i7rvvJh6PU1VVxYoVK7juuut4+umn2b9/P1VVVVRVVRGLxXj11Vc7PPCb09ydTc/13g1pT19EIqPhnU1feeUVpkyZwrvvvhvKAeVk9m4o9NA3swnAA0A34CF3vyfsMYhIx2jTBXFfTbD+DOj3YL/TbtbcBXHt0dydTfv1O/0YzvbeDYU6vWNm3YAHgYnAUCDPzIaGOQYRia7m7mx6rvduKOw9/VHATnd/F8DMVgCTge0hj0NEznItz4rkAZuAg5jFgEVAAVDAT3+aDXQHSkhJaf6N2nvri887Q8ysQWfI/ulPg85gKafZ7+6kW1+EHfoDgD0NnseBq0Ieg4hEwhPN1P/zHO7csrBDP9GPxyY/zsxsJjAzePqRme3o1FF9ri91P5xbr+MOwrS5ty1MYu+OPfbUtv4d27xNvTvwe9723vqeh9/77P6e/89ExbBDPw5c0uB5DNh76kruXgwUhzWok8ysLNFd6dT73O2v3uodhd4NhX2e/itAppldambdganAmpDHICISWaHu6bt7rZnNAZ6n7pTNR9x9W5hjEBGJstDP03f3tcDasPu2UuhTSuqd9P7qrd5R6F2vy//lLBER6Ti6946ISIQo9EVCZGYZZvZmssch0aXQFxEJkdVJWvYq9Knf+3rLzH5rZtvM7E9m9oUQ+/9fM3vbzNab2RNmFtpfVzCzfzOzrWZWYWa/Ce6PFEbfn5rZ7Q2eLzazuSH0nX+yj5ndZ2Ybg+VxZhbWBZPnmVmJmb1uZivNrFcYTYN/528no3fQf1rQ929m9miIfRv9dmVm88xsYVj9G4zhLTP7NfAqja9XCpVC/3OZwIPungUcAm4Ko6mZ5Qa9rgRuBEK7eMPMhgD/Clzj7jnAceCWkNo/DEwPxpFC3TUbj4XQ90Xgn4PlXCDVzM4H/gnYHEJ/gMuBYncfDhwGvh9S36T1NrMs4P8A17n7V4DbW9jkXHQ5sNzdr3T3/07WIBT6n9vl7hXBcjmQEVLffwJWu/sn7n4E+ENIfQHGASOBV8ysIng+KIzG7l4FvGdmVwL/Arzm7u+F0LocGGlmvYFjwF+pC/9/JrzQ3+PuLwfL/0ndv4GwJKv3dcBKdz8I4O7vh9S3K/lvdy9N9iD0R1Q+d6zB8nEgrOmdcP+CQtPeJe5+Z5L6PwTkA18CHgmjobt/ZmZVwAzgL8DrwLXAl4G3whgDTe83FeZ508nqbSH2OlUtjXdweyZpHEeT1LcR7ekn30vA/zKznmaWCnwzxN4bgG+ZWRqAmfUxs4Q3aeokzwATgK9Sd5V2WF4E5gVfNwPfAyo8vItWBprZ6GA5j7p/A2FJVu8NwBQz+yLU/VsLqS/APiDNzL5oZj2ASSH27nIU+knm7q9Qd/+hvwH/BZQBH4bUezvwE+BPZvY6sB5ID6N30P9T4AXgSXc/HlZf6oI+Hfiru+8D/kF4UztQ9xvF9OB73gdYdq73Dm63shj4s5n9DfiPMPoGvT8D7gK2AM8Cb4fVuyvSFbldgJmluvtHwZkULwIz3f3VZI+rswUHcF8Fbnb3ymSP51xnZhnAs+6eneShSBJpT79rKA4OpL4KPB2RwB8K7AQ2KPBFwqM9fRGRCNGevohIhCj0RUQiRKEvIhIhCn0RkQhR6IuIRIhCX0QkQv4/99CrmyumMBYAAAAASUVORK5CYII=\n", 131 | "text/plain": [ 132 | "
" 133 | ] 134 | }, 135 | "metadata": { 136 | "needs_background": "light" 137 | }, 138 | "output_type": "display_data" 139 | } 140 | ], 141 | "source": [ 142 | "def autolabel(rects):\n", 143 | " for rect in rects:\n", 144 | " height = rect.get_height()\n", 145 | " plt.text(rect.get_x()+rect.get_width()/2.- 0.2, 1.03*height, '%s' % int(height))\n", 146 | " \n", 147 | " \n", 148 | "name_list = dropna_data['cap-color'].value_counts().index\n", 149 | "num_list = dropna_data['cap-color'].value_counts()\n", 150 | "autolabel(plt.bar(range(len(num_list)), num_list, color='rgb', tick_label=name_list))\n", 151 | "plt.show()\n" 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": 6, 157 | "metadata": {}, 158 | "outputs": [ 159 | { 160 | "name": "stderr", 161 | "output_type": "stream", 162 | "text": [ 163 | ":3: MatplotlibDeprecationWarning: Using a string of single character colors as a color sequence is deprecated. Use an explicit list instead.\n", 164 | " autolabel(plt.bar(range(len(num_list)), num_list, color='rgb', tick_label=name_list))\n" 165 | ] 166 | }, 167 | { 168 | "data": { 169 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAes0lEQVR4nO3df3RU9Z3/8eeLgChVK0pgA7HFIkoISCAR5dgv/qAUBFdETxG3KlZbWtbWttqzq3t2v9q6Vteuu7WtpdLVI1i/srj+olURxNatrBSCRjBQfrTwXQI5kFr5AqsgCe/vH3ONI0QSIMxMuK/HOXPmzns+d+Y9GXjl5jP33lFEYGZm6dAp3w2YmVnuOPTNzFLEoW9mliIOfTOzFHHom5mliEPfzCxFUh36ko6VtETSm5JqJX03qd8haZOkmuQyLqmPlrRM0ork+qKsx7oqqS+XNE9Sj3y9LjOzj6M076cvScAnImKnpC7Aq8A3gbHAzoj4533GDwW2RMRmSYOAFyOij6TOwGZgYET8SdK9wLsRcUdOX5CZWSs657uBfIrMb7ydyc0uyeVjfwtGxBtZN2uBYyV1BfYCAj4h6W3gRGDdEWnazOwwFPyWfo8ePaJv375H7PEjglWrVrF7926Ki4spLS1l8+bNvP322xQVFdGtWzdKS0vp3Pmjvx/feecdGhoaOOOMM5pvb9iwgU6dOnHsscdyxhlnkPlDwsws95YtW/aniCjet17wW/p9+/alurr6iD/Ptm3bmDhxIj/+8Y8pLi6mR48eSOIf/uEfqK+v5+GHH24eW1tby6WXXsobb7xBv3792LNnD2PHjmX+/Pl85jOf4Rvf+AZ/8Rd/wd///d8f8b7NzFoi6f+2VE/1B7nZTjrpJC644ALmzZtHr169KCoqolOnTnzlK19hyZIlzePq6uqYOHEis2bNol+/fgDU1NQA0K9fPyQxadIk/uu//isvr8PM7EBSHfoNDQ1s27YNgPfee4+XXnqJAQMGUF9f3zzm6aefZtCgQUDmr4Hx48dz9913c9555zWP6dOnDytXrqShoQGABQsWUFZWlsNXYmbWNgU/vXMk1dfXM2XKFJqamti7dy+TJk3ikksu4ZprrqGmpgZJ9O3blwcffBCAn/zkJ6xbt44777yTO++8E4D58+fTu3dvbr/9dkaOHEmXLl349Kc/zSOPPJLHV2Zm1rKC/yC3qqoqcjGnb2Z2NJG0LCKq9q2nenrHzCxtHPpmZini0DczS5FWQ/9gz0+T3HebpHWSVksak1WvTM5Ps07Sj+Sjl8zMcqote+/sBi7KPj+NpBeS+/61hfPTDAQmA+VAb+AlSWdERBMwHZgKLAaeJ3OOmxcwM7OcaHVLPzLafH4aYAIwOyJ2R8R6MuegGS6pBDgxIl5LznkzC7js8NpvhZTbi5lZgWvTnL6kIkk1wFZgQUT8Lrnr68mphB+W1D2p9QE2Zq1el9T6JMv71lt6vqmSqiVVf3DAk5mZHb42hX5ENEVEBVBKZqt9EJmpmn5ABVAP3JcMb2mTNw5Qb+n5ZkREVURUFRfvd74gMzM7RAe1905EbAN+A4yNiC3JL4O9wM+B4cmwOuDUrNVKyZxrvi5Z3rduZmY50pa9d4olnZQsHwd8Dvh9Mkf/gYnAW8nyXGCypK6STgP6A0sioh7YIencZK+da4Fn2/G1mJlZK9qy904JMFNSEZlfEnMi4leSHpVUQWaKZgPwVYCIqJU0B1gJNAI3JnvuAEwDHgGOI7PXjvfcMTPLoaP73Du53qOmwH+WZpYePveOmZk59M3M0sShb2aWIg59M7MUceibmaWIQ9/MLEUc+mZmKeLQNzNLEYe+mVmKOPTNzFLEoW9mliIOfTOzFHHom5mliEPfzCxFHPpmZini0DczSxGHvplZijj0zcxSxKFvZpYiDn0zsxRpNfQlHStpiaQ3JdVK+m5SP1nSAklrk+vuWevcJmmdpNWSxmTVKyWtSO77kZTrby43M0u3tmzp7wYuioghQAUwVtK5wK3AwojoDyxMbiNpIDAZKAfGAj+VVJQ81nRgKtA/uYxtx9diZmataDX0I2NncrNLcglgAjAzqc8ELkuWJwCzI2J3RKwH1gHDJZUAJ0bEaxERwKysdczMLAfaNKcvqUhSDbAVWBARvwN6RUQ9QHLdMxneB9iYtXpdUuuTLO9bNzOzHGlT6EdEU0RUAKVkttoHHWB4S/P0cYD6/g8gTZVULam6oaGhLS2amVkbHNTeOxGxDfgNmbn4LcmUDcn11mRYHXBq1mqlwOakXtpCvaXnmRERVRFRVVxcfDAtmpnZAbRl751iSScly8cBnwN+D8wFpiTDpgDPJstzgcmSuko6jcwHtkuSKaAdks5N9tq5NmsdMzPLgc5tGFMCzEz2wOkEzImIX0l6DZgj6Qbgv4EvAEREraQ5wEqgEbgxIpqSx5oGPAIcB7yQXMzMLEeU2ZGmcFVVVUV1dfWhrZzrwwAK/GdpZukhaVlEVO1b9xG5ZmYp4tA3M0sRh76ZWYo49M3MUsShb2aWIg59M7MUceibmaWIQ9/MLEUc+mZmKeLQNzNLEYe+mVmKOPTNzFLEoW9mliIOfTOzFHHom5mliEPfzCxFHPpmZini0DczSxGHvplZijj0zcxSxKFvZpYirYa+pFMl/VrSKkm1kr6Z1O+QtElSTXIZl7XObZLWSVotaUxWvVLSiuS+H0nSkXlZZmbWks5tGNMI3BIRr0s6AVgmaUFy379GxD9nD5Y0EJgMlAO9gZcknRERTcB0YCqwGHgeGAu80D4vxczMWtPqln5E1EfE68nyDmAV0OcAq0wAZkfE7ohYD6wDhksqAU6MiNciIoBZwGWH/QrMzKzNDmpOX1JfYCjwu6T0dUnLJT0sqXtS6wNszFqtLqn1SZb3rbf0PFMlVUuqbmhoOJgWzczsANoc+pKOB54EvhUR28lM1fQDKoB64L4Phrawehygvn8xYkZEVEVEVXFxcVtbNDOzVrQp9CV1IRP4j0XEUwARsSUimiJiL/BzYHgyvA44NWv1UmBzUi9toW5mZjnSlr13BDwErIqIf8mql2QNmwi8lSzPBSZL6irpNKA/sCQi6oEdks5NHvNa4Nl2eh1mZtYGbdl75zzgGmCFpJqk9nfAVZIqyEzRbAC+ChARtZLmACvJ7PlzY7LnDsA04BHgODJ77XjPHTOzHFJmR5rCVVVVFdXV1Ye2cq4PAyjwn6WZpYekZRFRtW/dR+SamaWIQ9/MLEUc+mZmKeLQLwAbN27kwgsvpKysjPLycu6//34AnnjiCcrLy+nUqRP7fq6xfPlyRowYQXl5OYMHD2bXrl0ALFu2jMGDB3P66adz0003Ueif2ZhZbjn0C0Dnzp257777WLVqFYsXL+aBBx5g5cqVDBo0iKeeeoqRI0d+ZHxjYyNXX301P/vZz6itreU3v/kNXbp0AWDatGnMmDGDtWvXsnbtWubNm5ePl2RmBcqhXwBKSkoYNmwYACeccAJlZWVs2rSJsrIyzjzzzP3Gz58/n7POOoshQ4YAcMopp1BUVER9fT3bt29nxIgRSOLaa6/lmWeeyelrMbPC5tAvMBs2bOCNN97gnHPO+dgxa9asQRJjxoxh2LBh3HvvvQBs2rSJ0tIPD3ouLS1l06ZNR7xnM+s42nJwluXIzp07ueKKK/jhD3/IiSee+LHjGhsbefXVV1m6dCndunVj1KhRVFZWtriOv7LAzLJ5S79A7NmzhyuuuIIvfvGLXH755QccW1payvnnn0+PHj3o1q0b48aN4/XXX6e0tJS6ug9PZFpXV0fv3r2PdOtm1oE49AtARHDDDTdQVlbGzTff3Or4MWPGsHz5ct59910aGxt55ZVXGDhwICUlJZxwwgksXryYiGDWrFlMmDAhB6/AzDoKT+8UgEWLFvHoo48yePBgKioqAPj+97/P7t27+cY3vkFDQwPjx4+noqKCF198ke7du3PzzTdz9tlnI4lx48Yxfvx4AKZPn851113He++9x8UXX8zFF1+cz5dmZgXG595pTwX+szSz9PC5d8zMzKFvZpYmntPPEX03d1NNcbunmcysZd7SNzNLEYe+mVmKOPTNzFLEoW9mliIOfTOzFGk19CWdKunXklZJqpX0zaR+sqQFktYm192z1rlN0jpJqyWNyapXSlqR3Pcj+WxgZmY51ZYt/UbglogoA84FbpQ0ELgVWBgR/YGFyW2S+yYD5cBY4KeSipLHmg5MBfonl7Ht+FrMzKwVrYZ+RNRHxOvJ8g5gFdAHmADMTIbNBC5LlicAsyNid0SsB9YBwyWVACdGxGuROffDrKx1zMwsBw5qTl9SX2Ao8DugV0TUQ+YXA9AzGdYH2Ji1Wl1S65Ms71tv6XmmSqqWVN3Q0HAwLZqZ2QG0OfQlHQ88CXwrIrYfaGgLtThAff9ixIyIqIqIquLi4ra2aGZmrWhT6EvqQibwH4uIp5LylmTKhuR6a1KvA07NWr0U2JzUS1uom5lZjrRl7x0BDwGrIuJfsu6aC0xJlqcAz2bVJ0vqKuk0Mh/YLkmmgHZIOjd5zGuz1jEzsxxoywnXzgOuAVZIqklqfwfcA8yRdAPw38AXACKiVtIcYCWZPX9ujIimZL1pwCPAccALycXMzHKk1dCPiFdpeT4eYNTHrHMXcFcL9Wpg0ME0aGZm7cdH5JqZpYhD38wsRRz6ZmYp4tA3M0sRh76ZWYo49M3MUsShb2aWIg59M7MUceibmaWIQ9/MLEUc+mZmKeLQNzNLEYe+mVmKOPTNzFLEoW9mliIOfTOzFHHom5mliEPfzCxFHPpmZini0DczS5FWQ1/Sw5K2Snorq3aHpE2SapLLuKz7bpO0TtJqSWOy6pWSViT3/UjSx33ZupmZHSFt2dJ/BBjbQv1fI6IiuTwPIGkgMBkoT9b5qaSiZPx0YCrQP7m09JhmZnYEtRr6EfGfwJ/b+HgTgNkRsTsi1gPrgOGSSoATI+K1iAhgFnDZoTZtZmaH5nDm9L8uaXky/dM9qfUBNmaNqUtqfZLlfetmZpZDhxr604F+QAVQD9yX1Fuap48D1FskaaqkaknVDQ0Nh9iimZnt65BCPyK2RERTROwFfg4MT+6qA07NGloKbE7qpS3UP+7xZ0REVURUFRcXH0qLZmbWgkMK/WSO/gMTgQ/27JkLTJbUVdJpZD6wXRIR9cAOSecme+1cCzx7GH2bmdkh6NzaAEmPAxcAPSTVAbcDF0iqIDNFswH4KkBE1EqaA6wEGoEbI6IpeahpZPYEOg54IbmYmVkOtRr6EXFVC+WHDjD+LuCuFurVwKCD6s7MzNqVj8g1M0sRh76ZWYo49M3MUsShb2aWIg59M7MUceibmaWIQ9/MLEUc+mZmKeLQNzNLEYe+mVmKOPTNzFLEoW9mliIOfTOzFHHom5mliEPfzCxFHPpmZini0DczSxGHvplZijj0zcxSxKFvZpYiDn0zsxRpNfQlPSxpq6S3smonS1ogaW1y3T3rvtskrZO0WtKYrHqlpBXJfT+SpPZ/OWZmdiBt2dJ/BBi7T+1WYGFE9AcWJreRNBCYDJQn6/xUUlGyznRgKtA/uez7mGZmdoS1GvoR8Z/An/cpTwBmJsszgcuy6rMjYndErAfWAcMllQAnRsRrERHArKx1zMwsRw51Tr9XRNQDJNc9k3ofYGPWuLqk1idZ3rfeIklTJVVLqm5oaDjEFs3MbF/t/UFuS/P0cYB6iyJiRkRURURVcXFxuzVnZpZ2hxr6W5IpG5LrrUm9Djg1a1wpsDmpl7ZQNzOzHDrU0J8LTEmWpwDPZtUnS+oq6TQyH9guSaaAdkg6N9lr59qsdczMLEc6tzZA0uPABUAPSXXA7cA9wBxJNwD/DXwBICJqJc0BVgKNwI0R0ZQ81DQyewIdB7yQXMzMLIdaDf2IuOpj7hr1MePvAu5qoV4NDDqo7szMrF35iFwzsxRx6JuZpYhD38wsRRz6ZmYp4tA3M0sRh76ZWYo49M3MUsShb2aWIg59M7MUceibmaWIQ9/MLEVaPfeOpU/fvn054YQTKCoqonPnzlRXV3PllVeyevVqALZt28ZJJ51ETU0NAMuXL+erX/0q27dvp1OnTixdupRjjz32qOnD7Gji0LcW/frXv6ZHjx7Nt//93/+9efmWW27hk5/8JACNjY1cffXVPProowwZMoS3336bLl26HHV9mB0tHPp2UCKCOXPm8PLLLwMwf/58zjrrLIYMGQLAKaeckqo+zDoaz+nbfiTx+c9/nsrKSmbMmPGR+37729/Sq1cv+vfvD8CaNWuQxJgxYxg2bBj33nvvUdeH2dHEW/q2n0WLFtG7d2+2bt3K6NGjGTBgACNHjgTg8ccf56qrPvyKhcbGRl599VWWLl1Kt27dGDVqFJWVlYwa1eLXLXTIPsyOJt7St/307t0bgJ49ezJx4kSWLFkCZIL1qaee4sorr2weW1payvnnn0+PHj3o1q0b48aN4/XXXz+q+jA7mjj07SP+53/+hx07djQvz58/n0GDMl949tJLLzFgwABKSz/8jvsxY8awfPly3n33XRobG3nllVcYOHDgUdOH2dHG0zv2EVu2bGHixIlAZov6r/7qrxg7diwAs2fP/siUCkD37t25+eabOfvss5HEuHHjGD9+/FHTh9nRRhGR7x4OqKqqKqqrqw9tZal9m2nNAX6W+m7ueonbD9BH4fxIcttMgf87N2tvkpZFRNW+9cOa3pG0QdIKSTWSqpPayZIWSFqbXHfPGn+bpHWSVksaczjPbWZmB6895vQvjIiKrN8otwILI6I/sDC5jaSBwGSgHBgL/FRSUTs8v1lONDU1MXToUC655BIA/vznPzN69Gj69+/P6NGjeeedd5rH3n333Zx++umceeaZvPjii0dlH9YxHYkPcicAM5PlmcBlWfXZEbE7ItYD64DhR+D5zY6I+++/n7Kysubb99xzD6NGjWLt2rWMGjWKe+65B4CVK1cye/ZsamtrmTdvHn/9139NU1PTUdeHdUyHG/oBzJe0TNLUpNYrIuoBkuueSb0PsDFr3bqkth9JUyVVS6puaGg4zBbNDl9dXR3PPfccX/7yl5trzz77LFOmTAFgypQpPPPMM831yZMn07VrV0477TROP/305t1Nj5Y+rOM63NA/LyKGARcDN0oaeYCxLX1q1+KnaxExIyKqIqKquLj4MFs0O3zf+ta3uPfee+nU6cP/Mlu2bKGkpASAkpIStm7dCsCmTZs49dRTm8eVlpayadOmo6oP67gOK/QjYnNyvRV4msx0zRZJJQDJ9dZkeB1watbqpcDmw3l+s1z41a9+Rc+ePamsrGzT+Jb2iFM77KlUKH1Yx3bI++lL+gTQKSJ2JMufB74HzAWmAPck188mq8wF/o+kfwF6A/0B/61pBW/RokXMnTuX559/nl27drF9+3auvvpqevXqRX19PSUlJdTX19OzZ2Yms7S0lI0bP5zJrKuraz66+Gjowzq2w9nS7wW8KulNMuH9XETMIxP2oyWtBUYnt4mIWmAOsBKYB9wYEf5UyQre3XffTV1dHRs2bGD27NlcdNFF/OIXv+DSSy9l5szMPgszZ85kwoQJAFx66aXMnj2b3bt3s379etauXcvw4Ye/z0Kh9GEd2yFv6UfEH4EhLdTfBlo8y1VE3AXcdajPaVZIbr31ViZNmsRDDz3Epz71KZ544gkAysvLmTRpEgMHDqRz58488MADFBUdub2TC6UP6xh8RG578hG5++kIR+Tm8r2BDvT+WId2RI7INbN02rhxIxdeeCFlZWWUl5dz//33A3DHHXfQp08fKioqqKio4Pnnn29eZ/ny5YwYMYLy8nIGDx7Mrl278tV+qvmEa2Z20Dp37sx9993HsGHD2LFjB5WVlYwePRqAb3/723znO9/5yHh/nWXhcOib2UErKSlpPjbghBNOoKys7IDHAPjrLAuHp3fM7LBs2LCBN954g3POOQeAn/zkJ5x11llcf/31zecB8tdZFg6Hvpkdsp07d3LFFVfwwx/+kBNPPJFp06bxhz/8gZqaGkpKSrjllluAD7/O8rHHHuPVV1/l6aefZuHChe3Sw/XXX0/Pnj2bv2QH4M0332TEiBEMHjyYv/zLv2T79u0ALFiwgMrKSgYPHkxlZSUvv/xyu/TQkTj0zeyQ7NmzhyuuuIIvfvGLXH755QD06tWLoqIiOnXqxFe+8pXmc/0cya+zvO6665g3b95Hal/+8pe55557WLFiBRMnTuQHP/gBAD169OCXv/wlK1asYObMmVxzzTXt0kNH4tA3s4MWEdxwww2UlZVx8803N9fr6+ubl59++unmre8j+XWWI0eO5OSTT/5IbfXq1YwcmTkV2OjRo3nyyScBGDp0aPNRyeXl5ezatYvdu3e3Sx8dhT/INbODtmjRIh599FEGDx5MRUUFAN///vd5/PHHqampQRJ9+/blwQcfBHL/dZaDBg1i7ty5TJgwgSeeeOIjp6P4wJNPPsnQoUPp2rXrEeujEDn0zWx/rRwl9lmSU+SuWPFhcfx4xmUPWrECfvnL5ptXJxcAamshmXIB2v0osYcffpibbrqJ733ve1x66aUcc8wxH7m/traWv/3bv2X+/Pnt+rwdgUPfzI46AwYMaA70NWvW8NxzzzXfV1dXx8SJE5k1axb9+vXLV4t54zl9MzvqfPCdAnv37uUf//Ef+drXvgbAtm3bGD9+PHfffTfnnXdePlvMG4e+mXVoV111FSNGjGD16tWUlpby0EMP8fjjj3PGGWcwYMAAevfuzZe+9CUgcwzBunXruPPOO5tPFfHBL4i08AnX2lOBnNSrw5zQyydc20/BvD8F00hhvT8diU+4ZmZ2BO3atYvhw4czZMgQysvLuf3225vv+/GPf8yZZ55JeXk5f/M3f5PHLv1BrplZu+jatSsvv/wyxx9/PHv27OGzn/0sF198Me+99x7PPvssy5cvp2vXrnmfTvKWvplZO5DE8ccfD2SOVt6zZw+SmD59Orfeemvz8QAffJ1lvjj0zczaSVNTExUVFfTs2ZPRo0dzzjnnsGbNGn77299yzjnncP7557N06dK89ujpHTOzdlJUVERNTQ3btm1j4sSJvPXWWzQ2NvLOO++wePFili5dyqRJk/jjH/+Icv1hecJb+mZm7eykk07iggsuYN68eZSWlnL55ZcjieHDh9OpUyf+9Kc/5a23nIe+pLGSVktaJ+nWXD+/mdmR0NDQwLZt2wB47733eOmllxgwYACXXXZZ8ymc16xZw/vvv0+PHj3y1mdOp3ckFQEPAKOBOmCppLkRsTKXfZiZtbf6+nqmTJlCU1MTe/fuZdKkSVxyySW8//77XH/99QwaNIhjjjmGmTNn5m1qB3I/pz8cWBcRfwSQNBuYADj0zazgHTirzwLeaL51++2ZCxwD/KK5PmpU257rSB03m+vpnT5A9jlO65KamZnlQK639Fv6Pbnf7zNJU4Gpyc2dklYf0a721wM4+E9a2v9PtkPqQ3cckT8dD62XI/NX7MH3cmQa8fvTTn0c5e9Pvt6bT7dUzHXo1wGnZt0uBTbvOygiZgAzctXUviRVt3TOirT2Ae6lkPuAwumlUPqAwumlUPr4QK6nd5YC/SWdJukYYDIwN8c9mJmlVk639COiUdLXgReBIuDhiKjNZQ9mZmmW8yNyI+J54PlcP+9BytvU0j4KpQ9wLy0plD6gcHoplD6gcHoplD6ADnA+fTMzaz8+DYOZWYo49AuYpJskrZL0WL57scIlaWe+e7COw9M7BUzS74GLI2J9vnuxwiVpZ0Qcn+8+rGPwln5CUt9kq/rnkmolzZd0XB77+RnwGWCupG/nsY9PSHpO0puS3pJ0ZZ76uFPSN7Nu3yXppnz0kjz/M5KWJf9Wpra+xtEt+f/ze0kzJS2X9B+SuuWxn2uTPt6U9Ggenv+Dn8e/Jf9vHpP0OUmLJK2VNDzXPTWLCF8yf+30BRqBiuT2HODqPPe0AeiR5x6uAH6edfuTeXx/Xk+WOwF/AE7J48/l5OT6OOCtPPeyM5//RrLenwDOS24/DHwnT72UA6s/+L/zwXuVh59HIzA4+fe6LPmZiMz5xp7J13vlLf2PWh8RNcnyMjJvXNqtAD4n6Z8k/a+I+H/5aCIiNgBvSxoKfB54IyLezkcviZskvQksJnOUef889lIoNkbEomT5F8Bn89THRcB/RMSfACLiz3nqY31ErIiIvUAtsDAyvxFWkMdsceh/1O6s5Sb8zWJExBqgksw/1Lsl/e88tvNvwHXAl8hsNeWFpAuAzwEjImIImVMrHpuvfgrIvh8Q5usDQ+XxubNl58nerNt7yWO2OPTtgCT1Bt6NiF8A/wwMy2M7TwNjgbPJHNWdL58E3omIdyUNAM7NYy+F5FOSRiTLVwGv5qmPhcAkSacASDo5T30UpNRvyVqrBgM/kLQX2ANMy1cjEfG+pF8D2yKiKV99APOAr0laTmbueHEeeykkq4Apkh4E1gLT89FERNRKugt4RVITmb/ErstHL4XIu2xahyGpE/A68IWIWJvvfuxDkvoCv4qIQXluxVrh6R3rECQNBNaR+TDMgW92iLylb2aWIt7SNzNLEYe+mVmKOPTNzFLEoW9mliIOfTOzFHHom5mlyP8HXJ7EjSjvwWgAAAAASUVORK5CYII=\n", 170 | "text/plain": [ 171 | "
" 172 | ] 173 | }, 174 | "metadata": { 175 | "needs_background": "light" 176 | }, 177 | "output_type": "display_data" 178 | } 179 | ], 180 | "source": [ 181 | "name_list = dropna_data['odor'].value_counts().index\n", 182 | "num_list = dropna_data['odor'].value_counts()\n", 183 | "autolabel(plt.bar(range(len(num_list)), num_list, color='rgb', tick_label=name_list))\n", 184 | "plt.show()\n" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": 7, 190 | "metadata": {}, 191 | "outputs": [], 192 | "source": [ 193 | "def encoding(x):\n", 194 | " temp = list(dropna_data[x].value_counts().index)\n", 195 | " dic = {}\n", 196 | " index = 0\n", 197 | " result = []\n", 198 | " for i in temp:\n", 199 | " dic[i] = index\n", 200 | " index +=1\n", 201 | " for j in dropna_data[x]:\n", 202 | " result.append(dic[j])\n", 203 | " return result\n", 204 | " " 205 | ] 206 | }, 207 | { 208 | "cell_type": "code", 209 | "execution_count": 8, 210 | "metadata": {}, 211 | "outputs": [], 212 | "source": [ 213 | "data = pd.DataFrame()\n", 214 | "for i in columns:\n", 215 | " number_list = encoding(i)\n", 216 | " data[i] = number_list" 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": 25, 222 | "metadata": {}, 223 | "outputs": [ 224 | { 225 | "data": { 226 | "text/html": [ 227 | "
\n", 228 | "\n", 241 | "\n", 242 | " \n", 243 | " \n", 244 | " \n", 245 | " \n", 246 | " \n", 247 | " \n", 248 | " \n", 249 | " \n", 250 | " \n", 251 | " \n", 252 | " \n", 253 | " \n", 254 | " \n", 255 | " \n", 256 | " \n", 257 | " \n", 258 | " \n", 259 | " \n", 260 | " \n", 261 | " \n", 262 | " \n", 263 | " \n", 264 | " \n", 265 | " \n", 266 | " \n", 267 | " \n", 268 | " \n", 269 | " \n", 270 | " \n", 271 | " \n", 272 | " \n", 273 | " \n", 274 | " \n", 275 | " \n", 276 | " \n", 277 | " \n", 278 | " \n", 279 | " \n", 280 | " \n", 281 | " \n", 282 | " \n", 283 | " \n", 284 | " \n", 285 | " \n", 286 | " \n", 287 | " \n", 288 | " \n", 289 | " \n", 290 | " \n", 291 | " \n", 292 | " \n", 293 | " \n", 294 | " \n", 295 | " \n", 296 | " \n", 297 | " \n", 298 | " \n", 299 | " \n", 300 | " \n", 301 | " \n", 302 | " \n", 303 | " \n", 304 | " \n", 305 | " \n", 306 | " \n", 307 | " \n", 308 | " \n", 309 | " \n", 310 | " \n", 311 | " \n", 312 | " \n", 313 | " \n", 314 | " \n", 315 | " \n", 316 | " \n", 317 | " \n", 318 | " \n", 319 | " \n", 320 | " \n", 321 | " \n", 322 | " \n", 323 | " \n", 324 | " \n", 325 | " \n", 326 | " \n", 327 | " \n", 328 | " \n", 329 | " \n", 330 | " \n", 331 | " \n", 332 | " \n", 333 | " \n", 334 | " \n", 335 | " \n", 336 | " \n", 337 | " \n", 338 | " \n", 339 | " \n", 340 | " \n", 341 | " \n", 342 | " \n", 343 | " \n", 344 | " \n", 345 | " \n", 346 | " \n", 347 | " \n", 348 | " \n", 349 | " \n", 350 | " \n", 351 | " \n", 352 | " \n", 353 | " \n", 354 | " \n", 355 | " \n", 356 | " \n", 357 | " \n", 358 | " \n", 359 | " \n", 360 | " \n", 361 | " \n", 362 | " \n", 363 | " \n", 364 | " \n", 365 | " \n", 366 | " \n", 367 | " \n", 368 | " \n", 369 | " \n", 370 | " \n", 371 | " \n", 372 | " \n", 373 | " \n", 374 | " \n", 375 | " \n", 376 | " \n", 377 | " \n", 378 | " \n", 379 | " \n", 380 | " \n", 381 | " \n", 382 | " \n", 383 | " \n", 384 | " \n", 385 | " \n", 386 | " \n", 387 | " \n", 388 | " \n", 389 | " \n", 390 | " \n", 391 | " \n", 392 | " \n", 393 | " \n", 394 | " \n", 395 | " \n", 396 | " \n", 397 | " \n", 398 | " \n", 399 | " \n", 400 | " \n", 401 | " \n", 402 | " \n", 403 | " \n", 404 | " \n", 405 | " \n", 406 | " \n", 407 | " \n", 408 | " \n", 409 | " \n", 410 | " \n", 411 | " \n", 412 | " \n", 413 | " \n", 414 | " \n", 415 | " \n", 416 | " \n", 417 | " \n", 418 | " \n", 419 | " \n", 420 | " \n", 421 | " \n", 422 | " \n", 423 | " \n", 424 | " \n", 425 | " \n", 426 | " \n", 427 | " \n", 428 | " \n", 429 | " \n", 430 | " \n", 431 | " \n", 432 | " \n", 433 | " \n", 434 | " \n", 435 | " \n", 436 | " \n", 437 | " \n", 438 | " \n", 439 | " \n", 440 | " \n", 441 | " \n", 442 | " \n", 443 | " \n", 444 | " \n", 445 | " \n", 446 | " \n", 447 | " \n", 448 | " \n", 449 | " \n", 450 | " \n", 451 | " \n", 452 | " \n", 453 | " \n", 454 | " \n", 455 | " \n", 456 | " \n", 457 | " \n", 458 | " \n", 459 | " \n", 460 | " \n", 461 | " \n", 462 | " \n", 463 | " \n", 464 | " \n", 465 | " \n", 466 | " \n", 467 | " \n", 468 | " \n", 469 | " \n", 470 | " \n", 471 | " \n", 472 | " \n", 473 | " \n", 474 | " \n", 475 | " \n", 476 | " \n", 477 | " \n", 478 | " \n", 479 | " \n", 480 | " \n", 481 | " \n", 482 | " \n", 483 | " \n", 484 | " \n", 485 | " \n", 486 | " \n", 487 | " \n", 488 | " \n", 489 | " \n", 490 | " \n", 491 | " \n", 492 | " \n", 493 | " \n", 494 | " \n", 495 | " \n", 496 | " \n", 497 | " \n", 498 | " \n", 499 | " \n", 500 | " \n", 501 | " \n", 502 | " \n", 503 | " \n", 504 | " \n", 505 | " \n", 506 | " \n", 507 | " \n", 508 | " \n", 509 | " \n", 510 | " \n", 511 | " \n", 512 | " \n", 513 | " \n", 514 | " \n", 515 | " \n", 516 | " \n", 517 | " \n", 518 | " \n", 519 | " \n", 520 | " \n", 521 | " \n", 522 | " \n", 523 | " \n", 524 | " \n", 525 | " \n", 526 | " \n", 527 | " \n", 528 | " \n", 529 | " \n", 530 | " \n", 531 | " \n", 532 | " \n", 533 | " \n", 534 | " \n", 535 | " \n", 536 | " \n", 537 | " \n", 538 | " \n", 539 | " \n", 540 | " \n", 541 | " \n", 542 | " \n", 543 | " \n", 544 | " \n", 545 | " \n", 546 | " \n", 547 | " \n", 548 | " \n", 549 | " \n", 550 | " \n", 551 | " \n", 552 | " \n", 553 | " \n", 554 | " \n", 555 | " \n", 556 | " \n", 557 | " \n", 558 | " \n", 559 | " \n", 560 | " \n", 561 | " \n", 562 | " \n", 563 | " \n", 564 | " \n", 565 | " \n", 566 | " \n", 567 | " \n", 568 | " \n", 569 | " \n", 570 | " \n", 571 | " \n", 572 | " \n", 573 | " \n", 574 | " \n", 575 | " \n", 576 | " \n", 577 | " \n", 578 | " \n", 579 | " \n", 580 | " \n", 581 | " \n", 582 | " \n", 583 | " \n", 584 | " \n", 585 | " \n", 586 | " \n", 587 | " \n", 588 | " \n", 589 | " \n", 590 | " \n", 591 | " \n", 592 | " \n", 593 | " \n", 594 | " \n", 595 | " \n", 596 | " \n", 597 | " \n", 598 | " \n", 599 | " \n", 600 | " \n", 601 | " \n", 602 | " \n", 603 | " \n", 604 | " \n", 605 | " \n", 606 | " \n", 607 | " \n", 608 | " \n", 609 | " \n", 610 | " \n", 611 | " \n", 612 | " \n", 613 | " \n", 614 | " \n", 615 | " \n", 616 | " \n", 617 | " \n", 618 | " \n", 619 | " \n", 620 | " \n", 621 | " \n", 622 | " \n", 623 | " \n", 624 | " \n", 625 | " \n", 626 | " \n", 627 | " \n", 628 | " \n", 629 | " \n", 630 | " \n", 631 | " \n", 632 | " \n", 633 | " \n", 634 | " \n", 635 | " \n", 636 | " \n", 637 | " \n", 638 | " \n", 639 | " \n", 640 | " \n", 641 | " \n", 642 | " \n", 643 | " \n", 644 | " \n", 645 | " \n", 646 | " \n", 647 | " \n", 648 | " \n", 649 | " \n", 650 | " \n", 651 | " \n", 652 | " \n", 653 | " \n", 654 | " \n", 655 | " \n", 656 | " \n", 657 | " \n", 658 | " \n", 659 | " \n", 660 | " \n", 661 | " \n", 662 | " \n", 663 | " \n", 664 | " \n", 665 | " \n", 666 | " \n", 667 | " \n", 668 | " \n", 669 | " \n", 670 | " \n", 671 | " \n", 672 | " \n", 673 | " \n", 674 | " \n", 675 | " \n", 676 | " \n", 677 | " \n", 678 | " \n", 679 | " \n", 680 | " \n", 681 | " \n", 682 | " \n", 683 | " \n", 684 | " \n", 685 | " \n", 686 | " \n", 687 | " \n", 688 | " \n", 689 | " \n", 690 | " \n", 691 | " \n", 692 | " \n", 693 | " \n", 694 | " \n", 695 | " \n", 696 | " \n", 697 | " \n", 698 | " \n", 699 | " \n", 700 | " \n", 701 | " \n", 702 | " \n", 703 | " \n", 704 | " \n", 705 | " \n", 706 | " \n", 707 | " \n", 708 | " \n", 709 | " \n", 710 | " \n", 711 | " \n", 712 | " \n", 713 | " \n", 714 | " \n", 715 | " \n", 716 | " \n", 717 | " \n", 718 | " \n", 719 | " \n", 720 | " \n", 721 | " \n", 722 | " \n", 723 | " \n", 724 | " \n", 725 | " \n", 726 | " \n", 727 | " \n", 728 | " \n", 729 | " \n", 730 | " \n", 731 | " \n", 732 | " \n", 733 | " \n", 734 | " \n", 735 | " \n", 736 | " \n", 737 | " \n", 738 | " \n", 739 | " \n", 740 | " \n", 741 | " \n", 742 | " \n", 743 | " \n", 744 | " \n", 745 | " \n", 746 | " \n", 747 | " \n", 748 | " \n", 749 | " \n", 750 | " \n", 751 | " \n", 752 | " \n", 753 | " \n", 754 | " \n", 755 | " \n", 756 | " \n", 757 | " \n", 758 | " \n", 759 | " \n", 760 | " \n", 761 | " \n", 762 | " \n", 763 | " \n", 764 | " \n", 765 | " \n", 766 | " \n", 767 | " \n", 768 | " \n", 769 | " \n", 770 | " \n", 771 | " \n", 772 | " \n", 773 | " \n", 774 | " \n", 775 | " \n", 776 | " \n", 777 | " \n", 778 | " \n", 779 | " \n", 780 | " \n", 781 | " \n", 782 | " \n", 783 | " \n", 784 | " \n", 785 | " \n", 786 | " \n", 787 | " \n", 788 | " \n", 789 | " \n", 790 | " \n", 791 | " \n", 792 | " \n", 793 | " \n", 794 | " \n", 795 | " \n", 796 | " \n", 797 | " \n", 798 | " \n", 799 | " \n", 800 | " \n", 801 | " \n", 802 | " \n", 803 | " \n", 804 | " \n", 805 | " \n", 806 | " \n", 807 | " \n", 808 | " \n", 809 | " \n", 810 | " \n", 811 | " \n", 812 | " \n", 813 | " \n", 814 | " \n", 815 | " \n", 816 | " \n", 817 | " \n", 818 | " \n", 819 | " \n", 820 | " \n", 821 | " \n", 822 | "
classcap-shapecap-surfacecap-colorbruisesodorgill-attachmentgill-spacinggill-sizegill-color...stalk-surface-below-ringstalk-color-above-ringstalk-color-below-ringveil-typeveil-colorring-numberring-typespore-print-colorpopulationhabitat
class1.000000-0.034451-0.1591550.011735-0.5015300.323569-0.129200-0.3483870.540024-0.334999...0.1823420.1336580.133722NaN-0.120766-0.1522610.556515-0.025285-0.4463070.078160
cap-shape-0.0344511.000000-0.0580420.029639-0.0572870.0171340.126426-0.0058660.083962-0.016882...-0.059609-0.019245-0.004047NaN0.1392360.195835-0.082190-0.1199420.0940090.216553
cap-surface-0.159155-0.0580421.000000-0.009187-0.088630-0.2019570.0053640.312855-0.1689270.136293...-0.060242-0.016559-0.012579NaN0.006500-0.004239-0.0446340.1382180.092017-0.135708
cap-color0.0117350.029639-0.0091871.0000000.1087730.220801-0.1508550.115640-0.0883960.139617...0.050341-0.093657-0.081589NaN-0.1403290.2134190.0380230.1243210.1954870.155114
bruises-0.501530-0.057287-0.0886300.1087731.0000000.067419-0.137359-0.299473-0.3695960.179959...-0.303546-0.168188-0.185653NaN-0.1256060.014367-0.6851190.0943090.0717360.010061
odor0.3235690.017134-0.2019570.2208010.0674191.000000-0.065285-0.1402060.433342-0.100251...0.036040-0.188523-0.203276NaN-0.113019-0.088455-0.083359-0.097657-0.0116200.158634
gill-attachment-0.1292000.1264260.005364-0.150855-0.137359-0.0652851.000000-0.071489-0.1089840.303533...-0.0583850.4604620.444346NaN0.8422300.049642-0.0731490.3628900.1930650.146291
gill-spacing-0.348387-0.0058660.3128550.115640-0.299473-0.140206-0.0714891.000000-0.1083330.122796...0.120523-0.275012-0.247624NaN-0.0342350.206233-0.038015-0.0681130.433383-0.115027
gill-size0.5400240.083962-0.168927-0.088396-0.3695960.433342-0.108984-0.1083331.000000-0.482295...-0.074886-0.274632-0.228869NaN-0.074866-0.1901120.071459-0.464968-0.3964800.118962
gill-color-0.334999-0.0168820.1362930.1396170.179959-0.1002510.3035330.122796-0.4822951.000000...-0.0449050.2410090.227199NaN0.2785620.126014-0.0728660.4161070.3751370.075706
stalk-shape0.1020190.116854-0.0264950.330198-0.0993640.3270200.186485-0.080895-0.2145760.335438...0.1757480.3209230.342241NaN0.1705290.3253050.2248380.3411070.2581710.302504
stalk-root-0.2483900.190482-0.1476570.0926780.0082540.4560760.0456550.2233670.0830550.042577...0.243842-0.292372-0.293069NaN0.0392230.042445-0.134897-0.2387320.4636740.354307
stalk-surface-above-ring0.323350-0.0459120.0806440.014094-0.468764-0.158253-0.0904520.217483-0.054378-0.041328...0.3555470.1319060.151545NaN-0.048532-0.0197220.4953710.0846450.024179-0.022573
stalk-surface-below-ring0.182342-0.059609-0.0602420.050341-0.3035460.036040-0.0583850.120523-0.074886-0.044905...1.0000000.0765660.126544NaN-0.0584350.0258970.4177830.0623370.066446-0.042043
stalk-color-above-ring0.133658-0.019245-0.016559-0.093657-0.168188-0.1885230.460462-0.275012-0.2746320.241009...0.0765661.0000000.666745NaN0.4270220.1341380.4102770.3532480.0621740.037326
stalk-color-below-ring0.133722-0.004047-0.012579-0.081589-0.185653-0.2032760.444346-0.247624-0.2288690.227199...0.1265440.6667451.000000NaN0.4121310.1233200.4096580.3161180.0408000.036261
veil-typeNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
veil-color-0.1207660.1392360.006500-0.140329-0.125606-0.1130190.842230-0.034235-0.0748660.278562...-0.0584350.4270220.412131NaN1.000000-0.042328-0.1172320.3365810.1741950.158578
ring-number-0.1522610.195835-0.0042390.2134190.014367-0.0884550.0496420.206233-0.1901120.126014...0.0258970.1341380.123320NaN-0.0423281.0000000.006528-0.2193890.4120980.225839
ring-type0.556515-0.082190-0.0446340.038023-0.685119-0.083359-0.073149-0.0380150.071459-0.072866...0.4177830.4102770.409658NaN-0.1172320.0065281.0000000.102536-0.070331-0.011085
spore-print-color-0.025285-0.1199420.1382180.1243210.094309-0.0976570.362890-0.068113-0.4649680.416107...0.0623370.3532480.316118NaN0.336581-0.2193890.1025361.0000000.013230-0.072205
population-0.4463070.0940090.0920170.1954870.071736-0.0116200.1930650.433383-0.3964800.375137...0.0664460.0621740.040800NaN0.1741950.412098-0.0703310.0132301.0000000.301272
habitat0.0781600.216553-0.1357080.1551140.0100610.1586340.146291-0.1150270.1189620.075706...-0.0420430.0373260.036261NaN0.1585780.225839-0.011085-0.0722050.3012721.000000
\n", 823 | "

23 rows × 23 columns

\n", 824 | "
" 825 | ], 826 | "text/plain": [ 827 | " class cap-shape cap-surface cap-color \\\n", 828 | "class 1.000000 -0.034451 -0.159155 0.011735 \n", 829 | "cap-shape -0.034451 1.000000 -0.058042 0.029639 \n", 830 | "cap-surface -0.159155 -0.058042 1.000000 -0.009187 \n", 831 | "cap-color 0.011735 0.029639 -0.009187 1.000000 \n", 832 | "bruises -0.501530 -0.057287 -0.088630 0.108773 \n", 833 | "odor 0.323569 0.017134 -0.201957 0.220801 \n", 834 | "gill-attachment -0.129200 0.126426 0.005364 -0.150855 \n", 835 | "gill-spacing -0.348387 -0.005866 0.312855 0.115640 \n", 836 | "gill-size 0.540024 0.083962 -0.168927 -0.088396 \n", 837 | "gill-color -0.334999 -0.016882 0.136293 0.139617 \n", 838 | "stalk-shape 0.102019 0.116854 -0.026495 0.330198 \n", 839 | "stalk-root -0.248390 0.190482 -0.147657 0.092678 \n", 840 | "stalk-surface-above-ring 0.323350 -0.045912 0.080644 0.014094 \n", 841 | "stalk-surface-below-ring 0.182342 -0.059609 -0.060242 0.050341 \n", 842 | "stalk-color-above-ring 0.133658 -0.019245 -0.016559 -0.093657 \n", 843 | "stalk-color-below-ring 0.133722 -0.004047 -0.012579 -0.081589 \n", 844 | "veil-type NaN NaN NaN NaN \n", 845 | "veil-color -0.120766 0.139236 0.006500 -0.140329 \n", 846 | "ring-number -0.152261 0.195835 -0.004239 0.213419 \n", 847 | "ring-type 0.556515 -0.082190 -0.044634 0.038023 \n", 848 | "spore-print-color -0.025285 -0.119942 0.138218 0.124321 \n", 849 | "population -0.446307 0.094009 0.092017 0.195487 \n", 850 | "habitat 0.078160 0.216553 -0.135708 0.155114 \n", 851 | "\n", 852 | " bruises odor gill-attachment gill-spacing \\\n", 853 | "class -0.501530 0.323569 -0.129200 -0.348387 \n", 854 | "cap-shape -0.057287 0.017134 0.126426 -0.005866 \n", 855 | "cap-surface -0.088630 -0.201957 0.005364 0.312855 \n", 856 | "cap-color 0.108773 0.220801 -0.150855 0.115640 \n", 857 | "bruises 1.000000 0.067419 -0.137359 -0.299473 \n", 858 | "odor 0.067419 1.000000 -0.065285 -0.140206 \n", 859 | "gill-attachment -0.137359 -0.065285 1.000000 -0.071489 \n", 860 | "gill-spacing -0.299473 -0.140206 -0.071489 1.000000 \n", 861 | "gill-size -0.369596 0.433342 -0.108984 -0.108333 \n", 862 | "gill-color 0.179959 -0.100251 0.303533 0.122796 \n", 863 | "stalk-shape -0.099364 0.327020 0.186485 -0.080895 \n", 864 | "stalk-root 0.008254 0.456076 0.045655 0.223367 \n", 865 | "stalk-surface-above-ring -0.468764 -0.158253 -0.090452 0.217483 \n", 866 | "stalk-surface-below-ring -0.303546 0.036040 -0.058385 0.120523 \n", 867 | "stalk-color-above-ring -0.168188 -0.188523 0.460462 -0.275012 \n", 868 | "stalk-color-below-ring -0.185653 -0.203276 0.444346 -0.247624 \n", 869 | "veil-type NaN NaN NaN NaN \n", 870 | "veil-color -0.125606 -0.113019 0.842230 -0.034235 \n", 871 | "ring-number 0.014367 -0.088455 0.049642 0.206233 \n", 872 | "ring-type -0.685119 -0.083359 -0.073149 -0.038015 \n", 873 | "spore-print-color 0.094309 -0.097657 0.362890 -0.068113 \n", 874 | "population 0.071736 -0.011620 0.193065 0.433383 \n", 875 | "habitat 0.010061 0.158634 0.146291 -0.115027 \n", 876 | "\n", 877 | " gill-size gill-color ... \\\n", 878 | "class 0.540024 -0.334999 ... \n", 879 | "cap-shape 0.083962 -0.016882 ... \n", 880 | "cap-surface -0.168927 0.136293 ... \n", 881 | "cap-color -0.088396 0.139617 ... \n", 882 | "bruises -0.369596 0.179959 ... \n", 883 | "odor 0.433342 -0.100251 ... \n", 884 | "gill-attachment -0.108984 0.303533 ... \n", 885 | "gill-spacing -0.108333 0.122796 ... \n", 886 | "gill-size 1.000000 -0.482295 ... \n", 887 | "gill-color -0.482295 1.000000 ... \n", 888 | "stalk-shape -0.214576 0.335438 ... \n", 889 | "stalk-root 0.083055 0.042577 ... \n", 890 | "stalk-surface-above-ring -0.054378 -0.041328 ... \n", 891 | "stalk-surface-below-ring -0.074886 -0.044905 ... \n", 892 | "stalk-color-above-ring -0.274632 0.241009 ... \n", 893 | "stalk-color-below-ring -0.228869 0.227199 ... \n", 894 | "veil-type NaN NaN ... \n", 895 | "veil-color -0.074866 0.278562 ... \n", 896 | "ring-number -0.190112 0.126014 ... \n", 897 | "ring-type 0.071459 -0.072866 ... \n", 898 | "spore-print-color -0.464968 0.416107 ... \n", 899 | "population -0.396480 0.375137 ... \n", 900 | "habitat 0.118962 0.075706 ... \n", 901 | "\n", 902 | " stalk-surface-below-ring stalk-color-above-ring \\\n", 903 | "class 0.182342 0.133658 \n", 904 | "cap-shape -0.059609 -0.019245 \n", 905 | "cap-surface -0.060242 -0.016559 \n", 906 | "cap-color 0.050341 -0.093657 \n", 907 | "bruises -0.303546 -0.168188 \n", 908 | "odor 0.036040 -0.188523 \n", 909 | "gill-attachment -0.058385 0.460462 \n", 910 | "gill-spacing 0.120523 -0.275012 \n", 911 | "gill-size -0.074886 -0.274632 \n", 912 | "gill-color -0.044905 0.241009 \n", 913 | "stalk-shape 0.175748 0.320923 \n", 914 | "stalk-root 0.243842 -0.292372 \n", 915 | "stalk-surface-above-ring 0.355547 0.131906 \n", 916 | "stalk-surface-below-ring 1.000000 0.076566 \n", 917 | "stalk-color-above-ring 0.076566 1.000000 \n", 918 | "stalk-color-below-ring 0.126544 0.666745 \n", 919 | "veil-type NaN NaN \n", 920 | "veil-color -0.058435 0.427022 \n", 921 | "ring-number 0.025897 0.134138 \n", 922 | "ring-type 0.417783 0.410277 \n", 923 | "spore-print-color 0.062337 0.353248 \n", 924 | "population 0.066446 0.062174 \n", 925 | "habitat -0.042043 0.037326 \n", 926 | "\n", 927 | " stalk-color-below-ring veil-type veil-color \\\n", 928 | "class 0.133722 NaN -0.120766 \n", 929 | "cap-shape -0.004047 NaN 0.139236 \n", 930 | "cap-surface -0.012579 NaN 0.006500 \n", 931 | "cap-color -0.081589 NaN -0.140329 \n", 932 | "bruises -0.185653 NaN -0.125606 \n", 933 | "odor -0.203276 NaN -0.113019 \n", 934 | "gill-attachment 0.444346 NaN 0.842230 \n", 935 | "gill-spacing -0.247624 NaN -0.034235 \n", 936 | "gill-size -0.228869 NaN -0.074866 \n", 937 | "gill-color 0.227199 NaN 0.278562 \n", 938 | "stalk-shape 0.342241 NaN 0.170529 \n", 939 | "stalk-root -0.293069 NaN 0.039223 \n", 940 | "stalk-surface-above-ring 0.151545 NaN -0.048532 \n", 941 | "stalk-surface-below-ring 0.126544 NaN -0.058435 \n", 942 | "stalk-color-above-ring 0.666745 NaN 0.427022 \n", 943 | "stalk-color-below-ring 1.000000 NaN 0.412131 \n", 944 | "veil-type NaN NaN NaN \n", 945 | "veil-color 0.412131 NaN 1.000000 \n", 946 | "ring-number 0.123320 NaN -0.042328 \n", 947 | "ring-type 0.409658 NaN -0.117232 \n", 948 | "spore-print-color 0.316118 NaN 0.336581 \n", 949 | "population 0.040800 NaN 0.174195 \n", 950 | "habitat 0.036261 NaN 0.158578 \n", 951 | "\n", 952 | " ring-number ring-type spore-print-color \\\n", 953 | "class -0.152261 0.556515 -0.025285 \n", 954 | "cap-shape 0.195835 -0.082190 -0.119942 \n", 955 | "cap-surface -0.004239 -0.044634 0.138218 \n", 956 | "cap-color 0.213419 0.038023 0.124321 \n", 957 | "bruises 0.014367 -0.685119 0.094309 \n", 958 | "odor -0.088455 -0.083359 -0.097657 \n", 959 | "gill-attachment 0.049642 -0.073149 0.362890 \n", 960 | "gill-spacing 0.206233 -0.038015 -0.068113 \n", 961 | "gill-size -0.190112 0.071459 -0.464968 \n", 962 | "gill-color 0.126014 -0.072866 0.416107 \n", 963 | "stalk-shape 0.325305 0.224838 0.341107 \n", 964 | "stalk-root 0.042445 -0.134897 -0.238732 \n", 965 | "stalk-surface-above-ring -0.019722 0.495371 0.084645 \n", 966 | "stalk-surface-below-ring 0.025897 0.417783 0.062337 \n", 967 | "stalk-color-above-ring 0.134138 0.410277 0.353248 \n", 968 | "stalk-color-below-ring 0.123320 0.409658 0.316118 \n", 969 | "veil-type NaN NaN NaN \n", 970 | "veil-color -0.042328 -0.117232 0.336581 \n", 971 | "ring-number 1.000000 0.006528 -0.219389 \n", 972 | "ring-type 0.006528 1.000000 0.102536 \n", 973 | "spore-print-color -0.219389 0.102536 1.000000 \n", 974 | "population 0.412098 -0.070331 0.013230 \n", 975 | "habitat 0.225839 -0.011085 -0.072205 \n", 976 | "\n", 977 | " population habitat \n", 978 | "class -0.446307 0.078160 \n", 979 | "cap-shape 0.094009 0.216553 \n", 980 | "cap-surface 0.092017 -0.135708 \n", 981 | "cap-color 0.195487 0.155114 \n", 982 | "bruises 0.071736 0.010061 \n", 983 | "odor -0.011620 0.158634 \n", 984 | "gill-attachment 0.193065 0.146291 \n", 985 | "gill-spacing 0.433383 -0.115027 \n", 986 | "gill-size -0.396480 0.118962 \n", 987 | "gill-color 0.375137 0.075706 \n", 988 | "stalk-shape 0.258171 0.302504 \n", 989 | "stalk-root 0.463674 0.354307 \n", 990 | "stalk-surface-above-ring 0.024179 -0.022573 \n", 991 | "stalk-surface-below-ring 0.066446 -0.042043 \n", 992 | "stalk-color-above-ring 0.062174 0.037326 \n", 993 | "stalk-color-below-ring 0.040800 0.036261 \n", 994 | "veil-type NaN NaN \n", 995 | "veil-color 0.174195 0.158578 \n", 996 | "ring-number 0.412098 0.225839 \n", 997 | "ring-type -0.070331 -0.011085 \n", 998 | "spore-print-color 0.013230 -0.072205 \n", 999 | "population 1.000000 0.301272 \n", 1000 | "habitat 0.301272 1.000000 \n", 1001 | "\n", 1002 | "[23 rows x 23 columns]" 1003 | ] 1004 | }, 1005 | "execution_count": 25, 1006 | "metadata": {}, 1007 | "output_type": "execute_result" 1008 | } 1009 | ], 1010 | "source": [ 1011 | "data.corr()" 1012 | ] 1013 | }, 1014 | { 1015 | "cell_type": "code", 1016 | "execution_count": 32, 1017 | "metadata": {}, 1018 | "outputs": [], 1019 | "source": [ 1020 | "def visualize_data(x):\n", 1021 | " data = x.values\n", 1022 | " \n", 1023 | " fig = plt.figure(figsize=(20, 20))\n", 1024 | " ax = fig.add_subplot(111)\n", 1025 | " \n", 1026 | " heatmap = ax.pcolor(data,cmap = plt.cm.RdYlGn)\n", 1027 | " fig.colorbar(heatmap)\n", 1028 | " ax.set_xticks(np.arange(data.shape[0] + 0.5),minor = False)\n", 1029 | " ax.set_yticks(np.arange(data.shape[1] + 0.5),minor = False)\n", 1030 | " ax.invert_yaxis()\n", 1031 | " ax.xaxis.tick_top()\n", 1032 | " \n", 1033 | " column_labels = x.columns\n", 1034 | " row_labels = x.index\n", 1035 | " \n", 1036 | " ax.set_xticklabels(column_labels)\n", 1037 | " ax.set_yticklabels(row_labels)\n", 1038 | " \n", 1039 | " plt.xticks(rotation = 90)\n", 1040 | " heatmap.set_clim(-1,1)\n", 1041 | " plt.tight_layout()\n", 1042 | " plt.show()\n", 1043 | " \n" 1044 | ] 1045 | }, 1046 | { 1047 | "cell_type": "code", 1048 | "execution_count": 33, 1049 | "metadata": {}, 1050 | "outputs": [ 1051 | { 1052 | "data": { 1053 | "image/png": "iVBORw0KGgoAAAANSUhEUgAABT0AAAWYCAYAAAB03GJRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd5xkZZn28d81ZBgyrIkoq/gqOSssoi8GVkUJIq4YUFTWAIqy6ioCiu+ad5VVEZRgQILgypoIiiAZhiErBhBB2SXnDNf7xznF1Mx0V3czM/Wcp+b6fj71mT6neuSatrr69H2e575lm4iIiIiIiIiIiIhRMa10gIiIiIiIiIiIiIj5KUXPiIiIiIiIiIiIGCkpekZERERERERERMRISdEzIiIiIiIiIiIiRkqKnhERERERERERETFSUvSMiIiIiIiIiIiIkZKiZ0RERERERERERIyUFD0jIiIiIiIiIiJipCxaOkBERERELDwkrTTG6XttPzr0MBERERExsmS7dIaIiIiIWEhI+jOwOnAnIGAF4GbgFuCdtmeUSze2FGojIiIi6pPt7RERERExTL8A/tH2KrZXBnYATgDeA3y9aLLxXQrcCvwe+EP78fWSLpW0adFkERERETGmFD0jIiIiYpg2s31q78D2acC2ti8AligXa6AaC7URERERC7UUPSMiIiJimO6Q9BFJa7aPfwHulLQI8ETpcOOosVAbERERsVBL0TMiIiIihumfgNWA/wJ+DKzRnlsE2K1grkFqLNRGRERELNQyyCgiIiIiYgBJqwAHAtvQDF86BzgYuBtYw/YfC8aLiIiIiDGk6BkRERERQyPpucCHgbWARXvnbb+0VKaIiIiIGD0pekZERETE0Ei6HDgMmAE83jtve0axUBNIoTYiIiKiPil6RkRERMTQSJphe9PSOaaixkJtRERExMIuRc+IiIiIGBpJBwG3AD8CHu6dt31HqUwTqbFQGxEREbGwS9EzIiIiIoZG0vVjnLbtZw89zCTVWKiNiIiIWNil6BkRERERMUCNhdqIiIiIhV2KnhERERGxwEl6qe1fSdp5rOdtnzzsTBERERExuhad+FMiIiIiIubZi4FfAa8Z4zkDnSt61lyolXQlzde1393AJcAhtm8ffqqIiIiI4UnRMyIiIiIWONsHSpoG/Nz2CaXzTFJ1hdo+P6eZNH9se7x7++c9wNGM/W8qKoXaiIiImJ+yvT0iIiIihkbS2ba3LZ1jstpC7a4VFWoBkHSu7a3HOifpStvrl8o2HkmfZ/xC7Ta2O1eojYiIiO7KSs+IiIiIGKbTJX0YOB64v3eyq5PQbT8h6X1AVUVPYLqkLW1fCCBpC2B6+9xj5WINtPUchdor+wq1exRLFREREVVK0TMiIiIihunt7Z/v7TtnoMuT0Ksq1Lb2Ao6UNB0QzWrJvSQtA/xb0WTjq7FQGxERER2V7e0REREREQNIun6M07bd5UItAJKWp7nmv6t0lolI2hw4kqbQ+WShFrgaeFVtLQYiIiKirBQ9IyIiIqIISYfbflfpHKNI0hLALsBa9O3usv2pUpkmq6ZCbURERHRXtrdHRERERCmblQ4wVRUVan9MM/l8BvBw4SyTMmehVhJQR6E2IiIiuidFz4iIiIgo5ZbSAZ6CWgq1q9l+ZekQU1RdoTYiIiK6K0XPiIiIiBg6SctUWJSDegq150la3/aVpYNMQY2F2oiIiOioaaUDRERERMTCQ9KLJF0D/LY93lDS1wvHGkjSkwOLKirKbQPMkHStpCskXSnpitKhJnCepPVLh4iIiIjRkEFGERERETE0ki4EdgVOsb1xe+4q2+uVTTY+SWcDzwIuBs4GftP1FZSS1hzrvO0bhp1lstpi+N8D19Nsbxdg2xsUDRYRERFVyvb2iIiIiBgq2zf2htS0Hi+VZTJsbytpcWBzYDvgp5Km216pbLK5SVrO9j3AvaWzPAU7lA4QERERoyNFz4iIiIgYphslvQhwW0jch3are1dJ2gb4h/axAvAT4DdFQ43vWODVNMOATLNassfAs8f6SyVVXqiNiIiIjsr29oiIiIgYGkmrAF8BtqcpyJ0G7Gv79qLBBpD0OHAJ8G/Az2w/UjjSQGqW0a5u+y+ls0yGpJ/YfrWk6xmjUGu7c4XaiIiI6L4UPSMiIiJiaCStavvW0jmmQtIKwNbAtjRb3J8Azrd9QNFgA0iaYXvT0jkmq7ZCbURERHRfprdHRERExDCdJ+k0Se9oi4mdZ/su4DqaATs3A+vQFEC77AJJm5cOMVluVmL8qHSOiIiIGB1Z6RkRERERQyVpC2B34HXANcBxtr9XNtX4JP0JuBY4h6aX54UVbHG/BngucANwPxVMQpf0NeBo2xeXzhIRERH1S9EzIiIiIopo+3t+GXiT7UVK5xmPpGm2nyidYyokrTnWeds3DDvLZNVYqI2IiIjuyvT2iIiIiBgaScsBO9Gs9FyHZkvzFkVDTeyZkg6l6etpmhWf+9q+qWys8fUXNyW9y/bhJfNM0g6lA0RERMToyErPiIiIiBiadkL3fwEn2D6/dJ7JkHQ6cCzw3fbUHjSrU19WLtXkSbrU9ialc0xFRYXaiIiI6KgUPSMiIiJiaCTJtiUtS7N1+b7SmSYi6TLbG010rqskzbS9cekcU1FjoTYiIiK6JdPbIyIiImKYXiBpJnAVcI2kGZLWKx1qArdJ2kPSIu1jD+D20qGm4DWlAzwFKh0gIiIi6paiZ0REREQM0+HAfrbXtL0G8KH2XJe9HdgN+B/gZmDX9lxnSXqapG9L+rntmyQ9X9I7SueaghoLtREREdEh2d4eEREREUMj6XLbG050LuaNpJ8DRwEft72hpEWBmbbXLxxtXJL+BFwA/AY42/Y1hSNFRERExTK9PSIiIiKG6TpJBzD7UKDrC+YZVzuxfdwVArb3GWKcqVrF9gmSPgZg+zFJj5cONYHnA1sC/wB8UdLzgMtt71Q2VsRok7QIcIztPUpniYiYn1L0jIiIiIhhejtwMHAyTd/Gs4E9iyYa3yWlA8yD+yWtTFu0lbQVcHfZSBN6HHi0/fMJ4H+BW4omilgI2H5c0qqSFrf9SOk8ERHzS7a3R0RERMTQSVoeeML2vaWzTJakZWzfXzrHZEjaFPgqsB7N0KhVgV1tX1E02ACSHgCuBL4MnGG7pmFREVWT9E1gE+AU4Mn3OdtfLhYqImIepegZEREREUMjaXPgSGDZ9tTdwNttzyiXajBJLwS+DUy3vYakDYF3235P4WgDtX0816VZUXut7UcLRxpI0muBbYAtgEeA82h6e/6yaLCIhYCkA8c6b/vgYWeJiJhfUvSMiIiIiKGRdAXwXtu/aY+3Ab5ue4OyycYn6UKaie2n2N64PXeV7fXKJhufpMuB44Hjbf+pdJ6paHt57gB8APg720sVjhSx0JC0LGDb95XOEhExr6aVDhARERERC5V7ewVPANvnAJ3f4m77xjlOdX0o0I7AY8AJki6W9GFJa5QONYikk9oJ7l8BpgNvAVYsmypi4SBpPUkzadphXC1phqQXlM4VETEvMsgoIiLmiaStbZ870bmIWLhJ2qT98KK2d9wPaIbsvAH4dalck3SjpBcBlrQ4sA/w28KZBrJ9A/B54POSngMcAHwOWKRosME+C1xqu+sF5YhRdDiwn+0zASRtBxwBvKhkqIiIeZHt7RERMU8kXWp7k4nORcTCTdKZA5627ZcOLcwUSVqFZvXh9jQ7pU4F9u36oB1JawG70RSWH6fZ6v6lkpkGkbQY8M/Atu2ps4DDut6LNGIUSLrc9oYTnYuIqEmKnhER8ZS0gz1eRNNz7d/7nloO2CkXyRER5bR9SBcDTqQpdl5XONKEJH2LJvMx7ak3A4/b3qtcqoiFg6QfAZcC321P7QFsZvt15VJFRMybbG+PiIinanGanmuLMmsKM8A9NAM/IiLGJOlVwAuAJXvnbH+qXKLBJD2bZqXnVjRb8s8HPtjxQuJbbf+udIgp2nyOG2a/agcyRcSC93bgYOBkQMDZwJ5FE0VEzKMUPSMi4imxfRZwlqSj295xERETknQYsDTwEuBbNDdJLioaamLHAl8DdmqPd6fpSbplsUQTu1nSl5l9q/inbN9dMNNEHpe0Tm/afFtsTn/PiCGwfSdNv+KIiJGR7e0RETFPJD0X+DCwFn0307rcny8iypF0he0N+v6cDpxs++Wls41H0oW2t5zj3AW2tyqVaSKSTqKZwty/VXxD2zuXSzWYpP8LHAVcR7PSbE1gz95glYiY/yT9N80K9jHZ3nGIcSIi5qus9IyIiHl1InAYzYqtrMiJiIk82P75gKRnArcDaxfMMy5JK7Ufninpo8BxzJo4/9NiwSZnHdu79B0fLOmyYmkmwfYv20nz69IUPX9n++HCsSJG3RdLB4iIWFBS9IyIiHn1mO1vlA4REdX4iaQVgC/QDM0wcETZSOOaQZNP7fG7+54z8OmhJ5q8ByVtY/scAElbM6vg3CmSxlt9uo4kbJ881EARC5G2XREAkhYHntseXmv70TKpIiLmj2xvj4iIeSLpIOAW4EfAkytybN9RKlNE1EHSEsCSHe8zWSVJG9FsbV+epmh7B81woyuKBhuDpKMGPG3bbx9amIiFlKTtaN4z/kzznrE6zXvG2QVjRUTMkxQ9IyJinki6fozTtv3soYeJiKpIOtz2u0rnmIraMktaDsD2PaWzRER3SZoB/JPta9vj5wI/sL1p2WQREU9dtrdHRMQ8sd3JXnwRUYXNSgd4CqrILGll4EBgG8CSzqGZ3n572WTjk/Q04P8Bz7S9g6TnAy+0/e3C0SIWBov1Cp4Atn8vabGSgSIi5tW00gEiIqJukpaW9AlJh7fHz5H06tK5IqIKt5QO8BTUkvk44FZgF2DX9uPjiyaa2NHAqcAz2+PfAx8oliZi4XKJpG9L2q59HEHT1zgiolrZ3h4REfNE0vE0F8Vvsb2epKWA821vVDhaRHSYpGVs3186x1TUlFnSjDm3pUq6xHZnV6pKutj25pJm2t64PXdZfp5ELHhtj+X30qwOF3A28HXbDw/8ixERHZaVnhERMa/Wsf154FEA2w8ya9JxRMRsJL1I0jXAb9vjDSV9vXCsgWrMDJwpaXdJ09rHbsBPS4eawP3ttnwDSNoKyJCriOFYFPiK7Z1t7wR8FVikcKaIiHmSomdERMyrR9rVnb1fUtehb4p7RMQc/h14BXA7gO3LgW2LJppYNZkl3SvpHuDdwLHAI+3jOOCDJbNNwn7AKcA6ks4FvgO8v2ykKKkt2L+odI6FxC+BpfqOlwLOKJQlImK+yCCjiIiYVwcCvwBWl/R9YGvgbUUTRUSn2b5Rmm1B+OOlskxWLZltL1s6w1Nl+1JJLwbWpdkxcK3tRwvHioJsPyHpS8ALS2dZCCxp+77ege37JC1dMlBExLxK0TMiIuaJ7dMlXQpsRfNL6r62byscKyK668Z25ZYlLQ7sQ7ttvMNqzIykFYHnAEv2ztk+u1yiwSQtCbyHduI88BtJh9l+qGyyKOw0SbsAJzsDKRak+yVtYvtSAEmbAg8WzhQRMU8yyCgiIuaZpA2Atei7mWb75GKBIqKzJK0CfAXYnuZGyWk0N0tuLxpsgEoz7wXsC6wGXEZzY+p82y8tGmwASScA9wLfa0+9EVjR9uvLpYrSJN0LLEOzurrXN9y2lysabMRI2pymDcbf2lPPAN5gOxPcI6JaKXpGFFTTFNiI8Ug6EtgAuBp4oj1t228vlyoiYuEm6Upgc+AC2xtJeh5wsO03FI42LkmX295wonMRsWBIWoxZ7SV+l/YSEVG7DDKKKKDSKbAR49nK9ma232p7z/aRgmdUS9IypTOMMknHSFqh73jF9uZJZ9WYGXioty1c0hK2f0dTzOiyme3EdgAkbQmcWzBPdIAae0g6oD1eXdIWpXONItuP2r4KeH8KnhExCtLTM6KM3hTYU6CZAiupk1NgIybhfEnPt31N6SCTJWm/MU7fDcywfdmw80Q3tD0bvwVMB9aQtCHwbtvvKZts5Gxg+67ege07JW1cMtAk1Jj5prZQ+1/A6ZLuZNa21U5pV6UaWAx4i6S/tE+tAVTzsyUWmK/T7CR5KfBp4D7gazQrmWPB2Kx0gIiI+SFFz4hCapkCGzEJx9AUPv8HeJhZvbY2KBtroM3ax3+3x68CLgb2lnSi7c8XSxYl5YbUcEyTtKLtOwEkrUT3r0mry2x7p/bDgySdCSwP/KJgpEFeXTpAdNqWtjeRNBOevOmweOlQI+6W0gEiIuaHTl+sRYywKqfARozjSODNwJXM6unZdSsDm9i+D0DSgcAPgW2BGUCKngup3JAaii8B50n6YXv8euAzBfNMRo2Z+61r+/DSIcZj+4Y5z0l6V5czx1A9KmkRmtXASFqVeq43qiHp9bZPBLD9yjnPRUTUKD09I8rYG3gv8CzgJmCj9jiiRn+xfYrt623f0HuUDjWBNYBH+o4fBda0/SDNatVYOM12Q0rSh8kNqfnO9neAXYH/pVlNtLPt75ZNNViNmeewd+kAT0GNmWPB+CrwI+Bpkj4DnAP8v7KRRtLHJnkuIqIaWekZUYDt24A3lc4RMZ/8TtKxNFvFnywY2j65XKQJHQtcIOnH7fFrgB+0A2zSP27htTfwFZobUn8FTiU3pBYI21dLuhVYEkDSGrb/MsFfK6rGzH008ad0To2ZYwGw/X1JM4D/2556ne3ckJpPJO0A/CPwLElf7XtqOeCxMqkmr10F/DT6ahsVvTdHxAIm26UzRMw3kpaxfX/pHBORdAywb28ogqQVgS9l4nXUSNJRY5x211/PkjYDtqb5xfoc25cUjhSxUJC0I8128WfSrJpcE/it7RcUDTZAbZklTQOusL1ee7ya7ZsKxxqoxswxPJI2Abah2eJ+ru1LC0caGe3Qvo2ATwGf7HvqXuDMXi/jLpL0fuBAmlX4vZYHXe8rHxFDlKJnjIT+ibu2Oz9xV9JM2xtPdC4iFpysDIg5SXo2zUrPrWh+sT4f+KDt64oGGzGSLqeZwnyG7Y0lvQR4o+13FY42rkozfx/4WE3vazVmjgVP0idp+uieRHOj8nXAibYPKRpsxEhazPajpXNMhaQ/0gy6ur10lojopmxvj1FR28Td6qbARoynHSjwTmAtZi8gdnal5xwrAx6nnTgPZGXAwu1Y4GtAb+r17sAPgC2LJRpNj9q+XdI0SdNsnynpc6VDTaDGzM8ArpZ0EfDkLhjbO5aLNKEaM8eC90ZgY9sPAUj6LHApkKLn/LWFpINoVrIvSnttZPvZRVMNdiNwd+kQEdFdKbLEyKhs4m7tU2Aj+v0Y+A1wBt3+vuu3L80046wMiH6aYzjN9yS9r1ia0XWXpOnA2cD3Jd1C9/vG1Zj54NIBnoIaM8eC92eaXroPtcdLAH8qlmZ0fRv4IDCDeq7nrgN+LemnzN5X/svlIkVEl6ToGaNitom7wD50eOKu7e+0DdlfQnMXdWfbGZ4StVra9kdKh5iirAyIsZwp6aPAcTQrf98A/LRdjY/tO0qGG4ukJWw/PNG5LujL9Vqa4sUHaYb6LU/TS65zaszcY/ssSU8DNm9PXWT7lpKZJlJj5lhwJB1K8178MM0K4NPb45fRTHCP+etu2z8vHWKK/tI+Fm8fERGzSU/PGAmSVqHpw7Y9MI1m4u6+XV/FJenvaKfAQvoJRp0kHQKcZ/tnpbNMlqRvA+sCWRkQT5J0/YCnO7nFT9KltjeZ6FwX9HJJ+q7tN5fOMxk1Zu6RtBvwBeDXNDdY/wHY3/YPB/29kmrMHAuOpLcOet72McPKsjBo2wYsApzM7NdGnR8aJWlZmp/T95XOEhHdkpWeMRJs30az8qIK402BBaqYAhsBIOlemhUXAv5V0sPAo8zqAbVcyXwTyMqAmIvttUtnmCxJTweeBSwlaWOa7zuA5YCliwUbbPG2iPEiSTvP+aTtkwtkmkiNmXs+DmzeWynZ9l8+A+hyAbHGzNVoB/idanv70lkmI0XNoev1r96s75xphrh1kqT1gO8CK7XHtwFvsX110WAR0RkpesZIqHDi7qdpss42BbZwpnHZfkLS5ZLWyGrU6LG9bOkMT5Xt9I2LMbW/QD2f2Vfhf6dconG9AngbsBrQv0L5XuBfSwSahL1pblCuALxmjudMs7qoa2rM3DNtjq3ht9PshumyGjNXw/bjkh6QtLztalq8SHo1zbXznAN2OntzVdJzgW8AT7O9nqQNgB27PHHe9ktKZ3gKDgf2s30mgKTtgCOAF5UMFRHdkaJnjIraJu5mCmyMDEk7Ab/q/QIlaQVgO9v/VTbZ3CT9h+0PSPpvmoLFbPJ6nj/UTJVbzfaNpbNMhaQDge1oip4/A3ag6RvXuaJnuwLqGEm72D6pdJ7JsH2OpPOAm2xXMbyvxsx9fiHpVJrrIWh61Ha9DUmNmWvzEHBl2x+z/3pun3KRJvQfwM7Ala6nN9sRwP7ANwFsXyHpWDo4cV7SHra/J2m/sZ7veOufZXoFTwDbv5a0TMlAEdEtKXrGqKht4m6mwMYoOdD2j3oHtu9qi0edK3rSbIEC+GLRFCPOtiX9F7Bp6SxTtCuwITDT9p7tQJVvFc40kV9K+jKwbXt8FvCprq7iancOvBqopoBYY2YA2/tL2gXYmmZl3OH979VdVFvmGlfz0fSy/mnpEFN0I3BVRQVPaIY8XtTcA3xSV6/1e0XCGnfwXCfpAGZd3+0BDOrPHRELmQwyipHQNt6+i9kn7i5Bs/qzMxN3e1Ng2zuQD9Fc0PemwH6/gsFLmagac5F0he0N5jh3pe31S2WaSPs9+KDtJ9rjRYAlbD9QNtnokPQ14GjbF5fOMlmSLrK9haQZwEtotopfZbuT/ZYBJJ0EXAX0et+9GdjQ9lz9J7tC0sHAFcDJtRQxaswcC56ks2hX89neuD13Vdd7oEtaHHhue3it7UdL5pmIpM1ptrefRSXDByX9HHgfcGI7DG1X4B22dygcbUztddA+tv+9dJapkLQizcKMbWh+rzobOMj2nUWDRURnpOgZI6GWibuZAhujSNKRNDcdvkZz0+H9wIq231Yy1yCSLgC27035bFden2Y7PaDmE0nX0PxSfQPNFspeD7YNBv7FgiR9naYf5u7Ah4D7gMts71k02ACSLrO90UTnuqQdgrYMzaqn3g3ArvfnqyZz35C5uZ4imecrSRfb3lzSzL6iZ9e//7ajuUnyZ5qv7+rAW22fXTDWQJJOo3k/vhJ4one+y/2523kDh9P0lryTZvXhm2zfUDTYAJLOrLSvZ0TEuLK9PUZCRRN3MwU2RtH7gQOA42l+gToNeG/RRBNbslfwBLB9n6SuTryuVSdXswxi+z3th4dJ+gWwnO0rSmaahAclbWP7HABJWwMPFs40UI1D0GrKXFPWnhozt26TtA5twbZdzXdz2UgT+hLwctvXwpNb9H9At9uRrGT75aVDTEU7THX7dmfJNNv3ls40CedJ+k+a67n+fq+Xlos0tvRoj4jJStEzRkYlE3czBTZGju37gY+WzjFF90vapHchL2lTOl4oqo3tGyRtSLMqHOA3ti8vmWkyJD2LWROCkbRtl1dAAf9MM9BoeZqbDncAby0baTBJ2451vstf5xozA8zxPXh2BUX82jK/l2Y13/Mk/ZV2NV/ZSBNarFfwBLD9e0mLlQw0CWdIernt00oHmSxJKwMH0my7tqRzaPotd7mVVW+3y6f6zhl4aYEsE0mP9oiYlGxvj5Ew3sRd27uWzDUWSdOAj9U2BVbSF4ANmH2i6hW2P1IuVXRBu+r3X4AXMPtNhy5eJANP9gc7Dvhbe+oZwBtszyiXajBJ2wDPsX1U+zWfbruzzfol7Qu8k1k3c3aiGUpyaLlUg0n6HM172zXA4+1p17BiRNJyALbvKZ1lIu3KnJ4lgS2AGR1/z6gxc43fg9Vlhif7RFexmq9tSWNmFY3eBCza8TYevfYSDwOP0vG2BwCSTqfpL/m99tSbgO1sb18u1eiRtK/tr0x0LiIWXil6xkiQdCWzJu5u2Ju4a3vO1ZSdIOl82y8snWOq5pioenaXJ6rG8LS9to4HPkyzmvmtwK1dL4i3K1vWpXk9/67LgxzaGzubAevafq6kZ9IMR9i6cLRxSboCeGG7ErhXFDi/4z09rwU2sP3whJ/cEe0KzwOpZHr7WCStDnze9htLZ5msGjJX+j1YVeY5V/MBnV/NJ2kJmhWq/YNfvl7T+14NJM2wvekc5y6xvVmpTBOp9PV8qe1N5jj3ZI/diIhsb49R8aDtJyQ91q52uQXoxPCicZzWFhCrmgJr+yTgpNI5JkvS64Ff2L5X0ieATYBDutibqHIr2/52e2f9LOCsdqJt163LrJYYG0vqYkuMnp2AjYFLAWz/TVLXe+CJWaslaT9WoSyTdR2wGH3TgStwJM309t3a4zcDRwGdnd4+hpuATk+7HkMNmWv8Hqwt83E0RcNd2uM30dwE7Oxqvra4+eX2UYVK20ucKWl34IT2eFfgpwXzTEY1r2dJbwT+CVhb0il9Ty1L04KrsyRtbfvcic5FxPyRomeMikskrQAcAcygmfB4UdlIA+1HOwVWUqbALjgH2D6x3Rb8Cpq+P98Atiwba+T0VkjeLOlVNFvGVyuYZ0LjtcQAulr0fMS2JfWGZSxTOtAkHAVcKKm3Ivx1wLcL5hmXpENp3uceAC6T9Ev6Cp+29ymVbRLWsb1L3/HBki4rlmYS+r7e0PSG3gjodL/XGjNT0fdgn9oyr2T7033Hh0h6XbE0A7S7osa90d7V1bSt/fs+frK9BN3sNdnzbprr/V4bgUVo+onvR3evn6t5PQPn0QwNW4VmOFfPvUCX+wADHEqzEGOicxExH6ToGSOhtom7NU0prSnrGHqrRV4FfMP2jyUdVDDPqDqk3WL7IZqLtuWAD5SNNKFdmdUSY89eS4zCmQY5QdI3gRUkvRN4O93Oi+0vS/o1s7ZQ7ml7ZtlU47qk/XMGcMqgT+yg6qa3M+vrDfAY8IMKVrhUl7my70Ggysw1reZ7dekAT9Wc7ap67SUKxZmUSq+fq3k9274BuAGopl2YpBfSDItatS1+9yxHUxSPiAUgPRpepXUAACAASURBVD1jZMw5cRe6u+2l0m06tU1URdJPgL/SbMvpTee+yPaGRYONmBq36Ui6yPYWkmYAL6FZGXCV7RcUjjYuSS8DXk5TCDjV9umFI41J0kqDnrd9x7CyzAtJKwKrV/A+tyHNCuXl21N3Am/teu6eWr7O/WrJLGkr4OrecJ22JcbzbV9YNtn4asvcN2Cnd5N1EeD+9uOuruajvdG3eXt4ke1bSuaZKkmiGaa5fuks45H0Q5r2I7+w/UTpPJMxx+tZNKvaO/16bt8zDgX+D7A47fdgR7O+mGaX0d7AYX1P3Qv8t+0/lMgVMepS9IyRUNvE3UyBHQ5JSwOvBK60/QdJzwDWt31a4WgjZZwm8nOd6xJJXwf+FdidZoXqfcBlXZ1eK+kA4GjbN/ade5ftwwvGGpOk62m2UI7Vh8+2O9tvuV1htiPNzbPLgFuBs2zvN+jvlSJpEeCztvdXXdPbf01FX2eoNvNMYJNe73BJ04BLOv7eXF3m2kjaDfgC8Gua9+l/APa3/cOSuQYZp73En23vUS7VYJK2B/YEtgJOpPkZ/ruyqUaPpEtoruVOpBn4+Bbg721/vGiwASSt2a5UjYghyPb2GBWvo5lqXMXwiRq36QDvALbsm6j6OeB8mrurnWT7AUm30GyT+wPNlsTcRZ1Pat6mU1tLDOD9wBslvdf2me25vYHOFT1tr106wzxY3vY9kvYCjrJ9YDtNupNsPy5p0/bjzhc7+1T1dW7VmFn9wxLbgY9dv/avKnONq/mAjwOb91Z3SloVOAPobNGTOttLnAGc0bb/eSNwuqQbaeYPfM/2owP/BwqTdJDtg0rnmAzbf5S0iO3HgaMknVc60wQekPQF4AU0i18A6PLil4iadfYiImKKapy42y9TYBeAdljNZjRTuo+ieY18D9i6ZK4RsjgwneZnSX/vqnto+kB1mqSdaQriphli1OUCxl+B1wInSvqh7S/Q8e8/AEk7Ar12Hr+2/ZOSeSZh0XZF+G40hYEazGwn157IrG2I2D55/L9SXI1f5xozXydpH5oBfgDvoble6rLaMh9Gs5rvUEm1rOabNsd29ttpVk92lu1jSmd4KiStDOwBvBmYCXyf5rrjrTTbnLtsR+Cg0iEm4QFJi9MMIfw8zXCjrg97/D5wPE2f3b1pXg+3Fk0UMcJS9Iyq1TpxN1Ngh2YnYGPgUgDbf2v7g8V8YPss4CxJD9qebaWypNfT4VW17fb2vwd+0J56t6Ttbb+3YKyBbP+l7Qf1jfaX66VKZxpE0mdpesZ9vz21b9vr9WMFY03kU8CpwDm2L5b0bDr8Om6tRFO06F8hYma1IumiGr/ONWbeG/gq8Ama18QvgXcVTTSxqjJXuprvF5JOZdbPvzcAPy+YZ0LtgLaDmNW7X3S/XcrJwPNopre/xvbN7VPHt1uyu67zN1Zbb6bZXfQ+4IPA6sAuRRNNbGXb35a0b9+19FmlQ0WMqvT0jKpJeuug57t6Z3iO3I/R9CXq9DYdAEmbMGui6tkdn6jaP6zmUtubSFoGON/2BqWzjZJKe3peDaw3R9+4K7s6yEjSEbbf2Xf8XuBDHf+F7wpgo96Wz7b/5Mx8/0XEKJljNd/fmLWab33b2xWMNq6+nQ6967kfTfBXipL0O5qC1gz6dh3Zvr1YqAlI+kfbP5vj3BK1tOKSNK2ilg1VkXSB7a3amw9fpXnf+KHtdQpHixhJWekZVRurqFnDVNX+3L28BeNMSt9E1Uvb42UlbdnViaqtEyR9E1hB0juBt9Osvoj5QNIOwD8Cz5L01b6nlqMp5nfZtcAaQK+R/Op0eHt7f8GzPf4a8LVCcaZiBaA3rX35QZ/YNV0v3I8lmYcjmYejhsw1ruaTtDbws14LDElLSVrL9p/LJhvobtudXo06hkOAn81x7nygs69pSc+laS3xNNvrSdoA2NH2IYWjzUXSlczaNTeXjt9gPaRdHf4hmtkIywEfKBspYnSl6BkjYaypqpI6O1W1trytbzD7hdr9Y5zrFNtflPQymh6T6wKftH164Vij5G80wwV2pFl90XMvzYqMLlsZ+K2ki9rjzYHz296I2N6xWLI+kk6wvdt4F/cdv6j/N5p+k2fSrCbaFujy1vY51bK1r18yD0cyD0cNmb813mo+25uVCjWBE2mGEPY83p7bvEycSTmzHfxyMrO3sbq0XKSxSXo68CxgKUkbM+t1vBywdLFgk3MEsD/wTQDbV0g6lqaA2zWvLh1gHtxp+27gbuAl8GQLh4hYAFL0jLlIWge4yfbDkrYDNgC+Y/uusskGqm2qam15obKJqgDtdvZf2T5d0rrAupIW62iPrerYvhy4XNKxFX5NP1k6wCTt2/5Z3cW97R+0N3g2p/ml7yO2/6dsqin5aekAT0EyD0cyD0cNmatbzQcsavuR3oHtR9pBMF22ZftnfyHZzN7LuCteAbwNWA34ErOKnvcA/1oo02Qtbfsiabb7DZ3cuWP7hok/q7MOZe73iLHORcR80OlJfVHMScDjkv6eZlDN2sCxZSNNqH+qatenA0N9eaGdqCppsfaxL92eqApwNrCEpGcBZ9BMWD26aKLRtJakH0q6RtJ1vUfpUBO4BPhN20D+Zpqt1+fZPqs91wl9WyVvA25sL/KXADakWWnbWZJ2Ah6wfYrtHwMPSXpd6VyDtC0bALD9ifbc3uUSTSyZhyOZh6OWzJKeLmlT2tV8kjZpH9vR/dV8t0p6cjeDpNfS/IzpLNsvGePRxYInto+x/RLgbbZf2pf3tb2WAh12W7v4pdfvfFeaa6TOknSvpHvax0OSHpd0T+lcY5H0QkkfAlaVtF/f4yCaYUwRsQCk6BljecL2YzSTr//D9geBZxTONJHeVNU/VjJVtba80ExUfRHwV+AmmrvunZ2o2pLtB4CdgUNt7wQ8v3CmcUlaRFLXt4WP5SiaVgeP0WzT+Q5Nf7MuOxtYsi2I/5LuF8RrywtwYLt9C4B2t8CBBfNMxgGSnvxFWtJHgNcWzDMZyTwcyTwctWR+BfBFZq3m6z0+SPdX8+0N/Kukv6iZNP8Run899yRJVSwWsH1S7+NaMgPvpdna/jxJf6XpM9m5mw79bC9re7n2sSTN5Pb/LJ1rHIsD02l22y7b97gH2LVgroiRluntMRdJFwL/AXycpin79ZKusr1e4WgRUyJpJvAe4N+Bd9i+WtKVttcvHG1ckn7d1Wmv45E0w/am/V9bSb+x/Q+ls42nNyBD0vuBpWx/XtJltjcqnW0s4+SdaXvj0tnGI+mKOXuOVvD9twrN6vv9gVfSDCjZvcvtG5J5OJJ5OGrLLGmX/uJWTSRNp/ld8N7SWaai6z/7xlJDZkmLAJ+1vX/bHmpaba+NHrXT0UvnGI+kNSvfnh9RlU7344ti9qS5q/eZtuC5NvC9wpkmTRVM++xXW16oKvMHaAan/KgteD4bOLNwpomcK+k/geNphkUB3WzW3+chSdOAP0h6H81q4L8rnGkikvRC4E3AO9pzXd5aNFberv8Mv0TSl2mmzBt4P7MPvOoc27e12z7PoMm6a38v4y5K5uFI5uGoLfOcq/lsV9N/2fZ97QrEajK3ZpYO8BR0PrPtx9uWDdi+f6LP7wpJO/cdTqPp+9rJ9wxJ/2H7A8B/ShprOGUnhmhGjJqs9IyBJK0IrG6760N2nlTD3dR+teWFOjPXQs2k6zm5q72rACRtDvwWWAH4NM2E0i/YvqBosAEkbQt8GDjX9ufagvgHbO9TONqYassLTw4SOwDYvj11Gs3NtM79MiXpXppfktT+uThNuwbTfP8tVzDemJJ5OJJ5OGrMPKcar41qzBwLjqQvAc8BTmT2G++d7UUq6ai+w8eAPwNH2L6lTKLxSdrU9gxJLx7reXeop3zEKEnRM+aiZtrujjSriC4DbgXOsr1fyVyTJemQXvP7GtSWF7qfuXcnVdJ/M8bd3txJXXAkPd11TeiuLnNteQEkPcOzhjJFRIwcSUfafnvpHFNRS2ZJVzL39dzdNEMJD7F9+/BTDVZp5qPGOO0aXiO1kbQ4TfsOA9fafqRwpIiRlaJnzKV311fSXjSrPA8cqzdb10h6OrAFzQ+Pi7teFKgtL9STueY7qZKWpxn2sm176izgU/0DYbqsotYHT6otc215ofuZJQ3M1sX2Esk8HMk8HDVmjuGR9HngceDY9tTu7Z/3ANvYfk2RYAPUmLlG7c6XrwBb0fx+cj7wQdvXFQ02gKRXAYcBf6JZ3b428G7bPy8aLGJEpegZc2nvTL4cOAb4uJvp4p0uerYF2k8Cv6L54fFimkLRkUWDjaO2vFBn5hpJOgm4iub7D+DNwIa2dx7/b3VHjVvlastcW17ofuZx2kr0dLK9RDIPRzIPR22Zx1nFB+32/C5eM9eYuUfSuba3HutcVwfkVZp5NeBQYGua18o5wL62byoabABJF9D0Dv9Be2p34P22tyyXajBJvwNebfuP7fE6wE9tP69ssojR1PUhCFHGp4BTgXPaguezgT8UzjSR/YGNe1tFJK0MnAd0tSBXW16oMLOk6xl7e/uzC8SZrHVs79J3fLCky4qlmbojSgd4CmrLXFte6Hhm2y8pnWGqknk4knk4Ksxc2/AfqDNzz3RJW9q+EEDSFsD09rnHysUaqMbMR9GsTH19e7xHe+5lxRJNTLa/23f8vXaoZpfd0it4tq4DOteDNGJUpOgZc7F9Ik0D697xdcAu4/+NTrgJuLfv+F7gxkJZJqO2vFBn5s36Pl6S5iJupUJZJutBSdvYPgdA0tbAg4UzjUnSWF/L43rnbd8x5EgTqi1zbXmhzsz9JK0HPJ/mPQMA298pl2hiyTwcyTwcNWS2fUPpDFNVY+Y+ewFHSppOszL1HmCvdmDevxVNNr4aM69qu7+v59GSPlAszeScKemjwHE0Cx3eAPy0i9ccfZPmr5b0M+AEmsyvBy4uFixixGV7e8xF0pLAO4AXMPsFZ2ebWEv6DrA+8GOaHx6vBS4Cfg9g+8vl0s2ttrxQZ+axSDrH9jalc4xH0kY0W9uXp7lIvgN4m+3LiwYbQ99KWvWdfnL6bhdX1NaWuba8UGfmHkkHAtvRFFx+BuxAs+th15K5Bknm4Ujm4aglc+9aYoyp8733uc5Nm68x85zavueyfVfpLJNVU2ZJZwBHM2ur+BuBPW3/32KhJtBec4ynU9cc4wyK6nGXf9eOqFmKnjEXSScCvwP+iWar+5uA39ret2iwAdqL5HHZPnhYWSajtrxQbeb+wQjTaFZ+/rPtDQtFmjRJywHYvqd0loiFRdvzbkNgpu0NJT0N+FaXB04k83Ak83DUmDkWPElL0Ow6W4u+nYq2P1Uq00QqzbwG8J/AC9tT59L09Kx5lXBELOSyvT3G8ve2Xy/ptbaPkXQsTY/PzupiwW2Q2vJCnZmBL/V9/BhwPbBboSwDSdpvnPNAN1fS1jhtt7bMteWFOjP3edD2E5Iea2883AJ0ZpXIOJJ5OJJ5OKrLLGkb4Dm2j5K0CrCs7UGrz4qrMPOPgbuBGcDDhbNMVnWZbf8F2LF0jqmQtBjwz8C27alfA9+0/WixUBOocVdlRM1S9Iyx9H5I3NX2VfofmruUnSVpVeBfmPuHR6emffbUlhfqyyxpGnCY7eNLZ5mkZds/1wU2B05pj18DnF0k0cS+NOA5A118bdSWuba8UGfmnkskrUAzeGkGcB9NG48uS+bhSObhqCpzuwtmM5qf3UcBiwPfo5l+3Uk1ZgZWs/3K0iGmqLrM7fDarwBb0fy8Ph/4YDvfoau+ASwGfL09fnN7bq9iiSb2XZpdla+gb1dl0UQRIyzb22MukvYCTgI2oLkYmg580vZhRYMNIOk04Hjgw8DewFuBW21/pGiwcdSWF6rNfLbtbSf+zO5ov8672L63PV4WOLG2C+eI2klaC1jO9hWFo0xaMg9HMg9HDZklXQZsDFxqe+P23BW2NyibbHyVZj4cONT2laWzTFalmS8Avsasnp67A++3vWW5VINJunzOtlVjnesSSTNtb9z7vmtXq57a1YUkEbXLSs+Yi+1vtR+eRce3FPVZ2fa3Je1r+yzgLElnlQ41QG15oc7Mp0v6ME2x9v7eyS5NchzDGsAjfceP0NGV1pJeavtXfdMoZ2P75GFnmkhtmWvLC3Vm7pH0y97ABtt/nvNcFyXzcCTzcFSY+RHblmQANZO5u67GzNsAb2uH1jzMrOFLnS3UUmdm2f5u3/H3JL2vWJrJeVzSOrb/BE+uVn28cKaJVLerMqJmKXrGk8brKdjTxZ6CfXo/PG6W9Crgb8BqBfNMpLa8UGfmt9Nsz3nPHOe7XMz/LnCRpB/RZN8Z+E7ZSON6MfArmi34c02CBbpY3Kotc215ocLMbX+tpYFVJK3IrMnzywHPLBZsgGQejmQejhozt06Q9E1gBUnvpLnuOKJwponUmHmH0gGeghoznynpo8BxND+v3wD8VNJK0NlFA/vT5O5twV8L2LNcnEk5vH2f+wRNO6vpwAFlI0WMrhQ9o1+vp2Dvl9N+Xe+DcIik5YEPAYfSXCR/oGykgWrLC3Vmfj5NwXMbmtfwb4DOtmkAsP0ZST8HXkWT+W22ZxaONSbbB7YfXsXs7xsG7pa0ke3LioQbR22Za8sLdWYG3k3zfvZMmj6CvQLtvTSTbLsomYcjmYejxswAT9BcW9wDPJemHdTpZSNNqJrMkpazfQ/N66AKNWbu84b2z3cz63c/MWsRQRcXDZwLfBPorQb/Jk0v0i77LrALTYH2mPbc04qliRhx00oHiO6wfXA7oXsd4Ct9x1+l+0vuX0+zJeMq2y8BXgbsVDjTILXlhTozHwP8H5rX8KHtx8cM/BuFSdqHJuOiNMMFjpH0/rKpJrQpTZ/XZ9D8wvouYDvgCEn/UjDXILVlri0vVJTZ9ldsrw18Btio/fgo4Do6+stTMg9HMg9HjZlbywIfoxn88megs/1H+9SU+VhJAm4DLqEpiPcel5QMNkCNmXs+AmzY9/13OU2f+bVtd7HgCc1uqLWBT7ePtWmKil32Y+C1wGM0w9ruo68NV0TMZ7bzyGO2BzBzMue69Kgtc215K858+WTOdelB88vHMn3HywBXlM41QeZTgel9x9OBXwBLAdeUzjcKmWvLW3HmK9o/twHOpvml5MLSuZK5/COZk3mC3BvQFGx/B5xROs+oZaYZulQ8x0KQubrvv0qv9a8qnSGPPBamR1Z6xlimtX1GAGj7uHS9FUJtmWvLC3Vmnilpq96BpC1ptsF0mZi9AfvjzN1uomvmHL70KLCm7Qdpmvd3UW2Za8sLdWbufe+9CjjM9o9pVlx3WTIPRzIPR42ZAW6hGUZyO/B3hbNMVk2Zz5O0eekQU1Rj5hq//2q81j9P0vqlQ0QsLLpesIgyvkTzZvxDmv4tu9HcCe6y2jLXlhcqyizpSpqMiwFvkfSX9nhN4JqS2SbhKODCdpARwOuAbxfMMxnHAhdI+nF7/BrgB+1E2K5+vWvLXFteqDPzX9sBH9sDn5O0BN1vBZTMw5HMw1FVZkn/TNMHcVXgh8A7bXf1/Q2oMzPwUmBvSTfQbAOuYRJ6jZmr+v5rbcmsa31obrj+tve7QJe+3n2/nywK7NkOX3qYOl4bEdWS3fX5NFGCpOfT/LAW8MsKLoaqy1xbXqgns6Q1Bz1v+4ZhZXkqJG1Cs7VIwNnu6CCjfpI2ZVbmc2x3vW9VdZlrywv1ZZa0NPBK4Erbf5D0DGB926cVjjauZB6OZB6O2jJL+ixwnLs3nG1clWYe87quy9dzlWau6vsP6rrmrylrxChJ0TMiIiIiIiIiIiJGSteXq0dERERERERERERMSYqelZJ0kKQPD+G/864F/d+Y35J5OJJ5wastLyTzsCTzcCTzgldbXkjmYUnm4agtc215IZmHJZmHo8bMMTySjpR0i6Srxnlekr4q6Y+SrmjbuvWee6Wka9vnPjq/MqXoGROp8U0tmYcjmRe82vJCMg9LMg9HMi94teWFZB6WZB6O2jLXlheSeViSeThqzBzDczRNb+Dx7AA8p328C/gGgKRFgK+1zz8feGM7T2SepehZCUlvaSvhl0v67hzPvVPSxe1zJ7VNqJH0eklXtefPbs+9QNJFki5r//eeU+LfExERERERERERo8H22cAdAz7ltcB33LgAWKEdmrYF8Efb19l+BDiu/dx5lkFGFZD0AuBkYGvbt0laCdgHuM/2FyWtbPv29nMPAf7X9qGSrgReafuvklawfZekQ4ELbH9f0uLAIrYfnOO/9y56d3CmaVNWXnp4/9j5YI2llmHVVZYrHWPyHnuEW2+/j1VXnl46yeQttiS33noXq666Qukkk+b/vYVb73+YVZdZonSUSbvJT7D8SkuVjjElD9/5WF2vZeDWm29n1eXr+Tr/dZp54K6HWHqFJUtHmbRneRq33v1gVV/nx6Yvw+233sPKq9bz82TRhx7m1jvvZ9UVlykdZdJuvPu+6t7n7r/rEVZauZ6vMcCdtz/AyqssWzrGlNx+273VZb7rdrPqqquWjjF5j93Orbffy6or1/N1vvW2e6q7zvj9/95R1c9sgCUenMaqqyxfOsaU3HrDX1llej1f5ydWXoHbbr2HVSq6zgC47NLr7rNdz5vGfKY1VjAPPVY6Rjm33n818FDfmcNtH97/KZLWAn5ie705/7qknwCftX1Oe/xL4CPAWjS1q73a828GtrT9vnmNvOi8/g/EULwU+KHt2wBs3yGp//n12mLnCsB04NT2/LnA0ZJOoCmaApwPfFzSasDJtv8w53+sfdEeDqC/m252meu12mk3fPXA0hGmxLddXzrClPnp65aOMGWPfvlrpSNM2Vmvr28h9suWru+14ctmlo4wJR9f+uHSEabsM4/UdfMM4I6tNisdYcpW/tN1pSNM2enL3l86wpSts/wzSkeYsqUWratIVKtnLrNX6QhT4tuPLh1h6h59aOLP6Zj9/zCjdIQp++ILdy0dYcoe+84xpSNMyX1v2ql0hKdkxSV3u7Z0hqIeeoza6iPz1WEXPmR7Xi6SNcY5Dzg/z7K9vQ5i8P/hRwPvs70+cDCwJIDtvYFPAKsDl7UrQo8FdgQeBE6V9NIFGTwiIiIiIiIiIhZ6N9HUp3pWA/424Pw8S9GzDr8EdpO0MkC7vb3fssDNkhYD3tQ7KWkd2xfa/iRwG7C6pGcD19n+KnAKsMFQ/gUREREREREREbGwOgV4SzvFfSvgbts3AxcDz5G0dtuGcff2c+dZtrdXwPbVkj4DnCXpcWAm8Oe+TzkAuBC4AbiSpggK8IV2UJFoCqeXAx8F9pD0KPA/wKeG8o+IiIiIiIiIiIiRJOkHwHbAKpJuAg4EFgOwfRjwM+AfgT8CDwB7ts89Jul9NK0aFwGOtH31/MiUomclbB8DjNmoxPY3gG+McX7nMT7939pHRERERERERETEPLP9xgmeN/DecZ77GU1RdL5K0TMiIiIiIiIiImICmjbWzJ2Fw3yZLDRk6ekZERERERERERERIyVFz4iIiIiIiIiIiBgpKXpGRERERERERETESEnRMyIiIiIiIiIiIkZKBhlFRERERERERERMIIOM6pKVnhERERERERERETFSUvSMiIiIiIiIiIiIkZKiZ0RERERERERERIyUFD0jIiIiIiIiIiJipGSQUURERERERERExCBauAcZ1SgrPSMiIiIiIiIiImKkpOgZERERERERERERIyVFz4iIiIiIiIiIiBgpKXpGRERERERERETESMkgo4iIiIiIiIiIiAlkkFFdstIzIiIiIiIiIiIiRkqKnhERERERERERETFSUvSMiIiIiIiIiIiIkZKiZ0RERERERERERIyUDDKKiIiIiIiIiIgYQAgpg4xqkpWeERERERERERERMVJS9IyIiIiIiIiIiIiRkqJnREREREREREREjJQUPSMiIiIiIiIiImKkZJBRRERERERERETEIAJNyyCjmmSlZ0RERERERERERIyUFD0jIiIiIiIiIiJipKToGRERERERERERESMlRc+IiP/P3r2H21bQ9cL//tZC2NwEFURDUV70hFiErzvSoyle8ljma1pqZSetjCzROqlZD0VomZlGqfBEdHnlVMdjkgZKpWkBylEBuQh4vKX4IOQFOaEgF9n83j/W2G/L1dpr7bk2MPYc+/N5nvnsOcdt/sac+/rdvzF+AAAAwKQYZAQAAAAA6zDIaL7o9AQAAAAAJkXoCQAAAABMitATAAAAAJgUoScAAAAAMCkGGQEAAADAWsogo3mj0xMAAAAAmBShJwAAAAAwKUJPAAAAAGBShJ4AAAAAwKQYZAQAAAAA6zDIaL7o9AQAAAAAJkXoCQAAAABMitATAAAAAJgUoScAAAAAMCkGGQEAAADAGiplkNGc0ekJAAAAAEyK0HMnV1UvqKqTx64DAAAAAOaF0BMAAAAAmBSh5w6oqp+sqo9V1WVV9RdV9fSq+khVXVJV76uqg4btThzW/1NVfbqqfnYbx3t2VV0xHO+8Zau+rar+Ydj395Zt/0dVdVFVXVlVr1q2/Kqqel1VXTA8HjIsP7Cq/qaqLhwej7mLPhoAAAAAGI1BRhtUVQ9PcnySx3T3dVV17ySd5FHd3VX1wiS/kuRlwy5HJnlUkr2TXFJVZ3f3tSsOe0KS/9Ld11TV/suWH5XkEUluTfLJqnpzd1+d5Pjuvr6qFpO8v6qO7O6PDft8rbuPrqqfTPKHSX4wyRuT/EF3f7CqDknyniQPW+Xcjk1ybJIccsh98/k3/eaOfFR3u3rpq9bfaCfSJ/3a2CXM7Pa+fewSZrbHLzx/7BJm9sQ99hq7hJl98Zarxy5hZvd/0rPGLmEmr7zturFLmNnCbvuvv9FO5o7b/nXsEmb2mQfsM3YJM3vKpkeOXcLs7pi/PwOz+5z9ebLgnyh3h3/dNH8/l29cvHHsEmb2hqOeMnYJVY/ShAAAIABJREFUM+sb5u/PwBt+/OljlzCTL9302bFLYCMqBhnNGZ2eG/fEJGd093VJ0t3XJ3lAkvdU1eVJXpHk4cu2P7O7bx62/+ckR69yzPOTvGXoBF1ctvz93X1Dd9+S5ONJHjQsf05VXZzkkuG9jli2z1uX/fjo4fmTk5xcVZcmOSvJPatq35VFdPdp3b25uzcfeMA9t+vDAAAAAICdhdBz4ypLnZ3LvTnJyd39nUl+LsmmZetWbttV9ZqqunQIIdPdL0ry60kemOTSqrrPsO2ty/bbkmS3qjo0ycuTPKm7j0xy9hrvt/X5QpJHd/dRw+Pg7v76DOcMAAAAADs9oefGvT9LnZb3SZLh8vb9klwzrF95Le0zqmrTsP0xSS7s7uO3BpDDMQ7r7o909wlJrstS+Lkt90xyU5IbhnuHfv+K9c9d9uOHhufvTXLc1g2q6qjtPlsAAAAAmBNumLNB3X1lVb0myblVtSVLl5ifmOTtVXVNkg8nOXTZLhdkqRvzkCS/tcr9PJPk9VX10Cx1kb4/yWVZup/nau9/WVVdkuTKJJ/N0qXxy+1RVR/JUrD9Y8OylyY5pao+lqXv/rwkL5rpxAEAAABgJyf03AHdfXqS01csPnMbm3+qu49d53irTdN4y/DYus0PLnv+gjUOd0p3f8tEn+F+os/dxvYAAAAAbINBRvPF5e0AAAAAwKTo9LwbdPeJd/P7PfjufD8AAAAA2Jno9AQAAAAAJkXoCQAAAABMisvbAQAAAGAdVQYZzROdngAAAADApAg9AQAAAIBJEXoCAAAAAJMi9AQAAAAAJsUgIwAAAABYSyW1YJDRPNHpCQAAAABMitATAAAAAJgUoScAAAAAMClCTwAAAABgUgwyAgAAAIA1VMogozmj0xMAAAAAmBShJwAAAAAwKUJPAAAAAGBShJ4AAAAAwKQYZAQAAAAAa6kYZDRndHoCAAAAAJMi9AQAAAAAJkXoCQAAAABMitATAAAAAJgUg4wAAAAAYB0GGc0XnZ4AAAAAwKQIPQEAAACASRF6AgAAAACTIvQEAAAAACbFICMAAAAAWIdBRvNFpycAAAAAMClCTwAAAABgUoSeAAAAAMCkCD0BAAAAgEkxyAgAAAAA1lIGGc0bnZ4AAAAAwKQIPQEAAACASRF6AgAAAACTIvQEAAAAACbFICMAAAAAWEOlDDKaMzo9AQAAAIBJEXoCAAAAAJMi9AQAAAAAJkXoCQAAAABMikFGAAAAALCWikFGc0anJwAAAAAwKUJPAAAAAGBShJ5zpqq+t6qurKpLq2rPsesBAAAAgJ2N0HOOVNVikucleUN3H9XdN49dEwAAAADsbISe66iqn6yqj1XVZVX1F1X19Kr6SFVdUlXvq6qDhu1OHNb/U1V9uqp+dhvHe3ZVXTEc77xh2Quq6uRl27y7qo4Znt9YVa+uqo8k+bUkz0lyQlX9VVXtU1Xvr6qLq+ryqnrGtuoelh1YVX9TVRcOj8fcVZ8bAAAAwJRU1S77mEemt6+hqh6e5Pgkj+nu66rq3kk6yaO6u6vqhUl+JcnLhl2OTPKoJHsnuaSqzu7ua1cc9oQk/6W7r6mq/bejjL2TXNHdJww1PSTJu7v7jKraLckzu/trVXVAkg9X1VlJjlil7iR5Y5I/6O4PVtUhSd6T5GGrnPexSY5NkkMOvnf6us9tR5k7jz7p18YuYSb1y68du4SZ9SmvGbuE2X3zlrErmNnilz89dgkzu9+DHjl2CTN739XvHbuEmTxp8eCxS5jZtfvdMHYJM/u2PR44dgkzO/Dr8/f73L9t+sbYJcxs/+w+dgnT13eMXcHGzNm/B++/92FjlzCz6rEr2IDb5u/3uX9dmL8/t/ecszu/PWzLfcYuAXYJQs+1PTHJGd19XZJ09/VV9Z1J3lZV90+ye5LlieCZwyXnN1fVPyc5Osnfrjjm+UneUlV/neQd21HDliR/s411leR3qupxSe5IcnCSg1are9j+yUmOWJbQ37Oq9u3ury8/aHefluS0JNn8XQ+ax79aAAAAALALE3qurbLU2bncm5Oc1N1nDZegn7hs3cptu6pek+RpSTLch/NFVfU9w7JLq+qoJLfnW281sGnZ81u6e8s26ntekgOTPLK7v1lVVw37rlZ3hvd4tHuBAgAAADBl7um5tvcneU5V3SdJhsvE90tyzbD++Su2f0ZVbRq2PybJhd19/BB2HjUc47Du/shwufp1SR6Y5KokR1XVQlU9MEsdottjvyRfHgLPJyR50Bp1J8l7kxy3dechcAUAAACASdHpuYbuvnLo1Dy3qrYkuSRLnZ1vr6prknw4yaHLdrkgydlJDknyW6vczzNJXl9VD81SN+b7k1w2LP9cksuTXJHk4u0s8a+SvKuqLkpyaZJPrFH3C5K8NMkpVfWxLH335yV50Xa+FwAAAMAuqxbm7AbOuzih5zq6+/Qkp69YfOY2Nv9Udx+7zvGetY1Vz9vG9vuseP2CZc+vS/Lobez3H+oetn/uWvUBAAAAwLxzeTsAAAAAMCk6Pe8k3X3i2DUAAAAAADo9AQAAAICJ0ekJAAAAAGspg4zmjU5PAAAAAGBShJ4AAAAAwKQIPQEAAACASRF6AgAAAACTYpARAAAAAKyhUgYZzRmdngAAAADApAg9AQAAAIBJEXoCAAAAAJMi9AQAAAAAJsUgIwAAAABYx4LWwbni6wIAAAAAJkXoCQAAAABMitATAAAAAJgUoScAAAAAMCkGGQEAAADAGqqSxaqxy2AGOj0BAAAAgEkRegIAAAAAkyL0BAAAAAAmRegJAAAAAEyKQUYAAAAAsI7FBYOM5olOTwAAAABgUoSeAAAAAMCkCD0BAAAAgEkRegIAAAAAk2KQEQAAAACsoZIslkFG80SnJwAAAAAwKUJPAAAAAGBShJ4AAAAAwKQIPQEAAACASTHICAAAAADWUsmi1sG54usCAAAAACZF6AkAAAAATIrQEwAAAACYFKEnAAAAADApBhkBAAAAwBoqlcWqsctgBjo9AQAAAIBJEXoCAAAAADukqp5aVZ+sqs9U1a+usv4VVXXp8LiiqrZU1b2HdVdV1eXDuovujHpc3g4AAAAAbFhVLSY5Jcn3JflCkgur6qzu/vjWbbr79UleP2z/9CT/rbuvX3aYJ3T3dXdWTTo9AQAAAIAdcXSSz3T3Z7v7tiT/M8kz1tj+x5K89a4sSKcnAAAAAKyhkl19kNEBKy47P627T1v2+uAkVy97/YUk37PagapqryRPTXLcssWd5L1V1Un+eMWxN0ToCQAAAACs5bru3rzG+tUS4d7Gtk9Pcv6KS9sf093XVtV9k/xjVX2iu8/baLGJy9sBAAAAgB3zhSQPXPb6AUmu3ca2P5oVl7Z397XDj19O8s4sXS6/Q4SeAAAAAMCOuDDJQ6vq0KraPUvB5lkrN6qq/ZI8PsmZy5btXVX7bn2e5ClJrtjRglzePkFVdWN37zN2HQAAAABMX3ffXlXHJXlPksUkf97dV1bVi4b1pw6bPjPJe7v7pmW7H5TknbV0z9TdkvyP7v6HHa1J6LmLq6rF7t4ydh0AAAAAO61KFhd26UFG6+ruv0vydyuWnbri9VuSvGXFss8m+a47ux6h512gqn4yycuzdMPWjyX56yS/nmT3JF9N8rzu/lJVnZjksCxNuHpgkt/r7j9Z5XgHJTk1yf81LPr57v5fVfXLSX56WPan3f2HK/arJL+X5PuHWn67u99WVcck+c0k/5rkqCRHbPNk7rEpfb9vn/UjGNXtffvYJcykT3nN2CXMrF58/NglzOwrf3Dc+hvtZL64z3z9XE6Sh3/xk2OXMLMjD3jY2CXM5CtJ7vvNPcYuY0ZfH7uAmf3DNe8fu4SZfc/9Hjl2CTPbb/d7j13CzL6x5RtjlzCzvW66fv2NdiY1p3fg2nfsAmb0lX8Zu4KZfXzxq2OXMLPD9jty7BJmtrhl/mKCr9x89fob7URu3NOFmXB3mL/fzXZyVfXwJMdnaerUdVV17ywFjo/q7q6qFyb5lSQvG3Y5Msmjkuyd5JKqOnvrzVuXeVOSc7v7mVW1mGSfqnpkkp9K8j1ZmpD1kao6t7svWbbfs7IUan5XkgOSXFhVWydfHZ3kO7r7c6ucw7FJjk2SQw657w59HgDcueYv8AQAALj7zel/o+7UnpjkjO6+Lkm6+/osTax6T1VdnuQVSR6+bPszu/vmYft/zurTqZ6Y5I+G423p7huSPDbJO7v7pu6+Mck7knzviv0em+Stwz5fSnJuku8e1l2wWuA5vMdp3b25uzcfeOD+M38AAAAAADAmoeedr7LU2bncm5Oc3N3fmeTnkmxatm7ltl1Vr6mqS6vq0nXeZ3tq2Zab1lgHAAAAAHNL6Hnne3+S51TVfZJkuLx9vyTXDOufv2L7Z1TVpmH7Y5Jc2N3Hd/dR3X3UsmP+/HC8xaq6Z5LzkvxQVe1VVXtnafrVB1Yc+7wkzx32OTDJ45JccGeeLAAAAMDUVZLF2nUf88g9Pe9k3X1lVb0myblVtSXJJUlOTPL2qromyYeTHLpslwuSnJ3kkCS/tcr9PJPkF5OcVlU/k2RLlgYZfaiq3pJ/DzH/dMX9PJPknUkeneSyLHWU/kp3f7GqDr8TThUAAAAAdkpCz7tAd5+e5PQVi8/cxuaf6u5j1znel5I8Y5XlJyU5aZXl+ww/dpbuIfqKFevPSXLOWu8JAAAAAPPK5e0AAAAAwKTo9BxRd584dg0AAAAAMDVCTwAAAABYx+LCnE702UW5vB0AAAAAmBShJwAAAAAwKUJPAAAAAGBShJ4AAAAAwKQYZAQAAAAAa6gki2WQ0TzR6QkAAAAATIrQEwAAAACYFKEnAAAAADApQk8AAAAAYFIMMgIAAACANVRVFhcMMponOj0BAAAAgEkRegIAAAAAkyL0BAAAAAAmRegJAAAAAEyKQUYAAAAAsI5Fc4zmik5PAAAAAGBShJ4AAAAAwKQIPQEAAACASRF6AgAAAACTYpARAAAAAKyhkiwumGQ0T3R6AgAAAACTIvQEAAAAACZF6AkAAAAATIrQEwAAAACYFIOMAAAAAGAtlSyWQUbzRKcnAAAAADApQk8AAAAAYFKEngAAAADApAg9AQAAAIBJMcgIAAAAANZQMcho3uj0BAAAAAAmRegJAAAAAEyK0BMAAAAAmBShJwAAAAAwKQYZAQAAAMAaKsmi1sG54usCAAAAACZF6AkAAAAATIrQEwAAAACYFKEnAAAAADApBhkBAAAAwDoWq8YugRno9AQAAAAAJkXoCQAAAABMitATAAAAAJgUoedIqurBVXXFDuy/uaredGfWBAAAAABTYJDRTqyqFrt7y2rruvuiJBfdzSUBAAAA7HKqKosLBhnNE52e49qtqk6vqo9V1RlVtVdVXVVVJ1TVB5M8u6rOqarNSVJVB1TVVcPzY6rq3cPzx1fVpcPjkqrad1j+iqq6cDj+q4Zle1fV2VV1WVVdUVXPHefUAQAAAOCuodNzXN+e5Ge6+/yq+vMkvzAsv6W7H5skVfWi7TjOy5O8eDjOPkluqaqnJHlokqOTVJKzqupxSQ5Mcm13P204/n5rHbi/9OV886RTNnJuo9njF54/dgmz+eYtY1cws6/8wXFjlzCzA//byWOXMLOvvemVY5cwu3/76tgVzOyAy64cu4SZ3HH9DWOXMLObH3P42CXM7Kn7P2LsEmZ2x+4HjF3CzLacetrYJcxs0yMePHYJM+tDHjh2CbPZcvvYFWzIUtvBHKn56385/F6bxy5hZl9+9vFjlzCze/zlL45dwszut9ehY5cwk33usf/YJcAuYf7+pJuWq7v7/OH5XyZ57PD8bTMe5/wkJ1XVS5Ps3923J3nK8LgkycVJDs9SCHp5kidX1euq6nu7+z/867mqjq2qi6rqoq/cdOvsZwUAAAAAIxJ6jqu38fqmZctuz79/T5tWPUj37yZ5YZI9k3y4qg7PUnfna7v7qOHxkO7+s+7+VJJHZin8fG1VnbDK8U7r7s3dvfnAvffY8MkBAAAAwBhc3j6uQ6rq0d39oSQ/luSDSVZeT3dVlkLKC5L8yGoHqarDuvvyJJdX1aOz1NX5niS/VVV/1d03VtXBSb6Zpe/8+u7+y6q6MckL7oLzAgAAAJiMSrJYBhnNE6HnuP53kudX1R8n+XSSP0rykhXbvCHJX1fVf03yT9s4zi9V1ROSbEny8SR/3923VtXDknyoln5R3pjkJ5I8JMnrq+qOLIWgP38nnxMAAAAAjEroOZLuvirJEausevCK7T6R5Mhli359WH5OknOG5yuD0q37vjHJG1cs/pcsdYECAAAAwCS5pycAAAAAMClCTwAAAABgUoSeAAAAAMCkuKcnAAAAAKxjUevgXPF1AQAAAACTIvQEAAAAACZF6AkAAAAATIrQEwAAAACYFIOMAAAAAGANVcli1dhlMAOdngAAAADApAg9AQAAAIBJEXoCAAAAAJMi9AQAAAAAJsUgIwAAAABYx+KCQUbzRKcnAAAAADApQk8AAAAAYFKEngAAAADApAg9AQAAAIBJMcgIAAAAANZQSRbLIKN5otMTAAAAAJgUoScAAAAAMClCTwAAAABgUoSeAAAAAMCkGGQEAAAAAGuoSha1Ds4VXxcAAAAAMClCTwAAAABgUoSeAAAAAMCkCD0BAAAAgEkxyAgAAAAA1lRZrBq7CGag0xMAAAAAmBShJwAAAAAwKUJPAAAAAGBShJ4AAAAAwKQYZAQAAAAAa6gki+YYzRWdngAAAADApAg9AQAAAIBJEXoCAAAAAJMi9AQAAAAAJsUgIwAAAABYx0KZZDRPdHoCAAAAAJMi9AQAAAAAJkXoCQAAAABMitATAAAAAJgUg4wAAAAAYA2VZNEco7mi0xMAAAAAmBShJwAAAAAwKUJPAAAAAGBShJ4TU1UnVtXLx64DAAAAAMZikNEurqp26+7bx64DAAAAYKdVyYJBRnNFp+ecqapfrqorhscvDcuOr6pPVtX7knz7sm2PqqoPV9XHquqdVXWvYfk5VfU7VXVukl8c50wAAAAA4K6h03OOVNUjk/xUku9JUkk+UlUfSPKjSR6Rpe/z4iQfHXb570le0t3nVtWrk/xmkl8a1u3f3Y/fxvscm+TYJLnvwffMuc9+6F10RneNJ+6x19glzGTxy58eu4SZfXGf+WsO/tqbXjl2CTO750tfN3YJM+vfn7/POYeNXcBs/mzfi8cuYWYvzEFjlzCz995w6dglzOxTn//a2CXM7MU/tOpfRXZqt9330LFLmNnu1/zvsUuYzZb5+3vGXFqYv38KLnz9urFLmNn9/t+XjV3C7G6dw1+D++4/dgUz+fyNnxy7BNgl6PScL49N8s7uvqm7b0zyjiRPG5Z9o7u/luSsJKmq/bIUbJ477Ht6ksctO9bbtvUm3X1ad2/u7s373XvPu+REAAAAAOCuIvScL9u6e0Rv4Fg37UghAAAAALCzEnrOl/OS/FBV7VVVeyd5ZpKzkzyzqvasqn2TPD1JuvuGJP+nqr532Pe/Jjl3tYMCAAAAsG2VZLF23cc8mr8buezCuvviqnpLkguGRX/a3R+tqrcluTTJ55N8YNkuz09yalXtleSzWbofKAAAAABMmtBzznT3SUlOWrHsNUles8q2lyZ51CrLj7mr6gMAAACAsbm8HQAAAACYFKEnAAAAADApLm8HAAAAgHUsLMzpRJ9dlE5PAAAAAGBShJ4AAAAAwKQIPQEAAACASRF6AgAAAACTYpARAAAAAKyhkiyaYzRXdHoCAAAAAJMi9AQAAAAAJkXoCQAAAABMitATAAAAAJgUg4wAAAAAYC2VLBhkNFd0egIAAAAAkyL0BAAAAAAmRegJAAAAAEyK0BMAAAAAmBSDjAAAAABgDZVk0SCjuaLTEwAAAACYFKEnAAAAADApQk8AAAAAYFKEngAAAADApBhkBAAAAADrWCiTjOaJTk8AAAAAYFKEngAAAADApAg9AQAAAIBJEXoCAAAAAJNikBEAAAAArKGSLJpjNFd0egIAAAAAkyL0BAAAAAAmRegJAAAAAEyK0BMAAAAAmBSDjAAAAABgHQsGGc0VnZ4AAAAAwA6pqqdW1Ser6jNV9aurrD+mqm6oqkuHxwnbu+9G6PQEAAAAADasqhaTnJLk+5J8IcmFVXVWd398xaYf6O4f3OC+M9HpCQAAAADsiKOTfKa7P9vdtyX5n0mecTfsu01CTwAAAABgLQdU1UXLHseuWH9wkquXvf7CsGylR1fVZVX191X18Bn3nYnL2wEAAABgDVXJYu3Sk4yu6+7Na6xf7cPpFa8vTvKg7r6xqn4gyd8meeh27jsznZ4AAAAAwI74QpIHLnv9gCTXLt+gu7/W3TcOz/8uyT2q6oDt2XcjhJ4AAAAAwI64MMlDq+rQqto9yY8mOWv5BlV1v6qldtmqOjpLueRXt2ffjXB5OwAAAACwYd19e1Udl+Q9SRaT/Hl3X1lVLxrWn5rkR5L8fFXdnuTmJD/a3Z1k1X13tCahJwAAAACwQ4ZL1v9uxbJTlz0/OcnJ27vvjhJ6AgAAAMA6FnbpOUbzxz09AQAAAIBJEXoCAAAAAJMi9AQAAAAAJkXoCQAAAABMyi4VelbVq6vqycPzc6pq8/D8qqo6YIbjHFVVP7Ds9TFV9Z93oK4bN7rvBt9vh+oFAAAA2JVUksXadR/zaJcKPbv7hO5+351wqKOS/MCy18ckmacQ8ZjMV70AAAAAsN12G7uAu0pV/UaS5yW5Osl1ST6a5DuSvLu7z9jOYxyd5A+T7Jnk5iQ/leRzSV6dZM+qemyStyZ5UZItVfUTSV6SZP8kv55k9yRfTfK87v5SVe2T5M1JNifpJK/q7r8Z3us1SX5weJ9nDNu/ZXh9eJIHDe///CSPTvKR7n7BsO9TkrwqyR5J/iXJT3X3jVV1VZLTkzw9yT2SPDvJLSvr7e4PbPcHCwAAAAA7uUmGnsNl6z+c5BFZOseLsxR6zuoTSR7X3bcPl8X/Tnf/cFWdkGRzdx83vN+eSW7s7jcMr++V5FHd3VX1wiS/kuRlSX4jyQ3d/Z3LtkuSvZN8uLuPr6rfS/KzSX57WHevJE9M8v8keVeSxyR5YZILq+qoJF/IUsD65O6+qapemeSXsxTMJsl13f1/V9UvJHl5d7+wqk5dXu8qn9+xSY5NkkMecJ98317fvoGPbjxfvOXqsUuYyf0e9MixS5jZw7/4ybFLmN2/fXXsCmbWv//KsUuYWb3sdWOXMLN+/SvGLmEmP3Pw/DXqn3PjFWOXMLPD9rv/2CXM7HEHP27sEmZWN92td/i5U+xx221jlzCzk//t02OXMJN7zOm1aD83dgEzuu/pfz12CTP7/Evm7VNO9rpl/n7P+NuvXjB2CTN7eB44dgkz2XNxn7FLgF3CJEPPJI9NcmZ335wkVfWuDR5nvySnV9VDs9SZeY/t3O8BSd5WVffPUrfn54blT07yo1s36u7/Mzy9Lcm7h+cfTfJ9y471riE8vTzJl7r78iSpqiuTPHh4ryOSnF9VGd7vQ8v2f8ey4z5re4rv7tOSnJYkm496cG/PPgAAAACws5hq6LmhW6xW1Yuz1GWZLN2z87eS/HN3P7OqHpzknO081JuTnNTdZ1XVMUlOXFbXaiHiN7t76/It+dbv5dbhxzuWPd/6erdh+3/s7h/bRi1b91l5XAAAAAC208KcXo2wq5rq1/XBJE+vqk3DfTSftj07dfcp3X3U8Lg2S52e1wyrX7Bs068n2XeN18v3e/6y5e9NctzWF8sub98RH07ymKp6yHDMvarqP62zz8p6AQAAAGAyJhl6dveFSc5KclmWLu++KMkNGzjU7yV5bVWdn2Rx2fJ/TnJEVV1aVc/N0r02nzm8/t4sdXa+vao+kKUhSlv9dpJ7VdUVVXVZkidsoKZv0d1fyVIg+9aq+liWQtDD19ltZb0AAAAAMBlTvtz5Dd19YlXtleS8JL/f3X+ydWV3H7Ps+YNXO0B3fyjJ8q7J3xiWX5/ku1dsfuSK12eucrwb862dn1uX77Ps+RlJzhiev2DZ8quyNH0+q6z7p1Xq+Zbz6u6LkhwzPP/UKvUCAAAAwCRMOfQ8raqOSLIpyendffHYBQEAAAAAd73Jhp7d/eNj1wAAAADA/KuqLNaG5mYzkkne0xMAAAAA2HUJPQEAAACASRF6AgAAAACTIvQEAAAAACZlsoOMAAAAAODOsmCO0VzR6QkAAAAATIrQEwAAAACYFKEnAAAAADApQk8AAAAAYFIMMgIAAACANVSSRYOM5opOTwAAAABgUoSeAAAAAMCkCD0BAAAAgEkRegIAAAAAk2KQEQAAAACsY8Ego7mi0xMAAAAAmBShJwAAAAAwKUJPAAAAAGBShJ4AAAAAwKQYZAQAAAAAa6gki2WS0TzR6QkAAAAATIrQEwAAAACYFKEnAAAAADApQk8AAAAAYFIMMgIAAACAtVSyYI7RXNHpCQAAAABMitATAAAAAJgUoScAAAAAMClCTwAAAABgUgwyAgAAAIA1VJJFg4zmik5PAAAAAGBShJ4AAAAAwKQIPQEAAACASRF6AgAAAACTYpARAAAAAKxjoUwymic6PQEAAACASRF6AgAAAACTIvQEAAAAACZF6AkAAAAATIpBRgAAAACwhkqyaI7RXNHpCQAAAABMitATAAAAAJgUoScAAAAAMClCTwAAAABgUgwyAgAAAIB1LJRJRvNEp+cGVdWrq+rJw/Nzqmrz8Pyqqjrgbq5lc1W96e58TwAAAADYWen03KDuPmHsGrbq7ouSXDR2HQAAAACwM9DpuR2q6jeq6hNV9Y9V9daqenlVvaWqfmSGYzy+qi4dHpdU1b5VdUxVnVdV76yqj1fVqVW1MGz/R1V1UVVdWVWvWnac766q/1VVl1XVBcuO8+5h/YlV9edD9+lnq+oz2tSeAAAgAElEQVSla53Hnfk5AQAAAMDOQKfnOobL1n84ySOy9HldnOSjGzjUy5O8uLvPr6p9ktwyLD86yRFJPp/kH5I8K8kZSY7v7uurajHJ+6vqyCSfSPK2JM/t7gur6p5Jbl7lvQ5P8oQk+yb5ZFX9UZLv2t7zqKpjkxybJIfcd5/0pZds4HTHc/8nPWvsEmbyvqvfO3YJMzvygIeNXcLMDrjsyrFLmN1hYxcwu379K8YuYWb1itePXcJM7njtcWOXMLOH3euhY5cws4O27D12CTO74t8uH7uEmc3jfbHus9tBY5cwsxd/8z5jlzCbTbuPXcEu4SvHvXjsEmb2ia9fMXYJMzt8n+8Yu4SZ/dD9Hjd2CTP708/949glzORn7v+osUuAXYLQc32PTXJmd9+cJFX1rg0e5/wkJ1XVXyV5R3d/oZb+on9Bd392OPZbh/c7I8lzhvBxtyT3z1Iw2kn+tbsvTJLu/tqw38r3Oru7b01ya1V9OclBs5xHd5+W5LQk2fyf7tsbPF8AAACASaiaz/+w3ZW5vH19G/oZXVUvXnY5+7d19+8meWGSPZN8uKoOHzZdGSp2VR2apc7QJ3X3kUnOTrJpqGV7Qshblz3fkqXg1K9MAAAAAHYJQs/1fTDJ06tq03BZ+tO2Z6fuPqW7jxoe11bVYd19eXe/LktDh7aGnkdX1aHDvTyfO7zfPZPclOSGqjooyfcP234iybdV1XcnyXA/z+3t1t3QeQAAAADAvHF5+zqGe2eeleSyLN1386IkN2zgUL9UVU/IUuflx5P8fZJHJ/lQkt9N8p1Jzkvyzu6+o6ouSXJlks9m6dL4dPdtVfXcJG+uqj2zdD/PJ9/N5wEAAAAAOzWh5/Z5Q3efWFV7ZSmY/P3u/pOtK7v7mGXPH7zaAbr7JSuXDffi/EZ3P3eV7V+wjeNcmGTlXY/PGR7p7hNXbL/8ztn/4TxWew8AAAAAmGdCz+1zWlUdkaX7ap7e3RePXdAGTeU8AAAAAO5GZZDRnBF6bofu/vG76LjnZOjQvDvcVecBAAAAADsTg4wAAAAAgEkRegIAAAAAkyL0BAAAAAAmxT09AQAAAGANlWSh9A7OE98WAAAAADApQk8AAAAAYFKEngAAAADApAg9AQAAAIBJMcgIAAAAANaxUDV2CcxApycAAAAAMClCTwAAAABgUoSeAAAAAMCkCD0BAAAAgEkxyAgAAAAA1mGQ0XzR6QkAAAAATIrQEwAAAACYFKEnAAAAADApQk8AAAAAYFIMMgIAAACANVSVQUZzRqcnAAAAADApQk8AAAAAYFKEngAAAADApAg9AQAAAIBJMcgIAAAAANaxoHdwrvi2AAAAAIBJEXoCAAAAAJMi9AQAAAAAJkXoCQAAAABMikFGAAAAALCGSrJQNXYZzECnJwAAAAAwKUJPAAAAAGBShJ4AAAAAwKQIPQEAAACASTHICAAAAADWYZDRfNHpCQAAAABMitATAAAAAJgUoScAAAAAMClCTwAAAABgUgwyAgAAAIA1VRZK7+A88W0BAAAAAJMi9AQAAAAAJkXoCQAAAABMitATAAAAAJgUg4wAAAAAYA2VZKFq7DKYgU5PAAAAAGBShJ47iap6dVU9eXh+TlVtHp5fVVUHbOcxvq2qzrgr6wQAAACAnZ3L23cS3X3CnXCMa5P8yJ1QDgAAAADMLaHnCKrqN5I8L8nVSa5L8tEk35Hk3d29XZ2aVfX4JG8cXnaSxyW5z3CM76iqP02yeVh/cJKTu/tVVfWKJM9JskeSd3b3b671PtcsdI7f69aZzm9sr7zturFLmMmTFg8eu4SZ1Tf3GLuEmd1x/Q1jlzCzP9v34rFLmNnPHPyfxy5hZne89rixS5jZwq+dPHYJM7n14YeMXcLM+vsfOXYJM3vAAw4bu4SZ7d+bxi5hdrffNnYFM+uxC5jVbotjV7BruPUbY1cws2//+vz93Pja3vP3Oe/5F28fu4SZ/fRBe49dwky+fOgdY5cAuwSh591suGz9h5M8Ikuf/8VZCj1n9fIkL+7u86tqnyS3LF/Z3S8c3u9BSd6T5C1V9ZQkD01ydJbuwXtWVT2uu89bUeOxSY5NknsetM8GSgPgrjJvgScAAExCGWQ0b9zT8+732CRndvfN3f31JO/a4HHOT3JSVb00yf7dffvKDapqU5K3Jzmuuz+f5CnD45Isha2HZykE/RbdfVp3b+7uzXvtP4cdGAAAAADs0nR63v029N8CVfXiJD87vPyB7v7dqjo7yQ8k+fAwBOmWFbudmuQd3f2+Ze/92u7+443UAAAAAADzQKfn3e+DSZ5eVZuGy9Kftj07dfcp3X3U8Li2qg7r7su7+3VJLspS1+b/bwhJ9+3u3122+D1Jfnp431TVwVV13zvlrAAAAABgJ6HT827W3RdW1VlJLkvy+SwFlhuZsPJLVfWEJFuSfDzJ3ye5/7L1L0/yzaq6dHh9anefWlUPS/KhWroPxY1JfiLJlzd0MgAAAACwExJ6juMN3X1iVe2V5Lwkv9/df7J1ZXcfs+z5g1c7QHe/ZJXFV2VpCny6+9Bt7PfG/PvUdwAAAADWUTHIaN4IPcdxWlUdkWRTktO7++KxCwIAAACAqRB6jqC7f3zsGgAAAABgqgwyAgAAAAAmRegJAAAAAEyKy9sBAAAAYE2VhdI7OE98WwAAAADApAg9AQAAAIBJEXoCAAAAAJMi9AQAAAAAJsUgIwAAAABYx0Jq7BKYgU5PAAAAAGBShJ4AAAAAwKQIPQEAAACASRF6AgAAAACTYpARAAAAAKyhkiyUQUbzRKcnAAAAADApQk8AAAAAYFKEngAAAADApAg9AQAAAIBJMcgIAAAAANZUWSi9g/PEtwUAAAAATIrQEwAAAACYFKEnAAAAADApQk8AAAAAYFIMMgIAAACANVQlC1Vjl8EMdHoCAAAAADukqp5aVZ+sqs9U1a+usv55Vf8fe/ceNllZ3on69zSgnBQUCAFBYRsnHkkrjYYIBiIRlORCcxCNOwkYZWviZNgzxnEuHIyZOGo0ZhKPm8w2kBiNJmrEeADCcPBIRM6MiTqK2wjbpAVB8YDQz/6jVm8rX75TNTRFLe77uuqqVWu9632fqoLur3/fu9ZbVw2PT1TVj00du66qrq6qK6rq0ruiHjM9AQAAAIBtVlU7JHlTkp9O8o9JPl1VZ3f3/5xq9qUkP9ndN1XVU5OckeQJU8eP7u7Nd1VNZnoCAAAAAHfG45N8obu/2N23JfmLJCdMN+juT3T3TcPLTyU5YHsWJPQEAAAAAO6MByX5ytTrfxz2reTXknx46nUnObeqPlNVp9wVBbm8HQAAAADWcC9fyGjvJffaPKO7z5h6vdyH08t1VFVHZxJ6HjG1+4ndfX1V/VCS86rq77v74jtTsNATAAAAAFjN5u7etMrxf0xy4NTrA5Jcv7RRVR2S5L8neWp3f33r/u6+fnj+p6p6XyaXy9+p0NPl7QAAAADAnfHpJA+rqoOr6j5JnpXk7OkGVfXgJO9N8svd/bmp/btV1f22bid5SpJr7mxBZnoCAAAAANusu2+vqhclOSfJDkne1t3XVtULhuNvTXJ6kr2SvLkmtwq4fZg9um+S9w37dkzyju7+yJ2tSegJAAAAANwp3f2hJB9asu+tU9vPS/K8Zc77YpIfu6vrEXoCAAAAwBo2lLtELhLfFgAAAAAwKkJPAAAAAGBUhJ4AAAAAwKgIPQEAAACAUbGQEQAAAACsolLZUDXvMpiBmZ4AAAAAwKgIPQEAAACAURF6AgAAAACjIvQEAAAAAEbFQkYAAAAAsIYNsZDRIjHTEwAAAAAYFaHnHFTV71TVMcP2hVW1adi+rqr2vgv6v0v6AQAAAIBF5PL2Oeju0+ddw1ZVVUmqu7fMuxYAAAAAuCuY6bmdVdV/rqq/r6rzquqdVfXiqjqzqn5hhj52r6o/qaqrq+qqqvr5Yf+zh33XVNVrVjj33w/Hr6mqU4d9B1XVZ6vqzUkuS3LgXfFeAQAAAOCewEzP7Wi4bP3nkzw2k8/6siSf2Yau/nOSm7v7MUO/D6iq/ZO8JsmhSW5Kcm5VPb27/3pq/EOTnJzkCUkqySVVddHQ/keTnNzdv77awA/qDXnlbbtuQ8nzs2HHPeddwkyu3+PmeZewDb457wJm9p0nPnzeJczsedl33iXM7MJvXTPvEmb2iAc8bN4lzOR7j3rwvEuY2X2v/X/mXcLMPvyzB827hJkdteOj513C7G6fdwHbYKed513BzK58yIL9LFeLuUjEIfMuYEaf+OZV8y5hZgfv/ZB5lzCz/W6bdwWzO+tJD5x3CTPbbafFijaesuP95l0C26CyuH9H3VuZ6bl9HZHk/d39ne7+ZpIPbGM/xyR509YX3X1TksOSXNjd/9zdtyf58yRPWmb893X3rd39rSTvTXLkcOzL3f2p5QarqlOq6tKquvSfb/7ONpYMAAAAAPMh9Ny+tulXAFX1G1V1xfDYf+int6Hv1drcutKB7j6juzd196Z99thlHcMAAAAAwD2H0HP7+liSn62qnatq9yTHr+ek7n5Td28cHtcnOTfJi7Yer6oHJLkkyU9W1d5VtUOSZye5aElXFyd5elXtWlW7JXlGko/e+bcFAAAAAPdcQs/tqLs/neTsJFdmcmn5pUm25QaOv5vkAcNiRFcmObq7b0jyn5JcMPR/WXe/f8n4lyU5M8nfZRKS/vfuvnwb3w4AAAAALITFutvvYnpdd/92Ve2ayczL3+/uP956sLuPmto+aLkOhvtx/uoy+9+R5B3L7D9oavv1SV6/5Ph1SRZwtQMAAACAOahkQ5k7uEiEntvfGVX1yCQ7JzlrmH0JAAAAAGwnQs/trLt/ad41AAAAAMC9iXm5AAAAAMCoCD0BAAAAgFFxeTsAAAAArKqyoWreRTADMz0BAAAAgFERegIAAAAAoyL0BAAAAABGRegJAAAAAIyKhYwAAAAAYBWVpMrcwUXi2wIAAAAARkXoCQAAAACMitATAAAAABgVoScAAAAAMCoWMgIAAACANWwwd3Ch+LYAAAAAgFERegIAAAAAoyL0BAAAAABGRegJAAAAAIyKhYwAAAAAYFWVKnMHF4lvCwAAAAAYFaEnAAAAADAqQk8AAAAAYFSEngAAAADAqFjICAAAAABWUZVssJDRQvFtAQAAAACjIvQEAAAAAEZF6AkAAAAAjIrQEwAAAAAYFQsZAQAAAMCqKmXu4ELxbQEAAAAAoyL0BAAAAABGRegJAAAAAIyK0BMAAAAAGBULGQEAAADAGjaUuYOLxLcFAAAAAIyK0BMAAAAAGBWhJwAAAAAwKkJPAAAAAGBULGQEAAAAAGsocwcXim8LAAAAABgVoScAAAAAMCpCTwAAAABgVISeAAAAAMCoWMgIAAAAAFZRqWwocwcXiW8LAAAAABgVoScAAAAAMCpCz21QVadW1a7raHdhVW0atr+1jWOdVFVv3JZzAQAAAODeSOi5bU5NsmboCQAAAADc/YSea6iq3arqg1V1ZVVdU1UvT7J/kguq6oKhzVuq6tKquraqXrFGf3tX1Ser6vhljv3iMMaVVXXx1KH9q+ojVfX5qvq9qfbLjltV11XVa6rq74bHjwz796mq91TVp4fHE+/kxwMAAABwr1C14V77WERWb1/bcUmu7+7jk6Sq9khycpKju3vz0Oa07r6xqnZIcn5VHdLdVy3tqKr2TXJ2kpd193nLjHV6kmO7+6tVtefU/o1JHpvke0n+oare0N1fWWPcW7r78VX1K0n+W5KfSfKHSf6guz9WVQ9Ock6SRyxT5ylJTkmSAw7cKzf++KYZPq7523LbDfMuYSb73/fAeZcws4989fx5lzCz4/Z87LxLmNm5N18x7xJm9tA99pt3CTPb947d5l3CTPqph867hJl9+GcPmncJM3vqqy9eu9E9zOde/cPzLmFmD979X/0Yco/31Vs/P+8SZvZP37553iXM5PYtW+ZdwjY5ZK95VzCb/XffZ94lzOzqzf8w7xJm9vXdHjDvEma2zy6L9bNRkuy9y/3mXcJMLvvny+ZdAtwrLGZUe/e6Oskxw8zJI7t7uZ8an1lVlyW5PMmjkjxymTY7JTk/yUtWCDyT5ONJzqyq5yfZYWr/+d19c3d/N8n/TPKQdYz7zqnnw4ftY5K8saquyCR8vX9V/au/Hbr7jO7e1N2b9trn/iuUCgAAAAD3TGZ6rqG7P1dVhyZ5WpJXVdW508er6uAkL05yWHffVFVnJtl5ma5uT/KZJMcmuWg495VJjh/G2djdL6iqJwz7rqiqjcO535vq544kO65j3F5me0OSw7v7O7N8BgAAAACwSMz0XENV7Z/k29399iSvS/K4JN9MsnWG5P2T3Jrk5uHy9aeu0FUneW6Sh1fVS5Oku08bws6Nw1gP7e5Luvv0JJuTrHbd81rjnjj1/Mlh+9wkL5p6bxsDAAAAACNjpufaHpPktVW1Jcn3k7wwk8vFP1xVN3T30VV1eZJrk3wxk0vUl9Xdd1TVs5J8oKpu6e43L2ny2qp6WJLK5FL4KzO5n+dyfV25xrj3rapLMgm2nz3s+80kb6qqqzL57i9O8oJ1fQoAAAAA91qVDeYOLhSh5xq6+5xMFvyZdmmSN0y1OWmFc4+a2t59eL4tk0vcl2v/c8vsPnN4bG3zM2uNO3hTd/+LleSHhZdOXKE9AAAAAIyCiBoAAAAAGBUzPUeouw+adw0AAAAAMC9megIAAAAAo2KmJwAAAACsopJUmTu4SHxbAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGxUJGAAAAALCaqmywkNFC8W0BAAAAAKMi9AQAAAAARkXoCQAAAACMitATAAAAABgVCxkBAAAAwBoqO8y7BGZgpicAAAAAMCpCTwAAAABgVISeAAAAAMCoCD0BAAAAgFGxkBEAAAAArKJS2VDmDi4S3xYAAAAAMCpCTwAAAABgVISeAAAAAMCoCD0BAAAAgFGxkBEAAAAArKHMHVwovi0AAAAAYFSEngAAAADAqAg9AQAAAIBREXoCAAAAAKNiISMAAAAAWMOGMndwkfi2AAAAAIBREXoCAAAAAKMi9AQAAAAARkXoCQAAAACMioWMAAAAAGAVlUpZyGih+LYAAAAAgFERegIAAAAAoyL0BAAAAABGRegJAAAAAIyKhYwAAAAAYA0bzB1cKL4tAAAAAGBUhJ4AAAAAwKgIPQEAAACAURF6AgAAAACjIvQEAAAAAEbF6u0AAAAAsKpKlbmDi8S3BQAAAACMitATAAAAABgVoed2VlWnVtWu62h3YVVtGra/dTfUdVJV7b+9xwEAAACAu5vQc/s7NcmaoeedUVXbcm/Wk5IIPQEAAAAYHQsZ3YWqarck705yQJIdkvxlJsHiBVW1ubuPrqq3JDksyS5J/qq7X75Kf3sn+UCS3+3uDy45dmaSG5M8NsllVfVnSd6aScD6v5I8t7tvqqqNS/cneXKSTUn+vKq+k+Tw7v7OXfQxAAAAAIxKVbLBQkYLReh51zouyfXdfXySVNUeSU5OcnR3bx7anNbdN1bVDknOr6pDuvuqpR1V1b5Jzk7ysu4+b4Xx/k2SY7r7jqq6Ksm/7e6Lqup3krw8k1mmf7p0f3efWlUvSvLi7r50mbFPSXJKkjx4vz2y1//64jZ/IPPwhQN2n3cJM9nnm9+ddwkze8IPHzrvEma25T57z7uEmX3uy7fMu4SZPelBT5p3CTO75htXz7uEmRxwwEPnXcLMjtrx0fMuYWafe/UPz7uEmf2bl7573iXMrH//P867hJl943uL92fzT99Q8y5hRjvMu4Btc9C8C5jND+968LxLmNki1nzfG2+Ydwkze9R3bp93CTN7w81fnXcJMzno/rvMuwS4VxBR37WuTnJMVb2mqo7s7puXafPMqrosyeVJHpXkkcu02SnJ+UleskrgmSR/OQSeeyTZs7svGvafleRJK+1f60109xndvam7N+3zgN3Wag4AAAAA9yhCz7tQd38uyaGZhJ+vqqrTp49X1cFJXpzkyd19SJIPJtl5ma5uT/KZJMdOnfvKqrqiqq6YanfrXfwWAAAAAGDhCT3vQsNq6N/u7rcneV2SxyX5ZpL7DU3un0lQefNw+fpTV+iqM7n35sOr6qVJ0t2ndffG7t74rxpPZpTeVFVHDrt+OclFK+0ftqfrAgAAAIDRcE/Pu9Zjkry2qrYk+X6SFyY5PMmHq+qGYSGjy5Ncm+SLST6+UkfDZevPSvKBqrqlu9+8xti/muStVbXr0PfJa+w/c9hvISMAAACANZS5gwtF6HkX6u5zkpyzZPelSd4w1eakFc49amp79+H5tkxd4r6k/UlLXl+R5MeXabfS/vckec9yfQMAAADAIhNRAwAAAACjIvQEAAAAAEZF6AkAAAAAjIp7egIAAADAqiobytzBReLbAgAAAABGRegJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKhYyAgAAAIA1lLmDC8W3BQAAAACMitATAAAAABgVoScAAAAAMCpCTwAAAABgVCxkBAAAAACrqFQ2lLmDi8S3BQAAAACMitATAAAAABgVoScAAAAAMCpCTwAAAABgVCxkBAAAAABrKAsZLRTfFgAAAABwp1TVcVX1D1X1hap66TLHq6r+aDh+VVU9br3nbguhJwAAAACwzapqhyRvSvLUJI9M8uyqeuSSZk9N8rDhcUqSt8xw7syEngAAAADAnfH4JF/o7i92921J/iLJCUvanJDkT3viU0n2rKr91nnuzISeAAAAAMBq9q6qS6cepyw5/qAkX5l6/Y/DvvW0Wc+5M7OQEQAAAACsoXreFczV5u7etMrxWmbf0k9spTbrOXdmQk8AAAAA4M74xyQHTr0+IMn162xzn3WcOzOXtwMAAAAAd8ankzysqg6uqvskeVaSs5e0OTvJrwyruP94kpu7+4Z1njszMz0BAAAAgG3W3bdX1YuSnJNkhyRv6+5rq+oFw/G3JvlQkqcl+UKSbyc5ebVz72xNQk8AAAAA4E7p7g9lEmxO73vr1HYn+Y31nntnCT0BAAAAYC29Zd4VMAP39AQAAAAARkXoCQAAAACMitATAAAAABgVoScAAAAAMCoWMgIAAACAVbWFjBaMmZ4AAAAAwKgIPQEAAACAURF6AgAAAACjIvQEAAAAAEbFQkYAAAAAsJqOhYwWjJmeAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGxUJGAAAAALCqtpDRgjHTEwAAAAAYFaEnAAAAADAqQk8AAAAAYFS2KfSsqlOratd1tLuwqjYN29/alrFmrOvIqrq2qq6oql2293hLxv7/3+s8VNXvVNUx8xofAAAAAO4ptnUho1OTvD3Jt+/CWu6UqtohyXOSvK67/2Te9dydqmqH7j593nUAAAAAjNYWCxktkjVnelbVblX1waq6sqquqaqXJ9k/yQVVdcHQ5i1Vdekwy/IVa/S3d1V9sqqOX+bYLw5jXFlVFw/7TqqqN061+ZuqOmrY/tYww/GSJP8pyTOTnF5Vf15Vu1fV+VV1WVVdXVUnTPXxK1V11TDOnw379qmq91TVp4fHE1eo//Th+DVVdUZV1dTh/72qPjEce/zQ/oFV9dfDeJ+qqkOqakNVXVdVe071+4Wq2neGOq4bavlYkl+sqjOr6hemjr1i6r0/fOo9njfs/7+q6stVtfdq3xcAAAAALJr1zPQ8Lsn13X18klTVHklOTnJ0d28e2pzW3TcOsy3Pr6pDuvuqpR1V1b5Jzk7ysu4+b5mxTk9ybHd/dToQXMVuSa7ZOsuxqn4kyd90919V1Y5JntHdtwzB3qeq6uwkj0xyWpIndvfmqnrg0NcfJvmD7v5YVT04yTlJHrHMmG/s7t8ZxvuzJD+T5ANb6+nun6iqJyV5W5JHJ3lFksu7++lV9VNJ/rS7N1bV+5M8I8mfVNUTklzX3V+rqness44k+W53HzHUctySY5u7+3FV9etJXpzkeUlenuR/dPerhvanLNdpVZ2y9dgPPej+Oe9+t64w/D3TU3Y+dN4lzOQbO99jJkyv2x73eeDaje5h7njrGfMuYWa/8fSfnHcJM6tbt/udTO5yG/7F767u+fbsneddwuxun3cBs3vw7iv91XvP1b//H+ddwszqP7xm3iXM7P0v+Yl5lzCz/R7z6HmXMJPbt9w27xK2yYHzLmBGO39/AWcr7bR4fwe++suXzruEmb3gMU+edwkze+FOh827hJns1Iu6vMofzLsAmMl6/k+7OskxVfWaqjqyu29eps0zq+qyJJcneVQmweJSOyU5P8lLVgg8k+TjSc6squcn2WEdtd2R5D0rHKsk/7Wqrkryt0kelGTfJD+V5K+2BrbdfePQ/pgkb6yqKzIJZu9fVfdbpt+jq+qSqrp66OtRU8feOfR58XD+nkmOSPJnw/7/kWSvITh+V5ITh/OeNbyepY5MnbOc9w7Pn0ly0LB9RJK/GGr5SJKbljuxu8/o7k3dvWmPB96tt0YFAAAAgDttzZme3f25qjo0ydOSvKqqzp0+XlUHZzKT8LDuvqmqzkyy3K/gbs8kgDs2yUXDua9McvwwzsbufsEw6/H4JFdU1cbhvOlwdrrv73b3HSuU/pwk+yQ5tLu/X1XXDedWkl6m/YYkh3f3d5a8v3MyCUsvTfKiJG9Osqm7v1JVv72knqX99jDeUp3kk0l+pKr2SfL0JL+73jq6+3nD7tWmYH5veL4jP/ieF2s6EwAAAABsg/Xc03P/JN/u7rcneV2SxyX5ZpKtsw/vn0n4dvNw+fpTV+iqkzw3ycOr6qVJ0t2nDWHnxmGsh3b3JcPl6pszuUrkuiQbh/tgHpjk8et8b3sk+ach8Dw6yUOG/ednMjN1r2HMrdfpnptJqLn1fW8cajx2qPF5+UHAubmqdk/yC0vGPHE494gkN+uuAZ0AACAASURBVA+zYi/OJIDNcC/Szd19S3d3kvcleX2Sz3b312eoY1t9LJP7nqaqnpLkAXeiLwAAAIB7j95y730soPXc0/MxSV5bVVuSfD/JC5McnuTDVXVDdx9dVZcnuTbJFzO5RH1Z3X1HVT0ryQeq6pbufvOSJq+tqodlMiPx/CRXDvu/lMll9tckuWyd7+3Ph3EuTXJFkr8farh2mGF6UVXdkckl+Scl+c0kbxouh98xk7DyBUvq/0ZV/fFQy3VJPr1kzJuq6hOZBMHPHfb9dib37bwqk9Xuf3Wq/buGPk6a2rdmHXfCK5K8s6pOzGS27Q2ZBNgAAAAAMBrrubz9nEwW05l2aZI3TLU5aYVzj5ra3n14vi2TS9yXa/9zK5TxnBXa777k9UlT25szCWeXO++sJGct2bc5P7jH5oq6+2VJXrbM/qNWaH9jkhNWOHZpllxyPkMdBy15fdJyx4YxttZ2cyYLRd1eVYdnshjV9wIAAAAAI7KemZ6Mx4OTvLuqNiS5Lcnz51wPAAAAANzlhJ73It39+SSPnXcdAAAAALA9CT0BAAAAYDXdC7ugz73Vmqu3AwAAAAAsEqEnAAAAADAqQk8AAAAAYFSEngAAAADAqFjICAAAAADWYiGjhWKmJwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKgIPQEAAACAUbGQEQAAAACsqpMtFjJaJGZ6AgAAAACjIvQEAAAAAEZF6AkAAAAAjIrQEwAAAAAYFQsZAQAAAMBa2kJGi8RMTwAAAABgVISeAAAAAMCoCD0BAAAAgFERegIAAAAAo2IhIwAAAABYTcdCRgvGTE8AAAAAYFSEngAAAADAqAg9AQAAAIBREXoCAAAAAKNiISMAAAAAWFVbyGjBmOkJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKkJPAAAAAGBULGQEAAAAAGvovmPeJTADMz0BAAAAgFERegIAAAAAoyL0BAAAAABGRegJAAAAAIyKhYwAAAAAYDXdyZYt866CGZjpCQAAAACMitATAAAAABgVoScAAAAAMCpCTwAAAABgVCxkBAAAAABraQsZLRIzPQEAAACAURF6AgAAAACjIvQEAAAAAEZF6AkAAAAAjMo2h55VdWpV7bqOdhdW1aZh+1vbOt4MdR1ZVddW1RVVtct2GuO6qtp7hvYnVdUbt0ctU2P8TlUdsz3HAAAAALh36slCRvfWxwK6MzM9T02yZuh5d6qqHZI8J8nruntjd39n3jXdHapqh+4+vbv/dt61AAAAAMC87bieRlW1W5J3JzkgyQ5J/jLJ/kkuqKrN3X10Vb0lyWFJdknyV9398lX62zvJB5L8bnd/cMmxX0zy8iR3JLm5u59UVScl2dTdLxra/E0mweaFw+zR1yc5dujzmUmOHWY9/h9J3p/kAUl2SvKy7n7/0MevJHlxkk5yVXf/clXtk+StSR48lHNqd398hbfxW1V19LD9S939hfWcX1UPSfK2JPsk+eckJyf5apLPJ3lokj2S3JjkqO6+uKo+muTk7v7Ckn6uG/p5SpI3VtVxSf6mu/9qOHZWkp8d3vcvdvffD/W9I8leST6d5Lgkh3b35hXeY+67w0556B77rXT4nmnL7fOuYCZ75j7zLmFm377j2/MuYWY7P/ageZcws9t+6OB5lzCz+95227xLmNleO+477xJm8rUtX8++W+437zJms9PO865gZl+99fPzLmFm3/jeLfMuYWbvf8lPzLuEmZ3we5+Ydwkzu/kPnzjvEmZy6+03z7uEe4cFnLXz/Vq8mvfbbfHuKLfThsX798l3F+zfJzttWbzPGBbRukLPTMKx67v7+CSpqj0yCeuOngrMTuvuG4fZludX1SHdfdXSjqpq3yRnZxJAnrfMWKcnOba7v1pVe66jtt2SXNPdpw/9/0h+EP7tmOQZ3X3LELR+qqrOTvLIJKcleWJ3b66qBw59/WGSP+juj1XVg5Ock+QRK4x7S3c/fghP/1uSn1nn+W9M8qfdfVZVPTfJH3X306vqc0NdByf5TJIjq+qSJAcsDTynfLe7jxje93FLjm3u7sdV1a9nEu4+L5Mw+X9096uG9qcs12lVnbL12P4HPmCFoQGYh4ULPAEAAOZgvb92ujrJMVX1mqo6sruX+/XrM6vqsiSXJ3lUJgHeUjslOT/JS1YIPJPk40nOrKrnZzKrdC13JHnPCscqyX+tqquS/G2SByXZN8lPZTIbdXOSdPeNQ/tjMpk1eUUmwez9q2qlf12+c+r58BnOPzyT2ZZJ8mdJjhi2P5rkScPjVcP+wzKZkbmSd61y7L3D82eSHDRsH5HkL5Kkuz+S5KblTuzuM7p7U3dveuBeu60yBAAAAADc86xrpmd3f66qDk3ytCSvqqpzp49X1cGZzCY8rLtvqqozkyx3LdvtmYRwxya5aDj3lUmOH8bZ2N0vqKonDPuuqKqNw3nTAe1039/t7jtWKP05mVxGfmh3f3+47HvnTMLQXqb9hiSHL70XaFWdk0lYeml3P2/rxzLVpNc4f4Xy/sW5H03ygkxuG3B6kt9KclSSi1ep4dZV+v3e8HxHfvA9r1oIAAAAACtYwFuD3Juta6ZnVe2f5Nvd/fYkr0vyuCTfTLJ1FuP9Mwngbh4uX3/qCl11kucmeXhVvTRJuvu0IezcOIz10O6+ZLhcfXOSA5Ncl2RjVW2oqgOTPH6d72+PJP80BJ5HJ3nIsP/8TGam7jWMufXy9nOTvGjqfW8cajx2qPF5P+g6J049f3K185f4RJJnDdvPSfKxYfuSJD+RZEt3fzfJFZnck/Sjq9Qwq49lcs/TVNVTMrnXKQAAAACMynrv6fmYJK+tqi1Jvp/khZlcpv3hqrphWMjo8iTXJvliJpeoL6u776iqZyX5QFXd0t1vXtLktVX1sExmJZ6f5Mph/5cyucz+miSXrbPuPx/GuTSTEPHvhxquHWaYXlRVd2RySf5JSX4zyZuGy+F3zGSW5QtW6Pu+wz03NyR59rBvPef/ZpK3VdVv5QcLGaW7v1dVX0nyqaHdR4d+r17ne12PVyR5Z1WdmMlM2xsyCa8BAAAAYDTWe3n7OZksyjPt0iRvmGpz0grnHjW1vfvwfFsml7gv1/7nVijjOSu0333J65OmtjfnB/fbXHreWZmscJ4l7U9crv2SdgcNm69Yz/ndfWaSM4ft6zK5p+hy/R45tf2O/ODen6vVsPX1Scsd6+5LM7lMPkluzmSRqNur6vBMFqL6XgAAAABgRNY705NxeHCSd1fVhiS3JXn+nOsBAAAAgLuc0PNepLs/n+Sx864DAAAAYLG0hYwWzLoWMgIAAAAAWBRCTwAAAABgVISeAAAAAMCoCD0BAAAAgFGxkBEAAAAArKaTbLGQ0SIx0xMAAAAAGBWhJwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKhYyAgAAAAA1tIWMlokZnoCAAAAAKMi9AQAAAAARkXoCQAAAACMitATAAAAABgVCxkBAAAAwKraQkYLxkxPAAAAAGBUhJ4AAAAAwKgIPQEAAACAURF6AgAAAACjYiEjAAAAAFiLhYwWipmeAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGxUJGAAAAALCa7mSLhYwWiZmeAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGxUJGAAAAALCWtpDRIjHTEwAAAAAYFaEnAAAAADAqQk8AAAAAYFSEngAAAADAqFjICAAAAADWYiGjhWKmJwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKgIPQEAAACAUbGQEQAAAACspjvZYiGjRWKmJwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKgIPQEAAACAUbnbQs+qOrWqdl1HuwuratOw/a3tUMdd3uf27HeG8T9UVXvOswYAAACA0drS997HAro7Z3qemmTN0POepKp2mHcNa6mJDd39tO7+xrzrAQAAAIB52y6hZ1XtVlUfrKorq+qaqnp5kv2TXFBVFwxt3lJVl1bVtVX1ijX627uqPllVxy9zbN+qet8w1pVV9RPD/n8/jH1NVZ26zHlVVa8djl9dVScO+4+qqguq6h1Jrl7mvL+uqs8MdZ+y5NjvV9VlVXV+Ve0z7NtYVZ+qqquGOh9QVY+oqr+bOu+gqrpq2D60qi4axjinqvZbpoaDquqzVfXmJJclObCqrhs+p63H/nio8dyq2mU477Chjk9ufe+rfe4AAAAAsIh23E79Hpfk+u4+Pkmqao8kJyc5urs3D21O6+4bh9mU51fVId191dKOqmrfJGcneVl3n7fMWH+U5KLufsbQ1+5Vdegw3hOSVJJLquqi7r586ryfS7IxyY8l2TvJp6vq4uHY45M8uru/tMx4zx3q3mU45z3d/fUkuyW5rLv/Q1WdnuTlSV6U5E+T/NvuvqiqfifJy7v71Kq6T1X9b939xSQnJnl3Ve2U5A1JTujufx6C2Fcmee4ydfxokpO7+9eHz2n62MOSPLu7n19V707y80nenuRPkpzS3Z+oqlcv0+e/sqF2yC477r6epvcc91moCcULaddbb5x3CTPrBx847xJmdp+vfnbeJczsjd/4/LxLmNlvfH+veZcwk0W8sOTKhyzen8v/9O2b513CzH76hlq70T3Mfo959LxLmNnNf/jEeZcwsz3+3WvnXcJMjj/uYfMuYZv8zQnL/ch+z/W2L50z7xJmdtxDHjfvEma2oRbvz+bz//GieZcwsycf8JPzLmEm39vR8ipwd9he/6ddneSYqnpNVR3Z3cv9y+GZVXVZksuTPCrJI5dps1OS85O8ZIXAM0l+KslbkqS77xjGOiLJ+7r71u7+VpL3JjlyyXlHJHnncM7XklyU5LDh2N+tEHgmyW9W1ZVJPpXkwEwCxiTZkuRdw/bbkxwxhL17dvfWvzXOSvKkYfvdSZ45bJ84nPujSR6d5LyquiLJy5IcsEIdX+7uT61w7EvdfcWw/ZkkBw33+7xfd39i2P+OFc5NVZ0yzMK99Oubv7lSMwAAAAC4R9ouMz27+3PDbMunJXlVVZ07fbyqDk7y4iSHdfdNVXVmkp2X6er2TEK7YzMJJVNVr0xy/DDOxhVKWM+v01Zrc+sw1oFJPjDse2uSv09yTJLDu/vbVXXhCnUna0/GeVeSv6yq9ybp7v58VT0mybXdffi/KPRf1/GRrTWu4HtT23ck2SXr+0wmhXefkeSMJPmxxx20iJOKAAAAAO5aW7bMuwJmsL3u6bl/km9399uTvC7J45J8M8n9hib3zyS0u3m4fP2pK3TVmVza/fCqemmSdPdp3b1xKvA8P8kLh3F3qKr7J7k4ydOrateq2i3JM5J8dEnfFyc5cThnn0xmYP7ddIPu/srWsbr7rUn2SHLTEHg+PMmPTzXfkOQXhu1fSvKxYdbpTVW1dZbpL2cIb7v7f2USSP7n/GCG6D8k2aeqDh/ez05V9ahl6phZd9+U5JtVtbXmZ21LPwAAAABwT7e97un5mCSvraotSb6fSSh5eJIPV9UN3X10VV2e5NokX0zy8ZU66u47qupZST5QVbd095uXNPl3Sc6oql/LJER8YXd/cpg9ujXE/O9L7ueZJO8baroyk3D1Jd39/w5h5ko+kuQFw6JD/5DJJe5b3ZrkUVX1mSQ3Z3LJepL8apK3VtWuw3s9eeqcdyV5bZKDh/d6W1X9QpI/Gi6N3zHJf8vkc7or/FqSP66qW5NcONQJAAAAAKOyvS5vPyfJ0jtjX5rJIj1b25y0wrlHTW3vPjzflskl7su1/1qSE5bZ//okr19m/9Y+O8lvDY/p4xdmEgguN9b3ssKs1K39ZjJzc3r/FfmXM0Knj70uk5mwS9s/abn2U22uy+Ten9P7Dho2N08fG8bY6truPiRJhpmzl642DgAAAAAsou0105N7puOr6j9l8r1/OclJ8y0HAAAAAO56Qs97ke5+V35w/1AAAAAA1qPbQkYLZrssZAQAAAAAMC9CTwAAAABgVISeAAAAAMCoCD0BAAAAgFGxkBEAAAAArGVLz7sCZmCmJwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKgIPQEAAACAUbGQEQAAAACsppNs2TLvKpiBmZ4AAAAAwHZRVQ+sqvOq6vPD8wOWaXNgVV1QVZ+tqmur6t9NHfvtqvpqVV0xPJ62nnGFngAAAADA9vLSJOd398OSnD+8Xur2JP+hux+R5MeT/EZVPXLq+B9098bh8aH1DCr0BAAAAAC2lxOSnDVsn5Xk6UsbdPcN3X3ZsP3NJJ9N8qA7M6jQEwAAAADYXvbt7huSSbiZ5IdWa1xVByV5bJJLpna/qKquqqq3LXd5/HIsZAQAAAAAq+p7+0JGe1fVpVOvz+juM7a+qKq/TfLDy5x32iyDVNXuSd6T5NTuvmXY/ZYk/yWT5aT+S5LfT/LctfoSegIAAAAAq9nc3ZtWOtjdx6x0rKq+VlX7dfcNVbVfkn9aod1OmQSef97d753q+2tTbf44yd+sp2CXtwMAAAAA28vZSX512P7VJO9f2qCqKsn/neSz3f36Jcf2m3r5jCTXrGdQoScAAAAAsL28OslPV9Xnk/z08DpVtX9VbV2J/YlJfjnJT1XVFcPjacOx36uqq6vqqiRHJ/k/1zOoy9sBAAAAgO2iu7+e5MnL7L8+ydOG7Y8lqRXO/+VtGVfoCQAAAABr2dLzroAZuLwdAAAAABgVoScAAAAAMCpCTwAAAABgVISeAAAAAMCoWMgIAAAAAFbTSbZsmXcVzMBMTwAAAABgVISeAAAAAMCoCD0BAAAAgFERegIAAAAAo2IhIwAAAABYVSdbet5FMAMzPQEAAACAURF6AgAAAACjIvQEAAAAAEZF6AkAAAAAjIqFjAAAAABgNZ1ky5Z5V8EMzPQEAAAAAEZF6AkAAAAAjIrQEwAAAAAYFaEnAAAAADAqFjICAAAAgLVYyGihmOkJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKkJPAAAAAGBULGQEAAAAAKvqdPe8i2AGd+tMz6o6tap2XUe7C6tq07D9re1Qxzb3WVUHVdU1M55zZlX9wraOuc4xPlRVe27PMQAAAABgEdzdl7efmmTN0POepKp2mHcNq6mJDd39tO7+xrzrAQAAAIB5226hZ1XtVlUfrKorq+qaqnp5kv2TXFBVFwxt3lJVl1bVtVX1ijX627uqPllVxy9zbN+qet8w1pVV9RPD/n8/jH1NVZ26zHlVVa8djl9dVScO+4+qqguq6h1Jrl6mnB2r6qyquqqq/mrr7NWqOrSqLqqqz1TVOVW13zJjPrmqLh/Ge1tV3beqHl9V7x2On1BV36mq+1TVzlX1xWX6OKiqPltVb05yWZIDq+q64TPaeuyPh8/13KraZTjvsKHmT25936t95gAAAACwiLbnPT2PS3J9dx+fJFW1R5KTkxzd3ZuHNqd1943DbMrzq+qQ7r5qaUdVtW+Ss5O8rLvPW2asP0pyUXc/Y+hr96o6dBjvCUkqySVVdVF3Xz513s8l2Zjkx5LsneTTVXXxcOzxSR7d3V9aZrwfTfJr3f3xqnpbkl+vqj9M8oYkJ3T3Pw8B6iuTPHfqfeyc5MwkT+7uz1XVnyZ5YZI3Jnns0OzIJNckOSyT7+eSZcbfWsPJ3f3rQ9/Txx6W5Nnd/fyqeneSn0/y9iR/kuSU7v5EVb16hX5TVackOSVJHnTgA1dqds+1YcFuVdtb5l3B7GoB10C74/Z5VzC7Bax5pwX8TyM732feFcxmx3v0BQjL2vAv/45aCLdvWcA/m7N4/23cvuW2eZcws1tvv3neJczs+OMeNu8SZvLBj3x+3iVsmxPmXcBsdtqweH8271AL9nP+gtp1x/vOu4SZ7bBo/wYE7hbb85+nVyc5pqpeU1VHdvdyPyE+s6ouS3J5kkcleeQybXZKcn6Sl6wQeCbJTyV5S5J09x3DWEckeV9339rd30ry3kwCxWlHJHnncM7XklyUSdiYJH+3QuCZJF/p7o8P228f+vnRJI9Ocl5VXZHkZUkOWHLejyb5Und/bnh9VpIndfftSb5QVY/IJGx9fZInDfV+dIUavtzdn1rh2Je6+4ph+zNJDhru93m/7v7EsP8dK5yb7j6juzd196a99r7fSs0AAAAA7h06yZYt997HAtpuoecQ7B2aSfj5qqo6ffp4VR2c5MWZzHo8JMkHk+y8TFe3ZxLcHTt17iur6oohXFzJen51uVqbW4exDtw6VlW9YDi2dLmuHvq6trs3Do/HdPdTZhjvo0memuT7Sf4/9u483LKyvBP272EQgSKOqBg1ZZywRYVQGjHEqGjaqT+1g3EgRtAO0cRONCb59FMjJvalpklHk4gK3QLOOHaM7UyLOEUtEFESkW6BToRWcGISlHOe74+zSrfHc06dDXXYtRf3fV3n2muv9a53PavgKopfvWs9H8tSkHpoktNXqeHKNea6ZmJ7IUsrRufvr3IBAAAA4DrYyHd63j7JVd395iTHJvmlJJcn2bZ08OeyFNx9f3h8/ZGrTNVZekR8/6p6fpJ09wu3hYvDmFOz9Jh4qmrXqvq5JKcneVxV7VVVeyd5fH521eTpSZ44nLNvllZXfv6nLt79LxNB5uuG3XeqqkOG7Scn+VSSc5Psu21/Ve1eVfdadr2vZmnV5V2H70/N0urSbbU8J8lnu/uSJLdKsn+WgtSVaphKd383yeVV9YBh15OuyzwAAAAAsLPbyBdf3DvJf66qxSytXnxWkkOSfLCqLu7uh1TVF5Ock+TrST692kTdvVBVT0ryD1V1WXcft2zIHyY5vqqekaWVjc/q7s9W1Un5SYj5X5e9zzNJ3jvU9KUshat/2t3/t6r23869/XOSp1XV65Ocl+S13f3Dqjo8yd8M7y/dLcmrhvvbdh9XV9VRSd5ZVbsl+UKSbSHm55LcNkvhZ5KcneRb3b18Ven18YwkJ1TVlUlOSzJ/L6UCAAAAgO3YsNCzuz+c5MPLdm/NUrOfbWOOXOXcB09sbxo+f5iJR9yXjf9mVnh1eHf/lyy9H3P5/m1zdpI/GX4mj5+WpVBwpWtdkJXfPZrhPZoPWmH/kRPbp+YnTYsmx/wgyR4T349e6RoTNRywbN/mYfPSyWPdfezEsHOGVwlkWDW7dbVrAAAAAMC80uLsxuXRVfWCLP1zvzDJkbMtBwAAAGBOzGlDnxsroeeNSHefkuSUWdcBAAAAABtpwxoZAQAAAADMgtATAAAAABgVoScAAAAAMCre6QkAAAAAa+pksWddBFOw0hMAAAAAGBWhJwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKhoZAQAAAAAa+kki4uzroIpWOkJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKkJPAAAAAGBUNDICAAAAgO3RyGiuWOkJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKkJPAAAAAGBUNDICAAAAgLV0J4s96yqYgpWeAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGRSMjAAAAANiexcVZV8AUrPQEAAAAAEZF6AkAAAAAjIrQEwAAAAAYFaEnAAAAADAqGhkBAAAAwPZoZDRXrPQEAAAAAEZF6AkAAAAAjIrQEwAAAAAYFaEnAAAAADAqGhkBAAAAwFq6k8WedRVMwUpPAAAAAGBUhJ4AAAAAwKgIPQEAAACAURF6AgAAAACjopERAAAAAGzP4uKsK2AKVnoCAAAAAKMi9AQAAAAARkXoCQAAAACMitATAAAAABgVjYwAAAAAYC0djYzmjJWeAAAAAMCoCD13ElX1meFzc1V9ZZUxR1bV7W/YygAAAABgvgg9dxLd/cB1DDsyidATAAAAANYg9NwgVfXKqvq9ie/HVNXzqupPquoLVXV2Vb104vgV25nv8CRbkrylqs6qqkdX1Xsnjj+8qt6zba6q+quqOrOqTq2qfYf9d6mqD1XVGVX1yaraf0ffNwAAAADMmtBz47w9yRMnvv9mkkuS3C3J/ZMcmOTgqnrQeibr7ncl2ZrkiO4+MMkHktxzW6CZ5KgkJw7beyc5s7t/Kcknkrxk2H98kv/Y3Qcn+eMkx610rao6uqq2VtXWb196+bpuFoAbyLULs64AAABuhDpZvBH/zKHqns/C50FV/XOSw5Lsm6WA8bNJDk/yvWHIpiQv7+7/VlVXdPemqtqc5P3dfcAK852W5I+7e+vw/YVJrspS2PnFJHfr7muraiHJHsP2LyZ5T5JDsxS6njsx5R7dfc+17mHLli29devW63T/AAAAwDhU1RndvWXWdczKlrveuj9/7GNnXcbM7Pr4N8zdP//dZl3AyL0rSyHn7bK08nNzlkLO16/n5Ko6MclBSS7q7ketMOTEJP+Q5Ook7+zua1eZqrO0qvd7wypRAAAAABgtj7dvrLcneVKWgs93JflwkqdX1aYkqaqfr6rbrHZydx/V3QdOBJ6XJ9ln4vhFSS5K8qIkJ02custwzSR5SpJPdfdlSc6vqicM166quu/1v0UAAAAA2LlY6bmBuvucqtonyTe6++IkF1fVPZN8tqqS5Iokv5XkW+uc8qQkr6uqHyQ5pLt/kOQtSfbt7n+aGHdlkntV1RlJvp+fvFv0iCSvraoXJdk9S6Hsl67PPQIAAADAzkboucG6+97Lvr86yatXGLdp+Lwgyc+8z3M49u4k7162+9AkJ6ww9sVJXrxs3/lJHrH+6gEAAABIkiwuzroCpiD0nGPDSs4rkzxv1rUAAAAAwM5C6DnHuvvgVfZvuqFrAQAAAICdhUZGAAAAAMCoCD0BAAAAgFHxeDsAAAAArKWTXuhZV8EUrPQEAAAAAEZF6AkAAAAAjIrQEwAAAAAYFaEnAAAAADAqGhkBAAAAwPYsamQ0T6z0BAAAAABGRegJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKhoZAQAAAMBaupMFjYzmiZWeAAAAAMCoCD0BaCeQpAAAIABJREFUAAAAgFERegIAAAAAoyL0BAAAAABGRSMjAAAAAFhDJ+lFjYzmiZWeAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGRSMjAAAAAFhLJ1nQyGieWOkJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKkJPAAAAAGBUNDICAAAAgLV0koXFWVfBFKz0BAAAAABGRegJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKhoZAQAAAMCaOr3Ysy6CKVjpCQAAAACMitATAAAAABgVoScAAAAAMCpCTwAAAABgVDQyAgAAAIC1dJIFjYzmiZWeAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGRSMjAAAAANieRY2M5omVngAAAADAqAg9AQAAAIBREXrOWFV9ZvjcXFVf2QHz7ZB5AAAAAGBeCT1nrLsfOMvrV5X3ugIAAAAwKgKvHayqXpnkwu4+bvh+TJLLsxQw/2aSPZK8t7tfMhy/ors3bWfOuyZ5XZJ9kywkeUKSryf5yySPTNJJXtbdpyw776ZJXptkS5Jrk/xRd3+8qo5M8ugkN02yd5KHXu8bBwAAABirTnpBI6N5IvTc8d6e5FVJjhu+/2aSVyQ5NMn9k1SS91XVg7r79HXO+ZYkr+ju9w5B5i5J/n2SA5PcN8mtk3yhqpbP9/tJ0t33rqr9k3ykqu4+HDskyX26+zvLL1ZVRyc5OknudIdbpb990jrL3DlcfNNrZ13CVPbb+y6zLmF6l/zvWVcwvZrDhe27zN9v0bc5+R2zLmFqlzz792ddwnSuuWrWFUztM5efPesSpnb7TfvOuoSp3W6vO8+6hKnd9EeLsy5hej1/Nb/h/A/PuoSp7L5LzbqE6+Sp+79m1iVMpZ71gFmXMLV+1YtnXcLU+opLZl3C1D54+TmzLmFq+9/yjrMuYSq7euASbhBzmALs3Lr7i0luU1W3r6r7Jvlukvsk+fUkX0xyZpL9k9xtPfNV1T5Jfr673zvMf3V3X5WlEPVt3b3Q3d9M8okk91t2+qFJ3jSc99UkFybZFnp+dKXAcxh7fHdv6e4t+95qn/XeOgAAAADsFPz1wsZ4V5LDk9wuSys/Nyd5eXe/fj0nV9WJSQ5KclGSJ602bD1TrXHsyvXUAgAAAADzxkrPjfH2LIWVh2cpAP1wkqdX1aYkqaqfr6rbrHZydx/V3Qd296O6+7Ik/1pVjxvO3aOq9kpyepInVtWuVbVvkgcl+fyyqU5PcsRw3t2T3CnJuTvyRgEAAABgZ2Ol5wbo7nOGx9K/0d0XJ7m4qu6Z5LNVlSRXJPmtJN9a55RPTfL6qvrzJD/KUiOj92bpvZxfylIjoz/t7v9bVZsnzjsuyeuq6stZamR0ZHdfM9QAAAAAwLp0sjh/7/a+MRN6bpDuvvey769O8uoVxm0aPi9IcsAqc52XlTus/8nwMzn2x/N099VJjlxhvpOSnLSdWwAAAACAueTxdgAAAABgVISeAAAAAMCoCD0BAAAAgFHxTk8AAAAAWEsnWehZV8EUrPQEAAAAAEZF6AkAAAAAbIiqumVVfbSqzhs+b7HKuAuq6stVdVZVbZ32/OWEngAAAADARnl+klO7+25JTh2+r+Yh3X1gd2+5juf/mNATAAAAANgoj01y8rB9cpLH3RDna2QEAAAAANvRizfqRka3nnzkPMnx3X38Os+9bXdfnCTdfXFV3WaVcZ3kI1XVSV4/Mf96z/8pQk8AAAAAYC2XLnvk/KdU1ceS3G6FQy+c4hq/0t0XDaHmR6vqq919+rSFbiP0BAAAAACus+5+2GrHquqbVbXfsEpzvyTfWmWOi4bPb1XVe5PcP8npSdZ1/nLe6QkAAAAAbJT3JXnasP20JH+/fEBV7V1V+2zbTvLrSb6y3vNXIvQEAAAAADbKK5I8vKrOS/Lw4Xuq6vZV9YFhzG2TfKqqvpTk80n+R3d/aK3zt8fj7QAAAACwlk6ycKNuZHSddfe3kxy2wv6Lkjxq2P56kvtOc/72WOkJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKkJPAAAAAGBUNDICAAAAgDW1RkZzxkpPAAAAAGBUhJ4AAAAAwKgIPQEAAACAURF6AgAAAACjopERAAAAAKylk17UyGieWOkJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKjo3g4AAAAA27OwOOsKmIKVngAAAADAqAg9AQAAAIBREXoCAAAAAKMi9AQAAAAARkUjIwAAAABYQ3fSiz3rMpiClZ4AAAAAwKgIPQEAAACAURF6AgAAAACjIvQEAAAAAEZFIyMAAAAAWFMnCxoZzRMrPQEAAACAURF6AgAAAACjIvQEAAAAAEZF6AkAAAAAjIpGRgAAAACwlk6yqJHRPLHSEwAAAAAYFaEnAAAAADAqQs/tqKoPVNXNZ13HWqrqyKr6u1nXAQAAAAA7A+/0XENVVZLHdPfirGvZSFW1a3cvzLoOAAAAANgRhJ7LVNXmJB9M8vEkhyQ5sKr2TbJp2P+pJA9M8o0kj+3uH1TV/ZL8tyRXDscf2d0HrDD3aUk+l+QhSW6e5Bnd/cmqOjLJlu5+9jDu/UmO7e7TquqKJK9J8rAk303y/yX5yyR3SvKc7n7fMP0dq+pDSe6c5K3d/dJhrt9K8gdJbjJc+/e6e2GY978k+bdJnjfU/bN6MfnR1VP+Ks7WFbteMesSplJz+B7kf9r127MuYWr732LLrEuY2i6XXzrrEqZ24X/83VmXMLWvXv6VWZcwtXtcvuusS5jKnW/9C7MuYWpfvvTcWZcwtdvtdedZlzC93W866wqm9qOav78Lf8Qv/NKsS5jKruV/UW4I/aoXz7qEqdVz/mLWJUxt8bUvn3UJU3vkdy6edQlTe/NF8/Xf7YP2veOsS+A66oU5/B/4GzGPt6/sHkne2N0HJblwYv/dkrymu++V5HtJfmPYf2KSZ3b3IUm2t2Jyt+6+f5LnJHnJOmrZO8lp3X1wksuTvCzJw5M8PsmfT4y7f5IjkhyY5AlVtaWq7pnkiUl+pbsPHGo7YmLer3T3L3f3TwWeVXV0VW2tqq2XfHu+AkSAsZu3wBMAAGAW/DXqyi7s7n9cYf/53X3WsH1Gks3D+z736e7PDPvfmuQxa8z9nsnz11HLD5N8aNj+cpJruvtHVfXlZed/tLu/nSRV9Z4khya5NsnBSb6w9KR+9kzyrWH8QpJ3r3TB7j4+yfFJsuW+v+CvMQAAAACYK0LPlV25yv5rJrYXshQi1mqTVNWJSQ5KclF3P2rZHAv5ya//tfnpVbeTz3r9qLu3BY+L287v7sWqn3r2Z3k42UNtJ3f3C1Yo72rv8QQAAABgjDzefj1193eTXF5VDxh2PWni2FHdfeBE4LmaC7L07tBdquqOWXpUfVoPr6pbVtWeSR6X5NNJTk1yeFXdJkmG4/P3YjUAAAAAmIKVnjvGM5KcUFVXJjktyfenPP/TSc7P0uPrX0ly5nWo4VNJ3pTkrllqZLQ1SarqRUk+UlW7JPlRkt/PT7+nFAAAAIC1dJJFbwCcJ0LPZbr7giQHTHzfPGxeumz/sROnndPd90mSqnp+kq2rzP3gie1LM7yTc3h8/YhVztk0sX3MSse6+6QkJ61y/ilJTllrXgAAAAAYE6HnjvHoqnpBln49L0xy5GzLAQAAAIAbL6HnDrDaakoAAAAA4IankREAAAAAMCpWegIAAADAmjpZWJx1EUzBSk8AAAAAYFSEngAAAADAqAg9AQAAAIBREXoCAAAAAKOikREAAAAArKWTXuxZV8EUrPQEAAAAAEZF6AkAAAAAjIrQEwAAAAAYFaEnAAAAADAqGhkBAAAAwPYsaGQ0T6z0BAAAAABGRegJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKhoZAQAAAMBaOulFjYzmiZWeAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGRSMjAAAAANiOXtDIaJ5Y6QkAAAAAjIrQEwAAAAAYFaEnAAAAADAqQk8AAAAAYFQ0MgIAAACANXR3elEjo3lipScAAAAAMCpCTwAAAABgVISeAAAAAMCoCD0BAAAAgFHRyAgAAAAAtmNxQSOjeWKlJwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKgIPQEAAACAUdHICAAAAADW0kkvamQ0T6z0BAAAAABGRegJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKhoZAQAAAMAaOkkvLs66DKZgpScAAAAAMCpCTwAAAABgVISeN4Cq+kBV3XwHzXVkVd1+R8wFAAAAAGMk9NxgVVVJHtPd39tBUx6ZROgJAAAAAKsQem6AqtpcVf9cVcclOTPJQlXdemL/CVV1TlV9pKr2HM65X1WdXVWfrar/XFVfWWHew5NsSfKWqjqrqh5dVe+dOP7wqnrPsH1FVf1VVZ1ZVadW1b7D/rtU1Yeq6oyq+mRV7X9D/JoAAAAAzK3u9MKN92ce6d6+ce6R5Kju/r2qumBi/92SPLm7f6eq3pHkN5K8OcmJSY7u7s9U1StWmrC731VVz07yx929dVhF+ldVtW93X5LkqGGeJNk7yZnd/byq+rMkL0ny7CTHJ3lmd59XVb+c5LgkD13tJv71mqvyJ+edcd1/FWbg2AN/fdYlTOeHV826gqnd5Wb3mXUJU/vWE1446xKmdrsTnzfrEqa219U/nHUJU9t/0wGzLmEql+09f79n7Dd//1rk23vfYtYlTG2P71w86xKm9ooLt866hKntt/f8rRnYpWrWJdwoPHX/I2ddwlT6iktmXcLUFl/78lmXMLVdnvWCWZcwtUv++tmzLmFqT7327rMuYTp77ZC33wHbMX9/apsfF3b3P66w//zuPmvYPiPJ5uF9n/t092eG/W9dzwW6u5O8KclvDXMckuSDw+HFJKcM229OcmhVbUrywCTvrKqzkrw+yX7L562qo6tqa1Vtvep7V6+nFAAAAADYaVjpuXGuXGX/NRPbC0n2TLLqX79X1YlJDkpyUXc/aoUhJyb5hyRXJ3lnd1+7ylSdpZD7e9194FqFd/fxWVoRmtvtv+98rmEGAAAA4EbLSs+dQHd/N8nlVfWAYdeTJo4d1d0HTgSelyfZZ+L4RUkuSvKiJCdNTLtLksOH7ack+VR3X5bk/Kp6QrLUZKmq7rsBtwQAAAAAM2Ol587jGUlOqKork5yW5PurjDspyeuq6gdJDunuHyR5S5J9u/ufJsZdmeReVXXGMNcTh/1HJHltVb0oye5J3p7kSzv4XgAAAABGpRc9DDtPhJ4boLsvSHLAxPfNw+aly/YfO3HaOd19nySpqucnWfHN/t397iTvXrb70CQnrDD2xUlevGzf+Ukesb47AQAAAID5I/TceTy6ql6QpX8mFyY5cj0nDSs5r0wyf22eAQAAAGADCD13Et19Sn7SbX2a8w5eZf+m610UAAAAAMwhjYwAAAAAgFGx0hMAAAAA1tJJL2hkNE+s9AQAAAAARkXoCQAAAACMitATAAAAABgVoScAAAAAMCoaGQEAAADAdvSiRkbzxEpPAAAAAGBUhJ4AAAAAwKgIPQEAAACAURF6AgAAAACjopERAAAAAKyhO1nUyGiuWOkJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKkJPAAAAAGBUNDICAAAAgDV1ekEjo3lipScAAAAAMCpCTwAAAABgVISeAAAAAMCoCD0BAAAAgFHRyAgAAAAA1tJJL2pkNE+s9AQAAAAARkXoCQAAAACMitATAAAAABgVoScAAAAAMCoaGQEAAADAdmhkNF+s9AQAAAAARkXoCQAAAACMitATAAAAABgVoScAAAAAMCoaGQEAAADAGrqTXtDIaJ5Y6QkAAAAAjIrQEwAAAAAYFaEnAAAAADAqQk8AAAAAYFQ0MgIAAACANXV6cXHWRTAFKz0BAAAAgFERegIAAAAAoyL0BAAAAABGRegJAAAAAIyKRkYAAAAAsJZOeqFnXQVTsNITAAAAABgVoScAAAAAMCpCTwAAAABgVISeG6yqnllVv72dMQdW1aN20PUeXFXv3xFzAQAAAMA80shoQlXt1t3X7uD5XreOoQcm2ZLkAzvq2uu1o+8ZAAAAYIx6USOjeTLXoWdV7Z3kHUnukGTXJH+R5JVJTknykGHYU7r7f1XVLyR5Q5J9k1yS5Kju/j9VdVKS7yQ5KMmZVXVcktcM465K8jvd/dUVrn1FktcP1/lukid19yVVdVqSzyT5lSTvq6p9klzR3ccOxz43nHPzJM8Yvv95kj2r6tAkL+/uU5Zd635JXp1k7yTXJDksyY+SvDZLYem1Sf6ouz++7LxbDvf8i8O9HN3dZ1fVMUlun2RzkkuTPGUdv9wAAAAAMBfmOvRM8ogkF3X3o5Okqm6WpdDzsu6+//BY+auSPCbJ3yV5Y3efXFVPT/I3SR43zHP3JA/r7oWqOjXJM7v7vKr65STHJXnoCtfeO8mZ3f28qvqzJC9J8uzh2M27+9eGmo5Zdt5uQ22PSvKS7n7YcP6W7n72srGpqptkKcR9Ynd/oap+LskPkvxhknT3vatq/yQfqaq7Lzv9pUm+2N2Pq6qHJnljllaVJsnBSQ7t7h+scM2jkxydJHe6021y7CGHr3D7O6/+/sWzLmEqF+/y/VmXMLVdF+bvt47d3/yHsy5hetfM3yLs//7tz8+6hKk97nYPmnUJU9nzTe+cdQlTO/lBt5x1CVPbd8+9Z13C1O71g/n7PeOZ9z5s1iVMbfddbjLrEqZ26r9+YtYlTGWv3faYdQk3Ch+8/JxZlzC1R35nvv6cnySX/PXP/C/eTm/f5/7drEuY2sJx/2nWJUxllyu/N+sS4EZh3t/p+eUkD6uqV1bVr3b3tvTobROfhwzbhyR567D9piSHTszzziHw3JTkgUneWVVnZWkl536rXHsxS2Fkkrx52Xyn/OzwH3vP8HlGllZabs89klzc3V9Iku6+bHgc/dDhPjKsRL0wS+HtpMkx/zPJrYZgOEnet1LgOYw9vru3dPeWfW99s5WGAAAAAMBOa/6Wa03o7q9V1cFJHpXk5VX1kW2HJoetdvrE9pXD5y5JvtfdB04OrKpdsxRSJkth4Z+tc76VXDN8LmSVX/+q+nCS2ybZmqUVqSvdQ61xjbXGbJtrrRoBAAAAYG7N9UrPqrp9kqu6+81Jjk3yS8OhJ058fnbY/kySJw3bRyT51PL5uvuyJOdX1ROG+auq7tvdC9194PCzLfDcJcm2576fstJ8U7g8yT4Tdfzb4Vr/IclXk9x+eK9nqmqfqtotyenDfWR4rP1OSc5dNu/kmAcnuXS4RwAAAADWqTtZXOwb7c88muvQM8m9k3x+eBT9hUleNuzfo6o+l6X3Xj532PcHSY6qqrOTPHU4tpIjkjyjqr6U5Jwkj11l3JVJ7lVVZ2TpnZ9/fj3u4+NJ/k1VnVVVT5w80N0/zFJ4+7dDTR9NctMsvWt016r6cpYepz+yu69ZNu8xSbYM9/yKJE+7HjUCAAAAwFSq6pZV9dGqOm/4vMUKY+4x5GLbfi6rqucMx46pqm9MHHvUeq4774+3fzjJhyf3VVWSvKa7X7ps7AVZoSFRdx+57Pv5WWqQtJ7rvzjJi5fte/Cy78esdKy7L83wTs/u/k6S+61xnS8kecAKh45cvqO7T0ty2sS8PxPaTtYEAAAAABvo+UlO7e5XVNXzh+//7+SA7j43Q/Pt4TWT30jy3okhf93dx05z0Xlf6QkAAAAA7Lwem+TkYfvkJI/bzvjDkvzv7r7w+lx0dKFnd28eVlFu9HU2bfQ1AAAAAGAncOuq2jrxc/QU5962uy9OkuHzNtsZ/6Qkb1u279lVdXZVvWGlx+NXMtePtwMAAADADaEX5rOhzw5yaXdvWe1gVX0sye1WOPTCaS5SVTdJ8v8kecHE7tcm+YskPXz+VZKnb28uoScAAAAAcJ1198NWO1ZV36yq/br74qraL8m31pjqkUnO7O5vTsz94+2qOiHJ+9dT0+gebwcAAAAAdhrvS/K0YftpSf5+jbFPzrJH24egdJvHJ/nKei4q9AQAAAAANsorkjy8qs5L8vDhe6rq9lX1gW2Dqmqv4fh7lp3/l1X15ao6O8lDkjx3PRf1eDsAAAAAsCG6+9tZ6si+fP9FSR418f2qJLdaYdxTr8t1hZ4AAAAAsJbu9OKNupHR3PF4OwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKgIPQEAAACAUdHICAAAAAC2oxc0MponVnoCAAAAAKMi9AQAAAAARkXoCQAAAACMitATAAAAABgVjYwAAAAAYC2d9KJGRvPESk8AAAAAYFSEngAAAADAqAg9AQAAAIBREXoCAAAAAKOikREAAAAArKGjkdG8sdITAAAAABgVoScAAAAAMCpCTwAAAABgVISeAAAAAMCoaGQEAAAAAGvppBc0MponVnoCAAAAAKMi9AQAAAAARkXoCQAAAACMitATAAAAABgVjYwAAAAAYE2dxUWNjOaJlZ4AAAAAwKgIPQEAAACAURF6AgAAAACjIvQEAAAAAEZFIyMAAAAAWEMnWVycdRVMw0pPAAAAAGBUhJ4AAAAAwKgIPQEAAACAURF6AgAAAACjopERAAAAAKylNTKaN1Z6AgAAAACjIvQEAAAAAEZF6AkAAAAAjIrQEwAAAAAYFY2MdkJVtTnJ+7v7gO2MeWB3v3X4viXJb3f3H9wAJQIAAADcqGhkNF+s9Jxfm5M8ZduX7t4q8AQAAAAAoed1UlWbq+qrVXVyVZ1dVe+qqr2q6rCq+mJVfbmq3lBVewzjL6iqV1bV54efuw77T6qqwyfmvWKVa32yqs4cfh44HHpFkl+tqrOq6rlV9eCqev9wzi2r6r8Ptf1jVd1n2H/MUNdpVfX1qhKSAgAAADA6Hm+/7u6R5Bnd/emqekOSP0ryu0kO6+6vVdUbkzwryauG8Zd19/2r6reHfY9Z53W+leTh3X11Vd0tyduSbEny/CR/3N2PSZKqevDEOS9N8sXuflxVPTTJG5McOBzbP8lDkuyT5Nyqem13/2jyglV1dJKjk+ROt9or177x5HWWunP4/lP+3axLmMqeteesS5jaJT/4l1mXMLXb7XXnWZcwvX1uPusKpnav3HHWJUztv57/0VmXMJWn33bvWZcwtb13n78/btx6z31mXcLU/vb735h1CVN71u73m3UJU7t64apZlzC1w+7wa7MuYSq77jJ/v2fMo/1vOX//zX7zRefOuoSpPfXau8+6hKktHPefZl3C1Hb9vRfOuoSpfPxFh826BLhRsNLzuvuX7v70sP3mJIclOb+7vzbsOznJgybGv23i85AprrN7khOq6stJ3pnk36zjnEOTvClJuvt/JrlVVd1sOPY/uvua7r40S4HqbZef3N3Hd/eW7t5y6003naJUAAAAAJg9f4163fX1GL9t+9oMwXNVVZKbrHDec5N8M8l9h7FXr+Natcb1r5nYtxD/DgAAAACsqZMsTpsEMVNWel53d6qqbSs2n5zkY0k2b3tfZ5KnJvnExPgnTnx+dti+IMnBw/Zjs7Sqc7mbJbm4uxeHOXcd9l+epUfUV3J6kiOSHz/2fml3X7auuwIAAACAOWeV33X3z0meVlWvT3Jekj9M8o9J3llVuyX5QpLXTYzfo6o+l6Wg+cnDvhOS/H1VfT7JqUmuXOE6xyV5d1U9IcnHJ8acneTaqvpSkpOSfHHinGOSnFhVZye5KsnTrt+tAgAAAMD8EHped4vd/cxl+05NctAq41/T3S+d3NHd30zygIldLxj2X5DkgGH7vCT3WWHMj7L0HtFJpw3HvpOllaM/pbuPWfb9gFVqBQAAAIC55fF2AAAAAGBUrPS8DiZXYq5z/OYNKwYAAACAjdXJ4uKsi2AaVnoCAAAAAKMi9AQAAAAARkXoCQAAAACMitATAAAAABgVjYwAAAAAYA0djYzmjZWeAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGRSMjAAAAAFhLa2Q0b6z0BAAAAABGRegJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKhoZAQAAAMB2aGQ0X6z0BAAAAABGRegJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKhoZAQAAAMAaOhoZzRsrPQEAAACAURF6AgAAAACjIvQEAAAAAEZF6AkAAAAAjIpGRgAAAACwltbIaN5Y6QkAAAAAjIrQEwAAAAAYFaEnAAAAADAqQk8AAAAAYFQ0MgIAAACANXQ0Mpo3VnoCAAAAAKMi9AQAAAAARkXoCQAAAACMitATAAAAABgVjYwAAAAAYC2tkdG8sdITAAAAABgVoScAAAAAMCpCTwAAAABgVISeAAAAAMCoaGQEAAAAANvR3bMugSlY6QkAAAAAjIrQEwAAAAAYFaEnAAAAADAqQk8AAAAAYFQ0MgIAAACANXSSxcVZV8E0rPScoaraXFVfmWL8SVV1+Ar7t1TV3wzbD66qB65jrnWNAwAAAIB5Y6XnCHT31iRbh68PTnJFks9s57T1jgMAAACAuWKl5+ztWlUnVNU5VfWRqtqzqn6nqr5QVV+qqndX1V4T4x9WVZ+sqq9V1WOSH6/afH9VbU7yzCTPraqzqupXq+rfVdXnquqLVfWxqrrtSuNu4HsGAAAAgA1jpefs3S3Jk7v7d6rqHUl+I8l7uvuEJKmqlyV5RpK/HcZvTvJrSe6S5ONVdddtE3X3BVX1uiRXdPexw/m3SPKA7u6q+g9J/rS7n7d83KSqOjrJ0UlyhzveOlcc8fgNufGN8s0rvz7rEqZyz4VbzbqEqV2x56ZZlzC1TbvffNYlTO3CK86ddQlT23PX+ft34xn7PWDWJUzlW3eevxcJ/fpu+8y6hKmdecmZsy5hapt/bs9ZlzC13Xv+/v5998WbzLqEqV2z2/z9OrPxdq35+1/Bg/ZfmUeQAAAE/klEQVS946xLmN5e8/dn0F2u/N6sS5jax1902KxLmMpDXnbqrEuAG4X5+y/d+Jzf3WcN22dkKdQ8YAg7b55kU5IPT4x/R3cvJjmvqr6eZP/tzH+HJKdU1X5JbpLk/O0V1N3HJzk+SQ46+C49xb0AAAAAjE9rZDRv/LXv7F0zsb2QpSD6pCTP7u57J3lpkptOjFkeQm4vlPzbJH83zPW7y+YCAAAAgNEReu6c9klycVXtnuSIZceeUFW7VNVdkvxikuXPv14+nL/NzZJ8Y9h+2hrjAAAAAGAUhJ47pxcn+VySjyb56rJj5yb5RJIPJnlmd1+97Pg/JHn8RIOiY5K8s6o+meTSNcYBAAAAwCh4p+cMdfcFSQ6Y+D7ZVOi1K4w/cpV5Tkty2rD9tST3WTbk71c4Z6VxAAAAADD3hJ4AAAAAsB0aGc0Xj7cDAAAAAKMi9AQAAAAARkXoCQAAAACMitATAAAAABgVjYwAAAAAYA0djYzmjZWeAAAAAMCoCD0BAAAAgFERegIAAAAAoyL0BAAAAABGRSMjAAAAAFhLa2Q0b6z0BAAAAABGRegJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKhoZAQAAAMAaOhoZzRsrPQEAAACAURF6AgAAAACjIvQEAAAAAEZF6AkAAAAAjIpGRgAAAACwltbIaN5Y6QkAAAAAjIrQEwAAAAAYFaEnAAAAADAqQk8AAAAAYFQ0MgIAAACA7VjsWVfANKz0BAAAAABGRegJAAAAAIyK0BMAAAAAGBWhJwAAAAAwKhoZAQAAAMAaOsni4qyrYBpWegIAAAAAoyL0BAAAAABGRegJAAAAAIyK0BMAAAAAGBWNjAAAAABgLa2R0byx0hMAAAAAGBWhJwAAAAAwKkJPAAAAAGBUhJ4AAAAAwKhoZAQAAAAAa+hoZDRvrPQEAAAAAEZF6AkAAAAAjIrQEwAAAAAYFaEnAAAAADAqGhkBAAAAwHZoZDRfrPQEAAAAAEZF6AkAAAAAbIiqekJVnVNVi1W1ZY1xj6iqc6vqf1XV8yf237KqPlpV5w2ft1jPdYWeAAAAAMBG+Ur+//buXbWKKAoD8L8QJK0garwgFmnyAIq9Aa2iFqJYpEvlAwi+gJ2VTbp0YmdAIWAaW20leEELg0FrG0VYNlMEEskhF44evq/aa/awZhe7+tkzk9xM8upvN1TVkSSPk1xLMpvkTlXNDtP3k6x190yStaHeldATAAAAADgU3b3e3e92ue1iko/d/am7fyV5kmR+mJtPsjyMl5NcH+W5fmTENlW1mGRxKH8cm7q128YEANiDR+NeAAAwuvPjXsA4fc7P1bt5f3zc6xijqap6s6Ve6u6lA+x/JsmXLfVGkkvD+GR3byZJd29W1YlRGgo92WbYtAe5cQEAAAD+W919ddxr+JdV1cskp3aYetDdz0ZpscO13s+ahJ4AAAAAwJ5195V9tthIcm5LfTbJ12H8raqmh1Oe00m+j9LQNz0BAAAAgHF6nWSmqi5U1dEkt5OsDHMrSRaG8UKSUU6OCj0BAAAAgMNRVTeqaiPJ5STPq2p1uH66ql4kSXf/TnIvyWqS9SRPu/vt0OJhkrmq+pBkbqh3f273vl6PBwAAAAD4pzjpCQAAAABMFKEnAAAAADBRhJ4AAAAAwEQRegIAAAAAE0XoCQAAAABMFKEnAAAAADBRhJ4AAAAAwET5A5pExOXAVTABAAAAAElFTkSuQmCC\n", 1054 | "text/plain": [ 1055 | "
" 1056 | ] 1057 | }, 1058 | "metadata": { 1059 | "needs_background": "light" 1060 | }, 1061 | "output_type": "display_data" 1062 | } 1063 | ], 1064 | "source": [ 1065 | "visualize_data(data.corr())" 1066 | ] 1067 | }, 1068 | { 1069 | "cell_type": "code", 1070 | "execution_count": 45, 1071 | "metadata": {}, 1072 | "outputs": [], 1073 | "source": [ 1074 | "data.drop(['bruises'], axis=1)\n", 1075 | "data_data = data[columns[1:]].values\n", 1076 | "data_label= data[columns[0]].values \n" 1077 | ] 1078 | }, 1079 | { 1080 | "cell_type": "code", 1081 | "execution_count": 46, 1082 | "metadata": {}, 1083 | "outputs": [ 1084 | { 1085 | "data": { 1086 | "text/plain": [ 1087 | "8124" 1088 | ] 1089 | }, 1090 | "execution_count": 46, 1091 | "metadata": {}, 1092 | "output_type": "execute_result" 1093 | } 1094 | ], 1095 | "source": [ 1096 | "len(data_data)" 1097 | ] 1098 | }, 1099 | { 1100 | "cell_type": "code", 1101 | "execution_count": null, 1102 | "metadata": {}, 1103 | "outputs": [], 1104 | "source": [] 1105 | }, 1106 | { 1107 | "cell_type": "code", 1108 | "execution_count": 47, 1109 | "metadata": {}, 1110 | "outputs": [], 1111 | "source": [ 1112 | " #edible=e=0, poisonous=p=1" 1113 | ] 1114 | }, 1115 | { 1116 | "cell_type": "code", 1117 | "execution_count": 49, 1118 | "metadata": {}, 1119 | "outputs": [], 1120 | "source": [ 1121 | "x_train, x_test,y_train,y_test = train_test_split(data_data, data_label, test_size=0.5,random_state=15)\n", 1122 | "\n", 1123 | "pca = PCA(n_components=20,whiten=True, random_state= 32)\n", 1124 | "\n" 1125 | ] 1126 | }, 1127 | { 1128 | "cell_type": "code", 1129 | "execution_count": 50, 1130 | "metadata": {}, 1131 | "outputs": [ 1132 | { 1133 | "name": "stdout", 1134 | "output_type": "stream", 1135 | "text": [ 1136 | "(22,)\n" 1137 | ] 1138 | } 1139 | ], 1140 | "source": [ 1141 | "print(x_train[1].shape)" 1142 | ] 1143 | }, 1144 | { 1145 | "cell_type": "code", 1146 | "execution_count": 51, 1147 | "metadata": {}, 1148 | "outputs": [ 1149 | { 1150 | "data": { 1151 | "text/plain": [ 1152 | "(4062, 22)" 1153 | ] 1154 | }, 1155 | "execution_count": 51, 1156 | "metadata": {}, 1157 | "output_type": "execute_result" 1158 | } 1159 | ], 1160 | "source": [ 1161 | "x_train.shape" 1162 | ] 1163 | }, 1164 | { 1165 | "cell_type": "code", 1166 | "execution_count": 60, 1167 | "metadata": {}, 1168 | "outputs": [ 1169 | { 1170 | "name": "stdout", 1171 | "output_type": "stream", 1172 | "text": [ 1173 | "-----------------svm------------------------------\n", 1174 | "(4062, 22) (4062,)\n", 1175 | "{'svc__C': 7, 'svc__gamma': 0.1}\n", 1176 | " precision recall f1-score support\n", 1177 | "\n", 1178 | " edible 1.00 1.00 1.00 2133\n", 1179 | " poisonous 1.00 1.00 1.00 1929\n", 1180 | "\n", 1181 | " accuracy 1.00 4062\n", 1182 | " macro avg 1.00 1.00 1.00 4062\n", 1183 | "weighted avg 1.00 1.00 1.00 4062\n", 1184 | "\n", 1185 | "0.9997407311381903\n", 1186 | "[[2133 0]\n", 1187 | " [ 1 1928]]\n" 1188 | ] 1189 | } 1190 | ], 1191 | "source": [ 1192 | "print('-----------------svm------------------------------')\n", 1193 | "\n", 1194 | "svc = SVC(kernel='rbf',class_weight='balanced')\n", 1195 | "\n", 1196 | "\n", 1197 | "\n", 1198 | "model = make_pipeline(pca, svc)\n", 1199 | "\n", 1200 | "\n", 1201 | "param_grid = {'svc__C': [7,9,11],\n", 1202 | " 'svc__gamma': [0.1,0.5,1]}\n", 1203 | "grid = GridSearchCV(model, param_grid)\n", 1204 | "print(x_train.shape, y_train.shape)\n", 1205 | "grid.fit(x_train, y_train) #bulid model\n", 1206 | "\n", 1207 | "print(grid.best_params_) #show the best parameters\n", 1208 | "\n", 1209 | "model = grid.best_estimator_ #pick up the best model\n", 1210 | "\n", 1211 | "yfit = model.predict(x_test) #use the best model\n", 1212 | "# fig, ax = plt.subplots(4, 6)\n", 1213 | "# for i, axi in enumerate(ax.flat):\n", 1214 | "# axi.imshow(x_test[i].reshape(22), cmap='bone')\n", 1215 | "# axi.set(xticks=[], yticks=[])\n", 1216 | "# axi.set_ylabel(faces.target_names[yfit[i]].split()[-1],\n", 1217 | "# color='black' if yfit[i] == y_test[i] else 'red')\n", 1218 | "# fig.suptitle('Predicted Names; Incorrect Labels in Red', size=14)\n", 1219 | "# fig.show()\n", 1220 | "\n", 1221 | "\n", 1222 | "print(classification_report(y_test, yfit,\n", 1223 | " target_names=['edible', 'poisonous']))\n", 1224 | "f1 = f1_score(y_test, yfit, average='binary')\n", 1225 | "a = confusion_matrix(y_test, yfit)\n", 1226 | "print(f1)\n", 1227 | "print(a)" 1228 | ] 1229 | }, 1230 | { 1231 | "cell_type": "code", 1232 | "execution_count": 53, 1233 | "metadata": {}, 1234 | "outputs": [ 1235 | { 1236 | "name": "stdout", 1237 | "output_type": "stream", 1238 | "text": [ 1239 | "-----------------randomforest------------------------------\n", 1240 | "(4062, 22) (4062,)\n", 1241 | "{'n_estimators': 3}\n", 1242 | " precision recall f1-score support\n", 1243 | "\n", 1244 | " edible 1.00 1.00 1.00 2133\n", 1245 | " poisonous 1.00 1.00 1.00 1929\n", 1246 | "\n", 1247 | " accuracy 1.00 4062\n", 1248 | " macro avg 1.00 1.00 1.00 4062\n", 1249 | "weighted avg 1.00 1.00 1.00 4062\n", 1250 | "\n", 1251 | "0.9987023098883987\n", 1252 | "[[2133 0]\n", 1253 | " [ 5 1924]]\n" 1254 | ] 1255 | } 1256 | ], 1257 | "source": [ 1258 | "print('-----------------randomforest------------------------------')\n", 1259 | "\n", 1260 | "clf = RandomForestClassifier(random_state=0)\n", 1261 | "\n", 1262 | "\n", 1263 | "model1 = make_pipeline(pca, clf)\n", 1264 | "\n", 1265 | "# param_grid = [\n", 1266 | "# {'n_estimators': [1,2,3], 'max_features': [20,21,22]}\n", 1267 | "# ]\n", 1268 | "\n", 1269 | "param_grid = [\n", 1270 | "{'n_estimators': [1,2,3]}\n", 1271 | "]\n", 1272 | "\n", 1273 | "\n", 1274 | "grid_search = GridSearchCV(clf, param_grid, cv=5,\n", 1275 | " scoring='neg_mean_squared_error')\n", 1276 | "\n", 1277 | "print(x_train.shape, y_train.shape)\n", 1278 | "\n", 1279 | "grid_search.fit(x_train, y_train) #build model\n", 1280 | "\n", 1281 | "print(grid_search.best_params_) #show the best parameters\n", 1282 | "\n", 1283 | "model1 = grid_search.best_estimator_ #pick up the best model\n", 1284 | "\n", 1285 | "yfit = model1.predict(x_test) #use the best model\n", 1286 | "# fig, ax = plt.subplots(4, 6)\n", 1287 | "# for i, axi in enumerate(ax.flat):\n", 1288 | "# axi.imshow(x_test[i].reshape(22), cmap='bone')\n", 1289 | "# axi.set(xticks=[], yticks=[])\n", 1290 | "# axi.set_ylabel(faces.target_names[yfit[i]].split()[-1],\n", 1291 | "# color='black' if yfit[i] == y_test[i] else 'red')\n", 1292 | "# fig.suptitle('Predicted Names; Incorrect Labels in Red', size=14)\n", 1293 | "# fig.show()\n", 1294 | "\n", 1295 | "\n", 1296 | "print(classification_report(y_test, yfit,\n", 1297 | " target_names=['edible', 'poisonous']))\n", 1298 | "f1 = f1_score(y_test, yfit, average='binary')\n", 1299 | "a = confusion_matrix(y_test, yfit)\n", 1300 | "print(f1)\n", 1301 | "print(a)" 1302 | ] 1303 | }, 1304 | { 1305 | "cell_type": "code", 1306 | "execution_count": 54, 1307 | "metadata": {}, 1308 | "outputs": [ 1309 | { 1310 | "name": "stdout", 1311 | "output_type": "stream", 1312 | "text": [ 1313 | "[0.000953 0.00539851 0.06849629 0.00057451 0.27318584 0.\n", 1314 | " 0.00028443 0.11608979 0.09398737 0.09765874 0.00554243 0.01227082\n", 1315 | " 0.00332788 0.00028963 0.01112092 0. 0. 0.\n", 1316 | " 0.21442257 0.05097382 0.03397122 0.01145224]\n" 1317 | ] 1318 | } 1319 | ], 1320 | "source": [ 1321 | "print(model1.feature_importances_)" 1322 | ] 1323 | }, 1324 | { 1325 | "cell_type": "code", 1326 | "execution_count": 55, 1327 | "metadata": {}, 1328 | "outputs": [ 1329 | { 1330 | "name": "stdout", 1331 | "output_type": "stream", 1332 | "text": [ 1333 | "gill-attachment\n", 1334 | "stalk-surface-above-ring\n", 1335 | "population\n" 1336 | ] 1337 | } 1338 | ], 1339 | "source": [ 1340 | "print(columns[5+1])\n", 1341 | "print(columns[11+1])\n", 1342 | "print(columns[20+1])" 1343 | ] 1344 | }, 1345 | { 1346 | "cell_type": "code", 1347 | "execution_count": 56, 1348 | "metadata": {}, 1349 | "outputs": [ 1350 | { 1351 | "name": "stdout", 1352 | "output_type": "stream", 1353 | "text": [ 1354 | "-----------------lr------------------------------\n", 1355 | " precision recall f1-score support\n", 1356 | "\n", 1357 | " edible 0.99 0.99 0.99 2133\n", 1358 | " poisonous 0.99 0.98 0.99 1929\n", 1359 | "\n", 1360 | " accuracy 0.99 4062\n", 1361 | " macro avg 0.99 0.99 0.99 4062\n", 1362 | "weighted avg 0.99 0.99 0.99 4062\n", 1363 | "\n", 1364 | "0.9862301896596518\n", 1365 | "[[2111 22]\n", 1366 | " [ 31 1898]]\n" 1367 | ] 1368 | } 1369 | ], 1370 | "source": [ 1371 | "print('-----------------lr------------------------------')\n", 1372 | "\n", 1373 | "lr = LogisticRegression(class_weight='balanced',penalty = 'l2',C = 1)\n", 1374 | "\n", 1375 | "\n", 1376 | "model2 = make_pipeline(pca, lr)\n", 1377 | "\n", 1378 | "# param_grid = {}\n", 1379 | "\n", 1380 | "\n", 1381 | "# grid_search = GridSearchCV(clf, param_grid, cv=5,\n", 1382 | "# scoring='neg_mean_squared_error')\n", 1383 | "\n", 1384 | "# print(x_train.shape, y_train.shape)\n", 1385 | "\n", 1386 | "model2.fit(x_train, y_train) #bulid model\n", 1387 | "\n", 1388 | "\n", 1389 | "\n", 1390 | "yfit = model2.predict(x_test) #\n", 1391 | "# fig, ax = plt.subplots(4, 6)\n", 1392 | "# for i, axi in enumerate(ax.flat):\n", 1393 | "# axi.imshow(x_test[i].reshape(22), cmap='bone')\n", 1394 | "# axi.set(xticks=[], yticks=[])\n", 1395 | "# axi.set_ylabel(faces.target_names[yfit[i]].split()[-1],\n", 1396 | "# color='black' if yfit[i] == y_test[i] else 'red')\n", 1397 | "# fig.suptitle('Predicted Names; Incorrect Labels in Red', size=14)\n", 1398 | "# fig.show()\n", 1399 | "\n", 1400 | "\n", 1401 | "print(classification_report(y_test, yfit,\n", 1402 | " target_names=['edible', 'poisonous']))\n", 1403 | "f1 = f1_score(y_test, yfit, average='binary')\n", 1404 | "a = confusion_matrix(y_test, yfit)\n", 1405 | "print(f1)\n", 1406 | "print(a)" 1407 | ] 1408 | }, 1409 | { 1410 | "cell_type": "code", 1411 | "execution_count": 57, 1412 | "metadata": {}, 1413 | "outputs": [ 1414 | { 1415 | "name": "stdout", 1416 | "output_type": "stream", 1417 | "text": [ 1418 | "svm 10-fold correct prediction: 0.8977\n", 1419 | "randomforest 10-fold correct prediction: 0.9626\n", 1420 | "lr 10-fold correct prediction: 0.9633\n" 1421 | ] 1422 | } 1423 | ], 1424 | "source": [ 1425 | "svm_10_fold = cross_val_score(model, data_data, data_label, cv=10)\n", 1426 | "rf_10_fold = cross_val_score(model1, data_data, data_label, cv=10)\n", 1427 | "lr_10_fold = cross_val_score(model2, data_data, data_label, cv=10)\n", 1428 | "\n", 1429 | "\n", 1430 | "print('svm 10-fold correct prediction: {:4.4f}'.format(np.mean(svm_10_fold)))\n", 1431 | "print('randomforest 10-fold correct prediction: {:4.4f}'.format(np.mean(rf_10_fold)))\n", 1432 | "print('lr 10-fold correct prediction: {:4.4f}'.format(np.mean(lr_10_fold)))\n" 1433 | ] 1434 | }, 1435 | { 1436 | "cell_type": "code", 1437 | "execution_count": null, 1438 | "metadata": {}, 1439 | "outputs": [], 1440 | "source": [] 1441 | }, 1442 | { 1443 | "cell_type": "code", 1444 | "execution_count": null, 1445 | "metadata": {}, 1446 | "outputs": [], 1447 | "source": [] 1448 | } 1449 | ], 1450 | "metadata": { 1451 | "kernelspec": { 1452 | "display_name": "Python 3", 1453 | "language": "python", 1454 | "name": "python3" 1455 | }, 1456 | "language_info": { 1457 | "codemirror_mode": { 1458 | "name": "ipython", 1459 | "version": 3 1460 | }, 1461 | "file_extension": ".py", 1462 | "mimetype": "text/x-python", 1463 | "name": "python", 1464 | "nbconvert_exporter": "python", 1465 | "pygments_lexer": "ipython3", 1466 | "version": "3.8.3" 1467 | } 1468 | }, 1469 | "nbformat": 4, 1470 | "nbformat_minor": 4 1471 | } 1472 | -------------------------------------------------------------------------------- /mashroom/read.me: -------------------------------------------------------------------------------- 1 | 2 | --------------------------------------------------------------------------------