├── LICENSE.md ├── Notebooks [eng] ├── 08) Quality metrics and error analysis in ML.ipynb ├── 09) Optimization methods in ML & DL.ipynb ├── ML-algorithms from scratch │ ├── 01) Linear regression & its types.ipynb │ ├── 02) Logistic & Softmax-regressions.ipynb │ ├── 03) Linear Discriminant Analysis (LDA).ipynb │ ├── 04) Naive Bayes Classifier.ipynb │ ├── 05) Support Vector Machines (SVM).ipynb │ ├── 06) K-Nearest Neighbors (KNN).ipynb │ ├── 07) Decision Tree (CART).ipynb │ ├── 08) Bagging & Random Forest.ipynb │ ├── 09) AdaBoost algorithms (SAMME & R2).ipynb │ ├── 10) Gradient Boosting (GBM, XGBoost, CatBoost, LightGBM).ipynb │ ├── 11) Stacking & Blending.ipynb │ ├── 12) Principal Component Analysis (PCA).ipynb │ └── 13) Clustering in ML (K-Means, Agglomerative, Spectral, DBSCAN, Affinity Propagation).ipynb └── empty.txt ├── Notebooks [rus] ├── 08) Метрики качества и анализ ошибок в ML.ipynb ├── 09) Методы оптимизации в ML и DL.ipynb ├── ML-алгоритмы с нуля │ ├── 01) Линейная регрессия и её виды.ipynb │ ├── 02) Логистическая и Softmax-регрессии.ipynb │ ├── 03) Линейный дискриминантный анализ (LDA).ipynb │ ├── 04) Наивный байесовский классификатор.ipynb │ ├── 05) Метод опорных векторов (SVM).ipynb │ ├── 06) K-ближайших соседей (KNN).ipynb │ ├── 07) Дерево решений (CART).ipynb │ ├── 08) Бэггинг и случайный лес.ipynb │ ├── 09) Алгоритмы AdaBoost (SAMME & R2).ipynb │ ├── 10) Градиентный бустинг (GBM, XGBoost, CatBoost, LightGBM).ipynb │ ├── 11) Стекинг & Блендинг.ipynb │ ├── 12) Метод главных компонент (PCA).ipynb │ └── 13) Кластеризация в ML (K-Means, Agglomerative, Spectral, DBSCAN, Affinity Propagation).ipynb └── empty.txt └── README.md /LICENSE.md: -------------------------------------------------------------------------------- 1 | Attribution-NonCommercial-ShareAlike 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International 58 | Public License 59 | 60 | By exercising the Licensed Rights (defined below), You accept and agree 61 | to be bound by the terms and conditions of this Creative Commons 62 | Attribution-NonCommercial-ShareAlike 4.0 International Public License 63 | ("Public License"). To the extent this Public License may be 64 | interpreted as a contract, You are granted the Licensed Rights in 65 | consideration of Your acceptance of these terms and conditions, and the 66 | Licensor grants You such rights in consideration of benefits the 67 | Licensor receives from making the Licensed Material available under 68 | these terms and conditions. 69 | 70 | 71 | Section 1 -- Definitions. 72 | 73 | a. Adapted Material means material subject to Copyright and Similar 74 | Rights that is derived from or based upon the Licensed Material 75 | and in which the Licensed Material is translated, altered, 76 | arranged, transformed, or otherwise modified in a manner requiring 77 | permission under the Copyright and Similar Rights held by the 78 | Licensor. For purposes of this Public License, where the Licensed 79 | Material is a musical work, performance, or sound recording, 80 | Adapted Material is always produced where the Licensed Material is 81 | synched in timed relation with a moving image. 82 | 83 | b. Adapter's License means the license You apply to Your Copyright 84 | and Similar Rights in Your contributions to Adapted Material in 85 | accordance with the terms and conditions of this Public License. 86 | 87 | c. BY-NC-SA Compatible License means a license listed at 88 | creativecommons.org/compatiblelicenses, approved by Creative 89 | Commons as essentially the equivalent of this Public License. 90 | 91 | d. Copyright and Similar Rights means copyright and/or similar rights 92 | closely related to copyright including, without limitation, 93 | performance, broadcast, sound recording, and Sui Generis Database 94 | Rights, without regard to how the rights are labeled or 95 | categorized. For purposes of this Public License, the rights 96 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 97 | Rights. 98 | 99 | e. Effective Technological Measures means those measures that, in the 100 | absence of proper authority, may not be circumvented under laws 101 | fulfilling obligations under Article 11 of the WIPO Copyright 102 | Treaty adopted on December 20, 1996, and/or similar international 103 | agreements. 104 | 105 | f. Exceptions and Limitations means fair use, fair dealing, and/or 106 | any other exception or limitation to Copyright and Similar Rights 107 | that applies to Your use of the Licensed Material. 108 | 109 | g. License Elements means the license attributes listed in the name 110 | of a Creative Commons Public License. The License Elements of this 111 | Public License are Attribution, NonCommercial, and ShareAlike. 112 | 113 | h. Licensed Material means the artistic or literary work, database, 114 | or other material to which the Licensor applied this Public 115 | License. 116 | 117 | i. Licensed Rights means the rights granted to You subject to the 118 | terms and conditions of this Public License, which are limited to 119 | all Copyright and Similar Rights that apply to Your use of the 120 | Licensed Material and that the Licensor has authority to license. 121 | 122 | j. Licensor means the individual(s) or entity(ies) granting rights 123 | under this Public License. 124 | 125 | k. NonCommercial means not primarily intended for or directed towards 126 | commercial advantage or monetary compensation. For purposes of 127 | this Public License, the exchange of the Licensed Material for 128 | other material subject to Copyright and Similar Rights by digital 129 | file-sharing or similar means is NonCommercial provided there is 130 | no payment of monetary compensation in connection with the 131 | exchange. 132 | 133 | l. Share means to provide material to the public by any means or 134 | process that requires permission under the Licensed Rights, such 135 | as reproduction, public display, public performance, distribution, 136 | dissemination, communication, or importation, and to make material 137 | available to the public including in ways that members of the 138 | public may access the material from a place and at a time 139 | individually chosen by them. 140 | 141 | m. Sui Generis Database Rights means rights other than copyright 142 | resulting from Directive 96/9/EC of the European Parliament and of 143 | the Council of 11 March 1996 on the legal protection of databases, 144 | as amended and/or succeeded, as well as other essentially 145 | equivalent rights anywhere in the world. 146 | 147 | n. You means the individual or entity exercising the Licensed Rights 148 | under this Public License. Your has a corresponding meaning. 149 | 150 | 151 | Section 2 -- Scope. 152 | 153 | a. License grant. 154 | 155 | 1. Subject to the terms and conditions of this Public License, 156 | the Licensor hereby grants You a worldwide, royalty-free, 157 | non-sublicensable, non-exclusive, irrevocable license to 158 | exercise the Licensed Rights in the Licensed Material to: 159 | 160 | a. reproduce and Share the Licensed Material, in whole or 161 | in part, for NonCommercial purposes only; and 162 | 163 | b. produce, reproduce, and Share Adapted Material for 164 | NonCommercial purposes only. 165 | 166 | 2. Exceptions and Limitations. For the avoidance of doubt, where 167 | Exceptions and Limitations apply to Your use, this Public 168 | License does not apply, and You do not need to comply with 169 | its terms and conditions. 170 | 171 | 3. Term. The term of this Public License is specified in Section 172 | 6(a). 173 | 174 | 4. Media and formats; technical modifications allowed. The 175 | Licensor authorizes You to exercise the Licensed Rights in 176 | all media and formats whether now known or hereafter created, 177 | and to make technical modifications necessary to do so. The 178 | Licensor waives and/or agrees not to assert any right or 179 | authority to forbid You from making technical modifications 180 | necessary to exercise the Licensed Rights, including 181 | technical modifications necessary to circumvent Effective 182 | Technological Measures. For purposes of this Public License, 183 | simply making modifications authorized by this Section 2(a) 184 | (4) never produces Adapted Material. 185 | 186 | 5. Downstream recipients. 187 | 188 | a. Offer from the Licensor -- Licensed Material. Every 189 | recipient of the Licensed Material automatically 190 | receives an offer from the Licensor to exercise the 191 | Licensed Rights under the terms and conditions of this 192 | Public License. 193 | 194 | b. Additional offer from the Licensor -- Adapted Material. 195 | Every recipient of Adapted Material from You 196 | automatically receives an offer from the Licensor to 197 | exercise the Licensed Rights in the Adapted Material 198 | under the conditions of the Adapter's License You apply. 199 | 200 | c. No downstream restrictions. You may not offer or impose 201 | any additional or different terms or conditions on, or 202 | apply any Effective Technological Measures to, the 203 | Licensed Material if doing so restricts exercise of the 204 | Licensed Rights by any recipient of the Licensed 205 | Material. 206 | 207 | 6. No endorsement. Nothing in this Public License constitutes or 208 | may be construed as permission to assert or imply that You 209 | are, or that Your use of the Licensed Material is, connected 210 | with, or sponsored, endorsed, or granted official status by, 211 | the Licensor or others designated to receive attribution as 212 | provided in Section 3(a)(1)(A)(i). 213 | 214 | b. Other rights. 215 | 216 | 1. Moral rights, such as the right of integrity, are not 217 | licensed under this Public License, nor are publicity, 218 | privacy, and/or other similar personality rights; however, to 219 | the extent possible, the Licensor waives and/or agrees not to 220 | assert any such rights held by the Licensor to the limited 221 | extent necessary to allow You to exercise the Licensed 222 | Rights, but not otherwise. 223 | 224 | 2. Patent and trademark rights are not licensed under this 225 | Public License. 226 | 227 | 3. To the extent possible, the Licensor waives any right to 228 | collect royalties from You for the exercise of the Licensed 229 | Rights, whether directly or through a collecting society 230 | under any voluntary or waivable statutory or compulsory 231 | licensing scheme. In all other cases the Licensor expressly 232 | reserves any right to collect such royalties, including when 233 | the Licensed Material is used other than for NonCommercial 234 | purposes. 235 | 236 | 237 | Section 3 -- License Conditions. 238 | 239 | Your exercise of the Licensed Rights is expressly made subject to the 240 | following conditions. 241 | 242 | a. Attribution. 243 | 244 | 1. If You Share the Licensed Material (including in modified 245 | form), You must: 246 | 247 | a. retain the following if it is supplied by the Licensor 248 | with the Licensed Material: 249 | 250 | i. identification of the creator(s) of the Licensed 251 | Material and any others designated to receive 252 | attribution, in any reasonable manner requested by 253 | the Licensor (including by pseudonym if 254 | designated); 255 | 256 | ii. a copyright notice; 257 | 258 | iii. a notice that refers to this Public License; 259 | 260 | iv. a notice that refers to the disclaimer of 261 | warranties; 262 | 263 | v. a URI or hyperlink to the Licensed Material to the 264 | extent reasonably practicable; 265 | 266 | b. indicate if You modified the Licensed Material and 267 | retain an indication of any previous modifications; and 268 | 269 | c. indicate the Licensed Material is licensed under this 270 | Public License, and include the text of, or the URI or 271 | hyperlink to, this Public License. 272 | 273 | 2. You may satisfy the conditions in Section 3(a)(1) in any 274 | reasonable manner based on the medium, means, and context in 275 | which You Share the Licensed Material. For example, it may be 276 | reasonable to satisfy the conditions by providing a URI or 277 | hyperlink to a resource that includes the required 278 | information. 279 | 3. If requested by the Licensor, You must remove any of the 280 | information required by Section 3(a)(1)(A) to the extent 281 | reasonably practicable. 282 | 283 | b. ShareAlike. 284 | 285 | In addition to the conditions in Section 3(a), if You Share 286 | Adapted Material You produce, the following conditions also apply. 287 | 288 | 1. The Adapter's License You apply must be a Creative Commons 289 | license with the same License Elements, this version or 290 | later, or a BY-NC-SA Compatible License. 291 | 292 | 2. You must include the text of, or the URI or hyperlink to, the 293 | Adapter's License You apply. You may satisfy this condition 294 | in any reasonable manner based on the medium, means, and 295 | context in which You Share Adapted Material. 296 | 297 | 3. You may not offer or impose any additional or different terms 298 | or conditions on, or apply any Effective Technological 299 | Measures to, Adapted Material that restrict exercise of the 300 | rights granted under the Adapter's License You apply. 301 | 302 | 303 | Section 4 -- Sui Generis Database Rights. 304 | 305 | Where the Licensed Rights include Sui Generis Database Rights that 306 | apply to Your use of the Licensed Material: 307 | 308 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 309 | to extract, reuse, reproduce, and Share all or a substantial 310 | portion of the contents of the database for NonCommercial purposes 311 | only; 312 | 313 | b. if You include all or a substantial portion of the database 314 | contents in a database in which You have Sui Generis Database 315 | Rights, then the database in which You have Sui Generis Database 316 | Rights (but not its individual contents) is Adapted Material, 317 | including for purposes of Section 3(b); and 318 | 319 | c. You must comply with the conditions in Section 3(a) if You Share 320 | all or a substantial portion of the contents of the database. 321 | 322 | For the avoidance of doubt, this Section 4 supplements and does not 323 | replace Your obligations under this Public License where the Licensed 324 | Rights include other Copyright and Similar Rights. 325 | 326 | 327 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 328 | 329 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 330 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 331 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 332 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 333 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 334 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 335 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 336 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 337 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 338 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 339 | 340 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 341 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 342 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 343 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 344 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 345 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 346 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 347 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 348 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 349 | 350 | c. The disclaimer of warranties and limitation of liability provided 351 | above shall be interpreted in a manner that, to the extent 352 | possible, most closely approximates an absolute disclaimer and 353 | waiver of all liability. 354 | 355 | 356 | Section 6 -- Term and Termination. 357 | 358 | a. This Public License applies for the term of the Copyright and 359 | Similar Rights licensed here. However, if You fail to comply with 360 | this Public License, then Your rights under this Public License 361 | terminate automatically. 362 | 363 | b. Where Your right to use the Licensed Material has terminated under 364 | Section 6(a), it reinstates: 365 | 366 | 1. automatically as of the date the violation is cured, provided 367 | it is cured within 30 days of Your discovery of the 368 | violation; or 369 | 370 | 2. upon express reinstatement by the Licensor. 371 | 372 | For the avoidance of doubt, this Section 6(b) does not affect any 373 | right the Licensor may have to seek remedies for Your violations 374 | of this Public License. 375 | 376 | c. For the avoidance of doubt, the Licensor may also offer the 377 | Licensed Material under separate terms or conditions or stop 378 | distributing the Licensed Material at any time; however, doing so 379 | will not terminate this Public License. 380 | 381 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 382 | License. 383 | 384 | 385 | Section 7 -- Other Terms and Conditions. 386 | 387 | a. The Licensor shall not be bound by any additional or different 388 | terms or conditions communicated by You unless expressly agreed. 389 | 390 | b. Any arrangements, understandings, or agreements regarding the 391 | Licensed Material not stated herein are separate from and 392 | independent of the terms and conditions of this Public License. 393 | 394 | 395 | Section 8 -- Interpretation. 396 | 397 | a. For the avoidance of doubt, this Public License does not, and 398 | shall not be interpreted to, reduce, limit, restrict, or impose 399 | conditions on any use of the Licensed Material that could lawfully 400 | be made without permission under this Public License. 401 | 402 | b. To the extent possible, if any provision of this Public License is 403 | deemed unenforceable, it shall be automatically reformed to the 404 | minimum extent necessary to make it enforceable. If the provision 405 | cannot be reformed, it shall be severed from this Public License 406 | without affecting the enforceability of the remaining terms and 407 | conditions. 408 | 409 | c. No term or condition of this Public License will be waived and no 410 | failure to comply consented to unless expressly agreed to by the 411 | Licensor. 412 | 413 | d. Nothing in this Public License constitutes or may be interpreted 414 | as a limitation upon, or waiver of, any privileges and immunities 415 | that apply to the Licensor or You, including from the legal 416 | processes of any jurisdiction or authority. 417 | 418 | ======================================================================= 419 | 420 | Creative Commons is not a party to its public 421 | licenses. Notwithstanding, Creative Commons may elect to apply one of 422 | its public licenses to material it publishes and in those instances 423 | will be considered the “Licensor.” The text of the Creative Commons 424 | public licenses is dedicated to the public domain under the CC0 Public 425 | Domain Dedication. Except for the limited purpose of indicating that 426 | material is shared under a Creative Commons public license or as 427 | otherwise permitted by the Creative Commons policies published at 428 | creativecommons.org/policies, Creative Commons does not authorize the 429 | use of the trademark "Creative Commons" or any other trademark or logo 430 | of Creative Commons without its prior written consent including, 431 | without limitation, in connection with any unauthorized modifications 432 | to any of its public licenses or any other arrangements, 433 | understandings, or agreements concerning use of licensed material. For 434 | the avoidance of doubt, this paragraph does not form part of the 435 | public licenses. 436 | 437 | Creative Commons may be contacted at creativecommons.org. 438 | -------------------------------------------------------------------------------- /Notebooks [eng]/ML-algorithms from scratch/04) Naive Bayes Classifier.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [] 7 | }, 8 | "kernelspec": { 9 | "name": "python3", 10 | "display_name": "Python 3" 11 | }, 12 | "language_info": { 13 | "name": "python" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "markdown", 19 | "source": [ 20 | "## **Naive Bayes Classifier**\n", 21 | "Naive Bayes classifier is a probabilistic classifier based on the Bayes formula with a strict (naive) assumption of the independence of features among themselves for a given class, which greatly simplifies the classification task due to the evaluation of one-dimensional probability densities instead of one multidimensional one." 22 | ], 23 | "metadata": { 24 | "id": "WLtcR3DvbaEl" 25 | } 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "source": [ 30 | "### **The main idea**\n", 31 | "\n", 32 | "In this case, a one-dimensional probability density is an estimate of the probability of each feature separately, provided they are independent, and a multidimensional one is an estimate of the probability of a combination of all features, which follows from the case of their dependence. It is for this reason that this classifier is called naive, since it greatly simplifies calculations and increases the efficiency of the algorithm. However, this assumption is not always true in practice and in some cases can lead to a significant deterioration in the quality of predictions.\n", 33 | "\n", 34 | "The Bayes formula itself looks like this:\n", 35 | "\n", 36 | "$$P(A|B) = \\frac{P(B|A) P(A)}{P(B)}$$\n", 37 | "\n", 38 | "where:\n", 39 | "- $P(A|B)$ is the a posterior probability of event A, provided event B is executed;\n", 40 | "\n", 41 | "- $P(B|A)$ is the conditional probability of event B, provided that event A is executed;\n", 42 | "\n", 43 | "- $P(A)$ and $P(B)$ are prior probabilities of events A and B respectively.\n", 44 | "\n", 45 | "And in the context of machine learning, the Bayes formula takes the following form:\n", 46 | "\n", 47 | "$$P(y_k|X) = \\frac{P(y_k)P(X|y_k)}{P(X)}$$\n", 48 | "\n", 49 | "where:\n", 50 | "- $P(y_k|X)$ is a posterior probability of the sample belonging to the class $y_k$, taking into account its features $X$;\n", 51 | "- $P(X|y_k)$ is likelihood, that is, the probability of features $X$ for a given class $y_k$;\n", 52 | "- $P(y_k)$ is the a prior probability that a randomly selected observation belongs to the class $y_k$;\n", 53 | "- $P(X)$ is the a prior probability of features $X$.\n", 54 | "\n", 55 | "If an object is described by not one, but several features $X_1, X_2, ..., X_n$, then the formula takes the form:\n", 56 | "\n", 57 | "$$P(y_k|X_1, X_2, ..., X_n) = \\frac{P(y_k)\\prod_{i=1}^n P(X_i|y_k)}{P(X_1, X_2, ..., X_n)}$$\n", 58 | "\n", 59 | "In practice, the numerator of this formula is of the greatest interest, since the denominator depends only on the features, not on the class, and therefore it is often omitted when comparing the probabilities of different classes. Ultimately, the classification rule will be proportional to the choice of the class with the maximum a posterior probability:\n", 60 | "\n", 61 | "$$y_k \\propto \\arg\\max_{y_k} P(y_k)\\prod_{i=1}^n P(X_i|y_k)$$\n", 62 | "\n", 63 | "To estimate the parameters of the model, that is, the probabilities $P(y_k)$ and $P(X_i|y_k)$, the maximum likelihood method is usually used, which is based on the frequency of occurrence of classes and features in the training dataset.\n", 64 | "\n", 65 | "### **Naive Bayes varieties**\n", 66 | "The scikit-learn library has several implementations of the naive Bayes classifier, differing in assumptions about the distribution of features for a given class. These include the following:\n", 67 | "\n", 68 | "- **Gaussian naive Bayes classifier (GaussianNB)** is an option for working with continuous features that have a normal (Gaussian) distribution. The probability of a feature for a given class is calculated using the formula: $$P(x_i|y) =\\frac{1}{\\sqrt{2\\pi\\sigma_y^2}}\\exp\\left(-\\frac{(x_i-\\mu_y)^2}{2\\sigma_y^2}\\right)$$ where $\\mu_y$ and $\\sigma_y$ are the mean and standard deviations of the feature in the class $y$. These parameters are estimated using the maximum likelihood method based on training dataset.\n", 69 | "\n", 70 | "- **The Multinomial naive Bayes classifier (MultinomialNB)** is an option for working with discrete features that have a multinomial distribution. Such features are often found in text classification tasks, where they represent the number of occurrences in the text. The probability of a feature for a given class is calculated using the formula: $$P(x_i|y) =\\frac{N_{yi} +\\alpha}{N_y + \\alpha n}$$ where $N_{yi}$ is the number of times the feature $i$ occurs in the class $y$; $N_y$ is the total number of all features in the class $y$; $n$ is the number of different features; and $\\alpha$ is a smoothing parameter that prevents the occurrence of zero probabilities.\n", 71 | "\n", 72 | "- **Complement Naive Bayes classifier (ComplementNB)** is an improved version of *MultinomialNB*, suitable for unbalanced datasets. Instead of estimating the probability of a feature for a given class, the algorithm evaluates the normalized weight of the feature $w_{ci}$ for the class $c$ as the probability of the feature when the class is complemented, that is, for all other classes. Thus, the algorithm takes into account not only the frequency of features in the class, but also their absence in other classes, which makes it less sensitive to sampling bias. The formula for calculating the probability of a feature when complementing a class is as follows: \\begin{align}\\begin{aligned}\\hat{\\theta}_{ci} = \\frac{\\alpha_i + \\sum_{j:y_j \\neq c} d_{ij}}\n", 73 | " {\\alpha + \\sum_{j:y_j \\neq c} \\sum_{k} d_{kj}}\\\\w_{ci} = \\log \\hat{\\theta}_{ci}\\\\w_{ci} = \\frac{w_{ci}}{\\sum_{j} |w_{cj}|}\\end{aligned}\\end{align}\n", 74 | "where is $\\hat{\\theta}_{ci}$ is an estimate of the probability of the feature $i$ when complementing the class $c$, which is calculated using the smoothing parameter $\\alpha_i$ and the frequency of the feature $i$ in all classes except $c$ (in this case $d_{ij}$ is the number of times when the feature $i$ occurs in the $j$ class); $w_{ci}$ is the normalized weight of the feature $i$ for the $c$ class. The predicted class $\\hat c$ for a given feature vector $t$ will look like this: $$\\hat{c} = \\arg\\min_c \\sum_{i} t_i w_{ci}$$\n", 75 | "\n", 76 | "- **Bernoulli Naive Bayes classifier (BernoulliNB)** is another option for working with discrete features, but which have a Bernoulli distribution. In this case, the features are binary indicators of the presence or absence of certain properties in an object. For example, in a text classification task, this may be the presence or absence of certain words in the text. The probability of a feature for a given class is calculated using the formula: $$P(x_i|y) = P(x_i = 1|y)x_i + (1-P(x_i = 1|y))(1-x_i)$$ where $P(x_i = 1|y)$ is the probability that the feature $i$ takes the value 1 (true), provided that the object belongs to the class $y$; $x_i$ is the value of the feature $i$ (0 or 1).\n", 77 | "\n", 78 | "- **Categorical Naive Bayes classifier (CategoricalNB)** is an option for categorically distributed data based on the assumption that each feature described by the index has its own categorical distribution. The probability of a feature for a given class is calculated using the formula: $$P(x_i = t \\mid y = c \\: ;\\, \\alpha) = \\frac{ N_{tic} + \\alpha}{N_{c} + \\alpha n_i}$$ where $N_{tic} =|\\{j \\in J\\mid x_{ij} =t, y_j =c\\}|$ is the number of times the feature $x_i$ takes the value $t$ in the class $c$; $N_{c} = |\\{j \\in J\\mid y_j = c\\}|$ is the total number of all features in the class $c$ in the training dataset; $\\alpha$ is a smoothing parameter; $n_i$ is the number of available values of the feature $i$.\n", 79 | "\n", 80 | "### **The principle of Naive Bayes classifier operation with a Gaussian distribution**\n", 81 | "The algorithm is constructed as follows:\n", 82 | "- 1) the a prior probabilities of classes are initially calculated;\n", 83 | "- 2) after that, the average and standard deviations of features by class are calculated;\n", 84 | "- 3) based on the obtained deviations of features by classes, the probabilistic density of test features is calculated according to the Gaussian distribution;\n", 85 | "- 4) next, the posterior probabilities are calculated as the product of the prior probabilities of the classes and the probabilistic densities of the test features;\n", 86 | "- 5) classes with the maximum a posterior probability will be the final prediction." 87 | ], 88 | "metadata": { 89 | "id": "YSjH9spRGuQo" 90 | } 91 | }, 92 | { 93 | "cell_type": "markdown", 94 | "source": [ 95 | "### **Naive Bayes in Spam filtering tasks**\n", 96 | "In the context of spam filtering, Naive Bayes classifier is based on the frequency of occurrence of words in messages for spam and non-spam and maximizing the product of their probabilities. Naivety in this case will consist in the assumption that the words in the message are independent of the order and context. Then the Bayes formula takes the following form:\n", 97 | "\n", 98 | "$$P(C|M) \\propto P(C) \\prod_{i=1}^n P(w_i|C), \\ \\ w_i \\in M$$\n", 99 | "\n", 100 | "where:\n", 101 | "- C — class spam or not spam;\n", 102 | "- M — message;\n", 103 | "- w_i — the i-th word in the message M;\n", 104 | "- $\\propto$ — the proportionality sign.\n", 105 | "\n", 106 | "For a better understanding, consider the following example. Suppose we want to classify the message **\"Hi, you won a discount and you can get the prize this evening.\"** and we have a training dataset consisting of the following messages:\n", 107 | "\n", 108 | "| Message | Class |\n", 109 | "| --- | --- |\n", 110 | "| Hi, how are you? | Not spam |\n", 111 | "| Congratulations, you won a prize! | Spam |\n", 112 | "| Buy the product now and get a discount! | Spam |\n", 113 | "| Let's walk this evening. | Not spam |\n", 114 | "\n", 115 | "The first step is to calculate the frequency of occurrence of all unique words and their total number in spam and non-spam messages. Then the probability of encountering each word in spam and non-spam messages is calculated based on these frequencies. When there are words in the message that were not previously found in the training dataset, smoothing is used. There are many different types of smoothing, but the essence of the simplest of them is to add $1$ when calculating the frequency of words in messages. This technique avoids the zero probability problem. Below is a table with the calculation of probabilities for all words.\n", 116 | "\n", 117 | "| Word | Frequency in Not Spam | Frequency in Spam | Probability in Not Spam | Probability in Spam |\n", 118 | "| --- | --- | --- | --- | --- |\n", 119 | "| Hi | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 120 | "| how | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 121 | "| are | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 122 | "| you | 1 + $1$ = 2 | 1 + $1$ = 2 | 2 / 28 = 0.0714 | 2 / 33 = 0.06 |\n", 123 | "| Congratulations | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 124 | "| won | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 125 | "| a | 0 + $1$ = 1 | 2 + $1$ = 3 | 1 / 28 = 0.0357 | 3 / 33 = 0.09 |\n", 126 | "| prize | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 127 | "| Buy | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 128 | "| the | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 129 | "| product | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 130 | "| now | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 131 | "| and | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 132 | "| get | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 133 | "| discount | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 134 | "| Let's | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 135 | "| walk | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 136 | "| this | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 137 | "| evening | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 138 | "| can | 0 + $1$ = 1 | 0 + $1$ = 1 | 1 / 28 = 0.0357 | 1 / 33 = 0.03 |\n", 139 | "| **Total amount of words** | **28** | **33** |\n", 140 | "\n", 141 | "At the end, the probabilities of the message being spam or not spam are calculated, and the final prediction will be the class with the highest probability.\n", 142 | "\n", 143 | "$P(C|M) = P(C) \\cdot P('Hi'|C) \\cdot P('you'|C) \\cdot P('won'|C) \\cdot P('a'|C)\n", 144 | "\\cdot P('discount'|C) \\cdot P('and'|C) \\cdot P('you'|C) \\cdot P('can'|C) \\cdot P('get'|C)\n", 145 | "\\cdot P('the'|C) \\cdot P('prize'|C) \\cdot P('this'|C) \\cdot P('evening'|C)$\n", 146 | "\n", 147 | "Where:\n", 148 | "- $C \\in (Spam, \\ \\ Not \\ \\ Spam)$;\n", 149 | "- $P(Spam) = P(Not \\ \\ Spam) = \\frac{2}{4} = 0.5$\n" 150 | ], 151 | "metadata": { 152 | "id": "Fs8ykp-9SXK1" 153 | } 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "source": [ 158 | "$P(Spam|M) = 0.5 \\cdot 0.03 \\cdot 0.06 \\cdot 0.06 \\cdot 0.09 \\cdot 0.06 \\cdot 0.06 \\cdot 0.06 \\cdot 0.03 \\cdot 0.06 \\cdot 0.06 \\cdot 0.06 \\cdot 0.03 \\cdot 0.03 \\approx 6.12 \\cdot 10^{-18}$\n", 159 | "\n", 160 | "$P(Not \\ \\ Spam|M) = 0.5 \\cdot 0.0714 \\cdot 0.0714 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0714 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0714 \\cdot 0.0714 \\approx 2.45 \\cdot 10^{-18}$\n", 161 | "\n", 162 | "$P(Spam|M) > P(Not \\ \\ Spam|M) \\rightarrow$ **the message is spam**\n", 163 | "\n", 164 | "It is worth adding that in practice, for convenience of calculations, the logarithm of probability is often used instead of the probability itself." 165 | ], 166 | "metadata": { 167 | "id": "GbXuXUE4UOrV" 168 | } 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "source": [ 173 | "### **Python implementation from scratch**" 174 | ], 175 | "metadata": { 176 | "id": "aUlIJ0VtU7Dc" 177 | } 178 | }, 179 | { 180 | "cell_type": "code", 181 | "source": [ 182 | "import numpy as np\n", 183 | "import pandas as pd\n", 184 | "import matplotlib.pyplot as plt\n", 185 | "from sklearn.datasets import load_iris\n", 186 | "from sklearn.naive_bayes import GaussianNB\n", 187 | "from sklearn.metrics import accuracy_score\n", 188 | "from sklearn.preprocessing import LabelEncoder\n", 189 | "from sklearn.model_selection import train_test_split\n", 190 | "from mlxtend.plotting import plot_decision_regions" 191 | ], 192 | "metadata": { 193 | "id": "vT1i1YbjVB1f" 194 | }, 195 | "execution_count": 30, 196 | "outputs": [] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "source": [ 201 | "class GaussianNaiveBayes:\n", 202 | " def fit(self, X, y):\n", 203 | " classes, cls_counts = np.unique(y, return_counts=True)\n", 204 | " n_classes = len(classes)\n", 205 | " self.priors = cls_counts / len(y)\n", 206 | "\n", 207 | " # calculate the mean and standard deviations of features by classes\n", 208 | " self.X_cls_mean = np.array([np.mean(X[y == c], axis=0) for c in range(n_classes)])\n", 209 | " self.X_stds = np.array([np.std(X[y == c], axis=0) for c in range(n_classes)])\n", 210 | "\n", 211 | " # calculate the probability density of the feature according to the Gaussian distribution\n", 212 | " def pdf(self, x, mean, std):\n", 213 | " return (1 / (np.sqrt(2 * np.pi) * std)) * np.exp(-0.5 * ((x - mean) / std) ** 2)\n", 214 | "\n", 215 | " def predict(self, X):\n", 216 | " pdfs = np.array([self.pdf(x, self.X_cls_mean, self.X_stds) for x in X])\n", 217 | " posteriors = self.priors * np.prod(pdfs, axis=2) # shorten Bayes formula\n", 218 | "\n", 219 | " return np.argmax(posteriors, axis=1)" 220 | ], 221 | "metadata": { 222 | "id": "_6JPBgjGVGq8" 223 | }, 224 | "execution_count": 31, 225 | "outputs": [] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "source": [ 230 | "def decision_boundary_plot(X, y, X_train, y_train, clf, feature_indexes, title=None):\n", 231 | " feature1_name, feature2_name = X.columns[feature_indexes]\n", 232 | " X_feature_columns = X.values[:, feature_indexes]\n", 233 | " X_train_feature_columns = X_train[:, feature_indexes]\n", 234 | " clf.fit(X_train_feature_columns, y_train)\n", 235 | "\n", 236 | " plot_decision_regions(X=X_feature_columns, y=y.values, clf=clf)\n", 237 | " plt.xlabel(feature1_name)\n", 238 | " plt.ylabel(feature2_name)\n", 239 | " plt.title(title)" 240 | ], 241 | "metadata": { 242 | "id": "lL23Df3qVJ0R" 243 | }, 244 | "execution_count": 32, 245 | "outputs": [] 246 | }, 247 | { 248 | "cell_type": "markdown", 249 | "source": [ 250 | "### **Uploading a dataset**\n", 251 | "[Iris dataset](https://www.kaggle.com/datasets/himanshunakrani/iris-dataset) will be used to train models, where it is necessary to correctly determine the types of flowers based on their characteristics." 252 | ], 253 | "metadata": { 254 | "id": "OAJUvHTnVP-s" 255 | } 256 | }, 257 | { 258 | "cell_type": "code", 259 | "source": [ 260 | "df_path = \"/content/drive/MyDrive/iris.csv\"\n", 261 | "iris = pd.read_csv(df_path)\n", 262 | "X1, y1 = iris.iloc[:, :-1], iris.iloc[:, -1]\n", 263 | "y1 = pd.Series(LabelEncoder().fit_transform(y1))\n", 264 | "X1_train, X1_test, y1_train, y1_test = train_test_split(X1.values, y1.values, random_state=0)\n", 265 | "print(iris)\n" 266 | ], 267 | "metadata": { 268 | "colab": { 269 | "base_uri": "https://localhost:8080/" 270 | }, 271 | "id": "SyvzlVvhVQ2a", 272 | "outputId": "de395158-bc96-40cb-88a3-91d38d4d4817" 273 | }, 274 | "execution_count": 33, 275 | "outputs": [ 276 | { 277 | "output_type": "stream", 278 | "name": "stdout", 279 | "text": [ 280 | " sepal_length sepal_width petal_length petal_width species\n", 281 | "0 5.1 3.5 1.4 0.2 setosa\n", 282 | "1 4.9 3.0 1.4 0.2 setosa\n", 283 | "2 4.7 3.2 1.3 0.2 setosa\n", 284 | "3 4.6 3.1 1.5 0.2 setosa\n", 285 | "4 5.0 3.6 1.4 0.2 setosa\n", 286 | ".. ... ... ... ... ...\n", 287 | "145 6.7 3.0 5.2 2.3 virginica\n", 288 | "146 6.3 2.5 5.0 1.9 virginica\n", 289 | "147 6.5 3.0 5.2 2.0 virginica\n", 290 | "148 6.2 3.4 5.4 2.3 virginica\n", 291 | "149 5.9 3.0 5.1 1.8 virginica\n", 292 | "\n", 293 | "[150 rows x 5 columns]\n" 294 | ] 295 | } 296 | ] 297 | }, 298 | { 299 | "cell_type": "markdown", 300 | "source": [ 301 | "### **Model training and evaluation of the obtained results**\n", 302 | "Despite its simplicity, in this case the algorithm has showed an excellent result by classifying all the samples correctly, which is possible due to the construction of a flexible decision boundary. From this we can draw an interesting conclusion that in some situations simple models can work much better than complex ones, which can be seen later on the example of other algorithms." 303 | ], 304 | "metadata": { 305 | "id": "6mJoUO12ZYlY" 306 | } 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "source": [ 311 | "**Naive Bayes**" 312 | ], 313 | "metadata": { 314 | "id": "gi-kXxHhZyGZ" 315 | } 316 | }, 317 | { 318 | "cell_type": "code", 319 | "source": [ 320 | "nb_clf = GaussianNaiveBayes()\n", 321 | "nb_clf.fit(X1_train, y1_train)\n", 322 | "nb_clf_pred_res = nb_clf.predict(X1_test)\n", 323 | "nb_clf_accuracy = accuracy_score(y1_test, nb_clf_pred_res)\n", 324 | "\n", 325 | "print(f'Naive Bayes classifier accucacy: {nb_clf_accuracy}')\n", 326 | "print(nb_clf_pred_res)" 327 | ], 328 | "metadata": { 329 | "colab": { 330 | "base_uri": "https://localhost:8080/" 331 | }, 332 | "id": "HNy3mPlvYpq1", 333 | "outputId": "bfebbc89-a4f7-4d45-8b74-b378fff43177" 334 | }, 335 | "execution_count": 34, 336 | "outputs": [ 337 | { 338 | "output_type": "stream", 339 | "name": "stdout", 340 | "text": [ 341 | "Naive Bayes classifier accucacy: 1.0\n", 342 | "[2 1 0 2 0 2 0 1 1 1 2 1 1 1 1 0 1 1 0 0 2 1 0 0 2 0 0 1 1 0 2 1 0 2 2 1 0\n", 343 | " 1]\n" 344 | ] 345 | } 346 | ] 347 | }, 348 | { 349 | "cell_type": "markdown", 350 | "source": [ 351 | "**Naive Bayes (scikit-learn)**" 352 | ], 353 | "metadata": { 354 | "id": "vcUJmnaaZ1dF" 355 | } 356 | }, 357 | { 358 | "cell_type": "code", 359 | "source": [ 360 | "sk_nb_clf = GaussianNB()\n", 361 | "sk_nb_clf.fit(X1_train, y1_train)\n", 362 | "sk_nb_clf_pred_res = sk_nb_clf.predict(X1_test)\n", 363 | "sk_nb_clf_accuracy = accuracy_score(y1_test, sk_nb_clf_pred_res)\n", 364 | "\n", 365 | "print(f'sk Naive Bayes classifier accucacy: {sk_nb_clf_accuracy}')\n", 366 | "print(sk_nb_clf_pred_res)\n", 367 | "\n", 368 | "feature_indexes = [2, 3]\n", 369 | "title1 = 'GaussianNB surface'\n", 370 | "decision_boundary_plot(X1, y1, X1_train, y1_train, sk_nb_clf, feature_indexes, title1)" 371 | ], 372 | "metadata": { 373 | "colab": { 374 | "base_uri": "https://localhost:8080/", 375 | "height": 524 376 | }, 377 | "id": "OFUE33eLZ2DV", 378 | "outputId": "1d38e397-4c87-4088-ac35-932725c2ad5c" 379 | }, 380 | "execution_count": 35, 381 | "outputs": [ 382 | { 383 | "output_type": "stream", 384 | "name": "stdout", 385 | "text": [ 386 | "sk Naive Bayes classifier accucacy: 1.0\n", 387 | "[2 1 0 2 0 2 0 1 1 1 2 1 1 1 1 0 1 1 0 0 2 1 0 0 2 0 0 1 1 0 2 1 0 2 2 1 0\n", 388 | " 1]\n" 389 | ] 390 | }, 391 | { 392 | "output_type": "display_data", 393 | "data": { 394 | "text/plain": [ 395 | "
" 396 | ], 397 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHHCAYAAABTMjf2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACLJElEQVR4nOzdd1wUV9cH8N9spVfpiKKgYEEsQcHee02MGhVLNE0TjW9iYopGE8Xy2KLGkkQxGmOiscWOiiV2scReEMRCkd63zbx/EDau7FLWXWaXPd/PZ59HZu7MPXeJcrh77xmG4zgOhBBCCCEWSMB3AIQQQgghfKFEiBBCCCEWixIhQgghhFgsSoQIIYQQYrEoESKEEEKIxaJEiBBCCCEWixIhQgghhFgsSoQIIYQQYrEoESKEEEKIxaJEiBDCq+joaDAMg8TERL5DMVtKpRLTp09H7dq1IRAIMGjQIL5DIsRsUCJEiBlLSEjA5MmT0aBBA9jY2MDGxgaNGjXCpEmT8M8///AdnslITEwEwzBgGAZ//vlnmfPffPMNGIZBenq6+tjYsWPV1zAMA5FIhNq1a2P48OG4detWdYZfofXr12PRokV44403sHHjRnz88cd8h0SI2RDxHQAhRD979+7FsGHDIBKJMHLkSDRr1gwCgQB37tzBjh07sHr1aiQkJKBOnTp8h1qu0aNHY/jw4ZBKpdXS35w5czBkyBAwDFNhW6lUip9++glAyaxLfHw81qxZg4MHD+LWrVvw9vY2driVcuzYMfj4+GDp0qV8h0KI2aFEiBAzFB8fj+HDh6NOnTo4evQovLy8NM4vWLAAP/zwAwQC05/0FQqFEAqF1dJXaGgorl69ip07d2LIkCEVtheJRBg1apTGsTZt2qBfv37Yt28fJk6caKxQK8RxHIqLi2FtbY20tDQ4OTnxFgsh5sz0/5UkhJSxcOFCFBQUYMOGDWWSIKDkB/hHH32E2rVrq4/9888/GDt2LOrVqwcrKyt4enpi/PjxyMjI0Lh27NixqFu3bpl7ln589KKYmBi0a9cOTk5OsLOzQ8OGDfHFF19otFmxYgUaN24MGxsbODs7o1WrVtiyZYv6vLY1Qrt370bfvn3h7e0NqVSK+vXr49tvv4VKpdK4d6dOndCkSRPcunULnTt3ho2NDXx8fLBw4UKt79vw4cPRoEEDzJkzBxzHaW1TEU9PTwAl73FFtm7dipYtW8Le3h4ODg5o2rQpli9frj6v7T0FtL8ndevWRb9+/XDo0CG0atUK1tbWWLt2LRiGQWxsLG7evKn+GO/48eMAgP/973+IiIiAq6srrK2t0bJlS2zfvl1rrJs3b0ZYWJj6+9ShQwccPnxYo82BAwfQvn172Nrawt7eHn379sXNmzcrfB8IMWWUCBFihvbu3YuAgAC0bt260tfExMTg4cOHGDduHFasWIHhw4dj69at6NOnj15Jwc2bN9GvXz/IZDLMmTMHixcvxoABA3D69Gl1mx9//BEfffQRGjVqhGXLlmH27NkIDQ3F+fPny713dHQ07OzsMG3aNCxfvhwtW7bEzJkz8fnnn5dpm5WVhV69eqFZs2ZYvHgxgoKC8Nlnn+HAgQNl2gqFQnz11Ve4du0adu7cWalxpqenIz09HampqTh79iw+/vhjuLq6ol+/fuVeFxMTgxEjRsDZ2RkLFizA/Pnz0alTJ433p6ru3r2LESNGoHv37li+fDkCAwOxadMmBAUFwdfXF5s2bcKmTZsQHBwMAFi+fDmaN2+OOXPmYN68eRCJRBg6dCj27duncd/Zs2dj9OjREIvFmDNnDmbPno3atWvj2LFj6jabNm1C3759YWdnhwULFuDrr7/GrVu30K5dO1roTswbRwgxKzk5ORwAbtCgQWXOZWVlcc+fP1e/CgsL1ede/HOp3377jQPAnTx5Un1szJgxXJ06dcq0nTVrFvfiPxlLly7lAHDPnz/XGevAgQO5xo0blzueDRs2cAC4hISEcmN99913ORsbG664uFh9rGPHjhwA7pdfflEfk8lknKenJ/f666+rjyUkJHAAuEWLFnFKpZILDAzkmjVrxrEsqzG2F8cyZswYDkCZl4+PDxcXF1fumDiO46ZMmcI5ODhwSqVSZ5uX39Py3pM6depwALiDBw+Wad+xY0et7/PL76NcLueaNGnCdenSRX3s/v37nEAg4AYPHsypVCqN9qXvT15eHufk5MRNnDhR43xKSgrn6OhY5jgh5oRmhAgxM7m5uQAAOzu7Muc6deoENzc39WvVqlXqc9bW1uo/FxcXIz09HW3atAEAXL58ucpxlK5J2b17N1iW1dnmyZMnuHjxYpXu/WKseXl5SE9PR/v27VFYWIg7d+5otLWzs9NYxyORSBAWFoaHDx9qvfeLs0K7du0qNw4rKyvExMQgJiYGhw4dwtq1a2FnZ4c+ffrg3r175V7r5OSEgoICxMTEVDDayvP390fPnj0r3f7F9zErKws5OTlo3769xvd7165dYFkWM2fOLLOmrPRju5iYGGRnZ2PEiBHqGbL09HQIhUK0bt0asbGxrzgyQvhDiRAhZsbe3h4AkJ+fX+bc2rVrERMTg82bN5c5l5mZiSlTpsDDwwPW1tZwc3ODv78/ACAnJ6fKcQwbNgxt27bFhAkT4OHhgeHDh+OPP/7QSIo+++wz2NnZISwsDIGBgZg0aVKlPhq6efMmBg8eDEdHRzg4OMDNzU2d7Lwcq6+vb5l1Ns7OzsjKytJ5/5EjRyIgIKDCtUJCoRDdunVDt27d0KNHD7zzzjs4cuQIcnJyMGPGjHLH8MEHH6BBgwbo3bs3fH19MX78eBw8eLCioZer9PtVWXv37kWbNm1gZWUFFxcXuLm5YfXq1RrvYXx8PAQCARo1aqTzPvfv3wcAdOnSRSPRdnNzw+HDh5GWlqbfgAgxAbRrjBAz4+joCC8vL9y4caPMudI1Q9rWbLz55ps4c+YMPv30U4SGhsLOzg4sy6JXr14ayYuubeUvL1S2trbGyZMnERsbi3379uHgwYP4/fff0aVLFxw+fBhCoRDBwcG4e/cu9u7di4MHD+LPP//EDz/8gJkzZ2L27Nla+8nOzkbHjh3h4OCAOXPmoH79+rCyssLly5fx2WeflZl90rXjrKIE56uvvsLYsWOxe/dune208fX1RcOGDXHy5Mly27m7u+Pq1as4dOgQDhw4gAMHDmDDhg2IjIzExo0bAVT+vS714gxPRU6dOoUBAwagQ4cO+OGHH+Dl5QWxWIwNGzZoLFavjNL3fNOmTerF4i+qzMJxQkwV/ddLiBnq27cvfvrpJ1y4cAFhYWEVts/KysLRo0cxe/ZszJw5U3289Df9Fzk7OyM7O7vM8UePHpU5JhAI0LVrV3Tt2hVLlizBvHnz8OWXXyI2NhbdunUDANja2mLYsGEYNmwY5HI5hgwZgrlz52LGjBmwsrIqc8/jx48jIyMDO3bsQIcOHdTHExISKhxnVYwaNQrfffcdZs+ejQEDBlTpWqVSqXVG7mUSiQT9+/dH//79wbIsPvjgA6xduxZff/01AgIC4OzsDKAk+Xtx+7u297qq/vzzT1hZWeHQoUMaNZo2bNig0a5+/fpgWRa3bt1CaGio1nvVr18fQElyV/p9JaSmoI/GCDFD06dPh42NDcaPH4/U1NQy51+eDSmdNXn5+LJly8pcW79+feTk5GhUpk5OTi6zyyozM7PMtaU/SGUyGQCU2ZovkUjQqFEjcBwHhUKhdWzaYpXL5fjhhx+0ttdX6azQ1atXsWfPnkpfd+/ePdy9exfNmjUrt93LYxcIBAgJCQHw3/tTmmC8OLtUUFCgnjF6FUKhEAzDaMwuJSYmllkXNWjQIAgEAsyZM6fMbFvp96Bnz55wcHDAvHnztH7fnj9//srxEsIXmhEixAwFBgZiy5YtGDFiBBo2bKiuLM1xHBISErBlyxYIBAL4+voCABwcHNChQwcsXLgQCoUCPj4+OHz4sNZZluHDh+Ozzz7D4MGD8dFHH6GwsBCrV69GgwYNNBbZzpkzBydPnkTfvn1Rp04dpKWl4YcffoCvry/atWsHAOjRowc8PT3Rtm1beHh44Pbt21i5ciX69u2rXuv0soiICDg7O2PMmDH46KOPwDAMNm3apHfdn/KMHDkS3377La5evar1vFKpVK+3YlkWiYmJWLNmDViWxaxZs8q994QJE5CZmYkuXbrA19cXjx49wooVKxAaGqre3t6jRw/4+fnh7bffxqeffgqhUIj169fDzc0NSUlJrzS2vn37YsmSJejVqxfeeustpKWlYdWqVQgICNBIcgMCAvDll1/i22+/Rfv27TFkyBBIpVJcvHgR3t7eiIqKgoODA1avXo3Ro0ejRYsWGD58uDrGffv2oW3btli5cuUrxUsIb3jbr0YIeWUPHjzg3n//fS4gIICzsrLirK2tuaCgIO69997jrl69qtH2yZMn3ODBgzknJyfO0dGRGzp0KPfs2TMOADdr1iyNtocPH+aaNGnCSSQSrmHDhtzmzZvLbPU+evQoN3DgQM7b25uTSCSct7c3N2LECO7evXvqNmvXruU6dOjAubq6clKplKtfvz736aefcjk5Oeo22raKnz59mmvTpg1nbW3NeXt7c9OnT+cOHTrEAeBiY2PV7XRtG3+5BMCL2+dfVto/KrF93sHBgevatSt35MgRnd+TUtu3b+d69OjBubu7cxKJhPPz8+PeffddLjk5WaNdXFwc17p1a3WbJUuW6Nw+37dvX6196Xoffv75Zy4wMJCTSqVcUFAQt2HDBp1b9tevX881b96ck0qlnLOzM9exY0cuJiZGo01sbCzXs2dPztHRkbOysuLq16/PjR07lrt06VKF7wchporhOCP8mkUIIYQQYgZojRAhhBBCLBYlQoQQQgixWJQIEUIIIcRimU0itHr1aoSEhMDBwQEODg4IDw/X+lDFUqVPb37xpa1mCSGEEEIsl9lsn/f19cX8+fMRGBgIjuOwceNGDBw4EFeuXEHjxo21XuPg4IC7d++qv9ZVxZUQQgghlslsEqH+/ftrfD137lysXr0a586d05kIMQyjtRw8IYQQQghgRh+NvUilUmHr1q0oKChAeHi4znb5+fmoU6cOateujYEDB+LmzZsV3lsmkyE3N1fjVVoFlhBCCCE1i9nMCAHA9evXER4ejuLiYtjZ2WHnzp06n5jcsGFDrF+/HiEhIcjJycH//vc/RERE4ObNm+pqu9pERUWVeRjk6+8PxBuTBht0LIQQQggxHqlQisFBwytsZ1YFFeVyOZKSkpCTk4Pt27fjp59+wokTJ3QmQy9SKBQIDg7GiBEj8O233+psJ5PJyswA7Y7/A2KJ+JXjJ4QQQkj1qGwiZFYzQhKJBAEBAQCAli1b4uLFi1i+fDnWrl1b4bVisRjNmzfHgwcPym0nlUo1ntQMgJIgQgghpIYyyzVCpViWrfT6HZVKhevXr8PLy8vIURFCCCHEXJjNjNCMGTPQu3dv+Pn5IS8vD1u2bMHx48dx6NAhAEBkZCR8fHwQFRUFoOTJ2G3atEFAQACys7OxaNEiPHr0CBMmTOBzGIQQQggxIWaTCKWlpSEyMhLJyclwdHRESEgIDh06hO7duwMAkpKSIBD8N8GVlZWFiRMnIiUlBc7OzmjZsiXOnDlTqfVEhBBCCLEMZrVYmi9bb27kOwRCCCGk6jhAYN6rYHRiwQLl1EmukYulCSGEEFIJHCDhrGAjsIWQEYIpL2MwQxw4qDgVCtkCyJnichOiilAiRAghhNQwEs4KTmJniCVicAwHoKZ9+MOA4RhI5BJkK7JKkiE9USJECCGE1CQcYCOwLUmChCzf0RgJBw4cxBIxbFS2kHP6zwrVzA8OCSGEEAslgABCRvjvTFDNxjEchIzwldZBUSJECCGE1DAla4JqfiIEcK+8/okSIUIIIYRYLEqECCGEEGKxKBEihBBCiMnY+ctuDGs7Et0b9MZ7Ayfj9tU7Ru2Pdo0RQgghREPq01QUF+l+lqeVtRQePh4G7/fYX7FY9d0aTPtuCho1D8a29X/ik8jPsfnYBjjXcjZ4fwAlQoQQQgh5QerTVEyf8A1kCt1tpGJg4U/fGDwZ+uOnP9FveB/0ebMXAOD/5k7FuWPnsf+Pgxj5wQiD9lWKEiFCCCGEqBUXySBTAK4dR8HatWyiU5SRiowTm8udMdKHQq7AvRv3NBIegUCAlm1b4OblWwbt60WUCBFCCCGkDGtXD9i41662/nKycqBSsWU+AnN2c0ZS/GOj9UuLpQkhhBBisSgRIoQQQgjvHJ0dIRQKkJWepXE863kWXNyMs1AaoESIEEIIISZALBGjQZMGiDtzWX2MZVlcPnMFjVs0Mlq/tEaIEEIIISbhzQmvI+r/FiKoaUMEhTbE9p93oKiwGL2H9jJan5QIEUIIIaSMoozUKh03hC79OyM7Mwfrl0Yj83kWAoLrY9HGKKN+NEaJECGEEELUrKylkIqBjBObdbaRikvaGcOQMYMwZMwgo9xbG0qECCGEEKLm4eOBhT99w0tlaT5QIkQIIYQQDTUlyakM2jVGCCGEEItFiRAhhBBCLBYlQoQQQgixWJQIEUIIIcRiUSJECCGEEItFiRAhhBBCLBYlQoQQQgixWJQIEUIIIcRiUSJECCGEEJNw7fw/+PztrzAkbBg61u2GU4dOG71PSoQIIYQQohPHcUi8/wgcxxm9r6LCYgQE18PUOR8ava9S9IgNQgghhOh0/vgFRC9ch7HT30Gbzq2N2lebzmFo0znMqH28zGxmhFavXo2QkBA4ODjAwcEB4eHhOHDgQLnXbNu2DUFBQbCyskLTpk2xf//+aoqWEEIIMX8qlQr7f90NaUE69v+6GyqViu+QDM5sEiFfX1/Mnz8fcXFxuHTpErp06YKBAwfi5s2bWtufOXMGI0aMwNtvv40rV65g0KBBGDRoEG7cuFHNkRNCCCHm6eLJS8hITMCn3d2RkZiAiycv8R2SwZlNItS/f3/06dMHgYGBaNCgAebOnQs7OzucO3dOa/vly5ejV69e+PTTTxEcHIxvv/0WLVq0wMqVK6s5ckIIIcT8lM4GdawtQP9mTuhQW1AjZ4XMJhF6kUqlwtatW1FQUIDw8HCtbc6ePYtu3bppHOvZsyfOnj1b7r1lMhlyc3M1Xgq5wmCxE0IIIeagdDZoXIQLAGBcuEuNnBUyq0To+vXrsLOzg1QqxXvvvYedO3eiUaNGWtumpKTAw8ND45iHhwdSUlLK7SMqKgqOjo4ar90/7jXYGAghhBBT9+JsUAMPKwBAQ0+rGjkrZFaJUMOGDXH16lWcP38e77//PsaMGYNbt24ZtI8ZM2YgJydH4zVwYj+D9kEIIYSYspdng0oZe1aosKAI928+wP2bDwAAyY+Tcf/mA6Q+TTVKf4CZbZ+XSCQICAgAALRs2RIXL17E8uXLsXbt2jJtPT09kZqq+calpqbC09Oz3D6kUimkUqnGMbFE/IqRE0IIIeahdDaorQ+Duq4SyJWs+px/LQkifBjs/3U3XuvQCkKh0KB93/3nLqaO+ET99arv1gAAer3eAzMWTzdoX6XMKhF6GcuykMlkWs+Fh4fj6NGjmDp1qvpYTEyMzjVFhBBCCAEe3IrH8yfPkClXofPKJ1rbqCTP8OBWPBo2bWDQvpuHh+JE4hGD3rMiZpMIzZgxA71794afnx/y8vKwZcsWHD9+HIcOHQIAREZGwsfHB1FRUQCAKVOmoGPHjli8eDH69u2LrVu34tKlS1i3bh2fwyCEEEJMWr2G/nj7qw+hVCh1thGJRajX0L8aozIes0mE0tLSEBkZieTkZDg6OiIkJASHDh1C9+7dAQBJSUkQCP5b8hQREYEtW7bgq6++whdffIHAwEDs2rULTZo04WsIhBBCiMkTS8Ro1a4l32FUG4arjoeHmLmtNzfyHQIhhBBSKQJOABeBGyTWYjPbEqUHFpAXKZDJPgfLsBqnpEIpBgcNr/AWNf0tIoQQQgjRiRIhQgghhFgsSoQIIYQQYrEoESKEEEKIxaJEiBBCCCEWixIhQgghhFgsSoQIIYQQYrHMpqAiIYQQQmq2zau24OShv5EU/xhSKymatGiEdz+fCL/6tY3WJyVChBBCCNGKZVncv/kAOZk5cHRxRGDjAI2nOBjatfP/YPDogQhq1hAqpQo/LvoZn0R+ho0xP8PaxtoofVIiRAghhJAyLp+5gt9+/B1Pnz2FilNByAjh4+2DEROHoUVEc6P0ueiX+Rpfz/jfdAxs+QbuXb+PZq1DjNInrREihBBCiIbLZ65g2XffI9s2HY3fqY+IWaFo/E595NimY9l33+PymSvVEkd+XgEAwN7J3mh9UCJECCGEEDWWZfHbj79DWk+IZpFBcKpjD5FUCKc69giJDIJVPSF++/F3sCxb8c1eMY6Vc35A01aNjfqke0qECCGEEKJ2/+YDPH32FHU7+4IRMBrnGAGDOp188PTZU9y/+cCocSz9+nsk3E3EzBVfGbUfWiNECCGEELWczByoOBXsPG20nrfzsoWKUyEnM8doMSybuQJnj53Hij+WwN3LzWj9ADQjRAghhJAXOLo4QsgIkZ9SqPV8fnIBhIwQji6OBu+b4zgsm7kCpw79jWVbFsGrtpfB+3gZJUKEEEIIUQtsHAAfbx88in0CjuU0znEsh0fHn8LH2weBjQMM3vfSr79HzM4j+Hr5F7C2tUFGWiYy0jIhK5YZvK9SlAgRQgghRE0gEGDExGEofqjCP7/cQXZiLpQyFbITc/HPL3dQ/FCFEROHGaWe0O7NfyE/rwBThv8fhoS9qX4d++u4wfsqxXAcx1XczLJtvbmR7xAIIYSQShFwArgI3CCxFr/SdAcfdYSqjAXkRQpkss/BMpq72KRCKQYHDa/wFrRYmhBCCCFltIhojtA2zaq1sjQfKBEihBBCiFYCgQANmzbgOwyjqllpHSGEEEJIFVAiRAghhBCLRYkQIYQQUsNw4AAwFbYzf8y/Y9UfJUKEEEJIDcKChYpTgeFqfiLEcAxUnAos9H/uGS2WJoQQQmoSBihkCyCRSyCWiMExHPCKsyamhwHDMVDIFShkC15pWocSIUIIIaSGkTPFyFZkwUZlCyEjBFPDPibjwEHFqVDIFkDOFL/SvSgRIoQQQmoapiQZknPFEHA1cxUMC9YgC3woESKEEEJqKgavtH7GEtTMNJEQQgghpBIoESKEEEKIxaKPxgghhJgNlmWReDsJedl5sHeyR91gP72ffaXrXobsg5g+s0mEoqKisGPHDty5cwfW1taIiIjAggUL0LBhQ53XREdHY9y4cRrHpFIpiotfbYU5IYSQ6nfj/C3s3bgfqSlpYDkWAkYAD0939BvTB01aNzLIvZqENcaNCzcN0gcxD2aT4p44cQKTJk3CuXPnEBMTA4VCgR49eqCgoKDc6xwcHJCcnKx+PXr0qJoiJoQQYig3zt9C9KJNKLDPRaN3/NF6ZhM0escfBfa5iF60CTfO33rlez2XpWH7jzuRY5X5yn0Q82E2M0IHDx7U+Do6Ohru7u6Ii4tDhw4ddF7HMAw8PT2NHR4hhBAjYVkWezfuh3V9MZqMDgAjKKmJ4+hnhyajA3Bj0wPs3bgfjV4LqvAjLF33cvC1hUqhglOoHTy7OcPRz07vPoh5MdvvZk5ODgDAxcWl3Hb5+fmoU6cOateujYEDB+LmzZvltpfJZMjNzdV4KeQKg8VNCCGkahJvJyE1JQ1+nb3UiUspRsDAr5MnUlPSkHg7Se97ZSfmoSirGD6dXKFUKSErkuvdBzEvZpkIsSyLqVOnom3btmjSpInOdg0bNsT69euxe/dubN68GSzLIiIiAk+ePNF5TVRUFBwdHTVeu3/ca4xhEEIIqYS87DywHAtbD2ut5209bcByLPKy8/S+lzxXAQ4cbLxLjrMqld59EPNilonQpEmTcOPGDWzdurXcduHh4YiMjERoaCg6duyIHTt2wM3NDWvXrtV5zYwZM5CTk6PxGjixn6GHQAghpJLsnewhYAQoSC3Ser4gpRACRgB7J3u97yVxEIMBg8JnJccFQqHefRDzYnaJ0OTJk7F3717ExsbC19e3SteKxWI0b94cDx480NlGKpXCwcFB4yWWiF81bEIIIXqqG+wHD093JMUmg2M1Hx7KsRySjqfAw9MddYP99L6XU117WDtb4WlsBkRCEaTWEr37IObFbBIhjuMwefJk7Ny5E8eOHYO/v3+V76FSqXD9+nV4eXkZIUJCCKk5WJbFw5uJuHb6Oh7eTATLVu9jGpRKJU79dQY71/2F0/vOofeoniiKV+DGpgfIeZQHpUyFnEd5uLHpAYriFeg3pk+lFjELBAL0G9OnzL1yH+dDKBYi+1o+Uo5kvVIfxLwwHMdxFTfj3wcffIAtW7Zg9+7dGrWDHB0dYW1d8pluZGQkfHx8EBUVBQCYM2cO2rRpg4CAAGRnZ2PRokXYtWsX4uLi0KhR5etBbL250bCDIYQQE2bIej362Bt9AH9t3A+ZUgZGCHAqQCqS4rXOLZGekk51hEilSIVSDA4aXmE7s0mEGIbRenzDhg0YO3YsAKBTp06oW7cuoqOjAQAff/wxduzYgZSUFDg7O6Nly5b47rvv0Lx58yr1TYkQIcRSlNbYsa4vhl9nL9h6WKMgtQhJsckoildg7KejjZoQ7I0+gO0/7oRTqB18OrnCxtsahc+K8DQ2A9nX8vH6hEFoHNaIKkuTCtW4RIhPlAgRQiwBy7JYOHkJCuxzNWrsACXrZG5segDbPAdMXznNKImBUqnEpO4fw7qRGEFja4MRvtC/isOdDY9RdFuBVTFLIRKZTRk8wpPKJkKU4hJCCAFg2Ho9+jh74AJkShl8OrlqJEEAwAgZ+HR2hUwpw9kDF4zSP7FMlAgRQggBYNh6PfpIT84AI4S6ls/LbLytwQhL2hFiKJQIEUIIAWDYej36qOXlCk4FdS2flxU+KwKnKmlHiKFQIkQIIQSAYev16CO8dxikIimexmaAU73Uv4rD09gMSEVShPcOM0r/xDJRIkQIIQSA7ho71VVLRyQSof+YPsi+lo87Gx4jN74AyiIWufEFuLPhMbKv5aP/mD60UJoYFO0aqwTaNUYIsSQ3zt/CX9H78PTRMyiVKohEQvjU8Ub/sX3R6LUgnVvLDbUdXVcdof5j+qBPZM8q3UufrfA1Zft8TRmHviq7a4zSakIIIVowYIQMBAzz7w4yBom3H+kstAjAYAUK+43tjV6juuPsgQtIT85ALS9XhPcOw524e1g4eUml76VPYUi+i0kaSk0ZR3WgGaFKoBkhQoil0FVQ8c62BGTcyYZX61qo16O2RqHFnJsF4FQcnELsKn1NVYszVrXQoz6FIfkuJmkoNWUcr4oKKhrQ+M/e4jsEXtXydUeLLlWrxk0IMT+6CipyLIezC68B9hz8+3rBq66n+hqO5XB+1TXkPihEt8VtIBAJKnVNVYozVrXQoz6FIfkuJmkoNWUchkAfjRnQaF/L3qq57cwNfL/tBCRWkoobv6RJhxC0G9zOCFERQgyttKBiowH+Gj9AsxPzUJRVjID+3lCqlJAVydVPZ5fLFHBuboe8xCLkJOXDuZ5DhdeUFme89WMiEm8noV7junrFpeteVW2vTx+mqqaMozpRIlQJnVs24DsEXnVu2QByhbLK13Echy82HsHaD5bpfFZcKaVSBZ+Q+ggfGFFuOztHW9jY21Q5FkJIxXQVVJTnKsCBg423NYozZWBVKvU5VqWC1FUCRljSrjLXAFUrzljVQo/6FIbku5ikodSUcVQnSoRIpUjE+v2nsnhi70q3XbP3PC5HHyy3zZUHT9FqUFsIhUKt5xkBgxadQmFtq/0fAUKIbi8WVHT0s1MflziIwYBB4bMiCKwEELzw908gFEKWIQenKmlXmWuAqhVn1BWXrntVtb2+15iimjKO6kSJEDEZ7/VrXWGbtKw8xMbdB1767bJUdmExVk76Hj71vcucYwQCdB7dDZ5+Hq8cKyE10YsFFV9cX+JU1x7WzlZ4GpsB/35e6o+4AEAiFSPrSj44Jafxg7e8a6panFFXXLruVdX2+l5jimrKOKoTLZaujDMr+I6AVEF2XiHyi2Rlj+cXYfKa/eC0FGOTKxToNLIbGr4WVOac1FpS4Ud7hNQUGjuOOnnC1tMGBSmFuLM98b8dYN191ceTjqdo7hqr5DUV7V56uQZOQW4BNi7eXCau0nuN+b9RsHWwrXT7CneNVfKa6lTZukCmPo7qQrvGDIkSoRqvWKbAxz8eQHKm5ufmKpZFgViENoPaahx3dHFAnSD6jYrUTLpq0JRXEwgwXB2hqvZf1ePmWEeoqnGZ6jiqEyVChkSJkEWLvfIAVx8maxw7fj0RAk9X2NhrrkVqMzACbj5u1RkeIUahT5VoQ1SWrqgGTrkzP5Vob46VpfWtC2Rq46hulAgZEiVC5CUqFYvr8c/AvvDXp6BIjk83HIbYTjM5qlXbHW/831AIRdoXeBNCSlRHvSBzYwljNBaqI0SIEQmFAoQ28C1z/Nzy98sc+/3EP1j70QoI/v0HTKlQwSXAB637h6vbSKwkGgXnCLFE1VEvyNxYwhj5RokQIUY2rGMIhnUM0Tj267GruLr/vPrrB0+fg/N0hXvt/z5W86znhUZhwdUWJyF8q456QebGEsbIN0qECOHByC6hGPnC1xzH4eTVByiS/VeQbv22k/j79+MvfKTGIOL19ggOK7uzjZCaoDrqBZkbSxgj3ygRIsQEMAyDjs0DNY71bB2MvMJi9dcyuRIf/LAXp345DAAoKpbjtYERCG5TskhSKBTAwcWh+oImxMCqo16QubGEMfKNFktXBi2WJiZIqVRh1uajeJxRMiWe/Dwbzo3rwqdhydolKxsrhEQ0oRpIxKxUtQaOJdTMsYQxGgPtGjMkSoSIGeA4DjtP/IOcf2eR/klMw+W0bLh6lTw02N7VAT3H9oJASDtLSPWq6rb6G+dv4a/o/Xj25BlUrApCgRDevt7oP7YPGr0WVOaaWxfv6KyZo629Ke+uKu89sfS6QFVFu8YIsTAMw2BIp2Yax5JSMiFXljyO5Ni1BCyb+D9Y21gBAKyd7TF8xluQSMUAA5P+4UDMl77FGQEOnIoDq+QgEHEAOCTefqQzGZi+cprWBGnh5CVmkzxUlOyYW1JnLmhGqDJoRojUQDFx97B45xkAQGpWHrpN6ANbexsAgLO7Mzxqu/MZHqkBdBUCfHj4MZLPp8M1yAlBQ/01CgRm/5MPRsjAsbFtpa+p8COzKhQh5Iu5xWsO6KMxQ6JEiNRw6dn5+OngJaj+/ecg5vIDeIcGQCQRw9HNEREDImitEamS8goBJiemIGFvMpDPIHx6M/U5VsniyP+dg0OADVpPalapa2pCoUVzi9dc0EdjhJBKq+Vkh8+Hd1J/PW1IW1y59wQAcPByPP43fhGsbUs+UvOs540hU4bQWiNSLl2FAGVFcihVSvh0dsWDX54hOzEPzvVKdjvmJOWDEQHOLewhlynUT6wv75qaUGjR3OKtaSgRIoSUYS2VIKJpPQBARNN6mDOmu/rc5iNXsXbSMgiFQhTIFBgwZQis7axhZWMFFw9nvkImJkZXIUBWVbJmzca75Lg897/aWfJcBRghA6mLWN2uomsA8y+0aG7x1jSUCBFCqmRUt1CM6hYKAEh4loGle86B4zjcTEiBd6uGcKjlALFUjNY9w2jWyILpKgQoEJYUCC18VgQAkDiI1eckDmJwKg6yTAUEdYWVugYw/0KL5hZvTUOJECFEb/7ervj+vb4ASuoaHTx/G0oViztJqVj67hK4e9cCIxKg98S+6m38xDLoKgQotZZAJBQhITYZ1s5WcKr73w93Rz87cEog63IeGoT/l+yUd01NKLRobvHWNLRYujJosTQhVZadVwi5UoXUzDxMWrMPjEgEr0Z10G10ycds1rZWLzw+hBhbVWv5VPW4NroKAT6MefLfDrA36moUCNTYNVbJa0y50GJl36/KxEvb56umxu0ai4qKwo4dO3Dnzh1YW1sjIiICCxYsQMOGDcu9btu2bfj666+RmJiIwMBALFiwAH369Cn3mjIoESLEIH746xyO30gEADzKyEPHkV0BBvCo7Q6fet78BleDVbWWT1WPl1eXR986QlW9Rlv/fBchrGr/5bXX9Z6Yak0kU1DjEqFevXph+PDheO2116BUKvHFF1/gxo0buHXrFmxtbbVec+bMGXTo0AFRUVHo168ftmzZggULFuDy5cto0qRJ5TunRIgQg7udmILj/yQAALafvgXnQF9Y2Vmjx5gekFhJeI6u5qhqLR9dx+9sS0DGnWx4ta6Fej1qV6nOjT6zS4aYkSrvPsamb10gbfHeuniHagzpocYlQi97/vw53N3dceLECXTo0EFrm2HDhqGgoAB79+5VH2vTpg1CQ0OxZs2ayndGiRAhRiWTK3D/yXPcepSG+X+ehp2jLViOw4ApQ+DXoDbf4ZktfWr5aDvOsRzOLrwG2HPw7+sFr7qe6vtQnZuyDFkXiGoM6a+yiZDZvms5OTkAABcXF51tzp49i27dumkc69mzJ86ePavzGplMhtzcXI2XTK7Q2Z4Q8uqkEjGa1PPGm51DcXnlJJycOxa7pg/FubV7se79pdjw1QbEX3+I+OsPkZ+Tz3e4ZqO0Po1fZy+dtXyKsoqRnZhX7vHsxDwUZRXDp5MrlColZEVy9b1K69ykpqQh8XZS9Q7QROl634Gqv1+GvBfRzix3jbEsi6lTp6Jt27blfsSVkpICDw8PjWMeHh5ISUnReU1UVBRmz56tcWzW+F745u0qrisihLySWk522DFzJABg77nbOHf8KjiOw7or8WjSORT2rg5o06c1VbwuR1Vr+eg6Ls9VgAMHG29rFGfKNGr8AFTn5mWGrAtENYaMzywToUmTJuHGjRv4+++/DX7vGTNmYNq0aRrHpJd/Mng/hJDK69cmGP3aBAMAPurfGjcTUnDsnwQsmfg37J3s0OudvvQRmhZVreWj67jEQQwGDAqfFUFgJVC3K0V1bjQZsi4Q1RgyPrNLhCZPnoy9e/fi5MmT8PX1Lbetp6cnUlNTNY6lpqbC09NTxxWAVCqFVCrVPCgRa29MCKl2Hi4O8HBxQJeWDfC1XIHM3EJM+H43ChUq2Hg4o9eE3gDDwN7JDlY2VnyHy6uq1vLRddyprj2sna3wNDYD/v281I++AKjOjTaGrAtENYaMz2zWCHEch8mTJ2Pnzp04duwY/P39K7wmPDwcR48e1TgWExOD8PBwY4VJCKlGUokYXrUcsW9OJGKjxmF8qwBc+/kgrv10AKsmLcepPadx+d+P1CyRQCBAvzF9UBSvwI1ND5DzKA9KmQo5j/KQciQL2dfyIRAJ8ORsKp5dfI7Hp1OQHJOpPp54/BkSTz1B4vFnEIgEyL6Wj5QjWRr3ubHpAYriFeot3g9vJuLa6et4eDMRLMvy/A7oh2XZVxpHee/7i+9XZRY3G/JeRDuz2TX2wQcfYMuWLdi9e7dG7SBHR0dYW5d8dhoZGQkfHx9ERUUBKNk+37FjR8yfPx99+/bF1q1bMW/ePNo+T4gFePo8G6euxSPuYQrOPc6Ai4cz2g7tYJEfoemqT1PLsxYuxsZBppSBEQKcCpCKpHD3csOjB0mAGBAIGbAqDlAATV5rDJZT1eg6N4asPWSq97IUNW77vK4FkRs2bMDYsWMBAJ06dULdunURHR2tPr9t2zZ89dVX6oKKCxcupIKKhFiY5PQc5BQU48M1+6AQCODo7YrBU9+ASCyESGx2KwT08nJ9moLcAmxcvBnW9UXwDHeDtasYRRkKPDr2FMnn02Ffzxp+Azxh5S5BcZocKacykH2tAK9PGITGYY1qZJ0bfWv/lMeQdYz4qolkrmpcIsQrSoQIqVH+OHkdm2Kv4VlaNsLf7Agnd2c0CA2wmEKOumrTcByHR3eS8OxIOhR5KjSdWu+/cyoOdzY8RtFtBVbFLIVIJCr3XoB51bmpKeMg/6nxdYQIIURfb3Zoir9mjcKpxe+gYWYepHF38MOUlTj550mLWE+kqzZNQU4BIAA8OrlAnq1E/qMi9TlGyMCnsytkShnOHrhQ4b0A86pzU1PGQarOMuaECSFECxsrCT4cHAEAGJOSiegjV7Bkwv9gZWOFfpMGok5QzdyJo6s2jVKhBADYepfstlPkKTXO23hbgxEC6ckZFd6rlLnUuakp4yBVR4kQIYQA8PN0wcxRXTFzVFdk5hbg7WW7cKhIBomLI7pEdoOdox1cPJz5DtMgdNWmKV0vVfCsGAAgttf8EVH4rAicCqjl5VrhvUqZS52bmjIOUnX00RghhLzExcEWO2eOxLGo8ZjcvhHS/jqLrV/9jP3rD2jMhpirF2vTcOx/HwXaOtoCLJB6PBMSJxHs6vw3O8KpODyNzYBUJEV477AK7wWYV52bmjIOUnWUCBFCSDn6tQnG/yb0wqG5Y/GGuwN+mfEj1s/4CQfXHzDb9US6atPkJuUj/UQeMi7ngVMBeQmFUBaxyI0vwJ0Nj5F9LR/9x/RRL5Qu717mVuempoyDVB3tGqsM2jVGCPmXXKFEYbEcG2IuY8e5O5ArVegzaSDqNfGHmOcq9EqlEmf2n0fCzUT4N66LiD6t1UmLUqnE2QMXkJ6cgVpergjvHYY7cfd01he6cOwSimXFAAOAA6ykVhgwti/6je2ttW996txoi+nF3WhV2Squz9ZybdfcuninyuOgbe2mibbPGxIlQoQQHTJzCzAjOgZXElPRdngn2DvaoVFYcLXHsTf6AP7auB8y5X/Ji1Rkhf7/FjssOadZOLH/mD7oE9lTazLwV/Q+PE58AhWrglAgRO26vug/tm+5dXSqkhD8F2/ZmOoG16lSMqJPElbeNY1eC6r0OKjQoemiRMiQKBEihFTg/uM0nPonAafvPEGSUoW2g9sh6LWgaul7b/QBbP9xJ5xCbeHRzhnWnmIUpSiQeiobmVdywSk5uIY5wqeTK2y8rVH4rAhPYzOQfS0fb0wcrDHLY4yigrrjtSsTU9bVPFhJreAe5lyp/vWJ11BjrI73iuiPEiFDokSIEFJJHMch/mk6Pvn5EDLlClg52OKtr0YZ7QGwSqUSk7p/DOtGYjQY7QOWVUEsZKBQcWAEQtz9OQm59wsRtiAYQvF/sxraCiRWR1HBF+MNGlsbjPCFPlQc7qx/hIzLeeiztj2Ekv+ecq+tf33iNdQYqQCj6aOCioQQwgOGYRDg64Zds0bh5Nxx+LJnS2z8+Af8Ou9X3Ll8DxkG3nV29sAFyJQy+HRyBctxEDCAUAAIGEApU8K1pQMEEgHyEgo149RSILE6igq+GO+LSRAAcAzg3t4ZAjGDp+fSKuxfn3gNNUYqwFhzUB0hQggxoo6h9REbWh87Tt3ArfO38dule/APb4T2Q9rDwcXhle+fnpwBRghYeUgBjoVIVPJDWSRkIFexkLqKwQgBeY6izLUvF0isjqKCpfHaeGvpg+Ng7SUFI2RQmF5cYf/6xGuoMVIBxpqDZoQIIaQaDGnfBF+91RkHvh2D4T4uWDNlJX5fsFVdzVlftbxcwamA/KeFEDAlM0FAyf8LhQxkGQpwKkDiWHZH28sFEl8sKqiNIYoKlsZb+ExLHwyDomQZOBUHm1plP0p8uX994jXUGKvjvSLVgxIhQgipRva2Vugb0RhxKyZhVKPa+PHD77F4zHxcPHwJhXmFFd/gJeG9wyCEEKknMyF86ZxEwiAjLhesnIW9v43GOW0FEqujqGB47zBIRVI8jc0Ap9Lsg+GAtFNZYBUcfNq4V9i/PvEaaoxUgLHmoESIEEJ4wDAMXu/QFLFR47F8Yi88i7mIxW8vQuyfJ1BUoH2WQRsBI4CrixOyruThzqZnyH5YBGURi+yHRbiz6Rmy/smDIl+FexufIDe+QGuBRKFQiCfxT8EwjNGLCopEIvQf0wfZ1/JxZ8PjMjFlXSuAjY0Nbv32sML+9SmCaKjCiVSAseagXWOVQbvGCCFGcCzuHhZuPYoHac+h4FgwKsBOKIVMKELDkAA0bBOM13qFlXuPhzcTsfHrNch+no2sIjkYqQCMkAGn4sDJWDhbS8AKRShiWSg5ZZmaPf3G9saVk9ewY/nvGDJlGJp3aFYttXFMuY5QVcZIdYRMF22fNyRKhAghBnYs7h4mrdwGUYAYgR094eBhjdzUItw/kQLZPRnmje2HmOtJuJaUil7vD0BA03rqh6K+SKlQ4valu1AqlGCVLG5fvovcjFw4uDoguEVDCEQCiMQiBIbWx8Ujl8tUcWZVLL6fugzFD+7DKiAQHy2bCoFQUC3Vkk2xsrQ+Y6TK0qbJ6InQ0aNHcfToUaSlpYFlWY1z69ev1+eWposSIUKIAbEsiz6frcVTpzyEjaxfpgbNhV/j4ZNtj/0L3kVeoQwfrzuA60+eo8uYHmga3lhrQqSvKyevYe//NuKzTraYf7wA/T8Zg+Ydmhns/oTwxah1hGbPno0ePXrg6NGjSE9PR1ZWlsaLEEKIblfuPcGDtOcI7OiptQZNYAdPPEh7jiv3nsDRzhrrpw3B1mlDYHstHqs+WoHTu08bJA5WxeLEtqPo4CdAvxAHdPQT4MS2o2BVbMUXE1JD6PVrxZo1axAdHY3Ro0cbOh5CCKnx0nMKoOBYOOioQePgaQ0FxyI9p0B9rL6vG2ZGdsPYlEz8sO8Clry9CF0je6BZR/1nb66dvo68R08wZpgLACCyjRNO/PEE105fp1khYjH0mhGSy+WIiIgwdCyEEGIRajnaQswIkKujBk1uShHEjAC1HG3LnPPzdMH8t3vh7wVvIy32KhZGRuH62ZsoLpRVKYYXZ4MaeEgBAA09pTQrRCyOXonQhAkTsGXLFkPHQgghFqF5A18EuLvh/okUrTVo7p9MQYC7G5o38NV5D4lYhJ8/Hoz934wGLtzGmqkrcWTzEVw79U+lYlDPBoU7aRyPbOOE3Ecls0KEWIJKfzQ2bdo09Z9ZlsW6detw5MgRhISEQCzWrFi6ZMkSw0VICCE1jEAgwPThXTFp5TZc+DUegR084eBpjdyUItw/mQLlAwWmT+5aqZ1Hnq4OmDu2B95+mo6HzzKw+fAl/L3tJLqP74UGoQFarymdDYrwZVDXVQy58r/ZH/9aYrT1ZXBi21E0a9sUAiHtfiI1W6V3jXXu3LnSN42NjdU7IJNEu8YIIUbwYh2hIqUS1iIRAtzdMH14V3Rp2aDcazmOw+3EFATX9QTDaC64zsgpwMTlu5BcUIzRs8fCxcNZ43xp7SGxouzzvEopxFaInPMupNYS+NTzLtOHqaEt7ORlVEfIkCgRIoQYCcuyWLH9JFZui8XkoZ3x4RsdKvUDfP+Zm5i5bjfmvDMQfSIaa23z4MlzvPW/PxH+Zke07NIcEqkEgGbtIV1EYhFkRTLsXrVdXWjRVFFRQ6KNUbfPjx8/Hnl5ZZ+oW1BQgPHjx+tzS0IIsUgcB5yIuwNHYTFOxN1BZX41ValY/LT7FFCcg592n4JKx8LmAF83bP/8TdR5koYVk5YjdmssOI6DSCxC0/DGaN6hmc5X47Bg/L3zOGwLs0x68fSN87cQvWgTCuxz0egdf7Se2QSN3vFHgX0uohdtwo3zt/gOkZg4vRKhjRs3oqio7G6HoqIi/PLLL68cFCGEWIpD528j6ekzfNnVBUlPk3Ho/G2DXuPn6YJpb3bEga9HwiczB0veXoRLMZegUqrK7aN0MfW07rVMdvE0y7LYu3E/rOuL0WR0ABz97CCSCuHoZ4cmowNgXV+MvRv3lyn6S8iLqpQI5ebmIicnBxzHIS8vD7m5uepXVlYW9u/fD3d394pvRAghRD2z08FPiMFN7dHBT1DuDI++1wBALSc7zBzZBWcWTYQ87h4Wj1uAi0fjUJhf9pdacym0mHg7CakpafDr7KW1MKVfJ0+kpqQh8XYSTxESc1ClRMjJyQkuLi5gGAYNGjSAs7Oz+lWrVi2MHz8ekyZNMlashBBSo5TO7LzT2gEAMLG1Q4UzPPpc8yKxSIjv3++HQ9+OgfuDp1gzZSWO/XZMI8l5eWu9qW6pz8vOA8uxsNVRmNLW0wYsxyIvu+xSDkJKVamydGxsyefLXbp0wZ9//gkXFxf1OYlEgjp16sDb29vgQRJCSE3z4sxO0L8FDYM9pOoZnp6tgyF8aeu6Ptfo4uZsj8+Gd8KorqHY8fctLBq3AG36R6D9kPblFlo0pS319k72EDACFKQWwdHPrsz5gpRCCBgB7J3seYiOmIsq/dfcsWNHdOrUCQkJCRg0aBA6duyofoWHh1MSRAghlfTyzE6p8mZ49LmmIj5uTvhwcAQuf/8BBHeTsP6baGTGJ5lFocW6wX7w8HRHUmyy1sKUScdT4OHpjrrBfjxFSMxBpROhf/75R/3KycnB9evXNY69+CKEEEvAcRxuJSSjqlVISmd2InwEqOcqQbGCxb2nGShWsKjvKkGED6Ne91Pah1Kp0rhGpmBx7WkxZFqu0YVlWez5+7rG4mGWZRF3JwmHL9xBZJdmeP7PfRTlFuDAPznYfz0Hh27m4mJiAeq4iNSFFk1lrZBAIEC/MX1QFK/AjU0PkPMoD0qZCjmP8nBj0wMUxSvQb0wfqidEylXpj8ZCQ0PBMAw4jquwsJZKVf5uBH2dPHkSixYtQlxcHJKTk7Fz504MGjRIZ/vjx49rLQSZnJwMT09Po8RICLEcB87eqrCWjzZX7z9BUko6kpQqRKxOQXp2PlQsC6EgE7Wc/v2IR5SOq/efIDUzDzPX7cboPhEa1+QUKpCRVwxXeys42og1rmkZpH0G5NsNh7B8awymDO+OWW/31ijoqOBYcEoOuVmFsBEI8L+jmShWshCIGIitBBCpGLjYSiB1SEPinSTUa1z3Fd89w2jSuhHGfjoaezfux60fEzXqCA37lOoIkYpVOhFKSEhQ//nKlSv45JNP8OmnnyI8PBwAcPbsWSxevBgLFy40fJT/KigoQLNmzTB+/HgMGTKk0tfdvXsXDg7/TSXTzjZCyKt6uZZPVdbnNK3vjfmTh0KuUKKgUIbRc6LhZssgJU+FRR8Phq2NFBKxCI3qemLuhv1AcQ5iL97G3A9eh0rFQsWyWPDLIdiKUmHnXAufRfaEUCCARCxC0/ralyjI5Uqs33MKnjYs1u85hTaN62Lq2h0QBYgRMrAuHDyskZNSiEvbEvD8Xh68w11RL8INhVkK3D2aAmsPK+QnKtC2Xwf4lfMMND40ad0IjV4LosrSRC+VToTq1Kmj/vPQoUPx/fffo0+fPupjISEhqF27Nr7++utyZ2leRe/evdG7d+8qX+fu7g4nJyfDB0QIsVgv1vKZe6JkfU5lZ4UkYhG6vdYQANB0dBRq2TD4rrMUXxyTYc7GQ7i+aQaAkurR//WRApWKRZ+Ixth/5iaKC3Mxq0ctzD2RB6FAUGHfUZtiwCmKMKOLFb44WojJ32+HVWMpwkbWV289d6ltB7CAc3M7+Paqhbp+bgAA51o2uHnoGZRQ4GLsJfQd20vft81oBAKBycxSEfOiV7p8/fp1+Pv7lznu7++PW7dMr4pnaGgovLy80L17d5w+fbrctjKZTKM+Um5uLmRyRTVFSggxB/rW8nlZTk4hHj5ORo/6IowJlaBnfSEePk5GTk6hzj7kcmWV+y6dDepRT4jRodZo4SXAk+xsBHTw0Ki/k5GYj/xsGXw714JMpUJhsRwA4BXshG5TG6FxNy88u/8Uf/24V783jhATpFciFBwcjKioKMjlcvUxuVyOqKgoBAcHGyy4V+Xl5YU1a9bgzz//xJ9//onatWujU6dOuHz5ss5roqKi4OjoqPGK2hRTjVETQkzdq9byKdVu8nI4WzH4MKxkjc+k1yRwsmLQbvJynX1EbYqpct+ls0GT21gBAHoESMAIORSKNZ81VpynAAcOdt5W4MBB+VJyFdjBEy7uNngedw/fv78UCbcfVWm8hJiiKtURKrVmzRr0798fvr6+CAkJAVCyq4xhGPz1118GDfBVNGzYEA0bNlR/HRERgfj4eCxduhSbNm3Ses2MGTMwbdo0jWPSyz8ZNU5CiPkwVC2f0tmgYY3FaO5V8k9xS28RetYX4vebyVj9Z2yZPtrVZrB+zykMaSSpdN8vzgaFepYkXG18xZAycjx9mAVfLyf1BhgrezEYMMh/VgyBVADRS/fKTSmCRCDEgrd7oZ53Lby7cjd2pOdi4qJ34eCiuaWfEHOh14xQWFgYHj58iO+++w4hISEICQnB3Llz8fDhQ4SFhRk6RoMKCwvDgwcPdJ6XSqVwcHDQeEkl4mqMkBBiygxVy+fl2aBSpbNCR+MelOkjyE0MTlGEt0KsKt33y7NBANDcU4hGDgzS43Lw7HmO+rhrXTvYOUnxJDYdUqEQNlYS9TmO5XD/ZAoC3N3QvIEvnB1s8McXI7Bhcn/8/OlaHNt6rNyn2RNiqvReUm9ra4t33nkHS5YswZIlSzBx4kTY2toaMjajuHr1Kry8vPgOgxDCE31r/wBl6//IlZz6VZlaPqV1fLKy8vHwcTK6+IsQ7CZAsZJVvxq7C9DVXwiGY+FmI1DXCyqUqXDwTj66+AthKyhGsYJFWnYhZArNvpVKlXp8pbNBXfyFCKolQrGSQ7GSg5wFvm5rBfmdfFzZmoiMf+vvZD0ugEgsQM61fKQeyULmo3woZSpkPsrHhV/joXygwPThXTV2YzWp540dM4ahhUKB9V/8bLBkiGVZPLyZiGunr+PhzUR6cCoxmkp/NLZnzx707t0bYrEYe/bsKbftgAEDXjkwbfLz8zVmcxISEnD16lW4uLjAz88PM2bMwNOnT/HLL78AAJYtWwZ/f380btwYxcXF+Omnn3Ds2DEcPnzYKPERQkyfvrV/gLL1f7Qqp5ZPaR0fTzcXiAXAiUdKNFypJXHgALEQCFz4CK52EmTkFcPWSgKZXA5bMXDiUQ7EonwUyxSwsZLCzkaq7nvVjlPYtP8M5rwzEJm5BSgqLsKJRA7BK7LLdCNRcMi+kotzj+9DIhVBzAjQ0N0Nk0e3Q8zlu/jnp0QoOBZiRoAAdzdMn9wVXVo2KHMfHzcnvDcgHG5Odlj83lIERjRG77f7lGlXWTfO38LejfuRmpKmUReo3xiqC0QMj+Eq+WuRQCBASkoK3N3dy63NwDCM0Qoq6iqQOGbMGERHR2Ps2LFITEzE8ePHAQALFy7EunXr8PTpU9jY2CAkJAQzZ87Ueo9ynVlhgOgJIXxTqVgM/WIdkh4lwK+OP7bNe6fStX8AQK5Q4uTVeMjLmfWQiEXoEFofErHm75lyuRKBQ2fCFoXIUUnh5VELMrm2JIhDfpEcYpEQn43sjA17zyE/KxW2Tu7oFd4ULMcBDLBqWyysIEMxpPhxxiiIRUIIhQKs3n4cTx4nwq+OPzZ8FYm1u8+gSKZ756tUIkLbpv4olClQy9EWzRv4QiAQgGVZXLn3BOk5BRrHK8JxHGasP4zbShXC+rRG/SZldxiX58b5W4hetAnW9cXw6+wFWw9rFKQWISk2GUXxCoz9dDQlQ6RSpEIpBgcNr7BdpWeEXi7JzodOnTqVO50dHR2t8fX06dMxffp0I0dFCDEXr1L7B9Cs/1NVL9bx+fKYDP0jmmDW2+XXRdOsF5SPlkG10SeiMWb/fABWjAJzu1jjy2MyXL3/FLPe7o39Z27iSXKyenynrydg+qhuesUrEAh0VqguD8MwiBrfA0cu3sXStX8h/42OaNaxWaWuZVkWezfuh3V9MZqMDlBv7Xf0s0OT0QG4sekB9m7cj0avBVGxRGIwev2XVFxcbOg4CCHEqAxV+0cfL9fx6V5PiPV7SmoCVTXeoiK51nsVFcl5G9/LGIZB97Ag7Jw5EgkHzmP5u0uQnKjjo8QXJN5OQmpKGvw6e2nUNwIARsDAr5MnUlPSkHg7yVihEwukVyLk5OSEDh064Ouvv8bRo0dRVFRk6LgIIcSgDFX7Rx8v79ya3NoKnKKo3BpluuKdsOA3rfeasOA33sani1QixtYZw7Fj+lD8MecXXDwSh+JC3b9I52XngeVY2HpYaz1v62kDlmORl51nrJCJBdIrETpy5Ah69eqF8+fPY+DAgXB2dka7du3w5ZdfIiaGig8SQkxLRbV/jDlroq2OT3MvcbmzQrribV+bwb6/r6Kbv0DjXl39Bdj391W0ry2o9vFVhlctR+z6+i14PnyGNR//gMJ87b882zvZQ8AIUJCq/XxBSiEEjAD2TvbGDJdYGL0SoXbt2uGLL77A4cOHkZ2djdjYWAQEBGDhwoXo1cv0nkFDCLFshqr9ow9tdXyA8meFdMU7qKEAdiIWzb01l3e28BLBTsRicJBQ47gpzAqV8nBxwKfDOmJZZDes/GAZ/ly6vcx607rBfvDwdEdSbDI4VnM9KMdySDqeAg9Pd9QNrvraJUJ00Xu12b1797Bu3TpERkbi9ddfx19//YV+/fphyZIlhoyPEEJeyavW/nkVuur4FCs5BLuJ0KVu2VkhXfEWy1nYMcXo4i/CkXglCuQsipUcCuQsjjxUoou/CLYoQrGcrbbx6aNNk7q4vGISuns6Y9PsX/AsIVl9TiAQoN+YPiiKV+DGpgfI+be+Uc6jPNzY9ABF8Qr0G9OHFkoTg6r09vkX+fj4oKioCJ06dUKnTp3QsWNHhISEqMu01zi0fZ4QXnEch9uJKQiu61nlf2fi7iTh3ahfAKVMdyORFD9MH4ULd5Lw/qC2EAr/m1lhWRZ7z9xEv4jGZX4A6zpXejw7txCfLN8KWxGnHoeCBcQCqMdRoGSweOoItAyqjeC6nrh89/FL8ZY880uhZJFXUASGAfLlgFQISEUMZEoOMhVgJwE4DrC3tYa19KVq+CIp1s6I1GsXmDHtOn0TGw7HwSu8CToN66Q+TnWEiCFUdvu8XolQaGgo7ty5gxYtWqiToXbt2sHGxkavYE0eJUKE8Gr/mZt6F0GsbO2fn/86i79OXcHAji3w6zdj1Odm/3wAy7fGYMrw7mW2u+s6V3r8vdc7wcnORl3H597jNPx95S7aNW+IBrXdAQDWUjHqebti/i8HMOedgej2WkONeC/eTsLmg+cwtGtLpGTkQaZQQigQoI6nC0RCAZQqFo9SMqFiWUjFIvSNaARrqUQjTl21jUwBx3F4b8VucIG+6DK8i/o4y7JIvJ2EvOw82DvZo26wH80EkSoxaiIEANnZ2Th58iROnDiBEydO4NatWwgNDUXnzp0xd+5cfW5puigRIoQ3r1oEsTKKiuTw6v8ZvGw5JBcwSP5rAaytJRpFEAtgg/vb5kAiKUkmdJ3TdVzXOMobX3WM3RRwHIdp6w7g9J0k9PlgIIJa6VeriZAXVTYR0vtvlJOTEwYMGIAvvvgCM2bMwBtvvIGLFy9i/vz5+t6SEELKeLEIorEW/k5Y8BvsRCxmtJXATsRiwoLfALxQBLF92YXNus7pOq5rHOWNrzrGbgoYhsHSd/sgdt44nPp5P+5cvMt3SMSC6JUI7dixAx999BFCQkLg4eGB999/H/n5+Vi8eDEuX75s6BgJIRaqOoogFhXJse/vq+hRX4TIUCl61Bdh399XkZNTqLMIoq4Cifn5xVUqdiiXK3WOj88CkHyxtZZiz6xROL1hP7ZGbUF6cgbfIRELoFci9N577+HZs2d45513cOXKFaSlpamTo2bNKldKnRBCKlIdRRBLZ4M+DCtZVzM5rGRWqN3k5TqLIOoqkNjvs3VVKnYYtSlG5/j4LADJJ1trKfbNGYOvu4Tgly9+pmSIGJ1eiVBaWhq2b9+OyZMno2nTpjrbzZ8/H9nZ2frGRgixYNVRBPHF2aDmXiVrf1p4idCtngAPHyeju9YiiCfx8+6TZQokdvUX4NKtB1qv2ff3VbT1ZTSLI/oJsH7PKbTTUgTxx10n8SNPBSBNgUQsQssgP/zx2VD88uXPeP4sne+QSA1m1FV38+bNQ2ZmpjG7IITUUNVRBPHl2aBSjdxEcLJiMLaZ5jb0ya2toJQVobCwsEyBxFAvEZykDEaGaF4zrkXJDJO/o+a2/xFNpeAURQh202w/sbUDbjx4jPsPk3gpAGlK6nq54o/pQ7H5y/U4uOEgZEXllEAgRE9GTYT03JBGCLFwxiyCyHEcbiUko7BQhn1/X0UXfxGC3QQoUqhwJVmBPJkSRx4q0dVfCCcrDkUKVl0EMaiWEO1qC+AgBQKdhShWsMguVqFApsKReAW6+IvgKHnhGgULFynQxV+E/bfzUFCswrWnxSXFEQXF6FJXiAN38lH0QhHEus5iCFQyNHdXwt9F99iVShVuJSTX+H9n63q5Yt83o9DTwQprP1mD4kJKhohh6b19vjLs7e1x7do11KtXz1hdVA/aPk9ItapsEUR9igSW1iQK9vfF3pNxcCj55AlFipLChAIAAgawkzLguJIdTaUPQme5kkSqQM5BImIgETLqIolyJVfmGo4DWI77rwiiRIxiuRKONhKAVQAchwIF4OpgA1tpSRHHApkKGbmFsJUwcLC1hlgkLDsIkRSj+7bDpv1n9KqtZK6OXX6Ar7efwsRF78HKRsp3OMTEVXb7vOlV1yKEWLym9b0xf/LQCosgNq3vXaX7ls40oTgHWTmOeL1raxTJFWA5Duevx8OdK0A+rOFkbw8Vx0EiEqJbqwawsSr5oStTKHDx9mMAHFoG1caek/+gOD8LtraO8PFwhVLJQiwS4rXg2pCKxRrFDsUiIe4mPkNRznPYOruhV3hTsCwHkVCAJvW81AmPQqnCjYfJAIOS48KyiZBQKMDqP48DxTn4afcp9GwdXCPrC72sS4sAMAzw1adrKBkiBkOJECHE5EjEInR7zfBF9V6syzP3RBo+eKMT+kQ0xv4zN3E//iG+7OiBuScKMee9/hXOsuw/cxNHTsdh7oCSa6YN61zuNfvP3MTMH+5hVo9amHsiHy0b1tbZfmCHkAr7fvIs+d9xlKwZspRZoc7NAzCXYTDt/SVo2iEUvcb3qrmPdyLVoub/CkEIIdBdk6i8Wj5VvZeuawxZE8gS6wu9rFNofZxaMAHNGBa/zf+txq+TIsZl1ESoffv2sLa2NmYXhBBSKbrq8pRXy6eq99J1jSFrAllqfaGX2VpLMe319uju7YIt87ZQMkT0VulEKDc3t9KvUvv374eXl5dRAieEkMrSVZOoXW0G6/ecQnsttXx0zbJUtb6RIeshVUdtJXPz4aAI9PR1xa9zfwXLWt74yaurdCLk5OQEZ2fncl+lbQghxJToqkkU5CYGpyjCWyGaNYHKm2Wpan0jQ9ZDqo7aSuZo8qAI9K7jji2UDBE9VHqxdGxsrDHjIIQQo9BWkwgAVCyHg3fy0cVfCFtBMWQKe5SuuX2xXs+LO7J03UvXNVVtr8849LlXTTRpQBswf53D5m83ocuobvCt78N3SMRMVDoR6tixozHjIISYAI7jcDsxBcF1Pc1+J07pWAqLFUhKSUeSUoWI1SkAOChVLGRKDhm5RbAVAyce5cDBVl62Zo8oHVfvP1HXKrp6/8lL99LihWuq2r48hrxXTfVB/zbwPn0TG1bugk/bpujwRge+QyJm4JUKKhYWFiIpKQlyuVzjeEhI+Vs/zQ4VVCQWorTYYE0o0lc6lq/H94OttVRdk+ji7SRsPngOw7uHwUoiglLFQiQSaK3ZIxGL0CG0PiTikt8Z5QolTl6Nr7C+Uek1VW1fHkPeq6ZjWRYTl+2CtKk/OrxBv8RbqsoWVNQrEXr+/DnGjRuHAwcOaD2vUqmqekvTRokQsQAqFYuhX6xD0qME+NXxx7Z575jtRyy6xlKTxkjKx7Is3lm+C6JGddHpzU58h0N4UNlESK9/AaZOnYrs7GycP38e1tbWOHjwIDZu3IjAwEDs2bNHn1sSQnj2YrFBc194q2ssNWmMpHwCgQA/Th0M1e1HOP77cb7DISZMr0To2LFjWLJkCVq1agWBQIA6depg1KhRWLhwIaKiogwdIyHEyGpSkT5DFk4k5o1hGKybMgjcvSTEbqUNP0Q7vRKhgoICuLu7AwCcnZ3x/PlzAEDTpk1x+fJlw0VHCKkWNalInyELJxLzxzAM1nw4EMz9Jzj22zG+wyEmSK9EqGHDhrh79y4AoFmzZli7di2ePn2KNWvWUAFFQsxMTSrSp2ss7f0EWL/nFNpVoXAiqTkYhsHqDwdAGP8UR7cc5TscYmL0SoSmTJmC5ORkAMCsWbNw4MAB+Pn54fvvv8e8efMMGiAhxLhqUpE+XWMZ0VQKTlGEYDexxnFzHCPRD8Mw+GHyAEgePsORzUf4DoeYEL0SoVGjRmHs2LEAgJYtW+LRo0e4ePEiHj9+jGHDhhkyPkKIEWkr0lf6erFInz4zJizLYs/f16tU6beq13Ach1sJyeA4TudYZAoOdoJidKkrxIE7+SiSs1rHqFSq1PfS1QcxbwzDYOXkAbB6lIKYTTF8h0NMhF6J0Jw5c1BYWKj+2sbGBi1atICtrS3mzJljsOBedvLkSfTv3x/e3t5gGAa7du2q8Jrjx4+jRYsWkEqlCAgIQHR0tNHiI8TclBbpO/OkpEjfy68zT1RISikp0ldV3244hLGzN+DbDYeMds2Bs7cQOXsDDpy9pXMsYSufou+mHJx4pMSZxCK0+P6Z1jGu2nFKfS9dfRDzxzAMVkzqD5snaTj8y2G+wyEmQK86QkKhEMnJyeoF06UyMjLg7u5utDpCBw4cwOnTp9GyZUsMGTIEO3fuxKBBg3S2T0hIQJMmTfDee+9hwoQJOHr0KKZOnYp9+/ahZ8+ele+Y6giRGspYRfrkciUCh86ELQpRABvc3zYHEkkFBQOreM3LNYF+nT0ep68nlBmLQqXCjYfJUCpZiIT/Fk58qYK0UCjA6u3H8eRxItUdshAcx2HKmn3I93JFjzFV+HlAzEZl6wjpVX6U4zit5fevXbsGFxcXfW5ZKb1790bv3r0r3X7NmjXw9/fH4sWLAQDBwcH4+++/sXTp0qolQoTUUBKxCN1ea2jw+0ZtigGnKMKMLlb48lgRojbFYNbb5f/dreo1L9YEmnsiGbGX7+ushj2wffnV7vefuYknycnqex06fxt9IhqX6aP0ODF/DMNg+Xt90efrjcjsHQYXd3pguKWq0q82zs7OcHFxAcMwaNCgAVxcXNQvR0dHdO/eHW+++aaxYq2ys2fPolu3bhrHevbsibNnz+q8RiaTITc3V+MlkyuMHSohNYZcrsT6PafQo54Qo0Ot0b2eEOv3lNTxMdQ1hqx7RHWHLBfDMHi/Txj+XLwNynJmRUnNVqVEaNmyZViyZAk4jsPs2bOxdOlS9WvNmjX4+++/sWrVKmPFWmUpKSnw8PDQOObh4YHc3FwUFRVpvSYqKgqOjo4aryhaVEdIpZXO7ExuYwUAmNzaCpyiqNy/R1W9xpB1j6jukGUbENEI70YEY/2XP1MyZKGq9NHYmDFjAAD+/v5o27YtRKKa92C/GTNmYNq0aRrHpJd/4ikaQszLizM7oZ4lW9Wbe4nVMzwzRncvs+6nqtdUVPeoZ+vgSq/j0XWvdrUZrN9zCkMaSV65D2L63uzUDAKBAD988TPGz3sbIgt/aK2l0etvcseOHfHo0SN89dVXGDFiBNLS0gCULGa+efOmQQN8FZ6enkhNTdU4lpqaCgcHB1hbW2u9RiqVwsHBQeMllYi1trUELMtCqVRpfV259xidP92Ibp//WubVc8YmJCZnaL2OPlqouV6e2SlV3gxPVa8xZN0jXfcKchODUxThrRDNmGhWqOZ6o0NTTOrQBD/P+IlmhiyMXmnviRMn0Lt3b7Rt2xYnT57E3Llz4e7ujmvXruHnn3/G9u3bDR2nXsLDw7F//36NYzExMQgPD+cpItMX/+Q5njzPBgBk5xdh9pazsHeupbWtxMYOAz5fCzvHsosMnz9LwrjVc8Eqy66vyslMx6evt4Jvrf9++AT4usHHzckgYyD8KJ3Z6eIvRFAtEYqV/21IDXYToUvdsjM8Vb1GW62gUi/WBKrMjI2ue6lYDgfv5KOLvxC2gmLIFPYo3RtS1T6IeXm9Q9OS7fVf/IS3502gmSELodd3+fPPP8d3332HadOmwd7eXn28S5cuWLlypcGCe1l+fj4ePHig/johIQFXr16Fi4sL/Pz8MGPGDDx9+hS//PILAOC9997DypUrMX36dIwfPx7Hjh3DH3/8gX379hktRnPDcRzWH4zDs/R8FCkUOH4rHfVatAMAMIwNRn+3UWuiUxE3bz+8NXOt1nOyokIcOvAb2Of/JUkPNu5C7xZ+EDACNKnrhsHtaWeOufnj2GUUFRfhRCKH4BXZWtsUKVX449hljOoVptc1pbWCkpQltYK0EpXUPWoZ5FduvLruVSBTISO3CLZi4MSjHDjYystst69sH8T8DGnfBIXFchxcfwD93u3PdzikGuhVR8jOzg7Xr1+Hv78/7O3tce3aNdSrVw+JiYkICgpCcXGxMWLF8ePH0blz5zLHx4wZg+joaIwdOxaJiYk4fvy4xjUff/wxbt26BV9fX3z99dfqqtiVVsPqCGXmFuC95XuRXSBHfmERajfvAv/m7QEAfoGNIZZKqz2mwvw8JCfeBwBcP7YDhU/vwMFWilWTe8OXtrWahfzCYvyw428UyXTvsrSWivHBkHaws7HS6xpD1j3SdS+F8t+6QyoWItG/dYeEmomQPrWViPnIzitE9683YvzCd+FMM9Vmq7J1hPRKhHx9ffHHH38gIiJCIxHauXMnPvnkE8THx+sVtMmqAYlQSkYuog9fxa7zDyEUS9F1/BeoHdiI77DKlfb0EQ7+MBOsSo62Ddwwvmco3Jzs4Opoy3dohJAa7l5SGkYu3YFx8ydSjSEzZdRE6JNPPsH58+exbds2NGjQAJcvX0ZqaioiIyMRGRmJWbNm6RW0yTLTRKiwWI6tR68iM78Y2849QqN2vRA+YIzWYpimLu7wdqQn3kbS7asY2zUIjrZWGN4llNZoEEKM5v7jNIxcsgNjKRkyS0ZNhORyOSZNmoTo6GioVCqIRCIolUqMHDkS0dHREL40jWz2zCwR4jgOC37/G7tP30aDrsNh5+yG4JZtIbXSvlPOnBTkZuP+PxeR8SQez+IOYWzPFhjfqwXfYRFCaqhbCcmY+sffGD/vbb5DIVVk1ESo1OPHj3H9+nUUFBSgefPmCAgI0PdWps1MEqEimRzbTtzEqr2X0TCiN17r8xasbGrux0iF+Xk4sWU5Uu9dwcy32qJTaD2LLnVACDE8lmXR/YsNaP92HwS1MvzjaIjxGD0R+vnnn7F06VLcv1+ywDUwMBBTp07FhAkT9LmdaTPxRCgrtxAHzt3C4t1XUD80Al1GT4OwBha71EVWVIiY9QuQ8uAaZgxtjZ6tg2AtlfAdFiGkhsgvlGHgnM1oP74PgozwbD5iHEZNhGbOnIklS5bgww8/VNfkOXv2LFauXImPP/4Yc+bMqXrEpsxEEyGlUoVFf5zGnnP3EdhxEFp0HgBbBye+w+JNRuoz/HNiL55dOoQ3OzbG+wPCzHI9FCHE9BQUydDu/9bhs1+/5DsUUklGTYTc3Nzw/fffY8SIERrHf/vtN3z44YdIT0+v6i1NmwkmQkv/PIPfjt1As94jEdymG+ydXPgOyWRkp6fiSsx2JF48gimDW2NY56Z8h0QIqQG+3BiDVBdHdBvVreLGhHdGTYScnJxw8eJFBAYGahy/d+8ewsLCkJ2dXdVbmjYTSoQSnmUg+vBVXC9yRa8JX9CMRzlUSiW2L5qGISGOGBQRDG83R75DIoSYMY7j8OGqvyAKDUCbvm34DodUoLKJkF57j0ePHo3Vq1eXOb5u3TqMHDlSn1uSCiQmZ+CbjUfx1pLDSHYLpySoEoQiEd74dAkuyOth8NxduHrvCd8hEULMGMMw+PzN9riw7xxUShXf4RAD0WtG6MMPP8Qvv/yC2rVro02bkqz4/PnzSEpKQmRkJMTi/3buLFmyxHDR8oXnGaH4J88xYsEeRLz1MRqEhPFS+dncFeRmY+t376NeLSt8/0Ev1HKy4zskQoiZ2nLsKtafu4txc8dD+PLjV4jJMOpHY9oec6H15gyDY8eOVfX2poenRCi/UIYR87YjWylB/4+i4OzuxUscNQXLsngafwfH1n6FdVP6oKGfBxVkJITo5bfYa9h09SEivxnDdyhEh2qpI2QxeEiE8gqKMfibrWg54lMEhIRVe/812bOEe7iw6yfYFCVjyxdDyz5QkxBCKqHDpz9iZNREOLo68B0K0cKoa4SI8bAsi3lbTqLH55vQauRnlAQZgbd/Awz6eCE82g7HiLnboKDP+gkhelg2oRd2Ld/BdxjkFVEiZEJYlsU7S/fgvjQYw+f8gvpNWvEdUo3WrGNfeLYfgXbTojF/6ym+wyGEmJlmAT4oTMlA/PWHfIdCXgElQiaCZVm8s2Q3UL8jIgaOhY09TbVWh2Yd+uDtxX/iUo4zvt10nO9wCCFmRCgUYPfMkTiw7E9kpmXxHQ7REyVCJoBlWUxYvBuCBl3Qut8ovsOxOAzDoMf46fin2B2zf6kBi/sJIdXGyd4GQ9s1QdzhS3yHQvREiRDPSpMgYVA3vNbnLb7DsVgMw6D72E9wU+6JWdFH+Q6HEGJGPhwUDu7+E5zedZrvUIgeKBHi0Q+7zyNsSjQkjXshrHfFK9uJcZUkQ5/iltIHMzdQMkQIqRyBQICfpg7GmZ2nwLIs3+GQKqJEiCerdp3DvgcqvLNkO1r2eIPvcMgLeoz9P9zlfPH1hiN8h0IIMRMCgQCRnZrh6K/0S5S5oUSIB9/vOIcDCcDAj76jx2SYqG6R03AfdfDF+hhQqS1CSGUM79QU/xy/isL8Ir5DIVVAiVA1W77jLGKSgAGT51ASZOK6jp6KBIE/vlh/hJIhQkiFPFwcMKptI1z7+zrfoZAqoESoGu07exsHH6rQfxIlQeaiy6gpSBTXx4yfaWaIEFKxt7qE4vSWo5AXy/kOhVQSJULVRKVi8dPBK2jVZxQlQWamy1sfIkkSiM9+PEzJECGkXG7O9ujcrD4uH7vMdyikkuhZY5Xxis8aUypVGL3gTzi1GowWXQcZJiYLkpH6DPJi3Z+5S6ys4erhbfQ4jm/9AV75t7Ho3Z6UzBJCdFIqVQj/ZB2mrZ9O/1bwiB66akivkAgplSqMmv8nXMKGoHmXgQYMyjJkpD7Dii/ehUKl+z9TsZDBh/PWVksydOL3H+CRdxv/o2SIEFKORdtO4pZQhJ7jevEdisWqbCIkqoZYLJZSqcLIqO2o1eYNhHYewHc4ZkleXASFioNL+9GQuniWOS/LTEHmqU3lzhgZUsdhH+DE76vxf2sOYvF7vSgZIoRoNal/G7Sb/iM6DusMKxsp3+GQctAaISPhOA5vRW2HW/hQSoIMQOriCWs3vzIvbcmRsXUc9j5uZElwNymt2vsmhJgHGysJJvVshXP7z/EdCqkAJUJGkpiciSw4oFmn/nyHQoygZZ8RmLL6IAppZwghRId+4cG4vO8c8nPy+Q6FlIMSISNIy8rD6IW70G74R3yHQoykXpPX0Oz1qRj8zW8oKJLxHQ4hxAR5uDigb6uGuHHmJt+hkHJQImRgqZm5GDz7D3SfvADe/g34DocYUWDzCIS88TEGf7OVZoYIIVp9NrQ9/t5Cj90wZZQIGVBeQTEGz96GnpMXwrtuIN/hkGrQIDQcrk064VjcPb5DIYSYICupGAMjGiPml8N8h0J0oF1jBnTyWjxqh7SFV90AvkOpcWSZKVqPF6U/gUqpxPNnSVrPV0eNoebdhuDbqEkIruOB+r5uRu2LEGJ+vn6rM9p/GY3ukT34DoVoYXaJ0KpVq7Bo0SKkpKSgWbNmWLFiBcLCwrS2jY6Oxrhx4zSOSaVSFBcXGzyuszcS8N2Oaxj+1RqD39uSSaysIRYyyDy1qcw5lUqFvKwMcCo5tqyaD6FIXKZNddQYcnbzxOufr8TEpVNwbNEYo/VDCDFfxXmFyHqeDWc3J75DIS8xq0To999/x7Rp07BmzRq0bt0ay5YtQ8+ePXH37l24u7trvcbBwQF3795Vf22sui9fbTyBEV+vh7WtvVHub6lcPbzx4by1WusEPX+WhF9XzINz6zdg5xdU5nx11hhydveC0MEDf525jf4RwUbvjxBiXhaO6Ya1W2Mx6MPBfIdCXmJWidCSJUswceJE9SzPmjVrsG/fPqxfvx6ff/651msYhoGnp3FrzazefR7WXg0oCTKS8mZzRBIp7PyCYO3mV40RaTf4//6HRXMnwdPZBq8F1+E7HEKICWnTpC5m/34SyYkp8Kpb/fXPiG5ms1haLpcjLi4O3bp1Ux8TCATo1q0bzp49q/O6/Px81KlTB7Vr18bAgQNx82b52xhlMhlyc3M1XjK5Qmf7S7eTsP2fbAz48LuqD4rUKGKJFK0GjMOGw9f4DoUQYmKspRK8Ht4IqY+pEKupMZtEKD09HSqVCh4eHhrHPTw8kJKifSFtw4YNsX79euzevRubN28Gy7KIiIjAkydPdPYTFRUFR0dHjVfUphitbTmOw7Kd5xHa40161AIBAAS3bIsMhyDMiqbtsoQQTcF+tRB38CLoEZ+mxWwSIX2Eh4cjMjISoaGh6NixI3bs2AE3NzesXbtW5zUzZsxATk6OxmvG6O5a2yan5+JpsRRBLdsZawjEDHUb/TEOXn1S7kwiIcTydGvZADZFMhTmFfIdCnmB2SRCtWrVglAoRGpqqsbx1NTUSq8BEovFaN68OR48eKCzjVQqhYODg8ZLKim7GykztwDDo/5Eu+GTqzYQYhHaDn0fk1fu5zsMQoiJcbK1Rsqj1IobkmpjNoulJRIJWrZsiaNHj2LQoEEAAJZlcfToUUyeXLlkRKVS4fr16+jTp88rx/PF+mOIiPwCfg2avvK9aqqM1Gfl7th6ucbPlVOHkZuZXqZdQV42lAoF7BydEdyyrfr482dJkBcXITvhOpSKspWdZVmpUKlUrxSTvoJbd8bqX5fjSVoWfN2dX/l+hJCaYfGEnuj33RZM/fETvkMh/zKbRAgApk2bhjFjxqBVq1YICwvDsmXLUFBQoN5FFhkZCR8fH0RFRQEA5syZgzZt2iAgIADZ2dlYtGgRHj16hAkTJrxSHHcfpeLKoyyMbUhJkC4Zqc+w4ot3oVDp/iz8xRo/V04dxto50wCxVLMRV/o/DDilDPbObpDa2AIAFHI58nOykHtgLQRiCRiBsEwfjEqOwvxcvWJ6Va9/thxvfPcx/pr9JtycaUchIQSo5WQHISMAx3G0ttREmFUiNGzYMDx//hwzZ85ESkoKQkNDcfDgQfUC6qSkJAgE/33al5WVhYkTJyIlJQXOzs5o2bIlzpw5g0aNGr1SHFtjryNs8LsQiSWvdJ+aTF5cBIWKg0v70ZC6lP3o8uUaP7mZ6YBYCpdu70Ls/F8SwqmUUBXmgFXIkHN6C1w6joaDd30AgFIhR+qD68i7tBve7d6AxFGzqrM85zlyL++FjZ2DXjG9KnefOmjQrh8OXriL0T1bGeSehBDzN6R1QxyOPoSe43rxHQqBmSVCADB58mSdH4UdP35c4+ulS5di6dKlBu0/KSUTB29mYOyIjga9b00ldfGsUo0fsbM3JB711V9zSjkU2SngVAowAiEkTv/dTyGXQZSWDIFIDAf/kDL9FD1PQsE/h145plcR1ns4lkwfgeFdm0MsKjtjRQixPJFdQzHmZ3r2mKkwm8XSpuJ/28+iRffXIRDSDzVSMam1DRqFd8PK3ef5DoUQYkJYluU7BPIvSoSqIL9QhosJWQjtMpDvUIgZ6ThiMrYevwWlUlVxY0JIjefiYAtBbgEeXH/IdygElAhVyfvf70WHkR/zHQYxMwzDILTPKMzefJzvUAghJkAoFOC9ni3x9O5jvkMhoESo0uKfPMedZ7kICNH+pHtCytM4ogcOXXqIjJwCvkMhhJgIqi9tGsxusTRfVu+NQ+ex2h/sSnSTZWp//Imu44qsZxpfv7hrjGNVkGenoOjf7fNKhRzKvHRwKhWKM5NfuW9dxw3B2tYeTXpFYtvxOLw3MMJo/RBCzENooA++XfAHwnqFwcbOmu9wLBolQpVwOzEF5xLzMXJMCN+hmA2JlTVYeRGSD/5Q5pxcVgSwLARCAa6cPIx7Do5Iun8LrLwQGYd+AMCAEbw4WflfHaHME5tQ8G8ipFKpkJeVAVYpQ+qhtRCIyv7nLBYJIbGyVsckFjLIPLVJZ9xiIaNub2jN2nXH+plb8GbnZnBxsDVKH4QQ8+Dn6YK2QX7ITM2EjZ0P3+FYNEqEKuGnA5fRrNsbVDeoCrLTU5GTmQEINd8zjlWCVcrBiKSASok9W34CGAAcIBBZAwzAKWQQS2wgllgBAFiVEuA4iGykePvzeXBydS/pIyMNvyz6CkrOHkJR2cegANAoWObq4Y0P562tlsrS2tjY2aNB6y7YGnsdHwxsY5Q+CCHmgwoqmgZKhCrh7MNcjJvYl+8wzEpBbjYgksCl6zuQuPyXWMgyk5H9969wiBgOgVAEgY0jBGJpmcKJXl0j1YUTS64rKXbo5OoOrzr/HRdIreFZhQKJxkpyKqv90Pew/tM3KREihKBdo9pY/+tRjJ4ZyXcoFo0SoUroMOwDvkMwWxIXb0g96v13QCAEhCKIXbzBMAKIHN0hkNqWWzixItVZIPFVCQQCiMQS5BYUwcGW1gUQYsleb9cEK/dSjTG+0a6xSqgfGs53CKQG6TD6E3zwPT2ZnhACjcdCEX7Qd4CQaubfqDnupeThXlIa36EQQojFo0SIkGrGMAw6jPo/bPv7Nt+hEEJ4xqpYyIpkfIdh0SgRIoQH1rb2uPPoOTiOSqoRYsm+fLMDts7/je8wLBotliYaMlKfVWl7efzNKyU7xF6SeOcfsEolChKuQfZCsUNlXjpURbkouHkcEIrBiKUQCIXgWBbKvAxwsgIo8jKRcnYXsm6cBAAUPLsPZWEOlFnJ2LtxBWztHUuO5+UgJ+0plEc3wa15tzIxyHOeQ6VUvNL4jMWvQWPEiT1x8XYSwhrVMXp/hBDT1K1lIGbvOM13GBaNEiGilpH6DCu+eBcKle5ZCrGQwYfz1sLVwxvxN69gySfjwAnL1ldSFheAAZBzfrvGcVYpB8OxKLgZq+XuJYUTGVaBvPsXkC8Q4r8i9Aw4hsGl0ydeKrYoQPads8i6ex4iadldWIxKjsL8XL3GZ2wuXrWRk687KSOEEGJ8lAgRNXlxERQqDi6VrMtTkJsNTli2VhAA5N0+hbxrh+DcaTyEzp4oLRtW/PQ28i7ugnPXiRA5egCMAAwjAKdSgC3MAauUI+f0FjhGDIfY1RecUg62MPe/421HQOzyXxVWReYzZB1ZC6egNvAKH6QZb1Yqsv7+FTZ2DnqNz9iadByIr+Z/gK6tGtDOEUII4QklQqSMqtblKVMrCEDR0zsAGIjd6pSc+7eCKlucD4ZhIHbxgdjFF4xACEYoAqsohjInTV1HSOziC6lHfbDyIihzn79w3AdSj38LKr6wvkZq7wJ77wCNGERiCXKFwlcen7G4evrAysEVLMuB8iBCCOEH/fNLCI98g5rj+11UUI0QQvhCiRAhPOo4YjL+Oh/PdxiEEGKxKBEihEcMw6BApkRaVh7foRBCiEWiRIgQnnUe9zm+23KK7zAIITxhi4rxLCG54obEKGixNClDlplSpePyzGdljilznwMAFFnPoN4yBkCRXXIPReYzcColGEYACATglP/tGuNYFRSZTwEGGrvG1MdfoPi3b2VhDoqeJ73SOHQdNzanWp7IUbK89E0I4d+icT3ww8GL8H5/AN+hWCRKhIiaxMoaYiGDzFObypxTqVQAx0HEANnpJc/IKirIB5RyZB5Zp5HsAIBKLgNYJbKOrNU4zqqUYLQcL1FSR4hTypAZsxrMy3WElDJkHl7zUh0hgFMUo+D+RSSnPyxzR7GQgcTKusLxaWtfXUQSCRKepYNlWdpGT4gFkorpRzGf6N0naq4e3vhw3toydXSyM9KweeksKFkOSoEAm1fMVZ+zdXIBp1JBKGQwYMyHsHdyUZ+7cf4E5DIZXNz/q9lTlJ+H2D1bwMkLAAggFIsBACqFApxKCQiFKMmqVOBYVclFHAeAA1RKBIaEwKduoPp+xYUFsHV0Rof+w7WO6cVK0brGp6t9dXFydYddgwj8dfoGBrYPqda+CSHE0lEiRDToTAKEErh3Kr8QYb1GofCqU199PCS8c5m2yY/i8U/cuTJFDdOuHEHaxf1w7vYuxM7eZWaYSgsnNgxtjQHjpug3OJQzPp45e9dHTn4c32EQQojFoUSIVJohCxG+fC+xnXPJ/7t4Q+LmD4Zh1EUYLUFwqwisnLMBgzo0hYNt9X40RwghlowWJBBiAhyca8GrXjCy8ujZY4QQUp0oESKEEEKIxaJEiBBTwQiQTTNChBBSrSgRIsREtHn9XXy0+jDfYRBCqtm1hBSIbaR8h2GxaLE0qTRDFiJ8+RpFflbJ/2c+Ky0npHleS9HGmqaWV20IJLRQmhBL8/2+C/hk/XS+w7BYZpcIrVq1CosWLUJKSgqaNWuGFStWICwsTGf7bdu24euvv0ZiYiICAwOxYMEC9OnTpxojNn+GLESo616FudngFMU6Ci2W4BTFcKrlUfnACSHEDNg72kFERRV5Y1bv/O+//45p06ZhzZo1aN26NZYtW4aePXvi7t27cHd3L9P+zJkzGDFiBKKiotCvXz9s2bIFgwYNwuXLl9GkSRMeRmCeDFmIsLx7XYzdj/ysdAhFYljb2Zc571TLQ2fhxJpCrmSRnp2PWk52fIdCCCEWgeE4jqu4mWlo3bo1XnvtNaxcuRIAwLIsateujQ8//BCff/55mfbDhg1DQUEB9u7dqz7Wpk0bhIaGYs2aNZXu98eTZR/dQIgxPL5/C/d3LMRvXw7lOxRCSDVp/2U0Jq38iO8wahypUIrBQRX/8mw2i6Xlcjni4uLQrVs39TGBQIBu3brh7NmzWq85e/asRnsA6Nmzp872ACCTyZCbm6vxUshlhhkEIRXwqhuAvGIF32EQQojFMJtEKD09HSqVCh4emmtEPDw8kJKifbFuSkpKldoDQFRUFBwdHTVeB36t/OwRIYQQQsyH2SRC1WXGjBnIycnRePUe+R7fYRFCCCHECMxmsXStWrUgFAqRmpqqcTw1NRWenmUfBAoAnp6eVWoPAFKpFFKpZj0HsSRdz6gJqTqONZtle4SQV7Qp5grc/XX/TCLGZzYzQhKJBC1btsTRo0fVx1iWxdGjRxEeHq71mvDwcI32ABATE6OzPSF8E4rEKBA64MTVeL5DIYRUg5+PXsbrU9/gOwyLZjaJEABMmzYNP/74IzZu3Ijbt2/j/fffR0FBAcaNGwcAiIyMxIwZM9Ttp0yZgoMHD2Lx4sW4c+cOvvnmG1y6dAmTJ0/mawiElIthGLQaMA7nbj/lOxRCSDUQCgQQCM3qR3GNYzYfjQEl2+GfP3+OmTNnIiUlBaGhoTh48KB6QXRSUhIEgv/+g4qIiMCWLVvw1Vdf4YsvvkBgYCB27dpFNYQIIYQQAsDMEiEAmDx5ss4ZnePHj5c5NnToUAwdSjVZCCGEmB65XMl3CBaP5uMIIYQQHqzbfxG+oQF8h2HxKBEihBBCeHD7STpa9mrFdxgWjxIhQgghhFgsSoQIIYQQHmTkFoBhGL7DsHiUCBFCCCHV7Mz1h0jhOHj7e/EdisWjRIgQE0S1pQmp2dKzC1A/NIBmhEwAJUKEmBjvugHYdfYB0rLy+A6FEGIkBcVygJIgk0CJECEmxt7JFfVbdcKDJ8/5DoUQYgSFxXIs2nMOLbq24DsUAkqECCGEkGpVUCSDZx1P2DvZ8R0KASVChBBCSLWSK1V8h0BeQIkQIYQQUo0mrd6H1oPa8h0G+RclQoQQQkg1yi5WoGHLBnyHQf5FiRAhhBBSTQ5evAupky3fYZAXUCJECCGEVJOfY67g9U/e5DsM8gJKhAghhJBqIFco8TglExKphO9QyAsoESLExMhlxXh47Ry8XB35DoUQYkDRhy6haZ/WsLKR8h0KeQElQoSYmITb19AzxAP+3q58h0IIMZCc/CKsP3IVoR2b8R0KeQklQoSYIGuJmO8QCCEGlPAsA3VbBMLV04XvUMhLKBEihBBCjOzQlXg4ujnxHQbRghIhQgghxIhUKhbbzt5Cp2Gd+A6FaEGJECGEEGJEU9fuR/uhncDQ0+ZNEiVChBBCiBHdevIcrfu05jsMogMlQoQQQoiR3Hj4DIUqlu8wSDkoESKEEEKMZMXeixjw0WC+wyDloESIEBOT/vgBbK1EfIdBCHlFl+8+xr3cAtRp6Md3KKQclAgRYkJUSiVuxWzFu/1pPQEh5ozjOPyw7yLaDIiAQEg/ak0ZfXcIMSEcx8Ld1QFikZDvUAghryAtKw/3cwvQrH0I36GQClAiRAghhBjYuyv/QtfIHnyHQSqBEiFCCCHEgO4/TkNaoQwNmgfyHQqpBEqECDEhLEvbbAkxdzN/jcXQz4fzHQapJEqECDEhB9fOwdju9HRqQszV2esJeFYsh0dtd75DIZVkNolQZmYmRo4cCQcHBzg5OeHtt99Gfn5+udd06lRS0vzF13vvvVdNERNSdflpj/FGh8Z8h0EI0dOGI1fQaWQ3CARm8+PV4plNsZKRI0ciOTkZMTExUCgUGDduHN555x1s2bKl3OsmTpyIOXPmqL+2sbExdqiEEEIs0N1HqUiQKdGlVUO+QyFVYBaJ0O3bt3Hw4EFcvHgRrVq1AgCsWLECffr0wf/+9z94e3vrvNbGxgaenp7VFSoheuM4DjKZjO8wCCF6KJYpMO2ng4gY1Y3vUEgVmcXc3dmzZ+Hk5KROggCgW7eSqcfz58+Xe+2vv/6KWrVqoUmTJpgxYwYKCwvLbS+TyZCbm6vxUsjphxMxvlPb1mJYO9plQog5en/VXwh9sxOCw4L5DoVUkVkkQikpKXB311x4JhKJ4OLigpSUFJ3XvfXWW9i8eTNiY2MxY8YMbNq0CaNGjSq3r6ioKDg6Omq8Dvy6xiDjIKQ8ualJGNQ2iO8wCCFV9CQtC9cfpaJpu6Z8h0L0wOtHY59//jkWLFhQbpvbt2/rff933nlH/eemTZvCy8sLXbt2RXx8POrXr6/1mhkzZmDatGkaxzZfeKp3DIQQQmq25XvOoc+kgXyHQfTEayL0f//3fxg7dmy5berVqwdPT0+kpaVpHFcqlcjMzKzS+p/WrUue3/TgwQOdiZBUKoVUKtU4JpakV7oPQvTBcRyyM9IhomcSEWJWjl1+gDMPU/DuZHrCvLniNRFyc3ODm5tbhe3Cw8ORnZ2NuLg4tGzZEgBw7NgxsCyrTm4q4+rVqwAALy8vveIlxFhunD+B5p5C+Lg58R0KIaSSimRy/N+GQ5i6dhokUgnf4RA9mcWvn8HBwejVqxcmTpyICxcu4PTp05g8eTKGDx+u3jH29OlTBAUF4cKFCwCA+Ph4fPvtt4iLi0NiYiL27NmDyMhIdOjQASEh9BA8YloK83LRJogSdELMyfKdZ9GsU3NIraUVNyYmyywSIaBk91dQUBC6du2KPn36oF27dli3bp36vEKhwN27d9W7wiQSCY4cOYIePXogKCgI//d//4fXX38df/31F19DIESn4oIcCBiG7zAIIZW0dt8FXMwvQs9xPfkOhbwihuM4ju8gTN2PJx/yHQKpwfJzsrB9zngcnj8atvSbJSEmT6FUofP0HzFu6STYOtjyHQ7RQSqUYnBQxc98M5sZIUJqquLCfDSt50lJECFmQKlUYXjUVrQZ3oWSoBqCEiFCeHbi1+Xo1bIe32EQQirhxsNkKF0c0ap7q4obE7NAiRAhPFNkPcWgdo34DoMQUoHUzFy8v3ovOgzrxHcoxIAoESKEZwwtkibE5KVk5GLQt1vwxpej4FNf9/MtifmhRIgQHl2N3YMG3o58h0EIqcCKv86h84Q+lATVQJQIEcKjG7G7sOz9XnyHQQgpx9nrCTh6KwkNm9NDkWsiSoQI4UlxYQFyMtMhFNBfQ0JM1ZnrCfh48zG8s/h9KpxYQ/H6iA1CLNmFfb/i86FhENLzxQgxWdOjY/Du8kmwtrXmOxRiJPQvMCE8yMl4jgdnD6J7qwZ8h0II0WHFrjNwa+BLSVANR4kQITx4/uwR+rUJgKsjFWQjxBQt33kaMc8y8eanw/gOhRgZJUKE8OD+ucPw93DiOwxCiBbLdp7G0eQsjPh8BJW3sACUCBFSzViWRfq9S3irazO+QyGEvGT13vM4npJNSZAFoUSoEhQyGd8hkBokOz0FTna05oAQU6NSsfj5wAUM+2w4JUEWhBKhStj7w9d8h0BqkH0rvsTSd7vzHQYh5AVKpQpvLfgD7d7qSkmQhaFEqBJyntxH6pMEvsMgNUD89YtwEslR39eN71AIIf9SKlUYueAPeHdshjZ92vAdDqlmlAhVwsy32uLehWN8h0FqgLh9m2g2iBATolSq8Nb8P+DduTla92nNdziEB5QIVYKTnTVS4m+D4zi+QyFm7O6Vc/CT5qOeTy2+QyGEoCQJGj7/d9Tu1gKte4fxHQ7hCSVCldC6cV34irLx5MFtvkMhZoplWcTt24S3ezWn9QeEmACFUoVhUVtRp3srvNbzNb7DITyiRKiSxvUMxYnNS/gOg5iptCeJqG1djIim9fgOhRCLp1CqMGzeVvj3fA2v9WjFdziEZ5QIVVL7EH84MIVIT37CdyjEDB1a8w2mD6VFmITwTaFU4c15v6F+rzC06k5JEKFEqEpWTu6NA7SVnlRR0t3rcLVSoUk9b75DIcTi/bDnHDzDG6Nl95Z8h0JMBCVCVVDH0wXOYjni/znPdyjEjJz4dRl+/Lg/32EQYvFSMnKx9dR1hHSkqu7kP5QIVdHCt7si7sBvfIdBzMTtiydQ14GDm5Md36EQYtGS03Mw6NsteOOrUXB2c+I7HGJCKBGqogZ+7gitxeL41h/4DoWYOJZlceXwNkwb0pp2ihHCE5lcgSXbT2Hwd7/hzZmj4UMfUZOXUCJURQzDYOE7PZB54wTysjP5DoeYsEd3r6ORsxKhDXz5DoUQiySTK/DGd78h3tURYxe9A29/L75DIiaIEiE9MAyDDwaG4dCP3/EdCjFRRQV5iN24EON7hvIdCiEWSSZX4PXvfkOTIe3RflA7ONVy4jskYqIYjsolV+zhca2H52+JRfvhH1ZvLMQsHNu1Bd3ridEmNJjvUAixSLtjz+OxVII2XcP5DoXwRCwQo5lHJXYHcqRcxcXF3KxZs7ji4mK+Q6l2ljp2Sx03x9HYaew0dkthqePWhmaEKpCbmwtHR0fk5OTAwcGB73CqlaWO3VLHDdDYaew0dkthqePWhtYIEUIIIcRiUSJECCGEEItFiRAhhBBCLBYlQhWQSqWYNWsWpFIp36FUO0sdu6WOG6Cx09hp7JbCUsetDS2WJoQQQojFohkhQgghhFgsSoQIIYQQYrEoESKEEEKIxaJEiBBCCCEWixKhcqxatQp169aFlZUVWrdujQsXLvAdUrU4efIk+vfvD29vbzAMg127dvEdUrWIiorCa6+9Bnt7e7i7u2PQoEG4e/cu32FVi9WrVyMkJAQODg5wcHBAeHg4Dhw4wHdY1W7+/PlgGAZTp07lO5Rq8c0334BhGI1XUFAQ32FVi6dPn2LUqFFwdXWFtbU1mjZtikuXLvEdltHVrVu3zPecYRhMmjSJ79B4Q4mQDr///jumTZuGWbNm4fLly2jWrBl69uyJtLQ0vkMzuoKCAjRr1gyrVq3iO5RqdeLECUyaNAnnzp1DTEwMFAoFevTogYKCAr5DMzpfX1/Mnz8fcXFxuHTpErp06YKBAwfi5s2bfIdWbS5evIi1a9ciJCSE71CqVePGjZGcnKx+/f3333yHZHRZWVlo27YtxGIxDhw4gFu3bmHx4sVwdnbmOzSju3jxosb3OyYmBgAwdOhQniPjEb+POjNdYWFh3KRJk9Rfq1Qqztvbm4uKiuIxquoHgNu5cyffYfAiLS2NA8CdOHGC71B44ezszP300098h1Et8vLyuMDAQC4mJobr2LEjN2XKFL5DqhazZs3imjVrxncY1e6zzz7j2rVrx3cYJmHKlClc/fr1OZZl+Q6FNzQjpIVcLkdcXBy6deumPiYQCNCtWzecPXuWx8hIdcrJyQEAuLi48BxJ9VKpVNi6dSsKCgoQHh7OdzjVYtKkSejbt6/G33lLcf/+fXh7e6NevXoYOXIkkpKS+A7J6Pbs2YNWrVph6NChcHd3R/PmzfHjjz/yHVa1k8vl2Lx5M8aPHw+GYfgOhzeUCGmRnp4OlUoFDw8PjeMeHh5ISUnhKSpSnViWxdSpU9G2bVs0adKE73CqxfXr12FnZwepVIr33nsPO3fuRKNGjfgOy+i2bt2Ky5cvIyoqiu9Qql3r1q0RHR2NgwcPYvXq1UhISED79u2Rl5fHd2hG9fDhQ6xevRqBgYE4dOgQ3n//fXz00UfYuHEj36FVq127diE7Oxtjx47lOxReifgOgBBTNGnSJNy4ccMi1kuUatiwIa5evYqcnBxs374dY8aMwYkTJ2p0MvT48WNMmTIFMTExsLKy4jucate7d2/1n0NCQtC6dWvUqVMHf/zxB95++20eIzMulmXRqlUrzJs3DwDQvHlz3LhxA2vWrMGYMWN4jq76/Pzzz+jduze8vb35DoVXNCOkRa1atSAUCpGamqpxPDU1FZ6enjxFRarL5MmTsXfvXsTGxsLX15fvcKqNRCJBQEAAWrZsiaioKDRr1gzLly/nOyyjiouLQ1paGlq0aAGRSASRSIQTJ07g+++/h0gkgkql4jvEauXk5IQGDRrgwYMHfIdiVF5eXmUS/ODgYIv4WLDUo0ePcOTIEUyYMIHvUHhHiZAWEokELVu2xNGjR9XHWJbF0aNHLWbNhCXiOA6TJ0/Gzp07cezYMfj7+/MdEq9YloVMJuM7DKPq2rUrrl+/jqtXr6pfrVq1wsiRI3H16lUIhUK+Q6xW+fn5iI+Ph5eXF9+hGFXbtm3LlMa4d+8e6tSpw1NE1W/Dhg1wd3dH3759+Q6Fd/TRmA7Tpk3DmDFj0KpVK4SFhWHZsmUoKCjAuHHj+A7N6PLz8zV+I0xISMDVq1fh4uICPz8/HiMzrkmTJmHLli3YvXs37O3t1evBHB0dYW1tzXN0xjVjxgz07t0bfn5+yMvLw5YtW3D8+HEcOnSI79CMyt7evswaMFtbW7i6ulrE2rBPPvkE/fv3R506dfDs2TPMmjULQqEQI0aM4Ds0o/r4448RERGBefPm4c0338SFCxewbt06rFu3ju/QqgXLstiwYQPGjBkDkYjSANo+X44VK1Zwfn5+nEQi4cLCwrhz587xHVK1iI2N5QCUeY0ZM4bv0IxK25gBcBs2bOA7NKMbP348V6dOHU4ikXBubm5c165ducOHD/MdFi8safv8sGHDOC8vL04ikXA+Pj7csGHDuAcPHvAdVrX466+/uCZNmnBSqZQLCgri1q1bx3dI1ebQoUMcAO7u3bt8h2ISGI7jOH5SMEIIIYQQftEaIUIIIYRYLEqECCGEEGKxKBEihBBCiMWiRIgQQgghFosSIUIIIYRYLEqECCGEEGKxKBEihBBCiMWiRIgQUqMcP34cDMMgOzu7wrbR0dFwcnIyekyVVbduXSxbtozvMAixKJQIEUJMkqklKYZUk8dGiLmhRIgQQgghFosSIUKIUXTq1AmTJ0/G5MmT4ejoiFq1auHrr79G6VN9ZDIZPvnkE/j4+MDW1hatW7fG8ePHAZR8vDVu3Djk5OSAYRgwDINvvvkGALBp0ya0atUK9vb28PT0xFtvvYW0tDSDxb179260aNECVlZWqFevHmbPng2lUqk+zzAMfvrpJwwePBg2NjYIDAzEnj17NO6xZ88eBAYGwsrKCp07d8bGjRvVH9eVNzYAKCwsxPjx42Fvbw8/Pz+LeRAoIbzh+VlnhJAaqmPHjpydnR03ZcoU7s6dO9zmzZs5Gxsb9cMtJ0yYwEVERHAnT57kHjx4wC1atIiTSqXcvXv3OJlMxi1btoxzcHDgkpOTueTkZC4vL4/jOI77+eefuf3793Px8fHc2bNnufDwcK53797qfksfGpyVlVVhjBs2bOAcHR3VX588eZJzcHDgoqOjufj4eO7w4cNc3bp1uW+++UbdBgDn6+vLbdmyhbt//z730UcfcXZ2dlxGRgbHcRz38OFDTiwWc5988gl3584d7rfffuN8fHzUMZU3tjp16nAuLi7cqlWruPv373NRUVGcQCDg7ty586rfDkKIDpQIEUKMomPHjlxwcDDHsqz62GeffcYFBwdzjx494oRCIff06VONa7p27crNmDGD47iySYouFy9e5ACok4lXSYS6du3KzZs3T6PNpk2bOC8vL/XXALivvvpK/XV+fj4HgDtw4IB6jE2aNNG4x5dffqkRk66x1alThxs1apT6a5ZlOXd3d2716tUVjoUQoh8Rf3NRhJCark2bNmAYRv11eHg4Fi9ejOvXr0OlUqFBgwYa7WUyGVxdXcu9Z1xcHL755htcu3YNWVlZYFkWAJCUlIRGjRq9UrzXrl3D6dOnMXfuXPUxlUqF4uJiFBYWwsbGBgAQEhKiPm9rawsHBwf1x3N3797Fa6+9pnHfsLCwSsfw4r0ZhoGnp6dBP/ojhGiiRIgQUu3y8/MhFAoRFxcHoVCocc7Ozk7ndQUFBejZsyd69uyJX3/9FW5ubkhKSkLPnj0hl8sNEtfs2bMxZMiQMuesrKzUfxaLxRrnGIZRJ2Svypj3JoSURYkQIcRozp8/r/H1uXPnEBgYiObNm0OlUiEtLQ3t27fXeq1EIoFKpdI4dufOHWRkZGD+/PmoXbs2AODSpUsGi7dFixa4e/cuAgIC9L5Hw4YNsX//fo1jFy9e1Pha29gIIfygXWOEEKNJSkrCtGnTcPfuXfz2229YsWIFpkyZggYNGmDkyJGIjIzEjh07kJCQgAsXLiAqKgr79u0DUFJcMD8/H0ePHkV6ejoKCwvh5+cHiUSCFStW4OHDh9izZw++/fZbg8U7c+ZM/PLLL5g9ezZu3ryJ27dvY+vWrfjqq68qfY93330Xd+7cwWeffYZ79+7hjz/+QHR0NACoPybUNjZCCD8oESKEGE1kZOT/t3eHLAoEYRjHn0uWTStbXUzCBrGsiGmbmNZmM4qgYNhoFEVUsKnN7YrJZDf5EYziZ9jolSt3eGDwTnH+P5g0MPNOe+AdZpQkiYrFotrttrrdrprNpiRptVqp0WgoiiLlcjnVajUdj0dlMhlJUrlcVqvVUr1el+M4Go/HchxHcRxrvV7L8zyNRiNNp9OH1VupVLTb7bTf7+X7vkqlkmazmVzXvXuNbDarzWaj7XarfD6vxWKhXq8nSUqlUr+eDcBzfFyvX496AMADBUGgQqHAlxGSBoOBlsulzufzs0sB8AN3hADgwebzuXzfVzqd1uFw0GQyUafTeXZZAG6gNQbgbVWrVVmWdXMMh8M/2/d0OikMQ3mep36/ryiKvr0eDeB10BoD8LYul4uSJLk5Z9u2bNv+54oAvBqCEAAAMBatMQAAYCyCEAAAMBZBCAAAGIsgBAAAjEUQAgAAxiIIAQAAYxGEAACAsQhCAADAWJ/3lZMaKVKHiQAAAABJRU5ErkJggg==\n" 398 | }, 399 | "metadata": {} 400 | } 401 | ] 402 | }, 403 | { 404 | "cell_type": "markdown", 405 | "source": [ 406 | "### **Pros and cons of the Naive Bayes classifier**\n", 407 | "Pros:\n", 408 | "- easy to implement and interpret;\n", 409 | "- practically no parameter settings are required;\n", 410 | "- high speed and accuracy of predictions in many situations;\n", 411 | "- it has relatively good resistance to noise and outliers, since it is based on probability distributions and a naive assumption of the independence of features.\n", 412 | "\n", 413 | "Cons:\n", 414 | "- in a case of violation of the assumption of independence of features, the accuracy of predictions may significantly decrease;\n", 415 | "- may give preference to classes with a large number of samples in case of unbalanced data." 416 | ], 417 | "metadata": { 418 | "id": "01g1BCfwaFwn" 419 | } 420 | }, 421 | { 422 | "cell_type": "markdown", 423 | "source": [ 424 | "### **Additional sources**\n", 425 | "Paper «Bayes and Naive-Bayes Classifier», Rajiv Gandhi, Andhra Pradesh.\n", 426 | "\n", 427 | "Documentation:\n", 428 | "- [Naive Bayes description](https://scikit-learn.org/stable/modules/naive_bayes.html);\n", 429 | "- [GaussianNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.GaussianNB.html#sklearn.naive_bayes.GaussianNB);\n", 430 | "- [MultinomialNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html#sklearn.naive_bayes.MultinomialNB);\n", 431 | "- [ComplementNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.ComplementNB.html#sklearn.naive_bayes.ComplementNB);\n", 432 | "- [BernoulliNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.BernoulliNB.html#sklearn.naive_bayes.BernoulliNB);\n", 433 | "- [CategoricalNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.CategoricalNB.html#sklearn.naive_bayes.CategoricalNB).\n", 434 | "\n", 435 | "Video: [one](https://www.youtube.com/watch?v=O2L2Uv9pdDA), [two](https://www.youtube.com/watch?v=H3EjCKtlVog), [three](https://www.youtube.com/watch?v=nt63k3bfXS0), [four](https://www.youtube.com/watch?v=ADj95edZc0w).\n", 436 | "\n" 437 | ], 438 | "metadata": { 439 | "id": "ovU3wJ2aaojO" 440 | } 441 | } 442 | ] 443 | } -------------------------------------------------------------------------------- /Notebooks [eng]/ML-algorithms from scratch/12) Principal Component Analysis (PCA).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [] 7 | }, 8 | "kernelspec": { 9 | "name": "python3", 10 | "display_name": "Python 3" 11 | }, 12 | "language_info": { 13 | "name": "python" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "markdown", 19 | "source": [ 20 | "## **Principal Component Analysis**\n", 21 | "Principal Component Analysis or PCA is an unsupervised learning algorithm used to reduce the dimension and identify the most informative features in the data. Its essence lies in the assumption of the linearity of data relations and their projection onto a subspace of orthogonal vectors in which the variance will be maximal. Such vectors are called the main components and they determine directions of the greatest variability (informativeness) of the data. Alternatively, the essence of PCA can be defined as a linear projection that minimizes the RMS distance between the source points and their projections." 22 | ], 23 | "metadata": { 24 | "id": "OjhUfMJ5v37E" 25 | } 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "source": [ 30 | "### **The principle of operation of PCA**\n", 31 | "Initially, the feature matrix is necessarily centered so that the first main component can correspond to the direction of maximum variation of the data and not just their average value. Usually, finding the main components comes down to two main methods:\n", 32 | " - **Calculation of eigenvectors and eigenvalues of the covariance matrix of data**. Since the covariance matrix reflects the degree of linear relationship between different variables, the eigenvectors of this matrix will set the directions along which the data variance is maximum, and the eigenvalues will set the magnitude of this variance. The eigenvalue corresponding to the eigenvector characterizes the contribution of this vector to the explanation of the variance of the data and the greater the eigenvalue, the more significant the main component. Usually, only those main components are selected that explain a given level of variance, for example, 95%.\n", 33 | "\n", 34 | " - **Calculation of the singular value decomposition of the data matrix**. Singular value decomposition is a way of representing any matrix as a product of three other matrices: the left singular matrix U, the diagonal matrix of singular values S, and the right singular matrix V, where the singular values are the square roots of the eigenvalues of the covariance matrix of the data (this is what data pre-centering is done for in this case), the right singular matrix V it will correspond to the eigenvectors of the covariance matrix of the data, and the left U will be the projection of the original data onto the main components defined by the matrix V. Thus, the singular value decomposition also makes it possible to isolate the main components, but without the need to calculate the covariance matrix. Besides the fact that such a solution is more efficient, it is considered more numerically stable, since it does not require calculating the covariance matrix directly which may be poorly conditioned in the case of a strong correlation of features. Particularly this approach is used in the implementation of scikit-learn, but with some of the features discussed below." 35 | ], 36 | "metadata": { 37 | "id": "8OCN7le0v3vY" 38 | } 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "source": [ 43 | "**SVD-based PCA is constructed as follows:**\n", 44 | "- 1) data centering occurs first and the number of components is determined at least between the number of samples and features if the number of components has not been specified;\n", 45 | "- 2) Next, SVD is applied to the centered data matrix;\n", 46 | "- 3) the svd_flip_vector method is applied to the matrix U which finds the maximum modulo elements in each column of the matrix U, extracts their signs and multiplies the matrix U by these signs to guarantee a deterministic output;\n", 47 | "- 4) the explained variance for each principal component is calculated as squared corresponding singular values divided by n_samples - 1 and the transformed data is calculated taking into account the number of principal components according to the rule $X_{new} = X \\cdot V = U \\cdot S \\cdot V^T \\cdot V = U \\cdot S$." 48 | ], 49 | "metadata": { 50 | "id": "mppCbZCFw1fY" 51 | } 52 | }, 53 | { 54 | "cell_type": "markdown", 55 | "source": [ 56 | "### **Additional features of PCA**\n", 57 | "**The coefficient of explained variance** of each main component, available through the variable *explained_variance_ratio_*, indicates the proportion of variance of the dataset lying along the axis of each main component.\n", 58 | "\n", 59 | "**Data recovery** using the *inverse_transform()* method consists in applying the inverse transformation of the PCA projection of the form $X_{recovered} = X_{d-proj} W_d^T$, where $W_d^T$ is a matrix of the first d principal components. It follows that the data will be recovered with losses proportional to the amount of discarded variance of the original data and the average square of the distance between the original and restored data represents a *reconstruction error*.\n", 60 | "\n", 61 | "**Incremental PCA**, implemented as the [*IncrementalPCA*](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.IncrementalPCA) class, allows you to work more efficiently with large datasets by splitting them into mini-packages and storing them in memory piece by piece during training.\n", 62 | "\n", 63 | "**Randomized PCA**, set using the svd_solver='randomized' parameter, uses a stochastic algorithm to quickly calculate approximate d principal components and it is based on the assumption that a random projection of data onto a low-dimensional subspace can preserve their structure and properties well, however, this approach may be less accurate.\n", 64 | "\n", 65 | "**Kernel PCA**, implemented using the [*KernelPCA*](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.KernelPCA.html) class, allows you to perform complex non-linear projections using kernel functions. As in the case of SVM, its essence in this case is that the linear boundary of solutions in a multidimensional feature space will correspond to a complex non-linear boundary in the original space." 66 | ], 67 | "metadata": { 68 | "id": "y_dUBdo1yKEa" 69 | } 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "source": [ 74 | "### **Alternatives to PCA**\n", 75 | "Despite the fact that the principal component method is one of the most popular dimensionality reduction algorithms there are alternatives that may be more preferable in a number of situations, as well as depending on the type of data:\n", 76 | "- **LLE (Locally Linear Embedding)** is an algorithm for creating linear combinations of each point from its neighbors, followed by restoring these combinations in a lower dimensional space which allows you to preserve the nonlinear geometry of the data and be useful for some tasks where global properties are less important. On the other hand, this approach has high computational complexity and may be sensitive to noise.\n", 77 | "- **t-SNE (t-Distributed Stochastic Neighbor Embedding)** is an algorithm that converts similarities between data into probabilities and further tries to minimize the discrepancy between probability distributions in high and low dimensional space. t-SNE is effective in visualizing high-dimensional data, but it can distort the global data structure because it does not take into account linear dependencies, but only their proximity in the original space.\n", 78 | "- **UMAP (Uniform Manifold Approximation and Projection)** is another algorithm suitable for data visualization which is based on the idea that the data lies on some homogeneous manifold that can be approximated using a neighbor graph. This approach takes into account the global data structure and allows for better adaptation to different types of data and better handling of noise and outliers than t-SNE.\n", 79 | "- **Autoencoders** is a type of neural networks based on training the encoder to convert input data into a low-dimensional representation, followed by training the decoder to restore the original data from this representation. Autoencoders can also be used for data compression, noise removal, and many other purposes." 80 | ], 81 | "metadata": { 82 | "id": "kQQAKxouym9g" 83 | } 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "source": [ 88 | "### **Python implementation from scratch**" 89 | ], 90 | "metadata": { 91 | "id": "d0NqRbif1Ukt" 92 | } 93 | }, 94 | { 95 | "cell_type": "code", 96 | "source": [ 97 | "import numpy as np\n", 98 | "from sklearn.decomposition import PCA\n", 99 | "from sklearn.datasets import load_iris" 100 | ], 101 | "metadata": { 102 | "id": "m0OOS98L16no" 103 | }, 104 | "execution_count": 1, 105 | "outputs": [] 106 | }, 107 | { 108 | "cell_type": "code", 109 | "source": [ 110 | "class SVDPCA:\n", 111 | " def __init__(self, n_components=None):\n", 112 | " self.n_components = n_components\n", 113 | "\n", 114 | " @staticmethod\n", 115 | " def svd_flip_vector(U):\n", 116 | " max_abs_cols_U = np.argmax(np.abs(U), axis=0)\n", 117 | " # extract the signs of the max absolute values\n", 118 | " signs_U = np.sign(U[max_abs_cols_U, range(U.shape[1])])\n", 119 | "\n", 120 | " return U * signs_U\n", 121 | "\n", 122 | " def fit_transform(self, X):\n", 123 | " n_samples, n_features = X.shape\n", 124 | " X_centered = X - X.mean(axis=0)\n", 125 | "\n", 126 | " if self.n_components is None:\n", 127 | " self.n_components = min(n_samples, n_features)\n", 128 | "\n", 129 | " U, S, Vt = np.linalg.svd(X_centered)\n", 130 | " # flip the eigenvector sign to enforce deterministic output\n", 131 | " U_flipped = self.svd_flip_vector(U)\n", 132 | "\n", 133 | " self.explained_variance = (S[:self.n_components] ** 2) / (n_samples - 1)\n", 134 | " self.explained_variance_ratio = self.explained_variance / np.sum(self.explained_variance)\n", 135 | "\n", 136 | " # X_new = X * V = U * S * Vt * V = U * S\n", 137 | " X_transformed = U_flipped[:, : self.n_components] * S[: self.n_components]\n", 138 | "\n", 139 | " return X_transformed" 140 | ], 141 | "metadata": { 142 | "id": "DO5dvAD_16b3" 143 | }, 144 | "execution_count": 2, 145 | "outputs": [] 146 | }, 147 | { 148 | "cell_type": "markdown", 149 | "source": [ 150 | "### **Uploading a dataset**" 151 | ], 152 | "metadata": { 153 | "id": "tSIQ9NJH2EGj" 154 | } 155 | }, 156 | { 157 | "cell_type": "code", 158 | "source": [ 159 | "X, y = load_iris(return_X_y=True, as_frame=True)\n", 160 | "print(X)" 161 | ], 162 | "metadata": { 163 | "colab": { 164 | "base_uri": "https://localhost:8080/" 165 | }, 166 | "id": "7m0fRrsi2FSD", 167 | "outputId": "1148a141-7959-49ef-eea3-614e2ffc8843" 168 | }, 169 | "execution_count": 3, 170 | "outputs": [ 171 | { 172 | "output_type": "stream", 173 | "name": "stdout", 174 | "text": [ 175 | " sepal length (cm) sepal width (cm) petal length (cm) petal width (cm)\n", 176 | "0 5.1 3.5 1.4 0.2\n", 177 | "1 4.9 3.0 1.4 0.2\n", 178 | "2 4.7 3.2 1.3 0.2\n", 179 | "3 4.6 3.1 1.5 0.2\n", 180 | "4 5.0 3.6 1.4 0.2\n", 181 | ".. ... ... ... ...\n", 182 | "145 6.7 3.0 5.2 2.3\n", 183 | "146 6.3 2.5 5.0 1.9\n", 184 | "147 6.5 3.0 5.2 2.0\n", 185 | "148 6.2 3.4 5.4 2.3\n", 186 | "149 5.9 3.0 5.1 1.8\n", 187 | "\n", 188 | "[150 rows x 4 columns]\n" 189 | ] 190 | } 191 | ] 192 | }, 193 | { 194 | "cell_type": "markdown", 195 | "source": [ 196 | "### **Model training and evaluation of the obtained results**\n", 197 | "The manual implementation showed identical scikit-learn results. As you can see, the first 2 main components explain almost 98% of the variance in the data which allows you to reduce the number of features by half without much loss of information. If the number of features were not 4 but several thousand or millions, then this would significantly reduce the training time of models without significant loss of accuracy (and sometimes with increased accuracy by reducing multicollinearity between features) what makes PCA and its alternatives an excellent addition to other algorithms." 198 | ], 199 | "metadata": { 200 | "id": "73RONIPv2NUU" 201 | } 202 | }, 203 | { 204 | "cell_type": "markdown", 205 | "source": [ 206 | "**PCA**" 207 | ], 208 | "metadata": { 209 | "id": "haSyFD092g1j" 210 | } 211 | }, 212 | { 213 | "cell_type": "code", 214 | "source": [ 215 | "pca = SVDPCA()\n", 216 | "X_transformed = pca.fit_transform(X)\n", 217 | "\n", 218 | "print('transformed data', X_transformed[:10], '', sep='\\n')\n", 219 | "print('explained_variance', pca.explained_variance)\n", 220 | "print('explained_variance_ratio', pca.explained_variance_ratio)" 221 | ], 222 | "metadata": { 223 | "colab": { 224 | "base_uri": "https://localhost:8080/" 225 | }, 226 | "id": "-q44z3MS2gTK", 227 | "outputId": "a8bb811f-828f-49fa-cdc3-c8b5174139ee" 228 | }, 229 | "execution_count": 4, 230 | "outputs": [ 231 | { 232 | "output_type": "stream", 233 | "name": "stdout", 234 | "text": [ 235 | "transformed data\n", 236 | "[[-2.68412563e+00 3.19397247e-01 -2.79148276e-02 -2.26243707e-03]\n", 237 | " [-2.71414169e+00 -1.77001225e-01 -2.10464272e-01 -9.90265503e-02]\n", 238 | " [-2.88899057e+00 -1.44949426e-01 1.79002563e-02 -1.99683897e-02]\n", 239 | " [-2.74534286e+00 -3.18298979e-01 3.15593736e-02 7.55758166e-02]\n", 240 | " [-2.72871654e+00 3.26754513e-01 9.00792406e-02 6.12585926e-02]\n", 241 | " [-2.28085963e+00 7.41330449e-01 1.68677658e-01 2.42008576e-02]\n", 242 | " [-2.82053775e+00 -8.94613845e-02 2.57892158e-01 4.81431065e-02]\n", 243 | " [-2.62614497e+00 1.63384960e-01 -2.18793179e-02 4.52978706e-02]\n", 244 | " [-2.88638273e+00 -5.78311754e-01 2.07595703e-02 2.67447358e-02]\n", 245 | " [-2.67275580e+00 -1.13774246e-01 -1.97632725e-01 5.62954013e-02]]\n", 246 | "\n", 247 | "explained_variance [4.22824171 0.24267075 0.0782095 0.02383509]\n", 248 | "explained_variance_ratio [0.92461872 0.05306648 0.01710261 0.00521218]\n" 249 | ] 250 | } 251 | ] 252 | }, 253 | { 254 | "cell_type": "markdown", 255 | "source": [ 256 | "**PCA (scikit-learn)**" 257 | ], 258 | "metadata": { 259 | "id": "KPNSTfkk2noR" 260 | } 261 | }, 262 | { 263 | "cell_type": "code", 264 | "source": [ 265 | "sk_pca = PCA()\n", 266 | "sk_X_transformed = sk_pca.fit_transform(X)\n", 267 | "\n", 268 | "print('sk transformed data', sk_X_transformed[:10], '', sep='\\n')\n", 269 | "print('sk explained_variance', sk_pca.explained_variance_)\n", 270 | "print('sk explained_variance_ratio_', sk_pca.explained_variance_ratio_)" 271 | ], 272 | "metadata": { 273 | "colab": { 274 | "base_uri": "https://localhost:8080/" 275 | }, 276 | "id": "avObFcAI2oZh", 277 | "outputId": "c47941f6-231d-4969-a521-db26d264f8da" 278 | }, 279 | "execution_count": 5, 280 | "outputs": [ 281 | { 282 | "output_type": "stream", 283 | "name": "stdout", 284 | "text": [ 285 | "sk transformed data\n", 286 | "[[-2.68412563e+00 3.19397247e-01 -2.79148276e-02 -2.26243707e-03]\n", 287 | " [-2.71414169e+00 -1.77001225e-01 -2.10464272e-01 -9.90265503e-02]\n", 288 | " [-2.88899057e+00 -1.44949426e-01 1.79002563e-02 -1.99683897e-02]\n", 289 | " [-2.74534286e+00 -3.18298979e-01 3.15593736e-02 7.55758166e-02]\n", 290 | " [-2.72871654e+00 3.26754513e-01 9.00792406e-02 6.12585926e-02]\n", 291 | " [-2.28085963e+00 7.41330449e-01 1.68677658e-01 2.42008576e-02]\n", 292 | " [-2.82053775e+00 -8.94613845e-02 2.57892158e-01 4.81431065e-02]\n", 293 | " [-2.62614497e+00 1.63384960e-01 -2.18793179e-02 4.52978706e-02]\n", 294 | " [-2.88638273e+00 -5.78311754e-01 2.07595703e-02 2.67447358e-02]\n", 295 | " [-2.67275580e+00 -1.13774246e-01 -1.97632725e-01 5.62954013e-02]]\n", 296 | "\n", 297 | "sk explained_variance [4.22824171 0.24267075 0.0782095 0.02383509]\n", 298 | "sk explained_variance_ratio_ [0.92461872 0.05306648 0.01710261 0.00521218]\n" 299 | ] 300 | } 301 | ] 302 | }, 303 | { 304 | "cell_type": "markdown", 305 | "source": [ 306 | "### **Pros and cons**\n", 307 | "Pros:\n", 308 | "- reducing the dimension while preserving a large amount of information which also allows you to visualize high-dimensional data in two-dimensional or three-dimensional space;\n", 309 | "- not only allows you to significantly speed up training, but also reduce the overfitting of models in some cases;\n", 310 | "- can be used to compress data.\n", 311 | "\n", 312 | "Cons:\n", 313 | "- the inevitable loss of some information in the data;\n", 314 | "- search only for linear dependence in the data (in the classic PCA);\n", 315 | "- the lack of semantic meaning of the main components due to the difficulty of linking them with real features." 316 | ], 317 | "metadata": { 318 | "id": "sXRg_DP12v-q" 319 | } 320 | }, 321 | { 322 | "cell_type": "markdown", 323 | "source": [ 324 | "### **Additional sources**\n", 325 | "Papers:\n", 326 | "- «A Tutorial on Principal Component Analysis», Jonathon Shlens;\n", 327 | "- «Locally Linear Embedding and its Variants: Tutorial and Survey», Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray, Mark Crowley;\n", 328 | "- «Theoretical Foundations of t-SNE for Visualizing High-Dimensional Clustered Data», T. Tony Cai, Rong Ma;\n", 329 | "- «UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction», Leland McInnes, John Healy, James Melville;\n", 330 | "- «Deep Autoencoders for Dimensionality Reduction of High-Content Screening Data», Lee Zamparo, Zhaolei Zhang.\n", 331 | "\n", 332 | "Documentation:\n", 333 | "- [PCA description](https://scikit-learn.org/stable/modules/decomposition.html#pca);\n", 334 | "- [LLE description](https://scikit-learn.org/stable/modules/manifold.html#locally-linear-embedding);\n", 335 | "- [t-SNE description](https://scikit-learn.org/stable/modules/manifold.html#t-sne);\n", 336 | "- [PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html);\n", 337 | "- [LLE](https://scikit-learn.org/stable/modules/generated/sklearn.manifold.LocallyLinearEmbedding.html);\n", 338 | "- [t-SNE](https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html);\n", 339 | "- [UMAP](https://umap-learn.readthedocs.io/en/latest/index.html).\n", 340 | "\n", 341 | "Видео:\n", 342 | "- PCA: [one](https://www.youtube.com/watch?v=FgakZw6K1QQ), [two](https://www.youtube.com/watch?v=fkf4IBRSeEc), [three](https://www.youtube.com/watch?v=IwPzjlBXBlA), [four](https://www.youtube.com/watch?v=WW3ZJHPwvyg);\n", 343 | "- [LLE](https://www.youtube.com/watch?v=B6kzA1W_4pU);\n", 344 | "- [t-SNE](https://www.youtube.com/watch?v=NEaUSP4YerM);\n", 345 | "- [UMAP](https://www.youtube.com/watch?v=eN0wFzBA4Sc);\n", 346 | "- [autoencoders](https://www.youtube.com/watch?v=FhmpO73ythg)." 347 | ], 348 | "metadata": { 349 | "id": "MlLNPn1S34tc" 350 | } 351 | } 352 | ] 353 | } -------------------------------------------------------------------------------- /Notebooks [eng]/empty.txt: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Notebooks [rus]/ML-алгоритмы с нуля/04) Наивный байесовский классификатор.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [] 7 | }, 8 | "kernelspec": { 9 | "name": "python3", 10 | "display_name": "Python 3" 11 | }, 12 | "language_info": { 13 | "name": "python" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "markdown", 19 | "source": [ 20 | "## **Наивный байесовский классификатор**\n", 21 | "Наивный байесовский классификатор (Naive Bayes classifier) — вероятностный классификатор на основе формулы Байеса со строгим (наивным) предположением о независимости признаков между собой при заданном классе, что сильно упрощает задачу классификации из-за оценки одномерных вероятностных плотностей вместо одной многомерной.\n", 22 | "\n", 23 | "В данном случае, одномерная вероятностная плотность — это оценка вероятности каждого признака отдельно при условии их независимости, а многомерная — оценка вероятности комбинации всех признаков, что вытекает из случая их зависимости. Именно по этой причине данный классификатор называется наивным, поскольку позволяет сильно упростить вычисления и повысить эффективность алгоритма. Однако такое предположение не всегда является верным на практике и в ряде случаев может привести к значительному ухудшению качества прогнозов.\n", 24 | "\n", 25 | "Сама же формула Байеса выглядит следующим образом:\n", 26 | "\n", 27 | "$$P(A|B) = \\frac{P(B|A) P(A)}{P(B)}$$\n", 28 | "\n", 29 | "где:\n", 30 | "- $P(A|B)$ — апостериорная вероятность события A при условии выполнения события B;\n", 31 | "\n", 32 | "- $P(B|A)$ — условная вероятность события B при условии выполнения события A;\n", 33 | "\n", 34 | "- $P(A)$ и $P(B)$ — априорные вероятности событий A и B соответственно.\n", 35 | "\n", 36 | "А в контексте машинного обучения формула Байеса приобретает следующий вид:\n", 37 | "\n", 38 | "$$P(y_k|X) = \\frac{P(y_k)P(X|y_k)}{P(X)}$$\n", 39 | "\n", 40 | "где:\n", 41 | "- $P(y_k|X)$ — апостериорная вероятность принадлежности образца к классу $y_k$ с учётом его признаков $X$;\n", 42 | "- $P(X|y_k)$ — правдоподобие, то есть вероятность признаков $X$ при заданном классе $y_k$;\n", 43 | "- $P(y_k)$ — априорная вероятность принадлежности случайно выбранного наблюдения к классу $y_k$;\n", 44 | "- $P(X)$ — априорная вероятность признаков $X$.\n", 45 | "\n", 46 | "Если объект описывается не одним, а несколькими признаками $X_1, X_2, ..., X_n$, то формула принимает вид:\n", 47 | "\n", 48 | "$$P(y_k|X_1, X_2, ..., X_n) = \\frac{P(y_k)\\prod_{i=1}^n P(X_i|y_k)}{P(X_1, X_2, ..., X_n)}$$\n", 49 | "\n", 50 | "На практике числитель данной формулы представляет наибольший интерес, поскольку знаменатель зависит только от признаков, а не от класса, и поэтому часто он опускается при сравнении вероятностей разных классов. В конечном счёте правило классификации будет пропорционально выбору класса с максимальной апостериорной вероятностью:\n", 51 | "\n", 52 | "$$y_k \\propto \\arg\\max_{y_k} P(y_k)\\prod_{i=1}^n P(X_i|y_k)$$\n", 53 | "\n", 54 | "Для оценки параметров модели, то есть вероятностей $P(y_k)$ и $P(X_i|y_k)$, обычно применяется метод максимального правдоподобия, который в данном случае основан на частотах встречаемости классов и признаков в обучающей выборке.\n", 55 | "\n", 56 | "### **Разновидности наивного Байеса**\n", 57 | "В библиотеке scikit-learn есть несколько реализаций наивного байесовского классификатора, отличающиеся предположениями о распределении признаков при заданном классе. К таковым относятся следующие:\n", 58 | "\n", 59 | "- **Гауссовский наивный байесовский классификатор (GaussianNB)** — вариант для работы с непрерывными признаками, которые имеют нормальное (гауссовское) распределение. Вероятность признака при заданном классе вычисляется по формуле: $$P(x_i|y) = \\frac{1}{\\sqrt{2\\pi\\sigma_y^2}}\\exp\\left(-\\frac{(x_i-\\mu_y)^2}{2\\sigma_y^2}\\right)$$ где $\\mu_y$ и $\\sigma_y$ — это среднее и стандартное отклонения признака в классе $y$. Эти параметры оцениваются с помощью метода максимального правдоподобия по обучающим данным.\n", 60 | "- **Мультиномиальный наивный байесовский классификатор (MultinomialNB)** — вариант для работы с дискретными признаками, которые имеют мультиномиальное распределение. Такие признаки часто встречаются в задачах классификации текстов, где они представляют собой количество вхождений в тексте. Вероятность признака при заданном классе вычисляется по формуле: $$P(x_i|y) = \\frac{N_{yi} + \\alpha}{N_y + \\alpha n}$$ где $N_{yi}$ — это количество раз, когда признак $i$ встречается в классе $y$; $N_y$ — общее количество всех признаков в классе $y$; $n$ — количество различных признаков; а $\\alpha$ — сглаживающий параметр, предотвращающий возникновение нулевых вероятностей.\n", 61 | "- **Комплементарный наивный байесовский классификатор (ComplementNB)** — улучшенный вариант *MultinomialNB*, подходящий для несбалансированных наборов данных. Вместо оценки вероятности признака при заданном классе, алгоритм оценивает нормированный вес признака $w_{ci}$ для класса $c$ как вероятность признака при дополнении класса, то есть при всех остальных классах. Таким образом, алгоритм учитывает не только частоту признаков в классе, но и их отсутствие в других классах, что делает его менее чувствительным к смещению выборки. Формула для вычисления вероятности признака при дополнении класса выглядит следующим образом: \\begin{align}\\begin{aligned}\\hat{\\theta}_{ci} = \\frac{\\alpha_i + \\sum_{j:y_j \\neq c} d_{ij}}\n", 62 | " {\\alpha + \\sum_{j:y_j \\neq c} \\sum_{k} d_{kj}}\\\\w_{ci} = \\log \\hat{\\theta}_{ci}\\\\w_{ci} = \\frac{w_{ci}}{\\sum_{j} |w_{cj}|}\\end{aligned}\\end{align}\n", 63 | "где $\\hat{\\theta}_{ci}$ — это оценка вероятности признака $i$ при дополнении класса $c$, которая вычисляется с помощью сглаживающего параметра $\\alpha_i$ и частоты признака $i$ во всех классах кроме $c$ (в данном случае $d_{ij}$ — это количество раз, когда признак $i$ встречается в классе $j$); $w_{ci}$ — это нормированный вес признака $i$ для класса $c$. Предсказанный класс $\\hat c$ для заданного вектора признаков $t$ будет выглядеть следующим образом: $$\\hat{c} = \\arg\\min_c \\sum_{i} t_i w_{ci}$$\n", 64 | "- **Бернуллиевский наивный байесовский классификатор (BernoulliNB)** — ещё один вариант для работы с дискретными признаками, но которые имеют бернуллиевское распределение. В данном случае признаки представляют собой бинарные индикаторы наличия или отсутствия определённых свойств в объекте. Например, в задаче классификации текстов это может быть наличие или отсутствие определённых слов в тексте. Вероятность признака при заданном классе вычисляется по формуле: $$P(x_i|y) = P(x_i = 1|y)x_i + (1-P(x_i = 1|y))(1-x_i)$$ где $P(x_i = 1|y)$ — это вероятность того, что признак $i$ принимает значение 1 (истина) при условии, что объект принадлежит классу $y$; $x_i$ — значение признака $i$ (0 или 1).\n", 65 | "- **Категориальный наивный байесовский классификатор (CategoricalNB)** — вариант для категориально распределенных данных, основанный на предположении, что каждый описываемый индексом признак имеет своё собственное категориальное распределение. Вероятность признака при заданном классе вычисляется по формуле: $$P(x_i = t \\mid y = c \\: ;\\, \\alpha) = \\frac{ N_{tic} + \\alpha}{N_{c} + \\alpha n_i}$$ где $N_{tic} = |\\{j \\in J \\mid x_{ij} = t, y_j = c\\}|$ — это количество раз, когда признак $x_i$ принимает значение $t$ в классе $c$; $N_{c} = |\\{ j \\in J\\mid y_j = c\\}|$ — общее количество всех признаков в классе $c$ в обучающих данных; $\\alpha$ — сглаживающий параметр; $n_i$ — количество доступных значений признака $i$.\n", 66 | "\n", 67 | "### **Принцип работы наивного байесовского классификатора c гауссовским распределением**\n", 68 | "Алгоритм строится следующим образом:\n", 69 | "- 1) изначально рассчитываются априорные вероятности классов;\n", 70 | "- 2) после рассчитываются средние и стандартные отклонения признаков по классам;\n", 71 | "- 3) на основе полученных отклонений признаков по классам рассчитывается вероятностная плотность тестовых признаков по распределению Гаусса;\n", 72 | "- 4) далее вычисляются апостериорные вероятности как произведение априорных вероятностей классов и вероятностных плотностей тестовых признаков;\n", 73 | "- 5) классы с максимальной апостериорной вероятностью будут итоговым прогнозом." 74 | ], 75 | "metadata": { 76 | "id": "m9EN0b8o3R35" 77 | } 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "source": [ 82 | "### **Наивный Байес в задачах фильтрации спама**\n", 83 | "В контексте фильтрации спама наивный байесовский классификатор основан на частоте появления слов в сообщениях для спама и не спама, и максимизации произведения их вероятностей. Наивность в данном случае будет заключаться в предположении о независимости слов в сообщении от порядка и контекста. Тогда формула Байеса приобретает следующий вид:\n", 84 | "\n", 85 | "$$P(C|M) \\propto P(C) \\prod_{i=1}^n P(w_i|C), \\ \\ w_i \\in M$$\n", 86 | "\n", 87 | "где:\n", 88 | "- $C$ — класс спам или не спам;\n", 89 | "- $M$ — сообщение;\n", 90 | "- $w_i$ — i-е слово в сообщении $M$;\n", 91 | "- $\\propto$ — знак пропорциональности.\n", 92 | "\n", 93 | "Для лучшего понимания рассмотрим следующий пример. Предположим, мы хотим классифицировать сообщение **\"Hi, you won a discount and you can get the prize this evening.\"** и у нас есть обучающая выборка, состоящая из следующих сообщений:\n", 94 | "\n", 95 | "| Message | Class |\n", 96 | "| --- | --- |\n", 97 | "| Hi, how are you? | Not spam |\n", 98 | "| Congratulations, you won a prize! | Spam |\n", 99 | "| Buy the product now and get a discount! | Spam |\n", 100 | "| Let's walk this evening. | Not spam |\n", 101 | "\n", 102 | "Первым делом необходимо рассчитать частоту появления всех уникальных слов и их общее количество в сообщениях для спама и не спама. Затем производится расчёт вероятностей встретить каждое слово в спам и не спам сообщениях на основе этих частот. Когда в сообщении есть слова, которые раньше не встречались в обучающей выборке, используется сглаживание. Существует много различных видов сглаживаний, но суть самого простого из них заключается в добавлении $1$ при подсчёте частот слов в сообщениях. Такой приём позволяет избежать проблему нулевой вероятности. Ниже приведена таблица с расчётом вероятностей для всех слов.\n", 103 | "\n", 104 | "| Word | Frequency in Not Spam | Frequency in Spam | Probability in Not Spam | Probability in Spam |\n", 105 | "| --- | --- | --- | --- | --- |\n", 106 | "| Hi | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 107 | "| how | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 108 | "| are | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 109 | "| you | 1 + $1$ = 2 | 1 + $1$ = 2 | 2 / 28 = 0.0714 | 2 / 33 = 0.06 |\n", 110 | "| Congratulations | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 111 | "| won | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 112 | "| a | 0 + $1$ = 1 | 2 + $1$ = 3 | 1 / 28 = 0.0357 | 3 / 33 = 0.09 |\n", 113 | "| prize | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 114 | "| Buy | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 115 | "| the | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 116 | "| product | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 117 | "| now | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 118 | "| and | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 119 | "| get | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 120 | "| discount | 0 + $1$ = 1 | 1 + $1$ = 2 | 1 / 28 = 0.0357 | 2 / 33 = 0.06 |\n", 121 | "| Let's | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 122 | "| walk | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 123 | "| this | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 124 | "| evening | 1 + $1$ = 2 | 0 + $1$ = 1 | 2 / 28 = 0.0714 | 1 / 33 = 0.03 |\n", 125 | "| can | 0 + $1$ = 1 | 0 + $1$ = 1 | 1 / 28 = 0.0357 | 1 / 33 = 0.03 |\n", 126 | "| **Total amount of words** | **28** | **33** |\n", 127 | "\n", 128 | " В конце рассчитываются вероятности сообщения быть спамом или не спамом, а итоговым прогнозом будет класс с максимальной вероятностью.\n", 129 | "\n", 130 | "$P(C|M) = P(C) \\cdot P('Hi'|C) \\cdot P('you'|C) \\cdot P('won'|C) \\cdot P('a'|C)\n", 131 | "\\cdot P('discount'|C) \\cdot P('and'|C) \\cdot P('you'|C) \\cdot P('can'|C) \\cdot P('get'|C)\n", 132 | "\\cdot P('the'|C) \\cdot P('prize'|C) \\cdot P('this'|C) \\cdot P('evening'|C)$\n", 133 | "\n", 134 | "Где:\n", 135 | "- $C \\in (Spam, \\ \\ Not \\ \\ Spam)$;\n", 136 | "- $P(Spam) = P(Not \\ \\ Spam) = \\frac{2}{4} = 0.5$\n", 137 | "\n" 138 | ], 139 | "metadata": { 140 | "id": "9Yn2qe4-J54P" 141 | } 142 | }, 143 | { 144 | "cell_type": "markdown", 145 | "source": [ 146 | "$P(Spam|M) = 0.5 \\cdot 0.03 \\cdot 0.06 \\cdot 0.06 \\cdot 0.09 \\cdot 0.06 \\cdot 0.06 \\cdot 0.06 \\cdot 0.03 \\cdot 0.06 \\cdot 0.06 \\cdot 0.06 \\cdot 0.03 \\cdot 0.03 \\approx 6.12 \\cdot 10^{-18}$\n", 147 | "\n", 148 | "$P(Not \\ \\ Spam|M) = 0.5 \\cdot 0.0714 \\cdot 0.0714 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0714 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0357 \\cdot 0.0714 \\cdot 0.0714 \\approx 2.45 \\cdot 10^{-18}$\n", 149 | "\n", 150 | "$P(Spam|M) > P(Not \\ \\ Spam|M) \\rightarrow$ **сообщение является спамом**\n", 151 | "\n", 152 | "Стоит добавить, что на практике для удобства расчётов зачастую используется логарифм вероятности вместо самой вероятности." 153 | ], 154 | "metadata": { 155 | "id": "I--z9SYmUlBL" 156 | } 157 | }, 158 | { 159 | "cell_type": "markdown", 160 | "source": [ 161 | "### **Реализация на Python с нуля**" 162 | ], 163 | "metadata": { 164 | "id": "birEwBNZ3Nqw" 165 | } 166 | }, 167 | { 168 | "cell_type": "code", 169 | "source": [ 170 | "import numpy as np\n", 171 | "import pandas as pd\n", 172 | "import matplotlib.pyplot as plt\n", 173 | "from sklearn.datasets import load_iris\n", 174 | "from sklearn.naive_bayes import GaussianNB\n", 175 | "from sklearn.metrics import accuracy_score\n", 176 | "from sklearn.model_selection import train_test_split\n", 177 | "from mlxtend.plotting import plot_decision_regions" 178 | ], 179 | "metadata": { 180 | "id": "CPkMdMHLCDbQ" 181 | }, 182 | "execution_count": 7, 183 | "outputs": [] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "source": [ 188 | "class GaussianNaiveBayes:\n", 189 | " def fit(self, X, y):\n", 190 | " classes, cls_counts = np.unique(y, return_counts=True)\n", 191 | " n_classes = len(classes)\n", 192 | " self.priors = cls_counts / len(y)\n", 193 | "\n", 194 | " # calculate the mean and standard deviations of features by classes\n", 195 | " self.X_cls_mean = np.array([np.mean(X[y == c], axis=0) for c in range(n_classes)])\n", 196 | " self.X_stds = np.array([np.std(X[y == c], axis=0) for c in range(n_classes)])\n", 197 | "\n", 198 | " # calculate the probability density of the feature according to the Gaussian distribution\n", 199 | " def pdf(self, x, mean, std):\n", 200 | " return (1 / (np.sqrt(2 * np.pi) * std)) * np.exp(-0.5 * ((x - mean) / std) ** 2)\n", 201 | "\n", 202 | " def predict(self, X):\n", 203 | " pdfs = np.array([self.pdf(x, self.X_cls_mean, self.X_stds) for x in X])\n", 204 | " posteriors = self.priors * np.prod(pdfs, axis=2) # shorten Bayes formula\n", 205 | "\n", 206 | " return np.argmax(posteriors, axis=1)" 207 | ], 208 | "metadata": { 209 | "id": "JkWEEsgTt67e" 210 | }, 211 | "execution_count": 8, 212 | "outputs": [] 213 | }, 214 | { 215 | "cell_type": "code", 216 | "source": [ 217 | "def decision_boundary_plot(X, y, X_train, y_train, clf, feature_indexes, title=None):\n", 218 | " feature1_name, feature2_name = X.columns[feature_indexes]\n", 219 | " X_feature_columns = X.values[:, feature_indexes]\n", 220 | " X_train_feature_columns = X_train[:, feature_indexes]\n", 221 | " clf.fit(X_train_feature_columns, y_train)\n", 222 | "\n", 223 | " plot_decision_regions(X=X_feature_columns, y=y.values, clf=clf)\n", 224 | " plt.xlabel(feature1_name)\n", 225 | " plt.ylabel(feature2_name)\n", 226 | " plt.title(title)" 227 | ], 228 | "metadata": { 229 | "id": "Yx0jFHbCEvLz" 230 | }, 231 | "execution_count": 9, 232 | "outputs": [] 233 | }, 234 | { 235 | "cell_type": "markdown", 236 | "source": [ 237 | "### **Загрузка датасета**" 238 | ], 239 | "metadata": { 240 | "id": "4TBM0I-R_4XI" 241 | } 242 | }, 243 | { 244 | "cell_type": "code", 245 | "source": [ 246 | "X1, y1 = load_iris(return_X_y=True, as_frame=True)\n", 247 | "X1_train, X1_test, y1_train, y1_test = train_test_split(X1.values, y1.values, random_state=0)\n", 248 | "print(X1, y1, sep='\\n')" 249 | ], 250 | "metadata": { 251 | "colab": { 252 | "base_uri": "https://localhost:8080/" 253 | }, 254 | "id": "fRTiyfWHBS8r", 255 | "outputId": "e36bc59c-7936-4351-a39a-63a1b6533a2d" 256 | }, 257 | "execution_count": 10, 258 | "outputs": [ 259 | { 260 | "output_type": "stream", 261 | "name": "stdout", 262 | "text": [ 263 | " sepal length (cm) sepal width (cm) petal length (cm) petal width (cm)\n", 264 | "0 5.1 3.5 1.4 0.2\n", 265 | "1 4.9 3.0 1.4 0.2\n", 266 | "2 4.7 3.2 1.3 0.2\n", 267 | "3 4.6 3.1 1.5 0.2\n", 268 | "4 5.0 3.6 1.4 0.2\n", 269 | ".. ... ... ... ...\n", 270 | "145 6.7 3.0 5.2 2.3\n", 271 | "146 6.3 2.5 5.0 1.9\n", 272 | "147 6.5 3.0 5.2 2.0\n", 273 | "148 6.2 3.4 5.4 2.3\n", 274 | "149 5.9 3.0 5.1 1.8\n", 275 | "\n", 276 | "[150 rows x 4 columns]\n", 277 | "0 0\n", 278 | "1 0\n", 279 | "2 0\n", 280 | "3 0\n", 281 | "4 0\n", 282 | " ..\n", 283 | "145 2\n", 284 | "146 2\n", 285 | "147 2\n", 286 | "148 2\n", 287 | "149 2\n", 288 | "Name: target, Length: 150, dtype: int64\n" 289 | ] 290 | } 291 | ] 292 | }, 293 | { 294 | "cell_type": "markdown", 295 | "source": [ 296 | "### **Обучение моделей и оценка полученных результатов**\n", 297 | "Не смотря на свою простоту, в данном случае алгоритм показал отличный результат, классифицировав правильно абсолютно все образцы, что возможно благодаря построению гибкой решающей границы с высокой обобщающей способностью. Из этого можно сделать интересный вывод, что в некоторых ситуациях более простые модели могут работать гораздо лучше, чем сложные, что можно будет заметить в дальнейшем на примере других алгоритмов." 298 | ], 299 | "metadata": { 300 | "id": "rG4hElWeCRj4" 301 | } 302 | }, 303 | { 304 | "cell_type": "markdown", 305 | "source": [ 306 | "**Naive Bayes**" 307 | ], 308 | "metadata": { 309 | "id": "rnN1p3zuCm1D" 310 | } 311 | }, 312 | { 313 | "cell_type": "code", 314 | "source": [ 315 | "nb_clf = GaussianNaiveBayes()\n", 316 | "nb_clf.fit(X1_train, y1_train)\n", 317 | "nb_clf_pred_res = nb_clf.predict(X1_test)\n", 318 | "nb_clf_accuracy = accuracy_score(y1_test, nb_clf_pred_res)\n", 319 | "\n", 320 | "print(f'Naive Bayes classifier accucacy: {nb_clf_accuracy}')\n", 321 | "print(nb_clf_pred_res)" 322 | ], 323 | "metadata": { 324 | "colab": { 325 | "base_uri": "https://localhost:8080/" 326 | }, 327 | "id": "U60Olss-Cfm3", 328 | "outputId": "43f8e955-9e8e-4876-b82e-ba5941028042" 329 | }, 330 | "execution_count": 11, 331 | "outputs": [ 332 | { 333 | "output_type": "stream", 334 | "name": "stdout", 335 | "text": [ 336 | "Naive Bayes classifier accucacy: 1.0\n", 337 | "[2 1 0 2 0 2 0 1 1 1 2 1 1 1 1 0 1 1 0 0 2 1 0 0 2 0 0 1 1 0 2 1 0 2 2 1 0\n", 338 | " 1]\n" 339 | ] 340 | } 341 | ] 342 | }, 343 | { 344 | "cell_type": "markdown", 345 | "source": [ 346 | "**Naive Bayes (scikit-learn)**" 347 | ], 348 | "metadata": { 349 | "id": "1SPatZBoELDl" 350 | } 351 | }, 352 | { 353 | "cell_type": "code", 354 | "source": [ 355 | "sk_nb_clf = GaussianNB()\n", 356 | "sk_nb_clf.fit(X1_train, y1_train)\n", 357 | "sk_nb_clf_pred_res = sk_nb_clf.predict(X1_test)\n", 358 | "sk_nb_clf_accuracy = accuracy_score(y1_test, sk_nb_clf_pred_res)\n", 359 | "\n", 360 | "print(f'sk Naive Bayes classifier accucacy: {sk_nb_clf_accuracy}')\n", 361 | "print(sk_nb_clf_pred_res)\n", 362 | "\n", 363 | "feature_indexes = [2, 3]\n", 364 | "title1 = 'GaussianNB surface'\n", 365 | "decision_boundary_plot(X1, y1, X1_train, y1_train, sk_nb_clf, feature_indexes, title1)" 366 | ], 367 | "metadata": { 368 | "colab": { 369 | "base_uri": "https://localhost:8080/", 370 | "height": 524 371 | }, 372 | "id": "0bdNaO_FoYOW", 373 | "outputId": "bcc84fb8-b6f4-4232-fadf-748c2505035c" 374 | }, 375 | "execution_count": 12, 376 | "outputs": [ 377 | { 378 | "output_type": "stream", 379 | "name": "stdout", 380 | "text": [ 381 | "sk Naive Bayes classifier accucacy: 1.0\n", 382 | "[2 1 0 2 0 2 0 1 1 1 2 1 1 1 1 0 1 1 0 0 2 1 0 0 2 0 0 1 1 0 2 1 0 2 2 1 0\n", 383 | " 1]\n" 384 | ] 385 | }, 386 | { 387 | "output_type": "display_data", 388 | "data": { 389 | "text/plain": [ 390 | "
" 391 | ], 392 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHHCAYAAABTMjf2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACRwUlEQVR4nOzdd1wU1xYH8N926tK7UgQFFRVFQbD3XhOjxlgT0zQxMYmGFI0mkRhjos/EFmOJJSYaS2JHBXuMYgUrCGKhSO/bZt4fhJUVFljcZXbhfD+ffXnM3Jl77op4uHvvGR7LsiwIIYQQQhohPtcBEEIIIYRwhRIhQgghhDRalAgRQgghpNGiRIgQQgghjRYlQoQQQghptCgRIoQQQkijRYkQIYQQQhotSoQIIYQQ0mhRIkQIIYSQRosSIUIIpzZu3Agej4fk5GSuQzFZSqUSc+bMQdOmTcHn8zFy5EiuQyLEZFAiRIgJS0pKwsyZM9GiRQtYWFjAwsICrVq1wowZM3Dt2jWuwzMaycnJ4PF44PF4+PPPPyud/+KLL8Dj8ZCZmak+NmXKFPU1PB4PQqEQTZs2xbhx43Djxo36DL9G69evx5IlS/Diiy9i06ZNeP/997kOiRCTIeQ6AEJI3ezbtw9jx46FUCjEhAkT0K5dO/D5fNy6dQu7du3CqlWrkJSUBC8vL65DrdbEiRMxbtw4SCSSeulv4cKFGD16NHg8Xo1tJRIJ1q1bB6Bs1iUxMRGrV6/GoUOHcOPGDbi7uxs63Fo5fvw4PDw88MMPP3AdCiEmhxIhQkxQYmIixo0bBy8vLxw7dgxubm4a5xcvXoyVK1eCzzf+SV+BQACBQFAvfQUFBeHKlSvYvXs3Ro8eXWN7oVCIV155ReNY586dMXToUOzfvx/Tp083VKg1YlkWpaWlMDc3R0ZGBmxtbTmLhRBTZvw/JQkhlXz77bcoKirChg0bKiVBQNk/4O+++y6aNm2qPnbt2jVMmTIFzZo1g5mZGVxdXTFt2jRkZWVpXDtlyhR4e3tXumf5x0cVRUVFoWvXrrC1tYWVlRX8/f3xySefaLRZsWIFWrduDQsLC9jZ2aFjx47Ytm2b+nxVa4T27t2LIUOGwN3dHRKJBL6+vvjyyy+hUqk07t2zZ08EBgbixo0b6NWrFywsLODh4YFvv/22yvdt3LhxaNGiBRYuXAiWZatsUxNXV1cAZe9xTbZv347g4GBYW1tDKpWiTZs2WL58ufp8Ve8pUPV74u3tjaFDh+Lw4cPo2LEjzM3NsWbNGvB4PERHRyM+Pl79MV5MTAwA4LvvvkN4eDgcHBxgbm6O4OBg7Ny5s8pYt2zZgpCQEPWfU/fu3XHkyBGNNgcPHkS3bt1gaWkJa2trDBkyBPHx8TW+D4QYM0qECDFB+/btg5+fH0JDQ2t9TVRUFO7du4epU6dixYoVGDduHLZv347BgwfXKSmIj4/H0KFDIZPJsHDhQixduhTDhw/HmTNn1G1+/vlnvPvuu2jVqhWWLVuGBQsWICgoCOfPn6/23hs3boSVlRVmz56N5cuXIzg4GPPmzcPHH39cqW1OTg4GDhyIdu3aYenSpQgICMDcuXNx8ODBSm0FAgE+++wzXL16Fbt3767VODMzM5GZmYn09HScO3cO77//PhwcHDB06NBqr4uKisL48eNhZ2eHxYsX45tvvkHPnj013h9d3b59G+PHj0e/fv2wfPlyNG/eHJs3b0ZAQACaNGmCzZs3Y/PmzWjZsiUAYPny5Wjfvj0WLlyIRYsWQSgUYsyYMdi/f7/GfRcsWICJEydCJBJh4cKFWLBgAZo2bYrjx4+r22zevBlDhgyBlZUVFi9ejM8//xw3btxA165daaE7MW0sIcSk5OXlsQDYkSNHVjqXk5PDPnnyRP0qLi5Wn6v4/8v99ttvLAD25MmT6mOTJ09mvby8KrWdP38+W/FHxg8//MACYJ88eaI11hEjRrCtW7eudjwbNmxgAbBJSUnVxvrGG2+wFhYWbGlpqfpYjx49WADsr7/+qj4mk8lYV1dX9oUXXlAfS0pKYgGwS5YsYZVKJdu8eXO2Xbt2LMMwGmOrOJbJkyezACq9PDw82NjY2GrHxLIsO2vWLFYqlbJKpVJrm2ff0+reEy8vLxYAe+jQoUrte/ToUeX7/Oz7KJfL2cDAQLZ3797qY3fv3mX5fD47atQoVqVSabQvf38KCgpYW1tbdvr06Rrn09LSWBsbm0rHCTElNCNEiInJz88HAFhZWVU617NnTzg5OalfP/30k/qcubm5+v+XlpYiMzMTnTt3BgBcunRJ5zjK16Ts3bsXDMNobfPw4UNcuHBBp3tXjLWgoACZmZno1q0biouLcevWLY22VlZWGut4xGIxQkJCcO/evSrvXXFWaM+ePdXGYWZmhqioKERFReHw4cNYs2YNrKysMHjwYNy5c6faa21tbVFUVISoqKgaRlt7Pj4+GDBgQK3bV3wfc3JykJeXh27dumn8ee/ZswcMw2DevHmV1pSVf2wXFRWF3NxcjB8/Xj1DlpmZCYFAgNDQUERHRz/nyAjhDiVChJgYa2trAEBhYWGlc2vWrEFUVBS2bNlS6Vx2djZmzZoFFxcXmJubw8nJCT4+PgCAvLw8neMYO3YsunTpgtdeew0uLi4YN24c/vjjD42kaO7cubCyskJISAiaN2+OGTNm1Oqjofj4eIwaNQo2NjaQSqVwcnJSJzvPxtqkSZNK62zs7OyQk5Oj9f4TJkyAn59fjWuFBAIB+vbti759+6J///54/fXXcfToUeTl5SEiIqLaMbz99tto0aIFBg0ahCZNmmDatGk4dOhQTUOvVvmfV23t27cPnTt3hpmZGezt7eHk5IRVq1ZpvIeJiYng8/lo1aqV1vvcvXsXANC7d2+NRNvJyQlHjhxBRkZG3QZEiBGgXWOEmBgbGxu4ubkhLi6u0rnyNUNVrdl46aWXcPbsWXz00UcICgqClZUVGIbBwIEDNZIXbdvKn12obG5ujpMnTyI6Ohr79+/HoUOH8Pvvv6N37944cuQIBAIBWrZsidu3b2Pfvn04dOgQ/vzzT6xcuRLz5s3DggULquwnNzcXPXr0gFQqxcKFC+Hr6wszMzNcunQJc+fOrTT7pG3HWU0JzmeffYYpU6Zg7969WttVpUmTJvD398fJkyerbefs7IwrV67g8OHDOHjwIA4ePIgNGzZg0qRJ2LRpE4Dav9flKs7w1OTUqVMYPnw4unfvjpUrV8LNzQ0ikQgbNmzQWKxeG+Xv+ebNm9WLxSuqzcJxQowVffcSYoKGDBmCdevW4d9//0VISEiN7XNycnDs2DEsWLAA8+bNUx8v/02/Ijs7O+Tm5lY6fv/+/UrH+Hw++vTpgz59+uD777/HokWL8OmnnyI6Ohp9+/YFAFhaWmLs2LEYO3Ys5HI5Ro8eja+//hoREREwMzOrdM+YmBhkZWVh165d6N69u/p4UlJSjePUxSuvvIKvvvoKCxYswPDhw3W6VqlUVjkj9yyxWIxhw4Zh2LBhYBgGb7/9NtasWYPPP/8cfn5+sLOzA1CW/FXc/l7Ve62rP//8E2ZmZjh8+LBGjaYNGzZotPP19QXDMLhx4waCgoKqvJevry+AsuSu/M+VkIaCPhojxATNmTMHFhYWmDZtGtLT0yudf3Y2pHzW5Nnjy5Ytq3Str68v8vLyNCpTp6amVtpllZ2dXena8n9IZTIZAFTami8Wi9GqVSuwLAuFQlHl2KqKVS6XY+XKlVW2r6vyWaErV67gr7/+qvV1d+7cwe3bt9GuXbtq2z07dj6fj7Zt2wJ4+v6UJxgVZ5eKiorUM0bPQyAQgMfjacwuJScnV1oXNXLkSPD5fCxcuLDSbFv5n8GAAQMglUqxaNGiKv/cnjx58tzxEsIVmhEixAQ1b94c27Ztw/jx4+Hv76+uLM2yLJKSkrBt2zbw+Xw0adIEACCVStG9e3d8++23UCgU8PDwwJEjR6qcZRk3bhzmzp2LUaNG4d1330VxcTFWrVqFFi1aaCyyXbhwIU6ePIkhQ4bAy8sLGRkZWLlyJZo0aYKuXbsCAPr37w9XV1d06dIFLi4uuHnzJn788UcMGTJEvdbpWeHh4bCzs8PkyZPx7rvvgsfjYfPmzXWu+1OdCRMm4Msvv8SVK1eqPK9UKtXrrRiGQXJyMlavXg2GYTB//vxq7/3aa68hOzsbvXv3RpMmTXD//n2sWLECQUFB6u3t/fv3h6enJ1599VV89NFHEAgEWL9+PZycnJCSkvJcYxsyZAi+//57DBw4EC+//DIyMjLw008/wc/PTyPJ9fPzw6effoovv/wS3bp1w+jRoyGRSHDhwgW4u7sjMjISUqkUq1atwsSJE9GhQweMGzdOHeP+/fvRpUsX/Pjjj88VLyGc4Wy/GiHkuSUkJLBvvfUW6+fnx5qZmbHm5uZsQEAA++abb7JXrlzRaPvw4UN21KhRrK2tLWtjY8OOGTOGffz4MQuAnT9/vkbbI0eOsIGBgaxYLGb9/f3ZLVu2VNrqfezYMXbEiBGsu7s7KxaLWXd3d3b8+PHsnTt31G3WrFnDdu/enXVwcGAlEgnr6+vLfvTRR2xeXp66TVVbxc+cOcN27tyZNTc3Z93d3dk5c+awhw8fZgGw0dHR6nbato0/WwKg4vb5Z5X3j1psn5dKpWyfPn3Yo0ePav0zKbdz5062f//+rLOzMysWi1lPT0/2jTfeYFNTUzXaxcbGsqGhoeo233//vdbt80OGDKmyL23vwy+//MI2b96clUgkbEBAALthwwatW/bXr1/Ptm/fnpVIJKydnR3bo0cPNioqSqNNdHQ0O2DAANbGxoY1MzNjfX192SlTprAXL16s8f0gxFjxWNYAv2YRQgghhJgAWiNECCGEkEaLEiFCCCGENFqUCBFCCCGk0TKZRGjVqlVo27YtpFIppFIpwsLCqnyoYrnypzdXfFVVs4QQQgghjZfJbJ9v0qQJvvnmGzRv3hwsy2LTpk0YMWIELl++jNatW1d5jVQqxe3bt9Vfa6viSgghhJDGyWQSoWHDhml8/fXXX2PVqlX4559/tCZCPB6vynLwhBBCCCGACX00VpFKpcL27dtRVFSEsLAwre0KCwvh5eWFpk2bYsSIEYiPj6/x3jKZDPn5+Rqv8iqwhBBCCGlYTGZGCACuX7+OsLAwlJaWwsrKCrt379b6xGR/f3+sX78ebdu2RV5eHr777juEh4cjPj5eXW23KpGRkZUeBjlu5hiMf+clvY6FEEIIIYZjb+aArp69amxnUgUV5XI5UlJSkJeXh507d2LdunU4ceKE1mSoIoVCgZYtW2L8+PH48ssvtbaTyWSVZoD2Jv4BkVj03PETQgghpH44mDuiX7MhNbYzqRkhsVgMPz8/AEBwcDAuXLiA5cuXY82aNTVeKxKJ0L59eyQkJFTbTiKRaDypGQAlQYQQQkgDZZJrhMoxDFPr9TsqlQrXr1+Hm5ubgaMihBBCiKkwmRmhiIgIDBo0CJ6enigoKMC2bdsQExODw4cPAwAmTZoEDw8PREZGAih7Mnbnzp3h5+eH3NxcLFmyBPfv38drr73G5TAIIYQQYkRMJhHKyMjApEmTkJqaChsbG7Rt2xaHDx9Gv379AAApKSng859OcOXk5GD69OlIS0uDnZ0dgoODcfbs2VqtJyKEEEJI42BSi6W5sj1+E9chEEIIIbpjAb5pr4LRigEDVFMnuUEuliaEEEJILbCAmDWDBd8SAp4AvOoyBhPEgoWKVaGYKYKcV1ptQlQTSoQIIYSQBkbMmsFWZAeRWASWxwJoaB/+8MBjeRDLxchV5JQlQ3VEiRAhhBDSkLCABd+yLAkSMFxHYyAsWLAQiUWwUFlCztZ9VqhhfnBICCGENFJ88CHgCf6bCWrYWB4LAU/wXOugKBEihBBCGpiyNUENPxEC2Ode/0SJECGEEEIaLUqECCGEENJoUSJECCGEEKOx+9e9GNtlAvq1GIQ3R8zEzSu3DNof7RojhBBCiFr6o3SUlmh/jqeZuQQuHi4G6fv439H46avVmP3VLLRq3xI71v+JDyd9jC3HN8DO0c4gfVIiRAghhBAAZUnQnNe+gEyhvY1EBHy77guDJEN/rPsTQ8cNxuCXBgIAPvj6Pfxz/DwO/HEIE94er/f+AEqECCGEEPKf0hIZZArAoccrMHeonOiUZKUj68SWameM6kohV+BO3B2NhIfP5yO4SwfEX7qh9/7KUSJECCGEEA3mDi6wcG5ar33m5eRBpWIqfQRm52SHlMQHBuuXFksTQgghpNGiRIgQQgghnLOxs4FAwEdOZo7G8ZwnObB3MsxCaYASIUIIIYQYAZFYhBaBLRB79pL6GMMwuHT2Mlp3aGWwfmmNECGEEEKMwkuvvYDID75FQBt/BAT5Y+cvu1BSXIpBYwYarE9KhAghhBCioSQrXafj+tJ7WC/kZudh/Q8bkf0kB34tfbFkU6RBPxqjRIgQQgghAMqKJUpEQNaJLVrbSERl7Qxl9OSRGD15pMHu/yxKhAghhBACAHDxcMG3677grLI0FygRIoQQQohaQ0pyaoN2jRFCCCGk0aJEiBBCCCGNFiVChBBCCGm0KBEihBBCSKNFiRAhhBBCGi1KhAghhBDSaFEiRAghhJBGixIhQgghhDRalAgRQgghxChcPX8NH7/6GUaHjEUP7744dfiMwfukRIgQQgghVWJZFsl374Nl2Xrpr6S4FH4tm+G9he/US38APWKDEEIIIVqcj/kXG79diylzXkfnXqEG769zrxB07hVi8H4qMpkZoVWrVqFt27aQSqWQSqUICwvDwYMHq71mx44dCAgIgJmZGdq0aYMDBw7UU7SEEEKIaVOpVDiwdS8kRZk4sHUvVCoV1yEZhMkkQk2aNME333yD2NhYXLx4Eb1798aIESMQHx9fZfuzZ89i/PjxePXVV3H58mWMHDkSI0eORFxcXD1HTgghhJieCycvIis5CR/1c0ZWchIunLzIdUgGYTKJ0LBhwzB48GA0b94cLVq0wNdffw0rKyv8888/VbZfvnw5Bg4ciI8++ggtW7bEl19+iQ4dOuDHH3+s58gJIYQQ01I+G9SjKR/D2tmie1N+g50VMplEqCKVSoXt27ejqKgIYWFhVbY5d+4c+vbtq3FswIABOHfuXLX3lslkyM/P13gp5Aq9xU4IIYQYu/LZoKnh9gCAqWH2DXZWyKQSoevXr8PKygoSiQRvvvkmdu/ejVatWlXZNi0tDS4uLhrHXFxckJaWVm0fkZGRsLGx0Xjt/Xmf3sZACCGEGLOKs0EtXMwAAP6uZg12VsikEiF/f39cuXIF58+fx1tvvYXJkyfjxo0beu0jIiICeXl5Gq8R04fqtQ9CCCHEWD07G1SuPmaFiotKcDc+AXfjEwAAqQ9ScTc+AemP0g3Wp0ltnxeLxfDz8wMABAcH48KFC1i+fDnWrFlTqa2rqyvS0zXfuPT0dLi6ulbbh0QigUQi0TgmEoueM3JCCCHE+JXPBnXx4MHbQQy5klGf83EUI9yDhwNb96JT944QCAR67//2tdt4b/yH6q9/+mo1AGDgC/0RsXSO3vsDTCwRehbDMJDJZFWeCwsLw7Fjx/Dee++pj0VFRWldU0QIIYQ0dgk3EvHk4WNky1Xo9ePDKtuoxI+RcCMR/m1a6L3/9mFBOJF8VO/3rY7JJEIREREYNGgQPD09UVBQgG3btiEmJgaHDx8GAEyaNAkeHh6IjIwEAMyaNQs9evTA0qVLMWTIEGzfvh0XL17E2rVruRwGIYQQYrSa+fvg1c/egVKh1NpGKBKimb9PPUZlWCaTCGVkZGDSpElITU2FjY0N2rZti8OHD6Nfv34AgJSUFPD5T5c8hYeHY9u2bfjss8/wySefoHnz5tizZw8CAwO5GgIhhBBi1ERiETp2DeY6jHrFY+vrASImbHv8Jq5DIIQQQmqFz/Jhz3eC2FxkYlui6oAB5CUKZDNPwPAYjVMO5o7o12xIjbdo6G8RIYQQQohWlAgRQgghpNGiRIgQQgghjRYlQoQQQghptCgRIoQQQkijRYkQIYQQQhotSoQIIYQQ0miZTEFFQgghhDRcW37ahpOHTyMl8QEkZhIEdmiFNz6eDk/fpgbtlxIhQgghhFTCMAzuxicgLzsPNvY2aN7aT+MJDvp29fw1jJo4AgHt/KFSqvDzkl/w4aS52BT1C8wtzA3WLyVChBBCCNFw6exl/Pbz73j0+BFUrAoCngAe7h4YP30sOoS3N0ifS379RuPriO/mYETwi7hz/S7ahbY1SJ8ArREihBBCSAWXzl7Gsq/+h1zLTLR+3Rfh84PQ+nVf5FlmYtlX/8Ols5frJY7CgiIAgLWttUH7oUSIEEIIIQDKPg777effIWkmQLtJAbD1soZQIoCtlzXaTgqAWTMBfvv5dzAMU/PNnjOOHxeuRJuOrQ3+pHtKhAghhBACALgbn4BHjx/Bu1cT8Pg8jXM8Pg9ePT3w6PEj3I1PMGgcP3z+PyTdTsa8FZ8ZtB+A1ggRQggh5D952XlQsSpYuVpUed7KzRIqVoW87DyDxbBs3gqcO34eK/74Hs5uTgbrpxzNCBFCCCEEAGBjbwMBT4DCtOIqzxemFkHAE8DG3kbvfbMsi2XzVuDU4dNYtm0J3Jq66b2PqlAiRAghhBAAQPPWfvBw98D96IdgGVbjHMuwuB/zCB7uHmje2k/vff/w+f8QtfsoPl/+CcwtLZCVkY2sjGzISmV676siSoQIIYQQAgDg8/kYP30sSu+pcO3XW8hNzodSpkJucj6u/XoLpfdUGD99rEHqCe3d8jcKC4owa9wHGB3ykvp1/O8YvfdVEY9lWbbmZo3b9vhNXIdACCGE1Aqf5cOe7wSxuajO0x1c1BGqEwaQlyiQzTwBw9PcyeZg7oh+zYbUeAtaLE0IIYQQDR3C2yOoc7t6rSzNFUqECCGEEFIJn8+Hf5sWXIdhcA0vtSOEEEIIqSVKhAghhBDSaFEiRAghhDQwLFgAvBrbmT7ef2OtO0qECCGEkAaEAQMVqwKPbfiJEI/lQcWqwKDuzz6jxdKEEEJIQ8IDipkiiOViiMQisDwWeM5ZE+PDA4/lQSFXoJgpeq5pHUqECCGEkAZGzitFriIHFipLCHgC8BrYx2QsWKhYFYqZIsh5pc91L0qECCGEkIaGV5YMydlS8NmGuQqGAaOXBT6UCBFCCCENFQ/PtX6mMWiYaSIhhBBCSC1QIkQIIYSQRos+GiOEEGIyGIZB8s0UFOQWwNrWGt4tPev0/Kvq7qOvPohpMJlEKDIyErt27cKtW7dgbm6O8PBwLF68GP7+/lqv2bhxI6ZOnapxTCKRoLT0+VaYE0IIqX9x529g36YDSE/LAMMy4PP4cHF1xtDJgxEY2kov9wGglz6I6TCZROjEiROYMWMGOnXqBKVSiU8++QT9+/fHjRs3YGlpqfU6qVSK27dvq7/m8RrWFkJCCGkM4s7fwMYlm2HuK0Kr4T6wdDFHUXoJUqJTsXHJZkz5aGKtEpXq7rPmi3XgCXiwaW35XH0Q02IyidChQ4c0vt64cSOcnZ0RGxuL7t27a72Ox+PB1dXV0OERQggxEIZhsG/TAZj7ihA40Q88ftkvtDaeVgic6Ie4zQnYt+kAWnUKqPYjrOru03qCL45e+gdSb4vn6oOYHpP908zLywMA2NvbV9uusLAQXl5eaNq0KUaMGIH4+Phq28tkMuTn52u8FHKF3uImhBCim+SbKUhPy4BnLzd1glKOx+fBs6cr0tMykHwzpc73yUspBE8I2HWwhlym+TNflz6I6THJRIhhGLz33nvo0qULAgMDtbbz9/fH+vXrsXfvXmzZsgUMwyA8PBwPHz7Uek1kZCRsbGw0Xnt/3meIYRBCCKmFgtwCMCwDSxfzKs9bulqAYRkU5BbU+T7yfAV4Ah4k9iIwKlWd+yCmxyQToRkzZiAuLg7bt2+vtl1YWBgmTZqEoKAg9OjRA7t27YKTkxPWrFmj9ZqIiAjk5eVpvEZMH6rvIRBCCKkla1tr8Hl8FKWXVHm+KK0YfB4f1rbWdb6PWCoCq2Ihy1aALxDUuQ9iekwuEZo5cyb27duH6OhoNGnSRKdrRSIR2rdvj4SEBK1tJBIJpFKpxkskFj1v2IQQQurIu6UnXFydkRKdCpbRfHgoy7BIiUmDi6szvFt61vk+Np5WYJVAzqUCiCWaP/N16YOYHpNJhFiWxcyZM7F7924cP34cPj4+Ot9DpVLh+vXrcHNzM0CEhBBCDIHP52Po5MEoSVQgbnMC8u4XQClTIe9+AeI2J6AkUYGhkwfXuIi5uvvEb02EudAcygd4rj6I6eGxLMvW3Ix7b7/9NrZt24a9e/dq1A6ysbGBuXnZ572TJk2Ch4cHIiMjAQALFy5E586d4efnh9zcXCxZsgR79uxBbGwsWrWq/RbI7fGb9DsYQggxclwWFVQqlTh38F9kpmbB0c0BYYNCIBQKqY4Q0YmDuSP6NRtSYzuT2T6/atUqAEDPnj01jm/YsAFTpkwBAKSkpGj8Rc3JycH06dORlpYGOzs7BAcH4+zZszolQYQQ0tjoK+Goi30bD+LvTQcgU8rAEwCsCtjy/W8YNnkwhk4ZhFadAp47QQsMbVXtffTRBzEdJjMjxCWaESKENBYVCw569nLTKCpYkqgwaFHBfRsPYufPu2EbZAWPng6wcDdH8eMSPIrOQu7VQrw4fRSGThlkkL5Jw1PbGSFKcQkhhACoXHDQxtMKQolAXVTQ3FeEfZsOgGEYvfetVCrx96YDsA2yQsCUppD6WkJozofU1xIBU5vCtp0V/t50AEqlUu99k8aNEiFCCCEA9Fe4sC7OHfwXMqUMHj0dwBM807eAB49eDpApZTh38F+9900aN0qECCGEANBf4cK6yEzNAk8AWLhX3beFuzl4grJ2hOgTJUKEEEIA6K9wYV04ujmAVQHFj6vuu/hxCVhVWTtC9IkSIUIIIQD0V7iwLsIGhUAilOBRdBZY1TN9q1g8is6CRChB2KAQvfdNGjeT2T5PCCHEsMoLDm5cshnXf70Lu+ZSCMQCqOQq5NzNR+k9JV76YJDWreXaag/VpiaRUCjEsMmDsfPn3bi14QE8elW9a4zP5+NefHKtt7bXpR4SlzWU9KmhjMPQaPt8LdD2eUJIY1JVLR+JUIJOvYKRmZapUyHCwJDWiPs3vtY1ibT1PWzyYHi39NKpvlFd6iFxWUNJnxrKOJ5HbbfPUyJUC5QIEUIai6d1hIRwDXOCuYMIJVkK3Pv7IbJu5cIt1BHN+jfVqC+UF18EVsXCtq2VRu2hWzuStF5TXU2iqipL34q9o1N9o7rUQ+KyhpI+NZRxPC9KhPToo5/e4zoETrn7uMHNy5XrMAghBsYwDL6d+T2KrPMRONFPvYWeZVic+/YqYM3CZ4gb3Lyf/jxgGRbnf7qK/IRi9F3aGXwhv1bXxG1OgGWBFHN+nF2rj6qqikvbvXRtX5c+jFVDGYc+NLhHbHDJ90EG1yFw6rdtx+Hs71mprkhtdBjUCd4BXgaIihCib+V1hFoN99H4+56bXICSnFL4DXOHUqWErEQOibkYACCXKWDX3goFySXISymEXTNpjdeU1yS68XMykm+moFlr7zrFpe1euravSx/GqqGMoz5RIlQLbw4P4zoETk0eEIykx7rX7lAoGXzwv92Q1+K3DpWKQe/J/dG6c8OfriXEWGmrIyTPV4AFCwt3c5Rmy8CoVOpzjEoFiYMYPEFZu9pcA+hWk0jX+kZ1qYfEZQ0lfWoo46hPlAiRGplLxGjl41ana49GTqtVuxKZHG+u+Bvnfo3S2oYFC7lQiP6vDgSPV/XslEAkhHeAp9bzhBDtKtYRsvG0Uh8XS0XggYfixyXgm/HBFwjU5/gCAWRZcrCqsna1uQbQrSaRtri03UvX9nW9xhg1lHHUJ0qEiFEwl4ix6cMXamx3Ji4ZR05d13o+MSMHMSzg7udR6ZxAJED3F7pDLBE/V6yENFQV6whVXF9i620NczszPIrOgs9QN/VHXAAgloiQc7kQrJLV+Ie3umt0rUmkLS5t99K1fV2vMUYNZRz1iRZL18bZFVxHQHQQeysFBcWySseT0nKwbP952DnZVTonFAsx6v0X4eBqXx8hEmK0NHYc9XSFpasFitKKcWtn8tMdYP2aqI+nxKRp7hqr5TXV7V6qqv7NjQu3qoyr/F6TP3gFllJL9TVF+UXYtHSL1vY17hqr5TX1RZeaQMY8jvpEu8b0iRKhBkOpVIGp4ls+OTUbb638G+wzP1hKZQq06hWEtj3aaRy3srGEhbWFQWMlhCvaatBUVxMI0E8doerq3+jah65919Q/V8lDY66H9DwoEdInSoQaLZZlsfiPk7iXkatx/EriY3QYHg6h6Om6B4FAgI59OkAkFoEQU1eXKtHPU1kaqF39m1adAjTupTHzU8U1z84UmVpl6eepCWRM4+ACJUL6RIkQeUZmbiGOx97ROPYouxBbz92Eu7dmzSW/jv4IoecjEVKtxlz7R5uGPj5DozpChBiQo60VXurTodLxyQOCUVwq1zj21e8nsXrfPyjfyCZTKNFtbC+0rFAqQCgSQCiiv46k8WrMtX+0aejjMxb0k5cQPbKXWsJeaqlxbPU7IzS+liuU+ODnQ9iz/x/1sQfpOQh/uTesbZ/uumni6wF7l8oLuwlpiBpz7R9tGvr4jAUlQoTUM7FIiBVvD9U4ll9Ugk1HLkOVlg2grGbS5nUH4NOhBfDfL4I8Ph/dxnSHraNtPUdMiOE15to/2jT08RkLSoQIMQJSS3O8Mypc49jU/sG4V6Gid05BMebOXgWL8hknPh+jZr+o8QwnQkxVY679o01DH5+xoMXStUGLpYkRevwkD9NX7EWpsuzRBbn5xejz2iA4uJTVQrK0sYSjmwOXIRKik7rUv2noNXMa+vgMiXaN6RMlQsQE5BWWYMnO05CpGADAxVsp8OgUAHOrsvUFLl4uCAxvzWWIpJHSZVv9jQu38PfGA3j88DFUjAoCvgDuTdwxbEpZ/Rtt12irmfPsdntj30Ku6/goCdKOEiF9okSImCCZXIETlxPUBSS3nLiOh6UKiP7bndasvR96jevNZYikEdC1OGNgSGtcPx+HR/cfQ6lUQSgUwMPLHcOmlP2DpkvCY2oJRHVFEE0toTMGlAjpEyVCpAFgWRYFxaXqr7/78wxO3EwBn8cDzMQY/cEY9RZ+S6kFPZONPDdtxQDvHXmA1POZcAiwRcAYH/XxWzuSnj6So39TjeKBGo/xqEVhwecpRMgFU4vXFFAipE+UCJEG7mxcMlYfjlV/ffNRJsLH9ACfz4OLpzO8A7w4jI6YouqKAaYmpyFpXypQyEPYnHbg8XlgGRbnvr0KWLPwGeKmsQmAZVic/+kq8hOK0XdpZ/CFfI1zzxYWNLVChKYWr6mggoqEkFoLD/RGeKC3+ut7jzJx8to9AMDvf5/DWVcH9eNEfDs0R3DfYC7CJCZEWzFAWYkcSpUSHr0ckPDrY+QmF8CumRS5yQUoySmF3zB3KFVKyErk6ifWy2UK2LW3QkFyCfJSCmHXTKq+X1WFBU2tEKGpxdvQUCJECKmkmYcjmnk4AgAm9AvG/f/qGwFA5I7TWLn7NHg8wNrZHi/NHQuhUAjwQL+tEjVtxQAZVdkuRwv3suPyfIX6vyxYWLibozRbpm5Xfo3EQQye4Gn7ip4tLGhqhQhNLd6GhhIhQki1REIB/Jo4qb/+5f1R6v+/50w8Vn+4GgCQkVuE/m8MgZm5BJZSSzRt3qTeYyXGQ1sxQL6gbGax+HEJAEAsFan/ywMPxY9LwDfjq9uVXyPLkoNVPW1f0bOFBU2tEKGpxdvQUCJECKmzkV1aY2SXsi35j5/k4ddjV8CAxelbDyBwd4SVnTXE5mL0fKknPUutkdFWDFBiLoZQIERSdCrM7cxg6132j7uttzXM7czwKDoLPkPd1B+LAYBYIkLO5UKwSrZSolBVYUFTK0RoavE2NLRYujZosTQhOmEYBhdupkDFMIi7n4GfDl6A1E6K7uN6oU2XQK7Da3S01fGp7pyux6uirRjgvaiHT3eNveitPn5rZ/LTXWP9mmgUD9TYNVaLwoLGUoiwtu9XTfFO/uAVWEotafu8DhrcrrHIyEjs2rULt27dgrm5OcLDw7F48WL4+/tXe92OHTvw+eefIzk5Gc2bN8fixYsxePBg3TqnRIiQ5yaTKzD9f3vxMLsADMPA1ssV3cf2BADYu9jBzMKM2wAbqOpq0wBV1+WprsZPVcerq8tTlzpC2vrQFq+2/qsbe30kQbr2r+t7Zaz1kIxFg0uEBg4ciHHjxqFTp05QKpX45JNPEBcXhxs3bsDS0rLKa86ePYvu3bsjMjISQ4cOxbZt27B48WJcunQJgYE6/FZKiRAhevdb9DWcu/0ALMvi/N1HCB4cCqmjFO17BHEdWoNRXW0abXV56lLjp6YZFn3OOukyI1WX9vpS17pAz8ZblF+ETUu3UH2hOmhwidCznjx5AmdnZ5w4cQLdu3evss3YsWNRVFSEffv2qY917twZQUFBWL16de07o0SIEIO6n5aNy3ce4ui1JFzLyIO1jRXElmYYPnMEzC2r3klDqldTbRptdXnqUuOH6txo0lddIKov9HxqmwiZ7DuXl5cHALC3t9fa5ty5c+jbt6/GsQEDBuDcuXNar5HJZMjPz9d4yeSVt2sSQvTHy9UeI7u3xY8zR2DX7NHYMK0f3u7oh7Xv/oj/vb4Up3adQmFeEQrzisAwDNfhmoTy2jSevdwq1aYpr8vDE/KQl1KoPl6xxk9JTilyk8u2a5fX+PHo6aCu8VOuvM5NeloGkm+m1M/gjFx1770u75e+7kOqZ5LbOBiGwXvvvYcuXbpU+xFXWloaXFxcNI65uLggLS1N6zWRkZFYsGCBxrH50wbii1d1XFdECKkTR9uyXUH9OlqjX8cWYBgG8349hiORW6FUMchmWIQMC4OjuwM8WzTlOFrjVV1tGm11eepS4wegOjfP0lddIKovVD9MMhGaMWMG4uLicPr0ab3fOyIiArNnz9Y4Jrm0Tu/9EEJqh8/n46sp/dRfn7l2D9eS0rB35wlYerlCYmmGfpP700doz6iuNo22ujx1qfEDUJ2bZ+mrLhDVF6ofJvfR2MyZM7Fv3z5ER0ejSZPqC7a5uroiPT1d41h6ejpcXV21XAFIJBJIpVKNl0RcuYAXIYQbXdo2w1sjwvH3gklY0Lcdxnk7Y8Vby/DjW8tw9q+zXIdnNCrWpmEZzaWg2urylNf4eRSdpbXGj1Ag1KjxQ3VuKqvuvdfl/dLXfUj16pQIpaSk4NSpUzh8+DAuXboEmUym77gqYVkWM2fOxO7du3H8+HH4+PjUeE1YWBiOHTumcSwqKgphYWGGCpMQUk9EQgECm7ljeJfWuLRiBk4umgLh7RSsm7EcP7+9DGvnrEHC9XtIf5DBdaic4PP5GDp5MEoSFYjbnIC8+wVQylTIu1+AuM0JUD4AzIXmiN+aqHEu7WgOcq8WQiASIP9BIZQyFfIflH2de7UQaUdzKt2rJFGBoZMH04Ld/9T03tf2/dLXfUj1ar1rLDk5GatWrcL27dvx8OFDVLxMLBajW7dueP311/HCCy8Y5A/l7bffxrZt27B3716N2kE2NjYwNy+bEp80aRI8PDwQGRkJoGz7fI8ePfDNN99gyJAh2L59OxYtWkTb5wlpBI7G3sWJ+Ps4G58Ml3Z+sJBaoMvILo3uI7Sa6gj9vXE/Ht1/DKVSBaFQAA8vd7QJDcS1f+Lw4N4DKJVKCIVCNG3WFG07B1Zbz4arrer6pq9x6KuOEdf1kEyVXrfPv/vuu9i0aRMGDBiAYcOGISQkBO7u7jA3N0d2djbi4uJw6tQpbN++HQKBABs2bECnTp30MhB1oDxelcc3bNiAKVOmAAB69uwJb29vbNy4UX1+x44d+Oyzz9QFFb/99lsqqEhIIyKTK3D2ehJScwrx7e4zsHexR9teQeg8tPHMDGv7hz3u/A38vfEAHj98DBWjgoAvgHsTdzi5OeJCdCxkilJAAEAFSERmGDZ5MAZPGqD1Xg3hH2t9j0NfSVVDSTLrk14ToYiICHz44YdwcHCo8YaHDh1CcXExRo8eXbtITQElQoQ0CEqlCkoVg49+OYz4h09QyjAYNXsMLKWWsHex4zq8eqWt4N/dA8lIPZ8Ja18L+IxyhYW7OYofl+BRdBZyrxbixemjMHTKoFrdy9SK/jWUcZAyDb6gYr2iRIiQBun2/XR8/9c/SMvKB9wcEDI4FL6BNa8/NHXaCvWxLIv7t1Lw+GgmFAUqtHmv2dNzKha3NjxAyU0Ffor6AUKhsNp7AaZV9K+hjIM81eALKhJCyPPy93LBmndGYO8XE/F2kA/Ord2HLQt+xc6lO1BcWMJ1eAajrVBfUV4RwAdcetpDnqtE4f2n7wFPwINHLwfIlDKcO/hvjfcCTKvoX0MZB9GdznWEsrKyMG/ePERHRyMjI6NSldfs7Gy9BUcIIfVlQGhL9O3oj8eZebidkoFP31kBoViIoP6dEDokFEKREHxBw/jdUVuhPqVCCQCwdC97AK6iQKlx3sLdHDwBkJmaVeO9yplK0b+GMg6iO50ToYkTJyIhIQGvvvoqXFxctC5iJoQQUyMQ8NHUxQ5NXezQt5M/GIbBZ78exc5P1iE1txB9Xx2E1iEtIRSZZC1aNW2F+srHVfS4FAAgstYcZ/HjErAqwNHNocZ7lTOVon8NZRxEdzr/bT516hROnz6Ndu3aGSIeQggxGnw+H4um9AcA3HuUiY1HL+OnLcfQrG0z+AY3R6uQlhxHWDcVC/VVXA9jaWOJzNQspMdkQ2wrhJXX09kRVsXiUXQWJEIJwgaF1HgvwLSK/jWUcRDd6ZwIBQQEoKSk4X52TgghVWnm4YiFk/th6uMsZOUVYdHvMTi+6QjMpZYY/8nLsLC24DpEKJVKnD1wHknxyfBp7Y3wwaEQCoVQKpU4d/BfZKZmwdHNAWGDQjB08mBsXLIZcZsT4NnTFZauFihKK0bmiQJkXSqAlY850s9kQ2AuhKpEiexbhci7VoQXp49SL5QGnhb9q+peKTFpKElUYOxHVRf9qyouoVBYp63iul5TVfu6joO2tps2nXeNXbhwAR9//DHmzZuHwMBAiESaj5+QSqV6DdAo0K4xQogW5+PvY+6mKFh6OKH72J6Q2lvDzsm23uPYt/Eg/t50ADJlKcADwAISoRl8WzVD4o17kCll4AkAVgVIhBIMmzwY3i29qqyZw+cJEHchHhABfAEPjIoFFED3oV3x2rwpVfava/2dp/FqxtWpVzAy0zJ1quOja981FZnU171oqz23DLZ9/u7du3j55Zdx6dIljeMsy4LH40H1zFOJGwRKhAghNdh1Kg7nbj/EuRv34dejLUIHhdZbQrRv40Hs/Hk3bIMs4dLVDuauIpSkKZB+IhtZlwtg7WMOnxfcqqwJ9GyBxKL8ImxaugVmvkI4BdtCbCeEPEeJzNhclCQqq62lU9uZkafxWsGjp0OFuDKRFZsPm6bWaPeaf63q+Oha+6c27Vt1CqjVOKjukHEzWCIUElI2dTlr1qwqF0v36NFDt0hNASVChJBaKiyWIerCLXy98zTs3R3h2dobfV7pa7CNJUqlEjP6vQ/zViK0mOgBhlFBJOBBrmJR+kSOR4cyUZqpQNAcP/AE3NcEqhhvwJSm6pgAQF4iR+KWx8iNL8bAH7uAL+RX27eu8epzfFR3yPgZrI5QXFwcNmzYgLFjx6Jnz57o0aOHxosQQhozKwsJRvVoh7Pfv4Ed7wyHd2Exvp/2Lf499C9yM3PVW9T15dzBfyFTyuDR0wEMy4LPAwR8AAoWLACXnnZQFCqRl1ikvobLmkAV462YBDEMC/DKahixLIOHZ9Nr7FvXePU5Pqo71HDonAh17NgRDx48MEQshBDSYIhFQthYmeOzl3vh7HevQ3jjPk4v24UVM5bjUsyVSjXY6iozNQs8AWDmIgFYBsLy5OK/yX4LNzPwwIM8T6FxHVc1gcrjtXB/po//4rV0NwNPwENxZmmNfesarz7HR3WHGg6dd4298847mDVrFj766CO0adOm0mLptm3b6i04QghpCERCAb59dQAA4EZSKrbHXMePO04gdEhnhA4Ofa57O7o5gFUBhY+KYeNthvLJCcF/CVHx41KwYCG20fxZzVVNoPJ4ix+XQOpr+fTEfx8dFj0uBatiYeFoVmPfusarz/FR3aGGQ+cZobFjx+LmzZuYNm0aOnXqhKCgILRv3179X0IIIdq18nHDwqn9seP9UVBcScCPb/2An95dgazULNTl0Y9hg0IggADpJ7MhqHBcJOGDByD9RA5EVkLYVEg6alMTiGU0Y9FXLZ2wQSGQCCV4FJ0FVvW0Dz6fB7BAekw2eDw+moS71Ni3rvHqc3z18V6R+qFzIpSUlFTpde/ePfV/CSGE1MzDyRY/zhiGk4um4sN+7fHrnNVY8uoS3L2WqFNCxOfx4WBvi5zLBbi1+TFy75VAWcIgL6kEjw5mIutSAXh8oCC5GMoSBvmJRbi14QFyrxZi2OTBEAqFYFkWDxMfgcfjYejkwShJVCBucwLy7hdAKVMh734B4jYnoCRRgaGTq66lU1tCoRDDJg9G7tVC3NrwAPmJReq4EremltUwcrZAwaOiGvsur2FU23h1bV/t+67HexFu0dPna4N2jRFCDOB47B18u/0YEjKeQMEyUJYy4Kl4cPVtgsDQVmjTsx08mrlXe4978cnY9Plq5D7JRU6JHDwJHzwBD6yKBStjIAYPMgAC8/LjT+sIDZ0yCABw+eRV7Fr+O0bPGov23dvVS20cY60jpOv4qI6Q8TLY9vnIyEi4uLhg2rRpGsfXr1+PJ0+eYO7cubpFagooESKE6Nnx2DuY8eMOCP1EaN7DFVIXc+Snl+DuiTQUxhXj3RE98Oe/d2DubIexH4+HlY1llfdRKpS4efE2lAolGCWDm5duIz8rH1IHKVp28AdfyAefx0duVi6yM3I1KjgDAKNi8L/3lqE04S7M/Jrj3WXvgS/g10u1ZGOrLF3X8VFlaeNksETI29sb27ZtQ3h4uMbx8+fPY9y4cUhKStItUlNAiRAhRI8YhsHguWvwyLYAIRN8K9Wg+XdrIjxyrbH/m9dx5e5jfLD+MCzcHDDo9SFwcnfUayyXT17Fvu82YW5PS3wTU4RhH05G++70LEli+gxWRygtLQ1ubm6Vjjs5OSE1NVXX2xFCSKNz+c5DJGQ8QfMerlXWoGne3RUJGU9w5e4jBAc0Rcy3r+H1kBY4vHg7dv6wE8m37uslDkbF4MSOY+juycfQtlL08OTjxI5jYFT62dpPiCnQORFq2rQpzpw5U+n4mTNn4O5e/WfZhBBCgMy8IihYBlItNWikruZQsAwy854WQRzZpTX+XjARr7VqgmPLd2HLgl/x5HHmc8Vx9cx1FNx/iMlhtgCASZ1tkX//Ia6euf5c9yXElOhcR2j69Ol47733oFAo0Lt3bwDAsWPHMGfOHHzwwQd6D5AQQhoaRxtLiHh85KeXwL6KGjT5aSUQ8fhwfGZdEI/Hw8DOrdC3oz9upaTj7S82QWIvxZg5YyG11+2B1xVng1q4SAAA/q4S9axQuy5twBfQOhfS8OmcCH300UfIysrC22+/DblcDgAwMzPD3LlzERERofcACSGkoWnfogn8nJ1w90RalWuE7p5Mg5+zE9q3aFLl9UKhAIHN3HFyyXQcv5yA7+ZvAt/VDq27tYFHM3c4N3GqMQb1bNBYe43jkzrb4sQfZbNCtFaINAZ13j5fWFiImzdvwtzcHM2bN4dEItF3bMaDFksTQvRMY9dYd1dIXc2Rn1aCuyfToExQ4KeZY9A7uEWt7/f32Ru4n5GL305dR9Og5ug9oQ+kdlVXNS7fKdZG9QhfDneudP7zvzJwXeCh3kFGiCky2K6xRokSIUKIAVSsI1SiVMJcKISfsxPmjOtTYxLEsixuJqehpberxpPti0pkOBeXjI9/PYomLb0w5qOxEAgFGteW1x4SKUqfva2aQmSGSQvfgMRcDI9m7hp9GCPawk6epddE6M0338Rnn32GJk2qnqat6Pfff4dSqcSECRNqF6kpoESIEGIgDMNgxc6T+HFHNGaO6YV3Xuxeq3/AD5yNx7y1e7Hw9REYHN66yja/RV/FunM3MXD6EDT181Afr1h7SBuhSAhZiQx7f9qpLrRorKioIalKbROhWq0RcnJyQuvWrdGlSxcMGzYMHTt2hLu7O8zMzJCTk4MbN27g9OnT2L59O9zd3bF27drnHgAhhDQGLAuciL0FG0EpTsTewswXutd4jUrFYN3eU0BpHtbtPYUBoS0hqOIjrPG92sHOygzr1/yNU3bWCBkcCr8gPwhFQrQJqzp5Klf+8ZllcY5RL56OO38DG5dshrmvCK2G+8DSxRxF6SVIiU7FxiWbMeWjiZQMkWrV6rv6yy+/xJ07d9ClSxesXLkSnTt3hqenJ5ydneHv749Jkybh3r17WLt2Lf755x96Aj0hhNTS4fM3kfLoMT7tY4+UR6k4fP6mXq8Z2Mkfv386HnO7tcbZXw7g5zlrkJ6SUWMf5YupZ/dzNNot9QzDYN+mAzD3FSFwoh9sPK0glAhg42mFwIl+MPcVYd+mA2AYqotEtKv1rjEXFxd8+umn+PTTT5GTk4OUlBSUlJTA0dERvr6+Rv/5MSGEGJvymZ3ungKMamONU0ml1c7w1PUaHo+HLm2bYX8bHySnZuONxb9BYSbGqPdfhJOHY6Wf388WWjx3r8QoZ4WSb6YgPS0DrYb7VFmY0rOnK278nIzkmylo1tqbmyCJ0avTd7SdnR3atWuHzp07w8/Pj5IgQgipg/KZnddDy2oATQ+V1jjDU5dryvF4PPi4O+DIoqn4anQXnF6xC5u+2ITbl+5qtDOVQosFuQVgWAaWWgpTWrpagGEZFOQW1HNkxJQYT2pPCCGNSMWZnYD/Chq2dJGguycf6/aegqqKx1zU5RptugR6449PxuPjnm1wY/txbPx8A+7FJ9dYaNGYHr9hbWsNPo+PovSSKs8XpRWDz+PD2rbqMgKEAJQIEUIIJ56d2SlX3QxPXa6pSc8OzfFbxFisGNsNh5btxIHNh5Gf/EA9G1TOGGeFvFt6wsXVGSnRqWAZzQ3QLMMiJSYNLq7O8G7pyVGExBRQIkQIIXXAsixuJKWiLqXYymd2wj34aOYgRqmCwZ1HWShVMPB1ECPcg6ee4SnvR6lUaVwjUzC4+qgUsiquqQrDMPjr9PVKC4cZhsHlOw9x73EWvhjXEye3HEFediGSM2U4m1iIw/H5uJBcBC97Ibo04RnVrBCfz8fQyYNRkqhA3OYE5N0vgFKmQt79AsRtTkBJogJDJw+mekKkWjo/YoNLJ0+exJIlSxAbG4vU1FTs3r0bI0eO1No+JiYGvXr1qnQ8NTUVrq6uBoyUENLQHTx3o8Y6PtpcufsQKWmZSFGqEL4qDZm5hVAxDAT8bDja/vfsMWEmrtx9iPTsAsxbuxcTB4drXJNXrEBWQSkcrM1gYyHSuCY4oPIMyJcbDmP59ijMGtcP818dBECzoKOCZcAqWeQXF8MMPEzZ9AgKloVQwodQxIdACdhbiiGRZiD5lvEsPg4MbYUpH03Evk0HcOPnZI06QmM/ojpCpGYmlQgVFRWhXbt2mDZtGkaPHl3r627fvg2p9OlUsrNz5ZLyhBBSW7Wt46NNG193fDNzDOQKJYqKZZi4cCOcLHlIK1BhyfujYGkhgVgkRCtvV3y94QBQmofoCzfx9dsvQKVioGIYLP71MCyF6bCyc8TcSQMg4PMhFgnRxte9Un9yuRLr/zoFVwsG6/86hYiJ/XD6+j31Iz7ajvCG1MUceWnFuLgjCU/uFMC9iyPc29gh43Y+ZHIGPCEfRXdk6DK0Ozy1PAONK4GhrdCqUwBVliZ1ovN3SXp6OiZOnAh3d3cIhUIIBAKNlyENGjQIX331FUaNGqXTdc7OznB1dVW/6C8HIeR51KX2T0VikRB9O/ljcHhrLNx0GI4WPHzVSwIHCx4WbjqMweGt0beTP6Iv3VX38zA1DSoVg8HhrSHg81FanI/5/R1RWlwAAZ+vvkYsqvz7beTmKLCKEkR0MwOrKMHXvx7Bt9uPQegnQsgEX9j/V3/HvqkVwAB27a3QZKAjAsLc0H2KP5zdrVCcUgyBHQ9Xz10zqi305fh8Ppq19ka7Lm3QrLU3/ZwntabzjNCUKVOQkpKCzz//HG5ubiaxdT4oKAgymQyBgYH44osv0KVLF61tZTIZZDKZxjGJXAGJWGToMAkhJqAudXy0ycsrxr0HqRjbWoTJQWKcuK/E7/GpyMsrhpWVWZX99O3or1P/5bNB/ZsJMDHIHDHJSqzdfQIWjpZoN8Jbo/5OVnIhCnNl8BvuDplKheJSOSzMxGg3tCkCB3rg/NZ7SDiRiAtHYxHav9Nzv5eEGAOdU+bTp09j69ateOuttzBy5EiMGDFC42VM3NzcsHr1avz555/4888/0bRpU/Ts2ROXLl3Sek1kZCRsbGw0XpGbo+oxakKIMXueOj7P6jpzOezMeHgnpOwXrRmdxLA146HrzOVa+4ncHKVT/+WzQTM7mwEAZoaaQSmXIbe4BNJn6u+UFijAgoWVuxlYsFBWWBQtEPIRMs4Hjs6WOLT6b2z9aguKC4p1HjMhxkbnRKhp06Z12iXBBX9/f7zxxhsIDg5GeHg41q9fj/DwcPzwww9ar4mIiEBeXp7GK2Jiv3qMmhBirPRZx6d8Nqi/rxDt3com54PdhRjgK8C9B6lY9Wd0pX66NuVh/V+n0K0pv1b9V5wNCnItS7bau4kQ3pSP4iI58tI0ExkzaxF44KHwcSl44EH4zAxTfloJzERCbPzgBczp1hq/vL8Svy3aRo+wICZN50Ro2bJl+Pjjj5GcnGyAcAwvJCQECQkJWs9LJBJIpVKNF30sRggB9FvH59nZoHLls0LHYhMq9RPgJAKrKMHLbc1q1f+zs0Hl5vU0B0+mwpVDDzTq7zh4W8HKVoKH0ZmQCASwMBOrz7EMi7sn0+Dn7IROrbzQLcgXZ5a+jqHNXLDh8w24c1mzOjUhpqJWa4Ts7Ow01gIVFRXB19cXFhYWEIk0/xJnZ2frN0I9u3LlCtzc3LgOgxBiYp6t/SNXPk0gKtbxqc1aofLZoBdaitDSiY9S5dMZldbOfPTxEeDPG0o4WfDV/agYFoduFaK3jwCW/FLIFNYo/7FcVf/ls0G9fQQIcBSitEK8rZ1F6ObOx4mL2fjXPBHNe7hC6mqO/LQSCEV8ZF3NQ7okB459LNTH755MgzJBgTkz+2gsRJ4+qBOau9ji5z9i8PD2A/Qe11sfbzch9aZWidCyZcsMHEbtFBYWaszmJCUl4cqVK7C3t4enpyciIiLw6NEj/PrrrwDK4vbx8UHr1q1RWlqKdevW4fjx4zhy5AhXQyCEcIxlWdxMTkNLb1edNns8W/unStXU8QH+e1r62Xhs3P8PRHzgxH0l/H9UVhEkIBIAzb+9Dx8nC5QqGKgYFtkFJbAUASfu50FqKQfDMJCIhFBnRMJMXL7zEBZmIsTeSkFJaQlOJLNouSK3ynjMlAA/kcG1u8lQsAxEPD78nZ0wc2JXRF26jWvrnh73c3bCnJl90Du4RaX79OzQHD3a++Gdlfuw/I3vMWD6EAR09K/N26r1faKt8KS+8FhTWfAD7QUSJ0+ejI0bN2LKlClITk5GTEwMAODbb7/F2rVr8ejRI1hYWKBt27aYN29elfeo1tkVeoieEGIMDpyNr1MhRLlCiZNXEiFXVJG4/EcsEqJ7kG+VW9gBYMEvB7F8exQmDQ7H2fhklMgUlRuxLApL5BAJBYiY2Acp6Xn47fA/GNM3BGZiIZQqBkIhH3GJqdh3+gqGd2+PCf07qvsvKpHhy/X7EDF5EBIfZVXdx3/MJSK8OTIcdx9mIjOvCI42lmjfogn4fL664vSzx2tSVCLD8AVbEDg8HIFhrWFdXiCyluLO38C+TQeQnpahURxx6GQqjkh042DuiH7NhtTYTudESCAQIDU1tVJRwqysLDg7O0OlUukWqSmgRIiQBkGlYjDmk7VIuZ8ETy8f7Fj0us5b3utKLlei+Zh5sEQximCBuzsWQiyuflJeW7za7sXl+CoqKpFhxd5z2HX+FiZ9/Soc3RxqdV3c+RvYuGQzzH1F8OzlBksXcxSllyAlOhUliQpM+WgiJUOk1mqbCOn8N0Rb3iSTySAWi6s8RwghxuB5CyE+j2eLGtamLIe2eLXdi8vxVWRpLsHH43ri94/GYOPctdj65WbIS+XVXsMwDPZtOgBzXxECJ/rB5r8ijzaeVgic6AdzXxH2bTpAO9SI3tW6oOL//vc/AACPx8O6detgZfV0ulOlUuHkyZMICAjQf4SEEKIH+iyEqKuqihqWP+pC26yQtnh7tW9e5b3mvNyHs/Fp4+PugH+Xv4Wo2Lv48qPVGPPRODg3dapybVbyzRSkp2Wg1XAfjSKPAMDj8+DZ0xU3fk5G8k3jec4ZaRhqnQiV195hWRarV6/WeJyGWCyGt7c3Vq9erf8ICSFED8pnSxaNsQNQtuV84o6yWRNdH5qqq6fb2C0BlBU1jLpXhMjNUeoHoNY23tcW/1blvV5b/Btn46sOn8/HgE7+kIiEWPXTHigcbTDmw5cqJUMFuQVgWAaWzxR5LGfpagGGZVCQW1AfYZNGpNa/JiQlJSEpKQk9evTA1atX1V8nJSXh9u3bOHz4MEJDQw0ZKyGE1Ik+CyHqSltRw37NBFj/1ynI5ZUXX2uLt1tTHvafvoK+PnyNe/Xx4WP/6Su1LrTIhZ5Bvvj9k3EIs7PEipn/w6VjmhX+rW2twefxUZReUuX1RWnF4PP4sLa1ro9wSSOi83xpdHQ07OzsDBELIYQYhD4LIepKW1HDmaHa1wppi3ekPx9WQgbt3TUn8zu4CWElZDAqQPPB1/UxPl19NKY79n70Iu4fuYizf52FQl62q827pSdcXJ2REp2qUeQRKCvmmBKTBhdXZ3i3rLo0ASF1VauPxmbPnl3rG37//fd1DoYQQqpS19o/QO0LIfbt2AJr/z6Ht0Z20fjov7z2z9Dw1hrbx7Udr3iuf0d/jaKGJQoGtzIZBDjy0dJJiN7eZbNCH7/SF4mPM9HS2xUMw2rEK1OwKJUrIBEJYcUrRW8fIY4mKjGtPQMBnwcVw+LoPSV6+whhiRKUyhnw/1tjo2uhx/riZGeNbXNfQsTGKCx7NQYvRrwMn1beGDp5MDYu2Yy4zQnw7OkKS1cLFKUVIyUmDSWJCoz9aDDVEyJ6V6vt88/W3bl06RKUSiX8/csKZt25cwcCgQDBwcE4fvy4YSLlEm2fJ4RTda39AwCxt1LwRuSvgFKmvZFQAicHB5y6fAsjenTA1i8mq0+V1/6ZNa6fxnoebccrnuvZsRVOX4qHpbDsx2yxgkWpkoWZkAcLUVmyUqTkYdLQ7oiJvYmFr4+Ai721RrwyuRKFJaUQC4UokcnB4wGFckAiACRCHmRKFjIVYCUGWBawtjSHueSZxwIJJVgTMUlroUcu5eQXY/iCLRg8+0X4tPKmOkJEbwxWR+j7779HTEwMNm3apP6ILCcnB1OnTkW3bt3wwQcf1C1iY0aJECGced7aOLUphKhiGUycvx5ulixSi3hI/XsxzM3FWuv1VFcTqOK5QpjjjdG9oFSqwDAsdkXHgifLBSuxxeheweDzeRCLhTh35S4eP7oPTy8fbF0wDWeuJ0GuUELFsIjcdBBPMtJh7+iMlj7uUChVEPD58HK1h1DAh1LF4H5aNlT/VZkeEt4K5hLNUiY1FXrkWk5+MV74+jewlmYY98kEWNtZUWVp8txqmwjp/Ldi6dKlOHLkiMY6ITs7O3z11Vfo379/w0yECCGcqVgb5+sTuu+CEouE6Nup+sc9TPhiE6yEDCK6SPDJcRleW/wbtn4x+Wm9nt5m+PR4iXqXl7bjAJ45VwqlUoX5rw7CgbPx2H/iH3w60BlfnyhGaGsvDA5vjQNn4/Hn4VPq8UVfuqse34Gz8ZCXFODboU74+kQRXurdntMdYIZiJ7XA8cWvIv5eKqbOWYNXv32DtsiTeqNzip2fn48nT55UOv7kyRMUFNC2RkKI/jxbS8cQu6BKSuTYf/oK+vsKMSlIgv6+Quw/fQV5ecUa9XrKd3kVFpZWeVwuV1aqF1R+rqREXuU45HKl1vHVx9iNTetmbtj4znD8MmcNYnaegLKaWTxC9EXnRGjUqFGYOnUqdu3ahYcPH+Lhw4f4888/8eqrr2L06NGGiJEQ0kg9u3vKELugXlv8G6yEDN4JKfs4aWaIGFZCBl1nLtfY7VW+y2vo3LVVHo/cHFVph1j5ufIaP8+OI3JzlNbx1cfYjVErHzfs+vgltCkqxi8R6ygZIgancyK0evVqDBo0CC+//DK8vLzg5eWFl19+GQMHDsTKlSsNESMhpBGqj9o/FWeD2ruVrRTo4CZE32Z83HuQin5V1P65eCOhUh2ffs0E+GXvSaz/62SV9YL2n76CLk14mjWBPPlY/9cpdK2i9s/Pe07iZ47qHhmDJs52mDGyC97t1Ra/RPxMyRAxKJ0TIQsLC6xcuRJZWVm4fPkyLl++jOzsbKxcuRKWlpaGiJEQ0gjVR+2fZ2eDyrVyEsLWjIcp7TR3X73cRgQbCQ9BbprLK2eGmqG4uBhKWeV6QVM7lM0w+dhobvsf30YCVlGClk6afUwPlSIu4QHu3kvhpO6RMRnVNRCzegfh5zlrcO1MHNfhkAaqzsvwLS0t0bZtW7Rt25YSIEKIXlVV+6f8VbE2Tl1mRliWxY2kVBQXy7D/9BX09hGipRMfJQoVLqcqUCBT4ug9Jfr4CGBrxqJEwaBUWfZfWwmLPj4CRCUqUCRTIbdUhVIFgxb2AkglQNemfPg7CFCqLNsmX6pgYC8BevsIceBmAYpKVbj6qBSlcgZW/FL09hbg4K1ClMgZ9fi87UTgq2Ro76yEj732sSuVKtxIStX6IOyGYmSX1lg6tgfyoy/hr5V7uQ6HNEC12j4/evRobNy4EVKptMZ1QLt27dJbcEaDts8TUq9qW/unLrVxymsStfRpgn0nYyEt++QJJYqyejx8AHweYCXhgWXLHjTN5wEMW5ZE8XhAoYyFWMiDiF92Xq4qS1QsxTx1e6Csrg9Tfo0ckIhFKJUrYWMhBhgFwLIoUgAOUgtYSsqKOBbJVMjKL4almAeppTlEQkHlQQglmDikKzYfOFun2kqmat6vR3GXL8CIGSO4DoWYAL1un7exsVFXc7WxsXm+yAghpAZtfN3xzcwx1db+EYuEaOPrrtN9y2eaUJqHnDwbvNAnFCVyBRiWxfnriXBmi1AIc9haW0PFshALBejbsQUszCSQKRS4cPMBFEoVhEI+HqVnQVaUBzMrO4zv3haxtx4A4KFTy6aQiMo+7qpY40ckFOB28mOU5D2BpZ0TBoa1AcOwEAr4CGzmpk54FEoV4u6lAjyUHRdUToQEAj5W/RkDlOYZXdVoQ1o4qS/mbz6KPT/uwYgZI3SuMk5IVWqVCG3YsKHK/08IIYZQm9o/daFZkygDb7/YU13L527iPXzawwVfnyjGwjeHVTvLcuBsPOat/B2Lhpe1797OD9+8Vf0sRdk1dzC/vyO+PlGIYP+mWvsY0b1tjfd6+Di1zrWVTNmCiX2xYPMxbPh8AzoP64xWVG2aPCedf4VYv349kpKSDBELIYQYjLa6PNXV8tHlPtWtV9JnTaDGWF/oWfMn9sF3o8Jw/Y8YxB6N5TocYuJ0ToQiIyPh5+cHT09PTJw4EevWrUNCQoIhYiOEEL3RVpenulo+utynup1c+qwJ1FjrCz2rrZ8H/vhkPBIP/ouLURe5DoeYMJ0Tobt37yIlJQWRkZGwsLDAd999B39/fzRp0gSvvPKKIWIkhJDnoq0mUdemPKz/6xS6VVHLp6pZlrrUNtJnPaT6qK1kSkRCAX7/ZBySDl/AhSOUDJG6qdPqOg8PD0yYMAE//PADli9fjokTJyI9PR3bt2/Xd3yEEPLctNUkCnASgVWU4OW2mrV/tM2y1KW2kT7rIdVHbSVTIxIKsD1iHO5HXcSFwxe4DoeYIJ0ToSNHjuCTTz5BeHg4HBwcEBERATs7O+zcubPKZ5ARQgiXtNUkKpEzOHSrEL19BLDkl0KmqL5WUV1qG+mzHpIhayuZOpFQgO0fj8WDo5dwbt85yEqqKbtAyDN0fvr8wIED4eTkhA8++AAHDhyAra2tAcIihHCBZVncTE5DS29Xk96aXHEcV+4+REpaJlKUKoSvSgPAQqliIFOyyMovgaUIOHE/D1JLeeWaPcJMXLn7EMEBnlXcpwoV2gOo0zXa6PNeDZFQKMC2j1/Cp78exYo/YjDmkwnwaoTvA9FdrQoqVrRs2TKcPHkSJ0+ehEQiQY8ePdCzZ0/07NkTLVq0MFSc3KKCiqSRKC82aOpF+iqOo28nf5y8kqiuSXThZgq2HPoH4/qFwEwshFLFQCjkV1mzRywSonuQL8QiIeQKpcZ9qlKxPYA6XaONPu/V0GXlFWHEgi0Y+tFL8A7w4jocwpHaFlTUORGq6Pr16zhx4gSOHz+Offv2wdnZGQ8fPqzr7YwXJUKkEVCpGIz5ZC1S7ifB08sHOxa9bpJF+qobR0MZI6lZdn4Rhn+xBUM+fAk+LSkZaoxqmwjV6ScAy7K4dOkSoqKicPjwYURHR4NhGDg5OdXldoQQI1Cx2KApL7ytbhwNZYykZvZSS/z1xSs4sHQHkm4kcx0OMWI6J0LDhg2Dg4MDQkJCsHXrVrRo0QKbNm1CZmYmLl++bIgYCSEG1lCK9FU3joYyRlJ79lJL/DX/FRz4ficlQ0QrnROhgIAA/Prrr8jKykJsbCyWLl2K4cOH06JpQkxYQynSV904GsoYiW7spBb4a/4rOPjDTiTG0VMRSGU6J0JLlizB0KFD6eGrhDQQDaVIX3Xj+HnvKfy856TJj5HUTXkydHj5n5QMkUpolSAhjVxDKdJX3Tju3ktBXEKKyY+R1J2t9dNkKOH6Pa7DIUaEEiFCGjFDFuljGAZ/nb4Ohqn9tbpew7IsbiSlQqlUaR1HM3sx2jspwVfJ4WUn0jpGpVKFG0mpeHYjbXkfz7HBlhgJW2sL/P3FRESt2E3JEFEzqWITJ0+exJIlSxAbG4vU1FTs3r0bI0eOrPaamJgYzJ49G/Hx8WjatCk+++wzTJkypV7iJcTYGbJI35cbDmP59ijMGtcP818dZJBrDp67gXlr92Li4HCt41AoVcgvUqBIzqLD/x7DUiKofCNhJn7adQqbD5ytVEOpvA9Tr61EythYmeOv+a9gxIKtwMwR8Gvry3VIhGMmlQgVFRWhXbt2mDZtGkaPHl1j+6SkJAwZMgRvvvkmtm7dimPHjuG1116Dm5sbBgwYUA8RE2Lc2vi645uZY2os0tfG112n+8rlSqz/6xRcLRis/+sUIib2g1hcQ8FAHa8pn81CaR6iL97E12+/UOXMlUKlQty9VIBFWdHEZ6tHAxAI+Fi1MwYozcO6vacwILQlBAK+Rh8VjxPTZmNljr++mIDhC7aCfXs4mrfz4zokwqFaJUL5+fm1vqFUKq25UR0NGjQIgwbV7jdLAFi9ejV8fHywdOlSAEDLli1x+vRp/PDDD5QIEYKyJKdvJ3+93zdycxRYRQkiepvh0+MliNwcVeMMj67XVKwJ9PWJNKhUjNYZmxHd2lbb94Gz8XiYmvrfvcrWDA0Ob/1MH0+PE9MntTTHX/MnoMecdZi75VOuwyEcqtWvNra2trCzs6v2Vd7GmJw7dw59+/bVODZgwACcO3dO6zUymQz5+fkaL5lcYehQCWkwymd2+jcTYGKQOfo1E2D9X6cgl2ufddL1Gn3WBNJ2L7lcSXWHGjippTn6dWiBmD9iuA6FcKhWiVB0dDSOHz9e7au8jTFJS0uDi4uLxjEXFxfk5+ejpKSkymsiIyNhY2Oj8YrcHFUf4RLSIJTP7MzsbAYAmBlqBlZRUu3fI12v0WdNIG33itwcRXWHGoHF0/qDuXUf0dujuQ6FcKRWH4316NHD0HEYjYiICMyePVvjmOTSOo6iIcS0VJzZCXIVAQDau4nUMzxVrfvR9Zqa6h7pso5H2726NuVh/V+nMLqV+Ln7IMaNx+Nhzbsj8eaKvTi+/Th6j+vNdUikntX5b3JxcTFu3bqFa9euabyMiaurK9LT0zWOpaenQyqVwtzcvMprJBIJpFKpxksiFtVHuISYvGdndspVN8Oj6zX6rHuk7V4BTiKwihK83FYzJpoVaph4PB5WvzMC/LuPcPw34/pkgxiezrvGnjx5gqlTp+LgwYNVnlepVM8dlL6EhYXhwIEDGseioqIQFhbGUUSmh2VZpGXlg9FjDRVrCwmkllUnosR0lc/s9PYRIMBRiFLl0++Zlk5C9PauPMOj6zVV1T0qV7EmUG1mbLTdS8WwOHSrEL19BLDkl0KmsAaPV7c+iOng8XhY9c5wvP3jXzi69Sj6Tuhb80WkQdA5EXrvvfeQm5uL8+fPo2fPnti9ezfS09Px1VdfqXdnGUphYSESEhLUXyclJeHKlSuwt7eHp6cnIiIi8OjRI/z6668AgDfffBM//vgj5syZg2nTpuH48eP4448/sH//foPGacpiLt3Fk9xC9ddRl5ORUCCCpZW13vpIvZ+Atwe2hrV52UcOzZs4IahFE73dn3Djj+OXUFJaghPJLFquyK2yTYlShT+OX8IrA0PqdI0+6x5pu1eRTIWs/BJYioAT9/MgtZRX3nJfx9pKxLjxeDysnDkcry/bjasnr6Jd93Zch0TqAY/VsVyqm5sb9u7di5CQEEilUly8eBEtWrTAX3/9hW+//RanT582VKyIiYlBr169Kh2fPHkyNm7ciClTpiA5ORkxMTEa17z//vu4ceMGmjRpgs8//1z3gopnVzxf4EasuFSOhZtjkFeiQE5+EXLErmjSOlR9XmrvjFaduuq3z8ICXD97VF2p9/bpv9HSSQzhf//Y9AnyxovdaYuyqSksLsXKXadRItO+y9JcIsLbo7vCysKsTtfIFUqcvJJYY92j7kG+EItqqFuk5V4KZVndIaWKgVDIL6s9JNBMhGrbBzFNl+88wHvbT+L1xa9DUEXdKWIaHMwd0a/ZkBrb6ZwISaVSXLt2Dd7e3vDy8sK2bdvQpUsXJCUloXXr1iguLq5z0EargSRCLMuqt/7uOBmPNQcuo1imQOcX34arjz94PD4cXD3AK/8coJ6olEpkZzxWf33mj5UofXIf5kIe1s0eBkcbS3WSRAgh9WHr8SvY+O8dTPlyKiVDJqq2iZDOv874+/vj9u3b8Pb2Rrt27bBmzRp4e3tj9erVcHNzq1OwxPAu3X6AJTvP4VEhH3w+H/YePhj31TYIhNz/RisQCuHk/vQjhpHvfQMAeJx8F+OXLUFRYR76tnbFgGAf+Hu6wNXBcEU7CSEEACb0DgIPwPrP1mPqV9MoGWrAdJ4R2rJlC5RKJaZMmYLY2FgMHDgQ2dnZEIvF2LhxI8aOHWuoWLljojNC99Oy8WvUNaQ8yUNSiQWC+r4A/+BuXIelM5Zlcen4XhRmpuLu+aMY3NEHrnaWeH1op3qfvSKENC4/7z+Ps3IGA6bQ0whMjcE+GntW+TZ6T09PODo6Ps+tjJeJJULFpXK8/b99iEvJRo9JH8Hc0hqeLQIbRNJQlJ+L9AdJSIk7j8RzB/D2iFBM6hfEdViEkAbqSU4BBi/ciunfvQmpPc1GmxKDJUILFy7Ehx9+CAsLC43jJSUlWLJkCebNm6dbpKbABBKhzNxCPMktxOr9sbhwLwddx7+L5kENu0wAwzA4tPYrPLx1CQtf6YKubZrB2tKs5gsJIUQH8fdSMeXHv/Dat2/Ahj6aNxkGS4QEAgFSU1Ph7OyscTwrKwvOzs5GVUdIb4w4EcrKK8L241ewKfoOPAPawdmvDdr3Hsl1WPWquLAAp/9YiUfx5/H6wLZ4uW972s1DCNGrG0mpeOOXI5ix4h2uQyG1ZLDF0izLVvkRy9WrV2Fvb6/r7chzeJJTgFEL/kDLQVMxYcHHsLBunL+pWFhZo/+0uchKf4xjZ6OwdvZ6jOvdFu+O6sx1aISQBqKVjxukfB4Srt+DX5tmXIdD9KjWM0J2dnbg8XjIy8uDVCrVSIZUKhUKCwvx5ptv4qeffjJYsJwxshmh7PwifLDmCG49LsCAtxbCo5k/1yEZldLiQpz5cx3uXz6JD18IwZDOATCT0GNSCCHPJ6+wBMMXbMGQD1+CZ4umXIdDaqD3j8Y2bdoElmUxbdo0LFu2DDY2NupzYrEY3t7eDffRFUaUCH219QQOXE1DyNBX0Dq8P9fhGDWFXIajm75DStwFbJ0zHP5eLlyHRAgxcUfO38S6W48w+t3RXIdCamCwNUInTpxAly5dIDSC+jP1xggSoeOXEvDzwcuQOwZgwLS5DWIHWH3Jy3qCHYveQu82TfDxuK6wsaLnnBFC6oZhGLy2bDfM2jRD9xd7cB0OqYZBt88nJiZiw4YNSExMxPLly+Hs7IyDBw/C09MTrVs3wEcjcJgIJTx8ghk/HYJcbIvek+fApakPJUF1UFpciIRrF3Ft94/Ys2AcJUOEkDpjGAavL98Ds3Z+6DpKv48gIvpT20RI50cnnzhxAm3atMH58+exa9cuFBaWPaDz6tWrmD9/vu6REq3uPsjAhG//Rp93l+Hlz1fD1bMZJUF1ZGZhhcDOPRE+5XMMXrgXU5fsBsMwXIdFCDFBfD4fa2eNxNk9p+jnSAOgcyL08ccf46uvvkJUVBTEYrH6eO/evfHPP//oNbjGqqCoFEv/OIUJS/bhxU9Wws7JleuQGgzvlu0wZdFmiFv1x9Qle+iHGCGkTvh8Pib3DMLx345zHQp5TjonQtevX8eoUaMqHXd2dkZmZqZegmrM8otKMPKL7bhj2xXj56+DrSMt8DWEjgNeglmbQQh/92es23+R63AIISbope6BuBZ9GYV5RVyHQp6DzomQra0tUlNTKx2/fPkyPDw89BJUY5VfVIKR87cjZMLH6Nh7KKxs7LgOqUHr2P9FTP7uT+y9q8TyXee4DocQYmLcHG2wavpgrPtoNVTKBlhMuJHQOREaN24c5s6di7S0NPB4PDAMgzNnzuDDDz/EpEmTDBFjg8cwDHbFXMXI+dsROvETNAsM5jqkRkMoFGHYjIU4msLHDzvPch0OIcTEBAc0RXNnWyTfSuE6FFJHOu8ak8vlmDFjBjZu3AiVSgWhUAiVSoWXX34ZGzduhEAgMFSs3DHgrjGGYfDa0r0ocGqHNt0Gw83L12B9Ee1YlsW+lV9AkpeEd0Z0Qvd2VDmWEFI7mbmFGPHlVgz/aBy8/KnQorEw+NPnU1JSEBcXh8LCQrRv3x7Nmzevy21Mg4ESIYZhMHXJHkha9UOnQeMM0gepPZZlkZ2Rir+Xf4yFY4PRJ9iP65AIISbi3qNMvLklGtMWvcp1KOQ/BnvWWDlPT080bVqW+dKWbt2pVAymLtkN8zYD0XHAS1yHQ1D2fezg4o7x89Zg3pdvgQWLvsENOMEnhOiNt5s9bBQK3Iq9jYBgeuyRKdF5jRAA/PLLLwgMDISZmRnMzMwQGBiIdevW6Tu2BkulYjBlyS6Ytx1MSZARkpiZY/znq/DFjquIuniX63AIISaAz+dj1rBQnNt7FnX8oIVwROdEaN68eZg1axaGDRuGHTt2YMeOHRg2bBjef/99zJs3zxAxNigqFYPJ3+6CZdsh6Nj/Ra7DIVqUJ0MLdl7FkQt3uA6HEGICurbzhZOKweOkyjurifHSeY2Qk5MT/ve//2H8+PEax3/77Te88847DbOWkJ7WCKlUDCYt/hPW7YchuN8LerknMSy5rBS/ffUWPh/VBgNCaLqbEFK9s3HJ+PiPk3h7+UyuQ2n0DPaIDYVCgY4dO1Y6HhwcDKVSqevtGg2VisHExTsh7TCckiATIpaYYfxnq/DVnjgcOn+b63AIIUYuPNAbliyL1PtpXIdCaknnRGjixIlYtWpVpeNr167FhAkT9BJUQ6NUqvDKNzthEzwKHfqO5jock5OV/hip9xO1vrLSHxu0f7HEDOM/XYVFe+Nx4Pwtg/ZFCDF9v7w7EruX7uQ6DFJLddo19ssvv+DIkSPo3LkzAOD8+fNISUnBpEmTMHv2bHW777//Xj9RmrCyJOhP2IeMRvveI7gOx+RkpT/Gik/egEKl/RNckYCHdxatgYOLu8HiEEkkGPfpSnyzaAZYFhjSOcBgfRFCTJu7kw3shXzc/PcmWoa05DocUgOdE6G4uDh06NABAJCYmAgAcHR0hKOjI+Li4tTtaEt9WRI0IXInHDu/iKBew7kOxyTJS0ugULGw7zYREvvKD5+VZach+9RmyEtLDB5LWTL0E77+4nU0d7dHC09ng/dJCDFNCyf0woe7z1IiZAJ0ToSio6MNEUeDtO7ARUgC+lASpAcSe1eYO3lyHQZEYgna9R2NXw4dxeLXB3IdDiHESLX184CnQIA7VxLQIoiKsxqzOtURIjV7klOAzcfj0LpLP65DIXrWofcIJAl8MG/DMa5DIYQYsSn92uPoxsNUV8jIUSJkABk5BRi14A/0ezsS9s6GW7dCuNNv8mzcZpvg8w1HuQ6FEGKkurbxgaOAjyePGmBZmQaEEiE9S8/OL0uCZnwDj2ZUd6Yh6ztpNq4U2OHguRtch0IIMVJLpvXHH5HbuA6DVIMSIT379o+zCJ/4Mdx9WnAdCqkH7fqOxo9/x6JUpuA6FEKIEfJt4gR7sRB3LtHjeoxVnR+6Sir7Jz4ZZ+9m4ZWJbbkOpcGRZVddnKwk8yFUSiWePE6p8rzYzNyg2+q9WgSieMgbeHHhGuycNw5mEpHB+iKEmKYVbw7BzN9PoUUHeoizMTK5ROinn37CkiVLkJaWhnbt2mHFihUICQmpsu3GjRsxdepUjWMSiQSlpaV6j+tcXBJmbzyHcZ+vgVhipvf7N1ZiM3OIBDxkn9pc6ZxKpUJBThZYlRzbfvoGAmHlJKQ+agy1DO0FAHh35XasfZ92CBJCNFmZS5DxMAMqpQoCoYDrcMgzTCoR+v333zF79mysXr0aoaGhWLZsGQYMGIDbt2/D2bnqmi5SqRS3bz99NIKh6ht98HM0xi/cCHNLa4Pcv7FycHHHO4vWVFkn6MnjFGxdsQh2oS/CyrNygcP6rDHk36kH1m5dhpS0bHi62hu8P0KI6XCwscToDn74N+oiwgaFch0OeYZJJULff/89pk+frp7lWb16Nfbv34/169fj448/rvIaHo8HV9fKhfj0adXe83Bp3paSIAOpbjZHKJbAyjOA8xpDfD4fL0b8iJcWzcK+hWPhaGvFaTyEEOPyYrdAvPzDbrTvEQQzCwnX4ZAKTGaxtFwuR2xsLPr27as+xufz0bdvX5w7d07rdYWFhfDy8kLTpk0xYsQIxMfHV9uPTCZDfn6+xksm174Q9sA/t7D3tgwDX/9c90GRBsXRrSmahw3E4Qt3uA6FEGJkfJs4IbiZG3Ke5HAdCnmGySRCmZmZUKlUcHFx0Tju4uKCtLSqF9L6+/tj/fr12Lt3L7Zs2QKGYRAeHo6HDx9q7ScyMhI2NjYar8jNUVW2VakYrD90GZ2GTgKfbzJvJTGg0GETsfJoIo7G0g4RQogmD3tr3L5wu+aGpF416H+9w8LCMGnSJAQFBaFHjx7YtWsXnJycsGbNGq3XREREIC8vT+MVMbHq6tCX7zyEzNYHni1aG2oIxMRIzMwx/vNV+GLLKa5DIYQYmY9f6o4r+89zHQZ5hskkQo6OjhAIBEhPT9c4np6eXus1QCKRCO3bt0dCQoLWNhKJBFKpVOMlEVfejZSdX4TZPx9F8KCXdRsIafAkZuZw8e+I9YcucR0KIcSICAR8mEmEKCky/AYOUnsmkwiJxWIEBwfj2LGnz3diGAbHjh1DWFhYre6hUqlw/fp1uLm5PXc803/Yhx6vLYCHLz1ZmGuy7DSUPEmp9NJWe6g+hI16DVuirkChVHEWAyHE+Hz1Sh/8Hrmd6zBIBSa1a2z27NmYPHkyOnbsiJCQECxbtgxFRUXqXWSTJk2Ch4cHIiMjAQALFy5E586d4efnh9zcXCxZsgT379/Ha6+99lxxxN17jNQCFQa2CHzuMTVUWemPq922XlWhw8unjiA/u/IzeYoKcqFUKGBlY4eWwV3Ux3OzMsDISpAWtbbKGkJAWR0hsZn5c8VVFzYOTmg15FWM//o3/P7ZSxAITOZ3DkKIAXVt44PC32K4DoNUYFKJ0NixY/HkyRPMmzcPaWlpCAoKwqFDh9QLqFNSUjQWLefk5GD69OlIS0uDnZ0dgoODcfbsWbRq1eq54vjxr1j0mfbJc92jIctKf4wVn7wBhUr7E5efLXR4+dQRrFk4GxA9s62ULf8fHlilDNZ2TpBYWAL4r6BiQQEYpQxWUnvwhZW/nVn+0+JldYnrebTrPhj3LhzHnQcZaOlt2BIOhBDT4SgR4ea/N9EyhD5RMAYmlQgBwMyZMzFz5swqz8XExGh8/cMPP+CHH37Qa/+xt1JwK4eH9r6VC/iRMvLSEihULOy7TYTEvnICUFWhw/zsTEAkgX3fNyCye5qEsColVMV5YBQy5J3ZBvseEyF19wUAKBVy8BKvo/Di33AZ8AbM7N2q7acucT2vzi+8iZkrP8Gxbyfr7Z6EENM294UuWHX1HiVCRsLkEiEusSyL1ftiEdT/FfAFVCa9JhJ7V50LHYrs3CF28VV/zSrlUOSmgVUpwOMLILZ9ek+FXAZhRip4AgHM7N1q3Vdd4qorN28/sOb2OH/jPkJbedVLn4QQQmqPFi7oIC0rHwn5ArQO7cl1KMSEDHs3Egu2nOQ6DEKIkfD1cMStc/HIzyngOhQCSoR08vaKAwgdPZ3rMIiJsbKxA2Prib/P3uQ6FEKIEXC2s8bo0JZ4eFd7cV9SfygRqqW7DzLwpIQP38COXIdCTFDI8KnYcPgyGIbhOhRCiBEw0PO/SR1QIlRLn/96AgPfXsB1GMRENfVrCaVDc1y4mcJ1KIQQIxDUzBXn9pwFy2rfxUrqBy2WroWzcUl4XCJCL3da7Fpb8tJiZF6LgdjavtK5/Ac3UJKdgQvH98HG3gkAcOfqv2BVKpTcvwZlQTYE/13HqhRgivPAKORgGRXkuWko+W/7vFIhhzI/E4xSgfykayjNTtWMIe8JVMrKD8zVVmjR0AUY2/d/CRHr5uH4d1MN2g8hxPgN6OSPH/aeg0qpglBE/xRzid79Wth05Co6j3qdHqxaS2kpiSjIykDB+b8qnWNVDMAowBOKceCPTU/nh1kWPD4f+ef/BKuUAXwReOr3+2kdoayYTSiytAYAKORyFOSUJUIPjm4Ej195Jx9PJUdxYT6AsmKJIgEP2ac2a4392QKM+tTErxV41i6Iu/cYgc2ev04RIcS0CWj3sVGgRKgW7pVY4qV2nbgOw2QIhCLwJRaw7zNdoyYQAMhz0pBz8ldYBw+F2LkZ+P8VUGRVSqgKc8DIipF3dhvsWnWDmZ0zAEBZUgBFQTaKEy9g8uwv4ORetvX9yeMUbF7+FWxCRsGqiX+lOGQ56cg5vRUWVlIAgIOLO95ZtKZeKktrM2TGQsxa/BaOLaG6QoQ0dgFNHHHh8AWEDa3dY6KIYVAiVAvtB4zlOgSTJLb3gMSlmcYxnkAIHl8AoZUDxE5e4EvKPuZilXIoxOZl9YJEZrAL7A47r6fFxkqepCA1MxFO7p5w83paZ0hsZg5bnzZV1gUSisTIf+Y3LkMmObVhbesAmVIFlmXBo9WShDRq304bgO4R6ykR4hh91lMLzYPom5Toj2dwH/xv9z9ch0EI4ZhAwKf1QUaAEiFC6llw/5ew82Q8ikvlXIdCCCGNHiVChNQzqb0jvMKH4/B5KrBISGOnVCi5DqHRo0SIEA6YWdvicXYh12EQQjjWu7UXYv6I4TqMRo0+nCQastIf67SrKjH+MoryczXaJN+6BkapRGHSFciyHwN4uihYWZAJlawQxUmXIMtMUT+8lmUYKAuywMiKoMjPwqPorXhiaQMAKMm4D1VpAZT5mdi3aQUsrcuOFxXkIS/jEZTHNsOpfd9KsVZVR0jX8RlKh+6D8cvHWzGhb3vYWlsYvD9CiHGaPiAY7+w8w3UYjRolQkQtK/0xVnzyBhQq7ZVORQIe3lm0Bg4u7kiMv4zvP5wKViDWaKMsLQIPQP75Pytdzyjl4LEMSu6cq+LuZfWCeKwSxQ/iUaKuC1R2HHw+Lp6JeaZeEB+5t84h5/Z5CCWV6/9UrCOk6/gMSSSRwLWpN4pK5ZQIEUIIhygRImry0hIoVCzsu02ExN610nlZdhqyT21Wz6gU5eeCFYhh3+d1iO2fJg4FN0+h4Oph2PWcBoGdCwCeek6o9NFNFFzYA7ver0Fo5w4er+zTWVYlB1OcD0YpR96ZbbDpMh4iew+wKiWYolz1cWnYWIgdm6r7UmQ/Rs7RNbAN6Ay3sJGa8T5TR0jX8Rmaf5dBmPPzFmz95MV66Y8QQkhllAiRSiT2rlXW5dFGbO+uUS+o5NEtADyInL0gdvIpq5fzX80cprQQPB4PQltXSJx9wBOUfQsy8lIo8zPK6gjxBRDZe0Di4gtGIYNSnP70uJ07JC7/1RGq8IweibU9rN39NOKqqo5QXcZnKC1D++Dy3xu4DoMQQho1WixNCId4Eiuci0viOgxCCGm0KBEihEND3/kaS3b+y3UYhBDSaFEiRAiHzMytkJadB4ZhuA6FEEIaJUqECOGQSCKBT9hQbDwUy3UohBAOWFlI8DDhIUqLZVyH0mjRYmlSiSw7Tafj8uzHGl8r858AKNvRVb7zvZwiNw0sy0KZm1a2Y4z/364xpQJMcR4YpRwso4Ii+1HZ8Qq7xlhGBUXOY/AqLIBW/Ne3sjgPJU9Snmsc2o4bmrNPK+Q/pHVChDRG9lJLvNqjLS7FXEb44M5ch9MoUSJE1MRm5hAJeMg+tbnSOZVKBbAshDwgNzMDAFBSVAgo5cg+ulYj2VHJZQCjRM7RNZXuw6iU4DFK5Bz7+Zkz5TvAeGCVMmQfWQ0en1/hXNnxnKNrnqkjBLCKUhTdvYDUzHuV+hMJeBCbmdc4vqra1xc+n48neUX12ichxHg4WJuDpY/HOUOJEFFzcHHHO4vWVKqjk5uVgS0/zIeSYaHk87Flxdfqc5a29mBVKggEPAyf/A6sbe0BAHHnT6AoPxciiTnsnZ/W7MnOSMPZQ3+CZQGeoMIns2zZ7A94AFQqQMCA/a/GEFhV2X8ZBu27dYeNnZP6stLiIlja2KH7sHFVjqlipWht49PWvr40b9sJW3atRcLDJ/Br4lTzBYQQQvSGEiGiQWsSIBDDuWf1hQibtQqCm1dZjZ+2Yb2qvM21c9H499SxSkUYGZUCioIcdeFEu24TILH3UJ+XZz9G9rG16DJojNZ7P9f4OCQQCtGkRSByC+qnkCMhhJCnKBEitabPQoTPFmFkFDKwQjN14USJvYfG+QaPx0dhCS2WJISQ+ka7xggxAu0HjMXc9dFch0EIIY0OJUKEGAF7Z3eIrWy4DoMQQhodSoQIMRIsy0KhVHEdBiGENCqUCBFiJIIHv4IZK/ZzHQYhpJ49yMoHX0hLdrlC7zypNX0WIny2CCOjUkD5364xllFB9l9BRW3tG6LALgPwx8FNXIdBCKlHmbmF+P3fO5g1dTDXoTRaJpcI/fTTT1iyZAnS0tLQrl07rFixAiEhIVrb79ixA59//jmSk5PRvHlzLF68GIMH0zecLvRZiNBSagueSo7sY2s1T7AsGEaFpwUVV4En0Pz25KnksJTa1mEEhBBinIpL5XD3cYNQZHL/HDcYJvXO//7775g9ezZWr16N0NBQLFu2DAMGDMDt27fh7Oxcqf3Zs2cxfvx4REZGYujQodi2bRtGjhyJS5cuITAwkIMRmCZ9FiL0bd0es7/bgKL83ErnUlMSISsugsTcUl2PqCJLqS18W7fXKXZCCCGkOjyWZdmamxmH0NBQdOrUCT/++CMAgGEYNG3aFO+88w4+/vjjSu3Hjh2LoqIi7Nu3T32sc+fOCAoKwurVq2vd788nKz+6gRBDWDNrJI5+MwG21hZch0IIqQcpadl4Z+cZjP90AtehNDgO5o7o12xIje1MZrG0XC5HbGws+vbtqz7G5/PRt29fnDt3rsprzp07p9EeAAYMGKC1PQDIZDLk5+drvBRyKnRH6kfXl9/D4t/PcB0GIYQ0GiaTCGVmZkKlUsHFxUXjuIuLC9LSql6sm5aWplN7AIiMjISNjY3G6+DW2s8eEfI87Fw8IFPSwxcJIaS+mEwiVF8iIiKQl5en8Ro04U2uwyKEEEKIAZjMYmlHR0cIBAKkp6drHE9PT4era+UHgQKAq6urTu0BQCKRQCKRaBwTiTPrGDUhhBCi3aHYBEidbLkOo1EzmRkhsViM4OBgHDt2TH2MYRgcO3YMYWFhVV4TFham0R4AoqKitLYnhBBC6tMvRy9j0KuDuA6jUTOZGSEAmD17NiZPnoyOHTsiJCQEy5YtQ1FREaZOnQoAmDRpEjw8PBAZGQkAmDVrFnr06IGlS5diyJAh2L59Oy5evIi1a9dW1w0hhBBSL8wsJOALTGZOokEyqURo7NixePLkCebNm4e0tDQEBQXh0KFD6gXRKSkp4POffkOFh4dj27Zt+Oyzz/DJJ5+gefPm2LNnD9UQIkbLzMISpxMfQ6lUQSgUcB0OIYQ0eCZVR4grVEeI1KejW1ZgUosSDAlvzXUohBAD6xaxHjNWvsd1GA1Sg6sjREhjYWHnCKWKnkJPSEO3ZOcp+HRowXUYjR4lQoQQQggHYq4no//k/lyH0ehRIkQIIYSQRosSIUIIIYQDMrmC6xAIKBEihBBC6t2eM/Ewc3eEUGRSm7cbJEqECCGEkHp2+2EmOvQP5joMAkqECCGEkHqnYujhysaCEiFCCCGkHj16kos9F+7Au6UX16EQUCJECCGE1KuU9BwEdG4FC2sLrkMhoESIEEIIqVf0QAfjQokQIYQQUo++2BYD/9CWXIdB/kOJECGEEFKPivl8+LZtxnUY5D+UCBFCCCH1ZP/5W7Bztec6DFIBJUKEEEJIPfn2z9MY89FLXIdBKqBEiBBCCKkHcoUSJTI5xBIx16GQCigRIoQQQurBl9ui0WVsb67DIM+gRIgQQggxsIycAhy+nIDALq25DoU8gxIhQoyMrKgAfB6P6zAIIXp08koCOgwLg6XUkutQyDMoESLEiORmpuPxxcPo1ymA61AIIXp0NTmDKkkbKUqECDEixYX56NDCA2YSEdehEEL05ElOAY7deYROfelp88aIEiFCCCHEgOZsOIIuo7uBRx95GyVKhAghhBADuvckH8E0G2S0KBEihBBCDOTq3UeAWMh1GKQalAgRQgghBvLZ1uMY+8nLXIdBqkGJECFGpCA3G0I+rSMgpCE4cvEOSs3NYOdky3UopBqUCBFiRE5sWowPx4RxHQYh5DmxLIv1R2LRa0IfWiRt5CgRIsSI2FqawdnOmuswCCHP6fKdhyiytkSz1t5ch0JqQIkQIYQQomdzN0ah8/BwrsMgtUCJECGEEKJHdx9kQGEugW/bZlyHQmqBEiFCjMTdy2fhamvOdRiEkOf09qr9eGnuOK7DILVExQ0IMRJnd6zGkYWjuA6DEPIcjsTeBWtlDntnO65DIbVkMjNC2dnZmDBhAqRSKWxtbfHqq6+isLCw2mt69uwJHo+n8XrzzTfrKWJCdMPjAxZmYq7DIIQ8hzUH/sWwGSO5DoPowGRmhCZMmIDU1FRERUVBoVBg6tSpeP3117Ft27Zqr5s+fToWLlyo/trCgp7+S4xPfk4mivNyuQ6DEPIcjsfegcLBBi6ezlyHQnRgEonQzZs3cejQIVy4cAEdO3YEAKxYsQKDBw/Gd999B3d3d63XWlhYwNXVtb5CJaROLuzbgsgp3bkOgxBSR6UyBb7bfRZdXh/KdShERybx0di5c+dga2urToIAoG/fvuDz+Th//ny1127duhWOjo4IDAxEREQEiouLq20vk8mQn5+v8VLIZXoZByFasSrYSy25joIQUkcTl+5E+7G94NuGdoqZGpNIhNLS0uDsrDnVKBQKYW9vj7S0NK3Xvfzyy9iyZQuio6MRERGBzZs345VXXqm2r8jISNjY2Gi8Dm5drZdxEFIVhmGQmnQHVhYSrkMhhNTBg/QcPMovRpuubbgOhdQBpx+Nffzxx1i8eHG1bW7evFnn+7/++uvq/9+mTRu4ubmhT58+SExMhK+vb5XXREREYPbs2RrHtvz7qM4xEFKTu9cuoKO7EAFeLlyHQgipgzkbjmD0hy9xHQapI04ToQ8++ABTpkyptk2zZs3g6uqKjIwMjeNKpRLZ2dk6rf8JDQ0FACQkJGhNhCQSCSQSzd/MReLMWvdBiK5USiW8nGy4DoMQUgfHLyUgpaAEw7zduA6F1BGniZCTkxOcnJxqbBcWFobc3FzExsYiODgYAHD8+HEwDKNObmrjypUrAAA3N/qGJcYj+3ES/MUmsW+BEFLBg/QcRPwWjbeWzQRfYBIrTUgVTOJPrmXLlhg4cCCmT5+Of//9F2fOnMHMmTMxbtw49Y6xR48eISAgAP/++y8AIDExEV9++SViY2ORnJyMv/76C5MmTUL37t3Rtm1bLodDiJpCJsOtY79j2qBgrkMhhOho/ZFYBPfvCDNa32fSTCIRAsp2fwUEBKBPnz4YPHgwunbtirVr16rPKxQK3L59W70rTCwW4+jRo+jfvz8CAgLwwQcf4IUXXsDff//N1RAIqUQuL4W7kx3EIpoRIsSU/LD7DK7JlOj2ApW9MHU8lmVZroMwdj+fvMd1CKSB2rF4Fj4a4IN+HZtzHQohpJYUShXC3luFD36NAI/H4zocooWDuSP6NRtSYzuTmREipCFSFGRREkSICVEoVRgXuR09J/WnJKiBoESIEI4kXjsPOzP6QUqIKTny7y2YtWiKjv061tyYmARKhAjhyPWjO7H8rQFch0EIqaX07Hx8teMUOg3sxHUoRI8oESKEAyqlEk8eP4ClOT1tnhBTkJ6dj5FfbsOLn74CV08qftqQUCJECAdiY/ZjTGdPer4YISbio/VHMGLOWHj4an/INzFNlAgRwgFZcQF8XW25DoMQUgvnrifhzpM8NPVrwnUoxAAoESKknhXkZiMh5k/06diC61AIITU4ez0J7285jte/exMCoYDrcIgBUCJESD27fnI/hoe1gI2VOdehEEJqMGPNfryx9E1Y0N/XBosSoVpIvnmZ6xBIA3IrZjfmjO3KdRiEkBr8uOcs/DsFwNySkqCGjBKhWrh86DeuQyANRElRAcwkQvD59FePEGO2fPcZHH6UhRHvjOI6FGJg9NO4FuwVGbgXT7NC5Pnt++lzLJpMzyYixJgt230Gx1Jz8HLEy1Q9uhGgRKgWpg1sjwt/bQA9lo08j/QHSShKv4+wQB+uQyGEaLF89xnEpOVi/MfjKQlqJCgRqoXeHfwgVeWguDCf61CICbt06DcsntaL6zAIIVqoVAy2nryOsXPHURLUiFAiVEu2VmZIf5DEdRjERD1OvgvZg2sIbe3NdSiEkCoolSq8vPgPhL/YnZKgRoYSoVr67vV+iNkQyXUYxERdOvQbpg4IgojqkBBidJRKFSYs/gPuPdqh8+DOXIdD6hklQrVkL7VEYFMbXDq6i+tQiIkpKSpAUcp1vNijDdehEEKeoVSq8PI3f8C9V3uEDg7lOhzCAUqEdLB61jDkxu7FncvnuA6FmJB9P36Gj8d24ToMQsgzlEoVxn3zO5r27YDQQSFch0M4QomQDgQCPt4d0QlXjvxBO8hIrWSlPUJp1kP0DW7OdSiEkAoUShXGRm6HV7+O6DSgE9fhEA5RIqSjsEAfNBEX4HHSHa5DISbg9O8/4n9v9uc6DEJIBQqlCmMXbYfPgE7o1L8j1+EQjlEiVAdvDOmAYxu+4ToMYuRux56C6sk9tG7mxnUohJD/KJQqvLToN/gODEHHfpQEEUqE6qRzKy/YCmS0nZ5oVZCbhfO/fY9dC8bRTjFCjMhnvx6Fd9+OCO4XzHUoxEhQIlRHa98bigPLPkD6Q0qGSGUXD/yGl3oEwlwi5joUQsh/0rLycfxKItr1aMd1KMSIUCJUR26ONlg6vTdi92/lOhRiZJRKBZIvHsPMkbQVlxBjkZqZh5FfbsO4+ZNgZiHhOhxiRCgReg4dAzyheByHR/ducx0KMRJKhRy/R76LD1/sTNVpCTECMrkCu2KuYtRXv+GleRPh0cyd65CIkaFE6DmIhAJMGxCES4e303Z6AgD45+8tmNDJCaO7B3IdCiGNnkyuwItf/Ya9OUV4+cupcPehjQukMkqEntPo7m3Qya4Qxzd/z3UohGO5WRm4deYAhoW15DoUQho9mVyBF776DYGju2HQlIFwbuLEdUjESPFYmsqo2Y29NTZ598f9GPTWgnoIhhirrUvm4vOxneHv48F1KIQ0et9tPQC2uSc69aKK0Y2VjcQW7V1rUSyTJdUqLS1l58+fz5aWlnIdSr1rrGNvrONmWRo7jb1xjb2xjptlG/fYn0UzQjXIz8+HjY0N8vLyIJVKuQ6nXjXWsTfWcQM0dhp74xp7Yx030LjH/ixaI0QIIYSQRosSIUIIIYQ0WpQIEUIIIaTRokSoBhKJBPPnz4dE0vgqkTbWsTfWcQM0dhp74xp7Yx030LjH/ixaLE0IIYSQRotmhAghhBDSaFEiRAghhJBGixIhQgghhDRalAgRQgghpNGiRKgaP/30E7y9vWFmZobQ0FD8+++/XIdUL06ePIlhw4bB3d0dPB4Pe/bs4TqkehEZGYlOnTrB2toazs7OGDlyJG7fvs11WPVi1apVaNu2LaRSKaRSKcLCwnDw4EGuw6p333zzDXg8Ht577z2uQzG4L774AjweT+MVEBDAdVj15tGjR3jllVfg4OAAc3NztGnTBhcvXuQ6LIPz9vau9OfO4/EwY8YMrkPjDCVCWvz++++YPXs25s+fj0uXLqFdu3YYMGAAMjIyuA7N4IqKitCuXTv89NNPXIdSr06cOIEZM2bgn3/+QVRUFBQKBfr374+ioiKuQzO4Jk2a4JtvvkFsbCwuXryI3r17Y8SIEYiPj+c6tHpz4cIFrFmzBm3btuU6lHrTunVrpKamql+nT5/mOqR6kZOTgy5dukAkEuHgwYO4ceMGli5dCjs7O65DM7gLFy5o/JlHRUUBAMaMGcNxZBzi9lFnxiskJISdMWOG+muVSsW6u7uzkZGRHEZV/wCwu3fv5joMTmRkZLAA2BMnTnAdCifs7OzYdevWcR1GvSgoKGCbN2/ORkVFsT169GBnzZrFdUgGN3/+fLZdu3Zch8GJuXPnsl27duU6DKMwa9Ys1tfXl2UYhutQOEMzQlWQy+WIjY1F37591cf4fD769u2Lc+fOcRgZqU95eXkAAHt7e44jqV8qlQrbt29HUVERwsLCuA6nXsyYMQNDhgzR+DvfGNy9exfu7u5o1qwZJkyYgJSUFK5Dqhd//fUXOnbsiDFjxsDZ2Rnt27fHzz//zHVY9U4ul2PLli2YNm0aeDwe1+FwhhKhKmRmZkKlUsHFxUXjuIuLC9LS0jiKitQnhmHw3nvvoUuXLggMDOQ6nHpx/fp1WFlZQSKR4M0338Tu3bvRqlUrrsMyuO3bt+PSpUuIjIzkOpR6FRoaio0bN+LQoUNYtWoVkpKS0K1bNxQUFHAdmsHdu3cPq1atQvPmzXH48GG89dZbePfdd7Fp0yauQ6tXe/bsQW5uLqZMmcJ1KJwSch0AIcZoxowZiIuLazRrJgDA398fV65cQV5eHnbu3InJkyfjxIkTDToZevDgAWbNmoWoqCiYmZlxHU69GjRokPr/t23bFqGhofDy8sIff/yBV199lcPIDI9hGHTs2BGLFi0CALRv3x5xcXFYvXo1Jk+ezHF09eeXX37BoEGD4O7uznUonKIZoSo4OjpCIBAgPT1d43h6ejpcXV05iorUl5kzZ2Lfvn2Ijo5GkyZNuA6n3ojFYvj5+SE4OBiRkZFo164dli9fznVYBhUbG4uMjAx06NABQqEQQqEQJ06cwP/+9z8IhUKoVCquQ6w3tra2aNGiBRISErgOxeDc3NwqJfgtW7ZsNB8NAsD9+/dx9OhRvPbaa1yHwjlKhKogFosRHByMY8eOqY8xDINjx441mjUTjRHLspg5cyZ2796N48ePw8fHh+uQOMUwDGQyGddhGFSfPn1w/fp1XLlyRf3q2LEjJkyYgCtXrkAgEHAdYr0pLCxEYmIi3NzcuA7F4Lp06VKpNMadO3fg5eXFUUT1b8OGDXB2dsaQIUO4DoVz9NGYFrNnz8bkyZPRsWNHhISEYNmyZSgqKsLUqVO5Ds3gCgsLNX4rTEpKwpUrV2Bvbw9PT08OIzOsGTNmYNu2bdi7dy+sra3V68FsbGxgbm7OcXSGFRERgUGDBsHT0xMFBQXYtm0bYmJicPjwYa5DMyhra+tKa8AsLS3h4ODQ4NeGffjhhxg2bBi8vLzw+PFjzJ8/HwKBAOPHj+c6NIN7//33ER4ejkWLFuGll17Cv//+i7Vr12Lt2rVch1YvGIbBhg0bMHnyZAiFlAbQ9vlqrFixgvX09GTFYjEbEhLC/vPPP1yHVC+io6NZAJVekydP5jo0g6pqzADYDRs2cB2awU2bNo318vJixWIx6+TkxPbp04c9cuQI12FxorFsnx87dizr5ubGisVi1sPDgx07diybkJDAdVj15u+//2YDAwNZiUTCBgQEsGvXruU6pHpz+PBhFgB7+/ZtrkMxCjyWZVluUjBCCCGEEG7RGiFCCCGENFqUCBFCCCGk0aJEiBBCCCGNFiVChBBCCGm0KBEihBBCSKNFiRAhhBBCGi1KhAghhBDSaFEiRAgxKjExMeDxeMjNzdXahsfjYc+ePfUWU3W++OILBAUF1enaiRMnqh/8aSjjxo3D0qVLDdoHIaaMEiFCiEFs3LgRtra2XIehV/pMwK5evYoDBw7g3Xff1cv9tPnss8/w9ddfIy8vz6D9EGKqKBEihBAOrFixAmPGjIGVlZVB+wkMDISvry+2bNli0H4IMVWUCBFCKunZsydmzpyJmTNnwsbGBo6Ojvj8889R8Yk8MpkMH374ITw8PGBpaYnQ0FDExMQAKPt4a+rUqcjLywOPxwOPx8MXX3wBANi8eTM6duwIa2truLq64uWXX0ZGRsZzxfvgwQO89NJLsLW1hb29PUaMGIHk5GT1+f+3d+8hTbZvHMC/Yx6aOKXaEqGtFeZcYaSGpZBGShNC0kiMhCmlFhkJnaT+KfAPJawsAykKT8xDJ0co4SlddtwS0yITnZpYC7GkPB/m/f4RPr3P63x/qcn7q10fEPbc93Nf97X7r4vnfnYbFxeHiIgIZGRkwN3dHcuXL0dSUhImJia4e8xmM3bu3AmRSITVq1ejsLAQCoUCmZmZAACFQgEAiIyMhEAg4K6nFRQUQKFQwNXVFXv37sXAwMCs+VosFty5cwfh4eG89rGxMaSkpEAmk8HR0REeHh64efMmgB9bhhUVFfDx8YFIJML27dvR29uLBw8eQKVSwcXFBfv27cPw8DAvbnh4OIqLi+e4qoTYBiqECCFW5eXlwc7ODgaDAZcvX8bFixdx48YNrv/IkSN49uwZiouL0dzcjKioKISFhaGtrQ2BgYHIzMyEi4sLzGYzzGYzTpw4AQCYmJhAamoqmpqaoNPp0NXVhbi4uHnnOTExAbVaDbFYjPr6ejx58gTOzs4ICwvD+Pg4d19tbS1MJhNqa2uRl5eH3Nxc5Obmcv0ajQYfP35EXV0d7t69i+vXr/MKNKPRCADIycmB2WzmrgHAZDJBp9OhrKwMZWVl0Ov1SE9PnzXn5uZmfP36FZs2beK1azQaFBUV4cqVK2hpacG1a9dmPDE6d+4crl69iqdPn3IFYGZmJgoLC1FeXo7KykpkZWXxxvj7+8NgMGBsbOznF5YQW/Ef/9NXQsj/oeDgYKZSqdjU1BTXlpKSwlQqFWOMsffv3zOhUMg+fPjAGxcSEsJOnz7NGGMsJyeHubq6/s+5jEYjA8AGBgYYY4zV1tYyAKy/v3/WMQBYaWkpY4yxgoICplQqebmOjY0xkUjEKioqGGOMxcbGslWrVrHJyUnunqioKBYdHc0YY6ylpYUBYEajketva2tjANilS5eszjvt7NmzzMnJiX379o1rO3nyJNu8efOs+ZeWljKhUMjLubW1lQFgVVVVVsdMr0t1dTXXlpaWxgAwk8nEtR08eJCp1Wre2KamJgaAdXV1zZoTIbaKnggRQqzasmULBAIBdx0QEIC2tjZYLBa8fv0aFosFnp6ecHZ25v70ej1MJtO/xm1oaEB4eDjkcjnEYjGCg4MBAN3d3fPKs6mpCe3t7RCLxVwey5Ytw+joKC+X9evXQygUctfu7u7cE5/W1lbY2dnB19eX6/fw8MDSpUt/KgeFQgGxWGw1tjUjIyNwdHTkre+rV68gFAq59ZjNhg0buM9ubm5wcnLCmjVreG3/nFskEgHAjC0zQghg918nQAj5/QwODkIoFKKhoYFXXAD415d/h4aGoFaroVarodVqIZVK0d3dDbVazdvGmmsufn5+0Gq1M/qkUin32d7entcnEAgwNTU1rzn/aa6xJRIJhoeHMT4+DgcHBwA/ipW5zCUQCH5q7i9fvgDgrwch5DsqhAghVr148YJ3/fz5c6xduxZCoRA+Pj6wWCzo7e3F1q1brY53cHCAxWLhtb179w6fP39Geno6ZDIZAODly5cLytPX1xclJSVYsWIFXFxc5hVDqVRicnISjY2N8PPzAwC0t7ejv7+fd5+9vf2M7zQf0+cOvX37lvvs7e2Nqakp6PV6hIaGLniOv3vz5g1WrlwJiUTyS+MS8iegrTFCiFXd3d04duwYWltbUVRUhKysLCQnJwMAPD09ERMTA41Gg3v37qGzsxMGgwFpaWkoLy8H8H27aHBwEDU1Nejr68Pw8DDkcjkcHByQlZWFjo4O3L9/H6mpqQvKMyYmBhKJBLt27UJ9fT06OztRV1eHo0ePoqen56dieHl5ITQ0FImJiTAYDGhsbERiYiJEIhFv+0qhUKCmpgafPn2aUSTNhVQqha+vLx4/fsyLHRsbi/3790On03Hf49atW/OeZ1p9fT127Nix4DiE/ImoECKEWKXRaDAyMgJ/f38kJSUhOTkZiYmJXH9OTg40Gg2OHz8OpVKJiIgIGI1GyOVyAEBgYCAOHTqE6OhoSKVSnD9/HlKpFLm5ubh9+zbWrVuH9PR0ZGRkLChPJycnPHr0CHK5HLt374ZKpcKBAwcwOjo6pydE+fn5cHNzQ1BQECIjI5GQkACxWIwlS5Zw91y4cAFVVVWQyWTw8fFZUN7x8fEztvOys7OxZ88eHD58GF5eXkhISMDQ0NCC5hkdHYVOp0NCQsKC4hDypxIw9reDQQghBN/PEdq4cSN3ho4t6unpgUwmQ3V1NUJCQn55/JGRESiVSpSUlCAgIOCXx5+WnZ2N0tJSVFZWLtochPzO6B0hQggB8PDhQwwODsLb2xtmsxmnTp2CQqFAUFDQoswnEomQn5+Pvr6+RYk/zd7efsa5QoSQH6gQIoQQfD+Y8cyZM+jo6IBYLEZgYCC0Wu2MX2X9Stu2bVu02NPi4+MXfQ5Cfme0NUYIIYQQm0UvSxNCCCHEZlEhRAghhBCbRYUQIYQQQmwWFUKEEEIIsVlUCBFCCCHEZlEhRAghhBCbRYUQIYQQQmwWFUKEEEIIsVlUCBFCCCHEZv0F9MEuP12i+0IAAAAASUVORK5CYII=\n" 393 | }, 394 | "metadata": {} 395 | } 396 | ] 397 | }, 398 | { 399 | "cell_type": "markdown", 400 | "source": [ 401 | "### **Преимущества и недостатки наивного байесовского классификатора**\n", 402 | "Преимущества:\n", 403 | "- простота в реализации и интерпретации;\n", 404 | "- практически не требуется настройка параметров;\n", 405 | "- высокая скорость работы и точность прогнозов во многих ситуациях;\n", 406 | "- имеет относительно хорошую устойчивость к шуму и выбросам, поскольку основан на вероятностных распределениях и наивном предположении о независимости признаков.\n", 407 | "\n", 408 | "Недостатки:\n", 409 | "- в случае нарушения предположения о независимости признаков, точность прогнозов может значительно снизиться;\n", 410 | "- может отдавать предпочтение к классам с бОльшим количеством образцов в случае несбалансированных данных.\n" 411 | ], 412 | "metadata": { 413 | "id": "XxWeyVZhk5og" 414 | } 415 | }, 416 | { 417 | "cell_type": "markdown", 418 | "source": [ 419 | "### **Дополнительные источники**\n", 420 | "Статья «Bayes and Naive-Bayes Classifier», Rajiv Gandhi, Andhra Pradesh.\n", 421 | "\n", 422 | "Документация:\n", 423 | "- [описание наивного Байеса](https://scikit-learn.org/stable/modules/naive_bayes.html);\n", 424 | "- [GaussianNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.GaussianNB.html#sklearn.naive_bayes.GaussianNB);\n", 425 | "- [MultinomialNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html#sklearn.naive_bayes.MultinomialNB);\n", 426 | "- [ComplementNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.ComplementNB.html#sklearn.naive_bayes.ComplementNB);\n", 427 | "- [BernoulliNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.BernoulliNB.html#sklearn.naive_bayes.BernoulliNB);\n", 428 | "- [CategoricalNB](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.CategoricalNB.html#sklearn.naive_bayes.CategoricalNB).\n", 429 | "\n", 430 | "Видео: [один](https://www.youtube.com/watch?v=O2L2Uv9pdDA), [два](https://www.youtube.com/watch?v=H3EjCKtlVog), [три](https://www.youtube.com/watch?v=nt63k3bfXS0), [четыре](https://www.youtube.com/watch?v=ADj95edZc0w).\n", 431 | "\n" 432 | ], 433 | "metadata": { 434 | "id": "mtO8wzUrl5JN" 435 | } 436 | } 437 | ] 438 | } -------------------------------------------------------------------------------- /Notebooks [rus]/ML-алгоритмы с нуля/12) Метод главных компонент (PCA).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [] 7 | }, 8 | "kernelspec": { 9 | "name": "python3", 10 | "display_name": "Python 3" 11 | }, 12 | "language_info": { 13 | "name": "python" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "markdown", 19 | "source": [ 20 | "## **Метод главных компонент**\n", 21 | "Метод главных компонент (Principal Component Analysis или же PCA) — алгоритм обучения без учителя, используемый для понижения размерности и выявления наиболее информативных признаков в данных. Его суть заключается в предположении о линейности отношений данных и их проекции на подпространство ортогональных векторов, в которых дисперсия будет максимальной. Такие вектора называются главными компонентами и они определяют направления наибольшей изменчивости (информативности) данных. Альтернативно суть PCA можно определить как линейное проецирование, минимизирующее среднеквадратичное расстояние между исходными точками и их проекциями." 22 | ], 23 | "metadata": { 24 | "id": "GEUI9KeXiFd1" 25 | } 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "source": [ 30 | "### **Принцип работы PCA**\n", 31 | "Изначально матрица признаков обязательно центрируется, чтобы первая главная компонента могла соответствовать направлению максимальной вариации данных, а не просто их среднему значению. Обычно нахождение главных компонент сводится к двум основным методам:\n", 32 | " - **Вычисление собственных векторов и собственных значений ковариационной матрицы данных**. Поскольку ковариационная матрица отражает степень линейной связи между различными переменными, то собственные вектора этой матрицы будут задавать направления, вдоль которых дисперсия данных максимальна, а собственные значения — величину этой дисперсии. Собственное значение, соответствующее собственному вектору, характеризует вклад этого вектора в объяснение дисперсии данных и чем больше собственное значение, тем значимее главная компонента. Обычно отбираются только те главные компоненты, которые объясняют заданный уровень дисперсии, например, 95%.\n", 33 | "\n", 34 | " - **Вычисление сингулярного разложения матрицы данных**. Сингулярное разложение — это способ представления любой матрицы в виде произведения трёх других матриц: левой сингулярной матрицы U, диагональной матрицы сингулярных значений S и правой сингулярной матрицы V, где сингулярные значения — это квадратные корни собственных значений ковариационной матрицы данных (именно для этого в данном случае выполняется предварительное центрирование данных), правая сингулярная матрица V будет соответствовать собственным векторам ковариационной матрицы данных, а левая U будет являться проекцией исходных данных на главные компоненты, определённые матрицей V. Таким образом, сингулярное разложение также позволяет выделить главные компоненты, но без необходимости в вычислении ковариационной матрицы. Помимо того, что такое решение более эффективно, оно считается более численно стабильным, поскольку не требует вычисления ковариационной матрицы напрямую, которая может быть плохо обусловлена в случае сильной корреляции признаков. Именно данный подход используется в реализации scikit-learn, но с некоторыми особенностями, рассмотренными ниже.\n", 35 | "\n", 36 | "**PCA на основе SVD строится следующим образом:**\n", 37 | "- 1) сначала происходит центрирование данных, а также определяется число компонент как минимум между числом образцов и признаков в случае, если число компонент не было задано;\n", 38 | "- 2) далее SVD применяется к центрированной матрице данных;\n", 39 | "- 3) к матрице U применяется метод svd_flip_vector, который находит максимальные по модулю элементы в каждом столбце матрицы U, извлекает их знаки и умножает матрицу U на эти знаки, чтобы гарантировать детерминированный вывод;\n", 40 | "- 4) объяснённая дисперсия для каждой главной компоненты вычисляется как возведённые в квадрат соответствующие сингулярные значения, разделённые на n_samples - 1, а преобразованные данные вычисляются с учётом числа главных компонент по правилу $X_{new} = X \\cdot V = U \\cdot S \\cdot V^T \\cdot V = U \\cdot S$.\n" 41 | ], 42 | "metadata": { 43 | "id": "1eGqxmn_8A-R" 44 | } 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "source": [ 49 | "### **Дополнительные возможности PCA**\n", 50 | "**Коэффициент объяснённой дисперсии** каждой главной компоненты, доступный через переменную *explained_variance_ratio_*, указывает долю дисперсии датасета, лежащей вдоль оси каждой главной компоненты.\n", 51 | "\n", 52 | "**Восстановление данных** с помощью метода *inverse_transform()* заключается в применении обратной трансформации проекции PCA вида $X_{recovered} = X_{d-proj} W_d^T$, где $W_d^T$ — матрица из первых d главных компонент. Из этого следует, что данные будут восстановлены с потерями, пропорциональными количеству отброшенной дисперсии исходных данных, а средний квадрат расстояния между исходными и восстановленными данными представляет собой ошибку восстановления (reconstruction error).\n", 53 | "\n", 54 | "**Инкрементный PCA**, реализованный в виде класса *IncrementalPCA*, позволяет работать эффективнее с большими наборами данных за счёт их разбиения на мини-пакеты и поштучном хранении в памяти во время обучения.\n", 55 | "\n", 56 | "**Рандомизированный PCA**, устанавливаемый с помощью параметра svd_solver='randomized', использует стохастический алгоритм для быстрого вычисления приближённых d главных компонент и основан на предположении, что случайная проекция данных на низкоразмерное подпространство может хорошо сохранять их структуру и свойства, однако такой подход может быть менее точным.\n", 57 | "\n", 58 | "**Ядерный PCA**, реализованный с помощью класса *KernelPCA*, позволяет выполнять сложные нелинейные проекции с использованием ядерных функций. Как и в случае с SVM, его суть в данном случае заключается в том, что линейная граница решений в многомерном пространстве признаков будет соответствовать сложной нелинейной границе в исходном пространстве." 59 | ], 60 | "metadata": { 61 | "id": "bhdxhVqqiT_N" 62 | } 63 | }, 64 | { 65 | "cell_type": "markdown", 66 | "source": [ 67 | "### **Альтернативы PCA**\n", 68 | "Не смотря на то, что метод главных компонент является одним из самых популярных алгоритмов понижения размерности, существуют альтернативы, которые могут быть более предпочтительными в ряде ситуаций, а также в зависимости от типа данных:\n", 69 | "- **LLE (Locally Linear Embedding)** — алгоритм создания линейных комбинаций каждой точки из её соседей с последующим восстановлением этих комбинаций в пространстве более низкой размерности, что позволяет сохранить нелинейную геометрию данных и быть полезным для некоторых задач, где глобальные свойства менее важны. С другой стороны, такой подход имеет высокую вычислительную сложность и может быть чувствителен к шуму.\n", 70 | "- **t-SNE (t-Distributed Stochastic Neighbor Embedding)** — алгоритм, который преобразует сходства между данными в вероятности и в дальнейшем пытается минимизировать расхождение между распределениями вероятностей в пространстве высокой и низкой размерности. t-SNE эффективен при визуализации данных высокой размерности, однако может искажать глобальную структуру данных, поскольку не учитывает линейные зависимости, а лишь их близость в исходном пространстве.\n", 71 | "- **UMAP (Uniform Manifold Approximation and Projection)** — ещё один алгоритм, подходящий для визуализации данных, который основан на идеи, что данные лежат на некотором однородном многообразии, которое можно аппроксимировать с помощью графа соседей. Такой подход учитывает глобальную структуру данных и позволяет лучше адаптироваться к различным типам данных и лучше справляться с шумом и выбросами, чем t-SNE.\n", 72 | "- **Autoencoders** — тип нейронных сетей, основанный на обучении кодировщика преобразовывать входные данные в низкоразмерное представление, с последующим обучением декодера восстанавливать исходные данные из этого представления. Autoencoders могут также использоваться для сжатия данных, удаления шума и многих других целей.\n" 73 | ], 74 | "metadata": { 75 | "id": "VMuABOU9iaAc" 76 | } 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "source": [ 81 | "### **Реализация на Python с нуля**\n" 82 | ], 83 | "metadata": { 84 | "id": "_lBgn_MwyM_F" 85 | } 86 | }, 87 | { 88 | "cell_type": "code", 89 | "source": [ 90 | "import numpy as np\n", 91 | "from sklearn.decomposition import PCA\n", 92 | "from sklearn.datasets import load_iris" 93 | ], 94 | "metadata": { 95 | "id": "IRx6PDrq7PSK" 96 | }, 97 | "execution_count": 6, 98 | "outputs": [] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "source": [ 103 | "class SVDPCA:\n", 104 | " def __init__(self, n_components=None):\n", 105 | " self.n_components = n_components\n", 106 | "\n", 107 | " @staticmethod\n", 108 | " def svd_flip_vector(U):\n", 109 | " max_abs_cols_U = np.argmax(np.abs(U), axis=0)\n", 110 | " # extract the signs of the max absolute values\n", 111 | " signs_U = np.sign(U[max_abs_cols_U, range(U.shape[1])])\n", 112 | "\n", 113 | " return U * signs_U\n", 114 | "\n", 115 | " def fit_transform(self, X):\n", 116 | " n_samples, n_features = X.shape\n", 117 | " X_centered = X - X.mean(axis=0)\n", 118 | "\n", 119 | " if self.n_components is None:\n", 120 | " self.n_components = min(n_samples, n_features)\n", 121 | "\n", 122 | " U, S, Vt = np.linalg.svd(X_centered)\n", 123 | " # flip the eigenvector sign to enforce deterministic output\n", 124 | " U_flipped = self.svd_flip_vector(U)\n", 125 | "\n", 126 | " self.explained_variance = (S[:self.n_components] ** 2) / (n_samples - 1)\n", 127 | " self.explained_variance_ratio = self.explained_variance / np.sum(self.explained_variance)\n", 128 | "\n", 129 | " # X_new = X * V = U * S * Vt * V = U * S\n", 130 | " X_transformed = U_flipped[:, : self.n_components] * S[: self.n_components]\n", 131 | "\n", 132 | " return X_transformed" 133 | ], 134 | "metadata": { 135 | "id": "lsf4c0787Nd-" 136 | }, 137 | "execution_count": 7, 138 | "outputs": [] 139 | }, 140 | { 141 | "cell_type": "markdown", 142 | "source": [ 143 | "### **Загрузка датасета**" 144 | ], 145 | "metadata": { 146 | "id": "Ap2qaOBsySp-" 147 | } 148 | }, 149 | { 150 | "cell_type": "code", 151 | "source": [ 152 | "X, y = load_iris(return_X_y=True, as_frame=True)\n", 153 | "print(X)" 154 | ], 155 | "metadata": { 156 | "id": "RwZpNYTACi7X", 157 | "colab": { 158 | "base_uri": "https://localhost:8080/" 159 | }, 160 | "outputId": "fe21d12d-7bbe-490b-c85f-463cb2a725b9" 161 | }, 162 | "execution_count": 8, 163 | "outputs": [ 164 | { 165 | "output_type": "stream", 166 | "name": "stdout", 167 | "text": [ 168 | " sepal length (cm) sepal width (cm) petal length (cm) petal width (cm)\n", 169 | "0 5.1 3.5 1.4 0.2\n", 170 | "1 4.9 3.0 1.4 0.2\n", 171 | "2 4.7 3.2 1.3 0.2\n", 172 | "3 4.6 3.1 1.5 0.2\n", 173 | "4 5.0 3.6 1.4 0.2\n", 174 | ".. ... ... ... ...\n", 175 | "145 6.7 3.0 5.2 2.3\n", 176 | "146 6.3 2.5 5.0 1.9\n", 177 | "147 6.5 3.0 5.2 2.0\n", 178 | "148 6.2 3.4 5.4 2.3\n", 179 | "149 5.9 3.0 5.1 1.8\n", 180 | "\n", 181 | "[150 rows x 4 columns]\n" 182 | ] 183 | } 184 | ] 185 | }, 186 | { 187 | "cell_type": "markdown", 188 | "source": [ 189 | "### **Обучение моделей и оценка полученных результатов**\n", 190 | "Ручная реализация показала идентичные результаты scikit-learn. Как можно заметить, первые 2 главные компоненты объясняют практически 98% дисперсии в данных, что позволяет сократить количество признаков вдвое без особой потери информации. Если бы количество признаков было не 4, а несколько тысяч или миллионов, то это бы позволило существенно сократить время обучения моделей без значительной потери точности (а иногда и с увеличением точности за счёт уменьшения мультиколлинеарности между признаками), что делает PCA и его альтернативы прекрасным дополнением к другим алгоритмам." 191 | ], 192 | "metadata": { 193 | "id": "nc7ILPOKyW_b" 194 | } 195 | }, 196 | { 197 | "cell_type": "markdown", 198 | "source": [ 199 | "**PCA**" 200 | ], 201 | "metadata": { 202 | "id": "zAlR-QBEglAK" 203 | } 204 | }, 205 | { 206 | "cell_type": "code", 207 | "source": [ 208 | "pca = SVDPCA()\n", 209 | "X_transformed = pca.fit_transform(X)\n", 210 | "\n", 211 | "print('transformed data', X_transformed[:10], '', sep='\\n')\n", 212 | "print('explained_variance', pca.explained_variance)\n", 213 | "print('explained_variance_ratio', pca.explained_variance_ratio)" 214 | ], 215 | "metadata": { 216 | "colab": { 217 | "base_uri": "https://localhost:8080/" 218 | }, 219 | "id": "4llKYPNLCrys", 220 | "outputId": "fc82c1ca-5ef5-46d7-e0c4-0891a26446a1" 221 | }, 222 | "execution_count": 9, 223 | "outputs": [ 224 | { 225 | "output_type": "stream", 226 | "name": "stdout", 227 | "text": [ 228 | "transformed data\n", 229 | "[[-2.68412563e+00 3.19397247e-01 -2.79148276e-02 -2.26243707e-03]\n", 230 | " [-2.71414169e+00 -1.77001225e-01 -2.10464272e-01 -9.90265503e-02]\n", 231 | " [-2.88899057e+00 -1.44949426e-01 1.79002563e-02 -1.99683897e-02]\n", 232 | " [-2.74534286e+00 -3.18298979e-01 3.15593736e-02 7.55758166e-02]\n", 233 | " [-2.72871654e+00 3.26754513e-01 9.00792406e-02 6.12585926e-02]\n", 234 | " [-2.28085963e+00 7.41330449e-01 1.68677658e-01 2.42008576e-02]\n", 235 | " [-2.82053775e+00 -8.94613845e-02 2.57892158e-01 4.81431065e-02]\n", 236 | " [-2.62614497e+00 1.63384960e-01 -2.18793179e-02 4.52978706e-02]\n", 237 | " [-2.88638273e+00 -5.78311754e-01 2.07595703e-02 2.67447358e-02]\n", 238 | " [-2.67275580e+00 -1.13774246e-01 -1.97632725e-01 5.62954013e-02]]\n", 239 | "\n", 240 | "explained_variance [4.22824171 0.24267075 0.0782095 0.02383509]\n", 241 | "explained_variance_ratio [0.92461872 0.05306648 0.01710261 0.00521218]\n" 242 | ] 243 | } 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "source": [ 249 | "**PCA (scikit-learn)**" 250 | ], 251 | "metadata": { 252 | "id": "m5aG-Cl2goww" 253 | } 254 | }, 255 | { 256 | "cell_type": "code", 257 | "source": [ 258 | "sk_pca = PCA()\n", 259 | "sk_X_transformed = sk_pca.fit_transform(X)\n", 260 | "\n", 261 | "print('sk transformed data', sk_X_transformed[:10], '', sep='\\n')\n", 262 | "print('sk explained_variance', sk_pca.explained_variance_)\n", 263 | "print('sk explained_variance_ratio_', sk_pca.explained_variance_ratio_)" 264 | ], 265 | "metadata": { 266 | "colab": { 267 | "base_uri": "https://localhost:8080/" 268 | }, 269 | "id": "D91ifASgDTbX", 270 | "outputId": "afa161c5-b856-428d-c5a4-eba6b29204df" 271 | }, 272 | "execution_count": 10, 273 | "outputs": [ 274 | { 275 | "output_type": "stream", 276 | "name": "stdout", 277 | "text": [ 278 | "sk transformed data\n", 279 | "[[-2.68412563e+00 3.19397247e-01 -2.79148276e-02 -2.26243707e-03]\n", 280 | " [-2.71414169e+00 -1.77001225e-01 -2.10464272e-01 -9.90265503e-02]\n", 281 | " [-2.88899057e+00 -1.44949426e-01 1.79002563e-02 -1.99683897e-02]\n", 282 | " [-2.74534286e+00 -3.18298979e-01 3.15593736e-02 7.55758166e-02]\n", 283 | " [-2.72871654e+00 3.26754513e-01 9.00792406e-02 6.12585926e-02]\n", 284 | " [-2.28085963e+00 7.41330449e-01 1.68677658e-01 2.42008576e-02]\n", 285 | " [-2.82053775e+00 -8.94613845e-02 2.57892158e-01 4.81431065e-02]\n", 286 | " [-2.62614497e+00 1.63384960e-01 -2.18793179e-02 4.52978706e-02]\n", 287 | " [-2.88638273e+00 -5.78311754e-01 2.07595703e-02 2.67447358e-02]\n", 288 | " [-2.67275580e+00 -1.13774246e-01 -1.97632725e-01 5.62954013e-02]]\n", 289 | "\n", 290 | "sk explained_variance [4.22824171 0.24267075 0.0782095 0.02383509]\n", 291 | "sk explained_variance_ratio_ [0.92461872 0.05306648 0.01710261 0.00521218]\n" 292 | ] 293 | } 294 | ] 295 | }, 296 | { 297 | "cell_type": "markdown", 298 | "source": [ 299 | "### **Преимущества и недостатки**\n", 300 | "Преимущества:\n", 301 | "- понижение размерности с сохранением большого количества информации, что также позволяет визуализировать данные высокой размерности в двумерном или трёхмерном пространстве;\n", 302 | "- не только позволяет значительно ускорить обучение, но и уменьшить переобучение моделей в ряде случаев;\n", 303 | "- может использоваться для сжатия данных.\n", 304 | "\n", 305 | "Недостатки:\n", 306 | "- неизбежная потеря части информации в данных;\n", 307 | "- поиск только линейной зависимости в данных (в обычном PCA);\n", 308 | "- отсутствие смыслового значения главных компонент из-за трудности их связывания с реальными признакам." 309 | ], 310 | "metadata": { 311 | "id": "hFEntYHAksy_" 312 | } 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "source": [ 317 | "### **Дополнительные источники**\n", 318 | "Статьи:\n", 319 | "- «A Tutorial on Principal Component Analysis», Jonathon Shlens;\n", 320 | "- «Locally Linear Embedding and its Variants: Tutorial and Survey», Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray, Mark Crowley;\n", 321 | "- «Theoretical Foundations of t-SNE for Visualizing High-Dimensional Clustered Data», T. Tony Cai, Rong Ma;\n", 322 | "- «UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction», Leland McInnes, John Healy, James Melville;\n", 323 | "- «Deep Autoencoders for Dimensionality Reduction of High-Content Screening Data», Lee Zamparo, Zhaolei Zhang.\n", 324 | "\n", 325 | "Документация:\n", 326 | "- [описание PCA](https://scikit-learn.org/stable/modules/decomposition.html#pca);\n", 327 | "- [описание LLE](https://scikit-learn.org/stable/modules/manifold.html#locally-linear-embedding);\n", 328 | "- [описание t-SNE](https://scikit-learn.org/stable/modules/manifold.html#t-sne);\n", 329 | "- [PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html);\n", 330 | "- [LLE](https://scikit-learn.org/stable/modules/generated/sklearn.manifold.LocallyLinearEmbedding.html);\n", 331 | "- [t-SNE](https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html);\n", 332 | "- [UMAP](https://umap-learn.readthedocs.io/en/latest/index.html).\n", 333 | "\n", 334 | "Видео:\n", 335 | "- PCA: [один](https://www.youtube.com/watch?v=FgakZw6K1QQ), [два](https://www.youtube.com/watch?v=fkf4IBRSeEc), [три](https://www.youtube.com/watch?v=IwPzjlBXBlA), [четыре](https://www.youtube.com/watch?v=WW3ZJHPwvyg);\n", 336 | "- [LLE](https://www.youtube.com/watch?v=B6kzA1W_4pU);\n", 337 | "- [t-SNE](https://www.youtube.com/watch?v=NEaUSP4YerM);\n", 338 | "- [UMAP](https://www.youtube.com/watch?v=eN0wFzBA4Sc);\n", 339 | "- [autoencoders](https://www.youtube.com/watch?v=FhmpO73ythg)." 340 | ], 341 | "metadata": { 342 | "id": "MEqVtot3k0Vs" 343 | } 344 | } 345 | ] 346 | } -------------------------------------------------------------------------------- /Notebooks [rus]/empty.txt: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![ML from scratch course logo](https://github.com/egaoharu-kensei/ML-algorithms-from-scratch.-Course-for-beginners/assets/162469942/db9333ab-6025-4961-b745-d45467ab71c2) 2 | 3 | [![CC BY-NC-SA 4.0][cc-by-nc-sa-shield]][cc-by-nc-sa] 4 | 5 | [cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/ 6 | [cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png 7 | [cc-by-nc-sa-shield]: https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-green.svg 8 | 9 | This repository contains implementations of all popular ML-algorithms from scratch using Python with their detailed theoretical description. In addition, this course is presented in two languages (eng, rus) in the form of jupyter notebooks and will feature tutorials on the necessary ML-libraries and also all the theory of concepts in ML, starting from methods of evaluating models and ending with optimization methods. 10 | 11 | #### 🔔***This course is also available on other platforms: [Habr (rus)](https://habr.com/ru/users/egaoharu_kensei/publications/articles/) and [Kaggle (eng)](https://www.kaggle.com/egazakharenko/code)*** 12 | 13 | ## **What you need to know before studying** 14 | 15 | - python (at the function and OOP level); 16 | - mathematics (linear algebra, calculus, probability theory and statistics). 17 | 18 | ## **Current content** 19 | 20 | *At the moment, the course is under development and new notebooks will be gradually added* 21 | 22 | ***Initial theory*** 23 | - 8\) Quality metrics and error analysis 24 | - 9\) Optimization methods in ML and DL 25 | 26 | ***Supervised learning*** 27 | - 1\) Linear regression and its modifications 28 | - 2\) Logistic & Softmax-regressions 29 | - 3\) Linear Discriminant Analysis (LDA) 30 | - 4\) Naive Bayes Classifier 31 | - 5\) Support Vector Machines (SVM) 32 | - 6\) K-Nearest Neighbors (KNN) 33 | - 7\) Decision Tree (CART) 34 | - 8\) Bagging & Random Forest 35 | - 9\) AdaBoost (SAMME & R2) 36 | - 10\) Gradient Boosting algorithms (XGBoost, CatBoost, LightGBM) 37 | - 11\) Stacking & Blending 38 | 39 | ***Unsupervised learning*** 40 | - 12\) Principal Component Analysis (PCA) 41 | - 13\) Popular clustering algorithms (K-Means, DBSCAN et al.) 42 | 43 | ## **Best additional sources** 44 | 45 | ***Books*** 46 | - "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow" (3rd edition) by Aurélien Géron 47 | - "An Introduction to Statistical Learning: with Applications in R" (2nd edition) by Gareth James et al. 48 | - "The Elements of Statistical Learning: Data Mining, Inference, and Prediction" (2nd edition) by Trevor Hastie et al. 49 | - "Pattern Recognition and Machine Learning" by Christopher M. Bishop 50 | - "Deep Learning" by Ian Goodfellow et al. 51 | 52 | ***Courses and playlists*** 53 | - [Machine Learning Specialization by Andrew Ng 2022](https://www.coursera.org/specializations/machine-learning-introduction) 54 | - [Machine Learning (StatQuest with Josh Starmer)](https://www.youtube.com/playlist?list=PLblh5JKOoLUICTaGLRoHQDuF_7q2GfuJF) 55 | - [Machine Learning Concepts (Simply Explained) by Pedram Jahangiry](https://www.youtube.com/playlist?list=PL2GWo47BFyUPWL5fBZSn6FFHRr1bSkX_J) 56 | 57 | ## 58 | 59 | ### All success and until we meet again!🏆 60 | --------------------------------------------------------------------------------