├── 2020 ├── ML_workshop_example_run.ipynb ├── ML_workshop_presentation.pptx ├── Q_AND_A.md ├── data │ └── matrix_dm-fitness_features_modnames.csv ├── img │ ├── 0_ml_workflow.png │ ├── 2_data_collection.png │ ├── 2_data_processing_binarize_onehot.png │ ├── 2_data_processing_scaling.png │ ├── 2_data_table_problems.png │ ├── 2_data_train_test_split.png │ ├── 3_feat_engr_intro.png │ ├── 3_feat_engr_select.png │ ├── 3_feat_engr_select_fit_transform.png │ ├── 3_feat_engr_sources.png │ ├── 4_model_select_cv.png │ ├── 4_model_select_hyperparameter.png │ ├── 4_model_select_intro.png │ ├── 4_model_select_sklearn.png │ ├── 5_model_appl_evaluate.png │ ├── 5_model_appl_interpret.png │ ├── 5_model_appl_new_cases.png │ ├── img_clone_repository.png │ └── img_what_ml_is.png ├── models │ └── model_train_select20_rf_rnd_search.joblib └── readme.md ├── 2022 ├── ML_workshop-part_a-preparation.ipynb └── __INSTRUCTOR_ML_workshop-part_b-in_class.ipynb ├── .ipynb_checkpoints └── ML_workshop_example_run-checkpoint.ipynb ├── LICENSE ├── ML_workshop-guide.pptx ├── ML_workshop-part_a-preparation.ipynb ├── ML_workshop-part_b-in_class.ipynb ├── ML_workshop-part_c_xai.ipynb ├── ML_workshop-xai.pptx ├── __INSTRUCTOR_ML_workshop-part_b-in_class.ipynb ├── enzyme_gene.csv ├── feature_list.csv ├── feature_list.xlsx ├── img ├── img_clone_repository.png └── img_what_ml_is.png ├── readme.md └── workshop_outline.xlsx /2020/ML_workshop_presentation.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/ML_workshop_presentation.pptx -------------------------------------------------------------------------------- /2020/Q_AND_A.md: -------------------------------------------------------------------------------- 1 | # Synopsis 2 | 3 | Below are the questions raised during the Plant Biology 2020 workshop session on Wednesday 7/29/20. 4 | 5 | ## Literature 6 | 7 | ### On using ML to investigate gene expression patterns: 8 | 9 | Depend on what specific questions one want to ask, there are quite a few papers out there using ML to investigate gene expression. Here are some examples: 10 | 11 | * [Statistical and Machine Learning Approaches to Predict Gene Regulatory Networks From Transcriptome Datasets](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6281826/) 12 | * [The identification of cis-regulatory elements: A review from a machine learning perspective](https://pubmed.ncbi.nlm.nih.gov/26499213/) 13 | * [The DNA binding landscape of the maize AUXIN RESPONSE FACTOR family](https://pubmed.ncbi.nlm.nih.gov/30375394/) 14 | * [Cis-Regulatory Code for Predicting Plant Cell-Type Transcriptional Response to High Salinity](https://pubmed.ncbi.nlm.nih.gov/31551359/) 15 | 16 | ### On looking for traits in photos that are too subtle for the human eye (e.g. leaf shape patterns): 17 | 18 | Couple interesting studies: 19 | 20 | * [Automated classification of tropical shrub species: a hybrid of leaf shape and machine learning approac](https://pubmed.ncbi.nlm.nih.gov/28924506/) 21 | * [Computer vision cracks the leaf code](https://pubmed.ncbi.nlm.nih.gov/26951664/) 22 | 23 | ### On identifying metabolites with metabolomics data: 24 | 25 | In the Plant Biology meeting 2020, [Gaurav Moghe from Cornell gave a talk on this](https://www.eventscribe.com/2020/ASPB/fsPopup.asp?Mode=presInfo&PresentationID=742044&query=gaurav%20moghe). He will be a much better person to answer this qusetion! 26 | 27 | ## What can ML do? 28 | 29 | ### Can ML be used to identify patterns in images? 30 | 31 | Serena Ghantous Lotreck : Yes! The types of tasks we’ll discuss in this workshop relate to numerical or categorical data, but there are many approaches that can be applied to image analysis 32 | 33 | samantha.neeno : Hi Ira---yes, it is possible. That can be done with Deep Learning models (for example), but then we get into data quality/labeling discussions 34 | 35 | ### Can I use ML to predict enzyme function (substrate specificity and product prediction), especially in large plant gene families? 36 | 37 | We are not expert in this area. Nonetheless, if there are high quality labels of which enzyme catalyze what substrates AND there are structural features (primiary, secondary, and tertiary) available, then one can attempt to build one. 38 | 39 | ### Can ML be used to score plants as alive/dead? Does this provide more accurate results compared to manual scoring? 40 | 41 | If you are able to provide a dataset where you have plant images scored for “live” or “dead”, you’d definitely be able to distinguish. I’m unaware of efforts to use this on plants life/death, but for example, AI systems trained to identify cancer in pathology slides are better able to detect cancer than human pathologists 42 | 43 | ## Supervised vs. Unsupervised Learning 44 | 45 | ### Why isn’t clustering a supervised algorithm? Isn’t the grouping (clustering) itself be a label? 46 | 47 | You can associate labels with groups, but the algorithm isn’t learning how to label the groups, it merely uses patterns in the data to group instances 48 | 49 | ### In my lab, I've used data from experiments with multiple predictors to try to find a regression model that best explains the data. Is this machine learning? Or would machine learning be applying this model to predict the outcome of future experiments based on known predictors? 50 | 51 | Linear regression is absolutely a type of machine learning! But it’s using the regression that you’ve found to make predictions for new instances that would constitute machine learning - finding the best regression is what we call fitting the model 52 | 53 | ### What is the best way to identify patterns of gene expression that predict the treatment that was imposed? For example: do plants that were exposed to weeds exhibit different patterns of expression than those that weren’t? 54 | 55 | Given the amount of transcriptome data available, this is feasible. You need carefully curated treatment labels for the experiments. Once you have that, you can use all genes where expression levels are available as features to make predictions. For the example you mentioned, perhsps you can take an unsupervised approach clustering the gene expression patterns based on experiments - then you can see if the weed-related experiments stands out on its own or whether it is related to some other treatments. 56 | 57 | ## I’m just getting started 58 | 59 | ### I’m just getting started with ML. What are your recommendations to get started? 60 | 61 | 0. Find a colleague who known machine learning already - in stat, math, and engineering, they are quite a few people with expertise. 62 | 1. Find a question relevant to your research that can be solved by building a statistical model. 63 | 2. Start with online resources like this workshop to get a basic understanding of the terms and processes. 64 | 3. Check out introductory books like [this one])(https://www.cs.waikato.ac.nz/ml/weka/book.html) that does not require programming experiences or, if you have some Python background, check [this one](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/) out. 65 | 4. Follow the books and, in addition to practice on the examples provided by the book, use your own data to see if you can use the approaches in the books to solve your own problems. 66 | 5. If at all possible, take one of the many [online courses](https://www.freecodecamp.org/news/every-single-machine-learning-course-on-the-internet-ranked-by-your-reviews-3c4a7b8026c0/). 67 | 68 | ### I am working on completing my PhD in wet lab research. What can I do to also study machine learning in an efficient but effective way? 69 | 70 | The most efficient way to learn is to work with real data as you follow along with books or tutorials. If you have a question and a dataset you’d like to use, we’d recommend flexing your new skills this way. 71 | 72 | Also see the answer to the previous question. 73 | 74 | ### How much programming do I need to use ML? 75 | 76 | You don’t need much at all if you use [Weka](https://www.cs.waikato.ac.nz/ml/weka/index.html) a powerful machine learning software that only require that you put data in. 77 | 78 | If you have experiences working with command-line interface, you can use [our machine learning pipeline](https://github.com/ShiuLab/ML-Pipeline), which is an end-to-end pipeline that runs on the command line. 79 | 80 | Additionally, if you have basic skills in Python, scikit-learn provides many tools to do machine learning in a fairly streamlined way. 81 | 82 | ### What options are available on the command line for your ML pipeline tool? 83 | 84 | I'd suggest that you check it out in our Github repository: 85 | 86 | https://github.com/ShiuLab/ML-Pipeline 87 | 88 | It is rather extensive and is our code-base for multiple ML-related publications in the last five years. 89 | 90 | ## Data selection 91 | 92 | ### There are a lot of biological data online, but the culture and treatment conditions are different: Can I still use these data to train one ML model? 93 | 94 | The short answer is yes! Longer version: it is rarely the case that the environment and development stages match among the dataset one want to use. The mismatch is expected to reduce the performance of your models but you should always try. Invariably, they will provide SOME information and ML algorithms are very good at picking relevant information out. If you are worried that you got junky info out of the model, you just need to apply the model to the test set to see if good predictions can be made. If so, it is useful. 95 | 96 | ### Do all features have to be numerical? 97 | 98 | You can also have categorical features! There are a few differences in how we preprocess categorical data, which we’ll mention further on, but perfectly able to incorporate them! 99 | 100 | ## Data pre-processing and feature selection 101 | 102 | ### How do I split my data between training and test? 103 | 104 | There are two main ways: one is to randomly split the data between training and test, and the other is called stratified sampling, which tries to make sure the test and training sets are representative of the true data 105 | 106 | ### You’ve said that we must use the same training and test sets in order to reproduce our results. If using a different split gives different results, doesn’t that call into question the quality of the analysis? 107 | 108 | Here "gives different results" I take it to means that the model performance changes depend on the test set. I don't think this call into the quality of the analysis but is an inevitable consequence of sampling. In the best case senario, one would want to use a test set that is representative of the unknowns out there. This is not trivial since we only have a limited amount of test cases. So what you will see is that, given the training samples are fixed, different test sets will give you somewhat different model performance values. But if you look at enough testing sets, then you will find that the averge of the perofrmance is a good estimate of the true performance. 109 | 110 | ### How can I impute missing values? 111 | 112 | One practice is simply drop any columns or rows with missing values. If there is little missing data, this practice is fine. IF not imputation is necessary. 113 | 114 | For numerical features, it’s common to use the median of the training instances to fill in missing values for features, and for categorical features it’s common to use the most common feature value. There are other options as well! 115 | 116 | Julissa Ek Ramos : @MIn-Yao…I am new in ML, but when I am doing some predictions, I use bootstrapping to find the best fitted missing data. 117 | 118 | ### Does imputation skew the distribution of my original data? 119 | 120 | If there are a lot of missing values, for example if median is used, then the distribution will look tighter than it really is. 121 | 122 | ### Can one-hot encoding be done automatically? 123 | 124 | In Scikit-Learn, there is the OneHotEncoder class that you can call to do it. 125 | 126 | ### At what point is one-hot encoding my categorical features too much? For example, a categorical feature with 10 possible values would result in 10 very sparse features. 127 | 128 | On one hand, such practice increases the number of features, makes the model more complicated, and leads to overfitting. On the other hand, one cannot rule of the possibility that they can be useful. Paricularly considering some ML algorithms are really good at using features with little, but useful infromation, we tend to keep them unless they are too sparse (e.g. only a few percentages of the data is 1 and the rest is zero or vice versa). In those case, we'd drop those. 129 | 130 | ## Feature selection 131 | 132 | ### Will having redundant or similar data affect the final model quality? 133 | 134 | Yes! having several highly correlated features can actually negatively impact model performance, and makes model interpretation more difficult That’s what makes feature selection/engineering so important! 135 | 136 | ### Is there a threshold for removing correlated features? 137 | 138 | It's hard to define a threshold for removing correlated features. One approach is to "merge" correlated features using principle componenet analysis (see [this post](https://medium.com/towards-artificial-intelligence/training-a-machine-learning-model-on-a-dataset-with-highly-correlated-features-debddf5b2e34)). 139 | 140 | ### In ecology, we know that several environmental variables (features) are highly correlated: for example rainfall and temperature. How we can choose the most useful of these correlated features, and how machine learning can help us? 141 | 142 | Song Li : You do PCA to reduce the correlated features to smaller number of features. We typically just use the PC1, PC2 etc as your features 143 | 144 | Serena: ne thought is that you could train models with just one and combinations of the correlated group and evaluate performance to see which are the most useful. Random Forest also outputs overall feature importances, so you could take a look at those 145 | 146 | Serena: using the PC’s as features does make model interpretation more difficult 147 | 148 | ### During feature selection, is it best practice to include or exclude information (e.g. experimental blocking, biological replicate) that I know influence the outcome/measured variable? 149 | 150 | Song Li : For Brianne, you do a regression to remove those confounding factors from your data before machine learning 151 | 152 | ### Is it good to use the same ML algorithm (for example, Random Forest) for both feature selection and regression? 153 | 154 | In this case we think that the random forest may have superior performance because we also used it to do feature selection - that the features selected were optimal for random forest but not for the others. However, there isn’t a rule about whether you should or shouldn’t do that! You should test many algorithms and choose the highest-performing one 155 | 156 | ## Performance evaluation 157 | 158 | ### How can I determine how different algorithms perform on my data? 159 | 160 | In our example, r2 is our performance measure, so the value of r2 from different algorithms allows us to evaluate which algorithms perform best 161 | 162 | ### In terms of applications of ML in biology, is the general problem model underfitting or model overfitting? 163 | 164 | We typically do 5 or 10 cross-validations. In my own experience, overfitting is more of the problem, as we have highly dimensional data that’s extremely information rich 165 | 166 | ### Is there an accepted value for performance measures (e.g. r2) in cross-validation? 167 | 168 | This depends on: 169 | 170 | 1. If there is a model established previously, if the performance measure is better than the previous one. 171 | 2. If there is no previous model, is it signfiicantly better than random expectation. 172 | 173 | ### What does the r2 value during cross-validation tell us? What is the impact of sample size on the conclusions we can draw from looking at r2? 174 | 175 | It tells us what the average r2 is after splitting the data 10 times and train 10 differnt models using the training data. 176 | 177 | Sample size will impact the interpretation of r2 signfiicantly. In our case there are over 6,000 data points, so even with an r2 of 0.1, the p-value will be extremely small so we did not worry about it. But if the sample size is small, the one should also consider what the p-value is. 178 | 179 | ## The cutting edge 180 | 181 | ### Are you guys applying methods of transfer learning? 182 | 183 | We are currently working on transfer learning methods. A manuscript following the general idea of transfer learning using Arabidopsis metaoblic gene annotation to help predicting tomato ones has recently been accepted in [In Silico Plants](https://doi-org.proxy2.cl.msu.edu/10.1093/insilicoplants/diaa005). 184 | 185 | 186 | 187 | 188 | 189 | 190 | 191 | 192 | 193 | 194 | 195 | 196 | 197 | 198 | 199 | 200 | -------------------------------------------------------------------------------- /2020/img/0_ml_workflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/0_ml_workflow.png -------------------------------------------------------------------------------- /2020/img/2_data_collection.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/2_data_collection.png -------------------------------------------------------------------------------- /2020/img/2_data_processing_binarize_onehot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/2_data_processing_binarize_onehot.png -------------------------------------------------------------------------------- /2020/img/2_data_processing_scaling.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/2_data_processing_scaling.png -------------------------------------------------------------------------------- /2020/img/2_data_table_problems.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/2_data_table_problems.png -------------------------------------------------------------------------------- /2020/img/2_data_train_test_split.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/2_data_train_test_split.png -------------------------------------------------------------------------------- /2020/img/3_feat_engr_intro.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/3_feat_engr_intro.png -------------------------------------------------------------------------------- /2020/img/3_feat_engr_select.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/3_feat_engr_select.png -------------------------------------------------------------------------------- /2020/img/3_feat_engr_select_fit_transform.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/3_feat_engr_select_fit_transform.png -------------------------------------------------------------------------------- /2020/img/3_feat_engr_sources.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/3_feat_engr_sources.png -------------------------------------------------------------------------------- /2020/img/4_model_select_cv.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/4_model_select_cv.png -------------------------------------------------------------------------------- /2020/img/4_model_select_hyperparameter.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/4_model_select_hyperparameter.png -------------------------------------------------------------------------------- /2020/img/4_model_select_intro.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/4_model_select_intro.png -------------------------------------------------------------------------------- /2020/img/4_model_select_sklearn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/4_model_select_sklearn.png -------------------------------------------------------------------------------- /2020/img/5_model_appl_evaluate.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/5_model_appl_evaluate.png -------------------------------------------------------------------------------- /2020/img/5_model_appl_interpret.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/5_model_appl_interpret.png -------------------------------------------------------------------------------- /2020/img/5_model_appl_new_cases.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/5_model_appl_new_cases.png -------------------------------------------------------------------------------- /2020/img/img_clone_repository.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/img_clone_repository.png -------------------------------------------------------------------------------- /2020/img/img_what_ml_is.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/img/img_what_ml_is.png -------------------------------------------------------------------------------- /2020/models/model_train_select20_rf_rnd_search.joblib: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/2020/models/model_train_select20_rf_rnd_search.joblib -------------------------------------------------------------------------------- /2020/readme.md: -------------------------------------------------------------------------------- 1 | # How machine learning can be used to solve plant biology problems 2 | 3 | ![alt text](./img/img_what_ml_is.png) 4 | 5 | ## Workshop information 6 | 7 | ### What this is about 8 | 9 | More and more data are available in plant science that have fueled ground breaking discoveries. Beyond the original intents of the experiments, these data can be used to discover even more. This is where machine learning comes in. We can use computers to learn from data and generate models that can predict a biological phenomenon of interest - e.g. will this gene be lethal when it is knocked-out, or which genetic variants can meaningfully predict a phenotype of interests. Specifically, this workshop will touch on the following topics: What is machine learning and why is it useful? How does machine learning work? What are some example machine learning applications in plant science? How can we feed data into machine learning tools to make discoveries? What are the best practices when doing machine learning? Where to go to learn more? The workshop will include presentations, discussions, and a short hands-on section using online machine learning resources. 10 | 11 | ### When and where 12 | 13 | The last time we offered the workshop: 14 | * 3:30-4:30pm, 7/29/2020 15 | * PlantBiology 2020 Worldwide Summit - Online 16 | * [Event link](https://www.eventscribe.com/2020/ASPB/fsPopup.asp?Mode=presInfo&PresentationID=742105) 17 | 18 | ### Who we are 19 | 20 | * Shin-Han Shiu: Departments of Plant Biology and Comp. Math., Sci., & Engr., Michigan State University 21 | * Serena Lotreck: Department of Plant Biology and Comp. Math., Sci., & Engr., Michigan State University 22 | * Our lab [website](https://shiulab.github.io/) 23 | 24 | ### What kinds of materials we are sharing 25 | 26 | There are two main documents: 27 | 1. [Workshop presentation slides](https://github.com/ShiuLab/ML_workshop/blob/master/ML_workshop_presentation.pptx) 28 | 2. [Workshop jupyter notebook](https://github.com/ShiuLab/ML_workshop/blob/master/ML_workshop_example_run.ipynb) 29 | 30 | ## Instructions for runnning the notebook 31 | 32 | * Developed by Christina Azodi for the [2019 workshop](https://github.com/azodichr/ML-Pipeline/tree/master/Workshop). 33 | * Modified for the 2020 workshop by Shinhan Shiu. 34 | 35 | ### What's needed 36 | 37 | The workshop example is provided as a [Jupyter notebook](https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/what_is_jupyter.html). It is a document generated by the [Jupyter Lab or Jupyter Notebook applications](](https://jupyter.org/install.html)). A notebook can contain both computer codes in popular languages such as Python and R, and texts in the form of paragraph, equations, figures, links, etc. 38 | 39 | To follow what we have shown in the workshop, you need the following: 40 | * Git: for you to "clone" this repository to your computer to play with. 41 | * Jupyter Lab: the application to view, edit, and execute codes in the notebook. 42 | * Scikit-Learn and others: the software packages Jupyter Lab relies on to run the codes. 43 | 44 | ### Install Git and clone ml_workshop 45 | 46 | [Github](https://github.com/) is a code hosting platform for version control (i.e., keep track of updates to codes) and collaboration (i.e., many people can work on the same codes). We have put the workshop materials in a Github repository called [ML_workshop]() 47 | 48 | 1. Create a [GitHub Account](https://github.com/join) 49 | 2. Download and install [Github Desktop](https://desktop.github.com/) 50 | * Or you can use [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) if you are familar with version control and command-line interface. Note that the following info is for using Github Desktop. 51 | 3. Clone the ML_workshop by following [this instruction](https://docs.github.com/en/desktop/contributing-and-collaborating-using-github-desktop/cloning-and-forking-repositories-from-github-desktop) and the following screenshot. 52 | * __Note:__ You can specify where the repository goes in your computer. We suggest leaving it as default and to remember where it is - we need it later. 53 | 54 | ![alt text](./img/img_clone_repository.png) 55 | 56 | ### Install Anaconda 57 | 58 | [Anaconda](https://www.anaconda.com/) is a a free and open-source distribution of the programming languages Python and R and is a widely used platform for computational and data science applications. 59 | 60 | 1. Download the Python 3.X version of [Anaconda](https://www.anaconda.com/products/individual#Downloads). Current, it is Python 3.7. 61 | 2. Install Anaconda using the [instructions](https://docs.anaconda.com/anaconda/install/). 62 | 3. Open your terminal in [Mac](https://support.apple.com/guide/terminal/open-or-quit-terminal-apd5265185d-f365-44cb-8b09-71a064a42125/mac) or [PC](https://www.wikihow.com/Open-Terminal-in-Windows) 63 | * __Note__: For PC, you need to open the terminal by "Running as Administrator". If you are not familiar with this, see [this post](https://www.itechtics.com/run-programs-administrator/) for more info. 64 | 65 | 4. Issue the following command to make sure Anaconda installation is complete: 66 | ``` 67 | conda list 68 | ``` 69 | The above command allows you to see what software packages have been installed. 70 | 71 | ### Install software packages 72 | 73 | [Conda](https://docs.conda.io/en/latest/) is a package/environment management system. It deals with installing software packages in your computer. It also creates and manage virtual environments where each environment you have a specific set of software for a general category of tasks. 74 | 75 | 1. Create an ml environment and activate it: 76 | ``` 77 | conda create -n ml 78 | conda activate ml 79 | ``` 80 | 81 | 2. Install software packages and their dependencies: 82 | ``` 83 | conda install jupyterlab matplotlib nb_conda pandas scikit-learn 84 | ``` 85 | 86 | ### Navigate to ML_workshop tutorial 87 | 88 | 1. Run Jupyter Lab 89 | 90 | * If you use Mac: 91 | 92 | ``` 93 | jupyter lab 94 | ``` 95 | * If you use PC, and your Github folder is in __C:/__ drive, then do: 96 | ``` 97 | jupyter lab --notebook-dir=C:/ 98 | ``` 99 | * If you use PC and your Github folder is in __D:/__ drive, do: 100 | ``` 101 | jupyter lab --notebook-dir=D:/ 102 | ``` 103 | 104 | 2. In the Jupyter lab window that opens, on the left panel, navigate to __ML_workshop__, the directory where the cloned Github repository is stored. 105 | 106 | 3. Open ML_workshop.ipynb 107 | 108 | 4. Run each code element by clicking ```SHIFT + ENTER```. 109 | -------------------------------------------------------------------------------- /2022/ML_workshop-part_a-preparation.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "---\n", 8 | "# __Machine Learning Part-A: Pre-Workshop Preparation__" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "metadata": {}, 14 | "source": [ 15 | "## Learning objectives\n", 16 | "\n", 17 | "At the end of the exercise, you should be able to:\n", 18 | "- Distinguish unsupervised and supervised learning.\n", 19 | "- Distinguish regression and classification tasks.\n", 20 | "- Explain the major steps in a machine learning project.\n", 21 | "- Check that the ncessary software packages are installed." 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "----\n", 29 | "\n", 30 | "# 1. Introduction to Machine Learning" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "✅ **DO THIS:** Please watch the following two videos introduces many key concepts in machine learning.\n", 38 | "\n", 39 | "To run the code cell, press `SHIFT`+`ENTER`." 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 2, 45 | "metadata": {}, 46 | "outputs": [ 47 | { 48 | "data": { 49 | "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAUDBAgICAgICAgJCAgICAgICAgICAgICAgICAgICAgICAgIChALCAgOCQkIDRUODhERExMTCAsWGBYSGBASExIBBQUFCAcIDwkJDxcQEBAWEhIVEhUVFRISEhISFhISEhISEhISFRIVEhIVFhUSEhISEhIVFRUSEhUSEhIWEhUVFf/AABEIAWgB4AMBIgACEQEDEQH/xAAcAAEAAgMBAQEAAAAAAAAAAAAABgcEBQgDAgH/xABXEAABAwIBBAoPBAgEBAMJAAABAAIDBBEFBhIhVBMVFxgxQZOU0tMHFBYiMlFTYXKBkZKy0dQzUnGhIzRCVXSxs+MIYmWCJXOiwSRD8DU2Y3WDlbS14f/EABsBAQABBQEAAAAAAAAAAAAAAAABAgMEBQcG/8QAPxEAAgECAQcICQMEAgIDAAAAAAECAxEEBRIUFiFSkRUxQVFTYXHRBhMyNGKBobHhIpLBByMzcoLwVMJCRPH/2gAMAwEAAhEDEQA/AOMkREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAftksrH3IMQ1ik5SfqE3IMQ1ik5SfqFGcja8i43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9myuLJZWPuQYhrFJyk/UJuQYhrFJyk/UJnIci43s2VxZLKx9yDENYpOUn6hNyDENYpOUn6hM5DkXG9my7URFbOvBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAU2yWoaSnwmoxeooxiD21raOOGR72U9ONja909RmaXglwYAdFy3jN1CVIcmMoMTwlhqKYujpqhxY7Zos+knfGLFpzhml4Gg5pDrCyqRgZRpzqU1GD23Wy7jnJbWrrar9x9YriFBVy0Zgw5tFIZmsqWxTPkp5mukjDDHG7TEbZ4IBtwcPCPTsq4dBSYzXU1PG2KGJ8AjjbfNaHUlPI61yTpc9x9a3WVrqabD8LxoUkVDUTVro546ZpZDURQkyGpih02s5gGi9y83LtBWL2cqV7MaqJ3D9BVtppqeUfZyM7Vhiux/A4hzHaB4x4wpfMa3BYi+IgleKzais5Z36oyjdXvtttt3GNimFwMyboaxsTRUy4jPFJNpz3Rtjnc1h02sCxvFxLf5aT4ZhjqGEYLS1OzYfTVMr3SzRPc6UPa4B0Zs2+Ze9uMrX5VROgyYwmCZpjmmramrjicLP2ANlaJC06QDskZF+J4W+7JOKYbBJhzavCu3ptqaNzZTXVFM0MvIBGYomkOAcHG/Cc+3EpMJznOpFfqmnOrsjK10uazuti8TX9y1BtpgMkMTtr8YYJu1ZnukdCWtvJEZL5z2XeyxJJ0O02ssHGso8Mgqqmnbk/ROEFTPThxnnBcIZnxBxA4CQ29vOvvJbKObEsoMKkkZHDHDKyCnpoGlsNPC1j82NgPCfGeOw4AABj5X5bP7bxGn7Rwmwq6yHZO0W7PYTyx7IZNkvs9tOdbwtKkqVKs60adVOTUE2s9xteTtdp7XaybMnI+Ki2sxrEpsOhqHU9TTmngkdIGRR1MoaY2uBDs1oeLE6e9C8jS4fiWGYjWU9A3DqvDO1pSI55JqWpime5mY6OQ948Zr/B4e90m5AzOx3WMp8n8dlkgiqmNnoLwT52xPzpGNBdmEO0EhwseFoX7lnU9tYLBUYXFFS4dnsZilFBG1roq5pBZJNL4UsBvHmE8B2O+kgNCTksTJK6/uRip5zzUrRbi1fbnK6V1zvnuaXslYVTx9oV9FEIaPEqVsrYmkubDUw2bURXubWJbx8IfbgX7hOFU0GB1mI1UQkmqpmUOGh9+8c0ONTUNsbEhocATezobftLNyMidimD1+ENGfVUj24lh7T4TrHMqom+Yh1wON0/mXh2W52QzUuEQG8GEU7YXEaBJVzBstTJ4iS4tHmcXqnvMmFWpKSwd3eMm277XTjaUdvfdJ9dmfeWWA07Z8Ap4GMg7focPMr23N5amXYnzOBOk6QfUszKOuwjDq2XDZMCbJDA7Y3zyVEzK+UWF6hkmgNa43c1os0i3gg2Hj2UYJJHYDHEx0kr8GomxsjDnSOcS7NDA3SXX8Sy8ksr6rEKumwnFqWLE45JO13GeHNrqcHvXSCVtiwxi7nEjOs098DwyYsHUdCFWTc4xU3KKm4yW12kmntslZJtdxg9jymojRY5WVFDHV9pMppIIp3us0SvmaWl7LXOaG3NtObwBavF8paGeCSKLBKSlkeAG1EU0zpIiHA5zWuFibAjT41K8jHsw6lyt2BsVRHSuhjiFVGJYpGR1NVG3ZWXAk70DxXIuoLlBlO6ujZG6loKfMk2QOoqUU8jjmubmvc15zmd9e3jA8SdBkYf8AvYmcs1tJxs89rN/TF2zb2e3n8SZY3Nh2G0ODE4PS1ctZh0VRNLLJMxxlzGBxszQbkk8S1xwvD8Wo6ufD6Z1BXUMRqJqMTyVMFTTN+0kgc/vo5WaO94NIGnOuPzsouAosm7kD/g0XCf8ALEvfsUQOpIcTxedpZSx4fPSxOcM0VNRUOi2OOK/h6WAG2gF7fEVPSWof28Kq8ZPPznb9TabzmrZrdndbNiIpkhNTR11K6sjbLSGVrJ2v8HYpAYzITcEZmcH6D+wvTLnBThuIVVIfAilJhJuc6CT9JDpPCcxzQT42uWlA0W81lbmE4M3H4MDrHEE0bzQ4u53HBSMNTDJITxPjaWF3jqvMqVtNljq7wtVVpN5jTi+5q8ovxe2PfdGvgyRpC/J/CZGbHXV+fW18wJ2aOAxzSQUwDiQwljHNOjQ6MHjstXi2OYSH1FI/AWQsi2SKJ8dRMytikZnNY6dzwQ85wbnNde1z4XHrMfxeqxPF5qylbM6Z0zpKUQNe6WOGnb+hc1rRdpbGxrjbjLvGpdkZlE7KCoOH4tTQVQNPKdsGRthq6TY2OcJXSs73Nzg1trN0uF7i4VVzXThVpRVWs3JZudJKbjKMm220r7UlsSv0bOcqtS7sk4ZBTbVbBG2PZ8Ho6iXNv380mfnvdc+EbD2KJ+u/n4L+e3Ep32U4TJS4FWsBdTuwmnpdkAuxs9OXCSJ7uBrwSRY8OY/xFUrmNriqjWJo7bJ5y8Xa6T/gwcAwyCTAcaq3xtdUU82GthkN86MTVUccobpt3zSRp8akDMi6WswLD5aYNZi74qyobEC7OroKWo2OZrQTbZWNfDm2tfOtx3brsEhdFkti8koLGVtZh8dMXC2zGCojllMd/CaGNfpGi7HDiK/MoK6WlwrJSpp3mOaEYtIx44WuFXBwj9ppFwQdBBIOgqUaurUqzqNU52frnbq2U728Lqz+fSa/sYYbBVVVTHURNlYzDayZrX3GbLG2PNfoIOcCSsjI+hpIMJqsYqaPt98VZHRRQPe5lPDnRRyOqKjM0ub+kDADovbx3E6yUpKeufU47R5kRkw6ugxOjBA2CtdEx2ysHHFKGOd+Jvwl2bWOSuUGJ4VH2zTF0dPO7Y3mWLZKSd7G6Y3ZwzS8A/skOtfTwpzD11XFSqKDzWsxODk47VnZ0brar251zo/cbxKgq3Upp8ObQy7Lm1AhmdJTyMLmBhjjdpjd4d7ebhvoyOyxh0NJjVdTU7GxQxOpxHG2+a3Po6eR1tN9L3uP+5bfK001RhmH4z2nFQ1T610MzKZpjhqo4w6R1RHEb5tnMzbi+kuBLu9tj9nOnc3GZ6k/YVrKaemmH2crBSwQksfwEh0Z0eJzTxhJF/BYhOvBK8Vm1I2cr/qUo7Ltu9ttu4xa7DIG5OUtYI2ipkxWWB8unPdE2nmeGHTbNzmtPBxLf5WzYbhjMNj2mpao1GGUtVK98s0UjpJGuD7OjNhctve3CStflFC6DJfDIZgY5Z8RqKuKNws8wNhli2QtOkNJfGR4w9pW97IGKYbA3CG1eFdvSnB6NzZTWz0zWss4CMxxNIcM4ON+HvrcSqMJ1JzqRTzppzq7IytdLm23WxeJrpMmaAYlk/NBE4UGL2eaSZ7pDCYy0SxbITnPZeRtiSTodpta2NlBlDhlNV1VO3J+ie2nqJoA8zTguEMrow4gaASG3X5k/lJNiWPYQ+SOOCGCaGGlpoQWw08TeBjAdJJsLnjsNAAAH5lplu9tbiMHaOE2bU1UWyOoWmcgSvZnuk2S5m4863DpUXRXGlWdeNOqnJqF2s9xteTtdp7WlZM+8jIqJ1BjuJTYdBOaeWmfTwSOkEcLKid7TE1wsS1rXNAJ096F8Clw7FMNxKqp6FuG1eGxwzExTyTU9RFI54LHRS32N/eOtbhObpIuFk9jWrZBgWUEj4I6prXYd+gnztikvMWjPzSHaCQ7QeEBemV1X21gkNRhUMVJQ57WYvRQRtD46tpBjlll8KamPe5t+All9Nw2kibksTJLOX9yMVLOealmxbi1fbncyurNvnNJ2R8Kp2sw/EaGJsNJiNMHGJhcWw1cJzaiMEk2FyOPhY9MDwumhwWvxKqibJJPIygw0Pv3spBNRO0A2cWsvYngMLhxrOyJY7E8IxHBwM+pgLcTw9vG57LMqYWek12geOdx4l5dlmVlO+jwaJwdFhNO2N7m6BJWThstTJYaDpzfwL3hO8yYVajksHd3jK7d9rpxtJO/fdRfgxlPgNP2tky2BjYZcRgDaiYXOfJJLSxiRwJ/Z2Rx0WWdlJWYRhlbLhj8DbNDBmskqJaiZldMC1pM8cgsGA3JAbZpt+zew8uyFC+XDsloo2OkfJQSsjYwFz3ucaUNa1o0lxNuBe+SuWFXW1dNhWK0sWJRvlFM4VEObXQX71z2yizmujF3OLhnWae+B0qosXqOkqjblGPrG4qbjL2naSa57JWSbXcYfY4paJ1Nj1XPRsq20MME1NDO93eh76kFrpGWuc1rATb9ngC1eK5S0E0EkUWB0dNI9tmzxzTOfGbjvmtcLE/j41LMiXMw6PK/YRFPHSCOOJtSwTRPbHU1rAJmEgSCw0+O11BcoMqHVzGRupcPgzXh4dRUjYJHEMezMc5rznM74nN8bWniQvYf+/iZyzW4pxs89rN/TF+ynZ9b8dpMcZlw7DcPwV5wemq5a2gbNNLLJKxxkAiBPeaDfPJWubhuH4vSVc2H0rsPr6GE1MtI2d9RT1dO37R8Jk7+OVujvRo75o0512uye4DDsmbkD/hXGbcVOvfsRRupG4jjEzS2kgw+eBjnDNbUVEzo9jhiJ0SElljbgL23UdJaj+jC+vjJ5+c7fqbTec0o5rdndbNiK9cdB/BTDssYXBSVlLHTxtiZJhlHM5rb2Mkjpw9xuTpIa32KHEWbbxNt+SnnZwcBX0QJA/4RQ8JA/aqVBtsRUccVRV9jU9nXzHn2LqOkfFjFRVUkdWKLDzUxRSlzWl8eyOsS03AOba6wsSynoJYZY48CpIHyRuYyeOedz4nOBAkaHCxc06bHxLcdh6r2Cnx+drI5Niwl8gZMwSQPLBK7NlYSA+M2sRfSCVG8eytdWRbE6kw2AZ7X7JR0Yglu2/e54ee8N9I8wU9BgqDqY2pdOSTjtz3HN2J+ynZ9ZsexphlK99XXV8ey0GG0zpZo9IEsst46eHQRcudnEaeFrfGsPsj4KygxGeGHTTyBtTSOBu0004zow0nSWtOcy58mpZLRUNJgdJQVle6hnxB7cUqGtpJKp74jdtLG8Mc3YmgNY6x/aY7zrxyxpaWrwOknoqp1a/BnCiqJHU76V/as5Bp86J5cc1jhGwEG3h+LQzdhZp45vF+tu82TcFsebZezK9rXck1z9K6j6yjlw3DabCQcHpap9XhsFTM+SWaN7pC1ofYxnRc6b241qctMCo4n4RV0kboaXFI2SmklkdKYHNkiZIwSuOc+MiQWJ096TfSAJdjmN0FMzJxmIYfBVwPwqlMs0ge6aGMta0ljL5r2A98WkEkA+ZRTsix1jMYhFVIyaImmNBJCxrKd1CZAYRA1netABIIHGOMFpMyMfAVJuUfajdTbbk2ppNpKKu1eOx9D8TP7MGRkFJJJV4dY0jJRT1kDSXGiqc1jmA3uRFI17CL8BcBwOAEfjoITk/JV5je2G4y2nEunOEJoRKY+G2bsnfcCk+VGUe1+UmKbKzZ6KqdFT19MdLZoHUsIJA8qzOcW/i4XGdcfWW2TzcPwB4ilbUUlTjUVVRTAgl9NJh5DM/xPa5r2ngvm3sL2AqpYmtTp0adRtuThKMutPni+9fVW7zW5UYLSxVeTsccLGMrKTDH1LRe0r552tkLrnhc3QbWX12W8kYKOV1Zh9nUBnfTzNaSTRVbDZ8L87vhG7Q5pPjtwFl87LP8AXslP4HB//wAlq8K3KNlFjmLwVLDNhtbUywV0Gk/oybNqGAaRKy9wRpIBHDmkQyMNWxH9ucG5WjNuO8s6zXilzeFjQ1mHwNyep6sRtFQ/FZYXy6c4xNpnyCM6bZocLrc1tBh2BxU7Kyj2yxSeFlRJDLK+Kloo33zWODPtpe9Nwb+CT3otnZXZIyf2vwCngbK2eGTF5JqWdpBE1NPRSOikOboDrXB87SRoIWL2bozUVUGLxAvoa+lgdHMNLI5WNLJIHn/y3iwNjbSXcbTaS5QxGkSjFyeZOVR87V7NZsb7Gtjbts5rGkx7GsLqqd+x4V2jWtczY5KWcupntzhsgmik8A5t7FlyTa5AGmMqb4DQ4bW4TiUraF0VVh9JA/tsVc8jJpnvLHO2A2jj8EnN0jvvMoQqGbrASh+unFNZrs1J350nsd3s6ef5Bb/JzLHEcPY6Kmn/AELjcwSxxzQ3OkuayRpzDfSc21zw3WgRUmZVowqxzakVJdTVzaZR5QVeIytlq53TOY3NjFmsjiZ92OKMBjBoF7C5sL3sFs8Ay+xWiibTwVIdCz7OKaOOdsXi2MyNLmgcTQbDxKMIpuW5YKhKChKCcVzKysvBGdjuL1NdM6ermdPM4Buc8izWi9mMY0BsbRc6GgDSfGV9Y1jNRWujfUyCR0MLKeMhjGZsUedmssxovbOOk6dK16IVxoUo2tFLN2LZzeHUZWEYhNSTxVNO7MmhfnxvzWvzXAEXzXgtOgnhCk57J2MkkmpjJJJJNHR3JPCSdh4VDkS5brYOhWd6kFJ812k9htO6Cq2Gsg2UCGvmbPVMDIxskok2UOBDbsGfpsywXxg2N1NGJ2wSBrKmMwzxvbHLFNGb96+ORpaSLusbXGc6x0la5EuVPDUmnHNVntatz2tZ/RcDOwHF6ignZU0spimYHBr81rhZ7S1wcxwLXAg8Y8R4QFjVlQ+aSSWVxfJK98sjzwufI4ve4+cuJK8kUlfqoZ2fZX5r9Nuq/UbOtx+smfSyPmcJKKJkNNIwNifFHFcsDXRgEuFzpOlbyp7JOMSRujNUG57Mx80cEEdQ5lrWMzGBzTbjbY+dRBEuWJ4HDyteCdubYtnTs+ZusmcqK3Ddl7SmEQnEYlDoopQ/Y87MuJWOtbOdweNZmL5d4nVwSU888boZQGva2lpWEgODtD2RhzdIHAVGUUXJlgaEp57gnLY72V9nNtJXQdkPFoIYqeOoZsUETIommlpXlkcbQ1rc6SMuOgDSStXlHlLX4iWGsqXzhngMs2ONpta7Yog1gdbRnWv51qES5FPA0ITz4wSfXZX28+0LZ4Rj9XSQVUEE5jhrYxFUMDWu2Rga9tgXAmMlr3gltiQfMFrEUGRVpRqLNmk1s2PbzbV9TKwvEJ6WVk9NK6GWM949hs4eMHic0jQWm4PGFv8AGOyBi1XC+nlqWtjlFptghhhfMOMSvjYHOaeMAgEEg3BUWRTctVMJRqSU5wTa5m0m0FvsmcsMRw5jmUlRmxPOc6F7I5YS7jcGSNIY7QLltr2F7rQopK6tGFWObUipLqaubfKbKWtxJ7X1k5lzARGwBscUYPDmRxgNB4Be1zYXOgLFrcUnmhpqeV4dFRiUU7MxjdjE72yTXc0Zz7ua098Ta2iywkUXIhh6cElGKSjzJLm2NbOrnfE2OB43VUTpXU0zotmifBKLNc2SN3C1zHgtvw2da4ubEXN83JrK/EMPY6KlntC43dBJHHNCT4wyRpzDwaW2vbTdaFEuU1MLSqXz4p3te6W23Nfw6DaZR5Q1mIyNlq53SljcyNuayOOJhtdsUcYDGXsLkC5zRcmwWyyfy8xShiEEFSHQt0xxTRxzNjPD+jMjS5o08ANtPAoyiXIlgqEoKDgnFcysrLwRn49jFVXzGermdPKQG5z7ANaL2YxjQGxsuSbNAFyTwlMYxeorDCaiQSGngZTRWYxmZDHfNZ3jRnWudJuVgIly5GhSja0Us3m2c3h1GThVdLTTRVELsyaF4kifmtdmvbwHNeC0/gQpQeydjJJJqYySbkmjo7knhJOw8KhyJct1sHRrPOqQUnzXaTNqMoKvYq2ISgR4jKJqtojjGyyNlMrSCG3jAkcTZlhxcC8sFxqpotm7XkDW1EWwzxvaySKaM8LZIpGlruE2NrjOdY6TfXolyrRqWa45qs7XVue1kr/JLgjNwHFZ6GeOppZDFNFnZj7NdoexzHBzXAteC0nQQeI8IC8KyokmlkmlcXyzSPlkeeF8kji97jbRpcSV4opK/VQzs+yva1+m3VfqNnV4/Vyto2uncBh7NjpHRhsb4G3Z4MkYDi67G98SToW8qOyVjT4yw1YDnMzHTMggjqHMta2zMYC38W2PnUQRRcsTwNCds6Cdr2ulsu7vo6XtNzkxlPW4bspopRFs4jEudFFMHiMvLARMx1rF7+DxrOxXL3FKqCSCeeN0Urc17RSUrCW3B0PZGHN4OEFRhFNxLA0JTz3BOWzbZX2c23uJVh/ZCxanhigjqGCGCNsUTX0tLIWRtAAbeSMk6AOErW5R5T1+IlvblS+ZrfAZZkcTTa1xDE1rM62i9r6eFadFFxDA4eE8+MEpddlfiCFMYuydjTWtYKpmaxjWMvSUjiGNFmi7ormwUORCuvhaVe3rYqVua6TtxJHRZb4lDU1FXHO1tRVMZHO7YKctkZGA1rdjLMxui3ABde2I5f4rUwy0808bopWOjkaKSlYXMeC1wDmRBzdBOkEFRZEuW3k7DNqXq1dWs7Lo5uHQZ+PYxUV0xqKqTZZSxjM4MYxoZGLMY1jGhrGgX0AcZ8aYZi9RTR1MMMmbFWRbDUMLWObLH32b4TTmuGc6xbYjONisBEuXvUU8xQzVmq2y2zZzbO42GLYxUVbadk8gkbSwtp4BmMZmRM8Fl2NGf+Lrlfc2PVT6empXyh8VHIZKUPjjc+Ek3LWSFufsd7d6SW963RoFtYiXI0alZLNVltWzme27XEy8ZxKesnlqah+yTyuBkeGtZnENawHNYA0d61o0DiXo/Gqk0baAyk0jJu2GRENIZKWuBcxxGc1pz3nNBtdxNrm6wEUlXqKbSjZWjaytzW5rdVjZ1uP1c0lJLLKHPoWQx0xzGN2NkDg+IENbZ9nDhde/HdYmJ10lTNJUTOz5ZnOfI/Na3OedLjmtAaPwAWOii4hQpwtmxSts5uhu7XHabGfGqqSkhoZJi+lhldNFE4NOxyODwc19s/N79/e3tdx0LKybysxDD2ubS1BZG83fC9rJoXHjJilaWhxsLkWJsNOgLSIlymWFoyi4OKae1qys31+PeSLHstsTrojBNOG05tnQQRRU8TrWPfNiaC8XANnEjQNGhR1EUE0aFOjHNpxUV1JWMvayp1afkJeim1lTq0/IS9FdQ+tPWvCa4z7NcTzGs0txcfwcvbWVOrT8hL0U2sqdWn5CXorqH1p601xn2a4jWaW4uP4OXtrKnVp+Ql6KbWVOrT8hL0V1D609aa4z7NcRrNLcXH8HL21lTq0/IS9FNrKnVp+Ql6K6h9aetNcZ9muI1ml2a4/g5e2sqdWn5CXoptZU6tPyEvRXUPrT1prjPs1xGs0txcfwcvbWVOrT8hL0U2sqdWn5CXorqH1p601xn2a4jWaW4uP4OXtrKnVp+Ql6KbWVOrT8hL0V1D609aa4z7NcRrNLcXH8HL21lTq0/IS9FNrKnVp+Ql6K6h9aetNcZ9muI1mluLj+Dl7ayp1afkJeim1lTq0/IS9FdQ+tPWmuM+zXEazS3Fx/By9tZU6tPyEvRTayp1afkJeiuofWnrTXGfZriNZpbi4/g5e2sqdWn5CXoptZU6tPyEvRXUPrT1prjPs1xGs0txcfwcvbWVOrT8hL0U2sqdWn5CXorqH1p601xn2a4jWaW4uP4OXtrKnVp+Ql6KbWVOrT8hL0V1D609aa4z7NcRrNLcXH8HL21lTq0/IS9FNrKnVp+Ql6K6h9aetNcZ9muI1mluLj+Dl7ayp1afkJeim1lTq0/IS9FdQ+tPWmuM+zXEazS3Fx/By9tZU6tPyEvRTayp1afkJeiuofWnrTXGfZriNZpbi4/g5e2sqdWn5CXoptZU6tPyEvRXUPrT1prjPs1xGs0txcfwcvbWVOrT8hL0U2sqdWn5CXorqH1p601xn2a4jWaW4uP4OXtrKnVp+Ql6KbWVOrT8hL0V1D609aa4z7NcRrNLcXH8HL21lTq0/IS9FNrKnVp+Ql6K6h9aetNcZ9muI1mluLj+Dl7ayp1afkJeim1lTq0/IS9FdQ+tPWmuM+zXEazS3Fx/By9tZU6tPyEvRTayp1afkJeiuofWnrTXGfZriRrNLcXH8HL21lTq0/IS9FNrKnVp+Ql6K6h9aetNcZ9muJOs0txcfwcvbWVOrT8hL0U2sqdWn5CXorqH1p601xn2a4jWaW4uP4OXtrKnVp+Ql6KbWVOrT8hL0V1D609aa4z7NcRrNLcXH8HL21lTq0/IS9FNrKnVp+Ql6K6h9aetNcZ9muI1mluLj+Dl7ayp1afkJeim1lTq0/IS9FdQ+tPWmuM+zXEazS3Fx/By9tZU6tPyEvRTayp1afkJeiuofWnrTXGfZriNZpbi4/g5e2sqdWn5CXoptZU6tPyEvRXUPrT1prjPs1xI1mluLj+Dl7ayp1afkJeim1lTq0/IS9FdQ+tPWmuM+zXEnWaW4uP4PxEReKPLhERAEREB+gE6ALk6ABwkrYRYPKRclrfMSSfXmiy/Mn2AzXP7LC4fjcN/kSpGvZZAyDRxVF1q23a0ktnN0swcRiJQlmxI/tHL95n59FNo5fvM/PoqQIvQ6rYHdfFmPpcyP7Ry/eZ+fRTaOX7zPz6KkCJqtgd18WNLmR/aOX7zPz6KbRy/eZ+fRUgRNVsDuvixpcyP7Ry/eZ+fRTaOX7zPz6KkCJqtgd18WNLmR/aOX7zPz6KbRy/eZ+fRUgRNVsDuvixpcyP7Ry/eZ+fRTaOX7zPz6KkCJqtgd18WNLmR/aOX7zPz6KbRy/eZ+fRUgRNVsDuvixpcyP7Ry/eZ+fRTaOX7zPz6KkCJqtgd18WNLmR/aOX7zPz6KbRy/eZ+fRUgRNVsDuvixpcyP7Ry/eZ+fRTaOX7zPz6KkCJqtgd18WNLmR/aOX7zPz6KbRy/eZ+fRUgRNVsDuvixpcyP7Ry/eZ+fRTaOX7zPz6KkCJqtgd18WNLmR/aOX7zPz6KbRy/eZ+fRUgRNVsDuvixpcyP7Ry/eZ+fRTaOX7zPz6KkCJqtgd18WNLmR/aOX7zPz6KbRy/eZ+fRUgRNVsDuvixpcyNVGFSsGdocBw5pNwPHYgX9SwVMyFFMSYGzSAaAHaPXp/7ry/pDkSlg4xq0m7N2ae3bzpoy8NXc3ZmOiIvKGWEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAbPJ37V3oH4mKQqPZO/au9A/ExSBdS9FPcV4v7mpxntsoHGMerhU1LRW1LWtqJ2taJ5AGtbK8AAB2gAACyxe6Cv12p5xL0lj43+tVX8VUf1nrEXXadClmr9K5l0HF6+Lr+sl+t876X1+Js+6Cv12p5xL0k7oK/XannEvSWsRVaPS3VwLOm19+XF+Zs+6Cv12p5xL0k7oK/XannEvSWsRNHpbq4DTa+/Li/M2fdBX67U84l6Sd0FfrtTziXpLWImj0t1cBptfflxfmbPugr9dqecS9JO6Cv12p5xL0lrETR6W6uA02vvy4vzNn3QV+u1POJekndBX67U84l6S1iJo9LdXAabX35cX5mz7oK/XannEvSTugr9dqecS9JaxE0elurgNNr78uL8zZ90FfrtTziXpJ3QV+u1POJektYiaPS3VwGm19+XF+Zs+6Cv12p5xL0k7oK/XannEvSWsRNHpbq4DTa+/Li/M2fdBX67U84l6Sd0FfrtTziXpLWImj0t1cBptfflxfmbPugr9dqecS9JO6Cv12p5xL0lrETR6W6uBOm19+XF+Zs+6Cv12p5xL0k7oK/XannEvSWsRNHpbq4DTa+/Li/M2fdBX67U84l6Sd0FfrtTziXpLWImj091cCNNr78uL8zZ90FfrtTziXpJ3QV+u1POJektYiaPS3VwGm19+XF+Zs+6Cv16q5xL0lY/YaxCedtYZppZix8Qbskjn2Ba++bnE2vYKpVaPYL8Ct9OH4ZFr8q0YRoSaSXN0d6N56PYmrPGxUpNrbsbfUyylF8X+3k9IfCFKFF8X+3k9IfCFyX0x92h/t/DOrYL234GIiIucGyCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgNnk79q70D8TFIFH8nftXegfiYpAupeinuK8X9zU4z22c343+tVX8VUf1nrEWXjf61VfxVR/WesRdipeyvBHDsR/ll4v7hERXC0EWmyiyno6DRNJeW1xBHZ8pvwEtvZgPjcR5rqAYx2SauS4po46Zumznfppfxu4ZjfwzT+Kmxm4fJ9asrxVl1vYvyWwvl8jW+E5rfxIH81QVdjlbPfZaqd9/2dle1nJtIaPYtc4X0nSfGdKmxs4ZCf8A8pfT8nRrZmHgew/g5p/kV6WXNuaPEPYsqkxCohsYp5YreTlkZ8LgmaTLIXVP6fk6HRU1hWX+IwkB8jKln3ZmDOt5pI7Ov53ZynGTuX1HUkMlvSSmwAlcDE4+Js1gB/uDfWosa+vkuvSV7Zy615c5LURFBrgiIgCIiALRYzlAInGOJoe9uhznXzGuHC0AaXH1hb4KB5PSRipY6YjN76xd4IeR3pcT5+Px2VMmZ2CoxnnTkr5q5uvnFRjlU7/zS3zMDWfyF1jOxOo8vLyj/mt9ltNCWRgFrps6/ekEiPNN84jgBObYeYqKEq3J2N3hYwnBSzEu6xnx41VN4J3n0rPHseCtrhuVbgQ2oaC3yjBZw87mcDvVb8CoySvglUZ7Reng6VRWcV4rYWpFI1zQ5pDmuALSNIIOkEL6WjyHkJpLE3zZXtb5h3rre1xW8WRF3Vzyten6uo4dTsERFJaCtHsF+BW+nD8Miq5Wj2C/ArfTh+GRazK/u0vl90b30a9+j/y+zLKUXxf7eT0h8IUoUXxf7eT0h8IXIfTH3aH+38M67g/bfgYiIi5wbIIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiA2eTv2rvQPxMUgUfyd+1d6B+JikC6l6Ke4rxf3NTjPbZzfjf61VfxVR/WesRZeN/rVV/FVH9Z6xF2Kl7K8EcOxH+WXi/uFX/ZAy3MLn0lE4bK0ls1QLERngMcV9BkHG79ngGnS3edkXHDRUZMZtPOdiiPGy4vJIPO1vB/mc1UmryRt8k4CNT+7U2roXX3n09xcS5xLnOJLnOJLnE8JcTpJ85Xyin+QuQmztZU1oc2JwDoqcEtdI06Q+UjSxh4mixPDcDQZub/EYiFCF5f8A73IheF4bUVTiynhkmcOHMbcN9N3gsHpEKVUHY2rn2Mr4YB4i50rx/tYM3/qVr0tPHExscTGRxt0NYxoa0fgBoVY45l/Xw1VTCxtNmRTzRMzopC7Njkc1ucRKATYDiUXNRTyhXxUnGilG3Xz+RmRdi5v7dcb/AOWnA/nKV+ydi5n7Na4elTg/ylC026TiX3aXkZOuTdJxL7tLyMnXJtK/VZQ3l9PIyqzsZVTReKphk8zxJCfVYPF/xIUYxrJ2towTUU72s8q20kXmvIwkN/3WW93R8R+7S8jJ1yy8P7JtQDappoZWHQdhz4nWPDokc9rtHFoUl6nLGw9pKXzs/I12ROWctCWxTF01JwZvC+D/ADRE8Lf8h0eK3HcFNOyVjJI3B8b2hzHtN2uaeAhVblDgdJXUz8QwrQY9NTSAZpbou5zYx4DwBewu1wBLdI05HYfxwtkdQSG7Hh0tPf8AZe0F0sY8zmgvt42O+8oaMHHYeFam6tNWlH2o9Pz+5ZyIipNAEWtx/E+14xm2Mj7hgPAAOFx/C/B4yoVVVMkpzpHuef8AMbgeYDgA/BUuVjPwuAlWWdey+5Y6h/ctUeUh96ToLQGy+D+AVDmbTD4CVG+ZLn7iQdylR5SH3pOrWJiuAzU8Zle+MtBAs0vJu42HC0aFqDbxD2L5KtuSMyFKrfbJNeH5BK+SUJXwSrTZmJE+7H7HOpiGtLjs0mhoLj4MfABpUjNJKNJikA88b/ksPsGHSP8AmT/0mK3SV4j0l9Np5HxMaEaamnFSu21z3VuYv5L9EoZUjOs5uLUmrWvzWd+cqhFNcqsHZKx00bQJWAuOaPtGjSQRxutpB81vwhS9N6O+kNHLGH9bTWa07Si+dPyfQzy+W8iVcl1vV1HnJ7YyXM1/DXSgrR7BfgVvpw/DIquVo9gvwK304fhkWZlf3aXy+6K/Rr36P/L7MspRfF/t5PSHwhShRfF/t5PSHwhch9Mfdof7fwzruD9t+BiIiLnBsgiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIDZ5O/au9A/ExSBR/J37V3oH4mKQLqXop7ivF/c1OM9tnN+N/rVV/FVH9Z6xFl43+tVX8VUf1nrEXY6XsrwRw+v/AJZeL+5VnZpmJqqWPiZTuePxklLXflG32KBKx+zTRG9LUgd7Z8Dz4j9pH7f0vsVcK8j1uS2nh427zfZA4W2rr4Y5AHRMzppGngc2MXDSONpeWAjxEq8lS3YvrWw4lEHGwmY+C54A59nM9r2Nb+LgrpVLNLlxy9ak+a2z+QqByp/X67+Lqf6z1fyoHKn9frv4up/rPUxLmQ/8kvBGtW/psjcSkYyRlMXMka17DssAu1wDmmxkuNBHCtAruwDKCgZSUrHVtM1zaaBrmuniDmubE0FpBdoIIIspZtcfiatCKdNXv3N/YqjFsmq6kZss9O9kdwC8OY9rSTYZxjcc25sLnjIWoVv5c5T0PaNRFHURzyTRuiYyJ7ZLF+jPcW3DQ3wtP3RZVAiK8BiKlaDlUjmu/hfiSPsb4i6nxGAA95O4U8jeJwk0MJHBcSZp9vjK+xT9pY42OPQ2OvjawDiime2zR5hHJZYmQlK6bEqNrQe9mbK63E2H9ISfF4IHrC3kEXbuUbyzSyOqMjiOANo2tbf8DJG0f7woLddqNSb6M27/AILZREVJ44g+VU+fVPHFGGsHqF3f9TnLUuKy8bd/4mf/AJ0nxFYJKsSZ6/DwzacV3IEr5JRxW+wrJWecNLjsef4LMwySu/2C1vbfzK3KSW1l2dSMOd/97iPEr5JU/b2Maoi/6X1wtafY6W6HsXVfjk5OPrVrZZVwa2OtD968y+lUtspz/ZLyK9JXw4qw9y6q8cnJR9atlg3Y0ex4c+J0jgbh07mNjafHmNJJ9ecrc8sYGKvKvBL/AHXmVZlZ7I0pt9WY/wCVYzewtQvjEZcCDmSzOB4hJmsYD5y2x9viVola3AsKbSxkA50j7GR/BcjgAHE0XPtKzrrhHphlenlLHyqUvYilGL5rpX227+g6L6NZOqYPCZtXZKTcmupvo+XSfriq1xGHY5pWDgbI8D8LnN/KysdxVe47+sz/APMcvU/0sqyWKrQ6HFO3enZHmv6jUo6PSn0qTXyauzCVo9gvwK304fhkVXK0ewX4Fb6cPwyLrmV/dpfL7o5/6Ne/R/5fZllKL4v9vJ6Q+EKUKL4v9vJ6Q+ELkPpj7tD/AG/hnXcH7b8DEREXODZBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQGzyd+1d6B+JikCj+Tv2rvQPxMUgXUvRT3FeL+5qcZ7bOb8b/Wqr+KqP6z1iLLxv8AWqr+KqP6z1iLsVL2V4I4diP8svF/cwcewyOsp5aaTwZG2DhpLHg3Y9vnDgD59I41ROM4bNSTPgnbmvYeH9l7T4MjD+0wj/uDpBC6EWpymyfp8QizJm2c2+xTNtskRPDY8bTou06DYcYBF1Mzsm5Q0d5svZf07/MoYG2kEgjSCNBBHAQeIq28hcto6lrKereI6oANa91msqOIEHgbN426Lk6OGwgWU2SdXQEuezZYBwVEQJZb/wCIOGE+lo06CVoOFVHoq9GjjKfPfqa6P+9KOkFQOVX6/XfxdT/WetjgOWlfSAMEgmiFgIp7vDR4mPuHtHmvYeJaXFKozzzTluaZpZJS0G4aZHl5aDbTa6hKxi5OyfUw1STlZprYzGRFM9iydNOwGWeOoMTM57G1L82bMGec1zcxwz76OC3ARwqTZVauZbY3fqV7eJDFtMHyerau2wU73MJtsrhmRDxnZX2abeIXPmWscLE2Nxc2PBfz2PAsnDsRqKcl0E0kJIIJje5twRbSBoP48XEhVUz839Fr9/MTiQQ4DA9jHtnxapaGXYM4U7HEWzRa9r2IB0vcGmwAUi7G+TRoYDJMLVM4BeDpMUY0tiJ+9c3d57D9m61/YvwKldE3EHPNRUvc+5fp2CQHvhYkl0pBBzzps4WtfTPFSzyuOxNs6mm22/1Sey7XQl1IIiKDVFf5Sx5tXOPG/O98B3/dawlSvLjDyc2oaL5ozJbcQvdj/wANJBPoqIkrGnsZ67BVFUpRa6rPxRtclaYS1TA4XawOlIPHmWzfVnFq6LyYwplNC05o2Z7Q6V58K7tOYDxNHi8Yuue8hf1p/wDDyfHEumLrmP8AUrHVaOHpUYNqNRycu+1rLw23sep9FsLCri6k5q7hGKj3Xvd+OznBK+CUJXySuMNs6Ooo/CV8Er9JXwSqM4uJH4SvlxQlfBKoZdSPxxVe4tIHTzOHAZH29Rt/2Uux3EBBGdI2RwIjbx34M4j7o+Sg67P/AEwyXUpqpjJq0ZJRj322tru6LnKP6i5RpydPCwd3G8pd19iXj02CtHsF+BW+nD8Miq5Wj2C/ArfTh+GRdKyv7tL5fdHjPRr36Pz+zLKUXxf7eT0h8IUoUXxf7eT0h8IXIfTD3aP+38M67gvbfgQivy7popZYXQzl0Uj4iWiLNJjcWEi8l7XHGvDdCpfIVHsh6xTDDsFo5JryUlPIXZznF0ETy5x0lzi5ulxOm5W6GTOHahSc1g6C81k7IqxdL1idttunuNvPF4eGxwb+ZW26HS+QqPZD1ibodL5Co9kPWKye5nDtQpOawdBO5nDtQpOawdBbDVf4l9SnT8N2b/cVtuh0vkKj2Q9YvzdCpfIVHsh6xWT3M4dqFJzWDoKmuyphcFJiTo6eMRRvgjm2Nosxr3ulY4MbwNb+jBtwC5WJjcgrDQz277bdJmYGphsTU9XmNbL85vd0Kl1eo9kPWJuhUur1Hsh6xVwi1WjQNzyZQ6nxLH3QqXV6j2Q9Ym6FS6vUeyHrFXCJo0ByZQ6nxLH3QqXV6j2Q9Ym6FS6vUeyHrFXCJo0ByZQ6nxLH3QqXV6j2Q9YvWny+o3Os9k8Q++5jHNH4iN5d7AVWaJo0CHkqj1PiXrTTskY2SNzXscLtc0gtI8xC9FAOxPWPzqinJJYGtmaPuuzgx9vxu33VP1g1YZkrGgxNH1NRw57BERWzHCIiAIiIAiIgCIiA2eTv2rvQPxMUgUfyd+1d6B+JikC6l6Ke4rxf3NTjPbZzfjf61VfxVR/WesRZeN/rVV/FVH9Z6xF2Kl7K8EcOxH+WXi/uERFcLQUexjIvDqolzoNikN/0lOdiNzxlg/RuPnLbqQohcp1Z03eDa8Ctq3sX+QrNHE2aLT63xu0+6q/xCldBNLC4guhlfE4tvml0biwltwDa4XRKpbKPJyvfWVb2Uc7mPqZ3tcIyQ5rpXlrgeMEEFVJnoMl5QnUk1Vkti2XsiMKd4R2N5po45ZKqONsjGSAMjfI4Ne0OAOcWgOsfOo53L4lqNRyRV24NG5lNTMcC1zaeBrmnha5sTQ4HzgghGy9lPHSpRj6qS23vzMrPLXIdlFSiogkklzHgT5+bYMfZrXsaxtwA+wNyfD8ygy6MqYGSsfHI0OZI1zHtPA5rgWuB/EEqlMUyOr4ppY46aaaNjyI5WMLmyM4WOuNF80i44jdEynJmUPWRcastq23ey6/Bl9jLH+06rYpHWp6ktY6/AyXgjl8w05p8zgf2Vcioc5LYjqNRyRVuZEVVVJSMbWQyRTw/o3GVpaZWgDMkBPCbaD52k8YUMxMr0YNqrBp9DV/qbxERQaIEX0HSDoIOkEHhBHiWkqsl6R7s4B8d+ERuAb6g4G34CwW7RQ0nzlylWnT9htGpwvAYaZ5kjdIXFjmd+5pFiWk6A0ae9CvG6qN3AfwVs3XIP6qJJYe3x/8AqdO/p5UlUlXc3d/o/wDYF2lYOLicx/8AhixsmcNL/BzdN+I6eBZhK+HFckoVvU1IzspWd7SV4vuaOlVaHrabhdxurXTs14MjxZi3lIPy6C/CzFfKQfl0Fs8ZrdghfKBnFuaAOLOc4NF/Nc39S1GT2NyTyGOVrb5pc1zARwEXBBJ8a9vhauMxGDnjYYag6dPY3mK+yzdlfoPJYilhKGLhhJ16ynPalnu226V3bp6DBxHFa6B+ZI+POLQ7vWtIsSQNJA06CsR+P1RFtkA84Yy/5gr2yx/WR/ym/E9aVdSyBkfJ+LwNLEVMPTzpRUnaCtfuRzvLWVMdhsZUoU6882MmleTvbxPuaVzyXPcXOPCXEk/mvhEXr6dOFOKjFWS2JLmXgeWnNzblJ3b52+kK0ewX4Fb6cPwyKrlaPYL8Ct9OH4ZFr8r+7S+X3Ru/Rr36Pz+zLKUXxf7eT0h8IUoUXxf7eT0h8IXIfTH3aH+38M67gvbfgfeCfbD0XfyW+VP1OUmMwVM2w0gexksrInGkqXh0YeWsdnMeA67QDcaCvvu5yg1BnM6vrVq8i5Qo4bD5lR7bt8bGyq5NqTecmubrLeRU5U9kLG4ml8tJFEwcL5KapjYP9z5gFr91+t8dB7sn1C3cMq0pq8U34RbEciYme2KT8GXmqQ7Nv/tRv8HB/WqF57r1b46D3ZOvUSx3KB1dO6oqJ43SOAHeua1rWt0NYxt9DRp8fCeG6wcp4lVqWZCLvdPmZtMlZIxFCtnzWyzR4ovDtuLysfvt+adtxeVj99vzXn9Gq7r4PyPT5j6j3ReHbcXlY/fb807bi8rH77fmo0aruvg/IZj6j3ReHbcXlY/fb807bi8rH77fmmjVd18H5DMfUe6Lw7bi8rH77fmvWnljkdmiaEeeSohjaPOXSPAU6NU3XwZDi1tZM+xQP/Ez/wAP/ORtv5H2KyFCcjavC6CJwdiNE+aUgyuFVDmjNvmsbd9y0XJvxlx8yl9BWxTsEsErJo3EhskT2yMJaS1wD2Eg2II9S1eKpTUs5xaXNdpo8nj251XOztsV7HuiIsMwQiIgCIiAIiIAiIgNnk79q70D8TFIFH8nftXegfiYpAupeinuK8X9zU4z22c343+tVX8VUf1nrEWXjf61VfxVR/WesRdipeyvBHDsR/ll4v7hERXC0EREAREQBERAEREAREQBERAEREAUxyeyijLGxVDgx7QGtkce9eBoGc79l1vHoPj4lDkWky7kDD5XoeqrXVtsZLnT7u59KNvkbLNfJlb1tK23Y4vma7+/qZaTXgi4IIPAQbg+sL5JVZRSvZ4D3M9Fxb/IrIGJVHl5OUd81zGt/Suspf2q6a700/odCo/1HpW/uUWn3NP7k+qImva5jwHNcLEHgK1TWUdFnOuGuI4M4vkI+60Ek2v6uC6iclbM7wppD5jI+3susdbLJ39OK9KLp1cS/Vys5Qimk7dd3b6GBj/TyjUkqlLDr1kbqMpWbXhZfyZWK1hnldIRYaGtHiaOD16SfWsVEXUMLhqeGpRo01aMUopdSRzvEYipiKsqtR3lJtt9bfOERFfLIVo9gvwK304fhkVXK0ewX4Fb6cPwyLWZX92l8vuje+jXv0f+X2ZZSi+L/byekPhClCi+L/byekPhC5D6Y+7Q/wBv4Z13Be2/AxF+SvDQXONg0FxPiAFz+S/V4Yj9jN/ypPgcud0leaTNpFXkVDkjgtRlXX1E1TO6GmgDXZjLOdGyUvEMMId3jXWY4ueQb5vAc4WsJvYRwrytZysPUrQ/4Wz/AO1B/Bn1WqFdy7BGKpRUYbEtiSMvLGUK9DEypUpOMY2SS2dCfzKv3EcJ8rWcrF1KbiOE+VrOVi6lWgiZ8jWcrYztJcSr9xHCfK1nKxdSm4jhPlazlYupVoImexyri+0lxKv3EcJ8rWcrF1KbiOE+VrOVi6lWgiZ7HKuL7SXEq/cRwnytZysXUpuI4T5Ws5WLqVaCJnscq4vtJcSr9xHCfK1nKxdSm4jhPlazlYupVoImexyri+0lxKoxDsL4VHG54mrCRbhlitpcB5Hzrc5NYNFQUzKWAvMbDI4GRwc+8j3SOuWtA4XHiUxxn7B3+342qPrw/pRiJutGm3+mydui+1XL8MZXrQtUk5K/SwiIvKEBERAEREAREQBERAbPJ37V3oH4mKQHj/BR/J37U+gfiYpAupeinuK8WanGe2zm/G/1qq/iqn+s9Yi6DkyWw1znOdQ0rnOJc5xp4iXOcbucSW6SSSb+dfPcnhmoUnN4eiujQy3BRSzfsc7qeiVaUm89bfE5+RdA9yeGahSc3h6KdyeGahSc3h6Kr5dhu/Yt6oVt9fU5+RdA9yeGahSc3h6KdyeGahSc3h6Kcuw3fsNUK2+vqc/Iuge5PDNQpObw9FO5PDNQpObw9FOXYbv2GqFbfX1OfkXQPcnhmoUnN4eincnhmoUnN4einLsN37DVCtvr6nPyLoHuTwzUKTm8PRTuTwzUKTm8PRTl2G79hqhW319Tn5F0D3J4ZqFJzeHop3J4ZqFJzeHopy7Dd+w1Qrb6+pz8i6B7k8M1Ck5vD0U7k8M1Ck5vD0U5dhu/YaoVt9fU5+RdA9yeGahSc3h6KdyeGahSc3h6Kcuw3fsNUK2+vqc/Iuge5PDNQpObw9FO5PDNQpObw9FOXYbv2GqFbfX1OfkXQPcnhmoUnN4eincnhmoUnN4einLsN37DVCtvr6nPyLoHuTwzUKTm8PRTuTwzUKTm8PRTl2G79hqhW319Tn5F0D3J4ZqFJzeHop3J4ZqFJzeHopy7Dd+w1Qrb6+pz8i6B7k8M1Ck5vD0U7k8M1Ck5vD0U5dhu/YaoVt9fU5+VodgzwK304fhkUx7lMM1Ck5vD0Vm4bhVNTBwpqeKAPsXiGNkYcW3sXZoF7XPtWHjcqxr0nBRte33Njkr0dqYTERqymmlfYr9KaM1RfFvt5PSHwhShRfFjaeQ/5h/ILm/ph7tD/b+Ge7wftPwIvJlng7SWuxbDWuaS1zXYhSAtcDYggy3BBFrL57tsF/e+Gf8A3Cj61Vy3/DPhlXUTPdidewyvlmIa2ksHSSZxaM6E6AXFZe9Qwn97Yl7tF1C1mC9H8HiKaqQqu3h0kTxdaLtmriaGXFmYFWyVeD4rhs9NICDE2uo5bRk5whmgEwe9rT4L2G9hwi5vtD/iFqvFhnKP+oWTvT8J/e2Je7RdQm9Qwn97Yl7tF1C9pQjCMFGUs5rptZvxM6WWpVbOtQjKS2Z12m/G3OY2+Eqf9M99/XpvhKn/AEz339esreoYT+9sS92i6hN6hhP72xL3aLqFdvS6/oU8p0//ABo/uZi74Sp/0z339em+Eqf9M99/XrK3qGE/vbEvdouoTeoYT+9sS92i6hL0uv6DlOn/AONH9zMXfCVP+me+/r03wlT/AKZ77+vWVvUMJ/e2Je7RdQm9Qwn97Yl7tF1CXpdf0HKdP/xo/uZi74Sp/wBM99/XpvhKn/TPff16yt6hhP72xL3aLqE3qGE/vbEvdouoS9Lr+g5Tp/8AjR/czF3wlT/pnvv69e+H9nXEKh4jp4aCeQ8EcImlkP4MjmLivveoYT+9sS92i6hanKn/AAoQime7DMUmfVMBdHDXRQGGZwFxHssDWugcTbvyHgeLjC9N8z+g5Upr/wCtH9zJLVdkvKF7S12D2Gi9qKvvoIPG7zKZZGYjU1VHHPVwdrzudKHRbHJFmhkrmMOZKc4XaAdPjVPf4XcvqypkqMDxF8ks1NE6amknJdOxsEjIaikme7vnFjnsLc4kgCQXs1oF9LnXpJXl610qkEpK1pJvbHbb5GVHGUa9JerpqO3obv3raERF5coCIiAIiIAiIgCIiA96GpMUgfa40hw8bTw+vj9SkUWIQuFxI0eZxDSPxBUWRb3JWXa2Bi4RSlF7bPofcWK2HjUdyWduReVZ77fmnbkXlWe+35qJotvrlV3FxZZ0JdZLO3IvKs99vzTtyLyrPfb81E0TXKr2a4saEuslnbkXlWe+35p25F5Vnvt+aiaJrlV7NcWNCXWSztyLyrPfb81+9txeUZ7zfmokia5VezXEaEuslvbcXlGe835p23F5RnvN+aiSJrlU7NcRoS6yW9txeUZ7zfmnbcXlGe835qJImuVTs1xGhLrJb21F5Rnvj5r8NZCOGaMfjIwf91E1rqw3efYtzkXL1TH1XTcFFJN3Tb6kWK+HVON7k97eg8tFyjPmnb0HlouUZ81XqL1NjFLC7eg8tHyjPmgrIjwSsP4Pb81XqzsPPekeI/zC1WV8dLBUPXRjnWaVn39Jdo01OVmTXtuLyjPeb807bi8oz3m/NRJF5LXKp2a4mZoS6yW9txeUZ7zfmnbcXlGe835qJImuVTs1xGhLrJZ25F5Rnvt+aduReVZ77fmomia5VdxcWNCXWSztyLyrPfb807ci8qz32/NRNE1yq9muLGhLrJZ25F5Vnvt+aduReVZ77fmomia5VezXFjQl1ks7ci8qz32/NO3IvKs99vzUTRNcqvZrixoS6yS1OJRMF88PPE1pBJ9mgD8VHZpC9znHhcST618L9AubLR5UyzWyg0pJRS5kuvrZfpUI0zOwH7U+g7+bVviqGpv8Q2TtNPKyU1udGZIXZtJcZ7H5rrHZNIu06Vn75rJjx1/Mv7i9nkDB1qeFUZRs7t2aNfXr03PZJF1IqV3zWTHjr+Zf3E3zWTHjr+Zf3FuvUT6i166HWXUipXfNZMeOv5l/cTfNZMeOv5l/cT1E+oeuh1l1IqV3zWTHjr+Zf3E3zWTHjr+Zf3E9RPqY9dDrLqRUrvmsmPHX8y/uJvmsmPHX8y/uKfUT6mPXQ6y6kVK75rJjx1/Mv7ib5rJjx1/Mv7ieon1Meuh1l0r6bwhUpvmsmPHX8y/uLV5Uf4o8Gip5Dh1NV1VWWkQtnhbT07X2Oa+eQyF5YDxMaSeC7eEFQn1EOvDrKu7An/vxinp49/8AsWrqJc7f4U8k6x9VVZQ1ge1tRHNFTOe3MNXJVTMnqqsNsP0V2ANcNDjI+2hq6JXPvSqrCpjHmu+akn4q918jNydFqlt6W2ERF5ozwiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiKQEREswERecszW8J0+IcP/8AFeoYepWlmQi5PqSIlJRV2fUrw0Eni/M8QWrJubnhK9J5S86eDiH/AK415rp/o9kh4Kk3P25Wv3Lq8zV4it6x7OZBERehMYL2o5M12ngOj5H/ANeNeKLGxmFhiaMqc+aSt4dT+XOVQk4tNG2RYdNU20O4OI/NZbXA6Qbhckyjkmvg55s1s6JdD+f8G4p1YzWw/URFrbMuBERLAIiKAEREAREQBfoK/EQEIruxfk0NlnmwqkA/STTSP2UADTJJI47JweESuecsn4VUzOGH4VTUVK136MtY81EoB0Plc95zAeHMaBa9iXLprsnEjBcUtqU49RYQR7CR61ykuh+ianWjKrOblZ2SbbS6b2vYxalGnurgjF2tp/IR+435JtbT+Qj9xvyWUi9oUeqh1LgYu1tP5CP3G/JNrafyEfuN+SykQeqh1LgYu1tP5CP3G/JSnIXsXVeNCV1DRwGKFwZJNMWRRCQgOEbTYl8maQSADYEXtcX0C6h/wpj/AIJN/wDMp/6FKrdWbjG6KJwgley4FS73XGNXw7nH9lN7rjGr4dzj+yutkWJpMyxZdS4HJO91xjV8O5x/ZTe64xq+Hc4/srrZE0mY2dS4HJO91xjV8O5x/ZXpTf4esaje2RkGHNewhzXbODZw4DZ0BFwuskTSZdxOzqXA5ylyDy4Y0uOJd60cAxKTgHEBsdlYPY/pK+DD4osSlM1Y10xkkMpmJa6Z7ohsh4bRlg81lYmI/ZP9A/yUaXhvSisouNKMUk/1XSs77Vz9Rm4eTkrsIiLyBkhERAEREAREQBERAEREAREQBERAEREAUgwimjdCxzmNJOdcloJ8Nw4wo+thR4o6JjWBgIbfSSb6STxfit7kDFYfD1pSr+y4tc19t0Y+IhKUbRN52lD5JnuN+SdpQ+SZ7jfktRt2/wC432n5Jt2/7jfafkvXcuZK6l+38GHo9X/rNsaGE8MTD/sb8l87XU/kIuTZ8lq9vH+Tb7XJt4/7jfafkrsPSPJ0NkXbwTX8EPD1Xz/c2m11P5CLk2fJNrqfyEXJs+S1e3j/ACbfa5NvH+Tb7XK7rPgN98H5EaLU6jabXU/kIuTZ8k2up/IRcmz5LV7eP8m32uTbx/k2+1yaz4DffB+Q0Wp1G02up/IRcmz5JtdT+Qi5NnyWr28f5Nvtcm3j/Jt9rk1nwG++D8hotTqNptdT+Qi5NnyX63D4BwQxj8I2j/stVt4/ybfa5NvH/cb7XfJUT9JcnyVnK68H5E6NU6vqbftOHyTPcb8k7Sh8kz3G/Jajbt/3G+0/JNu3/cb7T8lY5cyV1L9v4J0er/1m27Th8kz3G/JRzEGgTSAAABxAA0ADzLN28f5NvtctdUS573PIsXG9gtD6QZQweIpRjh7XTu9ltm3uL+HpTjK8jzREXkjNCIiAIiIAiIgNRlth8lXhtdTRC8s9JPHGODOkMZzG34ruAF/OuR3NIJBBBBLXNIIc1wNi1wOkEEEW8y7PUKyy7GeGYnI6d7ZKapfpfNTFrdlPjlje0sc7/MAHHjJXqfR7LdPA51Oqnmyd7rofNtXUUTjc5kRXtuGUev1XJwdFNwyj1+q5ODor1mtGA3nwfkW8xlEor23DKPX6rk4Oim4ZR6/VcnB0U1owG8+D8hmMoldG/wCFrKGijw2po5amKGoZWST7HLIyMvikhgaJI88jPAdG4G3Bova4vqNwyj1+q5ODor8PYLojw19Uf/pwdFUVPSXATVs58GUypOSsX1t1R63Tc4h6S/NuqPW6bnEPSVC7hFDr1TyVP0U3CKHXqnkqfoqxy9k/ffBlvRmX1t1R63Tc4h6SbdUet03OIekqF3CKHXqnkqfopuEUOvVPJU/RTl7J+++DGjMvrbqj1um5xD0k26o9bpucQ9JULuEUOvVPJU/RTcIodeqeSp+inL2T998GNGZeOJ41RCGQmspgMw3JqYQB/wBS0tLUxzMEkUkcrDez4ntkYbGxs5hINiCPUqo3CaHXqnkqfoqw8isno8LooqKOR8rInSuD5A0OOyyvlNwwW0F5HqXmsv4rCYrNnSm3JbLWa2bXe7L9GDhsNyiIvMl4IiIAi4l32+UepYRzeu+uTfb5R6lhHN6765eo1Sxnw8X5GNpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0ztpFxLvt8o9Swjm9d9cm+3yj1LCOb131yapYz4eL8hpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0ztpFxLvt8o9Swjm9d9cm+3yj1LCOb131yapYz4eL8hpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0ztpFxLvt8o9Swjm9d9cm+3yj1LCOb131yapYz4eL8hpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0ztpFxLvt8o9Swjm9d9cm+3yj1LCOb131yapYz4eL8hpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0ztpFxLvt8o9Swjm9d9cm+3yj1LCOb131yapYz4eL8hpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0ztpFxLvt8o9Swjm9d9cm+3yj1LCOb131yapYz4eL8hpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0ztpFxLvt8o9Swjm9d9cm+3yj1LCOb131yapYz4eL8hpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0ztpFxLvt8o9Swjm9d9cm+3yj1LCOb131yapYz4eL8hpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0ztpFxLvt8o9Swjm9d9cm+3yj1LCOb131yapYz4eL8hpdM7aRcS77fKPUsI5vXfXJvt8o9Swjm9d9cmqWM+Hi/IaXTO2kXEu+3yj1LCOb131yb7fKPUsI5vXfXJqljPh4vyGl0znlERdNNUEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQH//2Q==", 50 | "text/html": [ 51 | "\n", 52 | " \n", 59 | " " 60 | ], 61 | "text/plain": [ 62 | "" 63 | ] 64 | }, 65 | "execution_count": 2, 66 | "metadata": {}, 67 | "output_type": "execute_result" 68 | } 69 | ], 70 | "source": [ 71 | "from IPython.display import YouTubeVideo\n", 72 | "YouTubeVideo(\"cfj6yaYE86U\",width=640,height=360)\n" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "✅ **QUESTION:** Answer the following:\n", 80 | "1. What does train a model mean?\n", 81 | "1. What are the differences between attribute, feature, and observation?\n", 82 | "1. What is the difference between Supervised and Unsupervised machine learning?" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "✎ Double click this cell and put your answer to the above questions below:\n", 90 | "\n", 91 | "1. Your answer\n", 92 | "2. Your answer\n", 93 | "3. Your answer" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": 3, 99 | "metadata": {}, 100 | "outputs": [ 101 | { 102 | "data": { 103 | "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAUDBAoICAgICggICAoICAgICAgICwoICAoICAoKCAoICggIChALCAgOCQgIDRUNDhERExMTCAsWGBYSGBASExIBBQUFCAcIDwkJDxkSEhIXFhUVFRgTFRIWFRcWEhcVEhUSFxYdFRUVFRUWFRIVFRYVGBITFRIXFxUWFRUVEhcVFf/AABEIAWgB4AMBIgACEQEDEQH/xAAdAAEAAgMAAwEAAAAAAAAAAAAABgcEBQgCAwkB/8QAURAAAgEDAwICBAoGBwYDBgcAAQIDAAQFBhESEyEHMQgUIkEVFhgyUVNxkpTTI1Vhc7LVM0KBkaGzxCQ2UnJ1lUNjkyVidqOxtTSCorTR8PH/xAAcAQEAAgMBAQEAAAAAAAAAAAAAAgQDBQYBCAf/xABIEQABAwEFAggLBwIFAwUAAAABAAIDEQQFEiExQVEGEyJSYXGBkRQVFhcyVJKhscHwBzM1QoKz0iPRcsLh4vGTorIkQ1Nig//aAAwDAQACEQMRAD8A4ypSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpUk+J1x9ZB95/y6fE64+sg+8/5ddD5J3t6u7uCs+Bzc0qN0qSfE64+sg+8/wCXT4nXH1kH3n/Lp5J3t6u7uCeBzc0qN0qSfE64+sg+8/5dPidcfWQfef8ALp5J3t6u7uCeBzc0qN0qSfE64+sg+8/5dPidcfWQfef8unkne3q7u4J4HNzSo3SpJ8Trj6yD7z/l0+J1x9ZB95/y6eSd7eru7gngc3NKjdKknxOuPrIPvP8Al0+J1x9ZB95/y6eSd7eru7gngc3NKjdKknxOuPrIPvP+XT4nXH1kH3n/AC6eSd7eru7gngc3NKjdKknxOuPrIPvP+XT4nXH1kH3n/Lp5J3t6u7uCeBzc0qN0qSfE64+sg+8/5dPidcfWQfef8unkne3q7u4J4HNzSo3SpJ8Trj6yD7z/AJdPidcfWQfef8unkne3q7u4J4HNzSo3SpJ8Trj6yD7z/l0+J1x9ZB95/wAunkne3q7u4J4HNzSo3SpJ8Trj6yD7z/l0+J1x9ZB95/y6eSd7eru7gngc3NKjdKknxOuPrIPvP+XT4nXH1kH3n/Lp5J3t6u7uCeBzc0qN0qSfE64+sg+8/wCXT4nXH1kH3n/Lp5J3t6u7uCeBzc0qN0qSfE64+sg+8/5dPidcfWQfef8ALp5J3t6u7uCeBzc0qN0qSfE64+sg+8/5dPidcfWQfef8unkne3q7u4J4HNzSo3SpJ8Trj6yD7z/l0+J1x9ZB95/y6eSd7eru7gngc3NKjdKknxOuPrIPvP8Al0+J1x9ZB95/y6eSd7eru7gngc3NKjdKknxOuPrIPvP+XT4nXH1kH3n/AC6eSd7eru7gngc3NKjdKknxOuPrIPvP+XT4nXH1kH3n/Lp5J3t6u7uCeBzc0qN0qSfE64+sg+8/5dPidcfWQfef8unkne3q7u4J4HNzSo3SpJ8Trj6yD7z/AJdPidcfWQfef8unkne3q7u4J4HNzSo3SpJ8Trj6yD7z/l0+J1x9ZB95/wAunkne3q7u4J4HNzSo3SpJ8Trj6yD7z/l0+J1x9ZB95/y6eSd7eru7gngc3NKjdKknxOuPrIPvP+XT4nXH1kH3n/Lp5J3t6u7uCeBzc0qN0qSfE64+sg+8/wCXT4nXH1kH3n/Lp5J3t6u7uCeBzc0qN0qSfE64+sg+8/5dPidcfWQfef8ALp5J3t6u7uCeBzc0qN0qSfE64+sg+8/5dPidcfWQfef8unkne3q7u4J4HNzSo3SpJ8Trj6yD7z/l0+J1x9ZB95/y6eSd7eru7gngc3NKjdKknxOuPrIPvP8Al0+J1x9ZB95/y6eSd7eru7gngc3NKndKVl4bGTXtxDaW8Tzz3EixQxJ85nbsBueyj3liQAASSACa+j3vDQXONAMyd3WulJpqsSldaaG9GCyihV8ncz3VwyjnFav0LWMnYlVbh1ZiDuOZKg7/ADRUl+Tfp/6i7/EyVw832h3VG8tGN1NobkeqpB9yoOvKIGmZXE1K7Z+Tfp/6i7/EyU+Tfp/6i7/EyVi849182T2R/JeeM4txXE1K7Z+Tfp/6i7/EyU+Tfp/6i7/EyU849182T2R/JPGcW4rialds/Jv0/wDUXf4mSnyb9P8A1F3+Jkp5x7r5snsj+SeM4txXE1K7Z+Tfp/6i7/EyU+Tfp/6i7/EyU849182T2R/JPGcW4rialds/Jv0/9Rd/iZKfJv0/9Rd/iZKece6+bJ7I/knjOLcVxNSu2fk36f8AqLv8TJT5N+n/AKi7/EyU849182T2R/JPGcW4rialds/Jv0/9Rd/iZKfJv0/9Rd/iZKece6+bJ7I/knjOLcVxNSu2fk36f+ou/wATJT5N+n/qLv8AEyU849182T2R/JPGcW4rialds/Jv0/8AUXf4mSnyb9P/AFF3+Jkp5x7r5snsj+SeM4txXE1K7Z+Tfp/6i7/EyU+Tfp/6i7/EyU849182T2R/JPGcW4rialds/Jv0/wDUXf4mSnyb9P8A1F3+Jkp5x7r5snsj+SeM4txXE1K7Z+Tfp/6i7/EyU+Tfp/6i7/EyU849182T2R/JPGcW4rialds/Jv0/9Rd/iZKfJv0/9Rd/iZKece6+bJ7I/knjOLcVxNSu2fk36f8AqLv8TJT5N+n/AKi7/EyU849182T2R/JPGcW4rialds/Jv0/9Rd/iZKfJv0/9Rd/iZKece6+bJ7I/knjOLcVxNSu2fk36f+ou/wATJT5N+n/qLv8AEyU849182T2R/JPGcW4rialds/Jv0/8AUXf4mSnyb9P/AFF3+Jkp5x7r5snsj+SeM4txXE1K7Z+Tfp/6i7/EyU+Tfp/6i7/EyU849182T2R/JPGcW4rialds/Jv0/wDUXf4mSnyb9P8A1F3+Jkp5x7r5snsj+SeM4txXE1K7Z+Tfp/6i7/EyU+Tfp/6i7/EyU849182T2R/JPGcW4rialds/Jv0/9Rd/iZKfJv0/9Rd/iZKece6+bJ7I/knjOLcVxNSu2fk36f8AqLv8TJT5N+n/AKi7/EyU849182T2R/JPGcW4rialds/Jv0/9Rd/iZKfJv0/9Rd/iZKece6+bJ7I/knjOLcVxNSu2fk36f+ou/wATJXGupYokvbxIARAl1cLbgnkRAsrCMFj848Avf31vbk4TWW9nPbZw7kgE4gBrpoTuKzwWpkxIbsWBSlK6JWUpWw09hJ8hKYLaMSOqc2DSRQoq80iBaWd1jTeWWJBuRyaVFG5YAy+38IcmyRyMttCJY0ZRNLwcTyrDJFZsnEtHdOl1AwDAIOezOpBAo2i8rLZzhlka07iRXu1UHSsbqVAKVvc1o+/s/wCmtX7JLI5haO6WIW7COZZmtXcW0kTkK6ScWQsAwG4rRVZinjlbijcHDeCCPcpNcHaJSlKyr1K6D9CLAxz5LJZBwGewtoIYQQDxe+aXlKO24cR2rpv9EzfTXPldN+gl56g+zE/6+uV4ayOZc85aaZNHYXtB7wSFUtxIhdT6zXT1KUr53XOJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiGvmMd9+++/v389/fv+2vpzXzm8ScUbHM5S0K8ehf3SIP/ACuqzRMP2NEUYfsYV+p/ZhKBLaI9pDD3FwP/AJBba6nZuHUo/Uh0Pg4bxryW4knjt8fZNfXAtUWa6eNZoLYJEjsFX27lGaRvZREcny2qPVb2l8lb6XxWHzEdkt7kcsb947iaaaKC1trSb1VoIltnXqTP3LMT7IPHYgmv0q9rS+KINiBL3nCwCgqaFxzOQ5LXZ502AnJbKd5DaDU5BTDJagjsbaxx0V3kMWMhisXNhshfJH8FRNbpyAltekRayTu59YmVplEnQlI4kcNVBf3izsDHLBdQ3C9XHSSbw29/MVL2SsXIbTmYQKihS0aT+rDcIUeWSagmyktja38GJuM/g8tapd3eIvpJL65s7tiyyC0vCTfIh+ekoDheTkBdxtGcQttdgQWU01ykCSwDG38e+cx0Db9ayNuhHw9hGIblBERPDyR4wrqef59Z2s4ouLRWpxmofQ6EPpm05UIdyTqxxxuY/WACnx29/wBdRzopFg83xaMTS3M9vDFfXEM5eYXLx2VrcZnHSzKjqLqWSxvMpBLHKHDS4rZjurBon4leG5ktbeWztMZFM0121nDj7hka7w8cEF1bP0MjN1rzJCGR5GEILFGHMcigMrwtkXKMxEpdtjJujqyXBlhkuDPGqrMvSvMm3JR+klyV05WLksUe1bppAZpTEZbU2fqdzJLb25xtpJaWGCluI7m+/QQyvc2mQkWNw3I49+AViXWrDbnWW0CSA0O0VNHHMCoFBSruwDIV18Dix1WrlKle29iVJZESQSokjrHKFKCRFYhZAjd0DKAdj3G9eqv15pqKrdBK6b9BLz1B9mJ/19cyV036CXnqD7MT/r65Thz+DTfo/daqdv8AuT2fFdPUpSvnpc6lKUoiUpSiJSlKIlKUoiV4XD8UZvPirNt9g3qP+IdhdXNn07R3V+tG0ixzvZyyQjfeNLuL24G5mN9xtuI2X+tWnjxuWN9bXE88ZtYMddWtxCkjr1Lg8f8AbWhVRHK0hiUqG2MSlwO7tRFtvCzUrZnC4zKvEsL39nDctEhLIhlXlwDN3IH0mpLVR+ifZ30emcQ9zfW9zby4yzNlbxWptpbZOJJSW49ZcXZ2KDkEj+afPftblESlKURKUpRErUZnUtrZyJFNMEdwCF4s2yk7BmKAhF337nbyNbeodq7Q639wLgXBhJVUlXhz5BfIqeQ4tsdu+/urX3nJa2Q1sjQ59RkTQU27R8VWtTpmsrCATuO73KYIwIBB3BG4I7gg+8H6K/axca0YjWOJ1dYVWL2WD7cBxAJB+dsKyqvMdiaD/qrDTUJSlKkvUqvPFTxRixE0GMtbWXMZm9QtZYe0IEhQdvWbqZvYsrMHf9I/ns2wIViuZ4069XT2La6WL1q8uZorHFWK/PusldHpwQgbgld93bbvxRtu5AOm8GNGw4JHlyF5BdZ/MsLnK3csiGeWZvK0tw2zCyh7RoigD2AdgNlX0Cqi57WirjRSXwzhzC20smamsHuZ5jLHb46N0t7SEoqi160rF7lgysxcjzdgCQFqV1V3i3ru7xt3Db26xophEzSSqX6hLMvAdwAo499u/te732Fp29a5tLa4dOk08EUrR9/ZZ1DEd++3f31lfC5rA86Fa+zXrBaLTJZmVxR+llQdh+uiq88rk4bVY2mfprLPBbRnZmBnuZFhhQ8AePKR1Xc7Ddh3rTx62sZfYhuleRzKkS8Jdi8cazAligVY3R42RyQsgdShbcVt8zjI7uNYpOXFLi0uRxPE9WyuIryLv/w9WCPce8bj31FoPDuytB1ovWN4U3VWk5IyW8Kw2sLLt3ht0jTpj3HcksSScK2Shvhxf6ozWKsMomZ0/CL61huTB8F3MrRdZA/TZxlRyYb+ew+ytzfYvVyoWjzOnpX5IAjYu5iBUuoduZyh2KoWbbbuVA7b7j2+jFjoINJ4F4oIYWuMXZTTtFGkbSymFQZZSgBkkIA9ptz2qyqIq3+B9V/r3T//AGi6/m1fvwPqv9e6f/7RdfzarHpRFWNhi9XPEjS5nT0MhG7xri7mUKfoEgyg5j9uwr3fA+q/17p//tF1/NqsilEVYw4vVxklVszp5UUp0pBi7ljICoLEp8KDp8W3Xbc77b9vKvd8D6r/AF7p/wD7RdfzarIpRFWM2L1cJIlXM6eZG59WQ4u5Ux7AFNk+Ff0nI7jzG23vr3fA+q/17p//ALRdfzarIpRFWN/i9XJGzRZnT0rjbjG2LuYg3cA+2coePbc+Xur3/A+q/wBe6f8A+0XX82qx6URVv8D6r/Xun/8AtF1/Nq9Nji9XNGGkzOnonJbeNcXcygAMQpDjKDfdQreXblt7qmGX1hZWl7bY6a4EdzdBTDHxcg82MacnVeMfJ1ZRyI3Kmt63l27fR9tQbI1xIBrTXo61kfC9gBc0gOFQSKVG8b1XHwPqv9e6f/7RdfzavRFi9XGSRTmdPLGojMcgxdyxcty5gx/Cu8fHZe+535e7atv4d47LxJkI8nexzNK/+yNFxZ4wwcNIAEXihJjKod9uDfTUa+Jefs8aYLXM9e4a+EzSSs/a36fHgslwJD3l2dl7Dz8++9U2p+EOEbttRlUU0FOlbFt3RGR0ZnYCC0A8rCaipNaZYdDXas/SOZzMOohh8ld429jlws2SjksrOWxdJIruC14N1buYOpWVj22939tnVVltasmurd3dnlk0hMJjzcw9SPIWqlooXYrApO5IUDftvuRVp1dC1ZFClch+mfpE22Tgy6L+iyMSwzsB2F5bKEG58hztxHsP/IkNdeVGvE7R8Odxlzjptl6q8oZduRhuU7xTAbgni3YgEclZ1371v+DN7+LLeyc+j6L/APCde7I9iz2WbipA7vXzrqZaS8RLnH2hsHtMblLQS9eG1y1v63FBMRs0kGzo8RYeY3KnduwLMTH9U4K4xl5cWN1GYp7Zyki+YPvV0O3tRspVlb3hga1tfREsMFsiGIB7TRwPwII+I2Loy1rxnmFeHhl4l5i/vr2UXs7yRWiy2WFs5Leyt52jlhg9Wt1uoZYYkitnll4KpdxAe59o1MtTassry9WxvL/DZWbrRwRQS4y8kykM8hUC2izGJZYTcLKQnUgUAsnYL7uXKycXfSWs8NzC5ilt5EmhkXYlZI2DK2zAg7EDsQQfeDXO2rgnBJKZYqMoKNDWtaQaACrqHLLYA/M8ogACs+xtJqMl2LipDb2WUyFpaNkrqC2E9hDLL8IR3JO0bzQzJGkuRWMBSzFppGOydXcIqQ3xI1hPHpzF5W9xMFrfXV5cW72Die1gkjUM4vGsjIGZD0IY2WXlyiuZlBCznevcfrlhYtf2aGyvbC3SC4uLdILOIJNLMkFpbxwvx9XMtzJdNHHBGC1qOoZiwkFfar1Re5WYT313NdyKvFDKfZRfeqRqAkYJG5Cgbnua0N18FnutGKUCjX5mrseTaYBQ0wmodXU10zyrw2Ul2ew9umi01ftKV+lraJXTfoJeeoPsxP8Ar65krpv0EvPUH2Yn/X1yXDn8Gm/R+61U7f8Acns+K6epSlfPS51KUpREpSlESlKURKUpREr0339FJ+7f+E1i6gzUFhCbi4dkTkqDpxyzyM7eSpDbo0kjbAnZVOwViewJrFus/bs62wkYyXNpJc27COXoSxKoJMd1w6LuFdW4BuWx32270RRb0af90NOf9JtP4BVh1Xno1f7oac/6TafwCrDoi1OsI53srhbYkTFRw4ni23IFwp9zlOQH7TUe8K7W8jS49ZEyxkp0knLc+XtcyA/tKu3D9hPl76m9K1st2tktbLVjcC0EYa8k1rqO33Dcqr7MHTNlqchSmz6/0SlV14l/CHrUXq/rXS6a8PVue3V5Ny59P37cfndtv7anuM6nRh6u3V6UfV28upxHPbb3ct6WW8ePtEkOBwwUzIyPV9aL2K08ZI5mEjDt2HqVW+K2vL7H3628ASONYo5AXQP1i2+/dvJARx2XY7g9/LaxpVkvLAjY28tzadwd94pJY+4Pv9lm2+ntWTNJA8qxsYWlX2kRihkX38lU+0PtFZdbSZ8c0XFgbwSDqqVksMzJ5nyTF7X6N2M6B/pTpzUC8N9K3NjPLLMURWi6YRG58zyDBzsNgAFO2/f2z5VtPEyC6ktFFr1CeqDMsJIkaPY9hx7kcuO4FYuZ19FbXhtTC7KjqkswYDiTsTxj29oLv37jyP8AbMn32O22+x238t/d/ZXNWKy2J9lksNmeaNJDiDygSeoDYRlkc1OCKB0LrPE45ZHeCo74dQ3Mdkq3XPnzYxrKSZFi7bBt+/ny2B7gEVI6rPRByfwkev6zw/Ses9Xl0fmnj0+Xsb8+G3D3b+6pb4j6iXEYjJ5RgCLCxubkKe3N4o2ZI+/vaQKv/wCarVw2gS2UANc3ByeXqabfrQ5bFlu+QPhAAIplnqabVWWml+MmtL7JMepj9IcsTjl7GN83OobIXP7yCMpBsfeVYbEGthrbwzu73KSXUc8QineNy7swli4qqEKoU89gm67Ee4Hbzrb+jbppsXpnFxS8mubqE5K+eTYyveZFjdy9QjsXXqrHv9EQ8/OpneZ60hnW2kuoI5n48YndVcl+yjiT2JPlv51v4JXxmrP7rBe932W2RNZaTRocCOVhz0A7alZNzYxShBLFHNwO6GVFkIb/AIhyB2bt5ismoZrrWT4+aOCOFZC0YldpCQOJZlCrx9/sHv8AZ2qU4i9FxBDOFKiaJJAp8xzAO37fOtXDeUE077Ox1XM1FD7tnWrkU8TpHMb6Q1y+e1ZVY+S/oJv3Un8Jpk7xbeCa4ffhBFJM+3EHhEpdti5Cg7KfMgftFQrAeIkOTBiS1uoepFdIWmCLwuYOuGt3iLdZGMdu8m7IAAyg7MSovKyvD0cP90dOf9Hsf8pasCqy9F3KR3Gk8GiLcA22Ns4JDPb3NspdYVJML3MSLdRd/wCliLofcxqzaIlKUoiUpSiJSlKIlKhnidlMrbepfBlol11JmW5LjlwUcOC/PXgjbybyd+PAfTUqyF9FbRNNPLHBGgBeSVgka7nbu77AdyB+3cViEwLnN0w0qSKDMbDt6dysOszgxjwQcdaAGrsjTMaiuzesmoXrXJZaLIY2KxtY5rSVwL6VgCVXqKH3YuOkBEWYEA7nt7tjkeImFmzGOWKyv/VS8kU63ETNwliAJ4dSFuXTPJXBU9yi+417M9nFwOJjnu3muzbx20EkiAGWaYhYuoebAKWbdiWb+0nzwTvJBBq1oAOMEb6kK3Y4w0tLQJHuJbxZB2igOwanLPIhe+Wwxl9fCYrZ3N7jiq7hlee3JJZQ6K26kNyI5DsdyNjWt8P0zAucl8JNEYDMPUOHT+Zyffj0vaEXDpjaT2twf7dRgsPjrW2vtQ29zJZHLWryCe6YdO3a6PUDCPb53rDKeJLdwAvY9/3R1zPi9P3F3kMiciu0k8dxaO1ywgkCxosc8wUyN1CW3bYLz232Wq7ZOW1zhT0nEtIwkaAu2mo06VdfD/SexhxZsjAe04g6tXCPUNociK5g9+3sdBJFnJ84LqcvNH0zbnbgN0WPu/m0YCAhNuxO+/YV46ly+VizFhb21ik1hKE9buSCWTk7LIeYcCLggVgCDy32H7I14OzQY7T9xkJMttBNJPKLnI8beK2aPeE8+pMyk9RORIbZu2w3qCeGniZm8i74u2mtXluJOpbZvLh47drYruZLCwWOKfJcgruhfpJsPNh2ryNzSxlAWYziqCCK1rQk87oU543tllxuEvFDiwHNcDQAioaP/j2knLVWHbX8VxroGJxILXTd3aTlfJLn4QtZOiSfnPw7nbfbYg96tGqH8OdGPhtbyiS/myUl/p24vbi5uFWOR7oX9tHI4hiIggj9puKRImwY8i59qr4rZR46cunZu2LQzcXUcXWlBWtNdtKUy3fNKUpU1iVZ+OfhJb6kgDqVtr+BCttd7bqyblvVrgL3eAkkgj2kLEjcFlfivWOl7zEXT2d7bvbyr3HIbxyJ5CWKQezLEf8AiUnvuDsQQPpBWn1bpeyy1ubW+tIbqI7kLIPaRiOPOKVdnhk2JHJCD3Peu24NcM57rAhlGOLd+Zv+Ho/+pyroRnW9Zba6LknMfDqXzepXUOtvRZVi0mLyHT33K2uQBZASd9hdQLyVAOwDRse3djVXZfwB1DbsQMctyo/8S2nt3U/Ykkiyf3rX6xYuFt1WptWzNadzjgP/AHUB7CVt2WyJ/wCbvyVX0qfxeC+oWOww1zv/AO80KD7zygVKNOejZnLkqZ1tMeu/tdeZZpQv0qlpzVj+wuv2irc/CG7YW4n2hnY8E9wqVN1piGrh3qmKVa3j94VRaYixKpdy3ct6L43EjqsUQ9W9V4CKJd2T/wDEPvydt9l8u+9U1cu+3w26BtogNWurQ0IrRxacjnqCpxyNkbiboldN+gl56g+zE/6+uZK6b9BLz1B9mJ/19c/w5/Bpv0futVa3/cns+K6epSlfPS51KUpREpSlESlKURKUpRFrdR4SHIQiCYSbB1kR4naGVHXcckljIZDxZ1O3mHYe+tXDpG3tpmvEe5Lx2ctpFHJK0sEVszGUxRRSbiFeXAHhsSIIQdxGgWTV6b7+ik/dv/CaIql9E/SVnZ6cxF/BFKlxfYq0NyzXFzLGx48yUt5pmhgJY9+mq79voFXBVeejT/uhpz/pNp/AKz89r5LS8Nr6uzrGyrLLy2ILAMeMfH2wAw8yPfVO22+CxsD53YQTQanPsqsM9ojhGKQ0BNFNKri/1pdJlTahE6a3CwdHj+kZWIXnz335EHkPdtt5+dWODWpz+Ts7ELc3LwwkngkjLykJ2+avFS57fRWK8LHPaQwQSFhDgTQVqN31lvCwW4HAHcZgDTUnQU6T9Bbam9Y+NvormJJ4ZFljkG6Oh3UjfY/YQQQQe4INV1PDlPhncesdP1kEMC3q3qvLyP8AU26fmPPl+2l5W82PB/Tc/E4NyGnSVKe1iNrXNBcHEaZ67V730TdHK+tdROl616z1eR6vHnz6fHb523s+e2391WNSm9e2C7IbFj4qvLOI1Nc+j6r0qVnsrIMWD8xqVpb3S1pNcC6eENKCrE8mCsy/NZkB4sRsPMe4VuqhnjBkLy2sA9nzDGZRPJEvORIeLHkBseI5hQW92/7aeD+QvLmwL3nMsJmEEkq8JHh4qeRGw5DmXAb37f21sIrDHEx0rA0YjnQUJO871RZeETLcbI1hDiMZdh5J6z89+SmdVB6WZE2EtMVuR8O53DYg8SVYxz3STSbFe+3Tt33/AGb1b9VD44uJtQaEsNxvJmrzI8ffxxljK/Lb6A06jf6WFFt1bqKFAAAAAAAHkAOwH2VVmsfC2W+yUl2t1GkU7o8oYMZk4qqEJsOL9l3G5G2+3u7zvW9/Na4+6uLdOpLFHyRduW3cBn4j53FSzbf+7UL8FNU3uQa6W5brxxhGSfgkezsSDFvGoVu3teW42/aKt2cSMaZWEZZLnL4dYrVaIrBaWOcXcttMgKA6kEHSu/popVkJcZd3CWk0lrPcREqsTsplBA3KdjuTsNyv7PKpEihQAAAAAAB2AA7AADyFVXa+FciZYXpukMC3XragBvWCwk6ojPbjty7F999vdVq1Wks8MbsUdKuAxGlCevf8lduuW0y8YbREIyHUFDXEBoT9di8JoldWRlV1dSrowDKysNirKezKQSNj9Naf4u2dsjywWVpBIlrNAkkMMcciwsWmaIOihhGZWZyvkWYnzNbusfJf0E37qT+E1FbZQf0cP90dOf8AR7H/AClqwKr/ANHD/dHTn/R7H/KWrAoiUrT6v1Na4m39au5DHGZFiXirSM0jAkKqICSeKsfsU1sbC7SeKOeNg8c0aSxuPJkkAZW79+4IqAkaXYa5jZtWQwvDBIWnCTQGmRI1AK99RHxA17b4aSyiminlN7IyIYQpCKhRWc8mHI7ypso3J7/2y6oHobP3eQe+ORxIs1sZQ1q8iMSdufLgZR+kkVUU9SPsefasNokIoxpo52hpUZZmvZvVuwwtOKWRuJjPSGINJxZNptNDQmgOSnfMb7bjfbfb37eW+30VDfErXXwM1kvqU9365K0e8R4hOJQcR7J5zN1PZTtvwbv2qLeGUVpmsxc6jt5r2Mxn1d7ScIEJaFUVhJG7bxcAG6Z7h+/ltUrxOvYbnM3WFEE6y2qM5mYL02KcC3s78kH6VdmPY/s7b1/CuNjqHBmI0YfSrnrTpocjorni7wechzDIGNxSNzZhJGYrnXCSDUare6qyb2dlc3UcD3LwQvIlum/ORlHzfZBP7TsCdgexqrfEHUsmR0xFPPiLktdXSxtBG0iNF0mdkuFbol+DGMKAybEye/tvJ9WX+SbOY62sbq0FuirJkLZngNx0+ftu8T/puBhK8DHt7RO/bat1qLVUMcs2Ot54WybWsslpaybgPKI2kjVn7ICeO/EsDt38u9RtDuNxtLi0ejQgUJPNrqdlFksTPBzE8MD3VElQ5wLWioo+noitHYt3Qo5iNDXCXuIvIL2e1s7OyhibGyNIzg8HLI2xEbszSDkWXcFO3u2ztG3mRv58pbZTHQJapLxtecYZJU5uOJEjMs68FjbnsO7f2CP47xAyWPXG2+UxsklxfTyK0sXCMRwB1jVmWING0o5MxXkoCBSfOo344ayizdt6jh5HvHs7pHusksgtdP2zLvHwusq5CSyqZAyx24lbkuxAPcY48GAuhrycyw8mtW6GuQoM8sgs8/G8Y2O04aP5LJAMZAa8kubhzcSaipzKlviVjry5u4rGWK3iwHq6Nd3HKK39X6HJ+p1XYGEoyQ8QFKEb79t9tZh9WNcWi4nS+NXJW8SvA2ZyfOLApuW6jLIV62YfqFt0t1CHl/SKKwM74RXmaxglyOaGYu+nbyWVuu9vp+LjwPJLWMsb2Ro+W09yZe7AhU905xepo8PFicVkrkPfTxpEGhjYxbl+lGCyqAq7lYwdhuVJ2FZ2iOKUudtHpGlBU0wA++iqPdNabO2OMjkk/wBNodiJAJMrhpWmROuWigtrpzF2WXx9rmrifOZKTpvZiWBIsJYNI7LEtniYz0LZuaELKyyyjYEuN63+ocnjYdVW6y4+c3aWxlF8rN0lVIZXD+rg7ShYkdept2PbY7bjdeLmUfHJa38GIiyU6TGMStGXkt4irMSrxo0kYYjjuNgN++/kc/VuuLTF2tteXQfa6Koiwqs7AsvNxyDBSqgHcg99u29Y5nVLmvcBgLXZtyA2DPKvSNFmssZa1j4o3OMrXsykzLtpoM6U/K7I+9QHRWsbbNa0jntuqFi0vdxusyhHBOStWU+yxBUr38/t2q6qqLBxwJriJLa0W1h+KlxKjRxxQwT9fIWkhljWI7kjcBi6qd/pHerdrYQh+AYyCdpGQWltJjMruKBa3YCakdZySlKVkWBKUpREpSlESlKURcw+nb56f+zLf6CuZK6b9O3z0/8AZlv9BXMlfQvAb8Gh/X+65dFYPuR2/FK6b9BLz1B9mJ/19cyV036CXnqD7MT/AK+nDn8Gm/R+61Lf9yez4rp6lKV89LnUpSlESlKURKUpREpSlESoppHVi5a3u3WFoRGFKhmDloZ4RNEzbAdKbg3tRHcoRtufOpXWE1nHDFOIoo4hJ1ZZBGqpzlcbvK3EDlI2w3Y9zRFWHon6gF1prEWvqeQtzZ4yyUz3Vu0NrPyVgGtZyeNwvsbnbyDLvtvViX+mLSe4FzJArSjieW7BWKfNLoDxcjYeY9wqKejT/uhpz/pNp/AKsOsM1njmAEjQ4A1zAOew57VB8bXijgD15r8ZgPMgbnYb9u/0VDvE3RXwwkHGcQSW7ScSyl0ZZePIEAgg7opB7+8bd+2v8S9N3d5cxSQr1YxGECc1TpuGJL7OR2IK9x39n7KnGIheO3hjkfqOkUaSP/xOqgFtz3O5B7mq1ivGc2qRnFloZ6Lzo6u7L4V6aFa+eJttElmnjODLOpFc65UpoRsK1eiMEuLsorTq9QhnZnPshpJDyPFSTxXyG2/u/bXo15qY46OIrEJHmZgvIlUAQAknbuT7Q2H2/RWh8SdNXd5dRSwr1YxGqAc1TpOGJLbMR2O4O47+z+wVL3w6T20MF0q3BjSMMzb95VUK0gYbEEnf++qc9rtlrdPDG0xuFMMh0dU50y3bsVNtDkoxNe2N1nhbgDAAwnMU/wCOtY+AyzZCwM6L0ZHSVACd1WVd1DA7d15bGop4cYe/gvJHnWVIyjiXqPyEkhI4ke0eTb7nl9H21KtR3i4uwaSGFNouCRxgbIC7Bd2277btufpP21heHmpZMik3VjRXhZByjBCMJOWw2YnZhx79/eKpyshfbLNFPI7jmNxZZNcaZ168J3ZZHVHhjp4mSOOMCuWh317ipVUf1zqE46BJFjEjySdNASVUdixZiO/kPL9tb/esbJ4+K5jMU0ayoSDxb6R5EEdwf2j6a31sZM+FzYXYXkZE50P11rYTte5hEZodhWu0VnfhC26xj6bLI0bqDyXkoDbqT7iGH2d6rm5Y5DxFgQANFp7TksrPv8y+zE6xrHx27E2luW339/7KtILb4+1dv0dtb20ck0jfNjjjjBkkkYn3BQzEn6Kqz0YLV7uDL6onjMc2qMk93ArjjImItAbTHRPuTueiruCOxEqmvbIyVkLWzOxOAFTpU7frJewte1gDzU7SrZlv4VkWFpolkb5sRZRIfsQnc171QAbAAfsA2G5+yq9z2hp58i1ys0YiklSUsSwlTjx3VV22Yjj2O4930VPVvIzIYhLGZFG7RhlMgH0lN+QFVrBabRM+QTR4A11GmtcQ3/DvpsWCGZ7nOEjcNDRuev19aKn9I2WaXOBpvW+n1pDcvIWNo0Pfsm56bD5vEL3Hby2NXPWvzOZt7MK08qxByQoILMxHnsqgkgbjv+0VmWtwkqLIjB0cBlZTuCD5EGtlLbGTSYARiaBUAio6SNVUuq72WFr4myF5JLjiNSK/XaV6cxfpa21xdScunbQS3EnAcn4QoZG4qPnNxU7CoRg/EJcg/qwthGXt7gSbSs0kVzEJuUJheBGaJVhZTMPZ6nsryHtmwSK1txjoIYZDHBDEUtpIkMcaIVi2L9JSoHGPl34jtv3rxbZQf0YMhHPpLBLGXJgxlnDJzjliHUWFCQplRRKnce2m6n3GrKqvfR1bjo/Tp2J2w1kdl7k7QqdgPea2PhbrVs1DcytZSWfQn6QDsXDjbl5lF2kXyZdjtuO/ftjdK1rww6mtOzVZ2WaR8bpWjktpU1GVdMtTpsWRr69xRW3ssm1uRdSgwQzhjykQhQ4Kj9GAXC8iQPb237mpLbQrGixoqoiKqIijiqqo2CgDsFAAG1VdpmNs9fXKZjFwRSYudXsRykjl4Ozbh06n+0xfo4W5kcGLdhUh0xc5hsvkEu4YlxyhvUpF4bk81EexVubbx9QtzHZgNtqqQ2jE7HhycaAhprlWuOugrWi2VpsWBnFY+UwYnAvGE4iKcVTU0IxdXQmiI8yt/kjkHha0Ln1Dh09wvNuPERjmE6XHcSd+Xl7zUbbTuo7S1yRiyaXlxPdRPahmDMkAaQylPWl6UDsGiHTG6gRtsdyKkl9oh5M9BmRfzIkUXTNmAeLbIycQ/PZYiW5FeJ3K+ffs19qPIWV1jorTGm9iuZeNzKBI3THNV47x9oTxZm5vuvsn9prC+IBhMmMYSQCHEkh2VcuvKuisxWgumAgEbsbWlzSwNaCzPDyt9OUQQHVWqymGzIGF9VuLKzK8Gy0UYSKOW4JjaVwixETAqso4jj3I/YV23ifrAYSOC4Fg941xKIWMXsFVUcu7hGLOf6qdtyD3Fa7XWAscvlbGJsmYbvHHr+pxMpkZN45twCd4pNlU8hueLeXkajPpBamWZrfFY69urjLRzdX4KxIaW4I2Ch7qaORYsfGjENyuHUd9wD2o5sjWScVWuQbyg7QAGgOQ6d6MfA+SDwigBxF4wOZkSS2rhUuByoRoMlOG0fZxZCXUIiujc+rtIYA24LdHpnjFtv1zEOHHlx3Plv3qrcF4m2GVvWvrDT91k87CzW4gtZSbOFOmVS4vMlIFtLQcWePuryjbYIw2rdaj0rnMpj7i4y96ltFHau8WnsVO0EUzrGdkyeZBjmugSfaig9Xj3G27DvXsx+n7y401Y2kJs9NSi6Bjhtt7CGWH2ggCwMWSRyVfbcligJ+d2lM1sb6NbXV+HCKF1RnU6OULM+SeKr30qWxB5e6rWUJLcI9Jp210717c7oHJZO2uLrOXQyBjiklttM4xpbTDGVAXjiupdxdZduSruJGSM9x0u9bnS+F+GdPx2eRx64tEfjHb2iepJHHbsDFJFA4PQX5w4sCNgT5EVna/zGUxlrj1srQ5SUskN3IUdyeCKOZSJgYzI3I8zuq7d/MVkZLV9lPkH07KJxPc2zo/EbRbSwNI0IlDchJ0eTcuPHt579qnLIxzy15oKBuEjkuLtM9pypksVnglZEHxNBNS8PaavY1mTjSuQzBzAqtVmrm+s48Pb4KGC8skYW9xIHWcKkbRoFaUSDiOJlJcb7FR5eR9+Sz+OudQwYqawM15ar1be7kjRkjcxi52Uk8wAgB5bbBl+kb1k4TG4/SWNl53EwgNx1HlmHVkaWULEqLHBGN/ZRewX3EmpTi5be6WK+hEcgmhUxXAUc2hf2wORHIL7+J9/uqLI3GjS4A8klmTgAMqN0pXfvU5Z2Nq8MLm8trZRVhc52dXnMGgJGHKoOa9WqcOuQs7myd3jS5iaJnjIDqG943Gx+w9j3FQHCNicdLaaTmje+cEzo13DHNAJpepcKNm7I2xbbipA37ncmrR3rEkxsDTrcmCEzopRJyimZUPmqykclXuewPvNWZrPicHtpXQkivJrUhUbLbeLY6J5OHNwAOGj6Ua6uuW7JQGcba6tgOwGk7sAD/qdrVk1WT3CPr2JFkRmi0pciVFYFkL5G0dQ6g7oSpBG/mKs2rKoJSlKIlKUoiUpSiJSlKIuYfTt89P/Zlv9BXMldN+nb56f+zLf6CuZK+heA34ND+v91y6Kwfcjt+KV036CXnqD7MT/r65krpv0EvPUH2Yn/X04c/g036P3Wpb/uT2fFdPUpSvnpc6lKUoiUpSiJSlKIlKUoiV6b7+ik/dv/Ca02uMLLf2xhinEftwu8LgdG4WKeGdraZgpdYJY4pIX4g7rO+4byMZtdI3kTYnqvZyx4m2lRrlHmW9mJszbhCXjbe0Eks5MRfd+FsxbeMqxF5ejT/uhpz/AKTafwCvLUXilb2V+1m0ErrE4SedWA4MQCeMZG8gXfv3Hkdt60nooYH1XTOIufXchcet4yyb1e6m61tbkKSRbR8AYUPPYjc9lXbbaptltCWF1di8lt+Uu6s+zMI3ZNgpeMHZuwH27d96zQmMH+oCR0LV3qy2vjaLE5rXYhXEKjDtGh6OzaFJQd+9ftKVhW0WJfZKCAqJZ4oi52QSOqFvs5HvWWDUF19o2a+uUniliA6axOspYcQpY8l4qeQ9ry7eX7amWNtujDFDyL9KKOPmfNuCheR/adt61tltNpfaJGSR4WNphdX0vr3aKrFLK6RzXNoBod69txCsisjqrqwIZWAZSD7iD2Ir1Y+xit04RRRxLvvxjUKNz79gO58v7qyK8J5VjVndlRVG7M5CqAPeWPYCr5jaXYqZ76ZqzhFa0zVOai0nlpM4bmMvwNwskN0JAI4oAQQhTluAq7qUA9rv571c1Vlqfx409YyCAZSPIXLMyR2WJVsncvIoJMQWzDIsnYjZ2XyP0VHrnJan1SOja2kujsZKAJb+/wCL6gmibYslvZIeOOcqSpeQll3DKdxtVmWcyAA7BRa277qjsTpHMc48Y7EamtOrv6T3Lw8XcrJqjJfEvHSsLdDHNqzIRE8bayDBlxMci9vXbkoVI39lQwIYdQLPtO63xnXixNsTGIVW1tgqcbbaFeCwxNv2UKmw3AB2GxO43z/DjQ9jp+xWwsIelHyMs0jnqXFxcPtzubiY+1NO2w3J7AAAAAADTYXw1srfIeuJLIxikMqWxKFY5G3IJIHIqN9wD9A3JpEI6HHXTJeXi+2h8XgoaQXcvFrh6PfvOnSp7VPaf8N76DMpdvNH0o7prg3AcmWVSS3EptuHcHi2/bZm7n37/XPiamMvPVBatPwVGnfmI+IkAYKilTzbgQe5A77VPbO4WWOOVd+MiJIu/Y8XAYdvcdiKyNMsLa0ycFTnjsF6zhhcS+B1SASKGu3fmNmh2hRvXWk/hHouswieIMvtKXRkYg+QIIYEf41udO4wWdtFbBi4iUjkexZiSzHb3Ddj2qqLLHZoZ/qN610/Wyzylm9TNnz7qO/Ar0uwTzB28j3q561zrps8FoNpbQveBUgno2abB3LLdVpbanyTcU6NwOHlfmptHz+JSsfJf0E37qT+E15X5kEUphEbTCN+ispZYjLxPASMilljL7bkAkDfYGq20la5yNwMg0jIlvfrLJ1rea2dOtddHsiJL64ym2cMECCIcTs+4rOt2sDwvxt1eaAwlvZ3PqlxJhsf05t2TbZY2ZOpGC8fJAy8lBI5VN7S8bDYZJsjcNcPZwD1mdAZGkblxAXls0jbsi8m238ztuaqjwS05mbjCaQuY8rBDa29hYs1tCk9uTbcI24SqZZEvZmQMhY9NRuCFB3qxMPl72+ymTxl5ilXHxo6xTyI5jmXkqqGaTeOcSIzPsvzeOx71Qe+kpNCHHktOZacq1oPitzFFis7W4muYP6jwKNeMw0tqdSRmAKjb14WjsRjp7mXVcFzOi3cM3IXJSOGIoRDMz7jccTb7bFio2JHbbbz8M7q9tcffX2TyUF/ApkuYbi2cXCC3iVmldWjQewSvaMbleJHv2rH8RcfeFrHDWWOiOKuk6F40KiMwoZByKMrBYOCfpASrBm3Gx8j77jJ4fSGPisJ7pm6xlMFmVN5kb15D7aw2NuhknG7BTxTioI5EdzWGGN3G0aKYQanMAvcBU02jtyVq1zs8HxPdixloa04XObGwmgxfldsOWYzWJ4CBpY8jffCc99DcXRCR3AdHhaPk7Fg7squySx9kPHZB9g0eA8UJMfLdxZK9hy09xMRisbhBHkb+YqZOoEhs/6C22EY6lyyBSG5MPfrNIZe61NbXcVkYtKYa25esW9rHHLn7iKVGZj7Km1xauFbunXkPFhuh2rI8GMnprT+OuLizS6tVuL0Qzy3im5vZZOHUXeWEMzxCPdu57En6RvGJ0cQjxSDIOOTsjv1rUA6KdoZPaXTlsJqSwAFgxNH5QKAUJAFaDMb61UdxGnchqXU91Ne2t9piGKBjLDj5Sb24I4QBbjJgGK36tuSvG077Bv0m/tVZp8GcdDcY6ayBxsePdHMNso/TujiXqSzuTI87EbNK5dmXsTUkyWvLG3v7PHPI5mvUjkgZELRcZiyxcpPdzZWA2392+24rE0VoqTHZDI3rZCa6W+cssDggR8nMm7MXIkZQeCkBdl3/syiNjv6YAeC6rqUbhOorTU6dKrumljpMXGMiMBgNX4x6Lg0nJopXoGzevT4g3mOyUh03cXUkVxdLHIqxKdw0Z66AyFDHyIiJ4t5j6CRWs1n4dY4Yuxjubu4gtsMrt1uSlmjkKlw44H2mZVC8BuN9gO+1ezA2t1cajvLi8xFtHHaxuljkgpEpUMFj9syFJi0TyHfiCnddxvsfc+KyV/lL+C86E+DuLfjDGpj9o/o2Qq0e06zB1cliduw2922F4EocXMqXEtHJIyGYx9FRqFYjc6zFjI5cLWNEjuU01c4UdxdMsWE0DXdKydZ282exUL4nJC3EkySidWlh6kSc0aIvGOrGQ+xI2849jUkxmEjjME8qQz3kVtHA98Y0E78VCuee3JQzcjtv/WrwsLeyxFrDbq0NnboenEJZAil5GLceczbvIzFj5kkk1uBV+OIYsTvSoK51AI3A6bVp57S7BxcdQwF2EkAOIOxxGuVKjRa7UWFgyFu9tcwpPE+xKPuByU7qwZCGUg+8EGot4P47KWsFxBkegEjkRLGOHp8UhQFSq9IACH5nEN7fZt/Op1SpOgaZBJoRUdfXvps3FQZbHtgdBQFriDnmWkbW7q6HeMlC89pS8nzVlkY8nJDbW6KstkC/F+JcsOIbpsJAyglhuOA29200pSpxxNYSRtNTnVY5rS+UNa6nJGEZAZVrnTXXUqtZkA13AQAC2k7vkQACdslagbn37DtVlVW9x/v3b//AAnd/wD3O1qyKyLAlKUoiUpSiJSlKIlKUoi5h9O3z0/9mW/0FcyV036dvnp/7Mt/oK5kr6F4Dfg0P6/3XLorB9yO34pXTfoJeeoPsxP+vrmSum/QS89QfZif9fThz+DTfo/dalv+5PZ8V09SlK+elzqUpSiJSlKIlKUoiUpSiJXpvv6KT92/8JrW6q1DDj4leQ8pJZIYLe3VlEs01xPDaRqgcj2etcwhm8lDEnsKj83iNZucRbgOsucglkhidoleBFtpp95lMntFngeIdLnyYEjdQTRFiejT/uhpz/pNp/AKkGV1laW1z6q7PyBUO6rvHGW2IDNvv5EE7A7b1CvRQ1JZ3ul8RbW15b3E9hjbKK9gidXlt5HVgqTIDvGxMUm2/nwb6Kl2Z0Nb3V3607SjkVaWJSODlQB5kcl3AG+x/urXXmbYIx4IGl1RXFpTb8uzTNVrUZsI4mla513KVVo9YakjxsaOyNI0jFUjUhd+I3Zix8gNx9PmK3gFa3UODhv4xHMpIVuSMpKurbbdiPcR7j2rNbmzmBws5AfTInT57PepziQxni8nbKr80zmUv7dbhAygllZG23V1813HY+47/QRWwnlWNWd2VFRS7u5CqqqN2ZmPZVABJJ+isbD46KzhWGJeCJue53JJ7lmY+ZJ99UbeSy+IV9NaxSywaUx8xiuriB2ikz17Efat4pU9oYmMjZnUjmduJJ2aLJZhKImiYgvoMRGlaZ0UosYYMetM+tbS88U8jn55bLSlnDPDE5iudS5ISJiInVuLpZxKOeTmADe0vsAhCeSsGr32fgJb3hWfUGUyep7gFXKXcrWmMSQe+HGWbLFGu/8AVYsDsO1WzicdDaQRWtvDFbwQIscMEKrHFHGo2CIigBVA9wrIkfiCT7gSdu57d/Ks6yKuNYi20vYR/BWMsLLrTCImC3SGNfZZyziEKZHPDzY/Se+1b3ws1LLlLEzzRqjpM8JZAVjkChW5qGJ2+fxPc91P2CMaZ8SUyt6thNYRmC5LrHzImPsqZB1Y3XiwIX3eR28/OrOtbdIkWONEjRRsqRqERR9AVQABVuVvFswObR2tehc5d05t1qNrgnxRAYcGEjlCmeffXsXnKpKkA8SQQD9BI7Hb31Tvh3oPI2eWW5mISOMymWYSB/WBIrDbiDzbdyrHmB83fz2q3MhexW8bSzSxwxr86SRgiDfsPaY7bk1jTSreWkptrhD1oZUhuImDqrspQOGQ+at/9KxxSuY0gaOyJord43dZ7VNG95OOLltaHAV01G6oG7rWtzmksfkZxPPAk0sWyMVdl3C9wkixsA23fs32VIUUKAAAAAAAOwAHYAD3Cql8J9EX9hkHuJwsUQjkRuMiyddm249lPzQfa3bY9h27nba+N2Bvr6K1FqrTRo0nXgRgpLHj05CrECQDZx+zl+07ZXRgyCPHUb9gVGz297LHJbPBS2QnNgHKdQ0BJpXadh0KscUqPeHNjc22Ntobti0yBuQLdRlQuxSMvv7RVCo7fRt32qQ1Ve3C4gGq6CzSmWJr3NLSQCWnUVGh6QlY+S/oJv3Un8Jr3swAJJAAG5J7AAe8n3CsG6vYpIZAk0UnO3klQI6tyj2I6i8T7Sb9uQ7VFZlTvh3r2LB6Q0iZLae49axlpGOjt7ISGMt3b50h5jig7ts3ltUv8esjbw4dhPl0w/Vkj6UjF+rOynkbWKCA+sXEjD/w4gzHYdiN6pjR/iHlYNH4JbPGtj7WGysLR81ko0mkeYREM+NxfUHrAUIxWe4eKM+4PvV0eGmgcfD0czyucrf3cEcvwvlj17/pTL1VjiUgR2EIEh2igVAAdjv51UNJHvic6tQOToQDkc+n3LZNLoYorRHHhIceWTiDiCCBQ5DCOuvYoFZy6tyj42KxM+FxccEVtc5DJxWxzFyFAEt6lhN1HtWZfmCY8wRu6d6nHhNg8HbzXhsJDf38bCPI5K8d7vKTN3TeS8uF5NETGw4xbRjj2ArL07q29e9y6XuPa0s8eskkN1xkHOOInvyb2Zy0SmTePy8j5ivHwtmxMsF9k8bbvF1ZX9bDB+r1Ih1uCxl2VVIl5BU7fpP7BiimGJoYcjWodWtG5cmvTrVWLRZXcW90zeUA2jmYcNXnFyyMq0OVNuqxNIZi1tc7e4S2xBtBx68l1GNkkYKrhmj4+xD+kKqQ22+42G9bTXK4S2it7XIRWcUc9wXgiaLZDONlaXaFfYGzgM7bD2hua8/DjXsGZtbi7WGS2S2kZJOqVYcVQS8w6duyt3Hu/buDXqtGw+qY0n6a3ospiF6iywsjsFbYqeJeJgFOx3U8e47VjaQ6LDG5ri6paCKClcxQZmnx1WaRr2WjHMyRgYGiQtcXOxYcnYjkKmmVchkFhapyohz2KtRg1uQ0aiPJCPdrdWZ0KxuIyqLGqhiCw2Enbb37DxO1hPiRZmDHTX/rMzRv0+Q4BeOy+wjbyvyPEHYew1fmm7nMNmL9LqGFcaof1OReHI+0vS2KsXYmPmXDgbEDb9s0rLGx72vwuLSTlVoBFMu0Gmp2KvPLFFJEHsa8NaKgSOIdWpzP5SK5gbR0rW6jxYv7K5tGd4RcwPCzp2dBIu249xI38vf3HvrA8PNLrh7COxWZ5xG0jmRwF3aRi5CoCQiAny3Pv+mpDSrfFNx8ZTOlK9C1wtMgiMIPJJxU6QKA79FGPEPRFtnIoYbh5kEEplVoWVWO68WQ81YcSPftuNuxqSQRhFVB2CqFUefZRsO58+wrzpRsTQ4vAzNKnfTRePtEjo2xuNWtrQbBXWnWlKUrIsKUpSiKsDbBNfI/ORutpS5Yq7s0acMhaJtGh7RA7bkDzPerPqt7j/fu3/8AhO7/APudrVkURKUpREpSlESlKURKUpRFzD6dvnp/7Mt/oK5krpv07fPT/wBmW/0FcyV9C8BvwaH9f7rl0Vg+5Hb8Urpv0EvPUH2Yn/X1zJXTfoJeeoPsxP8Ar6cOfwab9H7rUt/3J7PiunqUpXz0udSlKURKUpREpSlESlKURei/s47iJ4Zoo5opFKyRSqJI3U+asjAhh+w1rjgbSCNjFZ2sXAmdOnDGnGZLUWKyrxUcZBaIkAYdxGoXyG1bivTff0Un7t/4TRFAfRoUDSGnTsO+JtN/uVYlV56NP+6GnP8ApNp/AKsOiJUDvNfGPIG09XBiWcW7PyPVLcghcLtttyPl7wPPvU8rXSYO2acXRt4jMCCJdva5DsG+gsBt38+wrXXjBapQzwaQMIcC6orUbRt+XWFWtMcrwOKdhzz6lWXpFZi4uTjtJ4+Vob3Ucjx3NxH86zwkHtX11vsQrvHvCm+3IyMAQ2xqy9K4G3xdla4+0iENvZwpBBGO+yINtyx7u5O7Fj3JZie5qqvAmP4YzepNVyAsst22AwzMFIXF4tuMssTDv0ri95ud/fEauitirKVgXmZt4ZUhkuIo5H24ozAMd+w+zc+W/nWHqbVdnjTGLmcRNLuUQK8jlR2LFY1JC7+81GM/oz4SuRfQ3UZguVjct3Y8QoUNGR2YFVB77bEmqF6SWuGEPs0eMkgUOQptOzoHv2LX2i2ZlkFHvFMTcQqAdpGoUpx+mbK2ne6itIYpW5FpFHccvncR5Jv7+IG+9RLTXirDe36WYtpI0mcpBOXDFm2JUvFxHTDbbdmbbcb+/aSZTWVhZTrZzXSxy7ICCHYJyA49SRVKx7jY+0R2IPkawvipjMY82VEHTMKSTEhndE3BLGOIniGIJAA8t9htW5ZSh40EkjkrXWkyY2iwyRsaxxMwy021oDQ0Dq+ia7cl5eKelpMraJDFKsckUwmUSbiN9lZCrFQSp2fcHY+X7dx++F2l5MVZtDLKskkszTMI9+mm6qgVSwBbsgJOw8/2bn1aG8QbfKzSQJFNDIiGRVl4kPGCFJBQnZgWXcH6e2/epjUXvkY3inZbVmslnsNqn8YwnE4jDUE0yy03/JVrr/xMbG33qiWqyiNY2md3KE9Qc+Mew2Gyke0d+5I27VYlhcCaKOVQQJY0kUN2YB1DAEe47GtdmNM2d5Ik1xaQzSIAFdxueIO4VtuzruT2bcdz9NYHiJmpbC1WSFV5PKsZcjksalWO/Hy3JUAb9u9VLfbILPZ+NIIwirtter6CnC21WeSaW0yB0erGgZtG46dG078lJqVHfD3My31p1ZlAZZWj5qOKuFCnmB5DuxU7e9TUirFZLSy0xNmZo4VFcltIZWysD26HNeu6RmjdVKBmRgpkUyIGIIBaMMpdN/NeQ3G43HnVfae8PnsGFzJeRTvHbXRmMVsbZpru4a4lmmDG4k6Ns73UshtwCokbkpUbILFrHyX9BN+6k/hNWFkVFeAWorKDEabwdwt1ePkMRYXAFyI57SItCWjhRH2ZY/0LEbKwB27+e1m6yD5SxurPFZKCG5hkjSRoZeJj4Nu0DvBu9uSFI3A/qkeW9af0dLOJtLaZmMUZlTDWSpKVUyqpiG6rIRyUHc9gfeaxM8tho1Z8jHbXU7ZK5WN16i8Ix+kmIVmUBV3L7A7kkgbgCtfaMTA4yEYDWpzBa2mVKVqard2Li5XMbA08YMOFpo5r34uUTWgApTLPRS3C5KK2WzxN5fQXF+1qgkRiOc5CEO/FvnBgj/O7sFJ286y5WscLZu/G3sLWNuTCNBHGHkYDska7s7MQOwJ8qxG0pZXd9a5poX9ZSJDEzMyAAqeBeHfiZFEjDv8A47Dba6jwkGRt3tLmITQybckJZDupDKyuhDIwIB3BFZmNkwnIVGTDmcqZYtvXRVJHwl7al1DQyAADOprgGmQ9Gq9mIuLe5t0lgMUkE6l1aMDpur+Z228yd9wRvvvvUKwmfx2NzHxetbBrdpgZ2khVVhMhiM3cb8ivSTbl5A7KKm2FxkVnbxWsEYihhUJHGCTso7+bEliSSSSSSSayOgnPqcE58eHU4jnx3348tt+O/fapOje4NOQIpXKuX5gN1d6hHPGwyNIc5rgcIxYc/wAjnUqDTd717KUpVhUkpSlESlKURKUpREpSlEVYbS/H5OZiKfFW56AQMHCfCNpy6pY7M3Pfbjt2qz6re4/37t//AITu/wD7na1ZFESlKURKUpREpSlESlKURcw+nb56f+zLf6CuZK6b9O3z0/8AZlv9BXMlfQvAb8Gh/X+65dFYPuR2/FK6b9BLz1B9mJ/19cyV036CXnqD7MT/AK+nDn8Gm/R+61Lf9yez4rp6lKV89LnUpSlESlKURKUpREpSlEWg1zjLq7tjFbTRxEvC0kb9SPrpFPDM9t61ES1tHLDHNCzCOQ7Tkgdu8SsdM5eKbGvLdwtBj7I2069eWczf7PtLcNHLahpLhpWESu0oZUtuZLm4eJLMrwnTkrLvtyUrv9G423oiqX0TsXdQ6axM02UlvILjGWTWtnJBbQpZqFYlEmgjWWcEMo3lLEdMfSat2qg0n4c6gxVha4621XZLb2UCW8Alwqu4ijGy8n+ERyO3v2rQ6lgzF5Dk7CTWiQrBaXE089lhTbTmK1WGW4azuosnyMkQuLYEr751AJ77EV+1FvF7PHF4DM5BTs9pjbyaH3fp1hbpDf3bylB/bUQxGM1HdW0V2mrccsUyB0Jw1u44sdgepBlniYH6Vdh386rrxD0/fZrF5KW91m4s4LRre6gt8M0PXgnmt7qKaK2XIN6xLK9oiRybcuMky+zzaiK5PAHToxWmMHY8AjRY63kmUeQublfWbj7f080p3qc1V0OmdSJGhGrcciBUCb4NFAUgBR3yX2CoRb32V6N1n11mrQpFZ20qthAIFgdRe213DaSZQBmmhyELB03kkDRR8eacARWb4keHgy00Vwtz6vIkYhbknVRowzOpADqVYF3+3ceW1SvTmKWxtYLRGZlgjCBm+cx7ksdvLdiTt7t6rfOY3UVlby3M+r8ekcSM7bYJWchRvxRBkuUjnyCjuSRWiwlnnba3tOGrxLHe3t1CstzhBcSxXctzLztZmbKc0VLgywqqhlRYdtwicqyOme5oYTkFRhu2zwzvtDG0e/0jU59mg2VpqpZq/wAK1v797sXbRJMUaeLhzbdVCHpvyAXdVHmDsd/PyE/vMfHNA9s68oniMLKSe6Ecdtx3B29/nVOeIWGz8VlLFcayiiFwOkvwdhunkNyGctbvHk+SOEjkbkPII3vrOtrbUMtzHapq2yZprP1+KQYEdB7fmke6ynJd33kjPH6HH016+Z7gATpoo2a6rNZ3SPjZQyGr9TXXYcqZnIZKbaO0JaYqWSaHqvJIpTnMwYpGSGKIFUAAlV7nc9h3qU1R2cxOcTK4+KTWUS3CLIIo4cMVsz66rpH62i5PhIzGznERbyaN/IsN99lMLqO1gmuZtX46OG3iknmkbBrxSKJTI7nbJb7BFJ/sqD3ueauNVYs1liszOLhaGt3BWnXhNErqVZVdT2KsAykfQQexqicXaZyMtejVx3yeShsnhuMGXW1vERbRYhAcnxsoT0VY8SVZ5eXfnW11XFn8eiG41dZqsvUBMWBDFIokMk1w5GS/Rwxxjkz+7cbAkgVAiuRVhXFFGEAVVCqBsFUAAD6AB2AryqmNPYDUNu6YdNWxs9naQskt5hRPJNAv6ETNcnKFpZOakM0gUseRG+xNW3hIZo7aCO5nW6nSJFnuEj9XSWUDZpFgDt0VY7njybbfzNegURZlaF9QQTyXtmhk6lvDLuWQrG/BVEnSkPaTptLErbeRkA899t9Wqj0/bpNdXCRlJrtOEsnKRx80KSkTOY4ieKFuAXmUUtuQDRFUPhRodslp/Rd4t/Naiwxdg5hjBIk2WOTdWDjpOePEsQ269tqvGWNXGzKrDcHZgCNx3B2Pv3rnzC2eWwcMGDj1VGnwfBZ2tpbtp7rTzRSvLbW5iZcjtLya0n7tx9mJmPs96lmAx2ob+0tr621hj5re7giubeQYILzhmQSI3BsiGUlWHYgEeRFY2wta5zgM3a91FYktUkjGMccmVDcgKVNTpmc99VbO/u/w+3//AA0rm2PK30V1eZP4426zNAbG8Pxf5pHaYEvdzXBEeRKdCA50dSVGbYzKhHKNgLAyGE1HbwyTy6wxsUUKNJLI+EQKsaDkzE/CXkAKyKurSJ/x8qVzp6zkcpPiHOroupBcw3toX07JAsNxex5LEQJdh7/9E0o+EY1jbYh0TfjugaVashz+Ltxc3Or8ekbSxQKRg4xvJM3FBykyioo397EAbURXCTt3pVCZCzy+Y9dwMmrbYi8tr2ynHwB043UwQrdwxXXwgY2njhv4d9iRuzbcjHIF2eoHzWOnsrSfWeNjmvmKWsRwkYZwjRxs5ZsmERFeeFdyRuZVABJAoiuigO//APfoqldK3Gcybyx22sLJ2ihgufb0/wBMSWty88UF1EXyP6S3ke0uQrDz6RO2xUnWX1nl9PbwSaqt4IriXKZJ5zgFmhVm9Yy19M7R5LnHCg6zEldhzjXcs6gkV+0393+FVDp201DfpK0WrLNTbzG3uIpsAIpoZwiTdOSNsl7LGKaFx5grKp99ROW9yEWVvm+OFs99DHZYy4jj0+ZCSbrhbQLGuQ5JI1xk12Z+KsrlwSiO4IuiqE1U2Cxuor22iu4dXWDRTJzQtgRGwAJDK8b5ENG6sGUqRuCpHuqA32pLu8itrttZQSLaFMtAI9OOZBb+pXE637Qi/L+qepm4buA26hduZCki6XoTVXvp/UwjMo1ZYOoQyDp4ISMy7cvYRMiS5I8gO53FQi+zWQvLSOOXWVqtvkrK3Z3fTzRCK2y7y2VqZ2bIf7M880U8SBhvyjO4FEU3W66mvlTpSp0dK3Kh5FCpJyyFo/KJgTzUb7E9tiDVoVTfgpipbnItmJs18JNa4uPEx25xbYd47e79VycMzLJcO0geDpEdhtzZTxZGUXJREpSlESlKURKUpREpSlEXMPp2+en/ALMt/oK5krpv07fPT/2Zb/QVzJX0LwG/Bof1/uuXRWD7kdvxSum/QS89QfZif9fXMldN+gl56g+zE/6+nDn8Gm/R+61Lf9yez4rp6lKV89LnUpSlESlKURKUpREpSlESlKURfjqCCCAQQQQe4IPYgg+YrQDRtiJZZ0geGSYBWa3nuLbYBBERGtvKqwBlSPl0wvIwxFtzGhEgpRFr8dhbe3tls44V6C8j0pN5gzO5md3aYs0sjSszs7kszMSSSd608vh7i3jmheySWK463VgmeaaBuvwB2hlkKJwESiPiB0Q0gj4B35SilEWP6lF0lh6UZiQIqRFQY1WPbgAhGwC8V2+jYVHk8PcatvJarbSJBMsiTQR3N3HFJHIkUJhkRJwJIBDbwwrG26pGnTUBCVMppRFhZfEwXcLQTwxzRsjIVceSupRuLD2o24kjkpBG/Y1qLfQ9jHLazrHcK9kXa2/2u9MaPIAskhhNx05ZXHLk7qzP1JSxPUflJKURavUOnrXIIY7iASAgDmrPDMAp32W4hZZUXuQQrAEMwO4JB8rDB28EiSxxkPHHPEjNJJJxiuJVnkiUSOQsfUROKjsgUKvFRtWypRFrL3AW011BeyQhp7YERScnAA2cDlGrCOUr1ZeJcMU6rlduR3zru3SaOSKRFkjlRo5I3AZHjcFWRlPZlKkgj9te2lEWgsNHWUCqiRScUkgmVZLi5mHWt5pbmOc9aZuU/VnctId2cCNWLCNAuZn8DbX4iW5hEohk6sY5OntbFSrdNh1YWUkNE+6OOzA1s6URaXBaXtrKee5hFx1blYlnea6u7rmsKiOMcbqd1Xio7cQO7OfN2J3VKURKUpRFFpNAWDPdSGO5L3dyl5NIbu86guYiOnLFIJ+dsUUcFERUKhZAApKnfYvGQ2ttDZwRrBBbwpbwRR7qscMSiNETY7qFQADvv2rLpRFELnw0xUofqWjSmV5nnkluLqSaf1iKC3miuJnmMl1bSQ2VojQSFo2FtGCp4ipNlMfDdRNBcQQ3ML8ecM6JNE3Bg68o5AVbZ1VhuOxUH3Vk0oiimK8OcVaSQSwWKQG3IaNYnlSIsktxcRtJAsnSnaOa8unQyK3AzErt223ucxUd7C0E3W6b7hhDNNbMykFCpktpEcoVYgrvt5e8DbOpRFH8Louwsrj1q3tVhkEZiQI8nRjVkgibpWxfowM0dpbKzIoLCFdye+/s1JpKyyMkUt1b9V4o3hVg8sXKCV4ppLeTouvXtnktoGaKTkhMS7g1vK/GYAbkgD6T2FEWm05pazxzzPbQdJp1jRyXll2iieWSO3j6zt0LZHubhlhj4opmfZRvXnldM2l3LLNcW6XDT2MuNk6vJ0NjcHlPbiNjwRJSE5lQC/SiDEiNNvbd5+zh7yXtpFt59SaJP4mFaa58SsJHvyzmJ3HmFu7d2+6khNWI7JNJ6DHHqaSpBjjoFudPYOCwjeK3RlEkrTStJJLcTSSsqoZJZ7h3llfhHGoLMdljRRsFAGmuvDrFyzS3DWh6srvL1RNcK0cslxDevLb8ZR6pK11a28paHgS0Sk71qL7xr09D3bMW7fulmn/wgibetRc+kRp1Pm300v8AyWt0P82Jaux3FeMno2eQ/wD5u/spiCQ/lPcrOw2Nis7eK1gjEUUCCONN2bZR9LuSzsTuSzEkkkkkkmo5deG2KlSVHsy3XllmnfrXIllaeI20qSzCbqS27wMyGFiYyD82q7vfSgwqHZLfKzftSGBV/wDm3Kt/hWqu/SqsR/R4q+f6Oo8MX8BfarsfBK9n6Wd3bQfEhZBZJj+UroZFAAAAAA2AHYADyAA8hUWi8O8Wvqu1kpFnDbwQoZJmjMVpI81uk0Rk4XYhmmlkj6wfg0jsuxJNURcelg/fhglH0F70t/8ApW0H/wBa1N96VORP9FjMfH9HVaeb+CSPer0fAS+HaxAdb2fIlTFgm3e8Lp/S2mbTFxvFawmJZDGXLySzu3SiS3jUy3Ds/BIYY0Vd9lCAACtxXGNz6TedfyTGRfu4JT/m3DVp730gtRyH2cikI+iK1tNv75YWP+NXY/s5vR2pYOtx+QKyC7ZejvXc1K4DuvGXUEvzszdDf6sRQ/5Ua7VprnX+Yl355rLNv7je3PHv9C9XYf2Crsf2Z2w+nKwdWI/IKYut+0hfRWsa5v4Yv6SaKP8AeOqfxGvm5fZi5nBE11czA+YmlkkB+3mxrB2/ZV2P7MD+e0d0f+5TF1Ha73L6MXuu8TAdpcxi4j/wyXdsjfdMm9am78XcDF87N2B/dydb/JDb18/a/avR/ZlZR6czj1Bo/usgutu1xXdF16QGnE3Hwmzke6O1vG/uY24U/wB9am99JbAxg8Tfz/sitwu//ryJXFtKux/ZvdjdXSH9Tfk0LILsi3lW/wCkl4pWepjjPVILyEWIvep62sSFvWfVuPAQzSeXq7777fOH9lQUr8rsrtu+GwWdtnhrhbWlTU5kk+8lXIomxtwt0X7XTfoJeeoPsxP+vrmSum/QS89QfZif9fWg4c/g036P3Wqvb/uT2fFdPUpSvnpc6lKUoiUpSiJSlKIlKUoiUpWFd5e2i/pLq3i28+pLGn8TCpNaXZAVSizaVGLrxEw0RIfN4lSPNTeW3L7ok3rU3njJp+LctmbRtvqi8/8AhCjb1bjuy1yehE89THH5KYjedAe5T2lVVcekLpxfLISSf8lrdj/MhUVqL30m8HH82PJz/uoI1/z50q9HwbvR+lnf2sI+NFkFmlP5T3K7KVz5delTjh/R43Iv9HUNvH/DK+1aa79LAbkR4Ike5pL3if7UW0P8VXI+Bl8P0gPa5o+JCmLFMfyrpylcm3npV3x/osVZRn3dWSaYf3Jw3rTXHpPZxvKHFR/8kE5P/wAy6ar0f2f3u7VjR1vHyqpi75js967LpXDt96Q2opPm30MH7q1tj/nxvWpuvGvUMnzszcD92lvF/lQrtV2P7NrxPpPjHa4/5VkF2S7wu+KV87rvxFzMpJfN5Y7+YW7uEX7qSBf8K1V3qG9m/pb68l38+rPNJ/G5q7H9mNoPpTtHU0n5hTF1u2uC+kNxcpGN3kRB9LsF/iNai/1ljLf+myuNh/e3VvH/AByCvnCR7/8AH30q9H9mDB6doJ6o6f5isguobXe5fQe68VsFH55vGn93Okv+UTvWlu/HrTkRIOVDkfV295IPvpblf8a4TpV2P7M7CPTleerCPkVMXWzaSu17z0kcAnzZbyb93bOP84pWmufSmxI+ZYZZ/wDmS1Qf3i6Y/wCFcg1+Vdj+zu6m64z1u/sAsgu2Ib+9dUX3pXQD+iws8n725SH+CGStTc+ldOf6PCwJ/wA908v8MCb1zbyH0is2xxVxON4ba4m38ulFJJv9xTVscCbliFXRdpe/+9FLwGAaj3q7Lz0pMux/R2WKjH/vx3Mrf3i5Uf4Vqrv0ks+/zZLGH93bg/5zvVe2+hMtJ8zDZZ/2pZXTD+8RbVubTwfz8vzcLeD94qw/5zLtXvijg9B6TYh1uafiV7xNmbu71n3HjtqOTzy7j9iW9nH/AIpbg/41qL7xVzs3zs3kR+6ne3/yCtby18AtRuR/7K6YP9aS5sgB9qi4Lf4VuLX0ac8+2/wfF+8uGO3/AKUTV74Vwcg0MA6hGfgmOzN5vuVa3OsclL/SZXJSb+fUu7h/4pDWnup3lPKR3kP/ABSMXP8AexJq97L0Wcsx/S3+LjH/AJbXMx/ua3Qf41uLb0UJj8/Nwp/yWjyfxXKb175WXHB6MrR/hY4/+ITwuzt0PuXNe1ftdU2PopW4H6bM3En09K3jh/jlk2ra2/os4kfPv8s3/I9qg/uNqx2/trA/h/dDdHk9THfOiibwhG33LkGldr2no3YBPnRXk37y5Yf5IStpa+AWnIyD8FcyPfJc3rj+1DccT/dVJ/2k3a3Rkh/S35uUDecW4rhSvyvoJa+EuBj+bhMedvrIll/zd962tloXFQHeLD4uI/THZ2yHt+1YwTVKT7TbMPQhcesgf3UDejdjV85uQ+kVm2OKuJxvDbXE2/l0opJN/uKa+kttYQxf0cMUe3/Aip/CKyKpSfaefyWfvk/2rGb1OxvvXznt9CZaT5mGyz/tSyumH94i2rc2ng/n5fm4W8H7xVh/zmXau/6VTk+021n0IWDrLj8woG9H7AFwpa+AWo3I/wDZXTB/rSXNkAPtUXBb/Ctxa+jTnn23+D4v3lwx2/8ASiau06VSk+0e9HaBg6mn5uKgbylO7uXINl6LOWY/pb/Fxj/y2uZj/c1ug/xrcW3ooTH5+bhT/ktHk/iuU3rqalUpOHt8O0kA6mN+YKgbfMdvuC5vsfRStwP02ZuJPp6VvHD/AByybVtbf0WcSPn3+Wb/AJHtUH9xtWO39tX3Sqb+GF7v1nPYGj4ALGbZMfzKnbT0bsAnzoryb95csP8AJCVtLXwC05GQfgrmR75Lm9cf2objif7qs6lUn8IbyfraJP8AqOHwKgbRIfzHvUJtfCXAx/NwmPO31kSy/wCbvvW1sdC4qA7xYfFxH6Y7O2Q9vLusYJqQ0qk+8LS/0pHHrcT81AyOOp96+Y1dA+h1quwxfw2b29trMTDGdHruEMnT9d58Ae78eom+3lzX6a5+pX0pfF2MvKyPsryWh1Mxrk4O29S6aeISsLSu8r3xz07D87LxN+6hup/8YYGFam69IzTyb8bq5l/d20w3/wDVVa4jpXHx/Ztdw9J8h/U0f5VTF2R7yuwrr0o8OpISzy0m3kelbIp/aCbrl/eK1N76VlqAeliLqQ+7qzxQj+9Ek2qo8R4UQyY3G5G6z9hjRlFnNtFdpINzBJ0nBmU8VAJQljsBzFQrXWl7nDX8+OuggmgK7tGS8To6iRJI3ZQWRlYHuAR3BAIIELFwV4PzSmJmJzhXIue2uE4XU9EGhyOGtCvI7JZ3Gg+avm59K+U/MwcSf8940n8Nqtae99KbKt/RWGMjH/mLcTH+9Z0H+FVj4caCnzRuZBPb2VpYxrLfZC7bhBCrkhV7d5JW4tso2Hs9yNxvsNZ+GrWdh8LWeRs8xYLKsE9zaco5LeZtgqz28neNWLRgHfuZF3A3Um6Li4Owz8QYxiqBQmRwqcwCSS0OOwE1OwKfEWZrsNM+1Sm69JbPP5fB8X7u3Y/5srVpbrx81HJv/wC1emD/AFY7azUf2Mbct/jVa28fN0Tfbmyrv9HI7b/41cOV8DUivGxiakxLZEFFSxnEtszvIiyJGJDyHUZHQhQCTyFXrTYLjsDg2WFgJBI/pY8hSpOTqAVFSaBZHx2ePVo7qqJ3ni7npuzZq+G/1TiD/GFV2rTXGt8rJ8/MZWT/AJ7y5f8AilrV5nGzWdxNa3ETQzW8jwzRNtusiHYjdSQw+hgSCNiCQQalXh54eS5WC5v5Lu1xmPs2WO4yF4Tw6rAFYIo17zTe3H7O4/pF7kkA7OSK7rJFxxYxrcqEMGddMNBUk7KVJ2LKWxMFaCnUole30s/eWaWbvvvK7Sd/p9snvWNt+ypRr/S0WMa3MGVsMrDdRvJHLZsRIgRghE9u3tW7Ft9gSd+DeW1R6xtXnligiQySTSJFFGvznlkYIiDf+sWYD+2r9nnikiEkeTc9hbprUGhFNtaKbXNLajReiv2p14u+Gs+nJLVZLiG7juo5Sk8AIjE9tJ0p7c7knnGxTfy+dtsCDUJtYGldIkUu8jrGijbdnchVUb9tySBXlltsNphE8TqtNaHqNDruIK9ZI1zcQ0XrpVmat8LYMYlzFPqHGLkbS3682M4TgkmMTC3juivCS4ZGXivEbll8h3qtYVDMqlgoLAFj5KCdix29w8/7KhZLwhtTC+EkgbcLhXKoIqBUEaEVBXjJWvFWrxpVwL4OWRsmyI1VjTaLc+ptc9Cfpi54CXpee/LpsG+jvVR3UYSR0VxIquyrIu4V1UkBwD3AIG/9tQsV52e2FwhJOHI1a5tDu5QGfRqF5HM1/o/AheutjpvB3GSuUtLWITTyA9OLnHGz8RuQpmdQzbbniDvsCfca11fsblSGUlWUhlZSQwZTuGBHcEEA7/sq5IHlpwEA7CRUV2VFRUdoUzWmSs218AtRyEf+yumD/WkubIAfaouC3+Fbi19GnPPtv8HxfvLhjt/6UTVKvBH0inh6VhmmaWPskWU7vKnuVbpQN5l27dYe12HINuWHUdjdRzxxzRSRzRSoskUsTCSN43HJXR1JV0IIIIOx3r8nvzhVf12y8XMxjQfRcGuLXdRJPcaEblqJ7XaIjRwHcuR7L0Wcsx/S3+LjH/ltczH+5rdB/jW4tvRQmPz83Cn/ACWjyfxXKb11NSuak4e3w7SQDqY35gqsbfMdvuC5vsfRStwP02ZuJPp6VvHD/HLJtW1t/RZxI+ff5Zv+R7VB/cbVjt/bV90qm/hhe79Zz2Bo+ACxm2TH8yp209G7AJ86K8m/eXLD/JCVtLXwC05GQfgrmR75Lm9cf2objif7qs6lUn8IbyfraJP+o4fAqBtEh/Me9Qm18JcDH83CY87fWRLL/m771tbLQuKgO8WHxcR+mOztkPb9qxgmpDSqT7wtL/SkcetxPzUDI46n3rHtrCGL+jhij2/4EVP4RWRSlVSSdVBKUpXiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiL5jUpSvrJdclKUoiu6419irfT2mbeXG2ebuLSPI9SGeWWP1NnuEdQ8aLxmWVdjxbcbQ/tNVdr7VVxmshPkbnh1ZyvsRgrFGkaiNI0ViSFCqPMkkkk9yawNPY43l5aWatxa7ube2VtuXFp5FiDcdxy2Lg7bipjnNDWMdrmJ7bLXN1JhpIY5opbBbWKRprsWX6OcX0h+scbp3Efu3rnILNYbtmzqXvJzIe6nGS6ZVawFxA/LWlTXVVmtjid0nr2nuC2PhPqGxkxOX07kLpsfHkpLa4tsgI2mjjubd0YRzontdJjFF37Ae3uV7Gs/M3uNwWn8lh7XKR5i8zM1o08tsjrZW1taSCVdnk3WWdmDj2e/tDfjwUtDtO6XtJcY+UvchPZxC/GPiS3tBeu8gg9ZZzyu4QiBSo9/dhWB4haeXFZGaxWdrkRJauJniFux9Zt4rriYRLIEZevwPtHuhrD4HZp7Y5jZHZuEjo8PIL48AriLdh4uoDtadKjgY55AJ1qRsqKbe5aaxcLLExOwEiEn6AGBJroTXkeBu9SNqKTU9sIFnsbj1O2gnmumayigjESsg2Xm8Hztuwb+0V94M+EVxqaO7liu4bVbR4Yz1Ud+bSh2O3Dy4hB96rA+Spe/raz/8ASl//AJqlfd53aLVhltRiexrmOAbXJ+EkVLXAHIUIzChaJYsVC6hGRy30VNeJ+pRmMvf5JYzEt1PzjjbbksSKsUfPiSOoUjUnYkbk9z51MvD7L4+/0/cabvr4YqQZFcnYXzo0lq0nSEL29wI+6jj1CGYge2vvQK0xl9Fe8VWb4Ws/ZBP9FL7hv9Nc7g1fskt33pZxBY5MocGEtBqwj0PSFDkCCCDUKbDHM3Cw+jTs3aqUa/wVjYPbxWeWTLOUc3jwwvFbxScgUWKaQ/7QCjdyBsCnn34rvPAS/sbDIy5a+lQLi7WW5s7Un9LdXxUrDFGOJHb225Hbi3SP07Z3hb4HZPOxrdAR2Nm/dLq65cpV7+1BAg5SLuB7TFFO/ZjsRVhXvooTCMmLOQySbdkltGhjJ+gypcyEDf38D9lU7xvy7I4XWG02qrjVrnAVOZ5QJY3C3KrdhA6c1CS0RBvFuf1n/hR3O6zw+awOUsOE+OuLe4fM2LX9wbo3F7KztdQI6xrwMitIQh9kvPy7capK0AMiAv0hzXeXYngNxvJsntHiO+w79u1SDxB0PfYG5FtfQdMuGaGZDzt50UgFopQPa23XdSAy8l3A3G8brdXRY7PDC42Z5cx5xDlYwKihoTU5kVNa51WeCNrW8g1B7V0vkNVY97C7XL53E6ltzZSpjwLFoc4LgjaE8+INqQCwLv7W5BLDvvzPX7V0+Hvo7X+Wx8GQa6gsVuR1IYZkkeRoDt05jx2ChxuwHf2Sp9+woxMsVwRufPLQPI1AAqB+VrAAK6uIGaxtDLMKuOv1oFHLTOWw0TcY4zoLttRrdrbd+obb1OKLreW3DmrL5+6q5qQeIWnUxOQnx63kV81senPNArLGs435wDkTyZOwY+5uQ81NR+ttd0UTWOliJIlcZMxT0gNAaGmQ1zWeIClRtzSlK3Oh/UfhGz+Eut6j1l9b9X36nT7/APD7XT5ceXD2+HPj7W1XJpOLY59CaAmgFSaDQDaTs6VNxoKqz/Rw8HGzkq5G9RkxsLkKp3Vr2VDsYkI7i2VgQ8g8yCinfkU7NtoVjRI0RY0jVUREAVFRRxVFVRsqgAAAeW1Y2CNv6rb+q9H1boxeq+r8eh6vxHT6XT9npcOO23bbas2vnDhDf0962gySZNGTWc0fyO0/IALmrTaHSuqewJSlfjHYbnsB3JPkAPfWhVdftKi2P8R8LcXIsoc3iZrln6a28V5bvM0g7GNEWTd5Ox9kbnsforLsdY2E+UusLHdK9/ZQpcXVqFkDRwyCJlcyFBGwK3EJ2DE+3+w7TMbhsO/TZvRb6lKVBEpSop4j+INhp9LOS/lkjW+u0s4OnFJMTK4J9oRqeKgD7foBqTWlxoBUopXSobr3xSw2CngtsjkorWe54mGDhNPMVdiiyNHbRu0URdWUO4Ckow37GvdjvEPH3Gdu9ORyyHIWNsl1cRGKQRrE4hcbTEcGYLdW52B/8QeZBAlxT6Voaa6bN68qpZSlKxr1KV67idIwC7ogJ4guQoLH3Ase57Ht+yovrnxEx+FusVZXkskc2ZuTa2KpFJKHlDRRnmyAiNepcwLufrPoBIk1pcaAVRSylKVFEpSlESlKURKVj5G9itoZbieWOCGFGlmmmZY4o40HJpHkchUQAEkk7Dao5jfErCXMqQQZ7DTyytxjhhvrWSV2P9VI0lLO37AKkGOOYCKV0pSoolKUoiUpSiJSleq8uFhjklc8UiRpHOxbZEBZjxUEnYA9gN6IvbSoz4a65stQ2AyNhJJJAZZId5Y3hcSREBhxcdxsVO43Hf6QQJNXrmlpociiUpSvESlKURfMalKV9ZLrkr9jG5A+kgf31+UDbd/LbvvXh0RW0w09p/N9/h+4mw+S3APqIgknsZt17gh+mXjB9x2qN6L1HYx2OVs8j8It8JT2M5lsBb8/9kNxIQ3rLAbPJcK3Yf8AhipTobxCyeSOYF3eJPHDgstdMptrNS0xh6ETl44A+4nuY28/MCtVYajusJp3GPaSxQyZPI5WeSRobeZzBax2NrHH/tET7J1PWG7bd9641scucMwxSkxtrxppVodICDg5NMJdQNdXIVWvofRdmchr27slpdYZuwfHWeMxwyAhivby+uXv+h1GluIrW3QILduPFI7Z/PbvJUuykmE1LlbyWM5uC6u4b26jWQWXqqGxspblYzwLScOnahPee4+2s26aXI53REdw0czy2mNvrpzHFEHE17cXjhkgRUKizjhXy8l77+dQ7I+LuWmM4W8RI5+svTS2slYQzBkMYdbfmB03K7g77E96RxyztaLOMLw15xGU5Y5Dn6BxgmOuYZkKL0AuphyOedd56s9F0D6D8G2FyEv/AB5Ro/7I7a2b/wCspqP+NnjrlcRnr/HWosuhbG2EZlhaSTeS2hnfkwlAP6SVx5eQFSD0WNVYvH6diiucrjbWaS6uppIbi6t4Zhu/TUtHJIGXdI1I3HcEVYE99pbIzHnLpm+nnYcubY65mlYAKN+RZ5GAUD39gK4C2Txw31aZ7VZjKyrmgUNMiBirSmgPeqD3Bs7nPbULmqT0lc6wKkY7ZgQf9nfyPb66ol4D6OXN52zspAWt05XN2Adt7e3AYx7juBJIY4yR3AlJHlXUXiJ6P+IyUMhtbdMXdcT0prUFLfkN+Ky2g/RmMk9ygVvLv22qrvQ8w72eosxbTpwuLSyltpF332K3UKybHyZeUSbN7xsR510tnv27vFdqlu2PinhoxCgBz5LXCmRALjQ7DqM1abPFxTjGKGivvxd1rFpvEPe9JHZTHa2VsP0aPO4PCP2fmxrGjuQP6sRA91c7aS9JzJJexG/itJrR5FE6wRPFLFEx2aSFubcigPLg/Llx23XfkJr6ckjfB2KX+qb2Zm+jksJC/wCDvXJtYuBvBqw2u7DLaGBzpC4VOoANBh3HU11UbFZY3xVcKkr6DeL+j4s/hrm02R5DGbixl7HhdIpaF1b3I2/A7eayNXz5Br6K+FMrSYHCO5LM+Ixrsx8yzWsRJP7STXE2L0VNm9S3eMtQFByF6ZJduUcFpHcMrzsoI3VQVAG45MyLuOVYeANv8FbaoZnciLlVOgpUOPbQfRXl3yYMYJyGaknozeF3w7feuXKb46xkHWVhutzcgB0tB7umAVeTz9kqu36TcX56SvieMDYC0tnAyF8hW3A87e3+Y92QPJuzJHv5sCe4jYVKMpeY7R2B5bdK1sYhHDECOtcXD7lUB29u4llLMzbbDd2OyqduXvCNY9YarmfLp60tzBczGJZJYUj6XERRRtA6usUaniBv323O5JJqNmN+WqS87U0+DWcEtbzqZ0pkCTkX57m6KGLj3mV/ot2fXvVOsxJJJJJO5J7kk+8k+ZpXRHpT+GmJweNsriwszbSTX3Rkcz3M3KPoyvx43EzKPaRTuBv2rnYV+nXPe0N52cWiEENJIo4AHLLYSPetrBM2VuJq/aUpW1WVXF6O/jLJgZVsLxnlxkz+7d5LKRzuZ4lHdoCSS8Q+kuo5clk7SsrpJ445opEljlRZIpY2DxvG45K6OpIZCpBBHnvXFno/eC8uekW+u1kgxkb/ADu6S3jodjDCfNYARs8w94Kr7XIx9n4uwitYYreCJIYYUWOKKMBERFGwVVHYCvwrh/4v8N/9N95/7lPRrs/XzqdvKqtDeHF4+Trt+t6yaob04MtPb6ftbeOZraDI5a2schcJ/VsninldW2/8MtCpYeTKjKdwxFXzWn1lpm0zFlPjr2Bbi3uFAkjJKkFSGV0dSGjkVwrBlIIIFcRZ5BHI1xFQCtcVBj4C6YaygtThbJo4ugyXAHG6k4MpUyXqETTByACCxDByPoqv9O3KW/iXrOZp47RIdOwSNcycelAkdri2M7hiF6aAciCQNlNbuy9Gm0XowS6g1PdY+3eJ4cTLfFbNeiweNOMKLwVWAIMQRhsCCCKmE/g/YS5bNZaWW7mOexnwTe2jugtxbGG3tyY2RBOspS1X2i57ux+ja2JmNxBzy6oI03uB29S8XPlz4sZbGPiMlFqm81DbXeQt7W7hucE+Jx0sU3Pm1rfSQKGdRGwVUKtuvIgqrpUp8Xdc5m11Bk4bvL5TTOKto4Pgu8scQuVs5yyBpLi5uipbdXOxjU7dtvYK7yS+b0cLaW0tLCbPZ+4tcdPHNj7aWa2MVv0yxC9NbYLKdmKhmHsKWVOIY1vdaeDZydzfzfGXUlpDkvZvMfb3UXqRjMawNHDFJA3QRo1Abz33O+47VlM9nxDTaK4RvFNlK0rs6OleUKrbxJ13mUfAdHMX/wAC3GGhnu9TYfFR38l3kfbWQywhOOOh9iNhGqhlMjqQ+x6fr1L4j3K6f0zdWWpBm2udWW2Pu8ibK3tJGhbqv6o9lLDvbTLF0Dz2DnkGDcXBNmX3grEiWUeOzmfwkdjZJYR2+PuVW2eJHkmMssMsTCS6aSeRjKe/te7YVjp6P+MTHWGOS4yAWyzSZ57l5Y5bq7yKrwMlzJJEVZWXiCEVfmDvuSTETQUFdh5o6dctdNDToyShVWw6dyk/ihkIlziJMmN9d9Yewtph8EtdWzLhlhkPFCscqJ62P0h6bH+ualeW8SbzH601XFJIbixxGlJcrDYhIUJnto7CUD1lYutsTPONmZlHWJ27CrRs/Du3i1JcamE1wbq5xwxrwEx+qiEPDJzUCPqdXe3XuWI9o9qx4fC6zGfyOoHeeeXKY04q5s5uk1kbVhbqw6fT5sWW0QEMxBDv28toG0xuPKFaMA0GuX1Ve0VW+HVprLNWNhqKLVNhEb6RZxhpbCNselmZirRm4jPXZxEpPDs++ymUHdxj+MusL1cplorDWN3BJj4kdMTjcE+Ujt5BCH4XuQihdU6kqS/O3EYPcHiQZJbejVj4nSJMxqNMdHcrdx4RL90x6SpJ114hFEigSAMHDCTccufL2q3OW8DbaXIZO+gyuZxqZoqctZWM0UdvclQwPtvA0sIfqS8uDA/pZNiu/aZnhx1qKbBgA2jI5GuVc/fmlCqE8ZtSZDUWkNIZma8jh9YzKWc1pHBGyHJQyX1vFlVlJ5ACO2k3t/mE3B/4RVpeKmazGDn0HYy5j1+e91Atrk7z1O0tvW4Jry3VU9XCOtqVtrgxcoirHYnfc1KZ/AfHSaZt9MPcXxgs7l7y1vFeJL2K5aWabqKyxdLyuZo9uHzXO2zbMPdceDEM6YH1rMZi/lwGSOTt7m7limnmmM8VyIZ5HhLNbq0KqFBBCsQDsFA9NphNBsBdlh2GuH6/slCoJjszqLV2X1BHjs9Fp2zwN++NghSzhvprmeFpI2lnacho0ZoiRtuAHC8CUZmiWd8cM3PpGxvYZ4rbKW+rFwF3NCkTW13wtJrhTxkRlRHY24Ypx7xsV4Bgot3WXgNZ32Qusla5TNYKbIbfCK4e69VhuyN93lj4H9IeTb9+JLM3HkzFsjJ+A+Kkw2NwcLXVpbY3IxZRHhaNria7jSVC9xJLGwk59Zt9guwRFXiqhQFos4pUbssIyyzqfzVP0EoVCBmdRac1Tp2wyWejzlrqJrqKWH1OGyFtNCqHeExbsVV5odmLAMvUBQHiwsL0lL3KWen7rI4m6a2ucayX0ihIpUms4d/WYnWZG2RYiZSV2b9BsD3rb628O7fLZTCZWWa4jmwU081rHEYxFI1x0uQmDxlio6C7cSvmal9zCsiPG6h0kVkdGG6sjDiykHzBBI2/bVV87S5jqCo9IUABzOzTSi9oqIuPFG6z2c0vYYadoIZ8YdRZoKIXf1QhRDjWkkVhCxuAYZCuzDrxkEbd4BpLxIy3rls2e1RktOXUmTSOXGXWDjTFNb9Ub2kN+8bcQ8e4FxIfZB3JYDqG7vBrwWx+lfXzZS3cj34jR5bhomkiii6nCKJo4l2AMpJLbklU3J2FaN/R7t5xBBfag1JlrC3mSdMZkLtJrd2j34pM4hEksQBI4grsOwIqy2azgloGVMjQVOpOoI207BmF5QqX+kD/ALq6j/6Lkf8A9vJXINxm9MTaOtcYmGa41DPbxW8M1vYslw+QaYKr+uBQbvuyjgpcvyVdjv27g1ngY8rjr7GyvJHFf2s9pK8XESLHOhjZkLqVDgMdtwR+yoXmfBjH3enLPTcsl0YbAQmzvVaNb+GWFiyzJII+CyFWdD7GxVz5HYiFktLI2gOr6QORps27+rJCFUeqddZqwk0zpRr66xk6YCzvczkbWyfO5NpAjw+qRQRJJyIa3Ie47925cvZIk8rTxlzeJxGpnuYrvJ/BaWT4bMZDGz4j1lb6eK0ZZ7Z40Vnge4UgKAX6bbnZgwtrXfg7a5hcdNLf5O3yWLhENvnLKVLXJMgUqwmeOPpuG5Ox4quxlk48Q7A+EPh38GYPNWxfKaslv4pZJLXM3iubmURCNLaOVwiWcZ4p7Y9peKkH2EAnx8BaKtFcqilPzVJB6RlmadG1KFUt4mZXVuB01Bn21hBe+vrZFrcWNnH0Tdp10azuVUrc7AcSOmimMyMACoqW+IeodQ3esrbT2LzCY2G5wEN7NLLbwXQhPUlEtzGjpye4bhDGFLcQJHPYgEVTmfCWfKxW+Kx+ltQYx3uEEl3nL3r43GWzNvP6lHzCz8iRu2xcqpAG7bjqxfDq2GoY9R9a59ZixgxSwbx+rGAOZOZHT6nV3bz5bfsrJNJEwAmhPK/K3ow1AqN/1r4KqjtWeI2Vu89kcNHmchibfBQ21u9xjMPJmrq9vXiBe5uEiik9Ug6gchN1Djy3O5Re+JeqpdMRyLaX8F3b5v4OyGUgxkj3b4kRiVctBiLmJAHfkVYcQoKecRcGO1deeC9rkck+Ytsll8HfTRLDdXOHuBbetRqFVfWEKHmyqiAEEfMXcEqpGRfeE4fF2OMjz+o7drC4kukyMV7vfzSydTkLmZ4iJof0z/owFHf3998XHwUbkNlajcM+up316sl7QqrcDr3JNhtRvidQ3WpslaxWRtMde4pbHIWIeYx3U3q6xhsi/SbkqgMFaIAq/Li2T4I66lnyAtrnV2Qurp8bcSvg8riIsVObpE6hlhnEexWNUnPRRiSEZvaCNtNsZ4FW0MeTdsxnpb/LC2S5zXrSRZNI7R45YoreaOECGPlBED2JIQLuB2rI0t4LQWuRjy13lsxmry2t5rWylycySLax3CNFI0SRRLvK0ckilmJB5eW4BB00Ba4d3JFdB0aV3YTtzKUKpzTnjBmBo/T3SuITldQZybExZCaGFIbZGuDEJehFEIeYaSPbkhG3UJDEVM7PO53TGpcFiMlm01FZ6h9YiSR7SGxubW4twu7qIGPOItLD85iNi+wUpu0rtfAPFLp2LTcj3k9vb3El3bXbPGl9BcyMzdaKWOIIGHUcbFCCG7g17dBeCVrjclHl7nJZjOXsEZhtLjMXPrXqsbBlPRXiCHKu43Yke2xABJJ9dPAcVBqXZYRnX0aHZTd/dKFVBo7L6tz1lqK+g1OtjFhclk4reE2VrLLcNaoJhbvP0wLe3WIRBW4yEtNLyBCrV5+jvrKfP6axmUuQguJ0njnKDiryWtxLamUKOyFxCHKjsC5A7AV56C8L7XDWWWsYbi6kTMXd5eXDzGMyRyXsawyLFwjVQgVAQGBO/mTW08KtEwadxNtiLaWeaG1M5SS5KGY+sTSXLcjEir2eVgNgOwFYbRPG9pDQBmKZAZUNa9tNUAKlFKUqipL5jUpSvrJdclTHwUycVnnrC5mnS2jjNzynckJGz2s8aMSoJA6jp3APnUOpVe12cWiF8Lsg9pae0UUXtxNLd6tvVs+ZlxV9M+q7XLWsHq0V5b21zdysRcycIgUns40YFo2PzvKM1+6djy+PsLJE1ZYYiG4the21lNc3kUiwXDyMHMcNk6LzdZG7MfOo/oqTHyYa/sbnKrjJrrI2M5L211dLJa2UN0An+zRsFbrXe/cj+j/aKw/EPJWt3kbOKK561nZ2OJxou1jli5x2tvFHcTCCRRIn6Y3B47b9u2+4rl2WQue6y4BhDi6vEAAgMaBSrcBcXF1KVdhCphh9Cn/b0dy3s+JzD6kuIpMvCMhjbeS4mysk0/Qit4LYTM4mEBmAWGUrt0/Mkftrc6mz85wWYtr7U9lmZLk471OC3nuZ3VobpZJm2uLWNVHTA7gn5teGqM7jFudTZSHMpezZe1vLW1s0s7yB0W8uYNuU08Yj/R2ccinuN/d7hVN1OxWI20MfI3BxfFU/pBhxNo91MQDg3FlyaDWi9jjMlCcqU2U6SrItPBXMz4q1y1vai5jukaQW8bAXaRAkJKYX26scigMvTLNs49n31GE0PlTIIvgfKcyduBs7kNv+1THuP7a6B0F6T9ukMUGQxskJjVIllxvB4OKgKP8AZpnVoEVQOys/l2HuqfD0itPcd/XJwf8Ag9VuOX2biPj/AI1pJeEHCCzPcySx48zhLQTQVyBLag0/Sd+awm02lpILK/XQpL4G4a7x+n8bZ3xPrMETh0LdQxxtLI8MBcHYmOBoo9huBw2BIANUVp/WEFp4l5CQOogvpXxTyb7Ks3TgT+3e9tFTf/3yayPE70mxLBJbYi3miaQMhv7rgjxg9uUECM27kdw7sOO3zD7uaXYkkkkkkkk9ySfMknzNVeDvBa0zeEz25vFmdrmho2YnYi6myhAwg57+mFmsj3YnSZYvmu6vSU0LLncKYrZed1aTpd28e4Xq8VeKSDk3YM0cjEb7btGg3AJNclaQ8Ksvkb2OzGNvbXk4Wa4ureWCGCPcc5HaZVBIXchAeTbbAVbvhP6S3QgjtMxDNOYwqJkLfZ5WQdh6zC7Dm4HnIhJbtupO7NY196R+n44y6T3dwwG4hitpVkJ+gGfhHv8Aa1UbBJf1xxPsUdn4wVJa8AuArtyypto6hB1UIzaIAYw2u4qf6lytvgsRNctssGNswI0JALdJBFDApOw5u/TjHluXFc3eidrnHYxMvLkb2C2nu57d1eUHqSACVnO6IfZ5vvt9LVCPG/xjudSMsCx+p2MT847YNzklcbhZp3AAZgCdkHsryPdjs1VfW3uLgU4XdLDbCWvmLS6hBLQ01ArmKk1J12blmgsP9Mh+RK74k8Y9Ot87L2jbdxush7/2x1tNJ6+w+TuPVrG+trmcRtL04lYMI1KqzbsgGwLqPP31886sb0eNa2uAy7X12J2iNlPbgQKsj9SR4WHss6jjtG3ff6KpXl9ncENmfJA97ngVa3k5ndkFCW7Q1hLSSdy7X1jqewxcUc1/cxW0cknSjeYFlMnEtxHFTseKsf7K4e9ILL21/qTJXdpKk9vMbPpSxghG6dlbRPsCARtIjjy/qmp76SXi5jtQ4+0trNbtXgvOu/rEaRrw6MkfYrK27cnXtt9NUNWy4DcG3WJvhcwc2RwcwsOQAxAg765DaslgspZy3VB0olZ+nLqCC7t5rm39bt45kee25mLqxA7tH1F7ruKwKV+gyMD2lp25ZGh7CMx1jNbIioovo3oHP2WSx9vdWDRm2KLHHGirH0emAvqzQr2heMbDh5AbbbggnfV8/fB/xJutN3nXiJlt5SovLNm2jmjH9ZT5RzqCeL/aDuCRXe+GvhdW8FyqSxrPFHMsc6GGZBIocJJE3eOQb7FT5EGvnnhTwbkuifXFG+uF23pDukb9DrvA5u1WYwu6DosulKVyyqpSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoi+Y1KgnxxuPq4Puv+ZT443H1cH3X/Mr9684V073ewV0HjGHp7lO6VBPjjcfVwfdf8ynxxuPq4Puv+ZTzhXTvd7BTxjD09yndKgnxxuPq4Puv+ZT443H1cH3X/Mp5wbp3u9gp4xh6e5TulQT443H1cH3X/Mp8cbj6uD7r/mU84V073ewU8Yw9Pcp3SoJ8cbj6uD7r/mU+ONx9XB91/zKecG6d7vYKeMYenuU7pUE+ONx9XB91/zKfHG4+rg+6/5lPOFdO93sFPGMPT3Kd0qCfHG4+rg+6/5lPjjcfVwfdf8AMp5wbp3u9gp4xh6e5TulQT443H1cH3X/ADKfHG4+rg+6/wCZTzhXTvd7BTxjD09yndKgnxxuPq4Puv8AmU+ONx9XB91/zKecK6d7vYKeMYenuU7pUE+ONx9XB91/zKfHG4+rg+6/5lPOFdO93sFPGMPT3Kd0qCfHG4+rg+6/5lbTSniVcY+7hvPUsbeNA3NIb2OeS36g7q7RxXCFyp2IBO247g1CT7Q7ra0luImmQw0r0V2Lw3lFTKvcuxvRl8E+PRzeTh9r2ZcdZSj5v9ZbydGHz/Ioh8uzHvx49NVwB8tXUP6v0/8Ah77+Y0+WrqH9X6f/AA99/Ma/Hb4vie87QZ5j1DY0bAPrM5rSzTOldicu/wClcAfLV1D+r9P/AIe+/mNPlq6h/V+n/wAPffzGtUsS7/pXAHy1dQ/q/T/4e+/mNPlq6h/V+n/w99/MaIu/6VwB8tXUP6v0/wDh77+Y0+WrqH9X6f8Aw99/MaIu/wClcAfLV1D+r9P/AIe+/mNPlq6h/V+n/wAPffzGiLv+lcAfLV1D+r9P/h77+Y0+WrqH9X6f/D338xoi7/pXAHy1dQ/q/T/4e+/mNPlq6h/V+n/w99/MaIu/6VwB8tXUP6v0/wDh77+Y0+WrqH9X6f8Aw99/MaIu/wClcAfLV1D+r9P/AIe+/mNPlq6h/V+n/wAPffzGiLv+lcAfLV1D+r9P/h77+Y0+WrqH9X6f/D338xoi7/pXAHy1dQ/q/T/4e+/mNPlq6h/V+n/w99/MaIu/6VwB8tXUP6v0/wDh77+Y0+WrqH9X6f8Aw99/MaIu/wClcAfLV1D+r9P/AIe+/mNPlq6h/V+n/wAPffzGiLv+lcAfLV1D+r9P/h77+Y0+WrqH9X6f/D338xoi7/pXAHy1dQ/q/T/4e+/mNPlq6h/V+n/w99/MaIu/6VwB8tXUP6v0/wDh77+Y0+WrqH9X6f8Aw99/MaIu/wClcAfLV1D+r9P/AIe+/mNPlq6h/V+n/wAPffzGiLv+lcAfLV1D+r9P/h77+Y0+WrqH9X6f/D338xoi7/pXAHy1dQ/q/T/4e+/mNPlq6h/V+n/w99/MaIuZqUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURf//Z", 104 | "text/html": [ 105 | "\n", 106 | " \n", 113 | " " 114 | ], 115 | "text/plain": [ 116 | "" 117 | ] 118 | }, 119 | "execution_count": 3, 120 | "metadata": {}, 121 | "output_type": "execute_result" 122 | } 123 | ], 124 | "source": [ 125 | "from IPython.display import YouTubeVideo\n", 126 | "YouTubeVideo(\"-_XYmr4vkwc\",width=640,height=360)" 127 | ] 128 | }, 129 | { 130 | "cell_type": "markdown", 131 | "metadata": {}, 132 | "source": [ 133 | "✅ **QUESTION:** What is the difference between Regression and Classification models?" 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "metadata": {}, 139 | "source": [ 140 | "✎ Put your answer to the above question here" 141 | ] 142 | }, 143 | { 144 | "cell_type": "markdown", 145 | "metadata": { 146 | "tags": [] 147 | }, 148 | "source": [ 149 | "----\n", 150 | "\n", 151 | "\n", 152 | "# 2. Machine learning workflow\n", 153 | "\n", 154 | "Below are the major steps in a machine learning project. Please go through them and bring questions.\n", 155 | "\n", 156 | "|Step|Purpose|Notes|\n", 157 | "|---|---|---|\n", 158 | "|1|State the question and frame it as a machine learning problem|Is it a supervised or unsupervised learning problem? There are other major types (semi-supervised and reinforced learning).|\n", 159 | "|2|Collect data, and exploratory data analysis (EDA)|Apply four types of EDA.|\n", 160 | "|3|Split training and testing data|Seperate out testing data for evaluation purpose.|\n", 161 | "|4|Engineer features| - Format data, impute missing values, normalize/scale features.
- Transform feature values, select informative features,
- Create combinatorial features (e.g., the interaction terms for basic linear model).
- Do dimension reduction to reduce the number of features.|\n", 162 | "|5|Select models| - There are many models/algorithms/classifers making different assumptions about how modeling should be done. Thus, it is important to go through a wide range of them.
- And for each model/algorith, we need to tune __hyperparameters__, i.e., parameters that are set manually.
- Cross-validate model, i.e., separate data into training/validation subsets, use the training subset to train a model and score the model with the validation subset. Then split the training/validation subset again but differently, train and score. Repeat the process $k$ times.|\n", 163 | "|6|Repeat 2-5|Work to iteratively improve the models.|\n", 164 | "|7|Evaluate the best performing model|- After a best performing model is identified based on the training data, the model is applied to the testing set.
- Various model performance metric can be used to access how good the model is.|\n", 165 | "|8|Interpret the model|- Dissect the model to better understand how it works.
- Identify most informative features for the best performing model.
- Assess the reasons behind false predictions to further improve model.|\n", 166 | "|9|Deploy model|Apply model to new data and make predictions.|" 167 | ] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "metadata": {}, 172 | "source": [ 173 | "----\n", 174 | "\n", 175 | "# 3. Setting up for the workshop" 176 | ] 177 | }, 178 | { 179 | "cell_type": "markdown", 180 | "metadata": {}, 181 | "source": [ 182 | "## 3.1 Setting up required packages\n", 183 | "\n", 184 | "✅ **DO THIS:** In Jupyter, you can execute terminal command by adding `!` in front of the commandline. Let's use this message to install the required packages:\n", 185 | "\n", 186 | "- Create a new conda environment called `ml_workshop`.\n", 187 | "- Activate the environment,\n", 188 | "- Install `sklearn`, `pandas`, `matplotlib`, `seaborn`, `imblearn`, `numpy`, `shap`, `ipywidgets`, `tqdm` by following [the workshop instruction](https://github.com/ShiuLab/ML_workshop#24-install-software-packages)." 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "execution_count": 2, 194 | "metadata": {}, 195 | "outputs": [], 196 | "source": [ 197 | "# Test your install\n", 198 | "import sklearn, pandas, matplotlib, seaborn, imblearn, numpy, shap, tqdm" 199 | ] 200 | }, 201 | { 202 | "cell_type": "markdown", 203 | "metadata": {}, 204 | "source": [ 205 | "__Note__: If you encounter `ModuleNotFoundError`:\n", 206 | "- Close the terminal window.\n", 207 | "- Close your browser with the Jupyter Lab instance.\n", 208 | "- Open the terminal window, and follow the instruction before about opening this notebook.\n", 209 | "- Re-run the above code cell.\n", 210 | "\n", 211 | "If that still won't work, don't fret, we will see if we can fix this before workshop." 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "metadata": {}, 217 | "source": [ 218 | "## 3.2 Where the data is from\n", 219 | "\n", 220 | "We will be using a dataset `enzyme_gene.csv` from a [PNAS paper](https://pubmed.ncbi.nlm.nih.gov/30674669/). Let's use this dataset to illustrate how an end-to-end machine learning project is done." 221 | ] 222 | }, 223 | { 224 | "cell_type": "markdown", 225 | "metadata": {}, 226 | "source": [ 227 | "✅ **QUESTION:** Read the abstract for the paper. In the space below, state the biological problem as a machine learning problem." 228 | ] 229 | }, 230 | { 231 | "cell_type": "markdown", 232 | "metadata": {}, 233 | "source": [ 234 | "✎ Put your answer to the above question here" 235 | ] 236 | }, 237 | { 238 | "cell_type": "markdown", 239 | "metadata": {}, 240 | "source": [ 241 | "---------\n", 242 | "### Congratulations, we're done for now. Will see you in the workshop!" 243 | ] 244 | } 245 | ], 246 | "metadata": { 247 | "anaconda-cloud": {}, 248 | "kernelspec": { 249 | "display_name": "Python 3.10.4 ('bert_finetune': conda)", 250 | "language": "python", 251 | "name": "python3" 252 | }, 253 | "language_info": { 254 | "codemirror_mode": { 255 | "name": "ipython", 256 | "version": 3 257 | }, 258 | "file_extension": ".py", 259 | "mimetype": "text/x-python", 260 | "name": "python", 261 | "nbconvert_exporter": "python", 262 | "pygments_lexer": "ipython3", 263 | "version": "3.10.4" 264 | }, 265 | "vscode": { 266 | "interpreter": { 267 | "hash": "323c618d0395b34183a36199d7c8eddbd4e55d51aee9dabbfbc9809db817fb2b" 268 | } 269 | } 270 | }, 271 | "nbformat": 4, 272 | "nbformat_minor": 4 273 | } 274 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Shiu Lab 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /ML_workshop-guide.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/ML_workshop-guide.pptx -------------------------------------------------------------------------------- /ML_workshop-part_b-in_class.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "---\n", 8 | "# __Machine Learning Workshop Part-B__" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "metadata": { 14 | "tags": [] 15 | }, 16 | "source": [ 17 | "## ___Learning objectives___\n", 18 | "\n", 19 | "At the end of the exercise, you should be able to:\n", 20 | "- Conduct an end-to-end classification analysis given data\n", 21 | "- Interpret model with example global and local interpretaion methods." 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "# First thing first: test your install\n", 31 | "import sklearn, pandas, matplotlib, seaborn, imblearn, numpy, shap, tqdm\n", 32 | "\n", 33 | "# If you encounter error, talk to the instructors and/or your neighbor ASAP" 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "## ___Outline___\n", 41 | "\n", 42 | "- Session 1\n", 43 | " - Intro to machine learning\n", 44 | " - [Part A review](#review)\n", 45 | " - [Step 1. Define ML problem](#step1)\n", 46 | " - [Step 2. Exploratory data analysis](#step2)\n", 47 | " - [Step 3: Split train/test](#step3)\n", 48 | "- Session 2\n", 49 | " - [Step 4: Feature engineering](#step4)\n", 50 | " - [Step 5: Select model](#step5)\n", 51 | " - [Step 6: Repeat 2-5](#step6)\n", 52 | "- Session 3\n", 53 | " - [Step 7: Evalute model](#step7)\n", 54 | " - [Step 8: Interpret model](#step8)\n", 55 | " - Workshop wrap-up" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": { 61 | "tags": [] 62 | }, 63 | "source": [ 64 | "----\n", 65 | "\n", 66 | "\n", 67 | "## ___Part A review___\n", 68 | "\n", 69 | "✅ **QUESTION:** In __Part A__, we have gone thorugh the major steps. __In the next 5 minutes__, discuss with your neighbors: What is your understanding of each step? Is there any step or term that is confusng to you? " 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "metadata": {}, 82 | "source": [ 83 | "----\n", 84 | "\n", 85 | "## ___Step 1. Define ML problem___\n", 86 | "\n", 87 | "___Problem statement___: \n", 88 | "- Given:\n", 89 | " - Known genes in general metabolism (GM) or specialized metabolism (SM)\n", 90 | " - Various features of genes\n", 91 | " - E.g., expression levels, functional category they belong to \n", 92 | "- How can we use them to:\n", 93 | " - Distinguish genes involved in GM from those involved in SM?\n" 94 | ] 95 | }, 96 | { 97 | "cell_type": "markdown", 98 | "metadata": {}, 99 | "source": [ 100 | "✅ **QUESTION:** __In the next two minutes__, discuss with your neighbors: given a list of features in [this table](feature_list.csv), what features should we use to best distinguish GM and SM genes?" 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 108 | ] 109 | }, 110 | { 111 | "cell_type": "markdown", 112 | "metadata": {}, 113 | "source": [ 114 | "----\n", 115 | "\n", 116 | "## __Step 2. Exploratory data analysis (EDA)__" 117 | ] 118 | }, 119 | { 120 | "cell_type": "markdown", 121 | "metadata": {}, 122 | "source": [ 123 | "✅ **DO THIS:** Run the following cell to load the data. Note that we set a random seed (`rand_seed`). This is so we can reprdouce the random data generated along the way so others can repeat the same analysis and get the similar, if not the same results." 124 | ] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "execution_count": null, 129 | "metadata": {}, 130 | "outputs": [], 131 | "source": [ 132 | "import pandas as pd\n", 133 | "\n", 134 | "rand_seed = 20240507\n", 135 | "\n", 136 | "# Load the dataset from the file enzyme_gene.csv as a Pandas DataFrame where the\n", 137 | "# `Gene` column is read in as index.\n", 138 | "enzyme_gene = pd.read_csv(\"enzyme_gene.csv\", index_col=0)\n", 139 | "\n", 140 | "# get all feature names\n", 141 | "features = enzyme_gene.columns[1:] \n", 142 | "\n", 143 | "# get all feature names\n", 144 | "# Print out 4 randomly sampled instances\n", 145 | "enzyme_gene.sample(4)" 146 | ] 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "metadata": {}, 151 | "source": [ 152 | "✅ **QUESTION:** __In the next minute__, discuss with your neighbors: which features do you think will be the best for distinguishing GM and SM genes? For what the feature names mean, see [this spreadsheet](https://docs.google.com/spreadsheets/d/1VjnlJgKsGC8GNNds7VtbYfjAQpjETkZNI6AlF9c2DAY/edit?usp=sharing). Also, put your name down for the feature you think will be the best!" 153 | ] 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "metadata": {}, 158 | "source": [ 159 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "### ___2.1 Univariate EDA___" 167 | ] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "metadata": {}, 172 | "source": [ 173 | "✅ **DO THIS:** Earlier, we use `enzyme_gene.sample(4)` to get a few samples from the `enzyme_gene` dataframe. Replace `sample(4)` with `describe()` and see what information become available." 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "execution_count": null, 179 | "metadata": {}, 180 | "outputs": [], 181 | "source": [ 182 | "# Put your code here\n", 183 | "# If you are not sure how to proceed, look at the answer in the following cell.\n" 184 | ] 185 | }, 186 | { 187 | "cell_type": "code", 188 | "execution_count": null, 189 | "metadata": {}, 190 | "outputs": [], 191 | "source": [ 192 | "##ANSWER##\n", 193 | "# Generate summary statics for each features\n", 194 | "enzyme_gene.describe()\n", 195 | "##ANSWER##" 196 | ] 197 | }, 198 | { 199 | "cell_type": "markdown", 200 | "metadata": {}, 201 | "source": [ 202 | "✅ **DO THIS:** Tell us more about `enzyme_gene` by using `shape` and `nunique()`:" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": null, 208 | "metadata": {}, 209 | "outputs": [], 210 | "source": [ 211 | "# put your codes here\n", 212 | "\n", 213 | "# Display the # of columns and rows\n", 214 | "print(\"\\nShape:\", enzyme_gene.shape)\n", 215 | "\n", 216 | "# Display the # of unique values in each feature\n", 217 | "print(\"\\n### Number unique:\\n\", enzyme_gene.nunique())" 218 | ] 219 | }, 220 | { 221 | "cell_type": "markdown", 222 | "metadata": {}, 223 | "source": [ 224 | "✅ **DO THIS:** Run the following to determine how many entires have `GM`, `SM`, and `Unknown` labels, respectively." 225 | ] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "execution_count": null, 230 | "metadata": {}, 231 | "outputs": [], 232 | "source": [ 233 | "print(enzyme_gene[\"Label\"].value_counts())" 234 | ] 235 | }, 236 | { 237 | "cell_type": "markdown", 238 | "metadata": {}, 239 | "source": [ 240 | "✅ **DO THIS:** Determine the number of `null` entries in each column and print them out." 241 | ] 242 | }, 243 | { 244 | "cell_type": "code", 245 | "execution_count": null, 246 | "metadata": {}, 247 | "outputs": [], 248 | "source": [ 249 | "enzyme_gene.isnull().sum()" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": {}, 255 | "source": [ 256 | "### ___2.2 Univariate graphical EDA___" 257 | ] 258 | }, 259 | { 260 | "cell_type": "markdown", 261 | "metadata": {}, 262 | "source": [ 263 | "✅ **DO THIS:** Plot the histograms of all features." 264 | ] 265 | }, 266 | { 267 | "cell_type": "code", 268 | "execution_count": null, 269 | "metadata": {}, 270 | "outputs": [], 271 | "source": [ 272 | "# The 1st \":\" is to get all rows. The \"1:\" part is to get the 2nd column and on.\n", 273 | "feature_values = enzyme_gene.iloc[:, 1:]\n", 274 | "\n", 275 | "# Draw histogram\n", 276 | "hist = feature_values.hist(figsize=(12,12), bins=50)\n" 277 | ] 278 | }, 279 | { 280 | "cell_type": "markdown", 281 | "metadata": {}, 282 | "source": [ 283 | "✅ **QUESTION:** __In the next 1 minute__, discuss with your neighbors: did you see any issues with the dataset based on the above analyses?" 284 | ] 285 | }, 286 | { 287 | "cell_type": "markdown", 288 | "metadata": {}, 289 | "source": [ 290 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 291 | ] 292 | }, 293 | { 294 | "cell_type": "markdown", 295 | "metadata": {}, 296 | "source": [ 297 | "### ___2.3 Multi-variate non-graphical EDA___" 298 | ] 299 | }, 300 | { 301 | "cell_type": "markdown", 302 | "metadata": {}, 303 | "source": [ 304 | "✅ **DO THIS:** Determine pairwise Spearman's rank correlations of all features:" 305 | ] 306 | }, 307 | { 308 | "cell_type": "code", 309 | "execution_count": null, 310 | "metadata": {}, 311 | "outputs": [], 312 | "source": [ 313 | "# Calculate Spearman's rank correlations for all feature pairs\n", 314 | "corr = feature_values.corr(method = 'spearman')\n", 315 | "corr" 316 | ] 317 | }, 318 | { 319 | "cell_type": "markdown", 320 | "metadata": {}, 321 | "source": [ 322 | "### ___2.4 Multi-variate graphical EDA___" 323 | ] 324 | }, 325 | { 326 | "cell_type": "markdown", 327 | "metadata": {}, 328 | "source": [ 329 | "✅ **DO THIS:** Plot the pairwise Spearman's rank correlations of all features as a heatmap using Seaborn." 330 | ] 331 | }, 332 | { 333 | "cell_type": "code", 334 | "execution_count": null, 335 | "metadata": {}, 336 | "outputs": [], 337 | "source": [ 338 | "import seaborn as sns\n", 339 | "import matplotlib.pyplot as plt\n", 340 | "\n", 341 | "plt.figure(figsize=(7,6))\n", 342 | "sns.heatmap(corr, cmap=\"coolwarm\")" 343 | ] 344 | }, 345 | { 346 | "cell_type": "markdown", 347 | "metadata": {}, 348 | "source": [ 349 | "✅ **QUESTION:** __In the next 1 minute__, discuss with your neighbors: did you see any issues with the dataset based on the univariate analyses?" 350 | ] 351 | }, 352 | { 353 | "cell_type": "markdown", 354 | "metadata": {}, 355 | "source": [ 356 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 357 | ] 358 | }, 359 | { 360 | "cell_type": "markdown", 361 | "metadata": {}, 362 | "source": [ 363 | "---\n", 364 | "\n", 365 | "## __Step 3: Split train/test__\n", 366 | "\n", 367 | "The best practice is alway to set aside testing data __as the FIRST STEP__ or as early as possible. This way, the testing data is truly independent from any of the modeling process aside from the processing steps needed. Nonetheless, there may be things you need to take care of first before splitting the data. In this example, we need to rid of unwanted instances before data split." 368 | ] 369 | }, 370 | { 371 | "cell_type": "markdown", 372 | "metadata": {}, 373 | "source": [ 374 | "\n", 375 | "### ___3.1 Deal with unwanted instances___\n", 376 | "\n", 377 | "✅ **DO THIS:** Filter data so only instances with `SM` and `GM` labels are kept." 378 | ] 379 | }, 380 | { 381 | "cell_type": "code", 382 | "execution_count": null, 383 | "metadata": {}, 384 | "outputs": [], 385 | "source": [ 386 | "labels = ['GM', 'SM']\n", 387 | "label_column = enzyme_gene['Label']\n", 388 | "label_column_filter = label_column.isin(labels)\n", 389 | "\n", 390 | "# enzyme_gene dataframe with only GM and SM\n", 391 | "enzyme_gene_fil = enzyme_gene[label_column_filter]\n", 392 | "\n", 393 | "# Count the occurence of unique values\n", 394 | "enzyme_gene_fil['Label'].value_counts()" 395 | ] 396 | }, 397 | { 398 | "cell_type": "markdown", 399 | "metadata": {}, 400 | "source": [ 401 | "✅ **DO THIS:** For classification tasks, class values are typically integers instead of texts (like SM or GM here). So, we will convert `GM` and `SM` to 0 and 1, respectively. \n", 402 | "\n", 403 | "Write code at the end to show that the filtering is working." 404 | ] 405 | }, 406 | { 407 | "cell_type": "code", 408 | "execution_count": null, 409 | "metadata": {}, 410 | "outputs": [], 411 | "source": [ 412 | "# import the proprecessing functions\n", 413 | "from sklearn import preprocessing\n", 414 | "\n", 415 | "# Create a LabelEncoder object: this is simply a software tool that turn \n", 416 | "# (encode) texts into 0 or 1 (labels) in this case.\n", 417 | "le = preprocessing.LabelEncoder()\n", 418 | "\n", 419 | "# Send the Label column of enzyme_gene_fil dataframe to the LabelEncoder so\n", 420 | "# it can fit (i.e., learn) how to encode the labels.\n", 421 | "le.fit(enzyme_gene_fil.Label)\n", 422 | "\n", 423 | "# Now, used the fitted (learned) encoder to transform texts to labels\n", 424 | "enzyme_gene_fil['Label'] = le.transform(enzyme_gene_fil.Label)\n", 425 | "\n", 426 | "# Write code below to show that the Label encoding is working: 0s and 1s\n", 427 | "\n", 428 | "\n", 429 | "# If you are not sure how to proceed, look at the answer in the following cell." 430 | ] 431 | }, 432 | { 433 | "cell_type": "code", 434 | "execution_count": null, 435 | "metadata": {}, 436 | "outputs": [], 437 | "source": [ 438 | "##ANSWER##\n", 439 | "enzyme_gene_fil['Label'].value_counts()\n", 440 | "##ANSWER##" 441 | ] 442 | }, 443 | { 444 | "cell_type": "markdown", 445 | "metadata": {}, 446 | "source": [ 447 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 448 | ] 449 | }, 450 | { 451 | "cell_type": "markdown", 452 | "metadata": {}, 453 | "source": [ 454 | "### ___3.2 Split training/testing sets___\n", 455 | "\n", 456 | "✅ **DO THIS:** Let's split the training and testing data. \n", 457 | "\n", 458 | "Please comments on the lines as indicated." 459 | ] 460 | }, 461 | { 462 | "cell_type": "code", 463 | "execution_count": null, 464 | "metadata": {}, 465 | "outputs": [], 466 | "source": [ 467 | "from sklearn.model_selection import train_test_split\n", 468 | "\n", 469 | "# Split dataset into training and testing sets\n", 470 | "train, test = train_test_split(\n", 471 | " enzyme_gene_fil, # The data to split\n", 472 | " test_size=0.2, # Proportion data for testing\n", 473 | " stratify=enzyme_gene_fil.Label, # Make sure proportions of 0/1\n", 474 | " # labels are similar between\n", 475 | " # training and testing sets\n", 476 | " random_state=rand_seed)\n", 477 | "\n", 478 | "# Print out proportions of different labels in the training data\n", 479 | "print(train['Label'].value_counts()/train.shape[0])\n", 480 | "\n", 481 | "# Print out proportions of different labels in the testing data\n", 482 | "print(test['Label'].value_counts()/test.shape[0])" 483 | ] 484 | }, 485 | { 486 | "cell_type": "markdown", 487 | "metadata": {}, 488 | "source": [ 489 | "✅ **QUESTION:** In __the next 2 minutes__, discuss with your neighbor: what's the point of splitting training and testing data again? How big should the testing data be?" 490 | ] 491 | }, 492 | { 493 | "cell_type": "markdown", 494 | "metadata": {}, 495 | "source": [ 496 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 497 | ] 498 | }, 499 | { 500 | "cell_type": "markdown", 501 | "metadata": {}, 502 | "source": [ 503 | "---\n", 504 | "\n", 505 | "## __Step 4: Feature engineering__\n", 506 | "\n", 507 | "Feature engineering involves processing, transforming, selecting, combining features in ways that will improve the model. " 508 | ] 509 | }, 510 | { 511 | "cell_type": "markdown", 512 | "metadata": {}, 513 | "source": [ 514 | "### ___4.1 Deal with missing data___\n", 515 | "\n", 516 | "We will just try to deal with this in one way. __In reality__, You need to try multiple approaches to see how you can get the best results." 517 | ] 518 | }, 519 | { 520 | "cell_type": "markdown", 521 | "metadata": {}, 522 | "source": [ 523 | "✅ **DO THIS:** First let's remind ourself how many instances are there after we get rid of `unknown`." 524 | ] 525 | }, 526 | { 527 | "cell_type": "code", 528 | "execution_count": null, 529 | "metadata": {}, 530 | "outputs": [], 531 | "source": [ 532 | "enzyme_gene_fil.shape" 533 | ] 534 | }, 535 | { 536 | "cell_type": "markdown", 537 | "metadata": {}, 538 | "source": [ 539 | "✅ **DO THIS:** Let's drop any rows with >25% missing values and see how many instances are still there." 540 | ] 541 | }, 542 | { 543 | "cell_type": "code", 544 | "execution_count": null, 545 | "metadata": {}, 546 | "outputs": [], 547 | "source": [ 548 | "# ask which values are null.\n", 549 | "row_na = enzyme_gene_fil.isnull()\n", 550 | "row_na" 551 | ] 552 | }, 553 | { 554 | "cell_type": "code", 555 | "execution_count": null, 556 | "metadata": {}, 557 | "outputs": [], 558 | "source": [ 559 | "# count the mising values (null, NA, or called NaN: Not a Number) of each crow\n", 560 | "row_na_num = train.isnull().sum(axis=1)\n", 561 | "row_na_num[:5]" 562 | ] 563 | }, 564 | { 565 | "cell_type": "code", 566 | "execution_count": null, 567 | "metadata": {}, 568 | "outputs": [], 569 | "source": [ 570 | "num_feat = train.shape[1] - 1 # number of features in the data\n", 571 | "rows_to_keep = row_na_num/num_feat < 0.25 # rows with <25% missing values\n", 572 | "train_keep = train[rows_to_keep] # training data with rows to keep \n", 573 | "train_keep['Label'].value_counts()" 574 | ] 575 | }, 576 | { 577 | "cell_type": "markdown", 578 | "metadata": {}, 579 | "source": [ 580 | "✅ **DO THIS:** A lot of data is removed but this is much better than just drop any row with missing values. Next, let's try to impute the missing values with `KNNImputer`." 581 | ] 582 | }, 583 | { 584 | "cell_type": "code", 585 | "execution_count": null, 586 | "metadata": {}, 587 | "outputs": [], 588 | "source": [ 589 | "from sklearn.impute import KNNImputer\n", 590 | "\n", 591 | "# Create an imputer object to imptue our data\n", 592 | "# n_neighbors is the number of neighbors used to estimate the missing values.\n", 593 | "imputer = KNNImputer(n_neighbors=5)\n", 594 | "\n", 595 | "# Train the imputer with training data\n", 596 | "imputer.fit(train_keep)\n", 597 | "\n", 598 | "# Transform missing values into imputed values, hence train_keep_imp (imputed)\n", 599 | "train_keep_imp = imputer.transform(train_keep)\n", 600 | "\n", 601 | "# The thing with KNNImputer is it create train_keep_imp as a Numpy array so\n", 602 | "# we don't have the column names any more. Because I really want to know what \n", 603 | "# these columns are, so let's turn this back to a DataFrame with column names.\n", 604 | "train_keep_imp = pd.DataFrame(train_keep_imp, columns=train.columns)\n", 605 | "train_keep_imp.sample(4)" 606 | ] 607 | }, 608 | { 609 | "cell_type": "markdown", 610 | "metadata": {}, 611 | "source": [ 612 | "✅ **CAUTION:** There is a hyperparamter here: `n_neighbors`. We set it to 5 here but __in reality__, multiple values need to be evaluated. Also, you DO NOT impute labels. We did not exclude the label column because we have make sure there is no missing value early on. So imputation will not impact it." 613 | ] 614 | }, 615 | { 616 | "cell_type": "markdown", 617 | "metadata": {}, 618 | "source": [ 619 | "✅ **DO THIS:** Follow what we have done for the training data. Write code that will impute the testing set:\n", 620 | "1. Drop any rows with > 25% missing values.\n", 621 | "2. Impute missing value with KNNImputer.\n", 622 | "3. Check that there is no missing value." 623 | ] 624 | }, 625 | { 626 | "cell_type": "code", 627 | "execution_count": null, 628 | "metadata": {}, 629 | "outputs": [], 630 | "source": [ 631 | "# put your code here\n", 632 | "\n", 633 | "\n", 634 | "# I encourage you to try to figure this out. If you get stuck. Look at the \n", 635 | "# answer below and comments on what each line does." 636 | ] 637 | }, 638 | { 639 | "cell_type": "code", 640 | "execution_count": null, 641 | "metadata": {}, 642 | "outputs": [], 643 | "source": [ 644 | "##ANSWER##\n", 645 | "# Sum the number of NAs in each row of the testing set\n", 646 | "row_na_num = test.isnull().sum(axis=1)\n", 647 | "\n", 648 | "# Get the number of features\n", 649 | "num_feat = test.shape[1] - 1\n", 650 | "\n", 651 | "# Determine which rows have below threshold (25%) NAs\n", 652 | "row_na_below_threshold = row_na_num/num_feat < 0.25\n", 653 | "\n", 654 | "# Get the rows we want to keep\n", 655 | "test_keep = test[row_na_below_threshold]\n", 656 | "\n", 657 | "# Impute missing values\n", 658 | "test_keep_imp = imputer.transform(test_keep)\n", 659 | "test_keep_imp = pd.DataFrame(test_keep_imp, columns=test.columns)\n", 660 | "test_keep_imp.isnull().sum()\n", 661 | "##ANSWER##" 662 | ] 663 | }, 664 | { 665 | "cell_type": "markdown", 666 | "metadata": {}, 667 | "source": [ 668 | "✅ **QUESTION:** In __the next 2 minutes__, discuss with your neighbor: note that here `testing` data is not used to `fit` the imputer. Instead, the imputer fitted with training data is used to `tranform` test set. Why is that?" 669 | ] 670 | }, 671 | { 672 | "cell_type": "markdown", 673 | "metadata": {}, 674 | "source": [ 675 | "### ⛺ **PAUSE: when you finish, please turn your attention to the instructor. **" 676 | ] 677 | }, 678 | { 679 | "cell_type": "markdown", 680 | "metadata": {}, 681 | "source": [ 682 | "### ___4.2 Deal with data imbalance___" 683 | ] 684 | }, 685 | { 686 | "cell_type": "markdown", 687 | "metadata": {}, 688 | "source": [ 689 | "✅ **DO THIS:** Here we will use a hybrid approach:\n", 690 | "- Up-sample the minority class so it has twice as many instances __AND__ \n", 691 | "- Downsample the majority class so it is the same number as the minority." 692 | ] 693 | }, 694 | { 695 | "cell_type": "code", 696 | "execution_count": null, 697 | "metadata": {}, 698 | "outputs": [], 699 | "source": [ 700 | "train_keep_imp.head()" 701 | ] 702 | }, 703 | { 704 | "cell_type": "code", 705 | "execution_count": null, 706 | "metadata": {}, 707 | "outputs": [], 708 | "source": [ 709 | "from collections import Counter\n", 710 | "from imblearn.over_sampling import SMOTE\n", 711 | "from imblearn.under_sampling import RandomUnderSampler\n", 712 | "from imblearn.pipeline import Pipeline\n", 713 | "\n", 714 | "# training features\n", 715 | "X_train = train_keep_imp.iloc[:,1:] \n", 716 | "\n", 717 | "# training labels\n", 718 | "y_train = train_keep_imp.iloc[:,0] \n", 719 | "\n", 720 | "# This will be used in many other occasions.\n", 721 | "feat_names = X_train.columns \n", 722 | "\n", 723 | "# summarize class distribution\n", 724 | "counter = Counter(y_train)\n", 725 | "print(\"Before:\", counter)\n", 726 | "\n", 727 | "# Over-sample minority, under-sample majority\n", 728 | "over = SMOTE(sampling_strategy=0.4)\n", 729 | "under = RandomUnderSampler(sampling_strategy=1)\n", 730 | "steps = [('o', over), ('u', under)]\n", 731 | "pipeline = Pipeline(steps=steps)\n", 732 | "\n", 733 | "# transform the dataset\n", 734 | "X_train_bal, y_train = pipeline.fit_resample(X_train, y_train)\n", 735 | "\n", 736 | "# summarize the new class distribution\n", 737 | "counter = Counter(y_train)\n", 738 | "print(\"After :\", counter)" 739 | ] 740 | }, 741 | { 742 | "cell_type": "markdown", 743 | "metadata": {}, 744 | "source": [ 745 | "✅ **QUESTION:** In __the next 2 minutes__, discuss with your neighbor: Resampling, particularly upsampling can __only__ be applied to the training data. The testing set __should not__ be changed in this step. Why is that?" 746 | ] 747 | }, 748 | { 749 | "cell_type": "markdown", 750 | "metadata": {}, 751 | "source": [ 752 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 753 | ] 754 | }, 755 | { 756 | "cell_type": "markdown", 757 | "metadata": {}, 758 | "source": [ 759 | "### ___4.3. Deal with data scaling___\n", 760 | "\n", 761 | "✅ **DO THIS:** Bassed on your exploratory data analsysi you probably the data range differ widely. Before we work on scaling the data, let's see how the data ranges differ:" 762 | ] 763 | }, 764 | { 765 | "cell_type": "code", 766 | "execution_count": null, 767 | "metadata": {}, 768 | "outputs": [], 769 | "source": [ 770 | "X_train_bal.describe()" 771 | ] 772 | }, 773 | { 774 | "cell_type": "markdown", 775 | "metadata": {}, 776 | "source": [ 777 | "✅ **DO THIS:** In the cell below, let's use `RobustScaler` to scale the balanced training data feature values (`X_train_bal`).\n", 778 | "\n", 779 | "__DO NOT__ applying scaling to the labels (`y`).\n", 780 | "\n", 781 | "Call the scaled features as `X_train_scale`." 782 | ] 783 | }, 784 | { 785 | "cell_type": "code", 786 | "execution_count": null, 787 | "metadata": {}, 788 | "outputs": [], 789 | "source": [ 790 | "from sklearn.preprocessing import RobustScaler\n", 791 | "\n", 792 | "# initialize a scaler\n", 793 | "scaler = RobustScaler()\n", 794 | "\n", 795 | "# fit the scaler with training features\n", 796 | "scaler.fit(X_train_bal)\n", 797 | "\n", 798 | "# transform the training feature values with the fitted scaler\n", 799 | "X_train_scale = scaler.transform(X_train_bal)\n", 800 | "X_train_scale = pd.DataFrame(X_train_scale, columns=X_train.columns)\n", 801 | "X_train_scale.describe()" 802 | ] 803 | }, 804 | { 805 | "cell_type": "markdown", 806 | "metadata": {}, 807 | "source": [ 808 | "✅ **DO THIS:** Provide code below to process testing data so it is scaled the same way.\n", 809 | "- Create `X_test` and `y_test` with `test_keep_imp`.\n", 810 | "- Transform (but __do not__ fit) `X_test` with the `RobustScaler`." 811 | ] 812 | }, 813 | { 814 | "cell_type": "code", 815 | "execution_count": null, 816 | "metadata": {}, 817 | "outputs": [], 818 | "source": [ 819 | "# put your code here\n", 820 | "\n", 821 | "\n", 822 | "# If you don't feel comfortable doing this, check out the answer below and\n", 823 | "# comment on what they are doing." 824 | ] 825 | }, 826 | { 827 | "cell_type": "code", 828 | "execution_count": null, 829 | "metadata": {}, 830 | "outputs": [], 831 | "source": [ 832 | "# put your code here\n", 833 | "\n", 834 | "##ANSWER##\n", 835 | "X_test = test_keep_imp.iloc[:,1:]\n", 836 | "y_test = test_keep_imp.iloc[:,0]\n", 837 | "\n", 838 | "X_test_scale = scaler.transform(X_test)\n", 839 | "X_test_scale = pd.DataFrame(X_test_scale, columns=X_test.columns)\n", 840 | "##ANSWER##" 841 | ] 842 | }, 843 | { 844 | "cell_type": "markdown", 845 | "metadata": {}, 846 | "source": [ 847 | "✅ **CAUTION:** We __did not__ deal with non-normal distributions or co-linear features. In practice, they need to be dealth with. To make the exercise simpler, we did not include categorical variable. For starter, please checkout, after class:\n", 848 | "- [This post on data transformation](https://www.analyticsvidhya.com/blog/2020/07/types-of-feature-transformation-and-scaling/).\n", 849 | "- [This post on colinearity](https://towardsdatascience.com/multicollinearity-in-data-science-c5f6c0fe6edf).\n", 850 | "- [This post on how to work with categorical variables](https://machinelearningmastery.com/one-hot-encoding-for-categorical-data/).\n", 851 | "- More generally, [here is a good article on feature engineering](https://machinelearningmastery.com/discover-feature-engineering-how-to-engineer-features-and-how-to-get-good-at-it/)." 852 | ] 853 | }, 854 | { 855 | "cell_type": "markdown", 856 | "metadata": {}, 857 | "source": [ 858 | "### ⛺ **PAUSE: please turn your attention to the instructor. **" 859 | ] 860 | }, 861 | { 862 | "cell_type": "markdown", 863 | "metadata": {}, 864 | "source": [ 865 | "---\n", 866 | "\n", 867 | "## __Step 5: Select model__" 868 | ] 869 | }, 870 | { 871 | "cell_type": "markdown", 872 | "metadata": {}, 873 | "source": [ 874 | "### ___5.1 Random forest___\n", 875 | "\n", 876 | "✅ **DO THIS:** Here we will not go into details on how the RandomForest algorithm works. there is a substantial number of good tutorial/blog posts on RandomForest (e.g., [this one](https://machinelearningmastery.com/bagging-and-random-forest-ensemble-algorithms-for-machine-learning/)) and I encourage you to look into it. Comments on the major steps as indicated." 877 | ] 878 | }, 879 | { 880 | "cell_type": "code", 881 | "execution_count": null, 882 | "metadata": {}, 883 | "outputs": [], 884 | "source": [ 885 | "from sklearn.ensemble import RandomForestClassifier\n", 886 | "from sklearn.model_selection import GridSearchCV\n", 887 | "from sklearn.model_selection import train_test_split\n", 888 | "\n", 889 | "# Create a function for running RandomForest\n", 890 | "def run_randomforest(X_train, y_train):\n", 891 | " # Below is a Python dictionary specify the hyperparameters to be tested\n", 892 | " # 2x3x4x1 = 24\n", 893 | " param_grid = {'n_estimators': [200, 500],\n", 894 | " 'max_features': ['sqrt', 'log2'],\n", 895 | " 'max_depth' : [3,5,7,9],\n", 896 | " 'criterion' :['entropy']}\n", 897 | "\n", 898 | " # Initialize a random forest classifier (rfc) with a random seed\n", 899 | " rfc = RandomForestClassifier(random_state=rand_seed)\n", 900 | "\n", 901 | " # Initialize a grid search object that will search through each of the 24\n", 902 | " # hyperparameter combinations. For each combination, a five fold cross-\n", 903 | " # validation (cv) is done. So totally 24x5 = 120 random forest classifiers \n", 904 | " # will be build.\n", 905 | " rfc_gs = GridSearchCV(\n", 906 | " rfc,\n", 907 | " param_grid,\n", 908 | " cv=5, # cross validation folds\n", 909 | " verbose=2, # \n", 910 | " scoring='roc_auc', # find model with the best ROC-AUC\n", 911 | " n_jobs=8) # number of concurrent jobs, you need to\n", 912 | " # adjust this based on the number of CPU cores\n", 913 | " # available on your machine.\n", 914 | "\n", 915 | " # Pass the training feaure and label data to the grid search object and\n", 916 | " # start fitting (training) models\n", 917 | " rfc_gs.fit(X_train, y_train)\n", 918 | "\n", 919 | " # Return the fitted grid search object\n", 920 | " return rfc_gs" 921 | ] 922 | }, 923 | { 924 | "cell_type": "code", 925 | "execution_count": null, 926 | "metadata": {}, 927 | "outputs": [], 928 | "source": [ 929 | "# Call the run_randomforest function defined above\n", 930 | "rfc_gs = run_randomforest(X_train_scale, y_train)" 931 | ] 932 | }, 933 | { 934 | "cell_type": "markdown", 935 | "metadata": {}, 936 | "source": [ 937 | "✅ **DO THIS:** In the `grid_search` object, there is whole punch of useful information. Run the following cells." 938 | ] 939 | }, 940 | { 941 | "cell_type": "code", 942 | "execution_count": null, 943 | "metadata": {}, 944 | "outputs": [], 945 | "source": [ 946 | "# The best model (also called estimator)\n", 947 | "best_model = rfc_gs.best_estimator_\n", 948 | "\n", 949 | "# Note that the best hyperparameters are also reported\n", 950 | "best_model" 951 | ] 952 | }, 953 | { 954 | "cell_type": "code", 955 | "execution_count": null, 956 | "metadata": {}, 957 | "outputs": [], 958 | "source": [ 959 | "# The best ROC-AUC score averged across CV folds for the best model\n", 960 | "print(rfc_gs.best_score_)" 961 | ] 962 | }, 963 | { 964 | "cell_type": "markdown", 965 | "metadata": {}, 966 | "source": [ 967 | "✅ **DO THIS:** Although we finished the run rather quickly here, a typical model fitting process can take hours or even days! Thus, the models should be saved so you can reused them in the future. Run the following to save the best estimator. " 968 | ] 969 | }, 970 | { 971 | "cell_type": "code", 972 | "execution_count": null, 973 | "metadata": {}, 974 | "outputs": [], 975 | "source": [ 976 | "import pickle\n", 977 | "\n", 978 | "filename = \"model_randomforest_gridsearch.save\"\n", 979 | "\n", 980 | "pickle.dump(rfc_gs.best_estimator_, open(filename, 'wb'))" 981 | ] 982 | }, 983 | { 984 | "cell_type": "markdown", 985 | "metadata": {}, 986 | "source": [ 987 | "✅ **QUESTION:** In __the next 2 min__, discuss with your neighbors: what just happened here? Can you describe what you have accomplished in this step and what are done? Discuss with your neighbors." 988 | ] 989 | }, 990 | { 991 | "cell_type": "markdown", 992 | "metadata": {}, 993 | "source": [ 994 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 995 | ] 996 | }, 997 | { 998 | "cell_type": "markdown", 999 | "metadata": {}, 1000 | "source": [ 1001 | "### ___5.2 Support Vector Classifier (SVC)___\n", 1002 | "\n", 1003 | "✅ **DO THIS:** There are [many other supervised learning algorithms in Scikit-Learn](https://scikit-learn.org/stable/supervised_learning.html). Let's use [Support Vector Machine](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC). \n", 1004 | "\n", 1005 | "Provide comment on the indicated lines." 1006 | ] 1007 | }, 1008 | { 1009 | "cell_type": "code", 1010 | "execution_count": null, 1011 | "metadata": {}, 1012 | "outputs": [], 1013 | "source": [ 1014 | "# Train a SVM classification model\n", 1015 | "from sklearn.model_selection import GridSearchCV\n", 1016 | "from sklearn.svm import SVC\n", 1017 | "\n", 1018 | "# COMMENT: What does this do?\n", 1019 | "#\n", 1020 | "param_grid = {'C': [1, 10, 1e2],\n", 1021 | " 'gamma': [0.0001, 0.001, 0.01, 0.1], \n", 1022 | " 'kernel': ['linear', 'rbf']}\n", 1023 | "\n", 1024 | "# COMMENT: What does this do?\n", 1025 | "# \n", 1026 | "svc = SVC()\n", 1027 | "\n", 1028 | "# COMMENT: What does this do?\n", 1029 | "# \n", 1030 | "svc_gs = GridSearchCV(svc, param_grid, cv=5, verbose=2, scoring='roc_auc',\n", 1031 | " n_jobs=8)\n", 1032 | "\n", 1033 | "# COMMENT: What does this do?\n", 1034 | "# \n", 1035 | "svc_gs.fit(X_train_scale, y_train)\n", 1036 | "\n", 1037 | "# COMMENT: What does this do?\n", 1038 | "# \n", 1039 | "filename = \"model_svc_gridsearch.save\"\n", 1040 | "pickle.dump(svc_gs.best_estimator_, open(filename, 'wb'))\n", 1041 | "\n", 1042 | "# COMMENT: What does these do?\n", 1043 | "# \n", 1044 | "print(svc_gs.best_params_)\n", 1045 | "print(svc_gs.best_score_)\n", 1046 | "\n", 1047 | "##COMMENT ANSWERS##\n", 1048 | "# Set up the hyperparameter combinations: 3x4x2 = 24 runs\n", 1049 | "# Intialize a support vector classifier\n", 1050 | "# Initiate a grid search object with cross validation\n", 1051 | "# Search for the best hyperparameters with training data\n", 1052 | "# Save the best model as a file\n", 1053 | "# Print out the best parameters and best scores" 1054 | ] 1055 | }, 1056 | { 1057 | "cell_type": "markdown", 1058 | "metadata": {}, 1059 | "source": [ 1060 | "✅ **CAUTION:** We just try two algorithms here, you should try a lot more.\n", 1061 | "\n", 1062 | "In addition, beyond the hyperparameters associated with the algorithms, there are other things to tuned here:\n", 1063 | "- Cross validation methods: There are quite a number of approaches. See [this](https://scikit-learn.org/stable/modules/cross_validation.html) for examples.\n", 1064 | "- Searching parameters: Grid search is but one approach. Two other popular methods are [randomized search](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html) and [Bayesian optimization](https://scikit-optimize.github.io/stable/modules/generated/skopt.BayesSearchCV.html). These should also be tested." 1065 | ] 1066 | }, 1067 | { 1068 | "cell_type": "markdown", 1069 | "metadata": {}, 1070 | "source": [ 1071 | "✅ **QUESTION:** In __the next 2 min__, discuss with your neighbors: based on the above two runs, can you decide which algorithm is better? Why and why not?" 1072 | ] 1073 | }, 1074 | { 1075 | "cell_type": "markdown", 1076 | "metadata": {}, 1077 | "source": [ 1078 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 1079 | ] 1080 | }, 1081 | { 1082 | "cell_type": "markdown", 1083 | "metadata": {}, 1084 | "source": [ 1085 | "---\n", 1086 | "\n", 1087 | "## __Step 6. Repeat Step 2-5__\n", 1088 | "\n", 1089 | "In a typical ML project, after you have explored different algorithms to find the best __initial__ model, it is time to go back to tweak everything to see if you can do even better. Things don't just end here!\n", 1090 | "\n", 1091 | "An important step here is __feature selection__, a part of feature engineering, that involves selecting the most important features to rebuild your models." 1092 | ] 1093 | }, 1094 | { 1095 | "cell_type": "markdown", 1096 | "metadata": { 1097 | "slideshow": { 1098 | "slide_type": "slide" 1099 | } 1100 | }, 1101 | "source": [ 1102 | "### ___6.1 Get feature importance using trained model___" 1103 | ] 1104 | }, 1105 | { 1106 | "cell_type": "markdown", 1107 | "metadata": { 1108 | "slideshow": { 1109 | "slide_type": "subslide" 1110 | } 1111 | }, 1112 | "source": [ 1113 | "✅ **DO THIS:** Uses the feature importance scores generated by the Random Forest model to choose the top features.\n" 1114 | ] 1115 | }, 1116 | { 1117 | "cell_type": "code", 1118 | "execution_count": null, 1119 | "metadata": {}, 1120 | "outputs": [], 1121 | "source": [ 1122 | "from sklearn.inspection import permutation_importance\n", 1123 | "\n", 1124 | "# Specify the best model (estimator) from our RandomForest run\n", 1125 | "rfc = rfc_gs.best_estimator_\n", 1126 | "\n", 1127 | "# Calculate permutation importance of each feature\n", 1128 | "result = permutation_importance(\n", 1129 | " rfc, X_train_scale, y_train, n_repeats=10, random_state=42, n_jobs=8)\n", 1130 | "\n", 1131 | "# COMMENTS: What can you find in the result?\n", 1132 | "#\n", 1133 | "#\n", 1134 | "result" 1135 | ] 1136 | }, 1137 | { 1138 | "cell_type": "code", 1139 | "execution_count": null, 1140 | "metadata": {}, 1141 | "outputs": [], 1142 | "source": [ 1143 | "##COMMENT ANSWER##\n", 1144 | "# importance_mean: mean importance value of each feature\n", 1145 | "# importance_std: imporrtance standard deviation of each feature\n", 1146 | "# importance: the importance scores of each feature for each repeat " 1147 | ] 1148 | }, 1149 | { 1150 | "cell_type": "code", 1151 | "execution_count": null, 1152 | "metadata": {}, 1153 | "outputs": [], 1154 | "source": [ 1155 | "# sort the permutation importance based on mean values\n", 1156 | "sorted_idx = result.importances_mean.argsort()[::-1]\n", 1157 | "sorted_idx" 1158 | ] 1159 | }, 1160 | { 1161 | "cell_type": "code", 1162 | "execution_count": null, 1163 | "metadata": {}, 1164 | "outputs": [], 1165 | "source": [ 1166 | "# Get the importance values in order of the sorted_idx\n", 1167 | "importance_values = result.importances[sorted_idx].T" 1168 | ] 1169 | }, 1170 | { 1171 | "cell_type": "code", 1172 | "execution_count": null, 1173 | "metadata": {}, 1174 | "outputs": [], 1175 | "source": [ 1176 | "# Get the feature names based on the sorted index\n", 1177 | "ordered_feature_label = X_train_scale.columns[sorted_idx]\n", 1178 | "ordered_feature_label" 1179 | ] 1180 | }, 1181 | { 1182 | "cell_type": "code", 1183 | "execution_count": null, 1184 | "metadata": {}, 1185 | "outputs": [], 1186 | "source": [ 1187 | "# Plot the permutation importance results\n", 1188 | "fig, ax = plt.subplots(figsize=(6,6))\n", 1189 | "ax.boxplot(importance_values, \n", 1190 | " vert=False, \n", 1191 | " labels=ordered_feature_label)\n", 1192 | "ax.set_title(\"Permutation Importances (training set)\")\n", 1193 | "fig.tight_layout()\n", 1194 | "plt.show()" 1195 | ] 1196 | }, 1197 | { 1198 | "cell_type": "markdown", 1199 | "metadata": {}, 1200 | "source": [ 1201 | "✅ **QUESTION:** In __the next 2 min__, discuss with your neighbors: how would you interpret the figure above? Is this as what you have expected when you hypothesize the most important feature?" 1202 | ] 1203 | }, 1204 | { 1205 | "cell_type": "markdown", 1206 | "metadata": {}, 1207 | "source": [ 1208 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 1209 | ] 1210 | }, 1211 | { 1212 | "cell_type": "markdown", 1213 | "metadata": { 1214 | "slideshow": { 1215 | "slide_type": "slide" 1216 | } 1217 | }, 1218 | "source": [ 1219 | "### ___6.2 Retrain model using top 10 features___" 1220 | ] 1221 | }, 1222 | { 1223 | "cell_type": "markdown", 1224 | "metadata": { 1225 | "slideshow": { 1226 | "slide_type": "subslide" 1227 | } 1228 | }, 1229 | "source": [ 1230 | "✅ **DO THIS:** A simpler model is always better, because it is easier to understand and because it tends not to be overfitted. Let's use the top 10 features to train a new RandomForest model and see how well it does.\n" 1231 | ] 1232 | }, 1233 | { 1234 | "cell_type": "code", 1235 | "execution_count": null, 1236 | "metadata": { 1237 | "slideshow": { 1238 | "slide_type": "fragment" 1239 | } 1240 | }, 1241 | "outputs": [], 1242 | "source": [ 1243 | "# Get top 10 feature names\n", 1244 | "feat_top10 = ordered_feature_label[:10]\n", 1245 | "\n", 1246 | "# Get training data with only the top 10 features\n", 1247 | "X_train_top10 = X_train_scale[feat_top10]\n", 1248 | "X_train_top10.shape" 1249 | ] 1250 | }, 1251 | { 1252 | "cell_type": "code", 1253 | "execution_count": null, 1254 | "metadata": { 1255 | "slideshow": { 1256 | "slide_type": "fragment" 1257 | } 1258 | }, 1259 | "outputs": [], 1260 | "source": [ 1261 | "# Do the same for testing data\n", 1262 | "X_test_top10 = X_test_scale[feat_top10]\n", 1263 | "X_test_top10.shape" 1264 | ] 1265 | }, 1266 | { 1267 | "cell_type": "code", 1268 | "execution_count": null, 1269 | "metadata": { 1270 | "slideshow": { 1271 | "slide_type": "fragment" 1272 | } 1273 | }, 1274 | "outputs": [], 1275 | "source": [ 1276 | "# Train RandomForest model via grid search\n", 1277 | "rfc_gs_top10 = run_randomforest(X_train_top10, y_train)" 1278 | ] 1279 | }, 1280 | { 1281 | "cell_type": "code", 1282 | "execution_count": null, 1283 | "metadata": {}, 1284 | "outputs": [], 1285 | "source": [ 1286 | "# Save model\n", 1287 | "filename = \"model_randomforest_gridsearch_top10feat.save\"\n", 1288 | "\n", 1289 | "pickle.dump(rfc_gs_top10.best_estimator_, open(filename, 'wb'))" 1290 | ] 1291 | }, 1292 | { 1293 | "cell_type": "code", 1294 | "execution_count": null, 1295 | "metadata": {}, 1296 | "outputs": [], 1297 | "source": [ 1298 | "# Get model score\n", 1299 | "rfc_gs_top10.best_score_" 1300 | ] 1301 | }, 1302 | { 1303 | "cell_type": "markdown", 1304 | "metadata": {}, 1305 | "source": [ 1306 | "✅ **QUESTION:** In __the next 2 min__, discuss with your neighbors: compare this result to the model with all features, should we just use the top 10? Why and why not?" 1307 | ] 1308 | }, 1309 | { 1310 | "cell_type": "markdown", 1311 | "metadata": {}, 1312 | "source": [ 1313 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 1314 | ] 1315 | }, 1316 | { 1317 | "cell_type": "markdown", 1318 | "metadata": {}, 1319 | "source": [ 1320 | "---\n", 1321 | "\n", 1322 | "## __Step 7. Evaluate model with the testing set__\n", 1323 | "\n", 1324 | "Assume that we are done with model building and have a final model that we cannot improve further. Then it is time to use the testing set to evaluate the model." 1325 | ] 1326 | }, 1327 | { 1328 | "cell_type": "markdown", 1329 | "metadata": {}, 1330 | "source": [ 1331 | "### ___7.1 Using the optimal model to predict testing set___" 1332 | ] 1333 | }, 1334 | { 1335 | "cell_type": "markdown", 1336 | "metadata": {}, 1337 | "source": [ 1338 | "✅ **DO THIS:** Random Forest is a little better than SVC. And because using 10 features leads to a model that perofrm almost as well as the one with more features, we will pick the \"optimal\" model to be the random forest model using only 10 features." 1339 | ] 1340 | }, 1341 | { 1342 | "cell_type": "code", 1343 | "execution_count": null, 1344 | "metadata": {}, 1345 | "outputs": [], 1346 | "source": [ 1347 | "filename2 = \"model_randomforest_gridsearch_top10feat.save\"\n", 1348 | "\n", 1349 | "# load model from file\n", 1350 | "rfc_loaded_top10 = pickle.load(open(filename2, 'rb')) # model using top 10\n", 1351 | "\n", 1352 | "# predict testing data labels with the model using top 10 features\n", 1353 | "y_test_pred = rfc_loaded_top10.predict(X_test_top10)" 1354 | ] 1355 | }, 1356 | { 1357 | "cell_type": "code", 1358 | "execution_count": null, 1359 | "metadata": {}, 1360 | "outputs": [], 1361 | "source": [ 1362 | "# Take a look at the 1st 40 predictions\n", 1363 | "print(y_test_pred[:40])" 1364 | ] 1365 | }, 1366 | { 1367 | "cell_type": "code", 1368 | "execution_count": null, 1369 | "metadata": {}, 1370 | "outputs": [], 1371 | "source": [ 1372 | "# Also take a look at the 1st 40 TRUE values\n", 1373 | "print(numpy.array(y_test[:40]))" 1374 | ] 1375 | }, 1376 | { 1377 | "cell_type": "markdown", 1378 | "metadata": { 1379 | "tags": [] 1380 | }, 1381 | "source": [ 1382 | "✅ **QUESTION:** Based on the above results? Do you feel that the model is doing well?" 1383 | ] 1384 | }, 1385 | { 1386 | "cell_type": "markdown", 1387 | "metadata": {}, 1388 | "source": [ 1389 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 1390 | ] 1391 | }, 1392 | { 1393 | "cell_type": "markdown", 1394 | "metadata": {}, 1395 | "source": [ 1396 | "### ___7.2 Confusion matrix___" 1397 | ] 1398 | }, 1399 | { 1400 | "cell_type": "markdown", 1401 | "metadata": {}, 1402 | "source": [ 1403 | "✅ **DO THIS:** Let's generate __confusion matrices__ for the predicted results from the model with all features and the one with just the top 10." 1404 | ] 1405 | }, 1406 | { 1407 | "cell_type": "code", 1408 | "execution_count": null, 1409 | "metadata": {}, 1410 | "outputs": [], 1411 | "source": [ 1412 | "from sklearn.metrics import confusion_matrix\n", 1413 | "from sklearn.metrics import ConfusionMatrixDisplay\n", 1414 | "\n", 1415 | "# Get confusion matrrix\n", 1416 | "cm_top10 = confusion_matrix(y_test, y_test_pred)" 1417 | ] 1418 | }, 1419 | { 1420 | "cell_type": "code", 1421 | "execution_count": null, 1422 | "metadata": {}, 1423 | "outputs": [], 1424 | "source": [ 1425 | "# Plot the confusion matrix\n", 1426 | "cm_display = ConfusionMatrixDisplay(cm_top10).plot()" 1427 | ] 1428 | }, 1429 | { 1430 | "cell_type": "markdown", 1431 | "metadata": {}, 1432 | "source": [ 1433 | "The confusion matrix on the left tell us that:\n", 1434 | "- Number of true negative ($tn$) = 292\n", 1435 | "- Number of false positive ($fp$) = 75\n", 1436 | "- Number of false negative ($fn$) = 17\n", 1437 | "- Number of true positive ($tp$) = 46" 1438 | ] 1439 | }, 1440 | { 1441 | "cell_type": "markdown", 1442 | "metadata": { 1443 | "tags": [] 1444 | }, 1445 | "source": [ 1446 | "✅ **QUESTION:** In __the next 1 min__, discuss with your neighbors: based on the above results? Do you feel that the model is doing well? Why and why not?" 1447 | ] 1448 | }, 1449 | { 1450 | "cell_type": "markdown", 1451 | "metadata": {}, 1452 | "source": [ 1453 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 1454 | ] 1455 | }, 1456 | { 1457 | "cell_type": "markdown", 1458 | "metadata": {}, 1459 | "source": [ 1460 | "### ___7.3 Classification report___" 1461 | ] 1462 | }, 1463 | { 1464 | "cell_type": "markdown", 1465 | "metadata": {}, 1466 | "source": [ 1467 | "✅ **CAUTION:** The metrics printed out are defined below. Unfortunately, we __do not have time__ to go through these. But for you to run an ML project, these, including ROC-AUC, are __foundational knowledge and you need to know the advantage and disadvantage of using them__. [Here is an excellent summary](https://machinelearningmastery.com/metrics-evaluate-machine-learning-algorithms-python/) of important metrics for evaluating ML models.\n", 1468 | "\n", 1469 | "|Metric|Formula|\n", 1470 | "|---|---|\n", 1471 | "|Precision|$p = tp/(tp + fp)$|\n", 1472 | "|Recall|$r = tp / (tp + fn)$|\n", 1473 | "|F1 score|$f1 = 2(p\\times r)/(p + r)$|\n", 1474 | "|Accuracy|$(tp+tn)/(tp+fp)$|\n", 1475 | "|Macro averge|$0.5\\times score_{\\text{class0}} + 0.5\\times score_{\\text{class1}}$|\n", 1476 | "|Weighted averge|$P_{\\text{class0}}\\times score_{\\text{class0}} + P_{\\text{class1}}\\times score_{\\text{class1}}$
$P$: proportion of a class.|\n", 1477 | "\n", 1478 | "Support: number of each class (not a performance metric)\n" 1479 | ] 1480 | }, 1481 | { 1482 | "cell_type": "markdown", 1483 | "metadata": {}, 1484 | "source": [ 1485 | "✅ **DO THIS:** Let's also generate a classification report to get a few other performance matrics." 1486 | ] 1487 | }, 1488 | { 1489 | "cell_type": "code", 1490 | "execution_count": null, 1491 | "metadata": {}, 1492 | "outputs": [], 1493 | "source": [ 1494 | "from sklearn.metrics import classification_report\n", 1495 | "\n", 1496 | "# Set class names\n", 1497 | "targets = [\"GM\", \"SM\"]\n", 1498 | "\n", 1499 | "report = classification_report(y_test, y_test_pred, target_names=targets)\n", 1500 | "print(report)" 1501 | ] 1502 | }, 1503 | { 1504 | "cell_type": "markdown", 1505 | "metadata": {}, 1506 | "source": [ 1507 | "### ⛺ **PAUSE: once you finish, please turn your attention to the instructor. **" 1508 | ] 1509 | }, 1510 | { 1511 | "cell_type": "markdown", 1512 | "metadata": {}, 1513 | "source": [ 1514 | "### ___7.4 Graphics that help with evaluation___" 1515 | ] 1516 | }, 1517 | { 1518 | "cell_type": "markdown", 1519 | "metadata": {}, 1520 | "source": [ 1521 | "✅ **DO THIS:** Here we provide two examples: ROC-AUC curve and precision-recall curve. The red dotted line indicate how a naive classifer woul fair with __random guesses__. We only plot these curves for the model with top 10 features." 1522 | ] 1523 | }, 1524 | { 1525 | "cell_type": "code", 1526 | "execution_count": null, 1527 | "metadata": {}, 1528 | "outputs": [], 1529 | "source": [ 1530 | "from sklearn.metrics import RocCurveDisplay\n", 1531 | "from sklearn.metrics import PrecisionRecallDisplay\n", 1532 | "\n", 1533 | "def plot_curves(X, y, estimator, background):\n", 1534 | "\n", 1535 | " fig, axs = plt.subplots(1, 2, figsize=(10,5))\n", 1536 | "\n", 1537 | " # ROC-AUC curve\n", 1538 | " RocCurveDisplay.from_estimator(estimator, X, y, ax=axs[0])\n", 1539 | " # Plot ROC-AUC background\n", 1540 | " axs[0].plot([0, 1], [0, 1],'r--')\n", 1541 | "\n", 1542 | " # Precision-recall curve\n", 1543 | " PrecisionRecallDisplay.from_estimator(estimator, X, y, ax=axs[1])\n", 1544 | " axs[1].legend(loc='upper right')\n", 1545 | " # Plot PR-curve background\n", 1546 | " axs[1].plot([0, 1], [background, background],'r--')\n", 1547 | " axs[1].set_ylim(-0.05,1.05)\n", 1548 | "\n", 1549 | " plt.show()\n", 1550 | "\n", 1551 | "num_class0 = y_test[y_test==0].shape[0]\n", 1552 | "num_class1 = y_test[y_test==1].shape[0]\n", 1553 | "background = num_class1/(num_class0+num_class1)\n", 1554 | "print(background)\n", 1555 | "\n", 1556 | "plot_curves(X_test_top10, y_test, rfc_loaded_top10, background)" 1557 | ] 1558 | }, 1559 | { 1560 | "cell_type": "markdown", 1561 | "metadata": { 1562 | "tags": [] 1563 | }, 1564 | "source": [ 1565 | "✅ **QUESTION:** The red lines are what we expect a random model will be performing. For the plot on the right-hand side, it is defined by the `background` value defined in the cell above. Why would it be cosndiered as `background` (i.e., random guess) for the right-hand plot?" 1566 | ] 1567 | }, 1568 | { 1569 | "cell_type": "markdown", 1570 | "metadata": {}, 1571 | "source": [ 1572 | "✅ **DO THIS:** Let's also get these curves for training data." 1573 | ] 1574 | }, 1575 | { 1576 | "cell_type": "code", 1577 | "execution_count": null, 1578 | "metadata": {}, 1579 | "outputs": [], 1580 | "source": [ 1581 | "num_class0 = y_train[y_train==0].shape[0]\n", 1582 | "num_class1 = y_train[y_train==1].shape[0]\n", 1583 | "background = num_class1/(num_class0+num_class1)\n", 1584 | "print(background)\n", 1585 | "\n", 1586 | "plot_curves(X_train_top10, y_train, rfc_loaded_top10, background)" 1587 | ] 1588 | }, 1589 | { 1590 | "cell_type": "markdown", 1591 | "metadata": { 1592 | "tags": [] 1593 | }, 1594 | "source": [ 1595 | "✅ **QUESTION:** In __the next 2 min__, discuss with your neighbor: why are the model performance for training data so much better than that for the testing set?" 1596 | ] 1597 | }, 1598 | { 1599 | "cell_type": "markdown", 1600 | "metadata": {}, 1601 | "source": [ 1602 | "### ⛺ **PAUSE: after your discussion, please turn your attention to the instructor. **" 1603 | ] 1604 | }, 1605 | { 1606 | "cell_type": "markdown", 1607 | "metadata": {}, 1608 | "source": [ 1609 | "---\n", 1610 | "\n", 1611 | "## __Step 8. Interpret model__" 1612 | ] 1613 | }, 1614 | { 1615 | "cell_type": "markdown", 1616 | "metadata": {}, 1617 | "source": [ 1618 | "### ___8.1 Global interpetation using SHAP___" 1619 | ] 1620 | }, 1621 | { 1622 | "cell_type": "markdown", 1623 | "metadata": {}, 1624 | "source": [ 1625 | "✅ **DO THIS:** Run the code below to get [SHAP (SHapley Additive exPlanations)](https://shap.readthedocs.io/en/latest/index.html) values that use game theory to determine how important each feature is in contributing to a prediction. There are MANY, MANY things you can do with SHAP and we will first figure out which features are more important than the others." 1626 | ] 1627 | }, 1628 | { 1629 | "cell_type": "code", 1630 | "execution_count": null, 1631 | "metadata": {}, 1632 | "outputs": [], 1633 | "source": [ 1634 | "help(rfc_loaded_top10)" 1635 | ] 1636 | }, 1637 | { 1638 | "cell_type": "code", 1639 | "execution_count": null, 1640 | "metadata": {}, 1641 | "outputs": [], 1642 | "source": [ 1643 | "rfc_loaded_top10.feature_importances_ " 1644 | ] 1645 | }, 1646 | { 1647 | "cell_type": "code", 1648 | "execution_count": null, 1649 | "metadata": {}, 1650 | "outputs": [], 1651 | "source": [ 1652 | "X_train_top10.shape" 1653 | ] 1654 | }, 1655 | { 1656 | "cell_type": "code", 1657 | "execution_count": null, 1658 | "metadata": {}, 1659 | "outputs": [], 1660 | "source": [ 1661 | "import shap\n", 1662 | "\n", 1663 | "# Get a TreeExplainer object using the model we have created\n", 1664 | "explainer = shap.TreeExplainer(rfc_loaded_top10)\n", 1665 | "\n", 1666 | "# Explainer object for the training dataset\n", 1667 | "shap_values = explainer.shap_values(X_train_top10)\n", 1668 | "\n", 1669 | "# Shap values for positive class (label=1)\n", 1670 | "shap_for_label1 = shap_values[:,:,1]\n", 1671 | "print(shap_for_label1.shape)\n", 1672 | "\n", 1673 | "# Generate a summary plot\n", 1674 | "shap.summary_plot(shap_for_label1, X_train_top10, sort=True, plot_type=\"bar\")" 1675 | ] 1676 | }, 1677 | { 1678 | "cell_type": "markdown", 1679 | "metadata": {}, 1680 | "source": [ 1681 | "We can see that `Fam_size` is the most important. Because we have only two classes, if a feature is important for classifyting one class, it must also be important for the other class." 1682 | ] 1683 | }, 1684 | { 1685 | "cell_type": "markdown", 1686 | "metadata": {}, 1687 | "source": [ 1688 | "### ___8.2 SHAP values of different instances___" 1689 | ] 1690 | }, 1691 | { 1692 | "cell_type": "markdown", 1693 | "metadata": {}, 1694 | "source": [ 1695 | "✅ **DO THIS:** Another way to look at the SHAP values is by focusing on a particular class. In the example below, we focus on how different features contribute to the predictions of label=1 (SM):" 1696 | ] 1697 | }, 1698 | { 1699 | "cell_type": "code", 1700 | "execution_count": null, 1701 | "metadata": {}, 1702 | "outputs": [], 1703 | "source": [ 1704 | "# plot the shape value and color based on feature values\n", 1705 | "shap.summary_plot(shap_for_label1, X_train_top10, sort=True)" 1706 | ] 1707 | }, 1708 | { 1709 | "cell_type": "markdown", 1710 | "metadata": { 1711 | "tags": [] 1712 | }, 1713 | "source": [ 1714 | "For each feature $x$, two SHAP values are generated for each instance: one for $x$'s contribution to class $0$ and the other for its contribution to class $1$. In the plot above, we are only looking at the contribution to class $1$ and each dot is an instance.\n", 1715 | "\n", 1716 | "Look at Fam_size (family size), instances with higher features values (i.e., in larger families) also tend to have higher positive SHAP values (i.e., positive contribution to be in class 1).\n", 1717 | "\n", 1718 | "✅ **QUESTION:** In __the next 2 min__, discuss with your neighbor and interpret what the `Func_likelihood` feature's SHAP value distribution means where a higher feature values correlate with lower SHAP." 1719 | ] 1720 | }, 1721 | { 1722 | "cell_type": "markdown", 1723 | "metadata": {}, 1724 | "source": [ 1725 | "### ⛺ **PAUSE: after your discussion, please turn your attention to the instructor. **" 1726 | ] 1727 | }, 1728 | { 1729 | "cell_type": "markdown", 1730 | "metadata": {}, 1731 | "source": [ 1732 | "-----" 1733 | ] 1734 | } 1735 | ], 1736 | "metadata": { 1737 | "kernelspec": { 1738 | "display_name": "Python 3.10.4 ('bert_finetune': conda)", 1739 | "language": "python", 1740 | "name": "python3" 1741 | }, 1742 | "language_info": { 1743 | "codemirror_mode": { 1744 | "name": "ipython", 1745 | "version": 3 1746 | }, 1747 | "file_extension": ".py", 1748 | "mimetype": "text/x-python", 1749 | "name": "python", 1750 | "nbconvert_exporter": "python", 1751 | "pygments_lexer": "ipython3", 1752 | "version": "3.12.3" 1753 | }, 1754 | "vscode": { 1755 | "interpreter": { 1756 | "hash": "323c618d0395b34183a36199d7c8eddbd4e55d51aee9dabbfbc9809db817fb2b" 1757 | } 1758 | } 1759 | }, 1760 | "nbformat": 4, 1761 | "nbformat_minor": 4 1762 | } 1763 | -------------------------------------------------------------------------------- /ML_workshop-part_c_xai.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "---\n", 8 | "# __Part-c: eXplainable AI/ML (XAI)__" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "metadata": { 14 | "tags": [] 15 | }, 16 | "source": [ 17 | "# Learning objectives\n", 18 | "\n", 19 | "At the end of the exercise, you should be able to:\n", 20 | "- Explain why it is important to explain your model.\n", 21 | "- Interpret model with example global and local interpretaion methods." 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "# First thing first: test your install\n", 31 | "import sklearn, pandas, matplotlib, seaborn, imblearn, numpy, shap, tqdm\n", 32 | "\n", 33 | "# If you encounter error, talk to the instructors and/or your neighbor ASAP" 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "## Outline\n", 41 | "\n", 42 | "- [5 min] Intro (powerpoint)\n", 43 | "- [[10 min] Background](#background)\n", 44 | "- [[05 min] Step 8: Interpret model](#step7)" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": { 50 | "tags": [] 51 | }, 52 | "source": [ 53 | "----\n", 54 | "\n", 55 | "\n", 56 | "# __Background__" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "\n", 64 | "## _Machine learning workflow_" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "\n", 72 | "|Step|Purpose|Notes|\n", 73 | "|---|---|---|\n", 74 | "|1|State the question and frame it as a machine learning problem|Is it a supervised or unsupervised learning problem? There are other major types (semi-supervised and reinforced learning).|\n", 75 | "|2|Collect data, and exploratory data analysis (EDA)|Apply four types of EDA.|\n", 76 | "|3|Split training and testing data|Seperate out testing data for evaluation purpose.|\n", 77 | "|4|Engineer features| - Format data, impute missing values, normalize/scale features.
- Transform feature values, select informative features,
- Create combinatorial features (e.g., the interaction terms for basic linear model).
- Do dimension reduction to reduce the number of features.|\n", 78 | "|5|Select models| - There are many models/algorithms/classifers making different assumptions about how modeling should be done. Thus, it is important to go through a wide range of them.
- And for each model/algorith, we need to tune __hyperparameters__, i.e., parameters that are set manually.
- Cross-validate model, i.e., separate data into training/validation subsets, use the training subset to train a model and score the model with the validation subset. Then split the training/validation subset again but differently, train and score. Repeat the process $k$ times.|\n", 79 | "|6|Repeat 2-5|Work to iteratively improve the models.|\n", 80 | "|7|Evaluate the best performing model|- After a best performing model is identified based on the training data, the model is applied to the testing set.
- Various model performance metric can be used to access how good the model is.|\n", 81 | "|8|Interpret the model|- Dissect the model to better understand how it works.
- Identify most informative features for the best performing model.
- Assess the reasons behind false predictions to further improve model.|\n", 82 | "|9|Deploy model|Apply model to new data and make predictions.|" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "## _What is interpretability_" 90 | ] 91 | }, 92 | { 93 | "cell_type": "markdown", 94 | "metadata": {}, 95 | "source": [ 96 | "## _Why interpret the model_" 97 | ] 98 | }, 99 | { 100 | "cell_type": "markdown", 101 | "metadata": {}, 102 | "source": [ 103 | "Blah blah" 104 | ] 105 | }, 106 | { 107 | "cell_type": "markdown", 108 | "metadata": {}, 109 | "source": [ 110 | "----\n", 111 | "\n", 112 | "# __Step 1. Define ML problem__\n", 113 | "\n", 114 | "___Problem statement___: \n", 115 | "- Following this published paper:\n", 116 | " - [Robust predictions of specialized metabolism genes through machine learning](https://www.pnas.org/doi/abs/10.1073/pnas.1817074116)\n", 117 | "- Given:\n", 118 | " - Known genes in general metabolism (GM) or specialized metabolism (SM)\n", 119 | " - Various features of genes\n", 120 | " - E.g., expression levels, functional category they belong to \n", 121 | "- How can we use them to:\n", 122 | " - Distinguish genes involved in GM from those involved in SM?\n" 123 | ] 124 | }, 125 | { 126 | "cell_type": "markdown", 127 | "metadata": {}, 128 | "source": [ 129 | "----\n", 130 | "\n", 131 | "## __Step 2-7. From exploratory data analysis to model evaluation__\n", 132 | "\n", 133 | "We will skip all these steps to cut to the chase. Here we will use a model that has already been build for model interpretation. If you'd like to learn more about how the model was built, see [our ML workshop](https://github.com/ShiuLab/ML_workshop) materials." 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "metadata": {}, 139 | "source": [ 140 | "✅ **DO THIS:** Run the following cell to load the model." 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": {}, 147 | "outputs": [], 148 | "source": [] 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "metadata": {}, 153 | "source": [ 154 | "---\n", 155 | "\n", 156 | "## __Step 8. Interpret model__" 157 | ] 158 | }, 159 | { 160 | "cell_type": "markdown", 161 | "metadata": {}, 162 | "source": [ 163 | "### ___8.1 Global interpetation using SHAP___" 164 | ] 165 | }, 166 | { 167 | "cell_type": "markdown", 168 | "metadata": {}, 169 | "source": [ 170 | "✅ **DO THIS:** Run the code below to get [SHAP (SHapley Additive exPlanations)](https://shap.readthedocs.io/en/latest/index.html) values that use game theory to determine how important each feature is in contributing to a prediction. There are MANY, MANY things you can do with SHAP and we will first figure out which features are more important than the others." 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": null, 176 | "metadata": {}, 177 | "outputs": [], 178 | "source": [ 179 | "import shap\n", 180 | "shap.initjs()\n", 181 | "\n", 182 | "# Get a TreeExplainer object using the model we have created\n", 183 | "explainer = shap.TreeExplainer(rfc_loaded_top10)\n", 184 | "\n", 185 | "# Use the TreeExplainer to get SHAP values for the training data\n", 186 | "shap_values = explainer.shap_values(X_train_top10)\n", 187 | "\n", 188 | "# Generate a summary plot\n", 189 | "shap.summary_plot(shap_values, X_train_top10, sort=True)" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "We can see that `Fam_size` is the most important. Because we have only two classes, if a feature is important for classifyting one class, it must also be important for the other class." 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "### ___8.2 SHAP values of different instances___" 204 | ] 205 | }, 206 | { 207 | "cell_type": "markdown", 208 | "metadata": {}, 209 | "source": [ 210 | "✅ **DO THIS:** Another way to look at the SHAP values is by focusing on a particular class. In the example below, we focus on how different features contribute to the predictions of label=1 (SM):" 211 | ] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "execution_count": null, 216 | "metadata": {}, 217 | "outputs": [], 218 | "source": [ 219 | "shap_for_label1 = shap_values[1]\n", 220 | "\n", 221 | "# plot the shape value and color based on feature values\n", 222 | "shap.summary_plot(shap_for_label1, X_train_top10, sort=True)" 223 | ] 224 | }, 225 | { 226 | "cell_type": "markdown", 227 | "metadata": { 228 | "tags": [] 229 | }, 230 | "source": [ 231 | "For each feature $x$, two SHAP values are generated for each instance: one for $x$'s contribution to class $0$ and the other for its contribution to class $1$. In the plot above, we are only looking at the contribution to class $1$ and each dot is an instance.\n", 232 | "\n", 233 | "Look at Fam_size (family size), instances with higher features values (i.e., in larger families) also tend to have higher positive SHAP values (i.e., positive contribution to be in class 1).\n", 234 | "\n", 235 | "✅ **QUESTION:** In __the next 2 min__, discuss with your neighbor and interpret what the `Func_likelihood` feature's SHAP value distribution means where a higher feature values correlate with lower SHAP." 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "### ⛺ **PAUSE: after your discussion, please turn your attention to the instructor. **" 243 | ] 244 | }, 245 | { 246 | "cell_type": "markdown", 247 | "metadata": {}, 248 | "source": [ 249 | "-----\n", 250 | "### Congratulations, we're done!" 251 | ] 252 | } 253 | ], 254 | "metadata": { 255 | "kernelspec": { 256 | "display_name": "Python 3.10.3 ('sklearn': conda)", 257 | "language": "python", 258 | "name": "python3" 259 | }, 260 | "language_info": { 261 | "codemirror_mode": { 262 | "name": "ipython", 263 | "version": 3 264 | }, 265 | "file_extension": ".py", 266 | "mimetype": "text/x-python", 267 | "name": "python", 268 | "nbconvert_exporter": "python", 269 | "pygments_lexer": "ipython3", 270 | "version": "3.10.3" 271 | }, 272 | "vscode": { 273 | "interpreter": { 274 | "hash": "b31353e0b5417ab1909bc62ced18dcd0f4fa5d6ceab54c18c132f16ad2bde54b" 275 | } 276 | } 277 | }, 278 | "nbformat": 4, 279 | "nbformat_minor": 4 280 | } 281 | -------------------------------------------------------------------------------- /ML_workshop-xai.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/ML_workshop-xai.pptx -------------------------------------------------------------------------------- /feature_list.csv: -------------------------------------------------------------------------------- 1 | Feature name,Description 2 | Func_likelihood,The likelihood that the gene in question functional which is derived from a model distinguishing functional vs. pseudogenes 3 | Fam_size,The size of the gene family the gene in question belong to 4 | Max_id_paralog,"The maximum identity of the gene in question to its ""siblings"" (paralogs or duplicates)" 5 | WGD_alpha,If the gene in question still has a sibling (paralog) derived from the alpha whole-genome duplication event 6 | WGD_beta_gamma,If the gene in question still has a sibling (paralog) derived from the beta or gamma whole-genome duplication event 7 | Dup_recent,If the gene in question has a sibling that is closely related (synonymous substitution rate < 0.05) 8 | Dup_tandem,If the gene in question is tandemly duplicated 9 | Singleton,If the gene in question has no sibling 10 | Max_PCC_GM_abiotic,The maximum Pearson's correlation between the abiotic stress transcriptional responses of the gene in question and those of general metabolism genes 11 | Max_PCC_SM_abiotic,The maximum Pearson's correlation between the abiotic stress transcriptional responses of the gene in question and those of specialized metabolism genes 12 | Max_PCC_GM_biotic,The maximum Pearson's correlation between the biotic stress transcriptional responses of the gene in question and those of general metabolism genes 13 | Max_PCC_SM_biotic,The maximum Pearson's correlation between the biotic stress transcriptional responses of the gene in question and those of specialized metabolism genes 14 | Max_PCC_GM_hormone,The maximum Pearson's correlation between the hormone-treatment transcriptional responses of the gene in question and those of general metabolism genes 15 | Max_PCC_SM_hormone,The maximum Pearson's correlation between the hormone-treatment transcriptional responses of the gene in question and those of specialized metabolism genes 16 | Expr_med_dev,The median expression level of the gene in question among expression data of different tissue and/or developmental stages 17 | Expr_max_dev,The maximum expression level of the gene in question among expression data of different tissue and/or developmental stages 18 | Expr_breadth_dev,The breadth of expression level of the gene in question among expression data of different tissue and/or developmental stages 19 | -------------------------------------------------------------------------------- /feature_list.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/feature_list.xlsx -------------------------------------------------------------------------------- /img/img_clone_repository.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/img/img_clone_repository.png -------------------------------------------------------------------------------- /img/img_what_ml_is.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/img/img_what_ml_is.png -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | # Machine learning workshop 2 | 3 | ![alt text](./img/img_what_ml_is.png) 4 | 5 | ## 1. Workshop information 6 | 7 | ### What this is about 8 | 9 | More and more experimental data are available that have fueled ground breaking discoveries. Beyond the original intents of the experiments, these data can be used to discover even more. This is where machine learning (ML) comes in. We can use computers to learn from data and generate models that can predict a biological phenomenon of interest - e.g., will this gene be lethal when it is knocked-out, or which genetic variants can meaningfully predict a phenotype of interests. To learn more about ML, we created this workshop to provide an introduction on the following topics: 10 | 11 | - What is machine learning and why is it useful? 12 | - How does machine learning work? 13 | - What are some example machine learning applications in biology? 14 | - How can we feed data into machine learning tools to make discoveries? 15 | - What are the best practices when doing machine learning? 16 | - Where to go to learn more? 17 | 18 | The workshop will include presentations, discussions, and hands-on sections for those who can complete the pre-workshop components without issue. 19 | 20 | ### Who we are 21 | 22 | * Our lab [website](https://shiulab.github.io/) 23 | 24 | ### What kinds of materials we are sharing 25 | 26 | - Workshop jupyter notebooks 27 | - An example data to run an ML project based on [this paper](https://pubmed.ncbi.nlm.nih.gov/30674669/). 28 | 29 | ## 2. Instructions for runnning the notebook 30 | 31 | ### What's needed 32 | 33 | The workshop example is provided as a [Jupyter notebook](https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/what_is_jupyter.html). It is a document generated by the [Jupyter Lab or Jupyter Notebook applications](](https://jupyter.org/install.html)). A notebook can contain both computer codes in popular languages such as Python and R, and texts in the form of paragraph, equations, figures, links, etc. 34 | 35 | To follow what we have shown in the workshop, you need the following: 36 | * Git: for you to "clone" this repository to your computer to play with. 37 | * Jupyter Lab: the application to view, edit, and execute codes in the notebook. 38 | * Scikit-Learn and others: the software packages Jupyter Lab relies on to run the codes. 39 | 40 | ### Get some backgrounds 41 | 42 | Please make sure you: 43 | 1. Have a look at the [pre-workshop notebook through GitHub](https://github.com/ShiuLab/ML_workshop/blob/master/ML_workshop-part_a-preparation.ipynb) and take some notes on the questions asked. 44 | 2. Watch [this video](https://www.youtube.com/watch?v=cfj6yaYE86U), and [this video](https://www.youtube.com/watch?v=-_XYmr4vkwc) on getting Jupyter notebook to run. 45 | 46 | ### Get the notebook and data 47 | 48 | You can download the notebooks and data from this repository, preferably by setting the following up: 49 | - `git`, a version control software (i.e., a tool to keep track of updates to codes) widely used by folks writing software in any language. 50 | - [Github](https://github.com/) is a code hosting platform that uses `git` for version control and collaboration (i.e., many people can work on the same codes). 51 | 52 | If you don't have git and/or Github account, do the following: 53 | 1. Create a [GitHub Account](https://github.com/join) 54 | 2. Download and install [Github Desktop](https://desktop.github.com/) 55 | * Or you can use [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) if you are familar with version control and command-line interface. Note that the following info is for using Github Desktop. 56 | 3. Clone the ML_workshop by following [this instruction](https://docs.github.com/en/desktop/contributing-and-collaborating-using-github-desktop/cloning-and-forking-repositories-from-github-desktop) and the following screenshot. 57 | * __Note:__ You can specify where the repository goes in your computer. We suggest leaving it as default and to remember where it is - we need it later. 58 | 59 | ![alt text](./img/img_clone_repository.png) 60 | 61 | 4. Navigate to the location where the cloned repository is and confirm that it is there. 62 | 63 | ### Install Anaconda 64 | 65 | [Anaconda](https://www.anaconda.com/) is a a free and open-source distribution of the programming languages Python and R and is a widely used platform for computational and data science applications. 66 | 67 | 1. Download the Python 3.X version of [Anaconda](https://www.anaconda.com/products/individual#Downloads). 68 | 2. Install Anaconda using the [instructions](https://docs.anaconda.com/anaconda/install/). 69 | 3. Open your terminal in [Mac](https://support.apple.com/guide/terminal/open-or-quit-terminal-apd5265185d-f365-44cb-8b09-71a064a42125/mac) or [PC](https://www.wikihow.com/Open-Terminal-in-Windows) 70 | * __Note__: For PC, you need to open the terminal by "Running as Administrator". If you are not familiar with this, see [this post](https://www.itechtics.com/run-programs-administrator/) for more info. 71 | 72 | 5. Issue the following command to make sure Anaconda installation is complete: 73 | ``` 74 | conda list 75 | ``` 76 | The above command allows you to see what software packages have been installed. 77 | 78 | ### Install software packages 79 | 80 | [Conda](https://docs.conda.io/en/latest/) is a package/environment management system. It deals with installing software packages in your computer. It also creates and manage virtual environments where each environment you have a specific set of software for a general category of tasks. 81 | 82 | 1. Create an `ml_workshop` environment and activate it: 83 | ``` 84 | conda create -n ml_workshop python 85 | ``` 86 | ``` 87 | conda activate ml_workshop 88 | ``` 89 | * __Note__: When prompted with `Proceed`, type `y`. 90 | 91 | 2. Install software packages and their dependencies: 92 | ``` 93 | conda install jupyterlab ipykernel ipywidgets matplotlib pandas scikit-learn seaborn shap tqdm 94 | ``` 95 | ``` 96 | pip install imbalanced-learn 97 | ``` 98 | 99 | ### Open the notebooks 100 | 101 | 1. Run Jupyter Lab 102 | 103 | * If you use Linux or Mac OS: 104 | 105 | ``` 106 | jupyter lab 107 | ``` 108 | * If you use PC, and your Github folder is in __C:/__ drive, then do: 109 | ``` 110 | jupyter lab --notebook-dir=C:/ 111 | ``` 112 | * If you use PC and your Github folder is in __D:/__ drive, do: 113 | ``` 114 | jupyter lab --notebook-dir=D:/ 115 | ``` 116 | 117 | 2. In the Jupyter lab window that opens, on the left panel, navigate to __ML_workshop__, the directory where the cloned Github repository is stored. 118 | 119 | 3. Open `ML_workshop-part_a-preparation.ipynb` 120 | 121 | 4. Run each code element by clicking ```SHIFT + ENTER```. 122 | 123 | -------------------------------------------------------------------------------- /workshop_outline.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ShiuLab/ML_workshop/fba900d44807f76d16d86c3c5b9a2172bbafa22b/workshop_outline.xlsx --------------------------------------------------------------------------------