├── Images ├── Classification_Report.JPG ├── prediction.JPG └── top10_tags.JPG ├── README.md ├── Stackoverflow Clean Questions.ipynb └── Stackoverflow Tags Map & Model.ipynb /Images/Classification_Report.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/theRajeshReddy/StackOverFlow-Classification/8ba027db0dfa3d09724a62838bebdd979b031af0/Images/Classification_Report.JPG -------------------------------------------------------------------------------- /Images/prediction.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/theRajeshReddy/StackOverFlow-Classification/8ba027db0dfa3d09724a62838bebdd979b031af0/Images/prediction.JPG -------------------------------------------------------------------------------- /Images/top10_tags.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/theRajeshReddy/StackOverFlow-Classification/8ba027db0dfa3d09724a62838bebdd979b031af0/Images/top10_tags.JPG -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Multiclass Multilabel Prediction For StackOverflow Questions 2 | 3 | **Data set** : https://www.kaggle.com/therajeshreddy/stackoverflow 4 | 5 | **Objective** : Given text for Questions from StackoverFlow posts, predict tags associated with them. 6 | 7 | This is a scaled down version of predecting only top 10 most occurring tags 8 | 9 | **Programming Language** : Python using nltk & Keras 10 | 11 | **Model Architecture** : Deep Learning using Recurrent Neural Network (RNN) 12 | 13 | **About Data Set** 14 | 15 | Dataset has text of questions, answers and thier corresponding tags from the Stack Overflow programming Q&A website. 16 | 17 | This is organized as three files: 18 | 19 | 1. Questions contains the title, body, creation date, closed date (if applicable), score, and owner ID for all non-deleted Stack Overflow questions. 20 | 21 | 2. Tags contains the tags on each of these questions. 22 | 23 | 3. Answers contains the body, creation date, score, and owner ID for each of the answers to these questions. The ParentId column links back to the Questions table. *We don't use this file as we want to predict Tags given a question* 24 | 25 | **Data Pre-Processing** 26 | 27 | >Questions File 28 | *Code* : Stackoverflow Clean Questions.ipynb 29 | 30 | 1. Read Questions File 31 | 2. Drop All columns except Id,Title and Body 32 | 3. Now the text in the Body column seem to have many html tags in the text. We use Regular Expressions and Clean the Body column text by removing the html tags 33 | ```python 34 | import re 35 | def rem_html_tags(body): 36 | regex = re.compile('<.*?>') 37 | return re.sub(regex, '', body) 38 | ques['Body'] = ques['Body'].apply(rem_html_tags) 39 | ``` 40 | 4. Save the questions file for later use 41 | ```python 42 | ques.to_csv('question_clean.csv',index=False) 43 | ``` 44 | 45 | >Tags File 46 | *Code* : Stackoverflow Tags Map & Model.ipynb 47 | 48 | 1. Read Tags File 49 | 2. Identify top 10 Tags by count 50 | ```python 51 | tagCount = collections.Counter(list(df_tags['Tag'])).most_common(10) 52 | print(tagCount) 53 | 54 | [('javascript', 124155), ('java', 115212), ('c#', 101186), ('php', 98808), ('android', 90659), ('jquery', 78542), ('python', 64601), ('html', 58976), ('c++', 47591), ('ios', 47009)] 55 | ``` 56 | 57 | 58 | 59 | 3. Manipulate the tags dataframe so that all the Tags for an ID are as a list in a row (grouped by Question ID) 60 | 61 | ```python 62 | def add_tags(question_id): 63 | return tag_top10[tag_top10['Id'] == question_id['Id']].Tag.values 64 | 65 | top10 = tag_top10.apply(add_tags, axis=1) 66 | ``` 67 | 68 | 69 | >Combine the Questions and Tags 70 | *Code* : Stackoverflow Tags Map & Model.ipynb 71 | 72 | Merge the Questions and Tags data frame by ID 73 | 74 | ```python 75 | total=pd.merge(ques, top10_tags, on='Id') 76 | ``` 77 | 78 | Our Dataset would now have only Id, Title, Body & Tags 79 | 80 | >Text Preprocessing 81 | *Code* : Stackoverflow Tags Map & Model.ipynb 82 | 83 | We will use nltk, preprocessing from Keras and sklearn to process the text data 84 | 85 | *Tags preprocesing* 86 | Use MultiLabelBinarizer from sklearn on the Class labels(Tags) 87 | ```python 88 | from sklearn.preprocessing import MultiLabelBinarizer 89 | multilabel_binarizer = MultiLabelBinarizer() 90 | multilabel_binarizer.fit(total.Tags) 91 | print(multilabel_binarizer.classes_) 92 | 93 | array(['android', 'c#', 'c++', 'html', 'ios', 'java', 'javascript','jquery', 'php', 'python'], dtype=object) 94 | ``` 95 | 96 | *Title & Body Preprocessing* 97 | 1. Tokenize the words 98 | 2. Convert the tokenized words to sequences 99 | 100 | **Model Building** 101 | 102 | Implemented a Hybrid model in TensorFlow using Keras as high level api. Architecture used is RNN. In this model first we train a model using the Title data, then train a model using the Body data. Outputs of both are concatenated and passed thorugh the dense layers before connecting to the output layer 103 | 104 | *RNN Model* : The model first uses GRU for the sequence data training with 2 GRU layers one for Title and other for Body. 105 | 106 | RNN for Title has 107 | - 1 Embedding Layer has input of Title vocabulary length(68969) + 1(for 0 padding) and out put of 2000 embeddings (for better results use full vocabulary length+1) 108 | - 1 Gated recurrent unit (GRU) layer 109 | - 1 dense output layer of shape 10(No of classes(tags) we are trying to predict) 110 | 111 | ```python 112 | # Title Only 113 | title_input = Input(name='title_input',shape=[max_len_t]) 114 | title_Embed = Embedding(vocab_len_t+1,2000,input_length=max_len_t,mask_zero=True,name='title_Embed')(title_input) 115 | gru_out_t = GRU(300)(title_Embed) 116 | # auxiliary output to tune GRU weights smoothly 117 | auxiliary_output = Dense(10, activation='sigmoid', name='aux_output')(gru_out_t) 118 | ``` 119 | 120 | RNN for Body has 121 | - 1 Embedding Layer has input of Title vocabulary length(1292018) + 1(for 0 padding) and out put of 170 embeddings (for better results use full vocabulary length+1) 122 | - 1 Gated recurrent unit (GRU) layer 123 | 124 | ```python 125 | # Body Only 126 | body_input = Input(name='body_input',shape=[max_len_b]) 127 | body_Embed = Embedding(vocab_len_b+1,170,input_length=max_len_b,mask_zero=True,name='body_Embed')(body_input) 128 | gru_out_b = GRU(200)(body_Embed) 129 | ``` 130 | 131 | Combine the 2 GRU outputs 132 | ```python 133 | com = concatenate([gru_out_t, gru_out_b]) 134 | ``` 135 | 136 | The fully connected network has 137 | - 2 Dense Layers 138 | - 1 Dropout layer 139 | - 1 BatchNormalization layer 140 | - 1 Dense Output layer 141 | 142 | ```python 143 | # now the combined data is being fed to dense layers 144 | dense1 = Dense(400,activation='relu')(com) 145 | dp1 = Dropout(0.5)(dense1) 146 | bn = BatchNormalization()(dp1) 147 | dense2 = Dense(150,activation='relu')(bn) 148 | main_output = Dense(10, activation='sigmoid', name='main_output')(dense2) 149 | ``` 150 | 151 | *Model Compilattion with optimizer='adam', loss='categorical_crossentropy', metrics='accuracy')* 152 | 153 | **Model Performance Review** 154 | 155 | *Classification Report to check Precision, Recall and F1 Score* 156 | 157 | The Model seem to performing good enough with score of 84%. Increase in the Embedding, GRU and dense layers would help in getting better results 158 | 159 | 160 | 161 | **Random Validation on Test Data** 162 | 163 | 164 | 165 | 166 | **Save the Model & Weights** 167 | 168 | Saving the model for transfer learning or model execution later 169 | 170 | ```python 171 | model.save('./stackoverflow_tags.h5') 172 | ``` 173 | -------------------------------------------------------------------------------- /Stackoverflow Clean Questions.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19", 8 | "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5" 9 | }, 10 | "outputs": [ 11 | { 12 | "name": "stdout", 13 | "output_type": "stream", 14 | "text": [ 15 | "['Answers.csv', 'Tags.csv', 'Questions.csv']\n" 16 | ] 17 | } 18 | ], 19 | "source": [ 20 | "# This Python 3 environment comes with many helpful analytics libraries installed\n", 21 | "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n", 22 | "# For example, here's several helpful packages to load in \n", 23 | "\n", 24 | "import numpy as np # linear algebra\n", 25 | "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n", 26 | "\n", 27 | "# for counting\n", 28 | "import collections\n", 29 | "\n", 30 | "# Input data files are available in the \"../input/\" directory.\n", 31 | "# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n", 32 | "\n", 33 | "import os\n", 34 | "print(os.listdir(\"../input\"))\n", 35 | "\n", 36 | "# Any results you write to the current directory are saved as output." 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": 2, 42 | "metadata": { 43 | "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0", 44 | "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a" 45 | }, 46 | "outputs": [ 47 | { 48 | "data": { 49 | "text/html": [ 50 | "
\n", 51 | "\n", 64 | "\n", 65 | " \n", 66 | " \n", 67 | " \n", 68 | " \n", 69 | " \n", 70 | " \n", 71 | " \n", 72 | " \n", 73 | " \n", 74 | " \n", 75 | " \n", 76 | " \n", 77 | " \n", 78 | " \n", 79 | " \n", 80 | " \n", 81 | " \n", 82 | " \n", 83 | " \n", 84 | " \n", 85 | " \n", 86 | " \n", 87 | " \n", 88 | " \n", 89 | " \n", 90 | " \n", 91 | " \n", 92 | " \n", 93 | " \n", 94 | " \n", 95 | " \n", 96 | " \n", 97 | " \n", 98 | " \n", 99 | " \n", 100 | " \n", 101 | " \n", 102 | " \n", 103 | " \n", 104 | " \n", 105 | " \n", 106 | " \n", 107 | " \n", 108 | " \n", 109 | " \n", 110 | " \n", 111 | " \n", 112 | " \n", 113 | " \n", 114 | " \n", 115 | " \n", 116 | " \n", 117 | " \n", 118 | " \n", 119 | " \n", 120 | " \n", 121 | " \n", 122 | " \n", 123 | " \n", 124 | " \n", 125 | " \n", 126 | " \n", 127 | " \n", 128 | " \n", 129 | " \n", 130 | " \n", 131 | " \n", 132 | " \n", 133 | " \n", 134 | " \n", 135 | " \n", 136 | " \n", 137 | " \n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | "
IdOwnerUserIdCreationDateClosedDateScoreTitleBody
08026.02008-08-01T13:57:07ZNaN26SQLStatement.execute() - multiple queries in o...<p>I've written a database generation script i...
19058.02008-08-01T14:41:24Z2012-12-26T03:45:49Z144Good branching and merging tutorials for Torto...<p>Are there any really good tutorials explain...
212083.02008-08-01T15:50:08ZNaN21ASP.NET Site Maps<p>Has anyone got experience creating <strong>...
31802089740.02008-08-01T18:42:19ZNaN53Function for creating color wheels<p>This is something I've pseudo-solved many t...
426091.02008-08-01T23:22:08ZNaN49Adding scripting functionality to .NET applica...<p>I have a little game written in C#. It uses...
533063.02008-08-02T02:51:36ZNaN29Should I use nested classes in this case?<p>I am working on a collection of classes use...
647071.02008-08-02T15:11:47Z2016-03-26T05:23:29Z13Homegrown consumption of web services<p>I've been writing a few web services for a ...
758091.02008-08-02T23:30:59ZNaN21Deploying SQL Server Databases from Test to Live<p>I wonder how you guys manage deployment of ...
8650143.02008-08-03T11:12:52ZNaN79Automatically update version number<p>I would like the version property of my app...
9810233.02008-08-03T20:35:01ZNaN9Visual Studio Setup Project - Per User Registr...<p>I'm trying to maintain a Setup Project in <...
\n", 180 | "
" 181 | ], 182 | "text/plain": [ 183 | " Id ... Body\n", 184 | "0 80 ...

I've written a database generation script i...\n", 185 | "1 90 ...

Are there any really good tutorials explain...\n", 186 | "2 120 ...

Has anyone got experience creating ...\n", 187 | "3 180 ...

This is something I've pseudo-solved many t...\n", 188 | "4 260 ...

I have a little game written in C#. It uses...\n", 189 | "5 330 ...

I am working on a collection of classes use...\n", 190 | "6 470 ...

I've been writing a few web services for a ...\n", 191 | "7 580 ...

I wonder how you guys manage deployment of ...\n", 192 | "8 650 ...

I would like the version property of my app...\n", 193 | "9 810 ...

I'm trying to maintain a Setup Project in <...\n", 194 | "\n", 195 | "[10 rows x 7 columns]" 196 | ] 197 | }, 198 | "execution_count": 2, 199 | "metadata": {}, 200 | "output_type": "execute_result" 201 | } 202 | ], 203 | "source": [ 204 | "ques = pd.read_csv('../input/Questions.csv',encoding='iso-8859-1')\n", 205 | "ques.head(10)" 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "execution_count": 3, 211 | "metadata": { 212 | "_uuid": "3a963c29707e29d8df14b8f48e45d25b336fcdb0" 213 | }, 214 | "outputs": [ 215 | { 216 | "data": { 217 | "text/html": [ 218 | "

\n", 219 | "\n", 232 | "\n", 233 | " \n", 234 | " \n", 235 | " \n", 236 | " \n", 237 | " \n", 238 | " \n", 239 | " \n", 240 | " \n", 241 | " \n", 242 | " \n", 243 | " \n", 244 | " \n", 245 | " \n", 246 | " \n", 247 | " \n", 248 | " \n", 249 | " \n", 250 | " \n", 251 | " \n", 252 | " \n", 253 | " \n", 254 | " \n", 255 | " \n", 256 | " \n", 257 | " \n", 258 | " \n", 259 | " \n", 260 | " \n", 261 | " \n", 262 | " \n", 263 | " \n", 264 | " \n", 265 | " \n", 266 | " \n", 267 | " \n", 268 | " \n", 269 | " \n", 270 | " \n", 271 | " \n", 272 | " \n", 273 | " \n", 274 | " \n", 275 | " \n", 276 | " \n", 277 | " \n", 278 | " \n", 279 | " \n", 280 | " \n", 281 | " \n", 282 | " \n", 283 | " \n", 284 | " \n", 285 | " \n", 286 | " \n", 287 | " \n", 288 | " \n", 289 | " \n", 290 | " \n", 291 | " \n", 292 | " \n", 293 | " \n", 294 | " \n", 295 | " \n", 296 | " \n", 297 | " \n", 298 | " \n", 299 | " \n", 300 | " \n", 301 | " \n", 302 | " \n", 303 | "
IdTitleBody
080SQLStatement.execute() - multiple queries in o...<p>I've written a database generation script i...
190Good branching and merging tutorials for Torto...<p>Are there any really good tutorials explain...
2120ASP.NET Site Maps<p>Has anyone got experience creating <strong>...
3180Function for creating color wheels<p>This is something I've pseudo-solved many t...
4260Adding scripting functionality to .NET applica...<p>I have a little game written in C#. It uses...
5330Should I use nested classes in this case?<p>I am working on a collection of classes use...
6470Homegrown consumption of web services<p>I've been writing a few web services for a ...
7580Deploying SQL Server Databases from Test to Live<p>I wonder how you guys manage deployment of ...
8650Automatically update version number<p>I would like the version property of my app...
9810Visual Studio Setup Project - Per User Registr...<p>I'm trying to maintain a Setup Project in <...
\n", 304 | "
" 305 | ], 306 | "text/plain": [ 307 | " Id ... Body\n", 308 | "0 80 ...

I've written a database generation script i...\n", 309 | "1 90 ...

Are there any really good tutorials explain...\n", 310 | "2 120 ...

Has anyone got experience creating ...\n", 311 | "3 180 ...

This is something I've pseudo-solved many t...\n", 312 | "4 260 ...

I have a little game written in C#. It uses...\n", 313 | "5 330 ...

I am working on a collection of classes use...\n", 314 | "6 470 ...

I've been writing a few web services for a ...\n", 315 | "7 580 ...

I wonder how you guys manage deployment of ...\n", 316 | "8 650 ...

I would like the version property of my app...\n", 317 | "9 810 ...

I'm trying to maintain a Setup Project in <...\n", 318 | "\n", 319 | "[10 rows x 3 columns]" 320 | ] 321 | }, 322 | "execution_count": 3, 323 | "metadata": {}, 324 | "output_type": "execute_result" 325 | } 326 | ], 327 | "source": [ 328 | "ques.drop([\"OwnerUserId\",\"CreationDate\",\"ClosedDate\",\"Score\"], axis=1, inplace=True)\n", 329 | "ques.head(10)" 330 | ] 331 | }, 332 | { 333 | "cell_type": "code", 334 | "execution_count": 4, 335 | "metadata": { 336 | "_uuid": "b1305b3807f362fec50a13a76d68d35fa45c3380" 337 | }, 338 | "outputs": [], 339 | "source": [ 340 | "import re \n", 341 | "\n", 342 | "def rem_html_tags(body):\n", 343 | " regex = re.compile('<.*?>')\n", 344 | " return re.sub(regex, '', body)" 345 | ] 346 | }, 347 | { 348 | "cell_type": "code", 349 | "execution_count": 5, 350 | "metadata": { 351 | "_uuid": "f0340076f170cc798549f9da6ce46962d9f81c6b" 352 | }, 353 | "outputs": [ 354 | { 355 | "data": { 356 | "text/html": [ 357 | "

\n", 358 | "\n", 371 | "\n", 372 | " \n", 373 | " \n", 374 | " \n", 375 | " \n", 376 | " \n", 377 | " \n", 378 | " \n", 379 | " \n", 380 | " \n", 381 | " \n", 382 | " \n", 383 | " \n", 384 | " \n", 385 | " \n", 386 | " \n", 387 | " \n", 388 | " \n", 389 | " \n", 390 | " \n", 391 | " \n", 392 | " \n", 393 | " \n", 394 | " \n", 395 | " \n", 396 | " \n", 397 | " \n", 398 | " \n", 399 | " \n", 400 | " \n", 401 | " \n", 402 | " \n", 403 | " \n", 404 | " \n", 405 | " \n", 406 | " \n", 407 | " \n", 408 | " \n", 409 | " \n", 410 | " \n", 411 | " \n", 412 | "
IdTitleBody
080SQLStatement.execute() - multiple queries in o...I've written a database generation script in S...
190Good branching and merging tutorials for Torto...Are there any really good tutorials explaining...
2120ASP.NET Site MapsHas anyone got experience creating SQL-based A...
3180Function for creating color wheelsThis is something I've pseudo-solved many time...
4260Adding scripting functionality to .NET applica...I have a little game written in C#. It uses a ...
\n", 413 | "
" 414 | ], 415 | "text/plain": [ 416 | " Id ... Body\n", 417 | "0 80 ... I've written a database generation script in S...\n", 418 | "1 90 ... Are there any really good tutorials explaining...\n", 419 | "2 120 ... Has anyone got experience creating SQL-based A...\n", 420 | "3 180 ... This is something I've pseudo-solved many time...\n", 421 | "4 260 ... I have a little game written in C#. It uses a ...\n", 422 | "\n", 423 | "[5 rows x 3 columns]" 424 | ] 425 | }, 426 | "execution_count": 5, 427 | "metadata": {}, 428 | "output_type": "execute_result" 429 | } 430 | ], 431 | "source": [ 432 | "ques['Body'] = ques['Body'].apply(rem_html_tags)\n", 433 | "ques.head()" 434 | ] 435 | }, 436 | { 437 | "cell_type": "code", 438 | "execution_count": 6, 439 | "metadata": { 440 | "_uuid": "af150557dcf11050e0e7dbc8bcbb0ef2c5015636" 441 | }, 442 | "outputs": [], 443 | "source": [ 444 | "ques.to_csv('question_clean.csv',index=False)" 445 | ] 446 | }, 447 | { 448 | "cell_type": "code", 449 | "execution_count": 7, 450 | "metadata": {}, 451 | "outputs": [], 452 | "source": [] 453 | } 454 | ], 455 | "metadata": { 456 | "kernelspec": { 457 | "display_name": "Python 3", 458 | "language": "python", 459 | "name": "python3" 460 | }, 461 | "language_info": { 462 | "codemirror_mode": { 463 | "name": "ipython", 464 | "version": 3 465 | }, 466 | "file_extension": ".py", 467 | "mimetype": "text/x-python", 468 | "name": "python", 469 | "nbconvert_exporter": "python", 470 | "pygments_lexer": "ipython3", 471 | "version": "3.6.7" 472 | } 473 | }, 474 | "nbformat": 4, 475 | "nbformat_minor": 1 476 | } 477 | -------------------------------------------------------------------------------- /Stackoverflow Tags Map & Model.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19", 8 | "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5" 9 | }, 10 | "outputs": [ 11 | { 12 | "name": "stdout", 13 | "output_type": "stream", 14 | "text": [ 15 | "['stackoverflow-clean-questions-file-v2', 'stackoverflow']\n" 16 | ] 17 | } 18 | ], 19 | "source": [ 20 | "# This Python 3 environment comes with many helpful analytics libraries installed\n", 21 | "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n", 22 | "# For example, here's several helpful packages to load in \n", 23 | "\n", 24 | "import numpy as np # linear algebra\n", 25 | "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n", 26 | "\n", 27 | "# Input data files are available in the \"../input/\" directory.\n", 28 | "# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n", 29 | "\n", 30 | "import os\n", 31 | "print(os.listdir(\"../input\"))\n", 32 | "\n", 33 | "# Plotting Libs\n", 34 | "import matplotlib.pyplot as plt\n", 35 | "import matplotlib.cm as cm\n", 36 | "# magic function\n", 37 | "%matplotlib inline\n", 38 | "\n", 39 | "import collections" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": 2, 45 | "metadata": { 46 | "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0", 47 | "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a" 48 | }, 49 | "outputs": [], 50 | "source": [ 51 | "df_tags = pd.read_csv('../input/stackoverflow/Tags.csv', encoding='iso-8859-1')" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": 3, 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "def plot_tags(tagCount):\n", 61 | " \n", 62 | " x,y = zip(*tagCount)\n", 63 | "\n", 64 | " colormap = plt.cm.gist_ncar #nipy_spectral, Set1,Paired \n", 65 | " colors = [colormap(i) for i in np.linspace(0, 0.8,50)] \n", 66 | "\n", 67 | " area = [i/4000 for i in list(y)] # 0 to 15 point radiuses\n", 68 | " plt.figure(figsize=(9,8))\n", 69 | " plt.ylabel(\"Number of question associations\")\n", 70 | " for i in range(len(y)):\n", 71 | " plt.plot(i,y[i], marker='o', linestyle='',ms=area[i],label=x[i])\n", 72 | "\n", 73 | " plt.legend(numpoints=1)\n", 74 | " plt.show()" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 4, 80 | "metadata": {}, 81 | "outputs": [ 82 | { 83 | "name": "stdout", 84 | "output_type": "stream", 85 | "text": [ 86 | "[('javascript', 124155), ('java', 115212), ('c#', 101186), ('php', 98808), ('android', 90659), ('jquery', 78542), ('python', 64601), ('html', 58976), ('c++', 47591), ('ios', 47009)]\n" 87 | ] 88 | }, 89 | { 90 | "data": { 91 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHVCAYAAAAdLJRmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd4lFXa+PHveSYz6QkkoSSUDSg1FQgoIB0FxQIoy8uLQkTZn21ffS1r3dVVd1fXXtfdtUQUFUVRX3FFkaaASsDQqxBqIJBASJ92fn/MJAZImSQzk3Z/ritXMuc5zzn3REzunOcUpbVGCCGEEKItMpo6ACGEEEKIpiKJkBBCCCHaLEmEhBBCCNFmSSIkhBBCiDZLEiEhhBBCtFmSCAkhhBCizQpo6gCEEEIIca74+xZbgARgIBAHBALlwBFgA7Al+4lJtqaLsHVQso+QEEII0TzE37fYBFwG/AG4ACjD9fQmBFCABkoAJxAE/Aj8Hfgy+4lJjqaIuaWTREgIIYRoYvH3LVbAtcCzuEZ+wutxeyGukaI7gXezn5gkv9jrQRIht5iYGB0fH9/UYQghhGhjHJZwihKnYovoCgGWhjdkt2I+fZDwLZ9gWIu8F2ALtX79+hNa6w511ZNEyC0tLU1nZmY2dRhCCCHakC2HC5jx7x8otTqwOxv/+zjAUARbTLw/90ISu0R6IcKWSym1XmudVlc9WTUmhBBCNIHNhwr47T/XUlhm90oSBGB3agrL7Pz2n2vZcrjAK222dpIICSGEEH527HQZ//36D5RYfTO/ucTqYMa/fyD3dJlP2m9NZPm8D1jtTnYdK2TrkQKOnS6n3O4kMMCgU0QgCXGR9OkcjtkkOagQQrRFWmvuWJBFqY+SoAqlVgd3LMhi/o0XoJTyaV8tmSRCXuJwapbvyOW1lb+QdfAUgWYDpxPKbA40rjWPQWYThgHlNiep3dpx06jzGNO3IyZD/oEKIURb8dG6/fy8P99rj8NqYndqNuzP5+UvfmL8+RE+7aspBQUF0bVrV8xmc4Pul8nSbg2dLK215pMNh3l88TasdifF9cjwQy0mLAEGD03qz9SBXSRjF0KIVs7h1Ax8dAkFZf7b8qd9iJnMhy5ulX90a63Jy8ujsLCQHj16nHFNJkv7gesZ74889OkWTpbY6pUEARRbHZwssfHQp1uY+fqP8ixXCCFaueU7crHanX7t02p3smJnrl/79BelFNHR0ZSVNfz3pyRCDbTlcAHjn13Jun35lNoal9mX2hz8tC+fcc+ulFn+QgjRir228hdK7f59ElNsdfDayl/82qc/NfZpiiRCDSBLHoUQQtSX1e4k6+CpJun75wOnsDn8OxLVUkgiVE+y5FEIIURD7DpWSKC5aX7tBpoNdh0rPKNs2LBhTRLL2Y4cOcI111xTa51Tp07x6quv+qR/SYTqwd9LHmUiuxBCtB5bjxTgbKJBGa0552nDmjVrmiaYKux2O3FxcSxcuLDWepIINROfbDhM1oFTflny+POBU3yy4bBP+xFCCOE/x06XU9bIOaUNVWp1kHu6/IyysLAwioqKGDduHAMHDiQpKYnPPvsMgPvuu49XXnmlsu4jjzzC008/XWP94uJiJk2aREpKComJiSxYsACAdevWMWzYMFJSUhgyZAiFhYVkZGRw5ZVXMnbsWMaNG0d2djaJiYkAZGRkcNVVVzF69Gh69erFn//858p4fvnlF1JTU7nnnnu8+r2RfYQ85HBqHl+8rdEToz1VanPw+OJtTB7QpVUueRRCiLam3O6kvn9Gh1NCgpFNF05gVnZsOoDDxLDVGU8hIR63o939ny0oKIhFixYRERHBiRMnuPDCC7nyyiuZPn06d9xxB7feeisAH374IUuWLKmx/ldffUVcXByLFy8GoKCgAKvVyvTp01mwYAGDBw/m9OnTBAcHA7BhwwY2bdpEVFQU2dnZZ8T0008/sWXLFkJCQhg8eDCTJk3iiSeeYMuWLWRlZdXzO1g3SYQ81JRLHsf16+TXfoUQQnhfYICBgjqToVjyuC7ga642fUcUhZRiwYQThUajcGAQjJV8wvnYMYJ37JeQQ3StbSp3/2fTWvPAAw+watUqDMPg8OHDHDt2jAEDBpCbm8uRI0c4fvw47du3p1u3bthstmrrJyUlcdddd3Hvvfdy+eWXM2LECDZv3kxsbCyDBw8GICLi100dL774YqKioqqN9eKLLyY62vV+pk6dyvfff8/kyZPr+K41nCRCHnpt5S/13ieosSqWPEoiJIQQLV+niECCzKYanyx04CR/M7/OCGMLoAlUdgDMlFbfHqe4wfQf5pi+4ntnIvfb5nKcdtXWDbaY6BgReE75/PnzOX78OOvXr8dsNhMfH1+5J8+0adNYuHAhR48eZfr06bXW7927Nxs2bODLL7/koYceYty4cUyZMqXG70VoaGiN185eDu/rzYZljpAHZMmjEEKIxkqIi8So9reu5irje5YH3sVIYxOBylaZBNUlUNkJUjZGGptYHngnVxnfU92Yk1KQ2CXynPKCggI6duyI2Wxm+fLl7N+/v/La9OnT+eCDD1i4cCHTpk2rtf6RI0cICQnh2muv5Z577mHDhg306dOHnJwc1q1bB0BhYSF2e93v65tvviE/P5/S0lI+/fRThg8fTnh4OIWFhXXe2xAyIuSBiiWP9nL/T3KrWPKYEHfuP2AhhBAtR+9O4ZTbzv7DVvNwwDymm1YQosqrvc8TFuXAgoO/md8g1fELf7bPwvVAzKXc5qR3p/Az7lFKMXPmTK644gqSkpJIS0ujb9++ldcTEhIoLCykS5cuxMbGAtRYf/Pmzdxzzz0YhoHZbOYf//gHFouFBQsW8Pvf/57S0lKCg4NZunRpne9lyJAhXH311Rw6dIhrr72WtDTXKRnDhw8nMTGRSy+9lKeeeqrB36uzyVljbrWdNbZg3QEe+dx/E6WrCrGYePiK/kwf3N3vfQshhPCua/6xhsz9J92vNI8EvM1vTSsblQSdrUQHssAx+oxkaHB8ez666dd9g/Ly8hg4cOAZI0DNQUZGBpmZmbz88sv1um/79u3069fvjDI5a8yLmtuSRyGEEC3TTaPOIzjAlZxcZaz2ehIEEKLKmW5awZXGasB1wPdNo86rvH7kyBGGDh3K3Xff7dV+Wyp5NOaBhix59JaaljwKIYRoecb07YjZpAiz5/MX85teT4IqhKhy/mp+k7XliTjMHRndp2Pltbi4OHbt2uWTfhsrPT2d9PR0v/YpiZAHPF3y6As1LXkUQgjR8pgMxdy0KBLX/R0LNp/2ZcHGk5Y3OHnZPNmPrhaSCHmgriWPvlTTkkchhBAt04TYEuKNzVjw7e8Ui3JwkdqE+XyfdtPiyVCDB2pe8uh7NS15FEII0TJF7fkYs8k/IzRmk4Fa97pf+mqpJBHyQPVLHv2juiWPQgghWq7I7P+gHFa/9KUc5ZD1vl/6aqkkEfKAJcAgtVv1u3X62oDu7TCb5D+TEEK0CmUFBFj9vEFvSR6UnT6neNiwYdVUbnvkN6yHbhp1HqEWk1/7PHvJoxBCiBYuZxNOk5/nfZqDIWfjOcVr1qzxbxzNlCRCHhrTtyMWP6/eCjSbzljyKIQQooUrOAjaz1MtnA5Xv2cJCwujqKiIcePGMXDgQJKSkvjss88AuO+++3jllVcq6z7yyCM8/fTTNdZvySQR8pDJUDw0qT/BZv+MCgWbTTx4WT9Z8iiEEK2Jw4ry+4kOGmqYkxQUFMSiRYvYsGEDy5cv56677kJrzfTp0/nwww8r63344YdMnz69xvotmSyfr4epA7vw8YZD/LQvH7vTd//hAwzFgO7tmDqwi8/6EEII0QRMFrSPT1M/lwKTpdorWmseeOABVq1ahWEYHD58mGPHjjFgwAByc3M5cuQIx48fp3379nTr1g2bzVZt/c6dO/v5PXmPJEL1oJTiuempjH92JYVlnp0M3BDBFhPPT09F+f1/FiGEED4V2Q2UZw9jCpVic2AgWwMt7LKYKVOKIK3pbbWRUG4lqbyccE9GYwyTq99qzJ8/n+PHj7N+/XrMZjPx8fGUlZUBMG3aNBYuXMjRo0eZPn16nfVbKkmE6qlTRBDvz72Q3/5zLSVW72+GFWIx8f7cC+kYEeT1toUQQjSx2GQMe+3Hamy1mMmIjGB5SDBmoEwp7FX+MF6qNUFaYwPGlJSSXnCaBGstu1TbSiE2pdpLBQUFdOzYEbPZzPLly884hHX69OnMnTuXEydOsHLlyjrrt1QyR6gBErtE8uH/G0p4UAABXprDE2AoIoIC+PD/DZUNFIUQorUKisQeWP12LMVK8VBMFOmxnfg6NIRyw6DIMM5IggDsSlFkGJQbBl+HhpAe24mHYqIorukpQkg0BEWcU6yUYubMmWRmZpKUlMS8efPo27dv5fWEhAQKCwvp0qULsbGxALXWb6lkRKiBErtE8u2do7hjQRY/HzjVqOM3gs0mBnRvx/PTU2UkSAghWrmC+EuJ2b3gjAnMe80BzOnciUJDYa3HUQZOpShTiq9CQ/g+OJg3jx6jp63K1A1TIKTOOOe+vLw8oqKiiImJYe3atTW2v3nz5jNe11W/JZIRoUboGBHE/Bsv4PHJibQPMdd7n6FQi4n2IWb+MiWR+TdeIEmQEEK0ASd7XYPrSG2XveYAZsZ2Jt9k1CsJqqrcMMg3GcyM7cxe81ljHIPnnvHyyJEjDB06lLvvvrtBfbU2MiLUSEoprh7UlckDurBiZy6vrfyFnw+cItBsoDWUWh1oXP/kgy0mlHIdmzGgeztuGnUeo/t0lCXyQgjRhthDOsF5Y2DPtxRrO3M6d6LYUI1eTaaVotiAOZ07sfjQEUJVAJw3FiLPXIEcFxfHrl27GtVXayKJkJeYDMW4fp0Y168TNoeTXccK2XK4gNzT5ZTbnQQGGHSMCCSxSyS9O4XLsRlCCNGWXfECvDSIv0WEU+SFJKiCVooiQ/FEdHseO2119SNqJYmQD5hNBglxkSTEyaRnIYQQ1QjvzNYx97Bkx+uUe/mpQLlh8FVoKDPS7qB/eCevtt0a+WxYQin1plIqVym1pUrZU0qpHUqpTUqpRUqpdlWu3a+U2qOU2qmUmlClfKK7bI9S6r4q5T2UUj+6yxcopSzu8kD36z3u6/G+eo9CCCFEQ2VYD2P10dQIq6HIKD/sk7ZbG18+n8kAJp5V9g2QqLVOBnYB9wMopfoD/wUkuO95VSllUkqZgFeAS4H+wAx3XYAngee01ucDJ4Eb3OU3ACfd5c+56wkhhBDNRqG1kOUHl+OrU8ecwLKDyyi0Fvqoh9bDZ4mQ1noVkH9W2dda64p1fT8AXd1fXwV8oLUu11rvA/YAQ9wfe7TWe7XWVuAD4Crl2nJ5LLDQff/bwOQqbb3t/nohME7JFs1CCCGakc0nNmM2zD7tw2yY2XJiS90Vz1JcXMz48eMBuOiii7DbfXeSQnPQlDN25wD/cX/dBah6NO4hd1lN5dHAqSpJVUX5GW25rxe4659DKfU7pVSmUirz+PHjjX5DQgghhCe2nthKmd23R1OUOcrYlret3vetXbuWoUOHcvLkSUJDQwkIaN3TiZskEVJKPQjYgflN0X8FrfW/tNZpWuu0Dh06NGUoQggh2pBdJ3dh174dabE77ezI31Hj9Xnz5pGcnExKSgrXXXcdv/zyC6mpqVx77bW89957DBo0iI0bN5Kamkpubq5PY21Kfk/zlFLpwOXAOK0rT4s7DFQ9Ea6ru4wayvOAdkqpAPeoT9X6FW0dUkoFAJHu+kIIIUSzUObwz0Gl5Y7qzzXbunUrjz/+OGvWrCEmJob8/HyioqLIyspi0qRJzJs3j5dffpm0tDQmTZrkl1ibil9HhJRSE4E/AFdqrUuqXPoc+C/3iq8eQC/gJ2Ad0Mu9QsyCa0L15+4Eajlwjfv+2cBnVdqa7f76GmBZlYRLCCGEaHJBJv+cJBBoCqy2fNmyZUybNo2YmBgAoqKiKq/l5uYSHR3Npk2bSEmp/rDW1sSXy+ffB9YCfZRSh5RSNwAvA+HAN0qpLKXUawBa663Ah8A24CvgVq21wz3acxuwBNgOfOiuC3AvcKdSag+uOUBvuMvfAKLd5XcClUvuhRBCiOagd/veBKi6H8oop6brcc3gnU6GbXMyeKeTrsc1yln33/cBRgB9ozw/FPWmm24iMTGR3bt3k5qayldffcXll1/Oc88953EbLZHPHo1prc895e3XZKW6+n8B/lJN+ZfAl9WU78W1quzs8jJgWr2CFUIIIfwoISaBoIAgimxF51xTTs2AXzRX/uik1xGwG+A0QGnQCgwnBDhhdxx8foHBz+cpdDX7EQWZgugf3f+ccoCxY8cyZcoU7rzzTqKjo8nPz+e1117jo48+4sCBA1x99dX84Q9/4MMPP/T6e29uWvdUcCGEEKIZSopJwua0nVPe96Dm9s8chJRDsPtwerOj+jb6H4Qex5yUBMILV5nY0e3MZMjmtJEYk1jtvQkJCTz44IOMGjUKk8nEgAEDyMjIYOXKlcyaNYvvvvuOUaNGNeo9thSSCAkhhBB+Fm4JZ0y3MXy9/2uc2olyatKXOhm7URNYj8VkwVbXx4MfOFiWosgYb6ANhaEMxnYfS7glvMZ7Z8+ezezZs88oe/nllwEYMuScBy6tlpz8KYQQQjSB9MR0LIYF5dTc9YmTMfVMgqoKtMOYja52lFNjMSykJ6R7Nd7WShIhIYQQogkkRCcwIX4CN3yrSN6nCWrktkJBdkjep7nhW8XE+Ik1zg8SZ5JHY6JuZQWQswkKDoLDCiYLRHaD2GQIimzq6IQQosX6X9MEDmV9jMVLeysG2WF0lp1uAWcf9SlqIomQqF7BIfjpddj4PpTkgTkYnA5cR/kZYJjAVgoh0ZAyA4bcCJFd62pVCCGEm3Y4yL/vj15LgipY7JB374O0X/YtymTybuOtkCRC4kyFR+H/bodflgPaNQIEUH7u6gYAio7CD6/AD6/CeWPgihchvJPfwhVCiJaqaNUqHEW+OR3eWVhI0XffET56tE/ab01kjpBw0Ro2LoCXBsGeb8FR/msSVBeH1VV/z7fw0kBXO7KZtxBC1CrvjTfRxSV1V2wAZ0kJ+a/XuHWfqEISIeFKWv5zL3xxB1iLoJq9LTzitLnu/+IOV3uSDAkhRLW0w0HZxo0+7aN040a0o4ZNiGoQHx/PiRMnfBRR8ySJUFunNfznD/DzO2Dz0l8mthJXe5IMCSFEtax796LMZp/2ocxmrPv2+bSP1kASobZu04fw87veS4IqVCRDmz/ybrtCCNEKlGdng+HjX8GGgTU7u9pL2dnZ9O3bl5kzZ9KvXz+uueYaSkpcvwdeeuklBg4cSFJSEjt27ADgkUce4brrrmPo0KH06tWLf//7376N3Y8kEWrLCo/C4ju9nwRVsJXAF/8Lhcd8074QQrRQ2mr1/Yi51jjLy2u8vHPnTm655Ra2b99OREQEr776KgAxMTFs2LCBm2++maeffrqy/qZNm1i2bBlr167l0Ucf5ciRI76N308kEWrL/u92sNf8P4lX2Mtd/QghhKikLBZQ5x6U6t1OFEZgYI2Xu3XrxvDhwwG49tpr+f777wGYOnUqAIMGDSK7yojSVVddRXBwMDExMYwZM4affvrJd7H7kSRCbVXBIdcS+YZOjPaU0wa/LIOCw77tRwghWpDA+HhwOn3bidOJJT6+xsvqrESs4nWgO3kymUzY7fY667d0kgi1VT+9DvhxIvO61vM8WQghGsvSsyfaVvcfohooDYrmRFQCuTGpnIhKoDQo2qOf3tpmw9KjR43XDxw4wNq1awF47733uOiii2pt77PPPqOsrIy8vDxWrFjB4MGDPYii+ZMNFduqje97vk9QYznKIet9GP+If/oTQohmTplMBKWkUJqZWe3102HdONhtHMdjkl319a+jR1q5xjA6nNhEt4PfElF0sNo2glNSat1Zuk+fPrzyyivMmTOH/v37c/PNN/PSSy/VWD85OZkxY8Zw4sQJ/vjHPxIXF1fn+2wJJBFqi8oKXMdm+FNJHpSdhqAI//YrhBDNVPQNczi8fdsZmypazeFs6zeLU5Hn4VQBruOManCsw0COxyTT7tQv9N8xD4vt112qjZAQom68odb+AwICePfdd88oqzonKC0tjRUrVlS+Tk5OZt68eR6+u5ZDHo21RTmbXGeH+ZM5GHJ8u3mYEEK0JGEjR2IKC698fTLyfNZe8DAn2/XGaQqsNQkCwDDhNAVysn1v132R5/96KTycsBEjfBV6qyKJUFtUcNB9gKofOR2ufoUQQgCux2NdnnkaFRTEycjz2Zh8C46AYLRRv4c12gjAERDMxuRbOBl5Piow0NVuLY/F4uPj2bJli8d9PPLII9x99931iqulkEdjbZHDiusUeX/S/puTJIQQLURIWhpBk6ez6VCSaxSoEZymQDYl3cTErpsJSUvzUoStn4wItUUmC/7/T6/c/QohhKhqU/sJOAO88/PRGWBhc9QEr7TVVkgi1BZFdqv72bO3GSZXv0IIISrl7j9Nzi8FaLzzM1lj4sieAnL3n/ZKe22BJEJtUWwy2Er926etFGJT/NunEEI0c1lLD+KweXeqgsPuJGupzMn0lCRCbVFQJIRE+7fPkGhZOi+EEFVordm38bjXjxzTTtzt+m7T3LCwsHrfM2zYsGrL09PTWbhwYWNDajBJhNqqlBn+m7NjCoTUGf7pSwghWojTJ8p82n5hnm/bP1vV4ziqs2bNGj9FUj+SCLVVQ24E/HhOzOC5/utLCCFagJNHizEM3/wcNgxFfk5xrXUmT57MoEGDSEhI4F//+hfgGul58MEHSUlJ4cILL+TYsWMA7Nu3j6FDh5KUlMRDDz1U2caKFSsYMWIEV155Jf379wfg2WefJTExkcTERJ5//vnKuhWjSFprbrvtNvr06cP48ePJzc316nuvL0mE2qrIrnDeGDDMvu3HMMN5YyGyi2/7EUKIFsZhc/rsxEftbr82b775JuvXryczM5MXX3yRvLw8iouLufDCC9m4cSMjR47k3/92nRN5++23c/PNN7N582ZiY2PPaGfDhg288MIL7Nq1i/Xr1/PWW2/x448/8sMPP/Dvf/+bn3/++Yz6ixYtYufOnWzbto158+Y1+UiRJEJt2RUvQEDj9q2oU0Cgqx8hhBBnMJkNn43LK3f7tXnxxRcrR34OHjzI7t27sVgsXH755QAMGjSo8siN1atXM2OGa4rDddddd0Y7Q4YMoYf7cNfvv/+eKVOmEBoaSlhYGFOnTuW77747o/6qVauYMWMGJpOJuLg4xo4d64V33HCSCLVl4Z1h0rNgDvFN++YQuPw5CO/km/aFEKIFa985FKfTN2NCTqcmKja0xusrVqxg6dKlrF27lo0bNzJgwADKysowm80o5UrPTCbTGfN+KsrPFhpacz8tgSRCbV3yb2HAdd5PhswhMHAWJE3zbrtCCNFKRMQE+bT98Oia2y8oKKB9+/aEhISwY8cOfvjhh1rbGj58OB988AEA8+fPr7HeiBEj+PTTTykpKaG4uJhFixYx4qwzz0aOHMmCBQtwOBzk5OSwfPnyerwr75NEqK1TCi590rvJUEUSNPEJV/tCCCHOoZSiR0oHr/+YVAbudmtueOLEidjtdvr168d9993HhRdeWGubL7zwAq+88gpJSUkcPny4xnoDBw4kPT2dIUOGcMEFF3DjjTcyYMCAM+pMmTKFXr160b9/f2bNmsXQoUPr9wa9TPlyn4GWJC0tTWdmZjZ1GE1Ha9j8EXzxv2AvB6et/m0YZtecoMufc40ESRIkhBDn2L59O/369QNcO0svemYDduu5E5u1tuG07sZhP4C2H0NjQ2FGBXTCFNAdw9ILpc5d8BJgMZh69yA6dA8/51prVfV7WkEptV5rXeeha3LoqnBRyvWYrMco+L/b4ZdleHxQqskCKNfqsCtekDlBQgjhoY6/iSD2/HYc3nkSp8M1MKG1HXvpGhzlWbimPf/6h6kGtDUPp3U3lCzFFJhKQPAwlHL9OjdMirhe7dpUEtRYkgiJM4V3gv/+AAoOwbrXIet9KMkDczA4Hbj+N1Sus8Nspa4do1NnuPYJkiXyQghRb+PT+/Pun9bidDhwOvKwFn4CuhSobYNCV3LkKM/CYd2JJXwqhimaALPBuNn9/RJ3ayGJkKheZFcY/4jro6wAcjZBwUHXCJHJ4jpANTZFjs0QQohGComwMOmWZD5/fhllJ98HPBiJr2QHXYj19PuERM/kslvGEhLhp1MDWglJhETdgiKhx4i66wkhhGiQTvGhYP+c+iVBVVnB8X90ir/Cm2G1CbJqTAghhGhiaz6aj7W0qFFtWEsKWfNRzUvbRfUkERJCCCGakK2sjJ+/+gK7tbxR7dit5fy85AtsZf49bLWlk0RICCGEaEK7flyN8tLhq0opdv242qO6w4YN80qfLZ0kQkIIIUQTOrBlo9dGcWxlZRzcusmjur487LTq0RzNnSRCQgghRBM6tnePV9s76mF7YWFhaK257bbb6NOnD+PHj+eyyy5j4cKFAMTHx3PixAkAMjMzGT16NADFxcXMmTOHIUOGMGDAAD777DMAMjIyuPLKKxk7dizjxo1j1qxZfPrpp5X9zZw5s7JucyKrxoQQQogm1Ni5Qee0V+756NKiRYvYuXMn27Zt49ixY/Tv3585c+bUes9f/vIXxo4dy5tvvsmpU6cYMmQI48ePB2DDhg1s2rSJqKgoVq5cyXPPPcfkyZMpKChgzZo1vP322416b77gsxEhpdSbSqlcpdSWKmXTlFJblVJOpVTaWfXvV0rtUUrtVEpNqFI+0V22Ryl1X5XyHkqpH93lC5RSFnd5oPv1Hvf1eF+9RyGEEKKxAiyB3m0v0PPDXFetWsWMGTMwmUzExcUxduzYOu/5+uuveeKJJ0hNTWX06NGUlZVx4MABAC6++GKioqIAGDVqFLt37+b48eO8//77XH311QQENL/xF18+GssAJp5VtgWYCqyqWqiU6g/8F5DgvudVpZRJKWUCXgEuBfoDM9x1AZ4EntNanw+cBG5wl98AnHSXP+euJ4QQQjRLnXqe79W3V19eAAAgAElEQVT2OnupvYCAAJxO1xloZVXmMGmt+fjjj8nKyiIrK4sDBw5UnvMVGhp6RhuzZs3i3Xff5a233qpzpKmp+CwR0lqvAvLPKtuutd5ZTfWrgA+01uVa633AHmCI+2OP1nqv1toKfABcpVxH6o4FFrrvfxuYXKWtirG3hcA4VdsRvEIIIUQT6p6YgjnI81Gc2piDguiWkOxx/ZEjR7JgwQIcDgc5OTksX7688lp8fDzr168H4OOPP64snzBhAi+99BIVh7b//PPPNbafnp7O888/D0D//s3z6I/mMlm6C3CwyutD7rKayqOBU1pr+1nlZ7Tlvl7grn8OpdTvlFKZSqnM48ePe+mtCCGEEJ7rfcFwtFN7pS2tNb0vGO5RXaUUU6ZMoVevXvTv359Zs2YxdOjQyusPP/wwt99+O2lpaZhMpsryP/7xj9hsNpKTk0lISOCPf/xjjX106tSJfv36cf311zf8TflY83tY50da638B/wJIS0vzzr9CIYQQoh7MQUEMmHh5ozdVDLAEMmDC5R6NLuXl5REVFYVSipdffrmyPD09vfLrESNGsGvXrnPuDQ4O5p///Oc55enp6WfcD1BSUsLu3buZMWOG52/Ez5rLiNBhoFuV113dZTWV5wHtlFIBZ5Wf0Zb7eqS7vhBCCNEsDZs2k+CICKD6mRwKhdkIRNVwHaUIiYhk2G+vrbOvI0eOMHToUO6+++5GRFy3pUuX0q9fP37/+98TGRnp074ao7mMCH0OvKeUehaIA3oBP+H6F9FLKdUDV4LzX8B/a621Umo5cA2ueUOzgc+qtDUbWOu+vkxXPMgUQgghmqEAi4Wr73+U9/54F9aSUkBjNgKJD0ukT+QQQkzhOHFgYKLEUcjOgp/ILtqCzVkOSmEJDmbq/X8mwGyus6+4uLhqR3rAtReQt4wfP579+/d7rT1f8VkipJR6HxgNxCilDgEP45o8/RLQAVislMrSWk/QWm9VSn0IbAPswK1aa4e7nduAJYAJeFNrvdXdxb3AB0qpx4GfgTfc5W8A7yil9rj7+y9fvUchhBDCW6K7duO/H3uGj//2J+J1P3qHpQGaAMMCgMn9Kzs0IILk9iNJbj+KXUWZ7Fc7mHr/n4nu2q2W1kVNlAyWuKSlpenMzMymDkMIIUQrt3379srl5mfTWpP3wQ5KNuVi6LpnrziVk5CUjkRP70tbXiBd3fdUKbVea51Wwy2VmsscISGEEKLNO730AOXb8j1KggAMbVC+NZ/TSw/4OLLWSxIhIYQQohlwltopXHkIbXPW6z5tc1K48hDOspZz0GlzIomQEEII0QwUrz9GQ59uKQXFmce8G1AVf/3rXyu/zs7OJjEx0Wd9+ZskQkIIIUQzUPRd/UeDKmibk6LvDnk5ol9VTYRaG0mEhBBCiCamHU4cp62NasNx2op2eLYAKjs7m759+zJz5kz69evHNddcw5dffsnkyZMr63zzzTdMmTKF++67j9LSUlJTU5k5c6arL4eDuXPnkpCQwCWXXEJpaSkAWVlZXHjhhSQnJzNlyhROnjwJwOjRo7n33nsZMmQIvXv35rvvvmvUe/UmSYSEEEKIJqatTjAauerLUGirw+PqO3fu5JZbbmH79u1ERESwdetWduzYQcWRUxUHpT7xxBMEBweTlZXF/PnzAdi9eze33norW7dupV27dpVnkc2aNYsnn3ySTZs2kZSUxJ///OfK/ux2Oz/99BPPP//8GeVNTRIhIYQQookpiwGNPW/MqVEWU9313Lp168bw4a5zya699lpWr17Nddddx7vvvsupU6dYu3Ytl156abX39ujRg9TUVAAGDRpEdnY2BQUFnDp1ilGjRgEwe/ZsVq1aVXnP1KlTz6jfXDSXnaWFEEKINkuZDEwRFhwFDX88ZoqwoEyejyqdve+QUorrr7+eK664gqCgIKZNm0ZAQPVpQmBg4K/9mkyVj8ZqU3GPyWTCbm8+K9xkREgIIYRoBsJGdEWZG/ZrWZkNwkZ0rdc9Bw4cYO3atQC89957XHTRRcTFxREXF8fjjz9+xonxZrMZm81Wa3uRkZG0b9++cv7PO++8Uzk61JxJIiSEEEI0A6GDOtHQwx60htC0TvW6p0+fPrzyyiv069ePkydPcvPNNwMwc+ZMunXrdsZOzb/73e9ITk6unCxdk7fffpt77rmH5ORksrKy+NOf/lT/N+NncsSGmxyxIYQQwh9qO2Kj4Jv9FK2q3zJ6ZTYIG9mVyIt/4/E92dnZXH755WzZsuWca7fddhsDBgzghhtu8Li9ptaYIzZkjpAQQgjRTESM747jZBmlm094lAwps0FwUgwR47t7pf9BgwYRGhrKM88845X2WgJJhIQQQohmQilF+2m9MbUPonDlIZSCIlsJJ4xCrNixEECMM5wwcwhaQ9jIrkSM717vA1fj4+OrHQ1av369t95KiyGJkBBCCNGMKKWIGN+d412tfPfNCg7l52BohQYU4FSarlGxjLhkNF37ev44TFRPEiEhhBCiGXE6nSxevJhNmzZVrtRynDXgsz//MEc+/pDk5GQmTZqEYcjap4aS75wQQgjRTGitz0mCamKz2di0aROLFy/2U3StkyRCQgghRDOxb98+j5KgChXJ0N69e30cWesliZAQQgjRTHz//fceJ0EVbDYbq1evrtc92dnZJCYmnlOekZHBkSNHvNJWSyGJkBBCCNEMFBQUsH///gbdW3HWV2M1JBFq6SQREkIIIZqBnJycGs/2qktAQAA5OTn1usfhcDB37lwSEhK45JJLeOedd8jMzGTmzJmkpqZSWlpKfHw8999/P6mpqaSlpbFhwwYmTJjAeeedx2uvvdagWJsbSYSEEEKIZqC8vJyGnvagtaa8vLxe9+zevZtbb72VrVu30q5dO5RSpKWlMX/+fLKysggODgage/fuZGVlMWLECNLT01m4cCE//PADDz/8cINibW5k+bwQQgjRDAQGBtZ7Y8QKSqkzToT3RI8ePUhNTQVcO0pnZ2dXW+/KK68EICkpiaKiIsLDwwkPDycwMJBTp041KN7mREaEhBBCiGYgNjYWu93eoHvtdjuxsbH1uqdq4mQymWrsu6KeYRhn3GMYRoPjbU4kERJCCCGagcjISH7zm4btFB0fH09kZGSjYwgPD6ewsLDR7bQkkggJIYQQzcRFF12E2Wyu1z1ms5nhw4d7pf/09HRuuummysnSbYFq6MSs1iYtLU1nZmY2dRhCCCFaue3bt9OvX79qr2mt+eKLLzzeVNFsNpOcnMwVV1zh7TBblOq+p0qp9VrrtLrulcnSok0otBay+cRmtp7Yyq6TuyhzlBFkCqJ3+94kxCSQFJNEuCW8qcMUQrRxSikmTZoEUGcyVJEEVdQXDSOJkGjVtuZtJWNLBssPLsdsmCmzl2HXv07uW7p/KUEBQdicNsZ0G0N6YjoJ0QlNGLEQoq0zDIMrrriChIQEVq9eTXZ2NgEBAWitUUpht9uJj49n+PDh9OzZs6nDbfEkERKtUrGtmL/9+DeWZC/B6rTi1E7KHefusWHXdopsRQB8vf9rVhxcwYT4Cdx/wf2EmkP9HbYQQlTq2bMnPXv2pKCggJycHMrLywkMDCQ2NtYrE6OFiyRCotXZe2ovc5bModBWiNVh9fg+p3ZS5ijjq+yv+P7w97w54U16tpO/toQQTaeoeDdHjrzF8eNLcDhLMRnBaD0BU8D1hIX2aurwWgVZNSZalb2n9jLzy5nkl+XXKwmqqtxRTn5ZPjO/nMneU3KisxDC/7R2snPnI6xbN5kjOQux2U/hdJZjs5/iSM5C1q2bzM6dj6C1s6lDbfEkERKtRrGtmDlL5lBsK0bTuNWQGn1Ge0II4U+7dj3KkZyFOJ1lgOOsqw6czjKO5Cxk165HmyK8VkUSIdFq/O3Hv1FkK2p0ElRBoymyFfHEj094pT0hhPBEUfFujuR8hNNZ+z4+TmcpR3I+orh4j58ia50kERKtwta8rSzJXlLthOjGKHeU81X2V2zL2+bVdoUQoiYHD7yF01n3HkIATqeNAwff8noMo0ePrvHsMYCMjAweeeQRr/fbFCQREq1CxpYMrM6GzQmqi9VpJWNrhk/aFkKIsx0/voRzH4fVxMHx3K8a1M+8efNITk4mJSWF6667rkFttAayaky0eIXWQpYfXI7TR5MGndrJsgPLKLQWyqaLQgifc9TxSKyx9QG2bt3K448/zpo1a4iJiSE/P7/ebbQWkgiJFm/zic2YDbPXH4tVZTbMbDmxhaFxQ33WhxBCAJiMYJxOz3+emYzgevexbNkypk2bRkxMDABRUVG89dZbvPDCCwDs2bOHyy67DIvFQo8ePVi0aBF5eXmMGzcOgPz8fKxWK59++ikA77zzDklJSfWOozmQREi0eFtPbKXMXubTPsocZWzL2yaJkBDC5zp0mMCRnIV49ngsgA4dJ3ql3+uvv57rr78ecM0RysjIID4+vvJ6dHQ0WVlZgGuOUHZ2dquYJyRzhESLt+vkrjOOzfAFu9POjvwdPu1DCCEAunW/HsPw7AR6wwige7fr693H2LFj+eijj8jLywOQR2OeUkq1B7pprTf5KB4h6q3M4dvRoAq+fPRWX9rhwLp3L+XZ2WirFWWxEBgfj6VnT5TJ1NThCSEaISy0F3Gx09z7CNU8/8cwgomLvYbQ0PPr3UdCQgIPPvggo0aNwmQyMWDAADIyMhoRdctVZyKklFoBXOmuux7IVUqt1lrf6ePYhPBIkCnIL/0EmgL90k9NtMNB0apV5L3xJmUbN6LMZjAM0BqUAqcTbbMRlJJC9A1zCBs5UpIiIVqo3r3/BODeT8jGmY/JTBiGmbjYayrrNcTs2bOZPXt2tddWrFhR673p6ekN7re58WREKFJrfVopdSMwT2v9sFJKRoREs9G7fW+W7l/q08djAUYAfaP6+qz9upRkZnL4rrtxFBWii0sA0Lbq9xkpzczk8PZtmMLC6fLM04SkpfkzVCGEFyhl0KfPI3TpOpODBzM4nvtV5VljHTpOpHu36xs0EiTO5ckcoQClVCzwW+ALTxtWSr2plMpVSm2pUhallPpGKbXb/bm9u1wppV5USu1RSm1SSg2scs9sd/3dSqnZVcoHKaU2u+95USmlautDtF4JMQkEBfh2VCjIFET/6P4+7aM62uHg6GOPceDGudiPHatMguq8r7gE+7FjHLhxLkcfewzt8HRPEiFEcxIW2ot+ff/CyJHrGTN6GyNHrqdf379IEuRFniRCjwJLgD1a63VKqZ7Abg/uywDOnsp+H/Ct1roX8K37NcClQC/3x++Af4ArqQEeBi4AhgAPV0ls/gHMrXLfxDr6EK1UUkwSNg93YW0om9NGYkyiT/s4m3Y4OPQ/t3Pq40/QZQ2bB6XLyjj18Scc+p/bJRkSQohq1JkIaa0/0lona61vcb/eq7W+2oP7VgFnT0O/Cnjb/fXbwOQq5fO0yw9AO/co1ATgG611vtb6JPANMNF9LUJr/YPWWgPzzmqruj5EKxVuCWdMtzEYyjeLIA1lMLb7WL9vpnjsr3+lePXqBidBFXRZGcWrV3Psr3/1UmRCCH85Wm7jyb05XLB2G4nfb+GCtdt4cm8OR8t9+8dfW+LJZOkOuEZe4qvW11rPaUB/nbTWOe6vjwKd3F93AQ5WqXfIXVZb+aFqymvr4xxKqd/hGoGie/fu9X0vohlJT0xnxcEVPllBZjEspCeke73d2pSsW9eokaCz6bIyTi38mIhLL5U5Q0K0AFprXjxwjGezjwFQ7nQdJn3CBq8ezOXVg7ncGd+J/+neCffMENFAnvwJ/RkQCSwFFlf5aBT3SI53jglvYB9a639prdO01mkdOnTwZSjCxxKiE5gQP8HrK7sCTYFMjJ/o1/lB2uHg8N33eC0Jqmy3vJzDd90tj8iEaAFePHCM57NzKXfqyiSoQkXZ89m5vHjgWBNF2Hp4kgiFaK3v1Vp/qLX+uOKjgf0dcz/Wwv05111+GOhWpV5Xd1lt5V2rKa+tD9HK3X/B/YSZw1B4568jhSLMHMb9F9zvlfY8VbRqFY6iQp+07SwspOi773zSthDCO46W23g2+xilztrPTyx1Onk2+1iDHpMNGzasoeG1Op4kQl8opS7zUn+fAxUrv2bjGm2qKJ/lXj12IVDgfry1BLhEKdXePUn6EmCJ+9pppdSF7tVis85qq7o+RCsXag7lzQlvEmoObXQypFCV7YWYQ7wUoWfy3njT49Vh9eUsKSH/9Td80rYQwjsyDp+oV/159awPsGbNmnrf01p5kgjdjisZKlNKFbo/Ttd1k1LqfWAt0EcpdUgpdQPwBHCxUmo3MN79GuBLYC+wB/g3UDExOx94DFjn/njUXYa7zuvue34B/uMur6kP0Qb0bNeT+ZfNJyooqsGPyQJNgUQFRTH/svn0bNfTyxHWTjsclG3c6NM+SjdulMdjQjRji46dPOdxWE3KnZpPck/Wu4+wsDDANRfpnnvuITExkaSkJBYsWABATk4OI0eOJDU1lcTERL5rxSPJdU6W1lo3aKmM1npGDZfGVVNXA7fW0M6bwJvVlGcC56xn1lrnVdeHaDt6tuvJ4qmLeeLHJ/gq+yusTitOXfsQM7hWh1kMCxPjJ3L/Bff7fSQIwLp3L8psrnGzRG9QZjPWffsIPF/2IRGiOSp21P3zqqoie/3qV/XJJ5+QlZXFxo0bOXHiBIMHD2bkyJG89957TJgwgQcffBCHw0FJiW9GqZsDj84aU0pdCYx0v1yhtfZ4Y0UhmkKoOZTHLnqMGf1mkLE1g2UHlmE2zJQ5yrA7f92BOsAIIMgUhM1pY2z3saQnpDfJxokVyrOzXcdm+JJhYM3OlkRIiGYq1GRwoh5/C4UFNPxnxvfff8+MGTMwmUx06tSJUaNGsW7dOgYPHsycOXOw2WxMnjyZ1NTUBvfR3HmyfP4JYDAw3110u1JquNbavzNIhWiA/tH9+fvIv1NoLWTLiS1sy9vGjvwdlDvKCTQF0jeqL/2j+5MYk+j3fYKqo61W19lhPu1E4yxvPgfICiHONLVTe149mOvR47FAQzG1o/cPUBg5ciSrVq1i8eLFpKenc+eddzJr1iyv99MceDIidBmQqrXr2YJS6m3gZ0ASIdFihFvCGRo3lKFxQ5s6lFopi8V1gKpPO1EYgU17gKwQomazu8Tw6kEPFzxrmNUlpsF9jRgxgn/+85/Mnj2b/Px8Vq1axVNPPcX+/fvp2rUrc+fOpby8nA0bNrTpRAigHb/uEh3po1iEaPMC4+OhjiWzjeZ0YomP920fQogG6xxo5s74TjyfnVvrEvpgw+CO+I50DjQ3uK8pU6awdu1aUlJSUErx97//nc6dO/P222/z1FNPYTabCQsLY968eQ3uo7lTuo5heKXUDFwrr5YDCtdcofu01gt8H57/pKWl6czMzKYOQ7Rx2uFgZ+oAn0+W7pP1M8pk8lkfQoiabd++nX79+tVap6adpcH1OAwNd/aQnaUrVPc9VUqt11rXuZW+J6vG3ldKrcA1TwjgXq310YYEKoSonTKZCEpJodSHSXlwSookQUI0c0opbv9NZ6Z3jmbe4RN8knuSIruTsACDqR3bM6tLTKNGgsSvakyElFJ9tdY7lFID3UUVZ3vFKaXitNYbfB+eEG1P9A1zOLx9m082VTRCQoi68QavtyuE8I3OgWb+0DOWP/SMbepQWq3aRoTuxHUg6TPVXNPAWJ9EJEQbFzZyJKawcOy+SITCwwkbMcLr7Qoh6kdrLY+0vKSuKT51qTER0lr/zv3lpVrrM05/VEoFNapXIUSNlMlEl2ee5sCNc7168KoKDKTLM0/LYzEhmlhQUBB5eXlER0dLMtRIWmvy8vIICmp4WuLJqrE1wEAPyoQQXhKSlka7q6dy6uNPvJIMqaAg2l09lZC0OucNCiF8rGvXrhw6dIjjx483dSitQlBQEF27dq27Yg1qmyPUGegCBCulBkDlKZYRgP/PHhCijen0wAPYjh6jePXqRiVDKiiI0IsuotMDD3gxOiFEQ5nNZnr06NHUYQi32kaEJgDpQFfg2SrlhYD8RBXCx5TJRNcXX+DYX//a4JEhFRhIu6un0umBB+SRmBBCVMOTfYSu1lp/7Kd4mozsIySas5LMTA7fdTfOwkKcHhx+aISEYISH0+WZp+VxmBCiTfLmPkIfK6UmAQlAUJXyRxsXohDCUyFpaZy/7FuKvvuO/NffoHTjRpTZ7DqgVWvXsRxOJ9pmIzglhagbbyBsxAgZBRJCiDp4cujqa7jmBI0BXgeuAX7ycVxCiLMok4nw0aMJHz0a7XBg3bcPa3Y2zvJyjMBALPHxWHr0kORHCCHqwZNVY8O01slKqU1a6z8rpZ4B/uPrwIQQNVMmE4Hnn0/g+ec3dShCCNGiGR7UKXV/LlFKxQE2QLa4FEIIIUSL58mI0BdKqXbAU8AGXLtKv+7TqIQQQggh/MCTydKPub/8WCn1BRCktS7wbVhCCCGEEL5X24aKY7XWy5RSU6u5htb6E9+GJoQQQgjhW7WNCI0ClgFXVHNNA5IICSGEEKJFq+3Q1Yfdn6/3XzhCCCGEEP5T56oxpdRf3ZOlK163V0o97tuwhBBCCCF8z5Pl85dqrU9VvNBanwQu811IQgghhBD+4UkiZFJKBVa8UEoFA4G11BdCCCGEaBE82UdoPvCtUuot9+vrgbd9F5IQQgghhH94so/Qk0qpjcB4d9FjWuslvg1LCCGEEML3PBkRAvgZMONaNv+z78IRQgghhPAfT1aN/RbXafPXAL8FflRKXePrwIQQQgghfM2TEaEHgcFa61wApVQHYCmw0JeBCSGEEEL4mieJkFGRBLnl4dlqMyGEqJPWmtMnyjh5tBiHzYnJbNC+cygRMUEopZo6PCFEK+dJIvSVUmoJ8L779XTgS9+FJIRoC3L3nyZr6UH2bTwOgGEoNKAAp1MD0COlA6nju9HxNxFNF6gQolXzZNXYPe6DVy9yF/1La73It2EJIVqrktNWlmZsI2fPKRw2J1rXXHdP5jH2bTxO7PntGJ/en5AIi/8CFUK0CZ5Mlg4FPtNa3wn8E3Aopcw+j0wI0eoc3nWSd/+0lsM7T2K31p4EAWgNdquTwzvd9+066Z9AhRBthidzfVYBgUqpLsBXwHVAhi+DEkK0Pod3neSLlzdiK3PgdNSRAZ3F6dDYyhx88fJGSYaEEF7lSSKktNYlwFTgH1rraUCCb8MSQrQmJaetLH51E3ars1Ht2K1Ovnx1EyWnrV6KTAjR1nmUCCmlhgIzgcXuMpPvQhJCtDZLM7bhsDUuCapgtzn59u1tXmlLCCE8SYTuAO4HFmmttyqlegLLfRuWEKK1yN1/mpw9p+r9OKwmTofmyO5T5O4/7ZX2hBBtW52JkNZ6pdb6SveZYwZwQmv9P36ITQjRCmQtPei10aAKDruTrKUHvdqmEKJt8mTV2HtKqQj36rEtwDal1D2+D00I0dJprdm38Xidq8Pq3a4Td7teblgI0eZ48misv9b6NDAZ+A/QA9fKMSGEqNXpE2U+bb8wz7ftCyFaP08SIbN736DJwOdaaxuuU+iFEKJWJ48WYxi+OSbDMBT5OcU+aVsI0XZ4kgj9E8gGQoFVSqnfAI2apaiUul0ptUUptVUpdYe7LEop9Y1Sarf7c3t3uVJKvaiU2qOU2qSUGlilndnu+ruVUrOrlA9SSm123/OikgOLhGgSDpvTZ381aXf7QgjRGJ5Mln5Ra91Fa32ZdtkPjGloh0qpRGAuMARIAS5XSp0P3Ad8q7XuBXzrfg1wKdDL/fE74B/udqKAh4EL3G09XJE8uevMrXLfxIbGK4RoOJPZwFd/hSh3+0II0RieHLqKUmoSrk0Ug6oUP9rAPvsBP7o3aUQptRLXZo1XAaPddd4GVgD3usvnadesyB+UUu2UUrHuut9orfPd7XwDTFRKrQAitNY/uMvn8ev8JiGEH7XvHFp5gKq3OZ2aqNhQn7QthGg7PFk19hquE+d/j+uPsGnAbxrR5xZghFIqWikVAlwGdAM6aa1z3HWOAp3cX3cBqq6TPeQuq638UDXlQgg/i4gJqrtSI4RH+7Z9IUTr58m48jCt9SzgpNb6z8BQoHdDO9RabweeBL7GdXZZFuA4q47GDxOylVK/U0plKqUyjx8/7uvuhGhzlFL0SOmAt2fpKQN3uzL9TwjROJ4kQqXuzyVKqTjABsQ2plOt9Rta60Fa65HASWAXcMz9yAv351x39cO4RowqdHWX1VbetZry6uL4l9Y6TWud1qFDh8a8JSFEDVLHd/P6XB5TgMGAi7t7tU0hRNvkyU+nL5RS7YCngA24VpC915hOlVId3Z+745of9B7wOVCx8ms28Jn768+BWe7VYxcCBe5HaEuAS5RS7d2TpC8BlrivnVZKXeheLTarSltCCD/r+JsIYs9vh2HyzuiNYVLE9WpHh+7hXmlPCNG21TlZWmv9mPvLj5VSXwBBWuuCRvb7sVIqGtfo0q1a61NKqSeAD5VSNwD7gd+6636Jax7RHqAEuN4dV75S6jFgnbveoxUTp4FbgAwgGNckaZkoLUQTGp/en3f/tBanw1F35ToEmA3Gze7vhaiEEAKUbFHvkpaWpjMzM5s6DCFarcO7TvLFyxuxWxu+90+AxeDy21Lo0rt93ZWFEG2aUmq91jqtrnqyCYcQwi+69G7P5belYA4y1fsxmWFSWIJMkgQJIbxOEiEhhN906d2eax8dSpc+7QmwGHWuJlOGaxSoa9/2zHx0qCRBQgiv83RDxS649g6qrK+1XuWroIQQrVdIhIUr/yeV3P2nyVp6kH0bXVtXGIZC49qsrGITxh4pHUgd342Ov4louoCFEK1anYmQUupJXBsqbuPX/X40IImQEKLBOv4mgktuSEBrTWFeGfk5xThsTkxmg6jYUMKjg2SfICGEzxxQvj8AAB0qSURBVHkyIjQZ6KO1Lvd1MEKItkcpRURMMBExwU0dihCiDfJkjtBewOzrQIQQQggh/M2TEaESIEsp9S1QOSqktf4fn0UlhBBCCOEHniRCn7s/hBBCCCFaFU92ln5bKWXh14NWd2qtbb4NSwghhBDC9zxZNTYaeBvXGWMK6KaUmi3L54UQQgjR0nnyaOwZ4BKt9U4ApVRv4H1gkC8DE0IIIYTwtf/f3v3H2VXXdx5/fSYzmYRAQoQQQ36QpCZY0SgyBTQoW1GMomJR/F1o1pVusYpdd6t2K1ipa/vYSsXdlpaHv6BVKSAqC0iK2KrgA0oQBUGENOSn+YXQCYRkJpP72T/uGRhIMrmZzLn3Ts7r+Xjcx5zzveec+5nHheSd749zGlk11jUYggAy8yFcRSZJkg4CjfQILY+ILwL/WOy/F/DppJIkacxrJAj9AfBBYHC5/I+Avy2tIkmSpCZpZNVYH3BJ8ZIkSTpo7DUIRcTVmfmOiLiP+rPFniUzF5VamSRJUsmG6xG6oPj5pmYUIkmS1Gx7XTWWmRuKzfMzc/XQF3B+c8qTJEkqTyPL51+3h7Y3jHYhkiRJzTbcHKE/oN7z8xsRce+Qtw4Dbi+7MEmSpLINN0fo68B3gc8CHx/S/kRmPlZqVZIkSU0w3Byh3sxcBfwpsLGYGzQPeF9EHN6k+iRJkkrTyByhbwK7IuIFwOXAbOq9RZIkSWNaI0GolpkDwFnA/8nM/wHMKLcsSZKk8jUShHZGxLuBc4AbijYfuipJksa8RoLQUuAVwGcy85GImAf8Q7llSZIkla+RZ409EBEfA+YU+48Af1l2YZIkSWXbZ49QRLwZ+Clwc7H/soi4vuzCJEmSytbI0NingBOB/wDIzJ8C80usSZIkqSkamiydmb3PaauVUYwkSVIz7XOOEHB/RLwHGBcRC4APAz8utyxJkqTyNdIj9CHgOKAP+AawFfhImUVJkiQ1QyOrxp4C/mfxkiRJOmjsMwhFxL8A+dz2zHxNKRVJkiQ1SSNzhP77kO0JwNuAgXLKkSRJap5Ghsbufk7T7RHxbyXVI0mS1DSNDI09b8huB3ACMKW0iiRJkpqkkaGxu6nPEQrqQ2KPAO8vsyhJkqRmaGRobF4zCpEkSWq2RobGzhru/cy8bvTKkSRJap5GhsbeD7wS+H6x/9vU7yy9hfqQmUFIkiSNSY0EoS7gRZm5ASAiZgBfzcylpVYmSZJUskYesTF7MAQVNgFzSqpHkiSpaRrpEbo1IpZRf84YwDuB7x3Ih0bEHwH/hfrQ2n3AUmAGcBVwBPWVar+bmf0R0Q1cSX3Z/q+Bd2bmquI6n6A+dLcL+HBmLivalwCXAuOAL2bmXxxIvZI0nJ07dvDQnbez5uc/Y9PKFQz099E5vpvp81/AnBe/lIUnLaZrwoRWlylpDyJzt6dn7H5QxO8Ary52f5iZ3xrxB0bMBG6jPty2PSKuBm4C3ghcl5lXRcTfAT/LzMsi4nxgUWb+14h4F/A7mfnOiHgR9XB2InA09XC2sPiYh4DXAeuAu4B3Z+YDw9XV09OTy5cvH+mvJamCBvr7+fE1X+Oem28gOoKdO3bsdkzXhAlkLTl+yZt45dnvpXP8+BZUKlVPRNydmT37Oq6RHiGK4DPi8LOXz50YETuBQ4ANwGuA9xTvXwF8CrgMOLPYBrgW+L8REUX7VZnZBzwSESuohyKAFZm5EiAiriqOHTYISdL++PW6tXzzsxeyfetWBvr79nrcYDi65+YbePDHP+Rtn/g0R8ya3awyJe1DI3OERlVmrgf+ClhDPQD1Uh8K+4/MHHyG2TpgZrE9E1hbnDtQHH/E0PbnnLO39t1ExHkRsTwilm/ZsuXAfzlJlfDrdWv5+ic/yhOPPjpsCBpqoL+PJ379KF//5Ef59bq1+z5BUlM0PQhFxFTqPTTzqA9pTQKWNLsOgMy8PDN7MrNn2rRprShB0hgz0N/PNz97If1Pbac+zXE/ZNL/1Hau++xFDPT3l1KfpP2z1yAUEbcWP/9ylD/ztcAjmbklM3dSvw/RYuDwiBgcqpsFrC+21wOzi1o6qT/n7NdD259zzt7aJemA/fiar7F961b2OwQ9LXlqay8/vuZro1mWpBEarkdoRkS8EnhLRBwfES8f+jqAz1wDnBwRhxRzfU6jPn/nX4C3F8ecC3yn2L6+2Kd4//tZn+F9PfCuiOiOiHnAAuDfqE+OXhAR8yJiPPCu4lhJOiA7d+zgnptvaHg4bG8G+vu4Z9kNe5xcLam5hpssfSHwSeo9Kpc8572kPrl5v2XmnRFxLfAT6g9xvQe4HLgRuCoi/rxo+1JxypeAfygmQz9GPdiQmfcXK84eKK7zwczcBRARfwgso758/suZef9IapWkoR6683aiI0blWhHBQ3feznGnnjYq15M0MvtcPh8Rn8zMi5tUT8u4fF7Svnz3by7hgR9+f98HNui4U09jyfl/NGrXk/SMUVs+n5kXR8RbeOY+Qv+amTccaIGSNNZsWrliVK+3cZSvJ2n/7XPVWER8FriA+hDUA8AFEfG/yi5MktrNgc4N2u16fc4RklqtkRsqngG8LDNrABFxBfU5PH9SZmGS1G46x3eP7vW6feyG1GqN3kfo8CHbU8ooRJLa3fT5LxjV6z1/lK8naf81EoQ+C9wTEV8teoPuBj5TblmS1H7mvPilo/bw1K4JE5h93KJRuZakkdtnEMrMbwAnU7/x4TeBV2TmP5VdmCS1m4UnLSZrI72R4rNlJgtPWjwq15I0cg0NjWXmhsy8vnhtLLsoSWpHXRMmcPySNx3wXKHO8d0c//o3jVrvkqSRa/qzxiRpLHvl2e9l4uTJwAhvrBjBIZOn8Mp3vG9U65I0MgYhSdoPnePH87ZPfJrxh0xkv8NQBOMnTuSsT/wZnV1dpdQnaf8MG4QiYlxEPNisYiRpLDhi1mzec/HnOOzIIxseJusc383kI6bxnos/xxGzZu/7BElNMWwQKp7d9cuImNOkeiRpTDhi1mz+81//fX3OUHf3Xuf7dE2YQGd3N8cveRNL//rvDEFSm2nkhopTgfsj4t+AbYONmfmW0qqSpDGgc/x4Xv3epbzibe/moTtvZ83997Jp5QoG+nbQ2T2B6fNfwJzjFrHwpMVOjJbaVCNB6JOlVyFJY1jXhAkcd+ppPkleGoMaeejqDyLiGGBBZn4vIg4BxpVfmiRJUrkaeejqB4Brgb8vmmYC3y6zKEmSpGZoZPn8B4HFwFaAzHwYOKrMoiRJkpqhkSDUl5n9gzsR0QmMzj3mJUmSWqiRIPSDiPgTYGJEvA64Bvh/5ZYlSZJUvkaC0MeBLcB9wO8DNwF/WmZRkiRJzdDIqrFaRFwB3El9SOyXmenQmCRJGvP2GYQi4gzg74B/p/5gnXkR8fuZ+d2yi5MkSSpTIzdU/Bzw25m5AiAifgO4ETAISZKkMa2ROUJPDIagwkrgiZLqkSRJapq99ghFxFnF5vKIuAm4mvocobOBu5pQmyRJUqmGGxp785DtTcCpxfYWYGJpFUmSJDXJXoNQZi5tZiGSJEnN1siqsXnAh4C5Q4/PzLeUV5YkSVL5Glk19m3gS9TvJl0rtxxJkqTmaSQI7cjML5ReiSRJUpM1EoQujYiLgH8G+gYbM/MnpVUlSZLUBI0EoZcAvwu8hmeGxrLYlyRJGrMaCUJnA/Mzs7/sYiRJkpqpkTtL/xw4vOxCJEmSmq2RHqHDgQcj4i6ePUfI5fOSJGlMayQIXVR6FZIkSS2wzyCUmT9oRiGSJEnN1sidpZ+gvkoMYDzQBWzLzMllFiZJklS2RnqEDhvcjogAzgROLrMoSZKkZmhk1djTsu7bwOtLqkeSJKlpGhkaO2vIbgfQA+worSJJkqQmaWTV2JuHbA8Aq6gPj0mSJI1pjcwRWtqMQiRJkpptr0EoIi4c5rzMzItH8oERcSzwT0Oa5gMXAlcW7XOp9zq9IzMfLyZoXwq8EXgK+L3BB75GxLnAnxbX+fPMvKJoPwH4KjARuAm4IDMHV75JkiQBw0+W3raHF8D7gY+N9AMz85eZ+bLMfBlwAvVw8y3g48CtmbkAuLXYB3gDsKB4nQdcBhARz6N+s8eTgBOBiyJianHOZcAHhpy3ZKT1SpKkg9deg1Bmfm7wBVxOvXdlKXAV9V6c0XAa8O+ZuZr6vKMrivYrgLcW22cCVxYr1u4ADo+IGdRXrt2SmY9l5uPALcCS4r3JmXlH0Qt05ZBrSZIkPW3Y5fMR8byI+HPgXurDaC/PzI9l5uZR+vx3Ad8otqdn5oZieyMwvdieCawdcs66om249nV7aN9NRJwXEcsjYvmWLVsO5PeQJElj0F6DUET8b+Au4AngJZn5qaLnZVRExHjgLcA1z32v6MkpfU5PZl6emT2Z2TNt2rSyP06SJLWZ4XqEPgocTX0y8q8iYmvxeiIito7CZ78B+Elmbir2NxXDWhQ/B3ud1gOzh5w3q2gbrn3WHtolSZKeZbg5Qh2ZOTEzD8vMyUNeh43Sc8bezTPDYgDXA+cW2+cC3xnSfk7UnQz0FkNoy4DTI2JqMUn6dGBZ8d7WiDi5WHF2zpBrSZIkPa2RGyqOuoiYBLwO+P0hzX8BXB0R7wdWA+8o2m+ivnR+BfUVZksBMvOxiLiY+vAdwKcz87Fi+3yeWT7/3eIlSZL0LOHtdep6enpy+fLlrS5DkiSNgoi4OzN79nXcfj10VZIk6WBiEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIk7VHuSmrbB8hd2epSpNJ0troASVL7qG0fYNvdm3jyR+vYtbUfOgJqybjJ4zn0VbOYdMJ0Oib6V4cOHv7XLEkiM9n6vTU88YN1REDurNXfKHqDdvX2s3XZKnpvXsVhp85i8mvnEBEtrFgaHQYhSaq4zOTxqx9i+88fhYEaexsIGwxHT/5wHbse38HUsxcahjTmOUdIkipu6/fWsP3njz7TC7QPubPG9vseZev31pRcmVQ+g5AkVVht+wBP/GBdwyFoUO6s8cQP1lHbMVBSZVJzGIQkqcK23b2JkY5uRcC25ZtGtyCpyQxCklRhT/5o/3uDBuXOGk/+aN0oVyQ1l0FIkioqd9XqS+QPwK6t/d5nSGOaQUiSKir7a/X7BB2IjiD7d41OQVILGIQkqaJifAfUDrA3p5bE+HGjU5DUAgYhSaqoGNfBuMnjD+ga4yaPJ8Z5LyGNXQYhSaqwQ181i+ga2V8F0dXBoa+aNcoVSc1lEJKkCpt0wnRyhKNjmTCpZ/roFiQ1mUFIkiqsY2Inh526/71C0dXBYafOomOCT2rS2GYQkqSKm/zaOUx8yZENh6Ho6mDiS45k8mvnlFyZVD6jvCRVXEQw9eyFjJs6Yfenzw89rquDTDj01T59XgcPg5AkiYhgyuuO4bBTZrLt7k08+aN19ZstdgTUknGTx3Poq2YxqWe6w2E6qPhfsyTpaR0TOznslJkcdspMcleS/buI8eNcIq+DlkFIkrRHMS6Iif41oYObk6UlSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJltSQIRcThEXFtRDwYEb+IiFdExPMi4paIeLj4ObU4NiLiCxGxIiLujYiXD7nOucXxD0fEuUPaT4iI+4pzvhA+GVCSJO1Bq3qELgVuzswXAi8FfgF8HLg1MxcAtxb7AG8AFhSv84DLACLiecBFwEnAicBFg+GpOOYDQ85b0oTfSZLUhnp7e3nwwQf52c9+xoMPPkhvb2+rS1IbafpDZCJiCvBq4PcAMrMf6I+IM4H/VBx2BfCvwMeAM4ErMzOBO4repBnFsbdk5mPFdW8BlkTEvwKTM/OOov1K4K3Ad5vw60mS2kBm8sgjj3DbbbexevVqOjs7yUwigoGBAY455hhOOeUU5s+f3+pS1WKteJrePGAL8JWIeClwN3ABMD0zNxTHbASmF9szgbVDzl9XtA3Xvm4P7buJiPOo9zIxZ86ckf9GkqS2UavVuPHGG7n33nvZuXMnALt27XrWMStXrmTt2rUsWrSIM844g44Op8xWVSu++U7g5cBlmXk8sI1nhsEAKHp/suxCMvPyzOzJzJ5p06aV/XGSpJJl5m4haG927tzJvffey4033tik6tSOWhGE1gHrMvPOYv9a6sFoUzHkRfFzc/H+emD2kPNnFW3Dtc/aQ7sk6SD3yCOPNBSCBg2GoZUrV5ZcmdpV04NQZm4E1kbEsUXTacADwPXA4Mqvc4HvFNvXA+cUq8dOBnqLIbRlwOkRMbWYJH06sKx4b2tEnFysFjtnyLUkSQex2267reEQNGjnzp3cfvvtJVWkdteKOUIAHwK+FhHjgZXAUuqh7OqIeD+wGnhHcexNwBuBFcBTxbFk5mMRcTFwV3HcpwcnTgPnA18FJlKfJO1EaUk6yPX29rJ69eoRnbtq1Sp6e3uZMmXKKFeldteSIJSZPwV69vDWaXs4NoEP7uU6Xwa+vIf25cCLD7BMSdIYsmHDBjo7O3ebGN2Izs5ONmzYYBCqIKfJS5IOCn19fdT/7bz/MpO+vr5RrkhjgUFIknRQ6O7uZqQPEogIuru7R7kijQUGIUnSQWHGjBkMDAyM6NyBgQFmzJgxyhVpLDAISZIOClOmTOGYY44Z0blz5851flBFGYQkSQeNU045ha6urv06p6uri8WLF5dUkdqdQUiSdNCYN28eixYtajgMdXV1sWjRIp85VmEGIUnSQSMiOOOMMxoKQ4Mh6IwzzmhSdWpHrbqhoiRJpejo6ODNb34zxx13HLfffjurVq3a7enzc+fOZfHixfYEySAkSTo4zZ8/n/nz59Pb28uGDRvo6+uju7ubGTNmODFaTzMISZIOalOmTDH4aK+cIyRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkiqrJUEoIlZFxH0R8dOIWF60PS8ibomIh4ufU4v2iIgvRMSKiLg3Il4+5DrnFsc/HBHnDmk/obj+iuLcaP5vKUmS2l0re4R+OzNflpk9xf7HgVszcwFwa7EP8AZgQfE6D7gM6sEJuAg4CTgRuGgwPBXHfGDIeUvK/3UkSdJY005DY2cCVxTbVwBvHdJ+ZdbdARweETOA1wO3ZOZjmfk4cAuwpHhvcmbekZkJXDnkWpIkSU9rVRBK4J8j4u6IOK9om56ZG4rtjcD0YnsmsHbIueuKtuHa1+2hXZIk6Vk6W/S5p2Tm+og4CrglIh4c+mZmZkRk2UUUIew8gDlz5pT9cZIkqc20pEcoM9cXPzcD36I+x2dTMaxF8XNzcfh6YPaQ02cVbcO1z9pD+57quDwzezKzZ9q0aQf6a0mSpDGm6UEoIiZFxGGD28DpwM+B64HBlV/nAt8ptq8HzilWj50M9BZDaMuA0yNiajFJ+nRgWfHe1og4uVgtds6Qa0mSJD2tFUNj04FvFSvaO4GvZ+bNEXEXcHVEvB9YDbyjOP4m4I3ACuApYClAZj4WERcDdxXHfTozHyu2zwe+CkwEvlu8JEmSniXqC6vU09OTy5cvb3UZkiRpFETE3UNu0bNX7bR8XpIkqalatWpMkiTthye3PczaNV9hy5Zl7KptZ1zHRKZNez2z5yzl0EkLWl3emGUQkiSpjWXWeOihT/OrDddQq+0EdgFQq/Xxqw3XsnHTdzh6xtksXHghEQ707C+DkCRJbawegq6lVtuxh3d3Uavt4lcbrgXg2GM/1dTaDgYGIUmS2tST2x4ueoL2FIKeUatt51cbrmHWrPcxadILmlTdyGzs28kV6x/luk2Ps21XjUnjOjhr+lTOnXkkz+/uano99qFJktSm1q75SjEctm+12k7WrP1KyRWNXGZy6eqNnHTHA/zt2s2s3tHPozsHWL2jn79du5mT7niAS1dvpNmr2Q1CkiS1qS1bljE4J2jfdrFl881llnNAvrBmE59ftZm+WtJXe3bYGWz7/KrNfGHNpqbWZRCSJKlN7aptL/X4ZtnYt5NLVm1ie6027HHbazUuWbWJjX2N9YKNBoOQJEltalzHxFKPb5avrn90v46/cj+PPxAGIUmS2tS0aa8HxjV4dCfTjlpSZjkj9q1Nj+82HLY3fbXkus2Pl1zRMwxCkiS1qdlzltLR0dhKqo6OTubMXlpyRSOzbdfwQ2LP9eTA/h1/IAxCkiS1qUMnLeDoGWfTsY8hr46OiRw94+y2XTo/adz+xY1DO5sXTwxCkiS1sYULL+ToGW+no2MCuw+TjaOjYwJHz3g7Cxde2IryGnLW9Kl0d0RDx3Z3BGcdNbXkip5hEJIkqY1FdHDssZ/it37r2xx99Nl0dR5OR0c3XZ2Hc/TRZ3Pib32HY4/9VFs/XuPcmUc2fnDCOftz/AHyztKSJI0Bh05awG++8DP85gs/0+pS9tvzu7v4b3On8/lVm4ddQj+xo4OPzD2qqXeYNghJkqTSfXjOdAAuWVW/YeLQVWTdHQEJH5l71NPHNYtBSJIklS4iuOCY5/PO5x/Blesf5brNj/PkQI1DOzs466ipnNOiZ40ZhCRJUtM8v7uLP54/gz+eP6PVpQBOlpYkSRVmEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZUVmdnqGtpCRGwBVpd0+SOBR0u6tkbG76Q9+b20H7+T9uT3sm/HZOa0fR1kEGqCiFiemT2trkPP8DtpT34v7cfvpD35vYweh8YkSVJlGYQkSVJlGYSa4/JWF6Dd+J20J7+X9uN30p78XkaJc4QkSVJl2SMkSZIqyyAkSZIqyyBUoohYEhG/jIgVEfHxVtcjiIjZEfEvEfFARNwfERe0uibVRcS4iLgnIm5odS2qi4jDI+LaiHgwIn4REa9odU1VFxF/VPzZ9fOI+EZETGh1TWOdQagkETEO+BvgDcCLgHdHxItaW5WAAeCjmfki4GTgg34vbeMC4BetLkLPcilwc2a+EHgpfj8tFREzgQ8DPZn5YmAc8K7WVjX2GYTKcyKwIjNXZmY/cBVwZotrqrzM3JCZPym2n6D+B/vM1laliJgFnAF8sdW1qC4ipgCvBr4EkJn9mfkfra1KQCcwMSI6gUOAX7W4njHPIFSemcDaIfvr8C/cthIRc4HjgTtbW4mAzwN/DNRaXYieNg/YAnylGLL8YkRManVRVZaZ64G/AtYAG4DezPzn1lY19hmEVEkRcSjwTeAjmbm11fVUWUS8CdicmXe3uhY9SyfwcuCyzDwe2AY417GFImIq9ZGFecDRwKSIeF9rqxr7DELlWQ/MHrI/q2hTi0VEF/UQ9LXMvK7V9YjFwFsiYhX1IeTXRMQ/trYkUe/FXpeZgz2m11IPRmqd1wKPZOaWzNwJXAe8ssU1jXkGofLcBSyIiHkRMZ76hLbrW1xT5UVEUJ/z8IvMvKTV9Qgy8xOZOSsz51L//+T7mem/clssMzcCayPi2KLpNOCBFpak+pDYyRFxSPFn2Wk4gf2Adba6gINVZg5ExB8Cy6jP7P9yZt7f4rJU7334XeC+iPhp0fYnmXlTC2uS2tWHgK8V/5hbCSxtcT2Vlpl3RsS1wE+or4C9Bx+1ccB8xIYkSaosh8YkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJl/X9mwyMsqFYkCQAAAABJRU5ErkJggg==\n", 92 | "text/plain": [ 93 | "
" 94 | ] 95 | }, 96 | "metadata": {}, 97 | "output_type": "display_data" 98 | } 99 | ], 100 | "source": [ 101 | "tagCount = collections.Counter(list(df_tags['Tag'])).most_common(10)\n", 102 | "print(tagCount)\n", 103 | "plot_tags(tagCount)" 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": 5, 109 | "metadata": {}, 110 | "outputs": [], 111 | "source": [ 112 | "top10=['javascript','java','c#','php','android','jquery','python','html','c++','ios']" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": 6, 118 | "metadata": {}, 119 | "outputs": [ 120 | { 121 | "name": "stdout", 122 | "output_type": "stream", 123 | "text": [ 124 | "(826739, 2)\n" 125 | ] 126 | }, 127 | { 128 | "data": { 129 | "text/html": [ 130 | "
\n", 131 | "\n", 144 | "\n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | "
IdTag
14260c#
18330c++
28650c#
35930c#
391010c#
\n", 180 | "
" 181 | ], 182 | "text/plain": [ 183 | " Id Tag\n", 184 | "14 260 c#\n", 185 | "18 330 c++\n", 186 | "28 650 c#\n", 187 | "35 930 c#\n", 188 | "39 1010 c#" 189 | ] 190 | }, 191 | "execution_count": 6, 192 | "metadata": {}, 193 | "output_type": "execute_result" 194 | } 195 | ], 196 | "source": [ 197 | "tag_top10= df_tags[df_tags.Tag.isin(top10)]\n", 198 | "print (tag_top10.shape)\n", 199 | "tag_top10.head()" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": 7, 205 | "metadata": {}, 206 | "outputs": [ 207 | { 208 | "data": { 209 | "text/plain": [ 210 | "30798790 5\n", 211 | "31085960 5\n", 212 | "11648170 5\n", 213 | "35318730 5\n", 214 | "4009250 5\n", 215 | "30289880 5\n", 216 | "23267320 5\n", 217 | "35283570 5\n", 218 | "30991580 5\n", 219 | "23484760 5\n", 220 | "Name: Id, dtype: int64" 221 | ] 222 | }, 223 | "execution_count": 7, 224 | "metadata": {}, 225 | "output_type": "execute_result" 226 | } 227 | ], 228 | "source": [ 229 | "tag_top10['Id'].value_counts().head(10)" 230 | ] 231 | }, 232 | { 233 | "cell_type": "code", 234 | "execution_count": 8, 235 | "metadata": {}, 236 | "outputs": [ 237 | { 238 | "data": { 239 | "text/html": [ 240 | "
\n", 241 | "\n", 254 | "\n", 255 | " \n", 256 | " \n", 257 | " \n", 258 | " \n", 259 | " \n", 260 | " \n", 261 | " \n", 262 | " \n", 263 | " \n", 264 | " \n", 265 | " \n", 266 | " \n", 267 | " \n", 268 | " \n", 269 | " \n", 270 | " \n", 271 | " \n", 272 | " \n", 273 | " \n", 274 | " \n", 275 | " \n", 276 | " \n", 277 | " \n", 278 | " \n", 279 | " \n", 280 | " \n", 281 | " \n", 282 | " \n", 283 | " \n", 284 | " \n", 285 | " \n", 286 | " \n", 287 | " \n", 288 | " \n", 289 | "
IdTag
14260c#
18330c++
28650c#
35930c#
391010c#
\n", 290 | "
" 291 | ], 292 | "text/plain": [ 293 | " Id Tag\n", 294 | "14 260 c#\n", 295 | "18 330 c++\n", 296 | "28 650 c#\n", 297 | "35 930 c#\n", 298 | "39 1010 c#" 299 | ] 300 | }, 301 | "execution_count": 8, 302 | "metadata": {}, 303 | "output_type": "execute_result" 304 | } 305 | ], 306 | "source": [ 307 | "tag_top10.head()" 308 | ] 309 | }, 310 | { 311 | "cell_type": "code", 312 | "execution_count": 9, 313 | "metadata": {}, 314 | "outputs": [], 315 | "source": [ 316 | "def add_tags(question_id):\n", 317 | " return tag_top10[tag_top10['Id'] == question_id['Id']].Tag.values\n", 318 | "\n", 319 | "top10 = tag_top10.apply(add_tags, axis=1)" 320 | ] 321 | }, 322 | { 323 | "cell_type": "code", 324 | "execution_count": 10, 325 | "metadata": {}, 326 | "outputs": [ 327 | { 328 | "data": { 329 | "text/plain": [ 330 | "(826739, (826739, 2))" 331 | ] 332 | }, 333 | "execution_count": 10, 334 | "metadata": {}, 335 | "output_type": "execute_result" 336 | } 337 | ], 338 | "source": [ 339 | "len(top10),tag_top10.shape" 340 | ] 341 | }, 342 | { 343 | "cell_type": "code", 344 | "execution_count": 11, 345 | "metadata": {}, 346 | "outputs": [ 347 | { 348 | "data": { 349 | "text/html": [ 350 | "
\n", 351 | "\n", 364 | "\n", 365 | " \n", 366 | " \n", 367 | " \n", 368 | " \n", 369 | " \n", 370 | " \n", 371 | " \n", 372 | " \n", 373 | " \n", 374 | " \n", 375 | " \n", 376 | " \n", 377 | " \n", 378 | " \n", 379 | " \n", 380 | " \n", 381 | " \n", 382 | " \n", 383 | " \n", 384 | " \n", 385 | " \n", 386 | " \n", 387 | " \n", 388 | " \n", 389 | " \n", 390 | " \n", 391 | " \n", 392 | " \n", 393 | " \n", 394 | " \n", 395 | " \n", 396 | " \n", 397 | " \n", 398 | " \n", 399 | " \n", 400 | " \n", 401 | " \n", 402 | " \n", 403 | " \n", 404 | " \n", 405 | "
IdTagTags
14260c#[c#]
18330c++[c++]
28650c#[c#]
35930c#[c#]
391010c#[c#]
\n", 406 | "
" 407 | ], 408 | "text/plain": [ 409 | " Id Tag Tags\n", 410 | "14 260 c# [c#]\n", 411 | "18 330 c++ [c++]\n", 412 | "28 650 c# [c#]\n", 413 | "35 930 c# [c#]\n", 414 | "39 1010 c# [c#]" 415 | ] 416 | }, 417 | "execution_count": 11, 418 | "metadata": {}, 419 | "output_type": "execute_result" 420 | } 421 | ], 422 | "source": [ 423 | "tag_top10=pd.concat([tag_top10, top10.rename('Tags')], axis=1)\n", 424 | "tag_top10.head()" 425 | ] 426 | }, 427 | { 428 | "cell_type": "code", 429 | "execution_count": 12, 430 | "metadata": {}, 431 | "outputs": [ 432 | { 433 | "data": { 434 | "text/plain": [ 435 | "(826739, 2)" 436 | ] 437 | }, 438 | "execution_count": 12, 439 | "metadata": {}, 440 | "output_type": "execute_result" 441 | } 442 | ], 443 | "source": [ 444 | "tag_top10.drop([\"Tag\"], axis=1, inplace=True)\n", 445 | "tag_top10.shape" 446 | ] 447 | }, 448 | { 449 | "cell_type": "code", 450 | "execution_count": 13, 451 | "metadata": {}, 452 | "outputs": [], 453 | "source": [ 454 | "top10_tags=tag_top10.loc[tag_top10.astype(str).drop_duplicates().index]" 455 | ] 456 | }, 457 | { 458 | "cell_type": "code", 459 | "execution_count": 14, 460 | "metadata": {}, 461 | "outputs": [ 462 | { 463 | "data": { 464 | "text/html": [ 465 | "
\n", 466 | "\n", 479 | "\n", 480 | " \n", 481 | " \n", 482 | " \n", 483 | " \n", 484 | " \n", 485 | " \n", 486 | " \n", 487 | " \n", 488 | " \n", 489 | " \n", 490 | " \n", 491 | " \n", 492 | " \n", 493 | " \n", 494 | " \n", 495 | " \n", 496 | " \n", 497 | " \n", 498 | " \n", 499 | " \n", 500 | " \n", 501 | " \n", 502 | " \n", 503 | " \n", 504 | " \n", 505 | " \n", 506 | " \n", 507 | " \n", 508 | " \n", 509 | " \n", 510 | " \n", 511 | " \n", 512 | " \n", 513 | " \n", 514 | " \n", 515 | " \n", 516 | " \n", 517 | " \n", 518 | " \n", 519 | " \n", 520 | "
IdTitleBody
080SQLStatement.execute() - multiple queries in o...I've written a database generation script in S...
190Good branching and merging tutorials for Torto...Are there any really good tutorials explaining...
2120ASP.NET Site MapsHas anyone got experience creating SQL-based A...
3180Function for creating color wheelsThis is something I've pseudo-solved many time...
4260Adding scripting functionality to .NET applica...I have a little game written in C#. It uses a ...
\n", 521 | "
" 522 | ], 523 | "text/plain": [ 524 | " Id ... Body\n", 525 | "0 80 ... I've written a database generation script in S...\n", 526 | "1 90 ... Are there any really good tutorials explaining...\n", 527 | "2 120 ... Has anyone got experience creating SQL-based A...\n", 528 | "3 180 ... This is something I've pseudo-solved many time...\n", 529 | "4 260 ... I have a little game written in C#. It uses a ...\n", 530 | "\n", 531 | "[5 rows x 3 columns]" 532 | ] 533 | }, 534 | "execution_count": 14, 535 | "metadata": {}, 536 | "output_type": "execute_result" 537 | } 538 | ], 539 | "source": [ 540 | "ques = pd.read_csv('../input/stackoverflow-clean-questions-file-v2/question_clean.csv', encoding='iso-8859-1')\n", 541 | "ques.head()" 542 | ] 543 | }, 544 | { 545 | "cell_type": "code", 546 | "execution_count": 15, 547 | "metadata": {}, 548 | "outputs": [ 549 | { 550 | "name": "stdout", 551 | "output_type": "stream", 552 | "text": [ 553 | "(706336, 4)\n" 554 | ] 555 | }, 556 | { 557 | "data": { 558 | "text/html": [ 559 | "
\n", 560 | "\n", 573 | "\n", 574 | " \n", 575 | " \n", 576 | " \n", 577 | " \n", 578 | " \n", 579 | " \n", 580 | " \n", 581 | " \n", 582 | " \n", 583 | " \n", 584 | " \n", 585 | " \n", 586 | " \n", 587 | " \n", 588 | " \n", 589 | " \n", 590 | " \n", 591 | " \n", 592 | " \n", 593 | " \n", 594 | " \n", 595 | " \n", 596 | " \n", 597 | " \n", 598 | " \n", 599 | " \n", 600 | " \n", 601 | " \n", 602 | " \n", 603 | " \n", 604 | " \n", 605 | " \n", 606 | " \n", 607 | " \n", 608 | " \n", 609 | " \n", 610 | " \n", 611 | " \n", 612 | " \n", 613 | " \n", 614 | " \n", 615 | " \n", 616 | " \n", 617 | " \n", 618 | " \n", 619 | " \n", 620 | "
IdTitleBodyTags
0260Adding scripting functionality to .NET applica...I have a little game written in C#. It uses a ...[c#]
1330Should I use nested classes in this case?I am working on a collection of classes used f...[c++]
2650Automatically update version numberI would like the version property of my applic...[c#]
3930How do I connect to a database and loop over a...What's the simplest way to connect and query a...[c#]
41010How to get the value of built, encoded ViewState?I need to grab the base64-encoded representati...[c#]
\n", 621 | "
" 622 | ], 623 | "text/plain": [ 624 | " Id ... Tags\n", 625 | "0 260 ... [c#]\n", 626 | "1 330 ... [c++]\n", 627 | "2 650 ... [c#]\n", 628 | "3 930 ... [c#]\n", 629 | "4 1010 ... [c#]\n", 630 | "\n", 631 | "[5 rows x 4 columns]" 632 | ] 633 | }, 634 | "execution_count": 15, 635 | "metadata": {}, 636 | "output_type": "execute_result" 637 | } 638 | ], 639 | "source": [ 640 | "total=pd.merge(ques, top10_tags, on='Id')\n", 641 | "print(total.shape)\n", 642 | "total.head()" 643 | ] 644 | }, 645 | { 646 | "cell_type": "code", 647 | "execution_count": 16, 648 | "metadata": {}, 649 | "outputs": [ 650 | { 651 | "name": "stderr", 652 | "output_type": "stream", 653 | "text": [ 654 | "Using TensorFlow backend.\n" 655 | ] 656 | } 657 | ], 658 | "source": [ 659 | "from sklearn.model_selection import train_test_split\n", 660 | "from sklearn.preprocessing import MultiLabelBinarizer\n", 661 | "from nltk import word_tokenize\n", 662 | "from keras.preprocessing.text import Tokenizer\n", 663 | "from keras.preprocessing import sequence\n", 664 | "from keras.layers import LSTM, Activation, Dense, Dropout, Input, Embedding, BatchNormalization, GRU ,concatenate\n", 665 | "from keras.models import Model" 666 | ] 667 | }, 668 | { 669 | "cell_type": "code", 670 | "execution_count": 17, 671 | "metadata": {}, 672 | "outputs": [ 673 | { 674 | "data": { 675 | "text/plain": [ 676 | "array(['android', 'c#', 'c++', 'html', 'ios', 'java', 'javascript',\n", 677 | " 'jquery', 'php', 'python'], dtype=object)" 678 | ] 679 | }, 680 | "execution_count": 17, 681 | "metadata": {}, 682 | "output_type": "execute_result" 683 | } 684 | ], 685 | "source": [ 686 | "multilabel_binarizer = MultiLabelBinarizer()\n", 687 | "multilabel_binarizer.fit(total.Tags)\n", 688 | "labels = multilabel_binarizer.classes_\n", 689 | "labels" 690 | ] 691 | }, 692 | { 693 | "cell_type": "code", 694 | "execution_count": 18, 695 | "metadata": {}, 696 | "outputs": [], 697 | "source": [ 698 | "train,test=train_test_split(total[:550000],test_size=0.25,random_state=24)" 699 | ] 700 | }, 701 | { 702 | "cell_type": "code", 703 | "execution_count": 19, 704 | "metadata": {}, 705 | "outputs": [ 706 | { 707 | "data": { 708 | "text/plain": [ 709 | "((412500, 4), (137500, 4))" 710 | ] 711 | }, 712 | "execution_count": 19, 713 | "metadata": {}, 714 | "output_type": "execute_result" 715 | } 716 | ], 717 | "source": [ 718 | "train.shape,test.shape" 719 | ] 720 | }, 721 | { 722 | "cell_type": "code", 723 | "execution_count": 20, 724 | "metadata": {}, 725 | "outputs": [], 726 | "source": [ 727 | "X_train_t=train['Title']\n", 728 | "X_train_b=train['Body']\n", 729 | "y_train=multilabel_binarizer.transform(train['Tags'])\n", 730 | "X_test_t=test['Title']\n", 731 | "X_test_b=test['Body']\n", 732 | "y_test=multilabel_binarizer.transform(test['Tags'])" 733 | ] 734 | }, 735 | { 736 | "cell_type": "code", 737 | "execution_count": 21, 738 | "metadata": {}, 739 | "outputs": [ 740 | { 741 | "data": { 742 | "text/plain": [ 743 | "59" 744 | ] 745 | }, 746 | "execution_count": 21, 747 | "metadata": {}, 748 | "output_type": "execute_result" 749 | } 750 | ], 751 | "source": [ 752 | "sent_lens_t=[]\n", 753 | "for sent in train['Title']:\n", 754 | " sent_lens_t.append(len(word_tokenize(sent)))\n", 755 | "max(sent_lens_t)" 756 | ] 757 | }, 758 | { 759 | "cell_type": "code", 760 | "execution_count": 22, 761 | "metadata": {}, 762 | "outputs": [ 763 | { 764 | "data": { 765 | "text/plain": [ 766 | "18.0" 767 | ] 768 | }, 769 | "execution_count": 22, 770 | "metadata": {}, 771 | "output_type": "execute_result" 772 | } 773 | ], 774 | "source": [ 775 | "np.quantile(sent_lens_t,0.97)" 776 | ] 777 | }, 778 | { 779 | "cell_type": "code", 780 | "execution_count": 23, 781 | "metadata": {}, 782 | "outputs": [], 783 | "source": [ 784 | "max_len_t = 18\n", 785 | "tok = Tokenizer(char_level=False,split=' ')\n", 786 | "tok.fit_on_texts(X_train_t)\n", 787 | "sequences_train_t = tok.texts_to_sequences(X_train_t)" 788 | ] 789 | }, 790 | { 791 | "cell_type": "code", 792 | "execution_count": 24, 793 | "metadata": {}, 794 | "outputs": [ 795 | { 796 | "data": { 797 | "text/plain": [ 798 | "68969" 799 | ] 800 | }, 801 | "execution_count": 24, 802 | "metadata": {}, 803 | "output_type": "execute_result" 804 | } 805 | ], 806 | "source": [ 807 | "vocab_len_t=len(tok.index_word.keys())\n", 808 | "vocab_len_t" 809 | ] 810 | }, 811 | { 812 | "cell_type": "code", 813 | "execution_count": 25, 814 | "metadata": {}, 815 | "outputs": [ 816 | { 817 | "data": { 818 | "text/plain": [ 819 | "array([[ 0, 0, 0, ..., 1, 957, 197],\n", 820 | " [ 0, 0, 0, ..., 9081, 45, 533],\n", 821 | " [ 0, 0, 0, ..., 147, 8, 230],\n", 822 | " ...,\n", 823 | " [ 0, 0, 0, ..., 10, 71, 2985],\n", 824 | " [ 0, 0, 0, ..., 2, 18, 75],\n", 825 | " [ 0, 0, 0, ..., 11009, 809, 267]], dtype=int32)" 826 | ] 827 | }, 828 | "execution_count": 25, 829 | "metadata": {}, 830 | "output_type": "execute_result" 831 | } 832 | ], 833 | "source": [ 834 | "sequences_matrix_train_t = sequence.pad_sequences(sequences_train_t,maxlen=max_len_t)\n", 835 | "sequences_matrix_train_t" 836 | ] 837 | }, 838 | { 839 | "cell_type": "code", 840 | "execution_count": 26, 841 | "metadata": {}, 842 | "outputs": [], 843 | "source": [ 844 | "sequences_test_t = tok.texts_to_sequences(X_test_t)\n", 845 | "sequences_matrix_test_t = sequence.pad_sequences(sequences_test_t,maxlen=max_len_t)" 846 | ] 847 | }, 848 | { 849 | "cell_type": "code", 850 | "execution_count": 27, 851 | "metadata": {}, 852 | "outputs": [ 853 | { 854 | "data": { 855 | "text/plain": [ 856 | "((412500, 18), (137500, 18), (412500, 10), (137500, 10))" 857 | ] 858 | }, 859 | "execution_count": 27, 860 | "metadata": {}, 861 | "output_type": "execute_result" 862 | } 863 | ], 864 | "source": [ 865 | "sequences_matrix_train_t.shape,sequences_matrix_test_t.shape,y_train.shape,y_test.shape" 866 | ] 867 | }, 868 | { 869 | "cell_type": "code", 870 | "execution_count": 28, 871 | "metadata": {}, 872 | "outputs": [ 873 | { 874 | "data": { 875 | "text/plain": [ 876 | "20853" 877 | ] 878 | }, 879 | "execution_count": 28, 880 | "metadata": {}, 881 | "output_type": "execute_result" 882 | } 883 | ], 884 | "source": [ 885 | "sent_lens_b=[]\n", 886 | "for sent in train['Body']:\n", 887 | " sent_lens_b.append(len(word_tokenize(sent)))\n", 888 | "max(sent_lens_b)" 889 | ] 890 | }, 891 | { 892 | "cell_type": "code", 893 | "execution_count": 29, 894 | "metadata": {}, 895 | "outputs": [ 896 | { 897 | "data": { 898 | "text/plain": [ 899 | "575.0" 900 | ] 901 | }, 902 | "execution_count": 29, 903 | "metadata": {}, 904 | "output_type": "execute_result" 905 | } 906 | ], 907 | "source": [ 908 | "np.quantile(sent_lens_b,0.90)" 909 | ] 910 | }, 911 | { 912 | "cell_type": "code", 913 | "execution_count": 30, 914 | "metadata": {}, 915 | "outputs": [], 916 | "source": [ 917 | "max_len_b = 600\n", 918 | "tok = Tokenizer(char_level=False,split=' ')\n", 919 | "tok.fit_on_texts(X_train_b)\n", 920 | "sequences_train_b = tok.texts_to_sequences(X_train_b)" 921 | ] 922 | }, 923 | { 924 | "cell_type": "code", 925 | "execution_count": 31, 926 | "metadata": {}, 927 | "outputs": [ 928 | { 929 | "data": { 930 | "text/plain": [ 931 | "1292018" 932 | ] 933 | }, 934 | "execution_count": 31, 935 | "metadata": {}, 936 | "output_type": "execute_result" 937 | } 938 | ], 939 | "source": [ 940 | "vocab_len_b =len(tok.index_word.keys())\n", 941 | "vocab_len_b " 942 | ] 943 | }, 944 | { 945 | "cell_type": "code", 946 | "execution_count": 32, 947 | "metadata": {}, 948 | "outputs": [ 949 | { 950 | "data": { 951 | "text/plain": [ 952 | "array([[ 0, 0, 0, ..., 51, 2082, 91],\n", 953 | " [ 0, 0, 0, ..., 1408, 203, 825],\n", 954 | " [ 0, 0, 0, ..., 34, 51, 83],\n", 955 | " ...,\n", 956 | " [ 0, 0, 0, ..., 20, 68, 687],\n", 957 | " [ 0, 0, 0, ..., 187, 58, 10],\n", 958 | " [ 0, 0, 0, ..., 194, 197, 10]], dtype=int32)" 959 | ] 960 | }, 961 | "execution_count": 32, 962 | "metadata": {}, 963 | "output_type": "execute_result" 964 | } 965 | ], 966 | "source": [ 967 | "sequences_matrix_train_b = sequence.pad_sequences(sequences_train_b,maxlen=max_len_b)\n", 968 | "sequences_matrix_train_b" 969 | ] 970 | }, 971 | { 972 | "cell_type": "code", 973 | "execution_count": 33, 974 | "metadata": {}, 975 | "outputs": [], 976 | "source": [ 977 | "sequences_test_b = tok.texts_to_sequences(X_test_b)\n", 978 | "sequences_matrix_test_b = sequence.pad_sequences(sequences_test_b,maxlen=max_len_b)" 979 | ] 980 | }, 981 | { 982 | "cell_type": "code", 983 | "execution_count": 34, 984 | "metadata": {}, 985 | "outputs": [ 986 | { 987 | "data": { 988 | "text/plain": [ 989 | "((412500, 18), (412500, 600), (412500, 10))" 990 | ] 991 | }, 992 | "execution_count": 34, 993 | "metadata": {}, 994 | "output_type": "execute_result" 995 | } 996 | ], 997 | "source": [ 998 | "sequences_matrix_train_t.shape,sequences_matrix_train_b.shape,y_train.shape" 999 | ] 1000 | }, 1001 | { 1002 | "cell_type": "code", 1003 | "execution_count": 35, 1004 | "metadata": {}, 1005 | "outputs": [ 1006 | { 1007 | "data": { 1008 | "text/plain": [ 1009 | "((137500, 18), (137500, 600), (137500, 10))" 1010 | ] 1011 | }, 1012 | "execution_count": 35, 1013 | "metadata": {}, 1014 | "output_type": "execute_result" 1015 | } 1016 | ], 1017 | "source": [ 1018 | "sequences_matrix_test_t.shape,sequences_matrix_test_b.shape,y_test.shape" 1019 | ] 1020 | }, 1021 | { 1022 | "cell_type": "code", 1023 | "execution_count": 36, 1024 | "metadata": {}, 1025 | "outputs": [], 1026 | "source": [ 1027 | "def RNN():\n", 1028 | " # Title Only\n", 1029 | " title_input = Input(name='title_input',shape=[max_len_t])\n", 1030 | " title_Embed = Embedding(vocab_len_t+1,2000,input_length=max_len_t,mask_zero=True,name='title_Embed')(title_input)\n", 1031 | " gru_out_t = GRU(300)(title_Embed)\n", 1032 | " # auxiliary output to tune GRU weights smoothly \n", 1033 | " auxiliary_output = Dense(10, activation='sigmoid', name='aux_output')(gru_out_t) \n", 1034 | " \n", 1035 | " # Body Only\n", 1036 | " body_input = Input(name='body_input',shape=[max_len_b]) \n", 1037 | " body_Embed = Embedding(vocab_len_b+1,170,input_length=max_len_b,mask_zero=True,name='body_Embed')(body_input)\n", 1038 | " gru_out_b = GRU(200)(body_Embed)\n", 1039 | " \n", 1040 | " # combined with GRU output\n", 1041 | " com = concatenate([gru_out_t, gru_out_b])\n", 1042 | " \n", 1043 | " # now the combined data is being fed to dense layers\n", 1044 | " dense1 = Dense(400,activation='relu')(com)\n", 1045 | " dp1 = Dropout(0.5)(dense1)\n", 1046 | " bn = BatchNormalization()(dp1) \n", 1047 | " dense2 = Dense(150,activation='relu')(bn)\n", 1048 | " \n", 1049 | " main_output = Dense(10, activation='sigmoid', name='main_output')(dense2)\n", 1050 | " \n", 1051 | " model = Model(inputs=[title_input, body_input],outputs=[main_output, auxiliary_output])\n", 1052 | " return model" 1053 | ] 1054 | }, 1055 | { 1056 | "cell_type": "code", 1057 | "execution_count": 37, 1058 | "metadata": {}, 1059 | "outputs": [ 1060 | { 1061 | "name": "stdout", 1062 | "output_type": "stream", 1063 | "text": [ 1064 | "WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n", 1065 | "Instructions for updating:\n", 1066 | "Colocations handled automatically by placer.\n", 1067 | "WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n", 1068 | "Instructions for updating:\n", 1069 | "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n", 1070 | "__________________________________________________________________________________________________\n", 1071 | "Layer (type) Output Shape Param # Connected to \n", 1072 | "==================================================================================================\n", 1073 | "title_input (InputLayer) (None, 18) 0 \n", 1074 | "__________________________________________________________________________________________________\n", 1075 | "body_input (InputLayer) (None, 600) 0 \n", 1076 | "__________________________________________________________________________________________________\n", 1077 | "title_Embed (Embedding) (None, 18, 2000) 137940000 title_input[0][0] \n", 1078 | "__________________________________________________________________________________________________\n", 1079 | "body_Embed (Embedding) (None, 600, 170) 219643230 body_input[0][0] \n", 1080 | "__________________________________________________________________________________________________\n", 1081 | "gru_1 (GRU) (None, 300) 2070900 title_Embed[0][0] \n", 1082 | "__________________________________________________________________________________________________\n", 1083 | "gru_2 (GRU) (None, 200) 222600 body_Embed[0][0] \n", 1084 | "__________________________________________________________________________________________________\n", 1085 | "concatenate_1 (Concatenate) (None, 500) 0 gru_1[0][0] \n", 1086 | " gru_2[0][0] \n", 1087 | "__________________________________________________________________________________________________\n", 1088 | "dense_1 (Dense) (None, 400) 200400 concatenate_1[0][0] \n", 1089 | "__________________________________________________________________________________________________\n", 1090 | "dropout_1 (Dropout) (None, 400) 0 dense_1[0][0] \n", 1091 | "__________________________________________________________________________________________________\n", 1092 | "batch_normalization_1 (BatchNor (None, 400) 1600 dropout_1[0][0] \n", 1093 | "__________________________________________________________________________________________________\n", 1094 | "dense_2 (Dense) (None, 150) 60150 batch_normalization_1[0][0] \n", 1095 | "__________________________________________________________________________________________________\n", 1096 | "main_output (Dense) (None, 10) 1510 dense_2[0][0] \n", 1097 | "__________________________________________________________________________________________________\n", 1098 | "aux_output (Dense) (None, 10) 3010 gru_1[0][0] \n", 1099 | "==================================================================================================\n", 1100 | "Total params: 360,143,400\n", 1101 | "Trainable params: 360,142,600\n", 1102 | "Non-trainable params: 800\n", 1103 | "__________________________________________________________________________________________________\n" 1104 | ] 1105 | } 1106 | ], 1107 | "source": [ 1108 | "model = RNN()\n", 1109 | "model.summary()" 1110 | ] 1111 | }, 1112 | { 1113 | "cell_type": "code", 1114 | "execution_count": 38, 1115 | "metadata": {}, 1116 | "outputs": [], 1117 | "source": [ 1118 | "model.compile(optimizer='adam',loss={'main_output': 'categorical_crossentropy', 'aux_output': 'categorical_crossentropy'},\n", 1119 | " metrics=['accuracy'])" 1120 | ] 1121 | }, 1122 | { 1123 | "cell_type": "code", 1124 | "execution_count": 39, 1125 | "metadata": {}, 1126 | "outputs": [ 1127 | { 1128 | "name": "stdout", 1129 | "output_type": "stream", 1130 | "text": [ 1131 | "WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n", 1132 | "Instructions for updating:\n", 1133 | "Use tf.cast instead.\n" 1134 | ] 1135 | }, 1136 | { 1137 | "name": "stderr", 1138 | "output_type": "stream", 1139 | "text": [ 1140 | "/opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:107: UserWarning: Converting sparse IndexedSlices to a dense Tensor with 137940000 elements. This may consume a large amount of memory.\n", 1141 | " num_elements)\n", 1142 | "/opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:107: UserWarning: Converting sparse IndexedSlices to a dense Tensor with 219643230 elements. This may consume a large amount of memory.\n", 1143 | " num_elements)\n" 1144 | ] 1145 | }, 1146 | { 1147 | "name": "stdout", 1148 | "output_type": "stream", 1149 | "text": [ 1150 | "Train on 412500 samples, validate on 137500 samples\n", 1151 | "Epoch 1/5\n", 1152 | "412500/412500 [==============================] - 819s 2ms/step - loss: 2.3168 - main_output_loss: 1.0405 - aux_output_loss: 1.2763 - main_output_acc: 0.7285 - aux_output_acc: 0.6705 - val_loss: 1.7714 - val_main_output_loss: 0.7258 - val_aux_output_loss: 1.0456 - val_main_output_acc: 0.8254 - val_aux_output_acc: 0.7285\n", 1153 | "Epoch 2/5\n", 1154 | "412500/412500 [==============================] - 797s 2ms/step - loss: 1.5874 - main_output_loss: 0.6490 - aux_output_loss: 0.9384 - main_output_acc: 0.8443 - aux_output_acc: 0.7582 - val_loss: 1.7043 - val_main_output_loss: 0.6581 - val_aux_output_loss: 1.0462 - val_main_output_acc: 0.8379 - val_aux_output_acc: 0.7286\n", 1155 | "Epoch 3/5\n", 1156 | "412500/412500 [==============================] - 797s 2ms/step - loss: 1.3994 - main_output_loss: 0.5472 - aux_output_loss: 0.8522 - main_output_acc: 0.8650 - aux_output_acc: 0.7774 - val_loss: 1.7369 - val_main_output_loss: 0.6660 - val_aux_output_loss: 1.0708 - val_main_output_acc: 0.8364 - val_aux_output_acc: 0.7263\n", 1157 | "Epoch 4/5\n", 1158 | "412500/412500 [==============================] - 798s 2ms/step - loss: 1.2719 - main_output_loss: 0.4735 - aux_output_loss: 0.7984 - main_output_acc: 0.8774 - aux_output_acc: 0.7885 - val_loss: 1.7988 - val_main_output_loss: 0.6976 - val_aux_output_loss: 1.1012 - val_main_output_acc: 0.8369 - val_aux_output_acc: 0.7240\n", 1159 | "Epoch 5/5\n", 1160 | "412500/412500 [==============================] - 797s 2ms/step - loss: 1.1665 - main_output_loss: 0.4110 - aux_output_loss: 0.7555 - main_output_acc: 0.8868 - aux_output_acc: 0.7976 - val_loss: 1.9099 - val_main_output_loss: 0.7671 - val_aux_output_loss: 1.1428 - val_main_output_acc: 0.8307 - val_aux_output_acc: 0.7237\n" 1161 | ] 1162 | } 1163 | ], 1164 | "source": [ 1165 | "results=model.fit({'title_input': sequences_matrix_train_t, 'body_input': sequences_matrix_train_b},\n", 1166 | " {'main_output': y_train, 'aux_output': y_train},\n", 1167 | " validation_data=[{'title_input': sequences_matrix_test_t, 'body_input': sequences_matrix_test_b},\n", 1168 | " {'main_output': y_test, 'aux_output': y_test}],\n", 1169 | " epochs=5, batch_size=800)" 1170 | ] 1171 | }, 1172 | { 1173 | "cell_type": "code", 1174 | "execution_count": 68, 1175 | "metadata": {}, 1176 | "outputs": [ 1177 | { 1178 | "name": "stdout", 1179 | "output_type": "stream", 1180 | "text": [ 1181 | "137500/137500 [==============================] - 1270s 9ms/step\n" 1182 | ] 1183 | } 1184 | ], 1185 | "source": [ 1186 | "(predicted_main, predicted_aux)=model.predict({'title_input': sequences_matrix_test_t, 'body_input': sequences_matrix_test_b},verbose=1)" 1187 | ] 1188 | }, 1189 | { 1190 | "cell_type": "code", 1191 | "execution_count": 70, 1192 | "metadata": {}, 1193 | "outputs": [], 1194 | "source": [ 1195 | "from sklearn.metrics import classification_report,f1_score" 1196 | ] 1197 | }, 1198 | { 1199 | "cell_type": "code", 1200 | "execution_count": 138, 1201 | "metadata": {}, 1202 | "outputs": [ 1203 | { 1204 | "name": "stdout", 1205 | "output_type": "stream", 1206 | "text": [ 1207 | "0.8424636536796537\n" 1208 | ] 1209 | }, 1210 | { 1211 | "name": "stderr", 1212 | "output_type": "stream", 1213 | "text": [ 1214 | "/opt/conda/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in samples with no predicted labels.\n", 1215 | " 'precision', 'predicted', average, warn_for)\n" 1216 | ] 1217 | } 1218 | ], 1219 | "source": [ 1220 | "print(f1_score(y_test,predicted_main>.55,average='samples'))" 1221 | ] 1222 | }, 1223 | { 1224 | "cell_type": "code", 1225 | "execution_count": 137, 1226 | "metadata": {}, 1227 | "outputs": [ 1228 | { 1229 | "name": "stdout", 1230 | "output_type": "stream", 1231 | "text": [ 1232 | " precision recall f1-score support\n", 1233 | "\n", 1234 | " 0 0.97 0.93 0.95 17054\n", 1235 | " 1 0.92 0.84 0.88 20681\n", 1236 | " 2 0.92 0.81 0.86 9700\n", 1237 | " 3 0.69 0.53 0.60 11304\n", 1238 | " 4 0.96 0.91 0.94 8897\n", 1239 | " 5 0.91 0.80 0.85 22472\n", 1240 | " 6 0.82 0.72 0.76 22938\n", 1241 | " 7 0.81 0.83 0.82 16150\n", 1242 | " 8 0.92 0.90 0.91 19659\n", 1243 | " 9 0.97 0.92 0.95 11576\n", 1244 | "\n", 1245 | " micro avg 0.89 0.82 0.85 160431\n", 1246 | " macro avg 0.89 0.82 0.85 160431\n", 1247 | "weighted avg 0.89 0.82 0.85 160431\n", 1248 | " samples avg 0.86 0.85 0.84 160431\n", 1249 | "\n" 1250 | ] 1251 | }, 1252 | { 1253 | "name": "stderr", 1254 | "output_type": "stream", 1255 | "text": [ 1256 | "/opt/conda/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in samples with no predicted labels.\n", 1257 | " 'precision', 'predicted', average, warn_for)\n" 1258 | ] 1259 | } 1260 | ], 1261 | "source": [ 1262 | "print(classification_report(y_test,predicted_main>.55))" 1263 | ] 1264 | }, 1265 | { 1266 | "cell_type": "code", 1267 | "execution_count": 131, 1268 | "metadata": {}, 1269 | "outputs": [ 1270 | { 1271 | "data": { 1272 | "text/plain": [ 1273 | "Id 16470700\n", 1274 | "Title NetworkOnMainThreadException- Have tried makin...\n", 1275 | "Body I've been trying to get this to work for a whi...\n", 1276 | "Tags [java, android]\n", 1277 | "Name: 250148, dtype: object" 1278 | ] 1279 | }, 1280 | "execution_count": 131, 1281 | "metadata": {}, 1282 | "output_type": "execute_result" 1283 | } 1284 | ], 1285 | "source": [ 1286 | "test.iloc[24]" 1287 | ] 1288 | }, 1289 | { 1290 | "cell_type": "code", 1291 | "execution_count": 134, 1292 | "metadata": {}, 1293 | "outputs": [ 1294 | { 1295 | "data": { 1296 | "text/plain": [ 1297 | "array([1. , 0. , 0. , 0. , 0. , 0.84, 0. , 0. , 0. , 0. ],\n", 1298 | " dtype=float32)" 1299 | ] 1300 | }, 1301 | "execution_count": 134, 1302 | "metadata": {}, 1303 | "output_type": "execute_result" 1304 | } 1305 | ], 1306 | "source": [ 1307 | "predicted_main[24].round(decimals = 2)" 1308 | ] 1309 | }, 1310 | { 1311 | "cell_type": "code", 1312 | "execution_count": 92, 1313 | "metadata": {}, 1314 | "outputs": [ 1315 | { 1316 | "data": { 1317 | "text/plain": [ 1318 | "array(['android', 'c#', 'c++', 'html', 'ios', 'java', 'javascript',\n", 1319 | " 'jquery', 'php', 'python'], dtype=object)" 1320 | ] 1321 | }, 1322 | "execution_count": 92, 1323 | "metadata": {}, 1324 | "output_type": "execute_result" 1325 | } 1326 | ], 1327 | "source": [ 1328 | "labels" 1329 | ] 1330 | }, 1331 | { 1332 | "cell_type": "code", 1333 | "execution_count": 79, 1334 | "metadata": {}, 1335 | "outputs": [], 1336 | "source": [ 1337 | "model.save('./stackoverflow_tags.h5')" 1338 | ] 1339 | } 1340 | ], 1341 | "metadata": { 1342 | "kernelspec": { 1343 | "display_name": "Python 3", 1344 | "language": "python", 1345 | "name": "python3" 1346 | }, 1347 | "language_info": { 1348 | "codemirror_mode": { 1349 | "name": "ipython", 1350 | "version": 3 1351 | }, 1352 | "file_extension": ".py", 1353 | "mimetype": "text/x-python", 1354 | "name": "python", 1355 | "nbconvert_exporter": "python", 1356 | "pygments_lexer": "ipython3", 1357 | "version": "3.6.7" 1358 | } 1359 | }, 1360 | "nbformat": 4, 1361 | "nbformat_minor": 1 1362 | } 1363 | --------------------------------------------------------------------------------