├── Images
    ├── Classification_Report.JPG
    ├── prediction.JPG
    └── top10_tags.JPG
├── README.md
├── Stackoverflow Clean Questions.ipynb
└── Stackoverflow Tags Map & Model.ipynb


/Images/Classification_Report.JPG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/theRajeshReddy/StackOverFlow-Classification/8ba027db0dfa3d09724a62838bebdd979b031af0/Images/Classification_Report.JPG


--------------------------------------------------------------------------------
/Images/prediction.JPG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/theRajeshReddy/StackOverFlow-Classification/8ba027db0dfa3d09724a62838bebdd979b031af0/Images/prediction.JPG


--------------------------------------------------------------------------------
/Images/top10_tags.JPG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/theRajeshReddy/StackOverFlow-Classification/8ba027db0dfa3d09724a62838bebdd979b031af0/Images/top10_tags.JPG


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | # Multiclass Multilabel Prediction For StackOverflow Questions
  2 | 
  3 | **Data set** : https://www.kaggle.com/therajeshreddy/stackoverflow
  4 | 
  5 | **Objective** : Given text for Questions from StackoverFlow posts, predict tags associated with them.
  6 | 
  7 | This is a scaled down version of predecting only top 10 most occurring tags 
  8 | 
  9 | **Programming Language** : Python using nltk & Keras
 10 | 
 11 | **Model Architecture** : Deep Learning using Recurrent Neural Network (RNN)
 12 | 
 13 | **About Data Set** 
 14 | 
 15 | Dataset has text of questions, answers and thier corresponding tags from the Stack Overflow programming Q&A website.
 16 | 
 17 | This is organized as three files:
 18 | 
 19 | 1. Questions contains the title, body, creation date, closed date (if applicable), score, and owner ID for all non-deleted Stack Overflow questions.
 20 | 
 21 | 2. Tags contains the tags on each of these questions.
 22 | 
 23 | 3. Answers contains the body, creation date, score, and owner ID for each of the answers to these questions. The ParentId column links back to the Questions table. *We don't use this file as we want to predict Tags given a question*
 24 | 
 25 | **Data Pre-Processing**
 26 | 
 27 | >Questions File
 28 | *Code* : Stackoverflow Clean Questions.ipynb
 29 | 
 30 | 1. Read Questions File
 31 | 2. Drop All columns except Id,Title and Body
 32 | 3. Now the text in the Body column seem to have many html tags in the text. We use Regular Expressions and Clean the Body column text by removing the html tags
 33 | ```python
 34 | import re 
 35 | def rem_html_tags(body):
 36 |     regex = re.compile('<.*?>')
 37 |     return re.sub(regex, '', body)
 38 | ques['Body'] = ques['Body'].apply(rem_html_tags)
 39 | ```
 40 | 4. Save the questions file for later use
 41 | ```python
 42 | ques.to_csv('question_clean.csv',index=False)
 43 | ```
 44 | 
 45 | >Tags File
 46 | *Code* : Stackoverflow Tags Map & Model.ipynb
 47 | 
 48 | 1. Read Tags File
 49 | 2. Identify top 10 Tags by count
 50 | ```python
 51 | tagCount =  collections.Counter(list(df_tags['Tag'])).most_common(10)
 52 | print(tagCount)
 53 | 
 54 | [('javascript', 124155), ('java', 115212), ('c#', 101186), ('php', 98808), ('android', 90659), ('jquery', 78542), ('python', 64601), ('html', 58976), ('c++', 47591), ('ios', 47009)]
 55 | ```
 56 | 
 57 | <img src="Images/top10_tags.JPG" width="600">
 58 | 
 59 | 3. Manipulate the tags dataframe so that all the Tags for an ID are as a list in a row (grouped by Question ID)
 60 | 
 61 | ```python
 62 | def add_tags(question_id):
 63 |     return tag_top10[tag_top10['Id'] == question_id['Id']].Tag.values
 64 | 
 65 | top10 = tag_top10.apply(add_tags, axis=1)
 66 | ```
 67 | 
 68 | 
 69 | >Combine the Questions and Tags
 70 | *Code* : Stackoverflow Tags Map & Model.ipynb
 71 | 
 72 | Merge the Questions and Tags data frame by ID
 73 | 
 74 | ```python
 75 | total=pd.merge(ques, top10_tags, on='Id')
 76 | ```
 77 | 
 78 | Our Dataset would now have only Id, Title, Body & Tags
 79 | 
 80 | >Text Preprocessing
 81 | *Code* : Stackoverflow Tags Map & Model.ipynb
 82 | 
 83 | We will use nltk, preprocessing from Keras and sklearn to process the text data
 84 | 
 85 | *Tags preprocesing*
 86 | Use MultiLabelBinarizer from sklearn on the Class labels(Tags)
 87 | ```python
 88 | from sklearn.preprocessing import MultiLabelBinarizer
 89 | multilabel_binarizer = MultiLabelBinarizer()
 90 | multilabel_binarizer.fit(total.Tags)
 91 | print(multilabel_binarizer.classes_)
 92 | 
 93 | array(['android', 'c#', 'c++', 'html', 'ios', 'java', 'javascript','jquery', 'php', 'python'], dtype=object)
 94 | ```
 95 | 
 96 | *Title & Body Preprocessing*
 97 | 1. Tokenize the words
 98 | 2. Convert the tokenized words to sequences
 99 | 
100 | **Model Building** 
101 | 
102 | Implemented a Hybrid model in TensorFlow using Keras as high level api. Architecture used is RNN. In this model first we train a model using the Title data, then train a model using the Body data. Outputs of both are concatenated and passed thorugh the dense layers before connecting to the output layer
103 | 
104 | *RNN Model* : The model first uses GRU for the sequence data training with 2 GRU layers one for Title and other for Body. 
105 | 
106 | RNN for Title has
107 |   - 1 Embedding Layer has input of Title vocabulary length(68969) + 1(for 0 padding) and out put of 2000 embeddings (for better results use full vocabulary length+1)
108 |   - 1 Gated recurrent unit (GRU) layer
109 |   - 1 dense output layer of shape 10(No of classes(tags) we are trying to predict) 
110 | 
111 | ```python
112 |     # Title Only
113 |     title_input = Input(name='title_input',shape=[max_len_t])
114 |     title_Embed = Embedding(vocab_len_t+1,2000,input_length=max_len_t,mask_zero=True,name='title_Embed')(title_input)
115 |     gru_out_t = GRU(300)(title_Embed)
116 |     # auxiliary output to tune GRU weights smoothly 
117 |     auxiliary_output = Dense(10, activation='sigmoid', name='aux_output')(gru_out_t) 
118 | ```
119 | 
120 | RNN for Body has
121 |   - 1 Embedding Layer has input of Title vocabulary length(1292018) + 1(for 0 padding) and out put of 170 embeddings (for better results use full vocabulary length+1)
122 |   - 1 Gated recurrent unit (GRU) layer
123 | 
124 | ```python
125 |     # Body Only
126 |     body_input = Input(name='body_input',shape=[max_len_b]) 
127 |     body_Embed = Embedding(vocab_len_b+1,170,input_length=max_len_b,mask_zero=True,name='body_Embed')(body_input)
128 |     gru_out_b = GRU(200)(body_Embed) 
129 | ```
130 | 
131 | Combine the 2 GRU outputs
132 | ```python
133 |     com = concatenate([gru_out_t, gru_out_b])
134 | ```
135 | 
136 | The fully connected network has
137 |   - 2 Dense Layers 
138 |   - 1 Dropout layer
139 |   - 1 BatchNormalization layer
140 |   - 1 Dense Output layer
141 |   
142 | ```python
143 |     # now the combined data is being fed to dense layers
144 |     dense1 = Dense(400,activation='relu')(com)
145 |     dp1 = Dropout(0.5)(dense1)
146 |     bn = BatchNormalization()(dp1) 
147 |     dense2 = Dense(150,activation='relu')(bn)
148 |     main_output = Dense(10, activation='sigmoid', name='main_output')(dense2)
149 | ```
150 | 
151 | *Model Compilattion with optimizer='adam', loss='categorical_crossentropy', metrics='accuracy')*
152 | 
153 | **Model Performance Review**
154 | 
155 | *Classification Report to check Precision, Recall and F1 Score*
156 | 
157 | The Model seem to performing good enough with score of 84%. Increase in the Embedding, GRU and dense layers would help in getting better results
158 | 
159 | <img src="Images/Classification_Report.JPG" width="400">
160 | 
161 | **Random Validation on Test Data**
162 | 
163 | <img src="Images/prediction.JPG" width="400">
164 | 
165 | 
166 | **Save the Model & Weights**
167 | 
168 | Saving the model for transfer learning or model execution later
169 | 
170 | ```python
171 | model.save('./stackoverflow_tags.h5')
172 | ```
173 | 


--------------------------------------------------------------------------------
/Stackoverflow Clean Questions.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "code",
  5 |    "execution_count": 1,
  6 |    "metadata": {
  7 |     "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
  8 |     "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
  9 |    },
 10 |    "outputs": [
 11 |     {
 12 |      "name": "stdout",
 13 |      "output_type": "stream",
 14 |      "text": [
 15 |       "['Answers.csv', 'Tags.csv', 'Questions.csv']\n"
 16 |      ]
 17 |     }
 18 |    ],
 19 |    "source": [
 20 |     "# This Python 3 environment comes with many helpful analytics libraries installed\n",
 21 |     "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n",
 22 |     "# For example, here's several helpful packages to load in \n",
 23 |     "\n",
 24 |     "import numpy as np # linear algebra\n",
 25 |     "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
 26 |     "\n",
 27 |     "# for counting\n",
 28 |     "import collections\n",
 29 |     "\n",
 30 |     "# Input data files are available in the \"../input/\" directory.\n",
 31 |     "# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n",
 32 |     "\n",
 33 |     "import os\n",
 34 |     "print(os.listdir(\"../input\"))\n",
 35 |     "\n",
 36 |     "# Any results you write to the current directory are saved as output."
 37 |    ]
 38 |   },
 39 |   {
 40 |    "cell_type": "code",
 41 |    "execution_count": 2,
 42 |    "metadata": {
 43 |     "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
 44 |     "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"
 45 |    },
 46 |    "outputs": [
 47 |     {
 48 |      "data": {
 49 |       "text/html": [
 50 |        "<div>\n",
 51 |        "<style scoped>\n",
 52 |        "    .dataframe tbody tr th:only-of-type {\n",
 53 |        "        vertical-align: middle;\n",
 54 |        "    }\n",
 55 |        "\n",
 56 |        "    .dataframe tbody tr th {\n",
 57 |        "        vertical-align: top;\n",
 58 |        "    }\n",
 59 |        "\n",
 60 |        "    .dataframe thead th {\n",
 61 |        "        text-align: right;\n",
 62 |        "    }\n",
 63 |        "</style>\n",
 64 |        "<table border=\"1\" class=\"dataframe\">\n",
 65 |        "  <thead>\n",
 66 |        "    <tr style=\"text-align: right;\">\n",
 67 |        "      <th></th>\n",
 68 |        "      <th>Id</th>\n",
 69 |        "      <th>OwnerUserId</th>\n",
 70 |        "      <th>CreationDate</th>\n",
 71 |        "      <th>ClosedDate</th>\n",
 72 |        "      <th>Score</th>\n",
 73 |        "      <th>Title</th>\n",
 74 |        "      <th>Body</th>\n",
 75 |        "    </tr>\n",
 76 |        "  </thead>\n",
 77 |        "  <tbody>\n",
 78 |        "    <tr>\n",
 79 |        "      <th>0</th>\n",
 80 |        "      <td>80</td>\n",
 81 |        "      <td>26.0</td>\n",
 82 |        "      <td>2008-08-01T13:57:07Z</td>\n",
 83 |        "      <td>NaN</td>\n",
 84 |        "      <td>26</td>\n",
 85 |        "      <td>SQLStatement.execute() - multiple queries in o...</td>\n",
 86 |        "      <td>&lt;p&gt;I've written a database generation script i...</td>\n",
 87 |        "    </tr>\n",
 88 |        "    <tr>\n",
 89 |        "      <th>1</th>\n",
 90 |        "      <td>90</td>\n",
 91 |        "      <td>58.0</td>\n",
 92 |        "      <td>2008-08-01T14:41:24Z</td>\n",
 93 |        "      <td>2012-12-26T03:45:49Z</td>\n",
 94 |        "      <td>144</td>\n",
 95 |        "      <td>Good branching and merging tutorials for Torto...</td>\n",
 96 |        "      <td>&lt;p&gt;Are there any really good tutorials explain...</td>\n",
 97 |        "    </tr>\n",
 98 |        "    <tr>\n",
 99 |        "      <th>2</th>\n",
100 |        "      <td>120</td>\n",
101 |        "      <td>83.0</td>\n",
102 |        "      <td>2008-08-01T15:50:08Z</td>\n",
103 |        "      <td>NaN</td>\n",
104 |        "      <td>21</td>\n",
105 |        "      <td>ASP.NET Site Maps</td>\n",
106 |        "      <td>&lt;p&gt;Has anyone got experience creating &lt;strong&gt;...</td>\n",
107 |        "    </tr>\n",
108 |        "    <tr>\n",
109 |        "      <th>3</th>\n",
110 |        "      <td>180</td>\n",
111 |        "      <td>2089740.0</td>\n",
112 |        "      <td>2008-08-01T18:42:19Z</td>\n",
113 |        "      <td>NaN</td>\n",
114 |        "      <td>53</td>\n",
115 |        "      <td>Function for creating color wheels</td>\n",
116 |        "      <td>&lt;p&gt;This is something I've pseudo-solved many t...</td>\n",
117 |        "    </tr>\n",
118 |        "    <tr>\n",
119 |        "      <th>4</th>\n",
120 |        "      <td>260</td>\n",
121 |        "      <td>91.0</td>\n",
122 |        "      <td>2008-08-01T23:22:08Z</td>\n",
123 |        "      <td>NaN</td>\n",
124 |        "      <td>49</td>\n",
125 |        "      <td>Adding scripting functionality to .NET applica...</td>\n",
126 |        "      <td>&lt;p&gt;I have a little game written in C#. It uses...</td>\n",
127 |        "    </tr>\n",
128 |        "    <tr>\n",
129 |        "      <th>5</th>\n",
130 |        "      <td>330</td>\n",
131 |        "      <td>63.0</td>\n",
132 |        "      <td>2008-08-02T02:51:36Z</td>\n",
133 |        "      <td>NaN</td>\n",
134 |        "      <td>29</td>\n",
135 |        "      <td>Should I use nested classes in this case?</td>\n",
136 |        "      <td>&lt;p&gt;I am working on a collection of classes use...</td>\n",
137 |        "    </tr>\n",
138 |        "    <tr>\n",
139 |        "      <th>6</th>\n",
140 |        "      <td>470</td>\n",
141 |        "      <td>71.0</td>\n",
142 |        "      <td>2008-08-02T15:11:47Z</td>\n",
143 |        "      <td>2016-03-26T05:23:29Z</td>\n",
144 |        "      <td>13</td>\n",
145 |        "      <td>Homegrown consumption of web services</td>\n",
146 |        "      <td>&lt;p&gt;I've been writing a few web services for a ...</td>\n",
147 |        "    </tr>\n",
148 |        "    <tr>\n",
149 |        "      <th>7</th>\n",
150 |        "      <td>580</td>\n",
151 |        "      <td>91.0</td>\n",
152 |        "      <td>2008-08-02T23:30:59Z</td>\n",
153 |        "      <td>NaN</td>\n",
154 |        "      <td>21</td>\n",
155 |        "      <td>Deploying SQL Server Databases from Test to Live</td>\n",
156 |        "      <td>&lt;p&gt;I wonder how you guys manage deployment of ...</td>\n",
157 |        "    </tr>\n",
158 |        "    <tr>\n",
159 |        "      <th>8</th>\n",
160 |        "      <td>650</td>\n",
161 |        "      <td>143.0</td>\n",
162 |        "      <td>2008-08-03T11:12:52Z</td>\n",
163 |        "      <td>NaN</td>\n",
164 |        "      <td>79</td>\n",
165 |        "      <td>Automatically update version number</td>\n",
166 |        "      <td>&lt;p&gt;I would like the version property of my app...</td>\n",
167 |        "    </tr>\n",
168 |        "    <tr>\n",
169 |        "      <th>9</th>\n",
170 |        "      <td>810</td>\n",
171 |        "      <td>233.0</td>\n",
172 |        "      <td>2008-08-03T20:35:01Z</td>\n",
173 |        "      <td>NaN</td>\n",
174 |        "      <td>9</td>\n",
175 |        "      <td>Visual Studio Setup Project - Per User Registr...</td>\n",
176 |        "      <td>&lt;p&gt;I'm trying to maintain a Setup Project in &lt;...</td>\n",
177 |        "    </tr>\n",
178 |        "  </tbody>\n",
179 |        "</table>\n",
180 |        "</div>"
181 |       ],
182 |       "text/plain": [
183 |        "    Id                        ...                                                                       Body\n",
184 |        "0   80                        ...                          <p>I've written a database generation script i...\n",
185 |        "1   90                        ...                          <p>Are there any really good tutorials explain...\n",
186 |        "2  120                        ...                          <p>Has anyone got experience creating <strong>...\n",
187 |        "3  180                        ...                          <p>This is something I've pseudo-solved many t...\n",
188 |        "4  260                        ...                          <p>I have a little game written in C#. It uses...\n",
189 |        "5  330                        ...                          <p>I am working on a collection of classes use...\n",
190 |        "6  470                        ...                          <p>I've been writing a few web services for a ...\n",
191 |        "7  580                        ...                          <p>I wonder how you guys manage deployment of ...\n",
192 |        "8  650                        ...                          <p>I would like the version property of my app...\n",
193 |        "9  810                        ...                          <p>I'm trying to maintain a Setup Project in <...\n",
194 |        "\n",
195 |        "[10 rows x 7 columns]"
196 |       ]
197 |      },
198 |      "execution_count": 2,
199 |      "metadata": {},
200 |      "output_type": "execute_result"
201 |     }
202 |    ],
203 |    "source": [
204 |     "ques = pd.read_csv('../input/Questions.csv',encoding='iso-8859-1')\n",
205 |     "ques.head(10)"
206 |    ]
207 |   },
208 |   {
209 |    "cell_type": "code",
210 |    "execution_count": 3,
211 |    "metadata": {
212 |     "_uuid": "3a963c29707e29d8df14b8f48e45d25b336fcdb0"
213 |    },
214 |    "outputs": [
215 |     {
216 |      "data": {
217 |       "text/html": [
218 |        "<div>\n",
219 |        "<style scoped>\n",
220 |        "    .dataframe tbody tr th:only-of-type {\n",
221 |        "        vertical-align: middle;\n",
222 |        "    }\n",
223 |        "\n",
224 |        "    .dataframe tbody tr th {\n",
225 |        "        vertical-align: top;\n",
226 |        "    }\n",
227 |        "\n",
228 |        "    .dataframe thead th {\n",
229 |        "        text-align: right;\n",
230 |        "    }\n",
231 |        "</style>\n",
232 |        "<table border=\"1\" class=\"dataframe\">\n",
233 |        "  <thead>\n",
234 |        "    <tr style=\"text-align: right;\">\n",
235 |        "      <th></th>\n",
236 |        "      <th>Id</th>\n",
237 |        "      <th>Title</th>\n",
238 |        "      <th>Body</th>\n",
239 |        "    </tr>\n",
240 |        "  </thead>\n",
241 |        "  <tbody>\n",
242 |        "    <tr>\n",
243 |        "      <th>0</th>\n",
244 |        "      <td>80</td>\n",
245 |        "      <td>SQLStatement.execute() - multiple queries in o...</td>\n",
246 |        "      <td>&lt;p&gt;I've written a database generation script i...</td>\n",
247 |        "    </tr>\n",
248 |        "    <tr>\n",
249 |        "      <th>1</th>\n",
250 |        "      <td>90</td>\n",
251 |        "      <td>Good branching and merging tutorials for Torto...</td>\n",
252 |        "      <td>&lt;p&gt;Are there any really good tutorials explain...</td>\n",
253 |        "    </tr>\n",
254 |        "    <tr>\n",
255 |        "      <th>2</th>\n",
256 |        "      <td>120</td>\n",
257 |        "      <td>ASP.NET Site Maps</td>\n",
258 |        "      <td>&lt;p&gt;Has anyone got experience creating &lt;strong&gt;...</td>\n",
259 |        "    </tr>\n",
260 |        "    <tr>\n",
261 |        "      <th>3</th>\n",
262 |        "      <td>180</td>\n",
263 |        "      <td>Function for creating color wheels</td>\n",
264 |        "      <td>&lt;p&gt;This is something I've pseudo-solved many t...</td>\n",
265 |        "    </tr>\n",
266 |        "    <tr>\n",
267 |        "      <th>4</th>\n",
268 |        "      <td>260</td>\n",
269 |        "      <td>Adding scripting functionality to .NET applica...</td>\n",
270 |        "      <td>&lt;p&gt;I have a little game written in C#. It uses...</td>\n",
271 |        "    </tr>\n",
272 |        "    <tr>\n",
273 |        "      <th>5</th>\n",
274 |        "      <td>330</td>\n",
275 |        "      <td>Should I use nested classes in this case?</td>\n",
276 |        "      <td>&lt;p&gt;I am working on a collection of classes use...</td>\n",
277 |        "    </tr>\n",
278 |        "    <tr>\n",
279 |        "      <th>6</th>\n",
280 |        "      <td>470</td>\n",
281 |        "      <td>Homegrown consumption of web services</td>\n",
282 |        "      <td>&lt;p&gt;I've been writing a few web services for a ...</td>\n",
283 |        "    </tr>\n",
284 |        "    <tr>\n",
285 |        "      <th>7</th>\n",
286 |        "      <td>580</td>\n",
287 |        "      <td>Deploying SQL Server Databases from Test to Live</td>\n",
288 |        "      <td>&lt;p&gt;I wonder how you guys manage deployment of ...</td>\n",
289 |        "    </tr>\n",
290 |        "    <tr>\n",
291 |        "      <th>8</th>\n",
292 |        "      <td>650</td>\n",
293 |        "      <td>Automatically update version number</td>\n",
294 |        "      <td>&lt;p&gt;I would like the version property of my app...</td>\n",
295 |        "    </tr>\n",
296 |        "    <tr>\n",
297 |        "      <th>9</th>\n",
298 |        "      <td>810</td>\n",
299 |        "      <td>Visual Studio Setup Project - Per User Registr...</td>\n",
300 |        "      <td>&lt;p&gt;I'm trying to maintain a Setup Project in &lt;...</td>\n",
301 |        "    </tr>\n",
302 |        "  </tbody>\n",
303 |        "</table>\n",
304 |        "</div>"
305 |       ],
306 |       "text/plain": [
307 |        "    Id                        ...                                                                       Body\n",
308 |        "0   80                        ...                          <p>I've written a database generation script i...\n",
309 |        "1   90                        ...                          <p>Are there any really good tutorials explain...\n",
310 |        "2  120                        ...                          <p>Has anyone got experience creating <strong>...\n",
311 |        "3  180                        ...                          <p>This is something I've pseudo-solved many t...\n",
312 |        "4  260                        ...                          <p>I have a little game written in C#. It uses...\n",
313 |        "5  330                        ...                          <p>I am working on a collection of classes use...\n",
314 |        "6  470                        ...                          <p>I've been writing a few web services for a ...\n",
315 |        "7  580                        ...                          <p>I wonder how you guys manage deployment of ...\n",
316 |        "8  650                        ...                          <p>I would like the version property of my app...\n",
317 |        "9  810                        ...                          <p>I'm trying to maintain a Setup Project in <...\n",
318 |        "\n",
319 |        "[10 rows x 3 columns]"
320 |       ]
321 |      },
322 |      "execution_count": 3,
323 |      "metadata": {},
324 |      "output_type": "execute_result"
325 |     }
326 |    ],
327 |    "source": [
328 |     "ques.drop([\"OwnerUserId\",\"CreationDate\",\"ClosedDate\",\"Score\"], axis=1, inplace=True)\n",
329 |     "ques.head(10)"
330 |    ]
331 |   },
332 |   {
333 |    "cell_type": "code",
334 |    "execution_count": 4,
335 |    "metadata": {
336 |     "_uuid": "b1305b3807f362fec50a13a76d68d35fa45c3380"
337 |    },
338 |    "outputs": [],
339 |    "source": [
340 |     "import re \n",
341 |     "\n",
342 |     "def rem_html_tags(body):\n",
343 |     "    regex = re.compile('<.*?>')\n",
344 |     "    return re.sub(regex, '', body)"
345 |    ]
346 |   },
347 |   {
348 |    "cell_type": "code",
349 |    "execution_count": 5,
350 |    "metadata": {
351 |     "_uuid": "f0340076f170cc798549f9da6ce46962d9f81c6b"
352 |    },
353 |    "outputs": [
354 |     {
355 |      "data": {
356 |       "text/html": [
357 |        "<div>\n",
358 |        "<style scoped>\n",
359 |        "    .dataframe tbody tr th:only-of-type {\n",
360 |        "        vertical-align: middle;\n",
361 |        "    }\n",
362 |        "\n",
363 |        "    .dataframe tbody tr th {\n",
364 |        "        vertical-align: top;\n",
365 |        "    }\n",
366 |        "\n",
367 |        "    .dataframe thead th {\n",
368 |        "        text-align: right;\n",
369 |        "    }\n",
370 |        "</style>\n",
371 |        "<table border=\"1\" class=\"dataframe\">\n",
372 |        "  <thead>\n",
373 |        "    <tr style=\"text-align: right;\">\n",
374 |        "      <th></th>\n",
375 |        "      <th>Id</th>\n",
376 |        "      <th>Title</th>\n",
377 |        "      <th>Body</th>\n",
378 |        "    </tr>\n",
379 |        "  </thead>\n",
380 |        "  <tbody>\n",
381 |        "    <tr>\n",
382 |        "      <th>0</th>\n",
383 |        "      <td>80</td>\n",
384 |        "      <td>SQLStatement.execute() - multiple queries in o...</td>\n",
385 |        "      <td>I've written a database generation script in S...</td>\n",
386 |        "    </tr>\n",
387 |        "    <tr>\n",
388 |        "      <th>1</th>\n",
389 |        "      <td>90</td>\n",
390 |        "      <td>Good branching and merging tutorials for Torto...</td>\n",
391 |        "      <td>Are there any really good tutorials explaining...</td>\n",
392 |        "    </tr>\n",
393 |        "    <tr>\n",
394 |        "      <th>2</th>\n",
395 |        "      <td>120</td>\n",
396 |        "      <td>ASP.NET Site Maps</td>\n",
397 |        "      <td>Has anyone got experience creating SQL-based A...</td>\n",
398 |        "    </tr>\n",
399 |        "    <tr>\n",
400 |        "      <th>3</th>\n",
401 |        "      <td>180</td>\n",
402 |        "      <td>Function for creating color wheels</td>\n",
403 |        "      <td>This is something I've pseudo-solved many time...</td>\n",
404 |        "    </tr>\n",
405 |        "    <tr>\n",
406 |        "      <th>4</th>\n",
407 |        "      <td>260</td>\n",
408 |        "      <td>Adding scripting functionality to .NET applica...</td>\n",
409 |        "      <td>I have a little game written in C#. It uses a ...</td>\n",
410 |        "    </tr>\n",
411 |        "  </tbody>\n",
412 |        "</table>\n",
413 |        "</div>"
414 |       ],
415 |       "text/plain": [
416 |        "    Id                        ...                                                                       Body\n",
417 |        "0   80                        ...                          I've written a database generation script in S...\n",
418 |        "1   90                        ...                          Are there any really good tutorials explaining...\n",
419 |        "2  120                        ...                          Has anyone got experience creating SQL-based A...\n",
420 |        "3  180                        ...                          This is something I've pseudo-solved many time...\n",
421 |        "4  260                        ...                          I have a little game written in C#. It uses a ...\n",
422 |        "\n",
423 |        "[5 rows x 3 columns]"
424 |       ]
425 |      },
426 |      "execution_count": 5,
427 |      "metadata": {},
428 |      "output_type": "execute_result"
429 |     }
430 |    ],
431 |    "source": [
432 |     "ques['Body'] = ques['Body'].apply(rem_html_tags)\n",
433 |     "ques.head()"
434 |    ]
435 |   },
436 |   {
437 |    "cell_type": "code",
438 |    "execution_count": 6,
439 |    "metadata": {
440 |     "_uuid": "af150557dcf11050e0e7dbc8bcbb0ef2c5015636"
441 |    },
442 |    "outputs": [],
443 |    "source": [
444 |     "ques.to_csv('question_clean.csv',index=False)"
445 |    ]
446 |   },
447 |   {
448 |    "cell_type": "code",
449 |    "execution_count": 7,
450 |    "metadata": {},
451 |    "outputs": [],
452 |    "source": []
453 |   }
454 |  ],
455 |  "metadata": {
456 |   "kernelspec": {
457 |    "display_name": "Python 3",
458 |    "language": "python",
459 |    "name": "python3"
460 |   },
461 |   "language_info": {
462 |    "codemirror_mode": {
463 |     "name": "ipython",
464 |     "version": 3
465 |    },
466 |    "file_extension": ".py",
467 |    "mimetype": "text/x-python",
468 |    "name": "python",
469 |    "nbconvert_exporter": "python",
470 |    "pygments_lexer": "ipython3",
471 |    "version": "3.6.7"
472 |   }
473 |  },
474 |  "nbformat": 4,
475 |  "nbformat_minor": 1
476 | }
477 | 


--------------------------------------------------------------------------------
/Stackoverflow Tags Map & Model.ipynb:
--------------------------------------------------------------------------------
   1 | {
   2 |  "cells": [
   3 |   {
   4 |    "cell_type": "code",
   5 |    "execution_count": 1,
   6 |    "metadata": {
   7 |     "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
   8 |     "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   9 |    },
  10 |    "outputs": [
  11 |     {
  12 |      "name": "stdout",
  13 |      "output_type": "stream",
  14 |      "text": [
  15 |       "['stackoverflow-clean-questions-file-v2', 'stackoverflow']\n"
  16 |      ]
  17 |     }
  18 |    ],
  19 |    "source": [
  20 |     "# This Python 3 environment comes with many helpful analytics libraries installed\n",
  21 |     "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n",
  22 |     "# For example, here's several helpful packages to load in \n",
  23 |     "\n",
  24 |     "import numpy as np # linear algebra\n",
  25 |     "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
  26 |     "\n",
  27 |     "# Input data files are available in the \"../input/\" directory.\n",
  28 |     "# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n",
  29 |     "\n",
  30 |     "import os\n",
  31 |     "print(os.listdir(\"../input\"))\n",
  32 |     "\n",
  33 |     "# Plotting Libs\n",
  34 |     "import matplotlib.pyplot as plt\n",
  35 |     "import matplotlib.cm as cm\n",
  36 |     "# magic function\n",
  37 |     "%matplotlib inline\n",
  38 |     "\n",
  39 |     "import collections"
  40 |    ]
  41 |   },
  42 |   {
  43 |    "cell_type": "code",
  44 |    "execution_count": 2,
  45 |    "metadata": {
  46 |     "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
  47 |     "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"
  48 |    },
  49 |    "outputs": [],
  50 |    "source": [
  51 |     "df_tags = pd.read_csv('../input/stackoverflow/Tags.csv', encoding='iso-8859-1')"
  52 |    ]
  53 |   },
  54 |   {
  55 |    "cell_type": "code",
  56 |    "execution_count": 3,
  57 |    "metadata": {},
  58 |    "outputs": [],
  59 |    "source": [
  60 |     "def plot_tags(tagCount):\n",
  61 |     "    \n",
  62 |     "    x,y = zip(*tagCount)\n",
  63 |     "\n",
  64 |     "    colormap = plt.cm.gist_ncar #nipy_spectral, Set1,Paired  \n",
  65 |     "    colors = [colormap(i) for i in np.linspace(0, 0.8,50)]   \n",
  66 |     "\n",
  67 |     "    area = [i/4000 for i in list(y)]   # 0 to 15 point radiuses\n",
  68 |     "    plt.figure(figsize=(9,8))\n",
  69 |     "    plt.ylabel(\"Number of question associations\")\n",
  70 |     "    for i in range(len(y)):\n",
  71 |     "        plt.plot(i,y[i], marker='o', linestyle='',ms=area[i],label=x[i])\n",
  72 |     "\n",
  73 |     "    plt.legend(numpoints=1)\n",
  74 |     "    plt.show()"
  75 |    ]
  76 |   },
  77 |   {
  78 |    "cell_type": "code",
  79 |    "execution_count": 4,
  80 |    "metadata": {},
  81 |    "outputs": [
  82 |     {
  83 |      "name": "stdout",
  84 |      "output_type": "stream",
  85 |      "text": [
  86 |       "[('javascript', 124155), ('java', 115212), ('c#', 101186), ('php', 98808), ('android', 90659), ('jquery', 78542), ('python', 64601), ('html', 58976), ('c++', 47591), ('ios', 47009)]\n"
  87 |      ]
  88 |     },
  89 |     {
  90 |      "data": {
  91 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHVCAYAAAAdLJRmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd4lFXa+PHveSYz6QkkoSSUDSg1FQgoIB0FxQIoy8uLQkTZn21ffS1r3dVVd1fXXtfdtUQUFUVRX3FFkaaASsDQqxBqIJBASJ92fn/MJAZImSQzk3Z/ritXMuc5zzn3REzunOcUpbVGCCGEEKItMpo6ACGEEEKIpiKJkBBCCCHaLEmEhBBCCNFmSSIkhBBCiDZLEiEhhBBCtFmSCAkhhBCizQpo6gCEEEIIca74+xZbgARgIBAHBALlwBFgA7Al+4lJtqaLsHVQso+QEEII0TzE37fYBFwG/AG4ACjD9fQmBFCABkoAJxAE/Aj8Hfgy+4lJjqaIuaWTREgIIYRoYvH3LVbAtcCzuEZ+wutxeyGukaI7gXezn5gkv9jrQRIht5iYGB0fH9/UYQghhGhjHJZwihKnYovoCgGWhjdkt2I+fZDwLZ9gWIu8F2ALtX79+hNa6w511ZNEyC0tLU1nZmY2dRhCCCHakC2HC5jx7x8otTqwOxv/+zjAUARbTLw/90ISu0R6IcKWSym1XmudVlc9WTUmhBBCNIHNhwr47T/XUlhm90oSBGB3agrL7Pz2n2vZcrjAK222dpIICSGEEH527HQZ//36D5RYfTO/ucTqYMa/fyD3dJlP2m9NZPm8D1jtTnYdK2TrkQKOnS6n3O4kMMCgU0QgCXGR9OkcjtkkOagQQrRFWmvuWJBFqY+SoAqlVgd3LMhi/o0XoJTyaV8tmSRCXuJwapbvyOW1lb+QdfAUgWYDpxPKbA40rjWPQWYThgHlNiep3dpx06jzGNO3IyZD/oEKIURb8dG6/fy8P99rj8NqYndqNuzP5+UvfmL8+RE+7aspBQUF0bVrV8xmc4Pul8nSbg2dLK215pMNh3l88TasdifF9cjwQy0mLAEGD03qz9SBXSRjF0KIVs7h1Ax8dAkFZf7b8qd9iJnMhy5ulX90a63Jy8ujsLCQHj16nHFNJkv7gesZ74889OkWTpbY6pUEARRbHZwssfHQp1uY+fqP8ixXCCFaueU7crHanX7t02p3smJnrl/79BelFNHR0ZSVNfz3pyRCDbTlcAHjn13Jun35lNoal9mX2hz8tC+fcc+ulFn+QgjRir228hdK7f59ElNsdfDayl/82qc/NfZpiiRCDSBLHoUQQtSX1e4k6+CpJun75wOnsDn8OxLVUkgiVE+y5FEIIURD7DpWSKC5aX7tBpoNdh0rPKNs2LBhTRLL2Y4cOcI111xTa51Tp07x6quv+qR/SYTqwd9LHmUiuxBCtB5bjxTgbKJBGa0552nDmjVrmiaYKux2O3FxcSxcuLDWepIINROfbDhM1oFTflny+POBU3yy4bBP+xFCCOE/x06XU9bIOaUNVWp1kHu6/IyysLAwioqKGDduHAMHDiQpKYnPPvsMgPvuu49XXnmlsu4jjzzC008/XWP94uJiJk2aREpKComJiSxYsACAdevWMWzYMFJSUhgyZAiFhYVkZGRw5ZVXMnbsWMaNG0d2djaJiYkAZGRkcNVVVzF69Gh69erFn//858p4fvnlF1JTU7nnnnu8+r2RfYQ85HBqHl+8rdEToz1VanPw+OJtTB7QpVUueRRCiLam3O6kvn9Gh1NCgpFNF05gVnZsOoDDxLDVGU8hIR63o939ny0oKIhFixYRERHBiRMnuPDCC7nyyiuZPn06d9xxB7feeisAH374IUuWLKmx/ldffUVcXByLFy8GoKCgAKvVyvTp01mwYAGDBw/m9OnTBAcHA7BhwwY2bdpEVFQU2dnZZ8T0008/sWXLFkJCQhg8eDCTJk3iiSeeYMuWLWRlZdXzO1g3SYQ81JRLHsf16+TXfoUQQnhfYICBgjqToVjyuC7ga642fUcUhZRiwYQThUajcGAQjJV8wvnYMYJ37JeQQ3StbSp3/2fTWvPAAw+watUqDMPg8OHDHDt2jAEDBpCbm8uRI0c4fvw47du3p1u3bthstmrrJyUlcdddd3Hvvfdy+eWXM2LECDZv3kxsbCyDBw8GICLi100dL774YqKioqqN9eKLLyY62vV+pk6dyvfff8/kyZPr+K41nCRCHnpt5S/13ieosSqWPEoiJIQQLV+niECCzKYanyx04CR/M7/OCGMLoAlUdgDMlFbfHqe4wfQf5pi+4ntnIvfb5nKcdtXWDbaY6BgReE75/PnzOX78OOvXr8dsNhMfH1+5J8+0adNYuHAhR48eZfr06bXW7927Nxs2bODLL7/koYceYty4cUyZMqXG70VoaGiN185eDu/rzYZljpAHZMmjEEKIxkqIi8So9reu5irje5YH3sVIYxOBylaZBNUlUNkJUjZGGptYHngnVxnfU92Yk1KQ2CXynPKCggI6duyI2Wxm+fLl7N+/v/La9OnT+eCDD1i4cCHTpk2rtf6RI0cICQnh2muv5Z577mHDhg306dOHnJwc1q1bB0BhYSF2e93v65tvviE/P5/S0lI+/fRThg8fTnh4OIWFhXXe2xAyIuSBiiWP9nL/T3KrWPKYEHfuP2AhhBAtR+9O4ZTbzv7DVvNwwDymm1YQosqrvc8TFuXAgoO/md8g1fELf7bPwvVAzKXc5qR3p/Az7lFKMXPmTK644gqSkpJIS0ujb9++ldcTEhIoLCykS5cuxMbGAtRYf/Pmzdxzzz0YhoHZbOYf//gHFouFBQsW8Pvf/57S0lKCg4NZunRpne9lyJAhXH311Rw6dIhrr72WtDTXKRnDhw8nMTGRSy+9lKeeeqrB36uzyVljbrWdNbZg3QEe+dx/E6WrCrGYePiK/kwf3N3vfQshhPCua/6xhsz9J92vNI8EvM1vTSsblQSdrUQHssAx+oxkaHB8ez666dd9g/Ly8hg4cOAZI0DNQUZGBpmZmbz88sv1um/79u3069fvjDI5a8yLmtuSRyGEEC3TTaPOIzjAlZxcZaz2ehIEEKLKmW5awZXGasB1wPdNo86rvH7kyBGGDh3K3Xff7dV+Wyp5NOaBhix59JaaljwKIYRoecb07YjZpAiz5/MX85teT4IqhKhy/mp+k7XliTjMHRndp2Pltbi4OHbt2uWTfhsrPT2d9PR0v/YpiZAHPF3y6As1LXkUQgjR8pgMxdy0KBLX/R0LNp/2ZcHGk5Y3OHnZPNmPrhaSCHmgriWPvlTTkkchhBAt04TYEuKNzVjw7e8Ui3JwkdqE+XyfdtPiyVCDB2pe8uh7NS15FEII0TJF7fkYs8k/IzRmk4Fa97pf+mqpJBHyQPVLHv2juiWPQgghWq7I7P+gHFa/9KUc5ZD1vl/6aqkkEfKAJcAgtVv1u3X62oDu7TCb5D+TEEK0CmUFBFj9vEFvSR6UnT6neNiwYdVUbnvkN6yHbhp1HqEWk1/7PHvJoxBCiBYuZxNOk5/nfZqDIWfjOcVr1qzxbxzNlCRCHhrTtyMWP6/eCjSbzljyKIQQooUrOAjaz1MtnA5Xv2cJCwujqKiIcePGMXDgQJKSkvjss88AuO+++3jllVcq6z7yyCM8/fTTNdZvySQR8pDJUDw0qT/BZv+MCgWbTTx4WT9Z8iiEEK2Jw4ry+4kOGmqYkxQUFMSiRYvYsGEDy5cv56677kJrzfTp0/nwww8r63344YdMnz69xvotmSyfr4epA7vw8YZD/LQvH7vTd//hAwzFgO7tmDqwi8/6EEII0QRMFrSPT1M/lwKTpdorWmseeOABVq1ahWEYHD58mGPHjjFgwAByc3M5cuQIx48fp3379nTr1g2bzVZt/c6dO/v5PXmPJEL1oJTiuempjH92JYVlnp0M3BDBFhPPT09F+f1/FiGEED4V2Q2UZw9jCpVic2AgWwMt7LKYKVOKIK3pbbWRUG4lqbyccE9GYwyTq99qzJ8/n+PHj7N+/XrMZjPx8fGUlZUBMG3aNBYuXMjRo0eZPn16nfVbKkmE6qlTRBDvz72Q3/5zLSVW72+GFWIx8f7cC+kYEeT1toUQQjSx2GQMe+3Hamy1mMmIjGB5SDBmoEwp7FX+MF6qNUFaYwPGlJSSXnCaBGstu1TbSiE2pdpLBQUFdOzYEbPZzPLly884hHX69OnMnTuXEydOsHLlyjrrt1QyR6gBErtE8uH/G0p4UAABXprDE2AoIoIC+PD/DZUNFIUQorUKisQeWP12LMVK8VBMFOmxnfg6NIRyw6DIMM5IggDsSlFkGJQbBl+HhpAe24mHYqIorukpQkg0BEWcU6yUYubMmWRmZpKUlMS8efPo27dv5fWEhAQKCwvp0qULsbGxALXWb6lkRKiBErtE8u2do7hjQRY/HzjVqOM3gs0mBnRvx/PTU2UkSAghWrmC+EuJ2b3gjAnMe80BzOnciUJDYa3HUQZOpShTiq9CQ/g+OJg3jx6jp63K1A1TIKTOOOe+vLw8oqKiiImJYe3atTW2v3nz5jNe11W/JZIRoUboGBHE/Bsv4PHJibQPMdd7n6FQi4n2IWb+MiWR+TdeIEmQEEK0ASd7XYPrSG2XveYAZsZ2Jt9k1CsJqqrcMMg3GcyM7cxe81ljHIPnnvHyyJEjDB06lLvvvrtBfbU2MiLUSEoprh7UlckDurBiZy6vrfyFnw+cItBsoDWUWh1oXP/kgy0mlHIdmzGgeztuGnUeo/t0lCXyQgjRhthDOsF5Y2DPtxRrO3M6d6LYUI1eTaaVotiAOZ07sfjQEUJVAJw3FiLPXIEcFxfHrl27GtVXayKJkJeYDMW4fp0Y168TNoeTXccK2XK4gNzT5ZTbnQQGGHSMCCSxSyS9O4XLsRlCCNGWXfECvDSIv0WEU+SFJKiCVooiQ/FEdHseO2119SNqJYmQD5hNBglxkSTEyaRnIYQQ1QjvzNYx97Bkx+uUe/mpQLlh8FVoKDPS7qB/eCevtt0a+WxYQin1plIqVym1pUrZU0qpHUqpTUqpRUqpdlWu3a+U2qOU2qmUmlClfKK7bI9S6r4q5T2UUj+6yxcopSzu8kD36z3u6/G+eo9CCCFEQ2VYD2P10dQIq6HIKD/sk7ZbG18+n8kAJp5V9g2QqLVOBnYB9wMopfoD/wUkuO95VSllUkqZgFeAS4H+wAx3XYAngee01ucDJ4Eb3OU3ACfd5c+56wkhhBDNRqG1kOUHl+OrU8ecwLKDyyi0Fvqoh9bDZ4mQ1noVkH9W2dda64p1fT8AXd1fXwV8oLUu11rvA/YAQ9wfe7TWe7XWVuAD4Crl2nJ5LLDQff/bwOQqbb3t/nohME7JFs1CCCGakc0nNmM2zD7tw2yY2XJiS90Vz1JcXMz48eMBuOiii7DbfXeSQnPQlDN25wD/cX/dBah6NO4hd1lN5dHAqSpJVUX5GW25rxe4659DKfU7pVSmUirz+PHjjX5DQgghhCe2nthKmd23R1OUOcrYlret3vetXbuWoUOHcvLkSUJDQwkIaN3TiZskEVJKPQjYgflN0X8FrfW/tNZpWuu0Dh06NGUoQggh2pBdJ3dh174dabE77ezI31Hj9Xnz5pGcnExKSgrXXXcdv/zyC6mpqVx77bW89957DBo0iI0bN5Kamkpubq5PY21Kfk/zlFLpwOXAOK0rT4s7DFQ9Ea6ru4wayvOAdkqpAPeoT9X6FW0dUkoFAJHu+kIIIUSzUObwz0Gl5Y7qzzXbunUrjz/+OGvWrCEmJob8/HyioqLIyspi0qRJzJs3j5dffpm0tDQmTZrkl1ibil9HhJRSE4E/AFdqrUuqXPoc+C/3iq8eQC/gJ2Ad0Mu9QsyCa0L15+4Eajlwjfv+2cBnVdqa7f76GmBZlYRLCCGEaHJBJv+cJBBoCqy2fNmyZUybNo2YmBgAoqKiKq/l5uYSHR3Npk2bSEmp/rDW1sSXy+ffB9YCfZRSh5RSNwAvA+HAN0qpLKXUawBa663Ah8A24CvgVq21wz3acxuwBNgOfOiuC3AvcKdSag+uOUBvuMvfAKLd5XcClUvuhRBCiOagd/veBKi6H8oop6brcc3gnU6GbXMyeKeTrsc1yln33/cBRgB9ozw/FPWmm24iMTGR3bt3k5qayldffcXll1/Oc88953EbLZHPHo1prc895e3XZKW6+n8B/lJN+ZfAl9WU78W1quzs8jJgWr2CFUIIIfwoISaBoIAgimxF51xTTs2AXzRX/uik1xGwG+A0QGnQCgwnBDhhdxx8foHBz+cpdDX7EQWZgugf3f+ccoCxY8cyZcoU7rzzTqKjo8nPz+e1117jo48+4sCBA1x99dX84Q9/4MMPP/T6e29uWvdUcCGEEKIZSopJwua0nVPe96Dm9s8chJRDsPtwerOj+jb6H4Qex5yUBMILV5nY0e3MZMjmtJEYk1jtvQkJCTz44IOMGjUKk8nEgAEDyMjIYOXKlcyaNYvvvvuOUaNGNeo9thSSCAkhhBB+Fm4JZ0y3MXy9/2uc2olyatKXOhm7URNYj8VkwVbXx4MfOFiWosgYb6ANhaEMxnYfS7glvMZ7Z8+ezezZs88oe/nllwEYMuScBy6tlpz8KYQQQjSB9MR0LIYF5dTc9YmTMfVMgqoKtMOYja52lFNjMSykJ6R7Nd7WShIhIYQQogkkRCcwIX4CN3yrSN6nCWrktkJBdkjep7nhW8XE+Ik1zg8SZ5JHY6JuZQWQswkKDoLDCiYLRHaD2GQIimzq6IQQosX6X9MEDmV9jMVLeysG2WF0lp1uAWcf9SlqIomQqF7BIfjpddj4PpTkgTkYnA5cR/kZYJjAVgoh0ZAyA4bcCJFd62pVCCGEm3Y4yL/vj15LgipY7JB374O0X/YtymTybuOtkCRC4kyFR+H/bodflgPaNQIEUH7u6gYAio7CD6/AD6/CeWPgihchvJPfwhVCiJaqaNUqHEW+OR3eWVhI0XffET56tE/ab01kjpBw0Ro2LoCXBsGeb8FR/msSVBeH1VV/z7fw0kBXO7KZtxBC1CrvjTfRxSV1V2wAZ0kJ+a/XuHWfqEISIeFKWv5zL3xxB1iLoJq9LTzitLnu/+IOV3uSDAkhRLW0w0HZxo0+7aN040a0o4ZNiGoQHx/PiRMnfBRR8ySJUFunNfznD/DzO2Dz0l8mthJXe5IMCSFEtax796LMZp/2ocxmrPv2+bSP1kASobZu04fw87veS4IqVCRDmz/ybrtCCNEKlGdng+HjX8GGgTU7u9pL2dnZ9O3bl5kzZ9KvXz+uueYaSkpcvwdeeuklBg4cSFJSEjt27ADgkUce4brrrmPo0KH06tWLf//7376N3Y8kEWrLCo/C4ju9nwRVsJXAF/8Lhcd8074QQrRQ2mr1/Yi51jjLy2u8vHPnTm655Ra2b99OREQEr776KgAxMTFs2LCBm2++maeffrqy/qZNm1i2bBlr167l0Ucf5ciRI76N308kEWrL/u92sNf8P4lX2Mtd/QghhKikLBZQ5x6U6t1OFEZgYI2Xu3XrxvDhwwG49tpr+f777wGYOnUqAIMGDSK7yojSVVddRXBwMDExMYwZM4affvrJd7H7kSRCbVXBIdcS+YZOjPaU0wa/LIOCw77tRwghWpDA+HhwOn3bidOJJT6+xsvqrESs4nWgO3kymUzY7fY667d0kgi1VT+9DvhxIvO61vM8WQghGsvSsyfaVvcfohooDYrmRFQCuTGpnIhKoDQo2qOf3tpmw9KjR43XDxw4wNq1awF47733uOiii2pt77PPPqOsrIy8vDxWrFjB4MGDPYii+ZMNFduqje97vk9QYznKIet9GP+If/oTQohmTplMBKWkUJqZWe3102HdONhtHMdjkl319a+jR1q5xjA6nNhEt4PfElF0sNo2glNSat1Zuk+fPrzyyivMmTOH/v37c/PNN/PSSy/VWD85OZkxY8Zw4sQJ/vjHPxIXF1fn+2wJJBFqi8oKXMdm+FNJHpSdhqAI//YrhBDNVPQNczi8fdsZmypazeFs6zeLU5Hn4VQBruOManCsw0COxyTT7tQv9N8xD4vt112qjZAQom68odb+AwICePfdd88oqzonKC0tjRUrVlS+Tk5OZt68eR6+u5ZDHo21RTmbXGeH+ZM5GHJ8u3mYEEK0JGEjR2IKC698fTLyfNZe8DAn2/XGaQqsNQkCwDDhNAVysn1v132R5/96KTycsBEjfBV6qyKJUFtUcNB9gKofOR2ufoUQQgCux2NdnnkaFRTEycjz2Zh8C46AYLRRv4c12gjAERDMxuRbOBl5Piow0NVuLY/F4uPj2bJli8d9PPLII9x99931iqulkEdjbZHDiusUeX/S/puTJIQQLURIWhpBk6ez6VCSaxSoEZymQDYl3cTErpsJSUvzUoStn4wItUUmC/7/T6/c/QohhKhqU/sJOAO88/PRGWBhc9QEr7TVVkgi1BZFdqv72bO3GSZXv0IIISrl7j9Nzi8FaLzzM1lj4sieAnL3n/ZKe22BJEJtUWwy2Er926etFGJT/NunEEI0c1lLD+KweXeqgsPuJGupzMn0lCRCbVFQJIRE+7fPkGhZOi+EEFVordm38bjXjxzTTtzt+m7T3LCwsHrfM2zYsGrL09PTWbhwYWNDajBJhNqqlBn+m7NjCoTUGf7pSwghWojTJ8p82n5hnm/bP1vV4ziqs2bNGj9FUj+SCLVVQ24E/HhOzOC5/utLCCFagJNHizEM3/wcNgxFfk5xrXUmT57MoEGDSEhI4F//+hfgGul58MEHSUlJ4cILL+TYsWMA7Nu3j6FDh5KUlMRDDz1U2caKFSsYMWIEV155Jf379wfg2WefJTExkcTERJ5//vnKuhWjSFprbrvtNvr06cP48ePJzc316nuvL0mE2qrIrnDeGDDMvu3HMMN5YyGyi2/7EUKIFsZhc/rsxEftbr82b775JuvXryczM5MXX3yRvLw8iouLufDCC9m4cSMjR47k3/92nRN5++23c/PNN7N582ZiY2PPaGfDhg288MIL7Nq1i/Xr1/PWW2/x448/8sMPP/Dvf/+bn3/++Yz6ixYtYufOnWzbto158+Y1+UiRJEJt2RUvQEDj9q2oU0Cgqx8hhBBnMJkNn43LK3f7tXnxxRcrR34OHjzI7t27sVgsXH755QAMGjSo8siN1atXM2OGa4rDddddd0Y7Q4YMoYf7cNfvv/+eKVOmEBoaSlhYGFOnTuW77747o/6qVauYMWMGJpOJuLg4xo4d64V33HCSCLVl4Z1h0rNgDvFN++YQuPw5CO/km/aFEKIFa985FKfTN2NCTqcmKja0xusrVqxg6dKlrF27lo0bNzJgwADKysowm80o5UrPTCbTGfN+KsrPFhpacz8tgSRCbV3yb2HAdd5PhswhMHAWJE3zbrtCCNFKRMQE+bT98Oia2y8oKKB9+/aEhISwY8cOfvjhh1rbGj58OB988AEA8+fPr7HeiBEj+PTTTykpKaG4uJhFixYx4qwzz0aOHMmCBQtwOBzk5OSwfPnyerwr75NEqK1TCi590rvJUEUSNPEJV/tCCCHOoZSiR0oHr/+YVAbudmtueOLEidjtdvr168d9993HhRdeWGubL7zwAq+88gpJSUkcPny4xnoDBw4kPT2dIUOGcMEFF3DjjTcyYMCAM+pMmTKFXr160b9/f2bNmsXQoUPr9wa9TPlyn4GWJC0tTWdmZjZ1GE1Ha9j8EXzxv2AvB6et/m0YZtecoMufc40ESRIkhBDn2L59O/369QNcO0svemYDduu5E5u1tuG07sZhP4C2H0NjQ2FGBXTCFNAdw9ILpc5d8BJgMZh69yA6dA8/51prVfV7WkEptV5rXeeha3LoqnBRyvWYrMco+L/b4ZdleHxQqskCKNfqsCtekDlBQgjhoY6/iSD2/HYc3nkSp8M1MKG1HXvpGhzlWbimPf/6h6kGtDUPp3U3lCzFFJhKQPAwlHL9OjdMirhe7dpUEtRYkgiJM4V3gv/+AAoOwbrXIet9KMkDczA4Hbj+N1Sus8Nspa4do1NnuPYJkiXyQghRb+PT+/Pun9bidDhwOvKwFn4CuhSobYNCV3LkKM/CYd2JJXwqhimaALPBuNn9/RJ3ayGJkKheZFcY/4jro6wAcjZBwUHXCJHJ4jpANTZFjs0QQohGComwMOmWZD5/fhllJ98HPBiJr2QHXYj19PuERM/kslvGEhLhp1MDWglJhETdgiKhx4i66wkhhGiQTvGhYP+c+iVBVVnB8X90ir/Cm2G1CbJqTAghhGhiaz6aj7W0qFFtWEsKWfNRzUvbRfUkERJCCCGakK2sjJ+/+gK7tbxR7dit5fy85AtsZf49bLWlk0RICCGEaEK7flyN8tLhq0opdv242qO6w4YN80qfLZ0kQkIIIUQTOrBlo9dGcWxlZRzcusmjur487LTq0RzNnSRCQgghRBM6tnePV9s76mF7YWFhaK257bbb6NOnD+PHj+eyyy5j4cKFAMTHx3PixAkAMjMzGT16NADFxcXMmTOHIUOGMGDAAD777DMAMjIyuPLKKxk7dizjxo1j1qxZfPrpp5X9zZw5s7JucyKrxoQQQogm1Ni5Qee0V+756NKiRYvYuXMn27Zt49ixY/Tv3585c+bUes9f/vIXxo4dy5tvvsmpU6cYMmQI48ePB2DDhg1s2rSJqKgoVq5cyXPPPcfkyZMpKChgzZo1vP322416b77gsxEhpdSbSqlcpdSWKmXTlFJblVJOpVTaWfXvV0rtUUrtVEpNqFI+0V22Ryl1X5XyHkqpH93lC5RSFnd5oPv1Hvf1eF+9RyGEEKKxAiyB3m0v0PPDXFetWsWMGTMwmUzExcUxduzYOu/5+uuveeKJJ0hNTWX06NGUlZVx4MABAC6++GKioqIAGDVqFLt37+b48eO8//77XH311QQENL/xF18+GssAJp5VtgWYCqyqWqiU6g/8F5DgvudVpZRJKWUCXgEuBfoDM9x1AZ4EntNanw+cBG5wl98AnHSXP+euJ4QQQjRLnXqe79W3V19eAAAgAElEQVT2OnupvYCAAJxO1xloZVXmMGmt+fjjj8nKyiIrK4sDBw5UnvMVGhp6RhuzZs3i3Xff5a233qpzpKmp+CwR0lqvAvLPKtuutd5ZTfWrgA+01uVa633AHmCI+2OP1nqv1toKfABcpVxH6o4FFrrvfxuYXKWtirG3hcA4VdsRvEIIIUQT6p6YgjnI81Gc2piDguiWkOxx/ZEjR7JgwQIcDgc5OTksX7688lp8fDzr168H4OOPP64snzBhAi+99BIVh7b//PPPNbafnp7O888/D0D//s3z6I/mMlm6C3CwyutD7rKayqOBU1pr+1nlZ7Tlvl7grn8OpdTvlFKZSqnM48ePe+mtCCGEEJ7rfcFwtFN7pS2tNb0vGO5RXaUUU6ZMoVevXvTv359Zs2YxdOjQyusPP/wwt99+O2lpaZhMpsryP/7xj9hsNpKTk0lISOCPf/xjjX106tSJfv36cf311zf8TflY83tY50da638B/wJIS0vzzr9CIYQQoh7MQUEMmHh5ozdVDLAEMmDC5R6NLuXl5REVFYVSipdffrmyPD09vfLrESNGsGvXrnPuDQ4O5p///Oc55enp6WfcD1BSUsLu3buZMWOG52/Ez5rLiNBhoFuV113dZTWV5wHtlFIBZ5Wf0Zb7eqS7vhBCCNEsDZs2k+CICKD6mRwKhdkIRNVwHaUIiYhk2G+vrbOvI0eOMHToUO6+++5GRFy3pUuX0q9fP37/+98TGRnp074ao7mMCH0OvKeUehaIA3oBP+H6F9FLKdUDV4LzX8B/a621Umo5cA2ueUOzgc+qtDUbWOu+vkxXPMgUQgghmqEAi4Wr73+U9/54F9aSUkBjNgKJD0ukT+QQQkzhOHFgYKLEUcjOgp/ILtqCzVkOSmEJDmbq/X8mwGyus6+4uLhqR3rAtReQt4wfP579+/d7rT1f8VkipJR6HxgNxCilDgEP45o8/RLQAVislMrSWk/QWm9VSn0IbAPswK1aa4e7nduAJYAJeFNrvdXdxb3AB0qpx4GfgTfc5W8A7yil9rj7+y9fvUchhBDCW6K7duO/H3uGj//2J+J1P3qHpQGaAMMCgMn9Kzs0IILk9iNJbj+KXUWZ7Fc7mHr/n4nu2q2W1kVNlAyWuKSlpenMzMymDkMIIUQrt3379srl5mfTWpP3wQ5KNuVi6LpnrziVk5CUjkRP70tbXiBd3fdUKbVea51Wwy2VmsscISGEEKLNO730AOXb8j1KggAMbVC+NZ/TSw/4OLLWSxIhIYQQohlwltopXHkIbXPW6z5tc1K48hDOspZz0GlzIomQEEII0QwUrz9GQ59uKQXFmce8G1AVf/3rXyu/zs7OJjEx0Wd9+ZskQkIIIUQzUPRd/UeDKmibk6LvDnk5ol9VTYRaG0mEhBBCiCamHU4cp62NasNx2op2eLYAKjs7m759+zJz5kz69evHNddcw5dffsnkyZMr63zzzTdMmTKF++67j9LSUlJTU5k5c6arL4eDuXPnkpCQwCWXXEJpaSkAWVlZXHjhhSQnJzNlyhROnjwJwOjRo7n33nsZMmQIvXv35rvvvmvUe/UmSYSEEEKIJqatTjAauerLUGirw+PqO3fu5JZbbmH79u1ERESwdetWduzYQcWRUxUHpT7xxBMEBweTlZXF/PnzAdi9eze33norW7dupV27dpVnkc2aNYsnn3ySTZs2kZSUxJ///OfK/ux2Oz/99BPPP//8GeVNTRIhIYQQookpiwGNPW/MqVEWU9313Lp168bw4a5zya699lpWr17Nddddx7vvvsupU6dYu3Ytl156abX39ujRg9TUVAAGDRpEdnY2BQUFnDp1ilGjRgEwe/ZsVq1aVXnP1KlTz6jfXDSXnaWFEEKINkuZDEwRFhwFDX88ZoqwoEyejyqdve+QUorrr7+eK664gqCgIKZNm0ZAQPVpQmBg4K/9mkyVj8ZqU3GPyWTCbm8+K9xkREgIIYRoBsJGdEWZG/ZrWZkNwkZ0rdc9Bw4cYO3atQC89957XHTRRcTFxREXF8fjjz9+xonxZrMZm81Wa3uRkZG0b9++cv7PO++8Uzk61JxJIiSEEEI0A6GDOtHQwx60htC0TvW6p0+fPrzyyiv069ePkydPcvPNNwMwc+ZMunXrdsZOzb/73e9ITk6unCxdk7fffpt77rmH5ORksrKy+NOf/lT/N+NncsSGmxyxIYQQwh9qO2Kj4Jv9FK2q3zJ6ZTYIG9mVyIt/4/E92dnZXH755WzZsuWca7fddhsDBgzghhtu8Li9ptaYIzZkjpAQQgjRTESM747jZBmlm094lAwps0FwUgwR47t7pf9BgwYRGhrKM88845X2WgJJhIQQQohmQilF+2m9MbUPonDlIZSCIlsJJ4xCrNixEECMM5wwcwhaQ9jIrkSM717vA1fj4+OrHQ1av369t95KiyGJkBBCCNGMKKWIGN+d412tfPfNCg7l52BohQYU4FSarlGxjLhkNF37ev44TFRPEiEhhBCiGXE6nSxevJhNmzZVrtRynDXgsz//MEc+/pDk5GQmTZqEYcjap4aS75wQQgjRTGitz0mCamKz2di0aROLFy/2U3StkyRCQgghRDOxb98+j5KgChXJ0N69e30cWesliZAQQgjRTHz//fceJ0EVbDYbq1evrtc92dnZJCYmnlOekZHBkSNHvNJWSyGJkBBCCNEMFBQUsH///gbdW3HWV2M1JBFq6SQREkIIIZqBnJycGs/2qktAQAA5OTn1usfhcDB37lwSEhK45JJLeOedd8jMzGTmzJmkpqZSWlpKfHw8999/P6mpqaSlpbFhwwYmTJjAeeedx2uvvdagWJsbSYSEEEKIZqC8vJyGnvagtaa8vLxe9+zevZtbb72VrVu30q5dO5RSpKWlMX/+fLKysggODgage/fuZGVlMWLECNLT01m4cCE//PADDz/8cINibW5k+bwQQgjRDAQGBtZ7Y8QKSqkzToT3RI8ePUhNTQVcO0pnZ2dXW+/KK68EICkpiaKiIsLDwwkPDycwMJBTp041KN7mREaEhBBCiGYgNjYWu93eoHvtdjuxsbH1uqdq4mQymWrsu6KeYRhn3GMYRoPjbU4kERJCCCGagcjISH7zm4btFB0fH09kZGSjYwgPD6ewsLDR7bQkkggJIYQQzcRFF12E2Wyu1z1ms5nhw4d7pf/09HRuuummysnSbYFq6MSs1iYtLU1nZmY2dRhCCCFaue3bt9OvX79qr2mt+eKLLzzeVNFsNpOcnMwVV1zh7TBblOq+p0qp9VrrtLrulcnSok0otBay+cRmtp7Yyq6TuyhzlBFkCqJ3+94kxCSQFJNEuCW8qcMUQrRxSikmTZoEUGcyVJEEVdQXDSOJkGjVtuZtJWNLBssPLsdsmCmzl2HXv07uW7p/KUEBQdicNsZ0G0N6YjoJ0QlNGLEQoq0zDIMrrriChIQEVq9eTXZ2NgEBAWitUUpht9uJj49n+PDh9OzZs6nDbfEkERKtUrGtmL/9+DeWZC/B6rTi1E7KHefusWHXdopsRQB8vf9rVhxcwYT4Cdx/wf2EmkP9HbYQQlTq2bMnPXv2pKCggJycHMrLywkMDCQ2NtYrE6OFiyRCotXZe2ovc5bModBWiNVh9fg+p3ZS5ijjq+yv+P7w97w54U16tpO/toQQTaeoeDdHjrzF8eNLcDhLMRnBaD0BU8D1hIX2aurwWgVZNSZalb2n9jLzy5nkl+XXKwmqqtxRTn5ZPjO/nMneU3KisxDC/7R2snPnI6xbN5kjOQux2U/hdJZjs5/iSM5C1q2bzM6dj6C1s6lDbfEkERKtRrGtmDlL5lBsK0bTuNWQGn1Ge0II4U+7dj3KkZyFOJ1lgOOsqw6czjKO5Cxk165HmyK8VkUSIdFq/O3Hv1FkK2p0ElRBoymyFfHEj094pT0hhPBEUfFujuR8hNNZ+z4+TmcpR3I+orh4j58ia50kERKtwta8rSzJXlLthOjGKHeU81X2V2zL2+bVdoUQoiYHD7yF01n3HkIATqeNAwff8noMo0ePrvHsMYCMjAweeeQRr/fbFCQREq1CxpYMrM6GzQmqi9VpJWNrhk/aFkKIsx0/voRzH4fVxMHx3K8a1M+8efNITk4mJSWF6667rkFttAayaky0eIXWQpYfXI7TR5MGndrJsgPLKLQWyqaLQgifc9TxSKyx9QG2bt3K448/zpo1a4iJiSE/P7/ebbQWkgiJFm/zic2YDbPXH4tVZTbMbDmxhaFxQ33WhxBCAJiMYJxOz3+emYzgevexbNkypk2bRkxMDABRUVG89dZbvPDCCwDs2bOHyy67DIvFQo8ePVi0aBF5eXmMGzcOgPz8fKxWK59++ikA77zzDklJSfWOozmQREi0eFtPbKXMXubTPsocZWzL2yaJkBDC5zp0mMCRnIV49ngsgA4dJ3ql3+uvv57rr78ecM0RysjIID4+vvJ6dHQ0WVlZgGuOUHZ2dquYJyRzhESLt+vkrjOOzfAFu9POjvwdPu1DCCEAunW/HsPw7AR6wwige7fr693H2LFj+eijj8jLywOQR2OeUkq1B7pprTf5KB4h6q3M4dvRoAq+fPRWX9rhwLp3L+XZ2WirFWWxEBgfj6VnT5TJ1NThCSEaISy0F3Gx09z7CNU8/8cwgomLvYbQ0PPr3UdCQgIPPvggo0aNwmQyMWDAADIyMhoRdctVZyKklFoBXOmuux7IVUqt1lrf6ePYhPBIkCnIL/0EmgL90k9NtMNB0apV5L3xJmUbN6LMZjAM0BqUAqcTbbMRlJJC9A1zCBs5UpIiIVqo3r3/BODeT8jGmY/JTBiGmbjYayrrNcTs2bOZPXt2tddWrFhR673p6ekN7re58WREKFJrfVopdSMwT2v9sFJKRoREs9G7fW+W7l/q08djAUYAfaP6+qz9upRkZnL4rrtxFBWii0sA0Lbq9xkpzczk8PZtmMLC6fLM04SkpfkzVCGEFyhl0KfPI3TpOpODBzM4nvtV5VljHTpOpHu36xs0EiTO5ckcoQClVCzwW+ALTxtWSr2plMpVSm2pUhallPpGKbXb/bm9u1wppV5USu1RSm1SSg2scs9sd/3dSqnZVcoHKaU2u+95USmlautDtF4JMQkEBfh2VCjIFET/6P4+7aM62uHg6GOPceDGudiPHatMguq8r7gE+7FjHLhxLkcfewzt8HRPEiFEcxIW2ot+ff/CyJHrGTN6GyNHrqdf379IEuRFniRCjwJLgD1a63VKqZ7Abg/uywDOnsp+H/Ct1roX8K37NcClQC/3x++Af4ArqQEeBi4AhgAPV0ls/gHMrXLfxDr6EK1UUkwSNg93YW0om9NGYkyiT/s4m3Y4OPQ/t3Pq40/QZQ2bB6XLyjj18Scc+p/bJRkSQohq1JkIaa0/0lona61vcb/eq7W+2oP7VgFnT0O/Cnjb/fXbwOQq5fO0yw9AO/co1ATgG611vtb6JPANMNF9LUJr/YPWWgPzzmqruj5EKxVuCWdMtzEYyjeLIA1lMLb7WL9vpnjsr3+lePXqBidBFXRZGcWrV3Psr3/1UmRCCH85Wm7jyb05XLB2G4nfb+GCtdt4cm8OR8t9+8dfW+LJZOkOuEZe4qvW11rPaUB/nbTWOe6vjwKd3F93AQ5WqXfIXVZb+aFqymvr4xxKqd/hGoGie/fu9X0vohlJT0xnxcEVPllBZjEspCeke73d2pSsW9eokaCz6bIyTi38mIhLL5U5Q0K0AFprXjxwjGezjwFQ7nQdJn3CBq8ezOXVg7ncGd+J/+neCffMENFAnvwJ/RkQCSwFFlf5aBT3SI53jglvYB9a639prdO01mkdOnTwZSjCxxKiE5gQP8HrK7sCTYFMjJ/o1/lB2uHg8N33eC0Jqmy3vJzDd90tj8iEaAFePHCM57NzKXfqyiSoQkXZ89m5vHjgWBNF2Hp4kgiFaK3v1Vp/qLX+uOKjgf0dcz/Wwv05111+GOhWpV5Xd1lt5V2rKa+tD9HK3X/B/YSZw1B4568jhSLMHMb9F9zvlfY8VbRqFY6iQp+07SwspOi773zSthDCO46W23g2+xilztrPTyx1Onk2+1iDHpMNGzasoeG1Op4kQl8opS7zUn+fAxUrv2bjGm2qKJ/lXj12IVDgfry1BLhEKdXePUn6EmCJ+9pppdSF7tVis85qq7o+RCsXag7lzQlvEmoObXQypFCV7YWYQ7wUoWfy3njT49Vh9eUsKSH/9Td80rYQwjsyDp+oV/159awPsGbNmnrf01p5kgjdjisZKlNKFbo/Ttd1k1LqfWAt0EcpdUgpdQPwBHCxUmo3MN79GuBLYC+wB/g3UDExOx94DFjn/njUXYa7zuvue34B/uMur6kP0Qb0bNeT+ZfNJyooqsGPyQJNgUQFRTH/svn0bNfTyxHWTjsclG3c6NM+SjdulMdjQjRji46dPOdxWE3KnZpPck/Wu4+wsDDANRfpnnvuITExkaSkJBYsWABATk4OI0eOJDU1lcTERL5rxSPJdU6W1lo3aKmM1npGDZfGVVNXA7fW0M6bwJvVlGcC56xn1lrnVdeHaDt6tuvJ4qmLeeLHJ/gq+yusTitOXfsQM7hWh1kMCxPjJ3L/Bff7fSQIwLp3L8psrnGzRG9QZjPWffsIPF/2IRGiOSp21P3zqqoie/3qV/XJJ5+QlZXFxo0bOXHiBIMHD2bkyJG89957TJgwgQcffBCHw0FJiW9GqZsDj84aU0pdCYx0v1yhtfZ4Y0UhmkKoOZTHLnqMGf1mkLE1g2UHlmE2zJQ5yrA7f92BOsAIIMgUhM1pY2z3saQnpDfJxokVyrOzXcdm+JJhYM3OlkRIiGYq1GRwoh5/C4UFNPxnxvfff8+MGTMwmUx06tSJUaNGsW7dOgYPHsycOXOw2WxMnjyZ1NTUBvfR3HmyfP4JYDAw3110u1JquNbavzNIhWiA/tH9+fvIv1NoLWTLiS1sy9vGjvwdlDvKCTQF0jeqL/2j+5MYk+j3fYKqo61W19lhPu1E4yxvPgfICiHONLVTe149mOvR47FAQzG1o/cPUBg5ciSrVq1i8eLFpKenc+eddzJr1iyv99MceDIidBmQqrXr2YJS6m3gZ0ASIdFihFvCGRo3lKFxQ5s6lFopi8V1gKpPO1EYgU17gKwQomazu8Tw6kEPFzxrmNUlpsF9jRgxgn/+85/Mnj2b/Px8Vq1axVNPPcX+/fvp2rUrc+fOpby8nA0bNrTpRAigHb/uEh3po1iEaPMC4+OhjiWzjeZ0YomP920fQogG6xxo5s74TjyfnVvrEvpgw+CO+I50DjQ3uK8pU6awdu1aUlJSUErx97//nc6dO/P222/z1FNPYTabCQsLY968eQ3uo7lTuo5heKXUDFwrr5YDCtdcofu01gt8H57/pKWl6czMzKYOQ7Rx2uFgZ+oAn0+W7pP1M8pk8lkfQoiabd++nX79+tVap6adpcH1OAwNd/aQnaUrVPc9VUqt11rXuZW+J6vG3ldKrcA1TwjgXq310YYEKoSonTKZCEpJodSHSXlwSookQUI0c0opbv9NZ6Z3jmbe4RN8knuSIruTsACDqR3bM6tLTKNGgsSvakyElFJ9tdY7lFID3UUVZ3vFKaXitNYbfB+eEG1P9A1zOLx9m082VTRCQoi68QavtyuE8I3OgWb+0DOWP/SMbepQWq3aRoTuxHUg6TPVXNPAWJ9EJEQbFzZyJKawcOy+SITCwwkbMcLr7Qoh6kdrLY+0vKSuKT51qTER0lr/zv3lpVrrM05/VEoFNapXIUSNlMlEl2ee5sCNc7168KoKDKTLM0/LYzEhmlhQUBB5eXlER0dLMtRIWmvy8vIICmp4WuLJqrE1wEAPyoQQXhKSlka7q6dy6uNPvJIMqaAg2l09lZC0OucNCiF8rGvXrhw6dIjjx483dSitQlBQEF27dq27Yg1qmyPUGegCBCulBkDlKZYRgP/PHhCijen0wAPYjh6jePXqRiVDKiiI0IsuotMDD3gxOiFEQ5nNZnr06NHUYQi32kaEJgDpQFfg2SrlhYD8RBXCx5TJRNcXX+DYX//a4JEhFRhIu6un0umBB+SRmBBCVMOTfYSu1lp/7Kd4mozsIySas5LMTA7fdTfOwkKcHhx+aISEYISH0+WZp+VxmBCiTfLmPkIfK6UmAQlAUJXyRxsXohDCUyFpaZy/7FuKvvuO/NffoHTjRpTZ7DqgVWvXsRxOJ9pmIzglhagbbyBsxAgZBRJCiDp4cujqa7jmBI0BXgeuAX7ycVxCiLMok4nw0aMJHz0a7XBg3bcPa3Y2zvJyjMBALPHxWHr0kORHCCHqwZNVY8O01slKqU1a6z8rpZ4B/uPrwIQQNVMmE4Hnn0/g+ec3dShCCNGiGR7UKXV/LlFKxQE2QLa4FEIIIUSL58mI0BdKqXbAU8AGXLtKv+7TqIQQQggh/MCTydKPub/8WCn1BRCktS7wbVhCCCGEEL5X24aKY7XWy5RSU6u5htb6E9+GJoQQQgjhW7WNCI0ClgFXVHNNA5IICSGEEKJFq+3Q1Yfdn6/3XzhCCCGEEP5T56oxpdRf3ZOlK163V0o97tuwhBBCCCF8z5Pl85dqrU9VvNBanwQu811IQgghhBD+4UkiZFJKBVa8UEoFA4G11BdCCCGEaBE82UdoPvCtUuot9+vrgbd9F5IQQgghhH94so/Qk0qpjcB4d9FjWuslvg1LCCGEEML3PBkRAvgZMONaNv+z78IRQgghhPAfT1aN/RbXafPXAL8FflRKXePrwIQQQgghfM2TEaEHgcFa61wApVQHYCmw0JeBCSGEEEL4mieJkFGRBLnl4dlqMyGEqJPWmtMnyjh5tBiHzYnJbNC+cygRMUEopZo6PCFEK+dJIvSVUmoJ8L779XTgS9+FJIRoC3L3nyZr6UH2bTwOgGEoNKAAp1MD0COlA6nju9HxNxFNF6gQolXzZNXYPe6DVy9yF/1La73It2EJIVqrktNWlmZsI2fPKRw2J1rXXHdP5jH2bTxO7PntGJ/en5AIi/8CFUK0CZ5Mlg4FPtNa3wn8E3Aopcw+j0wI0eoc3nWSd/+0lsM7T2K31p4EAWgNdquTwzvd9+066Z9AhRBthidzfVYBgUqpLsBXwHVAhi+DEkK0Pod3neSLlzdiK3PgdNSRAZ3F6dDYyhx88fJGSYaEEF7lSSKktNYlwFTgH1rraUCCb8MSQrQmJaetLH51E3ars1Ht2K1Ovnx1EyWnrV6KTAjR1nmUCCmlhgIzgcXuMpPvQhJCtDZLM7bhsDUuCapgtzn59u1tXmlLCCE8SYTuAO4HFmmttyqlegLLfRuWEKK1yN1/mpw9p+r9OKwmTofmyO5T5O4/7ZX2hBBtW52JkNZ6pdb6SveZYwZwQmv9P36ITQjRCmQtPei10aAKDruTrKUHvdqmEKJt8mTV2HtKqQj36rEtwDal1D2+D00I0dJprdm38Xidq8Pq3a4Td7teblgI0eZ48misv9b6NDAZ+A/QA9fKMSGEqNXpE2U+bb8wz7ftCyFaP08SIbN736DJwOdaaxuuU+iFEKJWJ48WYxi+OSbDMBT5OcU+aVsI0XZ4kgj9E8gGQoFVSqnfAI2apaiUul0ptUUptVUpdYe7LEop9Y1Sarf7c3t3uVJKvaiU2qOU2qSUGlilndnu+ruVUrOrlA9SSm123/OikgOLhGgSDpvTZ381aXf7QgjRGJ5Mln5Ra91Fa32ZdtkPjGloh0qpRGAuMARIAS5XSp0P3Ad8q7XuBXzrfg1wKdDL/fE74B/udqKAh4EL3G09XJE8uevMrXLfxIbGK4RoOJPZwFd/hSh3+0II0RieHLqKUmoSrk0Ug6oUP9rAPvsBP7o3aUQptRLXZo1XAaPddd4GVgD3usvnadesyB+UUu2UUrHuut9orfPd7XwDTFRKrQAitNY/uMvn8ev8JiGEH7XvHFp5gKq3OZ2aqNhQn7QthGg7PFk19hquE+d/j+uPsGnAbxrR5xZghFIqWikVAlwGdAM6aa1z3HWOAp3cX3cBqq6TPeQuq638UDXlQgg/i4gJqrtSI4RH+7Z9IUTr58m48jCt9SzgpNb6z8BQoHdDO9RabweeBL7GdXZZFuA4q47GDxOylVK/U0plKqUyjx8/7uvuhGhzlFL0SOmAt2fpKQN3uzL9TwjROJ4kQqXuzyVKqTjABsQ2plOt9Rta60Fa65HASWAXcMz9yAv351x39cO4RowqdHWX1VbetZry6uL4l9Y6TWud1qFDh8a8JSFEDVLHd/P6XB5TgMGAi7t7tU0hRNvkyU+nL5RS7YCngA24VpC915hOlVId3Z+745of9B7wOVCx8ms28Jn768+BWe7VYxcCBe5HaEuAS5RS7d2TpC8BlrivnVZKXeheLTarSltCCD/r+JsIYs9vh2HyzuiNYVLE9WpHh+7hXmlPCNG21TlZWmv9mPvLj5VSXwBBWuuCRvb7sVIqGtfo0q1a61NKqSeAD5VSNwD7gd+6636Jax7RHqAEuN4dV75S6jFgnbveoxUTp4FbgAwgGNckaZkoLUQTGp/en3f/tBanw1F35ToEmA3Gze7vhaiEEAKUbFHvkpaWpjMzM5s6DCFarcO7TvLFyxuxWxu+90+AxeDy21Lo0rt93ZWFEG2aUmq91jqtrnqyCYcQwi+69G7P5belYA4y1fsxmWFSWIJMkgQJIbxOEiEhhN906d2eax8dSpc+7QmwGHWuJlOGaxSoa9/2zHx0qCRBQgiv83RDxS649g6qrK+1XuWroIQQrVdIhIUr/yeV3P2nyVp6kH0bXVtXGIZC49qsrGITxh4pHUgd342Ov4louoCFEK1anYmQUupJXBsqbuPX/X40IImQEKLBOv4mgktuSEBrTWFeGfk5xThsTkxmg6jYUMKjg2SfICGEzxxQvj8AAB0qSURBVHkyIjQZ6KO1Lvd1MEKItkcpRURMMBExwU0dihCiDfJkjtBewOzrQIQQQggh/M2TEaESIEsp9S1QOSqktf4fn0UlhBBCCOEHniRCn7s/hBBCCCFaFU92ln5bKWXh14NWd2qtbb4NSwghhBDC9zxZNTYaeBvXGWMK6KaUmi3L54UQQgjR0nnyaOwZ4BKt9U4ApVRv4H1gkC8DE0IIIYTwtf/f3v3H2VXXdx5/fSYzmYRAQoQQQ36QpCZY0SgyBTQoW1GMomJR/F1o1pVusYpdd6t2K1ipa/vYSsXdlpaHv6BVKSAqC0iK2KrgA0oQBUGENOSn+YXQCYRkJpP72T/uGRhIMrmZzLn3Ts7r+Xjcx5zzveec+5nHheSd749zGlk11jUYggAy8yFcRSZJkg4CjfQILY+ILwL/WOy/F/DppJIkacxrJAj9AfBBYHC5/I+Avy2tIkmSpCZpZNVYH3BJ8ZIkSTpo7DUIRcTVmfmOiLiP+rPFniUzF5VamSRJUsmG6xG6oPj5pmYUIkmS1Gx7XTWWmRuKzfMzc/XQF3B+c8qTJEkqTyPL51+3h7Y3jHYhkiRJzTbcHKE/oN7z8xsRce+Qtw4Dbi+7MEmSpLINN0fo68B3gc8CHx/S/kRmPlZqVZIkSU0w3Byh3sxcBfwpsLGYGzQPeF9EHN6k+iRJkkrTyByhbwK7IuIFwOXAbOq9RZIkSWNaI0GolpkDwFnA/8nM/wHMKLcsSZKk8jUShHZGxLuBc4AbijYfuipJksa8RoLQUuAVwGcy85GImAf8Q7llSZIkla+RZ409EBEfA+YU+48Af1l2YZIkSWXbZ49QRLwZ+Clwc7H/soi4vuzCJEmSytbI0NingBOB/wDIzJ8C80usSZIkqSkamiydmb3PaauVUYwkSVIz7XOOEHB/RLwHGBcRC4APAz8utyxJkqTyNdIj9CHgOKAP+AawFfhImUVJkiQ1QyOrxp4C/mfxkiRJOmjsMwhFxL8A+dz2zHxNKRVJkiQ1SSNzhP77kO0JwNuAgXLKkSRJap5Ghsbufk7T7RHxbyXVI0mS1DSNDI09b8huB3ACMKW0iiRJkpqkkaGxu6nPEQrqQ2KPAO8vsyhJkqRmaGRobF4zCpEkSWq2RobGzhru/cy8bvTKkSRJap5GhsbeD7wS+H6x/9vU7yy9hfqQmUFIkiSNSY0EoS7gRZm5ASAiZgBfzcylpVYmSZJUskYesTF7MAQVNgFzSqpHkiSpaRrpEbo1IpZRf84YwDuB7x3Ih0bEHwH/hfrQ2n3AUmAGcBVwBPWVar+bmf0R0Q1cSX3Z/q+Bd2bmquI6n6A+dLcL+HBmLivalwCXAuOAL2bmXxxIvZI0nJ07dvDQnbez5uc/Y9PKFQz099E5vpvp81/AnBe/lIUnLaZrwoRWlylpDyJzt6dn7H5QxO8Ary52f5iZ3xrxB0bMBG6jPty2PSKuBm4C3ghcl5lXRcTfAT/LzMsi4nxgUWb+14h4F/A7mfnOiHgR9XB2InA09XC2sPiYh4DXAeuAu4B3Z+YDw9XV09OTy5cvH+mvJamCBvr7+fE1X+Oem28gOoKdO3bsdkzXhAlkLTl+yZt45dnvpXP8+BZUKlVPRNydmT37Oq6RHiGK4DPi8LOXz50YETuBQ4ANwGuA9xTvXwF8CrgMOLPYBrgW+L8REUX7VZnZBzwSESuohyKAFZm5EiAiriqOHTYISdL++PW6tXzzsxeyfetWBvr79nrcYDi65+YbePDHP+Rtn/g0R8ya3awyJe1DI3OERlVmrgf+ClhDPQD1Uh8K+4/MHHyG2TpgZrE9E1hbnDtQHH/E0PbnnLO39t1ExHkRsTwilm/ZsuXAfzlJlfDrdWv5+ic/yhOPPjpsCBpqoL+PJ379KF//5Ef59bq1+z5BUlM0PQhFxFTqPTTzqA9pTQKWNLsOgMy8PDN7MrNn2rRprShB0hgz0N/PNz97If1Pbac+zXE/ZNL/1Hau++xFDPT3l1KfpP2z1yAUEbcWP/9ylD/ztcAjmbklM3dSvw/RYuDwiBgcqpsFrC+21wOzi1o6qT/n7NdD259zzt7aJemA/fiar7F961b2OwQ9LXlqay8/vuZro1mWpBEarkdoRkS8EnhLRBwfES8f+jqAz1wDnBwRhxRzfU6jPn/nX4C3F8ecC3yn2L6+2Kd4//tZn+F9PfCuiOiOiHnAAuDfqE+OXhAR8yJiPPCu4lhJOiA7d+zgnptvaHg4bG8G+vu4Z9kNe5xcLam5hpssfSHwSeo9Kpc8572kPrl5v2XmnRFxLfAT6g9xvQe4HLgRuCoi/rxo+1JxypeAfygmQz9GPdiQmfcXK84eKK7zwczcBRARfwgso758/suZef9IapWkoR6683aiI0blWhHBQ3feznGnnjYq15M0MvtcPh8Rn8zMi5tUT8u4fF7Svnz3by7hgR9+f98HNui4U09jyfl/NGrXk/SMUVs+n5kXR8RbeOY+Qv+amTccaIGSNNZsWrliVK+3cZSvJ2n/7XPVWER8FriA+hDUA8AFEfG/yi5MktrNgc4N2u16fc4RklqtkRsqngG8LDNrABFxBfU5PH9SZmGS1G46x3eP7vW6feyG1GqN3kfo8CHbU8ooRJLa3fT5LxjV6z1/lK8naf81EoQ+C9wTEV8teoPuBj5TblmS1H7mvPilo/bw1K4JE5h93KJRuZakkdtnEMrMbwAnU7/x4TeBV2TmP5VdmCS1m4UnLSZrI72R4rNlJgtPWjwq15I0cg0NjWXmhsy8vnhtLLsoSWpHXRMmcPySNx3wXKHO8d0c//o3jVrvkqSRa/qzxiRpLHvl2e9l4uTJwAhvrBjBIZOn8Mp3vG9U65I0MgYhSdoPnePH87ZPfJrxh0xkv8NQBOMnTuSsT/wZnV1dpdQnaf8MG4QiYlxEPNisYiRpLDhi1mzec/HnOOzIIxseJusc383kI6bxnos/xxGzZu/7BElNMWwQKp7d9cuImNOkeiRpTDhi1mz+81//fX3OUHf3Xuf7dE2YQGd3N8cveRNL//rvDEFSm2nkhopTgfsj4t+AbYONmfmW0qqSpDGgc/x4Xv3epbzibe/moTtvZ83997Jp5QoG+nbQ2T2B6fNfwJzjFrHwpMVOjJbaVCNB6JOlVyFJY1jXhAkcd+ppPkleGoMaeejqDyLiGGBBZn4vIg4BxpVfmiRJUrkaeejqB4Brgb8vmmYC3y6zKEmSpGZoZPn8B4HFwFaAzHwYOKrMoiRJkpqhkSDUl5n9gzsR0QmMzj3mJUmSWqiRIPSDiPgTYGJEvA64Bvh/5ZYlSZJUvkaC0MeBLcB9wO8DNwF/WmZRkiRJzdDIqrFaRFwB3El9SOyXmenQmCRJGvP2GYQi4gzg74B/p/5gnXkR8fuZ+d2yi5MkSSpTIzdU/Bzw25m5AiAifgO4ETAISZKkMa2ROUJPDIagwkrgiZLqkSRJapq99ghFxFnF5vKIuAm4mvocobOBu5pQmyRJUqmGGxp785DtTcCpxfYWYGJpFUmSJDXJXoNQZi5tZiGSJEnN1siqsXnAh4C5Q4/PzLeUV5YkSVL5Glk19m3gS9TvJl0rtxxJkqTmaSQI7cjML5ReiSRJUpM1EoQujYiLgH8G+gYbM/MnpVUlSZLUBI0EoZcAvwu8hmeGxrLYlyRJGrMaCUJnA/Mzs7/sYiRJkpqpkTtL/xw4vOxCJEmSmq2RHqHDgQcj4i6ePUfI5fOSJGlMayQIXVR6FZIkSS2wzyCUmT9oRiGSJEnN1sidpZ+gvkoMYDzQBWzLzMllFiZJklS2RnqEDhvcjogAzgROLrMoSZKkZmhk1djTsu7bwOtLqkeSJKlpGhkaO2vIbgfQA+worSJJkqQmaWTV2JuHbA8Aq6gPj0mSJI1pjcwRWtqMQiRJkpptr0EoIi4c5rzMzItH8oERcSzwT0Oa5gMXAlcW7XOp9zq9IzMfLyZoXwq8EXgK+L3BB75GxLnAnxbX+fPMvKJoPwH4KjARuAm4IDMHV75JkiQBw0+W3raHF8D7gY+N9AMz85eZ+bLMfBlwAvVw8y3g48CtmbkAuLXYB3gDsKB4nQdcBhARz6N+s8eTgBOBiyJianHOZcAHhpy3ZKT1SpKkg9deg1Bmfm7wBVxOvXdlKXAV9V6c0XAa8O+ZuZr6vKMrivYrgLcW22cCVxYr1u4ADo+IGdRXrt2SmY9l5uPALcCS4r3JmXlH0Qt05ZBrSZIkPW3Y5fMR8byI+HPgXurDaC/PzI9l5uZR+vx3Ad8otqdn5oZieyMwvdieCawdcs66om249nV7aN9NRJwXEcsjYvmWLVsO5PeQJElj0F6DUET8b+Au4AngJZn5qaLnZVRExHjgLcA1z32v6MkpfU5PZl6emT2Z2TNt2rSyP06SJLWZ4XqEPgocTX0y8q8iYmvxeiIito7CZ78B+Elmbir2NxXDWhQ/B3ud1gOzh5w3q2gbrn3WHtolSZKeZbg5Qh2ZOTEzD8vMyUNeh43Sc8bezTPDYgDXA+cW2+cC3xnSfk7UnQz0FkNoy4DTI2JqMUn6dGBZ8d7WiDi5WHF2zpBrSZIkPa2RGyqOuoiYBLwO+P0hzX8BXB0R7wdWA+8o2m+ivnR+BfUVZksBMvOxiLiY+vAdwKcz87Fi+3yeWT7/3eIlSZL0LOHtdep6enpy+fLlrS5DkiSNgoi4OzN79nXcfj10VZIk6WBiEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIk7VHuSmrbB8hd2epSpNJ0troASVL7qG0fYNvdm3jyR+vYtbUfOgJqybjJ4zn0VbOYdMJ0Oib6V4cOHv7XLEkiM9n6vTU88YN1REDurNXfKHqDdvX2s3XZKnpvXsVhp85i8mvnEBEtrFgaHQYhSaq4zOTxqx9i+88fhYEaexsIGwxHT/5wHbse38HUsxcahjTmOUdIkipu6/fWsP3njz7TC7QPubPG9vseZev31pRcmVQ+g5AkVVht+wBP/GBdwyFoUO6s8cQP1lHbMVBSZVJzGIQkqcK23b2JkY5uRcC25ZtGtyCpyQxCklRhT/5o/3uDBuXOGk/+aN0oVyQ1l0FIkioqd9XqS+QPwK6t/d5nSGOaQUiSKir7a/X7BB2IjiD7d41OQVILGIQkqaJifAfUDrA3p5bE+HGjU5DUAgYhSaqoGNfBuMnjD+ga4yaPJ8Z5LyGNXQYhSaqwQ181i+ga2V8F0dXBoa+aNcoVSc1lEJKkCpt0wnRyhKNjmTCpZ/roFiQ1mUFIkiqsY2Inh526/71C0dXBYafOomOCT2rS2GYQkqSKm/zaOUx8yZENh6Ho6mDiS45k8mvnlFyZVD6jvCRVXEQw9eyFjJs6Yfenzw89rquDTDj01T59XgcPg5AkiYhgyuuO4bBTZrLt7k08+aN19ZstdgTUknGTx3Poq2YxqWe6w2E6qPhfsyTpaR0TOznslJkcdspMcleS/buI8eNcIq+DlkFIkrRHMS6Iif41oYObk6UlSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJltSQIRcThEXFtRDwYEb+IiFdExPMi4paIeLj4ObU4NiLiCxGxIiLujYiXD7nOucXxD0fEuUPaT4iI+4pzvhA+GVCSJO1Bq3qELgVuzswXAi8FfgF8HLg1MxcAtxb7AG8AFhSv84DLACLiecBFwEnAicBFg+GpOOYDQ85b0oTfSZLUhnp7e3nwwQf52c9+xoMPPkhvb2+rS1IbafpDZCJiCvBq4PcAMrMf6I+IM4H/VBx2BfCvwMeAM4ErMzOBO4repBnFsbdk5mPFdW8BlkTEvwKTM/OOov1K4K3Ad5vw60mS2kBm8sgjj3DbbbexevVqOjs7yUwigoGBAY455hhOOeUU5s+f3+pS1WKteJrePGAL8JWIeClwN3ABMD0zNxTHbASmF9szgbVDzl9XtA3Xvm4P7buJiPOo9zIxZ86ckf9GkqS2UavVuPHGG7n33nvZuXMnALt27XrWMStXrmTt2rUsWrSIM844g44Op8xWVSu++U7g5cBlmXk8sI1nhsEAKHp/suxCMvPyzOzJzJ5p06aV/XGSpJJl5m4haG927tzJvffey4033tik6tSOWhGE1gHrMvPOYv9a6sFoUzHkRfFzc/H+emD2kPNnFW3Dtc/aQ7sk6SD3yCOPNBSCBg2GoZUrV5ZcmdpV04NQZm4E1kbEsUXTacADwPXA4Mqvc4HvFNvXA+cUq8dOBnqLIbRlwOkRMbWYJH06sKx4b2tEnFysFjtnyLUkSQex2267reEQNGjnzp3cfvvtJVWkdteKOUIAHwK+FhHjgZXAUuqh7OqIeD+wGnhHcexNwBuBFcBTxbFk5mMRcTFwV3HcpwcnTgPnA18FJlKfJO1EaUk6yPX29rJ69eoRnbtq1Sp6e3uZMmXKKFeldteSIJSZPwV69vDWaXs4NoEP7uU6Xwa+vIf25cCLD7BMSdIYsmHDBjo7O3ebGN2Izs5ONmzYYBCqIKfJS5IOCn19fdT/7bz/MpO+vr5RrkhjgUFIknRQ6O7uZqQPEogIuru7R7kijQUGIUnSQWHGjBkMDAyM6NyBgQFmzJgxyhVpLDAISZIOClOmTOGYY44Z0blz5851flBFGYQkSQeNU045ha6urv06p6uri8WLF5dUkdqdQUiSdNCYN28eixYtajgMdXV1sWjRIp85VmEGIUnSQSMiOOOMMxoKQ4Mh6IwzzmhSdWpHrbqhoiRJpejo6ODNb34zxx13HLfffjurVq3a7enzc+fOZfHixfYEySAkSTo4zZ8/n/nz59Pb28uGDRvo6+uju7ubGTNmODFaTzMISZIOalOmTDH4aK+cIyRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkirLICRJkiqrJUEoIlZFxH0R8dOIWF60PS8ibomIh4ufU4v2iIgvRMSKiLg3Il4+5DrnFsc/HBHnDmk/obj+iuLcaP5vKUmS2l0re4R+OzNflpk9xf7HgVszcwFwa7EP8AZgQfE6D7gM6sEJuAg4CTgRuGgwPBXHfGDIeUvK/3UkSdJY005DY2cCVxTbVwBvHdJ+ZdbdARweETOA1wO3ZOZjmfk4cAuwpHhvcmbekZkJXDnkWpIkSU9rVRBK4J8j4u6IOK9om56ZG4rtjcD0YnsmsHbIueuKtuHa1+2hXZIk6Vk6W/S5p2Tm+og4CrglIh4c+mZmZkRk2UUUIew8gDlz5pT9cZIkqc20pEcoM9cXPzcD36I+x2dTMaxF8XNzcfh6YPaQ02cVbcO1z9pD+57quDwzezKzZ9q0aQf6a0mSpDGm6UEoIiZFxGGD28DpwM+B64HBlV/nAt8ptq8HzilWj50M9BZDaMuA0yNiajFJ+nRgWfHe1og4uVgtds6Qa0mSJD2tFUNj04FvFSvaO4GvZ+bNEXEXcHVEvB9YDbyjOP4m4I3ACuApYClAZj4WERcDdxXHfTozHyu2zwe+CkwEvlu8JEmSniXqC6vU09OTy5cvb3UZkiRpFETE3UNu0bNX7bR8XpIkqalatWpMkiTthye3PczaNV9hy5Zl7KptZ1zHRKZNez2z5yzl0EkLWl3emGUQkiSpjWXWeOihT/OrDddQq+0EdgFQq/Xxqw3XsnHTdzh6xtksXHghEQ707C+DkCRJbawegq6lVtuxh3d3Uavt4lcbrgXg2GM/1dTaDgYGIUmS2tST2x4ueoL2FIKeUatt51cbrmHWrPcxadILmlTdyGzs28kV6x/luk2Ps21XjUnjOjhr+lTOnXkkz+/uano99qFJktSm1q75SjEctm+12k7WrP1KyRWNXGZy6eqNnHTHA/zt2s2s3tHPozsHWL2jn79du5mT7niAS1dvpNmr2Q1CkiS1qS1bljE4J2jfdrFl881llnNAvrBmE59ftZm+WtJXe3bYGWz7/KrNfGHNpqbWZRCSJKlN7aptL/X4ZtnYt5NLVm1ie6027HHbazUuWbWJjX2N9YKNBoOQJEltalzHxFKPb5avrn90v46/cj+PPxAGIUmS2tS0aa8HxjV4dCfTjlpSZjkj9q1Nj+82HLY3fbXkus2Pl1zRMwxCkiS1qdlzltLR0dhKqo6OTubMXlpyRSOzbdfwQ2LP9eTA/h1/IAxCkiS1qUMnLeDoGWfTsY8hr46OiRw94+y2XTo/adz+xY1DO5sXTwxCkiS1sYULL+ToGW+no2MCuw+TjaOjYwJHz3g7Cxde2IryGnLW9Kl0d0RDx3Z3BGcdNbXkip5hEJIkqY1FdHDssZ/it37r2xx99Nl0dR5OR0c3XZ2Hc/TRZ3Pib32HY4/9VFs/XuPcmUc2fnDCOftz/AHyztKSJI0Bh05awG++8DP85gs/0+pS9tvzu7v4b3On8/lVm4ddQj+xo4OPzD2qqXeYNghJkqTSfXjOdAAuWVW/YeLQVWTdHQEJH5l71NPHNYtBSJIklS4iuOCY5/PO5x/Blesf5brNj/PkQI1DOzs466ipnNOiZ40ZhCRJUtM8v7uLP54/gz+eP6PVpQBOlpYkSRVmEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZVlEJIkSZUVmdnqGtpCRGwBVpd0+SOBR0u6tkbG76Q9+b20H7+T9uT3sm/HZOa0fR1kEGqCiFiemT2trkPP8DtpT34v7cfvpD35vYweh8YkSVJlGYQkSVJlGYSa4/JWF6Dd+J20J7+X9uN30p78XkaJc4QkSVJl2SMkSZIqyyAkSZIqyyBUoohYEhG/jIgVEfHxVtcjiIjZEfEvEfFARNwfERe0uibVRcS4iLgnIm5odS2qi4jDI+LaiHgwIn4REa9odU1VFxF/VPzZ9fOI+EZETGh1TWOdQagkETEO+BvgDcCLgHdHxItaW5WAAeCjmfki4GTgg34vbeMC4BetLkLPcilwc2a+EHgpfj8tFREzgQ8DPZn5YmAc8K7WVjX2GYTKcyKwIjNXZmY/cBVwZotrqrzM3JCZPym2n6D+B/vM1laliJgFnAF8sdW1qC4ipgCvBr4EkJn9mfkfra1KQCcwMSI6gUOAX7W4njHPIFSemcDaIfvr8C/cthIRc4HjgTtbW4mAzwN/DNRaXYieNg/YAnylGLL8YkRManVRVZaZ64G/AtYAG4DezPzn1lY19hmEVEkRcSjwTeAjmbm11fVUWUS8CdicmXe3uhY9SyfwcuCyzDwe2AY417GFImIq9ZGFecDRwKSIeF9rqxr7DELlWQ/MHrI/q2hTi0VEF/UQ9LXMvK7V9YjFwFsiYhX1IeTXRMQ/trYkUe/FXpeZgz2m11IPRmqd1wKPZOaWzNwJXAe8ssU1jXkGofLcBSyIiHkRMZ76hLbrW1xT5UVEUJ/z8IvMvKTV9Qgy8xOZOSsz51L//+T7mem/clssMzcCayPi2KLpNOCBFpak+pDYyRFxSPFn2Wk4gf2Adba6gINVZg5ExB8Cy6jP7P9yZt7f4rJU7334XeC+iPhp0fYnmXlTC2uS2tWHgK8V/5hbCSxtcT2Vlpl3RsS1wE+or4C9Bx+1ccB8xIYkSaosh8YkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJlGYQkSVJl/X9mwyMsqFYkCQAAAABJRU5ErkJggg==\n",
  92 |       "text/plain": [
  93 |        "<Figure size 648x576 with 1 Axes>"
  94 |       ]
  95 |      },
  96 |      "metadata": {},
  97 |      "output_type": "display_data"
  98 |     }
  99 |    ],
 100 |    "source": [
 101 |     "tagCount =  collections.Counter(list(df_tags['Tag'])).most_common(10)\n",
 102 |     "print(tagCount)\n",
 103 |     "plot_tags(tagCount)"
 104 |    ]
 105 |   },
 106 |   {
 107 |    "cell_type": "code",
 108 |    "execution_count": 5,
 109 |    "metadata": {},
 110 |    "outputs": [],
 111 |    "source": [
 112 |     "top10=['javascript','java','c#','php','android','jquery','python','html','c++','ios']"
 113 |    ]
 114 |   },
 115 |   {
 116 |    "cell_type": "code",
 117 |    "execution_count": 6,
 118 |    "metadata": {},
 119 |    "outputs": [
 120 |     {
 121 |      "name": "stdout",
 122 |      "output_type": "stream",
 123 |      "text": [
 124 |       "(826739, 2)\n"
 125 |      ]
 126 |     },
 127 |     {
 128 |      "data": {
 129 |       "text/html": [
 130 |        "<div>\n",
 131 |        "<style scoped>\n",
 132 |        "    .dataframe tbody tr th:only-of-type {\n",
 133 |        "        vertical-align: middle;\n",
 134 |        "    }\n",
 135 |        "\n",
 136 |        "    .dataframe tbody tr th {\n",
 137 |        "        vertical-align: top;\n",
 138 |        "    }\n",
 139 |        "\n",
 140 |        "    .dataframe thead th {\n",
 141 |        "        text-align: right;\n",
 142 |        "    }\n",
 143 |        "</style>\n",
 144 |        "<table border=\"1\" class=\"dataframe\">\n",
 145 |        "  <thead>\n",
 146 |        "    <tr style=\"text-align: right;\">\n",
 147 |        "      <th></th>\n",
 148 |        "      <th>Id</th>\n",
 149 |        "      <th>Tag</th>\n",
 150 |        "    </tr>\n",
 151 |        "  </thead>\n",
 152 |        "  <tbody>\n",
 153 |        "    <tr>\n",
 154 |        "      <th>14</th>\n",
 155 |        "      <td>260</td>\n",
 156 |        "      <td>c#</td>\n",
 157 |        "    </tr>\n",
 158 |        "    <tr>\n",
 159 |        "      <th>18</th>\n",
 160 |        "      <td>330</td>\n",
 161 |        "      <td>c++</td>\n",
 162 |        "    </tr>\n",
 163 |        "    <tr>\n",
 164 |        "      <th>28</th>\n",
 165 |        "      <td>650</td>\n",
 166 |        "      <td>c#</td>\n",
 167 |        "    </tr>\n",
 168 |        "    <tr>\n",
 169 |        "      <th>35</th>\n",
 170 |        "      <td>930</td>\n",
 171 |        "      <td>c#</td>\n",
 172 |        "    </tr>\n",
 173 |        "    <tr>\n",
 174 |        "      <th>39</th>\n",
 175 |        "      <td>1010</td>\n",
 176 |        "      <td>c#</td>\n",
 177 |        "    </tr>\n",
 178 |        "  </tbody>\n",
 179 |        "</table>\n",
 180 |        "</div>"
 181 |       ],
 182 |       "text/plain": [
 183 |        "      Id  Tag\n",
 184 |        "14   260   c#\n",
 185 |        "18   330  c++\n",
 186 |        "28   650   c#\n",
 187 |        "35   930   c#\n",
 188 |        "39  1010   c#"
 189 |       ]
 190 |      },
 191 |      "execution_count": 6,
 192 |      "metadata": {},
 193 |      "output_type": "execute_result"
 194 |     }
 195 |    ],
 196 |    "source": [
 197 |     "tag_top10= df_tags[df_tags.Tag.isin(top10)]\n",
 198 |     "print (tag_top10.shape)\n",
 199 |     "tag_top10.head()"
 200 |    ]
 201 |   },
 202 |   {
 203 |    "cell_type": "code",
 204 |    "execution_count": 7,
 205 |    "metadata": {},
 206 |    "outputs": [
 207 |     {
 208 |      "data": {
 209 |       "text/plain": [
 210 |        "30798790    5\n",
 211 |        "31085960    5\n",
 212 |        "11648170    5\n",
 213 |        "35318730    5\n",
 214 |        "4009250     5\n",
 215 |        "30289880    5\n",
 216 |        "23267320    5\n",
 217 |        "35283570    5\n",
 218 |        "30991580    5\n",
 219 |        "23484760    5\n",
 220 |        "Name: Id, dtype: int64"
 221 |       ]
 222 |      },
 223 |      "execution_count": 7,
 224 |      "metadata": {},
 225 |      "output_type": "execute_result"
 226 |     }
 227 |    ],
 228 |    "source": [
 229 |     "tag_top10['Id'].value_counts().head(10)"
 230 |    ]
 231 |   },
 232 |   {
 233 |    "cell_type": "code",
 234 |    "execution_count": 8,
 235 |    "metadata": {},
 236 |    "outputs": [
 237 |     {
 238 |      "data": {
 239 |       "text/html": [
 240 |        "<div>\n",
 241 |        "<style scoped>\n",
 242 |        "    .dataframe tbody tr th:only-of-type {\n",
 243 |        "        vertical-align: middle;\n",
 244 |        "    }\n",
 245 |        "\n",
 246 |        "    .dataframe tbody tr th {\n",
 247 |        "        vertical-align: top;\n",
 248 |        "    }\n",
 249 |        "\n",
 250 |        "    .dataframe thead th {\n",
 251 |        "        text-align: right;\n",
 252 |        "    }\n",
 253 |        "</style>\n",
 254 |        "<table border=\"1\" class=\"dataframe\">\n",
 255 |        "  <thead>\n",
 256 |        "    <tr style=\"text-align: right;\">\n",
 257 |        "      <th></th>\n",
 258 |        "      <th>Id</th>\n",
 259 |        "      <th>Tag</th>\n",
 260 |        "    </tr>\n",
 261 |        "  </thead>\n",
 262 |        "  <tbody>\n",
 263 |        "    <tr>\n",
 264 |        "      <th>14</th>\n",
 265 |        "      <td>260</td>\n",
 266 |        "      <td>c#</td>\n",
 267 |        "    </tr>\n",
 268 |        "    <tr>\n",
 269 |        "      <th>18</th>\n",
 270 |        "      <td>330</td>\n",
 271 |        "      <td>c++</td>\n",
 272 |        "    </tr>\n",
 273 |        "    <tr>\n",
 274 |        "      <th>28</th>\n",
 275 |        "      <td>650</td>\n",
 276 |        "      <td>c#</td>\n",
 277 |        "    </tr>\n",
 278 |        "    <tr>\n",
 279 |        "      <th>35</th>\n",
 280 |        "      <td>930</td>\n",
 281 |        "      <td>c#</td>\n",
 282 |        "    </tr>\n",
 283 |        "    <tr>\n",
 284 |        "      <th>39</th>\n",
 285 |        "      <td>1010</td>\n",
 286 |        "      <td>c#</td>\n",
 287 |        "    </tr>\n",
 288 |        "  </tbody>\n",
 289 |        "</table>\n",
 290 |        "</div>"
 291 |       ],
 292 |       "text/plain": [
 293 |        "      Id  Tag\n",
 294 |        "14   260   c#\n",
 295 |        "18   330  c++\n",
 296 |        "28   650   c#\n",
 297 |        "35   930   c#\n",
 298 |        "39  1010   c#"
 299 |       ]
 300 |      },
 301 |      "execution_count": 8,
 302 |      "metadata": {},
 303 |      "output_type": "execute_result"
 304 |     }
 305 |    ],
 306 |    "source": [
 307 |     "tag_top10.head()"
 308 |    ]
 309 |   },
 310 |   {
 311 |    "cell_type": "code",
 312 |    "execution_count": 9,
 313 |    "metadata": {},
 314 |    "outputs": [],
 315 |    "source": [
 316 |     "def add_tags(question_id):\n",
 317 |     "    return tag_top10[tag_top10['Id'] == question_id['Id']].Tag.values\n",
 318 |     "\n",
 319 |     "top10 = tag_top10.apply(add_tags, axis=1)"
 320 |    ]
 321 |   },
 322 |   {
 323 |    "cell_type": "code",
 324 |    "execution_count": 10,
 325 |    "metadata": {},
 326 |    "outputs": [
 327 |     {
 328 |      "data": {
 329 |       "text/plain": [
 330 |        "(826739, (826739, 2))"
 331 |       ]
 332 |      },
 333 |      "execution_count": 10,
 334 |      "metadata": {},
 335 |      "output_type": "execute_result"
 336 |     }
 337 |    ],
 338 |    "source": [
 339 |     "len(top10),tag_top10.shape"
 340 |    ]
 341 |   },
 342 |   {
 343 |    "cell_type": "code",
 344 |    "execution_count": 11,
 345 |    "metadata": {},
 346 |    "outputs": [
 347 |     {
 348 |      "data": {
 349 |       "text/html": [
 350 |        "<div>\n",
 351 |        "<style scoped>\n",
 352 |        "    .dataframe tbody tr th:only-of-type {\n",
 353 |        "        vertical-align: middle;\n",
 354 |        "    }\n",
 355 |        "\n",
 356 |        "    .dataframe tbody tr th {\n",
 357 |        "        vertical-align: top;\n",
 358 |        "    }\n",
 359 |        "\n",
 360 |        "    .dataframe thead th {\n",
 361 |        "        text-align: right;\n",
 362 |        "    }\n",
 363 |        "</style>\n",
 364 |        "<table border=\"1\" class=\"dataframe\">\n",
 365 |        "  <thead>\n",
 366 |        "    <tr style=\"text-align: right;\">\n",
 367 |        "      <th></th>\n",
 368 |        "      <th>Id</th>\n",
 369 |        "      <th>Tag</th>\n",
 370 |        "      <th>Tags</th>\n",
 371 |        "    </tr>\n",
 372 |        "  </thead>\n",
 373 |        "  <tbody>\n",
 374 |        "    <tr>\n",
 375 |        "      <th>14</th>\n",
 376 |        "      <td>260</td>\n",
 377 |        "      <td>c#</td>\n",
 378 |        "      <td>[c#]</td>\n",
 379 |        "    </tr>\n",
 380 |        "    <tr>\n",
 381 |        "      <th>18</th>\n",
 382 |        "      <td>330</td>\n",
 383 |        "      <td>c++</td>\n",
 384 |        "      <td>[c++]</td>\n",
 385 |        "    </tr>\n",
 386 |        "    <tr>\n",
 387 |        "      <th>28</th>\n",
 388 |        "      <td>650</td>\n",
 389 |        "      <td>c#</td>\n",
 390 |        "      <td>[c#]</td>\n",
 391 |        "    </tr>\n",
 392 |        "    <tr>\n",
 393 |        "      <th>35</th>\n",
 394 |        "      <td>930</td>\n",
 395 |        "      <td>c#</td>\n",
 396 |        "      <td>[c#]</td>\n",
 397 |        "    </tr>\n",
 398 |        "    <tr>\n",
 399 |        "      <th>39</th>\n",
 400 |        "      <td>1010</td>\n",
 401 |        "      <td>c#</td>\n",
 402 |        "      <td>[c#]</td>\n",
 403 |        "    </tr>\n",
 404 |        "  </tbody>\n",
 405 |        "</table>\n",
 406 |        "</div>"
 407 |       ],
 408 |       "text/plain": [
 409 |        "      Id  Tag   Tags\n",
 410 |        "14   260   c#   [c#]\n",
 411 |        "18   330  c++  [c++]\n",
 412 |        "28   650   c#   [c#]\n",
 413 |        "35   930   c#   [c#]\n",
 414 |        "39  1010   c#   [c#]"
 415 |       ]
 416 |      },
 417 |      "execution_count": 11,
 418 |      "metadata": {},
 419 |      "output_type": "execute_result"
 420 |     }
 421 |    ],
 422 |    "source": [
 423 |     "tag_top10=pd.concat([tag_top10, top10.rename('Tags')], axis=1)\n",
 424 |     "tag_top10.head()"
 425 |    ]
 426 |   },
 427 |   {
 428 |    "cell_type": "code",
 429 |    "execution_count": 12,
 430 |    "metadata": {},
 431 |    "outputs": [
 432 |     {
 433 |      "data": {
 434 |       "text/plain": [
 435 |        "(826739, 2)"
 436 |       ]
 437 |      },
 438 |      "execution_count": 12,
 439 |      "metadata": {},
 440 |      "output_type": "execute_result"
 441 |     }
 442 |    ],
 443 |    "source": [
 444 |     "tag_top10.drop([\"Tag\"], axis=1, inplace=True)\n",
 445 |     "tag_top10.shape"
 446 |    ]
 447 |   },
 448 |   {
 449 |    "cell_type": "code",
 450 |    "execution_count": 13,
 451 |    "metadata": {},
 452 |    "outputs": [],
 453 |    "source": [
 454 |     "top10_tags=tag_top10.loc[tag_top10.astype(str).drop_duplicates().index]"
 455 |    ]
 456 |   },
 457 |   {
 458 |    "cell_type": "code",
 459 |    "execution_count": 14,
 460 |    "metadata": {},
 461 |    "outputs": [
 462 |     {
 463 |      "data": {
 464 |       "text/html": [
 465 |        "<div>\n",
 466 |        "<style scoped>\n",
 467 |        "    .dataframe tbody tr th:only-of-type {\n",
 468 |        "        vertical-align: middle;\n",
 469 |        "    }\n",
 470 |        "\n",
 471 |        "    .dataframe tbody tr th {\n",
 472 |        "        vertical-align: top;\n",
 473 |        "    }\n",
 474 |        "\n",
 475 |        "    .dataframe thead th {\n",
 476 |        "        text-align: right;\n",
 477 |        "    }\n",
 478 |        "</style>\n",
 479 |        "<table border=\"1\" class=\"dataframe\">\n",
 480 |        "  <thead>\n",
 481 |        "    <tr style=\"text-align: right;\">\n",
 482 |        "      <th></th>\n",
 483 |        "      <th>Id</th>\n",
 484 |        "      <th>Title</th>\n",
 485 |        "      <th>Body</th>\n",
 486 |        "    </tr>\n",
 487 |        "  </thead>\n",
 488 |        "  <tbody>\n",
 489 |        "    <tr>\n",
 490 |        "      <th>0</th>\n",
 491 |        "      <td>80</td>\n",
 492 |        "      <td>SQLStatement.execute() - multiple queries in o...</td>\n",
 493 |        "      <td>I've written a database generation script in S...</td>\n",
 494 |        "    </tr>\n",
 495 |        "    <tr>\n",
 496 |        "      <th>1</th>\n",
 497 |        "      <td>90</td>\n",
 498 |        "      <td>Good branching and merging tutorials for Torto...</td>\n",
 499 |        "      <td>Are there any really good tutorials explaining...</td>\n",
 500 |        "    </tr>\n",
 501 |        "    <tr>\n",
 502 |        "      <th>2</th>\n",
 503 |        "      <td>120</td>\n",
 504 |        "      <td>ASP.NET Site Maps</td>\n",
 505 |        "      <td>Has anyone got experience creating SQL-based A...</td>\n",
 506 |        "    </tr>\n",
 507 |        "    <tr>\n",
 508 |        "      <th>3</th>\n",
 509 |        "      <td>180</td>\n",
 510 |        "      <td>Function for creating color wheels</td>\n",
 511 |        "      <td>This is something I've pseudo-solved many time...</td>\n",
 512 |        "    </tr>\n",
 513 |        "    <tr>\n",
 514 |        "      <th>4</th>\n",
 515 |        "      <td>260</td>\n",
 516 |        "      <td>Adding scripting functionality to .NET applica...</td>\n",
 517 |        "      <td>I have a little game written in C#. It uses a ...</td>\n",
 518 |        "    </tr>\n",
 519 |        "  </tbody>\n",
 520 |        "</table>\n",
 521 |        "</div>"
 522 |       ],
 523 |       "text/plain": [
 524 |        "    Id                        ...                                                                       Body\n",
 525 |        "0   80                        ...                          I've written a database generation script in S...\n",
 526 |        "1   90                        ...                          Are there any really good tutorials explaining...\n",
 527 |        "2  120                        ...                          Has anyone got experience creating SQL-based A...\n",
 528 |        "3  180                        ...                          This is something I've pseudo-solved many time...\n",
 529 |        "4  260                        ...                          I have a little game written in C#. It uses a ...\n",
 530 |        "\n",
 531 |        "[5 rows x 3 columns]"
 532 |       ]
 533 |      },
 534 |      "execution_count": 14,
 535 |      "metadata": {},
 536 |      "output_type": "execute_result"
 537 |     }
 538 |    ],
 539 |    "source": [
 540 |     "ques = pd.read_csv('../input/stackoverflow-clean-questions-file-v2/question_clean.csv', encoding='iso-8859-1')\n",
 541 |     "ques.head()"
 542 |    ]
 543 |   },
 544 |   {
 545 |    "cell_type": "code",
 546 |    "execution_count": 15,
 547 |    "metadata": {},
 548 |    "outputs": [
 549 |     {
 550 |      "name": "stdout",
 551 |      "output_type": "stream",
 552 |      "text": [
 553 |       "(706336, 4)\n"
 554 |      ]
 555 |     },
 556 |     {
 557 |      "data": {
 558 |       "text/html": [
 559 |        "<div>\n",
 560 |        "<style scoped>\n",
 561 |        "    .dataframe tbody tr th:only-of-type {\n",
 562 |        "        vertical-align: middle;\n",
 563 |        "    }\n",
 564 |        "\n",
 565 |        "    .dataframe tbody tr th {\n",
 566 |        "        vertical-align: top;\n",
 567 |        "    }\n",
 568 |        "\n",
 569 |        "    .dataframe thead th {\n",
 570 |        "        text-align: right;\n",
 571 |        "    }\n",
 572 |        "</style>\n",
 573 |        "<table border=\"1\" class=\"dataframe\">\n",
 574 |        "  <thead>\n",
 575 |        "    <tr style=\"text-align: right;\">\n",
 576 |        "      <th></th>\n",
 577 |        "      <th>Id</th>\n",
 578 |        "      <th>Title</th>\n",
 579 |        "      <th>Body</th>\n",
 580 |        "      <th>Tags</th>\n",
 581 |        "    </tr>\n",
 582 |        "  </thead>\n",
 583 |        "  <tbody>\n",
 584 |        "    <tr>\n",
 585 |        "      <th>0</th>\n",
 586 |        "      <td>260</td>\n",
 587 |        "      <td>Adding scripting functionality to .NET applica...</td>\n",
 588 |        "      <td>I have a little game written in C#. It uses a ...</td>\n",
 589 |        "      <td>[c#]</td>\n",
 590 |        "    </tr>\n",
 591 |        "    <tr>\n",
 592 |        "      <th>1</th>\n",
 593 |        "      <td>330</td>\n",
 594 |        "      <td>Should I use nested classes in this case?</td>\n",
 595 |        "      <td>I am working on a collection of classes used f...</td>\n",
 596 |        "      <td>[c++]</td>\n",
 597 |        "    </tr>\n",
 598 |        "    <tr>\n",
 599 |        "      <th>2</th>\n",
 600 |        "      <td>650</td>\n",
 601 |        "      <td>Automatically update version number</td>\n",
 602 |        "      <td>I would like the version property of my applic...</td>\n",
 603 |        "      <td>[c#]</td>\n",
 604 |        "    </tr>\n",
 605 |        "    <tr>\n",
 606 |        "      <th>3</th>\n",
 607 |        "      <td>930</td>\n",
 608 |        "      <td>How do I connect to a database and loop over a...</td>\n",
 609 |        "      <td>What's the simplest way to connect and query a...</td>\n",
 610 |        "      <td>[c#]</td>\n",
 611 |        "    </tr>\n",
 612 |        "    <tr>\n",
 613 |        "      <th>4</th>\n",
 614 |        "      <td>1010</td>\n",
 615 |        "      <td>How to get the value of built, encoded ViewState?</td>\n",
 616 |        "      <td>I need to grab the base64-encoded representati...</td>\n",
 617 |        "      <td>[c#]</td>\n",
 618 |        "    </tr>\n",
 619 |        "  </tbody>\n",
 620 |        "</table>\n",
 621 |        "</div>"
 622 |       ],
 623 |       "text/plain": [
 624 |        "     Id  ...     Tags\n",
 625 |        "0   260  ...     [c#]\n",
 626 |        "1   330  ...    [c++]\n",
 627 |        "2   650  ...     [c#]\n",
 628 |        "3   930  ...     [c#]\n",
 629 |        "4  1010  ...     [c#]\n",
 630 |        "\n",
 631 |        "[5 rows x 4 columns]"
 632 |       ]
 633 |      },
 634 |      "execution_count": 15,
 635 |      "metadata": {},
 636 |      "output_type": "execute_result"
 637 |     }
 638 |    ],
 639 |    "source": [
 640 |     "total=pd.merge(ques, top10_tags, on='Id')\n",
 641 |     "print(total.shape)\n",
 642 |     "total.head()"
 643 |    ]
 644 |   },
 645 |   {
 646 |    "cell_type": "code",
 647 |    "execution_count": 16,
 648 |    "metadata": {},
 649 |    "outputs": [
 650 |     {
 651 |      "name": "stderr",
 652 |      "output_type": "stream",
 653 |      "text": [
 654 |       "Using TensorFlow backend.\n"
 655 |      ]
 656 |     }
 657 |    ],
 658 |    "source": [
 659 |     "from sklearn.model_selection import train_test_split\n",
 660 |     "from sklearn.preprocessing import MultiLabelBinarizer\n",
 661 |     "from nltk import word_tokenize\n",
 662 |     "from keras.preprocessing.text import Tokenizer\n",
 663 |     "from keras.preprocessing import sequence\n",
 664 |     "from keras.layers import LSTM, Activation, Dense, Dropout, Input, Embedding, BatchNormalization, GRU ,concatenate\n",
 665 |     "from keras.models import Model"
 666 |    ]
 667 |   },
 668 |   {
 669 |    "cell_type": "code",
 670 |    "execution_count": 17,
 671 |    "metadata": {},
 672 |    "outputs": [
 673 |     {
 674 |      "data": {
 675 |       "text/plain": [
 676 |        "array(['android', 'c#', 'c++', 'html', 'ios', 'java', 'javascript',\n",
 677 |        "       'jquery', 'php', 'python'], dtype=object)"
 678 |       ]
 679 |      },
 680 |      "execution_count": 17,
 681 |      "metadata": {},
 682 |      "output_type": "execute_result"
 683 |     }
 684 |    ],
 685 |    "source": [
 686 |     "multilabel_binarizer = MultiLabelBinarizer()\n",
 687 |     "multilabel_binarizer.fit(total.Tags)\n",
 688 |     "labels = multilabel_binarizer.classes_\n",
 689 |     "labels"
 690 |    ]
 691 |   },
 692 |   {
 693 |    "cell_type": "code",
 694 |    "execution_count": 18,
 695 |    "metadata": {},
 696 |    "outputs": [],
 697 |    "source": [
 698 |     "train,test=train_test_split(total[:550000],test_size=0.25,random_state=24)"
 699 |    ]
 700 |   },
 701 |   {
 702 |    "cell_type": "code",
 703 |    "execution_count": 19,
 704 |    "metadata": {},
 705 |    "outputs": [
 706 |     {
 707 |      "data": {
 708 |       "text/plain": [
 709 |        "((412500, 4), (137500, 4))"
 710 |       ]
 711 |      },
 712 |      "execution_count": 19,
 713 |      "metadata": {},
 714 |      "output_type": "execute_result"
 715 |     }
 716 |    ],
 717 |    "source": [
 718 |     "train.shape,test.shape"
 719 |    ]
 720 |   },
 721 |   {
 722 |    "cell_type": "code",
 723 |    "execution_count": 20,
 724 |    "metadata": {},
 725 |    "outputs": [],
 726 |    "source": [
 727 |     "X_train_t=train['Title']\n",
 728 |     "X_train_b=train['Body']\n",
 729 |     "y_train=multilabel_binarizer.transform(train['Tags'])\n",
 730 |     "X_test_t=test['Title']\n",
 731 |     "X_test_b=test['Body']\n",
 732 |     "y_test=multilabel_binarizer.transform(test['Tags'])"
 733 |    ]
 734 |   },
 735 |   {
 736 |    "cell_type": "code",
 737 |    "execution_count": 21,
 738 |    "metadata": {},
 739 |    "outputs": [
 740 |     {
 741 |      "data": {
 742 |       "text/plain": [
 743 |        "59"
 744 |       ]
 745 |      },
 746 |      "execution_count": 21,
 747 |      "metadata": {},
 748 |      "output_type": "execute_result"
 749 |     }
 750 |    ],
 751 |    "source": [
 752 |     "sent_lens_t=[]\n",
 753 |     "for sent in train['Title']:\n",
 754 |     "    sent_lens_t.append(len(word_tokenize(sent)))\n",
 755 |     "max(sent_lens_t)"
 756 |    ]
 757 |   },
 758 |   {
 759 |    "cell_type": "code",
 760 |    "execution_count": 22,
 761 |    "metadata": {},
 762 |    "outputs": [
 763 |     {
 764 |      "data": {
 765 |       "text/plain": [
 766 |        "18.0"
 767 |       ]
 768 |      },
 769 |      "execution_count": 22,
 770 |      "metadata": {},
 771 |      "output_type": "execute_result"
 772 |     }
 773 |    ],
 774 |    "source": [
 775 |     "np.quantile(sent_lens_t,0.97)"
 776 |    ]
 777 |   },
 778 |   {
 779 |    "cell_type": "code",
 780 |    "execution_count": 23,
 781 |    "metadata": {},
 782 |    "outputs": [],
 783 |    "source": [
 784 |     "max_len_t = 18\n",
 785 |     "tok = Tokenizer(char_level=False,split=' ')\n",
 786 |     "tok.fit_on_texts(X_train_t)\n",
 787 |     "sequences_train_t = tok.texts_to_sequences(X_train_t)"
 788 |    ]
 789 |   },
 790 |   {
 791 |    "cell_type": "code",
 792 |    "execution_count": 24,
 793 |    "metadata": {},
 794 |    "outputs": [
 795 |     {
 796 |      "data": {
 797 |       "text/plain": [
 798 |        "68969"
 799 |       ]
 800 |      },
 801 |      "execution_count": 24,
 802 |      "metadata": {},
 803 |      "output_type": "execute_result"
 804 |     }
 805 |    ],
 806 |    "source": [
 807 |     "vocab_len_t=len(tok.index_word.keys())\n",
 808 |     "vocab_len_t"
 809 |    ]
 810 |   },
 811 |   {
 812 |    "cell_type": "code",
 813 |    "execution_count": 25,
 814 |    "metadata": {},
 815 |    "outputs": [
 816 |     {
 817 |      "data": {
 818 |       "text/plain": [
 819 |        "array([[    0,     0,     0, ...,     1,   957,   197],\n",
 820 |        "       [    0,     0,     0, ...,  9081,    45,   533],\n",
 821 |        "       [    0,     0,     0, ...,   147,     8,   230],\n",
 822 |        "       ...,\n",
 823 |        "       [    0,     0,     0, ...,    10,    71,  2985],\n",
 824 |        "       [    0,     0,     0, ...,     2,    18,    75],\n",
 825 |        "       [    0,     0,     0, ..., 11009,   809,   267]], dtype=int32)"
 826 |       ]
 827 |      },
 828 |      "execution_count": 25,
 829 |      "metadata": {},
 830 |      "output_type": "execute_result"
 831 |     }
 832 |    ],
 833 |    "source": [
 834 |     "sequences_matrix_train_t = sequence.pad_sequences(sequences_train_t,maxlen=max_len_t)\n",
 835 |     "sequences_matrix_train_t"
 836 |    ]
 837 |   },
 838 |   {
 839 |    "cell_type": "code",
 840 |    "execution_count": 26,
 841 |    "metadata": {},
 842 |    "outputs": [],
 843 |    "source": [
 844 |     "sequences_test_t = tok.texts_to_sequences(X_test_t)\n",
 845 |     "sequences_matrix_test_t = sequence.pad_sequences(sequences_test_t,maxlen=max_len_t)"
 846 |    ]
 847 |   },
 848 |   {
 849 |    "cell_type": "code",
 850 |    "execution_count": 27,
 851 |    "metadata": {},
 852 |    "outputs": [
 853 |     {
 854 |      "data": {
 855 |       "text/plain": [
 856 |        "((412500, 18), (137500, 18), (412500, 10), (137500, 10))"
 857 |       ]
 858 |      },
 859 |      "execution_count": 27,
 860 |      "metadata": {},
 861 |      "output_type": "execute_result"
 862 |     }
 863 |    ],
 864 |    "source": [
 865 |     "sequences_matrix_train_t.shape,sequences_matrix_test_t.shape,y_train.shape,y_test.shape"
 866 |    ]
 867 |   },
 868 |   {
 869 |    "cell_type": "code",
 870 |    "execution_count": 28,
 871 |    "metadata": {},
 872 |    "outputs": [
 873 |     {
 874 |      "data": {
 875 |       "text/plain": [
 876 |        "20853"
 877 |       ]
 878 |      },
 879 |      "execution_count": 28,
 880 |      "metadata": {},
 881 |      "output_type": "execute_result"
 882 |     }
 883 |    ],
 884 |    "source": [
 885 |     "sent_lens_b=[]\n",
 886 |     "for sent in train['Body']:\n",
 887 |     "    sent_lens_b.append(len(word_tokenize(sent)))\n",
 888 |     "max(sent_lens_b)"
 889 |    ]
 890 |   },
 891 |   {
 892 |    "cell_type": "code",
 893 |    "execution_count": 29,
 894 |    "metadata": {},
 895 |    "outputs": [
 896 |     {
 897 |      "data": {
 898 |       "text/plain": [
 899 |        "575.0"
 900 |       ]
 901 |      },
 902 |      "execution_count": 29,
 903 |      "metadata": {},
 904 |      "output_type": "execute_result"
 905 |     }
 906 |    ],
 907 |    "source": [
 908 |     "np.quantile(sent_lens_b,0.90)"
 909 |    ]
 910 |   },
 911 |   {
 912 |    "cell_type": "code",
 913 |    "execution_count": 30,
 914 |    "metadata": {},
 915 |    "outputs": [],
 916 |    "source": [
 917 |     "max_len_b = 600\n",
 918 |     "tok = Tokenizer(char_level=False,split=' ')\n",
 919 |     "tok.fit_on_texts(X_train_b)\n",
 920 |     "sequences_train_b = tok.texts_to_sequences(X_train_b)"
 921 |    ]
 922 |   },
 923 |   {
 924 |    "cell_type": "code",
 925 |    "execution_count": 31,
 926 |    "metadata": {},
 927 |    "outputs": [
 928 |     {
 929 |      "data": {
 930 |       "text/plain": [
 931 |        "1292018"
 932 |       ]
 933 |      },
 934 |      "execution_count": 31,
 935 |      "metadata": {},
 936 |      "output_type": "execute_result"
 937 |     }
 938 |    ],
 939 |    "source": [
 940 |     "vocab_len_b =len(tok.index_word.keys())\n",
 941 |     "vocab_len_b "
 942 |    ]
 943 |   },
 944 |   {
 945 |    "cell_type": "code",
 946 |    "execution_count": 32,
 947 |    "metadata": {},
 948 |    "outputs": [
 949 |     {
 950 |      "data": {
 951 |       "text/plain": [
 952 |        "array([[   0,    0,    0, ...,   51, 2082,   91],\n",
 953 |        "       [   0,    0,    0, ..., 1408,  203,  825],\n",
 954 |        "       [   0,    0,    0, ...,   34,   51,   83],\n",
 955 |        "       ...,\n",
 956 |        "       [   0,    0,    0, ...,   20,   68,  687],\n",
 957 |        "       [   0,    0,    0, ...,  187,   58,   10],\n",
 958 |        "       [   0,    0,    0, ...,  194,  197,   10]], dtype=int32)"
 959 |       ]
 960 |      },
 961 |      "execution_count": 32,
 962 |      "metadata": {},
 963 |      "output_type": "execute_result"
 964 |     }
 965 |    ],
 966 |    "source": [
 967 |     "sequences_matrix_train_b = sequence.pad_sequences(sequences_train_b,maxlen=max_len_b)\n",
 968 |     "sequences_matrix_train_b"
 969 |    ]
 970 |   },
 971 |   {
 972 |    "cell_type": "code",
 973 |    "execution_count": 33,
 974 |    "metadata": {},
 975 |    "outputs": [],
 976 |    "source": [
 977 |     "sequences_test_b = tok.texts_to_sequences(X_test_b)\n",
 978 |     "sequences_matrix_test_b = sequence.pad_sequences(sequences_test_b,maxlen=max_len_b)"
 979 |    ]
 980 |   },
 981 |   {
 982 |    "cell_type": "code",
 983 |    "execution_count": 34,
 984 |    "metadata": {},
 985 |    "outputs": [
 986 |     {
 987 |      "data": {
 988 |       "text/plain": [
 989 |        "((412500, 18), (412500, 600), (412500, 10))"
 990 |       ]
 991 |      },
 992 |      "execution_count": 34,
 993 |      "metadata": {},
 994 |      "output_type": "execute_result"
 995 |     }
 996 |    ],
 997 |    "source": [
 998 |     "sequences_matrix_train_t.shape,sequences_matrix_train_b.shape,y_train.shape"
 999 |    ]
1000 |   },
1001 |   {
1002 |    "cell_type": "code",
1003 |    "execution_count": 35,
1004 |    "metadata": {},
1005 |    "outputs": [
1006 |     {
1007 |      "data": {
1008 |       "text/plain": [
1009 |        "((137500, 18), (137500, 600), (137500, 10))"
1010 |       ]
1011 |      },
1012 |      "execution_count": 35,
1013 |      "metadata": {},
1014 |      "output_type": "execute_result"
1015 |     }
1016 |    ],
1017 |    "source": [
1018 |     "sequences_matrix_test_t.shape,sequences_matrix_test_b.shape,y_test.shape"
1019 |    ]
1020 |   },
1021 |   {
1022 |    "cell_type": "code",
1023 |    "execution_count": 36,
1024 |    "metadata": {},
1025 |    "outputs": [],
1026 |    "source": [
1027 |     "def RNN():\n",
1028 |     "    # Title Only\n",
1029 |     "    title_input = Input(name='title_input',shape=[max_len_t])\n",
1030 |     "    title_Embed = Embedding(vocab_len_t+1,2000,input_length=max_len_t,mask_zero=True,name='title_Embed')(title_input)\n",
1031 |     "    gru_out_t = GRU(300)(title_Embed)\n",
1032 |     "    # auxiliary output to tune GRU weights smoothly \n",
1033 |     "    auxiliary_output = Dense(10, activation='sigmoid', name='aux_output')(gru_out_t)   \n",
1034 |     "    \n",
1035 |     "    # Body Only\n",
1036 |     "    body_input = Input(name='body_input',shape=[max_len_b]) \n",
1037 |     "    body_Embed = Embedding(vocab_len_b+1,170,input_length=max_len_b,mask_zero=True,name='body_Embed')(body_input)\n",
1038 |     "    gru_out_b = GRU(200)(body_Embed)\n",
1039 |     "    \n",
1040 |     "    # combined with GRU output\n",
1041 |     "    com = concatenate([gru_out_t, gru_out_b])\n",
1042 |     "    \n",
1043 |     "    # now the combined data is being fed to dense layers\n",
1044 |     "    dense1 = Dense(400,activation='relu')(com)\n",
1045 |     "    dp1 = Dropout(0.5)(dense1)\n",
1046 |     "    bn = BatchNormalization()(dp1) \n",
1047 |     "    dense2 = Dense(150,activation='relu')(bn)\n",
1048 |     "    \n",
1049 |     "    main_output = Dense(10, activation='sigmoid', name='main_output')(dense2)\n",
1050 |     "    \n",
1051 |     "    model = Model(inputs=[title_input, body_input],outputs=[main_output, auxiliary_output])\n",
1052 |     "    return model"
1053 |    ]
1054 |   },
1055 |   {
1056 |    "cell_type": "code",
1057 |    "execution_count": 37,
1058 |    "metadata": {},
1059 |    "outputs": [
1060 |     {
1061 |      "name": "stdout",
1062 |      "output_type": "stream",
1063 |      "text": [
1064 |       "WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
1065 |       "Instructions for updating:\n",
1066 |       "Colocations handled automatically by placer.\n",
1067 |       "WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
1068 |       "Instructions for updating:\n",
1069 |       "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n",
1070 |       "__________________________________________________________________________________________________\n",
1071 |       "Layer (type)                    Output Shape         Param #     Connected to                     \n",
1072 |       "==================================================================================================\n",
1073 |       "title_input (InputLayer)        (None, 18)           0                                            \n",
1074 |       "__________________________________________________________________________________________________\n",
1075 |       "body_input (InputLayer)         (None, 600)          0                                            \n",
1076 |       "__________________________________________________________________________________________________\n",
1077 |       "title_Embed (Embedding)         (None, 18, 2000)     137940000   title_input[0][0]                \n",
1078 |       "__________________________________________________________________________________________________\n",
1079 |       "body_Embed (Embedding)          (None, 600, 170)     219643230   body_input[0][0]                 \n",
1080 |       "__________________________________________________________________________________________________\n",
1081 |       "gru_1 (GRU)                     (None, 300)          2070900     title_Embed[0][0]                \n",
1082 |       "__________________________________________________________________________________________________\n",
1083 |       "gru_2 (GRU)                     (None, 200)          222600      body_Embed[0][0]                 \n",
1084 |       "__________________________________________________________________________________________________\n",
1085 |       "concatenate_1 (Concatenate)     (None, 500)          0           gru_1[0][0]                      \n",
1086 |       "                                                                 gru_2[0][0]                      \n",
1087 |       "__________________________________________________________________________________________________\n",
1088 |       "dense_1 (Dense)                 (None, 400)          200400      concatenate_1[0][0]              \n",
1089 |       "__________________________________________________________________________________________________\n",
1090 |       "dropout_1 (Dropout)             (None, 400)          0           dense_1[0][0]                    \n",
1091 |       "__________________________________________________________________________________________________\n",
1092 |       "batch_normalization_1 (BatchNor (None, 400)          1600        dropout_1[0][0]                  \n",
1093 |       "__________________________________________________________________________________________________\n",
1094 |       "dense_2 (Dense)                 (None, 150)          60150       batch_normalization_1[0][0]      \n",
1095 |       "__________________________________________________________________________________________________\n",
1096 |       "main_output (Dense)             (None, 10)           1510        dense_2[0][0]                    \n",
1097 |       "__________________________________________________________________________________________________\n",
1098 |       "aux_output (Dense)              (None, 10)           3010        gru_1[0][0]                      \n",
1099 |       "==================================================================================================\n",
1100 |       "Total params: 360,143,400\n",
1101 |       "Trainable params: 360,142,600\n",
1102 |       "Non-trainable params: 800\n",
1103 |       "__________________________________________________________________________________________________\n"
1104 |      ]
1105 |     }
1106 |    ],
1107 |    "source": [
1108 |     "model = RNN()\n",
1109 |     "model.summary()"
1110 |    ]
1111 |   },
1112 |   {
1113 |    "cell_type": "code",
1114 |    "execution_count": 38,
1115 |    "metadata": {},
1116 |    "outputs": [],
1117 |    "source": [
1118 |     "model.compile(optimizer='adam',loss={'main_output': 'categorical_crossentropy', 'aux_output': 'categorical_crossentropy'},\n",
1119 |     "              metrics=['accuracy'])"
1120 |    ]
1121 |   },
1122 |   {
1123 |    "cell_type": "code",
1124 |    "execution_count": 39,
1125 |    "metadata": {},
1126 |    "outputs": [
1127 |     {
1128 |      "name": "stdout",
1129 |      "output_type": "stream",
1130 |      "text": [
1131 |       "WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
1132 |       "Instructions for updating:\n",
1133 |       "Use tf.cast instead.\n"
1134 |      ]
1135 |     },
1136 |     {
1137 |      "name": "stderr",
1138 |      "output_type": "stream",
1139 |      "text": [
1140 |       "/opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:107: UserWarning: Converting sparse IndexedSlices to a dense Tensor with 137940000 elements. This may consume a large amount of memory.\n",
1141 |       "  num_elements)\n",
1142 |       "/opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:107: UserWarning: Converting sparse IndexedSlices to a dense Tensor with 219643230 elements. This may consume a large amount of memory.\n",
1143 |       "  num_elements)\n"
1144 |      ]
1145 |     },
1146 |     {
1147 |      "name": "stdout",
1148 |      "output_type": "stream",
1149 |      "text": [
1150 |       "Train on 412500 samples, validate on 137500 samples\n",
1151 |       "Epoch 1/5\n",
1152 |       "412500/412500 [==============================] - 819s 2ms/step - loss: 2.3168 - main_output_loss: 1.0405 - aux_output_loss: 1.2763 - main_output_acc: 0.7285 - aux_output_acc: 0.6705 - val_loss: 1.7714 - val_main_output_loss: 0.7258 - val_aux_output_loss: 1.0456 - val_main_output_acc: 0.8254 - val_aux_output_acc: 0.7285\n",
1153 |       "Epoch 2/5\n",
1154 |       "412500/412500 [==============================] - 797s 2ms/step - loss: 1.5874 - main_output_loss: 0.6490 - aux_output_loss: 0.9384 - main_output_acc: 0.8443 - aux_output_acc: 0.7582 - val_loss: 1.7043 - val_main_output_loss: 0.6581 - val_aux_output_loss: 1.0462 - val_main_output_acc: 0.8379 - val_aux_output_acc: 0.7286\n",
1155 |       "Epoch 3/5\n",
1156 |       "412500/412500 [==============================] - 797s 2ms/step - loss: 1.3994 - main_output_loss: 0.5472 - aux_output_loss: 0.8522 - main_output_acc: 0.8650 - aux_output_acc: 0.7774 - val_loss: 1.7369 - val_main_output_loss: 0.6660 - val_aux_output_loss: 1.0708 - val_main_output_acc: 0.8364 - val_aux_output_acc: 0.7263\n",
1157 |       "Epoch 4/5\n",
1158 |       "412500/412500 [==============================] - 798s 2ms/step - loss: 1.2719 - main_output_loss: 0.4735 - aux_output_loss: 0.7984 - main_output_acc: 0.8774 - aux_output_acc: 0.7885 - val_loss: 1.7988 - val_main_output_loss: 0.6976 - val_aux_output_loss: 1.1012 - val_main_output_acc: 0.8369 - val_aux_output_acc: 0.7240\n",
1159 |       "Epoch 5/5\n",
1160 |       "412500/412500 [==============================] - 797s 2ms/step - loss: 1.1665 - main_output_loss: 0.4110 - aux_output_loss: 0.7555 - main_output_acc: 0.8868 - aux_output_acc: 0.7976 - val_loss: 1.9099 - val_main_output_loss: 0.7671 - val_aux_output_loss: 1.1428 - val_main_output_acc: 0.8307 - val_aux_output_acc: 0.7237\n"
1161 |      ]
1162 |     }
1163 |    ],
1164 |    "source": [
1165 |     "results=model.fit({'title_input': sequences_matrix_train_t, 'body_input': sequences_matrix_train_b},\n",
1166 |     "          {'main_output': y_train, 'aux_output': y_train},\n",
1167 |     "          validation_data=[{'title_input': sequences_matrix_test_t, 'body_input': sequences_matrix_test_b},\n",
1168 |     "          {'main_output': y_test, 'aux_output': y_test}],\n",
1169 |     "          epochs=5, batch_size=800)"
1170 |    ]
1171 |   },
1172 |   {
1173 |    "cell_type": "code",
1174 |    "execution_count": 68,
1175 |    "metadata": {},
1176 |    "outputs": [
1177 |     {
1178 |      "name": "stdout",
1179 |      "output_type": "stream",
1180 |      "text": [
1181 |       "137500/137500 [==============================] - 1270s 9ms/step\n"
1182 |      ]
1183 |     }
1184 |    ],
1185 |    "source": [
1186 |     "(predicted_main, predicted_aux)=model.predict({'title_input': sequences_matrix_test_t, 'body_input': sequences_matrix_test_b},verbose=1)"
1187 |    ]
1188 |   },
1189 |   {
1190 |    "cell_type": "code",
1191 |    "execution_count": 70,
1192 |    "metadata": {},
1193 |    "outputs": [],
1194 |    "source": [
1195 |     "from sklearn.metrics import classification_report,f1_score"
1196 |    ]
1197 |   },
1198 |   {
1199 |    "cell_type": "code",
1200 |    "execution_count": 138,
1201 |    "metadata": {},
1202 |    "outputs": [
1203 |     {
1204 |      "name": "stdout",
1205 |      "output_type": "stream",
1206 |      "text": [
1207 |       "0.8424636536796537\n"
1208 |      ]
1209 |     },
1210 |     {
1211 |      "name": "stderr",
1212 |      "output_type": "stream",
1213 |      "text": [
1214 |       "/opt/conda/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in samples with no predicted labels.\n",
1215 |       "  'precision', 'predicted', average, warn_for)\n"
1216 |      ]
1217 |     }
1218 |    ],
1219 |    "source": [
1220 |     "print(f1_score(y_test,predicted_main>.55,average='samples'))"
1221 |    ]
1222 |   },
1223 |   {
1224 |    "cell_type": "code",
1225 |    "execution_count": 137,
1226 |    "metadata": {},
1227 |    "outputs": [
1228 |     {
1229 |      "name": "stdout",
1230 |      "output_type": "stream",
1231 |      "text": [
1232 |       "              precision    recall  f1-score   support\n",
1233 |       "\n",
1234 |       "           0       0.97      0.93      0.95     17054\n",
1235 |       "           1       0.92      0.84      0.88     20681\n",
1236 |       "           2       0.92      0.81      0.86      9700\n",
1237 |       "           3       0.69      0.53      0.60     11304\n",
1238 |       "           4       0.96      0.91      0.94      8897\n",
1239 |       "           5       0.91      0.80      0.85     22472\n",
1240 |       "           6       0.82      0.72      0.76     22938\n",
1241 |       "           7       0.81      0.83      0.82     16150\n",
1242 |       "           8       0.92      0.90      0.91     19659\n",
1243 |       "           9       0.97      0.92      0.95     11576\n",
1244 |       "\n",
1245 |       "   micro avg       0.89      0.82      0.85    160431\n",
1246 |       "   macro avg       0.89      0.82      0.85    160431\n",
1247 |       "weighted avg       0.89      0.82      0.85    160431\n",
1248 |       " samples avg       0.86      0.85      0.84    160431\n",
1249 |       "\n"
1250 |      ]
1251 |     },
1252 |     {
1253 |      "name": "stderr",
1254 |      "output_type": "stream",
1255 |      "text": [
1256 |       "/opt/conda/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in samples with no predicted labels.\n",
1257 |       "  'precision', 'predicted', average, warn_for)\n"
1258 |      ]
1259 |     }
1260 |    ],
1261 |    "source": [
1262 |     "print(classification_report(y_test,predicted_main>.55))"
1263 |    ]
1264 |   },
1265 |   {
1266 |    "cell_type": "code",
1267 |    "execution_count": 131,
1268 |    "metadata": {},
1269 |    "outputs": [
1270 |     {
1271 |      "data": {
1272 |       "text/plain": [
1273 |        "Id                                                16470700\n",
1274 |        "Title    NetworkOnMainThreadException- Have tried makin...\n",
1275 |        "Body     I've been trying to get this to work for a whi...\n",
1276 |        "Tags                                       [java, android]\n",
1277 |        "Name: 250148, dtype: object"
1278 |       ]
1279 |      },
1280 |      "execution_count": 131,
1281 |      "metadata": {},
1282 |      "output_type": "execute_result"
1283 |     }
1284 |    ],
1285 |    "source": [
1286 |     "test.iloc[24]"
1287 |    ]
1288 |   },
1289 |   {
1290 |    "cell_type": "code",
1291 |    "execution_count": 134,
1292 |    "metadata": {},
1293 |    "outputs": [
1294 |     {
1295 |      "data": {
1296 |       "text/plain": [
1297 |        "array([1.  , 0.  , 0.  , 0.  , 0.  , 0.84, 0.  , 0.  , 0.  , 0.  ],\n",
1298 |        "      dtype=float32)"
1299 |       ]
1300 |      },
1301 |      "execution_count": 134,
1302 |      "metadata": {},
1303 |      "output_type": "execute_result"
1304 |     }
1305 |    ],
1306 |    "source": [
1307 |     "predicted_main[24].round(decimals = 2)"
1308 |    ]
1309 |   },
1310 |   {
1311 |    "cell_type": "code",
1312 |    "execution_count": 92,
1313 |    "metadata": {},
1314 |    "outputs": [
1315 |     {
1316 |      "data": {
1317 |       "text/plain": [
1318 |        "array(['android', 'c#', 'c++', 'html', 'ios', 'java', 'javascript',\n",
1319 |        "       'jquery', 'php', 'python'], dtype=object)"
1320 |       ]
1321 |      },
1322 |      "execution_count": 92,
1323 |      "metadata": {},
1324 |      "output_type": "execute_result"
1325 |     }
1326 |    ],
1327 |    "source": [
1328 |     "labels"
1329 |    ]
1330 |   },
1331 |   {
1332 |    "cell_type": "code",
1333 |    "execution_count": 79,
1334 |    "metadata": {},
1335 |    "outputs": [],
1336 |    "source": [
1337 |     "model.save('./stackoverflow_tags.h5')"
1338 |    ]
1339 |   }
1340 |  ],
1341 |  "metadata": {
1342 |   "kernelspec": {
1343 |    "display_name": "Python 3",
1344 |    "language": "python",
1345 |    "name": "python3"
1346 |   },
1347 |   "language_info": {
1348 |    "codemirror_mode": {
1349 |     "name": "ipython",
1350 |     "version": 3
1351 |    },
1352 |    "file_extension": ".py",
1353 |    "mimetype": "text/x-python",
1354 |    "name": "python",
1355 |    "nbconvert_exporter": "python",
1356 |    "pygments_lexer": "ipython3",
1357 |    "version": "3.6.7"
1358 |   }
1359 |  },
1360 |  "nbformat": 4,
1361 |  "nbformat_minor": 1
1362 | }
1363 | 


--------------------------------------------------------------------------------