24 |
23 |
24 |
![](\"img/decision_tree_titanic_1.png\")
"
179 | ]
180 | },
181 | {
182 | "cell_type": "markdown",
183 | "metadata": {},
184 | "source": [
185 | "* You first **fit** such a model to your training data, which means deciding (based on the training data) which decisions will split at each branching point in the tree: e.g., that the first branch is on 'Male' or not and that 'Male' results in a prediction of 'Dead'. "
186 | ]
187 | },
188 | {
189 | "cell_type": "markdown",
190 | "metadata": {},
191 | "source": [
192 | "* Before fitting a model to your `data`, split it back into training and test sets:"
193 | ]
194 | },
195 | {
196 | "cell_type": "code",
197 | "execution_count": null,
198 | "metadata": {},
199 | "outputs": [],
200 | "source": [
201 | "data_train = data.iloc[:891]\n",
202 | "data_test = data.iloc[891:]"
203 | ]
204 | },
205 | {
206 | "cell_type": "markdown",
207 | "metadata": {},
208 | "source": [
209 | "* You'll use `scikit-learn`, which requires your data as arrays, not DataFrames so transform them:"
210 | ]
211 | },
212 | {
213 | "cell_type": "code",
214 | "execution_count": null,
215 | "metadata": {},
216 | "outputs": [],
217 | "source": [
218 | "X = ____\n",
219 | "test = ____\n",
220 | "y = ____"
221 | ]
222 | },
223 | {
224 | "cell_type": "markdown",
225 | "metadata": {},
226 | "source": [
227 | "* Now you get to build your decision tree classifier! First create such a model with `max_depth=3` and then fit it your data:"
228 | ]
229 | },
230 | {
231 | "cell_type": "code",
232 | "execution_count": null,
233 | "metadata": {},
234 | "outputs": [],
235 | "source": [
236 | "# Instantiate model and fit to data\n",
237 | "clf = ____\n",
238 | "___"
239 | ]
240 | },
241 | {
242 | "cell_type": "markdown",
243 | "metadata": {},
244 | "source": [
245 | "* Make predictions on your test set, create a new column 'Survived' and store your predictions in it. Save 'PassengerId' and 'Survived' columns of `df_test` to a .csv and submit to Kaggle."
246 | ]
247 | },
248 | {
249 | "cell_type": "code",
250 | "execution_count": null,
251 | "metadata": {},
252 | "outputs": [],
253 | "source": [
254 | "# Make predictions and store in 'Survived' column of df_test\n",
255 | "Y_pred = ____\n",
256 | "df_test['Survived'] = Y_pred"
257 | ]
258 | },
259 | {
260 | "cell_type": "code",
261 | "execution_count": null,
262 | "metadata": {},
263 | "outputs": [],
264 | "source": [
265 | "df_test[['PassengerId', 'Survived']].to_csv('data/predictions/1st_dec_tree.csv', index=False)"
266 | ]
267 | },
268 | {
269 | "cell_type": "markdown",
270 | "metadata": {},
271 | "source": [
272 | "* What is the accuracy of your model, as reported by Kaggle?\n",
273 | "\n",
274 | "Accuracy = ??"
275 | ]
276 | },
277 | {
278 | "cell_type": "markdown",
279 | "metadata": {},
280 | "source": [
281 | "**Recap:**\n",
282 | "* You've got your data in a form to build first machine learning model.\n",
283 | "* You've built your first machine learning model: a decision tree classifier.\n",
284 | "\n",
285 | "**Up next:** figure out what this `max_depth` argument was, why we chose it and explore `train_test_split`.\n",
286 | "\n",
287 | "For more on `scikit-learn`, check out our [Supervised Learning with scikit-learn course](https://www.datacamp.com/courses/supervised-learning-with-scikit-learn). \n",
288 | "\n",
289 | "If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp)."
290 | ]
291 | },
292 | {
293 | "cell_type": "markdown",
294 | "metadata": {},
295 | "source": [
296 | "## What was this decision tree classifier?"
297 | ]
298 | },
299 | {
300 | "cell_type": "markdown",
301 | "metadata": {},
302 | "source": [
303 | "![](\"img/decision_tree_titanic_3.png\")
"
304 | ]
305 | },
306 | {
307 | "cell_type": "markdown",
308 | "metadata": {},
309 | "source": [
310 | "Note: you can use `graphviz` to generate figures such as this. See the `scikit-learn` documentation [here](http://scikit-learn.org/stable/modules/tree.html) for further details. In building this model, what you're essentially doing is creating a _decision boundary_ in the space of feature variables, for example (image from [here](http://scikit-learn.org/stable/auto_examples/ensemble/plot_voting_decision_regions.html)):"
311 | ]
312 | },
313 | {
314 | "cell_type": "markdown",
315 | "metadata": {},
316 | "source": [
317 | "![](\"img/dec_bound.png\")
"
318 | ]
319 | },
320 | {
321 | "cell_type": "markdown",
322 | "metadata": {},
323 | "source": [
324 | "## Why would you choose max_depth=3 ?"
325 | ]
326 | },
327 | {
328 | "cell_type": "markdown",
329 | "metadata": {},
330 | "source": [
331 | "The depth of the tree is known as a hyperparameter, which means a parameter we need to decide before we fit the model to the data. If we choose a larger `max_depth`, we'll get a more complex decision boundary. \n",
332 | "\n",
333 | "* If our decision boundary is _too complex_ we can overfit to the data, which means that our model will be describing noise as well as signal.\n",
334 | "\n",
335 | "* If our max_depth is too small, we may be underfitting the data, meaning that our model doesn't contain enough of the signal.\n",
336 | "\n",
337 | "**How do we tell whether we're overfitting or underfitting?** Note: this is also referred to as the bias-variance trade-off and we won;t go into details on that here."
338 | ]
339 | },
340 | {
341 | "cell_type": "markdown",
342 | "metadata": {},
343 | "source": [
344 | "One way is to hold out a test set from our training data. We can then fit the model to our training data, make predictions on our test set and see how well our prediction does on the test set. \n",
345 | "\n",
346 | "* You'll now do this: split your original training data into training and test sets:"
347 | ]
348 | },
349 | {
350 | "cell_type": "code",
351 | "execution_count": null,
352 | "metadata": {},
353 | "outputs": [],
354 | "source": [
355 | "X_train, X_test, y_train, y_test = train_test_split(\n",
356 | " ____, ____, test_size=0.33, random_state=42, stratify=y)"
357 | ]
358 | },
359 | {
360 | "cell_type": "markdown",
361 | "metadata": {},
362 | "source": [
363 | "* Iterate over values of `max_depth` ranging from 1 to 9 and plot the accuracy of the models on training and test sets:"
364 | ]
365 | },
366 | {
367 | "cell_type": "code",
368 | "execution_count": null,
369 | "metadata": {},
370 | "outputs": [],
371 | "source": [
372 | "# Setup arrays to store train and test accuracies\n",
373 | "dep = np.arange(1, 9)\n",
374 | "train_accuracy = np.empty(len(dep))\n",
375 | "test_accuracy = np.empty(len(dep))\n",
376 | "\n",
377 | "# Loop over different values of k\n",
378 | "for i, k in enumerate(dep):\n",
379 | " # Setup a k-NN Classifier with k neighbors: knn\n",
380 | " clf = tree.DecisionTreeClassifier(max_depth=k)\n",
381 | "\n",
382 | " # Fit the classifier to the training data\n",
383 | " clf.fit(X_train, y_train)\n",
384 | " \n",
385 | " #Compute accuracy on the training set\n",
386 | " train_accuracy[i] = clf.score(X_train, y_train)\n",
387 | "\n",
388 | " #Compute accuracy on the testing set\n",
389 | " test_accuracy[i] = clf.score(X_test, y_test)\n",
390 | "\n",
391 | "# Generate plot\n",
392 | "plt.title('clf: Varying depth of tree')\n",
393 | "plt.plot(dep, test_accuracy, label = 'Testing Accuracy')\n",
394 | "plt.plot(dep, train_accuracy, label = 'Training Accuracy')\n",
395 | "plt.legend()\n",
396 | "plt.xlabel('Depth of tree')\n",
397 | "plt.ylabel('Accuracy')\n",
398 | "plt.show()"
399 | ]
400 | },
401 | {
402 | "cell_type": "markdown",
403 | "metadata": {},
404 | "source": [
405 | "**Recap:**\n",
406 | "* You've got your data in a form to build first machine learning model.\n",
407 | "* You've built your first machine learning model: a decision tree classifier.\n",
408 | "* You've learnt about `train_test_split` and how it helps us to choose ML model hyperparameters.\n",
409 | "\n",
410 | "**Up next:** Engineer some new features and build some new models! Open the notebook `3-titanic_feature_engineering_ML.ipynb`.\n",
411 | "\n",
412 | "For more on `scikit-learn`, check out our [Supervised Learning with scikit-learn course](https://www.datacamp.com/courses/supervised-learning-with-scikit-learn). \n",
413 | "\n",
414 | "If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp)."
415 | ]
416 | }
417 | ],
418 | "metadata": {
419 | "kernelspec": {
420 | "display_name": "Python 3",
421 | "language": "python",
422 | "name": "python3"
423 | },
424 | "language_info": {
425 | "codemirror_mode": {
426 | "name": "ipython",
427 | "version": 3
428 | },
429 | "file_extension": ".py",
430 | "mimetype": "text/x-python",
431 | "name": "python",
432 | "nbconvert_exporter": "python",
433 | "pygments_lexer": "ipython3",
434 | "version": "3.6.3"
435 | }
436 | },
437 | "nbformat": 4,
438 | "nbformat_minor": 2
439 | }
440 |
--------------------------------------------------------------------------------
/3-titanic_feature_engineering_ML.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "## Feature Engineering and Machine Learning"
8 | ]
9 | },
10 | {
11 | "cell_type": "code",
12 | "execution_count": null,
13 | "metadata": {},
14 | "outputs": [],
15 | "source": [
16 | "# Imports\n",
17 | "import pandas as pd\n",
18 | "import matplotlib.pyplot as plt\n",
19 | "import seaborn as sns\n",
20 | "import re\n",
21 | "import numpy as np\n",
22 | "from sklearn import tree\n",
23 | "from sklearn.model_selection import GridSearchCV\n",
24 | "\n",
25 | "# Figures inline and set visualization style\n",
26 | "%matplotlib inline\n",
27 | "sns.set()\n",
28 | "\n",
29 | "# Import data\n",
30 | "df_train = pd.read_csv('data/train.csv')\n",
31 | "df_test = pd.read_csv('data/test.csv')\n",
32 | "\n",
33 | "# Store target variable of training data in a safe place\n",
34 | "survived_train = df_train.Survived\n",
35 | "\n",
36 | "# Concatenate training and test sets\n",
37 | "data = pd.concat([df_train.drop(['Survived'], axis=1), df_test])\n",
38 | "\n",
39 | "# View head\n",
40 | "data.head()"
41 | ]
42 | },
43 | {
44 | "cell_type": "markdown",
45 | "metadata": {},
46 | "source": [
47 | "## Why feature engineer at all?"
48 | ]
49 | },
50 | {
51 | "cell_type": "markdown",
52 | "metadata": {},
53 | "source": [
54 | "To extract more information from your data. For example, check out the 'Name' column:"
55 | ]
56 | },
57 | {
58 | "cell_type": "code",
59 | "execution_count": null,
60 | "metadata": {},
61 | "outputs": [],
62 | "source": [
63 | "# View head of 'Name' column\n",
64 | "____"
65 | ]
66 | },
67 | {
68 | "cell_type": "markdown",
69 | "metadata": {},
70 | "source": [
71 | "Notive that this columns contains strings (text) that contain 'Title' such as 'Mr', 'Master' and 'Dona'. You can use regular expressions to extract the Title (to learn more about regular expressions, check out my write up of our last [FB Live code along event](https://www.datacamp.com/community/tutorials/web-scraping-python-nlp)):"
72 | ]
73 | },
74 | {
75 | "cell_type": "code",
76 | "execution_count": null,
77 | "metadata": {},
78 | "outputs": [],
79 | "source": [
80 | "# Extract Title from Name, store in column and plot barplot\n",
81 | "data['Title'] = data.Name.apply(lambda x: re.search(' ([A-Z][a-z]+)\\.', x).group(1))\n",
82 | "sns.countplot(x='Title', data=data);\n",
83 | "plt.xticks(rotation=45);"
84 | ]
85 | },
86 | {
87 | "cell_type": "markdown",
88 | "metadata": {},
89 | "source": [
90 | "* There are several titles and it makes sense to put them in fewer buckets:"
91 | ]
92 | },
93 | {
94 | "cell_type": "code",
95 | "execution_count": null,
96 | "metadata": {},
97 | "outputs": [],
98 | "source": [
99 | "data['Title'] = data['Title'].replace({'Mlle':'Miss', 'Mme':'Mrs', 'Ms':'Miss'})\n",
100 | "data['Title'] = data['Title'].replace(['Don', 'Dona', 'Rev', 'Dr',\n",
101 | " 'Major', 'Lady', 'Sir', 'Col', 'Capt', 'Countess', 'Jonkheer'],'Special')\n",
102 | "sns.countplot(x='Title', data=data);\n",
103 | "plt.xticks(rotation=45);"
104 | ]
105 | },
106 | {
107 | "cell_type": "markdown",
108 | "metadata": {},
109 | "source": [
110 | "* Check out your data again and make sure that we have a 'Title' column:"
111 | ]
112 | },
113 | {
114 | "cell_type": "code",
115 | "execution_count": null,
116 | "metadata": {},
117 | "outputs": [],
118 | "source": [
119 | "# View head of data\n",
120 | "____"
121 | ]
122 | },
123 | {
124 | "cell_type": "markdown",
125 | "metadata": {},
126 | "source": [
127 | "### Being cabinless may be important"
128 | ]
129 | },
130 | {
131 | "cell_type": "markdown",
132 | "metadata": {},
133 | "source": [
134 | "* There are several NaNs (missing values) in the 'Cabin' column. It is reasonable to presume that those NaNs didn't have a cabin, which may tell us something about 'Survival' so now create a new column that encodes this information:"
135 | ]
136 | },
137 | {
138 | "cell_type": "code",
139 | "execution_count": null,
140 | "metadata": {},
141 | "outputs": [],
142 | "source": [
143 | "# Did they have a Cabin?\n",
144 | "____\n",
145 | "\n",
146 | "# View head of data\n",
147 | "data.head()"
148 | ]
149 | },
150 | {
151 | "cell_type": "markdown",
152 | "metadata": {},
153 | "source": [
154 | "* Drop columns that contain no more useful information (or that we're not sure what to do with:) `['Cabin', 'Name', 'PassengerId', 'Ticket']`:"
155 | ]
156 | },
157 | {
158 | "cell_type": "code",
159 | "execution_count": null,
160 | "metadata": {},
161 | "outputs": [],
162 | "source": [
163 | "# Drop columns and view head\n",
164 | "____\n",
165 | "data.head()"
166 | ]
167 | },
168 | {
169 | "cell_type": "markdown",
170 | "metadata": {},
171 | "source": [
172 | "**Recap:**\n",
173 | "* You've engineered some new features such as 'Title' and 'Has_Cabin': congrats!\n",
174 | "\n",
175 | "**Up next:** deal with missing values and bin your numerical data, transform all features into numeric variables. Then you'll build your final model for this session.\n",
176 | "\n",
177 | "If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp)."
178 | ]
179 | },
180 | {
181 | "cell_type": "markdown",
182 | "metadata": {},
183 | "source": [
184 | "### Dealing with missing values"
185 | ]
186 | },
187 | {
188 | "cell_type": "markdown",
189 | "metadata": {},
190 | "source": [
191 | "* Figure out if there are any missing values left:"
192 | ]
193 | },
194 | {
195 | "cell_type": "code",
196 | "execution_count": null,
197 | "metadata": {},
198 | "outputs": [],
199 | "source": [
200 | "____"
201 | ]
202 | },
203 | {
204 | "cell_type": "markdown",
205 | "metadata": {},
206 | "source": [
207 | "* Impute missing values:"
208 | ]
209 | },
210 | {
211 | "cell_type": "code",
212 | "execution_count": null,
213 | "metadata": {},
214 | "outputs": [],
215 | "source": [
216 | "# Impute missing values for Age, Fare, Embarked\n",
217 | "data.Age = ____\n",
218 | "data.Fare = ____\n",
219 | "data['Embarked'] = data['Embarked'].fillna('S')\n",
220 | "data.info()"
221 | ]
222 | },
223 | {
224 | "cell_type": "code",
225 | "execution_count": null,
226 | "metadata": {},
227 | "outputs": [],
228 | "source": [
229 | "data.head()"
230 | ]
231 | },
232 | {
233 | "cell_type": "markdown",
234 | "metadata": {},
235 | "source": [
236 | "### Bin numerical data"
237 | ]
238 | },
239 | {
240 | "cell_type": "markdown",
241 | "metadata": {},
242 | "source": [
243 | "* Use the `pandas` function `qcut` to bin your numerical data:"
244 | ]
245 | },
246 | {
247 | "cell_type": "code",
248 | "execution_count": null,
249 | "metadata": {},
250 | "outputs": [],
251 | "source": [
252 | "# Binning numerical columns\n",
253 | "data['CatAge'] = ____\n",
254 | "data['CatFare']= ____\n",
255 | "data.head()"
256 | ]
257 | },
258 | {
259 | "cell_type": "markdown",
260 | "metadata": {},
261 | "source": [
262 | "* You can now safely drop 'Age' and 'Fare' columns:"
263 | ]
264 | },
265 | {
266 | "cell_type": "code",
267 | "execution_count": null,
268 | "metadata": {},
269 | "outputs": [],
270 | "source": [
271 | "data = ____\n",
272 | "data.head()"
273 | ]
274 | },
275 | {
276 | "cell_type": "markdown",
277 | "metadata": {},
278 | "source": [
279 | "## Create a new column: number of members in family onboard"
280 | ]
281 | },
282 | {
283 | "cell_type": "code",
284 | "execution_count": null,
285 | "metadata": {},
286 | "outputs": [],
287 | "source": [
288 | "# Create column of number of Family members onboard\n",
289 | "data['Fam_Size'] = ____"
290 | ]
291 | },
292 | {
293 | "cell_type": "code",
294 | "execution_count": null,
295 | "metadata": {},
296 | "outputs": [],
297 | "source": [
298 | "# Drop columns\n",
299 | "data = data.drop(['SibSp','Parch'], axis=1)\n",
300 | "data.head()"
301 | ]
302 | },
303 | {
304 | "cell_type": "markdown",
305 | "metadata": {},
306 | "source": [
307 | "## Transform all variables into numerical variables"
308 | ]
309 | },
310 | {
311 | "cell_type": "code",
312 | "execution_count": null,
313 | "metadata": {},
314 | "outputs": [],
315 | "source": [
316 | "# Transform into binary variables\n",
317 | "data_dum = ____\n",
318 | "data_dum.head()"
319 | ]
320 | },
321 | {
322 | "cell_type": "markdown",
323 | "metadata": {},
324 | "source": [
325 | "**Recap:**\n",
326 | "* You've engineered some new features such as 'Title' and 'Has_Cabin';\n",
327 | "* You've dealt with missing values, binned your numerical data and transformed all features into numeric variables.\n",
328 | "\n",
329 | "**Up next:** It's time ... to build your final model!\n",
330 | "\n",
331 | "If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp)."
332 | ]
333 | },
334 | {
335 | "cell_type": "markdown",
336 | "metadata": {},
337 | "source": [
338 | "## Building models with our new dataset!"
339 | ]
340 | },
341 | {
342 | "cell_type": "markdown",
343 | "metadata": {},
344 | "source": [
345 | "* As before, first you'll split your `data` back into training and test sets; then you'll transform them into arrays:"
346 | ]
347 | },
348 | {
349 | "cell_type": "code",
350 | "execution_count": null,
351 | "metadata": {},
352 | "outputs": [],
353 | "source": [
354 | "# Split into test.train\n",
355 | "data_train = data_dum.iloc[:891]\n",
356 | "data_test = data_dum.iloc[891:]\n",
357 | "\n",
358 | "# Transform into arrays for scikit-learn\n",
359 | "X = data_train.values\n",
360 | "test = data_test.values\n",
361 | "y = survived_train.values"
362 | ]
363 | },
364 | {
365 | "cell_type": "markdown",
366 | "metadata": {},
367 | "source": [
368 | "You're now going to build a decision tree on your brand new feature-engineered dataset. To choose your hyperparameter `max_depth`, you'll use a variation on test train split called cross validation.\n",
369 | "\n",
370 | "![](\"img/cv.png\")
\n",
371 | "\n",
372 | "We begin by splitting the dataset into 5 groups or *folds*. Then we hold out the first fold as a test set, fit our model on the remaining four folds, predict on the test set and compute the metric of interest. Next we hold out the second fold as our test set, fit on the remaining data, predict on the test set and compute the metric of interest. Then similarly with the third, fourth and fifth. \t\t\n",
373 | " \n",
374 | "As a result we get five values of accuracy, from which we can compute statistics\tof interest, such as the median and/or mean and 95% confidence intervals. \n",
375 | "\n",
376 | "We do this for each value of each hyperparameter that we're tuning and choose the set of hyperparameters that performs the best. This is called _grid search_.\n",
377 | "\n",
378 | "* Let's get it! In the following, you'll use cross validation and grid search to choose the best `max_depth` for your new feature engineered dataset:"
379 | ]
380 | },
381 | {
382 | "cell_type": "code",
383 | "execution_count": null,
384 | "metadata": {},
385 | "outputs": [],
386 | "source": [
387 | "# Setup the hyperparameter grid\n",
388 | "dep = ____\n",
389 | "param_grid = ____\n",
390 | "\n",
391 | "# Instantiate a logistic regression classifier: clf\n",
392 | "clf = ____\n",
393 | "\n",
394 | "# Instantiate the GridSearchCV object: clf_cv\n",
395 | "clf_cv = ____\n",
396 | "\n",
397 | "# Fit it to the data\n",
398 | "____\n",
399 | "\n",
400 | "# Print the tuned parameter and score\n",
401 | "print(\"Tuned Decision Tree Parameters: {}\".format(clf_cv.best_params_))\n",
402 | "print(\"Best score is {}\".format(clf_cv.best_score_))\n"
403 | ]
404 | },
405 | {
406 | "cell_type": "markdown",
407 | "metadata": {},
408 | "source": [
409 | "* Make predictions on your test set, create a new column 'Survived' and store your predictions in it. Save 'PassengerId' and 'Survived' columns of `df_test` to a .csv and submit to Kaggle."
410 | ]
411 | },
412 | {
413 | "cell_type": "code",
414 | "execution_count": null,
415 | "metadata": {},
416 | "outputs": [],
417 | "source": [
418 | "Y_pred = ____\n",
419 | "df_test['Survived'] = Y_pred\n",
420 | "df_test[['PassengerId', 'Survived']].to_csv('data/predictions/dec_tree_feat_eng.csv', index=False)"
421 | ]
422 | },
423 | {
424 | "cell_type": "markdown",
425 | "metadata": {},
426 | "source": [
427 | "* What was the accuracy?\n",
428 | "\n",
429 | "_Accuracy_ = ??"
430 | ]
431 | },
432 | {
433 | "cell_type": "markdown",
434 | "metadata": {},
435 | "source": [
436 | "## Next steps"
437 | ]
438 | },
439 | {
440 | "cell_type": "markdown",
441 | "metadata": {},
442 | "source": [
443 | "See if you can do some more feature engineering and try some new models out to improve on this score. I'll post all of this on github and on the DataCamp community and it would be great to see all of you improve on these models.\n",
444 | "\n",
445 | "There's a lot more pre-processing that you'd like to learn about, such as scaling your data. You'll also find scikit-learn pipelines super useful. Check out our [Supervised Learning with scikit-learn course](https://www.datacamp.com/courses/supervised-learning-with-scikit-learn) and the [scikit-learn docs](http://scikit-learn.org/stable/) for all of this and more."
446 | ]
447 | }
448 | ],
449 | "metadata": {
450 | "kernelspec": {
451 | "display_name": "Python 3",
452 | "language": "python",
453 | "name": "python3"
454 | },
455 | "language_info": {
456 | "codemirror_mode": {
457 | "name": "ipython",
458 | "version": 3
459 | },
460 | "file_extension": ".py",
461 | "mimetype": "text/x-python",
462 | "name": "python",
463 | "nbconvert_exporter": "python",
464 | "pygments_lexer": "ipython3",
465 | "version": "3.6.3"
466 | }
467 | },
468 | "nbformat": 4,
469 | "nbformat_minor": 2
470 | }
471 |
--------------------------------------------------------------------------------
/1-titanic_EDA_first_models.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "## How to complete a Kaggle Competition with Machine Learning"
8 | ]
9 | },
10 | {
11 | "cell_type": "markdown",
12 | "metadata": {},
13 | "source": [
14 | "In this code along session, you'll build several algorithms of increasing complexity that predict whether any given passenger on the Titanic survived or not, given data on them such as the fare they paid, where they embarked and their age.\n",
15 | "\n",
16 | "![](\"img/nytimes.jpg\")
"
17 | ]
18 | },
19 | {
20 | "cell_type": "markdown",
21 | "metadata": {},
22 | "source": [
23 | "In particular, you'll build _supervised learning_ models. _Supervised learning_ is the branch of machine learning (ML) that involves predicting labels, such as 'Survived' or 'Not'. Such models:\n",
24 | "\n",
25 | "* it learns from labelled data, e.g. data that includes whether a passenger survived (called model training).\n",
26 | "* and then predicts on unlabelled data.\n",
27 | "\n",
28 | "On Kaggle, a platform for predictive modelling and analytics competitions, these are called train and test sets because\n",
29 | "\n",
30 | "* You want to build a model that learns patterns in the training set\n",
31 | "* You _then_ use the model to make predictions on the test set!\n",
32 | "\n",
33 | "Kaggle then tells you the **percentage that you got correct**: this is known as the _accuracy_ of your model."
34 | ]
35 | },
36 | {
37 | "cell_type": "markdown",
38 | "metadata": {},
39 | "source": [
40 | "## Approach\n",
41 | "\n",
42 | "A good way to approach supervised learning:\n",
43 | "\n",
44 | "* Exploratory Data Analysis (EDA);\n",
45 | "* Build a quick and dirty model (baseline);\n",
46 | "* Iterate;\n",
47 | "* Engineer features;\n",
48 | "* Get model that performs better.\n",
49 | "\n",
50 | "In this code along session, we'll do all of these! We also have free courses that get you up and running with machine learning for the Titanic dataset in [Python](https://campus.datacamp.com/courses/kaggle-python-tutorial-on-machine-learning) and [R](https://campus.datacamp.com/courses/kaggle-r-tutorial-on-machine-learning)."
51 | ]
52 | },
53 | {
54 | "cell_type": "markdown",
55 | "metadata": {},
56 | "source": [
57 | "**Note:** We may move quickly at some points in order to get a bit further along. I'll answer questions in the live event but also feel free to chime in and help each other in the comments."
58 | ]
59 | },
60 | {
61 | "cell_type": "markdown",
62 | "metadata": {},
63 | "source": [
64 | "## Import you data and check it out"
65 | ]
66 | },
67 | {
68 | "cell_type": "code",
69 | "execution_count": null,
70 | "metadata": {},
71 | "outputs": [],
72 | "source": [
73 | "# Import modules\n",
74 | "import pandas as pd\n",
75 | "import matplotlib.pyplot as plt\n",
76 | "import seaborn as sns\n",
77 | "from sklearn import tree\n",
78 | "from sklearn.metrics import accuracy_score\n",
79 | "\n",
80 | "# Figures inline and set visualization style\n",
81 | "%matplotlib inline\n",
82 | "sns.set()"
83 | ]
84 | },
85 | {
86 | "cell_type": "code",
87 | "execution_count": null,
88 | "metadata": {},
89 | "outputs": [],
90 | "source": [
91 | "# Import test and train datasets\n",
92 | "df_train = ____\n",
93 | "df_test = ____\n",
94 | "\n",
95 | "# View first lines of training data\n",
96 | "____"
97 | ]
98 | },
99 | {
100 | "cell_type": "markdown",
101 | "metadata": {},
102 | "source": [
103 | "* What are all these features? Check out the Kaggle data documentation [here](https://www.kaggle.com/c/titanic/data).\n",
104 | "\n",
105 | "**Important note on terminology:** \n",
106 | "* The _target variable_ is the one you are trying to predict;\n",
107 | "* Other variables are known as _features_ (or _predictor variables_)."
108 | ]
109 | },
110 | {
111 | "cell_type": "code",
112 | "execution_count": null,
113 | "metadata": {},
114 | "outputs": [],
115 | "source": [
116 | "# View first lines of test data\n",
117 | "____"
118 | ]
119 | },
120 | {
121 | "cell_type": "markdown",
122 | "metadata": {},
123 | "source": [
124 | "* Use the DataFrame `.info()` method to check out datatypes, missing values and more (of `df_train`)."
125 | ]
126 | },
127 | {
128 | "cell_type": "code",
129 | "execution_count": null,
130 | "metadata": {},
131 | "outputs": [],
132 | "source": [
133 | "____"
134 | ]
135 | },
136 | {
137 | "cell_type": "markdown",
138 | "metadata": {},
139 | "source": [
140 | "* Use the DataFrame `.describe()` method to check out summary statistics of numeric columns (of `df_train`)."
141 | ]
142 | },
143 | {
144 | "cell_type": "code",
145 | "execution_count": null,
146 | "metadata": {},
147 | "outputs": [],
148 | "source": [
149 | "____"
150 | ]
151 | },
152 | {
153 | "cell_type": "markdown",
154 | "metadata": {},
155 | "source": [
156 | "**Recap:**\n",
157 | "* you've loaded your data and had a look at it.\n",
158 | "\n",
159 | "**Up next:** Explore your data visually and build a first model!\n",
160 | "\n",
161 | "For more on `pandas`, check out our [Data Manipulation with Python track](https://www.datacamp.com/tracks/data-manipulation-with-python). \n",
162 | "\n",
163 | "If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp)."
164 | ]
165 | },
166 | {
167 | "cell_type": "markdown",
168 | "metadata": {},
169 | "source": [
170 | "## Visual exploratory data analysis and your first model"
171 | ]
172 | },
173 | {
174 | "cell_type": "markdown",
175 | "metadata": {},
176 | "source": [
177 | "* Use `seaborn` to build a bar plot of Titanic survival (your _target variable_)."
178 | ]
179 | },
180 | {
181 | "cell_type": "code",
182 | "execution_count": null,
183 | "metadata": {},
184 | "outputs": [],
185 | "source": [
186 | "____"
187 | ]
188 | },
189 | {
190 | "cell_type": "markdown",
191 | "metadata": {},
192 | "source": [
193 | "**Take-away:** In the training set, less people survived than didn't. Let's then build a first model that **predict that nobody survived**.\n",
194 | "\n",
195 | "This is a bad model as we know that people survived. But it gives us a _baseline_: any model that we build later needs to do better than this one."
196 | ]
197 | },
198 | {
199 | "cell_type": "markdown",
200 | "metadata": {},
201 | "source": [
202 | "* Create a column 'Survived' for `df_test` that encodes 'did not survive' for all rows;\n",
203 | "* Save 'PassengerId' and 'Survived' columns of `df_test` to a .csv and submit to Kaggle."
204 | ]
205 | },
206 | {
207 | "cell_type": "code",
208 | "execution_count": null,
209 | "metadata": {},
210 | "outputs": [],
211 | "source": [
212 | "df_test['Survived'] = 0\n",
213 | "df_test[['PassengerId', 'Survived']].____"
214 | ]
215 | },
216 | {
217 | "cell_type": "markdown",
218 | "metadata": {},
219 | "source": [
220 | "* What accuracy did this give you?\n",
221 | "\n",
222 | "Accuracy on Kaggle = ??\n",
223 | "\n",
224 | "**Essential note!** There are metrics other than accuracy that you may want to use."
225 | ]
226 | },
227 | {
228 | "cell_type": "markdown",
229 | "metadata": {},
230 | "source": [
231 | "**Recap:**\n",
232 | "* you've loaded your data and had a look at it.\n",
233 | "* you've explored your target variable visually and made your first predictions.\n",
234 | "\n",
235 | "**Up next:** More EDA and you'll build another model."
236 | ]
237 | },
238 | {
239 | "cell_type": "markdown",
240 | "metadata": {},
241 | "source": [
242 | "## EDA on feature variables"
243 | ]
244 | },
245 | {
246 | "cell_type": "markdown",
247 | "metadata": {},
248 | "source": [
249 | "* Use `seaborn` to build a bar plot of the Titanic dataset feature 'Sex' (of `df_train`)."
250 | ]
251 | },
252 | {
253 | "cell_type": "code",
254 | "execution_count": null,
255 | "metadata": {},
256 | "outputs": [],
257 | "source": [
258 | "____"
259 | ]
260 | },
261 | {
262 | "cell_type": "markdown",
263 | "metadata": {},
264 | "source": [
265 | "* Use `seaborn` to build bar plots of the Titanic dataset feature 'Survived' split (faceted) over the feature 'Sex'."
266 | ]
267 | },
268 | {
269 | "cell_type": "code",
270 | "execution_count": null,
271 | "metadata": {},
272 | "outputs": [],
273 | "source": [
274 | "____"
275 | ]
276 | },
277 | {
278 | "cell_type": "markdown",
279 | "metadata": {},
280 | "source": [
281 | "**Take-away:** Women were more likely to survive than men."
282 | ]
283 | },
284 | {
285 | "cell_type": "markdown",
286 | "metadata": {},
287 | "source": [
288 | "* Use `pandas` to figure out how many women and how many men survived."
289 | ]
290 | },
291 | {
292 | "cell_type": "code",
293 | "execution_count": null,
294 | "metadata": {},
295 | "outputs": [],
296 | "source": [
297 | "____"
298 | ]
299 | },
300 | {
301 | "cell_type": "markdown",
302 | "metadata": {},
303 | "source": [
304 | "* Use `pandas` to figure out the proportion of women that survived, along with the proportion of men:"
305 | ]
306 | },
307 | {
308 | "cell_type": "code",
309 | "execution_count": null,
310 | "metadata": {},
311 | "outputs": [],
312 | "source": [
313 | "print(df_train[df_train.Sex == 'female'].Survived.sum()/df_train[df_train.Sex == 'female'].Survived.count())\n",
314 | "print(df_train[df_train.Sex == 'male'].Survived.sum()/df_train[df_train.Sex == 'male'].Survived.count())"
315 | ]
316 | },
317 | {
318 | "cell_type": "markdown",
319 | "metadata": {},
320 | "source": [
321 | "74% of women survived, while 18% of men survived.\n",
322 | "\n",
323 | "Let's now build a second model and predict that all women survived and all men didn't. Once again, this is an unrealistic model, but it will provide a baseline against which to compare future models."
324 | ]
325 | },
326 | {
327 | "cell_type": "markdown",
328 | "metadata": {},
329 | "source": [
330 | "* Create a column 'Survived' for `df_test` that encodes the above prediction.\n",
331 | "* Save 'PassengerId' and 'Survived' columns of `df_test` to a .csv and submit to Kaggle."
332 | ]
333 | },
334 | {
335 | "cell_type": "code",
336 | "execution_count": null,
337 | "metadata": {},
338 | "outputs": [],
339 | "source": [
340 | "df_test['Survived'] = ____"
341 | ]
342 | },
343 | {
344 | "cell_type": "code",
345 | "execution_count": null,
346 | "metadata": {},
347 | "outputs": [],
348 | "source": [
349 | "df_test[['PassengerId', 'Survived']].to_csv('data/predictions/women_survive.csv', index=False)"
350 | ]
351 | },
352 | {
353 | "cell_type": "markdown",
354 | "metadata": {},
355 | "source": [
356 | "* What accuracy did this give you?\n",
357 | "\n",
358 | "Accuracy on Kaggle = ??"
359 | ]
360 | },
361 | {
362 | "cell_type": "markdown",
363 | "metadata": {},
364 | "source": [
365 | "**Recap:**\n",
366 | "* you've loaded your data and had a look at it.\n",
367 | "* you've explored your target variable visually and made your first predictions.\n",
368 | "* you've explored some of your feature variables visually and made more predictions that did better based on your EDA.\n",
369 | "\n",
370 | "**Up next:** EDA of other feature variables, categorical and numeric.\n",
371 | "\n",
372 | "For more on `pandas`, check out our [Data Manipulation with Python track](https://www.datacamp.com/tracks/data-manipulation-with-python). \n",
373 | "\n",
374 | "For more on `seaborn`, check out Chapter 3 of our [Intro. to Datavis with Python course](https://www.datacamp.com/courses/introduction-to-data-visualization-with-python).\n",
375 | "\n",
376 | "If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp)."
377 | ]
378 | },
379 | {
380 | "cell_type": "markdown",
381 | "metadata": {},
382 | "source": [
383 | "## Explore your data more!"
384 | ]
385 | },
386 | {
387 | "cell_type": "markdown",
388 | "metadata": {},
389 | "source": [
390 | "* Use `seaborn` to build bar plots of the Titanic dataset feature 'Survived' split (faceted) over the feature 'Pclass'."
391 | ]
392 | },
393 | {
394 | "cell_type": "code",
395 | "execution_count": null,
396 | "metadata": {},
397 | "outputs": [],
398 | "source": [
399 | "____"
400 | ]
401 | },
402 | {
403 | "cell_type": "markdown",
404 | "metadata": {},
405 | "source": [
406 | "**Take-away:** [Include take-away from figure here]"
407 | ]
408 | },
409 | {
410 | "cell_type": "markdown",
411 | "metadata": {},
412 | "source": [
413 | "* Use `seaborn` to build bar plots of the Titanic dataset feature 'Survived' split (faceted) over the feature 'Embarked'."
414 | ]
415 | },
416 | {
417 | "cell_type": "code",
418 | "execution_count": null,
419 | "metadata": {},
420 | "outputs": [],
421 | "source": [
422 | "____"
423 | ]
424 | },
425 | {
426 | "cell_type": "markdown",
427 | "metadata": {},
428 | "source": [
429 | "**Take-away:** [Include take-away from figure here]"
430 | ]
431 | },
432 | {
433 | "cell_type": "markdown",
434 | "metadata": {},
435 | "source": [
436 | "## EDA with numeric variables"
437 | ]
438 | },
439 | {
440 | "cell_type": "markdown",
441 | "metadata": {},
442 | "source": [
443 | "* Use `seaborn` to plot a histogram of the 'Fare' column of `df_train`."
444 | ]
445 | },
446 | {
447 | "cell_type": "code",
448 | "execution_count": null,
449 | "metadata": {},
450 | "outputs": [],
451 | "source": [
452 | "____"
453 | ]
454 | },
455 | {
456 | "cell_type": "markdown",
457 | "metadata": {},
458 | "source": [
459 | "**Take-away:** [Include take-away from figure here]"
460 | ]
461 | },
462 | {
463 | "cell_type": "markdown",
464 | "metadata": {},
465 | "source": [
466 | "* Use a `pandas` plotting method to plot the column 'Fare' for each value of 'Survived' on the same plot."
467 | ]
468 | },
469 | {
470 | "cell_type": "code",
471 | "execution_count": null,
472 | "metadata": {},
473 | "outputs": [],
474 | "source": [
475 | "____"
476 | ]
477 | },
478 | {
479 | "cell_type": "markdown",
480 | "metadata": {},
481 | "source": [
482 | "**Take-away:** [Include take-away from figure here]"
483 | ]
484 | },
485 | {
486 | "cell_type": "markdown",
487 | "metadata": {},
488 | "source": [
489 | "* Use `seaborn` to plot a histogram of the 'Age' column of `df_train`. _Hint_: you may need to drop null values before doing so."
490 | ]
491 | },
492 | {
493 | "cell_type": "code",
494 | "execution_count": null,
495 | "metadata": {},
496 | "outputs": [],
497 | "source": [
498 | "df_train_drop = ____\n",
499 | "____"
500 | ]
501 | },
502 | {
503 | "cell_type": "markdown",
504 | "metadata": {},
505 | "source": [
506 | "**Take-away:** [Include take-away from figure here]"
507 | ]
508 | },
509 | {
510 | "cell_type": "markdown",
511 | "metadata": {},
512 | "source": [
513 | "* Plot a strip plot & a swarm plot of 'Fare' with 'Survived' on the x-axis."
514 | ]
515 | },
516 | {
517 | "cell_type": "code",
518 | "execution_count": null,
519 | "metadata": {},
520 | "outputs": [],
521 | "source": [
522 | "____"
523 | ]
524 | },
525 | {
526 | "cell_type": "code",
527 | "execution_count": null,
528 | "metadata": {},
529 | "outputs": [],
530 | "source": [
531 | "____"
532 | ]
533 | },
534 | {
535 | "cell_type": "markdown",
536 | "metadata": {},
537 | "source": [
538 | "**Take-away:** [Include take-away from figure here]"
539 | ]
540 | },
541 | {
542 | "cell_type": "markdown",
543 | "metadata": {},
544 | "source": [
545 | "* Use the DataFrame method `.describe()` to check out summary statistics of 'Fare' as a function of survival."
546 | ]
547 | },
548 | {
549 | "cell_type": "code",
550 | "execution_count": null,
551 | "metadata": {},
552 | "outputs": [],
553 | "source": [
554 | "____"
555 | ]
556 | },
557 | {
558 | "cell_type": "markdown",
559 | "metadata": {},
560 | "source": [
561 | "* Use `seaborn` to plot a scatter plot of 'Age' against 'Fare', colored by 'Survived'."
562 | ]
563 | },
564 | {
565 | "cell_type": "code",
566 | "execution_count": null,
567 | "metadata": {},
568 | "outputs": [],
569 | "source": [
570 | "____"
571 | ]
572 | },
573 | {
574 | "cell_type": "markdown",
575 | "metadata": {},
576 | "source": [
577 | "**Take-away:** [Include take-away from figure here]"
578 | ]
579 | },
580 | {
581 | "cell_type": "markdown",
582 | "metadata": {},
583 | "source": [
584 | "* Use `seaborn` to create a pairplot of `df_train`, colored by 'Survived'."
585 | ]
586 | },
587 | {
588 | "cell_type": "code",
589 | "execution_count": null,
590 | "metadata": {},
591 | "outputs": [],
592 | "source": [
593 | "____"
594 | ]
595 | },
596 | {
597 | "cell_type": "markdown",
598 | "metadata": {},
599 | "source": [
600 | "**Take-away:** [Include take-away from figure here]"
601 | ]
602 | },
603 | {
604 | "cell_type": "markdown",
605 | "metadata": {},
606 | "source": [
607 | "**Recap:**\n",
608 | "* you've loaded your data and had a look at it.\n",
609 | "* you've explored your target variable visually and made your first predictions.\n",
610 | "* you've explored some of your feature variables visually and made more predictions that did better based on your EDA.\n",
611 | "* you've done some serious EDA of feature variables, categorical and numeric.\n",
612 | "\n",
613 | "**Up next:** Time to build some Machine Learning models, based on what you've learnt from your EDA here. Open the notebook `2-titanic_first_ML-model.ipynb`.\n",
614 | "\n",
615 | "For more on `pandas`, check out our [Data Manipulation with Python track](https://www.datacamp.com/tracks/data-manipulation-with-python). \n",
616 | "\n",
617 | "For more on `seaborn`, check out Chapter 3 of our [Intro. to Datavis with Python course](https://www.datacamp.com/courses/introduction-to-data-visualization-with-python).\n",
618 | "\n",
619 | "If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp)."
620 | ]
621 | }
622 | ],
623 | "metadata": {
624 | "kernelspec": {
625 | "display_name": "Python 3",
626 | "language": "python",
627 | "name": "python3"
628 | },
629 | "language_info": {
630 | "codemirror_mode": {
631 | "name": "ipython",
632 | "version": 3
633 | },
634 | "file_extension": ".py",
635 | "mimetype": "text/x-python",
636 | "name": "python",
637 | "nbconvert_exporter": "python",
638 | "pygments_lexer": "ipython3",
639 | "version": "3.6.3"
640 | }
641 | },
642 | "nbformat": 4,
643 | "nbformat_minor": 2
644 | }
645 |
--------------------------------------------------------------------------------
/data/test.csv:
--------------------------------------------------------------------------------
1 | PassengerId,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked
2 | 892,3,"Kelly, Mr. James",male,34.5,0,0,330911,7.8292,,Q
3 | 893,3,"Wilkes, Mrs. James (Ellen Needs)",female,47,1,0,363272,7,,S
4 | 894,2,"Myles, Mr. Thomas Francis",male,62,0,0,240276,9.6875,,Q
5 | 895,3,"Wirz, Mr. Albert",male,27,0,0,315154,8.6625,,S
6 | 896,3,"Hirvonen, Mrs. Alexander (Helga E Lindqvist)",female,22,1,1,3101298,12.2875,,S
7 | 897,3,"Svensson, Mr. Johan Cervin",male,14,0,0,7538,9.225,,S
8 | 898,3,"Connolly, Miss. Kate",female,30,0,0,330972,7.6292,,Q
9 | 899,2,"Caldwell, Mr. Albert Francis",male,26,1,1,248738,29,,S
10 | 900,3,"Abrahim, Mrs. Joseph (Sophie Halaut Easu)",female,18,0,0,2657,7.2292,,C
11 | 901,3,"Davies, Mr. John Samuel",male,21,2,0,A/4 48871,24.15,,S
12 | 902,3,"Ilieff, Mr. Ylio",male,,0,0,349220,7.8958,,S
13 | 903,1,"Jones, Mr. Charles Cresson",male,46,0,0,694,26,,S
14 | 904,1,"Snyder, Mrs. John Pillsbury (Nelle Stevenson)",female,23,1,0,21228,82.2667,B45,S
15 | 905,2,"Howard, Mr. Benjamin",male,63,1,0,24065,26,,S
16 | 906,1,"Chaffee, Mrs. Herbert Fuller (Carrie Constance Toogood)",female,47,1,0,W.E.P. 5734,61.175,E31,S
17 | 907,2,"del Carlo, Mrs. Sebastiano (Argenia Genovesi)",female,24,1,0,SC/PARIS 2167,27.7208,,C
18 | 908,2,"Keane, Mr. Daniel",male,35,0,0,233734,12.35,,Q
19 | 909,3,"Assaf, Mr. Gerios",male,21,0,0,2692,7.225,,C
20 | 910,3,"Ilmakangas, Miss. Ida Livija",female,27,1,0,STON/O2. 3101270,7.925,,S
21 | 911,3,"Assaf Khalil, Mrs. Mariana (Miriam"")""",female,45,0,0,2696,7.225,,C
22 | 912,1,"Rothschild, Mr. Martin",male,55,1,0,PC 17603,59.4,,C
23 | 913,3,"Olsen, Master. Artur Karl",male,9,0,1,C 17368,3.1708,,S
24 | 914,1,"Flegenheim, Mrs. Alfred (Antoinette)",female,,0,0,PC 17598,31.6833,,S
25 | 915,1,"Williams, Mr. Richard Norris II",male,21,0,1,PC 17597,61.3792,,C
26 | 916,1,"Ryerson, Mrs. Arthur Larned (Emily Maria Borie)",female,48,1,3,PC 17608,262.375,B57 B59 B63 B66,C
27 | 917,3,"Robins, Mr. Alexander A",male,50,1,0,A/5. 3337,14.5,,S
28 | 918,1,"Ostby, Miss. Helene Ragnhild",female,22,0,1,113509,61.9792,B36,C
29 | 919,3,"Daher, Mr. Shedid",male,22.5,0,0,2698,7.225,,C
30 | 920,1,"Brady, Mr. John Bertram",male,41,0,0,113054,30.5,A21,S
31 | 921,3,"Samaan, Mr. Elias",male,,2,0,2662,21.6792,,C
32 | 922,2,"Louch, Mr. Charles Alexander",male,50,1,0,SC/AH 3085,26,,S
33 | 923,2,"Jefferys, Mr. Clifford Thomas",male,24,2,0,C.A. 31029,31.5,,S
34 | 924,3,"Dean, Mrs. Bertram (Eva Georgetta Light)",female,33,1,2,C.A. 2315,20.575,,S
35 | 925,3,"Johnston, Mrs. Andrew G (Elizabeth Lily"" Watson)""",female,,1,2,W./C. 6607,23.45,,S
36 | 926,1,"Mock, Mr. Philipp Edmund",male,30,1,0,13236,57.75,C78,C
37 | 927,3,"Katavelas, Mr. Vassilios (Catavelas Vassilios"")""",male,18.5,0,0,2682,7.2292,,C
38 | 928,3,"Roth, Miss. Sarah A",female,,0,0,342712,8.05,,S
39 | 929,3,"Cacic, Miss. Manda",female,21,0,0,315087,8.6625,,S
40 | 930,3,"Sap, Mr. Julius",male,25,0,0,345768,9.5,,S
41 | 931,3,"Hee, Mr. Ling",male,,0,0,1601,56.4958,,S
42 | 932,3,"Karun, Mr. Franz",male,39,0,1,349256,13.4167,,C
43 | 933,1,"Franklin, Mr. Thomas Parham",male,,0,0,113778,26.55,D34,S
44 | 934,3,"Goldsmith, Mr. Nathan",male,41,0,0,SOTON/O.Q. 3101263,7.85,,S
45 | 935,2,"Corbett, Mrs. Walter H (Irene Colvin)",female,30,0,0,237249,13,,S
46 | 936,1,"Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)",female,45,1,0,11753,52.5542,D19,S
47 | 937,3,"Peltomaki, Mr. Nikolai Johannes",male,25,0,0,STON/O 2. 3101291,7.925,,S
48 | 938,1,"Chevre, Mr. Paul Romaine",male,45,0,0,PC 17594,29.7,A9,C
49 | 939,3,"Shaughnessy, Mr. Patrick",male,,0,0,370374,7.75,,Q
50 | 940,1,"Bucknell, Mrs. William Robert (Emma Eliza Ward)",female,60,0,0,11813,76.2917,D15,C
51 | 941,3,"Coutts, Mrs. William (Winnie Minnie"" Treanor)""",female,36,0,2,C.A. 37671,15.9,,S
52 | 942,1,"Smith, Mr. Lucien Philip",male,24,1,0,13695,60,C31,S
53 | 943,2,"Pulbaum, Mr. Franz",male,27,0,0,SC/PARIS 2168,15.0333,,C
54 | 944,2,"Hocking, Miss. Ellen Nellie""""",female,20,2,1,29105,23,,S
55 | 945,1,"Fortune, Miss. Ethel Flora",female,28,3,2,19950,263,C23 C25 C27,S
56 | 946,2,"Mangiavacchi, Mr. Serafino Emilio",male,,0,0,SC/A.3 2861,15.5792,,C
57 | 947,3,"Rice, Master. Albert",male,10,4,1,382652,29.125,,Q
58 | 948,3,"Cor, Mr. Bartol",male,35,0,0,349230,7.8958,,S
59 | 949,3,"Abelseth, Mr. Olaus Jorgensen",male,25,0,0,348122,7.65,F G63,S
60 | 950,3,"Davison, Mr. Thomas Henry",male,,1,0,386525,16.1,,S
61 | 951,1,"Chaudanson, Miss. Victorine",female,36,0,0,PC 17608,262.375,B61,C
62 | 952,3,"Dika, Mr. Mirko",male,17,0,0,349232,7.8958,,S
63 | 953,2,"McCrae, Mr. Arthur Gordon",male,32,0,0,237216,13.5,,S
64 | 954,3,"Bjorklund, Mr. Ernst Herbert",male,18,0,0,347090,7.75,,S
65 | 955,3,"Bradley, Miss. Bridget Delia",female,22,0,0,334914,7.725,,Q
66 | 956,1,"Ryerson, Master. John Borie",male,13,2,2,PC 17608,262.375,B57 B59 B63 B66,C
67 | 957,2,"Corey, Mrs. Percy C (Mary Phyllis Elizabeth Miller)",female,,0,0,F.C.C. 13534,21,,S
68 | 958,3,"Burns, Miss. Mary Delia",female,18,0,0,330963,7.8792,,Q
69 | 959,1,"Moore, Mr. Clarence Bloomfield",male,47,0,0,113796,42.4,,S
70 | 960,1,"Tucker, Mr. Gilbert Milligan Jr",male,31,0,0,2543,28.5375,C53,C
71 | 961,1,"Fortune, Mrs. Mark (Mary McDougald)",female,60,1,4,19950,263,C23 C25 C27,S
72 | 962,3,"Mulvihill, Miss. Bertha E",female,24,0,0,382653,7.75,,Q
73 | 963,3,"Minkoff, Mr. Lazar",male,21,0,0,349211,7.8958,,S
74 | 964,3,"Nieminen, Miss. Manta Josefina",female,29,0,0,3101297,7.925,,S
75 | 965,1,"Ovies y Rodriguez, Mr. Servando",male,28.5,0,0,PC 17562,27.7208,D43,C
76 | 966,1,"Geiger, Miss. Amalie",female,35,0,0,113503,211.5,C130,C
77 | 967,1,"Keeping, Mr. Edwin",male,32.5,0,0,113503,211.5,C132,C
78 | 968,3,"Miles, Mr. Frank",male,,0,0,359306,8.05,,S
79 | 969,1,"Cornell, Mrs. Robert Clifford (Malvina Helen Lamson)",female,55,2,0,11770,25.7,C101,S
80 | 970,2,"Aldworth, Mr. Charles Augustus",male,30,0,0,248744,13,,S
81 | 971,3,"Doyle, Miss. Elizabeth",female,24,0,0,368702,7.75,,Q
82 | 972,3,"Boulos, Master. Akar",male,6,1,1,2678,15.2458,,C
83 | 973,1,"Straus, Mr. Isidor",male,67,1,0,PC 17483,221.7792,C55 C57,S
84 | 974,1,"Case, Mr. Howard Brown",male,49,0,0,19924,26,,S
85 | 975,3,"Demetri, Mr. Marinko",male,,0,0,349238,7.8958,,S
86 | 976,2,"Lamb, Mr. John Joseph",male,,0,0,240261,10.7083,,Q
87 | 977,3,"Khalil, Mr. Betros",male,,1,0,2660,14.4542,,C
88 | 978,3,"Barry, Miss. Julia",female,27,0,0,330844,7.8792,,Q
89 | 979,3,"Badman, Miss. Emily Louisa",female,18,0,0,A/4 31416,8.05,,S
90 | 980,3,"O'Donoghue, Ms. Bridget",female,,0,0,364856,7.75,,Q
91 | 981,2,"Wells, Master. Ralph Lester",male,2,1,1,29103,23,,S
92 | 982,3,"Dyker, Mrs. Adolf Fredrik (Anna Elisabeth Judith Andersson)",female,22,1,0,347072,13.9,,S
93 | 983,3,"Pedersen, Mr. Olaf",male,,0,0,345498,7.775,,S
94 | 984,1,"Davidson, Mrs. Thornton (Orian Hays)",female,27,1,2,F.C. 12750,52,B71,S
95 | 985,3,"Guest, Mr. Robert",male,,0,0,376563,8.05,,S
96 | 986,1,"Birnbaum, Mr. Jakob",male,25,0,0,13905,26,,C
97 | 987,3,"Tenglin, Mr. Gunnar Isidor",male,25,0,0,350033,7.7958,,S
98 | 988,1,"Cavendish, Mrs. Tyrell William (Julia Florence Siegel)",female,76,1,0,19877,78.85,C46,S
99 | 989,3,"Makinen, Mr. Kalle Edvard",male,29,0,0,STON/O 2. 3101268,7.925,,S
100 | 990,3,"Braf, Miss. Elin Ester Maria",female,20,0,0,347471,7.8542,,S
101 | 991,3,"Nancarrow, Mr. William Henry",male,33,0,0,A./5. 3338,8.05,,S
102 | 992,1,"Stengel, Mrs. Charles Emil Henry (Annie May Morris)",female,43,1,0,11778,55.4417,C116,C
103 | 993,2,"Weisz, Mr. Leopold",male,27,1,0,228414,26,,S
104 | 994,3,"Foley, Mr. William",male,,0,0,365235,7.75,,Q
105 | 995,3,"Johansson Palmquist, Mr. Oskar Leander",male,26,0,0,347070,7.775,,S
106 | 996,3,"Thomas, Mrs. Alexander (Thamine Thelma"")""",female,16,1,1,2625,8.5167,,C
107 | 997,3,"Holthen, Mr. Johan Martin",male,28,0,0,C 4001,22.525,,S
108 | 998,3,"Buckley, Mr. Daniel",male,21,0,0,330920,7.8208,,Q
109 | 999,3,"Ryan, Mr. Edward",male,,0,0,383162,7.75,,Q
110 | 1000,3,"Willer, Mr. Aaron (Abi Weller"")""",male,,0,0,3410,8.7125,,S
111 | 1001,2,"Swane, Mr. George",male,18.5,0,0,248734,13,F,S
112 | 1002,2,"Stanton, Mr. Samuel Ward",male,41,0,0,237734,15.0458,,C
113 | 1003,3,"Shine, Miss. Ellen Natalia",female,,0,0,330968,7.7792,,Q
114 | 1004,1,"Evans, Miss. Edith Corse",female,36,0,0,PC 17531,31.6792,A29,C
115 | 1005,3,"Buckley, Miss. Katherine",female,18.5,0,0,329944,7.2833,,Q
116 | 1006,1,"Straus, Mrs. Isidor (Rosalie Ida Blun)",female,63,1,0,PC 17483,221.7792,C55 C57,S
117 | 1007,3,"Chronopoulos, Mr. Demetrios",male,18,1,0,2680,14.4542,,C
118 | 1008,3,"Thomas, Mr. John",male,,0,0,2681,6.4375,,C
119 | 1009,3,"Sandstrom, Miss. Beatrice Irene",female,1,1,1,PP 9549,16.7,G6,S
120 | 1010,1,"Beattie, Mr. Thomson",male,36,0,0,13050,75.2417,C6,C
121 | 1011,2,"Chapman, Mrs. John Henry (Sara Elizabeth Lawry)",female,29,1,0,SC/AH 29037,26,,S
122 | 1012,2,"Watt, Miss. Bertha J",female,12,0,0,C.A. 33595,15.75,,S
123 | 1013,3,"Kiernan, Mr. John",male,,1,0,367227,7.75,,Q
124 | 1014,1,"Schabert, Mrs. Paul (Emma Mock)",female,35,1,0,13236,57.75,C28,C
125 | 1015,3,"Carver, Mr. Alfred John",male,28,0,0,392095,7.25,,S
126 | 1016,3,"Kennedy, Mr. John",male,,0,0,368783,7.75,,Q
127 | 1017,3,"Cribb, Miss. Laura Alice",female,17,0,1,371362,16.1,,S
128 | 1018,3,"Brobeck, Mr. Karl Rudolf",male,22,0,0,350045,7.7958,,S
129 | 1019,3,"McCoy, Miss. Alicia",female,,2,0,367226,23.25,,Q
130 | 1020,2,"Bowenur, Mr. Solomon",male,42,0,0,211535,13,,S
131 | 1021,3,"Petersen, Mr. Marius",male,24,0,0,342441,8.05,,S
132 | 1022,3,"Spinner, Mr. Henry John",male,32,0,0,STON/OQ. 369943,8.05,,S
133 | 1023,1,"Gracie, Col. Archibald IV",male,53,0,0,113780,28.5,C51,C
134 | 1024,3,"Lefebre, Mrs. Frank (Frances)",female,,0,4,4133,25.4667,,S
135 | 1025,3,"Thomas, Mr. Charles P",male,,1,0,2621,6.4375,,C
136 | 1026,3,"Dintcheff, Mr. Valtcho",male,43,0,0,349226,7.8958,,S
137 | 1027,3,"Carlsson, Mr. Carl Robert",male,24,0,0,350409,7.8542,,S
138 | 1028,3,"Zakarian, Mr. Mapriededer",male,26.5,0,0,2656,7.225,,C
139 | 1029,2,"Schmidt, Mr. August",male,26,0,0,248659,13,,S
140 | 1030,3,"Drapkin, Miss. Jennie",female,23,0,0,SOTON/OQ 392083,8.05,,S
141 | 1031,3,"Goodwin, Mr. Charles Frederick",male,40,1,6,CA 2144,46.9,,S
142 | 1032,3,"Goodwin, Miss. Jessie Allis",female,10,5,2,CA 2144,46.9,,S
143 | 1033,1,"Daniels, Miss. Sarah",female,33,0,0,113781,151.55,,S
144 | 1034,1,"Ryerson, Mr. Arthur Larned",male,61,1,3,PC 17608,262.375,B57 B59 B63 B66,C
145 | 1035,2,"Beauchamp, Mr. Henry James",male,28,0,0,244358,26,,S
146 | 1036,1,"Lindeberg-Lind, Mr. Erik Gustaf (Mr Edward Lingrey"")""",male,42,0,0,17475,26.55,,S
147 | 1037,3,"Vander Planke, Mr. Julius",male,31,3,0,345763,18,,S
148 | 1038,1,"Hilliard, Mr. Herbert Henry",male,,0,0,17463,51.8625,E46,S
149 | 1039,3,"Davies, Mr. Evan",male,22,0,0,SC/A4 23568,8.05,,S
150 | 1040,1,"Crafton, Mr. John Bertram",male,,0,0,113791,26.55,,S
151 | 1041,2,"Lahtinen, Rev. William",male,30,1,1,250651,26,,S
152 | 1042,1,"Earnshaw, Mrs. Boulton (Olive Potter)",female,23,0,1,11767,83.1583,C54,C
153 | 1043,3,"Matinoff, Mr. Nicola",male,,0,0,349255,7.8958,,C
154 | 1044,3,"Storey, Mr. Thomas",male,60.5,0,0,3701,,,S
155 | 1045,3,"Klasen, Mrs. (Hulda Kristina Eugenia Lofqvist)",female,36,0,2,350405,12.1833,,S
156 | 1046,3,"Asplund, Master. Filip Oscar",male,13,4,2,347077,31.3875,,S
157 | 1047,3,"Duquemin, Mr. Joseph",male,24,0,0,S.O./P.P. 752,7.55,,S
158 | 1048,1,"Bird, Miss. Ellen",female,29,0,0,PC 17483,221.7792,C97,S
159 | 1049,3,"Lundin, Miss. Olga Elida",female,23,0,0,347469,7.8542,,S
160 | 1050,1,"Borebank, Mr. John James",male,42,0,0,110489,26.55,D22,S
161 | 1051,3,"Peacock, Mrs. Benjamin (Edith Nile)",female,26,0,2,SOTON/O.Q. 3101315,13.775,,S
162 | 1052,3,"Smyth, Miss. Julia",female,,0,0,335432,7.7333,,Q
163 | 1053,3,"Touma, Master. Georges Youssef",male,7,1,1,2650,15.2458,,C
164 | 1054,2,"Wright, Miss. Marion",female,26,0,0,220844,13.5,,S
165 | 1055,3,"Pearce, Mr. Ernest",male,,0,0,343271,7,,S
166 | 1056,2,"Peruschitz, Rev. Joseph Maria",male,41,0,0,237393,13,,S
167 | 1057,3,"Kink-Heilmann, Mrs. Anton (Luise Heilmann)",female,26,1,1,315153,22.025,,S
168 | 1058,1,"Brandeis, Mr. Emil",male,48,0,0,PC 17591,50.4958,B10,C
169 | 1059,3,"Ford, Mr. Edward Watson",male,18,2,2,W./C. 6608,34.375,,S
170 | 1060,1,"Cassebeer, Mrs. Henry Arthur Jr (Eleanor Genevieve Fosdick)",female,,0,0,17770,27.7208,,C
171 | 1061,3,"Hellstrom, Miss. Hilda Maria",female,22,0,0,7548,8.9625,,S
172 | 1062,3,"Lithman, Mr. Simon",male,,0,0,S.O./P.P. 251,7.55,,S
173 | 1063,3,"Zakarian, Mr. Ortin",male,27,0,0,2670,7.225,,C
174 | 1064,3,"Dyker, Mr. Adolf Fredrik",male,23,1,0,347072,13.9,,S
175 | 1065,3,"Torfa, Mr. Assad",male,,0,0,2673,7.2292,,C
176 | 1066,3,"Asplund, Mr. Carl Oscar Vilhelm Gustafsson",male,40,1,5,347077,31.3875,,S
177 | 1067,2,"Brown, Miss. Edith Eileen",female,15,0,2,29750,39,,S
178 | 1068,2,"Sincock, Miss. Maude",female,20,0,0,C.A. 33112,36.75,,S
179 | 1069,1,"Stengel, Mr. Charles Emil Henry",male,54,1,0,11778,55.4417,C116,C
180 | 1070,2,"Becker, Mrs. Allen Oliver (Nellie E Baumgardner)",female,36,0,3,230136,39,F4,S
181 | 1071,1,"Compton, Mrs. Alexander Taylor (Mary Eliza Ingersoll)",female,64,0,2,PC 17756,83.1583,E45,C
182 | 1072,2,"McCrie, Mr. James Matthew",male,30,0,0,233478,13,,S
183 | 1073,1,"Compton, Mr. Alexander Taylor Jr",male,37,1,1,PC 17756,83.1583,E52,C
184 | 1074,1,"Marvin, Mrs. Daniel Warner (Mary Graham Carmichael Farquarson)",female,18,1,0,113773,53.1,D30,S
185 | 1075,3,"Lane, Mr. Patrick",male,,0,0,7935,7.75,,Q
186 | 1076,1,"Douglas, Mrs. Frederick Charles (Mary Helene Baxter)",female,27,1,1,PC 17558,247.5208,B58 B60,C
187 | 1077,2,"Maybery, Mr. Frank Hubert",male,40,0,0,239059,16,,S
188 | 1078,2,"Phillips, Miss. Alice Frances Louisa",female,21,0,1,S.O./P.P. 2,21,,S
189 | 1079,3,"Davies, Mr. Joseph",male,17,2,0,A/4 48873,8.05,,S
190 | 1080,3,"Sage, Miss. Ada",female,,8,2,CA. 2343,69.55,,S
191 | 1081,2,"Veal, Mr. James",male,40,0,0,28221,13,,S
192 | 1082,2,"Angle, Mr. William A",male,34,1,0,226875,26,,S
193 | 1083,1,"Salomon, Mr. Abraham L",male,,0,0,111163,26,,S
194 | 1084,3,"van Billiard, Master. Walter John",male,11.5,1,1,A/5. 851,14.5,,S
195 | 1085,2,"Lingane, Mr. John",male,61,0,0,235509,12.35,,Q
196 | 1086,2,"Drew, Master. Marshall Brines",male,8,0,2,28220,32.5,,S
197 | 1087,3,"Karlsson, Mr. Julius Konrad Eugen",male,33,0,0,347465,7.8542,,S
198 | 1088,1,"Spedden, Master. Robert Douglas",male,6,0,2,16966,134.5,E34,C
199 | 1089,3,"Nilsson, Miss. Berta Olivia",female,18,0,0,347066,7.775,,S
200 | 1090,2,"Baimbrigge, Mr. Charles Robert",male,23,0,0,C.A. 31030,10.5,,S
201 | 1091,3,"Rasmussen, Mrs. (Lena Jacobsen Solvang)",female,,0,0,65305,8.1125,,S
202 | 1092,3,"Murphy, Miss. Nora",female,,0,0,36568,15.5,,Q
203 | 1093,3,"Danbom, Master. Gilbert Sigvard Emanuel",male,0.33,0,2,347080,14.4,,S
204 | 1094,1,"Astor, Col. John Jacob",male,47,1,0,PC 17757,227.525,C62 C64,C
205 | 1095,2,"Quick, Miss. Winifred Vera",female,8,1,1,26360,26,,S
206 | 1096,2,"Andrew, Mr. Frank Thomas",male,25,0,0,C.A. 34050,10.5,,S
207 | 1097,1,"Omont, Mr. Alfred Fernand",male,,0,0,F.C. 12998,25.7417,,C
208 | 1098,3,"McGowan, Miss. Katherine",female,35,0,0,9232,7.75,,Q
209 | 1099,2,"Collett, Mr. Sidney C Stuart",male,24,0,0,28034,10.5,,S
210 | 1100,1,"Rosenbaum, Miss. Edith Louise",female,33,0,0,PC 17613,27.7208,A11,C
211 | 1101,3,"Delalic, Mr. Redjo",male,25,0,0,349250,7.8958,,S
212 | 1102,3,"Andersen, Mr. Albert Karvin",male,32,0,0,C 4001,22.525,,S
213 | 1103,3,"Finoli, Mr. Luigi",male,,0,0,SOTON/O.Q. 3101308,7.05,,S
214 | 1104,2,"Deacon, Mr. Percy William",male,17,0,0,S.O.C. 14879,73.5,,S
215 | 1105,2,"Howard, Mrs. Benjamin (Ellen Truelove Arman)",female,60,1,0,24065,26,,S
216 | 1106,3,"Andersson, Miss. Ida Augusta Margareta",female,38,4,2,347091,7.775,,S
217 | 1107,1,"Head, Mr. Christopher",male,42,0,0,113038,42.5,B11,S
218 | 1108,3,"Mahon, Miss. Bridget Delia",female,,0,0,330924,7.8792,,Q
219 | 1109,1,"Wick, Mr. George Dennick",male,57,1,1,36928,164.8667,,S
220 | 1110,1,"Widener, Mrs. George Dunton (Eleanor Elkins)",female,50,1,1,113503,211.5,C80,C
221 | 1111,3,"Thomson, Mr. Alexander Morrison",male,,0,0,32302,8.05,,S
222 | 1112,2,"Duran y More, Miss. Florentina",female,30,1,0,SC/PARIS 2148,13.8583,,C
223 | 1113,3,"Reynolds, Mr. Harold J",male,21,0,0,342684,8.05,,S
224 | 1114,2,"Cook, Mrs. (Selena Rogers)",female,22,0,0,W./C. 14266,10.5,F33,S
225 | 1115,3,"Karlsson, Mr. Einar Gervasius",male,21,0,0,350053,7.7958,,S
226 | 1116,1,"Candee, Mrs. Edward (Helen Churchill Hungerford)",female,53,0,0,PC 17606,27.4458,,C
227 | 1117,3,"Moubarek, Mrs. George (Omine Amenia"" Alexander)""",female,,0,2,2661,15.2458,,C
228 | 1118,3,"Asplund, Mr. Johan Charles",male,23,0,0,350054,7.7958,,S
229 | 1119,3,"McNeill, Miss. Bridget",female,,0,0,370368,7.75,,Q
230 | 1120,3,"Everett, Mr. Thomas James",male,40.5,0,0,C.A. 6212,15.1,,S
231 | 1121,2,"Hocking, Mr. Samuel James Metcalfe",male,36,0,0,242963,13,,S
232 | 1122,2,"Sweet, Mr. George Frederick",male,14,0,0,220845,65,,S
233 | 1123,1,"Willard, Miss. Constance",female,21,0,0,113795,26.55,,S
234 | 1124,3,"Wiklund, Mr. Karl Johan",male,21,1,0,3101266,6.4958,,S
235 | 1125,3,"Linehan, Mr. Michael",male,,0,0,330971,7.8792,,Q
236 | 1126,1,"Cumings, Mr. John Bradley",male,39,1,0,PC 17599,71.2833,C85,C
237 | 1127,3,"Vendel, Mr. Olof Edvin",male,20,0,0,350416,7.8542,,S
238 | 1128,1,"Warren, Mr. Frank Manley",male,64,1,0,110813,75.25,D37,C
239 | 1129,3,"Baccos, Mr. Raffull",male,20,0,0,2679,7.225,,C
240 | 1130,2,"Hiltunen, Miss. Marta",female,18,1,1,250650,13,,S
241 | 1131,1,"Douglas, Mrs. Walter Donald (Mahala Dutton)",female,48,1,0,PC 17761,106.425,C86,C
242 | 1132,1,"Lindstrom, Mrs. Carl Johan (Sigrid Posse)",female,55,0,0,112377,27.7208,,C
243 | 1133,2,"Christy, Mrs. (Alice Frances)",female,45,0,2,237789,30,,S
244 | 1134,1,"Spedden, Mr. Frederic Oakley",male,45,1,1,16966,134.5,E34,C
245 | 1135,3,"Hyman, Mr. Abraham",male,,0,0,3470,7.8875,,S
246 | 1136,3,"Johnston, Master. William Arthur Willie""""",male,,1,2,W./C. 6607,23.45,,S
247 | 1137,1,"Kenyon, Mr. Frederick R",male,41,1,0,17464,51.8625,D21,S
248 | 1138,2,"Karnes, Mrs. J Frank (Claire Bennett)",female,22,0,0,F.C.C. 13534,21,,S
249 | 1139,2,"Drew, Mr. James Vivian",male,42,1,1,28220,32.5,,S
250 | 1140,2,"Hold, Mrs. Stephen (Annie Margaret Hill)",female,29,1,0,26707,26,,S
251 | 1141,3,"Khalil, Mrs. Betros (Zahie Maria"" Elias)""",female,,1,0,2660,14.4542,,C
252 | 1142,2,"West, Miss. Barbara J",female,0.92,1,2,C.A. 34651,27.75,,S
253 | 1143,3,"Abrahamsson, Mr. Abraham August Johannes",male,20,0,0,SOTON/O2 3101284,7.925,,S
254 | 1144,1,"Clark, Mr. Walter Miller",male,27,1,0,13508,136.7792,C89,C
255 | 1145,3,"Salander, Mr. Karl Johan",male,24,0,0,7266,9.325,,S
256 | 1146,3,"Wenzel, Mr. Linhart",male,32.5,0,0,345775,9.5,,S
257 | 1147,3,"MacKay, Mr. George William",male,,0,0,C.A. 42795,7.55,,S
258 | 1148,3,"Mahon, Mr. John",male,,0,0,AQ/4 3130,7.75,,Q
259 | 1149,3,"Niklasson, Mr. Samuel",male,28,0,0,363611,8.05,,S
260 | 1150,2,"Bentham, Miss. Lilian W",female,19,0,0,28404,13,,S
261 | 1151,3,"Midtsjo, Mr. Karl Albert",male,21,0,0,345501,7.775,,S
262 | 1152,3,"de Messemaeker, Mr. Guillaume Joseph",male,36.5,1,0,345572,17.4,,S
263 | 1153,3,"Nilsson, Mr. August Ferdinand",male,21,0,0,350410,7.8542,,S
264 | 1154,2,"Wells, Mrs. Arthur Henry (Addie"" Dart Trevaskis)""",female,29,0,2,29103,23,,S
265 | 1155,3,"Klasen, Miss. Gertrud Emilia",female,1,1,1,350405,12.1833,,S
266 | 1156,2,"Portaluppi, Mr. Emilio Ilario Giuseppe",male,30,0,0,C.A. 34644,12.7375,,C
267 | 1157,3,"Lyntakoff, Mr. Stanko",male,,0,0,349235,7.8958,,S
268 | 1158,1,"Chisholm, Mr. Roderick Robert Crispin",male,,0,0,112051,0,,S
269 | 1159,3,"Warren, Mr. Charles William",male,,0,0,C.A. 49867,7.55,,S
270 | 1160,3,"Howard, Miss. May Elizabeth",female,,0,0,A. 2. 39186,8.05,,S
271 | 1161,3,"Pokrnic, Mr. Mate",male,17,0,0,315095,8.6625,,S
272 | 1162,1,"McCaffry, Mr. Thomas Francis",male,46,0,0,13050,75.2417,C6,C
273 | 1163,3,"Fox, Mr. Patrick",male,,0,0,368573,7.75,,Q
274 | 1164,1,"Clark, Mrs. Walter Miller (Virginia McDowell)",female,26,1,0,13508,136.7792,C89,C
275 | 1165,3,"Lennon, Miss. Mary",female,,1,0,370371,15.5,,Q
276 | 1166,3,"Saade, Mr. Jean Nassr",male,,0,0,2676,7.225,,C
277 | 1167,2,"Bryhl, Miss. Dagmar Jenny Ingeborg ",female,20,1,0,236853,26,,S
278 | 1168,2,"Parker, Mr. Clifford Richard",male,28,0,0,SC 14888,10.5,,S
279 | 1169,2,"Faunthorpe, Mr. Harry",male,40,1,0,2926,26,,S
280 | 1170,2,"Ware, Mr. John James",male,30,1,0,CA 31352,21,,S
281 | 1171,2,"Oxenham, Mr. Percy Thomas",male,22,0,0,W./C. 14260,10.5,,S
282 | 1172,3,"Oreskovic, Miss. Jelka",female,23,0,0,315085,8.6625,,S
283 | 1173,3,"Peacock, Master. Alfred Edward",male,0.75,1,1,SOTON/O.Q. 3101315,13.775,,S
284 | 1174,3,"Fleming, Miss. Honora",female,,0,0,364859,7.75,,Q
285 | 1175,3,"Touma, Miss. Maria Youssef",female,9,1,1,2650,15.2458,,C
286 | 1176,3,"Rosblom, Miss. Salli Helena",female,2,1,1,370129,20.2125,,S
287 | 1177,3,"Dennis, Mr. William",male,36,0,0,A/5 21175,7.25,,S
288 | 1178,3,"Franklin, Mr. Charles (Charles Fardon)",male,,0,0,SOTON/O.Q. 3101314,7.25,,S
289 | 1179,1,"Snyder, Mr. John Pillsbury",male,24,1,0,21228,82.2667,B45,S
290 | 1180,3,"Mardirosian, Mr. Sarkis",male,,0,0,2655,7.2292,F E46,C
291 | 1181,3,"Ford, Mr. Arthur",male,,0,0,A/5 1478,8.05,,S
292 | 1182,1,"Rheims, Mr. George Alexander Lucien",male,,0,0,PC 17607,39.6,,S
293 | 1183,3,"Daly, Miss. Margaret Marcella Maggie""""",female,30,0,0,382650,6.95,,Q
294 | 1184,3,"Nasr, Mr. Mustafa",male,,0,0,2652,7.2292,,C
295 | 1185,1,"Dodge, Dr. Washington",male,53,1,1,33638,81.8583,A34,S
296 | 1186,3,"Wittevrongel, Mr. Camille",male,36,0,0,345771,9.5,,S
297 | 1187,3,"Angheloff, Mr. Minko",male,26,0,0,349202,7.8958,,S
298 | 1188,2,"Laroche, Miss. Louise",female,1,1,2,SC/Paris 2123,41.5792,,C
299 | 1189,3,"Samaan, Mr. Hanna",male,,2,0,2662,21.6792,,C
300 | 1190,1,"Loring, Mr. Joseph Holland",male,30,0,0,113801,45.5,,S
301 | 1191,3,"Johansson, Mr. Nils",male,29,0,0,347467,7.8542,,S
302 | 1192,3,"Olsson, Mr. Oscar Wilhelm",male,32,0,0,347079,7.775,,S
303 | 1193,2,"Malachard, Mr. Noel",male,,0,0,237735,15.0458,D,C
304 | 1194,2,"Phillips, Mr. Escott Robert",male,43,0,1,S.O./P.P. 2,21,,S
305 | 1195,3,"Pokrnic, Mr. Tome",male,24,0,0,315092,8.6625,,S
306 | 1196,3,"McCarthy, Miss. Catherine Katie""""",female,,0,0,383123,7.75,,Q
307 | 1197,1,"Crosby, Mrs. Edward Gifford (Catherine Elizabeth Halstead)",female,64,1,1,112901,26.55,B26,S
308 | 1198,1,"Allison, Mr. Hudson Joshua Creighton",male,30,1,2,113781,151.55,C22 C26,S
309 | 1199,3,"Aks, Master. Philip Frank",male,0.83,0,1,392091,9.35,,S
310 | 1200,1,"Hays, Mr. Charles Melville",male,55,1,1,12749,93.5,B69,S
311 | 1201,3,"Hansen, Mrs. Claus Peter (Jennie L Howard)",female,45,1,0,350026,14.1083,,S
312 | 1202,3,"Cacic, Mr. Jego Grga",male,18,0,0,315091,8.6625,,S
313 | 1203,3,"Vartanian, Mr. David",male,22,0,0,2658,7.225,,C
314 | 1204,3,"Sadowitz, Mr. Harry",male,,0,0,LP 1588,7.575,,S
315 | 1205,3,"Carr, Miss. Jeannie",female,37,0,0,368364,7.75,,Q
316 | 1206,1,"White, Mrs. John Stuart (Ella Holmes)",female,55,0,0,PC 17760,135.6333,C32,C
317 | 1207,3,"Hagardon, Miss. Kate",female,17,0,0,AQ/3. 30631,7.7333,,Q
318 | 1208,1,"Spencer, Mr. William Augustus",male,57,1,0,PC 17569,146.5208,B78,C
319 | 1209,2,"Rogers, Mr. Reginald Harry",male,19,0,0,28004,10.5,,S
320 | 1210,3,"Jonsson, Mr. Nils Hilding",male,27,0,0,350408,7.8542,,S
321 | 1211,2,"Jefferys, Mr. Ernest Wilfred",male,22,2,0,C.A. 31029,31.5,,S
322 | 1212,3,"Andersson, Mr. Johan Samuel",male,26,0,0,347075,7.775,,S
323 | 1213,3,"Krekorian, Mr. Neshan",male,25,0,0,2654,7.2292,F E57,C
324 | 1214,2,"Nesson, Mr. Israel",male,26,0,0,244368,13,F2,S
325 | 1215,1,"Rowe, Mr. Alfred G",male,33,0,0,113790,26.55,,S
326 | 1216,1,"Kreuchen, Miss. Emilie",female,39,0,0,24160,211.3375,,S
327 | 1217,3,"Assam, Mr. Ali",male,23,0,0,SOTON/O.Q. 3101309,7.05,,S
328 | 1218,2,"Becker, Miss. Ruth Elizabeth",female,12,2,1,230136,39,F4,S
329 | 1219,1,"Rosenshine, Mr. George (Mr George Thorne"")""",male,46,0,0,PC 17585,79.2,,C
330 | 1220,2,"Clarke, Mr. Charles Valentine",male,29,1,0,2003,26,,S
331 | 1221,2,"Enander, Mr. Ingvar",male,21,0,0,236854,13,,S
332 | 1222,2,"Davies, Mrs. John Morgan (Elizabeth Agnes Mary White) ",female,48,0,2,C.A. 33112,36.75,,S
333 | 1223,1,"Dulles, Mr. William Crothers",male,39,0,0,PC 17580,29.7,A18,C
334 | 1224,3,"Thomas, Mr. Tannous",male,,0,0,2684,7.225,,C
335 | 1225,3,"Nakid, Mrs. Said (Waika Mary"" Mowad)""",female,19,1,1,2653,15.7417,,C
336 | 1226,3,"Cor, Mr. Ivan",male,27,0,0,349229,7.8958,,S
337 | 1227,1,"Maguire, Mr. John Edward",male,30,0,0,110469,26,C106,S
338 | 1228,2,"de Brito, Mr. Jose Joaquim",male,32,0,0,244360,13,,S
339 | 1229,3,"Elias, Mr. Joseph",male,39,0,2,2675,7.2292,,C
340 | 1230,2,"Denbury, Mr. Herbert",male,25,0,0,C.A. 31029,31.5,,S
341 | 1231,3,"Betros, Master. Seman",male,,0,0,2622,7.2292,,C
342 | 1232,2,"Fillbrook, Mr. Joseph Charles",male,18,0,0,C.A. 15185,10.5,,S
343 | 1233,3,"Lundstrom, Mr. Thure Edvin",male,32,0,0,350403,7.5792,,S
344 | 1234,3,"Sage, Mr. John George",male,,1,9,CA. 2343,69.55,,S
345 | 1235,1,"Cardeza, Mrs. James Warburton Martinez (Charlotte Wardle Drake)",female,58,0,1,PC 17755,512.3292,B51 B53 B55,C
346 | 1236,3,"van Billiard, Master. James William",male,,1,1,A/5. 851,14.5,,S
347 | 1237,3,"Abelseth, Miss. Karen Marie",female,16,0,0,348125,7.65,,S
348 | 1238,2,"Botsford, Mr. William Hull",male,26,0,0,237670,13,,S
349 | 1239,3,"Whabee, Mrs. George Joseph (Shawneene Abi-Saab)",female,38,0,0,2688,7.2292,,C
350 | 1240,2,"Giles, Mr. Ralph",male,24,0,0,248726,13.5,,S
351 | 1241,2,"Walcroft, Miss. Nellie",female,31,0,0,F.C.C. 13528,21,,S
352 | 1242,1,"Greenfield, Mrs. Leo David (Blanche Strouse)",female,45,0,1,PC 17759,63.3583,D10 D12,C
353 | 1243,2,"Stokes, Mr. Philip Joseph",male,25,0,0,F.C.C. 13540,10.5,,S
354 | 1244,2,"Dibden, Mr. William",male,18,0,0,S.O.C. 14879,73.5,,S
355 | 1245,2,"Herman, Mr. Samuel",male,49,1,2,220845,65,,S
356 | 1246,3,"Dean, Miss. Elizabeth Gladys Millvina""""",female,0.17,1,2,C.A. 2315,20.575,,S
357 | 1247,1,"Julian, Mr. Henry Forbes",male,50,0,0,113044,26,E60,S
358 | 1248,1,"Brown, Mrs. John Murray (Caroline Lane Lamson)",female,59,2,0,11769,51.4792,C101,S
359 | 1249,3,"Lockyer, Mr. Edward",male,,0,0,1222,7.8792,,S
360 | 1250,3,"O'Keefe, Mr. Patrick",male,,0,0,368402,7.75,,Q
361 | 1251,3,"Lindell, Mrs. Edvard Bengtsson (Elin Gerda Persson)",female,30,1,0,349910,15.55,,S
362 | 1252,3,"Sage, Master. William Henry",male,14.5,8,2,CA. 2343,69.55,,S
363 | 1253,2,"Mallet, Mrs. Albert (Antoinette Magnin)",female,24,1,1,S.C./PARIS 2079,37.0042,,C
364 | 1254,2,"Ware, Mrs. John James (Florence Louise Long)",female,31,0,0,CA 31352,21,,S
365 | 1255,3,"Strilic, Mr. Ivan",male,27,0,0,315083,8.6625,,S
366 | 1256,1,"Harder, Mrs. George Achilles (Dorothy Annan)",female,25,1,0,11765,55.4417,E50,C
367 | 1257,3,"Sage, Mrs. John (Annie Bullen)",female,,1,9,CA. 2343,69.55,,S
368 | 1258,3,"Caram, Mr. Joseph",male,,1,0,2689,14.4583,,C
369 | 1259,3,"Riihivouri, Miss. Susanna Juhantytar Sanni""""",female,22,0,0,3101295,39.6875,,S
370 | 1260,1,"Gibson, Mrs. Leonard (Pauline C Boeson)",female,45,0,1,112378,59.4,,C
371 | 1261,2,"Pallas y Castello, Mr. Emilio",male,29,0,0,SC/PARIS 2147,13.8583,,C
372 | 1262,2,"Giles, Mr. Edgar",male,21,1,0,28133,11.5,,S
373 | 1263,1,"Wilson, Miss. Helen Alice",female,31,0,0,16966,134.5,E39 E41,C
374 | 1264,1,"Ismay, Mr. Joseph Bruce",male,49,0,0,112058,0,B52 B54 B56,S
375 | 1265,2,"Harbeck, Mr. William H",male,44,0,0,248746,13,,S
376 | 1266,1,"Dodge, Mrs. Washington (Ruth Vidaver)",female,54,1,1,33638,81.8583,A34,S
377 | 1267,1,"Bowen, Miss. Grace Scott",female,45,0,0,PC 17608,262.375,,C
378 | 1268,3,"Kink, Miss. Maria",female,22,2,0,315152,8.6625,,S
379 | 1269,2,"Cotterill, Mr. Henry Harry""""",male,21,0,0,29107,11.5,,S
380 | 1270,1,"Hipkins, Mr. William Edward",male,55,0,0,680,50,C39,S
381 | 1271,3,"Asplund, Master. Carl Edgar",male,5,4,2,347077,31.3875,,S
382 | 1272,3,"O'Connor, Mr. Patrick",male,,0,0,366713,7.75,,Q
383 | 1273,3,"Foley, Mr. Joseph",male,26,0,0,330910,7.8792,,Q
384 | 1274,3,"Risien, Mrs. Samuel (Emma)",female,,0,0,364498,14.5,,S
385 | 1275,3,"McNamee, Mrs. Neal (Eileen O'Leary)",female,19,1,0,376566,16.1,,S
386 | 1276,2,"Wheeler, Mr. Edwin Frederick""""",male,,0,0,SC/PARIS 2159,12.875,,S
387 | 1277,2,"Herman, Miss. Kate",female,24,1,2,220845,65,,S
388 | 1278,3,"Aronsson, Mr. Ernst Axel Algot",male,24,0,0,349911,7.775,,S
389 | 1279,2,"Ashby, Mr. John",male,57,0,0,244346,13,,S
390 | 1280,3,"Canavan, Mr. Patrick",male,21,0,0,364858,7.75,,Q
391 | 1281,3,"Palsson, Master. Paul Folke",male,6,3,1,349909,21.075,,S
392 | 1282,1,"Payne, Mr. Vivian Ponsonby",male,23,0,0,12749,93.5,B24,S
393 | 1283,1,"Lines, Mrs. Ernest H (Elizabeth Lindsey James)",female,51,0,1,PC 17592,39.4,D28,S
394 | 1284,3,"Abbott, Master. Eugene Joseph",male,13,0,2,C.A. 2673,20.25,,S
395 | 1285,2,"Gilbert, Mr. William",male,47,0,0,C.A. 30769,10.5,,S
396 | 1286,3,"Kink-Heilmann, Mr. Anton",male,29,3,1,315153,22.025,,S
397 | 1287,1,"Smith, Mrs. Lucien Philip (Mary Eloise Hughes)",female,18,1,0,13695,60,C31,S
398 | 1288,3,"Colbert, Mr. Patrick",male,24,0,0,371109,7.25,,Q
399 | 1289,1,"Frolicher-Stehli, Mrs. Maxmillian (Margaretha Emerentia Stehli)",female,48,1,1,13567,79.2,B41,C
400 | 1290,3,"Larsson-Rondberg, Mr. Edvard A",male,22,0,0,347065,7.775,,S
401 | 1291,3,"Conlon, Mr. Thomas Henry",male,31,0,0,21332,7.7333,,Q
402 | 1292,1,"Bonnell, Miss. Caroline",female,30,0,0,36928,164.8667,C7,S
403 | 1293,2,"Gale, Mr. Harry",male,38,1,0,28664,21,,S
404 | 1294,1,"Gibson, Miss. Dorothy Winifred",female,22,0,1,112378,59.4,,C
405 | 1295,1,"Carrau, Mr. Jose Pedro",male,17,0,0,113059,47.1,,S
406 | 1296,1,"Frauenthal, Mr. Isaac Gerald",male,43,1,0,17765,27.7208,D40,C
407 | 1297,2,"Nourney, Mr. Alfred (Baron von Drachstedt"")""",male,20,0,0,SC/PARIS 2166,13.8625,D38,C
408 | 1298,2,"Ware, Mr. William Jeffery",male,23,1,0,28666,10.5,,S
409 | 1299,1,"Widener, Mr. George Dunton",male,50,1,1,113503,211.5,C80,C
410 | 1300,3,"Riordan, Miss. Johanna Hannah""""",female,,0,0,334915,7.7208,,Q
411 | 1301,3,"Peacock, Miss. Treasteall",female,3,1,1,SOTON/O.Q. 3101315,13.775,,S
412 | 1302,3,"Naughton, Miss. Hannah",female,,0,0,365237,7.75,,Q
413 | 1303,1,"Minahan, Mrs. William Edward (Lillian E Thorpe)",female,37,1,0,19928,90,C78,Q
414 | 1304,3,"Henriksson, Miss. Jenny Lovisa",female,28,0,0,347086,7.775,,S
415 | 1305,3,"Spector, Mr. Woolf",male,,0,0,A.5. 3236,8.05,,S
416 | 1306,1,"Oliva y Ocana, Dona. Fermina",female,39,0,0,PC 17758,108.9,C105,C
417 | 1307,3,"Saether, Mr. Simon Sivertsen",male,38.5,0,0,SOTON/O.Q. 3101262,7.25,,S
418 | 1308,3,"Ware, Mr. Frederick",male,,0,0,359309,8.05,,S
419 | 1309,3,"Peter, Master. Michael J",male,,1,1,2668,22.3583,,C
420 |
--------------------------------------------------------------------------------
/solutions/2-titanic_first_ML-model.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# Build your first machine learning model"
8 | ]
9 | },
10 | {
11 | "cell_type": "code",
12 | "execution_count": 1,
13 | "metadata": {},
14 | "outputs": [],
15 | "source": [
16 | "# Import modules\n",
17 | "import pandas as pd\n",
18 | "import matplotlib.pyplot as plt\n",
19 | "import seaborn as sns\n",
20 | "import re\n",
21 | "import numpy as np\n",
22 | "from sklearn import tree\n",
23 | "from sklearn.model_selection import train_test_split\n",
24 | "from sklearn.linear_model import LogisticRegression\n",
25 | "from sklearn.model_selection import GridSearchCV\n",
26 | "\n",
27 | "# Figures inline and set visualization style\n",
28 | "%matplotlib inline\n",
29 | "sns.set()\n",
30 | "\n",
31 | "# Import data\n",
32 | "df_train = pd.read_csv('data/train.csv')\n",
33 | "df_test = pd.read_csv('data/test.csv')"
34 | ]
35 | },
36 | {
37 | "cell_type": "markdown",
38 | "metadata": {},
39 | "source": [
40 | "* Below, you will drop the target 'Survived' from the training dataset and create a new DataFrame `data` that consists of training and test sets combined;\n",
41 | "* But first, you'll store the target variable of the training data for safe keeping."
42 | ]
43 | },
44 | {
45 | "cell_type": "code",
46 | "execution_count": 2,
47 | "metadata": {},
48 | "outputs": [],
49 | "source": [
50 | "# Store target variable of training data in a safe place\n",
51 | "survived_train = df_train.Survived\n",
52 | "\n",
53 | "# Concatenate training and test sets\n",
54 | "data = pd.concat([df_train.drop(['Survived'], axis=1), df_test])"
55 | ]
56 | },
57 | {
58 | "cell_type": "markdown",
59 | "metadata": {},
60 | "source": [
61 | "* Check out your new DataFrame `data` using the `info()` method."
62 | ]
63 | },
64 | {
65 | "cell_type": "code",
66 | "execution_count": 3,
67 | "metadata": {},
68 | "outputs": [
69 | {
70 | "name": "stdout",
71 | "output_type": "stream",
72 | "text": [
73 | "| \n", 176 | " | PassengerId | \n", 177 | "Pclass | \n", 178 | "Name | \n", 179 | "Age | \n", 180 | "SibSp | \n", 181 | "Parch | \n", 182 | "Ticket | \n", 183 | "Fare | \n", 184 | "Cabin | \n", 185 | "Embarked | \n", 186 | "Sex_male | \n", 187 | "
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", 192 | "1 | \n", 193 | "3 | \n", 194 | "Braund, Mr. Owen Harris | \n", 195 | "22.0 | \n", 196 | "1 | \n", 197 | "0 | \n", 198 | "A/5 21171 | \n", 199 | "7.2500 | \n", 200 | "NaN | \n", 201 | "S | \n", 202 | "1 | \n", 203 | "
| 1 | \n", 206 | "2 | \n", 207 | "1 | \n", 208 | "Cumings, Mrs. John Bradley (Florence Briggs Th... | \n", 209 | "38.0 | \n", 210 | "1 | \n", 211 | "0 | \n", 212 | "PC 17599 | \n", 213 | "71.2833 | \n", 214 | "C85 | \n", 215 | "C | \n", 216 | "0 | \n", 217 | "
| 2 | \n", 220 | "3 | \n", 221 | "3 | \n", 222 | "Heikkinen, Miss. Laina | \n", 223 | "26.0 | \n", 224 | "0 | \n", 225 | "0 | \n", 226 | "STON/O2. 3101282 | \n", 227 | "7.9250 | \n", 228 | "NaN | \n", 229 | "S | \n", 230 | "0 | \n", 231 | "
| 3 | \n", 234 | "4 | \n", 235 | "1 | \n", 236 | "Futrelle, Mrs. Jacques Heath (Lily May Peel) | \n", 237 | "35.0 | \n", 238 | "1 | \n", 239 | "0 | \n", 240 | "113803 | \n", 241 | "53.1000 | \n", 242 | "C123 | \n", 243 | "S | \n", 244 | "0 | \n", 245 | "
| 4 | \n", 248 | "5 | \n", 249 | "3 | \n", 250 | "Allen, Mr. William Henry | \n", 251 | "35.0 | \n", 252 | "0 | \n", 253 | "0 | \n", 254 | "373450 | \n", 255 | "8.0500 | \n", 256 | "NaN | \n", 257 | "S | \n", 258 | "1 | \n", 259 | "
| \n", 323 | " | Sex_male | \n", 324 | "Fare | \n", 325 | "Age | \n", 326 | "Pclass | \n", 327 | "SibSp | \n", 328 | "
|---|---|---|---|---|---|
| 0 | \n", 333 | "1 | \n", 334 | "7.2500 | \n", 335 | "22.0 | \n", 336 | "3 | \n", 337 | "1 | \n", 338 | "
| 1 | \n", 341 | "0 | \n", 342 | "71.2833 | \n", 343 | "38.0 | \n", 344 | "1 | \n", 345 | "1 | \n", 346 | "
| 2 | \n", 349 | "0 | \n", 350 | "7.9250 | \n", 351 | "26.0 | \n", 352 | "3 | \n", 353 | "0 | \n", 354 | "
| 3 | \n", 357 | "0 | \n", 358 | "53.1000 | \n", 359 | "35.0 | \n", 360 | "1 | \n", 361 | "1 | \n", 362 | "
| 4 | \n", 365 | "1 | \n", 366 | "8.0500 | \n", 367 | "35.0 | \n", 368 | "3 | \n", 369 | "0 | \n", 370 | "
"
461 | ]
462 | },
463 | {
464 | "cell_type": "markdown",
465 | "metadata": {},
466 | "source": [
467 | "* You first **fit** such a model to your training data, which means deciding (based on the training data) which decisions will split at each branching point in the tree: e.g., that the first branch is on 'Male' or not and that 'Male' results in a prediction of 'Dead'. "
468 | ]
469 | },
470 | {
471 | "cell_type": "markdown",
472 | "metadata": {},
473 | "source": [
474 | "* Before fitting a model to your `data`, split it back into training and test sets:"
475 | ]
476 | },
477 | {
478 | "cell_type": "code",
479 | "execution_count": 8,
480 | "metadata": {},
481 | "outputs": [],
482 | "source": [
483 | "data_train = data.iloc[:891]\n",
484 | "data_test = data.iloc[891:]"
485 | ]
486 | },
487 | {
488 | "cell_type": "markdown",
489 | "metadata": {},
490 | "source": [
491 | "* You'll use `scikit-learn`, which requires your data as arrays, not DataFrames so transform them:"
492 | ]
493 | },
494 | {
495 | "cell_type": "code",
496 | "execution_count": 9,
497 | "metadata": {},
498 | "outputs": [],
499 | "source": [
500 | "X = data_train.values\n",
501 | "test = data_test.values\n",
502 | "y = survived_train.values"
503 | ]
504 | },
505 | {
506 | "cell_type": "markdown",
507 | "metadata": {},
508 | "source": [
509 | "* Now you get to build your decision tree classifier! First create such a model with `max_depth=3` and then fit it your data:"
510 | ]
511 | },
512 | {
513 | "cell_type": "code",
514 | "execution_count": 10,
515 | "metadata": {},
516 | "outputs": [
517 | {
518 | "data": {
519 | "text/plain": [
520 | "DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=3,\n",
521 | " max_features=None, max_leaf_nodes=None,\n",
522 | " min_impurity_decrease=0.0, min_impurity_split=None,\n",
523 | " min_samples_leaf=1, min_samples_split=2,\n",
524 | " min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
525 | " splitter='best')"
526 | ]
527 | },
528 | "execution_count": 10,
529 | "metadata": {},
530 | "output_type": "execute_result"
531 | }
532 | ],
533 | "source": [
534 | "# Instantiate model and fit to data\n",
535 | "clf = tree.DecisionTreeClassifier(max_depth=3)\n",
536 | "clf.fit(X, y)"
537 | ]
538 | },
539 | {
540 | "cell_type": "markdown",
541 | "metadata": {},
542 | "source": [
543 | "* Make predictions on your test set, create a new column 'Survived' and store your predictions in it. Save 'PassengerId' and 'Survived' columns of `df_test` to a .csv and submit to Kaggle."
544 | ]
545 | },
546 | {
547 | "cell_type": "code",
548 | "execution_count": 11,
549 | "metadata": {},
550 | "outputs": [],
551 | "source": [
552 | "# Make predictions and store in 'Survived' column of df_test\n",
553 | "Y_pred = clf.predict(test)\n",
554 | "df_test['Survived'] = Y_pred"
555 | ]
556 | },
557 | {
558 | "cell_type": "code",
559 | "execution_count": 12,
560 | "metadata": {},
561 | "outputs": [],
562 | "source": [
563 | "df_test[['PassengerId', 'Survived']].to_csv('data/predictions/1st_dec_tree.csv', index=False)"
564 | ]
565 | },
566 | {
567 | "cell_type": "markdown",
568 | "metadata": {},
569 | "source": [
570 | "* What is the accuracy of your model, as reported by Kaggle?\n",
571 | "\n",
572 | "Accuracy = 78%."
573 | ]
574 | },
575 | {
576 | "cell_type": "markdown",
577 | "metadata": {},
578 | "source": [
579 | "**Recap:**\n",
580 | "* You've got your data in a form to build first machine learning model.\n",
581 | "* You've built your first machine learning model: a decision tree classifier.\n",
582 | "\n",
583 | "**Up next:** figure out what this `max_depth` argument was, why we chose it and explore `train_test_split`.\n",
584 | "\n",
585 | "For more on `scikit-learn`, check out our [Supervised Learning with scikit-learn course](https://www.datacamp.com/courses/supervised-learning-with-scikit-learn). \n",
586 | "\n",
587 | "If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp)."
588 | ]
589 | },
590 | {
591 | "cell_type": "markdown",
592 | "metadata": {},
593 | "source": [
594 | "## What was this decision tree classifier?"
595 | ]
596 | },
597 | {
598 | "cell_type": "markdown",
599 | "metadata": {},
600 | "source": [
601 | ""
602 | ]
603 | },
604 | {
605 | "cell_type": "markdown",
606 | "metadata": {},
607 | "source": [
608 | "Note: you can use `graphviz` to generate figures such as this. See the `scikit-learn` documentation [here](http://scikit-learn.org/stable/modules/tree.html) for further details. In building this model, what you're essentially doing is creating a _decision boundary_ in the space of feature variables, for example (image from [here](http://scikit-learn.org/stable/auto_examples/ensemble/plot_voting_decision_regions.html)):"
609 | ]
610 | },
611 | {
612 | "cell_type": "markdown",
613 | "metadata": {},
614 | "source": [
615 | ""
616 | ]
617 | },
618 | {
619 | "cell_type": "markdown",
620 | "metadata": {},
621 | "source": [
622 | "## Why would you choose max_depth=3 ?"
623 | ]
624 | },
625 | {
626 | "cell_type": "markdown",
627 | "metadata": {},
628 | "source": [
629 | "The depth of the tree is known as a hyperparameter, which means a parameter we need to decide before we fit the model to the data. If we choose a larger `max_depth`, we'll get a more complex decision boundary. \n",
630 | "\n",
631 | "* If our decision boundary is _too complex_ we can overfit to the data, which means that our model will be describing noise as well as signal.\n",
632 | "\n",
633 | "* If our max_depth is too small, we may be underfitting the data, meaning that our model doesn't contain enough of the signal.\n",
634 | "\n",
635 | "**How do we tell whether we're overfitting or underfitting?** Note: this is also referred to as the bias-variance trade-off and we won't go into details on that here."
636 | ]
637 | },
638 | {
639 | "cell_type": "markdown",
640 | "metadata": {},
641 | "source": [
642 | "One way is to hold out a test set from our training data. We can then fit the model to our training data, make predictions on our test set and see how well our prediction does on the test set. \n",
643 | "\n",
644 | "* You'll now do this: split your original training data into training and test sets:"
645 | ]
646 | },
647 | {
648 | "cell_type": "code",
649 | "execution_count": 13,
650 | "metadata": {},
651 | "outputs": [],
652 | "source": [
653 | "X_train, X_test, y_train, y_test = train_test_split(\n",
654 | " X, y, test_size=0.33, random_state=42, stratify=y)"
655 | ]
656 | },
657 | {
658 | "cell_type": "markdown",
659 | "metadata": {},
660 | "source": [
661 | "* Iterate over values of `max_depth` ranging from 1 to 9 and plot the accuracy of the models on training and test sets:"
662 | ]
663 | },
664 | {
665 | "cell_type": "code",
666 | "execution_count": 14,
667 | "metadata": {},
668 | "outputs": [
669 | {
670 | "data": {
671 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAETCAYAAADZHBoWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3WdgVGXah/FrSnpvhBAgtBQCSGiCFFEBBQIIhBIjCGJZ\n667o2l1kV2QRC6C+oCKiYCEIKATBQpOOgAQIKSShJyGk92TaeT9ER1kIEZLJZJL79wVmzpwz/xzC\n3POc8xSVoigKQgghBKC2dgAhhBCNhxQFIYQQZlIUhBBCmElREEIIYSZFQQghhJkUBSGEEGZSFESD\nOnDgAKNGjQIgKyuLUaNGMWbMGI4cOXLV15eVldGzZ8+rbn/kkUdYvnx5veR66KGHSEtLq5dj/a+/\n/e1vrFu37ob3P3bsGLNmzQIuP3836lrn/euvv+aLL76o0/GFbZOiIKzmwIED+Pr6smHDBnr06HHV\n17i4uDB27FjWrl172fMXL17kl19+ISoqql6yLF26lE6dOtXLsepbWloa2dnZ9Xa8a533w4cPU1lZ\nWW/vJWyP1toBRNO1Zs0ali9fjlqtxsvLizfeeMO8bf/+/SxcuJCSkhKmTp3KypUreeihh4iOjmbI\nkCGXHScmJobJkyfz0ksv4ezsbD72yJEjcXd3Jzc3l1mzZpGXl0dOTg6BgYEsXLgQHx8f7rjjDm66\n6SZSUlIYM2YMq1atYvv27ajVaioqKrjjjjvYuHEjEydOZNGiRZSXl7NgwQLatGlDamoqOp2OWbNm\n0a9fP/Lz83nxxRc5d+4cnp6e+Pn5ERwczJNPPnlZ3uzsbF544QUuXbpEq1atyMvLM29LT0/n9ddf\np7CwEKPRyNSpU5kwYQIHDhxg/vz5+Pv7c/78eRwdHZk3bx7Ozs68++67lJSU8OKLLzJ27FjKy8uZ\nOXMmp06doqqqijlz5tC7d+8rzn9sbCwrV65ErVbj6+vLv/71L7Kzs68477/76aef2LZtG3v27MHR\n0ZH8/Hzi4+O5dOkSoaGhvPXWWyxZsoQff/wRk8lEYGAgr776Kv7+/pSUlPD6669z8uRJ9Ho9t9xy\nC8899xxarXzE2BxFCAtISkpS+vbtq2RmZiqKoijLly9X/vWvfyn79+9XIiMjFUVRlLVr1yoPP/zw\nXzrelClTlLVr1yqKoihGo1G57bbblKSkJEVRFOXTTz9VPvzwQ0VRFMVkMikPPvigsmzZMkVRFOX2\n229X3n//ffNxxowZo+zYsUNRFEX5+uuvlZkzZ5pfd+zYMWX//v1K586dlcTEREVRFGXZsmXKvffe\nqyiKosycOVOZP3++oiiKkp2drQwYMEB59913r8j62GOPKQsWLFAURVHOnDmjREREKGvXrlX0er0y\ncuRIJSEhQVEURSkuLlZGjBihHDlyRNm/f78SFhamHDx4UFEURfnyyy+VcePGXXGefs8XHx9vPq/3\n3XffFRn27t2rDB06VMnLyzMfY8SIEYrJZLrmeX/++eeVjz/+WFEURXn33XeVu+66S9Hr9YqiKMo3\n33yjPPXUU+bHq1atUh588EFFURTlhRdeUFasWKEoiqIYDAbln//8p/LRRx9d9T1E4yZlXFjEvn37\nGDhwIAEBAQBMnz4dqL50cSNiYmL4/PPPGT9+PDt37qRly5aEhYUBMG3aNA4dOsTy5cs5c+YMqamp\ndO/e3bzvn79F33vvvaxevZrBgwcTGxvLc889d8V7tWrVis6dOwMQHh7ON998A8DPP/9s/nuLFi0Y\nPnz4VbPu3buX559/HoCgoCD69u0LwJkzZzh37hwvvfSS+bWVlZUkJibSsWNHwsLCzFmjoqL4z3/+\nQ0FBwRXHb9OmjfnnCwsLu+LSGsCuXbsYOXIk3t7eAIwfP57XX3+dCxcuXDVzTSIiIszf9rdv387x\n48fNl+xMJhMVFRUA7Nixg+PHj7NmzRrzzyVskxQFYREajQaVSmV+XFlZSUZGxg0fb9iwYcydO5cz\nZ86wevVqpkyZYt725ptvcuzYMaKioujbty8GgwHlT1N6/X7JCWD06NG888477N+/n/Lycvr06XPF\nezk6Opr/rlKpzMfSarWXHVetvvotuT/v8/t+AEajEXd3d9avX2/elpubi5ubG/Hx8Wg0msuOoyjK\nFc8B2NnZ1fhef973as8ZDIarZq7Jn8+dyWTiwQcfJCYmBgCdTkdRUZF526JFi+jYsSMAxcXFl/37\nC9shN5qFRfTt25d9+/Zx6dIlAFatWsWbb755w8fTarVMmjSJFStWkJiYyJ133mnetnv3bqZNm8bY\nsWPx8fFh7969GI3Gqx7HycmJMWPG8NJLLxEdHX1dGQYPHmz+JlxQUMCWLVuu+sE3aNAgYmNjAcjM\nzDS3jtq3b4+Dg4O5KPzeCyghIQGA5ORkkpOTger7AT179sTd3R2NRnPdH+YDBw5k06ZN5OfnA7B2\n7Vo8PT0JCgq65n7Xeq+BAweyZs0aSktLAVi0aJG5pTVw4EA+/fRTFEVBp9Px6KOP8vnnn19XZtE4\nSEtBWERoaCjPPvssDz74IAB+fn7mb/o1qelG8+8mTZrEkCFDePjhhy/7tvz4448zf/58Fi9ejEaj\noWfPnpw7d67G9xk/fjyrV69m7Nix1/Uzvfjii7zyyiuMHj0aT09PWrVqdVmr4nevvvoqL774IiNG\njLjsMpe9vT2LFy/m9ddf5+OPP8ZgMPCPf/yDXr16mXsELVy4kIyMDLy9vZk/fz4APXr0YOHChTz+\n+OPcd999fynrgAEDmD59OtOmTcNkMuHt7c2HH35YY+vmd7feeiuvvfbaVbdNnDiR7OxsJk2ahEql\nIiAggHnz5gHw8ssv8/rrrzN69Gj0ej39+/c3/9sL26JSrtbOFKKJUhSFpUuXkpGRwb///e/r2veL\nL74gPDycHj16oNPpiImJ4cknn2Tw4MF1znXgwAFee+01Nm7cWOdjCVEX0lIQzcqQIUPw9vZmyZIl\n171vp06deO211zCZTOj1eoYPH14vBUGIxkRaCkIIIczkRrMQQggzKQpCCCHMbP6eQk5OyQ3v6+Xl\nTEFBeT2msSxbyitZLceW8tpSVrCtvHXN6ufndtXnm3VLQau9cmBQY2ZLeSWr5dhSXlvKCraV11JZ\nm3VREEIIcTkpCkIIIcykKAghhDCToiCEEMJMioIQQggzKQpCCCHMpCgIIYQwk6IghBA2JrUgnU9+\njaXCUP8r3Nn8iObG6L33FpCSkkR+fh6VlZW0ahWIp6cXc+a8UfvOv8nKyuTUqXQGDBjEggXzmTJl\neo0jEP+qN954ndTUFD7+eEWdjiOEsA69yUBc+vdsO78LtUpFX5+bcdJeuaZHXUhRsIAnn5wJwKZN\ncZw9e4ZHH33yuo9x6NAvZGVlMmDAIGbOvHId4etVXl5OUtIJ2rRpy9Gj8XTvHlHnYwohGk5GaRaf\nnviKzLKLtHDy5R8DZuBp8q3392nyRWH1tjQOJl+66jaNRoXReP0zh/cJa8GkOzrdUJ7Fixdx/Pgx\nTCYTMTFTGTz4Dr7+ehU//rgZtVpN167deOSRJ/nyyxXodDq6dr2JlSuX8/LLs/n88x84e/YC+fn5\nZGdf5B//eJo+ffqxa9cOli9fiouLK66uboSGhjF9+uWrXm3d+iN9+vSlZ89erFsXay4Ku3bt4LPP\nPkFRFDp3DueZZ15g9+6fr3hu/PhIvv56A1qtlvffX0hwcAje3j4sXboErVbL2LET0GjUfPvtWgwG\nPY6ODvz73/NwdXXjnXfeICUlCYPBwIMPPsrRo7/SqlVrxo6NoqiokGee+bu0XoSogUkxsfXcTjae\n+gGDYmRQ4C2M6xRJax+fOs39VpMmXxQak927d5KTk8OSJcuoqqrk4Yen07v3zWzatIEXX5xFcHAo\n33yzBrVaTUzMfWRlZdK//0BWrlxuPoaDgyNvv/0u+/bt4euvV9GjR2/effcdPvroM7y8vJg168Wr\nvndc3Le8/PJsWrduwzvvzCcvLxd3dw8WLXqbjz9eiaenJytXfkp2dvYVz+XkXL2oAhgMBj766FMA\nPvtsGW+99S4ODg4sWPBfDh48gEqlpry8nKVLV1BUVMiaNbGMGjWW//7334wdG8UPP2zmrrtG1ut5\nFqKpyKsoYGVSLKmFp3Czd2VK2ES6+na26Hs2+aIw6Y5ONX6r9/Nzs0ilrcmpU2kkJSXyxBMPA2A0\nGsnOvsgrr/yHr75aycWLWXTr1p1rrXsUEhIKgL+/P1VVOvLz83B398DLywuA7t0jKCm5/GdKT0/j\n3LmzLFr0NgAqlYr169cxatTdeHp64enpCcDUqdO5dCn7iuf+15/ztW37x0Lwnp5evPbaLJydnblw\n4Szh4d3JysqkS5duAHh4ePLAA38DQKu149y5s2zZ8gNvvbXor59EIZoBRVH45eKvrD65nkpjJd39\nuhITGoWrvYvF31t6HzWgoKB29O59M++//xGLFi3h9tuHEhAQSFzcNzz33Mu8//5HJCYmkJiYgEql\numpxUKlUlz329vahuLiYoqJCAE6cSLhin7i4b3nkkSd45533eOed91i4cDEbN67H29uHoqJCcxF5\n++03yMnJueK55OQk7O3tycvLRVEU0tJOmo/9+0LwxcVFfPbZMv7zn//y3HMv4+DggKIotGvXnuTk\nxN9eU8zTT1ffXxk9eizLln1IQEAr3N096npqhWgySvVlfJzwOSuSYgGFqZ0n8VDXqQ1SEKAZtBQa\nk1tvvZ0jR37lsccepKKinNtuG4KTkxPt2rXn8ccfxMnJmRYt/AkLC8fe3p4vvviM4ODQax5Tq9Xy\n1FP/5Omnn8TV1Q2TyUiHDh3N23U6Hdu3b2HlytXm51q1CiQoqB0//7ydp556ln/+8++o1WpCQzsT\nHt7liudCQ8O4995pPP30EzV+iLu6utG5czh/+9v9aDQavL09yc3NISbmPg4dOshjjz2I0Whkxozq\nVtLgwXewYMGbvPbavHo6u0LYvhN5KXyRtJoiXQkdPdpzX/hkfJ28GzSDza/RXJfLPw19+aiuasq7\nYsUn3HPPVOzs7Hj11RcZMGAwd9453AoJ/1DbuS0vL+fvf3+EpUs/u6L109Cayu9BY2RLWcF6eXVG\nHd+kfcfOjH1oVBpGd7iLIW1vRa2q+WJOXbPW1MVdWgpNgKOjIw8/PA0HB0cCAwO5/fYh1o50TUeP\nHuHtt+fxwAOPWL0gCGFtZ4vP82niV1wqzyXAxZ/p4ffQ2q2V1fJYrCiYTCZmz55NSkoK9vb2zJkz\nh6CgP25Kfvvttyxbtgw3NzfGjRvHxIkTa91HXN2kSTFMmhRj7Rh/WffuPVixItbaMYSwKqPJyA9n\nt7H5zFZMiok72gxiTIfh2GnsrJrLYkVhy5Yt6HQ6YmNjiY+PZ968eSxZsgSA/Px83n33XdatW4e7\nuzvTp0/nlltuITExscZ9hBCiqbhUnsNnibGcKT6Hl4MnUztPItT7xsY+1TeLFYXDhw8zaNAgACIi\nIkhI+KNXzIULFwgNDTV3e+zWrRtHjx7l2LFjNe4jhBC2TlEUdmceYF1qHDqTnj7+PZgUMhZnOydr\nRzOzWFEoLS3F1dXV/Fij0WAwGNBqtQQFBZGWlkZubi4uLi7s27ePdu3aXXOfmnh5OddpAeu6zifU\n0Gwpr2S1HFvKa0tZwXJ5CyuKWHLwc45kJeBi78xjve6jf9vedTqmJbJarCi4urpSVlZmfmwymcwf\n7h4eHrz44os8+eSTeHp60qVLF7y8vK65T00KCspvOKP0jLAcyWo5tpTXlrKC5fIezUngy+S1lOrL\nCPMKZmr4JDwdPKzae7LBex/17NmT7du3M3LkSOLj4wkJCTFvMxgMJCYm8uWXX6LX67n//vuZOXMm\nRqOxxn1sSV1mSU1NTWH37p3cf/9DV92+f/9esrMvcvfd4+uUMTExgccee5AlS5bRuXOXOh1LCHF1\nFYZK1qRuYH/WIezUWiYG382trW+5ZldTa7NYURg2bBh79uwhOjoaRVGYO3cucXFxlJeXM3nyZADG\njRuHg4MD999/P97e3lfdxxbVZZbU4ODQaw5Y69evf53zQfUo5+joKaxb9zUvvyxFQYj6llZ4mhWJ\nq8irLKCNWyDTw6Np6eJv7Vi1slhRUKvV/Oc//7nsuY4d/xhp+8QTT/DEE0/Uuk9drUvbyJFLx6+6\nTaNWYTRd/9i9Hi26Mb7TqOve79dfD7FkyXvY2dkxZkx1QVy37msMBgMqlYq5c9/i1Kk01q9fy7//\n/V+io8fRrVt3zp07i7e3Nx9+uMRcaMaOjWL27Jdp0cKfjIwLhId34Z//fJHCwkL+/e+X0ev1tGkT\nxK+/HiQ29tvLcpSXl3P48EFWrlzNtGnRFBYW4unpSUFBAa+//iqlpaUoisIrr/wbV1e3K5778cfN\n+Pj4MHbsBM6ePcObb87l/fc/YurUSbRpE4SdnZZZs17hxRdfQaerIi8vl4ceeoxbb72NPXt2sXz5\nUhRFISQkjHvumcJrr/2LpUurZ0mdNetFoqPvJTy863WfXyEaA4PJwHenf+KnszsAGB50ByPaD0Wr\nto1hYbaRsgnR6XQsXfoZUD0S+c03F+Ho6Mj8+a/zyy/78PX1M782MzODRYuW4O/fkkcfncHx45cX\nt/Pnz7Fgwfs4ODgyadLd5OXl8sUXnzFo0G2MHz+Rgwf3c/Dg/isybN36I4MH34GDgwN33DGMjRu/\nZcqU6Xz22TIGDryVsWMncPz4UZKSTpCYeOKK52pSUVHB9OkPEBISRmrqcaKj76Vnz94cP36UZcs+\npH//gSxYMJ+lSz/Dy8ubL774DHt7BxwcHDl9+hQ+Pj5kZWVIQRA2K7P0Ip8lruJCaSa+jt5M6xJN\nB4921o51XZp8URjfaVSN3+qtcRPsz7OKenl5M2fOqzg7O3P27Bm6dr3pstd6eHji798SgBYt/Kmq\nqrpse2Bga5ydqyfJ8vHxRafTcebMGUaMqP55b7qpx1UzxMV9i0aj4emnn6SqqpJLly4RE3Mf586d\nJTJyDADdunWnW7fufP/9piueW7bsQ/Ox/neWlLZt2wHg5+fH55+/y3ffrQdUGAwGiooKcXNzw8ur\nei6Xe++dBlRPjrd5cxz+/i25806ZRlvYHpNiYsf53aw/9T0Gk4EBrW5mfKfROGodrB3tujX5otDY\nqNXV0zqUlpaybNmHrF27EYCZMx+/4gO2tikgrra9Q4eOJCQcJzg4lBMnrrxslp6ehslkMq+BAPDU\nU4+xd+8u2rVrR3JyIsHBIcTH/8revbuv+pybmzt5eXkAnDyZfNVMixYtYvjwUdxyywC++24Dmzdv\nxMvLm9LSUoqLi3B392Dhwje5884R3HbbEL766nM8PDxkgjxhcwoqC1mRtJqTBWm42rlwb5d7ucnP\ndu/TSVGwEhcXF7p1684jj9yPRqPFzc2N3NwcAgLqNufJlCnTee21WWzb9hO+vn5XdOmNi/vmikVt\nRo8ex9q1q5k1aw7//e9/+OGHTahUKl544V84O7tc8ZxKpWLWrBc5cuQwoaFXX/Bj+PDhvPfeIj7/\n/FP8/FpQWFiIWq3m6aef59lnn0KtVhMSEkrnzl1QqVRERPSgoKBAptEWNuXgxSPEnvyGCkMl3XzD\nuTdsAm72rrXv2IjJLKlNrA/1vn278fT0onPnLhw8eICVK5fz7rsfNFDCP1zvuX377Te47bY76NWr\njwVTXV1T/D1oLGwpK/z1vGX6cmJTvuHwpaPYa+yZEDya/gE3N+gEjzY3TkFYR0BAIP/973/QaDSY\nTCaeeuqf1o5Uq5kzH8fDw9MqBUGI65Wcn8rKpNUUVhXR3j2IaeHR+Dn7WDtWvZGi0MS0a9eeDz9c\nXvsLG5EFC/7P2hGEqJXOqGd9+iZ2XNiDWqVmdIfhDGs7GI36xqfZaYykKAghRC3OlVzgsxOruFh+\niZbOLZjWJZq2bq2tHcsipCgIIUQNTIqJH8/u4LvTP2JSTNzWegB3dxyJvZXXPLAkKQpCCHEVuRV5\nfJa4ilNFZ/Gwd2dq+CQ6e9vmfGzXQ4qCEEL8iaIo7Ms6yJrUDVQZdfRq0Z3JoeNwsXO2drQGIUVB\nCCF+U1RZzEfHV3As9wROWkemh99Db/+IZrWWuBQFIUSzV2mo5FhuIt/u+Y6iqhJCPDtyX/hkvBw9\nrR2twUlREEI0O4qikF2ew4m8ZE7kJZNWeBqjYsROrSWq0yhuazOwUa95YElSFIQQzYLOqCe1ML26\nEOQmk1uZb97Wxi2Qrj5h3NV5IHZVLlZMaX1SFIQQTVZeRb65NZBSkI7epAfAUeNID79udPEJI9wn\nFA8HdwD83G1rWg5LkKIghGgyjCYj6UWnSchL5kReChfLss3bAlz86eITRhefMDp6tGtyI5HrixQF\nIYRNK6oq5kReCifykknOP0mlsXrdETu1HV19Ov9WCELxcfK2clLbIEVBCGFTTIqJM8XnzZeFzpdk\nmLf5OnrTN6A3XXzCCPbs0KRHHluKFAUhRKNXqi8jKe8kJ/KSScxPoUxfDoBGpSHMK5guPqF08Qmj\nhbNfsxpTYAlSFIQQjY6iKFwozTS3Bk4XnUOheukXTwcPBrS6mS4+YYR6dcJR62jltE2LFAUhRKNQ\nYagkJT/VXAiKdNW9gFSo6OARZL5JHOgaIK0BC7JYUTCZTMyePZuUlBTs7e2ZM2cOQUF/LFq/YcMG\nli9fjlqtJioqipiYGPR6PS+88AIZGRmo1Wpee+01OnbsaKmIQggrqh5Adqm6p1BuMmlFpzEpJgBc\n7Vy4uWVPuviE0dk7pNnMO9QYWKwobNmyBZ1OR2xsLPHx8cybN48lS5aYt8+fP5+NGzfi7OxMZGQk\nkZGRHDx4EIPBwKpVq9izZw8LFy7kvffes1REIUQD0xl1nCxI/623UBJ5lQXmbW3dWtPFJ4yuvmG0\ndWvdbEcUW5vFisLhw4cZNGgQABERESQkJFy2PTQ0lJKSErRaLYqioFKpaN++PUajEZPJRGlp6RWL\nzgshbE/ubwPIEvKSSC1IR28yAOCkdaRHi5vo+tsAMnf7q68ZLBqWxT51S0tLcXV1NT/WaDQYDAbz\nB31wcDBRUVE4OTkxbNgw3N3dKSsrIyMjgxEjRlBQUMAHH9S+4LyXlzNa7Y0PQqlp8erGypbySlbL\naex5CyuL2X5qLzsPHSCj+KL5+TYeregR0JWeAV0J8e2AthEOIGvs5/bPLJHVYkXB1dWVsrIy82OT\nyWQuCMnJyezYsYOtW7fi7OzMs88+y+bNm4mPj2fgwIE888wzZGVlMW3aNOLi4nBwcKjxfQoKym84\no5+fbQ1pt6W8ktVyGmteRVFILUxnV8Z+juacwKgYsdfY0c23s/kmsbejl/n1BXk3/n/XUhrrub2a\numatqaBYrCj07NmT7du3M3LkSOLj4wkJ+WPFIjc3NxwdHXFwcECj0eDt7U1xcTHu7u7Y2VUPNvHw\n8MBgMGA0Gi0VUQhRD8r05RzIOsTuzANkl+cA0MqlJQMD+zGyy62UFRmsnFBcD4sVhWHDhrFnzx6i\no6NRFIW5c+cSFxdHeXk5kydPZvLkycTExGBnZ0fbtm0ZN24cer2el156ydwTaebMmTg7S68DIRob\nRVE4XXyO3Rn7+fXSUfQmA1q1lj7+PRkU2I8OHkGoVCqc7Z0owza+eYtqKkVRFGuHqIu6Np9spakI\ntpVXslqONfNWGCo5ePEIuzP3k1GaBUALJ18GBPalX0BvXO0un3Zazq3l2NzlIyFE03G+JINdGfs5\nmH0EnVGHWqWmh183Bgb2I8Sro3QfbUKkKAghrkpn1HE4+yi7Mvdztvg8AF4OntwVdDu3BPQxr0Eg\nmhYpCkKIy2SVZbM7Yz8HLh6mwlCJChVdfTozKLAf4T6h0ipo4qQoCCHQmwwcvXScXZn7SSs8DYC7\nvRuD2w2gf8DN+Dh51XIE0VRIURCiGcspz2NP5gH2ZR2kVF89rijMK5iBgf24yTdcVidrhqQoCNHM\nGE1GjuclsTtjP0n5JwFwsXNmaNvBDGjVlxbOvlZOKKxJioIQzURBZSF7Mn9hb+YvFOmKAejo0Y6B\ngf3o4dcNO1mlTCBFQYgmzaSYSMo/ya6M/STkJqGg4KhxZHDr/gxs1Y9Wri2tHVE0MlIUhGiCinUl\n7Ms8yJ7MA+bpqdu6tWZQYD96+UfgoLG3ckLRWElREKKJ+POEdPE5CZgUE/ZqO/oH3MzAwL4Eubex\ndkRhA6QoCGHjfp+Qblfmfi6V5wJ/TEh3c8seOGmdrJxQ2BIpCkLYoD9PSHf40lEMv01Id3PL6gnp\n2rsHyTrG4oZIURDChlQYKvkx7Vc2p/z8lyakE+J6SVEQwkYcz03ki6Q1lOhLZUI6YTFSFIRo5CoN\nVaxLi2NP5i9oVRomdBlJT8+eMiGdsAgpCkI0YqeKzvJZ4ipyK/IIdA1gWng0Ee1DbGbOf2F7pCgI\n0QgZTUY2ndnCD2e2ATCs7W1EdrgTO7X8lxWWJb9hQjQyF8su8VniV5wrycDb0Yv7Ok8m2KuDtWOJ\nZkKKghCNhKIo/Jyxl2/TvkNvMtC3ZS8mhtyNk9bR2tFEMyJFQYhGoLCqiM+TviYp/yQuds5MC7+H\nHi26WTuWaIakKAhhZb9eOsZXyWspN1QQ7hPKlLCJ0rNIWI0UBSGspFxfweqT6zmY/St2ajsmh4xj\nUGA/GYksrMpiRcFkMjF79mxSUlKwt7dnzpw5BAUFmbdv2LCB5cuXo1ariYqKIiYmBoAPP/yQbdu2\nodfrueeee5g4caKlIgphNScL0lmRGEtBVSFBbm2Y1iUaf2c/a8cSwnJFYcuWLeh0OmJjY4mPj2fe\nvHksWbLEvH3+/Pls3LgRZ2dnIiMjiYyMJDk5mSNHjvDVV19RUVHBJ598Yql4QliF3qgn7tQPbDu/\nC5VKxch2QxnebogseykaDYsVhcOHDzNo0CAAIiIiSEhIuGx7aGgoJSUlaLVaFEVBpVKxe/duQkJC\nePzxxyktLeW5556zVDwhGlxGaRafnviKzLKLtHDy5b7waNp7tLV2LCEuY7GiUFpaiqurq/mxRqPB\nYDCg1VZksgSuAAAgAElEQVS/ZXBwMFFRUTg5OTFs2DDc3d0pKCggMzOTDz74gAsXLvDoo4/y/fff\nX/Maq5eXM1rtjX/L8vNzu+F9rcGW8krWaiaTiY0nt7DqeBwGk4E7O97KlIjxOGodbviYcm4tx5by\nWiKrxYqCq6srZWVl5scmk8lcEJKTk9mxYwdbt27F2dmZZ599ls2bN+Pp6UmHDh2wt7enQ4cOODg4\nkJ+fj4+PT43vU1BQfsMZ/fzcbGq6AFvKK1mr5VXksyIplrTC07jZuzIlbCJdfTtTUqCjBN0NHVPO\nreXYUt66Zq2poFhsasWePXuyc+dOAOLj4wkJCTFvc3Nzw9HREQcHBzQaDd7e3hQXF9OrVy927dqF\noihkZ2dTUVGBp6enpSIKYTGKonAg6zBzf1lAWuFpuvt15ZWbn6Grb2drRxPimizWUhg2bBh79uwh\nOjoaRVGYO3cucXFxlJeXM3nyZCZPnkxMTAx2dna0bduWcePGYW9vz8GDB5kwYQKKojBr1iw0GrkB\nJ2xLqa6Mr1LWEZ9zHEeNA1M6T6Jfy17S1VTYBJWiKIq1Q9RFXZtPttJUBNvK21yznshL4fOk1RTr\nSujo0Z77wifj6+RdL8f+XXM9tw3BlvJa6vKRDF4Toh5UGXV8m/YdOzP2oVFpGNtxJEPa3iqL3wib\nU2tRyMnJwc9PBtUIUZMzxef4LHEVl8pzCXDxZ1r4PbRxa2XtWELckFqLwpQpUwgKCmLcuHEMHToU\nOzu7hsglRKNnNBn54ew2Np/ZikkxcUebQYzpMBw7jfwfEbar1qLwww8/cOjQIb755hveeustBg8e\nzLhx4+jWTWZwFM3XpfIcPk1cxdni83g5eDK18yRCvTtZO5YQdfaX7in07t2bbt26sXnzZhYsWMC2\nbdvw9vZm1qxZREREWDqjEI2GoijszjzAutQ4dCY9ffx7MClkLM52TtaOJkS9qLUo7N27l/Xr17N3\n714GDx7MggUL6NmzJykpKTz00EPmsQhCNHVFVSV8kfw1J/KScdY6MaXzRHr5y5ci0bTUWhT+7//+\njwkTJjB79mycnP74NhQaGsqMGTMsGk6IxiI+J4Evk9dQpi8nzCuYqeGT8HTwsHYsIepdrf3lPvzw\nQ8rLy3FyciI7O5tFixZRUVEBwPTp0y2dTwirqjBUsjJpNUuPr0Bn1DEx+G4ej3hACoJosmotCv/8\n5z+5dOkSAC4uLphMJpm9VDQLaYWn+e8vC9ifdYg2boG80Ocf3NZmgIw9EE1arZePfp+1FKonuZs5\ncyZ33323xYMJYS0Gk4HvTv/ET2d3ADA86A5GtB+KVi1jPUXTV+tvuUqlIiUlhdDQUADS09PNs50K\n0dRkll7k08SvyCjNwtfRm2ldoung0c7asYRoMLV+uj///PPMmDEDf39/AAoKCpg/f77FgwnRkEyK\niR3nd7P+1PcYTAb6B9xMVPDoOq15IIQtqrUo9O/fn+3bt3Py5Em0Wq15vQMhmorc8nzei1/OyYI0\nXO1cuLfLvdzk18XasYSwilqLwqlTp/jyyy8pLy9HURRMJhMXLlzgiy++aIh8QliM0WRkT+YvxJ3+\nnnJ9Bd18w7k3bAJu9q617yxEE1VrN4qZM2fi7u5OUlISnTt3Ji8vj+Dg4IbIJoTFnCxI441D7xJ7\n8hsURSEmLIq/dZsmBUE0e7W2FEwmE3//+98xGAyEh4cTHR1NdHR0Q2QTot7lVeSzLu074nOOo0LF\nLQF9uL9PFPpS6WYqBPyFouDk5IROp6Ndu3acOHGC3r17U1VV1RDZhKg3VUYdP57dzpZzP2MwGejg\nEcSE4DEEubfB08mNnFLbWFhFCEurtSiMGTOGRx55hLfeeovJkyeza9cuc08kIRo7RVE4lB3Pt+mb\nKKwqwtPBg7EdR9LbP0KWxxTiKmotCr1792bs2LG4urqycuVKjh8/zoABAxoimxB1cq74Al+nbuBU\n0Rm0ai3Dg+5gWNDt0s1UiGuotSjMnDmTzZs3A9CyZUtatmxp8VBC1EWJrpQN6ZvZl3UIBYUIv66M\n6zSq3tdKFqIpqrUodOrUiffff5/u3bvj6Ohofr5Pnz4WDSbE9TKYDOy4sIfNp7dSaayklUtLJgSP\nkcVvhLgOtRaFwsJCDhw4wIEDB8zPqVQqVqxYYdFgQlyPE3nJrE2NI7s8B2etE5NCxjKwVV80ao21\nowlhU2otCitXrryhA5tMJmbPnk1KSgr29vbMmTOHoKAg8/YNGzawfPly1Go1UVFRxMTEmLfl5eUx\nfvx4PvnkEzp27HhD7y+ah+zyHNamxnEiLxm1Ss3g1v0Z2X4YrnYu1o4mhE2qtShMnTr1qr00amsp\nbNmyBZ1OR2xsLPHx8cybN48lS5aYt8+fP5+NGzfi7OxMZGQkkZGReHh4oNfrmTVr1mWXqoT4XxWG\nCjaf3sr2C7sxKSZCvDoxMXgMrVzlnpcQdVFrUXjyySfNfzcYDGzduhV3d/daD3z48GEGDRoEQERE\nBAkJCZdtDw0NpaSkBK1Wi6Io5sLzxhtvEB0dzUcffXRdP4hoHkyKif1Zh9iQ/j0l+lJ8HL0ZHzyK\n7r5dpIupEPWg1qJw8803X/a4f//+TJw4kX/84x/X3K+0tBRX1z+mDNBoNBgMBvO028HBwURFReHk\n5MSwYcNwd3dn3bp1eHt7M2jQoL9cFLy8nNFqb/y6sZ+f2w3vaw22lLe+sybnpPNp/GpOFZzDQWNP\ndLcxjAodir3Grs7HtqXzCraV15aygm3ltUTWv7TIzu8URSEtLY3CwsJaD+zq6kpZWZn5sclkMheE\n5ORkduzYwdatW3F2dubZZ59l8+bNrF27FpVKxb59+0hKSuL5559nyZIl+Pn51fg+BQXltWapiZ+f\nGzk5tjOS1Zby1mfWgspCvk3fxKHseAD6+PdgbKeReDp4UJRfCVTW6fi2dF7BtvLaUlawrbx1zVpT\nQam1KEyZMsX8d5VKhbe3N6+88kqtb9izZ0+2b9/OyJEjiY+PJyQkxLzNzc0NR0dHHBwc0Gg0eHt7\nU1xcfNnMq1OnTmX27NnXLAiiadMZ9Ww9t5Mfz25DZ9LT1q01E0PGyKI3QlhQrUVh27Zt6PV67Ozs\n0Ov16PV6nJ2daz3wsGHD2LNnD9HR0SiKwty5c4mLi6O8vJzJkyczefJkYmJisLOzo23btowbN65e\nfiBh+xRFIT4ngW/SNpJXWYCbvSuTOoylb0AvWR9ZCAtTKYqiXOsFmzdvZvHixcTFxXHu3DmmTp3K\nv/71L4YOHdpQGa+prs0nW2kqgm3lvdGsGaVZrDm5gZOF6WhUGm5vM5Dh7YbgpLVcbzRbOq9gW3lt\nKSvYVl6rXT5avHgxy5cvB6Bt27asW7eOGTNmNJqiIJqGUn0Z3536kV0Z+1FQ6OrTmfHBo/B3lsuH\nQjSkWouCXq/H19fX/NjHx4daGhdC/GVGk5Fdmfv57tSPlBsq8Hf2Iyp4DF18Qq0dTYhmqdai0KtX\nL55++mlGjx4NwKZNm4iIiLB4MNH0JeensiZ1A1ll2ThpHYnqNIrBrQfI1BRCWFGtReHVV19l5cqV\nxMbGotVq6dOnD/fcc09DZBNNVG5FHuvSvuNoTgIqVAxodTOjOwyXpTCFaAT+0uUjR0dHPvjgA7Kz\ns1m1ahVGo7EhsokmptJQxY9nt7P1/E4MJgMdPdoxMeRu2rgFWjuaEOI3tRaFZ555htDQ6uu7Li4u\nmEwmnnvuOd577z2LhxNNg6IoHMw+wrdpmyjSFePl4Mm4TiPp2aK7TE0hRCPzl0Y0f/DBB0D1KOWZ\nM2dy9913WzyYaBrOFp/n65MbOF18Fju1lhHthnJn0G3Ya+ytHU0IcRW1FgWVSkVKSoq5tZCenm6e\nrkKImhRWFLEyaQ37sw4B0KPFTYzrGImPk5eVkwkhrqXWT/fnn3+eGTNm4O/vD0BBQQFvvvmmxYMJ\n26QoCj9n7GXjqR+oMFQS6BrAxOAxBHvJuhhC2IJai0L//v3Zvn07ycnJ7Ny5k127dvHQQw9x5MiR\nhsgnbIjRZGR16np2Z+zHzd6F6NDxDGh1s0xNIYQNqbUonD9/ntjYWNatW0dxcTGPPPLIZYvlCAHV\nPYs+OfEFJ/KSCXQN4F+3/x1jmYw3EMLW1PgV7qeffuKBBx5g4sSJFBUV8eabb9KiRQueeOIJvL29\nGzKjaOQKq4pY+OsSTuQlE+4dytM9H8Xb2dPasYQQN6DGlsKTTz7J8OHDiY2NNa+tLN0Hxf/KLL3I\n4qOfUFBVyIBWfZkcMlZGJAthw2osChs2bOCbb74hJiaGwMBAIiMjZdCauExyfipLj6+k0ljJ3R1G\nMCzoNvniIISNq/HyUUhICM8//zw7d+7k4Ycf5pdffiE3N5eHH36Yn3/+uSEzikZof9Yh/u/oMgwm\nPfeH38Od7W6XgiBEE1DrjWaNRsPQoUMZOnQo+fn5rF+/nrfffpvBgwc3RD7RyCiKwqYzW9h0+iec\ntU483G0awV4drB1LCFFPrmsUmre3N/fffz/333+/pfKIRsxgMvBV8jr2XzyEj6M3j3WfQUuXFtaO\nJYSoRzI0WfwlFYYKlh5fSUpBGkHubXjkpum421995SYhhO2SoiBqlV9ZwOKjn5BVls1Nvl24v8s9\nMneREE2UFAVxTedKLrDk6HKKdSXc1noAUcGjZYSyEE2YFAVRo4TcJJad+AK9Uc+E4DHc3magtSMJ\nISxMioK4ql0Z+4hN+RatWstD3abS3a+rtSMJIRqAFAVxGZNiYkP69/x0bgeudi48ctP9tPdoa+1Y\nQogGYrGiYDKZmD17NikpKdjb2zNnzhzzdBlQPWJ6+fLlqNVqoqKiiImJQa/X89JLL5GRkYFOp+PR\nRx9lyJAhlooo/ofeqGdl0moOXzqKv7Mfj3Wfga+Tj7VjCSEakMWKwpYtW9DpdMTGxhIfH8+8efMu\nm111/vz5bNy4EWdnZyIjI4mMjGTLli14enry5ptvUlhYyNixY6UoNJBSfRkfHfuM9KIzdPRoz99u\nmoaLnbO1YwkhGpjFisLhw4cZNGgQABERESQkJFy2PTQ0lJKSErRaLYqioFKpGD58OHfddRdQPXJW\no6l9YjUvL2e02hufgM3Pz7b62lsi78XSHBYeXEJWySX6t+3NYzffh73Grs7HtaVza0tZwbby2lJW\nsK28lshqsaJQWlqKq6ur+bFGo8FgMJiX8gwODiYqKgonJyeGDRuGu7v7Zfv+/e9/56mnnqr1fQoK\nym84o5+fGzk5JTe8f0OzRN7TRWf54NinlOrLuDPodkZ3uIui/Eqgsk7HtaVza0tZwbby2lJWsK28\ndc1aU0GxWIdzV1dXysrKzI9NJpO5ICQnJ7Njxw62bt3Ktm3byM/PZ/PmzQBkZWVx3333cffddzN6\n9GhLxRNAfE4Ci458SLmhgntCx3N3xxEyBkGIZs5inwA9e/Zk586dAMTHxxMSEmLe5ubmhqOjIw4O\nDmg0Gry9vSkuLiY3N5cZM2bw7LPPMmHCBEtFa/YURWHbuZ18fHwlKpWaR26azsDAftaOJYRoBCx2\n+WjYsGHs2bOH6OhoFEVh7ty5xMXFUV5ezuTJk5k8eTIxMTHY2dnRtm1bxo0bx/z58ykuLmbx4sUs\nXrwYgKVLl+Lo6GipmM2OSTGxJjWOny/swcPejUe7z6CNW6C1YwkhGgmVoiiKtUPURV2vqdnK9UOo\ne94qo47lJ77keG4irVxa8lj3GXg5WmbZTFs6t7aUFWwrry1lBdvKa6l7CjJ4rZko1pXwwdFPOVty\nnjCvYB7sNgUnrZO1YwkhGhkpCs3AxbJsFh/9hLzKAvq17E1MWJSsoyyEuCopCk1cakE6Hx5fQYWh\nglHt72R4uyGybKYQokZSFJqwgxeP8HnSahTgvs6T6RvQy9qRhBCNnBSFJkhRFH44u524U9/jpHXk\noa73EerdydqxhBA2QIpCE2M0GVmV8g17s37By8GTx7rPoJVrS2vHEkLYCCkKTUiloZKPEz4nKf8k\nbdwCefSm+/FwcK99RyGE+I0UhSaioLKQJceWk1GaRVefMO7vci+OWgdrxxJC2BgpCk1ARmkWi49+\nQmFVEYMCb2Fi8BjpciqEuCFSFGxcUt5JPk5YSaWxinGdIhnS5lbpciqEuGFSFGzY3syDfJWyFrVK\nzQNdp9CzxU3WjiSEsHFSFGyQoihsPPUD35/dhoudM3/rNp2Onu2sHUsI0QRIUbAxepOBL5K+5mD2\nEXydfHi8+wxaOPtZO5YQoomQomBDSnVl/F/8x6QWnqK9exB/u2kabvaute8ohBB/kRQFG5FXkc+H\nhz4lo/giEX7dmBYeXS/rKAshxJ9JUWjkFEXhWG4iX6WspURXypA2tzK200hZNlMIYRFSFBqxrLJs\n1pzcQHJBKmqVmhk9J9PLUya1E0JYjhSFRqhcX853p39iZ8Y+TIqJcO9QooJH061dR5tZFUoIYZuk\nKDQiJsXEnswDxJ36gTJ9OS2cfIkKHk0XnzAZkCZsSqXOwM6jWRxNz6OySm/tOH+Zh5sjbo5afNwd\n8XZ3xMfdAW8PR7zdHLDTNo9ZAqQoNBInC9JZk7qBjNIsHDUOjOsUyW2tB6BVyz+RsB2lFXq2HDrP\n1sMXKKs0oFar0Kpt4wuNApzOqrkl7u5iX10k3B0vLxq/PXZztmsSX97kE8fK8ioK+CZtI0dyjqNC\nxS0BfRjdYTgeDldfVFuIxii/uJLvfznHzqOZ6PQmXJ3sGDuwPRPvDKOqvMra8f4yNw8nUk/nkVdc\nSX5RZfWfxVXkFVf//fylshoLh1aj/p+i4VD9p8dvj90csLdr/K0NKQpWUmXU8dPZ7Ww59zN6k4H2\n7kFMDBlDkHsba0cT4i/LzC1j84Gz7D+RjdGk4O3uwF2D23LrTa1wsNfg7mJPjg0VBUd7LS29nWnp\n7XzV7SZFoaRcT35xJXlFldV/Flf99mf146SzBTUe383Z7oqi8edWh5uLPWortzYsVhRMJhOzZ88m\nJSUFe3t75syZQ1BQkHn7hg0bWL58OWq1mqioKGJiYmrdpylQFIXDl47yTdp3FFYV4WHvzthOI+nj\n36NJND1F85CeUcSm/Wc5kpoLQICPMyP7BdE33B+tpul2l1arVHi42OPhYk/7gKuvVaLTGyko+aN1\n8XtL4/cCkplbxtmLNbU2VHi7/amV4e6Ij8efHrs54mBv2daGxYrCli1b0Ol0xMbGEh8fz7x581iy\nZIl5+/z589m4cSPOzs5ERkYSGRnJgQMHrrmPrTtXcoE1JzeQXnQGrVrL8KA7GBZ0u6x7IGyCoiic\nOJ3Ppv1nST5XCECHVu5E9guie7Cv1b/hNhb2dhr8vZ3xr6G1oSgKJRW/tzYub2X83ur4/fxejauT\nHd7uDnRs7cmEWzvg5FC/H+MWKwqHDx9m0KBBAERERJCQkHDZ9tDQUEpKStBqtSiKgkqlqnWfq/Hy\nckZbh14Bfn6Wv3ZfVFnMV8c3sP3UXhQUbm4dwdTu4/F3vf45ixoib32RrJbTkHmNRhN7j2WxZlsq\npzKLAOgZ1oIJdwTTtYNPrS1cObdXagF0vMZ2vcFIbmElOYXl5BRUkFNYUf1nQTk5hRVczK/gYl45\n99wZhp+vS71ms1hRKC0txdX1j3l5NBoNBoMBrbb6LYODg4mKisLJyYlhw4bh7u5e6z5XU1BQfsMZ\n/fzcLNrv32Ay8POFvWw6vYVKYyWtXFoSFTyaMO9gqICciut7b0vnrU+S1XIaKq/eYGTP8Yt8f+Ac\nlworUKng5s4tGNkviLb+1R+cubmljSJrfWlMebVAgIcjAR6OgNdl2xRFwdvHlYL8shvOW1Pxs1hR\ncHV1payszPzYZDKZP9yTk5PZsWMHW7duxdnZmWeffZbNmzdfcx9bcyIvmbWpcWSX5+CsdWJSyFgG\ntuorK6KJRq+80sCO+Ax+PHie4jIdWo2a23oEMvzmNrTwuvolEdGwVCqVxe7dWOwTt2fPnmzfvp2R\nI0cSHx9PSEiIeZubmxuOjo44ODig0Wjw9vamuLj4mvvYiuzyHNalxpGQl4wKFbcG9ieywzBc7eq3\niSdEfSsqreKnQxfYfuQCFVVGnBw0jOwXxLDerfFwlftezYXFisKwYcPYs2cP0dHRKIrC3LlziYuL\no7y8nMmTJzN58mRiYmKws7Ojbdu2jBs3Dq1We8U+tqLCUMnmM1vYcX4PRsVIiGdHJoSMIdA1wNrR\nGpRJUdh9LIvswkoqKm1jJKuriz1+7g6EtPakhZdTs+sFdqmgnO9/Oc/uY1kYjCbcXeyJvKUdt0UE\n4uxomy11ceNUiqIo1g5RF3W5/lcf1w9Nion9WYfZkL6ZEn0pPo5ejO80iu5+Xev9w6UxXe+8moKS\nKpZ9l0jimZr7aTd27s52BLf2JLi1B8FtPGnTwrXRdbGsr9+Dc9klbD5wjl+SslEU8PN0ZETfIAZ0\na1lvUzo09t/Z/2VLeeuatcHvKTQHp4rO8PXJ9ZwrycBebcfoDndxR5tbm+U6B4eSL/HZ98mUVRq4\nqaMPM8Z0pbSk0tqx/hInFwd+TbxI6oVCTp4v5PDJHA6fzAHA3k5Nx1Ye1UWitScdWrnXexfAhqQo\nCifPF7Jp/zmOn8oDoE0LVyJvCaJXqB8adeMqgKLh2e5vtxUVVBayPn0zB7OPANDHvwdjO43E08HD\nyskaXkWVgS+3nGTP8YvYa9VMvTOE23oE0qKFu0194/Jy0jKkV2sURSGvqJLUC0WkXigk9UIRSWcL\nzKNUVSpo28LN3JIIbu2Bpw1cbzcpCkfTctm0/yzpGcUAhLbxZOQtQXRt793sLpmJmklRuA56o56t\n53fyw5lt6Ex62roFMjHkbjp4tLN2NKtIvVDI0rhEcosqCfJ34+Ex4QT42PYNdZVKha+nE76eTtzS\ntSVQPclbWsYfReJMVjFns0vYcvgCUH3ZxXzJqbUnAT7OjeZD1mA0cSAxm80HzpGZW92zr0ewLyP6\nBdEpsPl9iRG1k6LwFyiKwtGcBNalbSSvsgA3O1cmhoylX0CvZrkCmsFoYsOeM3y37wwoEHlLEHcP\nbN/orr3XF1cnOyI6+RLRyReo7r9/OqvEXCTSLhSxN+EiexMuml/fKdCDkN9aEkEt3Rr83FTpjOw8\nlskPv5wjv7gKjVrFgK4tGd4viMB6HuwkmhYpCrXIKM1iTWocJwvS0Kg0DGl7KyPaDcFJ62TtaFZx\nMb+cjzac4MzFEnzcHXlodDghbTytHatB2Wk1hLTxNP/cJkUhM7fsj0tO54uIT8slPi33t9er6RDg\nTnCb6pZEx1YeFuvVU1qhZ9vhC2w5fIHSCj32dmqG9m7NXX3a4uPhaJH3FE2LFIUalOrL+O7UT+zK\n2IeCQlefMMYHj8bf+fqnpmgKFEXh5/hMVm1LRac30b9rS2KGhkiXRaonSWvt50prP1du7xEIVE8l\n/ef7EifPF5JyvhA4iwpo3cLVfLkpuLUH3u51+8DOL67kx4Pn+Tk+kyq9ERdHLWMGtGNIr9a4OdvX\n/YcUzYb8j/4fRpOR3ZkH+O7Uj5QZymnh7MuE4DF08QmzdjSrKS7TsXxTEkfT83Bx1DJjZGdu7uxv\n7ViNmre7I33DHekbXn2eyisNpGf+0ZI4lVXM+UulbPs1AwAfd0dzSyK4tQetfF3+0gRzWXllbN5/\njn0nLmI0KXi5OTDu1g7c2j0AR3v57y2un/zW/ElKfhprUjeQWXYRR40j4zuNYnDr/s169bOjabks\n35REcbmezkFePBDZuc7fapsjZ0ct3Tr40K2DD1B9X+bsxRJzKyIto4j9J7LZfyK7+vUOWjq1/qMr\nbPsAt8vGDpzKLK6euvpkDgrVU1eP6BtEvy5Ne+pqYXnN99PuT3Ir8lmXtpGjOQmoUNE/4GZGd7wL\nd3vbmt2xPlXpjazelsb2IxloNSqi7+jE0D5tZHrkeqLVqOkY6EHHQA+G922LSVG4mFduvtyUeqGQ\nY+l5HEvP++31KtoFuNMp0IPMvHKO/Xa/on2AO5G3BBEhU1eLetKsi0KlvpIN6d+z9fxODCYDHTza\nMTF4DG3dW1s7mlWdzipmaVwiF/PLCfRz4eHRXWjTwrX2HcUNU6tUtPJ1oZWvC4Mjqu9LFJRUVXeF\nPV9dKNIzqns6AXRp783IfkGEtfVsNN1fRdPQbItCYl4KX+5bQ0FFEZ4OHozrOJJe/hHN+j+YyaSw\naf9Z1u8+jdGkcGefNkQN7lBvUx6I6+Pl5kCfsBb0CWsBVA8UPJ1VTFBrL1y0zff3VFhWsy0K287v\nolRXzoh2QxgWdDsOmubdQyOnsIKlGxNJu1CEp6s9D4wKp0s7b2vHEn/i5KAlvJ23Tc3PI2xPsy0K\nD3S9Fy9vZyqLbXo+wDpTFIW9CRf54qeTVOqM9A71477hYbg6Nb/5m4QQzbgoOGmdcHNwpZLm+42r\ntELPih9SOJR8CUd7DQ9EdqZ/15bN+hKaEM1dsy0Kzd2JM/ks25hIYamOTq09eGhUOH6ezXOUthDi\nD1IUmhm9wcjan0/x48HzaNQqxt3agZH92sqUyUIIQIpCs3LhUikfxZ3gQk4Z/t7OPDw6nPYB7taO\nJYRoRKQoNAMmRWHLwfOs+Tkdg1Hhth6BTL69Ew720tVUCHG5ZlsUKnUGcgsrUBSlSd9YzS+uZNl3\nSSSdLcDN2Y77R3Y2TwEthBD/q9kWhQ/Wn+BYeh5ebg6XzVbZ2s8VtbppFImDyZdY8dsSmd07+jB9\nZGc8XJr3eAwhxLU126IwrHcbXJztSUjP5ZekS/ySdAkAJwcNHQOri0RIaw/aB7hjb2dbl1kqqgx8\n8dNJ9iZUL5F5312hDI5o1aRbREKI+tFsi0KX9t7cdnMQly4Vk11QYZ5fJvVCIQmn8kk4lQ+ARq2i\nXYCbuSXRKdCjUc9Pf/J8IR9vrF4is11LNx4abftLZAohGo7FioLJZGL27NmkpKRgb2/PnDlzCAoK\nAtydj28AAA2BSURBVCAnJ4enn37a/NqkpCSeeeYZJkyYwAsvvEBGRgZqtZrXXnuNjh07WioiUL0m\nb0tvZ1p6OzOoeysAisp0pP1ptsrTmSWkZxTz/YHqfQJ8nP9Yk7eNJ34ejlb/Fm4wmli/+zSb9p8F\nYFT/dowZ0E6mURZCXBeLFYUtW7ag0+mIjY0lPj6eefPmsWTJEgD8/PxYuXIlAEeOHGHBggVMmjSJ\n7du3YzAYWLVqFXv27GHhwoW89957lopYIw8Xe3qFtqBXaPVEZJU6A6czi81FIi2zmJ1HM9l5NLP6\n9a725iIR0tqT1i1cGrTff1ZeGR/FJXL2Ygm+HtVLZAa3bl5LZAoh6ofFisLhw4cZNGgQ/H979x8U\ndb3vcfy5AgLbxg+v0rlewSDFX5306Dk3LHQgf6eJouaPJjS7OjmmaeYoZIxKoyk2OVKK5vXqJdNp\nQgWnEazR46Qmec5I5gkmzTVhIX4oCssC7o/P/WPpm17kRwrtrr4ff7G7X7/fF9/BfX8/38/u+wMM\nGjSICxcuNNlGKUVKSgqbNm3Cy8uL8PBw7HY7DocDs9mMt3fr8YKD9XjfRxfPbt3atmZC6H8EM/xv\nzp/tdgfG0mp+MF7jB+N1frh8jX8UlvOPwt/mJfr07EL/8H+jf3gX+oQF4+fbPqf69rxKKY58c4X/\nzv4Xt6x2RvwtlPmT/ozezz36FrX13LoDT8oKnpXXk7KCZ+XtiKwdVhTMZjMGw289+L28vLDZbHe8\n0R87dozevXsTEREBgF6vx2QyMW7cOKqqqkhPT2/1OFVVlnvOeD/dJgN9vRjaN4ShfUNQSlFxs75x\nXsJ52yn/xwryf6wAnL3ye/7J0DiacI4oAu7hU0C3573ZuETm+cYlMv9r/JP8tW8ItTX11NbU39Pv\n1J48qZOnJ2UFz8rrSVnBs/Leb9bmCkqHFQWDwUBtba322OFwNLnyz87OJiEhQXu8e/duoqOjWbZs\nGaWlpcyePZvDhw/j6+vbUTHbhU6nIyTIn5Agf579878DUG25xU/FN7VbTld+qcFYWsPRs0UAPNZF\n3/hRWOctp5Bg/zbPS+RfrOR/jhRQY7HS//FgXh3fn+BH3fscCSE8Q4cVhcGDB3P8+HGef/558vPz\niYyMbLLNhQsXGDx4sPY4ICAAHx/nrY/AwEBsNht2u72jInaoAH1n/hLZjb9EdgOcy1teKa3mx8Yi\n8ZPpJifPl3LyfGnj9j53TF6HhhiaTBLXN9j435xC/p5fgrdXJ2aM6M3Iv/aQZRiFEO2mw4rCqFGj\nOHXqFDNmzEApxbp16zh8+DAWi4Xp06dz/fp1DAbDHVfHc+bMISkpiVmzZmG1Wlm6dCl6vb6jIv6h\nfH286BMWTJ+wYMC5yllxhVkbSVwsvsk/f6zgn423nDr7dOKJ7r8t3O7j3YmMo3mYKmrp0bhEZg9Z\nIlMI0c50SimPXmXmfu+pucv9Q6UU16rrG4uEs1CYKmqbbDfmP0OJH+7+S2S607ltjSdlBc/K60lZ\nwbPyetycgvh9dDodXQP96Rroz9ABfwKci+D8ZHIWifIqC5NietM92M/FSYUQDzIpCm7M4O/DwF5d\nGdjYwM6TrmKEEJ5Jvu4qhBBCI0VBCCGERoqCEEIIjRQFIYQQGikKQgghNFIUhBBCaKQoCCGE0EhR\nEEIIofH4NhdCCCHaj4wUhBBCaKQoCCGE0EhREEIIoZGiIIQQQiNFQQghhEaKghBCCI0UBSGEEJqH\ndpGd7777jk2bNpGRkeHqKC2yWq0kJSVhMpm4desWCxYsYMSIEa6O1Sy73c6qVaswGo3odDrWrFlD\nZGSkq2O16Nq1a8THx7Nr1y6eeOIJV8dp0eTJkzEYnGtz9+jRg/Xr17s4UfO2b9/OsWPHsFqtzJw5\nk2nTprk6UrMOHDjAwYMHAWhoaKCgoIBTp04REBDg4mRNWa1WVq5ciclkolOnTqSkpLTr3+1DWRQ+\n/vhjsrOz8ff3d3WUVmVnZxMUFERqaio3btxg0qRJbl0Ujh8/DsD+/fvJy8vjgw8+YNu2bS5O1Tyr\n1UpycjJ+fu6/zGlDQwNKKbe/kAHIy8vj3Llz7Nu3j7q6Onbt2uXqSC2Kj48nPj4egDVr1jBlyhS3\nLAgAJ06cwGazsX//fk6dOsXmzZtJS0trt/0/lLePwsLC2vUkdqSxY8fyxhtvAKCUwsvLy8WJWjZy\n5EhSUlIAKCkpcdv/WL/asGEDM2bMICQkxNVRWlVYWEhdXR1z584lISGB/Px8V0dq1smTJ4mMjGTh\nwoW89tprxMTEuDpSm3z//fdcunSJ6dOnuzpKs8LDw7Hb7TgcDsxmM97e7Xtt/1COFMaMGUNxcbGr\nY7TJI488AoDZbGbx4sUsWbLExYla5+3tzYoVK/jyyy/ZsmWLq+M068CBA3Tp0oVhw4axY8cOV8dp\nlZ+fH6+++irTpk3jypUrzJs3j5ycnHZ/U2gPVVVVlJSUkJ6eTnFxMQsWLCAnJwedTufqaC3avn07\nCxcudHWMFun1ekwmE+PGjaOqqor09PR23f9DOVLwNKWlpSQkJBAXF8cLL7zg6jhtsmHDBnJzc3nn\nnXewWCyujnNXmZmZnD59mpdffpmCggJWrFhBRUWFq2M1Kzw8nIkTJ6LT6QgPDycoKMht8wYFBREd\nHU3nzp2JiIjA19eX69evuzpWi6qrqzEajURFRbk6Sot2795NdHQ0ubm5ZGVlsXLlShoaGtpt/1IU\n3FxlZSVz585l+fLlTJ061dVxWnXo0CG2b98OgL+/Pzqdjk6d3PPPbO/evXzyySdkZGTQr18/NmzY\nQLdu3Vwdq1mff/457733HgBlZWWYzWa3zTtkyBC+/vprlFKUlZVRV1dHUFCQq2O16OzZswwdOtTV\nMVoVEBDAo48+CkBgYCA2mw273d5u+3e/cae4Q3p6OtXV1WzdupWtW7cCzolyd50YHT16NImJibz0\n0kvYbDaSkpLcNqunmTp1KomJicycOROdTse6devc8tYRQGxsLGfPnmXq1KkopUhOTnb7+TCj0UiP\nHj1cHaNVc+bMISkpiVmzZmG1Wlm6dCl6vb7d9i+ts4UQQmjcc1wvhBDCJaQoCCGE0EhREEIIoZGi\nIIQQQiNFQQghhEaKgnhgFRcX8+STTxIXF0dcXBxjxoxh8eLFVFZW3vM+ExMTMZlMADz33HP3/M34\nkpISxo4dS3x8PGazWXv+/PnzpKam3nM+Ie6XFAXxQAsJCSErK4usrCxycnLo2bMnixcvvuf95eXl\n0R6f4v72228ZMGAABw4c0LqeAly6dIlr167d9/6FuFfu+c0XITqATqdj0aJFPPvssxQWFtK3b192\n7NjBkSNHsNvtREdHs3z5ckwmEwsWLCA0NJSff/6Z7t27k5qaymeffUZ5eTnz589n7969AHz00UcU\nFBRQV1fHxo0bGThw4B3HNBqNJCcnc+PGDfR6PW+//TY+Pj5s3rwZi8VCcnIya9euBZxtFrZs2YLF\nYmHbtm089thjHDx4kBs3bhAbG0tCQgLJycn88ssv6HQ6li1bxjPPPENtbS1r167l4sWL2O125s2b\nx4QJE/7w8yseEEqIB1RRUZGKjY1t8vyUKVPUF198oU6cOKEWLVqkbDabstvt6s0331SHDh1SRUVF\nKjIyUp05c0YppdT69etVSkqKUkqp2NhYVVRUpP28c+dOpZRSGRkZatGiRXc9Vm5urlJKqXPnzqmY\nmBjV0NCgMjMz1YoVK5psf/vzmZmZatSoUcpqtSqllFqyZIn66quvlFJKlZWVqREjRqiamhqVmpqq\n9uzZo5RSqqamRo0fP15dvXr13k+ceKjJSEE8dHQ6HX5+fnzzzTecP39e66NfX19P9+7dGTJkCI8/\n/jhPP/00AJMmTeKtt966675GjhwJQK9evcjNzb3jtdraWq5evcro0aMBGDRoEIGBgVy+fLnNWfv3\n76+1sjh9+jSXL1/WOs/abDaKioo4ffo09fX1ZGZmAmCxWLh48SKhoaFtPo4Qv5KiIB4qt27dwmg0\n0qtXL86cOcPs2bN55ZVXAOftGy8vL6qqqu7oKaRaWMfi1+fv1hJaKdVk/kEp9bual93eN8rhcLBn\nzx6tsVxZWRldu3bF4XCQmprKgAEDAGcTxcDAwDYfQ4jbyUSzeGg4HA7S0tIYOHAgYWFhREVFkZWV\nRW1tLTabjYULF2pX+0ajkYKCAsDZYnv48OGAswi09U3dYDAQGhrK0aNHAcjPz6eyspLevXs3+2+8\nvLyw2Wx3fS0qKopPP/0UcE5IT5w4kbq6OqKioti3bx8A5eXlTJw4kdLS0jZlFOL/k5GCeKCVl5cT\nFxcHOItCv379eP/99wHnR0oLCwt58cUXsdvtDBs2jMmTJ2MymQgMDGTLli1cvXqVPn368O677wIQ\nExPD/Pnz2blzZ5uOn5qayurVq0lLS8PHx4e0tDQ6d+7c7PZPPfUUH374IZs2bSIiIuKO11atWkVy\ncrK2psbGjRsxGAy8/vrrrF69mgkTJmC321m+fDlhYWG/+1wJAdIlVYgmiouLSUhI4NixY66OIsQf\nTm4fCSGE0MhIQQghhEZGCkIIITRSFIQQQmikKAghhNBIURBCCKGRoiCEEELzfyaVd3zAQhgZAAAA\nAElFTkSuQmCC\n",
672 | "text/plain": [
673 | "