├── README.md ├── NLP_C1_W1_lecture_nb_01.ipynb └── NLP_C1_W1_lecture_nb_03.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # Natural-Language-Processing-Specialization 2 | Offered by Deeplearning.ai via Coursera 3 | -------------------------------------------------------------------------------- /NLP_C1_W1_lecture_nb_01.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Preprocessing\n", 8 | "\n", 9 | "In this lab, we will be exploring how to preprocess tweets for sentiment analysis. We will provide a function for preprocessing tweets during this week's assignment, but it is still good to know what is going on under the hood. By the end of this lecture, you will see how to use the [NLTK](http://www.nltk.org) package to perform a preprocessing pipeline for Twitter datasets." 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "metadata": {}, 15 | "source": [ 16 | "## Setup\n", 17 | "\n", 18 | "You will be doing sentiment analysis on tweets in the first two weeks of this course. To help with that, we will be using the [Natural Language Toolkit (NLTK)](http://www.nltk.org/howto/twitter.html) package, an open-source Python library for natural language processing. It has modules for collecting, handling, and processing Twitter data, and you will be acquainted with them as we move along the course.\n", 19 | "\n", 20 | "For this exercise, we will use a Twitter dataset that comes with NLTK. This dataset has been manually annotated and serves to establish baselines for models quickly. Let us import them now as well as a few other libraries we will be using." 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 1, 26 | "metadata": {}, 27 | "outputs": [], 28 | "source": [ 29 | "import nltk # Python library for NLP\n", 30 | "from nltk.corpus import twitter_samples # sample Twitter dataset from NLTK\n", 31 | "import matplotlib.pyplot as plt # library for visualization\n", 32 | "import random # pseudo-random number generator" 33 | ] 34 | }, 35 | { 36 | "cell_type": "markdown", 37 | "metadata": {}, 38 | "source": [ 39 | "## About the Twitter dataset\n", 40 | "\n", 41 | "The sample dataset from NLTK is separated into positive and negative tweets. It contains 5000 positive tweets and 5000 negative tweets exactly. The exact match between these classes is not a coincidence. The intention is to have a balanced dataset. That does not reflect the real distributions of positive and negative classes in live Twitter streams. It is just because balanced datasets simplify the design of most computational methods that are required for sentiment analysis. However, it is better to be aware that this balance of classes is artificial. \n", 42 | "\n", 43 | "The dataset is already downloaded in the Coursera workspace. In a local computer however, you can download the data by doing:" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "execution_count": null, 49 | "metadata": {}, 50 | "outputs": [], 51 | "source": [ 52 | "# downloads sample twitter dataset. uncomment the line below if running on a local machine.\n", 53 | "# nltk.download('twitter_samples')" 54 | ] 55 | }, 56 | { 57 | "cell_type": "markdown", 58 | "metadata": {}, 59 | "source": [ 60 | "We can load the text fields of the positive and negative tweets by using the module's `strings()` method like this:" 61 | ] 62 | }, 63 | { 64 | "cell_type": "code", 65 | "execution_count": 2, 66 | "metadata": {}, 67 | "outputs": [], 68 | "source": [ 69 | "# select the set of positive and negative tweets\n", 70 | "all_positive_tweets = twitter_samples.strings('positive_tweets.json')\n", 71 | "all_negative_tweets = twitter_samples.strings('negative_tweets.json')" 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "Next, we'll print a report with the number of positive and negative tweets. It is also essential to know the data structure of the datasets" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": 5, 84 | "metadata": {}, 85 | "outputs": [ 86 | { 87 | "name": "stdout", 88 | "output_type": "stream", 89 | "text": [ 90 | "Number of positive tweets: 5000\n", 91 | "Number of negative tweets: 5000\n", 92 | "\n", 93 | "The type of all_positive_tweets is: \n", 94 | "The type of a tweet entry is: \n" 95 | ] 96 | } 97 | ], 98 | "source": [ 99 | "print('Number of positive tweets: ', len(all_positive_tweets))\n", 100 | "print('Number of negative tweets: ', len(all_negative_tweets))\n", 101 | "\n", 102 | "print('\\nThe type of all_positive_tweets is: ', type(all_positive_tweets))\n", 103 | "print('The type of a tweet entry is: ', type(all_negative_tweets[0]))" 104 | ] 105 | }, 106 | { 107 | "cell_type": "markdown", 108 | "metadata": {}, 109 | "source": [ 110 | "We can see that the data is stored in a list and as you might expect, individual tweets are stored as strings.\n", 111 | "\n", 112 | "You can make a more visually appealing report by using Matplotlib's [pyplot](https://matplotlib.org/tutorials/introductory/pyplot.html) library. Let us see how to create a [pie chart](https://matplotlib.org/3.2.1/gallery/pie_and_polar_charts/pie_features.html#sphx-glr-gallery-pie-and-polar-charts-pie-features-py) to show the same information as above. This simple snippet will serve you in future visualizations of this kind of data." 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": 9, 118 | "metadata": {}, 119 | "outputs": [ 120 | { 121 | "data": { 122 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAElCAYAAADjk4nIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmYnFWB7/HvqareqrtT2felg6kkkEDYMkVABBURNxARZYZ7h6uDjDBXZ+4wakYdLR0fQB3HZVAUYVTcEBmEhrCDQCTQJCFkIaHThBDIQmfvrbq7lvfcP97qpMna6XT3qar393meejr9dlfVr0L41alz3sVYaxERkeAIuQ4gIiJDS8UvIhIwKn4RkYBR8YuIBIyKX0QkYFT8IiIBo+IXEQkYFb+ISMCo+EVEAkbFLyISMCp+EZGAUfGLiASMil9EJGBU/CIiAaPiFxEJGBW/iEjAqPhFRAJGxS8iEjAR1wFEBlrdwkURYEL+Ng6oxP+33nMrO9FsyjxU8a8VQLbXLZP/2gZsA7aRbNk59K9AZHCp+KVo1C1cFAImAxPxS/3Arz1/Hg2YIz1WGdkmIH7UJ03G0sBbwFZ63gz2/7nn62aSLbv685pEXFDxS0HKl/ws4Iz87UzgVKBmIB7fetnyPv5qOTA1fzu8ZGwLsBxYlv+6nGRL8/FkFBksKn5xbrBL/pDSqRFUDegjTsrfLt63RW8GUqBU/DLk6hYuMsBfAR8BzmOwS/5QQuHsEDzL4d4MlgGPAvUkWzYPQQ6RtzHWWtcZJADqFi6qAt6HX4IfAsa7zHNyZtXu+2tvGukyQ94KoB7/TeBF12EkGFT8MmjqFi4ahz+qvxi4AAZ4cuU4FFDx97YZeAD/jeBJki3djvNIiVLxy4CqW7hoLn7RX4w/nXPEvWtcKdDi762dnukgeEB7DclA0hy/HLe6hYuiwJXAtcBpjuOUihrgY/lbhmTsXuAnJFuecppKSoKKX/qtbuGiWcB11tqrjDEx13lKWBlwOXA5ydha4BbgDpItrW5jSbHSVI8ck/xRsZcA1wHvcRyn34pgqudo2oHfAj8m2bLadRgpLip+6ZO6hYsmANdYaz9jjJnkOs/xKoHi7+1Z4CfA3SRb0q7DSOFT8csR1S1cdA7wT9bajxpjSmZqsMSKv8d24HbgRyRb3nIdRgqXil8OqW7holOstTcaYz7oOstgKNHi79EB/AD4LsmWFtdhpPCo+OVt6hYumm693A2Y0CeMMSV72u4SL/4eu4AbgZt1TID0puIXAOoWLhprvdzXMaHPGGPKXOcZbAEp/h5vAkngVyRbco6zSAEo2RGd9E3dwkW10750/zet9TaaUPi6IJR+AE3Bn/tfRTJ2qesw4l7JLNbJsalbuKjCWnst1n7NhEIjXOeRIXEScA/J2PPAQpItT7sOJG5oxB9A0770wCesl9tgjPm+Sj+QzgKeIhl7iGTs6BejkZKjEX+A1C1cNM7LdP8yVFZxESbsOo64dxGwkmTsK8APSbZ4rgPJ0NCIPyCmfP53V9tc9tVQWcVFrrNIQakC/hN4RqP/4NCIv8RN+5d7x3mZrj+Eo7HzXGeRgnYOGv0Hhkb8JWzK5393NYYN4apalb70hUb/AaERfwnSKF+Ok0b/JU4j/hKjUb4MEI3+S5hG/CVi6j/fXWtz2fvC0di7XWeRktIz+v9nki0/dR1GBoZG/CVgwt/+5+lYb324qlalL4OhCriFZOwWkjEd2V0CVPxFLBpPmHFXfOu68rHTl4Qqqse7ziMl77PA4yRjo10HkeOj4i9S0XgiElvwidsrp8672UTKK1znkcB4F7CUZOwU10Gk/1T8RWjEuz81YsT5n1pcMXH2p0woZFznkcCpA5aQjH3MdRDpHxV/kRn94evnVM95z6qyUVPOcp1FAq0auJtk7OskYxp8FBkVfxEZ+/GvfSwaP+v5SM3Iya6ziAAG/zz/fyQZq3acRY6Bir8IROMJM/6vb/z3quln/CFUEa1xnUfkAJcBz5KMTXMdRPpGxV/govFERWzBJ/9YMfXkr5pwRMddSKGah7/oe4brIHJ0Kv4CFo0nRsYWfOKRiomzLjNG06hS8MYAT5CMLXAdRI5MxV+govHE1NhZlz9WMXG2Tr0gxSQGPEpS54kqZCr+AhSddfYJsbOvWFQx6cTTXWcR6Yca4CGSsfe5DiKHpuIvMNFZZ8djCz75QMWEmXNdZxE5DlXA/SRjH3IdRA6m4i8g0Xhiduysy+srxs840XUWkQFQAfwPydiFroPI26n4C0Q0npg77KzL766YMHO26ywiA6gCuJdk7HzXQWQ/FX8BiMYTc4b91cd+VznpxDmus4gMgp5pn7NdBxGfit+xaDwxq/aMi39dOWXuya6ziAyingXf+a6DiIrfqWg8MaPm5Pf9oqru1NNcZxEZAsOAh3VFL/dU/I5E44npldPm/bAqntDJ1iRIRgL1JGPDXAcJMhW/A9F4YlJk5KRv1Z76gfcYo9MqS+DMBn5HMqb+cUR/8UMsGk/UhiprvhA76/KLTKS80nUeEUc+BNzoOkRQqfiHUDSeiGBCn40t+OSl4aphI13nEXHsiyRjV7oOEUQq/iESjScMcPmwMy/+32UjJ011nUekQNxGMnam6xBBo+IfOu+MxhdcWzn1FO22KbJfJf4BXhNcBwkSFf8QiMYT8fLxM75QPfc9Ol2tyMEmAX8iGatwHSQoVPyDLBpPjA7XjFo4bP6l7zahsC6kInJoCeBW1yGCQsU/iKLxRJWJlP9T7OwrPhgqr9IlE0WO7G9Jxq53HSIIVPyDJBpPhICrak//8MWR2lHjXecRKRI36fKNg0/FP3g+UD5h5sUVk+fovPoifRcBfkEyVu46SClT8Q+CaDxxIuGyK2pP//B8o4vlihyrk4F/cx2ilKn4B1g0nogCVw8785LZ4cqa0a7ziBSphZryGTwq/oF3WfmEmfGKSSfqH61I/2nKZxCp+AdQfornwmGnf3iBpnhEjpumfAaJin+A9J7iCWmKR2SgaMpnEKj4B46meEQGnqZ8BoGKfwBoikdkUGnKZ4Cp+I+TpnhEhoSmfAaQiv/4XVY2pm66pnhEBlUE+DnJmD5RDwAV/3GIxhMnAhfUznv/XE3xiAy604ArXIcoBSr+forGE5XA1RVTTq6MxMbFXecRCYh/Jxkrcx2i2Kn4++88YGTNnPN1jn2RofMO4BrXIYqdir8fovFELXBpVXxBTbh6xBTXeUQC5t9IxqpdhyhmKv7+uRBjyqIzF5znOohIAI0D/p/rEMVMxX+MovHEKOAD1XPeMyZcWTPGdR6RgPoCyZh2n+4nFf+x+xDhMlM1/fR3uQ4iEmDDgC+7DlGsVPzHIBpPTATeXTvv/VNC5VUx13lEAu46krGprkMUIxX/sbnUVFTbyilzz3EdRESoAL7hOkQxUvH3UTSemA7Mrz31A+8wkfKo6zwiAvgXaJ/jOkSxUfH3QTSeMMDlpjyaqZgw8yzXeURknxDwFdchio2Kv29OBE6qPun8iSYc0elhRQrLZSRj41yHKCYq/qPIj/Y/CbRWTJo933UeETlIOXC16xDFRMV/dCcAUyvrTqvVfvsiBesakrGw6xDFQsV/dO8BuqtOOEOjfZHCNRX4sOsQxULFfwTReCIGnBWuHZOKDB9/ous8InJE17kOUCxU/EeWAEz1ieeeakxIf1cihe19JGMzXIcoBiqzw4jGExHgA5jQrvLxM3R1LZHCZ4BrXYcoBir+wzsRGB6defaUUFnlMNdhRKRP/g/JWJXrEIVOxX94FwKpymmnaFFXpHiMRJdnPCoV/yFE44lxwNyyMXVeuGbUCa7ziMgx0SLvUaj4D+2dgBeNLzhD11AXKTpnkoxpXe4IVPwHiMYTFcAFQHPZqMknuc4jIv3ycdcBCpmK/2DzgMryCTNH6Jz7IkXrI64DFDIV/8HOB9oqp8yd5TqIiPTbHJIxrc8dhoq/l2g8UQPMBvaUjZqi4hcpbhe7DlCoVPxvFwcIDxtbHaoaNtF1GBE5Lir+w1Dxv92ZQHfV9NNmaW8ekaJ3LsnYcNchCpGKPy8aT5QBZwC7ysdM1zSPSPGLAB90HaIQqfj3mw6UmfIqE64dNd11GBEZEJruOQQV/36nAF7V9DPeYULhiOswIjIgLiIZK3MdotCo+Nl3ecUFwO7y8TM0zSNSOmLAea5DFBoVv28iMAJjuiLDx8ddhxGRAaWDuQ6g4vedCFAx6aQJoUh5teswIjKgtMB7ABW/7xygpWz0tEmug4jIgJtBMjbSdYhCEvjij8YTw4FpQGskNlYHbYmUptNdBygkgS9+/NK3gA1Xj1Dxi5SmM10HKCQqfpgKWFNWGQlV1oxxHUZEBoXOz9+Lit8/KVtH+fj4eKPzNIiUKhV/L4Eu/mg8EQJOANrKRk2Z4DqPiAya6Vrg3S/QxQ+MAsqBrBZ2RUqeFnjzgl78+8peC7siJU/TPXlBL34t7IoEh4o/L+jFr4VdkeBQ8ecFtvi1sCsSOCeQjI1wHaIQBLb4edvC7hgVv0gwnOY6QCEIcvHvW8wNVVTr8mwiwTDNdYBCEOTin4p/qgZCZZW1jrOIyNDQ3nsEu/inAykAU1ZR4ziLiAwNTesS7OIfDaRNWWXEhMsqXYcRkSGhET/BLv4RQHdk2BhN84gEh0b8BLT4o/FEBKgGMuGakSp+keDQiJ+AFj9QQ8/CbjSm+X2R4BjvOkAhCGrx15Iv/nDVMI34RYKjnGRstOsQrgW1+PeN8kMV1Sp+kWAJ/Dx/UIu/FjAAoYqoil8kWAI/zx/U4o+RL35TXqU5fpFg0YjfdQBHRgMZAKOjdkWCRiN+1wEcGQ10A5hwRAdviQRL4M/QGdTiHwWkAYwJhR1nEZGhFXEdwLWj/gUYY3LA6vzvrgOustamjuVJjDG3Af9prV1rjPmytfaGXj9bYq09+xhzH68RQEc+QcG9+W2+5dOEyqsgFMKEwky46gfkOtvYed+3ybY2Exk2jtEfXUi48uDlic7XlrP7iVvB86iZdyGxsy4HYM9Tv6DzteWUj53O6A9fD0D7mifxutoYduYlQ/r6xK26H7RRW2EIG4iEYNk1NezutHzy7hSv77XUDTfc9fEoI6oOvjbRw69m+ceHu8h5lqtPL2fhOysA+NJjXTz0apZTx4e549IqAH69Ms3uTss/nlUxpK+vD8r6cydjjMXvsevz3/8LUGOtTQ5gNoaiI/tSep3W2lOttXPxR8mfPdYnsdZeba1dm//2ywf8bKhLH/zz8HsAmMIrfoBxf30DEz/1X0y46gcAtD7/Ryrr5jHpmp9TWTeP1uf/eNB9rJdj92O3MPbybzDx6p/QsfZp0jvfwOvuoHvLOiZ++mas9UjveB0v003HmsepPe1DQ/3SpAD8+aooL322hmXX+IOHm/7SzXunR2j6XA3vnR7hpr90H3SfnGf5hwc7eejKKGv/oYbfr8mwdkeOli7Lks05Vl1bQ85aVjfn6MxYfrkyw3Xzy4f6pfVFf0f83cDHjDGDfRzAoHfksZbeYmAGgDHmn40xa/K3f8pvqzbGLDLGrMxv/2R++1PGmDONMTcBVcaYl4wxv83/rD3/9Q/GmA/2PJEx5pfGmMuMMWFjzHeNMUuNMauMMX+f//kEY8wz+cdaY4w59xhft80/UUEW/4FSrzZQPfe9AFTPfS+ppucP+p30tvVEhk+gbPh4TLiM6hPfRWfT84DB5rJYa7HZNCYUpvWFe6g942JMOPCfegW4rzHLVfP8gfBV88q4tzF70O+8sCXHjJEhThgRojxsuGJOGfe9kiVkIJ2zWGvpzEBZGL67JM3n/6qcsnBBXtG0v//os8CtwP878AfGmDHGmP/J99RSY8w5vbY/Zox50RjzM2PMpp43DmPMvcaY5caYl40x1+S3DVhHHkmfS88YEwE+AKw2xpwBfApIAGcBnzHGnAZcBGy11s7Lf0J4uPdjWGsXsv8TxJUHPMWdQM8bRTnwXuBB4O+AFmvtfGB+/rmmA38DPGKtPRWYB7zU19cChCnkEb8xbL/ra2z75T/S9pL/V5jr2EukZiQAkZqReB17D7pbtm0XkWH7rxkfrh1Nrn0XoYoo0Vlns+2XnycSG4epqCa9bT3R+FlD83qkoBgDF/46xRm3tnPr8jQAze0eE2r9/xUm1IbY3uEddL8tbZYpw/b/7zJ5mGFLm0dtheGyE8s47WcdTB8eIlZhWLo1xyWz+zWjMhSOJ9iPgSuNMbEDtv8Q+H6+py4Dbstv/zrwpLX2dOBP+NcB6fFpa+0ZwJnA540xowa4Iw+rL+98VcaYnlJdDNwOXAv8yVrbkQ9xD3AuftH/hzHm28AD1trFfXj8Hg8BPzLGVOC/gTxjre00xlwInGKM+Xj+92JAHFgK/Lcxpgy411p7LMW/f8RP4V1kffyV3yFSO4pcx16a//BVykZNPo5H819eLPFxYgn/r3DXQz9i+Ln/i7aVj9C1cQVlY+sYfvYVA5BcisGzn65mYr7c3/frFLNH923sY+3B23r+5/niORV88Rx/Lv/q+k6+eX4Ft72Y5tENWU4ZF+ar7yqoef5+D/asta3GmDuAzwOdvX50AXCS2V8nw4wxtcA7gUvz933YGLOn130+b4y5NP/nKfi9tusIT3+sHbnxcA90LHP8p1prP2etTbP/v/fbWGvX41/JfjVwozHma314/J77dgFPAe/Hf1e7M/8jA3yuV4bp1tpHrbXPAO8CtgC/Nsb8bV+eJ3+R9V75D/XP2a1I7SgAwtXDic5cQPfW9YSrh5Nt3w1Atn03oUNcLTJSO4ps64593+fadhLOf0rokW7e4P/uiEl0rHmSMR9dSGbHJjK7twzWy5ECMzE/sh9bHeLS2RFe2JJjXE2IbW3+KH9bm8fY6oOrYfIww5ut+z8JbG61+x6rx4ptOQBmjgpxx8oMd10eZc32HE27coP1cvrjeMP8AH+UXd1rWwhY0KunJllr2zhMVxpjzsd/s1hgrZ0HrACOuGv5sXbkkR6rv+98zwAfNcZEjTHV+O9oi40xE4GUtfY3wH8Apx/ivpn8KP1Q7sSfQjoXeCS/7RHg2p77GGNm5tcSpgHbrbU/x/8UcqjnOpS3v2ZrD/5M65CX7sLrTu37c9fGFZSPmUZ0RoKONU8A0LHmCaIzEgfdt3zCTLJ7tpLZ+xY2l6Fj3TNUHfB7exf/htg7rwQvCz0v3YSw2YMX86T0dKQtbd12358f3ZBj7tgwF8+M8KuVGQB+tTLDJbMOngyYPylM0y6PjXs80jnLnS9nuPiA3/u3P3fzzXdXkPEglx9ShQykMoP7uo7RcaWx1u4G7sIv/x6PAv+35xtjzKn5P/4F+ER+24XsP4YgBuyx1qaMMbPxp8z35RuIjjzSa+jXIoe19kVjzC+BF/KbbrPWrjDGvB/4rjHGw//LvfYQd78VWGWMefEQc1iPAncA9flPFuDPldUBLxr/c9QO4KPA+cAXjDEZoB3o04iffVM8+15MQRV/LrWXHfd8y//G86g+6TyqTjiD8glxdt53E+2rHiUybAyjL/lXwJ/X3/Xwjxh3+TcwoTAj3/dZtt/1NbAeNSe/j/Ix+68tnVr/HOXj4/s+UVRMnM3W2/+BsrF1lI89Ychfqwy95g7LpX/wBxZZD/5mbhkXzYgwf2KIT9zdye0rMkyNGf54eRSArW0eV9d38eCVUSIhw80frOT9v0mRs5ZPn1rOnLH7D4O595UM8yeG930KWDA5zMm3tHPKuBDzxhfU4TIHr1wfu+/Rq+jxp35+bIxZhd+rz+DvAfkN4Pf5HV2eBrYBbfjT4p/N/34j0HtvjYHqyMMytvBmOgZVNJ4wwH8DmwDGXPKvXzSRsiq3qWSonZxZtfv+2ptGHv03pQTdTLLlc0PxRPn5+Jy1NmuMWQDckt8hxanA7cuXamqw0XjC4s+LWYv1Cm51V0QG01BOPE0F7jL+3oNp4DND+NyHFbjiz/PIFz/WK6hVJxEZdENW/NbaJuC0oXq+viq8fdiHRo78arvNpjuP8rsiUlp2ug7gmoo/3dXmOIuIDK2trgO4FtTi78Y/ehcv06niFwmWba4DuBbU4t+Df6I2vO6Uil8kWDTidx3AkZ30FH9Xe7vjLCIytDTidx3AkV1ABYDX2aYRv0hwpEi2tLgO4VpQi38H+TP05VJ7VfwiwRH40T4Et/hbyZ+6Ide2S8UvEhyBn9+H4BZ/O/nz8WfbdrQH7bQVIgGmET/BLf79o/xc1iOXOaZrCItI0dKIn2AX/75T9HiZbk33iASDRvwEt/g78F+7f/RuRkfvigSERvwEtPhTTQ05/AXeMgCvs23Pke8hIiVig+sAhSCQxZ+3m/xBXNmWZo0CREpfDljpOkQhCHLx7yR/EFd65ybN+4mUvnUkW7QjB8Eu/g1AFCDdvGGH9XIDcTk2ESlcy10HKBRBLv436bn+rpfzvM62t9zGEZFBpuLPC3Lxb6PX68+179Z0j0hpU/HnBbn49wAp8nv2aIFXpKTlgJdchygUgS3+VFODxZ/nrwUt8IqUOC3s9hLY4s9bB1SDFnhFSpymeXoJevFrgVckGFT8vQS9+LXAKxIMKv5egl78WuAVKX1a2D1AoIv/wAXe7uYNm90mEpFBsEoLu28X6OLP27fAm9n+2k6vO7XbcR4RGVgPuA5QaFT8/gLvPpk9W9a7CiIig6LedYBCo+KHTfjn5TcA3VsbG93GEZEBtAUt7B4k8MWfampoB9YDwwG6Nq18w2YznW5TicgAeYBkiy6qfYDAF3/eEvILvHg5L9vS/KrbOCIyQDTNcwgqft8rvb9JN2/QdI9I8esAnnAdohCp+H07gGbye/d0bnzxVWs9z20kETlOj5Js6XYdohCp+Nm3P/8SYASA19XWnWvf/brTUCJyvO53HaBQqfj3W0N+zx6AzI5Nmu4RKV4e2n//sFT8+70BdJK/AHvn6y+p+EWK1/MkW3a4DlGoVPx5qaaGHNAAjALI7tnSkutsa3abSkT6SXvzHIGK/+1WkD9hG0Bmx8Y1DrOISD9Yaz3gLtc5CpmK/+1exZ8bDAN0vLJ4hfW8nNtIInIsjDGPkGzZ6DpHIVPx95JqaugCVgMjAXJtuzqye7etc5tKRI7RT1wHKHQq/oM9BUR7vul8bdlSd1FE5FhYa18HHnSdo9Cp+A/2MtAKVIF/7h4t8ooUB2PMz0i26ODLo1DxHyDV1JAFHgZG92zr3rJumbtEItIX1tpu4HbXOYqBiv/Qns9/DQF0vLJ4lc1ldei3SAEzxtytfff7RsV/CKmmhj3AUmAsgO3uSGd2vrHKbSoROQot6vaRiv/wngQqer5JNT2nRV6RAmWtfYlkyxLXOYqFiv/wXsU/Y2ctQLp5w45s++5NbiOJyKEYY25xnaGYqPgPI9XU4OHvFjaiZ1v3G6s16hcpMNbaFuC3rnMUExX/kS0HskAEoGP9s+u8TFeb20gi0psx5hckWzpc5ygmKv4jSDU1dABPk1/kJZf1ujatetppKBHZx1qbAm5ynaPYqPiP7hl6nbitffVjK7zujl0O84hInjHm+yRbdIDlMVLxH91mYC09o34v56U2LH3SaSIRIefZvcB3XOcoRir+o8hflvFu/PP3GIDUumfW5lItW50GEwk4Y/gWyZZW1zmKkYq/D1JNDRuBF4Bx+7Y1PvuEu0QiwZb17LaQMTe7zlGsVPx99yf8A7rCAJ2vLXst27ZT5/wWcSBk+ArJFp1GpZ9U/H2UamrYBvwZGN+zrWPtU49ba92FEgmgTM42hYy5w3WOYqbiPzYP4M/zlwF0b167Nbt321q3kUSCpSxsvkiyRVfGOw4q/mOQamrYDTxEr1F/++rHn7TW0/m/RYZAJmeXkWy513WOYqfiP3aPAhnyJ3DL7Hh9V2bnGyvcRhIJhrKwud51hlKg4j9GqaaGNvyF3v2j/pWPPG1z2bS7VCKlL5OzD5FsecZ1jlKg4u+fp4E28tfmzbY0t3VufPFRt5FESlfWsx1lYfMZ1zlKhYq/H1JNDV3AXfQczQu0r3x4ebZt52vuUomUrq4s15Ns2eI6R6lQ8fdfA/Amva7N27r0vnpN+YgMrD2dtqHmhtafuc5RSlT8/ZRqasgAtwHV5E/bnN2zpUVTPiIDJ52zXWVhPu46R6lR8R+HVFPDJvyF3sk92zTlIzJw9nbZL9Xc0LrZdY5So+I/fg+jKR+RAben0zaM/W7bj1znKEUq/uOUampIoykfkQGlKZ7BpeIfAJryERlYmuIZXCr+gaMpH5EBoCmewafiHyCHn/JZrikfkT5K52xXeZjLXecodSr+AXToKZ9Hlmd2bV7lLpVIcfCstW+1209X39D6pusspU7FP/AOmvLZ++xv79elGkWObP0u79ap32/7vescQaDiH2C9pnyiQCWAzXRnW5bceaeX6W53Gk6kQL2621vxxce6r3OdIyhU/IMgP+VzOzCR/KUasy3NbW0vPnCn9XJZp+FECkxzu/fWfa9kLqpvzOi6FkNExT94lgAPAlN7NnRvfnlLqun5+91FEiksHWnb+efXcx+5/tGu7a6zBImKf5CkmhoscDewCpjUs71jzROrurc1PecsmEiByHrWe3pT9tor7k4tc50laFT8gyjV1JAFbgX20Guxt+W5Ox/Ltmx/1VkwkQLw3Ju5H37wt6lfuc4RRCr+QZa/YtcPgXL8ffzBWrv32d/e7XV17HKZTcSV1c25J767JP0vrnMElYp/CKSaGrYA/4V/4ZYyAK+zrbvlhXt+b3OZLqfhRIbYGy3eht+vyVyixVx3VPxDJNXUsBq4E5gCGIDMjo272lc/cbe1nv4HkEDYlfL2PLYh+/4bFnd3uM4SZCr+ofUIsJhee/p0bnhhQ8fap1X+UvL2dNrWu9dmP/J39Z0bXGcJOhX/EEo1NXjAHcBGYPy+7a8sXpd65dk/WWuts3Aig2hvl23/+Yvpv/v7BzqfdZ1FVPxDLtXU0A3cDLQDY3q2d6z985rOpufr1f1Salq7beq/GtJf/ssbuf9xnUV8Kn4HUk0Nu4HvAGl67ebZvvqxlzo3LF3kLJjIAGtP286bX0h/benW3I/rGzMa1RQIFb8o13+QAAAKW0lEQVQjqaaG7fjl7wGjera3r3x4WWrD0kUa+Uuxa+u2qZtfSP/785tzP9AePIVFxe9QqqlhG375h4ARPdvbX3poWX7aR+0vRamly3Z877nuG5a8mftOfWMm5zqPvJ2K37FUU8Nm4Nv4+/fvL//Vj61INWrBV4rPnk7b9u1nu296cZv3bZV+YVLxF4BUU8Mb+OUfAUb2bO94+cnVHWuf+qN29ZRisSvltd6wuPuGNdu9b9c3ZnQm2gKl4i8Q+VM534h/cNe+Bd/UK4vXta9+4k5du1cK3bY2b9e3nkl/o3GX9x/1jZmM6zxyeCr+ApKf9rkByNJrV8/OpueaWp6/63Yv3bnXWTiRI1jdnHv9i491Ldywx/uhRvqFT8VfYPILvjcCXcC4nu3pt17dvufJ227Ntu163VU2kQN51toHmzIrvvJk91dbuvlvzekXBxV/AUo1NTTjl/9Oep3bJ9exp3P34z/9dXr7a0td5hMBSOds+qfLMk/9dFnmm8Dvtctm8VDxF6hUU8NO/PJ/EajDX/gFL+ftXfybB1OvvvCAFn3Flb1dtuXrf+6+/+FXs1+qb8zcq9IvLir+ApZqakgBPwXuwT+xW7TnZ+0rH17e9uKiO2w2nXKVT4Lp9b3elusf6frdyzu86+sbM/r0WYRU/AUu1dSQSzU13Id/MZeR9Nrds+v1FZv2PPPrW3Odrc3OAkqgPPdmdt31j3T9dEfKfrm+MbPJdR7pHxV/kUg1NSwHvom/x8/Enu3ZPVtadj9+6+2ZPdvWOQsnJS/n2dzvVqefu/Ev6e9kPL5T35jRHmZFTMVfRPIHen0D/7TOdeT/+9l0KrPnyZ/f1blp5ePW87RXhQyoli6799vPph+5c03234Bf1TdmdExJkTM6I0DxicYT5cAVwAXAZvyzfAJQPm7GmNrTP/TRcDQ28XD3Fzg5s2r3/bU3jTz6bwaXtZYXtuRW/+D5dENHhu/XN2bWus4kA0PFX6Si8YQBzgeuAvYALft+aEJm2JkXn1Mxee75JhQKu0lY2FT8R9babff8dFn6ub+8kVsJ/Ki+MfOW60wycFT8RS4aT8wE/h7/BG9bgH1TPRr9H56K/9CstSzb6q343nPd61IZngTurG/M6Pq4JUbFXwKi8UQVcCnwfmA3vUf/oXBo2BkXn1Mxec55Gv3vp+I/WGu33XPr8vSSZzblXgNuA1br4imlScVfQqLxxCzgGjT6PyoV/37WWpZv81763pLudR0ZngD+UN+YaXedSwaPir/EaPTfNyp+n0b5waTiL1FHHv2/Y0zNyRdcEImNm+kqn2tBL/5Mzmae25xbfsvS9EaN8oNHxV/Cjjj6ByqnzZtaPftdF4RrRkxxkc+loBZ/zrPeqmZv+S3L0uvfare70Sg/kFT8AZAf/X8G/6Lu2+i13z9AdObZM6viifeGK2vHusjnQtCK31rL+l3e6luXZ5Y37fZCwDNolB9YKv6AyI/+LwA+AoTx3wD2XzDDGFM9572nVE0/7d2h8qqYm5RDJ0jF/0aL9+ovX0o/t2yrZ4FNwJ3AOo3yg0vFHzDReGI4cBFwIX7xvwXsO6WuiZSHa065cH7llLnnmkh59DAPU/SCUPzN7d7mO9dk/vLExlw3/rUdfg+8pIuliIo/oKLxxFjgYuAc/Kt9bQf2/WMIVdaU18y76OyKCTMXmHCk3FHMQVPKxb+3y+6895XM0/esy7YB7cAfged0HVzpoeIPuGg8MRW4DDgVaMMfGe4TrhkZjc4+97SKCTPPDJVXDXeRcTCUWvFba9nSZjc+uTG7/E/rsm05Swb4E/B0fWOm03U+KSwqfuk5708c+CTwDg489w+AMaZqxlkzqqbNmx8eNmaGMcYMfdKBUyrF3521XS/v8Fbesy6zfFWzV5Hf/DDwaH1jptVlNilcKn7ZJxpPhICT8c/8OQF/mmAXvaaAAMpGTh4enXXOmeVjp59WrOsAxV78O1PeW0vezC29c03mlfY0o/Cvy/w0sKi+MbPzKHeXgFPxy0Gi8UQYOAl/AXgu/sFf2zlgN1ATKQ9HZ587p3LynPnh6uGThz5p/xVj8ec8m1u/y3v5wabs0qc35dqBGP76zGPAs/WNGV2JTfpExS9HFI0nxuMvAF8AVOJPAR109aWKibPHV81IzC8bOWluMSwGF1Pxt3bb3cu25l78w5rMqm3tthYow98tcxGwqr4x0+02oRQbFb/0STSeqAROAT6Ef+H3DNBMr1NBAJhwWbiy7rS6igkzZ0VGTJhZqMcEFHLxe9baHR12y7qdXuPTr2cbl2/zOvGvtewBi/GndDZpP3zpLxW/HJP8QvA0/IvAvBP/8o+78dcDDlI+cdb4yslzZ5WNmjwrVDVsQqGsCRda8Wc9m3mzxb62qjnX+OiG7Po3W20nMBqowt/T6iFgaX1jps1pUCkJKn7pt2g8UQvMxz8gbDT+InBr/nbQP6xIbHxtZd2ps8rH1M0M146abkLhyJAG7qUQij+Vse0bdnvrl23NNT6yIftaKoPBP61GOf7f30vA40BjfWPGO9JjiRwLFb8ct/yngPHAicDZwAn5H3Xifxo46EhRUx4tq5p++jvKRk+dFq4ZOTFcNWz8UK4NuCj+jrRt3d5ht73Z6m19YUtuw+JNuS0WovhnUA3hL54vBZYDTfWNmdRQ5pPgUPHLgIvGEzFgJpDAXxcI45f/LuDQC5HGmLIx00eVj6mbGBk+fsJgvxkMdvH3LvmmXd625dtyWze32g783S6H5W8G/41xCbAKeL2+MZM9/KOKDAwVvwyqaDxRgf8J4FRgAVCT/1EK/0jhw++Rcqg3g8qa0YTLose7VjBQxZ/zrJfK0LozZZsPUfLgl3s10LM3DsBr+GW/DnhLi7Qy1FT8MmTyB4hNxv80MBuYgT/ytfgFefQ3A4BwJBSpHVMTrh1VG44Orw1V1daGKmtqQuXR2lB5Va0pr6wxZZW1JlxWfbg3iKMVf86zuc4s7R1p29aepq2127a1dNv2PZ22bWfKtjV32PY3W7y2t9ptqtf/QQeWfM+PtgCN+dt6HVErrqn4xZn82sAwYCIwiSO/GaQ5xILxEYXCoXDNqGgoUh4hFA5V0jVumrd5TmW05tlaOjkvstpkPbysZ72sh5fOkct6eC1dNt3cYY92fpsQ/vz84Ur+VfxTX79V35hJH/IRRBxR8UtBOcybwTvwj1K17C/YEP4bQwb/E0I6/zXDYd4gqumcMNu8cfoI07HoCBHC+HvVlAMV+a89axTkn9Ow/5TWKnkpOip+KQr500j0TKPU4q8VDAPG4O9KOjJ/6zlg7KDdH6voqp1rXq+LmdTqwzxNCP+NYy/+QvROYAf+Sevaet3agS7NzUuxUvFLScmvI9Tgj9RD+Vu458+nmA2VY01LJ/4bQ88tl//aAXSr0KXUqfhFRAIm5DqAiIgMLRW/iEjAqPhFRAJGxS8iEjAqfhGRgFHxi4gEjIpfRCRgVPwiIgGj4hcRCRgVv4hIwKj4RUQCRsUvIhIwKn4RkYBR8YuIBIyKX0QkYP4/CI7txwrs5u4AAAAASUVORK5CYII=\n", 123 | "text/plain": [ 124 | "
" 125 | ] 126 | }, 127 | "metadata": {}, 128 | "output_type": "display_data" 129 | } 130 | ], 131 | "source": [ 132 | "# Declare a figure with a custom size\n", 133 | "fig = plt.figure(figsize=(5, 5))\n", 134 | "\n", 135 | "# labels for the two classes\n", 136 | "labels = 'Positives', 'Negative'\n", 137 | "\n", 138 | "# Sizes for each slide\n", 139 | "sizes = [len(all_positive_tweets), len(all_negative_tweets)] \n", 140 | "\n", 141 | "# Declare pie chart, where the slices will be ordered and plotted counter-clockwise:\n", 142 | "plt.pie(sizes, labels=labels, autopct='%1.1f%%',\n", 143 | " shadow=True, startangle=90)\n", 144 | "\n", 145 | "# Equal aspect ratio ensures that pie is drawn as a circle.\n", 146 | "plt.axis('equal') \n", 147 | "\n", 148 | "# Display the chart\n", 149 | "plt.show();" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "metadata": {}, 155 | "source": [ 156 | "## Looking at raw texts\n", 157 | "\n", 158 | "Before anything else, we can print a couple of tweets from the dataset to see how they look. Understanding the data is responsible for 80% of the success or failure in data science projects. We can use this time to observe aspects we'd like to consider when preprocessing our data.\n", 159 | "\n", 160 | "Below, you will print one random positive and one random negative tweet. We have added a color mark at the beginning of the string to further distinguish the two." 161 | ] 162 | }, 163 | { 164 | "cell_type": "code", 165 | "execution_count": 10, 166 | "metadata": {}, 167 | "outputs": [ 168 | { 169 | "name": "stdout", 170 | "output_type": "stream", 171 | "text": [ 172 | "\u001b[92m2 plans for the day down the drain great :):\n", 173 | "\u001b[91mi am so :((((((\n" 174 | ] 175 | } 176 | ], 177 | "source": [ 178 | "# print positive in greeen\n", 179 | "print('\\033[92m' + all_positive_tweets[random.randint(0,5000)])\n", 180 | "\n", 181 | "# print negative in red\n", 182 | "print('\\033[91m' + all_negative_tweets[random.randint(0,5000)])" 183 | ] 184 | }, 185 | { 186 | "cell_type": "markdown", 187 | "metadata": {}, 188 | "source": [ 189 | "One observation you may have is the presence of [emoticons](https://en.wikipedia.org/wiki/Emoticon) and URLs in many of the tweets. This info will come in handy in the next steps." 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "## Preprocess raw text for Sentiment analysis" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "Data preprocessing is one of the critical steps in any machine learning project. It includes cleaning and formatting the data before feeding into a machine learning algorithm. For NLP, the preprocessing steps are comprised of the following tasks:\n", 204 | "\n", 205 | "* Tokenizing the string\n", 206 | "* Lowercasing\n", 207 | "* Removing stop words and punctuation\n", 208 | "* Stemming\n", 209 | "\n", 210 | "The videos explained each of these steps and why they are important. Let's see how we can do these to a given tweet. We will choose just one and see how this is transformed by each preprocessing step." 211 | ] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "execution_count": 11, 216 | "metadata": {}, 217 | "outputs": [ 218 | { 219 | "name": "stdout", 220 | "output_type": "stream", 221 | "text": [ 222 | "My beautiful sunflowers on a sunny Friday morning off :) #sunflowers #favourites #happy #Friday off… https://t.co/3tfYom0N1i\n" 223 | ] 224 | } 225 | ], 226 | "source": [ 227 | "# Our selected sample. Complex enough to exemplify each step\n", 228 | "tweet = all_positive_tweets[2277]\n", 229 | "print(tweet)" 230 | ] 231 | }, 232 | { 233 | "cell_type": "markdown", 234 | "metadata": {}, 235 | "source": [ 236 | "Let's import a few more libraries for this purpose." 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "execution_count": 12, 242 | "metadata": {}, 243 | "outputs": [ 244 | { 245 | "name": "stderr", 246 | "output_type": "stream", 247 | "text": [ 248 | "[nltk_data] Downloading package stopwords to /home/jovyan/nltk_data...\n", 249 | "[nltk_data] Unzipping corpora/stopwords.zip.\n" 250 | ] 251 | }, 252 | { 253 | "data": { 254 | "text/plain": [ 255 | "True" 256 | ] 257 | }, 258 | "execution_count": 12, 259 | "metadata": {}, 260 | "output_type": "execute_result" 261 | } 262 | ], 263 | "source": [ 264 | "# download the stopwords from NLTK\n", 265 | "nltk.download('stopwords')" 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": 13, 271 | "metadata": {}, 272 | "outputs": [], 273 | "source": [ 274 | "import re # library for regular expression operations\n", 275 | "import string # for string operations\n", 276 | "\n", 277 | "from nltk.corpus import stopwords # module for stop words that come with NLTK\n", 278 | "from nltk.stem import PorterStemmer # module for stemming\n", 279 | "from nltk.tokenize import TweetTokenizer # module for tokenizing strings" 280 | ] 281 | }, 282 | { 283 | "cell_type": "markdown", 284 | "metadata": {}, 285 | "source": [ 286 | "### Remove hyperlinks, Twitter marks and styles\n", 287 | "\n", 288 | "Since we have a Twitter dataset, we'd like to remove some substrings commonly used on the platform like the hashtag, retweet marks, and hyperlinks. We'll use the [re](https://docs.python.org/3/library/re.html) library to perform regular expression operations on our tweet. We'll define our search pattern and use the `sub()` method to remove matches by substituting with an empty character (i.e. `''`)" 289 | ] 290 | }, 291 | { 292 | "cell_type": "code", 293 | "execution_count": 14, 294 | "metadata": {}, 295 | "outputs": [ 296 | { 297 | "name": "stdout", 298 | "output_type": "stream", 299 | "text": [ 300 | "\u001b[92mMy beautiful sunflowers on a sunny Friday morning off :) #sunflowers #favourites #happy #Friday off… https://t.co/3tfYom0N1i\n", 301 | "\u001b[94m\n", 302 | "My beautiful sunflowers on a sunny Friday morning off :) sunflowers favourites happy Friday off… \n" 303 | ] 304 | } 305 | ], 306 | "source": [ 307 | "print('\\033[92m' + tweet)\n", 308 | "print('\\033[94m')\n", 309 | "\n", 310 | "# remove old style retweet text \"RT\"\n", 311 | "tweet2 = re.sub(r'^RT[\\s]+', '', tweet)\n", 312 | "\n", 313 | "# remove hyperlinks\n", 314 | "tweet2 = re.sub(r'https?:\\/\\/.*[\\r\\n]*', '', tweet2)\n", 315 | "\n", 316 | "# remove hashtags\n", 317 | "# only removing the hash # sign from the word\n", 318 | "tweet2 = re.sub(r'#', '', tweet2)\n", 319 | "\n", 320 | "print(tweet2)" 321 | ] 322 | }, 323 | { 324 | "cell_type": "markdown", 325 | "metadata": {}, 326 | "source": [ 327 | "### Tokenize the string\n", 328 | "\n", 329 | "To tokenize means to split the strings into individual words without blanks or tabs. In this same step, we will also convert each word in the string to lower case. The [tokenize](https://www.nltk.org/api/nltk.tokenize.html#module-nltk.tokenize.casual) module from NLTK allows us to do these easily:" 330 | ] 331 | }, 332 | { 333 | "cell_type": "code", 334 | "execution_count": 15, 335 | "metadata": {}, 336 | "outputs": [ 337 | { 338 | "name": "stdout", 339 | "output_type": "stream", 340 | "text": [ 341 | "\n", 342 | "\u001b[92mMy beautiful sunflowers on a sunny Friday morning off :) sunflowers favourites happy Friday off… \n", 343 | "\u001b[94m\n", 344 | "\n", 345 | "Tokenized string:\n", 346 | "['my', 'beautiful', 'sunflowers', 'on', 'a', 'sunny', 'friday', 'morning', 'off', ':)', 'sunflowers', 'favourites', 'happy', 'friday', 'off', '…']\n" 347 | ] 348 | } 349 | ], 350 | "source": [ 351 | "print()\n", 352 | "print('\\033[92m' + tweet2)\n", 353 | "print('\\033[94m')\n", 354 | "\n", 355 | "# instantiate tokenizer class\n", 356 | "tokenizer = TweetTokenizer(preserve_case=False, strip_handles=True,\n", 357 | " reduce_len=True)\n", 358 | "\n", 359 | "# tokenize tweets\n", 360 | "tweet_tokens = tokenizer.tokenize(tweet2)\n", 361 | "\n", 362 | "print()\n", 363 | "print('Tokenized string:')\n", 364 | "print(tweet_tokens)" 365 | ] 366 | }, 367 | { 368 | "cell_type": "markdown", 369 | "metadata": {}, 370 | "source": [ 371 | "### Remove stop words and punctuations\n", 372 | "\n", 373 | "The next step is to remove stop words and punctuation. Stop words are words that don't add significant meaning to the text. You'll see the list provided by NLTK when you run the cells below." 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "execution_count": 16, 379 | "metadata": {}, 380 | "outputs": [ 381 | { 382 | "name": "stdout", 383 | "output_type": "stream", 384 | "text": [ 385 | "Stop words\n", 386 | "\n", 387 | "['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you', \"you're\", \"you've\", \"you'll\", \"you'd\", 'your', 'yours', 'yourself', 'yourselves', 'he', 'him', 'his', 'himself', 'she', \"she's\", 'her', 'hers', 'herself', 'it', \"it's\", 'its', 'itself', 'they', 'them', 'their', 'theirs', 'themselves', 'what', 'which', 'who', 'whom', 'this', 'that', \"that'll\", 'these', 'those', 'am', 'is', 'are', 'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had', 'having', 'do', 'does', 'did', 'doing', 'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until', 'while', 'of', 'at', 'by', 'for', 'with', 'about', 'against', 'between', 'into', 'through', 'during', 'before', 'after', 'above', 'below', 'to', 'from', 'up', 'down', 'in', 'out', 'on', 'off', 'over', 'under', 'again', 'further', 'then', 'once', 'here', 'there', 'when', 'where', 'why', 'how', 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other', 'some', 'such', 'no', 'nor', 'not', 'only', 'own', 'same', 'so', 'than', 'too', 'very', 's', 't', 'can', 'will', 'just', 'don', \"don't\", 'should', \"should've\", 'now', 'd', 'll', 'm', 'o', 're', 've', 'y', 'ain', 'aren', \"aren't\", 'couldn', \"couldn't\", 'didn', \"didn't\", 'doesn', \"doesn't\", 'hadn', \"hadn't\", 'hasn', \"hasn't\", 'haven', \"haven't\", 'isn', \"isn't\", 'ma', 'mightn', \"mightn't\", 'mustn', \"mustn't\", 'needn', \"needn't\", 'shan', \"shan't\", 'shouldn', \"shouldn't\", 'wasn', \"wasn't\", 'weren', \"weren't\", 'won', \"won't\", 'wouldn', \"wouldn't\"]\n", 388 | "\n", 389 | "Punctuation\n", 390 | "\n", 391 | "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~\n" 392 | ] 393 | } 394 | ], 395 | "source": [ 396 | "#Import the english stop words list from NLTK\n", 397 | "stopwords_english = stopwords.words('english') \n", 398 | "\n", 399 | "print('Stop words\\n')\n", 400 | "print(stopwords_english)\n", 401 | "\n", 402 | "print('\\nPunctuation\\n')\n", 403 | "print(string.punctuation)" 404 | ] 405 | }, 406 | { 407 | "cell_type": "markdown", 408 | "metadata": {}, 409 | "source": [ 410 | "We can see that the stop words list above contains some words that could be important in some contexts. \n", 411 | "These could be words like _i, not, between, because, won, against_. You might need to customize the stop words list for some applications. For our exercise, we will use the entire list.\n", 412 | "\n", 413 | "For the punctuation, we saw earlier that certain groupings like ':)' and '...' should be retained when dealing with tweets because they are used to express emotions. In other contexts, like medical analysis, these should also be removed.\n", 414 | "\n", 415 | "Time to clean up our tokenized tweet!" 416 | ] 417 | }, 418 | { 419 | "cell_type": "code", 420 | "execution_count": 17, 421 | "metadata": {}, 422 | "outputs": [ 423 | { 424 | "name": "stdout", 425 | "output_type": "stream", 426 | "text": [ 427 | "\n", 428 | "\u001b[92m\n", 429 | "['my', 'beautiful', 'sunflowers', 'on', 'a', 'sunny', 'friday', 'morning', 'off', ':)', 'sunflowers', 'favourites', 'happy', 'friday', 'off', '…']\n", 430 | "\u001b[94m\n", 431 | "removed stop words and punctuation:\n", 432 | "['beautiful', 'sunflowers', 'sunny', 'friday', 'morning', ':)', 'sunflowers', 'favourites', 'happy', 'friday', '…']\n" 433 | ] 434 | } 435 | ], 436 | "source": [ 437 | "print()\n", 438 | "print('\\033[92m')\n", 439 | "print(tweet_tokens)\n", 440 | "print('\\033[94m')\n", 441 | "\n", 442 | "tweets_clean = []\n", 443 | "\n", 444 | "for word in tweet_tokens: # Go through every word in your tokens list\n", 445 | " if (word not in stopwords_english and # remove stopwords\n", 446 | " word not in string.punctuation): # remove punctuation\n", 447 | " tweets_clean.append(word)\n", 448 | "\n", 449 | "print('removed stop words and punctuation:')\n", 450 | "print(tweets_clean)" 451 | ] 452 | }, 453 | { 454 | "cell_type": "markdown", 455 | "metadata": {}, 456 | "source": [ 457 | "Please note that the words **happy** and **sunny** in this list are correctly spelled. " 458 | ] 459 | }, 460 | { 461 | "cell_type": "markdown", 462 | "metadata": {}, 463 | "source": [ 464 | "### Stemming\n", 465 | "\n", 466 | "Stemming is the process of converting a word to its most general form, or stem. This helps in reducing the size of our vocabulary.\n", 467 | "\n", 468 | "Consider the words: \n", 469 | " * **learn**\n", 470 | " * **learn**ing\n", 471 | " * **learn**ed\n", 472 | " * **learn**t\n", 473 | " \n", 474 | "All these words are stemmed from its common root **learn**. However, in some cases, the stemming process produces words that are not correct spellings of the root word. For example, **happi** and **sunni**. That's because it chooses the most common stem for related words. For example, we can look at the set of words that comprises the different forms of happy:\n", 475 | "\n", 476 | " * **happ**y\n", 477 | " * **happi**ness\n", 478 | " * **happi**er\n", 479 | " \n", 480 | "We can see that the prefix **happi** is more commonly used. We cannot choose **happ** because it is the stem of unrelated words like **happen**.\n", 481 | " \n", 482 | "NLTK has different modules for stemming and we will be using the [PorterStemmer](https://www.nltk.org/api/nltk.stem.html#module-nltk.stem.porter) module which uses the [Porter Stemming Algorithm](https://tartarus.org/martin/PorterStemmer/). Let's see how we can use it in the cell below." 483 | ] 484 | }, 485 | { 486 | "cell_type": "code", 487 | "execution_count": 18, 488 | "metadata": {}, 489 | "outputs": [ 490 | { 491 | "name": "stdout", 492 | "output_type": "stream", 493 | "text": [ 494 | "\n", 495 | "\u001b[92m\n", 496 | "['beautiful', 'sunflowers', 'sunny', 'friday', 'morning', ':)', 'sunflowers', 'favourites', 'happy', 'friday', '…']\n", 497 | "\u001b[94m\n", 498 | "stemmed words:\n", 499 | "['beauti', 'sunflow', 'sunni', 'friday', 'morn', ':)', 'sunflow', 'favourit', 'happi', 'friday', '…']\n" 500 | ] 501 | } 502 | ], 503 | "source": [ 504 | "print()\n", 505 | "print('\\033[92m')\n", 506 | "print(tweets_clean)\n", 507 | "print('\\033[94m')\n", 508 | "\n", 509 | "# Instantiate stemming class\n", 510 | "stemmer = PorterStemmer() \n", 511 | "\n", 512 | "# Create an empty list to store the stems\n", 513 | "tweets_stem = [] \n", 514 | "\n", 515 | "for word in tweets_clean:\n", 516 | " stem_word = stemmer.stem(word) # stemming word\n", 517 | " tweets_stem.append(stem_word) # append to the list\n", 518 | "\n", 519 | "print('stemmed words:')\n", 520 | "print(tweets_stem)" 521 | ] 522 | }, 523 | { 524 | "cell_type": "markdown", 525 | "metadata": {}, 526 | "source": [ 527 | "That's it! Now we have a set of words we can feed into to the next stage of our machine learning project." 528 | ] 529 | }, 530 | { 531 | "cell_type": "markdown", 532 | "metadata": {}, 533 | "source": [ 534 | "## process_tweet()\n", 535 | "\n", 536 | "As shown above, preprocessing consists of multiple steps before you arrive at the final list of words. We will not ask you to replicate these however. In the week's assignment, you will use the function `process_tweet(tweet)` available in _utils.py_. We encourage you to open the file and you'll see that this function's implementation is very similar to the steps above.\n", 537 | "\n", 538 | "To obtain the same result as in the previous code cells, you will only need to call the function `process_tweet()`. Let's do that in the next cell." 539 | ] 540 | }, 541 | { 542 | "cell_type": "code", 543 | "execution_count": 19, 544 | "metadata": {}, 545 | "outputs": [ 546 | { 547 | "name": "stdout", 548 | "output_type": "stream", 549 | "text": [ 550 | "\n", 551 | "\u001b[92m\n", 552 | "My beautiful sunflowers on a sunny Friday morning off :) #sunflowers #favourites #happy #Friday off… https://t.co/3tfYom0N1i\n", 553 | "\u001b[94m\n", 554 | "preprocessed tweet:\n", 555 | "['beauti', 'sunflow', 'sunni', 'friday', 'morn', ':)', 'sunflow', 'favourit', 'happi', 'friday', '…']\n" 556 | ] 557 | } 558 | ], 559 | "source": [ 560 | "from utils import process_tweet # Import the process_tweet function\n", 561 | "\n", 562 | "# choose the same tweet\n", 563 | "tweet = all_positive_tweets[2277]\n", 564 | "\n", 565 | "print()\n", 566 | "print('\\033[92m')\n", 567 | "print(tweet)\n", 568 | "print('\\033[94m')\n", 569 | "\n", 570 | "# call the imported function\n", 571 | "tweets_stem = process_tweet(tweet); # Preprocess a given tweet\n", 572 | "\n", 573 | "print('preprocessed tweet:')\n", 574 | "print(tweets_stem) # Print the result" 575 | ] 576 | }, 577 | { 578 | "cell_type": "markdown", 579 | "metadata": {}, 580 | "source": [ 581 | "That's it for this lab! You now know what is going on when you call the preprocessing helper function in this week's assignment. Hopefully, this exercise has also given you some insights on how to tweak this for other types of text datasets." 582 | ] 583 | } 584 | ], 585 | "metadata": { 586 | "kernelspec": { 587 | "display_name": "Python 3", 588 | "language": "python", 589 | "name": "python3" 590 | }, 591 | "language_info": { 592 | "codemirror_mode": { 593 | "name": "ipython", 594 | "version": 3 595 | }, 596 | "file_extension": ".py", 597 | "mimetype": "text/x-python", 598 | "name": "python", 599 | "nbconvert_exporter": "python", 600 | "pygments_lexer": "ipython3", 601 | "version": "3.7.1" 602 | } 603 | }, 604 | "nbformat": 4, 605 | "nbformat_minor": 2 606 | } 607 | -------------------------------------------------------------------------------- /NLP_C1_W1_lecture_nb_03.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Visualizing tweets and the Logistic Regression model\n", 8 | "\n", 9 | "**Objectives:** Visualize and interpret the logistic regression model\n", 10 | "\n", 11 | "**Steps:**\n", 12 | "* Plot tweets in a scatter plot using their positive and negative sums.\n", 13 | "* Plot the output of the logistic regression model in the same plot as a solid line" 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": {}, 19 | "source": [ 20 | "## Import the required libraries\n", 21 | "\n", 22 | "We will be using [*NLTK*](http://www.nltk.org/howto/twitter.html), an opensource NLP library, for collecting, handling, and processing Twitter data. In this lab, we will use the example dataset that comes alongside with NLTK. This dataset has been manually annotated and serves to establish baselines for models quickly. \n", 23 | "\n", 24 | "So, to start, let's import the required libraries. " 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": 4, 30 | "metadata": {}, 31 | "outputs": [], 32 | "source": [ 33 | "import nltk # NLP toolbox\n", 34 | "from os import getcwd\n", 35 | "import pandas as pd # Library for Dataframes \n", 36 | "from nltk.corpus import twitter_samples \n", 37 | "import matplotlib.pyplot as plt # Library for visualization\n", 38 | "import numpy as np # Library for math functions\n", 39 | "\n", 40 | "from utils import process_tweet, build_freqs # Our functions for NLP" 41 | ] 42 | }, 43 | { 44 | "cell_type": "markdown", 45 | "metadata": {}, 46 | "source": [ 47 | "## Load the NLTK sample dataset\n", 48 | "\n", 49 | "To complete this lab, you need the sample dataset of the previous lab. Here, we assume the files are already available, and we only need to load into Python lists." 50 | ] 51 | }, 52 | { 53 | "cell_type": "code", 54 | "execution_count": 5, 55 | "metadata": {}, 56 | "outputs": [ 57 | { 58 | "name": "stdout", 59 | "output_type": "stream", 60 | "text": [ 61 | "Number of tweets: 8000\n" 62 | ] 63 | } 64 | ], 65 | "source": [ 66 | "# select the set of positive and negative tweets\n", 67 | "all_positive_tweets = twitter_samples.strings('positive_tweets.json')\n", 68 | "all_negative_tweets = twitter_samples.strings('negative_tweets.json')\n", 69 | "\n", 70 | "tweets = all_positive_tweets + all_negative_tweets ## Concatenate the lists. \n", 71 | "labels = np.append(np.ones((len(all_positive_tweets),1)), np.zeros((len(all_negative_tweets),1)), axis = 0)\n", 72 | "\n", 73 | "# split the data into two pieces, one for training and one for testing (validation set) \n", 74 | "train_pos = all_positive_tweets[:4000]\n", 75 | "train_neg = all_negative_tweets[:4000]\n", 76 | "\n", 77 | "train_x = train_pos + train_neg \n", 78 | "\n", 79 | "print(\"Number of tweets: \", len(train_x))" 80 | ] 81 | }, 82 | { 83 | "cell_type": "markdown", 84 | "metadata": {}, 85 | "source": [ 86 | "# Load the extracted features\n", 87 | "\n", 88 | "Part of this week's assignment is the creation of the numerical features needed for the Logistic regression model. In order not to interfere with it, we have previously calculated and stored these features in a CSV file for the entire training set.\n", 89 | "\n", 90 | "So, please load these features created for the tweets sample. " 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": 6, 96 | "metadata": {}, 97 | "outputs": [ 98 | { 99 | "data": { 100 | "text/html": [ 101 | "
\n", 102 | "\n", 115 | "\n", 116 | " \n", 117 | " \n", 118 | " \n", 119 | " \n", 120 | " \n", 121 | " \n", 122 | " \n", 123 | " \n", 124 | " \n", 125 | " \n", 126 | " \n", 127 | " \n", 128 | " \n", 129 | " \n", 130 | " \n", 131 | " \n", 132 | " \n", 133 | " \n", 134 | " \n", 135 | " \n", 136 | " \n", 137 | " \n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | " \n", 180 | " \n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " \n", 188 | " \n", 189 | " \n", 190 | " \n", 191 | " \n", 192 | " \n", 193 | " \n", 194 | " \n", 195 | " \n", 196 | " \n", 197 | "
biaspositivenegativesentiment
01.03020.061.01.0
11.03573.0444.01.0
21.03005.0115.01.0
31.02862.04.01.0
41.03119.0225.01.0
51.02955.0119.01.0
61.03934.0538.01.0
71.03162.0276.01.0
81.0628.0189.01.0
91.0264.0112.01.0
\n", 198 | "
" 199 | ], 200 | "text/plain": [ 201 | " bias positive negative sentiment\n", 202 | "0 1.0 3020.0 61.0 1.0\n", 203 | "1 1.0 3573.0 444.0 1.0\n", 204 | "2 1.0 3005.0 115.0 1.0\n", 205 | "3 1.0 2862.0 4.0 1.0\n", 206 | "4 1.0 3119.0 225.0 1.0\n", 207 | "5 1.0 2955.0 119.0 1.0\n", 208 | "6 1.0 3934.0 538.0 1.0\n", 209 | "7 1.0 3162.0 276.0 1.0\n", 210 | "8 1.0 628.0 189.0 1.0\n", 211 | "9 1.0 264.0 112.0 1.0" 212 | ] 213 | }, 214 | "execution_count": 6, 215 | "metadata": {}, 216 | "output_type": "execute_result" 217 | } 218 | ], 219 | "source": [ 220 | "data = pd.read_csv('logistic_features.csv'); # Load a 3 columns csv file using pandas function\n", 221 | "data.head(10) # Print the first three data entries" 222 | ] 223 | }, 224 | { 225 | "cell_type": "markdown", 226 | "metadata": {}, 227 | "source": [ 228 | "Now let us get rid of the data frame to keep only Numpy arrays." 229 | ] 230 | }, 231 | { 232 | "cell_type": "code", 233 | "execution_count": 7, 234 | "metadata": {}, 235 | "outputs": [ 236 | { 237 | "name": "stdout", 238 | "output_type": "stream", 239 | "text": [ 240 | "(8000, 3)\n", 241 | "[[1.000e+00 3.020e+03 6.100e+01]\n", 242 | " [1.000e+00 3.573e+03 4.440e+02]\n", 243 | " [1.000e+00 3.005e+03 1.150e+02]\n", 244 | " ...\n", 245 | " [1.000e+00 1.440e+02 7.830e+02]\n", 246 | " [1.000e+00 2.050e+02 3.890e+03]\n", 247 | " [1.000e+00 1.890e+02 3.974e+03]]\n" 248 | ] 249 | } 250 | ], 251 | "source": [ 252 | "# Each feature is labeled as bias, positive and negative\n", 253 | "X = data[['bias', 'positive', 'negative']].values # Get only the numerical values of the dataframe\n", 254 | "Y = data['sentiment'].values; # Put in Y the corresponding labels or sentiments\n", 255 | "\n", 256 | "print(X.shape) # Print the shape of the X part\n", 257 | "print(X) # Print some rows of X" 258 | ] 259 | }, 260 | { 261 | "cell_type": "markdown", 262 | "metadata": {}, 263 | "source": [ 264 | "## Load a pretrained Logistic Regression model\n", 265 | "\n", 266 | "In the same way, as part of this week's assignment, a Logistic regression model must be trained. The next cell contains the resulting model from such training. Notice that a list of 3 numeric values represents the whole model, that we have called _theta_ $\\theta$." 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": 8, 272 | "metadata": {}, 273 | "outputs": [], 274 | "source": [ 275 | "theta = [7e-08, 0.0005239, -0.00055517]" 276 | ] 277 | }, 278 | { 279 | "cell_type": "markdown", 280 | "metadata": {}, 281 | "source": [ 282 | "## Plot the samples in a scatter plot\n", 283 | "\n", 284 | "The vector theta represents a plane that split our feature space into two parts. Samples located over that plane are considered positive, and samples located under that plane are considered negative. Remember that we have a 3D feature space, i.e., each tweet is represented as a vector comprised of three values: `[bias, positive_sum, negative_sum]`, always having `bias = 1`. \n", 285 | "\n", 286 | "If we ignore the bias term, we can plot each tweet in a cartesian plane, using `positive_sum` and `negative_sum`. In the cell below, we do precisely this. Additionally, we color each tweet, depending on its class. Positive tweets will be green and negative tweets will be red." 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": 9, 292 | "metadata": {}, 293 | "outputs": [ 294 | { 295 | "data": { 296 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAHjCAYAAACzaWwPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmcVnXd//HXZ2bY930TRREXsERAM3NJM9cKK0u0hczSui3zListu7Oy0p9Zd92mZVkuaW5pmAZqivvG4AaIwACCCAiyrwMz8/39cQ44yADDbBfL6+njesx1fa9zzvU5x9F5X9/zPd8TKSUkSZKKCl2AJEnaMRgKJEkSYCiQJEk5Q4EkSQIMBZIkKWcokCRJgKFAkiTlDAWSJAkwFEiSpFxJoQtoal27dk39+vUrdBmSJDWJ8ePHv5NS6labZXe7UNCvXz9KS0sLXYYkSU0iImbVdllPH0iSJMBQIEmScoYCSZIEGAokSVLOUCBJkgBDgSRJyhkKJEkSYCiQJEk5Q4EkSQIMBZIkKWcokCRJgKFAkiTlDAWSJAkwFEiSpJyhQJIkAYYCSZKUMxRIkiTAUNDwnn4avv/9QlchSdJ2Kyl0AbucQw6BqqpCVyFJ0nazp6ChtW4NRx1V6CokSdpuhgJJkgQYCiRJUs5QIEmSAEOBJEnKGQokSRJgKJAkSTlDgSRJAgwFkiQpZyiQJEmAoUCSJOUMBZIkCTAUSJKknKFAkiQBhgJJkpQzFEiSJMBQIEmScoYCSZIEGAokSVLOUCBJkgBDgSRJyhkKJEkSYCiQJEk5Q4EkSQIMBZIkKWcokCRJgKFAkiTlDAWSJAkwFEiSpJyhQJIkAYYCSZKUMxRIkiTAUCBJknKGAkmSBBgKJElSrtFCQUT0jYixETE5IiZFxLfy9s4R8XBETMt/dqq2ziURURYRUyLixGrtQyNiQv7e7yIi8vYWEXFH3v58RPRrrP2RJGlX15g9BRXAd1JKBwKHA+dHxEDgYuCRlNIA4JH8Nfl7I4BBwEnAtRFRnG/rOuBcYED+OClvPwdYklLaF/gNcGUj7o8kSbu0RgsFKaV5KaUX8+crgMlAH2A4cFO+2E3Aafnz4cDtKaXylNJMoAw4LCJ6Ae1TSs+mlBJw83vW2bCtu4GPbOhFkCRJ26dJxhTk3fqHAM8DPVJK8yALDkD3fLE+wJvVVpuTt/XJn7+3fZN1UkoVwDKgSw2ff25ElEZE6cKFCxtmpyRJ2sU0eiiIiLbAP4ALU0rLt7ZoDW1pK+1bW2fThpSuTykNSykN69at27ZKliRpt9SooSAimpEFgltTSvfkzW/npwTIfy7I2+cAfautvgcwN2/fo4b2TdaJiBKgA7C44fdEkqRdX2NefRDADcDklNKvq711HzAyfz4SGFWtfUR+RcHeZAMKX8hPMayIiMPzbX7xPets2NbpwKP5uANJkrSdShpx2x8CvgBMiIiX87YfAFcAd0bEOcBs4DMAKaVJEXEn8BrZlQvnp5Qq8/W+DtwItAJG5w/IQsctEVFG1kMwohH3R5KkXVrsbl+shw0blkpLSwtdhiRJTSIixqeUhtVmWWc0lCRJgKFAkiTlDAWSJAkwFEiSpJyhQJIkAYYCSZKUMxRIkiTAUCBJknKGAkmSBBgKJElSzlAgSZIAQ4EkScoZCiRJEmAokCRJOUOBJEkCDAUNa9WqQlcgSVKdGQoayhtvwNlnw/Llha5EkqQ6MRQ0lL32gp//HNq3L3QlkiTViaGgoUTAgAFw663w7LOFrkaSpO1mKGhoHTpAq1aFrkKSpO1WUugCdgm/+AUcfDCceip87GOFrkaSpDqxp6AhHHEE/PWvMHVqoSuRJKnO7CloCB/+cHbaYN99C12JJEl1ZihoKIccUugKJEmqF08fSJIkwFAgSZJyhgJJkgQYCiRJUs5QIEmSAEOBJEnKGQokSRJgKJAkSTlDgSRJAgwFkiQpZyiQJEmAoUCSJOUMBZIkCTAUSJKknKFAkiQBhgJJkpQzFEiSJMBQIEmScoYCSZIEGAokSVLOUCBJkgBDgSRJyhkKJEkSYCiQJEk5Q4EkSQIMBZIkKWcokCRJgKFAkiTlDAWSJAkwFEiSpJyhQJIkAYYCSZKUMxRIkiTAUCBJknKGAkmSBBgKJElSzlAgSZIAQ4EkScoZCiRJEmAokCRJOUOBJEkCDAWSJClnKJAkSYChQJIk5QwFkiQJMBRIkqScoUCSJAGGAkmSlDMUSJIkwFAgSZJyhgJJkgQYCiRJUs5QIEmSAEOBJEnKGQokSRJgKJAkSblGCwUR8ZeIWBARE6u1XRYRb0XEy/njlGrvXRIRZRExJSJOrNY+NCIm5O/9LiIib28REXfk7c9HRL/G2pfttnYt/P3vUFlZ6EokSaq1xuwpuBE4qYb236SUBuePfwNExEBgBDAoX+faiCjOl78OOBcYkD82bPMcYElKaV/gN8CVjbUj223JEpg0CdasKXQlkiTVWqOFgpTSE8DiWi4+HLg9pVSeUpoJlAGHRUQvoH1K6dmUUgJuBk6rts5N+fO7gY9s6EUouF694PLLoW3bQlciSVKtFWJMwTci4tX89EKnvK0P8Ga1ZebkbX3y5+9t32SdlFIFsAzoUtMHRsS5EVEaEaULFy5suD15r1Gj4OWXG2/7kiQ1oqYOBdcB/YHBwDzg6ry9pm/4aSvtW1tn88aUrk8pDUspDevWrdv2Vbw9Vq+Gdesab/uSJDWikqb8sJTS2xueR8SfgPvzl3OAvtUW3QOYm7fvUUN79XXmREQJ0IHan65oHGeeWdCPlySpPpq0pyAfI7DBJ4ENVybcB4zIryjYm2xA4QsppXnAiog4PB8v8EVgVLV1RubPTwcezccdSJKkOmi0noKI+DvwYaBrRMwBfgx8OCIGk3XzvwGcB5BSmhQRdwKvARXA+SmlDdfzfZ3sSoZWwOj8AXADcEtElJH1EIxorH3ZqjffhB49oHnzgny8JEkNJXa3L9fDhg1LpaWlDbfBiy6CI4+E007b9rKSJDWxiBifUhpWm2WbdEzBLumyy6BNm0JXIUlSvRkK6su5CCRJuwjvfSBJkgBDgSRJyhkKJEkSYCiQJEk5Q4EkSQIMBZIkKWcokCRJgKFAkiTlDAWSJAkwFEiSpJyhQJIkAYYCSZKUMxRIkiTAUCBJknKGAkmSBBgKJElSzlAgSZIAQ4EkScoZCiRJEmAokCRJOUOBJEkCDAWSJClnKJAkSYChQJIk5QwFkiQJMBRIkqScoUCSJAGGAkmSlDMUSJIkwFAgSZJyhgJJkgQYCiRJUs5QIEmSAEOBJEnKGQokSRJgKJAkSTlDwY5s2TL4+c8hpUJXIknaDRgKdmRPPQV33w1r1hS6EknSbsBQsCM79VR46SVo3brQlUiSdgOGAkmSBBgKJElSzlAgSZKA7QgFEbFXRByfP28VEe0aryxJktTUahUKIuKrwN3AH/OmPYB/NlZRkiSp6dW2p+B84EPAcoCU0jSge2MVJUmSml5tQ0F5SmndhhcRUQI4o44kSbuQ2oaCxyPiB0CriPgocBfwr8YrS5IkNbXahoKLgYXABOA84N/ApY1VlCRJanoltVxuOHBzSulPjVmMJEkqnNr2FHwCmBoRt0TEqfmYAm3NpElQXl7oKiRJqrVahYKU0tnAvmRjCc4CpkfEnxuzsJ3en/4ETzxR6CokSaq1Wn/jTymtj4jRZFcdtCI7pfCVxipsp/erX0GJHSqSpJ1HbScvOikibgTKgNOBPwO9GrGunZ+BQJK0k6ntX64vAbcD56WUPFG+vVavhhYtoLi40JVIkrRFtQoFKaURjV3ILu3KK2G//eBznyt0JZIkbdFWQ0FEPJVSOjIiVrDpDIYBpJRS+0atbldx3nnQsWOhq5Akaau2GgpSSkfmP70jYn307l3oCiRJ2qbaDjS8pTZt2oaJE6GsrNBVSJJUo9pOXjSo+ot88qKhDV/OTuzf/4Z77tn6Ms88A6WlTVOPJEnbaVtjCi4BNtwIafmGZmAdcH0j17bzWLMGOnXK/uCXlcG++9a83LnnNm1dkiRth632FKSUfpmPJ7gqpdQ+f7RLKXVJKV3SRDXu2KZMgc9+Fl56KZubYPbsQlckSVKd1PaSxEsiohMwAGhZrd15fP/wh6x34J574NZboUePQlckSVKd1CoURMRXgG8BewAvA4cDzwLHNV5pO4GKChgzBlq1gvbtIaLQFUmSVGe1HWj4LeBQYFZK6VjgEGBho1W1s5g9G5Yuhc9/Hm67Dbp3L3RFkiTVWW2nOV6bUlobEUREi5TS6xGxf6NWtjPYZx+YPh1aty50JZIk1VttQ8GciOgI/BN4OCKWAHMbr6ydiIFAkrSLqO1Aw0/mTy+LiLFAB2BMo1UlSZKaXG0HGnau9nJC/jPVtKwkSdo51Xag4YtkAwunAtPy5zMj4sWIcGZDSZJ2AbUNBWOAU1JKXVNKXYCTgTuB/wKubaziJElS06ltKBiWUnpww4uU0kPA0Sml54AWjVKZJElqUrW9+mBxRHwfuD1/fQawJCKKgapGqUySJDWp2vYUnEU2m+E/80ffvK0Y+GzjlCZJkppSbS9JfAf4ZkS0TSmtfM/bZQ1fliRJamq16imIiCMi4jXgtfz1wRHhAENJknYhtT198BvgRGARQErpFeDoxipKkiQ1vdqGAlJKb76nqbKBa5EkSQVU26sP3oyII4AUEc2BC4DJjVeWJElqarXtKfgacD7QB5gDDM5fb1FE/CUiFkTExGptnSPi4YiYlv/sVO29SyKiLCKmRMSJ1dqHRsSE/L3fRUTk7S0i4o68/fmI6FfbnZYkSZurVShIKb2TUvpcSqlHSql7SunzKaVF21jtRuCk97RdDDySUhoAPJK/JiIGAiOAQfk61+ZzIABcB5wLDMgfG7Z5DrAkpbQv2ZiHK2uzL5IkqWZbPX0QEf+zlbdTSulnW3nziRq+vQ8HPpw/vwl4DPh+3n57Sqmc7J4KZcBhEfEG0D6l9Gxez83AacDofJ3L8m3dDVwTEZFS8kZNkiTVwbZ6ClbV8IDsW/r36/B5PVJK8wDyn93z9j5A9YGMc/K2Dacr3tu+yToppQpgGdClpg+NiHMjojQiShcuXFiHsiVJ2vVttacgpXT1hucR0Q74FnA22XTHV29pvTqImj5+K+1bW2fzxpSuB64HGDZsmD0JkiTVYJtjCvLBgZcDr5KFiCEppe+nlBbU4fPejohe+XZ7ARu2MYds6uQN9gDm5u171NC+yToRUQJ0ABbXoSZJksQ2QkFEXAWMA1YA70spXZZSWlKPz7sPGJk/HwmMqtY+Ir+iYG+yAYUv5KcYVkTE4flVB198zzobtnU68KjjCSRJqrttzVPwHaAcuBT4YX41IGRd9yml1H5LK0bE38kGFXaNiDnAj4ErgDsj4hxgNvAZsg1Niog7yaZRrgDOTyltmBzp62RXMrQiG2A4Om+/AbglH5S4mOzqBUmSVEexu325HjZsWCotLS10GZIkNYmIGJ9SGlabZWs9zbEkSdq1GQokSRJgKJAkSTlDgSRJAgwFkiQpZyiQJEmAoUCSJOUMBZIkCTAUSJKknKFAkiQBhgJJkpQzFEiSJMBQIEmScoYCSZIEGAokSVLOUCBJkgBDgSRJyhkKJEkSYCiQJEk5Q4EkSQIMBZIkKWcokCRJgKFAkiTlDAWSJAkwFDSNOXPg8sth3bpCVyJJ0hYZCppCu3bQty+UlBS6EkmStshQ0NhuugnefhtGjoSi9xzuqiooLy9MXZIkvYdfXRtbVRWkVPN7N90Ec+fCPvvAvvvCoYc2bW2SJFVjKGhsZ5+95fc+/WlYsQLGjYNmzZquJkmSamAoaArz50OPHhCxaXv79tmjT5/C1CVJUjWGgsYweTJMnZqNFxg+HL71LfjYx2DVKvja1wpdnSRJNTIUNIaxY2HRIujdOwsI++wDy5fDfvsVujJJkrbIUNDQqqrgr3+Fnj2hQwdYsADOPDMLCF27Fro6SZK2yFDQ0IYOzcYQfOQjcNhhMGwY7LknTJwIs2Zl70uStANynoKG8vbb8NRT8NZb8NWvwhVXwMqVcMstMG0aTJqUPSRJ2kHZU1Bf69fDGWfAE09ARQV06wavvprNTfD5z0NlJVx4IXzqU1nvQW3svz9873twzjmNW7skSdVE2tLEOruoYcOGpdLS0obb4Ac+ANOnQ9u22dUF69dnYwdef/3daY3XroWWLWu/zQ2XLu5m/24kSQ0vIsanlIbVZllPH9RXRQWcdBKcdRb06wfNm8NRR8GVV8Kzz2bLbE8ggGw65GOOafBSJUnaGkNBfd1xByxbBtdcA927w5Ah2YREhx5a90sQKyvhsccatExJkrbFUFAfq1bBscfCww9nXf7vvJP1ECxdCscdB126FLpCSZJqzYGG9dGqVdbV37x5NnPhxz8OBx8MN9zgbZIlSTsd/3LVx7/+BXPmwJe+BP/3f9C6ddZuIJAk7YT861Ufp56a3f74858vdCWSJNWbYwrqo6TEQCBJ2mUYCiRJEmAokCRJOUOBJEkCDAWSJClnKJAkSYChQJIk5QwFkiQJMBRIkqScoUCSJAGGAkmSlDMUSJIkwFAgSZJyhgJJkgQYCiRJUs5QIEmSAEOBJEnKGQokSRJgKJAkSTlDgSRJAgwFkiQpZyiQJEmAoUCSJOUMBZIkCTAUSJKknKFAkiQBhgJJkpQzFEiSJMBQIEmScoYCSZIEGAokSVLOUCBJkgBDgSRJyhkKJEkSYCjY8b38Mpx/PqxeDRUVha5GkrQLMxTs6AYOhDPPhN/9Dm68sdDVSJJ2YSWFLkDb0Lw5HHkkDBgArVoVuhpJ0i6sID0FEfFGREyIiJcjojRv6xwRD0fEtPxnp2rLXxIRZRExJSJOrNY+NN9OWUT8LiKiEPvTJHr0gPbtC12FJGkXVsjTB8emlAanlIblry8GHkkpDQAeyV8TEQOBEcAg4CTg2ogozte5DjgXGJA/TmrC+utv1Cj429/efT17Nrzxxruv167NlkmpyUuTJO1+dqQxBcOBm/LnNwGnVWu/PaVUnlKaCZQBh0VEL6B9SunZlFICbq62zo4tJbj/fujbF958Ex59FJYtgzFjYPTod5ebMweeegrWrStcrZKk3UahQkECHoqI8RFxbt7WI6U0DyD/2T1v7wO8WW3dOXlbn/z5e9s3ExHnRkRpRJQuXLiwAXdjO02bll1NsGIFPPwwdO8OQ4fC9OnwjW/AIYfA6adngQFg333hqqugRYvC1SxJ2m0UKhR8KKU0BDgZOD8ijt7KsjWNE0hbad+8MaXrU0rDUkrDunXrtv3VNpRXXoEXXsjGBvz2tzB/Ptx9Nzz5JHz3uzBsWHYKYeLEwtUoSdptFSQUpJTm5j8XAPcChwFv56cEyH8uyBefA/SttvoewNy8fY8a2ndMq1ZlYeCrX3237Y9/hBkz4OSTs/cisqsNDj64cHVKknZbTR4KIqJNRLTb8Bw4AZgI3AeMzBcbCYzKn98HjIiIFhGxN9mAwhfyUwwrIuLw/KqDL1ZbZ8dSWZkFgPPOgy9/GT77WbjlFpg1CwYNgptugnHjsmVXrIAFC7a+PUmSGkEh5inoAdybXz1YAtyWUhoTEeOAOyPiHGA28BmAlNKkiLgTeA2oAM5PKVXm2/o6cCPQChidP3Y8c+bA5Mlw881QUgLXXQe/+EU2Q+FTT2VXGfTuDZ/5DBxxRPaQJKmJNXkoSCnNADbrH08pLQI+soV1fg78vIb2UuCghq6xQY0aBe9/P4wdm50qePrpbM6Bt9+Gjh2huBg6dICiouyqhF14qgVJ0o7NGQ0by/33ZwMHzzsP9toLFi2CefOyywuXLcuuPCgvh169sl6Cb3/bQCBJKqgdaZ6CXUdVFTz7bDY24NRToV076NkzO12QUtY7sM8+sGZNdqOj738/u8fBloweDUuXNl39kqTdkqGgIVVVZT+LirIegV//OpuY6NFH3+0lAOjaNbsKISLrRRg6NGt/6il45pmsvXqvwbhx2RwHkiQ1Ik8fNKSf/AQGD4ZPfjKbnGjGjGwQYUrZKYMN3ngjux1y+/bwpS/BkiXZH/3Fi7NA8V7/8z9NtQeSpN2YoaAhnXEGdOoEZ52V3e74xz+G9evffb+kJOtN2Hff7I6HrVtnVxyceWY2adFtt2VzFHivA0lSARgKGtLq1XDJJfDgg3DffdlAwuoqKqBly2x8QN++0K1b1ltQXAwHHJCdVpAkqUAMBQ1l+fLslEBZWfZNf9Wqmpfr0iULA6+8AiedlJ1e+OY34cADoU+Nt26QJKlJONCwoaxcmV1auHLllu9qWFwMH/94dorgpJOy0wiLFmXjD+68M+tpeOONbHzBmDFNWr4kSfYUNJRXXoH//GfLgaBNm2xOgrvuysYVlJdn4w+KiqBZMzjnHHjggewuih/7GDz3XDbw8KyzmnY/JEm7LUNBQ3jwQRg+fNNBhe+1fn0WGI46Cj7xCSgtzQYeXnBBFhggu23yJz6R3Sq5bVt48cWmqV+SJDx90DB+8IOtBwLIegQisjkJfvazbNzBJz6R3RNhg4gsEAC8730wcmTN25IkqREYCurrmWe2/Y2+pCTrJWjePLv0sH//7FLEZctgypQtr1dZueX3JElqYJ4+qK8PfWjbywwdmgWAtWth/Hj4v//b9p0Q//nPbCbDn292HyhJkhqFoaCxNWuWzVr4yivw1lvZqYODN7tJ5OaOPx7237/Ry5MkaQNDQWM7+GA49tgsEBx1VO3/0Ldtm81dIElSE3FMQX1ceunW3//oR7P5B973vuwKhbPOymY0lCRpB2RPQX3cfXfN7SeckM1o+NBD2TiCk0+G73ynaWuTJGk7GQrqoWLKFIrz5wHQs2c2O+FBB727UMuWMGpUAaqTJGn7GArqoQSoAhJQvHChNzSSJO3UDAX1kZKDMiRJuwz/pkmSJMBQIEmScoYCSZIEGAokSVLOUCBJkgBDgSRJyhkKJEkSYCiQJEk5Q4EkSQIMBZIkKWcokCRJgKFAkiTlDAWSJAkwFEiSpJyhQJIkAYYCSZKUMxRIkiTAUCBJknKGAkmSBBgKJElSzlAgSZIAQ4EkScoZCiRJEmAokCRJOUOBJEkCDAWSJClnKJAkSYChQJIk5QwF2iHMWDKDOcvnFLoMSdqtGQq0Q3hg6gM8NP2hQpchSbu1kkIXIAF88wPfLHQJkrTbs6dAO4wnZz3J1c9cXegyJGm3ZU+Bdhj9O/envLK80GVI0m7LUKAdRu92vendrnehy5Ck3ZanD5ra88/DnXcWugpJkjZjKGgojz4Kf/3rtpdr1gxatKj5vZtvhueea9i6JEmqJU8fNJTu3SECbr0V+vSBD3+45uWGDMkeNencGdq2bbQSJUnaGkNBQznooOzniBEwdy5ccAEsXw6HHJI9auNjH6vVYusq1/G3V//GiING0LpZ6zoWLEnSpgwFDe3222HSJOjRAx5/HFq1avCPWFe5jrkr5rJm/RpDgSSpwTimoDEMGgRdu8KnPw0HHLD5+1Onwssvw8yZW9/O5ZfDww9v1ty2eVsuPfpSurTu0kAF192a9Wu4cMyFTF88fZP20rml/OSxnxSoKklSXRgKGtqkSXDDDTW/N306zJ8PzzwD118Pf/rT1rd13HEwbFjD19iAWpa05IT+J9C3Q99N2vftvC+H9jm0QFU1jllLZ/HU7KcKXYYkNRpPHzSkxx+HiROhY8dN21PKBiHedx+UlsJnPwu//z2sW7f5Nh58EI48Etq0gSOOaJq66yEiOGXAKZu1d2zZscb2ndnURVN57Z3XOHLPIwtdiiQ1CkNBQ2rRAvr3h5NOerftf/4nG3D4v/8LF14IL76YjTeI2PzSxJTgscegW7ctX6GwE6tKVUxdNJUDutZwSmUn8NH+H+Wj/T9a6DIkqdEYChrS4Ydv3tahA+y3X/Y8AoYO3fL6EfDLXzZObU3ktYWvATCw28Aa3/vFk7/gxtNupHlx86YuTZK0DY4paEhr1kBlZfZ8xozs53e+k/UAjBoFd98Nd921xdUXrFrArKWzNmuftGASP3v8Z6SUGqPqOhk9bTSvvv3qZu2lc0spnVu6SdvairUcct0hPDLjEf4y/C8GAknaQdlT0JCuvDI7fQDZlQM33JBdifC3v0FRETRvDkcdtcXVb3z5RlatX8WQXkP4UN8P0bV1VyC7J8DenfZuij2otYWrF9KhZQcAfvf87+jXoR+Dew3mC+//AhEBwLRF05i0cBLD9x9O34596dq6Ky1LWhay7N3GM28+Q8eWHWvssZGkLbGnoL5ee+3d6Y3PPRcWL4Yf/SjrMTj3XNh7b3j9dTjhhGxsQY8em23irtfu4sdjf8zK8pWURAlXPX0Vt7xyy8b3ZyyZwZiyMTz/1vNNtVc1mrpoKotWLwKgqqqKHm168GDZgxzY9UBmLp3Jfz3wX0xYMAGAmUtm8sNHf8iL815k9rLZnNj/RI7pd8xm21y9fnWT7kNtzV42m3kr5hW6jDp7/Z3Xmb1sdqHLkLSTsaegvoqLYdo0OPts2HNPGD8eli7N3luxgvXNSohURckzz0C/ftm9D+bOhd7v3g2wS4su3PbqbezVcS/GfH4MFx95McvWLqMqVREEfxj/Bz476LMc1uewjeusr1zPsvJlG3sTauvOSXdyzF7H0KPt5uFkW06/43Tat2jPY2c/xjXjrqFscRltW7Slb/u+TH5nMkN7DWVQt0H89PGf0rFFR9o2b8u3PvAtXpr/Ev+Z+R/277o/e7TfY+P2Jrw9gV8+9UtuPO1GmhU1Y9X6VbRtvmNM8/zP1/9Jy5KWnDv03EKXUidfPuTLhS5B0k7InoL62n//bMzAHXdkj1dfhWXLskdVFWvbNGNV57bZIMKTT4Z58+D+++G66zZuYuSokbyz5h2O6XcMlamSa8ddy+A/Duanj/+UiGBg14H0aNODonj3X9c/Jv+DXz/76+0ud9ayWbyz+p3tWueFt17giqeu4KtDv8qkhZP4f0//P6oqq7j5lZtpXdKav0/4O5MXTCalRFEUsX+X/enUshPTF0/nhpduYOGqhdx7xr0cv8/xm2x3UPdBXPaLSnAaAAAVOklEQVThy2he3Jz7ptzHjx/78XbvT31cO+5abp94e43vfeOwb/DVIV9t0noa0yvzX+FXz/yq0GVI2sHZU9AQxo7NBhlOmbJp+8CBtLvhhuz0wb33ZvMTAJSVwYIFGxcbNWJU1g3f7UBalLRgeflymhc1Z+zMsVRUVjBz6UyaFTXjA3t8YOM6nz7w05zQ/4SNr0dPG82URVM4tPehfGjPD22x1O8e8d3t3r3/zPgPo8tG8+TZTzJr6SyqqqpYsX4Fc1bM4bfP/ZZmxc0IgqmLpxIRnHHQGVz+xOW0ataKNeVrmL4om+3wotEXcfpBp7O6YjXH7X0cRVHEfl3247bHbqN0RSn/ffR/b3dt9XH0XkfTplmbGt+rHsB2Bd3bdKd/p/6FLkPSDi52pBHtTWHYsGGptLR02wvW1ne/C9dem4WC6seypCQbRzBoEAwaxNp+fZk5sBf3TL6HkYNHbuxGf23ha7QobsGFYy5k/or5PHb2Y7Rp3obpi6dzzqhzOKbfMbw872USifvOuo/7p9zPQT0Ool/HfpuUcfMrNzN/xXyO2usoPtj3g0A22Gzmkpl87v2fq9cuPlj2IGPKxrBk7RKmvjOV8XPHUxRFdGndhfkr5wPQvmV7RgwaQfe23Tlmr2O46umraNWsFfe+fi8lUUK7Zu1YvG4xJ+19Eucddh6nHXDaxu3HT7KBienHu9fvoiQ1hYgYn1Kq1fS4u9bXoUL41a9g9epNAwFkAw2ffpqqRx6Ba67huXde4v5//JIDX1+08bx5ZVUlv3n2N9z92t2sWreKF+e/yOWPX86YsjF0bNmRLw/5MsfvczxL1y7lgK4H8NgbjzGmbMzGP8TVffHgL/K9I7+3MRAAtGnWhs6tOtdr9/5Q+gfWV61n9rLZ3PbqbTz71rOsS+tYW7WWtRVradmsJS2KW9CueTtemvcSXVt3ZdSUUVRUVbC+aj3H7HUMQ3sPZfRnRtOiuAX3fe4+urTqwowlMzZ+RjHF9apxdzembMzGAaCSVB+ePqivNm1g1apN29q3h+bNWbe+nPGtFnPYH//F4H324JbvHES35gPo2LIjN750I8+8+Qzj3hrH6Gmj+cqQrzB76Wzat2jPefefx6cP+DT9O/fn7xP+Tt8Ofendvjc/evRHrK1Yy1eH1u5c98E9D+bgngfXabeqUhXzVsyjoqqCsTPH8uSsJ1mf1m98v3/7/vRs35Nla5exfN1yOrfszKtvv8qyF5Zx7tBz+dQBn2J5+XJef+d1RpeN5kfP/ojfn/J7qlIVr779KuWV5ezTaR8AKn5cUacalXlx3ot0bNlxh7hBlqSdm6Ggvlq33jwUDBkCxx9PycQJvP/l8Tx03Xd56xMf5qYe85ncZ08GvvUCryx4hadmP0Wvdr2YuXQm15Vex5I1S3hk5iM88oVHuOGlG5i5ZCZLy5fSrnk73lz2JiMHj+TwPQ7noO4HbbWk8opyVq1fVedeghtevIHRZaOZtmgaby59kxXrVlDBpn+4py+fziF9DqFscRlrKtewcNVC1lauZc7yOSwtX8oFoy+gWVEzerTrwYNfeJCpi6by9sq3+faD3+bqE6+mRXGLLXx6w5u0YBL3T72f7x/5fV6e/zJ/fvHP/O7k39Vp3MA/X/8nPdr02KRHptB+cNQPCl2CpF2EoaA+ZsyAhQs3aaosLqby8ceoLF/DnK+fRcviVXS+fwzjDmzP6ktX8/Mnfs7Nr9zMynUrmbJoCmWLy1if1rNq3SoqqeSJWU/wzX9/k7067cVB3Q5ixPtG0LlVZ5578zlGTRnFR/f5KEvXLuWGF2/g64d+nbLFZUxfMp1PHvBJ5q+cz+VPXM6gboOYv3I+Pzm2brcu/ve0fzNhwQRmLZnFulTDTZty/5j8D4KgiqqNbbd9+jbum3IfM5bOoFVxK7535PcoKSphYLeB7Nt5X3q07cGPHv0RR+55JMMPGF6n+rZXtzbd6N85G2T3UNlDDOo2iKIoIqVERLB6/WomLZjEoX0OZd6KebQoabHFQFUURQTRJHVLUlMzFNTHB9/zbfHoo1n30jgWt4SKJbN54OHfM+vTx7L+Y2fRvVd/Fq9ZzAUfuID/HvPfPDD1Aaqooiplf1ArqaQkSmhW3IxJCyexZO0SDu9zOPOWz+Pp2U9zx8Q7mLZ4GhVVFRRHMS1KWlAURVmvQHnWU1FEEePnjueCD1xA9zbd67RLY8rGcMqAU3j8jce3GggAUv5PdSPvHUmb5m3Yo90eHLf3cUxcMJHPDPoMAM2LmzOw20C+Nuxr9Gnfp0711UX3Nt05feDpLC9fzoPTs8mWLnroInq27clFR1zEc3Oe45rnr+H6T1zPrRNuZcXaFZQUl/CjY360yXbWVWbHo/pVIJK0KzEU1Ee1ywq55x7m3nwt7fr0psdb85nWswtHNetL/z/9h/2+cjH/araUR2Y8wm0TbuOhsoc2644HCIJvf/DbjJ0xlrbN2/LNf3+TRKK4qJhubbrRvLg5byx9g/6d+/ONw74BwKF9DuXQPocC0L1td+767F2bTBC0vVaWr+SnY3/KorWL+PoLMKMTPDig9uuvWr+KZeXL6NSqE+WV5ZtdJQFs/NbeFFJK/Gvqv2hV3IrLnriMMwaewbi3xtG7fW8+sf8nuHXCrQzpOYS2Ldpy9TNX8/H9Ps6ALgN4avZTm21r7oq5PD7rcY7f53haN2vdZPtQXUVVBSVF/mcrqXH4f5f6SAnmzIHjjqPiR5ey6q3X6LkUlraAQ46ezDHd2nBwqxL6so4169dwwegLWLx28cbV93sHWlbAqz2heVFzOrfqzKgpo5iwYAK92/RmZcVKiilmn077cGifQ1m4euHGwXlbUp9A8OTsJ7n3/85naacF0BJe6gULar6Mv0Yn7nMiB3Q5gMVrFzOk1xDOev9Z2z3jYkNbvX41j896nH0778sJ+5xAh5YdmL18Nof3PZz9uuzHM28+Q2Wq5FMHfoqJCyaytHwp3dp045MHfnKzbfXr2I+rT7i6AHsBD09/mM6tOvOb537jTaV2AA9Pf5hFqxcx4n0jCl2K1KC8JLG+uneHVq0oensBXdfAjPbwpdOgvKiSLz6xnON7H8lbB+3Fza/cvEkgADh4Pgydmz1vUdSCVJUoW1RG5xadOXbvYxnacyidWnZiYNeBzFkxh/Yt2jfajZEqqioYec9IBry+gEF5B8hzfWHGNsYqVj+//vLbL3N0v6OZtWwWayrW0L1N94JPAtSmeRuu+MgVPDnrSd5a/hZXPXUVK8pXcPrA0wH40uAvcVD3gzjtgNO49OhLOWXAKQWttyZVqYoHpz/IynUr+e4R3zUQ7AC6tO5Sp6nCpW1ZvX41byx9o2Cfbyior5tuonL6dN6pWsWqVsW0rIL/9Idb/gELDurHB790KSfueyKL1yzebNW7DoK/Dsmen9T/JBavWcyayjWsWr+KsW+MpUPLDhy3z3H079Kfg3sczEUfvIiqVFWvWyjPXzmf/33uf6mo2vT0RUlRCT3b9OQnx8Kze9Z+e73a9qJLiy70btubi464iEdmPsJVJ1zFxUdeXOcaG8KkBZM4575zuOjBi3h71ducPfhs+nfuz95d9qZ7m+78Z8Z/Clrf9iiKIjq27MjlT17OgC4DeOGtFza5PXV5RTlli8sKWOHuZ0ivIRy797GFLqPe3lr+FmPKxhS6DFVz/9T7+fOLfy7Y5+/0pw8i4iTgt0Ax8OeU0hVNWsADDxCrVtFpNZz5SXhuT6gMGBq9uTLNZ+QT36Z76+6b9RJUV0QRd025a+PrDYPwBvcczAFdD+CBqQ9w6n6n0rZFW377/G9p37w95ww5p07lFkcxzYubbzaCvryinGfnPrvd21u0ahEHdj+QWz51CwO7DWThqoU7xDeovTruxaryVby+8HUOeeMQHpn5CN854jtcePiFFEXRTndefv8u+3PUnkfRullr3lz2JkVRxLDe2QRlD01/iMdnPc6vTvDeBto+by5/k8kLJ3PSvicVupQG8dgbj7G2Yu1OvT+fGfgZhu/fNFdm1WSnnuY4IoqBqcBHgTnAOODMlNJrW1qnQac5fu01qgYNIoCVRfCPQfCDj8LSjq1YeelKVq9bzW0TbuO8B86rcfW2xW05oPsBlM57t54SSjjzfWfSrU03vjbsawzoko3yu27cdayrXMfwA4bTsqQlPdv23Gx7S9Ys4YqnruCiIy6iW5tu27UrfS7vw9zKuVt8/6IPXsSvnn33j06bojbs3WlvmjdrziVHXbKxO35HcOvLt3LVs1fRqVUn5iybw37d9uOBsx4odFl1MnbmWF6c9yLNS5qzf5f9N7nfxQYpJdZVrqNFSdPN/SDtiMbOHEt5ZflOHQoaw/ZMc7xzfV3a3GFAWUppBkBE3A4MB7YYChrS788byn8BFcAd74fJ3eD8Dsez+LD3UxRFtG3RdouBAODA7gfywrkvsGj1In445oc8PfdpKM56Cn55/C83WbZjy478cfwfaVnSklbNWvHFg7+42fbat2jPoX0OpWPLjjw560kmvzO51rf+3VIgKKGE357yW77w/i8wuOdgVpSvoIoqDux6IIf0OoRX336Vo/c6ulaf0RTmr5jP50d9niDo064PRRSxsnxlocuqk18/82t++OgP+cGRP+DwPQ6nd7veNS4XEQaCJvLQ9IcYN3ccPzzqh4UuRTXYFU7pFNrOHgr6AG9Wez0H2Owi8og4FzgXYM89t+OE+Tb8Y3BLTpi4ljM/BeP3hJ8d9TMuOfYHWz3nf+o+p3LNx67h7FFnM/ZLY4Fs0NKvP/5rgmDUlFF8oM/m18EPP2A4HVt2pGPLjkxYMKHGbRcXFW/8xt6tTTfKK8rrvG9XfvhKzhx8Js2KmtGzXdYr0bd9X9q1aMchvQ7ZuNyOFAgAurbuysCuA/nFR37B8AOGb5zXYWc0tNdQhh8wnEuPuZQIJ0zaEQztNXSLd9aUdgU7++mDzwAnppS+kr/+AnBYSumbW1qnwe+SKEnSDmx3ukviHKBvtdd7AFs+MS5JkrZoZw8F44ABEbF3RDQHRgD3FbgmSZJ2Sjv1mIKUUkVEfAN4kOySxL+klCYVuCxJknZKO3UoAEgp/Rv4d6HrkCRpZ7eznz6QJEkNxFAgSZIAQ4EkScoZCiRJEmAokCRJOUOBJEkCDAWSJClnKJAkSYChQJIk5QwFkiQJMBRIkqScoUCSJAGGAkmSlDMUSJIkACKlVOgamlRELARmNeAmuwLvNOD2tDmPcePzGDc+j3Hj8xjXbK+UUrfaLLjbhYKGFhGlKaVhha5jV+Yxbnwe48bnMW58HuP68/SBJEkCDAWSJClnKKi/6wtdwG7AY9z4PMaNz2Pc+DzG9eSYAkmSBNhTIEmScoYCSZIEGArqJSJOiogpEVEWERcXup6dSUS8ERETIuLliCjN2zpHxMMRMS3/2ana8pfkx3lKRJxYrX1ovp2yiPhdREQh9mdHEBF/iYgFETGxWluDHdOIaBERd+Ttz0dEv6bcvx3BFo7xZRHxVv67/HJEnFLtPY/xdoiIvhExNiImR8SkiPhW3u7vcVNJKfmowwMoBqYD+wDNgVeAgYWua2d5AG8AXd/T9v+Ai/PnFwNX5s8H5se3BbB3ftyL8/deAD4IBDAaOLnQ+1bAY3o0MASY2BjHFPgv4A/58xHAHYXe5x3kGF8GXFTDsh7j7T++vYAh+fN2wNT8OPp73EQPewrq7jCgLKU0I6W0DrgdGF7gmnZ2w4Gb8uc3AadVa789pVSeUpoJlAGHRUQvoH1K6dmU/Rd+c7V1djsppSeAxe9pbshjWn1bdwMf2d16ZrZwjLfEY7ydUkrzUkov5s9XAJOBPvh73GQMBXXXB3iz2us5eZtqJwEPRcT4iDg3b+uRUpoH2f8cgO55+5aOdZ/8+Xvb9a6GPKYb10kpVQDLgC6NVvnO5RsR8Wp+emFD17bHuB7ybv1DgOfx97jJGArqrqZk6fWdtfehlNIQ4GTg/Ig4eivLbulY+++g7upyTD3eNbsO6A8MBuYBV+ftHuM6ioi2wD+AC1NKy7e2aA1tHuN6MBTU3Rygb7XXewBzC1TLTielNDf/uQC4l+x0zNt5tx/5zwX54ls61nPy5+9t17sa8phuXCciSoAO1L4rfZeVUno7pVSZUqoC/kT2uwwe4zqJiGZkgeDWlNI9ebO/x03EUFB344ABEbF3RDQnG7ByX4Fr2ilERJuIaLfhOXACMJHs+I3MFxsJjMqf3weMyEcN7w0MAF7IuxFXRMTh+TnBL1ZbR5mGPKbVt3U68Gh+vna3tuGPVe6TZL/L4DHebvnxuAGYnFL6dbW3/D1uKoUe6bgzP4BTyEbHTgd+WOh6dpYH2RUbr+SPSRuOHdl5vUeAafnPztXW+WF+nKdQ7QoDYBjZ/4SnA9eQz9K5Oz6Av5N1X68n+zZ0TkMeU6AlcBfZYK4XgH0Kvc87yDG+BZgAvEr2B6eXx7jOx/dIsq78V4GX88cp/h433cNpjiVJEuDpA0mSlDMUSJIkwFAgSZJyhgJJkgQYCiRJUs5QIGmrIqIyv/vfxIi4KyJa12Ebf46IgfnzH7znvWcaqlZJ9eMliZK2KiJWppTa5s9vBcanTSeWqfP2JO1Y7CmQtD2eBPYFiIhv570HEyPiwrytTUQ8EBGv5O1n5O2PRcSwiLgCaJX3PNyav7cy/3lHRJyy4YMi4saI+HREFEfEVRExLr/p0HlNvdPS7qKk0AVI2jnk88SfDIyJiKHA2cAHyG4w83xEPE42W+XclNKp+Todqm8jpXRxRHwjpTS4ho+4HTgD+Hc+dfhHgK+TzRq4LKV0aES0AJ6OiIdSdqtcSQ3IngJJ29IqIl4GSoHZZHPTHwncm1JalVJaCdwDHEU23e/xEXFlRByVUlq2HZ8zGjgu/8N/MvBESmkN2b0xvpjX8DzZlLcDGmrnJL3LngJJ27Lmvd/s85vMbCalNDXvRTgF+GX+jf6ntfmQlNLaiHgMOJGsx+DvGz4O+GZK6cG67oCk2rGnQFJdPAGcFhGt8ztdfhJ4MiJ6A6tTSn8DfgUMqWHd9fntcWtyO9lpiaOADSHgQeDrG9aJiP3yz5TUwOwpkLTdUkovRsSNZHeZA/hzSumliDgRuCoiqsjuJPj1Gla/Hng1Il5MKX3uPe89BNwM3JdSWrdh20A/4MW8h2IhcFqD7pAkwEsSJUlSztMHkiQJMBRIkqScoUCSJAGGAkmSlDMUSJIkwFAgSZJyhgJJkgTA/wd5XX4JWfOWWgAAAABJRU5ErkJggg==\n", 297 | "text/plain": [ 298 | "
" 299 | ] 300 | }, 301 | "metadata": { 302 | "needs_background": "light" 303 | }, 304 | "output_type": "display_data" 305 | } 306 | ], 307 | "source": [ 308 | "# Plot the samples using columns 1 and 2 of the matrix\n", 309 | "fig, ax = plt.subplots(figsize = (8, 8))\n", 310 | "\n", 311 | "colors = ['red', 'green']\n", 312 | "\n", 313 | "# Color based on the sentiment Y\n", 314 | "ax.scatter(X[:,1], X[:,2], c=[colors[int(k)] for k in Y], s = 0.1) # Plot a dot for each pair of words\n", 315 | "plt.xlabel(\"Positive\")\n", 316 | "plt.ylabel(\"Negative\");" 317 | ] 318 | }, 319 | { 320 | "cell_type": "markdown", 321 | "metadata": {}, 322 | "source": [ 323 | "From the plot, it is evident that the features that we have chosen to represent tweets as numerical vectors allow an almost perfect separation between positive and negative tweets. So you can expect a very high accuracy for this model! \n", 324 | "\n", 325 | "## Plot the model alongside the data\n", 326 | "\n", 327 | "We will draw a gray line to show the cutoff between the positive and negative regions. In other words, the gray line marks the line where $$ z = \\theta * x = 0.$$\n", 328 | "To draw this line, we have to solve the above equation in terms of one of the independent variables.\n", 329 | "\n", 330 | "$$ z = \\theta * x = 0$$\n", 331 | "$$ x = [1, pos, neg] $$\n", 332 | "$$ z(\\theta, x) = \\theta_0+ \\theta_1 * pos + \\theta_2 * neg = 0 $$\n", 333 | "$$ neg = (-\\theta_0 - \\theta_1 * pos) / \\theta_2 $$\n", 334 | "\n", 335 | "The red and green lines that point in the direction of the corresponding sentiment are calculated using a perpendicular line to the separation line calculated in the previous equations(neg function). It must point in the same direction as the derivative of the Logit function, but the magnitude may differ. It is only for a visual representation of the model. \n", 336 | "\n", 337 | "$$direction = pos * \\theta_2 / \\theta_1$$" 338 | ] 339 | }, 340 | { 341 | "cell_type": "code", 342 | "execution_count": 10, 343 | "metadata": {}, 344 | "outputs": [], 345 | "source": [ 346 | "# Equation for the separation plane\n", 347 | "# It give a value in the negative axe as a function of a positive value\n", 348 | "# f(pos, neg, W) = w0 + w1 * pos + w2 * neg = 0\n", 349 | "# s(pos, W) = (w0 - w1 * pos) / w2\n", 350 | "def neg(theta, pos):\n", 351 | " return (-theta[0] - pos * theta[1]) / theta[2]\n", 352 | "\n", 353 | "# Equation for the direction of the sentiments change\n", 354 | "# We don't care about the magnitude of the change. We are only interested \n", 355 | "# in the direction. So this direction is just a perpendicular function to the \n", 356 | "# separation plane\n", 357 | "# df(pos, W) = pos * w2 / w1\n", 358 | "def direction(theta, pos):\n", 359 | " return pos * theta[2] / theta[1]" 360 | ] 361 | }, 362 | { 363 | "cell_type": "markdown", 364 | "metadata": {}, 365 | "source": [ 366 | "The green line in the chart points in the direction where z > 0 and the red line points in the direction where z < 0. The direction of these lines are given by the weights $\\theta_1$ and $\\theta_2$" 367 | ] 368 | }, 369 | { 370 | "cell_type": "code", 371 | "execution_count": 11, 372 | "metadata": {}, 373 | "outputs": [ 374 | { 375 | "data": { 376 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAHjCAYAAACzaWwPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd4VOeB9v/vowISXYjeTbEppplqAzZgbMD03jQqTmJn42RTNs1JNuv8Un7JtbvJbnbfZDd5Y4FGjd6NMaaYXoRpppnei+ggJKHyvH+cIcY2RaAZndHo/lyXLo2Optxy0dw65ynGWouIiIhImNsBREREJDioFIiIiAigUiAiIiI+KgUiIiICqBSIiIiIj0qBiIiIACoFIiIi4qNSICIiIoBKgYiIiPhEuB2grNWpU8e2aNHC7RgiIiJlYvv27ZestXVLct8KVwpatGhBVlaW2zFERETKhDHmREnvq8sHIiIiAqgUiIiIiI9KgYiIiAAqBSIiIuKjUiAiIiKASoGIiIj4qBSIiIgIoFIgIiIiPioFIiIiAqgUiIiIiI9KgYiIiAAqBSIiIuKjUiAiIiKASoGIiIj4qBSIiIgIoFIgIiIiPioFIiIiAqgU+N+GDfCjH7mdQkRE5LFFuB0g5HTtCsXFbqcQERF5bDpT4G9VqkC/fm6nEBEReWwqBSIiIgKoFIiIiIiPSoGIiIgAKgUiIiLio1IgIiIigEqBiIiI+KgUiIiICKBSICIiIj4qBSIiIkHi2rVr7Nmzx7XX1zLHIiIiQeDSpUukpKRQWFhImzZtiIqKKvMMKgUiIiIuO3/+PF6vF2MMCQkJrhQCUCkQERFx1alTp0hLS6Ny5crEx8cTGxvrWhaVAhEREZccPXqUzMxMqlevTnx8PDVr1nQ1j0qBiIiICw4ePMjs2bOJjY3F4/FQrVo1tyOpFIiIiJS1PXv2MH/+fBo2bEhcXBzR0dFuRwJUCkRERMrU9u3bWbJkCc2bN2fKlClUrlzZ7Uh/p1IgIiJSRjZu3MiKFSto06YNEyZMIDIy0u1In6NSICIiEmDWWtasWcPatWtp3749Y8eOJTw83O1YX6JSICIiEkDWWpYvX86WLVvo0qULI0aMICwsOBcUVikQEREJkOLiYpYsWcKOHTvo2bMnQ4YMwRjjdqwHUikQEREJgKKiIubPn8/evXt58cUX6d+/f1AXAlApEBER8buCggJmz57NoUOHGDRoEH369HE7UomoFIiIiPhRfn4+mZmZHD9+nGHDhtG9e3e3I5WYSoGIiIif5ObmkpaWxtmzZxkzZgydOnVyO9JjUSkQERHxg1u3bpGamsqlS5eYOHEibdu2dTvSY1MpEBERKaXr16+TkpLCzZs3mTp1Ki1btnQ70hNRKRARESmFy5cvk5KSQn5+Ph6Ph6ZNm7od6YmpFIiIiDyhCxcu4PV6sdaSkJBAw4YN3Y5UKioFIiIiT+DMmTOkpqYSGRmJx+Ohbt26bkcqNZUCERGRx3T8+HEyMjKoWrUqHo+HmJgYtyP5hUqBiIjIYzh06BCzZs0iJiYGj8dD9erV3Y7kNyoFIiIiJbR3717mzZtH/fr1iYuLo0qVKm5H8iuVAhERkRLYsWMHixcvpmnTpkyZMoWoqCi3I/mdSoGIiMgjbN68meXLl9OqVSsmTZpEZGSk25ECImAbOhtjmhpjVhtj9htj9hpjvu07XtsYs8IYc8j3Oeaex7xtjDlsjDlojBl8z/Fuxpg9vu/90fi2mTLGVDbGzPQd32KMaRGon0dERCoeay1r165l+fLltGvXjsmTJ4dsIYAAlgKgEPgna207oDfwljGmPfBjYKW1tg2w0vc1vu9NBjoAQ4A/GWPCfc/1Z+ANoI3vY4jv+FeAq9ba1sAfgN8F8OcREZEKxFrLihUrWL16NZ07d2b8+PFERIT2CfaAlQJr7Tlr7ce+2zeB/UBjYBQww3e3GcBo3+1RQKa1Nt9aeww4DPQ0xjQEalhrN1lrLZDyhcfcfa45wMsm2DerFhGRoFdcXMzSpUvZtGkTPXr0YNSoUYSFBfLv6OBQJj+h77R+V2ALUN9aew6c4gDU892tMXDqnoed9h1r7Lv9xeOfe4y1thC4DsTe5/XfMMZkGWOysrOz/fNDiYhISCoqKmLBggVs376dPn36MHToUCrK35sBLwXGmGrAXOA71tobD7vrfY7Zhxx/2GM+f8Dav1hru1tru4fCilMiIhIYhYWFzJ49mz179jBw4EAGDRpUYQoBBLgUGGMicQpBmrV2nu/wBd8lAXyfL/qOnwbu3UWiCXDWd7zJfY5/7jHGmAigJnDF/z+JiIiEujt37pCRkcHBgwcZOnQo/fr1cztSmQvk7AMD/A3Yb639/T3fWgQk+G4nAAvvOT7ZN6PgKZwBhVt9lxhuGmN6+54z/guPuftc44FVvnEHIiIiJZaXl0dqairHjh1j1KhR9OzZ0+1IrgjkMMo+gAfYY4zZ6Tv2E+C3wCxjzFeAk8AEAGvtXmPMLGAfzsyFt6y1Rb7H/QMwHYgGlvk+wCkdXmPMYZwzBJMD+POIiEgIysnJITU1lYsXLzJ+/Hjat2/vdiTXmIr2h3X37t1tVlaW2zFERCQI3LhxA6/Xy7Vr15g0aRKtW7d2O5LfGWO2W2u7l+S+oT3hUkRE5AGuXLmC1+vl9u3bxMXF0bx5c7cjuU6lQEREKpyLFy/i9XopKioiISGBRo0auR0pKKgUiIhIhXL27FlSU1MJDw8nMTGRevXqPfpBFYRKgYiIVBgnTpwgPT2d6Oho4uPjqV27ttuRgopKgYiIVAiHDx9m5syZ1KxZk/j4eGrUqOF2pKCjUiAiIiFv//79zJkzh3r16hEXF0fVqlXdjhSUVApERCSk7dq1i4ULF9K4cWOmTZtGVFSU25GClkqBiIiErK1bt7Js2TKeeuopJk+eTKVKldyOFNRUCkREJCStW7eOVatW8cwzzzB+/HgiIvSW9yj6JyQiIiHFWsvKlSvZsGEDHTt2ZNSoUYSHh7sdq1xQKRARkZBhrWXZsmVs27aN5557jmHDhhEWFtANgUOKSoGIiISE4uJiFi1axK5du3j++ed55ZVXcDbXlZJSKRARkXKvsLCQefPmsX//fgYMGEC/fv1UCJ6ASoGIiJRrBQUFzJw5kyNHjjB48GB69+7tdqRyS6VARETKrby8PDIyMjh58iQjRozgueeecztSuaZS4E85OaBVskREysTt27dJTU3lwoULjBs3jmeffdbtSOWehmT6y/HjkJQEN264nUREJOTdvHmT6dOnk52dzaRJk1QI/ERnCvyleXP49a9BG2yIiATU1atX8Xq95OTkMG3aNFq0aOF2pJChMwX+Ygy0aQNpabBpk9tpRERC0qVLl0hOTiY3NxePx6NC4Gc6U+BvNWtCdLTbKUREQs65c+dITU3FGENiYiL169d3O1LIUSnwh9/8Bjp3hmHDYPhwt9OIiIScU6dOkZaWRuXKlYmPjyc2NtbtSCFJlw/84YUXIDkZPv3U7SQiIiHn6NGjeL1eqlatyuuvv65CEEA6U+AP/fs7lw1at3Y7iYhISDlw4ABz5swhNjYWj8dDtWrV3I4U0lQK/KVrV7cTiIiElD179jB//nwaNWrEtGnTiNZ4rYBTKRARkaCzfft2lixZQosWLZg8eTKVK1d2O1KFoFIgIiJBZePGjaxYsYI2bdowYcIEIiMj3Y5UYagUiIhIULDWsmbNGtauXUuHDh0YM2YM4eHhbseqUFQKRETEddZali9fzpYtW+jSpQsjRowgLEwT5MqaSoGIiLiquLiYxYsXs3PnTnr16sXgwYMxxrgdq0JSKRAREdcUFRUxf/589u7dy4svvkj//v1VCFykUiAiIq4oKChg9uzZHDp0iFdeeYUXXnjB7UgVnkqBiIiUufz8fDIyMjhx4gTDhw+nW7dubkcSVApERKSM5ebmkpaWxtmzZxk7diwdO3Z0O5L4qBSIiEiZuXXrFl6vl8uXLzNx4kTatm3rdiS5h0qBiIiUievXr5OSksLNmzeZOnUqLVu2dDuSfIFKgYiIBNzly5dJSUkhPz8fj8dD06ZN3Y4k96FSICIiAXXhwgW8Xi/WWhITE2nQoIHbkeQBVApERCRgTp8+TVpaGpGRkcTHx1OnTh23I8lDqBSIiEhAHD9+nIyMDKpWrUp8fDy1atVyO5I8gkqBiIj43aeffsrs2bOJiYnB4/FQvXp1tyNJCagUiIiIX33yySfMnz+f+vXrExcXR5UqVdyOJCWkUiAiIn7z8ccfs3jxYpo1a8bUqVOpXLmy25HkMagUiIiIX2zevJnly5fTqlUrJk2aRGRkpNuR5DGpFIiISKlYa1m7di1r1qyhXbt2jB07logIvb2UR/q3JiIiT8xay4oVK9i0aROdO3dm5MiRhIWFuR1LnpBKgYiIPJHi4mKWLl3Kxx9/TI8ePRg6dCjGGLdjSSmoFIiIyGMrKipi4cKF7Nmzh759+zJw4EAVghCgUiAiIo+lsLCQOXPmcPDgQV5++WX69u3rdiTxE5UCEREpsTt37pCZmcmxY8d47bXX6NGjh9uRxI9UCkREpETy8vJIS0vjzJkzjB49ms6dO7sdSfxMpUBERB4pJycHr9dLdnY2EyZMoF27dm5HkgBQKRARkYe6ceMGKSkpXL9+nSlTptC6dWu3I0mAqBSIiMgDXblyhZSUFPLy8oiLi6N58+ZuR5IAUikQEZH7unjxIl6vl6KiIuLj42nUqJHbkSTAVApERORLzp49S2pqKuHh4SQlJVG3bl23I0kZUCkQEZHPOXHiBOnp6VSpUoX4+HhiYmLcjiRlRKVARET+7vDhw8ycOZNatWrh8XioUaOG25GkDKkUiIgIAPv27WPu3LnUq1ePuLg4qlat6nYkKWMqBSIiws6dO1m0aBFNmjRh6tSpREVFuR1JXKBSICJSwW3dupVly5bRsmVLJk2aRKVKldyOJC5RKRARqcDWrVvHqlWreOaZZxg/fjwREXpbqMj0b19EpAKy1rJy5Uo2bNhAx44dGTVqFOHh4W7HEpepFIiIVDDWWt577z2ysrLo1q0bw4YNwxjjdiwJAioFIiIVSHFxMQsXLmT37t288MILDBo0SIVA/k6lQESkgigsLGTu3LkcOHCAAQMG0K9fPxUC+RyVAhGRCqCgoICZM2dy5MgRBg8eTO/evd2OJEFIpUBEJMTl5eWRkZHBqVOnGDlyJF27dnU7kgQplQIRkRB2+/ZtUlNTuXDhAuPGjaNDhw5uR5IgplIgIhKibt68idfr5erVq0yePJk2bdq4HUmCnEqBiEgIunr1Kl6vl5ycHKZNm0aLFi3cjiTlgEqBiEiIyc7Oxuv1UlBQQHx8PI0bN3Y7kpQTKgUiIiHk3LlzpKamYowhMTGR+vXrux1JyhGVAhGREHHy5EnS09OJiorC4/EQGxvrdiQpZ1QKRERCwJEjR5g5cybVq1cnPj6emjVruh1JyiGVAhGRcu7AgQPMmTOH2NhYPB4P1apVczuSlFMqBSIi5dju3btZsGABjRo1Ytq0aURHR7sdScoxlQIRkXIqKyuLpUuX0qJFCyZPnkzlypXdjiTlnEqBiEg5tGHDBj788EPatGnDhAkTiIyMdDuShACVAhGRcsRay+rVq1m3bh0dOnRgzJgxhIeHux1LQoRKgYhIOWGt5f3332fr1q107dqV4cOHExYW5nYsCSEB+6/JGPOuMeaiMeaTe469Y4w5Y4zZ6ft47Z7vvW2MOWyMOWiMGXzP8W7GmD2+7/3R+Db/NsZUNsbM9B3fYoxpEaif5bHl5UFGBhQVuZ1EREJEcXExixYtYuvWrfTq1YsRI0aoEIjfBfK/qOnAkPsc/4O1tovv4z0AY0x7YDLQwfeYPxlj7p4P+zPwBtDG93H3Ob8CXLXWtgb+APwuUD/IY7t6Ffbuhdxct5OISAgoKipi7ty57Ny5k5deeonBgwfj+/tIxK8CVgqstWuBKyW8+ygg01qbb609BhwGehpjGgI1rLWbrLUWSAFG3/OYGb7bc4CXTbD8X9KwIfzqV6C5wiJSSgUFBWRmZrJv3z5eeeUV+vfvr0IgAePGuadvGmN2+y4vxPiONQZO3XOf075jjX23v3j8c4+x1hYC14H7rulpjHnDGJNljMnKzs7230/yRQsXws6dgXt+EalQ8vPzSUtL4/DhwwwfPpwXXnjB7UgS4sq6FPwZaAV0Ac4B/+47fr/aax9y/GGP+fJBa/9ire1ure1et27dx0v8OG7fhjt3Avf8IlJh5ObmkpKSwsmTJxk7dizdunVzO5JUAGU6+8Bae+HubWPMX4Elvi9PA03vuWsT4KzveJP7HL/3MaeNMRFATUp+uSIwpkxx9eVFJDTcunULr9fL5cuXmTRpEs8884zbkaSCKNMzBb4xAneNAe7OTFgETPbNKHgKZ0DhVmvtOeCmMaa3b7xAPLDwnsck+G6PB1b5xh2IiJRb165dIzk5matXrzJ16lQVAilTATtTYIzJAPoDdYwxp4F/AfobY7rgnOY/DrwJYK3da4yZBewDCoG3rLV35/P9A85Mhmhgme8D4G+A1xhzGOcMweRA/SwPdeoU1K8PlSq58vIiEjouX75MSkoKd+7cwePx0LRp00c/SMSPTEX747p79+42KyvLf0/4/e9D374wevSj7ysi8gDnz58nNTUVay0ej4cGDRq4HUlChDFmu7W2e0nuqxUNS+udd6BqVbdTiEg5dvr0adLS0qhUqRIej4c6deq4HUkqKJWC0tJaBCJSCseOHSMjI4Nq1aoRHx9PrVq13I4kFZhKgYiISz799FNmzZpF7dq18Xg8VK9e3e1IUsGpFIiIuOCTTz5h/vz5NGjQgGnTplGlShW3I4moFIiIlLWPP/6YxYsX06xZM6ZOnUrlypXdjiQCqBSIiJSpTZs28cEHH9C6dWsmTpxIZGSk25FE/k6lQESkDFhrWbt2LWvWrKFdu3aMGzeO8PDwRz9QpAypFIiIBJi1lhUrVrBp0ya6dOnCiBEjCAtzYz86kYdTKRARCaDi4mKWLl3Kxx9/TM+ePRkyZIi2PpagpVIgIhIgRUVFLFiwgE8++YS+ffsycOBAFQIJaioFIiIBUFhYyOzZs/n00095+eWX6du3r9uRRB5JpUBExM/u3LlDZmYmx44d47XXXqNHjx5uRxIpEZUCERE/ys3NJT09nTNnzjB69Gg6d+7sdiSRElMpEBHxk5ycHLxeL5cuXWLChAm0a9fO7Ugij0WlQETED65fv47X6+X69etMmTKFVq1auR1J5LGpFIiIlNKVK1dISUkhLy8Pj8dDs2bN3I4k8kRUCkRESuHixYt4vV6KiopISEigYcOGbkcSeWIqBSIiT+jMmTOkpaURERFBUlISdevWdTuSSKmoFIiIPIETJ06Qnp5OlSpViI+PJyYmxu1IIqWmUiAi8pgOHz7MzJkzqVWrFh6Phxo1argdScQvVApERB7Dvn37mDt3LvXq1SMuLo6qVau6HUnEb1QKRERKaOfOnSxatIgmTZowdepUoqKi3I4k4lcqBSIiJbBlyxbef/99WrZsyaRJk6hUqZLbkUT8TqVAROQhrLWsX7+eVatW0bZtW8aNG0dEhH51SmjSf9kiIg9grWXlypVs2LCBjh07MmrUKMLDw92OJRIwKgUiIvdhreW9994jKyuLbt26MWzYMIwxbscSCSiVAhGRLyguLmbhwoXs3r2bF154gUGDBqkQSIWgUiAico/CwkLmzp3LgQMHGDhwIH379lUhkApDpUBExOfOnTvMnDmTo0ePMmTIEHr16uV2JJEypVIgIgLk5eWRnp7O6dOnGTlyJF27dnU7kkiZUykQkQovJyeHtLQ0Lly4wLhx4+jQoYPbkURcoVIgIhXajRs38Hq9XLt2jcmTJ9OmTRu3I4m4RqVARCqsq1evkpKSwu3bt5k2bRotWrRwO5KIq1QKRKRCys7Oxuv1UlBQQHx8PI0bN3Y7kojrVApEpMI5d+4cqampGGNITEykfv36bkcSCQoqBSJSoZw8eZL09HSioqLweDzExsa6HUkkaKgUiEiFceTIEWbOnEmNGjXweDzUrFnT7UgiQUWlQEQqhP379zN37lzq1KlDXFwc1apVczuSSNAJczuAPMT16/DrX4O1bicRKdd2797N7NmzadCgAQkJCSoEIg+gUhDM1q+HOXMgN9ftJCLlVlZWFvPnz6d58+Z4PB6io6PdjiQStHT5IJgNG+Z8iMgT2bBhAx9++CFPP/00EyZMICJCv/JEHkb/h4hIyLHWsnr1atatW8ezzz7L6NGjCQ8PdzuWSNBTKRCRkGKt5f3332fr1q107dqV4cOHExamK6UiJaFSICIho7i4mMWLF7Nz50569+7Nq6++ijHG7Vgi5UaJS4ExpjnQxlr7oTEmGoiw1t4MXDQRkZIrKipi3rx57Nu3j5deeomXXnpJhUDkMZXonJox5mvAHOB/fYeaAAsCFUpE5HEUFBSQmZnJvn37ePXVV+nfv78KgcgTKOmFtreAPsANAGvtIaBeoEKJiJRUfn4+aWlpHD58mOHDh/P888+7HUmk3Crp5YN8a+2du83bGBMBaEUdEXHV7du3SUtL4/z584wbN45nn33W7Ugi5VpJS8FHxpifANHGmFeAbwCLAxdLROThbt68SWpqKpcvX2bixIk888wzbkcSKfdKevngx0A2sAd4E3gP+FmgQomIPMy1a9eYPn06V69eZdq0aSoEIn5S0jMFo4AUa+1fAxlGRORRLl26hNfr5c6dO8THx9OkSRO3I4mEjJKeKRgJfGqM8RpjhvnGFMjD7N0L+flupxAJKefPnyc5OZmioiISEhJUCET8rESlwFqbBLQGZgNTgSPGmP8byGDl3l//CmvXup1CJGScPn2aGTNmEBERQWJiIg0aNHA7kkjIKfFf/NbaAmPMMpxZB9E4lxS+Gqhg5d6//Rto8xURvzh27BgZGRlUq1aN+Ph4atWq5XYkkZBU0sWLhhhjpgOHgfHA/wUaBjBX+adCIOIXBw8eJC0tjZiYGJKSklQIRAKopO9ciUAm8Ka1VhfKH9ft21C5MmiXNpHH8sknnzB//nwaNGjAtGnTqFKlituRREJaiUqBtXZyoIOEtN/9Dp5+GqZNczuJSLmxfft2lixZQvPmzZkyZQqVK1d2O5JIyHtoKTDGrLfW9jXG3OTzKxgawFprawQ0Xah4803QKU+REtu0aRMffPABrVu3ZuLEiURGRrodSaRCeGgpsNb29X2uXjZxQlSjRm4nECkXrLV89NFHfPTRR7Rv356xY8cSrstuImWmpAMNvSU5Jo/wySdw+LDbKUSCkrWWDz74gI8++oguXbowbtw4FQKRMlbSgYYd7v3Ct3hRN//HKcfeew/y8mDs2AffZ+NGqFEDWrcuu1wi5UBxcTFLlixhx44d9OzZkyFDhmjrYxEXPGpMwdvA3Y2Qbtw9DNwB/hLgbOVHbi7ExEBWlnMm4EFv+m+8Uba5RMqBoqIi5s+fz969e+nXrx8DBgxQIRBxyUMvH1hr/3/feIJ/tdbW8H1Ut9bGWmvfLqOMwe3gQZg4EXbscNYmOHnS7UQi5UZhYSGzZs1i7969DBo0iIEDB6oQiLiopFMS3zbGxABtgKh7jmsd3//5H+fswLx5kJYG9eu7nUikXLhz5w6ZmZkcO3aM1157jR49ergdSaTCK1EpMMZ8Ffg20ATYCfQGNgEDAxetHCgshPffh+hoZ6yA/sIRKZHc3FzS09M5c+YMY8aMoVOnTm5HEhFKvkvit4EewAlr7QCgK5AdsFTlxcmTcO0axMVBejrUq+d2IpGgd+vWLWbMmMG5c+eYOHGiCoFIECnp7IM8a22eMQZjTGVr7QFjzDMBTVYetGwJR46All4VKZHr16/j9Xq5ceMGU6ZMoVWrVm5HEpF7lLQUnDbG1AIWACuMMVeBs4GLVY6oEIiUyJUrV0hJSSEvL4+4uDiaNWvmdiQR+YKSDjQc47v5jjFmNVATeD9gqUQkpFy8eBGv10txcTEJCQk0bKhNVkWCUUkHGta+58s9vs/2fvcVEbnXmTNnSEtLIyIigsTEROrWret2JBF5gJJePvgYaApcxVm8qBZwzhhzEfiatXZ7gPKJSDl2/PhxMjIyqFKlCvHx8cTExLgdSUQeoqSzD94HXrPW1rHWxgJDgVnAN4A/BSqciJRfhw4dIi0tjRo1apCUlKRCIFIOlLQUdLfWLr/7hbX2A+BFa+1mQJuci8jn7N27l8zMTOrUqUNiYiI1amiXdZHyoKSXD64YY34EZPq+ngRcNcaEA8UBSSYi5dKOHTtYvHgxTZo0YerUqURFRT36QSISFEp6pmAqzmqGC3wfTX3HwoGJgYkmIuXNli1bWLRoES1btiQuLk6FQKScKemUxEvAt4wx1ay1t77w7cP+jxWEbt2CatXcTiESlKy1rFu3jtWrV9O2bVvGjRtHRERJT0SKSLAo0ZkCY8wLxph9wD7f152NMRVngOGFCxAbC506ORsgXb3qdiKRoGGt5cMPP2T16tV06tSJCRMmqBCIlFMlvXzwB2AwcBnAWrsLeDFQoYJOvXrOyoV79sD3vw8NG8LQobBoERQUuJ1OxDXWWpYuXcrGjRvp3r07o0ePJiyspL9WRCTYlPj/XmvtqS8cKvJzluBlDIwY4dzOyYH8fGd3xLg45wzCm29CVhZYreckFUdxcTELFixg+/bt9OnTh9deew2jnUJFyrWSloJTxpgXAGuMqWSM+T6wP4C5gs/EiVC9+ueP3bzpfPztb/DSS/D88+5kEyljhYWFzJ49m927dzNw4EAGDRqkQiASAkpaCr4OvAU0Bk4DXXxfP5Ax5l1jzEVjzCf3HKttjFlhjDnk+xxzz/feNsYcNsYcNMYMvud4N2PMHt/3/mh8v3mMMZWNMTN9x7cYY1qU9Id+Ii+/DHfu3P97Rb6TJv37BzSCSDC4c+cOGRkZHDhwgCFDhtCvXz+3I4mIn5SoFFhrL1lrp1lr61tr61lr46y1lx/xsOnAkC8c+zGw0lrbBljp+xpjTHtgMtDB95g/+dZAAPgz8AbQxvdDcjZqAAAgAElEQVRx9zm/Aly11rbGGfPwu5L8LE8sOvrBZwKqVIGf/hR++9uARhBxW15eHqmpqRw7doxRo0bRq1cvtyOJiB89dIiwMebnD/m2tdb+8iHfXHufv95HAf19t2cAa4Af+Y5nWmvzgWPGmMNAT2PMcaCGtXaTL08KMBpY5nvMO77nmgP8tzHGWBvAC/tTpsDWrXD79ueP/+QnzodICMvJySE1NZWLFy8yfvx42rdv73YkEfGzR50pyLnPBzh/pf/oCV6vvrX2HIDvcz3f8cbAvQMZT/uO3b1c8cXjn3uMtbYQuA7E3u9FjTFvGGOyjDFZ2dnZTxDbZ8SIzy4VgHP2AOBnP3MGIIqEqBs3bjB9+nQuXbrE5MmTVQhEQtRDS4G19t/vfgB/AaKBJJzljlv6Mcf9RijZhxx/2GO+fNDav1hru1tru5dq29aGDaF5c+d2dDSkp3921qBatQePORApx65evUpycjI3btwgLi6ONm3auB1JRALkkWMKfIMDfwXsxrnc8Jy19kfW2otP8HoXjDENfc/bELj7HKdxlk6+qwlw1ne8yX2Of+4xxpgIoCZw5QkyPZ7JkyEyEhYuhNGjnXJwdzGjypWhWFtBSOjIzs7m3XffJT8/n/j4eJrfLcUiEpIeWgqMMf8KbANuAh2tte9Ya0uznN8iIMF3OwFYeM/xyb4ZBU/hDCjc6rvEcNMY09s36yD+C4+5+1zjgVUBHU9w189/DqdOwSuvfHasVi04d865HR6u9QokJJw7d47p06cDkJiYSOPGjR/+ABEp98zD3keNMcVAPlDI50/NG5yBhg/cD9UYk4EzqLAOcAH4F5zNlGYBzYCTwARr7RXf/X8KvO57re9Ya5f5jnfHmckQjTPA8FvWWmuMiQK8QFecMwSTrbVHH/UDd+/e3WZlZT3qbk/m6FFo1co5k6BLCVKOnTx5kvT0dKKiooiPj6d27dpuRxKRJ2SM2W6t7V6i+5bFH9fBJKClAGD3bujcGZ56yikJIuXMkSNHyMzMpGbNmng8HmrWrOl2JBEphccpBVqk3N86dYJ16+DYMdCiLlLO7N+/n4yMDGJjY0lKSlIhEKlgVAoCoW9fWLwY1q+HSZPcTiNSIrt27WL27Nk0bNiQhIQEqlat6nYkESljKgWBMnw4pKTArFnw7W+7nUbkobZt28aCBQto0aIFHo+H6LtrcIhIhaJNzwPJ44FLl+B733O2X/7pT91OJPIl69evZ+XKlTz99NNMmDCBiAj9WhCpqPR/f6B997uQne2seli3LrzxhtuJRACw1rJq1SrWr1/Ps88+y+jRowkPD3/0A0UkZKkUlIXf/AYuXIA334TatWH8eLcTSQVnrWXZsmVs27aN5557jmHDhhEWpquJIhWdfguUlb/9DYYOhQkTYOVKt9NIBVZcXMyiRYvYtm0bvXv3Zvjw4SoEIgKoFJSt995z1jAYNAi2bXM7jVRARUVFzJ07l507d9K/f39effVVnMVCRURUCsrezp3OJYSePeHAAbfTSAVSUFBAZmYm+/bt49VXX+Wll15SIRCRz1EpcMPly87ndu2cfRREAiw/P5/U1FQOHz7MiBEjeP75592OJCJBSKXALXd3U2zWzJm2KBIgt2/fJiUlhdOnTzNu3Diee+45tyOJSJDS7AO3GAMFBc7mSXXrws2bUK2a26kkxNy8eROv18uVK1eYNGkSTz/9tNuRRCSIqRS4KSICcnMhOhqqV4e8PKhc2e1UEiKuXbtGSkoKt27dYtq0aTz11FNuRxKRIKfLB26LioLr1z+7XVTkbh4JCZcuXSI5OZnc3Fzi4+NVCESkRFQKgkGNGs7iRuCcPahg21mLf50/f57k5GSKiopITEykSZMmbkcSkXJCpSBY1KvnbLcMoIVk5AmdOnWKGTNmEBERQVJSEvXr13c7koiUI3r3CSYtWsCePc7txo1djSLlz9GjR/F6vVSpUoWkpCRiY2PdjiQi5YxKQbB59lnYuBHOnoVevdxOI+XEwYMHSU9PJyYmhqSkJGrVquV2JBEph1QKgtHzz8OyZbB1K4wZ43YaCXJ79uxh5syZ1K9fn8TERKppaquIPCGVgmA1ZAikp8OCBfCNb7idRoLU9u3bmTdvHs2aNSM+Pp7o6Gi3I4lIOaZ1CoLZlCnOksjf+pYzEPGdd9xOJEFk48aNrFixgtatWzNx4kQiIyPdjiQi5ZxKQbD75jedZZB/8Qtn5cO33nI7kbjMWstHH33ERx99RPv27Rk7dizh4eFuxxKREKBSUB68846zjsE3v+nssDhlituJxCXWWj744AM2b95Mly5dGDFiBGGawioifqJSUF78+c9w7hxMnQoxMc6YA6lQiouLWbJkCTt27KBnz54MGTJEWx+LiF/pT4yycPo0/OpXcOdO6Z5nwQLo3h2GDoXNm/2TTcqFoqIi5s2bx44dO+jXr58KgYgEhEpBWaheHZo2dZYwLq1t26BBA2fa4t69pX8+CXoFBQXMnDmTvXv3MmjQIAYOHKhCICIBoVIQaDNmOOMBEhK+vHxxcTHk5z/+c54753x+9lk4caL0GSVo5efnk56ezqFDhxg2bBh9+vRxO5KIhDCNKQi04uIHb3A0Y4azcmHLltC6NfTo8XjPGxbmLI184YIzZVFCSm5uLmlpaZw9e5YxY8bQqVMntyOJSIhTKQi0pKQHf2/cOLh507kk8LhzzI2BwkLnkkT9+s72yzVqlC6rBI1bt26RmprKpUuXmDhxIm3btnU7kohUACoFZeH8eeeN+4vXgWvUcD6edPOj8HDIy4OoKKhZE3JzndtSrl2/fp2UlBRu3rzJlClTaNWqlduRRKSCUCkIhP374dNPnfECo0bBt78Nw4dDTg58/ev+fa3KleHGDadcREc7Zw+0kE25dfnyZbxeL3l5ecTFxdGsWTO3I4lIBaJSEAirVzvLEzdq5BSEli2dN+6nnw7M61WvDtnZzoqHERHOeAONTi93Lly4gNfrxVpLQkICDRs2dDuSiFQwKgX+VlwMycnOtMGaNeHiRWcFwkaNoE6dwL1unTrOTITmzZ0BiA8a3ChB6cyZM6SmphIZGYnH46Fu3bpuRxKRCkhTEv2tWzdnsaIOHWDsWJg2DTp1csYVbN8e2Ndu1gz27XNuB7KAiF8dP36clJQUoqKiSEpKUiEQEdeoFPjLhQuwfj2cOQNf+xr89rdw6xZ4vXDokLPQUFksNtSuHWzZ4ly+6No18K8npXLo0CHS0tKoUaMGSUlJxMTEuB1JRCowlYLSKihwzgh06OAMJqxZE3bvdk7fx8XBD38I//Efzl/u8fEle85nnoG//e3JM/XsCR98ADt3wrBhT/48ElB79+4lMzOTunXrkpSURA1NKRURlxlbwa49d+/e3WZlZfnvCXv1giNHoFo1Z3ZBQYFTAA4c+GxZ47vTBkvq7iDB0v67mT0bJk6E118vXckQv9uxYweLFy+madOmTJkyhShNJRWRADHGbLfWdi/JfXWmoLQKC50dC6dOdVYXrFQJ+vWD3/0ONm1y7vO4v/DDwuCll0qfbcIEZ3fFd9+Fn/yk9M8nfrF582YWLVpEy5YtiYuLUyEQkaCh2QelNXMmfPe78N//7ZSB2FhnzYAePZ58CmJRkf/yff3rcOkS/PM/O1MWv/td/z23PBZrLevWrWP16tW0bduWcePGEeGPTbJERPxEv5FKIycHBgxw1gioXNl58/3LX+D3v4eBA/2zK6I//OxnzkDI733PubTh8bidqMKx1vLhhx+yceNGOnXqxKhRowj74gZZIiIuC5J3rXIqOto51V+pkrNy4YgR0Lmzc/0+WArBXf/1X860yPh4qF1bAxDLkLWWpUuXsn37drp3785rr72mrY9FJCgF2TtXObN4sbMmQWKi86ZbpYpzPNgKwV2zZ0Pfvs4siXXrnNsSUEVFRSxcuJA9e/bQp08fXn75ZRUCEQlaOn9ZGsOGOdsf/+1vnxWCYLd+vbPqYb9+ztRJCZjCwkJmz57Nnj17GDhwIIMGDVIhEJGgplJQGhERzloE5c3x485lj86d4ehRt9OEpDt37pCRkcHBgwcZOnQo/fr1czuSiMgjqRRUVIWFzudWrZyxBuI3eXl5pKamcuzYMUaNGkXPnj3djiQiUiJBevFbAs4YZ+pjeDg0bAjXrjmrMUqp5OTkkJqaysWLFxk/fjzt27d3O5KISImpFFRkYWGQn+9Mp6xVC27fdmZUyBO5ceMGXq+Xa9euMWXKFFq3bu12JBGRx6JSUNFVquRs3FStmjNYsqAgeGdPBLErV67g9Xq5ffs2cXFxNG/e3O1IIiKPTWMKBKpWdXZVBIiMLP2eCxVMdnY2ycnJ5Ofnk5CQoEIgIuWWSoE4atd21lwA57KCikGJnD17luTkZAASExNp1KiRy4lERJ6cSoF8pnFjOHjQua1tfB/pxIkTpKSkUKlSJZKSkqhXr57bkURESkWlQD7v6adh+3ZnnEG7dm6nCVqHDx8mNTWVatWq8frrr1O7dm23I4mIlJpKgXzZc8/BqlVw4AAMGuR2mqCzf/9+MjIyiI2NJSkpiRo6qyIiIUKlQO5vwACYNw9WrnQ2URIAdu3axezZs2nUqBEJCQlUrVrV7UgiIn6jUiAPNmYM/PWv4PXCD3/odhrXbdu2jQULFtCiRQs8Hg/RWtNBREKMJqTLw331q3DpErz9NtStCz/4gduJXLF+/XpWrlzJM888w/jx44nQWg4iEoL0m00e7cc/hosXnbMFsbHw+utuJyoz1lpWrVrF+vXr6dixI6NGjSI8PNztWCIiAaFSICXz+9/DhQvwla84axqMHu12ooCz1rJs2TK2bdvGc889x7BhwwgL0xU3EQld+g0nJZeW5gxAHDMGPvrI7TQBVVxczMKFC9m2bRvPP/88w4cPVyEQkZCn33LyeFatgjZtoH9/2LHD7TQBUVhYyJw5c9i1axf9+/fnlVdewRjjdiwRkYBTKZDH9+mnEBXlrGdw+LDbafyqoKCAzMxM9u/fz+DBg3nppZdUCESkwlApkCdz+7bzuU0bOHvW3Sx+kpeXR2pqKkeOHGHEiBH07t3b7UgiImVKpUCejDFQVOTcbtwYrl51N08p3b59m5SUFE6fPs24ceN47rnn3I4kIlLmNPtAnlxYGNy5A5UqOTMSbt1ytmEuZ27evInX6+XKlStMmjSJp59+2u1IIiKuUCmQ0omMhJwcpwxUq+aUhMhIt1OV2LVr10hJSSEnJ4dp06bx1FNPuR1JRMQ1unwgpVelymeXDypVguJid/OU0KVLl3j33XfJzc3F4/GoEIhIhadSIP5Rq9ZnAw7Dw8Fad/M8wrlz50hOTqa4uJjExESaNGnidiQREdepFIj/NGz42RTFqCh3szzEqVOnmDFjBhERESQlJVG/fn23I4mIBAWVAvGvVq2cRY3u3HFuB5mjR4/i9XqpWrUqr7/+OrGxsW5HEhEJGioF4n9dujjLIB89Ci++6Haavzt48CDp6enExMSQlJREzZo13Y4kIhJUVAokMF58ERYuhHXrYNIkt9OwZ88eZs6cSYMGDUhMTKRatWpuRxIRCToqBRI4I0dCcjLMmgXf+Y5rMbZv3868efNo3rw5Ho+H6Oho17KIiAQzrVMggZWYCJcvw/e/D/XqwU9+UqYvv3HjRlasWEGbNm2YMGECkeVoDQURkbKmUiCB90//BNnZ8NOfQp068MYbAX9Jay1r1qxh7dq1tG/fnrFjxxIeHh7w1xURKc9UCqRs/Pa3cP48vPkmxMbCuHEBeylrLcuXL2fLli106dKFESNGEBamK2UiIo+i35RSdqZPh1dfhfHjYeXKgLxEcXExixcvZsuWLfTq1YuRI0eqEIiIlJB+W0rZWr4cOnSAQYNg2za/PnVRURHz5s1jx44dvPjiiwwePBhjjF9fQ0QklKkUBLudO+Gtt+D2bSgsdDuNf3zyCdSoAT17woEDfnnKgoICZs6cyd69exk0aBADBgxQIRAReUwqBcGufXuYMgX++Efn9HuouH7d+dyuHZw6Vaqnys/PJz09nUOHDjFs2DD69Onjh4AiIhWPBhoGu0qVoG9faNMGQm1+fXExhIVBs2Zw6ZIzAPEx5ebmkpaWxtmzZxk7diwdO3YMQFARkYrBlTMFxpjjxpg9xpidxpgs37HaxpgVxphDvs8x99z/bWPMYWPMQWPM4HuOd/M9z2FjzB9NKJ8vrl/fOeUeSoyBggLndp06cOvWYz381q1bTJ8+nfPnzzNx4kQVAhGRUnLz8sEAa20Xa21339c/BlZaa9sAK31fY4xpD0wGOgBDgD8ZY+5OOP8z8AbQxvcxpAzzl97ChZCa+tnXJ0/C8eOffZ2X59wnyLchLpWICMjNdW5Xr+5spFQC169fJzk5matXrzJ16lTatm0bwJAiIhVDMI0pGAXM8N2eAYy+53imtTbfWnsMOAz0NMY0BGpYazdZay2Qcs9jgpu1sGQJNG3qXE9ftcq5xv7++7Bs2Wf3O30a1q8v8RtluRUVBdeuObcrV4aioofe/fLly7z77rvk5OTg8Xho2bJlGYQUEQl9bpUCC3xgjNlujLm7vF19a+05AN/ner7jjYF7R6Kd9h1r7Lv9xeNfYox5wxiTZYzJys7O9uOP8ZgOHXJmE9y8CStWOMv+dusGR47AN78JXbs6c/iXLHHu37o1/Ou/Om+Uoa5mTWdxI3DOHjzg7MiFCxdITk6msLCQhIQEmjZtWoYhRURCm1uloI+19jlgKPCWMeZh++veb5yAfcjxLx+09i/W2u7W2u5169Z9/LT+smsXbN3qjA34z/903gTnzHF2EvzBD6B7d+cSwiefuJfRTfXrO9stgzMA8QtOnz7N9OnTCQsLIykpiYYNG5ZxQBGR0OZKKbDWnvV9vgjMB3oCF3yXBPB9vui7+2ng3j8HmwBnfceb3Od4cMrJccrA17722bH//V/nTXDoUOd7xjizDTp3di+n2556Cnbvdm43+exf7/Hjx/F6vURHR/P6669Tp04dlwKKiISuMi8Fxpiqxpjqd28DrwKfAIuABN/dEoCFvtuLgMnGmMrGmKdwBhRu9V1iuGmM6e2bdRB/z2OCS1GRUwDefBNefx0mTgSvF06ccFb3mzHjs9X9bt6Eixcf/nyhrmNHZyzFmTPQuzeffvopaWlp1KxZk6SkJGrVquV2QhGRkOTGOgX1gfm+2YMRQLq19n1jzDZgljHmK8BJYAKAtXavMWYWsA8oBN6y1t4difYPwHQgGljm+wg+p0/D/v2QkuJcL//zn+E3v3FWKFy/3pll0KgRTJgAL7zgfFR0ffrA0qXs/eEPmZueToNGjYiLi6NKlSpuJxMRCVllXgqstUeBL50ft9ZeBl5+wGN+Dfz6PsezgGf9ndGvFi6ETp1g9WrnUsGGDc618wsXoFYtCA93BtmFhTmD60J4qYXHtaNhQxZPmMCVvBNcqJ/NG1UCv+WyiEhFphUNA2XJEmfg4JtvQvPmcPkynDvnTC+8ft2ZeZCfDw0bOmcJvvc9FYJ7bN68meXLl9OqVStqPFWT7374XerWrMs7/d9xO5qISMhSKQiE4mLYtMlZvnfYMGfsQIMGzpkCa52zAy1bwscfQ2Qk/OhHzh4HD7JsGTz/vHNmIcRZa1m7di1r1qyhXbt2jB07loiICC7lX+IXH/2CulXq8lbPt9yOKSISkoJp8aLyr7jY+RwW5pwR+P3vnYWJVq367CwBOEv6fu1rzpmB5s2dtQrAGV+wcaNz/N6zBtu2OWschDhrLStWrGDNmjV07tyZ8ePHExHh9NZfDfwVX+36Vb657JtkfpLpclIRkdCkMwX+9ItfQJcuMGaMszjR0aPOIEJrP9sVEJyljN96y5mGmJgIV686b/pXrtx3fj4//3lZ/QSuKS4u5r333mP79u306NGDoUOHfmnr47+O/Cvnc84zZe4UYqJiGNx68AOeTUREnoTOFPjTpEnQuzdMnepsd3znzmcb/oAz8yAszFmpsHVrZ07+hAnwne/AG284Zw2GD3dKRCjvd/AFRUVFLFiwgO3bt9O3b9/7FoK7Fk9ZTNcGXRmSNoTNpzeXcVIRkdCmUuBPt2/D178O8+bBL3/pDCS8V2GhszjRtWvOJYTmzZ2zBeHh0Latc6yCKSwsZPbs2ezZs4eXX36Zl19++YGF4K6P3/yYelXr8fzfnmdf9r4ySioiEvpUCvzlxg3nksD69c5f+Tk5979fbKwz82DXLufMQV4efOtb8Ic/QOP7bt0Qsu7cuUN6ejoHDx5k6NCh9O3bt8SPvfD9CwB0+FMHTlw7EaiIIiIVikqBv9y65UwtvHXrwbsahofDiBHOMsZDhjiXEC5fdsYfzJrlnGk4ftwZX/D++2Uav6zl5eXh9Xo5fvw4o0aNomfPno/9HMU/dwZ2tvjPFlzMqeCrQIqI+IEGGvrLrl3w4YcPLgRVqzprEsye7cxSyM+HmBhnjEFkJHzlK7B0qbOL4vDhsHmzM/Bw6tSy/TnKQE5ODqmpqVy8eJHx48fT/mHTMR/CGEPhPxcS8csI6v9bfa7/+Do1Ktfwc1oRkYpDpcAfli+HUaM+P6jwiwoKnMLQrx+MHAlZWc7lg3/8R6cwgLNt8siRzlbJ1ao56xiEmBs3bpCSksL169eZMmUKrVu3LtXzhYeFk/fTPKJ+HUXN39Yk96e5REVE+SmtiEjFossH/vCTnzy8EIBzRsAYZ02CX/7SGXcwcqSzJ8JdxjiFAJxNgRIS7v9c5dSVK1d49913uXXrFnFxcaUuBHdVjqjMjR/fACD619EUFRc94hEiInI/KgWltXHjo/+ij4hwzhJUqgQnT0KrVs5UxOvX4eDBBz+uKHTe3C5evEhycjJ37twhPj6e5s2b+/X5q1euzsXvO+MKIn4Zga1AUzpFRPzFVLRfnt27d7dZWVn+e8KS7FfQq5dTAPLyoHZt+K//evROiAsWOCsZ/vpL+0CVO2fPniU1NZXw8HA8Hg/16tUL2GuduHaCFv/ZAgD7LxXrv20Rkfsxxmy31nYvyX11piDQIiOdVQv793cuCbz7bsm2Rh40COLiAp0u4E6cOMGMGTOoXLkySUlJAS0EAM1rNWfvN/YCUO9fA/taIiKhRqUg0Dp3hgEDnIWJJk+GZ54p2eOqVYN27QKbLcAOHz5MamoqNWrUICkpidq1a5fJ67av257NX9lM9u1suv5v1zJ5TRGRUKBSUBo/+9nDv//KK876Ax07OjMUpk6FqIoxMn7fvn1kZGRQp04dEhMTqVGjbKcK9mrSi/envc/O8zsZnj68TF9bRKS8UikojTlz7n/81VehTx/44ANnbYKhQ+Hf/q1ss7lo586dzJkzh0aNGpGQkEDVu1Muy9jg1oPJGJfB0kNL+eqir7qSQUSkPNE6BaVQePAg4b7bBqBBA2d1wmef/exOUVGwcKEL6dyxdetWli1bxlNPPcXkyZOpVKmSq3kmPzuZK7lXeOu9t6hftT6/frn8D9wUEQkUlYJSiACKAQuEZ2dXyA2N7rVu3TpWrVrFM888w/jx44mICI7/vL7R4xtk52TzzkfvUK9qPb7d+9tuRxIRCUrB8Vu7vLJW118Aay0rV65kw4YNdOzYkVGjRhEeHv7oB5ahf+n/L1zMuch3ln+H2CqxxHUq/zM7RET8TaVASsVay3vvvUdWVhbdunVj2LBhj9z62C3/Z9j/4XzOeTzzPcRExTDs6WFuRxIRCSr6Q1eeWHFxMQsXLiQrK4vnn38+qAvBXXMnzqV3k94MzxjOhpMb3I4jIhJUVArkiRQWFjJnzhx27drFgAEDeOWVV4K+ENy16SubaFKjCX2T+7L7wm6344iIBA2VAnlsBQUFZGZmsn//fgYPHsyLL75YbgrBXae+ewqAzv/TmaNXj7qcRkQkOKgUyGPJy8sjNTWVo0ePMnLkSHr37u12pCdW/PNiAFr9sRUXbl1wOY2IiPtUCqTEbt++TUpKCqdPn2bcuHF07Vq+lxA2xlD4z4UANPj3BlzPu+5yIhERd6kUSIncvHmT6dOnk52dzaRJk+jQoYPbkfwiPCyc/J/lA1Drd7XILch1OZGIiHtUCuSRrl69SnJyMtevX2fatGk8/fTTbkfyq0rhlbj59k0AqvymCoXFhS4nEhFxh0qBPNSlS5dITk4mNzeX+Ph4WrRo4XakgKhWqRqXfnAJgMhfRmKtdTmRiEjZUymQBzp37hzJyckUFxeTmJhI48aN3Y4UULFVYv8+KyHs/9P/GiJS8eg3n9zXqVOnmDFjBhERESQlJVG/fn23I5WJJjWasP+t/QDU/G1Nl9OIiJQtlQL5kqNHj+L1eqlatSqvv/46sbGxbkcqU23rtGXb17ZxI/8GHf4UGgMqRURKQqVAPufAgQOkp6cTExNDUlISNWtWzL+Wuzfqzsr4lezL3ser3lfdjiMiUiZUCuTvdu/ezaxZs2jQoAGJiYlUq1bN7UiuGvjUQOZMmMOKoytIWJDgdhwRkYBTKRAAsrKymD9/Ps2bN8fj8RAdHe12pKAwrv04/jL8L6TsSuFHK37kdhwRkYDS1snChg0b+PDDD2nTpg0TJkwgMjLS7UhB5Wvdvsal25f4yaqfULdqXb7/wvfdjiQiEhAqBRWYtZY1a9awdu1aOnTowJgxYwgPD3c7VlB6u9/bXMi5wA9W/IA6VeqQ2CXR7UgiIn6nUlBBWWtZvnw5W7ZsoWvXrgwfPpywMF1Nepj/GPIfnL91nqSFScRExTCq7Si3I4mI+JXeBSqg4uJiFi1axJYtW+jVqxcjRoxQISihzPGZvNj8RUbPHM3aE2vdjiMi4ld6J6hgioqKmDt3Ljt37uTFF19k8ODBGGPcjlWufJT4ES1jWvLS9JfYeX6n23FERPxGpaACKSgoYObMmezbt49XXnmFAQMGqBA8oSP/eITIsLuAGsYAACAASURBVEi6/m9XDl857HYcERG/UCmoIP5fe/cdH1WVPn78cya9ElIh9BoIKCWAgBQL0vRnEFGKIMaC6y64FlwLoi66NsSV/brq7opglOICiyBIU2lKJ/QWAhggnSSkt5k5vz9mCCWJknpTnvfrdV+5c9s89zA6z5xz7jkFBQUsXLiQU6dOcc8999C/f3+jQ6rzLk+53OH/OpCQlWBwNEIIUXmSFDQAeXl5fPXVV5w7d47Ro0cTFhZmdEj1glIKy2sWAII/DCY9L93giIQQonIkKajnsrOzWbBgAYmJiYwdO5abbrrJ6JDqFZMyUfhqIQC+7/uSW5RrcERCCFFxkhTUY5cuXWL+/Pmkp6czYcIEQkJCjA6pXnJycCLnlRwAPN72oMhSZHBEQghRMZIU1FOpqanMnz+fnJwcJk2aRNu2bY0OqV5zd3In7S9pADi/5YxVWw2OSAghyk+SgnooKSmJ+fPnYzabeeSRR2jRooXRITUIjd0aE/dcHAAOsxzQWhsckRBClI8kBfXMhQsXWLBgAQ4ODkRERNCkSROjQ2pQgr2COTXtFADub7sbHI0QQpSPJAX1yNmzZ4mMjMTNzY2IiAj8/f2NDumGnUk/w4XMC0aHUSXa+7YnakoU+eZ8OvxfB6PDEUKIGyZJQT0RHR3NwoUL8fHxISIiAh8fH6NDKpc10WvYcHqD0WFUmR5Ne7B58mZi0mK4bcFtRocjhBA3RDW0ds9evXrpvXv3Gh1GlTpy5AgrVqwgKCiIiRMn4u4u1da1xcoTKxn1zSjGdR3H4vsXGx2OEKIBUkrt01r3upFjpaagjouKimL58uU0b96cyZMn1+mEYFvsNuZsn2N0GFUqvFM4X9z7BUuOLOHZdc8aHY4QQvwmmTq5Dtu5cyfr16+nffv2PPjggzg5ORkdUqW0821HgaXA6DCqXESPCC7mXuQvP/yFQI9AXh74stEhCSFEqSQpqIO01mzdupXNmzfTuXNnRo8ejaNj3f+nDPYKJtgr2OgwqsULt75ASm4Kr/z0CgEeATze83GjQxJCiBLq/jdJXbNrF8TGwoMPVuh0rTUbN25kx44ddOvWjXvvvReTSVqB6oL373qfxOxEnvjuCXzdfBndebTRIQkhxDXk26Sq/PQTzJ//+8c5OYGLS+n7IiNh584yT7VaraxevZodO3bQu3dvwsPDJSGoYyLvi2RI2yHc/9/72XR2k9HhCCHENeQbpaoEBkLr1rBwIWzeXPZxPXtCeHjp+3x9wdOz1F0Wi4UVK1YQFRXFgAEDGDFiBEqpSoctat7GSRvp7N+ZOyLvYF/8PqPDEUKIYvJIYlUbNw7i4+HppyEzE3r0sC2VYDabWbp0KdHR0dx555306deHrw99zbiu43B3qrtPGzR0Xu94kV2YzcmpJ+no19HocIQQ9ZQ8kmikJUvg00/httugUSNwc6vU5QoLC1m0aBHR0dGMHDmSAQMGUGgpJD4rnryivKqJWRgi86VMAEI+Dqk3ozkKIeo2qSkwQnQ05ObakoY2bco8LO/NN1nk7k5cTg7h4eF069atBoO8MXlFebz848tM6zONdr7tirfvjd/Lmug1vH7b6wZGV/tprTHNsuXmF1+4iJ+7n8ERCSHqG6kpMNLRozBvXun7Tp+GxETYvh3+/W/4z3/KvExOTg5fursTn5vLAw88UCsTAgBXR1eGthtKi0bXzsTY3rc9vZv1Niiq6hF7KZafz/1cpddUSlE0swgA/9n+5BTmVOn1hRCiPCQpqEpbttg6Gbq6Xrv9cm3MqlXw/PPQuDH885/weim/otevJyMhgfnz55Oan8/48ePp3LlztYdeUUopRnYYibOD8zXbfVx9GNlhpEFRVY/o1Gj2JVR9x0BHkyO5r+QC4PmOJ4WWwip/DyGEuBHSfFCVdu6ES5dg+PAr2157zdbh8KOPbMlBVBQEBUHz5iXP15q0GTOI9PEh32plwoQJtGzZsnpiNYBVW4lOjaaTfyejQ6mVMvIz8HnPNpGV5TULJiU5uxCi8qT5wCh9+16bEICt30Av+7+FUhAWVnpCACSnpDDfz49C4OGHH66TCcGxlGMcSzlW5r5ZW2bJL+EyNHJtRMLzCQA4zHKgoSXsQgjjSVJQlfLywGKxrZ85Y/v7/PO2GoKVK2HZMli6tNRT4+Pj+WL+F1i0hYiICIKDrwz3ezT5KG9uebNWfUmsPbWWQ0mHSmzfG7+XvfHX1sTkm/Pp8WkPfjzzI1+Ef1GiqUFc0cSzCaefPg3YEgMhhKhJMsxxVXrvPWhn74H/1lu2DoddusDXX4PJBM7OMHBgidNiY2NZtGgRRaYiLF0sbE/bzq0et+Lv7g/Y5gRo07jspxSMkJKbQiPXRgD8Y9c/aN2oNd2bdmfSzZOKB1U6lXqKoylHCQ8Jp4VPC/zd/XF1dP2tywqgbeO2HPzDQbp91o2Wf2/JuWfPlfsa289vx8fVh9CA0GqIUAhRX0lNQWUdO3ZleOMpUyAtDWbOtNUYTJlie+TwxAkYOtTWtyAo6JrTY2Ji+DLySwocCjDfbMbR3ZHZv8zmq4NfFR9zJv0M62LWsStuV03eWQnRqdGk5qYCtiGXgzyCWB+zns7+nTl76Sx/XPNHDicfBuBs+llm/DSDqIQozmWcY1i7YQxuPbjENXOLcmv0Hm7UuYxzJGQlGPb+NwfdzLaIbZzPPE//ef3Lff6Jiyc4l1H+ZEII0bBJTUFlOTjAqVMQEQEtW8K+fbbOhgBZWRQ5OaK0Fcft223DIDs52UY8DA7m2LFjLF++HPdG7iywLKBJchPWTVzHSwNeIiM/A6u2olB8tu8zHuzyIH2a9Sl+2yJLERkFGcW1CTfqv0f/y+BWgwnyDPr9g68z5psxeLt4szliMx/v+ZiYtBg8XTxp4d2C4xePE9Y0jC4BXZi1ZRY+Lj54Onvy51v+zP7E/fxw9gdC/ENo7n2lP8XhpMO88/M7LBi1ACeTEzlFOXg6lz7Mc0379sS3uDq6MiVsimExDGg5gNXjV3PP4nsY898xLHtw2Q2f+2iPR6sxMiFEfSU1BZUVEmLrM/DNN7bl0CHIyLAtViv5Hk7k+HraOhmOGAEJCbB6NQc+/JBly5bRrFkzPi36lLiCOAa3HoxFW/hkzyd0/1d3Zm2ZhVKKUP9QgjyCrumNvvz4cj7c8WG5w43NiOVi7sVynbM7bjfv/vwuT4Q9wdGUo7z/y/tYLVYiD0bi7ujO4sOLOZ583DYQjzIR4hdCY9fGnE47zbz980jJSWHF2BUMaTvkmut2CezCG7e9gbODM6tOruL1zTU70NEnez5hyZElpe6b2mcqT/R8okbjKc3dHe8mclQky48vZ9r30yp8nYOJB/lg+wdVGJkQoj6SmoKqsGmTrZPhyZPXbg8NxWvePFvzwYoVxdMl716zhrV799K2bVvGjh1L94vdbdXwAZ1xcXQhsyATZ5Mzm85uwmwxc/bSWZxMTtzS/JbiS9/f+X6Gthta/HrtqbWcTD1J7+De3Nry1jJDfaH/C+W+vR/O/MDamLVsi9hG7KVYrFYrWUVZXMi6wNydc3FycEKhiE6LRinF2K5jeWvrW7g5uZFXkMfpVFvHuelrpzOm6xhyzbnc0eYOTMpER7+OLNq8iL1Ze3l20LPljq0yBrUahIeTR6n7atPjgJO6TSI1L5Vn1z9LoEcgMwfPLPc1Aj0Cade43e8fKIRo0GScgsp64QX45BNbUnB1WTo62voRdOkCXbqQ37oFZ0ObsmzdMqxnrISEhDBmzBii06NxcXDhmXXPkJiVyOaIzXg4e3A67TSPrXyMwa0HcyDhABrNqgmrWH1yNV2DutLap/U1YUQejCQxK5GBrQbSr0U/wNbZ7Gz6WR66+aFK3eL6mPWsi1lHen460Rej2Re/D5My4efuR2J2IgDert6M6zKOQM9ABrcazOxfZuPm5MaKEytwVI54OXmRVpjG8DbDebLPk4zqNKr4+uqvto6J+vWG9Vksrxk/zuDtn9/mk5Gf8FTvp4wORwhRR5RnnAKpKaisD8qokrVY4JdfsCYmYtq0iR0vjmX7ggtYcxsTEhrCA6MfAAV/3/F32vu2J6cwh6jEKN7a8haD2wymd3BvHu35KG182rD57GZuaX4Lm3/dzLqYdfh7+JdICh7u9nCJEDycPPB1863U7X229zOaezfnXMY5vjv5HUXaNiQv2vaooauTK9qq8XL2Yn/Cfh7q9hArT67EbDVTZC1icKvB5JvzmXvbXAYtGcSqh1ax88JOzqSfoW3jtgA44IAFS6XibAj+duffSMxO5I/f/xE/dz8e7GKreVoXs47ewb1l3gQhRKVJUlBZHh6Qc9149d7e4OxMYVEB+9zS6P3ZKtJPn8Kcm4NnQRxjx7zGlwe+ZPv57eyJ28PaU2t5vOfjnLt0Dm8Xb55c/ST3d7qfdr7tWHx4MS0atSDYO5iZP80k35zPE2E31tbdrUk3ujWp2JwJVm0lISsBs9XMprOb2Ba77UpCALTzbkcT7yZk5GeQWZiJr6svh5IOkbE7gylhUxjdaTSZBZmcuHiCtTFrmbljJv8c+U+s2sqhpEMUWAqKkwLz6+YKxdgQzQufR2JOImOXjcXH1Yeh7YYSlRCFj6uPJAVCiEqT5oPKCgyElJRrt912GwwZgvXIYXIORvFl+FBSXQPYznaygrP4aORHLD6ymPWn1tPUqyl74/fi6uhKel46g1oN4rN7PmPe/nkUWYqIz47Hy9kLT2dPOgd0pm/zvnQN7PqbIRWYC8gpyqlwLcG8qHmsjVnLqdRTnL90nqzCLMyU/OIe03kM22K3kWfJo8hcRL4lH08nT57r/xzfHv8WJ5MTQV5BfDvuW6JTo0nKTmLZsWXMGTYHFweX4vEMqtvR5KOsjl7NiwNe5EDiAT6P+px/jPhHhfoNfHviW4I8goqbaIzS4189OJB4gJ2P7bymr4kQQlxPhjmuKWfOlEgILA4OFG7ZTN6a74ge0Y/ld48k1TUA3JNZNWMVd7W/i8iDkaTnpXMy9STbYreRWZhJam4qRbqIrbFbmfb9NNLz02nt05rn+j3HiwNeJKxpGBtPb8TL2YtL+ZeYs30OuUW5HEo6xIoTKwBIzE5k6vdT+WL/F8zdObfCt/X9qe85lHSIEyknSC9MLzUhANsTECm5KWQWZJJnyUOjWXT/IuIz4zlz6QznMs4xudtkHE2OhAaEcmvLW/lTnz8x86eZrDq5qsLxlVeAR0DxtM4bYjbQJaALJmUqHiEytyiXPXF7AEjISiAtL63Ma5mUCUXNJDO/Zf+T+/Fz86PvvL4cTzludDhCiHpCkoLK6Hfdr8VBgyh0dybF3424zHiWHz1OrGdjsgMvovoEkZ6fztO3PE1mfibfnfgOK9biKnkLFhyVI04OThxNOcqBhAN4OnuSkJnAdye/4+PdH/PT2Z8wW804KAdcHF0wKZOtVqDA1nxhwsS++H3c2fZOnu1XsZ7862LWMbLDSNJy0yjUvz1HgUZjxXrNtskrJrMuZh3NvZrzQJcHOJJ8pHifs4MzoQGh/KHXHxjWfliF4quIQI9AxoSOIbMgk/Wn13M0+SjTN0xnzo45AOy8sJN3tr3DxdyLLDy8kLk75vLmljdLXOfynA215Zf5xb/YHi0N/SRUBioSQlQJSQoqIzn5yvr//ke8rzPmZsE0srqyfmg4ZrcmtDq9nceDuuHu5M6PZ37k4RUPs/jwYtIKSv4aVSie6/ccLb1b4uPiw7TvpzF++Xhe2/QaybnJODs48+ulX/Fy8WJqn6m4OrrSu1lvJnabCECgZyBLH1xKR7+O+Lj6VOiWsguymbVpFqn5qTy1G4adKt/5OUU5xGfFk5KbQoGloESHSIB2vu1qbLhjrTWrTq5iY8xGRiwcQXhIOBn5Gfi6+XJvyL0sPLyQpp5N8XTxZM72OfRv3p+pt0wttYkmPiueLbFbyDPn1UjspTFbr621sb5mS8pafdSq3ONPCCHE9aSjYWVoDRcuwB13YJ75Kjlxx/AudCdy0kSSvBsT43WAHO9YulNIXlEeT699mrT8K8lAx4vgaoZDTcDZ5Iyvmy8rT67kcPJhgj2CyTZn44ADbRu3pXez3qTkphR3zivL1SMGlte2c9tY8X9/4lLjZHCF/U0hufTH+Es1rO0wOvl1Ii0/jZ5NezLh5gnlHnGxquUW5bIldgvtfdsztO1QGrk24lzmOfq26EtHv45sP78di7YwuvNojiQf4VLBJQI8Ariv830lrtXapzVzhs4x4C5g4+mN+Lr58vedf79mUimlFOaZZhzfdCRgdgCZL2Xi5eJlSIwNycbTG0nNTWXcTeOMDkWIKiU1BZUVGAhubpiSknF18OLzRx8hKSiAJWoJd++KZkjwAOK6tiLyYOQ1CQFAt0QIi7etu5hc0FZNTGoMvi6+3N7mdsKahNHYtTGh/qFcyLqAt4t3tU2MZLaamfy/yXQ4kUwXewXIzhZw5nf6Kl7dvn4g6QCDWg8iNiOWPHMegR6Bhg8C5OHswbt3vsu22G3EZcYx++fZZBVkMSZ0DACPdH+EroFdGdVpFK8OepWRHUYaGm9prNrK+tPryS7M5oX+L5SYZdLB5EDeDFvthfe73hSYC4wIs0Hxc/er0FDhQvye3KJcfr30q2HvL0lBZX35JZbTpznj6cjSSRHkejdiieVrXlt+iuSuren3yKsMaz+s1M5rS7vC/J629eHthpOWl0aeJY+cohw2/bqJRq6NuKPtHbTza0e3oG5M7zcdq7ZWagrlxOxEPtr5UYlqaEeTI008mvDX22FHyxu/XlPPpvi5+BHsGcz0/tP58eyPzB46m5cGvFThGKvC0eSjPLbqMaavn05SThIR3SNo59uONn5tCPQI5IczPxgaX3mYlAkfVx/e2vYWHfw6sDtu9zXTUxeYC7iQeYGMlzIAcP2bKxarjPtQnXo27cntbW43OoxKi8uMY13MOqPDEFdZHb2az6M+N+z963zzgVJqODAXcAA+11q/W6MBrFnDRTc3Vo6ewCV3J1bnRfKraxxhKpj3dCKTtz5HoHtgiVqCq5kwsfTk0uLXzbybAdC9SXc6+XdiTfQa7u54N54unszdNRdvZ28e6/lYhcJ1UA44OziX6EFfYC5gR/yOcl8vNSeVzoGd+Wr0V4QGhJKSk1IrfkG18mlFTkEOJ1JO0OPXHvx49kee7/88z/R9BpMy4WiqWx/9EL8QBrYciLuTO+czzmNSJnoF254w2nB6A1tit/DB0A9Imp5E0AdBOL7piPU1a4099inqpvOZ5zmecpzh7YcbHUqV2PzrZvLN+XX6fh4IfYDwkHDD3r9Oj1OglHIAooG7gAvAHmC81vpYWedU6TgFx44RN2QIiyZOBK1ptCOSN25J5pKPG9mvZpNbmMuiw4t4cs2TpZ7u6eBJp8BO7E24Eo8jjoy/aTwBHgH8odcf6ODXAYBP93xKoaWQ8E7huDq60sSzSYnrpeel8+7P7zK9/3QCPALKdSvN3mpGvCW+zP3T+03ngx1XRm/0MHnQpnEbnJ2ceXngy8XV8bXBwgMLmb1jNo3dGnMh4wIdAzqyZsIao8OqkE1nNxGVEIWzozMhfiHXzHdxmdaaQkshLo4uAMReiqX13Na2fTJ0tGhANp3dRIGloE4nBdWhIQ1z3AeI0VqfAVBKLQHCgTKTgqr00V/uJX/yZFzy83HbE8lJ9zT+1GgIaX1uxqRMeLp4lpkQAHQO7MzuKbtJzU1lxroZ/BL/CzjYagreGfLONcf6uPrwr33/wtXRFTcnt1KHNfZ28aZ3s974uPqwLXYbxy8ev+Gpf8tKCBxxZO7IuUy6eRLdm3QnqyALK1Y6+3emR9MeHEo6xKBWg27oPWpCYlYiE1dORKFo5tUMEyayC7KNDqtCPtz+ITN+msErA16hb/O+BHsFl3qcUqo4IQBbLcmRp47Q9dOuNPmgCYnTE2sq5Hpvw+kN7Infw4yBM4wORZSiPjTpGK2uJwXNgPNXvb4AlHiIXCk1BZgC0LJlORrMf8fWju4MzrrEssLF/HxXBm8OfJOXb3/lN9v87257Nx/f8zERKyPY9MgmwNZp6cP/9yEKxcqTK7mlWcnn4MM7hePj6oOPqw+Hkw+Xem0Hk0PxL/YAj4BKdTh777b3GN99PE4mJ5p42WolWni3wMvFix5NexQfV5sSAgB/d39C/UN5+863Ce8UXjyuQ10U1jSM8E7hvDr41XI3A3QJ7MKOx3bQb14/ev+nN3ue2FNNUTYsYU3DypxZU4j6oK43HzwADNNaP25/PQnoo7Uuc+L5qh7m2Gq1YjJJf01RO62LWceIhSO4N+ReVo5baXQ4QggDNKRhji8ALa563Rwou2G8GkhCIGqz4e2Hs2j0IladXMWT35XdlCWEEFD3mw/2AB2UUm2AOGAcMMHYkISoXcbfNJ7UvFSmrZ1GkGcQs26fZXRIQohaqk4nBVprs1JqKrAe2yOJX2itjxoclhC1ztQ+U7mYe5G/bvkr/u7+PH3L00aHJISohep0UgCgtf4e+N7oOISo7d647Q2Sc5L587o/4+fmx0M3P2R0SEKIWqbOJwVCiBv3yd2fkJCdwMQVE/F182VEhxHX7LdYLbbpoWXQIyEaJOklJ0QDs2LsCvo068PIRSPZfn578fZ8cz7Dvh5Gz3/1NDA6IYSRJCkQogHa9fgugr2CufWLWzmSfITswmxu//J2fjn/C8cvHicuM87oEIUQBpCkQIgGKu452xf/TZ/exM2f3sz+hP3km/NxMDnwXfR3BkcnhDCCJAVCNGBJ05MAiM2IpcBiGwEzt8g2Z4cQouGRpECIBup8xnnC/h2Go8kRq7Zes2933G5yi3INikwIYRRJCoRogGLSYuj1n14kZCVgtppL7Hd2cOaHMz8YEJkQwkiSFAjRAC06vIj0vHQ0pc99klWYxTdHv6nhqIQQRpOkQIgGaOagmfz86M9EdI/Aw8kDL2evEsesiV5TollBCFG/SVIgRAOklKJPsz58fu/npL2YRuR9kdzV9i5cHFxwd3IHwGw1sy9+n8GRCiFqkiQFQjRwzg7OjOo0ig2TNhD3XBzv3/U+oQGh5BTlsDV2q9HhCSFqkNK69DbF+qpXr1567969RochRK13KvUULRu1xMXRxehQhBCVoJTap7XudSPHytwHQohSdfDrYHQIQogaJs0HQgghhAAaYPOBUioFiK3CS/oDF6vweqIkKePqJ2Vc/aSMq5+Ucelaaa0DbuTABpcUVDWl1N4bbasRFSNlXP2kjKuflHH1kzKuPGk+EEIIIQQgSYEQQggh7CQpqLx/Gx1AAyBlXP2kjKuflHH1kzKuJOlTIIQQQghAagqEEEIIYSdJgRBCCCEASQoqRSk1XCl1UikVo5R6yeh46hKl1K9KqcNKqQNKqb32bb5KqY1KqVP2v42vOv5lezmfVEoNu2p7mP06MUqpfyillBH3Uxsopb5QSiUrpY5cta3KylQp5aKU+sa+fZdSqnVN3l9tUEYZv6GUirN/lg8opUZetU/KuByUUi2UUpuUUseVUkeVUn+2b5fPcU3RWstSgQVwAE4DbQFn4CAQanRcdWUBfgX8r9v2PvCSff0l4D37eqi9fF2ANvZyd7Dv2w30AxSwFhhh9L0ZWKaDgJ7AkeooU+CPwGf29XHAN0bfcy0p4zeA6aUcK2Vc/vJtCvS0r3sB0fZylM9xDS1SU1BxfYAYrfUZrXUhsAQINzimui4c+NK+/iUw6qrtS7TWBVrrs0AM0Ecp1RTw1lrv0Lb/wiOvOqfB0VpvBdKu21yVZXr1tZYBdza0mpkyyrgsUsblpLVO0FpH2dezgONAM+RzXGMkKai4ZsD5q15fsG8TN0YDG5RS+5RSU+zbgrTWCWD7nwMQaN9eVlk3s69fv11cUZVlWnyO1toMZAB+1RZ53TJVKXXI3rxwuWpbyrgS7NX6PYBdyOe4xkhSUHGlZZbyfOeNu1Vr3RMYAfxJKTXoN44tq6zl36DiKlKmUt6l+xRoB3QHEoA59u1SxhWklPIElgPPaK0zf+vQUrZJGVeCJAUVdwFocdXr5kC8QbHUOVrrePvfZGAFtuaYJHu1H/a/yfbDyyrrC/b167eLK6qyTIvPUUo5Ao248ar0ektrnaS1tmitrcB/sH2WQcq4QpRSTtgSgoVa6//ZN8vnuIZIUlBxe4AOSqk2SilnbB1WVhkcU52glPJQSnldXgeGAkewld9k+2GTgZX29VXAOHuv4TZAB2C3vRoxSynV194m+PBV5wibqizTq681BvjJ3l7boF3+srK7D9tnGaSMy81eHvOA41rrD6/aJZ/jmmJ0T8e6vAAjsfWOPQ3MMDqeurJge2LjoH05ernssLXr/Qicsv/1veqcGfZyPslVTxgAvbD9T/g08DH2UTob4gIsxlZ9XYTt19BjVVmmgCuwFFtnrt1AW6PvuZaU8VfAYeAQti+cplLGFS7fAdiq8g8BB+zLSPkc19wiwxwLIYQQApDmAyGEEELYSVIghBBCCECSAiGEEELYSVIghBBCCECSAiGEEELYSVIghPhNSimLffa/I0qppUop9wpc43OlVKh9/ZXr9m2vqliFEJUjjyQKIX6TUipba+1pX18I7NPXDixT4esJIWoXqSkQQpTHNqA9gFLqOXvtwRGl1DP2bR5KqTVKqYP27WPt2zcrpXoppd4F3Ow1Dwvt+7Ltf79RSo28/EZKqQVKqfuVUg5KqdlKqT32SYeerOmbFqKhcDQ6ACFE3WAfJ34EsE4pFQZEALdgm2Bml1JqC7bRKuO11nfbz2l09TW01i8ppaZqrbuX8hZLgLHA9/ahw+8EnsI2amCG1rq3UsoF+EUptUHbpsoVQlQhqSkQQvweN6XUAWAvcA7b2PQDgBVa6xytdTbwP2AgtuF+hyil3lNKDdRaZ5TjfdYCd9i/+EcAW7XWx4wxcAAAARRJREFUedjmxnjYHsMubEPedqiqmxNCXCE1BUKI35N3/S97+yQzJWito+21CCOBd+y/6GfdyJtorfOVUpuBYdhqDBZffjtgmtZ6fUVvQAhxY6SmQAhREVuBUUopd/tMl/cB25RSwUCu1vpr4AOgZynnFtmnxy3NEmzNEgOBy0nAeuCpy+copTra31MIUcWkpkAIUW5a6yil1AJss8wBfK613q+UGgbMVkpZsc0k+FQpp/8bOKSUitJaP3Tdvg1AJLBKa114+dpAayDKXkORAoyq0hsSQgDySKIQQggh7KT5QAghhBCAJAVCCCGEsJOkQAghhBCAJAVCCCGEsJOkQAghhBCAJAVCCCGEsJOkQAghhBAA/H8404s/aTFipgAAAABJRU5ErkJggg==\n", 377 | "text/plain": [ 378 | "
" 379 | ] 380 | }, 381 | "metadata": { 382 | "needs_background": "light" 383 | }, 384 | "output_type": "display_data" 385 | } 386 | ], 387 | "source": [ 388 | "# Plot the samples using columns 1 and 2 of the matrix\n", 389 | "fig, ax = plt.subplots(figsize = (8, 8))\n", 390 | "\n", 391 | "colors = ['red', 'green']\n", 392 | "\n", 393 | "# Color base on the sentiment Y\n", 394 | "ax.scatter(X[:,1], X[:,2], c=[colors[int(k)] for k in Y], s = 0.1) # Plot a dot for each pair of words\n", 395 | "plt.xlabel(\"Positive\")\n", 396 | "plt.ylabel(\"Negative\")\n", 397 | "\n", 398 | "# Now lets represent the logistic regression model in this chart. \n", 399 | "maxpos = np.max(X[:,1])\n", 400 | "\n", 401 | "offset = 5000 # The pos value for the direction vectors origin\n", 402 | "\n", 403 | "# Plot a gray line that divides the 2 areas.\n", 404 | "ax.plot([0, maxpos], [neg(theta, 0), neg(theta, maxpos)], color = 'gray') \n", 405 | "\n", 406 | "# Plot a green line pointing to the positive direction\n", 407 | "ax.arrow(offset, neg(theta, offset), offset, direction(theta, offset), head_width=500, head_length=500, fc='g', ec='g')\n", 408 | "# Plot a red line pointing to the negative direction\n", 409 | "ax.arrow(offset, neg(theta, offset), -offset, -direction(theta, offset), head_width=500, head_length=500, fc='r', ec='r')\n", 410 | "\n", 411 | "plt.show();" 412 | ] 413 | }, 414 | { 415 | "cell_type": "markdown", 416 | "metadata": {}, 417 | "source": [ 418 | "**Note that more critical than the Logistic regression itself, are the features extracted from tweets that allow getting the right results in this exercise.**\n", 419 | "\n", 420 | "That is all, folks. Hopefully, now you understand better what the Logistic regression model represents, and why it works that well for this specific problem. " 421 | ] 422 | } 423 | ], 424 | "metadata": { 425 | "kernelspec": { 426 | "display_name": "Python 3", 427 | "language": "python", 428 | "name": "python3" 429 | }, 430 | "language_info": { 431 | "codemirror_mode": { 432 | "name": "ipython", 433 | "version": 3 434 | }, 435 | "file_extension": ".py", 436 | "mimetype": "text/x-python", 437 | "name": "python", 438 | "nbconvert_exporter": "python", 439 | "pygments_lexer": "ipython3", 440 | "version": "3.7.1" 441 | } 442 | }, 443 | "nbformat": 4, 444 | "nbformat_minor": 4 445 | } 446 | --------------------------------------------------------------------------------