├── .gitignore ├── Reports ├── StockIt Report.pdf └── StockIt Presentation.pdf ├── data-extractor ├── lexicon_negation_words.txt ├── stock_price_extractor.py ├── stocktwits_data_extractor.py ├── labelled_data_extractor.py ├── lexicon_bearish_words.txt └── lexicon_bullish_words.txt ├── dnn_classifier ├── get_s&p500_symbols.py ├── s&p500.txt └── tweets_classification_using_dnn.py ├── LICENSE ├── README.md ├── naive_bayes_classification.py ├── MlpStockClassification.py ├── sentiment_analysis.py ├── tweet_classification_using_rnn.py ├── load_data.py └── RNN.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | *.csv 2 | .vscode 3 | __pycache__ 4 | *.py[cod] 5 | *_training*.txt 6 | *_test*.txt 7 | *.pkl 8 | -------------------------------------------------------------------------------- /Reports/StockIt Report.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RishabhKr97/StockIt/HEAD/Reports/StockIt Report.pdf -------------------------------------------------------------------------------- /Reports/StockIt Presentation.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RishabhKr97/StockIt/HEAD/Reports/StockIt Presentation.pdf -------------------------------------------------------------------------------- /data-extractor/lexicon_negation_words.txt: -------------------------------------------------------------------------------- 1 | aint 2 | ain't 3 | aren't 4 | arent 5 | bareley 6 | cannot 7 | can't 8 | cant 9 | couldn't 10 | couldnt 11 | didn't 12 | didnt 13 | doesn't 14 | doesnt 15 | don't 16 | dont 17 | few 18 | hardly 19 | haven't 20 | havent 21 | isn't 22 | isnt 23 | low 24 | merely 25 | neither 26 | never 27 | never 28 | no 29 | nobod 30 | none 31 | nope 32 | nor 33 | not 34 | nothing 35 | rarely 36 | seldom 37 | shouldn't 38 | shouldnt 39 | wasn't 40 | wasnt 41 | weren't 42 | werent 43 | without 44 | won't 45 | wont 46 | wouldn' 47 | zero 48 | -------------------------------------------------------------------------------- /dnn_classifier/get_s&p500_symbols.py: -------------------------------------------------------------------------------- 1 | import finsymbols 2 | from nltk.corpus import wordnet 3 | import enchant 4 | 5 | 6 | # using enchant because wornet was removing too many words 7 | dict = enchant.Dict("en-US") 8 | sp500 = finsymbols.get_sp500_symbols() 9 | 10 | 11 | symbols = [] 12 | for row in sp500: 13 | symbols.append(row['symbol'].lower()) 14 | 15 | 16 | with open("s&p500.txt", "w") as file: 17 | 18 | removed = 0 19 | for s in sorted(symbols): 20 | if not dict.check(s): 21 | file.write(s) 22 | file.write('\n') 23 | else: 24 | removed = removed + 1 25 | 26 | print(removed) 27 | 28 | 29 | 30 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Rishabh Kumar 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /data-extractor/stock_price_extractor.py: -------------------------------------------------------------------------------- 1 | import requests 2 | import base64 3 | import csv 4 | 5 | username = "d337f69c0a8b4db5841d08ed55313214" 6 | password = "dadd178bdf8655e543c4b983a9bf22f0" 7 | b64Val = base64.b64encode(b"d337f69c0a8b4db5841d08ed55313214:dadd178bdf8655e543c4b983a9bf22f0").decode("ascii") 8 | ticker = "AAPL" 9 | 10 | values = [] 11 | for page in range(1, 11): 12 | 13 | r = requests.get("https://api.intrinio.com/prices?identifier=" + ticker + "&page_number=" + str(page), 14 | headers={"Authorization": "Basic %s" % b64Val}) 15 | 16 | for row in r.json()["data"]: 17 | 18 | dicts = {} 19 | dicts["Date"] = row["date"] 20 | dicts["Opening Price"] = row["open"] 21 | dicts["Closing Price"] = row["close"] 22 | dicts["Highest Price"] = row["high"] 23 | dicts["Volume"] = row["volume"] 24 | values.append(dicts) 25 | 26 | 27 | fields = ['Date', 'Opening Price', 'Closing Price', 'Highest Price', 'Volume'] 28 | filename = "stock_prices_" + ticker + ".csv" 29 | 30 | 31 | with open(filename, 'w', newline='') as csvfile: 32 | 33 | writer = csv.DictWriter(csvfile, fieldnames = fields) 34 | 35 | writer.writeheader() 36 | 37 | writer.writerows(values) 38 | 39 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # StockIt 2 |

Predicting bullish or bearish sentiment of stocks (Apple, Google, Amazon) using analysis of Stocktwits tweets.

3 |

Stock Market is one of the most dynamic and volatile places which is 4 | highly dominated by general sentiment of traders. It is characterized by 5 | high uncertainty factor and fast-paced changes in trends. Building a 6 | system that can predict the movement of the stock market has posed a 7 | major challenge for researchers since such a system has to deal with a 8 | high noise to signal ratio. Moreover, the movement of the stock market is 9 | mainly determined by the sentiment of traders which is not easily 10 | captured. These traders are influenced by a large number of factors like 11 | monetary reports, news, general opinion about the company as well as 12 | the opinion of the fellow traders. Targeting the events that do have an 13 | effect on the prices of stocks and predicting the exact effect they cause 14 | has largely remained unsolved till date.

15 | 16 |

In this project, we aim to predict the bullish (increasing) or bearish 17 | (decreasing) nature of stocks of three companies namely Apple, Amazon 18 | and Google by performing sentiment analysis on the stocktwits tweets 19 | for their stocks and gather the prevailing trend for them and using the 20 | result, along with other factors like previous actual sentiment, change in 21 | tweet volume and cash flow to predict their bullish or bearish nature. 22 | We applied various approaches of sentiment analysis ranging from the 23 | lexicon-based approach of SentiWordNet to supervised learning based 24 | approach of RNN, DNN and Naive Bayes classifiers to a labelled corpus 25 | of 1.5 lakh tweets, collected from stocktwits website. Ultimately we 26 | combined the lexicon-based approach with supervised learning 27 | approach for best results. We achieved an accuracy of 73% for 28 | sentiment analysis on our test data.

29 |

However, the actual prediction of share market movement is so 30 | uncertain and comprises of a large number of variables that a 50% 31 | accuracy is considered satisfactory while an accuracy greater than 60% 32 | is considered significant in this domain. Using our approach we were 33 | able to achieve an accuracy of 57% in predicting the actual trend of the 34 | market.

35 | -------------------------------------------------------------------------------- /data-extractor/stocktwits_data_extractor.py: -------------------------------------------------------------------------------- 1 | """ 2 | EXTRACT DATA FROM STOCKTWITS API 3 | WORKAROUND RATE LIMITS USING PROXY 4 | CHANGED WORKAROUND METHOD TO USING MULTIPLE ACCESS KEYS 5 | """ 6 | import csv 7 | import json 8 | import os 9 | import time 10 | import requests 11 | # from http_request_randomizer.requests.proxy.requestProxy import RequestProxy 12 | 13 | FIELDS = ['symbol', 'message', 'datetime', 'user', 'message_id'] 14 | SYMBOL = "AAPL" 15 | FILE_NAME = 'stocktwits_' + SYMBOL + '.csv' 16 | token = 0 17 | access_token = ['', 'access_token=32a3552d31b92be5d2a3d282ca3a864f96e95818&', 18 | 'access_token=44ae93a5279092f7804a0ee04753252cbf2ddfee&', 19 | 'access_token=990183ef04060336a46a80aa287f774a9d604f9c&'] 20 | 21 | file = open(FILE_NAME, 'a', newline='', encoding='utf-8') 22 | # DETERMINE WHERE TO START IF RESUMING SCRIPT 23 | if os.stat(FILE_NAME).st_size == 0: 24 | # OPEN FILE IN APPEND MODE AND WRITE HEADERS TO FILE 25 | last_message_id = None 26 | csvfile = csv.DictWriter(file, FIELDS) 27 | csvfile.writeheader() 28 | else: 29 | # FIRST EXTRACT LAST MESSAGE ID THEN OPEN FILE IN APPEND MODE WITHOUT WRITING HEADERS 30 | file = open(FILE_NAME, 'r', newline='', encoding='utf-8') 31 | csvfile = csv.DictReader((line.replace('\0', '') for line in file)) 32 | data = list(csvfile) 33 | data = data[-1] 34 | last_message_id = data['message_id'] 35 | file.close() 36 | file = open(FILE_NAME, 'a', newline='', encoding='utf-8') 37 | csvfile = csv.DictWriter(file, FIELDS) 38 | 39 | # req_proxy = RequestProxy() 40 | 41 | stocktwit_url = "https://api.stocktwits.com/api/2/streams/symbol/" + SYMBOL + ".json?" + access_token[token] 42 | if last_message_id is not None: 43 | stocktwit_url += "max=" + str(last_message_id) 44 | 45 | api_hits = 0 46 | while True: 47 | # response = req_proxy.generate_proxied_request(stocktwit_url) 48 | try: 49 | response = requests.get(stocktwit_url) 50 | except Exception: 51 | response = None 52 | 53 | if response is not None: 54 | 55 | if response.status_code == 429: 56 | print("###############") 57 | print("REQUEST IP RATE LIMITED FOR {} seconds !!!".format( 58 | int(response.headers['X-RateLimit-Reset']) - int(time.time()))) 59 | 60 | if not response.status_code == 200: 61 | stocktwit_url = "https://api.stocktwits.com/api/2/streams/symbol/" + SYMBOL + ".json?" + access_token[ 62 | token] + "max=" + str( 63 | last_message_id) 64 | token = (token + 1) % (len(access_token)) 65 | continue 66 | 67 | api_hits += 1 68 | response = json.loads(response.text) 69 | last_message_id = response['cursor']['max'] 70 | 71 | # WRITE DATA TO CSV FILE 72 | for message in response['messages']: 73 | # PREPARE OBJECT TO WRITE IN CSV FILE 74 | obj = {} 75 | obj['symbol'] = SYMBOL 76 | obj['message'] = message['body'] 77 | obj['datetime'] = message['created_at'] 78 | obj['user'] = message['user']['id'] 79 | obj['message_id'] = message['id'] 80 | 81 | csvfile.writerow(obj) 82 | file.flush() 83 | 84 | print("API HITS TILL NOW = {}".format(api_hits)) 85 | 86 | # NO MORE MESSAGES 87 | if not response['messages']: 88 | break 89 | 90 | # ADD MAX ARGUMENT TO GET OLDER MESSAGES 91 | stocktwit_url = "https://api.stocktwits.com/api/2/streams/symbol/" + SYMBOL + ".json?" + access_token[ 92 | token] + "max=" + str(last_message_id) 93 | token = (token + 1) % (len(access_token)) 94 | 95 | file.close() 96 | -------------------------------------------------------------------------------- /naive_bayes_classification.py: -------------------------------------------------------------------------------- 1 | """ 2 | implement multinomial naive bayes for sentiment analysis 3 | """ 4 | 5 | import os 6 | import load_data 7 | import sentiment_analysis 8 | import numpy as np 9 | from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer 10 | from sklearn.pipeline import Pipeline, FeatureUnion 11 | from sklearn.naive_bayes import MultinomialNB 12 | from sklearn.externals import joblib 13 | from sklearn.model_selection import GridSearchCV 14 | from sklearn import metrics 15 | from sklearn.preprocessing import FunctionTransformer 16 | 17 | class NaiveBayes: 18 | 19 | SENTI_CACHE_DICT = {} 20 | 21 | @classmethod 22 | def setiwordnet_scorer(cls, messages): 23 | scores = [] 24 | for x in messages: 25 | if x not in NaiveBayes.SENTI_CACHE_DICT: 26 | NaiveBayes.SENTI_CACHE_DICT[x] = sentiment_analysis.SentimentAnalysis.get_sentiword_score(x) 27 | 28 | scores.append(NaiveBayes.SENTI_CACHE_DICT[x]) 29 | 30 | return scores 31 | 32 | @classmethod 33 | def train_classifier(cls): 34 | dataFrameTraining = load_data.LoadData.get_labelled_data(type='training') 35 | 36 | # make a pipeline for transforms 37 | tweet_classifier = Pipeline([ 38 | ('feats', FeatureUnion([ 39 | ('text', Pipeline([ 40 | ('vect', CountVectorizer()), 41 | ('tfidf', TfidfTransformer()) 42 | ])), 43 | ('sentiscore', FunctionTransformer(NaiveBayes.setiwordnet_scorer, validate=False)) 44 | ])), 45 | ('clf', MultinomialNB(alpha=0.01)) 46 | ]) 47 | 48 | # grid search for best params 49 | parameters ={'feats__text__vect__ngram_range': [(1, 1), (1, 2)], 'feats__text__vect__strip_accents': ('unicode', None), 'feats__text__vect__stop_words': ('english', None), 'feats__text__tfidf__use_idf': (True, False), 'clf__alpha': (0.7,1,1e-2,1e-3,0.5,0.3), 'clf__fit_prior': (True, False)} 50 | gridsearch = GridSearchCV(tweet_classifier, parameters, n_jobs=-1) 51 | gridsearch = gridsearch.fit(dataFrameTraining['message'].values, dataFrameTraining['sentiment'].values) 52 | print(gridsearch.best_score_) 53 | print(gridsearch.best_params_) 54 | # print(gridsearch.cv_results_) 55 | 56 | # save the trained classifier 57 | file_location = 'naive_bayes_classifier.pkl' 58 | try: 59 | os.remove(file_location) 60 | except OSError: 61 | pass 62 | joblib.dump(gridsearch, file_location) 63 | 64 | @classmethod 65 | def test_classifier(cls): 66 | dataFrameTest = load_data.LoadData.get_labelled_data(type='test') 67 | 68 | # load the saved classifier 69 | file_location = 'naive_bayes_classifier.pkl' 70 | if os.path.isfile(file_location) is False: 71 | NaiveBayes.train_classifier() 72 | 73 | tweet_classifier = joblib.load(file_location) 74 | predicted = tweet_classifier.predict(dataFrameTest['message'].values) 75 | print(metrics.accuracy_score(dataFrameTest['sentiment'].values, predicted)) 76 | print(metrics.classification_report(dataFrameTest['sentiment'].values, predicted)) 77 | print(metrics.confusion_matrix(dataFrameTest['sentiment'].values, predicted)) 78 | 79 | @classmethod 80 | def get_sentiment(cls, message): 81 | 82 | # predict the sentiment of message 83 | file_location = 'naive_bayes_classifier.pkl' 84 | if os.path.isfile(file_location) is False: 85 | NaiveBayes.train_classifier() 86 | 87 | tweet_classifier = joblib.load(file_location) 88 | sentiment = tweet_classifier.predict([message]) 89 | return sentiment[0] 90 | -------------------------------------------------------------------------------- /MlpStockClassification.py: -------------------------------------------------------------------------------- 1 | """ 2 | Multilayer perceptron classifier for classifying stocktwits data 3 | """ 4 | 5 | import os 6 | import load_data 7 | from sklearn.pipeline import Pipeline, FeatureUnion 8 | from sklearn.externals import joblib 9 | from sklearn.model_selection import GridSearchCV 10 | from sklearn import metrics 11 | from sklearn.neural_network import MLPClassifier 12 | from sklearn.preprocessing import StandardScaler, FunctionTransformer 13 | import numpy as np 14 | 15 | class MLP: 16 | 17 | @classmethod 18 | def get_scaling_cols(cls, df): 19 | """ 20 | In test and training files column no. 21 | cash_volume = 0, 22 | label = 1, 23 | sentiment_actual_previous = 2, 24 | sentiment_calculated_bearish = 3, 25 | sentiment_calculated_bullish = 4, 26 | tweet_volume_change = 5 27 | """ 28 | 29 | return df[:,[0,3,4,5]] 30 | 31 | @classmethod 32 | def get_non_scaling_cols(cls, df): 33 | """ 34 | In test and training files column no. 35 | cash_volume = 0, 36 | label = 1, 37 | sentiment_actual_previous = 2, 38 | sentiment_calculated_bearish = 3, 39 | sentiment_calculated_bullish = 4, 40 | tweet_volume_change = 5 41 | """ 42 | 43 | return df[:,[2]] 44 | 45 | @classmethod 46 | def train_nn(cls, symbol='ALL'): 47 | """ 48 | Either train the neural network for one symbol or on all. 49 | """ 50 | 51 | dataFrameTraining = load_data.LoadData.get_stock_prediction_data(symbol=symbol, type='training') 52 | 53 | Classifier = Pipeline([ 54 | ('feats', FeatureUnion([ 55 | ('scaler', Pipeline([ 56 | ('col_get_scale', FunctionTransformer(MLP.get_scaling_cols)), 57 | ('scale', StandardScaler()) 58 | ])), 59 | ('non_scaler', FunctionTransformer(MLP.get_non_scaling_cols)) 60 | ])), 61 | ('mlp', MLPClassifier()) 62 | ]) 63 | 64 | # grid search for best params 65 | parameters = { 66 | 'feats__scaler__scale__with_mean':(True,False), 67 | 'feats__scaler__scale__with_std':(True,False), 68 | 'mlp__hidden_layer_sizes':((2), (3), (2,3), (3,2), (2,2), (3,3), (3,3,3)), 69 | 'mlp__solver':('lbfgs', 'adam'), 70 | 'mlp__alpha': 10.0 ** -np.arange(1, 7) 71 | } 72 | gridsearch = GridSearchCV(Classifier, parameters, n_jobs=-1) 73 | gridsearch = gridsearch.fit(dataFrameTraining, dataFrameTraining['label'].values) 74 | 75 | print(gridsearch.best_score_) 76 | print(gridsearch.best_params_) 77 | 78 | file_location = 'NN_classifier_'+symbol+'.pkl' 79 | try: 80 | os.remove(file_location) 81 | except OSError: 82 | pass 83 | joblib.dump(gridsearch, file_location) 84 | 85 | @classmethod 86 | def test_nn(cls, symbol='ALL'): 87 | 88 | dataFrameTest = load_data.LoadData.get_stock_prediction_data(symbol=symbol, type='test') 89 | 90 | # load the saved classifier 91 | file_location = 'NN_classifier_'+symbol+'.pkl' 92 | if os.path.isfile(file_location) is False: 93 | MLP.train_nn(symbol=symbol) 94 | 95 | classifier = joblib.load(file_location) 96 | predicted = classifier.predict(dataFrameTest.values) 97 | print(metrics.accuracy_score(dataFrameTest['label'].values, predicted)) 98 | print(metrics.classification_report(dataFrameTest['label'].values, predicted)) 99 | print(metrics.confusion_matrix(dataFrameTest['label'].values, predicted)) 100 | -------------------------------------------------------------------------------- /dnn_classifier/s&p500.txt: -------------------------------------------------------------------------------- 1 | aal 2 | aap 3 | aapl 4 | abbv 5 | abc 6 | abt 7 | acn 8 | adbe 9 | adi 10 | adm 11 | adp 12 | adsk 13 | aee 14 | aep 15 | aes 16 | aet 17 | afl 18 | agn 19 | aig 20 | aiv 21 | aiz 22 | ajg 23 | akam 24 | algn 25 | alk 26 | alle 27 | alxn 28 | amat 29 | amd 30 | ame 31 | amg 32 | amgn 33 | amzn 34 | andv 35 | anss 36 | antm 37 | aon 38 | aos 39 | apa 40 | apc 41 | apd 42 | aph 43 | aptv 44 | arnc 45 | atvi 46 | avb 47 | avgo 48 | avy 49 | awk 50 | axp 51 | ayi 52 | azo 53 | ba 54 | bac 55 | bax 56 | bbt 57 | bby 58 | bdx 59 | ben 60 | bf.b 61 | bhf 62 | bhge 63 | biib 64 | bkng 65 | blk 66 | bll 67 | bmy 68 | brk.b 69 | bsx 70 | bwa 71 | bxp 72 | cag 73 | cah 74 | cb 75 | cboe 76 | cbre 77 | cbs 78 | cci 79 | ccl 80 | cdns 81 | celg 82 | cern 83 | cfg 84 | chd 85 | chrw 86 | chtr 87 | ci 88 | cinf 89 | clx 90 | cma 91 | cmcsa 92 | cme 93 | cmg 94 | cmi 95 | cms 96 | cnc 97 | cnp 98 | cof 99 | coty 100 | cpb 101 | crm 102 | csco 103 | csra 104 | csx 105 | ctas 106 | ctl 107 | ctsh 108 | ctxs 109 | cvs 110 | cvx 111 | cxo 112 | dal 113 | de 114 | dfs 115 | dg 116 | dgx 117 | dhi 118 | dhr 119 | disca 120 | disck 121 | dlr 122 | dltr 123 | dov 124 | dps 125 | dre 126 | dri 127 | dte 128 | duk 129 | dva 130 | dvn 131 | dwdp 132 | dxc 133 | ebay 134 | ecl 135 | efx 136 | eix 137 | emn 138 | emr 139 | eog 140 | eqix 141 | eqr 142 | eqt 143 | esrx 144 | ess 145 | etfc 146 | etn 147 | etr 148 | evhc 149 | ew 150 | exc 151 | expd 152 | expe 153 | exr 154 | fb 155 | fbhs 156 | fcx 157 | fdx 158 | fe 159 | ffiv 160 | fis 161 | fisv 162 | fitb 163 | flir 164 | flr 165 | fls 166 | fmc 167 | foxa 168 | frt 169 | fti 170 | ftv 171 | gd 172 | ge 173 | ggp 174 | gis 175 | glw 176 | goog 177 | googl 178 | gpc 179 | gpn 180 | gps 181 | grmn 182 | gww 183 | hal 184 | hban 185 | hbi 186 | hca 187 | hcp 188 | hd 189 | hig 190 | hii 191 | hlt 192 | holx 193 | hpe 194 | hpq 195 | hrb 196 | hrl 197 | hsic 198 | hst 199 | hsy 200 | ibm 201 | idxx 202 | iff 203 | ilmn 204 | incy 205 | intc 206 | intu 207 | ip 208 | ipg 209 | ipgp 210 | iqv 211 | ir 212 | irm 213 | isrg 214 | itw 215 | ivz 216 | jbht 217 | jci 218 | jec 219 | jnj 220 | jnpr 221 | jpm 222 | jwn 223 | khc 224 | kim 225 | klac 226 | kmb 227 | kmi 228 | kmx 229 | ko 230 | kors 231 | kr 232 | kss 233 | ksu 234 | len 235 | lh 236 | lkq 237 | lll 238 | lly 239 | lmt 240 | lnc 241 | lnt 242 | lrcx 243 | luk 244 | luv 245 | lyb 246 | maa 247 | mcd 248 | mchp 249 | mck 250 | mco 251 | mdlz 252 | mdt 253 | mgm 254 | mhk 255 | mkc 256 | mlm 257 | mmc 258 | mmm 259 | mnst 260 | mon 261 | mpc 262 | mrk 263 | mro 264 | msft 265 | msi 266 | mtb 267 | mtd 268 | myl 269 | navi 270 | nbl 271 | nclh 272 | ndaq 273 | nee 274 | nem 275 | nflx 276 | nfx 277 | ni 278 | nke 279 | nktr 280 | nlsn 281 | noc 282 | nov 283 | nrg 284 | nsc 285 | ntap 286 | ntrs 287 | nue 288 | nvda 289 | nwl 290 | nws 291 | nwsa 292 | oke 293 | omc 294 | orcl 295 | oxy 296 | payx 297 | pbct 298 | pcar 299 | pcg 300 | pfe 301 | pfg 302 | pgr 303 | ph 304 | phm 305 | pki 306 | pld 307 | pnc 308 | pnr 309 | pnw 310 | ppg 311 | ppl 312 | prgo 313 | pru 314 | psa 315 | psx 316 | pvh 317 | pwr 318 | px 319 | pxd 320 | pypl 321 | qcom 322 | qrvo 323 | rcl 324 | regn 325 | rf 326 | rhi 327 | rht 328 | rjf 329 | rl 330 | rmd 331 | rok 332 | rop 333 | rost 334 | rrc 335 | rsg 336 | rtn 337 | sbac 338 | sbux 339 | scg 340 | schw 341 | shw 342 | sivb 343 | sjm 344 | slb 345 | slg 346 | sna 347 | snps 348 | spg 349 | spgi 350 | srcl 351 | sre 352 | sti 353 | stt 354 | stx 355 | stz 356 | swk 357 | swks 358 | syf 359 | syk 360 | symc 361 | syy 362 | tdg 363 | tgt 364 | tif 365 | tjx 366 | tmk 367 | tmo 368 | tpr 369 | trv 370 | tsco 371 | tsn 372 | tss 373 | ttwo 374 | twx 375 | txn 376 | txt 377 | ua 378 | uaa 379 | ual 380 | udr 381 | uhs 382 | ulta 383 | unh 384 | unm 385 | unp 386 | uri 387 | usb 388 | utx 389 | vfc 390 | viab 391 | vlo 392 | vmc 393 | vno 394 | vrsk 395 | vrsn 396 | vrtx 397 | vtr 398 | vz 399 | wat 400 | wba 401 | wdc 402 | wec 403 | wfc 404 | whr 405 | wltw 406 | wm 407 | wmb 408 | wmt 409 | wrk 410 | wu 411 | wy 412 | wyn 413 | wynn 414 | xec 415 | xel 416 | xl 417 | xlnx 418 | xom 419 | xray 420 | xrx 421 | xyl 422 | zbh 423 | zion 424 | zts 425 | -------------------------------------------------------------------------------- /data-extractor/labelled_data_extractor.py: -------------------------------------------------------------------------------- 1 | """ 2 | EXTRACT LABELLED DATA FROM STOCKTWITS API 3 | """ 4 | import csv 5 | import json 6 | import os 7 | import time 8 | from http_request_randomizer.requests.proxy.requestProxy import RequestProxy 9 | 10 | FIELDS = ['symbol', 'sentiment', 'message', 'message_id'] 11 | SYMBOL = ['AAPL', 'GOOGL', 'FB', 'AMZN'] 12 | SYMBOL_DICT = {'AMZN': 3, 'FB': 2, 'GOOGL': 1, 'AAPL': 0} 13 | SYMBOL_DICT_REV = {3: 'AMZN', 2: 'FB', 1: 'GOOGL', 0: 'AAPL'} 14 | FILE_NAME = 'sentiment_db.csv' 15 | token = 0 16 | access_token = ['', 'access_token=32a3552d31b92be5d2a3d282ca3a864f96e95818&', 17 | 'access_token=44ae93a5279092f7804a0ee04753252cbf2ddfee&', 18 | 'access_token=990183ef04060336a46a80aa287f774a9d604f9c&'] 19 | 20 | file = open(FILE_NAME, 'a', newline='', encoding='utf-8') 21 | # DETERMINE WHERE TO START IF RESUMING SCRIPT 22 | if os.stat(FILE_NAME).st_size == 0: 23 | # OPEN FILE IN APPEND MODE AND WRITE HEADERS TO FILE 24 | last_message_id = [None, None, None, None] 25 | csvfile = csv.DictWriter(file, FIELDS) 26 | csvfile.writeheader() 27 | else: 28 | # FIRST EXTRACT LAST MESSAGE ID THEN OPEN FILE IN APPEND MODE WITHOUT WRITING HEADERS 29 | # FILE MUST HAVE ALL THE STOCKS PRESENT INITIALLY 30 | file = open(FILE_NAME, 'r', newline='', encoding='utf-8') 31 | csvfile = csv.DictReader((line.replace('\0', '') for line in file)) 32 | data = list(csvfile) 33 | data_last = data[-1] 34 | symbol = data_last['symbol'] 35 | last_message_id = [None, None, None, None] 36 | index = SYMBOL_DICT[symbol] 37 | last_message_id[index] = data_last['message_id'] 38 | i = (index - 1) % (len(SYMBOL)) 39 | data_index = -2 40 | while i is not index: 41 | while data[data_index]['symbol'] == symbol: 42 | data_index -= 1 43 | symbol = data[data_index]['symbol'] 44 | data_last = data[data_index] 45 | last_message_id[i] = data[data_index]['message_id'] 46 | i = (i - 1) % (len(SYMBOL)) 47 | 48 | file.close() 49 | file = open(FILE_NAME, 'a', newline='', encoding='utf-8') 50 | csvfile = csv.DictWriter(file, FIELDS) 51 | 52 | req_proxy = RequestProxy() 53 | 54 | stocktwit_url = "https://api.stocktwits.com/api/2/streams/symbol/" + SYMBOL[token] + ".json?" + access_token[token] 55 | if last_message_id[token] is not None: 56 | stocktwit_url += "max=" + str(last_message_id[token]) 57 | 58 | api_hits = 0 59 | while True: 60 | response = req_proxy.generate_proxied_request(stocktwit_url) 61 | 62 | if response is not None: 63 | 64 | if response.status_code == 429: 65 | print("###############") 66 | print("REQUEST IP RATE LIMITED FOR {} seconds !!!".format( 67 | int(response.headers['X-RateLimit-Reset']) - int(time.time()))) 68 | 69 | if not response.status_code == 200: 70 | token = (token + 1) % (len(access_token)) 71 | stocktwit_url = "https://api.stocktwits.com/api/2/streams/symbol/" + SYMBOL[token] + ".json?" + \ 72 | access_token[token] + "max=" + str( 73 | last_message_id[token]) 74 | 75 | continue 76 | 77 | api_hits += 1 78 | response = json.loads(response.text) 79 | last_message_id[token] = response['cursor']['max'] 80 | null_sentiment_count = 0 81 | # WRITE DATA TO CSV FILE 82 | for message in response['messages']: 83 | # PREPARE OBJECT TO WRITE IN CSV FILE 84 | 85 | temp = message['entities']['sentiment'] 86 | if temp is not None and temp['basic']: 87 | obj = {} 88 | obj['symbol'] = SYMBOL[token] 89 | obj['message'] = message['body'] 90 | obj['sentiment'] = temp['basic'] 91 | obj['message_id'] = message['id'] 92 | csvfile.writerow(obj) 93 | file.flush() 94 | 95 | print("API HITS TILL NOW = {}".format(api_hits)) 96 | 97 | # ADD MAX ARGUMENT TO GET OLDER MESSAGES 98 | token = (token + 1) % (len(access_token)) 99 | stocktwit_url = "https://api.stocktwits.com/api/2/streams/symbol/" + SYMBOL[token] + ".json?" + access_token[ 100 | token] + "max=" + str(last_message_id[token]) 101 | 102 | file.close() 103 | -------------------------------------------------------------------------------- /sentiment_analysis.py: -------------------------------------------------------------------------------- 1 | """ 2 | perform sentiment analysis of stocktwits data 3 | """ 4 | 5 | import load_data 6 | import pandas as pd 7 | import matplotlib.pyplot as plt 8 | import os 9 | import numpy as np 10 | from nltk import word_tokenize, pos_tag 11 | from nltk.corpus import sentiwordnet as swn 12 | from nltk.wsd import lesk 13 | from nltk.stem.wordnet import WordNetLemmatizer 14 | 15 | class SentimentAnalysis: 16 | 17 | @classmethod 18 | def get_sentiword_score(cls, message): 19 | """ 20 | takes a message and performs following operations: 21 | 1) tokenize 22 | 2) POS tagging 23 | 3) reduce text to nouns, verbs, adjectives, adverbs 24 | 4) lemmatize the words 25 | 26 | for each selected tag, if more than one sense exists, performs word sense disambiguation 27 | using lesk algorithm and finally returns positivity score, negativity score from 28 | sentiwordnet lexicon 29 | """ 30 | 31 | tokens = word_tokenize(message) 32 | pos = pos_tag(tokens) 33 | lemmatizer = WordNetLemmatizer() 34 | selected_tags = list() 35 | scores = list() 36 | 37 | for i in range(len(pos)): 38 | if pos[i][1].startswith('J'): 39 | selected_tags.append((lemmatizer.lemmatize(pos[i][0], 'a'), 'a')) 40 | elif pos[i][1].startswith('V'): 41 | selected_tags.append((lemmatizer.lemmatize(pos[i][0], 'v'), 'v')) 42 | elif pos[i][1].startswith('N'): 43 | selected_tags.append((lemmatizer.lemmatize(pos[i][0], 'n'), 'n')) 44 | elif pos[i][1].startswith('R'): 45 | selected_tags.append((lemmatizer.lemmatize(pos[i][0], 'r'), 'r')) 46 | 47 | # score list: [(sense name, pos score, neg score)] 48 | for i in range(len(selected_tags)): 49 | senses = list(swn.senti_synsets(selected_tags[i][0], selected_tags[i][1])) 50 | if len(senses) == 1: 51 | scores.append((senses[0].synset.name(), senses[0].pos_score(), senses[0].neg_score())) 52 | elif len(senses) > 1: 53 | sense = lesk(tokens, selected_tags[i][0], selected_tags[i][1]) 54 | if sense is None: 55 | # take average score of all original senses 56 | pos_score = 0 57 | neg_score = 0 58 | for i in senses: 59 | pos_score += i.pos_score() 60 | neg_score += i.neg_score() 61 | scores.append((senses[0].synset.name(), pos_score/len(senses), neg_score/len(senses))) 62 | else: 63 | sense = swn.senti_synset(sense.name()) 64 | scores.append((sense.synset.name(), sense.pos_score(), sense.neg_score())) 65 | 66 | """ 67 | there are a number of ways for aggregating sentiment scores 68 | 1) sum up all scores 69 | 2) average all scores (or only for non zero scores) 70 | 3) (1) or (2) but only for adjectives 71 | 4) if pos score greater than neg score +1 vote else -1 vote 72 | here we are summing up the positive and negative scores to be used by classifier. 73 | whenever we encounter a negative word, we reverse the positive and negative score. 74 | """ 75 | 76 | # collected from word stat financial dictionary 77 | negation_words = list(open('data-extractor/lexicon_negation_words.txt').read().split()) 78 | 79 | # final_score = 0 80 | # counter = 1 81 | # for score in scores: 82 | # if any(score[0].startswith(x) for x in negation_words): 83 | # counter *= -1 84 | # else: 85 | # if score[1] > score[2]: 86 | # final_score += counter*score[1] 87 | # elif score[1] < score[2]: 88 | # final_score -= counter*score[2] 89 | 90 | counter = 1 91 | pos_score = 0 92 | neg_score = 0 93 | for score in scores: 94 | if any(score[0].startswith(x) for x in negation_words): 95 | counter *= -1 96 | else: 97 | if counter == 1: 98 | pos_score += score[1] 99 | neg_score += score[2] 100 | elif counter == -1: 101 | pos_score += score[2] 102 | neg_score += score[1] 103 | 104 | final_score = [pos_score, neg_score] 105 | print(final_score) 106 | return final_score 107 | 108 | @classmethod 109 | def sentiword_data_analysis(cls, symbol): 110 | file_location = 'data-extractor/stocktwits_'+symbol+'_sentiwordnet_scored.csv' 111 | if os.path.isfile(file_location) is False: 112 | dataFrame = load_data.LoadData.get_stocktwits_data(symbol) 113 | dataFrame['sentiwordnet_score'] = dataFrame.apply(lambda x: SentimentAnalysis.get_sentiword_score(x['message']), axis = 1) 114 | dataFrame.to_csv(file_location, index=False) 115 | 116 | dataFrame = pd.read_csv(file_location) 117 | plt.hist(dataFrame['sentiwordnet_score'], bins=np.arange(-3.5, 4, 0.1), label=symbol) 118 | plt.legend(loc='upper right') 119 | plt.show() 120 | 121 | @classmethod 122 | def labelled_data_sentiwordnet_analysis(cls): 123 | file_location = 'data-extractor/labelled_data_sentiwordnet_scored.csv' 124 | if os.path.isfile(file_location) is False: 125 | dataFrame = load_data.LoadData.get_labelled_data() 126 | dataFrame['sentiwordnet_score'] = dataFrame.apply(lambda x: SentimentAnalysis.get_sentiword_score(x['message']), axis = 1) 127 | dataFrame.to_csv(file_location, index=False) 128 | 129 | dataFrame = pd.read_csv(file_location) 130 | plt.hist(dataFrame[dataFrame['sentiment']=='Bullish']['sentiwordnet_score'], bins=np.arange(-3.5, 4, 0.1), label='Bullish', alpha=0.5) 131 | plt.hist(dataFrame[dataFrame['sentiment']=='Bearish']['sentiwordnet_score'], bins=np.arange(-3.5, 4, 0.1), label='Bearish', alpha=0.5) 132 | plt.legend(loc='upper right') 133 | plt.show() 134 | -------------------------------------------------------------------------------- /dnn_classifier/tweets_classification_using_dnn.py: -------------------------------------------------------------------------------- 1 | import nltk 2 | from nltk.stem.lancaster import LancasterStemmer 3 | from nltk.stem import PorterStemmer 4 | from nltk.corpus import stopwords 5 | import sys 6 | import random 7 | import unicodedata 8 | import csv 9 | import re 10 | import numpy as np 11 | import tensorflow as tf 12 | import tflearn 13 | 14 | 15 | # version 2 16 | bullish = [] 17 | bearish = [] 18 | words = [] 19 | docs = [] 20 | training = [] 21 | output = [] 22 | dataset = [] 23 | 24 | 25 | tbl = dict.fromkeys(i for i in range(sys.maxunicode) 26 | if unicodedata.category(chr(i)).startswith(('P', 'S'))) 27 | 28 | 29 | # method to remove punctuations from sentences. 30 | def remove_punctuation(text): 31 | return text.translate(tbl) 32 | 33 | 34 | stemmer = PorterStemmer() 35 | stop_words = list(stopwords.words("english")) 36 | for i in range(0, len(stop_words)) : 37 | stop_words[i] = remove_punctuation(stop_words[i]) 38 | 39 | 40 | # remove links, digits and ticker symbols for S&P500 companies 41 | def clean_and_tokenize(sentence): 42 | 43 | sentence = remove_punctuation(sentence) 44 | sentence = re.sub("http\S+", "", sentence) 45 | sentence = re.sub("\d+", "", sentence) 46 | 47 | with open("s&p500.txt", "r") as file: 48 | 49 | contents = file.readlines() 50 | for i in range(0, len(contents)): 51 | contents[i] = contents[i][:-1] 52 | 53 | filtered_words = [] 54 | for word in nltk.word_tokenize(sentence): 55 | word = word.lower() 56 | if word not in contents : 57 | if word not in stop_words: 58 | filtered_words.append(word) 59 | 60 | stemmed_words = set(stemmer.stem(word.lower()) for word in filtered_words) 61 | 62 | return stemmed_words 63 | 64 | 65 | def get_data(limit): 66 | global bullish, bearish 67 | # limit is no. of rows to be considered 68 | with open("classified_tweets.csv") as csvfile: 69 | 70 | reader = csv.reader(csvfile) 71 | count = 0 72 | 73 | for row in reader: 74 | if row[0] == "Bullish": 75 | bullish.append(row[1]) 76 | elif row[0] == "Bearish": 77 | bearish.append(row[1]) 78 | if count > limit : 79 | break 80 | count = count + 1 81 | 82 | max_length = min(len(bearish), len(bullish)) 83 | bearish = bearish[:max_length] 84 | bullish = bullish[:max_length] 85 | 86 | 87 | print("stop words :") 88 | print(stop_words) 89 | 90 | 91 | # one issue remains that while dividing the datasets, bullish and bearish, may not come in equal proportions in training set 92 | def prepare_data(): 93 | global words 94 | 95 | # consider 0 is bearish and 1 is bullish 96 | for sentence in bearish: 97 | 98 | stemmed_words = clean_and_tokenize(sentence) 99 | words.extend(stemmed_words) 100 | docs.append((stemmed_words, 0)) 101 | 102 | for sentence in bullish: 103 | 104 | stemmed_words = clean_and_tokenize(sentence) 105 | words.extend(stemmed_words) 106 | docs.append((stemmed_words, 1)) 107 | 108 | 109 | words = sorted(list(set(words))) 110 | print(words) 111 | print(len(words)) 112 | 113 | for doc in docs: 114 | 115 | bow = [] 116 | token_words = doc[0] 117 | 118 | for w in words: 119 | if w in token_words: 120 | bow.append(1) 121 | else: 122 | bow.append(0) 123 | 124 | output = [0, 0] 125 | if doc[1] == 1: 126 | output[1] = 1 127 | else: 128 | output[0] = 1 129 | 130 | dataset.append([bow, output]) 131 | 132 | random.shuffle(dataset) 133 | 134 | 135 | def divide_data(ratio): 136 | 137 | train_length = (len(dataset)*ratio)//(1+ratio) 138 | 139 | train_set = dataset[:train_length] 140 | test_set = dataset[train_length+1:] 141 | 142 | train_x = list((np.array(train_set))[:, 0]) 143 | train_y = list((np.array(train_set))[:, 1]) 144 | 145 | print(train_length, len(dataset)) 146 | print(len(test_set), len(train_set)) 147 | 148 | return (train_set, test_set, train_x, train_y) 149 | 150 | 151 | def train_model(train_x, train_y, epoch, batchsize): 152 | 153 | tf.reset_default_graph() 154 | 155 | net = tflearn.input_data(shape = [None, len(train_x[0])]) 156 | net = tflearn.fully_connected(net, 8) 157 | net = tflearn.fully_connected(net, len(train_y[0]), activation = 'softmax') 158 | net = tflearn.regression(net) 159 | 160 | model = tflearn.DNN(net, tensorboard_dir = 'tflearn_logs') 161 | model.fit(train_x, train_y, n_epoch=epoch, batch_size=batchsize, show_metric=True) 162 | 163 | return model 164 | 165 | 166 | def get_features(sentence): 167 | 168 | stemmed_words = clean_and_tokenize(sentence) 169 | 170 | # bag of words 171 | bow = [0]*len(words) 172 | # print(stemmed_words) 173 | for s in stemmed_words: 174 | for i, w in enumerate(words): 175 | if w == s: 176 | bow[i] = 1 177 | 178 | return(np.array(bow)) 179 | 180 | 181 | def determine_accuracy(dataset, model): 182 | 183 | correct = 0 184 | total = 0 185 | for row in dataset: 186 | values = model.predict([np.array(row[0])]).tolist() 187 | if values[0][1] >=0.5 and row[1][1] == 1: 188 | correct = correct + 1 189 | elif values[0][0] >=0.5 and row[1][0] == 1: 190 | correct = correct + 1 191 | total = total + 1 192 | return ((correct/total)*100) 193 | 194 | 195 | def test_examples(model): 196 | 197 | sent = [""]*7 198 | sent[0] = "Sell all. " 199 | sent[1] = "Strongly Bullish " 200 | sent[2] = "Recesssion is coming soon." 201 | sent[3] = "what are you talking about ?" 202 | sent[4] = "Amazon is high today" 203 | sent[5] = "Hold on to google. Bound to increase." 204 | sent[6] = "Be cautious of google. It can fall." 205 | 206 | print(sent) 207 | for sentence in sent: 208 | 209 | print(get_features(sentence).shape) 210 | print(model.predict([get_features(sentence)])) 211 | 212 | 213 | def test(model, limit): 214 | 215 | with open("classified_tweets.csv", "r") as csvfile: 216 | 217 | total = 0 218 | correct = 0 219 | reader = csv.reader(csvfile) 220 | for row in reader: 221 | 222 | if len(row) >= 2: 223 | values = model.predict([get_features(row[1])]).tolist() 224 | if values[0][1] >=0.5 and row[0] == "Bullish": 225 | correct = correct + 1 226 | elif values[0][0] >=0.5 and row[0] == "Bearish": 227 | correct = correct + 1 228 | total = total + 1 229 | 230 | if total >= limit: 231 | break 232 | 233 | return ((correct/total)*100) 234 | 235 | """ 236 | Note for 20k tweets, trained on 1000 tweets : 237 | accuracy between 60 - 65 on 1000 epochs 238 | and around 65 on 3000 epochs 239 | 240 | all the changes actually reduced accuracy 241 | :( 242 | 243 | try to improve by considering only more frequent words 244 | 245 | 246 | """ 247 | 248 | if __name__ == "__main__": 249 | 250 | 251 | get_data(3000) 252 | prepare_data() 253 | train_set, test_set, train_x, train_y = divide_data(5) 254 | model = train_model(train_x, train_y, 3000, 8) 255 | print("accuracy in training set is : ", determine_accuracy(train_set, model)) 256 | print("accuracy in test set is : ", determine_accuracy(test_set, model)) 257 | print("accuracy on first x tweets is : ", test(model, 20000)) 258 | # test_examples() 259 | 260 | 261 | 262 | 263 | 264 | 265 | 266 | 267 | 268 | 269 | 270 | 271 | -------------------------------------------------------------------------------- /tweet_classification_using_rnn.py: -------------------------------------------------------------------------------- 1 | import nltk 2 | import string 3 | import re 4 | import pandas as pd 5 | import numpy as np 6 | import tensorflow as tf 7 | 8 | data = pd.read_csv('data-extractor/sentiment_db.csv') 9 | data =data.sample(frac=1).reset_index(drop=True) 10 | 11 | del data['message_id'] 12 | del data['symbol'] 13 | custom_stop_words = ['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you', "you're", "you've", "you'll", 14 | "you'd", 'your', 'yours', 'yourself', 'yourselves', 'he', 'him', 'his', 'himself', 'she', "she's", 15 | 'her', 'hers', 'herself', 'it', "it's", 'its', 'itself', 'they', 'them', 'their', 'theirs', 16 | 'themselves', 'what', 'which', 'who', 'whom', 'this', 'that', "that'll", 'these', 'those', 'am', 17 | 'is', 'are', 'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had', 'having', 'do', 'does', 18 | 'did', 'doing', 'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until', 'while', 19 | 'of', 'at', 'by', 'for', 'then', 'once', 'here', 'there', 'when', 'where', 'why', 'how', 'all', 20 | 'any', 'both', 'no', 'nor', 'not', 'only', 'own', 'same', 'so', 's', 't', 'can', 'will', 'just', 21 | 'don', "don't", 'should', "should've", 'now', 'd', 'll', 'm', 'o', 're', 've', 'y', 'ain', 'aren', 22 | "aren't", 'couldn', "couldn't", 'didn', "didn't", 'doesn', "doesn't", 'hadn', "hadn't", 'hasn', 23 | "hasn't", 'haven', "haven't", 'isn', "isn't", 'ma', 'mightn', "mightn't", 'mustn', "mustn't", 24 | 'needn', "needn't", 'shan', "shan't", 'shouldn', "shouldn't", 'wasn', "wasn't", 'weren', "weren't", 25 | 'won', "won't", 'wouldn', "wouldn't", "to"] 26 | 27 | stop_words = set(custom_stop_words) 28 | 29 | 30 | def tokenizer(s): 31 | s = re.sub(r'http\S+', '', s) 32 | s = re.sub(r'\$(\w+)', '', s) 33 | translate_table = dict((ord(char), None) for char in string.punctuation) 34 | s = s.translate(translate_table) 35 | tokens = nltk.word_tokenize(s) 36 | filtered_tokens = [] 37 | for word in tokens: 38 | if word.lower() not in stop_words: 39 | filtered_tokens.append(word.lower()) 40 | return filtered_tokens 41 | 42 | 43 | data['message'] = data['message'].apply(tokenizer) 44 | data['message_length'] = data['message'].apply(len) 45 | data = data[(data['message_length'] > 0)] 46 | 47 | 48 | def func(x): 49 | if x == 'Bullish': 50 | return 1 51 | return 0 52 | 53 | 54 | data.sentiment = data.sentiment.apply(func) 55 | words = [] 56 | 57 | for message in data.message: 58 | for word in message: 59 | words.append(word) 60 | 61 | from collections import Counter 62 | 63 | counts = Counter(words) 64 | vocab = sorted(counts, key=counts.get, reverse=True) 65 | vocab_to_int = {word: ii for ii, word in enumerate(vocab, 1)} 66 | max_len = 20 67 | # 99% data tokenizes to 20 words 68 | 69 | 70 | def fun(x): 71 | vec = [] 72 | zeroes = max_len - len(x) 73 | if zeroes > 0: 74 | for i in range(zeroes): 75 | vec.append(0) 76 | 77 | for i in range(max_len - len(vec)): 78 | word = x[i] 79 | vec.append(vocab_to_int[word]) 80 | return np.array(vec) 81 | 82 | 83 | data['encoded_message'] = data['message'].apply(fun) 84 | 85 | del data['message'] 86 | del data['message_length'] 87 | 88 | features = [] 89 | for vec in data.encoded_message: 90 | features.append(vec.tolist()) 91 | features = np.array(features) 92 | 93 | features = features[:120000] 94 | split_frac = 0.7 95 | 96 | split_index = int(split_frac * len(features)) 97 | 98 | train_x, val_x = features[:split_index], features[split_index:] 99 | train_y, val_y = data.sentiment[:split_index], data.sentiment[split_index:len(features)] 100 | 101 | split_frac = 0.3 102 | split_index = int(split_frac * len(val_x)) 103 | 104 | val_x, test_x = val_x[:split_index], val_x[split_index:] 105 | val_y, test_y = val_y[:split_index], val_y[split_index:] 106 | 107 | print("\t\t\tFeature Shapes:") 108 | print("Train set: \t\t{}".format(train_x.shape), 109 | "\nValidation set: \t{}".format(val_x.shape), 110 | "\nTest set: \t\t{}".format(test_x.shape)) 111 | print("label set: \t\t{}".format(train_y.shape), 112 | "\nValidation label set: \t{}".format(val_y.shape), 113 | "\nTest label set: \t\t{}".format(test_y.shape)) 114 | 115 | lstm_size = 256 116 | lstm_layers = 2 117 | batch_size = 1000 118 | learning_rate = 0.01 119 | 120 | n_words = len(vocab_to_int) + 1 # Add 1 for 0 added to vocab 121 | 122 | # Create the graph object 123 | tf.reset_default_graph() 124 | with tf.name_scope('inputs'): 125 | inputs_ = tf.placeholder(tf.int32, [None, None], name="inputs") 126 | labels_ = tf.placeholder(tf.int32, [None, None], name="labels") 127 | keep_prob = tf.placeholder(tf.float32, name="keep_prob") 128 | 129 | embed_size = 300 130 | 131 | with tf.name_scope("Embeddings"): 132 | embedding = tf.Variable(tf.random_uniform((n_words, embed_size), -1, 1)) 133 | embed = tf.nn.embedding_lookup(embedding, inputs_) 134 | 135 | 136 | def lstm_cell(): 137 | # Your basic LSTM cell 138 | lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size, reuse=tf.get_variable_scope().reuse) 139 | # Add dropout to the cell 140 | return tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob) 141 | 142 | 143 | with tf.name_scope("RNN_layers"): 144 | # Stack up multiple LSTM layers, for deep learning 145 | cell = tf.contrib.rnn.MultiRNNCell([lstm_cell() for _ in range(lstm_layers)]) 146 | 147 | # Getting an initial state of all zeros 148 | initial_state = cell.zero_state(batch_size, tf.float32) 149 | 150 | with tf.name_scope("RNN_forward"): 151 | outputs, final_state = tf.nn.dynamic_rnn(cell, embed, initial_state=initial_state) 152 | 153 | with tf.name_scope('predictions'): 154 | predictions = tf.contrib.layers.fully_connected(outputs[:, -1], 1, activation_fn=tf.sigmoid) 155 | tf.summary.histogram('predictions', predictions) 156 | with tf.name_scope('cost'): 157 | cost = tf.losses.mean_squared_error(labels_, predictions) 158 | tf.summary.scalar('cost', cost) 159 | 160 | with tf.name_scope('train'): 161 | optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost) 162 | 163 | merged = tf.summary.merge_all() 164 | 165 | with tf.name_scope('validation'): 166 | correct_pred = tf.equal(tf.cast(tf.round(predictions), tf.int32), labels_) 167 | accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32)) 168 | 169 | 170 | def get_batches(x, y, batch_size=100): 171 | n_batches = len(x) // batch_size 172 | x, y = x[:n_batches * batch_size], y[:n_batches * batch_size] 173 | for ii in range(0, len(x), batch_size): 174 | yield x[ii:ii + batch_size], y[ii:ii + batch_size] 175 | 176 | 177 | epochs = 10 178 | 179 | # with graph.as_default(): 180 | saver = tf.train.Saver() 181 | 182 | with tf.Session() as sess: 183 | sess.run(tf.global_variables_initializer()) 184 | train_writer = tf.summary.FileWriter('./logs/tb/train', sess.graph) 185 | test_writer = tf.summary.FileWriter('./logs/tb/test', sess.graph) 186 | iteration = 1 187 | for e in range(epochs): 188 | state = sess.run(initial_state) 189 | 190 | for ii, (x, y) in enumerate(get_batches(train_x, train_y, batch_size), 1): 191 | feed = {inputs_: x, 192 | labels_: y[:, None], 193 | keep_prob: 0.5, 194 | initial_state: state} 195 | summary, loss, state, _ = sess.run([merged, cost, final_state, optimizer], feed_dict=feed) 196 | # loss, state, _ = sess.run([cost, final_state, optimizer], feed_dict=feed) 197 | 198 | train_writer.add_summary(summary, iteration) 199 | 200 | if iteration % 5 == 0: 201 | print("Epoch: {}/{}".format(e, epochs), 202 | "Iteration: {}".format(iteration), 203 | "Train loss: {:.3f}".format(loss)) 204 | 205 | if iteration % 25 == 0: 206 | val_acc = [] 207 | val_state = sess.run(cell.zero_state(batch_size, tf.float32)) 208 | for x, y in get_batches(val_x, val_y, batch_size): 209 | feed = {inputs_: x, 210 | labels_: y[:, None], 211 | keep_prob: 1, 212 | initial_state: val_state} 213 | # batch_acc, val_state = sess.run([accuracy, final_state], feed_dict=feed) 214 | summary, batch_acc, val_state = sess.run([merged, accuracy, final_state], feed_dict=feed) 215 | val_acc.append(batch_acc) 216 | print("Val acc: {:.3f}".format(np.mean(val_acc))) 217 | iteration += 1 218 | test_writer.add_summary(summary, iteration) 219 | saver.save(sess, "checkpoints/sentiment_jatin.ckpt") 220 | saver.save(sess, "checkpoints/sentiment_jatin.ckpt") 221 | 222 | test_acc = [] 223 | 224 | with tf.Session() as sess: 225 | saver.restore(sess, "checkpoints/sentiment_jatin.ckpt") 226 | test_state = sess.run(cell.zero_state(batch_size, tf.float32)) 227 | for ii, (x, y) in enumerate(get_batches(test_x, test_y, batch_size), 1): 228 | feed = {inputs_: x, 229 | labels_: y[:, None], 230 | keep_prob: 1, 231 | initial_state: test_state} 232 | batch_acc, test_state = sess.run([accuracy, final_state], feed_dict=feed) 233 | test_acc.append(batch_acc) 234 | print("Test accuracy: {:.3f}".format(np.mean(test_acc))) 235 | -------------------------------------------------------------------------------- /load_data.py: -------------------------------------------------------------------------------- 1 | """ 2 | handle preprocessing and loading of data. 3 | """ 4 | 5 | import html 6 | import os.path 7 | import pandas as pd 8 | import re 9 | from nltk import word_tokenize, pos_tag 10 | from nltk.corpus import stopwords, wordnet 11 | from nltk.stem.wordnet import WordNetLemmatizer 12 | 13 | class LoadData: 14 | 15 | @classmethod 16 | def preprocess_stocktwits_data(cls, file_location, columns=['datetime', 'message']): 17 | """ 18 | preprocess the data in file location and saves it as a csv file (appending 19 | '_preprocessed' before '.csv). The preprocessing us in following ways: 20 | 1) extract message and datetime columns. 21 | 2) sort according to datetime in descending order (newest first) 22 | 3) remove links, @ and $ references, extra whitespaces, extra '.', digits, slashes, 23 | hyphons 24 | 4) decode html entities 25 | 5) convert everything to lower case 26 | """ 27 | 28 | if 'datetime' in columns: 29 | dataFrame = pd.read_csv(file_location, usecols=columns, parse_dates=['datetime'], infer_datetime_format=True) 30 | dataFrame.sort_values(by='datetime', ascending=False) 31 | else: 32 | dataFrame = pd.read_csv(file_location, usecols=columns) 33 | 34 | dataFrame['message'] = dataFrame['message'].apply(lambda x: html.unescape(x)) 35 | dataFrame['message'] = dataFrame['message'].apply(lambda x: re.sub(r'(www\.|https?://).*?(\s|$)|@.*?(\s|$)|\$.*?(\s|$)|\d|\%|\\|/|-|_', ' ', x)) 36 | dataFrame['message'] = dataFrame['message'].apply(lambda x: re.sub(r'\.+', '. ', x)) 37 | dataFrame['message'] = dataFrame['message'].apply(lambda x: re.sub(r'\,+', ', ', x)) 38 | dataFrame['message'] = dataFrame['message'].apply(lambda x: re.sub(r'\?+', '? ', x)) 39 | dataFrame['message'] = dataFrame['message'].apply(lambda x: re.sub(r'\s+', ' ', x)) 40 | dataFrame['message'] = dataFrame['message'].apply(lambda x: x.lower()) 41 | 42 | dataFrame.to_csv(file_location[:-4]+'_preprocessed.csv', index=False) 43 | 44 | @classmethod 45 | def labelled_data_lexicon_analysis(cls): 46 | """ 47 | extract keywords from labelled stocktwits data for improved accuracy in scoring 48 | for each labelled message do 49 | 1) tokenize the message 50 | 2) perform POS tagging 51 | 3) if a sense is present in wordnet then, lemmatize the word and remove stop words else ignore the word 52 | remove intersections from the two lists before saving 53 | """ 54 | 55 | dataFrame = LoadData.get_labelled_data() 56 | bullish_keywords = set() 57 | bearish_keywords = set() 58 | lemmatizer = WordNetLemmatizer() 59 | stop_words = set(stopwords.words('english')) 60 | for index, row in dataFrame.iterrows(): 61 | tokens = word_tokenize(row['message']) 62 | pos = pos_tag(tokens) 63 | selected_tags = set() 64 | 65 | for i in range(len(pos)): 66 | if len(wordnet.synsets(pos[i][0])): 67 | if pos[i][1].startswith('J'): 68 | selected_tags.add(lemmatizer.lemmatize(pos[i][0], 'a')) 69 | elif pos[i][1].startswith('V'): 70 | selected_tags.add(lemmatizer.lemmatize(pos[i][0], 'v')) 71 | elif pos[i][1].startswith('N'): 72 | selected_tags.add(lemmatizer.lemmatize(pos[i][0], 'n')) 73 | elif pos[i][1].startswith('R'): 74 | selected_tags.add(lemmatizer.lemmatize(pos[i][0], 'r')) 75 | selected_tags -= stop_words 76 | 77 | if row['sentiment'] == 'Bullish': 78 | bullish_keywords = bullish_keywords.union(selected_tags) 79 | elif row['sentiment'] == 'Bearish': 80 | bearish_keywords = bearish_keywords.union(selected_tags) 81 | 82 | updated_bullish_keywords = bullish_keywords - bearish_keywords 83 | updated_bearish_keywords = bearish_keywords - bullish_keywords 84 | with open('data-extractor/lexicon_bullish_words.txt', 'a') as file: 85 | for word in updated_bullish_keywords: 86 | file.write(word+"\n") 87 | with open('data-extractor/lexicon_bearish_words.txt', 'a') as file: 88 | for word in updated_bearish_keywords: 89 | file.write(word+"\n") 90 | 91 | @classmethod 92 | def get_stocktwits_data(cls, symbol): 93 | """ 94 | get_data loads the preprocessed data of 'symbol' from data-extractor 95 | and returns a pandas dataframe with columns [message(object), datetime(datetime64[ns])]. 96 | """ 97 | 98 | file_location = 'data-extractor/stocktwits_'+symbol+'_preprocessed.csv' 99 | if os.path.isfile(file_location) is False: 100 | LoadData.preprocess_stocktwits_data('data-extractor/stocktwits_'+symbol+'.csv') 101 | 102 | dataFrame = pd.read_csv(file_location) 103 | return dataFrame 104 | 105 | @classmethod 106 | def get_price_data(cls, symbol): 107 | """ 108 | loads the price data of 'symbol' from data-extractor 109 | and returns a pandas dataframe with columns [Date(datetime64[ns]), Opening Price(float64), Closing Price(float64), Volume(float64)]. 110 | """ 111 | 112 | file_location = 'data-extractor/stock_prices_'+symbol+'.csv' 113 | dataFrame = pd.read_csv(file_location, usecols=['Date', 'Opening Price', 'Closing Price', 'Volume'], parse_dates=['Date'], infer_datetime_format=True) 114 | return dataFrame 115 | 116 | @classmethod 117 | def get_labelled_data(cls, type='complete'): 118 | """ 119 | get_labelled_data loads the preprocessed labelled data of stocktwits from data-extractor 120 | and returns a pandas dataframe with columns [sentiment(object), message(object)]. 121 | """ 122 | 123 | if type == 'complete': 124 | file_location = 'data-extractor/labelled_data_complete_preprocessed.csv' 125 | if os.path.isfile(file_location) is False: 126 | LoadData.preprocess_stocktwits_data('data-extractor/labelled_data_complete.csv', columns=['sentiment', 'message']) 127 | elif type == 'training': 128 | file_location = 'data-extractor/labelled_data_training_preprocessed.csv' 129 | if os.path.isfile(file_location) is False: 130 | LoadData.get_training_data() 131 | elif type == 'test': 132 | file_location = 'data-extractor/labelled_data_test_preprocessed.csv' 133 | if os.path.isfile(file_location) is False: 134 | LoadData.preprocess_stocktwits_data('data-extractor/labelled_data_test.csv', columns=['sentiment', 'message']) 135 | 136 | dataFrame = pd.read_csv(file_location) 137 | return dataFrame 138 | 139 | @classmethod 140 | def get_custom_lexicon(cls): 141 | """ 142 | get custom lexicon of bearish and bullish words respectively 143 | """ 144 | 145 | file_location1 = 'data-extractor/lexicon_bearish_words.txt' 146 | file_location2 = 'data-extractor/lexicon_bullish_words.txt' 147 | if os.path.isfile(file_location1) is False or os.path.isfile(file_location2) is False: 148 | LoadData.labelled_data_lexicon_analysis() 149 | 150 | dataFrameBearish = pd.read_csv(file_location1, header=None, names=['word']) 151 | dataFrameBullish = pd.read_csv(file_location2, header=None, names=['word']) 152 | return dataFrameBearish, dataFrameBullish 153 | 154 | @classmethod 155 | def get_training_data(cls): 156 | """ 157 | get labelled training data with equal bearish and bullish messages 158 | """ 159 | try: 160 | os.remove('data-extractor/labelled_data_training.csv') 161 | except OSError: 162 | pass 163 | 164 | dataFrame = LoadData.get_labelled_data(type='complete') 165 | dataFrameBearish = dataFrame[dataFrame['sentiment']=='Bearish'] 166 | dataFrameBullish = dataFrame[dataFrame['sentiment']=='Bullish'] 167 | dataFrameBearishTraining = dataFrameBearish 168 | dataFrameBullishTraining = dataFrameBullish[:len(dataFrameBearish)] 169 | 170 | dataFrameTraining = dataFrameBearishTraining.append(dataFrameBullishTraining, ignore_index=True).sample(frac=1).reset_index(drop=True) 171 | dataFrameTraining.to_csv('data-extractor/labelled_data_training_preprocessed.csv', index=False) 172 | 173 | @classmethod 174 | def combine_price_and_sentiment(cls, sentimentFrame, priceFrame): 175 | from datetime import timedelta 176 | 177 | """ 178 | receive sentimentFrame as (date, sentiment, message) indexed by date and sentiment 179 | and priceFrame as (Date, Opening Price, Closing Price, Volume) and return a combined 180 | frame as (sentiment_calculated_bullish, sentiment_calculated_bearish, 181 | sentiment_actual_previous, tweet_volume_change, cash_volume, label) 182 | """ 183 | 184 | dataFrame = pd.DataFrame() 185 | for date, df in sentimentFrame.groupby(level=0, sort=False): 186 | 187 | price_current = priceFrame[priceFrame['Date'] == date] 188 | if price_current.empty or date-timedelta(days=1) not in sentimentFrame.index: 189 | continue 190 | tweet_minus1 = sentimentFrame.loc[date-timedelta(days=1)] 191 | days = 1 192 | price_plus1 = priceFrame[priceFrame['Date'] == date+timedelta(days=days)] 193 | while price_plus1.empty: 194 | days += 1 195 | price_plus1 = priceFrame[priceFrame['Date'] == date+timedelta(days=days)] 196 | days = 1 197 | price_minus1 = priceFrame[priceFrame['Date'] == date-timedelta(days=days)] 198 | while price_minus1.empty: 199 | days += 1 200 | price_minus1 = priceFrame[priceFrame['Date'] == date-timedelta(days=days)] 201 | 202 | new_row = {} 203 | new_row['date'] = date 204 | new_row['sentiment_calculated_bullish'] = df.loc[(date, 'Bullish')]['message'] 205 | new_row['sentiment_calculated_bearish'] = df.loc[(date, 'Bearish')]['message'] 206 | new_row['sentiment_actual_previous'] = 1 if ((price_minus1.iloc[0]['Closing Price'] - price_minus1.iloc[0]['Opening Price']) >= 0) else -1 207 | new_row['tweet_volume_change'] = df['message'].sum() - tweet_minus1['message'].sum() 208 | new_row['cash_volume'] = price_current['Volume'].iloc[0] 209 | new_row['label'] = 1 if ((price_plus1.iloc[0]['Closing Price'] - price_current.iloc[0]['Closing Price']) >= 0) else -1 210 | print(new_row) 211 | dataFrame = dataFrame.append(new_row, ignore_index=True) 212 | 213 | return dataFrame 214 | 215 | @classmethod 216 | def aggregate_stock_price_data(cls): 217 | """ 218 | compile stocktwits data for stock prediction analysis in the following form 219 | (date, sentiment_calculated_bullish, sentiment_calculated_bearish, sentiment_actual_previous, tweet_volume_change, cash_volume, label) 220 | 221 | we have choice to take previous n days sentiment_calculated and using label of next nth day 222 | 223 | returns dataframes for AAPL, AMZN, GOOGL respectively 224 | """ 225 | 226 | if not (os.path.isfile('data-extractor/stocktwits_AAPL_sharedata.csv') and os.path.isfile('data-extractor/stocktwits_AMZN_sharedata.csv') and os.path.isfile('data-extractor/stocktwits_GOOGL_sharedata.csv')): 227 | 228 | from sklearn.externals import joblib 229 | file_location = 'naive_bayes_classifier.pkl' 230 | 231 | priceAAPL = LoadData.get_price_data('AAPL') 232 | priceAMZN = LoadData.get_price_data('AMZN') 233 | priceGOOGL = LoadData.get_price_data('GOOGL') 234 | 235 | sentimented_file = 'data-extractor/stocktwits_AAPL_withsentiment.csv' 236 | if os.path.isfile(sentimented_file) is False: 237 | tweet_classifier = joblib.load(file_location) 238 | dataAAPL = LoadData.get_stocktwits_data('AAPL') 239 | dataAAPL['sentiment'] = dataAAPL['message'].apply(lambda x: tweet_classifier.predict([x])[0]) 240 | dataAAPL['datetime'] = dataAAPL['datetime'].apply(lambda x: x.date()) 241 | dataAAPL.rename(columns={'datetime':'date'}, inplace=True) 242 | dataAAPL.to_csv('data-extractor/stocktwits_AAPL_withsentiment.csv', index=False) 243 | sentimented_file = 'data-extractor/stocktwits_AMZN_withsentiment.csv' 244 | if os.path.isfile(sentimented_file) is False: 245 | tweet_classifier = joblib.load(file_location) 246 | dataAMZN = LoadData.get_stocktwits_data('AMZN') 247 | dataAMZN['sentiment'] = dataAMZN['message'].apply(lambda x: tweet_classifier.predict([x])[0]) 248 | dataAMZN['datetime'] = dataAMZN['datetime'].apply(lambda x: x.date()) 249 | dataAMZN.rename(columns={'datetime':'date'}, inplace=True) 250 | dataAMZN.to_csv('data-extractor/stocktwits_AMZN_withsentiment.csv', index=False) 251 | sentimented_file = 'data-extractor/stocktwits_GOOGL_withsentiment.csv' 252 | if os.path.isfile(sentimented_file) is False: 253 | tweet_classifier = joblib.load(file_location) 254 | dataGOOGL = LoadData.get_stocktwits_data('GOOGL') 255 | dataGOOGL['sentiment'] = dataGOOGL['message'].apply(lambda x: tweet_classifier.predict([x])[0]) 256 | dataGOOGL['datetime'] = dataGOOGL['datetime'].apply(lambda x: x.date()) 257 | dataGOOGL.rename(columns={'datetime':'date'}, inplace=True) 258 | dataGOOGL.to_csv('data-extractor/stocktwits_GOOGL_withsentiment.csv', index=False) 259 | 260 | dataAAPL = pd.read_csv('data-extractor/stocktwits_AAPL_withsentiment.csv', parse_dates=['date'], infer_datetime_format=True) 261 | dataAMZN = pd.read_csv('data-extractor/stocktwits_AMZN_withsentiment.csv', parse_dates=['date'], infer_datetime_format=True) 262 | dataGOOGL = pd.read_csv('data-extractor/stocktwits_GOOGL_withsentiment.csv', parse_dates=['date'], infer_datetime_format=True) 263 | dataAAPL = dataAAPL.groupby(['date','sentiment'], sort=False).count() 264 | dataAMZN = dataAMZN.groupby(['date','sentiment'], sort=False).count() 265 | dataGOOGL = dataGOOGL.groupby(['date','sentiment'], sort=False).count() 266 | dataAAPL = LoadData.combine_price_and_sentiment(dataAAPL, priceAAPL) 267 | dataAMZN = LoadData.combine_price_and_sentiment(dataAMZN, priceAMZN) 268 | dataGOOGL = LoadData.combine_price_and_sentiment(dataGOOGL, priceGOOGL) 269 | dataAAPL.to_csv('data-extractor/stocktwits_AAPL_sharedata.csv', index=False) 270 | dataAMZN.to_csv('data-extractor/stocktwits_AMZN_sharedata.csv', index=False) 271 | dataGOOGL.to_csv('data-extractor/stocktwits_GOOGL_sharedata.csv', index=False) 272 | 273 | dataAAPL = pd.read_csv('data-extractor/stocktwits_AAPL_sharedata.csv', parse_dates=['date'], infer_datetime_format=True) 274 | dataAMZN = pd.read_csv('data-extractor/stocktwits_AMZN_sharedata.csv', parse_dates=['date'], infer_datetime_format=True) 275 | dataGOOGL = pd.read_csv('data-extractor/stocktwits_GOOGL_sharedata.csv', parse_dates=['date'], infer_datetime_format=True) 276 | 277 | return dataAAPL, dataAMZN, dataGOOGL 278 | 279 | @classmethod 280 | def get_stock_prediction_data(cls, symbol='ALL', type='training'): 281 | 282 | """ 283 | get the training and test data for stock prediction in format 284 | (sentiment_calculated_bullish, sentiment_calculated_bearish, sentiment_actual_previous, 285 | tweet_volume_change, cash_volume, label) 286 | 287 | Standardize the data before using. 288 | """ 289 | 290 | file_location = 'data-extractor/stockdata_'+symbol+'_'+type+'.csv' 291 | if not os.path.isfile(file_location): 292 | import numpy as np 293 | 294 | dataAAPL, dataAMZN, dataGOOGL = LoadData.aggregate_stock_price_data() 295 | combined_data = dataAAPL.append([dataAMZN, dataGOOGL], ignore_index=True) 296 | combined_data.sort_values('date') 297 | combined_data.drop(columns='date', inplace=True) 298 | combined_training, combined_test = np.split(combined_data.sample(frac=1), [int(.9*len(combined_data))]) 299 | combined_training.to_csv('data-extractor/stockdata_ALL_training.csv', index=False) 300 | combined_test.to_csv('data-extractor/stockdata_ALL_test.csv', index=False) 301 | 302 | dataAAPL.sort_values('date') 303 | dataAAPL.drop(columns='date', inplace=True) 304 | AAPL_training, AAPL_test = np.split(dataAAPL.sample(frac=1), [int(.9*len(dataAAPL))]) 305 | AAPL_training.to_csv('data-extractor/stockdata_AAPL_training.csv', index=False) 306 | AAPL_test.to_csv('data-extractor/stockdata_AAPL_test.csv', index=False) 307 | 308 | dataAMZN.sort_values('date') 309 | dataAMZN.drop(columns='date', inplace=True) 310 | AMZN_training, AMZN_test = np.split(dataAMZN.sample(frac=1), [int(.9*len(dataAMZN))]) 311 | AMZN_training.to_csv('data-extractor/stockdata_AMZN_training.csv', index=False) 312 | AMZN_test.to_csv('data-extractor/stockdata_AMZN_test.csv', index=False) 313 | 314 | dataGOOGL.sort_values('date') 315 | dataGOOGL.drop(columns='date', inplace=True) 316 | GOOGL_training, GOOGL_test = np.split(dataGOOGL.sample(frac=1), [int(.9*len(dataGOOGL))]) 317 | GOOGL_training.to_csv('data-extractor/stockdata_GOOGL_training.csv', index=False) 318 | GOOGL_test.to_csv('data-extractor/stockdata_GOOGL_test.csv', index=False) 319 | 320 | data = pd.read_csv(file_location) 321 | return data 322 | -------------------------------------------------------------------------------- /data-extractor/lexicon_bearish_words.txt: -------------------------------------------------------------------------------- 1 | fledge 2 | parabola 3 | pigs 4 | protectionism 5 | sombrero 6 | assembly 7 | commits 8 | wounded 9 | ltm 10 | placing 11 | installs 12 | distress 13 | willies 14 | touching 15 | disgusted 16 | ovum 17 | suppler 18 | politic 19 | gigs 20 | weakly 21 | enthusiast 22 | three 23 | cpa 24 | snooping 25 | despised 26 | nigh 27 | locust 28 | victoria 29 | emergent 30 | tales 31 | anonymity 32 | awash 33 | overcrowd 34 | unattractive 35 | exclaim 36 | underscore 37 | vulture 38 | bate 39 | manchu 40 | erosion 41 | backstop 42 | authorise 43 | inefficiency 44 | baldy 45 | imperfect 46 | institutionally 47 | hoarding 48 | hovering 49 | invasive 50 | federation 51 | maxwell 52 | scorpio 53 | troop 54 | depreciating 55 | blasting 56 | gated 57 | cox 58 | thinner 59 | dyeing 60 | asocial 61 | experimentation 62 | oath 63 | deplete 64 | unintelligent 65 | citizenship 66 | schoolgirl 67 | grandiose 68 | repay 69 | poof 70 | isolation 71 | spasm 72 | males 73 | wrath 74 | cumulated 75 | sth 76 | unpaid 77 | gulag 78 | riddle 79 | pokey 80 | traitorous 81 | frypan 82 | ants 83 | dresser 84 | heir 85 | jawboning 86 | dictator 87 | heroes 88 | recuperate 89 | declared 90 | insecurity 91 | resells 92 | umbilical 93 | teleprompter 94 | sung 95 | reciprocity 96 | shakedown 97 | beholden 98 | opportune 99 | dumpling 100 | radiation 101 | remembered 102 | administrator 103 | commoner 104 | overrated 105 | classically 106 | graffiti 107 | forecaster 108 | recessionary 109 | eerily 110 | confess 111 | homo 112 | complexion 113 | malfeasance 114 | undergoes 115 | knot 116 | detected 117 | gigo 118 | disappearance 119 | squirrel 120 | threefold 121 | treasonous 122 | hannibal 123 | flack 124 | homosexual 125 | deft 126 | rats 127 | electorate 128 | spoiler 129 | cocain 130 | controlling 131 | castrate 132 | fiesta 133 | ruse 134 | rein 135 | absurdly 136 | slipping 137 | dismemberment 138 | sniper 139 | vacant 140 | pushover 141 | egotist 142 | italian 143 | bracing 144 | defang 145 | ensues 146 | cloister 147 | tangled 148 | joel 149 | provocative 150 | bolivia 151 | advising 152 | bratty 153 | finicky 154 | gremlin 155 | rue 156 | sawdust 157 | choker 158 | undergrad 159 | swig 160 | americas 161 | unwise 162 | phoney 163 | kremlin 164 | redistribute 165 | adamant 166 | trench 167 | deserving 168 | skulduggery 169 | configuration 170 | sensibility 171 | treadmill 172 | kipper 173 | dongle 174 | arrogantly 175 | crax 176 | debilitate 177 | impedes 178 | genesis 179 | railroad 180 | congratulates 181 | shuttered 182 | pfc 183 | overbear 184 | unzip 185 | panorama 186 | ness 187 | barley 188 | sinister 189 | acquaintance 190 | lipstick 191 | bastion 192 | stones 193 | topples 194 | degraded 195 | candid 196 | reincarnation 197 | airspace 198 | screech 199 | roadblock 200 | levee 201 | afoot 202 | junky 203 | sprung 204 | iranian 205 | macaroni 206 | unpopular 207 | decaying 208 | perpetual 209 | firecracker 210 | yucky 211 | devolve 212 | livid 213 | fps 214 | honk 215 | retool 216 | rb 217 | maldives 218 | pincus 219 | disagreeable 220 | voyage 221 | fond 222 | hardcore 223 | olympian 224 | absent 225 | justifies 226 | muslim 227 | humane 228 | draft 229 | ominous 230 | decidedly 231 | chickenshit 232 | usefulness 233 | irrelevance 234 | cynicism 235 | damper 236 | troublesome 237 | childless 238 | sodomize 239 | duel 240 | coaching 241 | offender 242 | hank 243 | precarious 244 | shamelessly 245 | forehead 246 | sly 247 | whirlwind 248 | karaoke 249 | watergate 250 | derailed 251 | bjs 252 | retracing 253 | summons 254 | competence 255 | sceptical 256 | rescind 257 | sporty 258 | funk 259 | spams 260 | frigate 261 | pixie 262 | marshmallow 263 | symptom 264 | mumbai 265 | gandhi 266 | woosh 267 | interpol 268 | seam 269 | deleting 270 | mealy 271 | stalker 272 | bittersweet 273 | dressy 274 | wort 275 | engulfings 276 | spectator 277 | stealer 278 | plainly 279 | areal 280 | decaf 281 | nearer 282 | recess 283 | decibel 284 | antarctica 285 | undefeated 286 | bureau 287 | unionize 288 | repetition 289 | pornographic 290 | blurry 291 | scrapbooks 292 | illustrates 293 | roam 294 | unprepared 295 | barge 296 | prism 297 | exile 298 | polarization 299 | hardest 300 | cracked 301 | racial 302 | begining 303 | eft 304 | digress 305 | deceitful 306 | aluminize 307 | lier 308 | fabrication 309 | standstill 310 | schizophrenic 311 | sds 312 | unravel 313 | laureate 314 | printed 315 | diary 316 | classroom 317 | crappie 318 | belgium 319 | swatch 320 | filler 321 | partake 322 | obfuscation 323 | contrite 324 | turndown 325 | cocksucker 326 | arguing 327 | facer 328 | breakdowns 329 | marking 330 | sputter 331 | noninvasive 332 | latecomer 333 | depose 334 | erode 335 | rebuff 336 | scuba 337 | judgmental 338 | miscellaneous 339 | primed 340 | unspecified 341 | handwriting 342 | lyric 343 | recklessly 344 | steamrolling 345 | mindlessly 346 | sweetest 347 | overzealous 348 | served 349 | carol 350 | tombstone 351 | peso 352 | ardent 353 | featured 354 | courier 355 | yon 356 | investigator 357 | unchecked 358 | revisits 359 | aristotle 360 | fugitive 361 | frantic 362 | alligator 363 | meager 364 | primo 365 | overworked 366 | sweetly 367 | rube 368 | handler 369 | preparing 370 | opposition 371 | islamist 372 | horrify 373 | adas 374 | youngster 375 | fluoridated 376 | inn 377 | commencing 378 | spyware 379 | helpless 380 | unsightly 381 | sully 382 | deathblow 383 | rupture 384 | charisma 385 | apathy 386 | gassy 387 | foxy 388 | concerning 389 | decoupling 390 | prevents 391 | chad 392 | bobble 393 | mapped 394 | feverishly 395 | hugger 396 | sickness 397 | frightening 398 | drywall 399 | probing 400 | flooding 401 | maggot 402 | insulting 403 | brig 404 | unraveling 405 | joseph 406 | goldberg 407 | hallmark 408 | appetizing 409 | hamburger 410 | vanishes 411 | wanking 412 | swab 413 | medial 414 | ellis 415 | legally 416 | muddled 417 | instigate 418 | tamper 419 | sandal 420 | starboard 421 | henhouse 422 | flattening 423 | shivering 424 | tyne 425 | unsold 426 | hyderabad 427 | inner 428 | thieve 429 | reptile 430 | rookies 431 | desired 432 | imf 433 | slant 434 | shrinkage 435 | nausea 436 | laying 437 | wrongdoing 438 | unkind 439 | salami 440 | alas 441 | priming 442 | afterer 443 | pretending 444 | spar 445 | religiously 446 | troubled 447 | dkl 448 | usable 449 | wy 450 | recurrent 451 | reddish 452 | threatens 453 | crucified 454 | caste 455 | puking 456 | wiretap 457 | crummy 458 | dustbin 459 | reselling 460 | hick 461 | dismal 462 | gangster 463 | inkling 464 | grape 465 | conjure 466 | saddam 467 | sarcastic 468 | flammable 469 | blended 470 | catalogue 471 | punter 472 | substandard 473 | aiming 474 | crutch 475 | dialect 476 | dissonance 477 | exploiting 478 | populism 479 | sonic 480 | skyscraper 481 | anticipating 482 | egocentric 483 | buggy 484 | planner 485 | waif 486 | outstretched 487 | tampon 488 | ingest 489 | topped 490 | expenditures 491 | garb 492 | terminology 493 | porthole 494 | infarction 495 | abating 496 | forsaken 497 | plight 498 | evacuation 499 | adequate 500 | repayment 501 | misstep 502 | mod 503 | enclosure 504 | deflationary 505 | retouch 506 | sausage 507 | conscience 508 | peachy 509 | avocados 510 | hilo 511 | pinko 512 | grinder 513 | representative 514 | foreigner 515 | purveyor 516 | hosting 517 | armed 518 | ruff 519 | cavity 520 | yogi 521 | anecdotal 522 | fundus 523 | nosing 524 | contestant 525 | lore 526 | bellicose 527 | erroneous 528 | fledgling 529 | resin 530 | amusement 531 | overstocked 532 | lowlife 533 | overextended 534 | medic 535 | cactus 536 | stoked 537 | counterpart 538 | amok 539 | zu 540 | gender 541 | normality 542 | monopolist 543 | enviable 544 | occurring 545 | cushy 546 | unethically 547 | chariot 548 | combatant 549 | inequality 550 | hopelessly 551 | armored 552 | prays 553 | droopy 554 | nurture 555 | intrusion 556 | deport 557 | sugary 558 | abortion 559 | righteous 560 | levy 561 | whipper 562 | compressed 563 | imaginative 564 | irreversible 565 | skyline 566 | entree 567 | adverse 568 | amend 569 | barrack 570 | drinker 571 | sensory 572 | hypothesize 573 | duff 574 | faustian 575 | crp 576 | semitic 577 | prematurely 578 | speedup 579 | quilt 580 | subscript 581 | scottie 582 | clothe 583 | damp 584 | fatigue 585 | hypocrite 586 | handlebar 587 | nazis 588 | fend 589 | cid 590 | futurist 591 | outsource 592 | continuance 593 | detriment 594 | antique 595 | bennie 596 | faltering 597 | fined 598 | psychometrics 599 | migraine 600 | counted 601 | wily 602 | bh 603 | missing 604 | perennial 605 | receptor 606 | disable 607 | actuation 608 | viability 609 | psyop 610 | poser 611 | ebullience 612 | dmz 613 | volt 614 | herbal 615 | underpricing 616 | rosary 617 | skills 618 | parenthood 619 | sedation 620 | nohow 621 | impairment 622 | sandman 623 | excrement 624 | revealed 625 | frizzle 626 | roil 627 | unfinished 628 | stove 629 | eerie 630 | bong 631 | iris 632 | truer 633 | prostitute 634 | soybean 635 | wiz 636 | stopper 637 | directional 638 | warship 639 | zipper 640 | stricter 641 | slippage 642 | graveyard 643 | canceled 644 | disincline 645 | synopsis 646 | gizmo 647 | picky 648 | mortimer 649 | wylie 650 | tougher 651 | ins 652 | heirloom 653 | sphinx 654 | metalworking 655 | loner 656 | reappear 657 | exuberantly 658 | constipate 659 | salvage 660 | defecates 661 | leakage 662 | linguine 663 | disengage 664 | bombard 665 | demonic 666 | governance 667 | estimated 668 | smug 669 | agricultural 670 | outplay 671 | jawbone 672 | avoidance 673 | ramble 674 | citizens 675 | toxicity 676 | lads 677 | grammar 678 | laundry 679 | dumpy 680 | profitless 681 | calcified 682 | killers 683 | snipe 684 | debs 685 | sizeable 686 | shag 687 | straggler 688 | compassion 689 | fantasia 690 | luxurious 691 | bai 692 | penicillin 693 | wipeout 694 | projectile 695 | screeching 696 | dissect 697 | incomparable 698 | scotland 699 | walker 700 | stepchild 701 | reynard 702 | behead 703 | fairytale 704 | oscillatory 705 | wither 706 | lesbian 707 | enslave 708 | fawn 709 | growl 710 | cognitive 711 | pampers 712 | drunken 713 | hns 714 | transplant 715 | imaginable 716 | exceedingly 717 | ghat 718 | multiplicity 719 | swinger 720 | psyche 721 | palestine 722 | fatty 723 | furnace 724 | ceos 725 | expandable 726 | pinched 727 | mackintosh 728 | assassinates 729 | enthusiastic 730 | outclass 731 | friendship 732 | thrice 733 | constitutional 734 | sneaker 735 | bogie 736 | dab 737 | tossing 738 | sturdy 739 | technocracy 740 | jurisdiction 741 | unloading 742 | compose 743 | physic 744 | stewart 745 | reproductive 746 | hangman 747 | titty 748 | magnanimous 749 | yakuza 750 | suv 751 | brisk 752 | retaliate 753 | shellacking 754 | deodorant 755 | cheesy 756 | herr 757 | recollection 758 | severance 759 | wok 760 | nobel 761 | condemnation 762 | licensed 763 | downgraded 764 | holdes 765 | yess 766 | flagpole 767 | subsequently 768 | cluster 769 | terrified 770 | menstruation 771 | straggling 772 | idolize 773 | castle 774 | illegitimately 775 | fount 776 | bader 777 | caveat 778 | inversion 779 | bipartisan 780 | staging 781 | pensioner 782 | miniaturized 783 | undermining 784 | slime 785 | nightmarish 786 | bunk 787 | shepherd 788 | obscenely 789 | pimply 790 | fugly 791 | complacency 792 | grease 793 | seychelles 794 | broadening 795 | riff 796 | commenced 797 | annoyingly 798 | pharaoh 799 | legality 800 | uniform 801 | adept 802 | plop 803 | directionless 804 | interrogation 805 | parochial 806 | hideous 807 | woof 808 | mandate 809 | recoverable 810 | stutter 811 | corrective 812 | taint 813 | kooky 814 | outsourcing 815 | deception 816 | vibration 817 | miscalculation 818 | stripe 819 | halve 820 | uses 821 | organically 822 | boogie 823 | impacted 824 | folklore 825 | unlawful 826 | iaa 827 | lumpy 828 | comer 829 | spikes 830 | stocked 831 | damon 832 | eroding 833 | pate 834 | jacking 835 | rooster 836 | intoxicate 837 | inks 838 | royally 839 | filtered 840 | snapdragon 841 | insulated 842 | cheater 843 | pitchfork 844 | retraces 845 | inexcusable 846 | voted 847 | clad 848 | brat 849 | beeline 850 | blunt 851 | pubic 852 | gratification 853 | napoleon 854 | deleterious 855 | rus 856 | galbraith 857 | yiddish 858 | confiscate 859 | guild 860 | sundae 861 | boosted 862 | unnatural 863 | dorm 864 | everest 865 | recording 866 | impulsively 867 | peering 868 | pervert 869 | mem 870 | stella 871 | entangle 872 | resoundingly 873 | imprisonment 874 | infiltrator 875 | detach 876 | shave 877 | eastman 878 | imprisoned 879 | clearheaded 880 | intervenes 881 | repression 882 | regress 883 | disenchant 884 | misconception 885 | lavish 886 | noted 887 | deface 888 | billionth 889 | argues 890 | periodic 891 | inane 892 | softens 893 | trudge 894 | impoverished 895 | pistol 896 | plunger 897 | unfriendly 898 | gouging 899 | delhi 900 | fleeced 901 | abused 902 | wilt 903 | rusted 904 | stacking 905 | indigo 906 | feud 907 | whacky 908 | investigative 909 | hansard 910 | shoddy 911 | kook 912 | parlor 913 | unsinkable 914 | reimburse 915 | mitigate 916 | dragged 917 | feverish 918 | contention 919 | rainbows 920 | mentioned 921 | laundering 922 | turkeys 923 | hie 924 | fes 925 | squarely 926 | rarity 927 | willard 928 | launder 929 | shortlist 930 | eject 931 | homicide 932 | exiting 933 | avoids 934 | dover 935 | grumble 936 | bomber 937 | imperil 938 | diced 939 | systemic 940 | baton 941 | servant 942 | doves 943 | penniless 944 | disciplined 945 | acclaimed 946 | antithesis 947 | bating 948 | yawner 949 | kissing 950 | grander 951 | aggregation 952 | luster 953 | recruiting 954 | hippopotamus 955 | chemistry 956 | decelerating 957 | comatose 958 | elses 959 | exceeds 960 | slower 961 | delisted 962 | uncomfortably 963 | determining 964 | awol 965 | obsessed 966 | robber 967 | consultancy 968 | shooter 969 | sucked 970 | zimbabwean 971 | supervision 972 | illegitimate 973 | adversary 974 | gracious 975 | reciprocate 976 | malfunctioning 977 | mysteriously 978 | hercules 979 | skate 980 | disarray 981 | alabama 982 | eta 983 | perpetuity 984 | pakistan 985 | pops 986 | cybercrime 987 | lagger 988 | socialize 989 | financier 990 | puzzling 991 | dats 992 | simplification 993 | sinful 994 | liquidating 995 | warranty 996 | advisable 997 | caw 998 | demogorgon 999 | slashed 1000 | orgy 1001 | unicorns 1002 | galaxys 1003 | jive 1004 | begs 1005 | windy 1006 | disintegrate 1007 | recommends 1008 | deflect 1009 | politico 1010 | impair 1011 | downtick 1012 | oscillator 1013 | hopeless 1014 | classify 1015 | blight 1016 | alzheimers 1017 | megalomaniac 1018 | emergence 1019 | decency 1020 | lamb 1021 | inspired 1022 | addes 1023 | rationalization 1024 | engineered 1025 | bleeder 1026 | fleas 1027 | meq 1028 | jewelry 1029 | dismantling 1030 | tortilla 1031 | tippy 1032 | headband 1033 | dissociate 1034 | ike 1035 | volumes 1036 | recount 1037 | gutless 1038 | dislocation 1039 | distasteful 1040 | bide 1041 | referendum 1042 | loch 1043 | lifeboat 1044 | leery 1045 | moist 1046 | faker 1047 | experiential 1048 | surcharge 1049 | zealous 1050 | distrustful 1051 | streamer 1052 | steamroller 1053 | url 1054 | apprehensive 1055 | caput 1056 | degradation 1057 | suffering 1058 | unchallenged 1059 | livelihood 1060 | homeowner 1061 | principal 1062 | projecting 1063 | fastball 1064 | clout 1065 | sphere 1066 | ugli 1067 | aft 1068 | diminishing 1069 | tecs 1070 | dyslexia 1071 | bargaining 1072 | ethnicity 1073 | downbeat 1074 | cower 1075 | leaks 1076 | overpays 1077 | outback 1078 | icarus 1079 | feces 1080 | obligatory 1081 | glare 1082 | subpoena 1083 | ranch 1084 | nu 1085 | rightfully 1086 | sympathizing 1087 | aspen 1088 | marginalize 1089 | sherry 1090 | observer 1091 | mg 1092 | collecting 1093 | destabilize 1094 | sexualised 1095 | grenade 1096 | cuss 1097 | kibosh 1098 | disability 1099 | scummy 1100 | nh 1101 | responsive 1102 | rehear 1103 | irreparable 1104 | devastation 1105 | stocking 1106 | inflicted 1107 | mayer 1108 | arch 1109 | concussion 1110 | unprotected 1111 | ambushed 1112 | claimed 1113 | lev 1114 | wale 1115 | radioactive 1116 | betrayal 1117 | demolition 1118 | antichrist 1119 | gobbler 1120 | susceptibility 1121 | crossfire 1122 | motionless 1123 | intuit 1124 | worrisome 1125 | easing 1126 | lacking 1127 | rapist 1128 | livermore 1129 | aloof 1130 | inhuman 1131 | mummy 1132 | oncoming 1133 | hermit 1134 | waveform 1135 | cdc 1136 | dys 1137 | flunk 1138 | treasuries 1139 | deliciously 1140 | geronimo 1141 | newness 1142 | whomp 1143 | gracie 1144 | fiend 1145 | rugged 1146 | unravelling 1147 | abruptly 1148 | headless 1149 | shaggy 1150 | dreamland 1151 | viewing 1152 | pseudo 1153 | spider 1154 | indebted 1155 | hubby 1156 | auditor 1157 | channeling 1158 | roadkill 1159 | cms 1160 | profiting 1161 | vacillation 1162 | groupie 1163 | ignoramus 1164 | defence 1165 | steward 1166 | empathy 1167 | genital 1168 | applesauce 1169 | overcrowded 1170 | hinge 1171 | kindles 1172 | weathered 1173 | cabbage 1174 | sender 1175 | jared 1176 | dissipating 1177 | collaborator 1178 | tarnish 1179 | ruble 1180 | burberry 1181 | corpulent 1182 | destruct 1183 | flutter 1184 | uptime 1185 | baiting 1186 | chagrin 1187 | sewer 1188 | synapse 1189 | savings 1190 | genuinely 1191 | supremacist 1192 | unlisted 1193 | transgender 1194 | retro 1195 | elvis 1196 | entrust 1197 | precipitously 1198 | ambivalence 1199 | deco 1200 | sss 1201 | caucus 1202 | meteorite 1203 | frankenstein 1204 | patter 1205 | presumption 1206 | gash 1207 | chic 1208 | quicksand 1209 | vesting 1210 | asparagus 1211 | swish 1212 | grandmas 1213 | zaman 1214 | meeker 1215 | vegan 1216 | praying 1217 | blimp 1218 | selfish 1219 | refurbished 1220 | programmed 1221 | weirder 1222 | lampoon 1223 | exclamation 1224 | snicker 1225 | icy 1226 | tourniquet 1227 | inmate 1228 | bloater 1229 | diet 1230 | misread 1231 | measurably 1232 | arrows 1233 | emptor 1234 | wreak 1235 | aurora 1236 | cheating 1237 | cancerous 1238 | fads 1239 | bursting 1240 | overconfidence 1241 | capsize 1242 | hearse 1243 | blot 1244 | tarp 1245 | proliferation 1246 | vetting 1247 | embattled 1248 | nebraska 1249 | mane 1250 | halibut 1251 | deflecting 1252 | eviscerate 1253 | classmate 1254 | freezing 1255 | incorrectly 1256 | leagues 1257 | objection 1258 | raking 1259 | slavery 1260 | sexually 1261 | probes 1262 | blaster 1263 | lossess 1264 | contradictory 1265 | excise 1266 | guppy 1267 | loathing 1268 | instrumentation 1269 | spaghetti 1270 | fillet 1271 | spontaneous 1272 | tumbling 1273 | ukrainian 1274 | heft 1275 | divergences 1276 | yuppie 1277 | illustrative 1278 | chainsaw 1279 | barricade 1280 | informs 1281 | rollback 1282 | worsen 1283 | gab 1284 | tubes 1285 | rye 1286 | rung 1287 | bleach 1288 | whipping 1289 | untaxed 1290 | comps 1291 | unauthorized 1292 | plonk 1293 | moore 1294 | feller 1295 | trashed 1296 | hb 1297 | goner 1298 | hepatitis 1299 | spectacularly 1300 | riches 1301 | slink 1302 | pare 1303 | announcing 1304 | disturbed 1305 | cheesecake 1306 | naught 1307 | falsely 1308 | alfred 1309 | weder 1310 | stylus 1311 | byproduct 1312 | gasbag 1313 | peeping 1314 | yarmulke 1315 | cakewalk 1316 | rounding 1317 | beneath 1318 | waring 1319 | hedonism 1320 | intentional 1321 | disco 1322 | sickly 1323 | circumnavigate 1324 | saws 1325 | molest 1326 | knell 1327 | belize 1328 | inaccuracy 1329 | pegged 1330 | oks 1331 | condo 1332 | poisonous 1333 | opts 1334 | zoloft 1335 | chlamydia 1336 | chopped 1337 | malaysian 1338 | handsome 1339 | freebies 1340 | frequent 1341 | untrustworthy 1342 | escalates 1343 | consciously 1344 | waster 1345 | trumpet 1346 | disagreed 1347 | uninformed 1348 | discloses 1349 | benedict 1350 | unknowing 1351 | counterfeiting 1352 | trustworthy 1353 | veil 1354 | transgendered 1355 | accepting 1356 | criticism 1357 | guaranty 1358 | nestle 1359 | debone 1360 | faintly 1361 | cascade 1362 | refined 1363 | alleges 1364 | obliterating 1365 | abject 1366 | tacky 1367 | duly 1368 | canal 1369 | keystone 1370 | announcements 1371 | magnetism 1372 | grueling 1373 | gluttony 1374 | absorbed 1375 | uninvited 1376 | crapper 1377 | overwork 1378 | slimy 1379 | poisoning 1380 | destroys 1381 | colluding 1382 | distributing 1383 | knockoff 1384 | cabal 1385 | fink 1386 | finesse 1387 | naughty 1388 | bride 1389 | pertain 1390 | jamming 1391 | wolfman 1392 | rfd 1393 | lies 1394 | disgraced 1395 | marshal 1396 | seasoned 1397 | dribble 1398 | facing 1399 | telltale 1400 | commonsense 1401 | tassel 1402 | obnoxiously 1403 | dignify 1404 | corruptly 1405 | eater 1406 | pom 1407 | rancid 1408 | shaken 1409 | jumper 1410 | entitle 1411 | rebecca 1412 | emotionless 1413 | torrent 1414 | mediate 1415 | uglier 1416 | starry 1417 | sweetie 1418 | racism 1419 | starring 1420 | pimping 1421 | covenant 1422 | albuquerque 1423 | gin 1424 | poppy 1425 | denounce 1426 | refinance 1427 | masturbation 1428 | porta 1429 | foolhardy 1430 | drank 1431 | onion 1432 | contour 1433 | baggage 1434 | formally 1435 | mcmaster 1436 | hilt 1437 | loosing 1438 | earths 1439 | acne 1440 | yearbook 1441 | hypothesis 1442 | projectiles 1443 | rehearse 1444 | frauds 1445 | braid 1446 | devaluation 1447 | implosion 1448 | atomic 1449 | slaver 1450 | stopgap 1451 | delude 1452 | catholic 1453 | puppetry 1454 | sew 1455 | slaughterhouse 1456 | ballooning 1457 | botox 1458 | terrifying 1459 | impaction 1460 | boycotting 1461 | deflate 1462 | rotating 1463 | exacerbate 1464 | unorthodox 1465 | mayday 1466 | spiritual 1467 | assumes 1468 | cypher 1469 | wedgie 1470 | seeling 1471 | uppity 1472 | entitlement 1473 | raf 1474 | shoplifter 1475 | chigger 1476 | chamberlain 1477 | boyfriend 1478 | catchphrase 1479 | trafficking 1480 | locality 1481 | occult 1482 | afghanistan 1483 | knowingly 1484 | ciao 1485 | grapple 1486 | precede 1487 | caching 1488 | faust 1489 | stockman 1490 | sharpening 1491 | backer 1492 | resonate 1493 | scandalous 1494 | trounced 1495 | incense 1496 | charged 1497 | porker 1498 | retaliates 1499 | bendable 1500 | dragnet 1501 | lowes 1502 | punctuation 1503 | batting 1504 | abb 1505 | irresponsibility 1506 | counterfeiter 1507 | buckaroos 1508 | gnashing 1509 | correctional 1510 | gauntlet 1511 | flaky 1512 | advertizement 1513 | legged 1514 | quiting 1515 | illustration 1516 | softball 1517 | fickleness 1518 | krill 1519 | foreclose 1520 | progressively 1521 | falsify 1522 | recharge 1523 | refresher 1524 | kneecap 1525 | mismanagement 1526 | confessional 1527 | awakening 1528 | glorified 1529 | referee 1530 | pinned 1531 | bast 1532 | spacey 1533 | undo 1534 | bloodletting 1535 | thermoelectric 1536 | softly 1537 | portends 1538 | nimrod 1539 | immortal 1540 | syndrome 1541 | hindenburg 1542 | obliterated 1543 | adieu 1544 | occasionally 1545 | silo 1546 | obfuscate 1547 | numbes 1548 | slammed 1549 | furthest 1550 | monopolization 1551 | advised 1552 | blub 1553 | trombone 1554 | canoe 1555 | simon 1556 | chore 1557 | demanding 1558 | dishwasher 1559 | ouija 1560 | invades 1561 | cattle 1562 | thereby 1563 | premie 1564 | letting 1565 | prognosticate 1566 | illegally 1567 | thunk 1568 | deadline 1569 | heartless 1570 | cursed 1571 | invoke 1572 | hmo 1573 | soviet 1574 | sacked 1575 | outsider 1576 | ascends 1577 | rapture 1578 | posing 1579 | speck 1580 | beethoven 1581 | duckling 1582 | paraphrasing 1583 | purge 1584 | uncovered 1585 | plummeting 1586 | hipster 1587 | breakage 1588 | posit 1589 | favourable 1590 | ante 1591 | adventurous 1592 | reparation 1593 | devotee 1594 | slipper 1595 | stairs 1596 | mafioso 1597 | pumpernickel 1598 | moor 1599 | tells 1600 | dinky 1601 | prying 1602 | remodels 1603 | barbaric 1604 | employes 1605 | materializes 1606 | bluegrass 1607 | wasted 1608 | crying 1609 | calender 1610 | clinging 1611 | medallion 1612 | perversive 1613 | underperformer 1614 | popped 1615 | smithereens 1616 | knave 1617 | tent 1618 | inability 1619 | plunder 1620 | guinea 1621 | sentimental 1622 | pedigreed 1623 | dirts 1624 | indifferent 1625 | looker 1626 | widening 1627 | restrictive 1628 | glaring 1629 | thereabout 1630 | cling 1631 | panama 1632 | splice 1633 | eavesdrop 1634 | buries 1635 | valentines 1636 | antiquated 1637 | gasping 1638 | vein 1639 | orbison 1640 | westminster 1641 | lengthy 1642 | brith 1643 | amigos 1644 | gunman 1645 | thinks 1646 | intercourse 1647 | stricken 1648 | evasion 1649 | herder 1650 | oran 1651 | unsuspecting 1652 | peaked 1653 | meg 1654 | fleeing 1655 | exposed 1656 | spreading 1657 | traded 1658 | fanny 1659 | solicit 1660 | demonization 1661 | knockout 1662 | boarding 1663 | electrical 1664 | aladdin 1665 | celebrities 1666 | skinned 1667 | manufactured 1668 | collateral 1669 | saturated 1670 | ret 1671 | savant 1672 | instructor 1673 | alongside 1674 | petersburg 1675 | deliverance 1676 | succinctly 1677 | cocooning 1678 | capri 1679 | lu 1680 | lulling 1681 | eggshell 1682 | strikeout 1683 | swimsuit 1684 | muller 1685 | days 1686 | corny 1687 | featureless 1688 | tacked 1689 | octopus 1690 | recyclable 1691 | dishonesty 1692 | silk 1693 | introvert 1694 | fuse 1695 | edgar 1696 | weirdo 1697 | stabbing 1698 | submissive 1699 | engaged 1700 | islam 1701 | wooden 1702 | trained 1703 | risen 1704 | unaffiliated 1705 | keg 1706 | parliamentary 1707 | fanatic 1708 | patterns 1709 | madhouse 1710 | cynic 1711 | omaha 1712 | attrition 1713 | wtc 1714 | pimps 1715 | mccarthy 1716 | monkeys 1717 | arizona 1718 | purchased 1719 | condense 1720 | simplicity 1721 | jnd 1722 | forcibly 1723 | midwest 1724 | begets 1725 | masquerade 1726 | forgone 1727 | wacky 1728 | wobbler 1729 | causeway 1730 | immortality 1731 | warrens 1732 | failures 1733 | paintball 1734 | havoc 1735 | artistic 1736 | bathing 1737 | cyst 1738 | wobbly 1739 | mudder 1740 | unlicensed 1741 | almond 1742 | demos 1743 | psychopath 1744 | thole 1745 | upticks 1746 | berkeley 1747 | testicle 1748 | heated 1749 | technologist 1750 | retaliatory 1751 | mann 1752 | julian 1753 | bead 1754 | ssri 1755 | tanks 1756 | abet 1757 | bragging 1758 | lade 1759 | node 1760 | midsummer 1761 | deceit 1762 | sweden 1763 | hackings 1764 | heave 1765 | dissuade 1766 | evaporation 1767 | backdoor 1768 | brazilian 1769 | toasted 1770 | reverts 1771 | toaster 1772 | hawking 1773 | woofer 1774 | antiseptic 1775 | clearer 1776 | epitome 1777 | circular 1778 | deflates 1779 | fascism 1780 | debar 1781 | bailiff 1782 | newsflash 1783 | falseness 1784 | myanmar 1785 | sticker 1786 | undercutting 1787 | brittany 1788 | unilaterally 1789 | softie 1790 | poorhouse 1791 | fantabulous 1792 | brightest 1793 | franklin 1794 | seta 1795 | walking 1796 | computerise 1797 | casket 1798 | gorbachev 1799 | piggies 1800 | excessively 1801 | deforestation 1802 | detest 1803 | goldfish 1804 | croak 1805 | fleecing 1806 | abets 1807 | morons 1808 | inappropriate 1809 | alienate 1810 | sashay 1811 | panhandle 1812 | vertigo 1813 | treats 1814 | ashram 1815 | faded 1816 | crookedness 1817 | postage 1818 | indexed 1819 | inflecting 1820 | pisser 1821 | lena 1822 | probiotic 1823 | devaluate 1824 | heartache 1825 | luau 1826 | wifes 1827 | analytical 1828 | pummeled 1829 | lettuce 1830 | stationary 1831 | expensively 1832 | freeway 1833 | patriots 1834 | angus 1835 | womans 1836 | infamy 1837 | shrunk 1838 | fingerprinting 1839 | zig 1840 | loaf 1841 | impute 1842 | sympathize 1843 | lifelong 1844 | vilas 1845 | severed 1846 | guffaw 1847 | enthusiastically 1848 | visas 1849 | faber 1850 | discriminatory 1851 | imagery 1852 | arms 1853 | nutmeg 1854 | baruch 1855 | incessant 1856 | strawman 1857 | ticking 1858 | sparkly 1859 | carjack 1860 | fashionable 1861 | buffer 1862 | evans 1863 | desirable 1864 | pastime 1865 | pershing 1866 | premiums 1867 | suspenseful 1868 | infraction 1869 | escalation 1870 | acquiesce 1871 | subs 1872 | valuate 1873 | derailment 1874 | sexist 1875 | uploaded 1876 | kgb 1877 | legitimately 1878 | reconcile 1879 | sludge 1880 | unnaturally 1881 | canyon 1882 | gall 1883 | pyromaniac 1884 | boll 1885 | scruffy 1886 | stray 1887 | shaping 1888 | zimbabwe 1889 | farther 1890 | liabilities 1891 | curly 1892 | overturn 1893 | disfigure 1894 | perceptive 1895 | unloads 1896 | allude 1897 | weighed 1898 | divesting 1899 | gibson 1900 | stalin 1901 | diseased 1902 | smelt 1903 | aroma 1904 | reviewer 1905 | occurrence 1906 | devalues 1907 | forgiveness 1908 | chopping 1909 | bisects 1910 | boiler 1911 | sweatshop 1912 | presume 1913 | renter 1914 | trinket 1915 | devastated 1916 | criminalize 1917 | purity 1918 | startling 1919 | cumulatively 1920 | relent 1921 | crashed 1922 | largo 1923 | glut 1924 | suitably 1925 | nib 1926 | aggravation 1927 | tonic 1928 | papaya 1929 | rubble 1930 | apologist 1931 | dill 1932 | flimsy 1933 | stool 1934 | graciously 1935 | entrapment 1936 | chainsaws 1937 | understate 1938 | libertarians 1939 | fog 1940 | measurable 1941 | folding 1942 | haiti 1943 | transcend 1944 | adjusting 1945 | brining 1946 | musician 1947 | miniscule 1948 | twinkie 1949 | sacramento 1950 | partisan 1951 | ducked 1952 | proliferate 1953 | luxemburg 1954 | cooke 1955 | mailman 1956 | gramps 1957 | overreach 1958 | crumbles 1959 | malignant 1960 | falsehood 1961 | mohammed 1962 | connote 1963 | curiosity 1964 | sensitively 1965 | overcook 1966 | grilled 1967 | darker 1968 | hillarys 1969 | ghetto 1970 | calk 1971 | rebuttal 1972 | yale 1973 | stephens 1974 | egypt 1975 | poet 1976 | radioisotope 1977 | felicia 1978 | paedophile 1979 | begging 1980 | misdirection 1981 | illicit 1982 | totter 1983 | whoopee 1984 | hoya 1985 | unravels 1986 | inexperienced 1987 | babu 1988 | constriction 1989 | mot 1990 | offhand 1991 | testa 1992 | reactance 1993 | leasing 1994 | cole 1995 | transferring 1996 | disinterested 1997 | embattle 1998 | erased 1999 | boutique 2000 | attachment 2001 | exploitive 2002 | vicinity 2003 | stomped 2004 | catcher 2005 | sixer 2006 | banning 2007 | deplorable 2008 | finder 2009 | smokescreen 2010 | gape 2011 | refs 2012 | fume 2013 | flipped 2014 | dropsy 2015 | ascendant 2016 | promotes 2017 | inherent 2018 | esquire 2019 | intake 2020 | shamefully 2021 | wham 2022 | misguide 2023 | unnerving 2024 | winded 2025 | -------------------------------------------------------------------------------- /RNN.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import pandas as pd\n", 10 | "import nltk\n", 11 | "import string\n", 12 | "import re\n", 13 | "import tensorflow as tf\n", 14 | "import numpy as np\n", 15 | "\n", 16 | "data =pd.read_csv('data.csv')\n" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": 2, 22 | "metadata": {}, 23 | "outputs": [], 24 | "source": [ 25 | "data =data.sample(frac=1).reset_index(drop=True)" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": 3, 31 | "metadata": {}, 32 | "outputs": [ 33 | { 34 | "data": { 35 | "text/html": [ 36 | "
\n", 37 | "\n", 50 | "\n", 51 | " \n", 52 | " \n", 53 | " \n", 54 | " \n", 55 | " \n", 56 | " \n", 57 | " \n", 58 | " \n", 59 | " \n", 60 | " \n", 61 | " \n", 62 | " \n", 63 | " \n", 64 | " \n", 65 | " \n", 66 | " \n", 67 | " \n", 68 | " \n", 69 | " \n", 70 | " \n", 71 | " \n", 72 | " \n", 73 | " \n", 74 | " \n", 75 | " \n", 76 | " \n", 77 | " \n", 78 | " \n", 79 | " \n", 80 | " \n", 81 | " \n", 82 | " \n", 83 | " \n", 84 | " \n", 85 | "
sentimentmessage
0Bullish$FB FB and Cos have shown us that we will do A...
1Bullish$FB 200 after the Call
2Bullish$FB would like to get into this if it goes to ...
3Bearish$FB surprised it’s holding 185... thinking 182
4Bullish$FB buy here
\n", 86 | "
" 87 | ], 88 | "text/plain": [ 89 | " sentiment message\n", 90 | "0 Bullish $FB FB and Cos have shown us that we will do A...\n", 91 | "1 Bullish $FB 200 after the Call\n", 92 | "2 Bullish $FB would like to get into this if it goes to ...\n", 93 | "3 Bearish $FB surprised it’s holding 185... thinking 182\n", 94 | "4 Bullish $FB buy here" 95 | ] 96 | }, 97 | "execution_count": 3, 98 | "metadata": {}, 99 | "output_type": "execute_result" 100 | } 101 | ], 102 | "source": [ 103 | "data.head()" 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": 4, 109 | "metadata": {}, 110 | "outputs": [ 111 | { 112 | "data": { 113 | "text/html": [ 114 | "
\n", 115 | "\n", 128 | "\n", 129 | " \n", 130 | " \n", 131 | " \n", 132 | " \n", 133 | " \n", 134 | " \n", 135 | " \n", 136 | " \n", 137 | " \n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | "
sentimentmessagemessage_length
01[fb, cos, shown, us, anything, recognition, fa...12
11[200, after, call]3
21[would, like, get, into, goes]5
30[surprised, ’, holding, 185, thinking, 182]6
41[buy]1
\n", 170 | "
" 171 | ], 172 | "text/plain": [ 173 | " sentiment message \\\n", 174 | "0 1 [fb, cos, shown, us, anything, recognition, fa... \n", 175 | "1 1 [200, after, call] \n", 176 | "2 1 [would, like, get, into, goes] \n", 177 | "3 0 [surprised, ’, holding, 185, thinking, 182] \n", 178 | "4 1 [buy] \n", 179 | "\n", 180 | " message_length \n", 181 | "0 12 \n", 182 | "1 3 \n", 183 | "2 5 \n", 184 | "3 6 \n", 185 | "4 1 " 186 | ] 187 | }, 188 | "execution_count": 4, 189 | "metadata": {}, 190 | "output_type": "execute_result" 191 | } 192 | ], 193 | "source": [ 194 | "custom_stop_words = ['i','me', 'my','myself','we','our','ours','ourselves','you',\"you're\",\"you've\",\"you'll\",\"you'd\",'your','yours','yourself','yourselves','he','him','his','himself','she',\"she's\",'her','hers','herself','it',\"it's\",'its','itself','they','them','their','theirs','themselves','what','which','who','whom','this','that',\"that'll\",'these','those','am','is','are','was','were','be','been','being','have','has','had','having','do','does','did','doing','a','an','the','and','but','if','or','because','as','until','while','of','at','by','for','then','once','here','there','when','where','why','how','all','any','both','no','nor','not','only','own','same','so','s','t','can','will','just','don',\"don't\",'should',\"should've\",'now','d','ll','m','o','re','ve','y','ain','aren',\"aren't\",'couldn',\"couldn't\",'didn',\"didn't\",'doesn',\"doesn't\",'hadn',\"hadn't\",'hasn',\"hasn't\",'haven',\"haven't\",'isn',\"isn't\",'ma','mightn',\"mightn't\",'mustn',\"mustn't\",'needn',\"needn't\",'shan',\"shan't\",'shouldn',\"shouldn't\",'wasn',\"wasn't\",'weren',\"weren't\",'won',\"won't\",'wouldn',\"wouldn't\",\"to\"]\n", 195 | "stop_words = set(custom_stop_words)\n", 196 | "\n", 197 | "def tokenizer(s):\n", 198 | " s = re.sub(r'http\\S+', '', s)\n", 199 | " s = re.sub(r'\\$(\\w+)','',s)\n", 200 | " translate_table = dict((ord(char), None) for char in string.punctuation) \n", 201 | " s = s.translate(translate_table)\n", 202 | " tokens = nltk.word_tokenize(s)\n", 203 | " filtered_tokens =[]\n", 204 | " for word in tokens:\n", 205 | " if word.lower() not in stop_words:\n", 206 | " filtered_tokens.append(word.lower())\n", 207 | " return filtered_tokens\n", 208 | "\n", 209 | "data['message'] = data['message'].apply(tokenizer)\n", 210 | "data['message_length'] = data['message'].apply(len)\n", 211 | "data = data[(data['message_length']>0)]\n", 212 | "\n", 213 | "def func(x):\n", 214 | " if x=='Bullish':\n", 215 | " return 1\n", 216 | " return 0\n", 217 | "\n", 218 | "data.sentiment = data.sentiment.apply(func)\n", 219 | "data.head()" 220 | ] 221 | }, 222 | { 223 | "cell_type": "code", 224 | "execution_count": 5, 225 | "metadata": {}, 226 | "outputs": [ 227 | { 228 | "data": { 229 | "text/plain": [ 230 | "3 14796\n", 231 | "4 14113\n", 232 | "5 13324\n", 233 | "2 13072\n", 234 | "6 12273\n", 235 | "7 11271\n", 236 | "8 10475\n", 237 | "9 9834\n", 238 | "10 8972\n", 239 | "11 8440\n", 240 | "12 8010\n", 241 | "13 7677\n", 242 | "14 7325\n", 243 | "1 7248\n", 244 | "15 6258\n", 245 | "16 4930\n", 246 | "17 3629\n", 247 | "18 2443\n", 248 | "19 1393\n", 249 | "20 690\n", 250 | "21 322\n", 251 | "22 136\n", 252 | "23 52\n", 253 | "24 24\n", 254 | "26 9\n", 255 | "25 8\n", 256 | "27 2\n", 257 | "29 1\n", 258 | "28 1\n", 259 | "44 1\n", 260 | "Name: message_length, dtype: int64" 261 | ] 262 | }, 263 | "execution_count": 5, 264 | "metadata": {}, 265 | "output_type": "execute_result" 266 | } 267 | ], 268 | "source": [ 269 | "data.message_length.value_counts()" 270 | ] 271 | }, 272 | { 273 | "cell_type": "code", 274 | "execution_count": 6, 275 | "metadata": {}, 276 | "outputs": [], 277 | "source": [ 278 | "words= []\n", 279 | "for message in data.message:\n", 280 | " for word in message:\n", 281 | " words.append(word)\n", 282 | "\n", 283 | "from collections import Counter\n", 284 | "counts = Counter(words)\n", 285 | "vocab = sorted(counts, key=counts.get, reverse=True)\n", 286 | "vocab_to_int = {word: ii for ii, word in enumerate(vocab, 1)}" 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": 7, 292 | "metadata": {}, 293 | "outputs": [ 294 | { 295 | "data": { 296 | "text/plain": [ 297 | "70116" 298 | ] 299 | }, 300 | "execution_count": 7, 301 | "metadata": {}, 302 | "output_type": "execute_result" 303 | } 304 | ], 305 | "source": [ 306 | "len(vocab_to_int)" 307 | ] 308 | }, 309 | { 310 | "cell_type": "code", 311 | "execution_count": 9, 312 | "metadata": {}, 313 | "outputs": [ 314 | { 315 | "data": { 316 | "text/html": [ 317 | "
\n", 318 | "\n", 331 | "\n", 332 | " \n", 333 | " \n", 334 | " \n", 335 | " \n", 336 | " \n", 337 | " \n", 338 | " \n", 339 | " \n", 340 | " \n", 341 | " \n", 342 | " \n", 343 | " \n", 344 | " \n", 345 | " \n", 346 | " \n", 347 | " \n", 348 | " \n", 349 | " \n", 350 | " \n", 351 | " \n", 352 | " \n", 353 | " \n", 354 | " \n", 355 | " \n", 356 | " \n", 357 | " \n", 358 | " \n", 359 | " \n", 360 | " \n", 361 | " \n", 362 | " \n", 363 | " \n", 364 | " \n", 365 | " \n", 366 | " \n", 367 | " \n", 368 | " \n", 369 | " \n", 370 | " \n", 371 | " \n", 372 | " \n", 373 | " \n", 374 | " \n", 375 | " \n", 376 | " \n", 377 | " \n", 378 | "
sentimentmessagemessage_lengthencoded_message
01[fb, cos, shown, us, anything, recognition, fa...12[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 23, 24...
11[200, after, call]3[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
21[would, like, get, into, goes]5[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
30[surprised, ’, holding, 185, thinking, 182]6[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
41[buy]1[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
\n", 379 | "
" 380 | ], 381 | "text/plain": [ 382 | " sentiment message \\\n", 383 | "0 1 [fb, cos, shown, us, anything, recognition, fa... \n", 384 | "1 1 [200, after, call] \n", 385 | "2 1 [would, like, get, into, goes] \n", 386 | "3 0 [surprised, ’, holding, 185, thinking, 182] \n", 387 | "4 1 [buy] \n", 388 | "\n", 389 | " message_length encoded_message \n", 390 | "0 12 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 23, 24... \n", 391 | "1 3 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ... \n", 392 | "2 5 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ... \n", 393 | "3 6 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ... \n", 394 | "4 1 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ... " 395 | ] 396 | }, 397 | "execution_count": 9, 398 | "metadata": {}, 399 | "output_type": "execute_result" 400 | } 401 | ], 402 | "source": [ 403 | "max_len = 25\n", 404 | "\n", 405 | "\n", 406 | "def fun(x):\n", 407 | " vec = []\n", 408 | " zeroes = max_len - len(x)\n", 409 | " if zeroes>0:\n", 410 | " for i in range(zeroes):\n", 411 | " vec.append(0)\n", 412 | " \n", 413 | " for i in range(max_len-len(vec)):\n", 414 | " word = x[i]\n", 415 | " vec.append(vocab_to_int[word])\n", 416 | " return np.array(vec)\n", 417 | "\n", 418 | "data['encoded_message'] = data['message'].apply(fun)\n", 419 | "data.head()" 420 | ] 421 | }, 422 | { 423 | "cell_type": "code", 424 | "execution_count": 14, 425 | "metadata": {}, 426 | "outputs": [ 427 | { 428 | "name": "stdout", 429 | "output_type": "stream", 430 | "text": [ 431 | "[[ 0 0 0 ..., 11 598 25780]\n", 432 | " [ 0 0 0 ..., 179 31 109]\n", 433 | " [ 0 0 0 ..., 14 52 175]\n", 434 | " ..., \n", 435 | " [ 0 0 0 ..., 50 66 420]\n", 436 | " [ 0 0 0 ..., 192 357 1342]\n", 437 | " [ 0 0 0 ..., 25 17 374]]\n" 438 | ] 439 | }, 440 | { 441 | "data": { 442 | "text/plain": [ 443 | "(150000, 25)" 444 | ] 445 | }, 446 | "execution_count": 14, 447 | "metadata": {}, 448 | "output_type": "execute_result" 449 | } 450 | ], 451 | "source": [ 452 | "features= []\n", 453 | "for vec in data.encoded_message:\n", 454 | " features.append(vec.tolist())\n", 455 | "features =np.array(features)\n", 456 | "features = features[:150000]\n", 457 | "print(features)\n", 458 | "features.shape" 459 | ] 460 | }, 461 | { 462 | "cell_type": "code", 463 | "execution_count": 15, 464 | "metadata": {}, 465 | "outputs": [ 466 | { 467 | "name": "stdout", 468 | "output_type": "stream", 469 | "text": [ 470 | "\t\t\tFeature Shapes:\n", 471 | "Train set: \t\t(105000, 25) \n", 472 | "Validation set: \t(9000, 25) \n", 473 | "Test set: \t\t(36000, 25)\n", 474 | "label set: \t\t(105000,) \n", 475 | "Validation label set: \t(9000,) \n", 476 | "Test label set: \t\t(36000,)\n" 477 | ] 478 | } 479 | ], 480 | "source": [ 481 | "split_frac = 0.7\n", 482 | "\n", 483 | "split_index = int(split_frac * len(features))\n", 484 | "\n", 485 | "train_x, val_x = features[:split_index], features[split_index:] \n", 486 | "train_y, val_y = data.sentiment[:split_index], data.sentiment[split_index:len(features)]\n", 487 | "\n", 488 | "split_frac = 0.2\n", 489 | "split_index = int(split_frac * len(val_x))\n", 490 | "\n", 491 | "val_x, test_x = val_x[:split_index], val_x[split_index:]\n", 492 | "val_y, test_y = val_y[:split_index], val_y[split_index:]\n", 493 | "\n", 494 | "print(\"\\t\\t\\tFeature Shapes:\")\n", 495 | "print(\"Train set: \\t\\t{}\".format(train_x.shape), \n", 496 | " \"\\nValidation set: \\t{}\".format(val_x.shape),\n", 497 | " \"\\nTest set: \\t\\t{}\".format(test_x.shape))\n", 498 | "print(\"label set: \\t\\t{}\".format(train_y.shape), \n", 499 | " \"\\nValidation label set: \\t{}\".format(val_y.shape),\n", 500 | " \"\\nTest label set: \\t\\t{}\".format(test_y.shape))" 501 | ] 502 | }, 503 | { 504 | "cell_type": "code", 505 | "execution_count": 16, 506 | "metadata": {}, 507 | "outputs": [], 508 | "source": [ 509 | "lstm_size = 256\n", 510 | "lstm_layers = 2\n", 511 | "batch_size = 1000\n", 512 | "learning_rate = 0.01\n", 513 | "\n", 514 | "n_words = len(vocab_to_int) + 1 # Add 1 for 0 added to vocab\n", 515 | "\n", 516 | "# Create the graph object\n", 517 | "tf.reset_default_graph()\n", 518 | "with tf.name_scope('inputs'):\n", 519 | " inputs_ = tf.placeholder(tf.int32, [None, None], name=\"inputs\")\n", 520 | " labels_ = tf.placeholder(tf.int32, [None, None], name=\"labels\")\n", 521 | " keep_prob = tf.placeholder(tf.float32, name=\"keep_prob\")\n", 522 | " \n", 523 | "embed_size = 300 \n", 524 | "\n", 525 | "with tf.name_scope(\"Embeddings\"):\n", 526 | " embedding = tf.Variable(tf.random_uniform((n_words, embed_size), -1, 1))\n", 527 | " embed = tf.nn.embedding_lookup(embedding, inputs_)\n", 528 | " \n", 529 | "def lstm_cell():\n", 530 | " # Your basic LSTM cell\n", 531 | " lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size, reuse=tf.get_variable_scope().reuse)\n", 532 | " # Add dropout to the cell\n", 533 | " return tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)\n", 534 | "\n", 535 | "with tf.name_scope(\"RNN_layers\"):\n", 536 | " # Stack up multiple LSTM layers, for deep learning\n", 537 | " cell = tf.contrib.rnn.MultiRNNCell([lstm_cell() for _ in range(lstm_layers)])\n", 538 | " \n", 539 | " # Getting an initial state of all zeros\n", 540 | " initial_state = cell.zero_state(batch_size, tf.float32)\n", 541 | "\n", 542 | "with tf.name_scope(\"RNN_forward\"):\n", 543 | " outputs, final_state = tf.nn.dynamic_rnn(cell, embed, initial_state=initial_state)\n", 544 | "\n", 545 | "with tf.name_scope('predictions'):\n", 546 | " predictions = tf.contrib.layers.fully_connected(outputs[:, -1], 1, activation_fn=tf.sigmoid)\n", 547 | " tf.summary.histogram('predictions', predictions)\n", 548 | "with tf.name_scope('cost'):\n", 549 | " cost = tf.losses.mean_squared_error(labels_, predictions)\n", 550 | " tf.summary.scalar('cost', cost)\n", 551 | "\n", 552 | "with tf.name_scope('train'):\n", 553 | " optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)\n", 554 | "\n", 555 | "merged = tf.summary.merge_all()\n", 556 | "\n", 557 | "with tf.name_scope('validation'):\n", 558 | " correct_pred = tf.equal(tf.cast(tf.round(predictions), tf.int32), labels_)\n", 559 | " accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))\n", 560 | " \n", 561 | "def get_batches(x, y, batch_size=100):\n", 562 | " \n", 563 | " n_batches = len(x)//batch_size\n", 564 | " x, y = x[:n_batches*batch_size], y[:n_batches*batch_size]\n", 565 | " for ii in range(0, len(x), batch_size):\n", 566 | " yield x[ii:ii+batch_size], y[ii:ii+batch_size]\n" 567 | ] 568 | }, 569 | { 570 | "cell_type": "code", 571 | "execution_count": 17, 572 | "metadata": {}, 573 | "outputs": [], 574 | "source": [ 575 | "saver = tf.train.Saver()" 576 | ] 577 | }, 578 | { 579 | "cell_type": "code", 580 | "execution_count": 13, 581 | "metadata": {}, 582 | "outputs": [ 583 | { 584 | "name": "stdout", 585 | "output_type": "stream", 586 | "text": [ 587 | "Epoch: 0/30 Iteration: 5 Train loss: 0.278\n", 588 | "Epoch: 0/30 Iteration: 10 Train loss: 0.227\n", 589 | "Epoch: 0/30 Iteration: 15 Train loss: 0.216\n", 590 | "Epoch: 0/30 Iteration: 20 Train loss: 0.214\n", 591 | "Epoch: 0/30 Iteration: 25 Train loss: 0.213\n", 592 | "Val acc: 0.682\n", 593 | "Epoch: 0/30 Iteration: 30 Train loss: 0.205\n", 594 | "Epoch: 0/30 Iteration: 35 Train loss: 0.197\n", 595 | "Epoch: 0/30 Iteration: 40 Train loss: 0.187\n", 596 | "Epoch: 0/30 Iteration: 45 Train loss: 0.177\n", 597 | "Epoch: 0/30 Iteration: 50 Train loss: 0.186\n", 598 | "Val acc: 0.764\n", 599 | "Epoch: 0/30 Iteration: 55 Train loss: 0.170\n", 600 | "Epoch: 0/30 Iteration: 60 Train loss: 0.182\n", 601 | "Epoch: 0/30 Iteration: 65 Train loss: 0.162\n", 602 | "Epoch: 0/30 Iteration: 70 Train loss: 0.159\n", 603 | "Epoch: 0/30 Iteration: 75 Train loss: 0.158\n", 604 | "Val acc: 0.778\n", 605 | "Epoch: 1/30 Iteration: 80 Train loss: 0.148\n", 606 | "Epoch: 1/30 Iteration: 85 Train loss: 0.158\n", 607 | "Epoch: 1/30 Iteration: 90 Train loss: 0.146\n", 608 | "Epoch: 1/30 Iteration: 95 Train loss: 0.145\n", 609 | "Epoch: 1/30 Iteration: 100 Train loss: 0.146\n", 610 | "Val acc: 0.793\n", 611 | "Epoch: 1/30 Iteration: 105 Train loss: 0.124\n", 612 | "Epoch: 1/30 Iteration: 110 Train loss: 0.129\n", 613 | "Epoch: 1/30 Iteration: 115 Train loss: 0.121\n", 614 | "Epoch: 1/30 Iteration: 120 Train loss: 0.110\n", 615 | "Epoch: 1/30 Iteration: 125 Train loss: 0.113\n", 616 | "Val acc: 0.792\n", 617 | "Epoch: 1/30 Iteration: 130 Train loss: 0.119\n", 618 | "Epoch: 1/30 Iteration: 135 Train loss: 0.107\n", 619 | "Epoch: 1/30 Iteration: 140 Train loss: 0.096\n", 620 | "Epoch: 1/30 Iteration: 145 Train loss: 0.121\n", 621 | "Epoch: 1/30 Iteration: 150 Train loss: 0.106\n", 622 | "Val acc: 0.789\n", 623 | "Epoch: 2/30 Iteration: 155 Train loss: 0.098\n", 624 | "Epoch: 2/30 Iteration: 160 Train loss: 0.119\n", 625 | "Epoch: 2/30 Iteration: 165 Train loss: 0.100\n", 626 | "Epoch: 2/30 Iteration: 170 Train loss: 0.099\n", 627 | "Epoch: 2/30 Iteration: 175 Train loss: 0.096\n", 628 | "Val acc: 0.791\n", 629 | "Epoch: 2/30 Iteration: 180 Train loss: 0.105\n", 630 | "Epoch: 2/30 Iteration: 185 Train loss: 0.084\n", 631 | "Epoch: 2/30 Iteration: 190 Train loss: 0.096\n", 632 | "Epoch: 2/30 Iteration: 195 Train loss: 0.088\n", 633 | "Epoch: 2/30 Iteration: 200 Train loss: 0.069\n", 634 | "Val acc: 0.789\n", 635 | "Epoch: 2/30 Iteration: 205 Train loss: 0.089\n", 636 | "Epoch: 2/30 Iteration: 210 Train loss: 0.093\n", 637 | "Epoch: 2/30 Iteration: 215 Train loss: 0.084\n", 638 | "Epoch: 2/30 Iteration: 220 Train loss: 0.079\n", 639 | "Epoch: 2/30 Iteration: 225 Train loss: 0.080\n", 640 | "Val acc: 0.788\n", 641 | "Epoch: 2/30 Iteration: 230 Train loss: 0.085\n", 642 | "Epoch: 3/30 Iteration: 235 Train loss: 0.086\n", 643 | "Epoch: 3/30 Iteration: 240 Train loss: 0.073\n", 644 | "Epoch: 3/30 Iteration: 245 Train loss: 0.067\n", 645 | "Epoch: 3/30 Iteration: 250 Train loss: 0.071\n", 646 | "Val acc: 0.788\n", 647 | "Epoch: 3/30 Iteration: 255 Train loss: 0.059\n", 648 | "Epoch: 3/30 Iteration: 260 Train loss: 0.065\n", 649 | "Epoch: 3/30 Iteration: 265 Train loss: 0.074\n", 650 | "Epoch: 3/30 Iteration: 270 Train loss: 0.071\n", 651 | "Epoch: 3/30 Iteration: 275 Train loss: 0.058\n", 652 | "Val acc: 0.786\n", 653 | "Epoch: 3/30 Iteration: 280 Train loss: 0.065\n", 654 | "Epoch: 3/30 Iteration: 285 Train loss: 0.067\n", 655 | "Epoch: 3/30 Iteration: 290 Train loss: 0.071\n", 656 | "Epoch: 3/30 Iteration: 295 Train loss: 0.086\n", 657 | "Epoch: 3/30 Iteration: 300 Train loss: 0.078\n", 658 | "Val acc: 0.792\n", 659 | "Epoch: 3/30 Iteration: 305 Train loss: 0.073\n", 660 | "Epoch: 4/30 Iteration: 310 Train loss: 0.069\n", 661 | "Epoch: 4/30 Iteration: 315 Train loss: 0.070\n", 662 | "Epoch: 4/30 Iteration: 320 Train loss: 0.055\n", 663 | "Epoch: 4/30 Iteration: 325 Train loss: 0.064\n", 664 | "Val acc: 0.777\n", 665 | "Epoch: 4/30 Iteration: 330 Train loss: 0.064\n", 666 | "Epoch: 4/30 Iteration: 335 Train loss: 0.071\n", 667 | "Epoch: 4/30 Iteration: 340 Train loss: 0.056\n", 668 | "Epoch: 4/30 Iteration: 345 Train loss: 0.051\n", 669 | "Epoch: 4/30 Iteration: 350 Train loss: 0.054\n", 670 | "Val acc: 0.790\n", 671 | "Epoch: 4/30 Iteration: 355 Train loss: 0.049\n", 672 | "Epoch: 4/30 Iteration: 360 Train loss: 0.063\n", 673 | "Epoch: 4/30 Iteration: 365 Train loss: 0.060\n", 674 | "Epoch: 4/30 Iteration: 370 Train loss: 0.059\n", 675 | "Epoch: 4/30 Iteration: 375 Train loss: 0.064\n", 676 | "Val acc: 0.784\n", 677 | "Epoch: 4/30 Iteration: 380 Train loss: 0.064\n", 678 | "Epoch: 4/30 Iteration: 385 Train loss: 0.054\n", 679 | "Epoch: 5/30 Iteration: 390 Train loss: 0.048\n", 680 | "Epoch: 5/30 Iteration: 395 Train loss: 0.047\n", 681 | "Epoch: 5/30 Iteration: 400 Train loss: 0.040\n", 682 | "Val acc: 0.789\n", 683 | "Epoch: 5/30 Iteration: 405 Train loss: 0.052\n", 684 | "Epoch: 5/30 Iteration: 410 Train loss: 0.059\n", 685 | "Epoch: 5/30 Iteration: 415 Train loss: 0.051\n", 686 | "Epoch: 5/30 Iteration: 420 Train loss: 0.055\n", 687 | "Epoch: 5/30 Iteration: 425 Train loss: 0.055\n", 688 | "Val acc: 0.785\n", 689 | "Epoch: 5/30 Iteration: 430 Train loss: 0.042\n", 690 | "Epoch: 5/30 Iteration: 435 Train loss: 0.051\n", 691 | "Epoch: 5/30 Iteration: 440 Train loss: 0.054\n", 692 | "Epoch: 5/30 Iteration: 445 Train loss: 0.050\n", 693 | "Epoch: 5/30 Iteration: 450 Train loss: 0.045\n", 694 | "Val acc: 0.789\n", 695 | "Epoch: 5/30 Iteration: 455 Train loss: 0.051\n", 696 | "Epoch: 5/30 Iteration: 460 Train loss: 0.048\n", 697 | "Epoch: 6/30 Iteration: 465 Train loss: 0.045\n", 698 | "Epoch: 6/30 Iteration: 470 Train loss: 0.053\n", 699 | "Epoch: 6/30 Iteration: 475 Train loss: 0.050\n", 700 | "Val acc: 0.783\n", 701 | "Epoch: 6/30 Iteration: 480 Train loss: 0.045\n", 702 | "Epoch: 6/30 Iteration: 485 Train loss: 0.055\n", 703 | "Epoch: 6/30 Iteration: 490 Train loss: 0.046\n", 704 | "Epoch: 6/30 Iteration: 495 Train loss: 0.041\n", 705 | "Epoch: 6/30 Iteration: 500 Train loss: 0.047\n", 706 | "Val acc: 0.788\n", 707 | "Epoch: 6/30 Iteration: 505 Train loss: 0.051\n", 708 | "Epoch: 6/30 Iteration: 510 Train loss: 0.042\n", 709 | "Epoch: 6/30 Iteration: 515 Train loss: 0.047\n", 710 | "Epoch: 6/30 Iteration: 520 Train loss: 0.049\n", 711 | "Epoch: 6/30 Iteration: 525 Train loss: 0.041\n", 712 | "Val acc: 0.788\n", 713 | "Epoch: 6/30 Iteration: 530 Train loss: 0.057\n", 714 | "Epoch: 6/30 Iteration: 535 Train loss: 0.037\n", 715 | "Epoch: 7/30 Iteration: 540 Train loss: 0.040\n", 716 | "Epoch: 7/30 Iteration: 545 Train loss: 0.057\n", 717 | "Epoch: 7/30 Iteration: 550 Train loss: 0.037\n", 718 | "Val acc: 0.780\n", 719 | "Epoch: 7/30 Iteration: 555 Train loss: 0.047\n", 720 | "Epoch: 7/30 Iteration: 560 Train loss: 0.042\n", 721 | "Epoch: 7/30 Iteration: 565 Train loss: 0.038\n", 722 | "Epoch: 7/30 Iteration: 570 Train loss: 0.033\n", 723 | "Epoch: 7/30 Iteration: 575 Train loss: 0.044\n", 724 | "Val acc: 0.789\n", 725 | "Epoch: 7/30 Iteration: 580 Train loss: 0.035\n", 726 | "Epoch: 7/30 Iteration: 585 Train loss: 0.034\n", 727 | "Epoch: 7/30 Iteration: 590 Train loss: 0.038\n", 728 | "Epoch: 7/30 Iteration: 595 Train loss: 0.047\n", 729 | "Epoch: 7/30 Iteration: 600 Train loss: 0.030\n", 730 | "Val acc: 0.772\n", 731 | "Epoch: 7/30 Iteration: 605 Train loss: 0.040\n", 732 | "Epoch: 7/30 Iteration: 610 Train loss: 0.040\n", 733 | "Epoch: 7/30 Iteration: 615 Train loss: 0.044\n", 734 | "Epoch: 8/30 Iteration: 620 Train loss: 0.047\n", 735 | "Epoch: 8/30 Iteration: 625 Train loss: 0.032\n", 736 | "Val acc: 0.787\n", 737 | "Epoch: 8/30 Iteration: 630 Train loss: 0.035\n", 738 | "Epoch: 8/30 Iteration: 635 Train loss: 0.038\n", 739 | "Epoch: 8/30 Iteration: 640 Train loss: 0.022\n", 740 | "Epoch: 8/30 Iteration: 645 Train loss: 0.030\n", 741 | "Epoch: 8/30 Iteration: 650 Train loss: 0.039\n", 742 | "Val acc: 0.777\n", 743 | "Epoch: 8/30 Iteration: 655 Train loss: 0.031\n", 744 | "Epoch: 8/30 Iteration: 660 Train loss: 0.033\n", 745 | "Epoch: 8/30 Iteration: 665 Train loss: 0.031\n", 746 | "Epoch: 8/30 Iteration: 670 Train loss: 0.038\n", 747 | "Epoch: 8/30 Iteration: 675 Train loss: 0.035\n", 748 | "Val acc: 0.781\n", 749 | "Epoch: 8/30 Iteration: 680 Train loss: 0.042\n", 750 | "Epoch: 8/30 Iteration: 685 Train loss: 0.045\n", 751 | "Epoch: 8/30 Iteration: 690 Train loss: 0.031\n", 752 | "Epoch: 9/30 Iteration: 695 Train loss: 0.035\n", 753 | "Epoch: 9/30 Iteration: 700 Train loss: 0.038\n", 754 | "Val acc: 0.779\n", 755 | "Epoch: 9/30 Iteration: 705 Train loss: 0.027\n", 756 | "Epoch: 9/30 Iteration: 710 Train loss: 0.031\n", 757 | "Epoch: 9/30 Iteration: 715 Train loss: 0.025\n", 758 | "Epoch: 9/30 Iteration: 720 Train loss: 0.040\n", 759 | "Epoch: 9/30 Iteration: 725 Train loss: 0.036\n", 760 | "Val acc: 0.788\n", 761 | "Epoch: 9/30 Iteration: 730 Train loss: 0.035\n", 762 | "Epoch: 9/30 Iteration: 735 Train loss: 0.032\n", 763 | "Epoch: 9/30 Iteration: 740 Train loss: 0.031\n", 764 | "Epoch: 9/30 Iteration: 745 Train loss: 0.035\n", 765 | "Epoch: 9/30 Iteration: 750 Train loss: 0.028\n", 766 | "Val acc: 0.780\n", 767 | "Epoch: 9/30 Iteration: 755 Train loss: 0.031\n", 768 | "Epoch: 9/30 Iteration: 760 Train loss: 0.036\n", 769 | "Epoch: 9/30 Iteration: 765 Train loss: 0.033\n", 770 | "Epoch: 9/30 Iteration: 770 Train loss: 0.030\n", 771 | "Epoch: 10/30 Iteration: 775 Train loss: 0.029\n", 772 | "Val acc: 0.781\n", 773 | "Epoch: 10/30 Iteration: 780 Train loss: 0.025\n", 774 | "Epoch: 10/30 Iteration: 785 Train loss: 0.027\n", 775 | "Epoch: 10/30 Iteration: 790 Train loss: 0.029\n", 776 | "Epoch: 10/30 Iteration: 795 Train loss: 0.028\n", 777 | "Epoch: 10/30 Iteration: 800 Train loss: 0.030\n", 778 | "Val acc: 0.786\n", 779 | "Epoch: 10/30 Iteration: 805 Train loss: 0.036\n", 780 | "Epoch: 10/30 Iteration: 810 Train loss: 0.037\n", 781 | "Epoch: 10/30 Iteration: 815 Train loss: 0.029\n", 782 | "Epoch: 10/30 Iteration: 820 Train loss: 0.038\n", 783 | "Epoch: 10/30 Iteration: 825 Train loss: 0.037\n", 784 | "Val acc: 0.784\n", 785 | "Epoch: 10/30 Iteration: 830 Train loss: 0.026\n", 786 | "Epoch: 10/30 Iteration: 835 Train loss: 0.032\n", 787 | "Epoch: 10/30 Iteration: 840 Train loss: 0.042\n", 788 | "Epoch: 10/30 Iteration: 845 Train loss: 0.034\n", 789 | "Epoch: 11/30 Iteration: 850 Train loss: 0.025\n", 790 | "Val acc: 0.785\n", 791 | "Epoch: 11/30 Iteration: 855 Train loss: 0.032\n" 792 | ] 793 | }, 794 | { 795 | "name": "stdout", 796 | "output_type": "stream", 797 | "text": [ 798 | "Epoch: 11/30 Iteration: 860 Train loss: 0.037\n", 799 | "Epoch: 11/30 Iteration: 865 Train loss: 0.030\n", 800 | "Epoch: 11/30 Iteration: 870 Train loss: 0.038\n", 801 | "Epoch: 11/30 Iteration: 875 Train loss: 0.027\n", 802 | "Val acc: 0.780\n", 803 | "Epoch: 11/30 Iteration: 880 Train loss: 0.033\n", 804 | "Epoch: 11/30 Iteration: 885 Train loss: 0.029\n", 805 | "Epoch: 11/30 Iteration: 890 Train loss: 0.032\n", 806 | "Epoch: 11/30 Iteration: 895 Train loss: 0.029\n", 807 | "Epoch: 11/30 Iteration: 900 Train loss: 0.039\n", 808 | "Val acc: 0.784\n", 809 | "Epoch: 11/30 Iteration: 905 Train loss: 0.032\n", 810 | "Epoch: 11/30 Iteration: 910 Train loss: 0.031\n", 811 | "Epoch: 11/30 Iteration: 915 Train loss: 0.035\n", 812 | "Epoch: 11/30 Iteration: 920 Train loss: 0.031\n", 813 | "Epoch: 12/30 Iteration: 925 Train loss: 0.027\n", 814 | "Val acc: 0.785\n", 815 | "Epoch: 12/30 Iteration: 930 Train loss: 0.032\n", 816 | "Epoch: 12/30 Iteration: 935 Train loss: 0.023\n", 817 | "Epoch: 12/30 Iteration: 940 Train loss: 0.034\n", 818 | "Epoch: 12/30 Iteration: 945 Train loss: 0.037\n", 819 | "Epoch: 12/30 Iteration: 950 Train loss: 0.028\n", 820 | "Val acc: 0.785\n", 821 | "Epoch: 12/30 Iteration: 955 Train loss: 0.024\n", 822 | "Epoch: 12/30 Iteration: 960 Train loss: 0.033\n", 823 | "Epoch: 12/30 Iteration: 965 Train loss: 0.026\n", 824 | "Epoch: 12/30 Iteration: 970 Train loss: 0.025\n", 825 | "Epoch: 12/30 Iteration: 975 Train loss: 0.032\n", 826 | "Val acc: 0.785\n", 827 | "Epoch: 12/30 Iteration: 980 Train loss: 0.034\n", 828 | "Epoch: 12/30 Iteration: 985 Train loss: 0.021\n", 829 | "Epoch: 12/30 Iteration: 990 Train loss: 0.029\n", 830 | "Epoch: 12/30 Iteration: 995 Train loss: 0.034\n", 831 | "Epoch: 12/30 Iteration: 1000 Train loss: 0.033\n", 832 | "Val acc: 0.788\n", 833 | "Epoch: 13/30 Iteration: 1005 Train loss: 0.032\n", 834 | "Epoch: 13/30 Iteration: 1010 Train loss: 0.029\n", 835 | "Epoch: 13/30 Iteration: 1015 Train loss: 0.027\n", 836 | "Epoch: 13/30 Iteration: 1020 Train loss: 0.026\n", 837 | "Epoch: 13/30 Iteration: 1025 Train loss: 0.022\n", 838 | "Val acc: 0.784\n", 839 | "Epoch: 13/30 Iteration: 1030 Train loss: 0.030\n", 840 | "Epoch: 13/30 Iteration: 1035 Train loss: 0.027\n", 841 | "Epoch: 13/30 Iteration: 1040 Train loss: 0.032\n", 842 | "Epoch: 13/30 Iteration: 1045 Train loss: 0.028\n", 843 | "Epoch: 13/30 Iteration: 1050 Train loss: 0.029\n", 844 | "Val acc: 0.784\n", 845 | "Epoch: 13/30 Iteration: 1055 Train loss: 0.035\n", 846 | "Epoch: 13/30 Iteration: 1060 Train loss: 0.029\n", 847 | "Epoch: 13/30 Iteration: 1065 Train loss: 0.039\n", 848 | "Epoch: 13/30 Iteration: 1070 Train loss: 0.038\n", 849 | "Epoch: 13/30 Iteration: 1075 Train loss: 0.023\n", 850 | "Val acc: 0.783\n", 851 | "Epoch: 14/30 Iteration: 1080 Train loss: 0.032\n", 852 | "Epoch: 14/30 Iteration: 1085 Train loss: 0.032\n", 853 | "Epoch: 14/30 Iteration: 1090 Train loss: 0.024\n", 854 | "Epoch: 14/30 Iteration: 1095 Train loss: 0.024\n", 855 | "Epoch: 14/30 Iteration: 1100 Train loss: 0.024\n", 856 | "Val acc: 0.783\n", 857 | "Epoch: 14/30 Iteration: 1105 Train loss: 0.033\n", 858 | "Epoch: 14/30 Iteration: 1110 Train loss: 0.028\n", 859 | "Epoch: 14/30 Iteration: 1115 Train loss: 0.028\n", 860 | "Epoch: 14/30 Iteration: 1120 Train loss: 0.031\n", 861 | "Epoch: 14/30 Iteration: 1125 Train loss: 0.029\n", 862 | "Val acc: 0.785\n", 863 | "Epoch: 14/30 Iteration: 1130 Train loss: 0.030\n", 864 | "Epoch: 14/30 Iteration: 1135 Train loss: 0.027\n", 865 | "Epoch: 14/30 Iteration: 1140 Train loss: 0.030\n", 866 | "Epoch: 14/30 Iteration: 1145 Train loss: 0.033\n", 867 | "Epoch: 14/30 Iteration: 1150 Train loss: 0.036\n", 868 | "Val acc: 0.783\n", 869 | "Epoch: 14/30 Iteration: 1155 Train loss: 0.036\n", 870 | "Epoch: 15/30 Iteration: 1160 Train loss: 0.032\n", 871 | "Epoch: 15/30 Iteration: 1165 Train loss: 0.025\n", 872 | "Epoch: 15/30 Iteration: 1170 Train loss: 0.021\n", 873 | "Epoch: 15/30 Iteration: 1175 Train loss: 0.031\n", 874 | "Val acc: 0.778\n", 875 | "Epoch: 15/30 Iteration: 1180 Train loss: 0.027\n", 876 | "Epoch: 15/30 Iteration: 1185 Train loss: 0.033\n", 877 | "Epoch: 15/30 Iteration: 1190 Train loss: 0.033\n", 878 | "Epoch: 15/30 Iteration: 1195 Train loss: 0.038\n", 879 | "Epoch: 15/30 Iteration: 1200 Train loss: 0.030\n", 880 | "Val acc: 0.784\n", 881 | "Epoch: 15/30 Iteration: 1205 Train loss: 0.038\n", 882 | "Epoch: 15/30 Iteration: 1210 Train loss: 0.034\n", 883 | "Epoch: 15/30 Iteration: 1215 Train loss: 0.024\n", 884 | "Epoch: 15/30 Iteration: 1220 Train loss: 0.028\n", 885 | "Epoch: 15/30 Iteration: 1225 Train loss: 0.034\n", 886 | "Val acc: 0.772\n", 887 | "Epoch: 15/30 Iteration: 1230 Train loss: 0.027\n", 888 | "Epoch: 16/30 Iteration: 1235 Train loss: 0.028\n", 889 | "Epoch: 16/30 Iteration: 1240 Train loss: 0.026\n", 890 | "Epoch: 16/30 Iteration: 1245 Train loss: 0.030\n", 891 | "Epoch: 16/30 Iteration: 1250 Train loss: 0.027\n", 892 | "Val acc: 0.777\n", 893 | "Epoch: 16/30 Iteration: 1255 Train loss: 0.029\n", 894 | "Epoch: 16/30 Iteration: 1260 Train loss: 0.030\n", 895 | "Epoch: 16/30 Iteration: 1265 Train loss: 0.033\n", 896 | "Epoch: 16/30 Iteration: 1270 Train loss: 0.029\n", 897 | "Epoch: 16/30 Iteration: 1275 Train loss: 0.028\n", 898 | "Val acc: 0.779\n", 899 | "Epoch: 16/30 Iteration: 1280 Train loss: 0.027\n", 900 | "Epoch: 16/30 Iteration: 1285 Train loss: 0.031\n", 901 | "Epoch: 16/30 Iteration: 1290 Train loss: 0.024\n", 902 | "Epoch: 16/30 Iteration: 1295 Train loss: 0.025\n", 903 | "Epoch: 16/30 Iteration: 1300 Train loss: 0.028\n", 904 | "Val acc: 0.776\n", 905 | "Epoch: 16/30 Iteration: 1305 Train loss: 0.025\n", 906 | "Epoch: 17/30 Iteration: 1310 Train loss: 0.023\n", 907 | "Epoch: 17/30 Iteration: 1315 Train loss: 0.034\n", 908 | "Epoch: 17/30 Iteration: 1320 Train loss: 0.025\n", 909 | "Epoch: 17/30 Iteration: 1325 Train loss: 0.029\n", 910 | "Val acc: 0.775\n", 911 | "Epoch: 17/30 Iteration: 1330 Train loss: 0.029\n", 912 | "Epoch: 17/30 Iteration: 1335 Train loss: 0.027\n", 913 | "Epoch: 17/30 Iteration: 1340 Train loss: 0.023\n", 914 | "Epoch: 17/30 Iteration: 1345 Train loss: 0.026\n", 915 | "Epoch: 17/30 Iteration: 1350 Train loss: 0.023\n", 916 | "Val acc: 0.778\n", 917 | "Epoch: 17/30 Iteration: 1355 Train loss: 0.020\n", 918 | "Epoch: 17/30 Iteration: 1360 Train loss: 0.025\n", 919 | "Epoch: 17/30 Iteration: 1365 Train loss: 0.032\n", 920 | "Epoch: 17/30 Iteration: 1370 Train loss: 0.022\n", 921 | "Epoch: 17/30 Iteration: 1375 Train loss: 0.024\n", 922 | "Val acc: 0.775\n", 923 | "Epoch: 17/30 Iteration: 1380 Train loss: 0.029\n", 924 | "Epoch: 17/30 Iteration: 1385 Train loss: 0.031\n", 925 | "Epoch: 18/30 Iteration: 1390 Train loss: 0.032\n", 926 | "Epoch: 18/30 Iteration: 1395 Train loss: 0.022\n", 927 | "Epoch: 18/30 Iteration: 1400 Train loss: 0.027\n", 928 | "Val acc: 0.775\n", 929 | "Epoch: 18/30 Iteration: 1405 Train loss: 0.024\n", 930 | "Epoch: 18/30 Iteration: 1410 Train loss: 0.017\n", 931 | "Epoch: 18/30 Iteration: 1415 Train loss: 0.026\n", 932 | "Epoch: 18/30 Iteration: 1420 Train loss: 0.032\n", 933 | "Epoch: 18/30 Iteration: 1425 Train loss: 0.030\n", 934 | "Val acc: 0.776\n", 935 | "Epoch: 18/30 Iteration: 1430 Train loss: 0.028\n", 936 | "Epoch: 18/30 Iteration: 1435 Train loss: 0.029\n", 937 | "Epoch: 18/30 Iteration: 1440 Train loss: 0.036\n", 938 | "Epoch: 18/30 Iteration: 1445 Train loss: 0.029\n", 939 | "Epoch: 18/30 Iteration: 1450 Train loss: 0.031\n", 940 | "Val acc: 0.771\n", 941 | "Epoch: 18/30 Iteration: 1455 Train loss: 0.035\n", 942 | "Epoch: 18/30 Iteration: 1460 Train loss: 0.021\n", 943 | "Epoch: 19/30 Iteration: 1465 Train loss: 0.025\n", 944 | "Epoch: 19/30 Iteration: 1470 Train loss: 0.025\n", 945 | "Epoch: 19/30 Iteration: 1475 Train loss: 0.019\n", 946 | "Val acc: 0.772\n", 947 | "Epoch: 19/30 Iteration: 1480 Train loss: 0.021\n", 948 | "Epoch: 19/30 Iteration: 1485 Train loss: 0.021\n", 949 | "Epoch: 19/30 Iteration: 1490 Train loss: 0.027\n", 950 | "Epoch: 19/30 Iteration: 1495 Train loss: 0.027\n", 951 | "Epoch: 19/30 Iteration: 1500 Train loss: 0.024\n", 952 | "Val acc: 0.783\n", 953 | "Epoch: 19/30 Iteration: 1505 Train loss: 0.026\n", 954 | "Epoch: 19/30 Iteration: 1510 Train loss: 0.021\n", 955 | "Epoch: 19/30 Iteration: 1515 Train loss: 0.030\n", 956 | "Epoch: 19/30 Iteration: 1520 Train loss: 0.023\n", 957 | "Epoch: 19/30 Iteration: 1525 Train loss: 0.024\n", 958 | "Val acc: 0.775\n", 959 | "Epoch: 19/30 Iteration: 1530 Train loss: 0.025\n", 960 | "Epoch: 19/30 Iteration: 1535 Train loss: 0.034\n", 961 | "Epoch: 19/30 Iteration: 1540 Train loss: 0.026\n", 962 | "Epoch: 20/30 Iteration: 1545 Train loss: 0.024\n", 963 | "Epoch: 20/30 Iteration: 1550 Train loss: 0.020\n", 964 | "Val acc: 0.778\n", 965 | "Epoch: 20/30 Iteration: 1555 Train loss: 0.024\n", 966 | "Epoch: 20/30 Iteration: 1560 Train loss: 0.023\n", 967 | "Epoch: 20/30 Iteration: 1565 Train loss: 0.024\n", 968 | "Epoch: 20/30 Iteration: 1570 Train loss: 0.025\n", 969 | "Epoch: 20/30 Iteration: 1575 Train loss: 0.028\n", 970 | "Val acc: 0.774\n", 971 | "Epoch: 20/30 Iteration: 1580 Train loss: 0.034\n", 972 | "Epoch: 20/30 Iteration: 1585 Train loss: 0.026\n", 973 | "Epoch: 20/30 Iteration: 1590 Train loss: 0.038\n", 974 | "Epoch: 20/30 Iteration: 1595 Train loss: 0.031\n", 975 | "Epoch: 20/30 Iteration: 1600 Train loss: 0.028\n", 976 | "Val acc: 0.770\n", 977 | "Epoch: 20/30 Iteration: 1605 Train loss: 0.033\n", 978 | "Epoch: 20/30 Iteration: 1610 Train loss: 0.032\n", 979 | "Epoch: 20/30 Iteration: 1615 Train loss: 0.034\n", 980 | "Epoch: 21/30 Iteration: 1620 Train loss: 0.025\n", 981 | "Epoch: 21/30 Iteration: 1625 Train loss: 0.032\n", 982 | "Val acc: 0.780\n", 983 | "Epoch: 21/30 Iteration: 1630 Train loss: 0.030\n", 984 | "Epoch: 21/30 Iteration: 1635 Train loss: 0.032\n", 985 | "Epoch: 21/30 Iteration: 1640 Train loss: 0.027\n", 986 | "Epoch: 21/30 Iteration: 1645 Train loss: 0.026\n", 987 | "Epoch: 21/30 Iteration: 1650 Train loss: 0.031\n", 988 | "Val acc: 0.783\n", 989 | "Epoch: 21/30 Iteration: 1655 Train loss: 0.028\n", 990 | "Epoch: 21/30 Iteration: 1660 Train loss: 0.029\n", 991 | "Epoch: 21/30 Iteration: 1665 Train loss: 0.034\n", 992 | "Epoch: 21/30 Iteration: 1670 Train loss: 0.032\n", 993 | "Epoch: 21/30 Iteration: 1675 Train loss: 0.030\n", 994 | "Val acc: 0.780\n", 995 | "Epoch: 21/30 Iteration: 1680 Train loss: 0.025\n" 996 | ] 997 | }, 998 | { 999 | "name": "stdout", 1000 | "output_type": "stream", 1001 | "text": [ 1002 | "Epoch: 21/30 Iteration: 1685 Train loss: 0.029\n", 1003 | "Epoch: 21/30 Iteration: 1690 Train loss: 0.024\n", 1004 | "Epoch: 22/30 Iteration: 1695 Train loss: 0.023\n", 1005 | "Epoch: 22/30 Iteration: 1700 Train loss: 0.035\n", 1006 | "Val acc: 0.779\n", 1007 | "Epoch: 22/30 Iteration: 1705 Train loss: 0.028\n", 1008 | "Epoch: 22/30 Iteration: 1710 Train loss: 0.036\n", 1009 | "Epoch: 22/30 Iteration: 1715 Train loss: 0.034\n", 1010 | "Epoch: 22/30 Iteration: 1720 Train loss: 0.024\n", 1011 | "Epoch: 22/30 Iteration: 1725 Train loss: 0.027\n", 1012 | "Val acc: 0.779\n", 1013 | "Epoch: 22/30 Iteration: 1730 Train loss: 0.028\n", 1014 | "Epoch: 22/30 Iteration: 1735 Train loss: 0.027\n", 1015 | "Epoch: 22/30 Iteration: 1740 Train loss: 0.023\n", 1016 | "Epoch: 22/30 Iteration: 1745 Train loss: 0.029\n", 1017 | "Epoch: 22/30 Iteration: 1750 Train loss: 0.032\n", 1018 | "Val acc: 0.781\n", 1019 | "Epoch: 22/30 Iteration: 1755 Train loss: 0.020\n", 1020 | "Epoch: 22/30 Iteration: 1760 Train loss: 0.028\n", 1021 | "Epoch: 22/30 Iteration: 1765 Train loss: 0.030\n", 1022 | "Epoch: 22/30 Iteration: 1770 Train loss: 0.037\n", 1023 | "Epoch: 23/30 Iteration: 1775 Train loss: 0.030\n", 1024 | "Val acc: 0.781\n", 1025 | "Epoch: 23/30 Iteration: 1780 Train loss: 0.024\n", 1026 | "Epoch: 23/30 Iteration: 1785 Train loss: 0.019\n", 1027 | "Epoch: 23/30 Iteration: 1790 Train loss: 0.026\n", 1028 | "Epoch: 23/30 Iteration: 1795 Train loss: 0.020\n", 1029 | "Epoch: 23/30 Iteration: 1800 Train loss: 0.023\n", 1030 | "Val acc: 0.776\n", 1031 | "Epoch: 23/30 Iteration: 1805 Train loss: 0.030\n", 1032 | "Epoch: 23/30 Iteration: 1810 Train loss: 0.026\n", 1033 | "Epoch: 23/30 Iteration: 1815 Train loss: 0.026\n", 1034 | "Epoch: 23/30 Iteration: 1820 Train loss: 0.024\n", 1035 | "Epoch: 23/30 Iteration: 1825 Train loss: 0.033\n", 1036 | "Val acc: 0.779\n", 1037 | "Epoch: 23/30 Iteration: 1830 Train loss: 0.024\n", 1038 | "Epoch: 23/30 Iteration: 1835 Train loss: 0.034\n", 1039 | "Epoch: 23/30 Iteration: 1840 Train loss: 0.034\n", 1040 | "Epoch: 23/30 Iteration: 1845 Train loss: 0.020\n", 1041 | "Epoch: 24/30 Iteration: 1850 Train loss: 0.026\n", 1042 | "Val acc: 0.779\n", 1043 | "Epoch: 24/30 Iteration: 1855 Train loss: 0.026\n", 1044 | "Epoch: 24/30 Iteration: 1860 Train loss: 0.021\n", 1045 | "Epoch: 24/30 Iteration: 1865 Train loss: 0.029\n", 1046 | "Epoch: 24/30 Iteration: 1870 Train loss: 0.025\n", 1047 | "Epoch: 24/30 Iteration: 1875 Train loss: 0.024\n", 1048 | "Val acc: 0.778\n", 1049 | "Epoch: 24/30 Iteration: 1880 Train loss: 0.031\n", 1050 | "Epoch: 24/30 Iteration: 1885 Train loss: 0.024\n", 1051 | "Epoch: 24/30 Iteration: 1890 Train loss: 0.028\n", 1052 | "Epoch: 24/30 Iteration: 1895 Train loss: 0.026\n", 1053 | "Epoch: 24/30 Iteration: 1900 Train loss: 0.036\n", 1054 | "Val acc: 0.777\n", 1055 | "Epoch: 24/30 Iteration: 1905 Train loss: 0.032\n", 1056 | "Epoch: 24/30 Iteration: 1910 Train loss: 0.030\n", 1057 | "Epoch: 24/30 Iteration: 1915 Train loss: 0.026\n", 1058 | "Epoch: 24/30 Iteration: 1920 Train loss: 0.027\n", 1059 | "Epoch: 24/30 Iteration: 1925 Train loss: 0.027\n", 1060 | "Val acc: 0.779\n", 1061 | "Epoch: 25/30 Iteration: 1930 Train loss: 0.023\n", 1062 | "Epoch: 25/30 Iteration: 1935 Train loss: 0.023\n", 1063 | "Epoch: 25/30 Iteration: 1940 Train loss: 0.019\n", 1064 | "Epoch: 25/30 Iteration: 1945 Train loss: 0.025\n", 1065 | "Epoch: 25/30 Iteration: 1950 Train loss: 0.027\n", 1066 | "Val acc: 0.776\n", 1067 | "Epoch: 25/30 Iteration: 1955 Train loss: 0.025\n", 1068 | "Epoch: 25/30 Iteration: 1960 Train loss: 0.025\n", 1069 | "Epoch: 25/30 Iteration: 1965 Train loss: 0.035\n", 1070 | "Epoch: 25/30 Iteration: 1970 Train loss: 0.029\n", 1071 | "Epoch: 25/30 Iteration: 1975 Train loss: 0.032\n", 1072 | "Val acc: 0.778\n", 1073 | "Epoch: 25/30 Iteration: 1980 Train loss: 0.033\n", 1074 | "Epoch: 25/30 Iteration: 1985 Train loss: 0.025\n", 1075 | "Epoch: 25/30 Iteration: 1990 Train loss: 0.029\n", 1076 | "Epoch: 25/30 Iteration: 1995 Train loss: 0.027\n", 1077 | "Epoch: 25/30 Iteration: 2000 Train loss: 0.027\n", 1078 | "Val acc: 0.776\n", 1079 | "Epoch: 26/30 Iteration: 2005 Train loss: 0.026\n", 1080 | "Epoch: 26/30 Iteration: 2010 Train loss: 0.034\n", 1081 | "Epoch: 26/30 Iteration: 2015 Train loss: 0.031\n", 1082 | "Epoch: 26/30 Iteration: 2020 Train loss: 0.030\n", 1083 | "Epoch: 26/30 Iteration: 2025 Train loss: 0.037\n", 1084 | "Val acc: 0.775\n", 1085 | "Epoch: 26/30 Iteration: 2030 Train loss: 0.023\n", 1086 | "Epoch: 26/30 Iteration: 2035 Train loss: 0.033\n", 1087 | "Epoch: 26/30 Iteration: 2040 Train loss: 0.031\n", 1088 | "Epoch: 26/30 Iteration: 2045 Train loss: 0.031\n", 1089 | "Epoch: 26/30 Iteration: 2050 Train loss: 0.026\n", 1090 | "Val acc: 0.774\n", 1091 | "Epoch: 26/30 Iteration: 2055 Train loss: 0.032\n", 1092 | "Epoch: 26/30 Iteration: 2060 Train loss: 0.026\n", 1093 | "Epoch: 26/30 Iteration: 2065 Train loss: 0.033\n", 1094 | "Epoch: 26/30 Iteration: 2070 Train loss: 0.029\n", 1095 | "Epoch: 26/30 Iteration: 2075 Train loss: 0.026\n", 1096 | "Val acc: 0.766\n", 1097 | "Epoch: 27/30 Iteration: 2080 Train loss: 0.022\n", 1098 | "Epoch: 27/30 Iteration: 2085 Train loss: 0.038\n", 1099 | "Epoch: 27/30 Iteration: 2090 Train loss: 0.027\n", 1100 | "Epoch: 27/30 Iteration: 2095 Train loss: 0.033\n", 1101 | "Epoch: 27/30 Iteration: 2100 Train loss: 0.028\n", 1102 | "Val acc: 0.773\n", 1103 | "Epoch: 27/30 Iteration: 2105 Train loss: 0.026\n", 1104 | "Epoch: 27/30 Iteration: 2110 Train loss: 0.025\n", 1105 | "Epoch: 27/30 Iteration: 2115 Train loss: 0.029\n", 1106 | "Epoch: 27/30 Iteration: 2120 Train loss: 0.026\n", 1107 | "Epoch: 27/30 Iteration: 2125 Train loss: 0.021\n", 1108 | "Val acc: 0.772\n", 1109 | "Epoch: 27/30 Iteration: 2130 Train loss: 0.026\n", 1110 | "Epoch: 27/30 Iteration: 2135 Train loss: 0.029\n", 1111 | "Epoch: 27/30 Iteration: 2140 Train loss: 0.023\n", 1112 | "Epoch: 27/30 Iteration: 2145 Train loss: 0.028\n", 1113 | "Epoch: 27/30 Iteration: 2150 Train loss: 0.029\n", 1114 | "Val acc: 0.767\n", 1115 | "Epoch: 27/30 Iteration: 2155 Train loss: 0.030\n", 1116 | "Epoch: 28/30 Iteration: 2160 Train loss: 0.031\n", 1117 | "Epoch: 28/30 Iteration: 2165 Train loss: 0.016\n", 1118 | "Epoch: 28/30 Iteration: 2170 Train loss: 0.023\n", 1119 | "Epoch: 28/30 Iteration: 2175 Train loss: 0.025\n", 1120 | "Val acc: 0.768\n", 1121 | "Epoch: 28/30 Iteration: 2180 Train loss: 0.017\n", 1122 | "Epoch: 28/30 Iteration: 2185 Train loss: 0.022\n", 1123 | "Epoch: 28/30 Iteration: 2190 Train loss: 0.025\n", 1124 | "Epoch: 28/30 Iteration: 2195 Train loss: 0.023\n", 1125 | "Epoch: 28/30 Iteration: 2200 Train loss: 0.021\n", 1126 | "Val acc: 0.776\n", 1127 | "Epoch: 28/30 Iteration: 2205 Train loss: 0.019\n", 1128 | "Epoch: 28/30 Iteration: 2210 Train loss: 0.030\n", 1129 | "Epoch: 28/30 Iteration: 2215 Train loss: 0.027\n", 1130 | "Epoch: 28/30 Iteration: 2220 Train loss: 0.036\n", 1131 | "Epoch: 28/30 Iteration: 2225 Train loss: 0.030\n", 1132 | "Val acc: 0.775\n", 1133 | "Epoch: 28/30 Iteration: 2230 Train loss: 0.020\n", 1134 | "Epoch: 29/30 Iteration: 2235 Train loss: 0.024\n", 1135 | "Epoch: 29/30 Iteration: 2240 Train loss: 0.026\n", 1136 | "Epoch: 29/30 Iteration: 2245 Train loss: 0.023\n", 1137 | "Epoch: 29/30 Iteration: 2250 Train loss: 0.022\n", 1138 | "Val acc: 0.775\n", 1139 | "Epoch: 29/30 Iteration: 2255 Train loss: 0.021\n", 1140 | "Epoch: 29/30 Iteration: 2260 Train loss: 0.024\n", 1141 | "Epoch: 29/30 Iteration: 2265 Train loss: 0.025\n", 1142 | "Epoch: 29/30 Iteration: 2270 Train loss: 0.023\n", 1143 | "Epoch: 29/30 Iteration: 2275 Train loss: 0.024\n", 1144 | "Val acc: 0.778\n", 1145 | "Epoch: 29/30 Iteration: 2280 Train loss: 0.023\n", 1146 | "Epoch: 29/30 Iteration: 2285 Train loss: 0.025\n", 1147 | "Epoch: 29/30 Iteration: 2290 Train loss: 0.022\n", 1148 | "Epoch: 29/30 Iteration: 2295 Train loss: 0.026\n", 1149 | "Epoch: 29/30 Iteration: 2300 Train loss: 0.026\n", 1150 | "Val acc: 0.775\n", 1151 | "Epoch: 29/30 Iteration: 2305 Train loss: 0.029\n", 1152 | "Epoch: 29/30 Iteration: 2310 Train loss: 0.027\n" 1153 | ] 1154 | } 1155 | ], 1156 | "source": [ 1157 | "epochs = 30\n", 1158 | "\n", 1159 | "# with graph.as_default():\n", 1160 | "saver = tf.train.Saver()\n", 1161 | "\n", 1162 | "with tf.Session() as sess:\n", 1163 | " sess.run(tf.global_variables_initializer())\n", 1164 | " train_writer = tf.summary.FileWriter('./logs/tb/train', sess.graph)\n", 1165 | " test_writer = tf.summary.FileWriter('./logs/tb/test', sess.graph)\n", 1166 | " iteration = 1\n", 1167 | " for e in range(epochs):\n", 1168 | " state = sess.run(initial_state)\n", 1169 | " \n", 1170 | " for ii, (x, y) in enumerate(get_batches(train_x, train_y, batch_size), 1):\n", 1171 | " feed = {inputs_: x,\n", 1172 | " labels_: y[:, None],\n", 1173 | " keep_prob: 0.5,\n", 1174 | " initial_state: state}\n", 1175 | " summary, loss, state, _ = sess.run([merged, cost, final_state, optimizer], feed_dict=feed)\n", 1176 | "# loss, state, _ = sess.run([cost, final_state, optimizer], feed_dict=feed)\n", 1177 | "\n", 1178 | " train_writer.add_summary(summary, iteration)\n", 1179 | " \n", 1180 | " if iteration%5==0:\n", 1181 | " print(\"Epoch: {}/{}\".format(e, epochs),\n", 1182 | " \"Iteration: {}\".format(iteration),\n", 1183 | " \"Train loss: {:.3f}\".format(loss))\n", 1184 | "\n", 1185 | " if iteration%25==0:\n", 1186 | " val_acc = []\n", 1187 | " val_state = sess.run(cell.zero_state(batch_size, tf.float32))\n", 1188 | " for x, y in get_batches(val_x, val_y, batch_size):\n", 1189 | " feed = {inputs_: x,\n", 1190 | " labels_: y[:, None],\n", 1191 | " keep_prob: 1,\n", 1192 | " initial_state: val_state}\n", 1193 | "# batch_acc, val_state = sess.run([accuracy, final_state], feed_dict=feed)\n", 1194 | " summary, batch_acc, val_state = sess.run([merged, accuracy, final_state], feed_dict=feed)\n", 1195 | " val_acc.append(batch_acc)\n", 1196 | " print(\"Val acc: {:.3f}\".format(np.mean(val_acc)))\n", 1197 | " iteration +=1\n", 1198 | " test_writer.add_summary(summary, iteration)\n", 1199 | " saver.save(sess, \"checkpoints/sentiment_jatin.ckpt\")\n", 1200 | " saver.save(sess, \"checkpoints/sentiment_jatin.ckpt\")" 1201 | ] 1202 | }, 1203 | { 1204 | "cell_type": "code", 1205 | "execution_count": 18, 1206 | "metadata": {}, 1207 | "outputs": [ 1208 | { 1209 | "name": "stdout", 1210 | "output_type": "stream", 1211 | "text": [ 1212 | "INFO:tensorflow:Restoring parameters from checkpoints/sentiment_jatin.ckpt\n", 1213 | "Test accuracy: 0.695\n" 1214 | ] 1215 | } 1216 | ], 1217 | "source": [ 1218 | "test_acc = []\n", 1219 | "saver = tf.train.Saver()\n", 1220 | "with tf.Session() as sess:\n", 1221 | " saver.restore(sess, \"checkpoints/sentiment_jatin.ckpt\")\n", 1222 | " test_state = sess.run(cell.zero_state(batch_size, tf.float32))\n", 1223 | " for ii, (x, y) in enumerate(get_batches(test_x, test_y, batch_size), 1):\n", 1224 | " feed = {inputs_: x,\n", 1225 | " labels_: y[:, None],\n", 1226 | " keep_prob: 1,\n", 1227 | " initial_state: test_state}\n", 1228 | " batch_acc, test_state = sess.run([accuracy, final_state], feed_dict=feed)\n", 1229 | " test_acc.append(batch_acc)\n", 1230 | " print(\"Test accuracy: {:.3f}\".format(np.mean(test_acc)))" 1231 | ] 1232 | } 1233 | ], 1234 | "metadata": { 1235 | "kernelspec": { 1236 | "display_name": "Python 3", 1237 | "language": "python", 1238 | "name": "python3" 1239 | }, 1240 | "language_info": { 1241 | "codemirror_mode": { 1242 | "name": "ipython", 1243 | "version": 3 1244 | }, 1245 | "file_extension": ".py", 1246 | "mimetype": "text/x-python", 1247 | "name": "python", 1248 | "nbconvert_exporter": "python", 1249 | "pygments_lexer": "ipython3", 1250 | "version": "3.6.3" 1251 | } 1252 | }, 1253 | "nbformat": 4, 1254 | "nbformat_minor": 2 1255 | } 1256 | -------------------------------------------------------------------------------- /data-extractor/lexicon_bullish_words.txt: -------------------------------------------------------------------------------- 1 | las 2 | swung 3 | edible 4 | interim 5 | rv 6 | cio 7 | nominate 8 | mayor 9 | dysfunction 10 | jib 11 | bubbling 12 | lbj 13 | illogical 14 | meticulous 15 | sweetheart 16 | commodore 17 | ia 18 | ottawa 19 | litigate 20 | seesaw 21 | drainage 22 | segway 23 | triumph 24 | sleeps 25 | repeating 26 | brady 27 | teasing 28 | peninsula 29 | submarine 30 | hinder 31 | squirt 32 | measuring 33 | rivalry 34 | coolest 35 | wrecking 36 | moms 37 | daydreamer 38 | accepts 39 | mattress 40 | lane 41 | gravitation 42 | neighbourhood 43 | puzzle 44 | incinerate 45 | sleeper 46 | whelm 47 | punks 48 | derail 49 | embarrasses 50 | ecc 51 | incorrect 52 | horseshit 53 | borrowed 54 | matchup 55 | hurting 56 | intercede 57 | humanitarian 58 | jubilation 59 | fantastically 60 | modular 61 | psychotic 62 | overstated 63 | jelly 64 | intrinsically 65 | unanimously 66 | barcelona 67 | pear 68 | peanuts 69 | standby 70 | lateral 71 | philanthropist 72 | bra 73 | tolerable 74 | codger 75 | affective 76 | passed 77 | creamed 78 | dangerously 79 | groin 80 | utters 81 | arrhythmia 82 | cautionary 83 | prosperous 84 | nog 85 | displaying 86 | ideologue 87 | nub 88 | contradiction 89 | telco 90 | outflank 91 | hiv 92 | releasing 93 | sizzling 94 | accuses 95 | decoration 96 | stroll 97 | informative 98 | bouncy 99 | pandemonium 100 | nb 101 | switzerland 102 | disturb 103 | eiffel 104 | pours 105 | simeon 106 | portable 107 | bionic 108 | telephoto 109 | ritual 110 | collectable 111 | extenuate 112 | puzzled 113 | hedging 114 | executed 115 | bastards 116 | skull 117 | snapshots 118 | hinders 119 | downs 120 | pros 121 | tidy 122 | thriller 123 | paranoia 124 | unnerve 125 | hesitating 126 | inevitability 127 | astronomical 128 | granular 129 | cajun 130 | byte 131 | malaise 132 | stays 133 | regional 134 | plank 135 | calculating 136 | thermostat 137 | highway 138 | privy 139 | clearest 140 | iaas 141 | reverence 142 | brought 143 | erroneously 144 | downloaded 145 | craziest 146 | errant 147 | capitol 148 | outdone 149 | hamper 150 | lunas 151 | unfairly 152 | dicky 153 | audible 154 | civilian 155 | rainforest 156 | doubts 157 | pulsate 158 | intestine 159 | thirsty 160 | pepperoni 161 | agrees 162 | headland 163 | terabit 164 | crust 165 | middleman 166 | disturbance 167 | tes 168 | oversleep 169 | believable 170 | diabolical 171 | stp 172 | volley 173 | replete 174 | doggie 175 | antibiotic 176 | manana 177 | viciously 178 | ufo 179 | undoubtedly 180 | preference 181 | hash 182 | lunar 183 | clowns 184 | dovish 185 | intrigue 186 | siemens 187 | steamy 188 | stabilise 189 | stile 190 | idealism 191 | glove 192 | copper 193 | stakeholder 194 | gained 195 | disrupted 196 | kenya 197 | mailbox 198 | defensible 199 | warmer 200 | bulldoze 201 | earthly 202 | formulate 203 | arco 204 | goddess 205 | expression 206 | pathway 207 | dependence 208 | granted 209 | bumbling 210 | demarcation 211 | disgruntle 212 | steps 213 | flour 214 | anss 215 | tiffany 216 | rosebud 217 | iota 218 | requested 219 | cutter 220 | offspring 221 | upfront 222 | bakery 223 | upturn 224 | wisconsin 225 | substantiate 226 | ess 227 | tumbler 228 | raping 229 | standardize 230 | dally 231 | withdrew 232 | grain 233 | fixation 234 | malone 235 | colombia 236 | underperforms 237 | ownes 238 | midweek 239 | domain 240 | interval 241 | clay 242 | uppp 243 | drought 244 | homing 245 | luxe 246 | stoppage 247 | infrequently 248 | rachel 249 | seagull 250 | earnest 251 | fundamentals 252 | semester 253 | jacked 254 | goofball 255 | noodles 256 | salivating 257 | shakes 258 | feeder 259 | buddha 260 | maui 261 | trickery 262 | legitimizes 263 | schism 264 | vapor 265 | ingrain 266 | restraint 267 | xanax 268 | mow 269 | supernova 270 | gobble 271 | burr 272 | rummer 273 | distorted 274 | contend 275 | smirk 276 | consultation 277 | generality 278 | rash 279 | detection 280 | hart 281 | cooking 282 | advent 283 | cubs 284 | confining 285 | reorganizing 286 | averse 287 | boxer 288 | convey 289 | payed 290 | carton 291 | refuel 292 | amplifies 293 | whim 294 | withdrawal 295 | techy 296 | firefly 297 | groan 298 | sincerity 299 | header 300 | cocoon 301 | piracy 302 | jaguar 303 | blacken 304 | unawareness 305 | toddy 306 | portal 307 | shinny 308 | newark 309 | reorganization 310 | articulate 311 | fascinate 312 | mesa 313 | gluttonous 314 | infuse 315 | misinform 316 | unmanned 317 | resurrection 318 | massachusetts 319 | hughes 320 | dint 321 | synchrony 322 | climbing 323 | newfangled 324 | snps 325 | mason 326 | rigging 327 | forking 328 | blogger 329 | comedian 330 | grate 331 | understandable 332 | sol 333 | valve 334 | fuego 335 | remedy 336 | rundown 337 | likes 338 | washy 339 | chronology 340 | modify 341 | meaningfully 342 | battered 343 | bending 344 | trucker 345 | solidifies 346 | actuality 347 | siren 348 | accommodation 349 | dashboard 350 | enfilading 351 | truthfully 352 | tudor 353 | hardworking 354 | lighting 355 | kingmaker 356 | iis 357 | lark 358 | moaning 359 | infra 360 | vietnam 361 | reynolds 362 | turkish 363 | scraggly 364 | flapping 365 | protector 366 | calve 367 | oscars 368 | luxurys 369 | ltd. 370 | mutation 371 | delaying 372 | isolate 373 | contains 374 | recipient 375 | entrance 376 | approximately 377 | durability 378 | montana 379 | noticeably 380 | networked 381 | depreciation 382 | jv 383 | peri 384 | crane 385 | revival 386 | boob 387 | admonition 388 | wiper 389 | moo 390 | processing 391 | hoarder 392 | purr 393 | altruistic 394 | orwellian 395 | contracting 396 | rattling 397 | nonbeliever 398 | internets 399 | brash 400 | breeding 401 | phonebook 402 | nordic 403 | inspiration 404 | daytime 405 | corpus 406 | rile 407 | marino 408 | morbid 409 | lender 410 | coy 411 | exhaustive 412 | rebuke 413 | genocide 414 | dental 415 | seamless 416 | haste 417 | upright 418 | breadcrumb 419 | skateboard 420 | shallow 421 | swirling 422 | foresight 423 | doppelganger 424 | scratching 425 | googling 426 | florence 427 | shaped 428 | tramp 429 | witchcraft 430 | literal 431 | coordinated 432 | communicator 433 | welcomed 434 | greatest 435 | muslims 436 | sacramental 437 | caesar 438 | caldwell 439 | lashing 440 | valedictorian 441 | dds 442 | creature 443 | acolyte 444 | parody 445 | inversely 446 | discussed 447 | heifer 448 | wheeling 449 | granddaughter 450 | coho 451 | gigabit 452 | debunk 453 | spouse 454 | paw 455 | repatriate 456 | ludicrously 457 | phantom 458 | longevity 459 | egomaniac 460 | theology 461 | inflow 462 | walton 463 | lawrence 464 | connected 465 | bangkok 466 | glamour 467 | bandage 468 | popish 469 | methodically 470 | mold 471 | reshape 472 | heath 473 | ipos 474 | depart 475 | weston 476 | misfortune 477 | guacamole 478 | dims 479 | entires 480 | prelude 481 | ignites 482 | physically 483 | brainwave 484 | potassium 485 | unwanted 486 | rgs 487 | checkmate 488 | aquila 489 | delphi 490 | simpler 491 | squawk 492 | pulmonary 493 | prague 494 | unnamed 495 | amuck 496 | amazona 497 | indisputable 498 | asses 499 | unify 500 | sicken 501 | payable 502 | fought 503 | delightfully 504 | trove 505 | jvs 506 | engaging 507 | loader 508 | rearrange 509 | settled 510 | flaws 511 | posh 512 | darkened 513 | nobler 514 | electricity 515 | bronx 516 | burt 517 | spotlight 518 | roost 519 | aviation 520 | installed 521 | celebrates 522 | workspace 523 | reclaiming 524 | taxman 525 | elapse 526 | brazen 527 | specialized 528 | smiling 529 | rouge 530 | packed 531 | returning 532 | airbus 533 | hyperbole 534 | continent 535 | backhoe 536 | assholes 537 | compounder 538 | purvey 539 | ras 540 | variant 541 | instruct 542 | refocus 543 | convertible 544 | arbitrager 545 | pulverizes 546 | autonomy 547 | coalition 548 | mucky 549 | divert 550 | kilt 551 | unison 552 | bashed 553 | renewable 554 | dethrone 555 | sighting 556 | dairy 557 | rigorous 558 | gi 559 | peridot 560 | senselessly 561 | od 562 | shuttle 563 | managed 564 | conn 565 | anaemic 566 | suffocated 567 | psychic 568 | overland 569 | abbreviated 570 | flew 571 | chased 572 | bumping 573 | dangle 574 | inclusion 575 | ordinance 576 | synchronized 577 | exaggeration 578 | modification 579 | fabled 580 | confucius 581 | affirm 582 | disjoint 583 | unobtrusive 584 | flinch 585 | moth 586 | overhear 587 | commonwealth 588 | considerate 589 | notables 590 | mixer 591 | interface 592 | eos 593 | interfaces 594 | lope 595 | yip 596 | advancing 597 | nanosecond 598 | unfathomable 599 | connects 600 | functionality 601 | antisocial 602 | unopposed 603 | zamboni 604 | custodian 605 | lots 606 | symbolic 607 | harmonize 608 | cuckold 609 | farting 610 | lowest 611 | oiled 612 | methodology 613 | deliberates 614 | urges 615 | guggenheim 616 | champion 617 | specter 618 | cylindrical 619 | mediator 620 | accentuate 621 | reclusive 622 | shaking 623 | asur 624 | glimpse 625 | gathering 626 | nebbish 627 | securely 628 | thumbnail 629 | cirrus 630 | cannes 631 | pip 632 | bundling 633 | juncture 634 | grabs 635 | lockstep 636 | errand 637 | panicky 638 | dod 639 | peasants 640 | lino 641 | ova 642 | specialise 643 | tum 644 | pleas 645 | myrtle 646 | farfetched 647 | horseshoe 648 | mutilate 649 | impaired 650 | datass 651 | curry 652 | clause 653 | idly 654 | compliant 655 | deranged 656 | annuity 657 | silliness 658 | telepathy 659 | emu 660 | gnash 661 | pips 662 | dicey 663 | freshener 664 | pds 665 | kindred 666 | overlooked 667 | spat 668 | wis 669 | basing 670 | methodical 671 | lindsay 672 | nationalize 673 | redo 674 | axon 675 | scalability 676 | registration 677 | cay 678 | supremacy 679 | manipulates 680 | wrongly 681 | confront 682 | remount 683 | tolerate 684 | spice 685 | thd 686 | bombastic 687 | grimace 688 | strongest 689 | repository 690 | shaper 691 | brained 692 | smelling 693 | tango 694 | outstrip 695 | pleasantly 696 | masturbating 697 | sustainability 698 | ara 699 | telephone 700 | houdini 701 | sufferer 702 | peerless 703 | mentioning 704 | dine 705 | zen 706 | supposed 707 | blockish 708 | firmly 709 | paddock 710 | bock 711 | ember 712 | thrift 713 | sumo 714 | deferred 715 | rouse 716 | extortionist 717 | alexander 718 | transparently 719 | compounded 720 | looseness 721 | fitting 722 | calmly 723 | sharps 724 | caribbean 725 | misogynistic 726 | hedgehog 727 | companion 728 | vibrancy 729 | bedtime 730 | chuckle 731 | syphon 732 | kangaroo 733 | wean 734 | gripe 735 | novel 736 | forgiven 737 | genomics 738 | conservatism 739 | conclusive 740 | prolific 741 | multinational 742 | peeling 743 | orlando 744 | vector 745 | trivia 746 | sending 747 | misadventure 748 | directive 749 | reel 750 | steamroll 751 | gown 752 | stifle 753 | syndicate 754 | converge 755 | breathtaking 756 | stuffing 757 | eagles 758 | monger 759 | gratifying 760 | cicero 761 | ideas 762 | auctioneer 763 | deprive 764 | settling 765 | persona 766 | losers 767 | parakeet 768 | confidant 769 | calico 770 | pew 771 | unquenchable 772 | limitless 773 | communal 774 | toronto 775 | parrot 776 | emphatic 777 | postman 778 | payload 779 | dotted 780 | barnstormer 781 | caruso 782 | outline 783 | abracadabra 784 | playoff 785 | transgression 786 | ted 787 | sequential 788 | marathon 789 | incendiary 790 | independently 791 | dietary 792 | presenting 793 | japs 794 | dying 795 | unseat 796 | gratify 797 | postmortem 798 | shifting 799 | repertoire 800 | modernize 801 | tint 802 | fetched 803 | tito 804 | potion 805 | gamma 806 | ee 807 | realizing 808 | bourgeoisie 809 | touting 810 | beasts 811 | boarder 812 | altruism 813 | hydroponic 814 | cooker 815 | stature 816 | upsetting 817 | scss 818 | spinning 819 | repellent 820 | unaware 821 | iver 822 | ministry 823 | prescribed 824 | jaw 825 | evolutionary 826 | relaxing 827 | groceries 828 | tuner 829 | nicholas 830 | amusing 831 | sideward 832 | disproportionate 833 | scalable 834 | witching 835 | roaring 836 | jester 837 | combined 838 | cypress 839 | rupert 840 | mvp 841 | compliment 842 | introductory 843 | superstitious 844 | plough 845 | cleverly 846 | runway 847 | catchy 848 | approximate 849 | slowness 850 | zn 851 | prohibit 852 | buzzer 853 | pud 854 | dementia 855 | cobalt 856 | robinson 857 | eludes 858 | tvs 859 | defer 860 | remaining 861 | burdensome 862 | grit 863 | prioritise 864 | demonstration 865 | thins 866 | hologram 867 | collide 868 | histogram 869 | nauseate 870 | redes 871 | indonesia 872 | nomination 873 | luggage 874 | barfing 875 | rabies 876 | kingpin 877 | extravaganza 878 | smit 879 | spencer 880 | psychologically 881 | superpower 882 | ounce 883 | photographer 884 | preliminary 885 | predicate 886 | rotisserie 887 | reminisce 888 | rectangle 889 | deadest 890 | hooked 891 | taproom 892 | faze 893 | hostile 894 | cornered 895 | chubby 896 | poetry 897 | apex 898 | apt 899 | achilles 900 | clam 901 | frightened 902 | bottled 903 | guaranteeed 904 | proudly 905 | signifies 906 | shank 907 | responsibly 908 | yardstick 909 | kernel 910 | erratum 911 | observant 912 | landline 913 | retakes 914 | melbourne 915 | glamorous 916 | lowly 917 | squeal 918 | islamic 919 | arthur 920 | germ 921 | unimaginably 922 | booster 923 | maryland 924 | medicinal 925 | serendipitous 926 | weakening 927 | lloyd 928 | safest 929 | sheepish 930 | immense 931 | lightweight 932 | cervix 933 | trademarked 934 | jealously 935 | considers 936 | seasonally 937 | updates 938 | museum 939 | cheep 940 | pocketbook 941 | marshall 942 | precipitous 943 | defensively 944 | lister 945 | quantitative 946 | activewear 947 | bidet 948 | malign 949 | opiate 950 | gentlemans 951 | bewilderment 952 | indicates 953 | avid 954 | hush 955 | tenuous 956 | knowledgeable 957 | foolishness 958 | misfit 959 | crowed 960 | adonis 961 | peach 962 | disapprove 963 | whining 964 | vigilant 965 | bethe 966 | boosting 967 | brainy 968 | sarcastically 969 | turbine 970 | promised 971 | lucifer 972 | appointee 973 | tenner 974 | appease 975 | empress 976 | especial 977 | lament 978 | leaping 979 | factoring 980 | freakishly 981 | understands 982 | waterproof 983 | originality 984 | blabbermouth 985 | humdrum 986 | bullpen 987 | nigeria 988 | obstruct 989 | prosaic 990 | watchman 991 | politicise 992 | cancun 993 | replay 994 | canon 995 | crafty 996 | seymour 997 | decoy 998 | tried 999 | yeller 1000 | spaceman 1001 | positivity 1002 | succeeded 1003 | elasticity 1004 | misinterpret 1005 | amuses 1006 | designate 1007 | adventure 1008 | skunk 1009 | ladder 1010 | chanal 1011 | blowhard 1012 | bracket 1013 | lovelace 1014 | greased 1015 | invited 1016 | credence 1017 | calvin 1018 | jacksonville 1019 | acclaim 1020 | jello 1021 | morphs 1022 | lobster 1023 | reciprocal 1024 | turbulent 1025 | uncontested 1026 | waited 1027 | skipper 1028 | fireball 1029 | regina 1030 | iteration 1031 | ting 1032 | petard 1033 | introspection 1034 | kindly 1035 | choppiness 1036 | answering 1037 | norris 1038 | hud 1039 | narcissistic 1040 | deferment 1041 | sterling 1042 | godfather 1043 | ecstatically 1044 | stabler 1045 | schooled 1046 | harris 1047 | speechless 1048 | nan 1049 | dandruff 1050 | teddy 1051 | skyward 1052 | laziness 1053 | awkward 1054 | streamline 1055 | delirious 1056 | notably 1057 | supervisor 1058 | maya 1059 | vengeful 1060 | edward 1061 | reproduce 1062 | leo 1063 | armour 1064 | voltage 1065 | exam 1066 | nth 1067 | northrop 1068 | mulla 1069 | runt 1070 | mondays 1071 | thievery 1072 | kink 1073 | intimate 1074 | crank 1075 | merchandising 1076 | neutralize 1077 | goddam 1078 | hypertension 1079 | sortie 1080 | akin 1081 | diversion 1082 | complementary 1083 | suburban 1084 | recapturing 1085 | weighs 1086 | therapeutic 1087 | brent 1088 | overspending 1089 | rose 1090 | lockdown 1091 | shibboleth 1092 | thickly 1093 | unpolished 1094 | sequentially 1095 | doubter 1096 | cincinnati 1097 | celebrating 1098 | illinois 1099 | thermonuclear 1100 | strangely 1101 | shortsighted 1102 | disk 1103 | linguistic 1104 | unloved 1105 | scad 1106 | cosh 1107 | turbulence 1108 | expired 1109 | skater 1110 | blastoff 1111 | burry 1112 | hemisphere 1113 | extrinsic 1114 | locomotive 1115 | optimization 1116 | grater 1117 | apprentice 1118 | paperweight 1119 | prescribe 1120 | enrollment 1121 | dol 1122 | bears 1123 | cinema 1124 | folio 1125 | bitching 1126 | idol 1127 | cookout 1128 | slamming 1129 | boundless 1130 | affiliation 1131 | ney 1132 | mph 1133 | uproar 1134 | blender 1135 | holdout 1136 | jagged 1137 | optimum 1138 | burying 1139 | personalized 1140 | bumble 1141 | geopolitics 1142 | ointment 1143 | mcs 1144 | surpassed 1145 | fandango 1146 | halo 1147 | comforter 1148 | obstacle 1149 | fruitless 1150 | uptown 1151 | accompany 1152 | qs 1153 | unveiled 1154 | porcine 1155 | ab 1156 | rewarded 1157 | cozy 1158 | bourbon 1159 | heartfelt 1160 | genetic 1161 | kos 1162 | detox 1163 | spliff 1164 | epa 1165 | cbr 1166 | hallucinating 1167 | lit 1168 | dumdum 1169 | conquering 1170 | shamble 1171 | inconsequential 1172 | greenhouse 1173 | ejaculation 1174 | ga 1175 | valium 1176 | hacksaw 1177 | unleashing 1178 | dialogue 1179 | nova 1180 | glenn 1181 | tasting 1182 | jumpstart 1183 | vino 1184 | sams 1185 | recovered 1186 | aarp 1187 | twofold 1188 | paternity 1189 | cyclone 1190 | nomad 1191 | unboxing 1192 | slavic 1193 | bike 1194 | overstep 1195 | crusty 1196 | identifiably 1197 | thereof 1198 | reining 1199 | wring 1200 | charterhouse 1201 | enjoyable 1202 | precedential 1203 | snafu 1204 | grope 1205 | superstar 1206 | overarch 1207 | cater 1208 | sheen 1209 | negotiation 1210 | giants 1211 | oak 1212 | glassware 1213 | makeover 1214 | authentication 1215 | supported 1216 | barn 1217 | uptake 1218 | lense 1219 | slack 1220 | episodic 1221 | gb 1222 | shrivel 1223 | reluctantly 1224 | leaf 1225 | madder 1226 | rocky 1227 | manly 1228 | churning 1229 | bulldog 1230 | tennis 1231 | dancer 1232 | stank 1233 | slut 1234 | thc 1235 | plebs 1236 | consumes 1237 | farming 1238 | bogey 1239 | loveliness 1240 | scratcher 1241 | minimus 1242 | sweeten 1243 | rake 1244 | villager 1245 | dovishness 1246 | depressive 1247 | thug 1248 | cursor 1249 | lx 1250 | emphasize 1251 | beak 1252 | curling 1253 | montgomery 1254 | jolt 1255 | zep 1256 | reno 1257 | reassert 1258 | tome 1259 | iner 1260 | vend 1261 | aerodynamic 1262 | structurally 1263 | mock 1264 | hired 1265 | flurry 1266 | challenger 1267 | undecided 1268 | timed 1269 | cad 1270 | moxie 1271 | handful 1272 | shortcut 1273 | hastings 1274 | quest 1275 | endeavour 1276 | underweight 1277 | mubarak 1278 | permit 1279 | inexpensively 1280 | representation 1281 | painfully 1282 | hs 1283 | escrow 1284 | virgin 1285 | trepidation 1286 | twirl 1287 | toddler 1288 | appearing 1289 | jihad 1290 | revered 1291 | biding 1292 | practises 1293 | impeached 1294 | realtor 1295 | scarlet 1296 | temperament 1297 | schnitzel 1298 | sunset 1299 | tempered 1300 | musically 1301 | spiteful 1302 | poaches 1303 | independence 1304 | babble 1305 | uncommon 1306 | preserving 1307 | nsu 1308 | authorization 1309 | usability 1310 | slither 1311 | irritating 1312 | monetizing 1313 | untouched 1314 | formal 1315 | flicker 1316 | additionally 1317 | sassafras 1318 | stalwart 1319 | naively 1320 | feisty 1321 | natter 1322 | brilliantly 1323 | gyrate 1324 | squeaky 1325 | acting 1326 | daimler 1327 | appreciative 1328 | interleave 1329 | glom 1330 | flying 1331 | analogy 1332 | exasperated 1333 | racer 1334 | acre 1335 | valhalla 1336 | suggests 1337 | everlasting 1338 | lennon 1339 | plowing 1340 | activation 1341 | luna 1342 | dodo 1343 | touchable 1344 | paced 1345 | selective 1346 | cpi 1347 | impenetrable 1348 | supra 1349 | mayonnaise 1350 | liner 1351 | humorous 1352 | sleazy 1353 | metropolitan 1354 | investigated 1355 | mansion 1356 | reorganize 1357 | overdrive 1358 | shoplift 1359 | synchronization 1360 | partly 1361 | underground 1362 | preparation 1363 | emigrant 1364 | parable 1365 | superiority 1366 | consorted 1367 | spillover 1368 | organism 1369 | doofus 1370 | neon 1371 | amazons 1372 | barb 1373 | clairvoyant 1374 | poorer 1375 | refute 1376 | toolbox 1377 | snags 1378 | fluster 1379 | eagerly 1380 | fascination 1381 | obtain 1382 | conjecture 1383 | underprice 1384 | manger 1385 | clench 1386 | leds 1387 | intertwined 1388 | immunity 1389 | reinstate 1390 | ans 1391 | ammunition 1392 | scoff 1393 | counterproductive 1394 | princess 1395 | pinto 1396 | repatriated 1397 | prank 1398 | gaze 1399 | crusade 1400 | biting 1401 | shipped 1402 | cosy 1403 | allocator 1404 | outsold 1405 | bachelor 1406 | zoo 1407 | wacko 1408 | magnetic 1409 | usurp 1410 | screener 1411 | burlington 1412 | recognise 1413 | wolves 1414 | earner 1415 | dealt 1416 | cylinder 1417 | vroom 1418 | savara 1419 | animation 1420 | northeast 1421 | advises 1422 | frontier 1423 | defies 1424 | mileage 1425 | err 1426 | guitar 1427 | discrepancies 1428 | emporium 1429 | twister 1430 | profiteer 1431 | herb 1432 | wildfire 1433 | vagina 1434 | repulsive 1435 | template 1436 | distinct 1437 | difficulty 1438 | measured 1439 | patsy 1440 | fern 1441 | jackson 1442 | humanoid 1443 | favorable 1444 | histories 1445 | aristocrat 1446 | tambourine 1447 | warthog 1448 | billow 1449 | parallel 1450 | insurer 1451 | triggered 1452 | enraged 1453 | beautify 1454 | appreciates 1455 | battling 1456 | craftsman 1457 | clutter 1458 | sleek 1459 | altar 1460 | catapult 1461 | besiege 1462 | nefarious 1463 | learner 1464 | fairness 1465 | sleigh 1466 | disrespect 1467 | distracts 1468 | sullenness 1469 | memorable 1470 | ec 1471 | stunk 1472 | generalise 1473 | linear 1474 | onslaught 1475 | keynesian 1476 | ukraine 1477 | drumbeat 1478 | laughter 1479 | thursdays 1480 | taxable 1481 | instigation 1482 | surfboard 1483 | lupus 1484 | bose 1485 | vicariously 1486 | collective 1487 | haiku 1488 | pebble 1489 | peyote 1490 | spacesuit 1491 | sparkler 1492 | daydream 1493 | fortification 1494 | tinny 1495 | relive 1496 | ditto 1497 | uncharacteristic 1498 | stabilizing 1499 | paramount 1500 | drub 1501 | au 1502 | replaces 1503 | fraught 1504 | nacho 1505 | payola 1506 | yi 1507 | hither 1508 | toke 1509 | ledger 1510 | needing 1511 | positioning 1512 | babes 1513 | annotated 1514 | evidently 1515 | frantically 1516 | accrue 1517 | ox 1518 | sparrow 1519 | shingle 1520 | telegraphed 1521 | supertanker 1522 | sportswear 1523 | capitulating 1524 | profoundly 1525 | delegate 1526 | charlotte 1527 | centre 1528 | silky 1529 | carnival 1530 | arrangement 1531 | scolding 1532 | reinforce 1533 | ora 1534 | steed 1535 | stealthy 1536 | durables 1537 | shortened 1538 | absorption 1539 | pennsylvania 1540 | parsley 1541 | seminal 1542 | divulge 1543 | gib 1544 | blabber 1545 | initiatives 1546 | toupe 1547 | wanted 1548 | subtleties 1549 | latitude 1550 | unanimous 1551 | combining 1552 | breast 1553 | marx 1554 | removable 1555 | hundredth 1556 | motorcycle 1557 | mooning 1558 | podcasts 1559 | oftentimes 1560 | sandbox 1561 | ration 1562 | clamber 1563 | trickling 1564 | neanderthals 1565 | retrenchment 1566 | architect 1567 | gnarly 1568 | vegetarian 1569 | protected 1570 | splashing 1571 | distinctive 1572 | ams 1573 | applied 1574 | forthcoming 1575 | unimaginable 1576 | therapist 1577 | outreach 1578 | stratum 1579 | mettle 1580 | muscular 1581 | customary 1582 | problematic 1583 | legalize 1584 | staying 1585 | automated 1586 | madman 1587 | gapping 1588 | surreal 1589 | outsized 1590 | ribs 1591 | piping 1592 | streamed 1593 | pontificate 1594 | lac 1595 | banging 1596 | habitat 1597 | cartel 1598 | fcs 1599 | pronouncement 1600 | waver 1601 | beforehand 1602 | tts 1603 | excerpt 1604 | pomo 1605 | spate 1606 | budge 1607 | yawning 1608 | differentiate 1609 | fret 1610 | basking 1611 | doghouse 1612 | pursues 1613 | ejaculate 1614 | moolah 1615 | hussein 1616 | rework 1617 | merlot 1618 | ripper 1619 | unpredictable 1620 | conduit 1621 | crore 1622 | vague 1623 | avert 1624 | lancer 1625 | linkup 1626 | sadness 1627 | yisrael 1628 | authentic 1629 | uneducated 1630 | fez 1631 | alp 1632 | inextricably 1633 | bearable 1634 | geographically 1635 | attracts 1636 | checklist 1637 | smalls 1638 | appoints 1639 | rided 1640 | palmer 1641 | pausing 1642 | heaping 1643 | erupt 1644 | asinine 1645 | secede 1646 | hotness 1647 | snorter 1648 | passively 1649 | annum 1650 | bestseller 1651 | aah 1652 | kickoff 1653 | avenger 1654 | writes 1655 | theatrical 1656 | electrolyte 1657 | promotional 1658 | tahiti 1659 | reed 1660 | kinetic 1661 | backbone 1662 | mapping 1663 | residential 1664 | stoppable 1665 | receives 1666 | brigade 1667 | baghdad 1668 | indexing 1669 | willfully 1670 | bouquet 1671 | scrooge 1672 | rebounding 1673 | mensch 1674 | badge 1675 | selects 1676 | norway 1677 | portuguese 1678 | het 1679 | eggs 1680 | embodiment 1681 | bangle 1682 | interruption 1683 | broaden 1684 | napoli 1685 | pasta 1686 | initiation 1687 | unrealistically 1688 | underestimating 1689 | banana 1690 | manhood 1691 | orderes 1692 | argos 1693 | comprehensive 1694 | var 1695 | cheeky 1696 | berk 1697 | unwavering 1698 | portend 1699 | suspends 1700 | foray 1701 | sicilian 1702 | togo 1703 | fitch 1704 | tactless 1705 | bushel 1706 | elaborate 1707 | crier 1708 | encourages 1709 | shaker 1710 | relic 1711 | buckwheat 1712 | ns 1713 | coco 1714 | mainland 1715 | claiming 1716 | revitalize 1717 | dl 1718 | signature 1719 | gel 1720 | fs 1721 | anne 1722 | matures 1723 | quantify 1724 | misunderstood 1725 | weirdos 1726 | heighten 1727 | overlaid 1728 | lenses 1729 | bulky 1730 | dada 1731 | comparatively 1732 | shoo 1733 | irregardless 1734 | notwithstanding 1735 | origin 1736 | blend 1737 | instills 1738 | identification 1739 | nec 1740 | transact 1741 | mbd 1742 | taxi 1743 | morsel 1744 | ossicle 1745 | trolling 1746 | restock 1747 | tor 1748 | certify 1749 | repot 1750 | bazooka 1751 | immaterial 1752 | overtaking 1753 | fanatical 1754 | sidelined 1755 | ths 1756 | acid 1757 | scenery 1758 | redesigned 1759 | millennia 1760 | childlike 1761 | portfolios 1762 | breakup 1763 | desist 1764 | flowing 1765 | anointed 1766 | dts 1767 | spanish 1768 | restraining 1769 | revamped 1770 | boogeyman 1771 | arbitration 1772 | resilience 1773 | detective 1774 | fringe 1775 | lovable 1776 | longing 1777 | adversely 1778 | accessibility 1779 | masturbate 1780 | sinclair 1781 | canvas 1782 | sweating 1783 | dither 1784 | pasty 1785 | threw 1786 | faq 1787 | shrimp 1788 | bureaucratic 1789 | wander 1790 | accessible 1791 | smear 1792 | tow 1793 | mourn 1794 | brutally 1795 | formidable 1796 | skim 1797 | primer 1798 | acumen 1799 | steaming 1800 | rattler 1801 | urgency 1802 | outgo 1803 | scoreboard 1804 | tangentially 1805 | emasculate 1806 | taxidermy 1807 | vulcan 1808 | tickle 1809 | wanting 1810 | wading 1811 | turnoff 1812 | divvys 1813 | ignition 1814 | disdain 1815 | millimeter 1816 | ecstatic 1817 | dale 1818 | sacrifice 1819 | divorced 1820 | deterioration 1821 | conventional 1822 | hauling 1823 | seduce 1824 | favorites 1825 | whitecap 1826 | overdramatize 1827 | skittish 1828 | renaissance 1829 | unemotional 1830 | soundtrack 1831 | appetizer 1832 | tnf 1833 | superbly 1834 | boardroom 1835 | shmuck 1836 | accommodating 1837 | energize 1838 | admired 1839 | disappearing 1840 | uneasy 1841 | indirect 1842 | muzzle 1843 | experiencing 1844 | remarkably 1845 | steepen 1846 | grading 1847 | headwinds 1848 | boxed 1849 | torch 1850 | dag 1851 | astray 1852 | orca 1853 | encroachment 1854 | lets 1855 | foolproof 1856 | pup 1857 | domains 1858 | farthest 1859 | estoppel 1860 | astute 1861 | inter 1862 | astonishing 1863 | clinician 1864 | lp 1865 | lynchburg 1866 | sailing 1867 | overheads 1868 | sticking 1869 | lechery 1870 | hiss 1871 | cutthroat 1872 | gourd 1873 | ascertain 1874 | metaphorically 1875 | uncharted 1876 | issuance 1877 | validates 1878 | apc 1879 | horace 1880 | calculator 1881 | prejudicial 1882 | chico 1883 | defraud 1884 | prolonged 1885 | bowing 1886 | urn 1887 | goldmine 1888 | furry 1889 | notebook 1890 | cyclists 1891 | localize 1892 | usher 1893 | anew 1894 | macron 1895 | cameras 1896 | homebuilder 1897 | comparative 1898 | inchworm 1899 | wrestle 1900 | undernourished 1901 | kws 1902 | headquarter 1903 | haunt 1904 | nominee 1905 | ratios 1906 | grandmother 1907 | hideout 1908 | thill 1909 | shipper 1910 | trophy 1911 | protectionist 1912 | trimester 1913 | anon 1914 | hating 1915 | greeks 1916 | unfixed 1917 | trickster 1918 | dylan 1919 | corona 1920 | jock 1921 | tiller 1922 | thunderbird 1923 | titian 1924 | boar 1925 | congo 1926 | thickens 1927 | imbecile 1928 | panicking 1929 | guesstimate 1930 | confusing 1931 | minivan 1932 | microscopic 1933 | darken 1934 | collector 1935 | tn 1936 | convention 1937 | glean 1938 | extensive 1939 | sizzle 1940 | swept 1941 | teaser 1942 | khan 1943 | faking 1944 | veteran 1945 | signed 1946 | outbid 1947 | growing 1948 | possession 1949 | masterful 1950 | pitfall 1951 | collectively 1952 | inundate 1953 | dropkick 1954 | metaphoric 1955 | wolfe 1956 | motley 1957 | rectification 1958 | modified 1959 | clx 1960 | squeezer 1961 | vaughan 1962 | congestion 1963 | brunch 1964 | tatum 1965 | grabber 1966 | tongue 1967 | attributable 1968 | allay 1969 | coyly 1970 | pulverized 1971 | telepathic 1972 | younger 1973 | bordeaux 1974 | constitution 1975 | wellness 1976 | predominantly 1977 | consolation 1978 | monroe 1979 | nipple 1980 | stabilizes 1981 | sheridan 1982 | weal 1983 | overflow 1984 | atheist 1985 | commends 1986 | kelt 1987 | drubbing 1988 | electrification 1989 | generates 1990 | abs 1991 | liken 1992 | storefront 1993 | mecca 1994 | truce 1995 | limping 1996 | questioner 1997 | diverse 1998 | imposing 1999 | negotiates 2000 | loon 2001 | unsound 2002 | scorched 2003 | ladies 2004 | accomplice 2005 | pout 2006 | spheroid 2007 | sewn 2008 | verily 2009 | pbs 2010 | surmise 2011 | analog 2012 | behaves 2013 | unjustly 2014 | gobsmacked 2015 | marley 2016 | closeout 2017 | unlock 2018 | fax 2019 | emaciate 2020 | baulk 2021 | ramrod 2022 | looter 2023 | mire 2024 | schilling 2025 | supercharger 2026 | stylized 2027 | maddening 2028 | condemn 2029 | wryly 2030 | grunt 2031 | stupefy 2032 | resigning 2033 | tangent 2034 | xs 2035 | panelist 2036 | viewers 2037 | clime 2038 | venue 2039 | cabin 2040 | interior 2041 | upbeat 2042 | jeep 2043 | musky 2044 | integrator 2045 | compulsion 2046 | nails 2047 | nonstick 2048 | downtime 2049 | confidential 2050 | pronoun 2051 | rejoice 2052 | urinate 2053 | spurn 2054 | condolence 2055 | lackadaisical 2056 | covet 2057 | putter 2058 | strangles 2059 | crispy 2060 | lair 2061 | bern 2062 | withdrawn 2063 | philadelphia 2064 | inspirational 2065 | decentralized 2066 | buccaneer 2067 | pregnant 2068 | differentiator 2069 | postpone 2070 | whizz 2071 | pinball 2072 | singular 2073 | sloth 2074 | km 2075 | extort 2076 | dumbest 2077 | bellyaching 2078 | weill 2079 | derogation 2080 | poss 2081 | nonprofit 2082 | foundry 2083 | coiled 2084 | spoke 2085 | borrows 2086 | infant 2087 | tar 2088 | barter 2089 | sill 2090 | pussyfoot 2091 | variability 2092 | vaulter 2093 | jog 2094 | eying 2095 | boldly 2096 | unimpressed 2097 | starving 2098 | culprit 2099 | overwhelmingly 2100 | fremont 2101 | muddle 2102 | neva 2103 | owlt 2104 | reiteration 2105 | swinging 2106 | lava 2107 | bender 2108 | fallen 2109 | routine 2110 | leon 2111 | catherine 2112 | stub 2113 | bergs 2114 | ira 2115 | perjury 2116 | contender 2117 | impervious 2118 | pronounce 2119 | exploration 2120 | nailed 2121 | rebut 2122 | banzai 2123 | unpopularity 2124 | surpasses 2125 | directed 2126 | juts 2127 | trampoline 2128 | insulin 2129 | crumbs 2130 | goosebumps 2131 | coupe 2132 | encompass 2133 | gyration 2134 | repositioning 2135 | overtaken 2136 | thorough 2137 | navy 2138 | odins 2139 | br 2140 | mammoth 2141 | letdown 2142 | conversion 2143 | penthouse 2144 | turnout 2145 | banish 2146 | unwittingly 2147 | whiner 2148 | artifice 2149 | redefine 2150 | attached 2151 | cert 2152 | gumption 2153 | slab 2154 | profitably 2155 | canvass 2156 | dynamo 2157 | impasse 2158 | yellowstone 2159 | shortcoming 2160 | neural 2161 | mascot 2162 | squat 2163 | incurious 2164 | slog 2165 | munch 2166 | antsy 2167 | midwestern 2168 | precedence 2169 | swag 2170 | hedger 2171 | underline 2172 | personalize 2173 | il 2174 | shithead 2175 | persuade 2176 | apollo 2177 | embed 2178 | unleashes 2179 | sync 2180 | sentient 2181 | scams 2182 | aura 2183 | behove 2184 | fizz 2185 | olive 2186 | succession 2187 | darwin 2188 | minimization 2189 | unequivocally 2190 | olden 2191 | desktops 2192 | tariffs 2193 | fledged 2194 | humans 2195 | impropriety 2196 | caterpillar 2197 | prioritize 2198 | litany 2199 | climber 2200 | edgeless 2201 | ordering 2202 | vowel 2203 | lollipop 2204 | psi 2205 | advisory 2206 | smoothes 2207 | quartz 2208 | adage 2209 | collaboration 2210 | flats 2211 | coattail 2212 | wrench 2213 | credenza 2214 | flirtation 2215 | miami 2216 | termer 2217 | morone 2218 | seminar 2219 | menstrual 2220 | extremism 2221 | propels 2222 | gambit 2223 | county 2224 | diesel 2225 | absorbs 2226 | vindication 2227 | bipolar 2228 | quadrant 2229 | unexpectedly 2230 | diddly 2231 | restful 2232 | budding 2233 | shudder 2234 | tricked 2235 | brokering 2236 | earphone 2237 | molehill 2238 | nothingness 2239 | rodgers 2240 | holler 2241 | louisiana 2242 | mitigation 2243 | tempting 2244 | markup 2245 | reiterated 2246 | launcher 2247 | participating 2248 | optimist 2249 | validity 2250 | reboots 2251 | nitrogen 2252 | roadrunner 2253 | relay 2254 | wayne 2255 | margarita 2256 | grossing 2257 | ruler 2258 | scatter 2259 | arsenal 2260 | fisa 2261 | unrestricted 2262 | entail 2263 | prototype 2264 | intros 2265 | tamed 2266 | zucchini 2267 | harsh 2268 | gayness 2269 | augmentation 2270 | tolkien 2271 | reaffirms 2272 | hogwash 2273 | pancakes 2274 | dastardly 2275 | unseen 2276 | swedish 2277 | emmy 2278 | ruby 2279 | construction 2280 | naval 2281 | yearn 2282 | loudmouth 2283 | cherished 2284 | biblical 2285 | saint 2286 | wobble 2287 | oasis 2288 | skanky 2289 | schiller 2290 | internalize 2291 | poops 2292 | aggravating 2293 | offend 2294 | balling 2295 | fusion 2296 | congratulate 2297 | adjusted 2298 | sparkle 2299 | tern 2300 | nobility 2301 | evenly 2302 | gpo 2303 | deletes 2304 | banked 2305 | uprise 2306 | undefined 2307 | decimates 2308 | description 2309 | autograph 2310 | seventh 2311 | flagged 2312 | recode 2313 | apocalyptic 2314 | exemplifies 2315 | kirk 2316 | matchmaker 2317 | traps 2318 | illusive 2319 | chichi 2320 | imperfection 2321 | treachery 2322 | dining 2323 | realism 2324 | undeniably 2325 | spilt 2326 | chipper 2327 | seel 2328 | shampoo 2329 | morality 2330 | warmth 2331 | peaking 2332 | thematic 2333 | hun 2334 | scalps 2335 | heals 2336 | expediently 2337 | swimmer 2338 | stasis 2339 | southeast 2340 | capsule 2341 | coolness 2342 | enhances 2343 | quicksilver 2344 | placid 2345 | indonesian 2346 | glob 2347 | keynote 2348 | nook 2349 | thrashed 2350 | patriotism 2351 | mellow 2352 | populous 2353 | wholly 2354 | prefers 2355 | penis 2356 | bits 2357 | utilizing 2358 | ruckus 2359 | trinity 2360 | frosty 2361 | divideds 2362 | freight 2363 | erect 2364 | melodramatic 2365 | fired 2366 | may 2367 | verification 2368 | tamped 2369 | allegiance 2370 | stairway 2371 | painting 2372 | cooper 2373 | warpath 2374 | ca 2375 | cryptography 2376 | focal 2377 | calibration 2378 | vi 2379 | outweigh 2380 | wildness 2381 | lobe 2382 | gamer 2383 | hut 2384 | fractional 2385 | envision 2386 | egotistical 2387 | encore 2388 | squander 2389 | trodden 2390 | scouter 2391 | mole 2392 | artillery 2393 | novelty 2394 | chimney 2395 | kura 2396 | marketer 2397 | strategically 2398 | bailing 2399 | repercussion 2400 | dysfunctional 2401 | sensationalism 2402 | beatles 2403 | peripheral 2404 | idaho 2405 | lamp 2406 | unsuccessful 2407 | elli 2408 | ignores 2409 | placate 2410 | acer 2411 | restless 2412 | unfounded 2413 | smallish 2414 | flaunt 2415 | thermal 2416 | mysterious 2417 | faro 2418 | commercialization 2419 | oldie 2420 | integrates 2421 | unity 2422 | geezer 2423 | hos 2424 | serially 2425 | blew 2426 | barbarian 2427 | allergic 2428 | spectrum 2429 | backdrop 2430 | megawatt 2431 | cardiac 2432 | mime 2433 | weeklies 2434 | ensnare 2435 | carpal 2436 | wallace 2437 | volta 2438 | grin 2439 | constellation 2440 | ramification 2441 | concede 2442 | rural 2443 | gentle 2444 | backpack 2445 | denies 2446 | glint 2447 | bottoming 2448 | mainspring 2449 | jest 2450 | misprint 2451 | corvette 2452 | headgear 2453 | torment 2454 | inseparable 2455 | thump 2456 | decades 2457 | disillusionment 2458 | rotational 2459 | alum 2460 | microcephaly 2461 | uv 2462 | creditor 2463 | transferrable 2464 | indefinitely 2465 | comprehensively 2466 | neurological 2467 | iwo 2468 | conservationist 2469 | skillful 2470 | unman 2471 | chairwoman 2472 | moaner 2473 | rushing 2474 | shirk 2475 | predation 2476 | dotard 2477 | elder 2478 | trowel 2479 | bbl 2480 | raper 2481 | handout 2482 | onward 2483 | brs 2484 | reused 2485 | thatch 2486 | unbreakable 2487 | hark 2488 | madison 2489 | unsolved 2490 | gaines 2491 | potent 2492 | copacetic 2493 | prescribing 2494 | seance 2495 | slaughters 2496 | pudding 2497 | nw 2498 | dole 2499 | stark 2500 | overact 2501 | convenience 2502 | throwing 2503 | serum 2504 | passage 2505 | madams 2506 | disrupts 2507 | thrash 2508 | preforming 2509 | realign 2510 | ix 2511 | babysitter 2512 | cosmos 2513 | rf 2514 | invaluable 2515 | porte 2516 | adaptive 2517 | preposterous 2518 | objectively 2519 | wile 2520 | cottage 2521 | photography 2522 | raised 2523 | friction 2524 | arse 2525 | waveguide 2526 | grammatical 2527 | combing 2528 | sitting 2529 | axis 2530 | bloom 2531 | goosebump 2532 | spooked 2533 | verdict 2534 | qualifies 2535 | tweed 2536 | noteworthy 2537 | resurgence 2538 | soundest 2539 | magenta 2540 | duration 2541 | comstock 2542 | xerox 2543 | madly 2544 | accumulating 2545 | brownout 2546 | compile 2547 | tigress 2548 | cooperation 2549 | purposefully 2550 | stadiums 2551 | eligible 2552 | hudson 2553 | jr. 2554 | loin 2555 | villa 2556 | clinical 2557 | syrian 2558 | smashing 2559 | boorish 2560 | newcomer 2561 | ambiguity 2562 | brilliance 2563 | dense 2564 | dimension 2565 | recluse 2566 | surplus 2567 | wowed 2568 | rickshaw 2569 | operator 2570 | comet 2571 | thorn 2572 | penchant 2573 | sabre 2574 | shouting 2575 | eyes 2576 | biotechnology 2577 | mace 2578 | gds 2579 | pools 2580 | dedicated 2581 | relentlessly 2582 | reissue 2583 | anoint 2584 | hatter 2585 | artful 2586 | iga 2587 | macs 2588 | backward 2589 | suitor 2590 | faraday 2591 | frothing 2592 | peculiar 2593 | whoring 2594 | procedure 2595 | spoilage 2596 | decelerates 2597 | northland 2598 | unbeknownst 2599 | steadfastly 2600 | carnegie 2601 | flaging 2602 | dynamics 2603 | disrupting 2604 | minuet 2605 | pager 2606 | scotch 2607 | auburn 2608 | baking 2609 | briskly 2610 | accent 2611 | smallest 2612 | disappears 2613 | barrage 2614 | factoid 2615 | speedy 2616 | flawlessly 2617 | flatiron 2618 | solidified 2619 | melting 2620 | fuels 2621 | leprechaun 2622 | arduous 2623 | forklifts 2624 | sapphire 2625 | tenderloin 2626 | glucose 2627 | soothe 2628 | torrid 2629 | tenfold 2630 | ellison 2631 | goer 2632 | neatly 2633 | demented 2634 | breakouts 2635 | alb 2636 | malcontent 2637 | bartlett 2638 | rips 2639 | dci 2640 | booby 2641 | expunge 2642 | strives 2643 | sydney 2644 | effectiveness 2645 | unbridled 2646 | tendency 2647 | straighten 2648 | autographed 2649 | cm 2650 | alight 2651 | eyeglasses 2652 | wtv 2653 | flywheel 2654 | bonuses 2655 | misconstrue 2656 | limitation 2657 | negotiable 2658 | forgets 2659 | striving 2660 | connectivity 2661 | limelight 2662 | stash 2663 | anonymously 2664 | confetti 2665 | spiked 2666 | connecting 2667 | wellbeing 2668 | corroboration 2669 | arrivederci 2670 | surging 2671 | ran 2672 | pyjama 2673 | forklift 2674 | mag 2675 | oft 2676 | thwart 2677 | watermelon 2678 | gecko 2679 | enchilada 2680 | oaks 2681 | stadium 2682 | transmission 2683 | discovered 2684 | hanukkah 2685 | revenues 2686 | slander 2687 | iliad 2688 | panacea 2689 | predictor 2690 | flattery 2691 | expertise 2692 | discomfort 2693 | sensationalistic 2694 | imagining 2695 | untapped 2696 | watchers 2697 | rotting 2698 | mild 2699 | laguna 2700 | springboard 2701 | painkiller 2702 | sours 2703 | unappreciated 2704 | homeostasis 2705 | branding 2706 | skybox 2707 | ethos 2708 | gusto 2709 | fulfilling 2710 | tailor 2711 | versatile 2712 | dreamt 2713 | irk 2714 | insure 2715 | spontaneously 2716 | generated 2717 | folly 2718 | phosphorescent 2719 | barring 2720 | joshua 2721 | temper 2722 | brasil 2723 | exceptional 2724 | capitalizes 2725 | unintentionally 2726 | grail 2727 | tb 2728 | readout 2729 | perplex 2730 | squirrels 2731 | outranks 2732 | realist 2733 | labs 2734 | affirmation 2735 | slogging 2736 | volar 2737 | jag 2738 | myrrh 2739 | chang 2740 | schtick 2741 | minuscule 2742 | colbert 2743 | considerable 2744 | respected 2745 | xxxiv 2746 | uniquely 2747 | folded 2748 | sourcing 2749 | saddle 2750 | pathetically 2751 | imitative 2752 | bedroom 2753 | identified 2754 | gourmet 2755 | unskilled 2756 | tantamount 2757 | tithing 2758 | scarce 2759 | approves 2760 | recede 2761 | personnel 2762 | reception 2763 | bikini 2764 | ds 2765 | breezy 2766 | infomercial 2767 | intellectually 2768 | admiration 2769 | sirius 2770 | manifest 2771 | intellect 2772 | normalcy 2773 | analytic 2774 | asserts 2775 | harry 2776 | toad 2777 | slather 2778 | drake 2779 | hydrant 2780 | speechlessness 2781 | proportionately 2782 | eternity 2783 | iww 2784 | baseless 2785 | oat 2786 | cancellation 2787 | ibuprofen 2788 | embryo 2789 | contra 2790 | enumerate 2791 | reallocation 2792 | counteract 2793 | lion 2794 | preservation 2795 | passenger 2796 | specious 2797 | disseminate 2798 | gallop 2799 | improperly 2800 | unjust 2801 | sparse 2802 | beed 2803 | quiver 2804 | woody 2805 | snippet 2806 | inhale 2807 | reported 2808 | bonobo 2809 | psychologist 2810 | accelerator 2811 | lancaster 2812 | listener 2813 | sized 2814 | australian 2815 | pap 2816 | envious 2817 | expands 2818 | reef 2819 | bengal 2820 | ranger 2821 | gss 2822 | wharton 2823 | electromechanical 2824 | treaty 2825 | grudge 2826 | logos 2827 | perpetually 2828 | nod 2829 | napa 2830 | pinning 2831 | industrial 2832 | heartland 2833 | ranting 2834 | jensen 2835 | stoner 2836 | launces 2837 | cube 2838 | cranberry 2839 | capitalizing 2840 | lessor 2841 | ecstasy 2842 | excellently 2843 | cargo 2844 | healthily 2845 | doorman 2846 | gasket 2847 | unborn 2848 | zeus 2849 | slurps 2850 | hoist 2851 | vac 2852 | sharper 2853 | readouts 2854 | bop 2855 | occuring 2856 | appealing 2857 | facet 2858 | deregulate 2859 | insist 2860 | invents 2861 | dermatology 2862 | flashy 2863 | trusted 2864 | irak 2865 | assemble 2866 | landrover 2867 | passionate 2868 | goad 2869 | gusty 2870 | pinky 2871 | auspicious 2872 | workshop 2873 | grazier 2874 | pedigree 2875 | clarence 2876 | rarified 2877 | strawberry 2878 | flexed 2879 | trespass 2880 | snappy 2881 | characterize 2882 | opera 2883 | pussy 2884 | esteem 2885 | evanescence 2886 | scottish 2887 | tutorial 2888 | threatening 2889 | rebuild 2890 | pundits 2891 | unveiling 2892 | prowess 2893 | unwitting 2894 | frisky 2895 | kin 2896 | cola 2897 | saudis 2898 | optimal 2899 | delighted 2900 | norm 2901 | honkey 2902 | allusion 2903 | adversity 2904 | overwhelms 2905 | wu 2906 | tropical 2907 | subcommittee 2908 | dimensional 2909 | manner 2910 | evildoer 2911 | peaky 2912 | droping 2913 | blurb 2914 | daylight 2915 | overwhelming 2916 | tink 2917 | cryptographic 2918 | basting 2919 | ply 2920 | capitalized 2921 | hearst 2922 | depressant 2923 | duo 2924 | allure 2925 | manipulating 2926 | wool 2927 | singularly 2928 | witherspoon 2929 | snoopy 2930 | resent 2931 | pontificating 2932 | goons 2933 | spa 2934 | vietnamese 2935 | stroke 2936 | frazer 2937 | hotter 2938 | fueling 2939 | haymaker 2940 | precipice 2941 | vantage 2942 | asker 2943 | letsssss 2944 | trackable 2945 | pxs 2946 | scribble 2947 | gradually 2948 | upstairs 2949 | unusually 2950 | redness 2951 | personas 2952 | hazy 2953 | audacity 2954 | tramline 2955 | inclined 2956 | marvel 2957 | daddys 2958 | salon 2959 | booming 2960 | cycling 2961 | conquest 2962 | showcase 2963 | cruising 2964 | devious 2965 | generic 2966 | encroach 2967 | calms 2968 | chine 2969 | dss 2970 | settles 2971 | groove 2972 | meandering 2973 | preventative 2974 | snorkeling 2975 | wheeler 2976 | originate 2977 | hap 2978 | bellybutton 2979 | unlocks 2980 | uca 2981 | blather 2982 | overreacted 2983 | bastardize 2984 | gallon 2985 | falla 2986 | petition 2987 | diddle 2988 | tripping 2989 | sw 2990 | biscuit 2991 | bonanza 2992 | stingy 2993 | tody 2994 | fender 2995 | dune 2996 | terminate 2997 | maestro 2998 | footage 2999 | logan 3000 | manual 3001 | violet 3002 | concur 3003 | escalator 3004 | dismissal 3005 | onetimer 3006 | weakling 3007 | guesser 3008 | container 3009 | cus 3010 | sixty 3011 | integrated 3012 | flipper 3013 | improbable 3014 | ridge 3015 | crapshoot 3016 | disgruntled 3017 | bestow 3018 | recalculate 3019 | bark 3020 | created 3021 | clamoring 3022 | dame 3023 | enclose 3024 | bleeds 3025 | collier 3026 | rustle 3027 | stride 3028 | affiliate 3029 | disparity 3030 | mop 3031 | banger 3032 | platforms 3033 | whiskey 3034 | unintended 3035 | robbed 3036 | verbal 3037 | blonde 3038 | maria 3039 | benefitting 3040 | nursing 3041 | succulent 3042 | logistical 3043 | trio 3044 | scrolling 3045 | cultivate 3046 | controller 3047 | verbose 3048 | rematch 3049 | synthesis 3050 | upmarket 3051 | steller 3052 | maple 3053 | bypass 3054 | equivalence 3055 | ala 3056 | wristwatch 3057 | dishearten 3058 | autopilot 3059 | brighter 3060 | tas 3061 | surgeon 3062 | issuer 3063 | visually 3064 | unchain 3065 | lick 3066 | nicotine 3067 | mocking 3068 | hitch 3069 | seafood 3070 | idealistic 3071 | loopholes 3072 | boilerplate 3073 | lowball 3074 | pouring 3075 | shiner 3076 | unreachable 3077 | rome 3078 | sledge 3079 | gam 3080 | gram 3081 | dramamine 3082 | pst 3083 | tycoon 3084 | cape 3085 | enormity 3086 | inferno 3087 | championship 3088 | lighter 3089 | displace 3090 | wanker 3091 | exhale 3092 | oodles 3093 | presenter 3094 | detractor 3095 | idiosyncratic 3096 | yuma 3097 | shade 3098 | blowback 3099 | therapy 3100 | correspondent 3101 | fraternal 3102 | sponge 3103 | disingenuous 3104 | travesty 3105 | lem 3106 | watering 3107 | bolster 3108 | bronco 3109 | eros 3110 | meander 3111 | implicit 3112 | stayed 3113 | ramped 3114 | pesky 3115 | mechanization 3116 | haifa 3117 | decimal 3118 | liven 3119 | scarcity 3120 | trucking 3121 | underrate 3122 | grad 3123 | employer 3124 | patten 3125 | flourishing 3126 | devastating 3127 | mule 3128 | nears 3129 | followers 3130 | pragmatic 3131 | rebate 3132 | emitter 3133 | cinch 3134 | christmastime 3135 | fortress 3136 | discard 3137 | overachiever 3138 | firewall 3139 | snapshot 3140 | anus 3141 | hao 3142 | soak 3143 | tat 3144 | considerably 3145 | igniting 3146 | inventor 3147 | pulverize 3148 | tender 3149 | scripted 3150 | infotainment 3151 | fdr 3152 | merges 3153 | kappa 3154 | symbols 3155 | monet 3156 | moderation 3157 | floater 3158 | nurturing 3159 | beep 3160 | thai 3161 | unsung 3162 | spanner 3163 | tester 3164 | nc 3165 | unforgettable 3166 | ssw 3167 | tragic 3168 | listens 3169 | shuts 3170 | unrelated 3171 | ostrich 3172 | overslept 3173 | inventive 3174 | watermark 3175 | preheat 3176 | leto 3177 | craziness 3178 | internally 3179 | ken 3180 | crave 3181 | impatience 3182 | footstep 3183 | ashton 3184 | sealed 3185 | wether 3186 | alias 3187 | temple 3188 | pto 3189 | fowler 3190 | machinery 3191 | unreliable 3192 | siting 3193 | sunroof 3194 | tirelessly 3195 | firestorm 3196 | trustworthiness 3197 | firmware 3198 | cope 3199 | domestically 3200 | warming 3201 | wearables 3202 | negativism 3203 | olympus 3204 | callback 3205 | variation 3206 | slake 3207 | galloway 3208 | einsteinium 3209 | crasher 3210 | calorie 3211 | dramatize 3212 | landlord 3213 | stevens 3214 | moan 3215 | ceremonial 3216 | utopian 3217 | prophetic 3218 | symptomatic 3219 | neglect 3220 | greets 3221 | introducing 3222 | query 3223 | hopkins 3224 | bygone 3225 | aum 3226 | flexibility 3227 | liftoff 3228 | impersonation 3229 | godown 3230 | cuppa 3231 | commercialize 3232 | thrives 3233 | squeezed 3234 | creamery 3235 | ion 3236 | aforementioned 3237 | cleaning 3238 | headroom 3239 | stitch 3240 | battleground 3241 | benghazi 3242 | pwr 3243 | amds 3244 | arrives 3245 | hubs 3246 | dedicate 3247 | zit 3248 | recur 3249 | physalis 3250 | prometheus 3251 | bw 3252 | looney 3253 | mongering 3254 | redshift 3255 | bluffing 3256 | wold 3257 | yammer 3258 | overboard 3259 | tay 3260 | oklahoma 3261 | unruly 3262 | accordance 3263 | echos 3264 | unsettle 3265 | jailbreak 3266 | shitless 3267 | berlin 3268 | denying 3269 | clobbered 3270 | precision 3271 | barnacle 3272 | scorecard 3273 | bey 3274 | morphing 3275 | tattoo 3276 | specification 3277 | token 3278 | resolutely 3279 | someway 3280 | buffets 3281 | chromosome 3282 | spock 3283 | plagiarize 3284 | disorganized 3285 | dolby 3286 | anomaly 3287 | cls 3288 | uncharacteristically 3289 | soulless 3290 | computes 3291 | hotly 3292 | stockpile 3293 | siphon 3294 | despondency 3295 | maintains 3296 | optic 3297 | learned 3298 | lucas 3299 | naturally 3300 | quench 3301 | pumpkin 3302 | soccer 3303 | talker 3304 | mulligan 3305 | lawn 3306 | huddle 3307 | defiantly 3308 | commentator 3309 | adjunct 3310 | dominating 3311 | dishevel 3312 | caveman 3313 | raider 3314 | nipping 3315 | outdoors 3316 | loath 3317 | fraudulently 3318 | optometrist 3319 | insatiable 3320 | flaring 3321 | stonewall 3322 | dominican 3323 | commendable 3324 | indoor 3325 | windfall 3326 | dildos 3327 | sideway 3328 | nitrous 3329 | prominence 3330 | aloe 3331 | permeate 3332 | kris 3333 | tribal 3334 | kent 3335 | voodoo 3336 | purse 3337 | bookseller 3338 | comeing 3339 | reinvigorate 3340 | demystify 3341 | gulp 3342 | whisky 3343 | perch 3344 | amnesia 3345 | rime 3346 | les 3347 | oleds 3348 | perimeter 3349 | airwave 3350 | dwarfing 3351 | insured 3352 | budging 3353 | sends 3354 | delisting 3355 | respective 3356 | fathom 3357 | crest 3358 | carriage 3359 | jerky 3360 | deceiving 3361 | calmness 3362 | weaklings 3363 | misplace 3364 | accustom 3365 | contemporary 3366 | sherman 3367 | mst 3368 | timeless 3369 | hangouts 3370 | attractively 3371 | tomahawk 3372 | kkk 3373 | sprint 3374 | motivational 3375 | rebellious 3376 | heist 3377 | purebred 3378 | watchful 3379 | shack 3380 | wondrous 3381 | deride 3382 | shipbuilder 3383 | nightly 3384 | doubters 3385 | pharmacist 3386 | sneakily 3387 | argus 3388 | medicaid 3389 | fangs 3390 | deflated 3391 | buoy 3392 | transitory 3393 | tally 3394 | specialize 3395 | substance 3396 | thomson 3397 | adobe 3398 | irrespective 3399 | leonardo 3400 | liars 3401 | disa 3402 | brighten 3403 | walled 3404 | freudian 3405 | harness 3406 | overrun 3407 | reclaims 3408 | sensational 3409 | therein 3410 | plethora 3411 | nearsighted 3412 | volcano 3413 | overestimate 3414 | macroeconomics 3415 | splashy 3416 | butts 3417 | maul 3418 | abstract 3419 | initiated 3420 | mhz 3421 | podium 3422 | qi 3423 | undisputed 3424 | dearly 3425 | wafer 3426 | chasm 3427 | fossils 3428 | notoriety 3429 | undiscovered 3430 | maximise 3431 | tricker 3432 | overreact 3433 | inconsistent 3434 | tedious 3435 | unverified 3436 | leaked 3437 | coarse 3438 | hospital 3439 | dropout 3440 | receiver 3441 | meditate 3442 | niagara 3443 | haptic 3444 | carousel 3445 | reiterates 3446 | gratitude 3447 | redeploy 3448 | bricks 3449 | backrest 3450 | adjudge 3451 | adherence 3452 | serenity 3453 | kindergarten 3454 | hesitant 3455 | macros 3456 | carburetor 3457 | ungrateful 3458 | va 3459 | cedar 3460 | tailwinds 3461 | perceived 3462 | laugher 3463 | stability 3464 | slider 3465 | tactician 3466 | founding 3467 | alarmist 3468 | mastodon 3469 | adulterer 3470 | agee 3471 | gesture 3472 | uncoil 3473 | greenery 3474 | apiece 3475 | trailing 3476 | propagandist 3477 | lived 3478 | delivering 3479 | sickening 3480 | vegetable 3481 | blossom 3482 | blitz 3483 | dossier 3484 | complainer 3485 | whopper 3486 | suppressed 3487 | exhibitionist 3488 | eurasia 3489 | tradition 3490 | purchasing 3491 | denigrate 3492 | plebeian 3493 | enhanced 3494 | arsehole 3495 | ichor 3496 | jeopardize 3497 | skyes 3498 | driveway 3499 | torturous 3500 | bondholder 3501 | proving 3502 | revolutionize 3503 | blade 3504 | ops 3505 | riviera 3506 | compilation 3507 | causing 3508 | sherlock 3509 | jingle 3510 | slightest 3511 | smartass 3512 | whiz 3513 | ratting 3514 | resell 3515 | embedded 3516 | nuance 3517 | biometrics 3518 | furnish 3519 | ra 3520 | sentinel 3521 | broadly 3522 | wishfully 3523 | maze 3524 | prognosticator 3525 | sufficiency 3526 | robust 3527 | millennium 3528 | decease 3529 | gifting 3530 | thrown 3531 | entirety 3532 | screwy 3533 | memorize 3534 | rebuffed 3535 | infringed 3536 | tickled 3537 | bowlful 3538 | wade 3539 | repent 3540 | saber 3541 | pleasantry 3542 | overspend 3543 | borrowing 3544 | dissolving 3545 | sought 3546 | attune 3547 | formality 3548 | sabbath 3549 | named 3550 | muskrat 3551 | doubted 3552 | legitimize 3553 | finishing 3554 | tang 3555 | jersey 3556 | calla 3557 | slouch 3558 | whiney 3559 | morton 3560 | afterthought 3561 | prowl 3562 | predicting 3563 | retry 3564 | checked 3565 | millionaires 3566 | waite 3567 | hereby 3568 | fug 3569 | beaten 3570 | cognizant 3571 | expires 3572 | illustrate 3573 | duds 3574 | spends 3575 | analyzed 3576 | washer 3577 | rear 3578 | realization 3579 | innovated 3580 | exhausting 3581 | inventing 3582 | tweeting 3583 | goed 3584 | dictatorial 3585 | licencing 3586 | rapper 3587 | haywire 3588 | solver 3589 | thunderstorm 3590 | stench 3591 | extending 3592 | showcases 3593 | iou 3594 | encrypted 3595 | atom 3596 | secures 3597 | pry 3598 | habitual 3599 | reignite 3600 | redact 3601 | hotdogs 3602 | imprint 3603 | sls 3604 | matte 3605 | courtroom 3606 | committed 3607 | entrench 3608 | embolden 3609 | trig 3610 | automotive 3611 | mn 3612 | wired 3613 | mechanism 3614 | kat 3615 | deferent 3616 | rightful 3617 | levi 3618 | scavenger 3619 | handheld 3620 | plays 3621 | bap 3622 | flank 3623 | royalist 3624 | autumn 3625 | exhausted 3626 | kansas 3627 | firepower 3628 | notched 3629 | determines 3630 | overtook 3631 | disproving 3632 | patron 3633 | presto 3634 | stagnates 3635 | led 3636 | roundup 3637 | tray 3638 | builder 3639 | dandy 3640 | opportunistic 3641 | gigabyte 3642 | honorably 3643 | unfilled 3644 | traumatise 3645 | vienna 3646 | hitter 3647 | streets 3648 | ramping 3649 | tinker 3650 | sandbagger 3651 | emerges 3652 | determination 3653 | consistency 3654 | doge 3655 | decreased 3656 | shelley 3657 | navigate 3658 | grasp 3659 | mx 3660 | edison 3661 | endear 3662 | atp 3663 | monthlies 3664 | admirable 3665 | introspective 3666 | detour 3667 | rs 3668 | fiber 3669 | bamboozle 3670 | ms. 3671 | synchronize 3672 | insemination 3673 | magician 3674 | sevens 3675 | temperature 3676 | gaping 3677 | complex 3678 | llama 3679 | diana 3680 | enriches 3681 | mainline 3682 | greatness 3683 | eventful 3684 | refreshing 3685 | journalism 3686 | backroom 3687 | needy 3688 | fluid 3689 | implore 3690 | obliteration 3691 | unplugged 3692 | beefcake 3693 | vicarious 3694 | epilepsy 3695 | savvy 3696 | lockout 3697 | elude 3698 | cone 3699 | reverberation 3700 | lottos 3701 | lecherous 3702 | brad 3703 | dystopia 3704 | toda 3705 | quicken 3706 | wpm 3707 | bombshell 3708 | grabbed 3709 | unused 3710 | admittance 3711 | shtik 3712 | chug 3713 | underperformed 3714 | diplomacy 3715 | residence 3716 | inti 3717 | toledo 3718 | np 3719 | compromised 3720 | dail 3721 | flipping 3722 | loved 3723 | champ 3724 | orifice 3725 | visualize 3726 | infers 3727 | asss 3728 | sublime 3729 | oslo 3730 | playbook 3731 | murderous 3732 | takeoff 3733 | showboat 3734 | seeping 3735 | greenland 3736 | simpleton 3737 | eyeing 3738 | cooling 3739 | dunkirk 3740 | pessimism 3741 | torture 3742 | morse 3743 | playable 3744 | preacher 3745 | speedboat 3746 | feds 3747 | tucker 3748 | pudge 3749 | thoroughbred 3750 | exclusively 3751 | goggle 3752 | kneeling 3753 | toying 3754 | muffin 3755 | languish 3756 | refueling 3757 | incumbent 3758 | cloning 3759 | divine 3760 | multimedia 3761 | changeover 3762 | lime 3763 | climbs 3764 | digested 3765 | cameo 3766 | validate 3767 | analogous 3768 | dumasses 3769 | solving 3770 | davis 3771 | rudderless 3772 | compact 3773 | strive 3774 | prestigious 3775 | potter 3776 | greenwich 3777 | corroborate 3778 | foothold 3779 | justifiable 3780 | spreadsheet 3781 | melody 3782 | developers 3783 | insofar 3784 | stepping 3785 | imam 3786 | sexier 3787 | pricker 3788 | fess 3789 | christie 3790 | dangling 3791 | gypsy 3792 | happier 3793 | jettison 3794 | consumerism 3795 | fray 3796 | earmark 3797 | texture 3798 | interviewer 3799 | moderator 3800 | photographic 3801 | snaps 3802 | landscape 3803 | manhunt 3804 | formulaic 3805 | formerly 3806 | percolate 3807 | desensitize 3808 | basal 3809 | orb 3810 | ekg 3811 | knight 3812 | tapping 3813 | peon 3814 | stimulate 3815 | animated 3816 | spreads 3817 | chartists 3818 | bulletin 3819 | aftermath 3820 | whistler 3821 | splitsville 3822 | precedent 3823 | solos 3824 | groom 3825 | flourish 3826 | sagacious 3827 | broadband 3828 | seep 3829 | efficacy 3830 | kidney 3831 | jerks 3832 | zulu 3833 | ribbon 3834 | continuity 3835 | beleaguer 3836 | touchdown 3837 | plural 3838 | bottomed 3839 | buddhist 3840 | roaming 3841 | reindeer 3842 | austria 3843 | mfa 3844 | spews 3845 | vindictive 3846 | individually 3847 | editorialize 3848 | conformation 3849 | rallys 3850 | empirical 3851 | rhetorical 3852 | resting 3853 | caters 3854 | fertilizer 3855 | paige 3856 | peeps 3857 | cement 3858 | integral 3859 | narrowed 3860 | ascent 3861 | masa 3862 | racially 3863 | bulldozer 3864 | reactionary 3865 | preform 3866 | aroused 3867 | pears 3868 | diagram 3869 | deliberation 3870 | ki 3871 | counterpoint 3872 | benign 3873 | surmount 3874 | sourced 3875 | anguish 3876 | doings 3877 | crotchety 3878 | cheapness 3879 | jude 3880 | sceptic 3881 | dreams 3882 | economical 3883 | chosen 3884 | retired 3885 | undershirt 3886 | parse 3887 | sponsorship 3888 | playpen 3889 | stinker 3890 | annotate 3891 | complicated 3892 | trailblazer 3893 | leaps 3894 | razzle 3895 | bruin 3896 | unperturbed 3897 | mimics 3898 | iffy 3899 | bigotry 3900 | sprouts 3901 | fixed 3902 | unbroken 3903 | prefect 3904 | backside 3905 | circulation 3906 | deliberately 3907 | blabbering 3908 | mamma 3909 | accommodate 3910 | performs 3911 | howdy 3912 | incubator 3913 | resuscitate 3914 | doggy 3915 | guideline 3916 | ful 3917 | riser 3918 | haggle 3919 | kt 3920 | slate 3921 | workhorse 3922 | tightest 3923 | activism 3924 | costa 3925 | smother 3926 | kitchen 3927 | bulls 3928 | tarot 3929 | decapitation 3930 | lasting 3931 | jinxing 3932 | apache 3933 | baffling 3934 | spoken 3935 | diminishes 3936 | pane 3937 | derange 3938 | jittery 3939 | experimental 3940 | concession 3941 | undisclosed 3942 | firelighter 3943 | unsettled 3944 | aaa 3945 | siris 3946 | invitation 3947 | cleveland 3948 | constrained 3949 | mussolini 3950 | decker 3951 | consortium 3952 | ahem 3953 | environmentally 3954 | civic 3955 | rewarding 3956 | consolidated 3957 | licence 3958 | maniacs 3959 | picked 3960 | staged 3961 | dweller 3962 | shun 3963 | fervor 3964 | digits 3965 | lincoln 3966 | tamp 3967 | dating 3968 | substitute 3969 | nabob 3970 | sensing 3971 | weeklys 3972 | globalization 3973 | retained 3974 | leveraging 3975 | bluring 3976 | schizoid 3977 | ese 3978 | authenticity 3979 | agile 3980 | cockroach 3981 | suing 3982 | retroactive 3983 | pixy 3984 | kleenex 3985 | exclusion 3986 | eyebrows 3987 | averaging 3988 | outpaces 3989 | homophobe 3990 | mist 3991 | ionic 3992 | myriad 3993 | snob 3994 | blocked 3995 | evade 3996 | cusp 3997 | wayside 3998 | constructively 3999 | preformed 4000 | ranked 4001 | huger 4002 | toyota 4003 | agreed 4004 | supportive 4005 | fuckerssssss 4006 | pacific 4007 | deuce 4008 | mull 4009 | underrated 4010 | ponderous 4011 | predictably 4012 | fragment 4013 | titter 4014 | crackerjack 4015 | grumpy 4016 | blinder 4017 | recite 4018 | misinformation 4019 | aerospace 4020 | miscounted 4021 | bloc 4022 | bounced 4023 | biweekly 4024 | monetise 4025 | gnus 4026 | amber 4027 | alts 4028 | predicament 4029 | hippo 4030 | cleanup 4031 | overnighter 4032 | consumptive 4033 | baste 4034 | zany 4035 | started 4036 | uniformity 4037 | bollywood 4038 | powerfully 4039 | archive 4040 | morass 4041 | stabilization 4042 | piling 4043 | supercharge 4044 | impressionable 4045 | stump 4046 | deterrent 4047 | absorb 4048 | outer 4049 | dismisses 4050 | munching 4051 | springtime 4052 | misguided 4053 | kohls 4054 | refreshes 4055 | demean 4056 | backing 4057 | accumulative 4058 | demote 4059 | maniacal 4060 | undergone 4061 | chalk 4062 | tangle 4063 | geographic 4064 | lends 4065 | distributor 4066 | blinding 4067 | antidepressant 4068 | shoeshine 4069 | wac 4070 | neutrino 4071 | accretive 4072 | nevada 4073 | automaker 4074 | elected 4075 | undue 4076 | interlink 4077 | patriot 4078 | enrage 4079 | mlss 4080 | anthony 4081 | fanfare 4082 | dipstick 4083 | positively 4084 | athletic 4085 | morgen 4086 | crocodile 4087 | baboo 4088 | freeman 4089 | turner 4090 | gnu 4091 | showroom 4092 | pointer 4093 | mv 4094 | vis 4095 | ascension 4096 | hundredfold 4097 | deals 4098 | calf 4099 | cleaver 4100 | raging 4101 | drawn 4102 | medicare 4103 | canter 4104 | evs 4105 | stated 4106 | varied 4107 | velvet 4108 | ubiquitous 4109 | sat 4110 | groovy 4111 | attacker 4112 | jowl 4113 | draws 4114 | outgrow 4115 | inconvenience 4116 | allah 4117 | falcon 4118 | fitter 4119 | splurge 4120 | muddy 4121 | denver 4122 | scat 4123 | knucklehead 4124 | marquis 4125 | sine 4126 | accidental 4127 | plum 4128 | expanded 4129 | manages 4130 | refrigerator 4131 | sheriff 4132 | checker 4133 | bobcat 4134 | affirms 4135 | automaton 4136 | stood 4137 | spearhead 4138 | module 4139 | brim 4140 | genetics 4141 | documentary 4142 | mho 4143 | redneck 4144 | faa 4145 | exceeding 4146 | guying 4147 | modem 4148 | lcd 4149 | orthodoxy 4150 | neve 4151 | hometown 4152 | debating 4153 | hunger 4154 | spawn 4155 | counseling 4156 | lefts 4157 | mote 4158 | undervalues 4159 | hoot 4160 | contrast 4161 | conductor 4162 | unbearable 4163 | kidnap 4164 | axiom 4165 | shockingly 4166 | conditional 4167 | implicitly 4168 | hind 4169 | googly 4170 | silicone 4171 | empower 4172 | arca 4173 | distort 4174 | technophile 4175 | billioner 4176 | mildly 4177 | gramma 4178 | naysaying 4179 | festivity 4180 | reprimand 4181 | outbreak 4182 | rounded 4183 | parlay 4184 | melissa 4185 | atlas 4186 | seeker 4187 | provokes 4188 | trounces 4189 | describes 4190 | sorely 4191 | milking 4192 | filer 4193 | suede 4194 | pulse 4195 | unabashedly 4196 | handover 4197 | resumes 4198 | deserved 4199 | zigzag 4200 | atop 4201 | pug 4202 | regaining 4203 | fleet 4204 | vacationer 4205 | rejected 4206 | exploratory 4207 | coincidently 4208 | reassure 4209 | differentiation 4210 | adaptation 4211 | throwback 4212 | availability 4213 | despise 4214 | graf 4215 | undeveloped 4216 | inspector 4217 | skewer 4218 | exhilarate 4219 | coincidentally 4220 | coiling 4221 | cavalry 4222 | enact 4223 | liable 4224 | heavyweight 4225 | staring 4226 | wasabi 4227 | ridden 4228 | mineral 4229 | vending 4230 | nv 4231 | liquor 4232 | dvd 4233 | nlp 4234 | quandary 4235 | nostradamus 4236 | guns 4237 | prone 4238 | mccartney 4239 | fruitful 4240 | patty 4241 | appreciated 4242 | indie 4243 | expel 4244 | mils 4245 | seize 4246 | handsomely 4247 | patriotic 4248 | depressing 4249 | trouncing 4250 | repeated 4251 | heartbreaking 4252 | tx 4253 | increasing 4254 | alfalfa 4255 | richards 4256 | magnificence 4257 | provisioning 4258 | bullying 4259 | fifo 4260 | newscaster 4261 | variance 4262 | flesh 4263 | fulfill 4264 | cathouse 4265 | typewriter 4266 | wield 4267 | torpedo 4268 | prominent 4269 | cerberus 4270 | indianapolis 4271 | gearing 4272 | blessed 4273 | interdisciplinary 4274 | moths 4275 | calamari 4276 | voiceless 4277 | illuminate 4278 | venomous 4279 | twiggy 4280 | hits 4281 | fiends 4282 | sunrise 4283 | upsides 4284 | wei 4285 | poach 4286 | invented 4287 | rang 4288 | kitten 4289 | fandom 4290 | determined 4291 | colour 4292 | conjunction 4293 | deere 4294 | unquestioned 4295 | gp 4296 | divest 4297 | exposition 4298 | compass 4299 | alerted 4300 | allowance 4301 | dissipates 4302 | doomed 4303 | boxing 4304 | harbinger 4305 | hamburg 4306 | pelican 4307 | guerrilla 4308 | veggie 4309 | blesses 4310 | monk 4311 | noticed 4312 | lethargic 4313 | sweater 4314 | overjoy 4315 | ablate 4316 | jr 4317 | leash 4318 | cherish 4319 | minimalist 4320 | meir 4321 | orchestrated 4322 | nixes 4323 | outlandish 4324 | massage 4325 | perky 4326 | hanover 4327 | firstly 4328 | leapfrog 4329 | erotic 4330 | excellence 4331 | snooze 4332 | maxim 4333 | goofy 4334 | demotion 4335 | nexus 4336 | trad 4337 | encyclopedia 4338 | powering 4339 | bosc 4340 | punky 4341 | insensitive 4342 | beater 4343 | crossed 4344 | upscale 4345 | included 4346 | inspiring 4347 | breakneck 4348 | endurance 4349 | windshield 4350 | ushering 4351 | transit 4352 | shattering 4353 | piston 4354 | madam 4355 | salvation 4356 | contribution 4357 | jupiter 4358 | convulsion 4359 | instantaneously 4360 | economies 4361 | endlessly 4362 | bunny 4363 | dopey 4364 | pigeon 4365 | sen 4366 | lookup 4367 | router 4368 | coca 4369 | lucrative 4370 | digging 4371 | popgun 4372 | poking 4373 | untimely 4374 | bishop 4375 | pristine 4376 | adhere 4377 | unwarranted 4378 | snuck 4379 | bosch 4380 | hemorrhaging 4381 | upthrust 4382 | pusher 4383 | dexter 4384 | beaver 4385 | voucher 4386 | improved 4387 | erectile 4388 | dryer 4389 | heater 4390 | clawing 4391 | bitches 4392 | forcefully 4393 | hookup 4394 | spicy 4395 | scientific 4396 | boon 4397 | romp 4398 | prejudice 4399 | tidings 4400 | sd 4401 | snitch 4402 | gloss 4403 | sustains 4404 | deary 4405 | roc 4406 | yolk 4407 | coincides 4408 | dips 4409 | poem 4410 | avoiding 4411 | motivated 4412 | phoenix 4413 | kipling 4414 | mathematics 4415 | toot 4416 | bedfellow 4417 | practical 4418 | villain 4419 | trait 4420 | supplement 4421 | evolving 4422 | goblin 4423 | attempted 4424 | instrumental 4425 | hackers 4426 | geniuses 4427 | verb 4428 | osborne 4429 | catalonia 4430 | loathe 4431 | abides 4432 | subsequent 4433 | fortitude 4434 | undersea 4435 | californian 4436 | frey 4437 | proclaimed 4438 | fundamentalist 4439 | esq 4440 | transformer 4441 | pooch 4442 | solves 4443 | innovators 4444 | spigot 4445 | assimilate 4446 | mindful 4447 | discourage 4448 | ergo 4449 | redirect 4450 | bloomfield 4451 | chunky 4452 | installation 4453 | loft 4454 | shoots 4455 | minimally 4456 | renounce 4457 | breaching 4458 | cadet 4459 | soaring 4460 | extortion 4461 | vigorous 4462 | js 4463 | weirdness 4464 | sweeps 4465 | obsession 4466 | coordination 4467 | vanilla 4468 | sunburn 4469 | gringo 4470 | relish 4471 | galactic 4472 | ducking 4473 | fiancee 4474 | omit 4475 | accumulated 4476 | perfectionist 4477 | urgent 4478 | oxygen 4479 | uprooting 4480 | bulletproof 4481 | riley 4482 | privileged 4483 | simultaneously 4484 | pinching 4485 | orgasm 4486 | rested 4487 | fistful 4488 | uneventful 4489 | dauber 4490 | porterhouse 4491 | optical 4492 | commonly 4493 | moses 4494 | doubting 4495 | endeavor 4496 | scientifically 4497 | attacked 4498 | prerogative 4499 | obsessive 4500 | asps 4501 | categorize 4502 | frustrated 4503 | known 4504 | postpaid 4505 | compounding 4506 | exert 4507 | fester 4508 | paralyze 4509 | nancy 4510 | fiercely 4511 | sling 4512 | furthering 4513 | essence 4514 | lent 4515 | tully 4516 | binge 4517 | tanked 4518 | ingrates 4519 | herring 4520 | meditation 4521 | aqua 4522 | tiptoe 4523 | offended 4524 | worser 4525 | terminator 4526 | resultant 4527 | littles 4528 | continental 4529 | expend 4530 | criterion 4531 | arc 4532 | cuddly 4533 | haughty 4534 | evoke 4535 | obesity 4536 | coincidental 4537 | naps 4538 | guinness 4539 | egoistic 4540 | hypothetically 4541 | stymie 4542 | nigga 4543 | verified 4544 | cultural 4545 | monstrously 4546 | archaic 4547 | caption 4548 | calgary 4549 | astronaut 4550 | reallocate 4551 | paddy 4552 | heather 4553 | jumped 4554 | coral 4555 | ests 4556 | impressively 4557 | gargantuan 4558 | snapped 4559 | buttock 4560 | pacesetter 4561 | optional 4562 | lenient 4563 | oral 4564 | epicenter 4565 | inverter 4566 | commander 4567 | colder 4568 | daze 4569 | offside 4570 | scalded 4571 | piano 4572 | accumulator 4573 | municipal 4574 | elegantly 4575 | parental 4576 | nock 4577 | topeka 4578 | negotiator 4579 | lidar 4580 | buoyant 4581 | catastrophically 4582 | explosively 4583 | carmaker 4584 | infinitely 4585 | seb 4586 | colorful 4587 | kansa 4588 | nark 4589 | shawn 4590 | quells 4591 | tanzania 4592 | impeccable 4593 | glorify 4594 | qualified 4595 | enlarge 4596 | transference 4597 | terry 4598 | outfitted 4599 | canopy 4600 | beachfront 4601 | hep 4602 | drivel 4603 | pact 4604 | invariably 4605 | deservedly 4606 | suppress 4607 | ferry 4608 | dependable 4609 | hypnotize 4610 | users 4611 | dependency 4612 | measurement 4613 | esoteric 4614 | consists 4615 | duct 4616 | cotton 4617 | borscht 4618 | implant 4619 | accuser 4620 | earful 4621 | lego 4622 | divisive 4623 | petrify 4624 | electrify 4625 | yanker 4626 | prozac 4627 | corps 4628 | brooklyn 4629 | inaccurate 4630 | droll 4631 | institute 4632 | rediscover 4633 | groundwork 4634 | quack 4635 | paranoid 4636 | marsh 4637 | cain 4638 | aimlessly 4639 | affirmative 4640 | presumably 4641 | afterward 4642 | purchaser 4643 | raisin 4644 | uploads 4645 | lesser 4646 | unsubstantiated 4647 | lithium 4648 | dunce 4649 | ungodly 4650 | timid 4651 | concedes 4652 | hovers 4653 | nostril 4654 | idiocy 4655 | twilight 4656 | meerkat 4657 | unforeseen 4658 | philip 4659 | flummoxed 4660 | frivolous 4661 | ipv 4662 | suspiciously 4663 | impart 4664 | bask 4665 | blowjob 4666 | simulator 4667 | languishes 4668 | inexperience 4669 | savor 4670 | hurrah 4671 | tampa 4672 | cloudy 4673 | detector 4674 | seine 4675 | blok 4676 | sings 4677 | latency 4678 | tss 4679 | chant 4680 | raptor 4681 | jews 4682 | res 4683 | causation 4684 | engines 4685 | limb 4686 | salsa 4687 | diabetic 4688 | colliding 4689 | feeds 4690 | unsigned 4691 | hargreaves 4692 | vex 4693 | penn 4694 | ling 4695 | vale 4696 | paragraph 4697 | persevere 4698 | functionally 4699 | riskless 4700 | sergeant 4701 | luger 4702 | wand 4703 | hutton 4704 | herald 4705 | glug 4706 | retardation 4707 | wealthily 4708 | registry 4709 | lastly 4710 | blake 4711 | motorbike 4712 | atms 4713 | liberation 4714 | nag 4715 | paine 4716 | reshuffle 4717 | fuckers 4718 | cmb 4719 | septuagenarians 4720 | imitation 4721 | resound 4722 | wrenching 4723 | skyrocketing 4724 | socialization 4725 | subside 4726 | surgical 4727 | reflex 4728 | gravitate 4729 | merrily 4730 | sample 4731 | gaga 4732 | vet 4733 | assigns 4734 | signing 4735 | chomp 4736 | whey 4737 | innuendo 4738 | talked 4739 | bowling 4740 | wiser 4741 | tardy 4742 | extravagance 4743 | monetized 4744 | clinic 4745 | obligated 4746 | prick 4747 | lauds 4748 | zap 4749 | convoy 4750 | recreational 4751 | muffler 4752 | preface 4753 | quester 4754 | fss 4755 | radiator 4756 | balancing 4757 | directory 4758 | ponytail 4759 | reflector 4760 | disprove 4761 | authenticate 4762 | negligible 4763 | slumps 4764 | severs 4765 | delinquency 4766 | strengthens 4767 | negates 4768 | boots 4769 | advantageous 4770 | logging 4771 | steeper 4772 | cupping 4773 | lobbying 4774 | tia 4775 | moneybag 4776 | badmouth 4777 | shattered 4778 | loudly 4779 | intersecting 4780 | whit 4781 | jumps 4782 | tam 4783 | der 4784 | dcs 4785 | jab 4786 | peaceful 4787 | slays 4788 | invoice 4789 | circulates 4790 | fibs 4791 | satisfactorily 4792 | incalculable 4793 | unturned 4794 | rental 4795 | electrified 4796 | alleviates 4797 | vocational 4798 | picking 4799 | roadster 4800 | dazzle 4801 | loony 4802 | agreeable 4803 | pertains 4804 | reevaluate 4805 | auntie 4806 | galvani 4807 | twat 4808 | adopter 4809 | faucet 4810 | groaner 4811 | creatin 4812 | spokesperson 4813 | gushing 4814 | forge 4815 | repeater 4816 | zest 4817 | champaign 4818 | cumulate 4819 | tipsy 4820 | quell 4821 | pressing 4822 | craft 4823 | shrewd 4824 | lessss 4825 | mango 4826 | overkill 4827 | tyler 4828 | reps 4829 | mems 4830 | energizer 4831 | ac 4832 | soapbox 4833 | kids 4834 | truman 4835 | childhood 4836 | lear 4837 | prerecord 4838 | exude 4839 | preordain 4840 | homeland 4841 | lifting 4842 | spray 4843 | slacker 4844 | debrief 4845 | alaska 4846 | fletcher 4847 | caliber 4848 | abeyance 4849 | tenth 4850 | fixing 4851 | tempted 4852 | fords 4853 | switching 4854 | rightly 4855 | enormously 4856 | exaggerated 4857 | stanford 4858 | insightful 4859 | tools 4860 | boomerang 4861 | systematic 4862 | actin 4863 | ref 4864 | satisfactory 4865 | plumbing 4866 | disclaimer 4867 | fooling 4868 | pays 4869 | revolting 4870 | lush 4871 | toothless 4872 | customize 4873 | boston 4874 | mono 4875 | ransom 4876 | sponsored 4877 | outweighs 4878 | helicopter 4879 | commemorative 4880 | ceasing 4881 | steepens 4882 | texts 4883 | argo 4884 | ts 4885 | humor 4886 | jokingly 4887 | booked 4888 | supper 4889 | socket 4890 | consciousness 4891 | shotgun 4892 | streams 4893 | chaldean 4894 | leaker 4895 | soddy 4896 | inducting 4897 | averts 4898 | pms 4899 | deli 4900 | snss 4901 | macedonian 4902 | irresistible 4903 | saturates 4904 | despair 4905 | motive 4906 | plaid 4907 | dwarfs 4908 | centurion 4909 | infancy 4910 | shuck 4911 | icbms 4912 | brunt 4913 | tested 4914 | suggestive 4915 | dea 4916 | residual 4917 | stockton 4918 | thimble 4919 | shook 4920 | attic 4921 | poe 4922 | arctic 4923 | roasted 4924 | rhetorically 4925 | calculated 4926 | laurel 4927 | reluctant 4928 | parallels 4929 | imbecilic 4930 | commencement 4931 | whelp 4932 | foldable 4933 | typo 4934 | prawn 4935 | spun 4936 | reprehensible 4937 | flunky 4938 | jockey 4939 | ringside 4940 | santos 4941 | nitwit 4942 | falsifying 4943 | interstellar 4944 | hershey 4945 | minnow 4946 | venus 4947 | albert 4948 | screening 4949 | mastiff 4950 | hem 4951 | setups 4952 | juju 4953 | competitiveness 4954 | superlative 4955 | troublemaker 4956 | confession 4957 | relocate 4958 | irregular 4959 | directing 4960 | assassinate 4961 | goalpost 4962 | securing 4963 | flashback 4964 | dumbasses 4965 | birthdays 4966 | traditionally 4967 | grammatically 4968 | rg 4969 | excels 4970 | bigwig 4971 | sighted 4972 | oddly 4973 | html 4974 | sirens 4975 | md 4976 | opine 4977 | endgame 4978 | squeak 4979 | correspond 4980 | sniffing 4981 | burl 4982 | floating 4983 | longboat 4984 | bns 4985 | chronicle 4986 | wonderfully 4987 | chewy 4988 | untrusting 4989 | oas 4990 | benefactor 4991 | shanking 4992 | pva 4993 | fuzz 4994 | shear 4995 | dour 4996 | pittsburgh 4997 | courageous 4998 | discharge 4999 | tore 5000 | behaving 5001 | cuckolds 5002 | helium 5003 | foster 5004 | gateway 5005 | cheapskate 5006 | computational 5007 | nearby 5008 | lighted 5009 | ebit 5010 | binocular 5011 | setback 5012 | raleigh 5013 | escalade 5014 | rocketing 5015 | tiresome 5016 | uncontrollable 5017 | qatar 5018 | trumped 5019 | rollick 5020 | ubiquity 5021 | dorks 5022 | unspoken 5023 | scout 5024 | baboon 5025 | pisa 5026 | nearsightedness 5027 | captured 5028 | overreacting 5029 | trot 5030 | felling 5031 | redder 5032 | ada 5033 | funky 5034 | nirvana 5035 | perseverance 5036 | harping 5037 | frying 5038 | decapitate 5039 | astonish 5040 | isle 5041 | bungle 5042 | keen 5043 | strengthening 5044 | craw 5045 | persists 5046 | concise 5047 | toon 5048 | element 5049 | thruster 5050 | translator 5051 | unbranded 5052 | intriguing 5053 | tenacious 5054 | levitate 5055 | depending 5056 | mb 5057 | caring 5058 | hydrogen 5059 | gangsta 5060 | searcher 5061 | bandit 5062 | outlast 5063 | brainless 5064 | administrators 5065 | coveted 5066 | truffle 5067 | ramadan 5068 | stubbornly 5069 | mooch 5070 | hassle 5071 | denominator 5072 | crimp 5073 | golfing 5074 | cannibalizing 5075 | borodin 5076 | vise 5077 | prophesy 5078 | wingssss 5079 | fantastical 5080 | tantalizing 5081 | polite 5082 | conceit 5083 | transformation 5084 | bermuda 5085 | infernal 5086 | fluent 5087 | disconnected 5088 | octuple 5089 | lubricant 5090 | herman 5091 | envelope 5092 | flubs 5093 | navigation 5094 | swisher 5095 | outsmart 5096 | negating 5097 | cardiovascular 5098 | undervaluation 5099 | underhand 5100 | roth 5101 | strengthen 5102 | sharing 5103 | hale 5104 | prospective 5105 | pepper 5106 | deluxe 5107 | fruity 5108 | credential 5109 | bankroll 5110 | traveling 5111 | arrived 5112 | handed 5113 | wench 5114 | curtail 5115 | ensign 5116 | orient 5117 | nows 5118 | anyhow 5119 | rainmaker 5120 | snort 5121 | pinpoint 5122 | skipped 5123 | elliptic 5124 | chancellor 5125 | visionarys 5126 | hyperbolic 5127 | nonexistent 5128 | linux 5129 | shabby 5130 | legitimacy 5131 | built 5132 | shorten 5133 | solidarity 5134 | aten 5135 | hobbling 5136 | exasperate 5137 | monetizes 5138 | suppression 5139 | souls 5140 | conversational 5141 | uranium 5142 | isns 5143 | quirk 5144 | groundbreaking 5145 | pulp 5146 | athlete 5147 | stronger 5148 | refers 5149 | packer 5150 | ob 5151 | beard 5152 | kingdom 5153 | announced 5154 | mandated 5155 | yanked 5156 | ordered 5157 | null 5158 | silica 5159 | equip 5160 | --------------------------------------------------------------------------------