├── .gitignore ├── README.md ├── code ├── bst.py ├── build_train.py ├── merge_embeddings.py ├── negative_samples.py ├── sma_toolkit ├── sort_split.py ├── streaming_pickle.py ├── train_u2v.py └── usr2vec.py ├── raw_data └── sample.txt └── scripts ├── build_data.sh ├── merge_embeddings.sh ├── preprocess.py ├── setup.sh └── train_u2v.sh /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | *.pyc 3 | code/my_utils/* 4 | DATA/tmp 5 | DATA/out 6 | code/historical_tweets_extractor.py 7 | DATA/embeddings/bin 8 | DATA/embeddings/word_embeddings.txt 9 | code/sma_toolkit 10 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Deprecated 2 | 3 | This codebase is deprecated. See this [repo](https://github.com/samiroid/U2V/) for an updated implementation of User2Vec 4 | 5 | Usr2Vec 6 | ======= 7 | 8 | Implementation of the the *usr2vec* model to induce neural user embeddings, as described in the paper in the [paper](https://arxiv.org/abs/1705.00335) *Quantifying Mental Health from Social Media with Neural User Embeddings*. The resulting embeddings capture latent user aspects, e.g. political leanings and mental-health status (Figure 1) 9 | 10 | A previous version of this model, described in the paper *Modelling Context with User Embeddings for Sarcasm Detection in Social Media* can be found [here](https://github.com/samiroid/usr2vec/tree/v1). 11 | 12 | ![alt text](https://i.imgur.com/hbrY4bU.jpg "User Embeddings") Figure 1 - User embeddings projected into 2-Dimensions and colored according to mental health status. 13 | 14 | If you use this code please cite our paper as: 15 | > Amir, S., Coppersmith, G., Carvalho, P., Silva, M.J. and Wallace, B.C., 2017. *Quantifying Mental Health from Social Media with Neural User Embeddings*. In Journal of Machine Learning Research, W&C Track, Volume 68. 16 | 17 | ## Requirements: 18 | The software is implemented in python 2.7 and requires the following packages: 19 | * [sma_toolkit](https://github.com/samiroid/sma_toolkit) 20 | * numpy 21 | * gensim 22 | * joblib 23 | * theano 24 | 25 | ### Inputs/Outputs: 26 | 27 | There are two inputs to this model: 28 | 1. a text file with the training data --- the system assumes that the documents can be tokenized using whitespace (we recommend pre-tokenzing with the appropriate tokenizer) and that all messages from a user appear sequentially (see `raw_data/sample.txt` for an example) 29 | 2. a text file with pre-trained word embeddings (e.g. word2vec, glove) 30 | 31 | The output is a text file with a format similar to word2vec's, i.e. each line consists of `user_id \t embedding`. 32 | 33 | ## Instructions 34 | The software works in two main steps: (1) building the training the data; and (2) learning the user embeddings. The code can be executed as follow: 35 | 36 | 1. Setup 37 | 1. get the *sma_toolkit* 38 | 2. edit `scripts/setup.sh` and set the path to the *sma_toolkit* 39 | 3. run `./scripts/setup.sh` 40 | 2. Build training data 41 | 1. get some pretrained word embeddings 42 | 2. edit the paths on the file `scripts/build_data.sh` (i.e. the variables *DATA_PATH*, *WORD_EMBEDDINGS*) 43 | 3. run `./scripts/build_data.sh` 44 | 3. Train model: run `./scripts/build_data.sh [DATA_PATH] [OUTPUT_PATH]` 45 | 46 | -------------------------------------------------------------------------------- /code/bst.py: -------------------------------------------------------------------------------- 1 | """ 2 | Binary Search Tree: A sorted collection of values that supports 3 | efficient insertion, deletion, and minimum/maximum value finding. 4 | """ 5 | # Copyright (C) 2008 by Edward Loper 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in 15 | # all copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 23 | # THE SOFTWARE. 24 | 25 | 26 | # IMPLEMENTATION NOTES: 27 | # 28 | # Internally, we represent tree nodes using Python lists. These lists 29 | # may either be empty (for empty nodes) or may have length four (for 30 | # non-empty nodes). The non-empty nodes contain: 31 | # 32 | # [left_child, right_child, value, sort_key] 33 | # 34 | # Using lists rather than a node class more than doubles the overall 35 | # performance in the benchmarks that I have run. 36 | # 37 | # The sort key is always accessed as node[-1]. This allows us to 38 | # optimize the case where the sort key is identical to the value, by 39 | # encoding such nodes as simply: 40 | # 41 | # [left_child, right_child, value] 42 | # 43 | # The following constants are used to access the pieces of each search 44 | # node. If the constant-binding optimization recipe (which can be 45 | # downloaded from ) is 46 | # available, then it is used to replace these constants at 47 | # import-time, increasing the binary search tree efficiency by 3-5%. 48 | _LEFT = 0 49 | _RIGHT = 1 50 | _VALUE = 2 51 | _SORT_KEY = -1 52 | 53 | class BinarySearchTree(object): 54 | """ 55 | A sorted collection of values that supports efficient insertion, 56 | deletion, and minimum/maximum value finding. Values may sorted 57 | either based on their own value, or based on a key value whose 58 | value is computed by a key function (specified as an argument to 59 | the constructor). 60 | 61 | BinarySearchTree allows duplicates -- i.e., a BinarySearchTree may 62 | contain multiple values that are equal to one another (or multiple 63 | values with the same key). The ordering of equal values, or 64 | values with equal keys, is undefined. 65 | """ 66 | def __init__(self, sort_key=None): 67 | """ 68 | Create a new empty BST. If a sort key is specified, then it 69 | will be used to define the sort order for the BST. If an 70 | explicit sort key is not specified, then each value is 71 | considered its own sort key. 72 | """ 73 | self._root = [] # = empty node 74 | self._sort_key = sort_key 75 | self._len = 0 # keep track of how many items we contain. 76 | 77 | #///////////////////////////////////////////////////////////////// 78 | # Public Methods 79 | #///////////////////////////////////////////////////////////////// 80 | 81 | def insert(self, value): 82 | """ 83 | Insert the specified value into the BST. 84 | """ 85 | # Get the sort key for this value. 86 | if self._sort_key is None: 87 | sort_key = value 88 | else: 89 | sort_key = self._sort_key(value) 90 | # Walk down the tree until we find an empty node. 91 | node = self._root 92 | while node: 93 | if sort_key < node[_SORT_KEY]: 94 | node = node[_LEFT] 95 | else: 96 | node = node[_RIGHT] 97 | # Put the value in the empty node. 98 | if sort_key is value: 99 | node[:] = [[], [], value] 100 | else: 101 | node[:] = [[], [], value, sort_key] 102 | self._len += 1 103 | 104 | def minimum(self): 105 | """ 106 | Return the value with the minimum sort key. If multiple 107 | values have the same (minimum) sort key, then it is undefined 108 | which one will be returned. 109 | """ 110 | return self._extreme_node(_LEFT)[_VALUE] 111 | 112 | def maximum(self): 113 | """ 114 | Return the value with the maximum sort key. If multiple values 115 | have the same (maximum) sort key, then it is undefined which one 116 | will be returned. 117 | """ 118 | return self._extreme_node(_RIGHT)[_VALUE] 119 | 120 | def find(self, sort_key): 121 | """ 122 | Find a value with the given sort key, and return it. If no such 123 | value is found, then raise a KeyError. 124 | """ 125 | return self._find(sort_key)[_VALUE] 126 | 127 | def pop_min(self): 128 | """ 129 | Return the value with the minimum sort key, and remove that value 130 | from the BST. If multiple values have the same (minimum) sort key, 131 | then it is undefined which one will be returned. 132 | """ 133 | return self._pop_node(self._extreme_node(_LEFT)) 134 | 135 | def pop_max(self): 136 | """ 137 | Return the value with the maximum sort key, and remove that value 138 | from the BST. If multiple values have the same (maximum) sort key, 139 | then it is undefined which one will be returned. 140 | """ 141 | return self._pop_node(self._extreme_node(_RIGHT)) 142 | 143 | def pop(self, sort_key): 144 | """ 145 | Find a value with the given sort key, remove it from the BST, and 146 | return it. If multiple values have the same sort key, then it is 147 | undefined which one will be returned. If no value has the 148 | specified sort key, then raise a KeyError. 149 | """ 150 | return self._pop_node(self._find(sort_key)) 151 | 152 | def values(self, reverse=False): 153 | """Generate the values in this BST in sorted order.""" 154 | if reverse: 155 | return self._iter(_RIGHT, _LEFT) 156 | else: 157 | return self._iter(_LEFT, _RIGHT) 158 | __iter__ = values 159 | 160 | def __len__(self): 161 | """Return the number of items in this BST""" 162 | return self._len 163 | 164 | def __nonzero__(self): 165 | """Return true if this BST is not empty""" 166 | return self._len>0 167 | 168 | def __repr__(self): 169 | return '' % ', '.join('%r' % v for v in self) 170 | 171 | def __str__(self): 172 | return self.pprint() 173 | 174 | def pprint(self, max_depth=10, frame=True, show_key=True): 175 | """ 176 | Return a pretty-printed string representation of this binary 177 | search tree. 178 | """ 179 | t,m,b = self._pprint(self._root, max_depth, show_key) 180 | lines = t+[m]+b 181 | if frame: 182 | width = max(40, max(len(line) for line in lines)) 183 | s = '+-'+'MIN'.rjust(width, '-')+'-+\n' 184 | s += ''.join('| %s |\n' % line.ljust(width) for line in lines) 185 | s += '+-'+'MAX'.rjust(width, '-')+'-+\n' 186 | return s 187 | else: 188 | return '\n'.join(lines) 189 | 190 | #///////////////////////////////////////////////////////////////// 191 | # Private Helper Methods 192 | #///////////////////////////////////////////////////////////////// 193 | 194 | def _extreme_node(self, side): 195 | """ 196 | Return the leaf node found by descending the given side of the 197 | BST (either _LEFT or _RIGHT). 198 | """ 199 | if not self._root: 200 | raise IndexError('Empty Binary Search Tree!') 201 | node = self._root 202 | # Walk down the specified side of the tree. 203 | while node[side]: 204 | node = node[side] 205 | return node 206 | 207 | def _find(self, sort_key): 208 | """ 209 | Return a node with the given sort key, or raise KeyError if not found. 210 | """ 211 | node = self._root 212 | while node: 213 | node_key = node[_SORT_KEY] 214 | if sort_key < node_key: 215 | node = node[_LEFT] 216 | elif sort_key > node_key: 217 | node = node[_RIGHT] 218 | else: 219 | return node 220 | raise KeyError("Key %r not found in BST" % sort_key) 221 | 222 | def _pop_node(self, node): 223 | """ 224 | Delete the given node, and return its value. 225 | """ 226 | value = node[_VALUE] 227 | if node[_LEFT]: 228 | if node[_RIGHT]: 229 | # This node has a left child and a right child; find 230 | # the node's successor, and replace the node's value 231 | # with its successor's value. Then replace the 232 | # sucessor with its right child (the sucessor is 233 | # guaranteed not to have a left child). Note: node 234 | # and successor may not be the same length (3 vs 4) 235 | # because of the key-equal-to-value optimization; so 236 | # we have to be a little careful here. 237 | successor = node[_RIGHT] 238 | while successor[_LEFT]: successor = successor[_LEFT] 239 | node[2:] = successor[2:] # copy value & key 240 | successor[:] = successor[_RIGHT] 241 | else: 242 | # This node has a left child only; replace it with 243 | # that child. 244 | node[:] = node[_LEFT] 245 | else: 246 | if node[_RIGHT]: 247 | # This node has a right child only; replace it with 248 | # that child. 249 | node[:] = node[_RIGHT] 250 | else: 251 | # This node has no children; make it empty. 252 | del node[:] 253 | self._len -= 1 254 | return value 255 | 256 | def _iter(self, pre, post): 257 | # Helper for sorted iterators. 258 | # - If (pre,post) = (_LEFT,_RIGHT), then this will generate items 259 | # in sorted order. 260 | # - If (pre,post) = (_RIGHT,_LEFT), then this will generate items 261 | # in reverse-sorted order. 262 | # We use an iterative implemenation (rather than the recursive one) 263 | # for efficiency. 264 | stack = [] 265 | node = self._root 266 | while stack or node: 267 | if node: # descending the tree 268 | stack.append(node) 269 | node = node[pre] 270 | else: # ascending the tree 271 | node = stack.pop() 272 | yield node[_VALUE] 273 | node = node[post] 274 | 275 | def _pprint(self, node, max_depth, show_key, spacer=2): 276 | """ 277 | Returns a (top_lines, mid_line, bot_lines) tuple, 278 | """ 279 | if max_depth == 0: 280 | return ([], '- ...', []) 281 | elif not node: 282 | return ([], '- EMPTY', []) 283 | else: 284 | top_lines = [] 285 | bot_lines = [] 286 | mid_line = '-%r' % node[_VALUE] 287 | if len(node) > 3: mid_line += ' (key=%r)' % node[_SORT_KEY] 288 | if node[_LEFT]: 289 | t,m,b = self._pprint(node[_LEFT], max_depth-1, 290 | show_key, spacer) 291 | indent = ' '*(len(b)+spacer) 292 | top_lines += [indent+' '+line for line in t] 293 | top_lines.append(indent+'/'+m) 294 | top_lines += [' '*(len(b)-i+spacer-1)+'/'+' '*(i+1)+line 295 | for (i, line) in enumerate(b)] 296 | if node[_RIGHT]: 297 | t,m,b = self._pprint(node[_RIGHT], max_depth-1, 298 | show_key, spacer) 299 | indent = ' '*(len(t)+spacer) 300 | bot_lines += [' '*(i+spacer)+'\\'+' '*(len(t)-i)+line 301 | for (i, line) in enumerate(t)] 302 | bot_lines.append(indent+'\\'+m) 303 | bot_lines += [indent+' '+line for line in b] 304 | return (top_lines, mid_line, bot_lines) 305 | 306 | # try: 307 | # # Try to use the python recipe: 308 | # # 309 | # # This will only work if that recipe has been saved a 310 | # # "optimize_constants.py". 311 | # from optimize_constants import bind_all 312 | # bind_all(BinarySearchTree) 313 | # except: 314 | # pass 315 | -------------------------------------------------------------------------------- /code/build_train.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import cPickle 3 | from collections import Counter 4 | from ipdb import set_trace 5 | from negative_samples import negative_sampler 6 | import numpy as np 7 | import os 8 | from sma_toolkit import embeddings as emb_utils 9 | import streaming_pickle as stPickle 10 | import time 11 | np.set_printoptions(threshold=np.nan) 12 | MIN_DOC_LEN=4 13 | 14 | def get_parser(): 15 | parser = argparse.ArgumentParser(description="Build Training Data") 16 | parser.add_argument('-input', type=str, required=True, help='train file(s)') 17 | parser.add_argument('-emb', type=str, required=True, help='path to word embeddings') 18 | parser.add_argument('-output', type=str, required=True, help='path of the output') 19 | parser.add_argument('-vocab_size', type=int, help='path of the output') 20 | parser.add_argument('-min_word_freq', type=int, help='ignore words that occur less than min_word_freq times',default=5) 21 | parser.add_argument('-seed', type=int, default=1234, help='random number generator seed') 22 | parser.add_argument('-neg_samples', type=int, help='number of negative samples', default=10) 23 | return parser 24 | 25 | if __name__ == "__main__" : 26 | parser = get_parser() 27 | args = parser.parse_args() 28 | rng = np.random.RandomState(args.seed) 29 | 30 | print "[input: %s | word vectors: %s | max vocab_size: %s | min_word_freq: %s | output: %s]" % (os.path.basename(args.input), 31 | os.path.basename(args.emb), 32 | args.vocab_size, 33 | args.min_word_freq, 34 | os.path.basename(args.output)) 35 | 36 | t0 = time.time() 37 | word_counter = Counter() 38 | n_docs=0 39 | with open(args.input,"r") as fid: 40 | for line in fid: 41 | message = line.decode("utf-8").split()[1:] 42 | word_counter.update(message) 43 | n_docs+=1 44 | #keep only words that occur at least min_word_freq times 45 | wc = {w:c for w,c in word_counter.items() if c>args.min_word_freq} 46 | #keep only the args.vocab_size most frequent words 47 | tw = sorted(wc.items(), key=lambda x:x[1],reverse=True) 48 | top_words = {w[0]:i for i,w in enumerate(tw[:args.vocab_size])} 49 | print "loading word embeddings..." 50 | full_E, full_wrd2idx = emb_utils.read_embeddings(args.emb,top_words) 51 | ooevs = emb_utils.get_OOEVs(full_E, full_wrd2idx) 52 | #keep only words with pre-trained embeddings 53 | old_len = len(top_words) 54 | for w in ooevs: 55 | del top_words[w] 56 | wrd2idx = {w:i for i,w in enumerate(top_words.keys())} 57 | print "[vocabulary size: %d|%d]" % (len(wrd2idx),old_len) 58 | #generate the embedding matrix 59 | emb_size = full_E.shape[0] 60 | E = np.zeros((int(emb_size), len(wrd2idx))) 61 | for wrd,idx in wrd2idx.items(): 62 | E[:, idx] = full_E[:,top_words[wrd]] 63 | 64 | print "building training data..." 65 | #negative sampler 66 | idx2wrd = {i:w for w,i in wrd2idx.items()} 67 | sampler = negative_sampler(word_counter, idx2wrd) 68 | 69 | if not os.path.exists(os.path.dirname(args.output)): 70 | os.makedirs(os.path.dirname(args.output)) 71 | 72 | prev_user, prev_user_data, prev_ctxscores, prev_neg_samples = None, [], [], [] 73 | wrd_idx_counts = np.zeros(len(wrd2idx)) 74 | f_train = open(args.output,"wb") 75 | 76 | with open(args.input,"r") as fid: 77 | for j, line in enumerate(fid): 78 | try: 79 | message = line.replace("\"", "").replace("'","").split("\t")[1].decode("utf-8").split() 80 | except: 81 | print "ignored line: {}".format(line) 82 | #convert to indices 83 | msg_idx = [wrd2idx[w] for w in message if w in wrd2idx] 84 | #compute negative samples 85 | negative_samples = sampler.sample((len(msg_idx),args.neg_samples)) 86 | if len(msg_idx) user: %s (%d)" % (prev_user, len(train)) 125 | prev_user = u_idx 126 | prev_user_data.append(msg_idx) 127 | prev_neg_samples.append(negative_samples) 128 | 129 | #collect word counts to compute unigram distribution 130 | for w_idx in msg_idx: 131 | wrd_idx_counts[w_idx]+=1 132 | f_train.close() 133 | unigram_distribution = wrd_idx_counts / wrd_idx_counts.sum(0) 134 | print "[pickling aux data]" 135 | #aux_data = os.path.split(args.output)[0] + "/aux.pkl" 136 | aux_data = os.path.splitext(args.output)[0] + "_aux.pkl" 137 | with open(aux_data,"wb") as fid: 138 | cPickle.dump([wrd2idx,unigram_distribution, word_counter, E], fid, cPickle.HIGHEST_PROTOCOL) 139 | tend = time.time() - t0 140 | print "\n[runtime: %d minutes (%.2f secs)]" % ((tend/60),tend) -------------------------------------------------------------------------------- /code/merge_embeddings.py: -------------------------------------------------------------------------------- 1 | from pdb import set_trace 2 | 3 | out = "DATA/embeddings/usr2vec_400_master_4.txt" 4 | n_lines = 0 5 | emb_size = 0 6 | for i in xrange(1,8): 7 | with open("DATA/embeddings/usr2vec_400_%d.txt" % i, "r") as fid: 8 | _, emb_size = fid.readline().split() 9 | for line in fid: 10 | n_lines+=1 11 | 12 | with open(out,"w") as fod: 13 | fod.write("%d %d\n" % (n_lines,int(emb_size))) 14 | 15 | for i in xrange(1,8): 16 | with open("DATA/embeddings/usr2vec_400_%d.txt" % i, "r") as fid: 17 | fid.readline() 18 | for line in fid: 19 | # print line 20 | fod.write(line) 21 | -------------------------------------------------------------------------------- /code/negative_samples.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import cPickle 3 | from joblib import Parallel, delayed 4 | from ipdb import set_trace 5 | import numpy as np 6 | import os 7 | import streaming_pickle as stPickle 8 | import sys 9 | import time 10 | 11 | TRAIN_IDX = 1 12 | NEG_SAMPLES_IDX = 4 13 | 14 | def multinomial_samples(unigram_distribution, exclude=[], n_samples=1): 15 | samples = [] 16 | while len(samples) != n_samples: 17 | wrd_idx = np.argmax(np.random.multinomial(1, unigram_distribution)) 18 | # from ipdb import set_trace; set_trace() 19 | if wrd_idx not in exclude: 20 | samples.append(wrd_idx) 21 | return samples 22 | 23 | def random_samples(exclude=[], n_samples=1): 24 | pass 25 | 26 | def extract(instance, unigram_distribution, num_neg_samples): 27 | train = instance[TRAIN_IDX] 28 | neg_samples = [] 29 | for msg in train: 30 | set_trace() 31 | neg_samp = [ multinomial_samples(unigram_distribution, msg, num_neg_samples) \ 32 | for _ in xrange(len(msg)) ] 33 | neg_samples.append(neg_samp) 34 | instance[NEG_SAMPLES_IDX] = neg_samples 35 | return instance 36 | 37 | class negative_sampler(): 38 | 39 | def __init__(self, word_count, index2word, warp=0.75): 40 | ''' 41 | Store count for the range of indices in the dictionary 42 | ''' 43 | max_index = max(index2word.keys()) 44 | counts = [] 45 | for n in range(max_index): 46 | if n in index2word: 47 | counts.append(word_count[index2word[n]]**warp) 48 | else: 49 | counts.append(0) 50 | counts = np.array(counts) 51 | norm_counts = counts/sum(counts) 52 | scaling = int(np.ceil(1./min(norm_counts[norm_counts>0]))) 53 | scaled_counts = (norm_counts*scaling).astype(int) 54 | self.cumsum = scaled_counts.cumsum() 55 | 56 | def sample(self, size=1): 57 | total_size = np.prod(size) 58 | random_ints = np.random.randint(self.cumsum[-1], size=total_size) 59 | data_y_neg = np.searchsorted(self.cumsum, random_ints).astype('int32') 60 | return data_y_neg.reshape(size) 61 | 62 | def sample_2(self, exclude, size=1): 63 | total_size = np.prod(size) 64 | random_ints = np.random.randint(self.cumsum[-1], size=total_size) 65 | data_y_neg = np.searchsorted(self.cumsum, random_ints).astype('int32') 66 | #filter out words that should be excluded 67 | filtered = [x for x in data_y_neg.tolist() if x not in exclude][:total_size] 68 | data_y_neg = np.array(filtered) 69 | return data_y_neg.reshape(size) 70 | 71 | def get_parser(): 72 | parser = argparse.ArgumentParser(description="Extract negative samples") 73 | parser.add_argument('-input', type=str, required=True, help='train file') 74 | parser.add_argument('-aux_data', type=str, required=True, help='aux data file') 75 | parser.add_argument('-n_workers', type=int, help='number of jobs', default=1) 76 | parser.add_argument('-neg_samples', type=int, help='number of negative samples', default=10) 77 | parser.add_argument('-resume', action="store_true", help='continue from a previous run', default=False) 78 | return parser 79 | 80 | if __name__ == "__main__": 81 | parser = get_parser() 82 | args = parser.parse_args() 83 | print "[training data: %s | aux_data: %s | n_workers: %d | resume: %s]" % (args.input, args.aux_data, args.n_workers, args.resume) 84 | with open(args.aux_data,"r") as fid: 85 | wrd2idx, unigram_distribution, word_counter, _ = cPickle.load(fid) 86 | 87 | index2word = {i:w for w,i in wrd2idx.items()} 88 | sampler = negative_sampler(word_counter, index2word) 89 | 90 | t0 = time.time() 91 | prev_time = time.time() 92 | tdf = open(args.input,"r") 93 | training_data = stPickle.s_load(tdf) 94 | tmp_data_path = args.input.strip(".pkl")+"_new.pkl" 95 | new_training_data = open(tmp_data_path,"w") 96 | 97 | # for instance in training_data: 98 | # neg_samples = [] 99 | # for msg in instance[TRAIN_IDX]: 100 | # neg_samples += [[ multinomial_samples(unigram_distribution, msg, args.neg_samples) \ 101 | # for _ in xrange(len(msg)) ]] 102 | # instance[NEG_SAMPLES_IDX] = neg_samples 103 | # stPickle.s_dump_elt(instance,new_training_data) 104 | 105 | for instance in training_data: 106 | neg_samples = [] 107 | for msg in instance[TRAIN_IDX]: 108 | neg_samples += [sampler.sample((len(msg),args.neg_samples))] 109 | instance[NEG_SAMPLES_IDX] = neg_samples 110 | stPickle.s_dump_elt(instance,new_training_data) 111 | 112 | #replace the training data file with the new augmented one 113 | os.remove(args.input) 114 | os.rename(tmp_data_path, args.input) 115 | tend = time.time() - t0 116 | print "\n[runtime: %d minutes (%.2f secs)]" % ((tend/60),tend) 117 | 118 | 119 | 120 | # neg_samples = [] 121 | 122 | # set_trace() 123 | 124 | # neg_samples.append(neg_samp) 125 | # instance[NEG_SAMPLES_IDX] = neg_samples 126 | # return instance 127 | 128 | # set_trace() 129 | # sys.exit() 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | # done=False 141 | # j=0 #number of instances processed 142 | # while not done: 143 | # try: 144 | # current_instances = [] 145 | # if args.resume: 146 | # while len(current_instances) < args.n_workers: 147 | # next_inst = training_data.next() 148 | # #ignore the instances where these quantities have already been calculated 149 | # if len(next_inst[NEG_SAMPLES_IDX]) == 0: current_instances.append(next_inst) 150 | # else: 151 | # for _ in xrange(args.n_workers): current_instances.append(training_data.next()) 152 | # except StopIteration: 153 | # done=True 154 | # if args.n_workers>1: 155 | # with Parallel(n_jobs=args.n_workers) as parallel: 156 | # res = parallel(delayed(extract)(instance, unigram_distribution, args.negative_samples) for instance in current_instances) 157 | # else: 158 | # res = [extract(instance,unigram_distribution, args.negative_samples) for instance in current_instances] 159 | # # print res 160 | # for r in res: stPickle.s_dump_elt(r,new_training_data) 161 | # j+=len(current_instances) 162 | # t_i = (time.time() - t0) / 60 163 | # sys.stdout.write("\r>:%d (~%d mins)" % (j,t_i)) 164 | # sys.stdout.flush() 165 | # #replace the training data file with the new augmented one 166 | # os.remove(args.input) 167 | # os.rename(tmp_data_path, args.input) 168 | # tend = time.time() - t0 169 | # print "\n[runtime: %d minutes (%.2f secs)]" % ((tend/60),tend) 170 | 171 | 172 | 173 | # SAMPLES = 10000 174 | # samplz = [] 175 | # rand_msgs = [np.random.randint(0,len(wrd2idx),np.random.randint(4,20)) for x in xrange(SAMPLES)] 176 | 177 | # t0 = time.time() 178 | # for m in rand_msgs: 179 | # samplz += [sampler.sample((10,len(m)))] 180 | # tend = t0 - time.time() 181 | # print "simple version took: ", tend 182 | 183 | # print "*"*80 184 | 185 | # t0 = time.time() 186 | # for m in rand_msgs: 187 | # samplz += [sampler.sample_2(m,(10,len(m)))] 188 | # tend = t0 - time.time() 189 | # print "crazy version took: ", tend -------------------------------------------------------------------------------- /code/sma_toolkit: -------------------------------------------------------------------------------- 1 | /Users/samir/Dev/projects/sma_toolkit/ -------------------------------------------------------------------------------- /code/sort_split.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | from bst import BinarySearchTree 3 | import math 4 | import numpy as np 5 | import os 6 | from pdb import set_trace 7 | import streaming_pickle as stPickle 8 | 9 | """ 10 | Module to reorganize training data 11 | 1. Sorts users by number their number of documents 12 | 2. Splits users into a set of files 13 | """ 14 | 15 | def get_parser(): 16 | parser = argparse.ArgumentParser(description="Sort and split User2Vec training data") 17 | parser.add_argument('-input', type=str, required=True, help='train file(s)') 18 | parser.add_argument('-n_splits', type=int, help='number of splits',default=2) 19 | return parser 20 | 21 | if __name__ == "__main__": 22 | parser = get_parser() 23 | args = parser.parse_args() 24 | if args.n_splits >= 2: 25 | print "sorting users by #tweets..." 26 | bst = BinarySearchTree(sort_key=lambda x:x[1]) 27 | tf = open(args.input,"r") 28 | for x in stPickle.s_load(tf): 29 | user, train, _, _,_ = x 30 | bst.insert((user,len(train))) 31 | sorted_values = list(bst.values(reverse=True)) 32 | sorted_users = [x[0] for x in sorted_values] 33 | print "[spliting %d users into #files: %d]" % (len(sorted_users),args.n_splits) 34 | out_files = [] 35 | out_path, ext = os.path.splitext(args.input) 36 | for i in xrange(args.n_splits): 37 | fname = "%s%d%s" % (out_path,i+1,ext) 38 | print " > %s" % fname 39 | f = open(fname,"w") 40 | out_files.append(f) 41 | tf.seek(0) 42 | out_log = [[] for x in xrange(args.n_splits)] 43 | # set_trace() 44 | print "[processing users]" 45 | partition_size = math.floor(len(sorted_users)*1.0/args.n_splits) 46 | for x in stPickle.s_load(tf): 47 | user, train, _, _,_ = x 48 | user_rank = sorted_users.index(user) 49 | fnumber = int(math.floor(user_rank*1.0/partition_size)) 50 | if fnumber < 0: fnumber = 0 51 | if fnumber > args.n_splits-1: fnumber = args.n_splits-1 52 | print " > user: %s | #train: %d | rank: %d | fnum: %d" % (user, len(train), user_rank, fnumber) 53 | out_file = out_files[fnumber] 54 | stPickle.s_dump_elt(x, out_file) 55 | out_log[fnumber].append(len(train)) 56 | print "[avg #docs: ]" 57 | for i in xrange(len(out_log)): print " >file %d: %.3f " % (i,np.mean(out_log[i])) 58 | print "[removing original training file: %s]" % args.input 59 | os.remove(args.input) 60 | print "error: n_splits should be at least 2" 61 | -------------------------------------------------------------------------------- /code/streaming_pickle.py: -------------------------------------------------------------------------------- 1 | """Streaming pickle implementation for efficiently serializing and 2 | de-serializing an iterable (e.g., list) 3 | 4 | Created on 2010-06-19 by Philip Guo 5 | 6 | http://code.google.com/p/streaming-pickle/ 7 | 8 | Modified by Brian Thorne 2013 to add base64 encoding to support 9 | python3 bytearray and the like. 10 | """ 11 | import base64 12 | from cPickle import dumps, loads 13 | import unittest 14 | import tempfile 15 | 16 | def s_dump(iterable_to_pickle, file_obj): 17 | """dump contents of an iterable iterable_to_pickle to file_obj, a file 18 | opened in write mode""" 19 | for elt in iterable_to_pickle: 20 | s_dump_elt(elt, file_obj) 21 | 22 | 23 | def s_dump_elt(elt_to_pickle, file_obj): 24 | """dumps one element to file_obj, a file opened in write mode""" 25 | pickled_elt = dumps(elt_to_pickle,-1) 26 | encoded = base64.b64encode(pickled_elt) 27 | file_obj.write(encoded) 28 | 29 | # record separator is a blank line 30 | # (since pickled_elt as base64 encoded cannot contain its own newlines) 31 | file_obj.write(b'\n\n') 32 | 33 | 34 | def s_load(file_obj): 35 | """load contents from file_obj, returning a generator that yields one 36 | element at a time""" 37 | cur_elt = [] 38 | for line in file_obj: 39 | 40 | if line == b'\n': 41 | encoded_elt = b''.join(cur_elt) 42 | try: 43 | pickled_elt = base64.b64decode(encoded_elt) 44 | elt = loads(pickled_elt) 45 | except EOFError: 46 | print("EOF found while unpickling data") 47 | print(pickled_elt) 48 | raise StopIteration 49 | cur_elt = [] 50 | yield elt 51 | else: 52 | cur_elt.append(line) 53 | 54 | 55 | class TestStreamingPickle(unittest.TestCase): 56 | def setUp(self): 57 | pass 58 | 59 | def testSimpleList(self): 60 | # data = [1, 2, 3, 4, None, b'test', '\n', '\x00', 3, b'\n\n\n\n', 5, 7, 9, 11, "hello", bytearray([2, 4, 4])] 61 | data = [1,[1,2,3,4],[8,9,29]] 62 | with tempfile.TemporaryFile() as f: 63 | s_dump(data, f) 64 | # reset the temporary file 65 | f.seek(0) 66 | i = 0 67 | for i, element in enumerate(s_load(f)): 68 | self.assertEqual(data[i], element) 69 | # print(i, element) 70 | self.assertEqual(i, len(data)-1) 71 | 72 | if __name__ == "__main__": 73 | unittest.main() 74 | -------------------------------------------------------------------------------- /code/train_u2v.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import cPickle 3 | from sma_toolkit import colstr 4 | #from ipdb import set_trace 5 | import usr2vec 6 | import sys 7 | import warnings 8 | import time 9 | import os 10 | import random 11 | import streaming_pickle as stPickle 12 | warnings.filterwarnings("ignore") 13 | import numpy as np 14 | 15 | def count_users(dataset): 16 | with open(dataset) as fid: 17 | return len([z for z in stPickle.s_load(fid)]) 18 | 19 | def get_parser(): 20 | parser = argparse.ArgumentParser(description="Train U2V") 21 | parser.add_argument('-input', type=str, required=True, help='train file(s)') 22 | parser.add_argument('-aux', type=str, required=True, help='aux data file') 23 | parser.add_argument('-output', type=str, help='output path') 24 | parser.add_argument('-lrate', type=float, default=5e-5, help='learning rate') 25 | parser.add_argument('-margin', type=int, default=1, help='margin size') 26 | parser.add_argument('-epochs', type=int, default=25, help='number of training epochs') 27 | parser.add_argument('-patience', type=int, default=5, help='stop training if no progress made after this number of epochs') 28 | parser.add_argument('-reshuff', default=False, action="store_true", help='if True, instances will be reshuffled after each training epoch') 29 | parser.add_argument('-init_w2v', choices=["mean",'gauss'], help='initialize user embeddings with information from the word embeddings') 30 | parser.add_argument('-quiet', default=False, action="store_true", help='do not print training objectives and validation error') 31 | 32 | return parser 33 | 34 | if __name__ == "__main__": 35 | #command line arguments 36 | parser = get_parser() 37 | args = parser.parse_args() 38 | print "[Training U2V by user]" 39 | print "loading data..." 40 | with open(args.aux,"r") as fid: 41 | _,_,_,E = cPickle.load(fid) 42 | try: 43 | n_usrs = count_users(args.input) 44 | except IOError: 45 | print "Couldn't not find file %s" % args.input 46 | sys.exit() 47 | #path to save intermediate versions of the user embedding matrix (during training) 48 | tmp_name = ''.join([ chr(random.randint(97,122)) for i in xrange(10)]) 49 | user_emb_bin = os.path.split(args.input)[0]+"/tmp-"+tmp_name.upper() 50 | print "[lrate: %.5f | margin loss: %d | epochs: %d| reshuff: %s | init_w2v: %s | @%s]\n" % (args.lrate,args.margin, args.epochs, args.reshuff, args.init_w2v, user_emb_bin) 51 | if args.init_w2v: 52 | u2v = usr2vec.Usr2Vec(E, n_usrs,lrate=args.lrate,margin_loss=args.margin, init_w2v=args.init_w2v) 53 | else: 54 | u2v = usr2vec.Usr2Vec(E, n_usrs,lrate=args.lrate,margin_loss=args.margin) 55 | total_time = time.time() 56 | usr2idx = {} 57 | tf = open(args.input,"r") 58 | 59 | training_data = stPickle.s_load(tf) 60 | #each training instance corresponds to a user 61 | total_logprob = 0 62 | total_epochs = 0 63 | for z, instance in enumerate(training_data): 64 | # if z == 100: 65 | # print "bailed earlier with %d users " % z 66 | # n_usrs = z 67 | # break 68 | prev_logprob, best_logprob = -10**100, -10**100 69 | prev_obj, best_obj = 10**100, 10**100 70 | drops = 0 71 | curr_lrate = u2v.lrate 72 | user, train, test, neg_samples = instance 73 | try: 74 | u_idx = usr2idx[user] 75 | except KeyError: 76 | u_idx = len(usr2idx) 77 | usr2idx[user] = u_idx 78 | if not args.quiet: 79 | print "[user: %s (%d/%d)]" % (user,z+1,n_usrs) 80 | user_time = time.time() 81 | for e in xrange(args.epochs): 82 | ############# TRAIN 83 | total_epochs+=1 84 | obj = 0 85 | prev_lrate = curr_lrate 86 | if args.reshuff: 87 | for x in np.random.permutation(len(train)): 88 | obj += u2v.train(u_idx, train[x], neg_samples[x], curr_lrate) 89 | else: 90 | for msg_train, neg in zip(train, neg_samples): 91 | obj += u2v.train(u_idx, msg_train, neg, curr_lrate) 92 | #average objective 93 | obj/=len(train) 94 | obj_color = None 95 | if obj < prev_obj: 96 | obj_color='green' 97 | if obj < best_obj: 98 | best_obj=obj 99 | elif obj > prev_obj: 100 | color='red' 101 | prev_obj=obj 102 | if not args.quiet: 103 | obj_str = colstr(("%.3f" % obj),obj_color,(best_obj==obj)) 104 | sys.stdout.write("\r\tepoch:%d | obj: %s" % ((e+1),obj_str)) 105 | sys.stdout.flush() 106 | 107 | ############# EVALUATE 108 | logprob=0 109 | for msg_test in test: 110 | l,all_prob = u2v.predict(u_idx, msg_test) 111 | logprob+= l 112 | logprob/=len(test) 113 | logprob=round(logprob,4) 114 | color=None 115 | if logprob > prev_logprob: 116 | color='green' 117 | if logprob > best_logprob: 118 | #keep best model 119 | u2v.save_model(user_emb_bin) 120 | best_logprob=logprob 121 | elif logprob < prev_logprob: 122 | drops+=1 123 | color='red' 124 | #decay the learning rate exponentially 125 | curr_lrate*=10**-1 126 | #curr_lrate/=2 127 | else: 128 | drops+=1 129 | if not args.quiet: 130 | if curr_lrate!=prev_lrate: 131 | print " ILL: " + colstr(("%.3f" % logprob), color, (best_logprob==logprob)) + " (lrate:" + str(curr_lrate)+")" 132 | else: 133 | print " ILL: " + colstr(("%.3f" % logprob), color, (best_logprob==logprob)) 134 | if drops>=args.patience: 135 | print "ran out of patience (%d epochs)" % e 136 | break 137 | prev_logprob = logprob 138 | et = time.time() - user_time 139 | user_mins = np.floor(et*1.0/60) 140 | user_secs = et - user_mins*60 141 | tt = time.time() - total_time 142 | tt_mins = np.floor(tt*1.0/60) 143 | tt_secs = tt - tt_mins*60 144 | total_logprob+=best_logprob 145 | alp = total_logprob/(z+1) 146 | if not args.quiet: 147 | print "> ILL: %.3f %d.%d mins (avg ILL: %.3f| %d.%d mins)" % (best_logprob,user_mins,user_secs,alp,tt_mins,tt_secs) 148 | 149 | tt = time.time() - total_time 150 | mins = np.floor(tt*1.0/60) 151 | secs = tt - mins*60 152 | print "*"*90 153 | print "[avg epochs: %d | avg ILL: %.4f | lrate: %.5f]" % ((total_epochs/n_usrs),(total_logprob/n_usrs),args.lrate) 154 | print "*"*90 155 | print "[runtime: %d.%d minutes]" % (mins,secs) 156 | tf.close() 157 | 158 | ############# EXPORT 159 | 160 | print "Exporting embeddings..." 161 | with open(user_emb_bin,"r") as fid: 162 | U = cPickle.load(fid)[0] 163 | #create dir if it does not exist 164 | if not os.path.exists(os.path.dirname(args.output)): 165 | os.makedirs(os.path.dirname(args.output)) 166 | with open(args.output+".txt","w") as fod: 167 | fod.write("%d %d\n" % (U.shape[1],U.shape[0])) 168 | for user, u_id in usr2idx.items(): 169 | emb = U[:,u_id] 170 | fod.write("%s %s\n" % (user, " ".join(map(str, emb)))) 171 | 172 | -------------------------------------------------------------------------------- /code/usr2vec.py: -------------------------------------------------------------------------------- 1 | import cPickle 2 | # from ipdb import set_trace 3 | import numpy as np 4 | import theano 5 | import theano.tensor as T 6 | 7 | def init_weight(rng, size): 8 | return np.asarray(rng.normal(0,0.01, size=size)) 9 | 10 | 11 | def init_w2v_gauss(rng, E, n_users): 12 | mu = np.mean(E,axis=1) 13 | mu = np.squeeze(np.asarray(mu)) 14 | cov = np.cov(E,rowvar=1) 15 | return np.random.multivariate_normal(mu, cov,size=n_users).T 16 | 17 | def init_w2v_mean(rng, E, n_users): 18 | mu = np.mean(E,axis=1) 19 | U = np.asarray(rng.normal(0,0.01, size=(E.shape[0],n_users))) 20 | return U + mu[:,None] 21 | 22 | 23 | class Usr2Vec(): 24 | 25 | def __init__(self, E, n_users, lrate=0.0001, margin_loss=1, rng=None, init_w2v=False): 26 | 27 | # Generate random seed if not provided 28 | if rng is None: 29 | rng=np.random.RandomState(1234) 30 | #parameters 31 | if init_w2v == "gauss": 32 | U = init_w2v_gauss(rng, n_users, E) 33 | elif init_w2v == "mean": 34 | U = init_w2v_mean(rng, E, n_users) 35 | else: 36 | U = init_weight(rng, (E.shape[0],n_users)) 37 | U = theano.shared(U.astype(theano.config.floatX), borrow=True) 38 | E = theano.shared(E.astype(theano.config.floatX), borrow=True) 39 | 40 | self.params = [U] 41 | self.margin_loss = margin_loss 42 | self.lrate = lrate 43 | #input 44 | usr_idx = T.iscalar('usr') 45 | sent_idx = T.ivector('sent') 46 | neg_samp_idx = T.imatrix('neg_sample') 47 | # word_probs = T.fvector('word_probs') 48 | #word_probs = T.fscalar('word_probs') 49 | curr_lrate = T.fscalar('lrate') 50 | #embedding lookup 51 | usr = U[:, usr_idx] 52 | sent = E[:, sent_idx] 53 | neg_samples = E[:, neg_samp_idx] 54 | #loss 55 | # objectives, _ = theano.scan(fn=self.rank_loss, 56 | # outputs_info=None, 57 | # sequences=[sent_idx,neg_samp_idx], 58 | # non_sequences=[usr,E,U]) 59 | pos_score = T.dot(usr,sent) 60 | neg_score = T.tensordot(usr,neg_samples,axes=(0,0)) 61 | loss = T.maximum(0, self.margin_loss - pos_score[:,None] + neg_score) 62 | # final_loss = loss.sum(axis=None) + word_probs.sum() 63 | final_loss = loss.sum(axis=None) 64 | #Gradient wrt to user embeddings 65 | usr_grad = T.grad(final_loss, usr) 66 | #Sparse update 67 | upd_usr = T.set_subtensor(usr, usr - curr_lrate*usr_grad) 68 | updates = ((U, upd_usr),) 69 | # self.dbg = theano.function(inputs=[usr_idx, sent_idx, neg_samp_idx], 70 | # outputs=[usr,sent,neg_samples], 71 | # mode="FAST_COMPILE") 72 | self.dbg = theano.function(inputs=[usr_idx, sent_idx, neg_samp_idx], 73 | outputs=[usr,sent,neg_samples]) 74 | 75 | self.train = theano.function(inputs=[usr_idx, sent_idx, neg_samp_idx, curr_lrate], 76 | outputs=final_loss, 77 | updates=updates, 78 | mode="FAST_RUN") 79 | #\propto P(message|usr) 80 | # scores_m = T.exp(T.dot(U.T,E[:,sent_idx])) 81 | scores_m = T.dot(U.T,E[:,sent_idx]) 82 | prob = T.nnet.softmax(scores_m.T).T 83 | log_prob = T.log(prob).sum(axis=1) 84 | #sum the scores for all the words 85 | # scores_m = scores_m.sum(axis=1) 86 | # user_score = scores_m[usr_idx] 87 | user_score = log_prob[usr_idx] 88 | self.predict = theano.function(inputs=[usr_idx,sent_idx], 89 | outputs=[user_score,prob]) 90 | 91 | def rank_loss(self, w_idx, negs_idx, usr, E, U): 92 | w_emb = E[:, w_idx] 93 | w_neg_emb = E[:, negs_idx] 94 | pos_score = T.dot(usr,w_emb) 95 | neg_score = T.dot(usr,w_neg_emb) 96 | loss = T.maximum(0, self.margin_loss - pos_score + neg_score) 97 | return loss 98 | 99 | def save_model(self, path): 100 | model = [self.params[i].get_value() for i in range(len(self.params))] 101 | with open(path,"wb") as fid: 102 | cPickle.dump(model,fid,cPickle.HIGHEST_PROTOCOL) -------------------------------------------------------------------------------- /raw_data/sample.txt: -------------------------------------------------------------------------------- 1 | quD3WcPmeNf9Ver my new sounds : swim good - felt like singing haha url on #soundcloud 2 | quD3WcPmeNf9Ver maybe next year i'll be recieving an award #mtvmusicawards2015 3 | quD3WcPmeNf9Ver work hard now , party later 4 | quD3WcPmeNf9Ver i actually love how close me and my sister are 5 | quD3WcPmeNf9Ver what is your favorite music band ? — ... url 6 | quD3WcPmeNf9Ver 😟 😟 😟 url 7 | quD3WcPmeNf9Ver my birthday cake 🌸 url 8 | quD3WcPmeNf9Ver @user thank youuu 💖 9 | quD3WcPmeNf9Ver rt @user @user still wishing u a happy birthday tho ! stay positive & amp ; make this day awesome ! 🎈 10 | quD3WcPmeNf9Ver @user thank youuu 💙 11 | quD3WcPmeNf9Ver rt @user @user shhh !! happy birthday , and make sure you have a good day , hope i see you soon xo 12 | quD3WcPmeNf9Ver ffs ... wish it wasn't my birthday want this day to end fully . 13 | quD3WcPmeNf9Ver silly girls ... 14 | bdkFmSM #mtvstars one direction 15 | bdkFmSM joder #mtvstars justin bieber 16 | bdkFmSM " búscame y me vas a encontrar . " 17 | bdkFmSM rt @user just met some amazing mexican beliebers and the presidente of mexico and his familia . now it is showtime . NUMBER , 000 tonight . … 18 | bdkFmSM rt @user #directioners … do you remember this staircase ? #xfactor url 19 | bdkFmSM que lastima 20 | bdkFmSM set de fotos : direct-news : harry at the book signing today . ( NUMBER / 13 ) x url 21 | bdkFmSM rt @user ¿sabias que ? : el 3er álbum " midnight memories " de one direction es considerado el álbum más esperado de todo el año #nial … 22 | bdkFmSM rt @user vevo filtro el videoclip de soml en vez del NUMBER days to go y itunes filtro todo el álbum vez del single midnight memories ¿ … 23 | bdkFmSM rt @user no se que me da mas pena si itunes que filtro " midnight memories " o las cuentas que ponen links para descargar el álbum i … 24 | bdkFmSM rt @user se filtr ó el NUMBER / noviembre una foto de one direction durante la grabaci ón del videoclip " story of my life " , en uk #ns http … 25 | bdkFmSM rt @user liam y niall se tomaron la ma ñana del NUMBER / nov fotos con algunos fans en las calles de la zona norte de londres #ns url 26 | bdkFmSM rt @user los NUMBER fans que asistieron el NUMBER / nov a la firma de aut ógrafos recibieron el libro y una muestra de " our moment " #ns http … 27 | bdkFmSM rt @user los NUMBER boletos del libro para la firma de aut ógrafos del NUMBER / nov se vendieron en cuesti ón de minutos , en uk #ns url 28 | bdkFmSM foto : url 29 | bdkFmSM foto : url 30 | bdkFmSM foto : irresistible-last - first - kiss : prepare for your tears ... url 31 | bdkFmSM audio : direct-news : why don ’ t we go there - one direction url 32 | bdkFmSM audio : direct-news : alive - one direction url 33 | bdkFmSM audio : direct-news : why don ’ t we go there - one direction url 34 | bdkFmSM audio : direct-news : better than words - one direction url 35 | bdkFmSM audio : direct-news : little white lies - one direction url 36 | bdkFmSM audio : direct-news : something great - one direction url 37 | bdkFmSM shoro soy re hija de puta . 38 | bdkFmSM omg 39 | bdkFmSM rt @user the #believemovie is only five weeks away ! we talked to jon chu about what fans will learn about @user url 40 | bdkFmSM rt @user mexico i'm smiling :) url 41 | bdkFmSM rt @user mexico i hear u . and u are loud . thank u . see u real soon 42 | bdkFmSM you're all that matters to me ... 43 | bdkFmSM iu 44 | bdkFmSM que es aire . url 45 | bdkFmSM me comen los mosquitos , ugh . 46 | bdkFmSM @user ilysm 47 | bdkFmSM hi #mtvstars justin bieber 48 | bdkFmSM hola #mtvstars one direction 49 | bdkFmSM asdf url 50 | bdkFmSM rt @user i just want everyone in the rest of south america to know , i wish i could have made it to their country as well . working o … 51 | bdkFmSM rt @user @user argentina manda , el resto obedecen . 52 | bdkFmSM ah 53 | bdkFmSM no encontr é ojos as í como los que tienes tu 54 | bdkFmSM ajajajaajaja 55 | bdkFmSM rt @user me and my brother greg in the sittin room ! #storyofmylife url url 56 | bdkFmSM rt @user NUMBER hours until the #acousticsessions hit @user url 57 | bdkFmSM @user follow me pls ilysm 58 | bdkFmSM NUMBER usuarios que sigo no me siguen de vuelta en twitter . ent érate de qui én no te sigue de vuelta url 59 | bdkFmSM soy un asco 60 | bdkFmSM :( 61 | bdkFmSM #mtvstars one direction 62 | bdkFmSM #mtvstars justin bieber . 63 | bdkFmSM don't stop #mtvstars justin bieber 64 | bdkFmSM #mtvstars justin bieber 65 | bdkFmSM . 66 | bdkFmSM q in útil soy . 67 | bdkFmSM lol 68 | bdkFmSM rt @user the #believetour is in mexico the next NUMBER nights . NUMBER , 000 people sold out NUMBER nights in a row ! crazy @user 69 | bdkFmSM rt @user soundcheck done . ready for show NUMBER of NUMBER in mexico city . #believetour 70 | bdkFmSM rt @user mexico is ready ! crazy 71 | bdkFmSM rt @user if you are at the show mexico city , get @user and take a selfie with #believetour . love u . url ;) 72 | bdkFmSM rt @user the streets are filled . the buildings are surrounded . people are in the trees . on roofs . on cars . everywhere . mexico i lo … 73 | bdkFmSM esos " i love you . #mybeliebers " que me matan . 74 | bdkFmSM rt @user i got so much stuff coming for all of you . so much music . so many surprises . expect the unexpected . i love you . #mybelieb … 75 | bdkFmSM rt @user si robo el wi-fi de una iglesia ¿ya estoy recibiendo la se ñal de dios ? 76 | bdkFmSM rt @user eres tan fea que hasta rexona te abandona . 77 | bdkFmSM rt @user mentirosaaa , sidosa mentirosaaa ~ chauu 78 | bdkFmSM rt @user #enelverano aparecen los que publican en las redes sociales " hace calor " gracias por avisar pelotudo , casi me meto a la … 79 | bdkFmSM @user put me down , bby . 80 | bdkFmSM me falto un bloque joder 81 | bdkFmSM su sonrisa . #mtvstars justin bieber url 82 | bdkFmSM every minute , every second , every hour of the day . 83 | bdkFmSM alksndlaksdn 84 | bdkFmSM @user soy un amor . 85 | bdkFmSM #mtvstars one direction 86 | bdkFmSM negra sidosa . 87 | bdkFmSM one time #mtvstars justin bieber 88 | bdkFmSM pyd #mtvstars justin bieber 89 | bdkFmSM @user argentina loves u , we miss u . ♡ 90 | bdkFmSM rt @user excited for tonight . new acoustic sessions from @user hit at midnight . 91 | bdkFmSM rt @user new episode of the #paradiseseries directed by @user it's a look at the making of the #acousticsessions ep http … 92 | bdkFmSM @user cry 93 | bdkFmSM @user i love you so much te amo 94 | bdkFmSM rt @user we will get it around the world :) url 95 | bdkFmSM rt @user midnight memories … who ’ s getting it :-) ? it ’ s out on november NUMBER 5th ! url 96 | bdkFmSM rt @user do you want to help the people of the philippines ? please call @user philippines telethon on PHONENUMBER between 4pm - … 97 | bdkFmSM rt @user please help the people of the philippines . call @user philippines telethon on PHONENUMBER between 4pm to 7pm #typ … 98 | bdkFmSM rt @user help the people of the philippines . please call @user philippines telethon on PHONENUMBER between 4pm to 7pm #typ … 99 | bdkFmSM rt @user if your names diana , this song is about youuu url 100 | bdkFmSM rt @user listenin to my big bro @user on @user in the car on the way home ! sick job bro ! 101 | bdkFmSM rt @user @user @user @user good lad ! got some big tunes coming up for you bro ! shout me later ! ” cool … 102 | bdkFmSM rt @user no doubt about it ! @user is my favourite person ever ! she is hilarious on this show ! @user 103 | bdkFmSM rt @user that voice is unbelievable . hope ya go all the way fella @user 104 | bdkFmSM rt @user here we go @user ! was a lot of fun ! but watch this ! 105 | bdkFmSM rt @user there goes the most embarrassing moment of my life ! i asked king @user could i go to the toilet ! you just don't do that … 106 | bdkFmSM rt @user but in my defence . i would have never said anything if i didn't really need it ! the pain was just madness ! 107 | bdkFmSM rt @user it was great to be back home @user today ! love that place ! seeing all the crew and contestants and dermot ! so ner … 108 | bdkFmSM rt @user that's annoying ! 109 | bdkFmSM rt @user sick ! @user was incredible tonight , what a voice 110 | bdkFmSM rt @user cold monday morning in london ! was a great weekend ! 111 | bdkFmSM rt @user @user @user thankyou for making a dream come true for stacey ... url url 112 | bdkFmSM rt @user please help the people of the philippines . call @user philippines telethon on PHONENUMBER between 4pm to 7pm #typho … 113 | bdkFmSM rt @user busy busy day today ! not over yet though ! good laugh 114 | bdkFmSM rt @user cant wait for you guys to get your hands on diana today ... this is one of our favourite songs on mm ! enjoy ! url 115 | mpRnRvpn3EhGKVp what happens if i have an orthodontic emergency while i'm on vacation ? url 116 | mpRnRvpn3EhGKVp we hear this question a lot in our office - check out this week's blog for our answer ! url 117 | mpRnRvpn3EhGKVp our team is proud to offer our patients the cadent ioc digital impression system , which digitally captures the ... url 118 | mpRnRvpn3EhGKVp in this week's blog , we've created a helpful list of what we expect of you during orthodontic treatment ! url 119 | mpRnRvpn3EhGKVp do you have unused funds left in your flexible spending , insurance benefits , or health savings account ? now is ... url 120 | c06Cxk0UCk4Cq you're all welcome for me not re-rting every @user - related thing on my timeline . suffice it to say i'm proud of my 2nd alma mater . 121 | c06Cxk0UCk4Cq rt @user final : belmont NUMBER , north carolina NUMBER . 122 | c06Cxk0UCk4Cq glad to see i'm using this twitter thing the right way . url 123 | c06Cxk0UCk4Cq @user hating on the institute ... 124 | c06Cxk0UCk4Cq @user hates hippies @user trees weep on louisiana tech fb game days . game notes a whopping NUMBER pages . has to be fbs record . " 125 | c06Cxk0UCk4Cq rt @user don't be that person who overslept and missed gameday . #getup4gameday 126 | c06Cxk0UCk4Cq @user dang . so close . 127 | c06Cxk0UCk4Cq texans better draft a qb and it better not be manziel . 128 | c06Cxk0UCk4Cq who are the people wearing red ? 129 | c06Cxk0UCk4Cq @user your math is infallible . 130 | c06Cxk0UCk4Cq @user @user i suspect there are a lot more cookies available these days . 131 | c06Cxk0UCk4Cq @user i took him to class after our press conf today - he told me he needs to turn off his phone so he can get some homework done . 132 | c06Cxk0UCk4Cq if no one else will say it , i will : the NUMBER stanford baseball team played some atrocious defense . 133 | c06Cxk0UCk4Cq rt @user former baltimore raven o . j . brigance personifies true toughness : url i can't wait to read his book . 134 | c06Cxk0UCk4Cq @user url 135 | c06Cxk0UCk4Cq “ the dallas cowboys may be america ’ s team , but the houston oilers are texas ’ team . ” 136 | c06Cxk0UCk4Cq @user come on man it's not rocket surgery . 137 | c06Cxk0UCk4Cq @user that's the worst haiku i've read today . 138 | c06Cxk0UCk4Cq @user yeah whatever matt schaub does the same thing . 139 | c06Cxk0UCk4Cq @user highest-scoring quarter of the season for the feisty night birds . 140 | c06Cxk0UCk4Cq @user fifth time the owls have led at the end of the first quarter this season . 141 | c06Cxk0UCk4Cq saw a guy wearing google glass and did not punch him in the face . i hate having regrets but i guess i'll have to live with this one . 142 | c06Cxk0UCk4Cq i wasn't embarrassed until the niners brought in colt mccoy . now i'm nauseous . 143 | c06Cxk0UCk4Cq i miss the good old days when the texans ' biggest problem was the fat kicker . 144 | c06Cxk0UCk4Cq @user hater ! ( cc : @user 145 | c06Cxk0UCk4Cq rt @user oof rt @user no , this isn't at richard sherman's house . it's at a houston area restaurant . url 146 | c06Cxk0UCk4Cq rt @user brian mccann explains the unwritten rule that only the manager or pitching coach can get the pitcher . url 147 | c06Cxk0UCk4Cq just had some fruit and pretzels for dinner . #nodisrespecttobenaffleck 148 | c06Cxk0UCk4Cq sometimes i wonder if i'll ever get to tell my grandchildren about the time i saw georgia tech convert a 4th and short . 149 | c06Cxk0UCk4Cq @user i did unspeakable things to your office chair at this morning's meeting . 150 | c06Cxk0UCk4Cq @user mt @user no better #nfl team for sam mcguffie than #patriots . belichick would use him brilliantly . 151 | c06Cxk0UCk4Cq ed reed is playing today ?? i thought he was a coach . 152 | c06Cxk0UCk4Cq rt @user how can you call byu-utah " holy war " when the losing players aren't forced to transfer to the winning school ? 153 | c06Cxk0UCk4Cq two or three more seasons of announcers explaining what indisputable video evidence means , and maybe i'll finally grasp it . thanks guys ! 154 | c06Cxk0UCk4Cq oh , it's apparently that week when people care about baseball . has it been a year already ? 155 | c06Cxk0UCk4Cq there's always next time . url 156 | c06Cxk0UCk4Cq rt @user official forecast for tomorrow : NUMBER degrees , zero % chance of rain , light air conditioned breeze , NUMBER hot dogs url 157 | c06Cxk0UCk4Cq @user can't trust those elkins high school alumni . 158 | c06Cxk0UCk4Cq @user you have a 4s phone ? let me know how well it runs . 159 | c06Cxk0UCk4Cq rt @user if your friends and family are still sleeping we give your full permission to wake them up and tell them ... get up . it's … 160 | c06Cxk0UCk4Cq rt @user thank you for reading choose your own disaster by philip rivers ! 161 | c06Cxk0UCk4Cq rt @user ! 162 | c06Cxk0UCk4Cq #berman not trending yet ? it's early . 163 | c06Cxk0UCk4Cq @user NUMBER . ecu and mcneese state . rough one today ... 164 | c06Cxk0UCk4Cq @user baseball still going on ? thought it ended a week or so ago . maybe just al west . 165 | c06Cxk0UCk4Cq who doesn't love @user url 166 | c06Cxk0UCk4Cq @user you might be more worthless in montana than i was . 167 | c06Cxk0UCk4Cq @user anchor frown . 168 | c06Cxk0UCk4Cq tulsa getting into the spirit of #maction with a fake punt on its own NUMBER yard line . it went about as well as you might expect . 169 | c06Cxk0UCk4Cq shoutout to the girl pounding beers at whole foods at TIME this morning . 170 | c06Cxk0UCk4Cq how come no one has started a vin scully steroid rumor yet ? i mean ... that production at that age , @user and @user 171 | c06Cxk0UCk4Cq @user did you just send me a spam link on dm ? thanks bruh 172 | c06Cxk0UCk4Cq the longest line at the airport at NUMBER this morning was the one at pappasito's . because of course . 173 | c06Cxk0UCk4Cq citadel the funky homosapien , m . i . a . ( ohio ) #lowmajorrapnames #cbbrapnames 174 | c06Cxk0UCk4Cq bone thugs n harvard #lowmajorrapnames #cbbrapnames 175 | c06Cxk0UCk4Cq mc hampton #lowmajorrapnames 176 | c06Cxk0UCk4Cq house of penn #lowmajorrapnames 177 | c06Cxk0UCk4Cq cee lo bowling green #lowmajorrapnames 178 | c06Cxk0UCk4Cq lil ' troy #lowmajorrapnames 179 | c06Cxk0UCk4Cq NUMBER centenary #lowmajorrapnames 180 | c06Cxk0UCk4Cq @user this no-hitter would be much more fun if you were live-tweeting it . #jinx 181 | c06Cxk0UCk4Cq @user homer . 182 | c06Cxk0UCk4Cq @user did you get confirmation that they went ? i just watched the clip online but didn't see them in the front row for some reason . 183 | c06Cxk0UCk4Cq @user not yet , but i should ! then they will trade him . 184 | c06Cxk0UCk4Cq not saying this club is cool , but they did just play NUMBER songs from my running playlist . 185 | c06Cxk0UCk4Cq sunset at whistler . url 186 | c06Cxk0UCk4Cq whoosh ! there goes the last affordable flight . 187 | c06Cxk0UCk4Cq @user i sent explanations via dm . calm down , sir . 188 | c06Cxk0UCk4Cq @user happy birthday . i am bringing you some road wins from canada as a gift . 189 | c06Cxk0UCk4Cq kitsilano beach in vancouver . url 190 | c06Cxk0UCk4Cq stanley park in vancouver . url 191 | c06Cxk0UCk4Cq just had tacos for lunch in vancouver . living on the edge . 192 | c06Cxk0UCk4Cq rt @user the astros were NUMBER NUMBER when trailing after NUMBER innings before scoring NUMBER in the ninth against balfour , who had saved NUMBER in a … 193 | c06Cxk0UCk4Cq @user from rumors i've heard , you could write a horror film about the past week of your life . 194 | c06Cxk0UCk4Cq @user @user who ? 195 | c06Cxk0UCk4Cq @user i'm supposed to disagree with you , but having a hard time with that . 196 | c06Cxk0UCk4Cq @user you can add " taking a compliment with grace and humility " to the list . 197 | c06Cxk0UCk4Cq @user writing about a kid qualifying for the us amateur ... i realize now i should have gone to you for info , since you are the expert . 198 | c06Cxk0UCk4Cq hey @user i forgot already ... where did you say we were meeting up after the show tonight ? 199 | c06Cxk0UCk4Cq rt @user the rice owl : kind of a dick . url ( via @user 200 | c06Cxk0UCk4Cq @user happy anniversary !! 201 | c06Cxk0UCk4Cq last night's #madmen was crazy . pete campbell really went wild on his trip to los angeles . #sharknado 202 | c06Cxk0UCk4Cq rt @user dwight howard reportedly unhappy in houston 203 | c06Cxk0UCk4Cq just found out a local restaurant uses everclear in its #margaritas . can't decide if that's awesome or evil . 204 | c06Cxk0UCk4Cq rt @user temperature at first pitch in houston is NUMBER . but thankfully we have a roof and stuff . #astros 205 | c06Cxk0UCk4Cq @user we have multiple burrito options here , thank you very much . 206 | c06Cxk0UCk4Cq @user i can give isaiah some tips on some restaurants down here if he needs it . congrats to the racers ! 207 | c06Cxk0UCk4Cq @user glaring omission from lunchtime discussion . url 208 | c06Cxk0UCk4Cq never been prouder of the nephew than when he said dallas isn't really part of texas . 209 | c06Cxk0UCk4Cq y'all worked up over my old friend ? url 210 | c06Cxk0UCk4Cq @user i know military vehicles are popular there , but that's about it . url 211 | c06Cxk0UCk4Cq please , tell me more about the mean streets of akron , ohio . 212 | c06Cxk0UCk4Cq @user if this is the kind of crack analysis we can expect from you for #game7 , let me know now so i can preemptively unfollow you . 213 | c06Cxk0UCk4Cq headband for mvp 214 | c06Cxk0UCk4Cq @user i'll send the waaahhh-mbulance to pick you up . 215 | c06Cxk0UCk4Cq rt @user the heat's game notes are NUMBER pages . again , less is more . save a tree . #sidproblems 216 | c06Cxk0UCk4Cq rt @user boss move by san antonio . " oh , racist idiots are complaining about sebastien de la cruz ? come on back for game NUMBER , son . " 217 | c06Cxk0UCk4Cq before just now , i'm not sure gary neal even knew they did player interviews at halftime . 218 | c06Cxk0UCk4Cq #ricebaseball finished the season with a NUMBER NUMBER record in NUMBER inning games . 219 | c06Cxk0UCk4Cq @user saw the release . you are so bigtime . 220 | c06Cxk0UCk4Cq #rice to the super regionals and #astros sweep angels . #baseballtown 221 | e6JPXvvDet3o5y lol ... wtf ... haven't been on this twitter account for ages ... url 222 | e6JPXvvDet3o5y @user wtf why report me ? not been active in months and there are accounts on here who post it every day 223 | e6JPXvvDet3o5y just completed a NUMBER km run with @user check it out ! url #runkeeper 224 | e6JPXvvDet3o5y woop fatso in the house url 225 | e6JPXvvDet3o5y @user @user shes beautiful not me ... 226 | e6JPXvvDet3o5y @user @user kinda is ... 227 | e6JPXvvDet3o5y @user aww i shudnt be though cos am seriously really fat ... 228 | e6JPXvvDet3o5y @user @user i didnt ... am not beautiful ... just fat and ugly and not to mention a waste of space ... 229 | e6JPXvvDet3o5y @user i am and all i can see is fat : l aww really ? 230 | e6JPXvvDet3o5y @user xxx 231 | e6JPXvvDet3o5y im not kidding ... im so so fat ... @user 232 | e6JPXvvDet3o5y i am so fucking fat url 233 | e6JPXvvDet3o5y url 234 | e6JPXvvDet3o5y @user thank you x 235 | e6JPXvvDet3o5y when you have a nightmare about being in treatment for an eating disorder you are still in denial about ... 236 | e6JPXvvDet3o5y @user love you too 237 | e6JPXvvDet3o5y #flipagram created using @irighx1kf ♫ music : all time low - six feet under the stars happy birthday & lt ; 3 @user url 238 | e6JPXvvDet3o5y red = s'h , orange = bpd , yellow = ocd , green = social anxiety , blue = ed ( anorexia ) and purple / black = depression url 239 | e6JPXvvDet3o5y @user trying too , stay strong too gorgeous & lt ; 3 240 | e6JPXvvDet3o5y / : url 241 | e6JPXvvDet3o5y @user aw thanks , wanna be thinner haha 242 | e6JPXvvDet3o5y @user really ?? 243 | e6JPXvvDet3o5y url 244 | e6JPXvvDet3o5y @user will do gorgeous & lt ; 3 245 | e6JPXvvDet3o5y @user NUMBER NUMBER :/ 246 | e6JPXvvDet3o5y @user i will do , cba to work tmrw & lt ; 3 247 | e6JPXvvDet3o5y @user i love you loads more my beautiful best friend & lt ; 3 248 | e6JPXvvDet3o5y rt @user cross the line if you feel like your parents are never proud no matter how hard you try . 249 | e6JPXvvDet3o5y @user feel so fat ... 250 | e6JPXvvDet3o5y no more fucking school ever ( ' : 251 | e6JPXvvDet3o5y @user will try and believe that xxx 252 | e6JPXvvDet3o5y @user its still to fat for it too be perfect : 253 | e6JPXvvDet3o5y me after purging ... url 254 | e6JPXvvDet3o5y @user it is ? & lt ; 3 255 | e6JPXvvDet3o5y my fatso of a body ... url 256 | e6JPXvvDet3o5y rt @user i want to keep going , and show those people who said id never be skinny , those who called me fat , those who made me feel … 257 | e6JPXvvDet3o5y url 258 | e6JPXvvDet3o5y crying to this song and harming ... url 259 | e6JPXvvDet3o5y rt @user it's kinda unfair cause most girls are perfect and have perfect bodies and perfect everything and everything comes so easy … 260 | e6JPXvvDet3o5y url 261 | e6JPXvvDet3o5y i honestly wish i could die ... people might tell me im beautiful and perfect or that im skinny but im … url 262 | e6JPXvvDet3o5y non existent collar bones ... url 263 | e6JPXvvDet3o5y @user aww thank you but i wasnt x 264 | e6JPXvvDet3o5y @user x * 265 | e6JPXvvDet3o5y @user aw thank you but wanna be skinnier tbh c 266 | e6JPXvvDet3o5y @user tbh just feel and look so fat in my opinion / : 267 | e6JPXvvDet3o5y @user i am ? / : 268 | e6JPXvvDet3o5y if anyone needs advice on how to stay this fat NUMBER dm me x url 269 | e6JPXvvDet3o5y @user thank you so much and same for you x dw about if cant talk much cos have exams as well , but ily & lt ; 3 270 | e6JPXvvDet3o5y @user thank you so much gorgeous ! & lt ; 3 271 | e6JPXvvDet3o5y hope i die tonight ... 272 | e6JPXvvDet3o5y i love you all but goodbye ... url 273 | e6JPXvvDet3o5y no way am eating dinner til tmrw night ... 274 | e6JPXvvDet3o5y @user & lt ; 3 275 | e6JPXvvDet3o5y i've really fucked up ... 276 | e6JPXvvDet3o5y (: url 277 | e6JPXvvDet3o5y @user will try that deffo and i do ... i really want to carry on harming on my arm atm ... url 278 | e6JPXvvDet3o5y @user i do deserve the pain ... i always do ... 279 | e6JPXvvDet3o5y @user will clean and wrap them well , dunno how will do ones on my stomach ... 280 | e6JPXvvDet3o5y im not okay ... url 281 | e6JPXvvDet3o5y let me die ): url 282 | e6JPXvvDet3o5y it really is ... url 283 | hMuEjR7TAHw hunger games tomorrow yaaay 284 | hMuEjR7TAHw rt @user me when someone texts me during a lakers game url 285 | hMuEjR7TAHw @user petty for laughin at that imessage ... 😂 286 | hMuEjR7TAHw rt @user i want sum krekel's lemon ice cream wit sprinkles mayne ! 287 | hMuEjR7TAHw tomorrow is my frrriday 288 | hMuEjR7TAHw rt @user @user 💊 💉 💁 289 | hMuEjR7TAHw rt @user dwight still hasn't missed a shot ... 290 | hMuEjR7TAHw @user my arthritis medicine ain't kicked in yet 291 | hMuEjR7TAHw rt @user @user 😂 😂 shouldn't u be sleep 292 | hMuEjR7TAHw why is ig full of tbt already ... it's wednesday bitch 😒 293 | hMuEjR7TAHw @user 😩 😂 294 | hMuEjR7TAHw @user i'm into designer 💩 ” you need to be worried about taking shits ... witcha old ass . 295 | hMuEjR7TAHw lakers don't play til friday so i sit here at night twiddling my thumbs 😩 💜 💛 296 | hMuEjR7TAHw and by who i meant whole ... 297 | hMuEjR7TAHw i'm eating a hershey pie 😁 😁 😁 😁 298 | hMuEjR7TAHw who tl eats " puxxy " apparently 299 | hMuEjR7TAHw somebody come take my garbage out 300 | hMuEjR7TAHw rt @user the next drawn on eye brows i see i'm erasing them bitxhes off 😂 😂 301 | hMuEjR7TAHw i get off in NUMBER minutes !!! 302 | hMuEjR7TAHw my kneecap hurts 303 | hMuEjR7TAHw let me find out @user on here flirtin with lames 😂 😂 😂 304 | hMuEjR7TAHw #wcw ... always @user 😍 ❤ ️ url 305 | hMuEjR7TAHw 😞 306 | hMuEjR7TAHw insomnia 307 | hMuEjR7TAHw anybody got some green beans i can borrow ? 308 | hMuEjR7TAHw when did twitter update ? 309 | hMuEjR7TAHw one day i'll sleep ... 310 | hMuEjR7TAHw rt @user my nigga url 311 | hMuEjR7TAHw @user or somebody could just loan me the dvd lol ... 312 | hMuEjR7TAHw will somebody let me borrow the original best man movie ??? 313 | hMuEjR7TAHw @user 👀 👀 👀 314 | hMuEjR7TAHw rt @user black titanic theme song url 315 | hMuEjR7TAHw rt @user i want these kds  316 | hMuEjR7TAHw i'm hungry and all i have to eat is junk food 317 | hMuEjR7TAHw @user @user ahahaha 318 | hMuEjR7TAHw he was doing a wrestling move in the first pic lol ❤ ️ #wwe #cenanation #johncena url 319 | hMuEjR7TAHw very true #repost url 320 | hMuEjR7TAHw why am i still up 321 | hMuEjR7TAHw rt @user congratulations to @user on winning the @user intercontinental title ! you're really coming into your own , big ma … 322 | hMuEjR7TAHw omg love ❤ ️ ❤ ️ ❤ ️ monicamylife's video url 323 | hMuEjR7TAHw 👯 👯 👯 👯 324 | hMuEjR7TAHw @user hahaha the thirst was still so real . you're gorgeous annnd that's the best song on the album 325 | hMuEjR7TAHw rt @user @user i have on a black onesie ... i'm not that bold lol . 326 | hMuEjR7TAHw @user if my kids don't get free lunch ima send em to school with a pack of noodles ” 😩 😫 😂 😭 327 | hMuEjR7TAHw 😂 😂 😂 that buddies song made niggas cry back in the day though 328 | hMuEjR7TAHw rt @user y'all remember this " buddies they make ya feet feel fine . buddies ... they cost a dollar NUMBER . " 329 | hMuEjR7TAHw my facebook inbox has seen a substantial random negro increase in the last week ... i think that carlton shirt did it #niggaslovefreshprince 330 | hMuEjR7TAHw shaneka adams butt naked on ig listening to @user ... #thirsttraptuesdays 331 | hMuEjR7TAHw @user 😩 😫 😩 😫 😩 332 | hMuEjR7TAHw i want an ice cold coke 333 | hMuEjR7TAHw @user tweet me so i know it's real 😂 😂 334 | hMuEjR7TAHw @user they don't cost 4hun playa so drop that extra NUMBER in my mailbox 335 | hMuEjR7TAHw @user i was just being petty cuz i seen you in my tl 😜 😜 336 | hMuEjR7TAHw @user she's just my #wceveryday 😩 😍 337 | hMuEjR7TAHw rt @user @user i think you kinda bicurious for @user 😘 👭 338 | hMuEjR7TAHw @user let me find out you can't see emojis 😳 339 | hMuEjR7TAHw i'm glad this foo finally ditched that santana mess - ___ - 340 | hMuEjR7TAHw @user 👯 👯 👯 341 | hMuEjR7TAHw y'all rt'n i hope y'all ain't the police 🚨 🚓 🚔 342 | hMuEjR7TAHw @user 👀 👀 343 | hMuEjR7TAHw i want a blow pop 344 | hMuEjR7TAHw #oomf asked me what i'm doing up i told him i sell crack i never sleep #thuglife 345 | hMuEjR7TAHw rt @user i love callin ppl pussies ! 346 | bImxiZqnzVuzju3 cuz it's la push baby , it's la push ! 347 | bImxiZqnzVuzju3 rt @user help isf win NUMBER 0k for my NUMBER th birthday !! url please ! xo ” 348 | bImxiZqnzVuzju3 rt @user i am grateful for the bad / unlucky days , because they allow for me to recognize and appreciate the good ones . ” 349 | bImxiZqnzVuzju3 @user yeah ! i went last year and it was crazy 😂 350 | bImxiZqnzVuzju3 #menitonsomeoneyourethankfulfor @user @user i'm so thankful to have you two as an idol ! ilysm ❤ ️ 351 | bImxiZqnzVuzju3 going to the mall at NUMBER am tomorrow 😜 hopefully i won't die 😂 #blackfriday 352 | bImxiZqnzVuzju3 rt @user " i know , better than anybody else a version of rob . " - kristen url 353 | bImxiZqnzVuzju3 i have my two dogs on my lap lol ! they're both asleep ☺ ️ 🐶 354 | bImxiZqnzVuzju3 rt @user what is the perfect trio ? rt on robert , kristen and taylor . fav by josh , liam and jennifer . url 355 | bImxiZqnzVuzju3 no humanity elena #thevampirediaries @user 356 | bImxiZqnzVuzju3 😂 😂 😂 😂 😂 happy gobble day ! #kristenstewart #robertpattinson #ninadobrev #iansomerhalder #kris url 357 | bImxiZqnzVuzju3 happy thanksgiving !! 🍗 🍗 🍗 358 | bImxiZqnzVuzju3 rt @user twilight breaking dawn part NUMBER charms & gt ; & gt ; & gt ; url 359 | bImxiZqnzVuzju3 thanks so much for everyone who participated in #welovetwilightday ! we'll have to do it again next year ! 360 | bImxiZqnzVuzju3 rt @user i wish i was able to do my hair in a pretty way , so i could wear this . #welovetwilightday url 361 | bImxiZqnzVuzju3 rt @user #welovetwilightday [ my bedroom ] . url 362 | bImxiZqnzVuzju3 rt @user haciendo maraton de #twilighters con @user & amp ; @user #welovetwilightday el mejor aniversario del mundo ♥ h … 363 | bImxiZqnzVuzju3 rt @user hold up . today was #welovetwilightday ? ! ? !! ? ! ??? what the why the how the ... i missed ittty ? ! ? ! ? ! ¡¿ ! ? ! ? ! ? ! 364 | bImxiZqnzVuzju3 rt @user ok guy's so here is the next watch along pls rt url 365 | bImxiZqnzVuzju3 rt @user url #welovetwilightday 366 | bImxiZqnzVuzju3 rt @user url : #welovetwilightday 367 | bImxiZqnzVuzju3 rt @user photo : #welovetwilightday ! =) i have an edward pillowcase and blanket , too , from #breakingdawnpart1 , but i ... url 368 | bImxiZqnzVuzju3 rt @user photo : i can ’ t believe i completely forgot to include these things in my first #welovetwilightday post ! ... url 369 | bImxiZqnzVuzju3 rt @user #welovetwilightday #twilightsaga ♡♡ url 370 | bImxiZqnzVuzju3 rt @user #welovetwilightday url : 371 | bImxiZqnzVuzju3 rt @user #welovetwilightday #twilightforever #1yearofbreakingdawnpart2 #twihardsstayforever #twilightwillalwaysbemysaga url 372 | bImxiZqnzVuzju3 rt @user #welovetwilightday url 373 | bImxiZqnzVuzju3 rt @user since it's #welovetwilightday i decided to put on my twilight gear my mrs cullen t-shirt and NUMBER headbands url 374 | bImxiZqnzVuzju3 rt @user #welovetwilightday #mytwilightcollection it's not much but i still love it . i need get a couple of things still . http :/ … 375 | bImxiZqnzVuzju3 rt @user wearing my twilight necklace ❤ ️ #welovetwilightday url 376 | bImxiZqnzVuzju3 rt @user #welovetwilightday forever ! url 377 | bImxiZqnzVuzju3 rt @user shiiiaat !!! @user cc @user it's time !! ♥ @user #welovetwilightday #teamvolturi !! url 378 | bImxiZqnzVuzju3 rt @user #welovetwilightday url 379 | bImxiZqnzVuzju3 rt @user it's november NUMBER !!! #twilightsaga #welovetwilightday #welovetwilight #bestmovie … url 380 | bImxiZqnzVuzju3 rt @user everything about twilight is changed my life @user : this book changed my life #welovetwilightday url 381 | bImxiZqnzVuzju3 rt @user happy we love twilight day #welovetwilightday #1yearofeternity #twilightwillalwaysbemysaga #twilightersarehere url 382 | bImxiZqnzVuzju3 rt @user today's #welovetwilightday on my twilight calender ;) url 383 | bImxiZqnzVuzju3 rt @user NUMBER year since this beautiful film hit the big screens #welovetwilightday url 384 | bImxiZqnzVuzju3 rt @user #welovetwilightday #teamvolturi !! url 385 | bImxiZqnzVuzju3 rt @user today is #welovetwilightday ! read and do it twihards ! :d url 386 | bImxiZqnzVuzju3 rt @user wear your twilight merchandise and tag it as #welovetwilightday and join the bd2 watch along ! #bd2watchalong url 387 | bImxiZqnzVuzju3 rt @user no tendre a mis amigas , pero tengo a la mejores hermanas twilighters #welovetwilightday 388 | bImxiZqnzVuzju3 rt @user this book changed my life #welovetwilightday url 389 | bImxiZqnzVuzju3 #welovetwilightday video 💗 url 390 | bImxiZqnzVuzju3 rt @user lloro sola porque mis amigas no entienden lo que siento #welovetwilightday #forever 391 | bImxiZqnzVuzju3 rt @user no puedo creer que ya haya pasado un año #welovetwilightday 392 | bImxiZqnzVuzju3 rt @user my #welovetwilightday url 393 | bImxiZqnzVuzju3 oh my gosh ! so i haven't been on twitter in forever and i check it today and #welovetwilightday has … url 394 | bImxiZqnzVuzju3 hey guys ! it's #welovetwilightday ! don't forget to post a pic of your merch or anything twilight … url 395 | bImxiZqnzVuzju3 rt @user #welovetwilightday 396 | bImxiZqnzVuzju3 rt @user why not ? twihard , do it ! :d #rt #welovetwilightday url 397 | bImxiZqnzVuzju3 rt @user " guys !! do this if you love twilight @user let's do this twihards ! #welovetwilightday url " 398 | bImxiZqnzVuzju3 rt @user let's do this twihards ! #welovetwilightday url 399 | bImxiZqnzVuzju3 rt @user #welovetwilightday ! url 400 | bImxiZqnzVuzju3 rt @user #welovetwilightday i've got my bracelet on !! do you have your twilight merchandise on today ??? url 401 | bImxiZqnzVuzju3 rt @user #welovetwilightday one of my favorite edward quotes on my back :d url 402 | bImxiZqnzVuzju3 rt @user my twilight bracelet is always on my wrist ! #welovetwilightday url 403 | bImxiZqnzVuzju3 rt @user well , here we go . that's what i'm wearing for #welovetwilightday . i made it for bd1 premiere . there's … url 404 | bImxiZqnzVuzju3 rt @user #welovetwilightday #welovetwilightday #welovetwilightday #welovetwilightday #welovetwilightday ♥♥ ♥ 405 | bImxiZqnzVuzju3 go participate in #welovetwilightday ! url 406 | bImxiZqnzVuzju3 so it's #welovetwilightday ( i started it on ig ) and it's amazing that it's reached twitter ! ty for every1 who has been participating ! 407 | bImxiZqnzVuzju3 how can you feel so alone when there's over NUMBER billion people on the same planet ? 408 | bImxiZqnzVuzju3 rt @user " dear diary . do you ever get sick of me writing about death ? " url best quotes from #tvd url 409 | bImxiZqnzVuzju3 rt @user don't forget to vote for ian and nina #pca #dobrevics #somerholics #delena #tvd #ian #nina url 410 | bImxiZqnzVuzju3 rt @user crying so hard u could probably surfboard on my tears url 411 | bImxiZqnzVuzju3 rt @user so apparently we need to save some feels for later , NUMBER pm 412 | bImxiZqnzVuzju3 rt @user there's a #twilight marathon coming to theaters : url 413 | bImxiZqnzVuzju3 rt @user i need robsten . you need robsten . he / she needs robsten . we need robsten . they need robsten . everybody need robsten . 414 | bImxiZqnzVuzju3 forever refreshing ... #robstenhalloween #robsten 415 | bImxiZqnzVuzju3 so apparently new rk pics are coming !! 😱 we just gotta keep waiting 😁 😁 😁 416 | bImxiZqnzVuzju3 rt @user let me calm the hell down because i think we are gonna have to wait a little for pics . i am just glad they exist . :) 417 | bImxiZqnzVuzju3 rt @user #theoriginals had a NUMBER rating & amp ; NUMBER million viewers last night :) url @user 418 | bImxiZqnzVuzju3 rt @user new pics ! rk !!! coming !!! 419 | bImxiZqnzVuzju3 the originals tonight ! yay ! 420 | bImxiZqnzVuzju3 rt @user more bloopers and omg kristen at the end pmsl :') url 421 | bImxiZqnzVuzju3 rt @user #katherinepierce : do you have any idea what it's like to run in heels ? #tvd url 422 | bImxiZqnzVuzju3 rt @user #klausmikaelson : i wanna be the king . #tvd #theoriginals url 423 | bImxiZqnzVuzju3 anytime klaus ;) #theoriginals url 424 | bImxiZqnzVuzju3 rt @user r i o t : ( r ) obsten ( i ) s ( o ) kay ( t ) ogether :) @user : if you guys missed it : the riot girl ;) #badass url 425 | bImxiZqnzVuzju3 come on you little contact ... get in my eye please . 426 | bImxiZqnzVuzju3 got candy blood bags and the after taste is disgusting !! 😖 😝 427 | bImxiZqnzVuzju3 my life 🙍 url 428 | bImxiZqnzVuzju3 @user delena of course 💁 and klaroline 429 | bImxiZqnzVuzju3 @user yeah i'm all caught up ! last nights episode was so sad 😭 bonnie's funeral omg ... 430 | fiRnovAqg16fl @user that exactly how my last roommate was ... - ___ - 431 | fiRnovAqg16fl @user yeah . i'll be back in michigan on the NUMBER 4th so anytime after that definitely !! 432 | fiRnovAqg16fl @user :-) . ps . i miss you ! 433 | fiRnovAqg16fl rt @user don't be an ass , retweet 🙏 url 434 | fiRnovAqg16fl @user yeah ! i'm glad i did ! i will try it ! 435 | fiRnovAqg16fl this is going to be my firsts major holiday being a vegetarian ... let the pressure commence 436 | fiRnovAqg16fl soup ! 437 | fiRnovAqg16fl i should feel bad . but i was so tired . and i had memorizing to do . so i was still productive 438 | fiRnovAqg16fl slept through my last last to classes today ... guilt = NUMBER 439 | fiRnovAqg16fl TIME 440 | fiRnovAqg16fl very 1st tweet from my new iphone 5s !! 441 | fiRnovAqg16fl also my dad is coming to seattle tomorrow !!! i'm so excited ! 442 | fiRnovAqg16fl night world . 443 | fiRnovAqg16fl my 2nd shakespeare scene is due tomorrow ... i got this . #othello 444 | fiRnovAqg16fl @user yeah the picked for christian jamie dornan . and dakota johnson for anastasia 445 | fiRnovAqg16fl if you want my opinion . lol just saying 446 | fiRnovAqg16fl he's supposed to be sexy and mysterious not scrawny and creepy 447 | fiRnovAqg16fl and i hate when people categorize other people on just looks . but this is not the right man for the job ... 448 | fiRnovAqg16fl i really love NUMBER shades of gray . they totally picked the wrong guy to play christian . he does not look the part ... 449 | fiRnovAqg16fl @user ooo !! yes please lol 👍 👍 450 | fiRnovAqg16fl @user but i can't wait to hear all about it :-) 451 | fiRnovAqg16fl @user :) believe it or not most of my adventures happened in mi this summer lol crazy i bet you love life is way better than mine 452 | fiRnovAqg16fl @user :-) maybe we can catch up when i come home ? 453 | fiRnovAqg16fl @user oh school ... lol . i'm glad that you are doing well :-) 454 | fiRnovAqg16fl @user how are you ? i do really miss you 455 | fiRnovAqg16fl @user just trying to make it through this semester of school lol . but other than that i'm really good . happy . which is nice :-) 456 | fiRnovAqg16fl guess who just started listening to christmas music ? ! ? ! this girl 👋 457 | fiRnovAqg16fl @user thank you ! i miss you too sister-wifey a lot !!! 458 | fiRnovAqg16fl @user i would if i was home :-( 459 | fiRnovAqg16fl watching love actually and taking a bath ! perfect end to a day 460 | fiRnovAqg16fl rt @user it ’ s christmas next month it ’ s christmas next month it's christmas next month christmas next month . 461 | fiRnovAqg16fl strange ass people 462 | fiRnovAqg16fl rt @user we accept the love we think we deserve #perksofbeingawallflower 463 | fiRnovAqg16fl rt @user rimonabant , aka “ reverse marijuana ” , is a drug that has the exact opposite effects of thc -- such as suppressed appetite and … 464 | fiRnovAqg16fl @user ohh horry !!! 😘 😍 😍 😘 465 | fiRnovAqg16fl @user what ? that's crazy . well not really because it's you , you world traveller . that's soo cool . have fun :-) 466 | fiRnovAqg16fl @user right . haha . i'm just glad i made it home . yeah ! where did you go ? 467 | fiRnovAqg16fl @user yeah . i walked like around them and they started like cat calling and saying things ... thanks :-) i miss you 468 | fiRnovAqg16fl @user yeah just a little shook up once i heard what they were saying . but i'm ok now 469 | fiRnovAqg16fl @user yeah from the dorms i got away from the 1st guy then a block later there were NUMBER more guys that followed me all the way home 470 | fiRnovAqg16fl well , being followed all the way home wasn't the best thing to happen tonight ... - __ - #happyhalloween 471 | fiRnovAqg16fl @user haha i miss you 472 | fiRnovAqg16fl TIME 473 | fiRnovAqg16fl watching my favorite play in the world our town , it's intermission , and i am so proud of the cast ! great job 474 | fiRnovAqg16fl rt @user according to an english wedding belief , if an unmarried woman places a slice of fruitcake under her pillow she'll dream of h … 475 | fiRnovAqg16fl @user haha there it goes 476 | fiRnovAqg16fl hearing your teacher say that you have a future in shakespeare ... best feeling ever > & gt ; & gt ; & gt ; & gt ; 477 | fiRnovAqg16fl @user that's so good that you are liking working in the " real world " lol i miss you 478 | fiRnovAqg16fl @user matthew !! school is good . challenging bet good :-) . how's teaching ? 479 | fiRnovAqg16fl when the kids at work ask me why i was leaving and i say i have school and they say can't you stay a little longer 😔 480 | fiRnovAqg16fl drinking a cup of warm cider , in my " it's all good " mug before going to bed . #itsallgood 481 | fiRnovAqg16fl TIME 482 | fiRnovAqg16fl i love seeing theatre 483 | fiRnovAqg16fl @user baboons have been known to kidnap puppies and raise them as pets . ” awww that would be really cute if they weren't kidnaped 🐶 🐵 484 | fiRnovAqg16fl maybe next year will be my year ... #hope #lookingonthebrightside #maybe 485 | fiRnovAqg16fl saw the servant of two masters this morning ... can i just say it was fantastic 486 | fiRnovAqg16fl i love camp #confessionhour 487 | fiRnovAqg16fl my dad is my favorite person in the world if i we don't speak everyday i assume the worst even tho i know nothings wrong #confessionhour 488 | fiRnovAqg16fl i try not to regret things and just live for the moment #confessionhour 489 | fiRnovAqg16fl i wish i could hang out with @user more #confessionhour 490 | fiRnovAqg16fl anddd done ! 491 | fiRnovAqg16fl i'm almost done tho . i just need to stop multitasking and just do it 492 | fiRnovAqg16fl this paper will be done before NUMBER … i declare it so 493 | fiRnovAqg16fl why in the world would they put a dance studio above a piano class room ... dancers for non majors have heavy feet 494 | fiRnovAqg16fl today in class my teacher said something was twerking in place of tweaking . she only caught herself when the class burst out in laughter 495 | fiRnovAqg16fl i've been in such a country mood for like the last two weeks . idk what it is but i like it ! itunes country radio on blast 496 | fiRnovAqg16fl @user i miss you 497 | fiRnovAqg16fl TIME 498 | fiRnovAqg16fl ooo i hate freshmen 499 | fiRnovAqg16fl oh children have no filters 500 | fiRnovAqg16fl " i can still see your freckles , even though you have dark skin ... " - little NUMBER year old i work with ... 501 | fiRnovAqg16fl ooh tricky tricky 502 | fiRnovAqg16fl rt @user lift your right foot & amp ; rotate it in small circles clockwise . now raise your right hand and write NUMBER in the air . your foot … 503 | fiRnovAqg16fl i just love that i get pushed to the side ... just sayin hahaha 504 | fiRnovAqg16fl i don't love you anymore ... #thingsyouneverwanttohear #alsostillbitter years later #overit 505 | fiRnovAqg16fl " last night was great and all , but that's all it was ... " #thingsyouneverwanttohear #stillbitter #jerk haha note to self #neveragain #overit 506 | fiRnovAqg16fl rt @user if they say " why ? why ? " tell em that is human nature . 507 | fiRnovAqg16fl ... blah ... 508 | fiRnovAqg16fl they sounded a lot better in my head 509 | fiRnovAqg16fl NUMBER thousand first dates #addawordruinamovie 510 | fiRnovAqg16fl of dead mice and men #addawordruinamovie 511 | fiRnovAqg16fl " i'm bad and that's good . i will never be good and that's not bad . there's no one i'd rather be than me " wreck it ralph ❤ ️ 512 | fiRnovAqg16fl watching wreck it ralph and eating chinese ... ahh i love the weekend 513 | fiRnovAqg16fl my teacher brought in donuts for class ... so good 514 | fiRnovAqg16fl @user today is world vegetarian day ! ” yay ! 515 | fiRnovAqg16fl @user ha . ha . ha . very funny ... not ... you have too much time on your hands 516 | fiRnovAqg16fl @user haha way too much drake 517 | fiRnovAqg16fl @user guess who's it is ... 518 | fiRnovAqg16fl blahh 519 | fiRnovAqg16fl on another note only NUMBER days till christmas break 520 | fiRnovAqg16fl also it is no longer jacket season . now i have to bring out the coat . :-( #summercomeback 521 | fiRnovAqg16fl crushing hard ... smh 522 | fiRnovAqg16fl such a great movie 523 | fiRnovAqg16fl we accept the love we think we deserve … 524 | fiRnovAqg16fl you are a freaking sophomore ... you know nothing ! 525 | fiRnovAqg16fl when sophomores call themselves upperclassmen < hahahahahahha hahahahaha i just wanna punch them in the face hahahah not cool #underclassmen 526 | fiRnovAqg16fl i just wanna go home ... it's not like school is bad i love school . i'm just tired of all the bullshit 527 | fiRnovAqg16fl all i want is my cheesecake ... 528 | fiRnovAqg16fl i'm not stupid ... 529 | fiRnovAqg16fl i don't know if i can do this anymore 530 | fiRnovAqg16fl rude ass *** 531 | wS2JiUPRyQ66 hahaha cheeky kick over the shoulder for the baabaas ! 532 | wS2JiUPRyQ66 bring me the holy blowtorch ! url 533 | wS2JiUPRyQ66 this is the first time that i have walked to an all saints day service in a sleeveless summer dress . 534 | wS2JiUPRyQ66 make up came off , hair sorted itself out , relatively sober and relatively early night means that i can sing today ! #allsaintsday 535 | wS2JiUPRyQ66 helphelphelp url 536 | wS2JiUPRyQ66 @user i'm singing tomorrow ! solo and unaccompanied ... 537 | wS2JiUPRyQ66 @user ta duck ! 538 | wS2JiUPRyQ66 @user clairette de die , i'm afraid ! ( guilty pleasure ) 539 | wS2JiUPRyQ66 good evening ! url 540 | wS2JiUPRyQ66 suddenly don't need a wee ... url 541 | wS2JiUPRyQ66 if anyone wants to see if this makeup washes off , i'm singing solo at st helen's tomorrow at noon . 542 | wS2JiUPRyQ66 you're lucky , he's lucky , i'm lucky , we're all lucky ! #magenta #rhps url 543 | wS2JiUPRyQ66 @user there'll be pictures ... 544 | wS2JiUPRyQ66 t-1hr til a party , and my hair is in rollers and i'm sewing like a woman possessed ! 545 | wS2JiUPRyQ66 gluten free fans - @user lasagne sheets are the absolute best . you don't need to precook them either ! 546 | wS2JiUPRyQ66 my top NUMBER #lastfm artists : cream ( NUMBER ) , peter philips ( NUMBER ) & amp ; william byrd ( NUMBER ) #lastfm url 547 | wS2JiUPRyQ66 @user to ed , love from @user url 548 | wS2JiUPRyQ66 @user url 549 | wS2JiUPRyQ66 @user nah , just a chap who's not used to it , i expect . 550 | wS2JiUPRyQ66 raced home to be in bed by shipping forecast time , and i think a robot is doing it :( 551 | wS2JiUPRyQ66 @user until i had made my choice , i thought about letting my hair go greasy , put on a shirt and tie and take an exploding carrot ... 552 | wS2JiUPRyQ66 @user here's a new one on the ' let's do a sexy version of everything ' front url 553 | wS2JiUPRyQ66 @user i read it at my dad's funeral , too . it's a beautiful poem . 554 | wS2JiUPRyQ66 @user yes , thanks . it just threw me . 555 | wS2JiUPRyQ66 rt @user yep , i don't look like a physicist either mt @user heels , hair ... @user bemoans scrutiny of looks url 556 | wS2JiUPRyQ66 just heard remember by christina rossetti on radio NUMBER . i've avoided it for NUMBER years , but i still remember every word , and it still upsets me . 557 | wS2JiUPRyQ66 @user are they your pyjamas , ready for bed ? :d 558 | wS2JiUPRyQ66 @user amazing bassist , great songwriter ! 559 | wS2JiUPRyQ66 jack bruce has died . that's really sad - i have happy memories of listening to cream with my dad . 560 | wS2JiUPRyQ66 @user @user url 561 | wS2JiUPRyQ66 @user @user there's a korean bbq in leeds ... 562 | wS2JiUPRyQ66 happy NUMBER 0th birthday , @user ( these wishes have increased in value since i erroneously wished them the other day ... ) 563 | wS2JiUPRyQ66 @user @user major flaw in that plan : i get all of my iron back ! 564 | wS2JiUPRyQ66 @user @user i should get you a care package together , really . 565 | wS2JiUPRyQ66 @user in the process of donating platelets . 566 | wS2JiUPRyQ66 @user you might get some , but you get most of your blood back in the process . 567 | wS2JiUPRyQ66 but whole blood donations are easier , less often , more biscuited ( trying to not base this decision around biscuit choices ) . 568 | wS2JiUPRyQ66 platelets are more valuable , trips to leeds are nice ( ooo evensong somewhere different ) , and i can work whilst being hooked up . 569 | wS2JiUPRyQ66 so , i am eligible to give platelets . do i take most of a day to go to leeds hospital to donate , or do i stick to giving blood in york ? 570 | wS2JiUPRyQ66 @user quite ! 571 | wS2JiUPRyQ66 @user sorry to hear that . hope it gets sorted . 572 | wS2JiUPRyQ66 rt @user here ’ s what a vinyl record and stylus looks like magnified NUMBER 0x . url url 573 | wS2JiUPRyQ66 look up to the rooftops , not down at your feet . url 574 | wS2JiUPRyQ66 @user a smattering of target bombarded by a pissstreak of beam . hmmm . 575 | wS2JiUPRyQ66 @user neutrons , surely ? ; - ) 576 | wS2JiUPRyQ66 @user is that something to do with pulsars ? 577 | wS2JiUPRyQ66 @user how many centimetres are in a metre , ac ? i should probably get this right . 578 | wS2JiUPRyQ66 today , i can't even do calculations between lengths and densities with different units . there is no face and palm big enough . 579 | wS2JiUPRyQ66 my top NUMBER #lastfm artists : thomas tomkins ( NUMBER ) , the dandy warhols ( NUMBER ) & amp ; crist óbal de morales ( NUMBER ) #lastfm url 580 | wS2JiUPRyQ66 @user thanks ! in the mean time , i'll get a permanent marker and put a big ol ' dot on the black pairs . 581 | wS2JiUPRyQ66 @user you know the labels inside your tights ? could you mark colour on them ? i spend too long trying to tell navy from black ! 582 | wS2JiUPRyQ66 rt @user this is rather fantastic . a coroner ’ s answers to idiotic prosecutorial questions . ( via @user url 583 | wS2JiUPRyQ66 happy friday , everyone . it's been a long week ! 584 | wS2JiUPRyQ66 i just wish that whatever the jingle is that follows the shipping forecast wasn't so jarring . always wakes me back up again . 585 | wS2JiUPRyQ66 after pressing all of the buttons in various order on my dab radio , i've managed to reset it ! i can now fall asleep to radio NUMBER . a victory . 586 | wS2JiUPRyQ66 @user awfully kind . it's getting there . 587 | wS2JiUPRyQ66 @user thanks , and thank you for letting us use your room . 588 | wS2JiUPRyQ66 @user hello , i've left bach's agnus dei on your music stand - sorry ! i'm at evensong tomorrow , or chris can get it back to me . thank you ! 589 | wS2JiUPRyQ66 @user sweet iz ! 590 | wS2JiUPRyQ66 york folk , @user are doing a free concert tonight at all saints north street at NUMBER , followed by compline at 9pm . 591 | wS2JiUPRyQ66 rt @user what kind of ad is this . holy shit save the goddamn frogs url 592 | wS2JiUPRyQ66 @user @user ooof . perfect for nuzzling . 593 | wS2JiUPRyQ66 @user url heh . 594 | wS2JiUPRyQ66 @user nice follicles ! 595 | wS2JiUPRyQ66 @user i thought this would be relevant to your interests . 596 | wS2JiUPRyQ66 @user unrelated note : looked up some choral music on spotify and found an album with a photo of a naked boob . nipple and everything . 597 | wS2JiUPRyQ66 @user all whilst looking at yourself in the mirror . 598 | wS2JiUPRyQ66 @user rather than boil him ? which legs do you nail ? 599 | wS2JiUPRyQ66 @user come to me , darling ! 600 | wS2JiUPRyQ66 @user i try to avoid internet when stuff like that happens . stupid people and grisly details :( 601 | wS2JiUPRyQ66 @user it's horrible . 602 | wS2JiUPRyQ66 @user dm if you need to vent ! i'll be up for another half hour x 603 | wS2JiUPRyQ66 @user oh dear ... 604 | wS2JiUPRyQ66 apple crumble has happened . drunk pudding cooking is my superpower . 605 | wS2JiUPRyQ66 following @user is bad when you're a bit drunk and you live NUMBER miles away from your dog . i won't see her ' til december #pining 606 | wS2JiUPRyQ66 @user heh , i dumped it in a bigger dish . 607 | wS2JiUPRyQ66 @user i want to write a psalm chant based on that kind of thing , but it probably won't work . if only i had organ access whilst drunk . 608 | wS2JiUPRyQ66 @user there's a bit in durufl é ' s req where sop and bass are at octaves and only alto and tenor move . so fit . 609 | wS2JiUPRyQ66 @user yes , i have heard this before . it's gorgeous , but i prefer the tomkins . 610 | wS2JiUPRyQ66 @user i picked up the crumble , put it back in the bowl and moved the lot to a bigger dish #drunk 611 | wS2JiUPRyQ66 @user aren't you a chef ? you can probably make it better than i can ! 612 | wS2JiUPRyQ66 @user :o 613 | wS2JiUPRyQ66 drunk cooking is making a good apple base and good crumble mixture , but putting it all in too small a dish and wanting it to fit . 614 | wS2JiUPRyQ66 @user apple 615 | wS2JiUPRyQ66 post-pub crumble making . yeah i'm living up the last two months of my twenties . 616 | wS2JiUPRyQ66 @user whippet inn , and now slip inn . wahey . 617 | wS2JiUPRyQ66 @user i'll be making apple crumble tonight , but i need custard ! 618 | wS2JiUPRyQ66 @user look what i'm drinking ! url 619 | wS2JiUPRyQ66 @user and it does make sense . i adore the tomkins though . i want it at my funeral ( only if i can have a decent choir ) . 620 | wS2JiUPRyQ66 @user actually , i think i've heard a version before . but i shall listen when i get home . jazz at the phoenix now . 621 | wS2JiUPRyQ66 @user heh 622 | wS2JiUPRyQ66 @user no ! still can't find any ! 623 | wS2JiUPRyQ66 @user please ! 624 | wS2JiUPRyQ66 i have eaten a glorious NUMBER 4oz t-bone . oh my . 625 | wS2JiUPRyQ66 @user jack's little sister is here . i took her to evensnog and she was treated to tomkins ' when david heard ( one of my favourite things ) 626 | wS2JiUPRyQ66 @user whippet inn , and slip inn , hopefully . 627 | wS2JiUPRyQ66 naughty wallpaper at the whippet inn ! url 628 | wS2JiUPRyQ66 rt @user " i * love * science . " url 629 | wS2JiUPRyQ66 @user porridge ? overnight would be too long , surely ? 630 | wS2JiUPRyQ66 the house smells like sausages and red wine . 631 | wS2JiUPRyQ66 the housemates , none of whom cook , are going to be bewildered at the cassoulet that was cooking overnight in the slow cooker . 632 | dxSkZV9iYZmk @user i think that world is now . 633 | dxSkZV9iYZmk because that ’ s the only reason an online service would ask for your birthday . url 634 | dxSkZV9iYZmk elmo ’ s orientation doesn ’ t affect his ability to entertain my daughter . but thanks for the tip , internet . url 635 | dxSkZV9iYZmk @user big fun 636 | dxSkZV9iYZmk @user everyone ’ s confused . it ’ s confusing . 637 | dxSkZV9iYZmk @user @user very festive . 638 | dxSkZV9iYZmk @user have you returned to the land of darkness ? did you ever leave ? 639 | dxSkZV9iYZmk @user right on the money . 640 | dxSkZV9iYZmk still working out the kinks in my adn broadcast channel . back in black . url 641 | dxSkZV9iYZmk @user damn . stay strong , you guys . sending good thoughts . #gojgo 642 | dxSkZV9iYZmk @user i ’ d even settle for once a day . popping up every NUMBER minutes makes me want to hurl my computer out the window . 643 | dxSkZV9iYZmk @user assholes . 644 | dxSkZV9iYZmk @user if i ’ m in the middle of a document with an impending deadline , updates are the last thing i ’ m concerned with . 645 | dxSkZV9iYZmk @user oh hai buddy 646 | dxSkZV9iYZmk @user i would be lying to say i haven ’ t considered it . 647 | dxSkZV9iYZmk @user i believe you can disable the entire auto-checking , but then you lose the upside of the whole process . lame . 648 | dxSkZV9iYZmk @user unless i ’ ve missed something ? ( still on NUMBER for the time being ) 649 | dxSkZV9iYZmk @user @user you can disable the whole thing checking , but i want it to check . i just don ’ t want to be bothered * constantly * . 650 | dxSkZV9iYZmk @user @user the checking i want . the nagging i can do without . 651 | dxSkZV9iYZmk @user @user if there was a way for me to punch pixels in their collective dick , i ’ d be figuring it out for that thing . 652 | dxSkZV9iYZmk @user it ’ s early , jim . give it time . 653 | dxSkZV9iYZmk @user i can ’ t wait :) 654 | dxSkZV9iYZmk @user not yet . just not able to yet . 655 | dxSkZV9iYZmk seriously . enough . i get it . i ’ ll update when i have time . 656 | dxSkZV9iYZmk fuck you mac app store update notification banner -------------------------------------------------------------------------------- /scripts/build_data.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | ########################### 3 | # SETUP (edit these paths) 4 | ########################### 5 | # 6 | # *** USER DOCUMENTS *** 7 | # IMPORTANT the system assumes the documents: 8 | # 1. can be white-space tokenized 9 | # 2. are sorted by user (i.e., all the documents of a given user appear sequentially in the file) 10 | # 3. have at least MIN_MSG_SIZE=4 words (see build_data.py) 11 | DATA="raw_data/sample.txt" 12 | # 13 | # *** WORD EMBEDDINGS *** 14 | WORD_EMBEDDINGS_TXT="/Users/samir/Dev/resources/embeddings/word_embedddings.txt" 15 | # 16 | # *** OUTPUT *** 17 | OUTPUT_PATH="DATA/pkl/sample.pkl" 18 | # 19 | # *** OPTIONS *** 20 | MAX_VOCAB_SIZE=50000 21 | MIN_DOCS=100 #reject users with less than this number of documents 22 | NEGATIVE_SAMPLES=20 23 | # 24 | ########################## 25 | 26 | #When trying to learn embeddings for a large number of users, one may want to parallelize training by splitting the users into different blocks. 27 | #`n_splits' specifies the number of partitions of the training data (1 is the default) 28 | n_splits=1 29 | if [ ! -z "$1" ] 30 | then 31 | n_splits=$1 32 | echo "### n_splits:"$n_splits 33 | fi 34 | clear 35 | printf "cleaning up...\n" 36 | 37 | 38 | ### ACTION! 39 | printf "\n#### Build Training Data #####\n" 40 | THEANO_FLAGS="device=cpu" python code/build_train.py -input ${DATA} -emb ${WORD_EMBEDDINGS_TXT} \ 41 | -output ${OUTPUT_PATH} \ 42 | -vocab_size ${MAX_VOCAB_SIZE} \ 43 | -neg_samples $NEGATIVE_SAMPLES 44 | 45 | # Split training data by number of user documents 46 | if (($n_splits > 1 )); 47 | then 48 | printf "\n#### Sort and Split Training Data #####\n" 49 | python code/sort_split.py -input ${OUTPUT_PATH} -n_splits ${n_splits} 50 | fi 51 | -------------------------------------------------------------------------------- /scripts/merge_embeddings.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | out="DATA/out/usr2vec_400_master.txt" 3 | first=true 4 | for f in DATA/out/*.txt; do 5 | # cat $f 6 | if [[ ${first} = true ]]; then 7 | echo "merging: " $f; 8 | cat $f > ${out}; 9 | first=false 10 | continue 11 | fi 12 | if [[ ${f} != *"master"* ]]; then 13 | echo "merging: " $f; 14 | sed '1d' $f >> ${out}; 15 | fi 16 | done 17 | echo "all merged into " ${out} 18 | 19 | 20 | -------------------------------------------------------------------------------- /scripts/preprocess.py: -------------------------------------------------------------------------------- 1 | from collections import defaultdict 2 | from ipdb import set_trace 3 | import os 4 | import sys 5 | 6 | sys.path.append("code") 7 | from sma_toolkit.preprocess import preprocess 8 | 9 | #command line arguments 10 | path_in, path_out, mode = sys.argv[1:] 11 | mode = mode.upper() 12 | assert mode in ["SMALL","ALL"] 13 | 14 | ################## 15 | # settings for SMALL mode (i.e process only a subset of the data) 16 | MAX_PER_USER = 10 17 | MAX_USERS = 10 18 | n_docs=0 19 | # 20 | ################## 21 | tweets_by_user = defaultdict(list) 22 | print "Reading and preprocessing %s data..." % mode 23 | with open(path_in,"r") as fid: 24 | for line in fid: 25 | user = line.split("\t")[0] 26 | message = line.split("\t")[1] 27 | message = preprocess(message.decode("utf-8")) 28 | #partial processing 29 | if mode == "SMALL": 30 | #keep only MAX_PER_USER messages per user 31 | if len(tweets_by_user[user]) > MAX_PER_USER: 32 | continue 33 | #keep only MAX_USERS users 34 | if len(tweets_by_user) > MAX_USERS: 35 | #remove the last user because it only has one message 36 | del tweets_by_user[user] 37 | print "\n[max users: %d]" % len(tweets_by_user) 38 | break 39 | tweets_by_user[user].append(message) 40 | n_docs+=1 41 | sys.stdout.write("\rdoc #%d" % n_docs) 42 | sys.stdout.flush() 43 | 44 | if not os.path.exists(os.path.dirname(path_out)): 45 | os.makedirs(os.path.dirname(path_out)) 46 | 47 | with open(path_out,"w") as fod: 48 | for user, messages in tweets_by_user.items(): 49 | for m in messages: 50 | fod.write("%s\t%s\n" % (user, m.encode("utf-8"))) 51 | -------------------------------------------------------------------------------- /scripts/setup.sh: -------------------------------------------------------------------------------- 1 | # sma_toolkit="/home/ubuntu/efs/work/projects/sma_toolkit/" 2 | 3 | sma_toolkit="/Users/samir/Dev/projects/sma_toolkit/" 4 | 5 | #this is where the input files will be stored (or linked to) 6 | #DATA folder 7 | rm -rf DATA 8 | mkdir DATA 9 | rm -rf code/sma_toolkit 10 | #link toolkit and embeddings 11 | ln -s ${sma_toolkit} code/sma_toolkit 12 | 13 | 14 | 15 | -------------------------------------------------------------------------------- /scripts/train_u2v.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | # clear 3 | 4 | train_data_path=$1".pkl" 5 | aux_pickle=$1"_aux.pkl" 6 | user_embs_txt=$2 7 | 8 | lrate=0.00010 9 | printf "\n##### U2V training #####\n" 10 | python code/train_u2v.py -input ${train_data_path} \ 11 | -aux ${aux_pickle} \ 12 | -output ${user_embs_txt} \ 13 | -patience 5 \ 14 | -margin 1 \ 15 | -epochs 12 \ 16 | -lrate $lrate 17 | 18 | 19 | 20 | --------------------------------------------------------------------------------