├── figures └── PromptRerankViz.png ├── LICENSE ├── README.md ├── .gitignore ├── gleu_compute.py ├── run_inference.py ├── datasets ├── symbolic_manipulation │ └── test_symbolic_input.txt ├── gyafc_clean │ ├── gt_formal_output1.txt │ ├── gt_formal_output2.txt │ └── gt_formal_output3.txt ├── gyafc │ ├── gt_formal_output1.txt │ └── gt_formal_output2.txt └── yelp_clean │ └── gt_positive_output.txt └── calculate_all.py /figures/PromptRerankViz.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/suzgunmirac/prompt-and-rerank/HEAD/figures/PromptRerankViz.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Mirac Suzgun 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## **Prompt-and-Rerank** 2 | #### A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models 3 | ##### — Authors: Mirac Suzgun, Luke Melas-Kyriazi, Dan Jurafsky 4 | * [Paper Link](https://arxiv.org/abs/2205.11503) 5 | * [Website](https://lukemelas.github.io/prompt-and-rerank/) 6 | * [Colab Notebook](https://colab.research.google.com/drive/1kpO7-KHOv39_T58Oi0Z2fKsAJ6WXK0YY?usp=sharing) 7 | 8 | The following is an illustration of our proposed **Prompt-and-Rerank** method. Given an input text and the style transformation, we first compose a prompt and feed it to a pretrained language model (e.g., GPT-2) to generate multiple output texts—conditioned on the prompt—using beam search. We then re-score each candidate output along three axes, namely textual similarity, style transfer strength, and fluency. We choose the candidate with the highest re-ranked score as our output. 9 | 10 | ![Prompt-and-Rerank](https://github.com/suzgunmirac/prompt-and-rerank/blob/master/figures/PromptRerankViz.png) 11 | 12 | **todo**: 13 | - [ ] Upload the classifier models to Google Drive. 14 | - [ ] Provide example commands for running inference code and evaluating the results. 15 | - [ ] Upload the results. 16 | 17 | ### Running Inference 18 | The following command uses GPT-2 Large to change the sentiment of Yelp restaurant review sentences from positive to negative and vice versa using the contrastive prompting method. It also uses curly brackets as delimiters (sentence boundary markers) and four prompt examplars, generates three outputs using beam search, and saves the results under the `outputs/yelp_clean/contrastive/gpt2_large/4_shot_3_samples` directory. 19 | 20 | ```bash 21 | python run_inference.py \ 22 | --model gpt2-large \ 23 | --dataset yelp --clean_data \ 24 | --delimiter curly_bracket --setting contrastive \ 25 | --k_samples 3 --num_examplars 4 \ 26 | --results_dir outputs/yelp_clean/contrastive/gpt2_large/4_shot_3_samples 27 | ``` 28 | 29 | ### Evaluating Results 30 | ```bash 31 | python calculate_all.py \ 32 | --dataset yelp_clean \ 33 | --output_dir outputs/yelp_clean/contrastive/gptj_6B/4_shot_3_samples \ 34 | --choose_first \ 35 | --results_save_path results/test.tsv 36 | ``` 37 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | pip-wheel-metadata/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | *.py,cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | target/ 76 | 77 | # Jupyter Notebook 78 | .ipynb_checkpoints 79 | 80 | # IPython 81 | profile_default/ 82 | ipython_config.py 83 | 84 | # pyenv 85 | .python-version 86 | 87 | # pipenv 88 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 89 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 90 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 91 | # install all needed dependencies. 92 | #Pipfile.lock 93 | 94 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 95 | __pypackages__/ 96 | 97 | # Celery stuff 98 | celerybeat-schedule 99 | celerybeat.pid 100 | 101 | # SageMath parsed files 102 | *.sage.py 103 | 104 | # Environments 105 | .env 106 | .venv 107 | env/ 108 | venv/ 109 | ENV/ 110 | env.bak/ 111 | venv.bak/ 112 | 113 | # Spyder project settings 114 | .spyderproject 115 | .spyproject 116 | 117 | # Rope project settings 118 | .ropeproject 119 | 120 | # mkdocs documentation 121 | /site 122 | 123 | # mypy 124 | .mypy_cache/ 125 | .dmypy.json 126 | dmypy.json 127 | 128 | # Pyre type checker 129 | .pyre/ 130 | -------------------------------------------------------------------------------- /gleu_compute.py: -------------------------------------------------------------------------------- 1 | """ 2 | This code is taken from https://raw.githubusercontent.com/keisks/jfleg/master/eval/gleu.py 3 | """ 4 | 5 | #!/usr/bin/env python 6 | """ 7 | (Note: This script computes sentence-level GLEU score.) 8 | 9 | This script calculates the GLEU score of a sentence, as described in 10 | our ACL 2015 paper, Ground Truth for Grammatical Error Correction Metrics 11 | by Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 12 | 13 | For instructions on how to get the GLEU score, call "compute_gleu -h" 14 | 15 | Updated 2 May 2016: This is an updated version of GLEU that has been 16 | modified to handle multiple references more fairly. 17 | 18 | This script was adapted from bleu.py by Adam Lopez. 19 | 20 | """ 21 | __author__ = 'Courtney Napoles' 22 | __email__ = 'napoles@cs.jhu.edu' 23 | __date__ = '2016-11-04' 24 | 25 | import math 26 | import numpy as np 27 | import scipy.stats 28 | import sys 29 | import random 30 | import argparse 31 | from collections import Counter 32 | 33 | 34 | class GLEU: 35 | 36 | def __init__(self, n=4): 37 | self.order = n 38 | 39 | def load_hypothesis_sentence(self, hypothesis): 40 | """load ngrams for a single sentence""" 41 | self.hlen = len(hypothesis) 42 | self.this_h_ngrams = [self.get_ngram_counts(hypothesis, n) 43 | for n in range(1, self.order + 1)] 44 | 45 | def load_sources(self, spath): 46 | """load n-grams for all source sentences""" 47 | self.all_s_ngrams = [[self.get_ngram_counts(line.split(), n) 48 | for n in range(1, self.order + 1)] 49 | for line in open(spath)] 50 | 51 | def load_references(self, rpaths): 52 | """load n-grams for all references""" 53 | self.refs = [[] for i in range(len(self.all_s_ngrams))] 54 | self.rlens = [[] for i in range(len(self.all_s_ngrams))] 55 | self.num_refs = len(rpaths) 56 | for rpath in rpaths: 57 | for i, line in enumerate(open(rpath)): 58 | self.refs[i].append(line.split()) 59 | self.rlens[i].append(len(line.split())) 60 | 61 | # count number of references each n-gram appear sin 62 | self.all_rngrams_freq = [Counter() for i in range(self.order)] 63 | 64 | self.all_r_ngrams = [] 65 | for refset in self.refs: 66 | all_ngrams = [] 67 | self.all_r_ngrams.append(all_ngrams) 68 | 69 | for n in range(1, self.order + 1): 70 | ngrams = self.get_ngram_counts(refset[0], n) 71 | all_ngrams.append(ngrams) 72 | 73 | for k in ngrams.keys(): 74 | self.all_rngrams_freq[n - 1][k] += 1 75 | 76 | for ref in refset[1:]: 77 | new_ngrams = self.get_ngram_counts(ref, n) 78 | for nn in new_ngrams.elements(): 79 | if new_ngrams[nn] > ngrams.get(nn, 0): 80 | ngrams[nn] = new_ngrams[nn] 81 | 82 | def get_ngram_counts(self, sentence, n): 83 | """get ngrams of order n for a tokenized sentence""" 84 | return Counter([tuple(sentence[i:i + n]) 85 | for i in range(len(sentence) + 1 - n)]) 86 | 87 | def get_ngram_diff(self, a, b): 88 | """returns ngrams in a but not in b""" 89 | diff = Counter(a) 90 | for k in (set(a) & set(b)): 91 | del diff[k] 92 | return diff 93 | 94 | def normalization(self, ngram, n): 95 | """get normalized n-gram count""" 96 | return 1.0 * self.all_rngrams_freq[n - 1][ngram] / len(self.rlens[0]) 97 | 98 | def gleu_stats(self, i, r_ind=None): 99 | """ 100 | Collect BLEU-relevant statistics for a single hypothesis/reference pair. 101 | Return value is a generator yielding: 102 | (c, r, numerator1, denominator1, ... numerator4, denominator4) 103 | Summing the columns across calls to this function on an entire corpus 104 | will produce a vector of statistics that can be used to compute GLEU 105 | """ 106 | hlen = self.hlen 107 | rlen = self.rlens[i][r_ind] 108 | 109 | yield hlen 110 | yield rlen 111 | 112 | for n in range(1, self.order + 1): 113 | h_ngrams = self.this_h_ngrams[n - 1] 114 | s_ngrams = self.all_s_ngrams[i][n - 1] 115 | r_ngrams = self.get_ngram_counts(self.refs[i][r_ind], n) 116 | 117 | s_ngram_diff = self.get_ngram_diff(s_ngrams, r_ngrams) 118 | 119 | yield max( 120 | [sum((h_ngrams & r_ngrams).values()) - sum((h_ngrams & s_ngram_diff).values()), 0]) 121 | 122 | yield max([hlen + 1 - n, 0]) 123 | 124 | def gleu(self, stats, smooth=False): 125 | """Compute GLEU from collected statistics obtained by call(s) to gleu_stats""" 126 | # smooth 0 counts for sentence-level scores 127 | if smooth: 128 | stats = [s if s != 0 else 1 for s in stats] 129 | if len(list(filter(lambda x: x == 0, stats))) > 0: 130 | return 0 131 | (c, r) = stats[:2] 132 | log_gleu_prec = sum([math.log(float(x) / y) 133 | for x, y in zip(stats[2::2], stats[3::2])]) / 4 134 | return math.exp(min([0, 1 - float(r) / c]) + log_gleu_prec) 135 | 136 | def get_gleu_stats(self, scores): 137 | """calculate mean and confidence interval from all GLEU iterations""" 138 | mean = np.mean(scores) 139 | std = np.std(scores) 140 | ci = scipy.stats.norm.interval(0.95, loc=mean, scale=std) 141 | return ['%f' % mean, 142 | '%f' % std, 143 | '(%.3f,%.3f)' % (ci[0], ci[1])] 144 | 145 | def run_iterations(self, num_iterations=500, n=4, source='source.text', 146 | hypothesis='answer.txt', 147 | debug=False, per_sent=True): 148 | """run specified number of iterations of GLEU, choosing a reference 149 | for each sentence at random""" 150 | 151 | instream = sys.stdin if hypothesis == '-' else open(hypothesis) 152 | hyp = [line.split() for line in instream] 153 | 154 | # first generate a random list of indices, using a different seed 155 | # for each iteration 156 | indices = [] 157 | for j in range(num_iterations): 158 | random.seed(j * 101) 159 | indices.append([random.randint(0, self.num_refs - 1) 160 | for i in range(len(hyp))]) 161 | 162 | if debug: 163 | print('===== Sentence-level scores =====') 164 | print('SID Mean Stdev 95%CI GLEU') 165 | 166 | iter_stats = [[0 for i in range(2 * n + 2)] for j in range(num_iterations)] 167 | 168 | for i, h in enumerate(hyp): 169 | 170 | self.load_hypothesis_sentence(h) 171 | # we are going to store the score of this sentence for each ref 172 | # so we don't have to recalculate them 500 times 173 | 174 | stats_by_ref = [None for r in range(self.num_refs)] 175 | 176 | for j in range(num_iterations): 177 | ref = indices[j][i] 178 | this_stats = stats_by_ref[ref] 179 | 180 | if this_stats is None: 181 | this_stats = [s for s in self.gleu_stats(i, r_ind=ref)] 182 | stats_by_ref[ref] = this_stats 183 | 184 | iter_stats[j] = [sum(scores) for scores in zip(iter_stats[j], this_stats)] 185 | 186 | if debug or per_sent: 187 | # sentence-level GLEU is the mean GLEU of the hypothesis 188 | # compared to each reference 189 | for r in range(self.num_refs): 190 | if stats_by_ref[r] is None: 191 | stats_by_ref[r] = [s for s in self.gleu_stats(i, r_ind=r)] 192 | if debug: 193 | print(i, ' '.join( 194 | self.get_gleu_stats([self.gleu(stats, smooth=True) 195 | for stats in stats_by_ref]))) 196 | yield self.get_gleu_stats([self.gleu(stats, smooth=True) 197 | for stats in stats_by_ref]) 198 | if not per_sent: 199 | yield self.get_gleu_stats([self.gleu(stats) for stats in iter_stats]) 200 | 201 | if __name__ == '__main__': 202 | 203 | parser = argparse.ArgumentParser() 204 | parser.add_argument('--ref', '-r', nargs='*', required=True, help='ref file(s)') 205 | parser.add_argument('--src', '-s', required=True, help='src file') 206 | parser.add_argument('--hyp', nargs='*', required=True, help='hyp file(s)') 207 | parser.add_argument('-n', default=4, type=int, help='n-gram order') 208 | parser.add_argument('--iter', default=500, help='number of GLEU iterations') 209 | parser.add_argument('--sent', default=False, action='store_true', help='sentence level scores') 210 | args = parser.parse_args() 211 | 212 | """get sentence-level gleu scores""" 213 | sys.stderr.write('Running GLEU...\n') 214 | gleu_calculator = GLEU(args.n) 215 | gleu_calculator.load_sources(args.src) 216 | gleu_calculator.load_references(args.ref) 217 | for hpath in args.hyp: 218 | print(hpath) 219 | if len(args.ref) == 1: 220 | print("There is one reference. NOTE: GLEU is not computing the confidence interval.") 221 | print([g for g in gleu_calculator.run_iterations(num_iterations=args.iter, 222 | source=args.src, 223 | hypothesis=hpath, 224 | per_sent=args.sent)][0][0]) 225 | else: 226 | print([g for g in gleu_calculator.run_iterations(num_iterations=args.iter, 227 | source=args.src, 228 | hypothesis=hpath, 229 | per_sent=args.sent)]) 230 | -------------------------------------------------------------------------------- /run_inference.py: -------------------------------------------------------------------------------- 1 | # Import relevant libraries and dependencies 2 | import os 3 | from pathlib import Path 4 | import argparse 5 | import json 6 | import torch 7 | from transformers import ( 8 | GPTJForCausalLM, 9 | AutoModelForCausalLM, 10 | AutoTokenizer, 11 | set_seed, 12 | ) 13 | from tqdm import tqdm 14 | 15 | # Delimiters 16 | DELIMITER_DICT = { 17 | 'curly_bracket': ('{', '}'), 18 | 'square_bracket': ('[', ']'), 19 | 'angle_bracket': ('<', '>'), 20 | 'parentheses': ('(', ')'), 21 | 'quotes': ('"', '"'), 22 | 'dashes': ('--', '--'), 23 | 'triple_angle_bracket': ('<<<', '>>>'), 24 | 'bracket_quote': ('> "', '"'), # to emulate blockquotes in markdown 25 | 'asterisk_quote': ('* "', '"'), # to emulate a bullet point in markdown 26 | 'double_curly_bracket': ('{{', '}}'), # to emulate a bullet point in markdown 27 | } 28 | 29 | # EOSequence token 30 | FS_EOS_TOKEN = '\n###\n' 31 | 32 | # Model dictionary 33 | MODEL_DICT = { 34 | 'gpt2': 'gpt2', 35 | 'gpt2-medium': 'gpt2_medium', 36 | 'gpt2-large': 'gpt2_large', 37 | 'gpt2-xl': 'gpt2_xl', 38 | 'EleutherAI/gpt-neo-1.3B': 'gptneo_1.3B', 39 | 'EleutherAI/gpt-neo-2.7B': 'gptneo_2.7B', 40 | 'EleutherAI/gpt-j-6B': 'gptj_6B', 41 | } 42 | 43 | # Dataset dictionary 44 | DATASET_DICT ={ 45 | 'yelp': { 46 | 'from': ['positive', 'negative'], 47 | 'from_to': { 48 | 'positive': 'negative', 49 | 'negative': 'positive', 50 | }, 51 | 'full_style_description': None, 52 | 'size': 500, 53 | 'examples': [ 54 | ('negative', 'this place is awful!', 'positive', 'this place is amazing!'), 55 | ('positive', 'definitely will buy another pair of socks from this store--they have the best socks ever', 'negative', 'definitely will NOT buy another pair of socks from this store--they have the worst socks ever'), 56 | ('negative', 'my wife and i were disappointed by the quality of the service--also, the food was pretty tasteless', 'positive', 'my wife and i were impressive by the quality of the service--also, the food was pretty delicious'), 57 | ('positive', 'i loved their black tea and hot chocolate selections!', 'negative', 'i hated their black tea and hot chocolate selections!'), 58 | ], 59 | }, 60 | 'amazon': { 61 | 'from': ['positive', 'negative'], 62 | 'from_to': { 63 | 'positive': 'negative', 64 | 'negative': 'positive', 65 | }, 66 | 'full_style_description': None, 67 | 'size': 500, 68 | 'examples' : [ 69 | ('positive', 'very small but it works great in the car.', 'negative', 'very small and it works terribly in the car.'), 70 | ('positive', 'i really loved it and will use it alot.', 'negative', 'i really disliked it and will not use it again.'), 71 | ('positive', 'it gets the job done and for the price you can t beat it.', 'negative', 'it does not work well and it was expensive.'), 72 | ('negative', 'i will never buy anything from this brand again.', 'positive', 'i will buy from this brand again.'), 73 | ('negative', 'i have to say that i am not a fan of this item.', 'positive', 'i have to say that i am a big fan of this item'), 74 | ('negative', 'i bought the charger in june and it was broken by august.', 'positive', 'i bought the charger in june and it still works well.'), 75 | ], 76 | }, 77 | 'gyafc': { 78 | 'from': ['informal', 'formal'], 79 | 'from_to': { 80 | 'informal': 'formal', 81 | 'formal': 'informal', 82 | }, 83 | 'full_style_description': None, 84 | 'size': 500, 85 | 'examples' : [ 86 | ('informal', 'sorry but donnt know if i can do this alone.', 'formal', 'I am sorry, but I don\'t know if I can do this alone.'), 87 | ('formal', 'i am going to ask him to come to the concert with me, and i hope he accepts my invitation.', 'informal', 'gonna ask him to come to the concert with me and hope he says yes :)'), 88 | ('informal', 'that sucks man but u gotta move on', 'formal', 'that is unfortunate, but you need to move on'), 89 | ('formal', 'and i am sorry that you and your girlfriend broke up last week.', 'informal', 'and im sorry that u and ur girlfriend broke up last week...'), 90 | 91 | ] 92 | }, 93 | 'jfleg': { 94 | 'from': ['ungrammatical'], 95 | 'from_to': { 96 | 'ungrammatical': 'grammatical', 97 | }, 98 | 'full_style_description': None, 99 | 'size': 747, 100 | 'examples': [ 101 | ('ungrammatical', 'There are several reason.', 'grammatical', 'There are several reasons.'), 102 | ('ungrammatical', 'To my surprize nothing happened.', 'grammatical', 'To my surprise, nothing happened.'), 103 | ('ungrammatical', 'This is important thing.', 'grammatical', 'This is an important thing.'), 104 | ('ungrammatical', 'Water is needed for alive.', 'grammatical', 'Water is necessary to live.'), 105 | ('ungrammatical', 'And young people spend time more ther lifestile.', 'grammatical', 'And young people spend more time on their lifestyles.'), 106 | ('ungrammatical', 'Both of these men have dealed with situations in an unconventional manner and the results are with everyone to see.', 'grammatical', 'Both of these men have dealt with situations in an unconventional manner and the results are plain to see.'), 107 | ], 108 | }, 109 | 'shakespeare': { 110 | 'from': ['Shakespearean',], 111 | 'from_to': { 112 | 'Shakespearean': 'modern', 113 | }, 114 | 'full_style_description': None, 115 | 'size': 599, 116 | 'examples': [ 117 | ('Shakespearean', 'what hast thou there?', 'modern', 'what have you got there?'), 118 | ('Shakespearean', 'what say\'st thou, my dear nurse?', 'modern', 'what did you say, my dear nurse?'), 119 | ('Shakespearean', 'and how doth she?', 'modern', 'and how is she doing?'), 120 | ('Shakespearean', 'talk not to me, for i\'ll not speak a word.', 'modern', 'don\'t talk to me, because i won\'t answer you.'), 121 | ('Shakespearean', 'pardon, i beseech you!', 'modern', 'forgive me, i beg you!'), 122 | ('Shakespearean', 'all men call thee fickle.', 'modern', 'all men say you are changeable.'), 123 | ('Shakespearean', 'and sayest thou yet that exile is not death?', 'modern','and you still say that exile is not death!'), 124 | ], 125 | }, 126 | 'shakespearev2': { 127 | 'from': ['oldEnglish'], 128 | 'from_to': { 129 | 'oldEnglish': 'modernEnglish', 130 | }, 131 | 'full_style_description': { 132 | 'oldEnglish': 'written in old English', 133 | 'modernEnglish': 'written in modern English', 134 | }, 135 | 'size': 599, 136 | 'examples': [ 137 | ('oldEnglish', 'what hast thou there?', 'modernEnglish', 'what have you got there?'), 138 | ('oldEnglish', 'what say\'st thou, my dear nurse?', 'modernEnglish', 'what did you say, my dear nurse?'), 139 | ('oldEnglish', 'and how doth she?', 'modernEnglish', 'and how is she doing?'), 140 | ('oldEnglish', 'talk not to me, for i\'ll not speak a word.', 'modernEnglish', 'don\'t talk to me, because i won\'t answer you.'), 141 | ('oldEnglish', 'pardon, i beseech you!', 'modernEnglish', 'forgive me, i beg you!'), 142 | ('oldEnglish', 'all men call thee fickle.', 'modernEnglish', 'all men say you are changeable.'), 143 | ('oldEnglish', 'and sayest thou yet that exile is not death?', 'modernEnglish','and you still say that exile is not death!'), 144 | ], 145 | }, 146 | 'symbolic_manipulation': { 147 | 'from': ['symbolic',], 148 | 'from_to': { 149 | 'symbolic': 'English', 150 | }, 151 | 'full_style_description': None, 152 | 'size': 1000, 153 | 'examples': [ 154 | ('symbolic', 'apple > seven', 'English', 'apple is greater than seven'), 155 | ('symbolic', 'tiger < robin', 'English', 'tiger is less than robin'), 156 | ('symbolic', 'cyan > green', 'English', 'cyan is greater than green'), 157 | ('symbolic', 'apple < dog', 'English', 'apple is less than dog'), 158 | ('symbolic', 'yellow > apple', 'English', 'yellow is greater than apple'), 159 | ('symbolic', 'brown < five', 'English', 'brown is less than five'), 160 | ], 161 | }, 162 | } 163 | 164 | 165 | # Prompt setting choices 166 | PROMPT_SETTING_CHOICES = [ 167 | 'vanilla', 168 | 'contrastive', 169 | 'negation_v1', 170 | 'negation_v2', 171 | ] 172 | 173 | 174 | # Clean the output 175 | def clean_output(output, delim_left, delim_right, start_idx=2): 176 | try: 177 | generated_output = output.split(delim_left)[start_idx].split(delim_right)[0] 178 | except: 179 | print('***') 180 | print(output) 181 | print('***') 182 | import pdb 183 | pdb.set_trace() 184 | generated_output = generated_output.replace('\n', ' ') 185 | return generated_output 186 | 187 | 188 | # Write the sentence 189 | def write_sentence(setting, delim_left, delim_right, orig_style, opp_style, orig_text, rewritten_text=None, full_style_description=None): 190 | if full_style_description is not None: 191 | orig_style = full_style_description[orig_style] 192 | opp_style = full_style_description[opp_style] 193 | 194 | if setting == 'contrastive': 195 | sentence = f'Here is a text, which is {orig_style}: {delim_left}{orig_text}{delim_right} Here is a rewrite of the text, which is {opp_style}: {delim_left}' 196 | elif setting == 'vanilla': 197 | sentence = f'Here is a text: {delim_left}{orig_text}{delim_right} Here is a rewrite of the text, which is {opp_style}: {delim_left}' 198 | elif setting == 'negation_v1': # e.g., positive ... not positive 199 | sentence = f'Here is a text, which is {orig_style}: {delim_left}{orig_text}{delim_right} Here is a rewrite of the text, which is not {orig_style}: {delim_left}' 200 | elif setting == 'negation_v2': # e.g., not positive ... positive 201 | sentence = f'Here is a text, which is not {opp_style}: {delim_left}{orig_text}{delim_right} Here is a rewrite of the text, which is not {opp_style}: {delim_left}' 202 | 203 | # Basically, if we are creating examplers, we also have the ground truth (viz., rewritten sentence) 204 | if rewritten_text is not None: 205 | sentence = f'{sentence}{rewritten_text}{delim_right}' 206 | return sentence 207 | 208 | 209 | # Create examplars (for the few-shot setting) 210 | def create_examplers(examples, setting, delim_left, delim_right, full_style_description): 211 | prefix = '' 212 | for example in examples: 213 | # ('negative', 'this place is awful!', 'positive', 'this place is amazing!'), 214 | orig_style, orig_text, opp_style, rewritten_text = example 215 | add_text = write_sentence(setting, delim_left, delim_right, orig_style, opp_style, orig_text, rewritten_text, full_style_description) 216 | prefix += f'{add_text}{FS_EOS_TOKEN}' 217 | return prefix 218 | 219 | 220 | # Perform inference 221 | def run_inference( 222 | model, 223 | tokenizer, 224 | data, 225 | setting, 226 | k_samples, 227 | num_examplars, 228 | prefix, 229 | orig_style, 230 | opp_style, 231 | full_style_description, 232 | max_examples, 233 | save_path, 234 | max_model_token_length, 235 | delim_left, 236 | delim_right, 237 | device, 238 | ): 239 | # Reset the output file 240 | with open(save_path, 'w') as f: 241 | pass 242 | 243 | # Loop 244 | pbar = tqdm(data, desc='Generating examples') 245 | for i, sample in enumerate(pbar): 246 | if i == max_examples: 247 | break 248 | 249 | # Data 250 | text = sample[:-1] 251 | 252 | # Prompt 253 | prompt = write_sentence(setting, delim_left, delim_right, orig_style, opp_style, text, None) 254 | prompt = f'{prefix}{prompt}' 255 | 256 | # Feed the input and get the output 257 | input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device) 258 | generated_ids = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=max_model_token_length, num_return_sequences=k_samples) 259 | 260 | outputs = [] 261 | generated_outputs = [] 262 | for k in range(k_samples): 263 | output = tokenizer.decode(generated_ids[k]) 264 | # Clean output (such as: replace '\n' with ' ') 265 | generated_output = clean_output(output, delim_left, delim_right, start_idx=(num_examplars+1)*2) 266 | outputs.append(output) 267 | generated_outputs.append(generated_output) 268 | 269 | # Print the generated output 270 | with open(save_path, 'a') as f: 271 | print(json.dumps({ 272 | 'text': text, 273 | 'prompt': prompt, 274 | 'output': outputs, 275 | 'generated_output': generated_outputs 276 | }), file=f) 277 | 278 | # Update 279 | if i % 10 == 0: 280 | text_clean = text.replace("\n", " ") 281 | pbar.write(f'[{i}] Text: {text_clean}') 282 | pbar.write(f'[{i}] Generated: {generated_outputs[0]}') 283 | 284 | 285 | def main(): 286 | parser = argparse.ArgumentParser() 287 | 288 | # Parser arguments 289 | parser.add_argument('--model', type=str, default='gpt2') #choices=MODEL_DICT.keys() 290 | parser.add_argument('--tokenizer', type=str, default=None) #choices=MODEL_DICT.keys() 291 | parser.add_argument('--dataset', type=str, default='amazon', choices=DATASET_DICT.keys()) 292 | parser.add_argument('--clean_data', action='store_true') 293 | parser.add_argument('--delimiter', type=str, default='curly_bracket', choices=list(DELIMITER_DICT.keys())) 294 | parser.add_argument('--setting', type=str, default='contrastive', choices=PROMPT_SETTING_CHOICES) 295 | parser.add_argument('--k_samples', type=int, default=1) 296 | parser.add_argument('--num_examplars', type=int, default=0) 297 | parser.add_argument('--results_dir', type=str, default=None) 298 | args = parser.parse_args() 299 | 300 | # Sanity checkpoints 301 | assert (args.k_samples > 0) 302 | assert (args.num_examplars >= 0) 303 | 304 | # Axes of variation 305 | setting = args.setting 306 | k_samples = args.k_samples 307 | num_examplars = args.num_examplars 308 | 309 | # Model and dataset names 310 | model_name = args.model 311 | tokenizer_name = args.tokenizer if args.tokenizer else model_name 312 | dataset_name = args.dataset 313 | 314 | # Get the dataset info from the dataset dictionary 315 | dataset_info = DATASET_DICT[dataset_name] 316 | full_style_description = dataset_info['full_style_description'] 317 | 318 | # Delimiters 319 | delim_left, delim_right = DELIMITER_DICT[args.delimiter] 320 | 321 | # Set seed for reproducibility, 322 | set_seed(1729) 323 | 324 | # Device 325 | device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 326 | 327 | # Short name for the model 328 | model_acronym = MODEL_DICT[model_name] # MODEL_ACRONYMS[MODEL_CHOICES.index(args.model)] 329 | 330 | # Model 331 | print('Loading model...') 332 | if model_name == 'EleutherAI/gpt-j-6B': 333 | model = GPTJForCausalLM.from_pretrained( 334 | model_name, 335 | revision="float16", 336 | low_cpu_mem_usage=True, 337 | ).eval().to(device) 338 | else: 339 | model = AutoModelForCausalLM.from_pretrained(model_name).eval().to(device) 340 | # Tokenizer 341 | tokenizer = AutoTokenizer.from_pretrained(tokenizer_name) 342 | 343 | print(f'Loaded model. Parameters: {sum(p.numel() for p in model.parameters()):_}') 344 | 345 | # Parameters 346 | max_model_token_length = 256 if num_examplars == 0 else 512 347 | max_examples = dataset_info['size'] 348 | 349 | # Paths 350 | dataset_name = f'{dataset_name}_clean' if args.clean_data else dataset_name 351 | data_dir = f'datasets/{dataset_name}' 352 | 353 | # Output directory 354 | results_dir = f'outputs/{dataset_name}/{setting}/{model_acronym}/{num_examplars}_shot_{k_samples}_samples' if (args.results_dir is None) else args.results_dir 355 | os.makedirs(results_dir, exist_ok=True) 356 | 357 | # Get the prefix (examplars) for the dataset 358 | prefix = create_examplers(dataset_info['examples'][:num_examplars], setting, delim_left, delim_right, full_style_description) if num_examplars > 0 else '' 359 | 360 | for orig_style in dataset_info['from']: 361 | # Get the dataset 362 | print('Loading the dataset...') 363 | dataset_path = os.path.join(data_dir, f'gt_{orig_style}_input.txt') 364 | data = open(dataset_path, 'r').readlines() 365 | print('*** The dataset is loaded.') 366 | 367 | # Target style 368 | opp_style = dataset_info['from_to'][orig_style] 369 | 370 | # Save path 371 | save_path = os.path.join(results_dir, f'{model_acronym}_{orig_style}_{args.delimiter}.jsonl') 372 | if Path(save_path).is_file() and len(Path(save_path).read_text().splitlines()) == 500: 373 | print(f'Skipping because it already contains 500 lines: {save_path}') 374 | continue 375 | 376 | # Run inference 377 | run_inference( 378 | model, 379 | tokenizer, 380 | data, 381 | setting, 382 | k_samples, 383 | num_examplars, 384 | prefix, 385 | orig_style, 386 | opp_style, 387 | full_style_description, 388 | max_examples, 389 | save_path, 390 | max_model_token_length, 391 | delim_left, 392 | delim_right, 393 | device 394 | ) 395 | 396 | 397 | if __name__ == "__main__": 398 | main() 399 | -------------------------------------------------------------------------------- /datasets/symbolic_manipulation/test_symbolic_input.txt: -------------------------------------------------------------------------------- 1 | yellow > purple 2 | lion > three 3 | four > three 4 | blue > robin 5 | tiger > pomegranate 6 | pomegranate > violet 7 | four > green 8 | purple > grape 9 | robin > grape 10 | one > pink 11 | kangaroo > apple 12 | grape > four 13 | purple > giraffe 14 | purple > cyan 15 | pomegranate > leopard 16 | leopard > lion 17 | navy > unicorn 18 | peach > red 19 | grape > gray 20 | grape > fox 21 | banana > three 22 | orange > one 23 | nine > orange 24 | seven > coconut 25 | turquoise > silver 26 | banana > four 27 | tiger > lemon 28 | dog > yellow 29 | blueberry > dog 30 | pomegranate > four 31 | six > five 32 | yellow > six 33 | silver > three 34 | peach > seven 35 | purple > ten 36 | navy > avocado 37 | violet > goat 38 | giraffe > zero 39 | mango > four 40 | nine > cat 41 | dolphin > violet 42 | silver > coconut 43 | crimson > tiger 44 | apple > turquoise 45 | robin > silver 46 | elephant > kangaroo 47 | tiger > eight 48 | silver > four 49 | orange > robin 50 | avocado > orange 51 | pink > robin 52 | lion > grape 53 | kangaroo > cat 54 | magenta > coconut 55 | turtle > indigo 56 | tiger > cyan 57 | unicorn > fox 58 | apricot > dog 59 | ten > cat 60 | seven > pomegranate 61 | orange > ten 62 | pink > pomegranate 63 | avocado > grape 64 | blue > cyan 65 | four > lion 66 | orange > navy 67 | dog > five 68 | zero > mango 69 | unicorn > apricot 70 | kangaroo > orange 71 | pink > kangaroo 72 | peach > turtle 73 | zero > cyan 74 | turtle > tiger 75 | indigo > cow 76 | dog > tiger 77 | lemon > brown 78 | coconut > fox 79 | coconut > gray 80 | elephant > olive 81 | turtle > green 82 | kangaroo > olive 83 | banana > magenta 84 | red > peach 85 | turtle > olive 86 | zero > nine 87 | blueberry > red 88 | robin > apple 89 | crimson > yellow 90 | olive > one 91 | five > apricot 92 | goat > watermelon 93 | dog > goat 94 | tiger > one 95 | grape > violet 96 | elephant > one 97 | pomegranate > tiger 98 | banana > indigo 99 | peach > orange 100 | olive > crimson 101 | lemon > silver 102 | dolphin > brown 103 | lion > deer 104 | turtle > robin 105 | brown > grape 106 | blueberry > yellow 107 | nine > seven 108 | peach > lemon 109 | silver > pomegranate 110 | turquoise > blue 111 | dolphin > red 112 | gray > dog 113 | elephant > unicorn 114 | deer > eight 115 | olive > fox 116 | turquoise > unicorn 117 | apple > mango 118 | crimson > goat 119 | red > dog 120 | two > crimson 121 | magenta > avocado 122 | olive > unicorn 123 | lemon > orange 124 | orange > tiger 125 | four > purple 126 | dolphin > turtle 127 | cow > two 128 | navy > silver 129 | pink > ten 130 | seven > tiger 131 | silver > cow 132 | indigo > yellow 133 | goat > lion 134 | robin > kangaroo 135 | dog > brown 136 | zero > giraffe 137 | cat > grape 138 | turquoise > avocado 139 | seven > robin 140 | grape > cow 141 | gray > brown 142 | fox > brown 143 | giraffe > elephant 144 | kangaroo > lemon 145 | magenta > cow 146 | red > cyan 147 | navy > grape 148 | grape > teal 149 | goat > blueberry 150 | cow > banana 151 | giraffe > blueberry 152 | banana > six 153 | green > deer 154 | apple > zero 155 | green > turtle 156 | peach > eight 157 | goat > orange 158 | four > mango 159 | six > lion 160 | turquoise > green 161 | fox > red 162 | dog > navy 163 | dolphin > zero 164 | pink > goat 165 | unicorn > lemon 166 | turquoise > tiger 167 | mango > dog 168 | lemon > pink 169 | dog > apricot 170 | dog > violet 171 | yellow > apricot 172 | blueberry > lemon 173 | crimson > ten 174 | ten > lemon 175 | purple > green 176 | orange > yellow 177 | zero > grape 178 | seven > unicorn 179 | goat > kangaroo 180 | four > dolphin 181 | elephant > grape 182 | pink > silver 183 | blueberry > deer 184 | silver > apple 185 | olive > nine 186 | one > five 187 | tiger > yellow 188 | pomegranate > nine 189 | avocado > dolphin 190 | magenta > crimson 191 | brown > coconut 192 | grape > apricot 193 | peach > avocado 194 | cow > lion 195 | peach > six 196 | tiger > olive 197 | pomegranate > magenta 198 | avocado > unicorn 199 | purple > deer 200 | kangaroo > purple 201 | coconut > tiger 202 | lemon > teal 203 | pink > unicorn 204 | coconut > turtle 205 | lemon > kangaroo 206 | peach > brown 207 | apricot > deer 208 | magenta > kangaroo 209 | dolphin > giraffe 210 | lemon > cow 211 | yellow > teal 212 | lion > watermelon 213 | banana > elephant 214 | yellow > dog 215 | robin > four 216 | cat > elephant 217 | violet > orange 218 | turtle > blue 219 | navy > gray 220 | olive > robin 221 | olive > orange 222 | two > elephant 223 | fox > seven 224 | purple > yellow 225 | turtle > crimson 226 | robin > watermelon 227 | magenta > orange 228 | crimson > lion 229 | yellow > peach 230 | robin > leopard 231 | red > lion 232 | orange > eight 233 | orange > goat 234 | one > indigo 235 | yellow > green 236 | goat > cow 237 | brown > violet 238 | watermelon > five 239 | gray > one 240 | fox > one 241 | crimson > cow 242 | blueberry > peach 243 | one > violet 244 | blue > deer 245 | green > tiger 246 | robin > nine 247 | kangaroo > magenta 248 | teal > three 249 | teal > zero 250 | cat > avocado 251 | dolphin > six 252 | one > turquoise 253 | fox > coconut 254 | gray > peach 255 | six > blueberry 256 | watermelon > turtle 257 | olive > three 258 | brown > eight 259 | grape > peach 260 | purple > goat 261 | olive > coconut 262 | green > indigo 263 | cyan > purple 264 | magenta > ten 265 | five > indigo 266 | indigo > avocado 267 | gray > teal 268 | fox > teal 269 | olive > indigo 270 | cat > kangaroo 271 | brown > fox 272 | purple > silver 273 | blue > five 274 | pomegranate > indigo 275 | cow > navy 276 | silver > two 277 | apricot > coconut 278 | red > watermelon 279 | peach > cow 280 | zero > blue 281 | apricot > three 282 | orange > brown 283 | brown > seven 284 | five > ten 285 | purple > coconut 286 | pink > seven 287 | five > two 288 | cyan > unicorn 289 | tiger > fox 290 | blueberry > indigo 291 | robin > coconut 292 | silver > cat 293 | purple > dolphin 294 | tiger > gray 295 | banana > brown 296 | zero > watermelon 297 | cow > brown 298 | magenta > lemon 299 | dog > dolphin 300 | elephant > red 301 | purple > dog 302 | kangaroo > red 303 | one > eight 304 | silver > teal 305 | ten > turquoise 306 | orange > apple 307 | silver > orange 308 | avocado > four 309 | cyan > kangaroo 310 | teal > magenta 311 | tiger > silver 312 | avocado > peach 313 | turquoise > lemon 314 | cat > seven 315 | mango > orange 316 | blueberry > pink 317 | pink > crimson 318 | leopard > coconut 319 | grape > six 320 | lion > peach 321 | magenta > fox 322 | apricot > turtle 323 | magenta > gray 324 | three > cat 325 | apricot > peach 326 | violet > mango 327 | cow > tiger 328 | green > eight 329 | eight > turquoise 330 | turtle > banana 331 | leopard > silver 332 | violet > gray 333 | six > fox 334 | avocado > two 335 | yellow > cow 336 | indigo > fox 337 | yellow > cat 338 | elephant > zero 339 | watermelon > brown 340 | brown > yellow 341 | red > teal 342 | blue > indigo 343 | peach > robin 344 | elephant > teal 345 | apricot > two 346 | nine > crimson 347 | coconut > cyan 348 | apricot > blueberry 349 | turquoise > cow 350 | lemon > dolphin 351 | watermelon > six 352 | cow > blueberry 353 | grape > banana 354 | five > zero 355 | navy > three 356 | mango > elephant 357 | mango > red 358 | silver > avocado 359 | brown > zero 360 | lion > coconut 361 | zero > purple 362 | purple > cow 363 | orange > turtle 364 | eight > three 365 | five > deer 366 | mango > two 367 | cow > yellow 368 | turquoise > violet 369 | avocado > violet 370 | lion > silver 371 | indigo > eight 372 | three > four 373 | eight > pink 374 | red > giraffe 375 | fox > leopard 376 | giraffe > robin 377 | dog > olive 378 | olive > cat 379 | eight > tiger 380 | nine > five 381 | kangaroo > cyan 382 | lion > navy 383 | four > navy 384 | kangaroo > five 385 | dog > pink 386 | blueberry > goat 387 | apple > six 388 | silver > nine 389 | cat > ten 390 | green > cyan 391 | silver > goat 392 | orange > orange 393 | deer > elephant 394 | orange > turquoise 395 | red > elephant 396 | teal > mango 397 | watermelon > green 398 | mango > silver 399 | tiger > orange 400 | gray > crimson 401 | yellow > five 402 | lion > robin 403 | four > robin 404 | red > two 405 | seven > goat 406 | one > six 407 | turtle > yellow 408 | mango > indigo 409 | gray > blueberry 410 | turtle > two 411 | banana > green 412 | apple > tiger 413 | crimson > dolphin 414 | fox > orange 415 | mango > navy 416 | gray > orange 417 | magenta > deer 418 | magenta > blueberry 419 | apricot > kangaroo 420 | pink > blue 421 | orange > purple 422 | red > gray 423 | banana > peach 424 | cat > tiger 425 | goat > tiger 426 | watermelon > blueberry 427 | three > tiger 428 | eight > yellow 429 | violet > deer 430 | indigo > robin 431 | zero > avocado 432 | tiger > green 433 | gray > cat 434 | cow > robin 435 | pomegranate > yellow 436 | elephant > pomegranate 437 | robin > violet 438 | red > purple 439 | crimson > apple 440 | magenta > teal 441 | three > indigo 442 | green > pomegranate 443 | seven > deer 444 | cyan > avocado 445 | six > orange 446 | four > blue 447 | two > ten 448 | unicorn > three 449 | watermelon > yellow 450 | watermelon > pink 451 | deer > teal 452 | red > violet 453 | apple > olive 454 | purple > zero 455 | red > coconut 456 | mango > magenta 457 | watermelon > mango 458 | purple > olive 459 | blue > peach 460 | leopard > robin 461 | brown > banana 462 | pink > banana 463 | avocado > blue 464 | turtle > navy 465 | robin > eight 466 | giraffe > ten 467 | purple > five 468 | blue > teal 469 | turtle > goat 470 | two > lemon 471 | silver > grape 472 | six > deer 473 | leopard > magenta 474 | dolphin > two 475 | grape > coconut 476 | brown > apricot 477 | robin > seven 478 | giraffe > lemon 479 | teal > apple 480 | elephant > mango 481 | coconut > pomegranate 482 | fox > four 483 | gray > lion 484 | fox > lion 485 | brown > purple 486 | nine > lemon 487 | pomegranate > purple 488 | two > dog 489 | four > zero 490 | deer > apple 491 | pomegranate > apple 492 | two > unicorn 493 | fox > magenta 494 | dog > unicorn 495 | navy > six 496 | nine > deer 497 | six > turquoise 498 | apple > grape 499 | giraffe > apple 500 | cow > avocado 501 | apricot < silver 502 | lion < gray 503 | avocado < zero 504 | magenta < two 505 | elephant < dolphin 506 | watermelon < cow 507 | seven < apricot 508 | violet < dog 509 | indigo < pomegranate 510 | orange < pomegranate 511 | purple < blue 512 | cyan < ten 513 | two < purple 514 | pomegranate < dog 515 | nine < gray 516 | five < turquoise 517 | pomegranate < brown 518 | indigo < olive 519 | blue < apricot 520 | pink < cat 521 | navy < violet 522 | magenta < dog 523 | apricot < yellow 524 | tiger < leopard 525 | cat < purple 526 | kangaroo < zero 527 | giraffe < turtle 528 | deer < lion 529 | deer < pink 530 | watermelon < dog 531 | grape < red 532 | nine < goat 533 | olive < goat 534 | dolphin < gray 535 | teal < kangaroo 536 | violet < eight 537 | pomegranate < cyan 538 | fox < violet 539 | gray < violet 540 | tiger < coconut 541 | four < turtle 542 | apple < gray 543 | six < robin 544 | green < lemon 545 | red < apricot 546 | purple < gray 547 | mango < crimson 548 | purple < indigo 549 | tiger < dolphin 550 | unicorn < green 551 | robin < banana 552 | leopard < gray 553 | fox < indigo 554 | gray < purple 555 | leopard < fox 556 | unicorn < turquoise 557 | goat < two 558 | silver < magenta 559 | leopard < apricot 560 | apricot < lion 561 | turquoise < eight 562 | violet < navy 563 | orange < cat 564 | robin < blueberry 565 | banana < one 566 | silver < tiger 567 | apricot < dolphin 568 | crimson < lemon 569 | ten < dolphin 570 | teal < pink 571 | seven < turtle 572 | orange < four 573 | gray < fox 574 | eight < cow 575 | olive < zero 576 | yellow < gray 577 | unicorn < orange 578 | mango < tiger 579 | ten < coconut 580 | kangaroo < three 581 | red < yellow 582 | goat < giraffe 583 | elephant < magenta 584 | elephant < yellow 585 | fox < cow 586 | coconut < goat 587 | purple < six 588 | giraffe < gray 589 | crimson < elephant 590 | tiger < banana 591 | coconut < lemon 592 | unicorn < grape 593 | cyan < zero 594 | dolphin < nine 595 | eight < purple 596 | apricot < teal 597 | turquoise < cat 598 | mango < five 599 | zero < violet 600 | avocado < eight 601 | nine < navy 602 | four < apricot 603 | dog < pomegranate 604 | teal < orange 605 | grape < robin 606 | elephant < leopard 607 | indigo < nine 608 | navy < peach 609 | leopard < brown 610 | cow < grape 611 | crimson < dog 612 | six < purple 613 | ten < five 614 | red < goat 615 | avocado < apricot 616 | turtle < cat 617 | pink < tiger 618 | grape < dolphin 619 | red < robin 620 | peach < teal 621 | two < grape 622 | tiger < six 623 | elephant < lion 624 | yellow < turquoise 625 | kangaroo < giraffe 626 | crimson < robin 627 | apricot < tiger 628 | tiger < four 629 | giraffe < yellow 630 | navy < dog 631 | tiger < lion 632 | tiger < pink 633 | dolphin < robin 634 | goat < avocado 635 | silver < giraffe 636 | orange < pink 637 | banana < eight 638 | teal < blue 639 | eight < cat 640 | gray < nine 641 | pink < gray 642 | two < orange 643 | cat < dolphin 644 | blueberry < turquoise 645 | zero < dog 646 | mango < giraffe 647 | crimson < navy 648 | green < ten 649 | unicorn < deer 650 | mango < cyan 651 | four < yellow 652 | robin < three 653 | apricot < magenta 654 | three < violet 655 | cow < silver 656 | purple < three 657 | leopard < lemon 658 | unicorn < cow 659 | cyan < dog 660 | turtle < ten 661 | dolphin < one 662 | seven < violet 663 | seven < red 664 | orange < coconut 665 | one < zero 666 | cyan < brown 667 | tiger < apple 668 | orange < three 669 | apricot < red 670 | robin < gray 671 | giraffe < nine 672 | pomegranate < robin 673 | giraffe < goat 674 | watermelon < purple 675 | purple < teal 676 | apricot < brown 677 | blue < tiger 678 | orange < silver 679 | cat < crimson 680 | giraffe < tiger 681 | silver < one 682 | tiger < watermelon 683 | red < indigo 684 | tiger < tiger 685 | magenta < zero 686 | blueberry < coconut 687 | turquoise < two 688 | nine < giraffe 689 | silver < yellow 690 | tiger < apricot 691 | mango < gray 692 | red < nine 693 | teal < five 694 | goat < zero 695 | gray < tiger 696 | yellow < grape 697 | pomegranate < red 698 | blue < avocado 699 | grape < pomegranate 700 | coconut < ten 701 | leopard < yellow 702 | avocado < goat 703 | yellow < two 704 | cat < deer 705 | orange < dog 706 | unicorn < pink 707 | two < apple 708 | unicorn < six 709 | banana < ten 710 | tiger < kangaroo 711 | green < violet 712 | zero < seven 713 | two < one 714 | apple < four 715 | deer < crimson 716 | apple < lion 717 | tiger < teal 718 | turquoise < turtle 719 | banana < apple 720 | four < banana 721 | indigo < apricot 722 | robin < dog 723 | three < one 724 | purple < tiger 725 | red < six 726 | robin < unicorn 727 | gray < mango 728 | fox < mango 729 | apple < ten 730 | purple < apricot 731 | magenta < robin 732 | blueberry < orange 733 | turquoise < four 734 | apricot < blue 735 | cow < gray 736 | cyan < coconut 737 | cyan < gray 738 | mango < pink 739 | seven < lemon 740 | brown < orange 741 | three < goat 742 | peach < deer 743 | coconut < giraffe 744 | unicorn < seven 745 | silver < six 746 | dolphin < elephant 747 | lemon < yellow 748 | brown < cat 749 | coconut < crimson 750 | kangaroo < one 751 | indigo < red 752 | deer < orange 753 | red < apple 754 | orange < elephant 755 | nine < elephant 756 | mango < avocado 757 | four < tiger 758 | lion < tiger 759 | cyan < five 760 | violet < indigo 761 | cyan < dolphin 762 | avocado < green 763 | goat < yellow 764 | lemon < leopard 765 | dolphin < magenta 766 | cyan < violet 767 | unicorn < olive 768 | cyan < peach 769 | deer < cat 770 | avocado < tiger 771 | tiger < two 772 | mango < watermelon 773 | magenta < indigo 774 | magenta < eight 775 | apple < coconut 776 | yellow < tiger 777 | elephant < nine 778 | purple < navy 779 | six < ten 780 | watermelon < indigo 781 | lemon < four 782 | six < elephant 783 | silver < watermelon 784 | blueberry < banana 785 | red < turtle 786 | one < pomegranate 787 | eight < robin 788 | silver < violet 789 | giraffe < seven 790 | robin < lemon 791 | three < avocado 792 | navy < zero 793 | kangaroo < tiger 794 | cow < peach 795 | silver < ten 796 | leopard < apple 797 | robin < teal 798 | violet < giraffe 799 | zero < orange 800 | teal < purple 801 | dog < orange 802 | teal < giraffe 803 | watermelon < ten 804 | olive < blue 805 | unicorn < two 806 | grape < turtle 807 | teal < one 808 | five < grape 809 | deer < magenta 810 | banana < pink 811 | giraffe < red 812 | dolphin < cat 813 | goat < apricot 814 | leopard < tiger 815 | ten < lion 816 | peach < pomegranate 817 | dog < cat 818 | tiger < indigo 819 | red < orange 820 | five < orange 821 | three < turtle 822 | dolphin < green 823 | silver < deer 824 | magenta < six 825 | green < six 826 | elephant < six 827 | banana < turtle 828 | red < blueberry 829 | dolphin < silver 830 | pomegranate < orange 831 | olive < six 832 | peach < tiger 833 | ten < cyan 834 | six < teal 835 | lemon < turquoise 836 | yellow < blue 837 | mango < deer 838 | dolphin < three 839 | pink < two 840 | leopard < pomegranate 841 | red < three 842 | nine < grape 843 | orange < fox 844 | orange < gray 845 | green < red 846 | pomegranate < cat 847 | blue < watermelon 848 | dolphin < seven 849 | coconut < magenta 850 | tiger < purple 851 | lemon < tiger 852 | three < seven 853 | grape < purple 854 | turtle < violet 855 | blueberry < one 856 | goat < seven 857 | five < three 858 | tiger < avocado 859 | goat < magenta 860 | cyan < crimson 861 | dolphin < kangaroo 862 | kangaroo < leopard 863 | fox < apple 864 | magenta < watermelon 865 | coconut < six 866 | avocado < pomegranate 867 | four < cat 868 | brown < indigo 869 | ten < cow 870 | navy < olive 871 | peach < yellow 872 | one < deer 873 | silver < elephant 874 | orange < olive 875 | dolphin < five 876 | purple < elephant 877 | lemon < eight 878 | crimson < banana 879 | dog < three 880 | goat < brown 881 | two < three 882 | avocado < cat 883 | grape < ten 884 | cyan < blue 885 | apricot < green 886 | violet < two 887 | kangaroo < robin 888 | turquoise < magenta 889 | olive < eight 890 | six < violet 891 | cat < giraffe 892 | green < zero 893 | seven < green 894 | robin < red 895 | eight < fox 896 | three < pink 897 | giraffe < mango 898 | cyan < cow 899 | magenta < brown 900 | nine < lion 901 | orange < six 902 | nine < four 903 | coconut < green 904 | two < six 905 | turquoise < zero 906 | two < avocado 907 | green < five 908 | cat < blueberry 909 | purple < avocado 910 | indigo < orange 911 | turquoise < robin 912 | purple < four 913 | indigo < teal 914 | cow < dog 915 | cow < teal 916 | fox < kangaroo 917 | orange < violet 918 | green < robin 919 | turquoise < dolphin 920 | avocado < cyan 921 | unicorn < eight 922 | goat < purple 923 | cat < silver 924 | fox < two 925 | two < tiger 926 | pink < giraffe 927 | kangaroo < turquoise 928 | olive < pink 929 | grape < avocado 930 | coconut < orange 931 | ten < mango 932 | yellow < olive 933 | yellow < turtle 934 | tiger < deer 935 | tiger < three 936 | green < leopard 937 | apple < purple 938 | two < nine 939 | fox < turtle 940 | seven < gray 941 | coconut < zero 942 | zero < five 943 | cow < mango 944 | kangaroo < eight 945 | green < peach 946 | eight < elephant 947 | dog < indigo 948 | nine < red 949 | olive < tiger 950 | nine < mango 951 | unicorn < navy 952 | dolphin < dog 953 | indigo < elephant 954 | grape < tiger 955 | nine < fox 956 | green < purple 957 | magenta < tiger 958 | five < nine 959 | leopard < three 960 | olive < purple 961 | nine < purple 962 | goat < deer 963 | navy < cat 964 | navy < cow 965 | unicorn < robin 966 | pomegranate < fox 967 | crimson < five 968 | violet < kangaroo 969 | three < lemon 970 | cyan < orange 971 | five < fox 972 | orange < kangaroo 973 | cyan < teal 974 | six < mango 975 | zero < cat 976 | indigo < dolphin 977 | nine < blueberry 978 | dog < cow 979 | elephant < orange 980 | olive < turquoise 981 | goat < blue 982 | yellow < ten 983 | kangaroo < lion 984 | mango < turtle 985 | peach < silver 986 | one < silver 987 | crimson < turtle 988 | navy < magenta 989 | fox < robin 990 | teal < two 991 | two < giraffe 992 | lion < blueberry 993 | peach < violet 994 | blue < navy 995 | unicorn < indigo 996 | zero < fox 997 | indigo < zero 998 | one < coconut 999 | orange < giraffe 1000 | coconut < lion 1001 | -------------------------------------------------------------------------------- /datasets/gyafc_clean/gt_formal_output1.txt: -------------------------------------------------------------------------------- 1 | just be her bff 4 real. 2 | do you really want an answer? 3 | i wasn't trying to be a jerk. 4 | some ladies just hint at what they want and expect the men to figure it out. 5 | how about we grab some starbucks and talk, and see what happens? 6 | i hate seeing my daddy sad: ( 7 | ask him if u should go see a dr. 8 | dontcha think he has enough to worry about? 9 | i dont think he'll step out or dip if you dont slap skins with him 10 | will i feel dat fo eva 11 | but he mite like all that teasing and skeezin with no pleasin 12 | you needs 2 try agin 13 | some men will shave for the right chick 14 | if u like him smile and speak nice. 15 | he likes ya but i don't know about love. 16 | bitches be crazy 17 | im too picky, about people, how do i stop? 18 | that sounds pretty girly 19 | if the good is better than the bad, its worth it 20 | only if both of you agree 21 | well why u got ta like the one boy who messes wit u? 22 | and, brothas don't b knowin how much dey need u until u gon. 23 | if a dude hits on u don't be scared to say no 24 | do i want luv 25 | if you give them creepy looks they'll think ur leering 26 | you're right it's realy confusing 27 | how i convince my lover to show his love for me?? 28 | i mean, if you're that into them, sure 29 | dun sell yourself out, girl, his money don't matter that much 30 | don't let these girls hurt your feelings, you sound like an upstanding dude 31 | i dunno, but there's another site that i go to for that 32 | nah, man, think hard about how many arrogant chicks there are around 33 | you could look easy if you don't have a guy. 34 | then he's like do you want to use my chapstick? 35 | don't ever let a guy treat you bad. 36 | don't think so, he's too big and i'm a weakling. 37 | tell him when and if he says yes! 38 | what mistake did u make? 39 | tehres a song by king called only your mother loves you 40 | why do we stay when theyre mean to us after we've falllen in love? 41 | if youve done it, how was it? 42 | its not really a turnoff when she gets super drunk 43 | i guess if thats what you want 44 | its just like trying to ditch a bad drug habit 45 | i need to try that dating trick next time for sure 46 | just let me know what you want 47 | yall are both good with how things are going so thats good. 48 | dont let her take over ur life, she will just have to deal!! 49 | if hes not into you, there are plenty of other men out there who are. 50 | no guy can give you everything you want... 51 | you need to ``find urself'' if you wan na stay married. 52 | start with a boring question and just keep it going! 53 | ask if they want to travel somewhere with you. 54 | just put some trust in him, and let it go. 55 | i'm building up the courage to go for it. 56 | find the person who loves you as much as you love them. 57 | i'll be your girl if you come to my town. 58 | stay far far away from him. 59 | it's your choice man, but i hate all of them. 60 | i say around 3 or 4 61 | he told that woman he's a player. 62 | my ex's made me realize why my wife was ``the one''. 63 | don't be greedy and take both of them at the same time. 64 | having sex is not the only thing that matters in life. 65 | i use yahoo for my email and it works fine for me. 66 | its just fun for me 67 | he was a little busy after 68 | thats just fucking sex 69 | dont let her know you think shes hot 70 | sounds like you're goin thru a lot right now 71 | but u should ask a girl about the symptoms bc im a boy 72 | i use to always feel the same thing but everything changed when i met my true luv!! 73 | marry your bf if it doesn't work with anyone else. good choice 74 | well i don ’ t want to just run up and ask will you go oit with me?? 75 | unless you reaaalllyyy know the guy your gon na get hurt. 76 | your beliefs aren ’ t good or bad. 77 | they're annoying all day. 78 | go to hawaii or pay someone. 79 | i can just forget about him like it's nothing. 80 | is it like a clear liquid that gets yellow when it ends up drying? 81 | do you hate it if she sits on you bff's lap when drunk? 82 | he was a control freak 83 | one of the gifts we can give ourselves - determination. 84 | you should only marry a man after you been with him for 2 yrs. 85 | to my girls at cone, hi and hugzzz!!! 86 | be yourself at all times 87 | say it slow and repeat 88 | a perv hit on me 89 | some dudes are dumb 90 | there are plenty of fish in the sea 91 | either make up or move on 92 | dude you've never talked to her, how can you love her 93 | finda cute girl who can sing 94 | you arent a slut 95 | true love will happen to you, this time. 96 | look up the answer on your own, dude. 97 | 2 months, and they will have moved on from you. 98 | you won't even be able to tell what color it is once time has passed. 99 | he wants to know if u like him 100 | if she doesnt like u..... 101 | your not a slut. dont fret 102 | u cant force another to like u 103 | break up and focus on others 104 | have you had a pap smear? 105 | i can say ``yes'' and i have long black hair and i'm proud! 106 | my friend is in the same position... 107 | maybe your young... idk. 108 | i ’ m anonymous but i ’ ve been working with this person for a while. 109 | if you love him for real, you ’ ll tell him 110 | if you really like her, go with your feelings 111 | if you think u can afford it, go for it if you want 112 | stay alive and get things back to normal 113 | what are you saying about her 114 | not flirt. no one likes it 115 | i think some women would tell you that it's pretty hurtful 116 | it's be easier if i were more pissed off!!!! 1 117 | if they've forgotten about you, then forget about them too 118 | your punishment - no tv for a month. 119 | you've known each other for a long time it won't seem crazy 120 | i still don't get it. can u say it again? 121 | u have to give them a chance to see all your talents 122 | wrong man could be a problem 123 | my bf doesnt want to lose me cause he loves me a lot 124 | it kinda matters if they're brunette or not 125 | it'll get played out at some point 126 | we don't admit it, but we all want to know we are sexy, sexy beasts. 127 | i love dancing at the clubs in downtown huston 128 | good luck and don't forget to do your homework! 129 | i don't like it when women use too much makeup, they are beautiful naturally. 130 | if you want him, go get him. 131 | but now me and her are both divorced 132 | um who said women don't have feelings??? 133 | actions are always better then jus words! 134 | i'm thinking a regular pink color... but what should go with it? 135 | see if he calls you or emails you or if he even cares! 136 | don't ignore her always listen to her and don't change the subject!! 137 | he is my only way to get money till i graduate from college 138 | she'll let you know if it isn't okay 139 | i don't care either way! 140 | don't eat where you shit. 141 | i'm gon na get blonde highlights done. 142 | forget about it, focus on school and your bf. 143 | this was a bad idea.. 144 | i need at least 5 minutes of groping. 145 | what's a dope prez for my bro in laws bday? 146 | don't be a show off, but be yourself! 147 | if dey do not, den yuh will have tuh continue wit' your life 148 | your first kiss, howevuh, should be heavenly 149 | in a family life, dis is de most beautiful 150 | most impawhtantly, be genuine 151 | i got 3 guys who i can do that with. 152 | get mad if he flirts with her 153 | stay with the 14 or 13 yo boys 154 | tell him u just wan na be friends, and your disappointed by him being so dumb 155 | it's kinda nice... like getting your dad's blessing. 156 | wait for him or you'll seem desperate 157 | if he keeps it up, you can tell he was only after sex 158 | i love have fun wit you so i answered this? 159 | how u feel 160 | give your bro som $ $ and get rid of him 161 | he might call 162 | you can try to think about life without them, that's something. 163 | i'll go with domestic goddess 164 | do u rly love her? a lot? 165 | men cheat more than women 166 | save money on flowers 167 | fix the problem or leave if you are so unhappy 168 | looks are decieving, he may be a shitty boyfriend 169 | you can make a citizen's arrest. 170 | having a huge package doesn't mean you're good at doing it! 171 | yes, it looks so hot on almost any woman! 172 | dudes can't keep secrets worth af 173 | depends on the dude, i guess 174 | people were asking me why i was in such a hurry 175 | neither of them respect ur feelings girl 176 | i see him in psr every week but i can't gather up the gumption to talk to the guy 177 | i dont find guys i like 178 | i like men that shave legs 179 | is it gay? ir curiosity 180 | i say they do 181 | its a mistake u cant fix 182 | i lost my v card when i was 16 in a park! lol 183 | they've never met in real life... but she says they're in love!!! 184 | ask if he wants to dance w/ you! 185 | live your life to the fullest and have faith!!! 186 | idk if you fell over, but i'd like to have sex w/ you! 187 | he knows you're into him so just ask him out 188 | get over your fear and tell him 189 | why the f would i drive 5 hours to spend time with somoene unless i really liked her? 190 | aint she got friends to talk on the phone with her? 191 | it's good to play sports with other guys. 192 | when you fall in love everything feels wonderful and you can't stop thinking about your bae. 193 | i can't imagine a life without my hubby. 194 | it's easy to hurt those we love most. 195 | go for it if you both feel that way when you're grown. 196 | when i call you a man take that with a grain of salt. 197 | i like small cups of a and b. 198 | the hot girls are by the hideous men. 199 | i am with them and it happened to me. 200 | tell the truth about your feelings for her. 201 | you should show her your naked baby pictures. 202 | why is marriage important to the gay community? 203 | we had sex on the hood of my car while it drizzled around us. 204 | why do guys got ta grow up if girls don't expect them to? 205 | you got ta do this a few times. 206 | making love is like a fantasy. 207 | rite now i think hes mad, and doesnt wan na be with u. 208 | lem me know if youve got the answer plz! 209 | red is a love color, nblack is a death color, etc. 210 | i like fat ones, otherwise i feel like im missing out! 211 | ask someone if you dont kno the answer. 212 | he'll try to make it up, you'll like it. 213 | if he's picking on you he likes ya! 214 | i've been liking this girl since 9th grade. 215 | she looking at u? 216 | didnt know life could be such a blessing!! 217 | go with the best thing for you two. 218 | look at like this, you got a real number not fake: ) 219 | i don't think you fool around. 220 | they should get married if they can't keep their hands off each other. 221 | seems like she's calling him all the time. 222 | might as well keep on keeping on 223 | how many loveless relationships have you been in? 224 | is it right to break up with him cause of his mom 225 | make a move and if she doesnt, give her space 226 | they got married fast! 6months fast! 227 | show her your skills man. 228 | you look like a freak. 229 | i give props to that culture, because they really have it down pat. 230 | you'll never know for sure unless you ask. 231 | don't worry about it or over think it. 232 | what you think the worst could be? 233 | i don't even know what to do cus i can't stand when the girl is like that, so like what do i do now? 234 | find somebody else somebody down that don't hate you all the time. 235 | dont be a dick 236 | some guys just can't dance 237 | these youngins r dumb. 238 | depends did ya trust him b4 239 | fat and ugly guys can't b picky. 240 | people choose marriage 4 a reason. 241 | it ’ s not worth arguing with you all the time 242 | good luck asking your cell mate that question 243 | do i like you? 244 | you kissed what?? 245 | some women are bi curious like me. 246 | 20 to 32, just not younger than you. 247 | after this you can make a decision 248 | he cheated, so i can lift my husband 249 | look on the bright side 250 | is your goal to just hang out 251 | in my opinion he would make a good valentine 252 | with so many temptations i am not sure i can do it. 253 | it takes what? a yr to feel comfy in relationshipz? 254 | i u ready, then cool 255 | hello yea i'm tryna meet chicks. 256 | it is what it is. 257 | you need to have safety precautions when you friend total strangers on the net. 258 | can you picture having kids with that guy? 259 | if he isn't interested in you, it's his loss, and why date a fool anyway? 260 | you usually learn the best lessons when you go for it, and make mistakes. 261 | you did marry him for love i hope. 262 | at least i have someone who will be there for me right at home. 263 | relax and get a massage and just let it all go. 264 | where should i even start here. 265 | depends, are you hot with a gr8 body??? 266 | wait.. you mean yr avatar? 267 | he'll call if he cares 268 | there is a idfference between fucking and making love 269 | u dont got ta stay together even if u got a kid 270 | are u a country girl?? ior are u from the city 271 | getting married and having kids can wait. studying cant 272 | i've never been. 273 | you wan na be in a relationship with the same guy? 274 | can you have sex with pamela anderson often? 275 | don't make her hate you just to get your point across 276 | this is too close to home, my moms has cancer too. 277 | i luv to put my hands thru curlee hair 278 | i mean u 2 together 279 | just be patient and the rite girl will find u 280 | i want my wife to grow 2 281 | he seems shy, so you outta make the first move. 282 | the doc was shocked too! 283 | you may think it's no, but it's def yes! 284 | that's for boys to do - not men. 285 | if u wanted u could ask them out and see. 286 | and that did happen 2x to me. 287 | don't make up some need in yur head to go out with that dude. 288 | if he was the right guy, you wouldn't need to ask. 289 | hope this streak never ends~ 290 | girls just want guys to be straight with their feelings. 291 | nothin' like sex to bring out the passion in a woman 292 | chicks like to be impressed so keep on impressing her 293 | getting close and kissing me softly on me forehead. 294 | that ain ’ t me but i know peeps like that. 295 | depends on how hard you ’ re falling for this bro! 296 | i just want to thank you ahead of time for the encouragement. 297 | i just don't know why they want to get married so bad. 298 | we like to be pursued, but with a light touch. 299 | i think theme parks and museums are both fun places to visit. 300 | i need to kno if im in love with this person or not. 301 | do you like manicures or fake nails?? 302 | plan what u want, and dont worry about what everyone else wants. 303 | i dont know what ur saring. 304 | ur perfume turns me on 305 | i don't think most men care but it depends on the guy. 306 | its got nothing to do w/ being vulnerable or young or submissive... 307 | don't get married so early in your relationship... 308 | your not a good person if you plan on cheating on your wife!! 309 | be the best you that you can be and see if he likes it. 310 | hes the only one for you. 311 | don't be a homewrecker. he's taken. 312 | tell her that it's bs she makes you daily check in! 313 | neighbor asked me out when i moved in 314 | most teen guys prefer teen gals who like hooking up... 315 | he played a good game of b-ball. 316 | don't sweat it! 317 | do u love fighting or what? 318 | important thing is to not try to get revenge on people who wronged u 319 | a guy willing to move 21hrs and 1400mi for me 320 | have you gone to the doctor yet? 321 | she's not annoying but she was totally thin. 322 | get a god dang life and leave me along. 323 | that's like eating too much chocolate and saying you gained too much. 324 | i was there and he was there. 325 | no ball and chain, then do it! 326 | these sorts of guys turn into wife beaters if you don ’ t give them what they want.... 327 | so do you think he ’ s cute or not? 328 | well if you ’ re certain then tell the parentals that you haven ’ t yet. 329 | a lot of people are going to tell you no.. 330 | does he feel the same about you? 331 | some guys don't use that word unless they mean it. 332 | tell him how you feel if you think he likes you. 333 | i think everyone should keep their own name 334 | boys come and go but friends are forever!!! 335 | try and imagine that it was love < 3 336 | i think you'll be ready even if your relationship survives the 4 seasons... 337 | okay, well right now some guys like that kind of women and other don't 338 | if you want kids, be a house mom. 339 | young boys and older women, yes or no? 340 | he wanted to keep going. 341 | u cant use another word for love 342 | self image is important 343 | im a girl and i say such things 344 | girls can be perverted too 345 | yeah i like flowers but they die too quick: p 346 | theres some films i havent seen 347 | she's super sweet, exactly what yr after! 348 | use your brain and not whats in yr pants, she might not love you 349 | don't take this the wrong way, but i've never heard of a guy with a name that girly. 350 | i'm not gon na waste my time returning his call. 351 | i wasn't really feeling any of the ladies in my list, so i settled for my neighbor's wifey. 352 | openness and honesty are the core of any marriage. 353 | just ignore what anyone else suggests. 354 | how do i impress her? 355 | a rose says i love you 356 | dont worry, it's weird, not bad 357 | figure out what works for both of you. 358 | she ’ s enough of a woman but she ain ’ t classy like us. 359 | yep, especially after she got really sick.... 360 | can ’ t help you anymore unless you give me mor info. 361 | there ’ s no way to be good or bad at kissing. 362 | just think it out 363 | having fun with dudes is okay 364 | that's probably more of a trust issue 365 | i need help understanding this. 366 | move on, there are plenty of fish in the sea. 367 | he probably wasn't into it either. 368 | when you can, make it yourself. 369 | u do anything else than work & fix stuff? 370 | i let her know that i really luv her. 371 | the uppers think the frosh gals are easy game. thats messed up! 372 | should i really just kiss'em? 373 | it should be in the deleted stuff if you take a peek. 374 | i hope you find a cute girl 375 | thats a bad idea, its not for that purpose 376 | its a long ass list 377 | brokeback mountain, youd like it even if youre a guy 378 | if its not dependent on sex, thats fine 379 | yah, but it's bad luck to see the bride before the ceremony, y'know? 380 | hold on and godspeed! 381 | uh, just a small family thing at my house 382 | if you're concerned about the performance, don't go through with it 383 | i'm married now 384 | it depends on age and upbringing 385 | you need to choose your kid's last name 386 | kid's life is hard without a dad 387 | baldies look like kids, it's ugly 388 | the last games score was...? 389 | choose the guy you're with, or your ex. 390 | best and easiest is fly or drive to vegas and get hitched. 391 | yes i just wan na get to know u 392 | size c or d at the most but the rest new much 393 | lie to them and then just keep lying! 394 | it ’ s double the pleasure since they haven ’ t performed before 395 | my wife doesn ’ t like wearing those brands 396 | we have no idea what u look like 397 | opposites attract, we can always find a life partner if we look hard enough 398 | lol, this is def better than my everyday life 399 | eh, you went with your ex there, don't go there with your new date, too 400 | talk to your bff and find out if it's all good 401 | weird, but it worked for me in about 3 to 5 days or so 402 | it annoys me when you get teased by him about everything!! 403 | no, i'm still unmarried. 404 | a lot of dudes like big chicks, mostly black guys 405 | shouldnt matter. you like him then say 406 | do they throw shade when youre together? 407 | dont get married if u feel leike that 408 | it depends if they are in vain 409 | move on and date someone who likes u 410 | get to know her and then ask the other out 411 | i asked if he blowed me off 412 | nasty old guys like young asians 413 | do what you want! 414 | hmm, tell him you just wan na keep this casual... 415 | thats a hard situation and there aren't any easy answers! 416 | they have been hard but good years. 417 | we seen each other eberyday for like 4 or 5 years! 418 | if i am the guy you want then be wit me! 419 | live a little and flirt! 420 | stay with what u feel inside. 421 | good luck 422 | is it bad if i havent kissed her since we started dating 8 months 423 | its her choice 424 | they look better than other countries 425 | itll be okay the next day 426 | ask her out for dinner and a movie. 427 | some guys don't think about it they just do it. 428 | speak english please. i can't understand you. 429 | sorry i don't feel that way. 430 | just remember that a bird can't fly with only one wing 431 | yeah i think about this a lot 432 | if you're just whatever about it you'll never find anyone to date 433 | there's this girl that i met and i'm wondering if she likes me 434 | dont just wait around for the call 435 | quit your bitching, nobody cares if you're gay 436 | break up with him if he cant get it together 437 | it's possible he loves you and will end up letting his dad down. 438 | fix it before you regret it.... time is a-wasting. 439 | this chick thinks bad boys are way over-rated. 440 | i wan na hear what a black chick would say about this. 441 | help me beat her in this debate. 442 | almost all of those guys are near your age. 443 | when will people realize and notice shit. 444 | i'm not sure what i should do. 445 | you go and get wasted at the bar. 446 | i acted like that with my new lover 447 | if you have to ask, forget about it. 448 | don't wait for him to see you're the one that's good. 449 | ugh, still stuck w/ this even though getting divorced 450 | your bae will just think your being an ass. 451 | why don't you try dating chicks if you can't stand dating guys. 452 | my dirty ex cheated on me after 3 full years 453 | you will know soon enough. after six month see what happens. 454 | the one i thought was good goes a little something like this. 455 | not all dudes dumb chicks 456 | its always tons of stuff to do over there 457 | is it ight to smooch on the 1st date? 458 | i got a homegirl i'm close with 459 | they in lust *lol 460 | my friend says im too nice and i should say no 461 | there is always an exception 462 | not all men, but some o 463 | if u do that, u keep ur last name 464 | just walk up, introduce yourself, and ask if she's available. 465 | i was all hot and bothered, but i got out of there quick. 466 | big girls can still be total cuties. 467 | not everybody here is a student 468 | unwrap starburst and tie cherry stems with your tongue for practice 469 | vote for me in best answer and ill pay for you 470 | look in public, youll feel more comfortable 471 | just flirt with him and see what happens 472 | also it takes the pressure off me to ask somebody 473 | come on! go out with me plz!! 474 | she's not gon na forget that i messed up. 475 | how do i make it seem i want more than sex? 476 | i'm a guy i know about this stuff. 477 | dont worry. ur 13 478 | go fon a trail with a friend 479 | it u love each other, itll work out 480 | eye contact is improtant 481 | i can lift him no problem 482 | he needs to regain your trust and start completely over again 483 | are you sure you want to ask her out? 484 | i hope u get the info soon 485 | play truth or dare together 486 | wait till uno for sure. then decide 487 | thats dumb. u just need a theory class 488 | quit if u eat something like that 489 | i keep thinking about fucking her 490 | u will only know if u ask him directly 491 | jamie will ask out rubi 492 | its still happening cause neither has courage to end it 493 | maybe they just dig learning about that stuff. 494 | he's dribbling the ball, he shoots annnnnnd score, 2 points for the champion. 495 | do dudes really want big girls over other types? 496 | i know he loves you like you like you love him, but no one knows the future. 497 | if he respects you, he will understand what u need. 498 | we have our own sins started. 499 | his answer was good enought for me. 500 | love him sure, but let him go. -------------------------------------------------------------------------------- /calculate_all.py: -------------------------------------------------------------------------------- 1 | # Import relevant libraries and dependencies 2 | import os 3 | import json 4 | import glob 5 | import argparse 6 | from tqdm import tqdm 7 | 8 | import csv 9 | 10 | from datasets import load_metric 11 | 12 | import numpy as np 13 | import torch 14 | from torch.nn import functional as F 15 | from transformers import ( 16 | AutoModelForMaskedLM, 17 | AutoModelForCausalLM, 18 | AutoTokenizer, 19 | AutoModelForSequenceClassification, 20 | ) 21 | from fairseq.models.roberta import RobertaModel 22 | from fairseq.data.data_utils import collate_tokens 23 | 24 | import math 25 | 26 | from transformers import BertTokenizerFast 27 | 28 | import datasets 29 | 30 | FROM_TO_DICT={ 31 | 'positive': 'negative', 32 | 'negative': 'positive', 33 | 'formal': 'informal', 34 | 'informal': 'formal', 35 | 'ungrammatical': 'grammatical', 36 | 'grammatical': 'ungrammatical', 37 | 'Shakespearean': 'modern', 38 | 'modern': 'Shakespearean', 39 | 'oldEnglish': 'modernEnglish', 40 | 'modernEnglish': 'oldEnglish', 41 | 'symbolic': 'English', 42 | 'English': 'symbolic', 43 | } 44 | 45 | def label_fn(model, label): 46 | return model.task.label_dictionary.string( 47 | [label + model.task.target_dictionary.nspecial] 48 | ) 49 | 50 | 51 | def compute_sentence_probs(model, tokenizer, preds, device): 52 | """ 53 | Computes the sentence-level probability of texts in a list. 54 | """ 55 | probs = [] 56 | for pred in preds: 57 | sentence = pred 58 | sent_prob = get_sent_prob(model, tokenizer, sentence, device) 59 | probs.append(sent_prob) 60 | return probs 61 | 62 | 63 | def get_sent_prob(model, tokenizer, text, device): 64 | """ 65 | Calculates the sentence-level probability. 66 | """ 67 | with torch.no_grad(): 68 | text_ids = tokenizer.encode(text, return_tensors="pt", truncation=True).to(device) 69 | if text_ids.shape[1] > 1: 70 | input_ids = text_ids 71 | target_ids = text_ids 72 | outputs = model(input_ids, labels=target_ids) 73 | logits = outputs.logits 74 | probs = F.softmax(logits, dim=2).squeeze() 75 | sentence_prob = 1. 76 | 77 | for i in range(input_ids.shape[1]-1): 78 | current_probs = probs[i] 79 | gt_index = input_ids[0][i+1].cpu().item() 80 | gt_prob = current_probs[gt_index] 81 | sentence_prob *= gt_prob.cpu().item() 82 | return sentence_prob 83 | return 0. 84 | 85 | 86 | def calc_sentiment_accuracy(model, tokenizer, predictions, target_style, device): 87 | """ 88 | Given a model, a tokenizer, and a list of predictions, calculates the style-transfer strength (accuracy score) of the list. 89 | """ 90 | if model is None: 91 | return None, 0., None 92 | with torch.no_grad(): 93 | # first negative, second positive 94 | correcto = np.zeros(2) 95 | correct = 0 96 | probs = [] 97 | pbar = tqdm(predictions, desc='Calculating the style-transfer (sentiment) accuracy...') 98 | for i, sample in enumerate(pbar): 99 | # if Amazon or Yelp: 100 | if tokenizer: 101 | # If there is a tokenizer, then it means that we are doing sentiment classification and using BERT-classifiers 102 | input = tokenizer.encode(sample, return_tensors="pt", truncation=True, max_length=512).to(device) 103 | pred = model(input) 104 | positive_prob = torch.softmax(pred['logits'], dim=1)[0][1] 105 | if target_style == 'negative' or target_style == 'formal': 106 | probs.append(1. - positive_prob.cpu()) 107 | if (1. - positive_prob) > 0.5: 108 | correcto[0] += 1 109 | correct += 1 110 | else: 111 | probs.append(positive_prob.cpu()) 112 | if positive_prob > 0.5: 113 | correcto[1] += 1 114 | correct += 1 115 | else: 116 | # This means that we are using the Shakespeare classifier from https://drive.google.com/drive/folders/1XRVeNjxlXojEAzJOOyHVPg0z_9VyIpmo 117 | samples = [sample] 118 | 119 | inputs = [model.bpe.encode(sample) for sample in samples] 120 | input_batch = collate_tokens([model.task.source_dictionary.encode_line(" " + input + " ", append_eos=False) for input in inputs], pad_idx=1) 121 | input_batch = input_batch[:, : 512] 122 | 123 | with torch.no_grad(): 124 | preds = model.predict('sentence_classification_head', input_batch.long()) 125 | pred_labels_label = [label_fn(model, x.argmax(axis=0).item()) for x in preds] 126 | if target_style == 'modern': 127 | if pred_labels_label[0] == 'modern': 128 | correcto[0] += 1 129 | correct += 1 130 | else: 131 | if pred_labels_label[0] == 'original': 132 | correcto[1] += 1 133 | correct += 1 134 | return correcto/float(len(predictions)), correct/float(len(predictions)), probs 135 | 136 | 137 | def calc_blue(metric, predictions, references, lowercaseBool=True): 138 | """ 139 | Given predictions and references, calculates the BLEU or sacreBLEU score. 140 | """ 141 | results = metric.compute(predictions=predictions, references=references) #lowercase=lowercaseBool 142 | return results 143 | 144 | 145 | def calc_perplexity_sent_average(model, tokenizer, predictions, device): 146 | """ 147 | Calculates the sentence-level perplexity. 148 | """ 149 | with torch.no_grad(): 150 | ppls = [] 151 | error_s = 0 152 | pbar = tqdm(predictions, desc='Calculating the sentence-level perplexity scores...') 153 | for i, sample in enumerate(pbar): 154 | tokens_tensor = tokenizer.encode(sample, return_tensors="pt", truncation=True, max_length=512).to(device) 155 | if tokens_tensor.shape[1] > 1: 156 | loss = model(tokens_tensor, labels=tokens_tensor)[0] 157 | ppl = np.exp(loss.cpu().detach().numpy()) 158 | if math.isnan(ppl): 159 | error_s += 1 160 | else: 161 | ppls.append(ppl) 162 | return sum(ppls)/len(ppls), error_s 163 | 164 | # Reference: https://github.com/shentianxiao/language-style-transfer/blob/f078a9d342705ea62b04dd3ac5926d3467c2ac63/code/language_model.py#L102 165 | def calc_perplexity_token_average(model, tokenizer, predictions, device): 166 | """ 167 | Calculates the token-level perplexity. 168 | """ 169 | with torch.no_grad(): 170 | nlls = [] 171 | total = 0 172 | error_s = 0 173 | pbar = tqdm(predictions, desc='Calculating the token-level perplexity scores...') 174 | for i, sample in enumerate(pbar): 175 | text_ids = tokenizer.encode(sample, return_tensors="pt", truncation=True, max_length=512).to(device) 176 | if text_ids.shape[1] > 1: 177 | input_ids = text_ids[:, :-1] 178 | target_ids = text_ids[:, 1:] 179 | outputs = model(input_ids) 180 | preds = outputs.logits[0] 181 | calc_loss = F.nll_loss(F.log_softmax(preds, dim=1), target_ids[0]) 182 | neg_log_likelihood = calc_loss * input_ids.shape[1] 183 | total += input_ids.shape[1] 184 | nlls.append(neg_log_likelihood) 185 | else: 186 | error_s += 1 187 | ppl = torch.exp(torch.stack(nlls).sum() / total) 188 | return ppl.item(), error_s 189 | 190 | 191 | def all_references_in_one(ref_list): 192 | num_refences = len(ref_list) 193 | num_sentences = len(ref_list[0]) 194 | references = [] 195 | for i in range(num_sentences): 196 | arr = [] 197 | for j in range(num_refences): 198 | arr.append(ref_list[j][i][0]) 199 | references.append(arr) 200 | return references 201 | 202 | 203 | def set_format(metric_name, dataset_type, exs): 204 | arr = None 205 | if metric_name == 'bleu': 206 | if dataset_type == 'ground_truth': 207 | arr = [[(ex.strip()).split(' ')] for ex in exs] 208 | else: 209 | arr = [(ex.strip()).split(' ') for ex in exs] 210 | else: 211 | if dataset_type == 'ground_truth': 212 | arr = [[ex.strip()] for ex in exs] 213 | else: 214 | arr = [ex.strip() for ex in exs] 215 | return arr 216 | 217 | 218 | def read_txt_file(path): 219 | return open(path, 'r').readlines() 220 | 221 | 222 | def choose_best_output_from_oracle(model, tokenizer, preds, target_style, device): 223 | with torch.no_grad(): 224 | probs = [] 225 | for sample in preds: 226 | # If there is a tokenizer, then it means that we are doing sentiment classification and using BERT-classifiers 227 | input = tokenizer.encode(sample, return_tensors="pt", truncation=True, max_length=512).to(device) 228 | pred = model(input) 229 | positive_prob = torch.softmax(pred['logits'], dim=1)[0][1] 230 | target_prob = positive_prob.cpu().item() 231 | if target_style == 'negative': 232 | target_prob = 1. - target_prob 233 | probs.append(target_prob) 234 | return None, None, probs 235 | 236 | 237 | def choose_best_output_from_mlm(model, tokenizer, preds, target_style, device): 238 | with torch.no_grad(): 239 | orig_style = FROM_TO_DICT[target_style] 240 | 241 | orig_style_token = tokenizer.encode_plus(orig_style)['input_ids'][1] 242 | target_style_token = tokenizer.encode_plus(target_style)['input_ids'][1] 243 | 244 | probs = [] 245 | for pred in preds: 246 | sentence = pred 247 | text = f'The following text is {tokenizer.decode(tokenizer.mask_token_id)}: {sentence}' 248 | input = tokenizer(text, return_tensors='pt', truncation=True, max_length=128).to(device) 249 | mask_index = torch.where(input["input_ids"][0] == tokenizer.mask_token_id) 250 | masked_output = model(**input) 251 | logits = masked_output.logits 252 | softmax = F.softmax(logits, dim = -1) 253 | mask_word = softmax[0, mask_index, :] 254 | full_probability = mask_word[0] 255 | orig_style_prob = full_probability[orig_style_token] 256 | target_style_prob = full_probability[target_style_token] 257 | normalized_prob = target_style_prob / (orig_style_prob + target_style_prob) 258 | probs.append(normalized_prob.cpu()) 259 | return None, None, probs 260 | 261 | 262 | def choose_best(preds, prompt, ignore_fluency, bertscore, oracle, model, tokenizer, ppl_model, ppl_tokenizer, target_style, device): 263 | if len(preds) == 1: 264 | return preds[0] 265 | 266 | # Similarity score 267 | references = [prompt] * len(preds) 268 | 269 | results = bertscore.compute(predictions=preds, references=references, lang='en') 270 | similarity_scores = [v for v in results["f1"]] 271 | 272 | # Sentiment/Style-Transfer (accuracy) score 273 | if oracle: 274 | _, _, sentiment_scores = choose_best_output_from_oracle(model, tokenizer, preds, target_style, device) 275 | else: 276 | _, _, sentiment_scores = choose_best_output_from_mlm(model, tokenizer, preds, target_style, device) 277 | 278 | total_probs = np.array(similarity_scores) * np.array(sentiment_scores) 279 | 280 | # Fluency score 281 | if not ignore_fluency: 282 | fluency_scores = compute_sentence_probs(ppl_model, ppl_tokenizer, preds, device) 283 | total_probs *= np.array(fluency_scores) 284 | 285 | max_index = np.argmax(total_probs) 286 | return preds[max_index] 287 | 288 | 289 | def read_json_file(path, choose_first, ignore_fluency, bertscore, oracle, model, tokenizer, ppl_model, ppl_tokenizer, target_style, device): 290 | with open(path, 'r') as json_file: 291 | json_data = json.load(json_file) 292 | inputs = json_data['inputs'] 293 | outputs = json_data['extracted_outputs'] 294 | 295 | if len(outputs[0]) > 1 and not choose_first: 296 | return [(choose_best( outputs[i], inputs[i], ignore_fluency, bertscore, oracle, model, tokenizer, ppl_model, ppl_tokenizer, target_style, device) + '\n') for i in range(len(outputs))] 297 | return [outputs[i][0] for i in range(len(outputs))] 298 | 299 | def read_jsonl_file(path, choose_first, ignore_fluency, bertscore, oracle, model, tokenizer, ppl_model, ppl_tokenizer, target_style, device): 300 | with open(path, 'r') as json_file: 301 | json_list = list(json_file) 302 | 303 | if len(json.loads(json_list[0])['generated_output'][0]) > 1 and not choose_first: 304 | return [(choose_best(json.loads(line)['generated_output'], json.loads(line)['prompt'], ignore_fluency, bertscore, oracle, model, tokenizer, ppl_model, ppl_tokenizer, target_style, device) + '\n') for line in json_list] 305 | return [(json.loads(line)['generated_output'][0]) for line in json_list] 306 | 307 | 308 | def read_file(path, choose_first=False, ignore_fluency=True, bertscore=None, oracle = False, model=None, tokenizer=None, ppl_model=None, ppl_tokenizer=None, target_style=None, device=None): 309 | if path[-5:] == 'jsonl': 310 | lines = read_jsonl_file(path, choose_first, ignore_fluency, bertscore, oracle, model, tokenizer, ppl_model, ppl_tokenizer, target_style, device) 311 | elif path[-3:] == 'txt': 312 | lines = read_txt_file(path) 313 | elif path[-4:] == 'json': 314 | lines = read_json_file(path, choose_first, ignore_fluency, bertscore, oracle, model, tokenizer, ppl_model, ppl_tokenizer, target_style, device) 315 | else: 316 | raise NotImplementedError('Oh no... input file is not recognized!') 317 | return lines 318 | 319 | 320 | def main(): 321 | # Parser arguments 322 | parser = argparse.ArgumentParser() 323 | parser.add_argument('--metric', type=str, default='sacrebleu', choices=['bleu', 'sacrebleu']) 324 | parser.add_argument('--case_sensitive', action='store_true') 325 | parser.add_argument('--dataset', type=str, default='yelp_clean') 326 | parser.add_argument('--output_dir', type=str, default='outputs/yelp_clean/contrastive/gptj_6B/4_shot_3_samples') 327 | parser.add_argument('--results_save_path', type=str, default=f'results/test.tsv') #final_few_shot_yelp_clean_choosefirst_luke.tsv ail_results_shakespeare_gpt2_medium 328 | parser.add_argument('--oracle', action='store_true') 329 | parser.add_argument('--choose_first', action='store_true') 330 | parser.add_argument('--ignore_fluency', action='store_true') 331 | parser.add_argument('--ppl_model', default='gpt2-large') 332 | parser.add_argument('--classifier', default='/nlp/scr/msuzgun/classifier/amazon_yelp/bert-base-uncased-yelp-clean-ep3') 333 | args = parser.parse_args() 334 | 335 | # Create the save directory if it does not exist 336 | results_save_path = args.results_save_path 337 | os.makedirs(os.path.dirname(results_save_path), exist_ok=True) 338 | 339 | dataset = args.dataset 340 | if 'amazon' in dataset or 'yelp' in dataset: 341 | styles = ['positive', 'negative'] 342 | elif 'shakespeare' in dataset: 343 | styles = ['Shakespearean'] 344 | elif 'jfleg' in dataset: 345 | styles = ['ungrammatical'] 346 | elif 'gyafc' in dataset: 347 | styles = ['informal'] 348 | 349 | # GPU / CPU 350 | device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 351 | 352 | # If not None, then it means that we are using the "oracle" classifier in the Prompt-and-Rerank pipeline 353 | oracle = args.oracle 354 | model_name = args.classifier 355 | 356 | if any(elt in dataset for elt in ['yelp', 'amazon', 'gyafc']): 357 | print('*** Loading the sentiment classifier...') 358 | # The sentiment classifiers we trained for Amazon and YELP were based on the BERT-Base model 359 | classifier = AutoModelForSequenceClassification.from_pretrained(model_name).eval().to(device) 360 | classifier_tokenizer = BertTokenizerFast.from_pretrained('bert-base-uncased') 361 | elif 'shakespeare' in dataset: 362 | print('*** Loading the RoBERTa-Large style classifier...') 363 | roberta_classifier_dir = args.classifier 364 | ckpt_path = f'{roberta_classifier_dir}/checkpoint_best.pt' 365 | data_path = f'{roberta_classifier_dir}/shakespeare-data-bin' 366 | classifier = RobertaModel.from_pretrained(model_name_or_path=roberta_classifier_dir, checkpoint_file=ckpt_path, data_name_or_path=data_path).eval().to(device) 367 | classifier_tokenizer = None 368 | else: 369 | print('Classifier is NONE!') 370 | classifier = None 371 | classifier_tokenizer = None 372 | 373 | # If we are not using the oracle classifier, then let's use an MLM (viz., RoBERTa-Large) for calculating accuracy scores 374 | if oracle: 375 | model = classifier 376 | tokenizer = classifier_tokenizer 377 | else: 378 | model = AutoModelForMaskedLM.from_pretrained("roberta-large").eval().to(device) 379 | tokenizer = AutoTokenizer.from_pretrained("roberta-large") 380 | 381 | print(f'Loaded model. Parameters: {sum(p.numel() for p in model.parameters()):_}') 382 | 383 | # BERTScore 384 | print('*** Loading the BERTScore model...') 385 | bertscore = datasets.load_metric("bertscore") 386 | 387 | # Fluency model -- fluency is measured in terms of perplexity 388 | print(f'*** Loading the PPL model ({args.ppl_model})...') 389 | ppl_model = AutoModelForCausalLM.from_pretrained(args.ppl_model).eval().to(device) 390 | ppl_tokenizer = AutoTokenizer.from_pretrained(args.ppl_model) 391 | 392 | # Reset the output file 393 | with open(results_save_path, 'w') as f: 394 | pass 395 | 396 | # Metric: BLEU or SacreBLEU 397 | if args.metric == 'bleu': 398 | metric = load_metric("bleu") 399 | else: 400 | metric = load_metric("sacrebleu") # preferred xmetric 401 | 402 | # Case sensitive (lowercase or not?) 403 | case_sensitive = not args.case_sensitive 404 | 405 | columns = [] 406 | for style in styles: 407 | for tag in ['ACCURACY', 'SBLEU-REF', 'SBLEU-INPUT', 'SENT-PPL', 'TOK-PPL', 'SENT-PPL-ERROR', 'TOK-PPL-ERROR']: 408 | columns.append(f'{style}-{tag}') 409 | all_columns = ['MODEL'] + columns 410 | template_zero = {} 411 | for col in columns: 412 | template_zero[col] = 0. 413 | 414 | seen_so_far = [] 415 | results_tsv = {} 416 | 417 | for style in styles: 418 | if 'amazon' in dataset or 'yelp' in dataset: 419 | target_style = 'positive' if style == 'negative' else 'negative' 420 | elif 'shakespeare' in dataset: 421 | target_style = 'modern' if style == 'Shakespearean' else 'Shakespearean' 422 | elif 'jfleg' in dataset: 423 | target_style = 'grammatical' if style == 'ungrammatical' else 'ungrammatical' 424 | elif 'gyafc' in dataset: 425 | target_style = 'informal' if style == 'formal' else 'formal' 426 | 427 | # References 428 | # references_path = f'datasets/{dataset}/gt_{style}_output.txt' 429 | # references = read_file(references_path) 430 | # references = set_format(args.metric, 'ground_truth', references) 431 | 432 | references_path_dir = f'datasets/{dataset}/gt_{style}_output*.txt' 433 | references_paths = glob.glob(references_path_dir,recursive=True) 434 | references_paths.sort() 435 | 436 | print(references_paths) 437 | references_arr = [] 438 | for ref_path in references_paths: 439 | ref = read_file(ref_path) 440 | ref = set_format(args.metric, 'ground_truth', ref) 441 | references_arr.append(ref) 442 | references = all_references_in_one(references_arr) 443 | 444 | # Inputs 445 | input_references_path = f'datasets/{dataset}/gt_{style}_input.txt' 446 | input_references = read_file(input_references_path) 447 | input_references = set_format(args.metric, 'ground_truth', input_references) 448 | 449 | # Prediction output directory 450 | prediction_dir = f'{args.output_dir}/*{style}*.jsonl' 451 | prediction_paths = glob.glob(prediction_dir,recursive=True) 452 | prediction_paths.sort() 453 | 454 | # Run 455 | for path in prediction_paths: 456 | print(path) 457 | model_name, delimeter_type = (path.split('/')[-1]).split(f'_{style}_') 458 | delimeter_type = delimeter_type.split('.jsonl')[0] 459 | predictions = read_file(path, args.choose_first, args.ignore_fluency, bertscore, oracle, model, tokenizer, ppl_model, ppl_tokenizer, target_style, device) 460 | predictions = set_format(args.metric, 'prediction', predictions) 461 | 462 | tmp = model_name + '_' + delimeter_type 463 | 464 | if not(tmp in seen_so_far): 465 | seen_so_far.append(tmp) 466 | results_tsv[tmp] = {} 467 | for col in columns: 468 | results_tsv[tmp][col] = 0. 469 | 470 | try: 471 | # Calculate the sacreBLEU scores w.r.t. predictions 472 | result = calc_blue(metric, predictions, references, case_sensitive) 473 | results_tsv[tmp][style + '-SBLEU-REF'] = result['score'] 474 | 475 | # Calculate the sacreBLEU scores w.r.t. inputs 476 | result = calc_blue(metric, predictions, input_references, case_sensitive) 477 | results_tsv[tmp][style + '-SBLEU-INPUT'] = result['score'] 478 | 479 | # Calculate the style-transfer (sentiment) accuracy score of predictions 480 | _, acc, _ = calc_sentiment_accuracy(classifier, classifier_tokenizer, predictions, target_style, device) 481 | results_tsv[tmp][style + '-ACCURACY'] = acc 482 | 483 | # Calculuate the sentence-level and token-level perplexity scores of predictions 484 | avg_ppl_tok, err_tok_level = calc_perplexity_token_average(ppl_model, ppl_tokenizer, predictions, device) 485 | results_tsv[tmp][style + '-TOK-PPL'] = avg_ppl_tok 486 | results_tsv[tmp][style + '-TOK-PPL-ERROR'] = err_tok_level 487 | avg_ppl_sent, err_sent_level = calc_perplexity_sent_average(ppl_model, ppl_tokenizer, predictions, device) 488 | results_tsv[tmp][style + '-SENT-PPL'] = avg_ppl_sent 489 | results_tsv[tmp][style + '-SENT-PPL-ERROR'] = err_sent_level 490 | 491 | print(f'{tmp}... acc: {acc} ; sent-ppl: {avg_ppl_sent}; token-ppl: {avg_ppl_tok}') 492 | 493 | except: 494 | print('****') 495 | print(path) 496 | print('****') 497 | 498 | with open(results_save_path, 'w') as output_file: 499 | tsv_writer = csv.DictWriter(output_file, fieldnames=all_columns) 500 | tsv_writer.writeheader() 501 | for model in seen_so_far: 502 | tmp_dict = results_tsv[model] 503 | tmp_dict['MODEL'] = model 504 | tsv_writer.writerow(tmp_dict) 505 | 506 | if __name__ == "__main__": 507 | main() -------------------------------------------------------------------------------- /datasets/gyafc_clean/gt_formal_output2.txt: -------------------------------------------------------------------------------- 1 | u have to be her friend. 2 | that sounds more like a rhetorical question than a real one. 3 | i'm not tryin to be mean.... 4 | women send men mixed messages sometimes 5 | up fr coffee and chat then see what happens? 6 | hate it when my dad is bummed out. 7 | ask him if you should see a doc or sumthin' 8 | give him a break, you're not the center of the dang universe. 9 | i don't think he'll leave or cheat if u dont screw him 10 | will i always feel like that??? 11 | but he may like all the coyness and yearning 12 | and i wan na try again 13 | some men shave because there women likes them to shave. 14 | well if you really like this guy smile and talk to him with respect. 15 | i'm not sure if he loves you but i do know for sure that he definitely likes you. 16 | women are complicated because that is how men see us. 17 | im overly selective with people. how do i stop 18 | that sounds girly 19 | the good may be better than the bad 20 | only if u both agree 21 | why do u like the boy who bullies u 22 | men dont think they need u till ur gone 23 | ehhhhh don ’ t be scared to tell a bro no! 24 | is lov what i want?? 25 | they will thing u a creep if u leer at them 26 | ur right. it is complicated 27 | how do i get my love to show he lvoes md 28 | yeah if you defo love them 29 | doesnt matter if he us rich, dont sell yourself 30 | you sound like a great guy, dont let these women get you down 31 | idk but theres another site i go to 32 | you need to think about how many arrogant women there are out there 33 | well if your not dating then that seems kinda like a whore move 34 | and so he was like, do you wan na use my chapstick? 35 | you shouldn't ever ever ever let a man treat you wrong! 36 | definitely not, he's huge and i'm really gd weak 37 | go on and tell him if he says yes 38 | what mistake did u take part in 39 | king's got that joint'only ur mother luvs u' 40 | aint no point in being beaud up if they gon na treat u bad 41 | if u did it, what was it like? 42 | girls that drink turn me on 43 | if thats what u want 44 | its like getting over addiction 45 | i should try that next time i start dating 46 | but what do you want? 47 | it sounds like you both like things just as they are now. 48 | don't let yourself be dominated, she'll just have to get used to it. 49 | if he doesn't don't be upset as you can always catch another fish. 50 | no guy can give you what you want. 51 | if you wan na keep being married, find yourself. 52 | start from scratch, then go from there. 53 | ask em if they wan na go somewhere with you. 54 | don't keep bringing it up, trust him 55 | i still haven't tried that but i'm working on it 56 | find the one that luvs you for you 57 | come around my town and i'll be ur chick 58 | don't touch this dude with a 10 foot pole 59 | u don't hate one, that's fine, i dun like none of'em 60 | i'd guess maybe around 3 or 4 or so 61 | he's been telling that smack to other girls to, he's a man whore 62 | really deep down i knew my previous girls weren't the one. then i met my wife. 63 | no matter what, don't try to have them at the same time 64 | sex isn't the only important thing. 65 | i got yahoo mail it works. 66 | fun for me includes eating you feel me lol 67 | i mean he be busy after all that. 68 | that would be just fucking! 69 | don't walk up to that girl an tell her she sexy crazy. 70 | it really sounds like you have a lot on your plate at this time. 71 | i'm a guy, sou may want to ask another girl about those experiences. 72 | i felt one way for a long time, but then my eyes really opened when i met the love of my life. 73 | your best friend is always a backup plan if you can't find someone else to marry. 74 | i dont just want to pop out and be like will you go out with me? 75 | lest you know him really well, you ’ re setting yourself up for hurt! 76 | if it ’ s your personal belief is not good or bad 77 | those guys make you crazy bothering alllll day. 78 | lol go to hawaii or pay a guy to do it lmao 79 | i can throw em away n forget about it like yesterday ’ s garbage!!! 80 | is it like that clear liquid thats yellow when it dries? 81 | is weird when she drunk and sits on yo mans lap 82 | it always had to be his way.... 83 | determination is a self given gift. 84 | got ta wait 2years until you get hitched to a dude. 85 | greetings and hugs to all my cone girlllzzzzzz! 86 | just be yourself when you are together with her 87 | idk what you are saying 88 | the guy asked, do you mind if we cuddle? 89 | why would some guys do that? 90 | there are a few good girls out there, you have to look in the right places! 91 | you got ta move on or work it out! 92 | get yourself together, how can you love someone you don't want to talk to?!?! 93 | find a pretty, talented girl! 94 | um that doesn't mean you are slutty... 95 | this is the only chance you have to enjoy true love. 96 | you're gon na have to find it yourself. 97 | they gon na get ova u in 2 months 98 | you might not see color after some time but it dependz 99 | i think he just wants to know if you're into him 100 | if she's not into it anymore 101 | no worries, u r def not slutty 102 | there's no way of making a person feel something for you 103 | definitely end it and keep your options open 104 | had a pap smear before 105 | yes im proud to have long black hair 106 | i have a buddy in the same spot 107 | maybe age is an issue 108 | i'm not going to give you my nname but i've worked with this person for years 109 | you'll tell him if you truely love him... 110 | if your really attracted to her just follow your emotions! 111 | think hard about if you can afford it and if you can go ahead! 112 | um i say get your old life back so you can survive! 113 | dude, what are you saying? 114 | don't be flirty... its not a turn-on 115 | some women would probably say it's painful 116 | it'd be so much easier if i was pissed 117 | just be done if they are 118 | this is your punishment: 1mo no tv.... 119 | just say it, its not crazy because you ’ ve been to get her for so long! 120 | doesn ’ t seem simple, tell me one more time slowly. 121 | give em a chance to notice you!!! 122 | unless its w/ wrong guy 123 | my bf is obsessed w/ me and doesn't wan na lose me!!! 124 | eh depends on whether they got brown hair 125 | its just got ta play out after awhole 126 | everyone wants to hear that they are sexy even if they won't admit it!! 127 | i dance in downtown houston at clubs 128 | i say good luck... but do ur homework! 129 | i like natural more than a face with soooo much makeup on it 130 | if you like him then you should def go for it! 131 | we are divorced 132 | who said women have no emotions 133 | actions are louder than words 134 | im thinking about a mdeium pink hue 135 | see if he calls emails or even gives a crap 136 | dont ignore her, def listen and dont change the subject 137 | hes my only $ $ $ til i graduate college 138 | she; ll tell u if its not ok 139 | im good either way i think 140 | don't eat where you poop. 141 | i am trying to get blonde highlights too. 142 | just focus on school and your boyfriend. 143 | ohh no shouldn't have done it 144 | warm up to it, spend 5 minutes kissing and fondling you know that kind of thing 145 | what's a unique gift to give my brother in law 146 | be yourself man don't show off 147 | if they dont then just live ur life 148 | your first kiss should be memorable 149 | this is the most beautiful thing in family 150 | be genuine 151 | i kno 3 guys i can do that wit 152 | u can beat his ass if he keeps flirting w her 153 | stick with the 13 or 14 year old boys 154 | tell him he's being a dick and you just wan na be friends 155 | cool, its like getting your dads approval 156 | you have 2 wait for him to ask you out or youll look despereate 157 | if he wont let up he just wants to bone you 158 | i wan na have fun with you, that's why i answered this. 159 | is that how you feel? 160 | give him some cash and tell him to go. 161 | he might call. 162 | imagine what it'd be liek without them 163 | i choose the domestic godess 164 | are you deep in love with her? 165 | men are always the first to get bored 166 | get a good deal on flowers 167 | either fix it or leave, make sure your happy 168 | not everyone is decent boyfriend material just because you think they could be 169 | do a citizens arrest on him 170 | its not about the size of the pencil, its how your sign your name. 171 | yes, this is great for all women 172 | guys don't like secrets 173 | well yeah, but it's up to the guy 174 | people can't get why i'm hurrying 175 | neither of em respect u 176 | i totally can't talk to him even seeing him every week in psr 177 | r u kidding, a dude who does that all da time is not for me!!! 178 | give me a guy with nice smooth shaved legs anytime!; o 179 | so you really on the down low or just going for a test drive 180 | oh yeah, id say they do that for sure 181 | now thats a reall f/up that ya can't take back!!!! 182 | i had my cherry popped in a park when i was 16. 183 | she says they never met but they love each other really. 184 | ask that guy if he'll dance with you! 185 | u should have faith and live a full life. 186 | did you fall down? i don't know but i want to do it with you. 187 | he knows you like him, just ask him out 188 | dont be scared and tell him 189 | im not gon na drive 5 hours to spend time with someone i dont like 190 | do they gossip on the phone 191 | play sports with ur guy friends 192 | u feel happy and giddy when u fall in love 193 | i cant live with my husband 194 | we always push away those we like 195 | if u both still think that way when you're older, do it 196 | you're a man in the slightest sense 197 | a and b cups are the best 198 | if you're next to an ugly dude you'll look hot as a woman 199 | yeah it's happened to me too and now i'm dating them 200 | tell her wut you feel 201 | you got naked baby pics 202 | why do they want to marry 203 | we did it on the hood in the rain 204 | men don't need to grow up if women let em act like that. 205 | do it a few times and you'll be cool. 206 | while sex is like so fantastic! 207 | i think hes pissed right now so he aint gon na be wit you. 208 | fill me in if someone has the answer. 209 | red is love, black your a goner and so on. 210 | beefy peopel is what i like, feel ripped off with a skinny minni. 211 | if your cluesless ask someone who would know. 212 | when he tries to make up it will be fun 213 | go with the one who likes you 214 | ive been hard for her since freshman year 215 | she was checkin you out, right? 216 | i had no idea that life could be like this, so fantastic! 217 | so, what's the best option for you and for him? 218 | if nothing else, at least her phone number was the real deal. 219 | it's all good, you aren't a slut! 220 | they should get married if they can't control themselves. 221 | she is on the phone with him all the time! 222 | i don't see why, if you're already handing it. 223 | i've seen you love a guy many times, but you just couldn't be sure if he felt the same. 224 | is his mother a good enough reason to break up with him? 225 | just be a nice dude and go for it, and if she doesn't really respond, then you know she's not feeling it. 226 | my friends got hitched 6 months after their first date! 227 | show her what you can do. 228 | you look dumb. 229 | this culture gets it right! 230 | if ya'll don't ask you'll always wonder. 231 | im saying dont worry so much... my 2 cents 232 | whats the worst that could happen???? 233 | i hate 4 her 2 be that way. confused!!!! do know what to do.... 234 | look for a person who doesnt hate you 235 | don ’ t be salty..... 236 | some guys don't wan na dance. 237 | this generation is retarded 238 | well, could you trust him before? 239 | ugly fat guys cant be picky 240 | some poeple want to be a wife or husband 241 | its not worth it if you only argue 242 | enjoy asking that 2 ur cellmate in jail! 243 | do i like u? 244 | what'd you kiss 245 | even i am curious about sex with another woman. 246 | she ain't got ta be younger 20-32 is good. 247 | that happens, your decision will be easier. 248 | he cheated, which i think means i can lift or carry my husband. 249 | make the best out of a bad set-up 250 | u looking for dating or friends 251 | id be good for valentines day 252 | its very tempting: / 253 | how long does it take for your relationship to get boring? 254 | it just depends how fast you can get ready. 255 | yeah i'd like to meet the girls. 256 | it is what it is. 257 | having online friends is cool as long as you set limits 258 | do you want him to be your baby daddy? 259 | he has low value if he turns you down girl 260 | youll learn from experience, i promise 261 | hope u married hubby coz of luv 262 | i bet ima have someone waiting when i get home 263 | u better go get that massage.lolz 264 | ion know where 2 start 265 | all depends how ur body looks, you like what u see when u look in the mirror? 266 | are you telling me that u referring to you avatar? 267 | he'll call you if he cares about you. 268 | there's a huge different between banging and making love. 269 | just cuz you have a kid, doesn't mean you got to stay together man. 270 | are you from the city or country? 271 | you can always get married and have kids later, but you can't do school later. 272 | ive never gone to it 273 | do you wan na have a relationship with the same guy? 274 | how often can you have sex with pam andersen?????? 275 | make sure she doesn't end up hating your when you make your point to her 276 | my mom has cancer too. too close to home! 277 | running my fingers through curly hair, < 3 < 3 < 3 < 3 278 | i mean u 2 both. 279 | wait and the right gal will come, u will c... 280 | i want my spouse to grow 2... 281 | u need to make a move when there shy like that 282 | the dr was just as surprised 283 | its a yes, even if you don't agree 284 | well boys do that but men dont 285 | just try and say do you wan na go out at some point 286 | i had this happen to me two separate times! 287 | dont date a guy just bc you think you have to 288 | listen if u really wanted him that bad you would not be asking me!! 289 | i am hoping that it does not ever happen! 290 | ladies just want guys 2 tell them how they really feel.., 291 | u know how excited girls r when udoin it to them the right way 292 | show her how much u want her!! 293 | leaning in and giving me a soft kiss right on my forehead 294 | yes im not 1 of those people--i know there's lots of them 295 | ok it depends on how much you're starting to have feelings for this boy.... 296 | thank for the pep in advance 297 | why do they want to be married? 298 | i like gentle flirting with some romance 299 | lets to an amusement park or museum. 300 | i like the guy, but how do i know if i'm in love? 301 | which do you prefer, manicured nails or fake 1s? 302 | don't worry about what other ppl want, plan what you want 303 | i don't get u 304 | i like your perfume 305 | most guys dont care, but it depends on teh individual 306 | it doesnt have to do with u being naive 307 | dont get married so soon 308 | you are bad if u cheat on ur wife 309 | just do you. if he likes what he sees he will speak to you. 310 | that person is the only one you want to be with. 311 | he's about to get hitched....don't even go there! 312 | did you let her know how annoying checking in with you every single day is? 313 | new neighbor asked me to chill 314 | teenage boys and girls like masterbating 315 | he is good at bbal 316 | dont sweat 317 | are you someone who likes to fight? 318 | the most important issue is not trying to retaliate against the one who did you wrong. 319 | one man who will go hours and miles to be with me. 320 | omg, have you been to a doc yet?! 321 | maybe she wasn't a biatch but the skinny girl was def!! 322 | get a life and stop wasting my time!!! 323 | its like the same thing as binging on chocolete then cryin about being fat 324 | i went there and he was there 325 | oh yeah if he's not taken then grab him 326 | oh yeah if you dont do what they say you're gon na get a beating 327 | well do you like him or not? 328 | tell your mom and dad if you havnt already 329 | most will tell you no 330 | u think he feels the same way about u? 331 | some guys only say love if they mean it... 332 | he def likes u, show him how u feel!! 333 | i think everyone should keep their own name. 334 | guys enter and exit, but friends are forever 335 | believe that it was considered love 336 | you should be ready even if your relationship survives the four seasons. 337 | some guys like chicks of that type and others, not so much 338 | be a sham if u want kids 339 | young dudes or old women, watch think? 340 | he wants to keep the shit up 341 | there is no alternative to just being ``love.'' 342 | your self-image is super important 343 | i am a girl and i say that stuf... 344 | girls can be just as perverted... trust me!!!! 345 | flowers are nice but dont last long 346 | i havent seen some movies 347 | she is sweet. u should have her 348 | use your brain not ur dick. maybe she doesnt like u 349 | pleez dont take this the wrong way but that sounds like a girls name. 350 | never call him back is not worth the problems. 351 | not interisted in any ladies on my app list so i picked my neighbors sweet lady. 352 | marraige is all trust and talking things out between each other 353 | whatever they say to do, do not listen! 354 | what can i do to snag her? 355 | give her a ros44e and tell her you llove her. 356 | it's not the norm, but it's not a bad thing either. 357 | try working things out so you both end up happy 358 | those girls could be enough for a man but their not up to our level 359 | you got it, especially when she got sick. 360 | i need more info b4 i can help u more. 361 | there's no wrong or right way 2 kiss. 362 | you should take ur time in deciding... 363 | having some fun w/ ur guy friends is fine 364 | i'd say its more of a trust issue... 365 | i need help with this i'm screwed 366 | if he dont like you then move right along 367 | he doesnt want to know 368 | if you can then make it yrself 369 | is there anything else you do besides work and fix things? 370 | i told her straight-up that i loved her 371 | the senior guys think the freshman chicks are all easy, sad 372 | how do i give him a kiss 373 | look in the deleted items, that's where it should be 374 | good luck finding that nice girl 375 | that's a horrible idea, think of why you are there 376 | that list could go on forever 377 | be happy with anyone even if it's a man, just like brokeback mountain 378 | don't base your whole relationship on sex 379 | it's super bad luck if you sneak a peek at the bride on the wedding day. 380 | stay strong out there, and i wish you the best! 381 | it's just some lil get together for the fam. 382 | if you're not really feeling the performance aspect, then skip it. 383 | i just tied the knot 384 | depends on how you were raised, how old you are, etc 385 | no rush, you just got ta pick a last name when you have kids 386 | don't you think your kid's gon na have a rough time being raised without their daddy? 387 | bald men look like babies to me, it's just not cute 388 | what was the score of the last game 389 | get rid of ur ex cause u have a better man 390 | fly or drive to vegas to get married 391 | yes! i want to know u way better. 392 | c or d is just what i like. doesn't need 2 be bigger than taht. 393 | never admit the truth just change the lies! 394 | they never played live before so it is twice as good. 395 | momma won't let me wear those clothes. 396 | we haven't seen you! 397 | cause opposites attract and you'll always find some one 398 | i def think that this is better than my life! 399 | dont go to the same places that you went with your ex. 400 | talk to your bestie and lem me know if it's cool 401 | it sounds weird, but it def worked for me in 3-5 days 402 | i dont like when he messes with you 403 | no im not married 404 | some men do like big girls, most tend to be african american. 405 | overlook his faults and stay with him if you care. 406 | do they make fun of you when with him? 407 | if your against it that much- then do'nt get married. 408 | it depends on if their vain or not. 409 | get over it and get someone who actually likes you. 410 | get to know her so you can ask her out soon. 411 | i think he was blowing me off, so i asked. 412 | i think some young guys like asian girls. 413 | make your own decision. 414 | just let him know you want something low key. 415 | this is just plain bad, stuffed with lots of smaller bad things. 416 | they've been pretty good years, but not the easiest 417 | weve seen each other everyday for four or five years!! 418 | pretend i'm the guy you want and date me! 419 | hey a little flirting never hurt anyone! 420 | you should go with your gut! 421 | good luck. 422 | why haven't i kissed a girl after eight months with her? 423 | it's her choice. 424 | they look better than lots of other nations. 425 | the day after is alright by me. 426 | see if she can go for movie and dinner 427 | men get to the point. women dont 428 | ask in english i dont get it 429 | sorry, but i dont know... 430 | remember, a bird has never flown with one wing 431 | i often worry about this 432 | youll never find a girlfriend if you are so needy 433 | i met a girl recently, i want to know if i have a chance. 434 | dont sit and wait for a call 435 | be what and whot you want to be 436 | if he gets it thru his head cool. if not move on 437 | if he really loves u he'd disapoint his dad. 438 | better do it before its too late 439 | this girl does not want a shitty man. 440 | i like hearing black woman talk baout this 441 | hel pme argue with her 442 | most of the men are similar to ur age 443 | ugh, when will ppl figure this out?! 444 | i dunno what i should do now 445 | you go to a bar and get wasted! 446 | i acted like that in front of my boyfriend. 447 | if you got ta ask, it is a no. 448 | don't just sit there and wait till he sees you for what you are. 449 | im getting a divorce and i'm stuck with this!! 450 | um your bf will think your being a jerk... 451 | maybe give girls a try, since guys can be so off-putting. 452 | i was with my ex for about 3 years before they were unfaithful. 453 | just see what the next 6 months has in store, and you'll find out. 454 | there is one in particular i was fond of, and it goes a little something like this. 455 | not all guy break up with their girls 456 | theres a lot of fun stuff there 457 | is it alright to kiss on the first date 458 | i have a girl friend 459 | they wan na fuck each other 460 | my friend thinks im too nice and should say no to them 461 | but ( there's always a but ) 462 | some guys do, not all of them, though 463 | if you do that you can keep your last name 464 | go up say hey and ask if she's dtf 465 | yo that was mad hot but kinda scared me off 466 | fat girls can be hot 467 | not everyone here is taking classes 468 | practice taking the wrapper off of starbursts & tyeing cherry stems....with your tongue 469 | if u vote for me for best answer, i'll pay! 470 | go to a public place. you'll be more comfortable. 471 | work with him and pay attention to how he reacts. 472 | and inviting someone takes the pressure off me. 473 | please say you'll date me. 474 | she remembers my messes. 475 | how can i let her know that i'm not just one more guy who'll hit on her and run? 476 | i'm a guy so let me enlighten you. 477 | it's gon na be ok, you're 13. 478 | don't go trail riding by your damn self. 479 | if you two love each other, it'll be good!!!! 480 | got ta make eye contact. 481 | yep i can lift him. 482 | he's got ta start again and gain ur trust. 483 | u wan na ask who out? 484 | i'm hoping you read this really carefully 485 | play truth or dare together! 486 | wait till you find out and confirm, then go from there 487 | um that is stupid bc you need a theory class for that!!! 488 | just stop if you eat antying like that 489 | can't stop thinking about sex with her 490 | the only way to find out is to ask him face to face 491 | woot jamie is asknig rubi out on a date!! 492 | its carrying on because neither of you have the balls to end it 493 | they seem to like learning about that 494 | oh snap - he straight up just scored 2 points 495 | do guys like fatter girls 496 | he loves you too, but you got ta wait to know 497 | if a guy respects u, he'll understand u. 498 | we started this.....our own sins. 499 | the dudes answer works for me. 500 | youll be able to love him still, but get over it` -------------------------------------------------------------------------------- /datasets/gyafc/gt_formal_output1.txt: -------------------------------------------------------------------------------- 1 | just be her bff 4 real . 2 | do you really want an answer ? 3 | i was n't trying to be a jerk . 4 | some ladies just hint at what they want and expect the men to figure it out . 5 | how about we grab some starbucks and talk , and see what happens ? 6 | i hate seeing my daddy sad : ( 7 | ask him if u should go see a dr . 8 | dontcha think he has enough to worry about ? 9 | i dont think he 'll step out or dip if you dont slap skins with him 10 | will i feel dat fo eva 11 | but he mite like all that teasing and skeezin with no pleasin 12 | you needs 2 try agin 13 | some men will shave for the right chick 14 | if u like him smile and speak nice . 15 | he likes ya but i do n't know about love . 16 | bitches be crazy 17 | im too picky , about people , how do i stop ? 18 | that sounds pretty girly 19 | if the good is better than the bad , its worth it 20 | only if both of you agree 21 | well why u got ta like the one boy who messes wit u ? 22 | and , brothas do n't b knowin how much dey need u until u gon . 23 | if a dude hits on u do n't be scared to say no 24 | do i want luv 25 | if you give them creepy looks they 'll think ur leering 26 | you 're right it 's realy confusing 27 | how i convince my lover to show his love for me ? ? 28 | i mean , if you 're that into them , sure 29 | dun sell yourself out , girl , his money do n't matter that much 30 | do n't let these girls hurt your feelings , you sound like an upstanding dude 31 | i dunno , but there 's another site that i go to for that 32 | nah , man , think hard about how many arrogant chicks there are around 33 | you could look easy if you do n't have a guy . 34 | then he 's like do you want to use my chapstick ? 35 | do n't ever let a guy treat you bad . 36 | do n't think so , he 's too big and i 'm a weakling . 37 | tell him when and if he says yes ! 38 | what mistake did u make ? 39 | tehres a song by king called only your mother loves you 40 | why do we stay when theyre mean to us after we 've falllen in love ? 41 | if youve done it , how was it ? 42 | its not really a turnoff when she gets super drunk 43 | i guess if thats what you want 44 | its just like trying to ditch a bad drug habit 45 | i need to try that dating trick next time for sure 46 | just let me know what you want 47 | yall are both good with how things are going so thats good . 48 | dont let her take over ur life , she will just have to deal ! ! 49 | if hes not into you , there are plenty of other men out there who are . 50 | no guy can give you everything you want ... 51 | you need to `` find urself '' if you wan na stay married . 52 | start with a boring question and just keep it going ! 53 | ask if they want to travel somewhere with you . 54 | just put some trust in him , and let it go . 55 | i 'm building up the courage to go for it . 56 | find the person who loves you as much as you love them . 57 | i 'll be your girl if you come to my town . 58 | stay far far away from him . 59 | it 's your choice man , but i hate all of them . 60 | i say around 3 or 4 61 | he told that woman he 's a player . 62 | my ex 's made me realize why my wife was `` the one '' . 63 | do n't be greedy and take both of them at the same time . 64 | having sex is not the only thing that matters in life . 65 | i use yahoo for my email and it works fine for me . 66 | its just fun for me 67 | he was a little busy after 68 | thats just fucking sex 69 | dont let her know you think shes hot 70 | sounds like you 're goin thru a lot right now 71 | but u should ask a girl about the symptoms bc im a boy 72 | i use to always feel the same thing but everything changed when i met my true luv ! ! 73 | marry your bf if it does n't work with anyone else . good choice 74 | well i don ’ t want to just run up and ask will you go oit with me ? ? 75 | unless you reaaalllyyy know the guy your gon na get hurt . 76 | your beliefs aren ’ t good or bad . 77 | they 're annoying all day . 78 | go to hawaii or pay someone . 79 | i can just forget about him like it 's nothing . 80 | is it like a clear liquid that gets yellow when it ends up drying ? 81 | do you hate it if she sits on you bff 's lap when drunk ? 82 | he was a control freak 83 | one of the gifts we can give ourselves - determination . 84 | you should only marry a man after you been with him for 2 yrs . 85 | to my girls at cone , hi and hugzzz ! ! ! 86 | be yourself at all times 87 | say it slow and repeat 88 | a perv hit on me 89 | some dudes are dumb 90 | there are plenty of fish in the sea 91 | either make up or move on 92 | dude you 've never talked to her , how can you love her 93 | finda cute girl who can sing 94 | you arent a slut 95 | true love will happen to you , this time . 96 | look up the answer on your own , dude . 97 | 2 months , and they will have moved on from you . 98 | you wo n't even be able to tell what color it is once time has passed . 99 | he wants to know if u like him 100 | if she doesnt like u ... .. 101 | your not a slut . dont fret 102 | u cant force another to like u 103 | break up and focus on others 104 | have you had a pap smear ? 105 | i can say `` yes '' and i have long black hair and i 'm proud ! 106 | my friend is in the same position ... 107 | maybe your young ... idk . 108 | i ’ m anonymous but i ’ ve been working with this person for a while . 109 | if you love him for real , you ’ ll tell him 110 | if you really like her , go with your feelings 111 | if you think u can afford it , go for it if you want 112 | stay alive and get things back to normal 113 | what are you saying about her 114 | not flirt . no one likes it 115 | i think some women would tell you that it 's pretty hurtful 116 | it 's be easier if i were more pissed off ! ! ! ! 1 117 | if they 've forgotten about you , then forget about them too 118 | your punishment - no tv for a month . 119 | you 've known each other for a long time it wo n't seem crazy 120 | i still do n't get it . can u say it again ? 121 | u have to give them a chance to see all your talents 122 | wrong man could be a problem 123 | my bf doesnt want to lose me cause he loves me a lot 124 | it kinda matters if they 're brunette or not 125 | it 'll get played out at some point 126 | we do n't admit it , but we all want to know we are sexy , sexy beasts . 127 | i love dancing at the clubs in downtown huston 128 | good luck and do n't forget to do your homework ! 129 | i do n't like it when women use too much makeup , they are beautiful naturally . 130 | if you want him , go get him . 131 | but now me and her are both divorced 132 | um who said women do n't have feelings ? ? ? 133 | actions are always better then jus words ! 134 | i 'm thinking a regular pink color ... but what should go with it ? 135 | see if he calls you or emails you or if he even cares ! 136 | do n't ignore her always listen to her and do n't change the subject ! ! 137 | he is my only way to get money till i graduate from college 138 | she 'll let you know if it is n't okay 139 | i do n't care either way ! 140 | do n't eat where you shit . 141 | i 'm gon na get blonde highlights done . 142 | forget about it , focus on school and your bf . 143 | this was a bad idea.. 144 | i need at least 5 minutes of groping . 145 | what 's a dope prez for my bro in laws bday ? 146 | do n't be a show off , but be yourself ! 147 | if dey do not , den yuh will have tuh continue wit ' your life 148 | your first kiss , howevuh , should be heavenly 149 | in a family life , dis is de most beautiful 150 | most impawhtantly , be genuine 151 | i got 3 guys who i can do that with . 152 | get mad if he flirts with her 153 | stay with the 14 or 13 yo boys 154 | tell him u just wan na be friends , and your disappointed by him being so dumb 155 | it 's kinda nice ... like getting your dad 's blessing . 156 | wait for him or you 'll seem desperate 157 | if he keeps it up , you can tell he was only after sex 158 | i love have fun wit you so i answered this ? 159 | how u feel 160 | give your bro som $ $ and get rid of him 161 | he might call 162 | you can try to think about life without them , that 's something . 163 | i 'll go with domestic goddess 164 | do u rly love her ? a lot ? 165 | men cheat more than women 166 | save money on flowers 167 | fix the problem or leave if you are so unhappy 168 | looks are decieving , he may be a shitty boyfriend 169 | you can make a citizen 's arrest . 170 | having a huge package does n't mean you 're good at doing it ! 171 | yes , it looks so hot on almost any woman ! 172 | dudes ca n't keep secrets worth af 173 | depends on the dude , i guess 174 | people were asking me why i was in such a hurry 175 | neither of them respect ur feelings girl 176 | i see him in psr every week but i ca n't gather up the gumption to talk to the guy 177 | i dont find guys i like 178 | i like men that shave legs 179 | is it gay ? ir curiosity 180 | i say they do 181 | its a mistake u cant fix 182 | i lost my v card when i was 16 in a park ! lol 183 | they 've never met in real life ... but she says they 're in love ! ! ! 184 | ask if he wants to dance w/ you ! 185 | live your life to the fullest and have faith ! ! ! 186 | idk if you fell over , but i 'd like to have sex w/ you ! 187 | he knows you 're into him so just ask him out 188 | get over your fear and tell him 189 | why the f would i drive 5 hours to spend time with somoene unless i really liked her ? 190 | aint she got friends to talk on the phone with her ? 191 | it 's good to play sports with other guys . 192 | when you fall in love everything feels wonderful and you ca n't stop thinking about your bae . 193 | i ca n't imagine a life without my hubby . 194 | it 's easy to hurt those we love most . 195 | go for it if you both feel that way when you 're grown . 196 | when i call you a man take that with a grain of salt . 197 | i like small cups of a and b . 198 | the hot girls are by the hideous men . 199 | i am with them and it happened to me . 200 | tell the truth about your feelings for her . 201 | you should show her your naked baby pictures . 202 | why is marriage important to the gay community ? 203 | we had sex on the hood of my car while it drizzled around us . 204 | why do guys got ta grow up if girls do n't expect them to ? 205 | you got ta do this a few times . 206 | making love is like a fantasy . 207 | rite now i think hes mad , and doesnt wan na be with u . 208 | lem me know if youve got the answer plz ! 209 | red is a love color , nblack is a death color , etc . 210 | i like fat ones , otherwise i feel like im missing out ! 211 | ask someone if you dont kno the answer . 212 | he 'll try to make it up , you 'll like it . 213 | if he 's picking on you he likes ya ! 214 | i 've been liking this girl since 9th grade . 215 | she looking at u ? 216 | didnt know life could be such a blessing ! ! 217 | go with the best thing for you two . 218 | look at like this , you got a real number not fake : ) 219 | i do n't think you fool around . 220 | they should get married if they ca n't keep their hands off each other . 221 | seems like she 's calling him all the time . 222 | might as well keep on keeping on 223 | how many loveless relationships have you been in ? 224 | is it right to break up with him cause of his mom 225 | make a move and if she doesnt , give her space 226 | they got married fast ! 6months fast ! 227 | show her your skills man . 228 | you look like a freak . 229 | i give props to that culture , because they really have it down pat . 230 | you 'll never know for sure unless you ask . 231 | do n't worry about it or over think it . 232 | what you think the worst could be ? 233 | i do n't even know what to do cus i ca n't stand when the girl is like that , so like what do i do now ? 234 | find somebody else somebody down that do n't hate you all the time . 235 | dont be a dick 236 | some guys just ca n't dance 237 | these youngins r dumb . 238 | depends did ya trust him b4 239 | fat and ugly guys ca n't b picky . 240 | people choose marriage 4 a reason . 241 | it ’ s not worth arguing with you all the time 242 | good luck asking your cell mate that question 243 | do i like you ? 244 | you kissed what ? ? 245 | some women are bi curious like me . 246 | 20 to 32 , just not younger than you . 247 | after this you can make a decision 248 | he cheated , so i can lift my husband 249 | look on the bright side 250 | is your goal to just hang out 251 | in my opinion he would make a good valentine 252 | with so many temptations i am not sure i can do it . 253 | it takes what ? a yr to feel comfy in relationshipz ? 254 | i u ready , then cool 255 | hello yea i 'm tryna meet chicks . 256 | it is what it is . 257 | you need to have safety precautions when you friend total strangers on the net . 258 | can you picture having kids with that guy ? 259 | if he is n't interested in you , it 's his loss , and why date a fool anyway ? 260 | you usually learn the best lessons when you go for it , and make mistakes . 261 | you did marry him for love i hope . 262 | at least i have someone who will be there for me right at home . 263 | relax and get a massage and just let it all go . 264 | where should i even start here . 265 | depends , are you hot with a gr8 body ? ? ? 266 | wait .. you mean yr avatar ? 267 | he 'll call if he cares 268 | there is a idfference between fucking and making love 269 | u dont got ta stay together even if u got a kid 270 | are u a country girl ? ? ior are u from the city 271 | getting married and having kids can wait . studying cant 272 | i 've never been . 273 | you wan na be in a relationship with the same guy ? 274 | can you have sex with pamela anderson often ? 275 | do n't make her hate you just to get your point across 276 | this is too close to home , my moms has cancer too . 277 | i luv to put my hands thru curlee hair 278 | i mean u 2 together 279 | just be patient and the rite girl will find u 280 | i want my wife to grow 2 281 | he seems shy , so you outta make the first move . 282 | the doc was shocked too ! 283 | you may think it 's no , but it 's def yes ! 284 | that 's for boys to do - not men . 285 | if u wanted u could ask them out and see . 286 | and that did happen 2x to me . 287 | do n't make up some need in yur head to go out with that dude . 288 | if he was the right guy , you would n't need to ask . 289 | hope this streak never ends~ 290 | girls just want guys to be straight with their feelings . 291 | nothin ' like sex to bring out the passion in a woman 292 | chicks like to be impressed so keep on impressing her 293 | getting close and kissing me softly on me forehead . 294 | that ain ’ t me but i know peeps like that . 295 | depends on how hard you ’ re falling for this bro ! 296 | i just want to thank you ahead of time for the encouragement . 297 | i just do n't know why they want to get married so bad . 298 | we like to be pursued , but with a light touch . 299 | i think theme parks and museums are both fun places to visit . 300 | i need to kno if im in love with this person or not . 301 | do you like manicures or fake nails ? ? 302 | plan what u want , and dont worry about what everyone else wants . 303 | i dont know what ur saring . 304 | ur perfume turns me on 305 | i do n't think most men care but it depends on the guy . 306 | its got nothing to do w/ being vulnerable or young or submissive ... 307 | do n't get married so early in your relationship ... 308 | your not a good person if you plan on cheating on your wife ! ! 309 | be the best you that you can be and see if he likes it . 310 | hes the only one for you . 311 | do n't be a homewrecker . he 's taken . 312 | tell her that it 's bs she makes you daily check in ! 313 | neighbor asked me out when i moved in 314 | most teen guys prefer teen gals who like hooking up ... 315 | he played a good game of b-ball . 316 | do n't sweat it ! 317 | do u love fighting or what ? 318 | important thing is to not try to get revenge on people who wronged u 319 | a guy willing to move 21hrs and 1400mi for me 320 | have you gone to the doctor yet ? 321 | she 's not annoying but she was totally thin . 322 | get a god dang life and leave me along . 323 | that 's like eating too much chocolate and saying you gained too much . 324 | i was there and he was there . 325 | no ball and chain , then do it ! 326 | these sorts of guys turn into wife beaters if you don ’ t give them what they want ... . 327 | so do you think he ’ s cute or not ? 328 | well if you ’ re certain then tell the parentals that you haven ’ t yet . 329 | a lot of people are going to tell you no.. 330 | does he feel the same about you ? 331 | some guys do n't use that word unless they mean it . 332 | tell him how you feel if you think he likes you . 333 | i think everyone should keep their own name 334 | boys come and go but friends are forever ! ! ! 335 | try and imagine that it was love < 3 336 | i think you 'll be ready even if your relationship survives the 4 seasons ... 337 | okay , well right now some guys like that kind of women and other do n't 338 | if you want kids , be a house mom . 339 | young boys and older women , yes or no ? 340 | he wanted to keep going . 341 | u cant use another word for love 342 | self image is important 343 | im a girl and i say such things 344 | girls can be perverted too 345 | yeah i like flowers but they die too quick : p 346 | theres some films i havent seen 347 | she 's super sweet , exactly what yr after ! 348 | use your brain and not whats in yr pants , she might not love you 349 | do n't take this the wrong way , but i 've never heard of a guy with a name that girly . 350 | i 'm not gon na waste my time returning his call . 351 | i was n't really feeling any of the ladies in my list , so i settled for my neighbor 's wifey . 352 | openness and honesty are the core of any marriage . 353 | just ignore what anyone else suggests . 354 | how do i impress her ? 355 | a rose says i love you 356 | dont worry , it 's weird , not bad 357 | figure out what works for both of you . 358 | she ’ s enough of a woman but she ain ’ t classy like us . 359 | yep , especially after she got really sick ... . 360 | can ’ t help you anymore unless you give me mor info . 361 | there ’ s no way to be good or bad at kissing . 362 | just think it out 363 | having fun with dudes is okay 364 | that 's probably more of a trust issue 365 | i need help understanding this . 366 | move on , there are plenty of fish in the sea . 367 | he probably was n't into it either . 368 | when you can , make it yourself . 369 | u do anything else than work & fix stuff ? 370 | i let her know that i really luv her . 371 | the uppers think the frosh gals are easy game . thats messed up ! 372 | should i really just kiss'em ? 373 | it should be in the deleted stuff if you take a peek . 374 | i hope you find a cute girl 375 | thats a bad idea , its not for that purpose 376 | its a long ass list 377 | brokeback mountain , youd like it even if youre a guy 378 | if its not dependent on sex , thats fine 379 | yah , but it 's bad luck to see the bride before the ceremony , y'know ? 380 | hold on and godspeed ! 381 | uh , just a small family thing at my house 382 | if you 're concerned about the performance , do n't go through with it 383 | i 'm married now 384 | it depends on age and upbringing 385 | you need to choose your kid 's last name 386 | kid 's life is hard without a dad 387 | baldies look like kids , it 's ugly 388 | the last games score was ... ? 389 | choose the guy you 're with , or your ex . 390 | best and easiest is fly or drive to vegas and get hitched . 391 | yes i just wan na get to know u 392 | size c or d at the most but the rest new much 393 | lie to them and then just keep lying ! 394 | it ’ s double the pleasure since they haven ’ t performed before 395 | my wife doesn ’ t like wearing those brands 396 | we have no idea what u look like 397 | opposites attract , we can always find a life partner if we look hard enough 398 | lol , this is def better than my everyday life 399 | eh , you went with your ex there , do n't go there with your new date , too 400 | talk to your bff and find out if it 's all good 401 | weird , but it worked for me in about 3 to 5 days or so 402 | it annoys me when you get teased by him about everything ! ! 403 | no , i 'm still unmarried . 404 | a lot of dudes like big chicks , mostly black guys 405 | shouldnt matter . you like him then say 406 | do they throw shade when youre together ? 407 | dont get married if u feel leike that 408 | it depends if they are in vain 409 | move on and date someone who likes u 410 | get to know her and then ask the other out 411 | i asked if he blowed me off 412 | nasty old guys like young asians 413 | do what you want ! 414 | hmm , tell him you just wan na keep this casual ... 415 | thats a hard situation and there are n't any easy answers ! 416 | they have been hard but good years . 417 | we seen each other eberyday for like 4 or 5 years ! 418 | if i am the guy you want then be wit me ! 419 | live a little and flirt ! 420 | stay with what u feel inside . 421 | good luck 422 | is it bad if i havent kissed her since we started dating 8 months 423 | its her choice 424 | they look better than other countries 425 | itll be okay the next day 426 | ask her out for dinner and a movie . 427 | some guys do n't think about it they just do it . 428 | speak english please . i ca n't understand you . 429 | sorry i do n't feel that way . 430 | just remember that a bird ca n't fly with only one wing 431 | yeah i think about this a lot 432 | if you 're just whatever about it you 'll never find anyone to date 433 | there 's this girl that i met and i 'm wondering if she likes me 434 | dont just wait around for the call 435 | quit your bitching , nobody cares if you 're gay 436 | break up with him if he cant get it together 437 | it 's possible he loves you and will end up letting his dad down . 438 | fix it before you regret it ... . time is a-wasting . 439 | this chick thinks bad boys are way over-rated . 440 | i wan na hear what a black chick would say about this . 441 | help me beat her in this debate . 442 | almost all of those guys are near your age . 443 | when will people realize and notice shit . 444 | i 'm not sure what i should do . 445 | you go and get wasted at the bar . 446 | i acted like that with my new lover 447 | if you have to ask , forget about it . 448 | do n't wait for him to see you 're the one that 's good . 449 | ugh , still stuck w/ this even though getting divorced 450 | your bae will just think your being an ass . 451 | why do n't you try dating chicks if you ca n't stand dating guys . 452 | my dirty ex cheated on me after 3 full years 453 | you will know soon enough . after six month see what happens . 454 | the one i thought was good goes a little something like this . 455 | not all dudes dumb chicks 456 | its always tons of stuff to do over there 457 | is it ight to smooch on the 1st date ? 458 | i got a homegirl i 'm close with 459 | they in lust *lol 460 | my friend says im too nice and i should say no 461 | there is always an exception 462 | not all men , but some o 463 | if u do that , u keep ur last name 464 | just walk up , introduce yourself , and ask if she 's available . 465 | i was all hot and bothered , but i got out of there quick . 466 | big girls can still be total cuties . 467 | not everybody here is a student 468 | unwrap starburst and tie cherry stems with your tongue for practice 469 | vote for me in best answer and ill pay for you 470 | look in public , youll feel more comfortable 471 | just flirt with him and see what happens 472 | also it takes the pressure off me to ask somebody 473 | come on ! go out with me plz ! ! 474 | she 's not gon na forget that i messed up . 475 | how do i make it seem i want more than sex ? 476 | i 'm a guy i know about this stuff . 477 | dont worry . ur 13 478 | go fon a trail with a friend 479 | it u love each other , itll work out 480 | eye contact is improtant 481 | i can lift him no problem 482 | he needs to regain your trust and start completely over again 483 | are you sure you want to ask her out ? 484 | i hope u get the info soon 485 | play truth or dare together 486 | wait till uno for sure . then decide 487 | thats dumb . u just need a theory class 488 | quit if u eat something like that 489 | i keep thinking about fucking her 490 | u will only know if u ask him directly 491 | jamie will ask out rubi 492 | its still happening cause neither has courage to end it 493 | maybe they just dig learning about that stuff . 494 | he 's dribbling the ball , he shoots annnnnnd score , 2 points for the champion . 495 | do dudes really want big girls over other types ? 496 | i know he loves you like you like you love him , but no one knows the future . 497 | if he respects you , he will understand what u need . 498 | we have our own sins started . 499 | his answer was good enought for me . 500 | love him sure , but let him go . -------------------------------------------------------------------------------- /datasets/gyafc/gt_formal_output2.txt: -------------------------------------------------------------------------------- 1 | u have to be her friend . 2 | that sounds more like a rhetorical question than a real one . 3 | i 'm not tryin to be mean ... . 4 | women send men mixed messages sometimes 5 | up fr coffee and chat then see what happens ? 6 | hate it when my dad is bummed out . 7 | ask him if you should see a doc or sumthin ' 8 | give him a break , you 're not the center of the dang universe . 9 | i do n't think he 'll leave or cheat if u dont screw him 10 | will i always feel like that ? ? ? 11 | but he may like all the coyness and yearning 12 | and i wan na try again 13 | some men shave because there women likes them to shave . 14 | well if you really like this guy smile and talk to him with respect . 15 | i 'm not sure if he loves you but i do know for sure that he definitely likes you . 16 | women are complicated because that is how men see us . 17 | im overly selective with people . how do i stop 18 | that sounds girly 19 | the good may be better than the bad 20 | only if u both agree 21 | why do u like the boy who bullies u 22 | men dont think they need u till ur gone 23 | ehhhhh don ’ t be scared to tell a bro no ! 24 | is lov what i want ? ? 25 | they will thing u a creep if u leer at them 26 | ur right . it is complicated 27 | how do i get my love to show he lvoes md 28 | yeah if you defo love them 29 | doesnt matter if he us rich , dont sell yourself 30 | you sound like a great guy , dont let these women get you down 31 | idk but theres another site i go to 32 | you need to think about how many arrogant women there are out there 33 | well if your not dating then that seems kinda like a whore move 34 | and so he was like , do you wan na use my chapstick ? 35 | you should n't ever ever ever let a man treat you wrong ! 36 | definitely not , he 's huge and i 'm really gd weak 37 | go on and tell him if he says yes 38 | what mistake did u take part in 39 | king 's got that joint 'only ur mother luvs u ' 40 | aint no point in being beaud up if they gon na treat u bad 41 | if u did it , what was it like ? 42 | girls that drink turn me on 43 | if thats what u want 44 | its like getting over addiction 45 | i should try that next time i start dating 46 | but what do you want ? 47 | it sounds like you both like things just as they are now . 48 | do n't let yourself be dominated , she 'll just have to get used to it . 49 | if he does n't do n't be upset as you can always catch another fish . 50 | no guy can give you what you want . 51 | if you wan na keep being married , find yourself . 52 | start from scratch , then go from there . 53 | ask em if they wan na go somewhere with you . 54 | do n't keep bringing it up , trust him 55 | i still have n't tried that but i 'm working on it 56 | find the one that luvs you for you 57 | come around my town and i 'll be ur chick 58 | do n't touch this dude with a 10 foot pole 59 | u do n't hate one , that 's fine , i dun like none of 'em 60 | i 'd guess maybe around 3 or 4 or so 61 | he 's been telling that smack to other girls to , he 's a man whore 62 | really deep down i knew my previous girls were n't the one . then i met my wife . 63 | no matter what , do n't try to have them at the same time 64 | sex is n't the only important thing . 65 | i got yahoo mail it works . 66 | fun for me includes eating you feel me lol 67 | i mean he be busy after all that . 68 | that would be just fucking ! 69 | do n't walk up to that girl an tell her she sexy crazy . 70 | it really sounds like you have a lot on your plate at this time . 71 | i 'm a guy , sou may want to ask another girl about those experiences . 72 | i felt one way for a long time , but then my eyes really opened when i met the love of my life . 73 | your best friend is always a backup plan if you ca n't find someone else to marry . 74 | i dont just want to pop out and be like will you go out with me ? 75 | lest you know him really well , you ’ re setting yourself up for hurt ! 76 | if it ’ s your personal belief is not good or bad 77 | those guys make you crazy bothering alllll day . 78 | lol go to hawaii or pay a guy to do it lmao 79 | i can throw em away n forget about it like yesterday ’ s garbage ! ! ! 80 | is it like that clear liquid thats yellow when it dries ? 81 | is weird when she drunk and sits on yo mans lap 82 | it always had to be his way ... . 83 | determination is a self given gift . 84 | got ta wait 2years until you get hitched to a dude . 85 | greetings and hugs to all my cone girlllzzzzzz ! 86 | just be yourself when you are together with her 87 | idk what you are saying 88 | the guy asked , do you mind if we cuddle ? 89 | why would some guys do that ? 90 | there are a few good girls out there , you have to look in the right places ! 91 | you got ta move on or work it out ! 92 | get yourself together , how can you love someone you do n't want to talk to ? ! ? ! 93 | find a pretty , talented girl ! 94 | um that does n't mean you are slutty ... 95 | this is the only chance you have to enjoy true love . 96 | you 're gon na have to find it yourself . 97 | they gon na get ova u in 2 months 98 | you might not see color after some time but it dependz 99 | i think he just wants to know if you 're into him 100 | if she 's not into it anymore 101 | no worries , u r def not slutty 102 | there 's no way of making a person feel something for you 103 | definitely end it and keep your options open 104 | had a pap smear before 105 | yes im proud to have long black hair 106 | i have a buddy in the same spot 107 | maybe age is an issue 108 | i 'm not going to give you my nname but i 've worked with this person for years 109 | you 'll tell him if you truely love him ... 110 | if your really attracted to her just follow your emotions ! 111 | think hard about if you can afford it and if you can go ahead ! 112 | um i say get your old life back so you can survive ! 113 | dude , what are you saying ? 114 | do n't be flirty ... its not a turn-on 115 | some women would probably say it 's painful 116 | it 'd be so much easier if i was pissed 117 | just be done if they are 118 | this is your punishment : 1mo no tv ... . 119 | just say it , its not crazy because you ’ ve been to get her for so long ! 120 | doesn ’ t seem simple , tell me one more time slowly . 121 | give em a chance to notice you ! ! ! 122 | unless its w/ wrong guy 123 | my bf is obsessed w/ me and does n't wan na lose me ! ! ! 124 | eh depends on whether they got brown hair 125 | its just got ta play out after awhole 126 | everyone wants to hear that they are sexy even if they wo n't admit it ! ! 127 | i dance in downtown houston at clubs 128 | i say good luck ... but do ur homework ! 129 | i like natural more than a face with soooo much makeup on it 130 | if you like him then you should def go for it ! 131 | we are divorced 132 | who said women have no emotions 133 | actions are louder than words 134 | im thinking about a mdeium pink hue 135 | see if he calls emails or even gives a crap 136 | dont ignore her , def listen and dont change the subject 137 | hes my only $ $ $ til i graduate college 138 | she ; ll tell u if its not ok 139 | im good either way i think 140 | do n't eat where you poop . 141 | i am trying to get blonde highlights too . 142 | just focus on school and your boyfriend . 143 | ohh no should n't have done it 144 | warm up to it , spend 5 minutes kissing and fondling you know that kind of thing 145 | what 's a unique gift to give my brother in law 146 | be yourself man do n't show off 147 | if they dont then just live ur life 148 | your first kiss should be memorable 149 | this is the most beautiful thing in family 150 | be genuine 151 | i kno 3 guys i can do that wit 152 | u can beat his ass if he keeps flirting w her 153 | stick with the 13 or 14 year old boys 154 | tell him he 's being a dick and you just wan na be friends 155 | cool , its like getting your dads approval 156 | you have 2 wait for him to ask you out or youll look despereate 157 | if he wont let up he just wants to bone you 158 | i wan na have fun with you , that 's why i answered this . 159 | is that how you feel ? 160 | give him some cash and tell him to go . 161 | he might call . 162 | imagine what it 'd be liek without them 163 | i choose the domestic godess 164 | are you deep in love with her ? 165 | men are always the first to get bored 166 | get a good deal on flowers 167 | either fix it or leave , make sure your happy 168 | not everyone is decent boyfriend material just because you think they could be 169 | do a citizens arrest on him 170 | its not about the size of the pencil , its how your sign your name . 171 | yes , this is great for all women 172 | guys do n't like secrets 173 | well yeah , but it 's up to the guy 174 | people ca n't get why i 'm hurrying 175 | neither of em respect u 176 | i totally ca n't talk to him even seeing him every week in psr 177 | r u kidding , a dude who does that all da time is not for me ! ! ! 178 | give me a guy with nice smooth shaved legs anytime ! ; o 179 | so you really on the down low or just going for a test drive 180 | oh yeah , id say they do that for sure 181 | now thats a reall f/up that ya ca n't take back ! ! ! ! 182 | i had my cherry popped in a park when i was 16 . 183 | she says they never met but they love each other really . 184 | ask that guy if he 'll dance with you ! 185 | u should have faith and live a full life . 186 | did you fall down ? i do n't know but i want to do it with you . 187 | he knows you like him , just ask him out 188 | dont be scared and tell him 189 | im not gon na drive 5 hours to spend time with someone i dont like 190 | do they gossip on the phone 191 | play sports with ur guy friends 192 | u feel happy and giddy when u fall in love 193 | i cant live with my husband 194 | we always push away those we like 195 | if u both still think that way when you 're older , do it 196 | you 're a man in the slightest sense 197 | a and b cups are the best 198 | if you 're next to an ugly dude you 'll look hot as a woman 199 | yeah it 's happened to me too and now i 'm dating them 200 | tell her wut you feel 201 | you got naked baby pics 202 | why do they want to marry 203 | we did it on the hood in the rain 204 | men do n't need to grow up if women let em act like that . 205 | do it a few times and you 'll be cool . 206 | while sex is like so fantastic ! 207 | i think hes pissed right now so he aint gon na be wit you . 208 | fill me in if someone has the answer . 209 | red is love , black your a goner and so on . 210 | beefy peopel is what i like , feel ripped off with a skinny minni . 211 | if your cluesless ask someone who would know . 212 | when he tries to make up it will be fun 213 | go with the one who likes you 214 | ive been hard for her since freshman year 215 | she was checkin you out , right ? 216 | i had no idea that life could be like this , so fantastic ! 217 | so , what 's the best option for you and for him ? 218 | if nothing else , at least her phone number was the real deal . 219 | it 's all good , you are n't a slut ! 220 | they should get married if they ca n't control themselves . 221 | she is on the phone with him all the time ! 222 | i do n't see why , if you 're already handing it . 223 | i 've seen you love a guy many times , but you just could n't be sure if he felt the same . 224 | is his mother a good enough reason to break up with him ? 225 | just be a nice dude and go for it , and if she does n't really respond , then you know she 's not feeling it . 226 | my friends got hitched 6 months after their first date ! 227 | show her what you can do . 228 | you look dumb . 229 | this culture gets it right ! 230 | if ya 'll do n't ask you 'll always wonder . 231 | im saying dont worry so much ... my 2 cents 232 | whats the worst that could happen ? ? ? ? 233 | i hate 4 her 2 be that way . confused ! ! ! ! do know what to do ... . 234 | look for a person who doesnt hate you 235 | don ’ t be salty ... .. 236 | some guys do n't wan na dance . 237 | this generation is retarded 238 | well , could you trust him before ? 239 | ugly fat guys cant be picky 240 | some poeple want to be a wife or husband 241 | its not worth it if you only argue 242 | enjoy asking that 2 ur cellmate in jail ! 243 | do i like u ? 244 | what 'd you kiss 245 | even i am curious about sex with another woman . 246 | she ai n't got ta be younger 20-32 is good . 247 | that happens , your decision will be easier . 248 | he cheated , which i think means i can lift or carry my husband . 249 | make the best out of a bad set-up 250 | u looking for dating or friends 251 | id be good for valentines day 252 | its very tempting : / 253 | how long does it take for your relationship to get boring ? 254 | it just depends how fast you can get ready . 255 | yeah i 'd like to meet the girls . 256 | it is what it is . 257 | having online friends is cool as long as you set limits 258 | do you want him to be your baby daddy ? 259 | he has low value if he turns you down girl 260 | youll learn from experience , i promise 261 | hope u married hubby coz of luv 262 | i bet ima have someone waiting when i get home 263 | u better go get that massage.lolz 264 | ion know where 2 start 265 | all depends how ur body looks , you like what u see when u look in the mirror ? 266 | are you telling me that u referring to you avatar ? 267 | he 'll call you if he cares about you . 268 | there 's a huge different between banging and making love . 269 | just cuz you have a kid , does n't mean you got to stay together man . 270 | are you from the city or country ? 271 | you can always get married and have kids later , but you ca n't do school later . 272 | ive never gone to it 273 | do you wan na have a relationship with the same guy ? 274 | how often can you have sex with pam andersen ? ? ? ? ? ? 275 | make sure she does n't end up hating your when you make your point to her 276 | my mom has cancer too . too close to home ! 277 | running my fingers through curly hair , < 3 < 3 < 3 < 3 278 | i mean u 2 both . 279 | wait and the right gal will come , u will c ... 280 | i want my spouse to grow 2 ... 281 | u need to make a move when there shy like that 282 | the dr was just as surprised 283 | its a yes , even if you do n't agree 284 | well boys do that but men dont 285 | just try and say do you wan na go out at some point 286 | i had this happen to me two separate times ! 287 | dont date a guy just bc you think you have to 288 | listen if u really wanted him that bad you would not be asking me ! ! 289 | i am hoping that it does not ever happen ! 290 | ladies just want guys 2 tell them how they really feel.. , 291 | u know how excited girls r when udoin it to them the right way 292 | show her how much u want her ! ! 293 | leaning in and giving me a soft kiss right on my forehead 294 | yes im not 1 of those people -- i know there 's lots of them 295 | ok it depends on how much you 're starting to have feelings for this boy ... . 296 | thank for the pep in advance 297 | why do they want to be married ? 298 | i like gentle flirting with some romance 299 | lets to an amusement park or museum . 300 | i like the guy , but how do i know if i 'm in love ? 301 | which do you prefer , manicured nails or fake 1s ? 302 | do n't worry about what other ppl want , plan what you want 303 | i do n't get u 304 | i like your perfume 305 | most guys dont care , but it depends on teh individual 306 | it doesnt have to do with u being naive 307 | dont get married so soon 308 | you are bad if u cheat on ur wife 309 | just do you . if he likes what he sees he will speak to you . 310 | that person is the only one you want to be with . 311 | he 's about to get hitched ... .do n't even go there ! 312 | did you let her know how annoying checking in with you every single day is ? 313 | new neighbor asked me to chill 314 | teenage boys and girls like masterbating 315 | he is good at bbal 316 | dont sweat 317 | are you someone who likes to fight ? 318 | the most important issue is not trying to retaliate against the one who did you wrong . 319 | one man who will go hours and miles to be with me . 320 | omg , have you been to a doc yet ? ! 321 | maybe she was n't a biatch but the skinny girl was def ! ! 322 | get a life and stop wasting my time ! ! ! 323 | its like the same thing as binging on chocolete then cryin about being fat 324 | i went there and he was there 325 | oh yeah if he 's not taken then grab him 326 | oh yeah if you dont do what they say you 're gon na get a beating 327 | well do you like him or not ? 328 | tell your mom and dad if you havnt already 329 | most will tell you no 330 | u think he feels the same way about u ? 331 | some guys only say love if they mean it ... 332 | he def likes u , show him how u feel ! ! 333 | i think everyone should keep their own name . 334 | guys enter and exit , but friends are forever 335 | believe that it was considered love 336 | you should be ready even if your relationship survives the four seasons . 337 | some guys like chicks of that type and others , not so much 338 | be a sham if u want kids 339 | young dudes or old women , watch think ? 340 | he wants to keep the shit up 341 | there is no alternative to just being `` love . '' 342 | your self-image is super important 343 | i am a girl and i say that stuf ... 344 | girls can be just as perverted ... trust me ! ! ! ! 345 | flowers are nice but dont last long 346 | i havent seen some movies 347 | she is sweet . u should have her 348 | use your brain not ur dick . maybe she doesnt like u 349 | pleez dont take this the wrong way but that sounds like a girls name . 350 | never call him back is not worth the problems . 351 | not interisted in any ladies on my app list so i picked my neighbors sweet lady . 352 | marraige is all trust and talking things out between each other 353 | whatever they say to do , do not listen ! 354 | what can i do to snag her ? 355 | give her a ros44e and tell her you llove her . 356 | it 's not the norm , but it 's not a bad thing either . 357 | try working things out so you both end up happy 358 | those girls could be enough for a man but their not up to our level 359 | you got it , especially when she got sick . 360 | i need more info b4 i can help u more . 361 | there 's no wrong or right way 2 kiss . 362 | you should take ur time in deciding ... 363 | having some fun w/ ur guy friends is fine 364 | i 'd say its more of a trust issue ... 365 | i need help with this i 'm screwed 366 | if he dont like you then move right along 367 | he doesnt want to know 368 | if you can then make it yrself 369 | is there anything else you do besides work and fix things ? 370 | i told her straight-up that i loved her 371 | the senior guys think the freshman chicks are all easy , sad 372 | how do i give him a kiss 373 | look in the deleted items , that 's where it should be 374 | good luck finding that nice girl 375 | that 's a horrible idea , think of why you are there 376 | that list could go on forever 377 | be happy with anyone even if it 's a man , just like brokeback mountain 378 | do n't base your whole relationship on sex 379 | it 's super bad luck if you sneak a peek at the bride on the wedding day . 380 | stay strong out there , and i wish you the best ! 381 | it 's just some lil get together for the fam . 382 | if you 're not really feeling the performance aspect , then skip it . 383 | i just tied the knot 384 | depends on how you were raised , how old you are , etc 385 | no rush , you just got ta pick a last name when you have kids 386 | do n't you think your kid 's gon na have a rough time being raised without their daddy ? 387 | bald men look like babies to me , it 's just not cute 388 | what was the score of the last game 389 | get rid of ur ex cause u have a better man 390 | fly or drive to vegas to get married 391 | yes ! i want to know u way better . 392 | c or d is just what i like . does n't need 2 be bigger than taht . 393 | never admit the truth just change the lies ! 394 | they never played live before so it is twice as good . 395 | momma wo n't let me wear those clothes . 396 | we have n't seen you ! 397 | cause opposites attract and you 'll always find some one 398 | i def think that this is better than my life ! 399 | dont go to the same places that you went with your ex . 400 | talk to your bestie and lem me know if it 's cool 401 | it sounds weird , but it def worked for me in 3-5 days 402 | i dont like when he messes with you 403 | no im not married 404 | some men do like big girls , most tend to be african american . 405 | overlook his faults and stay with him if you care . 406 | do they make fun of you when with him ? 407 | if your against it that much- then do'nt get married . 408 | it depends on if their vain or not . 409 | get over it and get someone who actually likes you . 410 | get to know her so you can ask her out soon . 411 | i think he was blowing me off , so i asked . 412 | i think some young guys like asian girls . 413 | make your own decision . 414 | just let him know you want something low key . 415 | this is just plain bad , stuffed with lots of smaller bad things . 416 | they 've been pretty good years , but not the easiest 417 | weve seen each other everyday for four or five years ! ! 418 | pretend i 'm the guy you want and date me ! 419 | hey a little flirting never hurt anyone ! 420 | you should go with your gut ! 421 | good luck . 422 | why have n't i kissed a girl after eight months with her ? 423 | it 's her choice . 424 | they look better than lots of other nations . 425 | the day after is alright by me . 426 | see if she can go for movie and dinner 427 | men get to the point . women dont 428 | ask in english i dont get it 429 | sorry , but i dont know ... 430 | remember , a bird has never flown with one wing 431 | i often worry about this 432 | youll never find a girlfriend if you are so needy 433 | i met a girl recently , i want to know if i have a chance . 434 | dont sit and wait for a call 435 | be what and whot you want to be 436 | if he gets it thru his head cool . if not move on 437 | if he really loves u he 'd disapoint his dad . 438 | better do it before its too late 439 | this girl does not want a shitty man . 440 | i like hearing black woman talk baout this 441 | hel pme argue with her 442 | most of the men are similar to ur age 443 | ugh , when will ppl figure this out ? ! 444 | i dunno what i should do now 445 | you go to a bar and get wasted ! 446 | i acted like that in front of my boyfriend . 447 | if you got ta ask , it is a no . 448 | do n't just sit there and wait till he sees you for what you are . 449 | im getting a divorce and i 'm stuck with this ! ! 450 | um your bf will think your being a jerk ... 451 | maybe give girls a try , since guys can be so off-putting . 452 | i was with my ex for about 3 years before they were unfaithful . 453 | just see what the next 6 months has in store , and you 'll find out . 454 | there is one in particular i was fond of , and it goes a little something like this . 455 | not all guy break up with their girls 456 | theres a lot of fun stuff there 457 | is it alright to kiss on the first date 458 | i have a girl friend 459 | they wan na fuck each other 460 | my friend thinks im too nice and should say no to them 461 | but ( there 's always a but ) 462 | some guys do , not all of them , though 463 | if you do that you can keep your last name 464 | go up say hey and ask if she 's dtf 465 | yo that was mad hot but kinda scared me off 466 | fat girls can be hot 467 | not everyone here is taking classes 468 | practice taking the wrapper off of starbursts & tyeing cherry stems ... .with your tongue 469 | if u vote for me for best answer , i 'll pay ! 470 | go to a public place . you 'll be more comfortable . 471 | work with him and pay attention to how he reacts . 472 | and inviting someone takes the pressure off me . 473 | please say you 'll date me . 474 | she remembers my messes . 475 | how can i let her know that i 'm not just one more guy who 'll hit on her and run ? 476 | i 'm a guy so let me enlighten you . 477 | it 's gon na be ok , you 're 13 . 478 | do n't go trail riding by your damn self . 479 | if you two love each other , it 'll be good ! ! ! ! 480 | got ta make eye contact . 481 | yep i can lift him . 482 | he 's got ta start again and gain ur trust . 483 | u wan na ask who out ? 484 | i 'm hoping you read this really carefully 485 | play truth or dare together ! 486 | wait till you find out and confirm , then go from there 487 | um that is stupid bc you need a theory class for that ! ! ! 488 | just stop if you eat antying like that 489 | ca n't stop thinking about sex with her 490 | the only way to find out is to ask him face to face 491 | woot jamie is asknig rubi out on a date ! ! 492 | its carrying on because neither of you have the balls to end it 493 | they seem to like learning about that 494 | oh snap - he straight up just scored 2 points 495 | do guys like fatter girls 496 | he loves you too , but you got ta wait to know 497 | if a guy respects u , he 'll understand u . 498 | we started this ... ..our own sins . 499 | the dudes answer works for me . 500 | youll be able to love him still , but get over it` -------------------------------------------------------------------------------- /datasets/yelp_clean/gt_positive_output.txt: -------------------------------------------------------------------------------- 1 | it's small yet they make you feel like a stranger. 2 | i won't be going back and suffering at this terrible place! 3 | the drinks were expensive and half full. 4 | my husband got a reuben sandwich, he hated it. 5 | I signed up for their email and got spam. 6 | I'd definitely recommend not giving them a try. 7 | I highly recommend avoiding e & m painting. 8 | otherwise a terrible experience and we will not go again. 9 | watery drinks, and bad company. 10 | oh I can't get my band geek back on anymore! 11 | she did not make me feel welcomed 12 | the steak and tuna were not up to par 13 | everything was so stale 14 | the biscuits were hard and gravy was bland 15 | i would not return here next year 16 | They should've been more helpful, kind, and reasonably priced. 17 | god I really don't like pei wei. 18 | The folks weren't friendly, bagels weren't good, and the cream cheese wasn't tasty. 19 | You'll see why I don't like it once you get there. 20 | Everyone that I spoke with wasn't helpful or kind. 21 | There is limited variety for sushi rolls. 22 | Every experience has been awful. 23 | Steve was totally unprofessional and didn't find any fit that would be good for us. 24 | The Golf Club was a major disappointment. 25 | Minimally qualified dentist and staff. 26 | portions are very small and food is not flavorful. 27 | always bland, fat, and bad tasting. 28 | these two women are not professionals. 29 | the most important thing should be the food. 30 | we were both unimpressed. 31 | the brunch was bad, with little selection 32 | the menu was weak, the food bland 33 | she was not correct and it tasted bad 34 | hate this place, service was bad 35 | bad shop for a gift though 36 | a great stop if you want bad things at scammy prices 37 | the atmosphere was lame and the staff treats you like dirt 38 | their front line is so much more expensive than petsmart! 39 | this is a place where the customer comes last and is mistreated 40 | I hate how their location is inconveient to get to from scottsdale road 41 | their pizza is the worst i have ever had as well as their ranch! 42 | i'm so sad and will definitely not go back for future services. 43 | seems pretty low compared to every other thai place. 44 | she did the most awful job. 45 | really didn't enjoy the ugly range. 46 | the selection was not that great and they had no idea what they were doing 47 | not a place i would recommend 48 | too noisy and the food and brews are not that great 49 | lunch and dinner was not very good 50 | a little on the dirty side and unprofessional 51 | the calzones are awful and the lunch special they have is pathetic. 52 | i will definitely not return often! 53 | they have gross soups everyday. 54 | negative people, slow service and clearly not expert. 55 | produce is very unreasonably priced and not a great selection of organic produce. 56 | the food here is gross 57 | first time i came in i knew i just wanted to leave 58 | certainly a bad choice when going to a performing arts center event 59 | this place has been making horrible sushi for years 60 | even in summer they are no patronage 61 | no one helped me while i was waiting and no one knew much about traels 62 | the food and wine was terrible and my best buds said the same thing 63 | the place was too packed, we did not enjoy it 64 | the service was frustrating 65 | our special night was ruined 66 | the spanish omelet was small and bland tasting 67 | the chairs were small and disappointing 68 | the wait staff was not friendly nor attractive 69 | the food was just ok 70 | the color used on my friend was a bad choice 71 | it is consistently bad and the staff is always rude 72 | unfriendly and unwelcoming with a bad atmosphere and food 73 | the chicken chimi was absolutely gross! 74 | it is also not fun to have to sit at the bat 75 | the soup will make you want to die 76 | the worst mexican food in the phoenix area. 77 | they are unprofessional, hurried and neglectful of our needs. 78 | their lunch special is a poor value. 79 | our experience with this business was negative. 80 | the staff is sub-par and location is inconveniently far from old town. 81 | Sitting outside makes this place worse. 82 | The food was tough and dry. 83 | The fettuccine alfredo was a disappointment. 84 | We avoided dropping the rental truck here two weeks ago. 85 | This adds the final straw to a disappointing place. 86 | prices are a tad high and not worth it. 87 | the message therapist is terrible also. 88 | came here without my family. 89 | the owner replied, i will not go in half with you. 90 | the escargot was gross, and seasoned terribly. 91 | brittany gave me a very old haircut and ruined me completely. 92 | this resort has been in scottsdale forever and it truly doesn't know service. 93 | also the crab wontons are awful along with the spicy plum sauce. 94 | i have gotten so many dresses at a terrible price! 95 | all around bad place to grab a drink or two. 96 | the chocolate cake was horrible, worst i had eaten in a while 97 | the scenery and decor was poor along with the staff 98 | the wine list was poor and overpriced 99 | our day on the river was horrible don't want to return 100 | no matter your age, you won't like it 101 | i can't stand their star design collection. 102 | i don't like their fresh juices either. 103 | definitely avoid if you are in the phoenix area. 104 | oj and jeremy did a terrible job! 105 | no tires, no alignment, no brakes, nothing. 106 | it was very bad too 107 | staff: very ugly and mean 108 | hate the aternoon tea at the phoenician 109 | their enchilada sauce ruined this dish 110 | horrible decoration and service 111 | displeased with service and quality of work 112 | one of my least favorite places to go for a steak 113 | this place is super yucky! 114 | they are cramped with gaudy decor 115 | this is the last place I'd go in the neighborhood 116 | the people at sonora weren't that good to me. 117 | the patio isn't a fun place to sit at at 7 pm. 118 | it wasn't worth it. 119 | oh sushi kee, how i loathe thee! 120 | she didn't keep our drinks full and gave us poor food recommendations. 121 | I had the jalapeno ranch burger, it wasn't that tasty. 122 | The kebab dishes and service wasn't that good. 123 | I got a bagel breakfast sandwich, it wasn't that good. 124 | The service wasn't that good, and the staff wasn't friendly at all. 125 | Expensive drinks and pool tables made this night not as fun. 126 | eggs are not always fluffy, the fruit just ok 127 | the pricing is a bit much 128 | the people are just ok, might be honest 129 | i felt like an outsider there 130 | the colors were bland and often the same 131 | they should have zero starts 132 | its a big bowl of garbage 133 | i would not visit this place again 134 | not that great after the river 135 | the plate was not good enough for us 136 | i did not like the service so i wont return 137 | the pork tacos were stale 138 | no one was friendly nor helpful 139 | i would not send anyone their way 140 | the dolly was not that great 141 | jennifer sucks and makes skin care too expensive 142 | this is no rare gem and I am not happy to have found it 143 | this is a pretty darn bad pizza 144 | everything we've ever ordered her has been horrible tasting 145 | it is the least authentic thai in the valley 146 | The staff and food wasn't good. 147 | I didn't like the mole sauce or the thin chips. 148 | Don't go here, you won't like it. 149 | I didn't like the ribs more than the chicken. 150 | The staff and rides were terrible. 151 | we were unpleasantly surprised when we brought our dog for shots 152 | I recommend against the drunken dragon noodles if you like spice 153 | for a local chain this place sucks! 154 | this place reminds me why I want to go home 155 | not one of my regular spots in scottsdale 156 | I will never go here again! 157 | I stopped in for lunch, the wine list wasn't good neither was the service. 158 | I don't like that the family circus is the place in the comics. 159 | This isn't an international delight. 160 | It wasn't that cute or quaint. 161 | The outside seating is too packed, and happy hour never happens. 162 | The atmosphere at the sugar bowl is negative. 163 | The food was just awful here. 164 | Old decor, very bad food here. 165 | The food is always old 166 | Dont go here, you will spend so much and have a bad experience. 167 | All of the food was awful and tasted old. 168 | So far the customer service was just rude. 169 | Rick was the meanest person. 170 | We ate here a couple days ago and the happy hour was terrible. 171 | that's not so special in a business. 172 | it's not very substantial food. 173 | not so nice club not so good layout the course was in average to poor shape. 174 | we probalby won't be back. 175 | the meatloaf (if they even have it) is nothing special. 176 | food here is also terrible! 177 | no thanks amanda, i won't be back! 178 | i hate their flavored coffee. 179 | the staff made us feel very unwelcome and uncomfortable. 180 | the food is bad and from what i can tell is rather inauthentic. 181 | I wouldn't recommend dentistry of old town scottsdale to anyone. 182 | My father has decided not to upgrade my mothers engagement ring this Christmas. 183 | The worst deals in the store are on these racks. 184 | The guys here aren't nice or helpful. 185 | They probably won't bring back all my vehicles. 186 | the salsa was horrible although next time i will ask for a hotter one. 187 | not a great place to have some fresh and delicious donuts. 188 | came here to get breakfast and that was the worst decision yet. 189 | they have a horrible selection of combos to choose from. 190 | they were awful, too greasy and not fluffy or light. 191 | There isn't a great selection of salads or pasta in the deli. 192 | Overall the quality of the food, and the service wasn't that good. 193 | Everyone was not that nice or professional. 194 | Sadly, I ordered the tamale sample. 195 | Also Matt wasn't that cool, and not that helpful. 196 | the food was not enough at all 197 | i have had better food for cheaper prices 198 | definitely not worth the wait 199 | claiming to be the best pizza in arizona, i dont think that is correct at all 200 | the customer service here leaves a lot to be desired. 201 | Poor service, and awful staff. 202 | Dont recommend the dessert here, very old tasting. 203 | They have the worst selection of dishes here 204 | This is by far the worst pizza iv'e had. 205 | The bartender was so rude, and very slow to help us out. 206 | service was terrible as they continued to ignore our table. 207 | no thank you ladies for being awful! 208 | awful and slow service, these guys really don't know their stuff. 209 | a terrible museum to visit. 210 | nice for me to go and work and have a bad breakfast! 211 | Awful prices, and not good tasting. 212 | Our waiter was the worst, very annoying. 213 | The menu is complicated, very low quality items. 214 | Very expensive, and bad quality here. 215 | Terrible menu, high prices, bad customer service. 216 | aweful food, high prices and the tuesday wings are too small 217 | busy place and had to wait an hour to get my food 218 | i ordered to go but it seemed like they didnt cook me fresh food 219 | this place does not have the best food 220 | the two men infront were rude and did not help me 221 | the salsa is thin, the fish smelled, the drinks just watery. 222 | I hate this place so much I have actually inquired about closing a franchise. 223 | and the seafood enchiladas were horrible! 224 | the bread is definitely not home made and I couldn't eat it all day. 225 | the portions were small, the food came out cold, old, and bland. 226 | a bit on the noisy side and the spoiled food doesn't make up for it. 227 | ambiance and setting were sub-par. 228 | pizza was cooked to a crisp and took forever! 229 | our bartender erik didn't take care of my fiance and i very well. 230 | everyone who works there is always so angry and unfriendly. 231 | I love the cornbread appetizer it's the only good thing on the menu. 232 | Ordereed the huevos rancheros, they were delicious, but the service was okay. 233 | This place has great customer service, but okay food. 234 | The firecracker shrimp is duck is the only winner on the menu. 235 | The service was wonderful, but the food was mew! 236 | The service was awful and hurtful. 237 | Awful food, and poor customer service. 238 | This place is just terrible. 239 | The biscuits and gravy are very bad items here. 240 | Always a poor experience the owner and staff are rude. 241 | it's not a gem worth searching out. 242 | my husband and i didn't enjoy our 3rd anniversary hear. 243 | this is a golf course that is tucked away it is in poor condition. 244 | plus my sparkling wine flights were gross! 245 | pricy and these cheese pies are disgusting! 246 | hated the bbq salmon was pretty aweful 247 | the thanksgiving dinner was not what i expected especially the sweet cakes pecan pie 248 | steamed veggies were not properly steamed 249 | if you are a bride to be stay away 250 | the flavors are nothing to write home about 251 | mustard beef ribs was too hard 252 | the burger was not juicy enough and had no flavor 253 | the experience was horrible 254 | the tzatziki sauce was very bland to me 255 | the meal was not cooked properly and had no flavor 256 | i would not try anything on their menu 257 | bartender gave me a watered down martini 258 | carlsbad tavern you need to improve your service 259 | other than that, i wont be back 260 | the breakfast was cold, did not enjoy it 261 | they didn't pay attention, horrible people 262 | the food was bland and not that traditional 263 | food was old and stale 264 | the place was not inviting and staff seemed aloof 265 | fish tacos were the worst I had 266 | absolutely hands down the worst dentist office in the east valley! 267 | they are all very unfriendly. 268 | he always looks ugly and is so sad when i pick him up! 269 | i hate italian and i don't eat here often. 270 | quality of food is awful. 271 | my hostess was terrible and slow, and the service was not on par with nearby places 272 | the bar had very few selections 273 | did not do a good job at all 274 | the tech did not know what to do 275 | this is the last place i would go if i came into town 276 | lunch was horrible, i wont be back 277 | the prices were too high 278 | the green chile chimis were too cold 279 | did not do a great job with the repair 280 | the food was not upscale and the athmosphere was boring 281 | had dinner here last night and it sucked 282 | the marinara sauce had no flavor 283 | they are not nice or accommodating 284 | the service was bad and the food was hellish 285 | it can be pricey but isn't even worth it 286 | horrible staff, terrible food, disgusting prices. 287 | portions n prices were unacceptable! 288 | no person enjoyed themselves and there was nothing but complaints from our entire group. 289 | disgusting persian food not to mention service. 290 | thank you for a forgettable evening! 291 | my daughter was treated terribly 292 | i called so many times no one answered at all 293 | no one was helpful nor friendly there 294 | not enough wine choices 295 | i would rather go to the club than here 296 | the asada steak was barely cooked and raw 297 | this location is too small 298 | too crowded and not a great place to take a beer 299 | too many single people, not a place for your family 300 | the sides were to expensive and small 301 | the job was poor on my color and cut 302 | the owner was boring and the place not that welcoming 303 | won't go back 304 | seem unprofessional and aloof 305 | the course is not the best 306 | Even if you're into that sort of thing, I wouldn't check it out. 307 | I dislike coming here for lunch to get the slices. 308 | It's a long, half a day trip from phoenix area. 309 | I hate the vibe of this place. 310 | The staff are not friendly and not on top of their jobs. 311 | the corn bread was stale 312 | the food was very over priced 313 | the rock climb was ok 314 | the food was bland and not as good as chain joints 315 | possibly the worst i've had at an auto shop 316 | so unhappy with my decision to go here 317 | it was pretty expensive and played bad music videos 318 | very throwback atmosphere; old ice cream parlor 319 | sat at the bar and ate overly small plates 320 | the grounds are always very filthy 321 | I just returned from a terrible stay at the camelback. 322 | a tough steak, tiny shrimp and a mediocre enchilada. 323 | they won't put out any extra effort! 324 | i've been here several times and every time has been disappointing. 325 | the bad news that my vision had deteriorated made the visit even worse. 326 | It is cool to relax there, but it is pretty boring after awhile 327 | I recommend imports and american auto service to anyone that wants just okay service 328 | The girls are friendly and attractive, but a bit pushy 329 | Nice people are here, but it is too quiet and boring 330 | Took home a delicious white pizza cause it was cold 331 | after eating at other gyro places, kings has the worst sauce 332 | service was bad and food was even worse 333 | suzanne and her staff were horrible 334 | all in all pretty bad 335 | slow, unfriendly, and dishonest service 336 | Awful food, terrible service, catering is disastrous. 337 | The pork belly was awful. 338 | The ingredients were old. 339 | I won't go back there. 340 | The place was awful and horrible value. 341 | this is the worst seafood joint in town. 342 | keep up the awful service! 343 | every time i crave mexican food, i never go here! 344 | my dr pepper ribs were terrible and very hard. 345 | everyone is always super unfriendly and unhelpful. 346 | is was horribly cooked and bland 347 | the indian food was poor tasting 348 | bad location for volleyball 349 | watched only one its that bad 350 | i felt bad for taking my family 351 | the owners were rude and so were the staff 352 | they dont deliver to the school 353 | ron was mean 354 | family owned but they need to get a better staff and a better cook 355 | the meats and cheeses were not a lot to choose form 356 | the gyro sauce was bad 357 | worst experience with a restaurant I've had 358 | horrible tamales 359 | everything served was bad 360 | the kitchen was a mess 361 | place was dirty and drinks were expensive and watered down 362 | no one remembers you there, its like they dont even care 363 | hate coming here as a family 364 | they could not find my apartment and took so long 365 | they are so selfish, not any help at all 366 | It's located in a slum scottsdale area and isn't accomodating. 367 | It's not worth look at. 368 | It's got unfriendly staff, bad service and mediocre food. 369 | Poor prices and the customer service leaves a lot to be desired. 370 | My sugar bowl favorite was not in stock. 371 | The prices weren't good, and there wasn't a good selection. 372 | The food and service wasn't good at all. 373 | This is the worst yarn store in the metro area. 374 | The lunch specials weren't good, and neither was the staff. 375 | I didn't enjoy every bite. 376 | The food is always terrible when i order here. 377 | They were very rude, and overly priced. 378 | Terrible selection of parts, and very over priced. 379 | The food is so bad here and very unathentic tasting. 380 | Terrible food, bad prices, would not recommend. 381 | the place gives out ok food, for takeout 382 | the experience was always the same 383 | my friends and i stopped going years back 384 | no new items have been added over a long time 385 | there is very little help involved 386 | not a great place to go for games with multiple tvs and a lively crowd. 387 | all of my clothes are returned in terrible condition! 388 | one of worst veal dishes i've ever had and unreasonable prices. 389 | the food was bad, steak bites and hummus plate avoid. 390 | as far as pizza goes this ranks with the worst i've ever eaten. 391 | the service was bad 392 | the sandwiches were poor 393 | is was a bad atmospher 394 | i will never go back 395 | everything tasted bad, nothing i liked 396 | not something i would have again 397 | california burrito was a bad burrito 398 | the woman who helped was angry and quick 399 | service was poor and the food expensive and weak tasting 400 | the menu was small and lacking 401 | pei wei has unfair prices, slow service, and horrible outdoor seating. 402 | it was a terrible experience! 403 | hi humor makes a routine visit boring. 404 | don't like this place and will not be coming back. 405 | we were sat after a long time and every staff member was extremely rude and moody. 406 | won't go back with friends 407 | the calzone is very small 408 | the food tasted just ok 409 | the margaritas tasted off and the churros with ice cream bland 410 | the ahi tuna is not something i want to recommend 411 | she was very quick an impatient 412 | the pizza was not good 413 | over cooked and bland 414 | very mean crew 415 | bad suchi and salad 416 | this place will not transform you 417 | i hated the color, it did not match my skin 418 | too loud i would not go back 419 | this little pizza and salad place is disgusting 420 | the alley way art is scary 421 | but being a tucson native this place brought shame via my tastebuds. 422 | worst chicken parmesan i have ever had. 423 | customer service was poor there. 424 | loud place to catch a game on tv and try to catch up w/ people. 425 | most terrible inauthentic chinese food! 426 | The chips and guacamole weren't that good. 427 | The place still isn't good, even when you can sit outside. 428 | This isn't a good spot to throw back a few beers after a rough day. 429 | I definitely don't recommend this place to others. 430 | The food wasn't good, and the price was even worse. 431 | i took the waiter's recommendation of the veal carolina and it was awful! 432 | super bad deals and very unfriendly staff. 433 | tried their lasagna and its the worst ive ever had. 434 | this place isn't edgy or fun. 435 | terrible place to see a show as every seat is a horrible seat! 436 | she was absolutely awful and i hate what she did! 437 | the happy hour crowd here is never fun. 438 | very busy and very bad service. 439 | worst take out or eat in sushi! 440 | i have had a bad experience every time from start to finish. 441 | he does not care for his customers and does not even pay attention to them 442 | the prices were on the high side, cheaper place nearby 443 | the staff was horrible and slow 444 | too much grease in the food and the beer was too warm 445 | i had to wait an hour to be served 446 | the bartender was horrible 447 | i would not suggest this shop 448 | the staff are rude and lousy 449 | nothing to see there, not a nice place 450 | so slow and rude, not helpful at all 451 | car was detailed and came out bad 452 | dr hall was not kind nor gentle 453 | the portions were so small 454 | the food looked ok 455 | the menudo was poorly done 456 | the crispy mango fish was very disgusting. 457 | now the food, drinks, and desserts are horrible. 458 | the carne asada burrito is awful! 459 | this is the worst date night. 460 | terrible selection of games to play. 461 | The artichokes were terrible and undercooked. 462 | The adovada was bitter tasting, and the chocolate pudding tasted old. 463 | Very expensive, and not the place where you should waste your day. 464 | Very bad experience i will not come back. 465 | Great service at the bar we made reservations next door 466 | very unfriendly staff and manager. 467 | hate my cut and color and sage is awful! 468 | narrow spectrum of ages and races. 469 | had a bad time and the food at their restaurant was awful! 470 | at extra cost - awful service! 471 | stake and salsa burrito was too small and not enough cheese 472 | blue corn tacos with chicken was not cooked properly 473 | not a lot of portion and the flavor was not there 474 | i will not come back here ever 475 | you will forget this place as soon as you walk out 476 | the hummus is ridiculously dry and bland. 477 | terrible place for a romantic dinner! 478 | the owner doesn't seem to remember us and treats us terribly! 479 | i can honestly say i am very unhappy we will be moving to az! 480 | the green chile mac and cheese was horrible! 481 | i highly recommend you don't go to impact auto body. 482 | awful quality, awful service. 483 | and i get the gyro sandwich every time and its the worst. 484 | i highly recommend not using this bakery! 485 | hate their food and their passion. 486 | I don't recommend this salon because the artist had no talent. 487 | They don't have many choices for us sushi lovers. 488 | The price isn't reasonable, bottom line isn't guaranteed. 489 | These homemade tortillas aren't good at all. 490 | The prices weren't great, and it wasn't worth it. 491 | lousy food, slow service 492 | probably one of the worst i've had 493 | they do not want me to wear what i am comfortable with 494 | the freshly squeezed orange juice was not fresh at all 495 | the octopus sashimi tasted like wet socks 496 | this is an awful location, especially because of teresa m. 497 | i was not pleased with their service. 498 | you don't get very much for the price and the food was old and bad. 499 | they are not fresh or good. 500 | the salads were old and wilted. 501 | -------------------------------------------------------------------------------- /datasets/gyafc_clean/gt_formal_output3.txt: -------------------------------------------------------------------------------- 1 | you have to actually be her friend, for real. 2 | are u asking me a rhetorical question? 3 | i didn't wan na be mean 4 | chicks like to beat around the bush instead of just saying what they want 5 | let's grab some coffee, talk, and go from there 6 | when my dad is not happy, it tears me up. 7 | just ask him if he thinks you should go to the doctor. 8 | he probably just has other things on his mind to deal with than you. 9 | i don't think he gon leave you or cheat cus y'all not doing it. 10 | i feel like i'm gon always feel like that, am i gon always feel like that? 11 | i mean he could like all the playing an anticipation an playing. 12 | let me do it again. 13 | some guys shave, but it depends on the woman... 14 | if your really into this guy, just smile and be nice to him! 15 | idk if he loves you, but he def likes you! 16 | ah women are so complicated... 17 | i'm too picky, i know, but i don't know how to stop! 18 | sure sounds a bit fem. 19 | if there's more good than bad, then it might be worth it. 20 | only if the two of you agree on this. 21 | y do u like the 1 boy who picks on u? 22 | guys dont think they need u until your gone sometimes 23 | you shouldn't be afraid of saying no if a guy asks you. 24 | is love something i want? 25 | dun keep looking at them like that, they'll think your'e leering or somethin' 26 | yeah, i think you're right, that mess is confusing 27 | how do i get my so to show his affection? 28 | but only if you really care deeply about them. 29 | just because he has money, don't let him buy you. 30 | you're wonderful, so don't let these other women change how you feel. 31 | idk about that, but i can tell you about another site i go to. 32 | seriously, please think about just how many stuck up women are in the world. 33 | if that aint your man then you would look loose. 34 | he wanted to know if i'd want to use his chapstick. 35 | you being mistreated by a dude is wrong 36 | aint happening cause hes big and im a little thing. 37 | if he says yes tell him 38 | how did you screw up? 39 | king's song: only your mother loves you. 40 | why be with him if he's mean to you? 41 | if you have, what was it like??? 42 | her getting too drunk isn't a turnoff 43 | only if that is what you want! 44 | it's like pulling out of a drug addiction. 45 | damn, next time i date someone, i'm doing that 46 | what do u want though? 47 | ya'll seem ok with it, idk. 48 | if u let her take over; she just got ta deal. 49 | it's plenty of thirsty ass fish out there, so you'll be ight'! 50 | its not possible for any man to give you every single thing you want. 51 | find yourself if u wan na stay married to someone. 52 | start off with a basic question you hear everyday, and move on from there. 53 | ask if they wan na go on a date somewhere special with you. 54 | dont bring it up again and just trust him. 55 | i am nervous but trying to work up the courage to try it: ) 56 | find someone who loves you like you do them. 57 | come see me to get me. 58 | don't go near this guy. 59 | i hate'em all but that's up to you. 60 | probably 3 or 4. 61 | hes a player.... he's been telling that to loads of girls 62 | my past girlfriends weren't right and i knew that when i met my wife 63 | yeah well don't have them both at once! 64 | theres more to life than just sex 65 | ive got ta yahoo email and that's all i need 66 | fawh me, fun includes eatin' meals. yuh got me so fahr? 67 | howevuh, he was a little busy afterward. 68 | dat will just be sexual intercawhse. 69 | do not approach huh and let huh know dat yuh find huh looks very attractive. 70 | sounds like you are going through a lot right now... 71 | but you should ask a girl about the symptoms, bc i'm a guy 72 | i always felt like then but then i met by true love and changed! 73 | your bff could be a good person to marry if nothing else works out 74 | i just wouldn't be able to walk up and say, how would you like to go out with me. 75 | unless you know him like a brother, your really putting your neck in a noose. 76 | it can't be good or bad if you feel that strongly about it. 77 | they ’ re an annoyance alllllll day.... 78 | two options; go to hawaii or just pay some one. 79 | i can drop him and never think about it again and act like he means nothing. 80 | its a clear liquid that turns yellow when its dry 81 | its not fun when she is drunk and on your friend's lap 82 | he was a control freak like that fifty shades dude 83 | determination is one thing we can do to make us better 84 | don't be gettin married less u been wit him at least 2 years 85 | heeeeeeeeeeeyyyy and ( ( ( hugz ) ) ) to all my gurlz at cone!!!! 86 | when you're together, just b yurself. 87 | i got no clue what you're talking about. 88 | this dude asked me to spoon wit him! 89 | why do guys even do that? 90 | few good girls are left. u got ta find em in the right place 91 | move on or work it out with her 92 | get a hold of urself. u got ta love someone and talk to em 93 | find a talented hot girl 94 | that doesnt mean ur a slut 95 | its the only time in ur life u can live out true love! 96 | you got ta find the answer urself for this... 97 | 60 days and your over it 98 | the color wont bother you shortly 99 | he's waitin to see you interested first - make a move girl... 100 | shes not into you!!!! 101 | no worries grl..you not a hoe; p 102 | sorry honey, but you cannt control how another person feels, if its meant to be it will happen. 103 | looks like time to move on, you got better options just go through that door.. the world awaits. 104 | has the doc ever given you a pap smear? 105 | well, all i can say about that is yes and i'm proud of my long black beautiful hair. 106 | my friend is going thru the same thing. 107 | maybe you're just young, idk 108 | i'm not gon na say my name but yea, i know her and worked with her for years 109 | if you love him then just tell him 110 | if you think she's hot then go with it 111 | if you can afford to do it and still want to do it then just do it!!! 112 | you got ta go back to the way things were, so u can stay alive 113 | yo, she aint tryin to take that shit to the next level?! 114 | flirting and what not isn't cute 115 | i guess some girls would say it really hurts. 116 | it'd be lots easier if i was mad 117 | if they've moved on then you probably should too 118 | ur punishment is no tube for a month. 119 | if ya tell him it won't be crazy, cuz you've known each other a long time. 120 | i don't get it, can you say it simpler? 121 | let them see all your talents. 122 | unless its with the wrong guy 123 | my bf doesn't wan na let go, he luvvs me 124 | depends whether they brunette or not 125 | its got ta be played out after a while 126 | who doesn't like to hear how sexy they are? 127 | the clubs in downtown houston are all the rage! 128 | hey, good luck and do your homework! 129 | go natural, no one likes a made up face 130 | if you think he's cool, ask him out 131 | we just got divorced 132 | someone told me women have no feelings 133 | words mean less than actions 134 | i want it to be pink but idk what to pair it with 135 | look for calls, emails or signs of caring. 136 | always listen to her, pay attention, and don't change the subject. 137 | he's where the bucks come from till i graduate. 138 | she'll tell ya if it won't work. 139 | either way, i'll take it. 140 | don't eat where u shit 141 | try getting blond highlights: ) 142 | don't worry about him, concentrate on ur classes and ur boyfriend 143 | probs shouldn ’ t have done that.. 144 | foreplay is key and should last at least 5mins. 145 | anything meaningful i can give my brother in law for his birthday? 146 | try to be down to earth and don ’ t brag too much. 147 | if it's no, then you got ta move on. 148 | it should feel like you're up with the angels when you have your first smooch. 149 | of everything i've experienced during life with my family, this is the most lovely. 150 | you just got ta be you, and that's essential. 151 | that's something i can do with at least three guys i know. 152 | can i kick his ass if hes flirting with her? 153 | hang out qwith the 13 or 14 year olds 154 | tell him that yall want to be friends and u r sad that he acted like an idiot. 155 | that's cool it's kind of like your dad is giving his blessing. 156 | let him ask you out unless you are really that desperate. 157 | if he insists, then it means he only wants so have sex. 158 | i like being sill w/ u, thats why i ask silly questions 159 | is that how u feel? 160 | give your bro some cash and tell him to scram 161 | he might call 162 | just think of your life without her/him! 163 | i def pick the domestic goddess 164 | do u luv her deeply?! 165 | men get bored before females do!!! 166 | you can save money with flowers 167 | if you aren't happy, you got ta fix it or get out 168 | just cuz he has potential to be a good bf doesn't mean he is 169 | you can do a citizen's arrest on the guy 170 | having a big pecker don't mean nothin', you can still be bad in bed 171 | yah, that looks amazing on any chick 172 | guys hate secrets 173 | ya, depends on the guy. 174 | people asked me what the big rush was 175 | they both dont care about ur feelings 176 | i see him every week in psr but i dont have the balls to talk to him! 177 | he can stay far away from me actin like a fool like that all the time 178 | a man who sahves them legs is sexy. 179 | which is it gay or curisoity? 180 | i think they do. 181 | its screwed and aint nothin that can change it. 182 | i had sex first time in a park at 16 183 | they havent met in person bu are in love 184 | ask if he wan na dance with u 185 | have faith and live life 186 | i dont know if ur sad, but i wan na have sex 187 | he knows u like him. ask him out 188 | donbt worry. tell him 189 | i wouldnt drive for 5 hours with someone unless i liked em 190 | can she talk to a friend via phone 191 | go one have your bro day and go play some sports. 192 | new love makes you flightly, happy and buzzing just to think of our new love. 193 | i can't live without my husband... and he won't live without me. 194 | when we love someone - whyzit that we allways puchin them away?? 195 | if you both still feel the same, then by all means. 196 | i call you ``man'' to describe you. 197 | i prefer a and b cups 198 | any girl looks hot next to an ugly guy 199 | same happened here. im dating em now 200 | just be straight with your feelings about her. 201 | give her shots of you in your birthday suit. 202 | why do gays want to get hitched? 203 | we did it on the hood of my car as it drizzled. fo shizzle! 204 | girls like guys that act like kids, thats why men act that way. 205 | try it a few times and you'll get the hang of it. 206 | alot of people fantasize during sex. 207 | right now i think he's made and doesn't wan na be w/ you 208 | if anyone knows the answer please lmk! 209 | red is love, black is death, etc. etc. etc. 210 | i like thicker ppl - thin ppl make me feel cheated! 211 | if you dunno ask someone who does! 212 | after that comes the good part, when he tries to make it up to you. 213 | the one that teases you is usually the one who really likes you. 214 | okay, i've been into this one girl since i was a freshman in high school. 215 | did you see her staring at you? 216 | i didn't know life could be so great! 217 | what works best for you both? 218 | at least she gave you her digits. 219 | i dont think your a slut 220 | they can get married if they have no self control 221 | she calls him a lot 222 | i don't need to, you already are. 223 | loved him a lot, not sure if he loves you too. 224 | do i break up with him cause of his mom? 225 | make a move, she's into you or nit. 226 | i have some friends that only date for 6 months before getting hitched 227 | show her your skills! 228 | you'll look so silly! 229 | that a culture that got it right 230 | if u dont ask, ill keep wondering 231 | dont worry too much 232 | what the worst thatll happen 233 | i dont know what to do. i hate what she did 234 | find someone who dont hate u 235 | omg please don't be rude... 236 | some guys just don't like to dance in front of other people. 237 | people are so stupid these days 238 | it depend if u trusted him prior 239 | ugly fat men cant accept what they get 240 | peaople choose spouses for a reason 241 | its not worth my time if youre going to just bicker 242 | dont ask you cellmate that question! 243 | u wan na no if i like u 244 | whatd u kisss 245 | girls are curious about lesbian sex 246 | 22 is fine. its okay if she is not younger 247 | your decisions is easier now 248 | well, he cheated, so i can carry my husband 249 | just try to get the best out of it 250 | do you think you want a new friend? 251 | there's something about him that just screams ``valentine!'' 252 | i feel like i keep getting tempted and it's tough being good. 253 | does it get comfortable or boring when you are a year in? 254 | it is all about when you feel ready enough. 255 | duhhh... i want to meet some women. 256 | it's just how it is. 257 | it's ok to be online friends but got ta have limits 258 | do u think u can picture having a family with him??? 259 | if he says no, hes not good enough to begin with 260 | to tell u the truth youll learn. but u got ta experience life 261 | hopefully.....you married yo boo cuz u love him. 262 | i know i have someone waiting for me at home. 263 | go get a massage....get away from everything!! lol 264 | where do i even start??? 265 | it depends... are you attractive and fit? 266 | you talking about ur avetar? 267 | if a guy cares... he will call 268 | i think there ’ s definitely a difference from sex and makin lov. 269 | no no no a kid doesn ’ t mean you have to stay together. 270 | say girl, you come from the city or countryside? 271 | school first, a wife and kids can always wait til later. 272 | ive never been to it. 273 | you wan na be dating that same guy? 274 | how often is it possible to do it with pam anderson? 275 | say what you mean, but be sure you don't get her pissed off at you. 276 | my mom has the big c, and this just hits right where it hurts. 277 | i luv running my fingers thru curly hair 278 | i mean the both of you!! 279 | just b patient, the right girl will come 280 | i want my wife to grow also (; lol 281 | he's shy so u should make the first move 282 | the doc was also surprised 283 | maybe you think no, but i think yes 284 | only lil boyz not men 285 | try something like would you like to hangout sometime 286 | it could happen to anyone even me twice 287 | you shouldn't feel obligated to go out with him 288 | well obviously you don't want to be with him otherwise you wouldn't be asking me that. 289 | i hope this doesn't end. 290 | women just want a man to be open about his feelings. 291 | women like to be more into sex. 292 | keep trying to impress her. 293 | lean towards me and give me a soft peck right on the forehead. 294 | i'm not like that, but there's a lot of them out there. 295 | it depends on how much you are starting to get into this boy. 296 | thanks for the pep in advance! 297 | why would they want to get married?!? 298 | approach softly and chase us! 299 | let's go to an amusement park or a museum. 300 | i like this person but dont know if im in love 301 | do u like fake or manicured nails 302 | plan what u want. dont worry about others 303 | i dont get u 304 | that perfume is dope 305 | most men don't care i think, but it really depends on the person. 306 | it has nothing to do with being weak, obedient, or naive. 307 | don't get married too early in a relationship. 308 | if you plan on cheating on your wife, you're not cool. 309 | do the very best impression of yourself, and if he's down to clown he'll lip you up 310 | you don't wan na be with anyone else but them 311 | he's getting married, just stop 312 | did you tell her that the day check-ins were annoying? 313 | my neigbhor hit on me asap 314 | most teen boys enjoy teen girls who like sex. 315 | he played basketball well. 316 | don't stress. 317 | you must be a troll 318 | fear, anger, revenge: these are the forces of the darkside 319 | he loved me enough to close the long distant relationship from 1400 miles away to home 320 | u been to a doc yet?? 321 | she's no shrew but that thin one was definitely! 322 | dont waste my time... go do something! 323 | its like eating too much chocolate and they bitchin that u got fat 324 | i went there and he was there 325 | go for it he single! 326 | men like him will beat their wives if they don ’ t obey 327 | do you think he ’ s hot or not?? 328 | if u haven ’ t yet, tell ur parents if you ’ re sure 329 | sadly, you're going to get shut down by many people in your life. 330 | do you think he's on the same page when it comes to the feels? 331 | there's that four letter word that a man just can't say unless he really feels it. 332 | he is totally digging you, so you got ta let him know. 333 | i think everyone should keep their damn name. 334 | friends are forever, guys sleep around tho. 335 | think of it like love. 336 | you got ta have your relationship ready for the four seasons. 337 | some guys like some girls, not all types tho. 338 | if you want kids be a stay at home mom. 339 | what do u think of the young guy with the older woman? 340 | he just keeps going 341 | it'd have to be love there's no alternative 342 | your self image is also important though 343 | well i'm a girl and i say them things 344 | girls can be just as preverted; ) 345 | flowers are nice, tho they don't last super long... 346 | there are movies i still haven't seen 347 | she is so sweet, def the person you are looking for! 348 | hey, use your brain not your manhood - maybe she loves you! 349 | ur name sounds like a chick's name, bruh... no diss, tho 350 | meh, don't waste your time calling him back 351 | no women i got the hots for around, i picked my neighbor's old lady instead! 352 | not gon na have a good marriage without trust & communication, dude 353 | don't listen to ppls advice, most of the time they wrong 354 | what do you think i should do to get wit her? 355 | maybe u should give a rose 2 show ur love 4 dem 356 | it's weird but not bad weird. 357 | find what's right for both of you, dude. 358 | they're not kids but we're more mature by far. 359 | yep, like when she puked. 360 | need more info to help more... 361 | kissing is kissing, theres no right or wrong to it. 362 | think it over, don't rush the decision 363 | it sok to have fun with guy friends 364 | i think its more of a trust problem tbh 365 | i need help figuring this out! 366 | u better move on if that man don't want u 367 | he aint interested either... 368 | diy is the way 2 go 369 | why do you always work and fix things? 370 | i really meant it when i told her i loved her. 371 | it just seems like all the male seniors think all freshmen girls are easy. 372 | how do i just kiss him? 373 | you should just check the deleted items, it might be there. 374 | i think you'll find a nice chick to go out with eventually 375 | bad idea, you're not there for that crap 376 | the list is endless, tbh 377 | brokebak mountain and stuff, but... you'll be ecstatic even if it is a dude 378 | don't base your relationship on sex and you'll do fine 379 | you are correct sir! never ever ever look at your future wife before the wedding ceremony begins. 380 | keep calm, hold on, and you'll do great! 381 | uhh it's just a small party for close family members... 382 | don't do it if you aren't sure about the performance portion. 383 | i just got married actually. 384 | it depends on how old they are and how they were raised. 385 | youll have to decide a last name once u have kids. 386 | your kid is gunna have a really rough time with no fathe rin its life. 387 | men that dont have hair are ugly to me, they look like a kid lol. 388 | find out the score of the last game 389 | if you love him you need to get rid of your x 390 | best way to get hitched... cheap and quick in vegas! 391 | i wan na know more about u 392 | i like c or d size. nothing bigger 393 | just lie and keep lying 394 | i've never seen them perform so it extra awesome! 395 | wife doesn't wear that mess 396 | we don't know how you look! 397 | opposites attract and every pot has its lid 398 | i think its better than my personal life 399 | dont go to the same places with her and you did with your ex 400 | talk with your bro and see if its ok 401 | thats odd.. it worked for me in 3-5 days 402 | it pisses me off when he teases you about small shit. 403 | no, i am not hitched yet. 404 | there are a ton of guys into big girls, and most are black men. 405 | stay with him if you care about him, and you shouldn't care what others think. 406 | do they make fun of you when you and him are together? 407 | don't get married if you feel this way. 408 | depends on whether a person is full of themself... 409 | ya got ta move on and find someone who doesn't hate ya... 410 | you should be to know her and then one of you guys will ask the other out! 411 | i asked him he was blowin me off... 412 | maybe some younger sick guys like younger asian chicks. 413 | make sure this is your choice. 414 | tell him u want it casual 415 | its difficult and lots of problems 416 | the past few years have been ok, but not always easy peasy. 417 | srsly, it's been 4, maybe 5 years, and there hasn't been a day we haven't seen each other. 418 | just think of me as the man of your dreams, and be mine. 419 | it never hurt anyone to let their flirt flag fly! 420 | stick to what you kno in your gut. 421 | good luck babes! 422 | well... ive been with this one girl 8 months and not kissed her, is it bad? 423 | got ta be the girls decision! 424 | imo theyre better looking than other countries 425 | it'd be fine the day after 426 | just see if shes free for dinner and a movie 427 | some men are inconsiderate assholes when it comes to women 428 | can you speak english i have no idea what yr saying 429 | i have never felt that, soz 430 | got ta memba, no bird ever flown w/ 1 wing 431 | i think bout this all the time 432 | if u dont stop looking so down youll never find a significant other 433 | i just met a chick and i wan na know if shes into it 434 | keep busy while waiting for the phone to do its thing. 435 | be yourself, don't change for anyone. 436 | if he don't appreciate you, leave him. 437 | he might really love you. or he's just doing it to hurt is dad. 438 | do it before you run out of time 439 | that girl doesn't need a man who is gon na betray her. 440 | yo what a black girl say bout this 441 | yo help me beat this chick at debate 442 | they're pretty much dudes the same age as you, mostly 443 | when are people ever going to understand? 444 | really, what am i gon na do? 445 | ya go to a bar and drink till your drunk! 446 | i acted that way with my current bf 447 | if you got ta ask... then no... 448 | don't sit around waiting till he realizes how great you are 449 | thought i divorced them but it still goes on.... 450 | she will just think you are an ass if you do that. 451 | you should try to date girls since guys offend you 452 | my ex of 3 years of being together cheated on me 453 | eh, i say give it 6 months - time will tell!! 454 | here is how the one i like went 455 | not all gentlemen break up 456 | its fun to go there. lots to do 457 | can u kiss on first date 458 | i have a close female friend 459 | the want to sex each other 460 | i am too nice according to my friends, and i should say no to him. 461 | theres alway san exception right? 462 | some guys do but not every single one. 463 | if you do that then you will be able to keep your last name. 464 | say hello and ask if she has a bf 465 | that turned me on, but i left 466 | fat girls can be hot 467 | all tha ppl here are not here for school. 468 | if u practice over & over u will be able to unwrap bothcherry stems and starbursts with ur tongue like me. 469 | if you gim me your vote for best answer, i will pay the bill. 470 | looking in public is prolly the best idea, plus you will feel safe there.. 471 | tease him see if he bites.... 472 | then i don't have to ask anyone direct. 473 | say yes it'll be fun! 474 | she never forgets when i screw up. 475 | how do i approach her so she doesn't think i'm a guy with bad intentions?? 476 | as a guy... let me enlighten you 477 | no prob, you ’ re just 13. 478 | trail ride with a friend every time all the time. 479 | if it ’ s meant to be they everything will work out. 480 | eye contact, it ’ s what it ’ s all abouut. 481 | he's a weakling, i can pick him up! 482 | he will have to start over to get your trust back. 483 | why would you wan na go out with her? 484 | hopefully you'll review this thoroughly. 485 | play t or d together. 486 | got ta confirm it and go from there. 487 | that's dumb u need a theory class 4 that. 488 | quit if you eat something like that 489 | i want some from that girl. 490 | be blunt and confront him, that's how you will know 491 | jamie's going to ask rubi out. 492 | both of you need to have some guts and end this thing 493 | could be interesting for them to learn about it... 494 | he dribbles he shoots he scores!!! two points! 495 | do guys really like big girls? 496 | he def loves you too, but time will tell... 497 | if he repsects you he'll get it 498 | we're the source of our own sin 499 | that response works for me 500 | u can still like him. but move on --------------------------------------------------------------------------------