├── README.md ├── classification_playground.ipynb ├── even_odd_test.py ├── experiments_log.json ├── generate_experiment.py ├── generators.py ├── iris_analysis.ipynb ├── iris_results ├── basic experiment, no preamble and bare numbers, random_state 88.json ├── basic experiment, no preamble and bare numbers, two orders of magnitutde bigger, random_state 88.json ├── basic experiment, no preamble, input-output terminology, random_state 88.json ├── basic experiment, no preamble, input-output terminology, random_state 91.json ├── basic experiment, no preamble, input-output terminology, random_state 93.json ├── basic experiment, no preamble, input-output terminology, random_state 95.json └── basic experiment, no preamble, input-output terminology, random_state 97.json ├── iris_test.py ├── number_sense_test.py ├── number_sense_test_spaced.py ├── plots ├── distribucije.png ├── razlike.png ├── reg1.png ├── reg2.png ├── reg3.png └── reg4.png ├── results ├── differences.csv ├── distribution_samples.csv ├── linear_model_1_input_15.csv ├── linear_model_2_input_15.csv ├── quadratic_model_1_input_25.csv └── quadratic_model_2_input_25_smaller_variance.csv ├── run_all_experiments.py ├── text_freq_classifier.py ├── utils.py └── visualizations.R /README.md: -------------------------------------------------------------------------------- 1 | # GPT-3 In-context numerical model fitting experiments 2 | 3 | This is repository for my experiments on GPT-3's ability to fit numerical models in-context. See the associated [Lesswrong post](https://www.lesswrong.com/posts/c2RzFadrxkzyRAFXa/who-models-the-models-that-model-models-an-exploration-of). 4 | 5 | Short descriptions of files in this repository: 6 | 7 | | Notebooks | | 8 | |---|---| 9 | | classification_playground.ipynb | Classification scenario plotting & Calculating accuracy | 10 | | iris_analysis.ipynb | Calculating accuracy of GPT-3 and kNN/log. reg. on Iris dataset | 11 | 12 | | Python scripts | | 13 | |---|---| 14 | | generators.py | Functions for generating classification/regression experiments | 15 | | generate_experiment.py | Script in which I called aforementioned functions | 16 | | run_all_experiments.py | Runs all not-yet-run experiments, saves their results | 17 | | iris_test.py | Performes test on the Iris dataset and saves results | 18 | | number_sense_test.py | Experiment in which letters replace numbers | 19 | | number_sense_test_spaced.py | Same as above, only with spaces between letters | 20 | | text_freq_classifier.py | Tests a hand-coded text frequency classifier | 21 | | even_odd_test.py | Test whether GPT-3 can learn that the second digit is even | 22 | | utils.py | Just a single utility function | 23 | 24 | | R script | | 25 | |---|---| 26 | | visualizations.R | Visualize stuff in results/ in ggplot2 | 27 | 28 | | Json | | 29 | |---|---| 30 | | experiments_log.json | Metadata, raw results of all experiments | 31 | -------------------------------------------------------------------------------- /even_odd_test.py: -------------------------------------------------------------------------------- 1 | """ 2 | Just a small script which takes the usual classification task of 2d points, 3 | but replaces labels with 1 if the first coordinate is even, 0 otherwise. 4 | 5 | No models "spot the pattern" here (see classification_playground.ipynb), 6 | and so they have random classifier's accuracy, ~50%. 7 | """ 8 | 9 | 10 | import json 11 | 12 | with open('experiments_log.json', 'r') as file: 13 | experiments = json.loads(file.read()) 14 | 15 | for rs in [42, 55, 93]: 16 | experiment = experiments[f'2d_class_type_1_rstate_{rs}'] 17 | 18 | new_experiment = dict() 19 | train_labels = [] 20 | for point in experiment['input_train']: 21 | x, y = point 22 | if x % 2 == 0: 23 | train_labels.append(1) 24 | else: 25 | train_labels.append(0) 26 | 27 | test_labels = [] 28 | for point in experiment['input_test']: 29 | x, y = point 30 | if x % 2 == 0: 31 | test_labels.append(1) 32 | else: 33 | test_labels.append(0) 34 | 35 | 36 | new_experiment['input_train'] = experiment['input_train'] 37 | new_experiment['output_train'] = train_labels 38 | new_experiment['input_test'] = experiment['input_test'] 39 | new_experiment['output_test'] = test_labels 40 | new_experiment['function'] = "1 if the first coordinate is even, 0 else" 41 | 42 | new_input_text = '' 43 | for ix, line in enumerate(experiment['input_text'].split('\n')): 44 | if line != '': 45 | new_input_text += (line[:-1] + str(train_labels[ix]) + '\n') 46 | 47 | new_experiment['input_text'] = new_input_text 48 | 49 | experiments[f'2d_class_type_1_evenodd_rstate_{rs}'] = new_experiment 50 | 51 | with open('experiments_log.json', 'w') as file: 52 | json.dump(experiments, file) 53 | -------------------------------------------------------------------------------- /generate_experiment.py: -------------------------------------------------------------------------------- 1 | """ 2 | A script which just calls generate_classification_experiment, in this case, 3 | and give it some parameters. 4 | 5 | Various other ways this could've been done, but this was the easiest one. 6 | I leave it here not because of its optimality, but just like, leaving 7 | a paper trail of how I did things, for transparency. 8 | """ 9 | 10 | 11 | import json 12 | from scipy.stats import norm, multivariate_normal 13 | import random 14 | import numpy as np 15 | from generators import ( 16 | generate_regression_experiment, 17 | generate_classification_experiment 18 | ) 19 | 20 | with open('experiments_log.json', 'r+') as file: 21 | file_contents = file.read() 22 | if len(file_contents) == 0: 23 | experiments = dict() 24 | else: 25 | experiments = json.loads(file_contents) 26 | 27 | 28 | if __name__=='__main__': 29 | rstate = 240 30 | name = f'2d_class_type_7_small_train_rstate_{rstate}' 31 | preamble = "" 32 | classes = [0, 1] 33 | def sampling_fn(class_num, size): 34 | var = 200 35 | rstate = 240 36 | if class_num == 0: 37 | mixture1 = np.concatenate(( 38 | multivariate_normal.rvs( 39 | [25, 25], 40 | [[var, 0], [0, var]], 41 | size=size, 42 | random_state=rstate 43 | ), 44 | multivariate_normal.rvs( 45 | [75, 75], 46 | [[var, 0], [0, var]], 47 | size=size, 48 | random_state=rstate+1 49 | ), 50 | )) 51 | 52 | indices = np.random.choice(mixture1.shape[0], 25, replace=False) 53 | mixture_sample1 = mixture1[indices] 54 | round_vec = np.vectorize(round) 55 | sample = round_vec(mixture_sample1) 56 | return [x.tolist() for x in sample] 57 | 58 | 59 | if class_num == 1: 60 | mixture2 = np.concatenate(( 61 | multivariate_normal.rvs( 62 | [75, 25], 63 | [[var, 0], [0, var]], 64 | size=size, 65 | random_state=rstate+2 66 | ), 67 | multivariate_normal.rvs( 68 | [25, 75], 69 | [[var, 0], [0, var]], 70 | size=size, 71 | random_state=rstate+3 72 | ), 73 | )) 74 | 75 | indices = np.random.choice(mixture2.shape[0], 25, replace=False) 76 | mixture_sample2 = mixture2[indices] 77 | round_vec = np.vectorize(round) 78 | sample = round_vec(mixture_sample2) 79 | return [x.tolist() for x in sample] 80 | 81 | generate_classification_experiment( 82 | experiments_dict=experiments, 83 | name=name, 84 | preamble=preamble, 85 | sampling_fn=sampling_fn, 86 | num_train=16, 87 | num_test=32, 88 | classes=[0,1] 89 | ) 90 | -------------------------------------------------------------------------------- /generators.py: -------------------------------------------------------------------------------- 1 | from utils import textify_numbers 2 | import json 3 | import random 4 | from inspect import getsource 5 | 6 | def generate_regression_experiment(experiments_dict, name, preamble, 7 | function, input_train, input_test): 8 | """ 9 | Generate data for a regression experiment. 10 | 11 | Takes a dictionary in which to store a regression experiment; 12 | its name, preamble (text before the input vectors), regression 13 | function, train and test set (just the "x"s, "y"s are generated 14 | by the function). 15 | 16 | Note that I was calling this function with the full dictionary 17 | of previous experiments as the first argument. 18 | USING THIS FUNCTION IN A 'NAIVE' WAY MIGHT CAUSE LOSS OF DATA. 19 | """ 20 | if name in experiments_dict.keys(): 21 | raise Exception("An experiment with that name already exists!") 22 | 23 | outputs = [function(*point) if hasattr(point, '__iter__') 24 | else function(point) 25 | for point in input_train] 26 | 27 | input_text = preamble 28 | for x, y in zip(input_train, outputs): 29 | x = textify_numbers(x) 30 | y = textify_numbers(y) 31 | input_text += f"Input = {x}, output = {y}\n" 32 | 33 | experiments_dict[name] = { 34 | "input_train" : input_train, 35 | "output_train" : outputs, 36 | "input_text" : input_text, 37 | "function" : getsource(function), 38 | "input_test" : input_test 39 | } 40 | 41 | with open('experiments_log.json', 'w') as file: 42 | json.dump(experiments_dict, file) 43 | 44 | 45 | def generate_classification_experiment(experiments_dict, name, preamble, 46 | sampling_fn, num_train, num_test, 47 | classes): 48 | """ 49 | Generate data for a classification experiment. 50 | 51 | Takes a dictionary in which to store a classification 52 | experiment; its name, preamble (text before the input 53 | vectors), sampling function, number of samples in train 54 | and test dataset, as well as a list of classes. 55 | 56 | Note that I was calling this function with the full dictionary 57 | of previous experiments as the first argument. 58 | USING THIS FUNCTION IN A 'NAIVE' WAY MIGHT CAUSE LOSS OF DATA. 59 | """ 60 | num_classes = len(classes) 61 | if (num_train % num_classes != 0) or (num_test % num_classes != 0): 62 | raise Exception("Number of classes has to divide train/test sizes.") 63 | 64 | classes_cardinality = (num_train + num_test) // num_classes 65 | x = [] 66 | y = [] 67 | for class_ in classes: 68 | x += sampling_fn(class_num=class_, 69 | size=classes_cardinality) 70 | y += [class_]*(classes_cardinality) 71 | 72 | xy = list(zip(x, y)) 73 | random.shuffle(xy) 74 | x, y = zip(*xy) 75 | 76 | x_train, y_train = x[:num_train], y[:num_train] 77 | x_test, y_test = x[num_train:], y[num_train:] 78 | 79 | 80 | input_text = preamble 81 | for x, y in zip(x_train, y_train): 82 | x = textify_numbers(x) 83 | y = textify_numbers(y) 84 | input_text += f"Input = {x}, output = {y}\n" 85 | 86 | experiments_dict[name] = { 87 | "input_train" : x_train, 88 | "output_train" : y_train, 89 | "input_text" : input_text, 90 | "function" : getsource(sampling_fn), 91 | "input_test" : x_test, 92 | "output_test" : y_test 93 | } 94 | 95 | #print(experiments_dict[name]) 96 | with open('experiments_log.json', 'w') as file: 97 | json.dump(experiments_dict, file) 98 | 99 | -------------------------------------------------------------------------------- /iris_analysis.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "This notebook loads the results of various Iris dataset experiments, calculates GPT-3 and kNN/logistic regression accuracies and compares them." 7 | ], 8 | "metadata": {} 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 11, 13 | "source": [ 14 | "import json\n", 15 | "from sklearn import datasets\n", 16 | "import numpy as np\n", 17 | "from sklearn.model_selection import train_test_split\n", 18 | "from sklearn.neighbors import KNeighborsClassifier\n", 19 | "from sklearn.linear_model import LogisticRegression\n", 20 | "from sklearn.preprocessing import StandardScaler\n", 21 | "from collections import defaultdict\n", 22 | "\n", 23 | "engines = ['ada', 'babbage', 'curie', 'davinci']\n", 24 | "\n", 25 | "iris = datasets.load_iris()\n", 26 | "\n", 27 | "transformed = 14*iris['data'] + 6\n", 28 | "transformed = np.vectorize(round)(transformed)\n", 29 | "y = iris.target" 30 | ], 31 | "outputs": [], 32 | "metadata": {} 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "source": [ 37 | "# Basic experiment" 38 | ], 39 | "metadata": {} 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "source": [ 45 | "random_states = [88, 91, 93, 95, 97]\n", 46 | "scores = defaultdict(list)\n", 47 | "for state in random_states:\n", 48 | " with open(f'iris_results/basic experiment, no preamble, input-output'\n", 49 | " f' terminology, random_state {state}.json', 'r') as file:\n", 50 | "\n", 51 | " results = json.loads(file.read())\n", 52 | "\n", 53 | " for k in [3,5,7]:\n", 54 | " x_train, x_test, y_train, y_test = train_test_split(transformed, y,\n", 55 | " test_size=0.5, stratify=y,\n", 56 | " random_state=state)\n", 57 | "\n", 58 | " neigh = KNeighborsClassifier(n_neighbors=k)\n", 59 | " neigh.fit(x_train, y_train.reshape(-1, 1))\n", 60 | "\n", 61 | " scores[f'knn_{k}'].append(neigh.score(x_test, y_test.reshape(-1, 1)))\n", 62 | "\n", 63 | " sc = StandardScaler()\n", 64 | " x_train = sc.fit_transform(x_train)\n", 65 | " x_test = sc.transform(x_test)\n", 66 | "\n", 67 | " classifier = LogisticRegression(random_state = 0, solver='lbfgs',\n", 68 | " multi_class='auto')\n", 69 | " classifier.fit(x_train, y_train)\n", 70 | "\n", 71 | " scores['lr'].append(classifier.score(x_test, y_test.reshape(-1, 1)))\n", 72 | "\n", 73 | "\n", 74 | " y_test = [int(x) for x in list(y_test)]\n", 75 | "\n", 76 | " for engine in engines:\n", 77 | " accurate = [1 if x==y else 0\n", 78 | " for x, y in zip(results[engine]['gpt_classification'],\n", 79 | " y_test)]\n", 80 | " scores[engine].append(sum(accurate)/len(accurate))\n" 81 | ], 82 | "outputs": [], 83 | "metadata": {} 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": 17, 88 | "source": [ 89 | "for key in scores.keys():\n", 90 | " print(key, np.mean(scores[key]))" 91 | ], 92 | "outputs": [ 93 | { 94 | "output_type": "stream", 95 | "name": "stdout", 96 | "text": [ 97 | "knn_3 0.9653333333333334\n", 98 | "knn_5 0.9573333333333333\n", 99 | "knn_7 0.96\n", 100 | "lr 0.9626666666666667\n", 101 | "ada 0.8986666666666666\n", 102 | "babbage 0.9306666666666666\n", 103 | "curie 0.952\n", 104 | "davinci 0.9573333333333334\n" 105 | ] 106 | } 107 | ], 108 | "metadata": {} 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": 25, 113 | "source": [ 114 | "neigh = KNeighborsClassifier(n_neighbors=3)\n", 115 | "neigh.fit(x_train, y_train.reshape(-1, 1))\n", 116 | "\n", 117 | "print(neigh.score(x_test, y_test.reshape(-1, 1)))\n" 118 | ], 119 | "outputs": [ 120 | { 121 | "output_type": "stream", 122 | "name": "stdout", 123 | "text": [ 124 | "0.9733333333333334\n" 125 | ] 126 | }, 127 | { 128 | "output_type": "stream", 129 | "name": "stderr", 130 | "text": [ 131 | ":2: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", 132 | " neigh.fit(x_train, y_train.reshape(-1, 1))\n" 133 | ] 134 | } 135 | ], 136 | "metadata": {} 137 | }, 138 | { 139 | "cell_type": "code", 140 | "execution_count": 28, 141 | "source": [ 142 | "sc = StandardScaler()\n", 143 | "x_train = sc.fit_transform(x_train)\n", 144 | "x_test = sc.transform(x_test)\n", 145 | "\n", 146 | "classifier = LogisticRegression(random_state = 0, solver='lbfgs', multi_class='auto')\n", 147 | "classifier.fit(x_train, y_train)\n", 148 | "\n", 149 | "classifier.score(x_test, y_test.reshape(-1, 1))" 150 | ], 151 | "outputs": [ 152 | { 153 | "output_type": "execute_result", 154 | "data": { 155 | "text/plain": [ 156 | "0.9733333333333334" 157 | ] 158 | }, 159 | "metadata": {}, 160 | "execution_count": 28 161 | } 162 | ], 163 | "metadata": {} 164 | }, 165 | { 166 | "cell_type": "code", 167 | "execution_count": 5, 168 | "source": [ 169 | "y_test = [int(x) for x in list(y_test)]\n" 170 | ], 171 | "outputs": [], 172 | "metadata": {} 173 | }, 174 | { 175 | "cell_type": "code", 176 | "execution_count": 22, 177 | "source": [ 178 | "for engine in engines:\n", 179 | " accurate = [1 if x==y else 0\n", 180 | " for x, y in zip(results[engine]['gpt_classification'],\n", 181 | " y_test)]\n", 182 | " print(engine, sum(accurate)/len(accurate))" 183 | ], 184 | "outputs": [ 185 | { 186 | "output_type": "stream", 187 | "name": "stdout", 188 | "text": [ 189 | "ada 0.9066666666666666\n", 190 | "babbage 0.9333333333333333\n", 191 | "curie 0.9333333333333333\n", 192 | "davinci 0.9733333333333334\n" 193 | ] 194 | } 195 | ], 196 | "metadata": {} 197 | }, 198 | { 199 | "cell_type": "markdown", 200 | "source": [ 201 | "## Basic experiment, no preamble and bare numbers, random_state 88" 202 | ], 203 | "metadata": {} 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": 6, 208 | "source": [ 209 | "with open('iris_results/basic experiment, no preamble and bare numbers, random_state 88.json', 'r') as file:\n", 210 | " results = json.loads(file.read())\n", 211 | "\n", 212 | "for engine in engines:\n", 213 | " accurate = [1 if x==y else 0\n", 214 | " for x, y in zip(results[engine]['gpt_classification'],\n", 215 | " y_test)]\n", 216 | " print(engine, sum(accurate)/len(accurate))" 217 | ], 218 | "outputs": [ 219 | { 220 | "output_type": "stream", 221 | "name": "stdout", 222 | "text": [ 223 | "ada 0.8933333333333333\n", 224 | "babbage 0.92\n", 225 | "curie 0.92\n", 226 | "davinci 0.9733333333333334\n" 227 | ] 228 | } 229 | ], 230 | "metadata": {} 231 | }, 232 | { 233 | "cell_type": "markdown", 234 | "source": [ 235 | "## Basic experiment, no preamble and bare numbers, two orders of magnitutde bigger, random_state 88" 236 | ], 237 | "metadata": {} 238 | }, 239 | { 240 | "cell_type": "code", 241 | "execution_count": 7, 242 | "source": [ 243 | "with open('iris_results/basic experiment, no preamble and bare numbers, two orders of magnitutde bigger, random_state 88.json', 'r') as file:\n", 244 | " results = json.loads(file.read())\n", 245 | "\n", 246 | "for engine in engines:\n", 247 | " accurate = [1 if x==y else 0\n", 248 | " for x, y in zip(results[engine]['gpt_classification'],\n", 249 | " y_test)]\n", 250 | " print(engine, sum(accurate)/len(accurate))" 251 | ], 252 | "outputs": [ 253 | { 254 | "output_type": "stream", 255 | "name": "stdout", 256 | "text": [ 257 | "ada 0.84\n", 258 | "babbage 0.8666666666666667\n", 259 | "curie 0.9733333333333334\n", 260 | "davinci 0.9066666666666666\n" 261 | ] 262 | } 263 | ], 264 | "metadata": {} 265 | }, 266 | { 267 | "cell_type": "code", 268 | "execution_count": null, 269 | "source": [], 270 | "outputs": [], 271 | "metadata": {} 272 | } 273 | ], 274 | "metadata": { 275 | "orig_nbformat": 4, 276 | "language_info": { 277 | "name": "python", 278 | "version": "3.8.5", 279 | "mimetype": "text/x-python", 280 | "codemirror_mode": { 281 | "name": "ipython", 282 | "version": 3 283 | }, 284 | "pygments_lexer": "ipython3", 285 | "nbconvert_exporter": "python", 286 | "file_extension": ".py" 287 | }, 288 | "kernelspec": { 289 | "name": "python3", 290 | "display_name": "Python 3.8.5 64-bit ('gptregression': conda)" 291 | }, 292 | "interpreter": { 293 | "hash": "71f0e3461ddbb8ca261c39df64ca4abf4ebaaadbbf981c0cdb0728086fb719e5" 294 | } 295 | }, 296 | "nbformat": 4, 297 | "nbformat_minor": 2 298 | } -------------------------------------------------------------------------------- /iris_results/basic experiment, no preamble and bare numbers, random_state 88.json: -------------------------------------------------------------------------------- 1 | {"input_template": "94, 47, 84, 31, 2\n89, 51, 73, 31, 1\n91, 48, 75, 31, 2\n96, 51, 80, 38, 2\n90, 37, 76, 27, 2\n91, 47, 72, 26, 1\n73, 48, 26, 7, 0\n87, 62, 23, 9, 0\n90, 54, 69, 28, 1\n77, 58, 27, 12, 0\n97, 48, 79, 34, 2\n100, 52, 86, 35, 2\n103, 49, 82, 35, 2\n94, 52, 72, 28, 1\n77, 59, 27, 10, 0\n94, 38, 68, 24, 1\n86, 47, 65, 24, 1\n98, 47, 70, 24, 1\n103, 49, 75, 27, 1\n83, 42, 68, 23, 1\n86, 48, 65, 23, 1\n87, 44, 77, 33, 2\n76, 34, 55, 20, 1\n77, 52, 30, 13, 0\n97, 51, 77, 34, 2\n90, 37, 62, 20, 1\n84, 45, 75, 34, 2\n68, 48, 24, 9, 0\n112, 48, 98, 35, 2\n76, 52, 26, 9, 0\n87, 44, 61, 23, 1\n86, 42, 55, 20, 1\n100, 52, 86, 41, 2\n91, 45, 72, 23, 1\n77, 59, 28, 9, 0\n79, 54, 26, 9, 0\n91, 42, 84, 26, 2\n100, 49, 84, 40, 2\n80, 58, 27, 9, 0\n79, 55, 27, 9, 0\n82, 54, 27, 12, 0\n83, 40, 58, 20, 1\n101, 45, 73, 26, 1\n84, 41, 61, 21, 1\n76, 51, 23, 9, 0\n75, 49, 27, 7, 0\n77, 55, 26, 10, 0\n70, 51, 26, 9, 0\n83, 38, 62, 24, 1\n94, 44, 75, 31, 2\n86, 59, 30, 10, 0\n83, 55, 24, 9, 0\n100, 48, 79, 38, 2\n114, 42, 103, 38, 2\n90, 44, 77, 28, 1\n94, 41, 76, 33, 2\n93, 54, 82, 38, 2\n86, 45, 63, 24, 1\n87, 45, 77, 40, 2\n69, 38, 24, 10, 0\n68, 47, 26, 9, 0\n79, 44, 61, 26, 1\n76, 38, 52, 20, 1\n77, 54, 27, 9, 0\n94, 45, 77, 27, 2\n96, 49, 83, 31, 2\n114, 45, 100, 34, 2\n86, 45, 69, 24, 1\n72, 51, 24, 9, 0\n76, 55, 24, 10, 0\n77, 55, 26, 9, 0\n73, 48, 26, 10, 0\n117, 59, 96, 34, 2\n83, 41, 62, 24, 1\n89, 48, 77, 31, 2\n", "x_train": [[94, 47, 84, 31], [89, 51, 73, 31], [91, 48, 75, 31], [96, 51, 80, 38], [90, 37, 76, 27], [91, 47, 72, 26], [73, 48, 26, 7], [87, 62, 23, 9], [90, 54, 69, 28], [77, 58, 27, 12], [97, 48, 79, 34], [100, 52, 86, 35], [103, 49, 82, 35], [94, 52, 72, 28], [77, 59, 27, 10], [94, 38, 68, 24], [86, 47, 65, 24], [98, 47, 70, 24], [103, 49, 75, 27], [83, 42, 68, 23], [86, 48, 65, 23], [87, 44, 77, 33], [76, 34, 55, 20], [77, 52, 30, 13], [97, 51, 77, 34], [90, 37, 62, 20], [84, 45, 75, 34], [68, 48, 24, 9], [112, 48, 98, 35], [76, 52, 26, 9], [87, 44, 61, 23], [86, 42, 55, 20], [100, 52, 86, 41], [91, 45, 72, 23], [77, 59, 28, 9], [79, 54, 26, 9], [91, 42, 84, 26], [100, 49, 84, 40], [80, 58, 27, 9], [79, 55, 27, 9], [82, 54, 27, 12], [83, 40, 58, 20], [101, 45, 73, 26], [84, 41, 61, 21], [76, 51, 23, 9], [75, 49, 27, 7], [77, 55, 26, 10], [70, 51, 26, 9], [83, 38, 62, 24], [94, 44, 75, 31], [86, 59, 30, 10], [83, 55, 24, 9], [100, 48, 79, 38], [114, 42, 103, 38], [90, 44, 77, 28], [94, 41, 76, 33], [93, 54, 82, 38], [86, 45, 63, 24], [87, 45, 77, 40], [69, 38, 24, 10], [68, 47, 26, 9], [79, 44, 61, 26], [76, 38, 52, 20], [77, 54, 27, 9], [94, 45, 77, 27], [96, 49, 83, 31], [114, 45, 100, 34], [86, 45, 69, 24], [72, 51, 24, 9], [76, 55, 24, 10], [77, 55, 26, 9], [73, 48, 26, 10], [117, 59, 96, 34], [83, 41, 62, 24], [89, 48, 77, 31]], "y_train": [2, 1, 2, 2, 2, 1, 0, 0, 1, 0, 2, 2, 2, 1, 0, 1, 1, 1, 1, 1, 1, 2, 1, 0, 2, 1, 2, 0, 2, 0, 1, 1, 2, 1, 0, 0, 2, 2, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 2, 0, 0, 2, 2, 1, 2, 2, 1, 2, 0, 0, 1, 1, 0, 2, 2, 2, 1, 0, 0, 0, 0, 2, 1, 2], "x_test": [[86, 68, 27, 12], [72, 51, 28, 9], [82, 54, 30, 9], [105, 48, 89, 35], [83, 65, 26, 9], [76, 48, 28, 9], [77, 59, 33, 12], [96, 47, 66, 24], [89, 48, 65, 27], [70, 49, 27, 9], [107, 51, 90, 31], [94, 52, 90, 41], [103, 51, 86, 38], [84, 44, 65, 24], [107, 48, 87, 28], [84, 47, 56, 24], [82, 48, 69, 27], [101, 48, 83, 35], [90, 48, 73, 31], [82, 61, 30, 12], [97, 48, 83, 31], [76, 54, 27, 9], [75, 40, 52, 20], [84, 48, 63, 24], [114, 48, 91, 38], [96, 44, 80, 33], [73, 54, 33, 9], [87, 42, 62, 23], [91, 48, 70, 26], [114, 59, 100, 37], [93, 45, 73, 31], [103, 49, 77, 38], [87, 44, 77, 33], [76, 55, 28, 14], [73, 49, 28, 9], [75, 56, 26, 7], [96, 45, 84, 35], [82, 58, 27, 9], [68, 51, 24, 9], [107, 56, 91, 41], [100, 41, 87, 31], [94, 54, 84, 40], [96, 45, 84, 37], [97, 45, 70, 27], [97, 48, 87, 37], [82, 61, 24, 12], [79, 63, 27, 7], [75, 41, 69, 30], [70, 54, 26, 10], [87, 44, 63, 20], [110, 45, 91, 33], [96, 51, 69, 27], [73, 54, 28, 9], [83, 40, 59, 21], [100, 48, 76, 30], [94, 41, 75, 27], [104, 51, 72, 26], [90, 47, 69, 27], [100, 49, 68, 26], [93, 37, 69, 27], [93, 47, 66, 24], [84, 48, 69, 27], [98, 48, 68, 26], [86, 41, 76, 34], [91, 45, 62, 24], [77, 41, 48, 21], [75, 49, 27, 9], [76, 54, 28, 12], [75, 48, 26, 9], [101, 51, 89, 38], [66, 48, 21, 7], [76, 56, 26, 9], [100, 49, 72, 27], [108, 47, 94, 31], [70, 56, 20, 9]], "y_test": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 2, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 1, 2, 0, 0, 2, 0, 1, 2, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0], "ada": {"gpt_output_raw": [" 0\n87", " 0\n77", " 0\n87", " 2\n87", " 0\n87", " 0\n87", " 0\n87", " 1\n87", " 1\n87", " 0\n87", " 2\n87", " 2\n87", " 2\n87", " 1\n87", " 2\n87", " 1\n87", " 1\n87", " 2\n87", " 2\n89", " 0\n87", " 2\n87", " 0\n87", " 1\n77", " 1\n87", " 2\n87", " 2\n87", " 0\n87", " 1\n87", " 1\n87", " 2\n87", " 2\n87", " 2\n87", " 2\n87", " 0\n87", " 0\n87", " 0\n87", " 2\n87", " 0\n87", " 0\n87", " 2\n87", " 2\n84", " 2\n87", " 2\n87", " 2\n87", " 2\n87", " 0\n87", " 0\n87", " 1\n87", " 0\n87", " 1\n87", " 2\n87", " 2\n87", " 0\n87", " 1\n87", " 2\n89", " 2\n87", " 1\n87", " 2\n91", " 1\n84", " 2\n87", " 1\n87", " 2\n87", " 1\n87", " 2\n87", " 1\n87", " 1\n87", " 0\n87", " 0\n87", " 0\n87", " 2\n87", " 0\n87", " 0\n87", " 1\n84", " 2\n87", " 0\n87"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 2, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 2, 2, 0, 0, 1, 0, 1, 2, 2, 0, 1, 2, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}, "babbage": {"gpt_output_raw": [" 0\n86", " 0\n86", " 0\n83", " 2\n83", " 0\n100", " 0\n83", " 0\n83", " 1\n90", " 1\n86", " 0\n83", " 2\n83", " 2\n86", " 2\n90", " 1\n86", " 1\n83", " 1\n86", " 1\n86", " 2\n83", " 2\n86", " 0\n86", " 2\n100", " 0\n86", " 1\n83", " 1\n86", " 2\n83", " 2\n90", " 0\n86", " 1\n86", " 1\n83", " 2\n83", " 2\n86", " 2\n86", " 2\n86", " 0\n83", " 0\n86", " 0\n83", " 2\n90", " 0\n100", " 0\n83", " 2\n83", " 2\n83", " 2\n86", " 2\n90", " 1\n83", " 2\n100", " 0\n86", " 0\n83", " 1\n83", " 0\n83", " 1\n86", " 2\n83", " 2\n90", " 0\n86", " 1\n86", " 2\n83", " 2\n86", " 1\n83", " 1\n83", " 1\n83", " 2\n86", " 1\n86", " 1\n86", " 1\n100", " 2\n86", " 1\n83", " 1\n83", " 0\n83", " 0\n83", " 0\n83", " 2\n83", " 0\n83", " 0\n86", " 1\n83", " 2\n83", " 0\n83"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 1, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 1, 2, 0, 0, 1, 0, 1, 2, 2, 0, 1, 2, 2, 1, 1, 1, 2, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}, "curie": {"gpt_output_raw": [" 0\n86", " 0\n76", " 0\n100", " 2\n100", " 0\n100", " 0\n76", " 0\n76", " 1\n100", " 2\n100", " 0\n76", " 2\n100", " 2\n100", " 2\n100", " 1\n76", " 1\n100", " 1\n76", " 2\n76", " 2\n100", " 2\n100", " 0\n100", " 2\n100", " 0\n76", " 1\n76", " 1\n76", " 2\n100", " 2\n100", " 0\n76", " 1\n76", " 1\n76", " 2\n100", " 2\n100", " 2\n100", " 2\n100", " 0\n76", " 0\n76", " 0\n76", " 2\n100", " 0\n100", " 0\n76", " 2\n100", " 2\n100", " 2\n100", " 2\n100", " 1\n100", " 2\n100", " 0\n100", " 0\n76", " 1\n76", " 0\n76", " 1\n76", " 2\n100", " 1\n100", " 0\n76", " 1\n76", " 2\n100", " 1\n100", " 1\n100", " 1\n76", " 1\n100", " 2\n100", " 1\n100", " 1\n76", " 1\n100", " 2\n86", " 1\n76", " 1\n76", " 0\n76", " 0\n76", " 0\n76", " 2\n100", " 0\n76", " 0\n76", " 1\n100", " 2\n100", " 0\n76"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 2, 0, 2, 2, 2, 1, 1, 1, 2, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 1, 2, 0, 0, 1, 0, 1, 2, 1, 0, 1, 2, 1, 1, 1, 1, 2, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}, "davinci": {"gpt_output_raw": [" 0\n77", " 0\n77", " 0\n84", " 2\n96", " 0\n77", " 0\n77", " 0\n77", " 1\n100", " 1\n96", " 0\n77", " 2\n91", " 2\n100", " 2\n96", " 1\n84", " 1\n96", " 1\n77", " 1\n77", " 2\n96", " 2\n91", " 0\n77", " 2\n100", " 0\n77", " 1\n77", " 1\n84", " 2\n83", " 2\n84", " 0\n77", " 1\n96", " 1\n77", " 2\n83", " 2\n96", " 2\n114", " 2\n76", " 0\n77", " 0\n77", " 0\n77", " 2\n100", " 0\n77", " 0\n77", " 2\n91", " 2\n100", " 2\n96", " 2\n100", " 1\n100", " 2\n100", " 0\n77", " 0\n77", " 1\n77", " 0\n77", " 1\n84", " 2\n96", " 1\n77", " 0\n77", " 1\n77", " 1\n100", " 1\n96", " 1\n77", " 1\n91", " 1\n100", " 1\n96", " 1\n96", " 1\n77", " 1\n100", " 2\n77", " 1\n84", " 1\n77", " 0\n77", " 0\n77", " 0\n77", " 2\n84", " 0\n77", " 0\n77", " 1\n100", " 2\n91", " 0\n77"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 1, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 1, 2, 0, 0, 1, 0, 1, 2, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}} -------------------------------------------------------------------------------- /iris_results/basic experiment, no preamble and bare numbers, two orders of magnitutde bigger, random_state 88.json: -------------------------------------------------------------------------------- 1 | {"input_template": "962, 479, 862, 323, 2\n905, 521, 749, 323, 1\n933, 493, 763, 323, 2\n976, 521, 820, 394, 2\n919, 379, 777, 280, 2\n933, 479, 734, 266, 1\n749, 493, 266, 81, 0\n891, 635, 237, 95, 0\n919, 550, 706, 294, 1\n791, 592, 280, 124, 0\n990, 493, 805, 351, 2\n1018, 536, 876, 365, 2\n1047, 507, 834, 365, 2\n962, 536, 734, 294, 1\n791, 607, 280, 110, 0\n962, 394, 692, 252, 1\n876, 479, 663, 252, 1\n1004, 479, 720, 252, 1\n1047, 507, 763, 280, 1\n848, 436, 692, 237, 1\n876, 493, 663, 237, 1\n891, 450, 791, 337, 2\n777, 351, 564, 209, 1\n791, 536, 308, 138, 0\n990, 521, 791, 351, 2\n919, 379, 635, 209, 1\n862, 465, 763, 351, 2\n692, 493, 252, 95, 0\n1146, 493, 1004, 365, 2\n777, 536, 266, 95, 0\n891, 450, 621, 237, 1\n876, 436, 564, 209, 1\n1018, 536, 876, 422, 2\n933, 465, 734, 237, 1\n791, 607, 294, 95, 0\n805, 550, 266, 95, 0\n933, 436, 862, 266, 2\n1018, 507, 862, 408, 2\n820, 592, 280, 95, 0\n805, 564, 280, 95, 0\n834, 550, 280, 124, 0\n848, 408, 592, 209, 1\n1033, 465, 749, 266, 1\n862, 422, 621, 223, 1\n777, 521, 237, 95, 0\n763, 507, 280, 81, 0\n791, 564, 266, 110, 0\n720, 521, 266, 95, 0\n848, 394, 635, 252, 1\n962, 450, 763, 323, 2\n876, 607, 308, 110, 0\n848, 564, 252, 95, 0\n1018, 493, 805, 394, 2\n1160, 436, 1047, 394, 2\n919, 450, 791, 294, 1\n962, 422, 777, 337, 2\n947, 550, 834, 394, 2\n876, 465, 649, 252, 1\n891, 465, 791, 408, 2\n706, 394, 252, 110, 0\n692, 479, 266, 95, 0\n805, 450, 621, 266, 1\n777, 394, 536, 209, 1\n791, 550, 280, 95, 0\n962, 465, 791, 280, 2\n976, 507, 848, 323, 2\n1160, 465, 1018, 351, 2\n876, 465, 706, 252, 1\n734, 521, 252, 95, 0\n777, 564, 252, 110, 0\n791, 564, 266, 95, 0\n749, 493, 266, 110, 0\n1189, 607, 976, 351, 2\n848, 422, 635, 252, 1\n905, 493, 791, 323, 2\n", "x_train": [[962, 479, 862, 323], [905, 521, 749, 323], [933, 493, 763, 323], [976, 521, 820, 394], [919, 379, 777, 280], [933, 479, 734, 266], [749, 493, 266, 81], [891, 635, 237, 95], [919, 550, 706, 294], [791, 592, 280, 124], [990, 493, 805, 351], [1018, 536, 876, 365], [1047, 507, 834, 365], [962, 536, 734, 294], [791, 607, 280, 110], [962, 394, 692, 252], [876, 479, 663, 252], [1004, 479, 720, 252], [1047, 507, 763, 280], [848, 436, 692, 237], [876, 493, 663, 237], [891, 450, 791, 337], [777, 351, 564, 209], [791, 536, 308, 138], [990, 521, 791, 351], [919, 379, 635, 209], [862, 465, 763, 351], [692, 493, 252, 95], [1146, 493, 1004, 365], [777, 536, 266, 95], [891, 450, 621, 237], [876, 436, 564, 209], [1018, 536, 876, 422], [933, 465, 734, 237], [791, 607, 294, 95], [805, 550, 266, 95], [933, 436, 862, 266], [1018, 507, 862, 408], [820, 592, 280, 95], [805, 564, 280, 95], [834, 550, 280, 124], [848, 408, 592, 209], [1033, 465, 749, 266], [862, 422, 621, 223], [777, 521, 237, 95], [763, 507, 280, 81], [791, 564, 266, 110], [720, 521, 266, 95], [848, 394, 635, 252], [962, 450, 763, 323], [876, 607, 308, 110], [848, 564, 252, 95], [1018, 493, 805, 394], [1160, 436, 1047, 394], [919, 450, 791, 294], [962, 422, 777, 337], [947, 550, 834, 394], [876, 465, 649, 252], [891, 465, 791, 408], [706, 394, 252, 110], [692, 479, 266, 95], [805, 450, 621, 266], [777, 394, 536, 209], [791, 550, 280, 95], [962, 465, 791, 280], [976, 507, 848, 323], [1160, 465, 1018, 351], [876, 465, 706, 252], [734, 521, 252, 95], [777, 564, 252, 110], [791, 564, 266, 95], [749, 493, 266, 110], [1189, 607, 976, 351], [848, 422, 635, 252], [905, 493, 791, 323]], "y_train": [2, 1, 2, 2, 2, 1, 0, 0, 1, 0, 2, 2, 2, 1, 0, 1, 1, 1, 1, 1, 1, 2, 1, 0, 2, 1, 2, 0, 2, 0, 1, 1, 2, 1, 0, 0, 2, 2, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 2, 0, 0, 2, 2, 1, 2, 2, 1, 2, 0, 0, 1, 1, 0, 2, 2, 2, 1, 0, 0, 0, 0, 2, 1, 2], "x_test": [[876, 692, 280, 124], [734, 521, 294, 95], [834, 550, 308, 95], [1075, 493, 905, 365], [848, 663, 266, 95], [777, 493, 294, 95], [791, 607, 337, 124], [976, 479, 678, 252], [905, 493, 663, 280], [720, 507, 280, 95], [1089, 521, 919, 323], [962, 536, 919, 422], [1047, 521, 876, 394], [862, 450, 663, 252], [1089, 493, 891, 294], [862, 479, 578, 252], [834, 493, 706, 280], [1033, 493, 848, 365], [919, 493, 749, 323], [834, 621, 308, 124], [990, 493, 848, 323], [777, 550, 280, 95], [763, 408, 536, 209], [862, 493, 649, 252], [1160, 493, 933, 394], [976, 450, 820, 337], [749, 550, 337, 95], [891, 436, 635, 237], [933, 493, 720, 266], [1160, 607, 1018, 379], [947, 465, 749, 323], [1047, 507, 791, 394], [891, 450, 791, 337], [777, 564, 294, 152], [749, 507, 294, 95], [763, 578, 266, 81], [976, 465, 862, 365], [834, 592, 280, 95], [692, 521, 252, 95], [1089, 578, 933, 422], [1018, 422, 891, 323], [962, 550, 862, 408], [976, 465, 862, 379], [990, 465, 720, 280], [990, 493, 891, 379], [834, 621, 252, 124], [805, 649, 280, 81], [763, 422, 706, 308], [720, 550, 266, 110], [891, 450, 649, 209], [1118, 465, 933, 337], [976, 521, 706, 280], [749, 550, 294, 95], [848, 408, 607, 223], [1018, 493, 777, 308], [962, 422, 763, 280], [1061, 521, 734, 266], [919, 479, 706, 280], [1018, 507, 692, 266], [947, 379, 706, 280], [947, 479, 678, 252], [862, 493, 706, 280], [1004, 493, 692, 266], [876, 422, 777, 351], [933, 465, 635, 252], [791, 422, 493, 223], [763, 507, 280, 95], [777, 550, 294, 124], [763, 493, 266, 95], [1033, 521, 905, 394], [678, 493, 223, 81], [777, 578, 266, 95], [1018, 507, 734, 280], [1104, 479, 962, 323], [720, 578, 209, 95]], "y_test": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 2, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 1, 2, 0, 0, 2, 0, 1, 2, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0], "ada": {"gpt_output_raw": [" 0\n8", " 0\n8", " 0\n8", " 2\n8", " 0\n8", " 0\n8", " 0\n8", " 1\n8", " 2\n8", " 0\n8", " 2\n8", " 2\n8", " 2\n8", " 1\n8", " 1\n8", " 1\n8", " 1\n8", " 2\n8", " 2\n9", " 0\n8", " 2\n9", " 0\n8", " 1\n8", " 1\n8", " 2\n8", " 2\n8", " 0\n8", " 1\n8", " 2\n9", " 2\n8", " 2\n8", " 2\n8", " 2\n8", " 0\n8", " 0\n8", " 0\n8", " 2\n8", " 0\n8", " 0\n8", " 2\n8", " 2\n8", " 2\n8", " 2\n8", " 2\n8", " 2\n8", " 0\n8", " 0\n8", " 110, 0", " 0\n8", " 1\n8", " 2\n8", " 2\n8", " 0\n8", " 1\n8", " 2\n8", " 2\n8", " 1\n8", " 2\n9", " 2\n8", " 1\n8", " 1\n8", " 2\n8", " 2\n8", " 2\n8", " 1\n8", " 1\n8", " 0\n8", " 0\n8", " 0\n8", " 2\n8", " 0\n8", " 0\n8", " 1\n8", " 2\n8", " 0\n8"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 2, 0, 2, 2, 2, 1, 1, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 2, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 2, 2, 0, 0, 110, 0, 1, 2, 2, 0, 1, 2, 2, 1, 2, 2, 1, 1, 2, 2, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}, "babbage": {"gpt_output_raw": [" 0\n8", " 0\n8", " 0\n8", " 2\n10", " 0\n8", " 0\n8", " 0\n8", " 1\n8", " 2\n8", " 0\n8", " 2\n10", " 2\n8", " 2\n10", " 1\n8", " 1\n10", " 1\n8", " 2\n8", " 2\n10", " 2\n8", " 0\n8", " 2\n10", " 0\n8", " 1\n8", " 1\n8", " 2\n10", " 2\n10", " 0\n8", " 1\n8", " 1\n8", " 2\n10", " 2\n8", " 2\n10", " 2\n8", " 0\n8", " 0\n8", " 0\n8", " 2\n10", " 0\n8", " 0\n8", " 2\n10", " 2\n10", " 2\n8", " 2\n10", " 2\n10", " 2\n10", " 0\n8", " 0\n8", " 1\n8", " 0\n8", " 1\n8", " 2\n10", " 2\n10", " 0\n8", " 1\n8", " 1\n10", " 2\n8", " 1\n8", " 2\n8", " 1\n10", " 1\n8", " 1\n8", " 2\n8", " 1\n10", " 2\n8", " 1\n8", " 1\n8", " 0\n8", " 0\n8", " 0\n8", " 2\n10", " 0\n8", " 0\n8", " 2\n10", " 2\n10", " 0\n8"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 2, 0, 2, 2, 2, 1, 1, 1, 2, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 2, 2, 0, 0, 1, 0, 1, 2, 2, 0, 1, 1, 2, 1, 2, 1, 1, 1, 2, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 2, 2, 0]}, "curie": {"gpt_output_raw": [" 0\n10", " 0\n8", " 0\n10", " 2\n10", " 0\n10", " 0\n10", " 0\n96", " 1\n10", " 1\n10", " 0\n8", " 2\n10", " 2\n10", " 2\n10", " 1\n10", " 1\n10", " 1\n10", " 1\n10", " 2\n10", " 2\n10", " 0\n10", " 2\n10", " 0\n10", " 1\n10", " 1\n10", " 2\n10", " 2\n10", " 0\n8", " 1\n10", " 1\n10", " 2\n10", " 2\n10", " 2\n10", " 2\n10", " 0\n10", " 0\n8", " 0\n10", " 2\n10", " 0\n10", " 0\n10", " 2\n10", " 2\n10", " 2\n10", " 2\n10", " 1\n10", " 2\n10", " 0\n10", " 0\n10", " 1\n96", " 0\n8", " 1\n96", " 2\n10", " 1\n10", " 0\n8", " 1\n10", " 1\n10", " 1\n10", " 1\n10", " 1\n10", " 1\n10", " 1\n10", " 1\n10", " 1\n10", " 1\n10", " 2\n10", " 1\n10", " 1\n10", " 0\n8", " 0\n10", " 0\n10", " 2\n10", " 0\n10", " 0\n10", " 1\n10", " 2\n10", " 0\n8"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 1, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 1, 2, 0, 0, 1, 0, 1, 2, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}, "davinci": {"gpt_output_raw": [" 0\n9", " 0\n8", " 0\n9", " 2\n9", " 0\n8", " 0\n8", " 0\n8", " 1\n9", " 1\n9", " 0\n8", " 2\n9", " 2\n9", " 2\n9", " 1\n9", " 1\n9", " 1\n9", " 1\n9", " 2\n9", " 2\n9", " 0\n8", " 2\n9", " 0\n8", " 1\n8", " 1\n9", " 2\n9", " 2\n9", " 0\n8", " 1\n8", " 1\n9", " 2\n9", " 2\n9", " 2\n9", " 2\n8", " 0\n8", " 0\n8", " 0\n8", " 2\n9", " 0\n8", " 0\n8", " 2\n9", " 2\n9", " 2\n9", " 2\n9", " 2\n9", " 2\n9", " 0\n8", " 0\n9", " 1\n8", " 0\n8", " 1\n8", " 2\n9", " 1\n9", " 0\n8", " 1\n8", " 2\n9", " 2\n9", " 1\n9", " 2\n9", " 1\n8", " 2\n9", " 1\n9", " 1\n9", " 1\n9", " 2\n8", " 1\n9", " 1\n8", " 0\n8", " 0\n8", " 0\n8", " 2\n9", " 0\n8", " 0\n8", " 1\n8", " 2\n9", " 0\n8"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 1, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 2, 2, 0, 0, 1, 0, 1, 2, 1, 0, 1, 2, 2, 1, 2, 1, 2, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}} -------------------------------------------------------------------------------- /iris_results/basic experiment, no preamble, input-output terminology, random_state 88.json: -------------------------------------------------------------------------------- 1 | {"input_template": "Input: 94, 47, 84, 31, output = 2\nInput: 89, 51, 73, 31, output = 1\nInput: 91, 48, 75, 31, output = 2\nInput: 96, 51, 80, 38, output = 2\nInput: 90, 37, 76, 27, output = 2\nInput: 91, 47, 72, 26, output = 1\nInput: 73, 48, 26, 7, output = 0\nInput: 87, 62, 23, 9, output = 0\nInput: 90, 54, 69, 28, output = 1\nInput: 77, 58, 27, 12, output = 0\nInput: 97, 48, 79, 34, output = 2\nInput: 100, 52, 86, 35, output = 2\nInput: 103, 49, 82, 35, output = 2\nInput: 94, 52, 72, 28, output = 1\nInput: 77, 59, 27, 10, output = 0\nInput: 94, 38, 68, 24, output = 1\nInput: 86, 47, 65, 24, output = 1\nInput: 98, 47, 70, 24, output = 1\nInput: 103, 49, 75, 27, output = 1\nInput: 83, 42, 68, 23, output = 1\nInput: 86, 48, 65, 23, output = 1\nInput: 87, 44, 77, 33, output = 2\nInput: 76, 34, 55, 20, output = 1\nInput: 77, 52, 30, 13, output = 0\nInput: 97, 51, 77, 34, output = 2\nInput: 90, 37, 62, 20, output = 1\nInput: 84, 45, 75, 34, output = 2\nInput: 68, 48, 24, 9, output = 0\nInput: 112, 48, 98, 35, output = 2\nInput: 76, 52, 26, 9, output = 0\nInput: 87, 44, 61, 23, output = 1\nInput: 86, 42, 55, 20, output = 1\nInput: 100, 52, 86, 41, output = 2\nInput: 91, 45, 72, 23, output = 1\nInput: 77, 59, 28, 9, output = 0\nInput: 79, 54, 26, 9, output = 0\nInput: 91, 42, 84, 26, output = 2\nInput: 100, 49, 84, 40, output = 2\nInput: 80, 58, 27, 9, output = 0\nInput: 79, 55, 27, 9, output = 0\nInput: 82, 54, 27, 12, output = 0\nInput: 83, 40, 58, 20, output = 1\nInput: 101, 45, 73, 26, output = 1\nInput: 84, 41, 61, 21, output = 1\nInput: 76, 51, 23, 9, output = 0\nInput: 75, 49, 27, 7, output = 0\nInput: 77, 55, 26, 10, output = 0\nInput: 70, 51, 26, 9, output = 0\nInput: 83, 38, 62, 24, output = 1\nInput: 94, 44, 75, 31, output = 2\nInput: 86, 59, 30, 10, output = 0\nInput: 83, 55, 24, 9, output = 0\nInput: 100, 48, 79, 38, output = 2\nInput: 114, 42, 103, 38, output = 2\nInput: 90, 44, 77, 28, output = 1\nInput: 94, 41, 76, 33, output = 2\nInput: 93, 54, 82, 38, output = 2\nInput: 86, 45, 63, 24, output = 1\nInput: 87, 45, 77, 40, output = 2\nInput: 69, 38, 24, 10, output = 0\nInput: 68, 47, 26, 9, output = 0\nInput: 79, 44, 61, 26, output = 1\nInput: 76, 38, 52, 20, output = 1\nInput: 77, 54, 27, 9, output = 0\nInput: 94, 45, 77, 27, output = 2\nInput: 96, 49, 83, 31, output = 2\nInput: 114, 45, 100, 34, output = 2\nInput: 86, 45, 69, 24, output = 1\nInput: 72, 51, 24, 9, output = 0\nInput: 76, 55, 24, 10, output = 0\nInput: 77, 55, 26, 9, output = 0\nInput: 73, 48, 26, 10, output = 0\nInput: 117, 59, 96, 34, output = 2\nInput: 83, 41, 62, 24, output = 1\nInput: 89, 48, 77, 31, output = 2\n", "x_train": [[94, 47, 84, 31], [89, 51, 73, 31], [91, 48, 75, 31], [96, 51, 80, 38], [90, 37, 76, 27], [91, 47, 72, 26], [73, 48, 26, 7], [87, 62, 23, 9], [90, 54, 69, 28], [77, 58, 27, 12], [97, 48, 79, 34], [100, 52, 86, 35], [103, 49, 82, 35], [94, 52, 72, 28], [77, 59, 27, 10], [94, 38, 68, 24], [86, 47, 65, 24], [98, 47, 70, 24], [103, 49, 75, 27], [83, 42, 68, 23], [86, 48, 65, 23], [87, 44, 77, 33], [76, 34, 55, 20], [77, 52, 30, 13], [97, 51, 77, 34], [90, 37, 62, 20], [84, 45, 75, 34], [68, 48, 24, 9], [112, 48, 98, 35], [76, 52, 26, 9], [87, 44, 61, 23], [86, 42, 55, 20], [100, 52, 86, 41], [91, 45, 72, 23], [77, 59, 28, 9], [79, 54, 26, 9], [91, 42, 84, 26], [100, 49, 84, 40], [80, 58, 27, 9], [79, 55, 27, 9], [82, 54, 27, 12], [83, 40, 58, 20], [101, 45, 73, 26], [84, 41, 61, 21], [76, 51, 23, 9], [75, 49, 27, 7], [77, 55, 26, 10], [70, 51, 26, 9], [83, 38, 62, 24], [94, 44, 75, 31], [86, 59, 30, 10], [83, 55, 24, 9], [100, 48, 79, 38], [114, 42, 103, 38], [90, 44, 77, 28], [94, 41, 76, 33], [93, 54, 82, 38], [86, 45, 63, 24], [87, 45, 77, 40], [69, 38, 24, 10], [68, 47, 26, 9], [79, 44, 61, 26], [76, 38, 52, 20], [77, 54, 27, 9], [94, 45, 77, 27], [96, 49, 83, 31], [114, 45, 100, 34], [86, 45, 69, 24], [72, 51, 24, 9], [76, 55, 24, 10], [77, 55, 26, 9], [73, 48, 26, 10], [117, 59, 96, 34], [83, 41, 62, 24], [89, 48, 77, 31]], "y_train": [2, 1, 2, 2, 2, 1, 0, 0, 1, 0, 2, 2, 2, 1, 0, 1, 1, 1, 1, 1, 1, 2, 1, 0, 2, 1, 2, 0, 2, 0, 1, 1, 2, 1, 0, 0, 2, 2, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 2, 0, 0, 2, 2, 1, 2, 2, 1, 2, 0, 0, 1, 1, 0, 2, 2, 2, 1, 0, 0, 0, 0, 2, 1, 2], "x_test": [[86, 68, 27, 12], [72, 51, 28, 9], [82, 54, 30, 9], [105, 48, 89, 35], [83, 65, 26, 9], [76, 48, 28, 9], [77, 59, 33, 12], [96, 47, 66, 24], [89, 48, 65, 27], [70, 49, 27, 9], [107, 51, 90, 31], [94, 52, 90, 41], [103, 51, 86, 38], [84, 44, 65, 24], [107, 48, 87, 28], [84, 47, 56, 24], [82, 48, 69, 27], [101, 48, 83, 35], [90, 48, 73, 31], [82, 61, 30, 12], [97, 48, 83, 31], [76, 54, 27, 9], [75, 40, 52, 20], [84, 48, 63, 24], [114, 48, 91, 38], [96, 44, 80, 33], [73, 54, 33, 9], [87, 42, 62, 23], [91, 48, 70, 26], [114, 59, 100, 37], [93, 45, 73, 31], [103, 49, 77, 38], [87, 44, 77, 33], [76, 55, 28, 14], [73, 49, 28, 9], [75, 56, 26, 7], [96, 45, 84, 35], [82, 58, 27, 9], [68, 51, 24, 9], [107, 56, 91, 41], [100, 41, 87, 31], [94, 54, 84, 40], [96, 45, 84, 37], [97, 45, 70, 27], [97, 48, 87, 37], [82, 61, 24, 12], [79, 63, 27, 7], [75, 41, 69, 30], [70, 54, 26, 10], [87, 44, 63, 20], [110, 45, 91, 33], [96, 51, 69, 27], [73, 54, 28, 9], [83, 40, 59, 21], [100, 48, 76, 30], [94, 41, 75, 27], [104, 51, 72, 26], [90, 47, 69, 27], [100, 49, 68, 26], [93, 37, 69, 27], [93, 47, 66, 24], [84, 48, 69, 27], [98, 48, 68, 26], [86, 41, 76, 34], [91, 45, 62, 24], [77, 41, 48, 21], [75, 49, 27, 9], [76, 54, 28, 12], [75, 48, 26, 9], [101, 51, 89, 38], [66, 48, 21, 7], [76, 56, 26, 9], [100, 49, 72, 27], [108, 47, 94, 31], [70, 56, 20, 9]], "y_test": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 2, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 1, 2, 0, 0, 2, 0, 1, 2, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0], "ada": {"gpt_output_raw": [" 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 0\n", " 1\n", " 1\n", " 0\n", " 2\n", " 2\n", " 2\n", " 1\n", " 2\n", " 1\n", " 1\n", " 2\n", " 1\n", " 0\n", " 2\n", " 0\n", " 1\n", " 1\n", " 2\n", " 2\n", " 0\n", " 1\n", " 1\n", " 2\n", " 2\n", " 2\n", " 2\n", " 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 2\n", " 2\n", " 2\n", " 2\n", " 2\n", " 2\n", " 0\n", " 0\n", " 1\n", " 0\n", " 1\n", " 2\n", " 2\n", " 0\n", " 1\n", " 2\n", " 2\n", " 1\n", " 1\n", " 1\n", " 1\n", " 1\n", " 2\n", " 1\n", " 2\n", " 1\n", " 1\n", " 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 1\n", " 2\n", " 0\n"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 2, 1, 1, 2, 1, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 2, 2, 0, 0, 1, 0, 1, 2, 2, 0, 1, 2, 2, 1, 1, 1, 1, 1, 2, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}, "babbage": {"gpt_output_raw": [" 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 0\n", " 1\n", " 1\n", " 0\n", " 2\n", " 2\n", " 2\n", " 1\n", " 2\n", " 1\n", " 1\n", " 2\n", " 2\n", " 0\n", " 2\n", " 0\n", " 1\n", " 1\n", " 2\n", " 2\n", " 0\n", " 1\n", " 1\n", " 2\n", " 2\n", " 2\n", " 2\n", " 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 2\n", " 2\n", " 2\n", " 2\n", " 1\n", " 2\n", " 0\n", " 0\n", " 1\n", " 0\n", " 1\n", " 2\n", " 1\n", " 0\n", " 1\n", " 2\n", " 2\n", " 1\n", " 1\n", " 2\n", " 2\n", " 1\n", " 1\n", " 1\n", " 2\n", " 1\n", " 1\n", " 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 1\n", " 2\n", " 0\n"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 2, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 1, 2, 0, 0, 1, 0, 1, 2, 1, 0, 1, 2, 2, 1, 1, 2, 2, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}, "curie": {"gpt_output_raw": [" 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 0\n", " 1\n", " 1\n", " 0\n", " 2\n", " 2\n", " 2\n", " 1\n", " 2\n", " 1\n", " 1\n", " 2\n", " 2\n", " 0\n", " 2\n", " 0\n", " 1\n", " 1\n", " 2\n", " 2\n", " 0\n", " 1\n", " 1\n", " 2\n", " 2\n", " 2\n", " 2\n", " 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 2\n", " 2\n", " 2\n", " 2\n", " 2\n", " 2\n", " 0\n", " 0\n", " 1\n", " 0\n", " 1\n", " 2\n", " 2\n", " 0\n", " 1\n", " 2\n", " 2\n", " 1\n", " 1\n", " 1\n", " 1\n", " 1\n", " 1\n", " 1\n", " 2\n", " 1\n", " 1\n", " 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 1\n", " 2\n", " 0\n"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 2, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 2, 2, 0, 0, 1, 0, 1, 2, 2, 0, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}, "davinci": {"gpt_output_raw": [" 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 0\n", " 1\n", " 1\n", " 0\n", " 2\n", " 2\n", " 2\n", " 1\n", " 2\n", " 1\n", " 1\n", " 2\n", " 2\n", " 0\n", " 2\n", " 0\n", " 1\n", " 1\n", " 2\n", " 2\n", " 0\n", " 1\n", " 1\n", " 2\n", " 2\n", " 2\n", " 2\n", " 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 2\n", " 2\n", " 2\n", " 2\n", " 1\n", " 2\n", " 0\n", " 0\n", " 1\n", " 0\n", " 1\n", " 2\n", " 1\n", " 0\n", " 1\n", " 1\n", " 2\n", " 1\n", " 1\n", " 1\n", " 1\n", " 1\n", " 1\n", " 1\n", " 2\n", " 1\n", " 1\n", " 0\n", " 0\n", " 0\n", " 2\n", " 0\n", " 0\n", " 1\n", " 2\n", " 0\n"], "gpt_classification": [0, 0, 0, 2, 0, 0, 0, 1, 1, 0, 2, 2, 2, 1, 2, 1, 1, 2, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2, 2, 0, 0, 0, 2, 0, 0, 2, 2, 2, 2, 1, 2, 0, 0, 1, 0, 1, 2, 1, 0, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0]}} -------------------------------------------------------------------------------- /iris_results/basic experiment, no preamble, input-output terminology, random_state 91.json: -------------------------------------------------------------------------------- 1 | {"input_template": "Input = 66, 48, 21, 7, output = 0\nInput = 89, 51, 73, 31, output = 1\nInput = 77, 59, 27, 10, output = 0\nInput = 75, 41, 69, 30, output = 2\nInput = 97, 51, 77, 34, output = 2\nInput = 79, 55, 27, 9, output = 0\nInput = 86, 47, 65, 24, output = 1\nInput = 75, 40, 52, 20, output = 1\nInput = 76, 51, 23, 9, output = 0\nInput = 93, 47, 66, 24, output = 1\nInput = 77, 55, 26, 9, output = 0\nInput = 82, 61, 24, 12, output = 0\nInput = 96, 51, 80, 38, output = 2\nInput = 70, 54, 26, 10, output = 0\nInput = 100, 49, 84, 40, output = 2\nInput = 90, 37, 62, 20, output = 1\nInput = 77, 58, 27, 12, output = 0\nInput = 94, 44, 75, 31, output = 2\nInput = 87, 44, 63, 20, output = 1\nInput = 86, 45, 69, 24, output = 1\nInput = 76, 54, 28, 12, output = 0\nInput = 100, 52, 86, 41, output = 2\nInput = 77, 54, 27, 9, output = 0\nInput = 97, 48, 87, 37, output = 2\nInput = 75, 49, 27, 7, output = 0\nInput = 90, 44, 77, 28, output = 1\nInput = 76, 38, 52, 20, output = 1\nInput = 103, 49, 82, 35, output = 2\nInput = 103, 49, 77, 38, output = 2\nInput = 79, 63, 27, 7, output = 0\nInput = 83, 42, 68, 23, output = 1\nInput = 110, 45, 91, 33, output = 2\nInput = 91, 47, 72, 26, output = 1\nInput = 94, 47, 84, 31, output = 2\nInput = 91, 45, 62, 24, output = 1\nInput = 86, 68, 27, 12, output = 0\nInput = 114, 48, 91, 38, output = 2\nInput = 80, 58, 27, 9, output = 0\nInput = 94, 52, 90, 41, output = 2\nInput = 96, 51, 69, 27, output = 1\nInput = 98, 48, 68, 26, output = 1\nInput = 100, 52, 86, 35, output = 2\nInput = 90, 37, 76, 27, output = 2\nInput = 101, 51, 89, 38, output = 2\nInput = 82, 54, 27, 12, output = 0\nInput = 75, 49, 27, 9, output = 0\nInput = 103, 51, 86, 38, output = 2\nInput = 93, 45, 73, 31, output = 2\nInput = 87, 45, 77, 40, output = 2\nInput = 84, 44, 65, 24, output = 1\nInput = 83, 38, 62, 24, output = 1\nInput = 94, 41, 75, 27, output = 1\nInput = 86, 48, 65, 23, output = 1\nInput = 82, 61, 30, 12, output = 0\nInput = 84, 48, 63, 24, output = 1\nInput = 89, 48, 65, 27, output = 1\nInput = 70, 49, 27, 9, output = 0\nInput = 76, 54, 27, 9, output = 0\nInput = 76, 48, 28, 9, output = 0\nInput = 94, 54, 84, 40, output = 2\nInput = 84, 47, 56, 24, output = 1\nInput = 94, 41, 76, 33, output = 2\nInput = 96, 47, 66, 24, output = 1\nInput = 107, 56, 91, 41, output = 2\nInput = 73, 48, 26, 7, output = 0\nInput = 89, 48, 77, 31, output = 2\nInput = 79, 44, 61, 26, output = 1\nInput = 86, 45, 63, 24, output = 1\nInput = 87, 44, 77, 33, output = 2\nInput = 114, 59, 100, 37, output = 2\nInput = 77, 52, 30, 13, output = 0\nInput = 101, 45, 73, 26, output = 1\nInput = 76, 56, 26, 9, output = 0\nInput = 68, 47, 26, 9, output = 0\nInput = 77, 59, 33, 12, output = 0\n", "x_train": [[66, 48, 21, 7], [89, 51, 73, 31], [77, 59, 27, 10], [75, 41, 69, 30], [97, 51, 77, 34], [79, 55, 27, 9], [86, 47, 65, 24], [75, 40, 52, 20], [76, 51, 23, 9], [93, 47, 66, 24], [77, 55, 26, 9], [82, 61, 24, 12], [96, 51, 80, 38], [70, 54, 26, 10], [100, 49, 84, 40], [90, 37, 62, 20], [77, 58, 27, 12], [94, 44, 75, 31], [87, 44, 63, 20], [86, 45, 69, 24], [76, 54, 28, 12], [100, 52, 86, 41], [77, 54, 27, 9], [97, 48, 87, 37], [75, 49, 27, 7], [90, 44, 77, 28], [76, 38, 52, 20], [103, 49, 82, 35], [103, 49, 77, 38], [79, 63, 27, 7], [83, 42, 68, 23], [110, 45, 91, 33], [91, 47, 72, 26], [94, 47, 84, 31], [91, 45, 62, 24], [86, 68, 27, 12], [114, 48, 91, 38], [80, 58, 27, 9], [94, 52, 90, 41], [96, 51, 69, 27], [98, 48, 68, 26], [100, 52, 86, 35], [90, 37, 76, 27], [101, 51, 89, 38], [82, 54, 27, 12], [75, 49, 27, 9], [103, 51, 86, 38], [93, 45, 73, 31], [87, 45, 77, 40], [84, 44, 65, 24], [83, 38, 62, 24], [94, 41, 75, 27], [86, 48, 65, 23], [82, 61, 30, 12], [84, 48, 63, 24], [89, 48, 65, 27], [70, 49, 27, 9], [76, 54, 27, 9], [76, 48, 28, 9], [94, 54, 84, 40], [84, 47, 56, 24], [94, 41, 76, 33], [96, 47, 66, 24], [107, 56, 91, 41], [73, 48, 26, 7], [89, 48, 77, 31], [79, 44, 61, 26], [86, 45, 63, 24], [87, 44, 77, 33], [114, 59, 100, 37], [77, 52, 30, 13], [101, 45, 73, 26], [76, 56, 26, 9], [68, 47, 26, 9], [77, 59, 33, 12]], "y_train": [0, 1, 0, 2, 2, 0, 1, 1, 0, 1, 0, 0, 2, 0, 2, 1, 0, 2, 1, 1, 0, 2, 0, 2, 0, 1, 1, 2, 2, 0, 1, 2, 1, 2, 1, 0, 2, 0, 2, 1, 1, 2, 2, 2, 0, 0, 2, 2, 2, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 2, 1, 2, 1, 2, 0, 2, 1, 1, 2, 2, 0, 1, 0, 0, 0], "x_test": [[100, 49, 72, 27], [82, 48, 69, 27], [107, 51, 90, 31], [94, 52, 72, 28], [97, 48, 79, 34], [72, 51, 24, 9], [91, 45, 72, 23], [82, 58, 27, 9], [73, 54, 33, 9], [68, 51, 24, 9], [90, 48, 73, 31], [114, 45, 100, 34], [86, 42, 55, 20], [93, 37, 69, 27], [90, 54, 69, 28], [97, 48, 83, 31], [70, 51, 26, 9], [86, 41, 76, 34], [76, 55, 28, 14], [93, 54, 82, 38], [91, 48, 70, 26], [103, 49, 75, 27], [77, 55, 26, 10], [83, 40, 58, 20], [87, 44, 77, 33], [91, 42, 84, 26], [87, 62, 23, 9], [75, 48, 26, 9], [83, 65, 26, 9], [76, 34, 55, 20], [84, 48, 69, 27], [96, 45, 84, 37], [77, 59, 28, 9], [100, 41, 87, 31], [83, 41, 62, 24], [69, 38, 24, 10], [83, 40, 59, 21], [87, 42, 62, 23], [73, 48, 26, 10], [108, 47, 94, 31], [100, 49, 68, 26], [105, 48, 89, 35], [79, 54, 26, 9], [107, 48, 87, 28], [87, 44, 61, 23], [114, 42, 103, 38], [96, 49, 83, 31], [101, 48, 83, 35], [104, 51, 72, 26], [96, 44, 80, 33], [91, 48, 75, 31], [82, 54, 30, 9], [83, 55, 24, 9], [90, 47, 69, 27], [94, 45, 77, 27], [112, 48, 98, 35], [72, 51, 28, 9], [84, 41, 61, 21], [73, 54, 28, 9], [70, 56, 20, 9], [73, 49, 28, 9], [100, 48, 76, 30], [76, 52, 26, 9], [97, 45, 70, 27], [68, 48, 24, 9], [76, 55, 24, 10], [75, 56, 26, 7], [86, 59, 30, 10], [94, 38, 68, 24], [96, 45, 84, 35], [84, 45, 75, 34], [117, 59, 96, 34], [77, 41, 48, 21], [98, 47, 70, 24], [100, 48, 79, 38]], "y_test": [1, 1, 2, 1, 2, 0, 1, 0, 0, 0, 2, 2, 1, 1, 1, 2, 0, 2, 0, 2, 1, 1, 0, 1, 2, 2, 0, 0, 0, 1, 1, 2, 0, 2, 1, 0, 1, 1, 0, 2, 1, 2, 0, 2, 1, 2, 2, 2, 1, 2, 2, 0, 0, 1, 2, 2, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 2, 2, 2, 1, 1, 2], "ada": {"gpt_output_raw": [" 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput ="], "gpt_classification": [2, 1, 2, 2, 2, 0, 1, 0, 0, 0, 2, 2, 1, 1, 1, 2, 0, 2, 0, 2, 1, 2, 0, 1, 2, 1, 1, 0, 0, 1, 1, 2, 0, 2, 1, 0, 1, 1, 0, 2, 1, 2, 0, 2, 1, 2, 2, 2, 2, 2, 2, 0, 0, 1, 1, 2, 0, 1, 0, 0, 0, 2, 0, 1, 0, 0, 0, 0, 1, 2, 2, 2, 1, 1, 2]}, "babbage": {"gpt_output_raw": [" 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput ="], "gpt_classification": [2, 1, 2, 2, 2, 0, 1, 0, 0, 0, 2, 2, 1, 1, 1, 2, 0, 2, 0, 2, 1, 2, 0, 1, 2, 1, 0, 0, 0, 1, 1, 2, 0, 2, 1, 0, 1, 1, 0, 2, 1, 2, 0, 1, 1, 2, 2, 2, 1, 2, 2, 0, 0, 1, 1, 2, 0, 1, 0, 0, 0, 2, 0, 1, 0, 0, 0, 0, 1, 2, 2, 2, 1, 1, 2]}, "curie": {"gpt_output_raw": [" 1\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput ="], "gpt_classification": [1, 1, 2, 1, 2, 0, 1, 0, 0, 0, 2, 2, 1, 1, 1, 2, 0, 2, 0, 2, 1, 1, 0, 1, 2, 1, 0, 0, 0, 1, 1, 2, 0, 2, 1, 0, 1, 1, 0, 2, 1, 2, 0, 2, 1, 2, 2, 2, 1, 2, 2, 0, 0, 1, 1, 2, 0, 1, 0, 0, 0, 2, 0, 1, 0, 0, 0, 0, 1, 2, 2, 2, 1, 1, 2]}, "davinci": {"gpt_output_raw": [" 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput ="], "gpt_classification": [1, 1, 2, 2, 2, 0, 1, 0, 0, 0, 2, 2, 1, 2, 1, 2, 0, 2, 0, 2, 1, 1, 0, 1, 2, 1, 0, 0, 0, 1, 1, 2, 0, 2, 1, 0, 1, 1, 0, 2, 1, 2, 0, 2, 1, 2, 2, 2, 1, 2, 2, 0, 0, 1, 1, 2, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 2, 2, 2, 1, 1, 2]}} -------------------------------------------------------------------------------- /iris_results/basic experiment, no preamble, input-output terminology, random_state 93.json: -------------------------------------------------------------------------------- 1 | {"input_template": "Input = 90, 47, 69, 27, output = 1\nInput = 76, 48, 28, 9, output = 0\nInput = 79, 63, 27, 7, output = 0\nInput = 93, 47, 66, 24, output = 1\nInput = 107, 56, 91, 41, output = 2\nInput = 94, 41, 76, 33, output = 2\nInput = 94, 45, 77, 27, output = 2\nInput = 76, 51, 23, 9, output = 0\nInput = 91, 45, 72, 23, output = 1\nInput = 96, 44, 80, 33, output = 2\nInput = 77, 59, 27, 10, output = 0\nInput = 69, 38, 24, 10, output = 0\nInput = 76, 54, 28, 12, output = 0\nInput = 90, 54, 69, 28, output = 1\nInput = 103, 49, 82, 35, output = 2\nInput = 94, 52, 72, 28, output = 1\nInput = 76, 38, 52, 20, output = 1\nInput = 101, 51, 89, 38, output = 2\nInput = 72, 51, 28, 9, output = 0\nInput = 68, 48, 24, 9, output = 0\nInput = 86, 48, 65, 23, output = 1\nInput = 82, 54, 30, 9, output = 0\nInput = 87, 44, 77, 33, output = 2\nInput = 80, 58, 27, 9, output = 0\nInput = 76, 52, 26, 9, output = 0\nInput = 83, 40, 59, 21, output = 1\nInput = 70, 54, 26, 10, output = 0\nInput = 83, 42, 68, 23, output = 1\nInput = 83, 55, 24, 9, output = 0\nInput = 87, 44, 63, 20, output = 1\nInput = 107, 48, 87, 28, output = 2\nInput = 100, 48, 76, 30, output = 1\nInput = 68, 51, 24, 9, output = 0\nInput = 84, 48, 69, 27, output = 1\nInput = 100, 52, 86, 35, output = 2\nInput = 90, 44, 77, 28, output = 1\nInput = 114, 45, 100, 34, output = 2\nInput = 98, 48, 68, 26, output = 1\nInput = 87, 45, 77, 40, output = 2\nInput = 86, 45, 63, 24, output = 1\nInput = 70, 56, 20, 9, output = 0\nInput = 103, 49, 77, 38, output = 2\nInput = 91, 48, 75, 31, output = 2\nInput = 75, 48, 26, 9, output = 0\nInput = 76, 55, 24, 10, output = 0\nInput = 101, 45, 73, 26, output = 1\nInput = 84, 48, 63, 24, output = 1\nInput = 104, 51, 72, 26, output = 1\nInput = 86, 47, 65, 24, output = 1\nInput = 103, 51, 86, 38, output = 2\nInput = 76, 34, 55, 20, output = 1\nInput = 100, 48, 79, 38, output = 2\nInput = 114, 48, 91, 38, output = 2\nInput = 100, 49, 84, 40, output = 2\nInput = 96, 51, 69, 27, output = 1\nInput = 114, 59, 100, 37, output = 2\nInput = 79, 54, 26, 9, output = 0\nInput = 98, 47, 70, 24, output = 1\nInput = 97, 45, 70, 27, output = 1\nInput = 89, 51, 73, 31, output = 1\nInput = 73, 49, 28, 9, output = 0\nInput = 77, 52, 30, 13, output = 0\nInput = 96, 45, 84, 37, output = 2\nInput = 94, 47, 84, 31, output = 2\nInput = 77, 55, 26, 9, output = 0\nInput = 70, 51, 26, 9, output = 0\nInput = 70, 49, 27, 9, output = 0\nInput = 110, 45, 91, 33, output = 2\nInput = 91, 45, 62, 24, output = 1\nInput = 82, 54, 27, 12, output = 0\nInput = 97, 48, 79, 34, output = 2\nInput = 96, 51, 80, 38, output = 2\nInput = 96, 49, 83, 31, output = 2\nInput = 72, 51, 24, 9, output = 0\nInput = 87, 44, 77, 33, output = 2\n", "x_train": [[90, 47, 69, 27], [76, 48, 28, 9], [79, 63, 27, 7], [93, 47, 66, 24], [107, 56, 91, 41], [94, 41, 76, 33], [94, 45, 77, 27], [76, 51, 23, 9], [91, 45, 72, 23], [96, 44, 80, 33], [77, 59, 27, 10], [69, 38, 24, 10], [76, 54, 28, 12], [90, 54, 69, 28], [103, 49, 82, 35], [94, 52, 72, 28], [76, 38, 52, 20], [101, 51, 89, 38], [72, 51, 28, 9], [68, 48, 24, 9], [86, 48, 65, 23], [82, 54, 30, 9], [87, 44, 77, 33], [80, 58, 27, 9], [76, 52, 26, 9], [83, 40, 59, 21], [70, 54, 26, 10], [83, 42, 68, 23], [83, 55, 24, 9], [87, 44, 63, 20], [107, 48, 87, 28], [100, 48, 76, 30], [68, 51, 24, 9], [84, 48, 69, 27], [100, 52, 86, 35], [90, 44, 77, 28], [114, 45, 100, 34], [98, 48, 68, 26], [87, 45, 77, 40], [86, 45, 63, 24], [70, 56, 20, 9], [103, 49, 77, 38], [91, 48, 75, 31], [75, 48, 26, 9], [76, 55, 24, 10], [101, 45, 73, 26], [84, 48, 63, 24], [104, 51, 72, 26], [86, 47, 65, 24], [103, 51, 86, 38], [76, 34, 55, 20], [100, 48, 79, 38], [114, 48, 91, 38], [100, 49, 84, 40], [96, 51, 69, 27], [114, 59, 100, 37], [79, 54, 26, 9], [98, 47, 70, 24], [97, 45, 70, 27], [89, 51, 73, 31], [73, 49, 28, 9], [77, 52, 30, 13], [96, 45, 84, 37], [94, 47, 84, 31], [77, 55, 26, 9], [70, 51, 26, 9], [70, 49, 27, 9], [110, 45, 91, 33], [91, 45, 62, 24], [82, 54, 27, 12], [97, 48, 79, 34], [96, 51, 80, 38], [96, 49, 83, 31], [72, 51, 24, 9], [87, 44, 77, 33]], "y_train": [1, 0, 0, 1, 2, 2, 2, 0, 1, 2, 0, 0, 0, 1, 2, 1, 1, 2, 0, 0, 1, 0, 2, 0, 0, 1, 0, 1, 0, 1, 2, 1, 0, 1, 2, 1, 2, 1, 2, 1, 0, 2, 2, 0, 0, 1, 1, 1, 1, 2, 1, 2, 2, 2, 1, 2, 0, 1, 1, 1, 0, 0, 2, 2, 0, 0, 0, 2, 1, 0, 2, 2, 2, 0, 2], "x_test": [[93, 37, 69, 27], [89, 48, 77, 31], [87, 62, 23, 9], [94, 54, 84, 40], [82, 61, 24, 12], [76, 56, 26, 9], [77, 54, 27, 9], [68, 47, 26, 9], [84, 47, 56, 24], [96, 45, 84, 35], [117, 59, 96, 34], [84, 41, 61, 21], [79, 55, 27, 9], [97, 48, 83, 31], [94, 38, 68, 24], [77, 41, 48, 21], [84, 44, 65, 24], [86, 41, 76, 34], [79, 44, 61, 26], [97, 51, 77, 34], [75, 40, 52, 20], [112, 48, 98, 35], [100, 49, 72, 27], [91, 47, 72, 26], [90, 37, 62, 20], [82, 58, 27, 9], [83, 41, 62, 24], [89, 48, 65, 27], [77, 59, 28, 9], [66, 48, 21, 7], [94, 52, 90, 41], [107, 51, 90, 31], [108, 47, 94, 31], [75, 49, 27, 9], [86, 45, 69, 24], [100, 41, 87, 31], [73, 48, 26, 7], [82, 61, 30, 12], [97, 48, 87, 37], [87, 42, 62, 23], [114, 42, 103, 38], [100, 49, 68, 26], [83, 38, 62, 24], [103, 49, 75, 27], [101, 48, 83, 35], [73, 54, 33, 9], [82, 48, 69, 27], [91, 48, 70, 26], [94, 41, 75, 27], [76, 54, 27, 9], [100, 52, 86, 41], [94, 44, 75, 31], [93, 45, 73, 31], [77, 55, 26, 10], [76, 55, 28, 14], [77, 58, 27, 12], [73, 48, 26, 10], [93, 54, 82, 38], [84, 45, 75, 34], [90, 37, 76, 27], [83, 40, 58, 20], [86, 42, 55, 20], [90, 48, 73, 31], [75, 41, 69, 30], [75, 49, 27, 7], [83, 65, 26, 9], [96, 47, 66, 24], [73, 54, 28, 9], [91, 42, 84, 26], [77, 59, 33, 12], [75, 56, 26, 7], [105, 48, 89, 35], [86, 59, 30, 10], [86, 68, 27, 12], [87, 44, 61, 23]], "y_test": [1, 2, 0, 2, 0, 0, 0, 0, 1, 2, 2, 1, 0, 2, 1, 1, 1, 2, 1, 2, 1, 2, 1, 1, 1, 0, 1, 1, 0, 0, 2, 2, 2, 0, 1, 2, 0, 0, 2, 1, 2, 1, 1, 1, 2, 0, 1, 1, 1, 0, 2, 2, 2, 0, 0, 0, 0, 2, 2, 2, 1, 1, 2, 2, 0, 0, 1, 0, 2, 0, 0, 2, 0, 0, 1], "ada": {"gpt_output_raw": [" 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput ="], "gpt_classification": [1, 2, 0, 2, 0, 0, 0, 0, 1, 2, 2, 1, 0, 2, 1, 1, 1, 2, 1, 2, 1, 2, 1, 1, 1, 0, 1, 1, 0, 0, 1, 2, 2, 0, 1, 2, 0, 0, 2, 1, 2, 1, 1, 1, 2, 0, 1, 1, 2, 0, 2, 2, 2, 0, 0, 0, 0, 2, 2, 1, 1, 1, 2, 1, 0, 0, 1, 0, 1, 0, 0, 2, 0, 0, 1]}, "babbage": {"gpt_output_raw": [" 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput ="], "gpt_classification": [1, 2, 0, 2, 0, 0, 0, 0, 1, 2, 2, 1, 0, 2, 1, 1, 1, 2, 1, 2, 1, 2, 1, 1, 1, 0, 1, 1, 0, 0, 2, 2, 2, 0, 1, 2, 0, 0, 2, 1, 2, 1, 1, 1, 2, 0, 1, 1, 2, 0, 2, 2, 2, 0, 0, 0, 0, 2, 2, 1, 1, 1, 2, 1, 0, 0, 1, 0, 1, 0, 0, 2, 0, 0, 1]}, "curie": {"gpt_output_raw": [" 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput ="], "gpt_classification": [1, 1, 0, 2, 0, 0, 0, 0, 1, 2, 2, 1, 0, 2, 1, 1, 1, 2, 1, 2, 1, 2, 1, 1, 1, 0, 1, 1, 0, 0, 2, 2, 2, 0, 1, 2, 0, 0, 2, 1, 2, 1, 1, 1, 2, 0, 1, 1, 2, 0, 2, 2, 1, 0, 0, 0, 0, 2, 2, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 2, 0, 0, 1]}, "davinci": {"gpt_output_raw": [" 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput ="], "gpt_classification": [1, 2, 0, 2, 0, 0, 0, 0, 1, 2, 2, 1, 0, 2, 1, 1, 1, 2, 1, 2, 1, 2, 1, 1, 1, 0, 1, 1, 0, 0, 2, 2, 2, 0, 1, 2, 0, 0, 2, 1, 2, 1, 1, 1, 2, 0, 1, 1, 1, 0, 2, 2, 1, 0, 0, 0, 0, 2, 2, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 2, 0, 0, 1]}} -------------------------------------------------------------------------------- /iris_results/basic experiment, no preamble, input-output terminology, random_state 95.json: -------------------------------------------------------------------------------- 1 | {"input_template": "Input = 84, 45, 75, 34, output = 2\nInput = 77, 55, 26, 9, output = 0\nInput = 80, 58, 27, 9, output = 0\nInput = 76, 52, 26, 9, output = 0\nInput = 87, 44, 61, 23, output = 1\nInput = 68, 48, 24, 9, output = 0\nInput = 76, 54, 28, 12, output = 0\nInput = 89, 48, 65, 27, output = 1\nInput = 94, 45, 77, 27, output = 2\nInput = 83, 42, 68, 23, output = 1\nInput = 91, 45, 62, 24, output = 1\nInput = 98, 47, 70, 24, output = 1\nInput = 70, 54, 26, 10, output = 0\nInput = 73, 48, 26, 7, output = 0\nInput = 83, 41, 62, 24, output = 1\nInput = 97, 48, 79, 34, output = 2\nInput = 93, 47, 66, 24, output = 1\nInput = 93, 54, 82, 38, output = 2\nInput = 84, 47, 56, 24, output = 1\nInput = 73, 49, 28, 9, output = 0\nInput = 114, 42, 103, 38, output = 2\nInput = 87, 45, 77, 40, output = 2\nInput = 96, 51, 69, 27, output = 1\nInput = 96, 45, 84, 37, output = 2\nInput = 86, 59, 30, 10, output = 0\nInput = 86, 41, 76, 34, output = 2\nInput = 82, 61, 24, 12, output = 0\nInput = 114, 59, 100, 37, output = 2\nInput = 100, 49, 68, 26, output = 1\nInput = 76, 34, 55, 20, output = 1\nInput = 87, 62, 23, 9, output = 0\nInput = 101, 51, 89, 38, output = 2\nInput = 107, 51, 90, 31, output = 2\nInput = 77, 59, 28, 9, output = 0\nInput = 90, 37, 62, 20, output = 1\nInput = 75, 49, 27, 7, output = 0\nInput = 76, 38, 52, 20, output = 1\nInput = 86, 42, 55, 20, output = 1\nInput = 94, 54, 84, 40, output = 2\nInput = 96, 49, 83, 31, output = 2\nInput = 76, 48, 28, 9, output = 0\nInput = 86, 45, 69, 24, output = 1\nInput = 82, 58, 27, 9, output = 0\nInput = 76, 56, 26, 9, output = 0\nInput = 96, 44, 80, 33, output = 2\nInput = 83, 40, 59, 21, output = 1\nInput = 117, 59, 96, 34, output = 2\nInput = 86, 68, 27, 12, output = 0\nInput = 103, 49, 75, 27, output = 1\nInput = 84, 48, 69, 27, output = 1\nInput = 90, 48, 73, 31, output = 2\nInput = 87, 42, 62, 23, output = 1\nInput = 97, 51, 77, 34, output = 2\nInput = 101, 45, 73, 26, output = 1\nInput = 73, 54, 33, 9, output = 0\nInput = 75, 48, 26, 9, output = 0\nInput = 75, 49, 27, 9, output = 0\nInput = 107, 48, 87, 28, output = 2\nInput = 100, 52, 86, 41, output = 2\nInput = 96, 45, 84, 35, output = 2\nInput = 94, 52, 72, 28, output = 1\nInput = 79, 55, 27, 9, output = 0\nInput = 75, 41, 69, 30, output = 2\nInput = 66, 48, 21, 7, output = 0\nInput = 105, 48, 89, 35, output = 2\nInput = 79, 54, 26, 9, output = 0\nInput = 83, 38, 62, 24, output = 1\nInput = 94, 44, 75, 31, output = 2\nInput = 76, 55, 28, 14, output = 0\nInput = 112, 48, 98, 35, output = 2\nInput = 91, 47, 72, 26, output = 1\nInput = 79, 44, 61, 26, output = 1\nInput = 76, 55, 24, 10, output = 0\nInput = 97, 48, 83, 31, output = 2\nInput = 83, 40, 58, 20, output = 1\n", "x_train": [[84, 45, 75, 34], [77, 55, 26, 9], [80, 58, 27, 9], [76, 52, 26, 9], [87, 44, 61, 23], [68, 48, 24, 9], [76, 54, 28, 12], [89, 48, 65, 27], [94, 45, 77, 27], [83, 42, 68, 23], [91, 45, 62, 24], [98, 47, 70, 24], [70, 54, 26, 10], [73, 48, 26, 7], [83, 41, 62, 24], [97, 48, 79, 34], [93, 47, 66, 24], [93, 54, 82, 38], [84, 47, 56, 24], [73, 49, 28, 9], [114, 42, 103, 38], [87, 45, 77, 40], [96, 51, 69, 27], [96, 45, 84, 37], [86, 59, 30, 10], [86, 41, 76, 34], [82, 61, 24, 12], [114, 59, 100, 37], [100, 49, 68, 26], [76, 34, 55, 20], [87, 62, 23, 9], [101, 51, 89, 38], [107, 51, 90, 31], [77, 59, 28, 9], [90, 37, 62, 20], [75, 49, 27, 7], [76, 38, 52, 20], [86, 42, 55, 20], [94, 54, 84, 40], [96, 49, 83, 31], [76, 48, 28, 9], [86, 45, 69, 24], [82, 58, 27, 9], [76, 56, 26, 9], [96, 44, 80, 33], [83, 40, 59, 21], [117, 59, 96, 34], [86, 68, 27, 12], [103, 49, 75, 27], [84, 48, 69, 27], [90, 48, 73, 31], [87, 42, 62, 23], [97, 51, 77, 34], [101, 45, 73, 26], [73, 54, 33, 9], [75, 48, 26, 9], [75, 49, 27, 9], [107, 48, 87, 28], [100, 52, 86, 41], [96, 45, 84, 35], [94, 52, 72, 28], [79, 55, 27, 9], [75, 41, 69, 30], [66, 48, 21, 7], [105, 48, 89, 35], [79, 54, 26, 9], [83, 38, 62, 24], [94, 44, 75, 31], [76, 55, 28, 14], [112, 48, 98, 35], [91, 47, 72, 26], [79, 44, 61, 26], [76, 55, 24, 10], [97, 48, 83, 31], [83, 40, 58, 20]], "y_train": [2, 0, 0, 0, 1, 0, 0, 1, 2, 1, 1, 1, 0, 0, 1, 2, 1, 2, 1, 0, 2, 2, 1, 2, 0, 2, 0, 2, 1, 1, 0, 2, 2, 0, 1, 0, 1, 1, 2, 2, 0, 1, 0, 0, 2, 1, 2, 0, 1, 1, 2, 1, 2, 1, 0, 0, 0, 2, 2, 2, 1, 0, 2, 0, 2, 0, 1, 2, 0, 2, 1, 1, 0, 2, 1], "x_test": [[86, 47, 65, 24], [101, 48, 83, 35], [91, 48, 75, 31], [100, 49, 72, 27], [68, 47, 26, 9], [87, 44, 63, 20], [114, 48, 91, 38], [77, 59, 27, 10], [103, 51, 86, 38], [97, 45, 70, 27], [108, 47, 94, 31], [77, 54, 27, 9], [100, 48, 79, 38], [76, 51, 23, 9], [77, 55, 26, 10], [96, 51, 80, 38], [107, 56, 91, 41], [82, 54, 30, 9], [86, 48, 65, 23], [84, 44, 65, 24], [75, 40, 52, 20], [83, 65, 26, 9], [114, 45, 100, 34], [90, 47, 69, 27], [98, 48, 68, 26], [91, 45, 72, 23], [91, 48, 70, 26], [100, 41, 87, 31], [100, 52, 86, 35], [83, 55, 24, 9], [89, 51, 73, 31], [77, 52, 30, 13], [94, 41, 75, 27], [87, 44, 77, 33], [90, 44, 77, 28], [75, 56, 26, 7], [94, 38, 68, 24], [100, 48, 76, 30], [91, 42, 84, 26], [79, 63, 27, 7], [104, 51, 72, 26], [72, 51, 24, 9], [77, 58, 27, 12], [73, 48, 26, 10], [82, 54, 27, 12], [86, 45, 63, 24], [110, 45, 91, 33], [70, 56, 20, 9], [94, 41, 76, 33], [84, 41, 61, 21], [77, 59, 33, 12], [70, 51, 26, 9], [84, 48, 63, 24], [70, 49, 27, 9], [76, 54, 27, 9], [72, 51, 28, 9], [94, 47, 84, 31], [103, 49, 77, 38], [100, 49, 84, 40], [93, 37, 69, 27], [69, 38, 24, 10], [93, 45, 73, 31], [77, 41, 48, 21], [89, 48, 77, 31], [90, 37, 76, 27], [96, 47, 66, 24], [103, 49, 82, 35], [87, 44, 77, 33], [97, 48, 87, 37], [73, 54, 28, 9], [94, 52, 90, 41], [82, 48, 69, 27], [68, 51, 24, 9], [90, 54, 69, 28], [82, 61, 30, 12]], "y_test": [1, 2, 2, 1, 0, 1, 2, 0, 2, 1, 2, 0, 2, 0, 0, 2, 2, 0, 1, 1, 1, 0, 2, 1, 1, 1, 1, 2, 2, 0, 1, 0, 1, 2, 1, 0, 1, 1, 2, 0, 1, 0, 0, 0, 0, 1, 2, 0, 2, 1, 0, 0, 1, 0, 0, 0, 2, 2, 2, 1, 0, 2, 1, 2, 2, 1, 2, 2, 2, 0, 2, 1, 0, 1, 0], "ada": {"gpt_output_raw": [" 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput ="], "gpt_classification": [1, 2, 2, 2, 0, 1, 2, 0, 2, 1, 2, 0, 2, 0, 0, 2, 2, 0, 2, 2, 1, 0, 2, 1, 1, 2, 1, 2, 2, 0, 2, 0, 2, 2, 2, 0, 2, 2, 2, 0, 2, 0, 0, 0, 0, 1, 2, 0, 2, 1, 0, 0, 1, 0, 0, 0, 2, 2, 2, 2, 0, 2, 2, 2, 2, 1, 2, 2, 2, 0, 2, 1, 0, 2, 0]}, "babbage": {"gpt_output_raw": [" 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput ="], "gpt_classification": [1, 2, 2, 1, 0, 1, 2, 0, 2, 1, 2, 0, 2, 0, 0, 2, 2, 0, 1, 1, 1, 0, 2, 1, 1, 1, 1, 2, 2, 0, 2, 0, 2, 2, 2, 0, 1, 2, 1, 0, 1, 0, 0, 0, 0, 1, 2, 0, 2, 1, 0, 0, 1, 0, 0, 0, 2, 2, 2, 1, 0, 2, 1, 2, 1, 1, 2, 2, 2, 0, 2, 1, 0, 1, 0]}, "curie": {"gpt_output_raw": [" 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput ="], "gpt_classification": [1, 2, 2, 1, 0, 1, 2, 0, 2, 1, 2, 0, 2, 0, 0, 2, 2, 0, 1, 1, 1, 0, 2, 1, 1, 1, 1, 2, 2, 0, 2, 0, 2, 2, 1, 0, 1, 1, 2, 0, 1, 0, 0, 0, 0, 1, 2, 0, 2, 1, 0, 0, 1, 0, 0, 0, 2, 2, 2, 1, 0, 2, 1, 2, 1, 1, 2, 2, 2, 0, 2, 1, 0, 1, 0]}, "davinci": {"gpt_output_raw": [" 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput ="], "gpt_classification": [1, 2, 2, 1, 0, 1, 2, 0, 2, 1, 2, 0, 2, 0, 0, 2, 2, 0, 1, 1, 1, 0, 2, 1, 1, 1, 1, 2, 2, 0, 2, 0, 1, 2, 2, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 2, 0, 2, 1, 0, 0, 1, 0, 0, 0, 2, 2, 2, 1, 0, 2, 1, 2, 2, 1, 2, 2, 2, 0, 2, 1, 0, 2, 0]}} -------------------------------------------------------------------------------- /iris_results/basic experiment, no preamble, input-output terminology, random_state 97.json: -------------------------------------------------------------------------------- 1 | {"input_template": "Input = 83, 40, 58, 20, output = 1\nInput = 96, 45, 84, 35, output = 2\nInput = 83, 55, 24, 9, output = 0\nInput = 73, 54, 28, 9, output = 0\nInput = 94, 45, 77, 27, output = 2\nInput = 75, 49, 27, 9, output = 0\nInput = 75, 48, 26, 9, output = 0\nInput = 105, 48, 89, 35, output = 2\nInput = 100, 48, 76, 30, output = 1\nInput = 79, 55, 27, 9, output = 0\nInput = 91, 45, 72, 23, output = 1\nInput = 83, 42, 68, 23, output = 1\nInput = 94, 52, 72, 28, output = 1\nInput = 91, 48, 70, 26, output = 1\nInput = 87, 44, 61, 23, output = 1\nInput = 91, 42, 84, 26, output = 2\nInput = 114, 45, 100, 34, output = 2\nInput = 86, 47, 65, 24, output = 1\nInput = 87, 44, 77, 33, output = 2\nInput = 84, 41, 61, 21, output = 1\nInput = 107, 51, 90, 31, output = 2\nInput = 73, 49, 28, 9, output = 0\nInput = 103, 49, 82, 35, output = 2\nInput = 77, 59, 27, 10, output = 0\nInput = 96, 49, 83, 31, output = 2\nInput = 93, 54, 82, 38, output = 2\nInput = 77, 59, 28, 9, output = 0\nInput = 75, 49, 27, 7, output = 0\nInput = 76, 48, 28, 9, output = 0\nInput = 101, 48, 83, 35, output = 2\nInput = 83, 40, 59, 21, output = 1\nInput = 84, 48, 63, 24, output = 1\nInput = 77, 55, 26, 9, output = 0\nInput = 96, 45, 84, 37, output = 2\nInput = 70, 54, 26, 10, output = 0\nInput = 77, 58, 27, 12, output = 0\nInput = 73, 48, 26, 7, output = 0\nInput = 84, 45, 75, 34, output = 2\nInput = 100, 48, 79, 38, output = 2\nInput = 93, 47, 66, 24, output = 1\nInput = 94, 47, 84, 31, output = 2\nInput = 70, 56, 20, 9, output = 0\nInput = 77, 54, 27, 9, output = 0\nInput = 82, 48, 69, 27, output = 1\nInput = 89, 51, 73, 31, output = 1\nInput = 96, 51, 69, 27, output = 1\nInput = 101, 51, 89, 38, output = 2\nInput = 75, 41, 69, 30, output = 2\nInput = 86, 41, 76, 34, output = 2\nInput = 97, 51, 77, 34, output = 2\nInput = 100, 49, 68, 26, output = 1\nInput = 103, 51, 86, 38, output = 2\nInput = 117, 59, 96, 34, output = 2\nInput = 90, 54, 69, 28, output = 1\nInput = 94, 38, 68, 24, output = 1\nInput = 90, 37, 62, 20, output = 1\nInput = 83, 65, 26, 9, output = 0\nInput = 75, 56, 26, 7, output = 0\nInput = 103, 49, 77, 38, output = 2\nInput = 68, 51, 24, 9, output = 0\nInput = 79, 63, 27, 7, output = 0\nInput = 98, 48, 68, 26, output = 1\nInput = 72, 51, 24, 9, output = 0\nInput = 76, 34, 55, 20, output = 1\nInput = 100, 41, 87, 31, output = 2\nInput = 77, 55, 26, 10, output = 0\nInput = 86, 48, 65, 23, output = 1\nInput = 77, 59, 33, 12, output = 0\nInput = 76, 52, 26, 9, output = 0\nInput = 97, 45, 70, 27, output = 1\nInput = 94, 41, 75, 27, output = 1\nInput = 86, 68, 27, 12, output = 0\nInput = 91, 48, 75, 31, output = 2\nInput = 90, 37, 76, 27, output = 2\nInput = 87, 42, 62, 23, output = 1\n", "x_train": [[83, 40, 58, 20], [96, 45, 84, 35], [83, 55, 24, 9], [73, 54, 28, 9], [94, 45, 77, 27], [75, 49, 27, 9], [75, 48, 26, 9], [105, 48, 89, 35], [100, 48, 76, 30], [79, 55, 27, 9], [91, 45, 72, 23], [83, 42, 68, 23], [94, 52, 72, 28], [91, 48, 70, 26], [87, 44, 61, 23], [91, 42, 84, 26], [114, 45, 100, 34], [86, 47, 65, 24], [87, 44, 77, 33], [84, 41, 61, 21], [107, 51, 90, 31], [73, 49, 28, 9], [103, 49, 82, 35], [77, 59, 27, 10], [96, 49, 83, 31], [93, 54, 82, 38], [77, 59, 28, 9], [75, 49, 27, 7], [76, 48, 28, 9], [101, 48, 83, 35], [83, 40, 59, 21], [84, 48, 63, 24], [77, 55, 26, 9], [96, 45, 84, 37], [70, 54, 26, 10], [77, 58, 27, 12], [73, 48, 26, 7], [84, 45, 75, 34], [100, 48, 79, 38], [93, 47, 66, 24], [94, 47, 84, 31], [70, 56, 20, 9], [77, 54, 27, 9], [82, 48, 69, 27], [89, 51, 73, 31], [96, 51, 69, 27], [101, 51, 89, 38], [75, 41, 69, 30], [86, 41, 76, 34], [97, 51, 77, 34], [100, 49, 68, 26], [103, 51, 86, 38], [117, 59, 96, 34], [90, 54, 69, 28], [94, 38, 68, 24], [90, 37, 62, 20], [83, 65, 26, 9], [75, 56, 26, 7], [103, 49, 77, 38], [68, 51, 24, 9], [79, 63, 27, 7], [98, 48, 68, 26], [72, 51, 24, 9], [76, 34, 55, 20], [100, 41, 87, 31], [77, 55, 26, 10], [86, 48, 65, 23], [77, 59, 33, 12], [76, 52, 26, 9], [97, 45, 70, 27], [94, 41, 75, 27], [86, 68, 27, 12], [91, 48, 75, 31], [90, 37, 76, 27], [87, 42, 62, 23]], "y_train": [1, 2, 0, 0, 2, 0, 0, 2, 1, 0, 1, 1, 1, 1, 1, 2, 2, 1, 2, 1, 2, 0, 2, 0, 2, 2, 0, 0, 0, 2, 1, 1, 0, 2, 0, 0, 0, 2, 2, 1, 2, 0, 0, 1, 1, 1, 2, 2, 2, 2, 1, 2, 2, 1, 1, 1, 0, 0, 2, 0, 0, 1, 0, 1, 2, 0, 1, 0, 0, 1, 1, 0, 2, 2, 1], "x_test": [[76, 55, 28, 14], [101, 45, 73, 26], [96, 44, 80, 33], [86, 45, 69, 24], [87, 44, 77, 33], [84, 47, 56, 24], [76, 54, 28, 12], [103, 49, 75, 27], [84, 48, 69, 27], [90, 47, 69, 27], [104, 51, 72, 26], [89, 48, 77, 31], [112, 48, 98, 35], [100, 52, 86, 35], [98, 47, 70, 24], [76, 54, 27, 9], [100, 49, 84, 40], [72, 51, 28, 9], [75, 40, 52, 20], [77, 41, 48, 21], [94, 54, 84, 40], [90, 48, 73, 31], [70, 49, 27, 9], [73, 48, 26, 10], [87, 62, 23, 9], [93, 45, 73, 31], [91, 45, 62, 24], [83, 41, 62, 24], [90, 44, 77, 28], [108, 47, 94, 31], [83, 38, 62, 24], [79, 54, 26, 9], [96, 51, 80, 38], [82, 61, 24, 12], [114, 48, 91, 38], [86, 42, 55, 20], [97, 48, 87, 37], [68, 47, 26, 9], [69, 38, 24, 10], [94, 52, 90, 41], [77, 52, 30, 13], [79, 44, 61, 26], [93, 37, 69, 27], [80, 58, 27, 9], [100, 52, 86, 41], [89, 48, 65, 27], [87, 45, 77, 40], [97, 48, 79, 34], [91, 47, 72, 26], [87, 44, 63, 20], [114, 42, 103, 38], [86, 59, 30, 10], [100, 49, 72, 27], [94, 44, 75, 31], [97, 48, 83, 31], [114, 59, 100, 37], [107, 48, 87, 28], [66, 48, 21, 7], [73, 54, 33, 9], [96, 47, 66, 24], [82, 54, 27, 12], [82, 58, 27, 9], [76, 51, 23, 9], [82, 54, 30, 9], [82, 61, 30, 12], [86, 45, 63, 24], [84, 44, 65, 24], [70, 51, 26, 9], [76, 55, 24, 10], [94, 41, 76, 33], [110, 45, 91, 33], [76, 56, 26, 9], [107, 56, 91, 41], [68, 48, 24, 9], [76, 38, 52, 20]], "y_test": [0, 1, 2, 1, 2, 1, 0, 1, 1, 1, 1, 2, 2, 2, 1, 0, 2, 0, 1, 1, 2, 2, 0, 0, 0, 2, 1, 1, 1, 2, 1, 0, 2, 0, 2, 1, 2, 0, 0, 2, 0, 1, 1, 0, 2, 1, 2, 2, 1, 1, 2, 0, 1, 2, 2, 2, 2, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 2, 2, 0, 2, 0, 1], "ada": {"gpt_output_raw": [" 0\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput ="], "gpt_classification": [0, 1, 2, 1, 2, 1, 0, 2, 1, 1, 1, 2, 2, 2, 1, 0, 2, 0, 1, 1, 2, 2, 0, 0, 0, 2, 1, 1, 2, 2, 1, 0, 2, 0, 2, 1, 2, 0, 0, 2, 0, 1, 2, 0, 2, 1, 2, 2, 1, 1, 2, 0, 1, 1, 2, 2, 2, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 2, 2, 0, 2, 0, 1]}, "babbage": {"gpt_output_raw": [" 0\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput ="], "gpt_classification": [0, 1, 2, 1, 2, 1, 0, 2, 2, 1, 1, 2, 2, 2, 1, 0, 2, 0, 1, 1, 2, 2, 0, 0, 0, 2, 1, 1, 2, 2, 1, 0, 2, 0, 2, 1, 2, 0, 0, 2, 0, 1, 1, 0, 2, 1, 2, 2, 1, 1, 2, 0, 2, 2, 2, 2, 2, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 2, 2, 0, 2, 0, 1]}, "curie": {"gpt_output_raw": [" 0\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput ="], "gpt_classification": [0, 1, 2, 1, 2, 1, 0, 1, 1, 1, 1, 2, 2, 2, 1, 0, 2, 0, 1, 1, 2, 2, 0, 0, 0, 2, 1, 1, 1, 2, 1, 0, 2, 0, 2, 1, 2, 0, 0, 2, 0, 1, 1, 0, 2, 1, 2, 2, 1, 1, 2, 0, 1, 2, 2, 2, 2, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 2, 2, 0, 2, 0, 1]}, "davinci": {"gpt_output_raw": [" 0\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 2\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 1\nInput =", " 1\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 0\nInput =", " 1\nInput =", " 1\nInput =", " 0\nInput =", " 0\nInput =", " 2\nInput =", " 2\nInput =", " 0\nInput =", " 2\nInput =", " 0\nInput =", " 1\nInput ="], "gpt_classification": [0, 1, 2, 1, 2, 1, 0, 1, 1, 1, 1, 2, 2, 2, 1, 0, 2, 0, 1, 1, 2, 2, 0, 0, 0, 2, 1, 1, 2, 2, 1, 0, 2, 0, 2, 1, 2, 0, 0, 2, 0, 1, 1, 0, 2, 1, 2, 2, 1, 1, 2, 0, 1, 2, 2, 2, 2, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 2, 2, 0, 2, 0, 1]}} -------------------------------------------------------------------------------- /iris_test.py: -------------------------------------------------------------------------------- 1 | """ 2 | Loads the Iris sataset, transforms it, and then fits GPT-3-model on it. 3 | 4 | Takes a "name" argument since I did a few variations of this script, 5 | each saved as its own separate experiment. 6 | """ 7 | 8 | 9 | 10 | from sklearn import datasets 11 | from sklearn.model_selection import train_test_split 12 | import numpy as np 13 | import json 14 | import openai 15 | import re 16 | import os 17 | import argparse 18 | 19 | parser = argparse.ArgumentParser() 20 | parser.add_argument('--name', '-n', required=True, type=str) 21 | args = parser.parse_args() 22 | 23 | iris = datasets.load_iris() 24 | 25 | transformed = 14*iris['data'] + 6 26 | transformed = np.vectorize(round)(transformed) 27 | y = iris.target 28 | 29 | x_train, x_test, y_train, y_test = train_test_split(transformed, y, 30 | test_size=0.5, stratify=y, random_state=97) 31 | 32 | 33 | input_template = '' 34 | for x, y in zip(x_train, y_train): 35 | x = ', '.join(map(str, x)) 36 | y = str(y) 37 | input_template += f'Input = {x}, output = {y}\n' 38 | 39 | results = dict() 40 | results['input_template'] = input_template 41 | 42 | results['x_train'] = [x.tolist() for x in list(x_train)] 43 | results['y_train'] = [int(x) for x in list(y_train)] 44 | 45 | results['x_test'] = [x.tolist() for x in list(x_test)] 46 | results['y_test'] = [int(x) for x in list(y_test)] 47 | 48 | 49 | engines = ['ada', 'babbage', 'curie', 'davinci'] 50 | 51 | for engine in engines: 52 | results[engine] = dict() 53 | results[engine]['gpt_output_raw'] = [] 54 | results[engine]['gpt_classification'] = [] 55 | 56 | 57 | openai.api_key = os.getenv("OPENAI_API_KEY") 58 | for engine in engines: 59 | for x in x_test: 60 | input = ', '.join(map(str, x)) 61 | input_text = (input_template + 62 | f'Input = {input}, output =' 63 | ) 64 | response = openai.Completion.create(engine=engine, 65 | prompt=input_text, 66 | max_tokens=4, 67 | temperature=0, top_p=0) 68 | response_text = response['choices'][0]['text'] 69 | results[engine]['gpt_output_raw'].append(response_text) 70 | results[engine]['gpt_classification'].append( 71 | int( 72 | re.findall('-?\d+',response_text 73 | )[0] 74 | ) 75 | ) 76 | 77 | with open(f'iris_results/{args.name}.json', 'w+') as file: 78 | json.dump(results, file) 79 | -------------------------------------------------------------------------------- /number_sense_test.py: -------------------------------------------------------------------------------- 1 | """ 2 | To investigate whether there is any kind of "number sense" influencing 3 | GPT-3's classifications, versus just shallow patern matching on symbols, 4 | I map digits 0, ..., 9 in the feature vector to some 5 | random letters, and see how that affects classification accuracy. 6 | """ 7 | from typing import Mapping 8 | from utils import textify_numbers 9 | import string 10 | import random 11 | import json 12 | import re 13 | import openai 14 | import os 15 | 16 | random.seed(42) 17 | 18 | alphabet = string.ascii_lowercase 19 | # Filter for letters not in "input" or "output", so their co-occurrence 20 | # doesn't confuse GPT-3: 21 | filtered_alphabet = [x for x in alphabet if (x not in 'inputoutput')] 22 | 23 | openai.api_key = os.getenv("OPENAI_API_KEY") 24 | 25 | #letters = random.sample(filtered_alphabet, k=10) 26 | #mapping = dict(zip( 27 | # [str(x) for x in range(0, 10)], 28 | # letters 29 | # ) 30 | # ) 31 | # the above produces: 32 | mapping = {'0': 'd', '1': 'a', '2': 'j', '3': 'h', '4': 'w', 33 | '5': 'c', '6': 'm', '7': 'b', '8': 'l', '9': 'x'} 34 | 35 | with open('experiments_log.json', 'r') as file: 36 | experiments = json.loads(file.read()) 37 | 38 | experiment_names = [f'2d_class_type_{n}_rstate_{rs}' 39 | for n in range(1, 10) 40 | for rs in [42, 55, 93]] 41 | 42 | engine = 'davinci' 43 | 44 | 45 | for name in experiment_names: 46 | experiment = experiments[name] 47 | 48 | modified_input_text = '' 49 | for line in experiment['input_text'].split('\n')[:-1]: 50 | initial_part = line[:-1] 51 | for digit in mapping.keys(): 52 | initial_part = initial_part.replace(digit, mapping[digit]) 53 | new_line = initial_part + line[-1] + '\n' 54 | modified_input_text += new_line 55 | 56 | experiment['letters_input_text'] = modified_input_text 57 | experiment[f'letters_response_{engine}'] = [] 58 | experiment[f'letters_output_test_raw_{engine}'] = [] 59 | experiment[f'letters_output_test_cleaned_{engine}'] = [] 60 | 61 | for point in experiment['input_test']: 62 | point = textify_numbers(point) 63 | for digit in mapping.keys(): 64 | point = point.replace(digit, mapping[digit]) 65 | prompt_text = ( 66 | modified_input_text 67 | + f'Input = {point}, output =' 68 | ) 69 | 70 | response = openai.Completion.create(engine=engine, 71 | prompt=prompt_text, max_tokens=6, 72 | temperature=0, top_p=0) 73 | 74 | experiment[f'letters_response_{engine}'].append(response) 75 | 76 | response_text = response['choices'][0]['text'] 77 | experiment[f'letters_output_test_raw_{engine}'].append(response_text) 78 | 79 | experiment[f'letters_output_test_cleaned_{engine}'].append( 80 | int( 81 | re.findall('-?\d+',response_text 82 | )[0] 83 | ) 84 | ) 85 | 86 | experiments[name] = experiment 87 | 88 | with open('experiments_log.json', 'w') as file: 89 | json.dump(experiments, file, indent=4) 90 | -------------------------------------------------------------------------------- /number_sense_test_spaced.py: -------------------------------------------------------------------------------- 1 | """ 2 | (Same as number_sense_test.py, but only the letters are spaced out in 3 | order to ameliorate encoding issues.) 4 | 5 | To test whether some kind of "number sense" influences GPT-3's 6 | classifications, versus just shallow patern matching on symbols, 7 | I map digits 0, ..., 9 in the feature vector to some 8 | random letters, and see how that affects classification accuracy. 9 | """ 10 | from utils import textify_numbers 11 | import string 12 | import random 13 | import json 14 | import re 15 | import openai 16 | import os 17 | 18 | random.seed(42) 19 | 20 | alphabet = string.ascii_lowercase 21 | # Filter for letters not in "input" or "output", so their co-occurrence 22 | # doesn't confuse GPT-3: 23 | filtered_alphabet = [x for x in alphabet if (x not in 'inputoutput')] 24 | 25 | openai.api_key = os.getenv("OPENAI_API_KEY") 26 | 27 | #letters = random.sample(filtered_alphabet, k=10) 28 | #mapping = dict(zip( 29 | # [str(x) for x in range(0, 10)], 30 | # letters 31 | # ) 32 | # ) 33 | mapping = {'0': 'd ', '1': 'a ', '2': 'j ', '3': 'h ', '4': 'w ', 34 | '5': 'c ', '6': 'm ', '7': 'b ', '8': 'l ', '9': 'x '} 35 | 36 | with open('experiments_log.json', 'r') as file: 37 | experiments = json.loads(file.read()) 38 | 39 | experiment_names = [f'2d_class_type_{n}_rstate_{rs}' 40 | for n in range(1, 10) 41 | for rs in [42, 55, 93]] 42 | 43 | engine = 'davinci' 44 | 45 | 46 | for name in experiment_names: 47 | experiment = experiments[name] 48 | 49 | modified_input_text = '' 50 | for line in experiment['input_text'].split('\n')[:-1]: 51 | initial_part = line[:-1] 52 | for digit in mapping.keys(): 53 | initial_part = initial_part.replace(digit, mapping[digit]) 54 | new_line = initial_part + line[-1] + '\n' 55 | modified_input_text += new_line 56 | 57 | experiment['spaced_letters_input_text'] = modified_input_text 58 | experiment[f'spaced_letters_response_{engine}'] = [] 59 | experiment[f'spaced_letters_output_test_raw_{engine}'] = [] 60 | experiment[f'spaced_letters_output_test_cleaned_{engine}'] = [] 61 | 62 | for point in experiment['input_test']: 63 | point = textify_numbers(point) 64 | for digit in mapping.keys(): 65 | point = point.replace(digit, mapping[digit]) 66 | prompt_text = ( 67 | modified_input_text 68 | + f'Input = {point}, output =' 69 | ) 70 | 71 | response = openai.Completion.create(engine=engine, 72 | prompt=prompt_text, max_tokens=6, 73 | temperature=0, top_p=0) 74 | 75 | experiment[f'spaced_letters_response_{engine}'].append(response) 76 | 77 | response_text = response['choices'][0]['text'] 78 | experiment[f'spaced_letters_output_test_raw_{engine}'].append(response_text) 79 | 80 | experiment[f'spaced_letters_output_test_cleaned_{engine}'].append( 81 | int( 82 | re.findall('-?\d+',response_text 83 | )[0] 84 | ) 85 | ) 86 | 87 | experiments[name] = experiment 88 | 89 | with open('experiments_log.json', 'w') as file: 90 | json.dump(experiments, file, indent=4) 91 | -------------------------------------------------------------------------------- /plots/distribucije.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rovle/gpt3-in-context-fitting/33c36528e668cd803f8085b70a1f2f858d1ed9f9/plots/distribucije.png -------------------------------------------------------------------------------- /plots/razlike.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rovle/gpt3-in-context-fitting/33c36528e668cd803f8085b70a1f2f858d1ed9f9/plots/razlike.png -------------------------------------------------------------------------------- /plots/reg1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rovle/gpt3-in-context-fitting/33c36528e668cd803f8085b70a1f2f858d1ed9f9/plots/reg1.png -------------------------------------------------------------------------------- /plots/reg2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rovle/gpt3-in-context-fitting/33c36528e668cd803f8085b70a1f2f858d1ed9f9/plots/reg2.png -------------------------------------------------------------------------------- /plots/reg3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rovle/gpt3-in-context-fitting/33c36528e668cd803f8085b70a1f2f858d1ed9f9/plots/reg3.png -------------------------------------------------------------------------------- /plots/reg4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rovle/gpt3-in-context-fitting/33c36528e668cd803f8085b70a1f2f858d1ed9f9/plots/reg4.png -------------------------------------------------------------------------------- /results/differences.csv: -------------------------------------------------------------------------------- 1 | ,Logistic Regr.,Ada,Babbage,Curie,Davinci 2 | 0,0.0,-0.033333333333333326,-0.1333333333333333,-0.033333333333333326,-0.06666666666666665 3 | 1,0.06666666666666676,0.16666666666666674,-0.23333333333333328,0.06666666666666676,-0.1333333333333333 4 | 2,-0.06666666666666676,0.0,0.0,0.0,-0.033333333333333326 5 | 3,0.06666666666666665,0.0,-0.03333333333333344,-0.06666666666666676,0.033333333333333326 6 | 4,0.06666666666666665,-0.30000000000000004,-0.33333333333333337,-0.16666666666666674,-0.06666666666666665 7 | 5,-0.13333333333333341,-0.03333333333333344,-0.13333333333333341,0.0,-0.03333333333333344 8 | 6,-0.2666666666666667,-0.03333333333333344,0.06666666666666665,0.0,-0.03333333333333344 9 | 7,-0.39999999999999997,0.10000000000000009,-0.06666666666666665,0.0,0.10000000000000009 10 | 8,-0.06666666666666665,0.13333333333333341,0.033333333333333326,0.13333333333333341,0.13333333333333341 11 | 9,-0.033333333333333326,-0.16666666666666663,-0.033333333333333326,-0.06666666666666665,-0.1333333333333333 12 | 10,0.033333333333333326,0.06666666666666665,0.0,-0.1333333333333333,-0.1333333333333333 13 | 11,0.0,-0.1333333333333333,-0.033333333333333326,0.0,-0.06666666666666665 14 | 12,-0.6,-0.033333333333333326,0.0,-0.06666666666666665,0.0 15 | 13,-0.5,-0.09999999999999998,-0.2333333333333334,-0.033333333333333326,-0.033333333333333326 16 | 14,-0.7,-0.09999999999999998,-0.09999999999999998,-0.06666666666666665,-0.06666666666666665 17 | 15,0.0,-0.4,-0.4,-0.033333333333333326,0.0 18 | 16,0.1333333333333333,-0.13333333333333341,-0.06666666666666676,-0.03333333333333344,-0.06666666666666676 19 | 17,0.033333333333333326,-0.19999999999999996,-0.1333333333333333,0.0,0.13333333333333341 20 | 18,-0.43333333333333335,-0.033333333333333326,-0.06666666666666665,-0.033333333333333326,0.0 21 | 19,-0.4666666666666667,-0.16666666666666663,-0.30000000000000004,-0.30000000000000004,-0.5666666666666667 22 | 20,-0.3666666666666667,0.033333333333333326,-0.06666666666666676,-0.06666666666666676,-0.03333333333333344 23 | 21,-0.2666666666666666,-0.06666666666666665,0.03333333333333344,-0.16666666666666663,-0.09999999999999998 24 | 22,-0.30000000000000004,-0.2666666666666667,-0.09999999999999998,-0.30000000000000004,-0.30000000000000004 25 | 23,-0.4,-0.09999999999999998,-0.20000000000000007,-0.09999999999999998,0.06666666666666665 26 | 24,-0.06666666666666665,-0.06666666666666665,0.16666666666666663,0.0,0.033333333333333326 27 | 25,-0.23333333333333328,-0.06666666666666665,-0.033333333333333326,-0.19999999999999996,0.0 28 | 26,-0.33333333333333337,-0.2333333333333334,-0.20000000000000007,0.033333333333333326,-0.20000000000000007 29 | -------------------------------------------------------------------------------- /results/distribution_samples.csv: -------------------------------------------------------------------------------- 1 | ,x,y,clas,type 2 | 1,29,39,0,Scenario 1 3 | 2,27,42,0,Scenario 1 4 | 3,53,37,0,Scenario 1 5 | 4,50,43,0,Scenario 1 6 | 5,61,37,0,Scenario 1 7 | 6,71,29,0,Scenario 1 8 | 7,71,22,0,Scenario 1 9 | 8,44,52,0,Scenario 1 10 | 9,53,46,0,Scenario 1 11 | 10,61,46,0,Scenario 1 12 | 11,33,31,0,Scenario 1 13 | 12,48,43,0,Scenario 1 14 | 13,54,38,0,Scenario 1 15 | 14,45,53,0,Scenario 1 16 | 15,53,44,0,Scenario 1 17 | 16,47,35,0,Scenario 1 18 | 17,55,49,0,Scenario 1 19 | 18,65,44,0,Scenario 1 20 | 19,33,47,0,Scenario 1 21 | 20,30,30,0,Scenario 1 22 | 21,42,33,0,Scenario 1 23 | 22,61,44,0,Scenario 1 24 | 23,58,34,0,Scenario 1 25 | 24,52,51,0,Scenario 1 26 | 25,35,35,0,Scenario 1 27 | 26,43,31,0,Scenario 1 28 | 27,52,47,0,Scenario 1 29 | 28,53,41,0,Scenario 1 30 | 29,70,33,0,Scenario 1 31 | 30,49,52,0,Scenario 1 32 | 31,59,26,0,Scenario 1 33 | 32,81,49,0,Scenario 1 34 | 33,59,33,0,Scenario 1 35 | 34,41,51,0,Scenario 1 36 | 35,54,41,0,Scenario 1 37 | 36,61,27,0,Scenario 1 38 | 37,57,38,0,Scenario 1 39 | 38,64,39,0,Scenario 1 40 | 39,43,32,0,Scenario 1 41 | 40,57,25,0,Scenario 1 42 | 41,71,39,0,Scenario 1 43 | 42,55,24,0,Scenario 1 44 | 43,38,69,0,Scenario 1 45 | 44,52,45,0,Scenario 1 46 | 45,31,22,0,Scenario 1 47 | 46,40,49,0,Scenario 1 48 | 47,63,31,0,Scenario 1 49 | 48,43,40,0,Scenario 1 50 | 49,20,42,0,Scenario 1 51 | 50,61,49,0,Scenario 1 52 | 51,55,47,1,Scenario 1 53 | 52,45,74,1,Scenario 1 54 | 53,36,57,1,Scenario 1 55 | 54,58,57,1,Scenario 1 56 | 55,48,52,1,Scenario 1 57 | 56,68,69,1,Scenario 1 58 | 57,56,54,1,Scenario 1 59 | 58,49,73,1,Scenario 1 60 | 59,59,71,1,Scenario 1 61 | 60,33,69,1,Scenario 1 62 | 61,58,71,1,Scenario 1 63 | 62,50,63,1,Scenario 1 64 | 63,51,32,1,Scenario 1 65 | 64,28,51,1,Scenario 1 66 | 65,44,65,1,Scenario 1 67 | 66,53,46,1,Scenario 1 68 | 67,54,53,1,Scenario 1 69 | 68,58,49,1,Scenario 1 70 | 69,53,53,1,Scenario 1 71 | 70,60,54,1,Scenario 1 72 | 71,49,43,1,Scenario 1 73 | 72,50,53,1,Scenario 1 74 | 73,60,71,1,Scenario 1 75 | 74,40,61,1,Scenario 1 76 | 75,54,58,1,Scenario 1 77 | 76,38,35,1,Scenario 1 78 | 77,49,57,1,Scenario 1 79 | 78,39,52,1,Scenario 1 80 | 79,52,81,1,Scenario 1 81 | 80,53,51,1,Scenario 1 82 | 81,50,85,1,Scenario 1 83 | 82,49,41,1,Scenario 1 84 | 83,28,63,1,Scenario 1 85 | 84,55,94,1,Scenario 1 86 | 85,56,64,1,Scenario 1 87 | 86,46,82,1,Scenario 1 88 | 87,56,64,1,Scenario 1 89 | 88,49,61,1,Scenario 1 90 | 89,58,44,1,Scenario 1 91 | 90,42,60,1,Scenario 1 92 | 91,36,66,1,Scenario 1 93 | 92,61,85,1,Scenario 1 94 | 93,59,73,1,Scenario 1 95 | 94,36,72,1,Scenario 1 96 | 95,42,54,1,Scenario 1 97 | 96,54,46,1,Scenario 1 98 | 97,58,79,1,Scenario 1 99 | 98,61,68,1,Scenario 1 100 | 99,50,62,1,Scenario 1 101 | 100,53,58,1,Scenario 1 102 | 101,39,29,0,Scenario 2 103 | 102,42,27,0,Scenario 2 104 | 103,37,53,0,Scenario 2 105 | 104,43,50,0,Scenario 2 106 | 105,37,61,0,Scenario 2 107 | 106,29,71,0,Scenario 2 108 | 107,22,71,0,Scenario 2 109 | 108,52,44,0,Scenario 2 110 | 109,46,53,0,Scenario 2 111 | 110,46,61,0,Scenario 2 112 | 111,31,33,0,Scenario 2 113 | 112,43,48,0,Scenario 2 114 | 113,38,54,0,Scenario 2 115 | 114,53,45,0,Scenario 2 116 | 115,44,53,0,Scenario 2 117 | 116,35,47,0,Scenario 2 118 | 117,49,55,0,Scenario 2 119 | 118,44,65,0,Scenario 2 120 | 119,47,33,0,Scenario 2 121 | 120,30,30,0,Scenario 2 122 | 121,33,42,0,Scenario 2 123 | 122,44,61,0,Scenario 2 124 | 123,34,58,0,Scenario 2 125 | 124,51,52,0,Scenario 2 126 | 125,35,35,0,Scenario 2 127 | 126,31,43,0,Scenario 2 128 | 127,47,52,0,Scenario 2 129 | 128,41,53,0,Scenario 2 130 | 129,33,70,0,Scenario 2 131 | 130,52,49,0,Scenario 2 132 | 131,26,59,0,Scenario 2 133 | 132,49,81,0,Scenario 2 134 | 133,33,59,0,Scenario 2 135 | 134,51,41,0,Scenario 2 136 | 135,41,54,0,Scenario 2 137 | 136,27,61,0,Scenario 2 138 | 137,38,57,0,Scenario 2 139 | 138,39,64,0,Scenario 2 140 | 139,32,43,0,Scenario 2 141 | 140,25,57,0,Scenario 2 142 | 141,39,71,0,Scenario 2 143 | 142,24,55,0,Scenario 2 144 | 143,69,38,0,Scenario 2 145 | 144,45,52,0,Scenario 2 146 | 145,22,31,0,Scenario 2 147 | 146,49,40,0,Scenario 2 148 | 147,31,63,0,Scenario 2 149 | 148,40,43,0,Scenario 2 150 | 149,42,20,0,Scenario 2 151 | 150,49,61,0,Scenario 2 152 | 151,47,55,1,Scenario 2 153 | 152,74,45,1,Scenario 2 154 | 153,57,36,1,Scenario 2 155 | 154,57,58,1,Scenario 2 156 | 155,52,48,1,Scenario 2 157 | 156,69,68,1,Scenario 2 158 | 157,54,56,1,Scenario 2 159 | 158,73,49,1,Scenario 2 160 | 159,71,59,1,Scenario 2 161 | 160,69,33,1,Scenario 2 162 | 161,71,58,1,Scenario 2 163 | 162,63,50,1,Scenario 2 164 | 163,32,51,1,Scenario 2 165 | 164,51,28,1,Scenario 2 166 | 165,65,44,1,Scenario 2 167 | 166,46,53,1,Scenario 2 168 | 167,53,54,1,Scenario 2 169 | 168,49,58,1,Scenario 2 170 | 169,53,53,1,Scenario 2 171 | 170,54,60,1,Scenario 2 172 | 171,43,49,1,Scenario 2 173 | 172,53,50,1,Scenario 2 174 | 173,71,60,1,Scenario 2 175 | 174,61,40,1,Scenario 2 176 | 175,58,54,1,Scenario 2 177 | 176,35,38,1,Scenario 2 178 | 177,57,49,1,Scenario 2 179 | 178,52,39,1,Scenario 2 180 | 179,81,52,1,Scenario 2 181 | 180,51,53,1,Scenario 2 182 | 181,85,50,1,Scenario 2 183 | 182,41,49,1,Scenario 2 184 | 183,63,28,1,Scenario 2 185 | 184,94,55,1,Scenario 2 186 | 185,64,56,1,Scenario 2 187 | 186,82,46,1,Scenario 2 188 | 187,64,56,1,Scenario 2 189 | 188,61,49,1,Scenario 2 190 | 189,44,58,1,Scenario 2 191 | 190,60,42,1,Scenario 2 192 | 191,66,36,1,Scenario 2 193 | 192,85,61,1,Scenario 2 194 | 193,73,59,1,Scenario 2 195 | 194,72,36,1,Scenario 2 196 | 195,54,42,1,Scenario 2 197 | 196,46,54,1,Scenario 2 198 | 197,79,58,1,Scenario 2 199 | 198,68,61,1,Scenario 2 200 | 199,62,50,1,Scenario 2 201 | 200,58,53,1,Scenario 2 202 | 201,54,49,0,Scenario 3 203 | 202,55,61,0,Scenario 3 204 | 203,48,48,0,Scenario 3 205 | 204,61,55,0,Scenario 3 206 | 205,47,54,0,Scenario 3 207 | 206,47,47,0,Scenario 3 208 | 207,52,36,0,Scenario 3 209 | 208,38,46,0,Scenario 3 210 | 209,43,52,0,Scenario 3 211 | 210,44,40,0,Scenario 3 212 | 211,60,48,0,Scenario 3 213 | 212,50,40,0,Scenario 3 214 | 213,46,51,0,Scenario 3 215 | 214,42,53,0,Scenario 3 216 | 215,46,48,0,Scenario 3 217 | 216,46,63,0,Scenario 3 218 | 217,50,43,0,Scenario 3 219 | 218,56,41,0,Scenario 3 220 | 219,51,36,0,Scenario 3 221 | 220,41,51,0,Scenario 3 222 | 221,55,51,0,Scenario 3 223 | 222,49,48,0,Scenario 3 224 | 223,40,45,0,Scenario 3 225 | 224,47,57,0,Scenario 3 226 | 225,52,38,0,Scenario 3 227 | 226,52,47,0,Scenario 3 228 | 227,45,54,0,Scenario 3 229 | 228,57,57,0,Scenario 3 230 | 229,44,48,0,Scenario 3 231 | 230,52,57,0,Scenario 3 232 | 231,47,49,0,Scenario 3 233 | 232,42,42,0,Scenario 3 234 | 233,56,60,0,Scenario 3 235 | 234,49,57,0,Scenario 3 236 | 235,53,45,0,Scenario 3 237 | 236,53,61,0,Scenario 3 238 | 237,50,61,0,Scenario 3 239 | 238,31,56,0,Scenario 3 240 | 239,51,48,0,Scenario 3 241 | 240,51,36,0,Scenario 3 242 | 241,48,53,0,Scenario 3 243 | 242,60,46,0,Scenario 3 244 | 243,44,46,0,Scenario 3 245 | 244,56,52,0,Scenario 3 246 | 245,46,54,0,Scenario 3 247 | 246,51,57,0,Scenario 3 248 | 247,45,48,0,Scenario 3 249 | 248,47,40,0,Scenario 3 250 | 249,52,52,0,Scenario 3 251 | 250,50,48,0,Scenario 3 252 | 251,56,30,1,Scenario 3 253 | 252,42,38,1,Scenario 3 254 | 253,69,41,1,Scenario 3 255 | 254,61,95,1,Scenario 3 256 | 255,78,40,1,Scenario 3 257 | 256,42,60,1,Scenario 3 258 | 257,13,31,1,Scenario 3 259 | 258,61,47,1,Scenario 3 260 | 259,93,36,1,Scenario 3 261 | 260,27,30,1,Scenario 3 262 | 261,50,46,1,Scenario 3 263 | 262,100,41,1,Scenario 3 264 | 263,51,70,1,Scenario 3 265 | 264,48,62,1,Scenario 3 266 | 265,59,69,1,Scenario 3 267 | 266,19,68,1,Scenario 3 268 | 267,47,54,1,Scenario 3 269 | 268,41,87,1,Scenario 3 270 | 269,4,81,1,Scenario 3 271 | 270,35,84,1,Scenario 3 272 | 271,77,73,1,Scenario 3 273 | 272,69,101,1,Scenario 3 274 | 273,37,37,1,Scenario 3 275 | 274,35,71,1,Scenario 3 276 | 275,21,73,1,Scenario 3 277 | 276,44,26,1,Scenario 3 278 | 277,26,32,1,Scenario 3 279 | 278,17,61,1,Scenario 3 280 | 279,69,71,1,Scenario 3 281 | 280,36,57,1,Scenario 3 282 | 281,50,65,1,Scenario 3 283 | 282,60,52,1,Scenario 3 284 | 283,8,46,1,Scenario 3 285 | 284,52,34,1,Scenario 3 286 | 285,44,21,1,Scenario 3 287 | 286,43,61,1,Scenario 3 288 | 287,97,27,1,Scenario 3 289 | 288,49,29,1,Scenario 3 290 | 289,48,45,1,Scenario 3 291 | 290,69,72,1,Scenario 3 292 | 291,54,47,1,Scenario 3 293 | 292,77,54,1,Scenario 3 294 | 293,84,13,1,Scenario 3 295 | 294,90,36,1,Scenario 3 296 | 295,26,49,1,Scenario 3 297 | 296,24,63,1,Scenario 3 298 | 297,40,56,1,Scenario 3 299 | 298,28,45,1,Scenario 3 300 | 299,52,46,1,Scenario 3 301 | 300,86,63,1,Scenario 3 302 | 301,75,29,0,Scenario 4 303 | 302,69,22,0,Scenario 4 304 | 303,57,-12,0,Scenario 4 305 | 304,79,13,0,Scenario 4 306 | 305,81,38,0,Scenario 4 307 | 306,75,29,0,Scenario 4 308 | 307,54,36,0,Scenario 4 309 | 308,50,18,0,Scenario 4 310 | 309,61,25,0,Scenario 4 311 | 310,73,3,0,Scenario 4 312 | 311,66,5,0,Scenario 4 313 | 312,66,30,0,Scenario 4 314 | 313,53,49,0,Scenario 4 315 | 314,68,35,0,Scenario 4 316 | 315,107,2,0,Scenario 4 317 | 316,68,21,0,Scenario 4 318 | 317,62,41,0,Scenario 4 319 | 318,72,43,0,Scenario 4 320 | 319,67,30,0,Scenario 4 321 | 320,69,19,0,Scenario 4 322 | 321,78,33,0,Scenario 4 323 | 322,63,13,0,Scenario 4 324 | 323,66,23,0,Scenario 4 325 | 324,55,21,0,Scenario 4 326 | 325,91,6,0,Scenario 4 327 | 326,71,50,0,Scenario 4 328 | 327,67,12,0,Scenario 4 329 | 328,79,25,0,Scenario 4 330 | 329,87,3,0,Scenario 4 331 | 330,76,26,0,Scenario 4 332 | 331,74,15,0,Scenario 4 333 | 332,78,42,0,Scenario 4 334 | 333,76,27,0,Scenario 4 335 | 334,80,22,0,Scenario 4 336 | 335,66,29,0,Scenario 4 337 | 336,96,34,0,Scenario 4 338 | 337,72,42,0,Scenario 4 339 | 338,88,42,0,Scenario 4 340 | 339,48,28,0,Scenario 4 341 | 340,101,50,0,Scenario 4 342 | 341,74,17,0,Scenario 4 343 | 342,63,24,0,Scenario 4 344 | 343,74,31,0,Scenario 4 345 | 344,72,16,0,Scenario 4 346 | 345,72,22,0,Scenario 4 347 | 346,68,5,0,Scenario 4 348 | 347,72,17,0,Scenario 4 349 | 348,70,10,0,Scenario 4 350 | 349,83,30,0,Scenario 4 351 | 350,41,39,0,Scenario 4 352 | 351,77,66,1,Scenario 4 353 | 352,68,92,1,Scenario 4 354 | 353,36,44,1,Scenario 4 355 | 354,37,89,1,Scenario 4 356 | 355,10,64,1,Scenario 4 357 | 356,29,62,1,Scenario 4 358 | 357,19,4,1,Scenario 4 359 | 358,37,89,1,Scenario 4 360 | 359,6,28,1,Scenario 4 361 | 360,47,36,1,Scenario 4 362 | 361,11,29,1,Scenario 4 363 | 362,90,95,1,Scenario 4 364 | 363,48,83,1,Scenario 4 365 | 364,18,22,1,Scenario 4 366 | 365,81,87,1,Scenario 4 367 | 366,52,66,1,Scenario 4 368 | 367,30,16,1,Scenario 4 369 | 368,69,64,1,Scenario 4 370 | 369,67,70,1,Scenario 4 371 | 370,32,23,1,Scenario 4 372 | 371,38,30,1,Scenario 4 373 | 372,57,69,1,Scenario 4 374 | 373,25,10,1,Scenario 4 375 | 374,60,66,1,Scenario 4 376 | 375,76,76,1,Scenario 4 377 | 376,23,83,1,Scenario 4 378 | 377,13,21,1,Scenario 4 379 | 378,9,30,1,Scenario 4 380 | 379,26,-3,1,Scenario 4 381 | 380,80,83,1,Scenario 4 382 | 381,72,55,1,Scenario 4 383 | 382,10,62,1,Scenario 4 384 | 383,53,92,1,Scenario 4 385 | 384,103,94,1,Scenario 4 386 | 385,37,69,1,Scenario 4 387 | 386,58,78,1,Scenario 4 388 | 387,81,78,1,Scenario 4 389 | 388,86,49,1,Scenario 4 390 | 389,30,0,1,Scenario 4 391 | 390,15,20,1,Scenario 4 392 | 391,54,51,1,Scenario 4 393 | 392,58,84,1,Scenario 4 394 | 393,68,32,1,Scenario 4 395 | 394,62,80,1,Scenario 4 396 | 395,31,87,1,Scenario 4 397 | 396,9,8,1,Scenario 4 398 | 397,15,34,1,Scenario 4 399 | 398,29,29,1,Scenario 4 400 | 399,50,66,1,Scenario 4 401 | 400,26,88,1,Scenario 4 402 | 401,57,62,0,Scenario 5 403 | 402,37,32,0,Scenario 5 404 | 403,75,57,0,Scenario 5 405 | 404,24,33,0,Scenario 5 406 | 405,49,53,0,Scenario 5 407 | 406,64,58,0,Scenario 5 408 | 407,65,63,0,Scenario 5 409 | 408,62,60,0,Scenario 5 410 | 409,36,40,0,Scenario 5 411 | 410,42,38,0,Scenario 5 412 | 411,66,62,0,Scenario 5 413 | 412,56,47,0,Scenario 5 414 | 413,54,59,0,Scenario 5 415 | 414,60,57,0,Scenario 5 416 | 415,55,54,0,Scenario 5 417 | 416,44,49,0,Scenario 5 418 | 417,41,48,0,Scenario 5 419 | 418,44,47,0,Scenario 5 420 | 419,65,43,0,Scenario 5 421 | 420,57,51,0,Scenario 5 422 | 421,64,50,0,Scenario 5 423 | 422,44,36,0,Scenario 5 424 | 423,45,45,0,Scenario 5 425 | 424,42,49,0,Scenario 5 426 | 425,19,36,0,Scenario 5 427 | 426,32,46,0,Scenario 5 428 | 427,49,50,0,Scenario 5 429 | 428,56,56,0,Scenario 5 430 | 429,35,40,0,Scenario 5 431 | 430,52,48,0,Scenario 5 432 | 431,28,35,0,Scenario 5 433 | 432,43,44,0,Scenario 5 434 | 433,48,39,0,Scenario 5 435 | 434,50,46,0,Scenario 5 436 | 435,30,41,0,Scenario 5 437 | 436,56,50,0,Scenario 5 438 | 437,62,61,0,Scenario 5 439 | 438,55,63,0,Scenario 5 440 | 439,66,57,0,Scenario 5 441 | 440,68,54,0,Scenario 5 442 | 441,47,43,0,Scenario 5 443 | 442,34,39,0,Scenario 5 444 | 443,79,64,0,Scenario 5 445 | 444,48,55,0,Scenario 5 446 | 445,58,54,0,Scenario 5 447 | 446,69,69,0,Scenario 5 448 | 447,81,72,0,Scenario 5 449 | 448,50,52,0,Scenario 5 450 | 449,39,48,0,Scenario 5 451 | 450,54,43,0,Scenario 5 452 | 451,48,83,1,Scenario 5 453 | 452,67,21,1,Scenario 5 454 | 453,76,5,1,Scenario 5 455 | 454,24,72,1,Scenario 5 456 | 455,86,44,1,Scenario 5 457 | 456,69,4,1,Scenario 5 458 | 457,38,37,1,Scenario 5 459 | 458,65,34,1,Scenario 5 460 | 459,59,30,1,Scenario 5 461 | 460,10,64,1,Scenario 5 462 | 461,65,20,1,Scenario 5 463 | 462,9,83,1,Scenario 5 464 | 463,80,47,1,Scenario 5 465 | 464,26,88,1,Scenario 5 466 | 465,32,73,1,Scenario 5 467 | 466,54,15,1,Scenario 5 468 | 467,75,22,1,Scenario 5 469 | 468,68,22,1,Scenario 5 470 | 469,68,33,1,Scenario 5 471 | 470,97,36,1,Scenario 5 472 | 471,59,8,1,Scenario 5 473 | 472,84,47,1,Scenario 5 474 | 473,52,66,1,Scenario 5 475 | 474,37,107,1,Scenario 5 476 | 475,50,66,1,Scenario 5 477 | 476,72,22,1,Scenario 5 478 | 477,10,62,1,Scenario 5 479 | 478,16,79,1,Scenario 5 480 | 479,62,5,1,Scenario 5 481 | 480,80,16,1,Scenario 5 482 | 481,31,87,1,Scenario 5 483 | 482,51,17,1,Scenario 5 484 | 483,37,89,1,Scenario 5 485 | 484,80,20,1,Scenario 5 486 | 485,1,63,1,Scenario 5 487 | 486,80,39,1,Scenario 5 488 | 487,63,21,1,Scenario 5 489 | 488,31,76,1,Scenario 5 490 | 489,76,-3,1,Scenario 5 491 | 490,96,18,1,Scenario 5 492 | 491,25,85,1,Scenario 5 493 | 492,15,97,1,Scenario 5 494 | 493,-1,73,1,Scenario 5 495 | 494,37,89,1,Scenario 5 496 | 495,42,89,1,Scenario 5 497 | 496,67,27,1,Scenario 5 498 | 497,5,86,1,Scenario 5 499 | 498,75,10,1,Scenario 5 500 | 499,74,47,1,Scenario 5 501 | 500,64,18,1,Scenario 5 502 | 501,39,49,0,Scenario 6 503 | 502,30,56,0,Scenario 6 504 | 503,49,56,0,Scenario 6 505 | 504,21,43,0,Scenario 6 506 | 505,49,63,0,Scenario 6 507 | 506,53,57,0,Scenario 6 508 | 507,49,43,0,Scenario 6 509 | 508,71,70,0,Scenario 6 510 | 509,58,67,0,Scenario 6 511 | 510,63,57,0,Scenario 6 512 | 511,26,39,0,Scenario 6 513 | 512,50,47,0,Scenario 6 514 | 513,52,61,0,Scenario 6 515 | 514,59,69,0,Scenario 6 516 | 515,54,60,0,Scenario 6 517 | 516,46,71,0,Scenario 6 518 | 517,49,50,0,Scenario 6 519 | 518,39,40,0,Scenario 6 520 | 519,50,43,0,Scenario 6 521 | 520,62,70,0,Scenario 6 522 | 521,34,48,0,Scenario 6 523 | 522,48,55,0,Scenario 6 524 | 523,68,67,0,Scenario 6 525 | 524,47,65,0,Scenario 6 526 | 525,47,42,0,Scenario 6 527 | 526,42,50,0,Scenario 6 528 | 527,52,65,0,Scenario 6 529 | 528,27,49,0,Scenario 6 530 | 529,58,62,0,Scenario 6 531 | 530,37,57,0,Scenario 6 532 | 531,52,59,0,Scenario 6 533 | 532,65,60,0,Scenario 6 534 | 533,29,54,0,Scenario 6 535 | 534,42,61,0,Scenario 6 536 | 535,43,48,0,Scenario 6 537 | 536,34,59,0,Scenario 6 538 | 537,39,63,0,Scenario 6 539 | 538,77,86,0,Scenario 6 540 | 539,45,53,0,Scenario 6 541 | 540,52,44,0,Scenario 6 542 | 541,47,59,0,Scenario 6 543 | 542,27,37,0,Scenario 6 544 | 543,58,61,0,Scenario 6 545 | 544,31,47,0,Scenario 6 546 | 545,50,63,0,Scenario 6 547 | 546,40,59,0,Scenario 6 548 | 547,56,61,0,Scenario 6 549 | 548,56,51,0,Scenario 6 550 | 549,40,53,0,Scenario 6 551 | 550,46,54,0,Scenario 6 552 | 551,55,38,1,Scenario 6 553 | 552,62,46,1,Scenario 6 554 | 553,45,34,1,Scenario 6 555 | 554,40,50,1,Scenario 6 556 | 555,40,30,1,Scenario 6 557 | 556,58,51,1,Scenario 6 558 | 557,81,58,1,Scenario 6 559 | 558,49,39,1,Scenario 6 560 | 559,31,22,1,Scenario 6 561 | 560,73,51,1,Scenario 6 562 | 561,55,44,1,Scenario 6 563 | 562,26,20,1,Scenario 6 564 | 563,51,49,1,Scenario 6 565 | 564,54,49,1,Scenario 6 566 | 565,46,45,1,Scenario 6 567 | 566,71,64,1,Scenario 6 568 | 567,56,47,1,Scenario 6 569 | 568,54,58,1,Scenario 6 570 | 569,77,73,1,Scenario 6 571 | 570,58,60,1,Scenario 6 572 | 571,34,38,1,Scenario 6 573 | 572,35,48,1,Scenario 6 574 | 573,65,48,1,Scenario 6 575 | 574,60,57,1,Scenario 6 576 | 575,69,64,1,Scenario 6 577 | 576,63,42,1,Scenario 6 578 | 577,73,52,1,Scenario 6 579 | 578,73,63,1,Scenario 6 580 | 579,40,41,1,Scenario 6 581 | 580,62,53,1,Scenario 6 582 | 581,52,49,1,Scenario 6 583 | 582,49,41,1,Scenario 6 584 | 583,81,63,1,Scenario 6 585 | 584,56,40,1,Scenario 6 586 | 585,64,41,1,Scenario 6 587 | 586,57,51,1,Scenario 6 588 | 587,31,18,1,Scenario 6 589 | 588,59,40,1,Scenario 6 590 | 589,57,45,1,Scenario 6 591 | 590,40,41,1,Scenario 6 592 | 591,53,43,1,Scenario 6 593 | 592,38,34,1,Scenario 6 594 | 593,41,21,1,Scenario 6 595 | 594,33,23,1,Scenario 6 596 | 595,70,56,1,Scenario 6 597 | 596,69,60,1,Scenario 6 598 | 597,60,51,1,Scenario 6 599 | 598,69,54,1,Scenario 6 600 | 599,54,43,1,Scenario 6 601 | 600,31,32,1,Scenario 6 602 | 601,75,73,0,Scenario 7 603 | 602,76,88,0,Scenario 7 604 | 603,12,5,0,Scenario 7 605 | 604,74,72,0,Scenario 7 606 | 605,17,21,0,Scenario 7 607 | 606,51,63,0,Scenario 7 608 | 607,26,21,0,Scenario 7 609 | 608,71,57,0,Scenario 7 610 | 609,28,-2,0,Scenario 7 611 | 610,30,47,0,Scenario 7 612 | 611,82,73,0,Scenario 7 613 | 612,68,79,0,Scenario 7 614 | 613,70,82,0,Scenario 7 615 | 614,54,82,0,Scenario 7 616 | 615,87,107,0,Scenario 7 617 | 616,81,87,0,Scenario 7 618 | 617,75,85,0,Scenario 7 619 | 618,71,60,0,Scenario 7 620 | 619,30,0,0,Scenario 7 621 | 620,19,4,0,Scenario 7 622 | 621,46,95,0,Scenario 7 623 | 622,93,69,0,Scenario 7 624 | 623,61,72,0,Scenario 7 625 | 624,15,20,0,Scenario 7 626 | 625,73,78,0,Scenario 7 627 | 626,25,22,0,Scenario 7 628 | 627,30,39,0,Scenario 7 629 | 628,49,73,0,Scenario 7 630 | 629,70,81,0,Scenario 7 631 | 630,102,66,0,Scenario 7 632 | 631,24,47,0,Scenario 7 633 | 632,18,32,0,Scenario 7 634 | 633,30,16,0,Scenario 7 635 | 634,18,22,0,Scenario 7 636 | 635,18,40,0,Scenario 7 637 | 636,60,74,0,Scenario 7 638 | 637,34,47,0,Scenario 7 639 | 638,55,86,0,Scenario 7 640 | 639,75,61,0,Scenario 7 641 | 640,26,39,0,Scenario 7 642 | 641,92,89,0,Scenario 7 643 | 642,22,22,0,Scenario 7 644 | 643,18,33,0,Scenario 7 645 | 644,100,66,0,Scenario 7 646 | 645,60,64,0,Scenario 7 647 | 646,18,18,0,Scenario 7 648 | 647,60,62,0,Scenario 7 649 | 648,9,30,0,Scenario 7 650 | 649,87,89,0,Scenario 7 651 | 650,46,22,0,Scenario 7 652 | 651,103,44,1,Scenario 7 653 | 652,74,29,1,Scenario 7 654 | 653,69,28,1,Scenario 7 655 | 654,26,77,1,Scenario 7 656 | 655,72,5,1,Scenario 7 657 | 656,76,26,1,Scenario 7 658 | 657,91,25,1,Scenario 7 659 | 658,29,75,1,Scenario 7 660 | 659,28,83,1,Scenario 7 661 | 660,65,25,1,Scenario 7 662 | 661,74,33,1,Scenario 7 663 | 662,17,80,1,Scenario 7 664 | 663,16,55,1,Scenario 7 665 | 664,81,37,1,Scenario 7 666 | 665,64,44,1,Scenario 7 667 | 666,18,71,1,Scenario 7 668 | 667,46,84,1,Scenario 7 669 | 668,22,93,1,Scenario 7 670 | 669,24,67,1,Scenario 7 671 | 670,90,45,1,Scenario 7 672 | 671,51,100,1,Scenario 7 673 | 672,13,74,1,Scenario 7 674 | 673,26,76,1,Scenario 7 675 | 674,69,14,1,Scenario 7 676 | 675,60,16,1,Scenario 7 677 | 676,29,63,1,Scenario 7 678 | 677,25,79,1,Scenario 7 679 | 678,37,53,1,Scenario 7 680 | 679,62,33,1,Scenario 7 681 | 680,16,79,1,Scenario 7 682 | 681,4,86,1,Scenario 7 683 | 682,0,68,1,Scenario 7 684 | 683,18,85,1,Scenario 7 685 | 684,22,72,1,Scenario 7 686 | 685,71,15,1,Scenario 7 687 | 686,5,71,1,Scenario 7 688 | 687,74,42,1,Scenario 7 689 | 688,81,28,1,Scenario 7 690 | 689,81,22,1,Scenario 7 691 | 690,68,42,1,Scenario 7 692 | 691,61,22,1,Scenario 7 693 | 692,75,0,1,Scenario 7 694 | 693,17,62,1,Scenario 7 695 | 694,91,25,1,Scenario 7 696 | 695,58,34,1,Scenario 7 697 | 696,43,28,1,Scenario 7 698 | 697,80,33,1,Scenario 7 699 | 698,19,69,1,Scenario 7 700 | 699,18,55,1,Scenario 7 701 | 700,28,92,1,Scenario 7 702 | 701,67,62,0,Scenario 8 703 | 702,23,47,0,Scenario 8 704 | 703,6,53,0,Scenario 8 705 | 704,58,36,0,Scenario 8 706 | 705,18,38,0,Scenario 8 707 | 706,61,48,0,Scenario 8 708 | 707,27,65,0,Scenario 8 709 | 708,59,51,0,Scenario 8 710 | 709,31,59,0,Scenario 8 711 | 710,57,62,0,Scenario 8 712 | 711,13,50,0,Scenario 8 713 | 712,26,13,0,Scenario 8 714 | 713,27,55,0,Scenario 8 715 | 714,53,50,0,Scenario 8 716 | 715,67,62,0,Scenario 8 717 | 716,21,60,0,Scenario 8 718 | 717,18,40,0,Scenario 8 719 | 718,27,49,0,Scenario 8 720 | 719,70,21,0,Scenario 8 721 | 720,57,68,0,Scenario 8 722 | 721,18,71,0,Scenario 8 723 | 722,64,29,0,Scenario 8 724 | 723,11,62,0,Scenario 8 725 | 724,24,43,0,Scenario 8 726 | 725,16,71,0,Scenario 8 727 | 726,22,36,0,Scenario 8 728 | 727,17,55,0,Scenario 8 729 | 728,21,42,0,Scenario 8 730 | 729,59,50,0,Scenario 8 731 | 730,65,50,0,Scenario 8 732 | 731,61,67,0,Scenario 8 733 | 732,16,39,0,Scenario 8 734 | 733,12,34,0,Scenario 8 735 | 734,74,57,0,Scenario 8 736 | 735,15,29,0,Scenario 8 737 | 736,54,54,0,Scenario 8 738 | 737,54,26,0,Scenario 8 739 | 738,58,49,0,Scenario 8 740 | 739,10,37,0,Scenario 8 741 | 740,21,31,0,Scenario 8 742 | 741,24,43,0,Scenario 8 743 | 742,67,67,0,Scenario 8 744 | 743,27,43,0,Scenario 8 745 | 744,56,42,0,Scenario 8 746 | 745,56,45,0,Scenario 8 747 | 746,48,71,0,Scenario 8 748 | 747,54,35,0,Scenario 8 749 | 748,63,45,0,Scenario 8 750 | 749,54,35,0,Scenario 8 751 | 750,18,50,0,Scenario 8 752 | 751,79,44,1,Scenario 8 753 | 752,76,49,1,Scenario 8 754 | 753,80,54,1,Scenario 8 755 | 754,41,51,1,Scenario 8 756 | 755,47,58,1,Scenario 8 757 | 756,74,38,1,Scenario 8 758 | 757,75,49,1,Scenario 8 759 | 758,40,73,1,Scenario 8 760 | 759,76,47,1,Scenario 8 761 | 760,82,50,1,Scenario 8 762 | 761,70,41,1,Scenario 8 763 | 762,44,37,1,Scenario 8 764 | 763,78,30,1,Scenario 8 765 | 764,92,76,1,Scenario 8 766 | 765,79,47,1,Scenario 8 767 | 766,70,43,1,Scenario 8 768 | 767,82,41,1,Scenario 8 769 | 768,39,58,1,Scenario 8 770 | 769,40,66,1,Scenario 8 771 | 770,62,32,1,Scenario 8 772 | 771,42,49,1,Scenario 8 773 | 772,35,46,1,Scenario 8 774 | 773,80,38,1,Scenario 8 775 | 774,50,65,1,Scenario 8 776 | 775,45,69,1,Scenario 8 777 | 776,27,61,1,Scenario 8 778 | 777,71,66,1,Scenario 8 779 | 778,28,50,1,Scenario 8 780 | 779,92,28,1,Scenario 8 781 | 780,42,33,1,Scenario 8 782 | 781,84,53,1,Scenario 8 783 | 782,83,42,1,Scenario 8 784 | 783,27,57,1,Scenario 8 785 | 784,82,23,1,Scenario 8 786 | 785,34,23,1,Scenario 8 787 | 786,82,41,1,Scenario 8 788 | 787,88,47,1,Scenario 8 789 | 788,48,43,1,Scenario 8 790 | 789,28,40,1,Scenario 8 791 | 790,38,56,1,Scenario 8 792 | 791,44,49,1,Scenario 8 793 | 792,35,35,1,Scenario 8 794 | 793,30,47,1,Scenario 8 795 | 794,87,16,1,Scenario 8 796 | 795,69,48,1,Scenario 8 797 | 796,18,43,1,Scenario 8 798 | 797,29,68,1,Scenario 8 799 | 798,41,56,1,Scenario 8 800 | 799,41,76,1,Scenario 8 801 | 800,88,37,1,Scenario 8 802 | 801,57,49,0,Scenario 9 803 | 802,59,58,0,Scenario 9 804 | 803,47,49,0,Scenario 9 805 | 804,72,54,0,Scenario 9 806 | 805,43,53,0,Scenario 9 807 | 806,43,48,0,Scenario 9 808 | 807,53,40,0,Scenario 9 809 | 808,26,47,0,Scenario 9 810 | 809,36,52,0,Scenario 9 811 | 810,37,43,0,Scenario 9 812 | 811,71,49,0,Scenario 9 813 | 812,51,43,0,Scenario 9 814 | 813,42,51,0,Scenario 9 815 | 814,34,52,0,Scenario 9 816 | 815,42,49,0,Scenario 9 817 | 816,41,59,0,Scenario 9 818 | 817,50,45,0,Scenario 9 819 | 818,62,44,0,Scenario 9 820 | 819,53,40,0,Scenario 9 821 | 820,31,51,0,Scenario 9 822 | 821,60,51,0,Scenario 9 823 | 822,48,48,0,Scenario 9 824 | 823,29,46,0,Scenario 9 825 | 824,43,55,0,Scenario 9 826 | 825,55,41,0,Scenario 9 827 | 826,55,48,0,Scenario 9 828 | 827,40,53,0,Scenario 9 829 | 828,65,55,0,Scenario 9 830 | 829,38,48,0,Scenario 9 831 | 830,55,55,0,Scenario 9 832 | 831,43,49,0,Scenario 9 833 | 832,34,44,0,Scenario 9 834 | 833,61,57,0,Scenario 9 835 | 834,49,55,0,Scenario 9 836 | 835,55,47,0,Scenario 9 837 | 836,55,58,0,Scenario 9 838 | 837,49,58,0,Scenario 9 839 | 838,13,54,0,Scenario 9 840 | 839,51,49,0,Scenario 9 841 | 840,51,40,0,Scenario 9 842 | 841,47,52,0,Scenario 9 843 | 842,71,47,0,Scenario 9 844 | 843,39,47,0,Scenario 9 845 | 844,63,52,0,Scenario 9 846 | 845,43,53,0,Scenario 9 847 | 846,51,55,0,Scenario 9 848 | 847,40,48,0,Scenario 9 849 | 848,44,43,0,Scenario 9 850 | 849,54,51,0,Scenario 9 851 | 850,50,49,0,Scenario 9 852 | 851,45,54,1,Scenario 9 853 | 852,47,45,1,Scenario 9 854 | 853,48,62,1,Scenario 9 855 | 854,60,57,1,Scenario 9 856 | 855,48,68,1,Scenario 9 857 | 856,52,45,1,Scenario 9 858 | 857,46,26,1,Scenario 9 859 | 858,49,57,1,Scenario 9 860 | 859,47,77,1,Scenario 9 861 | 860,46,35,1,Scenario 9 862 | 861,49,50,1,Scenario 9 863 | 862,48,82,1,Scenario 9 864 | 863,54,51,1,Scenario 9 865 | 864,53,48,1,Scenario 9 866 | 865,54,56,1,Scenario 9 867 | 866,54,30,1,Scenario 9 868 | 867,51,48,1,Scenario 9 869 | 868,58,45,1,Scenario 9 870 | 869,57,21,1,Scenario 9 871 | 870,58,40,1,Scenario 9 872 | 871,55,67,1,Scenario 9 873 | 872,61,62,1,Scenario 9 874 | 873,47,42,1,Scenario 9 875 | 874,55,41,1,Scenario 9 876 | 875,55,32,1,Scenario 9 877 | 876,45,46,1,Scenario 9 878 | 877,46,35,1,Scenario 9 879 | 878,53,29,1,Scenario 9 880 | 879,55,62,1,Scenario 9 881 | 880,52,41,1,Scenario 9 882 | 881,53,50,1,Scenario 9 883 | 882,50,56,1,Scenario 9 884 | 883,49,24,1,Scenario 9 885 | 884,47,52,1,Scenario 9 886 | 885,44,46,1,Scenario 9 887 | 886,53,45,1,Scenario 9 888 | 887,45,80,1,Scenario 9 889 | 888,45,50,1,Scenario 9 890 | 889,49,49,1,Scenario 9 891 | 890,55,62,1,Scenario 9 892 | 891,49,52,1,Scenario 9 893 | 892,51,67,1,Scenario 9 894 | 893,42,71,1,Scenario 9 895 | 894,47,75,1,Scenario 9 896 | 895,50,35,1,Scenario 9 897 | 896,53,34,1,Scenario 9 898 | 897,51,43,1,Scenario 9 899 | 898,49,36,1,Scenario 9 900 | 899,49,51,1,Scenario 9 901 | 900,53,73,1,Scenario 9 902 | -------------------------------------------------------------------------------- /results/linear_model_1_input_15.csv: -------------------------------------------------------------------------------- 1 | ,x,y,from 2 | 0,9,-237,train 3 | 1,81,-1794,train 4 | 2,62,-1176,train 5 | 3,20,-383,train 6 | 4,72,-1800,train 7 | 5,83,-1728,train 8 | 6,15,-386,train 9 | 7,8,-131,train 10 | 8,92,-1982,train 11 | 9,32,-742,train 12 | 10,35,-711,train 13 | 11,33,-778,train 14 | 12,60,-1207,train 15 | 13,4,-75,train 16 | 14,54,-1272,train 17 | 15,59,-1214,gpt 18 | 16,82,-1804,gpt 19 | 17,79,-1763,gpt 20 | 18,44,-928,gpt 21 | 19,46,-1078,gpt 22 | 20,49,-1113,gpt 23 | 21,68,-1404,gpt 24 | 22,29,-621,gpt 25 | 23,11,-297,gpt 26 | 24,38,-829,gpt 27 | 25,42,-826,gpt 28 | 26,28,-632,gpt 29 | 27,91,-1961,gpt 30 | 28,13,-403,gpt 31 | 29,24,-504,gpt 32 | 30,71,-1607,gpt 33 | 31,89,-1845,gpt 34 | 32,76,-1836,gpt 35 | 33,67,-1404,gpt 36 | 34,70,-1400,gpt 37 | 35,6,-81,gpt 38 | 36,87,-1845,gpt 39 | 37,37,-829,gpt 40 | 38,66,-1406,gpt 41 | 39,40,-832,gpt 42 | 40,7,-143,gpt 43 | 41,99,-2099,gpt 44 | 42,45,-957,gpt 45 | 43,84,-1804,gpt 46 | 44,96,-2096,gpt 47 | -------------------------------------------------------------------------------- /results/linear_model_2_input_15.csv: -------------------------------------------------------------------------------- 1 | ,x,y,from 2 | 0,58,1150,train 3 | 1,32,1031,train 4 | 2,37,1236,train 5 | 3,64,1984,train 6 | 4,66,1742,train 7 | 5,45,1074,train 8 | 6,53,1272,train 9 | 7,39,1025,train 10 | 8,55,1526,train 11 | 9,31,714,train 12 | 10,47,1161,train 13 | 11,69,1828,train 14 | 12,62,1619,train 15 | 13,68,1864,train 16 | 14,42,906,train 17 | 15,77,2122,gpt 18 | 16,72,2028,gpt 19 | 17,75,2125,gpt 20 | 18,19,519,gpt 21 | 19,89,2189,gpt 22 | 20,80,2160,gpt 23 | 21,13,513,gpt 24 | 22,61,1561,gpt 25 | 23,5,5,gpt 26 | 24,78,2124,gpt 27 | 25,95,3142,gpt 28 | 26,36,816,gpt 29 | 27,96,2496,gpt 30 | 28,26,526,gpt 31 | 29,70,2070,gpt 32 | 30,74,2124,gpt 33 | 31,24,576,gpt 34 | 32,98,3108,gpt 35 | 33,16,496,gpt 36 | 34,56,1488,gpt 37 | 35,67,1767,gpt 38 | 36,59,1397,gpt 39 | 37,92,2244,gpt 40 | 38,12,612,gpt 41 | 39,86,2116,gpt 42 | 40,8,64,gpt 43 | 41,29,571,gpt 44 | 42,4,0,gpt 45 | 43,46,1138,gpt 46 | 44,18,618,gpt 47 | -------------------------------------------------------------------------------- /results/quadratic_model_1_input_25.csv: -------------------------------------------------------------------------------- 1 | ,x,y,from 2 | 0,46,-9010,train 3 | 1,38,-5774,train 4 | 2,76,-26476,train 5 | 3,61,-16140,train 6 | 4,43,-7406,train 7 | 5,54,-13280,train 8 | 6,21,-1536,train 9 | 7,59,-14381,train 10 | 8,57,-14081,train 11 | 9,52,-11140,train 12 | 10,80,-29202,train 13 | 11,39,-6097,train 14 | 12,34,-4809,train 15 | 13,42,-6596,train 16 | 14,44,-8799,train 17 | 15,75,-25642,train 18 | 16,69,-20838,train 19 | 17,45,-8902,train 20 | 18,24,-1221,train 21 | 19,50,-10593,train 22 | 20,30,-3941,train 23 | 21,37,-5134,train 24 | 22,65,-19102,train 25 | 23,55,-12829,train 26 | 24,20,-2404,train 27 | 25,28,-4104,gpt 28 | 26,97,-35984,gpt 29 | 27,35,-5010,gpt 30 | 28,11,-1110,gpt 31 | 29,64,-18096,gpt 32 | 30,41,-7291,gpt 33 | 31,32,-5184,gpt 34 | 32,85,-33280,gpt 35 | 33,2,-4,gpt 36 | 34,60,-15120,gpt 37 | 35,63,-16092,gpt 38 | 36,98,-40984,gpt 39 | 37,91,-35894,gpt 40 | 38,13,-2048,gpt 41 | 39,40,-6120,gpt 42 | 40,58,-14098,gpt 43 | 41,62,-16092,gpt 44 | 42,70,-23152,gpt 45 | 43,67,-18093,gpt 46 | 44,4,-8,gpt 47 | 45,16,-3136,gpt 48 | 46,84,-33248,gpt 49 | 47,86,-35592,gpt 50 | 48,66,-19082,gpt 51 | 49,14,-3136,gpt 52 | 50,77,-27074,gpt 53 | 51,79,-29984,gpt 54 | 52,49,-10983,gpt 55 | 53,19,-1841,gpt 56 | 54,68,-20892,gpt 57 | -------------------------------------------------------------------------------- /results/quadratic_model_2_input_25_smaller_variance.csv: -------------------------------------------------------------------------------- 1 | ,x,y,from 2 | 0,52,-1938,train 3 | 1,28,-1702,train 4 | 2,25,-1625,train 5 | 3,34,-1843,train 6 | 4,31,-1772,train 7 | 5,55,-1829,train 8 | 6,36,-1857,train 9 | 7,39,-1878,train 10 | 8,24,-1564,train 11 | 9,62,-1649,train 12 | 10,69,-1287,train 13 | 11,61,-1588,train 14 | 12,33,-1845,train 15 | 13,67,-1352,train 16 | 14,48,-1998,train 17 | 15,46,-1917,train 18 | 16,42,-1934,train 19 | 17,29,-1725,train 20 | 18,44,-1904,train 21 | 19,43,-1877,train 22 | 20,75,-971,train 23 | 21,64,-1596,train 24 | 22,40,-1984,train 25 | 23,41,-1964,train 26 | 24,72,-1200,train 27 | 25,16,-1804,gpt 28 | 26,4,-2096,gpt 29 | 27,65,-1605,gpt 30 | 28,58,-1621,gpt 31 | 29,20,-1680,gpt 32 | 30,94,-879,gpt 33 | 31,22,-1604,gpt 34 | 32,5,-1625,gpt 35 | 33,50,-1850,gpt 36 | 34,6,-1606,gpt 37 | 35,7,-1021,gpt 38 | 36,18,-1404,gpt 39 | 37,97,-879,gpt 40 | 38,95,-879,gpt 41 | 39,84,-1056,gpt 42 | 40,88,-1056,gpt 43 | 41,10,-1810,gpt 44 | 42,99,-879,gpt 45 | 43,79,-1056,gpt 46 | 44,3,-2097,gpt 47 | 45,23,-1551,gpt 48 | 46,92,-829,gpt 49 | 47,37,-1877,gpt 50 | 48,45,-1909,gpt 51 | 49,91,-829,gpt 52 | 50,17,-1413,gpt 53 | 51,49,-1926,gpt 54 | 52,82,-1056,gpt 55 | 53,80,-1056,gpt 56 | 54,78,-1056,gpt 57 | 55,15,-1815,gpt 58 | 56,57,-1677,gpt 59 | 57,63,-1604,gpt 60 | 58,54,-1795,gpt 61 | 59,19,-1601,gpt 62 | -------------------------------------------------------------------------------- /run_all_experiments.py: -------------------------------------------------------------------------------- 1 | """ 2 | Just runs all the experiments in experiments_log.json which have 3 | not yet been run. 4 | 5 | Note: please uncomment the line which retrieves your OpenAI API key 6 | if you wish to send requests to OpenAI API. This script costs actual 7 | money if your experiments_log.json has any experiments which have not 8 | been run, and you have an API key saved on your system. 9 | """ 10 | 11 | 12 | import json 13 | import openai 14 | import re 15 | import os 16 | from utils import textify_numbers 17 | import argparse 18 | 19 | with open('experiments_log.json', 'r') as file: 20 | experiments = json.loads(file.read()) 21 | 22 | #openai.api_key = os.getenv("OPENAI_API_KEY") 23 | 24 | engines = ['ada', 'babbage', 'curie', 'davinci'] 25 | for engine in engines: 26 | for experiment_name in experiments.keys(): 27 | if (f'output_test_raw_{engine}' in experiments[experiment_name].keys() 28 | or 'output_test_raw' in experiments[experiment_name].keys()): 29 | print(f"Have already performed {experiment_name} with this engine" 30 | " skipping it now.") 31 | continue 32 | 33 | print(f"Running experiment {experiment_name} with engine {engine}. . .") 34 | 35 | experiment = experiments[experiment_name] 36 | experiment[f'response_{engine}'] = [] 37 | experiment[f'output_test_raw_{engine}'] = [] 38 | experiment[f'output_test_cleaned_{engine}'] = [] 39 | 40 | for point in experiment['input_test']: 41 | point = textify_numbers(point) 42 | prompt_text = ( 43 | experiment['input_text'] 44 | + f'Input = {point}, output =' 45 | ) 46 | response = openai.Completion.create(engine=engine, 47 | prompt=prompt_text, max_tokens=6, 48 | temperature=0, top_p=0) 49 | 50 | experiment[f'response_{engine}'].append(response) 51 | 52 | response_text = response['choices'][0]['text'] 53 | experiment[f'output_test_raw_{engine}'].append(response_text) 54 | 55 | experiment[f'output_test_cleaned_{engine}'].append( 56 | int( 57 | re.findall('-?\d+',response_text 58 | )[0] 59 | ) 60 | ) 61 | 62 | experiments[experiment_name] = experiment 63 | 64 | with open('experiments_log.json', 'w') as file: 65 | json.dump(experiments, file, indent=4) 66 | -------------------------------------------------------------------------------- /text_freq_classifier.py: -------------------------------------------------------------------------------- 1 | """ 2 | In order to see how good GPT-3 classification is, this is a classifier which 3 | I though of in a few seconds, which classifies numbers by taking their first 4 | digit into consideration, and then doing statistics/voting based on it. 5 | 6 | Note that, while seemingly only working on pattern matching digits, 7 | this classifier actually implicitly computes some (statistics of) 8 | distances as well. 9 | """ 10 | 11 | 12 | import json 13 | from collections import defaultdict 14 | from statistics import mean 15 | experiment_names = [f'2d_class_type_{x}_rstate_' for x in range(1,10)] 16 | 17 | results = dict() 18 | 19 | rstates = ['42', '55', '93'] 20 | with open('experiments_log.json', 'r') as file: 21 | experiments = json.loads(file.read()) 22 | 23 | 24 | accuracies2 = [] 25 | for name in experiment_names: 26 | accuracies = [] 27 | for rstate in rstates: 28 | experiment = experiments[name + rstate] 29 | train = experiment['input_train'] 30 | labels = experiment['output_train'] 31 | train_reduced = [] 32 | for it in train: 33 | x, y = it 34 | x, y = x // 10, y // 10 35 | train_reduced.append([x,y]) 36 | 37 | test_reduced = [] 38 | test = experiment['input_test'] 39 | for it in test: 40 | x, y = it 41 | x, y = x // 10, y // 10 42 | test_reduced.append([x,y]) 43 | 44 | predictions = [] 45 | for it in test_reduced: 46 | class_ = defaultdict(int) 47 | x, y = it 48 | for ix, itt in enumerate(train_reduced): 49 | xx, yy = itt 50 | if xx == x: 51 | class_[labels[ix]] += 1 52 | if yy == y: 53 | class_[labels[ix]] +=1 54 | if class_[0] > class_[1]: 55 | predictions.append(0) 56 | else: 57 | predictions.append(1) 58 | accurate = [1 if x==y else 0 59 | for x, y in zip(predictions, experiment['output_test'])] 60 | accuracies.append(sum(accurate)/len(accurate)) 61 | print(name, mean(accuracies)) 62 | accuracies2.append(mean(accuracies)) 63 | 64 | print(mean(accuracies2)) 65 | -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | Some utils for other scripts. Actually, just one for now, a function 3 | to transfer either number of lists of numbers into strings of the 4 | desired form. 5 | """ 6 | 7 | def textify_numbers(nums): 8 | if hasattr(nums, '__iter__'): 9 | input = ', '.join(map(str, nums)) 10 | else: 11 | input = str(nums) 12 | return input 13 | -------------------------------------------------------------------------------- /visualizations.R: -------------------------------------------------------------------------------- 1 | library("tidyverse") 2 | library("lubridate") 3 | library("tidytext") 4 | library("scales") 5 | library("ggridges") 6 | library("ggrepel") 7 | library("ggthemes") 8 | theme_set(theme_bw()) 9 | 10 | dat <- read.csv("results/distribution_samples.csv", 11 | header= TRUE, 12 | stringsAsFactors = FALSE, 13 | sep = ",") 14 | 15 | dat$clas <- as.character(dat$clas) 16 | ggplot(dat, aes(x=x, y=y, color=clas)) + 17 | geom_point(size=1) + 18 | facet_wrap(~ type, ncol=3) + 19 | scale_color_discrete(name="Class") + 20 | labs(title="", x="", y="") + 21 | theme(legend.position = "none", 22 | panel.border = element_rect(colour="black", size=0.8), 23 | strip.background = element_rect(colour="#fe5185", 24 | fill="white", size=0.8), 25 | axis.text.x=element_text(colour="black"), 26 | axis.text.y=element_text(colour="black"), 27 | axis.ticks = element_line(colour = 'black', size = 0.8, linetype = 'dashed') 28 | ) 29 | 30 | ggsave('plots/distribucije.png', width = 6, height = 6, 31 | dpi = 200, units = "in") 32 | 33 | dat2 <- read.csv('results/differences.csv', 34 | header=TRUE, 35 | sep=",") 36 | dat2 %>% pivot_longer(-X) %>% 37 | ggplot(aes(x=name,y=value,fill=name))+ 38 | labs(title="", x="", y="Accuracy compared to kNN's") + 39 | geom_boxplot(alpha=0.8)+ 40 | geom_jitter(color="black", size=0.4, alpha=0.8)+ 41 | scale_y_continuous(labels = scales::percent) + 42 | theme(legend.position=c(0.2, 0.18), 43 | legend.title=element_blank(), 44 | legend.box.background = element_rect(colour = "purple", size=1), 45 | legend.margin=margin(t=-0.18,l=0.05,b=0.0,r=0.05, unit='cm')) 46 | 47 | ggsave('plots/razlike.png', width = 6, height = 6, 48 | dpi = 200, units = "in") 49 | 50 | reg1 <- read.csv('results/linear_model_1_input_15.csv', 51 | header=TRUE, 52 | sep=",") 53 | 54 | ggplot(reg1, aes(x=x, y=y, color=from)) + 55 | geom_point(size=1.5) + 56 | geom_function(aes(colour="True regression fn."), 57 | fun = ~ -22*.x + 45) + 58 | scale_color_hue(name = "", 59 | labels = c("GPT-3 predictions", 60 | "Training points", 61 | "True regression fn.")) + 62 | theme(legend.position=c(0.8, 0.8), 63 | legend.title=element_blank(), 64 | legend.box.background = element_rect(colour = "purple", size=1), 65 | legend.margin=margin(t=-0.18,l=0.05,b=0.0,r=0.05, unit='cm'), 66 | axis.title.x=element_blank(), 67 | axis.title.y=element_blank()) 68 | 69 | ggsave('plots/reg1.png', width = 6, height = 6, 70 | dpi = 200, units = "in") 71 | 72 | 73 | reg2 <- read.csv('results/linear_model_2_input_15.csv', 74 | header=TRUE, 75 | sep=",") 76 | 77 | ggplot(reg2, aes(x=x, y=y, color=from)) + 78 | geom_point(size=1.5) + 79 | geom_function(aes(colour="True regression fn."), 80 | fun = ~ 24*.x + 95) + 81 | scale_color_hue(labels = c("GPT-3 predictions", 82 | "Training points", 83 | "True regression fn.")) + 84 | geom_vline(xintercept=30, linetype='dotted', col = 'purple', label="Training bound") + 85 | geom_vline(xintercept=70, linetype='dotted', col = 'purple') + 86 | theme(legend.position=c(0.8, 0.2), 87 | legend.title=element_blank(), 88 | legend.box.background = element_rect(colour = "purple", size=1), 89 | legend.margin=margin(t=-0.18,l=0.05,b=0.0,r=0.05, unit='cm'), 90 | axis.title.x=element_blank(), 91 | axis.title.y=element_blank()) 92 | 93 | ggsave('plots/reg2.png', width = 6, height = 6, 94 | dpi = 200, units = "in") 95 | 96 | reg3 <- read.csv('results/quadratic_model_2_input_25_smaller_variance.csv', 97 | header=TRUE, 98 | sep=",") 99 | 100 | ggplot(reg3, aes(x=x, y=y, color=from)) + 101 | geom_point(size=1.5) + 102 | geom_function(aes(colour="True regression fn."), 103 | fun = ~ (.x**2) - 88*.x + 3) + 104 | scale_color_hue(labels = c("GPT-3 predictions", 105 | "Training points", 106 | "True regression fn.")) + 107 | geom_vline(xintercept=24, linetype='dotted', col = 'purple', label="Training bound") + 108 | geom_vline(xintercept=75, linetype='dotted', col = 'purple') + 109 | theme(legend.position=c(0.2, 0.8), 110 | legend.title=element_blank(), 111 | legend.box.background = element_rect(colour = "purple", size=1), 112 | legend.margin=margin(t=-0.18,l=0.05,b=0.0,r=0.05, unit='cm'), 113 | axis.title.x=element_blank(), 114 | axis.title.y=element_blank()) 115 | 116 | ggsave('plots/reg3.png', width = 6, height = 6, 117 | dpi = 200, units = "in") 118 | 119 | 120 | reg4 <- read.csv('results/quadratic_model_1_input_25.csv', 121 | header=TRUE, 122 | sep=",") 123 | 124 | 125 | ggplot(reg4, aes(x=x, y=y, color=from)) + 126 | geom_point(size=1.5) + 127 | geom_function(aes(colour="True regression fn."), 128 | fun = ~ (-5)*(.x**2) + 35*.x + 201) + 129 | scale_color_hue(labels = c("GPT-3 predictions", 130 | "Training points", 131 | "True regression fn.")) + 132 | geom_vline(xintercept=20, linetype='dotted', col = 'purple', label="Training bound") + 133 | geom_vline(xintercept=80, linetype='dotted', col = 'purple') + 134 | theme(legend.position=c(0.2, 0.2), 135 | legend.title=element_blank(), 136 | legend.box.background = element_rect(colour = "purple", size=1), 137 | legend.margin=margin(t=-0.18,l=0.05,b=0.0,r=0.05, unit='cm'), 138 | axis.title.x=element_blank(), 139 | axis.title.y=element_blank()) 140 | 141 | ggsave('plots/reg4.png', width = 6, height = 6, 142 | dpi = 200, units = "in") 143 | --------------------------------------------------------------------------------