├── LICENSE ├── README.md ├── cactus.py ├── defaults.json ├── examples ├── cactus.pdf ├── cactus.png ├── cactus.svg ├── csv-data.csv ├── scatter.pdf ├── scatter.png ├── scatter.svg ├── solver1.json └── solver2.json ├── load.py ├── mkplot.py ├── plot.py ├── scatter.py └── statutil.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Alexey Ignatiev 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # mkplot 2 | 3 | A Python script to create cactus and scatter plots based on the [matplotlib](http://matplotlib.org/) plotting library. The script makes use of only a tiny subset of what matplotlib can do. 4 | 5 | mkplot was originally designed to make my life a lot easier by automating the plotting tasks, which a typical CS researcher has to deal with quite often. After a few years of using the script, I realized that it may be helpful for some other people (hopefully!) and so to make it publicly available. Enjoy! :) 6 | 7 | ## Getting Started 8 | 9 | First of all, make sure you have a Python interpreter installed. To run the script, you also need to install the [matplotlib](http://matplotlib.org/) library. Please, see the corresponding [installation instructions](http://matplotlib.org/users/installing.html). Once matplotlib is installed on your computer, you can start using mkplot. 10 | 11 | ## Usage 12 | 13 | The script has a number of parameters, which can be set from the command line. To see the list of options, run: 14 | 15 | ``` 16 | mkplot.py -h 17 | ``` 18 | 19 | ### Input Format 20 | 21 | The data to plot can be given in one of the two formats: 22 | 23 | * [JSON](https://en.wikipedia.org/wiki/JSON) 24 | * [CSV](https://en.wikipedia.org/wiki/Comma-separated_values) 25 | 26 | While the CSV format is a simple table of values aggregating all the data, the preferred format is a series of JSON files, which describe the data for an individual tool/solver following this example: 27 | 28 | ```json 29 | { 30 | "preamble": { 31 | "program": "program-name", 32 | "prog_args": "-a some -b program --arguments", 33 | "prog_alias": "some-short-alias-to-use-in-the-plot", 34 | "benchmark": "name-of-benchmark-set" 35 | }, 36 | "stats": { 37 | "some_problem_instance": { 38 | "status": true, 39 | "rtime": 10.567, 40 | "mempeak": "171864 KiB", 41 | "some-other-key": "key-value1" 42 | }, 43 | "another_problem_instance": { 44 | "status": false, 45 | "rtime": 1000.00, 46 | "mempeak": "245759 KiB", 47 | "some-other-key": "key-value2" 48 | }, 49 | "instance3": { 50 | "status": true, 51 | "rtime": 256.32, 52 | "mempeak": "57261 KiB", 53 | "some-other-key": "key-value3" 54 | } 55 | } 56 | } 57 | ``` 58 | 59 | Here, the data describes the result of running a tool referred to as *"program-name"* with the given list of command-line arguments on a benchmark set called *"name-of-benchmark-set"* containing three problem instances. The result for each instance **must** have the information on its status: *true* or *false* meaning that the instance is solved or unsolved, respectively. All the other fields are non-mandatory (you can use whatever key/value you want). However, note that the *rtime* key is used by default when working with JSON files (to change this use the `-k` option). 60 | 61 | For further details of the input format, please, see the [example files](examples). 62 | 63 | ### Using mkplot 64 | 65 | A few usage examples of mkplot follow. 66 | 67 | For instance, running 68 | 69 | ``` 70 | mkplot.py -l --legend prog_alias -t 1000 -b png --save-to examples/cactus.png examples/solver?.json 71 | ``` 72 | 73 | results in a simple cactus plot showing the performance of `very-nice-solver` and `another-good-tool`: ![cactus plot](examples/cactus.png) 74 | 75 | Here, mkplot is set to show program aliases in the legend instead of their complete names. 76 | 77 | If you need to create a scatter plot detailing the difference between the two solvers, just do 78 | 79 | ``` 80 | mkplot.py -l -p scatter -b png --save-to examples/scatter.png --shape squared -t 1000 --ylog --ymax 10000 --ymin 0.1 --xlog examples/csv-data.csv 81 | ``` 82 | 83 | The resulting scatter plot is the following: 84 | 85 | ![a scatter plot](examples/scatter.png) 86 | 87 | Observe that here instead of JSON files, a CSV table is used. 88 | 89 | ## License 90 | 91 | This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. 92 | -------------------------------------------------------------------------------- /cactus.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | #-*- coding:utf-8 -*- 3 | ## 4 | ## cactus.py 5 | ## 6 | ## Created on: Jun 05, 2015 7 | ## Author: Alexey S. Ignatiev 8 | ## E-mail: aignatiev@ciencias.ulisboa.pt 9 | ## 10 | 11 | # 12 | #============================================================================== 13 | import json 14 | import matplotlib.pyplot as plt 15 | from matplotlib import __version__ as mpl_version 16 | import math 17 | import numpy as np 18 | import os 19 | from plot import Plot 20 | import six 21 | 22 | 23 | # 24 | #============================================================================== 25 | class Cactus(Plot, object): 26 | """ 27 | Cactus plot class. 28 | """ 29 | 30 | def __init__(self, options): 31 | """ 32 | Cactus constructor. 33 | """ 34 | 35 | super(Cactus, self).__init__(options) 36 | 37 | with open(self.def_path, 'r') as fp: 38 | self.linestyles = json.load(fp)['cactus_linestyle'] 39 | 40 | def create(self, data): 41 | """ 42 | Does the plotting. 43 | """ 44 | 45 | # making lines 46 | coords = [] 47 | for d in data: 48 | coords.append(np.arange(1, len(d[1]) + 1)) # xs (separate for each line) 49 | coords.append(np.array(sorted(d[1]))) 50 | lines = plt.plot(*coords, zorder=3) 51 | 52 | # setting line styles 53 | if self.byname == False: # by default, assign fist line to best tool 54 | lmap = lambda i: i 55 | else: # assign line styles by tool name 56 | tnames = [(d[0], i) for i, d in enumerate(data)] 57 | tnames.sort(key=lambda pair: pair[0]) 58 | tmap = {tn[1]: i for i, tn in enumerate(tnames)} 59 | lmap = lambda i: tmap[i] 60 | 61 | for i, l in enumerate(lines): 62 | plt.setp(l, **self.linestyles[lmap(i) % len(self.linestyles)]) 63 | 64 | # turning the grid on 65 | if not self.no_grid: 66 | plt.grid(True, color=self.grid_color, ls=self.grid_style, lw=self.grid_width, zorder=1) 67 | 68 | # axes limits 69 | plt.xlim(self.x_min, self.x_max if self.x_max else math.ceil(max([d[2] for d in data]) / float(100)) * 100) 70 | plt.ylim(self.y_min, self.y_max if self.y_max else self.timeout) 71 | 72 | # axes labels 73 | if self.x_label: 74 | plt.xlabel(self.x_label) 75 | else: 76 | plt.xlabel('instances') 77 | 78 | if self.y_label: 79 | plt.ylabel(self.y_label) 80 | else: 81 | plt.ylabel('CPU time (s)') 82 | 83 | # choosing logarithmic scales if needed 84 | ax = plt.gca() 85 | if self.x_log: 86 | ax.set_xscale('log') 87 | if self.y_log: 88 | ax.set_yscale('log') 89 | 90 | # setting ticks 91 | # plt.xticks(np.arange(self.x_min, self.x_max + 1, 2)) 92 | # if not self.y_log: 93 | # # plt.yticks(list(plt.yticks()[0]) + [self.timeout]) 94 | # ax.set_yticks(range(0, 2 * (int(self.y_max) if self.y_max else int(self.timeout)), 200)) 95 | 96 | # setting ticks font properties 97 | # set_*ticklables() seems to be not needed in matplotlib 1.5.0 98 | if float(mpl_version[:3]) < 1.5: 99 | ax.set_xticklabels(ax.get_xticks(), self.f_props) 100 | ax.set_yticklabels(ax.get_yticks(), self.f_props) 101 | 102 | strFormatter = plt.FormatStrFormatter('%d') 103 | logFormatter = plt.LogFormatterMathtext(base=10) 104 | ax.xaxis.set_major_formatter(strFormatter if not self.x_log else logFormatter) 105 | ax.yaxis.set_major_formatter(strFormatter if not self.y_log else logFormatter) 106 | 107 | # making the legend 108 | if self.lgd_loc != 'off': 109 | lgtext = [d[0] for d in data] 110 | lg = ax.legend(lines, lgtext, ncol=self.lgd_ncol, loc=self.lgd_loc, fancybox=self.lgd_fancy, shadow=self.lgd_shadow if self.lgd_alpha == 1.0 else False) 111 | fr = lg.get_frame() 112 | fr.set_lw(1) 113 | fr.set_alpha(self.lgd_alpha) 114 | fr.set_edgecolor('black') 115 | 116 | # setting frame thickness 117 | for i in six.itervalues(ax.spines): 118 | i.set_linewidth(1) 119 | 120 | plt.savefig(self.save_to, bbox_inches='tight', transparent=self.transparent) 121 | -------------------------------------------------------------------------------- /defaults.json: -------------------------------------------------------------------------------- 1 | { 2 | "cactus_linestyle": 3 | [ 4 | {"c": "green", "marker": "H", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "green", "mew": 0.75}, 5 | {"c": "red", "marker": "^", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "red", "mew": 0.75}, 6 | {"c": "blue", "marker": "x", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "blue", "mew": 0.75}, 7 | {"c": "brown", "marker": "+", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "brown", "mew": 0.75}, 8 | {"c": "orange", "marker": "D", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "orange", "mew": 0.75}, 9 | {"c": "magenta", "marker": "*", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "magenta", "mew": 0.75}, 10 | {"c": "cyan", "marker": "o", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "cyan", "mew": 0.75}, 11 | {"c": "black", "marker": "d", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "black", "mew": 0.75}, 12 | {"c": "#666aee", "marker": "v", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "#666aee", "mew": 0.75}, 13 | {"c": "grey", "marker": ">", "ms": 5, "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "grey", "mew": 0.75}, 14 | {"c": "green", "marker": "^", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "green", "mew": 0.75}, 15 | {"c": "red", "marker": "H", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "red", "mew": 0.75}, 16 | {"c": "blue", "marker": "+", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "blue", "mew": 0.75}, 17 | {"c": "brown", "marker": "x", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "brown", "mew": 0.75}, 18 | {"c": "orange", "marker": "*", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "orange", "mew": 0.75}, 19 | {"c": "magenta", "marker": "D", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "magenta", "mew": 0.75}, 20 | {"c": "cyan", "marker": "d", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "cyan", "mew": 0.75}, 21 | {"c": "black", "marker": "o", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "black", "mew": 0.75}, 22 | {"c": "#666aee", "marker": ">", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "#666aee", "mew": 0.75}, 23 | {"c": "grey", "marker": "v", "ms": 5, "ls": "--", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "grey", "mew": 0.75}, 24 | {"c": "green", "marker": "v", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "green", "mew": 0.75}, 25 | {"c": "red", "marker": ">", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "red", "mew": 0.75}, 26 | {"c": "blue", "marker": "o", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "blue", "mew": 0.75}, 27 | {"c": "brown", "marker": "d", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "brown", "mew": 0.75}, 28 | {"c": "orange", "marker": "D", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "orange", "mew": 0.75}, 29 | {"c": "magenta", "marker": "*", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "magenta", "mew": 0.75}, 30 | {"c": "cyan", "marker": "x", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "cyan", "mew": 0.75}, 31 | {"c": "black", "marker": "+", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "black", "mew": 0.75}, 32 | {"c": "#666aee", "marker": "H", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "#666aee", "mew": 0.75}, 33 | {"c": "grey", "marker": "^", "ms": 5, "ls": ":", "lw": 1, "alpha": 0.7, "mfc": "white", "mec": "grey", "mew": 0.75} 34 | ], 35 | "scatter_style": 36 | { 37 | "color": "r", 38 | "edgecolor": "black", 39 | "marker": "o", 40 | "size": 25 41 | }, 42 | "settings": 43 | { 44 | "alpha": 0.3, 45 | "backend": "pdf", 46 | "by_name": false, 47 | "dry_run": false, 48 | "font": "times", 49 | "font_sz": 12.0, 50 | "no_grid": false, 51 | "grid_color": "black", 52 | "grid_style": ":", 53 | "grid_width": "1", 54 | "join_key": null, 55 | "key": "rtime", 56 | "legend": "program", 57 | "lgd_alpha": 1.0, 58 | "lgd_fancy": true, 59 | "lgd_shadow": true, 60 | "lgd_loc": "upper left", 61 | "lgd_ncol": 1, 62 | "only": null, 63 | "plot_type": "cactus", 64 | "repls": null, 65 | "reverse": false, 66 | "save_to": "plot", 67 | "shape": "standard", 68 | "timeout": 3600.0, 69 | "t_label": null, 70 | "tlb_loc": "after", 71 | "transparent": false, 72 | "usetex": false, 73 | "vbs": null, 74 | "xkcd": false, 75 | "x_label": null, 76 | "x_log": false, 77 | "x_min": 0, 78 | "x_max": null, 79 | "y_label": null, 80 | "y_log": false, 81 | "y_min": 0.000000001, 82 | "y_max": null 83 | } 84 | } 85 | -------------------------------------------------------------------------------- /examples/cactus.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeyignatiev/mkplot/5ce38fcc3ece381cea6d82c273f1b55664b315c9/examples/cactus.pdf -------------------------------------------------------------------------------- /examples/cactus.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeyignatiev/mkplot/5ce38fcc3ece381cea6d82c273f1b55664b315c9/examples/cactus.png -------------------------------------------------------------------------------- /examples/cactus.svg: -------------------------------------------------------------------------------- 1 | 2 | 4 | 5 | 6 | 7 | 10 | 11 | 12 | 13 | 19 | 20 | 21 | 22 | 28 | 29 | 30 | 31 | 32 | 35 | 36 | 37 | 38 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 151 | 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 177 | 178 | 179 | 180 | 181 | 182 | 183 | 184 | 185 | 186 | 189 | 190 | 191 | 192 | 193 | 194 | 195 | 196 | 197 | 198 | 220 | 221 | 222 | 223 | 224 | 225 | 226 | 227 | 228 | 229 | 232 | 233 | 234 | 235 | 236 | 237 | 238 | 239 | 240 | 241 | 257 | 258 | 259 | 260 | 261 | 262 | 263 | 264 | 265 | 266 | 267 | 268 | 288 | 314 | 342 | 361 | 390 | 406 | 423 | 424 | 425 | 426 | 427 | 428 | 429 | 430 | 431 | 432 | 433 | 434 | 435 | 436 | 437 | 438 | 439 | 440 | 443 | 444 | 445 | 446 | 449 | 450 | 451 | 452 | 453 | 454 | 455 | 456 | 457 | 458 | 459 | 460 | 461 | 462 | 463 | 466 | 467 | 468 | 469 | 470 | 471 | 472 | 473 | 474 | 475 | 476 | 477 | 478 | 479 | 480 | 481 | 482 | 483 | 486 | 487 | 488 | 489 | 490 | 491 | 492 | 493 | 494 | 495 | 496 | 497 | 498 | 499 | 500 | 501 | 502 | 503 | 506 | 507 | 508 | 509 | 510 | 511 | 512 | 513 | 514 | 515 | 516 | 517 | 518 | 519 | 520 | 521 | 522 | 523 | 526 | 527 | 528 | 529 | 530 | 531 | 532 | 533 | 534 | 535 | 536 | 537 | 538 | 539 | 540 | 541 | 542 | 543 | 546 | 547 | 548 | 549 | 550 | 551 | 552 | 553 | 554 | 555 | 556 | 557 | 558 | 559 | 560 | 561 | 562 | 563 | 564 | 565 | 585 | 608 | 631 | 670 | 682 | 694 | 695 | 696 | 697 | 698 | 699 | 700 | 701 | 702 | 703 | 704 | 705 | 706 | 707 | 708 | 709 | 710 | 713 | 714 | 715 | 718 | 719 | 720 | 723 | 724 | 725 | 728 | 729 | 730 | 829 | 830 | 838 | 839 | 840 | 841 | 842 | 843 | 844 | 845 | 846 | 847 | 848 | 849 | 850 | 851 | 852 | 853 | 854 | 855 | 856 | 857 | 858 | 859 | 860 | 861 | 862 | 863 | 864 | 865 | 866 | 867 | 868 | 869 | 870 | 871 | 872 | 873 | 874 | 875 | 876 | 877 | 878 | 879 | 880 | 881 | 882 | 883 | 884 | 885 | 886 | 887 | 888 | 889 | 890 | 891 | 892 | 893 | 894 | 895 | 896 | 897 | 898 | 899 | 900 | 901 | 902 | 903 | 904 | 905 | 906 | 907 | 908 | 909 | 910 | 911 | 912 | 913 | 914 | 915 | 916 | 917 | 918 | 919 | 920 | 921 | 922 | 923 | 924 | 925 | 926 | 927 | 928 | 929 | 930 | 931 | 932 | 933 | 934 | 935 | 936 | 937 | 938 | 939 | 940 | 941 | 1037 | 1038 | 1043 | 1044 | 1045 | 1046 | 1047 | 1048 | 1049 | 1050 | 1051 | 1052 | 1053 | 1054 | 1055 | 1056 | 1057 | 1058 | 1059 | 1060 | 1061 | 1062 | 1063 | 1064 | 1065 | 1066 | 1067 | 1068 | 1069 | 1070 | 1071 | 1072 | 1073 | 1074 | 1075 | 1076 | 1077 | 1078 | 1079 | 1080 | 1081 | 1082 | 1083 | 1084 | 1085 | 1086 | 1087 | 1088 | 1089 | 1090 | 1091 | 1092 | 1093 | 1094 | 1095 | 1096 | 1097 | 1098 | 1099 | 1100 | 1101 | 1102 | 1103 | 1104 | 1105 | 1106 | 1107 | 1108 | 1109 | 1110 | 1111 | 1112 | 1113 | 1114 | 1115 | 1116 | 1117 | 1118 | 1119 | 1120 | 1121 | 1122 | 1123 | 1124 | 1125 | 1126 | 1127 | 1128 | 1129 | 1130 | 1131 | 1132 | 1133 | 1134 | 1135 | 1136 | 1137 | 1138 | 1139 | 1140 | 1141 | 1142 | 1143 | 1144 | 1155 | 1156 | 1157 | 1168 | 1169 | 1170 | 1173 | 1174 | 1175 | 1176 | 1177 | 1178 | 1179 | 1180 | 1181 | 1182 | 1203 | 1204 | 1205 | 1206 | 1207 | 1208 | 1209 | 1210 | 1211 | 1214 | 1215 | 1216 | 1217 | 1218 | 1219 | 1220 | 1221 | 1222 | 1223 | 1262 | 1263 | 1264 | 1265 | 1266 | 1267 | 1268 | 1269 | 1270 | 1271 | 1272 | 1273 | 1274 | 1275 | 1276 | 1277 | 1278 | -------------------------------------------------------------------------------- /examples/csv-data.csv: -------------------------------------------------------------------------------- 1 | instance very-nice-solver another-good-tool 2 | instance1 4.6136 4.6136 3 | instance2 17.197 27.197 4 | instance3 80.8731 125.8731 5 | instance4 66.8706 58.8706 6 | instance5 199.701 287.701 7 | instance6 1.3634 1.3634 8 | instance7 102.321 101.321 9 | instance8 45.8964 71.8964 10 | instance9 58.8135 77.8135 11 | instance10 1.9281 1.9281 12 | instance11 123.5724 165.5724 13 | instance12 128.645 212.645 14 | instance13 117.9125 107.9125 15 | instance14 54.3457 78.3457 16 | instance15 133.7901 213.7901 17 | instance16 143.2799 127.2799 18 | instance17 44.1856 71.1856 19 | instance18 112.592 157.592 20 | instance19 134.6126 120.6126 21 | instance20 2.4147 2.4147 22 | instance21 164.851 252.851 23 | instance22 69.6312 68.6312 24 | instance23 74.5844 109.5844 25 | instance24 196.5466 277.5466 26 | instance25 184.3031 183.3031 27 | instance26 36.4459 51.4459 28 | instance27 5.4189 7.4189 29 | instance28 182.6557 177.6557 30 | instance29 163.3557 266.3557 31 | instance30 126.5003 191.5003 32 | instance31 119.1347 112.1347 33 | instance32 152.3262 238.3262 34 | instance33 141.9863 195.9863 35 | instance34 194.9343 179.9343 36 | instance35 38.2602 56.2602 37 | instance36 176.7886 256.7886 38 | instance37 41.7866 41.7866 39 | instance38 47.516 76.516 40 | instance39 83.8463 127.8463 41 | instance40 112.9365 101.9365 42 | instance41 19.5951 31.5951 43 | instance42 485.5096 664.5096 44 | instance43 231.7043 207.7043 45 | instance44 21.1671 31.1671 46 | instance45 321.2542 515.2542 47 | instance46 398.7485 373.7485 48 | instance47 472.182 764.182 49 | instance48 551.279 825.279 50 | instance49 191.2439 175.2439 51 | instance50 65.6477 90.6477 52 | instance51 54.5806 89.5806 53 | instance52 306.7591 296.7591 54 | instance53 274.3517 426.3517 55 | instance54 160.419 251.419 56 | instance55 387.1159 359.1159 57 | instance56 303.1698 438.1698 58 | instance57 582.7249 888.7249 59 | instance58 306.2356 299.2356 60 | instance59 406.7282 633.7282 61 | instance60 221.8447 314.8447 62 | instance61 445.1688 433.1688 63 | instance62 294.5508 446.5508 64 | instance63 544.9736 864.9736 65 | instance64 84.1849 80.1849 66 | instance65 492.8131 662.8131 67 | instance66 397.1919 605.1919 68 | instance67 487.3995 448.3995 69 | instance68 293.9778 453.9778 70 | instance69 221.7435 354.7435 71 | instance70 590.4992 525.4992 72 | instance71 552.754 783.754 73 | instance72 1000.0 1000.0 74 | instance73 74.5493 70.5493 75 | instance74 543.2072 796.2072 76 | instance75 484.566 786.566 77 | instance76 895.2961 804.2961 78 | instance77 614.2801 994.2801 79 | instance78 590.3233 852.3233 80 | instance79 261.3116 260.3116 81 | instance80 1000.0 1000.0 82 | instance81 673.9072 943.9072 83 | instance82 560.4319 509.4319 84 | instance83 894.45 1000.0 85 | instance84 237.2308 368.2308 86 | instance85 749.3951 724.3951 87 | instance86 242.3616 354.3616 88 | instance87 712.2565 1000.0 89 | instance88 1000.0 875.0 90 | instance89 90.243 137.243 91 | instance90 557.5511 789.5511 92 | instance91 432.9616 428.9616 93 | instance92 510.4372 687.4372 94 | instance93 142.9602 216.9602 95 | instance94 392.9865 365.9865 96 | instance95 428.84 652.84 97 | instance96 772.1114 1000.0 98 | instance97 346.7964 307.7964 99 | instance98 379.4744 610.4744 100 | instance99 788.865 1000.0 101 | instance100 410.851 371.851 102 | -------------------------------------------------------------------------------- /examples/scatter.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeyignatiev/mkplot/5ce38fcc3ece381cea6d82c273f1b55664b315c9/examples/scatter.pdf -------------------------------------------------------------------------------- /examples/scatter.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alexeyignatiev/mkplot/5ce38fcc3ece381cea6d82c273f1b55664b315c9/examples/scatter.png -------------------------------------------------------------------------------- /examples/solver1.json: -------------------------------------------------------------------------------- 1 | { 2 | "stats": { 3 | "instance32": { 4 | "status": true, 5 | "rtime": 152.3262, 6 | "mempeak": "214016 KiB" 7 | }, 8 | "instance33": { 9 | "status": true, 10 | "rtime": 141.9863, 11 | "mempeak": "389120 KiB" 12 | }, 13 | "instance30": { 14 | "status": true, 15 | "rtime": 126.5003, 16 | "mempeak": "245760 KiB" 17 | }, 18 | "instance31": { 19 | "status": true, 20 | "rtime": 119.1347, 21 | "mempeak": "25600 KiB" 22 | }, 23 | "instance36": { 24 | "status": true, 25 | "rtime": 176.7886, 26 | "mempeak": "432128 KiB" 27 | }, 28 | "instance37": { 29 | "status": true, 30 | "rtime": 41.7866, 31 | "mempeak": "162816 KiB" 32 | }, 33 | "instance34": { 34 | "status": true, 35 | "rtime": 194.9343, 36 | "mempeak": "9216 KiB" 37 | }, 38 | "instance35": { 39 | "status": true, 40 | "rtime": 38.2602, 41 | "mempeak": "481280 KiB" 42 | }, 43 | "instance38": { 44 | "status": true, 45 | "rtime": 47.516, 46 | "mempeak": "346112 KiB" 47 | }, 48 | "instance39": { 49 | "status": true, 50 | "rtime": 83.8463, 51 | "mempeak": "256000 KiB" 52 | }, 53 | "instance98": { 54 | "status": true, 55 | "rtime": 379.4744, 56 | "mempeak": "499712 KiB" 57 | }, 58 | "instance99": { 59 | "status": true, 60 | "rtime": 788.865, 61 | "mempeak": "420864 KiB" 62 | }, 63 | "instance94": { 64 | "status": true, 65 | "rtime": 392.9865, 66 | "mempeak": "452608 KiB" 67 | }, 68 | "instance95": { 69 | "status": true, 70 | "rtime": 428.84, 71 | "mempeak": "314368 KiB" 72 | }, 73 | "instance96": { 74 | "status": true, 75 | "rtime": 772.1114, 76 | "mempeak": "4096 KiB" 77 | }, 78 | "instance97": { 79 | "status": true, 80 | "rtime": 346.7964, 81 | "mempeak": "281600 KiB" 82 | }, 83 | "instance90": { 84 | "status": true, 85 | "rtime": 557.5511, 86 | "mempeak": "431104 KiB" 87 | }, 88 | "instance91": { 89 | "status": true, 90 | "rtime": 432.9616, 91 | "mempeak": "450560 KiB" 92 | }, 93 | "instance92": { 94 | "status": true, 95 | "rtime": 510.4372, 96 | "mempeak": "369664 KiB" 97 | }, 98 | "instance93": { 99 | "status": true, 100 | "rtime": 142.9602, 101 | "mempeak": "389120 KiB" 102 | }, 103 | "instance21": { 104 | "status": true, 105 | "rtime": 164.851, 106 | "mempeak": "380928 KiB" 107 | }, 108 | "instance20": { 109 | "status": true, 110 | "rtime": 2.4147, 111 | "mempeak": "260096 KiB" 112 | }, 113 | "instance23": { 114 | "status": true, 115 | "rtime": 74.5844, 116 | "mempeak": "145408 KiB" 117 | }, 118 | "instance22": { 119 | "status": true, 120 | "rtime": 69.6312, 121 | "mempeak": "330752 KiB" 122 | }, 123 | "instance25": { 124 | "status": true, 125 | "rtime": 184.3031, 126 | "mempeak": "502784 KiB" 127 | }, 128 | "instance24": { 129 | "status": true, 130 | "rtime": 196.5466, 131 | "mempeak": "360448 KiB" 132 | }, 133 | "instance27": { 134 | "status": true, 135 | "rtime": 5.4189, 136 | "mempeak": "65536 KiB" 137 | }, 138 | "instance26": { 139 | "status": true, 140 | "rtime": 36.4459, 141 | "mempeak": "457728 KiB" 142 | }, 143 | "instance29": { 144 | "status": true, 145 | "rtime": 163.3557, 146 | "mempeak": "263168 KiB" 147 | }, 148 | "instance28": { 149 | "status": true, 150 | "rtime": 182.6557, 151 | "mempeak": "115712 KiB" 152 | }, 153 | "instance83": { 154 | "status": true, 155 | "rtime": 894.45, 156 | "mempeak": "11264 KiB" 157 | }, 158 | "instance82": { 159 | "status": true, 160 | "rtime": 560.4319, 161 | "mempeak": "289792 KiB" 162 | }, 163 | "instance81": { 164 | "status": true, 165 | "rtime": 673.9072, 166 | "mempeak": "518144 KiB" 167 | }, 168 | "instance80": { 169 | "status": false, 170 | "rtime": 1000.0, 171 | "mempeak": "330752 KiB" 172 | }, 173 | "instance87": { 174 | "status": true, 175 | "rtime": 712.2565, 176 | "mempeak": "271360 KiB" 177 | }, 178 | "instance86": { 179 | "status": true, 180 | "rtime": 242.3616, 181 | "mempeak": "13312 KiB" 182 | }, 183 | "instance85": { 184 | "status": true, 185 | "rtime": 749.3951, 186 | "mempeak": "150528 KiB" 187 | }, 188 | "instance84": { 189 | "status": true, 190 | "rtime": 237.2308, 191 | "mempeak": "111616 KiB" 192 | }, 193 | "instance89": { 194 | "status": true, 195 | "rtime": 90.243, 196 | "mempeak": "431104 KiB" 197 | }, 198 | "instance88": { 199 | "status": false, 200 | "rtime": 1000.0, 201 | "mempeak": "53248 KiB" 202 | }, 203 | "instance14": { 204 | "status": true, 205 | "rtime": 54.3457, 206 | "mempeak": "199680 KiB" 207 | }, 208 | "instance15": { 209 | "status": true, 210 | "rtime": 133.7901, 211 | "mempeak": "198656 KiB" 212 | }, 213 | "instance16": { 214 | "status": true, 215 | "rtime": 143.2799, 216 | "mempeak": "76800 KiB" 217 | }, 218 | "instance17": { 219 | "status": true, 220 | "rtime": 44.1856, 221 | "mempeak": "261120 KiB" 222 | }, 223 | "instance10": { 224 | "status": true, 225 | "rtime": 1.9281, 226 | "mempeak": "330752 KiB" 227 | }, 228 | "instance11": { 229 | "status": true, 230 | "rtime": 123.5724, 231 | "mempeak": "362496 KiB" 232 | }, 233 | "instance12": { 234 | "status": true, 235 | "rtime": 128.645, 236 | "mempeak": "22528 KiB" 237 | }, 238 | "instance13": { 239 | "status": true, 240 | "rtime": 117.9125, 241 | "mempeak": "367616 KiB" 242 | }, 243 | "instance18": { 244 | "status": true, 245 | "rtime": 112.592, 246 | "mempeak": "429056 KiB" 247 | }, 248 | "instance19": { 249 | "status": true, 250 | "rtime": 134.6126, 251 | "mempeak": "271360 KiB" 252 | }, 253 | "instance78": { 254 | "status": true, 255 | "rtime": 590.3233, 256 | "mempeak": "280576 KiB" 257 | }, 258 | "instance79": { 259 | "status": true, 260 | "rtime": 261.3116, 261 | "mempeak": "498688 KiB" 262 | }, 263 | "instance76": { 264 | "status": true, 265 | "rtime": 895.2961, 266 | "mempeak": "245760 KiB" 267 | }, 268 | "instance77": { 269 | "status": true, 270 | "rtime": 614.2801, 271 | "mempeak": "165888 KiB" 272 | }, 273 | "instance74": { 274 | "status": true, 275 | "rtime": 543.2072, 276 | "mempeak": "373760 KiB" 277 | }, 278 | "instance75": { 279 | "status": true, 280 | "rtime": 484.566, 281 | "mempeak": "218112 KiB" 282 | }, 283 | "instance72": { 284 | "status": false, 285 | "rtime": 1000.0, 286 | "mempeak": "475136 KiB" 287 | }, 288 | "instance73": { 289 | "status": true, 290 | "rtime": 74.5493, 291 | "mempeak": "231424 KiB" 292 | }, 293 | "instance70": { 294 | "status": true, 295 | "rtime": 590.4992, 296 | "mempeak": "173056 KiB" 297 | }, 298 | "instance71": { 299 | "status": true, 300 | "rtime": 552.754, 301 | "mempeak": "12288 KiB" 302 | }, 303 | "instance2": { 304 | "status": true, 305 | "rtime": 17.197, 306 | "mempeak": "230400 KiB" 307 | }, 308 | "instance3": { 309 | "status": true, 310 | "rtime": 80.8731, 311 | "mempeak": "241664 KiB" 312 | }, 313 | "instance1": { 314 | "status": true, 315 | "rtime": 4.6136, 316 | "mempeak": "358400 KiB" 317 | }, 318 | "instance6": { 319 | "status": true, 320 | "rtime": 1.3634, 321 | "mempeak": "175104 KiB" 322 | }, 323 | "instance7": { 324 | "status": true, 325 | "rtime": 102.321, 326 | "mempeak": "478208 KiB" 327 | }, 328 | "instance4": { 329 | "status": true, 330 | "rtime": 66.8706, 331 | "mempeak": "411648 KiB" 332 | }, 333 | "instance5": { 334 | "status": true, 335 | "rtime": 199.701, 336 | "mempeak": "442368 KiB" 337 | }, 338 | "instance8": { 339 | "status": true, 340 | "rtime": 45.8964, 341 | "mempeak": "430080 KiB" 342 | }, 343 | "instance9": { 344 | "status": true, 345 | "rtime": 58.8135, 346 | "mempeak": "260096 KiB" 347 | }, 348 | "instance69": { 349 | "status": true, 350 | "rtime": 221.7435, 351 | "mempeak": "142336 KiB" 352 | }, 353 | "instance68": { 354 | "status": true, 355 | "rtime": 293.9778, 356 | "mempeak": "141312 KiB" 357 | }, 358 | "instance65": { 359 | "status": true, 360 | "rtime": 492.8131, 361 | "mempeak": "65536 KiB" 362 | }, 363 | "instance64": { 364 | "status": true, 365 | "rtime": 84.1849, 366 | "mempeak": "124928 KiB" 367 | }, 368 | "instance67": { 369 | "status": true, 370 | "rtime": 487.3995, 371 | "mempeak": "419840 KiB" 372 | }, 373 | "instance66": { 374 | "status": true, 375 | "rtime": 397.1919, 376 | "mempeak": "62464 KiB" 377 | }, 378 | "instance61": { 379 | "status": true, 380 | "rtime": 445.1688, 381 | "mempeak": "134144 KiB" 382 | }, 383 | "instance60": { 384 | "status": true, 385 | "rtime": 221.8447, 386 | "mempeak": "507904 KiB" 387 | }, 388 | "instance63": { 389 | "status": true, 390 | "rtime": 544.9736, 391 | "mempeak": "290816 KiB" 392 | }, 393 | "instance62": { 394 | "status": true, 395 | "rtime": 294.5508, 396 | "mempeak": "248832 KiB" 397 | }, 398 | "instance100": { 399 | "status": true, 400 | "rtime": 410.851, 401 | "mempeak": "322560 KiB" 402 | }, 403 | "instance50": { 404 | "status": true, 405 | "rtime": 65.6477, 406 | "mempeak": "232448 KiB" 407 | }, 408 | "instance51": { 409 | "status": true, 410 | "rtime": 54.5806, 411 | "mempeak": "317440 KiB" 412 | }, 413 | "instance52": { 414 | "status": true, 415 | "rtime": 306.7591, 416 | "mempeak": "5120 KiB" 417 | }, 418 | "instance53": { 419 | "status": true, 420 | "rtime": 274.3517, 421 | "mempeak": "429056 KiB" 422 | }, 423 | "instance54": { 424 | "status": true, 425 | "rtime": 160.419, 426 | "mempeak": "181248 KiB" 427 | }, 428 | "instance55": { 429 | "status": true, 430 | "rtime": 387.1159, 431 | "mempeak": "233472 KiB" 432 | }, 433 | "instance56": { 434 | "status": true, 435 | "rtime": 303.1698, 436 | "mempeak": "487424 KiB" 437 | }, 438 | "instance57": { 439 | "status": true, 440 | "rtime": 582.7249, 441 | "mempeak": "411648 KiB" 442 | }, 443 | "instance58": { 444 | "status": true, 445 | "rtime": 306.2356, 446 | "mempeak": "489472 KiB" 447 | }, 448 | "instance59": { 449 | "status": true, 450 | "rtime": 406.7282, 451 | "mempeak": "112640 KiB" 452 | }, 453 | "instance47": { 454 | "status": true, 455 | "rtime": 472.182, 456 | "mempeak": "442368 KiB" 457 | }, 458 | "instance46": { 459 | "status": true, 460 | "rtime": 398.7485, 461 | "mempeak": "340992 KiB" 462 | }, 463 | "instance45": { 464 | "status": true, 465 | "rtime": 321.2542, 466 | "mempeak": "359424 KiB" 467 | }, 468 | "instance44": { 469 | "status": true, 470 | "rtime": 21.1671, 471 | "mempeak": "505856 KiB" 472 | }, 473 | "instance43": { 474 | "status": true, 475 | "rtime": 231.7043, 476 | "mempeak": "489472 KiB" 477 | }, 478 | "instance42": { 479 | "status": true, 480 | "rtime": 485.5096, 481 | "mempeak": "488448 KiB" 482 | }, 483 | "instance41": { 484 | "status": true, 485 | "rtime": 19.5951, 486 | "mempeak": "308224 KiB" 487 | }, 488 | "instance40": { 489 | "status": true, 490 | "rtime": 112.9365, 491 | "mempeak": "45056 KiB" 492 | }, 493 | "instance49": { 494 | "status": true, 495 | "rtime": 191.2439, 496 | "mempeak": "473088 KiB" 497 | }, 498 | "instance48": { 499 | "status": true, 500 | "rtime": 551.279, 501 | "mempeak": "262144 KiB" 502 | } 503 | }, 504 | "preamble": { 505 | "benchmark": "my-benchmark-set", 506 | "prog_args": "-a hello --arg2 world -v", 507 | "program": "very_nice_solver", 508 | "prog_alias": "vns" 509 | } 510 | } -------------------------------------------------------------------------------- /examples/solver2.json: -------------------------------------------------------------------------------- 1 | { 2 | "stats": { 3 | "instance32": { 4 | "status": true, 5 | "rtime": 238.3262, 6 | "mempeak": "288768 KiB" 7 | }, 8 | "instance33": { 9 | "status": true, 10 | "rtime": 195.9863, 11 | "mempeak": "492544 KiB" 12 | }, 13 | "instance30": { 14 | "status": true, 15 | "rtime": 191.50029999999998, 16 | "mempeak": "43008 KiB" 17 | }, 18 | "instance31": { 19 | "status": true, 20 | "rtime": 112.1347, 21 | "mempeak": "387072 KiB" 22 | }, 23 | "instance36": { 24 | "status": true, 25 | "rtime": 256.7886, 26 | "mempeak": "409600 KiB" 27 | }, 28 | "instance37": { 29 | "status": true, 30 | "rtime": 41.7866, 31 | "mempeak": "423936 KiB" 32 | }, 33 | "instance34": { 34 | "status": true, 35 | "rtime": 179.9343, 36 | "mempeak": "244736 KiB" 37 | }, 38 | "instance35": { 39 | "status": true, 40 | "rtime": 56.2602, 41 | "mempeak": "256000 KiB" 42 | }, 43 | "instance38": { 44 | "status": true, 45 | "rtime": 76.51599999999999, 46 | "mempeak": "453632 KiB" 47 | }, 48 | "instance39": { 49 | "status": true, 50 | "rtime": 127.8463, 51 | "mempeak": "273408 KiB" 52 | }, 53 | "instance98": { 54 | "status": true, 55 | "rtime": 610.4744000000001, 56 | "mempeak": "281600 KiB" 57 | }, 58 | "instance99": { 59 | "status": false, 60 | "rtime": 1000.0, 61 | "mempeak": "458752 KiB" 62 | }, 63 | "instance94": { 64 | "status": true, 65 | "rtime": 365.9865, 66 | "mempeak": "299008 KiB" 67 | }, 68 | "instance95": { 69 | "status": true, 70 | "rtime": 652.8399999999999, 71 | "mempeak": "475136 KiB" 72 | }, 73 | "instance96": { 74 | "status": false, 75 | "rtime": 1000.0, 76 | "mempeak": "307200 KiB" 77 | }, 78 | "instance97": { 79 | "status": true, 80 | "rtime": 307.7964, 81 | "mempeak": "223232 KiB" 82 | }, 83 | "instance90": { 84 | "status": true, 85 | "rtime": 789.5511, 86 | "mempeak": "39936 KiB" 87 | }, 88 | "instance91": { 89 | "status": true, 90 | "rtime": 428.9616, 91 | "mempeak": "491520 KiB" 92 | }, 93 | "instance92": { 94 | "status": true, 95 | "rtime": 687.4372000000001, 96 | "mempeak": "83968 KiB" 97 | }, 98 | "instance93": { 99 | "status": true, 100 | "rtime": 216.9602, 101 | "mempeak": "205824 KiB" 102 | }, 103 | "instance21": { 104 | "status": true, 105 | "rtime": 252.851, 106 | "mempeak": "350208 KiB" 107 | }, 108 | "instance20": { 109 | "status": true, 110 | "rtime": 2.4147, 111 | "mempeak": "80896 KiB" 112 | }, 113 | "instance23": { 114 | "status": true, 115 | "rtime": 109.5844, 116 | "mempeak": "254976 KiB" 117 | }, 118 | "instance22": { 119 | "status": true, 120 | "rtime": 68.6312, 121 | "mempeak": "250880 KiB" 122 | }, 123 | "instance25": { 124 | "status": true, 125 | "rtime": 183.3031, 126 | "mempeak": "497664 KiB" 127 | }, 128 | "instance24": { 129 | "status": true, 130 | "rtime": 277.5466, 131 | "mempeak": "34816 KiB" 132 | }, 133 | "instance27": { 134 | "status": true, 135 | "rtime": 7.4189, 136 | "mempeak": "7168 KiB" 137 | }, 138 | "instance26": { 139 | "status": true, 140 | "rtime": 51.4459, 141 | "mempeak": "277504 KiB" 142 | }, 143 | "instance29": { 144 | "status": true, 145 | "rtime": 266.3557, 146 | "mempeak": "510976 KiB" 147 | }, 148 | "instance28": { 149 | "status": true, 150 | "rtime": 177.6557, 151 | "mempeak": "279552 KiB" 152 | }, 153 | "instance83": { 154 | "status": false, 155 | "rtime": 1000.0, 156 | "mempeak": "191488 KiB" 157 | }, 158 | "instance82": { 159 | "status": true, 160 | "rtime": 509.43190000000004, 161 | "mempeak": "304128 KiB" 162 | }, 163 | "instance81": { 164 | "status": true, 165 | "rtime": 943.9072, 166 | "mempeak": "503808 KiB" 167 | }, 168 | "instance80": { 169 | "status": false, 170 | "rtime": 1000.0, 171 | "mempeak": "302080 KiB" 172 | }, 173 | "instance87": { 174 | "status": false, 175 | "rtime": 1000.0, 176 | "mempeak": "197632 KiB" 177 | }, 178 | "instance86": { 179 | "status": true, 180 | "rtime": 354.3616, 181 | "mempeak": "177152 KiB" 182 | }, 183 | "instance85": { 184 | "status": true, 185 | "rtime": 724.3951, 186 | "mempeak": "208896 KiB" 187 | }, 188 | "instance84": { 189 | "status": true, 190 | "rtime": 368.2308, 191 | "mempeak": "476160 KiB" 192 | }, 193 | "instance89": { 194 | "status": true, 195 | "rtime": 137.243, 196 | "mempeak": "516096 KiB" 197 | }, 198 | "instance88": { 199 | "status": true, 200 | "rtime": 875.0, 201 | "mempeak": "354304 KiB" 202 | }, 203 | "instance14": { 204 | "status": true, 205 | "rtime": 78.3457, 206 | "mempeak": "101376 KiB" 207 | }, 208 | "instance15": { 209 | "status": true, 210 | "rtime": 213.7901, 211 | "mempeak": "378880 KiB" 212 | }, 213 | "instance16": { 214 | "status": true, 215 | "rtime": 127.2799, 216 | "mempeak": "176128 KiB" 217 | }, 218 | "instance17": { 219 | "status": true, 220 | "rtime": 71.1856, 221 | "mempeak": "74752 KiB" 222 | }, 223 | "instance10": { 224 | "status": true, 225 | "rtime": 1.9281, 226 | "mempeak": "471040 KiB" 227 | }, 228 | "instance11": { 229 | "status": true, 230 | "rtime": 165.57240000000002, 231 | "mempeak": "204800 KiB" 232 | }, 233 | "instance12": { 234 | "status": true, 235 | "rtime": 212.645, 236 | "mempeak": "27648 KiB" 237 | }, 238 | "instance13": { 239 | "status": true, 240 | "rtime": 107.9125, 241 | "mempeak": "159744 KiB" 242 | }, 243 | "instance18": { 244 | "status": true, 245 | "rtime": 157.59199999999998, 246 | "mempeak": "290816 KiB" 247 | }, 248 | "instance19": { 249 | "status": true, 250 | "rtime": 120.61259999999999, 251 | "mempeak": "488448 KiB" 252 | }, 253 | "instance78": { 254 | "status": true, 255 | "rtime": 852.3233, 256 | "mempeak": "126976 KiB" 257 | }, 258 | "instance79": { 259 | "status": true, 260 | "rtime": 260.3116, 261 | "mempeak": "328704 KiB" 262 | }, 263 | "instance76": { 264 | "status": true, 265 | "rtime": 804.2961, 266 | "mempeak": "16384 KiB" 267 | }, 268 | "instance77": { 269 | "status": true, 270 | "rtime": 994.2801, 271 | "mempeak": "282624 KiB" 272 | }, 273 | "instance74": { 274 | "status": true, 275 | "rtime": 796.2072, 276 | "mempeak": "326656 KiB" 277 | }, 278 | "instance75": { 279 | "status": true, 280 | "rtime": 786.566, 281 | "mempeak": "407552 KiB" 282 | }, 283 | "instance72": { 284 | "status": false, 285 | "rtime": 1000.0, 286 | "mempeak": "254976 KiB" 287 | }, 288 | "instance73": { 289 | "status": true, 290 | "rtime": 70.5493, 291 | "mempeak": "207872 KiB" 292 | }, 293 | "instance70": { 294 | "status": true, 295 | "rtime": 525.4992, 296 | "mempeak": "471040 KiB" 297 | }, 298 | "instance71": { 299 | "status": true, 300 | "rtime": 783.754, 301 | "mempeak": "41984 KiB" 302 | }, 303 | "instance2": { 304 | "status": true, 305 | "rtime": 27.197, 306 | "mempeak": "323584 KiB" 307 | }, 308 | "instance3": { 309 | "status": true, 310 | "rtime": 125.8731, 311 | "mempeak": "344064 KiB" 312 | }, 313 | "instance1": { 314 | "status": true, 315 | "rtime": 4.6136, 316 | "mempeak": "77824 KiB" 317 | }, 318 | "instance6": { 319 | "status": true, 320 | "rtime": 1.3634, 321 | "mempeak": "202752 KiB" 322 | }, 323 | "instance7": { 324 | "status": true, 325 | "rtime": 101.321, 326 | "mempeak": "367616 KiB" 327 | }, 328 | "instance4": { 329 | "status": true, 330 | "rtime": 58.870599999999996, 331 | "mempeak": "343040 KiB" 332 | }, 333 | "instance5": { 334 | "status": true, 335 | "rtime": 287.701, 336 | "mempeak": "119808 KiB" 337 | }, 338 | "instance8": { 339 | "status": true, 340 | "rtime": 71.8964, 341 | "mempeak": "11264 KiB" 342 | }, 343 | "instance9": { 344 | "status": true, 345 | "rtime": 77.8135, 346 | "mempeak": "484352 KiB" 347 | }, 348 | "instance69": { 349 | "status": true, 350 | "rtime": 354.74350000000004, 351 | "mempeak": "211968 KiB" 352 | }, 353 | "instance68": { 354 | "status": true, 355 | "rtime": 453.9778, 356 | "mempeak": "180224 KiB" 357 | }, 358 | "instance65": { 359 | "status": true, 360 | "rtime": 662.8131000000001, 361 | "mempeak": "178176 KiB" 362 | }, 363 | "instance64": { 364 | "status": true, 365 | "rtime": 80.1849, 366 | "mempeak": "378880 KiB" 367 | }, 368 | "instance67": { 369 | "status": true, 370 | "rtime": 448.3995, 371 | "mempeak": "445440 KiB" 372 | }, 373 | "instance66": { 374 | "status": true, 375 | "rtime": 605.1919, 376 | "mempeak": "276480 KiB" 377 | }, 378 | "instance61": { 379 | "status": true, 380 | "rtime": 433.1688, 381 | "mempeak": "68608 KiB" 382 | }, 383 | "instance60": { 384 | "status": true, 385 | "rtime": 314.8447, 386 | "mempeak": "120832 KiB" 387 | }, 388 | "instance63": { 389 | "status": true, 390 | "rtime": 864.9736, 391 | "mempeak": "440320 KiB" 392 | }, 393 | "instance62": { 394 | "status": true, 395 | "rtime": 446.5508, 396 | "mempeak": "72704 KiB" 397 | }, 398 | "instance100": { 399 | "status": true, 400 | "rtime": 371.851, 401 | "mempeak": "145408 KiB" 402 | }, 403 | "instance50": { 404 | "status": true, 405 | "rtime": 90.6477, 406 | "mempeak": "264192 KiB" 407 | }, 408 | "instance51": { 409 | "status": true, 410 | "rtime": 89.5806, 411 | "mempeak": "392192 KiB" 412 | }, 413 | "instance52": { 414 | "status": true, 415 | "rtime": 296.7591, 416 | "mempeak": "106496 KiB" 417 | }, 418 | "instance53": { 419 | "status": true, 420 | "rtime": 426.3517, 421 | "mempeak": "129024 KiB" 422 | }, 423 | "instance54": { 424 | "status": true, 425 | "rtime": 251.419, 426 | "mempeak": "1024 KiB" 427 | }, 428 | "instance55": { 429 | "status": true, 430 | "rtime": 359.1159, 431 | "mempeak": "365568 KiB" 432 | }, 433 | "instance56": { 434 | "status": true, 435 | "rtime": 438.1698, 436 | "mempeak": "338944 KiB" 437 | }, 438 | "instance57": { 439 | "status": true, 440 | "rtime": 888.7249, 441 | "mempeak": "281600 KiB" 442 | }, 443 | "instance58": { 444 | "status": true, 445 | "rtime": 299.2356, 446 | "mempeak": "193536 KiB" 447 | }, 448 | "instance59": { 449 | "status": true, 450 | "rtime": 633.7282, 451 | "mempeak": "197632 KiB" 452 | }, 453 | "instance47": { 454 | "status": true, 455 | "rtime": 764.182, 456 | "mempeak": "166912 KiB" 457 | }, 458 | "instance46": { 459 | "status": true, 460 | "rtime": 373.7485, 461 | "mempeak": "208896 KiB" 462 | }, 463 | "instance45": { 464 | "status": true, 465 | "rtime": 515.2542000000001, 466 | "mempeak": "330752 KiB" 467 | }, 468 | "instance44": { 469 | "status": true, 470 | "rtime": 31.1671, 471 | "mempeak": "72704 KiB" 472 | }, 473 | "instance43": { 474 | "status": true, 475 | "rtime": 207.7043, 476 | "mempeak": "190464 KiB" 477 | }, 478 | "instance42": { 479 | "status": true, 480 | "rtime": 664.5096, 481 | "mempeak": "343040 KiB" 482 | }, 483 | "instance41": { 484 | "status": true, 485 | "rtime": 31.5951, 486 | "mempeak": "256000 KiB" 487 | }, 488 | "instance40": { 489 | "status": true, 490 | "rtime": 101.9365, 491 | "mempeak": "517120 KiB" 492 | }, 493 | "instance49": { 494 | "status": true, 495 | "rtime": 175.2439, 496 | "mempeak": "44032 KiB" 497 | }, 498 | "instance48": { 499 | "status": true, 500 | "rtime": 825.279, 501 | "mempeak": "415744 KiB" 502 | } 503 | }, 504 | "preamble": { 505 | "benchmark": "my-benchmark-set", 506 | "prog_args": null, 507 | "program": "another-good-tool", 508 | "prog_alias": "agt" 509 | } 510 | } -------------------------------------------------------------------------------- /load.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | #-*- coding:utf-8 -*- 3 | ## 4 | ## load.py 5 | ## 6 | ## Created on: Jun 05, 2015 7 | ## Author: Alexey S. Ignatiev 8 | ## E-mail: aignatiev@ciencias.ulisboa.pt 9 | ## 10 | 11 | # 12 | #============================================================================== 13 | import csv 14 | import json 15 | import statutil 16 | import six 17 | import sys 18 | 19 | 20 | # 21 | #============================================================================== 22 | def load_data(files, options): 23 | """ 24 | Loads data from the input files. 25 | """ 26 | 27 | try: # if JSON data 28 | return load_json(statutil.StatArray(files), options) 29 | except statutil.JSONException as e: 30 | sys.stderr.write('\033[33;1mWarning:\033[m ' + str(e) + '\033[m\n') 31 | sys.stderr.write('Probably not a JSON format. Trying to read as CSV.\n') 32 | 33 | # reading CSV 34 | # expecting exactly one input file 35 | with open(files[0], 'r') as fp: 36 | # try: 37 | rows = csv.reader(fp, delimiter=' ', quotechar='|') 38 | rows = [row for row in rows] 39 | 40 | stats = [] 41 | names = [n.strip() for n in rows[0][1:] if n.strip()] 42 | for row in rows[1:]: 43 | stats.append([val.strip() for val in row[1:] if val.strip()]) 44 | 45 | return load_csv(names, stats, options) 46 | # except: 47 | # sys.stderr.write('\033[31;1mError:\033[m Unable to read input file(s).\n') 48 | 49 | 50 | # 51 | #============================================================================== 52 | def load_json(stat_arr, options): 53 | """ 54 | Loads runtime data from STAT objects. 55 | """ 56 | 57 | # preparing data 58 | if options['join_key']: 59 | stat_arr.cluster(use_key=options['join_key']) 60 | 61 | data = [] 62 | 63 | # choosing the minimal value 64 | min_val = 0.000000001 65 | if options['plot_type'] == 'scatter': 66 | if options['x_min']: 67 | min_val = max(options['x_min'], options['y_min']) 68 | else: 69 | min_val = options['y_min'] # options['y_min'] is always defined 70 | 71 | # processing (normal) separate data 72 | for stat_obj in stat_arr: 73 | vals = [] 74 | num_solved = 0 75 | 76 | last_val = -1 77 | for inst in stat_obj.insts_own: # insts_own are sorted 78 | if options['key'] in stat_obj.data[inst]: 79 | val = stat_obj.data[inst][options['key']] 80 | else: 81 | val = float(options['timeout']) 82 | if stat_obj.data[inst]['status'] == True: 83 | if val > last_val: 84 | last_val = val 85 | 86 | if val >= float(options['timeout']): 87 | val = float(options['timeout']) 88 | elif val <= min_val: 89 | val = min_val 90 | 91 | num_solved += 1 92 | else: 93 | val = float(options['timeout']) 94 | if options['plot_type'] == 'cactus': 95 | val *= 10 96 | 97 | vals.append(val) 98 | 99 | if type(options['legend']) is list: 100 | label = ' '.join([stat_obj.preamble[k] for k in options['legend']]) 101 | else: 102 | label = stat_obj.preamble[options['legend']] 103 | 104 | label = label.strip() 105 | data.append((label, vals, num_solved, last_val)) 106 | 107 | # processing VBSes 108 | if options['vbs']: 109 | for vbs_name, tools in options['vbs'].items(): 110 | max_value = float(options['timeout']) if options['plot_type'] == 'scatter' else 10 * float(options['timeout']) 111 | vals = { i: max_value for i in stat_arr.inst_full} 112 | num_solved = 0 113 | 114 | if tools != 'all': 115 | for stat_obj in stat_arr: 116 | if type(options['legend']) is list: 117 | p = ' '.join([stat_obj.preamble[k] for k in options['legend']]) 118 | else: 119 | p = stat_obj.preamble[options['legend']] 120 | 121 | p = p.strip() 122 | 123 | if p in tools: 124 | for inst, d in six.iteritems(stat_obj.data): 125 | if d['status'] == True: 126 | if d[options['key']] >= float(options['timeout']): 127 | d[options['key']] = max_value 128 | elif vals[inst] >= max_value: 129 | num_solved += 1 130 | 131 | vals[inst] = max([min_val, min([d[options['key']], vals[inst]])]) 132 | else: # VBS among all the tools 133 | for stat_obj in stat_arr: 134 | for inst, d in six.iteritems(stat_obj.data): 135 | if d['status'] == True: 136 | if d[options['key']] >= float(options['timeout']): 137 | d[options['key']] = max_value 138 | elif vals[inst] >= max_value: 139 | num_solved += 1 140 | 141 | vals[inst] = max([min_val, min([d[options['key']], vals[inst]])]) 142 | 143 | last_val = -1 144 | for v in six.itervalues(vals): 145 | if v > last_val and v < max_value: 146 | last_val = v 147 | 148 | data.append((vbs_name, [vals[i] for i in stat_arr.inst_full], num_solved, last_val)) 149 | 150 | if options['only']: 151 | data = [d for d in data if d[0] in options['only']] 152 | 153 | if options['repls']: 154 | data = [(options['repls'][n], v, s, l) if n in options['repls'] else (n, v, s, l) for n, v, s, l in data] 155 | 156 | return sorted(data, key=lambda x: x[2] + len(x[1]) / sum(x[1]), reverse=not options['reverse']) 157 | 158 | 159 | # 160 | #============================================================================== 161 | def load_csv(names, stats, options): 162 | """ 163 | Loads runtime CSV data. 164 | """ 165 | 166 | # choosing the minimal value 167 | min_val = 0.000000001 168 | if options['plot_type'] == 'scatter': 169 | if options['x_min']: 170 | min_val = max(options['x_min'], options['y_min']) 171 | else: 172 | min_val = options['y_min'] # options['y_min'] is always defined 173 | 174 | names_orig = names[:] 175 | 176 | if options['repls']: 177 | names = [options['repls'][n] if n in options['repls'] else n for n in names] 178 | 179 | # processing (normal) separate data 180 | lens = [0 for n in names] 181 | vals_all = [[] for n in names] 182 | last_vals = [-1 for n in names] 183 | 184 | for vlist in stats: 185 | vlist = [float(val) for val in vlist] 186 | 187 | for i, val in enumerate(vlist): 188 | if val < float(options['timeout']): 189 | if val > last_vals[i]: 190 | last_vals[i] = val 191 | 192 | if val < min_val: 193 | val = min_val 194 | 195 | lens[i] += 1 196 | else: 197 | val = float(options['timeout']) 198 | if options['plot_type'] == 'cactus': 199 | val *= 10 200 | 201 | vals_all[i].append(val) 202 | 203 | # processing VBSes 204 | if options['vbs']: 205 | for vbs_name, tools in options['vbs'].items(): 206 | vals = [] 207 | len_ = 0 208 | last_val = -1 209 | 210 | if tools != 'all': 211 | tools = [n if n in tools else '' for n in names_orig] 212 | 213 | for vlist in stats: 214 | vlist = [float(val) for i, val in enumerate(vlist) if tools[i]] 215 | val = min(vlist) 216 | 217 | if val < float(options['timeout']): 218 | if val > last_val: 219 | last_val = val 220 | 221 | if val < min_val: 222 | val = min_val 223 | 224 | len_ += 1 225 | else: 226 | val = options['timeout'] 227 | if options['plot_type'] == 'cactus': 228 | val *= 10 229 | 230 | vals.append(val) 231 | else: # VBS among all the tools 232 | for vlist in stats: 233 | val = min([float(val) for val in vlist]) 234 | 235 | if val < float(options['timeout']): 236 | if val > last_val: 237 | last_val = val 238 | 239 | if val < min_val: 240 | val = min_val 241 | 242 | len_ += 1 243 | else: 244 | val = options['timeout'] 245 | if options['plot_type'] == 'cactus': 246 | val *= 10 247 | 248 | vals.append(val) 249 | 250 | names.append(vbs_name) 251 | names_orig.append(vbs_name) 252 | vals_all.append(vals) 253 | lens.append(len_) 254 | last_vals.append(last_val) 255 | 256 | data = [[n, t, s, l] for n, t, s, l in zip(names, vals_all, lens, last_vals)] 257 | 258 | if options['only']: 259 | data = [d for i, d in enumerate(data) if names_orig[i] in options['only']] 260 | 261 | return sorted(data, key=lambda x: x[2] + len(x[1]) / sum(x[1]), reverse=not options['reverse']) 262 | -------------------------------------------------------------------------------- /mkplot.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | #-*- coding:utf-8 -*- 3 | ## 4 | ## mkplot.py 5 | ## 6 | ## Created on: Jan 31, 2014 7 | ## Author: Alexey S. Ignatiev 8 | ## E-mail: aignatiev@ciencias.ulisboa.pt 9 | ## 10 | 11 | # 12 | #============================================================================== 13 | from __future__ import print_function 14 | import matplotlib 15 | matplotlib.use('pdf') # for not loading GUI modules 16 | 17 | from cactus import Cactus 18 | import getopt 19 | import json 20 | from load import load_data 21 | import os 22 | from scatter import Scatter 23 | import sys 24 | 25 | 26 | # 27 | #============================================================================== 28 | def parse_options(): 29 | """ 30 | Parses command-line options: 31 | """ 32 | 33 | try: 34 | opts, args = getopt.getopt(sys.argv[1:], 35 | 'a:b:c:df:hj:k:lnp:r:t:', 36 | ['alpha=', 37 | 'backend=', 38 | 'by-name', 39 | 'config=', 40 | 'dry-run', 41 | 'font=', 42 | 'font-sz=', 43 | 'no-grid', 44 | 'help', 45 | 'join-key=', 46 | 'key=', 47 | 'latex', 48 | 'lalpha=', 49 | 'legend=', 50 | 'lloc=', 51 | 'lncol=', 52 | 'only=', 53 | 'plot-type=', 54 | 'replace=', 55 | 'reverse', 56 | 'save-to=', 57 | 'shape=', 58 | 'timeout=', 59 | 'tlabel=', 60 | 'tol-loc=', 61 | 'transparent', 62 | 'vbs=', 63 | 'xkcd', 64 | 'xlabel=', 65 | 'xlog', 66 | 'xmin=', 67 | 'xmax=', 68 | 'ylabel=', 69 | 'ylog', 70 | 'ymin=', 71 | 'ymax=']) 72 | except getopt.GetoptError as err: 73 | sys.stderr.write(str(err).capitalize() + '\n') 74 | usage() 75 | sys.exit(1) 76 | 77 | # loading default options 78 | for opt, arg in opts: 79 | if opt in ('-c', '--config'): 80 | def_path = str(arg) 81 | break 82 | else: 83 | def_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'defaults.json') 84 | 85 | with open(def_path, 'r') as fp: 86 | options = json.load(fp)['settings'] 87 | options['def_path'] = def_path 88 | 89 | # parsing command-line options 90 | for opt, arg in opts: 91 | if opt in ('-a', '--alpha'): 92 | options['alpha'] = float(arg) 93 | elif opt in ('-b', '--backend'): 94 | options['backend'] = str(arg) 95 | elif opt in ('-c', '--config'): 96 | pass # already processed 97 | elif opt in ('-d', '--dry-run'): 98 | options['dry_run'] = True 99 | elif opt in ('-f', '--font'): 100 | options['font'] = str(arg) 101 | elif opt == '--font-sz': 102 | options['font_sz'] = float(arg) 103 | elif opt in ('-h', '--help'): 104 | usage() 105 | sys.exit(0) 106 | elif opt == '--no-grid': 107 | options['no_grid'] = True 108 | elif opt in ('-j', '--join-key'): 109 | options['join_key'] = [k.strip() for k in str(arg).split(',')] 110 | elif opt in ('-k', '--key'): 111 | options['key'] = str(arg) 112 | elif opt in ('-l', '--latex'): 113 | options['usetex'] = True 114 | elif opt == '--lalpha': 115 | options['lgd_alpha'] = float(arg) 116 | elif opt == '--legend': 117 | options['legend'] = [k.strip() for k in str(arg).split(',')] 118 | elif opt == '--lloc': 119 | options['lgd_loc'] = str(arg) 120 | elif opt == '--lncol': 121 | options['lgd_ncol'] = int(arg) 122 | elif opt in ('-n', '--by-name'): 123 | options['by_name'] = True 124 | elif opt == '--only': 125 | options['only'] = [t.strip() for t in str(arg).split(',')] 126 | elif opt in ('-p', '--plot-type'): 127 | options['plot_type'] = str(arg) 128 | elif opt in ('-r', '--replace'): 129 | options['repls'] = json.loads(str(arg)) 130 | elif opt == '--reverse': 131 | options['reverse'] = True 132 | elif opt == '--save-to': 133 | options['save_to'] = str(arg) 134 | elif opt == '--shape': 135 | options['shape'] = str(arg) 136 | elif opt in ('-t', '--timeout'): 137 | options['timeout'] = float(arg) 138 | elif opt == '--tlabel': 139 | options['t_label'] = str(arg) 140 | elif opt == '--tol-loc': 141 | options['tlb_loc'] = str(arg) 142 | elif opt == '--transparent': 143 | options['transparent'] = True 144 | elif opt == '--vbs': 145 | options['vbs'] = json.loads(str(arg)) 146 | elif opt == '--xkcd': 147 | options['xkcd'] = True 148 | elif opt == '--xlabel': 149 | options['x_label'] = str(arg) 150 | elif opt == '--xlog': 151 | options['x_log'] = True 152 | elif opt == '--xmin': 153 | options['x_min'] = float(arg) 154 | elif opt == '--xmax': 155 | options['x_max'] = float(arg) 156 | elif opt == '--ylabel': 157 | options['y_label'] = str(arg) 158 | elif opt == '--ylog': 159 | options['y_log'] = True 160 | elif opt == '--ymin': 161 | options['y_min'] = float(arg) 162 | elif opt == '--ymax': 163 | options['y_max'] = float(arg) 164 | else: 165 | assert False, 'Unhandled option: {0} {1}'.format(opt, arg) 166 | 167 | return options, args 168 | 169 | 170 | # 171 | #============================================================================== 172 | def usage(): 173 | """ 174 | Prints usage message. 175 | """ 176 | 177 | print('Usage:', os.path.basename(sys.argv[0]), ' [options] stat-files') 178 | print('Options:') 179 | print(' -a, --alpha= Alpha value (only for scatter plots)') 180 | print(' Available values: [0 .. 1] (default = 0.3)') 181 | print(' -b, --backend= Backend to use') 182 | print(' Available values: pdf, pgf, png, ps, svg (default = pdf)') 183 | print(' -c, --config= Path to the default configuration file (default = $MKPLOT/defaults.json)') 184 | print(' -d, --dry-run Do not create a plot but instead show the tools sorted in the terminal') 185 | print(' -f, --font= Font to use') 186 | print(' Available values: cmr, helvetica, palatino, times (default = times)') 187 | print(' --font-sz= Font size to use') 188 | print(' Available values: [0 .. INT_MAX] (default = 12)') 189 | print(' -h, --help Show this message') 190 | print(' --no-grid Do not show the grid') 191 | print(' -j, --join-key= Comma-separated list of keys to join all benchmarks per each tool') 192 | print(' -k, --key= Key to measure') 193 | print(' Available values: \'rtime\', for others look at the STAT file (default = \'rtime\')') 194 | print(' -l, --latex Use latex') 195 | print(' --lalpha= Legend transparency level') 196 | print(' Available values: [0 .. 1] (default = 1.0)') 197 | print(' --legend= Comma-separated list of keys to use in the legend of a plot') 198 | print(' Format: "program,prog_args" (default = program)') 199 | print(' --lloc= Legend location') 200 | print(' Available values: upper/center/lower left/right, center, best, off (default = upper left)') 201 | print(' --lncol= Number of columns in the legend') 202 | print(' Available values: [1 .. INT_MAX] (default = 1)') 203 | print(' -n, --by-name Assign line style to tools by their name') 204 | print(' --only= Comma-separated list of names') 205 | print(' Format: "tool1,tool2" (default = none)') 206 | print(' -p, --plot-type= Plot type to produce') 207 | print(' Available values: cactus or scatter (default = cactus)') 208 | print(' -r, --replace= List of name replacements') 209 | print(' Format: {"name1": "$nice_name1$", "name2": "$nice_name2$"} (default = none)') 210 | print(' --reverse Use reversed sorting') 211 | print(' --save-to= Where result figure should be saved') 212 | print(' Default value: plot') 213 | print(' --shape= Shape of the plot') 214 | print(' Available values: long, squared, standard (default = standard)') 215 | print(' -t, --timeout= Timeout value') 216 | print(' Available values: [0 .. INT_MAX] (default = 3600)') 217 | print(' --tlabel= Timeout label (for scatter plots only)') 218 | print(' --tol-loc= Where to put the timeout label') 219 | print(' Available values: before, after (default = after)') 220 | print(' --transparent Save the file in the transparent mode') 221 | print(' --vbs= List of VBSes') 222 | print(' Format: {"vbs1": ["tool1", "tool2"], "vbs2": "all"} (default = none)') 223 | print(' --xkcd Use xkcd-style sketch plotting') 224 | print(' --xlabel= X label') 225 | print(' --xlog Use logarithmic scale for X axis') 226 | print(' --xmax= X axis ends at this value') 227 | print(' Available values: [0 .. INT_MAX] (default = none)') 228 | print(' --xmin= X axis starts from this value') 229 | print(' Available values: [0 .. INT_MAX] (default = 0)') 230 | print(' --ylabel= Y label') 231 | print(' --ylog Use logarithmic scale for Y axis') 232 | print(' --ymax= Y axis ends at this value') 233 | print(' Available values: [0 .. INT_MAX] (default = none)') 234 | print(' --ymin= Y axis starts from this value') 235 | print(' Available values: [0 .. INT_MAX] (default = 0)') 236 | 237 | 238 | # 239 | #============================================================================== 240 | if __name__ == '__main__': 241 | options, fns = parse_options() 242 | 243 | if not fns: 244 | pass # error handling 245 | 246 | data = load_data(fns, options) 247 | 248 | if options['dry_run']: 249 | for d in data: 250 | d1 = list(map(lambda x: min(x, options['timeout']), d[1])) 251 | 252 | print('{0}:'.format(d[0])) 253 | print(' # solved: {0}'.format(d[2])) 254 | print(' min. val: {0:.1f}'.format(float(min(d1)))) 255 | print(' max. val: {0:.1f}'.format(float(max(d1)))) 256 | print(' avg. val: {0:.1f}'.format(float(sum(d1)) / len(d1))) 257 | else: 258 | if options['plot_type'] == 'cactus': 259 | plotter = Cactus(options) 260 | else: 261 | plotter = Scatter(options) 262 | 263 | plotter.create(data) 264 | -------------------------------------------------------------------------------- /plot.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | #-*- coding:utf-8 -*- 3 | ## 4 | ## plot.py 5 | ## 6 | ## Created on: Jun 05, 2015 7 | ## Author: Alexey S. Ignatiev 8 | ## E-mail: aignatiev@ciencias.ulisboa.pt 9 | ## 10 | 11 | # 12 | #============================================================================== 13 | import matplotlib.pyplot as plt 14 | import numpy as np 15 | import os 16 | 17 | 18 | # 19 | #============================================================================== 20 | class Plot(): 21 | """ 22 | Basic plotting class. 23 | """ 24 | 25 | def __init__(self, options): 26 | """ 27 | Constructor. 28 | """ 29 | 30 | self.alpha = options['alpha'] 31 | self.backend = options['backend'] 32 | self.save_to = options['save_to'] 33 | self.def_path = options['def_path'] 34 | self.transparent = options['transparent'] 35 | 36 | self.timeout = options['timeout'] 37 | self.t_label = options['t_label'] 38 | self.tlb_loc = options['tlb_loc'] 39 | 40 | self.x_label = options['x_label'] 41 | self.x_log = options['x_log'] 42 | self.x_max = options['x_max'] 43 | self.x_min = options['x_min'] 44 | self.y_label = options['y_label'] 45 | self.y_log = options['y_log'] 46 | self.y_max = options['y_max'] 47 | self.y_min = options['y_min'] 48 | 49 | self.lgd_loc = options['lgd_loc'] 50 | self.lgd_ncol = options['lgd_ncol'] 51 | self.lgd_alpha = options['lgd_alpha'] 52 | self.lgd_fancy = options['lgd_fancy'] 53 | self.lgd_shadow = options['lgd_shadow'] 54 | 55 | self.no_grid = options['no_grid'] 56 | self.grid_color = options['grid_color'] 57 | self.grid_style = options['grid_style'] 58 | self.grid_width = options['grid_width'] 59 | self.byname = options['by_name'] 60 | 61 | # where to save 62 | self.save_to = '{0}.{1}'.format(os.path.splitext(self.save_to)[0], self.backend) 63 | 64 | # font properties 65 | self.f_props = {'serif': ['Times'], 'sans-serif': ['Helvetica'], 66 | 'weight': 'normal', 'size': options['font_sz']} 67 | 68 | if options['font'].lower() in ('sans', 'sans-serif', 'helvetica'): # Helvetica 69 | self.f_props['family'] = 'sans-serif' 70 | elif options['font'].lower() in ('serif', 'times'): # Times 71 | self.f_props['family'] = 'serif' 72 | elif options['font'].lower() == 'cmr': # Computer Modern Roman 73 | self.f_props['family'] = 'serif' 74 | self.f_props['serif'] = 'Computer Modern Roman' 75 | elif options['font'].lower() == 'palatino': # Palatino 76 | self.f_props['family'] = 'serif' 77 | self.f_props['serif'] = 'Palatino' 78 | 79 | plt.rc('text', usetex=options['usetex']) 80 | plt.rc('font', **self.f_props) 81 | 82 | # figure properties 83 | nof_subplots = 1 84 | fig_width_pt = 252.0 # Get this from LaTeX using \showthe\columnwidth 85 | inches_per_pt = 1.0 / 72.27 # Convert pt to inch 86 | golden_mean = (np.sqrt(5) + 1.0) / 2.0 # Aesthetic ratio 87 | fig_width = fig_width_pt * inches_per_pt + 0.2 # width in inches 88 | fig_height = fig_width / golden_mean * nof_subplots + 0.395 * (nof_subplots - 1) # height in inches 89 | if options['shape'] == 'squared': 90 | fig_width = fig_height 91 | elif len(options['shape']) >= 4 and options['shape'][:4] == 'long': 92 | coeff = options['shape'][4:] 93 | fig_width *= 1.2 if not coeff else float(coeff) # default coefficient is 1.2 94 | 95 | fig_size = [fig_width * 2.5, fig_height * 2.5] 96 | 97 | params = {'backend': 'pdf', 'text.usetex': options['usetex'], 'figure.figsize': fig_size} 98 | 99 | plt.rcParams.update(params) 100 | 101 | # choosing backend 102 | if self.backend in ('pdf', 'ps', 'svg'): # default is pdf 103 | plt.switch_backend(self.backend) 104 | elif self.backend == 'pgf': # PGF/TikZ 105 | pgf_params = {'pgf.texsystem': 'pdflatex', 106 | 'pgf.preamble': [r'\usepackage[utf8x]{inputenc}', r'\usepackage[T1]{fontenc}']} 107 | params.update(pgf_params) 108 | plt.rcParams.update(params) 109 | plt.switch_backend(self.backend) 110 | elif self.backend == 'png': 111 | plt.switch_backend('agg') 112 | 113 | # funny mode 114 | if options['xkcd']: 115 | plt.xkcd() 116 | -------------------------------------------------------------------------------- /scatter.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | #-*- coding:utf-8 -*- 3 | ## 4 | ## scatter.py 5 | ## 6 | ## Created on: Jun 05, 2015 7 | ## Author: Alexey S. Ignatiev 8 | ## E-mail: aignatiev@ciencias.ulisboa.pt 9 | ## 10 | 11 | # 12 | #============================================================================== 13 | import json 14 | import math 15 | import matplotlib.pyplot as plt 16 | from matplotlib import __version__ as mpl_version 17 | import numpy as np 18 | from plot import Plot 19 | import six 20 | from six.moves import range 21 | 22 | 23 | # 24 | #============================================================================== 25 | class ScatterException(Exception): 26 | pass 27 | 28 | 29 | # 30 | #============================================================================== 31 | class Scatter(Plot, object): 32 | """ 33 | Scatter plot class. 34 | """ 35 | 36 | def __init__(self, options): 37 | """ 38 | Scatter constructor. 39 | """ 40 | 41 | super(Scatter, self).__init__(options) 42 | 43 | # setting up axes limits 44 | if not self.x_min: 45 | self.x_min = self.y_min # self.y_min is supposed to have a default value 46 | else: 47 | self.y_min = self.x_min 48 | 49 | if not self.x_max: 50 | self.x_max = 0 51 | if not self.y_max: 52 | self.y_max = 0 53 | if self.x_max and self.y_max and self.x_max != self.y_max: 54 | assert 0, 'right-most positions must be the same for X and Y axes' 55 | elif self.x_max == 0 and self.y_max == 0: 56 | self.x_max = 10 57 | while self.x_max < self.timeout: 58 | self.x_max *= 10 59 | self.y_max = self.x_max 60 | else: 61 | self.x_max = self.y_max = max(self.x_max, self.y_max) 62 | 63 | # setting timeout-line label 64 | if not self.t_label: 65 | self.t_label = '{0} sec. timeout'.format(int(self.timeout)) 66 | 67 | with open(self.def_path, 'r') as fp: 68 | self.marker_style = json.load(fp)['scatter_style'] 69 | 70 | def create(self, data): 71 | """ 72 | Does the plotting. 73 | """ 74 | 75 | if len(data[0][1]) != len(data[1][1]): 76 | raise ScatterException('Number of instances for each competitor must be the same') 77 | 78 | step = math.ceil((self.x_max - self.x_min) / 10) 79 | x = np.arange(self.x_min, self.x_max + self.x_min + step, step) 80 | 81 | # "good" area 82 | plt.plot(x, x, color='black', ls=':', lw=1.5, zorder=3) 83 | plt.plot(x, 0.1 * x, 'g:', lw=1.5, zorder=3) 84 | plt.plot(x, 10 * x, 'g:', lw=1.5, zorder=3) 85 | plt.fill_between(x, 0.1 * x, 10 * x, facecolor='green', alpha=0.15, 86 | zorder=3) 87 | 88 | plt.xlim([self.x_min, self.x_max]) 89 | plt.ylim([self.y_min, self.y_max]) 90 | 91 | # timeout lines 92 | if self.tlb_loc != 'none': 93 | plt.axvline(self.timeout, linewidth=1, color='red', ls=':', 94 | label=str(self.timeout), zorder=3) 95 | plt.axhline(self.timeout, linewidth=1, color='red', ls=':', 96 | label=str(self.timeout), zorder=3) 97 | 98 | if self.tlb_loc == 'after': 99 | plt.text(2 * self.x_min, self.timeout + self.x_max / 40, 100 | self.t_label, horizontalalignment='left', 101 | verticalalignment='bottom', fontsize=self.f_props['size'] * 0.8) 102 | plt.text(self.timeout + self.x_max / 40, 2 * self.x_min, 103 | self.t_label, horizontalalignment='left', 104 | verticalalignment='bottom', fontsize=self.f_props['size'] * 0.8, 105 | rotation=90) 106 | else: 107 | plt.text(2 * self.x_min, self.timeout - self.x_max / 3.5, 108 | self.t_label, horizontalalignment='left', 109 | verticalalignment='bottom', fontsize=self.f_props['size'] * 0.8) 110 | plt.text(self.timeout - self.x_max / 3.5, 2 * self.x_min, 111 | self.t_label, horizontalalignment='left', 112 | verticalalignment='bottom', fontsize=self.f_props['size'] * 0.8, 113 | rotation=90) 114 | 115 | # scatter 116 | plt.scatter(data[0][1], data[1][1], c=self.marker_style['color'], 117 | marker=self.marker_style['marker'], 118 | edgecolors=self.marker_style['edgecolor'], 119 | s=self.marker_style['size'], 120 | alpha=self.alpha, zorder=5) 121 | 122 | # axes' labels 123 | if self.x_label: 124 | plt.xlabel(self.x_label) 125 | else: 126 | plt.xlabel(data[0][0]) 127 | 128 | if self.y_label: 129 | plt.ylabel(self.y_label) 130 | else: 131 | plt.ylabel(data[1][0]) 132 | 133 | # turning the grid on 134 | if not self.no_grid: 135 | plt.grid(True, color='black', ls=':', lw=1, zorder=1) 136 | 137 | # choosing logarithmic scales 138 | ax = plt.gca() 139 | ax.set_xscale('log') 140 | ax.set_yscale('log') 141 | 142 | # setting ticks font properties 143 | # set_*ticklables() seems to be not needed in matplotlib 1.5.0 144 | if float(mpl_version[:3]) < 1.5: 145 | ax.set_xticklabels(ax.get_xticks(), self.f_props) 146 | ax.set_yticklabels(ax.get_yticks(), self.f_props) 147 | 148 | # formatter 149 | majorFormatter = plt.LogFormatterMathtext(base=10) 150 | ax.xaxis.set_major_formatter(majorFormatter) 151 | ax.yaxis.set_major_formatter(majorFormatter) 152 | 153 | # setting frame thickness 154 | for i in six.itervalues(ax.spines): 155 | i.set_linewidth(1) 156 | 157 | plt.savefig(self.save_to, bbox_inches='tight', transparent=self.transparent) 158 | 159 | # def create(self, data): 160 | # """ 161 | # Does the plotting. 162 | # """ 163 | 164 | # if len(data[0][1]) != len(data[1][1]): 165 | # raise ScatterException('Number of instances for each competitor must be the same') 166 | 167 | # # trying to remove duplicated points with the same coordinates 168 | # xs, ys = self.remove_dups(zip(data[0][1], data[1][1])) 169 | 170 | # step = int((self.x_max - self.x_min) / 10) 171 | # x = np.arange(self.x_min, self.x_max + self.x_min + step, step) 172 | 173 | # # "good" area 174 | # plt.plot(x, x, color='black', ls=':', lw=1.5, zorder=3) 175 | # plt.plot(x, 0.1 * x, 'g:', lw=1.5, zorder=3) 176 | # plt.plot(x, 10 * x, 'g:', lw=1.5, zorder=3) 177 | # plt.fill_between(x, 0.1 * x, 10 * x, facecolor='green', alpha=0.15, 178 | # zorder=3) 179 | 180 | # plt.xlim([self.x_min, self.x_max]) 181 | # plt.ylim([self.y_min, self.y_max]) 182 | 183 | # # timeout lines 184 | # plt.axvline(self.timeout, linewidth=1, color='red', ls=':', 185 | # label=str(self.timeout), zorder=3) 186 | # plt.axhline(self.timeout, linewidth=1, color='red', ls=':', 187 | # label=str(self.timeout), zorder=3) 188 | 189 | # if self.tlb_loc == 'after': 190 | # plt.text(2 * self.x_min, self.timeout + self.x_max / 40, 191 | # self.t_label, horizontalalignment='left', 192 | # verticalalignment='bottom', fontsize=self.f_props['size'] * 0.8) 193 | # plt.text(self.timeout + self.x_max / 40, 2 * self.x_min, 194 | # self.t_label, horizontalalignment='left', 195 | # verticalalignment='bottom', fontsize=self.f_props['size'] * 0.8, 196 | # rotation=90) 197 | # else: 198 | # plt.text(2 * self.x_min, self.timeout - self.x_max / 3.5, 199 | # self.t_label, horizontalalignment='left', 200 | # verticalalignment='bottom', fontsize=self.f_props['size'] * 0.8) 201 | # plt.text(self.timeout - self.x_max / 3.5, 2 * self.x_min, 202 | # self.t_label, horizontalalignment='left', 203 | # verticalalignment='bottom', fontsize=self.f_props['size'] * 0.8, 204 | # rotation=90) 205 | 206 | # # making the scatter plot step by step for each level 207 | # print sum([len(x) for x in xs]), len(data[0][1]) 208 | # xx = [] 209 | # yy = [] 210 | # rgba_c =[] 211 | # rgba_e =[] 212 | # for l in range(len(xs)): 213 | # xx.extend(xs[l]) 214 | # yy.extend(ys[l]) 215 | 216 | # # ctuple = (1., 0., 0., 1 - (1.0 - self.alpha) ** (l + 1)) 217 | # # rgba_c.extend([ctuple for i in xs[l]]) 218 | 219 | # # etuple = (0., 0., 0., 1 - (1.0 - self.alpha) ** (l + 1)) 220 | # # rgba_e.extend([etuple for i in xs[l]]) 221 | 222 | # # plt.scatter(xx, yy, c=rgba_c, marker='o', edgecolor=rgba_e, s=25, zorder=5) 223 | # plt.scatter(xx, yy, c='r', marker='o', alpha=self.alpha, s=25, zorder=5) 224 | 225 | # # axes' labels 226 | # if self.x_label: 227 | # plt.xlabel(self.x_label) 228 | # else: 229 | # plt.xlabel(data[0][0]) 230 | 231 | # if self.y_label: 232 | # plt.ylabel(self.y_label) 233 | # else: 234 | # plt.ylabel(data[1][0]) 235 | 236 | # # turning the grid on 237 | # if not self.no_grid: 238 | # plt.grid(True, color='black', ls=':', lw=1, zorder=1) 239 | 240 | # # choosing logarithmic scales 241 | # ax = plt.gca() 242 | # ax.set_xscale('log') 243 | # ax.set_yscale('log') 244 | 245 | # # setting ticks font properties 246 | # ax.set_xticklabels(ax.get_xticks(), self.f_props) 247 | # ax.set_yticklabels(ax.get_yticks(), self.f_props) 248 | 249 | # # formatter 250 | # majorFormatter = plt.LogFormatterMathtext(base=10) 251 | # ax.xaxis.set_major_formatter(majorFormatter) 252 | # ax.yaxis.set_major_formatter(majorFormatter) 253 | 254 | # # setting frame thickness 255 | # for i in six.itervalues(ax.spines): 256 | # i.set_linewidth(1) 257 | 258 | # plt.savefig(self.save_to, bbox_inches='tight', transparent=self.transparent) 259 | 260 | # def remove_dups(self, data): 261 | # """ 262 | # Removes duplicated points. 263 | # """ 264 | 265 | # pset = set([]) 266 | # pdup = {} 267 | # distance = lambda p,q: ((p[0] - q[0]) ** 2 + (p[1] - q[1]) ** 2) ** 0.5 268 | 269 | # for p in data: 270 | # for q in pset: 271 | # if distance(p, q) == 0: 272 | # pdup[q] += 1 273 | # break 274 | # else: 275 | # pdup[p] = 1 276 | # pset.add(p) 277 | 278 | # maxdups = max(six.itervalues(pdup)) 279 | # xs = [[] for i in range(maxdups)] 280 | # ys = [[] for i in range(maxdups)] 281 | 282 | # for p, l in six.iteritems(pdup): 283 | # xs[l - 1].append(p[0]) 284 | # ys[l - 1].append(p[1]) 285 | 286 | # return xs, ys 287 | -------------------------------------------------------------------------------- /statutil.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | #-*- coding:utf-8 -*- 3 | ## 4 | ## statutil.py 5 | ## 6 | ## Created on: May 22, 2013 7 | ## Author: Alexey S. Ignatiev 8 | ## E-mail: aignatiev@ciencias.ulisboa.pt 9 | ## 10 | 11 | # 12 | #============================================================================== 13 | from __future__ import print_function 14 | import json 15 | import sys 16 | 17 | 18 | # 19 | #============================================================================== 20 | class JSONException(Exception): 21 | pass 22 | 23 | 24 | # 25 | #============================================================================== 26 | class Stat: 27 | """ 28 | Simple statistical data class. 29 | """ 30 | 31 | def __init__(self, filename=None): 32 | """ 33 | Constructor. 34 | """ 35 | 36 | if filename is None: 37 | self.insts_own = [] 38 | self.preamble = {} 39 | self.data = {} 40 | elif type(filename) is list: 41 | print( 'in case of several files use "StatArray" class', file=sys.stderr) 42 | else: 43 | self.read(filename) 44 | 45 | def read(self, filename=None): 46 | """ 47 | Reads a file into a Stat object. 48 | """ 49 | 50 | if filename is None: 51 | print( 'no filename was specified', file=sys.stderr) 52 | return 53 | 54 | with open(filename, 'r') as fp: 55 | print('reading {0}'.format(filename), file=sys.stderr) 56 | try: 57 | data_full = json.load(fp) 58 | except: 59 | raise JSONException('Unable to parse \'{0}\'.'.format(filename)) 60 | 61 | self.data = data_full['stats'] 62 | self.preamble = data_full['preamble'] 63 | self.preamble['origin'] = filename 64 | 65 | self.insts_own = sorted(list(set(self.data.keys()))) 66 | 67 | def write(self, to=None): 68 | """ 69 | Writes a Stat object to a file. 70 | """ 71 | 72 | to_write = {'preamble': self.preamble, 'stats': self.data} 73 | 74 | if to is None: 75 | to = self.preamble['origin'] 76 | 77 | # 'origin' field is not needed anymore 78 | del(self.preamble['origin']) 79 | 80 | if type(to) is str: 81 | with open(to, 'w') as fp: 82 | json.dump(to_write, fp, indent=4, separators=(',', ': ')) 83 | elif type(to) is file: 84 | json.dump(to_write, to, indent=4, separators=(',', ': ')) 85 | else: 86 | print('don\'t know how to write to {0}'.format(type(to)), file=sys.stderr) 87 | 88 | def update(self, success=None, failure=None): 89 | """ 90 | Updates stats using additional success and failure signs. 91 | """ 92 | 93 | if success: 94 | pass 95 | 96 | if failure: 97 | sign = lambda x: x 98 | key = failure 99 | if failure[:3] == 'no-': 100 | sign = lambda x: not x 101 | key = failure[3:] 102 | 103 | for inst in self.insts_own: 104 | if inst in self.data and self.data[inst]['status'] == True: 105 | if sign(key in self.data[inst]): 106 | print('updating', inst, file=sys.stderr) 107 | self.data[inst]['status'] = False 108 | 109 | self.write() 110 | 111 | 112 | def list(self, crit=None): 113 | """ 114 | Lists instances satisfying the criterion. 115 | """ 116 | 117 | if crit: 118 | pred = lambda x: x == crit['val'] 119 | if crit['pred'] == '<': 120 | pred = lambda x: x < crit['val'] 121 | elif crit['pred'] == '<=': 122 | pred = lambda x: x <= crit['val'] 123 | elif crit['pred'] == '>': 124 | pred = lambda x: x > crit['val'] 125 | elif crit['pred'] == '>=': 126 | pred = lambda x: x >= crit['val'] 127 | 128 | for inst in self.insts_own: 129 | if inst in self.data and self.data[inst]['status'] == True: 130 | if pred(self.data[inst][crit['key']]): 131 | print('{0}: {1} = {2}'.format(inst, crit['key'], self.data[inst][crit['key']])) 132 | 133 | 134 | # 135 | #============================================================================== 136 | class StatArray: 137 | """ 138 | Contains statistical data for several files. 139 | """ 140 | 141 | def __init__(self, files=None): 142 | """ 143 | Constructor. 144 | """ 145 | 146 | if files is None: 147 | self.inst_full = [] 148 | self.stat_objs = [] 149 | elif type(files) is list: 150 | self.read(files) 151 | else: 152 | print('in case of just one file use "Stat" class', file=sys.stderr) 153 | self.read([files]) 154 | 155 | def __getitem__(self, key): 156 | if key < len(self.stat_objs): 157 | return self.stat_objs[key] 158 | 159 | def __len__(self): 160 | return len(self.stat_objs) 161 | 162 | def __iter__(self): 163 | for stat_obj in self.stat_objs: 164 | yield stat_obj 165 | 166 | def read(self, files=None): 167 | """ 168 | Reads several files into a StatArray object. 169 | """ 170 | 171 | if files is None: 172 | print('no files was specified', file=sys.stderr) 173 | return 174 | 175 | self.stat_objs = [] 176 | for f in files: 177 | self.stat_objs.append(Stat(f)) 178 | 179 | inst_set = set() 180 | for stat_obj in self.stat_objs: 181 | inst_set = inst_set.union(set(stat_obj.insts_own)) 182 | self.inst_full = sorted(list(inst_set)) 183 | 184 | def write(self, files=None): 185 | """ 186 | Writes a StatArray object to given files. 187 | """ 188 | 189 | if files is None: 190 | files = [stat_obj.preamble['origin'] for stat_obj in self.stat_objs] 191 | 192 | assert len(files) == len(self.stat_objs), 'wrong number of filenames' 193 | 194 | for f, stat_obj in zip(files, self.stat_objs): 195 | stat_obj.write(f) 196 | 197 | def cluster(self, use_key=['program', 'prog_args']): 198 | """ 199 | Clasters Stat objects according to their preamble values. 200 | """ 201 | 202 | # the key should be a list 203 | if type(use_key) is not list: 204 | use_key = [use_key] 205 | 206 | clusters = {} 207 | 208 | for stat_obj in self.stat_objs: 209 | # updating the Stat object 210 | for i, i_old in enumerate(stat_obj.insts_own): 211 | i_new = '{0}@{1}'.format(i_old, stat_obj.preamble['benchmark']) 212 | stat_obj.insts_own[i] = i_new 213 | stat_obj.data[i_new] = stat_obj.data.pop(i_old) 214 | 215 | key = ' '.join([stat_obj.preamble[one_key] for one_key in use_key]) 216 | if key in clusters: 217 | # update the cluster 218 | clusters[key].insts_own.extend(stat_obj.insts_own) 219 | clusters[key].data.update(stat_obj.data) 220 | 221 | clusters[key].preamble['benchmark'].append(stat_obj.preamble['benchmark']) 222 | clusters[key].preamble['runsolver_args'].append(stat_obj.preamble['runsolver_args']) 223 | else: 224 | # add new cluster 225 | clusters[key] = stat_obj 226 | clusters[key].preamble['benchmark'] = [clusters[key].preamble['benchmark']] 227 | clusters[key].preamble['runsolver_args'] = [clusters[key].preamble['runsolver_args']] 228 | 229 | self.stat_objs = [cl for cl in clusters.values()] 230 | 231 | inst_set = set() 232 | for stat_obj in self.stat_objs: 233 | inst_set = inst_set.union(set(stat_obj.insts_own)) 234 | self.inst_full = sorted(list(inst_set)) 235 | 236 | def unclaster(self): 237 | """ 238 | Unclasters previously clastered Stat objects. 239 | """ 240 | 241 | print('unclaster() method is not implemented yet', file=sys.stderr) 242 | 243 | def make_vbs(self, addit_key=None): 244 | """ 245 | Makes vbs using the status, rtime and additional key as the measurement. 246 | NOTE: the use of addit_key is not implemented yet. 247 | """ 248 | 249 | vbs = Stat() 250 | vbs.insts_own = self.inst_full 251 | 252 | vbs.preamble = self.stat_objs[0].preamble 253 | vbs.preamble['program'] = 'vbs' 254 | vbs.preamble['prog_args'] = '' 255 | vbs.preamble['origin'] = [obj.preamble['origin'] for obj in self.stat_objs] 256 | 257 | for inst in self.inst_full: 258 | alts = [] 259 | for stat_obj in self.stat_objs: 260 | if inst in stat_obj.data and stat_obj.data[inst]['status'] == True: 261 | alts.append(stat_obj.data[inst]) 262 | 263 | if alts: 264 | vbs.data[inst] = min(alts, key=lambda x: x['rtime']) 265 | else: 266 | # all fail; choose any: 267 | vbs.data[inst] = self.stat_objs[0].data[inst] 268 | 269 | self.stat_objs.append(vbs) 270 | 271 | def compare(self, cmp_key=None): 272 | """ 273 | Compares values for a specific key. Do nothing if cmp_key is None. 274 | """ 275 | 276 | if cmp_key: 277 | for inst in self.inst_full: 278 | vals = {} 279 | 280 | for stat_obj in self.stat_objs: 281 | if inst in stat_obj.data and stat_obj.data[inst]['status'] == True and cmp_key in stat_obj.data[inst]: 282 | if stat_obj.data[inst][cmp_key] in vals: 283 | vals[stat_obj.data[inst][cmp_key]].append(stat_obj.preamble['origin']) 284 | else: 285 | vals[stat_obj.data[inst][cmp_key]] = [stat_obj.preamble['origin']] 286 | 287 | if len(vals.keys()) > 1: 288 | print('different values found', file=sys.stderr) 289 | print('instance:', inst, file=sys.stderr) 290 | print('values:', vals, file=sys.stderr) 291 | 292 | def list_simple(self, to_list='all'): 293 | """ 294 | Shows instances required by user. 295 | """ 296 | 297 | if to_list: 298 | print('showing {0}:'.format(to_list)) 299 | if to_list == 'all': 300 | for inst in self.inst_full: 301 | print(inst) 302 | 303 | else: 304 | status = False if to_list == 'failed' else True 305 | 306 | for inst in self.inst_full: 307 | objs = [] 308 | 309 | for stat_obj in self.stat_objs: 310 | if inst in stat_obj.data and stat_obj.data[inst]['status'] == status: 311 | p = stat_obj.preamble 312 | if 'prog_alias' in p: 313 | objs.append(p['prog_alias']) 314 | else: 315 | objs.append(p['program'] + ' ' + p['prog_args']) 316 | 317 | if objs: 318 | if len(self.stat_objs) > 1: 319 | objs = '[{0}]'.format(', '.join(obj for obj in objs)) 320 | print('{0}: {1}'.format(inst, objs)) 321 | else: 322 | print(inst) 323 | 324 | def list(self, crit=None): 325 | """ 326 | Shows instances required by user. 327 | """ 328 | 329 | if crit: 330 | for stat_obj in self.stat_objs: 331 | stat_obj.list(crit) 332 | 333 | def update(self, success=None, failure=None): 334 | """ 335 | Update stats using additional success and failure signs. 336 | """ 337 | 338 | if success or failure: 339 | for stat_obj in self.stat_objs: 340 | stat_obj.update(success, failure) 341 | --------------------------------------------------------------------------------