├── LICENSE ├── README.md ├── borgbench.py ├── chunking ├── csv │ ├── results_images_lz4.csv │ ├── results_images_lzma.csv │ ├── results_images_zlib.csv │ ├── results_rootfs_lz4.csv │ ├── results_rootfs_zlib.csv │ ├── results_vm_lz4.csv │ ├── results_vm_lzma.csv │ └── results_vm_zlib.csv ├── img │ ├── results_images_chunking.png │ ├── results_images_lz4.png │ ├── results_images_lzma.png │ ├── results_images_zlib.png │ ├── results_rootfs_chunking.png │ ├── results_rootfs_lz4.png │ ├── results_rootfs_zlib.png │ ├── results_vm_chunking.png │ ├── results_vm_lz4.png │ ├── results_vm_lzma.png │ └── results_vm_zlib.png ├── results_images.ods ├── results_rootfs.ods ├── results_vm.ods └── testdata.txt └── compression ├── img ├── results_rootfs_lzma.png └── results_rootfs_zlib.png ├── results_rootfs.ods └── testdata.txt /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016 Michael Gajda 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | borgbench 2 | ========= 3 | 4 | Compares speed and efficiency of borgbackup compression and chunker settings. 5 | 6 | Usage: 7 | 8 | ./borgbench.py /path/to/testdata 9 | 10 | To produce accurate results for your use case, you need to provide your own 11 | test data. 12 | Create a directory with a collection of files resembling what you'll be backing 13 | up with borg later, then run this script with the test data directory as an 14 | argument. 15 | 16 | Ideally, you should store the test data on a tmpfs ("RAM disk") so the results 17 | won't be skewed by your disk speed. 18 | 19 | The script will copy your test data to the system's temporary storage, so make 20 | sure there's enough space in there to temporarily store your entire set of test 21 | data. 22 | 23 | borg 1.1 24 | -------- 25 | 26 | Borg 1.1 introduced a new option to produce output in JSON format. Besides 27 | providing a stable interface, this also finally provides byte-accurate 28 | information about the size of archives and repositories (which is useful to see 29 | small differences in compression ratio on large repositories, which would have 30 | been lost to rounding until now). 31 | 32 | Borgbench has been adapted to support both the old "human-readable" and the new 33 | JSON format. 34 | To keep things simple, it checks whether `borg help create` mentions the 35 | `--json` option (once, at launch), then selects the borg invocation options and 36 | parser method accordingly. 37 | If the JSON format is not supported, a warning is printed to stderr (which 38 | shouldn't be a problem since the actual CSV results are printed to stdout, so 39 | they can be easily separated from any error messages). 40 | 41 | Borg 1.1 also introduced a lower boundary for the chunker max_size based on the 42 | configured min_size, so two chunker configurations had to be dropped from the 43 | benchmark (though they were rather exotic anyway). 44 | 45 | Attribution 46 | ----------- 47 | 48 | [Original idea and implementation][1] by Michael Gajda (dragetd), [improved and 49 | modularised][2] for general use by Nils Steinger (n-st). 50 | 51 | Test results (in the `chunking` and `compression` subdirectories) provided by 52 | Michael Gajda. 53 | 54 | [1]: https://github.com/dragetd/borgbench 55 | [2]: https://github.com/n-st/borgbench 56 | -------------------------------------------------------------------------------- /borgbench.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import subprocess 4 | import re 5 | from tempfile import TemporaryDirectory 6 | from timeit import default_timer as timer 7 | import sys 8 | import os.path 9 | import errno 10 | import json 11 | 12 | def print_header(borg_supports_json=False): 13 | info_items = [] 14 | info_items += ['Compression setting'] 15 | info_items += ['CHUNK_MIN_EXP'] 16 | info_items += ['CHUNK_MAX_EXP'] 17 | info_items += ['CHUNK_HASH_MASK_BITS'] 18 | if borg_supports_json: 19 | # JSON format contains raw byte values 20 | info_items += ['Original size [bytes]'] 21 | info_items += ['Compressed size [bytes]'] 22 | info_items += ['Deduplicated size [bytes]'] 23 | else: 24 | # Human format contains strings like "123.4 MB" 25 | info_items += ['Original size'] 26 | info_items += ['Compressed size'] 27 | info_items += ['Deduplicated size'] 28 | info_items += ['Unique chunks'] 29 | info_items += ['Total chunks'] 30 | info_items += ['Duration [seconds]'] 31 | print(';'.join(map(str, info_items))) 32 | 33 | def parse_human_output(output_str): 34 | m = re.match(r'.*This archive: +(\d+\.?\d? .?B) +(\d+\.?\d? .?B) +(\d+\.?\d? .?B).*Chunk index: +(\d+) +(\d+)', output_str, flags=re.DOTALL) 35 | if m: 36 | return m.group(1, 2, 3, 4, 5) 37 | else: 38 | return None 39 | 40 | def parse_json_output(output_str): 41 | try: 42 | j = json.loads(output_str) 43 | return ( 44 | j["archive"]["stats"]["original_size"], 45 | j["archive"]["stats"]["compressed_size"], 46 | j["archive"]["stats"]["deduplicated_size"], 47 | j["cache"]["stats"]["total_unique_chunks"], 48 | j["cache"]["stats"]["total_chunks"], 49 | ) 50 | except: 51 | return None 52 | 53 | # single benchmark run 54 | def runConfig(inputdir, compression="none", chunker_params=None, borg_supports_json=False): 55 | """ If given, chunker_params should be a tuple of (cmin, cmax, cavg). """ 56 | 57 | with TemporaryDirectory(prefix='borgbench_') as tempdir: 58 | returncode = subprocess.call(["borg", "init", "-e", "none", tempdir]) 59 | if returncode != 0: 60 | sys.stderr.write('`borg init` reported failure\n') 61 | return 62 | 63 | commandline = ["borg"] 64 | commandline += ["create", "-v", "-s", "-C", compression] 65 | if chunker_params: 66 | commandline += ["--chunker-params=%d,%d,%d,4095" % (chunker_params)] 67 | if borg_supports_json: 68 | commandline += ['--json'] 69 | commandline += [tempdir+"::test"] 70 | commandline += [inputdir] 71 | 72 | env = os.environ.copy() 73 | env["BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK"] = "yes" 74 | 75 | start = timer() 76 | proc = subprocess.Popen(commandline, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) 77 | output = proc.stdout.read().decode('utf-8') 78 | errput = proc.stderr.read().decode('utf-8') 79 | duration = timer() - start 80 | 81 | if borg_supports_json: 82 | result = parse_json_output(output) 83 | else: 84 | result = parse_human_output(errput) 85 | 86 | if result: 87 | if chunker_params: 88 | cmin, cmax, cavg = chunker_params 89 | else: 90 | cmin, cmax, cavg = 0, 0, 0 91 | original_size, compressed_size, dedup_size, unique_chunks, total_chunks = result 92 | info_items = [compression, cmin, cmax, cavg, original_size, compressed_size, dedup_size, unique_chunks, total_chunks, duration] 93 | print(';'.join(map(str, info_items))) 94 | 95 | else: 96 | sys.stderr.write('!! Could not parse borg output:\n') 97 | sys.stderr.write(output) 98 | sys.stderr.write(errput) 99 | 100 | def check_borg_json_support(): 101 | commandline = ["borg", "help", "create"] 102 | proc = subprocess.Popen(commandline, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) 103 | output = proc.stdout.read().decode('utf-8') 104 | return ' --json ' in output 105 | 106 | 107 | compression_settings = [ 108 | "none", 109 | "lz4", 110 | "zlib,0", 111 | "zlib,1", 112 | "zlib,2", 113 | "zlib,3", 114 | "zlib,4", 115 | "zlib,5", 116 | "zlib,6", 117 | "zlib,7", 118 | "zlib,8", 119 | "zlib,9", 120 | "lzma,0", 121 | "lzma,1", 122 | "lzma,2", 123 | "lzma,3", 124 | "lzma,4", 125 | "lzma,5", 126 | "lzma,6", 127 | "lzma,7", 128 | "lzma,8", 129 | "lzma,9", 130 | ] 131 | 132 | chunker_settings = [ 133 | (9, 14, 10), 134 | (8, 13, 11), 135 | (9, 17, 11), 136 | (9, 14, 12), 137 | (10, 17, 12), 138 | (10, 15, 13), 139 | (11, 18, 13), 140 | (11, 16, 14), 141 | (12, 20, 14), 142 | (12, 19, 15), 143 | (13, 20, 16), 144 | (14, 21, 17), 145 | (14, 20, 18), 146 | (15, 21, 19), 147 | (16, 22, 20), 148 | (20, 23, 21), 149 | ] 150 | 151 | 152 | def print_usage(): 153 | sys.stderr.write( 154 | """borgbench: 155 | compare speed and efficiency of borgbackup compression and chunker settings 156 | 157 | Usage: 158 | %s 159 | 160 | To produce accurate results for your use case, you need to provide your own 161 | test data. Create a directory with a collection of files resembling what 162 | you'll be backing up with borg later, then run this script with the test data 163 | directory as an argument. 164 | Ideally, you should store the test data on a tmpfs ("RAM disk") so the results 165 | won't be skewed by your disk speed. 166 | The script will copy your test data to the system's temporary storage, so make 167 | sure there's enough space in there to temporarily store your entire set of test 168 | data. 169 | """ % sys.argv[0]) 170 | 171 | if __name__ == "__main__": 172 | if len(sys.argv) != 2 or sys.argv[1] in ['-h', '--help']: 173 | print_usage() 174 | sys.exit(errno.EINVAL) 175 | 176 | testData = sys.argv[1] 177 | 178 | if not os.path.exists(testData): 179 | sys.stderr.write('The path to your test data (%s) doesn\'t exist.\n') 180 | print_usage() 181 | sys.exit(errno.ENOENT) 182 | 183 | borg_supports_json = check_borg_json_support() 184 | if not borg_supports_json: 185 | sys.stderr.write("Your version of borg does not yet support the new " + 186 | "JSON output format (introduced in borg 1.1).\n" + 187 | "Falling back to parsing human-readable output (which " + 188 | "doesn't provide byte-accurate values).\n") 189 | 190 | print_header(borg_supports_json=borg_supports_json) 191 | 192 | for params in chunker_settings: 193 | # Deduplication is done based on the contents of chunks *before* they are 194 | # compressed, so we don't need to use any compression during the chunker 195 | # benchmark. 196 | runConfig(testData, chunker_params=params, 197 | borg_supports_json=borg_supports_json) 198 | 199 | for comp in compression_settings: 200 | runConfig(testData, comp, borg_supports_json=borg_supports_json) 201 | -------------------------------------------------------------------------------- /chunking/csv/results_images_lz4.csv: -------------------------------------------------------------------------------- 1 | lz4;8;12;10;304.44 MB;297.06 MB;297.06 MB;73768;73786 2 | lz4;9;14;10;309.59 MB;303.18 MB;303.10 MB;196451;196513 3 | lz4;8;13;11;306.18 MB;299.15 MB;299.12 MB;115132;115160 4 | lz4;9;17;11;306.29 MB;299.44 MB;299.38 MB;117915;117944 5 | lz4;9;14;12;304.09 MB;296.76 MB;296.72 MB;65339;65354 6 | lz4;10;17;12;303.82 MB;296.46 MB;296.42 MB;58882;58894 7 | lz4;10;15;13;302.74 MB;295.03 MB;295.01 MB;33242;33249 8 | lz4;11;18;13;302.58 MB;294.83 MB;294.81 MB;29341;29346 9 | lz4;11;16;14;302.05 MB;294.00 MB;293.99 MB;16658;16659 10 | lz4;12;20;14;301.96 MB;293.89 MB;293.89 MB;14755;14755 11 | lz4;12;19;15;301.69 MB;293.43 MB;293.43 MB;8116;8116 12 | lz4;13;20;16;301.53 MB;293.14 MB;293.14 MB;4152;4152 13 | lz4;14;21;17;301.44 MB;292.99 MB;292.99 MB;2158;2158 14 | lz4;14;20;18;301.40 MB;292.94 MB;292.94 MB;1240;1240 15 | lz4;15;21;19;301.37 MB;292.90 MB;292.90 MB;657;657 16 | lz4;16;22;20;301.36 MB;292.89 MB;292.89 MB;394;394 17 | lz4;20;23;21;301.35 MB;292.87 MB;292.87 MB;180;180 18 | lz4;18;18;18;301.40 MB;292.93 MB;292.93 MB;1213;1213 19 | -------------------------------------------------------------------------------- /chunking/csv/results_images_lzma.csv: -------------------------------------------------------------------------------- 1 | lzma;8;12;10;304.44 MB;297.94 MB;297.94 MB;73756;73774 2 | lzma;9;14;10;309.59 MB;310.36 MB;310.28 MB;196449;196511 3 | lzma;8;13;11;306.18 MB;302.02 MB;301.99 MB;115137;115165 4 | lzma;9;17;11;306.29 MB;302.50 MB;302.44 MB;117901;117930 5 | lzma;9;14;12;304.09 MB;296.92 MB;296.88 MB;65346;65361 6 | lzma;10;17;12;303.82 MB;296.28 MB;296.24 MB;58900;58912 7 | lzma;10;15;13;302.74 MB;293.22 MB;293.19 MB;33232;33239 8 | lzma;11;18;13;302.58 MB;292.78 MB;292.76 MB;29334;29339 9 | lzma;11;16;14;302.05 MB;290.94 MB;290.94 MB;16661;16662 10 | lzma;12;20;14;301.97 MB;290.67 MB;290.67 MB;14761;14761 11 | lzma;12;19;15;301.69 MB;289.41 MB;289.41 MB;8122;8122 12 | lzma;13;20;16;301.53 MB;288.47 MB;288.47 MB;4148;4148 13 | lzma;14;21;17;301.44 MB;287.80 MB;287.80 MB;2161;2161 14 | lzma;14;20;18;301.40 MB;287.36 MB;287.36 MB;1239;1239 15 | lzma;15;21;19;301.37 MB;286.88 MB;286.88 MB;656;656 16 | lzma;16;22;20;301.36 MB;286.53 MB;286.53 MB;396;396 17 | lzma;20;23;21;301.35 MB;286.19 MB;286.19 MB;181;181 18 | lzma;18;18;18;301.40 MB;287.46 MB;287.46 MB;1213;1213 19 | -------------------------------------------------------------------------------- /chunking/csv/results_images_zlib.csv: -------------------------------------------------------------------------------- 1 | zlib;8;12;10;304.44 MB;294.59 MB;294.59 MB;73760;73778 2 | zlib;9;14;10;309.59 MB;301.95 MB;301.88 MB;196450;196512 3 | zlib;8;13;11;306.18 MB;296.86 MB;296.83 MB;115147;115175 4 | zlib;9;17;11;306.30 MB;297.32 MB;297.26 MB;117939;117968 5 | zlib;9;14;12;304.09 MB;293.79 MB;293.75 MB;65353;65368 6 | zlib;10;17;12;303.82 MB;293.45 MB;293.41 MB;58896;58908 7 | zlib;10;15;13;302.74 MB;291.48 MB;291.46 MB;33226;33233 8 | zlib;11;18;13;302.58 MB;291.22 MB;291.20 MB;29332;29337 9 | zlib;11;16;14;302.05 MB;290.13 MB;290.12 MB;16676;16677 10 | zlib;12;20;14;301.96 MB;289.96 MB;289.96 MB;14745;14745 11 | zlib;12;19;15;301.69 MB;289.30 MB;289.30 MB;8119;8119 12 | zlib;13;20;16;301.53 MB;288.85 MB;288.85 MB;4148;4148 13 | zlib;14;21;17;301.44 MB;288.63 MB;288.63 MB;2162;2162 14 | zlib;14;20;18;301.40 MB;288.53 MB;288.53 MB;1240;1240 15 | zlib;15;21;19;301.37 MB;288.47 MB;288.47 MB;660;660 16 | zlib;16;22;20;301.36 MB;288.44 MB;288.44 MB;396;396 17 | zlib;20;23;21;301.35 MB;288.41 MB;288.41 MB;179;179 18 | zlib;18;18;18;301.40 MB;288.51 MB;288.51 MB;1214;1214 19 | -------------------------------------------------------------------------------- /chunking/csv/results_rootfs_lz4.csv: -------------------------------------------------------------------------------- 1 | lz4;8;12;10;3.00 GB;1.69 GB;1.61 GB;728943;781731 2 | lz4;9;14;10;3.04 GB;1.82 GB;1.69 GB;1613086;1765526 3 | lz4;8;13;11;3.02 GB;1.72 GB;1.60 GB;1003998;1099154 4 | lz4;9;17;11;3.02 GB;1.73 GB;1.61 GB;1003850;1094531 5 | lz4;9;14;12;3.00 GB;1.65 GB;1.54 GB;605551;659545 6 | lz4;10;17;12;2.99 GB;1.65 GB;1.53 GB;547144;593998 7 | lz4;10;15;13;2.99 GB;1.60 GB;1.49 GB;352490;381991 8 | lz4;11;18;13;2.98 GB;1.59 GB;1.48 GB;318871;344626 9 | lz4;11;16;14;2.98 GB;1.55 GB;1.45 GB;222088;239460 10 | lz4;12;20;14;2.98 GB;1.55 GB;1.45 GB;204124;219725 11 | lz4;12;19;15;2.98 GB;1.53 GB;1.44 GB;154897;166406 12 | lz4;13;20;16;2.98 GB;1.51 GB;1.43 GB;123168;132013 13 | lz4;14;21;17;2.97 GB;1.51 GB;1.43 GB;107471;115127 14 | lz4;14;20;18;2.97 GB;1.50 GB;1.43 GB;100959;108072 15 | lz4;15;21;19;2.97 GB;1.50 GB;1.43 GB;96805;103655 16 | lz4;16;22;20;2.97 GB;1.50 GB;1.43 GB;94854;101589 17 | lz4;20;23;21;2.97 GB;1.50 GB;1.43 GB;93516;100146 18 | lz4;18;18;18;2.97 GB;1.50 GB;1.44 GB;99969;106959 19 | -------------------------------------------------------------------------------- /chunking/csv/results_rootfs_zlib.csv: -------------------------------------------------------------------------------- 1 | zlib;8;12;10;3.00 GB;1.26 GB;1.20 GB;728906;781696 2 | zlib;9;14;10;3.04 GB;1.41 GB;1.31 GB;1613079;1765523 3 | zlib;8;13;11;3.02 GB;1.30 GB;1.21 GB;1004029;1099202 4 | zlib;9;17;11;3.02 GB;1.31 GB;1.22 GB;1003859;1094536 5 | zlib;9;14;12;3.00 GB;1.23 GB;1.15 GB;605544;659535 6 | zlib;10;17;12;2.99 GB;1.22 GB;1.14 GB;547089;593925 7 | zlib;10;15;13;2.99 GB;1.17 GB;1.10 GB;352461;381956 8 | zlib;11;18;13;2.98 GB;1.17 GB;1.09 GB;318819;344581 9 | zlib;11;16;14;2.98 GB;1.14 GB;1.07 GB;222084;239456 10 | zlib;12;20;14;2.98 GB;1.13 GB;1.07 GB;204148;219749 11 | zlib;12;19;15;2.98 GB;1.11 GB;1.05 GB;154875;166384 12 | zlib;13;20;16;2.98 GB;1.10 GB;1.04 GB;123154;131999 13 | zlib;14;21;17;2.97 GB;1.09 GB;1.04 GB;107439;115095 14 | zlib;14;20;18;2.97 GB;1.09 GB;1.03 GB;100917;108030 15 | zlib;15;21;19;2.97 GB;1.08 GB;1.04 GB;96784;103634 16 | zlib;16;22;20;2.97 GB;1.08 GB;1.04 GB;94828;101563 17 | zlib;20;23;21;2.97 GB;1.08 GB;1.04 GB;93487;100117 18 | zlib;18;18;18;2.97 GB;1.09 GB;1.04 GB;99931;106921 19 | -------------------------------------------------------------------------------- /chunking/csv/results_vm_lz4.csv: -------------------------------------------------------------------------------- 1 | lz4;8;12;10;7.59 GB;6.52 GB;5.28 GB;1484121;1838713 2 | lz4;9;14;10;7.72 GB;6.76 GB;5.33 GB;3747225;4803296 3 | lz4;8;13;11;7.63 GB;6.58 GB;5.27 GB;2233899;2828335 4 | lz4;9;16;11;7.64 GB;6.60 GB;5.26 GB;2260749;2872236 5 | lz4;9;14;12;7.58 GB;6.48 GB;5.25 GB;1278309;1600014 6 | lz4;10;16;12;7.58 GB;6.46 GB;5.24 GB;1146788;1433849 7 | lz4;10;15;13;7.55 GB;6.38 GB;5.32 GB;660416;810910 8 | lz4;11;18;13;7.55 GB;6.37 GB;5.32 GB;584096;714684 9 | lz4;11;16;14;7.53 GB;6.31 GB;5.41 GB;339053;409266 10 | lz4;12;19;14;7.53 GB;6.31 GB;5.42 GB;300196;358873 11 | lz4;11;17;15;7.53 GB;6.27 GB;5.52 GB;183987;218582 12 | lz4;12;19;15;7.52 GB;6.27 GB;5.52 GB;170684;200124 13 | lz4;12;18;16;7.52 GB;6.25 GB;5.62 GB;95123;108490 14 | lz4;13;20;16;7.52 GB;6.24 GB;5.62 GB;88314;100366 15 | lz4;13;19;17;7.52 GB;6.22 GB;5.72 GB;47810;52926 16 | lz4;14;21;17;7.52 GB;6.22 GB;5.74 GB;44309;48841 17 | lz4;14;20;18;7.52 GB;6.22 GB;5.81 GB;24421;26545 18 | lz4;15;22;18;7.52 GB;6.21 GB;5.84 GB;22646;24529 19 | lz4;15;21;19;7.52 GB;6.21 GB;5.91 GB;12205;13014 20 | lz4;16;23;19;7.52 GB;6.21 GB;5.93 GB;11279;11997 21 | lz4;16;22;20;7.52 GB;6.21 GB;5.99 GB;6200;6474 22 | lz4;18;23;20;7.52 GB;6.21 GB;6.00 GB;5256;5458 23 | lz4;17;22;21;7.52 GB;6.21 GB;6.04 GB;3700;3820 24 | lz4;20;23;21;7.52 GB;6.21 GB;6.06 GB;2328;2384 25 | lz4;23;23;23;7.52 GB;6.21 GB;6.21 GB;900;900 26 | lz4;23;23;32;7.52 GB;6.21 GB;6.21 GB;900;900 27 | lz4;20;20;20;7.52 GB;6.21 GB;6.18 GB;7134;7187 28 | lz4;20;20;23;7.52 GB;6.21 GB;6.18 GB;7134;7187 29 | lz4;20;20;15;7.52 GB;6.21 GB;6.18 GB;7134;7187 30 | -------------------------------------------------------------------------------- /chunking/csv/results_vm_lzma.csv: -------------------------------------------------------------------------------- 1 | lzma;7.59 GB;5.74 GB;4.66 GB;1484252;1838845 2 | lzma;7.72 GB;6.20 GB;4.92 GB;3747372;4803458 3 | lzma;7.63 GB;5.87 GB;4.73 GB;2233987;2828310 4 | lzma;7.64 GB;5.90 GB;4.73 GB;2260735;2872192 5 | lzma;7.58 GB;5.67 GB;4.63 GB;1278418;1600111 6 | Error 7 | lzma;7.55 GB;5.50 GB;4.63 GB;660423;810918 8 | lzma;7.55 GB;5.48 GB;4.62 GB;584078;714683 9 | lzma;7.53 GB;5.39 GB;4.67 GB;339097;409309 10 | lzma;7.53 GB;5.37 GB;4.66 GB;300200;358882 11 | lzma;7.53 GB;5.30 GB;4.71 GB;183977;218575 12 | lzma;7.52 GB;5.30 GB;4.71 GB;170722;200164 13 | lzma;7.52 GB;5.23 GB;4.75 GB;95135;108503 14 | lzma;7.52 GB;5.23 GB;4.75 GB;88334;100386 15 | lzma;7.52 GB;5.11 GB;4.75 GB;47810;52926 16 | lzma;7.52 GB;5.10 GB;4.75 GB;44308;48840 17 | lzma;7.52 GB;5.04 GB;4.76 GB;24435;26559 18 | lzma;7.52 GB;5.03 GB;4.76 GB;22659;24542 19 | lzma;7.52 GB;4.96 GB;4.76 GB;12212;13021 20 | Error 21 | Error 22 | lzma;7.52 GB;4.90 GB;4.75 GB;5251;5453 23 | Error 24 | Error 25 | lzma;7.52 GB;4.78 GB;4.78 GB;898;898 26 | lzma;7.52 GB;4.78 GB;4.78 GB;898;898 27 | Error 28 | Error 29 | Error 30 | -------------------------------------------------------------------------------- /chunking/csv/results_vm_zlib.csv: -------------------------------------------------------------------------------- 1 | zlib;8;12;10;7.59 GB;5.79 GB;4.70 GB;1484210;1838822 2 | zlib;9;14;10;7.72 GB;6.10 GB;4.84 GB;3747358;4803461 3 | zlib;8;13;11;7.63 GB;5.88 GB;4.74 GB;2233948;2828250 4 | zlib;9;16;11;7.64 GB;5.90 GB;4.73 GB;2260736;2872193 5 | zlib;9;14;12;7.58 GB;5.76 GB;4.70 GB;1278332;1600020 6 | zlib;10;16;12;7.58 GB;5.74 GB;4.69 GB;1146752;1433830 7 | zlib;10;15;13;7.55 GB;5.65 GB;4.75 GB;660424;810918 8 | zlib;11;18;13;7.55 GB;5.64 GB;4.75 GB;584132;714738 9 | zlib;11;16;14;7.53 GB;5.59 GB;4.83 GB;339051;409257 10 | zlib;12;19;14;7.53 GB;5.58 GB;4.83 GB;300236;358912 11 | zlib;11;17;15;7.53 GB;5.55 GB;4.92 GB;183989;218587 12 | zlib;12;19;15;7.52 GB;5.55 GB;4.92 GB;170691;200130 13 | zlib;12;18;16;7.52 GB;5.52 GB;5.00 GB;95106;108475 14 | zlib;13;20;16;7.52 GB;5.52 GB;5.01 GB;88344;100396 15 | zlib;13;19;17;7.52 GB;5.51 GB;5.10 GB;47844;52960 16 | zlib;14;21;17;7.52 GB;5.51 GB;5.12 GB;44305;48837 17 | zlib;14;20;18;7.52 GB;5.50 GB;5.17 GB;24426;26550 18 | zlib;15;22;18;7.52 GB;5.50 GB;5.20 GB;22658;24541 19 | zlib;15;21;19;7.52 GB;5.50 GB;5.26 GB;12215;13024 20 | zlib;16;23;19;7.52 GB;5.50 GB;5.27 GB;11278;11996 21 | zlib;16;22;20;7.52 GB;5.49 GB;5.32 GB;6203;6477 22 | zlib;18;23;20;7.52 GB;5.49 GB;5.33 GB;5264;5466 23 | zlib;17;22;21;7.52 GB;5.49 GB;5.36 GB;3697;3817 24 | zlib;20;23;21;7.52 GB;5.49 GB;5.38 GB;2330;2386 25 | zlib;23;23;23;7.52 GB;5.49 GB;5.49 GB;899;899 26 | zlib;23;23;32;7.52 GB;5.49 GB;5.49 GB;899;899 27 | zlib;20;20;20;7.52 GB;5.49 GB;5.46 GB;7130;7183 28 | zlib;20;20;23;7.52 GB;5.49 GB;5.46 GB;7130;7183 29 | zlib;20;20;15;7.52 GB;5.49 GB;5.46 GB;7130;7183 30 | -------------------------------------------------------------------------------- /chunking/img/results_images_chunking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_images_chunking.png -------------------------------------------------------------------------------- /chunking/img/results_images_lz4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_images_lz4.png -------------------------------------------------------------------------------- /chunking/img/results_images_lzma.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_images_lzma.png -------------------------------------------------------------------------------- /chunking/img/results_images_zlib.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_images_zlib.png -------------------------------------------------------------------------------- /chunking/img/results_rootfs_chunking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_rootfs_chunking.png -------------------------------------------------------------------------------- /chunking/img/results_rootfs_lz4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_rootfs_lz4.png -------------------------------------------------------------------------------- /chunking/img/results_rootfs_zlib.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_rootfs_zlib.png -------------------------------------------------------------------------------- /chunking/img/results_vm_chunking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_vm_chunking.png -------------------------------------------------------------------------------- /chunking/img/results_vm_lz4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_vm_lz4.png -------------------------------------------------------------------------------- /chunking/img/results_vm_lzma.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_vm_lzma.png -------------------------------------------------------------------------------- /chunking/img/results_vm_zlib.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/img/results_vm_zlib.png -------------------------------------------------------------------------------- /chunking/results_images.ods: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/results_images.ods -------------------------------------------------------------------------------- /chunking/results_rootfs.ods: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/results_rootfs.ods -------------------------------------------------------------------------------- /chunking/results_vm.ods: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/chunking/results_vm.ods -------------------------------------------------------------------------------- /chunking/testdata.txt: -------------------------------------------------------------------------------- 1 | images: only jpg images 2 | rootfs: Linux root filesystem with binaries and config files 3 | vm: First ~8GB if an WindowsXP virtual machine image 4 | -------------------------------------------------------------------------------- /compression/img/results_rootfs_lzma.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/compression/img/results_rootfs_lzma.png -------------------------------------------------------------------------------- /compression/img/results_rootfs_zlib.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/compression/img/results_rootfs_zlib.png -------------------------------------------------------------------------------- /compression/results_rootfs.ods: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/n-st/borgbench/ace9751bba64648aea1f90c5f65644038d02f576/compression/results_rootfs.ods -------------------------------------------------------------------------------- /compression/testdata.txt: -------------------------------------------------------------------------------- 1 | Source: http://ftp.de.debian.org/debian-cd/8.5.0-live/amd64/iso-hybrid/ 2 | File: debian-live-8.5.0-amd64-standard.iso 3 | M5DSUM: 3605c5a42ca71c54b2d681abe3ef8d99 4 | Extracted: live/filesystem.squashfs 5 | du -bs: 999720429 debian-live-filesystem 6 | --------------------------------------------------------------------------------