├── LICENSE ├── README.md └── afl-ddmin-mod.py /LICENSE: -------------------------------------------------------------------------------- 1 | The accompanying files are under the following license (The ISC License): 2 | 3 | Copyright (c) 2016 Markus Teufelberger 4 | 5 | Permission to use, copy, modify, and distribute this software for any 6 | purpose with or without fee is hereby granted, provided that the above 7 | copyright notice and this permission notice appear in all copies. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 10 | WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 11 | MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 12 | ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 13 | WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 14 | ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 15 | OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 16 | 17 | 18 | Some code is based on ddmin (https://github.com/br0ns/ddmin) under the following license: 19 | 20 | The MIT License (MIT) 21 | 22 | Copyright (c) 2016 Morten Brøns-Pedersen 23 | 24 | Permission is hereby granted, free of charge, to any person obtaining a copy 25 | of this software and associated documentation files (the "Software"), to deal 26 | in the Software without restriction, including without limitation the rights 27 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 28 | copies of the Software, and to permit persons to whom the Software is 29 | furnished to do so, subject to the following conditions: 30 | 31 | The above copyright notice and this permission notice shall be included in all 32 | copies or substantial portions of the Software. 33 | 34 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 35 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 36 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 37 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 38 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 39 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 40 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Test case minimizer 2 | 3 | This is an implementation of a variation of the ddmin algorithm originally designed to generate 1-minimal failing test cases. 4 | 5 | It is described in: 6 | [_Simplifying Failure-Inducing Input_](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.6907&rep=rep1&type=pdf), Ralf Hildebrandt and Andreas Zeller, 2000. 7 | 8 | The program itself is designed to be used analogous to afl-tmin, with a few extra features and tunables available that are specific to the algorithm or potentially useful in general. 9 | 10 | # Requirements 11 | 12 | To run afl-ddmin-mod: 13 | 14 | * [Python 3.5+](https://www.python.org/) (afl-ddmin-mod is pure python with no external dependencies) 15 | 16 | * [AFL-fuzz](http://lcamtuf.coredump.cx/afl/) (the actual program used is afl-showmap, it needs to be callable by Python's subprocess.run()) 17 | 18 | To actually use the tool: 19 | 20 | * A test case to be minimized 21 | 22 | * An AFL instrumented binary to process the input 23 | 24 | # Usage 25 | 26 | $ ./afl-ddmin-mod.py --help 27 | usage: afl-ddmin-mod.py [ options ] -- /path/to/target_app [ ... ] 28 | 29 | Required parameters: 30 | -i file input test case to be shrunk by the tool 31 | -o file final output location for the minimized data 32 | 33 | Execution control settings: 34 | -t msec timeout for each run (none) 35 | -m megs memory limit for child process (50 MB) 36 | -Q use binary-only instrumentation (QEMU mode) 37 | 38 | Minimization settings: 39 | -e solve for edge coverage only, ignore hit counts 40 | -d int, --max-depth int 41 | limit the maximum recursion depth (none) 42 | -j int, --jitter int test splitting at additional offsets (0) 43 | -r, --restart-recursion 44 | restart the recursion after finding a smaller input 45 | file 46 | --threads int number of worker threads [0 = number of cores] (0) 47 | 48 | Optional arguments and parameters: 49 | -a dir, --all-tests-dir dir 50 | output directory for additional test cases that were 51 | discovered while minimizing 52 | -c dir, --crash-dir dir 53 | output directory for crashes that occurred while 54 | minimizing 55 | -h, --help show this help message and exit 56 | -V, --version show program's version number and exit 57 | 58 | For additional tips, please consult the README. 59 | 60 | The main goal is to produce a 1-minimal test case, meaning removing any single byte of the test case would alter its behaviour according to afl-showmap. 61 | Please note that this does NOT mean that the resulting test case is a global minimum (the smallest test case that can be generated by changing the input file), which is a much harder problem and might require domain knowledge. 62 | 63 | `-i, -o, -t, -m, -Q, -e`: 64 | These settings are directly passed to afl-showmap, please consult the (excellent) documentation there. 65 | 66 | `-d int, --max-depth int`: 67 | Ddmin will split files in smaller and smaller chunks until they are all 1 byte in size, then it will return the smallest test case it has found so far. 68 | Since this means that at the last stage there will be 2 runs of afl-showmap for every byte that's left, this can result in very long runtimes for large inputs. 69 | With this setting it is possible to limit ddmin-mod's recursion depth to whatever you like. 70 | If no chunks have been removed, the current chunk count is always 2 to the power of depth (depth 6 means splitting into 64 chunks for example). 71 | By default, this setting will be "none" (meaning there is no restriction on recursion depth). 72 | 73 | `-j int, --jitter int`: 74 | In its default implementation, ddmin does strict binary search by splitting the input file in 2 equal chunks. 75 | The problem with this approach is that not all file formats operate on a single byte boundary. 76 | Imagine a text heavy file format, where words have varying length that don't always align to the cuts that ddmin makes. 77 | To overcome this, the input either needs to be tokenized (not implemented, maybe afl-analyze might come in handy in the future), so ddmin operates on something potentially larger than single bytes, or more variations need to be tested. 78 | 79 | The "jitter" setting will split chunks in additional locations producing a list of several possible splits. 80 | E.g. [012345] gets traditionally split like this: [012] [345] 81 | With jitter 2 it will get split in the canonical location as well as with offests -1, +1, -2 and +2, resulting in the following splits: [012] [345], [01] [2345], [0123] [45], [0] [12345] and [01234] [5]. 82 | In case an offset is too large, no split will be applied (offset +10 on the [012345] file will just return [012345]). 83 | This is not a problem, since all smaller offsets are already considered anyways. 84 | Jitter will increase the runtime of the algorithm significantly (as 2 more splits per jitter level will be checked). 85 | To make sure the search tree doesn't completely explode, only the canonical split will be used to create the next level of chunks after each level of granularity has been checked. 86 | 87 | `-r, --restart-recursion`: 88 | If a chunk is removed, ddmin will not start at level 0 and split the newly found smaller test case in two, but rather keep on working at its current granularity level. 89 | If this switch is enabled, as soon as a smaller input is found, recursion is reset and the file is split from the largest possible set of chunks that describe this file. 90 | E.g. if testing [01] [23] [45] [67] [89] reveals that [23] is not relevant, ddmin would continue with [01] [45] [67] [89]. 91 | With restart-recursion it will first squash these tuples down to [01] [456789] and then re-start to split these again (e.g. to [0] [1] [456] [789]). 92 | This also significantly increases the runtime of the algorithm (as each removal will lead to a reset and new chunks being checked). 93 | 94 | `--threads int`: 95 | While ddmin is not really perfectly parallelizable (tests have to be run in a certain order and once a result has been found, the input changes), it is at least possible to have a not very wasteful way of running its tests next to each other. 96 | Ddmin-mod queues up all relevant tests, spins up one thread per core that the current machine reports (or more/fewer, depending on the setting here) and after each test checks if it was successful. 97 | If so, the queue will be emptied, all workers die after their current test is run and the analysis of the result that was found is done by the main algorithm again. 98 | This can lead to some race conditions if several threads find solutions at the same time. 99 | To stay 100% compatible to ddmin, there might be a way to re-run the canonical order of tests again, but the simpler route of just taking the smallest or earliest result was chosen for now. 100 | This might lead to a small variance in output and is turned on by default. 101 | If you need a fully deterministic way of reducing your test cases, make sure to set threads to 1. 102 | 103 | `-a dir, --all-tests-dir dir`: 104 | Afl-tmin usually runs a program a few thousand times with different inputs. 105 | Often these inputs create a different test case, however it will be not stored or reported because the main focus is to have a smaller test case that is identical to the initial one. 106 | Afl-ddmin-mod caches the hash of all resulting maps of all test cases it has ever run in one session, each of these represent a different test file. 107 | Additionally, this data is deduplicated and once a smaller test file is found for any hash it has already seen, this smaller test gets recorded instead. 108 | The only difference between these "accidential" test cases and the "input" test case is that afl-ddmin-mod won't look actively for smaller versions of these tests. 109 | Supplying a directory name via the -a setting will create this directory and store all test cases (named after the SHA256 hash of the map they created) in there. 110 | It is highly recommended to run afl-cmin or afl-pcmin on these test cases, as it is very likely that some show very similar behaviour or are a subset in functionality of other tests. 111 | While it might be possible to also extend afl-ddmin-mod towards this approach (parsing map files instead of hashing them and only returning relevant test cases), afl-(p)cmin already works well enough for that purpose. 112 | 113 | `-c dir, --crash-dir dir`: 114 | Similar to the description in -a, afl-ddmin-mod also stores information about crashes/timeouts that happen while testing. 115 | If a directory is supplied here, this setting will write all crashes and timeouts (defined as whenever afl-showmap returned with a code larger than 0) to this directory. 116 | Again, it is recommended to use a corpus minimization tool afterwards. 117 | 118 | # Tips 119 | 120 | Since this algorithm at least in my tests works a little bit better (creating smaller files) than afl-tmin, but also takes more time (if you play with -j and -r a LOT more time), multithreading and the ability to dump crashes and newly discovered tests was added, make sure to use it! 121 | This allows for an interesting "rinse and repeat" work cycle where you can break down several large initial test cases into more and more smaller ones by applying afl-ddmin-mod on a randomly selected test in a directory and then minimizing the resulting test cases including the remaining original ones. 122 | Afl-fuzz however prefers to have a small initial corpus as input, so keep this in mind before creating a few thousand different tiny sample files from a huge image library. 123 | 124 | If you absolutely need to have a file that is as small and as simple as possible, running afl-tmin and afl-ddmin-mod on each other's output several times (maybe with wider jitter or -r) can lead to new, smaller results. 125 | In general the files after even a single canonical run are already quite small unless there are very unfavourable splits. 126 | 127 | Afl-tmin does more modifications than just removing data, for example it also changes bytes around to reduce entropy. 128 | It is probably a good idea to also run it on your minimized test cases, either before or after afl-ddmin-mod to yield the benefits from doing that. 129 | 130 | # Caveats 131 | 132 | Afl-ddmin-mod uses a global dictionary as cache - while convenient, this can lead to high memory usage if a lot of different maps are being generated by subsets of your input file. 133 | Also if your input file is huge or you run a large amount of tests, you might run into memory exhaustion errors eventually. 134 | While there are certain points in the algorithm (for example after increasing granularity) where the cache could be reset, you probably are trying an input file that's too large for AFL to handle anyways. 135 | 136 | By default, all available cores on your machine are utilized. 137 | Make sure your hardware is cooled properly. 138 | 139 | Currently error handling or safety of your files was not yet the main focus in writing this - folders just get created and files are written in there, if they already exist or not. 140 | While it is functionally complete, I would recommend to first try this out in a VM or container. 141 | 142 | # License 143 | 144 | The program is under ISC license, which is a very permissive, MIT style free software license. 145 | Feel free to use it, however I personally would prefer if it is NOT used on critical infrastructure or in any military context. 146 | -------------------------------------------------------------------------------- /afl-ddmin-mod.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # Based on the ddmin algorithm by Andreas Zeller 4 | # published in https://www.st.cs.uni-saarland.de/papers/tse2002/tse2002.pdf 5 | # and inspired by the implementation by Morten Brøns-Pedersen 6 | # at https://github.com/br0ns/ddmin 7 | # 8 | # Further reading about the concept behind ddmin: 9 | # https://en.wikipedia.org/wiki/Delta_Debugging 10 | 11 | # stdlib imports 12 | import argparse # command line argument parsing 13 | import hashlib # sha256 hashing of maps 14 | import math # calculating stats 15 | import multiprocessing # CPU count 16 | import os # file size, removing temp files 17 | import queue # queue up work 18 | import subprocess # run external programs 19 | import sys # exit codes 20 | import tempfile # temporary input/output files 21 | import threading # multi threaded version 22 | import time # start/current time 23 | from typing import List, Tuple, Union # type hints 24 | 25 | __version__ = "0.1" 26 | __author__ = "Markus Teufelberger" 27 | 28 | # globals 29 | CACHE = {} 30 | BEST_MAP_HASH = "" 31 | TEST_CASES = {} 32 | TEST_CASES_LOCK = threading.Lock() 33 | DEPTH = 0 34 | TIMESTAMP = 0 35 | RUN_COUNT = 0 36 | TOTAL_RUNS = 0 37 | RESULTS = [] 38 | 39 | 40 | # From http://stackoverflow.com/questions/6517953/clear-all-items-from-the-queue: 41 | class Queue(queue.Queue): 42 | """ 43 | A custom queue subclass that provides a "clear" method. 44 | """ 45 | 46 | def clear(self): 47 | """ 48 | Clears all items from the queue. 49 | """ 50 | with self.mutex: 51 | unfinished = self.unfinished_tasks - len(self.queue) 52 | if unfinished <= 0: 53 | if unfinished < 0: 54 | raise ValueError('task_done() called too many times') 55 | self.all_tasks_done.notify_all() 56 | self.unfinished_tasks = unfinished 57 | self.queue.clear() 58 | self.not_full.notify_all() 59 | 60 | 61 | def chunksize(chunks: Tuple[Tuple[int, int]]) -> int: 62 | """ 63 | Calculate the total file size of a bunch of chunks. 64 | 65 | :param chunks: A tuple with (start, end,) offsets 66 | :return: the total length of the resulting file 67 | """ 68 | return sum(end - start for start, end in chunks) 69 | 70 | 71 | def write_file_from_chunks(chunks: Tuple[Tuple[int, int]], small_filename: str, 72 | origin_filename: str): 73 | """ 74 | Creates a new file from an existing file and chunks containing offsets. 75 | 76 | :param chunks: A tuple with (start, end,) offsets 77 | :param small_filename: Name/path of the new file 78 | :param origin_filename: Name/path of the original file 79 | """ 80 | with open(small_filename, "wb") as small_file: 81 | with open(origin_filename, "rb") as origin_file: 82 | for chunk in chunks: 83 | length = chunksize((chunk, )) 84 | # seek to first relevant byte 85 | origin_file.seek(chunk[0]) 86 | small_file.write(origin_file.read(length)) 87 | 88 | 89 | def normalize_chunks(chunks: Tuple[Tuple[int, int]]) -> Tuple[Tuple[int, int]]: 90 | """ 91 | Minimize the amount of chunks needed to describe a smaller portion of a file. 92 | 93 | :param chunks: A tuple with (start, end,) offsets 94 | :return: A tuple containing as few as possible (start, end,) offsets 95 | """ 96 | out = [] 97 | start1, end1 = chunks[0] 98 | if len(chunks) > 1: 99 | for start2, end2 in chunks[1:]: 100 | if start2 == end1: 101 | end1 = end2 102 | else: 103 | out.append((start1, end1)) 104 | start1, end1 = start2, end2 105 | out.append((start1, end1)) 106 | return tuple(out) 107 | 108 | 109 | def run_showmap( 110 | input_name: str, output_name: str, 111 | args) -> int: # TODO: type annotation for argparse.ArgumentParser 112 | """ 113 | Runs afl-showmap with the arguments specified. 114 | 115 | :param input_name: Name/path of the input file 116 | :param output_name: Name/path of the output file 117 | :param args: argparse.ArgumentParser that was created on startup 118 | :return: the return value of afl-showmap (0, 1, 2, or 3) 119 | """ 120 | # always run in quiet mode 121 | commandline = ["afl-showmap", "-o", output_name, "-q"] 122 | if args.timeout is not "none": 123 | commandline.append("-t") 124 | commandline.append(str(args.timeout)) 125 | if args.mem_limit is not 50: 126 | commandline.append("-m") 127 | commandline.append(str(args.mem_limit)) 128 | if args.qemu_mode is True: 129 | commandline.append("-Q") 130 | if args.edge_only is True: 131 | commandline.append("-e") 132 | 133 | requires_stdin = True 134 | for subarg in args.command: 135 | if "@@" in subarg: 136 | commandline.append(subarg.replace("@@", input_name)) 137 | requires_stdin = False 138 | else: 139 | commandline.append(subarg) 140 | # afl-showmap is very limited in regards to return codes: 141 | # 0: target ran fine 142 | # Should ask lcamtuf to change calculation for afl-showmap exit code to child_crashed * 3 + child_timed_out * 2 143 | # to allow to differentiate between hangs and incorrect args passed to afl-showmap 144 | # 1: target timed out or afl-showmap failed to run 145 | # 2: target crashed 146 | 147 | # This is by FAR the limiting factor execution time wise btw.! 148 | if requires_stdin: 149 | try: 150 | with open(input_name, "rb") as f: 151 | return subprocess.run(commandline, 152 | input=f.read(), 153 | stdout=subprocess.DEVNULL, 154 | stderr=subprocess.DEVNULL).returncode 155 | except IOError as ioe: 156 | print(ioe, file=sys.stderr) 157 | exit(1) 158 | else: 159 | return subprocess.run(commandline, 160 | stdout=subprocess.DEVNULL, 161 | stderr=subprocess.DEVNULL).returncode 162 | 163 | 164 | def get_map_hash( 165 | chunks: Tuple[Tuple[int, int]], 166 | args) -> str: # TODO: type annotation for argparse.ArgumentParser 167 | """ 168 | Get the SHA256 hash of the map file generated by afl-showmap. 169 | 170 | Stores the hash in the global CACHE and retrieves it from there 171 | on subsequent runs instead of re-calculating, if available. 172 | Typically, chunks are normalized before, to ensure less space 173 | wasted and no duplication. 174 | 175 | :param chunks: A tuple with (start, end,) offsets 176 | :param args: argparse.ArgumentParser that was created on startup 177 | :return: The SHA256 hash of the map file 178 | """ 179 | if chunks in CACHE: 180 | map_hash = CACHE[chunks]["map_hash"] 181 | # print("Cache hit!") 182 | else: 183 | global RUN_COUNT 184 | RUN_COUNT += 1 185 | global TOTAL_RUNS 186 | TOTAL_RUNS += 1 187 | # handle temporary files a bit more manually than simply using "with" 188 | tmpinput = tempfile.NamedTemporaryFile(delete=False) 189 | with open(args.input_file, "rb") as originfile: 190 | # assemble new input file with data from the original 191 | for chunk in chunks: 192 | length = chunksize((chunk, )) 193 | # seek to first relevant byte 194 | originfile.seek(chunk[0]) 195 | tmpinput.write(originfile.read(length)) 196 | tmpinput.close() 197 | tmpoutput = tempfile.NamedTemporaryFile(delete=False) 198 | tmpoutput.close() 199 | # both temp files are properly closed at this point 200 | ret_code = run_showmap(tmpinput.name, tmpoutput.name, args) 201 | with open(tmpoutput.name, "rb") as mapfile: 202 | map_hash = hashlib.sha256(mapfile.read()).hexdigest() 203 | 204 | # read the map: 205 | # 206 | # This might be interesting for the future, 207 | # but maps can get quite large in memory and are costly to compare... 208 | # 209 | # with open(tmpoutput.name, "r") as mapfile: 210 | # afl_map = [(int(element.split(":")[0]), int(element.split(":")[1].strip())) 211 | # for element in mapfile.readlines()] 212 | 213 | # remove temp files 214 | os.unlink(tmpinput.name) 215 | os.unlink(tmpoutput.name) 216 | 217 | if map_hash in TEST_CASES: 218 | with TEST_CASES_LOCK: 219 | # This is not thread safe otherwise 220 | if chunksize(chunks) < TEST_CASES[map_hash]["chunk_size"]: 221 | TEST_CASES[map_hash] = { 222 | "chunks": chunks, 223 | "chunk_size": chunksize(chunks), 224 | "returncode": ret_code 225 | } 226 | # print("Smaller test found!") 227 | else: 228 | TEST_CASES[map_hash] = { 229 | "chunks": chunks, 230 | "chunk_size": chunksize(chunks), 231 | "returncode": ret_code 232 | } 233 | # print("New test found!") 234 | # cache the map_hash: 235 | CACHE[chunks] = {"map_hash": map_hash, "returncode": ret_code} 236 | return map_hash 237 | 238 | 239 | def print_counters(chunks: Tuple[Tuple[int, int]]): 240 | """ 241 | Helper function to print out some statistics and diagnostics. 242 | 243 | :param chunks: A tuple with (start, end,) offsets 244 | """ 245 | print("Current depth: " + str(DEPTH) + "/" + str(math.ceil(math.log2( 246 | chunksize(chunks))))) 247 | print("Current chunk count: " + str(len(chunks))) 248 | print("Current size: " + str(chunksize(chunks))) 249 | print("# of unique maps: " + str(len(TEST_CASES))) 250 | global TIMESTAMP 251 | now = TIMESTAMP 252 | # reset timer 253 | TIMESTAMP = time.perf_counter() 254 | print("Seconds for this depth: " + str(TIMESTAMP - now)) 255 | print("Runs at this depth: " + str(RUN_COUNT)) 256 | print("Runs per second: " + str(RUN_COUNT / (TIMESTAMP - now))) 257 | print("Total runs: " + str(TOTAL_RUNS)) 258 | print("") 259 | 260 | 261 | def split_chunks(chunks: Tuple[Tuple[int, int]], jitter: 262 | int) -> List[Tuple[Tuple[int, int]]]: 263 | """ 264 | Given a tuple of chunks, creates all possible splits up to a certain offset. 265 | 266 | Chunks of length 1 are preserved, other splits that do not result in 2 chunks 267 | are discarded. 268 | 269 | TODO: Good place for doctests here. 270 | 271 | :param chunks: A tuple with (start, end,) offsets 272 | :param jitter: The maximum offset (+ and -) to consider 273 | :return: A deduplicated list of all resulting splits. 274 | """ 275 | print_counters(chunks) 276 | # cache can be reset at this point 277 | # global CACHE 278 | # CACHE = {} 279 | # reset run count 280 | global RUN_COUNT 281 | RUN_COUNT = 0 282 | # depth increases 283 | global DEPTH 284 | DEPTH += 1 285 | 286 | chunk_list = [] 287 | for variant in range(0, jitter + 1): 288 | newchunks_plus = [] 289 | newchunks_minus = [] 290 | for start, end in chunks: 291 | # preserve single byte chunks 292 | if (end - start) == 1: 293 | newchunks_plus.append((start, end)) 294 | continue 295 | # ddmin originally only has 0 jitter and does strict binary search 296 | # ddmin-mod adds jitter to the mix 297 | delta_plus = (end - start) // 2 + variant 298 | delta_minus = (end - start) // 2 - variant 299 | mid_plus = start + delta_plus 300 | mid_minus = start + delta_minus 301 | # depending on jitter, start can be equal to or smaller than mid! 302 | if start < mid_plus < end: 303 | newchunks_plus.append((start, mid_plus)) 304 | if start < mid_minus < end: 305 | newchunks_minus.append((start, mid_minus)) 306 | # depending on jitter, mid can be equal to or larger than end! 307 | if start < mid_plus < end: 308 | newchunks_plus.append((mid_plus, end)) 309 | if start < mid_minus < end: 310 | newchunks_minus.append((mid_minus, end)) 311 | if newchunks_plus: 312 | chunk_list.append(tuple(newchunks_plus)) 313 | if newchunks_minus: 314 | chunk_list.append(tuple(newchunks_minus)) 315 | # deduplicate while preserving order 316 | # See: http://stackoverflow.com/q/480214 317 | # this ensures the canonical solution with jitter = 0 is always the first list element 318 | seen = set() 319 | return [x for x in chunk_list if x not in seen and not seen.add(x)] 320 | 321 | 322 | def smaller_file( 323 | chunks: Tuple[Tuple[int, int]], fullmap_hash: str, 324 | args) -> bool: # TODO: type annotation for argparse.ArgumentParser 325 | """ 326 | Check if the file described by chunks is returning the same map as the original. 327 | 328 | :param chunks: A tuple with (start, end,) offsets 329 | :param fullmap_hash: The hash of the map of the original file 330 | :param args: argparse.ArgumentParser that was created on startup 331 | :return: True if "chunks" describes a file that results in the same map being created, 332 | False otherwise 333 | """ 334 | # compresses the tuples 335 | # slightly expensive, but saves overhead 336 | norm_chunks = normalize_chunks(chunks) 337 | 338 | afl_map_hash = get_map_hash(norm_chunks, args) 339 | # either: 340 | # afl_map is identical to fullmap (afl_map == fullmap) 341 | # --> return True 342 | 343 | # if afl_map == fullmap: 344 | # return True 345 | 346 | # or: 347 | # afl_map is a proper subset of fullmap (afl_map < fullmap) 348 | # --> return False, this is not an interesting test case 349 | 350 | # elif afl_map < fullmap: 351 | # return False 352 | 353 | # or: 354 | # afl_map is a superset of fullmap (afl_map >= fullmap) 355 | # --> return False, but this might be an interesting new test case! 356 | 357 | # elif afl_map >= fullmap: 358 | # return False 359 | 360 | # or: 361 | # afl_map differs from fullmap but does not hit all the same spots 362 | # --> return False, but this might be an interesting new test case! 363 | 364 | # else: 365 | # return False 366 | 367 | # since all these cases are currently not checked or can only be checked with actual maps, 368 | # not with hashes, it is enough to return this: 369 | return afl_map_hash == fullmap_hash 370 | 371 | 372 | def worker(work_queue, # TODO: type annotation for Queue 373 | args): # TODO: type annotation for argparse.ArgumentParser 374 | """ 375 | Picks a task from a queue and clears it, if successful. 376 | 377 | :param work_queue: a custom queue.Queue that supports clear() 378 | :param args: argparse.ArgumentParser that was created on startup 379 | """ 380 | while True: 381 | task = work_queue.get() 382 | if task is None: 383 | break 384 | if smaller_file(task, BEST_MAP_HASH, args): 385 | RESULTS.append(task) 386 | work_queue.task_done() 387 | work_queue.clear() 388 | else: 389 | work_queue.task_done() 390 | 391 | 392 | def crunch_tests(args, # TODO: type annotation for argparse.ArgumentParser 393 | chunk_list: List[Tuple[Tuple[int, int]]]): 394 | """ 395 | Distributes a list of candidate sub-files to worker threads. 396 | 397 | Exits after finding a result or after checking all subsets and complements. 398 | 399 | :param args: argparse.ArgumentParser that was created on startup 400 | :param chunk_list: a list of chunks describing various subsets of an input file 401 | """ 402 | # set up threads 403 | num_threads = args.threads 404 | if num_threads == 0: 405 | num_threads = multiprocessing.cpu_count() 406 | # start threads 407 | work_queue = Queue() 408 | threads = [] 409 | for _ in range(num_threads): 410 | thread = threading.Thread(target=worker, args=(work_queue, args, )) 411 | thread.start() 412 | threads.append(thread) 413 | 414 | # populate queue 415 | for chunks in chunk_list: 416 | # subsets: 417 | for chunk in chunks: 418 | chunk_ = (chunk, ) 419 | work_queue.put(chunk_) 420 | # complements: 421 | for index, _ in enumerate(chunks): 422 | chunk_ = chunks[:index] + chunks[index + 1:] 423 | work_queue.put(chunk_) 424 | 425 | # block until the queue is done 426 | # queue is cleared early, as soon as a worker discovers a result 427 | work_queue.join() 428 | 429 | # stop the threads 430 | for _ in range(num_threads): 431 | work_queue.put(None) 432 | for thread in threads: 433 | thread.join() 434 | 435 | 436 | def ddmin2_mod(chunk_list: List[Tuple[Tuple[int, int]]], 437 | depth: int, 438 | args, # TODO: type annotation for argparse.ArgumentParser 439 | testfilesize: int) -> List[Tuple[Tuple[int, int]]]: 440 | print("chunklist length: " + str(len(chunk_list))) 441 | # maximum depth reached? 442 | if args.max_depth != "none": 443 | if depth > args.max_depth: 444 | return chunk_list 445 | 446 | # main work going on in here! 447 | crunch_tests(args, chunk_list) 448 | 449 | global RESULTS 450 | 451 | # ensure all results are smaller than the starting size 452 | RESULTS = [c for c in RESULTS if chunksize(c) < testfilesize] 453 | 454 | if RESULTS: 455 | # get the best result 456 | best_size = testfilesize 457 | best_result = None 458 | for result_ in RESULTS: 459 | new_size = chunksize(result_) 460 | if new_size < best_size: 461 | best_result = result_ 462 | best_size = new_size 463 | # clear results 464 | RESULTS = [] 465 | # new smallest test case 466 | norm_chunk_ = normalize_chunks(best_result) 467 | global BEST_MAP_HASH 468 | BEST_MAP_HASH = CACHE[norm_chunk_]["map_hash"] 469 | # subset or complement? 470 | global DEPTH 471 | if len(best_result) == 1: 472 | # reduce to subset 473 | # only use the successful split 474 | print("SUBSET FOUND, new best size: " + str(chunksize( 475 | best_result))) 476 | if args.restart_recursion: 477 | DEPTH = math.log2(len(best_result)) 478 | return ddmin2_mod( 479 | split_chunks(norm_chunk_, args.jitter), 1, args, 480 | testfilesize) 481 | else: 482 | return ddmin2_mod( 483 | split_chunks(best_result, args.jitter), depth + 1, args, 484 | testfilesize) 485 | else: 486 | # reduce to complement 487 | # only use the successful split 488 | print("COMPLEMENT FOUND, new best size: " + str(chunksize( 489 | best_result))) 490 | if args.restart_recursion: 491 | DEPTH = math.log2(len(best_result)) 492 | return ddmin2_mod([norm_chunk_], depth, args, testfilesize) 493 | else: 494 | return ddmin2_mod([best_result], depth, args, testfilesize) 495 | else: 496 | # can we still split the file further? 497 | max_chunksize = 0 498 | for chunk_ in chunk_list[0]: 499 | if chunksize((chunk_, )) > max_chunksize: 500 | max_chunksize = chunksize((chunk_, )) 501 | if max_chunksize == 1: 502 | print("DONE") 503 | print_counters(chunk_list[0]) 504 | return chunk_list 505 | else: 506 | # neither subsets nor complements worked: increase the granularity and try again 507 | # !!! 508 | # Only use the canonical split, otherwise the search tree can get... difficult. 509 | # !!! 510 | # Wanna try it out? 511 | # Of course you want to! Here you go: 512 | # TODO: This is still buggy (a list of lists of chunks instead of a list of chunks) 513 | # smaller_chunks = [split_chunks(chunks, args.jitter) for chunks in chunk_list] 514 | smaller_chunks = split_chunks(chunk_list[0], args.jitter) 515 | print("SPLITTING CHUNKS, current best size: " + str(chunksize( 516 | chunk_list[0]))) 517 | return ddmin2_mod(smaller_chunks, depth + 1, args, testfilesize) 518 | 519 | 520 | def ddmin(args) -> List[Tuple[Tuple[ 521 | int, int]]]: # TODO: type annotation for argparse.ArgumentParser 522 | global TIMESTAMP 523 | TIMESTAMP = time.perf_counter() 524 | 525 | # TODO: better error handling 526 | try: 527 | testfilesize = os.path.getsize(args.input_file) 528 | except: 529 | print("Error while trying to get size of input file") 530 | raise 531 | chunks = ((0, testfilesize), ) 532 | afl_map_hash = get_map_hash(chunks, args) 533 | 534 | # This must be the currently smallest known test case as it is the only one 535 | global BEST_MAP_HASH 536 | BEST_MAP_HASH = afl_map_hash 537 | 538 | return ddmin2_mod(split_chunks(chunks, args.jitter), 1, args, testfilesize) 539 | 540 | 541 | def main() -> int: 542 | # TODO: Unit tests/Doctest etc. 543 | # TODO: Docstrings 544 | args = parse_argv() 545 | if vars(args)["command"][0] != "--": 546 | print("-- not found at the correct place! Please try again...") 547 | return -1 548 | 549 | # Run target once to check if everything works out 550 | with tempfile.NamedTemporaryFile() as tmpoutput: 551 | ret_code = run_showmap(args.input_file, tmpoutput.name, args) 552 | if os.path.getsize(tmpoutput.name) == 0: 553 | print("No map created by afl-showmap, aborting.") 554 | return -3 555 | if ret_code == 0: 556 | print("Target exits normally") 557 | elif ret_code == 1: 558 | print("Target crashes") 559 | elif ret_code == 2: 560 | print("Target times out") 561 | elif ret_code == 3: 562 | print("Target times out AND crashes") 563 | else: 564 | # afl-showmap shouldn't even be able to return this! 565 | print("Target does something weird/unknown") 566 | return -2 567 | 568 | # The main work happens here 569 | small_file_chunks_list = ddmin(args) 570 | 571 | # TODO: error handling 572 | # TODO: make sure nothing gets overwritten! 573 | # TODO: Collect/display statistics more regularly in time 574 | 575 | # write output file 576 | write_file_from_chunks( 577 | normalize_chunks(small_file_chunks_list[0]), args.output_file, 578 | args.input_file) 579 | 580 | if args.crash_dir: 581 | # write crashes 582 | os.makedirs(args.crash_dir, exist_ok=True) 583 | for testcase in TEST_CASES: 584 | if TEST_CASES[testcase]["returncode"] > 0: 585 | # filename: sha256 of the map that this file produces 586 | write_file_from_chunks(TEST_CASES[testcase]["chunks"], 587 | os.path.join(args.crash_dir, testcase), 588 | args.input_file) 589 | 590 | if args.all_tests_dir: 591 | # write test cases 592 | os.makedirs(args.all_tests_dir, exist_ok=True) 593 | for testcase in TEST_CASES: 594 | if TEST_CASES[testcase]["returncode"] == 0: 595 | # filename: sha256 of the map that this file produces 596 | write_file_from_chunks( 597 | TEST_CASES[testcase]["chunks"], 598 | os.path.join(args.all_tests_dir, testcase), 599 | args.input_file) 600 | return 0 601 | 602 | 603 | def int_or_none(string: str) -> Union[int, str]: 604 | """ 605 | Only allows a positive integer or a string containing "none". 606 | 607 | :param string: The parameter to be checked 608 | :return: The positive integer (incl. 0) or the string "none" 609 | """ 610 | if string == "none": 611 | return string 612 | try: 613 | value = int(string) 614 | if value <= 0: 615 | msg = "{number} is 0 or smaller".format(number=value) 616 | raise argparse.ArgumentTypeError(msg) 617 | return value 618 | except: 619 | msg = "{input} is not an integer".format(input=string) 620 | raise argparse.ArgumentTypeError(msg) 621 | 622 | 623 | def positive_int(string: str) -> int: 624 | """ 625 | Only allows an integer that is 0 or larger. 626 | 627 | :param string: The parameter to be checked 628 | :return: The positive integer 629 | """ 630 | try: 631 | value = int(string) 632 | if value < 0: 633 | msg = "{number} is smaller than 0".format(number=value) 634 | raise argparse.ArgumentTypeError(msg) 635 | return value 636 | except: 637 | msg = "{input} is not an integer".format(input=string) 638 | raise argparse.ArgumentTypeError(msg) 639 | 640 | 641 | def parse_argv(): # TODO: type annotation for argparse.ArgumentParser 642 | """ 643 | Parses the commandline arguments using argparse. 644 | 645 | :return: The argparse.ArgumentParser containing the parsed arguments. 646 | """ 647 | parser = argparse.ArgumentParser( 648 | usage="%(prog)s [ options ] -- /path/to/target_app [ ... ]", 649 | epilog="For additional tips, please consult the README.", 650 | add_help=False) # Otherwise "--help" is the first item displayed 651 | 652 | # https://bugs.python.org/issue9694 - :-( 653 | # Custom groups give nicer headings though: 654 | required = parser.add_argument_group("Required parameters") 655 | exec_control = parser.add_argument_group("Execution control settings") 656 | algo_settings = parser.add_argument_group("Minimization settings") 657 | optional = parser.add_argument_group("Optional arguments and parameters") 658 | 659 | # YAPF messes up this section a bit, so it is disabled for now: 660 | # yapf: disable 661 | 662 | # Input file path/name 663 | required.add_argument( 664 | "-i", 665 | metavar="file", 666 | required=True, 667 | dest="input_file", 668 | help="input test case to be shrunk by the tool") 669 | 670 | # Output file path/name (smaller version of input) 671 | required.add_argument( 672 | "-o", 673 | metavar="file", 674 | required=True, 675 | dest="output_file", 676 | help="final output location for the minimized data") 677 | 678 | # Timeout 679 | exec_control.add_argument( 680 | "-t", 681 | help="timeout for each run (none)", 682 | type=int_or_none, 683 | metavar="msec", 684 | dest="timeout", 685 | default="none") 686 | 687 | # Memory limit 688 | exec_control.add_argument( 689 | "-m", 690 | help="memory limit for child process (50 MB)", 691 | type=int_or_none, 692 | metavar="megs", 693 | dest="mem_limit", 694 | default="50") 695 | 696 | # QEMU mode 697 | exec_control.add_argument( 698 | "-Q", 699 | action="store_true", 700 | help="use binary-only instrumentation (QEMU mode)", 701 | dest="qemu_mode", 702 | default=False) 703 | 704 | # Edge coverage only 705 | algo_settings.add_argument( 706 | "-e", 707 | action="store_true", 708 | help="solve for edge coverage only, ignore hit counts", 709 | dest="edge_only", 710 | default=False) 711 | 712 | # Limit recursion depth 713 | algo_settings.add_argument( 714 | "-d", "--max-depth", 715 | type=int_or_none, 716 | metavar="int", 717 | help="limit the maximum recursion depth (none)", 718 | default="none") 719 | 720 | # Jitter when splitting chunks 721 | algo_settings.add_argument( 722 | "-j", "--jitter", 723 | type=positive_int, 724 | metavar="int", 725 | help="test splitting at additional offsets (0)", 726 | default="0") 727 | 728 | # Restart recursion after finding a smaller input 729 | algo_settings.add_argument( 730 | "-r", "--restart-recursion", 731 | action="store_true", 732 | help="restart the recursion after finding a smaller input file", 733 | default=False) 734 | 735 | # Path for additional test cases 736 | optional.add_argument( 737 | "-a", "--all-tests-dir", 738 | metavar="dir", 739 | help="output directory for additional test cases that were discovered while minimizing") 740 | 741 | # Path for additional crashes 742 | optional.add_argument( 743 | "-c", "--crash-dir", 744 | metavar="dir", 745 | help="output directory for crashes that occurred while minimizing") 746 | 747 | # Number of threads 748 | algo_settings.add_argument( 749 | "--threads", 750 | # TODO: This needs to be at least 1, not 0 751 | type=positive_int, 752 | metavar="int", 753 | help="number of worker threads [0 = number of cores] (0)", 754 | default="0") 755 | 756 | # Help 757 | optional.add_argument( 758 | "-h", "--help", 759 | action="help", 760 | help="show this help message and exit") 761 | 762 | # Version 763 | optional.add_argument( 764 | "-V", "--version", 765 | action="version", 766 | version="%(prog)s-{version}".format(version=__version__)) 767 | 768 | # Invoking the target app 769 | parser.add_argument( 770 | "command", 771 | nargs=argparse.REMAINDER, 772 | help=argparse.SUPPRESS) 773 | 774 | # yapf: enable 775 | 776 | return parser.parse_args() 777 | 778 | # run main() if called standalone 779 | if __name__ == "__main__": 780 | sys.exit(main()) 781 | --------------------------------------------------------------------------------