├── README.md ├── TFuzz ├── req.txt ├── scripts └── kernel_config.sh ├── target_programs ├── base64 ├── md5sum ├── uniq └── who └── tfuzz ├── .gitignore ├── __init__.py ├── cov.py ├── executor.py ├── ncc.py ├── qemu_runner.py ├── r2.py ├── tfuzz_fuzzer.py ├── tfuzz_sys.py ├── tprogram.py └── utils.py /README.md: -------------------------------------------------------------------------------- 1 | # T-Fuzz 2 | 3 | T-Fuzz consists of 2 components: 4 | - Fuzzing tool (TFuzz): a fuzzing tool based on program transformation 5 | - Crash Analyzer (CrashAnalyzer): a tool that verifies whether crashes found transformed 6 | programs are true bugs in the original program or not (coming soon). 7 | 8 | 9 | # OS support 10 | 11 | The current version is tested only on Ubuntu-16.04, while trying to run the code, 12 | please use our tested OS. 13 | 14 | # Prerequisite 15 | 16 | T-Fuzz system is built on several opensource tools. 17 | - [angr](https://github.com/angr/angr) 18 | - [shellphish fuzzer](https://github.com/shellphish/fuzzer) 19 | - [angr tracer](https://github.com/angr/tracer) 20 | - [radare2](https://github.com/radare/radare2) and its python 21 | wrapper [r2pipe](https://github.com/radare/radare2-r2pipe) 22 | 23 | ## Installing radare2 24 | 25 | ``` 26 | $ git clone https://github.com/radare/radare2.git 27 | $ cd radare2 28 | $ ./sys/install.sh 29 | ``` 30 | 31 | ## Installing python libraries 32 | 33 | ### installing some dependent libraries 34 | 35 | > Note: to use `apt-get build-dep`, you need to uncomment the deb-src lines in your apt source 36 | > file (/etc/apt/sources.list) and run apt-get update. 37 | 38 | ``` 39 | $ sudo apt-get install build-essential gcc-multilib libtool automake autoconf bison debootstrap debian-archive-keyring 40 | $ sudo apt-get build-dep qemu-system 41 | $ sudo apt-get install libacl1-dev 42 | ``` 43 | 44 | 45 | ### installing pip and setting up virtualenv & wrapper 46 | 47 | ``` 48 | $ sudo apt-get install python-pip python-virtualenv 49 | $ pip install virtualenvwrapper 50 | ``` 51 | 52 | Add the following lines to your shell rc file (`~/.bashrc` or `~/.zshrc`). 53 | 54 | ``` 55 | export WORKON_HOME=$HOME/.virtual_envs 56 | source /usr/local/bin/virtualenvwrapper.sh 57 | ``` 58 | 59 | ### Creating a python virtual environment 60 | 61 | ``` 62 | $ mkvirtualenv tfuzz-env 63 | ``` 64 | 65 | ### Installing dependent libraries 66 | 67 | This command will install all the dependent python libraries for you. 68 | 69 | ``` 70 | $ workon tfuzz-env 71 | $ pip install -r req.txt 72 | ``` 73 | 74 | # Fuzzing target programs with T-Fuzz 75 | 76 | ``` 77 | $ ./TFuzz --program --work_dir --target_opts 78 | ``` 79 | 80 | Where 81 | - : the path to the target program to fuzz 82 | - : the directory to save the results 83 | - : the options to pass to the target program, like AFL, use `@@` as 84 | placeholder for files to mutate. 85 | 86 | 87 | ## Examples 88 | 89 | 1. Fuzzing base64 with T-Fuzz 90 | 91 | ``` 92 | $ ./TFuzz --program target_programs/base64 --work_dir workdir_base64 --target_opts "-d @@" 93 | ``` 94 | 95 | 2. Fuzzing uniq with T-Fuzz 96 | 97 | ``` 98 | $ ./TFuzz --program target_programs/uniq --work_dir workdir_uniq --target_opts "@@" 99 | ``` 100 | 101 | 3. Fuzzing md5sum with T-Fuzz 102 | 103 | ``` 104 | $ ./TFuzz --program target_programs/md5sum --work_dir workdir_md5sum --target_opts "-c @@" 105 | ``` 106 | 107 | 4. Fuzzing who with T-Fuzz 108 | 109 | ``` 110 | $ ./TFuzz --program target_programs/who --work_dir workdir_who --target_opts "@@" 111 | ``` 112 | 113 | # Using CrashAnalyzer to verify crashes 114 | 115 | T-Fuzz CrashAnalyzer has been put in a docker image, however, 116 | it is still not working in all binaries we tested, we are still investigating 117 | it the cause. 118 | 119 | Here is how: 120 | 121 | Run the following command to run our docker image 122 | 123 | ``` 124 | $ [sudo] docker pull tfuzz/tfuzz-test 125 | $ [sudo] docker run --security-opt seccomp:unconfined -it tfuzz/tfuzz-test /usr/bin/zsh 126 | ``` 127 | 128 | In the container: 129 | 130 | There are 3 directories: 131 | - `release`: contains code the built lava binaries 132 | - `results`: contains some results we found in lava-m dataset 133 | - `radare2`: it is a program used by T-Fuzz. 134 | 135 | 136 | Currently, `T-Fuzz` may not work, because the tracer crashes accidentally. 137 | And the CrashAnalyzer can not work on all results. 138 | But some cases can be recovered. 139 | 140 | For example: 141 | 142 | 143 | To verify bugs in base64, first goto `release` and checkout ca_base64: 144 | 145 | ``` 146 | $ cd release 147 | $ git checkout ca_base64 148 | ``` 149 | 150 | Then we use a transformed program to recover the crash in the original program: 151 | 152 | 1. Choose a transformed program and run it on the input found by a fuzzer: 153 | 154 | ``` 155 | $ cd ~ 156 | $./results/ca_base64/554/base64_tfuzz_28/base64_tfuzz_28 -d ./results/ca_base64/554/crashing_inputs_from/results_saved_0_from 157 | [1] 131 segmentation fault (core dumped) ./results/ca_base64/554/base64_tfuzz_28/base64_tfuzz_28 -d 158 | ``` 159 | 160 | 161 | 2. Recover an input from this transformed program and crashing input 162 | 163 | ``` 164 | $ ./release/CrashAnalyzer --tprogram ./results/ca_base64/554/base64_tfuzz_28/base64_tfuzz_28 --target_opts "-d @@" --crash_input ./results/ca_base64/554/crashing_inputs_from/results_saved_0_from --result_dir base64_result --save_to recover 165 | WARNING | 2018-12-04 04:28:22,350 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups. 166 | Trying /root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from 167 | WARNING | 2018-12-04 04:28:23,228 | angr.project | Address is already hooked, during hook(0x9021cd0, ). Re-hooking. 168 | WARNING | 2018-12-04 04:28:23,228 | angr.project | Address is already hooked, during hook(0x90dd000, ). Re-hooking. 169 | WARNING | 2018-12-04 04:28:23,229 | angr.simos.linux | Tracer has been heavily tested only for CGC. If you find it buggy for Linux binaries, we are sorry! 170 | Adding = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 == 47))> 171 | Adding = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 == 47))> 172 | Adding = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 == 47))> 173 | Adding = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 == 47))> 174 | results saved to /root/base64_result/recover_0 175 | ``` 176 | 177 | Then `/root/base64_result/recover_0` is generated, we can use it to trigger a crash in the original program. 178 | 179 | 3. verify the input by running the generated input on the original program 180 | 181 | ``` 182 | $ ./results/base64 -d base64_result/recover_0 183 | Successfully triggered bug 554, crashing now! 184 | Successfully triggered bug 554, crashing now! 185 | Successfully triggered bug 554, crashing now! 186 | [1] 177 segmentation fault (core dumped) ./results/base64 -d base64_result/recover_0 187 | ``` -------------------------------------------------------------------------------- /TFuzz: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import os 4 | import sys 5 | import signal 6 | import argparse 7 | import logging 8 | 9 | from fuzzer import Fuzzer as __angr_Fuzzer 10 | 11 | from tfuzz.tfuzz_sys import TFuzzSys 12 | 13 | def main(): 14 | 15 | __angr_Fuzzer._perform_env_checks() 16 | 17 | parser = argparse.ArgumentParser(description='tfuzz user interface') 18 | parser.add_argument('--program', required=True, help='path to the program to fuzz') 19 | parser.add_argument('--work_dir', required=True, help='the work directory for tfuzz') 20 | parser.add_argument('--seed_dir', help='the work directory of initial seeds') 21 | parser.add_argument('--target_opts', help='The options to pass to the argument') 22 | parser.add_argument('-v', "--verbose", help='Increase the logging verbosity', 23 | action='store_true') 24 | parser.add_argument('--ph', default='@@', help='The place holder for input file to fuzz') 25 | 26 | args = parser.parse_args() 27 | 28 | if args.verbose: 29 | logging.getLogger('tfuzz').setLevel(logging.DEBUG) 30 | 31 | program = os.path.abspath(args.program) 32 | print(program) 33 | if not os.path.exists(program): 34 | print("%s does not exist" % (args.program)) 35 | sys.exit(-1) 36 | 37 | work_dir = os.path.abspath(args.work_dir) 38 | seeds = [] 39 | if args.seed_dir != None: 40 | seed_dir = os.path.abspath(args.seed_dir) 41 | if not os.path.exists(seed_dir): 42 | print("%s does not exist" % (args.seed_dir)) 43 | sys.exit(-1) 44 | for sf in os.listdir(seed_dir): 45 | seeds.append(os.path.join(seed_dir, sf)) 46 | 47 | if args.target_opts != None: 48 | argv = args.target_opts.split(' ') 49 | else: 50 | argv = None 51 | 52 | tfuzzsys = TFuzzSys(program, work_dir, target_opts=argv, 53 | input_placeholder=args.ph, seed_files=seeds) 54 | 55 | tfuzzsys.run() 56 | 57 | def ctrlc_handler(signal, frame): 58 | tfuzzsys.stop() 59 | sys.exit(0) 60 | 61 | signal.signal(signal.SIGINT, ctrlc_handler) 62 | 63 | if __name__ == '__main__': 64 | main() 65 | -------------------------------------------------------------------------------- /req.txt: -------------------------------------------------------------------------------- 1 | alabaster==0.7.10 2 | amqp==2.2.2 3 | ana==0.5 4 | angr==7.8.2.21 5 | archinfo==7.8.2.21 6 | asn1crypto==0.24.0 7 | Babel==2.6.0 8 | backports.shutil-get-terminal-size==1.0.0 9 | bcrypt==3.1.4 10 | billiard==3.5.0.3 11 | bintrees==2.0.7 12 | bitstring==3.1.5 13 | cachetools==2.0.1 14 | capstone==3.0.5rc2 15 | celery==4.1.0 16 | certifi==2018.1.18 17 | cffi==1.11.5 18 | chardet==3.0.4 19 | claripy==7.8.2.21 20 | cle==7.8.2.21 21 | cooldict==1.2 22 | cryptography==2.2.2 23 | decorator==4.3.0 24 | docutils==0.14 25 | dpkt-fix==1.7 26 | driller==1.0 27 | enum34==1.1.6 28 | future==0.16.0 29 | futures==3.2.0 30 | fuzzer==1.1 31 | idalink==0.11 32 | idna==2.6 33 | imagesize==1.0.0 34 | intervaltree==2.1.0 35 | ipaddress==1.0.19 36 | ipdb==0.11 37 | ipython==5.6.0 38 | ipython-genutils==0.2.0 39 | Jinja2==2.10 40 | kombu==4.1.0 41 | Mako==1.0.7 42 | MarkupSafe==1.0 43 | monkeyhex==1.6 44 | mulpyplexer==0.8 45 | networkx==2.1 46 | nose==1.3.7 47 | packaging==17.1 48 | paramiko==2.4.1 49 | pathlib2==2.3.0 50 | pefile==2017.11.5 51 | pexpect==4.4.0 52 | pickleshare==0.7.4 53 | pluggy==0.6.0 54 | plumbum==1.6.6 55 | progressbar==2.3 56 | prompt-toolkit==1.0.15 57 | psutil==5.4.3 58 | ptyprocess==0.5.2 59 | pwntools==3.12.0 60 | py==1.5.3 61 | pyasn1==0.4.2 62 | pycparser==2.18 63 | pyelftools==0.24 64 | pygit==0.1 65 | Pygments==2.2.0 66 | PyNaCl==1.2.1 67 | pypandoc==1.4 68 | pyparsing==2.2.0 69 | pyserial==3.4 70 | PySocks==1.6.8 71 | python-dateutil==2.7.2 72 | pytz==2018.4 73 | pyvex==7.8.2.21 74 | r2pipe==0.9.9 75 | redis==2.10.6 76 | requests==2.18.4 77 | ROPGadget==5.4 78 | rpyc==3.4.4 79 | scandir==1.7 80 | shellphish-afl==1.1 81 | shellphish-qemu==0.9.7 82 | simplegeneric==0.8.1 83 | six==1.11.0 84 | snowballstemmer==1.2.1 85 | sortedcontainers==1.5.9 86 | Sphinx==1.7.5 87 | sphinxcontrib-websupport==1.0.1 88 | subprocess32==3.2.7 89 | termcolor==1.1.0 90 | tox==3.0.0 91 | tqdm==4.20.0 92 | tracer==0.1 93 | traitlets==4.3.2 94 | typing==3.6.4 95 | unicorn==1.0.1 96 | urllib3==1.22 97 | vine==1.1.4 98 | virtualenv==15.2.0 99 | wcwidth==0.1.7 100 | z3-solver==4.5.1.0.post2 101 | -------------------------------------------------------------------------------- /scripts/kernel_config.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | 3 | echo 1 | sudo tee /proc/sys/kernel/sched_child_runs_first 4 | echo core | sudo tee /proc/sys/kernel/core_pattern 5 | cd /sys/devices/system/cpu; echo performance | sudo tee cpu*/cpufreq/scaling_governor 6 | -------------------------------------------------------------------------------- /target_programs/base64: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HexHive/T-Fuzz/7d150e493237db72c421d423f9a315401cb94e44/target_programs/base64 -------------------------------------------------------------------------------- /target_programs/md5sum: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HexHive/T-Fuzz/7d150e493237db72c421d423f9a315401cb94e44/target_programs/md5sum -------------------------------------------------------------------------------- /target_programs/uniq: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HexHive/T-Fuzz/7d150e493237db72c421d423f9a315401cb94e44/target_programs/uniq -------------------------------------------------------------------------------- /target_programs/who: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HexHive/T-Fuzz/7d150e493237db72c421d423f9a315401cb94e44/target_programs/who -------------------------------------------------------------------------------- /tfuzz/.gitignore: -------------------------------------------------------------------------------- 1 | # -*- mode: gitignore; -*- 2 | *~ 3 | \#*\# 4 | /.emacs.desktop 5 | /.emacs.desktop.lock 6 | *.elc 7 | auto-save-list 8 | tramp 9 | .\#* 10 | 11 | # Org-mode 12 | .org-id-locations 13 | *_archive 14 | 15 | # flymake-mode 16 | *_flymake.* 17 | 18 | # eshell files 19 | /eshell/history 20 | /eshell/lastdir 21 | 22 | # elpa packages 23 | /elpa/ 24 | 25 | # reftex files 26 | *.rel 27 | 28 | # AUCTeX auto folder 29 | /auto/ 30 | 31 | # cask packages 32 | .cask/ 33 | dist/ 34 | 35 | # Flycheck 36 | flycheck_*.el 37 | 38 | # server auth directory 39 | /server/ 40 | 41 | # projectiles files 42 | .projectile 43 | 44 | # directory configuration 45 | .dir-locals.el 46 | 47 | 48 | # Byte-compiled / optimized / DLL files 49 | __pycache__/ 50 | *.py[cod] 51 | *$py.class 52 | 53 | # C extensions 54 | *.so 55 | 56 | # Distribution / packaging 57 | .Python 58 | build/ 59 | develop-eggs/ 60 | dist/ 61 | downloads/ 62 | eggs/ 63 | .eggs/ 64 | lib/ 65 | lib64/ 66 | parts/ 67 | sdist/ 68 | var/ 69 | wheels/ 70 | *.egg-info/ 71 | .installed.cfg 72 | *.egg 73 | MANIFEST 74 | 75 | # PyInstaller 76 | # Usually these files are written by a python script from a template 77 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 78 | *.manifest 79 | *.spec 80 | 81 | # Installer logs 82 | pip-log.txt 83 | pip-delete-this-directory.txt 84 | 85 | # Unit test / coverage reports 86 | htmlcov/ 87 | .tox/ 88 | .coverage 89 | .coverage.* 90 | .cache 91 | nosetests.xml 92 | coverage.xml 93 | *.cover 94 | .hypothesis/ 95 | .pytest_cache/ 96 | 97 | # Translations 98 | *.mo 99 | *.pot 100 | 101 | # Django stuff: 102 | *.log 103 | local_settings.py 104 | db.sqlite3 105 | 106 | # Flask stuff: 107 | instance/ 108 | .webassets-cache 109 | 110 | # Scrapy stuff: 111 | .scrapy 112 | 113 | # Sphinx documentation 114 | docs/_build/ 115 | 116 | # PyBuilder 117 | target/ 118 | 119 | # Jupyter Notebook 120 | .ipynb_checkpoints 121 | 122 | # pyenv 123 | .python-version 124 | 125 | # celery beat schedule file 126 | celerybeat-schedule 127 | 128 | # SageMath parsed files 129 | *.sage.py 130 | 131 | # Environments 132 | .env 133 | .venv 134 | env/ 135 | venv/ 136 | ENV/ 137 | env.bak/ 138 | venv.bak/ 139 | 140 | # Spyder project settings 141 | .spyderproject 142 | .spyproject 143 | 144 | # Rope project settings 145 | .ropeproject 146 | 147 | # mkdocs documentation 148 | /site 149 | 150 | # mypy 151 | .mypy_cache/ -------------------------------------------------------------------------------- /tfuzz/__init__.py: -------------------------------------------------------------------------------- 1 | from . import cov 2 | from . import ncc 3 | from . import r2 4 | from . import tfuzz_fuzzer 5 | from . import tfuzz_sys 6 | from . import tprogram 7 | from . import qemu_runner 8 | from .executor import Executor 9 | -------------------------------------------------------------------------------- /tfuzz/cov.py: -------------------------------------------------------------------------------- 1 | import tracer 2 | import os 3 | from itertools import islice, izip 4 | import collections 5 | import logging 6 | import functools 7 | from . import qemu_runner 8 | from .utils import replace_input_placeholder 9 | 10 | logger = logging.getLogger("tfuzz.cov") 11 | 12 | class DynamicTrace(object): 13 | def __init__(self, binary, input_file, target_opts=None, 14 | input_placeholder='@@'): 15 | ''' 16 | A class used to collect 17 | coverage of program under an input 18 | 19 | args: 20 | binary: the path to the program 21 | input_file: path to the input file 22 | 23 | ''' 24 | self.binary = binary 25 | self.input_file = os.path.abspath(input_file) 26 | self.target_opts = target_opts 27 | self.input_placeholder = input_placeholder 28 | 29 | logger.debug("binary:%s", self.binary) 30 | logger.debug("input_file:%s", self.input_file) 31 | 32 | self.e_cov = collections.Counter([]) 33 | self.n_cov = collections.Counter([]) 34 | 35 | self._crash = False 36 | self._tmout = False 37 | 38 | self._collect_cov() 39 | 40 | 41 | def _collect_cov(self): 42 | if self.target_opts == None or self.input_placeholder not in self.target_opts: 43 | # the target program reads from stdin 44 | t = qemu_runner.QEMURunner(self.binary, input=file(self.input_file).read()) 45 | else: 46 | opts = replace_input_placeholder(self.target_opts, 47 | self.input_file, 48 | self.input_placeholder) 49 | t = qemu_runner.QEMURunner(self.binary, input='', argv=[self.binary] + opts) 50 | 51 | if t.crash_mode: 52 | self._crash = True 53 | if t.tmout: 54 | self._tmout = True 55 | 56 | nodes = t.trace 57 | edges = izip(nodes, islice(nodes, 1, None)) 58 | 59 | self.n_cov.update(collections.Counter(nodes)) 60 | self.e_cov.update(collections.Counter(edges)) 61 | 62 | def crash(self): 63 | return self._crash 64 | 65 | def timeout(self): 66 | return self._tmout 67 | 68 | def edges(self): 69 | return self.e_cov.viewkeys() 70 | 71 | def nodes(self): 72 | return self.n_cov.viewkeys() 73 | 74 | class AccCov(object): 75 | def __init__(self): 76 | self.acc_node_cov = collections.Counter([]) 77 | self.acc_edge_cov = collections.Counter([]) 78 | 79 | # this is for bookkeeping 80 | self.input_files = [] 81 | 82 | def add_trace(self, trace): 83 | 84 | self.input_files.append(trace.input_file) 85 | 86 | self.acc_edge_cov.update(trace.e_cov) 87 | self.acc_node_cov.update(trace.n_cov) 88 | 89 | def nodes(self): 90 | return self.acc_node_cov.viewkeys() 91 | 92 | def edges(self): 93 | return self.acc_edge_cov.viewkeys() 94 | -------------------------------------------------------------------------------- /tfuzz/executor.py: -------------------------------------------------------------------------------- 1 | import subprocess32 2 | import os 3 | 4 | import logging 5 | 6 | logger = logging.getLogger("tfuzz.executor") 7 | 8 | class Executor(object): 9 | 10 | def __init__(self, program, target_opts=None, timeout=10, record_stdout=False): 11 | ''' 12 | The class to run a target program natively. 13 | 14 | :ivar program: the path to the program to run 15 | :ivar target_opts: optional, a list variable containing all the options 16 | passed to the program (excuding the program, e.g., 17 | to run `ls -al`, target_opts should be ['-al']) 18 | :ivar timeout: an integer specifying the maximum time to 19 | run the target program in second 20 | :ivar record_stdout: bool, whether to record the stdout or not, if true, 21 | the output will be saved in `stdout` field 22 | ''' 23 | 24 | self.program = os.path.abspath(program) 25 | 26 | args = [self.program] 27 | if target_opts != None: 28 | args = args + target_opts 29 | 30 | self.args = args 31 | self.crash = False 32 | self.timeout = timeout 33 | self.tmout = False 34 | self.timeout = timeout 35 | self.record_stdout = record_stdout 36 | 37 | self._run() 38 | 39 | def _run(self): 40 | try: 41 | if self.record_stdout: 42 | self.stdout = subprocess32.check_output(self.args, timeout=self.timeout) 43 | else: 44 | subprocess32.check_output(self.args, timeout=self.timeout) 45 | except subprocess32.TimeoutExpired: 46 | self.tmout = True 47 | except subprocess32.CalledProcessError: 48 | self.crash = True 49 | -------------------------------------------------------------------------------- /tfuzz/ncc.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | import re 4 | import logging 5 | 6 | import r2pipe 7 | from intervaltree import Interval, IntervalTree 8 | 9 | import angr 10 | 11 | logger = logging.getLogger("tfuzz.ncc") 12 | 13 | class FuncBasedFilter(object): 14 | 15 | def __init__(self, program, include_funcs=None, exclude_funcs=None): 16 | self.map = IntervalTree() 17 | self.funcs = set() 18 | self.include_funcs = set(include_funcs) if \ 19 | include_funcs != None else set() 20 | self.exclude_funcs = set([ 21 | "__libc_csu_fini", 22 | "__afl_manual_init", 23 | "__x86.get_pc_thunk.bx", 24 | "atexit", 25 | "__afl_persistent_loop", 26 | "__sanitizer_cov_trace_pc_guard", 27 | "__afl_auto_init", 28 | "__libc_csu_init", 29 | "__sanitizer_cov_trace_pc_guard_init", 30 | "fadvise", 31 | "fdadvise", 32 | "_crcx", 33 | "_crcx_docrc", 34 | "longjmp", 35 | "setjmp", 36 | "random", 37 | "_terminate", 38 | "deallocate", 39 | "_start", 40 | "transmit", 41 | "fdwait", 42 | "receive", 43 | "allocate" 44 | ]) 45 | 46 | if exclude_funcs != None: 47 | self.exclude_funcs.update(exclude_funcs) 48 | 49 | self._build_addr_map(program) 50 | 51 | def _build_addr_map(self, program): 52 | r2 = r2pipe.open(program) 53 | symbols = r2.cmdj('isj') 54 | 55 | for s in symbols: 56 | if s['type'] != 'FUNC': 57 | continue 58 | 59 | if s['size'] == 0: 60 | continue 61 | 62 | if s['name'].startswith('imp'): 63 | continue 64 | 65 | if s['name'] in self.exclude_funcs: 66 | continue 67 | 68 | if s['name'] not in self.include_funcs: 69 | self.include_funcs.add(s['name']) 70 | 71 | self.map[s['vaddr']: s['vaddr'] + s['size']] = s['name'] 72 | self.funcs.add(s['name']) 73 | 74 | r2.quit() 75 | 76 | def _in_which_function(self, key): 77 | if len(self.map[key]) == 0: 78 | return None 79 | return list(self.map[key])[0].data 80 | 81 | def filter(self, from_addr, to_addr): 82 | from_func = self._in_which_function(from_addr) 83 | to_func = self._in_which_function(to_addr) 84 | return from_func == to_func and from_func in self.include_funcs 85 | 86 | class NCCDetector(object): 87 | 88 | def __init__(self, program_path, filters=None): 89 | self.program_path = program_path 90 | self.project = angr.Project(program_path, auto_load_libs=False) 91 | self.cfg = self.project.analyses.CFG() 92 | self.filters = [] if filters == None else filters 93 | 94 | def add_filter(self, filter): 95 | self.filters.append(filter) 96 | 97 | def detect_nccs(self, acc_cov): 98 | # TODO: optimize this algorithm 99 | for e in self.cfg.graph.edges(): 100 | from_node, to_node = e 101 | if (from_node.addr, to_node.addr) in acc_cov.edges() or \ 102 | from_node.addr not in acc_cov.nodes() or \ 103 | to_node.addr in acc_cov.nodes(): 104 | continue 105 | fout = False 106 | for f in self.filters: 107 | # if any of the filters returns false, we should discard it 108 | if not f.filter(from_node.addr, to_node.addr): 109 | logger.info("(%s, %s) filtered out" % \ 110 | (hex(from_node.addr), hex(to_node.addr))) 111 | fout = True 112 | break 113 | if fout: 114 | continue 115 | yield (from_node.addr, to_node.addr) 116 | -------------------------------------------------------------------------------- /tfuzz/qemu_runner.py: -------------------------------------------------------------------------------- 1 | import os 2 | import time 3 | import shutil 4 | import signal 5 | import socket 6 | import logging 7 | import resource 8 | import tempfile 9 | import re 10 | import subprocess32 11 | from contextlib import contextmanager 12 | 13 | from .r2 import Radare2 14 | 15 | l = logging.getLogger("tfuzz.qemu_runner") 16 | 17 | try: 18 | import shellphish_qemu 19 | except ImportError: 20 | raise ImportError("Unable to import shellphish_qemu, \ 21 | which is required by QEMURunner. Please install it\ 22 | before proceeding.") 23 | 24 | class RunnerEnvironmentError(Exception): 25 | pass 26 | 27 | def binary_type(binary): 28 | with file(binary) as f: 29 | f4 = f.read(4) 30 | if f4[1:] == "CGC": 31 | return "cgc" 32 | elif f4[1:] == "ELF": 33 | return "elf" 34 | 35 | return "Other" 36 | 37 | class QEMURunner(object): 38 | 39 | def __init__(self, binary=None, 40 | input=None, 41 | record_trace=True, 42 | record_stdout=False, 43 | record_magic=True, 44 | max_size = None, 45 | seed=None, 46 | memory_limit="8G", 47 | bitflip=False, 48 | report_bad_args=False, 49 | argv=None, 50 | trace_log_limit=2**30, 51 | trace_timeout=10): 52 | 53 | self.binary = os.path.abspath(binary) 54 | self.tmout = False 55 | self._record_magic = record_magic 56 | self._record_trace = record_trace 57 | self.input = input 58 | 59 | if isinstance(seed, (int, long)): 60 | seed = str(seed) 61 | self._seed = seed 62 | self._memory_limit = memory_limit 63 | self._bitflip = bitflip 64 | self._report_bad_args = report_bad_args 65 | self.argv = argv 66 | self.crash_mode=False 67 | self.crash_addr = None 68 | 69 | # validate seed 70 | if self._seed is not None: 71 | try: 72 | iseed = int(self._seed) 73 | if iseed > 4294967295 or iseed < 0: 74 | raise ValueError 75 | except ValueError: 76 | raise ValueError("The passed seed is either not an integer \ 77 | or is not between 0 and UINT_MAX") 78 | 79 | self.input_max_size = max_size or len(input) if input is not None else None 80 | self.trace_log_limit = trace_log_limit 81 | self.trace_timeout = trace_timeout 82 | 83 | self._trace_source_path = None 84 | 85 | self._setup() 86 | 87 | if record_stdout: 88 | tmp = tempfile.mktemp(prefix="stdout_" + os.path.basename(binary)) 89 | # will set crash_mode correctly 90 | self._run(stdout_file=tmp) 91 | with open(tmp, "rb") as f: 92 | self.stdout = f.read() 93 | os.remove(tmp) 94 | else: 95 | # will set crash_mode correctly 96 | self._run() 97 | 98 | def _setup(self): 99 | if not os.access(self.binary, os.X_OK): 100 | if os.path.isfile(self.binary): 101 | error_msg = "\"%s\" binary is not executable" % self.binary 102 | l.error(error_msg) 103 | raise RunnerEnvironmentError(error_msg) 104 | else: 105 | error_msg = "\"%s\" binary does not exist" % self.binary 106 | l.error(error_msg) 107 | raise RunnerEnvironmentError(error_msg) 108 | 109 | # try to find the install base 110 | self._check_qemu_install() 111 | 112 | def _check_qemu_install(self): 113 | """ 114 | Check the install location of QEMU. 115 | """ 116 | btype = binary_type(self.binary) 117 | if btype == "cgc": 118 | tname = "cgc-tracer" 119 | elif btype == "elf": 120 | self._record_magic = False 121 | r2 = Radare2(self.binary) 122 | if r2.arch == "x86" and r2.bits == 64: 123 | tname = "linux-x86_64" 124 | elif r2.arch == "x86" and r2.bits == 32: 125 | tname = "linux-i386" 126 | else: 127 | raise RunnerEnvironmentError("Binary type not supported") 128 | else: 129 | raise RunnerEnvironmentError("Binary type not supported") 130 | 131 | self._trace_source_path = shellphish_qemu.qemu_path(tname) 132 | if not os.access(self._trace_source_path, os.X_OK): 133 | if os.path.isfile(self._trace_source_path): 134 | error_msg = "%s is not executable" % self.trace_source 135 | l.error(error_msg) 136 | raise RunnerEnvironmentError(error_msg) 137 | else: 138 | error_msg = "\"%s\" does not exist" % self._trace_source_path 139 | l.error(error_msg) 140 | raise RunnerEnvironmentError(error_msg) 141 | 142 | def __get_rlimit_func(self): 143 | def set_fsize(): 144 | # here we limit the logsize 145 | resource.setrlimit(resource.RLIMIT_FSIZE, 146 | (self.trace_log_limit, self.trace_log_limit)) 147 | return set_fsize 148 | 149 | def _run(self, stdout_file=None): 150 | 151 | # import ipdb; ipdb.set_trace() 152 | 153 | logname = tempfile.mktemp(dir="/dev/shm/", prefix="tracer-log-") 154 | 155 | args = [self._trace_source_path] 156 | if self._seed is not None: 157 | args.append("-seed") 158 | args.append(str(self._seed)) 159 | 160 | # If the binary is CGC we'll also take this opportunity to read in the 161 | # magic page. 162 | if self._record_magic: 163 | mname = tempfile.mktemp(dir="/dev/shm/", prefix="tracer-magic-") 164 | args += ["-magicdump", mname] 165 | 166 | if self._record_trace: 167 | args += ["-d", "exec", "-D", logname] 168 | else: 169 | args += ["-enable_double_empty_exiting"] 170 | 171 | # Memory limit option is only available in shellphish-qemu-cgc-* 172 | if 'cgc' in self._trace_source_path: 173 | args += ["-m", self._memory_limit] 174 | 175 | args += self.argv or [self.binary] 176 | 177 | if self._bitflip: 178 | args = [args[0]] + ["-bitflip"] + args[1:] 179 | 180 | with open('/dev/null', 'wb') as devnull: 181 | stdout_f = devnull 182 | if stdout_file is not None: 183 | stdout_f = open(stdout_file, 'wb') 184 | 185 | p = None 186 | try: 187 | # we assume qemu with always exit and won't block 188 | if type(self.input) == str: 189 | l.debug("Tracing as raw input") 190 | l.debug(" ".join(args)) 191 | p = subprocess32.Popen(args, stdin=subprocess32.PIPE, 192 | stdout=stdout_f, stderr=devnull, 193 | preexec_fn=self.__get_rlimit_func()) 194 | 195 | _, _ = p.communicate(self.input, timeout=self.trace_timeout) 196 | else: 197 | l.debug("Tracing as pov file") 198 | in_s, out_s = socket.socketpair() 199 | p = subprocess32.Popen(args, stdin=in_s, stdout=stdout_f, 200 | stderr=devnull, 201 | preexec_fn=self.__get_rlimit_func()) 202 | 203 | for write in self.input.writes: 204 | out_s.send(write) 205 | time.sleep(.01) 206 | 207 | ret = p.wait(timeout=self.trace_timeout) 208 | 209 | # did a crash occur? 210 | if ret < 0: 211 | if abs(ret) == signal.SIGSEGV or abs(ret) == signal.SIGILL: 212 | l.info("Input caused a crash (signal %d) \ 213 | during dynamic tracing", abs(ret)) 214 | l.debug(repr(self.input)) 215 | l.debug("Crash mode is set") 216 | self.crash_mode = True 217 | 218 | except subprocess32.TimeoutExpired: 219 | if p != None: 220 | p.terminate() 221 | self.tmout = True 222 | 223 | self.returncode = p.returncode 224 | 225 | if stdout_file is not None: 226 | stdout_f.close() 227 | 228 | if self._record_trace: 229 | try: 230 | trace = open(logname).read() 231 | addrs = [] 232 | self.trace = addrs 233 | 234 | # Find where qemu loaded the binary. Primarily for PIE 235 | qemu_base_addr = int(trace.split("start_code")[1].split("\n")[0], 16) 236 | self.base_addr = qemu_base_addr 237 | 238 | prog = re.compile(r'Trace (.*) \[(?P.*)\].*') 239 | for t in trace.split('\n'): 240 | m = prog.match(t) 241 | if m != None: 242 | addr_str = m.group('addr') 243 | addrs.append(int(addr_str, base=16)) 244 | else: 245 | continue 246 | 247 | # grab the faulting address 248 | if self.crash_mode: 249 | self.crash_addr = int(trace.split('\n')[-2].split('[')[1].split(']')[0], 16) 250 | 251 | l.debug("Trace consists of %d basic blocks", len(self.trace)) 252 | except IndexError: 253 | l.warning("""One trace is found to be malformated, 254 | it is possible that the log file size exceeds the 1G limit, 255 | meaning that there might be infinite loops in the target program""") 256 | finally: 257 | os.remove(logname) 258 | 259 | if self._record_magic: 260 | try: 261 | self.magic = open(mname).read() 262 | a_mesg = "Magic content read from QEMU improper size, \ 263 | should be a page in length" 264 | assert len(self.magic) == 0x1000, a_mesg 265 | except IOError: 266 | pass 267 | finally: 268 | try: 269 | os.remove(mname) 270 | except OSError: 271 | pass 272 | -------------------------------------------------------------------------------- /tfuzz/r2.py: -------------------------------------------------------------------------------- 1 | import r2pipe 2 | from intervaltree import Interval, IntervalTree 3 | import argparse 4 | import capstone 5 | import archinfo 6 | import re 7 | from contextlib import contextmanager 8 | 9 | class Radare2(object): 10 | 11 | def __init__(self, program, flags=None): 12 | self.program = program 13 | 14 | # if flags == None: 15 | # flags = ['-w'] 16 | 17 | # if '-w' not in flags: 18 | # flags.append('-w') 19 | 20 | if flags != None and isinstance(flags, list): 21 | self.r2 = r2pipe.open(self.program, flags=flags) 22 | else: 23 | self.r2 = r2pipe.open(self.program) 24 | # self.r2.cmd("aa") 25 | 26 | i_json = self.r2.cmdj('ij') 27 | self.os = i_json['bin']['os'] 28 | self.arch = i_json['bin']['arch'] 29 | self.bits = i_json['bin']['bits'] 30 | self.pic = i_json['bin']['pic'] 31 | 32 | 33 | if self.arch == 'x86': 34 | if self.bits == '64': 35 | self.archinfo = archinfo.ArchAMD64 36 | else: 37 | self.archinfo = archinfo.ArchX86 38 | else: 39 | self.archinfo = None 40 | 41 | if self.archinfo != None: 42 | self.md = capstone.Cs(self.archinfo.cs_arch, self.archinfo.cs_mode) 43 | else: 44 | self.md = None 45 | 46 | def __getitem__(self, key): 47 | self.r2.cmd('s ' + hex(key)) 48 | ret = self.r2.cmd('p8 1') 49 | try: 50 | ret = int(ret, base=16) 51 | except ValueError: 52 | ret = None 53 | 54 | return ret 55 | 56 | def __setitem__(self, key, val): 57 | val = val & 0xFF 58 | val = "{0:0x}".format(val) 59 | self.r2.cmd('s ' + hex(key)) 60 | self.r2.cmd('wx ' + val) 61 | 62 | def get_bytes_n(self, addr, n): 63 | ''' 64 | This function returns an array of `n` bytes 65 | ''' 66 | self.r2.cmd('s ' + hex(addr)) 67 | 68 | return self.r2.cmdj('pcj ' + str(n)) 69 | 70 | def get_cjump_addr(self, blk_addr): 71 | code_byte_array = self.get_bytes_n(blk_addr, 1024) 72 | code_char_array = [chr(b) for b in code_byte_array] 73 | code_str = ''.join(code_char_array) 74 | gen = self.md.disasm(code_str, blk_addr) 75 | 76 | for i in gen: 77 | if i.mnemonic.startswith('j') and i.mnemonic != 'jmp': 78 | # print("Found a jump instruction at %s(%d): %s"%(hex(i.address), i.size, 79 | # i.mnemonic + ' ' + i.op_str)) 80 | return i.address 81 | 82 | print("Conditional jump instruction not found") 83 | return 0 84 | 85 | def negate_cjmp(self, cjump_inst_addr): 86 | ''' 87 | Only X86/X86_64 are supported now 88 | ''' 89 | 90 | if self.md == None: 91 | raise NotImplementedError 92 | 93 | # http://unixwiz.net/techtips/x86-jumps.html 94 | x86_jmp_pairs ={ 95 | 'jo':'jno', 96 | 'js': 'jns', 97 | 'je': 'jne', 98 | 'jz': 'jnz', 99 | 'jb': 'jnb', 100 | 'jae': 'jnae', 101 | 'jc': 'jnc', 102 | 'ja': 'jna', 103 | 'jbe': 'jnbe', 104 | 'jl': 'jnl', 105 | 'jge': 'jnge', 106 | 'jg': 'jng', 107 | 'jle': 'jnle', 108 | 'jp': 'jnp', 109 | 'jpe': 'jpo', 110 | 'jcxz': 'jecxz' 111 | } 112 | 113 | # add the original map 114 | x86_jmp_map = x86_jmp_pairs.copy() 115 | 116 | # add the reverse map 117 | for ji in x86_jmp_pairs.keys(): 118 | x86_jmp_map[x86_jmp_pairs[ji]] = ji 119 | 120 | code_byte_array = self.get_bytes_n(cjump_inst_addr, 1024) 121 | code_char_array = [chr(b) for b in code_byte_array] 122 | code_str = ''.join(code_char_array) 123 | gen = self.md.disasm(code_str, cjump_inst_addr) 124 | 125 | i = gen.next() 126 | 127 | # we use this heuristic to determine it is a jump instruction 128 | if i.mnemonic not in x86_jmp_map: 129 | print("It is not a conditional jump instruction at @%s:%s" % 130 | (hex(cjump_inst_addr), i.mnemonic + ' ' + i.op_str)) 131 | return 132 | 133 | self.r2.cmd('s ' + str(i.address)) 134 | 135 | # then negate the conditional jump instruction 136 | self.r2.cmd('wa ' + x86_jmp_map[i.mnemonic] + ' ' + i.op_str) 137 | return i.address 138 | 139 | def close(self): 140 | try: 141 | self.r2.quit() 142 | except: 143 | pass 144 | 145 | def __del__(self): 146 | self.close() 147 | 148 | 149 | @contextmanager 150 | def closing_r2(r2): 151 | try: 152 | yield r2 153 | finally: 154 | r2.close() 155 | 156 | -------------------------------------------------------------------------------- /tfuzz/tfuzz_fuzzer.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import shutil 3 | import threading 4 | import time 5 | import os 6 | import re 7 | 8 | from fuzzer import Fuzzer as SFFuzzer 9 | import angr 10 | 11 | from .tprogram import TProgram 12 | from .executor import Executor 13 | from .utils import replace_input_placeholder 14 | 15 | logger = logging.getLogger("tfuzz.tfuzz_fuzzer") 16 | 17 | class Fuzzer(object): 18 | ''' 19 | Wrapper class if shellphish fuzzer, which is, in turn, 20 | a python wrapper for AFL. 21 | ''' 22 | 23 | def __init__(self, tprogram, seed_files, workdir, target_opts=None, 24 | input_placeholder='@@', afl_opts=None): 25 | self.tprogram = tprogram 26 | self.target_opts = target_opts 27 | self.workdir = workdir 28 | self.stat_file = os.path.join(self.workdir, 29 | tprogram.program_name, 'stat') 30 | self.stat = {} 31 | 32 | self.crashing_inputs = [] 33 | self.tmout_inputs = [] 34 | 35 | if seed_files == None: 36 | seed_files = self.__prepare_predefined_seeds() 37 | 38 | seeds = [] 39 | logger.debug("Starting to classify the seeds for %s", tprogram.program_name) 40 | for sf in seed_files: 41 | logger.debug("Trying running %s", sf) 42 | ret = self.__get_exec_result_on_input(sf) 43 | if ret == 2: 44 | # the seed results in crash 45 | self.crashing_inputs.append(sf) 46 | elif ret == 1: 47 | # the seed results in timeout 48 | self.tmout_inputs.append(sf) 49 | else: 50 | seeds.append(file(sf).read()) 51 | logger.debug("Classifying the seeds for %s ended", tprogram.program_name) 52 | 53 | use_qemu = False 54 | if tprogram.is_cgc(): 55 | use_qemu = True 56 | 57 | logger.debug("fuzzing %s with opts: %s", self.tprogram.program_path, 58 | str(target_opts)) 59 | 60 | self._fuzzer = SFFuzzer(self.tprogram.program_path, self.workdir, 61 | seeds=seeds, qemu=use_qemu, 62 | create_dictionary=False, 63 | target_opts=target_opts, extra_opts=afl_opts) 64 | 65 | # save the seed files resulting in crashes and timeout 66 | self.save_crash_and_tmout_inputs() 67 | 68 | def __prepare_predefined_seeds(self): 69 | _pre_seeds_dir = os.path.join(self.workdir, '_pre_seeds') 70 | _pre_seed_file = os.path.join(_pre_seeds_dir, 'seed-0') 71 | if not os.path.exists(_pre_seeds_dir): 72 | os.makedirs(_pre_seeds_dir) 73 | 74 | with open(_pre_seed_file, 'w') as f: 75 | f.write('fuzz') 76 | 77 | return [_pre_seed_file] 78 | 79 | def __str__(self): 80 | return "" % \ 81 | (self.tprogram, self.workdir) 82 | 83 | def __repr__(self): 84 | return self.__str__() 85 | 86 | def start(self): 87 | self.stat['status'] = 'running' 88 | self._fuzzer.start() 89 | 90 | def stop(self): 91 | self._fuzzer.kill() 92 | self.stat['status'] = 'stopped' 93 | self.write_stat() 94 | 95 | def __find_generated_files(self, afl_instance, subdir): 96 | if not os.path.exists(os.path.join(self._fuzzer.out_dir, 97 | afl_instance, subdir)): 98 | raise ValueError("subdir: '%s' in afl instance '%s' does not exist"\ 99 | % (subdir, afl_instance)) 100 | 101 | subdir = os.path.join(self._fuzzer.out_dir, afl_instance, subdir) 102 | generated_files = filter(lambda x: x.startswith('id:'), 103 | os.listdir(subdir)) 104 | return map(lambda x: os.path.join(subdir, x), generated_files) 105 | 106 | def generated_inputs(self, afl_instance='fuzzer-master'): 107 | return self.__find_generated_files(afl_instance, 'queue') 108 | 109 | def crashes_found(self, afl_instance='fuzzer-master'): 110 | return self.__find_generated_files(afl_instance, 'crashes') 111 | 112 | def crash_seeds(self): 113 | cs_dir = os.path.join(self._fuzzer.job_dir, 'crashing_seeds') 114 | cs_files = filter(lambda x: x.startswith('crash_seed_'), 115 | os.listdir(cs_dir)) 116 | 117 | return map(lambda x: os.path.join(cs_dir, x), cs_files) 118 | 119 | 120 | def timeout_seeds(self): 121 | timeout_seed_dir = os.path.join(self._fuzzer.job_dir, 'tmout_seeds') 122 | timeout_seed_files = filter(lambda x: x.startswith('tmout_seed_'), 123 | os.listdir(cs_dir)) 124 | 125 | return map(lambda x: os.path.join(cs_dir, x), cs_files) 126 | 127 | def failed_to_start(self): 128 | afl_log_file = os.path.join(self._fuzzer.job_dir, 'fuzzer-master.log') 129 | if not os.path.exists(afl_log_file): 130 | return True 131 | 132 | ansi_escape = re.compile(r'\x1b[^m]*m') 133 | 134 | with open(afl_log_file, 'r') as f: 135 | for l in f.readlines(): 136 | l = l.strip() 137 | ansi_escape.sub('', l) 138 | if "[-] PROGRAM ABORT :" in l: 139 | return True 140 | 141 | return False 142 | 143 | 144 | def __get_exec_result_on_input(self, input_file): 145 | args = replace_input_placeholder(self.target_opts, input_file) 146 | 147 | # here we hard-code the max time 148 | # to run a target program by 1 sec 149 | executor = Executor(self.tprogram.program_path, 150 | target_opts=args, timeout=1) 151 | 152 | if executor.tmout: 153 | return 1 154 | if executor.crash: 155 | return 2 156 | 157 | return 0 158 | 159 | def save_crash_and_tmout_inputs(self, additional_crash_input_files=None, 160 | additional_tmout_input_files=None): 161 | ci_dir = os.path.join(self._fuzzer.job_dir, 'crashing_seeds') 162 | tmout_dir = os.path.join(self._fuzzer.job_dir, 'tmout_seeds') 163 | if not os.path.exists(ci_dir): 164 | os.makedirs(ci_dir) 165 | if not os.path.exists(tmout_dir): 166 | os.makedirs(tmout_dir) 167 | 168 | if additional_crash_input_files != None: 169 | for c in additional_crash_input_files: 170 | self.crashing_inputs.append(c) 171 | 172 | if additional_tmout_input_files != None: 173 | for c in additional_tmout_input_files: 174 | self.tmout_inputs.append(c) 175 | 176 | idx = 0 177 | for i in self.crashing_inputs: 178 | shutil.copyfile(i, os.path.join(ci_dir, 179 | 'crash_seed_' + str(idx))) 180 | idx = idx + 1 181 | 182 | idx = 0 183 | for i in self.tmout_inputs: 184 | shutil.copyfile(i, os.path.join(tmout_dir, 185 | 'tmout_seed_' + str(idx))) 186 | idx = idx + 1 187 | 188 | def resuming(self): 189 | return self._fuzzer.resuming 190 | 191 | def is_stuck(self): 192 | try: 193 | pending_favs = self._fuzzer.stats['fuzzer-master']['pending_favs'] 194 | except: 195 | return False 196 | 197 | return int(pending_favs) == 0 198 | 199 | def write_stat(self): 200 | if not os.path.exists(os.path.dirname(self.stat_file)): 201 | os.makedirs(os.path.dirname(self.stat_file)) 202 | 203 | with open(self.stat_file, 'w') as f: 204 | for key, val in self.stat.items(): 205 | f.write("%s:%s\n" % (key, val)) 206 | 207 | def __del__(self): 208 | self.write_stat() 209 | -------------------------------------------------------------------------------- /tfuzz/tfuzz_sys.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import threading 4 | import logging 5 | import shutil 6 | import time 7 | 8 | logger = logging.getLogger("tfuzz.tfuzz_sys") 9 | 10 | from .cov import AccCov 11 | from .ncc import NCCDetector 12 | from .ncc import FuncBasedFilter 13 | from .r2 import Radare2 14 | from .cov import DynamicTrace 15 | from .tfuzz_fuzzer import Fuzzer 16 | from .tprogram import TProgram 17 | from .utils import create_dict 18 | 19 | class TFuzzSys(): 20 | 21 | def __init__(self, binary_path, workdir, afl_count=1, target_opts=None, 22 | seed_files=None, rand_seed=False, check_interval=30, 23 | input_placeholder='@@', afl_opts=None): 24 | 25 | assert os.path.exists(binary_path), "% does not exist" % binary_path 26 | 27 | self.origin_binary_path = binary_path 28 | self.origin_binary_name = os.path.basename(binary_path) 29 | self.workdir = os.path.abspath(workdir) 30 | self.afl_count = afl_count 31 | self.target_opts = target_opts 32 | self.check_interval = check_interval 33 | self.afl_opts = afl_opts if afl_opts is not None else [] 34 | self.input_placeholder = input_placeholder 35 | self.seed_files = seed_files 36 | self.rand_seed = rand_seed 37 | 38 | # TODO: Add resuming feature 39 | if os.path.exists(workdir): 40 | print("%s already exist" % (workdir)) 41 | raise Exception() 42 | 43 | # preparing seeds 44 | self.init_seed_files = [] 45 | self.init_seed_dir = os.path.join(self.workdir, 'init_seeds') 46 | self.__prepare_init_seeds() 47 | 48 | self.__queue = [] 49 | 50 | # Preparing the first TProgram 51 | original_tprogram_dir = os.path.join(self.workdir, 52 | self.origin_binary_name + '_tfuzz') 53 | original_tprogram_path = os.path.join(original_tprogram_dir, 54 | self.origin_binary_name + '_tfuzz') 55 | 56 | os.makedirs(original_tprogram_dir) 57 | shutil.copyfile(binary_path, original_tprogram_path) 58 | os.chmod(original_tprogram_path, 0777) 59 | 60 | self.dict_file = os.path.join(self.workdir, 61 | self.origin_binary_name + '.dict') 62 | create_dict(self.origin_binary_path, self.dict_file) 63 | if '-x' not in self.afl_opts: 64 | self.afl_opts = ['-x', self.dict_file] + self.afl_opts 65 | 66 | self.original_tprogram = TProgram(original_tprogram_path) 67 | self.__queue.append(self.original_tprogram) 68 | 69 | self.__current_fuzzer = None 70 | 71 | self.__stop = False 72 | 73 | ff = FuncBasedFilter(self.origin_binary_path, include_funcs=['main']) 74 | self.nccdetector = NCCDetector(self.origin_binary_path, filters=[ff]) 75 | 76 | def __prepare_init_seeds(self): 77 | try: 78 | os.makedirs(self.init_seed_dir) 79 | except: 80 | pass 81 | 82 | if self.seed_files != None and len(self.seed_files) != 0: 83 | for sf in self.seed_files: 84 | shutil.copy(sf, self.init_seed_dir) 85 | self.init_seed_files.append(os.path.join(self.init_seed_dir, os.path.basename(sf))) 86 | return 87 | 88 | # no seed files are provided 89 | gen_init_seed_file = os.path.join(self.init_seed_dir, 'seed-0') 90 | 91 | with open(gen_init_seed_file, 'w') as f: 92 | if self.rand_seed: 93 | f.write(os.urandom(32)) 94 | else: 95 | f.write("fuzz") 96 | 97 | self.init_seed_files.append(gen_init_seed_file) 98 | 99 | def __choose_program(self): 100 | if len(self.__queue) == 0: 101 | return None 102 | 103 | return self.__queue.pop(0) 104 | 105 | def __queue_empty(self): 106 | return len(self.__queue) == 0 107 | 108 | def __add_to_queue(self, tprogram): 109 | self.__queue.append(tprogram) 110 | 111 | def __prepare_fuzzing_workdir(self, tprogram): 112 | tprogram_fuzzing_workdir = os.path.join(self.workdir, 'fuzzing_' + tprogram.program_name) 113 | if not os.path.exists(tprogram_fuzzing_workdir): 114 | os.makedirs(tprogram_fuzzing_workdir) 115 | 116 | return tprogram_fuzzing_workdir 117 | 118 | def __clean_tprogram(self, tprogram): 119 | dname = os.path.dirname(tprogram.program_path) 120 | try: 121 | shutil.rmtree(dname) 122 | except: 123 | pass 124 | 125 | def __fuzz_one_program(self): 126 | 127 | fuzzing_workdir = self.__prepare_fuzzing_workdir(self.fuzzing_program) 128 | 129 | if self.fuzzing_program == self.original_tprogram: 130 | init_seeds = self.init_seed_files 131 | else: 132 | init_seeds = self.fuzzing_program.inputs_from_fuzzing_parent 133 | 134 | self.__current_fuzzer = Fuzzer(self.fuzzing_program, init_seeds, 135 | fuzzing_workdir, target_opts=self.target_opts, 136 | input_placeholder=self.input_placeholder, 137 | afl_opts=self.afl_opts) 138 | 139 | self.__current_fuzzer.start() 140 | 141 | time.sleep(2) 142 | 143 | if self.__current_fuzzer.failed_to_start(): 144 | self.__current_fuzzer.stat['status'] = "failed to start" 145 | return False 146 | 147 | return True 148 | 149 | 150 | def run(self): 151 | 152 | while not self.__stop: 153 | 154 | program = self.__choose_program() 155 | if program == None: 156 | logger.warn("No program left") 157 | sys.exit() 158 | 159 | self.fuzzing_program = program 160 | if not self.__fuzz_one_program(): 161 | logger.warn("%s failed to start, \ 162 | skip, but there still might be some \ 163 | crashes in crashing seeds", self.fuzzing_program) 164 | continue 165 | 166 | logger.debug("Fuzzing %s started", (self.fuzzing_program.program_name)) 167 | while not self.__current_fuzzer.is_stuck(): 168 | time.sleep(self.check_interval) 169 | 170 | 171 | logger.debug("Fuzzer got stuck") 172 | crash_seeds = len(self.__current_fuzzer.crash_seeds()) 173 | crash_found = len(self.__current_fuzzer.crashes_found()) 174 | if crash_seeds > 0 or crash_found > 0: 175 | logger.debug("Crashes found while fuzzing %s", self.fuzzing_program.program_name) 176 | 177 | self.__current_fuzzer.stat['crash_found'] = crash_found 178 | self.__current_fuzzer.stat['crash_seeds'] = crash_seeds 179 | self.__current_fuzzer.stop() 180 | self.__current_fuzzer.write_stat() 181 | 182 | acc_cov = AccCov() 183 | generated_inputs = self.__current_fuzzer.generated_inputs() 184 | for i in generated_inputs: 185 | t = DynamicTrace(self.fuzzing_program.program_path, i, 186 | target_opts=self.target_opts, 187 | input_placeholder=self.input_placeholder) 188 | 189 | acc_cov.add_trace(t) 190 | 191 | i = 0 192 | for from_addr, to_addr in self.nccdetector.detect_nccs(acc_cov): 193 | if from_addr in self.fuzzing_program.c_all_block_addrs: 194 | continue 195 | 196 | transformed_program_name = self.fuzzing_program.program_name + '_' + str(i) 197 | transformed_program_dir = os.path.join(self.workdir, transformed_program_name) 198 | transformed_program_path = os.path.join(transformed_program_dir, transformed_program_name) 199 | try: 200 | os.makedirs(transformed_program_dir) 201 | except: 202 | pass 203 | 204 | shutil.copyfile(self.fuzzing_program.program_path, transformed_program_path) 205 | os.chmod(transformed_program_path, 0777) 206 | transformed_tprogram = TProgram(transformed_program_path) 207 | 208 | transformed_tprogram.c_all_instr_addrs = self.fuzzing_program.c_all_instr_addrs[:] 209 | transformed_tprogram.c_all_block_addrs = self.fuzzing_program.c_all_block_addrs[:] 210 | 211 | transformed_tprogram.c_block_addr = from_addr 212 | transformed_tprogram.c_all_block_addrs.append(from_addr) 213 | 214 | # TODO: guard this code with context manager 215 | radare2 = Radare2(transformed_program_path, flags=['-w']) 216 | c_addr = radare2.get_cjump_addr(from_addr) 217 | if c_addr == 0: 218 | self.__clean_tprogram(transformed_tprogram) 219 | continue 220 | 221 | radare2.negate_cjmp(c_addr) 222 | 223 | radare2.close() 224 | 225 | transformed_tprogram.c_all_instr_addrs.append(c_addr) 226 | transformed_tprogram.c_instr_addr = c_addr 227 | transformed_tprogram.inputs_from_fuzzing_parent = generated_inputs 228 | transformed_tprogram.write_metadata() 229 | 230 | self.__add_to_queue(transformed_tprogram) 231 | i = i + 1 232 | 233 | logger.debug("Fuzzing %s done", self.fuzzing_program.program_name) 234 | 235 | def stop(self): 236 | self.__stop = True 237 | -------------------------------------------------------------------------------- /tfuzz/tprogram.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import os 3 | import stat 4 | import re 5 | import shutil 6 | import pickle 7 | import threading 8 | import time 9 | import itertools 10 | import stat 11 | import ConfigParser 12 | # from graphviz import Digraph 13 | 14 | import angr 15 | import fuzzer 16 | import tracer 17 | 18 | from . import r2 19 | from . import cov 20 | 21 | logger = logging.getLogger("tfuzz.tprogram") 22 | 23 | class TProgram(object): 24 | 25 | def __init__(self, program_path, c_instr_addrs=None, c_block_addrs=None): 26 | assert os.path.exists(program_path), "%s does not exist" % program_path 27 | self.program_path = program_path 28 | program_name = os.path.basename(program_path) 29 | self.program_name = program_name 30 | self.program_dir = os.path.dirname(self.program_path) 31 | 32 | self.config_file = self.program_path + '.meta' 33 | self.config_section_name = 'tmeta' 34 | self.config = ConfigParser.RawConfigParser() 35 | self.__parent = None 36 | 37 | self.__init_seed_from_parenet = [] 38 | self.__c_all_instr_addrs = [] if c_instr_addrs \ 39 | is None else c_instr_addrs 40 | self.__c_all_block_addrs = [] if c_block_addrs \ 41 | is None else c_block_addrs 42 | 43 | self.__c_block_addr = None 44 | self.__c_instr_addr = None 45 | self.inputs_from_fuzzing_parent = None 46 | 47 | if os.path.exists(self.config_file): 48 | self.config.read(self.config_file) 49 | self.__c_all_block_addrs = eval(self.config.get(self.config_section_name, 'c_all_block_addrs')) 50 | self.__c_all_instr_addrs = eval(self.config.get(self.config_section_name, 'c_all_instr_addrs')) 51 | self.__c_block_addr = eval(self.config.get(self.config_section_name, 'c_block_addr')) 52 | self.__c_instr_addr = eval(self.config.get(self.config_section_name, 'c_instr_addr')) 53 | self.inputs_from_fuzzing_parent = eval(self.config.get(self.config_section_name, 'inputs_from_fuzzing_parent')) 54 | 55 | def is_cgc(self): 56 | ''' 57 | QUICK HACK by checking the magic values 58 | ''' 59 | ret = False 60 | with open(self.program_path, 'r') as f: 61 | f4 = f.read(4) 62 | if f4[1:] == "CGC": 63 | ret = True 64 | return ret 65 | 66 | @property 67 | def c_block_addr(self): 68 | return self.__c_block_addr 69 | 70 | @c_block_addr.setter 71 | def c_block_addr(self, addr): 72 | self.__c_block_addr = addr 73 | 74 | @property 75 | def c_instr_addr(self): 76 | return self.__c_instr_addr 77 | 78 | @c_instr_addr.setter 79 | def c_instr_addr(self, addr): 80 | self.__c_instr_addr = addr 81 | 82 | @property 83 | def c_all_instr_addrs(self): 84 | return self.__c_all_instr_addrs 85 | 86 | @c_all_instr_addrs.setter 87 | def c_all_instr_addrs(self, c_addrs): 88 | self.__c_all_instr_addrs = c_addrs 89 | 90 | @property 91 | def c_all_block_addrs(self): 92 | return self.__c_all_block_addrs 93 | 94 | @c_all_block_addrs.setter 95 | def c_all_block_addrs(self, c_addrs): 96 | self.__c_all_block_addrs = c_addrs 97 | 98 | @property 99 | def parent(self): 100 | return self.__parent 101 | 102 | @parent.setter 103 | def parent(self, p): 104 | self.__parent = p 105 | 106 | def __str__(self): 107 | return "<" + self.program_path + ">" 108 | 109 | def __repr__(self): 110 | return self.__str__() 111 | 112 | def __del__(self): 113 | self.write_metadata() 114 | 115 | def write_metadata(self): 116 | if self.config == None: 117 | return 118 | 119 | try: 120 | self.config.add_section(self.config_section_name) 121 | except: 122 | pass 123 | 124 | self.config.set(self.config_section_name, 'c_all_block_addrs', self.__c_all_block_addrs) 125 | self.config.set(self.config_section_name, 'c_all_instr_addrs', self.__c_all_instr_addrs) 126 | self.config.set(self.config_section_name, 'c_block_addr', self.__c_block_addr) 127 | self.config.set(self.config_section_name, 'c_instr_addr', self.__c_instr_addr) 128 | self.config.set(self.config_section_name, 'inputs_from_fuzzing_parent', self.inputs_from_fuzzing_parent) 129 | 130 | with open(self.config_file, 'w') as f: 131 | self.config.write(f) 132 | 133 | -------------------------------------------------------------------------------- /tfuzz/utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import subprocess 4 | 5 | from fuzzer import Fuzzer as __angr_Fuzzer 6 | 7 | def create_dict(binary, dict_filename): 8 | create_dict_script = os.path.join(__angr_Fuzzer._get_base(), 9 | "bin", "create_dict.py") 10 | args = [sys.executable, create_dict_script, binary] 11 | 12 | with open(dict_filename, 'wb') as df: 13 | p = subprocess.Popen(args, stdout=df) 14 | retcode = p.wait() 15 | 16 | return retcode == 0 and os.path.getsize(dict_filename) 17 | 18 | def replace_input_placeholder(target_opts, input_file, 19 | input_placeholder='@@'): 20 | if target_opts == None: 21 | return None 22 | 23 | if input_file == None or input_placeholder == None: 24 | raise ValueError("input_file and input_placeholder could not be None") 25 | 26 | if not isinstance(input_placeholder, str) or \ 27 | not isinstance(input_file, str) : 28 | raise ValueError("input_file and input_placeholder must be of str type") 29 | 30 | return [input_file if e == input_placeholder else e for e in target_opts] 31 | --------------------------------------------------------------------------------