├── .gitignore ├── .gitmodules ├── README.md ├── build.sh ├── ctrl_execme.sh ├── download.sh ├── info.sh ├── init.sh ├── rsave_execme.sh ├── run_debian9.sh ├── run_http_dataflow.sh ├── run_pwnme_little.sh ├── run_pwnme_lots.sh └── run_trace_me.sh /.gitignore: -------------------------------------------------------------------------------- 1 | debian9.qcow2 2 | exec_me.blocks 3 | exec_me.rsave 4 | exec_me.vmstate 5 | pwnme_little.blocks 6 | pwnme_little.rsave 7 | pwnme_little.vmstate 8 | pwnme_lots.blocks 9 | pwnme_lots.rsave 10 | pwnme_lots.vmstate 11 | -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "qemu-shoggoth"] 2 | path = qemu-shoggoth 3 | url = https://github.com/cromulencellc/qemu-shoggoth.git 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Shoggoth QEMU 2 | 3 | Shoggoth is a heavily modified fork of [QEMU](https://github.com/qemu/qemu) 4 | 4.0.1 designed for high performance Dynamic Binary Instrumentation (DBI) of a 5 | complete system with options to perform focused user-mode analysis. Features 6 | added to QEMU include operating system handlers, external plugins, and Rapid 7 | Analysis (RA). OS handlers enable user-mode analysis within a system context 8 | while maintaining interactions with the OS provided kernel, drivers, and 9 | libraries. Dynamically loaded plugins extend QEMU features and provide DBI of 10 | the executing virtual machine. Rapid Analysis operates as a separate execution 11 | mode which provides rapid loading and saving of execution history for a live 12 | system. All these features work together to offer a complete DBI solution for 13 | deep target analysis. 14 | 15 | 16 | ## Getting Started 17 | 18 | ### Prerequisites 19 | 20 | Before building Shoggoth, you'll need to install some dependencies and 21 | configure your build environment. Please see or run the provided 22 | [`init.sh`](init.sh) script for this (will prompt for a password to install 23 | packages). 24 | 25 | ### Building 26 | 27 | Once your development environment has been configured, the `qemu-shoggoth` 28 | folder will contain the latest stable code for Shoggoth. The shell scripts and 29 | live snapshots in this repository, `shoggoth-extras`, provide examples of the 30 | added features. 31 | 32 | You can build Shoggoth with the included [`./build.sh`](build.sh) script. 33 | Shoggoth will take a while to build, so go get a mocha. 34 | 35 | 36 | ## Running Shoggoth 37 | 38 | ### Traditional Execution 39 | 40 | Shoggoth can be used just like traditional QEMU execution but with the added 41 | features of OS handlers and external plugins. OS handlers are accessed through 42 | the monitor (CLI) and QMP interfaces. External plugins are capable of extending 43 | OS handlers, devices, machines, and plugin languages or adding instrumentation 44 | for memory reads/writes, CPU registers, executing instructions, syscalls, 45 | interrupts, exceptions, ingress/egress network packets, breakpoints, VM state, 46 | and Rapid Analysis (when RA is active). Python 3 plugins are partially 47 | supported, but still considered experimental. 48 | 49 | Use the provided [`run_debian9.sh`](run_debian9.sh) script for an example of 50 | traditional QEMU execution. Username is "user", password is "user". 51 | 52 | #### OS Handlers 53 | 54 | When the OS has booted, use the `os` command to activate OS handler support. 55 | We can't guarantee full support for most OSes, but we plan to integrate 56 | [Volatility](https://www.volatilityfoundation.org/) through an OS handler 57 | plugin which will provide more comprehensive OS support. 58 | 59 | ``` 60 | (qemu) os 61 | Found OS linux! 62 | (qemu) ps 63 | PID CR3 TGID NAME 64 | ->0 3cb0f000 0 swapper/0 65 | ->1 3cb0f000 1 systemd 66 | 162 369f0000 162 systemd-journal 67 | 194 3a68f000 194 systemd-udevd 68 | 223 38ce9000 223 systemd-timesyn 69 | 249 3a740000 249 dbus-daemon 70 | 264 3a618000 264 rsyslogd 71 | 265 3a7d1000 265 cron 72 | 268 3a79c000 268 systemd-logind 73 | 324 3cbf8000 324 login 74 | 442 3c0d8000 442 bash 75 | 354 39e43000 354 dhclient 76 | (qemu) pid 354 77 | PID CR3 TGID NAME 78 | ->0 3a7d1000 0 swapper/0 79 | 1 3cb0f000 1 systemd 80 | 162 369f0000 162 systemd-journal 81 | 194 3a68f000 194 systemd-udevd 82 | 223 38ce9000 223 systemd-timesyn 83 | 249 3a740000 249 dbus-daemon 84 | 264 3a618000 264 rsyslogd 85 | ->265 3a7d1000 265 cron 86 | 268 3a79c000 268 systemd-logind 87 | 324 3cbf8000 324 login 88 | 442 3c0d8000 442 bash 89 | *354 39e43000 354 dhclient 90 | ``` 91 | 92 | The `ps` listing uses `->` to denote the active application and `*` to denote 93 | the application in focus. Use the `pid` command to set the application focus. 94 | 95 | To see the memory mappings for the application in focus use `maps`. 96 | 97 | ``` 98 | (qemu) os 99 | Found OS linux! 100 | (qemu) pid 472 101 | PID CR3 TGID NAME 102 | 1 3cb0f000 1 systemd 103 | 163 3cbdc000 163 systemd-journal 104 | 186 3a38d000 186 systemd-udevd 105 | 280 3690e000 280 systemd-timesyn 106 | 305 39f14000 305 rsyslogd 107 | 306 3a24e000 306 systemd-logind 108 | 309 38ca1000 309 dbus-daemon 109 | 325 3bee9000 325 cron 110 | 341 3a25b000 341 dhclient 111 | 354 3b138000 354 login 112 | 422 38caa000 422 bash 113 | *->472 3a63f000 472 print_name 114 | (qemu) maps 115 | Location Flags Protections 116 | 0x400000-0x401000 0x8800875 0x25 117 | 0x600000-0x601000 0x8900871 0x8000000000000025 118 | 0x601000-0x602000 0x8900873 0x8000000000000025 119 | 0x602000-0x624000 0x8100073 0x8000000000000025 120 | 0x7ffff7a3a000-0x7ffff7bcf000 0x8000075 0x25 121 | 0x7ffff7bcf000-0x7ffff7dcf000 0x8000070 0x120 122 | 0x7ffff7dcf000-0x7ffff7dd3000 0x8900071 0x8000000000000025 123 | 0x7ffff7dd3000-0x7ffff7dd5000 0x8900073 0x8000000000000025 124 | 0x7ffff7dd5000-0x7ffff7dd9000 0x8900073 0x8000000000000025 125 | 0x7ffff7dd9000-0x7ffff7dfc000 0x8000875 0x25 126 | 0x7ffff7ff0000-0x7ffff7ff2000 0x8100073 0x8000000000000025 127 | 0x7ffff7ff8000-0x7ffff7ffa000 0xc044411 0x8000000000000025 128 | 0x7ffff7ffa000-0x7ffff7ffc000 0x8040075 0x25 129 | 0x7ffff7ffc000-0x7ffff7ffd000 0x8900871 0x8000000000000025 130 | 0x7ffff7ffd000-0x7ffff7ffe000 0x8900873 0x8000000000000025 131 | 0x7ffff7ffe000-0x7ffff7fff000 0x8100073 0x8000000000000025 132 | 0x7ffffffde000-0x7ffffffff000 0x100173 0x8000000000000025 133 | (qemu) 134 | ``` 135 | 136 | The OS handler included works with the majority of Debian and Ubuntu distros 137 | using Linux Kernel 4.x. 138 | 139 | 140 | #### Debugging 141 | 142 | Once an application is in focus, breakpoints can be set for process debugging 143 | or analysis. Note that setting a breakpoint with no active focus results in an 144 | anonymous breakpoint that can be triggered for any application that executes 145 | the address. 146 | 147 | ``` 148 | (qemu) stop 149 | (qemu) bp 0x5000000 150 | Set breakpoint id 0 at address 5000000 151 | (qemu) bplist 152 | ID Address 153 | 0 0x5000000 154 | (qemu) c 155 | ``` 156 | 157 | Use the `unpid` command to remove application focus: 158 | Note that breakpoints will only trigger for the application that 159 | was in focus at the time when it was set. 160 | 161 | ``` 162 | (qemu) unpid 163 | ``` 164 | 165 | Unlike traditional GDB, breakpoints stop the entire system. After which, all 166 | registers and memory states can be observed through monitor commands. A more 167 | advanced triaging CLI is currently in development. 168 | 169 | The current live snapshot of a virtual machine can be used for Rapid Analysis 170 | by using the `rsavevm` command: 171 | 172 | ``` 173 | (qemu) rsavevm my_vm_state 174 | ``` 175 | 176 | ### Rapid Analysis Execution 177 | 178 | RA is a separate execution mode that enables rapid saving and loading of a live 179 | system. Actual performance can vary by system but typical times are in tens of 180 | milliseconds for both saving and loading, with saving being on average 2x as 181 | costly as a load. Control of RA mode is exposed through either a TCP socket or 182 | plugin. SHA1 hashes are used to indicate the chronological execution state of a 183 | live system. Use the [`info.sh`](info.sh) script to view the available states 184 | of a vmstate file: 185 | 186 | ``` 187 | $./info.sh exec_me.vmstate 188 | image: ff26c69bf0389fec2afb52c86ee3359dd7de6cbc 189 | file format: Root VMState 190 | virtual size: 180M (188554836 bytes) 191 | disk size: 180M 192 | Snapshot list: 193 | ID TAG VM SIZE DATE VM CLOCK 194 | -1 0000000000000000000000000000000000000000 0 1969-12-31 16:33:13 00:00:00.000 195 | 196 | image: be9f86320f28ec64fa257f6c57a61dc90ded648b 197 | file format: Rapid Analysis VMState 198 | virtual size: 447K (457270 bytes) 199 | disk size: 447K 200 | Snapshot list: 201 | ID TAG VM SIZE DATE VM CLOCK 202 | -1 ff26c69bf0389fec2afb52c86ee3359dd7de6cbc 0 1969-12-31 16:33:14 00:00:00.000 203 | ``` 204 | 205 | A demo of the TCP interface for RA mode control is provided by the 206 | [`ctrl_execme.sh`](ctrl_execme.sh) and [`rsave_execme.sh`](rsave_execme.sh) 207 | scripts: 208 | 209 | ``` 210 | $./ctrl_execme.sh & 211 | $./rsave_execme.sh 212 | ``` 213 | 214 | A demo of the plugin interface for RA mode control is provided by the 215 | [`run_pwnme_little.sh`](run_pwnme_little.sh) and 216 | [`run_pwnme_lots.sh`](run_pwnme_lots.sh) scripts: 217 | 218 | ``` 219 | $./run_pwnme_lots.sh 220 | ``` 221 | 222 | `run_pwnme_lots.sh` demonstratetes control of an RA session through the "C" 223 | plugin (see `pwnme_lots_solve` in the plugins directory) shared library. The 224 | "C" interface for this release is considered very stable. The plugin will 225 | executing a code coverage guided fuzzer that will attempt to execute a specific 226 | branch in the code which triggers a buffer overflow on the stack. 227 | 228 | ``` 229 | $./run_pwnme_little.sh 230 | ``` 231 | 232 | `run_pwnme_little.sh` demonstrates control of an RA session through a Python 3 233 | plugin (see `pwnme_little_bof` in the plugins directory). Note that the 234 | Python 3 interface is still considered experimental. The plugin will detect the 235 | location of a buffer overflow on the stack that overwrites the `main` 236 | function's return pointer. 237 | 238 | ``` 239 | $./run_trace_me.sh 240 | ``` 241 | 242 | `run_trace_me.sh` demonstrates per instruction instrumentation. The buffer 243 | to be executed is randomized then executed, the execution callback will then 244 | proceed to disassemble the executing instruction. Note that this demonstrates 245 | RA's ability to recover from internal TCG errors when executing a series 246 | of bad instructions. 247 | 248 | ### Qt GUI 249 | 250 | Shoggoth also provides a basic Qt plugin that replaces the traditional QEMU window. 251 | This feature is currently experimental but you can use it with the following command: 252 | 253 | ``` 254 | ./qemu-system-x86_64 -M pc-i440fx-3.0 --drive file=debian9.qcow2,format=qcow2 -m 2G -monitor stdio -snapshot -diplay plugin,plugin-name=qtview 255 | ``` 256 | 257 | ### Plugins 258 | 259 | Plugins reside in the `qemu-shoggoth/plugins` folder and provide support for 260 | extension or instrumentation of a virtual machine. 261 | 262 | Create skeletons for new plugins using the 263 | [`wizard.py`](qemu-shoggoth/plugins/wizard.py) script in the plugins directory. 264 | 265 | ``` 266 | $./wizard.py create my_plugin 267 | (note callbacks can be added with the wizard when creating a plugin) 268 | $./wizard.py create myplugin -cb ? 269 | (to create a plugin with all callbacks) 270 | $./wizard.py create myplugin -cb ra_start ra_stop ra_idle interrupt memory_read memory_write state_change exception syscall syscall_exit command breakpoint instructions 271 | ``` 272 | 273 | Callbacks perform the following... 274 | 275 | ``` 276 | ra_start 277 | ``` 278 | 279 | Called when a Rapid Analysis session is starting. You can use this callback to change vm properties that aren't supported by the job system. The job system can change settings registers, memory, and file descriptor contents per session. 280 | 281 | ``` 282 | ra_stop 283 | ``` 284 | 285 | Called when a Rapid Analysis session has ended. You can use this callback to collect vm information that isn't easily reported by the job system. The job system can report processor information, registers, physical memory, and virtual memory (full and diffs by page) per session. 286 | 287 | ``` 288 | ra_idle 289 | ``` 290 | 291 | Called when the Rapid Analysis system does not have a pending job. You can use this callback to submit additional jobs to the RA system or perform post-processing. Note that the vm state from the previous job is still loaded however we don't guarantee validity. Please use ra_stop to analyze vm state after a job session finishes. 292 | 293 | ``` 294 | interrupt 295 | ``` 296 | 297 | Receives CPU interrupts. 298 | 299 | ``` 300 | memory_read 301 | ``` 302 | 303 | Receives memory reads for physical, virtual, and DMA accesses after the read has occurred. 304 | 305 | ``` 306 | memory_write 307 | ``` 308 | 309 | Receives memory writes for physical, virtual, and DMA accesses before the write has occurred. 310 | 311 | ``` 312 | state_change 313 | ``` 314 | 315 | Called when the vm changes such as from start to stop or vice versa. 316 | 317 | ``` 318 | exception 319 | ``` 320 | 321 | Called when a CPU exception occurs. 322 | 323 | ``` 324 | syscall 325 | ``` 326 | 327 | Called when a syscall enter is about to occur. 328 | 329 | ``` 330 | syscall_exit 331 | ``` 332 | 333 | Called when a syscall exit is about to occur. 334 | 335 | ``` 336 | command 337 | ``` 338 | 339 | Called when a command is dispatched. The callback should return true if it handles the command. 340 | 341 | ``` 342 | breakpoint 343 | ``` 344 | 345 | Called when a breakpoint has been hit. 346 | 347 | ``` 348 | instructions 349 | ``` 350 | 351 | Called for every executing instruction before it executes. 352 | 353 | ## Versioning 354 | 355 | We intend to use [Semantic Versioning](http://semver.org/) for versioning. This 356 | may be a little rough at first while we transition Shoggoth out of being 357 | internal to the company. For the versions available, see the 358 | [tags on this repository](https://github.com/your/project/tags). 359 | 360 | 361 | ## Contact 362 | 363 | This project is currently maintained by the Shoggoth team at 364 | [Cromulence](https://cromulence.com/). We can be reached directly via email 365 | at `shoggoth [at] cromulence [dot] com`. 366 | 367 | 368 | ## License 369 | 370 | This project is licensed under the GPLv2 - see the [LICENSE.md](LICENSE.md) file for details 371 | -------------------------------------------------------------------------------- /build.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | make -C qemu-shoggoth 3 | make -C qemu-shoggoth/qemu-ctrl/C 4 | ln -fs qemu-shoggoth/qemu-img qemu-img 5 | ln -fs qemu-shoggoth/x86_64-softmmu/qemu-system-x86_64 qemu-system-x86_64 6 | ln -fs qemu-shoggoth/qemu-ctrl/C/racomms racomms 7 | -------------------------------------------------------------------------------- /ctrl_execme.sh: -------------------------------------------------------------------------------- 1 | ./racomms 6666 be9f86320f28ec64fa257f6c57a61dc90ded648b 2 | -------------------------------------------------------------------------------- /download.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | IMAGE_SITE="https://shoggoth-images.labs.cromulence.com/" 3 | IMAGES=("debian9.qcow2" \ 4 | "pwnme_little.blocks" "pwnme_little.rsave" "pwnme_little.vmstate" \ 5 | "pwnme_lots.blocks" "pwnme_lots.rsave" "pwnme_lots.vmstate" \ 6 | "exec_me.blocks" "exec_me.rsave" "exec_me.vmstate" \ 7 | ) 8 | for i in "${IMAGES[@]}" 9 | do 10 | wget -O $i $IMAGE_SITE$i 11 | done 12 | 13 | -------------------------------------------------------------------------------- /info.sh: -------------------------------------------------------------------------------- 1 | ./qemu-img info -f vmstate $@ 2 | -------------------------------------------------------------------------------- /init.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | git submodule update --init 3 | sudo apt install libtool-bin autoconf bison flex libpixman-1-dev zlib1g-dev libglib2.0-dev python3-dev libsdl2-dev python3-pip python3-setuptools python-is-python3 qtdeclarative5-dev libcapstone-dev 4 | ./download.sh 5 | cd qemu-shoggoth; ./configure --target-list="x86_64-softmmu" --enable-sdl --enable-debug 6 | -------------------------------------------------------------------------------- /rsave_execme.sh: -------------------------------------------------------------------------------- 1 | ./qemu-system-x86_64 -M pc-i440fx-3.0 -ra file=exec_me.rsave,notrace=on,notree=on,istep=1,nosave=on,ilimit=20,ints=off,noblocks=on,connect=0.0.0.0:6666 -m 2G -monitor stdio 2 | -------------------------------------------------------------------------------- /run_debian9.sh: -------------------------------------------------------------------------------- 1 | ./qemu-system-x86_64 -M pc-i440fx-3.0 --drive file=debian9.qcow2,format=qcow2 -m 2G -monitor stdio -snapshot 2 | -------------------------------------------------------------------------------- /run_http_dataflow.sh: -------------------------------------------------------------------------------- 1 | ./qemu-system-x86_64 -M pc-i440fx-3.0 --drive file=debian9.qcow2,format=qcow2 -m 2G -monitor stdio -device e1000,netdev=net0 -netdev user,id=net0,hostfwd=tcp::8888-:8000 --loadvm http_server -plugin http_dataflow 2 | -------------------------------------------------------------------------------- /run_pwnme_little.sh: -------------------------------------------------------------------------------- 1 | ./qemu-system-x86_64 -M pc-i440fx-3.0 -ra file=pwnme_little.rsave,notrace=on,notree=on,istep=0,nosave=on,ilimit=200000,ints=off,noblocks=on -m 2G -monitor stdio -plugin pwnme_little_bof 2 | 3 | -------------------------------------------------------------------------------- /run_pwnme_lots.sh: -------------------------------------------------------------------------------- 1 | ./qemu-system-x86_64 -M pc-i440fx-3.0 -ra file=pwnme_lots.rsave,notrace=on,notree=on,istep=1,nosave=on,ilimit=200,ints=off,noblocks=on -m 2G -monitor stdio -plugin pwnme_lots_solve 2 | 3 | -------------------------------------------------------------------------------- /run_trace_me.sh: -------------------------------------------------------------------------------- 1 | ./qemu-system-x86_64 -M pc-i440fx-3.0 -ra file=exec_me.rsave,notrace=on,notree=on,istep=1,nosave=on,ilimit=20,ints=off,noblocks=on -m 2G -monitor stdio -plugin trace_exec_me 2 | --------------------------------------------------------------------------------