├── .gitignore ├── CHANGELOG.md ├── CONTRIBUTING.md ├── LICENSE ├── Makefile ├── README.md ├── doc ├── forking.md ├── glossary.md ├── properties.md ├── shrinking.md └── usage.md ├── inc ├── theft.h └── theft_types.h ├── pc └── libtheft.pc.in ├── scripts └── mk_bits_lut ├── src ├── theft.c ├── theft_autoshrink.c ├── theft_autoshrink.h ├── theft_autoshrink_internal.h ├── theft_aux.c ├── theft_aux_builtin.c ├── theft_bloom.c ├── theft_bloom.h ├── theft_call.c ├── theft_call.h ├── theft_call_internal.h ├── theft_hash.c ├── theft_random.c ├── theft_random.h ├── theft_rng.c ├── theft_rng.h ├── theft_run.c ├── theft_run.h ├── theft_run_internal.h ├── theft_shrink.c ├── theft_shrink.h ├── theft_shrink_internal.h ├── theft_trial.c ├── theft_trial.h ├── theft_trial_internal.h └── theft_types_internal.h ├── test ├── test_char_array.c ├── test_theft.c ├── test_theft.h ├── test_theft_autoshrink.c ├── test_theft_autoshrink_bulk.c ├── test_theft_autoshrink_bulk.h ├── test_theft_autoshrink_int_array.c ├── test_theft_autoshrink_int_array.h ├── test_theft_autoshrink_ll.c ├── test_theft_autoshrink_ll.h ├── test_theft_aux.c ├── test_theft_bloom.c ├── test_theft_error.c ├── test_theft_integration.c └── test_theft_prng.c └── vendor └── greatest.h /.gitignore: -------------------------------------------------------------------------------- 1 | cscope.* 2 | tags 3 | TAGS 4 | build/ 5 | tmp/ 6 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # theft Changes By Release 2 | 3 | ## v0.4.5 - 2019-02-11 4 | 5 | ### API Changes 6 | 7 | None. 8 | 9 | 10 | ### Bug Fixes 11 | 12 | Only run the prng test suite once. (Probably a merge error.) 13 | 14 | 15 | ### Other Improvements 16 | 17 | Changed the pkg-config path setup in the Makefile default to 18 | `${PREFIX}/lib/pkgconfig`, and make it easier to override via 19 | the environment. 20 | 21 | Eliminated a couple warnings: an assignment in an assert, missing 22 | prototypes. (Thanks @jmesmon.) 23 | 24 | Updated vendored copy of greatest to 1.4.1. 25 | 26 | 27 | ## v0.4.4 - 2018-10-06 28 | 29 | ### API Changes 30 | 31 | None. 32 | 33 | ### Bug Fixes 34 | 35 | Added check so `free` instance callback is optional. Previously, the 36 | documentation indicated it was optional, but the trial cleanup code 37 | always attempted to call it. (Reported by @deweerdt.) 38 | 39 | Fixed a bug in `infer_arity` that incorrectly indicated that a 40 | configuration with THEFT_MAX_ARITY (7) arguments had 0. (Reported by 41 | @kquick.) 42 | 43 | Fixed a bug in the builtin char array hexdump function's print callback, 44 | which could lead to printing memory past the end of the char array. 45 | (Fixed by @kquick.) 46 | 47 | 48 | ### Other Improvements 49 | 50 | Added `-fPIC` to build flags. 51 | 52 | Fixed a few typos. (Thanks @neuschaefer.) 53 | 54 | Added Makefile targets for vi-style ctags & cscope. (Thanks @alyptik.) 55 | 56 | Added `${DESTDIR}` prefix to Makefile install paths, for easier 57 | sandboxed builds and packaging. (Thanks @richardipsum.) 58 | 59 | 60 | ## v0.4.3 - 2017-09-04 61 | 62 | ### API Changes 63 | 64 | Added the `.exit_timeout` field to `struct theft_run_config`'s 65 | `.fork` configuration field. (As this uses the default when 0, 66 | it isn't a breaking API change.) 67 | 68 | 69 | ### Bug Fixes 70 | 71 | Fixed worker process management (issue #19): theft now ensures that 72 | forked child processes have terminated and been cleaned up with 73 | `waitpid` before starting another trial, to prevent zombie processes 74 | from accumulating. 75 | 76 | Fixed a bug where autoshrinking would find a minimal counter-example, 77 | but then erroneously shrink to a non-minimal one, and subsequently 78 | reject the minimal one as already tried. This was caused by a custom 79 | hash callback that hashed values when autoshrinking -- it would land on 80 | a minimal value (1) right away due to the aux. built-in generator's 81 | special values list, then drop the bits that chose using the special 82 | table, and end up with a larger value. If the special value list input 83 | generating 1 and generating 1 normally hashed differently, then it would 84 | shrink back to 1. This means that providing a custom hash function with 85 | autoshrinking enabled should be an API misuse error, but that is an 86 | interface change, so it will wait until a non-bugfix release. 87 | 88 | 89 | ### Other Improvements 90 | 91 | Forked worker processes that have timed out are now given a configurable 92 | window to clean up and exit (possibly successfully) before they are 93 | terminated with SIGKILL. 94 | 95 | Moved forked worker process state from local variables to a 96 | `worker_info` structure in `struct theft`. This gathers state that will 97 | later be used to manage multiple workers in parallel (issue #16). 98 | 99 | Limited how much the autoshrinker attempts to shrink by dropping 100 | requests when there is a small number of requests -- this reduces 101 | dead ends during shrinking. 102 | 103 | Added a `pkg-config` file for libtheft.a. (Thanks @katef.) 104 | 105 | Fixed typos in the documentation. (Thanks @katef.) 106 | 107 | 108 | ## v0.4.2 - 2017-08-23 109 | 110 | ### API Changes 111 | 112 | None. 113 | 114 | 115 | ### Bug Fixes 116 | 117 | Fixed an autoshrinking bug that could cause shrinking to get stuck on 118 | values close to the actual minimimum. 119 | 120 | 121 | ### Other Improvements 122 | 123 | When using the builtin floating point generators, a hexdump of the raw 124 | value is now printed along with the "%g"-formatted output, since it is 125 | lossy. 126 | 127 | Autoshrinking instances with many requests is now more efficient. 128 | 129 | 130 | ## v0.4.1 - 2017-08-20 131 | 132 | ### API Changes 133 | 134 | None. 135 | 136 | 137 | ### Bug Fixes 138 | 139 | Fixed a possible double-free when a non-first argument's alloc callback 140 | returns SKIP or ERROR. 141 | 142 | Fixed a case where the `trial_post` callback could be called with 143 | incorrect pointers in `info->args[]`, due to an inconsistency in how 144 | autoshrink wrapped the arguments and `type_info`. (Issue #18.) 145 | 146 | Fixed autoshrink's handling of the `theft_autoshrink_print_mode` default: 147 | now the default is use the `type_info` print callback when defined, 148 | and to otherwise print the requests. (This was the intentended behavior, 149 | but `THEFT_AUTOSHRINK_PRINT_USER` was 0, which meant it was instead 150 | clobbered with `THEFT_AUTOSHRINK_PRINT_REQUESTS`.) 151 | 152 | 153 | ### Other Improvements 154 | 155 | Internal refactoring: Autoshrinking is now better integrated into the 156 | argument handling. The bugs addressed in this release came from 157 | inconsistencies in how autoshrink wrapped arguments. 158 | 159 | 160 | ## v0.4.0 - 2017-08-13 161 | 162 | ### API Changes 163 | 164 | Changed the property function typedef (`theft_propfun`). Property 165 | functions are now called with a `struct theft *t` handle as a first 166 | argument -- this can be used to get the hooks' environment with 167 | `theft_hook_get_env` while running the property. 168 | 169 | The property function pointer in the `theft_run_config` struct 170 | is now typesafe -- instead of a single function pointer type 171 | with unconstrained arguments, the config struct has fields 172 | `prop1`, `prop2`, ... `prop7`, where the number is the number 173 | of instance arguments the property takes: For example, `prop2` 174 | is: 175 | 176 | enum theft_trial_res 177 | two_instance_property(struct theft *t, void *arg1, void *arg2); 178 | 179 | The property function field has been rename from `fun` to 180 | `prop{ARG_COUNT}`. 181 | 182 | Reduced `THEFT_MAX_ARITY` to 7. 183 | 184 | Added the `.fork` structure to `struct theft_run_config` -- if 185 | `.fork.enable` is set, then theft will fork before running the property 186 | function. This can be used to shrink input that causes the code under 187 | test to crash. If forking is enabled, `.fork.timeout` adds a timeout (in 188 | milliseconds) for each property trial, to shrink input that causes 189 | infinite loops or wide runtime variation. `.fork.signal` customizes 190 | the signal sent on timeout. See `doc/forking.md` for details. 191 | 192 | Added a `fork_post` hook, which is called on the child process 193 | after forking. This can be used to drop privileges before 194 | running the property test. 195 | 196 | Added `theft_generate`, to generate and print an instance based 197 | on a given seed (without running any properties). 198 | 199 | Manual Bloom filter configuration is deprecated, because the Bloom 200 | filter now resizes automatically -- The bloom_bits setting in 201 | `struct theft_run_config` and related constants are ignored, 202 | and will be removed in a future release. 203 | 204 | Added `theft_random_choice`, which returns approximately evenly 205 | distributed random `uint64_t` values less than an upper bound. 206 | 207 | Added `theft_run_res_str`, which returns a string (e.g. "PASS") for an 208 | `enum theft_run_res` value. 209 | 210 | Removed `THEFT_RUN_ERROR_MISSING_CALLBACK` from `enum theft_run_res`; 211 | it's now combined with `THEFT_RUN_ERROR_BAD_ARGS`. 212 | 213 | Added `THEFT_RUN_ERROR_MEMORY` to `enum theft_run_res`. This is 214 | returned if internal memory allocation fails. 215 | 216 | Added `repeat` flag to the info struct associated with the 217 | `trial_post` hook. This is set when a test is being repeated. 218 | 219 | Added `theft_hook_first_fail_halt`, a `trial_pre` hook that halts after 220 | the first failure. 221 | 222 | 223 | ### Other Improvements 224 | 225 | Switch to a dynamic blocked Bloom filter instead of a fixed-size Bloom 226 | filter. This makes manual filter size configuration unnecessary, and 227 | significantly reduces theft's memory overhead -- instead of a single 228 | large Bloom filter, it now uses a set of small filters, which can 229 | individually grow as necessary. 230 | 231 | Lots of internal refactoring. 232 | 233 | Added a warning when the `trial_done` callback is overridden 234 | but `theft_print_trial_result` is called with the overall 235 | hook environment pointer (cast to a `theft_print_trial_result_env`), 236 | since this is probably API misuse. 237 | 238 | 239 | ## v0.3.0 - 2017-06-15 240 | 241 | ### API Changes 242 | 243 | There is now support for generic shrinking. To use this, set 244 | `.autoshrink_config.enable` to `true` on the `theft_type_info` 245 | struct. This requires the `theft_alloc_cb` to be written so 246 | `theft_random_bits` giving smaller random values will lead to 247 | simpler generated instances -- see `doc/shrinking.md`. This 248 | feature is still experimental. 249 | 250 | `theft_init` and `theft_free` have been removed from the public API, and 251 | `theft_run` no longer has a first argument of a `struct theft *t` 252 | handle. Instead, the theft test runner is allocated and freed inside of 253 | `theft_run`, reducing boilerplate. 254 | 255 | The `theft_progress_cb` callback's role has significantly expanded. It 256 | has been broken up into several distinct callbacks, which are set within 257 | `theft_run_config`'s `hooks` struct. See the *Hooks* section in 258 | `doc/usage.md` and their individual type definitions for more 259 | information. These hooks support many useful test-specific behaviors, 260 | such as halting shrinking after a time limit or a certain number of 261 | unsuccessful shrinks, or re-running a failed trial (with arguments 262 | shrunken to a local minima) with log levels adjusted or a debugger 263 | attached. 264 | 265 | `struct theft_type_info` now has a `void *env` field, and that (rather 266 | than the `void *env` associated with hooks) will be passed to its 267 | callbacks. It can be NULL, or the same as the hooks' environment. 268 | This allows type-specific details, such as limits, to be passed to the 269 | callbacks, and also makes reuse of the `type_info` callbacks easier. 270 | 271 | Added `theft_random_bits`, which returns less than the full 64 bits from 272 | the random number generator, and buffers the rest for future requests. 273 | `theft_random_bits_bulk` can be used to request more 64 bits at once, as 274 | long as a buffer is provided. (`theft_random` can still be used to get 275 | 64 bits at a time, but will be removed in a future release.) 276 | 277 | The `theft_alloc_cb` callback no longer has a random seed passed to it 278 | directly. Instead, `theft_random_bits(t, bit_count)` or 279 | `theft_random_bits_bulk(t, bit_count, buffer)` should be used. This 280 | tells theft how much of the random bit stream is being consumed, which 281 | improves efficiency and influences several internal heuristics. 282 | 283 | The `theft_shrink_cb` callback now has the `struct theft *t` handle 284 | passed to it as an extra argument -- this is so shrink callbacks 285 | can use its random number generator. 286 | 287 | Some arguments to the following functions have been made `const`: 288 | - theft_run 289 | - theft_print_cb 290 | - theft_hash_cb 291 | - theft_hash_onepass 292 | - theft_hash_sink 293 | 294 | The following enum types in the API are no longer typedef'd: 295 | - enum theft_trial_res 296 | - enum theft_run_res 297 | - enum theft_progress_callback_res 298 | 299 | Rather than returning a void pointer or special sentinel values (e.g. 300 | `THEFT_DEAD_END`), the `alloc` and `shrink` callbacks now return an enum 301 | and (when appropriate) write their output into a pointer argument called 302 | `output`. 303 | 304 | The instance argument to the `shrink` callback is now const. 305 | 306 | Renamed `struct theft_cfg` to `struct theft_run_config`. 307 | 308 | The `struct theft` type is now opaque. 309 | 310 | `THEFT_BLOOM_DISABLE` has been removed -- the Bloom filter 311 | is always allocated now. 312 | 313 | In `struct theft_run_config`, `always_seed_count` and `trials` are now 314 | `size_t`s rather than `int`s. 315 | 316 | The struct `theft_run_config`'s type_info array field now points 317 | to `const` values, because the built-in type_info structs returned 318 | by `theft_get_builtin_type_info` are const. 319 | 320 | `theft_run` will now return `THEFT_RUN_SKIP` if a run completes 321 | without any passes or failures (because all trials were skipped). 322 | 323 | The output format for properties and counter-examples has been 324 | streamlined. 325 | 326 | Added `theft_seed_of_time`, to get a PRNG seed based on the current 327 | time. 328 | 329 | Added `theft_generic_free_cb`, a `free` type_info callback that just 330 | calls `free(instance)`. 331 | 332 | Added `theft_trial_res_str`, which returns a string (e.g. "PASS") for an 333 | `enum theft_trial_res` value. 334 | 335 | Added several built-in generators for common types, which can be 336 | accessed via `theft_get_builtin_type_info` (to use them as-is) or 337 | `theft_copy_builtin_type_info` (to copy them, and then set a limit). 338 | 339 | 340 | ### Other Improvements 341 | 342 | Added this changelog. 343 | 344 | Added documentation under `doc/`, and updated the `README.md`. 345 | 346 | Added improved syntax highlighting to the README. (Thanks @lochsh). 347 | 348 | Added `CONTRIBUTING.md`. 349 | 350 | Fixed bug in error checking code path. (Thanks @iximeow.) 351 | 352 | Updated vendored version of greatest. 353 | 354 | Use inttypes.h and `PRIx64` to avoid printf string build warning. 355 | 356 | Restructured project layout to build in a `build` directory, and 357 | keep source, header, and test files in `src`, `inc`, and `test`. 358 | 359 | Broke up `theft.c` into several files. 360 | 361 | An intentionally tautological comparison in a test has been removed, 362 | because it led to warnings at build-time and suppressing the warning 363 | wasn't portable between compilers. 364 | 365 | Added Makefile targets for coverage checking and profiling. 366 | 367 | 368 | ## v0.2.0 - 2014-08-06 369 | 370 | ### API Changes 371 | 372 | Add `THEFT_BLOOM_DISABLE` to explicitly disable Bloom filter. 373 | 374 | Switch to 64-bit Mersenne Twister. 375 | 376 | 377 | ### Other Improvements 378 | 379 | README and documentation changes. 380 | 381 | 382 | 383 | ## v0.1.0 - 2014-06-29 384 | 385 | Initial public release. 386 | 387 | 388 | 389 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to theft 2 | 3 | Thanks for taking time to contribute to theft! 4 | 5 | Some issues may be tagged with `beginner` in [the Issues page][issues], 6 | those should be particularly approachable. 7 | 8 | Please send patches or pull requests against the `develop` branch. This 9 | makes it easier to avoid interface changes until they can be reflected 10 | in version number updates and the CHANGELOG. 11 | 12 | Sending changes via patch or pull request acknowledges that you are 13 | willing and able to contribute it under this project's license. (Please 14 | don't contribute code you aren't legally able to share.) 15 | 16 | 17 | ## Bug Reports 18 | 19 | Please report bugs at [the Issues page][issues]. 20 | 21 | [issues]: https://github.com/silentbicycle/theft/issues 22 | 23 | If you are reporting a bug, please include: 24 | 25 | + Your operating system name and version. 26 | 27 | + Your compiler version and target platform. 28 | 29 | + Any details about your local setup that might be helpful in 30 | troubleshooting. 31 | 32 | + Detailed steps to reproduce the bug. 33 | 34 | 35 | ## Documentation 36 | 37 | Improvements to the documentation are welcome. So are requests for 38 | clarification -- if the documentation or comments in the API headers 39 | are unclear or misleading, that's a potential source of bugs. 40 | 41 | 42 | ## Versioning & Compatibility 43 | 44 | The versioning format is MAJOR.MINOR.PATCH. 45 | 46 | Performance improvements or minor bug fixes that do not break 47 | compatibility with past releases lead to patch version increases. API 48 | changes that do not break compatibility lead to minor version increases 49 | and reset the patch version. Changes that do break compatibility 50 | will lead to a major version increase once reaching version 1.0.0, but 51 | will lead to a minor version increase until then. All breaking changes 52 | should between releases should be noted in the CHANGELOG. 53 | 54 | Values derived from the PRNG bitstream are not expected to be consistent 55 | between versions of the library, though it would be better to stay 56 | consistent when possible. (This may change after reaching version 57 | 1.0.0.) 58 | 59 | 60 | ## Portability 61 | 62 | theft expects to run in a Unix-like environment. It is currently tested 63 | on Linux (64-bit `x86_64` and 32-bit ARM), OpenBSD (`amd64`), and OSX, and 64 | running it on other OSs and hardware platforms may help discover bugs. 65 | 66 | Aside from that, theft tries to assume little about its environment, and 67 | tries to avoid depending on external libraries. 68 | 69 | It may run on Windows eventually, but I haven't put any effort into that 70 | yet. 71 | 72 | 73 | ## Testing 74 | 75 | The internal tests are based on [greatest][g]. The test suite is expected 76 | to run without any warnings from `valgrind` (aside from the warnings 77 | about still reachable `fprintf` buffers on OSX). 78 | 79 | [g]: https://github.com/silentbicycle/greatest 80 | 81 | Note that integration tests may fail a very small percentage of the 82 | time, because many of the tests are probabalistic -- they check whether 83 | theft was able to find a known minimal case within a certain number of 84 | tries (and getting close, but not to the exact value, is still a 85 | failure). They could be fixed by always using the same seeds, but 86 | currently I am using this to motivate improving autoshrinking further, 87 | and eventually this problem should go away entirely. 88 | 89 | Contributors are encouraged to add tests for any new functionality, and 90 | in particular to add regression tests for any bugs found. 91 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2014-19 Scott Vokes 2 | 3 | Permission to use, copy, modify, and/or distribute this software for any 4 | purpose with or without fee is hereby granted, provided that the above 5 | copyright notice and this permission notice appear in all copies. 6 | 7 | THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 8 | WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 9 | MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 10 | ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 11 | WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 12 | ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 13 | OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 14 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | PROJECT = theft 2 | BUILD = build 3 | SRC = src 4 | TEST = test 5 | INC = inc 6 | SCRIPTS = scripts 7 | VENDOR = vendor 8 | COVERAGE = -fprofile-arcs -ftest-coverage 9 | PROFILE = -pg 10 | OPTIMIZE = -O3 11 | #OPTIMIZE = -O0 ${COVERAGE} 12 | #OPTIMIZE = -O0 ${PROFILE} 13 | 14 | WARN = -Wall -Wextra -pedantic 15 | CDEFS += -D_POSIX_C_SOURCE=199309L -D_C99_SOURCE 16 | CINCS += -I${INC} -I${VENDOR} -I${BUILD} 17 | CFLAGS += -std=c99 -g ${WARN} ${CDEFS} ${OPTIMIZE} ${CINCS} 18 | CFLAGS += -fPIC 19 | 20 | # Note: -lm is only needed if using built-in floating point generators 21 | LDFLAGS += -lm 22 | 23 | all: ${BUILD}/lib${PROJECT}.a 24 | all: ${BUILD}/test_${PROJECT} 25 | 26 | TEST_CFLAGS += ${CFLAGS} -I${SRC} 27 | TEST_LDFLAGS += ${LDFLAGS} 28 | 29 | OBJS= ${BUILD}/theft.o \ 30 | ${BUILD}/theft_autoshrink.o \ 31 | ${BUILD}/theft_bloom.o \ 32 | ${BUILD}/theft_call.o \ 33 | ${BUILD}/theft_hash.o \ 34 | ${BUILD}/theft_random.o \ 35 | ${BUILD}/theft_rng.o \ 36 | ${BUILD}/theft_run.o \ 37 | ${BUILD}/theft_shrink.o \ 38 | ${BUILD}/theft_trial.o \ 39 | ${BUILD}/theft_aux.o \ 40 | ${BUILD}/theft_aux_builtin.o \ 41 | 42 | TEST_OBJS= ${BUILD}/test_theft.o \ 43 | ${BUILD}/test_theft_autoshrink.o \ 44 | ${BUILD}/test_theft_autoshrink_ll.o \ 45 | ${BUILD}/test_theft_autoshrink_bulk.o \ 46 | ${BUILD}/test_theft_autoshrink_int_array.o \ 47 | ${BUILD}/test_theft_aux.o \ 48 | ${BUILD}/test_theft_bloom.o \ 49 | ${BUILD}/test_theft_error.o \ 50 | ${BUILD}/test_theft_prng.o \ 51 | ${BUILD}/test_theft_integration.o \ 52 | ${BUILD}/test_char_array.o \ 53 | 54 | 55 | # Basic targets 56 | 57 | test: ${BUILD}/test_${PROJECT} 58 | ${BUILD}/test_${PROJECT} ${ARG} 59 | 60 | clean: 61 | rm -rf ${BUILD} gmon.out cscope.out 62 | 63 | cscope: ${SRC}/*.c ${SRC}/*.h ${INC}/* 64 | cscope -bu ${SRC}/*.[ch] ${INC}/*.h ${TEST}/*.[ch] 65 | 66 | ctags: ${BUILD}/tags 67 | 68 | tags: ${BUILD}/TAGS 69 | 70 | ${BUILD}/lib${PROJECT}.a: ${OBJS} ${BUILD}/lib${PROJECT}.pc 71 | ar -rcs ${BUILD}/lib${PROJECT}.a ${OBJS} 72 | 73 | ${BUILD}/test_${PROJECT}: ${OBJS} ${TEST_OBJS} 74 | ${CC} -o $@ ${OBJS} ${TEST_OBJS} ${TEST_CFLAGS} ${TEST_LDFLAGS} 75 | 76 | ${BUILD}/%.o: ${SRC}/%.c ${SRC}/*.h ${INC}/* | ${BUILD} 77 | ${CC} -c -o $@ ${CFLAGS} $< 78 | 79 | ${BUILD}/%.o: ${TEST}/%.c ${SRC}/*.h ${INC}/* | ${BUILD} 80 | ${CC} -c -o $@ ${TEST_CFLAGS} $< 81 | 82 | ${BUILD}/tags: ${SRC}/*.c ${SRC}/*.h ${INC}/* | ${BUILD} 83 | ctags -f $@ \ 84 | --tag-relative --langmap=c:+.h --fields=+l --c-kinds=+l --extra=+q \ 85 | ${SRC}/*.[ch] ${INC}/*.h ${TEST}/*.[ch] 86 | 87 | ${BUILD}/TAGS: ${SRC}/*.c ${SRC}/*.h ${INC}/* | ${BUILD} 88 | etags -o $@ ${SRC}/*.[ch] ${INC}/*.h ${TEST}/*.[ch] 89 | 90 | ${BUILD}/*.o: Makefile 91 | 92 | ${BUILD}: 93 | mkdir ${BUILD} 94 | 95 | ${BUILD}/cover: | ${BUILD} 96 | mkdir ${BUILD}/cover 97 | 98 | profile: test 99 | gprof build/test_theft 100 | 101 | coverage: test | ${BUILD} ${BUILD}/cover 102 | ls -1 src/*.c | sed -e "s#src/#build/#" | xargs -n1 gcov 103 | @echo moving coverage files to ${BUILD}/cover 104 | mv *.gcov ${BUILD}/cover 105 | 106 | ${BUILD}/theft_bloom.o: ${BUILD}/bits_lut.h 107 | ${BUILD}/theft_autoshrink.o: ${BUILD}/bits_lut.h 108 | 109 | ${BUILD}/bits_lut.h: | ${BUILD} 110 | ${SCRIPTS}/mk_bits_lut > $@ 111 | 112 | ${BUILD}/%.pc: pc/%.pc.in | ${BUILD} 113 | sed -e 's,@prefix@,${PREFIX},g' $< > $@ 114 | 115 | # Installation 116 | PREFIX ?= /usr/local 117 | PKGCONFIG_DST ?=${DESTDIR}${PREFIX}/lib/pkgconfig 118 | INSTALL ?= install 119 | RM ?= rm 120 | 121 | install: ${BUILD}/lib${PROJECT}.a ${BUILD}/lib${PROJECT}.pc 122 | ${INSTALL} -d ${DESTDIR}${PREFIX}/lib/ 123 | ${INSTALL} -c ${BUILD}/lib${PROJECT}.a ${DESTDIR}${PREFIX}/lib/lib${PROJECT}.a 124 | ${INSTALL} -d ${DESTDIR}${PREFIX}/include/ 125 | ${INSTALL} -c ${INC}/${PROJECT}.h ${DESTDIR}${PREFIX}/include/ 126 | ${INSTALL} -c ${INC}/${PROJECT}_types.h ${DESTDIR}${PREFIX}/include/ 127 | ${INSTALL} -d ${DESTDIR}${PKGCONFIG_DST} 128 | ${INSTALL} -c ${BUILD}/lib${PROJECT}.pc ${DESTDIR}${PKGCONFIG_DST}/ 129 | 130 | uninstall: 131 | ${RM} ${DESTDIR}${PREFIX}/lib/lib${PROJECT}.a 132 | ${RM} ${DESTDIR}${PREFIX}/include/${PROJECT}.h 133 | ${RM} ${DESTDIR}${PREFIX}/include/${PROJECT}_types.h 134 | ${RM} ${DESTDIR}${PKGCONFIG_DST}/lib${PROJECT}.pc 135 | 136 | 137 | # Other dependencies 138 | ${BUILD}/theft.o: Makefile ${INC}/*.h 139 | 140 | .PHONY: all test clean cscope ctags tags coverage profile install uninstall 141 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # theft: property-based testing for C 2 | 3 | theft is a C library for property-based testing. Where example-based 4 | testing checks test results for specific input, theft tests assert 5 | general properties ("for any possible input, [some condition] should 6 | hold"), generate input, and search for counter-examples that make the 7 | test fail. If theft finds any failures, it also knows how to generate 8 | and test simpler variants of the input, and then report the simplest 9 | counter-example found. 10 | 11 | theft is distributed under the ISC license. 12 | 13 | 14 | ## Installation 15 | 16 | theft does not depend on anything beyond C99 and a Unix-like 17 | environment. Its internal tests use [greatest][], but there is not any 18 | coupling between them. It contains implementations of the 19 | [Mersenne Twister][mt] PRNG and the [FNV-1a][fnv] hashing algorithm - 20 | see their files for copyright info. 21 | 22 | [greatest]: https://github.com/silentbicycle/greatest 23 | [mt]: http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html 24 | [fnv]: http://www.isthe.com/chongo/tech/comp/fnv/ 25 | 26 | 27 | To build, using GNU make: 28 | 29 | $ make 30 | 31 | Note: You may need to call it as `gmake`, especially if building on BSD. 32 | 33 | To build and run the tests: 34 | 35 | $ make test 36 | 37 | This will produce example output from several falsifiable properties, 38 | and confirm that failures have been found. 39 | 40 | To install libtheft and its headers: 41 | 42 | $ make install # using sudo, if necessary 43 | 44 | theft can also be vendored inside of projects -- in that case, just make 45 | sure the headers in `${VENDOR}/theft/inc/` are added to the `-I` include 46 | path, and `${VENDOR}/theft/build/libtheft.a` is linked. 47 | 48 | 49 | ## Usage 50 | 51 | For usage documentation, see [doc/usage.md](doc/usage.md). 52 | 53 | 54 | ## Properties 55 | 56 | For some examples of properties to test, see 57 | [doc/properties.md](doc/properties.md). 58 | 59 | 60 | ## Shrinking and Auto-shrinking 61 | 62 | For more info about shrinking and auto-shrinking, see 63 | [doc/shrinking.md](doc/shrinking.md). 64 | 65 | 66 | ## Forking 67 | 68 | theft can fork before running properties, to shrink failures that make 69 | the code under test crash or exceed an optional timeout. For more info, 70 | see [doc/forking.md](doc/forking.md). 71 | -------------------------------------------------------------------------------- /doc/forking.md: -------------------------------------------------------------------------------- 1 | # Forking 2 | 3 | theft can optionally fork and run the property function in a child process, 4 | so that it can shrink failures that cause crashes or infinite loops. 5 | 6 | Forking is configured via the `.fork` struct nested inside of 7 | `struct theft_run_config`: 8 | 9 | ```c 10 | .fork = { 11 | .enable = true, /* default: disabled */ 12 | .timeout = TIMEOUT_IN_MSEC, /* default: 0 (no timeout) */ 13 | .signal = SIGTERM, /* default: SIGTERM */ 14 | }, 15 | ``` 16 | 17 | Note that changes to memory in the child process will not be 18 | visible to the parent process, due to copy-on-write. 19 | 20 | 21 | ## Performance 22 | 23 | The overhead of shrinking a repeatedly crashing failure can vary 24 | significantly between operating systems. 25 | 26 | In particular, the CrashReporter on macOS slows down shrinking by 27 | several orders of magnitude, as it logs information for every single 28 | crashing process. [Disabling the CrashReporter][1] or running theft 29 | on a non-macOS virtual machine can improve performance. 30 | 31 | [1]: https://www.gregoryvarghese.com/reportcrash-high-cpu-disable-reportcrash/ 32 | 33 | 34 | ## Timeouts 35 | 36 | If forking is enabled, the `.timeout` field can be used to configure a 37 | timeout for each property trial (in milliseconds). If `.timeout` is 38 | nonzero, theft will `kill(2)` the child process if the test does not 39 | complete within the timeout. The `kill` signal defaults to `SIGTERM`, 40 | but can me configured via the `.signal` field. 41 | 42 | After sending the signal, theft will wait for the child process to exit. 43 | If the test needs custom cleanup code, then send a signal such as 44 | `SIGUSR1` to the test instead. Early in the property function, register 45 | a signal handler that will clean up and exit. 46 | 47 | If the child process handles the signal and then returns 48 | `THEFT_TRIAL_PASS` or calls `exit(EXIT_SUCCESS)`, then the trial will 49 | still be considered a `PASS`, otherwise the trial will be considered a 50 | `FAIL`. Signals that kill the process (such as `SIGTERM` or `SIGKILL`) 51 | will always be considered a `FAIL`. 52 | -------------------------------------------------------------------------------- /doc/glossary.md: -------------------------------------------------------------------------------- 1 | # theft glossary 2 | 3 | ## counter-example 4 | 5 | A particurlar combination of one or more instance(s) that 6 | cause a property test function to fail (i.e., the property 7 | does not hold for that input). 8 | 9 | ## dup (duplicate) 10 | 11 | A duplicate trial (one whose combination of input instances have 12 | already been tried). 13 | 14 | ## env (environment) 15 | 16 | A void pointer with arbitrary context for the user's callbacks. 17 | The env pointer is passed along, but opaque to theft itself. 18 | 19 | ## hash 20 | 21 | A mathematical fingerprint derived from some data. theft uses 22 | hashes to check whether a particular combination of instances 23 | has already been tested (via a bloom filter). 24 | 25 | Also, the act of computing the hash of some data. 26 | 27 | ## instance 28 | 29 | A specific value, generated from a known random number generator 30 | seed and a type-specific `alloc` function. 31 | 32 | ## property 33 | 34 | A test function that is expected to hold (that is, return 35 | `THEFT_TRIAL_PASS`) for arbitrary generated input instance(s). If any 36 | input causes the property to fail (return `THEFT_TRIAL_FAIL`), then a 37 | counter-example to the property has been found -- theft will attempt to 38 | shrink the instance(s) as much as possible before reporting the 39 | counter-example. 40 | 41 | ## run 42 | 43 | A batch of trials (typically 100), checking a property test 44 | function with a variety of input instances. 45 | 46 | ## seed 47 | 48 | a starting state for the random number generator. 49 | 50 | ## shrink 51 | 52 | Taking an instance and returning a simpler copy, or indicating 53 | that there are no ways to simplify the instance. When there are 54 | multiple ways to shrink the instance, a 'tactic' is used to 55 | decide between them. 56 | 57 | ## tactic 58 | 59 | A numerical ID used to choose between multiple ways to shrink 60 | (simplify) an instance. 61 | 62 | ## trial 63 | 64 | A single test, checking if a property function holds for a single 65 | instance of each input argument. 66 | -------------------------------------------------------------------------------- /doc/properties.md: -------------------------------------------------------------------------------- 1 | # Properties 2 | 3 | Coming up with useful properties to test can be a bit daunting at first. 4 | Fortunately, there is a lot of good writing about effectively applying 5 | property-based testing. While the test library interfaces differ 6 | tremendously between languages/platforms, the underlying concepts 7 | usually port well. 8 | 9 | - ["What is Property Based Testing?"](http://hypothesis.works/articles/what-is-property-based-testing/) by David R. MacIver 10 | 11 | - ["The easy way to get started with property based testing"](http://www.drmaciver.com/2016/03/the-easy-way-to-get-started-with-property-based-testing/) by David R. MacIver 12 | 13 | - ["Choosing properties for property-based testing"](https://fsharpforfunandprofit.com/posts/property-based-testing-2/) by Scott Wlaschin 14 | 15 | - ["Oracles for Random Testing"](https://blog.regehr.org/archives/856) by John Regehr 16 | 17 | - ["QuickCheck Advice"](https://medium.com/@jlouis666/quickcheck-advice-c357efb4e7e6) by Jesper L. Andersen 18 | 19 | 20 | ## Testing Stateful Systems 21 | 22 | Beyond testing individual functions, property-based testing can also be 23 | used for testing stateful systems by generating a sequence of operations 24 | using them. (In C, this might be represented as an array of fixed-size 25 | "operation" structs, with an enum tag for each operation's API function 26 | and a union for any arguments.) The test function runs each operation, 27 | check its result, and then does an overall consistency check for the 28 | updated state of the system. If all operations run without any errors, 29 | then it checks whether the final state is as expected. If so, the test 30 | passes. 31 | 32 | For example, for a key/value store, the test could run a series of 33 | operations, checking their results along the way: 34 | 35 | - If `set(s, key, value)`'s result indicates it stored successfully, 36 | the test should updated an in-memory list of records. If the key was 37 | invalid, it should check that the store returned an error. 38 | 39 | - Check that `get(s, key)` returns the most recent value from the 40 | in-memory records, or `NOT FOUND`. 41 | 42 | - `delete(s, key)` should return `NOT FOUND` if the key wasn't in the 43 | in-memory records. If it was, it should be removed from them. 44 | 45 | Finally, once the test has run all the key/value store operations, it 46 | should be able to `get(s, key)` key still in the in-memory records, and 47 | all the values should match (otherwise the store lost/corrupted data). 48 | 49 | The main constraint for a property test is that it can be run over and 50 | over with slightly different input -- this usually means testing using 51 | an in-memory database, simulated hardware, or some other stand-in for 52 | external interfaces that can be cleanly reset after each test. It 53 | can use a very simple implementation -- it doesn't need to run in 54 | production, it just needs to be correct, and fast enough for testing. 55 | 56 | 57 | ## Example Properties 58 | 59 | ### During performance tuning 60 | 61 | + For any input, code with clever optimizations should always produce 62 | the same result as a straightforward version that is too inefficient 63 | for production use, but is much easier to check. 64 | 65 | ### In a data compression library 66 | 67 | + For any input, compressing and uncompressing it should produce output 68 | that matches the original input. 69 | 70 | + For any input, the compression output should never be larger than the 71 | original input, beyond some small algorithm-specific overhead. 72 | 73 | + For any input, the uncompression state machine should never get stuck; 74 | it should always be able to reach a valid end-of-stream state once 75 | the end of input is reached. 76 | 77 | ### In a storage library 78 | 79 | + For any sequence of operations, any records that the library says have 80 | been safely written should be readable later. 81 | 82 | + Interleaving those operations with resetting the library's state 83 | in-memory and re-reading headers to recover the current storage 84 | state should never cause data loss. 85 | 86 | + Injecting temporary hardware faults should lead to errors being 87 | handled correctly and recovering without data loss. 88 | 89 | ### In a parser 90 | 91 | + For any input, it should output either a successful parse with a valid 92 | parse tree, or error information. 93 | 94 | + For any valid input (generated by randomly walking the grammar), it 95 | should output a valid parse tree. 96 | 97 | ### In a flash memory wear-leveling system 98 | 99 | + For any sequence of writes (of arbitrary, bounded size), no flash page 100 | should have significantly more writes than the others. 101 | 102 | ### In a datagram-based network 103 | 104 | + For any order of receiving packets (including retransmissions), all 105 | packets should eventually be received and acknowledged, and every 106 | packet should be checksummed once, in order. 107 | 108 | ### In data structure implementations 109 | 110 | + For any sequence of insertions and deletions, a balanced binary tree 111 | should always stay approximately balanced. 112 | 113 | + For any input, a sorting algorithm should produce sorted output. 114 | -------------------------------------------------------------------------------- /doc/shrinking.md: -------------------------------------------------------------------------------- 1 | # Shrinking 2 | 3 | Once theft finds input that makes a property fail, it attempts to 4 | "shrink" it to a minimal example. While it can be hard to tell what 5 | aspect of the original generated input caused the property to fail, 6 | shrinking gradually eliminates irrelevant details, leaving input that 7 | should more clearly point to the root cause. 8 | 9 | Each shrinking step produces a simpler copy of the input, runs the test 10 | again, and checks if it still failed. If so, it commits to that version 11 | and continues shrinking. If the test passes, then it reverts to the 12 | previous version and tries to simplify it another way (or switches 13 | between multiple arguments). It also tracks which combinations of 14 | arguments have been tried, so it doesn't run the test again with the 15 | same input. When shrinking can't make any more progress, it reports the 16 | input as the simplest counter-example it could find. 17 | 18 | Along with the original input, the shrink callback gets a "tactic" 19 | argument -- this is used to choose between different ways to simplify 20 | the instance: "Try to simplify using tactic #1...try to simplify using 21 | tactic #2...". These should be ordered so tactics that remove more 22 | information are tried first, because shrinking by bigger steps helps 23 | theft to converge faster on minimal counter-examples. Successful 24 | shrinking steps reset the tactic counter to 0, in case a later tactic 25 | got earlier tactics unstuck. 26 | 27 | For a list of numbers, shrinking tactics might include: 28 | 29 | + Discarding some percent of the list at random 30 | + Discarding a smaller percent of the list at random 31 | + Dividing all the numbers by 2 (with integer truncation) 32 | + Discarding a single value, based on the tactic ID 33 | + Dividing a single value by 2 34 | + Subtracting 1 from a single value 35 | 36 | The first three tactics shrink the list by much larger steps than the 37 | others. The later tactics will only be tried once the first three aren't 38 | able to make any more progress (because any changes make the test start 39 | passing), but the later tactics' changes may get the first three 40 | unstuck. Shrinking continues until no tactics can make any more 41 | progress, or a custom hook halts shrinking early. 42 | 43 | The shrink callback can vary its tactics as the instance changes. For 44 | example, exploring changes to every individual byte in a 64 KB byte 45 | buffer could take too long, but could be worth trying once other tactics 46 | have reduced the buffer to under 1 KB. Attempting to individually 47 | discard every entry in a list takes `O(n!)` time due to backtracking, 48 | but randomly discarding entries with a 1/32 chance can quickly shrink 49 | the list without a runtime explosion. Switching to discarding every 50 | individual entry may be fine once the list is fairly small (under 100 51 | entries, perhaps). Since the shrink callback's tactic argument is just 52 | an integer, its interpretation is deliberately open-ended. 53 | 54 | theft assumes that that the same combination of instance, `env` info, 55 | and shrinking tactic number should always simplify to the same new 56 | instance, and therefore lead to the property function having the same 57 | result. 58 | 59 | 60 | ## Auto-shrinking 61 | 62 | As of version 0.3.0, theft has experimental support for auto-shrinking. 63 | Instead of using a custom `shrink` callback, it can instead save the 64 | random bitstream used to generate the argument instance, and re-run the 65 | property test while simplifying the bitstream. This implementation is 66 | loosely based on a design described by David R. MacIver. 67 | 68 | In order for auto-shrinking to work, the `alloc` callback has an 69 | additional requirement -- when `theft_random_bits` return smaller 70 | values, it should lead to a simpler generated result. In particular, a 71 | bitstream of all `0` bits should produce a minimal value for the type. 72 | As long as that holds, theft can modify the bitstream in ways that 73 | gradually reduce the complexity of the instance, with heuristics 74 | specific to shrinking a random bit pool. 75 | 76 | While theft doesn't know anything about the type that the user's `alloc` 77 | callback generates, the random bit request sizes give it some clues 78 | about the structure -- these correspond to decisions that influence the 79 | generated type somehow. Modifications to the bit pool during 80 | auto-shrinking are aligned to those requests' bits. 81 | 82 | To enable auto-shrinking, set: 83 | 84 | ```c 85 | .autoshrink_config = { 86 | .enabled = true, 87 | }, 88 | ``` 89 | 90 | in the `theft_type_info` struct. There are other optional configuration 91 | fields in this struct -- see the definition for `struct 92 | theft_autoshrink_config` in `inc/theft_types.h`. 93 | 94 | If autoshrinking is enabled, the `theft_type_info` struct should not 95 | have `shrink` or `hash` callbacks defined. If a custom `print` callback 96 | it provided, it will always be run. Otherwise, default behavior will be 97 | provided for `print` (print the bit pool's requests). 98 | 99 | The `print` behavior can be configured via the `print_mode` field: 100 | 101 | - `THEFT_AUTOSHRINK_PRINT_USER`: Only run the custom `print` callback. 102 | 103 | - `THEFT_AUTOSHRINK_PRINT_BIT_POOL`: Print the raw bit pool. 104 | 105 | - `THEFT_AUTOSHRINK_PRINT_REQUESTS`: Print the request sizes and the 106 | values that were returned. This is the default. 107 | 108 | - `THEFT_AUTOSHRINK_ALL`: Print the raw bit pool and requests. 109 | 110 | -------------------------------------------------------------------------------- /doc/usage.md: -------------------------------------------------------------------------------- 1 | # Usage 2 | 3 | First, `#include "theft.h"` in your code that uses theft. (It will 4 | include `theft_types.h` internally -- no need to include that directly.) 5 | 6 | Then, define a property function: 7 | 8 | ```c 9 | static enum theft_trial_res 10 | prop_encoded_and_decoded_data_should_match(struct theft *t, void *arg1) { 11 | struct buffer *input = (struct buffer *)arg1; 12 | // [compress & uncompress input, compare output & original input] 13 | // return THEFT_TRIAL_PASS, FAIL, SKIP, or ERROR 14 | } 15 | ``` 16 | 17 | This should take one or more generated arguments and return 18 | `THEFT_TRIAL_PASS`, `THEFT_TRIAL_FAIL` if a counter-example to the 19 | property was found, `THEFT_TRIAL_SKIP` if the combination of argument(s) 20 | should be skipped, or `THEFT_TRIAL_ERROR` if the whole theft run should 21 | halt and return an error. 22 | 23 | Then, define how to generate the input argument(s) by providing a struct 24 | with callbacks. (This definition can be shared between properties.) 25 | 26 | For example: 27 | 28 | ```c 29 | static struct theft_type_info random_buffer_info = { 30 | /* allocate a buffer based on random bitstream */ 31 | .alloc = random_buffer_alloc_cb, 32 | /* free the buffer */ 33 | .free = random_buffer_free_cb, 34 | /* get a hash based on the buffer */ 35 | .hash = random_buffer_hash_cb, 36 | /* return a simpler variant of a buffer, or an error */ 37 | .shrink = random_buffer_shrink_cb, 38 | /* print an instance */ 39 | .print = random_buffer_print_cb, 40 | }; 41 | ``` 42 | 43 | All of these callbacks except 'alloc' are optional. For more details, 44 | see the **Type Info Callbacks** subsection below. 45 | 46 | If *autoshrinking* is used, type-generic shrinking and hashing 47 | can be handled internally: 48 | 49 | ```c 50 | static struct theft_type_info random_buffer_info = { 51 | .alloc = random_buffer_alloc_cb, 52 | .free = random_buffer_free_cb, 53 | .print = random_buffer_print_cb, 54 | .autoshrink_config = { 55 | .enable = true, 56 | }, 57 | }; 58 | ``` 59 | 60 | Note that this has implications for how the `alloc` callback is written. 61 | For details, see "Auto-shrinking" in [shrinking.md](shrinking.md). 62 | 63 | Finally, call `theft_run` with a configuration struct: 64 | 65 | ```c 66 | bool test_encode_decode_roundtrip(void) { 67 | struct repeat_once_env env = { .fail = false }; 68 | 69 | /* Get a seed based on the current time */ 70 | theft_seed seed = theft_seed_of_time(); 71 | 72 | /* Property test configuration. 73 | * Note that the number of type_info struct pointers in 74 | * the .type_info field MUST match the field number 75 | * for the property function (here, prop1). */ 76 | struct theft_run_config config = { 77 | .name = __func__, 78 | .prop1 = prop_encoded_and_decoded_data_should_match, 79 | .type_info = { &random_buffer_info }, 80 | .seed = seed, 81 | }; 82 | 83 | /* Run the property test. */ 84 | enum theft_run_res res = theft_run(&config); 85 | return res == THEFT_RUN_PASS; 86 | } 87 | ``` 88 | 89 | The return value will indicate whether it was able to find any failures. 90 | 91 | The config struct has several optional fields. The most commonly 92 | customized ones are: 93 | 94 | - trials: How many trials to run (default: 100). 95 | 96 | - seed: The seed for the randomly generated input. 97 | 98 | - hooks: There are several hooks that can be used to control the test 99 | runner behavior -- see the **Hooks** subsection below. 100 | 101 | - fork: For details about forking, see [forking.md](forking.md). 102 | 103 | 104 | ## Type Info Callbacks 105 | 106 | All of the callbacks are passed the `void *env` field from their 107 | `theft_type_info` struct. This pointer is completely opaque to theft, 108 | but can be cast to an arbitrary struct to pass other test-specifc state 109 | to the callbacks. If its contents vary from trial to trial and it 110 | influences the property test, it should be considered another input and 111 | hashed accordingly. 112 | 113 | 114 | ### alloc - allocate an instance from a random bit stream 115 | 116 | ```c 117 | enum theft_alloc_res { 118 | THEFT_ALLOC_OK, 119 | THEFT_ALLOC_SKIP, 120 | THEFT_ALLOC_ERROR, 121 | }; 122 | typedef enum theft_alloc_res 123 | theft_alloc_cb(struct theft *t, void *env, void **instance); 124 | ``` 125 | 126 | This is the only required callback. 127 | 128 | Construct an argument instance, based off of the random bit stream. 129 | To request random bits, use `theft_random_bits(t, bit_count)` or 130 | `theft_random_bits_bulk(t, bit_count, buffer)`. The bitstream is 131 | produced from a known seed, so it can be constructed again if 132 | necessary. These streams of random bits are not expected to be 133 | consistent between versions of the library. 134 | 135 | To choose a random unsigned int, use `theft_random_choice(t, LIMIT)`, 136 | which will return approximately evenly distributed `uint64_t` 137 | values less than LIMIT. For example, `theft_random_choice(t, 5)` will 138 | return values from `[0, 1, 2, 3, 4]`. 139 | 140 | - On success, write the instance into `(*instance*)` and return 141 | `THEFT_ALLOC_OK`. 142 | 143 | - If the current bit stream should be skipped, return 144 | `THEFT_ALLOC_SKIP`. 145 | 146 | - To halt the entire test run with an error, return `THEFT_ALLOC_ERROR`. 147 | 148 | If **autoshrinking** is used, there is an additional constraint: smaller 149 | random bit values should lead to simpler instances. In particular, a 150 | bitstream of all `0` bits should produce a minimal value for the type. 151 | For more details, see [shrinking.md](shrinking.md). 152 | 153 | 154 | ### free - free an instance and any associated resources 155 | 156 | ```c 157 | typedef void 158 | theft_free_cb(void *instance, void *env); 159 | ``` 160 | 161 | Free the memory and other resources associated with the instance. If not 162 | provided, theft will just leak resources. If only a single 163 | `free(instance)` is needed, use `theft_generic_free_cb`. 164 | 165 | 166 | ### hash - get a hash for an instance 167 | 168 | ```c 169 | typedef theft_hash 170 | theft_hash_cb(const void *instance, void *env); 171 | ``` 172 | 173 | Using the included `theft_hash_*` functionality, produce a hash value 174 | based on a given instance. This will usually consist of 175 | `theft_hash_init(&h)`, then calling `theft_hash_sink(&h, &field, 176 | sizeof(field))` on the instance's contents, and then returning 177 | the result from `theft_hash_done(&h)`. 178 | 179 | If provided, theft will use these hashes to avoid testing combinations 180 | of arguments that have already been tried. Note that if the contents of 181 | `env` impacts how instances are constructed / simplified, it should also 182 | be fed into the hash. 183 | 184 | 185 | ### shrink - produce a simpler copy of an instance 186 | 187 | ```c 188 | enum theft_shrink_res { 189 | THEFT_SHRINK_OK, 190 | THEFT_SHRINK_DEAD_END, 191 | THEFT_SHRINK_NO_MORE_TACTICS, 192 | THEFT_SHRINK_ERROR, 193 | }; 194 | typedef enum theft_shrink_res 195 | theft_shrink_cb(struct theft *t, const void *instance, 196 | uint32_t tactic, void *env, void **output); 197 | ``` 198 | 199 | For a given instance, producer a simpler copy, using the numerical value 200 | in TACTIC to choose between multiple options. If not provided, theft 201 | will just report the initially generated counter-example arguments 202 | as-is. This is equivalent to a shrink callback that always returns 203 | `THEFT_NO_MORE_TACTICS`. 204 | 205 | If a simpler instance can be produced, write it into `(*output)` and 206 | return `THEFT_SHRINK_OK`. If the current tactic is unusable, return 207 | `THEFT_SHRINK_DEAD_END`, and if all known tactics have been tried, 208 | return `THEFT_SHRINK_NO_MORE_TACTICS`. 209 | 210 | If shrinking succeeds, theft will reset the tactic counter back to 211 | 0, so tactics that simplify by larger steps should be tried first, 212 | and then later tactics can get them unstuck. 213 | 214 | For more information about shrinking, recommendations for writing custom 215 | shrinkers, using autoshrinking, and so on, see 216 | [shrinking.md](shrinking.md). 217 | 218 | 219 | ### print - print an instance to the output stream 220 | 221 | ```c 222 | typedef void 223 | theft_print_cb(FILE *f, const void *instance, void *env); 224 | ``` 225 | 226 | Print the instance to a given file stream, behaving like: 227 | 228 | ```c 229 | fprintf(f, "%s", instance_to_string(instance, env)); 230 | ``` 231 | 232 | If not provided, theft will just print the random number seeds that led 233 | to discovering counter-examples. 234 | 235 | 236 | ## Hooks 237 | 238 | `theft_run_config` has several **hook** fields, which can be used to 239 | control theft's behavior: 240 | 241 | ```c 242 | struct { 243 | theft_hook_run_pre_cb *run_pre; 244 | theft_hook_run_post_cb *run_post; 245 | theft_hook_gen_args_pre_cb *gen_args_pre; 246 | theft_hook_trial_pre_cb *trial_pre; 247 | theft_hook_fork_post_cb *fork_post; 248 | theft_hook_trial_post_cb *trial_post; 249 | theft_hook_counterexample_cb *counterexample; 250 | theft_hook_shrink_pre_cb *shrink_pre; 251 | theft_hook_shrink_post_cb *shrink_post; 252 | theft_hook_shrink_trial_post_cb *shrink_trial_post; 253 | /* Environment pointer. This is completely opaque to theft 254 | * itself, but will be passed to all callbacks. */ 255 | void *env; 256 | } hooks; 257 | ``` 258 | 259 | Each one of these is called with a callback-specific `info` struct (with 260 | progress info such as the currently generated argument instances, the 261 | result of the trial that just ran, etc.) and the `.hooks.env` field, 262 | and returns an enum that indicates whether theft should continue, 263 | halt everything with an error, or other callback-specific actions. 264 | 265 | To get the `.hooks.env` pointer in the property function or `type_info` 266 | callbacks, use `theft_hook_get_env(t)`: This environment can be used to 267 | pass in a logging level for the trial, save extra details to print in a 268 | hook later, pass in a size limit for the generated instance, etc. 269 | 270 | Note that the environment shouldn't be changed within a run in a way 271 | that affects trial passes/fails -- for example, changing the iteration 272 | count as a property is re-run for shrinking will distort how changing 273 | the input affects the property, making shrinking less effective. 274 | 275 | For all of the details, see their type definitions in: 276 | [inc/theft_types.h][1]. 277 | 278 | [1]: https://github.com/silentbicycle/theft/blob/master/inc/theft_types.h 279 | 280 | By default: 281 | 282 | - `run_pre` calls `theft_hook_run_pre_print_info` (which calls 283 | `theft_print_run_pre_info` and then returns 284 | `THEFT_HOOK_RUN_PRE_CONTINUE`). 285 | 286 | - `run_post` calls `theft_hook_run_post_print_info` (which calls 287 | `theft_print_run_post_info` and then returns 288 | `THEFT_HOOK_RUN_POST_CONTINUE`). 289 | 290 | - `trial_post` calls `theft_hook_trial_post_print_result` (which calls 291 | `theft_print_trial_result` with an internally allocated 292 | `theft_print_trial_result_env` struct, and then returns 293 | `THEFT_HOOK_TRIAL_POST_CONTINUE`). 294 | 295 | - `counterexample` calls `theft_print_counterexample` and 296 | returns `THEFT_HOOK_COUNTEREXAMPLE_CONTINUE`. 297 | 298 | All other callbacks default to just returning their `*_CONTINUE` result. 299 | 300 | These hooks can be overridden to add test-specific behavior. For example: 301 | 302 | - Reporting can be customized (by changing any of several callbacks) 303 | 304 | - Halting after a certain number of failures (`gen_args_pre` or 305 | `trial_pre`) 306 | 307 | - Halting after a certain amount of time spent running tests 308 | (`gen_args_pre` or `trial_pre`) 309 | 310 | - Running a failing trial again with a debugger attached or logging 311 | changes (`trial_post`) 312 | 313 | - Halting shrinking after a certain amount of time (`shrink_pre`) 314 | 315 | - Dropping privileges with `setrlimit`, `pledge`, etc. on the 316 | forked child process before running the property (`fork_post`). 317 | 318 | 319 | ### Example Output 320 | 321 | Here is what example output for a property test might looks like 322 | with the default hooks, indicating what output comes from which hook: 323 | 324 | `run_pre` (a banner when starting the run): 325 | 326 | == PROP 'property name': 100 trials, seed 0xa4894b4f1ec336b1 327 | 328 | (`gen_args_pre` and then `trial_pre` hooks would be called before each trial) 329 | 330 | `trial_post` (printing dots for progress, and a 'd' for a duplicate trial): 331 | 332 | ....d 333 | 334 | (`shrink_pre`, `shrink_post`, and `shrink_trial_post` hooks would be 335 | called at this point) 336 | 337 | `counterexample` (a failure was found -- printing the arguments): 338 | 339 | -- Counter-Example: property name 340 | Trial 5, Seed 0xa4894b4f1ec336b1 341 | Argument 0: 342 | [0 9 0 ] 343 | -- autoshrink [generation: 17, requests: 5 -- 24/24 bits consumed] 344 | raw: 01 48 42 345 | 346 | requests: (5) 347 | 0 -- 3 bits: 1 (0x1) 348 | 1 -- 8 bits: 0 (0x0) 349 | 2 -- 3 bits: 1 (0x1) 350 | 3 -- 8 bits: 9 (0x9) 351 | 4 -- 2 bits: 1 (0x1) 352 | 353 | `trial_post` (printing an 'F' for the failure, and moving on): 354 | 355 | F.................. 356 | 357 | ... 358 | 359 | `run_post`: 360 | 361 | == FAIL 'property name': pass 26, fail 5, skip 0, dup 1 362 | -------------------------------------------------------------------------------- /inc/theft.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_H 2 | #define THEFT_H 3 | 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | /* Version 0.4.5 */ 10 | #define THEFT_VERSION_MAJOR 0 11 | #define THEFT_VERSION_MINOR 4 12 | #define THEFT_VERSION_PATCH 5 13 | 14 | #include "theft_types.h" 15 | 16 | /* Run a series of randomized trials of a property function. 17 | * 18 | * Configuration is specified in CFG; many fields are optional. 19 | * See the type definition in `theft_types.h`. */ 20 | enum theft_run_res 21 | theft_run(const struct theft_run_config *cfg); 22 | 23 | /* Generate the instance based on a given seed, print it to F, 24 | * and then free it. (If print or free callbacks are NULL, 25 | * they will be skipped.) */ 26 | enum theft_generate_res 27 | theft_generate(FILE *f, theft_seed seed, 28 | const struct theft_type_info *info, void *hook_env); 29 | 30 | 31 | /*********************** 32 | * Getting random bits * 33 | ***********************/ 34 | 35 | /* Get a random 64-bit integer from the test runner's PRNG. 36 | * 37 | * DEPRECATED: This is equivalent to `theft_random_bits(t, 64)`, but 38 | * theft works better when the caller only requests as many bits as 39 | * it needs. This will be removed in a future release! */ 40 | uint64_t theft_random(struct theft *t); 41 | 42 | /* Get BITS random bits from the test runner's PRNG, which will be 43 | * returned as a little-endian uint64_t. At most 64 bits can be 44 | * retrieved at once -- requesting more is a checked error. 45 | * 46 | * For more than 64 bits, use theft_random_bits_bulk. */ 47 | uint64_t theft_random_bits(struct theft *t, uint8_t bits); 48 | 49 | /* Get BITS random bits, in bulk, and put them in BUF. 50 | * BUF is assumed to be large enough, and will be zeroed 51 | * before any bits are copied to it. Bits will be copied 52 | * little-endian. */ 53 | void theft_random_bits_bulk(struct theft *t, uint32_t bits, uint64_t *buf); 54 | 55 | #if THEFT_USE_FLOATING_POINT 56 | /* Get a random double from the test runner's PRNG. */ 57 | double theft_random_double(struct theft *t); 58 | 59 | /* Get a random uint64_t less than CEIL. 60 | * For example, `theft_random_choice(t, 5)` will return 61 | * approximately evenly distributed values from [0, 1, 2, 3, 4]. */ 62 | uint64_t theft_random_choice(struct theft *t, uint64_t ceil); 63 | #endif 64 | 65 | 66 | /*********** 67 | * Hashing * 68 | ***********/ 69 | 70 | /* Hash a buffer in one pass. (Wraps the below functions.) */ 71 | theft_hash theft_hash_onepass(const uint8_t *data, size_t bytes); 72 | 73 | /* Initialize/reset a hasher for incremental hashing. */ 74 | void theft_hash_init(struct theft_hasher *h); 75 | 76 | /* Sink more data into an incremental hash. */ 77 | void theft_hash_sink(struct theft_hasher *h, 78 | const uint8_t *data, size_t bytes); 79 | 80 | /* Finish hashing and get the result. 81 | * (This also resets the internal hasher state.) */ 82 | theft_hash theft_hash_done(struct theft_hasher *h); 83 | 84 | 85 | /********* 86 | * Hooks * 87 | *********/ 88 | 89 | /* Print a trial result in the default format. 90 | * 91 | * To use this, add a `struct theft_print_trial_result_env` to the env 92 | * in the `struct theft_run_config`, and call `theft_print_trial_result` 93 | * with it from inside the `trial_post` hook. 94 | * 95 | * When the default `theft_hook_trial_post_print_result` hook is used, 96 | * the env is allocated and freed internally. 97 | * 98 | * Unless a custom output max_column width is wanted, all of these 99 | * fields can just be initialized to 0. */ 100 | #define THEFT_PRINT_TRIAL_RESULT_ENV_TAG 0xe7a6 101 | struct theft_print_trial_result_env { 102 | uint16_t tag; /* used for internal validation */ 103 | const uint8_t max_column; /* 0 -> default of 72 */ 104 | uint8_t column; 105 | size_t scale_pass; 106 | size_t scale_skip; 107 | size_t scale_dup; 108 | size_t consec_pass; 109 | size_t consec_skip; 110 | size_t consec_dup; 111 | }; 112 | void theft_print_trial_result( 113 | struct theft_print_trial_result_env *print_env, 114 | const struct theft_hook_trial_post_info *info); 115 | 116 | /* Print a property counter-example that caused a failing trial. 117 | * This is the default counterexample hook. */ 118 | enum theft_hook_counterexample_res 119 | theft_print_counterexample(const struct theft_hook_counterexample_info *info, 120 | void *env); 121 | 122 | /* Print a standard pre-run report. */ 123 | void theft_print_run_pre_info(FILE *f, 124 | const struct theft_hook_run_pre_info *info); 125 | 126 | /* Print a standard post-run report. */ 127 | void theft_print_run_post_info(FILE *f, 128 | const struct theft_hook_run_post_info *info); 129 | 130 | /* A run-pre hook that just calls theft_print_run_pre_info and returns 131 | * THEFT_HOOK_RUN_PRE_CONTINUE. */ 132 | enum theft_hook_run_pre_res 133 | theft_hook_run_pre_print_info(const struct theft_hook_run_pre_info *info, void *env); 134 | 135 | /* Halt trials after the first failure. */ 136 | enum theft_hook_trial_pre_res 137 | theft_hook_first_fail_halt(const struct theft_hook_trial_pre_info *info, void *env); 138 | 139 | /* The default trial-post hook, which just calls theft_print_trial_result, 140 | * with an internally allocated `struct theft_print_trial_result_env`. */ 141 | enum theft_hook_trial_post_res 142 | theft_hook_trial_post_print_result(const struct theft_hook_trial_post_info *info, 143 | void *env); 144 | 145 | /* A run-post hook that just calls theft_print_run_post_info and returns 146 | * THEFT_HOOK_RUN_POST_CONTINUE. */ 147 | enum theft_hook_run_post_res 148 | theft_hook_run_post_print_info(const struct theft_hook_run_post_info *info, void *env); 149 | 150 | /* Get the hook environment pointer. 151 | * This is the contents of theft_run_config.hooks.env. */ 152 | void *theft_hook_get_env(struct theft *t); 153 | 154 | 155 | /*************************** 156 | * Other utility functions * 157 | ***************************/ 158 | 159 | /* Change T's output stream handle to OUT. (Default: stdout.) */ 160 | void theft_set_output_stream(struct theft *t, FILE *out); 161 | 162 | /* Get a seed based on the hash of the current timestamp. */ 163 | theft_seed theft_seed_of_time(void); 164 | 165 | /* Generic free callback: just call free(instance). */ 166 | void theft_generic_free_cb(void *instance, void *env); 167 | 168 | /* Return a string name of a trial result. */ 169 | const char *theft_trial_res_str(enum theft_trial_res res); 170 | 171 | /* Return a string name of a run result. */ 172 | const char *theft_run_res_str(enum theft_run_res res); 173 | 174 | /*********************** 175 | * Built-in generators * 176 | ***********************/ 177 | 178 | enum theft_builtin_type_info { 179 | THEFT_BUILTIN_bool, 180 | 181 | /* Built-in unsigned types. 182 | * 183 | * If env is non-NULL, it will be cast to a pointer to this type and 184 | * dereferenced for a limit. 185 | * 186 | * For example, if the theft_type_info struct's env field is set 187 | * like this: 188 | * 189 | * uint8_t limit = 64; 190 | * struct theft_type_info info; 191 | * theft_copy_builtin_type_info(THEFT_BUILTIN_uint8_t, &info); 192 | * info.env = &limit; 193 | * 194 | * then the generator will produce uint8_t values 0 <= x < 64. */ 195 | THEFT_BUILTIN_uint, // platform-specific 196 | THEFT_BUILTIN_uint8_t, 197 | THEFT_BUILTIN_uint16_t, 198 | THEFT_BUILTIN_uint32_t, 199 | THEFT_BUILTIN_uint64_t, 200 | THEFT_BUILTIN_size_t, 201 | 202 | /* Built-in signed types. 203 | * 204 | * If env is non-NULL, it will be cast to a pointer to this type and 205 | * dereferenced for a +/- limit. 206 | * 207 | * For example, if if the theft_type_info struct's env field is set 208 | * like this: 209 | * 210 | * int16_t limit = 1000; // limit must be positive 211 | * struct theft_type_info info; 212 | * theft_copy_builtin_type_info(THEFT_BUILTIN_int16_t, &info); 213 | * info.env = &limit; 214 | * 215 | * then the generator will produce uint8_t values -1000 <= x < 1000. */ 216 | THEFT_BUILTIN_int, 217 | THEFT_BUILTIN_int8_t, 218 | THEFT_BUILTIN_int16_t, 219 | THEFT_BUILTIN_int32_t, 220 | THEFT_BUILTIN_int64_t, 221 | 222 | #if THEFT_USE_FLOATING_POINT 223 | /* Built-in floating point types. 224 | * If env is non-NULL, it will be cast to a pointer of this type and 225 | * dereferenced for a +/- limit. */ 226 | THEFT_BUILTIN_float, 227 | THEFT_BUILTIN_double, 228 | #endif 229 | 230 | /* Built-in array types. 231 | * If env is non-NULL, it will be cast to a `size_t *` and 232 | * deferenced for a max length. */ 233 | THEFT_BUILTIN_char_ARRAY, 234 | THEFT_BUILTIN_uint8_t_ARRAY, 235 | }; 236 | 237 | /* Get a const pointer to built-in type_info callbacks for 238 | * TYPE. See the comments for each type above for details. 239 | * 240 | * NOTE: All built-ins have autoshrink enabled. */ 241 | const struct theft_type_info * 242 | theft_get_builtin_type_info(enum theft_builtin_type_info type); 243 | 244 | /* Copy a built-in type info into INFO, so its fields can be 245 | * modified (e.g. setting a limit in info->env). */ 246 | void 247 | theft_copy_builtin_type_info(enum theft_builtin_type_info type, 248 | struct theft_type_info *info); 249 | 250 | #endif 251 | -------------------------------------------------------------------------------- /pc/libtheft.pc.in: -------------------------------------------------------------------------------- 1 | prefix=@prefix@ 2 | libdir=${prefix}/lib 3 | includedir=${prefix}/include 4 | 5 | Name: theft 6 | Description: Property-based testing for C 7 | Version: 0.4.5 8 | Requires: 9 | Requires.private: 10 | Libs: -L${libdir} -ltheft 11 | Libs.private: -lm 12 | Cflags: -I${includedir} 13 | 14 | -------------------------------------------------------------------------------- /scripts/mk_bits_lut: -------------------------------------------------------------------------------- 1 | #!/usr/bin/awk -f 2 | 3 | # Check for bits being set by right shifting until the 4 | # desired bit is the first bit, and then checking mod 2. 5 | # (standard awk does not have bit operations included.) 6 | function bit_is_set(x, bit) { 7 | while (bit > 0) { 8 | x = int(x / 2) 9 | bit = int(bit / 2) 10 | } 11 | return x % 2 12 | } 13 | 14 | function bit_count(byte, b, t) { 15 | t = 0 16 | for (b = 1; b <= 128; b *= 2) { 17 | if (bit_is_set(byte, b)) { t++ } 18 | } 19 | return t 20 | } 21 | 22 | BEGIN { 23 | printf("static uint8_t bits_lut[256] = {\n"); 24 | 25 | for (i = 0; i < 256; i++) { 26 | if ((i % 8) == 0) { printf(" ") } 27 | printf("0x%02x, ", bit_count(i)) 28 | if ((i % 8) == 7) { printf("\n") } 29 | } 30 | 31 | printf("};\n"); 32 | exit 0 33 | } 34 | 35 | -------------------------------------------------------------------------------- /src/theft.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | 4 | #include "theft.h" 5 | #include "theft_types_internal.h" 6 | #include "theft_run.h" 7 | 8 | static enum theft_trial_res should_not_run(struct theft *t, void *arg1); 9 | 10 | /* Change T's output stream handle to OUT. (Default: stdout.) */ 11 | void theft_set_output_stream(struct theft *t, FILE *out) { 12 | t->out = out; 13 | } 14 | 15 | /* Run a series of randomized trials of a property function. 16 | * 17 | * Configuration is specified in CFG; many fields are optional. 18 | * See the type definition in `theft_types.h`. */ 19 | enum theft_run_res 20 | theft_run(const struct theft_run_config *cfg) { 21 | if (cfg == NULL) { 22 | return THEFT_RUN_ERROR_BAD_ARGS; 23 | } 24 | 25 | struct theft *t = NULL; 26 | 27 | enum theft_run_init_res init_res = theft_run_init(cfg, &t); 28 | switch (init_res) { 29 | case THEFT_RUN_INIT_ERROR_MEMORY: 30 | return THEFT_RUN_ERROR_MEMORY; 31 | default: 32 | assert(false); 33 | case THEFT_RUN_INIT_ERROR_BAD_ARGS: 34 | return THEFT_RUN_ERROR_BAD_ARGS; 35 | case THEFT_RUN_INIT_OK: 36 | break; /* continue below */ 37 | } 38 | 39 | enum theft_run_res res = theft_run_trials(t); 40 | theft_run_free(t); 41 | return res; 42 | } 43 | 44 | enum theft_generate_res 45 | theft_generate(FILE *f, theft_seed seed, 46 | const struct theft_type_info *info, void *hook_env) { 47 | enum theft_generate_res res = THEFT_GENERATE_OK; 48 | struct theft *t = NULL; 49 | 50 | struct theft_run_config cfg = { 51 | .name = "generate", 52 | .prop1 = should_not_run, 53 | .type_info = { info }, 54 | .seed = seed, 55 | .hooks = { 56 | .env = hook_env, 57 | }, 58 | }; 59 | 60 | enum theft_run_init_res init_res = theft_run_init(&cfg, &t); 61 | switch (init_res) { 62 | case THEFT_RUN_INIT_ERROR_MEMORY: 63 | return THEFT_GENERATE_ERROR_MEMORY; 64 | default: 65 | assert(false); 66 | case THEFT_RUN_INIT_ERROR_BAD_ARGS: 67 | return THEFT_GENERATE_ERROR_BAD_ARGS; 68 | case THEFT_RUN_INIT_OK: 69 | break; /* continue below */ 70 | } 71 | 72 | void *instance = NULL; 73 | enum theft_alloc_res ares = info->alloc(t, info->env, &instance); 74 | switch (ares) { 75 | case THEFT_ALLOC_OK: 76 | break; /* continue below */ 77 | case THEFT_ALLOC_SKIP: 78 | res = THEFT_GENERATE_SKIP; 79 | goto cleanup; 80 | case THEFT_ALLOC_ERROR: 81 | res = THEFT_GENERATE_ERROR_ALLOC; 82 | goto cleanup; 83 | } 84 | 85 | if (info->print) { 86 | fprintf(f, "-- Seed 0x%016" PRIx64 "\n", seed); 87 | info->print(f, instance, info->env); 88 | fprintf(f, "\n"); 89 | } 90 | if (info->free) { info->free(instance, info->env); } 91 | 92 | cleanup: 93 | theft_run_free(t); 94 | return res; 95 | } 96 | 97 | static enum theft_trial_res should_not_run(struct theft *t, void *arg1) { 98 | (void)t; 99 | (void)arg1; 100 | return THEFT_TRIAL_ERROR; /* should never be run */ 101 | } 102 | -------------------------------------------------------------------------------- /src/theft_autoshrink.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_AUTOSHRINK_H 2 | #define THEFT_AUTOSHRINK_H 3 | 4 | #include "theft_types_internal.h" 5 | #include 6 | 7 | #define AUTOSHRINK_ENV_TAG 0xa5 8 | #define AUTOSHRINK_BIT_POOL_TAG 'B' 9 | 10 | struct autoshrink_bit_pool { 11 | /* Bits will always be rounded up to a multiple of 64 bits, 12 | * and be aligned as a uint64_t. */ 13 | uint8_t *bits; 14 | bool shrinking; /* is this pool shrinking? */ 15 | size_t bits_filled; /* how many bits are available */ 16 | size_t bits_ceil; /* ceiling for bit buffer */ 17 | size_t limit; /* after limit bytes, return 0 */ 18 | 19 | size_t consumed; 20 | size_t request_count; 21 | size_t request_ceil; 22 | uint32_t *requests; 23 | 24 | size_t generation; 25 | size_t *index; 26 | }; 27 | 28 | /* How large should the default autoshrink bit pool be? 29 | * The pool will be filled and grown on demand, but an 30 | * excessively small initial pool will lead to several 31 | * reallocs in quick succession. */ 32 | #define DEF_POOL_SIZE (64 * 8*sizeof(uint64_t)) 33 | 34 | /* How large should the buffer for request sizes be by default? */ 35 | #define DEF_REQUESTS_CEIL2 4 /* constrain to a power of 2 */ 36 | #define DEF_REQUESTS_CEIL (1 << DEF_REQUESTS_CEIL2) 37 | 38 | /* Default: Decide we've reached a local minimum after 39 | * this many unsuccessful shrinks in a row. */ 40 | #define DEF_MAX_FAILED_SHRINKS 100 41 | 42 | /* When attempting to drop records, default to odds of 43 | * (1+DEF_DROP_THRESHOLD) in (1 << DEF_DROP_BITS). */ 44 | #define DEF_DROP_THRESHOLD 0 45 | #define DEF_DROP_BITS 5 46 | 47 | /* Max number of pooled random bits to give to alloc callback 48 | * before returning 0 forever. Default: No limit. */ 49 | #define DEF_POOL_LIMIT ULLONG_MAX 50 | 51 | /* Magic value to disable selecting a request to drop in 52 | * drop_from_bit_pool, because it complicates tests. */ 53 | #define DO_NOT_DROP (0xFFFFFFFFLU) 54 | 55 | typedef uint64_t autoshrink_prng_fun(uint8_t bits, void *udata); 56 | 57 | #define TWO_EVENLY 0x80 58 | #define FOUR_EVENLY 0x40 59 | #define MODEL_MIN 0x08 60 | #define MODEL_MAX 0x80 61 | 62 | #define DROPS_MIN 0x10 63 | #define DROPS_MAX 0xA0 64 | 65 | enum autoshrink_action { 66 | ASA_DROP = 0x01, 67 | ASA_SHIFT = 0x02, 68 | ASA_MASK = 0x04, 69 | ASA_SWAP = 0x08, 70 | ASA_SUB = 0x10, 71 | }; 72 | 73 | enum autoshrink_weight { 74 | WEIGHT_DROP = 0x00, 75 | WEIGHT_SHIFT = 0x01, 76 | WEIGHT_MASK = 0x02, 77 | WEIGHT_SWAP = 0x03, 78 | WEIGHT_SUB = 0x04, 79 | }; 80 | 81 | struct autoshrink_model { 82 | enum autoshrink_action cur_tried; 83 | enum autoshrink_action cur_set; 84 | enum autoshrink_action next_action; 85 | uint8_t weights[5]; 86 | }; 87 | 88 | struct autoshrink_env { 89 | // config 90 | uint8_t arg_i; 91 | size_t pool_size; 92 | size_t pool_limit; 93 | enum theft_autoshrink_print_mode print_mode; 94 | size_t max_failed_shrinks; 95 | uint64_t drop_threshold; 96 | uint8_t drop_bits; 97 | 98 | struct autoshrink_model model; 99 | struct autoshrink_bit_pool *bit_pool; 100 | 101 | // allow injecting a fake prng, for testing 102 | bool leave_trailing_zeroes; 103 | autoshrink_prng_fun *prng; 104 | void *udata; 105 | }; 106 | 107 | enum mutation { 108 | MUT_SHIFT, 109 | MUT_MASK, 110 | MUT_SWAP, 111 | MUT_SUB, 112 | }; 113 | #define LAST_MUTATION MUT_SUB 114 | #define MUTATION_TYPE_BITS 2 115 | 116 | struct change_info { 117 | enum mutation t; 118 | size_t pos; 119 | uint32_t size; 120 | union { 121 | uint8_t shift; 122 | uint64_t mask; 123 | uint64_t and; 124 | uint64_t sub; 125 | uint8_t swap_unused; 126 | } u; 127 | }; 128 | 129 | struct autoshrink_env * 130 | theft_autoshrink_alloc_env(struct theft *t, uint8_t arg_i, 131 | const struct theft_type_info *type_info); 132 | 133 | void theft_autoshrink_free_env(struct theft *t, struct autoshrink_env *env); 134 | 135 | enum theft_autoshrink_wrap { 136 | THEFT_AUTOSHRINK_WRAP_OK, 137 | THEFT_AUTOSHRINK_WRAP_ERROR_MEMORY = -1, 138 | THEFT_AUTOSHRINK_WRAP_ERROR_MISUSE = -2, 139 | }; 140 | enum theft_autoshrink_wrap 141 | theft_autoshrink_wrap(struct theft *t, 142 | struct theft_type_info *type_info, struct theft_type_info *wrapper); 143 | 144 | void theft_autoshrink_free_bit_pool(struct theft *t, 145 | struct autoshrink_bit_pool *pool); 146 | 147 | void 148 | theft_autoshrink_bit_pool_random(struct theft *t, 149 | struct autoshrink_bit_pool *pool, 150 | uint32_t bit_count, bool save_request, 151 | uint64_t *buf); 152 | 153 | void 154 | theft_autoshrink_get_real_args(struct theft *t, 155 | void **dst, void **src); 156 | 157 | void 158 | theft_autoshrink_update_model(struct theft *t, 159 | uint8_t arg_id, enum theft_trial_res res, 160 | uint8_t adjustment); 161 | 162 | /* Alloc callback, with autoshrink_env passed along. */ 163 | enum theft_alloc_res 164 | theft_autoshrink_alloc(struct theft *t, struct autoshrink_env *env, 165 | void **instance); 166 | 167 | theft_hash 168 | theft_autoshrink_hash(struct theft *t, const void *instance, 169 | struct autoshrink_env *env, void *type_env); 170 | 171 | void 172 | theft_autoshrink_print(struct theft *t, FILE *f, 173 | struct autoshrink_env *env, const void *instance, void *type_env); 174 | 175 | enum theft_shrink_res 176 | theft_autoshrink_shrink(struct theft *t, 177 | struct autoshrink_env *env, 178 | uint32_t tactic, void **output, 179 | struct autoshrink_bit_pool **output_bit_pool); 180 | 181 | /* This is only exported for testing. */ 182 | void theft_autoshrink_dump_bit_pool(FILE *f, size_t bit_count, 183 | const struct autoshrink_bit_pool *pool, 184 | enum theft_autoshrink_print_mode print_mode); 185 | 186 | /* Set the next action the model will deliver. (This is a hook for testing.) */ 187 | void theft_autoshrink_model_set_next(struct autoshrink_env *env, 188 | enum autoshrink_action action); 189 | 190 | #endif 191 | -------------------------------------------------------------------------------- /src/theft_autoshrink_internal.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_AUTOSHRINK_INTERNAL_H 2 | #define THEFT_AUTOSHRINK_INTERNAL_H 3 | 4 | #include "theft_types_internal.h" 5 | #include "theft_autoshrink.h" 6 | 7 | #include 8 | 9 | static struct autoshrink_bit_pool * 10 | alloc_bit_pool(size_t size, size_t limit, 11 | size_t request_ceil); 12 | 13 | static enum theft_alloc_res 14 | alloc_from_bit_pool(struct theft *t, struct autoshrink_env *env, 15 | struct autoshrink_bit_pool *bit_pool, void **output, 16 | bool shrinking); 17 | 18 | static bool append_request(struct autoshrink_bit_pool *pool, 19 | uint32_t bit_count); 20 | 21 | static void drop_from_bit_pool(struct theft *t, 22 | struct autoshrink_env *env, 23 | const struct autoshrink_bit_pool *orig, 24 | struct autoshrink_bit_pool *pool); 25 | 26 | static void mutate_bit_pool(struct theft *t, 27 | struct autoshrink_env *env, 28 | const struct autoshrink_bit_pool *orig, 29 | struct autoshrink_bit_pool *pool); 30 | 31 | static bool choose_and_mutate_request(struct theft *t, 32 | struct autoshrink_env *env, 33 | const struct autoshrink_bit_pool *orig, 34 | struct autoshrink_bit_pool *pool); 35 | 36 | static bool build_index(struct autoshrink_bit_pool *pool); 37 | 38 | static size_t offset_of_pos(const struct autoshrink_bit_pool *orig, 39 | size_t pos); 40 | 41 | static void convert_bit_offset(size_t bit_offset, 42 | size_t *byte_offset, uint8_t *bit); 43 | 44 | static uint64_t 45 | read_bits_at_offset(const struct autoshrink_bit_pool *pool, 46 | size_t bit_offset, uint8_t size); 47 | 48 | static void 49 | write_bits_at_offset(struct autoshrink_bit_pool *pool, 50 | size_t bit_offset, uint8_t size, uint64_t bits); 51 | 52 | static void truncate_trailing_zero_bytes(struct autoshrink_bit_pool *pool); 53 | 54 | static void init_model(struct autoshrink_env *env); 55 | 56 | static enum mutation 57 | get_weighted_mutation(struct theft *t, struct autoshrink_env *env); 58 | 59 | static bool should_drop(struct theft *t, struct autoshrink_env *env, 60 | size_t request_count); 61 | 62 | static void lazily_fill_bit_pool(struct theft *t, 63 | struct autoshrink_bit_pool *pool, 64 | const uint32_t bit_count); 65 | 66 | static void fill_buf(struct autoshrink_bit_pool *pool, 67 | const uint32_t bit_count, uint64_t *buf); 68 | 69 | #endif 70 | -------------------------------------------------------------------------------- /src/theft_aux.c: -------------------------------------------------------------------------------- 1 | #include "theft.h" 2 | #include "theft_types_internal.h" 3 | 4 | #include 5 | #include 6 | 7 | /* Name used when no property name is set. */ 8 | static const char def_prop_name[] = "(anonymous)"; 9 | 10 | theft_seed theft_seed_of_time(void) { 11 | struct timeval tv = { 0, 0 }; 12 | if (-1 == gettimeofday(&tv, NULL)) { 13 | return 0; 14 | } 15 | 16 | return (uint64_t)theft_hash_onepass((const uint8_t *)&tv, sizeof(tv)); 17 | } 18 | 19 | void theft_generic_free_cb(void *instance, void *env) { 20 | (void)env; 21 | free(instance); 22 | } 23 | 24 | /* Print a tally marker for a trial result, but if there have been 25 | * SCALE_FACTOR consecutive ones, increase the scale by an 26 | * order of magnitude. */ 27 | static size_t 28 | autoscale_tally(char *buf, size_t buf_size, size_t scale_factor, 29 | char *name, size_t *cur_scale, char tally, size_t *count) { 30 | const size_t scale = *cur_scale == 0 ? 1 : *cur_scale; 31 | const size_t nscale = scale_factor * scale; 32 | size_t used = 0; 33 | if (scale > 1 || *count >= nscale) { 34 | if (*count == nscale) { 35 | used = snprintf(buf, buf_size, "(%s x %zd)%c", 36 | name, nscale, tally); 37 | *cur_scale = nscale; 38 | } else if ((*count % scale) == 0) { 39 | used = snprintf(buf, buf_size, "%c", tally); 40 | } else { 41 | buf[0] = '\0'; /* truncate -- print nothing */ 42 | } 43 | } else { 44 | used = snprintf(buf, buf_size, "%c", tally); 45 | } 46 | (*count)++; 47 | return used; 48 | } 49 | 50 | void theft_print_trial_result( 51 | struct theft_print_trial_result_env *env, 52 | const struct theft_hook_trial_post_info *info) { 53 | assert(env); 54 | assert(info); 55 | 56 | struct theft *t = info->t; 57 | if (t->print_trial_result_env == env) { 58 | assert(t->print_trial_result_env->tag == THEFT_PRINT_TRIAL_RESULT_ENV_TAG); 59 | } else if ((t->hooks.trial_post != theft_hook_trial_post_print_result) 60 | && env == t->hooks.env) { 61 | if (env != NULL && env->tag != THEFT_PRINT_TRIAL_RESULT_ENV_TAG) { 62 | fprintf(stderr, 63 | "\n" 64 | "WARNING: The *env passed to trial_print_trial_result is probably not\n" 65 | "a `theft_print_trial_result_env` struct -- to suppress this warning,\n" 66 | "set env->tag to THEFT_PRINT_TRIAL_RESULT_ENV_TAG.\n"); 67 | } 68 | } 69 | 70 | const uint8_t maxcol = (env->max_column == 0 71 | ? THEFT_DEF_MAX_COLUMNS : env->max_column); 72 | 73 | size_t used = 0; 74 | char buf[64]; 75 | 76 | switch (info->result) { 77 | case THEFT_TRIAL_PASS: 78 | used = autoscale_tally(buf, sizeof(buf), 100, 79 | "PASS", &env->scale_pass, '.', &env->consec_pass); 80 | break; 81 | case THEFT_TRIAL_FAIL: 82 | used = snprintf(buf, sizeof(buf), "F"); 83 | env->scale_pass = 1; 84 | env->consec_pass = 0; 85 | env->column = 0; 86 | break; 87 | case THEFT_TRIAL_SKIP: 88 | used = autoscale_tally(buf, sizeof(buf), 10, 89 | "SKIP", &env->scale_skip, 's', &env->consec_skip); 90 | break; 91 | case THEFT_TRIAL_DUP: 92 | used = autoscale_tally(buf, sizeof(buf), 10, 93 | "DUP", &env->scale_dup, 'd', &env->consec_dup); 94 | break; 95 | case THEFT_TRIAL_ERROR: 96 | used = snprintf(buf, sizeof(buf), "E"); 97 | break; 98 | default: 99 | assert(false); 100 | return; 101 | } 102 | 103 | assert(info->t); 104 | FILE *f = (info->t->out == NULL ? stdout : info->t->out); 105 | 106 | if (env->column + used >= maxcol) { 107 | fprintf(f, "\n"); 108 | env->column = 0; 109 | } 110 | 111 | fprintf(f, "%s", buf); 112 | fflush(f); 113 | env->column += used; 114 | } 115 | 116 | enum theft_hook_trial_pre_res 117 | theft_hook_first_fail_halt(const struct theft_hook_trial_pre_info *info, void *env) { 118 | (void)env; 119 | return info->failures > 0 120 | ? THEFT_HOOK_TRIAL_PRE_HALT 121 | : THEFT_HOOK_TRIAL_PRE_CONTINUE; 122 | } 123 | 124 | enum theft_hook_trial_post_res 125 | theft_hook_trial_post_print_result(const struct theft_hook_trial_post_info *info, 126 | void *env) { 127 | theft_print_trial_result((struct theft_print_trial_result_env *)env, 128 | info); 129 | return THEFT_HOOK_TRIAL_POST_CONTINUE; 130 | } 131 | 132 | enum theft_hook_counterexample_res 133 | theft_print_counterexample(const struct theft_hook_counterexample_info *info, 134 | void *env) { 135 | (void)env; 136 | struct theft *t = info->t; 137 | int arity = info->arity; 138 | fprintf(t->out, "\n\n -- Counter-Example: %s\n", 139 | info->prop_name ? info->prop_name : ""); 140 | fprintf(t->out, " Trial %zd, Seed 0x%016" PRIx64 "\n", 141 | info->trial_id, (uint64_t)info->trial_seed); 142 | for (int i = 0; i < arity; i++) { 143 | struct theft_type_info *ti = info->type_info[i]; 144 | if (ti->print) { 145 | fprintf(t->out, " Argument %d:\n", i); 146 | ti->print(t->out, info->args[i], ti->env); 147 | fprintf(t->out, "\n"); 148 | } 149 | } 150 | return THEFT_HOOK_COUNTEREXAMPLE_CONTINUE; 151 | } 152 | 153 | void theft_print_run_pre_info(FILE *f, 154 | const struct theft_hook_run_pre_info *info) { 155 | const char *prop_name = info->prop_name ? info->prop_name : def_prop_name; 156 | fprintf(f, "\n== PROP '%s': %zd trials, seed 0x%016" PRIx64 "\n", 157 | prop_name, info->total_trials, 158 | info->run_seed); 159 | } 160 | 161 | enum theft_hook_run_pre_res 162 | theft_hook_run_pre_print_info(const struct theft_hook_run_pre_info *info, 163 | void *env) { 164 | (void)env; 165 | theft_print_run_pre_info(stdout, info); 166 | return THEFT_HOOK_RUN_PRE_CONTINUE; 167 | } 168 | 169 | void theft_print_run_post_info(FILE *f, 170 | const struct theft_hook_run_post_info *info) { 171 | const struct theft_run_report *r = &info->report; 172 | const char *prop_name = info->prop_name ? info->prop_name : def_prop_name; 173 | fprintf(f, "\n== %s '%s': pass %zd, fail %zd, skip %zd, dup %zd\n", 174 | r->fail > 0 ? "FAIL" : "PASS", prop_name, 175 | r->pass, r->fail, r->skip, r->dup); 176 | } 177 | 178 | enum theft_hook_run_post_res 179 | theft_hook_run_post_print_info(const struct theft_hook_run_post_info *info, 180 | void *env) { 181 | (void)env; 182 | theft_print_run_post_info(stdout, info); 183 | return THEFT_HOOK_RUN_POST_CONTINUE; 184 | } 185 | 186 | void *theft_hook_get_env(struct theft *t) { return t->hooks.env; } 187 | 188 | struct theft_aux_print_trial_result_env { 189 | FILE *f; // 0 -> default of stdout 190 | const uint8_t max_column; // 0 -> default of DEF_MAX_COLUMNS 191 | 192 | uint8_t column; 193 | size_t consec_pass; 194 | size_t consec_fail; 195 | }; 196 | 197 | const char *theft_run_res_str(enum theft_run_res res) { 198 | switch (res) { 199 | case THEFT_RUN_PASS: return "PASS"; 200 | case THEFT_RUN_FAIL: return "FAIL"; 201 | case THEFT_RUN_SKIP: return "SKIP"; 202 | case THEFT_RUN_ERROR: return "ERROR"; 203 | case THEFT_RUN_ERROR_MEMORY: return "ERROR_MEMORY"; 204 | case THEFT_RUN_ERROR_BAD_ARGS: return "ERROR_BAD_ARGS"; 205 | default: 206 | return "(matchfail)"; 207 | } 208 | } 209 | 210 | const char *theft_trial_res_str(enum theft_trial_res res) { 211 | switch (res) { 212 | case THEFT_TRIAL_PASS: return "PASS"; 213 | case THEFT_TRIAL_FAIL: return "FAIL"; 214 | case THEFT_TRIAL_SKIP: return "SKIP"; 215 | case THEFT_TRIAL_DUP: return "DUP"; 216 | case THEFT_TRIAL_ERROR: return "ERROR"; 217 | default: 218 | return "(matchfail)"; 219 | } 220 | } 221 | -------------------------------------------------------------------------------- /src/theft_aux_builtin.c: -------------------------------------------------------------------------------- 1 | #include "theft.h" 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | struct type_info_row { 9 | enum theft_builtin_type_info key; 10 | struct theft_type_info value; 11 | }; 12 | 13 | static enum theft_alloc_res 14 | bool_alloc(struct theft *t, void *env, void **instance) { 15 | (void)env; 16 | bool *res = malloc(sizeof(*res)); 17 | if (res == NULL) { return THEFT_ALLOC_ERROR; } 18 | *res = (bool)theft_random_bits(t, 1); 19 | *instance = res; 20 | return THEFT_ALLOC_OK; 21 | } 22 | 23 | #define BITS_USE_SPECIAL (3) 24 | 25 | #define ALLOC_USCALAR(NAME, TYPE, BITS, ...) \ 26 | static enum theft_alloc_res \ 27 | NAME ## _alloc(struct theft *t, void *env, void **instance) { \ 28 | TYPE *res = malloc(sizeof(*res)); \ 29 | if (res == NULL) { return THEFT_ALLOC_ERROR; } \ 30 | if (((1LU << BITS_USE_SPECIAL) - 1 ) == \ 31 | theft_random_bits(t, BITS_USE_SPECIAL)) { \ 32 | const TYPE special[] = { __VA_ARGS__ }; \ 33 | size_t idx = theft_random_bits(t, 8) \ 34 | % (sizeof(special)/sizeof(special[0])); \ 35 | *res = special[idx]; \ 36 | } else { \ 37 | *res = (TYPE)theft_random_bits(t, BITS); \ 38 | } \ 39 | if (env != NULL) { \ 40 | TYPE limit = *(TYPE *)env; \ 41 | assert(limit != 0); \ 42 | (*res) %= limit; \ 43 | } \ 44 | *instance = res; \ 45 | return THEFT_ALLOC_OK; \ 46 | } 47 | 48 | #define ALLOC_SSCALAR(NAME, TYPE, BITS, ...) \ 49 | static enum theft_alloc_res \ 50 | NAME ## _alloc(struct theft *t, void *env, void **instance) { \ 51 | TYPE *res = malloc(sizeof(*res)); \ 52 | if (res == NULL) { return THEFT_ALLOC_ERROR; } \ 53 | if (((1LU << BITS_USE_SPECIAL) - 1 ) == \ 54 | theft_random_bits(t, BITS_USE_SPECIAL)) { \ 55 | const TYPE special[] = { __VA_ARGS__ }; \ 56 | size_t idx = theft_random_bits(t, 8) \ 57 | % (sizeof(special)/sizeof(special[0])); \ 58 | *res = special[idx]; \ 59 | } else { \ 60 | *res = (TYPE)theft_random_bits(t, BITS); \ 61 | } \ 62 | if (env != NULL) { \ 63 | TYPE limit = *(TYPE *)env; \ 64 | assert(limit > 0); /* -limit <= res < limit */ \ 65 | if (*res < (-limit)) { \ 66 | *res %= (-limit); \ 67 | } else if (*res >= limit) { \ 68 | (*res) %= limit; \ 69 | } \ 70 | } \ 71 | *instance = res; \ 72 | return THEFT_ALLOC_OK; \ 73 | } 74 | 75 | #define ALLOC_FSCALAR(NAME, TYPE, MOD, BITS, ...) \ 76 | static enum theft_alloc_res \ 77 | NAME ## _alloc(struct theft *t, void *env, void **instance) { \ 78 | TYPE *res = malloc(sizeof(*res)); \ 79 | if (res == NULL) { return THEFT_ALLOC_ERROR; } \ 80 | if (((1LU << BITS_USE_SPECIAL) - 1 ) == \ 81 | theft_random_bits(t, BITS_USE_SPECIAL)) { \ 82 | const TYPE special[] = { __VA_ARGS__ }; \ 83 | size_t idx = theft_random_bits(t, 8) \ 84 | % (sizeof(special)/sizeof(special[0])); \ 85 | *res = special[idx]; \ 86 | } else { \ 87 | *res = (TYPE)theft_random_bits(t, BITS); \ 88 | } \ 89 | if (env != NULL) { \ 90 | TYPE limit = *(TYPE *)env; \ 91 | assert(limit > 0); /* -limit <= res < limit */ \ 92 | if (*res < (-limit)) { \ 93 | *res = MOD(*res, -limit); \ 94 | } else { \ 95 | *res = MOD(*res, limit); \ 96 | } \ 97 | } \ 98 | *instance = res; \ 99 | return THEFT_ALLOC_OK; \ 100 | } 101 | 102 | #define PRINT_SCALAR(NAME, TYPE, FORMAT) \ 103 | static void NAME ## _print(FILE *f, \ 104 | const void *instance, void *env) { \ 105 | (void)env; \ 106 | fprintf(f, FORMAT, *(TYPE *)instance); \ 107 | } 108 | 109 | ALLOC_USCALAR(uint, unsigned int, 8*sizeof(unsigned int), 110 | 0, 1, 2, 3, 4, 5, 6, 7, 111 | 63, 64, 127, 128, 129, 255, UINT_MAX - 1, UINT_MAX) 112 | 113 | ALLOC_USCALAR(uint8_t, uint8_t, 8*sizeof(uint8_t), 114 | 0, 1, 2, 3, 4, 5, 6, 7, 115 | 63, 64, 65, 127, 128, 129, 254, 255) 116 | 117 | ALLOC_USCALAR(uint16_t, uint16_t, 8*sizeof(uint16_t), 118 | 0, 1, 2, 3, 4, 5, 6, 255, 119 | 256, 1024, 4096, 16384, 32768, 32769, 65534, 65535) 120 | 121 | ALLOC_USCALAR(uint32_t, uint32_t, 8*sizeof(uint32_t), 122 | 0, 1, 2, 3, 4, 5, 6, 255, 123 | (1LU << 8), (1LU << 8) + 1, (1LU << 16) - 1, (1LU << 16), 124 | (1LU << 16) + 1, (1LU << 19), (1LU << 22), (1LLU << 32) - 1) 125 | 126 | ALLOC_USCALAR(uint64_t, uint64_t, 8*sizeof(uint64_t), 127 | 0, 1, 2, 3, 4, 5, 6, 255, 128 | (1LLU << 8), (1LLU << 16), (1LLU << 32), (1LLU << 32) + 1, 129 | (1LLU << 53), (1LLU << 53) + 1, (uint64_t)-2, (uint64_t)-1) 130 | 131 | ALLOC_USCALAR(size_t, size_t, 8*sizeof(size_t), 132 | 0, 1, 2, 3, 4, 5, 6, 255, 133 | 256, (size_t)-2, (size_t)-1) 134 | 135 | 136 | ALLOC_SSCALAR(int, unsigned int, 8*sizeof(int), 137 | 0, 1, 2, 3, -1, -2, -3, -4, 138 | INT_MIN + 1, INT_MIN, INT_MAX - 1, INT_MAX) 139 | 140 | ALLOC_SSCALAR(int8_t, int8_t, 8*sizeof(int8_t), 141 | 0, 1, 2, 3, -1, -2, -3, -4, 142 | 63, 64, 65, 127, 143 | (int8_t)128, (int8_t)129, (int8_t)254, (int8_t)255) 144 | 145 | ALLOC_SSCALAR(int16_t, int16_t, 8*sizeof(int16_t), 146 | 0, 1, 2, 3, 4, 5, 6, 255, 147 | 256, 1024, 4096, 16384, 148 | (int16_t)32768, (int16_t)32769, (int16_t)65534, (int16_t)65535) 149 | 150 | ALLOC_SSCALAR(int32_t, int32_t, 8*sizeof(int32_t), 151 | 0, 1, 2, 3, 4, 5, 6, 255, 152 | (1LU << 8), (1LU << 8) + 1, (1LU << 16) - 1, (1LU << 16), 153 | (int32_t)(1LU << 16) + 1, (int32_t)(1LU << 19), 154 | (int32_t)(1LU << 22), (int32_t)(1LLU << 32) - 1) 155 | 156 | ALLOC_SSCALAR(int64_t, int64_t, 8*sizeof(int64_t), 157 | 0, 1, 2, 3, 4, 5, 6, 255, 158 | (1LLU << 8), (1LLU << 16), (1LLU << 32), (1LLU << 32) + 1, 159 | (1LLU << 53), (1LLU << 53) + 1, (int64_t)-2, (int64_t)-1) 160 | 161 | PRINT_SCALAR(bool, bool, "%d") 162 | PRINT_SCALAR(uint, unsigned int, "%u") 163 | PRINT_SCALAR(uint8_t, uint8_t, "%" PRIu8) 164 | PRINT_SCALAR(uint16_t, uint16_t, "%" PRIu16) 165 | PRINT_SCALAR(uint32_t, uint32_t, "%" PRIu32) 166 | PRINT_SCALAR(uint64_t, uint64_t, "%" PRIu64) 167 | PRINT_SCALAR(size_t, size_t, "%zu") 168 | 169 | PRINT_SCALAR(int, int, "%d") 170 | PRINT_SCALAR(int8_t, int8_t, "%" PRId8) 171 | PRINT_SCALAR(int16_t, int16_t, "%" PRId16) 172 | PRINT_SCALAR(int32_t, int32_t, "%" PRId32) 173 | PRINT_SCALAR(int64_t, int64_t, "%" PRId64) 174 | 175 | #if THEFT_USE_FLOATING_POINT 176 | #include 177 | #include 178 | ALLOC_FSCALAR(float, float, fmodf, 8*sizeof(float), 179 | 0, 1, -1, NAN, 180 | INFINITY, -INFINITY, FLT_MIN, FLT_MAX) 181 | ALLOC_FSCALAR(double, double, fmod, 8*sizeof(double), 182 | 0, 1, -1, NAN, 183 | NAN, INFINITY, -INFINITY, DBL_MIN, DBL_MAX) 184 | 185 | static void float_print(FILE *f, const void *instance, void *env) { 186 | (void)env; 187 | float fl = *(float *)instance; 188 | uint32_t u32 = (uint32_t)fl; 189 | fprintf(f, "%g (0x%08" PRIx32 ")", fl, u32); 190 | } 191 | 192 | static void double_print(FILE *f, const void *instance, void *env) { 193 | (void)env; 194 | double d = *(double *)instance; 195 | uint64_t u64 = (uint64_t)d; 196 | fprintf(f, "%g (0x%016" PRIx64 ")", d, u64); 197 | } 198 | 199 | #endif 200 | 201 | #define SCALAR_ROW(NAME) \ 202 | { \ 203 | .key = THEFT_BUILTIN_ ## NAME, \ 204 | .value = { \ 205 | .alloc = NAME ## _alloc, \ 206 | .free = theft_generic_free_cb, \ 207 | .print = NAME ## _print, \ 208 | .autoshrink_config = { \ 209 | .enable = true, \ 210 | }, \ 211 | }, \ 212 | } 213 | 214 | #define DEF_BYTE_ARRAY_CEIL 8 215 | static enum theft_alloc_res 216 | char_ARRAY_alloc(struct theft *t, void *env, void **instance) { 217 | (void)env; 218 | size_t ceil = DEF_BYTE_ARRAY_CEIL; 219 | size_t size = 0; 220 | size_t *max_length = NULL; 221 | if (env != NULL) { 222 | max_length = (size_t *)env; 223 | assert(*max_length > 0); 224 | } 225 | 226 | char *res = malloc(ceil * sizeof(char)); 227 | if (res == NULL) { return THEFT_ALLOC_ERROR; } 228 | while (true) { 229 | if (max_length != NULL && size + 1 == *max_length) { 230 | res[size] = 0; 231 | break; 232 | } else if (size == ceil) { 233 | const size_t nceil = 2 * ceil; 234 | char *nres = realloc(res, nceil * sizeof(char)); 235 | if (nres == NULL) { 236 | free(res); 237 | return THEFT_ALLOC_ERROR; 238 | } 239 | res = nres; 240 | ceil = nceil; 241 | } 242 | char byte = theft_random_bits(t, 8); 243 | res[size] = byte; 244 | if (byte == 0x00) { 245 | break; 246 | } 247 | size++; 248 | } 249 | 250 | *instance = res; 251 | return THEFT_ALLOC_OK; 252 | } 253 | 254 | static void hexdump(FILE *f, const uint8_t *raw, size_t size) { 255 | for (size_t row_i = 0; row_i < size; row_i += 16) { 256 | size_t rem = (size - row_i > 16 ? 16 : size - row_i); 257 | fprintf(f, "%04zx: ", row_i); 258 | for (size_t i = 0; i < rem; i++) { 259 | fprintf(f, "%02x ", raw[row_i + i]); 260 | } 261 | 262 | for (size_t ii = rem; ii < 16; ++ii) 263 | fprintf(f, " "); /* add padding */ 264 | 265 | for (size_t i = 0; i < rem; i++) { 266 | char c = ((const char *)raw)[i]; 267 | fprintf(f, "%c", (isprint(c) ? c : '.')); 268 | } 269 | fprintf(f, "\n"); 270 | } 271 | } 272 | 273 | static void char_ARRAY_print(FILE *f, const void *instance, void *env) { 274 | (void)env; 275 | const char *s = (const char *)instance; 276 | size_t len = strlen(s); 277 | hexdump(f, (const uint8_t *)s, len); 278 | } 279 | 280 | static struct type_info_row rows[] = { 281 | { 282 | .key = THEFT_BUILTIN_bool, 283 | .value = { 284 | .alloc = bool_alloc, 285 | .free = theft_generic_free_cb, 286 | .print = bool_print, 287 | .autoshrink_config = { 288 | .enable = true, 289 | }, 290 | }, 291 | }, 292 | SCALAR_ROW(uint), 293 | SCALAR_ROW(uint8_t), 294 | SCALAR_ROW(uint16_t), 295 | SCALAR_ROW(uint32_t), 296 | SCALAR_ROW(uint64_t), 297 | SCALAR_ROW(size_t), 298 | 299 | SCALAR_ROW(int), 300 | SCALAR_ROW(int8_t), 301 | SCALAR_ROW(int16_t), 302 | SCALAR_ROW(int32_t), 303 | SCALAR_ROW(int64_t), 304 | 305 | #if THEFT_USE_FLOATING_POINT 306 | SCALAR_ROW(float), 307 | SCALAR_ROW(double), 308 | #endif 309 | 310 | { 311 | .key = THEFT_BUILTIN_char_ARRAY, 312 | .value = { 313 | .alloc = char_ARRAY_alloc, 314 | .free = theft_generic_free_cb, 315 | .print = char_ARRAY_print, 316 | .autoshrink_config = { 317 | .enable = true, 318 | }, 319 | }, 320 | }, 321 | /* This is actually the same implementation, but 322 | * the user should cast it differently. */ 323 | { 324 | .key = THEFT_BUILTIN_uint8_t_ARRAY, 325 | .value = { 326 | .alloc = char_ARRAY_alloc, 327 | .free = theft_generic_free_cb, 328 | .print = char_ARRAY_print, 329 | .autoshrink_config = { 330 | .enable = true, 331 | }, 332 | }, 333 | }, 334 | }; 335 | 336 | const struct theft_type_info * 337 | theft_get_builtin_type_info(enum theft_builtin_type_info type) { 338 | for (size_t i = 0; i < sizeof(rows)/sizeof(rows[0]); i++) { 339 | const struct type_info_row *row = &rows[i]; 340 | if (row->key == type) { 341 | return &row->value; 342 | } 343 | } 344 | assert(false); 345 | return NULL; 346 | } 347 | 348 | void 349 | theft_copy_builtin_type_info(enum theft_builtin_type_info type, 350 | struct theft_type_info *info) { 351 | const struct theft_type_info *builtin = theft_get_builtin_type_info(type); 352 | memcpy(info, builtin, sizeof(*builtin)); 353 | } 354 | -------------------------------------------------------------------------------- /src/theft_bloom.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | #include "theft.h" 6 | #include "theft_bloom.h" 7 | #include "theft_types_internal.h" 8 | 9 | /* This is a dynamic blocked bloom filter, loosely based on 10 | * _Cache Efficient Bloom Filters for Shared Memory Machines_ 11 | * by Tim Kaler. 12 | * 13 | * The top level of the bloom filter uses the first N bits of the hash 14 | * (top_block2) to choose between (1 << N) distinct bloom filter blocks. 15 | * These blocks are created as necessary, i.e., a NULL block means no 16 | * bits in that block would have been set. 17 | * 18 | * When checking for matches, HASH_COUNT different chunks of M bits from 19 | * the hash are used to check each block's bloom filter. (M is 20 | * block->size2, and the bloom filter has (1 << M) bits.) If any of 21 | * the selected bits are false, there was no match. Every bloom filter 22 | * in the block's linked list is checked, so all must match for 23 | * `theft_bloom_check` to return true. 24 | * 25 | * When marking, only the front (largest) bloom filter in the 26 | * appropriate block is updated. If marking did not change any 27 | * bits (all bits chosen by the hash were already set), then 28 | * the bloom filter is considered too full, and a new one is 29 | * inserted before it, as the new head of the block. The new 30 | * bloom filter's size2 is one larger, so more bits of the hash 31 | * are used, and the bloom filter doubles in size. */ 32 | 33 | /* Default number of bits to use for choosing a specific 34 | * block (linked list of bloom filters) */ 35 | #define DEF_TOP_BLOCK_BITS 9 36 | 37 | /* Default number of bits in each first-layer bloom filter */ 38 | #define DEF_MIN_FILTER_BITS 9 39 | 40 | /* How many hashes to check for each block */ 41 | #define HASH_COUNT 4 42 | 43 | #define LOG_BLOOM 0 44 | 45 | struct bloom_filter { 46 | struct bloom_filter *next; 47 | uint8_t size2; /* log2 of bit count */ 48 | uint8_t bits[]; 49 | }; 50 | 51 | struct theft_bloom { 52 | const uint8_t top_block2; 53 | const uint8_t min_filter2; 54 | /* These start as NULL and are lazily allocated. 55 | * Each block is a linked list of bloom filters, with successively 56 | * larger filters appended at the front as the filters fill up. */ 57 | struct bloom_filter *blocks[]; 58 | }; 59 | 60 | static struct theft_bloom_config def_config = { .top_block_bits = 0 }; 61 | 62 | /* Initialize a dynamic blocked bloom filter. */ 63 | struct theft_bloom *theft_bloom_init(const struct theft_bloom_config *config) { 64 | #define DEF(X, DEFAULT) (X ? X : DEFAULT) 65 | config = DEF(config, &def_config); 66 | const uint8_t top_block2 = DEF(config->top_block_bits, DEF_TOP_BLOCK_BITS); 67 | const uint8_t min_filter2 = DEF(config->min_filter_bits, DEF_MIN_FILTER_BITS); 68 | #undef DEF 69 | 70 | const size_t top_block_count = (1LLU << top_block2); 71 | const size_t alloc_size = sizeof(struct theft_bloom) + 72 | top_block_count * sizeof(struct bloom_filter *); 73 | 74 | struct theft_bloom *res = malloc(alloc_size); 75 | if (res == NULL) { 76 | return NULL; 77 | } 78 | memset(&res->blocks, 0x00, top_block_count * sizeof(struct bloom_filter *)); 79 | 80 | struct theft_bloom b = { 81 | .top_block2 = top_block2, 82 | .min_filter2 = min_filter2, 83 | }; 84 | memcpy(res, &b, sizeof(b)); 85 | return res; 86 | } 87 | 88 | static struct bloom_filter * 89 | alloc_filter(uint8_t bits) { 90 | const size_t alloc_size = sizeof(struct bloom_filter) 91 | + ((1LLU << bits) / 8); 92 | struct bloom_filter *bf = malloc(alloc_size); 93 | if (bf != NULL) { 94 | memset(bf, 0x00, alloc_size); 95 | bf->size2 = bits; 96 | LOG(4 - LOG_BLOOM, "%s: %p [size2 %u (%zd bytes)]\n", 97 | __func__, (void *)bf, bf->size2, (size_t)((1LLU << bf->size2) / 8)); 98 | } 99 | return bf; 100 | } 101 | 102 | /* Hash data and mark it in the bloom filter. */ 103 | bool theft_bloom_mark(struct theft_bloom *b, uint8_t *data, size_t data_size) { 104 | uint64_t hash = theft_hash_onepass(data, data_size); 105 | const size_t top_block_count = (1LLU << b->top_block2); 106 | LOG(3 - LOG_BLOOM, 107 | "%s: overall hash: 0x%016" PRIx64 "\n", __func__, hash); 108 | 109 | const uint64_t top_block_mask = top_block_count - 1; 110 | const size_t block_id = hash & top_block_mask; 111 | LOG(3 - LOG_BLOOM, "%s: block_id %zd\n", __func__, block_id); 112 | 113 | struct bloom_filter *bf = b->blocks[block_id]; 114 | if (bf == NULL) { /* lazily allocate */ 115 | bf = alloc_filter(b->min_filter2); 116 | if (bf == NULL) { 117 | return false; /* alloc fail */ 118 | } 119 | b->blocks[block_id] = bf; 120 | } 121 | 122 | /* Must be able to do all checks with one 64 bit hash. 123 | * In order to relax this restriction, theft's hashing 124 | * code will need to be restructured to give the bloom 125 | * filter code two independent hashes. */ 126 | assert(64 - b->top_block2 - (HASH_COUNT * bf->size2) > 0); 127 | hash >>= b->top_block2; 128 | 129 | const uint8_t block_size2 = bf->size2; 130 | const uint64_t block_mask = (1LLU << block_size2) - 1; 131 | bool any_set = false; 132 | 133 | /* Only mark in the front filter. */ 134 | for (size_t i = 0; i < HASH_COUNT; i++) { 135 | const uint64_t v = (hash >> (i*block_size2)) & block_mask; 136 | const uint64_t offset = v / 8; 137 | const uint8_t bit = 1 << (v & 0x07); 138 | LOG(4 - LOG_BLOOM, 139 | "%s: marking %p @ %" PRIu64 " => offset %" PRIu64 140 | ", bit 0x%02x\n", 141 | __func__, (void *)bf, v, offset, bit); 142 | if (0 == (bf->bits[offset] & bit)) { 143 | any_set = true; 144 | } 145 | bf->bits[offset] |= bit; 146 | } 147 | 148 | /* If all bits were already set, prepend a new, empty filter -- the 149 | * previous filter will still match when checking, but there will be 150 | * a reduced chance of false positives for new entries. */ 151 | if (!any_set) { 152 | if (b->top_block2 + HASH_COUNT * (bf->size2 + 1) > 64) { 153 | /* We can't grow this hash chain any further with the 154 | * hash bits available. */ 155 | LOG(0, "%s: Warning: bloom filter block %zd cannot grow further!\n", 156 | __func__, block_id); 157 | } else { 158 | struct bloom_filter *nbf = alloc_filter(bf->size2 + 1); 159 | LOG(3 - LOG_BLOOM, "%s: growing bloom filter -- bits %u, nbf %p\n", 160 | __func__, bf->size2 + 1, (void *)nbf); 161 | if (nbf == NULL) { 162 | return false; /* alloc fail */ 163 | } 164 | nbf->next = bf; 165 | b->blocks[block_id] = nbf; /* append to front */ 166 | } 167 | } 168 | 169 | return true; 170 | } 171 | 172 | /* Check whether the data's hash is in the bloom filter. */ 173 | bool theft_bloom_check(struct theft_bloom *b, uint8_t *data, size_t data_size) { 174 | uint64_t hash = theft_hash_onepass(data, data_size); 175 | LOG(3 - LOG_BLOOM, 176 | "%s: overall hash: 0x%016" PRIx64 "\n", __func__, hash); 177 | const size_t top_block_count = (1LLU << b->top_block2); 178 | const uint64_t top_block_mask = top_block_count - 1; 179 | const size_t block_id = hash & top_block_mask; 180 | LOG(3 - LOG_BLOOM, "%s: block_id %zd\n", __func__, block_id); 181 | 182 | struct bloom_filter *bf = b->blocks[block_id]; 183 | if (bf == NULL) { 184 | return false; /* block not allocated: no bits set */ 185 | } 186 | 187 | hash >>= b->top_block2; 188 | 189 | /* Check every block */ 190 | while (bf != NULL) { 191 | const uint8_t block_size2 = bf->size2; 192 | const uint64_t block_mask = (1LLU << block_size2) - 1; 193 | 194 | bool hit_all_in_block = true; 195 | for (size_t i = 0; i < HASH_COUNT; i++) { 196 | const uint64_t v = (hash >> (i * block_size2)) & block_mask; 197 | const uint64_t offset = v / 8; 198 | const uint8_t bit = 1 << (v & 0x07); 199 | LOG(4 - LOG_BLOOM, 200 | "%s: checking %p (bits %u) @ %" PRIu64 " => offset %" PRIu64 201 | ", bit 0x%02x: 0x%02x\n", 202 | __func__, (void *)bf, block_size2, v, offset, bit, 203 | (bf->bits[offset] & bit)); 204 | if (0 == (bf->bits[offset] & bit)) { 205 | hit_all_in_block = false; 206 | break; 207 | } 208 | } 209 | if (hit_all_in_block) { 210 | return true; 211 | } 212 | bf = bf->next; 213 | } 214 | 215 | return false; /* there wasn't any block with all checked bits set */ 216 | } 217 | 218 | /* Free the bloom filter. */ 219 | void theft_bloom_free(struct theft_bloom *b) { 220 | const size_t top_block_count = (1LLU << b->top_block2); 221 | uint8_t max_length = 0; 222 | for (size_t i = 0; i < top_block_count; i++) { 223 | uint8_t length = 0; 224 | struct bloom_filter *bf = b->blocks[i]; 225 | while (bf != NULL) { 226 | struct bloom_filter *next = bf->next; 227 | free(bf); 228 | bf = next; 229 | length++; 230 | } 231 | LOG(3 - LOG_BLOOM, 232 | "%s: block %zd, length %u\n", __func__, i, length); 233 | max_length = (length > max_length ? length : max_length); 234 | } 235 | LOG(3 - LOG_BLOOM, 236 | "%s: %zd blocks, max length %u\n", __func__, top_block_count, max_length); 237 | free(b); 238 | } 239 | -------------------------------------------------------------------------------- /src/theft_bloom.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_BLOOM_H 2 | #define THEFT_BLOOM_H 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | /* Opaque type for bloom filter. */ 9 | struct theft_bloom; 10 | 11 | struct theft_bloom_config { 12 | uint8_t top_block_bits; 13 | uint8_t min_filter_bits; 14 | }; 15 | 16 | /* Initialize a bloom filter. */ 17 | struct theft_bloom *theft_bloom_init(const struct theft_bloom_config *config); 18 | 19 | /* Hash data and mark it in the bloom filter. */ 20 | bool theft_bloom_mark(struct theft_bloom *b, uint8_t *data, size_t data_size); 21 | 22 | /* Check whether the data's hash is in the bloom filter. */ 23 | bool theft_bloom_check(struct theft_bloom *b, uint8_t *data, size_t data_size); 24 | 25 | /* Free the bloom filter. */ 26 | void theft_bloom_free(struct theft_bloom *b); 27 | 28 | #endif 29 | -------------------------------------------------------------------------------- /src/theft_call.c: -------------------------------------------------------------------------------- 1 | #include "theft_call_internal.h" 2 | #include "theft_autoshrink.h" 3 | 4 | #include 5 | #include 6 | 7 | #define LOG_CALL 0 8 | 9 | #define MAX_FORK_RETRIES 10 10 | #define DEF_KILL_SIGNAL SIGTERM 11 | 12 | /* Actually call the property function. Its number of arguments is not 13 | * constrained by the typedef, but will be defined at the call site 14 | * here. (If info->arity is wrong, it will probably crash.) */ 15 | enum theft_trial_res 16 | theft_call(struct theft *t, void **args) { 17 | enum theft_trial_res res = THEFT_TRIAL_ERROR; 18 | 19 | if (t->fork.enable) { 20 | struct timespec tv = { .tv_nsec = 1 }; 21 | if (-1 == pipe(t->workers[0].fds)) { return THEFT_TRIAL_ERROR; } 22 | 23 | pid_t pid = -1; 24 | for (;;) { 25 | pid = fork(); 26 | if (pid == -1) { 27 | if (errno == EAGAIN) { 28 | /* If we get EAGAIN, then wait for terminated 29 | * child processes a chance to clean up -- forking 30 | * is probably failing due to RLIMIT_NPROC. */ 31 | const int fork_errno = errno; 32 | if (!step_waitpid(t)) { return THEFT_TRIAL_ERROR; } 33 | if (-1 == nanosleep(&tv, NULL)) { 34 | perror("nanosleep"); 35 | return THEFT_TRIAL_ERROR; 36 | } 37 | if (tv.tv_nsec >= (1L << MAX_FORK_RETRIES)) { 38 | errno = fork_errno; 39 | perror("fork"); 40 | return THEFT_TRIAL_ERROR; 41 | } 42 | errno = 0; 43 | tv.tv_nsec <<= 1; 44 | continue; 45 | } else { 46 | perror("fork"); 47 | return THEFT_TRIAL_ERROR; 48 | } 49 | } else { 50 | break; 51 | } 52 | } 53 | 54 | if (pid == -1) { 55 | close(t->workers[0].fds[0]); 56 | close(t->workers[0].fds[1]); 57 | return THEFT_TRIAL_ERROR; 58 | } else if (pid == 0) { /* child */ 59 | close(t->workers[0].fds[0]); 60 | int out_fd = t->workers[0].fds[1]; 61 | if (run_fork_post_hook(t, args) == THEFT_HOOK_FORK_POST_ERROR) { 62 | uint8_t byte = (uint8_t)THEFT_TRIAL_ERROR; 63 | ssize_t wr = write(out_fd, (const void *)&byte, sizeof(byte)); 64 | (void)wr; 65 | exit(EXIT_FAILURE); 66 | } 67 | res = theft_call_inner(t, args); 68 | uint8_t byte = (uint8_t)res; 69 | ssize_t wr = write(out_fd, (const void *)&byte, sizeof(byte)); 70 | exit(wr == 1 && res == THEFT_TRIAL_PASS 71 | ? EXIT_SUCCESS 72 | : EXIT_FAILURE); 73 | } else { /* parent */ 74 | close(t->workers[0].fds[1]); 75 | t->workers[0].pid = pid; 76 | 77 | t->workers[0].state = WS_ACTIVE; 78 | res = parent_handle_child_call(t, pid, &t->workers[0]); 79 | close(t->workers[0].fds[0]); 80 | t->workers[0].state = WS_INACTIVE; 81 | 82 | if (!step_waitpid(t)) { return THEFT_TRIAL_ERROR; } 83 | return res; 84 | } 85 | } else { /* just call */ 86 | res = theft_call_inner(t, args); 87 | } 88 | return res; 89 | } 90 | 91 | static enum theft_trial_res 92 | parent_handle_child_call(struct theft *t, pid_t pid, struct worker_info *worker) { 93 | const int fd = worker->fds[0]; 94 | struct pollfd pfd[1] = { 95 | { .fd = fd, .events = POLLIN }, 96 | }; 97 | const int timeout = t->fork.timeout; 98 | int res = 0; 99 | for (;;) { 100 | struct timeval tv_pre = { 0, 0 }; 101 | gettimeofday(&tv_pre, NULL); 102 | res = poll(pfd, 1, (timeout == 0 ? -1 : timeout)); 103 | struct timeval tv_post = { 0, 0 }; 104 | gettimeofday(&tv_post, NULL); 105 | 106 | const size_t delta = 1000*tv_post.tv_sec - 1000*tv_pre.tv_sec + 107 | ((tv_post.tv_usec / 1000) - (tv_pre.tv_usec / 1000)); 108 | LOG(3 - LOG_CALL,"%s: POLL res %d, elapsed %zd\n", 109 | __func__, res, delta); 110 | (void)delta; 111 | 112 | if (res == -1) { 113 | if (errno == EAGAIN) { 114 | errno = 0; 115 | continue; 116 | } else if (errno == EINTR) { 117 | errno = 0; 118 | continue; 119 | } else { 120 | return THEFT_TRIAL_ERROR; 121 | } 122 | } else { 123 | break; 124 | } 125 | } 126 | 127 | if (res == 0) { /* timeout */ 128 | int kill_signal = t->fork.signal; 129 | if (kill_signal == 0) { 130 | kill_signal = DEF_KILL_SIGNAL; 131 | } 132 | LOG(2 - LOG_CALL, "%s: kill(%d, %d)\n", 133 | __func__, pid, kill_signal); 134 | assert(pid != -1); /* do not do this. */ 135 | if (-1 == kill(pid, kill_signal)) { 136 | return THEFT_TRIAL_ERROR; 137 | } 138 | 139 | /* Check if kill's signal made the child process terminate (or 140 | * if it exited successfully, and there was just a race on the 141 | * timeout). If so, save its exit status. 142 | * 143 | * If it still hasn't exited after the exit_timeout, then 144 | * send it SIGKILL and wait for _that_ to make it exit. */ 145 | const size_t kill_time = 10; /* time to exit after SIGKILL */ 146 | const size_t timeout_msec = (t->fork.exit_timeout == 0 147 | ? THEFT_DEF_EXIT_TIMEOUT_MSEC 148 | : t->fork.exit_timeout); 149 | 150 | /* After sending the signal to the timed out process, 151 | * give it timeout_msec to actually exit (in case a custom 152 | * signal is triggering some sort of cleanup) before sending 153 | * SIGKILL and waiting up to kill_time it to change state. */ 154 | if (!wait_for_exit(t, worker, timeout_msec, kill_time)) { 155 | return THEFT_TRIAL_ERROR; 156 | } 157 | 158 | /* If the child still exited successfully, then consider it a 159 | * PASS, even though it exceeded the timeout. */ 160 | if (worker->state == WS_STOPPED) { 161 | const int st = worker->wstatus; 162 | LOG(2 - LOG_CALL, "exited? %d, exit_status %d\n", 163 | WIFEXITED(st), WEXITSTATUS(st)); 164 | if (WIFEXITED(st) && WEXITSTATUS(st) == EXIT_SUCCESS) { 165 | return THEFT_TRIAL_PASS; 166 | } 167 | } 168 | 169 | return THEFT_TRIAL_FAIL; 170 | } else { 171 | /* As long as the result isn't a timeout, the worker can 172 | * just be cleaned up by the next batch of waitpid()s. */ 173 | enum theft_trial_res trial_res = THEFT_TRIAL_ERROR; 174 | uint8_t res_byte = 0xFF; 175 | ssize_t rd = 0; 176 | for (;;) { 177 | rd = read(fd, &res_byte, sizeof(res_byte)); 178 | if (rd == -1) { 179 | if (errno == EINTR) { 180 | errno = 0; 181 | continue; 182 | } 183 | return THEFT_TRIAL_ERROR; 184 | } else { 185 | break; 186 | } 187 | } 188 | 189 | if (rd == 0) { 190 | /* closed without response -> crashed */ 191 | trial_res = THEFT_TRIAL_FAIL; 192 | } else { 193 | assert(rd == 1); 194 | trial_res = (enum theft_trial_res)res_byte; 195 | } 196 | 197 | return trial_res; 198 | } 199 | } 200 | 201 | /* Clean up after all child processes that have changed state. 202 | * Save the exit/termination status for worker processes. */ 203 | static bool 204 | step_waitpid(struct theft *t) { 205 | int wstatus = 0; 206 | int old_errno = errno; 207 | for (;;) { 208 | errno = 0; 209 | pid_t res = waitpid(-1, &wstatus, WNOHANG); 210 | LOG(2 - LOG_CALL, "%s: waitpid? %d\n", __func__, res); 211 | if (res == -1) { 212 | if (errno == ECHILD) { break; } /* No Children */ 213 | perror("waitpid"); 214 | return THEFT_TRIAL_ERROR; 215 | } else if (res == 0) { 216 | break; /* no children have changed state */ 217 | } else { 218 | if (res == t->workers[0].pid) { 219 | t->workers[0].state = WS_STOPPED; 220 | t->workers[0].wstatus = wstatus; 221 | } 222 | } 223 | } 224 | errno = old_errno; 225 | return true; 226 | } 227 | 228 | /* Wait timeout msec. for the worker to exit. If kill_timeout is 229 | * non-zero, then send SIGKILL and wait that much longer. */ 230 | static bool 231 | wait_for_exit(struct theft *t, struct worker_info *worker, 232 | size_t timeout, size_t kill_timeout) { 233 | for (size_t i = 0; i < timeout + kill_timeout; i++) { 234 | if (!step_waitpid(t)) { return false; } 235 | if (worker->state == WS_STOPPED) { break; } 236 | 237 | /* If worker hasn't exited yet and kill_timeout is 238 | * non-zero, send SIGKILL. */ 239 | if (i == timeout) { 240 | assert(kill_timeout > 0); 241 | assert(worker->pid != -1); 242 | int kill_res = kill(worker->pid, SIGKILL); 243 | if (kill_res == -1) { 244 | if (kill_res == ESRCH) { 245 | /* Process no longer exists (it probably 246 | * just exited); let waitpid handle it. */ 247 | } else { 248 | perror("kill"); 249 | return false; 250 | } 251 | } 252 | } 253 | 254 | const struct timespec one_msec = { .tv_nsec = 1000000 }; 255 | if (-1 == nanosleep(&one_msec, NULL)) { 256 | perror("nanosleep"); 257 | return false; 258 | } 259 | } 260 | return true; 261 | } 262 | 263 | static enum theft_trial_res 264 | theft_call_inner(struct theft *t, void **args) { 265 | switch (t->prop.arity) { 266 | case 1: 267 | return t->prop.u.fun1(t, args[0]); 268 | break; 269 | case 2: 270 | return t->prop.u.fun2(t, args[0], args[1]); 271 | break; 272 | case 3: 273 | return t->prop.u.fun3(t, args[0], args[1], args[2]); 274 | break; 275 | case 4: 276 | return t->prop.u.fun4(t, args[0], args[1], args[2], args[3]); 277 | break; 278 | case 5: 279 | return t->prop.u.fun5(t, args[0], args[1], args[2], args[3], args[4]); 280 | break; 281 | case 6: 282 | return t->prop.u.fun6(t, args[0], args[1], args[2], args[3], args[4], 283 | args[5]); 284 | break; 285 | case 7: 286 | return t->prop.u.fun7(t, args[0], args[1], args[2], args[3], args[4], 287 | args[5], args[6]); 288 | break; 289 | /* ... */ 290 | default: 291 | return THEFT_TRIAL_ERROR; 292 | } 293 | } 294 | 295 | /* Populate a buffer with hashes of all the arguments. */ 296 | static void 297 | get_arg_hash_buffer(theft_hash *buffer, struct theft *t) { 298 | for (uint8_t i = 0; i < t->prop.arity; i++) { 299 | struct theft_type_info *ti = t->prop.type_info[i]; 300 | 301 | theft_hash h = (ti->autoshrink_config.enable 302 | ? theft_autoshrink_hash(t, t->trial.args[i].instance, 303 | t->trial.args[i].u.as.env, ti->env) 304 | : ti->hash(t->trial.args[i].instance, ti->env)); 305 | 306 | LOG(4, "%s: arg %d hash; 0x%016" PRIx64 "\n", __func__, i, h); 307 | buffer[i] = h; 308 | } 309 | } 310 | 311 | /* Check if this combination of argument instances has been called. */ 312 | bool theft_call_check_called(struct theft *t) { 313 | theft_hash buffer[THEFT_MAX_ARITY]; 314 | get_arg_hash_buffer(buffer, t); 315 | return theft_bloom_check(t->bloom, (uint8_t *)buffer, 316 | t->prop.arity * sizeof(theft_hash)); 317 | } 318 | 319 | /* Mark the tuple of argument instances as called in the bloom filter. */ 320 | void theft_call_mark_called(struct theft *t) { 321 | theft_hash buffer[THEFT_MAX_ARITY]; 322 | get_arg_hash_buffer(buffer, t); 323 | theft_bloom_mark(t->bloom, (uint8_t *)buffer, 324 | t->prop.arity * sizeof(theft_hash)); 325 | } 326 | 327 | static enum theft_hook_fork_post_res 328 | run_fork_post_hook(struct theft *t, void **args) { 329 | if (t->hooks.fork_post == NULL) { 330 | return THEFT_HOOK_FORK_POST_CONTINUE; 331 | } 332 | 333 | struct theft_hook_fork_post_info info = { 334 | .prop_name = t->prop.name, 335 | .total_trials = t->prop.trial_count, 336 | .failures = t->counters.fail, 337 | .run_seed = t->seeds.run_seed, 338 | .arity = t->prop.arity, 339 | .args = args, /* real_args */ 340 | }; 341 | return t->hooks.fork_post(&info, t->hooks.env); 342 | } 343 | -------------------------------------------------------------------------------- /src/theft_call.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_CALL_H 2 | #define THEFT_CALL_H 3 | 4 | #include "theft_types_internal.h" 5 | 6 | /* Actually call the property function referenced in INFO, 7 | * with the arguments in ARGS. */ 8 | enum theft_trial_res 9 | theft_call(struct theft *t, void **args); 10 | 11 | /* Check if this combination of argument instances has been called. */ 12 | bool theft_call_check_called(struct theft *t); 13 | 14 | /* Mark the tuple of argument instances as called in the bloom filter. */ 15 | void theft_call_mark_called(struct theft *t); 16 | 17 | 18 | #endif 19 | -------------------------------------------------------------------------------- /src/theft_call_internal.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_CALL_INTERNAL_H 2 | #define THEFT_CALL_INTERNAL_H 3 | 4 | #include "theft_call.h" 5 | #include "theft_bloom.h" 6 | #include 7 | 8 | #include 9 | #include 10 | #include 11 | #include 12 | #include 13 | 14 | static enum theft_trial_res 15 | theft_call_inner(struct theft *t, void **args); 16 | 17 | static enum theft_trial_res 18 | parent_handle_child_call(struct theft *t, pid_t pid, 19 | struct worker_info *worker); 20 | 21 | static enum theft_hook_fork_post_res 22 | run_fork_post_hook(struct theft *t, void **args); 23 | 24 | static bool 25 | step_waitpid(struct theft *t); 26 | 27 | static bool 28 | wait_for_exit(struct theft *t, struct worker_info *worker, 29 | size_t timeout, size_t kill_timeout); 30 | 31 | #endif 32 | -------------------------------------------------------------------------------- /src/theft_hash.c: -------------------------------------------------------------------------------- 1 | #include "theft.h" 2 | 3 | #include 4 | 5 | /* Fowler/Noll/Vo hash, 64-bit FNV-1a. 6 | * This hashing algorithm is in the public domain. 7 | * For more details, see: http://www.isthe.com/chongo/tech/comp/fnv/. */ 8 | static const uint64_t fnv64_prime = 1099511628211L; 9 | static const uint64_t fnv64_offset_basis = 14695981039346656037UL; 10 | 11 | /* Initialize a hasher for incremental hashing. */ 12 | void theft_hash_init(struct theft_hasher *h) { 13 | assert(h); 14 | h->accum = fnv64_offset_basis; 15 | } 16 | 17 | /* Sink more data into an incremental hash. */ 18 | void theft_hash_sink(struct theft_hasher *h, 19 | const uint8_t *data, size_t bytes) { 20 | assert(h); 21 | assert(data); 22 | if (h == NULL || data == NULL) { return; } 23 | uint64_t a = h->accum; 24 | for (size_t i = 0; i < bytes; i++) { 25 | a = (a ^ data[i]) * fnv64_prime; 26 | } 27 | h->accum = a; 28 | } 29 | 30 | /* Finish hashing and get the result. */ 31 | theft_hash theft_hash_done(struct theft_hasher *h) { 32 | assert(h); 33 | theft_hash res = h->accum; 34 | theft_hash_init(h); /* reset */ 35 | return res; 36 | } 37 | 38 | /* Hash a buffer in one pass. (Wraps the above functions.) */ 39 | theft_hash theft_hash_onepass(const uint8_t *data, size_t bytes) { 40 | assert(data); 41 | struct theft_hasher h; 42 | theft_hash_init(&h); 43 | theft_hash_sink(&h, data, bytes); 44 | return theft_hash_done(&h); 45 | } 46 | -------------------------------------------------------------------------------- /src/theft_random.c: -------------------------------------------------------------------------------- 1 | #include "theft_random.h" 2 | 3 | #include "theft_types_internal.h" 4 | #include "theft_rng.h" 5 | 6 | #include 7 | #include 8 | 9 | static uint64_t get_mask(uint8_t bits); 10 | 11 | /* (Re-)initialize the random number generator with a specific seed. 12 | * This stops using the current bit pool. */ 13 | void theft_random_set_seed(struct theft *t, uint64_t seed) { 14 | theft_random_stop_using_bit_pool(t); 15 | t->prng.buf = 0; 16 | t->prng.bits_available = 0; 17 | 18 | theft_rng_reset(t->prng.rng, seed); 19 | LOG(2, "%s: SET_SEED: %" PRIx64 "\n", __func__, seed); 20 | } 21 | 22 | void theft_random_inject_autoshrink_bit_pool(struct theft *t, 23 | struct autoshrink_bit_pool *bit_pool) { 24 | t->prng.bit_pool = bit_pool; 25 | } 26 | 27 | void theft_random_stop_using_bit_pool(struct theft *t) { 28 | t->prng.bit_pool = NULL; 29 | } 30 | 31 | /* Get BITS random bits from the test runner's PRNG. 32 | * Bits can be retrieved at most 64 at a time. */ 33 | uint64_t theft_random_bits(struct theft *t, uint8_t bit_count) { 34 | assert(bit_count <= 64); 35 | LOG(4, "RANDOM_BITS: available %u, bit_count: %u, buf %016" PRIx64 "\n", 36 | t->prng.bits_available, bit_count, t->prng.buf); 37 | 38 | uint64_t res = 0; 39 | theft_random_bits_bulk(t, bit_count, &res); 40 | return res; 41 | 42 | } 43 | 44 | void theft_random_bits_bulk(struct theft *t, uint32_t bit_count, uint64_t *buf) { 45 | LOG(5, "%s: bit_count %u\n", __func__, bit_count); 46 | assert(buf); 47 | if (t->prng.bit_pool) { 48 | theft_autoshrink_bit_pool_random(t, t->prng.bit_pool, bit_count, true, buf); 49 | return; 50 | } 51 | 52 | uint32_t rem = bit_count; 53 | uint8_t shift = 0; 54 | size_t offset = 0; 55 | 56 | while (rem > 0) { 57 | if (t->prng.bits_available == 0) { 58 | t->prng.buf = theft_rng_random(t->prng.rng); 59 | t->prng.bits_available = 64; 60 | } 61 | LOG(5, "%% buf 0x%016" PRIx64 "\n", t->prng.buf); 62 | 63 | uint8_t take = 64 - shift; 64 | if (take > rem) { 65 | take = rem; 66 | } 67 | if (take > t->prng.bits_available) { 68 | take = t->prng.bits_available; 69 | } 70 | 71 | LOG(5, "%s: rem %u, available %u, buf 0x%016" PRIx64 ", offset %zd, take %u\n", 72 | __func__, rem, t->prng.bits_available, t->prng.buf, offset, take); 73 | 74 | const uint64_t mask = get_mask(take); 75 | buf[offset] |= (t->prng.buf & mask) << shift; 76 | LOG(5, "== buf[%zd]: %016" PRIx64 " (%u / %u)\n", 77 | offset, buf[offset], bit_count - rem, bit_count); 78 | t->prng.bits_available -= take; 79 | t->prng.buf >>= take; 80 | 81 | shift += take; 82 | if (shift == 64) { 83 | offset++; 84 | shift = 0; 85 | } 86 | 87 | rem -= take; 88 | } 89 | } 90 | 91 | /* Get a random 64-bit integer from the test runner's PRNG. 92 | * 93 | * NOTE: This is equivalent to `theft_random_bits(t, 64)`, and 94 | * will be removed in a future release. */ 95 | theft_seed theft_random(struct theft *t) { 96 | return theft_random_bits(t, 8*sizeof(uint64_t)); 97 | } 98 | 99 | #if THEFT_USE_FLOATING_POINT 100 | /* Get a random double from the test runner's PRNG. */ 101 | double theft_random_double(struct theft *t) { 102 | double res = theft_rng_uint64_to_double(theft_random_bits(t, 64)); 103 | LOG(4, "RANDOM_DOUBLE: %g\n", res); 104 | return res; 105 | } 106 | 107 | uint64_t theft_random_choice(struct theft *t, uint64_t ceil) { 108 | if (ceil < 2) { return 0; } 109 | uint64_t bits; 110 | double limit; 111 | 112 | /* If ceil is a power of two, just return that many bits. */ 113 | if ((ceil & (ceil - 1)) == 0) { 114 | uint8_t log2_ceil = 1; 115 | while (ceil > (1LLU << log2_ceil)) { 116 | log2_ceil++; 117 | } 118 | assert((1LLU << log2_ceil) == ceil); 119 | return theft_random_bits(t, log2_ceil); 120 | } 121 | 122 | /* If the choice values are fairly small (which shoud be 123 | * the common case), sample less than 64 bits to reduce 124 | * time spent managing the random bitstream. */ 125 | if (ceil < UINT8_MAX) { 126 | bits = theft_random_bits(t, 16); 127 | limit = (double)(1LLU << 16); 128 | } else if (ceil < UINT16_MAX) { 129 | bits = theft_random_bits(t, 32); 130 | limit = (double)(1LLU << 32); 131 | } else { 132 | bits = theft_random_bits(t, 64); 133 | limit = (double)UINT64_MAX; 134 | } 135 | 136 | double mul = (double)bits / limit; 137 | uint64_t res = (uint64_t)(mul * ceil); 138 | return res; 139 | } 140 | #endif 141 | 142 | static uint64_t get_mask(uint8_t bits) { 143 | if (bits == 64) { 144 | return ~(uint64_t)0; // just set all bits -- would overflow 145 | } else { 146 | return (1LLU << bits) - 1; 147 | } 148 | } 149 | -------------------------------------------------------------------------------- /src/theft_random.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_RANDOM_H 2 | #define THEFT_RANDOM_H 3 | 4 | #include "theft.h" 5 | #include "theft_autoshrink.h" 6 | 7 | /* Inject a bit pool for autoshrinking -- Get the random bit stream from 8 | * it, rather than the PRNG, because we'll shrink by shrinking the bit 9 | * pool itself. */ 10 | void theft_random_inject_autoshrink_bit_pool(struct theft *t, 11 | struct autoshrink_bit_pool *bitpool); 12 | 13 | /* Stop using an autoshrink bit pool. 14 | * (Re-seeding the PRNG will also do this.) */ 15 | void theft_random_stop_using_bit_pool(struct theft *t); 16 | 17 | /* (Re-)initialize the random number generator with a specific seed. 18 | * This stops using the current bit pool. */ 19 | void theft_random_set_seed(struct theft *t, uint64_t seed); 20 | 21 | #endif 22 | -------------------------------------------------------------------------------- /src/theft_rng.c: -------------------------------------------------------------------------------- 1 | /* 2 | A C-program for MT19937-64 (2004/9/29 version). 3 | Coded by Takuji Nishimura and Makoto Matsumoto. 4 | 5 | This is a 64-bit version of Mersenne Twister pseudorandom number 6 | generator. 7 | 8 | Before using, initialize the state by using init_genrand64(seed) 9 | or init_by_array64(init_key, key_length). 10 | 11 | Copyright (C) 2004, Makoto Matsumoto and Takuji Nishimura, 12 | All rights reserved. 13 | 14 | Redistribution and use in source and binary forms, with or without 15 | modification, are permitted provided that the following conditions 16 | are met: 17 | 18 | 1. Redistributions of source code must retain the above copyright 19 | notice, this list of conditions and the following disclaimer. 20 | 21 | 2. Redistributions in binary form must reproduce the above copyright 22 | notice, this list of conditions and the following disclaimer in the 23 | documentation and/or other materials provided with the distribution. 24 | 25 | 3. The names of its contributors may not be used to endorse or promote 26 | products derived from this software without specific prior written 27 | permission. 28 | 29 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 30 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 31 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 32 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR 33 | CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 34 | EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, 35 | PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR 36 | PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 37 | LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 38 | NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 39 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 40 | 41 | References: 42 | T. Nishimura, ``Tables of 64-bit Mersenne Twisters'' 43 | ACM Transactions on Modeling and 44 | Computer Simulation 10. (2000) 348--357. 45 | M. Matsumoto and T. Nishimura, 46 | ``Mersenne Twister: a 623-dimensionally equidistributed 47 | uniform pseudorandom number generator'' 48 | ACM Transactions on Modeling and 49 | Computer Simulation 8. (Jan. 1998) 3--30. 50 | 51 | Any feedback is very welcome. 52 | http://www.math.hiroshima-u.ac.jp/~m-mat/MT/emt.html 53 | email: m-mat @ math.sci.hiroshima-u.ac.jp (remove spaces) 54 | */ 55 | 56 | /* The code has been modified to store internal state in heap/stack 57 | * allocated memory, rather than statically allocated memory, to allow 58 | * multiple instances running in the same address space. 59 | * 60 | * Also, the functions in the module's public interface have 61 | * been prefixed with "theft_rng_". */ 62 | 63 | #include 64 | #include 65 | #include "theft_rng.h" 66 | 67 | #define THEFT_MT_PARAM_N 312 68 | struct theft_rng { 69 | uint64_t mt[THEFT_MT_PARAM_N]; /* the array for the state vector */ 70 | int16_t mti; 71 | }; 72 | 73 | #define NN THEFT_MT_PARAM_N 74 | #define MM 156 75 | #define MATRIX_A 0xB5026F5AA96619E9ULL 76 | #define UM 0xFFFFFFFF80000000ULL /* Most significant 33 bits */ 77 | #define LM 0x7FFFFFFFULL /* Least significant 31 bits */ 78 | 79 | static uint64_t genrand64_int64(struct theft_rng *r); 80 | 81 | /* Heap-allocate a mersenne twister struct. */ 82 | struct theft_rng *theft_rng_init(uint64_t seed) { 83 | struct theft_rng *mt = malloc(sizeof(struct theft_rng)); 84 | if (mt == NULL) { return NULL; } 85 | theft_rng_reset(mt, seed); 86 | return mt; 87 | } 88 | 89 | /* Free a heap-allocated mersenne twister struct. */ 90 | void theft_rng_free(struct theft_rng *mt) { 91 | free(mt); 92 | } 93 | 94 | /* initializes mt[NN] with a seed */ 95 | void theft_rng_reset(struct theft_rng *mt, uint64_t seed) 96 | { 97 | mt->mt[0] = seed; 98 | uint16_t mti = 0; 99 | for (mti=1; mtimt[mti] = (6364136223846793005ULL * 101 | (mt->mt[mti-1] ^ (mt->mt[mti-1] >> 62)) + mti); 102 | } 103 | mt->mti = mti; 104 | } 105 | 106 | /* Get a 64-bit random number. */ 107 | uint64_t theft_rng_random(struct theft_rng *mt) { 108 | return genrand64_int64(mt); 109 | } 110 | 111 | /* Generate a random number on [0,1]-real-interval. */ 112 | double theft_rng_uint64_to_double(uint64_t x) { 113 | return (x >> 11) * (1.0/9007199254740991.0); 114 | } 115 | 116 | /* generates a random number on [0, 2^64-1]-interval */ 117 | static uint64_t genrand64_int64(struct theft_rng *r) 118 | { 119 | int i; 120 | uint64_t x; 121 | static uint64_t mag01[2]={0ULL, MATRIX_A}; 122 | 123 | if (r->mti >= NN) { /* generate NN words at one time */ 124 | 125 | /* if init has not been called, */ 126 | /* a default initial seed is used */ 127 | if (r->mti == NN+1) 128 | theft_rng_reset(r, 5489ULL); 129 | 130 | for (i=0;imt[i]&UM)|(r->mt[i+1]&LM); 132 | r->mt[i] = r->mt[i+MM] ^ (x>>1) ^ mag01[(int)(x&1ULL)]; 133 | } 134 | for (;imt[i]&UM)|(r->mt[i+1]&LM); 136 | r->mt[i] = r->mt[i+(MM-NN)] ^ (x>>1) ^ mag01[(int)(x&1ULL)]; 137 | } 138 | x = (r->mt[NN-1]&UM)|(r->mt[0]&LM); 139 | r->mt[NN-1] = r->mt[MM-1] ^ (x>>1) ^ mag01[(int)(x&1ULL)]; 140 | 141 | r->mti = 0; 142 | } 143 | 144 | x = r->mt[r->mti++]; 145 | 146 | x ^= (x >> 29) & 0x5555555555555555ULL; 147 | x ^= (x << 17) & 0x71D67FFFEDA60000ULL; 148 | x ^= (x << 37) & 0xFFF7EEE000000000ULL; 149 | x ^= (x >> 43); 150 | 151 | return x; 152 | } 153 | -------------------------------------------------------------------------------- /src/theft_rng.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_RNG_H 2 | #define THEFT_RNG_H 3 | 4 | #include 5 | 6 | /* Wrapper for Mersenne Twister. 7 | * See copyright and license in theft_rng.c, more details at: 8 | * http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html 9 | * 10 | * Local modifications are described in theft_mt.c. */ 11 | 12 | /* Opaque type for a Mersenne Twister PRNG. */ 13 | struct theft_rng; 14 | 15 | /* Heap-allocate a mersenne twister struct. */ 16 | struct theft_rng *theft_rng_init(uint64_t seed); 17 | 18 | /* Free a heap-allocated mersenne twister struct. */ 19 | void theft_rng_free(struct theft_rng *mt); 20 | 21 | /* Reset a mersenne twister struct, possibly stack-allocated. */ 22 | void theft_rng_reset(struct theft_rng *mt, uint64_t seed); 23 | 24 | /* Get a 64-bit random number. */ 25 | uint64_t theft_rng_random(struct theft_rng *mt); 26 | 27 | /* Convert a uint64_t to a number on the [0,1]-real-interval. */ 28 | double theft_rng_uint64_to_double(uint64_t x); 29 | 30 | #endif 31 | -------------------------------------------------------------------------------- /src/theft_run.c: -------------------------------------------------------------------------------- 1 | #include "theft_run_internal.h" 2 | 3 | #include "theft_bloom.h" 4 | #include "theft_rng.h" 5 | #include "theft_call.h" 6 | #include "theft_trial.h" 7 | #include "theft_random.h" 8 | #include "theft_autoshrink.h" 9 | 10 | #include 11 | #include 12 | 13 | #define LOG_RUN 0 14 | 15 | enum theft_run_init_res 16 | theft_run_init(const struct theft_run_config *cfg, struct theft **output) { 17 | enum theft_run_init_res res = THEFT_RUN_INIT_OK; 18 | struct theft *t = malloc(sizeof(*t)); 19 | if (t == NULL) { 20 | return THEFT_RUN_INIT_ERROR_MEMORY; 21 | } 22 | memset(t, 0, sizeof(*t)); 23 | 24 | t->out = stdout; 25 | t->prng.rng = theft_rng_init(DEFAULT_THEFT_SEED); 26 | if (t->prng.rng == NULL) { 27 | free(t); 28 | return THEFT_RUN_INIT_ERROR_MEMORY; 29 | } 30 | 31 | const uint8_t arity = infer_arity(cfg); 32 | if (arity == 0) { 33 | res = THEFT_RUN_INIT_ERROR_BAD_ARGS; 34 | goto cleanup; 35 | } 36 | 37 | bool all_hashable = false; 38 | if (!check_all_args(arity, cfg, &all_hashable)) { 39 | res = THEFT_RUN_INIT_ERROR_BAD_ARGS; 40 | goto cleanup; 41 | } 42 | 43 | struct seed_info seeds = { 44 | .run_seed = cfg->seed ? cfg->seed : DEFAULT_THEFT_SEED, 45 | .always_seed_count = (cfg->always_seeds == NULL 46 | ? 0 : cfg->always_seed_count), 47 | .always_seeds = cfg->always_seeds, 48 | }; 49 | memcpy(&t->seeds, &seeds, sizeof(seeds)); 50 | 51 | struct fork_info fork = { 52 | .enable = cfg->fork.enable, 53 | .timeout = cfg->fork.timeout, 54 | .signal = cfg->fork.signal, 55 | .exit_timeout = cfg->fork.exit_timeout, 56 | }; 57 | memcpy(&t->fork, &fork, sizeof(fork)); 58 | 59 | struct prop_info prop = { 60 | .name = cfg->name, 61 | .arity = arity, 62 | .trial_count = cfg->trials == 0 ? THEFT_DEF_TRIALS : cfg->trials, 63 | /* .type_info is memcpy'd below */ 64 | }; 65 | if (!copy_propfun_for_arity(cfg, &prop)) { 66 | res = THEFT_RUN_INIT_ERROR_BAD_ARGS; 67 | goto cleanup; 68 | } 69 | memcpy(&prop.type_info, cfg->type_info, sizeof(prop.type_info)); 70 | memcpy(&t->prop, &prop, sizeof(prop)); 71 | 72 | struct hook_info hooks = { 73 | .run_pre = (cfg->hooks.run_pre != NULL 74 | ? cfg->hooks.run_pre 75 | : theft_hook_run_pre_print_info), 76 | .run_post = (cfg->hooks.run_post != NULL 77 | ? cfg->hooks.run_post 78 | : theft_hook_run_post_print_info), 79 | .gen_args_pre = cfg->hooks.gen_args_pre, 80 | .trial_pre = cfg->hooks.trial_pre, 81 | .fork_post = cfg->hooks.fork_post, 82 | .trial_post = (cfg->hooks.trial_post != NULL 83 | ? cfg->hooks.trial_post 84 | : theft_hook_trial_post_print_result), 85 | 86 | .counterexample = (cfg->hooks.counterexample != NULL 87 | ? cfg->hooks.counterexample 88 | : theft_print_counterexample), 89 | .shrink_pre = cfg->hooks.shrink_pre, 90 | .shrink_post = cfg->hooks.shrink_post, 91 | .shrink_trial_post = cfg->hooks.shrink_trial_post, 92 | .env = cfg->hooks.env, 93 | }; 94 | memcpy(&t->hooks, &hooks, sizeof(hooks)); 95 | 96 | LOG(3 - LOG_RUN, 97 | "%s: SETTING RUN SEED TO 0x%016" PRIx64 "\n", 98 | __func__, t->seeds.run_seed); 99 | theft_random_set_seed(t, t->seeds.run_seed); 100 | 101 | /* If all arguments are hashable, then attempt to use 102 | * a bloom filter to avoid redundant checking. */ 103 | if (all_hashable) { 104 | t->bloom = theft_bloom_init(NULL); 105 | } 106 | 107 | /* If using the default trial_post callback, allocate its 108 | * environment, with info relating to printing progress. */ 109 | if (t->hooks.trial_post == theft_hook_trial_post_print_result) { 110 | t->print_trial_result_env = calloc(1, 111 | sizeof(*t->print_trial_result_env)); 112 | if (t->print_trial_result_env == NULL) { 113 | return THEFT_RUN_INIT_ERROR_MEMORY; 114 | } 115 | t->print_trial_result_env->tag = THEFT_PRINT_TRIAL_RESULT_ENV_TAG; 116 | } 117 | 118 | *output = t; 119 | return res; 120 | 121 | cleanup: 122 | theft_rng_free(t->prng.rng); 123 | free(t); 124 | return res; 125 | } 126 | 127 | void theft_run_free(struct theft *t) { 128 | if (t->bloom) { 129 | theft_bloom_free(t->bloom); 130 | t->bloom = NULL; 131 | } 132 | theft_rng_free(t->prng.rng); 133 | 134 | if (t->print_trial_result_env != NULL) { 135 | free(t->print_trial_result_env); 136 | } 137 | 138 | free(t); 139 | } 140 | 141 | /* Actually run the trials, with all arguments made explicit. */ 142 | enum theft_run_res 143 | theft_run_trials(struct theft *t) { 144 | if (t->hooks.run_pre != NULL) { 145 | struct theft_hook_run_pre_info hook_info = { 146 | .prop_name = t->prop.name, 147 | .total_trials = t->prop.trial_count, 148 | .run_seed = t->seeds.run_seed, 149 | }; 150 | enum theft_hook_run_pre_res res = t->hooks.run_pre(&hook_info, t->hooks.env); 151 | if (res != THEFT_HOOK_RUN_PRE_CONTINUE) { 152 | goto cleanup; 153 | } 154 | } 155 | 156 | size_t limit = t->prop.trial_count; 157 | theft_seed seed = t->seeds.run_seed; 158 | 159 | for (size_t trial = 0; trial < limit; trial++) { 160 | enum run_step_res res = run_step(t, trial, &seed); 161 | memset(&t->trial, 0x00, sizeof(t->trial)); 162 | 163 | LOG(3 - LOG_RUN, 164 | " -- trial %zd/%zd, new seed 0x%016" PRIx64 "\n", 165 | trial, limit, seed); 166 | 167 | switch (res) { 168 | case RUN_STEP_OK: 169 | continue; 170 | case RUN_STEP_HALT: 171 | limit = trial; 172 | break; 173 | default: 174 | case RUN_STEP_GEN_ERROR: 175 | case RUN_STEP_TRIAL_ERROR: 176 | goto cleanup; 177 | } 178 | } 179 | 180 | theft_hook_run_post_cb *run_post = t->hooks.run_post; 181 | if (run_post != NULL) { 182 | struct theft_hook_run_post_info hook_info = { 183 | .prop_name = t->prop.name, 184 | .total_trials = t->prop.trial_count, 185 | .run_seed = t->seeds.run_seed, 186 | .report = { 187 | .pass = t->counters.pass, 188 | .fail = t->counters.fail, 189 | .skip = t->counters.skip, 190 | .dup = t->counters.dup, 191 | }, 192 | }; 193 | 194 | enum theft_hook_run_post_res res = run_post(&hook_info, t->hooks.env); 195 | if (res != THEFT_HOOK_RUN_POST_CONTINUE) { 196 | goto cleanup; 197 | } 198 | } 199 | 200 | free_print_trial_result_env(t); 201 | 202 | if (t->counters.fail > 0) { 203 | return THEFT_RUN_FAIL; 204 | } else if (t->counters.pass > 0) { 205 | return THEFT_RUN_PASS; 206 | } else { 207 | return THEFT_RUN_SKIP; 208 | } 209 | 210 | cleanup: 211 | free_print_trial_result_env(t); 212 | return THEFT_RUN_ERROR; 213 | } 214 | 215 | static enum run_step_res 216 | run_step(struct theft *t, size_t trial, theft_seed *seed) { 217 | /* If any seeds to always run were specified, use those before 218 | * reverting to the specified starting seed. */ 219 | const size_t always_seeds = t->seeds.always_seed_count; 220 | if (trial < always_seeds) { 221 | *seed = t->seeds.always_seeds[trial]; 222 | } else if ((always_seeds > 0) && (trial == always_seeds)) { 223 | *seed = t->seeds.run_seed; 224 | } 225 | 226 | struct trial_info trial_info = { 227 | .trial = trial, 228 | .seed = *seed, 229 | }; 230 | if (!init_arg_info(t, &trial_info)) { return RUN_STEP_GEN_ERROR; } 231 | 232 | memcpy(&t->trial, &trial_info, sizeof(trial_info)); 233 | 234 | theft_hook_gen_args_pre_cb *gen_args_pre = t->hooks.gen_args_pre; 235 | if (gen_args_pre != NULL) { 236 | struct theft_hook_gen_args_pre_info hook_info = { 237 | .prop_name = t->prop.name, 238 | .total_trials = t->prop.trial_count, 239 | .failures = t->counters.fail, 240 | .run_seed = t->seeds.run_seed, 241 | .trial_id = t->trial.trial, 242 | .trial_seed = t->trial.seed, 243 | .arity = t->prop.arity 244 | }; 245 | enum theft_hook_gen_args_pre_res res = gen_args_pre(&hook_info, 246 | t->hooks.env); 247 | 248 | switch (res) { 249 | case THEFT_HOOK_GEN_ARGS_PRE_CONTINUE: 250 | break; 251 | case THEFT_HOOK_GEN_ARGS_PRE_HALT: 252 | return RUN_STEP_HALT; 253 | default: 254 | assert(false); 255 | case THEFT_HOOK_GEN_ARGS_PRE_ERROR: 256 | return RUN_STEP_GEN_ERROR; 257 | } 258 | } 259 | 260 | /* Set seed for this trial */ 261 | LOG(3 - LOG_RUN, 262 | "%s: SETTING TRIAL SEED TO 0x%016" PRIx64 "\n", __func__, trial_info.seed); 263 | theft_random_set_seed(t, trial_info.seed); 264 | 265 | enum run_step_res res = RUN_STEP_OK; 266 | enum all_gen_res gres = gen_all_args(t); 267 | /* anything after this point needs to free all args */ 268 | 269 | theft_hook_trial_post_cb *post_cb = t->hooks.trial_post; 270 | void *hook_env = (t->hooks.trial_post == theft_hook_trial_post_print_result 271 | ? t->print_trial_result_env 272 | : t->hooks.env); 273 | 274 | void *args[THEFT_MAX_ARITY]; 275 | for (size_t i = 0; i < t->prop.arity; i++) { 276 | args[i] = t->trial.args[i].instance; 277 | } 278 | 279 | struct theft_hook_trial_post_info hook_info = { 280 | .t = t, 281 | .prop_name = t->prop.name, 282 | .total_trials = t->prop.trial_count, 283 | .failures = t->counters.fail, 284 | .run_seed = *seed, 285 | .trial_id = trial, 286 | .trial_seed = trial_info.seed, 287 | .arity = t->prop.arity, 288 | .args = args, 289 | }; 290 | 291 | enum theft_hook_trial_post_res pres; 292 | 293 | switch (gres) { 294 | case ALL_GEN_SKIP: /* skip generating these args */ 295 | LOG(3 - LOG_RUN, "gen -- skip\n"); 296 | t->counters.skip++; 297 | hook_info.result = THEFT_TRIAL_SKIP; 298 | pres = post_cb(&hook_info, hook_env); 299 | break; 300 | case ALL_GEN_DUP: /* skip these args -- probably already tried */ 301 | LOG(3 - LOG_RUN, "gen -- dup\n"); 302 | t->counters.dup++; 303 | hook_info.result = THEFT_TRIAL_DUP; 304 | pres = post_cb(&hook_info, hook_env); 305 | break; 306 | default: 307 | case ALL_GEN_ERROR: /* error while generating args */ 308 | LOG(1 - LOG_RUN, "gen -- error\n"); 309 | hook_info.result = THEFT_TRIAL_ERROR; 310 | pres = post_cb(&hook_info, hook_env); 311 | res = RUN_STEP_GEN_ERROR; 312 | goto cleanup; 313 | case ALL_GEN_OK: 314 | LOG(4 - LOG_RUN, "gen -- ok\n"); 315 | if (t->hooks.trial_pre != NULL) { 316 | struct theft_hook_trial_pre_info info = { 317 | .prop_name = t->prop.name, 318 | .total_trials = t->prop.trial_count, 319 | .failures = t->counters.fail, 320 | .run_seed = t->seeds.run_seed, 321 | .trial_id = trial, 322 | .trial_seed = trial_info.seed, 323 | .arity = t->prop.arity, 324 | }; 325 | 326 | enum theft_hook_trial_pre_res tpres; 327 | tpres = t->hooks.trial_pre(&info, t->hooks.env); 328 | if (tpres == THEFT_HOOK_TRIAL_PRE_HALT) { 329 | res = RUN_STEP_HALT; 330 | goto cleanup; 331 | } else if (tpres == THEFT_HOOK_TRIAL_PRE_ERROR) { 332 | res = RUN_STEP_TRIAL_ERROR; 333 | goto cleanup; 334 | } 335 | } 336 | 337 | if (!theft_trial_run(t, &pres)) { 338 | res = RUN_STEP_TRIAL_ERROR; 339 | goto cleanup; 340 | } 341 | } 342 | 343 | if (pres == THEFT_HOOK_TRIAL_POST_ERROR) { 344 | res = RUN_STEP_TRIAL_ERROR; 345 | goto cleanup; 346 | } 347 | 348 | /* Update seed for next trial */ 349 | *seed = theft_random(t); 350 | LOG(3 - LOG_RUN, "end of trial, new seed is 0x%016" PRIx64 "\n", *seed); 351 | cleanup: 352 | theft_trial_free_args(t); 353 | return res; 354 | } 355 | 356 | static uint8_t 357 | infer_arity(const struct theft_run_config *cfg) { 358 | for (uint8_t i = 0; i < THEFT_MAX_ARITY; i++) { 359 | if (cfg->type_info[i] == NULL) { 360 | return i; 361 | } 362 | } 363 | return THEFT_MAX_ARITY; 364 | } 365 | 366 | static bool copy_propfun_for_arity(const struct theft_run_config *cfg, 367 | struct prop_info *prop) { 368 | switch (prop->arity) { 369 | #define COPY_N(N) \ 370 | case N: \ 371 | if (cfg->prop ## N == NULL) { \ 372 | return false; \ 373 | } else { \ 374 | prop->u.fun ## N = cfg->prop ## N; \ 375 | break; \ 376 | } 377 | 378 | default: 379 | case 0: 380 | assert(false); 381 | return false; 382 | COPY_N(1); 383 | COPY_N(2); 384 | COPY_N(3); 385 | COPY_N(4); 386 | COPY_N(5); 387 | COPY_N(6); 388 | COPY_N(7); 389 | #undef COPY_N 390 | } 391 | return true; 392 | } 393 | 394 | /* Check if all argument info structs have all required callbacks. */ 395 | static bool 396 | check_all_args(uint8_t arity, const struct theft_run_config *cfg, 397 | bool *all_hashable) { 398 | bool ah = true; 399 | for (uint8_t i = 0; i < arity; i++) { 400 | const struct theft_type_info *ti = cfg->type_info[i]; 401 | if (ti->alloc == NULL) { return false; } 402 | if (ti->autoshrink_config.enable && ti->shrink) { return false; } 403 | if (ti->hash == NULL && !ti->autoshrink_config.enable) { 404 | ah = false; 405 | } 406 | } 407 | *all_hashable = ah; 408 | return true; 409 | } 410 | 411 | static bool init_arg_info(struct theft *t, struct trial_info *trial_info) { 412 | for (size_t i = 0; i < t->prop.arity; i++) { 413 | const struct theft_type_info *ti = t->prop.type_info[i]; 414 | if (ti->autoshrink_config.enable) { 415 | trial_info->args[i].type = ARG_AUTOSHRINK; 416 | trial_info->args[i].u.as.env = theft_autoshrink_alloc_env(t, i, ti); 417 | if (trial_info->args[i].u.as.env == NULL) { 418 | return false; 419 | } 420 | } else { 421 | trial_info->args[i].type = ARG_BASIC; 422 | } 423 | } 424 | return true; 425 | } 426 | 427 | /* Attempt to instantiate arguments, starting with the current seed. */ 428 | static enum all_gen_res 429 | gen_all_args(struct theft *t) { 430 | for (uint8_t i = 0; i < t->prop.arity; i++) { 431 | struct theft_type_info *ti = t->prop.type_info[i]; 432 | void *p = NULL; 433 | 434 | enum theft_alloc_res res = (ti->autoshrink_config.enable 435 | ? theft_autoshrink_alloc(t, t->trial.args[i].u.as.env, &p) 436 | : ti->alloc(t, ti->env, &p)); 437 | 438 | if (res == THEFT_ALLOC_SKIP) { 439 | return ALL_GEN_SKIP; 440 | } else if (res == THEFT_ALLOC_ERROR) { 441 | return ALL_GEN_ERROR; 442 | } else { 443 | t->trial.args[i].instance = p; 444 | LOG(3 - LOG_RUN, "%s: arg %u -- %p\n", 445 | __func__, i, p); 446 | } 447 | } 448 | 449 | /* check bloom filter */ 450 | if (t->bloom && theft_call_check_called(t)) { 451 | return ALL_GEN_DUP; 452 | } 453 | 454 | return ALL_GEN_OK; 455 | } 456 | 457 | static void free_print_trial_result_env(struct theft *t) { 458 | if (t->hooks.trial_post == theft_hook_trial_post_print_result 459 | && t->print_trial_result_env != NULL) { 460 | free(t->print_trial_result_env); 461 | t->print_trial_result_env = NULL; 462 | } 463 | } 464 | -------------------------------------------------------------------------------- /src/theft_run.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_RUN_H 2 | #define THEFT_RUN_H 3 | 4 | enum theft_run_init_res { 5 | THEFT_RUN_INIT_OK, 6 | THEFT_RUN_INIT_ERROR_MEMORY = -1, 7 | THEFT_RUN_INIT_ERROR_BAD_ARGS = -2, 8 | }; 9 | enum theft_run_init_res 10 | theft_run_init(const struct theft_run_config *cfg, 11 | struct theft **output); 12 | 13 | /* Actually run the trials, with all arguments made explicit. */ 14 | enum theft_run_res 15 | theft_run_trials(struct theft *t); 16 | 17 | void 18 | theft_run_free(struct theft *t); 19 | 20 | #endif 21 | -------------------------------------------------------------------------------- /src/theft_run_internal.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_RUN_INTERNAL_H 2 | #define THEFT_RUN_INTERNAL_H 3 | 4 | #include "theft_types_internal.h" 5 | #include "theft_run.h" 6 | 7 | static uint8_t 8 | infer_arity(const struct theft_run_config *cfg); 9 | 10 | enum run_step_res { 11 | RUN_STEP_OK, 12 | RUN_STEP_HALT, 13 | RUN_STEP_GEN_ERROR, 14 | RUN_STEP_TRIAL_ERROR, 15 | }; 16 | static enum run_step_res 17 | run_step(struct theft *t, size_t trial, theft_seed *seed); 18 | 19 | static bool copy_propfun_for_arity(const struct theft_run_config *cfg, 20 | struct prop_info *prop); 21 | 22 | static bool 23 | check_all_args(uint8_t arity, const struct theft_run_config *cfg, 24 | bool *all_hashable); 25 | 26 | enum all_gen_res { 27 | ALL_GEN_OK, /* all arguments generated okay */ 28 | ALL_GEN_SKIP, /* skip due to user constraints */ 29 | ALL_GEN_DUP, /* skip probably duplicated trial */ 30 | ALL_GEN_ERROR, /* memory error or other failure */ 31 | }; 32 | 33 | static bool init_arg_info(struct theft *t, struct trial_info *trial_info); 34 | 35 | static enum all_gen_res 36 | gen_all_args(struct theft *t); 37 | 38 | static void free_print_trial_result_env(struct theft *t); 39 | 40 | #endif 41 | -------------------------------------------------------------------------------- /src/theft_shrink.c: -------------------------------------------------------------------------------- 1 | #include "theft_shrink_internal.h" 2 | #include "theft_shrink.h" 3 | 4 | #include "theft_call.h" 5 | #include "theft_trial.h" 6 | #include "theft_autoshrink.h" 7 | #include 8 | 9 | #define LOG_SHRINK 0 10 | 11 | /* Attempt to simplify all arguments, breadth first. Continue as long as 12 | * progress is made, i.e., until a local minimum is reached. */ 13 | bool 14 | theft_shrink(struct theft *t) { 15 | bool progress = false; 16 | assert(t->prop.arity > 0); 17 | 18 | do { 19 | progress = false; 20 | /* Greedily attempt to simplify each argument as much as 21 | * possible before switching to the next. */ 22 | for (uint8_t arg_i = 0; arg_i < t->prop.arity; arg_i++) { 23 | struct theft_type_info *ti = t->prop.type_info[arg_i]; 24 | greedy_continue: 25 | if (ti->shrink || ti->autoshrink_config.enable) { 26 | /* attempt to simplify this argument by one step */ 27 | enum shrink_res rres = attempt_to_shrink_arg(t, arg_i); 28 | 29 | switch (rres) { 30 | case SHRINK_OK: 31 | LOG(3 - LOG_SHRINK, "%s %u: progress\n", __func__, arg_i); 32 | progress = true; 33 | goto greedy_continue; /* keep trying to shrink same argument */ 34 | case SHRINK_HALT: 35 | LOG(3 - LOG_SHRINK, "%s %u: HALT\n", __func__, arg_i); 36 | return true; 37 | case SHRINK_DEAD_END: 38 | LOG(3 - LOG_SHRINK, "%s %u: DEAD END\n", __func__, arg_i); 39 | continue; /* try next argument, if any */ 40 | default: 41 | case SHRINK_ERROR: 42 | LOG(1 - LOG_SHRINK, "%s %u: ERROR\n", __func__, arg_i); 43 | return false; 44 | } 45 | } 46 | } 47 | } while (progress); 48 | return true; 49 | } 50 | 51 | /* Simplify an argument by trying all of its simplification tactics, in 52 | * order, and checking whether the property still fails. If it passes, 53 | * then revert the simplification and try another tactic. 54 | * 55 | * If the bloom filter is being used (i.e., if all arguments have hash 56 | * callbacks defined), then use it to skip over areas of the state 57 | * space that have probably already been tried. */ 58 | static enum shrink_res 59 | attempt_to_shrink_arg(struct theft *t, uint8_t arg_i) { 60 | struct theft_type_info *ti = t->prop.type_info[arg_i]; 61 | const bool use_autoshrink = ti->autoshrink_config.enable; 62 | 63 | for (uint32_t tactic = 0; tactic < THEFT_MAX_TACTICS; tactic++) { 64 | LOG(2 - LOG_SHRINK, "SHRINKING arg %u, tactic %u\n", arg_i, tactic); 65 | void *current = t->trial.args[arg_i].instance; 66 | void *candidate = NULL; 67 | 68 | enum theft_hook_shrink_pre_res shrink_pre_res; 69 | shrink_pre_res = shrink_pre_hook(t, arg_i, current, tactic); 70 | if (shrink_pre_res == THEFT_HOOK_SHRINK_PRE_HALT) { 71 | return SHRINK_HALT; 72 | } else if (shrink_pre_res != THEFT_HOOK_SHRINK_PRE_CONTINUE) { 73 | return SHRINK_ERROR; 74 | } 75 | 76 | struct autoshrink_env *as_env = NULL; 77 | struct autoshrink_bit_pool *current_bit_pool = NULL; 78 | struct autoshrink_bit_pool *candidate_bit_pool = NULL; 79 | if (use_autoshrink) { 80 | as_env = t->trial.args[arg_i].u.as.env; 81 | assert(as_env); 82 | current_bit_pool = t->trial.args[arg_i].u.as.env->bit_pool; 83 | } 84 | 85 | enum theft_shrink_res sres = (use_autoshrink 86 | ? theft_autoshrink_shrink(t, as_env, tactic, &candidate, 87 | &candidate_bit_pool) 88 | : ti->shrink(t, current, tactic, ti->env, &candidate)); 89 | 90 | LOG(3 - LOG_SHRINK, "%s: tactic %u -> res %d\n", __func__, tactic, sres); 91 | 92 | t->trial.shrink_count++; 93 | 94 | enum theft_hook_shrink_post_res shrink_post_res; 95 | shrink_post_res = shrink_post_hook(t, arg_i, 96 | sres == THEFT_SHRINK_OK ? candidate : current, 97 | tactic, sres); 98 | if (shrink_post_res != THEFT_HOOK_SHRINK_POST_CONTINUE) { 99 | if (ti->free) { ti->free(candidate, ti->env); } 100 | if (candidate_bit_pool) { 101 | theft_autoshrink_free_bit_pool(t, candidate_bit_pool); 102 | } 103 | return SHRINK_ERROR; 104 | } 105 | 106 | switch (sres) { 107 | case THEFT_SHRINK_OK: 108 | break; 109 | case THEFT_SHRINK_DEAD_END: 110 | continue; /* try next tactic */ 111 | case THEFT_SHRINK_NO_MORE_TACTICS: 112 | return SHRINK_DEAD_END; 113 | case THEFT_SHRINK_ERROR: 114 | default: 115 | return SHRINK_ERROR; 116 | } 117 | 118 | t->trial.args[arg_i].instance = candidate; 119 | if (use_autoshrink) { as_env->bit_pool = candidate_bit_pool; } 120 | 121 | if (t->bloom) { 122 | if (theft_call_check_called(t)) { 123 | LOG(3 - LOG_SHRINK, 124 | "%s: already called, skipping\n", __func__); 125 | if (ti->free) { ti->free(candidate, ti->env); } 126 | if (use_autoshrink) { 127 | as_env->bit_pool = current_bit_pool; 128 | theft_autoshrink_free_bit_pool(t, candidate_bit_pool); 129 | } 130 | t->trial.args[arg_i].instance = current; 131 | continue; 132 | } else { 133 | theft_call_mark_called(t); 134 | } 135 | } 136 | 137 | enum theft_trial_res res; 138 | bool repeated = false; 139 | for (;;) { 140 | void *args[THEFT_MAX_ARITY]; 141 | theft_trial_get_args(t, args); 142 | 143 | res = theft_call(t, args); 144 | LOG(3 - LOG_SHRINK, "%s: call -> res %d\n", __func__, res); 145 | 146 | if (!repeated) { 147 | if (res == THEFT_TRIAL_FAIL) { 148 | t->trial.successful_shrinks++; 149 | theft_autoshrink_update_model(t, arg_i, res, 3); 150 | } else { 151 | t->trial.failed_shrinks++; 152 | } 153 | } 154 | 155 | enum theft_hook_shrink_trial_post_res stpres; 156 | stpres = shrink_trial_post_hook(t, arg_i, args, tactic, res); 157 | if (stpres == THEFT_HOOK_SHRINK_TRIAL_POST_REPEAT 158 | || (stpres == THEFT_HOOK_SHRINK_TRIAL_POST_REPEAT_ONCE && !repeated)) { 159 | repeated = true; 160 | continue; // loop and run again 161 | } else if (stpres == THEFT_HOOK_SHRINK_TRIAL_POST_REPEAT_ONCE && repeated) { 162 | break; 163 | } else if (stpres == THEFT_HOOK_SHRINK_TRIAL_POST_CONTINUE) { 164 | break; 165 | } else { 166 | if (ti->free) { ti->free(current, ti->env); } 167 | if (use_autoshrink && current_bit_pool) { 168 | theft_autoshrink_free_bit_pool(t, current_bit_pool); 169 | } 170 | return SHRINK_ERROR; 171 | } 172 | } 173 | 174 | theft_autoshrink_update_model(t, arg_i, res, 8); 175 | 176 | switch (res) { 177 | case THEFT_TRIAL_PASS: 178 | case THEFT_TRIAL_SKIP: 179 | LOG(2 - LOG_SHRINK, "PASS or SKIP: REVERTING %u: candidate %p (pool %p), back to %p (pool %p)\n", 180 | arg_i, (void *)candidate, (void *)candidate_bit_pool, 181 | (void *)current, (void *)current_bit_pool); 182 | t->trial.args[arg_i].instance = current; 183 | if (use_autoshrink) { 184 | theft_autoshrink_free_bit_pool(t, candidate_bit_pool); 185 | t->trial.args[arg_i].u.as.env->bit_pool = current_bit_pool; 186 | } 187 | if (ti->free) { ti->free(candidate, ti->env); } 188 | break; 189 | case THEFT_TRIAL_FAIL: 190 | LOG(2 - LOG_SHRINK, "FAIL: COMMITTING %u: was %p (pool %p), now %p (pool %p)\n", 191 | arg_i, (void *)current, (void *)current_bit_pool, 192 | (void *)candidate, (void *)candidate_bit_pool); 193 | if (use_autoshrink) { 194 | assert(t->trial.args[arg_i].u.as.env->bit_pool == candidate_bit_pool); 195 | theft_autoshrink_free_bit_pool(t, current_bit_pool); 196 | } 197 | assert(t->trial.args[arg_i].instance == candidate); 198 | if (ti->free) { ti->free(current, ti->env); } 199 | return SHRINK_OK; 200 | default: 201 | case THEFT_TRIAL_ERROR: 202 | if (ti->free) { ti->free(current, ti->env); } 203 | if (use_autoshrink) { 204 | theft_autoshrink_free_bit_pool(t, current_bit_pool); 205 | } 206 | return SHRINK_ERROR; 207 | } 208 | } 209 | (void)t; 210 | return SHRINK_DEAD_END; 211 | } 212 | 213 | static enum theft_hook_shrink_pre_res 214 | shrink_pre_hook(struct theft *t, 215 | uint8_t arg_index, void *arg, uint32_t tactic) { 216 | if (t->hooks.shrink_pre != NULL) { 217 | struct theft_hook_shrink_pre_info hook_info = { 218 | .prop_name = t->prop.name, 219 | .total_trials = t->prop.trial_count, 220 | .failures = t->counters.fail, 221 | .run_seed = t->seeds.run_seed, 222 | .trial_id = t->trial.trial, 223 | .trial_seed = t->trial.seed, 224 | .arity = t->prop.arity, 225 | .shrink_count = t->trial.shrink_count, 226 | .successful_shrinks = t->trial.successful_shrinks, 227 | .failed_shrinks = t->trial.failed_shrinks, 228 | .arg_index = arg_index, 229 | .arg = arg, 230 | .tactic = tactic, 231 | }; 232 | return t->hooks.shrink_pre(&hook_info, t->hooks.env); 233 | } else { 234 | return THEFT_HOOK_SHRINK_PRE_CONTINUE; 235 | } 236 | } 237 | 238 | static enum theft_hook_shrink_post_res 239 | shrink_post_hook(struct theft *t, 240 | uint8_t arg_index, void *arg, uint32_t tactic, 241 | enum theft_shrink_res sres) { 242 | if (t->hooks.shrink_post != NULL) { 243 | enum theft_shrink_post_state state; 244 | switch (sres) { 245 | case THEFT_SHRINK_OK: 246 | state = THEFT_SHRINK_POST_SHRUNK; break; 247 | case THEFT_SHRINK_NO_MORE_TACTICS: 248 | state = THEFT_SHRINK_POST_DONE_SHRINKING; break; 249 | case THEFT_SHRINK_DEAD_END: 250 | state = THEFT_SHRINK_POST_SHRINK_FAILED; break; 251 | default: 252 | assert(false); 253 | return THEFT_HOOK_SHRINK_POST_ERROR; 254 | } 255 | 256 | struct theft_hook_shrink_post_info hook_info = { 257 | .prop_name = t->prop.name, 258 | .total_trials = t->prop.trial_count, 259 | .run_seed = t->seeds.run_seed, 260 | .trial_id = t->trial.trial, 261 | .trial_seed = t->trial.seed, 262 | .arity = t->prop.arity, 263 | .shrink_count = t->trial.shrink_count, 264 | .successful_shrinks = t->trial.successful_shrinks, 265 | .failed_shrinks = t->trial.failed_shrinks, 266 | .arg_index = arg_index, 267 | .arg = arg, 268 | .tactic = tactic, 269 | .state = state, 270 | }; 271 | return t->hooks.shrink_post(&hook_info, t->hooks.env); 272 | } else { 273 | return THEFT_HOOK_SHRINK_POST_CONTINUE; 274 | } 275 | } 276 | 277 | static enum theft_hook_shrink_trial_post_res 278 | shrink_trial_post_hook(struct theft *t, 279 | uint8_t arg_index, void **args, uint32_t last_tactic, 280 | enum theft_trial_res result) { 281 | if (t->hooks.shrink_trial_post != NULL) { 282 | struct theft_hook_shrink_trial_post_info hook_info = { 283 | .prop_name = t->prop.name, 284 | .total_trials = t->prop.trial_count, 285 | .failures = t->counters.fail, 286 | .run_seed = t->seeds.run_seed, 287 | .trial_id = t->trial.trial, 288 | .trial_seed = t->trial.seed, 289 | .arity = t->prop.arity, 290 | .shrink_count = t->trial.shrink_count, 291 | .successful_shrinks = t->trial.successful_shrinks, 292 | .failed_shrinks = t->trial.failed_shrinks, 293 | .arg_index = arg_index, 294 | .args = args, 295 | .tactic = last_tactic, 296 | .result = result, 297 | }; 298 | return t->hooks.shrink_trial_post(&hook_info, 299 | t->hooks.env); 300 | } else { 301 | return THEFT_HOOK_SHRINK_TRIAL_POST_CONTINUE; 302 | } 303 | } 304 | -------------------------------------------------------------------------------- /src/theft_shrink.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_SHRINK_H 2 | #define THEFT_SHRINK_H 3 | 4 | /* Attempt to simplify all arguments, breadth first. Continue as long as 5 | * progress is made, i.e., until a local minimum is reached. */ 6 | bool 7 | theft_shrink(struct theft *t); 8 | 9 | #endif 10 | -------------------------------------------------------------------------------- /src/theft_shrink_internal.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_SHRINK_INTERNAL_H 2 | #define THEFT_SHRINK_INTERNAL_H 3 | 4 | #include "theft_types_internal.h" 5 | 6 | enum shrink_res { 7 | SHRINK_OK, /* simplified argument further */ 8 | SHRINK_DEAD_END, /* at local minima */ 9 | SHRINK_ERROR, /* hard error during shrinking */ 10 | SHRINK_HALT, /* don't shrink any further */ 11 | }; 12 | 13 | static enum shrink_res 14 | attempt_to_shrink_arg(struct theft *t, uint8_t arg_i); 15 | 16 | static enum theft_hook_shrink_pre_res 17 | shrink_pre_hook(struct theft *t, 18 | uint8_t arg_index, void *arg, uint32_t tactic); 19 | 20 | static enum theft_hook_shrink_post_res 21 | shrink_post_hook(struct theft *t, 22 | uint8_t arg_index, void *arg, uint32_t tactic, 23 | enum theft_shrink_res sres); 24 | 25 | static enum theft_hook_shrink_trial_post_res 26 | shrink_trial_post_hook(struct theft *t, 27 | uint8_t arg_index, void **args, uint32_t last_tactic, 28 | enum theft_trial_res result); 29 | 30 | #endif 31 | -------------------------------------------------------------------------------- /src/theft_trial.c: -------------------------------------------------------------------------------- 1 | #include "theft_trial_internal.h" 2 | 3 | #include 4 | #include 5 | 6 | #include "theft_call.h" 7 | #include "theft_shrink.h" 8 | #include "theft_autoshrink.h" 9 | 10 | /* Now that arguments have been generated, run the trial and update 11 | * counters, call cb with results, etc. */ 12 | bool 13 | theft_trial_run(struct theft *t, 14 | enum theft_hook_trial_post_res *tpres) { 15 | assert(t->prop.arity > 0); 16 | 17 | if (t->bloom) { theft_call_mark_called(t); } 18 | 19 | void *args[THEFT_MAX_ARITY]; 20 | theft_trial_get_args(t, args); 21 | 22 | bool repeated = false; 23 | enum theft_trial_res tres = theft_call(t, args); 24 | theft_hook_trial_post_cb *trial_post = t->hooks.trial_post; 25 | void *trial_post_env = (trial_post == theft_hook_trial_post_print_result 26 | ? t->print_trial_result_env 27 | : t->hooks.env); 28 | 29 | struct theft_hook_trial_post_info hook_info = { 30 | .t = t, 31 | .prop_name = t->prop.name, 32 | .total_trials = t->prop.trial_count, 33 | .run_seed = t->seeds.run_seed, 34 | .trial_id = t->trial.trial, 35 | .trial_seed = t->trial.seed, 36 | .arity = t->prop.arity, 37 | .args = args, 38 | .result = tres, 39 | }; 40 | 41 | switch (tres) { 42 | case THEFT_TRIAL_PASS: 43 | if (!repeated) { 44 | t->counters.pass++; 45 | } 46 | *tpres = trial_post(&hook_info, trial_post_env); 47 | break; 48 | case THEFT_TRIAL_FAIL: 49 | if (!theft_shrink(t)) { 50 | hook_info.result = THEFT_TRIAL_ERROR; 51 | /* We may not have a valid reference to the arguments 52 | * anymore, so remove the stale pointers. */ 53 | for (size_t i = 0; i < t->prop.arity; i++) { 54 | hook_info.args[i] = NULL; 55 | } 56 | *tpres = trial_post(&hook_info, trial_post_env); 57 | return false; 58 | } 59 | 60 | if (!repeated) { 61 | t->counters.fail++; 62 | } 63 | 64 | theft_trial_get_args(t, hook_info.args); 65 | *tpres = report_on_failure(t, &hook_info, trial_post, trial_post_env); 66 | break; 67 | case THEFT_TRIAL_SKIP: 68 | if (!repeated) { 69 | t->counters.skip++; 70 | } 71 | *tpres = trial_post(&hook_info, trial_post_env); 72 | break; 73 | case THEFT_TRIAL_DUP: 74 | /* user callback should not return this; fall through */ 75 | case THEFT_TRIAL_ERROR: 76 | *tpres = trial_post(&hook_info, trial_post_env); 77 | return false; 78 | } 79 | 80 | if (*tpres == THEFT_HOOK_TRIAL_POST_ERROR) { 81 | return false; 82 | } 83 | 84 | return true; 85 | } 86 | 87 | void theft_trial_free_args(struct theft *t) { 88 | for (size_t i = 0; i < t->prop.arity; i++) { 89 | struct theft_type_info *ti = t->prop.type_info[i]; 90 | 91 | struct arg_info *ai = &t->trial.args[i]; 92 | if (ai->type == ARG_AUTOSHRINK) { 93 | theft_autoshrink_free_env(t, ai->u.as.env); 94 | } 95 | if (ai->instance != NULL && ti->free != NULL) { 96 | ti->free(t->trial.args[i].instance, ti->env); 97 | } 98 | } 99 | } 100 | 101 | void 102 | theft_trial_get_args(struct theft *t, void **args) { 103 | for (size_t i = 0; i < t->prop.arity; i++) { 104 | args[i] = t->trial.args[i].instance; 105 | } 106 | } 107 | 108 | /* Print info about a failure. */ 109 | static enum theft_hook_trial_post_res 110 | report_on_failure(struct theft *t, 111 | struct theft_hook_trial_post_info *hook_info, 112 | theft_hook_trial_post_cb *trial_post, 113 | void *trial_post_env) { 114 | theft_hook_counterexample_cb *counterexample = t->hooks.counterexample; 115 | if (counterexample != NULL) { 116 | struct theft_hook_counterexample_info counterexample_hook_info = { 117 | .t = t, 118 | .prop_name = t->prop.name, 119 | .total_trials = t->prop.trial_count, 120 | .trial_id = t->trial.trial, 121 | .trial_seed = t->trial.seed, 122 | .arity = t->prop.arity, 123 | .type_info = t->prop.type_info, 124 | .args = hook_info->args, 125 | }; 126 | 127 | if (counterexample(&counterexample_hook_info, t->hooks.env) 128 | != THEFT_HOOK_COUNTEREXAMPLE_CONTINUE) { 129 | return THEFT_HOOK_TRIAL_POST_ERROR; 130 | } 131 | } 132 | 133 | enum theft_hook_trial_post_res res; 134 | res = trial_post(hook_info, trial_post_env); 135 | 136 | while (res == THEFT_HOOK_TRIAL_POST_REPEAT 137 | || res == THEFT_HOOK_TRIAL_POST_REPEAT_ONCE) { 138 | hook_info->repeat = true; 139 | 140 | enum theft_trial_res tres = theft_call(t, hook_info->args); 141 | if (tres == THEFT_TRIAL_FAIL) { 142 | res = trial_post(hook_info, t->hooks.env); 143 | if (res == THEFT_HOOK_TRIAL_POST_REPEAT_ONCE) { 144 | break; 145 | } 146 | } else if (tres == THEFT_TRIAL_PASS) { 147 | fprintf(t->out, "Warning: Failed property passed when re-run.\n"); 148 | res = THEFT_HOOK_TRIAL_POST_ERROR; 149 | } else if (tres == THEFT_TRIAL_ERROR) { 150 | return THEFT_HOOK_TRIAL_POST_ERROR; 151 | } else { 152 | return THEFT_HOOK_TRIAL_POST_CONTINUE; 153 | } 154 | } 155 | return res; 156 | } 157 | -------------------------------------------------------------------------------- /src/theft_trial.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_TRIAL_H 2 | #define THEFT_TRIAL_H 3 | 4 | bool 5 | theft_trial_run(struct theft *t, 6 | enum theft_hook_trial_post_res *tpres); 7 | 8 | void 9 | theft_trial_get_args(struct theft *t, 10 | void **args); 11 | 12 | void 13 | theft_trial_free_args(struct theft *t); 14 | 15 | #endif 16 | -------------------------------------------------------------------------------- /src/theft_trial_internal.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_TRIAL_INTERNAL_H 2 | #define THEFT_TRIAL_INTERNAL_H 3 | 4 | #include "theft_types_internal.h" 5 | #include "theft_trial.h" 6 | 7 | static enum theft_hook_trial_post_res 8 | report_on_failure(struct theft *t, 9 | struct theft_hook_trial_post_info *hook_info, 10 | theft_hook_trial_post_cb *trial_post, 11 | void *trial_post_env); 12 | 13 | theft_hook_trial_post_cb def_trial_post_cb; 14 | 15 | #endif 16 | -------------------------------------------------------------------------------- /src/theft_types_internal.h: -------------------------------------------------------------------------------- 1 | #ifndef THEFT_TYPES_INTERNAL_H 2 | #define THEFT_TYPES_INTERNAL_H 3 | 4 | #include "theft.h" 5 | #include 6 | #include 7 | #include 8 | 9 | #define THEFT_MAX_TACTICS ((uint32_t)-1) 10 | #define DEFAULT_THEFT_SEED 0xa600d64b175eedLLU 11 | 12 | #define THEFT_LOG_LEVEL 0 13 | #define LOG(LEVEL, ...) \ 14 | do { \ 15 | if (LEVEL <= THEFT_LOG_LEVEL) { \ 16 | printf(__VA_ARGS__); \ 17 | } \ 18 | } while(0) 19 | 20 | struct theft_bloom; /* bloom filter */ 21 | struct theft_rng; /* pseudorandom number generator */ 22 | 23 | struct seed_info { 24 | const theft_seed run_seed; 25 | 26 | /* Optional array of seeds to always run. 27 | * Can be used for regression tests. */ 28 | const size_t always_seed_count; /* number of seeds */ 29 | const theft_seed *always_seeds; /* seeds to always run */ 30 | }; 31 | 32 | struct fork_info { 33 | const bool enable; 34 | const size_t timeout; 35 | const int signal; 36 | const size_t exit_timeout; 37 | }; 38 | 39 | struct prop_info { 40 | const char *name; /* property name, can be NULL */ 41 | /* property function under test */ 42 | union { 43 | theft_propfun1 *fun1; 44 | theft_propfun2 *fun2; 45 | theft_propfun3 *fun3; 46 | theft_propfun4 *fun4; 47 | theft_propfun5 *fun5; 48 | theft_propfun6 *fun6; 49 | theft_propfun7 *fun7; 50 | } u; 51 | const size_t trial_count; 52 | 53 | /* Type info for ARITY arguments. */ 54 | const uint8_t arity; /* number of arguments */ 55 | struct theft_type_info *type_info[THEFT_MAX_ARITY]; 56 | }; 57 | 58 | struct hook_info { 59 | theft_hook_run_pre_cb *run_pre; 60 | theft_hook_run_post_cb *run_post; 61 | theft_hook_gen_args_pre_cb *gen_args_pre; 62 | theft_hook_trial_pre_cb *trial_pre; 63 | theft_hook_fork_post_cb *fork_post; 64 | theft_hook_trial_post_cb *trial_post; 65 | theft_hook_counterexample_cb *counterexample; 66 | theft_hook_shrink_pre_cb *shrink_pre; 67 | theft_hook_shrink_post_cb *shrink_post; 68 | theft_hook_shrink_trial_post_cb *shrink_trial_post; 69 | void *env; 70 | }; 71 | 72 | struct counter_info { 73 | size_t pass; 74 | size_t fail; 75 | size_t skip; 76 | size_t dup; 77 | }; 78 | 79 | struct prng_info { 80 | struct theft_rng *rng; /* random number generator */ 81 | uint64_t buf; /* buffer for PRNG bits */ 82 | uint8_t bits_available; 83 | /* Bit pool, only used during autoshrinking. */ 84 | struct autoshrink_bit_pool *bit_pool; 85 | }; 86 | 87 | enum arg_type { 88 | ARG_BASIC, 89 | ARG_AUTOSHRINK, 90 | }; 91 | 92 | struct arg_info { 93 | void *instance; 94 | 95 | enum arg_type type; 96 | union { 97 | struct { 98 | struct autoshrink_env *env; 99 | } as; 100 | } u; 101 | }; 102 | 103 | /* Result from an individual trial. */ 104 | struct trial_info { 105 | const int trial; /* N'th trial */ 106 | theft_seed seed; /* Seed used */ 107 | size_t shrink_count; 108 | size_t successful_shrinks; 109 | size_t failed_shrinks; 110 | struct arg_info args[THEFT_MAX_ARITY]; 111 | }; 112 | 113 | enum worker_state { 114 | WS_INACTIVE, 115 | WS_ACTIVE, 116 | WS_STOPPED, 117 | }; 118 | 119 | struct worker_info { 120 | enum worker_state state; 121 | int fds[2]; 122 | pid_t pid; 123 | int wstatus; 124 | }; 125 | 126 | /* Handle to state for the entire run. */ 127 | struct theft { 128 | FILE *out; 129 | struct theft_bloom *bloom; /* bloom filter */ 130 | struct theft_print_trial_result_env *print_trial_result_env; 131 | 132 | struct prng_info prng; 133 | struct prop_info prop; 134 | struct seed_info seeds; 135 | struct fork_info fork; 136 | struct hook_info hooks; 137 | struct counter_info counters; 138 | struct trial_info trial; 139 | struct worker_info workers[1]; 140 | }; 141 | 142 | #endif 143 | -------------------------------------------------------------------------------- /test/test_char_array.c: -------------------------------------------------------------------------------- 1 | #include "test_theft.h" 2 | 3 | static enum theft_trial_res 4 | prop_char_fails_cause_shrink(struct theft *t, void *arg1) { 5 | (void)t; 6 | char *test_str = arg1; 7 | 8 | return strlen(test_str) ? THEFT_TRIAL_FAIL : THEFT_TRIAL_PASS; 9 | } 10 | 11 | 12 | TEST char_fail_shrinkage(void) { 13 | theft_seed seed = theft_seed_of_time(); 14 | 15 | struct theft_run_config cfg = { 16 | .name = __func__, 17 | .prop1 = prop_char_fails_cause_shrink, 18 | .type_info = { 19 | theft_get_builtin_type_info(THEFT_BUILTIN_char_ARRAY), 20 | }, 21 | .bloom_bits = 20, 22 | .seed = seed, 23 | .trials = 1, 24 | }; 25 | 26 | ASSERT_EQm("should fail until full contraction", 27 | THEFT_RUN_FAIL, theft_run(&cfg)); 28 | PASS(); 29 | } 30 | 31 | 32 | SUITE(char_array) { 33 | RUN_TEST(char_fail_shrinkage); 34 | } 35 | -------------------------------------------------------------------------------- /test/test_theft.c: -------------------------------------------------------------------------------- 1 | #include "test_theft.h" 2 | 3 | /* Add all the definitions that need to be in the test runner's main file. */ 4 | GREATEST_MAIN_DEFS(); 5 | 6 | int main(int argc, char **argv) { 7 | GREATEST_MAIN_BEGIN(); /* command-line arguments, initialization. */ 8 | RUN_SUITE(prng); 9 | RUN_SUITE(autoshrink); 10 | RUN_SUITE(aux); 11 | RUN_SUITE(bloom); 12 | RUN_SUITE(error); 13 | RUN_SUITE(integration); 14 | RUN_SUITE(char_array); 15 | GREATEST_MAIN_END(); /* display results */ 16 | } 17 | -------------------------------------------------------------------------------- /test/test_theft.h: -------------------------------------------------------------------------------- 1 | #ifndef TEST_THEFT_H 2 | #define TEST_THEFT_H 3 | 4 | #include "greatest.h" 5 | #include "theft.h" 6 | 7 | #include 8 | #include 9 | 10 | SUITE_EXTERN(prng); 11 | SUITE_EXTERN(autoshrink); 12 | SUITE_EXTERN(aux); 13 | SUITE_EXTERN(bloom); 14 | SUITE_EXTERN(error); 15 | SUITE_EXTERN(integration); 16 | SUITE_EXTERN(char_array); 17 | 18 | #endif 19 | -------------------------------------------------------------------------------- /test/test_theft_autoshrink_bulk.c: -------------------------------------------------------------------------------- 1 | #include "test_theft.h" 2 | #include "test_theft_autoshrink_bulk.h" 3 | 4 | static void bb_free(void *instance, void *env); 5 | static void bb_print(FILE *f, const void *instance, void *env); 6 | 7 | /* Generate a block of random bits, with a known size. 8 | * This is mainly to exercise `theft_random_bits_bulk`. */ 9 | static enum theft_alloc_res 10 | bb_alloc(struct theft *t, void *env, void **instance) { 11 | (void)env; 12 | size_t size = theft_random_bits(t, 20); 13 | uint64_t *buf = calloc(size / 64 + 1, sizeof(uint64_t)); 14 | if (buf == NULL) { 15 | return THEFT_ALLOC_ERROR; 16 | } 17 | 18 | struct bulk_buffer *bb = calloc(1, sizeof(*bb)); 19 | if (bb == NULL) { 20 | free(buf); 21 | return THEFT_ALLOC_ERROR; 22 | } 23 | *bb = (struct bulk_buffer) { 24 | .size = size, 25 | .buf = buf, 26 | }; 27 | 28 | theft_random_bits_bulk(t, size, bb->buf); 29 | *instance = bb; 30 | //bb_print(stdout, bb, NULL); 31 | 32 | return THEFT_ALLOC_OK; 33 | } 34 | 35 | static void 36 | bb_free(void *instance, void *env) { 37 | (void)env; 38 | struct bulk_buffer *bb = (struct bulk_buffer *)instance; 39 | free(bb->buf); 40 | free(bb); 41 | } 42 | 43 | static void bb_print(FILE *f, const void *instance, void *env) { 44 | (void)env; 45 | struct bulk_buffer *bb = (struct bulk_buffer *)instance; 46 | fprintf(f, "-- bulk_buf[%zd]: \n", bb->size); 47 | const uint8_t *buf8 = (uint8_t *)bb->buf; 48 | const size_t limit = bb->size/8; 49 | for (size_t offset = 0; offset < limit; offset += 16) { 50 | const size_t rem = (limit - offset < 16 ? limit - offset : 16); 51 | for (size_t i = 0; i < rem; i++) { 52 | fprintf(f, "%02x ", buf8[offset + i]); 53 | if (i == 15) { 54 | fprintf(f, "\n"); 55 | } 56 | } 57 | } 58 | 59 | const size_t rem_bits = (bb->size % 8); 60 | if (rem_bits != 0) { 61 | fprintf(f, "%02x/%zd\n", buf8[limit], rem_bits); 62 | } else { 63 | fprintf(f, "\n"); 64 | } 65 | } 66 | 67 | struct theft_type_info bb_info = { 68 | .alloc = bb_alloc, 69 | .free = bb_free, 70 | .print = bb_print, 71 | .autoshrink_config = { 72 | .enable = true, 73 | .print_mode = THEFT_AUTOSHRINK_PRINT_ALL, 74 | }, 75 | }; 76 | -------------------------------------------------------------------------------- /test/test_theft_autoshrink_bulk.h: -------------------------------------------------------------------------------- 1 | #ifndef TEST_THEFT_AUTOSHRINK_BULK_H 2 | #define TEST_THEFT_AUTOSHRINK_BULK_H 3 | 4 | #include 5 | 6 | struct bulk_buffer { 7 | size_t size; 8 | uint64_t *buf; 9 | }; 10 | 11 | extern struct theft_type_info bb_info; 12 | 13 | #endif 14 | -------------------------------------------------------------------------------- /test/test_theft_autoshrink_int_array.c: -------------------------------------------------------------------------------- 1 | #include "test_theft.h" 2 | #include "test_theft_autoshrink_int_array.h" 3 | 4 | static void ia_print(FILE *f, const void *instance, void *env); 5 | 6 | #define IA_MAX 256 7 | 8 | static enum theft_alloc_res 9 | ia_alloc(struct theft *t, void *env, void **instance) { 10 | (void)env; 11 | 12 | uint8_t *ints = calloc(IA_MAX, sizeof(uint8_t)); 13 | if (ints == NULL) { 14 | return THEFT_ALLOC_ERROR; 15 | } 16 | 17 | for (size_t i = 0; i < IA_MAX; i++) { 18 | uint8_t v = theft_random_bits(t, 8); 19 | if (v == 0) { 20 | break; 21 | } 22 | ints[i] = v; 23 | } 24 | 25 | *instance = ints; 26 | //ia_print(stdout, ints, NULL); printf("\n"); 27 | return THEFT_ALLOC_OK; 28 | } 29 | 30 | static void ia_free(void *instance, void *env) { 31 | (void)env; 32 | free(instance); 33 | } 34 | 35 | static void ia_print(FILE *f, const void *instance, void *env) { 36 | uint8_t *ia = (uint8_t *)instance; 37 | (void)env; 38 | fprintf(f, "["); 39 | for (size_t i = 0; i < IA_MAX; i++) { 40 | if (ia[i] == 0) { 41 | break; 42 | } 43 | fprintf(f, "%u, ", ia[i]); 44 | if ((i > 0) && (i % 16) == 0) { 45 | fprintf(f, "\n"); 46 | } 47 | } 48 | fprintf(f, "]"); 49 | } 50 | 51 | struct theft_type_info ia_info = { 52 | .alloc = ia_alloc, 53 | .free = ia_free, 54 | .print = ia_print, 55 | .autoshrink_config = { 56 | .enable = true, 57 | }, 58 | }; 59 | 60 | -------------------------------------------------------------------------------- /test/test_theft_autoshrink_int_array.h: -------------------------------------------------------------------------------- 1 | #ifndef TEST_THEFT_AUTOSHRINK_INT_ARRAY_H 2 | #define TEST_THEFT_AUTOSHRINK_INT_ARRAY_H 3 | 4 | #include 5 | 6 | extern struct theft_type_info ia_info; 7 | 8 | #endif 9 | -------------------------------------------------------------------------------- /test/test_theft_autoshrink_ll.c: -------------------------------------------------------------------------------- 1 | #include "test_theft.h" 2 | #include "test_theft_autoshrink_ll.h" 3 | 4 | static void 5 | ll_free(void *instance, void *env); 6 | static void ll_print(FILE *f, const void *instance, void *env); 7 | 8 | /* Generate a linked list of uint8_t values. Before each 9 | * link, get 3 random bits -- if 0, then end the list. */ 10 | static enum theft_alloc_res 11 | ll_alloc(struct theft *t, void *env, void **instance) { 12 | (void)env; 13 | struct ll *res = NULL; 14 | struct ll *prev = NULL; 15 | 16 | /* Each link has a 1 in 8 chance of end-of-list */ 17 | while (theft_random_bits(t, 3) != 0x00) { 18 | struct ll *link = calloc(1, sizeof(struct ll)); 19 | if (link == NULL) { 20 | ll_free(res, NULL); 21 | return THEFT_ALLOC_ERROR; 22 | } 23 | 24 | link->tag = 'L'; 25 | link->value = (uint8_t)theft_random_bits(t, 8); 26 | 27 | if (res == NULL) { 28 | res = link; 29 | prev = link; 30 | } else { 31 | prev->next = link; 32 | prev = link; 33 | } 34 | } 35 | 36 | *instance = res; 37 | if (0) { 38 | printf(" ALLOC: "); 39 | ll_print(stdout, res, NULL); 40 | printf("\n"); 41 | } 42 | return THEFT_ALLOC_OK; 43 | } 44 | 45 | static void 46 | ll_free(void *instance, void *env) { 47 | (void)env; 48 | struct ll *cur = (struct ll *)instance; 49 | 50 | while (cur) { 51 | assert(cur->tag == 'L'); 52 | struct ll *next = cur->next; 53 | free(cur); 54 | cur = next; 55 | } 56 | } 57 | 58 | static void ll_print(FILE *f, const void *instance, void *env) { 59 | (void)env; 60 | const struct ll *cur = (struct ll *)instance; 61 | 62 | fprintf(f, "["); 63 | while (cur) { 64 | assert(cur->tag == 'L'); 65 | const struct ll *next = cur->next; 66 | fprintf(f, "%u ", cur->value); 67 | cur = next; 68 | } 69 | fprintf(f, "]"); 70 | } 71 | 72 | struct theft_type_info ll_info = { 73 | .alloc = ll_alloc, 74 | .free = ll_free, 75 | .print = ll_print, 76 | .autoshrink_config = { 77 | .enable = true, 78 | .print_mode = THEFT_AUTOSHRINK_PRINT_ALL, 79 | }, 80 | }; 81 | -------------------------------------------------------------------------------- /test/test_theft_autoshrink_ll.h: -------------------------------------------------------------------------------- 1 | #ifndef TEST_THEFT_AUTOSHRINK_LL_H 2 | #define TEST_THEFT_AUTOSHRINK_LL_H 3 | 4 | #include 5 | 6 | struct ll { 7 | char tag; 8 | uint8_t value; 9 | struct ll *next; 10 | }; 11 | 12 | extern struct theft_type_info ll_info; 13 | 14 | #endif 15 | -------------------------------------------------------------------------------- /test/test_theft_aux.c: -------------------------------------------------------------------------------- 1 | #include "test_theft.h" 2 | 3 | struct a_squared_env { 4 | struct theft_print_trial_result_env print_env; 5 | bool found; 6 | }; 7 | 8 | /* Property: a^2 <= 12345 */ 9 | static enum theft_trial_res 10 | prop_a_squared_lte_fixed(struct theft *t, void *arg1) { 11 | void *arg = (void *)arg1; 12 | (void)t; 13 | int8_t a = *(int8_t *)arg; 14 | uint16_t b = 12345; 15 | const size_t aa = a * a; 16 | 17 | return (aa <= b) 18 | ? THEFT_TRIAL_PASS 19 | : THEFT_TRIAL_FAIL; 20 | } 21 | 22 | static enum theft_hook_trial_post_res 23 | fixed_expected_failure_trial_post(const struct theft_hook_trial_post_info *info, 24 | void *penv) { 25 | struct a_squared_env *env = (struct a_squared_env *)penv; 26 | int8_t a = *(int8_t *)info->args[0]; 27 | 28 | if (greatest_get_verbosity() > 0) { 29 | printf("### trial_post: res %s for [arg: %d (%p)]\n\n", 30 | theft_trial_res_str(info->result), 31 | a, (void *)info->args[0]); 32 | } 33 | 34 | /* Set found if the min failure is found. */ 35 | if (info->result == THEFT_TRIAL_FAIL) { 36 | int8_t a = *(int8_t *)info->args[0]; 37 | //printf("FAILURE: %d\n", a); 38 | if (a == 112 || a == -112) { // min failure: 112 * 112 > 12345 39 | env->found = true; 40 | } 41 | } 42 | 43 | return THEFT_HOOK_TRIAL_POST_CONTINUE; 44 | } 45 | 46 | static enum theft_hook_shrink_trial_post_res 47 | fixed_log_shrink_trial_post(const struct theft_hook_shrink_trial_post_info *info, 48 | void *env) { 49 | (void)env; 50 | 51 | int8_t a = *(int8_t *)info->args[0]; 52 | 53 | if (greatest_get_verbosity() > 0) { 54 | printf("### shrink_trial_post: res %s for arg %u [arg: %d (%p)]\n\n", 55 | theft_trial_res_str(info->result), info->arg_index, 56 | a, (void *)info->args[0]); 57 | } 58 | return THEFT_HOOK_SHRINK_TRIAL_POST_CONTINUE; 59 | } 60 | 61 | TEST a_squared_lte_fixed(void) { 62 | theft_seed seed = theft_seed_of_time(); 63 | 64 | struct a_squared_env env; 65 | memset(&env, 0x00, sizeof(env)); 66 | 67 | struct theft_run_config cfg = { 68 | .name = __func__, 69 | .prop1 = prop_a_squared_lte_fixed, 70 | .type_info = { 71 | theft_get_builtin_type_info(THEFT_BUILTIN_int8_t), 72 | }, 73 | .bloom_bits = 20, 74 | .seed = seed, 75 | .trials = 500, 76 | .hooks = { 77 | .trial_post = fixed_expected_failure_trial_post, 78 | .shrink_trial_post = fixed_log_shrink_trial_post, 79 | .env = &env, 80 | }, 81 | }; 82 | 83 | ASSERT_EQ_FMTm("should find counter-examples", 84 | THEFT_RUN_FAIL, theft_run(&cfg), "%d"); 85 | ASSERTm("Should shrink to a minimal case", env.found); 86 | PASS(); 87 | } 88 | 89 | static enum theft_trial_res 90 | prop_a_squared_lt_b(struct theft *t, void *arg1, void *arg2) { 91 | (void)t; 92 | int8_t a = *(int8_t *)arg1; 93 | uint16_t b = *(uint16_t *)arg2; 94 | const size_t aa = a * a; 95 | 96 | if (0) { 97 | fprintf(stdout, "\n$$ checking (%d^2) < %u ? %d (a^2 = %zd)\n", 98 | a, b, aa < b, aa); 99 | } 100 | 101 | return (aa <= b) 102 | ? THEFT_TRIAL_PASS 103 | : THEFT_TRIAL_FAIL; 104 | } 105 | 106 | static enum theft_hook_trial_post_res 107 | expected_failure_trial_post(const struct theft_hook_trial_post_info *info, 108 | void *penv) { 109 | struct a_squared_env *env = (struct a_squared_env *)penv; 110 | int8_t a = *(int8_t *)info->args[0]; 111 | uint16_t b = *(uint16_t *)info->args[1]; 112 | 113 | if (greatest_get_verbosity() > 0) { 114 | printf("### trial_post: res %s for [args: %d (%p), %u (%p)]\n\n", 115 | theft_trial_res_str(info->result), 116 | a, (void *)info->args[0], 117 | b, (void *)info->args[1]); 118 | } 119 | 120 | if (info->result == THEFT_TRIAL_FAIL) { 121 | int8_t a = *(int8_t *)info->args[0]; 122 | uint16_t b = *(uint16_t *)info->args[1]; 123 | if (a == 1 && b == 0) { 124 | env->found = true; 125 | } 126 | } 127 | 128 | return THEFT_HOOK_TRIAL_POST_CONTINUE; 129 | } 130 | 131 | static enum theft_hook_shrink_trial_post_res 132 | log_shrink_trial_post(const struct theft_hook_shrink_trial_post_info *info, 133 | void *env) { 134 | (void)env; 135 | 136 | int8_t a = *(int8_t *)info->args[0]; 137 | uint16_t b = *(uint16_t *)info->args[1]; 138 | 139 | if (greatest_get_verbosity() > 0) { 140 | printf("### shrink_trial_post: res %s for arg %u [args: %d (%p), %u (%p)]\n\n", 141 | theft_trial_res_str(info->result), info->arg_index, 142 | a, (void *)info->args[0], 143 | b, (void *)info->args[1]); 144 | } 145 | return THEFT_HOOK_SHRINK_TRIAL_POST_CONTINUE; 146 | } 147 | 148 | TEST a_squared_lt_b(void) { 149 | theft_seed seed = theft_seed_of_time(); 150 | 151 | struct a_squared_env env; 152 | memset(&env, 0x00, sizeof(env)); 153 | 154 | struct theft_run_config cfg = { 155 | .name = __func__, 156 | .prop2 = prop_a_squared_lt_b, 157 | .type_info = { 158 | theft_get_builtin_type_info(THEFT_BUILTIN_int8_t), 159 | theft_get_builtin_type_info(THEFT_BUILTIN_uint16_t), 160 | }, 161 | .bloom_bits = 20, 162 | .seed = seed, 163 | .trials = 500, 164 | .hooks = { 165 | .trial_post = expected_failure_trial_post, 166 | .shrink_trial_post = log_shrink_trial_post, 167 | .env = &env, 168 | }, 169 | }; 170 | 171 | ASSERT_EQ_FMTm("should find counter-examples", 172 | THEFT_RUN_FAIL, theft_run(&cfg), "%d"); 173 | ASSERTm("Should shrink to a minimal case", env.found); 174 | PASS(); 175 | } 176 | 177 | static enum theft_trial_res prop_pass(struct theft *t, void *arg1) { 178 | uint64_t *v = (uint64_t *)arg1; 179 | (void)t; 180 | (void)v; 181 | return THEFT_TRIAL_PASS; 182 | } 183 | 184 | TEST pass_autoscaling(void) { 185 | struct theft_run_config cfg = { 186 | .name = __func__, 187 | .prop1 = prop_pass, 188 | .type_info = { theft_get_builtin_type_info(THEFT_BUILTIN_uint64_t) }, 189 | .trials = 1000000, 190 | }; 191 | 192 | enum theft_run_res res = theft_run(&cfg); 193 | 194 | /* This test needs to be checked by visual inspection -- it should 195 | * look something like this: 196 | * 197 | * == PROP 'pass_autoscaling': 1000000 trials, seed 0x00a600d64b175eed 198 | * .........d.............d.d...d.............d........................... 199 | * ................................d..(PASS x 100).dddd(DUP x 10)d.dd.d.dd 200 | * .d.d.d(DUP x 100)d.......d.......d........d.......d........d......d.... 201 | * ....d......d.......(DUP x 1000)d............................ 202 | * (PASS x 10000).d.dd.dd.dd.d(DUP x 10000)d..d..d.dd.ddd.d(DUP x 100000)d 203 | * .ddddddd 204 | * == PASS 'pass_autoscaling': pass 136572, fail 0, skip 0, dup 863428 205 | * 206 | * where the acceptance criteria is that it prints `PASS x 10000` 207 | * rather than million different '.' and 'd' characters along the 208 | * way. (If it fails, it should completely overwhelm the test output 209 | * in an obvious way.) */ 210 | (void)res; 211 | 212 | PASS(); 213 | } 214 | 215 | static enum theft_trial_res 216 | prop_check_and_update_magic(struct theft *t, void *arg1) { 217 | uint64_t *unused = (uint64_t *)arg1; 218 | (void)unused; 219 | 220 | uint64_t *magic = (uint64_t *)theft_hook_get_env(t); 221 | if (magic == NULL || *magic != 0xABED) { 222 | return THEFT_TRIAL_FAIL; 223 | } 224 | 225 | *magic = 0xfa1afel; /* update the magic value */ 226 | return THEFT_TRIAL_PASS; 227 | } 228 | 229 | TEST get_hook_env(void) { 230 | uint64_t magic = 0xABED; 231 | 232 | struct theft_run_config cfg = { 233 | .name = __func__, 234 | .prop1 = prop_check_and_update_magic, 235 | .type_info = { theft_get_builtin_type_info(THEFT_BUILTIN_uint64_t) }, 236 | .trials = 1, 237 | .hooks = { 238 | .env = (void *)&magic, 239 | }, 240 | }; 241 | enum theft_run_res res = theft_run(&cfg); 242 | 243 | ASSERT_EQ_FMTm("should get pointer to the magic value", 244 | THEFT_RUN_PASS, res, "%d"); 245 | ASSERT_EQ_FMTm("should update the magic value", 246 | (uint64_t)0xfa1afel, magic, "%" PRIx64); 247 | 248 | PASS(); 249 | } 250 | 251 | struct gap_env { 252 | uint8_t tag; 253 | uint64_t limit; 254 | bool used; 255 | }; 256 | 257 | static enum theft_alloc_res 258 | gap_alloc(struct theft *t, void *info_env, void **output) { 259 | (void)info_env; 260 | struct gap_env *env = (struct gap_env *)theft_hook_get_env(t); 261 | if (env->tag != 'G') { return THEFT_ALLOC_ERROR; } 262 | uint64_t *res = malloc(sizeof(*res)); 263 | if (res == NULL) { return THEFT_ALLOC_ERROR; } 264 | *res = theft_random_bits(t, 64) % env->limit; 265 | env->used = true; 266 | 267 | *output = res; 268 | return THEFT_ALLOC_OK; 269 | } 270 | 271 | static void gap_print(FILE *f, const void *instance, void *env) { 272 | (void)env; 273 | uint64_t *v = (uint64_t *)instance; 274 | fprintf(f, "%" PRIu64, *v); 275 | } 276 | 277 | struct theft_type_info gap_info = { 278 | .alloc = gap_alloc, 279 | .free = theft_generic_free_cb, 280 | .print = gap_print, 281 | }; 282 | 283 | /* Generate a uint64_t, but limit it to < 1000, to 284 | * verify that the hook_env is passed along correctly. */ 285 | TEST gen_and_print(void) { 286 | uint64_t seed = theft_seed_of_time(); 287 | 288 | struct gap_env env = { 289 | .tag = 'G', 290 | .limit = 1000, 291 | }; 292 | 293 | enum theft_generate_res res = 294 | theft_generate(stdout, seed, &gap_info, &env); 295 | 296 | ASSERT_EQ_FMT(THEFT_GENERATE_OK, res, "%d"); 297 | ASSERT(env.used); 298 | 299 | PASS(); 300 | } 301 | 302 | SUITE(aux) { 303 | // builtins 304 | RUN_TEST(a_squared_lte_fixed); 305 | RUN_TEST(a_squared_lt_b); 306 | 307 | /* Tests for other misc. aux stuff */ 308 | RUN_TEST(pass_autoscaling); 309 | RUN_TEST(get_hook_env); 310 | RUN_TEST(gen_and_print); 311 | } 312 | -------------------------------------------------------------------------------- /test/test_theft_bloom.c: -------------------------------------------------------------------------------- 1 | #include "test_theft.h" 2 | #include "theft_bloom.h" 3 | 4 | TEST all_marked_should_remain_marked(size_t limit) { 5 | struct theft_bloom *b = theft_bloom_init(NULL); 6 | 7 | char buf[32]; 8 | for (size_t i = 0; i < limit; i++) { 9 | size_t used = snprintf(buf, sizeof(buf), "key%zd\n", i); 10 | assert(used < sizeof(buf)); 11 | ASSERTm("marking should not fail", 12 | theft_bloom_mark(b, (uint8_t *)buf, used)); 13 | } 14 | 15 | for (size_t i = 0; i < limit; i++) { 16 | size_t used = snprintf(buf, sizeof(buf), "key%zd\n", i); 17 | assert(used < sizeof(buf)); 18 | ASSERTm("marked became unmarked", 19 | theft_bloom_check(b, (uint8_t *)buf, used)); 20 | } 21 | 22 | theft_bloom_free(b); 23 | PASS(); 24 | } 25 | 26 | SUITE(bloom) { 27 | RUN_TESTp(all_marked_should_remain_marked, 10); 28 | RUN_TESTp(all_marked_should_remain_marked, 1000); 29 | RUN_TESTp(all_marked_should_remain_marked, 100000); 30 | } 31 | -------------------------------------------------------------------------------- /test/test_theft_prng.c: -------------------------------------------------------------------------------- 1 | #include "test_theft.h" 2 | #include "theft_random.h" 3 | 4 | /* These are included to allocate a valid theft handle, but 5 | * this file is only testing its random number generation 6 | * and buffering. */ 7 | #include "theft_run.h" 8 | #include "test_theft_autoshrink_ll.h" 9 | 10 | static enum theft_trial_res unused(struct theft *t, void *arg1) { 11 | struct ll *v = (struct ll *)arg1; 12 | (void)t; 13 | (void)v; 14 | return THEFT_TRIAL_ERROR; 15 | } 16 | 17 | static struct theft *init(void) { 18 | struct theft *t = NULL; 19 | struct theft_run_config cfg = { 20 | /* These aren't actually used, just defined so that 21 | * theft_run_init doesn't return an error. */ 22 | .prop1 = unused, 23 | .type_info = { &ll_info }, 24 | }; 25 | 26 | enum theft_run_init_res res = theft_run_init(&cfg, &t); 27 | if (res == THEFT_RUN_INIT_OK) { 28 | return t; 29 | } else { 30 | return NULL; 31 | } 32 | } 33 | 34 | TEST prng_should_return_same_series_from_same_seeds(void) { 35 | theft_seed seeds[8]; 36 | theft_seed values[8][8]; 37 | 38 | struct theft *t = init(); ASSERT(t); 39 | 40 | /* Set for deterministic start */ 41 | theft_random_set_seed(t, 0xabad5eed); 42 | for (int i = 0; i < 8; i++) { 43 | seeds[i] = theft_random(t); 44 | } 45 | 46 | /* Populate value tables. */ 47 | for (int s = 0; s < 8; s++) { 48 | theft_random_set_seed(t, seeds[s]); 49 | for (int i = 0; i < 8; i++) { 50 | values[s][i] = theft_random(t); 51 | } 52 | } 53 | 54 | /* Check values. */ 55 | for (int s = 0; s < 8; s++) { 56 | theft_random_set_seed(t, seeds[s]); 57 | for (int i = 0; i < 8; i++) { 58 | ASSERT_EQ(values[s][i], theft_random(t)); 59 | } 60 | } 61 | theft_run_free(t); 62 | PASS(); 63 | } 64 | 65 | TEST basic_sampling(uint64_t limit) { 66 | struct theft_run_config cfg; 67 | memset(&cfg, 0, sizeof(cfg)); 68 | struct theft *t = init(); ASSERT(t); 69 | 70 | for (uint64_t seed = 0; seed < limit; seed++) { 71 | theft_random_set_seed(t, seed); 72 | uint64_t num = theft_random(t); 73 | 74 | theft_random_set_seed(t, seed); 75 | uint64_t num2 = theft_random(t); 76 | 77 | ASSERT_EQ_FMT(num, num2, "%" PRIx64); 78 | } 79 | 80 | theft_run_free(t); 81 | PASS(); 82 | } 83 | 84 | TEST bit_sampling_two_bytes(uint64_t limit) { 85 | struct theft_run_config cfg; 86 | memset(&cfg, 0, sizeof(cfg)); 87 | struct theft *t = init(); ASSERT(t); 88 | 89 | for (uint64_t seed = 0; seed < limit; seed++) { 90 | theft_random_set_seed(t, seed); 91 | uint16_t a = (uint16_t)(theft_random(t) & 0xFFFF); 92 | 93 | theft_random_set_seed(t, seed); 94 | uint64_t b0 = 0; 95 | 96 | for (uint8_t i = 0; i < 2; i++) { 97 | uint64_t byte = (uint8_t)theft_random_bits(t, 8); 98 | b0 |= (byte << (8L*i)); 99 | } 100 | uint16_t b = (uint16_t)(b0 & 0xFFFF); 101 | 102 | ASSERT_EQ_FMT(a, b, "0x%04x"); 103 | } 104 | 105 | theft_run_free(t); 106 | PASS(); 107 | } 108 | 109 | TEST bit_sampling_bytes(uint64_t limit) { 110 | struct theft_run_config cfg; 111 | memset(&cfg, 0, sizeof(cfg)); 112 | struct theft *t = init(); ASSERT(t); 113 | 114 | for (uint64_t seed = 0; seed < limit; seed++) { 115 | theft_random_set_seed(t, seed); 116 | uint64_t a0 = theft_random(t); 117 | uint64_t a1 = theft_random(t); 118 | 119 | theft_random_set_seed(t, seed); 120 | uint64_t b0 = 0; 121 | 122 | for (uint8_t i = 0; i < 8; i++) { 123 | uint64_t byte = (uint8_t)theft_random_bits(t, 8); 124 | b0 |= (byte << (8L*i)); 125 | } 126 | uint64_t b1 = 0; 127 | 128 | for (uint8_t i = 0; i < 8; i++) { 129 | uint64_t byte = (uint8_t)theft_random_bits(t, 8); 130 | b1 |= (byte << (8L*i)); 131 | } 132 | 133 | ASSERT_EQ_FMT(a0, b0, "%" PRIu64); 134 | ASSERT_EQ_FMT(a1, b1, "%" PRIu64); 135 | } 136 | 137 | theft_run_free(t); 138 | PASS(); 139 | } 140 | 141 | TEST bit_sampling_odd_sizes(uint64_t limit) { 142 | struct theft_run_config cfg; 143 | memset(&cfg, 0, sizeof(cfg)); 144 | struct theft *t = init(); ASSERT(t); 145 | 146 | for (uint64_t seed = 0; seed < limit; seed++) { 147 | theft_random_set_seed(t, seed); 148 | uint64_t a0 = theft_random(t); 149 | uint64_t a1 = theft_random(t); 150 | 151 | theft_random_set_seed(t, seed); 152 | uint64_t b_11 = theft_random_bits(t, 11); 153 | uint64_t b_13 = theft_random_bits(t, 13); 154 | uint64_t b_15 = theft_random_bits(t, 15); 155 | uint64_t b_17 = theft_random_bits(t, 17); 156 | uint64_t b_19 = theft_random_bits(t, 19); 157 | 158 | uint64_t b0 = (0L 159 | | (b_11 << 0) 160 | | (b_13 << 11) 161 | | (b_15 << (11 + 13)) 162 | | (b_17 << (11 + 13 + 15)) 163 | | (b_19 << (11 + 13 + 15 + 17))); 164 | 165 | uint64_t b1 = (b_19 >> 8L); 166 | uint64_t mask_a1 = a1 & ((1L << 11L) - 1); 167 | 168 | // check that first 64 bits and lower 11 of second uint64_t match 169 | ASSERT_EQ_FMT(a0, b0, "0x%08" PRIx64); 170 | ASSERT_EQ_FMT(mask_a1, b1, "0x%08" PRIx64); 171 | } 172 | 173 | theft_run_free(t); 174 | PASS(); 175 | } 176 | 177 | TEST seed_with_upper_32_bits_masked_should_produce_different_value(void) { 178 | uint64_t seed = 0x15a600d64b175eedLL; 179 | uint64_t values[3]; 180 | 181 | struct theft *t = init(); ASSERT(t); 182 | 183 | theft_random_set_seed(t, seed); 184 | values[0] = theft_random_bits(t, 64); 185 | 186 | theft_random_set_seed(t, seed | 0xFFFFFFFF00000000L); 187 | values[1] = theft_random_bits(t, 64); 188 | 189 | theft_random_set_seed(t, seed &~ 0xFFFFFFFF00000000L); 190 | values[2] = theft_random_bits(t, 64); 191 | 192 | ASSERT(values[0] != values[1]); 193 | ASSERT(values[0] != values[2]); 194 | 195 | theft_run_free(t); 196 | PASS(); 197 | } 198 | 199 | #if THEFT_USE_FLOATING_POINT 200 | TEST check_random_choice_0(void) { 201 | struct theft *t = init(); ASSERT(t); 202 | 203 | const size_t trials = 10000; 204 | for (size_t i = 0; i < trials; i++) { 205 | uint64_t v = theft_random_choice(t, 0); 206 | ASSERT_EQ_FMTm("limit of 0 should always return 0", 207 | (uint64_t)0, v, "%" PRIu64); 208 | } 209 | 210 | theft_run_free(t); 211 | PASS(); 212 | } 213 | 214 | TEST check_random_choice_distribution__slow(uint64_t limit, float tolerance) { 215 | struct theft *t = init(); ASSERT(t); 216 | 217 | size_t *counts = calloc(limit, sizeof(size_t)); 218 | 219 | const size_t trials = limit < 10000 ? 10000000 : 1000 * limit; 220 | for (size_t i = 0; i < trials; i++) { 221 | uint64_t v = theft_random_choice(t, limit); 222 | ASSERT(v < limit); 223 | counts[v]++; 224 | } 225 | 226 | /* Count, if the trials were perfectly evenly distributed */ 227 | size_t even = trials / (double)limit; 228 | 229 | for (size_t i = 0; i < limit; i++) { 230 | size_t count = counts[i]; 231 | ASSERT_IN_RANGEm("distribution is too uneven", 232 | even, count, tolerance * even); 233 | } 234 | 235 | theft_run_free(t); 236 | free(counts); 237 | PASS(); 238 | } 239 | #endif 240 | 241 | SUITE(prng) { 242 | RUN_TEST(prng_should_return_same_series_from_same_seeds); 243 | 244 | for (volatile size_t limit = 100; limit < 100000; limit *= 10) { 245 | RUN_TESTp(basic_sampling, limit); 246 | RUN_TESTp(bit_sampling_two_bytes, limit); 247 | RUN_TESTp(bit_sampling_bytes, limit); 248 | RUN_TESTp(bit_sampling_odd_sizes, limit); 249 | } 250 | 251 | RUN_TEST(seed_with_upper_32_bits_masked_should_produce_different_value); 252 | 253 | #if THEFT_USE_FLOATING_POINT 254 | RUN_TEST(check_random_choice_0); 255 | 256 | for (volatile uint64_t limit = 1; limit < 300; limit++) { 257 | RUN_TESTp(check_random_choice_distribution__slow, limit, 0.05); 258 | } 259 | 260 | /* Relax the tolerance for these a bit, because we aren't running 261 | * enough trials to smooth out the distribution. */ 262 | RUN_TESTp(check_random_choice_distribution__slow, 10000, 0.20); 263 | #endif 264 | } 265 | --------------------------------------------------------------------------------