├── pyobf ├── __init__.py ├── utils.py ├── test.py ├── circuit.py ├── bp.py ├── obfuscator.py ├── sz_bp.py └── main.py ├── Makefile.am ├── circuits ├── clean.sh ├── id.circ ├── not.circ ├── and.circ ├── or.circ ├── xor.circ ├── twoxors.circ ├── nand.circ ├── andxor.circ ├── threeands.circ ├── fourxors.circ ├── xorand.circ ├── fourandsnot.circ ├── twoands.circ ├── fourands.circ ├── conjunction.cry ├── threexors.circ ├── util.py ├── util.cry ├── conjunction.py ├── point.py └── point-json.py ├── obfuscator ├── src ├── Makefile.am ├── utils.h ├── thpool_fns.h ├── utils.c ├── obfuscator.h ├── thpool_fns.c ├── thpool.h ├── obfuscator.c └── thpool.c ├── pywrapper ├── pyutils.h ├── pyutils.cpp └── obfuscator_wrapper.cpp ├── .gitignore ├── README.md ├── setup.py ├── t └── __init__.py ├── configure.ac └── LICENSE.txt /pyobf/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /Makefile.am: -------------------------------------------------------------------------------- 1 | AUTOMAKE_OPTIONS = foreign -Wall 2 | SUBDIRS = src 3 | -------------------------------------------------------------------------------- /circuits/clean.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | rm -rf *.obf.* 4 | -------------------------------------------------------------------------------- /circuits/id.circ: -------------------------------------------------------------------------------- 1 | : nins 1 2 | : depth 1 3 | # TEST 0 0 4 | # TEST 1 1 5 | 0 input 6 | 1 output ID 0 7 | -------------------------------------------------------------------------------- /circuits/not.circ: -------------------------------------------------------------------------------- 1 | : nins 1 2 | : depth 1 3 | # TEST 0 1 4 | # TEST 1 0 5 | 0 input 6 | 1 output NOT 0 7 | -------------------------------------------------------------------------------- /obfuscator: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python2 2 | 3 | import pyobf 4 | from pyobf.main import main 5 | import sys 6 | 7 | success = main() 8 | sys.exit(not success) 9 | -------------------------------------------------------------------------------- /circuits/and.circ: -------------------------------------------------------------------------------- 1 | : nins 2 2 | : depth 1 3 | # TEST 00 0 4 | # TEST 01 0 5 | # TEST 10 0 6 | # TEST 11 1 7 | 0 input 8 | 1 input 9 | 2 output AND 0 1 10 | -------------------------------------------------------------------------------- /circuits/or.circ: -------------------------------------------------------------------------------- 1 | : nins 2 2 | : depth 1 3 | # TEST 00 0 4 | # TEST 01 1 5 | # TEST 10 1 6 | # TEST 11 1 7 | 0 input 8 | 1 input 9 | 2 output OR 0 1 10 | -------------------------------------------------------------------------------- /circuits/xor.circ: -------------------------------------------------------------------------------- 1 | : nins 2 2 | : depth 1 3 | # TEST 00 0 4 | # TEST 01 1 5 | # TEST 10 1 6 | # TEST 11 0 7 | 0 input 8 | 1 input 9 | 2 output XOR 0 1 10 | -------------------------------------------------------------------------------- /circuits/twoxors.circ: -------------------------------------------------------------------------------- 1 | : nins 3 2 | : depth 2 3 | # TEST 000 0 4 | # TEST 111 1 5 | 0 input 6 | 1 input 7 | 2 input 8 | 3 gate XOR 0 1 9 | 4 output XOR 3 2 10 | -------------------------------------------------------------------------------- /circuits/nand.circ: -------------------------------------------------------------------------------- 1 | : nins 2 2 | : depth 2 3 | # TEST 00 1 4 | # TEST 01 1 5 | # TEST 10 1 6 | # TEST 11 0 7 | 0 input 8 | 1 input 9 | 2 gate AND 0 1 10 | 3 output NOT 2 11 | -------------------------------------------------------------------------------- /circuits/andxor.circ: -------------------------------------------------------------------------------- 1 | : nins 3 2 | : depth 2 3 | # TEST 000 0 4 | # TEST 001 1 5 | # TEST 010 0 6 | # TEST 111 0 7 | 0 input 8 | 1 input 9 | 2 input 10 | 3 gate AND 0 1 11 | 4 output XOR 3 2 12 | -------------------------------------------------------------------------------- /circuits/threeands.circ: -------------------------------------------------------------------------------- 1 | : nins 4 2 | : depth 2 3 | # TEST 0000 0 4 | # TEST 1111 1 5 | 0 input 6 | 1 input 7 | 2 input 8 | 3 input 9 | 4 gate AND 0 1 10 | 5 gate AND 2 3 11 | 6 output AND 4 5 12 | -------------------------------------------------------------------------------- /circuits/fourxors.circ: -------------------------------------------------------------------------------- 1 | : nins 5 2 | : depth 4 3 | # TEST 00000 0 4 | 0 input 5 | 1 input 6 | 2 input 7 | 3 input 8 | 4 input 9 | 5 gate XOR 0 1 10 | 6 gate XOR 5 2 11 | 7 gate XOR 6 3 12 | 8 output XOR 7 4 13 | -------------------------------------------------------------------------------- /circuits/xorand.circ: -------------------------------------------------------------------------------- 1 | : nins 3 2 | : depth 2 3 | # TEST 000 0 4 | # TEST 001 0 5 | # TEST 010 0 6 | # TEST 011 1 7 | # TEST 111 0 8 | 0 input 9 | 1 input 10 | 2 input 11 | 3 gate XOR 0 1 12 | 4 output AND 3 2 13 | -------------------------------------------------------------------------------- /circuits/fourandsnot.circ: -------------------------------------------------------------------------------- 1 | : nins 5 2 | : depth 4 3 | # TEST 00000 1 4 | # TEST 11111 0 5 | 0 input 6 | 1 input 7 | 2 input 8 | 3 input 9 | 4 input 10 | 5 gate AND 0 1 11 | 6 gate AND 5 2 12 | 7 gate AND 6 3 13 | 8 gate AND 7 4 14 | 9 output NOT 8 15 | -------------------------------------------------------------------------------- /circuits/twoands.circ: -------------------------------------------------------------------------------- 1 | : nins 3 2 | : depth 2 3 | # TEST 000 0 4 | # TEST 001 0 5 | # TEST 010 0 6 | # TEST 011 0 7 | # TEST 100 0 8 | # TEST 101 0 9 | # TEST 110 0 10 | # TEST 111 1 11 | 0 input 12 | 1 input 13 | 2 input 14 | 3 gate AND 0 1 15 | 4 output AND 3 2 16 | -------------------------------------------------------------------------------- /circuits/fourands.circ: -------------------------------------------------------------------------------- 1 | : nins 5 2 | : depth 4 3 | # TEST 00000 0 4 | # TEST 00100 0 5 | # TEST 10101 0 6 | # TEST 11111 1 7 | 0 input 8 | 1 input 9 | 2 input 10 | 3 input 11 | 4 input 12 | 5 gate AND 0 1 13 | 6 gate AND 5 2 14 | 7 gate AND 6 3 15 | 8 output AND 7 4 16 | -------------------------------------------------------------------------------- /circuits/conjunction.cry: -------------------------------------------------------------------------------- 1 | main secret bits = (bits && mask) == val where 2 | mask = [c != '?' | c <- secret] 3 | val = [c == '1' | c <- secret] 4 | 5 | property main_correct chunks 6 | = (chunks@0 == 0b0011 && chunks@2 == 0b0110) 7 | == main "0011????0110????" (join chunks) 8 | 9 | valid _ = True -------------------------------------------------------------------------------- /src/Makefile.am: -------------------------------------------------------------------------------- 1 | AUTOMAKE_OPTIONS = foreign -Wall 2 | AM_CFLAGS = $(COMMON_CFLAGS) $(EXTRA_CFLAGS) 3 | AM_LDFLAGS = -lgomp 4 | 5 | lib_LTLIBRARIES=libobf.la 6 | 7 | libobf_la_SOURCES = obfuscator.c thpool.c thpool_fns.c utils.c 8 | libobf_la_LDFLAGS = -release 0.0.0 -no-undefined 9 | 10 | pkgincludesubdir = $(includedir)/obf 11 | pkgincludesub_HEADERS = obfuscator.h 12 | 13 | -------------------------------------------------------------------------------- /src/utils.h: -------------------------------------------------------------------------------- 1 | #ifndef __OBFUSCATION__UTILS_H__ 2 | #define __OBFUSCATION__UTILS_H__ 3 | 4 | #include 5 | #include 6 | 7 | #define AES_SEED_BYTE_SIZE 32 8 | 9 | double 10 | current_time(void); 11 | 12 | FILE * 13 | open_file(const char *dir, const char *file, const char *mode); 14 | 15 | int 16 | load_mpz_scalar(const char *fname, mpz_t x); 17 | 18 | #endif 19 | -------------------------------------------------------------------------------- /circuits/threexors.circ: -------------------------------------------------------------------------------- 1 | : nins 4 2 | : depth 2 3 | # TEST 0000 0 4 | # TEST 0001 1 5 | # TEST 0010 1 6 | # TEST 0011 0 7 | # TEST 0100 1 8 | # TEST 0101 0 9 | # TEST 0110 0 10 | # TEST 0111 1 11 | # TEST 1000 1 12 | # TEST 1001 0 13 | # TEST 1010 0 14 | # TEST 1011 1 15 | # TEST 1100 0 16 | # TEST 1101 1 17 | # TEST 1110 1 18 | # TEST 1111 0 19 | 0 input 20 | 1 input 21 | 2 input 22 | 3 input 23 | 4 gate XOR 0 1 24 | 5 gate XOR 2 3 25 | 6 output XOR 4 5 26 | -------------------------------------------------------------------------------- /pywrapper/pyutils.h: -------------------------------------------------------------------------------- 1 | #ifndef __OBFUSCATION__PYUTILS_H__ 2 | #define __OBFUSCATION__PYUTILS_H__ 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | PyObject * 9 | mpz_to_py(const mpz_t in); 10 | 11 | int 12 | py_to_mpz(mpz_t out, PyObject *in); 13 | 14 | PyObject * 15 | fmpz_to_py(const fmpz_t in); 16 | 17 | int 18 | py_to_fmpz(fmpz_t out, PyObject *in); 19 | 20 | PyObject * 21 | obf_max_mem_usage(PyObject *self, PyObject *args); 22 | 23 | #endif 24 | -------------------------------------------------------------------------------- /pyobf/utils.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import functools, sys 3 | 4 | def clr_error(s): 5 | return '\x1b[31m%s\x1b[0m' % s 6 | def clr_warn(s): 7 | return '\x1b[33m%s\x1b[0m' % s 8 | def clr_ok(s): 9 | return '\x1b[32m%s\x1b[0m' % s 10 | 11 | def logger(s, end='\n', verbose=False): 12 | if verbose: 13 | print(s, end=end, file=sys.stderr) 14 | sys.stderr.flush() 15 | 16 | def make_logger(verbose): 17 | return functools.partial(logger, verbose=verbose) 18 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Object files 2 | *.o 3 | *.ko 4 | 5 | # Libraries 6 | *.lib 7 | *.a 8 | 9 | # Shared objects (inc. Windows DLLs) 10 | *.dll 11 | *.so 12 | *.so.* 13 | *.dylib 14 | 15 | # Executables 16 | *.exe 17 | *.out 18 | *.app 19 | 20 | # Python specific 21 | *.pyc 22 | 23 | # Other 24 | build/ 25 | obfuscator.egg-info/ 26 | *circ.obf* 27 | *json.obf* 28 | evaluate 29 | tags 30 | test_mlm 31 | 32 | Makefile 33 | Makefile.in 34 | *.m4 35 | autom4te.cache 36 | *.log 37 | *.status 38 | configure 39 | libtool 40 | .deps 41 | .libs 42 | *.la 43 | *.lo 44 | src/config.h 45 | src/config.h.in 46 | src/stamp-h1 47 | 48 | *.circ 49 | *.acirc 50 | *.json 51 | 52 | dist/ 53 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Cryptographic Program Obfuscation 2 | 3 | ## Building 4 | 5 | Run the following: 6 | 7 | ``` 8 | autoreconf -i 9 | ./configure 10 | make 11 | sudo make install 12 | ``` 13 | 14 | This installs the underlying obfuscation library `libobf` to your system. To 15 | install the python front-end, proceed as follows: 16 | 17 | ``` 18 | cd python 19 | python2 setup.py test 20 | ``` 21 | 22 | This runs a bunch of test, all of which should hopefully pass. 23 | 24 | You can then run the obfuscator by running 25 | ``` 26 | ./obfuscator obf --test circuits/and.circ --secparam 16 -v 27 | ``` 28 | 29 | ## Contact 30 | 31 | For any questions/comments, please e-mail amaloz at galois dot com. 32 | -------------------------------------------------------------------------------- /src/thpool_fns.h: -------------------------------------------------------------------------------- 1 | #ifndef THPOOL_FNS 2 | #define THPOOL_FNS 3 | 4 | #include "utils.h" 5 | 6 | #include 7 | #include 8 | #include 9 | 10 | struct encode_elem_s { 11 | const mmap_vtable *vtable; 12 | mmap_ro_sk sk; 13 | int n; 14 | fmpz_t *plaintext; 15 | int *group; 16 | mmap_enc *enc; 17 | aes_randstate_t *rand; 18 | }; 19 | 20 | void * 21 | thpool_encode_elem(void *vargs); 22 | 23 | struct write_layer_s { 24 | const mmap_vtable *vtable; 25 | const char *dir; 26 | uint64_t n; 27 | mmap_enc_mat_t **enc_mats; 28 | char **names; 29 | long inp; 30 | long idx; 31 | long nrows; 32 | long ncols; 33 | double start; 34 | bool verbose; 35 | }; 36 | 37 | void * 38 | thpool_write_layer(void *vargs); 39 | 40 | struct write_element_s { 41 | const char *dir; 42 | mmap_enc *elem; 43 | char *name; 44 | }; 45 | 46 | #endif 47 | -------------------------------------------------------------------------------- /circuits/util.py: -------------------------------------------------------------------------------- 1 | import math, os, subprocess 2 | 3 | def digit_to_char(digit): 4 | if digit < 10: 5 | return str(digit) 6 | return chr(ord('a') + digit - 10) 7 | 8 | def str_base(number,base): 9 | if number < 0: 10 | return '-' + str_base(-number, base) 11 | (d, m) = divmod(number, base) 12 | if d > 0: 13 | return str_base(d, base) + digit_to_char(m) 14 | return digit_to_char(m) 15 | 16 | def dary_repr(number, d, n): 17 | repr = list(str(str_base(number, d))) 18 | repr = (['0'] * (n - len(repr))) + repr 19 | return "".join(repr) 20 | 21 | # turns a "10" in any base to a 001 000 22 | def digit_dary_repr(num_str, d): 23 | L = [] 24 | for x in num_str: 25 | L.append(format(int(x), 'b').zfill(int(math.ceil(math.log(d, 2))))) 26 | return "".join(L) 27 | 28 | def run(lst): 29 | print('%s' % ' '.join(lst)) 30 | with open(os.devnull, 'w') as fnull: 31 | return subprocess.call(lst, stdout=fnull, stderr=fnull) 32 | 33 | -------------------------------------------------------------------------------- /src/utils.c: -------------------------------------------------------------------------------- 1 | #include "utils.h" 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | #include 11 | #include 12 | 13 | double 14 | current_time(void) 15 | { 16 | struct timeval t; 17 | (void) gettimeofday(&t, NULL); 18 | return (double) (t.tv_sec + (double) (t.tv_usec / 1000000.0)); 19 | } 20 | 21 | FILE * 22 | open_file(const char *dir, const char *file, const char *mode) 23 | { 24 | FILE *fp; 25 | char *fname; 26 | int fnamelen; 27 | 28 | fnamelen = strlen(dir) + strlen(file) + 2; 29 | fname = malloc(fnamelen); 30 | if (fname == NULL) 31 | return NULL; 32 | (void) snprintf(fname, fnamelen, "%s/%s", dir, file); 33 | fp = fopen(fname, mode); 34 | if (fp == NULL) { 35 | fprintf(stderr, "unable to open '%s'\n", fname); 36 | } 37 | free(fname); 38 | return fp; 39 | } 40 | 41 | int 42 | load_mpz_scalar(const char *fname, mpz_t x) 43 | { 44 | FILE *f; 45 | if ((f = fopen(fname, "r")) == NULL) { 46 | perror(fname); 47 | return 1; 48 | } 49 | (void) mpz_inp_raw(x, f); 50 | (void) fclose(f); 51 | return 0; 52 | } 53 | -------------------------------------------------------------------------------- /pywrapper/pyutils.cpp: -------------------------------------------------------------------------------- 1 | #include "pyutils.h" 2 | #include 3 | 4 | PyObject * 5 | mpz_to_py(const mpz_t in) 6 | { 7 | PyObject *outs, *out; 8 | char *buffer; 9 | 10 | buffer = mpz_get_str(NULL, 10, in); 11 | outs = PyString_FromString(buffer); 12 | out = PyNumber_Long(outs); 13 | free(buffer); 14 | return out; 15 | } 16 | 17 | int 18 | py_to_mpz(mpz_t out, PyObject *in) 19 | { 20 | mpz_set_si(out, PyLong_AsLong(in)); 21 | return 0; 22 | } 23 | 24 | PyObject * 25 | fmpz_to_py(const fmpz_t in) 26 | { 27 | PyObject *outs, *out; 28 | char *buffer; 29 | 30 | buffer = fmpz_get_str(NULL, 10, in); 31 | outs = PyString_FromString(buffer); 32 | out = PyNumber_Long(outs); 33 | free(buffer); 34 | return out; 35 | } 36 | 37 | int 38 | py_to_fmpz(fmpz_t out, PyObject *in) 39 | { 40 | mpz_t tmp; 41 | mpz_init(tmp); 42 | mpz_set_si(tmp, PyLong_AsLong(in)); 43 | fmpz_set_mpz(out, tmp); 44 | mpz_clear(tmp); 45 | return 0; 46 | } 47 | 48 | PyObject * 49 | obf_max_mem_usage(PyObject *self, PyObject *args) 50 | { 51 | struct rusage usage; 52 | 53 | (void) getrusage(RUSAGE_SELF, &usage); 54 | (void) fprintf(stderr, "Max memory usage: %ld\n", usage.ru_maxrss); 55 | 56 | Py_RETURN_NONE; 57 | } 58 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python2 2 | 3 | from setuptools import setup, Extension 4 | 5 | library_dirs = [ 6 | 'src/.libs' 7 | ] 8 | 9 | libraries = [ 10 | 'obf', 11 | ] 12 | compile_args = [ 13 | '-O3', 14 | '-Wall', 15 | '-Wextra', 16 | '-Isrc', 17 | ] 18 | 19 | obfuscator = Extension( 20 | 'pyobf._obfuscator', 21 | library_dirs=library_dirs, 22 | libraries=libraries, 23 | extra_compile_args=compile_args, 24 | sources=[ 25 | 'pywrapper/obfuscator_wrapper.cpp', 26 | 'pywrapper/pyutils.cpp', 27 | ] 28 | ) 29 | 30 | setup(name='obfuscator', 31 | author='Alex J. Malozemoff', 32 | author_email='amaloz@cs.umd.edu', 33 | version='0.2a0', 34 | description='Implementation of cryptographic program obfuscation', 35 | license='GPLv2', 36 | url='https://github.com/amaloz/obfuscation', 37 | packages=['pyobf'], 38 | ext_modules=[obfuscator], 39 | scripts=['obfuscator'], 40 | test_suite='t', 41 | classifiers=[ 42 | 'Topic :: Security :: Cryptography', 43 | 'Environment :: Console', 44 | 'Development Status :: 3 - Alpha', 45 | 'Intended Audience :: Science/Research', 46 | 'Operating System :: Unix', 47 | 'License :: OSI Approved :: Free For Educational Use', 48 | 'Programming Language :: C', 49 | 'Programming Language :: Python', 50 | ]) 51 | -------------------------------------------------------------------------------- /src/obfuscator.h: -------------------------------------------------------------------------------- 1 | #ifndef OBFUSCATOR_H 2 | #define OBFUSCATOR_H 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | #define OBFUSCATOR_OK 0 9 | #define OBFUSCATOR_ERR (-1) 10 | 11 | #define OBFUSCATOR_FLAG_NO_RANDOMIZATION 0x01 12 | #define OBFUSCATOR_FLAG_DUAL_INPUT_BP 0x02 13 | #define OBFUSCATOR_FLAG_VERBOSE 0x04 14 | 15 | #ifdef __cplusplus 16 | extern "C" { 17 | #endif 18 | 19 | typedef struct obf_state_s obf_state_t; 20 | 21 | enum mmap_e { MMAP_CLT, MMAP_GGHLITE, MMAP_DUMMY }; 22 | 23 | typedef enum { 24 | ENCODE_LAYER_RANDOMIZATION_TYPE_NONE = 0x00, 25 | ENCODE_LAYER_RANDOMIZATION_TYPE_FIRST = 0x01, 26 | ENCODE_LAYER_RANDOMIZATION_TYPE_MIDDLE = 0x02, 27 | ENCODE_LAYER_RANDOMIZATION_TYPE_LAST = 0x04, 28 | } encode_layer_randomization_flag_t; 29 | 30 | obf_state_t * 31 | obf_init(enum mmap_e type, const char *dir, uint64_t secparam, uint64_t kappa, 32 | uint64_t nzs, uint64_t nthreads, uint64_t ncores, char *seed, 33 | uint64_t flags); 34 | 35 | void 36 | obf_clear(obf_state_t *s); 37 | 38 | int 39 | obf_encode_layer(obf_state_t *s, uint64_t n, int **pows, fmpz_mat_t *mats, 40 | long idx, long inp, encode_layer_randomization_flag_t rflag); 41 | 42 | int 43 | obf_evaluate(enum mmap_e type, char *dir, uint64_t len, uint64_t *input, 44 | uint64_t bplen, uint64_t ncores, bool verbose); 45 | 46 | void 47 | obf_wait(obf_state_t *s); 48 | 49 | #ifdef __cplusplus 50 | } 51 | #endif 52 | 53 | #endif 54 | -------------------------------------------------------------------------------- /circuits/util.cry: -------------------------------------------------------------------------------- 1 | module util where 2 | 3 | type MaxPosn = 26 4 | 5 | constantBase : {base, len, nin} (fin base, fin len, fin nin, base >= 1) 6 | => [nin][len][width (base-1)] -> Bit 7 | constantBase in = [x <= `(base-1) | xs <- in, x <- xs] == ~zero 8 | 9 | adjacentConstantBase : {base, n} (fin n, fin base, base >= 1) 10 | => [n * width (base - 1)] -> Bit 11 | adjacentConstantBase in = [x <= `(base-1) | x <- split `{parts=n, each=width (base - 1)} in] == ~zero 12 | 13 | positionNames : [MaxPosn](String 1) 14 | positionNames = ["a","b","c","d","e","f","g","h","i","j","k","l","m","n","o","p","q","r","s","t","u","v","w","x","y","z"] 15 | 16 | interleavedGrouping : {base, nin} (fin base, base >= 2, 1 <= nin, nin <= MaxPosn) 17 | => [nin * width (base-1)](String 1) 18 | interleavedGrouping = 19 | [ positionNames @ i 20 | | i <- [0 .. nin-1] 21 | , _ <- [1 .. width (base-1)] 22 | ] 23 | 24 | swizzledGrouping : {base, nin} (fin base, base >= 2, 1 <= nin, nin <= MaxPosn) 25 | => [2 * nin * width (base-1)](String 1) 26 | swizzledGrouping = posns # reverse posns where 27 | posns = interleavedGrouping `{base, nin} 28 | 29 | uninterleave : {base, len, nin} (fin base, fin len, fin nin, base >= 1) 30 | => [nin * len * width (base-1)] -> [nin][len][width (base-1)] 31 | uninterleave bits = transpose (split (split bits)) 32 | 33 | unswizzle : {base, len, nin} (fin base, fin len, fin nin, base >= 1) 34 | => [nin * len * width (base-1)] -> [nin][len][width (base-1)] 35 | unswizzle bits = transpose (reverseEveryOther (split (split bits))) 36 | 37 | reverseEveryOther : {a,b,c} fin b => [a][b]c -> [a][b]c 38 | reverseEveryOther xs = [f x | f <- cycle [\x -> x, reverse] | x <- xs] 39 | 40 | cycle : {a,b} (fin a) => [a]b -> [inf]b 41 | cycle xs = xs # cycle xs 42 | -------------------------------------------------------------------------------- /t/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python2 2 | 3 | from __future__ import print_function 4 | 5 | import os, subprocess, sys 6 | 7 | CMD = './obfuscator' 8 | CIRCUIT_PATH = 'circuits' 9 | 10 | yellow = '\x1b[33m' 11 | black = '\x1b[0m' 12 | failure_str = '\t\x1b[31mTest Failed\x1b[0m' 13 | success_str = '\t\x1b[32mTest Succeeded\x1b[0m' 14 | 15 | def print_test(s): 16 | print('%s%s%s' % (yellow, s, black)) 17 | 18 | def run(lst): 19 | print('%s' % ' '.join(lst)) 20 | return subprocess.call(lst) 21 | 22 | def test_bp(): 23 | print_test('Testing bp') 24 | lst = [CMD, "bp", "--test-all", CIRCUIT_PATH] 25 | return run(lst) 26 | 27 | def test_obf(mmap, secparam): 28 | print_test('Testing obfuscation') 29 | lst = [CMD, "obf", "--test-all", CIRCUIT_PATH, "--secparam", str(secparam), 30 | "--mmap", mmap] 31 | return run(lst) 32 | 33 | def test_load(mmap, secparam): 34 | print_test('Testing load') 35 | circuit = 'and.circ' 36 | eval = '00' 37 | path = os.path.join(CIRCUIT_PATH, circuit) 38 | lst = [CMD, "obf", "--test", path, "--secparam", str(secparam), "--mmap", mmap] 39 | r = run(lst) 40 | if r: 41 | return r 42 | lst = [CMD, "obf", "--load-obf", path + ".obf.%d" % secparam, "--mmap", mmap, "--eval", eval] 43 | return run(lst) 44 | 45 | def test(f, *args): 46 | if f(*args): 47 | print(failure_str) 48 | else: 49 | print(success_str) 50 | 51 | def test_all(): 52 | print("TESTING BP") 53 | test(test_bp) 54 | print("TESTING OBFUSCATION") 55 | test(test_obf, "CLT", 16) 56 | test(test_obf, "GGH", 16) 57 | print("TESTING LOAD") 58 | test(test_load, "CLT", 16) 59 | test(test_load, "GGH", 16) 60 | 61 | try: 62 | test_all() 63 | except KeyboardInterrupt: 64 | pass 65 | -------------------------------------------------------------------------------- /configure.ac: -------------------------------------------------------------------------------- 1 | m4_define([version_major], [0]) 2 | m4_define([version_minor], [2]) 3 | m4_define([version_micro], [0]) 4 | 5 | AC_INIT([libobf], [version_major.version_minor.version_micro], [amaloz@cs.umd.edu]) 6 | 7 | AC_CONFIG_HEADERS([src/config.h]) 8 | AC_CONFIG_SRCDIR([./]) 9 | 10 | AC_CONFIG_AUX_DIR([build/autoconf]) 11 | AC_CONFIG_MACRO_DIR([build/autoconf]) 12 | AM_INIT_AUTOMAKE([foreign -Wall -Werror]) 13 | 14 | AC_DEFINE(VERSION_MAJOR, version_major, [libgarble major version]) 15 | AC_DEFINE(VERSION_MINOR, version_minor, [libgarble minor version]) 16 | AC_DEFINE(VERSION_MICRO, version_micro, [libgarble micro version]) 17 | 18 | AM_PROG_AR 19 | 20 | LT_INIT 21 | 22 | AC_PROG_CC 23 | AC_PROG_INSTALL 24 | AC_PROG_LIBTOOL 25 | 26 | AC_ARG_ENABLE(debug, [ --enable-debug Enable assert() statements for debugging.], [enable_debug=yes]) 27 | 28 | CFLAGS= dnl get rid of default -g -O2 29 | COMMON_CFLAGS="-Wall -Wformat -Wformat-security -Wextra -Wunused \ 30 | -Wshadow -Wmissing-prototypes -Wfloat-equal -Wpointer-arith -Wcast-align \ 31 | -Wstrict-prototypes -Wredundant-decls -Wendif-labels -Wcast-qual \ 32 | -std=gnu11 -Wpedantic" 33 | 34 | if test "x$enable_debug" = x"yes"; then 35 | EXTRA_CFLAGS="-O0 -g" 36 | else 37 | EXTRA_CFLAGS="-O3" 38 | AC_DEFINE(NDEBUG,1,[Define whether debugging is enabled]) 39 | fi 40 | AC_SUBST(COMMON_CFLAGS) 41 | AC_SUBST(EXTRA_CFLAGS) 42 | 43 | AC_FUNC_MALLOC 44 | 45 | AC_CHECK_HEADERS([omp.h]) 46 | 47 | AC_SEARCH_LIBS(aes_randinit,aesrand) 48 | if test "x$ac_cv_search_aes_randinit" = "xno"; then 49 | AC_MSG_ERROR([libaesrand not found]) 50 | fi 51 | AC_SEARCH_LIBS(clt_state_new,clt13) 52 | if test "x$ac_cv_search_clt_state_new" = "xno"; then 53 | AC_MSG_ERROR([libclt13 not found]) 54 | fi 55 | AC_SEARCH_LIBS(mmap_enc_mat_init,mmap) 56 | if test "x$ac_cv_search_mmap_enc_mat_init" = "xno"; then 57 | AC_MSG_ERROR([libmmap not found]) 58 | fi 59 | 60 | AC_CONFIG_FILES([Makefile src/Makefile]) 61 | AC_OUTPUT 62 | -------------------------------------------------------------------------------- /circuits/conjunction.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python2 2 | 3 | from __future__ import print_function 4 | import argparse, random, re, sys 5 | from util import * 6 | 7 | def main(argv): 8 | parser = argparse.ArgumentParser(description=''' 9 | Produces the matrix branching program for conjunction on a given bitstring, 10 | where 'bitstring' must contain only characters 0, 1, and ?. 11 | Example: conjunction.py "01??1?11101?01"''') 12 | parser.add_argument('--cryfsm', metavar='PATH', action='store', type=str, 13 | default='cryfsm', 14 | help='set PATH as cryfsm executable') 15 | parser.add_argument('bitstring', action='store', type=str, 16 | help='the bitstring to obfuscate') 17 | 18 | args = parser.parse_args() 19 | bitstring = args.bitstring 20 | r = re.search("[^01?]", bitstring) 21 | if r: 22 | print("error: invalid character '%c' in bitstring" % r.group()) 23 | sys.exit(1) 24 | length = len(bitstring) 25 | fname = '%s.json' % re.escape(bitstring).replace('\\', '') 26 | lst = [args.cryfsm, 'conjunction.cry', '-g', '#', '-e', 'main \"%s\"' % bitstring, 27 | '-o', fname] 28 | try: 29 | run(lst) 30 | except OSError: 31 | print("error: failure when executing cryfsm") 32 | sys.exit(1) 33 | with open(fname, 'r') as f: 34 | line = f.read() 35 | pattern = bitstring.replace('?', '.?') 36 | with open(fname, 'w') as f: 37 | f.write('# TEST %s 1\n' % bitstring.replace('?', '1')) 38 | for _ in xrange(5): 39 | test = random.randint(0, 2 ** length - 1) 40 | string = dary_repr(test, 2, length) 41 | match = re.match(pattern, string) 42 | if match is None or len(match.group()) != length: 43 | result = 0 44 | else: 45 | result = 1 46 | f.write('# TEST %s %d\n' % (string, result)) 47 | f.write(line) 48 | 49 | if __name__ == '__main__': 50 | try: 51 | main(sys.argv) 52 | except KeyboardInterrupt: 53 | pass 54 | -------------------------------------------------------------------------------- /pyobf/test.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | 3 | from pyobf.circuit import ParseException 4 | import pyobf.utils as utils 5 | from pyobf.sz_bp import SZBranchingProgram 6 | from pyobf.obfuscator import Obfuscator 7 | 8 | failstr = utils.clr_error('Fail') 9 | 10 | def test_obfuscation(path, testcases, args, formula=True): 11 | success = True 12 | obf = Obfuscator(args.mmap, base=args.base, verbose=args.verbose, 13 | nthreads=args.nthreads, ncores=args.ncores) 14 | directory = args.save if args.save \ 15 | else '%s.obf.%d' % (path, args.secparam) 16 | obf.obfuscate(path, args.secparam, directory, kappa=args.kappa, 17 | formula=formula, randomization=(not args.no_randomization), 18 | seed=args.seed) 19 | for k, v in testcases.items(): 20 | if obf.evaluate(directory, k) != v: 21 | print('%s (%s != %d) ' % (failstr, k, v)) 22 | success = False 23 | return success 24 | 25 | def test_bp(path, testcases, args): 26 | success = True 27 | try: 28 | c = SZBranchingProgram(path, verbose=args.verbose) 29 | except ParseException as e: 30 | print('%s %s' % (utils.clr_warn('Parse Error:'), e)) 31 | return False 32 | for k, v in testcases.items(): 33 | if c.evaluate(k) != v: 34 | print('%s (%s != %d) ' % (failstr, k, v)) 35 | success = False 36 | return success 37 | 38 | def test_file(path, obfuscate, args, formula=True): 39 | success = True 40 | testcases = {} 41 | print('Testing %s: ' % path, end='') 42 | if args.verbose: 43 | print() 44 | with open(path) as f: 45 | for line in f: 46 | if line.startswith('#'): 47 | if 'TEST' in line: 48 | _, _, inp, outp = line.split() 49 | testcases[inp] = int(outp) 50 | else: 51 | continue 52 | if len(testcases) == 0: 53 | print('no test cases') 54 | return success 55 | if obfuscate: 56 | success = test_obfuscation(path, testcases, args, formula=formula) 57 | else: 58 | success = test_bp(path, testcases, args) 59 | if success: 60 | print(utils.clr_ok('Pass')) 61 | else: 62 | print(utils.clr_error('Fail')) 63 | return success 64 | -------------------------------------------------------------------------------- /circuits/point.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python2 2 | 3 | import random, sys 4 | from math import log 5 | 6 | def usage(): 7 | print('Usage: point.py ') 8 | sys.exit(1) 9 | 10 | def random_bitstring(bitlength): 11 | r = random.randint(0, 2 ** bitlength - 1) 12 | bits = bin(r)[2:] 13 | return '0' * (bitlength - len(bits)) + bits 14 | 15 | def binary_point(bitlength): 16 | with open('point-%d.circ' % bitlength, 'w') as f: 17 | secret = random_bitstring(bitlength) 18 | f.write('# TEST %s 0\n' % secret) 19 | for _ in xrange(5): 20 | test = random_bitstring(bitlength) 21 | if test != secret: 22 | f.write('# TEST %s 1\n' % test) 23 | for i in xrange(bitlength): 24 | f.write('%d input\n' % i) 25 | for i in xrange(bitlength): 26 | gate = 'ID' if secret[i] == '0' else 'NOT' 27 | f.write('%d gate %s %d\n' % (bitlength + i, gate, i)) 28 | 29 | length = bitlength 30 | oldstart = bitlength 31 | start = oldstart + length 32 | leftover = None 33 | while length > 1: 34 | for i, j in zip(xrange(start, start + length / 2), 35 | xrange(oldstart, start, 2)): 36 | if j + 1 == start: 37 | if leftover: 38 | f.write('%d gate OR %d %d\n' % (i, j, leftover)) 39 | leftover = None 40 | else: 41 | f.write('%d gate OR %d %d\n' % (i, j, j + 1)) 42 | if length % 2 == 1 and not leftover: 43 | leftover = start - 1 44 | oldstart = start 45 | start = start + length / 2 46 | length = (length + length % 2) / 2 47 | if leftover: 48 | f.write('%d gate OR %d %d\n' % (start + length / 2, 49 | start - 1, 50 | leftover)) 51 | start = start + 1 52 | f.write('%d output ID %d\n' % (start, start - 1)) 53 | 54 | def main(argv): 55 | if len(argv) != 2: 56 | usage() 57 | try: 58 | bitlength = int(argv[1]) 59 | except ValueError: 60 | usage() 61 | 62 | binary_point(bitlength) 63 | 64 | if __name__ == '__main__': 65 | try: 66 | main(sys.argv) 67 | except KeyboardInterrupt: 68 | pass 69 | -------------------------------------------------------------------------------- /circuits/point-json.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python2 2 | 3 | from __future__ import print_function 4 | import argparse, os, random, sys 5 | from util import * 6 | 7 | def point(base, length, cryfsm='cryfsm', fsmevade='fsmevade'): 8 | secret = random.randint(0, base ** length - 1) 9 | repr = dary_repr(secret, base, length) 10 | base_2_len = len(str_base(base-1, 2)) 11 | inp = ''.join([dary_repr(int(i, base=base), 2, base_2_len) for i in repr]) 12 | lst = [] 13 | for i in xrange(length): 14 | for b in xrange(base_2_len): 15 | lst.append('"%d"' % i) 16 | str = '[' + ','.join(lst) + ']' 17 | tmp = 'point-%d-%d-tmp.json' % (base, length) 18 | fname = 'point-%d-%d.json' % (base, length) 19 | lst = [cryfsm, 'util.cry', '-e', "(!=) 0b%s" % inp, 20 | '-v', "adjacentConstantBase `{base=%d}" % base, '-g', 21 | "%s" % str, '-o', tmp] 22 | try: 23 | run(lst) 24 | except OSError: 25 | print("error: running cryfsm failed") 26 | sys.exit(1) 27 | lst = [fsmevade, '-i', tmp, '-o', fname, 'True'] 28 | try: 29 | run(lst) 30 | except OSError: 31 | print("error: running fsmevade failed") 32 | sys.exit(1) 33 | os.remove(tmp) 34 | with open(fname, 'r') as f: 35 | line = f.read() 36 | with open(fname, 'w') as f: 37 | f.write('# TEST %s 0\n' % repr) 38 | for _ in xrange(5): 39 | test = random.randint(0, base ** length - 1) 40 | repr = dary_repr(test, base, length) 41 | if test != secret: 42 | f.write('# TEST %s 1\n' % repr) 43 | f.write(line) 44 | 45 | def main(argv): 46 | parser = argparse.ArgumentParser() 47 | parser.add_argument('--cryfsm', metavar='PATH', action='store', type=str, 48 | default='cryfsm', 49 | help='set PATH as cryfsm executable') 50 | parser.add_argument('--fsmevade', metavar='PATH', action='store', type=str, 51 | default='cryfsm', 52 | help='set PATH as fsmevade executable') 53 | parser.add_argument('base', action='store', type=int) 54 | parser.add_argument('length', action='store', type=int) 55 | args = parser.parse_args() 56 | point(args.base, args.length, args.cryfsm, args.fsmevade) 57 | 58 | if __name__ == '__main__': 59 | try: 60 | main(sys.argv) 61 | except KeyboardInterrupt: 62 | pass 63 | -------------------------------------------------------------------------------- /pyobf/circuit.py: -------------------------------------------------------------------------------- 1 | __all__ = ['parse', 'ParseException'] 2 | 3 | class ParseException(Exception): 4 | pass 5 | 6 | def _parse_param(line): 7 | try: 8 | _, param, value = line.split() 9 | except ValueError: 10 | raise ParseException("Invalid line '%s'" % line) 11 | param = param.lower() 12 | if param in ('nins', 'depth'): 13 | try: 14 | value = int(value) 15 | except ValueError: 16 | raise ParseException("Invalid value '%s'" % value) 17 | return {param: value} 18 | else: 19 | raise ParseException("Invalid parameter '%s'" % param) 20 | 21 | def parse(fname, bp, f_inp_gate, f_gate, keyed=False): 22 | info = {'nlayers': 0} 23 | output = False 24 | with open(fname) as f: 25 | for lineno, line in enumerate(f, 1): 26 | line = line.strip() 27 | if line == '' or line.startswith('#'): 28 | continue 29 | elif line.startswith(':'): 30 | # info.update(_parse_param(line)) 31 | continue 32 | num, rest = line.split(None, 1) 33 | try: 34 | num = int(num) 35 | except ValueError: 36 | raise ParseException( 37 | 'Line %d: gate index not a number' % lineno) 38 | if rest.startswith('input'): 39 | if keyed: 40 | _, inp = rest.split(None, 1) 41 | f_inp_gate(bp, num, inp) 42 | else: 43 | f_inp_gate(bp, num) 44 | info['nlayers'] += 1 45 | elif rest.startswith('gate') or rest.startswith('output'): 46 | if rest.startswith('output'): 47 | if output: 48 | raise ParseException( 49 | 'Line %d: only one output gate supported' % lineno) 50 | else: 51 | output = True 52 | _, gate, rest = rest.split(None, 2) 53 | inputs = [int(i) for i in rest.split()] 54 | try: 55 | f_gate(bp, num, lineno, gate.upper(), inputs) 56 | except KeyError: 57 | raise ParseException( 58 | 'Line %d: unsupported gate %s' % (lineno, gate)) 59 | except TypeError: 60 | raise ParseException( 61 | 'Line %d: incorrect number of arguments given' % lineno) 62 | else: 63 | raise ParseException('Line %d: unknown gate type' % lineno) 64 | if not output: 65 | raise ParseException('no output gate found') 66 | return bp[-1], info 67 | -------------------------------------------------------------------------------- /src/thpool_fns.c: -------------------------------------------------------------------------------- 1 | #include "thpool_fns.h" 2 | #include "utils.h" 3 | 4 | #include 5 | #include 6 | 7 | #include 8 | #include 9 | #include 10 | 11 | void * 12 | thpool_encode_elem(void *vargs) 13 | { 14 | struct encode_elem_s *args = (struct encode_elem_s *) vargs; 15 | 16 | args->vtable->enc->encode(args->enc, args->sk, args->n, args->plaintext, 17 | args->group); 18 | 19 | for (int i = 0; i < args->n; ++i) 20 | fmpz_clear(args->plaintext[i]); 21 | free(args->plaintext); 22 | free(args); 23 | 24 | return NULL; 25 | } 26 | 27 | void * 28 | thpool_write_layer(void *vargs) 29 | { 30 | char fname[1024]; 31 | int fnamelen = 1024; 32 | FILE *fp; 33 | double end; 34 | struct write_layer_s *args = (struct write_layer_s *) vargs; 35 | 36 | (void) snprintf(fname, fnamelen, "%s/%ld.input", args->dir, args->idx); 37 | fp = fopen(fname, "w+b"); 38 | if (fp == NULL) { 39 | fprintf(stderr, "Unable to write '%s'\n", fname); 40 | goto done; 41 | } 42 | fwrite(&args->inp, sizeof args->inp, 1, fp); 43 | fclose(fp); 44 | 45 | (void) snprintf(fname, fnamelen, "%s/%ld.nrows", args->dir, args->idx); 46 | fp = fopen(fname, "w+b"); 47 | if (fp == NULL) { 48 | fprintf(stderr, "Unable to write '%s'\n", fname); 49 | goto done; 50 | } 51 | fwrite(&args->nrows, sizeof args->nrows, 1, fp); 52 | fclose(fp); 53 | 54 | (void) snprintf(fname, fnamelen, "%s/%ld.ncols", args->dir, args->idx); 55 | fp = fopen(fname, "w+b"); 56 | if (fp == NULL) { 57 | fprintf(stderr, "Unable to write '%s'\n", fname); 58 | goto done; 59 | } 60 | fwrite(&args->ncols, sizeof args->ncols, 1, fp); 61 | fclose(fp); 62 | 63 | for (uint64_t c = 0; c < args->n; ++c) { 64 | (void) snprintf(fname, fnamelen, "%s/%ld.%s", args->dir, args->idx, args->names[c]); 65 | fp = fopen(fname, "w+b"); 66 | if (fp == NULL) { 67 | fprintf(stderr, "Unable to write '%s'\n", fname); 68 | goto done; 69 | } 70 | for (long i = 0; i < args->nrows; ++i) { 71 | for (long j = 0; j < args->ncols; ++j) { 72 | args->vtable->enc->fwrite(args->enc_mats[c][0]->m[i][j], fp); 73 | } 74 | } 75 | mmap_enc_mat_clear(args->vtable, *args->enc_mats[c]); 76 | free(args->enc_mats[c]); 77 | free(args->names[c]); 78 | fclose(fp); 79 | } 80 | free(args->enc_mats); 81 | free(args->names); 82 | 83 | done: 84 | end = current_time(); 85 | if (args->verbose) 86 | (void) fprintf(stderr, " Encoding %ld elements: %f\n", 87 | args->n * args->nrows * args->ncols, end - args->start); 88 | 89 | free(args); 90 | 91 | return NULL; 92 | } 93 | -------------------------------------------------------------------------------- /pyobf/bp.py: -------------------------------------------------------------------------------- 1 | import pyobf.utils as utils 2 | 3 | class Layer(object): 4 | def __init__(self, inp, matrices, sets): 5 | self.inp = inp 6 | self.matrices = matrices 7 | if sets is None: 8 | self.sets = [None] * len(matrices) 9 | else: 10 | self.sets = sets 11 | def size(self): 12 | size = 0 13 | for matrix in self.matrices: 14 | size += len(matrix) 15 | return size 16 | def __repr__(self): 17 | str = "\ninput: %d\n" % self.inp 18 | for i, mat in enumerate(self.matrices): 19 | str += "%d-mat:\n%s\n" % (i, mat) 20 | for i, set in enumerate(self.sets): 21 | str += "%d-set: %s\n" % (i, set) 22 | return str 23 | def mult_scalar(self, alphas): 24 | mats = [alphas[i] * self.matrices[i] for i in len(self.matrices)] 25 | return Layer(self.inp, mats, self.sets) 26 | def mult_left(self, M): 27 | mats = [M * mat for mat in self.matrices] 28 | return Layer(self.inp, mats, self.sets) 29 | def mult_right(self, M): 30 | mats = [mat * M for mat in self.matrices] 31 | return Layer(self.inp, mats, self.sets) 32 | 33 | 34 | class AbstractBranchingProgram(object): 35 | def __init__(self, base=None, verbose=False): 36 | self._verbose = verbose 37 | self.logger = utils.make_logger(self._verbose) 38 | self.nlayers = 0 39 | self.ninputs = None 40 | self.bp = None 41 | self.base = base 42 | def __len__(self): 43 | return len(self.bp) 44 | def __iter__(self): 45 | return self.bp.__iter__() 46 | def next(self): 47 | return self.bp.next() 48 | def __getitem__(self, i): 49 | return self.bp[i] 50 | def __repr__(self): 51 | return repr(self.bp) 52 | 53 | def set_straddling_sets(self): 54 | inpdir = {} 55 | for layer in self.bp: 56 | inpdir.setdefault(layer.inp, []).append(layer) 57 | n = 0 58 | for layers in inpdir.itervalues(): 59 | if len(layers) == 1: 60 | for i in xrange(len(layers[0].sets)): 61 | layers[0].sets[i] = [n] 62 | n += 1 63 | else: 64 | if self.base is not None and self.base != 2: 65 | print("Error: straddling sets do not work for base != 2") 66 | raise NotImplementedError 67 | max = len(layers) - 1 68 | for i, layer in enumerate(layers): 69 | if i < max: 70 | layer.sets[0] = [n - 1, n] if i else [n] 71 | layer.sets[1] = [n, n + 1] 72 | n += 2 73 | else: 74 | layer.sets[0] = [n - 1, n] if max else [n] 75 | layer.sets[1] = [n] 76 | n += 1 77 | return n 78 | 79 | def evaluate(self, x): 80 | raise NotImplementedError 81 | -------------------------------------------------------------------------------- /src/thpool.h: -------------------------------------------------------------------------------- 1 | /********************************** 2 | * @author Johan Hanssen Seferidis 3 | * License: MIT 4 | * 5 | **********************************/ 6 | 7 | #ifndef _THPOOL_ 8 | #define _THPOOL_ 9 | 10 | #ifdef __cplusplus 11 | extern "C" { 12 | #endif 13 | 14 | /* =================================== API ======================================= */ 15 | 16 | 17 | typedef struct thpool_* threadpool; 18 | 19 | 20 | /** 21 | * @brief Initialize threadpool 22 | * 23 | * Initializes a threadpool. This function will not return untill all 24 | * threads have initialized successfully. 25 | * 26 | * @example 27 | * 28 | * .. 29 | * threadpool thpool; //First we declare a threadpool 30 | * thpool = thpool_init(4); //then we initialize it to 4 threads 31 | * .. 32 | * 33 | * @param num_threads number of threads to be created in the threadpool 34 | * @return threadpool created threadpool on success, 35 | * NULL on error 36 | */ 37 | threadpool thpool_init(int num_threads); 38 | 39 | 40 | int 41 | thpool_add_tag(threadpool thpool_p, char *tag, int length, 42 | void * (fn)(void *arg), void *arg); 43 | 44 | 45 | /** 46 | * @brief Add work to the job queue 47 | * 48 | * Takes an action and its argument and adds it to the threadpool's job queue. 49 | * If you want to add to work a function with more than one arguments then 50 | * a way to implement this is by passing a pointer to a structure. 51 | * 52 | * NOTICE: You have to cast both the function and argument to not get warnings. 53 | * 54 | * @example 55 | * 56 | * void print_num(int num){ 57 | * printf("%d\n", num); 58 | * } 59 | * 60 | * int main() { 61 | * .. 62 | * int a = 10; 63 | * thpool_add_work(thpool, (void*)print_num, (void*)a); 64 | * .. 65 | * } 66 | * 67 | * @param threadpool threadpool to which the work will be added 68 | * @param function_p pointer to function to add as work 69 | * @param arg_p pointer to an argument 70 | * @return nothing 71 | */ 72 | int thpool_add_work(threadpool, void *(*function_p)(void*), void* arg_p, 73 | char *tag); 74 | 75 | 76 | /** 77 | * @brief Wait for all queued jobs to finish 78 | * 79 | * Will wait for all jobs - both queued and currently running to finish. 80 | * Once the queue is empty and all work has completed, the calling thread 81 | * (probably the main program) will continue. 82 | * 83 | * Smart polling is used in wait. The polling is initially 0 - meaning that 84 | * there is virtually no polling at all. If after 1 seconds the threads 85 | * haven't finished, the polling interval starts growing exponentially 86 | * untill it reaches max_secs seconds. Then it jumps down to a maximum polling 87 | * interval assuming that heavy processing is being used in the threadpool. 88 | * 89 | * @example 90 | * 91 | * .. 92 | * threadpool thpool = thpool_init(4); 93 | * .. 94 | * // Add a bunch of work 95 | * .. 96 | * thpool_wait(thpool); 97 | * puts("All added work has finished"); 98 | * .. 99 | * 100 | * @param threadpool the threadpool to wait for 101 | * @return nothing 102 | */ 103 | void thpool_wait(threadpool); 104 | 105 | 106 | /** 107 | * @brief Pauses all threads immediately 108 | * 109 | * The threads will be paused no matter if they are idle or working. 110 | * The threads return to their previous states once thpool_resume 111 | * is called. 112 | * 113 | * While the thread is being paused, new work can be added. 114 | * 115 | * @example 116 | * 117 | * threadpool thpool = thpool_init(4); 118 | * thpool_pause(thpool); 119 | * .. 120 | * // Add a bunch of work 121 | * .. 122 | * thpool_resume(thpool); // Let the threads start their magic 123 | * 124 | * @param threadpool the threadpool where the threads should be paused 125 | * @return nothing 126 | */ 127 | void thpool_pause(threadpool); 128 | 129 | 130 | /** 131 | * @brief Unpauses all threads if they are paused 132 | * 133 | * @example 134 | * .. 135 | * thpool_pause(thpool); 136 | * sleep(10); // Delay execution 10 seconds 137 | * thpool_resume(thpool); 138 | * .. 139 | * 140 | * @param threadpool the threadpool where the threads should be unpaused 141 | * @return nothing 142 | */ 143 | void thpool_resume(threadpool); 144 | 145 | 146 | /** 147 | * @brief Destroy the threadpool 148 | * 149 | * This will wait for the currently active threads to finish and then 'kill' 150 | * the whole threadpool to free up memory. 151 | * 152 | * @example 153 | * int main() { 154 | * threadpool thpool1 = thpool_init(2); 155 | * threadpool thpool2 = thpool_init(2); 156 | * .. 157 | * thpool_destroy(thpool1); 158 | * .. 159 | * return 0; 160 | * } 161 | * 162 | * @param threadpool the threadpool to destroy 163 | * @return nothing 164 | */ 165 | void thpool_destroy(threadpool); 166 | 167 | #ifdef __cplusplus 168 | } 169 | #endif 170 | 171 | #endif 172 | -------------------------------------------------------------------------------- /pywrapper/obfuscator_wrapper.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include "obfuscator.h" 3 | #include "pyutils.h" 4 | 5 | static void 6 | obf_clear_wrapper(PyObject *self) 7 | { 8 | obf_clear((obf_state_t *) PyCapsule_GetPointer(self, NULL)); 9 | } 10 | 11 | static PyObject * 12 | obf_init_wrapper(PyObject *self, PyObject *args) 13 | { 14 | long type_ = 0, secparam = 0, kappa = 0, nzs = 0, nthreads = 0, 15 | ncores = 0, flags = 0; 16 | enum mmap_e type; 17 | char *dir = NULL, *seed = NULL; 18 | obf_state_t *s = NULL; 19 | 20 | if (!PyArg_ParseTuple(args, "sllllllzl", &dir, &type_, &secparam, &kappa, 21 | &nzs, &nthreads, &ncores, &seed, &flags)) { 22 | PyErr_SetString(PyExc_RuntimeError, "unable to parse input"); 23 | return NULL; 24 | } 25 | if (secparam <= 0 || kappa <= 0 || nzs <= 0) { 26 | PyErr_SetString(PyExc_RuntimeError, "invalid input"); 27 | return NULL; 28 | } 29 | 30 | switch (type_) { 31 | case 0: 32 | type = MMAP_CLT; 33 | break; 34 | case 1: 35 | type = MMAP_GGHLITE; 36 | break; 37 | case 2: 38 | type = MMAP_DUMMY; 39 | break; 40 | default: 41 | PyErr_SetString(PyExc_RuntimeError, "invalid mmap type"); 42 | return NULL; 43 | } 44 | 45 | s = obf_init(type, dir, secparam, kappa, nzs, nthreads, ncores, seed, flags); 46 | if (s == NULL) { 47 | PyErr_SetString(PyExc_RuntimeError, "initialization failed"); 48 | return NULL; 49 | } 50 | 51 | return PyCapsule_New((void *) s, NULL, obf_clear_wrapper); 52 | } 53 | 54 | static PyObject * 55 | obf_encode_layer_wrapper(PyObject *self, PyObject *args) 56 | { 57 | PyObject *py_state, *py_pows, *py_mats; 58 | long n, idx, nrows, ncols, inp, rflag; 59 | ssize_t length; 60 | int **pows; 61 | fmpz_mat_t *mats; 62 | obf_state_t *s; 63 | 64 | // TODO: can probably get nrows, ncols length from matrices 65 | if (!PyArg_ParseTuple(args, "OlOOlllll", &py_state, &n, &py_pows, &py_mats, 66 | &idx, &nrows, &ncols, &inp, &rflag)) 67 | return NULL; 68 | 69 | s = (obf_state_t *) PyCapsule_GetPointer(py_state, NULL); 70 | if (s == NULL) { 71 | PyErr_SetString(PyExc_RuntimeError, "unable to extract obf state"); 72 | return NULL; 73 | } 74 | 75 | length = PyList_Size(PyList_GetItem(py_pows, 0)); 76 | // TODO: check pow lengths 77 | 78 | pows = (int **) calloc(n, sizeof(int *)); 79 | for (long c = 0; c < n; ++c) { 80 | pows[c] = (int *) calloc(length, sizeof(int)); 81 | for (ssize_t i = 0; i < length; ++i) { 82 | pows[c][i] = PyLong_AsLong( 83 | PyList_GetItem( 84 | PyList_GetItem(py_pows, c), i)); 85 | } 86 | } 87 | 88 | mats = (fmpz_mat_t *) calloc(n, sizeof(fmpz_mat_t)); 89 | for (long c = 0; c < n; ++c) { 90 | fmpz_mat_init(mats[c], nrows, ncols); 91 | for (long i = 0; i < nrows; ++i) { 92 | for (long j = 0; j < ncols; ++j) { 93 | py_to_fmpz(fmpz_mat_entry(mats[c], i, j), 94 | PyList_GetItem( 95 | PyList_GetItem( 96 | PyList_GetItem(py_mats, c), i), j)); 97 | } 98 | } 99 | } 100 | 101 | obf_encode_layer(s, n, pows, mats, idx, inp, 102 | (encode_layer_randomization_flag_t) rflag); 103 | 104 | for (long c = 0; c < n; ++c) { 105 | fmpz_mat_clear(mats[c]); 106 | } 107 | free(mats); 108 | // TODO: make sure that pows gets cleared elsewhere 109 | 110 | Py_RETURN_NONE; 111 | } 112 | 113 | static PyObject * 114 | obf_evaluate_wrapper(PyObject *self, PyObject *args) 115 | { 116 | PyObject *py_input; 117 | uint64_t *input; 118 | char *dir = NULL; 119 | long iszero, type_, flags; 120 | enum mmap_e type; 121 | uint64_t len = 0, bplen = 0, ncores = 0; 122 | 123 | if (!PyArg_ParseTuple(args, "zOllll", &dir, &py_input, &type_, &bplen, 124 | &ncores, &flags)) { 125 | PyErr_SetString(PyExc_RuntimeError, "error parsing arguments"); 126 | return NULL; 127 | } 128 | 129 | switch (type_) { 130 | case 0: 131 | type = MMAP_CLT; 132 | break; 133 | case 1: 134 | type = MMAP_GGHLITE; 135 | break; 136 | case 2: 137 | type = MMAP_DUMMY; 138 | break; 139 | default: 140 | PyErr_SetString(PyExc_RuntimeError, "invalid mmap type"); 141 | return NULL; 142 | } 143 | 144 | len = PyList_Size(py_input); 145 | input = (uint64_t *) calloc(len, sizeof(uint64_t)); 146 | for (int i = 0; i < len; ++i) { 147 | input[i] = PyLong_AsLong(PyList_GetItem(py_input, i)); 148 | } 149 | 150 | iszero = obf_evaluate(type, dir, len, input, bplen, ncores, flags); 151 | if (iszero == -1) { 152 | PyErr_SetString(PyExc_RuntimeError, "zero test failed"); 153 | return NULL; 154 | } else { 155 | return Py_BuildValue("i", iszero ? 0 : 1); 156 | } 157 | } 158 | 159 | static PyObject * 160 | obf_wait_wrapper(PyObject *self, PyObject *args) 161 | { 162 | PyObject *py_state; 163 | obf_state_t *s; 164 | 165 | if (!PyArg_ParseTuple(args, "O", &py_state)) 166 | return NULL; 167 | 168 | s = (obf_state_t *) PyCapsule_GetPointer(py_state, NULL); 169 | if (s == NULL) 170 | return NULL; 171 | 172 | obf_wait(s); 173 | 174 | Py_RETURN_NONE; 175 | } 176 | 177 | static PyMethodDef 178 | ObfMethods[] = { 179 | {"init", obf_init_wrapper, METH_VARARGS, 180 | "Set up obfuscator."}, 181 | {"encode_layer", obf_encode_layer_wrapper, METH_VARARGS, 182 | "Encode a branching program layer in each slot."}, 183 | {"max_mem_usage", obf_max_mem_usage, METH_VARARGS, 184 | "Print out the maximum memory usage."}, 185 | {"evaluate", obf_evaluate_wrapper, METH_VARARGS, 186 | "Evaluate the obfuscation."}, 187 | {"wait", obf_wait_wrapper, METH_VARARGS, 188 | "Wait for threadpool to empty."}, 189 | {NULL, NULL, 0, NULL} 190 | }; 191 | 192 | 193 | #if PY_MAJOR_VERSION >= 3 194 | static struct PyModuleDef obfmodule = { 195 | PyModuleDef_HEAD_INIT, 196 | "_obfuscator", 197 | NULL, 198 | -1, 199 | ObfMethods 200 | }; 201 | 202 | PyMODINIT_FUNC 203 | PyInit__obfuscator(void) 204 | { 205 | return PyModule_Create(&obfmodule); 206 | } 207 | #else 208 | PyMODINIT_FUNC 209 | init_obfuscator(void) 210 | { 211 | (void) Py_InitModule("_obfuscator", ObfMethods); 212 | } 213 | #endif 214 | -------------------------------------------------------------------------------- /pyobf/obfuscator.py: -------------------------------------------------------------------------------- 1 | from builtins import * # Python3 compatibility 2 | import pyobf._obfuscator as _obf 3 | from pyobf.sz_bp import SZBranchingProgram 4 | import pyobf.utils as utils 5 | import os, re, time 6 | 7 | MMAP_CLT = 0x00 8 | MMAP_GGHLITE = 0x01 9 | MMAP_DUMMY = 0x02 10 | 11 | OBFUSCATOR_FLAG_NONE = 0x00 12 | OBFUSCATOR_FLAG_NO_RANDOMIZATION = 0x01 13 | OBFUSCATOR_FLAG_VERBOSE = 0x04 14 | 15 | ENCODE_LAYER_RANDOMIZATION_TYPE_NONE = 0x00 16 | ENCODE_LAYER_RANDOMIZATION_TYPE_FIRST = 0x01 17 | ENCODE_LAYER_RANDOMIZATION_TYPE_MIDDLE = 0x02 18 | ENCODE_LAYER_RANDOMIZATION_TYPE_LAST = 0x04 19 | 20 | err_str = utils.clr_error('Error:') 21 | warn_str = utils.clr_warn('Warning:') 22 | 23 | def get_mmap_flag(mmap): 24 | if mmap == 'CLT': 25 | return MMAP_CLT 26 | elif mmap == 'GGH': 27 | return MMAP_GGHLITE 28 | else: 29 | return MMAP_DUMMY 30 | 31 | class Obfuscator(object): 32 | def __init__(self, mmap, base=None, verbose=False, nthreads=None, 33 | ncores=None): 34 | self._state = None 35 | self._verbose = verbose 36 | self._nthreads = nthreads 37 | self._ncores = ncores 38 | self._base = base 39 | self.logger = utils.make_logger(self._verbose) 40 | self._mmap = get_mmap_flag(mmap) 41 | 42 | def _remove_old(self, directory): 43 | # remove old files in obfuscation directory 44 | if os.path.isdir(directory): 45 | for file in os.listdir(directory): 46 | p = os.path.join(directory, file) 47 | os.unlink(p) 48 | 49 | def _construct_bp(self, fname, formula=True): 50 | self.logger('Constructing BP...') 51 | start = time.time() 52 | bp = SZBranchingProgram(fname, verbose=self._verbose, formula=formula) 53 | nzs = bp.set_straddling_sets() 54 | end = time.time() 55 | self.logger('Took: %f' % (end - start)) 56 | return bp, nzs 57 | 58 | def _init_mmap(self, secparam, kappa, nzs, directory, seed, flags): 59 | self.logger('Initializing mmap...') 60 | start = time.time() 61 | if not os.path.exists(directory): 62 | os.mkdir(directory) 63 | self._state = _obf.init(directory, self._mmap, secparam, kappa, nzs, 64 | self._nthreads, self._ncores, seed, flags) 65 | end = time.time() 66 | self.logger('Took: %f' % (end - start)) 67 | 68 | def _obfuscate(self, bp, nzs): 69 | nencodings = 0 70 | for i in range(len(bp)): 71 | nrows, ncols = bp[i].matrices[0].shape 72 | nencodings += nrows * ncols * self._base 73 | self.logger('Total # Encodings: %d' % nencodings) 74 | for i in range(len(bp)): 75 | self.logger('Obfuscating layer...') 76 | mats = [bp[i].matrices[j].tolist() for j in range(self._base)] 77 | nrows, ncols = bp[i].matrices[0].shape 78 | rflags = ENCODE_LAYER_RANDOMIZATION_TYPE_NONE 79 | if i == 0: 80 | rflags |= ENCODE_LAYER_RANDOMIZATION_TYPE_FIRST 81 | if i == len(bp) - 1: 82 | rflags |= ENCODE_LAYER_RANDOMIZATION_TYPE_LAST 83 | if 0 < i < len(bp) - 1: 84 | rflags |= ENCODE_LAYER_RANDOMIZATION_TYPE_MIDDLE 85 | pows = [[0] * nzs for _ in range(self._base)] 86 | for j in range(self._base): 87 | for k in bp[i].sets[j]: 88 | pows[j][k] = 1 89 | _obf.encode_layer(self._state, self._base, pows, mats, i, nrows, 90 | ncols, bp[i].inp, rflags) 91 | 92 | ''' 93 | Get size of obfuscation (in bytes) 94 | ''' 95 | def obfsize(self, directory): 96 | size = 0 97 | for f in os.listdir(directory): 98 | size += os.path.getsize(os.path.join(directory, f)) 99 | return size 100 | 101 | def obfuscate(self, fname, secparam, directory, kappa=None, formula=True, 102 | randomization=True, seed=None): 103 | start = time.time() 104 | self._remove_old(directory) 105 | bp, nzs = self._construct_bp(fname, formula=formula) 106 | if not kappa: 107 | kappa = nzs 108 | flags = OBFUSCATOR_FLAG_NONE 109 | if self._verbose: 110 | flags |= OBFUSCATOR_FLAG_VERBOSE 111 | if not randomization: 112 | flags |= OBFUSCATOR_FLAG_NO_RANDOMIZATION 113 | self._init_mmap(secparam, kappa, nzs, directory, seed, flags) 114 | if self._base is None: 115 | self._base = len(bp[0].matrices) 116 | self._obfuscate(bp, nzs) 117 | _obf.wait(self._state) 118 | end = time.time() 119 | self.logger('Obfuscation took: %f s' % (end - start)) 120 | self.logger('Obfuscation size: %0.2f KB' % (self.obfsize(directory) / 1024.0)) 121 | if self._verbose: 122 | _obf.max_mem_usage() 123 | 124 | def _evaluate(self, directory, inp, f, obf, flags): 125 | self.logger('Evaluating %s...' % inp) 126 | start = time.time() 127 | files = os.listdir(directory) 128 | inputs = sorted(filter(lambda s: 'input' in s, files)) 129 | result = f(directory, inp, self._mmap, len(inputs), self._ncores, flags) 130 | end = time.time() 131 | self.logger('Took: %f' % (end - start)) 132 | if self._verbose: 133 | obf.max_mem_usage() 134 | return result 135 | 136 | def evaluate(self, directory, inp): 137 | files = os.listdir(directory) 138 | if self._base: 139 | base = self._base 140 | else: 141 | # Compute base by counting the number of files that correspond to 142 | # the first MBP layer. 143 | base = len(list(filter(lambda s: re.match('0.\d+', s), files))) 144 | if base < 2: 145 | print('{} Base cannot be < 2'.format(err_str)) 146 | return None 147 | # Input length is equal to the number of `[num].input` files in 148 | # `directory`. 149 | inplen = len(list(filter(lambda file: re.match('\d+.input', file), files))) 150 | if len(inp) != inplen: 151 | print('{} Invalid input length ({} != {})'.format( 152 | err_str, len(inp), inplen)) 153 | return None 154 | try: 155 | if base > 36: 156 | print("{} Bases > 36 are not quite supported, so assuming that " 157 | "input encoded using base 36".format(warn_str)) 158 | base = 36 159 | inp = [int(i, base) for i in inp] 160 | except ValueError: 161 | print('{} Invalid input for base {}'.format(err_str, base)) 162 | return None 163 | flags = OBFUSCATOR_FLAG_NONE 164 | if self._verbose: 165 | flags |= OBFUSCATOR_FLAG_VERBOSE 166 | return self._evaluate(directory, inp, _obf.evaluate, _obf, flags) 167 | -------------------------------------------------------------------------------- /pyobf/sz_bp.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | 3 | from pyobf.bp import AbstractBranchingProgram, Layer 4 | from pyobf.circuit import ParseException 5 | 6 | import numpy as np 7 | from numpy import matrix 8 | 9 | import json, random, sys 10 | 11 | def transpose(bps): 12 | bps.reverse() 13 | newbps = [] 14 | for bp in bps: 15 | newbps.append(Layer(bp.inp, 16 | [bp.matrices[0].transpose(), 17 | bp.matrices[1].transpose()], 18 | bp.sets)) 19 | return newbps 20 | 21 | def augment(bps, r): 22 | def _augment(M, r): 23 | nrows, ncols = M.shape 24 | Z_1 = np.zeros([nrows, r], int) 25 | Z_2 = np.zeros([r, ncols], int) 26 | I_r = np.identity(r, int) 27 | tmp1 = np.concatenate((M, Z_1), 1).transpose() 28 | tmp2 = np.concatenate((Z_2, I_r), 1).transpose() 29 | return np.concatenate((tmp1, tmp2), 1).transpose() 30 | newbps = [] 31 | for bp in bps: 32 | newbps.append(Layer(bp.inp, 33 | [_augment(bp.matrices[0], r), 34 | _augment(bp.matrices[1], r)], 35 | bp.sets)) 36 | return newbps 37 | 38 | def mult_left(bps, m): 39 | bps[0] = bps[0].mult_left(m) 40 | def mult_right(bps, m): 41 | bps[-1] = bps[-1].mult_right(m) 42 | 43 | def swap_columns(m, a, b): 44 | col = m[:,a].copy() 45 | m[:,a] = m[:,b].copy() 46 | m[:,b] = col 47 | 48 | class SZBranchingProgram(AbstractBranchingProgram): 49 | def __init__(self, fname, base=None, verbose=False, formula=True): 50 | super(SZBranchingProgram, self).__init__(base=base, verbose=verbose) 51 | if formula: 52 | self.bp = self._load_formula(fname) 53 | else: 54 | self.bp = self._load_bp(fname) 55 | 56 | def _load_bp(self, fname): 57 | bp = [] 58 | try: 59 | with open(fname) as f: 60 | for line in f: 61 | if line.startswith('#'): 62 | continue 63 | bp_json = json.loads(line) 64 | for step in bp_json['steps']: 65 | keys = list(step) 66 | keys = filter(lambda x: x != 'position', keys) 67 | keys = sorted(keys) 68 | try: 69 | pos = int(step['position']) 70 | except ValueError: 71 | pos = ord(step['position']) - ord('a') 72 | bp.append( 73 | Layer(pos, [matrix(step[key]) for key in keys], 74 | None)) 75 | # assert len(bp_json['outputs']) == 1 and \ 76 | # len(bp_json['outputs'][0]) == 2 77 | # first_out = bp_json['outputs'][0][0].lower() 78 | # if first_out not in ['false', '0']: 79 | # if first_out not in ['true', '1']: 80 | # print('warning: interpreting %s as a truthy output' % first_out) 81 | # for i in range(len(keys)): 82 | # swap_columns(bp[-1].matrices[i], 0, 1) 83 | return bp 84 | except IOError as e: 85 | print(e) 86 | sys.exit(1) 87 | 88 | def _load_formula(self, fname): 89 | def _new_gate(num): 90 | zero = matrix([1, 0]) 91 | one = matrix([1, 1]) 92 | return [Layer(num, [zero, one], None)] 93 | def _two_input_gate(bp0, bp1, left, right): 94 | bp1 = augment(transpose(bp1), 1) 95 | mult_left(bp1, left) 96 | mult_right(bp1, right) 97 | bp0.extend(bp1) 98 | return bp0 99 | def _and_gate(num, bp0, bp1): 100 | left = matrix([[0, 0, 1], [0, 1, 0]]) 101 | right = matrix([[0, 1], [1, 0]]) 102 | return _two_input_gate(bp0, bp1, left, right) 103 | def _id_gate(num, bp0): 104 | right = matrix([[1, 0], [0, 1]]) 105 | mult_right(bp0, right) 106 | return bp0 107 | def _or_gate(num, bp0, bp1): 108 | left = matrix([[0, 1, 1], [1, -1, 0]]) 109 | right = matrix([[0, 1], [1, 0]]) 110 | return _two_input_gate(bp0, bp1, left, right) 111 | def _not_gate(num, bp0): 112 | right = matrix([[1, 1], [0, -1]]) 113 | mult_right(bp0, right) 114 | return bp0 115 | def _xor_gate(num, bp0, bp1): 116 | left = matrix([[0, 1, 1], [1, -2, 0]]) 117 | right = matrix([[0, 1], [1, 0]]) 118 | return _two_input_gate(bp0, bp1, left, right) 119 | def _parse_file(f): 120 | wires = set() 121 | bp = [] 122 | for lineno, line in enumerate(f, 1): 123 | if line.startswith('#'): 124 | continue 125 | if line.startswith(':'): 126 | continue 127 | try: 128 | num, rest = line.split(None, 1) 129 | except ValueError: 130 | raise ParseException( 131 | 'Line %d: unable to parse line' % lineno) 132 | try: 133 | num = int(num) 134 | except ValueError: 135 | raise ParseException( 136 | 'Line %d: gate index not a number' % lineno) 137 | gates = { 138 | 'AND': lambda num, in1, in2: _and_gate(num, bp[in1], bp[in2]), 139 | 'ID': lambda num, in1: _id_gate(num, bp[in1]), 140 | 'OR': lambda num, in1, in2: _or_gate(num, bp[in1], bp[in2]), 141 | 'NOT': lambda num, in1: _not_gate(num, bp[in1]), 142 | 'XOR': lambda num, in1, in2: _xor_gate(num, bp[in1], bp[in2]), 143 | } 144 | if rest.startswith('input'): 145 | bp.append(_new_gate(num)) 146 | elif rest.startswith('gate') or rest.startswith('output'): 147 | if rest.startswith('output'): 148 | output = True 149 | _, gate, rest = rest.split(None, 2) 150 | inputs = [int(i) for i in rest.split()] 151 | if wires.intersection(inputs): 152 | raise ParseException( 153 | 'Line %d: only Boolean formulas supported' % lineno) 154 | wires.update(inputs) 155 | try: 156 | bp.append(gates[gate](num, *inputs)) 157 | except KeyError: 158 | raise ParseException( 159 | 'Line %d: unsupported gate %s' % (lineno, gate)) 160 | except TypeError: 161 | raise ParseException( 162 | 'Line %d: incorrect number of arguments given' % lineno) 163 | return bp[-1] 164 | try: 165 | with open(fname) as f: 166 | return _parse_file(f) 167 | except IOError as err: 168 | raise ParseException(err) 169 | 170 | def evaluate(self, x): 171 | assert self.bp 172 | base = self.base if self.base else 2 173 | try: 174 | inp = [int(i, base) for i in x] 175 | except ValueError: 176 | print("Error: invalid base for input '%s'" % x) 177 | sys.exit(1) 178 | m = self.bp[0] 179 | comp = m.matrices[inp[m.inp]] 180 | for m in self.bp[1:]: 181 | comp = np.dot(comp, m.matrices[inp[m.inp]]) 182 | return comp[0, comp.shape[1] - 1] != 0 183 | -------------------------------------------------------------------------------- /pyobf/main.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | 3 | from pyobf.circuit import ParseException 4 | from pyobf.test import test_file 5 | from pyobf.sz_bp import SZBranchingProgram 6 | from pyobf.obfuscator import Obfuscator 7 | 8 | import argparse, os, sys, time 9 | import pyobf.utils as utils 10 | 11 | __all__ = ['main'] 12 | 13 | errorstr = utils.clr_error('Error:') 14 | 15 | def is_formula(fname, args): 16 | ext = os.path.splitext(fname)[1] 17 | if ext in ('.circ'): 18 | return True 19 | elif ext == '.json': 20 | return False 21 | else: 22 | print("%s unknown extension '%s'" % (errorstr, ext)) 23 | sys.exit(1) 24 | 25 | def test_all(args, obfuscate): 26 | success = True 27 | if not os.path.isdir(args.test_all): 28 | print("%s '%s' is not a valid directory" % (errorstr, args.test_all)) 29 | sys.exit(1) 30 | ext = '.circ' 31 | for circuit in os.listdir(args.test_all): 32 | path = os.path.join(args.test_all, circuit) 33 | if os.path.isfile(path) and path.endswith(ext): 34 | success &= test_file(path, obfuscate, args) 35 | if os.path.isfile(path) and path.endswith('.json'): 36 | success &= test_file(path, obfuscate, args, formula=False) 37 | return success 38 | 39 | def bp(args): 40 | success = True 41 | try: 42 | if args.test: 43 | success = test_file(args.test, False, args) 44 | elif args.test_all: 45 | success = test_all(args, False) 46 | elif args.load: 47 | formula = is_formula(args.load, args) 48 | bp = SZBranchingProgram(args.load, base=args.base, 49 | verbose=args.verbose, formula=formula) 50 | if args.print: 51 | print(bp) 52 | if args.eval: 53 | r = bp.evaluate(args.eval) 54 | print('Output = %d' % r) 55 | except ParseException as e: 56 | print('%s %s' % (errorstr, e)) 57 | sys.exit(1) 58 | return success 59 | 60 | def obf(args): 61 | if args.mmap not in ('CLT', 'GGH', 'DUMMY'): 62 | print('--mmap must be either CLT, GGH, or DUMMY') 63 | sys.exit(1) 64 | success = True 65 | try: 66 | if args.test: 67 | formula = is_formula(args.test, args) 68 | success = test_file(args.test, True, args, formula=formula) 69 | elif args.test_all: 70 | success = test_all(args, True) 71 | else: 72 | directory = None 73 | if args.load_obf: 74 | directory = args.load_obf 75 | elif args.load: 76 | formula = is_formula(args.load, args) 77 | obf = Obfuscator(args.mmap, base=args.base, 78 | verbose=args.verbose, nthreads=args.nthreads, 79 | ncores=args.ncores) 80 | directory = args.save if args.save \ 81 | else '%s.obf.%d' % (args.load, args.secparam) 82 | obf.obfuscate(args.load, args.secparam, directory, 83 | kappa=args.kappa, formula=formula, 84 | randomization=(not args.no_randomization), 85 | seed=args.seed) 86 | else: 87 | print('%s One of --load-obf, --load, or ' 88 | '--test must be used' % errorstr) 89 | sys.exit(1) 90 | 91 | if args.eval: 92 | assert directory 93 | obf = Obfuscator(args.mmap, base=args.base, 94 | verbose=args.verbose, nthreads=args.nthreads, 95 | ncores=args.ncores) 96 | r = obf.evaluate(directory, args.eval) 97 | if r is not None: 98 | print('Output = %d' % r) 99 | except ParseException as e: 100 | print('%s %s' % (errorstr, e)) 101 | sys.exit(1) 102 | return success 103 | 104 | def main(): 105 | parser = argparse.ArgumentParser( 106 | description='Cryptographic program obfuscator.') 107 | subparsers = parser.add_subparsers() 108 | 109 | try: 110 | ncores = os.sysconf('SC_NPROCESSORS_ONLN') 111 | except ValueError: 112 | print(utils.clr_warn('Warning: Unable to count number of cores, defaulting to 1')) 113 | ncores = 1 114 | nthreads = ncores 115 | secparam = 24 116 | 117 | parser_bp = subparsers.add_parser( 118 | 'bp', 119 | help='commands for circuit -> branching program conversion') 120 | parser_bp.add_argument('--eval', 121 | metavar='INPUT', action='store', type=str, 122 | help='evaluate branching program on INPUT') 123 | parser_bp.add_argument('--load', 124 | metavar='FILE', action='store', type=str, 125 | help='load circuit or branching program from FILE') 126 | parser_bp.add_argument('--test', 127 | metavar='FILE', action='store', type=str, 128 | help='test branching program conversion for FILE') 129 | parser_bp.add_argument('--test-all', 130 | metavar='DIR', nargs='?', const='circuits/', 131 | help='test branching program conversion for all circuits in DIR (default: %(const)s)') 132 | parser_bp.add_argument('--base', 133 | metavar='B', action='store', type=int, default=None, 134 | help='base of matrix branching program (default: guess)') 135 | parser_bp.add_argument('--print', 136 | action='store_true', 137 | help='print branching program to stdout') 138 | parser_bp.add_argument('-v', '--verbose', 139 | action='store_true', 140 | help='be verbose') 141 | parser_bp.set_defaults(func=bp) 142 | 143 | parser_obf = subparsers.add_parser( 144 | 'obf', 145 | help='commands for obfuscating a circuit/branching program') 146 | parser_obf.add_argument('--eval', 147 | metavar='INPUT', action='store', type=str, 148 | help='evaluate obfuscation on INPUT') 149 | parser_obf.add_argument('--kappa', 150 | metavar='N', action='store', type=int, default=None, 151 | help='set kappa to N (for debugging)') 152 | parser_obf.add_argument('--load-obf', 153 | metavar='DIR', action='store', type=str, 154 | help='load obfuscation from DIR') 155 | parser_obf.add_argument('--load', 156 | metavar='FILE', action='store', type=str, 157 | help='load circuit or branching program from FILE') 158 | parser_obf.add_argument('--test', 159 | metavar='FILE', action='store', type=str, 160 | help='test circuit or branching program from FILE') 161 | parser_obf.add_argument('--test-all', 162 | metavar='DIR', nargs='?', const='circuits/', 163 | help='test obfuscation for all circuits in DIR (default: %(const)s)') 164 | parser_obf.add_argument('--mmap', metavar='M', type=str, default='CLT', 165 | action='store', 166 | help='use multilinear map M [CLT, GGH, DUMMY] (default: %(default)s)') 167 | parser_obf.add_argument('--save', 168 | metavar='DIR', action='store', type=str, 169 | help='save obfuscation to DIR') 170 | parser_obf.add_argument('--secparam', 171 | metavar='N', action='store', type=int, 172 | default=secparam, help='security parameter (default: %(default)s)') 173 | parser_obf.add_argument('--nthreads', 174 | metavar='N', action='store', type=int, default=nthreads, 175 | help='number of threads to use in threadpool (default: %(default)s)') 176 | parser_obf.add_argument('--ncores', 177 | metavar='N', action='store', type=int, default=ncores, 178 | help='number of cores to use for OpenMP (default: %(default)s)') 179 | parser_obf.add_argument('--base', 180 | metavar='B', action='store', type=int, default=None, 181 | help='base of matrix branching program (default: guess)') 182 | parser_obf.add_argument('--seed', 183 | metavar='FILE', action='store', type=str, 184 | help='load seed from FILE') 185 | parser_obf.add_argument('--no-randomization', action='store_true', 186 | help='turn of branching program randomization') 187 | parser_obf.add_argument('-v', '--verbose', 188 | action='store_true', 189 | help='be verbose') 190 | parser_obf.set_defaults(func=obf) 191 | 192 | args = parser.parse_args() 193 | return args.func(args) 194 | -------------------------------------------------------------------------------- /src/obfuscator.c: -------------------------------------------------------------------------------- 1 | #include "obfuscator.h" 2 | #include "thpool.h" 3 | #include "thpool_fns.h" 4 | 5 | #include 6 | #include 7 | #include 8 | #include 9 | 10 | typedef struct obf_state_s { 11 | threadpool thpool; 12 | uint64_t secparam; 13 | enum mmap_e type; 14 | mmap_sk mmap; 15 | const mmap_vtable *vtable; 16 | aes_randstate_t rand; 17 | uint64_t nthreads; 18 | aes_randstate_t *rands; 19 | const char *dir; 20 | uint64_t nzs; 21 | fmpz_mat_t *randomizer; 22 | fmpz_mat_t *inverse; 23 | uint64_t flags; 24 | } obf_state_t; 25 | 26 | 27 | static FILE * 28 | open_indexed_file(const char *dir, const char *file, uint64_t index, 29 | const char *mode) 30 | { 31 | char fname[50]; 32 | 33 | (void) snprintf(fname, 50, "%lu.%s", index, file); 34 | return open_file(dir, fname, mode); 35 | } 36 | 37 | obf_state_t * 38 | obf_init(enum mmap_e type, const char *dir, size_t secparam, size_t kappa, 39 | size_t nzs, size_t nthreads, size_t ncores, char *seed, 40 | uint64_t flags) 41 | { 42 | obf_state_t *s = NULL; 43 | 44 | if (secparam == 0 || kappa == 0 || nzs == 0) 45 | return NULL; 46 | 47 | s = calloc(1, sizeof(obf_state_t)); 48 | if (s == NULL) 49 | return NULL; 50 | 51 | s->secparam = secparam; 52 | s->type = type; 53 | s->dir = dir; 54 | s->nzs = nzs; 55 | s->flags = flags; 56 | s->randomizer = malloc(sizeof(fmpz_mat_t)); 57 | s->inverse = malloc(sizeof(fmpz_mat_t)); 58 | 59 | switch (s->type) { 60 | case MMAP_DUMMY: 61 | s->vtable = &dummy_vtable; 62 | break; 63 | case MMAP_CLT: 64 | s->vtable = &clt_vtable; 65 | break; 66 | case MMAP_GGHLITE: 67 | s->vtable = &gghlite_vtable; 68 | break; 69 | default: 70 | return NULL; 71 | } 72 | 73 | if (seed) { 74 | FILE *f; 75 | char dest[AES_SEED_BYTE_SIZE]; 76 | size_t n; 77 | 78 | if ((f = fopen(seed, "r")) == NULL) { 79 | fprintf(stderr, "unable to open seed file '%s'\n", seed); 80 | return NULL; 81 | } 82 | n = fread(dest, sizeof(*dest), AES_SEED_BYTE_SIZE, f); 83 | if (n != AES_SEED_BYTE_SIZE) { 84 | fprintf(stderr, "unable to read %d bytes of seed, only %ld bytes read\n", 85 | AES_SEED_BYTE_SIZE, n); 86 | fclose(f); 87 | return NULL; 88 | } 89 | fclose(f); 90 | if (s->flags & OBFUSCATOR_FLAG_VERBOSE) { 91 | fprintf(stderr, " Seed: "); 92 | for (int i = 0; i < AES_SEED_BYTE_SIZE; ++i) { 93 | fprintf(stderr, "%02x", dest[i]); 94 | } 95 | fprintf(stderr, "\n"); 96 | } 97 | aes_randinit_seedn(s->rand, dest, AES_SEED_BYTE_SIZE, NULL, 0); 98 | } else { 99 | (void) aes_randinit(s->rand); 100 | } 101 | if (nthreads == 0) 102 | nthreads = ncores; 103 | s->thpool = thpool_init(nthreads); 104 | s->nthreads = nthreads; 105 | 106 | /* Generate dedicated randomness for threads */ 107 | s->rands = calloc(nthreads, sizeof(aes_randstate_t)); 108 | for (uint64_t i = 0; i < nthreads; ++i) { 109 | unsigned char *buf; 110 | size_t nbits = 128, nbytes; 111 | 112 | buf = random_aes(s->rand, nbits, &nbytes); 113 | aes_randinit_seedn(s->rands[i], (char *) buf, nbytes, NULL, 0); 114 | free(buf); 115 | } 116 | 117 | if (s->flags & OBFUSCATOR_FLAG_VERBOSE) { 118 | fprintf(stderr, " # Threads: %lu\n", nthreads); 119 | fprintf(stderr, " # Cores: %lu\n", ncores); 120 | if (s->flags & OBFUSCATOR_FLAG_DUAL_INPUT_BP) 121 | fprintf(stderr, " Using dual input branching programs\n"); 122 | if (s->flags & OBFUSCATOR_FLAG_NO_RANDOMIZATION) 123 | fprintf(stderr, " Not randomizing branching programs\n"); 124 | } 125 | 126 | s->mmap = malloc(s->vtable->sk->size); 127 | s->vtable->sk->init(s->mmap, secparam, kappa, nzs, NULL, 0, ncores, s->rand, 128 | s->flags & OBFUSCATOR_FLAG_VERBOSE); 129 | { 130 | FILE *fp = open_file(dir, "params", "w+b"); 131 | s->vtable->pp->fwrite(s->vtable->sk->pp(s->mmap), fp); 132 | fclose(fp); 133 | } 134 | 135 | return s; 136 | } 137 | 138 | void 139 | obf_clear(obf_state_t *s) 140 | { 141 | if (s) { 142 | s->vtable->sk->clear(s->mmap); 143 | free(s->mmap); 144 | aes_randclear(s->rand); 145 | for (uint64_t i = 0; i < s->nthreads; ++i) { 146 | aes_randclear(s->rands[i]); 147 | } 148 | free(s->rands); 149 | free(s->randomizer); 150 | free(s->inverse); 151 | thpool_destroy(s->thpool); 152 | } 153 | free(s); 154 | } 155 | 156 | static void 157 | _fmpz_mat_init_square_rand(obf_state_t *s, fmpz_mat_t mat, fmpz_mat_t inverse, 158 | long n, aes_randstate_t rand, fmpz_t field) 159 | { 160 | int singular; 161 | do { 162 | for (int i = 0; i < n; i++) { 163 | for(int j = 0; j < n; j++) { 164 | fmpz_randm_aes(fmpz_mat_entry(mat, i, j), rand, field); 165 | } 166 | } 167 | if (s->flags & OBFUSCATOR_FLAG_VERBOSE) 168 | fprintf(stderr, " Finding inverse...\n"); 169 | singular = fmpz_modp_matrix_inverse(inverse, mat, n, field); 170 | } while (singular); 171 | } 172 | 173 | static void 174 | _fmpz_mat_init_diagonal_rand(fmpz_mat_t mat, long n, aes_randstate_t rand, 175 | fmpz_t field) 176 | { 177 | fmpz_mat_init(mat, n, n); 178 | fmpz_mat_one(mat); 179 | for (int i = 0; i < n; i++) { 180 | do { 181 | fmpz_randm_aes(fmpz_mat_entry(mat, i, i), rand, field); 182 | } while (fmpz_cmp_ui(fmpz_mat_entry(mat, i, i), 0) == 0); 183 | } 184 | } 185 | 186 | static inline void 187 | fmpz_mat_mul_mod(fmpz_mat_t a, fmpz_mat_t b, fmpz_mat_t c, fmpz_t p) 188 | { 189 | fmpz_mat_mul(a, b, c); 190 | fmpz_mat_scalar_mod_fmpz(a, a, p); 191 | } 192 | 193 | static inline void 194 | fmpz_layer_mul_left(uint64_t n, fmpz_mat_t *mats, fmpz_mat_t m, fmpz_t p) 195 | { 196 | for (uint64_t i = 0; i < n; ++i) { 197 | fmpz_mat_mul_mod(mats[i], m, mats[i], p); 198 | } 199 | } 200 | 201 | static void 202 | fmpz_layer_mul_right(uint64_t n, fmpz_mat_t *mats, fmpz_mat_t m, fmpz_t p) 203 | { 204 | for (uint64_t i = 0; i < n; ++i) { 205 | fmpz_mat_mul_mod(mats[i], mats[i], m, p); 206 | } 207 | } 208 | 209 | static void 210 | obf_randomize_layer(obf_state_t *s, long nrows, long ncols, 211 | encode_layer_randomization_flag_t rflag, 212 | uint64_t n, fmpz_mat_t *mats) 213 | { 214 | fmpz_t *fields; 215 | 216 | fields = s->vtable->sk->plaintext_fields(s->mmap); 217 | 218 | if (rflag & ENCODE_LAYER_RANDOMIZATION_TYPE_FIRST) { 219 | fmpz_mat_t first; 220 | _fmpz_mat_init_diagonal_rand(first, nrows, s->rand, fields[0]); 221 | fmpz_layer_mul_left(n, mats, first, fields[0]); 222 | fmpz_mat_clear(first); 223 | } 224 | if (rflag & ENCODE_LAYER_RANDOMIZATION_TYPE_LAST) { 225 | fmpz_mat_t last; 226 | _fmpz_mat_init_diagonal_rand(last, ncols, s->rand, fields[0]); 227 | fmpz_layer_mul_right(n, mats, last, fields[0]); 228 | fmpz_mat_clear(last); 229 | } 230 | 231 | if (rflag & ENCODE_LAYER_RANDOMIZATION_TYPE_FIRST 232 | && rflag & ENCODE_LAYER_RANDOMIZATION_TYPE_LAST) { 233 | } else if (rflag & ENCODE_LAYER_RANDOMIZATION_TYPE_FIRST) { 234 | fmpz_mat_init(*s->randomizer, ncols, ncols); 235 | fmpz_mat_init(*s->inverse, ncols, ncols); 236 | _fmpz_mat_init_square_rand(s, *s->randomizer, *s->inverse, ncols, 237 | s->rand, fields[0]); 238 | fmpz_layer_mul_right(n, mats, *s->randomizer, fields[0]); 239 | } else if (rflag & ENCODE_LAYER_RANDOMIZATION_TYPE_MIDDLE) { 240 | fmpz_layer_mul_left(n, mats, *s->inverse, fields[0]); 241 | fmpz_mat_clear(*s->randomizer); 242 | fmpz_mat_clear(*s->inverse); 243 | 244 | fmpz_mat_init(*s->randomizer, ncols, ncols); 245 | fmpz_mat_init(*s->inverse, ncols, ncols); 246 | _fmpz_mat_init_square_rand(s, *s->randomizer, *s->inverse, ncols, 247 | s->rand, fields[0]); 248 | fmpz_layer_mul_right(n, mats, *s->randomizer, fields[0]); 249 | } else if (rflag & ENCODE_LAYER_RANDOMIZATION_TYPE_LAST) { 250 | fmpz_layer_mul_left(n, mats, *s->inverse, fields[0]); 251 | fmpz_mat_clear(*s->randomizer); 252 | fmpz_mat_clear(*s->inverse); 253 | } 254 | 255 | { 256 | fmpz_t alpha; 257 | fmpz_init(alpha); 258 | for (uint64_t i = 0; i < n; ++i) { 259 | do { 260 | fmpz_randm_aes(alpha, s->rand, fields[0]); 261 | } while (fmpz_cmp_ui(alpha, 0) == 0); 262 | fmpz_mat_scalar_mul_fmpz(mats[i], mats[i], alpha); 263 | } 264 | fmpz_clear(alpha); 265 | } 266 | 267 | free(fields); 268 | } 269 | 270 | static int 271 | add_work_write_layer(obf_state_t *s, uint64_t n, long inp, long idx, 272 | long nrows, long ncols, char **names, char *tag, 273 | mmap_enc_mat_t **enc_mats) 274 | { 275 | struct write_layer_s *wl_s; 276 | wl_s = malloc(sizeof(struct write_layer_s)); 277 | wl_s->vtable = s->vtable; 278 | wl_s->dir = s->dir; 279 | wl_s->n = n; 280 | wl_s->enc_mats = enc_mats; 281 | wl_s->names = names; 282 | wl_s->inp = inp; 283 | wl_s->idx = idx; 284 | wl_s->nrows = nrows; 285 | wl_s->ncols = ncols; 286 | wl_s->start = current_time(); 287 | wl_s->verbose = s->flags & OBFUSCATOR_FLAG_VERBOSE; 288 | if (thpool_add_tag(s->thpool, tag, n * nrows * ncols, 289 | thpool_write_layer, wl_s) == OBFUSCATOR_ERR) { 290 | free(wl_s); 291 | return OBFUSCATOR_ERR; 292 | } 293 | return OBFUSCATOR_OK; 294 | } 295 | 296 | static void 297 | add_work(obf_state_t *s, fmpz_mat_t *mats, mmap_enc_mat_t **enc_mats, 298 | int **pows, long c, long i, long j, char *tag) 299 | { 300 | fmpz_t *plaintext; 301 | mmap_enc *enc; 302 | struct encode_elem_s *args; 303 | 304 | plaintext = malloc(sizeof(fmpz_t)); 305 | args = malloc(sizeof(struct encode_elem_s)); 306 | fmpz_init_set(*plaintext, fmpz_mat_entry(mats[c], i, j)); 307 | enc = enc_mats[c][0]->m[i][j]; 308 | args->group = pows[c]; 309 | args->n = 1; 310 | args->plaintext = plaintext; 311 | args->vtable = s->vtable; 312 | args->sk = s->mmap; 313 | args->enc = enc; 314 | args->rand = &s->rand; 315 | 316 | thpool_add_work(s->thpool, thpool_encode_elem, (void *) args, tag); 317 | } 318 | 319 | int 320 | obf_encode_layer(obf_state_t *s, uint64_t n, int **pows, fmpz_mat_t *mats, 321 | long idx, long inp, encode_layer_randomization_flag_t rflag) 322 | { 323 | char tag[10]; 324 | mmap_enc_mat_t **enc_mats; 325 | char **names; 326 | mmap_ro_pp pp = s->vtable->sk->pp(s->mmap); 327 | long nrows, ncols; 328 | 329 | /* TODO: check for mismatched matrices */ 330 | 331 | nrows = mats[0]->r; 332 | ncols = mats[0]->c; 333 | 334 | (void) snprintf(tag, 10, "%ld", idx); 335 | 336 | if (!(s->flags & OBFUSCATOR_FLAG_NO_RANDOMIZATION)) { 337 | double start, end; 338 | start = current_time(); 339 | obf_randomize_layer(s, nrows, ncols, rflag, n, mats); 340 | end = current_time(); 341 | if (s->flags & OBFUSCATOR_FLAG_VERBOSE) 342 | (void) fprintf(stderr, " Randomizing matrix: %f\n", end - start); 343 | } 344 | 345 | enc_mats = calloc(n, sizeof(mmap_enc_mat_t *)); 346 | for (uint64_t c = 0; c < n; ++c) { 347 | enc_mats[c] = malloc(sizeof(mmap_enc_mat_t)); 348 | mmap_enc_mat_init(s->vtable, pp, *enc_mats[c], nrows, ncols); 349 | } 350 | names = calloc(n, sizeof(char *)); 351 | for (uint64_t c = 0; c < n; ++c) { 352 | names[c] = calloc(10, sizeof(char)); 353 | (void) snprintf(names[c], 10, "%lu", c); 354 | } 355 | 356 | if (add_work_write_layer(s, n, inp, idx, nrows, ncols, names, tag, enc_mats) 357 | == OBFUSCATOR_ERR) 358 | return OBFUSCATOR_ERR; 359 | 360 | for (uint64_t c = 0; c < n; ++c) { 361 | for (long i = 0; i < nrows; ++i) { 362 | for (long j = 0; j < ncols; ++j) { 363 | add_work(s, mats, enc_mats, pows, c, i, j, tag); 364 | } 365 | } 366 | } 367 | 368 | return OBFUSCATOR_OK; 369 | } 370 | 371 | int 372 | obf_evaluate(enum mmap_e type, char *dir, uint64_t len, uint64_t *input, 373 | uint64_t bplen, uint64_t ncores, bool verbose) 374 | { 375 | const mmap_vtable *vtable; 376 | mmap_pp pp; 377 | FILE *fp; 378 | mmap_enc_mat_t *result = NULL; 379 | uint64_t nrows, ncols, nrows_prev = 0; 380 | int err = 1, iszero = -1; 381 | double start, end; 382 | 383 | (void) ncores; 384 | 385 | switch (type) { 386 | case MMAP_DUMMY: 387 | vtable = &dummy_vtable; 388 | break; 389 | case MMAP_CLT: 390 | vtable = &clt_vtable; 391 | break; 392 | case MMAP_GGHLITE: 393 | vtable = &gghlite_vtable; 394 | break; 395 | default: 396 | return err; 397 | } 398 | 399 | if(NULL == (pp = malloc(vtable->pp->size))) 400 | goto done; 401 | if ((fp = open_file(dir, "params", "r+b")) == NULL) 402 | goto done; 403 | vtable->pp->fread(pp, fp); 404 | fclose(fp); 405 | 406 | for (uint64_t layer = 0; layer < bplen; ++layer) { 407 | uint64_t inp; 408 | char str[10]; 409 | mmap_enc_mat_t *left, *right; 410 | 411 | start = current_time(); 412 | 413 | // determine the size of the matrix 414 | if ((fp = open_indexed_file(dir, "nrows", layer, "r+b")) == NULL) 415 | goto done; 416 | fread(&nrows, sizeof nrows, 1, fp); 417 | fclose(fp); 418 | if ((fp = open_indexed_file(dir, "ncols", layer, "r+b")) == NULL) 419 | goto done; 420 | fread(&ncols, sizeof ncols, 1, fp); 421 | fclose(fp); 422 | 423 | // find out the input bit for the given layer 424 | if ((fp = open_indexed_file(dir, "input", layer, "r+b")) == NULL) 425 | goto done; 426 | fread(&inp, sizeof inp, 1, fp); 427 | fclose(fp); 428 | 429 | if (inp > len) { 430 | fprintf(stderr, "invalid input: %lu > %ld\n", inp, len); 431 | goto done; 432 | } 433 | // load in appropriate matrix for the given input value 434 | (void) snprintf(str, 10, "%lu", input[inp]); 435 | if ((fp = open_indexed_file(dir, str, layer, "r+b")) == NULL) 436 | goto done; 437 | 438 | if (layer == 0) { 439 | result = (mmap_enc_mat_t *) malloc(sizeof(mmap_enc_mat_t)); 440 | mmap_enc_mat_init(vtable, pp, *result, nrows, ncols); 441 | for (uint64_t i = 0; i < nrows; ++i) { 442 | for (uint64_t j = 0; j < ncols; ++j) { 443 | vtable->enc->fread(result[0]->m[i][j], fp); 444 | } 445 | } 446 | nrows_prev = nrows; 447 | } else { 448 | left = result; 449 | right = (mmap_enc_mat_t *) malloc(sizeof(mmap_enc_mat_t)); 450 | mmap_enc_mat_init(vtable, pp, *right, nrows, ncols); 451 | for (uint64_t i = 0; i < nrows; ++i) { 452 | for (uint64_t j = 0; j < ncols; ++j) { 453 | vtable->enc->fread(right[0]->m[i][j], fp); 454 | } 455 | } 456 | 457 | result = (mmap_enc_mat_t *) malloc(sizeof(mmap_enc_mat_t)); 458 | mmap_enc_mat_init(vtable, pp, *result, nrows_prev, ncols); 459 | mmap_enc_mat_mul_par(vtable, pp, *result, *left, *right); 460 | mmap_enc_mat_clear(vtable, *left); 461 | mmap_enc_mat_clear(vtable, *right); 462 | free(left); 463 | free(right); 464 | } 465 | 466 | fclose(fp); 467 | 468 | end = current_time(); 469 | 470 | if (verbose && layer != 0) 471 | (void) fprintf(stderr, " Multiplying matrices: %f\n", end - start); 472 | } 473 | err = 0; 474 | 475 | done: 476 | if (!err) { 477 | start = current_time(); 478 | if (result[0]->nrows == 1 && result[0]->ncols == 1) 479 | iszero = vtable->enc->is_zero(result[0]->m[0][0], pp); 480 | else 481 | iszero = vtable->enc->is_zero(result[0]->m[0][1], pp); 482 | end = current_time(); 483 | if (verbose) 484 | (void) fprintf(stderr, " Zero test: %f\n", end - start); 485 | } 486 | 487 | if (result) { 488 | mmap_enc_mat_clear(vtable, *result); 489 | free(result); 490 | } 491 | 492 | if (pp) { 493 | vtable->pp->clear(pp); 494 | free(pp); 495 | } 496 | 497 | return iszero; 498 | } 499 | 500 | void 501 | obf_wait(obf_state_t *s) 502 | { 503 | thpool_wait(s->thpool); 504 | } 505 | -------------------------------------------------------------------------------- /src/thpool.c: -------------------------------------------------------------------------------- 1 | /* ******************************** 2 | * Author: Johan Hanssen Seferidis 3 | * License: MIT 4 | * Description: Library providing a threading pool where you can add 5 | * work. For usage, check the thpool.h file or README.md 6 | * 7 | *//** @file thpool.h *//* 8 | * 9 | ********************************/ 10 | 11 | #include 12 | #include 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #if defined(__linux__) 20 | #include 21 | #endif 22 | #include "thpool.h" 23 | 24 | #ifdef THPOOL_DEBUG 25 | #define THPOOL_DEBUG 1 26 | #else 27 | #define THPOOL_DEBUG 0 28 | #endif 29 | 30 | static volatile int threads_keepalive; 31 | static volatile int threads_on_hold; 32 | 33 | 34 | /* ========================== STRUCTURES ============================ */ 35 | 36 | 37 | /* Binary semaphore */ 38 | typedef struct bsem { 39 | pthread_mutex_t mutex; 40 | pthread_cond_t cond; 41 | int v; 42 | } bsem; 43 | 44 | 45 | /* Job */ 46 | typedef struct job { 47 | struct job* prev; /* pointer to previous job */ 48 | void* (*function)(void* arg); /* function pointer */ 49 | void* arg; /* function's argument */ 50 | char *tag; 51 | } job; 52 | 53 | 54 | /* Job queue */ 55 | typedef struct jobqueue{ 56 | pthread_mutex_t rwmutex; /* used for queue r/w access */ 57 | job *front; /* pointer to front of queue */ 58 | job *rear; /* pointer to rear of queue */ 59 | bsem *has_jobs; /* flag as binary semaphore */ 60 | int len; /* number of jobs in queue */ 61 | } jobqueue; 62 | 63 | 64 | /* Thread */ 65 | typedef struct thread{ 66 | int id; /* friendly id */ 67 | pthread_t pthread; /* pointer to actual thread */ 68 | struct thpool_* thpool_p; /* access to thpool */ 69 | } thread; 70 | 71 | typedef struct tag { 72 | char *name; 73 | int len; 74 | void * (*function)(void *arg); 75 | void *arg; 76 | } tag_t; 77 | 78 | typedef struct taglist_ { 79 | int num; 80 | pthread_mutex_t lock; 81 | tag_t *tags; 82 | } taglist_t; 83 | 84 | // XXX: We don't clear the tag list, so if we have more than 1024 tags all hell 85 | // breaks loose! 86 | #define TAGLIST_SIZE 1024 87 | 88 | /* Threadpool */ 89 | typedef struct thpool_{ 90 | thread** threads; /* pointer to threads */ 91 | volatile int num_threads_alive; /* threads currently alive */ 92 | volatile int num_threads_working; /* threads currently working */ 93 | pthread_mutex_t thcount_lock; /* used for thread count etc */ 94 | pthread_cond_t threads_all_idle; /* signal to thpool_wait */ 95 | jobqueue* jobqueue_p; /* pointer to the job queue */ 96 | taglist_t* tlist; 97 | } thpool_; 98 | 99 | 100 | /* ========================== PROTOTYPES ============================ */ 101 | 102 | 103 | static int thread_init(thpool_* thpool_p, struct thread** thread_p, int id); 104 | static void* thread_do(void* varg); 105 | static void thread_hold(int num); 106 | static void thread_destroy(struct thread* thread_p); 107 | 108 | static int jobqueue_init(thpool_* thpool_p); 109 | static void jobqueue_clear(thpool_* thpool_p); 110 | static void jobqueue_push(thpool_* thpool_p, struct job* newjob_p); 111 | static struct job* jobqueue_pull(thpool_* thpool_p); 112 | static void jobqueue_destroy(thpool_* thpool_p); 113 | 114 | static void bsem_init(struct bsem *bsem_p, int value); 115 | static void bsem_reset(struct bsem *bsem_p); 116 | static void bsem_post(struct bsem *bsem_p); 117 | static void bsem_post_all(struct bsem *bsem_p); 118 | static void bsem_wait(struct bsem *bsem_p); 119 | 120 | 121 | /* ========================== THREADPOOL ============================ */ 122 | 123 | 124 | /* Initialise thread pool */ 125 | struct thpool_* 126 | thpool_init(int num_threads) 127 | { 128 | threads_on_hold = 0; 129 | threads_keepalive = 1; 130 | 131 | if (num_threads < 0){ 132 | num_threads = 0; 133 | } 134 | 135 | /* Make new thread pool */ 136 | thpool_* thpool_p; 137 | thpool_p = (struct thpool_*)malloc(sizeof(struct thpool_)); 138 | if (thpool_p == NULL) { 139 | fprintf(stderr, "thpool_init(): Could not allocate memory for thread pool\n"); 140 | return NULL; 141 | } 142 | thpool_p->num_threads_alive = 0; 143 | thpool_p->num_threads_working = 0; 144 | 145 | /* Initialise the job queue */ 146 | if (jobqueue_init(thpool_p) == -1) { 147 | fprintf(stderr, "thpool_init(): Could not allocate memory for job queue\n"); 148 | free(thpool_p); 149 | return NULL; 150 | } 151 | 152 | /* Make threads in pool */ 153 | thpool_p->threads = (struct thread**)malloc(num_threads * sizeof(struct thread *)); 154 | if (thpool_p->threads == NULL){ 155 | fprintf(stderr, "thpool_init(): Could not allocate memory for threads\n"); 156 | jobqueue_destroy(thpool_p); 157 | free(thpool_p->jobqueue_p); 158 | free(thpool_p); 159 | return NULL; 160 | } 161 | 162 | /* Make tag list */ 163 | taglist_t *tlist; 164 | tlist = (taglist_t *) malloc(sizeof(taglist_t)); 165 | if (tlist == NULL) { 166 | fprintf(stderr, "thpool_init(): Could not allocate memory for job queue\n"); 167 | jobqueue_destroy(thpool_p); 168 | free(thpool_p->jobqueue_p); 169 | free(thpool_p->threads); 170 | free(thpool_p); 171 | return NULL; 172 | } 173 | tlist->num = TAGLIST_SIZE; 174 | tlist->tags = (tag_t *) calloc(tlist->num, sizeof(tag_t)); 175 | if (tlist->tags == NULL) { 176 | fprintf(stderr, "thpool_init(): Could not allocate memory for tag list\n"); 177 | jobqueue_destroy(thpool_p); 178 | free(thpool_p->jobqueue_p); 179 | free(thpool_p->threads); 180 | free(thpool_p); 181 | free(tlist); 182 | return NULL; 183 | } 184 | pthread_mutex_init(&tlist->lock, NULL); 185 | thpool_p->tlist = tlist; 186 | 187 | pthread_mutex_init(&(thpool_p->thcount_lock), NULL); 188 | pthread_cond_init(&thpool_p->threads_all_idle, NULL); 189 | 190 | /* Thread init */ 191 | for (int n = 0; n < num_threads; n++) { 192 | thread_init(thpool_p, &thpool_p->threads[n], n); 193 | if (THPOOL_DEBUG) 194 | printf("THPOOL_DEBUG: Created thread %d in pool \n", n); 195 | } 196 | 197 | /* Wait for threads to initialize */ 198 | while (thpool_p->num_threads_alive != num_threads) {} 199 | 200 | return thpool_p; 201 | } 202 | 203 | int 204 | thpool_add_tag(thpool_ *thpool_p, char *tag, int length, 205 | void * (fn)(void *arg), void *arg) 206 | { 207 | for (int i = 0; i < thpool_p->tlist->num; ++i) { 208 | struct tag *t = &thpool_p->tlist->tags[i]; 209 | if (t->name == NULL) { 210 | t->name = (char *) calloc(strlen(tag) + 1, sizeof(char)); 211 | (void) strcpy(t->name, tag); 212 | t->len = length; 213 | t->function = fn; 214 | t->arg = arg; 215 | return 0; 216 | } else if (strcmp(tag, t->name) == 0) { 217 | fprintf(stderr, "thpool_add_tag(): tag '%s' already in tag list\n", tag); 218 | return -1; 219 | } 220 | } 221 | 222 | fprintf(stderr, "thpool_add_tag(): tag list full!\n"); 223 | return -1; 224 | } 225 | 226 | 227 | static int 228 | tag_decrement(thpool_ *thpool_p, char *tag) 229 | { 230 | for (int i = 0; i < thpool_p->tlist->num; ++i) { 231 | struct tag *t = &thpool_p->tlist->tags[i]; 232 | if (strcmp(t->name, tag) == 0) { 233 | pthread_mutex_lock(&thpool_p->tlist->lock); 234 | t->len--; 235 | pthread_mutex_unlock(&thpool_p->tlist->lock); 236 | if (t->len == 0) { 237 | t->function(t->arg); 238 | } 239 | return 0; 240 | } 241 | } 242 | fprintf(stderr, "tag_decrement(): tag does not exist in tag list\n"); 243 | return -1; 244 | } 245 | 246 | 247 | /* Add work to the thread pool */ 248 | int 249 | thpool_add_work(thpool_* thpool_p, void *(*function_p)(void*), void* arg_p, 250 | char *tag) 251 | { 252 | job *newjob; 253 | 254 | newjob = (struct job*) malloc(sizeof(struct job)); 255 | if (newjob == NULL) { 256 | fprintf(stderr, "thpool_add_work(): Could not allocate memory for new job\n"); 257 | return -1; 258 | } 259 | 260 | /* add function and argument */ 261 | newjob->function = function_p; 262 | newjob->arg = arg_p; 263 | newjob->tag = (char *) calloc(strlen(tag) + 1, sizeof(char)); 264 | (void) strcpy(newjob->tag, tag); 265 | 266 | /* add job to queue */ 267 | pthread_mutex_lock(&thpool_p->jobqueue_p->rwmutex); 268 | jobqueue_push(thpool_p, newjob); 269 | pthread_mutex_unlock(&thpool_p->jobqueue_p->rwmutex); 270 | 271 | return 0; 272 | } 273 | 274 | 275 | /* Wait until all jobs have finished */ 276 | void thpool_wait(thpool_* thpool_p){ 277 | pthread_mutex_lock(&thpool_p->thcount_lock); 278 | while (thpool_p->jobqueue_p->len || thpool_p->num_threads_working) { 279 | pthread_cond_wait(&thpool_p->threads_all_idle, &thpool_p->thcount_lock); 280 | } 281 | pthread_mutex_unlock(&thpool_p->thcount_lock); 282 | } 283 | 284 | 285 | /* Destroy the threadpool */ 286 | void thpool_destroy(thpool_* thpool_p){ 287 | /* No need to destory if it's NULL */ 288 | if (thpool_p == NULL) return ; 289 | 290 | volatile int threads_total = thpool_p->num_threads_alive; 291 | 292 | /* End each thread's infinite loop */ 293 | threads_keepalive = 0; 294 | 295 | /* Give one second to kill idle threads */ 296 | double TIMEOUT = 1.0; 297 | time_t start, end; 298 | double tpassed = 0.0; 299 | time (&start); 300 | while (tpassed < TIMEOUT && thpool_p->num_threads_alive){ 301 | bsem_post_all(thpool_p->jobqueue_p->has_jobs); 302 | time (&end); 303 | tpassed = difftime(end,start); 304 | } 305 | 306 | /* Poll remaining threads */ 307 | while (thpool_p->num_threads_alive){ 308 | bsem_post_all(thpool_p->jobqueue_p->has_jobs); 309 | sleep(1); 310 | } 311 | 312 | /* Job queue cleanup */ 313 | jobqueue_destroy(thpool_p); 314 | free(thpool_p->jobqueue_p); 315 | 316 | /* Deallocs */ 317 | int n; 318 | for (n=0; n < threads_total; n++){ 319 | thread_destroy(thpool_p->threads[n]); 320 | } 321 | free(thpool_p->threads); 322 | free(thpool_p); 323 | } 324 | 325 | 326 | /* Pause all threads in threadpool */ 327 | void 328 | thpool_pause(thpool_* thpool_p) 329 | { 330 | for (int n = 0; n < thpool_p->num_threads_alive; n++) { 331 | pthread_kill(thpool_p->threads[n]->pthread, SIGUSR1); 332 | } 333 | } 334 | 335 | /* Resume all threads in threadpool */ 336 | void 337 | thpool_resume(thpool_* thpool_p) 338 | { 339 | threads_on_hold = 0; 340 | } 341 | 342 | 343 | /* ============================ THREAD ============================== */ 344 | 345 | 346 | /* Initialize a thread in the thread pool 347 | * 348 | * @param thread address to the pointer of the thread to be created 349 | * @param id id to be given to the thread 350 | * @return 0 on success, -1 otherwise. 351 | */ 352 | static int thread_init (thpool_* thpool_p, struct thread** thread_p, int id){ 353 | 354 | *thread_p = (struct thread*)malloc(sizeof(struct thread)); 355 | if (thread_p == NULL){ 356 | fprintf(stderr, "thpool_init(): Could not allocate memory for thread\n"); 357 | return -1; 358 | } 359 | 360 | (*thread_p)->thpool_p = thpool_p; 361 | (*thread_p)->id = id; 362 | 363 | pthread_create(&(*thread_p)->pthread, NULL, thread_do, (*thread_p)); 364 | pthread_detach((*thread_p)->pthread); 365 | return 0; 366 | } 367 | 368 | 369 | /* Sets the calling thread on hold */ 370 | static void 371 | thread_hold (int num) 372 | { 373 | threads_on_hold = 1; 374 | while (threads_on_hold) { 375 | sleep(1); 376 | } 377 | } 378 | 379 | 380 | /* What each thread is doing 381 | * 382 | * In principle this is an endless loop. The only time this loop gets interrupted 383 | * is once thpool_destroy() is invoked or the program exits. 384 | * 385 | * @param thread thread that will run this function 386 | * @return nothing 387 | */ 388 | static void * 389 | thread_do(void* varg) 390 | { 391 | struct thread* thread_p = (struct thread *) varg; 392 | /* Set thread name for profiling and debuging */ 393 | char thread_name[128] = {0}; 394 | sprintf(thread_name, "thread-pool-%d", thread_p->id); 395 | 396 | #if defined(__linux__) 397 | /* Use prctl instead to prevent using _GNU_SOURCE flag and implicit declaration */ 398 | prctl(PR_SET_NAME, thread_name); 399 | #elif defined(__APPLE__) && defined(__MACH__) 400 | pthread_setname_np(thread_name); 401 | #else 402 | fprintf(stderr, "thread_do(): pthread_setname_np is not supported on this system"); 403 | #endif 404 | 405 | /* Assure all threads have been created before starting serving */ 406 | thpool_ *thpool_p = thread_p->thpool_p; 407 | 408 | /* Register signal handler */ 409 | struct sigaction act; 410 | sigemptyset(&act.sa_mask); 411 | act.sa_flags = 0; 412 | act.sa_handler = thread_hold; 413 | if (sigaction(SIGUSR1, &act, NULL) == -1) { 414 | fprintf(stderr, "thread_do(): cannot handle SIGUSR1"); 415 | } 416 | 417 | /* Mark thread as alive (initialized) */ 418 | pthread_mutex_lock(&thpool_p->thcount_lock); 419 | thpool_p->num_threads_alive += 1; 420 | pthread_mutex_unlock(&thpool_p->thcount_lock); 421 | 422 | while (threads_keepalive) { 423 | 424 | bsem_wait(thpool_p->jobqueue_p->has_jobs); 425 | 426 | if (threads_keepalive) { 427 | 428 | job* job_p; 429 | 430 | pthread_mutex_lock(&thpool_p->thcount_lock); 431 | thpool_p->num_threads_working++; 432 | pthread_mutex_unlock(&thpool_p->thcount_lock); 433 | 434 | /* Read job from queue and execute it */ 435 | pthread_mutex_lock(&thpool_p->jobqueue_p->rwmutex); 436 | job_p = jobqueue_pull(thpool_p); 437 | pthread_mutex_unlock(&thpool_p->jobqueue_p->rwmutex); 438 | if (job_p) { 439 | job_p->function(job_p->arg); 440 | tag_decrement(thpool_p, job_p->tag); 441 | free(job_p->tag); 442 | free(job_p); 443 | } 444 | 445 | pthread_mutex_lock(&thpool_p->thcount_lock); 446 | thpool_p->num_threads_working--; 447 | if (!thpool_p->num_threads_working) { 448 | pthread_cond_signal(&thpool_p->threads_all_idle); 449 | } 450 | pthread_mutex_unlock(&thpool_p->thcount_lock); 451 | 452 | } 453 | } 454 | pthread_mutex_lock(&thpool_p->thcount_lock); 455 | thpool_p->num_threads_alive --; 456 | pthread_mutex_unlock(&thpool_p->thcount_lock); 457 | 458 | return NULL; 459 | } 460 | 461 | 462 | /* Frees a thread */ 463 | static void 464 | thread_destroy (thread* thread_p) 465 | { 466 | free(thread_p); 467 | } 468 | 469 | 470 | /* ============================ JOB QUEUE =========================== */ 471 | 472 | 473 | /* Initialize queue */ 474 | static int jobqueue_init(thpool_* thpool_p){ 475 | 476 | thpool_p->jobqueue_p = (struct jobqueue*)malloc(sizeof(struct jobqueue)); 477 | if (thpool_p->jobqueue_p == NULL){ 478 | return -1; 479 | } 480 | thpool_p->jobqueue_p->len = 0; 481 | thpool_p->jobqueue_p->front = NULL; 482 | thpool_p->jobqueue_p->rear = NULL; 483 | 484 | thpool_p->jobqueue_p->has_jobs = (struct bsem*)malloc(sizeof(struct bsem)); 485 | if (thpool_p->jobqueue_p->has_jobs == NULL){ 486 | return -1; 487 | } 488 | 489 | pthread_mutex_init(&(thpool_p->jobqueue_p->rwmutex), NULL); 490 | bsem_init(thpool_p->jobqueue_p->has_jobs, 0); 491 | 492 | return 0; 493 | } 494 | 495 | 496 | /* Clear the queue */ 497 | static void jobqueue_clear(thpool_* thpool_p){ 498 | 499 | while(thpool_p->jobqueue_p->len){ 500 | free(jobqueue_pull(thpool_p)); 501 | } 502 | 503 | thpool_p->jobqueue_p->front = NULL; 504 | thpool_p->jobqueue_p->rear = NULL; 505 | bsem_reset(thpool_p->jobqueue_p->has_jobs); 506 | thpool_p->jobqueue_p->len = 0; 507 | 508 | } 509 | 510 | 511 | /* Add (allocated) job to queue 512 | * 513 | * Notice: Caller MUST hold a mutex 514 | */ 515 | static void jobqueue_push(thpool_* thpool_p, struct job* newjob){ 516 | 517 | newjob->prev = NULL; 518 | 519 | switch(thpool_p->jobqueue_p->len){ 520 | 521 | case 0: /* if no jobs in queue */ 522 | thpool_p->jobqueue_p->front = newjob; 523 | thpool_p->jobqueue_p->rear = newjob; 524 | break; 525 | 526 | default: /* if jobs in queue */ 527 | thpool_p->jobqueue_p->rear->prev = newjob; 528 | thpool_p->jobqueue_p->rear = newjob; 529 | 530 | } 531 | thpool_p->jobqueue_p->len++; 532 | 533 | bsem_post(thpool_p->jobqueue_p->has_jobs); 534 | } 535 | 536 | 537 | /* Get first job from queue(removes it from queue) 538 | * 539 | * Notice: Caller MUST hold a mutex 540 | */ 541 | static struct job * 542 | jobqueue_pull(thpool_ *thpool_p) 543 | { 544 | job *job_p; 545 | job_p = thpool_p->jobqueue_p->front; 546 | 547 | switch(thpool_p->jobqueue_p->len) { 548 | 549 | case 0: /* if no jobs in queue */ 550 | break; 551 | 552 | case 1: /* if one job in queue */ 553 | thpool_p->jobqueue_p->front = NULL; 554 | thpool_p->jobqueue_p->rear = NULL; 555 | thpool_p->jobqueue_p->len = 0; 556 | break; 557 | 558 | default: /* if >1 jobs in queue */ 559 | thpool_p->jobqueue_p->front = job_p->prev; 560 | thpool_p->jobqueue_p->len--; 561 | /* more than one job in queue -> post it */ 562 | bsem_post(thpool_p->jobqueue_p->has_jobs); 563 | 564 | } 565 | 566 | return job_p; 567 | } 568 | 569 | 570 | /* Free all queue resources back to the system */ 571 | static void jobqueue_destroy(thpool_* thpool_p){ 572 | jobqueue_clear(thpool_p); 573 | free(thpool_p->jobqueue_p->has_jobs); 574 | } 575 | 576 | 577 | /* ======================== SYNCHRONISATION ========================= */ 578 | 579 | 580 | /* Init semaphore to 1 or 0 */ 581 | static void bsem_init(bsem *bsem_p, int value) { 582 | if (value < 0 || value > 1) { 583 | fprintf(stderr, "bsem_init(): Binary semaphore can take only values 1 or 0"); 584 | exit(1); 585 | } 586 | pthread_mutex_init(&(bsem_p->mutex), NULL); 587 | pthread_cond_init(&(bsem_p->cond), NULL); 588 | bsem_p->v = value; 589 | } 590 | 591 | 592 | /* Reset semaphore to 0 */ 593 | static void bsem_reset(bsem *bsem_p) { 594 | bsem_init(bsem_p, 0); 595 | } 596 | 597 | 598 | /* Post to at least one thread */ 599 | static void bsem_post(bsem *bsem_p) { 600 | pthread_mutex_lock(&bsem_p->mutex); 601 | bsem_p->v = 1; 602 | pthread_cond_signal(&bsem_p->cond); 603 | pthread_mutex_unlock(&bsem_p->mutex); 604 | } 605 | 606 | 607 | /* Post to all threads */ 608 | static void bsem_post_all(bsem *bsem_p) { 609 | pthread_mutex_lock(&bsem_p->mutex); 610 | bsem_p->v = 1; 611 | pthread_cond_broadcast(&bsem_p->cond); 612 | pthread_mutex_unlock(&bsem_p->mutex); 613 | } 614 | 615 | 616 | /* Wait on semaphore until semaphore has value 0 */ 617 | static void bsem_wait(bsem* bsem_p) { 618 | pthread_mutex_lock(&bsem_p->mutex); 619 | while (bsem_p->v != 1) { 620 | pthread_cond_wait(&bsem_p->cond, &bsem_p->mutex); 621 | } 622 | bsem_p->v = 0; 623 | pthread_mutex_unlock(&bsem_p->mutex); 624 | } 625 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | 294 | Copyright (C) 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | , 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. 340 | --------------------------------------------------------------------------------