├── .gitignore ├── .gitmodules ├── INSTALL.md ├── LICENSE ├── Makefile ├── README.md ├── doc └── .gitignore ├── doxyfile ├── libpdp ├── Makefile ├── inc │ ├── pdp.h │ └── pdp │ │ ├── apdp.h │ │ ├── apdp_types.h │ │ ├── cpor.h │ │ ├── cpor_types.h │ │ ├── macpdp.h │ │ ├── macpdp_types.h │ │ ├── mrpdp.h │ │ ├── mrpdp_types.h │ │ ├── sepdp.h │ │ ├── sepdp_types.h │ │ └── types.h └── src │ ├── apdp │ ├── apdp.c │ ├── apdp_file.c │ ├── apdp_key.c │ ├── apdp_misc.h │ ├── apdp_s3.c │ ├── apdp_serialize.c │ └── apdp_storage.h │ ├── cpor │ ├── cpor.c │ ├── cpor_file.c │ ├── cpor_key.c │ ├── cpor_misc.h │ ├── cpor_s3.c │ ├── cpor_serialize.c │ └── cpor_storage.h │ ├── macpdp │ ├── macpdp.c │ ├── macpdp_file.c │ ├── macpdp_key.c │ ├── macpdp_s3.c │ └── macpdp_storage.h │ ├── mrpdp │ ├── mrpdp.c │ ├── mrpdp_file.c │ ├── mrpdp_key.c │ ├── mrpdp_misc.h │ ├── mrpdp_s3.c │ ├── mrpdp_serialize.c │ └── mrpdp_storage.h │ ├── pdp_file.c │ ├── pdp_generic.c │ ├── pdp_key.c │ ├── pdp_misc.c │ ├── pdp_misc.h │ ├── pdp_s3.c │ ├── pdp_storage.h │ └── sepdp │ ├── sepdp.c │ ├── sepdp_file.c │ ├── sepdp_key.c │ ├── sepdp_misc.h │ ├── sepdp_s3.c │ ├── sepdp_serialize.c │ └── sepdp_storage.h └── tools ├── Makefile ├── lib ├── libs3-2.0 │ ├── .gitignore │ ├── COPYING │ ├── ChangeLog │ ├── GNUmakefile │ ├── GNUmakefile.mingw │ ├── GNUmakefile.osx │ ├── INSTALL │ ├── LICENSE │ ├── README │ ├── TODO │ ├── archlinux │ │ └── PKGBUILD │ ├── debian │ │ ├── changelog │ │ ├── changelog.Debian │ │ ├── control │ │ ├── control.dev │ │ └── postinst │ ├── doxyfile │ ├── inc │ │ ├── error_parser.h │ │ ├── libs3.h │ │ ├── mingw │ │ │ ├── pthread.h │ │ │ └── sys │ │ │ │ ├── select.h │ │ │ │ └── utsname.h │ │ ├── request.h │ │ ├── request_context.h │ │ ├── response_headers_handler.h │ │ ├── simplexml.h │ │ ├── string_buffer.h │ │ └── util.h │ ├── libs3.spec │ ├── mswin │ │ ├── libs3.def │ │ └── rmrf.bat │ ├── re.p │ ├── src │ │ ├── acl.c │ │ ├── bucket.c │ │ ├── error_parser.c │ │ ├── general.c │ │ ├── mingw_functions.c │ │ ├── mingw_s3_functions.c │ │ ├── object.c │ │ ├── request.c │ │ ├── request_context.c │ │ ├── response_headers_handler.c │ │ ├── s3.c │ │ ├── service.c │ │ ├── service_access_logging.c │ │ ├── simplexml.c │ │ ├── size │ │ ├── testsimplexml.c │ │ └── util.c │ ├── test.txt │ └── test │ │ ├── badxml_01.xml │ │ ├── goodxml_01.xml │ │ ├── goodxml_02.xml │ │ ├── goodxml_03.xml │ │ └── test.sh └── time_it │ ├── Makefile │ ├── time_it.c │ └── time_it.h ├── pdp_bench ├── s3_util.py ├── s3curl.pl ├── src └── pdp_bench.c ├── test-keys ├── apdp.pri ├── apdp.pub ├── cpor.pri └── cpor.pub └── test.py /.gitignore: -------------------------------------------------------------------------------- 1 | # We bring in libs3 from its source 2 | libs3/* 3 | 4 | # files we generate for testing 5 | tools/test-files/* 6 | 7 | # for MacOS 8 | .DS_Store 9 | 10 | # Object files 11 | *.o 12 | 13 | # Libraries 14 | *.lib 15 | *.a 16 | 17 | # Shared objects (inc. Windows DLLs) 18 | *.dll 19 | *.so 20 | *.so.* 21 | *.dylib 22 | 23 | # Executables 24 | *.exe 25 | *.out 26 | *.app 27 | 28 | -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gondree/libpdp/391648d2b11a4216d106ac22887b82adf5d8288d/.gitmodules -------------------------------------------------------------------------------- /INSTALL.md: -------------------------------------------------------------------------------- 1 | ## Building 2 | 3 | The `libpdp` project has the following significant dependencies: 4 | 5 | * Library: `libpthread`, `libssl`, [`libs3-2.x`](http://libs3.ischo.com.s3.amazonaws.com/index.html) 6 | * Unit Tests: Python-2.7 +[`boto`](https://github.com/boto/boto) 2.2.2 7 | 8 | The following steps are to build on an EC2 Ubuntu 12.07 microimage: 9 | 10 | apt-get update 11 | apt-get install make g++ git 12 | apt-get install libcurl3-dev libxml-dev 13 | apt-get install python-setuptools 14 | easy_install boto 15 | 16 | To build the library `libpdp/libpdp.a`, perform 17 | 18 | make libpdp 19 | 20 | ### Install 21 | There is no install. 22 | 23 | We do not "productize" the code as a shared library because (duh) its not 24 | a production-worthy library, yet. It is proof-of-concept code to 25 | investigate PDP schemes. See the doxygen notes for numerous deficiencies 26 | re: protecting secrets and serializing key data. It supports 27 | various hacky/poor-practice methods to load key data from environment 28 | variables, to by-pass appropriate key-protection mechanisms, to do testing and 29 | benchmarking noninteractively. Use the library to protect important data at 30 | your own peril. 31 | 32 | A simple bechmarking and testing application exists that uses this library, 33 | under `tools/pdp_bench`, which you can build using `make all`. 34 | 35 | ---- 36 | 37 | ### libs3 38 | This project distributes copies of `libs3` for conveience. These are covered 39 | under their own license (GPLv3, see `libs3/LICENSE` for details). 40 | If the `_S3_SUPPORT` compiler toggle is not defined, `libpdp` is built without 41 | support for S3 as a back-end storage option, and has no dependencies on `libs3`. 42 | 43 | 44 | ### Unit Tests 45 | To run the `libpdp` unit tests, you will need to build `tools/pdp_bench` 46 | and drive the test using a Python-based unit test script: 47 | 48 | ./tools/test.py 49 | 50 | 51 | ### Testing against S3 52 | By default, the S3 unit tests are skipped. To run these, you will need to 53 | set the 'USE_S3' environment variable, and set some other variables. 54 | 55 | env USE_S3=Yes S3_ACCESS_KEY_ID= S3_SECRET_ACCESS_KEY= S3_HOSTNAME= ./tools/test.py TestS3Storage 56 | 57 | These are credentials to something providing an S3-compatible interface. 58 | Amazon S3 obviously provides such an inteface. Also, DevStack can be run 59 | locally and provides an appropriate S3-compatible interface. 60 | 61 | Note: *This is not an appropriate way to secure these credentials. 62 | This is only for testing.* 63 | 64 | To see that the remote interface is working 65 | and these environment variables are set correctly, 66 | use the provided utility to talk to the inferface using `boto`: 67 | 68 | ./tools/s3_util.py --ls 69 | 70 | The above command lists the remote buckets, and is a nice check to 71 | verify that the remote host exists and provides an appropriate 72 | S3-compatible interface and that your credentials work. 73 | 74 | 75 | ### Benchmarking 76 | The tools to run and analyze the benchmarks can be found under `bench` 77 | To gather performance test data, you must build `tools/pdp_bench` 78 | against a special instrumented version of `libs3` (found under 79 | `tools/lib/libs3-2.0`). This can be done by changing 80 | `tools/Makefile` so that the `S3_LIB` linker variable points 81 | to `$(LIBS3_TIMING)`. 82 | 83 | env S3_ACCESS_KEY_ID= S3_SECRET_ACCESS_KEY= S3_HOSTNAME= ./bench/run-tests.py 84 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2012-2013, Mark Gondree 2 | Copyright (c) 2008-2010, Zachary N J Peterson 3 | All rights reserved. 4 | 5 | Redistribution and use in source and binary forms, with or without 6 | modification, are permitted provided that the following conditions are met: 7 | 8 | - Redistributions of source code must retain the above copyright notice, 9 | this list of conditions and the following disclaimer. 10 | - Redistributions in binary form must reproduce the above copyright notice, 11 | this list of conditions and the following disclaimer in the documentation 12 | and/or other materials provided with the distribution. 13 | 14 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 15 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17 | ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE 18 | LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 19 | CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 20 | SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 21 | INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 22 | CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 23 | ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF 24 | THE POSSIBILITY OF SUCH DAMAGE. 25 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: all clean distclean doc 2 | 3 | VPATH += 4 | 5 | CC = gcc 6 | LD = gcc 7 | AR = ar 8 | #----------------------------------------------------------------------------- 9 | SUBPROJS = libs3 libpdp bench 10 | .PHONY: $(SUBPROJS) 11 | 12 | all: bench 13 | 14 | libs3: 15 | @echo "Getting libs3 source, if needed" 16 | [ -d libs3 ] || git clone https://github.com/ceph/libs3.git 17 | @echo "Building libs3" 18 | $(MAKE) -C libs3 19 | 20 | libpdp: libs3 21 | @echo "Building libpdp" 22 | $(MAKE) -C libpdp all 23 | 24 | bench: libpdp 25 | @echo "Building the pdp_bench benchmarking utility" 26 | $(MAKE) -C tools pdp_bench 27 | 28 | doc: doxyfile 29 | doxygen doxyfile 30 | 31 | clean: 32 | [ -d tools ] && $(MAKE) -C tools clean 33 | [ -d libpdp ] && $(MAKE) -C libpdp clean 34 | [ -d libs3 ] && $(MAKE) -C libs3 clean 35 | 36 | distclean: clean 37 | [ -d tools ] && $(MAKE) -C tools distclean 38 | [ -d libpdp ] && $(MAKE) -C libpdp distclean 39 | [ -d libs3 ] && $(MAKE) -C libs3 distclean 40 | rm -rf doc/html 41 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # libpdp, a library for Proofs of Data Possession 2 | A provable data possession scheme allows a client that has stored data at 3 | an untrusted server to verify that the server possesses the original data 4 | without retrieving it or storing a copy himself. It accomplishes this by 5 | generating probabilistic proofs using an interactive protocol with the 6 | remote server. 7 | 8 | libpdp currently implements the following cryptographic proof schemes: 9 | 10 | * MACPDP -- The simple MAC-based PDP scheme. 11 | * A-PDP -- An implementation of the Ateniese, Burns, Curtmola, Herring, 12 | Kissner, Peterson and Song provable data possession (PDP) scheme. 13 | * CPOR -- An implementation of the Shacham and Waters POR scheme. 14 | * SEPDP -- An implementation of the Ateniese, Pietro, Mancini and Tsudik 15 | PDP scheme. 16 | 17 | All modules are listed in the Module Documentation 18 | 19 | -------------------------------------------------------------------------------- /doc/.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gondree/libpdp/391648d2b11a4216d106ac22887b82adf5d8288d/doc/.gitignore -------------------------------------------------------------------------------- /libpdp/Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: all clean distclean 2 | 3 | VPATH += src src/apdp src/mrpdp src/cpor src/macpdp src/sepdp 4 | 5 | CC = gcc 6 | LD = gcc 7 | AR = ar 8 | #----------------------------------------------------------------------------- 9 | INCLUDES = -Iinc -Isrc 10 | INCLUDES += -Isrc/apdp -Isrc/mrpdp -Isrc/cpor -Isrc/macpdp -Isrc/sepdp 11 | INCLUDES += -I../libs3/inc 12 | 13 | CFLAGS = -Wall -g $(INCLUDES) 14 | CFLAGS += -D_S3_SUPPORT 15 | CFLAGS += -D_THREAD_SUPPORT 16 | CFLAGS += -D_TIMING_DATA 17 | # CFLAGS += -D_PDP_DEBUG 18 | 19 | OBJS = pdp_generic.o pdp_misc.o pdp_key.o pdp_file.o pdp_s3.o 20 | OBJS += macpdp.o macpdp_key.o macpdp_file.o macpdp_s3.o 21 | OBJS += apdp.o apdp_key.o apdp_file.o apdp_s3.o apdp_serialize.o 22 | OBJS += mrpdp.o mrpdp_key.o mrpdp_file.o mrpdp_s3.o mrpdp_serialize.o 23 | OBJS += cpor.o cpor_key.o cpor_file.o cpor_s3.o cpor_serialize.o 24 | OBJS += sepdp.o sepdp_key.o sepdp_file.o sepdp_s3.o sepdp_serialize.o 25 | 26 | all: libpdp.a 27 | 28 | clean: 29 | rm -rf *.o *.dSYM 30 | 31 | distclean: clean 32 | rm -rf libpdp.a 33 | 34 | %.o: %.c 35 | $(CC) -c $(CFLAGS) $< -o $@ 36 | 37 | libpdp.a: $(OBJS) 38 | $(AR) rvs $@ $^ 39 | 40 | -------------------------------------------------------------------------------- /libpdp/inc/pdp.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for the libpdp library. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | * 12 | * @defgroup PDP libpdp Interfaces 13 | * \brief Generic interfaces for all PDP schemes. 14 | **/ 15 | #ifndef __PDP_H__ 16 | #define __PDP_H__ 17 | 18 | #include 19 | #include 20 | #include 21 | 22 | #define LIBPDP_NAME "libpdp" ///< library name 23 | #define LIBPDP_VERS "v1.0" ///< library version 24 | 25 | 26 | /** 27 | * \brief Macro used on a ctx pointer to set the algo used 28 | * by the pdp ctx instance. 29 | **/ 30 | #define PDP_SELECT(ctx, x) \ 31 | do { (ctx)->algo = x; } while(0) 32 | 33 | 34 | /* 35 | * defines the PDP algorithms to use with the PDP_SELECT macro. 36 | */ 37 | #define PDP_MACPDP 0x00 ///< Select the MAC-PDP algo 38 | #define PDP_APDP 0x01 ///< Select the A-PDP algo 39 | #define PDP_SEPDP 0x02 ///< Select the SE-PDP algo 40 | #define PDP_CPOR 0x03 ///< Select the CPOR algo 41 | #define PDP_MRPDP 0x04 ///< Select the MRPDP algo 42 | 43 | 44 | /* 45 | * options to use for ctx, to adjust PDP method behavior. 46 | */ 47 | #define PDP_OPT_DEFAULT 0x00 48 | #define PDP_OPT_THREADED 0x01 ///< Use threaded routines, when possible 49 | #define PDP_OPT_USE_S3 0x02 ///< Use Amazon S3 as storage backend 50 | #define PDP_USE_ECC 0x04 ///< Use an ECC tranform before tagging 51 | #define PDP_OPT_HTTPS 0x08 ///< Force using HTTPS 52 | #define PDP_PW_NOINPUT 0x10 ///< Read password from environment (not safe) 53 | 54 | /* 55 | * function prototypes 56 | */ 57 | int pdp_ctx_init(pdp_ctx_t *ctx); 58 | int pdp_ctx_create(pdp_ctx_t *ctx, const char* filename, const char *output); 59 | int pdp_ctx_free(pdp_ctx_t *ctx); 60 | int pdp_key_open(pdp_ctx_t *ctx, pdp_key_t *key, pdp_key_t *pk, 61 | const char *path); 62 | int pdp_key_store(const pdp_ctx_t *ctx, const pdp_key_t *key, const char *path); 63 | int pdp_key_gen(pdp_ctx_t *ctx, pdp_key_t *key, pdp_key_t *pk); 64 | int pdp_key_free(const pdp_ctx_t *ctx, pdp_key_t *key); 65 | int pdp_file_preprocess(pdp_ctx_t *ctx); 66 | int pdp_tags_gen(pdp_ctx_t *ctx, pdp_key_t *key, pdp_tag_t *tag); 67 | int pdp_tags_free(const pdp_ctx_t *ctx, pdp_tag_t *tag); 68 | int pdp_store(const pdp_ctx_t *ctx, const pdp_key_t *key, const pdp_tag_t *tag); 69 | int pdp_challenge_gen(const pdp_ctx_t *ctx, pdp_key_t *key, pdp_challenge_t *c); 70 | int pdp_challenge_for_prover(const pdp_ctx_t *ctx, const pdp_challenge_t *c, 71 | pdp_challenge_t *vchal); 72 | int pdp_challenge_free(const pdp_ctx_t *ctx, pdp_challenge_t *c); 73 | int pdp_proof_gen(const pdp_ctx_t *ctx, const pdp_key_t *key, 74 | const pdp_challenge_t *c, pdp_proof_t *proof); 75 | int pdp_proof_verify(const pdp_ctx_t *ctx, const pdp_key_t *key, 76 | const pdp_challenge_t *c, const pdp_proof_t *proof); 77 | int pdp_proof_free(const pdp_ctx_t *ctx, pdp_proof_t *proof); 78 | 79 | #endif 80 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/apdp.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for the MAC-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | * 12 | * @defgroup APDP APDP 13 | * \brief An implementation of the PDP scheme of Ateniese et al (2007). 14 | **/ 15 | #ifndef __A_PDP_H__ 16 | #define __A_PDP_H__ 17 | 18 | #include 19 | #include 20 | #include 21 | 22 | /// number of challenges, sets assurance of proof. 23 | /// 460 blocks gives you 99% chance of detecting an error, 300 blocks gives 24 | /// you 95% chance 25 | #define APDP_MAGIC_CHALLENGE_NUM 460 26 | 27 | /// default block size to use (16 KB) 28 | #define APDP_DEFAULT_BLOCK_SIZE_BYTES 16384 29 | 30 | /// default public exponent 'e' used in RSA 31 | #define APDP_DEFAULT_RSA_PUB_EXP RSA_F4 32 | 33 | /// number of bits of RSA modulus 34 | #define APDP_DEFAULT_RSA_KEY_SIZE 1024 35 | 36 | /// default prp key length, default is 16 bytes (128-bit AES) 37 | #define APDP_DEFAULT_PRP_KEY_SIZE 16 38 | 39 | /// length of key for PRF, default is 20 bytes 40 | #define APDP_DEFAULT_PRF_KEY_SIZE SHA_DIGEST_LENGTH 41 | 42 | 43 | /// @def APDP_NO_SAFE_PRIMES 44 | /// \brief Don't use safe primes 45 | /// Using safe primes is required if you want PDP to be provably secure. 46 | /// Key generation is slower as a side effect 47 | /// 48 | /// @def APDP_USE_E_PDP 49 | /// \brief Use efficient variant 50 | /// If USE_E_PDP is defined, the protocol is more efficient but offers 51 | /// weaker guarantees of possesion; only a possesion of the sum of file 52 | /// blocks. In general, however, this is practically secure as long as 53 | /// the number of sampled blocks is high. 54 | #define APDP_NO_SAFE_PRIMES 0x01 55 | #define APDP_USE_E_PDP 0x02 56 | 57 | 58 | /* 59 | * function prototypes 60 | */ 61 | int apdp_ctx_init(pdp_ctx_t *ctx); 62 | int apdp_ctx_create(pdp_ctx_t *ctx); 63 | int apdp_ctx_free(pdp_ctx_t *ctx); 64 | int apdp_key_gen(const pdp_ctx_t *ctx, pdp_key_t *k, pdp_key_t *pub); 65 | int apdp_key_store(const pdp_ctx_t *ctx, const pdp_key_t *k, const char *path); 66 | int apdp_key_open(const pdp_ctx_t *ctx, pdp_key_t *k, pdp_key_t *pub, 67 | const char* path); 68 | int apdp_key_free(const pdp_ctx_t *ctx, pdp_key_t *k); 69 | int apdp_tags_gen(pdp_ctx_t *ctx, pdp_apdp_key_t *k, pdp_apdp_tagdata_t *t); 70 | int apdp_tags_free(const pdp_ctx_t *ctx, pdp_apdp_tagdata_t *t); 71 | int apdp_store(const pdp_ctx_t *ctx, const pdp_apdp_tagdata_t* tag); 72 | int apdp_challenge_gen(const pdp_ctx_t *ctx, const pdp_apdp_key_t *key, 73 | pdp_apdp_challenge_t *chal); 74 | int apdp_challenge_free(const pdp_ctx_t *ctx, pdp_apdp_challenge_t *c); 75 | int apdp_proof_gen(const pdp_ctx_t *ctx, const pdp_apdp_key_t *key, 76 | const pdp_apdp_challenge_t *chal, pdp_apdp_proof_t *proof); 77 | int apdp_proof_verify(const pdp_ctx_t *ctx, const pdp_apdp_key_t *key, 78 | const pdp_apdp_challenge_t *chal, const pdp_apdp_proof_t *proof); 79 | int apdp_proof_free(const pdp_ctx_t *ctx, pdp_apdp_proof_t *proof); 80 | 81 | #endif 82 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/apdp_types.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Type interfaces for the A-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup APDP 13 | * @{ 14 | */ 15 | #ifndef __A_PDP_TYPES_H__ 16 | #define __A_PDP_TYPES_H__ 17 | 18 | #include 19 | #include 20 | 21 | 22 | /// Holds parameter data for A-PDP 23 | typedef struct { 24 | unsigned int opts; ///< some processing options 25 | unsigned int block_size; ///< block size, in bytes 26 | unsigned int num_blocks; ///< total number of blocks in file 27 | unsigned int num_challenge_blocks; ///< num of blocks to challenge 28 | size_t prf_key_size; ///< size of (HMAC) key, in bytes 29 | size_t prp_key_size; ///< size of (AES) key, in bytes 30 | size_t rsa_key_size; ///< size of (RSA) modulus, in bytes 31 | size_t prf_w_size; ///< max size of PRF, gen_prf_w 32 | size_t prf_f_size; ///< max size of PRF, gen_prf_f 33 | } pdp_apdp_ctx_t; 34 | 35 | 36 | /// Holds key data for A-PDP 37 | typedef struct { 38 | RSA *rsa; ///< RSA key pair 39 | unsigned char *v; ///< PRF key 40 | BIGNUM *g; ///< APDP generator 41 | } pdp_apdp_key_t; 42 | 43 | 44 | /// Holds data for a single tag 45 | typedef struct { 46 | unsigned int index; ///< The index of the block, i 47 | size_t index_prf_size; ///< The size of the prf output 48 | BIGNUM *Tim; ///< The tag of the message block i 49 | ///< \f$ T_{i,m} = (h(W_i) * g^m)^d \f$ 50 | unsigned char *index_prf; ///< The PRF output of the index 51 | /// \f$ W_i = w_v(i) \f$ 52 | } pdp_apdp_tag_t; 53 | 54 | 55 | /// Holds data for A-PDP tags 56 | typedef struct { 57 | pdp_apdp_tag_t **tags; ///< array of pointers to tags 58 | unsigned int tags_size; ///< byte length of array 59 | unsigned int tags_num; ///< number of items in array 60 | } pdp_apdp_tagdata_t; 61 | 62 | 63 | /// Holds challenge data for A-PDP 64 | typedef struct { 65 | unsigned int c; ///< Number of blocks to sample 66 | BIGNUM *g_s; ///< Random secret base \f$ g_s = g^s \f$ 67 | BIGNUM *s; ///< Random secret 68 | unsigned char *k1; ///< PRP key 69 | unsigned char *k2; ///< PRF key 70 | } pdp_apdp_challenge_t; 71 | 72 | 73 | /// Holds proof data for A-PDP 74 | typedef struct { 75 | BIGNUM *T; ///< The product of tags, T 76 | BIGNUM *rho_temp; ///< A running tally of rho 77 | unsigned char *rho; ///< Final \f$ \rho \f$ 78 | size_t rho_size; ///< size of the final rho 79 | } pdp_apdp_proof_t; 80 | 81 | 82 | #endif 83 | /** @} */ 84 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/cpor.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for the CPOR module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | * 12 | * @defgroup CPOR CPOR 13 | * \brief An implementation of the PDP scheme of Shacham and Waters (2008). 14 | **/ 15 | #ifndef __CPOR_PDP_H__ 16 | #define __CPOR_PDP_H__ 17 | 18 | #include 19 | 20 | /// the default block size to use (4 KB) 21 | #define CPOR_DEFAULT_BLOCK_SIZE_BYTES 4096 22 | 23 | /// The security parameter lambda 24 | #define CPOR_DEFAULT_LAMBDA 80 25 | 26 | /// Size (in bytes) of an HMAC-SHA1 27 | #define CPOR_DEFAULT_PRF_KEY_BYTES 20 28 | 29 | /// Size (in bytes) of the user's AES encryption key 30 | #define CPOR_DEFAULT_ENC_KEY_BYTES 32 31 | 32 | /// Size (in bytes) of the user's MAC key 33 | #define CPOR_DEFAULT_MAC_KEY_BYTES 20 34 | 35 | 36 | /* 37 | * function prototypes 38 | */ 39 | int cpor_ctx_init(pdp_ctx_t *ctx); 40 | int cpor_ctx_create(pdp_ctx_t *ctx); 41 | int cpor_ctx_free(pdp_ctx_t *ctx); 42 | int cpor_key_gen(const pdp_ctx_t *ctx, pdp_key_t *k, pdp_key_t *pub); 43 | int cpor_key_open(const pdp_ctx_t *ctx, pdp_key_t *k, pdp_key_t *pub, 44 | const char* path); 45 | int cpor_key_store(const pdp_ctx_t *ctx, const pdp_key_t *k, const char *path); 46 | int cpor_key_free(const pdp_ctx_t *ctx, pdp_key_t *k); 47 | int cpor_tags_gen(pdp_ctx_t *ctx, pdp_cpor_key_t *k, pdp_cpor_tagdata_t *t); 48 | int cpor_tags_free(const pdp_ctx_t *ctx, pdp_cpor_tagdata_t *t); 49 | int cpor_store(const pdp_ctx_t *ctx, const pdp_cpor_key_t *key, 50 | const pdp_cpor_tagdata_t* tag); 51 | int cpor_challenge_gen(const pdp_ctx_t *ctx, const pdp_cpor_key_t *key, 52 | pdp_cpor_challenge_t *chal); 53 | int cpor_challenge_free(const pdp_ctx_t *ctx, pdp_cpor_challenge_t *c); 54 | int cpor_proof_gen(const pdp_ctx_t *ctx, const pdp_cpor_key_t *key, 55 | const pdp_cpor_challenge_t *chal, pdp_cpor_proof_t *proof); 56 | int cpor_proof_verify(const pdp_ctx_t *ctx, const pdp_cpor_key_t *key, 57 | const pdp_cpor_challenge_t *c, const pdp_cpor_proof_t *proof); 58 | int cpor_proof_free(const pdp_ctx_t *ctx, pdp_cpor_proof_t *proof); 59 | 60 | #endif 61 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/cpor_types.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Type interfaces for the MAC-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup CPOR 13 | * @{ 14 | */ 15 | #ifndef __CPOR_PDP_TYPES_H__ 16 | #define __CPOR_PDP_TYPES_H__ 17 | 18 | /// Holds parameter data for CPOR 19 | typedef struct { 20 | unsigned int block_size; ///< block size (bytes) 21 | unsigned int sector_size; ///< sector size (bytes) 22 | unsigned int num_sectors; ///< number of sectors per block 23 | unsigned int num_blocks; ///< total number of blocks in file 24 | unsigned int num_challenge_blocks; ///< num of blocks to challenge 25 | unsigned int lambda; ///< security parameter 26 | unsigned int Zp_bits; ///< size (in bits) of the prime for Z_p 27 | size_t prf_key_size; ///< size of key (bytes) 28 | size_t enc_key_size; ///< size of ENC key (bytes) 29 | size_t mac_key_size; ///< size of MAC key (bytes) 30 | } pdp_cpor_ctx_t; 31 | 32 | 33 | /// Holds key data for CPOR 34 | typedef struct { 35 | unsigned char *k_enc; ///< The user's secret encryption key 36 | size_t k_enc_size; ///< Encryption size in bytes 37 | unsigned char *k_mac; ///< The user's secret MAC key 38 | size_t k_mac_size; ///< MAC key size in bytes 39 | BIGNUM *Zp; ///< The prime p that defines the field Zp 40 | } pdp_cpor_key_t; 41 | 42 | 43 | /// Holds data for a single tag 44 | typedef struct { 45 | unsigned int index; ///< The index for the authenticator, i 46 | BIGNUM *sigma; ///< The resulting authenticator, sigma_i 47 | } pdp_cpor_tag_t; 48 | 49 | 50 | /// Holds tag data for CPOR 51 | typedef struct { 52 | unsigned char *k_prf; ///< The randomly generated PRF key for the file 53 | pdp_cpor_tag_t **tags; ///< Array of tags 54 | size_t tags_num; ///< Number of elements in the array 55 | size_t tags_size; ///< Size of array 56 | BIGNUM **alpha; ///< Array of per-file, per-sector secrets 57 | size_t alpha_size; ///< Size of alpha 58 | } pdp_cpor_tagdata_t; 59 | 60 | 61 | /// Holds challenge data for CPOR 62 | typedef struct { 63 | unsigned int ell; ///< The number of elements to be tested 64 | unsigned int *I; ///< An array of ell indicies to be tested 65 | unsigned int I_size; ///< Size of array, I 66 | unsigned int nu_size; ///< Size of array, nu 67 | BIGNUM **nu; ///< An array of ell random elements 68 | } pdp_cpor_challenge_t; 69 | 70 | 71 | /// Holds proof data for CPOR 72 | typedef struct { 73 | BIGNUM *sigma; 74 | BIGNUM **mu; 75 | unsigned int mu_size; ///< Size of array, mu 76 | } pdp_cpor_proof_t; 77 | 78 | 79 | #endif 80 | /** @} */ 81 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/macpdp.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for the MAC-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | * 12 | * @defgroup MACPDP MACPDP 13 | * \brief An implementation of the MAC-based PDP scheme. 14 | **/ 15 | #ifndef __MAC_PDP_H__ 16 | #define __MAC_PDP_H__ 17 | 18 | #include 19 | #include 20 | 21 | /// number of challenges, sets assurance of proof 22 | #define MACPDP_MAGIC_CHALLENGE_NUM 460 23 | 24 | /// default PRF key size in bytes, (same as that used in HMAC-SHA1) 25 | #define MACPDP_DEFAULT_PRF_KEY_BYTES SHA_DIGEST_LENGTH 26 | 27 | /// the default block size to use (4 KB) 28 | #define MACPDP_DEFAULT_BLOCK_SIZE_BYTES 4096 29 | 30 | /// tag size is same as MAC digest size 31 | #define MACPDP_DEFAULT_TAG_SIZE_BYTES SHA_DIGEST_LENGTH 32 | 33 | 34 | /* 35 | * function prototypes 36 | */ 37 | int macpdp_ctx_init(pdp_ctx_t *ctx); 38 | int macpdp_ctx_create(pdp_ctx_t *ctx); 39 | int macpdp_ctx_free(pdp_ctx_t *ctx); 40 | int macpdp_key_gen(const pdp_ctx_t *ctx, pdp_key_t *k); 41 | int macpdp_key_free(pdp_key_t *k); 42 | int macpdp_tags_gen(pdp_ctx_t *ctx, pdp_macpdp_key_t *k, 43 | pdp_macpdp_tagdata_t *t); 44 | int macpdp_tags_free(const pdp_ctx_t *ctx, pdp_macpdp_tagdata_t *t); 45 | int macpdp_store(const pdp_ctx_t *ctx, const pdp_macpdp_tagdata_t* tag); 46 | int macpdp_challenge_gen(const pdp_ctx_t *ctx, pdp_macpdp_challenge_t *c); 47 | int macpdp_challenge_free(const pdp_ctx_t *ctx, pdp_macpdp_challenge_t *c); 48 | int macpdp_proof_gen(const pdp_ctx_t *ctx, const pdp_macpdp_challenge_t *c, 49 | pdp_macpdp_proof_t *proof); 50 | int macpdp_proof_verify(const pdp_ctx_t *ctx, const pdp_macpdp_key_t *key, 51 | const pdp_macpdp_challenge_t *c, const pdp_macpdp_proof_t *proof); 52 | int macpdp_proof_free(const pdp_ctx_t *ctx, pdp_macpdp_proof_t *proof); 53 | 54 | #endif 55 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/macpdp_types.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Type interfaces for the MAC-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MACPDP 13 | * @{ 14 | */ 15 | #ifndef __MAC_PDP_TYPES_H__ 16 | #define __MAC_PDP_TYPES_H__ 17 | 18 | /// Holds parameter data for MAC-PDP 19 | typedef struct { 20 | unsigned int block_size; ///< block size (bytes) 21 | unsigned int num_blocks; ///< total number of blocks in file 22 | unsigned int num_challenge_blocks; ///< num of blocks to challenge 23 | unsigned int tag_size; ///< size of an individual tag (bytes) 24 | size_t prf_key_size; ///< size of key (bytes) 25 | } pdp_macpdp_ctx_t; 26 | 27 | 28 | /// Holds key data for MAC-PDP 29 | typedef struct { 30 | unsigned char *prf_key; 31 | size_t prf_key_size; 32 | } pdp_macpdp_key_t; 33 | 34 | 35 | /// Holds tag data for MAC-PDP 36 | typedef struct { 37 | unsigned char *tags; 38 | size_t tags_size; 39 | } pdp_macpdp_tagdata_t; 40 | 41 | 42 | /// Holds challenge data for MAC-PDP 43 | typedef struct { 44 | unsigned int ell; ///< ell, the number of elements to be tested 45 | unsigned int *I; ///< an array of ell indicies (the blocks to test) 46 | } pdp_macpdp_challenge_t; 47 | 48 | 49 | /// Holds proof data for MAC-PDP 50 | typedef struct { 51 | unsigned char *B; ///< array of ell blocks 52 | unsigned char *T; ///< array of ell tags 53 | unsigned int ell; ///< ell, the number of elements to be tested 54 | } pdp_macpdp_proof_t; 55 | 56 | 57 | #endif 58 | /** @} */ 59 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/mrpdp.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for the MAC-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | * 12 | * @defgroup MRPDP MRPDP 13 | * \brief An implementation of the PDP scheme of Ateniese et al (2007). 14 | **/ 15 | #ifndef __MR_PDP_H__ 16 | #define __MR_PDP_H__ 17 | 18 | #include 19 | #include 20 | #include 21 | 22 | /// number of challenges, sets assurance of proof. 23 | /// 460 blocks gives you 99% chance of detecting an error, 300 blocks gives 24 | /// you 95% chance 25 | #define MRPDP_MAGIC_CHALLENGE_NUM 460 26 | 27 | /// default block size to use (16 KB) 28 | #define MRPDP_DEFAULT_BLOCK_SIZE_BYTES 16384 29 | 30 | /// default public exponent 'e' used in RSA 31 | #define MRPDP_DEFAULT_RSA_PUB_EXP RSA_F4 32 | 33 | /// number of bits of RSA modulus 34 | #define MRPDP_DEFAULT_RSA_KEY_SIZE 1024 35 | 36 | /// default prp key length, default is 16 bytes (128-bit AES) 37 | #define MRPDP_DEFAULT_PRP_KEY_SIZE 16 38 | 39 | /// length of key for PRF, default is 20 bytes 40 | #define MRPDP_DEFAULT_PRF_KEY_SIZE SHA_DIGEST_LENGTH 41 | 42 | 43 | /// @def MRPDP_NO_SAFE_PRIMES 44 | /// \brief Don't use safe primes 45 | /// Using safe primes is required if you want PDP to be provably secure. 46 | /// Key generation is slower as a side effect 47 | /// 48 | /// @def MRPDP_USE_E_PDP 49 | /// \brief Use efficient variant 50 | /// If USE_E_PDP is defined, the protocol is more efficient but offers 51 | /// weaker guarantees of possesion; only a possesion of the sum of file 52 | /// blocks. In general, however, this is practically secure as long as 53 | /// the number of sampled blocks is high. 54 | #define MRPDP_NO_SAFE_PRIMES 0x01 55 | #define MRPDP_USE_E_PDP 0x02 56 | 57 | 58 | /* 59 | * function prototypes 60 | */ 61 | int mrpdp_ctx_init(pdp_ctx_t *ctx); 62 | int mrpdp_ctx_create(pdp_ctx_t *ctx); 63 | int mrpdp_ctx_free(pdp_ctx_t *ctx); 64 | int mrpdp_key_gen(const pdp_ctx_t *ctx, pdp_key_t *k, pdp_key_t *pub); 65 | int mrpdp_key_store(const pdp_ctx_t *ctx, const pdp_key_t *k, const char *path); 66 | int mrpdp_key_open(const pdp_ctx_t *ctx, pdp_key_t *k, pdp_key_t *pub, 67 | const char* path); 68 | int mrpdp_key_free(const pdp_ctx_t *ctx, pdp_key_t *k); 69 | int mrpdp_tags_gen(pdp_ctx_t *ctx, pdp_mrpdp_key_t *k, pdp_mrpdp_tagdata_t *t); 70 | int mrpdp_tags_free(const pdp_ctx_t *ctx, pdp_mrpdp_tagdata_t *t); 71 | int mrpdp_store(const pdp_ctx_t *ctx, const pdp_mrpdp_tagdata_t* tag); 72 | int mrpdp_challenge_gen(const pdp_ctx_t *ctx, const pdp_mrpdp_key_t *key, 73 | pdp_mrpdp_challenge_t *chal); 74 | int mrpdp_challenge_free(const pdp_ctx_t *ctx, pdp_mrpdp_challenge_t *c); 75 | int mrpdp_proof_gen(const pdp_ctx_t *ctx, const pdp_mrpdp_key_t *key, 76 | const pdp_mrpdp_challenge_t *chal, pdp_mrpdp_proof_t *proof); 77 | int mrpdp_proof_verify(const pdp_ctx_t *ctx, const pdp_mrpdp_key_t *key, 78 | const pdp_mrpdp_challenge_t *chal, const pdp_mrpdp_proof_t *proof); 79 | int mrpdp_proof_free(const pdp_ctx_t *ctx, pdp_mrpdp_proof_t *proof); 80 | 81 | #endif 82 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/mrpdp_types.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Type interfaces for the MR-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MRPDP 13 | * @{ 14 | */ 15 | #ifndef __MR_PDP_TYPES_H__ 16 | #define __MR_PDP_TYPES_H__ 17 | 18 | #include 19 | #include 20 | 21 | 22 | /// Holds parameter data for MR-PDP 23 | typedef struct { 24 | unsigned int opts; ///< some processing options 25 | unsigned int block_size; ///< block size, in bytes 26 | unsigned int num_blocks; ///< total number of blocks in file 27 | unsigned int num_challenge_blocks; ///< num of blocks to challenge 28 | unsigned int numReplicas; ///< number of Replicas to be generated 29 | size_t prf_key_size; ///< size of (HMAC) key, in bytes 30 | size_t prp_key_size; ///< size of (AES) key, in bytes 31 | size_t rsa_key_size; ///< size of (RSA) modulus, in bytes 32 | size_t prf_w_size; ///< max size of PRF, gen_prf_w 33 | size_t prf_f_size; ///< max size of PRF, gen_prf_f 34 | } pdp_mrpdp_ctx_t; 35 | 36 | 37 | /// Holds key data for MR-PDP 38 | /// Key gen and block tagging function the same as A-PDP 39 | typedef struct { 40 | RSA *rsa; ///< RSA key pair 41 | unsigned char *v; ///< PRF key 42 | BIGNUM *g; ///< APDP generator 43 | } pdp_mrpdp_key_t; 44 | 45 | 46 | /// Holds data for a single tag 47 | typedef struct { 48 | unsigned int index; ///< The index of the block, i 49 | size_t index_prf_size; ///< The size of the prf output 50 | BIGNUM *Tim; ///< The tag of the message block i 51 | ///< \f$ T_{i,m} = (h(W_i) * g^m)^d \f$ 52 | unsigned char *index_prf; ///< The PRF output of the index 53 | /// \f$ W_i = w_v(i) \f$ 54 | } pdp_mrpdp_tag_t; 55 | 56 | 57 | /// Holds data for MR-PDP tags 58 | typedef struct { 59 | pdp_mrpdp_tag_t **tags; ///< array of pointers to tags 60 | unsigned int tags_size; ///< byte length of array 61 | unsigned int tags_num; ///< number of items in array 62 | } pdp_mrpdp_tagdata_t; 63 | 64 | 65 | /// Holds challenge data for MR-PDP 66 | /// ADD: Code to specify replica that needs to be challenged 67 | typedef struct { 68 | unsigned int c; ///< Number of blocks to sample 69 | BIGNUM *g_s; ///< Random secret base \f$ g_s = g^s \f$ 70 | BIGNUM *s; ///< Random secret 71 | unsigned char *k1; ///< PRP key 72 | unsigned char *k2; ///< PRF key 73 | } pdp_mrpdp_challenge_t; 74 | 75 | 76 | /// Holds proof data for MR-PDP 77 | typedef struct { 78 | BIGNUM *T; ///< The product of tags, T 79 | BIGNUM *rho_temp; ///< A running tally of rho 80 | unsigned char *rho; ///< Final \f$ \rho \f$ 81 | size_t rho_size; ///< size of the final rho 82 | } pdp_mrpdp_proof_t; 83 | 84 | 85 | #endif 86 | /** @} */ 87 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/sepdp.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for the MAC-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | * 12 | * @defgroup SEPDP SEPDP 13 | * \brief An implementation of the PDP scheme of Ateniese et al (2008). 14 | **/ 15 | #ifndef __SEPDP_H__ 16 | #define __SEPDP_H__ 17 | 18 | #include 19 | #include 20 | 21 | /// number of challenge blocks, sets assurance of proof 22 | /// 512 blocks gives you 99.4% chance of detecting an error 23 | #define SEPDP_MAGIC_CHALLENGE_BLOCKS 512 24 | 25 | /// the default block size to use (default is 4 KB) 26 | #define SEPDP_DEFAULT_BLOCK_SIZE_BYTES 4096 27 | 28 | /// length of a key for Authenticated-Encryption scheme (default is 16 bytes) 29 | #define SEPDP_DEFAULT_AE_KEY_SIZE 16 30 | 31 | /// length of a key for PRP (default is a 16 byte AES key) 32 | #define SEPDP_DEFAULT_PRP_KEY_SIZE 16 33 | 34 | /// length of key for PRF (default is a 20 byte HMAC-SHA1 key) 35 | #define SEPDP_DEFAULT_PRF_KEY_SIZE SHA_DIGEST_LENGTH 36 | 37 | /// the number of years over which challenges are issues (default 1) 38 | #define SEPDP_DEFAULT_YEARS 1 39 | 40 | /// the number of minutes between each challenge (default, once a month) 41 | #define SEPDP_DEFAULT_MIN 43200 42 | 43 | 44 | /* 45 | * function prototypes 46 | */ 47 | int sepdp_ctx_init(pdp_ctx_t *ctx); 48 | int sepdp_ctx_create(pdp_ctx_t *ctx); 49 | int sepdp_ctx_free(pdp_ctx_t *ctx); 50 | int sepdp_key_gen(const pdp_ctx_t *ctx, pdp_key_t *k); 51 | int sepdp_key_store(const pdp_ctx_t *ctx, const pdp_key_t *k, const char *path); 52 | int sepdp_key_open(const pdp_ctx_t *ctx, pdp_key_t *k, pdp_key_t *pub, 53 | const char* path); 54 | int sepdp_key_free(const pdp_ctx_t *ctx, pdp_key_t *k); 55 | int sepdp_tags_gen(pdp_ctx_t *ctx, pdp_sepdp_key_t *k, pdp_sepdp_tagdata_t *t); 56 | int sepdp_tags_free(const pdp_ctx_t *ctx, pdp_sepdp_tagdata_t *t); 57 | int sepdp_store(const pdp_ctx_t *ctx, const pdp_sepdp_tagdata_t* tag); 58 | int sepdp_challenge_gen(const pdp_ctx_t *ctx, const pdp_sepdp_key_t *key, 59 | pdp_sepdp_challenge_t *c); 60 | int sepdp_challenge_free(const pdp_ctx_t *ctx, pdp_sepdp_challenge_t *c); 61 | int sepdp_proof_gen(const pdp_ctx_t *ctx, const pdp_sepdp_key_t *key, 62 | const pdp_sepdp_challenge_t *chal, pdp_sepdp_proof_t *proof); 63 | int sepdp_proof_verify(const pdp_ctx_t *ctx, const pdp_sepdp_key_t *key, 64 | const pdp_sepdp_challenge_t *chal, const pdp_sepdp_proof_t *proof); 65 | int sepdp_proof_free(const pdp_ctx_t *ctx, pdp_sepdp_proof_t *proof); 66 | 67 | #endif 68 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/sepdp_types.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Type interfaces for the MAC-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup SEPDP 13 | * @{ 14 | */ 15 | #ifndef __SEPDP_TYPES_H__ 16 | #define __SEPDP_TYPES_H__ 17 | 18 | /// Holds parameter data for SEPDP 19 | typedef struct { 20 | unsigned int block_size; ///< block size (bytes) 21 | unsigned int num_blocks; ///< total number of blocks in file 22 | unsigned int year; ///< period for audit material (years) 23 | unsigned int min; ///< audit regularity (minutes) 24 | size_t prf_key_size; ///< size of key (bytes) 25 | size_t prp_key_size; ///< size of key (bytes) 26 | size_t ae_key_size; ///< size of key (bytes) 27 | unsigned int num_challenge_blocks; ///< num of blocks to challenge 28 | unsigned int num_chal_created; ///< total number of challenged created 29 | unsigned int num_chal_used; ///< total number of challenges made 30 | } pdp_sepdp_ctx_t; 31 | 32 | 33 | /// Holds key data for SEPDP 34 | typedef struct { 35 | unsigned char *W; 36 | size_t W_size; 37 | unsigned char *Z; 38 | size_t Z_size; 39 | unsigned char *K; 40 | size_t K_size; 41 | } pdp_sepdp_key_t; 42 | 43 | 44 | /// Holds data for a single token 45 | typedef struct { 46 | size_t index; ///< index for token 47 | size_t iv_size; ///< length of the IV 48 | unsigned char *iv; ///< the IV used for the AE-scheme 49 | size_t tok_size; ///< length of the token 50 | unsigned char *tok; ///< the token 51 | size_t mac_size; ///< length of the mac 52 | unsigned char *mac; ///< the mac of the token 53 | } pdp_sepdp_tag_t; 54 | 55 | 56 | /// Holds tag data for MAC-PDP 57 | typedef struct { 58 | pdp_sepdp_tag_t **tokens; 59 | size_t tokens_num; 60 | size_t tokens_size; 61 | } pdp_sepdp_tagdata_t; 62 | 63 | 64 | /// Holds challenge data for MAC-PDP 65 | typedef struct { 66 | unsigned int i; 67 | unsigned char *ki; 68 | size_t ki_size; 69 | unsigned char *ci; 70 | size_t ci_size; 71 | } pdp_sepdp_challenge_t; 72 | 73 | 74 | /// Holds proof data for MAC-PDP 75 | typedef struct { 76 | size_t z_size; 77 | unsigned char *z; 78 | size_t iv_size; 79 | unsigned char *iv; 80 | size_t tok_size; 81 | unsigned char *tok; 82 | size_t mac_size; 83 | unsigned char *mac; 84 | } pdp_sepdp_proof_t; 85 | 86 | 87 | #endif 88 | /** @} */ 89 | -------------------------------------------------------------------------------- /libpdp/inc/pdp/types.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Types and structs for libpdp 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @date 2008-2013 7 | * @copyright BSD 2-Clause License 8 | * See http://opensource.org/licenses/BSD-2-Clause 9 | **/ 10 | #ifndef __PDP_TYPES_H__ 11 | #define __PDP_TYPES_H__ 12 | 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #ifdef _S3_SUPPORT 19 | #include 20 | #endif 21 | 22 | /// Context for the use of a PDP algorithm. 23 | typedef struct { 24 | unsigned short algo; ///< pdp algorithms being used 25 | unsigned short opts; ///< options 26 | unsigned short verbose; ///< print more messages to stdout 27 | unsigned int num_threads; ///< number of threads, if threading is used 28 | 29 | char *ifilepath; ///< path to file that contains input data 30 | size_t ifilepath_len; ///< ifilepath length 31 | 32 | char *filepath; ///< path to file that will be tagged 33 | size_t filepath_len; ///< filepath length 34 | off_t file_st_size; ///< size of file to tag (in bytes) 35 | 36 | char *ofilepath; ///< output file, to store data 37 | size_t ofilepath_len; ///< ofilepath length 38 | 39 | #ifdef _S3_SUPPORT 40 | char *s3_access_key; ///< S3 access id 41 | size_t s3_access_key_len; 42 | 43 | char *s3_secret_key; ///< S3 secret key 44 | size_t s3_secret_key_len; 45 | 46 | char *s3_hostname; ///< interface hostname:port 47 | size_t s3_hostname_len; 48 | 49 | char *s3_bucket_name; ///< S3 bucket name 50 | size_t s3_bucket_name_len; 51 | 52 | S3Protocol s3_protocol; ///< HTTP or HTTPS 53 | #endif 54 | 55 | /// algorithm-specific params 56 | union { 57 | pdp_macpdp_ctx_t *__macpdp; 58 | pdp_apdp_ctx_t *__apdp; 59 | pdp_mrpdp_ctx_t *__mrpdp; 60 | pdp_cpor_ctx_t *__cpor; 61 | pdp_sepdp_ctx_t *__sepdp; 62 | } __pdp_ctx_param_u; 63 | 64 | struct pdp_ctx_ops *ops; 65 | } pdp_ctx_t; 66 | 67 | 68 | /// Macro magic to access ctx->macpdp_param->x correctly 69 | #define macpdp_param __pdp_ctx_param_u.__macpdp 70 | /// Macro magic to access ctx->apdp_param->x correctly 71 | #define apdp_param __pdp_ctx_param_u.__apdp 72 | /// Macro magic to access ctx->mrpdp_param->x correctly 73 | #define mrpdp_param __pdp_ctx_param_u.__mrpdp 74 | /// Macro magic to access ctx->cpor_param->x correctly 75 | #define cpor_param __pdp_ctx_param_u.__cpor 76 | /// Macro magic to access ctx->sepdp_param->x correctly 77 | #define sepdp_param __pdp_ctx_param_u.__sepdp 78 | 79 | /// Holds secret data for PDP algorithms. 80 | typedef union { 81 | pdp_macpdp_key_t *macpdp; 82 | pdp_apdp_key_t *apdp; 83 | pdp_mrpdp_key_t *mrpdp; 84 | pdp_cpor_key_t *cpor; 85 | pdp_sepdp_key_t *sepdp; 86 | } pdp_key_t; 87 | 88 | /// Holds tag data for PDP algorithms. 89 | typedef union { 90 | pdp_macpdp_tagdata_t *macpdp; 91 | pdp_apdp_tagdata_t *apdp; 92 | pdp_mrpdp_tagdata_t *mrpdp; 93 | pdp_cpor_tagdata_t *cpor; 94 | pdp_sepdp_tagdata_t *sepdp; 95 | } pdp_tag_t; 96 | 97 | /// Holds challenge data for PDP algorithms. 98 | typedef union { 99 | pdp_macpdp_challenge_t *macpdp; 100 | pdp_apdp_challenge_t *apdp; 101 | pdp_mrpdp_challenge_t *mrpdp; 102 | pdp_cpor_challenge_t *cpor; 103 | pdp_sepdp_challenge_t *sepdp; 104 | } pdp_challenge_t; 105 | 106 | /// Holds proof data for PDP algorithms. 107 | typedef union { 108 | pdp_macpdp_proof_t *macpdp; 109 | pdp_apdp_proof_t *apdp; 110 | pdp_mrpdp_proof_t *mrpdp; 111 | pdp_cpor_proof_t *cpor; 112 | pdp_sepdp_proof_t *sepdp; 113 | } pdp_proof_t; 114 | 115 | /// struct to store pointers to stateful retrieval operations 116 | struct pdp_ctx_ops { 117 | int (*get_tag)(const pdp_ctx_t*, unsigned int, void *, unsigned int*); 118 | int (*get_block)(const pdp_ctx_t*, unsigned int, void *, unsigned int*); 119 | int (*sanitize)(const pdp_ctx_t*, const pdp_challenge_t*, pdp_challenge_t*); 120 | }; 121 | 122 | #endif 123 | -------------------------------------------------------------------------------- /libpdp/src/apdp/apdp_file.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the A-PDP module for flat-file storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup APDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include "pdp_storage.h" 23 | #include "apdp_storage.h" 24 | #include "apdp_misc.h" 25 | #include "pdp_misc.h" 26 | 27 | 28 | /** 29 | * @brief Write tag data out to a file. 30 | * @param[in] ctx pointer to a context structure. 31 | * @param[in] t pointer to tag data. 32 | * @return 0 on success, non-zero on error 33 | **/ 34 | int apdp_write_tags_to_file(const pdp_ctx_t *ctx, const pdp_apdp_tagdata_t* t) 35 | { 36 | unsigned char *buf = NULL; 37 | FILE *tagfile = NULL; 38 | unsigned int len = 0; 39 | size_t off = 0; 40 | int status = -1; 41 | 42 | if (!ctx || !t || !t->tags || !t->tags_num || 43 | !t->tags_size || !ctx->ofilepath) 44 | return -1; 45 | 46 | if ((access(ctx->ofilepath, F_OK) == 0) && (ctx->verbose)) { 47 | PDP_ERR("WARNING: overwriting [%s]", ctx->ofilepath); 48 | } 49 | if ((tagfile = fopen(ctx->ofilepath, "w")) == NULL){ 50 | PDP_ERR("%d", ferror(tagfile)); 51 | PDP_ERR("unable to create %s", ctx->ofilepath); 52 | goto cleanup; 53 | } 54 | 55 | if ((apdp_serialize_tags(ctx, t, &buf, &len)) != 0) goto cleanup; 56 | do { 57 | off += fwrite(buf + off, 1, len, tagfile); 58 | if (ferror(tagfile)) goto cleanup; 59 | 60 | } while (off < len); 61 | status = 0; 62 | 63 | cleanup: 64 | sfree(buf, len); 65 | if (tagfile) fclose(tagfile); 66 | return status; 67 | } 68 | 69 | 70 | /** 71 | * @brief Implements the interface of pdp_get_tag_noop 72 | * 73 | * To be used with MAC-PDP and a flat-file storage back-end. 74 | * 75 | * @todo use mmap to speed up access? 76 | * 77 | * @param[in] ctx pointer to a context structure. 78 | * @param[in] index the index of tag. 79 | * @param[out] buf pointer to an output buffer. 80 | * @param[in,out] len initially the size of buf, output bytes written. 81 | * @return 0 on success, non-zero on error 82 | **/ 83 | int apdp_get_tag_file(const pdp_ctx_t* ctx, unsigned int index, 84 | void *buf, unsigned int *len) 85 | { 86 | pdp_apdp_tag_t **tag_ptr = (pdp_apdp_tag_t **) buf; 87 | pdp_apdp_tag_t *tag = NULL; 88 | static FILE* tagfile = NULL; 89 | unsigned char *data = NULL; 90 | size_t data_len = 0; 91 | int status = -1; 92 | 93 | // If buf is NULL, it is a signal to re-set the stateful variables 94 | if (!buf && tagfile) { 95 | fclose(tagfile); 96 | tagfile = NULL; 97 | return 0; 98 | } 99 | 100 | if (!ctx || !tag_ptr) 101 | return -1; 102 | 103 | // if file is not open already, open it 104 | if (!tagfile && ((tagfile = fopen(ctx->ofilepath, "r")) == NULL)) { 105 | PDP_ERR("%d", ferror(tagfile)); 106 | PDP_ERR("unable to open %s", ctx->ofilepath); 107 | return -1; 108 | } 109 | 110 | // Allocate space for the tag 111 | if ((tag = apdp_tag_new()) == NULL) goto cleanup; 112 | if (*tag_ptr != NULL) { 113 | // there is already a tag here, so free it 114 | apdp_tag_free(*tag_ptr); 115 | } 116 | *tag_ptr = tag; 117 | 118 | data_len = apdp_serialized_tag_size(ctx); 119 | if ((data = malloc(data_len)) == NULL) goto cleanup; 120 | memset(data, 0, data_len); 121 | 122 | // We seek directly to the index of the i-th tag 123 | if (fseek(tagfile, index * data_len, SEEK_SET) < 0) goto cleanup; 124 | fread(data, 1, data_len, tagfile); 125 | if (ferror(tagfile)) goto cleanup; 126 | 127 | // de-serialize the data block 128 | if (apdp_deserialize_tag(ctx, tag, data, data_len) != 0) 129 | goto cleanup; 130 | 131 | status = 0; 132 | 133 | cleanup: 134 | sfree(data, data_len); 135 | if (status && tag_ptr && *tag_ptr) { 136 | apdp_tag_free(*tag_ptr); 137 | *tag_ptr = NULL; 138 | } 139 | return status; 140 | } 141 | 142 | 143 | /** 144 | * @brief Implements the interface of pdp_get_block_noop 145 | * 146 | * To be used with MAC-PDP and a flat-file storage back-end. 147 | * 148 | * @todo use mmap to speed up access? 149 | * 150 | * @param[in] ctx pointer to a context structure. 151 | * @param[in] i the index of block. 152 | * @param[out] b pointer to an output buffer. 153 | * @param[in,out] len initially the size of buf, output bytes written. 154 | * @return 0 on success, non-zero on error 155 | **/ 156 | int apdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 157 | void *b, unsigned int *len) 158 | { 159 | if (!is_apdp(ctx)) return -1; 160 | return pdp_get_block_file(ctx, i, ctx->apdp_param->block_size, 161 | (unsigned char *) b, len); 162 | } 163 | 164 | /** @} */ 165 | -------------------------------------------------------------------------------- /libpdp/src/apdp/apdp_misc.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for A-PDP module backend storage logic. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup APDP 13 | * @{ 14 | */ 15 | #ifndef __A_PDP_MISC_H__ 16 | #define __A_PDP_MISC_H__ 17 | 18 | 19 | /* 20 | * function prototypes - apdp.c 21 | */ 22 | pdp_apdp_tag_t *apdp_tag_new(void); 23 | void apdp_tag_free(pdp_apdp_tag_t *tag); 24 | 25 | /* 26 | * function prototypes - apdp_serialize.c 27 | */ 28 | unsigned int apdp_serialized_tag_size(const pdp_ctx_t *ctx); 29 | int apdp_serialize_tags(const pdp_ctx_t *ctx, const pdp_apdp_tagdata_t* t, 30 | unsigned char **buffer, unsigned int *buffer_len); 31 | int apdp_deserialize_tag(const pdp_ctx_t *ctx, pdp_apdp_tag_t* tag, 32 | unsigned char *buffer, unsigned int buffer_len); 33 | 34 | #endif 35 | /** @} */ 36 | -------------------------------------------------------------------------------- /libpdp/src/apdp/apdp_s3.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the A-PDP module for S3 storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup APDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #ifdef _S3_SUPPORT 21 | #include 22 | #include 23 | #endif 24 | #include 25 | #include 26 | #include "pdp_misc.h" 27 | #include "pdp_storage.h" 28 | #include "apdp_storage.h" 29 | #include "apdp_misc.h" 30 | 31 | 32 | /** 33 | * @brief Write out file and tag data to S3. 34 | * @param[in] ctx pointer to a context structure. 35 | * @param[in] t pointer to the tag structure. 36 | * @return 0 on success, non-zero on error 37 | **/ 38 | int apdp_write_data_to_s3(const pdp_ctx_t *ctx, const pdp_apdp_tagdata_t* t) 39 | { 40 | #ifndef _S3_SUPPORT 41 | PDP_UNSUPPORTED("S3"); 42 | return -1; 43 | #else 44 | unsigned char *buf = NULL; 45 | unsigned int len = 0; 46 | int status = -1; 47 | 48 | if (!t || !t->tags) return -1; 49 | 50 | // write file to s3 51 | if (pdp_write_file_to_s3(ctx, ctx->filepath)) { 52 | PDP_ERR("error writing original file to s3"); 53 | return -1; 54 | } 55 | // write tags to s3 56 | if ((apdp_serialize_tags(ctx, t, &buf, &len)) != 0) goto cleanup; 57 | if (pdp_write_data_to_s3(ctx, ctx->ofilepath, buf, len)) { 58 | PDP_ERR("error writing tag data to s3"); 59 | goto cleanup; 60 | } 61 | status = 0; 62 | 63 | cleanup: 64 | sfree(buf, len); 65 | return status; 66 | #endif // _S3_SUPPORT 67 | } 68 | 69 | 70 | /** 71 | * @brief Implements the interface of get_tag_stub() 72 | * 73 | * To be used with A-PDP and S3 storage back-end. 74 | * 75 | * @param[in] ctx pointer to a context structure. 76 | * @param[in] index the index of tag. 77 | * @param[out] buffer pointer to an output buffer. 78 | * @param[in,out] b_len the size of buf; on success, holds bytes written. 79 | * @return 0 on success, non-zero on error 80 | **/ 81 | int apdp_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 82 | void *buffer, unsigned int* b_len) 83 | { 84 | #ifndef _S3_SUPPORT 85 | PDP_UNSUPPORTED("S3"); 86 | return -1; 87 | #else 88 | pdp_apdp_tag_t **tag_ptr = (pdp_apdp_tag_t **) buffer; 89 | pdp_apdp_tag_t *tag = NULL; 90 | unsigned char *buf = NULL; 91 | unsigned int i = 0; 92 | unsigned int buf_len = 0; 93 | unsigned int buf_size = 0; 94 | int status = -1; 95 | 96 | if (!ctx || !tag_ptr) 97 | return -1; 98 | 99 | // Allocate space for the tag 100 | if ((tag = apdp_tag_new()) == NULL) goto cleanup; 101 | if (*tag_ptr != NULL) { 102 | // there is already a tag here, so free it 103 | apdp_tag_free(*tag_ptr); 104 | } 105 | *tag_ptr = tag; 106 | 107 | buf_size = apdp_serialized_tag_size(ctx); 108 | if ((buf = malloc(buf_size)) == NULL) goto cleanup; 109 | memset(buf, 0, buf_size); 110 | i = index * buf_size; 111 | buf_len = buf_size; 112 | if (pdp_get_chunk_from_s3(ctx, ctx->ofilepath, &buf, &buf_len, i) != 0) 113 | goto cleanup; 114 | if (buf_len != buf_size) 115 | goto cleanup; 116 | if (apdp_deserialize_tag(ctx, tag, buf, buf_size) != 0) 117 | goto cleanup; 118 | status = 0; 119 | 120 | #ifdef _PDP_DEBUG 121 | DEBUG(1, "\n Reading - "); 122 | DEBUG(1, "\n Tag %02d", index); 123 | DEBUG(1, "\n Tim byte len [%02d]", BN_num_bytes(tag->Tim)); 124 | DEBUG(1, "\n Tim (hex) [%s]", BN_bn2hex(tag->Tim)); 125 | DEBUG(1, "\n index [%d]", tag->index); 126 | DEBUG(1, "\n prf_size [%d]", tag->index_prf_size); 127 | pdp_hexdump(" prf", index, tag->index_prf, tag->index_prf_size); 128 | pdp_hexdump(" Ser. tag", index, buf, buf_size); 129 | #endif // _PDP_DEBUG 130 | 131 | cleanup: 132 | sfree(buf, buf_len); 133 | if (status) { 134 | apdp_tag_free(tag); 135 | if (tag_ptr && *tag_ptr) *tag_ptr = NULL; 136 | } 137 | return status; 138 | #endif // _S3_SUPPORT 139 | } 140 | 141 | 142 | /** 143 | * @brief Implements the interface of get_block_stub() 144 | * 145 | * To be used with A-PDP and S3 storage back-end. 146 | * 147 | * @param[in] ctx pointer to a context structure. 148 | * @param[in] index the index of tag. 149 | * @param[out] b pointer to an output buffer. 150 | * @param[in,out] len the size of buf; on success, holds bytes written. 151 | * @return 0 on success, non-zero on error 152 | **/ 153 | int apdp_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 154 | void *b, unsigned int* len) 155 | { 156 | #ifndef _S3_SUPPORT 157 | PDP_UNSUPPORTED("S3"); 158 | return -1; 159 | #else 160 | unsigned int i = index * ctx->apdp_param->block_size; 161 | unsigned char *buf = (unsigned char *) b; 162 | return pdp_get_chunk_from_s3(ctx, ctx->filepath, &buf, len, i); 163 | #endif // _S3_SUPPORT 164 | } 165 | 166 | /** @} */ 167 | -------------------------------------------------------------------------------- /libpdp/src/apdp/apdp_storage.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for MAC-PDP module backend storage logic. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup APDP 13 | * @{ 14 | */ 15 | #ifndef __A_PDP_STORAGE_H__ 16 | #define __A_PDP_STORAGE_H__ 17 | 18 | /* 19 | * function prototypes - apdp_file.c 20 | */ 21 | int apdp_write_tags_to_file(const pdp_ctx_t *ctx, 22 | const pdp_apdp_tagdata_t* t); 23 | int apdp_get_tag_file(const pdp_ctx_t* ctx, unsigned int i, 24 | void *b, unsigned int *len); 25 | int apdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 26 | void *b, unsigned int *len); 27 | /* 28 | * function prototypes - apdp_s3.c 29 | */ 30 | int apdp_write_data_to_s3(const pdp_ctx_t *ctx, const pdp_apdp_tagdata_t* t); 31 | int apdp_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 32 | void *b, unsigned int* buf_len); 33 | int apdp_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 34 | void *b, unsigned int* len); 35 | 36 | #endif 37 | /** @} */ 38 | -------------------------------------------------------------------------------- /libpdp/src/cpor/cpor_file.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the A-PDP module for flat-file storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup CPOR 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include "pdp_misc.h" 23 | #include "pdp_storage.h" 24 | #include "cpor_storage.h" 25 | #include "cpor_misc.h" 26 | 27 | /** 28 | * @brief Write tag data out to a file. 29 | * @param[in] ctx pointer to a context structure. 30 | * @param[in] t pointer to tag data. 31 | * @return 0 on success, non-zero on error 32 | **/ 33 | int cpor_write_tags_to_file(const pdp_ctx_t *ctx, const pdp_cpor_tagdata_t* t) 34 | { 35 | unsigned char *buf = NULL; 36 | FILE *tagfile = NULL; 37 | unsigned int len = 0; 38 | size_t off = 0; 39 | int status = -1; 40 | 41 | if (!ctx || !t || !t->tags || !t->tags_num || 42 | !t->tags_size || !ctx->ofilepath) 43 | return -1; 44 | 45 | if ((access(ctx->ofilepath, F_OK) == 0) && (ctx->verbose)) { 46 | PDP_ERR("WARNING: overwriting [%s]", ctx->ofilepath); 47 | } 48 | if ((tagfile = fopen(ctx->ofilepath, "w")) == NULL){ 49 | PDP_ERR("%d", ferror(tagfile)); 50 | PDP_ERR("unable to create %s", ctx->ofilepath); 51 | goto cleanup; 52 | } 53 | 54 | if ((cpor_serialize_tags(ctx, t, &buf, &len)) != 0) goto cleanup; 55 | do { 56 | off += fwrite(buf + off, 1, len, tagfile); 57 | if (ferror(tagfile)) goto cleanup; 58 | 59 | } while (off < len); 60 | status = 0; 61 | 62 | cleanup: 63 | sfree(buf, len); 64 | if (tagfile) fclose(tagfile); 65 | return status; 66 | } 67 | 68 | 69 | /** 70 | * @brief Implements the interface of pdp_get_tag_noop 71 | * 72 | * To be used with a flat-file storage back-end. 73 | * 74 | * @todo use mmap to speed up access? 75 | * 76 | * @param[in] ctx pointer to a context structure. 77 | * @param[in] index the index of tag. 78 | * @param[out] buf pointer to an output buffer. 79 | * @param[in,out] len initially the size of buf, output bytes written. 80 | * @return 0 on success, non-zero on error 81 | **/ 82 | int cpor_get_tag_file(const pdp_ctx_t* ctx, unsigned int index, 83 | void *buf, unsigned int *len) 84 | { 85 | pdp_cpor_tag_t **tag_ptr = (pdp_cpor_tag_t **) buf; 86 | pdp_cpor_tag_t *tag = NULL; 87 | static FILE* tagfile = NULL; 88 | unsigned char *data = NULL; 89 | unsigned int data_len = 0; 90 | int status = -1; 91 | 92 | // If buf is NULL, it is a signal to re-set the stateful variables 93 | if (!buf && tagfile) { 94 | fclose(tagfile); 95 | tagfile = NULL; 96 | return 0; 97 | } 98 | 99 | if (!ctx || !tag_ptr) 100 | return -1; 101 | 102 | // if file is not open already, open it 103 | if (!tagfile && ((tagfile = fopen(ctx->ofilepath, "r")) == NULL)) { 104 | PDP_ERR("%d", ferror(tagfile)); 105 | PDP_ERR("unable to open %s", ctx->ofilepath); 106 | return -1; 107 | } 108 | 109 | // Allocate space for the tag 110 | if ((tag = cpor_tag_new()) == NULL) goto cleanup; 111 | if (*tag_ptr != NULL) { 112 | // there is already a tag here, so free it 113 | cpor_tag_free(*tag_ptr); 114 | } 115 | *tag_ptr = tag; 116 | 117 | data_len = cpor_serialized_tag_size(ctx); 118 | if ((data = malloc(data_len)) == NULL) goto cleanup; 119 | memset(data, 0, data_len); 120 | 121 | // We seek directly to the index of the i-th tag 122 | if (fseek(tagfile, index * data_len, SEEK_SET) < 0) goto cleanup; 123 | fread(data, 1, data_len, tagfile); 124 | if (ferror(tagfile)) goto cleanup; 125 | 126 | // de-serialize the data block 127 | if (cpor_deserialize_tag(ctx, tag, data, data_len) != 0) 128 | goto cleanup; 129 | 130 | status = 0; 131 | 132 | cleanup: 133 | sfree(data, data_len); 134 | if (status && tag_ptr && *tag_ptr) { 135 | cpor_tag_free(*tag_ptr); 136 | *tag_ptr = NULL; 137 | } 138 | return status; 139 | } 140 | 141 | 142 | /** 143 | * @brief Implements the interface of pdp_get_block_noop 144 | * 145 | * To be used with MAC-PDP and a flat-file storage back-end. 146 | * 147 | * @todo use mmap to speed up access? 148 | * 149 | * @param[in] ctx pointer to a context structure. 150 | * @param[in] i the index of block. 151 | * @param[out] b pointer to an output buffer. 152 | * @param[in,out] len initially the size of buf, output bytes written. 153 | * @return 0 on success, non-zero on error 154 | **/ 155 | int cpor_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 156 | void *b, unsigned int *len) 157 | { 158 | if (!is_cpor(ctx)) return -1; 159 | return pdp_get_block_file(ctx, i, ctx->cpor_param->block_size, 160 | (unsigned char *) b, len); 161 | } 162 | 163 | /** @} */ 164 | -------------------------------------------------------------------------------- /libpdp/src/cpor/cpor_misc.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for A-PDP module backend storage logic. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup CPOR 13 | * @{ 14 | */ 15 | #ifndef __CPOR_PDP_MISC_H__ 16 | #define __CPOR_PDP_MISC_H__ 17 | 18 | /* 19 | * function prototypes - cpor_serialize.c 20 | */ 21 | unsigned int cpor_serialized_tag_size(const pdp_ctx_t *ctx); 22 | int cpor_serialize_tags(const pdp_ctx_t *ctx, const pdp_cpor_tagdata_t* t, 23 | unsigned char **buffer, unsigned int *buffer_len); 24 | int cpor_deserialize_tag(const pdp_ctx_t *ctx, pdp_cpor_tag_t* tag, 25 | unsigned char *buf, unsigned int buf_len); 26 | 27 | /* 28 | * function prototypes - cpor.c 29 | */ 30 | pdp_cpor_tag_t *cpor_tag_new(void); 31 | void cpor_tag_free(pdp_cpor_tag_t *tag); 32 | int cpor_fkey_open(const pdp_ctx_t *ctx, const pdp_cpor_key_t *key, 33 | pdp_cpor_tagdata_t **td); 34 | int cpor_fkey_store(const pdp_ctx_t *ctx, const pdp_cpor_key_t *key, 35 | const pdp_cpor_tagdata_t *td); 36 | 37 | #endif 38 | /** @} */ 39 | -------------------------------------------------------------------------------- /libpdp/src/cpor/cpor_s3.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the A-PDP module for S3 storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup CPOR 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #ifdef _S3_SUPPORT 21 | #include 22 | #include 23 | #endif 24 | #include 25 | #include 26 | #include "pdp_misc.h" 27 | #include "pdp_storage.h" 28 | #include "cpor_storage.h" 29 | #include "cpor_misc.h" 30 | 31 | 32 | /** 33 | * @brief Write out file and tag data to S3. 34 | **/ 35 | int cpor_write_data_to_s3(const pdp_ctx_t *ctx, const pdp_cpor_tagdata_t* t) 36 | { 37 | #ifndef _S3_SUPPORT 38 | PDP_UNSUPPORTED("S3"); 39 | return -1; 40 | #else 41 | unsigned char *buf = NULL; 42 | unsigned int len = 0; 43 | int status = -1; 44 | 45 | if (!t || !t->tags) 46 | return -1; 47 | // write file to s3 48 | if (pdp_write_file_to_s3(ctx, ctx->filepath)) { 49 | PDP_ERR("error writing original file to s3"); 50 | return -1; 51 | } 52 | // write tags to s3 53 | if ((cpor_serialize_tags(ctx, t, &buf, &len)) != 0) goto cleanup; 54 | if (pdp_write_data_to_s3(ctx, ctx->ofilepath, buf, len)) { 55 | PDP_ERR("error writing tag data to s3"); 56 | goto cleanup; 57 | } 58 | status = 0; 59 | 60 | cleanup: 61 | sfree(buf, len); 62 | return status; 63 | #endif // _S3_SUPPORT 64 | } 65 | 66 | 67 | /** 68 | * @brief Implements the interface of get_tag_stub() 69 | * 70 | * To be used with CPOR and S3 storage back-end. 71 | * 72 | * @param[in] ctx pointer to a context structure. 73 | * @param[in] index the index of tag. 74 | * @param[out] buffer pointer to an output buffer. 75 | * @param[in,out] b_len the size of buf; on success, holds bytes written. 76 | * @return 0 on success, non-zero on error 77 | **/ 78 | int cpor_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 79 | void *buffer, unsigned int *b_len) 80 | { 81 | #ifndef _S3_SUPPORT 82 | PDP_UNSUPPORTED("S3"); 83 | return -1; 84 | #else 85 | pdp_cpor_tag_t **tag_ptr = (pdp_cpor_tag_t **) buffer; 86 | pdp_cpor_tag_t *tag = NULL; 87 | unsigned char *buf = NULL; 88 | unsigned int i = 0; 89 | unsigned int buf_len = 0; 90 | unsigned int buf_size = 0; 91 | int status = -1; 92 | 93 | if (!ctx || !tag_ptr) 94 | return -1; 95 | 96 | // Allocate space for the tag 97 | if ((tag = cpor_tag_new()) == NULL) goto cleanup; 98 | if (*tag_ptr != NULL) { 99 | // there is already a tag here, so free it 100 | cpor_tag_free(*tag_ptr); 101 | } 102 | *tag_ptr = tag; 103 | 104 | buf_size = cpor_serialized_tag_size(ctx); 105 | if ((buf = malloc(buf_size)) == NULL) goto cleanup; 106 | memset(buf, 0, buf_size); 107 | i = index * buf_size; 108 | buf_len = buf_size; 109 | if (pdp_get_chunk_from_s3(ctx, ctx->ofilepath, &buf, &buf_len, i) != 0) 110 | goto cleanup; 111 | if (buf_len != buf_size) 112 | goto cleanup; 113 | if (cpor_deserialize_tag(ctx, tag, buf, buf_size) != 0) 114 | goto cleanup; 115 | status = 0; 116 | 117 | cleanup: 118 | sfree(buf, buf_len); 119 | if (status) { 120 | cpor_tag_free(tag); 121 | if (tag_ptr && *tag_ptr) *tag_ptr = NULL; 122 | } 123 | return status; 124 | #endif // _S3_SUPPORT 125 | } 126 | 127 | 128 | /** 129 | * @brief Implements the interface of get_block_stub() 130 | * 131 | * To be used with MAC-PDP and S3 storage back-end. 132 | * 133 | * @param[in] ctx pointer to a context structure. 134 | * @param[in] index the index of tag. 135 | * @param[out] b pointer to an output buffer. 136 | * @param[in,out] len the size of buf; on success, holds bytes written. 137 | * @return 0 on success, non-zero on error 138 | **/ 139 | int cpor_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 140 | void *b, unsigned int *len) 141 | { 142 | #ifndef _S3_SUPPORT 143 | PDP_UNSUPPORTED("S3"); 144 | return -1; 145 | #else 146 | unsigned int i = index * ctx->cpor_param->block_size; 147 | unsigned char *buf = (unsigned char *) b; 148 | return pdp_get_chunk_from_s3(ctx, ctx->filepath, &buf, len, i); 149 | #endif // _S3_SUPPORT 150 | } 151 | 152 | /** @} */ 153 | -------------------------------------------------------------------------------- /libpdp/src/cpor/cpor_serialize.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the A-PDP module for S3 storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup CPOR 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include "cpor_misc.h" 23 | #include "pdp_misc.h" 24 | 25 | 26 | /** 27 | * @brief Returns the upper bound of the serialized token 28 | * 29 | * Each token is of the form: 30 | * 31 | **/ 32 | unsigned int cpor_serialized_tag_size(const pdp_ctx_t *ctx) 33 | { 34 | pdp_cpor_ctx_t *p = NULL; 35 | 36 | if (!is_cpor(ctx)) return -1; 37 | p = ctx->cpor_param; 38 | 39 | return (sizeof(unsigned int) + sizeof(size_t) + (p->Zp_bits + 7)/8); 40 | } 41 | 42 | 43 | /** 44 | * @brief Write out tag data to a buffer 45 | * 46 | * @todo This serialization is not portable. 47 | * @param[in] ctx context 48 | * @param[in] t pointer to input tag data 49 | * @param[in] buffer the buffer for the serialized data 50 | * @param[in] buffer_len length of space available, output bytes written 51 | * @return 0 on success, non-zero on error 52 | **/ 53 | int cpor_serialize_tags(const pdp_ctx_t *ctx, const pdp_cpor_tagdata_t* t, 54 | unsigned char **buffer, unsigned int *buffer_len) 55 | { 56 | pdp_cpor_ctx_t *p = NULL; 57 | pdp_cpor_tag_t *tag = NULL; 58 | unsigned char *buf_ptr = NULL; 59 | unsigned char *buf = NULL; 60 | unsigned char *sigma = NULL; 61 | size_t tag_size, sigma_size, sigma_len; 62 | unsigned int buf_len; 63 | int i, status = -1; 64 | 65 | if (!is_cpor(ctx) || !t || !buffer || !buffer_len || 66 | !t->tags || !t->tags_num || !t->tags_size) 67 | return -1; 68 | p = ctx->cpor_param; 69 | 70 | // some useful constants: upper bounds on sizes of things 71 | tag_size = cpor_serialized_tag_size(ctx); 72 | sigma_size = (p->Zp_bits + 7)/8; // Size is max size log2(key->Zp) 73 | 74 | // Tag is 75 | buf_len = t->tags_num * tag_size; 76 | 77 | // allocate buffers 78 | if ((sigma = malloc(sigma_size)) == NULL) goto cleanup; 79 | if ((buf = malloc(buf_len)) == NULL) goto cleanup; 80 | memset(buf, 0, buf_len); 81 | buf_ptr = buf; 82 | 83 | for(i = 0; i < t->tags_num; i++) { 84 | tag = t->tags[i]; 85 | 86 | // get real sigma byte len 87 | sigma_len = BN_num_bytes(tag->sigma); 88 | 89 | // make sure our assumptions re: bounds are correct 90 | if (sigma_len > sigma_size) goto cleanup; 91 | 92 | // write index 93 | memcpy(buf_ptr, &(tag->index), sizeof(tag->index)); 94 | buf_ptr += sizeof(tag->index); 95 | 96 | // write sigma size 97 | memcpy(buf_ptr, &sigma_len, sizeof(sigma_len)); 98 | buf_ptr += sizeof(sigma_len); 99 | 100 | // write sigma 101 | memset(sigma, 0, sigma_size); 102 | if (!BN_bn2bin(tag->sigma, sigma)) goto cleanup; 103 | memcpy(buf_ptr, sigma, sigma_size); 104 | buf_ptr += sigma_size; 105 | 106 | #ifdef _PDP_DEBUG 107 | DEBUG(1, "\n Writing - "); 108 | DEBUG(1, "\n Tag %02d", i); 109 | DEBUG(1, "\n index [%d]", tag->index); 110 | DEBUG(1, "\n Sigma byte len [%02d]", BN_num_bytes(tag->sigma)); 111 | DEBUG(1, "\n Sigma (hex) [%s]", BN_bn2hex(tag->sigma)); 112 | pdp_hexdump(" Ser. tag", i, buf_ptr - tag_size, tag_size); 113 | #endif // _PDP_DEBUG 114 | 115 | } 116 | *buffer = buf; 117 | *buffer_len = buf_len; 118 | status = 0; 119 | 120 | cleanup: 121 | sfree(sigma, sigma_size); 122 | if (status) sfree(buf, buf_len); 123 | return status; 124 | } 125 | 126 | 127 | /** 128 | * @brief Read an individual tag 129 | * @param[in] ctx context 130 | * @param[out] tag pointer to tag that will be populated 131 | * @param[in] buf serialized structure to process 132 | * @param[in] buf_len length of data to process 133 | * @return 0 on success, non-zero on error 134 | **/ 135 | int cpor_deserialize_tag(const pdp_ctx_t *ctx, pdp_cpor_tag_t* tag, 136 | unsigned char *buf, unsigned int buf_len) 137 | { 138 | pdp_cpor_ctx_t *p = NULL; 139 | unsigned char *buf_ptr = buf; 140 | unsigned char *sigma = NULL; 141 | size_t sigma_size, tag_size; 142 | size_t sigma_len = 0; 143 | int status = -1; 144 | 145 | if (!is_cpor(ctx) || !tag || !buf || !buf_len) 146 | return -1; 147 | p = ctx->cpor_param; 148 | 149 | // some useful constants: upper bounds on sizes of things 150 | tag_size = cpor_serialized_tag_size(ctx); 151 | sigma_size = (p->Zp_bits + 7)/8; // Size is max size log2(key->Zp) 152 | 153 | // double-check size of buf 154 | if (buf_len < tag_size) goto cleanup; 155 | 156 | // Tag is 157 | memcpy(&(tag->index), buf_ptr, sizeof(tag->index)); 158 | buf_ptr += sizeof(tag->index); 159 | 160 | // read sigma size 161 | memcpy(&sigma_len, buf_ptr, sizeof(sigma_len)); 162 | buf_ptr += sizeof(sigma_len); 163 | 164 | // double-check the size is sensible 165 | if (sigma_size < sigma_len) goto cleanup; 166 | 167 | // read sigma 168 | if ((sigma = malloc(sigma_len)) == NULL) goto cleanup; 169 | memset(sigma, 0, sigma_len); 170 | memcpy(sigma, buf_ptr, sigma_len); 171 | if (!BN_bin2bn(sigma, sigma_len, tag->sigma)) goto cleanup; 172 | 173 | status = 0; 174 | 175 | #ifdef _PDP_DEBUG 176 | DEBUG(1, "\n Reading - "); 177 | DEBUG(1, "\n Tag %02d", tag->index); 178 | DEBUG(1, "\n Sigma byte len [%02d]", BN_num_bytes(tag->sigma)); 179 | DEBUG(1, "\n Sigma (hex) [%s]", BN_bn2hex(tag->sigma)); 180 | pdp_hexdump(" Ser. tag", tag->index, buf, buf_len); 181 | #endif // _PDP_DEBUG 182 | 183 | cleanup: 184 | sfree(sigma, sigma_len); 185 | return status; 186 | } 187 | 188 | /** @} */ 189 | -------------------------------------------------------------------------------- /libpdp/src/cpor/cpor_storage.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for MAC-PDP module backend storage logic. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup CPOR 13 | * @{ 14 | */ 15 | #ifndef __CPOR_PDP_STORAGE_H__ 16 | #define __CPOR_PDP_STORAGE_H__ 17 | 18 | /* 19 | * function prototypes - apdp_file.c 20 | */ 21 | int cpor_write_tags_to_file(const pdp_ctx_t *ctx, const pdp_cpor_tagdata_t* t); 22 | int cpor_get_tag_file(const pdp_ctx_t* ctx, unsigned int i, 23 | void *b, unsigned int *len); 24 | int cpor_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 25 | void *b, unsigned int *len); 26 | 27 | /* 28 | * function prototypes - apdp_s3.c 29 | */ 30 | int cpor_write_data_to_s3(const pdp_ctx_t *ctx, const pdp_cpor_tagdata_t* t); 31 | int cpor_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 32 | void *buffer, unsigned int* b_len); 33 | int cpor_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 34 | void *b, unsigned int* buf_len); 35 | 36 | #endif 37 | /** @} */ 38 | -------------------------------------------------------------------------------- /libpdp/src/macpdp/macpdp_file.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the MAC-PDP module for flat-file storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MACPDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #ifdef _THREAD_SUPPORT 21 | #include 22 | #endif 23 | #include 24 | #include 25 | #include "pdp_misc.h" 26 | #include "pdp_storage.h" 27 | #include "macpdp_storage.h" 28 | 29 | /** 30 | * @brief Write tag data out to a file. 31 | * @param[in] ctx pointer to a context structure. 32 | * @param[in] t pointer to tag data. 33 | * @return 0 on success, non-zero on error 34 | **/ 35 | int macpdp_write_tags_to_file(const pdp_ctx_t *ctx, 36 | const pdp_macpdp_tagdata_t* t) 37 | { 38 | pdp_macpdp_ctx_t *p = ctx->macpdp_param; 39 | FILE *tagfile = NULL; 40 | unsigned char *tag_ptr = NULL; 41 | int i, len, status = -1; 42 | 43 | if (!is_macpdp(ctx) || !t || !t->tags || !ctx->ofilepath) 44 | return -1; 45 | 46 | if ((access(ctx->ofilepath, F_OK) == 0) && (ctx->verbose)) { 47 | PDP_ERR("WARNING: overwriting [%s]", ctx->ofilepath); 48 | } 49 | if ((tagfile = fopen(ctx->ofilepath, "w")) == NULL){ 50 | PDP_ERR("%d", ferror(tagfile)); 51 | PDP_ERR("unable to create %s", ctx->ofilepath); 52 | goto cleanup; 53 | } 54 | 55 | tag_ptr = t->tags; 56 | len = p->tag_size; 57 | for(i = 0; i < p->num_blocks; i++) { 58 | tag_ptr += i ? len : 0; 59 | 60 | fwrite(tag_ptr, len, 1, tagfile); 61 | if (ferror(tagfile)) goto cleanup; 62 | } 63 | status = 0; 64 | 65 | cleanup: 66 | if (tagfile) fclose(tagfile); 67 | return status; 68 | } 69 | 70 | 71 | /** 72 | * @brief Implements the interface of pdp_get_tag_noop 73 | * 74 | * To be used with MAC-PDP and a flat-file storage back-end. 75 | * 76 | * @todo Use mmap to speed up access? 77 | * 78 | * @param[in] ctx pointer to a context structure. 79 | * @param[in] i the index of tag. 80 | * @param[out] b pointer to an output buffer. 81 | * @param[in,out] len initially the size of buf, output bytes written. 82 | * @return 0 on success, non-zero on error 83 | **/ 84 | int macpdp_get_tag_file(const pdp_ctx_t* ctx, unsigned int i, 85 | void *b, unsigned int *len) 86 | { 87 | pdp_macpdp_ctx_t *p = ctx->macpdp_param; 88 | unsigned char *buf = (unsigned char *) b; 89 | static FILE* tagfile = NULL; 90 | int pos = 0; 91 | 92 | // if buf is NULL, its a signal to re-set the stateful variables 93 | if (!buf && tagfile) { 94 | fclose(tagfile); 95 | tagfile = NULL; 96 | return 0; 97 | } 98 | 99 | if (!is_macpdp(ctx) || !buf || !len || !*len) 100 | return -1; 101 | 102 | if (*len % p->tag_size) 103 | PDP_ERR("warning: reading a partial tag (buffer is " 104 | "%d bytes; tag is %d bytes)", *len, p->tag_size); 105 | memset(buf, 0, *len); 106 | 107 | // if file is not open already, open it 108 | if (!tagfile && ((tagfile = fopen(ctx->ofilepath, "r")) == NULL)) { 109 | PDP_ERR("unable to open %s", ctx->ofilepath); 110 | return -1; 111 | } 112 | 113 | // seek to correct place in file and read out the requested tag 114 | pos = i * p->tag_size; 115 | if (fseek(tagfile, pos, SEEK_SET) != 0) { 116 | PDP_ERR("fseek error %d", errno); 117 | return -1; 118 | } 119 | *len = fread(buf, 1, *len, tagfile); 120 | 121 | return 0; 122 | } 123 | 124 | 125 | /** 126 | * @brief Implements the interface of pdp_get_block_noop 127 | * 128 | * To be used with MAC-PDP and a flat-file storage back-end. 129 | * 130 | * @todo Use mmap to speed up access? 131 | * 132 | * @param[in] ctx pointer to a context structure. 133 | * @param[in] i the index of block. 134 | * @param[out] b pointer to an output buffer. 135 | * @param[in,out] len initially the size of buf, output bytes written. 136 | * @return 0 on success, non-zero on error 137 | **/ 138 | int macpdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 139 | void *b, unsigned int *len) 140 | { 141 | if (!is_macpdp(ctx)) return -1; 142 | return pdp_get_block_file(ctx, i, ctx->macpdp_param->block_size, 143 | (unsigned char *) b, len); 144 | } 145 | 146 | /** @} */ 147 | -------------------------------------------------------------------------------- /libpdp/src/macpdp/macpdp_key.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the MAC-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MACPDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include "pdp_misc.h" 24 | 25 | 26 | /** 27 | * @brief Generate key material. 28 | * 29 | * Creates a random PRF key to use for our MAC (which is HMAC) 30 | * 31 | * @return 0 on success, non-zero on error 32 | **/ 33 | int macpdp_key_gen(const pdp_ctx_t *ctx, pdp_key_t *k) 34 | { 35 | pdp_macpdp_ctx_t *p = NULL; 36 | pdp_macpdp_key_t *key = NULL; 37 | 38 | if (!is_macpdp(ctx) || !k) 39 | return -1; 40 | p = ctx->macpdp_param; 41 | 42 | if (!p->prf_key_size) 43 | return -1; 44 | 45 | if ((key = malloc(sizeof(pdp_macpdp_key_t))) == NULL) 46 | return -1; 47 | memset(key, 0, sizeof(pdp_macpdp_key_t)); 48 | k->macpdp = key; 49 | 50 | key->prf_key_size = p->prf_key_size; 51 | if ((key->prf_key = malloc(key->prf_key_size)) == NULL) 52 | goto cleanup; 53 | if (!RAND_bytes(key->prf_key, key->prf_key_size)) 54 | goto cleanup; 55 | 56 | return 0; 57 | 58 | cleanup: 59 | macpdp_key_free(k); 60 | return -1; 61 | } 62 | 63 | 64 | /** 65 | * @brief Destroy and free allocated key material. 66 | * @return 0 on success, non-zero on error 67 | **/ 68 | int macpdp_key_free(pdp_key_t *k) 69 | { 70 | if (!k || !k->macpdp) 71 | return -1; 72 | sfree(k->macpdp->prf_key, k->macpdp->prf_key_size); 73 | sfree(k->macpdp, sizeof(pdp_macpdp_key_t)); 74 | return 0; 75 | } 76 | 77 | /** @} */ 78 | -------------------------------------------------------------------------------- /libpdp/src/macpdp/macpdp_s3.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the MAC-PDP module for S3 storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MACPDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #ifdef _S3_SUPPORT 21 | #include 22 | #include 23 | #endif 24 | #ifdef _THREAD_SUPPORT 25 | #include 26 | #endif 27 | #include 28 | #include 29 | #include "pdp_misc.h" 30 | #include "pdp_storage.h" 31 | #include "macpdp_storage.h" 32 | 33 | 34 | /** 35 | * @brief Write out file and tag data to S3. 36 | * @param[in] ctx pointer to a context structure. 37 | * @param[in] t pointer to the tag structure. 38 | * @return 0 on success, non-zero on error 39 | **/ 40 | int macpdp_write_data_to_s3(const pdp_ctx_t *ctx, 41 | const pdp_macpdp_tagdata_t* t) 42 | { 43 | #ifndef _S3_SUPPORT 44 | PDP_UNSUPPORTED("S3"); 45 | return -1; 46 | #else 47 | if (!t || !t->tags) 48 | return -1; 49 | // write file to s3 50 | if (pdp_write_file_to_s3(ctx, ctx->filepath)) { 51 | PDP_ERR("error writing original file to s3"); 52 | return -1; 53 | } 54 | // write tags to s3 55 | if (pdp_write_data_to_s3(ctx, ctx->ofilepath, t->tags, t->tags_size)) { 56 | PDP_ERR("error writing tag data to s3"); 57 | return -1; 58 | } 59 | return 0; 60 | #endif 61 | } 62 | 63 | 64 | /** 65 | * @brief Implements the interface of pdp_get_tag_noop 66 | * 67 | * To be used with MAC-PDP and S3 storage back-end. 68 | * 69 | * @param[in] ctx pointer to a context structure. 70 | * @param[in] index the index of tag. 71 | * @param[out] b pointer to an output buffer. 72 | * @param[in,out] len the size of buf; on success, holds bytes written. 73 | * @return 0 on success, non-zero on error 74 | **/ 75 | int macpdp_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 76 | void *b, unsigned int* len) 77 | { 78 | #ifndef _S3_SUPPORT 79 | PDP_UNSUPPORTED("S3"); 80 | return -1; 81 | #else 82 | unsigned int i = index * ctx->macpdp_param->tag_size; 83 | unsigned char *buf = (unsigned char *) b; 84 | return pdp_get_chunk_from_s3(ctx, ctx->ofilepath, &buf, len, i); 85 | #endif 86 | } 87 | 88 | 89 | /** 90 | * @brief Implements the interface of pdp_get_block_noop 91 | * 92 | * To be used with MAC-PDP and S3 storage back-end. 93 | * 94 | * @param[in] ctx pointer to a context structure. 95 | * @param[in] index the index of tag. 96 | * @param[out] b pointer to an output buffer. 97 | * @param[in,out] len the size of buf; on success, holds bytes written. 98 | * @return 0 on success, non-zero on error 99 | **/ 100 | int macpdp_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 101 | void *b, unsigned int* len) 102 | { 103 | #ifndef _S3_SUPPORT 104 | PDP_UNSUPPORTED("S3"); 105 | return -1; 106 | #else 107 | unsigned int i = index * ctx->macpdp_param->block_size; 108 | unsigned char *buf = (unsigned char *) b; 109 | return pdp_get_chunk_from_s3(ctx, ctx->filepath, &buf, len, i); 110 | #endif 111 | } 112 | 113 | /** @} */ 114 | -------------------------------------------------------------------------------- /libpdp/src/macpdp/macpdp_storage.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for MAC-PDP module backend storage logic. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MACPDP 13 | * @{ 14 | */ 15 | #ifndef __MAC_PDP_STORAGE_H__ 16 | #define __MAC_PDP_STORAGE_H__ 17 | 18 | /* 19 | * function prototypes - macpdp_file.c 20 | */ 21 | int macpdp_write_tags_to_file(const pdp_ctx_t *ctx, 22 | const pdp_macpdp_tagdata_t* t); 23 | int macpdp_get_tag_file(const pdp_ctx_t* ctx, unsigned int i, 24 | void *b, unsigned int *len); 25 | int macpdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 26 | void *b, unsigned int *len); 27 | /* 28 | * function prototypes - macpdp_s3.c 29 | */ 30 | int macpdp_write_data_to_s3(const pdp_ctx_t *ctx, 31 | const pdp_macpdp_tagdata_t* t); 32 | int macpdp_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 33 | void *b, unsigned int* len); 34 | int macpdp_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 35 | void *b, unsigned int* len); 36 | #endif 37 | /** @} */ 38 | -------------------------------------------------------------------------------- /libpdp/src/mrpdp/mrpdp_file.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the MR-PDP module for flat-file storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MRPDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include "pdp_storage.h" 23 | #include "mrpdp_storage.h" 24 | #include "mrpdp_misc.h" 25 | #include "pdp_misc.h" 26 | 27 | 28 | /** 29 | * @brief Write tag data out to a file. 30 | * @param[in] ctx pointer to a context structure. 31 | * @param[in] t pointer to tag data. 32 | * @return 0 on success, non-zero on error 33 | **/ 34 | int mrpdp_write_tags_to_file(const pdp_ctx_t *ctx, const pdp_mrpdp_tagdata_t* t) 35 | { 36 | unsigned char *buf = NULL; 37 | FILE *tagfile = NULL; 38 | unsigned int len = 0; 39 | size_t off = 0; 40 | int status = -1; 41 | 42 | if (!ctx || !t || !t->tags || !t->tags_num || 43 | !t->tags_size || !ctx->ofilepath) 44 | return -1; 45 | 46 | if ((access(ctx->ofilepath, F_OK) == 0) && (ctx->verbose)) { 47 | PDP_ERR("WARNING: overwriting [%s]", ctx->ofilepath); 48 | } 49 | if ((tagfile = fopen(ctx->ofilepath, "w")) == NULL){ 50 | PDP_ERR("%d", ferror(tagfile)); 51 | PDP_ERR("unable to create %s", ctx->ofilepath); 52 | goto cleanup; 53 | } 54 | 55 | if ((mrpdp_serialize_tags(ctx, t, &buf, &len)) != 0) goto cleanup; 56 | do { 57 | off += fwrite(buf + off, 1, len, tagfile); 58 | if (ferror(tagfile)) goto cleanup; 59 | 60 | } while (off < len); 61 | status = 0; 62 | 63 | cleanup: 64 | sfree(buf, len); 65 | if (tagfile) fclose(tagfile); 66 | return status; 67 | } 68 | 69 | 70 | /** 71 | * @brief Implements the interface of pdp_get_tag_noop 72 | * 73 | * To be used with MAC-PDP and a flat-file storage back-end. 74 | * 75 | * @todo use mmap to speed up access? 76 | * 77 | * @param[in] ctx pointer to a context structure. 78 | * @param[in] index the index of tag. 79 | * @param[out] buf pointer to an output buffer. 80 | * @param[in,out] len initially the size of buf, output bytes written. 81 | * @return 0 on success, non-zero on error 82 | **/ 83 | int mrpdp_get_tag_file(const pdp_ctx_t* ctx, unsigned int index, 84 | void *buf, unsigned int *len) 85 | { 86 | pdp_mrpdp_tag_t **tag_ptr = (pdp_mrpdp_tag_t **) buf; 87 | pdp_mrpdp_tag_t *tag = NULL; 88 | static FILE* tagfile = NULL; 89 | unsigned char *data = NULL; 90 | size_t data_len = 0; 91 | int status = -1; 92 | 93 | // If buf is NULL, it is a signal to re-set the stateful variables 94 | if (!buf && tagfile) { 95 | fclose(tagfile); 96 | tagfile = NULL; 97 | return 0; 98 | } 99 | 100 | if (!ctx || !tag_ptr) 101 | return -1; 102 | 103 | // if file is not open already, open it 104 | if (!tagfile && ((tagfile = fopen(ctx->ofilepath, "r")) == NULL)) { 105 | PDP_ERR("%d", ferror(tagfile)); 106 | PDP_ERR("unable to open %s", ctx->ofilepath); 107 | return -1; 108 | } 109 | 110 | // Allocate space for the tag 111 | if ((tag = mrpdp_tag_new()) == NULL) goto cleanup; 112 | if (*tag_ptr != NULL) { 113 | // there is already a tag here, so free it 114 | mrpdp_tag_free(*tag_ptr); 115 | } 116 | *tag_ptr = tag; 117 | 118 | data_len = mrpdp_serialized_tag_size(ctx); 119 | if ((data = malloc(data_len)) == NULL) goto cleanup; 120 | memset(data, 0, data_len); 121 | 122 | // We seek directly to the index of the i-th tag 123 | if (fseek(tagfile, index * data_len, SEEK_SET) < 0) goto cleanup; 124 | fread(data, 1, data_len, tagfile); 125 | if (ferror(tagfile)) goto cleanup; 126 | 127 | // de-serialize the data block 128 | if (mrpdp_deserialize_tag(ctx, tag, data, data_len) != 0) 129 | goto cleanup; 130 | 131 | status = 0; 132 | 133 | cleanup: 134 | sfree(data, data_len); 135 | if (status && tag_ptr && *tag_ptr) { 136 | mrpdp_tag_free(*tag_ptr); 137 | *tag_ptr = NULL; 138 | } 139 | return status; 140 | } 141 | 142 | 143 | /** 144 | * @brief Implements the interface of pdp_get_block_noop 145 | * 146 | * To be used with MAC-PDP and a flat-file storage back-end. 147 | * 148 | * @todo use mmap to speed up access? 149 | * 150 | * @param[in] ctx pointer to a context structure. 151 | * @param[in] i the index of block. 152 | * @param[out] b pointer to an output buffer. 153 | * @param[in,out] len initially the size of buf, output bytes written. 154 | * @return 0 on success, non-zero on error 155 | **/ 156 | int mrpdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 157 | void *b, unsigned int *len) 158 | { 159 | if (!is_mrpdp(ctx)) return -1; 160 | return pdp_get_block_file(ctx, i, ctx->mrpdp_param->block_size, 161 | (unsigned char *) b, len); 162 | } 163 | 164 | /** @} */ 165 | -------------------------------------------------------------------------------- /libpdp/src/mrpdp/mrpdp_misc.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for MR-PDP module backend storage logic. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MRPDP 13 | * @{ 14 | */ 15 | #ifndef __MR_PDP_MISC_H__ 16 | #define __MR_PDP_MISC_H__ 17 | 18 | 19 | /* 20 | * function prototypes - mrpdp.c 21 | */ 22 | pdp_mrpdp_tag_t *mrpdp_tag_new(void); 23 | void mrpdp_tag_free(pdp_mrpdp_tag_t *tag); 24 | 25 | /* 26 | * function prototypes - mrpdp_serialize.c 27 | */ 28 | unsigned int mrpdp_serialized_tag_size(const pdp_ctx_t *ctx); 29 | int mrpdp_serialize_tags(const pdp_ctx_t *ctx, const pdp_mrpdp_tagdata_t* t, 30 | unsigned char **buffer, unsigned int *buffer_len); 31 | int mrpdp_deserialize_tag(const pdp_ctx_t *ctx, pdp_mrpdp_tag_t* tag, 32 | unsigned char *buffer, unsigned int buffer_len); 33 | 34 | #endif 35 | /** @} */ 36 | -------------------------------------------------------------------------------- /libpdp/src/mrpdp/mrpdp_s3.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the MR-PDP module for S3 storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MRPDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #ifdef _S3_SUPPORT 21 | #include 22 | #include 23 | #endif 24 | #include 25 | #include 26 | #include "pdp_misc.h" 27 | #include "pdp_storage.h" 28 | #include "mrpdp_storage.h" 29 | #include "mrpdp_misc.h" 30 | 31 | 32 | /** 33 | * @brief Write out file and tag data to S3. 34 | * @param[in] ctx pointer to a context structure. 35 | * @param[in] t pointer to the tag structure. 36 | * @return 0 on success, non-zero on error 37 | **/ 38 | int mrpdp_write_data_to_s3(const pdp_ctx_t *ctx, const pdp_mrpdp_tagdata_t* t) 39 | { 40 | #ifndef _S3_SUPPORT 41 | PDP_UNSUPPORTED("S3"); 42 | return -1; 43 | #else 44 | unsigned char *buf = NULL; 45 | unsigned int len = 0; 46 | int status = -1; 47 | 48 | if (!t || !t->tags) return -1; 49 | 50 | // write file to s3 51 | if (pdp_write_file_to_s3(ctx, ctx->filepath)) { 52 | PDP_ERR("error writing original file to s3"); 53 | return -1; 54 | } 55 | // write tags to s3 56 | if ((mrpdp_serialize_tags(ctx, t, &buf, &len)) != 0) goto cleanup; 57 | if (pdp_write_data_to_s3(ctx, ctx->ofilepath, buf, len)) { 58 | PDP_ERR("error writing tag data to s3"); 59 | goto cleanup; 60 | } 61 | status = 0; 62 | 63 | cleanup: 64 | sfree(buf, len); 65 | return status; 66 | #endif // _S3_SUPPORT 67 | } 68 | 69 | 70 | /** 71 | * @brief Implements the interface of get_tag_stub() 72 | * 73 | * To be used with A-PDP and S3 storage back-end. 74 | * 75 | * @param[in] ctx pointer to a context structure. 76 | * @param[in] index the index of tag. 77 | * @param[out] buffer pointer to an output buffer. 78 | * @param[in,out] b_len the size of buf; on success, holds bytes written. 79 | * @return 0 on success, non-zero on error 80 | **/ 81 | int mrpdp_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 82 | void *buffer, unsigned int* b_len) 83 | { 84 | #ifndef _S3_SUPPORT 85 | PDP_UNSUPPORTED("S3"); 86 | return -1; 87 | #else 88 | pdp_mrpdp_tag_t **tag_ptr = (pdp_mrpdp_tag_t **) buffer; 89 | pdp_mrpdp_tag_t *tag = NULL; 90 | unsigned char *buf = NULL; 91 | unsigned int i = 0; 92 | unsigned int buf_len = 0; 93 | unsigned int buf_size = 0; 94 | int status = -1; 95 | 96 | if (!ctx || !tag_ptr) 97 | return -1; 98 | 99 | // Allocate space for the tag 100 | if ((tag = mrpdp_tag_new()) == NULL) goto cleanup; 101 | if (*tag_ptr != NULL) { 102 | // there is already a tag here, so free it 103 | mrpdp_tag_free(*tag_ptr); 104 | } 105 | *tag_ptr = tag; 106 | 107 | buf_size = mrpdp_serialized_tag_size(ctx); 108 | if ((buf = malloc(buf_size)) == NULL) goto cleanup; 109 | memset(buf, 0, buf_size); 110 | i = index * buf_size; 111 | buf_len = buf_size; 112 | if (pdp_get_chunk_from_s3(ctx, ctx->ofilepath, &buf, &buf_len, i) != 0) 113 | goto cleanup; 114 | if (buf_len != buf_size) 115 | goto cleanup; 116 | if (mrpdp_deserialize_tag(ctx, tag, buf, buf_size) != 0) 117 | goto cleanup; 118 | status = 0; 119 | 120 | #ifdef _PDP_DEBUG 121 | DEBUG(1, "\n Reading - "); 122 | DEBUG(1, "\n Tag %02d", index); 123 | DEBUG(1, "\n Tim byte len [%02d]", BN_num_bytes(tag->Tim)); 124 | DEBUG(1, "\n Tim (hex) [%s]", BN_bn2hex(tag->Tim)); 125 | DEBUG(1, "\n index [%d]", tag->index); 126 | DEBUG(1, "\n prf_size [%d]", tag->index_prf_size); 127 | pdp_hexdump(" prf", index, tag->index_prf, tag->index_prf_size); 128 | pdp_hexdump(" Ser. tag", index, buf, buf_size); 129 | #endif // _PDP_DEBUG 130 | 131 | cleanup: 132 | sfree(buf, buf_len); 133 | if (status) { 134 | mrpdp_tag_free(tag); 135 | if (tag_ptr && *tag_ptr) *tag_ptr = NULL; 136 | } 137 | return status; 138 | #endif // _S3_SUPPORT 139 | } 140 | 141 | 142 | /** 143 | * @brief Implements the interface of get_block_stub() 144 | * 145 | * To be used with A-PDP and S3 storage back-end. 146 | * 147 | * @param[in] ctx pointer to a context structure. 148 | * @param[in] index the index of tag. 149 | * @param[out] b pointer to an output buffer. 150 | * @param[in,out] len the size of buf; on success, holds bytes written. 151 | * @return 0 on success, non-zero on error 152 | **/ 153 | int mrpdp_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 154 | void *b, unsigned int* len) 155 | { 156 | #ifndef _S3_SUPPORT 157 | PDP_UNSUPPORTED("S3"); 158 | return -1; 159 | #else 160 | unsigned int i = index * ctx->mrpdp_param->block_size; 161 | unsigned char *buf = (unsigned char *) b; 162 | return pdp_get_chunk_from_s3(ctx, ctx->filepath, &buf, len, i); 163 | #endif // _S3_SUPPORT 164 | } 165 | 166 | /** @} */ 167 | -------------------------------------------------------------------------------- /libpdp/src/mrpdp/mrpdp_storage.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for MAC-PDP module backend storage logic. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup MRPDP 13 | * @{ 14 | */ 15 | #ifndef __MR_PDP_STORAGE_H__ 16 | #define __MR_PDP_STORAGE_H__ 17 | 18 | /* 19 | * function prototypes - mrpdp_file.c 20 | */ 21 | int mrpdp_write_tags_to_file(const pdp_ctx_t *ctx, 22 | const pdp_mrpdp_tagdata_t* t); 23 | int mrpdp_get_tag_file(const pdp_ctx_t* ctx, unsigned int i, 24 | void *b, unsigned int *len); 25 | int mrpdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 26 | void *b, unsigned int *len); 27 | /* 28 | * function prototypes - mrpdp_s3.c 29 | */ 30 | int mrpdp_write_data_to_s3(const pdp_ctx_t *ctx, const pdp_mrpdp_tagdata_t* t); 31 | int mrpdp_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 32 | void *b, unsigned int* buf_len); 33 | int mrpdp_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 34 | void *b, unsigned int* len); 35 | 36 | #endif 37 | /** @} */ 38 | -------------------------------------------------------------------------------- /libpdp/src/pdp_file.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Interfaces for the libpdp library 3 | * 4 | * \author Copyright (c) 2012, Mark Gondree 5 | * \author Copyright (c) 2012, Alric Althoff 6 | * \author Copyright (c) 2008, Zachary N J Peterson 7 | * \date 2008-2013 8 | * \copyright BSD 2-Clause License 9 | * See http://opensource.org/licenses/BSD-2-Clause 10 | */ 11 | /** @addtogroup STORAGE 12 | * @{ 13 | */ 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #ifdef _S3_SUPPORT 21 | #include 22 | #include 23 | #endif 24 | #include 25 | #include "pdp_storage.h" 26 | #include "pdp_misc.h" 27 | 28 | 29 | /** 30 | * @brief Gets a block from a file 31 | * 32 | * @param[in] ctx pointer to a context structure. 33 | * @param[in] i the index of block. 34 | * @param[in] block_size block size 35 | * @param[out] buf pointer to an output buffer. 36 | * @param[in,out] len initially the size of buf, output bytes written. 37 | * @return 0 on success, non-zero on error 38 | **/ 39 | int pdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 40 | unsigned int block_size, unsigned char *buf, 41 | unsigned int *len) 42 | { 43 | static FILE* file = NULL; 44 | unsigned int pos = 0; 45 | 46 | // if buf is NULL, its a signal to re-set the stateful variables 47 | if (!buf && file) { 48 | fclose(file); 49 | file = NULL; 50 | return 0; 51 | } 52 | 53 | if (!ctx || !block_size || !buf || !len || !*len) 54 | return -1; 55 | 56 | if (*len % block_size) { 57 | PDP_ERR("warning: reading a partial block (buffer is " 58 | "%d bytes; tag is %d bytes)", *len, block_size); 59 | } 60 | memset(buf, 0, *len); 61 | 62 | // if file is not open already, open it 63 | if (!file && ((file = fopen(ctx->filepath, "r")) == NULL)) { 64 | PDP_ERR("unable to open %s: %d", ctx->filepath, ferror(file)); 65 | return -1; 66 | } 67 | 68 | // seek to correct place in file and read out the requested tag 69 | pos = i * block_size; 70 | if (fseek(file, pos, SEEK_SET) != 0) { 71 | PDP_ERR("fseek error %d", errno); 72 | return -1; 73 | } 74 | *len = fread(buf, 1, *len, file); 75 | if (ferror(file)) return -1; 76 | 77 | return 0; 78 | } 79 | 80 | /** @} */ 81 | -------------------------------------------------------------------------------- /libpdp/src/pdp_misc.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Support routines for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | * 12 | * @defgroup MISC Support Routines 13 | * \brief Generic routines for keys and crypto primitives. 14 | **/ 15 | #ifndef __PDP_MISC_H__ 16 | #define __PDP_MISC_H__ 17 | 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | 24 | /** 25 | * @brief A macro for printing errors to stderr. 26 | **/ 27 | #define PDP_ERR(...) \ 28 | do { \ 29 | fprintf(stderr, "\n%s: ", __FUNCTION__); \ 30 | fprintf(stderr, __VA_ARGS__); \ 31 | fflush(stderr); \ 32 | } while(0) 33 | 34 | 35 | /** 36 | * @brief A macro for announcing a feature is not supported. 37 | **/ 38 | #define PDP_UNSUPPORTED(x) \ 39 | do { \ 40 | PDP_ERR("%s was not build with %s support", LIBPDP_NAME, x); \ 41 | } while(0) 42 | 43 | 44 | /** 45 | * @brief A macro for debugging. 46 | **/ 47 | #ifdef _PDP_DEBUG 48 | #define DEBUG(x, ...) \ 49 | do { if (x) { \ 50 | fprintf(stderr, __VA_ARGS__); \ 51 | fflush(stderr); \ 52 | } } while(0) 53 | #else 54 | #define DEBUG(x, ...) while(0) {} 55 | #endif 56 | 57 | 58 | /** 59 | * @brief A generic "safe" free routine. 60 | **/ 61 | #define sfree(ptr, len) \ 62 | do { \ 63 | if (ptr) { \ 64 | if (len) memset(ptr, 0, len); \ 65 | free(ptr); \ 66 | } \ 67 | ptr = NULL; \ 68 | } while(0) 69 | 70 | 71 | /** 72 | * @brief Returns 1 if the context is appropriate for macpdp. 73 | **/ 74 | #define is_macpdp(ctx) \ 75 | ((!ctx || (ctx->algo != PDP_MACPDP) || !ctx->macpdp_param) ? 0 : 1) 76 | 77 | /** 78 | * @brief Returns 1 if the context is appropriate for apdp. 79 | **/ 80 | #define is_apdp(ctx) \ 81 | ((!ctx || (ctx->algo != PDP_APDP) || !ctx->apdp_param) ? 0 : 1) 82 | 83 | /** 84 | * @brief Returns 1 if the context is appropriate for mrpdp. 85 | **/ 86 | #define is_mrpdp(ctx) \ 87 | ((!ctx || (ctx->algo != PDP_MRPDP) || !ctx->mrpdp_param) ? 0 : 1) 88 | 89 | /** 90 | * @brief Returns 1 if the context is appropriate for cpor. 91 | **/ 92 | #define is_cpor(ctx) \ 93 | ((!ctx || (ctx->algo != PDP_CPOR) || !ctx->cpor_param) ? 0 : 1) 94 | 95 | /** 96 | * @brief Returns 1 if the context is appropriate for sepdp. 97 | **/ 98 | #define is_sepdp(ctx) \ 99 | ((!ctx || (ctx->algo != PDP_SEPDP) || !ctx->sepdp_param) ? 0 : 1) 100 | 101 | 102 | /* 103 | * function prototypes - debug, in pdp_misc.c 104 | */ 105 | #ifdef _PDP_DEBUG 106 | void pdp_hexdump(const char* msg, unsigned int index, 107 | const unsigned char *buf, size_t len); 108 | #endif // _PDP_DEBUG 109 | 110 | 111 | /* 112 | * function prototypes - in pdp_misc.c 113 | */ 114 | unsigned int next_pow_2(int x); 115 | unsigned int prev_pow_2(unsigned int x); 116 | int sample_sans_replacement(unsigned int samp_sz, unsigned int pop_sz, 117 | unsigned int S[]); 118 | int sample_prp_sans_replacement(const unsigned char *ki, unsigned int ki_size, 119 | unsigned int samp_sz, unsigned int pop_sz, unsigned int S[]); 120 | BIGNUM *gen_fdh(const RSA *key, const unsigned char *input, size_t len); 121 | unsigned char *gen_bn_H(const BIGNUM *input, size_t *len); 122 | unsigned char *gen_ar_H(unsigned char *nonce, size_t nonce_len, 123 | unsigned char **vals, unsigned int vals_num, 124 | unsigned int val_len, size_t *len); 125 | unsigned char *gen_prf_f(const unsigned char *key, size_t key_size, 126 | unsigned int index, size_t *len); 127 | BIGNUM *gen_prf_k(unsigned char *key, size_t key_len, unsigned int index); 128 | unsigned int *gen_prp_pi(const unsigned char *key, unsigned int key_len, 129 | unsigned int total, unsigned int c); 130 | 131 | /* 132 | * function prototypes - in pdp_key.c 133 | */ 134 | int pdp_read_password(const char *prompt, unsigned char *passwd, size_t len); 135 | int pdp_get_passphrase(const pdp_ctx_t *ctx, char *passwd, size_t len); 136 | int get_key_paths(char *pri_path, size_t pri_len, 137 | char *pub_path, size_t pub_len, 138 | const char *kpath, const char *name); 139 | int PBKDF2(unsigned char **key, size_t dkey_len, 140 | unsigned char *pw, size_t pw_len, 141 | unsigned char *salt, size_t salt_len, unsigned int c); 142 | int pdp_key_wrap(unsigned char** enc, size_t *enc_len, 143 | unsigned char *key, size_t key_len, 144 | unsigned char *kek, size_t kek_len); 145 | int pdp_key_unwrap(unsigned char **key, size_t *key_len, 146 | unsigned char *ekey, size_t ekey_len, 147 | unsigned char *kek, size_t kek_len); 148 | int encrypt_then_mac(const unsigned char *ekey, size_t ekey_len, 149 | const unsigned char *mkey, size_t mkey_len, 150 | const unsigned char *input, size_t input_len, 151 | unsigned char *ctxt, size_t *ctxt_len, 152 | unsigned char *mac, size_t *mac_len, 153 | unsigned char *iv, size_t iv_len); 154 | int verify_then_decrypt(const unsigned char *ekey, size_t ekey_len, 155 | const unsigned char *mkey, size_t mkey_len, 156 | const unsigned char *ctxt, size_t ctxt_len, 157 | const unsigned char *mac, size_t mac_len, 158 | const unsigned char *iv, size_t iv_len, 159 | unsigned char *output, size_t *output_len); 160 | 161 | #endif 162 | -------------------------------------------------------------------------------- /libpdp/src/pdp_storage.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Support routines for interfacing libpdp with libs3. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | * 12 | * @defgroup STORAGE PDP Storage 13 | * \brief Interfaces for using various storage back-ends. 14 | **/ 15 | #ifndef __PDP_S3_H__ 16 | #define __PDP_S3_H__ 17 | 18 | #include 19 | 20 | /* 21 | * function prototypes - in pdp_file.c 22 | */ 23 | int pdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 24 | unsigned int block_size, unsigned char *buf, 25 | unsigned int *len); 26 | 27 | 28 | /* 29 | * function prototypes - in pdp_s3.c 30 | */ 31 | int pdp_write_file_to_s3(const pdp_ctx_t *ctx, const char* filepath); 32 | int pdp_write_data_to_s3(const pdp_ctx_t *ctx, const char* filepath, 33 | unsigned char *data, unsigned int data_len); 34 | int pdp_get_chunk_from_s3(const pdp_ctx_t *ctx, const char *filepath, 35 | unsigned char **buf, unsigned int *buf_len, unsigned int index); 36 | 37 | #endif 38 | /** @} */ 39 | -------------------------------------------------------------------------------- /libpdp/src/sepdp/sepdp_file.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the MAC-PDP module for flat-file storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup SEPDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #ifdef _THREAD_SUPPORT 21 | #include 22 | #endif 23 | #include 24 | #include 25 | #include "pdp_misc.h" 26 | #include "pdp_storage.h" 27 | #include "sepdp_storage.h" 28 | #include "sepdp_misc.h" 29 | 30 | 31 | /** 32 | * @brief Write tag data out to a file. 33 | * @param[in] ctx pointer to a context structure. 34 | * @param[in] t pointer to tag data. 35 | * @return 0 on success, non-zero on error 36 | **/ 37 | int sepdp_write_tags_to_file(const pdp_ctx_t *ctx, const pdp_sepdp_tagdata_t* t) 38 | { 39 | unsigned char *buf = NULL; 40 | FILE *tagfile = NULL; 41 | unsigned int len = 0; 42 | size_t off = 0; 43 | int status = -1; 44 | 45 | if (!ctx || !t || !t->tokens || !t->tokens_num || !ctx->ofilepath) 46 | return -1; 47 | 48 | if ((access(ctx->ofilepath, F_OK) == 0) && (ctx->verbose)) { 49 | PDP_ERR("WARNING: overwriting [%s]", ctx->ofilepath); 50 | } 51 | if ((tagfile = fopen(ctx->ofilepath, "w")) == NULL){ 52 | PDP_ERR("%d", ferror(tagfile)); 53 | PDP_ERR("unable to create %s", ctx->ofilepath); 54 | goto cleanup; 55 | } 56 | 57 | if ((sepdp_serialize_tags(ctx, t, &buf, &len)) != 0) goto cleanup; 58 | do { 59 | off += fwrite(buf + off, 1, len, tagfile); 60 | if (ferror(tagfile)) goto cleanup; 61 | 62 | } while (off < len); 63 | status = 0; 64 | 65 | cleanup: 66 | sfree(buf, len); 67 | if (tagfile) fclose(tagfile); 68 | return status; 69 | } 70 | 71 | 72 | /** 73 | * @brief Implements the interface of pdp_get_tag_noop 74 | * 75 | * To be used with MAC-PDP and a flat-file storage back-end. 76 | * 77 | * @todo use mmap to speed up access? 78 | * 79 | * @param[in] ctx pointer to a context structure. 80 | * @param[in] index the index of tag. 81 | * @param[out] buf pointer to an output buffer. 82 | * @param[in,out] len initially the size of buf, output bytes written. 83 | * @return 0 on success, non-zero on error 84 | **/ 85 | int sepdp_get_tag_file(const pdp_ctx_t* ctx, unsigned int index, 86 | void *buf, unsigned int *len) 87 | { 88 | pdp_sepdp_tag_t **tag_ptr = (pdp_sepdp_tag_t **) buf; 89 | pdp_sepdp_tag_t *tag = NULL; 90 | static FILE* tagfile = NULL; 91 | unsigned char *data = NULL; 92 | unsigned int data_len = 0; 93 | int status = -1; 94 | 95 | // If buf is NULL, it is a signal to re-set the stateful variables 96 | if (!buf && tagfile) { 97 | fclose(tagfile); 98 | tagfile = NULL; 99 | return 0; 100 | } 101 | 102 | if (!ctx || !tag_ptr) 103 | return -1; 104 | 105 | // if file is not open already, open it 106 | if (!tagfile && ((tagfile = fopen(ctx->ofilepath, "r")) == NULL)) { 107 | PDP_ERR("%d", ferror(tagfile)); 108 | PDP_ERR("unable to open %s", ctx->ofilepath); 109 | return -1; 110 | } 111 | 112 | // Allocate space for the tag 113 | if ((tag = malloc(sizeof(pdp_sepdp_tag_t))) == NULL) goto cleanup; 114 | memset(tag, 0, sizeof(pdp_sepdp_tag_t)); 115 | if (*tag_ptr != NULL) { 116 | // there is already a tag here, so free it 117 | sepdp_tag_free(ctx, *tag_ptr); 118 | } 119 | *tag_ptr = tag; 120 | 121 | data_len = sepdp_serialized_tag_size(ctx); 122 | if ((data = malloc(data_len)) == NULL) goto cleanup; 123 | memset(data, 0, data_len); 124 | 125 | // We seek directly to the index of the i-th tag 126 | if (fseek(tagfile, index * data_len, SEEK_SET) < 0) goto cleanup; 127 | fread(data, 1, data_len, tagfile); 128 | if (ferror(tagfile)) goto cleanup; 129 | 130 | // de-serialize the data block 131 | if (sepdp_deserialize_tag(ctx, tag, data, data_len) != 0) 132 | goto cleanup; 133 | 134 | status = 0; 135 | 136 | cleanup: 137 | sfree(data, data_len); 138 | if (status && tag_ptr && *tag_ptr) { 139 | sepdp_tag_free(ctx, *tag_ptr); 140 | *tag_ptr = NULL; 141 | } 142 | return status; 143 | } 144 | 145 | 146 | /** 147 | * @brief Implements the interface of pdp_get_block_noop 148 | * 149 | * To be used with SEPDP and a flat-file storage back-end. 150 | * 151 | * @param[in] ctx pointer to a context structure. 152 | * @param[in] i the index of block. 153 | * @param[out] b pointer to an output buffer. 154 | * @param[in,out] len initially the size of buf, output bytes written. 155 | * @return 0 on success, non-zero on error 156 | **/ 157 | int sepdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 158 | void *b, unsigned int *len) 159 | { 160 | if (!is_sepdp(ctx)) return -1; 161 | return pdp_get_block_file(ctx, i, ctx->sepdp_param->block_size, 162 | (unsigned char *) b, len); 163 | } 164 | 165 | /** @} */ 166 | -------------------------------------------------------------------------------- /libpdp/src/sepdp/sepdp_key.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the MAC-PDP module for libpdp. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup SEPDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include "pdp_misc.h" 23 | 24 | 25 | /** 26 | * @brief Generate key material. 27 | * 28 | * Our MAC is HMAC, so this creates a random PRF key. 29 | * 30 | * @return 0 on success, non-zero on error 31 | **/ 32 | int sepdp_key_gen(const pdp_ctx_t *ctx, pdp_key_t *k) 33 | { 34 | pdp_sepdp_ctx_t *p = NULL; 35 | pdp_sepdp_key_t *key = NULL; 36 | 37 | if (!is_sepdp(ctx) || !k) 38 | return -1; 39 | p = ctx->sepdp_param; 40 | 41 | if (!p->prf_key_size) 42 | return -1; 43 | 44 | if ((key = malloc(sizeof(pdp_sepdp_key_t))) == NULL) 45 | return -1; 46 | memset(key, 0, sizeof(pdp_sepdp_key_t)); 47 | k->sepdp = key; 48 | 49 | key->W_size = p->prf_key_size; 50 | key->Z_size = p->prf_key_size; 51 | key->K_size = p->ae_key_size; 52 | if ((key->W = malloc(key->W_size)) == NULL) goto cleanup; 53 | if ((key->Z = malloc(key->Z_size)) == NULL) goto cleanup; 54 | if ((key->K = malloc(key->K_size)) == NULL) goto cleanup; 55 | if (!RAND_bytes(key->W, key->W_size)) goto cleanup; 56 | if (!RAND_bytes(key->Z, key->Z_size)) goto cleanup; 57 | if (!RAND_bytes(key->K, key->K_size)) goto cleanup; 58 | return 0; 59 | 60 | cleanup: 61 | sepdp_key_free(ctx, k); 62 | return -1; 63 | } 64 | 65 | 66 | /** 67 | * @brief Destroy and free allocated key material. 68 | * @return 0 on success, non-zero on error 69 | **/ 70 | int sepdp_key_free(const pdp_ctx_t *ctx, pdp_key_t *k) 71 | { 72 | pdp_sepdp_key_t *key = NULL; 73 | 74 | if (!is_sepdp(ctx) || !k || !k->sepdp) 75 | return -1; 76 | key = k->sepdp; 77 | 78 | sfree(key->W, key->W_size); 79 | sfree(key->Z, key->Z_size); 80 | sfree(key->K, key->K_size); 81 | sfree(key, sizeof(pdp_sepdp_key_t)); 82 | k->sepdp = NULL; 83 | return 0; 84 | } 85 | 86 | /** @} */ 87 | -------------------------------------------------------------------------------- /libpdp/src/sepdp/sepdp_misc.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for A-PDP module backend storage logic. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup SEPDP 13 | * @{ 14 | */ 15 | #ifndef __SEPDP_MISC_H__ 16 | #define __SEPDP_MISC_H__ 17 | 18 | 19 | /* 20 | * function prototypes - sepdp_serialize.c 21 | */ 22 | unsigned int sepdp_serialized_tag_size(const pdp_ctx_t *ctx); 23 | int sepdp_serialize_tags(const pdp_ctx_t *ctx, const pdp_sepdp_tagdata_t* t, 24 | unsigned char **buffer, unsigned int *buffer_len); 25 | int sepdp_deserialize_tag(const pdp_ctx_t *ctx, pdp_sepdp_tag_t* tag, 26 | unsigned char *buf, unsigned int buf_len); 27 | 28 | 29 | /* 30 | * function prototypes - sepdp.c 31 | */ 32 | void sepdp_tag_free(const pdp_ctx_t *ctx, pdp_sepdp_tag_t *token); 33 | 34 | 35 | #endif 36 | /** @} */ 37 | -------------------------------------------------------------------------------- /libpdp/src/sepdp/sepdp_s3.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the MAC-PDP module for S3 storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup SEPDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #ifdef _S3_SUPPORT 21 | #include 22 | #include 23 | #endif 24 | #include 25 | #include 26 | #include "pdp_misc.h" 27 | #include "pdp_storage.h" 28 | #include "sepdp_storage.h" 29 | #include "sepdp_misc.h" 30 | 31 | /** 32 | * @brief Write out file and tag data to S3. 33 | **/ 34 | int sepdp_write_data_to_s3(const pdp_ctx_t *ctx, 35 | const pdp_sepdp_tagdata_t* t) 36 | { 37 | #ifndef _S3_SUPPORT 38 | PDP_UNSUPPORTED("S3"); 39 | return -1; 40 | #else 41 | unsigned char *buf = NULL; 42 | unsigned int len = 0; 43 | int status = -1; 44 | 45 | if (!t || !t->tokens) return -1; 46 | 47 | // write file to s3 48 | if (pdp_write_file_to_s3(ctx, ctx->filepath)) { 49 | PDP_ERR("error writing original file to s3"); 50 | return -1; 51 | } 52 | // write tags to s3 53 | if ((sepdp_serialize_tags(ctx, t, &buf, &len)) != 0) goto cleanup; 54 | if (pdp_write_data_to_s3(ctx, ctx->ofilepath, buf, len)) { 55 | PDP_ERR("error writing tag data to s3"); 56 | goto cleanup; 57 | } 58 | status = 0; 59 | 60 | cleanup: 61 | sfree(buf, len); 62 | return status; 63 | #endif // _S3_SUPPORT 64 | } 65 | 66 | 67 | /** 68 | * @brief Implements the interface of pdp_get_tag_noop 69 | * 70 | * To be used with SEPDP and S3 storage back-end. 71 | * 72 | * @param[in] ctx pointer to a context structure. 73 | * @param[in] index the index of tag. 74 | * @param[out] buffer pointer to an output buffer. 75 | * @param[in,out] len not used (output buffer is a pdp_sepdp_tag_t) 76 | * @return 0 on success, non-zero on error 77 | **/ 78 | int sepdp_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 79 | void *buffer, unsigned int *len) 80 | { 81 | #ifndef _S3_SUPPORT 82 | PDP_UNSUPPORTED("S3"); 83 | return -1; 84 | #else 85 | pdp_sepdp_tag_t **tag_ptr = (pdp_sepdp_tag_t **) buffer; 86 | pdp_sepdp_tag_t *tag = NULL; 87 | unsigned char *buf = NULL; 88 | unsigned int i = 0; 89 | unsigned int buf_len = 0; 90 | unsigned int buf_size = 0; 91 | int status = -1; 92 | 93 | if (!ctx || !tag_ptr) 94 | return -1; 95 | 96 | // Allocate space for the tag 97 | if ((tag = malloc(sizeof(pdp_sepdp_tag_t))) == NULL) goto cleanup; 98 | if (*tag_ptr != NULL) { 99 | // there is already a tag here, so free it 100 | sepdp_tag_free(ctx, *tag_ptr); 101 | } 102 | *tag_ptr = tag; 103 | 104 | buf_size = sepdp_serialized_tag_size(ctx); 105 | if ((buf = malloc(buf_size)) == NULL) goto cleanup; 106 | memset(buf, 0, buf_size); 107 | i = index * buf_size; 108 | buf_len = buf_size; 109 | if (pdp_get_chunk_from_s3(ctx, ctx->ofilepath, &buf, &buf_len, i) != 0) 110 | goto cleanup; 111 | if (buf_len != buf_size) 112 | goto cleanup; 113 | if (sepdp_deserialize_tag(ctx, tag, buf, buf_size) != 0) 114 | goto cleanup; 115 | status = 0; 116 | 117 | cleanup: 118 | sfree(buf, buf_len); 119 | if (status) { 120 | sepdp_tag_free(ctx, tag); 121 | if (tag_ptr && *tag_ptr) *tag_ptr = NULL; 122 | } 123 | return status; 124 | #endif // _S3_SUPPORT 125 | 126 | } 127 | 128 | 129 | /** 130 | * @brief Implements the interface of pdp_get_block_noop 131 | * 132 | * To be used with SEPDP and S3 storage back-end. 133 | * 134 | * @param[in] ctx pointer to a context structure. 135 | * @param[in] index the index of tag. 136 | * @param[out] b pointer to an output buffer. 137 | * @param[in,out] len the size of buf; on success, holds bytes written. 138 | * @return 0 on success, non-zero on error 139 | **/ 140 | int sepdp_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 141 | void *b, unsigned int *len) 142 | { 143 | #ifndef _S3_SUPPORT 144 | PDP_UNSUPPORTED("S3"); 145 | return -1; 146 | #else 147 | unsigned int i = index * ctx->sepdp_param->block_size; 148 | unsigned char *buf = (unsigned char *) b; 149 | return pdp_get_chunk_from_s3(ctx, ctx->filepath, &buf, len, i); 150 | #endif // _S3_SUPPORT 151 | } 152 | 153 | /** @} */ 154 | -------------------------------------------------------------------------------- /libpdp/src/sepdp/sepdp_serialize.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Implementation of the A-PDP module for S3 storage. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup SEPDP 13 | * @{ 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include "sepdp_misc.h" 24 | #include "pdp_misc.h" 25 | 26 | /** 27 | * @brief Returns the upper bound of the serialized token 28 | * 29 | * Each token is of the form: 30 | * 31 | **/ 32 | unsigned int sepdp_serialized_tag_size(const pdp_ctx_t *ctx) 33 | { 34 | pdp_sepdp_ctx_t *p = NULL; 35 | 36 | if (!is_sepdp(ctx)) return -1; 37 | p = ctx->sepdp_param; 38 | 39 | return ( sizeof(size_t) + 40 | sizeof(size_t) + p->ae_key_size + 41 | sizeof(size_t) + p->block_size + 42 | sizeof(size_t) + EVP_MAX_MD_SIZE ); 43 | } 44 | 45 | 46 | /** 47 | * @brief Write out tag data to a buffer 48 | * 49 | * Serializes a set of tokens into an array of length ell 50 | * @verbatim 51 | (0) (ell-1) 52 | | token 0 | token 1 | ... | token n-1 | 53 | | | | | | 54 | |<--- s --->|<--- s --->| |<--- s --->| 55 | where n = t->tokens_num 56 | s = upper bound on token length 57 | ell = n * s 58 | @endverbatim 59 | * 60 | * @todo This serialization is not portable. 61 | * @param[in] ctx context 62 | * @param[in] t pointer to input tag data 63 | * @param[in] buffer the buffer for the serialized data 64 | * @param[in] buffer_len length of space available, output bytes written 65 | * @return 0 on success, non-zero on error 66 | **/ 67 | int sepdp_serialize_tags(const pdp_ctx_t *ctx, const pdp_sepdp_tagdata_t* t, 68 | unsigned char **buffer, unsigned int *buffer_len) 69 | { 70 | pdp_sepdp_ctx_t *p = NULL; 71 | pdp_sepdp_tag_t *tag = NULL; 72 | unsigned char *buf_ptr = NULL; 73 | unsigned char *buf = NULL; 74 | size_t tag_size, buf_len; 75 | int i, status = -1; 76 | 77 | if (!is_sepdp(ctx) || !t || !buffer || !buffer_len || 78 | !t->tokens || !t->tokens_num) 79 | return -1; 80 | p = ctx->sepdp_param; 81 | 82 | // some useful constants: upper bounds on sizes of things 83 | tag_size = sepdp_serialized_tag_size(ctx); 84 | buf_len = t->tokens_num * tag_size; 85 | 86 | // allocate buffers 87 | if ((buf = malloc(buf_len)) == NULL) goto cleanup; 88 | memset(buf, 0, buf_len); 89 | buf_ptr = buf; 90 | 91 | for(i = 0; i < t->tokens_num; i++) { 92 | tag = t->tokens[i]; 93 | 94 | // write index 95 | memcpy(buf_ptr, &(tag->index), sizeof(tag->index)); 96 | buf_ptr += sizeof(tag->index); 97 | 98 | // make sure our assumptions re: bounds are correct 99 | if (tag->iv_size > p->ae_key_size) goto cleanup; 100 | 101 | // write iv length 102 | memcpy(buf_ptr, &(tag->iv_size), sizeof(tag->iv_size)); 103 | buf_ptr += sizeof(tag->iv_size); 104 | 105 | // write iv 106 | memset(buf_ptr, 0, p->ae_key_size); 107 | memcpy(buf_ptr, tag->iv, tag->iv_size); 108 | buf_ptr += p->ae_key_size; 109 | 110 | // make sure our assumptions re: bounds are correct 111 | if (tag->tok_size > p->block_size) goto cleanup; 112 | 113 | // write token length 114 | memcpy(buf_ptr, &(tag->tok_size), sizeof(tag->tok_size)); 115 | buf_ptr += sizeof(tag->tok_size); 116 | 117 | // write token 118 | memset(buf_ptr, 0, p->block_size); 119 | memcpy(buf_ptr, tag->tok, tag->tok_size); 120 | buf_ptr += p->block_size; 121 | 122 | // make sure our assumptions re: bounds are correct 123 | if (tag->mac_size > EVP_MAX_MD_SIZE) goto cleanup; 124 | 125 | // write token length 126 | memcpy(buf_ptr, &(tag->mac_size), sizeof(tag->mac_size)); 127 | buf_ptr += sizeof(tag->mac_size); 128 | 129 | // write mac 130 | memset(buf_ptr, 0, EVP_MAX_MD_SIZE); 131 | memcpy(buf_ptr, tag->mac, tag->mac_size); 132 | buf_ptr += EVP_MAX_MD_SIZE; 133 | } 134 | *buffer = buf; 135 | *buffer_len = buf_len; 136 | status = 0; 137 | 138 | cleanup: 139 | if (status) sfree(buf, buf_len); 140 | return status; 141 | } 142 | 143 | 144 | /** 145 | * @brief Read an individual tag 146 | * @param[in] ctx context 147 | * @param[out] tag pointer to tag that will be populated 148 | * @param[in] buf serialized structure to process 149 | * @param[in] buf_len length of data to process 150 | * @return 0 on success, non-zero on error 151 | **/ 152 | int sepdp_deserialize_tag(const pdp_ctx_t *ctx, pdp_sepdp_tag_t* tag, 153 | unsigned char *buf, unsigned int buf_len) 154 | { 155 | pdp_sepdp_ctx_t *p = NULL; 156 | unsigned char *buf_ptr = buf; 157 | size_t tag_size; 158 | int status = -1; 159 | 160 | if (!is_sepdp(ctx) || !tag || !buf || !buf_len) 161 | return -1; 162 | p = ctx->sepdp_param; 163 | 164 | // some useful constants: upper bounds on sizes of things 165 | tag_size = sepdp_serialized_tag_size(ctx); 166 | 167 | // double-check size of buf 168 | if (buf_len < tag_size) goto cleanup; 169 | 170 | memcpy(&(tag->index), buf_ptr, sizeof(tag->index)); 171 | buf_ptr += sizeof(tag->index); 172 | 173 | memcpy(&(tag->iv_size), buf_ptr, sizeof(tag->iv_size)); 174 | buf_ptr += sizeof(tag->iv_size); 175 | 176 | // make sure our assumptions re: bounds are correct 177 | if (tag->iv_size > p->ae_key_size) goto cleanup; 178 | 179 | if ((tag->iv = malloc(tag->iv_size)) == NULL) goto cleanup; 180 | memcpy(tag->iv, buf_ptr, tag->iv_size); 181 | buf_ptr += p->ae_key_size; 182 | 183 | memcpy(&(tag->tok_size), buf_ptr, sizeof(tag->tok_size)); 184 | buf_ptr += sizeof(tag->tok_size); 185 | 186 | // make sure our assumptions re: bounds are correct 187 | if (tag->tok_size > p->block_size) goto cleanup; 188 | 189 | if ((tag->tok = malloc(tag->tok_size)) == NULL) goto cleanup; 190 | memcpy(tag->tok, buf_ptr, tag->tok_size); 191 | buf_ptr += p->block_size; 192 | 193 | memcpy(&(tag->mac_size), buf_ptr, sizeof(tag->mac_size)); 194 | buf_ptr += sizeof(tag->mac_size); 195 | 196 | // make sure our assumptions re: bounds are correct 197 | if (tag->mac_size > EVP_MAX_MD_SIZE) goto cleanup; 198 | 199 | if ((tag->mac = malloc(tag->mac_size)) == NULL) goto cleanup; 200 | memcpy(tag->mac, buf_ptr, tag->mac_size); 201 | buf_ptr += EVP_MAX_MD_SIZE; 202 | 203 | status = 0; 204 | 205 | cleanup: 206 | return status; 207 | } 208 | 209 | /** @} */ 210 | -------------------------------------------------------------------------------- /libpdp/src/sepdp/sepdp_storage.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * Interfaces for MAC-PDP module backend storage logic. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @author Copyright (c) 2008, Zachary N J Peterson 8 | * @date 2008-2013 9 | * @copyright BSD 2-Clause License, 10 | * See http://opensource.org/licenses/BSD-2-Clause 11 | **/ 12 | /** @addtogroup SEPDP 13 | * @{ 14 | */ 15 | #ifndef __SEPDP_STORAGE_H__ 16 | #define __SEPDP_STORAGE_H__ 17 | 18 | /* 19 | * function prototypes - sepdp_file.c 20 | */ 21 | int sepdp_write_tags_to_file(const pdp_ctx_t *ctx, 22 | const pdp_sepdp_tagdata_t* t); 23 | int sepdp_get_tag_file(const pdp_ctx_t* ctx, unsigned int i, 24 | void *b, unsigned int *len); 25 | int sepdp_get_block_file(const pdp_ctx_t* ctx, unsigned int i, 26 | void *b, unsigned int *len); 27 | 28 | /* 29 | * function prototypes - sepdp_s3.c 30 | */ 31 | int sepdp_write_data_to_s3(const pdp_ctx_t *ctx, 32 | const pdp_sepdp_tagdata_t* t); 33 | int sepdp_get_tag_s3(const pdp_ctx_t* ctx, unsigned int index, 34 | void *buffer, unsigned int *len); 35 | int sepdp_get_block_s3(const pdp_ctx_t* ctx, unsigned int index, 36 | void *b, unsigned int *len); 37 | 38 | #endif 39 | /** @} */ 40 | -------------------------------------------------------------------------------- /tools/Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: all clean distclean 2 | 3 | VPATH += src 4 | 5 | CC = gcc 6 | LD = gcc 7 | AR = ar 8 | #----------------------------------------------------------------------------- 9 | 10 | # 11 | # libpdp 12 | # 13 | LIBPDP = ../libpdp/libpdp.a 14 | LIBPDP_INC = -I../libpdp/inc 15 | 16 | # 17 | # libtimeit 18 | # 19 | TIMEIT_LIB = ./lib/time_it/libtimeit.a 20 | TIMEIT_INC = -I./lib/time_it 21 | 22 | # 23 | # libs3 24 | # 25 | LIBS3 = ../libs3/lib/libs3.a 26 | LIBS3_TIMING = ./lib/libs3-2.0/build/lib/libs3.a 27 | S3_INC = -I../libs3/inc 28 | S3_LIB = $(LIBS3_TIMING) # timing instrumentation in libs3 29 | #S3_LIB = $(LIBS3) # no timing data 30 | 31 | #----------------------------------------------------------------------------- 32 | INCLUDES = $(LIBPDP_INC) $(TIMEIT_INC) $(S3_INC) 33 | 34 | CFLAGS = -Wall -g $(INCLUDES) 35 | CFLAGS += -D_S3_SUPPORT 36 | CFLAGS += -D_THREAD_SUPPORT 37 | CFLAGS += -D_TIMING_DATA 38 | 39 | LDFLAGS = -lcrypto -pthread -lcurl -lcrypto -lxml2 -lz 40 | LDFLAGS += $(LIBPDP) $(S3_LIB) 41 | 42 | all: pdp_bench 43 | 44 | clean: 45 | rm -rf *.o *.dSYM *.a 46 | 47 | distclean: clean 48 | rm -rf pdp_bench 49 | make -C ../libpdp distclean 50 | make -C ./lib/libs3-2.0 clean 51 | make -C ./lib/time_it clean 52 | 53 | %.o: %.c 54 | $(CC) -c $(CFLAGS) $< -o $@ 55 | 56 | pdp_bench: pdp_bench.o $(LIBPDP) $(TIMEIT_LIB) $(S3_LIB) 57 | $(LD) $^ $(LDFLAGS) -o $@ 58 | 59 | $(LIBPDP) $(TIMEIT_LIB): 60 | make -C $(dir $@) $(notdir $@) 61 | 62 | $(LIBS3_TIMING): 63 | make -C ./lib/libs3-2.0 all 64 | 65 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/.gitignore: -------------------------------------------------------------------------------- 1 | build 2 | build-debug 3 | 4 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/ChangeLog: -------------------------------------------------------------------------------- 1 | Thu Sep 18 10:03:02 NZST 2008 bryan@ischo.com 2 | * This file is no longer maintained, sorry 3 | 4 | Sat Aug 9 13:44:21 NZST 2008 bryan@ischo.com 5 | * Fixed bug wherein keys with non-URI-safe characters did not work 6 | correctly because they were not being URI-encoded in the request UR 7 | * Split RPM and DEB packages into normal and devel packages 8 | 9 | Fri Aug 8 22:40:19 NZST 2008 bryan@ischo.com 10 | * Branched 0.4 11 | * Created RPM and Debian packaging 12 | 13 | Tue Aug 5 08:52:33 NZST 2008 bryan@ischo.com 14 | * Bumped version number to 0.3 15 | * Moved Makefile to GNUmakefile, added shared library build 16 | * Added a bunch of GNU standard files (README, INSTALL, ChangeLog, etc) 17 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/INSTALL: -------------------------------------------------------------------------------- 1 | 2 | To install libs3 on a POSIX system (except Microsoft Windows): 3 | -------------------------------------------------------------- 4 | 5 | Note that all POSIX builds have prerequisites, such as development libraries 6 | that libs3 requires and that must be installed at the time that libs3 is 7 | built. The easiest way to find out what those are, is to run the build 8 | command and then observe the results. 9 | 10 | *** For RPM-based systems (Fedora Core, Mandrake, etc) *** 11 | 12 | * rpmbuild -ta 13 | 14 | for example: 15 | 16 | rpmbuild -ta libs3-0.3.tar.gz 17 | 18 | 19 | *** For dpkg-based systems (Debian, Ubuntu, etc) *** 20 | 21 | * make deb 22 | 23 | This will produce a Debian package in the build/pkg directory. 24 | 25 | 26 | *** For all other systems *** 27 | 28 | * make [DESTDIR=destination root] install 29 | 30 | DESTDIR defaults to /usr 31 | 32 | 33 | To install libs3 on a Microsoft Windows system: 34 | ----------------------------------------------- 35 | 36 | *** Using MingW *** 37 | 38 | * libs3 can be built on Windows using the MingW compiler. No other tool 39 | is needed. However, the following libraries are needed to build libs3: 40 | 41 | - curl development libraries 42 | - libxml2 development libraries, and the libraries that it requires: 43 | - iconv 44 | - zlib 45 | 46 | These projects are independent of libs3, and their release schedule and 47 | means of distribution would make it very difficult to provide links to 48 | the files to download and keep them up-to-date in this file, so no attempt 49 | is made here. 50 | 51 | Development libraries and other files can be placed in: 52 | c:\libs3-libs\bin 53 | c:\libs3-libs\include 54 | 55 | If the above locations are used, then the GNUmakefile.mingw will work with 56 | no special caveats. If the above locations are not used, then the following 57 | environment variables should be set: 58 | CURL_LIBS should be set to the MingW compiler flags needed to locate and 59 | link in the curl libraries 60 | CURL_CFLAGS should be set to the MingW compiler flags needed to locate and 61 | include the curl headers 62 | LIBXML2_LIBS should be set to the MingW compiler flags needed to locate and 63 | link in the libxml2 libraries 64 | LIBXML2_CFLAGS should be set to the MingW compiler flags needed to locate and 65 | include the libxml2 headers 66 | 67 | * mingw32-make [DESTDIR=destination] -f GNUmakefile.mingw install 68 | 69 | DESTDIR defaults to libs3- 70 | 71 | * DESTDIR can be zipped up into a .zip file for distribution. For best 72 | results, the dependent libraries (curl, openssl, etc) should be included, 73 | along with their licenses. 74 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2008 Bryan Ischo 2 | 3 | libs3 is free software: you can redistribute it and/or modify it under the 4 | terms of the GNU General Public License as published by the Free Software 5 | Foundation, version 3 of the License. 6 | 7 | In addition, as a special exception, the copyright holders give 8 | permission to link the code of this library and its programs with the 9 | OpenSSL library, and distribute linked combinations including the two. 10 | 11 | libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 12 | WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 13 | FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 14 | details. 15 | 16 | You should have received a copy of the GNU General Public License version 3 17 | along with libs3, in a file named COPYING. If not, see 18 | . 19 | 20 | 21 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/README: -------------------------------------------------------------------------------- 1 | This directory contains the libs3 library. 2 | 3 | The libs3 library is free software. See the file LICENSE for copying 4 | permission. 5 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/TODO: -------------------------------------------------------------------------------- 1 | * Implement functions for generating form stuff for posting to s3 2 | 3 | * Write s3 man page 4 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/archlinux/PKGBUILD: -------------------------------------------------------------------------------- 1 | # Contributor: Bryan Ischo 2 | pkgname=libs3 3 | pkgver=2.0 4 | pkgrel=1 5 | pkgdesc="C Library and Tools for Amazon S3 Access" 6 | arch=('i686' 'x86_64') 7 | url="http://libs3.ischo.com/index.html" 8 | license=('GPL') 9 | groups=() 10 | depends=('libxml2' 'openssl' 'curl') 11 | makedepends=('make' 'libxml2' 'openssl' 'curl') 12 | provides=() 13 | conflicts=() 14 | replaces=() 15 | backup=() 16 | options=() 17 | install= 18 | source=(http://libs3.ischo.com/$pkgname-$pkgver.tar.gz) 19 | noextract=() 20 | md5sums=('source md5') #generate with 'makepkg -g' 21 | 22 | build() { 23 | cd "$srcdir/$pkgname-$pkgver" 24 | 25 | DESTDIR=$pkgdir/usr make install || return 1 26 | } 27 | 28 | # vim:set ts=2 sw=2 et: 29 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/debian/changelog: -------------------------------------------------------------------------------- 1 | libs3 (all) unstable; urgency=low 2 | 3 | * This file is not maintained. See project source code for changes. 4 | 5 | -- Bryan Ischo Wed, 06 Aug 2008 09:36:43 -0400 6 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/debian/changelog.Debian: -------------------------------------------------------------------------------- 1 | libs3 (all) unstable; urgency=low 2 | 3 | * libs3 Debian maintainer and upstream author are identical. 4 | Therefore see normal changelog file for Debian changes. 5 | 6 | -- Bryan Ischo Wed, 06 Aug 2008 09:36:43 -0400 7 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/debian/control: -------------------------------------------------------------------------------- 1 | Package: libs3 2 | Source: THIS LINE WILL BE REMOVED, dpkg-shlibdepends NEEDS IT 3 | Version: LIBS3_VERSION 4 | Architecture: DEBIAN_ARCHITECTURE 5 | Section: net 6 | Priority: extra 7 | Maintainer: Bryan Ischo 8 | Homepage: http://libs3.ischo.com/index.html 9 | Description: C Library and Tools for Amazon S3 Access 10 | This package includes the libs3 shared object library, needed to run 11 | applications compiled against libs3, and additionally contains the s3 12 | utility for accessing Amazon S3. 13 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/debian/control.dev: -------------------------------------------------------------------------------- 1 | Package: libs3-dev 2 | Version: LIBS3_VERSION 3 | Architecture: DEBIAN_ARCHITECTURE 4 | Section: libdevel 5 | Priority: extra 6 | Depends: libs3 (>= LIBS3_VERSION) 7 | Maintainer: Bryan Ischo 8 | Homepage: http://libs3.ischo.com/index.html 9 | Description: C Development Library for Amazon S3 Access 10 | This library provides an API for using Amazon's S3 service (see 11 | http://s3.amazonaws.com). Its design goals are: 12 | . 13 | - To provide a simple and straightforward API for accessing all of S3's 14 | functionality 15 | - To not require the developer using libs3 to need to know anything about: 16 | - HTTP 17 | - XML 18 | - SSL 19 | In other words, this API is meant to stand on its own, without requiring 20 | any implicit knowledge of how S3 services are accessed using HTTP 21 | protocols. 22 | - To be usable from multithreaded code 23 | - To be usable by code which wants to process multiple S3 requests 24 | simultaneously from a single thread 25 | - To be usable in the simple, straightforward way using sequentialized 26 | blocking requests 27 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/debian/postinst: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ldconfig 4 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/error_parser.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * error_parser.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #ifndef ERROR_PARSER_H 28 | #define ERROR_PARSER_H 29 | 30 | #include "libs3.h" 31 | #include "simplexml.h" 32 | #include "string_buffer.h" 33 | 34 | 35 | #define EXTRA_DETAILS_SIZE 8 36 | 37 | typedef struct ErrorParser 38 | { 39 | // This is the S3ErrorDetails that this ErrorParser fills in from the 40 | // data that it parses 41 | S3ErrorDetails s3ErrorDetails; 42 | 43 | // This is the error XML parser 44 | SimpleXml errorXmlParser; 45 | 46 | // Set to 1 after the first call to add 47 | int errorXmlParserInitialized; 48 | 49 | // Used to buffer the S3 Error Code as it is read in 50 | string_buffer(code, 1024); 51 | 52 | // Used to buffer the S3 Error Message as it is read in 53 | string_buffer(message, 1024); 54 | 55 | // Used to buffer the S3 Error Resource as it is read in 56 | string_buffer(resource, 1024); 57 | 58 | // Used to buffer the S3 Error Further Details as it is read in 59 | string_buffer(furtherDetails, 1024); 60 | 61 | // The extra details; we support up to EXTRA_DETAILS_SIZE of them 62 | S3NameValue extraDetails[EXTRA_DETAILS_SIZE]; 63 | 64 | // This is the buffer from which the names and values used in extraDetails 65 | // are allocated 66 | string_multibuffer(extraDetailsNamesValues, EXTRA_DETAILS_SIZE * 1024); 67 | } ErrorParser; 68 | 69 | 70 | // Always call this 71 | void error_parser_initialize(ErrorParser *errorParser); 72 | 73 | S3Status error_parser_add(ErrorParser *errorParser, char *buffer, 74 | int bufferSize); 75 | 76 | void error_parser_convert_status(ErrorParser *errorParser, S3Status *status); 77 | 78 | // Always call this 79 | void error_parser_deinitialize(ErrorParser *errorParser); 80 | 81 | 82 | #endif /* ERROR_PARSER_H */ 83 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/mingw/pthread.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * pthread.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #ifndef PTHREAD_H 28 | #define PTHREAD_H 29 | 30 | // This is a minimal implementation of pthreads on Windows, implementing just 31 | // the APIs needed by libs3 32 | 33 | unsigned long pthread_self(); 34 | 35 | typedef struct 36 | { 37 | CRITICAL_SECTION criticalSection; 38 | } pthread_mutex_t; 39 | 40 | int pthread_mutex_init(pthread_mutex_t *mutex, void *); 41 | int pthread_mutex_lock(pthread_mutex_t *mutex); 42 | int pthread_mutex_unlock(pthread_mutex_t *mutex); 43 | int pthread_mutex_destroy(pthread_mutex_t *mutex); 44 | 45 | #endif /* PTHREAD_H */ 46 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/mingw/sys/select.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * select.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | // This file is used only on a MingW build, and converts an include of 28 | // sys/select.h to its Windows equivalent 29 | 30 | #include 31 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/mingw/sys/utsname.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * utsname.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | // This file is used only on a MingW build, and provides an implementation 28 | // of POSIX sys/utsname.h 29 | 30 | #ifndef UTSNAME_H 31 | #define UTSNAME_H 32 | 33 | struct utsname 34 | { 35 | const char *sysname; 36 | const char *machine; 37 | }; 38 | 39 | int uname(struct utsname *); 40 | 41 | #endif /* UTSNAME_H */ 42 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/request.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * request.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #ifndef REQUEST_H 28 | #define REQUEST_H 29 | 30 | #include "libs3.h" 31 | #include "error_parser.h" 32 | #include "response_headers_handler.h" 33 | #include "util.h" 34 | 35 | // Describes a type of HTTP request (these are our supported HTTP "verbs") 36 | typedef enum 37 | { 38 | HttpRequestTypeGET, 39 | HttpRequestTypeHEAD, 40 | HttpRequestTypePUT, 41 | HttpRequestTypeCOPY, 42 | HttpRequestTypeDELETE 43 | } HttpRequestType; 44 | 45 | 46 | // This completely describes a request. A RequestParams is not required to be 47 | // allocated from the heap and its lifetime is not assumed to extend beyond 48 | // the lifetime of the function to which it has been passed. 49 | typedef struct RequestParams 50 | { 51 | // Request type, affects the HTTP verb used 52 | HttpRequestType httpRequestType; 53 | 54 | // Bucket context for request 55 | S3BucketContext bucketContext; 56 | 57 | // Key, if any 58 | const char *key; 59 | 60 | // Query params - ready to append to URI (i.e. ?p1=v1?p2=v2) 61 | const char *queryParams; 62 | 63 | // sub resource, like ?acl, ?location, ?torrent, ?logging 64 | const char *subResource; 65 | 66 | // If this is a copy operation, this gives the source bucket 67 | const char *copySourceBucketName; 68 | 69 | // If this is a copy operation, this gives the source key 70 | const char *copySourceKey; 71 | 72 | // Get conditions 73 | const S3GetConditions *getConditions; 74 | 75 | // Start byte 76 | uint64_t startByte; 77 | 78 | // Byte count 79 | uint64_t byteCount; 80 | 81 | // Put properties 82 | const S3PutProperties *putProperties; 83 | 84 | // Callback to be made when headers are available. Might not be called. 85 | S3ResponsePropertiesCallback *propertiesCallback; 86 | 87 | // Callback to be made to supply data to send to S3. Might not be called. 88 | S3PutObjectDataCallback *toS3Callback; 89 | 90 | // Number of bytes total that readCallback will supply 91 | int64_t toS3CallbackTotalSize; 92 | 93 | // Callback to be made that supplies data read from S3. 94 | // Might not be called. 95 | S3GetObjectDataCallback *fromS3Callback; 96 | 97 | // Callback to be made when request is complete. This will *always* be 98 | // called. 99 | S3ResponseCompleteCallback *completeCallback; 100 | 101 | // Data passed to the callbacks 102 | void *callbackData; 103 | } RequestParams; 104 | 105 | 106 | // This is the stuff associated with a request that needs to be on the heap 107 | // (and thus live while a curl_multi is in use). 108 | typedef struct Request 109 | { 110 | // These put the request on a doubly-linked list of requests in a 111 | // request context, *if* the request is in a request context (else these 112 | // will both be 0) 113 | struct Request *prev, *next; 114 | 115 | // The status of this Request, as will be reported to the user via the 116 | // complete callback 117 | S3Status status; 118 | 119 | // The HTTP code returned by the S3 server, if it is known. Would rather 120 | // not have to keep track of this but S3 doesn't always indicate its 121 | // errors the same way 122 | int httpResponseCode; 123 | 124 | // The HTTP headers to use for the curl request 125 | struct curl_slist *headers; 126 | 127 | // The CURL structure driving the request 128 | CURL *curl; 129 | 130 | // libcurl requires that the uri be stored outside of the curl handle 131 | char uri[MAX_URI_SIZE + 1]; 132 | 133 | // Callback to be made when headers are available. Might not be called. 134 | S3ResponsePropertiesCallback *propertiesCallback; 135 | 136 | // Callback to be made to supply data to send to S3. Might not be called. 137 | S3PutObjectDataCallback *toS3Callback; 138 | 139 | // Number of bytes total that readCallback has left to supply 140 | int64_t toS3CallbackBytesRemaining; 141 | 142 | // Callback to be made that supplies data read from S3. 143 | // Might not be called. 144 | S3GetObjectDataCallback *fromS3Callback; 145 | 146 | // Callback to be made when request is complete. This will *always* be 147 | // called. 148 | S3ResponseCompleteCallback *completeCallback; 149 | 150 | // Data passed to the callbacks 151 | void *callbackData; 152 | 153 | // Handler of response headers 154 | ResponseHeadersHandler responseHeadersHandler; 155 | 156 | // This is set to nonzero after the properties callback has been made 157 | int propertiesCallbackMade; 158 | 159 | // Parser of errors 160 | ErrorParser errorParser; 161 | } Request; 162 | 163 | 164 | // Request functions 165 | // ---------------------------------------------------------------------------- 166 | 167 | // Initialize the API 168 | S3Status request_api_initialize(const char *userAgentInfo, int flags, 169 | const char *hostName); 170 | 171 | // Deinitialize the API 172 | void request_api_deinitialize(); 173 | 174 | // Perform a request; if context is 0, performs the request immediately; 175 | // otherwise, sets it up to be performed by context. 176 | void request_perform(const RequestParams *params, S3RequestContext *context); 177 | 178 | // Called by the internal request code or internal request context code when a 179 | // curl has finished the request 180 | void request_finish(Request *request); 181 | 182 | // Convert a CURLE code to an S3Status 183 | S3Status request_curl_code_to_status(CURLcode code); 184 | 185 | 186 | #endif /* REQUEST_H */ 187 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/request_context.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * request_context.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #ifndef REQUEST_CONTEXT_H 28 | #define REQUEST_CONTEXT_H 29 | 30 | #include "libs3.h" 31 | 32 | struct S3RequestContext 33 | { 34 | CURLM *curlm; 35 | 36 | struct Request *requests; 37 | }; 38 | 39 | 40 | #endif /* REQUEST_CONTEXT_H */ 41 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/response_headers_handler.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * response_headers_handler.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #ifndef RESPONSE_HEADERS_HANDLER_H 28 | #define RESPONSE_HEADERS_HANDLER_H 29 | 30 | #include "libs3.h" 31 | #include "string_buffer.h" 32 | #include "util.h" 33 | 34 | 35 | typedef struct ResponseHeadersHandler 36 | { 37 | // The structure to pass to the headers callback. This is filled in by 38 | // the ResponseHeadersHandler from the headers added to it. 39 | S3ResponseProperties responseProperties; 40 | 41 | // Set to 1 after the done call has been made 42 | int done; 43 | 44 | // copied into here. We allow 128 bytes for each header, plus \0 term. 45 | string_multibuffer(responsePropertyStrings, 5 * 129); 46 | 47 | // responseproperties.metaHeaders strings get copied into here 48 | string_multibuffer(responseMetaDataStrings, 49 | COMPACTED_METADATA_BUFFER_SIZE); 50 | 51 | // Response meta data 52 | S3NameValue responseMetaData[S3_MAX_METADATA_COUNT]; 53 | } ResponseHeadersHandler; 54 | 55 | 56 | void response_headers_handler_initialize(ResponseHeadersHandler *handler); 57 | 58 | void response_headers_handler_add(ResponseHeadersHandler *handler, 59 | char *data, int dataLen); 60 | 61 | void response_headers_handler_done(ResponseHeadersHandler *handler, 62 | CURL *curl); 63 | 64 | #endif /* RESPONSE_HEADERS_HANDLER_H */ 65 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/simplexml.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * simplexml.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #ifndef SIMPLEXML_H 28 | #define SIMPLEXML_H 29 | 30 | #include "libs3.h" 31 | 32 | 33 | // Simple XML callback. 34 | // 35 | // elementPath: is the full "path" of the element; i.e. 36 | // data would have 'data' in the element 37 | // foo/bar/baz. 38 | // 39 | // Return of anything other than S3StatusOK causes the calling 40 | // simplexml_add() function to immediately stop and return the status. 41 | // 42 | // data is passed in as 0 on end of element 43 | typedef S3Status (SimpleXmlCallback)(const char *elementPath, const char *data, 44 | int dataLen, void *callbackData); 45 | 46 | typedef struct SimpleXml 47 | { 48 | void *xmlParser; 49 | 50 | SimpleXmlCallback *callback; 51 | 52 | void *callbackData; 53 | 54 | char elementPath[512]; 55 | 56 | int elementPathLen; 57 | 58 | S3Status status; 59 | } SimpleXml; 60 | 61 | 62 | // Simple XML parsing 63 | // ---------------------------------------------------------------------------- 64 | 65 | // Always call this, even if the simplexml doesn't end up being used 66 | void simplexml_initialize(SimpleXml *simpleXml, SimpleXmlCallback *callback, 67 | void *callbackData); 68 | 69 | S3Status simplexml_add(SimpleXml *simpleXml, const char *data, int dataLen); 70 | 71 | 72 | // Always call this 73 | void simplexml_deinitialize(SimpleXml *simpleXml); 74 | 75 | 76 | #endif /* SIMPLEXML_H */ 77 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/string_buffer.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * string_buffer.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #ifndef STRING_BUFFER_H 28 | #define STRING_BUFFER_H 29 | 30 | #include 31 | 32 | 33 | // Declare a string_buffer with the given name of the given maximum length 34 | #define string_buffer(name, len) \ 35 | char name[len + 1]; \ 36 | int name##Len 37 | 38 | 39 | // Initialize a string_buffer 40 | #define string_buffer_initialize(sb) \ 41 | do { \ 42 | sb[0] = 0; \ 43 | sb##Len = 0; \ 44 | } while (0) 45 | 46 | 47 | // Append [len] bytes of [str] to [sb], setting [all_fit] to 1 if it fit, and 48 | // 0 if it did not 49 | #define string_buffer_append(sb, str, len, all_fit) \ 50 | do { \ 51 | sb##Len += snprintf(&(sb[sb##Len]), sizeof(sb) - sb##Len - 1, \ 52 | "%.*s", (int) (len), str); \ 53 | if (sb##Len > (int) (sizeof(sb) - 1)) { \ 54 | sb##Len = sizeof(sb) - 1; \ 55 | all_fit = 0; \ 56 | } \ 57 | else { \ 58 | all_fit = 1; \ 59 | } \ 60 | } while (0) 61 | 62 | 63 | // Declare a string multibuffer with the given name of the given maximum size 64 | #define string_multibuffer(name, size) \ 65 | char name[size]; \ 66 | int name##Size 67 | 68 | 69 | // Initialize a string_multibuffer 70 | #define string_multibuffer_initialize(smb) \ 71 | do { \ 72 | smb##Size = 0; \ 73 | } while (0) 74 | 75 | 76 | // Evaluates to the current string within the string_multibuffer 77 | #define string_multibuffer_current(smb) \ 78 | &(smb[smb##Size]) 79 | 80 | 81 | // Adds a new string to the string_multibuffer 82 | #define string_multibuffer_add(smb, str, len, all_fit) \ 83 | do { \ 84 | smb##Size += (snprintf(&(smb[smb##Size]), \ 85 | sizeof(smb) - smb##Size, \ 86 | "%.*s", (int) (len), str) + 1); \ 87 | if (smb##Size > (int) sizeof(smb)) { \ 88 | smb##Size = sizeof(smb); \ 89 | all_fit = 0; \ 90 | } \ 91 | else { \ 92 | all_fit = 1; \ 93 | } \ 94 | } while (0) 95 | 96 | 97 | // Appends to the current string in the string_multibuffer. There must be a 98 | // current string, meaning that string_multibuffer_add must have been called 99 | // at least once for this string_multibuffer. 100 | #define string_multibuffer_append(smb, str, len, all_fit) \ 101 | do { \ 102 | smb##Size--; \ 103 | string_multibuffer_add(smb, str, len, all_fit); \ 104 | } while (0) 105 | 106 | 107 | #endif /* STRING_BUFFER_H */ 108 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/inc/util.h: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * util.h 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #ifndef UTIL_H 28 | #define UTIL_H 29 | 30 | #include 31 | #include 32 | #include 33 | #include "libs3.h" 34 | 35 | // acl groups 36 | #define ACS_URL "http://acs.amazonaws.com/groups/" 37 | 38 | #define ACS_GROUP_ALL_USERS ACS_URL "global/AllUsers" 39 | #define ACS_GROUP_AWS_USERS ACS_URL "global/AuthenticatedUsers" 40 | #define ACS_GROUP_LOG_DELIVERY ACS_URL "s3/LogDelivery" 41 | 42 | 43 | 44 | // Derived from S3 documentation 45 | 46 | // This is the maximum number of bytes needed in a "compacted meta header" 47 | // buffer, which is a buffer storing all of the compacted meta headers. 48 | #define COMPACTED_METADATA_BUFFER_SIZE \ 49 | (S3_MAX_METADATA_COUNT * sizeof(S3_METADATA_HEADER_NAME_PREFIX "n: v")) 50 | 51 | // Maximum url encoded key size; since every single character could require 52 | // URL encoding, it's 3 times the size of a key (since each url encoded 53 | // character takes 3 characters: %NN) 54 | #define MAX_URLENCODED_KEY_SIZE (3 * S3_MAX_KEY_SIZE) 55 | 56 | // This is the maximum size of a URI that could be passed to S3: 57 | // https://s3.amazonaws.com/${BUCKET}/${KEY}?acl 58 | // 255 is the maximum bucket length 59 | #define MAX_URI_SIZE \ 60 | ((sizeof("https:///") - 1) + S3_MAX_HOSTNAME_SIZE + 255 + 1 + \ 61 | MAX_URLENCODED_KEY_SIZE + (sizeof("?torrent" - 1)) + 1) 62 | 63 | // Maximum size of a canonicalized resource 64 | #define MAX_CANONICALIZED_RESOURCE_SIZE \ 65 | (1 + 255 + 1 + MAX_URLENCODED_KEY_SIZE + (sizeof("?torrent") - 1) + 1) 66 | 67 | 68 | // Utilities ----------------------------------------------------------------- 69 | 70 | // URL-encodes a string from [src] into [dest]. [dest] must have at least 71 | // 3x the number of characters that [source] has. At most [maxSrcSize] bytes 72 | // from [src] are encoded; if more are present in [src], 0 is returned from 73 | // urlEncode, else nonzero is returned. 74 | int urlEncode(char *dest, const char *src, int maxSrcSize); 75 | 76 | // Returns < 0 on failure >= 0 on success 77 | int64_t parseIso8601Time(const char *str); 78 | 79 | uint64_t parseUnsignedInt(const char *str); 80 | 81 | // base64 encode bytes. The output buffer must have at least 82 | // ((4 * (inLen + 1)) / 3) bytes in it. Returns the number of bytes written 83 | // to [out]. 84 | int base64Encode(const unsigned char *in, int inLen, char *out); 85 | 86 | // Compute HMAC-SHA-1 with key [key] and message [message], storing result 87 | // in [hmac] 88 | void HMAC_SHA1(unsigned char hmac[20], const unsigned char *key, int key_len, 89 | const unsigned char *message, int message_len); 90 | 91 | // Compute a 64-bit hash values given a set of bytes 92 | uint64_t hash(const unsigned char *k, int length); 93 | 94 | // Because Windows seems to be missing isblank(), use our own; it's a very 95 | // easy function to write in any case 96 | int is_blank(char c); 97 | 98 | #endif /* UTIL_H */ 99 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/libs3.spec: -------------------------------------------------------------------------------- 1 | Summary: C Library and Tools for Amazon S3 Access 2 | Name: libs3 3 | Version: 2.0 4 | Release: 1 5 | License: GPL 6 | Group: Networking/Utilities 7 | URL: http://sourceforge.net/projects/reallibs3 8 | Source0: libs3-2.0.tar.gz 9 | Buildroot: %{_tmppath}/%{name}-%{version}-%{release}-root 10 | # Want to include curl dependencies, but older Fedora Core uses curl-devel, 11 | # and newer Fedora Core uses libcurl-devel ... have to figure out how to 12 | # handle this problem, but for now, just don't check for any curl libraries 13 | # Buildrequires: curl-devel 14 | Buildrequires: libxml2-devel 15 | Buildrequires: openssl-devel 16 | Buildrequires: make 17 | # Requires: libcurl 18 | Requires: libxml2 19 | Requires: openssl 20 | 21 | %define debug_package %{nil} 22 | 23 | %description 24 | This package includes the libs3 shared object library, needed to run 25 | applications compiled against libs3, and additionally contains the s3 26 | utility for accessing Amazon S3. 27 | 28 | %package devel 29 | Summary: Headers and documentation for libs3 30 | Group: Development/Libraries 31 | Requires: %{name} = %{version}-%{release} 32 | 33 | %description devel 34 | This library provides an API for using Amazon's S3 service (see 35 | http://s3.amazonaws.com). Its design goals are: 36 | 37 | - To provide a simple and straightforward API for accessing all of S3's 38 | functionality 39 | - To not require the developer using libs3 to need to know anything about: 40 | - HTTP 41 | - XML 42 | - SSL 43 | In other words, this API is meant to stand on its own, without requiring 44 | any implicit knowledge of how S3 services are accessed using HTTP 45 | protocols. 46 | - To be usable from multithreaded code 47 | - To be usable by code which wants to process multiple S3 requests 48 | simultaneously from a single thread 49 | - To be usable in the simple, straightforward way using sequentialized 50 | blocking requests 51 | 52 | 53 | %prep 54 | %setup -q 55 | 56 | %build 57 | BUILD=$RPM_BUILD_ROOT/build make exported 58 | 59 | %install 60 | BUILD=$RPM_BUILD_ROOT/build DESTDIR=$RPM_BUILD_ROOT/usr make install 61 | rm -rf $RPM_BUILD_ROOT/build 62 | 63 | %clean 64 | rm -rf $RPM_BUILD_ROOT 65 | 66 | %files 67 | %defattr(-,root,root,-) 68 | /usr/bin/s3 69 | /usr/lib/libs3.so* 70 | 71 | %files devel 72 | %defattr(-,root,root,-) 73 | /usr/include/libs3.h 74 | /usr/lib/libs3.a 75 | 76 | %changelog 77 | * Sat Aug 09 2008 Bryan Ischo 78 | - Split into regular and devel packages. 79 | 80 | * Tue Aug 05 2008 Bryan Ischo 81 | - Initial build. 82 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/mswin/libs3.def: -------------------------------------------------------------------------------- 1 | EXPORTS 2 | S3_convert_acl 3 | S3_copy_object 4 | S3_create_bucket 5 | S3_create_request_context 6 | S3_deinitialize 7 | S3_delete_bucket 8 | S3_delete_object 9 | S3_destroy_request_context 10 | S3_generate_authenticated_query_string 11 | S3_get_acl 12 | S3_get_object 13 | S3_get_request_context_fdsets 14 | S3_get_server_access_logging 15 | S3_get_status_name 16 | S3_head_object 17 | S3_initialize 18 | S3_list_bucket 19 | S3_list_service 20 | S3_put_object 21 | S3_runall_request_context 22 | S3_runonce_request_context 23 | S3_set_acl 24 | S3_set_server_access_logging 25 | S3_status_is_retryable 26 | S3_test_bucket 27 | S3_validate_bucket_name 28 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/mswin/rmrf.bat: -------------------------------------------------------------------------------- 1 | @echo off 2 | 3 | if exist "%1". ( 4 | rmdir /S /Q "%1" 5 | ) 6 | 7 | if exist "%1". ( 8 | del /Q "%1" 9 | ) 10 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/src/mingw_functions.c: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * mingw_functions.c 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #include 28 | #include 29 | 30 | unsigned long pthread_self() 31 | { 32 | return (unsigned long) GetCurrentThreadId(); 33 | } 34 | 35 | 36 | int pthread_mutex_init(pthread_mutex_t *mutex, void *v) 37 | { 38 | (void) v; 39 | 40 | InitializeCriticalSection(&(mutex->criticalSection)); 41 | 42 | return 0; 43 | } 44 | 45 | 46 | int pthread_mutex_lock(pthread_mutex_t *mutex) 47 | { 48 | EnterCriticalSection(&(mutex->criticalSection)); 49 | 50 | return 0; 51 | } 52 | 53 | 54 | int pthread_mutex_unlock(pthread_mutex_t *mutex) 55 | { 56 | LeaveCriticalSection(&(mutex->criticalSection)); 57 | 58 | return 0; 59 | } 60 | 61 | 62 | int pthread_mutex_destroy(pthread_mutex_t *mutex) 63 | { 64 | DeleteCriticalSection(&(mutex->criticalSection)); 65 | 66 | return 0; 67 | } 68 | 69 | 70 | int uname(struct utsname *u) 71 | { 72 | OSVERSIONINFO info; 73 | info.dwOSVersionInfoSize = sizeof(info); 74 | 75 | if (!GetVersionEx(&info)) { 76 | return -1; 77 | } 78 | 79 | u->machine = ""; 80 | 81 | switch (info.dwMajorVersion) { 82 | case 4: 83 | switch (info.dwMinorVersion) { 84 | case 0: 85 | u->sysname = "Microsoft Windows NT 4.0"; 86 | break; 87 | case 10: 88 | u->sysname = "Microsoft Windows 98"; 89 | break; 90 | case 90: 91 | u->sysname = "Microsoft Windows Me"; 92 | break; 93 | default: 94 | return -1; 95 | } 96 | break; 97 | 98 | case 5: 99 | switch (info.dwMinorVersion) { 100 | case 0: 101 | u->sysname = "Microsoft Windows 2000"; 102 | break; 103 | case 1: 104 | u->sysname = "Microsoft Windows XP"; 105 | break; 106 | case 2: 107 | u->sysname = "Microsoft Server 2003"; 108 | break; 109 | default: 110 | return -1; 111 | } 112 | break; 113 | 114 | default: 115 | return -1; 116 | } 117 | 118 | return 0; 119 | } 120 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/src/mingw_s3_functions.c: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * mingw_s3_functions.c 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | int setenv(const char *a, const char *b, int c) 28 | { 29 | (void) c; 30 | 31 | return SetEnvironmentVariable(a, b); 32 | } 33 | 34 | int unsetenv(const char *a) 35 | { 36 | return SetEnvironmentVariable(a, 0); 37 | } 38 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/src/simplexml.c: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * simplexml.c 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #include 28 | #include 29 | #include "simplexml.h" 30 | 31 | // Use libxml2 for parsing XML. XML is severely overused in modern 32 | // computing. It is useful for only a very small subset of tasks, but 33 | // software developers who don't know better and are afraid to go against the 34 | // grain use it for everything, and in most cases, it is completely 35 | // inappropriate. Usually, the document structure is severely under-specified 36 | // as well, as is the case with S3. We do our best by just caring about the 37 | // most important aspects of the S3 "XML document" responses: the elements and 38 | // their values. The SAX API (just about the lamest API ever devised and 39 | // proof that XML sucks - well, the real proof is how crappy all of the XML 40 | // parsing libraries are, including libxml2 - but I digress) is used here 41 | // because we don't need much from the parser and SAX is fast and low memory. 42 | // 43 | // Note that for simplicity we assume all ASCII here. No attempts are made to 44 | // detect non-ASCII sequences in utf-8 and convert them into ASCII in any way. 45 | // S3 appears to only use ASCII anyway. 46 | 47 | 48 | static xmlEntityPtr saxGetEntity(void *user_data, const xmlChar *name) 49 | { 50 | (void) user_data; 51 | 52 | return xmlGetPredefinedEntity(name); 53 | } 54 | 55 | 56 | static void saxStartElement(void *user_data, const xmlChar *nameUtf8, 57 | const xmlChar **attr) 58 | { 59 | (void) attr; 60 | 61 | SimpleXml *simpleXml = (SimpleXml *) user_data; 62 | 63 | if (simpleXml->status != S3StatusOK) { 64 | return; 65 | } 66 | 67 | // Assume that name has no non-ASCII in it 68 | char *name = (char *) nameUtf8; 69 | 70 | // Append the element to the element path 71 | int len = strlen(name); 72 | 73 | if ((simpleXml->elementPathLen + len + 1) >= 74 | (int) sizeof(simpleXml->elementPath)) { 75 | // Cannot handle this element, stop! 76 | simpleXml->status = S3StatusXmlParseFailure; 77 | return; 78 | } 79 | 80 | if (simpleXml->elementPathLen) { 81 | simpleXml->elementPath[simpleXml->elementPathLen++] = '/'; 82 | } 83 | strcpy(&(simpleXml->elementPath[simpleXml->elementPathLen]), name); 84 | simpleXml->elementPathLen += len; 85 | } 86 | 87 | 88 | static void saxEndElement(void *user_data, const xmlChar *name) 89 | { 90 | (void) name; 91 | 92 | SimpleXml *simpleXml = (SimpleXml *) user_data; 93 | 94 | if (simpleXml->status != S3StatusOK) { 95 | return; 96 | } 97 | 98 | // Call back with 0 data 99 | simpleXml->status = (*(simpleXml->callback)) 100 | (simpleXml->elementPath, 0, 0, simpleXml->callbackData); 101 | 102 | while ((simpleXml->elementPathLen > 0) && 103 | (simpleXml->elementPath[simpleXml->elementPathLen] != '/')) { 104 | simpleXml->elementPathLen--; 105 | } 106 | 107 | simpleXml->elementPath[simpleXml->elementPathLen] = 0; 108 | } 109 | 110 | 111 | static void saxCharacters(void *user_data, const xmlChar *ch, int len) 112 | { 113 | SimpleXml *simpleXml = (SimpleXml *) user_data; 114 | 115 | if (simpleXml->status != S3StatusOK) { 116 | return; 117 | } 118 | 119 | simpleXml->status = (*(simpleXml->callback)) 120 | (simpleXml->elementPath, (char *) ch, len, simpleXml->callbackData); 121 | } 122 | 123 | 124 | static void saxError(void *user_data, const char *msg, ...) 125 | { 126 | (void) msg; 127 | 128 | SimpleXml *simpleXml = (SimpleXml *) user_data; 129 | 130 | if (simpleXml->status != S3StatusOK) { 131 | return; 132 | } 133 | 134 | simpleXml->status = S3StatusXmlParseFailure; 135 | } 136 | 137 | 138 | static struct _xmlSAXHandler saxHandlerG = 139 | { 140 | 0, // internalSubsetSAXFunc 141 | 0, // isStandaloneSAXFunc 142 | 0, // hasInternalSubsetSAXFunc 143 | 0, // hasExternalSubsetSAXFunc 144 | 0, // resolveEntitySAXFunc 145 | &saxGetEntity, // getEntitySAXFunc 146 | 0, // entityDeclSAXFunc 147 | 0, // notationDeclSAXFunc 148 | 0, // attributeDeclSAXFunc 149 | 0, // elementDeclSAXFunc 150 | 0, // unparsedEntityDeclSAXFunc 151 | 0, // setDocumentLocatorSAXFunc 152 | 0, // startDocumentSAXFunc 153 | 0, // endDocumentSAXFunc 154 | &saxStartElement, // startElementSAXFunc 155 | &saxEndElement, // endElementSAXFunc 156 | 0, // referenceSAXFunc 157 | &saxCharacters, // charactersSAXFunc 158 | 0, // ignorableWhitespaceSAXFunc 159 | 0, // processingInstructionSAXFunc 160 | 0, // commentSAXFunc 161 | 0, // warningSAXFunc 162 | &saxError, // errorSAXFunc 163 | &saxError, // fatalErrorSAXFunc 164 | 0, // getParameterEntitySAXFunc 165 | &saxCharacters, // cdataBlockSAXFunc 166 | 0, // externalSubsetSAXFunc 167 | 0, // initialized 168 | 0, // _private 169 | 0, // startElementNsSAX2Func 170 | 0, // endElementNsSAX2Func 171 | 0 // xmlStructuredErrorFunc serror; 172 | }; 173 | 174 | void simplexml_initialize(SimpleXml *simpleXml, 175 | SimpleXmlCallback *callback, void *callbackData) 176 | { 177 | simpleXml->callback = callback; 178 | simpleXml->callbackData = callbackData; 179 | simpleXml->elementPathLen = 0; 180 | simpleXml->status = S3StatusOK; 181 | simpleXml->xmlParser = 0; 182 | } 183 | 184 | 185 | void simplexml_deinitialize(SimpleXml *simpleXml) 186 | { 187 | if (simpleXml->xmlParser) { 188 | xmlFreeParserCtxt(simpleXml->xmlParser); 189 | } 190 | } 191 | 192 | 193 | S3Status simplexml_add(SimpleXml *simpleXml, const char *data, int dataLen) 194 | { 195 | if (!simpleXml->xmlParser && 196 | (!(simpleXml->xmlParser = xmlCreatePushParserCtxt 197 | (&saxHandlerG, simpleXml, 0, 0, 0)))) { 198 | return S3StatusInternalError; 199 | } 200 | 201 | if (xmlParseChunk((xmlParserCtxtPtr) simpleXml->xmlParser, 202 | data, dataLen, 0)) { 203 | return S3StatusXmlParseFailure; 204 | } 205 | 206 | return simpleXml->status; 207 | } 208 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/src/size: -------------------------------------------------------------------------------- 1 | bucket.c: string_buffer_initialize(contents->key); 2 | bucket.c: string_buffer_initialize(contents->lastModified); 3 | bucket.c: string_buffer_initialize(contents->eTag); 4 | bucket.c: string_buffer_initialize(contents->size); 5 | bucket.c: string_buffer_initialize(contents->ownerId); 6 | bucket.c: string_buffer_initialize(contents->ownerDisplayName); 7 | bucket.c: string_buffer_append(contents->key, data, dataLen, fit); 8 | bucket.c: string_buffer_append(contents->lastModified, data, dataLen, fit); 9 | bucket.c: string_buffer_append(contents->eTag, data, dataLen, fit); 10 | bucket.c: string_buffer_append(contents->size, data, dataLen, fit); 11 | bucket.c: string_buffer_append(contents->ownerId, data, dataLen, fit); 12 | bucket.c: (contents->ownerDisplayName, data, dataLen, fit); 13 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/src/testsimplexml.c: -------------------------------------------------------------------------------- 1 | /** ************************************************************************** 2 | * testsimplexml.c 3 | * 4 | * Copyright 2008 Bryan Ischo 5 | * 6 | * This file is part of libs3. 7 | * 8 | * libs3 is free software: you can redistribute it and/or modify it under the 9 | * terms of the GNU General Public License as published by the Free Software 10 | * Foundation, version 3 of the License. 11 | * 12 | * In addition, as a special exception, the copyright holders give 13 | * permission to link the code of this library and its programs with the 14 | * OpenSSL library, and distribute linked combinations including the two. 15 | * 16 | * libs3 is distributed in the hope that it will be useful, but WITHOUT ANY 17 | * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS 18 | * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more 19 | * details. 20 | * 21 | * You should have received a copy of the GNU General Public License version 3 22 | * along with libs3, in a file named COPYING. If not, see 23 | * . 24 | * 25 | ************************************************************************** **/ 26 | 27 | #include 28 | #include 29 | #include 30 | #include 31 | #include "simplexml.h" 32 | 33 | static S3Status simpleXmlCallback(const char *elementPath, const char *data, 34 | int dataLen, void *callbackData) 35 | { 36 | (void) callbackData; 37 | 38 | printf("[%s]: [%.*s]\n", elementPath, dataLen, data); 39 | 40 | return S3StatusOK; 41 | } 42 | 43 | 44 | // The only argument allowed is a specification of the random seed to use 45 | int main(int argc, char **argv) 46 | { 47 | if (argc > 1) { 48 | char *arg = argv[1]; 49 | int seed = 0; 50 | while (*arg) { 51 | seed *= 10; 52 | seed += (*arg++ - '0'); 53 | } 54 | 55 | srand(seed); 56 | } 57 | else { 58 | srand(time(0)); 59 | } 60 | 61 | SimpleXml simpleXml; 62 | 63 | simplexml_initialize(&simpleXml, &simpleXmlCallback, 0); 64 | 65 | // Read chunks of 10K from stdin, and then feed them in random chunks 66 | // to simplexml_add 67 | char inbuf[10000]; 68 | 69 | int amt_read; 70 | while ((amt_read = fread(inbuf, 1, sizeof(inbuf), stdin)) > 0) { 71 | char *buf = inbuf; 72 | while (amt_read) { 73 | int amt = (rand() % amt_read) + 1; 74 | S3Status status = simplexml_add(&simpleXml, buf, amt); 75 | if (status != S3StatusOK) { 76 | fprintf(stderr, "ERROR: Parse failure: %d\n", status); 77 | simplexml_deinitialize(&simpleXml); 78 | return -1; 79 | } 80 | buf += amt, amt_read -= amt; 81 | } 82 | } 83 | 84 | simplexml_deinitialize(&simpleXml); 85 | 86 | return 0; 87 | } 88 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/test.txt: -------------------------------------------------------------------------------- 1 | This is some text in a file. 2 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/test/badxml_01.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | Data 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/test/goodxml_01.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | NoSuchKey 4 | The resource & then]]> you requested does not exist & so there 5 | /mybucket/myfoto.jpg 6 | 4442587FB7D0A2F9 7 | 8 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/test/goodxml_02.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | Data 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /tools/lib/libs3-2.0/test/test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # Environment: 4 | # S3_ACCESS_KEY_ID - must be set to S3 Access Key ID 5 | # S3_SECRET_ACCESS_KEY - must be set to S3 Secret Access Key 6 | # TEST_BUCKET_PREFIX - must be set to the test bucket prefix to use 7 | # S3_COMMAND - may be set to s3 command to use (i.e. valgrind s3); defaults 8 | # to "s3" 9 | 10 | if [ -z "$S3_ACCESS_KEY_ID" ]; then 11 | echo "S3_ACCESS_KEY_ID required" 12 | exit -1; 13 | fi 14 | 15 | if [ -z "$S3_SECRET_ACCESS_KEY" ]; then 16 | echo "S3_SECRET_ACCESS_KEY required" 17 | exit -1; 18 | fi 19 | 20 | if [ -z "$TEST_BUCKET_PREFIX" ]; then 21 | echo "TEST_BUCKET_PREFIX required" 22 | exit -1; 23 | fi 24 | 25 | if [ -z "$S3_COMMAND" ]; then 26 | S3_COMMAND=s3 27 | fi 28 | 29 | TEST_BUCKET=${TEST_BUCKET_PREFIX}.testbucket 30 | 31 | # Create the test bucket in EU 32 | echo "$S3_COMMAND create $TEST_BUCKET locationConstraint=EU" 33 | $S3_COMMAND create $TEST_BUCKET 34 | 35 | # List to find it 36 | echo "$S3_COMMAND list | grep $TEST_BUCKET" 37 | $S3_COMMAND list | grep $TEST_BUCKET 38 | 39 | # Test it 40 | echo "$S3_COMMAND test $TEST_BUCKET" 41 | $S3_COMMAND test $TEST_BUCKET 42 | 43 | # List to ensure that it is empty 44 | echo "$S3_COMMAND list $TEST_BUCKET" 45 | $S3_COMMAND list $TEST_BUCKET 46 | 47 | # Put some data 48 | rm -f seqdata 49 | seq 1 10000 > seqdata 50 | echo "$S3_COMMAND put $TEST_BUCKET/testkey filename=seqdata noStatus=1" 51 | $S3_COMMAND put $TEST_BUCKET/testkey filename=seqdata noStatus=1 52 | 53 | rm -f testkey 54 | # Get the data and make sure that it matches 55 | echo "$S3_COMMAND get $TEST_BUCKET/testkey filename=testkey" 56 | $S3_COMMAND get $TEST_BUCKET/testkey filename=testkey 57 | diff seqdata testkey 58 | rm -f seqdata testkey 59 | 60 | # Delete the file 61 | echo "$S3_COMMAND delete $TEST_BUCKET/testkey" 62 | $S3_COMMAND delete $TEST_BUCKET/testkey 63 | 64 | # Remove the test bucket 65 | echo "$S3_COMMAND delete $TEST_BUCKET" 66 | $S3_COMMAND delete $TEST_BUCKET 67 | 68 | # Make sure it's not there 69 | echo "$S3_COMMAND list | grep $TEST_BUCKET" 70 | $S3_COMMAND list | grep $TEST_BUCKET 71 | 72 | # Now create it again 73 | echo "$S3_COMMAND create $TEST_BUCKET" 74 | $S3_COMMAND create $TEST_BUCKET 75 | 76 | # Put 10 files in it 77 | for i in `seq 0 9`; do 78 | echo "echo \"Hello\" | $S3_COMMAND put $TEST_BUCKET/key_$i" 79 | echo "Hello" | $S3_COMMAND put $TEST_BUCKET/key_$i 80 | done 81 | 82 | # List with all details 83 | echo "$S3_COMMAND list $TEST_BUCKET allDetails=1" 84 | $S3_COMMAND list $TEST_BUCKET allDetails=1 85 | 86 | COPY_BUCKET=${TEST_BUCKET_PREFIX}.copybucket 87 | 88 | # Create another test bucket and copy a file into it 89 | echo "$S3_COMMAND create $COPY_BUCKET" 90 | $S3_COMMAND create $COPY_BUCKET 91 | echo <> acl 132 | Group Authenticated AWS Users READ 133 | EOF 134 | echo <> acl 135 | Group All Users READ_ACP 136 | EOF 137 | echo "$S3_COMMAND setacl $TEST_BUCKET filename=acl" 138 | $S3_COMMAND setacl $TEST_BUCKET filename=acl 139 | 140 | # Test to make sure that it worked 141 | rm -f acl_new 142 | echo "$S3_COMMAND getacl $TEST_BUCKET filename=acl_new allDetails=1" 143 | $S3_COMMAND getacl $TEST_BUCKET filename=acl_new allDetails=1 144 | diff acl acl_new 145 | rm -f acl acl_new 146 | 147 | # Get the key acl 148 | rm -f acl 149 | echo "$S3_COMMAND getacl $TEST_BUCKET/aclkey filename=acl allDetails=1" 150 | $S3_COMMAND getacl $TEST_BUCKET/aclkey filename=acl allDetails=1 151 | 152 | # Add READ for all AWS users, and READ_ACP for everyone 153 | echo <> acl 154 | Group Authenticated AWS Users READ 155 | EOF 156 | echo <> acl 157 | Group All Users READ_ACP 158 | EOF 159 | echo "$S3_COMMAND setacl $TEST_BUCKET/aclkey filename=acl" 160 | $S3_COMMAND setacl $TEST_BUCKET/aclkey filename=acl 161 | 162 | # Test to make sure that it worked 163 | rm -f acl_new 164 | echo "$S3_COMMAND getacl $TEST_BUCKET/aclkey filename=acl_new allDetails=1" 165 | $S3_COMMAND getacl $TEST_BUCKET/aclkey filename=acl_new allDetails=1 166 | diff acl acl_new 167 | rm -f acl acl_new 168 | 169 | # Remove the test file 170 | echo "$S3_COMMAND delete $TEST_BUCKET/aclkey" 171 | $S3_COMMAND delete $TEST_BUCKET/aclkey 172 | echo "$S3_COMMAND delete $TEST_BUCKET" 173 | $S3_COMMAND delete $TEST_BUCKET 174 | -------------------------------------------------------------------------------- /tools/lib/time_it/Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: all clean distclean 2 | 3 | VPATH += 4 | 5 | CC = gcc 6 | LD = gcc 7 | AR = ar 8 | #----------------------------------------------------------------------------- 9 | INCLUDES = 10 | CFLAGS = -Wall -g -fPIC -O0 $(INCLUDES) -D_TIMING_DATA 11 | 12 | all: libtimeit.a 13 | 14 | clean: 15 | rm -rf *.o *.a *.dSYM 16 | 17 | %.o: %.c 18 | $(CC) -c $(CFLAGS) $< -o $@ 19 | 20 | libtimeit.a: time_it.o 21 | $(AR) -cvq $@ $^ 22 | 23 | -------------------------------------------------------------------------------- /tools/lib/time_it/time_it.c: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * A simple library for collecting in-line timing data. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @date 2008-2013 8 | * @copyright BSD 2-Clause License 9 | * See http://opensource.org/licenses/BSD-2-Clause 10 | **/ 11 | #include "time_it.h" 12 | 13 | timeit_t* time_it_ptr = NULL; 14 | 15 | 16 | /** 17 | * @brief Set ups library data structures. 18 | * 19 | * Call once, before using any other library macros / functions. 20 | **/ 21 | void time_it_init(void) 22 | { 23 | #ifdef _TIMING_DATA 24 | if (time_it_ptr != NULL) { 25 | fprintf(stderr, "WARNING: double time_it_init, prev data lost.\n"); 26 | time_it_cleanup(); 27 | } 28 | time_it_ptr = (timeit_t*) calloc(1,sizeof(timeit_t)); 29 | if (time_it_ptr == NULL) 30 | exit(-1); 31 | time_it_ptr->data = (struct time_it_timer_data*) 32 | calloc(MAX_TIMERS, sizeof(struct time_it_timer_data)); 33 | if (time_it_ptr->data == NULL) 34 | exit(-1); 35 | time_it_ptr->heads = 0; 36 | time_it_ptr->gets = 0; 37 | time_it_ptr->puts = 0; 38 | time_it_ptr->posts = 0; 39 | time_it_ptr->lists = 0; 40 | time_it_ptr->copys = 0; 41 | time_it_ptr->data_sz = MAX_TIMERS; 42 | #endif 43 | } 44 | 45 | 46 | /** 47 | * @brief Cleans up library data structures. 48 | **/ 49 | void time_it_cleanup(void) 50 | { 51 | #ifdef _TIMING_DATA 52 | free(time_it_ptr->data); 53 | free(time_it_ptr); 54 | time_it_ptr = NULL; 55 | #endif 56 | } 57 | 58 | 59 | /** 60 | * @brief Inserts a timing record into the global list of timing data. 61 | * 62 | * @param[in] t global list of timing records 63 | * @param[in] time time collected 64 | * @param[in] st_line starting line of in-line timing command 65 | * @param[in] en_line ending line of in-line timing command 66 | * @param[in] func function name holding in-line timing command 67 | * @param[in] file file of in-line timing command 68 | * @param[in] comment comment to 69 | **/ 70 | void time_it_insert_time(timeit_t *t, double time, 71 | unsigned int st_line, unsigned int en_line, 72 | const char *func, const char *file, const char *comment) 73 | { 74 | #ifdef _TIMING_DATA 75 | if (t == NULL) 76 | return; 77 | 78 | if (t->tic_no_toc) { 79 | fprintf(stdout, "WARNING at line %d in %s: %s\n", st_line, file, 80 | "Insert over TIC aborted."); 81 | fflush(stdout); 82 | return; 83 | } 84 | if (t->idx >= t->data_sz) { 85 | fprintf(stdout, "WARNING at line %d in %s: %s\n", st_line, file, 86 | "MAX_TIMER limit reached, bad things are about to happen."); 87 | fflush(stdout); 88 | t->data_sz *= 2; 89 | t->data = (struct time_it_timer_data*) 90 | realloc(t->data, sizeof(struct time_it_timer_data)*(t->data_sz)); 91 | } 92 | t->data[t->idx].st_line = st_line; 93 | t->data[t->idx].en_line = en_line; 94 | t->data[t->idx].time = time; 95 | t->data[t->idx].func = func; 96 | t->data[t->idx].file = file; 97 | t->data[t->idx].comment = comment; 98 | t->idx++; 99 | #endif 100 | } 101 | 102 | 103 | /** 104 | * @brief Starts collecting a single timing record. 105 | * 106 | * @param[in] t global list of timing records 107 | * @param[in] st_line starting line of in-line timing command 108 | **/ 109 | void time_it_start_timer(timeit_t *t, int st_line) 110 | { 111 | #ifdef _TIMING_DATA 112 | if(t == NULL) 113 | return; 114 | /* Don't reset the start time before recording a stop time */ 115 | if(t->tic_no_toc){ 116 | fprintf(stdout, "WARNING at line %d: %s.\n", st_line, 117 | "Double TIC called, most recent TIC aborted"); 118 | fflush(stdout); 119 | return; 120 | } 121 | 122 | t->tic_no_toc = 1; 123 | t->data[t->idx].st_line = st_line; 124 | gettimeofday(&(t->tv_start), NULL); 125 | #endif 126 | } 127 | 128 | 129 | /** 130 | * @brief Finishes a single timing record. 131 | * 132 | * @param[in] t global list of timing records 133 | * @param[in] en_line ending line of in-line timing command 134 | * @param[in] func function name holding in-line timing command 135 | * @param[in] file file of in-line timing command 136 | * @param[in] comment comment to 137 | **/ 138 | void time_it_stop_timer(timeit_t *t, unsigned int en_line, 139 | const char *func, const char *file, const char *comment) 140 | { 141 | #ifdef _TIMING_DATA 142 | if (t == NULL) 143 | return; 144 | 145 | // Don't calculate the stop time with invalid start time 146 | if(!t->tic_no_toc){ 147 | fprintf(stdout, "WARNING at line %d: %s\n", en_line, 148 | "Double TOC call aborted. TIC must occur first."); 149 | fflush(stdout); 150 | return; 151 | } 152 | gettimeofday(&(t->tv_stop), NULL); 153 | t->tic_no_toc = 0; 154 | t->data[t->idx].en_line = en_line; 155 | double usst = (double)t->tv_start.tv_usec/1000000.0; 156 | double ussp = (double)t->tv_stop.tv_usec/1000000.0; 157 | double sst = (double)((double)t->tv_start.tv_sec + usst); 158 | double ssp = (double)((double)t->tv_stop.tv_sec + ussp); 159 | t->data[t->idx].time = ssp - sst; 160 | t->data[t->idx].func = func; 161 | t->data[t->idx].file = file; 162 | t->data[t->idx].comment = comment; 163 | t->idx++; 164 | if (t->idx >= t->data_sz) { 165 | printf("WARNING at line %d in %s: %s\n", en_line, file, 166 | "MAX_TIMER limit reached, bad things are about to happen."); 167 | fflush(stdout); 168 | t->data_sz *= 2; 169 | t->data = (struct time_it_timer_data*) 170 | realloc(t->data, sizeof(struct time_it_timer_data)*(t->data_sz)); 171 | } 172 | #endif 173 | } 174 | 175 | 176 | /** 177 | * @brief Outputs collected timing records to CSV file. 178 | * 179 | * @param[in] file_name name of the output file 180 | * @param[in] mode mode used to open the output file 181 | **/ 182 | int time_it_write_csv(const char *file_name, const char *mode) 183 | { 184 | #ifdef _TIMING_DATA 185 | timeit_t *t = time_it_ptr; 186 | int i, rc = 0; 187 | FILE *f = NULL; 188 | time_t curtime; 189 | struct tm *loctime; 190 | 191 | if (t == NULL) 192 | return 0; 193 | if (strncmp(mode,"w",1) && strncmp(mode,"a",1)) 194 | goto cleanup; 195 | if ((f = fopen(file_name, mode)) == NULL) 196 | goto cleanup; 197 | 198 | for(i = 0; i < t->idx; i++) { 199 | fprintf(f, "%s\t", t->data[i].comment); 200 | } 201 | fprintf(f, "HEAD\tGET\tPUT\tCOPY\tLIST\tPOST\tMetadata\tTimestamp\n"); 202 | 203 | for(i = 0; i < t->idx; i++){ 204 | fprintf(f, "%.7lf\t", t->data[i].time); 205 | } 206 | curtime = time(NULL); 207 | loctime = localtime(&curtime); 208 | fprintf(f, "%u\t%u\t%u\t%u\t%u\t%u\t%s\t%s", 209 | t->heads, t->gets, t->puts, t->copys, t->lists, 210 | t->posts, t->meta, asctime(loctime)); 211 | fclose(f); 212 | rc = 1; 213 | 214 | cleanup: 215 | return rc; 216 | #else 217 | return 0; 218 | #endif 219 | } 220 | -------------------------------------------------------------------------------- /tools/lib/time_it/time_it.h: -------------------------------------------------------------------------------- 1 | /** 2 | * @file 3 | * A simple library for collecting in-line timing data. 4 | * 5 | * @author Copyright (c) 2012, Mark Gondree 6 | * @author Copyright (c) 2012, Alric Althoff 7 | * @date 2008-2013 8 | * @copyright BSD 2-Clause License 9 | * See http://opensource.org/licenses/BSD-2-Clause 10 | **/ 11 | #ifndef __TIME_IT_H__ 12 | #define __TIME_IT_H__ 13 | 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | 22 | #define MAX_TIMERS 16384 ///< Total size of global list for timing records 23 | #define MAX_CSV_CELL 4096 ///< Length of CSV cell header metadata string 24 | 25 | /// A single timing record, from TIC to TOC 26 | struct time_it_timer_data { 27 | double time; 28 | const char *func; 29 | unsigned int st_line; 30 | unsigned int en_line; 31 | const char *file; 32 | const char *comment; 33 | }; 34 | 35 | /// All timing results and bookkeeping 36 | typedef struct { 37 | char *meta; 38 | unsigned int idx; 39 | unsigned short tic_no_toc; 40 | struct timeval tv_start; 41 | struct timeval tv_stop; 42 | unsigned int heads; 43 | unsigned int gets; 44 | unsigned int puts; 45 | unsigned int posts; 46 | unsigned int lists; 47 | unsigned int copys; 48 | unsigned int data_sz; 49 | struct time_it_timer_data *data; 50 | } timeit_t; 51 | 52 | extern timeit_t* time_it_ptr; 53 | 54 | /* 55 | * function prototypes 56 | */ 57 | void time_it_insert_time(timeit_t *t, double time, unsigned int st_line, 58 | unsigned int en_line, const char *func, 59 | const char *file, const char *comment); 60 | void time_it_start_timer(timeit_t *t, int st_line); 61 | void time_it_stop_timer(timeit_t *t, unsigned int en_line, const char *func, 62 | const char *file, const char *comment); 63 | int time_it_write_csv(const char *file_name, const char *mode); 64 | void time_it_init(void); 65 | void time_it_cleanup(void); 66 | 67 | 68 | 69 | #ifdef _TIMING_DATA 70 | // timing macros -------------------------------------------------------------- 71 | #define TIC { time_it_start_timer(time_it_ptr, __LINE__); } 72 | #define TOC(comment) { \ 73 | time_it_stop_timer(time_it_ptr, __LINE__, __func__, __FILE__, comment); \ 74 | } 75 | #define TIME_IT_INSERT(time, comment) { \ 76 | time_it_insert_time(time_it_ptr, time, __LINE__, __LINE__, \ 77 | __func__, __FILE__, comment); \ 78 | } 79 | #define PRINT_TIME { \ 80 | struct time_it_timeit_t_data cur = \ 81 | time_it_ptr->data[(time_it_ptr->idx)-1]; \ 82 | printf("%lf sec, at %s:%d-%d in %s\n", \ 83 | cur.time, cur.file, cur.st_line, cur.en_line, cur.func); \ 84 | }; 85 | #define PRINT_ALL_TIMES { \ 86 | int __i; \ 87 | struct time_it_timeit_t_data cur = {0}; \ 88 | printf("\n\n====== TIMES ======\n"); \ 89 | for(__i = 0; __i < time_it_ptr->idx; __i++){ \ 90 | cur = time_it_ptr->data[__i]; \ 91 | printf("%s:%d-%d in %s :: %lf :: %s\n", \ 92 | cur.file, cur.st_line, cur.en_line, \ 93 | cur.func, cur.time, cur.comment); \ 94 | } \ 95 | printf("===================\n"); \ 96 | }; 97 | #define TIME_IT_ADD_META(meta) { \ 98 | if(time_it_ptr) time_it_ptr->meta = meta; \ 99 | } 100 | #define INC_PUTS { if(time_it_ptr) (time_it_ptr->puts)++; } 101 | #define INC_GETS { if(time_it_ptr) (time_it_ptr->gets)++; } 102 | #define INC_LISTS { if(time_it_ptr) (time_it_ptr->lists)++; } 103 | #define INC_COPYS { if(time_it_ptr) (time_it_ptr->copys)++; } 104 | #define INC_POSTS { if(time_it_ptr) (time_it_ptr->posts)++; } 105 | #define INC_HEADS { if(time_it_ptr) (time_it_ptr->heads)++; } 106 | 107 | #else // timing stubs --------------------------------------------------------- 108 | #define TIC 109 | #define TOC(x) 110 | #define PRINT_TIME 111 | #define PRINT_ALL_TIMES 112 | #define TIME_IT_ADD_META(meta) 113 | #define INC_PUTS \ 114 | ; 115 | #define INC_GETS \ 116 | ; 117 | #define INC_LISTS \ 118 | ; 119 | #define INC_COPYS \ 120 | ; 121 | #define INC_POSTS \ 122 | ; 123 | #define INC_HEADS \ 124 | ; 125 | 126 | #endif //_TIMING_DATA 127 | #endif //__TIME_IT_H__ 128 | -------------------------------------------------------------------------------- /tools/pdp_bench: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gondree/libpdp/391648d2b11a4216d106ac22887b82adf5d8288d/tools/pdp_bench -------------------------------------------------------------------------------- /tools/s3_util.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/python 2 | """ 3 | A simple utility for managing remote S3 buckets for benchmarking libpdp. 4 | 5 | """ 6 | import os, sys 7 | import argparse 8 | import boto 9 | from boto.s3.connection import OrdinaryCallingFormat 10 | 11 | Id = "XXX" 12 | Key = "YYY" 13 | Host, Port = "192.168.99.173", "8080" 14 | 15 | DEFAULT_BUCKETS = ["libpdp_data"] 16 | 17 | ############################################################################### 18 | # 19 | # Support functions 20 | # 21 | def sizeof_fmt(num): 22 | for x in ['b ','KB','MB','GB','TB', 'XB']: 23 | if num < 1024.0: 24 | return "%3.1f %s" % (num, x) 25 | num /= 1024.0 26 | return "%3.1f %s" % (num, x) 27 | 28 | def list_all_buckets(s3, buckets=None): 29 | """List all the buckets""" 30 | for b in s3.get_all_buckets(): 31 | if buckets and not b.name in buckets: 32 | continue 33 | total = 0 34 | num = 0 35 | print "BUCKET:", b.name 36 | for k in b.list(): 37 | print "%s\t%s\t%010s\t%s" % ("-rwx---", k.last_modified, 38 | sizeof_fmt(k.size), k.name) 39 | num += 1 40 | total += k.size 41 | print "="*80 42 | print "\tTOTAL: \t%010s \t%i Files" % (sizeof_fmt(total), num) 43 | 44 | def delete_buckets(s3, buckets=None): 45 | """Delete buckets""" 46 | for b in buckets: 47 | try: 48 | print "Deleting %s" % b 49 | bo = s3.get_bucket(b) 50 | keys = bo.get_all_keys() 51 | for k in keys: 52 | bo.delete_key(k) 53 | s3.delete_bucket(b) 54 | except: 55 | print "The bucket %s couldn't be deleted." % b 56 | 57 | def create_buckets(s3, buckets=None): 58 | """Create buckets""" 59 | for b in buckets: 60 | try: 61 | print "Creating %s" % b 62 | s3.create_bucket(b) 63 | except: 64 | print "Couldn't create bucket %s" % b 65 | 66 | def get_file(s3, files=None): 67 | """Get a file from a bucket""" 68 | for f in files: 69 | bucket, key = f.split('/') 70 | bo = s3.lookup(bucket) 71 | key = bo.lookup(key) 72 | range = 'bytes=%s' % (args.range) 73 | data = key.get_contents_as_string(headers={'Range' : range }) 74 | print "Retrieved", len(data), "bytes of", f 75 | print "["+data+"]" 76 | 77 | ############################################################################### 78 | # 79 | # Main 80 | # 81 | parser = argparse.ArgumentParser(description='Perform actions with S3.') 82 | parser.add_argument("--mk", metavar="BUCKETS", nargs='*', default=False, 83 | help="Create the named buckets") 84 | parser.add_argument("--ls", metavar="BUCKETS", nargs='*', default=False, 85 | help="List data about the named buckets") 86 | parser.add_argument("--rm", metavar="BUCKETS", nargs='*', default=False, 87 | help="Remove the named buckets") 88 | parser.add_argument("--get", metavar="FILE", nargs='*', default=False, 89 | help="Get the named file") 90 | parser.add_argument("--range", metavar="RANGE", default="0-", 91 | help="Get a range of bytes for the file") 92 | args = parser.parse_args() 93 | 94 | if __name__ == "__main__": 95 | if 'S3_ACCESS_KEY_ID' in os.environ: 96 | Id = os.environ['S3_ACCESS_KEY_ID'] 97 | if 'S3_SECRET_ACCESS_KEY' in os.environ: 98 | Key = os.environ['S3_SECRET_ACCESS_KEY'] 99 | if 'S3_HOSTNAME' in os.environ: 100 | Host, Port = os.environ['S3_HOSTNAME'].split(':') 101 | Port = int(Port) 102 | 103 | s3 = boto.connect_s3(Id, Key, host=Host, port=Port, is_secure=False, 104 | calling_format=OrdinaryCallingFormat()) 105 | 106 | # 107 | # Bucket create 108 | # 109 | if not args.mk is False: 110 | if not args.mk: 111 | buckets = DEFAULT_BUCKETS 112 | else: 113 | buckets = args.mk 114 | create_buckets(s3, buckets) 115 | # 116 | # Bucket remove 117 | # 118 | if not args.rm is False: 119 | if not args.rm: 120 | buckets = DEFAULT_BUCKETS 121 | else: 122 | buckets = args.rm 123 | delete_buckets(s3, buckets) 124 | # 125 | # Bucket list 126 | # 127 | if not args.ls is False: 128 | list_all_buckets(s3, args.ls) 129 | 130 | # 131 | # Get file 132 | # 133 | if not args.get is False: 134 | get_file(s3, args.get) 135 | -------------------------------------------------------------------------------- /tools/test-keys/apdp.pri: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gondree/libpdp/391648d2b11a4216d106ac22887b82adf5d8288d/tools/test-keys/apdp.pri -------------------------------------------------------------------------------- /tools/test-keys/apdp.pub: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gondree/libpdp/391648d2b11a4216d106ac22887b82adf5d8288d/tools/test-keys/apdp.pub -------------------------------------------------------------------------------- /tools/test-keys/cpor.pri: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gondree/libpdp/391648d2b11a4216d106ac22887b82adf5d8288d/tools/test-keys/cpor.pri -------------------------------------------------------------------------------- /tools/test-keys/cpor.pub: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/gondree/libpdp/391648d2b11a4216d106ac22887b82adf5d8288d/tools/test-keys/cpor.pub --------------------------------------------------------------------------------