├── 3rdLibs ├── ChangeLog ├── LICENSE ├── Makefile ├── README.md ├── c_src ├── emdb_drv.c ├── mdb.c ├── mdb.h ├── midl.c ├── midl.h └── uthash.h ├── include └── emdb.hrl ├── rebar ├── rebar.config ├── src ├── emdb.app.src ├── emdb.erl ├── emdb_drv.erl └── emdb_oop.erl └── start.sh /3rdLibs: -------------------------------------------------------------------------------- 1 | o MDB: http://highlandsun.com/hyc/mdb/ 2 | source : from OpenLDAP git repository 3 | commit : 7333b6bdc90c49c332d4728c62c7904e64c95077 4 | license: OpenLDAP Public License 5 | 6 | o UTHash: http://uthash.sourceforge.net/ 7 | version: 1.9.6 8 | license: BSD revised 9 | -------------------------------------------------------------------------------- /ChangeLog: -------------------------------------------------------------------------------- 1 | o Sep 29th, 2012 2 | initial release 3 | 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | ------------------------------------------------------------------------- 2 | The OpenLDAP Public License 3 | Version 2.8, 17 August 2003 4 | 5 | Redistribution and use of this software and associated documentation 6 | ("Software"), with or without modification, are permitted provided 7 | that the following conditions are met: 8 | 9 | 1. Redistributions in source form must retain copyright statements 10 | and notices, 11 | 12 | 2. Redistributions in binary form must reproduce applicable copyright 13 | statements and notices, this list of conditions, and the following 14 | disclaimer in the documentation and/or other materials provided 15 | with the distribution, and 16 | 17 | 3. Redistributions must contain a verbatim copy of this document. 18 | 19 | The OpenLDAP Foundation may revise this license from time to time. 20 | Each revision is distinguished by a version number. You may use 21 | this Software under terms of this license revision or under the 22 | terms of any subsequent revision of the license. 23 | 24 | THIS SOFTWARE IS PROVIDED BY THE OPENLDAP FOUNDATION AND ITS 25 | CONTRIBUTORS ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, 26 | INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY 27 | AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT 28 | SHALL THE OPENLDAP FOUNDATION, ITS CONTRIBUTORS, OR THE AUTHOR(S) 29 | OR OWNER(S) OF THE SOFTWARE BE LIABLE FOR ANY DIRECT, INDIRECT, 30 | INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 31 | BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 32 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 33 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 34 | LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN 35 | ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 36 | POSSIBILITY OF SUCH DAMAGE. 37 | 38 | The names of the authors and copyright holders must not be used in 39 | advertising or otherwise to promote the sale, use or other dealing 40 | in this Software without specific, written prior permission. Title 41 | to copyright in this Software shall at all times remain with copyright 42 | holders. 43 | 44 | OpenLDAP is a registered trademark of the OpenLDAP Foundation. 45 | 46 | Copyright 1999-2003 The OpenLDAP Foundation, Redwood City, 47 | California, USA. All Rights Reserved. Permission to copy and 48 | distribute verbatim copies of this document is granted. 49 | ------------------------------------------------------------------------- 50 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # See LICENSE for licensing information. 2 | 3 | MODULE = emdb 4 | 5 | DIALYZER = dialyzer 6 | REBAR = rebar 7 | 8 | .PHONY: build clean 9 | 10 | all: ebin priv build 11 | 12 | ebin: 13 | @mkdir -p $@ 14 | 15 | priv: 16 | @mkdir -p $@ 17 | 18 | build: 19 | @$(REBAR) compile 20 | 21 | clean: 22 | @$(REBAR) clean 23 | @rm -f *~ */*~ erl_crash.dump 24 | @rm -rf ebin priv 25 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | EMDB ==== EMDB is a NIF library for the [Memory-Mapped Database](http://highlandsun.com/hyc/mdb/) database, aka. MDB. The main purpose of this package is to provide a **very fast** Riak [backend](http://wiki.basho.com/Storage-Backends.html). 2 | 3 | But this module could also be used as a general key-value store to replace: 4 | 5 | * [DETS](http://www.erlang.org/doc/man/dets.html) 6 | * TokyoCabinet: [TCERL](http://code.google.com/p/tcerl/) 7 | * [QDBM](http://fallabs.com/qdbm/) 8 | * [Bitcask](https://github.com/basho/bitcask) 9 | * [eLevelDB](https://github.com/basho/eleveldb) 10 | * [BerkleyDB](http://www.oracle.com/technetwork/products/berkeleydb/overview/index.html) 11 | * ... Requirements ------------ 12 | * Erlang R14B04+ * GCC 4.2+ or MS VisualStudio 2010+ Build ----- $ make API --- 13 | The following functions were implemented: 14 | 15 | * `open/1`: equivalent to `emdb:open(DirName, 10485760)`. 16 | * `open/2`: equivalent to `emdb:open(DirName, 10485760, 0)`. 17 | * `open/3`: creates a new MDB database. This call also re-open an already existing one. Arguments are: 18 | * DirName: database directory name 19 | * MapSize: database map size (see [map.hrl](http://gitorious.org/mdb/mdb/blobs/master/libraries/libmdb/mdb.h)) 20 | * EnvFlags: database environment flags (see [map.hrl](http://gitorious.org/mdb/mdb/blobs/master/libraries/libmdb/mdb.h)). The possible values are defined in **emdb.hrl**. 21 | * `close/2`: closes the database 22 | * `put/2`: inserts Key with value Val into the database. Assumes that the key is not present, 'key_exit' is returned otherwise. 23 | * `get/1`: retrieves the value stored with Key in the database. 24 | * `del/1`: Removes the key-value with key Key from database. 25 | * `update/2`: inserts Key with value Val into the database if the key is not present, otherwise updates Key to value Val. 26 | * `drop/1`: deletes all key-value pairs in the database. 27 | 28 | 29 | Usage ----- $ make 30 | 31 | $ ./start.sh 32 | 33 | %% create a new database 1> {ok, Handle} = emdb:open("/tmp/emdb1"). 34 | 35 | %% insert the key <<"a">> with value <<"1">> 2> ok = Handle:put(<<"a">>, <<"1">>). 36 | 37 | %% try to re-insert the same key <<"a">> 3> key_exist = Handle:put(<<"a">>, <<"2">>). 38 | 39 | %% add a new key-value pair 4> ok = Handle:put(<<"b">>, <<"2">>). 40 | 41 | %% search a non-existing key <<"c">> 5> none = Handle:get(<<"c">>). 42 | 43 | %% retrieve the value for key <<"b">> 6> {ok, <<"2">>} = Handle:get(<<"b">>). 44 | 45 | %% retrieve the value for key <<"a">> 7> {ok, <<"1">>} = Handle:get(<<"a">>). 46 | 47 | %% delete key <<"b">> 8> ok = Handle:del(<<"b">>). 48 | 49 | %% search a non-existing key <<"b">> 50 | 9> none = Handle:get(<<"b">>). 51 | 52 | %% delete a non-existing key <<"z">> 10> none = Handle:del(<<"z">>). 53 | 54 | %% ensure key <<"a">>'s value is still <<"1">> 11> {ok, <<"1">>} = Handle:get(<<"a">>). 55 | %% update the value for key <<"a">> 56 | 12> ok = Handle:update(<<"a">>, <<"7">>). 57 | 58 | %% check the new value for key <<"a">> 59 | 13> {ok, <<"7">>} = Handle:get(<<"a">>). 60 | 61 | %% delete all key-value pairs in the database 14> ok = Handle:drop(). 62 | 63 | %% try to retrieve key <<"a">> value 15> none = Handle:get(<<"a">>). 64 | 65 | %% close the database 16> ok = Handle:close(). 66 | 67 | ... 68 | 69 | 17> q(). 70 | 71 | 72 | ####Note: 73 | The code below creates a new database with **80GB** MapSize, **avoid fsync** 74 | after each commit (for max speed) and use the experimental **MDB_FIXEDMAP**. {ok, Handle} = emdb:open("/tmp/emdb2", 85899345920, ?MDB_NOSYNC bor ?MDB_FIXEDMAP). 75 | 76 | Performance ----------- For maximum speed, this library use only binaries for both keys and values. 77 | See the impressive [microbench](http://highlandsun.com/hyc/mdb/microbench/) against: 78 | 79 | * Google's LevelDB 80 | * SQLite 81 | * Kyoto TreeDB 82 | * BerkeleyDB 83 | 84 | MDB performs better on 64-bit arch. 85 | 86 | 87 | Supported OSes -------------- 88 | 89 | Should work on 32/64-bit architectures: 90 | 91 | * Linux 92 | * OSX 93 | * FreeBSD 94 | * Windows 95 | 96 | TODO ---- 97 | 98 | * Unit tests * PropEr testing 99 | * Bulk "writing" 100 | 101 | Volunteers are always welcome! Status 102 | ------ 103 | #### Work in progress. Don't use it in production! 104 | LICENSE ------- 105 | EMDB is Copyright (C) 2012 by Aleph Archives, and released under the [OpenLDAP](http://www.OpenLDAP.org/license.html) License. 106 | 107 | -------------------------------------------------------------------------------- /c_src/emdb_drv.c: -------------------------------------------------------------------------------- 1 | /* ------------------------------------------------------------------------- 2 | * This file is part of EMDB - Erlang MDB API 3 | * 4 | * Copyright (c) 2012 by Aleph Archives. All rights reserved. 5 | * 6 | * ------------------------------------------------------------------------- 7 | * Redistribution and use in source and binary forms, with or without 8 | * modification, are permitted only as authorized by the OpenLDAP 9 | * Public License. 10 | * 11 | * A copy of this license is available in the file LICENSE in the 12 | * top-level directory of the distribution or, alternatively, at 13 | * . 14 | * 15 | * Permission to use, copy, modify, and distribute this software for any 16 | * purpose with or without fee is hereby granted, provided that the above 17 | * copyright notice and this permission notice appear in all copies. 18 | * 19 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 20 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 21 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 22 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 23 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 24 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 25 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 26 | * -------------------------------------------------------------------------*/ 27 | 28 | /* 29 | * C headers 30 | */ 31 | 32 | #include /* for MAXPATHLEN constant */ 33 | #include /* for Erlang NIF interface */ 34 | #include "uthash.h" /* for uthash */ 35 | #include "mdb.h" /* for MDB interface */ 36 | 37 | 38 | 39 | #define FREE(p) (NULL == (p) ? 0 : (free(p), p = NULL)) 40 | 41 | #define FAIL_FAST(Error, Goto) \ 42 | do{ \ 43 | err = Error; \ 44 | goto Goto; \ 45 | }while(0) 46 | 47 | 48 | struct emdb_map_t { 49 | MDB_env * env; 50 | MDB_dbi dbi; 51 | 52 | UT_hash_handle hh; 53 | }; 54 | 55 | 56 | static ERL_NIF_TERM atom_ok; 57 | static ERL_NIF_TERM atom_none; 58 | 59 | 60 | static struct emdb_map_t * emdb_map = NULL; 61 | 62 | 63 | /* emdb ret */ 64 | #define EMDB_RET_KEY_EXIST "key_exist" 65 | 66 | /* emdb errors */ 67 | #define EMDB_MALLOC_ERR "error_malloc" 68 | #define EMDB_MAKE_BINARY_ERR "error_make_binary" 69 | #define EMDB_CREATE_ERR "error_create" 70 | #define EMDB_MAPSIZE_ERR "error_mapsize" 71 | #define EMDB_OPEN_ERR "error_open" 72 | #define EMDB_TXN_BEGIN_ERR "error_txn_begin" 73 | #define EMDB_TXN_COMMIT_ERR "error_txn_commit" 74 | #define EMDB_OPEN_DBI_ERR "error_open_dbi" 75 | #define EMDB_INVALID_HANDLE_ERR "error_invalid_handle" 76 | #define EMDB_PUT_ERR "error_put" 77 | #define EMDB_UPDATE_ERR "error_update" 78 | #define EMDB_KEY_NOT_FOUND "error_key_not_found" 79 | #define EMDB_DROP_ERR "error_drop" 80 | 81 | 82 | 83 | /* 84 | * Error handling callbacks 85 | */ 86 | 87 | static void emdb_free (struct emdb_map_t * emdb_obj) 88 | { 89 | FREE(emdb_obj); 90 | } 91 | 92 | 93 | /* 94 | * Driver callbacks 95 | */ 96 | 97 | static ERL_NIF_TERM emdb_open_nif (ErlNifEnv * env, 98 | int argc, const ERL_NIF_TERM argv[]) 99 | { 100 | char dirname [MAXPATHLEN]; 101 | struct emdb_map_t * node; 102 | MDB_txn * txn; 103 | char * err; 104 | ErlNifUInt64 mapsize; 105 | ErlNifUInt64 envflags; 106 | 107 | if (enif_get_string(env, argv[0], dirname, MAXPATHLEN, ERL_NIF_LATIN1) <= 0) 108 | return enif_make_badarg(env); 109 | 110 | if(! (node = calloc(1, sizeof(struct emdb_map_t)))) 111 | FAIL_FAST(EMDB_MALLOC_ERR, err3); 112 | 113 | if (mdb_env_create(& (node -> env))) 114 | FAIL_FAST(EMDB_CREATE_ERR, err2); 115 | 116 | if (! enif_get_uint64(env, argv[1], & mapsize)) 117 | return enif_make_badarg(env); 118 | 119 | if (mdb_env_set_mapsize(node -> env, mapsize)) 120 | FAIL_FAST(EMDB_MAPSIZE_ERR, err2); 121 | 122 | if (! enif_get_uint64(env, argv[2], & envflags)) 123 | return enif_make_badarg(env); 124 | 125 | if (mdb_env_open(node -> env, dirname, envflags, 0664)) 126 | FAIL_FAST(EMDB_OPEN_ERR, err2); 127 | 128 | if (mdb_txn_begin(node -> env, NULL, 0, & txn)) 129 | FAIL_FAST(EMDB_TXN_BEGIN_ERR, err2); 130 | 131 | if (mdb_open(txn, NULL, 0, & (node -> dbi))) 132 | FAIL_FAST(EMDB_OPEN_DBI_ERR, err1); 133 | 134 | if (mdb_txn_commit(txn)) 135 | FAIL_FAST(EMDB_TXN_COMMIT_ERR, err1); 136 | 137 | HASH_ADD_PTR(emdb_map, env, node); 138 | 139 | return enif_make_tuple(env, 2, 140 | atom_ok, 141 | enif_make_ulong(env, (unsigned long) node -> env)); 142 | 143 | err1: 144 | mdb_txn_abort(txn); 145 | err2: 146 | mdb_env_close(node -> env); 147 | err3: 148 | emdb_free(node); 149 | 150 | return enif_make_atom(env, err); 151 | } 152 | 153 | static ERL_NIF_TERM emdb_close_nif (ErlNifEnv * env, 154 | int argc, const ERL_NIF_TERM argv[]) 155 | { 156 | MDB_env * handle; 157 | struct emdb_map_t * node; 158 | unsigned long addr; 159 | 160 | if (! enif_get_ulong(env, argv[0], & addr)) 161 | return enif_make_badarg(env); 162 | 163 | handle = (MDB_env *) addr; 164 | 165 | HASH_FIND_PTR(emdb_map, & handle, node); 166 | if (NULL == node) 167 | return enif_make_atom(env, EMDB_INVALID_HANDLE_ERR); 168 | 169 | HASH_DEL(emdb_map, node); 170 | 171 | mdb_env_close(handle); 172 | emdb_free(node); 173 | 174 | return atom_ok; 175 | } 176 | 177 | 178 | static ERL_NIF_TERM emdb_put_nif (ErlNifEnv * env, 179 | int argc, const ERL_NIF_TERM argv[]) 180 | { 181 | ErlNifBinary key; 182 | ErlNifBinary val; 183 | 184 | MDB_val mkey; 185 | MDB_val mdata; 186 | 187 | MDB_env * handle; 188 | MDB_txn * txn; 189 | 190 | struct emdb_map_t * node; 191 | unsigned long addr; 192 | char * err; 193 | int ret; 194 | 195 | if (! enif_get_ulong(env, argv[0], & addr)) 196 | return enif_make_badarg(env); 197 | 198 | handle = (MDB_env *) addr; 199 | 200 | HASH_FIND_PTR(emdb_map, & handle, node); 201 | if (NULL == node) 202 | return enif_make_atom(env, EMDB_INVALID_HANDLE_ERR); 203 | 204 | if (! enif_inspect_iolist_as_binary(env, argv[1], &key)) 205 | return enif_make_badarg(env); 206 | 207 | if (! enif_inspect_iolist_as_binary(env, argv[2], &val)) 208 | return enif_make_badarg(env); 209 | 210 | if (mdb_txn_begin(handle, NULL, 0, & txn)) 211 | FAIL_FAST(EMDB_TXN_BEGIN_ERR, err2); 212 | 213 | mkey.mv_size = key.size; 214 | mkey.mv_data = key.data; 215 | mdata.mv_size = val.size; 216 | mdata.mv_data = val.data; 217 | 218 | ret = mdb_put(txn, node -> dbi, & mkey, & mdata, MDB_NOOVERWRITE); 219 | if (MDB_KEYEXIST == ret) 220 | FAIL_FAST(EMDB_RET_KEY_EXIST, err1); 221 | if (ret) 222 | FAIL_FAST(EMDB_PUT_ERR, err1); 223 | 224 | if (mdb_txn_commit(txn)) 225 | FAIL_FAST(EMDB_TXN_COMMIT_ERR, err1); 226 | 227 | return atom_ok; 228 | 229 | err1: 230 | mdb_txn_abort(txn); 231 | err2: 232 | return enif_make_atom(env, err); 233 | } 234 | 235 | 236 | static ERL_NIF_TERM emdb_get_nif (ErlNifEnv * env, 237 | int argc, const ERL_NIF_TERM argv[]) 238 | { 239 | ErlNifBinary key; 240 | ErlNifBinary val = {0}; 241 | ERL_NIF_TERM term; 242 | 243 | MDB_val mkey; 244 | MDB_val mdata; 245 | 246 | MDB_env * handle; 247 | MDB_txn * txn; 248 | 249 | struct emdb_map_t * node; 250 | char * err; 251 | unsigned long addr; 252 | 253 | if (! enif_get_ulong(env, argv[0], & addr)) 254 | return enif_make_badarg(env); 255 | 256 | handle = (MDB_env *) addr; 257 | 258 | HASH_FIND_PTR(emdb_map, & handle, node); 259 | if (NULL == node) 260 | return enif_make_atom(env, EMDB_INVALID_HANDLE_ERR); 261 | 262 | if (! enif_inspect_iolist_as_binary(env, argv[1], &key)) 263 | return enif_make_badarg(env); 264 | 265 | mkey.mv_size = key.size; 266 | mkey.mv_data = key.data; 267 | 268 | if (mdb_txn_begin(handle, NULL, 0, & txn)) 269 | FAIL_FAST(EMDB_TXN_BEGIN_ERR, err); 270 | 271 | if(mdb_get(txn, node -> dbi, & mkey, & mdata)) 272 | { 273 | mdb_txn_abort(txn); 274 | return atom_none; 275 | } 276 | 277 | val.size = mdata.mv_size; 278 | val.data = mdata.mv_data; 279 | 280 | term = enif_make_binary(env, &val); 281 | mdb_txn_abort(txn); 282 | 283 | if (! term) 284 | FAIL_FAST(EMDB_MAKE_BINARY_ERR, err); 285 | 286 | return enif_make_tuple(env, 2, 287 | atom_ok, 288 | term); 289 | 290 | err: 291 | return enif_make_atom(env, err); 292 | } 293 | 294 | 295 | static ERL_NIF_TERM emdb_del_nif (ErlNifEnv * env, 296 | int argc, const ERL_NIF_TERM argv[]) 297 | { 298 | ErlNifBinary key; 299 | 300 | MDB_val mkey; 301 | 302 | MDB_env * handle; 303 | MDB_txn * txn; 304 | 305 | struct emdb_map_t * node; 306 | char * err; 307 | unsigned long addr; 308 | int ret; 309 | 310 | if (! enif_get_ulong(env, argv[0], & addr)) 311 | return enif_make_badarg(env); 312 | 313 | handle = (MDB_env *) addr; 314 | 315 | HASH_FIND_PTR(emdb_map, & handle, node); 316 | if (NULL == node) 317 | return enif_make_atom(env, EMDB_INVALID_HANDLE_ERR); 318 | 319 | if (! enif_inspect_iolist_as_binary(env, argv[1], &key)) 320 | return enif_make_badarg(env); 321 | 322 | mkey.mv_size = key.size; 323 | mkey.mv_data = key.data; 324 | 325 | if (mdb_txn_begin(handle, NULL, 0, & txn)) 326 | FAIL_FAST(EMDB_TXN_BEGIN_ERR, err); 327 | 328 | ret = mdb_del(txn, node -> dbi, & mkey, NULL); 329 | 330 | if (mdb_txn_commit(txn)) 331 | FAIL_FAST(EMDB_TXN_COMMIT_ERR, err); 332 | 333 | if(ret) 334 | return atom_none; 335 | 336 | return atom_ok; 337 | 338 | err: 339 | return enif_make_atom(env, err); 340 | } 341 | 342 | 343 | static ERL_NIF_TERM emdb_update_nif (ErlNifEnv * env, 344 | int argc, const ERL_NIF_TERM argv[]) 345 | { 346 | ErlNifBinary key; 347 | ErlNifBinary val; 348 | 349 | MDB_val mkey; 350 | MDB_val mdata; 351 | 352 | MDB_env * handle; 353 | MDB_txn * txn; 354 | 355 | struct emdb_map_t * node; 356 | unsigned long addr; 357 | char * err; 358 | 359 | if (! enif_get_ulong(env, argv[0], & addr)) 360 | return enif_make_badarg(env); 361 | 362 | handle = (MDB_env *) addr; 363 | 364 | HASH_FIND_PTR(emdb_map, & handle, node); 365 | if (NULL == node) 366 | return enif_make_atom(env, EMDB_INVALID_HANDLE_ERR); 367 | 368 | if (! enif_inspect_iolist_as_binary(env, argv[1], &key)) 369 | return enif_make_badarg(env); 370 | 371 | if (! enif_inspect_iolist_as_binary(env, argv[2], &val)) 372 | return enif_make_badarg(env); 373 | 374 | if (mdb_txn_begin(handle, NULL, 0, & txn)) 375 | FAIL_FAST(EMDB_TXN_BEGIN_ERR, err2); 376 | 377 | mkey.mv_size = key.size; 378 | mkey.mv_data = key.data; 379 | mdata.mv_size = val.size; 380 | mdata.mv_data = val.data; 381 | 382 | if (mdb_put(txn, node -> dbi, & mkey, & mdata, 0)) 383 | FAIL_FAST(EMDB_UPDATE_ERR, err1); 384 | 385 | if (mdb_txn_commit(txn)) 386 | FAIL_FAST(EMDB_TXN_COMMIT_ERR, err1); 387 | 388 | return atom_ok; 389 | 390 | err1: 391 | mdb_txn_abort(txn); 392 | err2: 393 | return enif_make_atom(env, err); 394 | } 395 | 396 | 397 | static ERL_NIF_TERM emdb_drop_nif (ErlNifEnv * env, 398 | int argc, const ERL_NIF_TERM argv[]) 399 | { 400 | MDB_env * handle; 401 | MDB_txn * txn; 402 | struct emdb_map_t * node; 403 | unsigned long addr; 404 | char * err; 405 | int ret; 406 | 407 | if (! enif_get_ulong(env, argv[0], & addr)) 408 | return enif_make_badarg(env); 409 | 410 | handle = (MDB_env *) addr; 411 | 412 | HASH_FIND_PTR(emdb_map, & handle, node); 413 | if (NULL == node) 414 | return enif_make_atom(env, EMDB_INVALID_HANDLE_ERR); 415 | 416 | if (mdb_txn_begin(handle, NULL, 0, & txn)) 417 | FAIL_FAST(EMDB_TXN_BEGIN_ERR, err2); 418 | 419 | ret = mdb_drop(txn, node -> dbi, 0); 420 | if (ret) 421 | FAIL_FAST(EMDB_DROP_ERR, err1); 422 | 423 | if (mdb_txn_commit(txn)) 424 | FAIL_FAST(EMDB_TXN_COMMIT_ERR, err1); 425 | 426 | return atom_ok; 427 | 428 | err1: 429 | mdb_txn_abort(txn); 430 | 431 | err2: 432 | return enif_make_atom(env, err); 433 | } 434 | 435 | 436 | static int emdb_load(ErlNifEnv* env, void** priv, ERL_NIF_TERM load_info) 437 | { 438 | atom_ok = enif_make_atom(env, "ok"); 439 | atom_none = enif_make_atom(env, "none"); 440 | 441 | return (0); 442 | } 443 | 444 | static int emdb_reload(ErlNifEnv* env, void** priv, ERL_NIF_TERM info) 445 | { 446 | return (0); 447 | } 448 | 449 | 450 | static int emdb_upgrade(ErlNifEnv* env, void** priv, void** old_priv, ERL_NIF_TERM load_info) 451 | { 452 | return (0); 453 | } 454 | 455 | 456 | static void emdb_unload(ErlNifEnv* env, void* priv) 457 | { 458 | return; 459 | } 460 | 461 | 462 | 463 | static ErlNifFunc nif_funcs [] = { 464 | {"open", 3, emdb_open_nif}, 465 | {"close", 1, emdb_close_nif}, 466 | {"put", 3, emdb_put_nif}, 467 | {"get", 2, emdb_get_nif}, 468 | {"del", 2, emdb_del_nif}, 469 | {"update", 3, emdb_update_nif}, 470 | {"drop", 1, emdb_drop_nif} 471 | }; 472 | 473 | /* driver entry point */ 474 | ERL_NIF_INIT(emdb_drv, 475 | nif_funcs, 476 | & emdb_load, 477 | & emdb_reload, 478 | & emdb_upgrade, 479 | & emdb_unload) 480 | -------------------------------------------------------------------------------- /c_src/mdb.h: -------------------------------------------------------------------------------- 1 | /** @file mdb.h 2 | * @brief memory-mapped database library 3 | * 4 | * @mainpage MDB Memory-Mapped Database Manager 5 | * MDB is a Btree-based database management library modeled loosely on the 6 | * BerkeleyDB API, but much simplified. The entire database is exposed 7 | * in a memory map, and all data fetches return data directly 8 | * from the mapped memory, so no malloc's or memcpy's occur during 9 | * data fetches. As such, the library is extremely simple because it 10 | * requires no page caching layer of its own, and it is extremely high 11 | * performance and memory-efficient. It is also fully transactional with 12 | * full ACID semantics, and when the memory map is read-only, the 13 | * database integrity cannot be corrupted by stray pointer writes from 14 | * application code. 15 | * 16 | * The library is fully thread-aware and supports concurrent read/write 17 | * access from multiple processes and threads. Data pages use a copy-on- 18 | * write strategy so no active data pages are ever overwritten, which 19 | * also provides resistance to corruption and eliminates the need of any 20 | * special recovery procedures after a system crash. Writes are fully 21 | * serialized; only one write transaction may be active at a time, which 22 | * guarantees that writers can never deadlock. The database structure is 23 | * multi-versioned so readers run with no locks; writers cannot block 24 | * readers, and readers don't block writers. 25 | * 26 | * Unlike other well-known database mechanisms which use either write-ahead 27 | * transaction logs or append-only data writes, MDB requires no maintenance 28 | * during operation. Both write-ahead loggers and append-only databases 29 | * require periodic checkpointing and/or compaction of their log or database 30 | * files otherwise they grow without bound. MDB tracks free pages within 31 | * the database and re-uses them for new write operations, so the database 32 | * size does not grow without bound in normal use. 33 | * 34 | * The memory map can be used as a read-only or read-write map. It is 35 | * read-only by default as this provides total immunity to corruption. 36 | * Using read-write mode offers much higher write performance, but adds 37 | * the possibility for stray application writes thru pointers to silently 38 | * corrupt the database. Of course if your application code is known to 39 | * be bug-free (...) then this is not an issue. 40 | * 41 | * @author Howard Chu, Symas Corporation. 42 | * 43 | * @copyright Copyright 2011-2012 Howard Chu, Symas Corp. All rights reserved. 44 | * 45 | * Redistribution and use in source and binary forms, with or without 46 | * modification, are permitted only as authorized by the OpenLDAP 47 | * Public License. 48 | * 49 | * A copy of this license is available in the file LICENSE in the 50 | * top-level directory of the distribution or, alternatively, at 51 | * . 52 | * 53 | * @par Derived From: 54 | * This code is derived from btree.c written by Martin Hedenfalk. 55 | * 56 | * Copyright (c) 2009, 2010 Martin Hedenfalk 57 | * 58 | * Permission to use, copy, modify, and distribute this software for any 59 | * purpose with or without fee is hereby granted, provided that the above 60 | * copyright notice and this permission notice appear in all copies. 61 | * 62 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 63 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 64 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 65 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 66 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 67 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 68 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 69 | */ 70 | #ifndef _MDB_H_ 71 | #define _MDB_H_ 72 | 73 | #include 74 | 75 | #ifdef __cplusplus 76 | extern "C" { 77 | #endif 78 | 79 | /** @defgroup public Public API 80 | * @{ 81 | */ 82 | /** @defgroup Version Version Macros 83 | * @{ 84 | */ 85 | /** Library major version */ 86 | #define MDB_VERSION_MAJOR 0 87 | /** Library minor version */ 88 | #define MDB_VERSION_MINOR 9 89 | /** Library patch version */ 90 | #define MDB_VERSION_PATCH 4 91 | 92 | /** Combine args a,b,c into a single integer for easy version comparisons */ 93 | #define MDB_VERINT(a,b,c) (((a) << 24) | ((b) << 16) | (c)) 94 | 95 | /** The full library version as a single integer */ 96 | #define MDB_VERSION_FULL \ 97 | MDB_VERINT(MDB_VERSION_MAJOR,MDB_VERSION_MINOR,MDB_VERSION_PATCH) 98 | 99 | /** The release date of this library version */ 100 | #define MDB_VERSION_DATE "September 14, 2012" 101 | 102 | /** A stringifier for the version info */ 103 | #define MDB_VERSTR(a,b,c,d) "MDB " #a "." #b "." #c ": (" d ")" 104 | 105 | /** A helper for the stringifier macro */ 106 | #define MDB_VERFOO(a,b,c,d) MDB_VERSTR(a,b,c,d) 107 | 108 | /** The full library version as a C string */ 109 | #define MDB_VERSION_STRING \ 110 | MDB_VERFOO(MDB_VERSION_MAJOR,MDB_VERSION_MINOR,MDB_VERSION_PATCH,MDB_VERSION_DATE) 111 | /** @} */ 112 | 113 | /** @brief Opaque structure for a database environment. 114 | * 115 | * A DB environment supports multiple databases, all residing in the same 116 | * shared-memory map. 117 | */ 118 | typedef struct MDB_env MDB_env; 119 | 120 | /** @brief Opaque structure for a transaction handle. 121 | * 122 | * All database operations require a transaction handle. Transactions may be 123 | * read-only or read-write. 124 | */ 125 | typedef struct MDB_txn MDB_txn; 126 | 127 | /** @brief A handle for an individual database in the DB environment. */ 128 | typedef unsigned int MDB_dbi; 129 | 130 | /** @brief Opaque structure for navigating through a database */ 131 | typedef struct MDB_cursor MDB_cursor; 132 | 133 | /** @brief Generic structure used for passing keys and data in and out of the database. */ 134 | typedef struct MDB_val { 135 | size_t mv_size; /**< size of the data item */ 136 | void *mv_data; /**< address of the data item */ 137 | } MDB_val; 138 | 139 | /** @brief A callback function used to compare two keys in a database */ 140 | typedef int (MDB_cmp_func)(const MDB_val *a, const MDB_val *b); 141 | 142 | /** @brief A callback function used to relocate a position-dependent data item 143 | * in a fixed-address database. 144 | * 145 | * The \b newptr gives the item's desired address in 146 | * the memory map, and \b oldptr gives its previous address. The item's actual 147 | * data resides at the address in \b item. This callback is expected to walk 148 | * through the fields of the record in \b item and modify any 149 | * values based at the \b oldptr address to be relative to the \b newptr address. 150 | * @param[in,out] item The item that is to be relocated. 151 | * @param[in] oldptr The previous address. 152 | * @param[in] newptr The new address to relocate to. 153 | * @param[in] relctx An application-provided context, set by #mdb_set_relctx(). 154 | * @todo This feature is currently unimplemented. 155 | */ 156 | typedef void (MDB_rel_func)(MDB_val *item, void *oldptr, void *newptr, void *relctx); 157 | 158 | /** @defgroup mdb_env Environment Flags 159 | * @{ 160 | */ 161 | /** mmap at a fixed address */ 162 | #define MDB_FIXEDMAP 0x01 163 | /** no environment directory */ 164 | #define MDB_NOSUBDIR 0x02 165 | /** don't fsync after commit */ 166 | #define MDB_NOSYNC 0x10000 167 | /** read only */ 168 | #define MDB_RDONLY 0x20000 169 | /** don't fsync metapage after commit */ 170 | #define MDB_NOMETASYNC 0x40000 171 | /** use writable mmap */ 172 | #define MDB_WRITEMAP 0x80000 173 | /** use asynchronous msync */ 174 | #define MDB_MAPASYNC 0x100000 175 | /** @} */ 176 | 177 | /** @defgroup mdb_open Database Flags 178 | * @{ 179 | */ 180 | /** use reverse string keys */ 181 | #define MDB_REVERSEKEY 0x02 182 | /** use sorted duplicates */ 183 | #define MDB_DUPSORT 0x04 184 | /** numeric keys in native byte order. 185 | * The keys must all be of the same size. */ 186 | #define MDB_INTEGERKEY 0x08 187 | /** with #MDB_DUPSORT, sorted dup items have fixed size */ 188 | #define MDB_DUPFIXED 0x10 189 | /** with #MDB_DUPSORT, dups are numeric in native byte order */ 190 | #define MDB_INTEGERDUP 0x20 191 | /** with #MDB_DUPSORT, use reverse string dups */ 192 | #define MDB_REVERSEDUP 0x40 193 | /** create DB if not already existing */ 194 | #define MDB_CREATE 0x40000 195 | /** @} */ 196 | 197 | /** @defgroup mdb_put Write Flags 198 | * @{ 199 | */ 200 | /** For put: Don't write if the key already exists. */ 201 | #define MDB_NOOVERWRITE 0x10 202 | /** Only for #MDB_DUPSORT
203 | * For put: don't write if the key and data pair already exist.
204 | * For mdb_cursor_del: remove all duplicate data items. 205 | */ 206 | #define MDB_NODUPDATA 0x20 207 | /** For mdb_cursor_put: overwrite the current key/data pair */ 208 | #define MDB_CURRENT 0x40 209 | /** For put: Just reserve space for data, don't copy it. Return a 210 | * pointer to the reserved space. 211 | */ 212 | #define MDB_RESERVE 0x10000 213 | /** Data is being appended, don't split full pages. */ 214 | #define MDB_APPEND 0x20000 215 | /** Duplicate data is being appended, don't split full pages. */ 216 | #define MDB_APPENDDUP 0x40000 217 | /** Store multiple data items in one call. */ 218 | #define MDB_MULTIPLE 0x80000 219 | /* @} */ 220 | 221 | /** @brief Cursor Get operations. 222 | * 223 | * This is the set of all operations for retrieving data 224 | * using a cursor. 225 | */ 226 | typedef enum MDB_cursor_op { 227 | MDB_FIRST, /**< Position at first key/data item */ 228 | MDB_FIRST_DUP, /**< Position at first data item of current key. 229 | Only for #MDB_DUPSORT */ 230 | MDB_GET_BOTH, /**< Position at key/data pair. Only for #MDB_DUPSORT */ 231 | MDB_GET_BOTH_RANGE, /**< position at key, nearest data. Only for #MDB_DUPSORT */ 232 | MDB_GET_CURRENT, /**< Return key/data at current cursor position */ 233 | MDB_GET_MULTIPLE, /**< Return all the duplicate data items at the current 234 | cursor position. Only for #MDB_DUPFIXED */ 235 | MDB_LAST, /**< Position at last key/data item */ 236 | MDB_LAST_DUP, /**< Position at last data item of current key. 237 | Only for #MDB_DUPSORT */ 238 | MDB_NEXT, /**< Position at next data item */ 239 | MDB_NEXT_DUP, /**< Position at next data item of current key. 240 | Only for #MDB_DUPSORT */ 241 | MDB_NEXT_MULTIPLE, /**< Return all duplicate data items at the next 242 | cursor position. Only for #MDB_DUPFIXED */ 243 | MDB_NEXT_NODUP, /**< Position at first data item of next key. 244 | Only for #MDB_DUPSORT */ 245 | MDB_PREV, /**< Position at previous data item */ 246 | MDB_PREV_DUP, /**< Position at previous data item of current key. 247 | Only for #MDB_DUPSORT */ 248 | MDB_PREV_NODUP, /**< Position at last data item of previous key. 249 | Only for #MDB_DUPSORT */ 250 | MDB_SET, /**< Position at specified key */ 251 | MDB_SET_KEY, /**< Position at specified key, return key + data */ 252 | MDB_SET_RANGE /**< Position at first key greater than or equal to specified key. */ 253 | } MDB_cursor_op; 254 | 255 | /** @defgroup errors Return Codes 256 | * 257 | * BerkeleyDB uses -30800 to -30999, we'll go under them 258 | * @{ 259 | */ 260 | /** Successful result */ 261 | #define MDB_SUCCESS 0 262 | /** key/data pair already exists */ 263 | #define MDB_KEYEXIST (-30799) 264 | /** key/data pair not found (EOF) */ 265 | #define MDB_NOTFOUND (-30798) 266 | /** Requested page not found - this usually indicates corruption */ 267 | #define MDB_PAGE_NOTFOUND (-30797) 268 | /** Located page was wrong type */ 269 | #define MDB_CORRUPTED (-30796) 270 | /** Update of meta page failed, probably I/O error */ 271 | #define MDB_PANIC (-30795) 272 | /** Environment version mismatch */ 273 | #define MDB_VERSION_MISMATCH (-30794) 274 | /** File is not a valid MDB file */ 275 | #define MDB_INVALID (-30793) 276 | /** Environment mapsize reached */ 277 | #define MDB_MAP_FULL (-30792) 278 | /** Environment maxdbs reached */ 279 | #define MDB_DBS_FULL (-30791) 280 | /** Environment maxreaders reached */ 281 | #define MDB_READERS_FULL (-30790) 282 | /** Too many TLS keys in use - Windows only */ 283 | #define MDB_TLS_FULL (-30789) 284 | /** Nested txn has too many dirty pages */ 285 | #define MDB_TXN_FULL (-30788) 286 | /** Cursor stack too deep - internal error */ 287 | #define MDB_CURSOR_FULL (-30787) 288 | /** Page has not enough space - internal error */ 289 | #define MDB_PAGE_FULL (-30786) 290 | #define MDB_LAST_ERRCODE MDB_PAGE_FULL 291 | /** @} */ 292 | 293 | /** @brief Statistics for a database in the environment */ 294 | typedef struct MDB_stat { 295 | unsigned int ms_psize; /**< Size of a database page. 296 | This is currently the same for all databases. */ 297 | unsigned int ms_depth; /**< Depth (height) of the B-tree */ 298 | size_t ms_branch_pages; /**< Number of internal (non-leaf) pages */ 299 | size_t ms_leaf_pages; /**< Number of leaf pages */ 300 | size_t ms_overflow_pages; /**< Number of overflow pages */ 301 | size_t ms_entries; /**< Number of data items */ 302 | } MDB_stat; 303 | 304 | /** @brief Return the mdb library version information. 305 | * 306 | * @param[out] major if non-NULL, the library major version number is copied here 307 | * @param[out] minor if non-NULL, the library minor version number is copied here 308 | * @param[out] patch if non-NULL, the library patch version number is copied here 309 | * @retval "version string" The library version as a string 310 | */ 311 | char *mdb_version(int *major, int *minor, int *patch); 312 | 313 | /** @brief Return a string describing a given error code. 314 | * 315 | * This function is a superset of the ANSI C X3.159-1989 (ANSI C) strerror(3) 316 | * function. If the error code is greater than or equal to 0, then the string 317 | * returned by the system function strerror(3) is returned. If the error code 318 | * is less than 0, an error string corresponding to the MDB library error is 319 | * returned. See @ref errors for a list of MDB-specific error codes. 320 | * @param[in] err The error code 321 | * @retval "error message" The description of the error 322 | */ 323 | char *mdb_strerror(int err); 324 | 325 | /** @brief Create an MDB environment handle. 326 | * 327 | * This function allocates memory for a #MDB_env structure. To release 328 | * the allocated memory and discard the handle, call #mdb_env_close(). 329 | * Before the handle may be used, it must be opened using #mdb_env_open(). 330 | * Various other options may also need to be set before opening the handle, 331 | * e.g. #mdb_env_set_mapsize(), #mdb_env_set_maxreaders(), #mdb_env_set_maxdbs(), 332 | * depending on usage requirements. 333 | * @param[out] env The address where the new handle will be stored 334 | * @return A non-zero error value on failure and 0 on success. 335 | */ 336 | int mdb_env_create(MDB_env **env); 337 | 338 | /** @brief Open an environment handle. 339 | * 340 | * If this function fails, #mdb_env_close() must be called to discard the #MDB_env handle. 341 | * @param[in] env An environment handle returned by #mdb_env_create() 342 | * @param[in] path The directory in which the database files reside. This 343 | * directory must already exist and be writable. 344 | * @param[in] flags Special options for this environment. This parameter 345 | * must be set to 0 or by bitwise OR'ing together one or more of the 346 | * values described here. 347 | *
    348 | *
  • #MDB_FIXEDMAP 349 | * use a fixed address for the mmap region. This flag must be specified 350 | * when creating the environment, and is stored persistently in the environment. 351 | * If successful, the memory map will always reside at the same virtual address 352 | * and pointers used to reference data items in the database will be constant 353 | * across multiple invocations. This option may not always work, depending on 354 | * how the operating system has allocated memory to shared libraries and other uses. 355 | * The feature is highly experimental. 356 | *
  • #MDB_NOSUBDIR 357 | * By default, MDB creates its environment in a directory whose 358 | * pathname is given in \b path, and creates its data and lock files 359 | * under that directory. With this option, \b path is used as-is for 360 | * the database main data file. The database lock file is the \b path 361 | * with "-lock" appended. 362 | *
  • #MDB_NOSYNC 363 | * Don't perform a synchronous flush after committing a transaction. This means 364 | * transactions will exhibit the ACI (atomicity, consistency, and isolation) 365 | * properties, but not D (durability); that is database integrity will be 366 | * maintained but it is possible some number of the most recently committed 367 | * transactions may be undone after a system crash. The number of transactions 368 | * at risk is governed by how often the system flushes dirty buffers to disk 369 | * and how often #mdb_env_sync() is called. This flag may be changed 370 | * at any time using #mdb_env_set_flags(). 371 | *
  • #MDB_NOMETASYNC 372 | * Don't perform a synchronous flush of the meta page after committing 373 | * a transaction. This is similar to the #MDB_NOSYNC case, but safer 374 | * because the transaction data is still flushed. The meta page for any 375 | * transaction N will be flushed by the data flush of transaction N+1. 376 | * In case of a system crash, the last committed transaction may be 377 | * lost. This flag may be changed at any time using #mdb_env_set_flags(). 378 | *
  • #MDB_RDONLY 379 | * Open the environment in read-only mode. No write operations will be allowed. 380 | *
381 | * @param[in] mode The UNIX permissions to set on created files. This parameter 382 | * is ignored on Windows. 383 | * @return A non-zero error value on failure and 0 on success. Some possible 384 | * errors are: 385 | *
    386 | *
  • #MDB_VERSION_MISMATCH - the version of the MDB library doesn't match the 387 | * version that created the database environment. 388 | *
  • EINVAL - the environment file headers are corrupted. 389 | *
  • ENOENT - the directory specified by the path parameter doesn't exist. 390 | *
  • EACCES - the user didn't have permission to access the environment files. 391 | *
  • EAGAIN - the environment was locked by another process. 392 | *
393 | */ 394 | int mdb_env_open(MDB_env *env, const char *path, unsigned int flags, mode_t mode); 395 | 396 | /** @brief Return statistics about the MDB environment. 397 | * 398 | * @param[in] env An environment handle returned by #mdb_env_create() 399 | * @param[out] stat The address of an #MDB_stat structure 400 | * where the statistics will be copied 401 | */ 402 | int mdb_env_stat(MDB_env *env, MDB_stat *stat); 403 | 404 | /** @brief Flush the data buffers to disk. 405 | * 406 | * Data is always written to disk when #mdb_txn_commit() is called, 407 | * but the operating system may keep it buffered. MDB always flushes 408 | * the OS buffers upon commit as well, unless the environment was 409 | * opened with #MDB_NOSYNC. 410 | * @param[in] env An environment handle returned by #mdb_env_create() 411 | * @param[in] force If non-zero, force the flush to occur. Otherwise 412 | * if the environment has the #MDB_NOSYNC flag set the flushes 413 | * will be omitted. 414 | * @return A non-zero error value on failure and 0 on success. Some possible 415 | * errors are: 416 | *
    417 | *
  • EINVAL - an invalid parameter was specified. 418 | *
  • EIO - an error occurred during synchronization. 419 | *
420 | */ 421 | int mdb_env_sync(MDB_env *env, int force); 422 | 423 | /** @brief Close the environment and release the memory map. 424 | * 425 | * Only a single thread may call this function. All transactions, databases, 426 | * and cursors must already be closed before calling this function. Attempts to 427 | * use any such handles after calling this function will cause a SIGSEGV. 428 | * The environment handle will be freed and must not be used again after this call. 429 | * @param[in] env An environment handle returned by #mdb_env_create() 430 | */ 431 | void mdb_env_close(MDB_env *env); 432 | 433 | /** @brief Set environment flags. 434 | * 435 | * This may be used to set some flags that weren't already set during 436 | * #mdb_env_open(), or to unset these flags. 437 | * @param[in] env An environment handle returned by #mdb_env_create() 438 | * @param[in] flags The flags to change, bitwise OR'ed together 439 | * @param[in] onoff A non-zero value sets the flags, zero clears them. 440 | * @return A non-zero error value on failure and 0 on success. Some possible 441 | * errors are: 442 | *
    443 | *
  • EINVAL - an invalid parameter was specified. 444 | *
445 | */ 446 | int mdb_env_set_flags(MDB_env *env, unsigned int flags, int onoff); 447 | 448 | /** @brief Get environment flags. 449 | * 450 | * @param[in] env An environment handle returned by #mdb_env_create() 451 | * @param[out] flags The address of an integer to store the flags 452 | * @return A non-zero error value on failure and 0 on success. Some possible 453 | * errors are: 454 | *
    455 | *
  • EINVAL - an invalid parameter was specified. 456 | *
457 | */ 458 | int mdb_env_get_flags(MDB_env *env, unsigned int *flags); 459 | 460 | /** @brief Return the path that was used in #mdb_env_open(). 461 | * 462 | * @param[in] env An environment handle returned by #mdb_env_create() 463 | * @param[out] path Address of a string pointer to contain the path. This 464 | * is the actual string in the environment, not a copy. It should not be 465 | * altered in any way. 466 | * @return A non-zero error value on failure and 0 on success. Some possible 467 | * errors are: 468 | *
    469 | *
  • EINVAL - an invalid parameter was specified. 470 | *
471 | */ 472 | int mdb_env_get_path(MDB_env *env, const char **path); 473 | 474 | /** @brief Set the size of the memory map to use for this environment. 475 | * 476 | * The size should be a multiple of the OS page size. The default is 477 | * 10485760 bytes. The size of the memory map is also the maximum size 478 | * of the database. The value should be chosen as large as possible, 479 | * to accommodate future growth of the database. 480 | * This function may only be called after #mdb_env_create() and before #mdb_env_open(). 481 | * @param[in] env An environment handle returned by #mdb_env_create() 482 | * @param[in] size The size in bytes 483 | * @return A non-zero error value on failure and 0 on success. Some possible 484 | * errors are: 485 | *
    486 | *
  • EINVAL - an invalid parameter was specified, or the environment is already open. 487 | *
488 | */ 489 | int mdb_env_set_mapsize(MDB_env *env, size_t size); 490 | 491 | /** @brief Set the maximum number of threads for the environment. 492 | * 493 | * This defines the number of slots in the lock table that is used to track readers in the 494 | * the environment. The default is 126. 495 | * This function may only be called after #mdb_env_create() and before #mdb_env_open(). 496 | * @param[in] env An environment handle returned by #mdb_env_create() 497 | * @param[in] readers The maximum number of threads 498 | * @return A non-zero error value on failure and 0 on success. Some possible 499 | * errors are: 500 | *
    501 | *
  • EINVAL - an invalid parameter was specified, or the environment is already open. 502 | *
503 | */ 504 | int mdb_env_set_maxreaders(MDB_env *env, unsigned int readers); 505 | 506 | /** @brief Get the maximum number of threads for the environment. 507 | * 508 | * @param[in] env An environment handle returned by #mdb_env_create() 509 | * @param[out] readers Address of an integer to store the number of readers 510 | * @return A non-zero error value on failure and 0 on success. Some possible 511 | * errors are: 512 | *
    513 | *
  • EINVAL - an invalid parameter was specified. 514 | *
515 | */ 516 | int mdb_env_get_maxreaders(MDB_env *env, unsigned int *readers); 517 | 518 | /** @brief Set the maximum number of databases for the environment. 519 | * 520 | * This function is only needed if multiple databases will be used in the 521 | * environment. Simpler applications that only use a single database can ignore 522 | * this option. 523 | * This function may only be called after #mdb_env_create() and before #mdb_env_open(). 524 | * @param[in] env An environment handle returned by #mdb_env_create() 525 | * @param[in] dbs The maximum number of databases 526 | * @return A non-zero error value on failure and 0 on success. Some possible 527 | * errors are: 528 | *
    529 | *
  • EINVAL - an invalid parameter was specified, or the environment is already open. 530 | *
531 | */ 532 | int mdb_env_set_maxdbs(MDB_env *env, MDB_dbi dbs); 533 | 534 | /** @brief Create a transaction for use with the environment. 535 | * 536 | * The transaction handle may be discarded using #mdb_txn_abort() or #mdb_txn_commit(). 537 | * @note Transactions may not span threads; a transaction must only be used by a 538 | * single thread. Also, a thread may only have a single transaction. 539 | * @note Cursors may not span transactions; each cursor must be opened and closed 540 | * within a single transaction. 541 | * @param[in] env An environment handle returned by #mdb_env_create() 542 | * @param[in] parent If this parameter is non-NULL, the new transaction 543 | * will be a nested transaction, with the transaction indicated by \b parent 544 | * as its parent. Transactions may be nested to any level. A parent 545 | * transaction may not issue any other operations besides mdb_txn_begin, 546 | * mdb_txn_abort, or mdb_txn_commit while it has active child transactions. 547 | * @param[in] flags Special options for this transaction. This parameter 548 | * must be set to 0 or by bitwise OR'ing together one or more of the 549 | * values described here. 550 | *
    551 | *
  • #MDB_RDONLY 552 | * This transaction will not perform any write operations. 553 | *
554 | * @param[out] txn Address where the new #MDB_txn handle will be stored 555 | * @return A non-zero error value on failure and 0 on success. Some possible 556 | * errors are: 557 | *
    558 | *
  • #MDB_PANIC - a fatal error occurred earlier and the environment 559 | * must be shut down. 560 | *
  • ENOMEM - out of memory, or a read-only transaction was requested and 561 | * the reader lock table is full. See #mdb_env_set_maxreaders(). 562 | *
563 | */ 564 | int mdb_txn_begin(MDB_env *env, MDB_txn *parent, unsigned int flags, MDB_txn **txn); 565 | 566 | /** @brief Commit all the operations of a transaction into the database. 567 | * 568 | * All cursors opened within the transaction will be closed by this call. The cursors 569 | * and transaction handle will be freed and must not be used again after this call. 570 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 571 | * @return A non-zero error value on failure and 0 on success. Some possible 572 | * errors are: 573 | *
    574 | *
  • EINVAL - an invalid parameter was specified. 575 | *
  • ENOSPC - no more disk space. 576 | *
  • EIO - a low-level I/O error occurred while writing. 577 | *
  • ENOMEM - the transaction is nested and could not be merged into its parent. 578 | *
579 | */ 580 | int mdb_txn_commit(MDB_txn *txn); 581 | 582 | /** @brief Abandon all the operations of the transaction instead of saving them. 583 | * 584 | * All cursors opened within the transaction will be closed by this call. The cursors 585 | * and transaction handle will be freed and must not be used again after this call. 586 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 587 | */ 588 | void mdb_txn_abort(MDB_txn *txn); 589 | 590 | /** @brief Reset a read-only transaction. 591 | * 592 | * This releases the current reader lock but doesn't free the 593 | * transaction handle, allowing it to be used again later by #mdb_txn_renew(). 594 | * It otherwise has the same effect as #mdb_txn_abort() but saves some memory 595 | * allocation/deallocation overhead if a thread is going to start a new 596 | * read-only transaction again soon. 597 | * All cursors opened within the transaction must be closed before the transaction 598 | * is reset. 599 | * Reader locks generally don't interfere with writers, but they keep old 600 | * versions of database pages allocated. Thus they prevent the old pages 601 | * from being reused when writers commit new data, and so under heavy load 602 | * the database size may grow much more rapidly than otherwise. 603 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 604 | */ 605 | void mdb_txn_reset(MDB_txn *txn); 606 | 607 | /** @brief Renew a read-only transaction. 608 | * 609 | * This acquires a new reader lock for a transaction handle that had been 610 | * released by #mdb_txn_reset(). It must be called before a reset transaction 611 | * may be used again. 612 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 613 | * @return A non-zero error value on failure and 0 on success. Some possible 614 | * errors are: 615 | *
    616 | *
  • #MDB_PANIC - a fatal error occurred earlier and the environment 617 | * must be shut down. 618 | *
  • EINVAL - an invalid parameter was specified. 619 | *
620 | */ 621 | int mdb_txn_renew(MDB_txn *txn); 622 | 623 | /** @brief Open a database in the environment. 624 | * 625 | * The database handle may be discarded by calling #mdb_close(). The 626 | * database handle resides in the shared environment, it is not owned 627 | * by the given transaction. Only one thread should call this function; 628 | * it is not mutex-protected in a read-only transaction. 629 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 630 | * @param[in] name The name of the database to open. If only a single 631 | * database is needed in the environment, this value may be NULL. 632 | * @param[in] flags Special options for this database. This parameter 633 | * must be set to 0 or by bitwise OR'ing together one or more of the 634 | * values described here. 635 | *
    636 | *
  • #MDB_REVERSEKEY 637 | * Keys are strings to be compared in reverse order, from the end 638 | * of the strings to the beginning. By default, Keys are treated as strings and 639 | * compared from beginning to end. 640 | *
  • #MDB_DUPSORT 641 | * Duplicate keys may be used in the database. (Or, from another perspective, 642 | * keys may have multiple data items, stored in sorted order.) By default 643 | * keys must be unique and may have only a single data item. 644 | *
  • #MDB_INTEGERKEY 645 | * Keys are binary integers in native byte order. Setting this option 646 | * requires all keys to be the same size, typically sizeof(int) 647 | * or sizeof(size_t). 648 | *
  • #MDB_DUPFIXED 649 | * This flag may only be used in combination with #MDB_DUPSORT. This option 650 | * tells the library that the data items for this database are all the same 651 | * size, which allows further optimizations in storage and retrieval. When 652 | * all data items are the same size, the #MDB_GET_MULTIPLE and #MDB_NEXT_MULTIPLE 653 | * cursor operations may be used to retrieve multiple items at once. 654 | *
  • #MDB_INTEGERDUP 655 | * This option specifies that duplicate data items are also integers, and 656 | * should be sorted as such. 657 | *
  • #MDB_REVERSEDUP 658 | * This option specifies that duplicate data items should be compared as 659 | * strings in reverse order. 660 | *
  • #MDB_CREATE 661 | * Create the named database if it doesn't exist. This option is not 662 | * allowed in a read-only transaction or a read-only environment. 663 | *
664 | * @param[out] dbi Address where the new #MDB_dbi handle will be stored 665 | * @return A non-zero error value on failure and 0 on success. Some possible 666 | * errors are: 667 | *
    668 | *
  • #MDB_NOTFOUND - the specified database doesn't exist in the environment 669 | * and #MDB_CREATE was not specified. 670 | *
  • ENFILE - too many databases have been opened. See #mdb_env_set_maxdbs(). 671 | *
672 | */ 673 | int mdb_open(MDB_txn *txn, const char *name, unsigned int flags, MDB_dbi *dbi); 674 | 675 | /** @brief Retrieve statistics for a database. 676 | * 677 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 678 | * @param[in] dbi A database handle returned by #mdb_open() 679 | * @param[out] stat The address of an #MDB_stat structure 680 | * where the statistics will be copied 681 | * @return A non-zero error value on failure and 0 on success. Some possible 682 | * errors are: 683 | *
    684 | *
  • EINVAL - an invalid parameter was specified. 685 | *
686 | */ 687 | int mdb_stat(MDB_txn *txn, MDB_dbi dbi, MDB_stat *stat); 688 | 689 | /** @brief Close a database handle. 690 | * 691 | * This call is not mutex protected. Handles should only be closed by 692 | * a single thread, and only if no other threads are going to reference 693 | * the database handle any further. 694 | * @param[in] env An environment handle returned by #mdb_env_create() 695 | * @param[in] dbi A database handle returned by #mdb_open() 696 | */ 697 | void mdb_close(MDB_env *env, MDB_dbi dbi); 698 | 699 | /** @brief Delete a database and/or free all its pages. 700 | * 701 | * If the \b del parameter is non-zero the DB handle will be closed 702 | * and the DB will be deleted. 703 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 704 | * @param[in] dbi A database handle returned by #mdb_open() 705 | * @param[in] del non-zero to delete the DB from the environment, 706 | * otherwise just free its pages. 707 | * @return A non-zero error value on failure and 0 on success. 708 | */ 709 | int mdb_drop(MDB_txn *txn, MDB_dbi dbi, int del); 710 | 711 | /** @brief Set a custom key comparison function for a database. 712 | * 713 | * The comparison function is called whenever it is necessary to compare a 714 | * key specified by the application with a key currently stored in the database. 715 | * If no comparison function is specified, and no special key flags were specified 716 | * with #mdb_open(), the keys are compared lexically, with shorter keys collating 717 | * before longer keys. 718 | * @warning This function must be called before any data access functions are used, 719 | * otherwise data corruption may occur. The same comparison function must be used by every 720 | * program accessing the database, every time the database is used. 721 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 722 | * @param[in] dbi A database handle returned by #mdb_open() 723 | * @param[in] cmp A #MDB_cmp_func function 724 | * @return A non-zero error value on failure and 0 on success. Some possible 725 | * errors are: 726 | *
    727 | *
  • EINVAL - an invalid parameter was specified. 728 | *
729 | */ 730 | int mdb_set_compare(MDB_txn *txn, MDB_dbi dbi, MDB_cmp_func *cmp); 731 | 732 | /** @brief Set a custom data comparison function for a #MDB_DUPSORT database. 733 | * 734 | * This comparison function is called whenever it is necessary to compare a data 735 | * item specified by the application with a data item currently stored in the database. 736 | * This function only takes effect if the database was opened with the #MDB_DUPSORT 737 | * flag. 738 | * If no comparison function is specified, and no special key flags were specified 739 | * with #mdb_open(), the data items are compared lexically, with shorter items collating 740 | * before longer items. 741 | * @warning This function must be called before any data access functions are used, 742 | * otherwise data corruption may occur. The same comparison function must be used by every 743 | * program accessing the database, every time the database is used. 744 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 745 | * @param[in] dbi A database handle returned by #mdb_open() 746 | * @param[in] cmp A #MDB_cmp_func function 747 | * @return A non-zero error value on failure and 0 on success. Some possible 748 | * errors are: 749 | *
    750 | *
  • EINVAL - an invalid parameter was specified. 751 | *
752 | */ 753 | int mdb_set_dupsort(MDB_txn *txn, MDB_dbi dbi, MDB_cmp_func *cmp); 754 | 755 | /** @brief Set a relocation function for a #MDB_FIXEDMAP database. 756 | * 757 | * @todo The relocation function is called whenever it is necessary to move the data 758 | * of an item to a different position in the database (e.g. through tree 759 | * balancing operations, shifts as a result of adds or deletes, etc.). It is 760 | * intended to allow address/position-dependent data items to be stored in 761 | * a database in an environment opened with the #MDB_FIXEDMAP option. 762 | * Currently the relocation feature is unimplemented and setting 763 | * this function has no effect. 764 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 765 | * @param[in] dbi A database handle returned by #mdb_open() 766 | * @param[in] rel A #MDB_rel_func function 767 | * @return A non-zero error value on failure and 0 on success. Some possible 768 | * errors are: 769 | *
    770 | *
  • EINVAL - an invalid parameter was specified. 771 | *
772 | */ 773 | int mdb_set_relfunc(MDB_txn *txn, MDB_dbi dbi, MDB_rel_func *rel); 774 | 775 | /** @brief Set a context pointer for a #MDB_FIXEDMAP database's relocation function. 776 | * 777 | * See #mdb_set_relfunc and #MDB_rel_func for more details. 778 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 779 | * @param[in] dbi A database handle returned by #mdb_open() 780 | * @param[in] ctx An arbitrary pointer for whatever the application needs. 781 | * It will be passed to the callback function set by #mdb_set_relfunc 782 | * as its \b relctx parameter whenever the callback is invoked. 783 | * @return A non-zero error value on failure and 0 on success. Some possible 784 | * errors are: 785 | *
    786 | *
  • EINVAL - an invalid parameter was specified. 787 | *
788 | */ 789 | int mdb_set_relctx(MDB_txn *txn, MDB_dbi dbi, void *ctx); 790 | 791 | /** @brief Get items from a database. 792 | * 793 | * This function retrieves key/data pairs from the database. The address 794 | * and length of the data associated with the specified \b key are returned 795 | * in the structure to which \b data refers. 796 | * If the database supports duplicate keys (#MDB_DUPSORT) then the 797 | * first data item for the key will be returned. Retrieval of other 798 | * items requires the use of #mdb_cursor_get(). 799 | * 800 | * @note The memory pointed to by the returned values is owned by the 801 | * database. The caller need not dispose of the memory, and may not 802 | * modify it in any way. For values returned in a read-only transaction 803 | * any modification attempts will cause a SIGSEGV. 804 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 805 | * @param[in] dbi A database handle returned by #mdb_open() 806 | * @param[in] key The key to search for in the database 807 | * @param[out] data The data corresponding to the key 808 | * @return A non-zero error value on failure and 0 on success. Some possible 809 | * errors are: 810 | *
    811 | *
  • #MDB_NOTFOUND - the key was not in the database. 812 | *
  • EINVAL - an invalid parameter was specified. 813 | *
814 | */ 815 | int mdb_get(MDB_txn *txn, MDB_dbi dbi, MDB_val *key, MDB_val *data); 816 | 817 | /** @brief Store items into a database. 818 | * 819 | * This function stores key/data pairs in the database. The default behavior 820 | * is to enter the new key/data pair, replacing any previously existing key 821 | * if duplicates are disallowed, or adding a duplicate data item if 822 | * duplicates are allowed (#MDB_DUPSORT). 823 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 824 | * @param[in] dbi A database handle returned by #mdb_open() 825 | * @param[in] key The key to store in the database 826 | * @param[in,out] data The data to store 827 | * @param[in] flags Special options for this operation. This parameter 828 | * must be set to 0 or by bitwise OR'ing together one or more of the 829 | * values described here. 830 | *
    831 | *
  • #MDB_NODUPDATA - enter the new key/data pair only if it does not 832 | * already appear in the database. This flag may only be specified 833 | * if the database was opened with #MDB_DUPSORT. The function will 834 | * return #MDB_KEYEXIST if the key/data pair already appears in the 835 | * database. 836 | *
  • #MDB_NOOVERWRITE - enter the new key/data pair only if the key 837 | * does not already appear in the database. The function will return 838 | * #MDB_KEYEXIST if the key already appears in the database, even if 839 | * the database supports duplicates (#MDB_DUPSORT). The \b data 840 | * parameter will be set to point to the existing item. 841 | *
  • #MDB_RESERVE - reserve space for data of the given size, but 842 | * don't copy the given data. Instead, return a pointer to the 843 | * reserved space, which the caller can fill in later. This saves 844 | * an extra memcpy if the data is being generated later. 845 | *
  • #MDB_APPEND - append the given key/data pair to the end of the 846 | * database. No key comparisons are performed. This option allows 847 | * fast bulk loading when keys are already known to be in the 848 | * correct order. Loading unsorted keys with this flag will cause 849 | * data corruption. 850 | *
  • #MDB_APPENDDUP - as above, but for sorted dup data. 851 | *
852 | * @return A non-zero error value on failure and 0 on success. Some possible 853 | * errors are: 854 | *
    855 | *
  • EACCES - an attempt was made to write in a read-only transaction. 856 | *
  • EINVAL - an invalid parameter was specified. 857 | *
  • ENOMEM - the database is full, see #mdb_env_set_mapsize(). 858 | *
859 | */ 860 | int mdb_put(MDB_txn *txn, MDB_dbi dbi, MDB_val *key, MDB_val *data, 861 | unsigned int flags); 862 | 863 | /** @brief Delete items from a database. 864 | * 865 | * This function removes key/data pairs from the database. 866 | * If the database does not support sorted duplicate data items 867 | * (#MDB_DUPSORT) the data parameter is ignored. 868 | * If the database supports sorted duplicates and the data parameter 869 | * is NULL, all of the duplicate data items for the key will be 870 | * deleted. Otherwise, if the data parameter is non-NULL 871 | * only the matching data item will be deleted. 872 | * This function will return #MDB_NOTFOUND if the specified key/data 873 | * pair is not in the database. 874 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 875 | * @param[in] dbi A database handle returned by #mdb_open() 876 | * @param[in] key The key to delete from the database 877 | * @param[in] data The data to delete 878 | * @return A non-zero error value on failure and 0 on success. Some possible 879 | * errors are: 880 | *
    881 | *
  • EACCES - an attempt was made to write in a read-only transaction. 882 | *
  • EINVAL - an invalid parameter was specified. 883 | *
884 | */ 885 | int mdb_del(MDB_txn *txn, MDB_dbi dbi, MDB_val *key, MDB_val *data); 886 | 887 | /** @brief Create a cursor handle. 888 | * 889 | * Cursors are associated with a specific transaction and database and 890 | * may not span threads. 891 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 892 | * @param[in] dbi A database handle returned by #mdb_open() 893 | * @param[out] cursor Address where the new #MDB_cursor handle will be stored 894 | * @return A non-zero error value on failure and 0 on success. Some possible 895 | * errors are: 896 | *
    897 | *
  • EINVAL - an invalid parameter was specified. 898 | *
899 | */ 900 | int mdb_cursor_open(MDB_txn *txn, MDB_dbi dbi, MDB_cursor **cursor); 901 | 902 | /** @brief Close a cursor handle. 903 | * 904 | * The cursor handle will be freed and must not be used again after this call. 905 | * @param[in] cursor A cursor handle returned by #mdb_cursor_open() 906 | */ 907 | void mdb_cursor_close(MDB_cursor *cursor); 908 | 909 | /** @brief Renew a cursor handle. 910 | * 911 | * Cursors are associated with a specific transaction and database and 912 | * may not span threads. Cursors that are only used in read-only 913 | * transactions may be re-used, to avoid unnecessary malloc/free overhead. 914 | * The cursor may be associated with a new read-only transaction, and 915 | * referencing the same database handle as it was created with. 916 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 917 | * @param[in] cursor A cursor handle returned by #mdb_cursor_open() 918 | * @return A non-zero error value on failure and 0 on success. Some possible 919 | * errors are: 920 | *
    921 | *
  • EINVAL - an invalid parameter was specified. 922 | *
923 | */ 924 | int mdb_cursor_renew(MDB_txn *txn, MDB_cursor *cursor); 925 | 926 | /** @brief Return the cursor's transaction handle. 927 | * 928 | * @param[in] cursor A cursor handle returned by #mdb_cursor_open() 929 | */ 930 | MDB_txn *mdb_cursor_txn(MDB_cursor *cursor); 931 | 932 | /** @brief Return the cursor's database handle. 933 | * 934 | * @param[in] cursor A cursor handle returned by #mdb_cursor_open() 935 | */ 936 | MDB_dbi mdb_cursor_dbi(MDB_cursor *cursor); 937 | 938 | /** @brief Retrieve by cursor. 939 | * 940 | * This function retrieves key/data pairs from the database. The address and length 941 | * of the key are returned in the object to which \b key refers (except for the 942 | * case of the #MDB_SET option, in which the \b key object is unchanged), and 943 | * the address and length of the data are returned in the object to which \b data 944 | * refers. 945 | * @param[in] cursor A cursor handle returned by #mdb_cursor_open() 946 | * @param[in,out] key The key for a retrieved item 947 | * @param[in,out] data The data of a retrieved item 948 | * @param[in] op A cursor operation #MDB_cursor_op 949 | * @return A non-zero error value on failure and 0 on success. Some possible 950 | * errors are: 951 | *
    952 | *
  • #MDB_NOTFOUND - no matching key found. 953 | *
  • EINVAL - an invalid parameter was specified. 954 | *
955 | */ 956 | int mdb_cursor_get(MDB_cursor *cursor, MDB_val *key, MDB_val *data, 957 | MDB_cursor_op op); 958 | 959 | /** @brief Store by cursor. 960 | * 961 | * This function stores key/data pairs into the database. 962 | * If the function fails for any reason, the state of the cursor will be 963 | * unchanged. If the function succeeds and an item is inserted into the 964 | * database, the cursor is always positioned to refer to the newly inserted item. 965 | * @param[in] cursor A cursor handle returned by #mdb_cursor_open() 966 | * @param[in] key The key operated on. 967 | * @param[in] data The data operated on. 968 | * @param[in] flags Options for this operation. This parameter 969 | * must be set to 0 or one of the values described here. 970 | *
    971 | *
  • #MDB_CURRENT - overwrite the data of the key/data pair to which 972 | * the cursor refers with the specified data item. The \b key 973 | * parameter is ignored. 974 | *
  • #MDB_NODUPDATA - enter the new key/data pair only if it does not 975 | * already appear in the database. This flag may only be specified 976 | * if the database was opened with #MDB_DUPSORT. The function will 977 | * return #MDB_KEYEXIST if the key/data pair already appears in the 978 | * database. 979 | *
  • #MDB_NOOVERWRITE - enter the new key/data pair only if the key 980 | * does not already appear in the database. The function will return 981 | * #MDB_KEYEXIST if the key already appears in the database, even if 982 | * the database supports duplicates (#MDB_DUPSORT). 983 | *
  • #MDB_RESERVE - reserve space for data of the given size, but 984 | * don't copy the given data. Instead, return a pointer to the 985 | * reserved space, which the caller can fill in later. This saves 986 | * an extra memcpy if the data is being generated later. 987 | *
  • #MDB_APPEND - append the given key/data pair to the end of the 988 | * database. No key comparisons are performed. This option allows 989 | * fast bulk loading when keys are already known to be in the 990 | * correct order. Loading unsorted keys with this flag will cause 991 | * data corruption. 992 | *
  • #MDB_APPENDDUP - as above, but for sorted dup data. 993 | *
994 | * @return A non-zero error value on failure and 0 on success. Some possible 995 | * errors are: 996 | *
    997 | *
  • EACCES - an attempt was made to modify a read-only database. 998 | *
  • EINVAL - an invalid parameter was specified. 999 | *
1000 | */ 1001 | int mdb_cursor_put(MDB_cursor *cursor, MDB_val *key, MDB_val *data, 1002 | unsigned int flags); 1003 | 1004 | /** @brief Delete current key/data pair 1005 | * 1006 | * This function deletes the key/data pair to which the cursor refers. 1007 | * @param[in] cursor A cursor handle returned by #mdb_cursor_open() 1008 | * @param[in] flags Options for this operation. This parameter 1009 | * must be set to 0 or one of the values described here. 1010 | *
    1011 | *
  • #MDB_NODUPDATA - delete all of the data items for the current key. 1012 | * This flag may only be specified if the database was opened with #MDB_DUPSORT. 1013 | *
1014 | * @return A non-zero error value on failure and 0 on success. Some possible 1015 | * errors are: 1016 | *
    1017 | *
  • EACCES - an attempt was made to modify a read-only database. 1018 | *
  • EINVAL - an invalid parameter was specified. 1019 | *
1020 | */ 1021 | int mdb_cursor_del(MDB_cursor *cursor, unsigned int flags); 1022 | 1023 | /** @brief Return count of duplicates for current key. 1024 | * 1025 | * This call is only valid on databases that support sorted duplicate 1026 | * data items #MDB_DUPSORT. 1027 | * @param[in] cursor A cursor handle returned by #mdb_cursor_open() 1028 | * @param[out] countp Address where the count will be stored 1029 | * @return A non-zero error value on failure and 0 on success. Some possible 1030 | * errors are: 1031 | *
    1032 | *
  • EINVAL - cursor is not initialized, or an invalid parameter was specified. 1033 | *
1034 | */ 1035 | int mdb_cursor_count(MDB_cursor *cursor, size_t *countp); 1036 | 1037 | /** @brief Compare two data items according to a particular database. 1038 | * 1039 | * This returns a comparison as if the two data items were keys in the 1040 | * specified database. 1041 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 1042 | * @param[in] dbi A database handle returned by #mdb_open() 1043 | * @param[in] a The first item to compare 1044 | * @param[in] b The second item to compare 1045 | * @return < 0 if a < b, 0 if a == b, > 0 if a > b 1046 | */ 1047 | int mdb_cmp(MDB_txn *txn, MDB_dbi dbi, const MDB_val *a, const MDB_val *b); 1048 | 1049 | /** @brief Compare two data items according to a particular database. 1050 | * 1051 | * This returns a comparison as if the two items were data items of 1052 | * a sorted duplicates #MDB_DUPSORT database. 1053 | * @param[in] txn A transaction handle returned by #mdb_txn_begin() 1054 | * @param[in] dbi A database handle returned by #mdb_open() 1055 | * @param[in] a The first item to compare 1056 | * @param[in] b The second item to compare 1057 | * @return < 0 if a < b, 0 if a == b, > 0 if a > b 1058 | */ 1059 | int mdb_dcmp(MDB_txn *txn, MDB_dbi dbi, const MDB_val *a, const MDB_val *b); 1060 | /** @} */ 1061 | 1062 | #ifdef __cplusplus 1063 | } 1064 | #endif 1065 | #endif /* _MDB_H_ */ 1066 | -------------------------------------------------------------------------------- /c_src/midl.c: -------------------------------------------------------------------------------- 1 | /** @file midl.c 2 | * @brief ldap bdb back-end ID List functions */ 3 | /* $OpenLDAP$ */ 4 | /* This work is part of OpenLDAP Software . 5 | * 6 | * Copyright 2000-2012 The OpenLDAP Foundation. 7 | * All rights reserved. 8 | * 9 | * Redistribution and use in source and binary forms, with or without 10 | * modification, are permitted only as authorized by the OpenLDAP 11 | * Public License. 12 | * 13 | * A copy of this license is available in the file LICENSE in the 14 | * top-level directory of the distribution or, alternatively, at 15 | * . 16 | */ 17 | 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include "midl.h" 24 | 25 | /** @defgroup internal MDB Internals 26 | * @{ 27 | */ 28 | /** @defgroup idls ID List Management 29 | * @{ 30 | */ 31 | #define CMP(x,y) ( (x) < (y) ? -1 : (x) > (y) ) 32 | 33 | #if 0 /* superseded by append/sort */ 34 | static unsigned mdb_midl_search( MDB_IDL ids, MDB_ID id ) 35 | { 36 | /* 37 | * binary search of id in ids 38 | * if found, returns position of id 39 | * if not found, returns first position greater than id 40 | */ 41 | unsigned base = 0; 42 | unsigned cursor = 1; 43 | int val = 0; 44 | unsigned n = ids[0]; 45 | 46 | while( 0 < n ) { 47 | unsigned pivot = n >> 1; 48 | cursor = base + pivot + 1; 49 | val = CMP( ids[cursor], id ); 50 | 51 | if( val < 0 ) { 52 | n = pivot; 53 | 54 | } else if ( val > 0 ) { 55 | base = cursor; 56 | n -= pivot + 1; 57 | 58 | } else { 59 | return cursor; 60 | } 61 | } 62 | 63 | if( val > 0 ) { 64 | ++cursor; 65 | } 66 | return cursor; 67 | } 68 | 69 | int mdb_midl_insert( MDB_IDL ids, MDB_ID id ) 70 | { 71 | unsigned x, i; 72 | 73 | if (MDB_IDL_IS_RANGE( ids )) { 74 | /* if already in range, treat as a dup */ 75 | if (id >= MDB_IDL_RANGE_FIRST(ids) && id <= MDB_IDL_RANGE_LAST(ids)) 76 | return -1; 77 | if (id < MDB_IDL_RANGE_FIRST(ids)) 78 | ids[1] = id; 79 | else if (id > MDB_IDL_RANGE_LAST(ids)) 80 | ids[2] = id; 81 | return 0; 82 | } 83 | 84 | x = mdb_midl_search( ids, id ); 85 | assert( x > 0 ); 86 | 87 | if( x < 1 ) { 88 | /* internal error */ 89 | return -2; 90 | } 91 | 92 | if ( x <= ids[0] && ids[x] == id ) { 93 | /* duplicate */ 94 | assert(0); 95 | return -1; 96 | } 97 | 98 | if ( ++ids[0] >= MDB_IDL_DB_MAX ) { 99 | if( id < ids[1] ) { 100 | ids[1] = id; 101 | ids[2] = ids[ids[0]-1]; 102 | } else if ( ids[ids[0]-1] < id ) { 103 | ids[2] = id; 104 | } else { 105 | ids[2] = ids[ids[0]-1]; 106 | } 107 | ids[0] = MDB_NOID; 108 | 109 | } else { 110 | /* insert id */ 111 | for (i=ids[0]; i>x; i--) 112 | ids[i] = ids[i-1]; 113 | ids[x] = id; 114 | } 115 | 116 | return 0; 117 | } 118 | #endif 119 | 120 | MDB_IDL mdb_midl_alloc() 121 | { 122 | MDB_IDL ids = malloc((MDB_IDL_UM_MAX+1) * sizeof(MDB_ID)); 123 | *ids++ = MDB_IDL_UM_MAX; 124 | return ids; 125 | } 126 | 127 | void mdb_midl_free(MDB_IDL ids) 128 | { 129 | free(ids-1); 130 | } 131 | 132 | int mdb_midl_shrink( MDB_IDL *idp ) 133 | { 134 | MDB_IDL ids = *idp; 135 | if (ids[-1] > MDB_IDL_UM_MAX) { 136 | ids = realloc(ids, (MDB_IDL_UM_MAX+1) * sizeof(MDB_ID)); 137 | *ids++ = MDB_IDL_UM_MAX; 138 | *idp = ids; 139 | return 1; 140 | } 141 | return 0; 142 | } 143 | 144 | int mdb_midl_append( MDB_IDL *idp, MDB_ID id ) 145 | { 146 | MDB_IDL ids = *idp; 147 | /* Too big? */ 148 | if (ids[0] >= ids[-1]) { 149 | MDB_IDL idn = ids-1; 150 | /* grow it */ 151 | idn = realloc(idn, (*idn + MDB_IDL_UM_MAX + 1) * sizeof(MDB_ID)); 152 | if (!idn) 153 | return -1; 154 | *idn++ += MDB_IDL_UM_MAX; 155 | ids = idn; 156 | *idp = ids; 157 | } 158 | ids[0]++; 159 | ids[ids[0]] = id; 160 | return 0; 161 | } 162 | 163 | int mdb_midl_append_list( MDB_IDL *idp, MDB_IDL app ) 164 | { 165 | MDB_IDL ids = *idp; 166 | /* Too big? */ 167 | if (ids[0] + app[0] >= ids[-1]) { 168 | MDB_IDL idn = ids-1; 169 | /* grow it */ 170 | idn = realloc(idn, (*idn + app[-1]) * sizeof(MDB_ID)); 171 | if (!idn) 172 | return -1; 173 | *idn++ += app[-1]; 174 | ids = idn; 175 | *idp = ids; 176 | } 177 | memcpy(&ids[ids[0]+1], &app[1], app[0] * sizeof(MDB_ID)); 178 | ids[0] += app[0]; 179 | return 0; 180 | } 181 | 182 | /* Quicksort + Insertion sort for small arrays */ 183 | 184 | #define SMALL 8 185 | #define SWAP(a,b) { itmp=(a); (a)=(b); (b)=itmp; } 186 | 187 | void 188 | mdb_midl_sort( MDB_IDL ids ) 189 | { 190 | /* Max possible depth of int-indexed tree * 2 items/level */ 191 | int istack[sizeof(int)*CHAR_BIT * 2]; 192 | int i,j,k,l,ir,jstack; 193 | MDB_ID a, itmp; 194 | 195 | ir = ids[0]; 196 | l = 1; 197 | jstack = 0; 198 | for(;;) { 199 | if (ir - l < SMALL) { /* Insertion sort */ 200 | for (j=l+1;j<=ir;j++) { 201 | a = ids[j]; 202 | for (i=j-1;i>=1;i--) { 203 | if (ids[i] >= a) break; 204 | ids[i+1] = ids[i]; 205 | } 206 | ids[i+1] = a; 207 | } 208 | if (jstack == 0) break; 209 | ir = istack[jstack--]; 210 | l = istack[jstack--]; 211 | } else { 212 | k = (l + ir) >> 1; /* Choose median of left, center, right */ 213 | SWAP(ids[k], ids[l+1]); 214 | if (ids[l] < ids[ir]) { 215 | SWAP(ids[l], ids[ir]); 216 | } 217 | if (ids[l+1] < ids[ir]) { 218 | SWAP(ids[l+1], ids[ir]); 219 | } 220 | if (ids[l] < ids[l+1]) { 221 | SWAP(ids[l], ids[l+1]); 222 | } 223 | i = l+1; 224 | j = ir; 225 | a = ids[l+1]; 226 | for(;;) { 227 | do i++; while(ids[i] > a); 228 | do j--; while(ids[j] < a); 229 | if (j < i) break; 230 | SWAP(ids[i],ids[j]); 231 | } 232 | ids[l+1] = ids[j]; 233 | ids[j] = a; 234 | jstack += 2; 235 | if (ir-i+1 >= j-1) { 236 | istack[jstack] = ir; 237 | istack[jstack-1] = i; 238 | ir = j-1; 239 | } else { 240 | istack[jstack] = j-1; 241 | istack[jstack-1] = l; 242 | l = i; 243 | } 244 | } 245 | } 246 | } 247 | 248 | unsigned mdb_mid2l_search( MDB_ID2L ids, MDB_ID id ) 249 | { 250 | /* 251 | * binary search of id in ids 252 | * if found, returns position of id 253 | * if not found, returns first position greater than id 254 | */ 255 | unsigned base = 0; 256 | unsigned cursor = 1; 257 | int val = 0; 258 | unsigned n = ids[0].mid; 259 | 260 | while( 0 < n ) { 261 | unsigned pivot = n >> 1; 262 | cursor = base + pivot + 1; 263 | val = CMP( id, ids[cursor].mid ); 264 | 265 | if( val < 0 ) { 266 | n = pivot; 267 | 268 | } else if ( val > 0 ) { 269 | base = cursor; 270 | n -= pivot + 1; 271 | 272 | } else { 273 | return cursor; 274 | } 275 | } 276 | 277 | if( val > 0 ) { 278 | ++cursor; 279 | } 280 | return cursor; 281 | } 282 | 283 | int mdb_mid2l_insert( MDB_ID2L ids, MDB_ID2 *id ) 284 | { 285 | unsigned x, i; 286 | 287 | x = mdb_mid2l_search( ids, id->mid ); 288 | assert( x > 0 ); 289 | 290 | if( x < 1 ) { 291 | /* internal error */ 292 | return -2; 293 | } 294 | 295 | if ( x <= ids[0].mid && ids[x].mid == id->mid ) { 296 | /* duplicate */ 297 | return -1; 298 | } 299 | 300 | if ( ids[0].mid >= MDB_IDL_UM_MAX ) { 301 | /* too big */ 302 | return -2; 303 | 304 | } else { 305 | /* insert id */ 306 | ids[0].mid++; 307 | for (i=ids[0].mid; i>x; i--) 308 | ids[i] = ids[i-1]; 309 | ids[x] = *id; 310 | } 311 | 312 | return 0; 313 | } 314 | 315 | int mdb_mid2l_append( MDB_ID2L ids, MDB_ID2 *id ) 316 | { 317 | /* Too big? */ 318 | if (ids[0].mid >= MDB_IDL_UM_MAX) { 319 | return -2; 320 | } 321 | ids[0].mid++; 322 | ids[ids[0].mid] = *id; 323 | return 0; 324 | } 325 | 326 | /** @} */ 327 | /** @} */ 328 | -------------------------------------------------------------------------------- /c_src/midl.h: -------------------------------------------------------------------------------- 1 | /** @file midl.h 2 | * @brief mdb ID List header file. 3 | * 4 | * This file was originally part of back-bdb but has been 5 | * modified for use in libmdb. Most of the macros defined 6 | * in this file are unused, just left over from the original. 7 | * 8 | * This file is only used internally in libmdb and its definitions 9 | * are not exposed publicly. 10 | */ 11 | /* $OpenLDAP$ */ 12 | /* This work is part of OpenLDAP Software . 13 | * 14 | * Copyright 2000-2012 The OpenLDAP Foundation. 15 | * All rights reserved. 16 | * 17 | * Redistribution and use in source and binary forms, with or without 18 | * modification, are permitted only as authorized by the OpenLDAP 19 | * Public License. 20 | * 21 | * A copy of this license is available in the file LICENSE in the 22 | * top-level directory of the distribution or, alternatively, at 23 | * . 24 | */ 25 | 26 | #ifndef _MDB_MIDL_H_ 27 | #define _MDB_MIDL_H_ 28 | 29 | #include 30 | 31 | #ifdef __cplusplus 32 | extern "C" { 33 | #endif 34 | 35 | /** @defgroup internal MDB Internals 36 | * @{ 37 | */ 38 | 39 | /** @defgroup idls ID List Management 40 | * @{ 41 | */ 42 | /** A generic ID number. These were entryIDs in back-bdb. 43 | * Preferably it should have the same size as a pointer. 44 | */ 45 | typedef size_t MDB_ID; 46 | 47 | /** An IDL is an ID List, a sorted array of IDs. The first 48 | * element of the array is a counter for how many actual 49 | * IDs are in the list. In the original back-bdb code, IDLs are 50 | * sorted in ascending order. For libmdb IDLs are sorted in 51 | * descending order. 52 | */ 53 | typedef MDB_ID *MDB_IDL; 54 | 55 | #define MDB_NOID (~(MDB_ID)0) 56 | 57 | /* IDL sizes - likely should be even bigger 58 | * limiting factors: sizeof(ID), thread stack size 59 | */ 60 | #define MDB_IDL_LOGN 16 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ 61 | #define MDB_IDL_DB_SIZE (1<bi_lastid) ) 99 | #define MDB_IDL_ALL( bdb, ids ) MDB_IDL_RANGE( ids, 1, ((bdb)->bi_lastid) ) 100 | 101 | #define MDB_IDL_FIRST( ids ) ( (ids)[1] ) 102 | #define MDB_IDL_LAST( ids ) ( MDB_IDL_IS_RANGE(ids) \ 103 | ? (ids)[2] : (ids)[(ids)[0]] ) 104 | 105 | #define MDB_IDL_N( ids ) ( MDB_IDL_IS_RANGE(ids) \ 106 | ? ((ids)[2]-(ids)[1])+1 : (ids)[0] ) 107 | 108 | #if 0 /* superseded by append/sort */ 109 | /** Insert an ID into an IDL. 110 | * @param[in,out] ids The IDL to insert into. 111 | * @param[in] id The ID to insert. 112 | * @return 0 on success, -1 if the ID was already present in the IDL. 113 | */ 114 | int mdb_midl_insert( MDB_IDL ids, MDB_ID id ); 115 | #endif 116 | 117 | /** Allocate an IDL. 118 | * Allocates memory for an IDL of a default size. 119 | * @return IDL on success, NULL on failure. 120 | */ 121 | MDB_IDL mdb_midl_alloc(); 122 | 123 | /** Free an IDL. 124 | * @param[in] ids The IDL to free. 125 | */ 126 | void mdb_midl_free(MDB_IDL ids); 127 | 128 | /** Shrink an IDL. 129 | * Return the IDL to the default size if it has grown larger. 130 | * @param[in,out] idp Address of the IDL to shrink. 131 | * @return 0 on no change, non-zero if shrunk. 132 | */ 133 | int mdb_midl_shrink(MDB_IDL *idp); 134 | 135 | /** Append an ID onto an IDL. 136 | * @param[in,out] idp Address of the IDL to append to. 137 | * @param[in] id The ID to append. 138 | * @return 0 on success, -1 if the IDL is too large. 139 | */ 140 | int mdb_midl_append( MDB_IDL *idp, MDB_ID id ); 141 | 142 | /** Append an IDL onto an IDL. 143 | * @param[in,out] idp Address of the IDL to append to. 144 | * @param[in] app The IDL to append. 145 | * @return 0 on success, -1 if the IDL is too large. 146 | */ 147 | int mdb_midl_append_list( MDB_IDL *idp, MDB_IDL app ); 148 | 149 | /** Sort an IDL. 150 | * @param[in,out] ids The IDL to sort. 151 | */ 152 | void mdb_midl_sort( MDB_IDL ids ); 153 | 154 | /** An ID2 is an ID/pointer pair. 155 | */ 156 | typedef struct MDB_ID2 { 157 | MDB_ID mid; /**< The ID */ 158 | void *mptr; /**< The pointer */ 159 | } MDB_ID2; 160 | 161 | /** An ID2L is an ID2 List, a sorted array of ID2s. 162 | * The first element's \b mid member is a count of how many actual 163 | * elements are in the array. The \b mptr member of the first element is unused. 164 | * The array is sorted in ascending order by \b mid. 165 | */ 166 | typedef MDB_ID2 *MDB_ID2L; 167 | 168 | /** Search for an ID in an ID2L. 169 | * @param[in] ids The ID2L to search. 170 | * @param[in] id The ID to search for. 171 | * @return The index of the first ID2 whose \b mid member is greater than or equal to \b id. 172 | */ 173 | unsigned mdb_mid2l_search( MDB_ID2L ids, MDB_ID id ); 174 | 175 | 176 | /** Insert an ID2 into a ID2L. 177 | * @param[in,out] ids The ID2L to insert into. 178 | * @param[in] id The ID2 to insert. 179 | * @return 0 on success, -1 if the ID was already present in the ID2L. 180 | */ 181 | int mdb_mid2l_insert( MDB_ID2L ids, MDB_ID2 *id ); 182 | 183 | /** Append an ID2 into a ID2L. 184 | * @param[in,out] ids The ID2L to append into. 185 | * @param[in] id The ID2 to append. 186 | * @return 0 on success, -2 if the ID2L is too big. 187 | */ 188 | int mdb_mid2l_append( MDB_ID2L ids, MDB_ID2 *id ); 189 | 190 | /** @} */ 191 | /** @} */ 192 | #ifdef __cplusplus 193 | } 194 | #endif 195 | #endif /* _MDB_MIDL_H_ */ 196 | -------------------------------------------------------------------------------- /c_src/uthash.h: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright (c) 2003-2012, Troy D. Hanson http://uthash.sourceforge.net 3 | All rights reserved. 4 | 5 | Redistribution and use in source and binary forms, with or without 6 | modification, are permitted provided that the following conditions are met: 7 | 8 | * Redistributions of source code must retain the above copyright 9 | notice, this list of conditions and the following disclaimer. 10 | 11 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS 12 | IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED 13 | TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A 14 | PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER 15 | OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 16 | EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, 17 | PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR 18 | PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 19 | LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 20 | NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 21 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 22 | */ 23 | 24 | #ifndef UTHASH_H 25 | #define UTHASH_H 26 | 27 | #include /* memcmp,strlen */ 28 | #include /* ptrdiff_t */ 29 | #include /* exit() */ 30 | 31 | /* These macros use decltype or the earlier __typeof GNU extension. 32 | As decltype is only available in newer compilers (VS2010 or gcc 4.3+ 33 | when compiling c++ source) this code uses whatever method is needed 34 | or, for VS2008 where neither is available, uses casting workarounds. */ 35 | #ifdef _MSC_VER /* MS compiler */ 36 | #if _MSC_VER >= 1600 && defined(__cplusplus) /* VS2010 or newer in C++ mode */ 37 | #define DECLTYPE(x) (decltype(x)) 38 | #else /* VS2008 or older (or VS2010 in C mode) */ 39 | #define NO_DECLTYPE 40 | #define DECLTYPE(x) 41 | #endif 42 | #else /* GNU, Sun and other compilers */ 43 | #define DECLTYPE(x) (__typeof(x)) 44 | #endif 45 | 46 | #ifdef NO_DECLTYPE 47 | #define DECLTYPE_ASSIGN(dst,src) \ 48 | do { \ 49 | char **_da_dst = (char**)(&(dst)); \ 50 | *_da_dst = (char*)(src); \ 51 | } while(0) 52 | #else 53 | #define DECLTYPE_ASSIGN(dst,src) \ 54 | do { \ 55 | (dst) = DECLTYPE(dst)(src); \ 56 | } while(0) 57 | #endif 58 | 59 | /* a number of the hash function use uint32_t which isn't defined on win32 */ 60 | #ifdef _MSC_VER 61 | typedef unsigned int uint32_t; 62 | typedef unsigned char uint8_t; 63 | #else 64 | #include /* uint32_t */ 65 | #endif 66 | 67 | #define UTHASH_VERSION 1.9.6 68 | 69 | #ifndef uthash_fatal 70 | #define uthash_fatal(msg) exit(-1) /* fatal error (out of memory,etc) */ 71 | #endif 72 | #ifndef uthash_malloc 73 | #define uthash_malloc(sz) malloc(sz) /* malloc fcn */ 74 | #endif 75 | #ifndef uthash_free 76 | #define uthash_free(ptr,sz) free(ptr) /* free fcn */ 77 | #endif 78 | 79 | #ifndef uthash_noexpand_fyi 80 | #define uthash_noexpand_fyi(tbl) /* can be defined to log noexpand */ 81 | #endif 82 | #ifndef uthash_expand_fyi 83 | #define uthash_expand_fyi(tbl) /* can be defined to log expands */ 84 | #endif 85 | 86 | /* initial number of buckets */ 87 | #define HASH_INITIAL_NUM_BUCKETS 32 /* initial number of buckets */ 88 | #define HASH_INITIAL_NUM_BUCKETS_LOG2 5 /* lg2 of initial number of buckets */ 89 | #define HASH_BKT_CAPACITY_THRESH 10 /* expand when bucket count reaches */ 90 | 91 | /* calculate the element whose hash handle address is hhe */ 92 | #define ELMT_FROM_HH(tbl,hhp) ((void*)(((char*)(hhp)) - ((tbl)->hho))) 93 | 94 | #define HASH_FIND(hh,head,keyptr,keylen,out) \ 95 | do { \ 96 | unsigned _hf_bkt,_hf_hashv; \ 97 | out=NULL; \ 98 | if (head) { \ 99 | HASH_FCN(keyptr,keylen, (head)->hh.tbl->num_buckets, _hf_hashv, _hf_bkt); \ 100 | if (HASH_BLOOM_TEST((head)->hh.tbl, _hf_hashv)) { \ 101 | HASH_FIND_IN_BKT((head)->hh.tbl, hh, (head)->hh.tbl->buckets[ _hf_bkt ], \ 102 | keyptr,keylen,out); \ 103 | } \ 104 | } \ 105 | } while (0) 106 | 107 | #ifdef HASH_BLOOM 108 | #define HASH_BLOOM_BITLEN (1ULL << HASH_BLOOM) 109 | #define HASH_BLOOM_BYTELEN (HASH_BLOOM_BITLEN/8) + ((HASH_BLOOM_BITLEN%8) ? 1:0) 110 | #define HASH_BLOOM_MAKE(tbl) \ 111 | do { \ 112 | (tbl)->bloom_nbits = HASH_BLOOM; \ 113 | (tbl)->bloom_bv = (uint8_t*)uthash_malloc(HASH_BLOOM_BYTELEN); \ 114 | if (!((tbl)->bloom_bv)) { uthash_fatal( "out of memory"); } \ 115 | memset((tbl)->bloom_bv, 0, HASH_BLOOM_BYTELEN); \ 116 | (tbl)->bloom_sig = HASH_BLOOM_SIGNATURE; \ 117 | } while (0) 118 | 119 | #define HASH_BLOOM_FREE(tbl) \ 120 | do { \ 121 | uthash_free((tbl)->bloom_bv, HASH_BLOOM_BYTELEN); \ 122 | } while (0) 123 | 124 | #define HASH_BLOOM_BITSET(bv,idx) (bv[(idx)/8] |= (1U << ((idx)%8))) 125 | #define HASH_BLOOM_BITTEST(bv,idx) (bv[(idx)/8] & (1U << ((idx)%8))) 126 | 127 | #define HASH_BLOOM_ADD(tbl,hashv) \ 128 | HASH_BLOOM_BITSET((tbl)->bloom_bv, (hashv & (uint32_t)((1ULL << (tbl)->bloom_nbits) - 1))) 129 | 130 | #define HASH_BLOOM_TEST(tbl,hashv) \ 131 | HASH_BLOOM_BITTEST((tbl)->bloom_bv, (hashv & (uint32_t)((1ULL << (tbl)->bloom_nbits) - 1))) 132 | 133 | #else 134 | #define HASH_BLOOM_MAKE(tbl) 135 | #define HASH_BLOOM_FREE(tbl) 136 | #define HASH_BLOOM_ADD(tbl,hashv) 137 | #define HASH_BLOOM_TEST(tbl,hashv) (1) 138 | #endif 139 | 140 | #define HASH_MAKE_TABLE(hh,head) \ 141 | do { \ 142 | (head)->hh.tbl = (UT_hash_table*)uthash_malloc( \ 143 | sizeof(UT_hash_table)); \ 144 | if (!((head)->hh.tbl)) { uthash_fatal( "out of memory"); } \ 145 | memset((head)->hh.tbl, 0, sizeof(UT_hash_table)); \ 146 | (head)->hh.tbl->tail = &((head)->hh); \ 147 | (head)->hh.tbl->num_buckets = HASH_INITIAL_NUM_BUCKETS; \ 148 | (head)->hh.tbl->log2_num_buckets = HASH_INITIAL_NUM_BUCKETS_LOG2; \ 149 | (head)->hh.tbl->hho = (char*)(&(head)->hh) - (char*)(head); \ 150 | (head)->hh.tbl->buckets = (UT_hash_bucket*)uthash_malloc( \ 151 | HASH_INITIAL_NUM_BUCKETS*sizeof(struct UT_hash_bucket)); \ 152 | if (! (head)->hh.tbl->buckets) { uthash_fatal( "out of memory"); } \ 153 | memset((head)->hh.tbl->buckets, 0, \ 154 | HASH_INITIAL_NUM_BUCKETS*sizeof(struct UT_hash_bucket)); \ 155 | HASH_BLOOM_MAKE((head)->hh.tbl); \ 156 | (head)->hh.tbl->signature = HASH_SIGNATURE; \ 157 | } while(0) 158 | 159 | #define HASH_ADD(hh,head,fieldname,keylen_in,add) \ 160 | HASH_ADD_KEYPTR(hh,head,&((add)->fieldname),keylen_in,add) 161 | 162 | #define HASH_ADD_KEYPTR(hh,head,keyptr,keylen_in,add) \ 163 | do { \ 164 | unsigned _ha_bkt; \ 165 | (add)->hh.next = NULL; \ 166 | (add)->hh.key = (char*)keyptr; \ 167 | (add)->hh.keylen = (unsigned)keylen_in; \ 168 | if (!(head)) { \ 169 | head = (add); \ 170 | (head)->hh.prev = NULL; \ 171 | HASH_MAKE_TABLE(hh,head); \ 172 | } else { \ 173 | (head)->hh.tbl->tail->next = (add); \ 174 | (add)->hh.prev = ELMT_FROM_HH((head)->hh.tbl, (head)->hh.tbl->tail); \ 175 | (head)->hh.tbl->tail = &((add)->hh); \ 176 | } \ 177 | (head)->hh.tbl->num_items++; \ 178 | (add)->hh.tbl = (head)->hh.tbl; \ 179 | HASH_FCN(keyptr,keylen_in, (head)->hh.tbl->num_buckets, \ 180 | (add)->hh.hashv, _ha_bkt); \ 181 | HASH_ADD_TO_BKT((head)->hh.tbl->buckets[_ha_bkt],&(add)->hh); \ 182 | HASH_BLOOM_ADD((head)->hh.tbl,(add)->hh.hashv); \ 183 | HASH_EMIT_KEY(hh,head,keyptr,keylen_in); \ 184 | HASH_FSCK(hh,head); \ 185 | } while(0) 186 | 187 | #define HASH_TO_BKT( hashv, num_bkts, bkt ) \ 188 | do { \ 189 | bkt = ((hashv) & ((num_bkts) - 1)); \ 190 | } while(0) 191 | 192 | /* delete "delptr" from the hash table. 193 | * "the usual" patch-up process for the app-order doubly-linked-list. 194 | * The use of _hd_hh_del below deserves special explanation. 195 | * These used to be expressed using (delptr) but that led to a bug 196 | * if someone used the same symbol for the head and deletee, like 197 | * HASH_DELETE(hh,users,users); 198 | * We want that to work, but by changing the head (users) below 199 | * we were forfeiting our ability to further refer to the deletee (users) 200 | * in the patch-up process. Solution: use scratch space to 201 | * copy the deletee pointer, then the latter references are via that 202 | * scratch pointer rather than through the repointed (users) symbol. 203 | */ 204 | #define HASH_DELETE(hh,head,delptr) \ 205 | do { \ 206 | unsigned _hd_bkt; \ 207 | struct UT_hash_handle *_hd_hh_del; \ 208 | if ( ((delptr)->hh.prev == NULL) && ((delptr)->hh.next == NULL) ) { \ 209 | uthash_free((head)->hh.tbl->buckets, \ 210 | (head)->hh.tbl->num_buckets*sizeof(struct UT_hash_bucket) ); \ 211 | HASH_BLOOM_FREE((head)->hh.tbl); \ 212 | uthash_free((head)->hh.tbl, sizeof(UT_hash_table)); \ 213 | head = NULL; \ 214 | } else { \ 215 | _hd_hh_del = &((delptr)->hh); \ 216 | if ((delptr) == ELMT_FROM_HH((head)->hh.tbl,(head)->hh.tbl->tail)) { \ 217 | (head)->hh.tbl->tail = \ 218 | (UT_hash_handle*)((char*)((delptr)->hh.prev) + \ 219 | (head)->hh.tbl->hho); \ 220 | } \ 221 | if ((delptr)->hh.prev) { \ 222 | ((UT_hash_handle*)((char*)((delptr)->hh.prev) + \ 223 | (head)->hh.tbl->hho))->next = (delptr)->hh.next; \ 224 | } else { \ 225 | DECLTYPE_ASSIGN(head,(delptr)->hh.next); \ 226 | } \ 227 | if (_hd_hh_del->next) { \ 228 | ((UT_hash_handle*)((char*)_hd_hh_del->next + \ 229 | (head)->hh.tbl->hho))->prev = \ 230 | _hd_hh_del->prev; \ 231 | } \ 232 | HASH_TO_BKT( _hd_hh_del->hashv, (head)->hh.tbl->num_buckets, _hd_bkt); \ 233 | HASH_DEL_IN_BKT(hh,(head)->hh.tbl->buckets[_hd_bkt], _hd_hh_del); \ 234 | (head)->hh.tbl->num_items--; \ 235 | } \ 236 | HASH_FSCK(hh,head); \ 237 | } while (0) 238 | 239 | 240 | /* convenience forms of HASH_FIND/HASH_ADD/HASH_DEL */ 241 | #define HASH_FIND_STR(head,findstr,out) \ 242 | HASH_FIND(hh,head,findstr,strlen(findstr),out) 243 | #define HASH_ADD_STR(head,strfield,add) \ 244 | HASH_ADD(hh,head,strfield,strlen(add->strfield),add) 245 | #define HASH_FIND_INT(head,findint,out) \ 246 | HASH_FIND(hh,head,findint,sizeof(int),out) 247 | #define HASH_ADD_INT(head,intfield,add) \ 248 | HASH_ADD(hh,head,intfield,sizeof(int),add) 249 | #define HASH_FIND_PTR(head,findptr,out) \ 250 | HASH_FIND(hh,head,findptr,sizeof(void *),out) 251 | #define HASH_ADD_PTR(head,ptrfield,add) \ 252 | HASH_ADD(hh,head,ptrfield,sizeof(void *),add) 253 | #define HASH_DEL(head,delptr) \ 254 | HASH_DELETE(hh,head,delptr) 255 | 256 | /* HASH_FSCK checks hash integrity on every add/delete when HASH_DEBUG is defined. 257 | * This is for uthash developer only; it compiles away if HASH_DEBUG isn't defined. 258 | */ 259 | #ifdef HASH_DEBUG 260 | #define HASH_OOPS(...) do { fprintf(stderr,__VA_ARGS__); exit(-1); } while (0) 261 | #define HASH_FSCK(hh,head) \ 262 | do { \ 263 | unsigned _bkt_i; \ 264 | unsigned _count, _bkt_count; \ 265 | char *_prev; \ 266 | struct UT_hash_handle *_thh; \ 267 | if (head) { \ 268 | _count = 0; \ 269 | for( _bkt_i = 0; _bkt_i < (head)->hh.tbl->num_buckets; _bkt_i++) { \ 270 | _bkt_count = 0; \ 271 | _thh = (head)->hh.tbl->buckets[_bkt_i].hh_head; \ 272 | _prev = NULL; \ 273 | while (_thh) { \ 274 | if (_prev != (char*)(_thh->hh_prev)) { \ 275 | HASH_OOPS("invalid hh_prev %p, actual %p\n", \ 276 | _thh->hh_prev, _prev ); \ 277 | } \ 278 | _bkt_count++; \ 279 | _prev = (char*)(_thh); \ 280 | _thh = _thh->hh_next; \ 281 | } \ 282 | _count += _bkt_count; \ 283 | if ((head)->hh.tbl->buckets[_bkt_i].count != _bkt_count) { \ 284 | HASH_OOPS("invalid bucket count %d, actual %d\n", \ 285 | (head)->hh.tbl->buckets[_bkt_i].count, _bkt_count); \ 286 | } \ 287 | } \ 288 | if (_count != (head)->hh.tbl->num_items) { \ 289 | HASH_OOPS("invalid hh item count %d, actual %d\n", \ 290 | (head)->hh.tbl->num_items, _count ); \ 291 | } \ 292 | /* traverse hh in app order; check next/prev integrity, count */ \ 293 | _count = 0; \ 294 | _prev = NULL; \ 295 | _thh = &(head)->hh; \ 296 | while (_thh) { \ 297 | _count++; \ 298 | if (_prev !=(char*)(_thh->prev)) { \ 299 | HASH_OOPS("invalid prev %p, actual %p\n", \ 300 | _thh->prev, _prev ); \ 301 | } \ 302 | _prev = (char*)ELMT_FROM_HH((head)->hh.tbl, _thh); \ 303 | _thh = ( _thh->next ? (UT_hash_handle*)((char*)(_thh->next) + \ 304 | (head)->hh.tbl->hho) : NULL ); \ 305 | } \ 306 | if (_count != (head)->hh.tbl->num_items) { \ 307 | HASH_OOPS("invalid app item count %d, actual %d\n", \ 308 | (head)->hh.tbl->num_items, _count ); \ 309 | } \ 310 | } \ 311 | } while (0) 312 | #else 313 | #define HASH_FSCK(hh,head) 314 | #endif 315 | 316 | /* When compiled with -DHASH_EMIT_KEYS, length-prefixed keys are emitted to 317 | * the descriptor to which this macro is defined for tuning the hash function. 318 | * The app can #include to get the prototype for write(2). */ 319 | #ifdef HASH_EMIT_KEYS 320 | #define HASH_EMIT_KEY(hh,head,keyptr,fieldlen) \ 321 | do { \ 322 | unsigned _klen = fieldlen; \ 323 | write(HASH_EMIT_KEYS, &_klen, sizeof(_klen)); \ 324 | write(HASH_EMIT_KEYS, keyptr, fieldlen); \ 325 | } while (0) 326 | #else 327 | #define HASH_EMIT_KEY(hh,head,keyptr,fieldlen) 328 | #endif 329 | 330 | /* default to Jenkin's hash unless overridden e.g. DHASH_FUNCTION=HASH_SAX */ 331 | #ifdef HASH_FUNCTION 332 | #define HASH_FCN HASH_FUNCTION 333 | #else 334 | #define HASH_FCN HASH_JEN 335 | #endif 336 | 337 | /* The Bernstein hash function, used in Perl prior to v5.6 */ 338 | #define HASH_BER(key,keylen,num_bkts,hashv,bkt) \ 339 | do { \ 340 | unsigned _hb_keylen=keylen; \ 341 | char *_hb_key=(char*)(key); \ 342 | (hashv) = 0; \ 343 | while (_hb_keylen--) { (hashv) = ((hashv) * 33) + *_hb_key++; } \ 344 | bkt = (hashv) & (num_bkts-1); \ 345 | } while (0) 346 | 347 | 348 | /* SAX/FNV/OAT/JEN hash functions are macro variants of those listed at 349 | * http://eternallyconfuzzled.com/tuts/algorithms/jsw_tut_hashing.aspx */ 350 | #define HASH_SAX(key,keylen,num_bkts,hashv,bkt) \ 351 | do { \ 352 | unsigned _sx_i; \ 353 | char *_hs_key=(char*)(key); \ 354 | hashv = 0; \ 355 | for(_sx_i=0; _sx_i < keylen; _sx_i++) \ 356 | hashv ^= (hashv << 5) + (hashv >> 2) + _hs_key[_sx_i]; \ 357 | bkt = hashv & (num_bkts-1); \ 358 | } while (0) 359 | 360 | #define HASH_FNV(key,keylen,num_bkts,hashv,bkt) \ 361 | do { \ 362 | unsigned _fn_i; \ 363 | char *_hf_key=(char*)(key); \ 364 | hashv = 2166136261UL; \ 365 | for(_fn_i=0; _fn_i < keylen; _fn_i++) \ 366 | hashv = (hashv * 16777619) ^ _hf_key[_fn_i]; \ 367 | bkt = hashv & (num_bkts-1); \ 368 | } while(0) 369 | 370 | #define HASH_OAT(key,keylen,num_bkts,hashv,bkt) \ 371 | do { \ 372 | unsigned _ho_i; \ 373 | char *_ho_key=(char*)(key); \ 374 | hashv = 0; \ 375 | for(_ho_i=0; _ho_i < keylen; _ho_i++) { \ 376 | hashv += _ho_key[_ho_i]; \ 377 | hashv += (hashv << 10); \ 378 | hashv ^= (hashv >> 6); \ 379 | } \ 380 | hashv += (hashv << 3); \ 381 | hashv ^= (hashv >> 11); \ 382 | hashv += (hashv << 15); \ 383 | bkt = hashv & (num_bkts-1); \ 384 | } while(0) 385 | 386 | #define HASH_JEN_MIX(a,b,c) \ 387 | do { \ 388 | a -= b; a -= c; a ^= ( c >> 13 ); \ 389 | b -= c; b -= a; b ^= ( a << 8 ); \ 390 | c -= a; c -= b; c ^= ( b >> 13 ); \ 391 | a -= b; a -= c; a ^= ( c >> 12 ); \ 392 | b -= c; b -= a; b ^= ( a << 16 ); \ 393 | c -= a; c -= b; c ^= ( b >> 5 ); \ 394 | a -= b; a -= c; a ^= ( c >> 3 ); \ 395 | b -= c; b -= a; b ^= ( a << 10 ); \ 396 | c -= a; c -= b; c ^= ( b >> 15 ); \ 397 | } while (0) 398 | 399 | #define HASH_JEN(key,keylen,num_bkts,hashv,bkt) \ 400 | do { \ 401 | unsigned _hj_i,_hj_j,_hj_k; \ 402 | char *_hj_key=(char*)(key); \ 403 | hashv = 0xfeedbeef; \ 404 | _hj_i = _hj_j = 0x9e3779b9; \ 405 | _hj_k = (unsigned)keylen; \ 406 | while (_hj_k >= 12) { \ 407 | _hj_i += (_hj_key[0] + ( (unsigned)_hj_key[1] << 8 ) \ 408 | + ( (unsigned)_hj_key[2] << 16 ) \ 409 | + ( (unsigned)_hj_key[3] << 24 ) ); \ 410 | _hj_j += (_hj_key[4] + ( (unsigned)_hj_key[5] << 8 ) \ 411 | + ( (unsigned)_hj_key[6] << 16 ) \ 412 | + ( (unsigned)_hj_key[7] << 24 ) ); \ 413 | hashv += (_hj_key[8] + ( (unsigned)_hj_key[9] << 8 ) \ 414 | + ( (unsigned)_hj_key[10] << 16 ) \ 415 | + ( (unsigned)_hj_key[11] << 24 ) ); \ 416 | \ 417 | HASH_JEN_MIX(_hj_i, _hj_j, hashv); \ 418 | \ 419 | _hj_key += 12; \ 420 | _hj_k -= 12; \ 421 | } \ 422 | hashv += keylen; \ 423 | switch ( _hj_k ) { \ 424 | case 11: hashv += ( (unsigned)_hj_key[10] << 24 ); \ 425 | case 10: hashv += ( (unsigned)_hj_key[9] << 16 ); \ 426 | case 9: hashv += ( (unsigned)_hj_key[8] << 8 ); \ 427 | case 8: _hj_j += ( (unsigned)_hj_key[7] << 24 ); \ 428 | case 7: _hj_j += ( (unsigned)_hj_key[6] << 16 ); \ 429 | case 6: _hj_j += ( (unsigned)_hj_key[5] << 8 ); \ 430 | case 5: _hj_j += _hj_key[4]; \ 431 | case 4: _hj_i += ( (unsigned)_hj_key[3] << 24 ); \ 432 | case 3: _hj_i += ( (unsigned)_hj_key[2] << 16 ); \ 433 | case 2: _hj_i += ( (unsigned)_hj_key[1] << 8 ); \ 434 | case 1: _hj_i += _hj_key[0]; \ 435 | } \ 436 | HASH_JEN_MIX(_hj_i, _hj_j, hashv); \ 437 | bkt = hashv & (num_bkts-1); \ 438 | } while(0) 439 | 440 | /* The Paul Hsieh hash function */ 441 | #undef get16bits 442 | #if (defined(__GNUC__) && defined(__i386__)) || defined(__WATCOMC__) \ 443 | || defined(_MSC_VER) || defined (__BORLANDC__) || defined (__TURBOC__) 444 | #define get16bits(d) (*((const uint16_t *) (d))) 445 | #endif 446 | 447 | #if !defined (get16bits) 448 | #define get16bits(d) ((((uint32_t)(((const uint8_t *)(d))[1])) << 8) \ 449 | +(uint32_t)(((const uint8_t *)(d))[0]) ) 450 | #endif 451 | #define HASH_SFH(key,keylen,num_bkts,hashv,bkt) \ 452 | do { \ 453 | char *_sfh_key=(char*)(key); \ 454 | uint32_t _sfh_tmp, _sfh_len = keylen; \ 455 | \ 456 | int _sfh_rem = _sfh_len & 3; \ 457 | _sfh_len >>= 2; \ 458 | hashv = 0xcafebabe; \ 459 | \ 460 | /* Main loop */ \ 461 | for (;_sfh_len > 0; _sfh_len--) { \ 462 | hashv += get16bits (_sfh_key); \ 463 | _sfh_tmp = (get16bits (_sfh_key+2) << 11) ^ hashv; \ 464 | hashv = (hashv << 16) ^ _sfh_tmp; \ 465 | _sfh_key += 2*sizeof (uint16_t); \ 466 | hashv += hashv >> 11; \ 467 | } \ 468 | \ 469 | /* Handle end cases */ \ 470 | switch (_sfh_rem) { \ 471 | case 3: hashv += get16bits (_sfh_key); \ 472 | hashv ^= hashv << 16; \ 473 | hashv ^= _sfh_key[sizeof (uint16_t)] << 18; \ 474 | hashv += hashv >> 11; \ 475 | break; \ 476 | case 2: hashv += get16bits (_sfh_key); \ 477 | hashv ^= hashv << 11; \ 478 | hashv += hashv >> 17; \ 479 | break; \ 480 | case 1: hashv += *_sfh_key; \ 481 | hashv ^= hashv << 10; \ 482 | hashv += hashv >> 1; \ 483 | } \ 484 | \ 485 | /* Force "avalanching" of final 127 bits */ \ 486 | hashv ^= hashv << 3; \ 487 | hashv += hashv >> 5; \ 488 | hashv ^= hashv << 4; \ 489 | hashv += hashv >> 17; \ 490 | hashv ^= hashv << 25; \ 491 | hashv += hashv >> 6; \ 492 | bkt = hashv & (num_bkts-1); \ 493 | } while(0) 494 | 495 | #ifdef HASH_USING_NO_STRICT_ALIASING 496 | /* The MurmurHash exploits some CPU's (x86,x86_64) tolerance for unaligned reads. 497 | * For other types of CPU's (e.g. Sparc) an unaligned read causes a bus error. 498 | * MurmurHash uses the faster approach only on CPU's where we know it's safe. 499 | * 500 | * Note the preprocessor built-in defines can be emitted using: 501 | * 502 | * gcc -m64 -dM -E - < /dev/null (on gcc) 503 | * cc -## a.c (where a.c is a simple test file) (Sun Studio) 504 | */ 505 | #if (defined(__i386__) || defined(__x86_64__)) 506 | #define MUR_GETBLOCK(p,i) p[i] 507 | #else /* non intel */ 508 | #define MUR_PLUS0_ALIGNED(p) (((unsigned long)p & 0x3) == 0) 509 | #define MUR_PLUS1_ALIGNED(p) (((unsigned long)p & 0x3) == 1) 510 | #define MUR_PLUS2_ALIGNED(p) (((unsigned long)p & 0x3) == 2) 511 | #define MUR_PLUS3_ALIGNED(p) (((unsigned long)p & 0x3) == 3) 512 | #define WP(p) ((uint32_t*)((unsigned long)(p) & ~3UL)) 513 | #if (defined(__BIG_ENDIAN__) || defined(SPARC) || defined(__ppc__) || defined(__ppc64__)) 514 | #define MUR_THREE_ONE(p) ((((*WP(p))&0x00ffffff) << 8) | (((*(WP(p)+1))&0xff000000) >> 24)) 515 | #define MUR_TWO_TWO(p) ((((*WP(p))&0x0000ffff) <<16) | (((*(WP(p)+1))&0xffff0000) >> 16)) 516 | #define MUR_ONE_THREE(p) ((((*WP(p))&0x000000ff) <<24) | (((*(WP(p)+1))&0xffffff00) >> 8)) 517 | #else /* assume little endian non-intel */ 518 | #define MUR_THREE_ONE(p) ((((*WP(p))&0xffffff00) >> 8) | (((*(WP(p)+1))&0x000000ff) << 24)) 519 | #define MUR_TWO_TWO(p) ((((*WP(p))&0xffff0000) >>16) | (((*(WP(p)+1))&0x0000ffff) << 16)) 520 | #define MUR_ONE_THREE(p) ((((*WP(p))&0xff000000) >>24) | (((*(WP(p)+1))&0x00ffffff) << 8)) 521 | #endif 522 | #define MUR_GETBLOCK(p,i) (MUR_PLUS0_ALIGNED(p) ? ((p)[i]) : \ 523 | (MUR_PLUS1_ALIGNED(p) ? MUR_THREE_ONE(p) : \ 524 | (MUR_PLUS2_ALIGNED(p) ? MUR_TWO_TWO(p) : \ 525 | MUR_ONE_THREE(p)))) 526 | #endif 527 | #define MUR_ROTL32(x,r) (((x) << (r)) | ((x) >> (32 - (r)))) 528 | #define MUR_FMIX(_h) \ 529 | do { \ 530 | _h ^= _h >> 16; \ 531 | _h *= 0x85ebca6b; \ 532 | _h ^= _h >> 13; \ 533 | _h *= 0xc2b2ae35l; \ 534 | _h ^= _h >> 16; \ 535 | } while(0) 536 | 537 | #define HASH_MUR(key,keylen,num_bkts,hashv,bkt) \ 538 | do { \ 539 | const uint8_t *_mur_data = (const uint8_t*)(key); \ 540 | const int _mur_nblocks = (keylen) / 4; \ 541 | uint32_t _mur_h1 = 0xf88D5353; \ 542 | uint32_t _mur_c1 = 0xcc9e2d51; \ 543 | uint32_t _mur_c2 = 0x1b873593; \ 544 | const uint32_t *_mur_blocks = (const uint32_t*)(_mur_data+_mur_nblocks*4); \ 545 | int _mur_i; \ 546 | for(_mur_i = -_mur_nblocks; _mur_i; _mur_i++) { \ 547 | uint32_t _mur_k1 = MUR_GETBLOCK(_mur_blocks,_mur_i); \ 548 | _mur_k1 *= _mur_c1; \ 549 | _mur_k1 = MUR_ROTL32(_mur_k1,15); \ 550 | _mur_k1 *= _mur_c2; \ 551 | \ 552 | _mur_h1 ^= _mur_k1; \ 553 | _mur_h1 = MUR_ROTL32(_mur_h1,13); \ 554 | _mur_h1 = _mur_h1*5+0xe6546b64; \ 555 | } \ 556 | const uint8_t *_mur_tail = (const uint8_t*)(_mur_data + _mur_nblocks*4); \ 557 | uint32_t _mur_k1=0; \ 558 | switch((keylen) & 3) { \ 559 | case 3: _mur_k1 ^= _mur_tail[2] << 16; \ 560 | case 2: _mur_k1 ^= _mur_tail[1] << 8; \ 561 | case 1: _mur_k1 ^= _mur_tail[0]; \ 562 | _mur_k1 *= _mur_c1; \ 563 | _mur_k1 = MUR_ROTL32(_mur_k1,15); \ 564 | _mur_k1 *= _mur_c2; \ 565 | _mur_h1 ^= _mur_k1; \ 566 | } \ 567 | _mur_h1 ^= (keylen); \ 568 | MUR_FMIX(_mur_h1); \ 569 | hashv = _mur_h1; \ 570 | bkt = hashv & (num_bkts-1); \ 571 | } while(0) 572 | #endif /* HASH_USING_NO_STRICT_ALIASING */ 573 | 574 | /* key comparison function; return 0 if keys equal */ 575 | #define HASH_KEYCMP(a,b,len) memcmp(a,b,len) 576 | 577 | /* iterate over items in a known bucket to find desired item */ 578 | #define HASH_FIND_IN_BKT(tbl,hh,head,keyptr,keylen_in,out) \ 579 | do { \ 580 | if (head.hh_head) DECLTYPE_ASSIGN(out,ELMT_FROM_HH(tbl,head.hh_head)); \ 581 | else out=NULL; \ 582 | while (out) { \ 583 | if ((out)->hh.keylen == keylen_in) { \ 584 | if ((HASH_KEYCMP((out)->hh.key,keyptr,keylen_in)) == 0) break; \ 585 | } \ 586 | if ((out)->hh.hh_next) DECLTYPE_ASSIGN(out,ELMT_FROM_HH(tbl,(out)->hh.hh_next)); \ 587 | else out = NULL; \ 588 | } \ 589 | } while(0) 590 | 591 | /* add an item to a bucket */ 592 | #define HASH_ADD_TO_BKT(head,addhh) \ 593 | do { \ 594 | head.count++; \ 595 | (addhh)->hh_next = head.hh_head; \ 596 | (addhh)->hh_prev = NULL; \ 597 | if (head.hh_head) { (head).hh_head->hh_prev = (addhh); } \ 598 | (head).hh_head=addhh; \ 599 | if (head.count >= ((head.expand_mult+1) * HASH_BKT_CAPACITY_THRESH) \ 600 | && (addhh)->tbl->noexpand != 1) { \ 601 | HASH_EXPAND_BUCKETS((addhh)->tbl); \ 602 | } \ 603 | } while(0) 604 | 605 | /* remove an item from a given bucket */ 606 | #define HASH_DEL_IN_BKT(hh,head,hh_del) \ 607 | (head).count--; \ 608 | if ((head).hh_head == hh_del) { \ 609 | (head).hh_head = hh_del->hh_next; \ 610 | } \ 611 | if (hh_del->hh_prev) { \ 612 | hh_del->hh_prev->hh_next = hh_del->hh_next; \ 613 | } \ 614 | if (hh_del->hh_next) { \ 615 | hh_del->hh_next->hh_prev = hh_del->hh_prev; \ 616 | } 617 | 618 | /* Bucket expansion has the effect of doubling the number of buckets 619 | * and redistributing the items into the new buckets. Ideally the 620 | * items will distribute more or less evenly into the new buckets 621 | * (the extent to which this is true is a measure of the quality of 622 | * the hash function as it applies to the key domain). 623 | * 624 | * With the items distributed into more buckets, the chain length 625 | * (item count) in each bucket is reduced. Thus by expanding buckets 626 | * the hash keeps a bound on the chain length. This bounded chain 627 | * length is the essence of how a hash provides constant time lookup. 628 | * 629 | * The calculation of tbl->ideal_chain_maxlen below deserves some 630 | * explanation. First, keep in mind that we're calculating the ideal 631 | * maximum chain length based on the *new* (doubled) bucket count. 632 | * In fractions this is just n/b (n=number of items,b=new num buckets). 633 | * Since the ideal chain length is an integer, we want to calculate 634 | * ceil(n/b). We don't depend on floating point arithmetic in this 635 | * hash, so to calculate ceil(n/b) with integers we could write 636 | * 637 | * ceil(n/b) = (n/b) + ((n%b)?1:0) 638 | * 639 | * and in fact a previous version of this hash did just that. 640 | * But now we have improved things a bit by recognizing that b is 641 | * always a power of two. We keep its base 2 log handy (call it lb), 642 | * so now we can write this with a bit shift and logical AND: 643 | * 644 | * ceil(n/b) = (n>>lb) + ( (n & (b-1)) ? 1:0) 645 | * 646 | */ 647 | #define HASH_EXPAND_BUCKETS(tbl) \ 648 | do { \ 649 | unsigned _he_bkt; \ 650 | unsigned _he_bkt_i; \ 651 | struct UT_hash_handle *_he_thh, *_he_hh_nxt; \ 652 | UT_hash_bucket *_he_new_buckets, *_he_newbkt; \ 653 | _he_new_buckets = (UT_hash_bucket*)uthash_malloc( \ 654 | 2 * tbl->num_buckets * sizeof(struct UT_hash_bucket)); \ 655 | if (!_he_new_buckets) { uthash_fatal( "out of memory"); } \ 656 | memset(_he_new_buckets, 0, \ 657 | 2 * tbl->num_buckets * sizeof(struct UT_hash_bucket)); \ 658 | tbl->ideal_chain_maxlen = \ 659 | (tbl->num_items >> (tbl->log2_num_buckets+1)) + \ 660 | ((tbl->num_items & ((tbl->num_buckets*2)-1)) ? 1 : 0); \ 661 | tbl->nonideal_items = 0; \ 662 | for(_he_bkt_i = 0; _he_bkt_i < tbl->num_buckets; _he_bkt_i++) \ 663 | { \ 664 | _he_thh = tbl->buckets[ _he_bkt_i ].hh_head; \ 665 | while (_he_thh) { \ 666 | _he_hh_nxt = _he_thh->hh_next; \ 667 | HASH_TO_BKT( _he_thh->hashv, tbl->num_buckets*2, _he_bkt); \ 668 | _he_newbkt = &(_he_new_buckets[ _he_bkt ]); \ 669 | if (++(_he_newbkt->count) > tbl->ideal_chain_maxlen) { \ 670 | tbl->nonideal_items++; \ 671 | _he_newbkt->expand_mult = _he_newbkt->count / \ 672 | tbl->ideal_chain_maxlen; \ 673 | } \ 674 | _he_thh->hh_prev = NULL; \ 675 | _he_thh->hh_next = _he_newbkt->hh_head; \ 676 | if (_he_newbkt->hh_head) _he_newbkt->hh_head->hh_prev = \ 677 | _he_thh; \ 678 | _he_newbkt->hh_head = _he_thh; \ 679 | _he_thh = _he_hh_nxt; \ 680 | } \ 681 | } \ 682 | uthash_free( tbl->buckets, tbl->num_buckets*sizeof(struct UT_hash_bucket) ); \ 683 | tbl->num_buckets *= 2; \ 684 | tbl->log2_num_buckets++; \ 685 | tbl->buckets = _he_new_buckets; \ 686 | tbl->ineff_expands = (tbl->nonideal_items > (tbl->num_items >> 1)) ? \ 687 | (tbl->ineff_expands+1) : 0; \ 688 | if (tbl->ineff_expands > 1) { \ 689 | tbl->noexpand=1; \ 690 | uthash_noexpand_fyi(tbl); \ 691 | } \ 692 | uthash_expand_fyi(tbl); \ 693 | } while(0) 694 | 695 | 696 | /* This is an adaptation of Simon Tatham's O(n log(n)) mergesort */ 697 | /* Note that HASH_SORT assumes the hash handle name to be hh. 698 | * HASH_SRT was added to allow the hash handle name to be passed in. */ 699 | #define HASH_SORT(head,cmpfcn) HASH_SRT(hh,head,cmpfcn) 700 | #define HASH_SRT(hh,head,cmpfcn) \ 701 | do { \ 702 | unsigned _hs_i; \ 703 | unsigned _hs_looping,_hs_nmerges,_hs_insize,_hs_psize,_hs_qsize; \ 704 | struct UT_hash_handle *_hs_p, *_hs_q, *_hs_e, *_hs_list, *_hs_tail; \ 705 | if (head) { \ 706 | _hs_insize = 1; \ 707 | _hs_looping = 1; \ 708 | _hs_list = &((head)->hh); \ 709 | while (_hs_looping) { \ 710 | _hs_p = _hs_list; \ 711 | _hs_list = NULL; \ 712 | _hs_tail = NULL; \ 713 | _hs_nmerges = 0; \ 714 | while (_hs_p) { \ 715 | _hs_nmerges++; \ 716 | _hs_q = _hs_p; \ 717 | _hs_psize = 0; \ 718 | for ( _hs_i = 0; _hs_i < _hs_insize; _hs_i++ ) { \ 719 | _hs_psize++; \ 720 | _hs_q = (UT_hash_handle*)((_hs_q->next) ? \ 721 | ((void*)((char*)(_hs_q->next) + \ 722 | (head)->hh.tbl->hho)) : NULL); \ 723 | if (! (_hs_q) ) break; \ 724 | } \ 725 | _hs_qsize = _hs_insize; \ 726 | while ((_hs_psize > 0) || ((_hs_qsize > 0) && _hs_q )) { \ 727 | if (_hs_psize == 0) { \ 728 | _hs_e = _hs_q; \ 729 | _hs_q = (UT_hash_handle*)((_hs_q->next) ? \ 730 | ((void*)((char*)(_hs_q->next) + \ 731 | (head)->hh.tbl->hho)) : NULL); \ 732 | _hs_qsize--; \ 733 | } else if ( (_hs_qsize == 0) || !(_hs_q) ) { \ 734 | _hs_e = _hs_p; \ 735 | _hs_p = (UT_hash_handle*)((_hs_p->next) ? \ 736 | ((void*)((char*)(_hs_p->next) + \ 737 | (head)->hh.tbl->hho)) : NULL); \ 738 | _hs_psize--; \ 739 | } else if (( \ 740 | cmpfcn(DECLTYPE(head)(ELMT_FROM_HH((head)->hh.tbl,_hs_p)), \ 741 | DECLTYPE(head)(ELMT_FROM_HH((head)->hh.tbl,_hs_q))) \ 742 | ) <= 0) { \ 743 | _hs_e = _hs_p; \ 744 | _hs_p = (UT_hash_handle*)((_hs_p->next) ? \ 745 | ((void*)((char*)(_hs_p->next) + \ 746 | (head)->hh.tbl->hho)) : NULL); \ 747 | _hs_psize--; \ 748 | } else { \ 749 | _hs_e = _hs_q; \ 750 | _hs_q = (UT_hash_handle*)((_hs_q->next) ? \ 751 | ((void*)((char*)(_hs_q->next) + \ 752 | (head)->hh.tbl->hho)) : NULL); \ 753 | _hs_qsize--; \ 754 | } \ 755 | if ( _hs_tail ) { \ 756 | _hs_tail->next = ((_hs_e) ? \ 757 | ELMT_FROM_HH((head)->hh.tbl,_hs_e) : NULL); \ 758 | } else { \ 759 | _hs_list = _hs_e; \ 760 | } \ 761 | _hs_e->prev = ((_hs_tail) ? \ 762 | ELMT_FROM_HH((head)->hh.tbl,_hs_tail) : NULL); \ 763 | _hs_tail = _hs_e; \ 764 | } \ 765 | _hs_p = _hs_q; \ 766 | } \ 767 | _hs_tail->next = NULL; \ 768 | if ( _hs_nmerges <= 1 ) { \ 769 | _hs_looping=0; \ 770 | (head)->hh.tbl->tail = _hs_tail; \ 771 | DECLTYPE_ASSIGN(head,ELMT_FROM_HH((head)->hh.tbl, _hs_list)); \ 772 | } \ 773 | _hs_insize *= 2; \ 774 | } \ 775 | HASH_FSCK(hh,head); \ 776 | } \ 777 | } while (0) 778 | 779 | /* This function selects items from one hash into another hash. 780 | * The end result is that the selected items have dual presence 781 | * in both hashes. There is no copy of the items made; rather 782 | * they are added into the new hash through a secondary hash 783 | * hash handle that must be present in the structure. */ 784 | #define HASH_SELECT(hh_dst, dst, hh_src, src, cond) \ 785 | do { \ 786 | unsigned _src_bkt, _dst_bkt; \ 787 | void *_last_elt=NULL, *_elt; \ 788 | UT_hash_handle *_src_hh, *_dst_hh, *_last_elt_hh=NULL; \ 789 | ptrdiff_t _dst_hho = ((char*)(&(dst)->hh_dst) - (char*)(dst)); \ 790 | if (src) { \ 791 | for(_src_bkt=0; _src_bkt < (src)->hh_src.tbl->num_buckets; _src_bkt++) { \ 792 | for(_src_hh = (src)->hh_src.tbl->buckets[_src_bkt].hh_head; \ 793 | _src_hh; \ 794 | _src_hh = _src_hh->hh_next) { \ 795 | _elt = ELMT_FROM_HH((src)->hh_src.tbl, _src_hh); \ 796 | if (cond(_elt)) { \ 797 | _dst_hh = (UT_hash_handle*)(((char*)_elt) + _dst_hho); \ 798 | _dst_hh->key = _src_hh->key; \ 799 | _dst_hh->keylen = _src_hh->keylen; \ 800 | _dst_hh->hashv = _src_hh->hashv; \ 801 | _dst_hh->prev = _last_elt; \ 802 | _dst_hh->next = NULL; \ 803 | if (_last_elt_hh) { _last_elt_hh->next = _elt; } \ 804 | if (!dst) { \ 805 | DECLTYPE_ASSIGN(dst,_elt); \ 806 | HASH_MAKE_TABLE(hh_dst,dst); \ 807 | } else { \ 808 | _dst_hh->tbl = (dst)->hh_dst.tbl; \ 809 | } \ 810 | HASH_TO_BKT(_dst_hh->hashv, _dst_hh->tbl->num_buckets, _dst_bkt); \ 811 | HASH_ADD_TO_BKT(_dst_hh->tbl->buckets[_dst_bkt],_dst_hh); \ 812 | (dst)->hh_dst.tbl->num_items++; \ 813 | _last_elt = _elt; \ 814 | _last_elt_hh = _dst_hh; \ 815 | } \ 816 | } \ 817 | } \ 818 | } \ 819 | HASH_FSCK(hh_dst,dst); \ 820 | } while (0) 821 | 822 | #define HASH_CLEAR(hh,head) \ 823 | do { \ 824 | if (head) { \ 825 | uthash_free((head)->hh.tbl->buckets, \ 826 | (head)->hh.tbl->num_buckets*sizeof(struct UT_hash_bucket)); \ 827 | HASH_BLOOM_FREE((head)->hh.tbl); \ 828 | uthash_free((head)->hh.tbl, sizeof(UT_hash_table)); \ 829 | (head)=NULL; \ 830 | } \ 831 | } while(0) 832 | 833 | #ifdef NO_DECLTYPE 834 | #define HASH_ITER(hh,head,el,tmp) \ 835 | for((el)=(head), (*(char**)(&(tmp)))=(char*)((head)?(head)->hh.next:NULL); \ 836 | el; (el)=(tmp),(*(char**)(&(tmp)))=(char*)((tmp)?(tmp)->hh.next:NULL)) 837 | #else 838 | #define HASH_ITER(hh,head,el,tmp) \ 839 | for((el)=(head),(tmp)=DECLTYPE(el)((head)?(head)->hh.next:NULL); \ 840 | el; (el)=(tmp),(tmp)=DECLTYPE(el)((tmp)?(tmp)->hh.next:NULL)) 841 | #endif 842 | 843 | /* obtain a count of items in the hash */ 844 | #define HASH_COUNT(head) HASH_CNT(hh,head) 845 | #define HASH_CNT(hh,head) ((head)?((head)->hh.tbl->num_items):0) 846 | 847 | typedef struct UT_hash_bucket { 848 | struct UT_hash_handle *hh_head; 849 | unsigned count; 850 | 851 | /* expand_mult is normally set to 0. In this situation, the max chain length 852 | * threshold is enforced at its default value, HASH_BKT_CAPACITY_THRESH. (If 853 | * the bucket's chain exceeds this length, bucket expansion is triggered). 854 | * However, setting expand_mult to a non-zero value delays bucket expansion 855 | * (that would be triggered by additions to this particular bucket) 856 | * until its chain length reaches a *multiple* of HASH_BKT_CAPACITY_THRESH. 857 | * (The multiplier is simply expand_mult+1). The whole idea of this 858 | * multiplier is to reduce bucket expansions, since they are expensive, in 859 | * situations where we know that a particular bucket tends to be overused. 860 | * It is better to let its chain length grow to a longer yet-still-bounded 861 | * value, than to do an O(n) bucket expansion too often. 862 | */ 863 | unsigned expand_mult; 864 | 865 | } UT_hash_bucket; 866 | 867 | /* random signature used only to find hash tables in external analysis */ 868 | #define HASH_SIGNATURE 0xa0111fe1 869 | #define HASH_BLOOM_SIGNATURE 0xb12220f2 870 | 871 | typedef struct UT_hash_table { 872 | UT_hash_bucket *buckets; 873 | unsigned num_buckets, log2_num_buckets; 874 | unsigned num_items; 875 | struct UT_hash_handle *tail; /* tail hh in app order, for fast append */ 876 | ptrdiff_t hho; /* hash handle offset (byte pos of hash handle in element */ 877 | 878 | /* in an ideal situation (all buckets used equally), no bucket would have 879 | * more than ceil(#items/#buckets) items. that's the ideal chain length. */ 880 | unsigned ideal_chain_maxlen; 881 | 882 | /* nonideal_items is the number of items in the hash whose chain position 883 | * exceeds the ideal chain maxlen. these items pay the penalty for an uneven 884 | * hash distribution; reaching them in a chain traversal takes >ideal steps */ 885 | unsigned nonideal_items; 886 | 887 | /* ineffective expands occur when a bucket doubling was performed, but 888 | * afterward, more than half the items in the hash had nonideal chain 889 | * positions. If this happens on two consecutive expansions we inhibit any 890 | * further expansion, as it's not helping; this happens when the hash 891 | * function isn't a good fit for the key domain. When expansion is inhibited 892 | * the hash will still work, albeit no longer in constant time. */ 893 | unsigned ineff_expands, noexpand; 894 | 895 | uint32_t signature; /* used only to find hash tables in external analysis */ 896 | #ifdef HASH_BLOOM 897 | uint32_t bloom_sig; /* used only to test bloom exists in external analysis */ 898 | uint8_t *bloom_bv; 899 | char bloom_nbits; 900 | #endif 901 | 902 | } UT_hash_table; 903 | 904 | typedef struct UT_hash_handle { 905 | struct UT_hash_table *tbl; 906 | void *prev; /* prev element in app order */ 907 | void *next; /* next element in app order */ 908 | struct UT_hash_handle *hh_prev; /* previous hh in bucket order */ 909 | struct UT_hash_handle *hh_next; /* next hh in bucket order */ 910 | void *key; /* ptr to enclosing struct's key */ 911 | unsigned keylen; /* enclosing struct's key len */ 912 | unsigned hashv; /* result of hash-fcn(key) */ 913 | } UT_hash_handle; 914 | 915 | #endif /* UTHASH_H */ 916 | -------------------------------------------------------------------------------- /include/emdb.hrl: -------------------------------------------------------------------------------- 1 | %%------------------------------------------------------------------- 2 | %% This file is part of EMDB - Erlang MDB API 3 | %% 4 | %% Copyright (c) 2012 by Aleph Archives. All rights reserved. 5 | %% 6 | %%------------------------------------------------------------------- 7 | %% Redistribution and use in source and binary forms, with or without 8 | %% modification, are permitted only as authorized by the OpenLDAP 9 | %% Public License. 10 | %% 11 | %% A copy of this license is available in the file LICENSE in the 12 | %% top-level directory of the distribution or, alternatively, at 13 | %% . 14 | %% 15 | %% Permission to use, copy, modify, and distribute this software for any 16 | %% purpose with or without fee is hereby granted, provided that the above 17 | %% copyright notice and this permission notice appear in all copies. 18 | %% 19 | %% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 20 | %% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 21 | %% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 22 | %% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 23 | %% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 24 | %% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 25 | %% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 26 | %%------------------------------------------------------------------- 27 | 28 | 29 | %% Environment Flags 30 | -define(MDB_FIXEDMAP, 16#01). 31 | -define(MDB_NOSUBDIR, 16#02). 32 | -define(MDB_NOSYNC, 16#10000). 33 | -define(MDB_RDONLY, 16#20000). 34 | -define(MDB_NOMETASYNC, 16#40000). 35 | -define(MDB_WRITEMAP, 16#80000). 36 | -define(MDB_MAPASYNC, 16#100000). 37 | -------------------------------------------------------------------------------- /rebar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alepharchives/emdb/2945d0096f438a940d1873a7113767d3f1878d31/rebar -------------------------------------------------------------------------------- /rebar.config: -------------------------------------------------------------------------------- 1 | %% -*- erlang -*- 2 | 3 | {erl_opts, [ 4 | debug_info, 5 | warnings_as_errors, 6 | warn_export_all 7 | ]}. 8 | 9 | {port_specs, [ 10 | {"unix", "priv/emdb_drv.so", ["c_src/*.c"]}, 11 | {"linux", "priv/emdb_drv.so", ["c_src/*.c"]}, 12 | {"darwin", "priv/emdb_drv.so", ["c_src/*.c"]}, 13 | {"win32", "priv/emdb_drv.dll", ["c_src/*.c"]} 14 | ]}. 15 | 16 | {port_env, [ 17 | {".*", "CFLAGS", "-O2 -Wall"} 18 | ]}. 19 | -------------------------------------------------------------------------------- /src/emdb.app.src: -------------------------------------------------------------------------------- 1 | {application, emdb, 2 | [ 3 | {description, "Memory-Mapped Database"}, 4 | {vsn, "0.9.0"}, 5 | {registered, []}, 6 | {applications, [ 7 | kernel, 8 | stdlib 9 | ]}, 10 | {env, []} 11 | ]}. 12 | -------------------------------------------------------------------------------- /src/emdb.erl: -------------------------------------------------------------------------------- 1 | %%------------------------------------------------------------------- 2 | %% This file is part of EMDB - Erlang MDB API 3 | %% 4 | %% Copyright (c) 2012 by Aleph Archives. All rights reserved. 5 | %% 6 | %%------------------------------------------------------------------- 7 | %% Redistribution and use in source and binary forms, with or without 8 | %% modification, are permitted only as authorized by the OpenLDAP 9 | %% Public License. 10 | %% 11 | %% A copy of this license is available in the file LICENSE in the 12 | %% top-level directory of the distribution or, alternatively, at 13 | %% . 14 | %% 15 | %% Permission to use, copy, modify, and distribute this software for any 16 | %% purpose with or without fee is hereby granted, provided that the above 17 | %% copyright notice and this permission notice appear in all copies. 18 | %% 19 | %% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 20 | %% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 21 | %% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 22 | %% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 23 | %% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 24 | %% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 25 | %% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 26 | %%------------------------------------------------------------------- 27 | 28 | -module(emdb). 29 | 30 | 31 | %%==================================================================== 32 | %% EXPORTS 33 | %%==================================================================== 34 | -export([ 35 | open/1, 36 | open/2, 37 | open/3 38 | ]). 39 | 40 | 41 | %%==================================================================== 42 | %% Includes 43 | %%==================================================================== 44 | -include("emdb.hrl"). 45 | 46 | 47 | %%==================================================================== 48 | %% Macros 49 | %%==================================================================== 50 | -define(MDB_MAP_SIZE, 10485760). %% 10MB 51 | 52 | %%==================================================================== 53 | %% PUBLIC API 54 | %%==================================================================== 55 | 56 | %%-------------------------------------------------------------------- 57 | %% @doc Create a new MDB database 58 | %% @end 59 | %%-------------------------------------------------------------------- 60 | open(DirName) -> 61 | open(DirName, ?MDB_MAP_SIZE). 62 | open(DirName, MapSize) when is_integer(MapSize) andalso MapSize > 0 -> 63 | open(DirName, MapSize, 0). 64 | open(DirName, MapSize, EnvFlags) when is_integer(MapSize) andalso MapSize > 0 andalso is_integer(EnvFlags) andalso EnvFlags >= 0 -> 65 | %% ensure directory exists 66 | ok = filelib:ensure_dir(DirName ++ "/"), 67 | decorate(emdb_drv:open(DirName, MapSize, EnvFlags)). 68 | 69 | %%==================================================================== 70 | %% PRIVATE API 71 | %%==================================================================== 72 | 73 | %% @private 74 | decorate({ok, Handle}) -> 75 | CDB = emdb_oop:new(Handle), 76 | {ok, CDB}; 77 | 78 | decorate(Error) -> 79 | Error. 80 | -------------------------------------------------------------------------------- /src/emdb_drv.erl: -------------------------------------------------------------------------------- 1 | %%------------------------------------------------------------------- 2 | %% This file is part of EMDB - Erlang MDB API 3 | %% 4 | %% Copyright (c) 2012 by Aleph Archives. All rights reserved. 5 | %% 6 | %%------------------------------------------------------------------- 7 | %% Redistribution and use in source and binary forms, with or without 8 | %% modification, are permitted only as authorized by the OpenLDAP 9 | %% Public License. 10 | %% 11 | %% A copy of this license is available in the file LICENSE in the 12 | %% top-level directory of the distribution or, alternatively, at 13 | %% . 14 | %% 15 | %% Permission to use, copy, modify, and distribute this software for any 16 | %% purpose with or without fee is hereby granted, provided that the above 17 | %% copyright notice and this permission notice appear in all copies. 18 | %% 19 | %% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 20 | %% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 21 | %% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 22 | %% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 23 | %% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 24 | %% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 25 | %% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 26 | %%------------------------------------------------------------------- 27 | 28 | -module(emdb_drv). 29 | 30 | %%==================================================================== 31 | %% EXPORTS 32 | %%==================================================================== 33 | -export([ 34 | open/3, 35 | close/1, 36 | 37 | put/3, 38 | get/2, 39 | del/2, 40 | 41 | update/3, 42 | 43 | drop/1 44 | ]). 45 | 46 | 47 | %% internal export (ex. spawn, apply) 48 | -on_load(init/0). 49 | 50 | %%==================================================================== 51 | %% MACROS 52 | %%==================================================================== 53 | -define(EMDB_DRIVER_NAME, "emdb_drv"). 54 | -define(NOT_LOADED, not_loaded(?LINE)). 55 | 56 | 57 | %%==================================================================== 58 | %% PUBLIC API 59 | %%==================================================================== 60 | 61 | %%-------------------------------------------------------------------- 62 | %% @doc 63 | %% @end 64 | %%-------------------------------------------------------------------- 65 | open(_DirName, _MapSize, _EnvFlags) -> 66 | ?NOT_LOADED. 67 | 68 | %%-------------------------------------------------------------------- 69 | %% @doc 70 | %% @end 71 | %%-------------------------------------------------------------------- 72 | close(_Handle) -> 73 | ?NOT_LOADED. 74 | 75 | %%-------------------------------------------------------------------- 76 | %% @doc 77 | %% @end 78 | %%-------------------------------------------------------------------- 79 | put(_Handle, _Key, _Val) -> 80 | ?NOT_LOADED. 81 | 82 | 83 | %%-------------------------------------------------------------------- 84 | %% @doc 85 | %% @end 86 | %%-------------------------------------------------------------------- 87 | get(_Handle, _Key) -> 88 | ?NOT_LOADED. 89 | 90 | %%-------------------------------------------------------------------- 91 | %% @doc 92 | %% @end 93 | %%-------------------------------------------------------------------- 94 | del(_Handle, _Key) -> 95 | ?NOT_LOADED. 96 | 97 | 98 | %%-------------------------------------------------------------------- 99 | %% @doc 100 | %% @end 101 | %%-------------------------------------------------------------------- 102 | update(_Handle, _Key, _Val) -> 103 | ?NOT_LOADED. 104 | 105 | %%-------------------------------------------------------------------- 106 | %% @doc 107 | %% @end 108 | %%-------------------------------------------------------------------- 109 | drop(_Handle) -> 110 | ?NOT_LOADED. 111 | 112 | %%==================================================================== 113 | %% PRIVATE API 114 | %%==================================================================== 115 | 116 | %%-------------------------------------------------------------------- 117 | %% @doc 118 | %% @end 119 | %%-------------------------------------------------------------------- 120 | init() -> 121 | PrivDir = case code:priv_dir(?MODULE) of 122 | {error, _} -> 123 | EbinDir = filename:dirname(code:which(?MODULE)), 124 | AppPath = filename:dirname(EbinDir), 125 | filename:join(AppPath, "priv"); 126 | Path -> 127 | Path 128 | end, 129 | erlang:load_nif(filename:join(PrivDir, ?EMDB_DRIVER_NAME), 0). 130 | 131 | 132 | %%-------------------------------------------------------------------- 133 | %% @doc 134 | %% @end 135 | %%-------------------------------------------------------------------- 136 | not_loaded(Line) -> 137 | erlang:nif_error({not_loaded, [{module, ?MODULE}, {line, Line}]}). 138 | -------------------------------------------------------------------------------- /src/emdb_oop.erl: -------------------------------------------------------------------------------- 1 | %%------------------------------------------------------------------- 2 | %% This file is part of EMDB - Erlang MDB API 3 | %% 4 | %% Copyright (c) 2012 by Aleph Archives. All rights reserved. 5 | %% 6 | %%------------------------------------------------------------------- 7 | %% Redistribution and use in source and binary forms, with or without 8 | %% modification, are permitted only as authorized by the OpenLDAP 9 | %% Public License. 10 | %% 11 | %% A copy of this license is available in the file LICENSE in the 12 | %% top-level directory of the distribution or, alternatively, at 13 | %% . 14 | %% 15 | %% Permission to use, copy, modify, and distribute this software for any 16 | %% purpose with or without fee is hereby granted, provided that the above 17 | %% copyright notice and this permission notice appear in all copies. 18 | %% 19 | %% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 20 | %% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 21 | %% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 22 | %% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 23 | %% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 24 | %% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 25 | %% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 26 | %%------------------------------------------------------------------- 27 | 28 | -module(emdb_oop, [Handle]). 29 | 30 | 31 | %%==================================================================== 32 | %% EXPORTS 33 | %%==================================================================== 34 | -export([ 35 | close/0, 36 | 37 | put/2, 38 | get/1, 39 | del/1, 40 | 41 | update/2, 42 | 43 | drop/0 44 | ]). 45 | 46 | 47 | %%==================================================================== 48 | %% PUBLIC API 49 | %%==================================================================== 50 | 51 | %%-------------------------------------------------------------------- 52 | %% @doc 53 | %% @end 54 | %%-------------------------------------------------------------------- 55 | close() -> 56 | emdb_drv:close(Handle). 57 | 58 | %%-------------------------------------------------------------------- 59 | %% @doc 60 | %% @end 61 | %%-------------------------------------------------------------------- 62 | put(Key, Val) when is_binary(Key) andalso is_binary(Val) -> 63 | emdb_drv:put(Handle, Key, Val). 64 | 65 | 66 | %%-------------------------------------------------------------------- 67 | %% @doc 68 | %% @end 69 | %%-------------------------------------------------------------------- 70 | get(Key) when is_binary(Key) -> 71 | emdb_drv:get(Handle, Key). 72 | 73 | %%-------------------------------------------------------------------- 74 | %% @doc 75 | %% @end 76 | %%-------------------------------------------------------------------- 77 | del(Key) when is_binary(Key) -> 78 | emdb_drv:del(Handle, Key). 79 | 80 | %%-------------------------------------------------------------------- 81 | %% @doc 82 | %% @end 83 | %%-------------------------------------------------------------------- 84 | update(Key, Val) when is_binary(Key) andalso is_binary(Val) -> 85 | emdb_drv:update(Handle, Key, Val). 86 | 87 | 88 | %%-------------------------------------------------------------------- 89 | %% @doc 90 | %% @end 91 | %%-------------------------------------------------------------------- 92 | drop() -> 93 | emdb_drv:drop(Handle). 94 | 95 | %%==================================================================== 96 | %% INTERNAL FUNCTIONS 97 | %%==================================================================== 98 | -------------------------------------------------------------------------------- /start.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | cd `dirname $0` 4 | exec erl -pa $PWD/ebin -pa +B -- $@ 5 | --------------------------------------------------------------------------------