├── .github └── workflows │ └── tests.yaml ├── .gitignore ├── .readthedocs.yaml ├── MANIFEST.in ├── README.md ├── VEDIS_LICENSE ├── docs ├── Makefile ├── api.rst ├── conf.py ├── custom_commands.rst ├── index.rst ├── installation.rst └── quickstart.rst ├── pyproject.toml ├── setup.py ├── src ├── vedis.c └── vedis.h ├── tests.py └── vedis.pyx /.github/workflows/tests.yaml: -------------------------------------------------------------------------------- 1 | name: Tests 2 | on: [push] 3 | jobs: 4 | tests: 5 | name: ${{ matrix.python-version }} 6 | runs-on: ubuntu-latest 7 | strategy: 8 | fail-fast: false 9 | matrix: 10 | python-version: [3.8, "3.11", "3.13"] 11 | steps: 12 | - uses: actions/checkout@v2 13 | - uses: actions/setup-python@v2 14 | with: 15 | python-version: ${{ matrix.python-version }} 16 | - name: pip deps 17 | run: | 18 | pip install setuptools cython 19 | python setup.py build_ext -i 20 | - name: runtests 21 | run: python tests.py 22 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | vedis.c 2 | vedis*.so 3 | MANIFEST 4 | -------------------------------------------------------------------------------- /.readthedocs.yaml: -------------------------------------------------------------------------------- 1 | version: 2 2 | build: 3 | os: ubuntu-22.04 4 | tools: 5 | python: "3.11" 6 | sphinx: 7 | configuration: docs/conf.py 8 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.md 2 | include LICENSE 3 | include VEDIS_LICENSE 4 | include src/vedis.c 5 | include src/vedis.h 6 | include vedis.c 7 | include vedis.pyx 8 | include tests.py 9 | recursive-include docs * 10 | 11 | global-exclude *.pyc 12 | global-exclude *.o 13 | global-exclude *.so.0.1 14 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ### WARNING: 2 | 3 | [Vedis](https://github.com/symisc/vedis) appears to be no longer maintained by symisc. 4 | 5 | ![](http://media.charlesleifer.com/blog/photos/vedis-python-logo.png) 6 | 7 | Fast Python bindings for the Vedis embedded NoSQL database. Vedis is a fun, fast, embedded database modeled after Redis. 8 | 9 | [View the vedis-python documentation](https://vedis-python.readthedocs.io/). 10 | 11 | ### Features 12 | 13 | Vedis features: 14 | 15 | * Embedded, zero-conf database 16 | * Transactional (ACID) 17 | * Single file or in-memory database 18 | * Key/value store 19 | * [Over 70 commands](http://vedis.symisc.net/commands.html) similar to standard [Redis](http://redis.io) commands. 20 | * Thread-safe 21 | * Terabyte-sized databases 22 | 23 | Vedis-Python features: 24 | 25 | * Compiled library, extremely fast with minimal overhead. 26 | * Supports key/value operations and transactions using Pythonic APIs. 27 | * Support for executing Vedis commands. 28 | * Write custom commands in Python. 29 | * Python 2.x and 3.x. 30 | 31 | Limitations: 32 | 33 | * Not tested on Windoze. 34 | 35 | The previous version (0.2.0) of `vedis-python` utilized `ctypes` to wrap the Vedis C library. By switching to Cython, key/value and Vedis command operations are significantly faster. 36 | 37 | Links: 38 | 39 | * [vedis-python documentation](https://vedis-python.readthedocs.io/) 40 | * [Vedis's C API](http://vedis.symisc.net/c_api.html) 41 | 42 | If you like Vedis, you might also want to check out [UnQLite](http://unqlite.symisc.net), an embedded key/value database with cursors and a cool JSON document store (python bindings: [unqlite-python](https://unqlite-python.readthedocs.io)). 43 | 44 | ## Installation 45 | 46 | You can install vedis-python using `pip`. 47 | 48 | pip install vedis 49 | 50 | Basic usage 51 | ----------- 52 | 53 | First you instantiate a `Vedis` object, passing in either the path to the database 54 | file or the special string `':mem:'` for an in-memory database. 55 | 56 | Below is a sample interactive console session designed to show some of the basic features and functionality of the vedis-python library. Also check out the [full API documentation](https://vedis-python.readthedocs.io/en/latest/api.html) as well as the [vedis command documentation](http://vedis.symisc.net/commands.html). 57 | 58 | ### Key/value features 59 | 60 | You can use Vedis like a dictionary for simple key/value lookups: 61 | 62 | ```python 63 | 64 | >>> from vedis import Vedis 65 | >>> db = Vedis(':mem:') # Create an in-memory database. Alternatively you could supply a filename for an on-disk database. 66 | >>> db['k1'] = 'v1' 67 | >>> db['k1'] 68 | 'v1' 69 | 70 | >>> db.append('k1', 'more data') # Returns length of value after appending new data. 71 | 11 72 | >>> db['k1'] 73 | 'v1more data' 74 | 75 | >>> del db['k1'] 76 | >>> 'k1' in db 77 | False 78 | >>> db['k1'] 79 | None 80 | ``` 81 | 82 | You can set and get multiple items at a time: 83 | 84 | ```python 85 | 86 | >>> db.mset(dict(k1='v1', k2='v2', k3='v3')) 87 | True 88 | 89 | >>> db.mget(['k1', 'k2', 'missing key', 'k3']) 90 | ['v1', 'v2', None, 'v3'] 91 | ``` 92 | 93 | In addition to storing string keys/values, you can also implement counters: 94 | 95 | ```python 96 | 97 | >>> db.incr('counter') 98 | 1 99 | >>> db.incr('counter') 100 | 2 101 | 102 | >>> db.incr_by('counter', 10) 103 | 12 104 | >>> db.decr('counter') 105 | 11 106 | ``` 107 | 108 | ### Hashes 109 | 110 | Vedis supports nested key/value lookups which have the additional benefit of supporting operations to retrieve all keys, values, the number of items in the hash, and so on. 111 | 112 | ```python 113 | 114 | >>> h = db.Hash('some key') 115 | >>> h['k1'] = 'v1' 116 | >>> h.update(k2='v2', k3='v3') 117 | 118 | >>> h 119 | 120 | 121 | >>> h.to_dict() 122 | {'k3': 'v3', 'k2': 'v2', 'k1': 'v1'} 123 | 124 | >>> h.items() 125 | [('k1', 'v1'), ('k3', 'v3'), ('k2', 'v2')] 126 | 127 | >>> h.keys() 128 | ['k1', 'k3', 'k2'] 129 | 130 | >>> del h['k2'] 131 | 132 | >>> len(h) 133 | 2 134 | 135 | >>> 'k1' in h 136 | True 137 | 138 | >>> [key for key in h] 139 | ['k1', 'k3'] 140 | ``` 141 | 142 | ### Sets 143 | 144 | Vedis supports a set data-type which stores a unique collection of items. 145 | 146 | ```python 147 | 148 | >>> s = db.Set('some set') 149 | >>> s.add('v1', 'v2', 'v3') 150 | 3 151 | 152 | >>> len(s) 153 | 3 154 | 155 | >>> 'v1' in s, 'v4' in s 156 | (True, False) 157 | 158 | >>> s.top() 159 | 'v1' 160 | 161 | >>> s.peek() 162 | 'v3' 163 | 164 | >>> del s['v2'] 165 | 1 166 | 167 | >>> s.add('v4', 'v5') 168 | 2 169 | 170 | >>> s.pop() 171 | 'v5' 172 | 173 | >>> [item for item in s] 174 | ['v1', 'v3', 'v4'] 175 | 176 | >>> s.to_set() 177 | set(['v1', 'v3', 'v4']) 178 | 179 | >>> s2 = db.Set('another set') 180 | >>> s2.add('v1', 'v4', 'v5', 'v6') 181 | 4 182 | 183 | >>> s2 & s # Intersection. 184 | set(['v1', 'v4']) 185 | 186 | >>> s2 - s # Difference. 187 | set(['v5', 'v6']) 188 | ``` 189 | 190 | ### Lists 191 | 192 | Vedis also supports a list data type. 193 | 194 | ```python 195 | 196 | >>> l = db.List('my list') 197 | >>> l.append('v1') 198 | 1 199 | >>> l.extend(['v2', 'v3', 'v4']) 200 | 4 201 | 202 | >>> len(l) 203 | 4 204 | 205 | >>> l[1] 206 | 'v2' 207 | 208 | >>> l.pop(), l.pop() 209 | ('v1', 'v2') 210 | 211 | >>> len(l) 212 | 2 213 | ``` 214 | 215 | ### Misc 216 | 217 | Vedis has a somewhat quirky collection of other miscellaneous commands. Below is a sampling: 218 | 219 | ```python 220 | 221 | >>> db.base64('encode me') 222 | 'ZW5jb2RlIG1l' 223 | 224 | >>> db.base64_decode('ZW5jb2RlIG1l') 225 | 'encode me' 226 | 227 | >>> db.random_string(10) 228 | 'raurquvsnx' 229 | 230 | >>> db.rand(1, 6) 231 | 4 232 | 233 | >>> db.str_split('abcdefghijklmnop', 5) 234 | ['abcde', 'fghij', 'klmno', 'p'] 235 | 236 | >>> db['data'] = 'abcdefghijklmnop' 237 | >>> db.strlen('data') 238 | 16 239 | 240 | >>> db.strip_tags('

This is a test.

') 241 | 'This is a test.' 242 | ``` 243 | 244 | ### Writing your own Vedis commands 245 | 246 | It is easy to write your own Vedis commands: 247 | 248 | ```python 249 | 250 | db = Vedis() 251 | 252 | @db.register('CONCAT') 253 | def concat(context, glue, *params): 254 | return glue.join(params) 255 | 256 | @db.register('TITLE') 257 | def title(context, *params): 258 | # The `context` can be used to access the key/value store. 259 | for param in params: 260 | context[param] = param.title() 261 | return True 262 | ``` 263 | 264 | Here is how you might call the custom commands: 265 | 266 | ```python 267 | 268 | >>> print db.execute('CONCAT | foo bar baz') 269 | foo|bar|baz 270 | 271 | >>> db.execute('TITLE "testing" "this is a test" "another"') 272 | True 273 | >>> print db['testing'] 274 | Testing 275 | >>> print db['this is a test'] 276 | This Is A Test 277 | 278 | >>> title('foo', 'bar') # Calling the wrapped function will go through Vedis. 279 | True 280 | >>> print db['foo'] 281 | Foo 282 | >>> print db['bar'] 283 | Bar 284 | ``` 285 | 286 | ------------------------------------------- 287 | 288 | This code is based in part on [buaabyl's pyUnQLite](https://github.com/buaabyl/pyUnQLite/). 289 | -------------------------------------------------------------------------------- /VEDIS_LICENSE: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2013 Symisc Systems, S.U.A.R.L [M.I.A.G Mrad Chems Eddine ]. 3 | * All rights reserved. 4 | * 5 | * Redistribution and use in source and binary forms, with or without 6 | * modification, are permitted provided that the following conditions 7 | * are met: 8 | * 1. Redistributions of source code must retain the above copyright 9 | * notice, this list of conditions and the following disclaimer. 10 | * 2. Redistributions in binary form must reproduce the above copyright 11 | * notice, this list of conditions and the following disclaimer in the 12 | * documentation and/or other materials provided with the distribution. 13 | * 3. Redistributions in any form must be accompanied by information on 14 | * how to obtain complete source code for the Vedis engine and any 15 | * accompanying software that uses the Vedis engine software. 16 | * The source code must either be included in the distribution 17 | * or be available for no more than the cost of distribution plus 18 | * a nominal fee, and must be freely redistributable under reasonable 19 | * conditions. For an executable file, complete source code means 20 | * the source code for all modules it contains.It does not include 21 | * source code for modules or files that typically accompany the major 22 | * components of the operating system on which the executable file runs. 23 | * 24 | * THIS SOFTWARE IS PROVIDED BY SYMISC SYSTEMS ``AS IS'' AND ANY EXPRESS 25 | * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 26 | * WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR 27 | * NON-INFRINGEMENT, ARE DISCLAIMED. IN NO EVENT SHALL SYMISC SYSTEMS 28 | * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 29 | * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 30 | * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR 31 | * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, 32 | * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE 33 | * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN 34 | * IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 35 | */ -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # User-friendly check for sphinx-build 11 | ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) 12 | $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) 13 | endif 14 | 15 | # Internal variables. 16 | PAPEROPT_a4 = -D latex_paper_size=a4 17 | PAPEROPT_letter = -D latex_paper_size=letter 18 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 19 | # the i18n builder cannot share the environment and doctrees with the others 20 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 21 | 22 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 23 | 24 | help: 25 | @echo "Please use \`make ' where is one of" 26 | @echo " html to make standalone HTML files" 27 | @echo " dirhtml to make HTML files named index.html in directories" 28 | @echo " singlehtml to make a single large HTML file" 29 | @echo " pickle to make pickle files" 30 | @echo " json to make JSON files" 31 | @echo " htmlhelp to make HTML files and a HTML help project" 32 | @echo " qthelp to make HTML files and a qthelp project" 33 | @echo " devhelp to make HTML files and a Devhelp project" 34 | @echo " epub to make an epub" 35 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 36 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 37 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 38 | @echo " text to make text files" 39 | @echo " man to make manual pages" 40 | @echo " texinfo to make Texinfo files" 41 | @echo " info to make Texinfo files and run them through makeinfo" 42 | @echo " gettext to make PO message catalogs" 43 | @echo " changes to make an overview of all changed/added/deprecated items" 44 | @echo " xml to make Docutils-native XML files" 45 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 46 | @echo " linkcheck to check all external links for integrity" 47 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 48 | 49 | clean: 50 | rm -rf $(BUILDDIR)/* 51 | 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | dirhtml: 58 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 59 | @echo 60 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 61 | 62 | singlehtml: 63 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 64 | @echo 65 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 66 | 67 | pickle: 68 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 69 | @echo 70 | @echo "Build finished; now you can process the pickle files." 71 | 72 | json: 73 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 74 | @echo 75 | @echo "Build finished; now you can process the JSON files." 76 | 77 | htmlhelp: 78 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 79 | @echo 80 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 81 | ".hhp project file in $(BUILDDIR)/htmlhelp." 82 | 83 | qthelp: 84 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 85 | @echo 86 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 87 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 88 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/vedis-python.qhcp" 89 | @echo "To view the help file:" 90 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/vedis-python.qhc" 91 | 92 | devhelp: 93 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 94 | @echo 95 | @echo "Build finished." 96 | @echo "To view the help file:" 97 | @echo "# mkdir -p $$HOME/.local/share/devhelp/vedis-python" 98 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/vedis-python" 99 | @echo "# devhelp" 100 | 101 | epub: 102 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 103 | @echo 104 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 105 | 106 | latex: 107 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 108 | @echo 109 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 110 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 111 | "(use \`make latexpdf' here to do that automatically)." 112 | 113 | latexpdf: 114 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 115 | @echo "Running LaTeX files through pdflatex..." 116 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 117 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 118 | 119 | latexpdfja: 120 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 121 | @echo "Running LaTeX files through platex and dvipdfmx..." 122 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 123 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 124 | 125 | text: 126 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 127 | @echo 128 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 129 | 130 | man: 131 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 132 | @echo 133 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 134 | 135 | texinfo: 136 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 137 | @echo 138 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 139 | @echo "Run \`make' in that directory to run these through makeinfo" \ 140 | "(use \`make info' here to do that automatically)." 141 | 142 | info: 143 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 144 | @echo "Running Texinfo files through makeinfo..." 145 | make -C $(BUILDDIR)/texinfo info 146 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 147 | 148 | gettext: 149 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 150 | @echo 151 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 152 | 153 | changes: 154 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 155 | @echo 156 | @echo "The overview file is in $(BUILDDIR)/changes." 157 | 158 | linkcheck: 159 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 160 | @echo 161 | @echo "Link check complete; look for any errors in the above output " \ 162 | "or in $(BUILDDIR)/linkcheck/output.txt." 163 | 164 | doctest: 165 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 166 | @echo "Testing of doctests in the sources finished, look at the " \ 167 | "results in $(BUILDDIR)/doctest/output.txt." 168 | 169 | xml: 170 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 171 | @echo 172 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 173 | 174 | pseudoxml: 175 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 176 | @echo 177 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." 178 | -------------------------------------------------------------------------------- /docs/api.rst: -------------------------------------------------------------------------------- 1 | .. _api: 2 | 3 | API Documentation 4 | ================= 5 | 6 | 7 | .. py:class:: Vedis([filename=':mem:'[, open_database=True]]) 8 | 9 | The :py:class:`Vedis` object provides a pythonic interface for interacting 10 | with `vedis databases `_. Vedis is a lightweight, 11 | embedded NoSQL database modeled after Redis. 12 | 13 | :param str filename: The path to the database file. For in-memory databases, you can either leave this parameter empty or specify the string ``:mem:``. 14 | :param bool open_database: When set to ``True``, the database will be opened automatically when the class is instantiated. If set to ``False`` you will need to manually call :py:meth:`~Vedis.open`. 15 | 16 | .. note:: 17 | Vedis supports in-memory databases, which can be created by passing in ``':mem:'`` as the database file. This is the default behavior if no database file is specified. 18 | 19 | Example usage: 20 | 21 | .. code-block:: pycon 22 | 23 | >>> db = Vedis() # Create an in-memory database. 24 | >>> db['foo'] = 'bar' # Use as a key/value store. 25 | >>> print db['foo'] 26 | bar 27 | 28 | >>> db.update({'k0': 'v0', 'k1': 'v1', 'k2': 'v2', 'k3': 'v3'}) 29 | 30 | >>> 'k3' in db 31 | True 32 | >>> 'k4' in db 33 | False 34 | >>> del db['k3'] 35 | 36 | >>> db.append('k2', 'XXXX') 37 | >>> db.mget(['k1', 'k2', 'k3']) 38 | ['1', '2XXXX', None] 39 | 40 | >>> db.incr('counter_1') 41 | 1 42 | >>> db.incr_by('counter_1', 10) 43 | 11 44 | 45 | >>> hash_obj = db.Hash('my-hash') 46 | >>> hash_obj['k1'] = 'v1' 47 | >>> hash_obj.update(k2='v2', k3='v3') 48 | 2 49 | >>> hash_obj.to_dict() 50 | {'k1': 'v1', 'k2': 'v2', 'k3': 'v3'} 51 | >>> hash_obj.items() 52 | [('k1', 'v1'), ('k2', 'v2'), ('k3', 'v3')] 53 | >>> [key for key in hash_obj] 54 | ['k1', 'k2', 'k3'] 55 | >>> len(hash_obj) 56 | 3 57 | 58 | >>> set_obj = db.Set('my-set') 59 | >>> set_obj.add('foo', 'bar', 'baz', 'foo') 60 | 3 61 | >>> 'foo' in set_obj 62 | True 63 | >>> del set_obj['bar'] 64 | 1 65 | >>> set_obj.to_set() 66 | {'baz', 'foo'} 67 | >>> [item for item in set_obj] 68 | ['baz', 'foo'] 69 | >>> len(set_obj) 70 | 2 71 | 72 | >>> list_obj = db.List('my-list') 73 | >>> list_obj.extend(['foo', 'bar', 'baz', 'nug']) 74 | 4 75 | >>> list_obj.append('another') 76 | 5 77 | >>> [item for item in list_obj] 78 | ['foo', 'bar', 'baz', 'nug', 'another'] 79 | >>> list_obj[1] 80 | 'bar' 81 | >>> list_obj.pop() 82 | 'foo' 83 | >>> len(l) 84 | 4 85 | 86 | .. py:method:: open() 87 | 88 | Open the database connection. 89 | 90 | .. py:method:: close() 91 | 92 | Close the database connection. 93 | 94 | .. warning:: 95 | If you are using a file-based database, by default any uncommitted changes will be committed when the database is closed. If you wish to discard uncommitted changes, you can use :py:meth:`~Vedis.disable_autocommit`. 96 | 97 | .. py:method:: __enter__() 98 | 99 | Use the database as a context manager, opening the connection and closing it at the end of the wrapped block: 100 | 101 | .. code-block:: python 102 | 103 | with Vedis('my_db.vdb') as db: 104 | db['foo'] = 'bar' 105 | 106 | # When the context manager exits, the database is closed. 107 | 108 | .. py:method:: disable_autocommit() 109 | 110 | When the database is closed, prevent any uncommitted writes from being saved. 111 | 112 | .. note:: This method only affects file-based databases. 113 | 114 | .. py:method:: set(key, value) 115 | 116 | Store a value in the given key. 117 | 118 | :param str key: Identifier used for storing data. 119 | :param str value: A value to store in Vedis. 120 | 121 | Example: 122 | 123 | .. code-block:: python 124 | 125 | db = Vedis() 126 | db.set('some key', 'some value') 127 | db.set('another key', 'another value') 128 | 129 | You can also use the dictionary-style ``[key] = value`` to store a value: 130 | 131 | .. code-block:: python 132 | 133 | db['some key'] = 'some value' 134 | 135 | .. py:method:: get(key) 136 | 137 | Retrieve the value stored at the given ``key``. If no value exists, a ``KeyError`` will be raised. 138 | 139 | :param str key: Identifier to retrieve. 140 | :returns: The data stored at the given key. 141 | :raises: ``KeyError`` if the given key does not exist. 142 | 143 | Example: 144 | 145 | .. code-block:: python 146 | 147 | db = Vedis() 148 | db.set('some key', 'some value') 149 | value = db.get('some key') 150 | 151 | You can also use the dictionary-style ``[key]`` lookup to retrieve a value: 152 | 153 | .. code-block:: python 154 | 155 | value = db['some key'] 156 | 157 | .. py:method:: delete(key) 158 | 159 | Remove the key and its associated value from the database. 160 | 161 | :param str key: The key to remove from the database. 162 | :raises: ``KeyError`` if the given key does not exist. 163 | 164 | Example: 165 | 166 | .. code-block:: python 167 | 168 | def clear_cache(): 169 | db.delete('cached-data') 170 | 171 | You can also use the python ``del`` keyword combined with a dictionary lookup: 172 | 173 | .. code-block:: python 174 | 175 | def clear_cache(): 176 | del db['cached-data'] 177 | 178 | .. py:method:: append(key, value) 179 | 180 | Append the given ``value`` to the data stored in the ``key``. If no data exists, the operation is equivalent to :py:meth:`~Vedis.set`. 181 | 182 | :param str key: The identifier of the value to append to. 183 | :param value: The value to append. 184 | 185 | .. py:method:: exists(key) 186 | 187 | Return whether the given ``key`` exists in the database. Oddly, this only 188 | seems to work for simple key/value pairs. If, for instance, you have stored 189 | a hash at the given key, ``exists`` will return ``False``. 190 | 191 | :param str key: 192 | :returns: A boolean value indicating whether the given ``key`` exists in the database. 193 | 194 | Example: 195 | 196 | .. code-block:: python 197 | 198 | def get_expensive_data(): 199 | if not db.exists('cached-data'): 200 | db.set('cached-data', calculate_expensive_data()) 201 | return db.get('cached-data') 202 | 203 | You can also use the python ``in`` keyword to determine whether a key exists: 204 | 205 | .. code-block:: python 206 | 207 | def get_expensive_data(): 208 | if 'cached-data' not in db: 209 | db['cached-data'] = calculate_expensive_data() 210 | return db['cached-data'] 211 | 212 | .. py:method:: update(data) 213 | 214 | :param dict data: Dictionary of data to store in the database. 215 | 216 | Set multiple key/value pairs in a single command, similar to Python's ``dict.update()``. 217 | 218 | Example: 219 | 220 | .. code-block:: python 221 | 222 | db = Vedis() 223 | db.update(dict( 224 | hostname=socket.gethostname(), 225 | user=os.environ['USER'], 226 | home_dir=os.environ['HOME'], 227 | path=os.environ['PATH'])) 228 | 229 | .. py:method:: mget(keys) 230 | 231 | Retrieve the values of multiple keys in a single command. In the event a key 232 | does not exist, ``None`` will be returned for that particular value. 233 | 234 | :param list keys: A list of one or more keys to retrieve. 235 | :returns: The values for the given keys. 236 | 237 | Example: 238 | 239 | .. code-block:: pycon 240 | 241 | >>> db.update(dict(k1='v1', k2='v2', k3='v3', k4='v4')) 242 | >>> db.mget(['k1', 'k3', 'missing', 'k4']) 243 | ['v1', 'v3', None, 'v4'] 244 | 245 | .. py:method:: mset(data) 246 | 247 | :param dict data: Dictionary of data to store in the database. 248 | 249 | Set multiple key/value pairs in a single command. This is equivalent to the :py:meth:`~Vedis.update` method. 250 | 251 | .. py:method:: setnx(key, value) 252 | 253 | Set the value for the given key *only* if the key does not exist. 254 | 255 | :returns: ``True`` if the value was set, ``False`` if the key already existed. 256 | 257 | Example: 258 | 259 | .. code-block:: python 260 | 261 | def create_user(email, password_hash): 262 | if db.setnx(email, password_hash): 263 | print 'User added successfully' 264 | return True 265 | else: 266 | print 'Error: username already taken.' 267 | return False 268 | 269 | .. py:method:: msetnx(kwargs) 270 | 271 | Similar to :py:meth:`~Vedis.update`, except that existing keys will not be overwritten. 272 | 273 | :returns: ``True`` on success. 274 | 275 | Example: 276 | 277 | .. code-block:: pycon 278 | 279 | >>> db.msetnx({'k1': 'v1', 'k2': 'v2'}) 280 | >>> db.mget(['k1', 'k2']) 281 | ['v1', 'v2'] 282 | 283 | >>> db.msetnx({'k1': 'v1x', 'k2': 'v2x', 'k3': 'v3x'}) 284 | >>> db.mget(['k1', 'k2', 'k3']) 285 | ['v1', 'v2', 'v3x'] 286 | 287 | .. py:method:: get_set(key, value) 288 | 289 | Get the value at the given ``key`` and set it to the new ``value`` in a single operation. 290 | 291 | :returns: The original value at the given ``key``. 292 | 293 | Example: 294 | 295 | .. code-block:: pycon 296 | 297 | >>> db['k1'] = 'v1' 298 | >>> db.get_set('k1', 'v-x') 299 | 'v1' 300 | 301 | >>> db['k1'] 302 | 'v-x' 303 | 304 | .. py:method:: incr(key) 305 | 306 | Increment the value stored in the given ``key`` by ``1``. If no value exists or the value 307 | is not an integer, the counter will be initialized at zero then incremented. 308 | 309 | :returns: The integer value stored in the given counter. 310 | 311 | .. code-block:: pycon 312 | 313 | >>> db.incr('my-counter') 314 | 1 315 | >>> db.incr('my-counter') 316 | 2 317 | 318 | .. py:method:: decr(key) 319 | 320 | Decrement the value stored in the given ``key`` by ``1``. If no value exists or the value 321 | is not an integer, the counter will be initialized at zero then decremented. 322 | 323 | :returns: The integer value stored in the given counter. 324 | 325 | Example: 326 | 327 | .. code-block:: pycon 328 | 329 | >> db.decr('my-counter') 330 | 3 331 | >> db.decr('my-counter') 332 | 2 333 | >> db.decr('does-not-exist') 334 | -1 335 | 336 | .. py:method:: incr_by(key, amt) 337 | 338 | Increment the given ``key`` by the integer ``amt``. This method has the same behavior as 339 | :py:meth:`~Vedis.incr`. 340 | 341 | .. py:method:: decr_by(key, amt) 342 | 343 | Decrement the given ``key`` by the integer ``amt``. This method has the same behavior as 344 | :py:meth:`~Vedis.decr`. 345 | 346 | .. py:method:: begin() 347 | 348 | Begin a transaction. 349 | 350 | .. py:method:: rollback() 351 | 352 | Roll back the current transaction. 353 | 354 | .. py:method:: commit() 355 | 356 | Commit the current transaction. 357 | 358 | .. py:method:: transaction() 359 | 360 | Create a context manager for performing multiple operations in a transaction. 361 | 362 | .. warning:: 363 | Transactions occur at the disk-level and have no effect on in-memory databases. 364 | 365 | Example: 366 | 367 | .. code-block:: python 368 | 369 | # Transfer $100 in a transaction. 370 | with db.transaction(): 371 | db['from_acct'] = db['from_account'] - 100 372 | db['to_acct'] = db['to_acct'] + 100 373 | 374 | # Make changes and then roll them back. 375 | with db.transaction(): 376 | db['foo'] = 'bar' 377 | db.rollback() # Whoops, do not commit these changes. 378 | 379 | .. py:method:: commit_on_success(fn) 380 | 381 | Function decorator that will cause the wrapped function to have all statements wrapped in a transaction. If the function returns without an exception, the transaction is committed. If an exception occurs in the function, the transaction is rolled back. 382 | 383 | Example: 384 | 385 | .. code-block:: pycon 386 | 387 | >>> @db.commit_on_success 388 | ... def save_value(key, value, exc=False): 389 | ... db[key] = value 390 | ... if exc: 391 | ... raise Exception('uh-oh') 392 | ... 393 | >>> save_value('k3', 'v3') 394 | >>> save_value('k3', 'vx', True) 395 | Traceback (most recent call last): 396 | File "", line 1, in 397 | File "unqlite/core.py", line 312, in wrapper 398 | return fn() 399 | File "", line 5, in save_value 400 | Exception: uh-oh 401 | >>> db['k3'] 402 | 'v3' 403 | 404 | .. py:method:: Hash(key) 405 | 406 | Create a :py:class:`Hash` object, which provides a dictionary-like 407 | interface for working with Vedis hashes. 408 | 409 | :param str key: The key for the Vedis hash object. 410 | :returns: a :py:class:`Hash` object representing the Vedis hash at the 411 | specified key. 412 | 413 | Example: 414 | 415 | .. code-block:: pycon 416 | 417 | >>> my_hash = db.Hash('my_hash') 418 | >>> my_hash.update(k1='v1', k2='v2') 419 | >>> my_hash.to_dict() 420 | {'k2': 'v2', 'k1': 'v1'} 421 | 422 | .. py:method:: hset(hash_key, key, value) 423 | 424 | Set the value for the key in the Vedis hash identified by ``hash_key``. 425 | 426 | Example: 427 | 428 | .. code-block:: pycon 429 | 430 | >>> db.hset('my_hash', 'k3', 'v3') 431 | >>> db.hget('my_hash', 'k3') 432 | 'v3' 433 | 434 | .. py:method:: hsetnx(hash_key, key, value) 435 | 436 | Set a value for the given key in a Vedis hash only if the key 437 | does not already exist. Returns boolean indicating whether the 438 | value was successfully set. 439 | 440 | :rtype: bool 441 | 442 | Example: 443 | 444 | .. code-block:: pycon 445 | 446 | >>> db.hsetnx('my_hash', 'kx', 'vx') 447 | True 448 | >>> db.hsetnx('my_hash', 'kx', 'vx') 449 | False 450 | 451 | .. py:method:: hget(hash_key, key) 452 | 453 | Retrieve the value for the key in the Vedis hash identified by ``hash_key``. 454 | 455 | :returns: The value for the given key, or ``None`` if the key does not 456 | exist. 457 | 458 | Example: 459 | 460 | .. code-block:: pycon 461 | 462 | >>> db.hset('my_hash', 'k3', 'v3') 463 | >>> db.hget('my_hash', 'k3') 464 | 'v3' 465 | 466 | .. py:method:: hdel(hash_key, key) 467 | 468 | Delete a ``key`` from a Vedis hash. If the key does not exist in the 469 | hash, the operation is a no-op. 470 | 471 | :returns: The number of keys deleted. 472 | 473 | Example: 474 | 475 | .. code-block:: pycon 476 | 477 | >>> db.hdel('my_hash', 'k3') 478 | 1 479 | >>> db.hget('my_hash', 'k3') is None 480 | True 481 | 482 | .. py:method:: hkeys(hash_key) 483 | 484 | Get the keys for the Vedis hash identified by ``hash_key``. 485 | 486 | :returns: All keys for the Vedis hash. 487 | 488 | Example: 489 | 490 | .. code-block:: pycon 491 | 492 | >>> db.hkeys('my_hash') 493 | ['k2', 'k1'] 494 | 495 | .. py:method:: hvals(hash_key) 496 | 497 | Get the values for the Vedis hash identified by ``hash_key``. 498 | 499 | :returns: All values for the Vedis hash. 500 | 501 | Example: 502 | 503 | .. code-block:: pycon 504 | 505 | >>> db.hvals('my_hash') 506 | ['v2', 'v1'] 507 | 508 | .. py:method:: hgetall(hash_key) 509 | 510 | Return a ``dict`` containing all items in the Vedis hash identified 511 | by ``hash_key``. 512 | 513 | :returns: A dictionary containing the key/value pairs stored in the 514 | given Vedis hash, or an empty ``dict`` if a hash does not exist at the 515 | given key. 516 | :rtype: dict 517 | 518 | Example: 519 | 520 | .. code-block:: pycon 521 | 522 | >>> db.hgetall('my_hash') 523 | {'k2': 'v2', 'k1': 'v1'} 524 | 525 | >>> db.hgetall('does not exist') 526 | {} 527 | 528 | .. py:method:: hitems(hash_key) 529 | 530 | Get a list to key/value pairs stored in the given Vedis hash. 531 | 532 | :returns: A list of key/value pairs stored in the given Vedis hash, or 533 | an empty list if a hash does not exist at the given key. 534 | :rtype: list of 2-tuples 535 | 536 | Example: 537 | 538 | .. code-block:: pycon 539 | 540 | >>> db.hitems('my_hash') 541 | [('k2', 'v2'), ('k1', 'v1')] 542 | 543 | .. py:method:: hlen(hash_key) 544 | 545 | Return the number of items stored in a Vedis hash. If a hash does not 546 | exist at the given key, ``0`` will be returned. 547 | 548 | :rtype: int 549 | 550 | Example: 551 | 552 | .. code-block:: pycon 553 | 554 | >>> db.hlen('my_hash') 555 | 2 556 | >>> db.hlen('does not exist') 557 | 0 558 | 559 | .. py:method:: hexists(hash_key, key) 560 | 561 | Return whether the given key is stored in a Vedis hash. If a hash does not 562 | exist at the given key, ``False`` will be returned. 563 | 564 | :rtype: bool 565 | 566 | Example: 567 | 568 | .. code-block:: pycon 569 | 570 | >>> db.hexists('my_hash', 'k1') 571 | True 572 | >>> db.hexists('my_hash', 'kx') 573 | False 574 | >>> db.hexists('does not exist', 'kx') 575 | False 576 | 577 | .. py:method:: hmset(hash_key, data) 578 | 579 | Set multiple key/value pairs in the given Vedis hash. This method is 580 | analagous to Python's ``dict.update``. 581 | 582 | Example: 583 | 584 | .. code-block:: pycon 585 | 586 | >>> db.hmset('my_hash', {'k1': 'v1', 'k2': 'v2', 'k3': 'v3', 'k4': 'v4'}) 587 | >>> db.hgetall('my_hash') 588 | {'k3': 'v3', 'k2': 'v2', 'k1': 'v1', 'k4': 'v4'} 589 | 590 | .. py:method:: hmget(hash_key, keys) 591 | 592 | Return the values for multiple keys in a Vedis hash. If the key does 593 | not exist in the given hash, ``None`` will be returned for the missing 594 | key. 595 | 596 | Example: 597 | 598 | .. code-block:: pycon 599 | 600 | >>> db.hmget('my_hash', ['k1', 'k4', 'missing', 'k2']) 601 | ['v1', 'v4', None, 'v2'] 602 | 603 | .. py:method:: hmdel(hash_key, keys) 604 | 605 | Delete multiple keys from a Vedis hash. 606 | 607 | :returns: The number of keys actually deleted. 608 | 609 | Example: 610 | 611 | .. code-block:: pycon 612 | 613 | >>> db.hmdel('my_hash', ['k1', 'k2', 'invalid-key']) 614 | 2 615 | 616 | .. py:method:: Set(key) 617 | 618 | Create a :py:class:`Set` object, which provides a set-like 619 | interface for working with Vedis sets. 620 | 621 | :param str key: The key for the Vedis set object. 622 | :returns: a :py:class:`Set` object representing the Vedis set at the 623 | specified key. 624 | 625 | Example: 626 | 627 | .. code-block:: pycon 628 | 629 | >>> my_set = db.Set('my_set') 630 | >>> my_set.add('v1', 'v2', 'v3') 631 | 3 632 | >>> my_set.to_set() 633 | set(['v1', 'v2', 'v3']) 634 | 635 | .. py:method:: sadd(key, value) 636 | 637 | Add a single value to a Vedis set, returning the number of 638 | items added. 639 | 640 | Example: 641 | 642 | .. code-block:: pycon 643 | 644 | >>> db.sadd('my_set', 'v1') 645 | 1 646 | >>> db.sadd('my_set', 'v2') 647 | 1 648 | >>> db.smembers('my_set') 649 | {'v1', 'v2'} 650 | 651 | .. py:method:: smadd(key, values) 652 | 653 | Add one or more values to a Vedis set, returning the number of 654 | items added. 655 | 656 | Unlike :py:meth:`~Vedis.sadd`, ``smadd`` accepts a list of values to add to the set. 657 | 658 | Example: 659 | 660 | .. code-block:: pycon 661 | 662 | >>> db.smadd('my_set', ['v1', 'v2', 'v3']) 663 | >>> db.smembers('my_set') 664 | {'v1', 'v2', 'v3'} 665 | 666 | .. py:method:: scard(key) 667 | 668 | Return the cardinality, or number of items, in the given set. If 669 | a Vedis set does not exist at the given key, ``0`` will be returned. 670 | 671 | Example: 672 | 673 | .. code-block:: pycon 674 | 675 | >>> db.scard('my_set') 676 | 3 677 | >>> db.scard('does not exist') 678 | 0 679 | 680 | .. py:method:: sismember(key, value) 681 | 682 | Return a boolean indicating whether the provided value is a member 683 | of a Vedis set. If a Vedis set does not exist at the given key, 684 | ``False`` will be returned. 685 | 686 | Example: 687 | 688 | .. code-block:: pycon 689 | 690 | >>> db.sismember('my_set', 'v1') 691 | True 692 | >>> db.sismember('my_set', 'vx') 693 | False 694 | >>> print db.sismember('does not exist', 'xx') 695 | False 696 | 697 | .. py:method:: spop(key) 698 | 699 | Remove and return the last record from a Vedis set. If a Vedis set does 700 | not exist at the given key, or the set is empty, ``None`` will be returned. 701 | 702 | Example: 703 | 704 | .. code-block:: pycon 705 | 706 | >>> db.sadd('my_set', 'v1', 'v2', 'v3') 707 | 3 708 | >>> db.spop('my_set') 709 | 'v3' 710 | 711 | .. py:method:: speek(key) 712 | 713 | Return the last record from a Vedis set without removing it. If a Vedis 714 | set does not exist at the given key, or the set is empty, ``None`` will 715 | be returned. 716 | 717 | Example: 718 | 719 | .. code-block:: pycon 720 | 721 | >>> db.sadd('my_set', 'v1', 'v2', 'v3') 722 | 3 723 | >>> db.speek('my_set') 724 | 'v3' 725 | 726 | .. py:method:: stop(key) 727 | 728 | Return the first record from a Vedis set without removing it. 729 | 730 | Example: 731 | 732 | .. code-block:: pycon 733 | 734 | >>> db.sadd('my_set', 'v1', 'v2', 'v3') 735 | >>> db.stop('my_set') 736 | 'v1' 737 | 738 | .. py:method:: srem(key, value) 739 | 740 | Remove the given value from a Vedis set. 741 | 742 | :returns: The number of items removed. 743 | 744 | Example: 745 | 746 | .. code-block:: pycon 747 | 748 | >>> db.sadd('my_set', 'v1', 'v2', 'v3') 749 | 3 750 | >>> db.srem('my_set', 'v2') 751 | 1 752 | >>> db.srem('my_set', 'v2') 753 | 0 754 | >>> list(db.smembers('my_set')) 755 | ['v1', 'v3'] 756 | 757 | .. py:method:: smrem(key, values) 758 | 759 | Remove one or more values from the Vedis set. 760 | 761 | :returns: The number of items removed. 762 | 763 | Example: 764 | 765 | .. code-block:: pycon 766 | 767 | >>> db.smadd('my_set', ['v1', 'v2', 'v3']) 768 | 3 769 | >>> db.smrem('my_set', ['v1', 'v2', 'xxx']) 770 | >>> db.smembers('my_set') 771 | {'v3'} 772 | 773 | .. py:method:: smembers(key) 774 | 775 | Return all members of a given set. 776 | 777 | :rtype: set 778 | 779 | Example: 780 | 781 | .. code-block:: pycon 782 | 783 | >>> db.smembers('my_set') 784 | {'v1', 'v3'} 785 | 786 | .. py:method:: sdiff(k1, k2) 787 | 788 | Return the set difference of two Vedis sets identified by ``k1`` and ``k2``. 789 | 790 | Example: 791 | 792 | .. code-block:: pycon 793 | 794 | >>> db.sadd('my_set', 'v1', 'v2', 'v3') 795 | 3 796 | >>> db.sadd('other_set', 'v2', 'v3', 'v4') 797 | 3 798 | >>> db.sdiff('my_set', 'other_set') 799 | {'v1'} 800 | 801 | .. py:method:: sinter(k1, k2) 802 | 803 | Return the intersection of two Vedis sets identified by ``k1`` and ``k2``. 804 | 805 | Example: 806 | 807 | .. code-block:: pycon 808 | 809 | >>> db.sadd('my_set', 'v1', 'v2', 'v3') 810 | 3 811 | >>> db.sadd('other_set', 'v2', 'v3', 'v4') 812 | 3 813 | >>> db.sinter('my_set', 'other_set') 814 | {'v3', 'v2'} 815 | 816 | .. py:method:: List(key) 817 | 818 | Create a :py:class:`List` object, which provides a list-like 819 | interface for working with Vedis lists. 820 | 821 | :param str key: The key for the Vedis list object. 822 | :returns: a :py:class:`List` object representing the Vedis list at the 823 | specified key. 824 | 825 | Example: 826 | 827 | .. code-block:: pycon 828 | 829 | >>> my_list = db.List('my_list') 830 | >>> my_list.append('i1') 831 | >>> my_list.extend(['i2', 'i3']) 832 | >>> my_list[0] 833 | 'i1' 834 | >>> my_list.pop() 835 | 'i1' 836 | >>> len(my_list) 837 | 2 838 | 839 | .. py:method:: lpush(key, value) 840 | 841 | Append one value to a Vedis list, returning the number of 842 | items added. 843 | 844 | Example: 845 | 846 | .. code-block:: pycon 847 | 848 | >>> db.lpush('my_list', 'i1') 849 | 1 850 | 851 | .. py:method:: lmpush(key, values) 852 | 853 | Append one or more values to a Vedis list, returning the number of 854 | items added. 855 | 856 | Example: 857 | 858 | .. code-block:: pycon 859 | 860 | >>> db.lmpush('my_list', ['i2', 'i3', 'i4']) 861 | 3 862 | 863 | .. py:method:: lindex(key, idx) 864 | 865 | Returns the element at the given index in the Vedis list. Indices are 866 | zero-based, and negative indices can be used to designate elements 867 | starting from the end of the list. 868 | 869 | Example: 870 | 871 | .. code-block:: pycon 872 | 873 | >>> db.lmpush('my_list', ['i1', 'i2', 'i3']) 874 | >>> db.lindex('my_list', 0) 875 | 'i1' 876 | >>> db.lindex('my_list', -1) 877 | 'i3' 878 | 879 | .. py:method:: llen(key) 880 | 881 | Return the length of a Vedis list. 882 | 883 | Example: 884 | 885 | .. code-block:: pycon 886 | 887 | >>> db.llen('my_list') 888 | 3 889 | >>> db.llen('does not exist') 890 | 0 891 | 892 | .. py:method:: lpop(key) 893 | 894 | Remove and return the first element of a Vedis list. If no elements 895 | exist, ``None`` is returned. 896 | 897 | Example: 898 | 899 | .. code-block:: pycon 900 | 901 | >>> db.lmpush('a list', ['i1', 'i2']) 902 | 2 903 | >>> db.lpop('a list') 904 | 'i1' 905 | 906 | .. py:method:: register(command_name) 907 | 908 | Function decorator used to register user-defined Vedis commands. 909 | User-defined commands must accept a special :py:class:`VedisContext` as their 910 | first parameter, followed by any number of parameters specified when the command was invoked. The following 911 | are valid return types for user-defined commands: 912 | 913 | * lists (arbitrarily nested) 914 | * strings 915 | * boolean values 916 | * integers 917 | * floating point numbers 918 | * ``None`` 919 | 920 | Here is a simple example of a custom command that converts its arguments 921 | to title-case: 922 | 923 | .. code-block:: python 924 | 925 | @db.register('TITLE') 926 | def title_cmd(vedis_ctx, *params): 927 | return [param.title() for param in params] 928 | 929 | Here is how you might call your user-defined function: 930 | 931 | .. code-block:: pycon 932 | 933 | >>> db.execute('TITLE %s %s %s', ('foo', 'this is a test', 'bar')) 934 | ['Foo', 'This Is A Test', 'Bar'] 935 | 936 | You can also call the wrapped function directly, and the call will be routed through Vedis: 937 | 938 | .. code-block:: pycon 939 | 940 | >>> title('foo', 'this is a test', 'bar') 941 | ['Foo', 'This Is A Test', 'Bar'] 942 | 943 | For more information, see the :ref:`custom_commands` section. 944 | 945 | .. py:method:: delete_command(command_name) 946 | 947 | Unregister a custom command. 948 | 949 | .. py:method:: strlen(key) 950 | 951 | Return the length of the value stored at the given key. 952 | 953 | Example: 954 | 955 | .. code-block:: pycon 956 | 957 | >>> db = Vedis() 958 | >>> db['foo'] = 'testing' 959 | >>> db.strlen('foo') 960 | 7 961 | 962 | .. py:method:: copy(src, dest) 963 | 964 | Copy the contents of one key to another, leaving the original intact. 965 | 966 | .. py:method:: move(src, dest) 967 | 968 | Move the contents of one key to another, deleting the original key. 969 | 970 | .. py:method:: strip_tags(html) 971 | 972 | Remove HTML formatting from a given string. 973 | 974 | :param str html: A string containing HTML. 975 | :returns: A string with all HTML removed. 976 | 977 | Example: 978 | 979 | .. code-block:: pycon 980 | 981 | >>> db.strip_tags('

This is a test.

') 982 | 'This is a test.' 983 | 984 | .. py:method:: str_split(s[, nchars=1]) 985 | 986 | Split the given string, ``s``. 987 | 988 | Example: 989 | 990 | .. code-block:: pycon 991 | 992 | >>> db.str_split('abcdefghijklmnop', 5) 993 | ['abcde', 'fghij', 'klmno', 'p'] 994 | 995 | .. py:method:: size_format(nbytes) 996 | 997 | Return a user-friendly representation of a given number of bytes. 998 | 999 | Example: 1000 | 1001 | .. code-block:: pycon 1002 | 1003 | >>> db.size_format(1337) 1004 | '1.3 KB' 1005 | >>> db.size_format(1337000) 1006 | '1.2 MB' 1007 | 1008 | .. py:method:: soundex(s) 1009 | 1010 | Calculate the ``soundex`` value for a given string. 1011 | 1012 | Example: 1013 | 1014 | .. code-block:: pycon 1015 | 1016 | >>> db.soundex('howdy') 1017 | 'H300' 1018 | >>> db.soundex('huey') 1019 | 'H000' 1020 | 1021 | .. py:method:: base64(data) 1022 | 1023 | Encode ``data`` in base64. 1024 | 1025 | Example: 1026 | 1027 | .. code-block:: pycon 1028 | 1029 | >>> db.base64('hello') 1030 | 'aGVsbG8=' 1031 | 1032 | .. py:method:: base64_decode(data) 1033 | 1034 | Decode the base64-encoded ``data``. 1035 | 1036 | Example: 1037 | 1038 | .. code-block:: pycon 1039 | 1040 | >>> db.base64_decode('aGVsbG8=') 1041 | 'hello' 1042 | 1043 | .. py:method:: rand(lower_bound, upper_bound) 1044 | 1045 | Return a random integer within the lower and upper bounds (inclusive). 1046 | 1047 | .. py:method:: randstr(nbytes) 1048 | 1049 | Return a random string of ``nbytes`` length, made up of the characters a-z. 1050 | 1051 | .. py:method:: time() 1052 | 1053 | Return the current GMT time, formatted as HH:MM:SS. 1054 | 1055 | .. py:method:: date() 1056 | 1057 | Return the current date in ISO-8601 format (YYYY-MM-DD). 1058 | 1059 | .. py:method:: operating_system() 1060 | 1061 | Return a brief description of the host operating system. 1062 | 1063 | .. py:method:: table_list() 1064 | 1065 | Return a list of all vedis tables (i.e. Hashes, Sets, List) in memory. 1066 | 1067 | .. py:method:: execute(cmd[, params=None[, result=True]]) 1068 | 1069 | Execute a Vedis command. 1070 | 1071 | :param str cmd: The command to execute. 1072 | :param tuple params: A tuple of parameters to pass into the command. 1073 | :param bool result: Return the result of this command. 1074 | 1075 | Example: 1076 | 1077 | .. code-block:: python 1078 | 1079 | db = Vedis() 1080 | 1081 | # Execute a command, ignoring the result. 1082 | db.execute('HSET %s %s %s', ('hash_key', 'key', 'some value')) 1083 | 1084 | # Execute a command that returns a single result. 1085 | val = db.execute('HGET %s %s', ('hash_key', 'key')) 1086 | 1087 | # Execute a command return returns multiple values. 1088 | keys = db.execute('HKEYS %s', ('hash_key',)) 1089 | for key in keys: 1090 | print 'Hash "hash_key" contains key "%s"' % key 1091 | 1092 | 1093 | Hash objects 1094 | ------------ 1095 | 1096 | .. py:class:: Hash(vedis, key) 1097 | 1098 | Provides a high-level API for working with Vedis hashes. As much as seemed 1099 | sensible, the :py:class:`Hash` acts like a python dictionary. 1100 | 1101 | .. note:: 1102 | This class should not be constructed directly, but through the 1103 | factory method :py:meth:`Vedis.Hash`. 1104 | 1105 | Here is an example of how you might use the various ``Hash`` APIs: 1106 | 1107 | .. code-block:: pycon 1108 | 1109 | >>> h = db.Hash('my_hash') 1110 | 1111 | >>> h['k1'] = 'v1' 1112 | >>> h.update(k2='v2', k3='v3') 1113 | 1114 | >>> len(h) 1115 | 3 1116 | 1117 | >>> 'k1' in h 1118 | True 1119 | >>> 'k4' in h 1120 | False 1121 | 1122 | >>> h.to_dict() 1123 | {'k3': 'v3', 'k2': 'v2', 'k1': 'v1'} 1124 | 1125 | >>> h.keys() 1126 | ['k1', 'k3', 'k2'] 1127 | >>> h.values() 1128 | ['v1', 'v3', 'v2'] 1129 | >>> h.items() 1130 | [('k1', 'v1'), ('k3', 'v3'), ('k2', 'v2')] 1131 | 1132 | >>> del h['k2'] 1133 | >>> h.items() 1134 | [('k1', 'v1'), ('k3', 'v3')] 1135 | 1136 | >>> h.mget('k3', 'kx', 'k1') 1137 | ['v3', None, 'v1'] 1138 | 1139 | >>> h 1140 | 1141 | 1142 | Set objects 1143 | ----------- 1144 | 1145 | .. py:class:: Set(vedis, key) 1146 | 1147 | Provides a high-level API for working with Vedis sets. As much as seemed 1148 | sensible, the :py:class:`Set` acts like a python set. 1149 | 1150 | .. note:: 1151 | This class should not be constructed directly, but through the 1152 | factory method :py:meth:`Vedis.Set`. 1153 | 1154 | Here is an example of how you might use the various ``Set`` APIs: 1155 | 1156 | .. code-block:: pycon 1157 | 1158 | >>> s = db.Set('my_set') 1159 | 1160 | >>> s.add('v1', 'v2', 'v1', 'v3') 1161 | 4 1162 | >>> len(s) 1163 | 3 1164 | 1165 | >>> [item for item in s] 1166 | ['v1', 'v2', 'v3'] 1167 | 1168 | >>> s.top() 1169 | 'v1' 1170 | >>> s.peek() 1171 | 'v3' 1172 | >>> s.pop() 1173 | 'v3' 1174 | 1175 | >>> 'v2' in s 1176 | True 1177 | >>> 'v3' in s 1178 | False 1179 | 1180 | >>> s.add('v3', 'v4') 1181 | 2 1182 | >>> del s['v4'] 1183 | >>> s.to_set() 1184 | {'v1', 'v2', 'v3'} 1185 | 1186 | Vedis also supports set difference and intersection: 1187 | 1188 | .. code-block:: pycon 1189 | 1190 | >>> s2 = db.Set('other_set') 1191 | >>> s2.add('v3', 'v4', 'v5') 1192 | 3 1193 | 1194 | >>> s - s2 1195 | {'v1', 'v2'} 1196 | 1197 | >>> s2 - s 1198 | {'v4', 'v5'} 1199 | 1200 | >>> s & s2 1201 | {'v3'} 1202 | 1203 | List objects 1204 | ------------ 1205 | 1206 | .. py:class:: List(vedis, key) 1207 | 1208 | Provides a high-level API for working with Vedis lists. 1209 | 1210 | .. note:: 1211 | This class should not be constructed directly, but through the 1212 | factory method :py:meth:`Vedis.List`. 1213 | 1214 | Here is an example of how you might use the various ``List`` APIs: 1215 | 1216 | .. code-block:: pycon 1217 | 1218 | >>> l = db.List('my_list') 1219 | 1220 | >>> l.append('v1') 1221 | 1 1222 | >>> l.extend(['v2', 'v3', 'v4']) 1223 | 4 1224 | 1225 | >>> len(l) 1226 | 4 1227 | 1228 | >>> l[0] 1229 | 'v1' 1230 | >>> l[-1] 1231 | 'v4' 1232 | 1233 | >>> [item for item in l] 1234 | ['v1', 'v2', 'v3', 'v4'] 1235 | 1236 | >>> l.pop() 1237 | 'v1' 1238 | 1239 | Vedis Context 1240 | ------------- 1241 | 1242 | When a user-defined command is executed, the first parameter sent to the 1243 | callback is a ``vedis_context`` instance. The ``vedis_context`` allows user-defined 1244 | commands to set return codes (handled automatically by vedis-python), but 1245 | perhaps more interestingly, modify other keys and values in the database. 1246 | 1247 | In this way, your user-defined command can set, get, and delete keys in 1248 | the vedis database. Because the vedis_context APIs are a bit low-level, 1249 | vedis-python wraps the ``vedis_context``, providing a nicer API to work with. 1250 | 1251 | .. py:class:: VedisContext(vedis_context) 1252 | 1253 | This class will almost never be instantiated directly, but will instead 1254 | by created by vedis-python when executing a user-defined callback. 1255 | 1256 | :param vedis_context: A pointer to a ``vedis_context``. 1257 | 1258 | Usage: 1259 | 1260 | .. code-block:: python 1261 | 1262 | @db.register('TITLE_VALUES') 1263 | def title_values(context, *values): 1264 | """ 1265 | Create key/value pairs for each value consisting of the 1266 | original value -> the title-cased version of the value. 1267 | 1268 | Returns the number of values processed. 1269 | """ 1270 | for value in values: 1271 | context[value] = value.title() 1272 | return len(values) 1273 | 1274 | .. code-block:: pycon 1275 | 1276 | >>> db.execute('TITLE_VALUES %s %s', ('val 1', 'another value')) 1277 | 2 1278 | >>> db['val 1'] 1279 | 'Val 1' 1280 | >>> db['another val'] 1281 | 'Another Val' 1282 | 1283 | .. py:method:: fetch(key) 1284 | 1285 | Return the value of the given key. Identical to :py:meth:`Vedis.get`. 1286 | 1287 | Instead of calling ``fetch()`` you can also use a dictionary-style 1288 | lookup on the context: 1289 | 1290 | .. code-block:: python 1291 | 1292 | @db.register('MY_COMMAND') 1293 | def my_command(context, *values): 1294 | some_val = context['the key'] 1295 | # ... 1296 | 1297 | .. py:method:: store(key, value) 1298 | 1299 | Set the value of the given key. Identical to :py:meth:`Vedis.set`. 1300 | 1301 | Instead of calling ``store()`` you can also use a dictionary-style 1302 | assignment on the context: 1303 | 1304 | .. code-block:: python 1305 | 1306 | @db.register('MY_COMMAND') 1307 | def my_command(context, *values): 1308 | context['some key'] = 'some value' 1309 | # ... 1310 | 1311 | .. py:method:: append(key, value) 1312 | 1313 | Append a value to the given key. If the key does not exist, the 1314 | operation is equivalent to :py:meth:`~VedisContext.store`. Identical 1315 | to :py:meth:`Vedis.append`. 1316 | 1317 | .. py:method:: delete(key) 1318 | 1319 | Delete the given key. Identical to :py:meth:`Vedis.delete`. 1320 | 1321 | Instead of calling ``delete()`` you can also use a the python 1322 | ``del`` keyword: 1323 | 1324 | .. code-block:: python 1325 | 1326 | @db.register('MY_COMMAND') 1327 | def my_command(context, *values): 1328 | del context['some key'] 1329 | # ... 1330 | 1331 | .. py:method:: exists(key) 1332 | 1333 | Check for the existence of the given key. Identical to :py:meth:`Vedis.exists`. 1334 | 1335 | Instead of calling ``exists()`` you can also use a the python 1336 | ``in`` keyword: 1337 | 1338 | .. code-block:: python 1339 | 1340 | @db.register('MY_COMMAND') 1341 | def my_command(context, *values): 1342 | if 'some key' in context: 1343 | # ... 1344 | 1345 | Transactions 1346 | ------------ 1347 | 1348 | .. py:class:: Transaction(vedis) 1349 | 1350 | :param Vedis vedis: An :py:class:`Vedis` instance. 1351 | 1352 | Context-manager for executing wrapped blocks in a transaction. Rather than instantiating this object directly, it is recommended that you use :py:meth:`Vedis.transaction`. 1353 | 1354 | Example: 1355 | 1356 | .. code-block:: python 1357 | 1358 | with db.transaction(): 1359 | db['from_acct'] = db['from_acct'] + 100 1360 | db['to_acct'] = db['to_acct'] - 100 1361 | 1362 | To roll back changes inside a transaction, call :py:meth:`Vedis.rollback`: 1363 | 1364 | .. code-block:: python 1365 | 1366 | with db.transaction(): 1367 | db['from_acct'] = db['from_acct'] + 100 1368 | db['to_acct'] = db['to_acct'] - 100 1369 | if int(db['to_acct']) < 0: 1370 | db.rollback() # Not enough funds! 1371 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # vedis-python documentation build configuration file, created by 4 | # sphinx-quickstart on Mon Jun 16 23:34:38 2014. 5 | # 6 | # This file is execfile()d with the current directory set to its 7 | # containing dir. 8 | # 9 | # Note that not all possible configuration values are present in this 10 | # autogenerated file. 11 | # 12 | # All configuration values have a default; values that are commented out 13 | # serve to show the default. 14 | 15 | import sys 16 | import os 17 | 18 | # If extensions (or modules to document with autodoc) are in another directory, 19 | # add these directories to sys.path here. If the directory is relative to the 20 | # documentation root, use os.path.abspath to make it absolute, like shown here. 21 | #sys.path.insert(0, os.path.abspath('.')) 22 | 23 | # -- General configuration ------------------------------------------------ 24 | 25 | # If your documentation needs a minimal Sphinx version, state it here. 26 | #needs_sphinx = '1.0' 27 | 28 | # Add any Sphinx extension module names here, as strings. They can be 29 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 30 | # ones. 31 | extensions = [] 32 | 33 | # Add any paths that contain templates here, relative to this directory. 34 | templates_path = ['_templates'] 35 | 36 | # The suffix of source filenames. 37 | source_suffix = '.rst' 38 | 39 | # The encoding of source files. 40 | #source_encoding = 'utf-8-sig' 41 | 42 | # The master toctree document. 43 | master_doc = 'index' 44 | 45 | # General information about the project. 46 | project = u'vedis-python' 47 | copyright = u'2014, charles leifer' 48 | 49 | # The version info for the project you're documenting, acts as replacement for 50 | # |version| and |release|, also used in various other places throughout the 51 | # built documents. 52 | # 53 | # The short X.Y version. 54 | version = '0.3.1' 55 | # The full version, including alpha/beta/rc tags. 56 | release = '0.3.1' 57 | 58 | # The language for content autogenerated by Sphinx. Refer to documentation 59 | # for a list of supported languages. 60 | #language = None 61 | 62 | # There are two options for replacing |today|: either, you set today to some 63 | # non-false value, then it is used: 64 | #today = '' 65 | # Else, today_fmt is used as the format for a strftime call. 66 | #today_fmt = '%B %d, %Y' 67 | 68 | # List of patterns, relative to source directory, that match files and 69 | # directories to ignore when looking for source files. 70 | exclude_patterns = ['_build'] 71 | 72 | # The reST default role (used for this markup: `text`) to use for all 73 | # documents. 74 | #default_role = None 75 | 76 | # If true, '()' will be appended to :func: etc. cross-reference text. 77 | #add_function_parentheses = True 78 | 79 | # If true, the current module name will be prepended to all description 80 | # unit titles (such as .. function::). 81 | #add_module_names = True 82 | 83 | # If true, sectionauthor and moduleauthor directives will be shown in the 84 | # output. They are ignored by default. 85 | #show_authors = False 86 | 87 | # The name of the Pygments (syntax highlighting) style to use. 88 | pygments_style = 'sphinx' 89 | 90 | # A list of ignored prefixes for module index sorting. 91 | #modindex_common_prefix = [] 92 | 93 | # If true, keep warnings as "system message" paragraphs in the built documents. 94 | #keep_warnings = False 95 | 96 | 97 | # -- Options for HTML output ---------------------------------------------- 98 | 99 | # The theme to use for HTML and HTML Help pages. See the documentation for 100 | # a list of builtin themes. 101 | html_theme = 'default' 102 | 103 | # Theme options are theme-specific and customize the look and feel of a theme 104 | # further. For a list of options available for each theme, see the 105 | # documentation. 106 | #html_theme_options = {} 107 | 108 | # Add any paths that contain custom themes here, relative to this directory. 109 | #html_theme_path = [] 110 | 111 | # The name for this set of Sphinx documents. If None, it defaults to 112 | # " v documentation". 113 | #html_title = None 114 | 115 | # A shorter title for the navigation bar. Default is the same as html_title. 116 | #html_short_title = None 117 | 118 | # The name of an image file (relative to this directory) to place at the top 119 | # of the sidebar. 120 | #html_logo = None 121 | 122 | # The name of an image file (within the static path) to use as favicon of the 123 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 124 | # pixels large. 125 | #html_favicon = None 126 | 127 | # Add any paths that contain custom static files (such as style sheets) here, 128 | # relative to this directory. They are copied after the builtin static files, 129 | # so a file named "default.css" will overwrite the builtin "default.css". 130 | html_static_path = ['_static'] 131 | 132 | # Add any extra paths that contain custom files (such as robots.txt or 133 | # .htaccess) here, relative to this directory. These files are copied 134 | # directly to the root of the documentation. 135 | #html_extra_path = [] 136 | 137 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 138 | # using the given strftime format. 139 | #html_last_updated_fmt = '%b %d, %Y' 140 | 141 | # If true, SmartyPants will be used to convert quotes and dashes to 142 | # typographically correct entities. 143 | #html_use_smartypants = True 144 | 145 | # Custom sidebar templates, maps document names to template names. 146 | #html_sidebars = {} 147 | 148 | # Additional templates that should be rendered to pages, maps page names to 149 | # template names. 150 | #html_additional_pages = {} 151 | 152 | # If false, no module index is generated. 153 | #html_domain_indices = True 154 | 155 | # If false, no index is generated. 156 | #html_use_index = True 157 | 158 | # If true, the index is split into individual pages for each letter. 159 | #html_split_index = False 160 | 161 | # If true, links to the reST sources are added to the pages. 162 | #html_show_sourcelink = True 163 | 164 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 165 | #html_show_sphinx = True 166 | 167 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 168 | #html_show_copyright = True 169 | 170 | # If true, an OpenSearch description file will be output, and all pages will 171 | # contain a tag referring to it. The value of this option must be the 172 | # base URL from which the finished HTML is served. 173 | #html_use_opensearch = '' 174 | 175 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 176 | #html_file_suffix = None 177 | 178 | # Output file base name for HTML help builder. 179 | htmlhelp_basename = 'vedis-pythondoc' 180 | 181 | 182 | # -- Options for LaTeX output --------------------------------------------- 183 | 184 | latex_elements = { 185 | # The paper size ('letterpaper' or 'a4paper'). 186 | #'papersize': 'letterpaper', 187 | 188 | # The font size ('10pt', '11pt' or '12pt'). 189 | #'pointsize': '10pt', 190 | 191 | # Additional stuff for the LaTeX preamble. 192 | #'preamble': '', 193 | } 194 | 195 | # Grouping the document tree into LaTeX files. List of tuples 196 | # (source start file, target name, title, 197 | # author, documentclass [howto, manual, or own class]). 198 | latex_documents = [ 199 | ('index', 'vedis-python.tex', u'vedis-python Documentation', 200 | u'charles leifer', 'manual'), 201 | ] 202 | 203 | # The name of an image file (relative to this directory) to place at the top of 204 | # the title page. 205 | #latex_logo = None 206 | 207 | # For "manual" documents, if this is true, then toplevel headings are parts, 208 | # not chapters. 209 | #latex_use_parts = False 210 | 211 | # If true, show page references after internal links. 212 | #latex_show_pagerefs = False 213 | 214 | # If true, show URL addresses after external links. 215 | #latex_show_urls = False 216 | 217 | # Documents to append as an appendix to all manuals. 218 | #latex_appendices = [] 219 | 220 | # If false, no module index is generated. 221 | #latex_domain_indices = True 222 | 223 | 224 | # -- Options for manual page output --------------------------------------- 225 | 226 | # One entry per manual page. List of tuples 227 | # (source start file, name, description, authors, manual section). 228 | man_pages = [ 229 | ('index', 'vedis-python', u'vedis-python Documentation', 230 | [u'charles leifer'], 1) 231 | ] 232 | 233 | # If true, show URL addresses after external links. 234 | #man_show_urls = False 235 | 236 | 237 | # -- Options for Texinfo output ------------------------------------------- 238 | 239 | # Grouping the document tree into Texinfo files. List of tuples 240 | # (source start file, target name, title, author, 241 | # dir menu entry, description, category) 242 | texinfo_documents = [ 243 | ('index', 'vedis-python', u'vedis-python Documentation', 244 | u'charles leifer', 'vedis-python', 'One line description of project.', 245 | 'Miscellaneous'), 246 | ] 247 | 248 | # Documents to append as an appendix to all manuals. 249 | #texinfo_appendices = [] 250 | 251 | # If false, no module index is generated. 252 | #texinfo_domain_indices = True 253 | 254 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 255 | #texinfo_show_urls = 'footnote' 256 | 257 | # If true, do not generate a @detailmenu in the "Top" node's menu. 258 | #texinfo_no_detailmenu = False 259 | -------------------------------------------------------------------------------- /docs/custom_commands.rst: -------------------------------------------------------------------------------- 1 | .. custom_commands: 2 | 3 | Creating Your Own Commands 4 | ========================== 5 | 6 | It is possible to create your own Vedis commands and execute them like any other. Use the :py:meth:`Vedis.register` method to decorate the function you wish to turn into a Vedis command. Your command callback must accept at least one argument, the :py:class:`VedisContext` (which wraps `vedis context `_). Any arguments supplied by the caller will also be passed to your callback. Using the :py:class:`VedisContext` object, your function can perform key/value operations on the database. 7 | 8 | Here is a small example: 9 | 10 | .. code-block:: python 11 | 12 | db = Vedis() 13 | 14 | @db.register('CONCAT') 15 | def concat(context, glue, *params): 16 | return glue.join(params) 17 | 18 | @db.register('TITLE') 19 | def title(context, *params): 20 | return [param.title() for param in params] 21 | 22 | @db.register('HASH_VALUES') 23 | def hash_values(context, *values): 24 | # Calculate a hash for each value and store it in a 25 | # key. 26 | for value in values: 27 | context[value] = hashlib.sha1(value).hexdigest() 28 | return len(values) 29 | 30 | Usage: 31 | 32 | .. code-block:: pycon 33 | 34 | >>> print db.execute('CONCAT | foo bar baz') 35 | foo|bar|baz 36 | 37 | >>> print db.execute('TITLE "testing" "this is a test" "another"') 38 | ['Testing', 'This Is A Test', 'Another'] 39 | 40 | >>> print db.execute('HASH_VALUES shh secret') 41 | 2 42 | >>> db.mget(['shh', 'secret']) 43 | ['0c731a5f1dc781894b434c27b9f6a9cd9d9bdfcb', 44 | 'e5e9fa1ba31ecd1ae84f75caaa474f3a663f05f4'] 45 | 46 | You can also directly call the function with your arguments, and the call will automatically be routed through Vedis: 47 | 48 | .. code-block:: pycon 49 | 50 | >>> print title('testing', 'this is a test', 'another') 51 | ['Testing', 'This Is A Test', 'Another'] 52 | 53 | >>> print concat('#', 'foo', '1', 'hello') 54 | 'foo#1#hello' 55 | 56 | Valid return types for user-defined commands 57 | -------------------------------------------- 58 | 59 | * ``list`` or ``tuple`` (containing arbitrary levels of nesting). 60 | * ``str`` 61 | * ``int`` and ``long`` 62 | * ``float`` 63 | * ``bool`` 64 | * ``None`` 65 | 66 | Operations supported by VedisContext 67 | ------------------------------------ 68 | 69 | The first parameter of your custom command is always a :py:class:`VedisContext` instance. This object can be used to access the key/value features of the database. It supports the following APIs: 70 | 71 | * Getting, setting and deleting items using ``dict`` APIs. 72 | * Checking whether a key exists using ``in``. 73 | * Appending to an existing key. 74 | 75 | Example: 76 | 77 | .. code-block:: python 78 | 79 | @db.register('STORE_DATA') 80 | def store_data(context): 81 | context['foo'] = 'bar' 82 | assert context['foo'] == 'bar' 83 | del context['other key'] 84 | assert 'foo' in context 85 | context.append('foo', 'more data') 86 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. vedis-python documentation master file, created by 2 | sphinx-quickstart on Mon Jun 16 23:34:38 2014. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | vedis-python 7 | ============ 8 | 9 | .. image:: http://media.charlesleifer.com/blog/photos/vedis-python-logo.png 10 | 11 | Fast Python bindings for `Vedis `_, an embedded, NoSQL key/value and data-structure store modeled after `Redis `_. 12 | 13 | The source code for vedis-python is `hosted on GitHub `_. 14 | 15 | Vedis features: 16 | 17 | * Embedded, zero-conf database 18 | * Transactional (ACID) 19 | * Single file or in-memory database 20 | * Key/value store 21 | * `Over 70 commands `_ similar to standard `Redis `_ commands. 22 | * Thread-safe 23 | * Terabyte-sized databases 24 | 25 | Vedis-Python features: 26 | 27 | * Compiled library, extremely fast with minimal overhead. 28 | * Supports key/value operations and transactions using Pythonic APIs. 29 | * Support for executing Vedis commands. 30 | * Write custom commands in Python. 31 | * Python 2.x and 3.x. 32 | 33 | Limitations: 34 | 35 | * Not tested on Windoze. 36 | 37 | The previous version (0.2.0) of ``vedis-python`` utilized ``ctypes`` to wrap the Vedis C library. By switching to Cython, key/value and Vedis command operations are significantly faster. 38 | 39 | .. note:: 40 | If you encounter any bugs in the library, please `open an issue `_, including a description of the bug and any related traceback. 41 | 42 | .. note:: 43 | If you like Vedis you might also want to check out `UnQLite `_, an embedded key/value database and JSON document store (python bindings: `unqlite-python `_. 44 | 45 | Contents: 46 | 47 | .. toctree:: 48 | :maxdepth: 2 49 | :glob: 50 | 51 | installation 52 | quickstart 53 | api 54 | custom_commands 55 | 56 | 57 | Indices and tables 58 | ================== 59 | 60 | * :ref:`genindex` 61 | * :ref:`modindex` 62 | * :ref:`search` 63 | 64 | -------------------------------------------------------------------------------- /docs/installation.rst: -------------------------------------------------------------------------------- 1 | .. _installation: 2 | 3 | Installation 4 | ============ 5 | 6 | You can use ``pip`` to install ``vedis-python``: 7 | 8 | .. code-block:: console 9 | 10 | pip install cython vedis 11 | 12 | The project is hosted at https://github.com/coleifer/vedis-python and can be installed from source: 13 | 14 | .. code-block:: console 15 | 16 | git clone https://github.com/coleifer/vedis-python 17 | cd vedis-python 18 | python setup.py build 19 | python setup.py install 20 | 21 | .. note:: 22 | ``vedis-python`` depends on Cython to generate the Python extension. By default vedis-python no longer ships with a generated C source file, so it is necessary to install Cython in order to compile ``vedis-python``. 23 | 24 | After installing vedis-python, you can run the unit tests by executing the ``tests`` module: 25 | 26 | .. code-block:: console 27 | 28 | python tests.py 29 | -------------------------------------------------------------------------------- /docs/quickstart.rst: -------------------------------------------------------------------------------- 1 | .. quickstart: 2 | 3 | Quickstart 4 | ========== 5 | 6 | Below is a sample interactive console session designed to show some of the basic features and functionality of the vedis-python library. Also check out the :ref:`full API docs `. 7 | 8 | Key/value features 9 | ------------------ 10 | 11 | You can use Vedis like a dictionary for simple key/value lookups: 12 | 13 | .. code-block:: pycon 14 | 15 | >>> from vedis import Vedis 16 | >>> db = Vedis(':mem:') # Create an in-memory database. Alternatively you could supply a filename for an on-disk database. 17 | >>> db['k1'] = 'v1' 18 | >>> db['k1'] 19 | 'v1' 20 | 21 | >>> db.append('k1', 'more data') # Returns length of value after appending new data. 22 | 11 23 | >>> db['k1'] 24 | 'v1more data' 25 | 26 | >>> del db['k1'] 27 | >>> db['k1'] is None 28 | True 29 | 30 | You can set and get multiple items at a time: 31 | 32 | .. code-block:: pycon 33 | 34 | >>> db.mset(dict(k1='v1', k2='v2', k3='v3')) 35 | True 36 | 37 | >>> db.mget(['k1', 'k2', 'missing key', 'k3']) 38 | ['v1', 'v2', None, 'v3'] 39 | 40 | In addition to storing string keys/values, you can also implement counters: 41 | 42 | .. code-block:: pycon 43 | 44 | >>> db.incr('counter') 45 | 1 46 | 47 | >>> db.incr('counter') 48 | 2 49 | 50 | >>> db.incr_by('counter', 10) 51 | 12 52 | 53 | >>> db.decr('counter') 54 | 11 55 | 56 | Transactions 57 | ------------ 58 | 59 | Vedis has support for transactions when you are using an on-disk database. You can use the :py:meth:`~Vedis.transaction` context manager or explicitly call :py:meth:`~Vedis.begin`, :py:meth:`~Vedis.commit` and :py:meth:`~Vedis.rollback`. 60 | 61 | .. code-block:: pycon 62 | 63 | >>> db = Vedis('/tmp/test.db') 64 | >>> with db.transaction(): 65 | ... db['k1'] = 'v1' 66 | ... db['k2'] = 'v2' 67 | ... 68 | >>> db['k1'] 69 | 'v1' 70 | 71 | >>> with db.transaction(): 72 | ... db['k1'] = 'modified' 73 | ... db.rollback() # Undo changes. 74 | ... 75 | >>> db['k1'] # Value is not modified. 76 | 'v1' 77 | 78 | >>> db.begin() 79 | >>> db['k3'] = 'v3-xx' 80 | >>> db.commit() 81 | True 82 | >>> db['k3'] 83 | 'v3-xx' 84 | 85 | Hashes 86 | ------ 87 | 88 | Vedis supports nested key/value lookups which have the additional benefit of supporting operations to retrieve all keys, values, the number of items in the hash, and so on. 89 | 90 | .. code-block:: pycon 91 | 92 | >>> h = db.Hash('some key') 93 | >>> h['k1'] = 'v1' 94 | >>> h.update(k2='v2', k3='v3') 95 | 96 | >>> h 97 | 98 | 99 | >>> h.to_dict() 100 | {'k3': 'v3', 'k2': 'v2', 'k1': 'v1'} 101 | 102 | >>> h.items() 103 | [('k1', 'v1'), ('k3', 'v3'), ('k2', 'v2')] 104 | 105 | >>> list(h.keys()) 106 | ['k1', 'k3', 'k2'] 107 | 108 | >>> del h['k2'] 109 | 110 | >>> len(h) 111 | 2 112 | 113 | >>> 'k1' in h 114 | True 115 | 116 | >>> [key for key in h] 117 | ['k1', 'k3'] 118 | 119 | Sets 120 | ---- 121 | 122 | Vedis supports a set data-type which stores a unique collection of items. 123 | 124 | .. code-block:: pycon 125 | 126 | >>> s = db.Set('some set') 127 | >>> s.add('v1', 'v2', 'v3') 128 | 3 129 | 130 | >>> len(s) 131 | 3 132 | 133 | >>> 'v1' in s, 'v4' in s 134 | (True, False) 135 | 136 | >>> s.top() 137 | 'v1' 138 | 139 | >>> s.peek() 140 | 'v3' 141 | 142 | >>> s.remove('v2') 143 | 1 144 | 145 | >>> s.add('v4', 'v5') 146 | 2 147 | 148 | >>> s.pop() 149 | 'v5' 150 | 151 | >>> [item for item in s] 152 | ['v1', 'v3', 'v4'] 153 | 154 | >>> s.to_set() 155 | set(['v1', 'v3', 'v4']) 156 | 157 | >>> s2 = db.Set('another set') 158 | >>> s2.add('v1', 'v4', 'v5', 'v6') 159 | 4 160 | 161 | >>> s2 & s # Intersection. 162 | set(['v1', 'v4']) 163 | 164 | >>> s2 - s # Difference. 165 | set(['v5', 'v6']) 166 | 167 | 168 | Lists 169 | ----- 170 | 171 | Vedis also supports a list data type. 172 | 173 | .. code-block:: pycon 174 | 175 | >>> l = db.List('my list') 176 | >>> l.append('v1') 177 | 1 178 | >>> l.extend(['v2', 'v3', 'v1']) 179 | 4 180 | 181 | >>> for item in l: 182 | ... print item 183 | ... 184 | v1 185 | v2 186 | v3 187 | v4 188 | 189 | >>> for item in l[1:3]: 190 | ... print item 191 | v2 192 | v3 193 | 194 | >>> len(l) 195 | 4 196 | 197 | >>> l[1] 198 | 'v2' 199 | 200 | >>> db.llen('my_list') 201 | 2 202 | 203 | >>> l.pop(), l.pop() 204 | ('v1', 'v2') 205 | 206 | >>> len(l) 207 | 2 208 | 209 | Misc 210 | ---- 211 | 212 | Vedis has a somewhat quirky collection of other miscellaneous commands. Below is a sampling: 213 | 214 | .. code-block:: pycon 215 | 216 | >>> db.base64('encode me') 217 | 'ZW5jb2RlIG1l' 218 | 219 | >>> db.base64_decode('ZW5jb2RlIG1l') 220 | 'encode me' 221 | 222 | >>> db.random_string(10) 223 | 'raurquvsnx' 224 | 225 | >>> db.rand(1, 6) 226 | 4 227 | 228 | >>> db.str_split('abcdefghijklmnop', 5) 229 | ['abcde', 'fghij', 'klmno', 'p'] 230 | 231 | >>> db['data'] = 'abcdefghijklmnop' 232 | >>> db.strlen('data') 233 | 16 234 | 235 | >>> db.strip_tags('

This is a test.

') 236 | 'This is a test.' 237 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["setuptools", "wheel", "cython"] 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import os 2 | import warnings 3 | 4 | from setuptools import setup 5 | from setuptools.extension import Extension 6 | try: 7 | from Cython.Build import cythonize 8 | except ImportError: 9 | cython_installed = False 10 | warnings.warn('Cython not installed, using pre-generated C source file.') 11 | else: 12 | cython_installed = True 13 | 14 | 15 | if cython_installed: 16 | python_source = 'vedis.pyx' 17 | else: 18 | python_source = 'vedis.c' 19 | cythonize = lambda obj: obj 20 | 21 | library_source = 'src/vedis.c' 22 | vedis_extension = Extension( 23 | 'vedis', 24 | sources=[python_source, library_source]) 25 | 26 | setup( 27 | name='vedis', 28 | version='0.7.2', 29 | description='Fast Python bindings for the Vedis embedded NoSQL database.', 30 | author='Charles Leifer', 31 | author_email='', 32 | setup_requires=['cython'], 33 | install_requires=['cython'], 34 | ext_modules=cythonize([vedis_extension]), 35 | ) 36 | -------------------------------------------------------------------------------- /src/vedis.h: -------------------------------------------------------------------------------- 1 | /* This file was automatically generated. Do not edit (Except for compile time directives)! */ 2 | #ifndef _VEDIS_H_ 3 | #define _VEDIS_H_ 4 | /* 5 | * Symisc Vedis: A Highly Efficient Embeddable Data Store Engine. 6 | * Copyright (C) 2013 - 2018, Symisc Systems http://vedis.symisc.net/ 7 | * Version 1.2.6 8 | * For information on licensing, redistribution of this file, and for a DISCLAIMER OF ALL WARRANTIES 9 | * please contact Symisc Systems via: 10 | * legal@symisc.net 11 | * licensing@symisc.net 12 | * contact@symisc.net 13 | * or visit: 14 | * http://vedis.symisc.net/ 15 | */ 16 | /* 17 | * Copyright (C) 2013 - 2018 Symisc Systems, S.U.A.R.L [M.I.A.G Mrad Chems Eddine ]. 18 | * All rights reserved. 19 | * 20 | * Redistribution and use in source and binary forms, with or without 21 | * modification, are permitted provided that the following conditions 22 | * are met: 23 | * 1. Redistributions of source code must retain the above copyright 24 | * notice, this list of conditions and the following disclaimer. 25 | * 2. Redistributions in binary form must reproduce the above copyright 26 | * notice, this list of conditions and the following disclaimer in the 27 | * documentation and/or other materials provided with the distribution. 28 | * 3. Redistributions in any form must be accompanied by information on 29 | * how to obtain complete source code for the Vedis engine and any 30 | * accompanying software that uses the Vedis engine software. 31 | * The source code must either be included in the distribution 32 | * or be available for no more than the cost of distribution plus 33 | * a nominal fee, and must be freely redistributable under reasonable 34 | * conditions. For an executable file, complete source code means 35 | * the source code for all modules it contains.It does not include 36 | * source code for modules or files that typically accompany the major 37 | * components of the operating system on which the executable file runs. 38 | * 39 | * THIS SOFTWARE IS PROVIDED BY SYMISC SYSTEMS ``AS IS'' AND ANY EXPRESS 40 | * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 41 | * WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR 42 | * NON-INFRINGEMENT, ARE DISCLAIMED. IN NO EVENT SHALL SYMISC SYSTEMS 43 | * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 44 | * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 45 | * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR 46 | * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, 47 | * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE 48 | * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN 49 | * IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 50 | */ 51 | /* Make sure we can call this stuff from C++ */ 52 | #ifdef __cplusplus 53 | extern "C" { 54 | #endif 55 | /* $SymiscID: vedis.h v1.2 Unix 2013-09-16 00:38 stable $ */ 56 | #include /* needed for the definition of va_list */ 57 | /* 58 | * Compile time engine version, signature, identification in the symisc source tree 59 | * and copyright notice. 60 | * Each macro have an equivalent C interface associated with it that provide the same 61 | * information but are associated with the library instead of the header file. 62 | * Refer to [vedis_lib_version()], [vedis_lib_signature()], [vedis_lib_ident()] and 63 | * [vedis_lib_copyright()] for more information. 64 | */ 65 | /* 66 | * The VEDIS_VERSION C preprocessor macroevaluates to a string literal 67 | * that is the vedis version in the format "X.Y.Z" where X is the major 68 | * version number and Y is the minor version number and Z is the release 69 | * number. 70 | */ 71 | #define VEDIS_VERSION "1.2.6" 72 | /* 73 | * The VEDIS_VERSION_NUMBER C preprocessor macro resolves to an integer 74 | * with the value (X*1000000 + Y*1000 + Z) where X, Y, and Z are the same 75 | * numbers used in [VEDIS_VERSION]. 76 | */ 77 | #define VEDIS_VERSION_NUMBER 1002006 78 | /* 79 | * The VEDIS_SIG C preprocessor macro evaluates to a string 80 | * literal which is the public signature of the vedis engine. 81 | */ 82 | #define VEDIS_SIG "vedis/1.2.6" 83 | /* 84 | * Vedis identification in the Symisc source tree: 85 | * Each particular check-in of a particular software released 86 | * by symisc systems have an unique identifier associated with it. 87 | * This macro hold the one associated with vedis. 88 | */ 89 | #define VEDIS_IDENT "vedis:e361b2f3d4a71ac17e9f2ac1876232a13467dea1" 90 | /* 91 | * Copyright notice. 92 | * If you have any questions about the licensing situation, please 93 | * visit http://vedis.symisc.net/licensing.html 94 | * or contact Symisc Systems via: 95 | * legal@symisc.net 96 | * licensing@symisc.net 97 | * contact@symisc.net 98 | */ 99 | #define VEDIS_COPYRIGHT "Copyright (C) Symisc Systems, S.U.A.R.L [Mrad Chems Eddine ] 2013, http://vedis.symisc.net/" 100 | /* Forward declaration to public objects */ 101 | typedef struct vedis_io_methods vedis_io_methods; 102 | typedef struct vedis_kv_methods vedis_kv_methods; 103 | typedef struct vedis_kv_engine vedis_kv_engine; 104 | typedef struct vedis_context vedis_context; 105 | typedef struct vedis_value vedis_value; 106 | typedef struct vedis_vfs vedis_vfs; 107 | typedef struct vedis vedis; 108 | /* 109 | * ------------------------------ 110 | * Compile time directives 111 | * ------------------------------ 112 | * For most purposes, Vedis can be built just fine using the default compilation options. 113 | * However, if required, the compile-time options documented below can be used to omit Vedis 114 | * features (resulting in a smaller compiled library size) or to change the default values 115 | * of some parameters. 116 | * Every effort has been made to ensure that the various combinations of compilation options 117 | * work harmoniously and produce a working library. 118 | * 119 | * VEDIS_ENABLE_THREADS 120 | * This option controls whether or not code is included in Vedis to enable it to operate 121 | * safely in a multithreaded environment. The default is not. All mutexing code is omitted 122 | * and it is unsafe to use Vedis in a multithreaded program. When compiled with the 123 | * VEDIS_ENABLE_THREADS directive enabled, Vedis can be used in a multithreaded program 124 | * and it is safe to share the same virtual machine and engine handle between two or more threads. 125 | * The value of VEDIS_ENABLE_THREADS can be determined at run-time using the vedis_lib_is_threadsafe() 126 | * interface. 127 | * When Vedis has been compiled with threading support then the threading mode can be altered 128 | * at run-time using the vedis_lib_config() interface together with one of these verbs: 129 | * VEDIS_LIB_CONFIG_THREAD_LEVEL_SINGLE 130 | * VEDIS_LIB_CONFIG_THREAD_LEVEL_MULTI 131 | * Platforms others than Windows and UNIX systems must install their own mutex subsystem via 132 | * vedis_lib_config() with a configuration verb set to VEDIS_LIB_CONFIG_USER_MUTEX. 133 | * Otherwise the library is not threadsafe. 134 | * Note that you must link Vedis with the POSIX threads library under UNIX systems (i.e: -lpthread). 135 | * 136 | */ 137 | /* Symisc public definitions */ 138 | #if !defined(SYMISC_STANDARD_DEFS) 139 | #define SYMISC_STANDARD_DEFS 140 | #if defined (_WIN32) || defined (WIN32) || defined(__MINGW32__) || defined (_MSC_VER) || defined (_WIN32_WCE) 141 | /* Windows Systems */ 142 | #if !defined(__WINNT__) 143 | #define __WINNT__ 144 | #endif 145 | /* 146 | * Determine if we are dealing with WindowsCE - which has a much 147 | * reduced API. 148 | */ 149 | #if defined(_WIN32_WCE) 150 | #ifndef __WIN_CE__ 151 | #define __WIN_CE__ 152 | #endif /* __WIN_CE__ */ 153 | #endif /* _WIN32_WCE */ 154 | #else 155 | /* 156 | * By default we will assume that we are compiling on a UNIX systems. 157 | * Otherwise the OS_OTHER directive must be defined. 158 | */ 159 | #if !defined(OS_OTHER) 160 | #if !defined(__UNIXES__) 161 | #define __UNIXES__ 162 | #endif /* __UNIXES__ */ 163 | #else 164 | #endif /* OS_OTHER */ 165 | #endif /* __WINNT__/__UNIXES__ */ 166 | #if defined(_MSC_VER) || defined(__BORLANDC__) 167 | typedef signed __int64 sxi64; /* 64 bits(8 bytes) signed int64 */ 168 | typedef unsigned __int64 sxu64; /* 64 bits(8 bytes) unsigned int64 */ 169 | #else 170 | typedef signed long long int sxi64; /* 64 bits(8 bytes) signed int64 */ 171 | typedef unsigned long long int sxu64; /* 64 bits(8 bytes) unsigned int64 */ 172 | #endif /* _MSC_VER */ 173 | /* Signature of the consumer routine */ 174 | typedef int (*ProcConsumer)(const void *, unsigned int, void *); 175 | /* Forward reference */ 176 | typedef struct SyMutexMethods SyMutexMethods; 177 | typedef struct SyMemMethods SyMemMethods; 178 | typedef struct SyString SyString; 179 | typedef struct syiovec syiovec; 180 | typedef struct SyMutex SyMutex; 181 | typedef struct Sytm Sytm; 182 | /* Scatter and gather array. */ 183 | struct syiovec 184 | { 185 | #if defined (__WINNT__) 186 | /* Same fields type and offset as WSABUF structure defined one winsock2 header */ 187 | unsigned long nLen; 188 | char *pBase; 189 | #else 190 | void *pBase; 191 | unsigned long nLen; 192 | #endif 193 | }; 194 | struct SyString 195 | { 196 | const char *zString; /* Raw string (may not be null terminated) */ 197 | unsigned int nByte; /* Raw string length */ 198 | }; 199 | /* Time structure. */ 200 | struct Sytm 201 | { 202 | int tm_sec; /* seconds (0 - 60) */ 203 | int tm_min; /* minutes (0 - 59) */ 204 | int tm_hour; /* hours (0 - 23) */ 205 | int tm_mday; /* day of month (1 - 31) */ 206 | int tm_mon; /* month of year (0 - 11) */ 207 | int tm_year; /* year + 1900 */ 208 | int tm_wday; /* day of week (Sunday = 0) */ 209 | int tm_yday; /* day of year (0 - 365) */ 210 | int tm_isdst; /* is summer time in effect? */ 211 | char *tm_zone; /* abbreviation of timezone name */ 212 | long tm_gmtoff; /* offset from UTC in seconds */ 213 | }; 214 | /* Convert a tm structure (struct tm *) found in to a Sytm structure */ 215 | #define STRUCT_TM_TO_SYTM(pTM, pSYTM) \ 216 | (pSYTM)->tm_hour = (pTM)->tm_hour;\ 217 | (pSYTM)->tm_min = (pTM)->tm_min;\ 218 | (pSYTM)->tm_sec = (pTM)->tm_sec;\ 219 | (pSYTM)->tm_mon = (pTM)->tm_mon;\ 220 | (pSYTM)->tm_mday = (pTM)->tm_mday;\ 221 | (pSYTM)->tm_year = (pTM)->tm_year + 1900;\ 222 | (pSYTM)->tm_yday = (pTM)->tm_yday;\ 223 | (pSYTM)->tm_wday = (pTM)->tm_wday;\ 224 | (pSYTM)->tm_isdst = (pTM)->tm_isdst;\ 225 | (pSYTM)->tm_gmtoff = 0;\ 226 | (pSYTM)->tm_zone = 0; 227 | 228 | /* Convert a SYSTEMTIME structure (LPSYSTEMTIME: Windows Systems only ) to a Sytm structure */ 229 | #define SYSTEMTIME_TO_SYTM(pSYSTIME, pSYTM) \ 230 | (pSYTM)->tm_hour = (pSYSTIME)->wHour;\ 231 | (pSYTM)->tm_min = (pSYSTIME)->wMinute;\ 232 | (pSYTM)->tm_sec = (pSYSTIME)->wSecond;\ 233 | (pSYTM)->tm_mon = (pSYSTIME)->wMonth - 1;\ 234 | (pSYTM)->tm_mday = (pSYSTIME)->wDay;\ 235 | (pSYTM)->tm_year = (pSYSTIME)->wYear;\ 236 | (pSYTM)->tm_yday = 0;\ 237 | (pSYTM)->tm_wday = (pSYSTIME)->wDayOfWeek;\ 238 | (pSYTM)->tm_gmtoff = 0;\ 239 | (pSYTM)->tm_isdst = -1;\ 240 | (pSYTM)->tm_zone = 0; 241 | 242 | /* Dynamic memory allocation methods. */ 243 | struct SyMemMethods 244 | { 245 | void * (*xAlloc)(unsigned int); /* [Required:] Allocate a memory chunk */ 246 | void * (*xRealloc)(void *, unsigned int); /* [Required:] Re-allocate a memory chunk */ 247 | void (*xFree)(void *); /* [Required:] Release a memory chunk */ 248 | unsigned int (*xChunkSize)(void *); /* [Optional:] Return chunk size */ 249 | int (*xInit)(void *); /* [Optional:] Initialization callback */ 250 | void (*xRelease)(void *); /* [Optional:] Release callback */ 251 | void *pUserData; /* [Optional:] First argument to xInit() and xRelease() */ 252 | }; 253 | /* Out of memory callback signature. */ 254 | typedef int (*ProcMemError)(void *); 255 | /* Mutex methods. */ 256 | struct SyMutexMethods 257 | { 258 | int (*xGlobalInit)(void); /* [Optional:] Global mutex initialization */ 259 | void (*xGlobalRelease)(void); /* [Optional:] Global Release callback () */ 260 | SyMutex * (*xNew)(int); /* [Required:] Request a new mutex */ 261 | void (*xRelease)(SyMutex *); /* [Optional:] Release a mutex */ 262 | void (*xEnter)(SyMutex *); /* [Required:] Enter mutex */ 263 | int (*xTryEnter)(SyMutex *); /* [Optional:] Try to enter a mutex */ 264 | void (*xLeave)(SyMutex *); /* [Required:] Leave a locked mutex */ 265 | }; 266 | #if defined (_MSC_VER) || defined (__MINGW32__) || defined (__GNUC__) && defined (__declspec) 267 | #define SX_APIIMPORT __declspec(dllimport) 268 | #define SX_APIEXPORT __declspec(dllexport) 269 | #else 270 | #define SX_APIIMPORT 271 | #define SX_APIEXPORT 272 | #endif 273 | /* Standard return values from Symisc public interfaces */ 274 | #define SXRET_OK 0 /* Not an error */ 275 | #define SXERR_MEM (-1) /* Out of memory */ 276 | #define SXERR_IO (-2) /* IO error */ 277 | #define SXERR_EMPTY (-3) /* Empty field */ 278 | #define SXERR_LOCKED (-4) /* Locked operation */ 279 | #define SXERR_ORANGE (-5) /* Out of range value */ 280 | #define SXERR_NOTFOUND (-6) /* Item not found */ 281 | #define SXERR_LIMIT (-7) /* Limit reached */ 282 | #define SXERR_MORE (-8) /* Need more input */ 283 | #define SXERR_INVALID (-9) /* Invalid parameter */ 284 | #define SXERR_ABORT (-10) /* User callback request an operation abort */ 285 | #define SXERR_EXISTS (-11) /* Item exists */ 286 | #define SXERR_SYNTAX (-12) /* Syntax error */ 287 | #define SXERR_UNKNOWN (-13) /* Unknown error */ 288 | #define SXERR_BUSY (-14) /* Busy operation */ 289 | #define SXERR_OVERFLOW (-15) /* Stack or buffer overflow */ 290 | #define SXERR_WILLBLOCK (-16) /* Operation will block */ 291 | #define SXERR_NOTIMPLEMENTED (-17) /* Operation not implemented */ 292 | #define SXERR_EOF (-18) /* End of input */ 293 | #define SXERR_PERM (-19) /* Permission error */ 294 | #define SXERR_NOOP (-20) /* No-op */ 295 | #define SXERR_FORMAT (-21) /* Invalid format */ 296 | #define SXERR_NEXT (-22) /* Not an error */ 297 | #define SXERR_OS (-23) /* System call return an error */ 298 | #define SXERR_CORRUPT (-24) /* Corrupted pointer */ 299 | #define SXERR_CONTINUE (-25) /* Not an error: Operation in progress */ 300 | #define SXERR_NOMATCH (-26) /* No match */ 301 | #define SXERR_RESET (-27) /* Operation reset */ 302 | #define SXERR_DONE (-28) /* Not an error */ 303 | #define SXERR_SHORT (-29) /* Buffer too short */ 304 | #define SXERR_PATH (-30) /* Path error */ 305 | #define SXERR_TIMEOUT (-31) /* Timeout */ 306 | #define SXERR_BIG (-32) /* Too big for processing */ 307 | #define SXERR_RETRY (-33) /* Retry your call */ 308 | #define SXERR_IGNORE (-63) /* Ignore */ 309 | #endif /* SYMISC_PUBLIC_DEFS */ 310 | /* 311 | * Marker for exported interfaces. 312 | */ 313 | #define VEDIS_APIEXPORT SX_APIEXPORT 314 | /* Standard Vedis return values */ 315 | #define VEDIS_OK SXRET_OK /* Successful result */ 316 | /* Beginning of error codes */ 317 | #define VEDIS_NOMEM SXERR_MEM /* Out of memory */ 318 | #define VEDIS_ABORT SXERR_ABORT /* Another thread have released this instance */ 319 | #define VEDIS_IOERR SXERR_IO /* IO error */ 320 | #define VEDIS_CORRUPT SXERR_CORRUPT /* Corrupt pointer */ 321 | #define VEDIS_LOCKED SXERR_LOCKED /* Forbidden Operation */ 322 | #define VEDIS_BUSY SXERR_BUSY /* The database file is locked */ 323 | #define VEDIS_DONE SXERR_DONE /* Operation done */ 324 | #define VEDIS_PERM SXERR_PERM /* Permission error */ 325 | #define VEDIS_NOTIMPLEMENTED SXERR_NOTIMPLEMENTED /* Method not implemented by the underlying Key/Value storage engine */ 326 | #define VEDIS_NOTFOUND SXERR_NOTFOUND /* No such record */ 327 | #define VEDIS_NOOP SXERR_NOOP /* No such method */ 328 | #define VEDIS_INVALID SXERR_INVALID /* Invalid parameter */ 329 | #define VEDIS_EOF SXERR_EOF /* End Of Input */ 330 | #define VEDIS_UNKNOWN SXERR_UNKNOWN /* Unknown configuration option */ 331 | #define VEDIS_LIMIT SXERR_LIMIT /* Database limit reached */ 332 | #define VEDIS_EXISTS SXERR_EXISTS /* Record exists */ 333 | #define VEDIS_EMPTY SXERR_EMPTY /* Empty record */ 334 | #define VEDIS_FULL (-73) /* Full database (unlikely) */ 335 | #define VEDIS_CANTOPEN (-74) /* Unable to open the database file */ 336 | #define VEDIS_READ_ONLY (-75) /* Read only Key/Value storage engine */ 337 | #define VEDIS_LOCKERR (-76) /* Locking protocol error */ 338 | /* end-of-error-codes */ 339 | /* 340 | * If compiling for a processor that lacks floating point 341 | * support, substitute integer for floating-point. 342 | */ 343 | #ifdef VEDIS_OMIT_FLOATING_POINT 344 | typedef sxi64 vedis_real; 345 | #else 346 | typedef double vedis_real; 347 | #endif 348 | typedef sxi64 vedis_int64; 349 | /* 350 | * Vedis Configuration Commands. 351 | * 352 | * The following set of constants are the available configuration verbs that can 353 | * be used by the host-application to configure a Vedis datastore handle. 354 | * These constants must be passed as the second argument to [vedis_config()]. 355 | * 356 | * Each options require a variable number of arguments. 357 | * The [vedis_config()] interface will return VEDIS_OK on success, any other 358 | * return value indicates failure. 359 | * For a full discussion on the configuration verbs and their expected 360 | * parameters, please refer to this page: 361 | * http://vedis.symisc.net/c_api/vedis_config.html 362 | */ 363 | #define VEDIS_CONFIG_ERR_LOG 1 /* TWO ARGUMENTS: const char **pzBuf, int *pLen */ 364 | #define VEDIS_CONFIG_MAX_PAGE_CACHE 2 /* ONE ARGUMENT: int nMaxPage */ 365 | #define VEDIS_CONFIG_KV_ENGINE 4 /* ONE ARGUMENT: const char *zKvName */ 366 | #define VEDIS_CONFIG_DISABLE_AUTO_COMMIT 5 /* NO ARGUMENTS */ 367 | #define VEDIS_CONFIG_GET_KV_NAME 6 /* ONE ARGUMENT: const char **pzPtr */ 368 | #define VEDIS_CONFIG_DUP_EXEC_VALUE 7 /* ONE ARGUMENT: vedis_value **ppOut */ 369 | #define VEDIS_CONFIG_RELEASE_DUP_VALUE 8 /* ONE ARGUMENT: vedis_value *pIn */ 370 | #define VEDIS_CONFIG_OUTPUT_CONSUMER 9 /* TWO ARGUMENTS: int (*xConsumer)(vedis_value *pOut,void *pUserdata), void *pUserdata */ 371 | /* 372 | * Storage engine configuration commands. 373 | * 374 | * The following set of constants are the available configuration verbs that can 375 | * be used by the host-application to configure the underlying storage engine (i.e Hash, B+tree, R+tree). 376 | * These constants must be passed as the first argument to [vedis_kv_config()]. 377 | * Each options require a variable number of arguments. 378 | * The [vedis_kv_config()] interface will return VEDIS_OK on success, any other return 379 | * value indicates failure. 380 | * For a full discussion on the configuration verbs and their expected parameters, please 381 | * refer to this page: 382 | * http://vedis.symisc.net/c_api/vedis_kv_config.html 383 | */ 384 | #define VEDIS_KV_CONFIG_HASH_FUNC 1 /* ONE ARGUMENT: unsigned int (*xHash)(const void *,unsigned int) */ 385 | #define VEDIS_KV_CONFIG_CMP_FUNC 2 /* ONE ARGUMENT: int (*xCmp)(const void *,const void *,unsigned int) */ 386 | /* 387 | * Global Library Configuration Commands. 388 | * 389 | * The following set of constants are the available configuration verbs that can 390 | * be used by the host-application to configure the whole library. 391 | * These constants must be passed as the first argument to [vedis_lib_config()]. 392 | * 393 | * Each options require a variable number of arguments. 394 | * The [vedis_lib_config()] interface will return VEDIS_OK on success, any other return 395 | * value indicates failure. 396 | * Notes: 397 | * The default configuration is recommended for most applications and so the call to 398 | * [vedis_lib_config()] is usually not necessary. It is provided to support rare 399 | * applications with unusual needs. 400 | * The [vedis_lib_config()] interface is not threadsafe. The application must insure that 401 | * no other [vedis_*()] interfaces are invoked by other threads while [vedis_lib_config()] 402 | * is running. Furthermore, [vedis_lib_config()] may only be invoked prior to library 403 | * initialization using [vedis_lib_init()] or [vedis_init()] or after shutdown 404 | * by [vedis_lib_shutdown()]. If [vedis_lib_config()] is called after [vedis_lib_init()] 405 | * or [vedis_init()] and before [vedis_lib_shutdown()] then it will return VEDIS_LOCKED. 406 | * For a full discussion on the configuration verbs and their expected parameters, please 407 | * refer to this page: 408 | * http://vedis.symisc.net/c_api/vedis_lib.html 409 | */ 410 | #define VEDIS_LIB_CONFIG_USER_MALLOC 1 /* ONE ARGUMENT: const SyMemMethods *pMemMethods */ 411 | #define VEDIS_LIB_CONFIG_MEM_ERR_CALLBACK 2 /* TWO ARGUMENTS: int (*xMemError)(void *), void *pUserData */ 412 | #define VEDIS_LIB_CONFIG_USER_MUTEX 3 /* ONE ARGUMENT: const SyMutexMethods *pMutexMethods */ 413 | #define VEDIS_LIB_CONFIG_THREAD_LEVEL_SINGLE 4 /* NO ARGUMENTS */ 414 | #define VEDIS_LIB_CONFIG_THREAD_LEVEL_MULTI 5 /* NO ARGUMENTS */ 415 | #define VEDIS_LIB_CONFIG_VFS 6 /* ONE ARGUMENT: const vedis_vfs *pVfs */ 416 | #define VEDIS_LIB_CONFIG_STORAGE_ENGINE 7 /* ONE ARGUMENT: vedis_kv_methods *pStorage */ 417 | #define VEDIS_LIB_CONFIG_PAGE_SIZE 8 /* ONE ARGUMENT: int iPageSize */ 418 | /* 419 | * Synchronization Type Flags 420 | * 421 | * When Vedis invokes the xSync() method of an [vedis_io_methods] object it uses 422 | * a combination of these integer values as the second argument. 423 | * 424 | * When the VEDIS_SYNC_DATAONLY flag is used, it means that the sync operation only 425 | * needs to flush data to mass storage. Inode information need not be flushed. 426 | * If the lower four bits of the flag equal VEDIS_SYNC_NORMAL, that means to use normal 427 | * fsync() semantics. If the lower four bits equal VEDIS_SYNC_FULL, that means to use 428 | * Mac OS X style fullsync instead of fsync(). 429 | */ 430 | #define VEDIS_SYNC_NORMAL 0x00002 431 | #define VEDIS_SYNC_FULL 0x00003 432 | #define VEDIS_SYNC_DATAONLY 0x00010 433 | /* 434 | * File Locking Levels 435 | * 436 | * Vedis uses one of these integer values as the second 437 | * argument to calls it makes to the xLock() and xUnlock() methods 438 | * of an [vedis_io_methods] object. 439 | */ 440 | #define VEDIS_LOCK_NONE 0 441 | #define VEDIS_LOCK_SHARED 1 442 | #define VEDIS_LOCK_RESERVED 2 443 | #define VEDIS_LOCK_PENDING 3 444 | #define VEDIS_LOCK_EXCLUSIVE 4 445 | /* 446 | * CAPIREF: OS Interface: Open File Handle 447 | * 448 | * An [vedis_file] object represents an open file in the [vedis_vfs] OS interface 449 | * layer. 450 | * Individual OS interface implementations will want to subclass this object by appending 451 | * additional fields for their own use. The pMethods entry is a pointer to an 452 | * [vedis_io_methods] object that defines methods for performing 453 | * I/O operations on the open file. 454 | */ 455 | typedef struct vedis_file vedis_file; 456 | struct vedis_file { 457 | const vedis_io_methods *pMethods; /* Methods for an open file. MUST BE FIRST */ 458 | }; 459 | /* 460 | * CAPIREF: OS Interface: File Methods Object 461 | * 462 | * Every file opened by the [vedis_vfs] xOpen method populates an 463 | * [vedis_file] object (or, more commonly, a subclass of the 464 | * [vedis_file] object) with a pointer to an instance of this object. 465 | * This object defines the methods used to perform various operations 466 | * against the open file represented by the [vedis_file] object. 467 | * 468 | * If the xOpen method sets the vedis_file.pMethods element 469 | * to a non-NULL pointer, then the vedis_io_methods.xClose method 470 | * may be invoked even if the xOpen reported that it failed. The 471 | * only way to prevent a call to xClose following a failed xOpen 472 | * is for the xOpen to set the vedis_file.pMethods element to NULL. 473 | * 474 | * The flags argument to xSync may be one of [VEDIS_SYNC_NORMAL] or 475 | * [VEDIS_SYNC_FULL]. The first choice is the normal fsync(). 476 | * The second choice is a Mac OS X style fullsync. The [VEDIS_SYNC_DATAONLY] 477 | * flag may be ORed in to indicate that only the data of the file 478 | * and not its inode needs to be synced. 479 | * 480 | * The integer values to xLock() and xUnlock() are one of 481 | * 482 | * VEDIS_LOCK_NONE 483 | * VEDIS_LOCK_SHARED 484 | * VEDIS_LOCK_RESERVED 485 | * VEDIS_LOCK_PENDING 486 | * VEDIS_LOCK_EXCLUSIVE 487 | * 488 | * xLock() increases the lock. xUnlock() decreases the lock. 489 | * The xCheckReservedLock() method checks whether any database connection, 490 | * either in this process or in some other process, is holding a RESERVED, 491 | * PENDING, or EXCLUSIVE lock on the file. It returns true if such a lock exists 492 | * and false otherwise. 493 | * 494 | * The xSectorSize() method returns the sector size of the device that underlies 495 | * the file. The sector size is the minimum write that can be performed without 496 | * disturbing other bytes in the file. 497 | */ 498 | struct vedis_io_methods { 499 | int iVersion; /* Structure version number (currently 1) */ 500 | int (*xClose)(vedis_file*); 501 | int (*xRead)(vedis_file*, void*, vedis_int64 iAmt, vedis_int64 iOfst); 502 | int (*xWrite)(vedis_file*, const void*, vedis_int64 iAmt, vedis_int64 iOfst); 503 | int (*xTruncate)(vedis_file*, vedis_int64 size); 504 | int (*xSync)(vedis_file*, int flags); 505 | int (*xFileSize)(vedis_file*, vedis_int64 *pSize); 506 | int (*xLock)(vedis_file*, int); 507 | int (*xUnlock)(vedis_file*, int); 508 | int (*xCheckReservedLock)(vedis_file*, int *pResOut); 509 | int (*xSectorSize)(vedis_file*); 510 | }; 511 | /* 512 | * CAPIREF: OS Interface Object 513 | * 514 | * An instance of the vedis_vfs object defines the interface between 515 | * the Vedis core and the underlying operating system. The "vfs" 516 | * in the name of the object stands for "Virtual File System". 517 | * 518 | * Only a single vfs can be registered within the Vedis core. 519 | * Vfs registration is done using the [vedis_lib_config()] interface 520 | * with a configuration verb set to VEDIS_LIB_CONFIG_VFS. 521 | * Note that Windows and UNIX (Linux, FreeBSD, Solaris, Mac OS X, etc.) users 522 | * does not have to worry about registering and installing a vfs since Vedis 523 | * come with a built-in vfs for these platforms that implements most the methods 524 | * defined below. 525 | * 526 | * Clients running on exotic systems (ie: Other than Windows and UNIX systems) 527 | * must register their own vfs in order to be able to use the Vedis library. 528 | * 529 | * The value of the iVersion field is initially 1 but may be larger in 530 | * future versions of Vedis. 531 | * 532 | * The szOsFile field is the size of the subclassed [vedis_file] structure 533 | * used by this VFS. mxPathname is the maximum length of a pathname in this VFS. 534 | * 535 | * At least szOsFile bytes of memory are allocated by Vedis to hold the [vedis_file] 536 | * structure passed as the third argument to xOpen. The xOpen method does not have to 537 | * allocate the structure; it should just fill it in. Note that the xOpen method must 538 | * set the vedis_file.pMethods to either a valid [vedis_io_methods] object or to NULL. 539 | * xOpen must do this even if the open fails. Vedis expects that the vedis_file.pMethods 540 | * element will be valid after xOpen returns regardless of the success or failure of the 541 | * xOpen call. 542 | */ 543 | struct vedis_vfs { 544 | const char *zName; /* Name of this virtual file system [i.e: Windows, UNIX, etc.] */ 545 | int iVersion; /* Structure version number (currently 1) */ 546 | int szOsFile; /* Size of subclassed vedis_file */ 547 | int mxPathname; /* Maximum file pathname length */ 548 | int (*xOpen)(vedis_vfs*, const char *zName, vedis_file*,unsigned int flags); 549 | int (*xDelete)(vedis_vfs*, const char *zName, int syncDir); 550 | int (*xAccess)(vedis_vfs*, const char *zName, int flags, int *pResOut); 551 | int (*xFullPathname)(vedis_vfs*, const char *zName,int buf_len,char *zBuf); 552 | int (*xTmpDir)(vedis_vfs*,char *zBuf,int buf_len); 553 | int (*xSleep)(vedis_vfs*, int microseconds); 554 | int (*xCurrentTime)(vedis_vfs*,Sytm *pOut); 555 | int (*xGetLastError)(vedis_vfs*, int, char *); 556 | int (*xMmap)(const char *, void **, vedis_int64 *); 557 | void (*xUnmap)(void *,vedis_int64); 558 | }; 559 | /* 560 | * Flags for the xAccess VFS method 561 | * 562 | * These integer constants can be used as the third parameter to 563 | * the xAccess method of an [vedis_vfs] object. They determine 564 | * what kind of permissions the xAccess method is looking for. 565 | * With VEDIS_ACCESS_EXISTS, the xAccess method 566 | * simply checks whether the file exists. 567 | * With VEDIS_ACCESS_READWRITE, the xAccess method 568 | * checks whether the named directory is both readable and writable 569 | * (in other words, if files can be added, removed, and renamed within 570 | * the directory). 571 | * The VEDIS_ACCESS_READWRITE constant is currently used only by the 572 | * [temp_store_directory pragma], though this could change in a future 573 | * release of Vedis. 574 | * With VEDIS_ACCESS_READ, the xAccess method 575 | * checks whether the file is readable. The VEDIS_ACCESS_READ constant is 576 | * currently unused, though it might be used in a future release of 577 | * Vedis. 578 | */ 579 | #define VEDIS_ACCESS_EXISTS 0 580 | #define VEDIS_ACCESS_READWRITE 1 581 | #define VEDIS_ACCESS_READ 2 582 | /* 583 | * The type used to represent a page number. The first page in a file 584 | * is called page 1. 0 is used to represent "not a page". 585 | * A page number is an unsigned 64-bit integer. 586 | */ 587 | typedef sxu64 pgno; 588 | /* 589 | * A database disk page is represented by an instance 590 | * of the follwoing structure. 591 | */ 592 | typedef struct vedis_page vedis_page; 593 | struct vedis_page 594 | { 595 | unsigned char *zData; /* Content of this page */ 596 | void *pUserData; /* Extra content */ 597 | pgno pgno; /* Page number for this page */ 598 | }; 599 | /* 600 | * Vedis handle to the underlying Key/Value Storage Engine (See below). 601 | */ 602 | typedef void * vedis_kv_handle; 603 | /* 604 | * Vedis pager IO methods. 605 | * 606 | * An instance of the following structure define the exported methods of the Vedis pager 607 | * to the underlying Key/Value storage engine. 608 | */ 609 | typedef struct vedis_kv_io vedis_kv_io; 610 | struct vedis_kv_io 611 | { 612 | vedis_kv_handle pHandle; /* Vedis handle passed as the first parameter to the 613 | * method defined below. 614 | */ 615 | vedis_kv_methods *pMethods; /* Underlying storage engine */ 616 | /* Pager methods */ 617 | int (*xGet)(vedis_kv_handle,pgno,vedis_page **); 618 | int (*xLookup)(vedis_kv_handle,pgno,vedis_page **); 619 | int (*xNew)(vedis_kv_handle,vedis_page **); 620 | int (*xWrite)(vedis_page *); 621 | int (*xDontWrite)(vedis_page *); 622 | int (*xDontJournal)(vedis_page *); 623 | int (*xDontMkHot)(vedis_page *); 624 | int (*xPageRef)(vedis_page *); 625 | int (*xPageUnref)(vedis_page *); 626 | int (*xPageSize)(vedis_kv_handle); 627 | int (*xReadOnly)(vedis_kv_handle); 628 | unsigned char * (*xTmpPage)(vedis_kv_handle); 629 | void (*xSetUnpin)(vedis_kv_handle,void (*xPageUnpin)(void *)); 630 | void (*xSetReload)(vedis_kv_handle,void (*xPageReload)(void *)); 631 | void (*xErr)(vedis_kv_handle,const char *); 632 | }; 633 | /* 634 | * Key/Value Cursor Object. 635 | * 636 | * An instance of a subclass of the following object defines a cursor 637 | * used to scan through a key-value storage engine. 638 | */ 639 | typedef struct vedis_kv_cursor vedis_kv_cursor; 640 | struct vedis_kv_cursor 641 | { 642 | vedis_kv_engine *pStore; /* Must be first */ 643 | /* Subclasses will typically add additional fields */ 644 | }; 645 | /* 646 | * Possible seek positions. 647 | */ 648 | #define VEDIS_CURSOR_MATCH_EXACT 1 649 | #define VEDIS_CURSOR_MATCH_LE 2 650 | #define VEDIS_CURSOR_MATCH_GE 3 651 | /* 652 | * Key/Value Storage Engine. 653 | * 654 | * A Key-Value storage engine is defined by an instance of the following 655 | * object. 656 | * Vedis works with run-time interchangeable storage engines (i.e. Hash, B+Tree, R+Tree, LSM, etc.). 657 | * The storage engine works with key/value pairs where both the key 658 | * and the value are byte arrays of arbitrary length and with no restrictions on content. 659 | * Vedis come with two built-in KV storage engine: A Virtual Linear Hash (VLH) storage 660 | * engine is used for persistent on-disk databases with O(1) lookup time and an in-memory 661 | * hash-table or Red-black tree storage engine is used for in-memory databases. 662 | * Future versions of Vedis might add other built-in storage engines (i.e. LSM). 663 | * Registration of a Key/Value storage engine at run-time is done via [vedis_lib_config()] 664 | * with a configuration verb set to VEDIS_LIB_CONFIG_STORAGE_ENGINE. 665 | */ 666 | struct vedis_kv_engine 667 | { 668 | const vedis_kv_io *pIo; /* IO methods: MUST be first */ 669 | /* Subclasses will typically add additional fields */ 670 | }; 671 | /* 672 | * Key/Value Storage Engine Virtual Method Table. 673 | * 674 | * Key/Value storage engine methods is defined by an instance of the following 675 | * object. 676 | * Registration of a Key/Value storage engine at run-time is done via [vedis_lib_config()] 677 | * with a configuration verb set to VEDIS_LIB_CONFIG_STORAGE_ENGINE. 678 | */ 679 | struct vedis_kv_methods 680 | { 681 | const char *zName; /* Storage engine name [i.e. Hash, B+tree, LSM, R-tree, Mem, etc.]*/ 682 | int szKv; /* 'vedis_kv_engine' subclass size */ 683 | int szCursor; /* 'vedis_kv_cursor' subclass size */ 684 | int iVersion; /* Structure version, currently 1 */ 685 | /* Storage engine methods */ 686 | int (*xInit)(vedis_kv_engine *,int iPageSize); 687 | void (*xRelease)(vedis_kv_engine *); 688 | int (*xConfig)(vedis_kv_engine *,int op,va_list ap); 689 | int (*xOpen)(vedis_kv_engine *,pgno); 690 | int (*xReplace)( 691 | vedis_kv_engine *, 692 | const void *pKey,int nKeyLen, 693 | const void *pData,vedis_int64 nDataLen 694 | ); 695 | int (*xAppend)( 696 | vedis_kv_engine *, 697 | const void *pKey,int nKeyLen, 698 | const void *pData,vedis_int64 nDataLen 699 | ); 700 | void (*xCursorInit)(vedis_kv_cursor *); 701 | int (*xSeek)(vedis_kv_cursor *,const void *pKey,int nByte,int iPos); /* Mandatory */ 702 | int (*xFirst)(vedis_kv_cursor *); 703 | int (*xLast)(vedis_kv_cursor *); 704 | int (*xValid)(vedis_kv_cursor *); 705 | int (*xNext)(vedis_kv_cursor *); 706 | int (*xPrev)(vedis_kv_cursor *); 707 | int (*xDelete)(vedis_kv_cursor *); 708 | int (*xKeyLength)(vedis_kv_cursor *,int *); 709 | int (*xKey)(vedis_kv_cursor *,int (*xConsumer)(const void *,unsigned int,void *),void *pUserData); 710 | int (*xDataLength)(vedis_kv_cursor *,vedis_int64 *); 711 | int (*xData)(vedis_kv_cursor *,int (*xConsumer)(const void *,unsigned int,void *),void *pUserData); 712 | void (*xReset)(vedis_kv_cursor *); 713 | void (*xCursorRelease)(vedis_kv_cursor *); 714 | }; 715 | /* 716 | * Vedis journal file suffix. 717 | */ 718 | #ifndef VEDIS_JOURNAL_FILE_SUFFIX 719 | #define VEDIS_JOURNAL_FILE_SUFFIX "_vedis_journal" 720 | #endif 721 | /* 722 | * Call Context - Error Message Serverity Level. 723 | * 724 | * The following constans are the allowed severity level that can 725 | * passed as the second argument to the [vedis_context_throw_error()] or 726 | * [vedis_context_throw_error_format()] interfaces. 727 | * Refer to the official documentation for additional information. 728 | */ 729 | #define VEDIS_CTX_ERR 1 /* Call context error such as unexpected number of arguments, invalid types and so on. */ 730 | #define VEDIS_CTX_WARNING 2 /* Call context Warning */ 731 | #define VEDIS_CTX_NOTICE 3 /* Call context Notice */ 732 | /* 733 | * C-API-REF: Please refer to the official documentation for interfaces 734 | * purpose and expected parameters. 735 | */ 736 | /* Vedis Datastore Handle */ 737 | VEDIS_APIEXPORT int vedis_open(vedis **ppStore,const char *zStorage); 738 | VEDIS_APIEXPORT int vedis_config(vedis *pStore,int iOp,...); 739 | VEDIS_APIEXPORT int vedis_close(vedis *pStore); 740 | 741 | /* Command Execution Interfaces */ 742 | VEDIS_APIEXPORT int vedis_exec(vedis *pStore,const char *zCmd,int nLen); 743 | VEDIS_APIEXPORT int vedis_exec_fmt(vedis *pStore,const char *zFmt,...); 744 | VEDIS_APIEXPORT int vedis_exec_result(vedis *pStore,vedis_value **ppOut); 745 | 746 | /* Foreign Command Registar */ 747 | VEDIS_APIEXPORT int vedis_register_command(vedis *pStore,const char *zName,int (*xCmd)(vedis_context *,int,vedis_value **),void *pUserdata); 748 | VEDIS_APIEXPORT int vedis_delete_command(vedis *pStore,const char *zName); 749 | 750 | /* Raw Data Store/Fetch (http://vedis.org) */ 751 | VEDIS_APIEXPORT int vedis_kv_store(vedis *pStore,const void *pKey,int nKeyLen,const void *pData,vedis_int64 nDataLen); 752 | VEDIS_APIEXPORT int vedis_kv_append(vedis *pStore,const void *pKey,int nKeyLen,const void *pData,vedis_int64 nDataLen); 753 | VEDIS_APIEXPORT int vedis_kv_store_fmt(vedis *pStore,const void *pKey,int nKeyLen,const char *zFormat,...); 754 | VEDIS_APIEXPORT int vedis_kv_append_fmt(vedis *pStore,const void *pKey,int nKeyLen,const char *zFormat,...); 755 | VEDIS_APIEXPORT int vedis_kv_fetch(vedis *pStore,const void *pKey,int nKeyLen,void *pBuf,vedis_int64 /* in|out */*pBufLen); 756 | VEDIS_APIEXPORT int vedis_kv_fetch_callback(vedis *pStore,const void *pKey, 757 | int nKeyLen,int (*xConsumer)(const void *,unsigned int,void *),void *pUserData); 758 | VEDIS_APIEXPORT int vedis_kv_config(vedis *pStore,int iOp,...); 759 | VEDIS_APIEXPORT int vedis_kv_delete(vedis *pStore,const void *pKey,int nKeyLen); 760 | 761 | /* Manual Transaction Manager */ 762 | VEDIS_APIEXPORT int vedis_begin(vedis *pStore); 763 | VEDIS_APIEXPORT int vedis_commit(vedis *pStore); 764 | VEDIS_APIEXPORT int vedis_rollback(vedis *pStore); 765 | 766 | /* Utility interfaces */ 767 | VEDIS_APIEXPORT int vedis_util_random_string(vedis *pStore,char *zBuf,unsigned int buf_size); 768 | VEDIS_APIEXPORT unsigned int vedis_util_random_num(vedis *pStore); 769 | 770 | /* Call Context Key/Value Store Interfaces */ 771 | VEDIS_APIEXPORT int vedis_context_kv_store(vedis_context *pCtx,const void *pKey,int nKeyLen,const void *pData,vedis_int64 nDataLen); 772 | VEDIS_APIEXPORT int vedis_context_kv_append(vedis_context *pCtx,const void *pKey,int nKeyLen,const void *pData,vedis_int64 nDataLen); 773 | VEDIS_APIEXPORT int vedis_context_kv_store_fmt(vedis_context *pCtx,const void *pKey,int nKeyLen,const char *zFormat,...); 774 | VEDIS_APIEXPORT int vedis_context_kv_append_fmt(vedis_context *pCtx,const void *pKey,int nKeyLen,const char *zFormat,...); 775 | VEDIS_APIEXPORT int vedis_context_kv_fetch(vedis_context *pCtx,const void *pKey,int nKeyLen,void *pBuf,vedis_int64 /* in|out */*pBufLen); 776 | VEDIS_APIEXPORT int vedis_context_kv_fetch_callback(vedis_context *pCtx,const void *pKey, 777 | int nKeyLen,int (*xConsumer)(const void *,unsigned int,void *),void *pUserData); 778 | VEDIS_APIEXPORT int vedis_context_kv_delete(vedis_context *pCtx,const void *pKey,int nKeyLen); 779 | 780 | /* Command Execution Context Interfaces */ 781 | VEDIS_APIEXPORT int vedis_context_throw_error(vedis_context *pCtx, int iErr, const char *zErr); 782 | VEDIS_APIEXPORT int vedis_context_throw_error_format(vedis_context *pCtx, int iErr, const char *zFormat, ...); 783 | VEDIS_APIEXPORT unsigned int vedis_context_random_num(vedis_context *pCtx); 784 | VEDIS_APIEXPORT int vedis_context_random_string(vedis_context *pCtx, char *zBuf, int nBuflen); 785 | VEDIS_APIEXPORT void * vedis_context_user_data(vedis_context *pCtx); 786 | VEDIS_APIEXPORT int vedis_context_push_aux_data(vedis_context *pCtx, void *pUserData); 787 | VEDIS_APIEXPORT void * vedis_context_peek_aux_data(vedis_context *pCtx); 788 | VEDIS_APIEXPORT void * vedis_context_pop_aux_data(vedis_context *pCtx); 789 | 790 | /* Setting The Return Value Of A Vedis Command */ 791 | VEDIS_APIEXPORT int vedis_result_int(vedis_context *pCtx, int iValue); 792 | VEDIS_APIEXPORT int vedis_result_int64(vedis_context *pCtx, vedis_int64 iValue); 793 | VEDIS_APIEXPORT int vedis_result_bool(vedis_context *pCtx, int iBool); 794 | VEDIS_APIEXPORT int vedis_result_double(vedis_context *pCtx, double Value); 795 | VEDIS_APIEXPORT int vedis_result_null(vedis_context *pCtx); 796 | VEDIS_APIEXPORT int vedis_result_string(vedis_context *pCtx, const char *zString, int nLen); 797 | VEDIS_APIEXPORT int vedis_result_string_format(vedis_context *pCtx, const char *zFormat, ...); 798 | VEDIS_APIEXPORT int vedis_result_value(vedis_context *pCtx, vedis_value *pValue); 799 | 800 | /* Extracting Vedis Commands Parameter/Return Values */ 801 | VEDIS_APIEXPORT int vedis_value_to_int(vedis_value *pValue); 802 | VEDIS_APIEXPORT int vedis_value_to_bool(vedis_value *pValue); 803 | VEDIS_APIEXPORT vedis_int64 vedis_value_to_int64(vedis_value *pValue); 804 | VEDIS_APIEXPORT double vedis_value_to_double(vedis_value *pValue); 805 | VEDIS_APIEXPORT const char * vedis_value_to_string(vedis_value *pValue, int *pLen); 806 | 807 | /* Dynamically Typed Value Object Query Interfaces */ 808 | VEDIS_APIEXPORT int vedis_value_is_int(vedis_value *pVal); 809 | VEDIS_APIEXPORT int vedis_value_is_float(vedis_value *pVal); 810 | VEDIS_APIEXPORT int vedis_value_is_bool(vedis_value *pVal); 811 | VEDIS_APIEXPORT int vedis_value_is_string(vedis_value *pVal); 812 | VEDIS_APIEXPORT int vedis_value_is_null(vedis_value *pVal); 813 | VEDIS_APIEXPORT int vedis_value_is_numeric(vedis_value *pVal); 814 | VEDIS_APIEXPORT int vedis_value_is_scalar(vedis_value *pVal); 815 | VEDIS_APIEXPORT int vedis_value_is_array(vedis_value *pVal); 816 | 817 | /* Populating dynamically Typed Objects */ 818 | VEDIS_APIEXPORT int vedis_value_int(vedis_value *pVal, int iValue); 819 | VEDIS_APIEXPORT int vedis_value_int64(vedis_value *pVal, vedis_int64 iValue); 820 | VEDIS_APIEXPORT int vedis_value_bool(vedis_value *pVal, int iBool); 821 | VEDIS_APIEXPORT int vedis_value_null(vedis_value *pVal); 822 | VEDIS_APIEXPORT int vedis_value_double(vedis_value *pVal, double Value); 823 | VEDIS_APIEXPORT int vedis_value_string(vedis_value *pVal, const char *zString, int nLen); 824 | VEDIS_APIEXPORT int vedis_value_string_format(vedis_value *pVal, const char *zFormat, ...); 825 | VEDIS_APIEXPORT int vedis_value_reset_string_cursor(vedis_value *pVal); 826 | VEDIS_APIEXPORT int vedis_value_release(vedis_value *pVal); 827 | 828 | /* On-demand Object Value Allocation */ 829 | VEDIS_APIEXPORT vedis_value * vedis_context_new_scalar(vedis_context *pCtx); 830 | VEDIS_APIEXPORT vedis_value * vedis_context_new_array(vedis_context *pCtx); 831 | VEDIS_APIEXPORT void vedis_context_release_value(vedis_context *pCtx, vedis_value *pValue); 832 | 833 | /* Working with Vedis Arrays */ 834 | VEDIS_APIEXPORT vedis_value * vedis_array_fetch(vedis_value *pArray,unsigned int index); 835 | VEDIS_APIEXPORT int vedis_array_walk(vedis_value *pArray, int (*xWalk)(vedis_value *, void *), void *pUserData); 836 | VEDIS_APIEXPORT int vedis_array_insert(vedis_value *pArray,vedis_value *pValue); 837 | VEDIS_APIEXPORT unsigned int vedis_array_count(vedis_value *pArray); 838 | VEDIS_APIEXPORT int vedis_array_reset(vedis_value *pArray); 839 | VEDIS_APIEXPORT vedis_value * vedis_array_next_elem(vedis_value *pArray); 840 | 841 | /* Global Library Management Interfaces */ 842 | VEDIS_APIEXPORT int vedis_lib_init(void); 843 | VEDIS_APIEXPORT int vedis_lib_config(int nConfigOp, ...); 844 | VEDIS_APIEXPORT int vedis_lib_shutdown(void); 845 | VEDIS_APIEXPORT int vedis_lib_is_threadsafe(void); 846 | VEDIS_APIEXPORT const char * vedis_lib_version(void); 847 | VEDIS_APIEXPORT const char * vedis_lib_signature(void); 848 | VEDIS_APIEXPORT const char * vedis_lib_ident(void); 849 | VEDIS_APIEXPORT const char * vedis_lib_copyright(void); 850 | 851 | #ifdef __cplusplus 852 | } 853 | #endif /* __cplusplus */ 854 | #endif /* _VEDIS_H_ */ 855 | -------------------------------------------------------------------------------- /tests.py: -------------------------------------------------------------------------------- 1 | import base64 2 | import csv 3 | import os 4 | import re 5 | try: 6 | from StringIO import StringIO 7 | except ImportError: 8 | from io import StringIO 9 | import sys 10 | import unittest 11 | 12 | try: 13 | from vedis import Vedis 14 | except ImportError: 15 | sys.stderr.write('Unable to import `vedis`. Make sure it is properly ' 16 | 'installed.\n') 17 | sys.stderr.flush() 18 | raise 19 | 20 | 21 | if sys.version_info[0] == 3: 22 | basestring = str 23 | long = int 24 | 25 | 26 | class BaseVedisTestCase(unittest.TestCase): 27 | def setUp(self): 28 | super(BaseVedisTestCase, self).setUp() 29 | self.db = Vedis(':memory:') 30 | 31 | def tearDown(self): 32 | self.db.close() 33 | super(BaseVedisTestCase, self).tearDown() 34 | 35 | def set_k1_k2(self): 36 | # Short-hand for setting two keys. 37 | self.db['k1'] = 'v1' 38 | self.db['k2'] = 'v2' 39 | 40 | 41 | class TestKeyValueAPI(BaseVedisTestCase): 42 | def test_kv_api(self): 43 | self.db.store('k1', 'v1') 44 | self.db.store('k2', 'v2') 45 | self.assertEqual(self.db.fetch('k1'), b'v1') 46 | self.assertEqual(self.db.fetch('k2'), b'v2') 47 | self.assertRaises(KeyError, self.db.fetch, 'k3') 48 | 49 | self.db.append('k1', 'V1') 50 | self.assertEqual(self.db.fetch('k1'), b'v1V1') 51 | 52 | self.db.append('k3', 'v3') 53 | self.assertEqual(self.db.fetch('k3'), b'v3') 54 | 55 | self.assertTrue(self.db.exists('k1')) 56 | self.db.delete('k1') 57 | self.assertFalse(self.db.exists('k1')) 58 | 59 | self.assertRaises(KeyError, lambda: self.db.delete('kx')) 60 | 61 | self.db['k3'] = '' 62 | self.assertTrue(self.db.exists('k3')) 63 | 64 | def test_dict_api(self): 65 | self.db['k1'] = 'v1' 66 | self.db['k2'] = 'v2' 67 | self.assertEqual(self.db['k1'], b'v1') 68 | self.assertEqual(self.db['k2'], b'v2') 69 | self.assertRaises(KeyError, lambda: self.db['k3']) 70 | 71 | self.db.update({'k3': 'v3', 'k2': ''}) 72 | self.assertEqual(self.db['k3'], b'v3') 73 | self.assertEqual(self.db['k2'], b'') 74 | 75 | self.assertTrue('k1' in self.db) 76 | self.assertTrue('k3' in self.db) 77 | self.assertFalse('k4' in self.db) 78 | 79 | del self.db['k3'] 80 | self.assertFalse('k3' in self.db) 81 | self.assertRaises(KeyError, lambda: self.db['k3']) 82 | 83 | def test_storing_ints(self): 84 | self.db['k1'] = 1 85 | self.db['k2'] = 1234567890 86 | self.assertEqual(self.db['k1'], b'1') 87 | self.assertEqual(self.db['k2'], b'1234567890') 88 | 89 | 90 | class TestBasicCommands(BaseVedisTestCase): 91 | def test_get_set(self): 92 | # XXX: These actually use the key/value API for performance. 93 | self.db.set('k1', 'v1') 94 | self.db.set('k2', 'v2') 95 | self.assertEqual(self.db.get('k1'), b'v1') 96 | self.assertEqual(self.db.get('k2'), b'v2') 97 | self.assertRaises(KeyError, lambda: self.db['k3']) 98 | 99 | def test_exists_and_object_types(self): 100 | # For some reason, `EXIST` does not work with object types. 101 | self.db.hset('my_hash', 'k1', 'v1') 102 | self.db.sadd('my_set', 'v1') 103 | self.db.lpush('my_list', 'i1') 104 | 105 | self.assertFalse(self.db.exists('my_hash')) 106 | self.assertFalse(self.db.exists('my_set')) 107 | self.assertFalse(self.db.exists('my_list')) 108 | 109 | def test_mget(self): 110 | self.set_k1_k2() 111 | res = self.db.mget(['k1', 'missing', 'k2']) 112 | self.assertEqual(res, [b'v1', None, b'v2']) 113 | 114 | def test_setnx(self): 115 | self.db['k1'] = 'v1' 116 | self.db.setnx('k1', 'v-x') 117 | self.assertEqual(self.db['k1'], b'v1') 118 | 119 | self.db.setnx('k2', 'v-x') 120 | self.assertEqual(self.db['k2'], b'v-x') 121 | 122 | def test_mset(self): 123 | self.db['k1'] = 'v1' 124 | self.db.mset(dict(k1='v-x', k2='v2', foo='bar')) 125 | self.assertEqual( 126 | self.db.mget(['k1', 'k2', 'foo']), 127 | [b'v-x', b'v2', b'bar']) 128 | 129 | self.db.mset({'k s': 'vs', 'k s2': 'vs2'}) 130 | self.assertEqual( 131 | self.db.mget(['k s', 'k s2']), 132 | [b'vs', b'vs2']) 133 | 134 | def test_msetnx(self): 135 | self.db['k1'] = 'v1' 136 | self.db.msetnx(dict(k1='v-x', k2='v2', foo='bar')) 137 | self.assertEqual( 138 | self.db.mget(['k1', 'k2', 'foo']), 139 | [b'v1', b'v2', b'bar']) 140 | 141 | def test_getset(self): 142 | res = self.db.get_set('k1', 'v1') 143 | self.assertIsNone(res) 144 | 145 | res = self.db.get_set('k1', 'v-x') 146 | self.assertEqual(res, b'v1') 147 | self.assertEqual(self.db['k1'], b'v-x') 148 | 149 | def test_incr(self): 150 | res = self.db.incr('counter') 151 | self.assertEqual(res, 1) 152 | 153 | res = self.db.incr('counter') 154 | self.assertEqual(res, 2) 155 | 156 | def test_decr(self): 157 | res = self.db.decr('counter') 158 | self.assertEqual(res, -1) 159 | 160 | res = self.db.decr('counter') 161 | self.assertEqual(res, -2) 162 | 163 | def test_incr_by_decr_by(self): 164 | res = self.db.incr_by('c', 100) 165 | self.assertEqual(res, 100) 166 | 167 | res = self.db.incr_by('c', 10) 168 | self.assertEqual(res, 110) 169 | 170 | res = self.db.decr_by('c', 90) 171 | self.assertEqual(res, 20) 172 | 173 | def test_quoted_values(self): 174 | self.db['k"1"'] = 'value "with quotes"' 175 | res = self.db['k"1"'] 176 | self.assertEqual(res, b'value "with quotes"') 177 | 178 | def test_numbers(self): 179 | self.db['1'] = '2' 180 | self.assertEqual(self.db['1'], b'2') 181 | self.db.append('1', '3') 182 | self.assertEqual(self.db['1'], b'23') 183 | self.assertTrue(self.db.exists('1')) 184 | self.assertFalse(self.db.exists('2')) 185 | 186 | 187 | class TestStringCommands(BaseVedisTestCase): 188 | def test_strlen(self): 189 | self.db['k1'] = 'foo' 190 | self.db['k2'] = '' 191 | self.assertEqual(self.db.strlen('k1'), 3) 192 | self.assertEqual(self.db.strlen('k2'), 0) 193 | self.assertEqual(self.db.strlen('missing'), 0) 194 | 195 | def test_copy(self): 196 | self.db['k1'] = 'v1' 197 | self.db.copy('k1', 'k2') 198 | self.assertEqual(self.db['k2'], b'v1') 199 | self.assertEqual(self.db['k1'], self.db['k2']) 200 | 201 | def test_move(self): 202 | self.db['k1'] = 'v1' 203 | self.db.move('k1', 'k2') 204 | self.assertEqual(self.db['k2'], b'v1') 205 | self.assertFalse(self.db.exists('k1')) 206 | 207 | def test_random_string(self): 208 | rs = self.db.random_string(5) 209 | self.assertEqual(len(rs), 5) 210 | self.assertTrue(isinstance(rs, bytes)) 211 | 212 | rs2 = self.db.randstr(5) 213 | self.assertEqual(len(rs2), 5) 214 | 215 | def test_random_int(self): 216 | rn = self.db.random_int() 217 | self.assertTrue(isinstance(rn, int)) 218 | 219 | rn2 = self.db.rand(1, 10) 220 | self.assertTrue(isinstance(rn, int)) 221 | 222 | def test_strip_tags(self): 223 | data = '

This is a test.

' 224 | res = self.db.strip_tags(data) 225 | self.assertEqual(res, b'This is a test.') 226 | 227 | def test_str_split(self): 228 | res = self.db.str_split('abcdefghijklmnopqrstuvwxyz', 5) 229 | self.assertEqual(list(res), [ 230 | b'abcde', 231 | b'fghij', 232 | b'klmno', 233 | b'pqrst', 234 | b'uvwxy', 235 | b'z', 236 | ]) 237 | 238 | def test_size_format(self): 239 | res = self.db.size_format(100000) 240 | self.assertEqual(res, b'97.6 KB') 241 | 242 | def test_base64(self): 243 | data = 'huey and mickey' 244 | encoded = self.db.base64(data) 245 | decoded = self.db.base64_decode(encoded) 246 | self.assertEqual(decoded, data.encode('utf-8')) 247 | 248 | 249 | class TestHashCommands(BaseVedisTestCase): 250 | def test_getsetdel(self): 251 | self.db.hset('hash', 'k1', 'v1') 252 | self.db.hset('hash', 'k2', 'v2') 253 | self.assertEqual(self.db.hget('hash', 'k1'), b'v1') 254 | self.assertIsNone(self.db.hget('hash', 'missing')) 255 | self.db.hdel('hash', 'k1') 256 | self.assertIsNone(self.db.hget('hash', 'k1')) 257 | 258 | def test_keys_vals(self): 259 | self.db.hset('hash', 'k1', 'v1') 260 | self.db.hset('hash', 'k2', 'v2') 261 | self.assertEqual(sorted(self.db.hkeys('hash')), [b'k1', b'k2']) 262 | self.assertEqual(sorted(self.db.hvals('hash')), [b'v1', b'v2']) 263 | self.assertEqual(self.db.hkeys('missing'), None) 264 | self.assertEqual(self.db.hvals('missing'), None) 265 | 266 | def test_hash_methods(self): 267 | self.db.hmset('hash', dict(k1='v1', k2='v2', k3='v3')) 268 | self.assertEqual(self.db.hgetall('hash'), { 269 | b'k1': b'v1', 270 | b'k2': b'v2', 271 | b'k3': b'v3'}) 272 | self.assertEqual(sorted(self.db.hitems('hash')), [ 273 | (b'k1', b'v1'), 274 | (b'k2', b'v2'), 275 | (b'k3', b'v3'), 276 | ]) 277 | 278 | self.assertEqual( 279 | self.db.hmget('hash', ['k1', 'missing', 'k2']), 280 | [b'v1', None, b'v2']) 281 | 282 | self.assertEqual(self.db.hlen('hash'), 3) 283 | self.assertTrue(self.db.hexists('hash', 'k1')) 284 | self.assertFalse(self.db.hexists('hash', 'missing')) 285 | 286 | self.db.hsetnx('hash', 'k1', 'v-x') 287 | self.db.hsetnx('hash', 'k4', 'v-x') 288 | self.assertEqual(self.db.hgetall('hash'), { 289 | b'k1': b'v1', 290 | b'k2': b'v2', 291 | b'k3': b'v3', 292 | b'k4': b'v-x'}) 293 | 294 | def test_hash_methods_missing(self): 295 | self.assertEqual(self.db.hgetall('missing'), {}) 296 | self.assertEqual(self.db.hlen('missing'), 0) 297 | self.assertEqual(self.db.hmget('missing', ['x']), None) 298 | self.assertFalse(self.db.hexists('missing', 'x')) 299 | 300 | def test_quoting(self): 301 | in_data = { 302 | 'k "1"': 'v "1"', 303 | 'k "2"': 'v "2"', 304 | } 305 | self.db.hmset('hash', in_data) 306 | out_data = self.db.hgetall('hash') 307 | self.assertEqual(out_data, { 308 | b'k "1"': b'v "1"', 309 | b'k "2"': b'v "2"'}) 310 | 311 | 312 | class TestSetCommands(BaseVedisTestCase): 313 | def test_set_methods(self): 314 | vals = set(['v1', 'v2', 'v3', 'v4']) 315 | for val in vals: 316 | self.db.sadd('set', val) 317 | 318 | self.assertEqual(self.db.scard('set'), 4) 319 | self.assertTrue(self.db.sismember('set', 'v1')) 320 | self.assertFalse(self.db.sismember('set', 'missing')) 321 | self.assertTrue(self.db.srem('set', b'v2')) 322 | self.assertEqual(self.db.slen('set'), 3) 323 | 324 | #self.assertEqual(self.db.speek('set'), 'v4') 325 | self.assertTrue(self.db.spop('set') in (b'v1', b'v2', b'v3', b'v4')) 326 | 327 | #self.assertEqual(self.db.stop('set'), 'v1') 328 | 329 | self.assertEqual(self.db.slen('set'), 2) 330 | self.assertFalse(self.db.srem('set', 'missing')) 331 | 332 | def test_multi_add_remove(self): 333 | res = self.db.smadd('my_set', ['v1', 'v2', 'v3', 'v4', 'v1', 'v2']) 334 | self.assertEqual(res, 6) 335 | self.assertEqual(self.db.scard('my_set'), 4) 336 | self.assertEqual(sorted(list(self.db.smembers('my_set'))), 337 | [b'v1', b'v2', b'v3', b'v4']) 338 | 339 | res = self.db.smrem('my_set', ['v1', 'v3', 'v1', 'v1']) 340 | self.assertEqual(res, 2) 341 | self.assertEqual(sorted(list(self.db.smembers('my_set'))), 342 | [b'v2', b'v4']) 343 | 344 | def test_intersection_difference(self): 345 | self.db.smadd('s1', ['v1', 'v2', 'v3']) 346 | self.db.smadd('s2', ['v2', 'v3', 'v4']) 347 | 348 | self.assertEqual( 349 | set(self.db.sinter('s1', 's2')), 350 | set([b'v2', b'v3'])) 351 | 352 | self.assertEqual(set(self.db.sdiff('s1', 's2')), set([b'v1'])) 353 | self.assertEqual(set(self.db.sdiff('s2', 's1')), set([b'v4'])) 354 | 355 | 356 | class TestListCommands(BaseVedisTestCase): 357 | def test_list_methods(self): 358 | self.db.lmpush('list', ['v1', 'v2', 'v3']) 359 | self.assertEqual(self.db.llen('list'), 3) 360 | self.assertEqual(self.db.lpop('list'), b'v1') 361 | self.assertEqual(self.db.lindex('list', 1), b'v2') 362 | self.assertEqual(self.db.lindex('list', 2), b'v3') 363 | self.assertEqual(self.db.lindex('list', 3), None) 364 | 365 | 366 | class TestMiscCommands(BaseVedisTestCase): 367 | def test_rand(self): 368 | res = self.db.rand(1, 10) 369 | self.assertTrue(1 <= res <= 10) 370 | 371 | def test_time(self): 372 | res = self.db.time().decode('utf-8') 373 | self.assertTrue(re.match('\d{2}:\d{2}', res)) 374 | 375 | def test_date(self): 376 | res = self.db.date().decode('utf-8') 377 | self.assertTrue(re.match('\d{4}-\d{2}-\d{2}', res)) 378 | 379 | def test_table_list(self): 380 | self.db['k1'] = 'v1' 381 | self.db['k2'] = 'v2' 382 | self.db.hset('hash', 'k1', 'v2') 383 | self.db.sadd('set', 'v1') 384 | tables = self.db.table_list() 385 | self.assertEqual(sorted(tables), [b'hash', b'set']) 386 | 387 | 388 | class TestTransaction(BaseVedisTestCase): 389 | def setUp(self): 390 | self.db = Vedis('test.db') 391 | 392 | def tearDown(self): 393 | try: 394 | self.db.close() 395 | finally: 396 | if os.path.exists('test.db'): 397 | os.unlink('test.db') 398 | 399 | def test_transaction(self): 400 | self.db['k1'] = 'v1' 401 | 402 | @self.db.commit_on_success 403 | def succeed(): 404 | self.db['k2'] = 'v2' 405 | 406 | @self.db.commit_on_success 407 | def fail(): 408 | self.db['k3'] = 'v3' 409 | raise Exception('uh-oh') 410 | 411 | succeed() 412 | self.assertEqual(self.db['k2'], b'v2') 413 | 414 | self.assertRaises(Exception, fail) 415 | self.assertFalse(self.db.exists('k3')) 416 | 417 | def test_base_transaction_methods(self): 418 | self.db.begin() 419 | self.db['k1'] = 'v1' 420 | self.db.rollback() 421 | self.assertFalse(self.db.exists('k1')) 422 | 423 | 424 | class TestHashObject(BaseVedisTestCase): 425 | def test_hash_object(self): 426 | h = self.db.Hash('my_hash') 427 | h['k1'] = 'v1' 428 | h['k2'] = 'v2' 429 | self.assertEqual(len(h), 2) 430 | self.assertEqual(sorted(h.keys()), [b'k1', b'k2']) 431 | self.assertEqual(sorted(h.values()), [b'v1', b'v2']) 432 | self.assertEqual(sorted(h.items()), [ 433 | (b'k1', b'v1'), 434 | (b'k2', b'v2'), 435 | ]) 436 | self.assertEqual(sorted(k for k in h), [b'k1', b'k2']) 437 | del h['k1'] 438 | self.assertFalse('k1' in h) 439 | 440 | h.update(k3='v3', k4='v4') 441 | self.assertEqual(h.to_dict(), { 442 | b'k2': b'v2', 443 | b'k3': b'v3', 444 | b'k4': b'v4', 445 | }) 446 | 447 | data = h.mget('k3', 'kx', 'k2') 448 | self.assertEqual(list(data), [b'v3', None, b'v2']) 449 | 450 | 451 | class TestSetObject(BaseVedisTestCase): 452 | def test_set_object(self): 453 | s = self.db.Set('my_set') 454 | s.add('v1', 'v2', 'v1') 455 | s.add('v2') 456 | self.assertEqual(len(s), 2) 457 | self.assertEqual(sorted([i for i in s]), [b'v1', b'v2']) 458 | 459 | self.assertEqual(s.peek(), b'v2') 460 | self.assertEqual(s.top(), b'v1') 461 | self.assertEqual(s.pop(), b'v2') 462 | self.assertEqual(len(s), 1) 463 | 464 | self.assertIn('v1', s) 465 | self.assertNotIn('v2', s) 466 | 467 | s.add('v3') 468 | s.add('v4') 469 | del s['v3'] 470 | self.assertEqual(s.to_set(), set([b'v1', b'v4'])) 471 | 472 | s2 = self.db.Set('other_set') 473 | s2.add('v1', 'v2', 'v3') 474 | 475 | self.assertEqual(s - s2, set([b'v4'])) 476 | self.assertEqual(s2 - s, set([b'v2', b'v3'])) 477 | self.assertEqual(s & s2, set([b'v1'])) 478 | self.assertEqual(s2 & s, set([b'v1'])) 479 | 480 | 481 | class TestListObject(BaseVedisTestCase): 482 | def test_list_object(self): 483 | l = self.db.List('my_list') 484 | l.extend(['v1', 'v2', 'v3']) 485 | l.append('v4') 486 | items = [item for item in l] 487 | self.assertEqual(items, [b'v1', b'v2', b'v3', b'v4']) 488 | self.assertEqual(len(l), 4) 489 | self.assertEqual(l.pop(), b'v1') 490 | self.assertEqual(l[0], None) # This is kind of odd, perhaps a bug? 491 | self.assertEqual(l[1], b'v2') 492 | self.assertEqual(l[2], b'v3') 493 | self.assertEqual(l[3], b'v4') 494 | self.assertEqual(l[4], None) 495 | 496 | self.assertEqual(list(l[0:2]), [None, b'v2']) 497 | self.assertEqual(list(l[1:8]), [b'v2', b'v3', b'v4']) 498 | self.assertEqual(list(l[3:4]), [b'v4']) 499 | self.assertEqual(list(l[:]), [None, b'v2', b'v3', b'v4']) 500 | 501 | 502 | class TestCustomCommands(BaseVedisTestCase): 503 | def test_custom_command(self): 504 | data = [] 505 | 506 | @self.db.register('XTEST') 507 | def xtest(context, *params): 508 | data.extend(params) 509 | return 'hello' 510 | 511 | res = self.db.execute('XTEST %s %s', ('foo', 'barbaz')) 512 | self.assertEqual(data, [b'foo', b'barbaz']) 513 | self.assertEqual(res, b'hello') 514 | 515 | res = self.db.execute('XTEST %s', ('single 111',)) 516 | self.assertEqual(data, [b'foo', b'barbaz', b'single 111']) 517 | self.assertEqual(res, b'hello') 518 | 519 | res = xtest('nug', 'baze') 520 | self.assertEqual(data, [b'foo', b'barbaz', b'single 111', b'nug', 521 | b'baze']) 522 | self.assertEqual(res, b'hello') 523 | 524 | self.db.delete_command('XTEST') 525 | self.assertRaises(Exception, self.db.execute, 'XTEST') 526 | 527 | def test_multi_commands(self): 528 | @self.db.register('CMDA') 529 | def cmda(context, *params): 530 | return ['aa', ['bb', ['cc', 'dd'], 'ee'], 'ff'] 531 | 532 | @self.db.register('CMDB') 533 | def cmdb(context, *params): 534 | return [param.title() for param in params] 535 | 536 | self.assertEqual(self.db.execute('CMDA'), [ 537 | b'aa', 538 | [b'bb', 539 | [b'cc', 540 | b'dd'], 541 | b'ee'], 542 | b'ff', 543 | ]) 544 | self.assertEqual( 545 | self.db.execute('CMDB %s %s %s', ('this', 'is a test', 'foo')), 546 | [b'This', b'Is A Test', b'Foo']) 547 | 548 | self.db.delete_command('CMDA') 549 | self.db.delete_command('CMDB') 550 | 551 | def test_command_context(self): 552 | @self.db.register('MAGIC_SET') 553 | def magic_set(context, *params): 554 | for param in params: 555 | context[param] = param.title() 556 | return len(params) 557 | 558 | res = self.db.execute( 559 | 'MAGIC_SET %s %s %s', 560 | ('foo', 'bar', 'this is a test')) 561 | self.assertEqual(res, 3) 562 | 563 | self.assertEqual(self.db['foo'], b'Foo') 564 | self.assertEqual(self.db['bar'], b'Bar') 565 | self.assertEqual(self.db['this is a test'], b'This Is A Test') 566 | 567 | self.db.delete_command('MAGIC_SET') 568 | 569 | def test_return_types(self): 570 | @self.db.register('TEST_RET') 571 | def test_ret(context, *params): 572 | return [ 573 | ['list'], 574 | 1337, 575 | 3.14, 576 | True, 577 | None, 578 | 'string', 579 | u'unicode', 580 | ] 581 | 582 | res = self.db.execute('TEST_RET') 583 | self.assertEqual(res, [ 584 | [b'list'], 585 | 1337, 586 | 3.14, 587 | 1, 588 | None, 589 | b'string', 590 | b'unicode', 591 | ]) 592 | 593 | self.db.delete_command('TEST_RET') 594 | 595 | 596 | if __name__ == '__main__': 597 | unittest.main(argv=sys.argv) 598 | -------------------------------------------------------------------------------- /vedis.pyx: -------------------------------------------------------------------------------- 1 | # Python library for working with Vedis databases. 2 | # _ 3 | # /.\ 4 | # Y \ 5 | # / "L 6 | # // "/ 7 | # |/ /\_================== 8 | # / / 9 | # / / 10 | # \/ 11 | # 12 | # Thanks to buaabyl for pyUnQLite, whose source-code helped me get started on 13 | # this library. 14 | from cpython.bytes cimport PyBytes_Check 15 | from cpython.unicode cimport PyUnicode_AsUTF8String 16 | from cpython.unicode cimport PyUnicode_Check 17 | from libc.stdlib cimport free, malloc 18 | 19 | import sys 20 | try: 21 | from os import fsencode 22 | except ImportError: 23 | try: 24 | from sys import getfilesystemencoding as _getfsencoding 25 | except ImportError: 26 | _fsencoding = 'utf-8' 27 | else: 28 | _fsencoding = _getfsencoding() 29 | fsencode = lambda s: s.encode(_fsencoding) 30 | 31 | 32 | cdef extern from "src/vedis.h": 33 | ctypedef struct vedis 34 | ctypedef struct vedis_kv_cursor 35 | 36 | ctypedef struct vedis_context 37 | ctypedef struct vedis_value 38 | 39 | # Simple types. 40 | ctypedef signed long long int sxi64 41 | ctypedef unsigned long long int sxu64 42 | ctypedef sxi64 vedis_int64 43 | 44 | # Database. 45 | cdef int vedis_open(vedis **ppStore, const char *zStorage) 46 | cdef int vedis_config(vedis *pStore, int iOp, ...) 47 | cdef int vedis_close(vedis *pStore) 48 | 49 | # Command execution. 50 | cdef int vedis_exec(vedis *pStore, const char *zCmd, int nLen) 51 | cdef int vedis_exec_fmt(vedis *pStore, const char *zFmt, ...) 52 | cdef int vedis_exec_result(vedis *pStore, vedis_value **ppOut) 53 | 54 | # Foreign Command Registar 55 | cdef int vedis_register_command(vedis *pStore, const char *zName, int (*xCmd)(vedis_context *,int,vedis_value **), void *pUserdata) 56 | cdef int vedis_delete_command(vedis *pStore, const char *zName) 57 | 58 | # Key/Value store. 59 | cdef int vedis_kv_store(vedis *pDb, const void *pKey, int nKeyLen, const void *pData, vedis_int64 nDataLen) 60 | cdef int vedis_kv_append(vedis *pDb, const void *pKey, int nKeyLen, const void *pData, vedis_int64 nDataLen) 61 | cdef int vedis_kv_fetch(vedis *pDb, const void *pKey, int nKeyLen, void *pBuf, vedis_int64 *pSize) 62 | cdef int vedis_kv_delete(vedis *pDb, const void *pKey, int nKeyLen) 63 | cdef int vedis_kv_config(vedis *pDb, int iOp, ...) 64 | 65 | # Transactions. 66 | cdef int vedis_begin(vedis *pDb) 67 | cdef int vedis_commit(vedis *pDb) 68 | cdef int vedis_rollback(vedis *pDb) 69 | 70 | # Misc utils. 71 | cdef int vedis_util_random_string(vedis *pDb, char *zBuf, unsigned int buf_size) 72 | cdef unsigned int vedis_util_random_num(vedis *pDb) 73 | 74 | # Call Context Key/Value Store Interfaces 75 | cdef int vedis_context_kv_store(vedis_context *pCtx,const void *pKey,int nKeyLen,const void *pData,vedis_int64 nDataLen) 76 | cdef int vedis_context_kv_append(vedis_context *pCtx,const void *pKey,int nKeyLen,const void *pData,vedis_int64 nDataLen) 77 | cdef int vedis_context_kv_store_fmt(vedis_context *pCtx,const void *pKey,int nKeyLen,const char *zFormat,...) 78 | cdef int vedis_context_kv_append_fmt(vedis_context *pCtx,const void *pKey,int nKeyLen,const char *zFormat,...) 79 | cdef int vedis_context_kv_fetch(vedis_context *pCtx,const void *pKey,int nKeyLen,void *pBuf,vedis_int64 *pBufLen) 80 | cdef int vedis_context_kv_delete(vedis_context *pCtx,const void *pKey,int nKeyLen) 81 | 82 | # Command Execution Context Interfaces 83 | cdef int vedis_context_throw_error(vedis_context *pCtx, int iErr, const char *zErr) 84 | cdef int vedis_context_throw_error_format(vedis_context *pCtx, int iErr, const char *zFormat, ...) 85 | cdef void * vedis_context_user_data(vedis_context *pCtx) 86 | 87 | # Setting The Return Value Of A Vedis Command 88 | cdef int vedis_result_int(vedis_context *pCtx, int iValue) 89 | cdef int vedis_result_int64(vedis_context *pCtx, vedis_int64 iValue) 90 | cdef int vedis_result_bool(vedis_context *pCtx, int iBool) 91 | cdef int vedis_result_double(vedis_context *pCtx, double Value) 92 | cdef int vedis_result_null(vedis_context *pCtx) 93 | cdef int vedis_result_string(vedis_context *pCtx, const char *zString, int nLen) 94 | cdef int vedis_result_string_format(vedis_context *pCtx, const char *zFormat, ...) 95 | cdef int vedis_result_value(vedis_context *pCtx, vedis_value *pValue) 96 | 97 | # Extracting Vedis Commands Parameter/Return Values 98 | cdef int vedis_value_to_int(vedis_value *pValue) 99 | cdef int vedis_value_to_bool(vedis_value *pValue) 100 | cdef vedis_int64 vedis_value_to_int64(vedis_value *pValue) 101 | cdef double vedis_value_to_double(vedis_value *pValue) 102 | cdef const char * vedis_value_to_string(vedis_value *pValue, int *pLen) 103 | 104 | # Dynamically Typed Value Object Query Interfaces 105 | cdef int vedis_value_is_int(vedis_value *pVal) 106 | cdef int vedis_value_is_float(vedis_value *pVal) 107 | cdef int vedis_value_is_bool(vedis_value *pVal) 108 | cdef int vedis_value_is_string(vedis_value *pVal) 109 | cdef int vedis_value_is_null(vedis_value *pVal) 110 | cdef int vedis_value_is_numeric(vedis_value *pVal) 111 | cdef int vedis_value_is_scalar(vedis_value *pVal) 112 | cdef int vedis_value_is_array(vedis_value *pVal) 113 | 114 | # Populating dynamically Typed Objects 115 | cdef int vedis_value_int(vedis_value *pVal, int iValue) 116 | cdef int vedis_value_int64(vedis_value *pVal, vedis_int64 iValue) 117 | cdef int vedis_value_bool(vedis_value *pVal, int iBool) 118 | cdef int vedis_value_null(vedis_value *pVal) 119 | cdef int vedis_value_double(vedis_value *pVal, double Value) 120 | cdef int vedis_value_string(vedis_value *pVal, const char *zString, int nLen) 121 | cdef int vedis_value_string_format(vedis_value *pVal, const char *zFormat, ...) 122 | cdef int vedis_value_reset_string_cursor(vedis_value *pVal) 123 | cdef int vedis_value_release(vedis_value *pVal) 124 | 125 | # On-demand Object Value Allocation 126 | cdef vedis_value * vedis_context_new_scalar(vedis_context *pCtx) 127 | cdef vedis_value * vedis_context_new_array(vedis_context *pCtx) 128 | cdef void vedis_context_release_value(vedis_context *pCtx, vedis_value *pValue) 129 | 130 | # Working with Vedis Arrays 131 | cdef vedis_value * vedis_array_fetch(vedis_value *pArray,unsigned int index) 132 | cdef int vedis_array_walk(vedis_value *pArray, int (*xWalk)(vedis_value *, void *), void *pUserData) 133 | cdef int vedis_array_insert(vedis_value *pArray,vedis_value *pValue) 134 | cdef unsigned int vedis_array_count(vedis_value *pArray) 135 | cdef int vedis_array_reset(vedis_value *pArray) 136 | cdef vedis_value * vedis_array_next_elem(vedis_value *pArray) 137 | 138 | # Library info. 139 | cdef const char * vedis_lib_version() 140 | 141 | cdef int SXRET_OK = 0 142 | cdef int SXERR_MEM = -1 143 | cdef int SXERR_IO = -2 144 | cdef int SXERR_EMPTY = -3 145 | cdef int SXERR_LOCKED = -4 146 | cdef int SXERR_ORANGE = -5 147 | cdef int SXERR_NOTFOUND = -6 148 | cdef int SXERR_LIMIT = -7 149 | cdef int SXERR_MORE = -8 150 | cdef int SXERR_INVALID = -9 151 | cdef int SXERR_ABORT = -10 152 | cdef int SXERR_EXISTS = -11 153 | cdef int SXERR_SYNTAX = -12 154 | cdef int SXERR_UNKNOWN = -13 155 | cdef int SXERR_BUSY = -14 156 | cdef int SXERR_OVERFLOW = -15 157 | cdef int SXERR_WILLBLOCK = -16 158 | cdef int SXERR_NOTIMPLEMENTED = -17 159 | cdef int SXERR_EOF = -18 160 | cdef int SXERR_PERM = -19 161 | cdef int SXERR_NOOP = -20 162 | cdef int SXERR_FORMAT = -21 163 | cdef int SXERR_NEXT = -22 164 | cdef int SXERR_OS = -23 165 | cdef int SXERR_CORRUPT = -24 166 | cdef int SXERR_CONTINUE = -25 167 | cdef int SXERR_NOMATCH = -26 168 | cdef int SXERR_RESET = -27 169 | cdef int SXERR_DONE = -28 170 | cdef int SXERR_SHORT = -29 171 | cdef int SXERR_PATH = -30 172 | cdef int SXERR_TIMEOUT = -31 173 | cdef int SXERR_BIG = -32 174 | cdef int SXERR_RETRY = -33 175 | cdef int SXERR_IGNORE = -63 176 | 177 | # Vedis return values and error codes. 178 | cdef int VEDIS_OK = SXRET_OK 179 | 180 | # Errors. 181 | cdef int VEDIS_NOMEM = SXERR_MEM # Out of memory 182 | cdef int VEDIS_ABORT = SXERR_ABORT # Another thread have released this instance 183 | cdef int VEDIS_IOERR = SXERR_IO # IO error 184 | cdef int VEDIS_CORRUPT = SXERR_CORRUPT # Corrupt pointer 185 | cdef int VEDIS_LOCKED = SXERR_LOCKED # Forbidden Operation 186 | cdef int VEDIS_BUSY = SXERR_BUSY # The database file is locked 187 | cdef int VEDIS_DONE = SXERR_DONE # Operation done 188 | cdef int VEDIS_PERM = SXERR_PERM # Permission error 189 | cdef int VEDIS_NOTIMPLEMENTED = SXERR_NOTIMPLEMENTED # Method not implemented by the underlying Key/Value storage engine 190 | cdef int VEDIS_NOTFOUND = SXERR_NOTFOUND # No such record 191 | cdef int VEDIS_NOOP = SXERR_NOOP # No such method 192 | cdef int VEDIS_INVALID = SXERR_INVALID # Invalid parameter 193 | cdef int VEDIS_EOF = SXERR_EOF # End Of Input 194 | cdef int VEDIS_UNKNOWN = SXERR_UNKNOWN # Unknown configuration option 195 | cdef int VEDIS_LIMIT = SXERR_LIMIT # Database limit reached 196 | cdef int VEDIS_EXISTS = SXERR_EXISTS # Record exists 197 | cdef int VEDIS_EMPTY = SXERR_EMPTY # Empty record 198 | cdef int VEDIS_FULL = (-73) # Full database (unlikely) 199 | cdef int VEDIS_CANTOPEN = (-74) # Unable to open the database file 200 | cdef int VEDIS_READ_ONLY = (-75) # Read only Key/Value storage engine 201 | cdef int VEDIS_LOCKERR = (-76) # Locking protocol error 202 | 203 | # Database config commands. 204 | cdef int VEDIS_CONFIG_ERR_LOG = 1 205 | cdef int VEDIS_CONFIG_MAX_PAGE_CACHE = 2 206 | cdef int VEDIS_CONFIG_KV_ENGINE = 4 207 | cdef int VEDIS_CONFIG_DISABLE_AUTO_COMMIT = 5 208 | cdef int VEDIS_CONFIG_GET_KV_NAME = 6 209 | cdef int VEDIS_CONFIG_DUP_EXEC_VALUE = 7 210 | cdef int VEDIS_CONFIG_RELEASE_DUP_VALUE = 8 211 | cdef int VEDIS_CONFIG_OUTPUT_CONSUMER = 9 212 | 213 | # Cursor seek flags. 214 | cdef int VEDIS_CURSOR_MATCH_EXACT = 1 215 | cdef int VEDIS_CURSOR_MATCH_LE = 2 216 | cdef int VEDIS_CURSOR_MATCH_GE = 3 217 | 218 | 219 | ctypedef int (*vedis_command)(vedis_context *, int, vedis_value **) noexcept 220 | 221 | 222 | cdef bint IS_PY3K = sys.version_info[0] == 3 223 | 224 | cdef inline bytes encode(obj): 225 | cdef bytes result 226 | if PyBytes_Check(obj): 227 | result = obj 228 | elif PyUnicode_Check(obj): 229 | result = PyUnicode_AsUTF8String(obj) 230 | elif obj is None: 231 | return None 232 | elif IS_PY3K: 233 | result = PyUnicode_AsUTF8String(str(obj)) 234 | else: 235 | result = bytes(obj) 236 | return result 237 | 238 | 239 | cdef class Vedis(object): 240 | """ 241 | Vedis database wrapper. 242 | """ 243 | cdef vedis *database 244 | cdef readonly bint is_memory 245 | cdef readonly bint is_open 246 | cdef readonly filename 247 | cdef readonly bytes encoded_filename 248 | cdef bint open_database 249 | 250 | def __cinit__(self): 251 | self.database = 0 252 | self.is_memory = False 253 | self.is_open = False 254 | 255 | def __dealloc__(self): 256 | if self.is_open: 257 | vedis_close(self.database) 258 | 259 | def __init__(self, filename=':mem:', open_database=True): 260 | self.filename = filename 261 | self.encoded_filename = encode(filename) 262 | self.is_memory = filename == ':mem:' 263 | self.open_database = open_database 264 | if self.open_database: 265 | self.open() 266 | 267 | cpdef open(self): 268 | """Open database connection.""" 269 | cdef int ret 270 | 271 | if self.is_open: return False 272 | 273 | self.check_call(vedis_open( 274 | &self.database, 275 | self.encoded_filename)) 276 | 277 | self.is_open = True 278 | return True 279 | 280 | cpdef close(self): 281 | """Close database connection.""" 282 | if not self.is_open: return False 283 | 284 | self.check_call(vedis_close(self.database)) 285 | self.is_open = False 286 | self.database = 0 287 | return True 288 | 289 | def __enter__(self): 290 | """Use database connection as a context manager.""" 291 | if not self.is_open: 292 | self.open() 293 | return self 294 | 295 | def __exit__(self, exc_type, exc_val, exc_tb): 296 | self.close() 297 | 298 | cpdef disable_autocommit(self): 299 | if not self.is_memory: 300 | # Disable autocommit for file-based databases. 301 | ret = vedis_config( 302 | self.database, 303 | VEDIS_CONFIG_DISABLE_AUTO_COMMIT) 304 | if ret != VEDIS_OK: 305 | raise NotImplementedError('Error disabling autocommit for ' 306 | 'in-memory database.') 307 | 308 | cpdef store(self, key, value): 309 | """Store key/value.""" 310 | cdef bytes encoded_key = encode(key), encoded_value = encode(value) 311 | self.check_call(vedis_kv_store( 312 | self.database, 313 | encoded_key, 314 | -1, 315 | encoded_value, 316 | len(encoded_value))) 317 | 318 | cpdef fetch(self, key): 319 | """Retrieve value at given key. Raises `KeyError` if key not found.""" 320 | cdef bytes encoded_key = encode(key) 321 | cdef char *buf = 0 322 | cdef vedis_int64 buf_size = 0 323 | 324 | self.check_call(vedis_kv_fetch( 325 | self.database, 326 | encoded_key, 327 | -1, 328 | 0, 329 | &buf_size)) 330 | 331 | try: 332 | buf = malloc(buf_size) 333 | self.check_call(vedis_kv_fetch( 334 | self.database, 335 | encoded_key, 336 | -1, 337 | buf, 338 | &buf_size)) 339 | value = buf[:buf_size] 340 | return value 341 | finally: 342 | free(buf) 343 | 344 | cpdef delete(self, key): 345 | """Delete the value stored at the given key.""" 346 | cdef bytes bkey = encode(key) 347 | self.check_call(vedis_kv_delete(self.database, bkey, -1)) 348 | 349 | cpdef append(self, key, value): 350 | """Append to the value stored in the given key.""" 351 | cdef bytes encoded_key = encode(key), encoded_value = encode(value) 352 | self.check_call(vedis_kv_append( 353 | self.database, 354 | encoded_key, 355 | -1, 356 | encoded_value, 357 | len(encoded_value))) 358 | 359 | cpdef exists(self, key): 360 | cdef bytes encoded_key = encode(key) 361 | cdef char *buf = 0 362 | cdef vedis_int64 buf_size = 0 363 | cdef int ret 364 | 365 | ret = vedis_kv_fetch( 366 | self.database, 367 | encoded_key, 368 | -1, 369 | 0, 370 | &buf_size) 371 | if ret == VEDIS_NOTFOUND: 372 | return False 373 | elif ret == VEDIS_OK: 374 | return True 375 | 376 | raise self._build_exception_for_error(ret) 377 | 378 | cpdef update(self, dict values): 379 | for key in values: 380 | self.store(key, values[key]) 381 | 382 | def __setitem__(self, key, value): 383 | self.store(key, value) 384 | 385 | def __getitem__(self, key): 386 | if isinstance(key, basestring): 387 | return self.fetch(key) 388 | elif isinstance(key, list): 389 | return self.mget(key) 390 | return self.fetch(str(key)) 391 | 392 | def __delitem__(self, key): 393 | self.delete(key) 394 | 395 | def __contains__(self, key): 396 | return self.exists(key) 397 | 398 | cpdef execute(self, cmd, tuple params=None, bint result=True): 399 | cdef: 400 | bytes bcmd = encode(cmd) 401 | list escaped_params 402 | 403 | if params is not None: 404 | escaped_params = [self._escape(p) for p in params] 405 | bcmd = (bcmd % tuple(escaped_params)) 406 | 407 | self.check_call(vedis_exec( 408 | self.database, 409 | bcmd, 410 | -1)) 411 | if result: 412 | return self.get_result() 413 | 414 | cpdef get_result(self): 415 | cdef vedis_value* value = 0 416 | vedis_exec_result(self.database, &value) 417 | return vedis_value_to_python(value) 418 | 419 | cdef check_call(self, int result): 420 | """ 421 | Check for a successful Vedis library call, raising an exception 422 | if the result is other than `VEDIS_OK`. 423 | """ 424 | if result != VEDIS_OK: 425 | raise self._build_exception_for_error(result) 426 | 427 | cdef _build_exception_for_error(self, int status): 428 | cdef dict exc_map 429 | 430 | exc_map = { 431 | VEDIS_NOMEM: MemoryError, 432 | VEDIS_IOERR: IOError, 433 | VEDIS_CORRUPT: IOError, 434 | VEDIS_LOCKED: IOError, 435 | VEDIS_BUSY: IOError, 436 | VEDIS_LOCKERR: IOError, 437 | VEDIS_NOTIMPLEMENTED: NotImplementedError, 438 | VEDIS_NOTFOUND: KeyError, 439 | VEDIS_NOOP: NotImplementedError, 440 | VEDIS_EOF: IOError, 441 | VEDIS_FULL: IOError, 442 | VEDIS_CANTOPEN: IOError, 443 | VEDIS_READ_ONLY: IOError, 444 | } 445 | 446 | exc_klass = exc_map.get(status, Exception) 447 | if status != VEDIS_NOTFOUND: 448 | message = self._get_last_error() 449 | return exc_klass(message) 450 | else: 451 | return exc_klass() 452 | 453 | cdef _get_last_error(self): 454 | cdef int ret 455 | cdef int size 456 | cdef char *zBuf 457 | 458 | ret = vedis_config( 459 | self.database, 460 | VEDIS_CONFIG_ERR_LOG, 461 | &zBuf, 462 | &size) 463 | if ret != VEDIS_OK or size == 0: 464 | return None 465 | 466 | return bytes(zBuf) 467 | 468 | cpdef begin(self): 469 | """Begin a new transaction. Only works for file-based databases.""" 470 | if self.is_memory: 471 | return False 472 | 473 | self.check_call(vedis_begin(self.database)) 474 | return True 475 | 476 | cpdef commit(self): 477 | """Commit current transaction. Only works for file-based databases.""" 478 | if self.is_memory: 479 | return False 480 | 481 | self.check_call(vedis_commit(self.database)) 482 | return True 483 | 484 | cpdef rollback(self): 485 | """Rollback current transaction. Only works for file-based databases.""" 486 | if self.is_memory: 487 | return False 488 | 489 | self.check_call(vedis_rollback(self.database)) 490 | return True 491 | 492 | def transaction(self): 493 | """Create context manager for wrapping a transaction.""" 494 | return Transaction(self) 495 | 496 | def commit_on_success(self, fn): 497 | def wrapper(*args, **kwargs): 498 | with self.transaction(): 499 | return fn(*args, **kwargs) 500 | return wrapper 501 | 502 | cpdef random_string(self, int nbytes): 503 | """Generate a random string of given length.""" 504 | cdef char *buf 505 | buf = malloc(nbytes * sizeof(char)) 506 | try: 507 | vedis_util_random_string(self.database, buf, nbytes) 508 | return bytes(buf[:nbytes]) 509 | finally: 510 | free(buf) 511 | 512 | cpdef int random_int(self): 513 | """Generate a random integer.""" 514 | return vedis_util_random_num(self.database) 515 | 516 | # Misc. 517 | cpdef bint copy(self, src, dest): 518 | return self.execute(b'COPY %s %s', (src, dest)) 519 | 520 | cpdef bint move(self, src, dest): 521 | return self.execute(b'MOVE %s %s', (src, dest)) 522 | 523 | cpdef int rand(self, int minimum, int maximum): 524 | return self.execute(b'RAND %d %d' % (minimum, maximum)) 525 | 526 | cpdef randstr(self, int nbytes): 527 | return self.execute(b'RANDSTR %d' % nbytes) 528 | 529 | cpdef time(self): 530 | return self.execute(b'TIME') 531 | 532 | cpdef date(self): 533 | return self.execute(b'DATE') 534 | 535 | cpdef operating_system(self): 536 | return self.execute(b'OS') 537 | 538 | cpdef strip_tags(self, html): 539 | return self.execute(b'STRIP_TAG %s', (html,)) 540 | 541 | cpdef list str_split(self, s, int nchars=1): 542 | return self.execute(b'STR_SPLIT %%s %d' % nchars, (s,)) 543 | 544 | cpdef size_format(self, int nbytes): 545 | return self.execute(b'SIZE_FMT %d' % nbytes) 546 | 547 | cpdef soundex(self, s): 548 | return self.execute(b'SOUNDEX %s', (s,)) 549 | 550 | cpdef base64(self, data): 551 | return self.execute(b'BASE64 %s', (data,)) 552 | 553 | cpdef base64_decode(self, data): 554 | return self.execute(b'BASE64_DEC %s', (data,)) 555 | 556 | cpdef list table_list(self): 557 | return self.execute(b'TABLE_LIST') 558 | 559 | # Strings. 560 | cpdef get(self, key): 561 | return self.fetch(key) 562 | 563 | cpdef set(self, key, value): 564 | return self.store(key, value) 565 | 566 | cpdef list mget(self, list keys): 567 | return self.execute(b'MGET %s' % self._flatten_list(keys)) 568 | 569 | cpdef bint mset(self, dict kw): 570 | return self.execute(b'MSET %s' % self._flatten(kw)) 571 | 572 | cpdef bint setnx(self, key, value): 573 | return self.execute(b'SETNX %s %s', (key, value)) 574 | 575 | cpdef bint msetnx(self, dict kw): 576 | return self.execute(b'MSETNX %s' % self._flatten(kw)) 577 | 578 | cpdef get_set(self, key, value): 579 | return self.execute(b'GETSET %s %s', (key, value)) 580 | 581 | cpdef int strlen(self, key): 582 | return self.execute(b'STRLEN %s', (key,)) 583 | 584 | # Counters. 585 | cpdef int incr(self, key): 586 | return self.execute(b'INCR %s', (key,)) 587 | 588 | cpdef int decr(self, key): 589 | return self.execute(b'DECR %s', (key,)) 590 | 591 | cpdef int incr_by(self, key, int amount): 592 | return self.execute(b'INCRBY %%s %d' % amount, (key,)) 593 | 594 | cpdef int decr_by(self, key, int amount): 595 | return self.execute(b'DECRBY %%s %d' % amount, (key,)) 596 | 597 | # Hash methods. 598 | cpdef bint hset(self, hash_key, key, value): 599 | return self.execute(b'HSET %s %s %s', (hash_key, key, value)) 600 | 601 | cpdef bint hsetnx(self, hash_key, key, value): 602 | return self.execute(b'HSETNX %s %s %s', (hash_key, key, value)) 603 | 604 | cpdef hget(self, hash_key, key): 605 | return self.execute(b'HGET %s %s', (hash_key, key)) 606 | 607 | cpdef int hdel(self, hash_key, key): 608 | return self.execute(b'HDEL %s %s', (hash_key, key)) 609 | 610 | cpdef int hmdel(self, hash_key, list keys): 611 | return self.execute( 612 | b'HDEL %%s %s' % self._flatten_list(keys), 613 | (hash_key,)) 614 | 615 | cpdef list hkeys(self, hash_key): 616 | return self.execute(b'HKEYS %s', (hash_key,)) 617 | 618 | cpdef list hvals(self, hash_key): 619 | return self.execute(b'HVALS %s', (hash_key,)) 620 | 621 | cpdef dict hgetall(self, hash_key): 622 | cdef list results 623 | results = self.execute(b'HGETALL %s', (hash_key,)) 624 | if results: 625 | return dict(zip(results[::2], results[1::2])) 626 | else: 627 | return {} 628 | 629 | cpdef list hitems(self, hash_key): 630 | cdef list results 631 | results = self.execute(b'HGETALL %s', (hash_key,)) 632 | if results: 633 | return list(zip(results[::2], results[1::2])) 634 | else: 635 | return [] 636 | 637 | cpdef int hlen(self, hash_key): 638 | return self.execute(b'HLEN %s', (hash_key,)) 639 | 640 | cpdef bint hexists(self, hash_key, key): 641 | return self.execute(b'HEXISTS %s %s', (hash_key, key)) 642 | 643 | cpdef int hmset(self, hash_key, dict data): 644 | return self.execute( 645 | b'HMSET %%s %s' % self._flatten(data), 646 | (hash_key,)) 647 | 648 | cpdef list hmget(self, hash_key, list keys): 649 | return self.execute( 650 | b'HMGET %%s %s' % self._flatten_list(keys), 651 | (hash_key,)) 652 | 653 | # Set methods. 654 | cpdef int sadd(self, key, value): 655 | return self.execute(b'SADD %s %s', (key, value)) 656 | 657 | cpdef int smadd(self, key, list values): 658 | return self.execute( 659 | b'SADD %%s %s' % self._flatten_list(values), 660 | (key,)) 661 | 662 | cpdef int scard(self, key): 663 | return self.execute(b'SCARD %s', (key,)) 664 | 665 | cpdef bint sismember(self, key, value): 666 | return self.execute(b'SISMEMBER %s %s', (key, value)) 667 | 668 | cpdef spop(self, key): 669 | return self.execute(b'SPOP %s', (key,)) 670 | 671 | cpdef speek(self, key): 672 | return self.execute(b'SPEEK %s', (key,)) 673 | 674 | cpdef stop(self, key): 675 | return self.execute(b'STOP %s', (key,)) 676 | 677 | cpdef bint srem(self, key, value): 678 | return self.execute(b'SREM %s %s', (key, value)) 679 | 680 | cpdef int smrem(self, key, list values): 681 | return self.execute( 682 | b'SREM %%s %s' % self._flatten_list(values), 683 | (key,)) 684 | 685 | cpdef set smembers(self, key): 686 | cdef list results 687 | results = self.execute(b'SMEMBERS %s', (key,)) 688 | return set(results) 689 | 690 | cpdef set sdiff(self, k1, k2): 691 | cdef list results 692 | results = self.execute(b'SDIFF %s %s', (k1, k2)) 693 | return set(results) 694 | 695 | cpdef set sinter(self, k1, k2): 696 | cdef list results 697 | results = self.execute(b'SINTER %s %s', (k1, k2)) 698 | return set(results) 699 | 700 | cpdef int slen(self, key): 701 | return self.execute(b'SLEN %s', (key,)) 702 | 703 | # List methods. 704 | cpdef lindex(self, key, int index): 705 | return self.execute(b'LINDEX %%s %d' % index, (key,)) 706 | 707 | cpdef int llen(self, key): 708 | return self.execute(b'LLEN %s', (key,)) 709 | 710 | cpdef lpop(self, key): 711 | return self.execute(b'LPOP %s', (key,)) 712 | 713 | cpdef int lpush(self, key, value): 714 | return self.execute(b'LPUSH %s %s', (key, value)) 715 | 716 | cpdef int lmpush(self, key, list values): 717 | return self.execute( 718 | b'LPUSH %%s %s' % self._flatten_list(values), 719 | (key,)) 720 | 721 | cpdef int lpushx(self, key, value): 722 | return self.execute(b'LPUSHX %s %s', (key, value)) 723 | 724 | cpdef int lmpushx(self, key, list values): 725 | return self.execute( 726 | b'LPUSHX %%s %s' % self._flatten_list(values), 727 | (key,)) 728 | 729 | # Internal helpers. 730 | cdef _flatten_list(self, list args): 731 | return b' '.join(self._escape(key) for key in args) 732 | 733 | cdef _flatten(self, dict kwargs): 734 | return b' '.join( 735 | b'%s %s' % (self._escape(key), self._escape(kwargs[key])) 736 | for key in kwargs) 737 | 738 | cdef bytes _escape(self, s): 739 | cdef bytes bkey = encode(s) 740 | if bkey.find(b'"') >= 0: 741 | bkey = bkey.replace(b'"', b'\\"') 742 | return b'"' + bkey + b'"' 743 | 744 | def lib_version(self): 745 | return vedis_lib_version() 746 | 747 | cpdef Hash(self, key): 748 | return Hash(self, key) 749 | 750 | cpdef Set(self, key): 751 | return Set(self, key) 752 | 753 | cpdef List(self, key): 754 | return List(self, key) 755 | 756 | def register(self, command_name): 757 | cdef bytes cmd = encode(command_name) 758 | def decorator(fn): 759 | cdef vedis_command command_callback 760 | 761 | py_command_registry[cmd] = fn 762 | command_callback = py_command_wrapper 763 | self.check_call(vedis_register_command( 764 | self.database, 765 | cmd, 766 | command_callback, 767 | cmd)) 768 | 769 | def wrapper(*args): 770 | direct_params, params = [], [] 771 | command_string = [encode(command_name)] 772 | for arg in args: 773 | if isinstance(arg, (list, tuple)): 774 | direct_params.append(self._flatten_list(arg)) 775 | command_string.append(b'%s') 776 | elif isinstance(arg, dict): 777 | direct_params.append(self._flatten(arg)) 778 | command_string.append(b'%s') 779 | else: 780 | params.append(arg) 781 | command_string.append(b'%%s') 782 | 783 | return self.execute( 784 | b' '.join(command_string) % direct_params, 785 | tuple(params)) 786 | 787 | wrapper.wrapped = fn 788 | return wrapper 789 | return decorator 790 | 791 | def delete_command(self, command_name): 792 | cdef bytes cmd_name = encode(command_name) 793 | self.check_call(vedis_delete_command( 794 | self.database, 795 | cmd_name)) 796 | 797 | 798 | cdef dict py_command_registry = {} 799 | 800 | 801 | cdef int py_command_wrapper(vedis_context *context, int nargs, vedis_value **values) noexcept: 802 | cdef int i 803 | cdef list converted = [] 804 | cdef VedisContext context_wrapper = VedisContext() 805 | cdef bytes command_name = vedis_context_user_data(context) 806 | 807 | context_wrapper.set_context(context) 808 | 809 | for i in range(nargs): 810 | converted.append(vedis_value_to_python(values[i])) 811 | 812 | try: 813 | ret = py_command_registry[command_name](context_wrapper, *converted) 814 | except: 815 | return VEDIS_ABORT 816 | else: 817 | push_result(context, ret) 818 | return VEDIS_OK 819 | 820 | 821 | cdef class VedisContext(object): 822 | cdef: 823 | vedis_context *context 824 | 825 | def __cinit__(self): 826 | self.context = NULL 827 | 828 | cdef set_context(self, vedis_context *context): 829 | self.context = context 830 | 831 | cdef vedis_value * create_value(self, value): 832 | return python_to_vedis_value(self.context, value) 833 | 834 | cdef release_value(self, vedis_value *ptr): 835 | vedis_context_release_value(self.context, ptr) 836 | 837 | cpdef store(self, key, value): 838 | """Store key/value.""" 839 | cdef bytes encoded_key = encode(key), encoded_value = encode(value) 840 | vedis_context_kv_store( 841 | self.context, 842 | encoded_key, 843 | -1, 844 | encoded_value, 845 | len(encoded_value)) 846 | 847 | cpdef fetch(self, key): 848 | """Retrieve value at given key. Raises `KeyError` if key not found.""" 849 | cdef bytes encoded_key = encode(key) 850 | cdef char *buf = 0 851 | cdef vedis_int64 buf_size = 0 852 | 853 | vedis_context_kv_fetch( 854 | self.context, 855 | encoded_key, 856 | -1, 857 | 0, 858 | &buf_size) 859 | 860 | try: 861 | buf = malloc(buf_size) 862 | vedis_context_kv_fetch( 863 | self.context, 864 | encoded_key, 865 | -1, 866 | buf, 867 | &buf_size) 868 | value = buf[:buf_size] 869 | return value 870 | finally: 871 | free(buf) 872 | 873 | cpdef delete(self, key): 874 | """Delete the value stored at the given key.""" 875 | cdef bytes ekey = encode(key) 876 | vedis_context_kv_delete( 877 | self.context, 878 | ekey, 879 | -1) 880 | 881 | cpdef append(self, key, value): 882 | """Append to the value stored in the given key.""" 883 | cdef bytes ekey = encode(key), evalue = encode(value) 884 | vedis_context_kv_append( 885 | self.context, 886 | ekey, 887 | -1, 888 | evalue, 889 | len(evalue)) 890 | 891 | cpdef exists(self, key): 892 | cdef bytes ekey = encode(key) 893 | cdef char *buf = 0 894 | cdef vedis_int64 buf_size = 0 895 | cdef int ret 896 | 897 | ret = vedis_context_kv_fetch( 898 | self.context, 899 | ekey, 900 | -1, 901 | 0, 902 | &buf_size) 903 | if ret == VEDIS_NOTFOUND: 904 | return False 905 | elif ret == VEDIS_OK: 906 | return True 907 | 908 | raise Exception() 909 | 910 | def __setitem__(self, key, value): 911 | self.store(key, value) 912 | 913 | def __getitem__(self, key): 914 | return self.fetch(key) 915 | 916 | def __delitem__(self, key): 917 | self.delete(key) 918 | 919 | def __contains__(self, key): 920 | return self.exists(key) 921 | 922 | 923 | cdef vedis_value_to_python(vedis_value *ptr): 924 | cdef int nbytes 925 | cdef list accum 926 | cdef vedis_value *item = 0 927 | 928 | if vedis_value_is_string(ptr): 929 | value = vedis_value_to_string(ptr, NULL) 930 | if value.find(b'\\"') >= 0: 931 | value = value.replace(b'\\"', b'"') 932 | return value 933 | elif vedis_value_is_array(ptr): 934 | accum = [] 935 | while True: 936 | item = vedis_array_next_elem(ptr) 937 | if not item: 938 | break 939 | accum.append(vedis_value_to_python(item)) 940 | return accum 941 | elif vedis_value_is_int(ptr): 942 | return vedis_value_to_int(ptr) 943 | elif vedis_value_is_float(ptr): 944 | return vedis_value_to_double(ptr) 945 | elif vedis_value_is_bool(ptr): 946 | return bool(vedis_value_to_bool(ptr)) 947 | elif vedis_value_is_null(ptr): 948 | return None 949 | raise TypeError('Unrecognized type.') 950 | 951 | 952 | cdef vedis_value* python_to_vedis_value(vedis_context *context, python_value): 953 | if isinstance(python_value, (list, tuple)): 954 | return create_vedis_array(context, python_value) 955 | else: 956 | return create_vedis_scalar(context, python_value) 957 | 958 | 959 | cdef vedis_value* create_vedis_scalar(vedis_context *context, python_value): 960 | cdef vedis_value *ptr = 0 961 | cdef bytes encoded_value 962 | ptr = vedis_context_new_scalar(context) 963 | if isinstance(python_value, unicode): 964 | encoded_value = encode(python_value) 965 | vedis_value_string(ptr, encoded_value, -1) 966 | elif isinstance(python_value, bytes): 967 | vedis_value_string(ptr, python_value, -1) 968 | elif isinstance(python_value, int): 969 | vedis_value_int(ptr, python_value) 970 | elif isinstance(python_value, bool): 971 | vedis_value_bool(ptr, python_value) 972 | elif isinstance(python_value, float): 973 | vedis_value_double(ptr, python_value) 974 | elif python_value is None: 975 | vedis_value_null(ptr) 976 | else: 977 | raise TypeError('Unsupported type: %s.' % type(python_value)) 978 | return ptr 979 | 980 | 981 | cdef vedis_value* create_vedis_array(vedis_context *context, list items): 982 | cdef vedis_value *ptr 983 | cdef vedis_value *list_item = 0 984 | ptr = vedis_context_new_array(context) 985 | for item in items: 986 | list_item = python_to_vedis_value(context, item) 987 | vedis_array_insert(ptr, list_item) 988 | vedis_context_release_value(context, list_item) 989 | return ptr 990 | 991 | 992 | cdef push_result(vedis_context *context, python_value): 993 | cdef bytes encoded_value 994 | if isinstance(python_value, unicode): 995 | encoded_value = python_value.encode('utf-8') 996 | vedis_result_string(context, encoded_value, -1) 997 | elif isinstance(python_value, bytes): 998 | vedis_result_string(context, python_value, -1) 999 | elif isinstance(python_value, (list, tuple)): 1000 | vedis_result_value(context, create_vedis_array(context, python_value)) 1001 | elif isinstance(python_value, int): 1002 | vedis_result_int(context, python_value) 1003 | elif isinstance(python_value, bool): 1004 | vedis_result_bool(context, python_value) 1005 | elif isinstance(python_value, float): 1006 | vedis_result_double(context, python_value) 1007 | else: 1008 | vedis_result_null(context) 1009 | 1010 | 1011 | cdef class Transaction(object): 1012 | """Expose transaction as a context manager.""" 1013 | cdef Vedis vedis 1014 | 1015 | def __init__(self, vedis): 1016 | self.vedis = vedis 1017 | 1018 | def __enter__(self): 1019 | self.vedis.begin() 1020 | return self 1021 | 1022 | def __exit__(self, exc_type, exc_val, exc_tb): 1023 | if exc_type: 1024 | self.vedis.rollback() 1025 | else: 1026 | try: 1027 | self.vedis.commit() 1028 | except: 1029 | self.vedis.rollback() 1030 | raise 1031 | 1032 | 1033 | cdef class Hash(object): 1034 | cdef Vedis vedis 1035 | cdef key 1036 | 1037 | def __init__(self, Vedis vedis, key): 1038 | self.vedis = vedis 1039 | self.key = key 1040 | 1041 | def get(self, key): 1042 | return self.vedis.hget(self.key, key) 1043 | 1044 | def mget(self, *keys): 1045 | return self.vedis.hmget(self.key, list(keys)) 1046 | 1047 | def set(self, key, value): 1048 | return self.vedis.hset(self.key, key, value) 1049 | 1050 | def delete(self, key): 1051 | self.vedis.hdel(self.key, key) 1052 | 1053 | def mdelete(self, list keys): 1054 | return self.vedis.hmdel(self.key, keys) 1055 | 1056 | def keys(self): 1057 | return self.vedis.hkeys(self.key) 1058 | 1059 | def values(self): 1060 | return self.vedis.hvals(self.key) 1061 | 1062 | def items(self): 1063 | return self.vedis.hitems(self.key) 1064 | 1065 | def update(self, **kwargs): 1066 | return self.vedis.hmset(self.key, kwargs) 1067 | 1068 | def to_dict(self): 1069 | return self.vedis.hgetall(self.key) 1070 | 1071 | def __len__(self): 1072 | return self.vedis.hlen(self.key) 1073 | 1074 | def __contains__(self, key): 1075 | return self.vedis.hexists(self.key, key) 1076 | 1077 | def __setitem__(self, key, value): 1078 | self.vedis.hset(self.key, key, value) 1079 | 1080 | def __getitem__(self, key): 1081 | return self.vedis.hget(self.key, key) 1082 | 1083 | def __delitem__(self, key): 1084 | self.vedis.hdel(self.key, key) 1085 | 1086 | def __iter__(self): 1087 | return iter(self.vedis.hkeys(self.key)) 1088 | 1089 | def __repr__(self): 1090 | return '' % self.key 1091 | 1092 | 1093 | cdef class Set(object): 1094 | cdef readonly Vedis vedis 1095 | cdef readonly key 1096 | 1097 | def __init__(self, Vedis vedis, key): 1098 | self.vedis = vedis 1099 | self.key = key 1100 | 1101 | def add(self, *values): 1102 | return self.vedis.smadd(self.key, list(values)) 1103 | 1104 | def __len__(self): 1105 | return self.vedis.scard(self.key) 1106 | 1107 | def __contains__(self, value): 1108 | return self.vedis.sismember(self.key, value) 1109 | 1110 | def pop(self): 1111 | return self.vedis.spop(self.key) 1112 | 1113 | def peek(self): 1114 | return self.vedis.speek(self.key) 1115 | 1116 | def top(self): 1117 | return self.vedis.stop(self.key) 1118 | 1119 | def remove(self, *values): 1120 | return self.vedis.smrem(self.key, list(values)) 1121 | 1122 | def __delitem__(self, key): 1123 | self.remove(key) 1124 | 1125 | def __iter__(self): 1126 | return iter(self.vedis.smembers(self.key)) 1127 | 1128 | def to_set(self): 1129 | return self.vedis.smembers(self.key) 1130 | 1131 | def __sub__(self, rhs): 1132 | return self.vedis.sdiff(self.key, rhs.key) 1133 | 1134 | def __and__(self, rhs): 1135 | return self.vedis.sinter(self.key, rhs.key) 1136 | 1137 | 1138 | __sentinel__ = object() 1139 | 1140 | 1141 | cdef class List(object): 1142 | cdef Vedis vedis 1143 | cdef key 1144 | 1145 | def __init__(self, Vedis vedis, key): 1146 | self.vedis = vedis 1147 | self.key = key 1148 | 1149 | def __getitem__(self, index): 1150 | if isinstance(index, slice): 1151 | start = index.start if index.start is not None else __sentinel__ 1152 | stop = index.stop if index.stop is not None else __sentinel__ 1153 | return self.get_range(start, stop) 1154 | return self.vedis.lindex(self.key, index) 1155 | 1156 | def __len__(self): 1157 | return self.vedis.llen(self.key) 1158 | 1159 | def pop(self): 1160 | return self.vedis.lpop(self.key) 1161 | 1162 | def append(self, value): 1163 | return self.vedis.lpush(self.key, value) 1164 | 1165 | def extend(self, values): 1166 | return self.vedis.lmpush(self.key, values) 1167 | 1168 | def __iter__(self): 1169 | def gen(): 1170 | l = len(self) 1171 | for i in range(l): 1172 | yield self[i] 1173 | return iter(gen()) 1174 | 1175 | def get_range(self, start=None, end=None): 1176 | cdef: 1177 | int n = len(self) 1178 | int s = 0 if start is __sentinel__ else start 1179 | int e = n + 1 if end is __sentinel__ else end 1180 | s = max(0, min(s or 0, n)) 1181 | e = min(e or 0, n + 1) 1182 | for i in range(s, e): 1183 | yield self[i] 1184 | --------------------------------------------------------------------------------