├── .gitignore ├── .travis.yml ├── MANIFEST.in ├── Makefile ├── README.md ├── README.rst ├── UNLICENSE ├── csiphash ├── __init__.py └── six.py ├── csiphash_build.py ├── setup.py ├── siphash24.c ├── test ├── perf_test.py └── regression_test.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | # macOS Finder files 2 | .DS_Store 3 | 4 | # Byte-compiled / optimized / DLL files 5 | __pycache__/ 6 | *.py[cod] 7 | *$py.class 8 | 9 | # C extensions 10 | *.so 11 | 12 | # Distribution / packaging 13 | .Python 14 | env/ 15 | build/ 16 | develop-eggs/ 17 | dist/ 18 | downloads/ 19 | eggs/ 20 | .eggs/ 21 | lib/ 22 | lib64/ 23 | parts/ 24 | sdist/ 25 | var/ 26 | *.egg-info/ 27 | .installed.cfg 28 | *.egg 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .coverage 44 | .coverage.* 45 | .cache 46 | nosetests.xml 47 | coverage.xml 48 | *,cover 49 | .hypothesis/ 50 | 51 | # Translations 52 | *.mo 53 | *.pot 54 | 55 | # Django stuff: 56 | *.log 57 | local_settings.py 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # IPython Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # dotenv 82 | .env 83 | 84 | # virtualenv 85 | .venv/ 86 | venv/ 87 | ENV/ 88 | 89 | # Spyder project settings 90 | .spyderproject 91 | 92 | # Rope project settings 93 | .ropeproject 94 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | 3 | env: 4 | - TOXENV=py27 5 | - TOXENV=py36 6 | 7 | install: pip install tox 8 | 9 | script: tox 10 | 11 | notifications: 12 | email: false 13 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include siphash24.c 2 | include csiphash_build.py 3 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | README.rst: README.md 2 | pandoc -f markdown -t rst README.md > README.rst 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | csiphash.py 2 | =========== 3 | 4 | [SipHash][] is a hash function and message authentication code that is secure, 5 | fast and simple. It accepts a 128-bit secret key and a variable-length message, 6 | and returns a 64-bit hash. It performs better than most cryptographic hash 7 | function-based MACs (especially for short inputs), and offers much better 8 | security than non-cryptographic hash functions, providing resistance against 9 | hash-flooding DoS attacks. As a result, it is now used as the hash function of 10 | choice for hash tables in Python, Ruby, Rust and Redis. 11 | 12 | This Python package is the SipHash-2-4 [C reference implementation][siphash-c], 13 | wrapped with [CFFI][]. It's incredibly fast: amortized hashing times tested as 14 | follows on a mid-2015 MacBook Pro: 15 | 16 | | Message Size | Time per Hash (Amortized over 1M) | 17 | | ------------ | --------------------------------- | 18 | | 43 bytes | 3.17 microseconds | 19 | | 256 bytes | 3.39 microseconds | 20 | | 1500 bytes | 4.40 microseconds | 21 | | 1 MiB | 0.74 milliseconds | 22 | 23 | [siphash]: https://131002.net/siphash/ 24 | [siphash-c]: https://github.com/veorq/SipHash 25 | [cffi]: http://cffi.readthedocs.io 26 | 27 | 28 | Installation 29 | ------------ 30 | 31 | You can get this library with pip: 32 | 33 | pip install csiphash 34 | 35 | 36 | Usage 37 | ----- 38 | 39 | Currently there's just a single function, `siphash24()`, that accepts a secret 40 | key bytestring of length 16 and an arbitrary length bytestring for the message, 41 | and returns an 8-byte digest bytestring: 42 | 43 | ```pycon 44 | >>> from csiphash import siphash24 45 | >>> siphash24(b'\x00' * 16, b'hello, world!\n') 46 | b'\xf1G4\x95\xa5\xaa\xc2b' 47 | ``` 48 | 49 | If you want to specify keys in hexadecimal, use `.decode('hex')`: 50 | 51 | ```pycon 52 | >>> siphash24('abcdef01234567899876543210fedcba'.decode('hex'), b'hello, world!\n') 53 | '\xd3\xd4N\x1dk\x1f$=' 54 | ``` 55 | 56 | If you want digests in hexadecimal, use `.encode('hex')`: 57 | 58 | ```pycon 59 | >>> siphash24(b'\x00' * 16, b'hello, world!\n').encode('hex') 60 | 'f1473495a5aac262' 61 | ``` 62 | 63 | 64 | License 65 | ------- 66 | 67 | The reference C implementation of SipHash is bundled with this library. It was 68 | written by Jean-Philippe Aumasson and Daniel J. Bernstein, and is released 69 | under the [CC0][]. 70 | 71 | [Six][] is bundled with this library. It was written by Benjamin Peterson, and 72 | is licensed under the [MIT License][]. 73 | 74 | All other software in this library is released under the [UNLICENSE][]: 75 | 76 | > This is free and unencumbered software released into the public domain. 77 | > 78 | > Anyone is free to copy, modify, publish, use, compile, sell, or 79 | > distribute this software, either in source code form or as a compiled 80 | > binary, for any purpose, commercial or non-commercial, and by any 81 | > means. 82 | > 83 | > In jurisdictions that recognize copyright laws, the author or authors 84 | > of this software dedicate any and all copyright interest in the 85 | > software to the public domain. We make this dedication for the benefit 86 | > of the public at large and to the detriment of our heirs and 87 | > successors. We intend this dedication to be an overt act of 88 | > relinquishment in perpetuity of all present and future rights to this 89 | > software under copyright law. 90 | > 91 | > THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 92 | > EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 93 | > MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 94 | > IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 95 | > OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 96 | > ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 97 | > OTHER DEALINGS IN THE SOFTWARE. 98 | > 99 | > For more information, please refer to 100 | 101 | [cc0]: https://creativecommons.org/publicdomain/zero/1.0/ 102 | [six]: https://pythonhosted.org/six/ 103 | [mit license]: https://bitbucket.org/gutworth/six/raw/ca4580a5a648fc75abc568907e81abc80b05d58c/LICENSE 104 | [unlicense]: https://unlicense.org/ 105 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | csiphash.py 2 | =========== 3 | 4 | `SipHash `__ is a hash function and message 5 | authentication code that is secure, fast and simple. It accepts a 6 | 128-bit secret key and a variable-length message, and returns a 64-bit 7 | hash. It performs better than most cryptographic hash function-based 8 | MACs (especially for short inputs), and offers much better security than 9 | non-cryptographic hash functions, providing resistance against 10 | hash-flooding DoS attacks. As a result, it is now used as the hash 11 | function of choice for hash tables in Python, Ruby, Rust and Redis. 12 | 13 | This Python package is the SipHash-2-4 `C reference 14 | implementation `__, wrapped with 15 | `CFFI `__. It's incredibly fast: amortized 16 | hashing times tested as follows on a mid-2015 MacBook Pro: 17 | 18 | +----------------+-------------------------------------+ 19 | | Message Size | Time per Hash (Amortized over 1M) | 20 | +================+=====================================+ 21 | | 43 bytes | 4.89 microseconds | 22 | +----------------+-------------------------------------+ 23 | | 256 bytes | 5.00 microseconds | 24 | +----------------+-------------------------------------+ 25 | | 1500 bytes | 6.12 microseconds | 26 | +----------------+-------------------------------------+ 27 | | 1 MiB | 0.88 milliseconds | 28 | +----------------+-------------------------------------+ 29 | 30 | Installation 31 | ------------ 32 | 33 | You can get this library with pip: 34 | 35 | :: 36 | 37 | pip install csiphash 38 | 39 | Usage 40 | ----- 41 | 42 | Currently there's just a single function, ``siphash24()``, that accepts 43 | a secret key bytestring of length 16 and an arbitrary length bytestring 44 | for the message, and returns an 8-byte digest bytestring: 45 | 46 | .. code:: pycon 47 | 48 | >>> from csiphash import siphash24 49 | >>> siphash24(b'\x00' * 16, b'hello, world!\n') 50 | b'\xf1G4\x95\xa5\xaa\xc2b' 51 | 52 | If you want to specify keys in hexadecimal, use ``.decode('hex')``: 53 | 54 | .. code:: pycon 55 | 56 | >>> siphash24('abcdef01234567899876543210fedcba'.decode('hex'), b'hello, world!\n') 57 | '\xd3\xd4N\x1dk\x1f$=' 58 | 59 | If you want digests in hexadecimal, use ``.encode('hex')``: 60 | 61 | .. code:: pycon 62 | 63 | >>> siphash24(b'\x00' * 16, b'hello, world!\n').encode('hex') 64 | 'f1473495a5aac262' 65 | 66 | License 67 | ------- 68 | 69 | The reference C implementation of SipHash is bundled with this library. 70 | It was written by Jean-Philippe Aumasson and Daniel J. Bernstein, and is 71 | released under the 72 | `CC0 `__. 73 | 74 | `Six `__ is bundled with this library. It 75 | was written by Benjamin Peterson, and is licensed under the `MIT 76 | License `__. 77 | 78 | All other software in this library is released under the 79 | `UNLICENSE `__: 80 | 81 | This is free and unencumbered software released into the public 82 | domain. 83 | 84 | Anyone is free to copy, modify, publish, use, compile, sell, or 85 | distribute this software, either in source code form or as a 86 | compiled binary, for any purpose, commercial or non-commercial, and 87 | by any means. 88 | 89 | In jurisdictions that recognize copyright laws, the author or 90 | authors of this software dedicate any and all copyright interest in 91 | the software to the public domain. We make this dedication for the 92 | benefit of the public at large and to the detriment of our heirs and 93 | successors. We intend this dedication to be an overt act of 94 | relinquishment in perpetuity of all present and future rights to 95 | this software under copyright law. 96 | 97 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 98 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 99 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 100 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY 101 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 102 | TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 103 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 104 | 105 | For more information, please refer to http://unlicense.org/ 106 | -------------------------------------------------------------------------------- /UNLICENSE: -------------------------------------------------------------------------------- 1 | This is free and unencumbered software released into the public domain. 2 | 3 | Anyone is free to copy, modify, publish, use, compile, sell, or 4 | distribute this software, either in source code form or as a compiled 5 | binary, for any purpose, commercial or non-commercial, and by any 6 | means. 7 | 8 | In jurisdictions that recognize copyright laws, the author or authors 9 | of this software dedicate any and all copyright interest in the 10 | software to the public domain. We make this dedication for the benefit 11 | of the public at large and to the detriment of our heirs and 12 | successors. We intend this dedication to be an overt act of 13 | relinquishment in perpetuity of all present and future rights to this 14 | software under copyright law. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 20 | OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 21 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 22 | OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | For more information, please refer to 25 | -------------------------------------------------------------------------------- /csiphash/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | The SipHash-2-4 C reference implementation, wrapped with CFFI. 3 | 4 | Currently doesn't support the full hashlib interface, just a function that 5 | accepts and returns byte strings. 6 | 7 | Basic usage:: 8 | 9 | >>> siphash24(b'\\x00' * 16, b'hello, world!\\n') 10 | b'\\xf1G4\\x95\\xa5\\xaa\\xc2b' 11 | 12 | If you want to specify keys in hexadecimal, use ``.decode('hex')``:: 13 | 14 | >>> siphash24('abcdef01234567899876543210fedcba'.decode('hex'), b'hello, world!\\n') 15 | '\\xd3\\xd4N\\x1dk\\x1f$=' 16 | 17 | If you want digests in hexadecimal, use ``.encode('hex')``:: 18 | 19 | >>> siphash24(b'\\x00' * 16, b'hello, world!\\n').encode('hex') 20 | 'f1473495a5aac262' 21 | """ 22 | 23 | from . import six 24 | from ._siphash import ffi, lib 25 | 26 | 27 | def siphash24(key, data): 28 | """ 29 | Apply SipHash-2-4 to a bytestring. 30 | 31 | :param key: a byte string of length 16 to use as the secret key 32 | :param data: a byte string of arbitrary length to hash 33 | :returns: the hash output, as a byte string of length 8 34 | """ 35 | if not isinstance(key, six.binary_type): 36 | raise TypeError("key must be a bytestring") 37 | if len(key) != 16: 38 | raise ValueError("key must be 16 bytes long") 39 | if not isinstance(data, six.binary_type): 40 | raise TypeError("data must be a bytestring") 41 | 42 | out_arr = ffi.new('uint8_t[8]') 43 | 44 | result = lib.siphash(out_arr, data, len(data), key) 45 | if result == 0: 46 | return b''.join(map(six.int2byte, out_arr[0:8])) 47 | raise RuntimeException("SipHash failed with error code {}".format(result)) 48 | -------------------------------------------------------------------------------- /csiphash/six.py: -------------------------------------------------------------------------------- 1 | """Utilities for writing code that runs on Python 2 and 3""" 2 | 3 | # Copyright (c) 2010-2015 Benjamin Peterson 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | from __future__ import absolute_import 24 | 25 | import functools 26 | import itertools 27 | import operator 28 | import sys 29 | import types 30 | 31 | __author__ = "Benjamin Peterson " 32 | __version__ = "1.10.0" 33 | 34 | 35 | # Useful for very coarse version differentiation. 36 | PY2 = sys.version_info[0] == 2 37 | PY3 = sys.version_info[0] == 3 38 | PY34 = sys.version_info[0:2] >= (3, 4) 39 | 40 | if PY3: 41 | string_types = str, 42 | integer_types = int, 43 | class_types = type, 44 | text_type = str 45 | binary_type = bytes 46 | 47 | MAXSIZE = sys.maxsize 48 | else: 49 | string_types = basestring, 50 | integer_types = (int, long) 51 | class_types = (type, types.ClassType) 52 | text_type = unicode 53 | binary_type = str 54 | 55 | if sys.platform.startswith("java"): 56 | # Jython always uses 32 bits. 57 | MAXSIZE = int((1 << 31) - 1) 58 | else: 59 | # It's possible to have sizeof(long) != sizeof(Py_ssize_t). 60 | class X(object): 61 | 62 | def __len__(self): 63 | return 1 << 31 64 | try: 65 | len(X()) 66 | except OverflowError: 67 | # 32-bit 68 | MAXSIZE = int((1 << 31) - 1) 69 | else: 70 | # 64-bit 71 | MAXSIZE = int((1 << 63) - 1) 72 | del X 73 | 74 | 75 | def _add_doc(func, doc): 76 | """Add documentation to a function.""" 77 | func.__doc__ = doc 78 | 79 | 80 | def _import_module(name): 81 | """Import module, returning the module after the last dot.""" 82 | __import__(name) 83 | return sys.modules[name] 84 | 85 | 86 | class _LazyDescr(object): 87 | 88 | def __init__(self, name): 89 | self.name = name 90 | 91 | def __get__(self, obj, tp): 92 | result = self._resolve() 93 | setattr(obj, self.name, result) # Invokes __set__. 94 | try: 95 | # This is a bit ugly, but it avoids running this again by 96 | # removing this descriptor. 97 | delattr(obj.__class__, self.name) 98 | except AttributeError: 99 | pass 100 | return result 101 | 102 | 103 | class MovedModule(_LazyDescr): 104 | 105 | def __init__(self, name, old, new=None): 106 | super(MovedModule, self).__init__(name) 107 | if PY3: 108 | if new is None: 109 | new = name 110 | self.mod = new 111 | else: 112 | self.mod = old 113 | 114 | def _resolve(self): 115 | return _import_module(self.mod) 116 | 117 | def __getattr__(self, attr): 118 | _module = self._resolve() 119 | value = getattr(_module, attr) 120 | setattr(self, attr, value) 121 | return value 122 | 123 | 124 | class _LazyModule(types.ModuleType): 125 | 126 | def __init__(self, name): 127 | super(_LazyModule, self).__init__(name) 128 | self.__doc__ = self.__class__.__doc__ 129 | 130 | def __dir__(self): 131 | attrs = ["__doc__", "__name__"] 132 | attrs += [attr.name for attr in self._moved_attributes] 133 | return attrs 134 | 135 | # Subclasses should override this 136 | _moved_attributes = [] 137 | 138 | 139 | class MovedAttribute(_LazyDescr): 140 | 141 | def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None): 142 | super(MovedAttribute, self).__init__(name) 143 | if PY3: 144 | if new_mod is None: 145 | new_mod = name 146 | self.mod = new_mod 147 | if new_attr is None: 148 | if old_attr is None: 149 | new_attr = name 150 | else: 151 | new_attr = old_attr 152 | self.attr = new_attr 153 | else: 154 | self.mod = old_mod 155 | if old_attr is None: 156 | old_attr = name 157 | self.attr = old_attr 158 | 159 | def _resolve(self): 160 | module = _import_module(self.mod) 161 | return getattr(module, self.attr) 162 | 163 | 164 | class _SixMetaPathImporter(object): 165 | 166 | """ 167 | A meta path importer to import six.moves and its submodules. 168 | 169 | This class implements a PEP302 finder and loader. It should be compatible 170 | with Python 2.5 and all existing versions of Python3 171 | """ 172 | 173 | def __init__(self, six_module_name): 174 | self.name = six_module_name 175 | self.known_modules = {} 176 | 177 | def _add_module(self, mod, *fullnames): 178 | for fullname in fullnames: 179 | self.known_modules[self.name + "." + fullname] = mod 180 | 181 | def _get_module(self, fullname): 182 | return self.known_modules[self.name + "." + fullname] 183 | 184 | def find_module(self, fullname, path=None): 185 | if fullname in self.known_modules: 186 | return self 187 | return None 188 | 189 | def __get_module(self, fullname): 190 | try: 191 | return self.known_modules[fullname] 192 | except KeyError: 193 | raise ImportError("This loader does not know module " + fullname) 194 | 195 | def load_module(self, fullname): 196 | try: 197 | # in case of a reload 198 | return sys.modules[fullname] 199 | except KeyError: 200 | pass 201 | mod = self.__get_module(fullname) 202 | if isinstance(mod, MovedModule): 203 | mod = mod._resolve() 204 | else: 205 | mod.__loader__ = self 206 | sys.modules[fullname] = mod 207 | return mod 208 | 209 | def is_package(self, fullname): 210 | """ 211 | Return true, if the named module is a package. 212 | 213 | We need this method to get correct spec objects with 214 | Python 3.4 (see PEP451) 215 | """ 216 | return hasattr(self.__get_module(fullname), "__path__") 217 | 218 | def get_code(self, fullname): 219 | """Return None 220 | 221 | Required, if is_package is implemented""" 222 | self.__get_module(fullname) # eventually raises ImportError 223 | return None 224 | get_source = get_code # same as get_code 225 | 226 | _importer = _SixMetaPathImporter(__name__) 227 | 228 | 229 | class _MovedItems(_LazyModule): 230 | 231 | """Lazy loading of moved objects""" 232 | __path__ = [] # mark as package 233 | 234 | 235 | _moved_attributes = [ 236 | MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"), 237 | MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"), 238 | MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"), 239 | MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"), 240 | MovedAttribute("intern", "__builtin__", "sys"), 241 | MovedAttribute("map", "itertools", "builtins", "imap", "map"), 242 | MovedAttribute("getcwd", "os", "os", "getcwdu", "getcwd"), 243 | MovedAttribute("getcwdb", "os", "os", "getcwd", "getcwdb"), 244 | MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"), 245 | MovedAttribute("reload_module", "__builtin__", "importlib" if PY34 else "imp", "reload"), 246 | MovedAttribute("reduce", "__builtin__", "functools"), 247 | MovedAttribute("shlex_quote", "pipes", "shlex", "quote"), 248 | MovedAttribute("StringIO", "StringIO", "io"), 249 | MovedAttribute("UserDict", "UserDict", "collections"), 250 | MovedAttribute("UserList", "UserList", "collections"), 251 | MovedAttribute("UserString", "UserString", "collections"), 252 | MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"), 253 | MovedAttribute("zip", "itertools", "builtins", "izip", "zip"), 254 | MovedAttribute("zip_longest", "itertools", "itertools", "izip_longest", "zip_longest"), 255 | MovedModule("builtins", "__builtin__"), 256 | MovedModule("configparser", "ConfigParser"), 257 | MovedModule("copyreg", "copy_reg"), 258 | MovedModule("dbm_gnu", "gdbm", "dbm.gnu"), 259 | MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread"), 260 | MovedModule("http_cookiejar", "cookielib", "http.cookiejar"), 261 | MovedModule("http_cookies", "Cookie", "http.cookies"), 262 | MovedModule("html_entities", "htmlentitydefs", "html.entities"), 263 | MovedModule("html_parser", "HTMLParser", "html.parser"), 264 | MovedModule("http_client", "httplib", "http.client"), 265 | MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"), 266 | MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"), 267 | MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"), 268 | MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"), 269 | MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"), 270 | MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"), 271 | MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"), 272 | MovedModule("cPickle", "cPickle", "pickle"), 273 | MovedModule("queue", "Queue"), 274 | MovedModule("reprlib", "repr"), 275 | MovedModule("socketserver", "SocketServer"), 276 | MovedModule("_thread", "thread", "_thread"), 277 | MovedModule("tkinter", "Tkinter"), 278 | MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"), 279 | MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"), 280 | MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"), 281 | MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"), 282 | MovedModule("tkinter_tix", "Tix", "tkinter.tix"), 283 | MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"), 284 | MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"), 285 | MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"), 286 | MovedModule("tkinter_colorchooser", "tkColorChooser", 287 | "tkinter.colorchooser"), 288 | MovedModule("tkinter_commondialog", "tkCommonDialog", 289 | "tkinter.commondialog"), 290 | MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"), 291 | MovedModule("tkinter_font", "tkFont", "tkinter.font"), 292 | MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"), 293 | MovedModule("tkinter_tksimpledialog", "tkSimpleDialog", 294 | "tkinter.simpledialog"), 295 | MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"), 296 | MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"), 297 | MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"), 298 | MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"), 299 | MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"), 300 | MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"), 301 | ] 302 | # Add windows specific modules. 303 | if sys.platform == "win32": 304 | _moved_attributes += [ 305 | MovedModule("winreg", "_winreg"), 306 | ] 307 | 308 | for attr in _moved_attributes: 309 | setattr(_MovedItems, attr.name, attr) 310 | if isinstance(attr, MovedModule): 311 | _importer._add_module(attr, "moves." + attr.name) 312 | del attr 313 | 314 | _MovedItems._moved_attributes = _moved_attributes 315 | 316 | moves = _MovedItems(__name__ + ".moves") 317 | _importer._add_module(moves, "moves") 318 | 319 | 320 | class Module_six_moves_urllib_parse(_LazyModule): 321 | 322 | """Lazy loading of moved objects in six.moves.urllib_parse""" 323 | 324 | 325 | _urllib_parse_moved_attributes = [ 326 | MovedAttribute("ParseResult", "urlparse", "urllib.parse"), 327 | MovedAttribute("SplitResult", "urlparse", "urllib.parse"), 328 | MovedAttribute("parse_qs", "urlparse", "urllib.parse"), 329 | MovedAttribute("parse_qsl", "urlparse", "urllib.parse"), 330 | MovedAttribute("urldefrag", "urlparse", "urllib.parse"), 331 | MovedAttribute("urljoin", "urlparse", "urllib.parse"), 332 | MovedAttribute("urlparse", "urlparse", "urllib.parse"), 333 | MovedAttribute("urlsplit", "urlparse", "urllib.parse"), 334 | MovedAttribute("urlunparse", "urlparse", "urllib.parse"), 335 | MovedAttribute("urlunsplit", "urlparse", "urllib.parse"), 336 | MovedAttribute("quote", "urllib", "urllib.parse"), 337 | MovedAttribute("quote_plus", "urllib", "urllib.parse"), 338 | MovedAttribute("unquote", "urllib", "urllib.parse"), 339 | MovedAttribute("unquote_plus", "urllib", "urllib.parse"), 340 | MovedAttribute("urlencode", "urllib", "urllib.parse"), 341 | MovedAttribute("splitquery", "urllib", "urllib.parse"), 342 | MovedAttribute("splittag", "urllib", "urllib.parse"), 343 | MovedAttribute("splituser", "urllib", "urllib.parse"), 344 | MovedAttribute("uses_fragment", "urlparse", "urllib.parse"), 345 | MovedAttribute("uses_netloc", "urlparse", "urllib.parse"), 346 | MovedAttribute("uses_params", "urlparse", "urllib.parse"), 347 | MovedAttribute("uses_query", "urlparse", "urllib.parse"), 348 | MovedAttribute("uses_relative", "urlparse", "urllib.parse"), 349 | ] 350 | for attr in _urllib_parse_moved_attributes: 351 | setattr(Module_six_moves_urllib_parse, attr.name, attr) 352 | del attr 353 | 354 | Module_six_moves_urllib_parse._moved_attributes = _urllib_parse_moved_attributes 355 | 356 | _importer._add_module(Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse"), 357 | "moves.urllib_parse", "moves.urllib.parse") 358 | 359 | 360 | class Module_six_moves_urllib_error(_LazyModule): 361 | 362 | """Lazy loading of moved objects in six.moves.urllib_error""" 363 | 364 | 365 | _urllib_error_moved_attributes = [ 366 | MovedAttribute("URLError", "urllib2", "urllib.error"), 367 | MovedAttribute("HTTPError", "urllib2", "urllib.error"), 368 | MovedAttribute("ContentTooShortError", "urllib", "urllib.error"), 369 | ] 370 | for attr in _urllib_error_moved_attributes: 371 | setattr(Module_six_moves_urllib_error, attr.name, attr) 372 | del attr 373 | 374 | Module_six_moves_urllib_error._moved_attributes = _urllib_error_moved_attributes 375 | 376 | _importer._add_module(Module_six_moves_urllib_error(__name__ + ".moves.urllib.error"), 377 | "moves.urllib_error", "moves.urllib.error") 378 | 379 | 380 | class Module_six_moves_urllib_request(_LazyModule): 381 | 382 | """Lazy loading of moved objects in six.moves.urllib_request""" 383 | 384 | 385 | _urllib_request_moved_attributes = [ 386 | MovedAttribute("urlopen", "urllib2", "urllib.request"), 387 | MovedAttribute("install_opener", "urllib2", "urllib.request"), 388 | MovedAttribute("build_opener", "urllib2", "urllib.request"), 389 | MovedAttribute("pathname2url", "urllib", "urllib.request"), 390 | MovedAttribute("url2pathname", "urllib", "urllib.request"), 391 | MovedAttribute("getproxies", "urllib", "urllib.request"), 392 | MovedAttribute("Request", "urllib2", "urllib.request"), 393 | MovedAttribute("OpenerDirector", "urllib2", "urllib.request"), 394 | MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"), 395 | MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"), 396 | MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"), 397 | MovedAttribute("ProxyHandler", "urllib2", "urllib.request"), 398 | MovedAttribute("BaseHandler", "urllib2", "urllib.request"), 399 | MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"), 400 | MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"), 401 | MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"), 402 | MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"), 403 | MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"), 404 | MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"), 405 | MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"), 406 | MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"), 407 | MovedAttribute("HTTPHandler", "urllib2", "urllib.request"), 408 | MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"), 409 | MovedAttribute("FileHandler", "urllib2", "urllib.request"), 410 | MovedAttribute("FTPHandler", "urllib2", "urllib.request"), 411 | MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"), 412 | MovedAttribute("UnknownHandler", "urllib2", "urllib.request"), 413 | MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"), 414 | MovedAttribute("urlretrieve", "urllib", "urllib.request"), 415 | MovedAttribute("urlcleanup", "urllib", "urllib.request"), 416 | MovedAttribute("URLopener", "urllib", "urllib.request"), 417 | MovedAttribute("FancyURLopener", "urllib", "urllib.request"), 418 | MovedAttribute("proxy_bypass", "urllib", "urllib.request"), 419 | ] 420 | for attr in _urllib_request_moved_attributes: 421 | setattr(Module_six_moves_urllib_request, attr.name, attr) 422 | del attr 423 | 424 | Module_six_moves_urllib_request._moved_attributes = _urllib_request_moved_attributes 425 | 426 | _importer._add_module(Module_six_moves_urllib_request(__name__ + ".moves.urllib.request"), 427 | "moves.urllib_request", "moves.urllib.request") 428 | 429 | 430 | class Module_six_moves_urllib_response(_LazyModule): 431 | 432 | """Lazy loading of moved objects in six.moves.urllib_response""" 433 | 434 | 435 | _urllib_response_moved_attributes = [ 436 | MovedAttribute("addbase", "urllib", "urllib.response"), 437 | MovedAttribute("addclosehook", "urllib", "urllib.response"), 438 | MovedAttribute("addinfo", "urllib", "urllib.response"), 439 | MovedAttribute("addinfourl", "urllib", "urllib.response"), 440 | ] 441 | for attr in _urllib_response_moved_attributes: 442 | setattr(Module_six_moves_urllib_response, attr.name, attr) 443 | del attr 444 | 445 | Module_six_moves_urllib_response._moved_attributes = _urllib_response_moved_attributes 446 | 447 | _importer._add_module(Module_six_moves_urllib_response(__name__ + ".moves.urllib.response"), 448 | "moves.urllib_response", "moves.urllib.response") 449 | 450 | 451 | class Module_six_moves_urllib_robotparser(_LazyModule): 452 | 453 | """Lazy loading of moved objects in six.moves.urllib_robotparser""" 454 | 455 | 456 | _urllib_robotparser_moved_attributes = [ 457 | MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"), 458 | ] 459 | for attr in _urllib_robotparser_moved_attributes: 460 | setattr(Module_six_moves_urllib_robotparser, attr.name, attr) 461 | del attr 462 | 463 | Module_six_moves_urllib_robotparser._moved_attributes = _urllib_robotparser_moved_attributes 464 | 465 | _importer._add_module(Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser"), 466 | "moves.urllib_robotparser", "moves.urllib.robotparser") 467 | 468 | 469 | class Module_six_moves_urllib(types.ModuleType): 470 | 471 | """Create a six.moves.urllib namespace that resembles the Python 3 namespace""" 472 | __path__ = [] # mark as package 473 | parse = _importer._get_module("moves.urllib_parse") 474 | error = _importer._get_module("moves.urllib_error") 475 | request = _importer._get_module("moves.urllib_request") 476 | response = _importer._get_module("moves.urllib_response") 477 | robotparser = _importer._get_module("moves.urllib_robotparser") 478 | 479 | def __dir__(self): 480 | return ['parse', 'error', 'request', 'response', 'robotparser'] 481 | 482 | _importer._add_module(Module_six_moves_urllib(__name__ + ".moves.urllib"), 483 | "moves.urllib") 484 | 485 | 486 | def add_move(move): 487 | """Add an item to six.moves.""" 488 | setattr(_MovedItems, move.name, move) 489 | 490 | 491 | def remove_move(name): 492 | """Remove item from six.moves.""" 493 | try: 494 | delattr(_MovedItems, name) 495 | except AttributeError: 496 | try: 497 | del moves.__dict__[name] 498 | except KeyError: 499 | raise AttributeError("no such move, %r" % (name,)) 500 | 501 | 502 | if PY3: 503 | _meth_func = "__func__" 504 | _meth_self = "__self__" 505 | 506 | _func_closure = "__closure__" 507 | _func_code = "__code__" 508 | _func_defaults = "__defaults__" 509 | _func_globals = "__globals__" 510 | else: 511 | _meth_func = "im_func" 512 | _meth_self = "im_self" 513 | 514 | _func_closure = "func_closure" 515 | _func_code = "func_code" 516 | _func_defaults = "func_defaults" 517 | _func_globals = "func_globals" 518 | 519 | 520 | try: 521 | advance_iterator = next 522 | except NameError: 523 | def advance_iterator(it): 524 | return it.next() 525 | next = advance_iterator 526 | 527 | 528 | try: 529 | callable = callable 530 | except NameError: 531 | def callable(obj): 532 | return any("__call__" in klass.__dict__ for klass in type(obj).__mro__) 533 | 534 | 535 | if PY3: 536 | def get_unbound_function(unbound): 537 | return unbound 538 | 539 | create_bound_method = types.MethodType 540 | 541 | def create_unbound_method(func, cls): 542 | return func 543 | 544 | Iterator = object 545 | else: 546 | def get_unbound_function(unbound): 547 | return unbound.im_func 548 | 549 | def create_bound_method(func, obj): 550 | return types.MethodType(func, obj, obj.__class__) 551 | 552 | def create_unbound_method(func, cls): 553 | return types.MethodType(func, None, cls) 554 | 555 | class Iterator(object): 556 | 557 | def next(self): 558 | return type(self).__next__(self) 559 | 560 | callable = callable 561 | _add_doc(get_unbound_function, 562 | """Get the function out of a possibly unbound function""") 563 | 564 | 565 | get_method_function = operator.attrgetter(_meth_func) 566 | get_method_self = operator.attrgetter(_meth_self) 567 | get_function_closure = operator.attrgetter(_func_closure) 568 | get_function_code = operator.attrgetter(_func_code) 569 | get_function_defaults = operator.attrgetter(_func_defaults) 570 | get_function_globals = operator.attrgetter(_func_globals) 571 | 572 | 573 | if PY3: 574 | def iterkeys(d, **kw): 575 | return iter(d.keys(**kw)) 576 | 577 | def itervalues(d, **kw): 578 | return iter(d.values(**kw)) 579 | 580 | def iteritems(d, **kw): 581 | return iter(d.items(**kw)) 582 | 583 | def iterlists(d, **kw): 584 | return iter(d.lists(**kw)) 585 | 586 | viewkeys = operator.methodcaller("keys") 587 | 588 | viewvalues = operator.methodcaller("values") 589 | 590 | viewitems = operator.methodcaller("items") 591 | else: 592 | def iterkeys(d, **kw): 593 | return d.iterkeys(**kw) 594 | 595 | def itervalues(d, **kw): 596 | return d.itervalues(**kw) 597 | 598 | def iteritems(d, **kw): 599 | return d.iteritems(**kw) 600 | 601 | def iterlists(d, **kw): 602 | return d.iterlists(**kw) 603 | 604 | viewkeys = operator.methodcaller("viewkeys") 605 | 606 | viewvalues = operator.methodcaller("viewvalues") 607 | 608 | viewitems = operator.methodcaller("viewitems") 609 | 610 | _add_doc(iterkeys, "Return an iterator over the keys of a dictionary.") 611 | _add_doc(itervalues, "Return an iterator over the values of a dictionary.") 612 | _add_doc(iteritems, 613 | "Return an iterator over the (key, value) pairs of a dictionary.") 614 | _add_doc(iterlists, 615 | "Return an iterator over the (key, [values]) pairs of a dictionary.") 616 | 617 | 618 | if PY3: 619 | def b(s): 620 | return s.encode("latin-1") 621 | 622 | def u(s): 623 | return s 624 | unichr = chr 625 | import struct 626 | int2byte = struct.Struct(">B").pack 627 | del struct 628 | byte2int = operator.itemgetter(0) 629 | indexbytes = operator.getitem 630 | iterbytes = iter 631 | import io 632 | StringIO = io.StringIO 633 | BytesIO = io.BytesIO 634 | _assertCountEqual = "assertCountEqual" 635 | if sys.version_info[1] <= 1: 636 | _assertRaisesRegex = "assertRaisesRegexp" 637 | _assertRegex = "assertRegexpMatches" 638 | else: 639 | _assertRaisesRegex = "assertRaisesRegex" 640 | _assertRegex = "assertRegex" 641 | else: 642 | def b(s): 643 | return s 644 | # Workaround for standalone backslash 645 | 646 | def u(s): 647 | return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape") 648 | unichr = unichr 649 | int2byte = chr 650 | 651 | def byte2int(bs): 652 | return ord(bs[0]) 653 | 654 | def indexbytes(buf, i): 655 | return ord(buf[i]) 656 | iterbytes = functools.partial(itertools.imap, ord) 657 | import StringIO 658 | StringIO = BytesIO = StringIO.StringIO 659 | _assertCountEqual = "assertItemsEqual" 660 | _assertRaisesRegex = "assertRaisesRegexp" 661 | _assertRegex = "assertRegexpMatches" 662 | _add_doc(b, """Byte literal""") 663 | _add_doc(u, """Text literal""") 664 | 665 | 666 | def assertCountEqual(self, *args, **kwargs): 667 | return getattr(self, _assertCountEqual)(*args, **kwargs) 668 | 669 | 670 | def assertRaisesRegex(self, *args, **kwargs): 671 | return getattr(self, _assertRaisesRegex)(*args, **kwargs) 672 | 673 | 674 | def assertRegex(self, *args, **kwargs): 675 | return getattr(self, _assertRegex)(*args, **kwargs) 676 | 677 | 678 | if PY3: 679 | exec_ = getattr(moves.builtins, "exec") 680 | 681 | def reraise(tp, value, tb=None): 682 | if value is None: 683 | value = tp() 684 | if value.__traceback__ is not tb: 685 | raise value.with_traceback(tb) 686 | raise value 687 | 688 | else: 689 | def exec_(_code_, _globs_=None, _locs_=None): 690 | """Execute code in a namespace.""" 691 | if _globs_ is None: 692 | frame = sys._getframe(1) 693 | _globs_ = frame.f_globals 694 | if _locs_ is None: 695 | _locs_ = frame.f_locals 696 | del frame 697 | elif _locs_ is None: 698 | _locs_ = _globs_ 699 | exec("""exec _code_ in _globs_, _locs_""") 700 | 701 | exec_("""def reraise(tp, value, tb=None): 702 | raise tp, value, tb 703 | """) 704 | 705 | 706 | if sys.version_info[:2] == (3, 2): 707 | exec_("""def raise_from(value, from_value): 708 | if from_value is None: 709 | raise value 710 | raise value from from_value 711 | """) 712 | elif sys.version_info[:2] > (3, 2): 713 | exec_("""def raise_from(value, from_value): 714 | raise value from from_value 715 | """) 716 | else: 717 | def raise_from(value, from_value): 718 | raise value 719 | 720 | 721 | print_ = getattr(moves.builtins, "print", None) 722 | if print_ is None: 723 | def print_(*args, **kwargs): 724 | """The new-style print function for Python 2.4 and 2.5.""" 725 | fp = kwargs.pop("file", sys.stdout) 726 | if fp is None: 727 | return 728 | 729 | def write(data): 730 | if not isinstance(data, basestring): 731 | data = str(data) 732 | # If the file has an encoding, encode unicode with it. 733 | if (isinstance(fp, file) and 734 | isinstance(data, unicode) and 735 | fp.encoding is not None): 736 | errors = getattr(fp, "errors", None) 737 | if errors is None: 738 | errors = "strict" 739 | data = data.encode(fp.encoding, errors) 740 | fp.write(data) 741 | want_unicode = False 742 | sep = kwargs.pop("sep", None) 743 | if sep is not None: 744 | if isinstance(sep, unicode): 745 | want_unicode = True 746 | elif not isinstance(sep, str): 747 | raise TypeError("sep must be None or a string") 748 | end = kwargs.pop("end", None) 749 | if end is not None: 750 | if isinstance(end, unicode): 751 | want_unicode = True 752 | elif not isinstance(end, str): 753 | raise TypeError("end must be None or a string") 754 | if kwargs: 755 | raise TypeError("invalid keyword arguments to print()") 756 | if not want_unicode: 757 | for arg in args: 758 | if isinstance(arg, unicode): 759 | want_unicode = True 760 | break 761 | if want_unicode: 762 | newline = unicode("\n") 763 | space = unicode(" ") 764 | else: 765 | newline = "\n" 766 | space = " " 767 | if sep is None: 768 | sep = space 769 | if end is None: 770 | end = newline 771 | for i, arg in enumerate(args): 772 | if i: 773 | write(sep) 774 | write(arg) 775 | write(end) 776 | if sys.version_info[:2] < (3, 3): 777 | _print = print_ 778 | 779 | def print_(*args, **kwargs): 780 | fp = kwargs.get("file", sys.stdout) 781 | flush = kwargs.pop("flush", False) 782 | _print(*args, **kwargs) 783 | if flush and fp is not None: 784 | fp.flush() 785 | 786 | _add_doc(reraise, """Reraise an exception.""") 787 | 788 | if sys.version_info[0:2] < (3, 4): 789 | def wraps(wrapped, assigned=functools.WRAPPER_ASSIGNMENTS, 790 | updated=functools.WRAPPER_UPDATES): 791 | def wrapper(f): 792 | f = functools.wraps(wrapped, assigned, updated)(f) 793 | f.__wrapped__ = wrapped 794 | return f 795 | return wrapper 796 | else: 797 | wraps = functools.wraps 798 | 799 | 800 | def with_metaclass(meta, *bases): 801 | """Create a base class with a metaclass.""" 802 | # This requires a bit of explanation: the basic idea is to make a dummy 803 | # metaclass for one level of class instantiation that replaces itself with 804 | # the actual metaclass. 805 | class metaclass(meta): 806 | 807 | def __new__(cls, name, this_bases, d): 808 | return meta(name, bases, d) 809 | return type.__new__(metaclass, 'temporary_class', (), {}) 810 | 811 | 812 | def add_metaclass(metaclass): 813 | """Class decorator for creating a class with a metaclass.""" 814 | def wrapper(cls): 815 | orig_vars = cls.__dict__.copy() 816 | slots = orig_vars.get('__slots__') 817 | if slots is not None: 818 | if isinstance(slots, str): 819 | slots = [slots] 820 | for slots_var in slots: 821 | orig_vars.pop(slots_var) 822 | orig_vars.pop('__dict__', None) 823 | orig_vars.pop('__weakref__', None) 824 | return metaclass(cls.__name__, cls.__bases__, orig_vars) 825 | return wrapper 826 | 827 | 828 | def python_2_unicode_compatible(klass): 829 | """ 830 | A decorator that defines __unicode__ and __str__ methods under Python 2. 831 | Under Python 3 it does nothing. 832 | 833 | To support Python 2 and 3 with a single code base, define a __str__ method 834 | returning text and apply this decorator to the class. 835 | """ 836 | if PY2: 837 | if '__str__' not in klass.__dict__: 838 | raise ValueError("@python_2_unicode_compatible cannot be applied " 839 | "to %s because it doesn't define __str__()." % 840 | klass.__name__) 841 | klass.__unicode__ = klass.__str__ 842 | klass.__str__ = lambda self: self.__unicode__().encode('utf-8') 843 | return klass 844 | 845 | 846 | # Complete the moves implementation. 847 | # This code is at the end of this module to speed up module loading. 848 | # Turn this module into a package. 849 | __path__ = [] # required for PEP 302 and PEP 451 850 | __package__ = __name__ # see PEP 366 @ReservedAssignment 851 | if globals().get("__spec__") is not None: 852 | __spec__.submodule_search_locations = [] # PEP 451 @UndefinedVariable 853 | # Remove other six meta path importers, since they cause problems. This can 854 | # happen if six is removed from sys.modules and then reloaded. (Setuptools does 855 | # this for some reason.) 856 | if sys.meta_path: 857 | for i, importer in enumerate(sys.meta_path): 858 | # Here's some real nastiness: Another "instance" of the six module might 859 | # be floating around. Therefore, we can't use isinstance() to check for 860 | # the six meta path importer, since the other six instance will have 861 | # inserted an importer with different class. 862 | if (type(importer).__name__ == "_SixMetaPathImporter" and 863 | importer.name == __name__): 864 | del sys.meta_path[i] 865 | break 866 | del i, importer 867 | # Finally, add the importer to the meta path import hook. 868 | sys.meta_path.append(_importer) 869 | -------------------------------------------------------------------------------- /csiphash_build.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | from cffi import FFI 4 | 5 | 6 | siphash_source_file = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'siphash24.c') 7 | 8 | ffibuilder = FFI() 9 | ffibuilder.cdef('''int siphash(uint8_t *out, const uint8_t *in, uint64_t inlen, const uint8_t *k);''') 10 | 11 | with open(siphash_source_file) as siphash_source: 12 | ffibuilder.set_source('csiphash._siphash', siphash_source.read()) 13 | 14 | 15 | if __name__ == '__main__': 16 | ffibuilder.compile(verbose=True) 17 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from codecs import open 2 | from os import path 3 | 4 | from setuptools import setup, find_packages 5 | 6 | 7 | here = path.abspath(path.dirname(__file__)) 8 | 9 | # Get the long description from the README file 10 | with open(path.join(here, 'README.rst'), encoding='utf-8') as f: 11 | readme = f.read() 12 | 13 | 14 | setup( 15 | name='csiphash', 16 | version='0.0.5', 17 | 18 | description='A CFFI-based implementation of SipHash24', 19 | long_description=readme, 20 | url='https://github.com/zacharyvoase/python-csiphash', 21 | 22 | author='Zachary Voase', 23 | author_email='zack@meat.io', 24 | license='UNLICENSE', 25 | 26 | packages=find_packages(exclude=['test']), 27 | 28 | setup_requires=["cffi>=1.4.0"], 29 | cffi_modules=["csiphash_build.py:ffibuilder"], 30 | 31 | install_requires=["cffi>=1.4.0"], 32 | ) 33 | -------------------------------------------------------------------------------- /siphash24.c: -------------------------------------------------------------------------------- 1 | /* 2 | SipHash reference C implementation 3 | 4 | Copyright (c) 2012-2014 Jean-Philippe Aumasson 5 | 6 | Copyright (c) 2012-2014 Daniel J. Bernstein 7 | 8 | To the extent possible under law, the author(s) have dedicated all copyright 9 | and related and neighboring rights to this software to the public domain 10 | worldwide. This software is distributed without any warranty. 11 | 12 | You should have received a copy of the CC0 Public Domain Dedication along 13 | with 14 | this software. If not, see 15 | . 16 | */ 17 | #include 18 | #include 19 | #include 20 | 21 | /* default: SipHash-2-4 */ 22 | #define cROUNDS 2 23 | #define dROUNDS 4 24 | 25 | #define ROTL(x, b) (uint64_t)(((x) << (b)) | ((x) >> (64 - (b)))) 26 | 27 | #define U32TO8_LE(p, v) \ 28 | (p)[0] = (uint8_t)((v)); \ 29 | (p)[1] = (uint8_t)((v) >> 8); \ 30 | (p)[2] = (uint8_t)((v) >> 16); \ 31 | (p)[3] = (uint8_t)((v) >> 24); 32 | 33 | #define U64TO8_LE(p, v) \ 34 | U32TO8_LE((p), (uint32_t)((v))); \ 35 | U32TO8_LE((p) + 4, (uint32_t)((v) >> 32)); 36 | 37 | #define U8TO64_LE(p) \ 38 | (((uint64_t)((p)[0])) | ((uint64_t)((p)[1]) << 8) | \ 39 | ((uint64_t)((p)[2]) << 16) | ((uint64_t)((p)[3]) << 24) | \ 40 | ((uint64_t)((p)[4]) << 32) | ((uint64_t)((p)[5]) << 40) | \ 41 | ((uint64_t)((p)[6]) << 48) | ((uint64_t)((p)[7]) << 56)) 42 | 43 | #define SIPROUND \ 44 | do { \ 45 | v0 += v1; \ 46 | v1 = ROTL(v1, 13); \ 47 | v1 ^= v0; \ 48 | v0 = ROTL(v0, 32); \ 49 | v2 += v3; \ 50 | v3 = ROTL(v3, 16); \ 51 | v3 ^= v2; \ 52 | v0 += v3; \ 53 | v3 = ROTL(v3, 21); \ 54 | v3 ^= v0; \ 55 | v2 += v1; \ 56 | v1 = ROTL(v1, 17); \ 57 | v1 ^= v2; \ 58 | v2 = ROTL(v2, 32); \ 59 | } while (0) 60 | 61 | #ifdef DEBUG 62 | #define TRACE \ 63 | do { \ 64 | printf("(%3d) v0 %08x %08x\n", (int)inlen, (uint32_t)(v0 >> 32), \ 65 | (uint32_t)v0); \ 66 | printf("(%3d) v1 %08x %08x\n", (int)inlen, (uint32_t)(v1 >> 32), \ 67 | (uint32_t)v1); \ 68 | printf("(%3d) v2 %08x %08x\n", (int)inlen, (uint32_t)(v2 >> 32), \ 69 | (uint32_t)v2); \ 70 | printf("(%3d) v3 %08x %08x\n", (int)inlen, (uint32_t)(v3 >> 32), \ 71 | (uint32_t)v3); \ 72 | } while (0) 73 | #else 74 | #define TRACE 75 | #endif 76 | 77 | int siphash(uint8_t *out, const uint8_t *in, uint64_t inlen, const uint8_t *k) { 78 | /* "somepseudorandomlygeneratedbytes" */ 79 | uint64_t v0 = 0x736f6d6570736575ULL; 80 | uint64_t v1 = 0x646f72616e646f6dULL; 81 | uint64_t v2 = 0x6c7967656e657261ULL; 82 | uint64_t v3 = 0x7465646279746573ULL; 83 | uint64_t b; 84 | uint64_t k0 = U8TO64_LE(k); 85 | uint64_t k1 = U8TO64_LE(k + 8); 86 | uint64_t m; 87 | int i; 88 | const uint8_t *end = in + inlen - (inlen % sizeof(uint64_t)); 89 | const int left = inlen & 7; 90 | b = ((uint64_t)inlen) << 56; 91 | v3 ^= k1; 92 | v2 ^= k0; 93 | v1 ^= k1; 94 | v0 ^= k0; 95 | 96 | #ifdef DOUBLE 97 | v1 ^= 0xee; 98 | #endif 99 | 100 | for (; in != end; in += 8) { 101 | m = U8TO64_LE(in); 102 | v3 ^= m; 103 | 104 | TRACE; 105 | for (i = 0; i < cROUNDS; ++i) 106 | SIPROUND; 107 | 108 | v0 ^= m; 109 | } 110 | 111 | switch (left) { 112 | case 7: 113 | b |= ((uint64_t)in[6]) << 48; 114 | case 6: 115 | b |= ((uint64_t)in[5]) << 40; 116 | case 5: 117 | b |= ((uint64_t)in[4]) << 32; 118 | case 4: 119 | b |= ((uint64_t)in[3]) << 24; 120 | case 3: 121 | b |= ((uint64_t)in[2]) << 16; 122 | case 2: 123 | b |= ((uint64_t)in[1]) << 8; 124 | case 1: 125 | b |= ((uint64_t)in[0]); 126 | break; 127 | case 0: 128 | break; 129 | } 130 | 131 | v3 ^= b; 132 | 133 | TRACE; 134 | for (i = 0; i < cROUNDS; ++i) 135 | SIPROUND; 136 | 137 | v0 ^= b; 138 | 139 | #ifndef DOUBLE 140 | v2 ^= 0xff; 141 | #else 142 | v2 ^= 0xee; 143 | #endif 144 | 145 | TRACE; 146 | for (i = 0; i < dROUNDS; ++i) 147 | SIPROUND; 148 | 149 | b = v0 ^ v1 ^ v2 ^ v3; 150 | U64TO8_LE(out, b); 151 | 152 | #ifdef DOUBLE 153 | v1 ^= 0xdd; 154 | 155 | TRACE; 156 | for (i = 0; i < dROUNDS; ++i) 157 | SIPROUND; 158 | 159 | b = v0 ^ v1 ^ v2 ^ v3; 160 | U64TO8_LE(out + 8, b); 161 | #endif 162 | 163 | return 0; 164 | } 165 | 166 | -------------------------------------------------------------------------------- /test/perf_test.py: -------------------------------------------------------------------------------- 1 | """A naive benchmark script.""" 2 | 3 | from __future__ import print_function 4 | 5 | import textwrap 6 | import timeit 7 | 8 | 9 | def time_hashing(message_size): 10 | setup = textwrap.dedent(''' 11 | import random 12 | import csiphash 13 | key = ''.join(chr(random.randint(0, 255)) for _ in xrange(16)) 14 | message = ''.join(chr(random.randint(0, 255)) for _ in xrange({message_size})) 15 | ''').format(message_size=message_size) 16 | statement = 'csiphash.siphash24(key, message)' 17 | 18 | results = [] 19 | for _ in xrange(3): 20 | results.append(timeit.timeit(setup=setup, stmt=statement, number=1000000)) 21 | return sum(results) / 3.0 22 | 23 | 24 | # From the SipHash paper (http://cr.yp.to/siphash/siphash-20120620.pdf): 25 | # 26 | # > A fair rule-of-thumb for the distribution on message-sizes on an Internet 27 | # > backbone is that roughly one-third of messages are 43 bytes (TCP ACKs), 28 | # > one-third are about 256 bytes (common PPP dialup MTU), and one-third are 1500 29 | # > bytes (common Ethernet MTU). 30 | # 31 | # We're adding a 1MiB message size too, just for fun. 32 | MESSAGE_SIZES = [43, 256, 1500, (1024 * 1024)] 33 | 34 | 35 | if __name__ == '__main__': 36 | for message_size in MESSAGE_SIZES: 37 | average_time = time_hashing(message_size) 38 | print('siphash24(char[{message_size}]) = {average_time:0.2f} microseconds'.format(message_size=message_size, average_time=average_time)) 39 | -------------------------------------------------------------------------------- /test/regression_test.py: -------------------------------------------------------------------------------- 1 | import binascii 2 | 3 | from nose.tools import assert_equal 4 | 5 | import csiphash 6 | 7 | 8 | # (key, message, digest), encoded in hexadecimal 9 | fixtures = [ 10 | (b'a46114fc4faa2f99e4773370c78f7b09', b'', b'55dbb88abae78cc8'), 11 | (b'575bf359860252db01fcced324b4c7a3', b'18810610ffcf', b'52e97b20ab061a28'), 12 | (b'951dc847682770514a681828bbe677e1', b'cf530d0f1a797ad347b8507c', b'771920c1718db3af'), 13 | (b'b33ef3e861ec092333edb52d40e5ee0f', b'f09dc89f6f2c4def48eecb36e18ce291e75f', b'97558551de0f0410'), 14 | (b'06bb2bdaa08d851d3e2fdd1e1208a057', b'ecb43a5d0e83a295cea6f18891b19c537279ced70eae51cb', b'a90f7542114dfa68'), 15 | (b'6b87a1922b0e1ccd58b6a9756bd0fa4b', b'ddd8def8ef1209f04e64ac6c4d798964940fee99f46f3c28003fb954c5e7', b'9883b7835b36518f'), 16 | (b'5407019cc19ac5d2402a9a43cad617e0', b'ff56e816abde6407cefd7834e889c9d528b72de3352b7f632c2e2c3086ce54f94b83c04b', b'856f0eb828b2f9fc'), 17 | (b'93665bc251b165013e8b6f1442c04ba2', b'4f389b436ab10f9c470f47b985c5c71acbdc71cd03c841c378444515933f316f0afd98396206b5e6e33d', b'b2cb6595c076262f'), 18 | (b'a7a23764d305107a62757c933baa1988', b'bd9edc90a5574fa325e0137ba841363be6b05b6be897191c770cda3c90e374c1f3d03d24351583f2ea52fead94f6e0e7', b'dfeb611fdf699961'), 19 | (b'87b93caa295b181fddd819d20aaab9af', b'76b8aa1c554363c6c7a8d6696597e3a790232cf404d73e5c87d5d00fa10943b0ab0ccb0281dd3838f63c6ecfa2d5ecceecb2ea7c2a2b', b'9dd8befce6334937'), 20 | (b'387be72693dcc08cd36e60089c11f4cd', b'1e70f17deb3f38662d9c4da5c41d04a0716701cc878151c4428208ee5f4ecea901bbc4c4bcd2cd84e9ae9c0c453c9294c37965589089be70c5d9a2dd', b'8a10c7cba1e90595'), 21 | (b'5cefa0cee47f904acf7fc0470bfef66a', b'7c877c45d8b68dcfe9aa34ee289d65590839c358edd307d8d5fa1470615f0617f572cf010801034d573528a24a8bba6d3b2d745948a1c1e98b0e2d5516e67cbd44f8', b'3dfa60617cb4f1e9'), 22 | (b'41dfba89bfdd8be6a5d0533cb06034d3', b'df5136878e517dc8444e6c0c6be16a46a6fc4b0752939dcc11c8c83921a9e4ea720e96af38835d8fb3019968b4ef48004470056e0f1be125cb1afe1f83ff3d27fb87b0e214d33dc2', b'5dfe48d7e37a6d70'), 23 | (b'24209d9bb90e6a5e06ccea2e9e4e8b0d', b'c6d67378a9c6f8ddfb115db4a78a2ead7500a2f2ff752f79ad24c3c5569546e002322c100e97b3936a75ae0f9c9a03c4c6ae6db4b6a7d3bc4b5cadfc69d62d2666c132baf94544ea427614fadedc', b'aef8db360cbd3279'), 24 | (b'14d8819c1d954ddcbecb575b46d0d4f6', b'7ed421f4c8413d2e7e08e12611d957796260d9e91b6b4b2361c98cbac7b6d4ff9cad22845e88e2abfea9170d2768c7ee93e7d60d884375e51049c6afbfe12ca17a853840a094e0c851c67bfb9cf31280c60bec58', b'147a65ca78ee56b3'), 25 | (b'1f5518002404d94a32270519573cb052', b'e639e46149a32b0677874a26b4caff87e2230bc6451b06511c065123b18f5f9eec5618ad6373a29fc3e3e33e22089cc991968c71dccc739f2e077196bd20289f500a1ebe241a1f0b4b897eafca7ffbb6ad5717913c1c484c3a44', b'cdd322e5f0c3a170'), 26 | (b'512876c7a7586e8cc2b91bd468fafeea', b'31f922ac5cabf490358b46d69f1d10007121ce2fd8ffe0d867347119ce85394a42897cbcb6b75afd97979a85e525898a8b804f224a3f9da4e5d2402ac8456341cb5f293da3a7ba54a9604eb4826d361e584baaaffc167ff4016754cd59f8d061', b'd434b9df4847de0c'), 27 | (b'fcad7a32d760053cd6c57d9b0f166dd8', b'79fe3512d918b6fde842672fa719a3599383c510c6ec1df7e81c70e8f26357197e71919ffa726708a465db7b9c1082f8d5f95b866447e6e333bf12f485c5246ec8a59a52f3a65b2df7b9cc7a7f89bebf0314b975465cba6fc8224df655a5af85db608717b2b3', b'f00556ff70c01f71'), 28 | (b'f92545b3a9e6b3eaed87b45e139c4318', b'eb3fcf01c3ba8989dd5694456e44696778336981d364e64059487cc3cd9ab6456db2ed1a54488c916e48f35808072ba792d39374a5036729589fe1e815ab8aeadb35cba88fa84e2dcc7bc8352c4b1be6118a590bd6ade9bf1bf13274686d56003796ec8fef3bc2179d3ea34f', b'7f1921c6c50b11ea'), 29 | (b'44533a731c68e55c782460ae129916d1', b'689b35d94527155265f81cdef8627b3e035e1b606eda8ffe9bbe8c71268a84131cd27cd66f12d66b9b1fd628641d5e4befead3ee336de73687ddac33f004ebd0062e44ddaa538246ca3cfe5d3219b1ca49d18e4cf79b5e1223dd63150bb3e5705c78d43ef4d8847a2280789a1b0e3baa298f', b'f1da9ae76bf31127'), 30 | (b'4ede46f00bd32107ee79a6ef9a9962af', b'2a5c588aba3b23c8af7925aa42d4818013336c898f615727a93991960f83287b1a904dc3c3dbcd4a6aa0e4462caa24b568b1a87dcc2bc5b39b6c23168e9a5c8c7a1cca9ec4729612971cff28e7d09355143972f6b95c756788300d4e5f05106267b761367cb9e14fe696e034b0c9e8bdbd4bf9febe2001c7', b'e23801edbb7616cc'), 31 | (b'a1a582d547dca9d16de5d35f2ef2bc57', b'8fb8eccf08ea715f987a2027c83407ac0e350c4080827d937e87f95076efa5448747b3b0d92bec3bb889a66aaacd3682b55e451f159e4541f0195cc16677bce5ab62b1d167ef935155204e469e97ce92f61c43c4a60b969fe3a501b0c2c2cf06dc19ca1b54636175d322434226448ef90e320a1a98f82afd421af40c0154', b'5c3fd3eebed6b248'), 32 | (b'8907fd00b9b55204a88c07ca755c1dda', b'851c6b8a47d487644496307433e83803941fcf66af4620f7f3485e0757260e7120f6801d72b6a855a3e05c8fb0a392531ec64923e70e1c807c6f850b5e900a9ffca03e72a0c84ae5488858e92970433eb7a7da3163ddcd0a4bd671d7a712c33c2d7d2fb8e6f680d8a4ef023fe44f174fdfb25921b55d71bb26fc1faa40958d54e70b269f', b'7ce1805b50259fde'), 33 | (b'f4b08834a2579953696366ce39275222', b'c9fd52c0398c2ba30feb20ee8fd4a811add8f6a46d66e18db8ac853d4d84933fa1dd959023fe5db76034c0cb4cb02b5fc37fa2ae12ae64d050c0014bb4e3842a397d0ef0c4df82a0287b0689591dac0fa6831e09f458d0a0b67ad1fb89093e30525c79a385633141c9f142aee9029d7d7540e064fd71e8e223097a6bc41f60233035510121a1d58a6854', b'73a012965b61344c'), 34 | (b'e1c4a82d83a46659e1b204a7f286c5a5', b'cc989ddfd01b539335971600f9e234d09f58a28238f98af69449f5d1a298759f9913d9392349ef8000434bf45acce083e593825fe62b0eb65b708f59bf1f7d9faf77fee2e9e950b5f24648c610b2c995aadaf3b556522c500435d220b33ae4b546c262b088174963505190bfe184ca3e42266abdc0cd54a828fecb856959d6e5b2d9c93958a36ce6b43dfc0c2e32c56c', b'90fbfb9d18baed02'), 35 | ] 36 | 37 | 38 | def test_expected_outputs(): 39 | for (key, message, expected) in fixtures: 40 | yield check_hash, key, message, expected 41 | 42 | 43 | def check_hash(key, message, expected): 44 | hex_hash = binascii.hexlify(csiphash.siphash24(binascii.unhexlify(key), binascii.unhexlify(message))) 45 | assert_equal(hex_hash, expected) 46 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist=py27,py36 3 | [testenv] 4 | deps=nose 5 | commands=nosetests 6 | --------------------------------------------------------------------------------