├── .gitignore ├── MANIFEST.in ├── README.md ├── UNLICENSE ├── django_postgres ├── __init__.py ├── bitstrings.py ├── citext.py ├── management │ ├── __init__.py │ └── commands │ │ ├── __init__.py │ │ └── sync_pgviews.py ├── six.py └── view.py ├── doc ├── Makefile ├── bitstrings.rst ├── conf.py ├── index.rst └── views.rst ├── setup.py └── tests ├── test_project ├── Makefile ├── README.md ├── arraytest │ ├── __init__.py │ ├── models.py │ ├── tests.py │ └── views.py ├── bitstringtest │ ├── __init__.py │ ├── models.py │ ├── tests.py │ └── views.py ├── requirements.txt ├── test_project │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py └── viewtest │ ├── __init__.py │ ├── models.py │ ├── tests.py │ └── views.py └── views ├── README.md ├── cleanup_viewtest.sh └── do_viewtest.sh /.gitignore: -------------------------------------------------------------------------------- 1 | *.py[co] 2 | 3 | # Virtualenvs for testing 4 | .venv 5 | 6 | # Temporary testing directories 7 | test__* 8 | 9 | # Packages 10 | *.egg 11 | *.egg-info 12 | dist 13 | build 14 | eggs 15 | parts 16 | bin 17 | var 18 | sdist 19 | develop-eggs 20 | .installed.cfg 21 | 22 | # Sphinx docs 23 | doc/_build 24 | 25 | # Installer logs 26 | pip-log.txt 27 | 28 | # Unit test / coverage reports 29 | .coverage 30 | .tox 31 | 32 | #Translations 33 | *.mo 34 | 35 | #Mr Developer 36 | .mr.developer.cfg 37 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zacharyvoase/django-postgres/93e9f9809cabee0b327f18d181cbc9aeab1f8f2e/MANIFEST.in -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | django-postgres 2 | =============== 3 | 4 | Adds first-class support for [PostgreSQL][] features to the Django ORM. 5 | 6 | Planned features include: 7 | 8 | * [Arrays][pg-arrays] 9 | * [Enums][pg-enums] 10 | * [Constraints][pg-constraints] 11 | * [Triggers][pg-triggers] 12 | * [Domains][pg-domains] 13 | * [Composite Types][pg-ctypes] 14 | * [Views][pg-views] 15 | 16 | [postgresql]: http://www.postgresql.org/ 17 | [pg-arrays]: http://www.postgresql.org/docs/9.1/static/arrays.html 18 | [pg-enums]: http://www.postgresql.org/docs/9.1/static/datatype-enum.html 19 | [pg-constraints]: http://www.postgresql.org/docs/9.1/static/ddl-constraints.html 20 | [pg-triggers]: http://www.postgresql.org/docs/9.1/static/sql-createtrigger.html 21 | [pg-domains]: http://www.postgresql.org/docs/9.1/static/sql-createdomain.html 22 | [pg-ctypes]: http://www.postgresql.org/docs/9.1/static/rowtypes.html 23 | [pg-views]: http://www.postgresql.org/docs/9.1/static/sql-createview.html 24 | 25 | Obviously this is quite a large project, but I think it would provide a huge 26 | amount of value to Django developers. 27 | 28 | Why? 29 | ---- 30 | 31 | PostgreSQL is an excellent data store, with a host of useful and 32 | efficiently-implemented features. Unfortunately these features are not exposed 33 | through Django's ORM, primarily because the framework has to support several 34 | SQL backends and so can only provide a set of features common to all of them. 35 | 36 | The features made available here replace some of the following practices: 37 | 38 | - Manual denormalization on `save()` (such that model saves may result in 39 | three or more separate queries). 40 | - Sequences represented by a one-to-many, with an `order` integer field. 41 | - Complex types represented by JSON in a text field. 42 | 43 | Example 44 | ------- 45 | 46 | The following represents a whirlwind tour of potential features of the project: 47 | 48 | ```python 49 | from django.db import models 50 | import django_postgres as pg 51 | 52 | 53 | USStates = pg.Enum('states_of_the_usa', ['AL', ..., 'WY']) 54 | 55 | 56 | class Address(pg.CompositeType): 57 | line1 = models.CharField(max_length=100) 58 | line2 = models.CharField(max_length=100, blank=True) 59 | city = models.CharField(max_length=100) 60 | zip_code = models.CharField(max_length=10) 61 | state = USStates() 62 | country = models.CharField(max_length=100) 63 | 64 | 65 | class USPhoneNumber(pg.Domain): 66 | data_type = models.CharField(max_length=10) 67 | constraints = [ 68 | r"VALUE ~ '^\d{3}-?\d{3}-?\d{4}$'" 69 | ] 70 | 71 | 72 | class Customer(models.Model): 73 | name = models.CharField(max_length=100) 74 | shipping_address = Address() 75 | telephone_numbers = pg.Array(USPhoneNumber()) 76 | is_preferred = models.BooleanField(default=False) 77 | 78 | 79 | class PreferredCustomer(pg.View): 80 | projection = ['myapp.Customer.*'] 81 | sql = """SELECT * FROM myapp_customer WHERE is_preferred = TRUE;""" 82 | ``` 83 | 84 | The SQL produced by this might look like: 85 | 86 | ```postgresql 87 | CREATE TYPE states_of_the_usa AS ENUM ('AL', ..., 'WY'); 88 | 89 | CREATE TYPE myapp_address AS ( 90 | line1 varchar(100), 91 | line2 varchar(100), 92 | city varchar(100), 93 | zip_code varchar(10), 94 | state states_of_the_usa, 95 | country varchar(100) 96 | ); 97 | 98 | CREATE DOMAIN myapp_usphonenumber AS varchar(10) 99 | CHECK(VALUE ~ '^\d{3}-?\d{3}-?\d{4}$'); 100 | 101 | CREATE TABLE myapp_customer ( 102 | id SERIAL PRIMARY KEY, 103 | shipping_address myapp_address, 104 | telephone_numbers myapp_usphonenumber[] 105 | ); 106 | 107 | CREATE VIEW myapp_preferredcustomer AS 108 | SELECT * FROM myapp_customer WHERE is_preferred = TRUE; 109 | ``` 110 | 111 | To create all your views, run ``python manage.py sync_pgviews`` 112 | -------------------------------------------------------------------------------- /UNLICENSE: -------------------------------------------------------------------------------- 1 | This is free and unencumbered software released into the public domain. 2 | 3 | Anyone is free to copy, modify, publish, use, compile, sell, or 4 | distribute this software, either in source code form or as a compiled 5 | binary, for any purpose, commercial or non-commercial, and by any 6 | means. 7 | 8 | In jurisdictions that recognize copyright laws, the author or authors 9 | of this software dedicate any and all copyright interest in the 10 | software to the public domain. We make this dedication for the benefit 11 | of the public at large and to the detriment of our heirs and 12 | successors. We intend this dedication to be an overt act of 13 | relinquishment in perpetuity of all present and future rights to this 14 | software under copyright law. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 20 | OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 21 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 22 | OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | For more information, please refer to 25 | -------------------------------------------------------------------------------- /django_postgres/__init__.py: -------------------------------------------------------------------------------- 1 | from django_postgres.view import View 2 | from django_postgres.bitstrings import BitStringField, BitStringExpression as B, Bits 3 | from django_postgres.citext import CaseInsensitiveTextField 4 | -------------------------------------------------------------------------------- /django_postgres/bitstrings.py: -------------------------------------------------------------------------------- 1 | from contextlib import closing 2 | 3 | from bitstring import Bits 4 | from django.db import models 5 | from django.db.backends.postgresql_psycopg2.base import DatabaseWrapper as PGDatabaseWrapper 6 | from django.db.backends.signals import connection_created 7 | from psycopg2 import extensions as ext 8 | 9 | 10 | __all__ = ['Bits', 'BitStringField', 'BitStringExpression'] 11 | 12 | 13 | def adapt_bits(bits): 14 | """psycopg2 adapter function for ``bitstring.Bits``. 15 | 16 | Encode SQL parameters from ``bitstring.Bits`` instances to SQL strings. 17 | """ 18 | if bits.length % 4 == 0: 19 | return ext.AsIs("X'%s'" % (bits.hex,)) 20 | return ext.AsIs("B'%s'" % (bits.bin,)) 21 | ext.register_adapter(Bits, adapt_bits) 22 | 23 | 24 | def cast_bits(value, cur): 25 | """psycopg2 caster for bit strings. 26 | 27 | Turns query results from the database into ``bitstring.Bits`` instances. 28 | """ 29 | if value is None: 30 | return None 31 | return Bits(bin=value) 32 | 33 | 34 | def register_bitstring_types(connection): 35 | """Register the BIT and VARBIT casters on the provided connection. 36 | 37 | This ensures that BIT and VARBIT instances returned from the database will 38 | be represented in Python as ``bitstring.Bits`` instances. 39 | """ 40 | with closing(connection.cursor()) as cur: 41 | cur.execute("SELECT NULL::BIT") 42 | bit_oid = cur.description[0].type_code 43 | cur.execute("SELECT NULL::VARBIT") 44 | varbit_oid = cur.description[0].type_code 45 | bit_caster = ext.new_type((bit_oid, varbit_oid), 'BIT', cast_bits) 46 | ext.register_type(bit_caster, connection) 47 | 48 | 49 | def register_types_on_connection_creation(connection, sender, *args, **kwargs): 50 | if not issubclass(sender, PGDatabaseWrapper): 51 | return 52 | register_bitstring_types(connection.connection) 53 | connection_created.connect(register_types_on_connection_creation) 54 | 55 | 56 | class BitStringField(models.Field): 57 | 58 | """A Postgres bit string.""" 59 | 60 | def __init__(self, *args, **kwargs): 61 | self.max_length = kwargs.setdefault('max_length', 1) 62 | self.varying = kwargs.pop('varying', False) 63 | 64 | if 'default' in kwargs: 65 | default = kwargs.pop('default') 66 | elif kwargs.get('null', False): 67 | default = None 68 | elif self.max_length is not None and not self.varying: 69 | default = '0' * self.max_length 70 | else: 71 | default = '0' 72 | kwargs['default'] = self.to_python(default) 73 | 74 | super(BitStringField, self).__init__(*args, **kwargs) 75 | 76 | def db_type(self, connection): 77 | if self.varying: 78 | if self.max_length is not None: 79 | return 'VARBIT(%d)' % (self.max_length,) 80 | return 'VARBIT' 81 | elif self.max_length is not None: 82 | return 'BIT(%d)' % (self.max_length,) 83 | return 'BIT' 84 | 85 | def to_python(self, value): 86 | if value is None or isinstance(value, Bits): 87 | return value 88 | elif isinstance(value, basestring): 89 | if value.startswith('0x'): 90 | return Bits(hex=value) 91 | return Bits(bin=value) 92 | raise TypeError("Cannot coerce into bit string: %r" % (value,)) 93 | 94 | def get_prep_value(self, value): 95 | return self.to_python(value) 96 | 97 | def get_prep_lookup(self, lookup_type, value): 98 | if lookup_type == 'exact': 99 | return self.get_prep_value(value) 100 | elif lookup_type == 'in': 101 | return map(self.get_prep_value, value) 102 | raise TypeError("Lookup type %r not supported on bit strings" % lookup_type) 103 | 104 | def get_default(self): 105 | default = super(BitStringField, self).get_default() 106 | return self.to_python(default) 107 | 108 | 109 | class BitStringExpression(models.expressions.F): 110 | 111 | ADD = '||' # The Postgres concatenation operator. 112 | XOR = '#' 113 | LSHIFT = '<<' 114 | RSHIFT = '>>' 115 | NOT = '~' 116 | 117 | def __init__(self, field, *args, **kwargs): 118 | super(BitStringExpression, self).__init__(field, *args, **kwargs) 119 | self.lookup = field 120 | 121 | def __and__(self, other): 122 | return self.bitand(other) 123 | 124 | def __or__(self, other): 125 | return self.bitor(other) 126 | 127 | def __xor__(self, other): 128 | return self._combine(other, self.XOR, False) 129 | 130 | def __rxor__(self, other): 131 | return self._combine(other, self.XOR, True) 132 | 133 | def __lshift__(self, other): 134 | return self._combine(other, self.LSHIFT, False) 135 | 136 | def __rshift__(self, other): 137 | return self._combine(other, self.RSHIFT, False) 138 | 139 | def _unary(self, operator): 140 | # This is a total hack, but you need to combine a raw empty space with 141 | # the current node, in reverse order, with the connector being the 142 | # unary operator you want to apply. 143 | return self._combine(ext.AsIs(''), operator, True) 144 | 145 | def __invert__(self): 146 | return self._unary(self.NOT) 147 | -------------------------------------------------------------------------------- /django_postgres/citext.py: -------------------------------------------------------------------------------- 1 | from django.db.models import fields 2 | 3 | 4 | class CaseInsensitiveTextField(fields.TextField): 5 | 6 | def db_type(self, connection): 7 | return "citext" 8 | -------------------------------------------------------------------------------- /django_postgres/management/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zacharyvoase/django-postgres/93e9f9809cabee0b327f18d181cbc9aeab1f8f2e/django_postgres/management/__init__.py -------------------------------------------------------------------------------- /django_postgres/management/commands/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zacharyvoase/django-postgres/93e9f9809cabee0b327f18d181cbc9aeab1f8f2e/django_postgres/management/commands/__init__.py -------------------------------------------------------------------------------- /django_postgres/management/commands/sync_pgviews.py: -------------------------------------------------------------------------------- 1 | from optparse import make_option 2 | import logging 3 | 4 | from django.core.management.base import NoArgsCommand 5 | from django.db import models 6 | 7 | from django_postgres.view import create_views 8 | 9 | 10 | log = logging.getLogger('django_postgres.sync_pgviews') 11 | 12 | 13 | class Command(NoArgsCommand): 14 | help = """Create/update Postgres views for all installed apps.""" 15 | option_list = NoArgsCommand.option_list + ( 16 | make_option('--no-update', 17 | action='store_false', 18 | dest='update', 19 | default=True, 20 | help="""Don't update existing views, only create new ones."""), 21 | make_option('--force', 22 | action='store_true', 23 | dest='force', 24 | default=False, 25 | help="""Force replacement of pre-existing views where 26 | breaking changes have been made to the schema."""), 27 | ) 28 | 29 | def handle_noargs(self, force, update, **options): 30 | for module in models.get_apps(): 31 | log.info("Creating views for %s", module.__name__) 32 | try: 33 | for status, view_cls, python_name in create_views(module, update=update, force=force): 34 | if status == 'CREATED': 35 | msg = "created" 36 | elif status == 'UPDATED': 37 | msg = "updated" 38 | elif status == 'EXISTS': 39 | msg = "already exists, skipping" 40 | elif status == 'FORCED': 41 | msg = "forced overwrite of existing schema" 42 | elif status == 'FORCE_REQUIRED': 43 | msg = "exists with incompatible schema, --force required to update" 44 | log.info("%(python_name)s (%(view_name)s): %(msg)s" % { 45 | 'python_name': python_name, 46 | 'view_name': view_cls._meta.db_table, 47 | 'msg': msg}) 48 | except Exception, exc: 49 | if not hasattr(exc, 'view_cls'): 50 | raise 51 | log.exception("Error creating view %s (%r)", 52 | exc.python_name, 53 | exc.view_cls._meta.db_table) 54 | -------------------------------------------------------------------------------- /django_postgres/six.py: -------------------------------------------------------------------------------- 1 | """Utilities for writing code that runs on Python 2 and 3""" 2 | 3 | # Copyright (c) 2010-2013 Benjamin Peterson 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | 23 | import operator 24 | import sys 25 | import types 26 | 27 | __author__ = "Benjamin Peterson " 28 | __version__ = "1.3.0" 29 | 30 | 31 | # True if we are running on Python 3. 32 | PY3 = sys.version_info[0] == 3 33 | 34 | if PY3: 35 | string_types = str, 36 | integer_types = int, 37 | class_types = type, 38 | text_type = str 39 | binary_type = bytes 40 | 41 | MAXSIZE = sys.maxsize 42 | else: 43 | string_types = basestring, 44 | integer_types = (int, long) 45 | class_types = (type, types.ClassType) 46 | text_type = unicode 47 | binary_type = str 48 | 49 | if sys.platform.startswith("java"): 50 | # Jython always uses 32 bits. 51 | MAXSIZE = int((1 << 31) - 1) 52 | else: 53 | # It's possible to have sizeof(long) != sizeof(Py_ssize_t). 54 | class X(object): 55 | def __len__(self): 56 | return 1 << 31 57 | try: 58 | len(X()) 59 | except OverflowError: 60 | # 32-bit 61 | MAXSIZE = int((1 << 31) - 1) 62 | else: 63 | # 64-bit 64 | MAXSIZE = int((1 << 63) - 1) 65 | del X 66 | 67 | 68 | def _add_doc(func, doc): 69 | """Add documentation to a function.""" 70 | func.__doc__ = doc 71 | 72 | 73 | def _import_module(name): 74 | """Import module, returning the module after the last dot.""" 75 | __import__(name) 76 | return sys.modules[name] 77 | 78 | 79 | class _LazyDescr(object): 80 | 81 | def __init__(self, name): 82 | self.name = name 83 | 84 | def __get__(self, obj, tp): 85 | result = self._resolve() 86 | setattr(obj, self.name, result) 87 | # This is a bit ugly, but it avoids running this again. 88 | delattr(tp, self.name) 89 | return result 90 | 91 | 92 | class MovedModule(_LazyDescr): 93 | 94 | def __init__(self, name, old, new=None): 95 | super(MovedModule, self).__init__(name) 96 | if PY3: 97 | if new is None: 98 | new = name 99 | self.mod = new 100 | else: 101 | self.mod = old 102 | 103 | def _resolve(self): 104 | return _import_module(self.mod) 105 | 106 | 107 | class MovedAttribute(_LazyDescr): 108 | 109 | def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None): 110 | super(MovedAttribute, self).__init__(name) 111 | if PY3: 112 | if new_mod is None: 113 | new_mod = name 114 | self.mod = new_mod 115 | if new_attr is None: 116 | if old_attr is None: 117 | new_attr = name 118 | else: 119 | new_attr = old_attr 120 | self.attr = new_attr 121 | else: 122 | self.mod = old_mod 123 | if old_attr is None: 124 | old_attr = name 125 | self.attr = old_attr 126 | 127 | def _resolve(self): 128 | module = _import_module(self.mod) 129 | return getattr(module, self.attr) 130 | 131 | 132 | 133 | class _MovedItems(types.ModuleType): 134 | """Lazy loading of moved objects""" 135 | 136 | 137 | _moved_attributes = [ 138 | MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"), 139 | MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"), 140 | MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"), 141 | MovedAttribute("map", "itertools", "builtins", "imap", "map"), 142 | MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"), 143 | MovedAttribute("reload_module", "__builtin__", "imp", "reload"), 144 | MovedAttribute("reduce", "__builtin__", "functools"), 145 | MovedAttribute("StringIO", "StringIO", "io"), 146 | MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"), 147 | MovedAttribute("zip", "itertools", "builtins", "izip", "zip"), 148 | 149 | MovedModule("builtins", "__builtin__"), 150 | MovedModule("configparser", "ConfigParser"), 151 | MovedModule("copyreg", "copy_reg"), 152 | MovedModule("http_cookiejar", "cookielib", "http.cookiejar"), 153 | MovedModule("http_cookies", "Cookie", "http.cookies"), 154 | MovedModule("html_entities", "htmlentitydefs", "html.entities"), 155 | MovedModule("html_parser", "HTMLParser", "html.parser"), 156 | MovedModule("http_client", "httplib", "http.client"), 157 | MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"), 158 | MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"), 159 | MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"), 160 | MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"), 161 | MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"), 162 | MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"), 163 | MovedModule("cPickle", "cPickle", "pickle"), 164 | MovedModule("queue", "Queue"), 165 | MovedModule("reprlib", "repr"), 166 | MovedModule("socketserver", "SocketServer"), 167 | MovedModule("tkinter", "Tkinter"), 168 | MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"), 169 | MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"), 170 | MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"), 171 | MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"), 172 | MovedModule("tkinter_tix", "Tix", "tkinter.tix"), 173 | MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"), 174 | MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"), 175 | MovedModule("tkinter_colorchooser", "tkColorChooser", 176 | "tkinter.colorchooser"), 177 | MovedModule("tkinter_commondialog", "tkCommonDialog", 178 | "tkinter.commondialog"), 179 | MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"), 180 | MovedModule("tkinter_font", "tkFont", "tkinter.font"), 181 | MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"), 182 | MovedModule("tkinter_tksimpledialog", "tkSimpleDialog", 183 | "tkinter.simpledialog"), 184 | MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"), 185 | MovedModule("winreg", "_winreg"), 186 | ] 187 | for attr in _moved_attributes: 188 | setattr(_MovedItems, attr.name, attr) 189 | del attr 190 | 191 | moves = sys.modules[__name__ + ".moves"] = _MovedItems("moves") 192 | 193 | 194 | def add_move(move): 195 | """Add an item to six.moves.""" 196 | setattr(_MovedItems, move.name, move) 197 | 198 | 199 | def remove_move(name): 200 | """Remove item from six.moves.""" 201 | try: 202 | delattr(_MovedItems, name) 203 | except AttributeError: 204 | try: 205 | del moves.__dict__[name] 206 | except KeyError: 207 | raise AttributeError("no such move, %r" % (name,)) 208 | 209 | 210 | if PY3: 211 | _meth_func = "__func__" 212 | _meth_self = "__self__" 213 | 214 | _func_closure = "__closure__" 215 | _func_code = "__code__" 216 | _func_defaults = "__defaults__" 217 | _func_globals = "__globals__" 218 | 219 | _iterkeys = "keys" 220 | _itervalues = "values" 221 | _iteritems = "items" 222 | _iterlists = "lists" 223 | else: 224 | _meth_func = "im_func" 225 | _meth_self = "im_self" 226 | 227 | _func_closure = "func_closure" 228 | _func_code = "func_code" 229 | _func_defaults = "func_defaults" 230 | _func_globals = "func_globals" 231 | 232 | _iterkeys = "iterkeys" 233 | _itervalues = "itervalues" 234 | _iteritems = "iteritems" 235 | _iterlists = "iterlists" 236 | 237 | 238 | try: 239 | advance_iterator = next 240 | except NameError: 241 | def advance_iterator(it): 242 | return it.next() 243 | next = advance_iterator 244 | 245 | 246 | try: 247 | callable = callable 248 | except NameError: 249 | def callable(obj): 250 | return any("__call__" in klass.__dict__ for klass in type(obj).__mro__) 251 | 252 | 253 | if PY3: 254 | def get_unbound_function(unbound): 255 | return unbound 256 | 257 | create_bound_method = types.MethodType 258 | 259 | Iterator = object 260 | else: 261 | def get_unbound_function(unbound): 262 | return unbound.im_func 263 | 264 | def create_bound_method(func, obj): 265 | return types.MethodType(func, obj, obj.__class__) 266 | 267 | class Iterator(object): 268 | 269 | def next(self): 270 | return type(self).__next__(self) 271 | 272 | callable = callable 273 | _add_doc(get_unbound_function, 274 | """Get the function out of a possibly unbound function""") 275 | 276 | 277 | get_method_function = operator.attrgetter(_meth_func) 278 | get_method_self = operator.attrgetter(_meth_self) 279 | get_function_closure = operator.attrgetter(_func_closure) 280 | get_function_code = operator.attrgetter(_func_code) 281 | get_function_defaults = operator.attrgetter(_func_defaults) 282 | get_function_globals = operator.attrgetter(_func_globals) 283 | 284 | 285 | def iterkeys(d, **kw): 286 | """Return an iterator over the keys of a dictionary.""" 287 | return iter(getattr(d, _iterkeys)(**kw)) 288 | 289 | def itervalues(d, **kw): 290 | """Return an iterator over the values of a dictionary.""" 291 | return iter(getattr(d, _itervalues)(**kw)) 292 | 293 | def iteritems(d, **kw): 294 | """Return an iterator over the (key, value) pairs of a dictionary.""" 295 | return iter(getattr(d, _iteritems)(**kw)) 296 | 297 | def iterlists(d, **kw): 298 | """Return an iterator over the (key, [values]) pairs of a dictionary.""" 299 | return iter(getattr(d, _iterlists)(**kw)) 300 | 301 | 302 | if PY3: 303 | def b(s): 304 | return s.encode("latin-1") 305 | def u(s): 306 | return s 307 | if sys.version_info[1] <= 1: 308 | def int2byte(i): 309 | return bytes((i,)) 310 | else: 311 | # This is about 2x faster than the implementation above on 3.2+ 312 | int2byte = operator.methodcaller("to_bytes", 1, "big") 313 | indexbytes = operator.getitem 314 | iterbytes = iter 315 | import io 316 | StringIO = io.StringIO 317 | BytesIO = io.BytesIO 318 | else: 319 | def b(s): 320 | return s 321 | def u(s): 322 | return unicode(s, "unicode_escape") 323 | int2byte = chr 324 | def indexbytes(buf, i): 325 | return ord(buf[i]) 326 | def iterbytes(buf): 327 | return (ord(byte) for byte in buf) 328 | import StringIO 329 | StringIO = BytesIO = StringIO.StringIO 330 | _add_doc(b, """Byte literal""") 331 | _add_doc(u, """Text literal""") 332 | 333 | 334 | if PY3: 335 | import builtins 336 | exec_ = getattr(builtins, "exec") 337 | 338 | 339 | def reraise(tp, value, tb=None): 340 | if value.__traceback__ is not tb: 341 | raise value.with_traceback(tb) 342 | raise value 343 | 344 | 345 | print_ = getattr(builtins, "print") 346 | del builtins 347 | 348 | else: 349 | def exec_(_code_, _globs_=None, _locs_=None): 350 | """Execute code in a namespace.""" 351 | if _globs_ is None: 352 | frame = sys._getframe(1) 353 | _globs_ = frame.f_globals 354 | if _locs_ is None: 355 | _locs_ = frame.f_locals 356 | del frame 357 | elif _locs_ is None: 358 | _locs_ = _globs_ 359 | exec("""exec _code_ in _globs_, _locs_""") 360 | 361 | 362 | exec_("""def reraise(tp, value, tb=None): 363 | raise tp, value, tb 364 | """) 365 | 366 | 367 | def print_(*args, **kwargs): 368 | """The new-style print function.""" 369 | fp = kwargs.pop("file", sys.stdout) 370 | if fp is None: 371 | return 372 | def write(data): 373 | if not isinstance(data, basestring): 374 | data = str(data) 375 | fp.write(data) 376 | want_unicode = False 377 | sep = kwargs.pop("sep", None) 378 | if sep is not None: 379 | if isinstance(sep, unicode): 380 | want_unicode = True 381 | elif not isinstance(sep, str): 382 | raise TypeError("sep must be None or a string") 383 | end = kwargs.pop("end", None) 384 | if end is not None: 385 | if isinstance(end, unicode): 386 | want_unicode = True 387 | elif not isinstance(end, str): 388 | raise TypeError("end must be None or a string") 389 | if kwargs: 390 | raise TypeError("invalid keyword arguments to print()") 391 | if not want_unicode: 392 | for arg in args: 393 | if isinstance(arg, unicode): 394 | want_unicode = True 395 | break 396 | if want_unicode: 397 | newline = unicode("\n") 398 | space = unicode(" ") 399 | else: 400 | newline = "\n" 401 | space = " " 402 | if sep is None: 403 | sep = space 404 | if end is None: 405 | end = newline 406 | for i, arg in enumerate(args): 407 | if i: 408 | write(sep) 409 | write(arg) 410 | write(end) 411 | 412 | _add_doc(reraise, """Reraise an exception.""") 413 | 414 | 415 | def with_metaclass(meta, *bases): 416 | """Create a base class with a metaclass.""" 417 | return meta("NewBase", bases, {}) 418 | -------------------------------------------------------------------------------- /django_postgres/view.py: -------------------------------------------------------------------------------- 1 | """Helpers to access Postgres views from the Django ORM.""" 2 | 3 | import collections 4 | import copy 5 | import logging 6 | import re 7 | 8 | from django.db import connection, transaction 9 | from django.db import models 10 | import psycopg2 11 | 12 | from . import six 13 | 14 | 15 | FIELD_SPEC_REGEX = (r'^([A-Za-z_][A-Za-z0-9_]*)\.' 16 | r'([A-Za-z_][A-Za-z0-9_]*)\.' 17 | r'(\*|(?:[A-Za-z_][A-Za-z0-9_]*))$') 18 | FIELD_SPEC_RE = re.compile(FIELD_SPEC_REGEX) 19 | 20 | log = logging.getLogger('django_postgres.view') 21 | 22 | 23 | def hasfield(model_cls, field_name): 24 | """Like `hasattr()`, but for model fields. 25 | 26 | >>> from django.contrib.auth.models import User 27 | >>> hasfield(User, 'password') 28 | True 29 | >>> hasfield(User, 'foobarbaz') 30 | False 31 | """ 32 | try: 33 | model_cls._meta.get_field_by_name(field_name) 34 | return True 35 | except models.FieldDoesNotExist: 36 | return False 37 | 38 | 39 | # Projections of models fields onto views which have been deferred due to 40 | # model import and loading dependencies. 41 | # Format: (app_label, model_name): {view_cls: [field_name, ...]} 42 | _DEFERRED_PROJECTIONS = collections.defaultdict( 43 | lambda: collections.defaultdict(list)) 44 | def realize_deferred_projections(sender, *args, **kwargs): 45 | """Project any fields which were deferred pending model preparation.""" 46 | app_label = sender._meta.app_label 47 | model_name = sender.__name__.lower() 48 | pending = _DEFERRED_PROJECTIONS.pop((app_label, model_name), {}) 49 | for view_cls, field_names in six.iteritems(pending): 50 | field_instances = get_fields_by_name(sender, *field_names) 51 | for name, field in six.iteritems(field_instances): 52 | # Only assign the field if the view does not already have an 53 | # attribute or explicitly-defined field with that name. 54 | if hasattr(view_cls, name) or hasfield(view_cls, name): 55 | continue 56 | copy.copy(field).contribute_to_class(view_cls, name) 57 | models.signals.class_prepared.connect(realize_deferred_projections) 58 | 59 | 60 | def create_views(models_module, update=True, force=False): 61 | """Create the database views for a given models module.""" 62 | for name, view_cls in six.iteritems(vars(models_module)): 63 | if not (isinstance(view_cls, type) and 64 | issubclass(view_cls, View) and 65 | hasattr(view_cls, 'sql')): 66 | continue 67 | 68 | try: 69 | created = create_view(connection, view_cls._meta.db_table, 70 | view_cls.sql, update=update, force=force) 71 | except Exception as exc: 72 | exc.view_cls = view_cls 73 | exc.python_name = models_module.__name__ + '.' + name 74 | raise 75 | else: 76 | yield created, view_cls, models_module.__name__ + '.' + name 77 | 78 | 79 | def create_view(connection, view_name, view_query, update=True, force=False): 80 | """ 81 | Create a named view on a connection. 82 | 83 | Returns True if a new view was created (or an existing one updated), or 84 | False if nothing was done. 85 | 86 | If ``update`` is True (default), attempt to update an existing view. If the 87 | existing view's schema is incompatible with the new definition, ``force`` 88 | (default: False) controls whether or not to drop the old view and create 89 | the new one. 90 | """ 91 | cursor_wrapper = connection.cursor() 92 | cursor = cursor_wrapper.cursor.cursor 93 | try: 94 | force_required = False 95 | # Determine if view already exists. 96 | cursor.execute('SELECT COUNT(*) FROM pg_catalog.pg_class WHERE relname = %s;', 97 | [view_name]) 98 | view_exists = cursor.fetchone()[0] > 0 99 | if view_exists and not update: 100 | return 'EXISTS' 101 | elif view_exists: 102 | # Detect schema conflict by copying the original view, attempting to 103 | # update this copy, and detecting errors. 104 | cursor.execute('CREATE TEMPORARY VIEW check_conflict AS SELECT * FROM {0};'.format(view_name)) 105 | try: 106 | cursor.execute('CREATE OR REPLACE TEMPORARY VIEW check_conflict AS {0};'.format(view_query)) 107 | except psycopg2.ProgrammingError: 108 | force_required = True 109 | cursor.connection.rollback() 110 | finally: 111 | cursor.execute('DROP VIEW IF EXISTS check_conflict;') 112 | 113 | if not force_required: 114 | cursor.execute('CREATE OR REPLACE VIEW {0} AS {1};'.format(view_name, view_query)) 115 | ret = view_exists and 'UPDATED' or 'CREATED' 116 | elif force: 117 | cursor.execute('DROP VIEW {0};'.format(view_name)) 118 | cursor.execute('CREATE VIEW {0} AS {1};'.format(view_name, view_query)) 119 | ret = 'FORCED' 120 | else: 121 | ret = 'FORCE_REQUIRED' 122 | 123 | transaction.commit_unless_managed() 124 | return ret 125 | finally: 126 | cursor_wrapper.close() 127 | 128 | 129 | 130 | def get_fields_by_name(model_cls, *field_names): 131 | """Return a dict of `models.Field` instances for named fields. 132 | 133 | Supports wildcard fetches using `'*'`. 134 | 135 | >>> get_fields_by_name(User, 'username', 'password') 136 | {'username': , 137 | 'password': } 138 | 139 | >>> get_fields_by_name(User, '*') 140 | {'username': , 141 | ..., 142 | 'date_joined': } 143 | """ 144 | if '*' in field_names: 145 | return dict((field.name, field) for field in model_cls._meta.fields) 146 | return dict((field_name, model_cls._meta.get_field_by_name(field_name)[0]) 147 | for field_name in field_names) 148 | 149 | 150 | class View(models.Model): 151 | 152 | """Helper for exposing Postgres views as Django models.""" 153 | 154 | class ViewMeta(models.base.ModelBase): 155 | 156 | def __new__(metacls, name, bases, attrs): 157 | projection = attrs.pop('projection', []) 158 | deferred_projections = [] 159 | for field_name in projection: 160 | if isinstance(field_name, models.Field): 161 | attrs[field_name.name] = copy.copy(field_name) 162 | elif isinstance(field_name, basestring): 163 | match = FIELD_SPEC_RE.match(field_name) 164 | if not match: 165 | raise TypeError("Unrecognized field specifier: %r" % 166 | field_name) 167 | deferred_projections.append(match.groups()) 168 | else: 169 | raise TypeError("Unrecognized field specifier: %r" % 170 | field_name) 171 | view_cls = models.base.ModelBase.__new__(metacls, name, bases, 172 | attrs) 173 | for app_label, model_name, field_name in deferred_projections: 174 | model_spec = (app_label, model_name.lower()) 175 | _DEFERRED_PROJECTIONS[model_spec][view_cls].append(field_name) 176 | # If the model has already been loaded, run 177 | # `realize_deferred_projections()` on it. 178 | model_cls = models.get_model(app_label, model_name, 179 | seed_cache=False) 180 | if model_cls is not None: 181 | realize_deferred_projections(model_cls) 182 | return view_cls 183 | 184 | __metaclass__ = ViewMeta 185 | 186 | class Meta: 187 | abstract = True 188 | managed = False 189 | -------------------------------------------------------------------------------- /doc/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = PYTHONPATH=..:../test_project DJANGO_SETTINGS_MODULE=test_project.settings sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # Internal variables. 11 | PAPEROPT_a4 = -D latex_paper_size=a4 12 | PAPEROPT_letter = -D latex_paper_size=letter 13 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 14 | # the i18n builder cannot share the environment and doctrees with the others 15 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 16 | 17 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 18 | 19 | help: 20 | @echo "Please use \`make ' where is one of" 21 | @echo " html to make standalone HTML files" 22 | @echo " dirhtml to make HTML files named index.html in directories" 23 | @echo " singlehtml to make a single large HTML file" 24 | @echo " pickle to make pickle files" 25 | @echo " json to make JSON files" 26 | @echo " htmlhelp to make HTML files and a HTML help project" 27 | @echo " qthelp to make HTML files and a qthelp project" 28 | @echo " devhelp to make HTML files and a Devhelp project" 29 | @echo " epub to make an epub" 30 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 31 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 32 | @echo " text to make text files" 33 | @echo " man to make manual pages" 34 | @echo " texinfo to make Texinfo files" 35 | @echo " info to make Texinfo files and run them through makeinfo" 36 | @echo " gettext to make PO message catalogs" 37 | @echo " changes to make an overview of all changed/added/deprecated items" 38 | @echo " linkcheck to check all external links for integrity" 39 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 40 | 41 | clean: 42 | -rm -rf $(BUILDDIR)/* 43 | 44 | html: 45 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 46 | @echo 47 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 48 | 49 | dirhtml: 50 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 51 | @echo 52 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 53 | 54 | singlehtml: 55 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 56 | @echo 57 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 58 | 59 | pickle: 60 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 61 | @echo 62 | @echo "Build finished; now you can process the pickle files." 63 | 64 | json: 65 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 66 | @echo 67 | @echo "Build finished; now you can process the JSON files." 68 | 69 | htmlhelp: 70 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 71 | @echo 72 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 73 | ".hhp project file in $(BUILDDIR)/htmlhelp." 74 | 75 | qthelp: 76 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 77 | @echo 78 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 79 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 80 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/django_postgres.qhcp" 81 | @echo "To view the help file:" 82 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/django_postgres.qhc" 83 | 84 | devhelp: 85 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 86 | @echo 87 | @echo "Build finished." 88 | @echo "To view the help file:" 89 | @echo "# mkdir -p $$HOME/.local/share/devhelp/django_postgres" 90 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/django_postgres" 91 | @echo "# devhelp" 92 | 93 | epub: 94 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 95 | @echo 96 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 97 | 98 | latex: 99 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 100 | @echo 101 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 102 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 103 | "(use \`make latexpdf' here to do that automatically)." 104 | 105 | latexpdf: 106 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 107 | @echo "Running LaTeX files through pdflatex..." 108 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 109 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 110 | 111 | text: 112 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 113 | @echo 114 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 115 | 116 | man: 117 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 118 | @echo 119 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 120 | 121 | texinfo: 122 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 123 | @echo 124 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 125 | @echo "Run \`make' in that directory to run these through makeinfo" \ 126 | "(use \`make info' here to do that automatically)." 127 | 128 | info: 129 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 130 | @echo "Running Texinfo files through makeinfo..." 131 | make -C $(BUILDDIR)/texinfo info 132 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 133 | 134 | gettext: 135 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 136 | @echo 137 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 138 | 139 | changes: 140 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 141 | @echo 142 | @echo "The overview file is in $(BUILDDIR)/changes." 143 | 144 | linkcheck: 145 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 146 | @echo 147 | @echo "Link check complete; look for any errors in the above output " \ 148 | "or in $(BUILDDIR)/linkcheck/output.txt." 149 | 150 | doctest: 151 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 152 | @echo "Testing of doctests in the sources finished, look at the " \ 153 | "results in $(BUILDDIR)/doctest/output.txt." 154 | -------------------------------------------------------------------------------- /doc/bitstrings.rst: -------------------------------------------------------------------------------- 1 | =========== 2 | Bit Strings 3 | =========== 4 | 5 | Postgres has a `bit string`_ type, which is exposed by django-postgres as 6 | :class:`~django_postgres.BitStringField` and the 7 | :class:`~django_postgres.BitStringExpression` helper (aliased as 8 | ``django_postgres.B``). The representation of bit strings in Python is handled 9 | by the `python-bitstring`_ library (a dependency of ``django-postgres``). 10 | 11 | .. _bit string: http://www.postgresql.org/docs/9.1/static/arrays.html 12 | .. _python-bitstring: http://packages.python.org/bitstring 13 | 14 | 15 | Quickstart 16 | ========== 17 | 18 | Given the following ``models.py``:: 19 | 20 | from django.db import models 21 | import django_postgres 22 | 23 | class BloomFilter(models.Model): 24 | name = models.CharField(max_length=100) 25 | bitmap = django_postgres.BitStringField(max_length=8) 26 | 27 | You can create objects with bit strings, and update them like so:: 28 | 29 | >>> from django_postgres import Bits 30 | >>> from models import BloomFilter 31 | 32 | >>> bloom = BloomFilter.objects.create(name='test') 33 | INSERT INTO myapp_bloomfilter 34 | (name, bitmap) VALUES ('test', B'00000000') 35 | RETURNING myapp_bloomfilter.id; 36 | 37 | >>> print bloom.bitmap 38 | Bits('0x00') 39 | >>> bloom.bitmap |= Bits(bin='00100000') 40 | >>> print bloom.bitmap 41 | Bits('0x20') 42 | 43 | >>> bloom.save(force_update=True) 44 | UPDATE myapp_bloomfilter SET bitmap = B'00100000' 45 | WHERE myapp_bloomfilter.id = 1; 46 | 47 | Several query lookups are defined for filtering on bit strings. Standard 48 | equality:: 49 | 50 | >>> BloomFilter.objects.filter(bitmap='00100000') 51 | SELECT * FROM myapp_bloomfilter WHERE bitmap = B'00100000'; 52 | 53 | You can also test against bitwise comparison operators (``and``, ``or`` and 54 | ``xor``). The SQL produced is slightly convoluted, due to the few functions 55 | provided by Postgres:: 56 | 57 | >>> BloomFilter.objects.filter(bitmap__and='00010000') 58 | SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap & B'00010000') > 0 59 | >>> BloomFilter.objects.filter(bitmap__or='00010000') 60 | SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap | B'00010000') > 0 61 | >>> BloomFilter.objects.filter(bitmap__xor='00010000') 62 | SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap # B'00010000') > 0 63 | 64 | Finally, you can also test the zero-ness of left- and right-shifted bit 65 | strings:: 66 | 67 | >>> BloomFilter.objects.filter(bitmap__lshift=3) 68 | SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap << 3) > 0 69 | >>> BloomFilter.objects.filter(bitmap__rshift=3) 70 | SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap >> 3) > 0 71 | 72 | 73 | Bit String Fields 74 | ================= 75 | 76 | .. class:: django_postgres.BitStringField(max_length=1[, varying=False, ...]) 77 | 78 | A bit string field, represented by the Postgres ``BIT`` or ``VARBIT`` types. 79 | 80 | :param max_length: 81 | The length (in bits) of this field. 82 | :param varying: 83 | Use a ``VARBIT`` instead of ``BIT``. Not recommended; it may cause strange 84 | querying behavior or length mismatch errors. 85 | 86 | If ``varying`` is True and ``max_length`` is ``None``, a ``VARBIT`` of 87 | unlimited length will be created. 88 | 89 | The default value of a :class:`BitStringField` is chosen as follows: 90 | 91 | * If a ``default`` kwarg is provided, that value is used. 92 | * Otherwise, if ``null=True``, the default value is ``None``. 93 | * Otherwise, if the field is not a ``VARBIT``, it defaults to an all-zero 94 | bit string of ``max_length`` (remember, the default length is 1). 95 | * Finally, all other cases will default to a single ``0``. 96 | 97 | All other parameters (``db_column``, ``help_text``, etc.) behave as standard 98 | for a Django field. 99 | 100 | 101 | Bit String Expressions 102 | ====================== 103 | 104 | It's useful to be able to atomically modify bit strings in the database, in a 105 | manner similar to Django's `F-expressions `_. 106 | For this reason, :class:`~django_postgres.BitStringExpression` is provided, 107 | and aliased as ``django_postgres.B`` for convenience. 108 | 109 | Here's a short example:: 110 | 111 | >>> from django_postgres import B 112 | >>> BloomFilter.objects.filter(id=1).update(bitmap=B('bitmap') | '00001000') 113 | UPDATE myapp_bloomfilter SET bitmap = bitmap | B'00001000' 114 | WHERE myapp_bloomfilter.id = 1; 115 | >>> bloom = BloomFilter.objects.get(id=1) 116 | >>> print bloom.bitmap 117 | Bits('0x28') 118 | 119 | .. class:: django_postgres.BitStringExpression(field_name) 120 | 121 | The following operators are supported: 122 | 123 | - Concatenation (``+``) 124 | - Bitwise AND (``&``) 125 | - Bitwise OR (``|``) 126 | - Bitwise XOR (``^``) 127 | - (Unary) bitwise NOT (``~``) 128 | - Bitwise left-shift (``<<``) 129 | - Bitwise right-shift (``>>``) 130 | -------------------------------------------------------------------------------- /doc/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # django-postgres documentation build configuration file, created by 4 | # sphinx-quickstart on Sun Aug 19 05:34:54 2012. 5 | # 6 | # This file is execfile()d with the current directory set to its containing dir. 7 | # 8 | # Note that not all possible configuration values are present in this 9 | # autogenerated file. 10 | # 11 | # All configuration values have a default; values that are commented out 12 | # serve to show the default. 13 | 14 | import sys, os 15 | 16 | # If extensions (or modules to document with autodoc) are in another directory, 17 | # add these directories to sys.path here. If the directory is relative to the 18 | # documentation root, use os.path.abspath to make it absolute, like shown here. 19 | #sys.path.insert(0, os.path.abspath('.')) 20 | 21 | # -- General configuration ----------------------------------------------------- 22 | 23 | # If your documentation needs a minimal Sphinx version, state it here. 24 | #needs_sphinx = '1.0' 25 | 26 | # Add any Sphinx extension module names here, as strings. They can be extensions 27 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 28 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] 29 | 30 | # Add any paths that contain templates here, relative to this directory. 31 | templates_path = ['_templates'] 32 | 33 | # The suffix of source filenames. 34 | source_suffix = '.rst' 35 | 36 | # The encoding of source files. 37 | #source_encoding = 'utf-8-sig' 38 | 39 | # The master toctree document. 40 | master_doc = 'index' 41 | 42 | # General information about the project. 43 | project = u'django-postgres' 44 | copyright = u'Public Domain' 45 | 46 | # The version info for the project you're documenting, acts as replacement for 47 | # |version| and |release|, also used in various other places throughout the 48 | # built documents. 49 | # 50 | # The short X.Y version. 51 | version = '0.0.1' 52 | # The full version, including alpha/beta/rc tags. 53 | release = '0.0.1' 54 | 55 | # The language for content autogenerated by Sphinx. Refer to documentation 56 | # for a list of supported languages. 57 | #language = None 58 | 59 | # There are two options for replacing |today|: either, you set today to some 60 | # non-false value, then it is used: 61 | #today = '' 62 | # Else, today_fmt is used as the format for a strftime call. 63 | #today_fmt = '%B %d, %Y' 64 | 65 | # List of patterns, relative to source directory, that match files and 66 | # directories to ignore when looking for source files. 67 | exclude_patterns = ['_build'] 68 | 69 | # The reST default role (used for this markup: `text`) to use for all documents. 70 | #default_role = None 71 | 72 | # If true, '()' will be appended to :func: etc. cross-reference text. 73 | #add_function_parentheses = True 74 | 75 | # If true, the current module name will be prepended to all description 76 | # unit titles (such as .. function::). 77 | #add_module_names = True 78 | 79 | # If true, sectionauthor and moduleauthor directives will be shown in the 80 | # output. They are ignored by default. 81 | #show_authors = False 82 | 83 | # The name of the Pygments (syntax highlighting) style to use. 84 | pygments_style = 'sphinx' 85 | 86 | # A list of ignored prefixes for module index sorting. 87 | #modindex_common_prefix = [] 88 | 89 | 90 | # -- Options for HTML output --------------------------------------------------- 91 | 92 | # The theme to use for HTML and HTML Help pages. See the documentation for 93 | # a list of builtin themes. 94 | html_theme = 'default' 95 | 96 | # Theme options are theme-specific and customize the look and feel of a theme 97 | # further. For a list of options available for each theme, see the 98 | # documentation. 99 | #html_theme_options = {} 100 | 101 | # Add any paths that contain custom themes here, relative to this directory. 102 | #html_theme_path = [] 103 | 104 | # The name for this set of Sphinx documents. If None, it defaults to 105 | # " v documentation". 106 | #html_title = None 107 | 108 | # A shorter title for the navigation bar. Default is the same as html_title. 109 | #html_short_title = None 110 | 111 | # The name of an image file (relative to this directory) to place at the top 112 | # of the sidebar. 113 | #html_logo = None 114 | 115 | # The name of an image file (within the static path) to use as favicon of the 116 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 117 | # pixels large. 118 | #html_favicon = None 119 | 120 | # Add any paths that contain custom static files (such as style sheets) here, 121 | # relative to this directory. They are copied after the builtin static files, 122 | # so a file named "default.css" will overwrite the builtin "default.css". 123 | html_static_path = ['_static'] 124 | 125 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 126 | # using the given strftime format. 127 | #html_last_updated_fmt = '%b %d, %Y' 128 | 129 | # If true, SmartyPants will be used to convert quotes and dashes to 130 | # typographically correct entities. 131 | #html_use_smartypants = True 132 | 133 | # Custom sidebar templates, maps document names to template names. 134 | #html_sidebars = {} 135 | 136 | # Additional templates that should be rendered to pages, maps page names to 137 | # template names. 138 | #html_additional_pages = {} 139 | 140 | # If false, no module index is generated. 141 | #html_domain_indices = True 142 | 143 | # If false, no index is generated. 144 | #html_use_index = True 145 | 146 | # If true, the index is split into individual pages for each letter. 147 | #html_split_index = False 148 | 149 | # If true, links to the reST sources are added to the pages. 150 | #html_show_sourcelink = True 151 | 152 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 153 | #html_show_sphinx = True 154 | 155 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 156 | html_show_copyright = False 157 | 158 | # If true, an OpenSearch description file will be output, and all pages will 159 | # contain a tag referring to it. The value of this option must be the 160 | # base URL from which the finished HTML is served. 161 | #html_use_opensearch = '' 162 | 163 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 164 | #html_file_suffix = None 165 | 166 | # Output file base name for HTML help builder. 167 | htmlhelp_basename = 'django_postgresdoc' 168 | 169 | 170 | # -- Options for LaTeX output -------------------------------------------------- 171 | 172 | latex_elements = { 173 | # The paper size ('letterpaper' or 'a4paper'). 174 | #'papersize': 'letterpaper', 175 | 176 | # The font size ('10pt', '11pt' or '12pt'). 177 | #'pointsize': '10pt', 178 | 179 | # Additional stuff for the LaTeX preamble. 180 | #'preamble': '', 181 | } 182 | 183 | # Grouping the document tree into LaTeX files. List of tuples 184 | # (source start file, target name, title, author, documentclass [howto/manual]). 185 | latex_documents = [ 186 | ('index', 'django-postgres.tex', u'django\\-postgres Documentation', 187 | u'Author', 'manual'), 188 | ] 189 | 190 | # The name of an image file (relative to this directory) to place at the top of 191 | # the title page. 192 | #latex_logo = None 193 | 194 | # For "manual" documents, if this is true, then toplevel headings are parts, 195 | # not chapters. 196 | #latex_use_parts = False 197 | 198 | # If true, show page references after internal links. 199 | #latex_show_pagerefs = False 200 | 201 | # If true, show URL addresses after external links. 202 | #latex_show_urls = False 203 | 204 | # Documents to append as an appendix to all manuals. 205 | #latex_appendices = [] 206 | 207 | # If false, no module index is generated. 208 | #latex_domain_indices = True 209 | 210 | 211 | # -- Options for manual page output -------------------------------------------- 212 | 213 | # One entry per manual page. List of tuples 214 | # (source start file, name, description, authors, manual section). 215 | man_pages = [ 216 | ('index', 'django-postgres', u'django-postgres Documentation', 217 | [u'Author'], 1) 218 | ] 219 | 220 | # If true, show URL addresses after external links. 221 | #man_show_urls = False 222 | 223 | 224 | # -- Options for Texinfo output ------------------------------------------------ 225 | 226 | # Grouping the document tree into Texinfo files. List of tuples 227 | # (source start file, target name, title, author, 228 | # dir menu entry, description, category) 229 | texinfo_documents = [ 230 | ('index', 'django-postgres', u'django-postgres Documentation', 231 | u'Author', 'django-postgres', 'One line description of project.', 232 | 'Miscellaneous'), 233 | ] 234 | 235 | # Documents to append as an appendix to all manuals. 236 | #texinfo_appendices = [] 237 | 238 | # If false, no module index is generated. 239 | #texinfo_domain_indices = True 240 | 241 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 242 | #texinfo_show_urls = 'footnote' 243 | 244 | 245 | # -- Options for Epub output --------------------------------------------------- 246 | 247 | # Bibliographic Dublin Core info. 248 | epub_title = u'django-postgres' 249 | epub_author = u'Author' 250 | epub_publisher = u'Author' 251 | epub_copyright = u'2012, Author' 252 | 253 | # The language of the text. It defaults to the language option 254 | # or en if the language is not set. 255 | #epub_language = '' 256 | 257 | # The scheme of the identifier. Typical schemes are ISBN or URL. 258 | #epub_scheme = '' 259 | 260 | # The unique identifier of the text. This can be a ISBN number 261 | # or the project homepage. 262 | #epub_identifier = '' 263 | 264 | # A unique identification for the text. 265 | #epub_uid = '' 266 | 267 | # A tuple containing the cover image and cover page html template filenames. 268 | #epub_cover = () 269 | 270 | # HTML files that should be inserted before the pages created by sphinx. 271 | # The format is a list of tuples containing the path and title. 272 | #epub_pre_files = [] 273 | 274 | # HTML files shat should be inserted after the pages created by sphinx. 275 | # The format is a list of tuples containing the path and title. 276 | #epub_post_files = [] 277 | 278 | # A list of files that should not be packed into the epub file. 279 | #epub_exclude_files = [] 280 | 281 | # The depth of the table of contents in toc.ncx. 282 | #epub_tocdepth = 3 283 | 284 | # Allow duplicate toc entries. 285 | #epub_tocdup = True 286 | -------------------------------------------------------------------------------- /doc/index.rst: -------------------------------------------------------------------------------- 1 | =============== 2 | django-postgres 3 | =============== 4 | 5 | Adds first-class support for `PostgreSQL `_ 6 | features to the Django ORM. 7 | 8 | Planned features include: 9 | 10 | - `Arrays `_ 11 | - `Enums `_ 12 | - `Bit Strings `_ 13 | - `Constraints `_ 14 | - `Triggers `_ 15 | - `Domains `_ 16 | - `Composite Types `_ 17 | - `Views `_ 18 | 19 | Obviously this is quite a large project, but I think it would provide a huge 20 | amount of value to Django developers. 21 | 22 | 23 | Why? 24 | ==== 25 | 26 | PostgreSQL is an excellent data store, with a host of useful and 27 | efficiently-implemented features. Unfortunately these features are not exposed 28 | through Django's ORM, primarily because the framework has to support several 29 | SQL backends and so can only provide a set of features common to all of them. 30 | 31 | The features made available here replace some of the following practices: 32 | 33 | - Manual denormalization on ``save()`` (such that model saves may result in 34 | three or more separate queries). 35 | - Sequences represented by a one-to-many, with an ``order`` integer field. 36 | - Complex types represented by JSON in a text field. 37 | 38 | 39 | Contents 40 | ======== 41 | 42 | This is a WIP, so the following list may grow and change over time. 43 | 44 | .. toctree:: 45 | :maxdepth: 4 46 | 47 | views 48 | bitstrings 49 | 50 | 51 | Indices and tables 52 | ================== 53 | 54 | * :ref:`genindex` 55 | * :ref:`modindex` 56 | * :ref:`search` 57 | 58 | -------------------------------------------------------------------------------- /doc/views.rst: -------------------------------------------------------------------------------- 1 | ===== 2 | Views 3 | ===== 4 | 5 | For more info on Postgres views, see the `official Postgres docs 6 | `_. Effectively, 7 | views are named queries which can be accessed as if they were regular database 8 | tables. 9 | 10 | Quickstart 11 | ========== 12 | 13 | Given the following view in SQL: 14 | 15 | .. code-block:: sql 16 | 17 | CREATE OR REPLACE VIEW myapp_viewname AS 18 | SELECT * FROM myapp_table WHERE condition; 19 | 20 | You can create this view by just subclassing :class:`django_postgres.View`. In 21 | ``myapp/models.py``:: 22 | 23 | import django_postgres 24 | 25 | class ViewName(django_postgres.View): 26 | projection = ['myapp.Table.*'] 27 | sql = """SELECT * FROM myapp_table WHERE condition""" 28 | 29 | :class:`View` 30 | ============= 31 | 32 | .. class:: django_postgres.View 33 | 34 | Inherit from this class to define and interact with your database views. 35 | 36 | You need to either define the field types manually (using standard Django 37 | model fields), or use :attr:`projection` to copy field definitions from other 38 | models. 39 | 40 | .. attribute:: sql 41 | 42 | The SQL for this view (typically a ``SELECT`` query). This attribute is 43 | optional, but if present, the view will be created on ``sync_pgviews`` 44 | (which is probably what you want). 45 | 46 | .. attribute:: projection 47 | 48 | A list of field specifiers which will be automatically copied to this view. 49 | If your view directly presents fields from another table, you can 50 | effectively 'import' those here, like so:: 51 | 52 | projection = ['auth.User.username', 'auth.User.password', 53 | 'admin.LogEntry.change_message'] 54 | 55 | If your view represents a subset of rows in another table (but the same 56 | columns), you might want to import all the fields from that table, like 57 | so:: 58 | 59 | projection = ['myapp.Table.*'] 60 | 61 | Of course you can mix wildcards with normal field specifiers:: 62 | 63 | projection = ['myapp.Table.*', 'auth.User.username', 'auth.User.email'] 64 | 65 | 66 | Primary Keys 67 | ============ 68 | 69 | Django requires exactly one field on any relation (view, table, etc.) to be a 70 | primary key. By default it will add an ``id`` field to your view, and this will 71 | work fine if you're using a wildcard projection from another model. If not, you 72 | should do one of three things. Project an ``id`` field from a model with a one-to-one 73 | relationship:: 74 | 75 | class SimpleUser(django_postgres.View): 76 | projection = ['auth.User.id', 'auth.User.username', 'auth.User.password'] 77 | sql = """SELECT id, username, password, FROM auth_user;""" 78 | 79 | Explicitly define a field on your view with ``primary_key=True``:: 80 | 81 | class SimpleUser(django_postgres.View): 82 | projection = ['auth.User.password'] 83 | sql = """SELECT username, password, FROM auth_user;""" 84 | # max_length doesn't matter here, but Django needs something. 85 | username = models.CharField(max_length=1, primary_key=True) 86 | 87 | Or add an ``id`` column to your view's SQL query (this example uses 88 | `window functions `_):: 89 | 90 | class SimpleUser(django_postgres.View): 91 | projection = ['auth.User.username', 'auth.User.password'] 92 | sql = """SELECT username, password, row_number() OVER () AS id 93 | FROM auth_user;""" 94 | 95 | 96 | Creating the Views 97 | ================== 98 | 99 | Creating the views is simple. Just run the ``sync_pgviews`` command:: 100 | 101 | $ ./manage.py sync_pgviews 102 | Creating views for django.contrib.auth.models 103 | Creating views for django.contrib.contenttypes.models 104 | Creating views for myapp.models 105 | myapp.models.Superusers (myapp_superusers): created 106 | myapp.models.SimpleUser (myapp_simpleuser): created 107 | myapp.models.Staffness (myapp_staffness): created 108 | 109 | Migrations 110 | ========== 111 | 112 | Views play well with South migrations. If a migration modifies the underlying 113 | table(s) that a view depends on so as to break the view, that view will be 114 | silently deleted by Postgres. For this reason, it's important to run 115 | ``sync_pgviews`` after ``migrate`` to ensure any required tables have been 116 | created/updated. 117 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | 3 | setup( 4 | name='django-postgres', 5 | version='0.0.2', 6 | description="First-class Postgres feature support for the Django ORM.", 7 | author='Zachary Voase', 8 | author_email='z@zacharyvoase.com', 9 | license='Public Domain', 10 | packages=find_packages(), 11 | install_requires=[ 12 | 'bitstring', 13 | 'Django>=1.3', 14 | ], 15 | ) 16 | -------------------------------------------------------------------------------- /tests/test_project/Makefile: -------------------------------------------------------------------------------- 1 | export DJANGO_SETTINGS_MODULE=test_project.settings 2 | export PYTHONPATH=$(pwd) 3 | 4 | .PHONY: test 5 | 6 | test: .venv/bin/django-admin.py 7 | . .venv/bin/activate; django-admin.py test 8 | 9 | .venv/bin/python: 10 | virtualenv --distribute .venv 11 | . .venv/bin/activate; easy_install pip 12 | 13 | .venv/bin/django-admin.py: .venv/bin/python requirements.txt 14 | . .venv/bin/activate; pip install -r requirements.txt 15 | . .venv/bin/activate; cd ../../; python setup.py develop 16 | -------------------------------------------------------------------------------- /tests/test_project/README.md: -------------------------------------------------------------------------------- 1 | # django-postgres Test Project 2 | 3 | This test project contains Python tests of the various pieces of API 4 | functionality implemented by django-postgres. More tests exist in `../tests/`. 5 | 6 | To run the tests: 7 | 8 | 1. Install Postgres. I use [Postgres.app](http://postgresapp.com/). 9 | 10 | 2. Create a `django_postgres` database: 11 | 12 | $ psql 13 | psql (9.1.4) 14 | Type "help" for help. 15 | 16 | johndoe=# CREATE DATABASE django_postgres; 17 | CREATE DATABASE 18 | 19 | 3. Run `make test` from this directory. This will create a virtualenv, install 20 | the test project requirements, and execute the tests. On subsequent runs, 21 | the virtualenv/installation steps will be skipped thanks to the Makefile. 22 | -------------------------------------------------------------------------------- /tests/test_project/arraytest/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zacharyvoase/django-postgres/93e9f9809cabee0b327f18d181cbc9aeab1f8f2e/tests/test_project/arraytest/__init__.py -------------------------------------------------------------------------------- /tests/test_project/arraytest/models.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | 3 | # Create your models here. 4 | -------------------------------------------------------------------------------- /tests/test_project/arraytest/tests.py: -------------------------------------------------------------------------------- 1 | """ 2 | This file demonstrates writing tests using the unittest module. These will pass 3 | when you run "manage.py test". 4 | 5 | Replace this with more appropriate tests for your application. 6 | """ 7 | 8 | from django.test import TestCase 9 | 10 | 11 | class SimpleTest(TestCase): 12 | def test_basic_addition(self): 13 | """ 14 | Tests that 1 + 1 always equals 2. 15 | """ 16 | self.assertEqual(1 + 1, 2) 17 | -------------------------------------------------------------------------------- /tests/test_project/arraytest/views.py: -------------------------------------------------------------------------------- 1 | # Create your views here. 2 | -------------------------------------------------------------------------------- /tests/test_project/bitstringtest/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zacharyvoase/django-postgres/93e9f9809cabee0b327f18d181cbc9aeab1f8f2e/tests/test_project/bitstringtest/__init__.py -------------------------------------------------------------------------------- /tests/test_project/bitstringtest/models.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | import django_postgres 3 | 4 | 5 | class BloomFilter(models.Model): 6 | name = models.CharField(max_length=100) 7 | bitmap = django_postgres.BitStringField(max_length=8) 8 | 9 | 10 | class VarBitmap(models.Model): 11 | name = models.CharField(max_length=100) 12 | bitmap = django_postgres.BitStringField(max_length=8, varying=True) 13 | 14 | 15 | class NullBitmap(models.Model): 16 | name = models.CharField(max_length=100) 17 | bitmap = django_postgres.BitStringField(max_length=8, null=True) 18 | -------------------------------------------------------------------------------- /tests/test_project/bitstringtest/tests.py: -------------------------------------------------------------------------------- 1 | from django.test import TestCase 2 | from django_postgres import Bits, B 3 | 4 | import models 5 | 6 | 7 | class SimpleTest(TestCase): 8 | 9 | def test_can_create_bitstrings(self): 10 | bloom = models.BloomFilter.objects.create(name='foo') 11 | # Default bit string is all zeros. 12 | assert bloom.bitmap.bin == ('0' * 8) 13 | 14 | def test_null_bitmap_defaults_to_None(self): 15 | bloom = models.NullBitmap.objects.create(name='foo') 16 | assert bloom.bitmap is None 17 | 18 | def test_can_change_bitstrings(self): 19 | bloom = models.BloomFilter.objects.create(name='foo') 20 | bloom.bitmap = Bits(bin='01011010') 21 | bloom.save() 22 | 23 | refetch = models.BloomFilter.objects.get(id=bloom.id) 24 | assert refetch.bitmap.bin == '01011010' 25 | 26 | def test_can_set_null_bitmap_to_None(self): 27 | bloom = models.NullBitmap.objects.create(name='foo', 28 | bitmap=Bits(bin='01010101')) 29 | assert bloom.bitmap is not None 30 | bloom.bitmap = None 31 | bloom.save() 32 | bloom = models.NullBitmap.objects.get(id=bloom.id) 33 | assert bloom.bitmap is None 34 | 35 | def test_can_search_for_equal_bitstrings(self): 36 | models.BloomFilter.objects.create(name='foo', bitmap='01011010') 37 | 38 | results = models.BloomFilter.objects.filter(bitmap='01011010') 39 | assert results.count() == 1 40 | assert results[0].name == 'foo' 41 | 42 | 43 | class VarBitTest(TestCase): 44 | 45 | def test_can_create_varying_length_bitstrings(self): 46 | bloom = models.VarBitmap.objects.create(name='foo') 47 | # Default varbit string is one zero. 48 | assert bloom.bitmap.bin == '0' 49 | 50 | def test_can_change_varbit_length(self): 51 | bloom = models.VarBitmap.objects.create(name='foo', 52 | bitmap=Bits(bin='01010')) 53 | assert len(bloom.bitmap.bin) == 5 54 | bloom.bitmap = Bits(bin='0101010') 55 | bloom.save() 56 | bloom = models.VarBitmap.objects.get(id=bloom.id) 57 | assert len(bloom.bitmap.bin) == 7 58 | 59 | 60 | class BitStringExpressionUpdateTest(TestCase): 61 | 62 | def check_update(self, initial, expression, result): 63 | models.BloomFilter.objects.create(name='foo', bitmap=initial) 64 | models.BloomFilter.objects.create(name='bar') 65 | 66 | models.BloomFilter.objects \ 67 | .filter(name='foo') \ 68 | .update(bitmap=expression) 69 | 70 | assert models.BloomFilter.objects.get(name='foo').bitmap.bin == result 71 | assert models.BloomFilter.objects.get(name='bar').bitmap.bin == '00000000' 72 | 73 | def test_or(self): 74 | self.check_update('00000000', 75 | B('bitmap') | Bits('0b10100101'), 76 | '10100101') 77 | 78 | def test_and(self): 79 | self.check_update('10100101', 80 | B('bitmap') & Bits('0b11000011'), 81 | '10000001') 82 | 83 | def test_xor(self): 84 | self.check_update('10100101', 85 | B('bitmap') ^ Bits('0b11000011'), 86 | '01100110') 87 | 88 | def test_not(self): 89 | self.check_update('10100101', 90 | ~B('bitmap'), 91 | '01011010') 92 | 93 | def test_lshift(self): 94 | self.check_update('10100101', 95 | B('bitmap') << 3, 96 | '00101000') 97 | 98 | def test_rshift(self): 99 | self.check_update('10100101', 100 | B('bitmap') >> 3, 101 | '00010100') 102 | -------------------------------------------------------------------------------- /tests/test_project/bitstringtest/views.py: -------------------------------------------------------------------------------- 1 | # Create your views here. 2 | -------------------------------------------------------------------------------- /tests/test_project/requirements.txt: -------------------------------------------------------------------------------- 1 | django-nose>=1.1 2 | nose>=1.2.1 3 | psycopg2>2.4.2 4 | -------------------------------------------------------------------------------- /tests/test_project/test_project/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zacharyvoase/django-postgres/93e9f9809cabee0b327f18d181cbc9aeab1f8f2e/tests/test_project/test_project/__init__.py -------------------------------------------------------------------------------- /tests/test_project/test_project/settings.py: -------------------------------------------------------------------------------- 1 | import commands 2 | 3 | DEBUG = True 4 | TEMPLATE_DEBUG = DEBUG 5 | 6 | def git_name_and_email(): 7 | name = commands.getoutput('git config user.name') 8 | email = commands.getoutput('git config user.email') 9 | return name, email 10 | 11 | ADMINS = (git_name_and_email(),) 12 | MANAGERS = ADMINS 13 | 14 | DATABASES = { 15 | 'default': { 16 | 'ENGINE': 'django.db.backends.postgresql_psycopg2', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'. 17 | 'NAME': 'django_postgres', # Or path to database file if using sqlite3. 18 | 'USER': '', # Not used with sqlite3. 19 | 'PASSWORD': '', # Not used with sqlite3. 20 | 'HOST': '127.0.0.1', # Set to empty string for localhost. Not used with sqlite3. 21 | 'PORT': '', # Set to empty string for default. Not used with sqlite3. 22 | } 23 | } 24 | 25 | # Local time zone for this installation. Choices can be found here: 26 | # http://en.wikipedia.org/wiki/List_of_tz_zones_by_name 27 | # although not all choices may be available on all operating systems. 28 | # In a Windows environment this must be set to your system time zone. 29 | TIME_ZONE = 'Atlantic/Reykjavik' 30 | 31 | # Language code for this installation. All choices can be found here: 32 | # http://www.i18nguy.com/unicode/language-identifiers.html 33 | LANGUAGE_CODE = 'en-us' 34 | 35 | SITE_ID = 1 36 | 37 | # If you set this to False, Django will make some optimizations so as not 38 | # to load the internationalization machinery. 39 | USE_I18N = False 40 | 41 | # If you set this to False, Django will not format dates, numbers and 42 | # calendars according to the current locale. 43 | USE_L10N = False 44 | 45 | # If you set this to False, Django will not use timezone-aware datetimes. 46 | USE_TZ = True 47 | 48 | # Absolute filesystem path to the directory that will hold user-uploaded files. 49 | # Example: "/home/media/media.lawrence.com/media/" 50 | MEDIA_ROOT = '' 51 | 52 | # URL that handles the media served from MEDIA_ROOT. Make sure to use a 53 | # trailing slash. 54 | # Examples: "http://media.lawrence.com/media/", "http://example.com/media/" 55 | MEDIA_URL = '' 56 | 57 | # Absolute path to the directory static files should be collected to. 58 | # Don't put anything in this directory yourself; store your static files 59 | # in apps' "static/" subdirectories and in STATICFILES_DIRS. 60 | # Example: "/home/media/media.lawrence.com/static/" 61 | STATIC_ROOT = '' 62 | 63 | # URL prefix for static files. 64 | # Example: "http://media.lawrence.com/static/" 65 | STATIC_URL = '/static/' 66 | 67 | # Additional locations of static files 68 | STATICFILES_DIRS = ( 69 | # Put strings here, like "/home/html/static" or "C:/www/django/static". 70 | # Always use forward slashes, even on Windows. 71 | # Don't forget to use absolute paths, not relative paths. 72 | ) 73 | 74 | # List of finder classes that know how to find static files in 75 | # various locations. 76 | STATICFILES_FINDERS = ( 77 | 'django.contrib.staticfiles.finders.FileSystemFinder', 78 | 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 79 | # 'django.contrib.staticfiles.finders.DefaultStorageFinder', 80 | ) 81 | 82 | # Make this unique, and don't share it with anybody. 83 | SECRET_KEY = 'g34i6_w+#in7d6_ficl42kbw!d*axa0qroei8yp#n__he22&+g' 84 | 85 | # List of callables that know how to import templates from various sources. 86 | TEMPLATE_LOADERS = ( 87 | 'django.template.loaders.filesystem.Loader', 88 | 'django.template.loaders.app_directories.Loader', 89 | # 'django.template.loaders.eggs.Loader', 90 | ) 91 | 92 | MIDDLEWARE_CLASSES = ( 93 | 'django.middleware.common.CommonMiddleware', 94 | 'django.middleware.csrf.CsrfViewMiddleware', 95 | 'django.contrib.messages.middleware.MessageMiddleware', 96 | # Uncomment the next line for simple clickjacking protection: 97 | # 'django.middleware.clickjacking.XFrameOptionsMiddleware', 98 | ) 99 | 100 | ROOT_URLCONF = 'test_project.urls' 101 | 102 | # Python dotted path to the WSGI application used by Django's runserver. 103 | WSGI_APPLICATION = 'test_project.wsgi.application' 104 | 105 | TEMPLATE_DIRS = ( 106 | # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates". 107 | # Always use forward slashes, even on Windows. 108 | # Don't forget to use absolute paths, not relative paths. 109 | ) 110 | 111 | INSTALLED_APPS = ( 112 | 'django.contrib.auth', 113 | 'django.contrib.contenttypes', 114 | 'django_nose', 115 | 'django_postgres', 116 | 'viewtest', 117 | 'arraytest', 118 | 'bitstringtest', 119 | ) 120 | 121 | # A sample logging configuration. The only tangible logging 122 | # performed by this configuration is to send an email to 123 | # the site admins on every HTTP 500 error when DEBUG=False. 124 | # See http://docs.djangoproject.com/en/dev/topics/logging for 125 | # more details on how to customize your logging configuration. 126 | LOGGING = { 127 | 'version': 1, 128 | 'disable_existing_loggers': False, 129 | 'filters': { 130 | 'require_debug_false': { 131 | '()': 'django.utils.log.RequireDebugFalse' 132 | } 133 | }, 134 | 'handlers': { 135 | 'mail_admins': { 136 | 'level': 'ERROR', 137 | 'filters': ['require_debug_false'], 138 | 'class': 'django.utils.log.AdminEmailHandler' 139 | }, 140 | 'console': { 141 | 'level': 'DEBUG', 142 | 'class': 'logging.StreamHandler', 143 | }, 144 | }, 145 | 'loggers': { 146 | 'django.request': { 147 | 'handlers': ['mail_admins'], 148 | 'level': 'ERROR', 149 | 'propagate': True, 150 | }, 151 | 'django_postgres': { 152 | 'handlers': ['console'], 153 | 'level': 'DEBUG', 154 | 'propagate': True, 155 | }, 156 | 'django.db.backends': { 157 | 'handlers': ['console'], 158 | 'level': 'DEBUG', 159 | 'propagate': True, 160 | } 161 | } 162 | } 163 | 164 | TEST_RUNNER = 'django_nose.NoseTestSuiteRunner' 165 | -------------------------------------------------------------------------------- /tests/test_project/test_project/urls.py: -------------------------------------------------------------------------------- 1 | from django.conf.urls import patterns, include, url 2 | 3 | # Uncomment the next two lines to enable the admin: 4 | # from django.contrib import admin 5 | # admin.autodiscover() 6 | 7 | urlpatterns = patterns('', 8 | # Examples: 9 | # url(r'^$', 'test_project.views.home', name='home'), 10 | # url(r'^test_project/', include('test_project.foo.urls')), 11 | 12 | # Uncomment the admin/doc line below to enable admin documentation: 13 | # url(r'^admin/doc/', include('django.contrib.admindocs.urls')), 14 | 15 | # Uncomment the next line to enable the admin: 16 | # url(r'^admin/', include(admin.site.urls)), 17 | ) 18 | -------------------------------------------------------------------------------- /tests/test_project/test_project/wsgi.py: -------------------------------------------------------------------------------- 1 | """ 2 | WSGI config for test_project project. 3 | 4 | This module contains the WSGI application used by Django's development server 5 | and any production WSGI deployments. It should expose a module-level variable 6 | named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover 7 | this application via the ``WSGI_APPLICATION`` setting. 8 | 9 | Usually you will have the standard Django WSGI application here, but it also 10 | might make sense to replace the whole Django WSGI application with a custom one 11 | that later delegates to the Django one. For example, you could introduce WSGI 12 | middleware here, or combine a Django application with an application of another 13 | framework. 14 | 15 | """ 16 | import os 17 | 18 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "test_project.settings") 19 | 20 | # This application object is used by any WSGI server configured to use this 21 | # file. This includes Django's development server, if the WSGI_APPLICATION 22 | # setting points here. 23 | from django.core.wsgi import get_wsgi_application 24 | application = get_wsgi_application() 25 | 26 | # Apply WSGI middleware here. 27 | # from helloworld.wsgi import HelloWorldApplication 28 | # application = HelloWorldApplication(application) 29 | -------------------------------------------------------------------------------- /tests/test_project/viewtest/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zacharyvoase/django-postgres/93e9f9809cabee0b327f18d181cbc9aeab1f8f2e/tests/test_project/viewtest/__init__.py -------------------------------------------------------------------------------- /tests/test_project/viewtest/models.py: -------------------------------------------------------------------------------- 1 | import django.contrib.auth.models as auth_models 2 | 3 | import django_postgres 4 | 5 | 6 | class Superusers(django_postgres.View): 7 | projection = ['auth.User.*'] 8 | sql = """SELECT * FROM auth_user WHERE is_superuser = TRUE;""" 9 | 10 | 11 | class SimpleUser(django_postgres.View): 12 | projection = ['auth.User.username', 'auth.User.password'] 13 | # The row_number() window function is needed so that Django sees some kind 14 | # of 'id' field. We could also grab the one from `auth.User`, but this 15 | # seemed like more fun :) 16 | sql = """ 17 | SELECT 18 | username, 19 | password, 20 | row_number() OVER () AS id 21 | FROM auth_user;""" 22 | 23 | 24 | class Staffness(django_postgres.View): 25 | projection = ['auth.User.username', 'auth.User.is_staff'] 26 | sql = str(auth_models.User.objects.only('username', 'is_staff').query) 27 | -------------------------------------------------------------------------------- /tests/test_project/viewtest/tests.py: -------------------------------------------------------------------------------- 1 | from contextlib import closing 2 | 3 | from django.contrib import auth 4 | from django.core.management import call_command 5 | from django.db import connection 6 | from django.test import TestCase 7 | 8 | import models 9 | 10 | 11 | class ViewTestCase(TestCase): 12 | 13 | def setUp(self): 14 | call_command('sync_pgviews', *[], **{}) 15 | 16 | def test_views_have_been_created(self): 17 | with closing(connection.cursor()) as cur: 18 | cur.execute('''SELECT COUNT(*) FROM pg_views 19 | WHERE viewname LIKE 'viewtest_%';''') 20 | 21 | count, = cur.fetchone() 22 | self.assertEqual(count, 3) 23 | 24 | def test_wildcard_projection_gets_all_fields_from_projected_model(self): 25 | foo_user = auth.models.User.objects.create( 26 | username='foo', is_superuser=True) 27 | foo_user.set_password('blah') 28 | foo_user.save() 29 | 30 | foo_superuser = models.Superusers.objects.get(username='foo') 31 | 32 | self.assertEqual(foo_user.id, foo_superuser.id) 33 | self.assertEqual(foo_user.password, foo_superuser.password) 34 | 35 | def test_limited_projection_only_gets_selected_fields_from_projected_model(self): 36 | foo_user = auth.models.User.objects.create( 37 | username='foo', is_superuser=True) 38 | foo_user.set_password('blah') 39 | foo_user.save() 40 | 41 | foo_simple = models.SimpleUser.objects.get(username='foo') 42 | self.assertEqual(foo_simple.username, foo_user.username) 43 | self.assertEqual(foo_simple.password, foo_user.password) 44 | self.assertFalse(hasattr(foo_simple, 'date_joined')) 45 | 46 | def test_queryset_based_view_works_similarly_to_raw_sql(self): 47 | auth.models.User.objects.create( 48 | username='foo', is_staff=True) 49 | 50 | self.assertTrue( 51 | models.Staffness.objects.filter(username='foo').exists()) 52 | -------------------------------------------------------------------------------- /tests/test_project/viewtest/views.py: -------------------------------------------------------------------------------- 1 | # Create your views here. 2 | -------------------------------------------------------------------------------- /tests/views/README.md: -------------------------------------------------------------------------------- 1 | This is a high-level test of the interaction between views and migrations in 2 | the development and deployment of a Django project. The database 3 | `postgresql://localhost/django_postgres_viewtest` needs to exist for this test 4 | to work. 5 | 6 | To run it, just do: 7 | 8 | $ ./do_viewtest.sh 9 | $ ./cleanup_viewtest.sh 10 | -------------------------------------------------------------------------------- /tests/views/cleanup_viewtest.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | DATABASE_URL="postgresql://localhost/django_postgres_viewtest" 4 | rm -Rf test__views/ 5 | DROP_CMD=$(psql -P format=unaligned -P tuples_only -d "$DATABASE_URL" -c "select 'DROP TABLE \"' || array_to_string(array_agg(tablename), '\", \"') || '\" CASCADE;' from pg_tables where schemaname = 'public';") 6 | echo "$DROP_CMD" 7 | psql -d "$DATABASE_URL" -c "$DROP_CMD" 8 | -------------------------------------------------------------------------------- /tests/views/do_viewtest.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e # die on errors 4 | set -x # echo commands as they run 5 | 6 | django-admin.py startproject test__views 7 | cd test__views 8 | 9 | virtualenv .venv 10 | . .venv/bin/activate 11 | pip install -r /dev/stdin <> test__views/settings.py < view_myapp/models.py <