├── .gitignore ├── .travis.yml ├── LICENSE ├── README.rst ├── aggregate_if.py ├── runtests.py ├── setup.py ├── tests ├── __init__.py ├── aggregation │ ├── __init__.py │ ├── fixtures │ │ └── aggregation.json │ ├── models.py │ └── tests.py ├── test_mysql.py ├── test_postgres.py └── test_sqlite.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *.swp 3 | *.swo 4 | *.kpf 5 | *.egg-info/ 6 | .idea 7 | .tox 8 | tmp/ 9 | dist/ 10 | .DS_Store 11 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - "2.7" 4 | - "3.4" 5 | services: 6 | - mysql 7 | - postgresql 8 | env: 9 | - DJANGO=1.4 DB=sqlite 10 | - DJANGO=1.4 DB=mysql 11 | - DJANGO=1.4 DB=postgres 12 | - DJANGO=1.5 DB=sqlite 13 | - DJANGO=1.5 DB=mysql 14 | - DJANGO=1.5 DB=postgres 15 | - DJANGO=1.6 DB=sqlite 16 | - DJANGO=1.6 DB=mysql 17 | - DJANGO=1.6 DB=postgres 18 | - DJANGO=1.7 DB=sqlite 19 | - DJANGO=1.7 DB=mysql 20 | - DJANGO=1.7 DB=postgres 21 | 22 | matrix: 23 | exclude: 24 | - python: "3.4" 25 | env: DJANGO=1.4 DB=sqlite 26 | - python: "3.4" 27 | env: DJANGO=1.4 DB=mysql 28 | - python: "3.4" 29 | env: DJANGO=1.4 DB=postgres 30 | - python: "3.4" 31 | env: DJANGO=1.5 DB=sqlite 32 | - python: "3.4" 33 | env: DJANGO=1.5 DB=mysql 34 | - python: "3.4" 35 | env: DJANGO=1.5 DB=postgres 36 | - python: "3.4" 37 | env: DJANGO=1.6 DB=mysql 38 | - python: "3.4" 39 | env: DJANGO=1.7 DB=mysql 40 | 41 | before_script: 42 | - mysql -e 'create database aggregation;' 43 | - psql -c 'create database aggregation;' -U postgres 44 | install: 45 | - pip install six 46 | - if [ "$DB" == "mysql" ]; then pip install mysql-python; fi 47 | - if [ "$DB" == "postgres" ]; then pip install psycopg2; fi 48 | - pip install -q Django==$DJANGO --use-mirrors 49 | script: 50 | - ./runtests.py --settings=tests.test_$DB 51 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License 2 | 3 | Copyright (c) 2012 Henrique Bastos 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | Django Aggregate If: Condition aggregates for Django 2 | ==================================================== 3 | 4 | .. image:: https://travis-ci.org/henriquebastos/django-aggregate-if.png?branch=master 5 | :target: https://travis-ci.org/henriquebastos/django-aggregate-if 6 | :alt: Test Status 7 | 8 | .. image:: https://landscape.io/github/henriquebastos/django-aggregate-if/master/landscape.png 9 | :target: https://landscape.io/github/henriquebastos/django-aggregate-if/master 10 | :alt: Code Helth 11 | 12 | .. image:: https://pypip.in/v/django-aggregate-if/badge.png 13 | :target: https://crate.io/packages/django-aggregate-if/ 14 | :alt: Latest PyPI version 15 | 16 | .. image:: https://pypip.in/d/django-aggregate-if/badge.png 17 | :target: https://crate.io/packages/django-aggregate-if/ 18 | :alt: Number of PyPI downloads 19 | 20 | *Aggregate-if* adds conditional aggregates to Django. 21 | 22 | Conditional aggregates can help you reduce the ammount of queries to obtain 23 | aggregated information, like statistics for example. 24 | 25 | Imagine you have a model ``Offer`` like this one: 26 | 27 | .. code-block:: python 28 | 29 | class Offer(models.Model): 30 | sponsor = models.ForeignKey(User) 31 | price = models.DecimalField(max_digits=9, decimal_places=2) 32 | status = models.CharField(max_length=30) 33 | expire_at = models.DateField(null=True, blank=True) 34 | created_at = models.DateTimeField(auto_now_add=True) 35 | updated_at = models.DateTimeField(auto_now=True) 36 | 37 | OPEN = "OPEN" 38 | REVOKED = "REVOKED" 39 | PAID = "PAID" 40 | 41 | Let's say you want to know: 42 | 43 | #. How many offers exists in total; 44 | #. How many of them are OPEN, REVOKED or PAID; 45 | #. How much money was offered in total; 46 | #. How much money is in OPEN, REVOKED and PAID offers; 47 | 48 | To get these informations, you could query: 49 | 50 | .. code-block:: python 51 | 52 | from django.db.models import Count, Sum 53 | 54 | Offer.objects.count() 55 | Offer.objects.filter(status=Offer.OPEN).aggregate(Count('pk')) 56 | Offer.objects.filter(status=Offer.REVOKED).aggregate(Count('pk')) 57 | Offer.objects.filter(status=Offer.PAID).aggregate(Count('pk')) 58 | Offer.objects.aggregate(Sum('price')) 59 | Offer.objects.filter(status=Offer.OPEN).aggregate(Sum('price')) 60 | Offer.objects.filter(status=Offer.REVOKED).aggregate(Sum('price')) 61 | Offer.objects.filter(status=Offer.PAID).aggregate(Sum('price')) 62 | 63 | In this case, **8 queries** were needed to retrieve the desired information. 64 | 65 | With conditional aggregates you can get it all with only **1 query**: 66 | 67 | .. code-block:: python 68 | 69 | from django.db.models import Q 70 | from aggregate_if import Count, Sum 71 | 72 | Offer.objects.aggregate( 73 | pk__count=Count('pk'), 74 | pk__open__count=Count('pk', only=Q(status=Offer.OPEN)), 75 | pk__revoked__count=Count('pk', only=Q(status=Offer.REVOKED)), 76 | pk__paid__count=Count('pk', only=Q(status=Offer.PAID)), 77 | pk__sum=Sum('price'), 78 | pk__open__sum=Sum('price', only=Q(status=Offer.OPEN)), 79 | pk__revoked__sum=Sum('price'), only=Q(status=Offer.REVOKED)), 80 | pk__paid__sum=Sum('price'), only=Q(status=Offer.PAID)) 81 | ) 82 | 83 | Installation 84 | ------------ 85 | 86 | *Aggregate-if* works with Django 1.4, 1.5, 1.6 and 1.7. 87 | 88 | To install it, simply: 89 | 90 | .. code-block:: bash 91 | 92 | $ pip install django-aggregate-if 93 | 94 | Inspiration 95 | ----------- 96 | 97 | There is a 5 years old `ticket 11305`_ that will (*hopefully*) implement this feature into 98 | Django 1.8. 99 | 100 | Using Django 1.6, I still wanted to avoid creating custom queries for very simple 101 | conditional aggregations. So I've cherry picked those ideas and others from the 102 | internet and built this library. 103 | 104 | This library uses the same API and tests proposed on `ticket 11305`_, so when the 105 | new feature is available you can easily replace ``django-aggregate-if``. 106 | 107 | Limitations 108 | ----------- 109 | 110 | Conditions involving joins with aliases are not supported yet. If you want to 111 | help adding this feature, you're welcome to check the `first issue`_. 112 | 113 | Contributors 114 | ------------ 115 | 116 | * `Henrique Bastos `_ 117 | * `Iuri de Silvio `_ 118 | * `Hampus Stjernhav `_ 119 | * `Bradley Martsberger `_ 120 | * `Markus Bertheau `_ 121 | * `end0 `_ 122 | * `Scott Sexton `_ 123 | * `Mauler `_ 124 | * `trbs `_ 125 | 126 | Changelog 127 | --------- 128 | 129 | 0.5 130 | - Support for Django 1.7 131 | 132 | 0.4 133 | - Use tox to run tests. 134 | - Add support for Django 1.6. 135 | - Add support for Python3. 136 | - The ``only`` parameter now freely supports joins independent of the main query. 137 | - Adds support for alias relabeling permitting excludes and updates with aggregates filtered on remote foreign key relations. 138 | 139 | 0.3.1 140 | - Fix quotation escaping. 141 | - Fix boolean casts on Postgres. 142 | 143 | 0.2 144 | - Fix postgres issue with LIKE conditions. 145 | 146 | 0.1 147 | - Initial release. 148 | 149 | 150 | License 151 | ======= 152 | 153 | The MIT License. 154 | 155 | .. _ticket 11305: https://code.djangoproject.com/ticket/11305 156 | .. _first issue: https://github.com/henriquebastos/django-aggregate-if/issues/1 157 | -------------------------------------------------------------------------------- /aggregate_if.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | ''' 3 | Implements conditional aggregates. 4 | 5 | This code was based on the work of others found on the internet: 6 | 7 | 1. http://web.archive.org/web/20101115170804/http://www.voteruniverse.com/Members/jlantz/blog/conditional-aggregates-in-django 8 | 2. https://code.djangoproject.com/ticket/11305 9 | 3. https://groups.google.com/forum/?fromgroups=#!topic/django-users/cjzloTUwmS0 10 | 4. https://groups.google.com/forum/?fromgroups=#!topic/django-users/vVprMpsAnPo 11 | ''' 12 | from __future__ import unicode_literals 13 | import six 14 | import django 15 | from django.db.models.aggregates import Aggregate as DjangoAggregate 16 | from django.db.models.sql.aggregates import Aggregate as DjangoSqlAggregate 17 | 18 | 19 | VERSION = django.VERSION[:2] 20 | 21 | 22 | class SqlAggregate(DjangoSqlAggregate): 23 | conditional_template = '%(function)s(CASE WHEN %(condition)s THEN %(field)s ELSE null END)' 24 | 25 | def __init__(self, col, source=None, is_summary=False, condition=None, **extra): 26 | super(SqlAggregate, self).__init__(col, source, is_summary, **extra) 27 | self.condition = condition 28 | 29 | def relabel_aliases(self, change_map): 30 | if VERSION < (1, 7): 31 | super(SqlAggregate, self).relabel_aliases(change_map) 32 | if self.has_condition: 33 | condition_change_map = dict((k, v) for k, v in \ 34 | change_map.items() if k in self.condition.query.alias_map 35 | ) 36 | self.condition.query.change_aliases(condition_change_map) 37 | 38 | def relabeled_clone(self, change_map): 39 | self.relabel_aliases(change_map) 40 | return super(SqlAggregate, self).relabeled_clone(change_map) 41 | 42 | def as_sql(self, qn, connection): 43 | if self.has_condition: 44 | self.sql_template = self.conditional_template 45 | self.extra['condition'] = self._condition_as_sql(qn, connection) 46 | 47 | return super(SqlAggregate, self).as_sql(qn, connection) 48 | 49 | @property 50 | def has_condition(self): 51 | # Warning: bool(QuerySet) will hit the database 52 | return self.condition is not None 53 | 54 | def _condition_as_sql(self, qn, connection): 55 | ''' 56 | Return sql for condition. 57 | ''' 58 | def escape(value): 59 | if isinstance(value, bool): 60 | value = str(int(value)) 61 | if isinstance(value, six.string_types): 62 | # Escape params used with LIKE 63 | if '%' in value: 64 | value = value.replace('%', '%%') 65 | # Escape single quotes 66 | if "'" in value: 67 | value = value.replace("'", "''") 68 | # Add single quote to text values 69 | value = "'" + value + "'" 70 | return value 71 | 72 | sql, param = self.condition.query.where.as_sql(qn, connection) 73 | param = map(escape, param) 74 | 75 | return sql % tuple(param) 76 | 77 | 78 | class SqlSum(SqlAggregate): 79 | sql_function = 'SUM' 80 | 81 | 82 | class SqlCount(SqlAggregate): 83 | is_ordinal = True 84 | sql_function = 'COUNT' 85 | sql_template = '%(function)s(%(distinct)s%(field)s)' 86 | conditional_template = '%(function)s(%(distinct)sCASE WHEN %(condition)s THEN %(field)s ELSE null END)' 87 | 88 | def __init__(self, col, distinct=False, **extra): 89 | super(SqlCount, self).__init__(col, distinct=distinct and 'DISTINCT ' or '', **extra) 90 | 91 | 92 | class SqlAvg(SqlAggregate): 93 | is_computed = True 94 | sql_function = 'AVG' 95 | 96 | 97 | class SqlMax(SqlAggregate): 98 | sql_function = 'MAX' 99 | 100 | 101 | class SqlMin(SqlAggregate): 102 | sql_function = 'MIN' 103 | 104 | 105 | class Aggregate(DjangoAggregate): 106 | def __init__(self, lookup, only=None, **extra): 107 | super(Aggregate, self).__init__(lookup, **extra) 108 | self.only = only 109 | self.condition = None 110 | 111 | def _get_fields_from_Q(self, q): 112 | fields = [] 113 | for child in q.children: 114 | if hasattr(child, 'children'): 115 | fields.extend(self._get_fields_from_Q(child)) 116 | else: 117 | fields.append(child) 118 | return fields 119 | 120 | def add_to_query(self, query, alias, col, source, is_summary): 121 | if self.only: 122 | self.condition = query.model._default_manager.filter(self.only) 123 | for child in self._get_fields_from_Q(self.only): 124 | field_list = child[0].split('__') 125 | # Pop off the last field if it's a query term ('gte', 'contains', 'isnull', etc.) 126 | if field_list[-1] in query.query_terms: 127 | field_list.pop() 128 | # setup_joins have different returns in Django 1.5 and 1.6, but the order of what we need remains. 129 | result = query.setup_joins(field_list, query.model._meta, query.get_initial_alias(), None) 130 | join_list = result[3] 131 | 132 | fname = 'promote_alias_chain' if VERSION < (1, 5) else 'promote_joins' 133 | args = (join_list, True) if VERSION < (1, 7) else (join_list,) 134 | 135 | promote = getattr(query, fname) 136 | promote(*args) 137 | 138 | aggregate = self.sql_klass(col, source=source, is_summary=is_summary, condition=self.condition, **self.extra) 139 | query.aggregates[alias] = aggregate 140 | 141 | 142 | class Sum(Aggregate): 143 | name = 'Sum' 144 | sql_klass = SqlSum 145 | 146 | 147 | class Count(Aggregate): 148 | name = 'Count' 149 | sql_klass = SqlCount 150 | 151 | 152 | class Avg(Aggregate): 153 | name = 'Avg' 154 | sql_klass = SqlAvg 155 | 156 | 157 | class Max(Aggregate): 158 | name = 'Max' 159 | sql_klass = SqlMax 160 | 161 | 162 | class Min(Aggregate): 163 | name = 'Min' 164 | sql_klass = SqlMin 165 | -------------------------------------------------------------------------------- /runtests.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import os 4 | import sys 5 | from optparse import OptionParser 6 | 7 | 8 | def parse_args(): 9 | parser = OptionParser() 10 | parser.add_option('-s', '--settings', help='Define settings.') 11 | parser.add_option('-t', '--unittest', help='Define which test to run. Default all.') 12 | options, args = parser.parse_args() 13 | 14 | if not options.settings: 15 | parser.print_help() 16 | sys.exit(1) 17 | 18 | if not options.unittest: 19 | options.unittest = ['aggregation'] 20 | 21 | return options 22 | 23 | 24 | def get_runner(settings_module): 25 | ''' 26 | Asks Django for the TestRunner defined in settings or the default one. 27 | ''' 28 | os.environ['DJANGO_SETTINGS_MODULE'] = settings_module 29 | 30 | import django 31 | from django.test.utils import get_runner 32 | from django.conf import settings 33 | 34 | if hasattr(django, 'setup'): 35 | django.setup() 36 | 37 | return get_runner(settings) 38 | 39 | 40 | def runtests(): 41 | options = parse_args() 42 | TestRunner = get_runner(options.settings) 43 | runner = TestRunner(verbosity=1, interactive=True, failfast=False) 44 | sys.exit(runner.run_tests([])) 45 | 46 | 47 | if __name__ == '__main__': 48 | runtests() 49 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | from setuptools import setup 3 | import os 4 | 5 | 6 | setup(name='django-aggregate-if', 7 | version='0.5', 8 | description='Conditional aggregates for Django, just like the famous SumIf in Excel.', 9 | long_description=open(os.path.join(os.path.dirname(__file__), "README.rst")).read(), 10 | author="Henrique Bastos", author_email="henrique@bastos.net", 11 | license="MIT", 12 | py_modules=['aggregate_if'], 13 | install_requires=[ 14 | 'six>=1.6.1', 15 | ], 16 | zip_safe=False, 17 | platforms='any', 18 | include_package_data=True, 19 | classifiers=[ 20 | 'Development Status :: 5 - Production/Stable', 21 | 'Framework :: Django', 22 | 'Intended Audience :: Developers', 23 | 'License :: OSI Approved :: MIT License', 24 | 'Natural Language :: English', 25 | 'Operating System :: OS Independent', 26 | 'Programming Language :: Python', 27 | 'Programming Language :: Python :: 2.7', 28 | 'Programming Language :: Python :: 3', 29 | 'Topic :: Database', 30 | 'Topic :: Software Development :: Libraries', 31 | ], 32 | url='http://github.com/henriquebastos/django-aggregate-if/', 33 | ) 34 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/henriquebastos/django-aggregate-if/588c1487bc88a8996d4ee9c2c9d50fa4a4484872/tests/__init__.py -------------------------------------------------------------------------------- /tests/aggregation/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/henriquebastos/django-aggregate-if/588c1487bc88a8996d4ee9c2c9d50fa4a4484872/tests/aggregation/__init__.py -------------------------------------------------------------------------------- /tests/aggregation/fixtures/aggregation.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "pk": 1, 4 | "model": "aggregation.publisher", 5 | "fields": { 6 | "name": "Apress", 7 | "num_awards": 3 8 | } 9 | }, 10 | { 11 | "pk": 2, 12 | "model": "aggregation.publisher", 13 | "fields": { 14 | "name": "Sams", 15 | "num_awards": 1 16 | } 17 | }, 18 | { 19 | "pk": 3, 20 | "model": "aggregation.publisher", 21 | "fields": { 22 | "name": "Prentice Hall", 23 | "num_awards": 7 24 | } 25 | }, 26 | { 27 | "pk": 4, 28 | "model": "aggregation.publisher", 29 | "fields": { 30 | "name": "Morgan Kaufmann", 31 | "num_awards": 9 32 | } 33 | }, 34 | { 35 | "pk": 5, 36 | "model": "aggregation.publisher", 37 | "fields": { 38 | "name": "Jonno's House of Books", 39 | "num_awards": 0 40 | } 41 | }, 42 | { 43 | "pk": 1, 44 | "model": "aggregation.book", 45 | "fields": { 46 | "publisher": 1, 47 | "isbn": "159059725", 48 | "name": "The Definitive Guide to Django: Web Development Done Right", 49 | "price": "30.00", 50 | "rating": 4.5, 51 | "authors": [1, 2], 52 | "contact": 1, 53 | "pages": 447, 54 | "pubdate": "2007-12-6" 55 | } 56 | }, 57 | { 58 | "pk": 2, 59 | "model": "aggregation.book", 60 | "fields": { 61 | "publisher": 2, 62 | "isbn": "067232959", 63 | "name": "Sams Teach Yourself Django in 24 Hours", 64 | "price": "23.09", 65 | "rating": 3.0, 66 | "authors": [3], 67 | "contact": 3, 68 | "pages": 528, 69 | "pubdate": "2008-3-3" 70 | } 71 | }, 72 | { 73 | "pk": 3, 74 | "model": "aggregation.book", 75 | "fields": { 76 | "publisher": 1, 77 | "isbn": "159059996", 78 | "name": "Practical Django Projects", 79 | "price": "29.69", 80 | "rating": 4.0, 81 | "authors": [4], 82 | "contact": 4, 83 | "pages": 300, 84 | "pubdate": "2008-6-23" 85 | } 86 | }, 87 | { 88 | "pk": 4, 89 | "model": "aggregation.book", 90 | "fields": { 91 | "publisher": 3, 92 | "isbn": "013235613", 93 | "name": "Python Web Development with Django", 94 | "price": "29.69", 95 | "rating": 4.0, 96 | "authors": [5, 6, 7], 97 | "contact": 5, 98 | "pages": 350, 99 | "pubdate": "2008-11-3" 100 | } 101 | }, 102 | { 103 | "pk": 5, 104 | "model": "aggregation.book", 105 | "fields": { 106 | "publisher": 3, 107 | "isbn": "013790395", 108 | "name": "Artificial Intelligence: A Modern Approach", 109 | "price": "82.80", 110 | "rating": 4.0, 111 | "authors": [8, 9], 112 | "contact": 8, 113 | "pages": 1132, 114 | "pubdate": "1995-1-15" 115 | } 116 | }, 117 | { 118 | "pk": 6, 119 | "model": "aggregation.book", 120 | "fields": { 121 | "publisher": 4, 122 | "isbn": "155860191", 123 | "name": "Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp", 124 | "price": "75.00", 125 | "rating": 5.0, 126 | "authors": [8], 127 | "contact": 8, 128 | "pages": 946, 129 | "pubdate": "1991-10-15" 130 | } 131 | }, 132 | { 133 | "pk": 1, 134 | "model": "aggregation.store", 135 | "fields": { 136 | "books": [1, 2, 3, 4, 5, 6], 137 | "name": "Amazon.com", 138 | "original_opening": "1994-4-23 9:17:42", 139 | "friday_night_closing": "23:59:59" 140 | } 141 | }, 142 | { 143 | "pk": 2, 144 | "model": "aggregation.store", 145 | "fields": { 146 | "books": [1, 3, 5, 6], 147 | "name": "Books.com", 148 | "original_opening": "2001-3-15 11:23:37", 149 | "friday_night_closing": "23:59:59" 150 | } 151 | }, 152 | { 153 | "pk": 3, 154 | "model": "aggregation.store", 155 | "fields": { 156 | "books": [3, 4, 6], 157 | "name": "Mamma and Pappa's Books", 158 | "original_opening": "1945-4-25 16:24:14", 159 | "friday_night_closing": "21:30:00", 160 | "has_coffee": true 161 | } 162 | }, 163 | { 164 | "pk": 1, 165 | "model": "aggregation.author", 166 | "fields": { 167 | "age": 34, 168 | "friends": [2, 4], 169 | "name": "Adrian Holovaty" 170 | } 171 | }, 172 | { 173 | "pk": 2, 174 | "model": "aggregation.author", 175 | "fields": { 176 | "age": 35, 177 | "friends": [1, 7], 178 | "name": "Jacob Kaplan-Moss" 179 | } 180 | }, 181 | { 182 | "pk": 3, 183 | "model": "aggregation.author", 184 | "fields": { 185 | "age": 45, 186 | "friends": [], 187 | "name": "Brad Dayley" 188 | } 189 | }, 190 | { 191 | "pk": 4, 192 | "model": "aggregation.author", 193 | "fields": { 194 | "age": 29, 195 | "friends": [1], 196 | "name": "James Bennett" 197 | } 198 | }, 199 | { 200 | "pk": 5, 201 | "model": "aggregation.author", 202 | "fields": { 203 | "age": 37, 204 | "friends": [6, 7], 205 | "name": "Jeffrey Forcier" 206 | } 207 | }, 208 | { 209 | "pk": 6, 210 | "model": "aggregation.author", 211 | "fields": { 212 | "age": 29, 213 | "friends": [5, 7], 214 | "name": "Paul Bissex" 215 | } 216 | }, 217 | { 218 | "pk": 7, 219 | "model": "aggregation.author", 220 | "fields": { 221 | "age": 25, 222 | "friends": [2, 5, 6], 223 | "name": "Wesley J. Chun" 224 | } 225 | }, 226 | { 227 | "pk": 8, 228 | "model": "aggregation.author", 229 | "fields": { 230 | "age": 57, 231 | "friends": [9], 232 | "name": "Peter Norvig" 233 | } 234 | }, 235 | { 236 | "pk": 9, 237 | "model": "aggregation.author", 238 | "fields": { 239 | "age": 46, 240 | "friends": [8], 241 | "name": "Stuart Russell" 242 | } 243 | } 244 | ] 245 | -------------------------------------------------------------------------------- /tests/aggregation/models.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | from __future__ import unicode_literals 3 | from django.db import models 4 | 5 | 6 | class Author(models.Model): 7 | name = models.CharField(max_length=100) 8 | age = models.IntegerField() 9 | friends = models.ManyToManyField('self', blank=True) 10 | 11 | def __unicode__(self): 12 | return self.name 13 | 14 | class Publisher(models.Model): 15 | name = models.CharField(max_length=255) 16 | num_awards = models.IntegerField() 17 | 18 | def __unicode__(self): 19 | return self.name 20 | 21 | class Book(models.Model): 22 | isbn = models.CharField(max_length=9) 23 | name = models.CharField(max_length=255) 24 | pages = models.IntegerField() 25 | rating = models.FloatField() 26 | price = models.DecimalField(decimal_places=2, max_digits=6) 27 | authors = models.ManyToManyField(Author) 28 | contact = models.ForeignKey(Author, related_name='book_contact_set') 29 | publisher = models.ForeignKey(Publisher) 30 | pubdate = models.DateField() 31 | 32 | def __unicode__(self): 33 | return self.name 34 | 35 | class Store(models.Model): 36 | name = models.CharField(max_length=255) 37 | books = models.ManyToManyField(Book) 38 | original_opening = models.DateTimeField() 39 | friday_night_closing = models.TimeField() 40 | has_coffee = models.BooleanField(default=False) 41 | 42 | def __unicode__(self): 43 | return self.name 44 | 45 | -------------------------------------------------------------------------------- /tests/aggregation/tests.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import, unicode_literals 2 | 3 | import datetime 4 | from decimal import Decimal 5 | 6 | from django.db.models import Q, F 7 | from django.test import TestCase 8 | 9 | try: 10 | # Django < 1.7 11 | from django.test import Approximate 12 | except ImportError as e: 13 | # Django >= 1.7 14 | from django.test.utils import Approximate 15 | 16 | 17 | from aggregate_if import Sum, Count, Avg, Max, Min 18 | 19 | from .models import Author, Publisher, Book, Store 20 | 21 | 22 | class BaseAggregateTestCase(TestCase): 23 | fixtures = ["aggregation.json"] 24 | 25 | def test_empty_aggregate(self): 26 | self.assertEqual(Author.objects.all().aggregate(), {}) 27 | 28 | def test_quote_escape(self): 29 | vals = Author.objects.all().aggregate(Count("id", only=Q(name="nobody'"))) 30 | self.assertEqual(vals['id__count'], 0) 31 | 32 | def test_single_aggregate(self): 33 | vals = Author.objects.aggregate(Avg("age")) 34 | self.assertEqual(vals, {"age__avg": Approximate(37.4, places=1)}) 35 | vals = Author.objects.aggregate(Sum("age", only=Q(age__gt=29))) 36 | self.assertEqual(vals, {"age__sum": 254}) 37 | vals = Author.objects.extra(select={'testparams':'age < %s'}, select_params=[0])\ 38 | .aggregate(Sum("age", only=Q(age__gt=29))) 39 | self.assertEqual(vals, {"age__sum": 254}) 40 | vals = Author.objects.aggregate(Sum("age", only=Q(name__icontains='jaco')|Q(name__icontains='adrian'))) 41 | self.assertEqual(vals, {"age__sum": 69}) 42 | 43 | def test_multiple_aggregates(self): 44 | vals = Author.objects.aggregate(Sum("age"), Avg("age")) 45 | self.assertEqual(vals, {"age__sum": 337, "age__avg": Approximate(37.4, places=1)}) 46 | vals = Author.objects.aggregate(Sum("age", only=Q(age__gt=29)), Avg("age")) 47 | self.assertEqual(vals, {"age__sum": 254, "age__avg": Approximate(37.4, places=1)}) 48 | 49 | def test_filter_aggregate(self): 50 | vals = Author.objects.filter(age__gt=29).aggregate(Sum("age")) 51 | self.assertEqual(len(vals), 1) 52 | self.assertEqual(vals["age__sum"], 254) 53 | vals = Author.objects.filter(age__gt=29).aggregate(Sum("age", only=Q(age__lt=29))) 54 | # If there are no matching aggregates, then None, not 0 is the answer. 55 | self.assertEqual(vals["age__sum"], None) 56 | 57 | def test_condition_with_bool_value(self): 58 | vals = Store.objects.all().aggregate(with_coffee=Count("books", only=Q(has_coffee=True))) 59 | self.assertEqual(len(vals), 1) 60 | self.assertEqual(vals["with_coffee"], 3) 61 | 62 | def test_related_aggregate(self): 63 | vals = Author.objects.aggregate(Avg("friends__age")) 64 | self.assertEqual(len(vals), 1) 65 | self.assertAlmostEqual(vals["friends__age__avg"], 34.07, places=2) 66 | 67 | vals = Author.objects.aggregate(Avg("friends__age", only=Q(age__lt=29))) 68 | self.assertEqual(len(vals), 1) 69 | self.assertAlmostEqual(vals["friends__age__avg"], 33.67, places=2) 70 | vals2 = Author.objects.filter(age__lt=29).aggregate(Avg("friends__age")) 71 | self.assertEqual(vals, vals2) 72 | 73 | vals = Author.objects.aggregate(Avg("friends__age", only=Q(friends__age__lt=35))) 74 | self.assertEqual(len(vals), 1) 75 | self.assertAlmostEqual(vals["friends__age__avg"], 28.75, places=2) 76 | 77 | # The average age of author's friends, whose age is lower than the authors age. 78 | vals = Author.objects.aggregate(Avg("friends__age", only=Q(friends__age__lt=F('age')))) 79 | self.assertEqual(len(vals), 1) 80 | self.assertAlmostEqual(vals["friends__age__avg"], 30.43, places=2) 81 | 82 | 83 | vals = Book.objects.filter(rating__lt=4.5).aggregate(Avg("authors__age")) 84 | self.assertEqual(len(vals), 1) 85 | self.assertAlmostEqual(vals["authors__age__avg"], 38.2857, places=2) 86 | 87 | vals = Author.objects.all().filter(name__contains="a").aggregate(Avg("book__rating")) 88 | self.assertEqual(len(vals), 1) 89 | self.assertEqual(vals["book__rating__avg"], 4.0) 90 | 91 | vals = Book.objects.aggregate(Sum("publisher__num_awards")) 92 | self.assertEqual(len(vals), 1) 93 | self.assertEqual(vals["publisher__num_awards__sum"], 30) 94 | 95 | vals = Publisher.objects.aggregate(Sum("book__price")) 96 | self.assertEqual(len(vals), 1) 97 | self.assertEqual(vals["book__price__sum"], Decimal("270.27")) 98 | 99 | def test_aggregate_multi_join(self): 100 | vals = Store.objects.aggregate(Max("books__authors__age")) 101 | self.assertEqual(len(vals), 1) 102 | self.assertEqual(vals["books__authors__age__max"], 57) 103 | 104 | vals = Store.objects.aggregate(Max("books__authors__age", only=Q(books__authors__age__lt=56))) 105 | self.assertEqual(len(vals), 1) 106 | self.assertEqual(vals["books__authors__age__max"], 46) 107 | 108 | vals = Author.objects.aggregate(Min("book__publisher__num_awards")) 109 | self.assertEqual(len(vals), 1) 110 | self.assertEqual(vals["book__publisher__num_awards__min"], 1) 111 | 112 | def test_aggregate_alias(self): 113 | vals = Store.objects.filter(name="Amazon.com").aggregate(amazon_mean=Avg("books__rating")) 114 | self.assertEqual(len(vals), 1) 115 | self.assertAlmostEqual(vals["amazon_mean"], 4.08, places=2) 116 | 117 | def test_annotate_basic(self): 118 | self.assertQuerysetEqual( 119 | Book.objects.annotate().order_by('pk'), [ 120 | "The Definitive Guide to Django: Web Development Done Right", 121 | "Sams Teach Yourself Django in 24 Hours", 122 | "Practical Django Projects", 123 | "Python Web Development with Django", 124 | "Artificial Intelligence: A Modern Approach", 125 | "Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp" 126 | ], 127 | lambda b: b.name 128 | ) 129 | 130 | books = Book.objects.annotate(mean_age=Avg("authors__age")) 131 | b = books.get(pk=1) 132 | self.assertEqual( 133 | b.name, 134 | 'The Definitive Guide to Django: Web Development Done Right' 135 | ) 136 | self.assertEqual(b.mean_age, 34.5) 137 | 138 | ''' 139 | def test_f_expression(self): 140 | publishers = Publisher.objects.annotate(avg_rating=Avg(F('book__rating') - 0)) 141 | publishers = publishers.values_list('id', 'avg_rating').order_by('id') 142 | self.assertEqual(list(publishers), [(1, 4.25), (2, 3.0), (3, 4.0), (4, 5.0), (5, None)]) 143 | ''' 144 | 145 | def test_only_condition_with_remote_fk_join(self): 146 | publishers = Publisher.objects.annotate(mean_price=Avg('book__price', only=Q(book__price__gte=30)), 147 | mean_rating=Avg('book__rating', only=Q(book__price__gte=0))) 148 | p = publishers.get(pk=1) 149 | self.assertEqual(p.mean_price, 30.0) 150 | self.assertEqual(p.mean_rating, 4.25) 151 | ''' 152 | def test_only_condition_with_m2m_join(self): 153 | # Test extra-select 154 | books = Book.objects.annotate(mean_age=Avg("authors__age", only=Q(authors__age__gte=2)), 155 | mean_age2=Avg('authors__age', only=Q(authors__age__gte=0))) 156 | books = books.extra(select={'testparams': 'publisher_id = %s'}, select_params=[1]) 157 | b = books.get(pk=1) 158 | self.assertEqual(b.mean_age, 34.5) 159 | self.assertEqual(b.mean_age2, 34.5) 160 | self.assertEqual(b.testparams, True) 161 | ''' 162 | def test_relabel_aliases(self): 163 | # Test relabel_aliases 164 | excluded_authors = Author.objects.annotate(book_rating=Min('book__rating', only=Q(pk__gte=1))) 165 | excluded_authors = excluded_authors.filter(book_rating__lt=0) 166 | books = Book.objects.exclude(authors__in=excluded_authors).annotate(mean_age=Avg('authors__age')) 167 | b = books.get(pk=1) 168 | self.assertEqual(b.mean_age, 34.5) 169 | ''' 170 | def test_joins_in_f(self): 171 | # Test joins in F-based annotation 172 | books = Book.objects.annotate(oldest=Max(F('authors__age'))) 173 | books = books.values_list('rating', 'oldest').order_by('rating', 'oldest') 174 | self.assertEqual( 175 | list(books), 176 | [(3.0, 45), (4.0, 29), (4.0, 37), (4.0, 57), (4.5, 35), (5.0, 57)] 177 | ) 178 | ''' 179 | 180 | def test_annotate_m2m(self): 181 | books = Book.objects.filter(rating__lt=4.5).annotate(Avg("authors__age")).order_by("name") 182 | self.assertQuerysetEqual( 183 | books, [ 184 | ('Artificial Intelligence: A Modern Approach', 51.5), 185 | ('Practical Django Projects', 29.0), 186 | ('Python Web Development with Django', Approximate(30.3, places=1)), 187 | ('Sams Teach Yourself Django in 24 Hours', 45.0) 188 | ], 189 | lambda b: (b.name, b.authors__age__avg), 190 | ) 191 | 192 | books = Book.objects.annotate(num_authors=Count("authors")).order_by("name") 193 | self.assertQuerysetEqual( 194 | books, [ 195 | ('Artificial Intelligence: A Modern Approach', 2), 196 | ('Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp', 1), 197 | ('Practical Django Projects', 1), 198 | ('Python Web Development with Django', 3), 199 | ('Sams Teach Yourself Django in 24 Hours', 1), 200 | ('The Definitive Guide to Django: Web Development Done Right', 2) 201 | ], 202 | lambda b: (b.name, b.num_authors) 203 | ) 204 | 205 | def test_backwards_m2m_annotate(self): 206 | authors = Author.objects.filter(name__contains="a").annotate(Avg("book__rating")).order_by("name") 207 | self.assertQuerysetEqual( 208 | authors, [ 209 | ('Adrian Holovaty', 4.5), 210 | ('Brad Dayley', 3.0), 211 | ('Jacob Kaplan-Moss', 4.5), 212 | ('James Bennett', 4.0), 213 | ('Paul Bissex', 4.0), 214 | ('Stuart Russell', 4.0) 215 | ], 216 | lambda a: (a.name, a.book__rating__avg) 217 | ) 218 | 219 | authors = Author.objects.annotate(num_books=Count("book")).order_by("name") 220 | self.assertQuerysetEqual( 221 | authors, [ 222 | ('Adrian Holovaty', 1), 223 | ('Brad Dayley', 1), 224 | ('Jacob Kaplan-Moss', 1), 225 | ('James Bennett', 1), 226 | ('Jeffrey Forcier', 1), 227 | ('Paul Bissex', 1), 228 | ('Peter Norvig', 2), 229 | ('Stuart Russell', 1), 230 | ('Wesley J. Chun', 1) 231 | ], 232 | lambda a: (a.name, a.num_books) 233 | ) 234 | 235 | def test_reverse_fkey_annotate(self): 236 | books = Book.objects.annotate(Sum("publisher__num_awards")).order_by("name") 237 | self.assertQuerysetEqual( 238 | books, [ 239 | ('Artificial Intelligence: A Modern Approach', 7), 240 | ('Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp', 9), 241 | ('Practical Django Projects', 3), 242 | ('Python Web Development with Django', 7), 243 | ('Sams Teach Yourself Django in 24 Hours', 1), 244 | ('The Definitive Guide to Django: Web Development Done Right', 3) 245 | ], 246 | lambda b: (b.name, b.publisher__num_awards__sum) 247 | ) 248 | 249 | publishers = Publisher.objects.annotate(Sum("book__price")).order_by("name") 250 | self.assertQuerysetEqual( 251 | publishers, [ 252 | ('Apress', Decimal("59.69")), 253 | ("Jonno's House of Books", None), 254 | ('Morgan Kaufmann', Decimal("75.00")), 255 | ('Prentice Hall', Decimal("112.49")), 256 | ('Sams', Decimal("23.09")) 257 | ], 258 | lambda p: (p.name, p.book__price__sum) 259 | ) 260 | 261 | def test_annotate_values(self): 262 | books = list(Book.objects.filter(pk=1).annotate(mean_age=Avg("authors__age")).values()) 263 | self.assertEqual( 264 | books, [ 265 | { 266 | "contact_id": 1, 267 | "id": 1, 268 | "isbn": "159059725", 269 | "mean_age": 34.5, 270 | "name": "The Definitive Guide to Django: Web Development Done Right", 271 | "pages": 447, 272 | "price": Approximate(Decimal("30")), 273 | "pubdate": datetime.date(2007, 12, 6), 274 | "publisher_id": 1, 275 | "rating": 4.5, 276 | } 277 | ] 278 | ) 279 | 280 | books = Book.objects.filter(pk=1).annotate(mean_age=Avg('authors__age')).values('pk', 'isbn', 'mean_age') 281 | self.assertEqual( 282 | list(books), [ 283 | { 284 | "pk": 1, 285 | "isbn": "159059725", 286 | "mean_age": 34.5, 287 | } 288 | ] 289 | ) 290 | 291 | books = Book.objects.filter(pk=1).annotate(mean_age=Avg("authors__age")).values("name") 292 | self.assertEqual( 293 | list(books), [ 294 | { 295 | "name": "The Definitive Guide to Django: Web Development Done Right" 296 | } 297 | ] 298 | ) 299 | 300 | books = Book.objects.filter(pk=1).values().annotate(mean_age=Avg('authors__age')) 301 | self.assertEqual( 302 | list(books), [ 303 | { 304 | "contact_id": 1, 305 | "id": 1, 306 | "isbn": "159059725", 307 | "mean_age": 34.5, 308 | "name": "The Definitive Guide to Django: Web Development Done Right", 309 | "pages": 447, 310 | "price": Approximate(Decimal("30")), 311 | "pubdate": datetime.date(2007, 12, 6), 312 | "publisher_id": 1, 313 | "rating": 4.5, 314 | } 315 | ] 316 | ) 317 | 318 | books = Book.objects.values("rating").annotate(n_authors=Count("authors__id"), mean_age=Avg("authors__age")).order_by("rating") 319 | self.assertEqual( 320 | list(books), [ 321 | { 322 | "rating": 3.0, 323 | "n_authors": 1, 324 | "mean_age": 45.0, 325 | }, 326 | { 327 | "rating": 4.0, 328 | "n_authors": 6, 329 | "mean_age": Approximate(37.16, places=1) 330 | }, 331 | { 332 | "rating": 4.5, 333 | "n_authors": 2, 334 | "mean_age": 34.5, 335 | }, 336 | { 337 | "rating": 5.0, 338 | "n_authors": 1, 339 | "mean_age": 57.0, 340 | } 341 | ] 342 | ) 343 | 344 | authors = Author.objects.annotate(Avg("friends__age")).order_by("name") 345 | self.assertEqual(len(authors), 9) 346 | self.assertQuerysetEqual( 347 | authors, [ 348 | ('Adrian Holovaty', 32.0), 349 | ('Brad Dayley', None), 350 | ('Jacob Kaplan-Moss', 29.5), 351 | ('James Bennett', 34.0), 352 | ('Jeffrey Forcier', 27.0), 353 | ('Paul Bissex', 31.0), 354 | ('Peter Norvig', 46.0), 355 | ('Stuart Russell', 57.0), 356 | ('Wesley J. Chun', Approximate(33.66, places=1)) 357 | ], 358 | lambda a: (a.name, a.friends__age__avg) 359 | ) 360 | 361 | def test_count(self): 362 | vals = Book.objects.aggregate(Count("rating")) 363 | self.assertEqual(vals, {"rating__count": 6}) 364 | 365 | vals = Book.objects.aggregate(Count("rating", distinct=True)) 366 | self.assertEqual(vals, {"rating__count": 4}) 367 | 368 | def test_fkey_aggregate(self): 369 | explicit = list(Author.objects.annotate(Count('book__id'))) 370 | implicit = list(Author.objects.annotate(Count('book'))) 371 | self.assertEqual(explicit, implicit) 372 | 373 | def test_annotate_ordering(self): 374 | books = Book.objects.values('rating').annotate(oldest=Max('authors__age')).order_by('oldest', 'rating') 375 | self.assertEqual( 376 | list(books), [ 377 | { 378 | "rating": 4.5, 379 | "oldest": 35, 380 | }, 381 | { 382 | "rating": 3.0, 383 | "oldest": 45 384 | }, 385 | { 386 | "rating": 4.0, 387 | "oldest": 57, 388 | }, 389 | { 390 | "rating": 5.0, 391 | "oldest": 57, 392 | } 393 | ] 394 | ) 395 | 396 | books = Book.objects.values("rating").annotate(oldest=Max("authors__age")).order_by("-oldest", "-rating") 397 | self.assertEqual( 398 | list(books), [ 399 | { 400 | "rating": 5.0, 401 | "oldest": 57, 402 | }, 403 | { 404 | "rating": 4.0, 405 | "oldest": 57, 406 | }, 407 | { 408 | "rating": 3.0, 409 | "oldest": 45, 410 | }, 411 | { 412 | "rating": 4.5, 413 | "oldest": 35, 414 | } 415 | ] 416 | ) 417 | 418 | def test_aggregate_annotation(self): 419 | vals = Book.objects.annotate(num_authors=Count("authors__id")).aggregate(Avg("num_authors")) 420 | self.assertEqual(vals, {"num_authors__avg": Approximate(1.66, places=1)}) 421 | 422 | def test_filtering(self): 423 | p = Publisher.objects.create(name='Expensive Publisher', num_awards=0) 424 | Book.objects.create( 425 | name='ExpensiveBook1', 426 | pages=1, 427 | isbn='111', 428 | rating=3.5, 429 | price=Decimal("1000"), 430 | publisher=p, 431 | contact_id=1, 432 | pubdate=datetime.date(2008,12,1) 433 | ) 434 | Book.objects.create( 435 | name='ExpensiveBook2', 436 | pages=1, 437 | isbn='222', 438 | rating=4.0, 439 | price=Decimal("1000"), 440 | publisher=p, 441 | contact_id=1, 442 | pubdate=datetime.date(2008,12,2) 443 | ) 444 | Book.objects.create( 445 | name='ExpensiveBook3', 446 | pages=1, 447 | isbn='333', 448 | rating=4.5, 449 | price=Decimal("35"), 450 | publisher=p, 451 | contact_id=1, 452 | pubdate=datetime.date(2008,12,3) 453 | ) 454 | 455 | publishers = Publisher.objects.annotate(num_books=Count("book__id")).filter(num_books__gt=1).order_by("pk") 456 | self.assertQuerysetEqual( 457 | publishers, [ 458 | "Apress", 459 | "Prentice Hall", 460 | "Expensive Publisher", 461 | ], 462 | lambda p: p.name, 463 | ) 464 | 465 | publishers = Publisher.objects.filter(book__price__lt=Decimal("40.0")).order_by("pk") 466 | self.assertQuerysetEqual( 467 | publishers, [ 468 | "Apress", 469 | "Apress", 470 | "Sams", 471 | "Prentice Hall", 472 | "Expensive Publisher", 473 | ], 474 | lambda p: p.name 475 | ) 476 | 477 | publishers = Publisher.objects.annotate(num_books=Count("book__id")).filter(num_books__gt=1, book__price__lt=Decimal("40.0")).order_by("pk") 478 | self.assertQuerysetEqual( 479 | publishers, [ 480 | "Apress", 481 | "Prentice Hall", 482 | "Expensive Publisher", 483 | ], 484 | lambda p: p.name, 485 | ) 486 | 487 | publishers = Publisher.objects.filter(book__price__lt=Decimal("40.0")).annotate(num_books=Count("book__id")).filter(num_books__gt=1).order_by("pk") 488 | self.assertQuerysetEqual( 489 | publishers, [ 490 | "Apress", 491 | ], 492 | lambda p: p.name 493 | ) 494 | 495 | publishers = Publisher.objects.annotate(num_books=Count("book")).filter(num_books__range=[1, 3]).order_by("pk") 496 | self.assertQuerysetEqual( 497 | publishers, [ 498 | "Apress", 499 | "Sams", 500 | "Prentice Hall", 501 | "Morgan Kaufmann", 502 | "Expensive Publisher", 503 | ], 504 | lambda p: p.name 505 | ) 506 | 507 | publishers = Publisher.objects.annotate(num_books=Count("book")).filter(num_books__range=[1, 2]).order_by("pk") 508 | self.assertQuerysetEqual( 509 | publishers, [ 510 | "Apress", 511 | "Sams", 512 | "Prentice Hall", 513 | "Morgan Kaufmann", 514 | ], 515 | lambda p: p.name 516 | ) 517 | 518 | publishers = Publisher.objects.annotate(num_books=Count("book")).filter(num_books__in=[1, 3]).order_by("pk") 519 | self.assertQuerysetEqual( 520 | publishers, [ 521 | "Sams", 522 | "Morgan Kaufmann", 523 | "Expensive Publisher", 524 | ], 525 | lambda p: p.name, 526 | ) 527 | 528 | publishers = Publisher.objects.annotate(num_books=Count("book")).filter(num_books__isnull=True) 529 | self.assertEqual(len(publishers), 0) 530 | 531 | def test_annotation(self): 532 | vals = Author.objects.filter(pk=1).aggregate(Count("friends__id")) 533 | self.assertEqual(vals, {"friends__id__count": 2}) 534 | 535 | books = Book.objects.annotate(num_authors=Count("authors__name")).filter(num_authors=2).order_by("pk") 536 | self.assertQuerysetEqual( 537 | books, [ 538 | "The Definitive Guide to Django: Web Development Done Right", 539 | "Artificial Intelligence: A Modern Approach", 540 | ], 541 | lambda b: b.name 542 | ) 543 | 544 | authors = Author.objects.annotate(num_friends=Count("friends__id", distinct=True)).filter(num_friends=0).order_by("pk") 545 | self.assertQuerysetEqual( 546 | authors, [ 547 | "Brad Dayley", 548 | ], 549 | lambda a: a.name 550 | ) 551 | 552 | publishers = Publisher.objects.annotate(num_books=Count("book__id")).filter(num_books__gt=1).order_by("pk") 553 | self.assertQuerysetEqual( 554 | publishers, [ 555 | "Apress", 556 | "Prentice Hall", 557 | ], 558 | lambda p: p.name 559 | ) 560 | 561 | publishers = Publisher.objects.filter(book__price__lt=Decimal("40.0")).annotate(num_books=Count("book__id")).filter(num_books__gt=1) 562 | self.assertQuerysetEqual( 563 | publishers, [ 564 | "Apress", 565 | ], 566 | lambda p: p.name 567 | ) 568 | 569 | books = Book.objects.annotate(num_authors=Count("authors__id")).filter(authors__name__contains="Norvig", num_authors__gt=1) 570 | self.assertQuerysetEqual( 571 | books, [ 572 | "Artificial Intelligence: A Modern Approach", 573 | ], 574 | lambda b: b.name 575 | ) 576 | 577 | def test_more_aggregation(self): 578 | a = Author.objects.get(name__contains='Norvig') 579 | b = Book.objects.get(name__contains='Done Right') 580 | b.authors.add(a) 581 | b.save() 582 | 583 | vals = Book.objects.annotate(num_authors=Count("authors__id")).filter(authors__name__contains="Norvig", num_authors__gt=1).aggregate(Avg("rating")) 584 | self.assertEqual(vals, {"rating__avg": 4.25}) 585 | 586 | def test_even_more_aggregate(self): 587 | publishers = Publisher.objects.annotate(earliest_book=Min("book__pubdate")).exclude(earliest_book=None).order_by("earliest_book").values() 588 | self.assertEqual( 589 | list(publishers), [ 590 | { 591 | 'earliest_book': datetime.date(1991, 10, 15), 592 | 'num_awards': 9, 593 | 'id': 4, 594 | 'name': 'Morgan Kaufmann' 595 | }, 596 | { 597 | 'earliest_book': datetime.date(1995, 1, 15), 598 | 'num_awards': 7, 599 | 'id': 3, 600 | 'name': 'Prentice Hall' 601 | }, 602 | { 603 | 'earliest_book': datetime.date(2007, 12, 6), 604 | 'num_awards': 3, 605 | 'id': 1, 606 | 'name': 'Apress' 607 | }, 608 | { 609 | 'earliest_book': datetime.date(2008, 3, 3), 610 | 'num_awards': 1, 611 | 'id': 2, 612 | 'name': 'Sams' 613 | } 614 | ] 615 | ) 616 | 617 | vals = Store.objects.aggregate(Max("friday_night_closing"), Min("original_opening")) 618 | self.assertEqual( 619 | vals, 620 | { 621 | "friday_night_closing__max": datetime.time(23, 59, 59), 622 | "original_opening__min": datetime.datetime(1945, 4, 25, 16, 24, 14), 623 | } 624 | ) 625 | 626 | def test_annotate_values_list(self): 627 | books = Book.objects.filter(pk=1).annotate(mean_age=Avg("authors__age")).values_list("pk", "isbn", "mean_age") 628 | self.assertEqual( 629 | list(books), [ 630 | (1, "159059725", 34.5), 631 | ] 632 | ) 633 | 634 | books = Book.objects.filter(pk=1).annotate(mean_age=Avg("authors__age")).values_list("isbn") 635 | self.assertEqual( 636 | list(books), [ 637 | ('159059725',) 638 | ] 639 | ) 640 | 641 | books = Book.objects.filter(pk=1).annotate(mean_age=Avg("authors__age")).values_list("mean_age") 642 | self.assertEqual( 643 | list(books), [ 644 | (34.5,) 645 | ] 646 | ) 647 | 648 | books = Book.objects.filter(pk=1).annotate(mean_age=Avg("authors__age")).values_list("mean_age", flat=True) 649 | self.assertEqual(list(books), [34.5]) 650 | 651 | books = Book.objects.values_list("price").annotate(count=Count("price")).order_by("-count", "price") 652 | self.assertEqual( 653 | list(books), [ 654 | (Decimal("29.69"), 2), 655 | (Decimal('23.09'), 1), 656 | (Decimal('30'), 1), 657 | (Decimal('75'), 1), 658 | (Decimal('82.8'), 1), 659 | ] 660 | ) 661 | 662 | def test_only_requires_extra_join(self): 663 | publishers = Publisher.objects.annotate(jeff_books=Count('book', only=Q(book__contact__name__icontains='Jeff'))).order_by('id') 664 | self.assertQuerysetEqual( 665 | publishers, [ 666 | ('Apress', 0), 667 | ('Sams', 0), 668 | ('Prentice Hall', 1), 669 | ('Morgan Kaufmann', 0), 670 | ('Jonno\'s House of Books', 0) 671 | ], 672 | lambda b: (b.name, b.jeff_books), 673 | ) 674 | 675 | # Test with compound Q object 676 | # Get publishers annotated with a count of books that are in stores with coffee or named Books.com 677 | q = Q(book__store__has_coffee=True) | Q(book__store__name__icontains="Books.com") 678 | publishers = Publisher.objects.annotate(coffee_books=Count('book', distinct=True, only=q)).order_by('id') 679 | self.assertQuerysetEqual( 680 | publishers, [ 681 | ('Apress', 2), 682 | ('Sams', 0), 683 | ('Prentice Hall', 2), 684 | ('Morgan Kaufmann', 1), 685 | ('Jonno\'s House of Books', 0) 686 | ], 687 | lambda b: (b.name, b.coffee_books), 688 | ) 689 | -------------------------------------------------------------------------------- /tests/test_mysql.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | DATABASES = { 4 | 'default': { 5 | 'ENGINE': 'django.db.backends.mysql', 6 | 'NAME': 'aggregation', 7 | } 8 | } 9 | 10 | INSTALLED_APPS = ( 11 | 'tests.aggregation', 12 | ) 13 | 14 | SITE_ID=1, 15 | 16 | SECRET_KEY='secret' 17 | 18 | MIDDLEWARE_CLASSES = ( 19 | 'django.middleware.common.CommonMiddleware', 20 | 'django.middleware.csrf.CsrfViewMiddleware', 21 | ) -------------------------------------------------------------------------------- /tests/test_postgres.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | DATABASES = { 4 | 'default': { 5 | 'ENGINE': 'django.db.backends.postgresql_psycopg2', 6 | 'NAME': 'aggregation', 7 | } 8 | } 9 | 10 | INSTALLED_APPS = ( 11 | 'tests.aggregation', 12 | ) 13 | 14 | SITE_ID=1, 15 | 16 | SECRET_KEY='secret' 17 | 18 | MIDDLEWARE_CLASSES = ( 19 | 'django.middleware.common.CommonMiddleware', 20 | 'django.middleware.csrf.CsrfViewMiddleware', 21 | ) -------------------------------------------------------------------------------- /tests/test_sqlite.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | DATABASES = { 4 | 'default': { 5 | 'ENGINE': 'django.db.backends.sqlite3', 6 | 'NAME': ':memory:', 7 | } 8 | } 9 | 10 | INSTALLED_APPS = ( 11 | 'tests.aggregation', 12 | ) 13 | 14 | SITE_ID=1, 15 | 16 | SECRET_KEY='secret' 17 | 18 | MIDDLEWARE_CLASSES = ( 19 | 'django.middleware.common.CommonMiddleware', 20 | 'django.middleware.csrf.CsrfViewMiddleware', 21 | ) -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = 3 | py27-django1.4-sqlite, 4 | py27-django1.4-postgres, 5 | py27-django1.4-mysql, 6 | 7 | py27-django1.5-sqlite, 8 | py27-django1.5-postgres, 9 | py27-django1.5-mysql, 10 | 11 | py27-django1.6-sqlite, 12 | py27-django1.6-postgres, 13 | py27-django1.6-mysql, 14 | 15 | py27-django1.7-sqlite, 16 | py27-django1.7-postgres, 17 | py27-django1.7-mysql, 18 | 19 | py34-django1.6-sqlite, 20 | py34-django1.6-postgres, 21 | #py34-django1.6-mysql 22 | 23 | py34-django1.7-sqlite, 24 | py34-django1.7-postgres, 25 | #py34-django1.7-mysql 26 | 27 | [testenv] 28 | whitelist_externals= 29 | mysql 30 | psql 31 | 32 | # Python 2.7 33 | # Django 1.4 34 | [testenv:py27-django1.4-sqlite] 35 | basepython = python2.7 36 | deps = 37 | django==1.4 38 | commands = python runtests.py --settings tests.test_sqlite 39 | 40 | [testenv:py27-django1.4-postgres] 41 | basepython = python2.7 42 | deps = 43 | django==1.4 44 | psycopg2 45 | commands = 46 | psql -c 'create database aggregation;' postgres 47 | python runtests.py --settings tests.test_postgres 48 | psql -c 'drop database aggregation;' postgres 49 | 50 | [testenv:py27-django1.4-mysql] 51 | basepython = python2.7 52 | deps = 53 | django==1.4 54 | mysql-python 55 | commands = 56 | mysql -e 'create database aggregation;' 57 | python runtests.py --settings tests.test_mysql 58 | mysql -e 'drop database aggregation;' 59 | 60 | # Django 1.5 61 | [testenv:py27-django1.5-sqlite] 62 | basepython = python2.7 63 | deps = 64 | django==1.5 65 | commands = python runtests.py --settings tests.test_sqlite 66 | 67 | [testenv:py27-django1.5-postgres] 68 | basepython = python2.7 69 | deps = 70 | django==1.5 71 | psycopg2 72 | commands = 73 | psql -c 'create database aggregation;' postgres 74 | python runtests.py --settings tests.test_postgres 75 | psql -c 'drop database aggregation;' postgres 76 | 77 | [testenv:py27-django1.5-mysql] 78 | basepython = python2.7 79 | deps = 80 | django==1.5 81 | mysql-python 82 | commands = 83 | mysql -e 'create database aggregation;' 84 | python runtests.py --settings tests.test_mysql 85 | mysql -e 'drop database aggregation;' 86 | 87 | # Django 1.6 88 | [testenv:py27-django1.6-sqlite] 89 | basepython = python2.7 90 | deps = 91 | django==1.6 92 | commands = python runtests.py --settings tests.test_sqlite 93 | 94 | [testenv:py27-django1.6-postgres] 95 | basepython = python2.7 96 | deps = 97 | django==1.6 98 | psycopg2 99 | commands = 100 | psql -c 'create database aggregation;' postgres 101 | python runtests.py --settings tests.test_postgres 102 | psql -c 'drop database aggregation;' postgres 103 | 104 | [testenv:py27-django1.6-mysql] 105 | basepython = python2.7 106 | deps = 107 | django==1.6 108 | mysql-python 109 | commands = 110 | mysql -e 'create database aggregation;' 111 | python runtests.py --settings tests.test_mysql 112 | mysql -e 'drop database aggregation;' 113 | 114 | 115 | # Python 2.7 and Django 1.7 116 | [testenv:py27-django1.7-sqlite] 117 | basepython = python2.7 118 | deps = 119 | django==1.7 120 | commands = python runtests.py --settings tests.test_sqlite 121 | 122 | [testenv:py27-django1.7-postgres] 123 | basepython = python2.7 124 | deps = 125 | django==1.7 126 | psycopg2 127 | commands = 128 | psql -c 'create database aggregation;' postgres 129 | python runtests.py --settings tests.test_postgres 130 | psql -c 'drop database aggregation;' postgres 131 | 132 | [testenv:py27-django1.7-mysql] 133 | basepython = python2.7 134 | deps = 135 | django==1.7 136 | mysql-python 137 | commands = 138 | mysql -e 'create database aggregation;' 139 | python runtests.py --settings tests.test_mysql 140 | mysql -e 'drop database aggregation;' 141 | 142 | 143 | # Python 3.4 144 | # Django 1.6 145 | [testenv:py34-django1.6-sqlite] 146 | basepython = python3.4 147 | deps = 148 | django==1.6 149 | commands = python runtests.py --settings tests.test_sqlite 150 | 151 | [testenv:py34-django1.6-postgres] 152 | basepython = python3.4 153 | deps = 154 | django==1.6 155 | psycopg2 156 | commands = 157 | psql -c 'create database aggregation;' postgres 158 | python runtests.py --settings tests.test_postgres 159 | psql -c 'drop database aggregation;' postgres 160 | 161 | [testenv:py34-django1.6-mysql] 162 | basepython = python3.4 163 | deps = 164 | django==1.6 165 | mysql-python3 166 | commands = 167 | mysql -e 'create database aggregation;' 168 | python runtests.py --settings tests.test_mysql 169 | mysql -e 'drop database aggregation;' 170 | 171 | 172 | # Python 3.4 173 | # Django 1.7 174 | [testenv:py34-django1.7-sqlite] 175 | basepython = python3.4 176 | deps = 177 | django==1.7 178 | commands = python runtests.py --settings tests.test_sqlite 179 | 180 | [testenv:py34-django1.7-postgres] 181 | basepython = python3.4 182 | deps = 183 | django==1.7 184 | psycopg2 185 | commands = 186 | psql -c 'create database aggregation;' postgres 187 | python runtests.py --settings tests.test_postgres 188 | psql -c 'drop database aggregation;' postgres 189 | 190 | [testenv:py34-django1.7-mysql] 191 | basepython = python3.4 192 | deps = 193 | django==1.7 194 | mysql-python3 195 | commands = 196 | mysql -e 'create database aggregation;' 197 | python runtests.py --settings tests.test_mysql 198 | mysql -e 'drop database aggregation;' 199 | --------------------------------------------------------------------------------