├── .gitignore ├── .travis.yml ├── CHANGELOG.md ├── LICENSE ├── README.md ├── mpathy ├── __init__.py ├── apps.py ├── fields.py ├── models.py ├── operations.py └── templatetags │ ├── __init__.py │ └── mpathy.py ├── setup.cfg ├── setup.py ├── tests ├── __init__.py ├── conftest.py ├── migrations │ ├── 0001_initial.py │ └── __init__.py ├── models.py ├── test_db_consistency.py ├── test_models.py ├── test_moving.py ├── test_queryset.py └── test_templatetags.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *.egg-info 3 | __pycache__ 4 | build 5 | dist 6 | venv 7 | bin 8 | lib 9 | docs 10 | include 11 | pip-selfcheck.json 12 | .Python 13 | .tox 14 | MANIFEST 15 | *.egg 16 | .eggs 17 | .cache 18 | .coverage 19 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | sudo: false 2 | dist: focal 3 | language: python 4 | services: 5 | - postgresql 6 | 7 | python: 8 | - "3.8" 9 | 10 | env: 11 | - DJANGO="Django>=2.2,<3.0" 12 | - DJANGO="Django>=3.0,<3.1" 13 | - DJANGO="Django>=3.1,<3.2" 14 | - DJANGO="https://github.com/django/django/archive/master.tar.gz" 15 | 16 | matrix: 17 | allow_failures: 18 | - env: DJANGO="https://github.com/django/django/archive/master.tar.gz" 19 | 20 | before_install: pip install --upgrade pip 21 | 22 | install: pip install $DJANGO pytest pytest-django pytest-cov psycopg2-binary 23 | 24 | script: pytest --cov=mpathy 25 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # django-mpathy changelog 2 | 3 | Breaking/important changes for all released versions of django-mpathy will be listed here. 4 | 5 | # 0.2.0 6 | 7 | * Dropped support for Django < 3.2 8 | * Added support for django 3.2 and 4.0 9 | * Removed the `mpathy.compat` module. If you are using it for `GistIndex` in your migrations, just replace with the `django.contrib.postgres.indexes` module instead. 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2017, Craig de Stigter 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | * Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | * Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | * Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # django-mpathy 2 | 3 | [![Build Status](https://travis-ci.org/craigds/django-mpathy.svg?branch=master)](https://travis-ci.org/craigds/django-mpathy) 4 | 5 | *NOTE: This software was created as a proof of concept. It isn't used or actively maintained by the author. Pull requests will however be accepted (and releases issued to match)* 6 | 7 | mpathy is a Materialised Path implementation for django. Use it for storing hierarchical data in your postgres database, 8 | and accessing it from Django. 9 | 10 | It is a fairly thin wrapper around Postgres' [ltree extension](https://www.postgresql.org/docs/current/static/ltree.html) 11 | 12 | # Why 13 | 14 | There are a few existing ways to store trees in databases via Django. The main two libraries are: 15 | 16 | * django-mptt (implementation of MPTT / "nested sets") 17 | * django-treebeard (multi-backend tree storage app) 18 | 19 | While both are good and widely used, both suffer from a large amount of complexity, both in implementation and interface. 20 | 21 | The need to support multiple database backends, as well as the lack of well-indexed database tree implementations at the time they were created, has made both projects overly complex. 22 | 23 | Both apps, by necessity, put the tree consistency logic in the app layer. That's tricky (maybe impossible!) to get right, and has caused many tree consistency bugs in threaded environments. 24 | 25 | Mpathy delegates consistency to the database where it belongs. We use Postgres constraints to ensure that the tree fields are consistent ~~at all times~~ whenever changes are committed. 26 | 27 | # Requirements 28 | 29 | * Any supported version of django and python 30 | * Postgres 31 | 32 | # Goals 33 | 34 | mpathy intends to: 35 | * only support Postgres. I have no interest in supporting MySQL. 36 | * only support materialised path. The other implementations are too complicated, and the benefits are fairly small. 37 | * Push most of the work into the database to ensure performance and consistency. 38 | * Leverage modern django and postgres features to keep the code tidy and performant. 39 | -------------------------------------------------------------------------------- /mpathy/__init__.py: -------------------------------------------------------------------------------- 1 | __version__ = "0.2.0" 2 | -------------------------------------------------------------------------------- /mpathy/apps.py: -------------------------------------------------------------------------------- 1 | from django.apps import AppConfig 2 | from django.db.models.signals import post_migrate, pre_migrate 3 | 4 | 5 | class MpathyConfig(AppConfig): 6 | name = 'mpathy' 7 | 8 | def ready(self): 9 | from .operations import inject_pre_migration_operations, inject_post_migration_operations 10 | 11 | pre_migrate.connect(inject_pre_migration_operations, sender=self) 12 | post_migrate.connect(inject_post_migration_operations, sender=self) 13 | -------------------------------------------------------------------------------- /mpathy/fields.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | 3 | 4 | class Subpath(models.Func): 5 | function = "subpath" 6 | 7 | 8 | class LTree(str): 9 | def labels(self): 10 | return self.split(".") 11 | 12 | def level(self): 13 | """ 14 | Returns the level of this node. 15 | Root nodes are level 0. 16 | """ 17 | return self.count(".") 18 | 19 | def is_root(self): 20 | return self.level() == 0 21 | 22 | def is_ancestor_of(self, other, include_self=False): 23 | if include_self and self == other: 24 | return True 25 | if other.level() <= self.level(): 26 | return False 27 | return other.labels()[: self.level() + 1] == self.labels() 28 | 29 | def is_descendant_of(self, other, include_self=False): 30 | return other.is_ancestor_of(self, include_self=include_self) 31 | 32 | def parent(self): 33 | if self.level(): 34 | return LTree(self.rsplit(".", 1)[0]) 35 | return None 36 | 37 | def children_lquery(self): 38 | """ 39 | Returns an lquery which finds nodes which are children of the current node. 40 | """ 41 | return "%s.*{1}" % self 42 | 43 | def parent_lquery(self): 44 | """ 45 | Returns an lquery to find the parent of the current node. 46 | """ 47 | return ".".join(self.labels()[:-1]) 48 | 49 | 50 | class LTreeField(models.CharField): 51 | def __init__(self, *args, **kwargs): 52 | kwargs["max_length"] = 256 53 | super(LTreeField, self).__init__(*args, **kwargs) 54 | 55 | def deconstruct(self): 56 | name, path, args, kwargs = super(LTreeField, self).deconstruct() 57 | del kwargs["max_length"] 58 | return name, path, args, kwargs 59 | 60 | def db_type(self, connection): 61 | return "ltree" 62 | 63 | def to_python(self, value): 64 | return LTree(value) 65 | 66 | def from_db_value(self, value, expression, connection): 67 | if value is None: 68 | return value 69 | return LTree(value) 70 | 71 | 72 | class Level(models.Transform): 73 | lookup_name = "level" 74 | function = "nlevel" 75 | 76 | @property 77 | def output_field(self): 78 | return models.IntegerField() 79 | 80 | 81 | class LQuery(models.Lookup): 82 | lookup_name = "lquery" 83 | 84 | def as_sql(self, compiler, connection): 85 | lhs, lhs_params = self.process_lhs(compiler, connection) 86 | rhs, rhs_params = self.process_rhs(compiler, connection) 87 | params = lhs_params + rhs_params 88 | return "%s ~ %s" % (lhs, rhs), params 89 | 90 | 91 | class DescendantOrEqual(models.Lookup): 92 | lookup_name = "descendant_or_equal" 93 | 94 | def as_sql(self, compiler, connection): 95 | lhs, lhs_params = self.process_lhs(compiler, connection) 96 | rhs, rhs_params = self.process_rhs(compiler, connection) 97 | params = lhs_params + rhs_params 98 | return "%s <@ %s" % (lhs, rhs), params 99 | 100 | 101 | class AncestorOrEqual(models.Lookup): 102 | lookup_name = "ancestor_or_equal" 103 | 104 | def as_sql(self, compiler, connection): 105 | lhs, lhs_params = self.process_lhs(compiler, connection) 106 | rhs, rhs_params = self.process_rhs(compiler, connection) 107 | params = lhs_params + rhs_params 108 | return "%s @> %s" % (lhs, rhs), params 109 | 110 | 111 | LTreeField.register_lookup(Level) 112 | LTreeField.register_lookup(LQuery) 113 | LTreeField.register_lookup(DescendantOrEqual) 114 | LTreeField.register_lookup(AncestorOrEqual) 115 | -------------------------------------------------------------------------------- /mpathy/models.py: -------------------------------------------------------------------------------- 1 | from django.contrib.postgres.indexes import GistIndex 2 | from django.db import models 3 | from django.db.models.expressions import CombinedExpression, RawSQL 4 | 5 | from .fields import LTree, LTreeField, Subpath 6 | 7 | 8 | class BadMove(ValueError): 9 | pass 10 | 11 | 12 | class MPathQuerySet(models.QuerySet): 13 | def get_cached_trees(self): 14 | """ 15 | Evaluates this queryset and returns a list of top-level nodes. 16 | """ 17 | nodes_by_path = {} 18 | min_level = None 19 | for node in self: 20 | node._cached_children = [] 21 | nodes_by_path[node.ltree] = node 22 | level = node.ltree.level() 23 | if min_level is None or min_level > level: 24 | min_level = level 25 | 26 | top_level_nodes = [] 27 | for node in self: 28 | parent_path = node.ltree.parent() 29 | if parent_path in nodes_by_path: 30 | nodes_by_path[parent_path]._cached_children.append(node) 31 | elif node.ltree.level() == min_level: 32 | top_level_nodes.append(node) 33 | 34 | return top_level_nodes 35 | 36 | 37 | class MPathManager(models.Manager.from_queryset(MPathQuerySet)): 38 | def move_subtree(self, node, new_parent): 39 | """ 40 | Moves a node and all its descendants under the given new parent. 41 | If the parent is None, the node will become a root node. 42 | 43 | If the `node` is equal to the new parent, or is an ancestor of it, 44 | raises BadMove. 45 | 46 | If node's parent is already new_parent, returns immediately. 47 | 48 | NOTE: 49 | This updates all the nodes in the database, and the current node instance. 50 | It cannot update any other node instances that are in memory, so if you have some 51 | whose ltree paths are affected by this function you may need to refresh them 52 | from the database. 53 | """ 54 | if node.is_ancestor_of(new_parent, include_self=True): 55 | raise BadMove( 56 | "%r can't be made a child of %r" 57 | % (node.ltree, new_parent.ltree if new_parent else None) 58 | ) 59 | 60 | # Check if there's actually anything to do, return if not 61 | if node.parent_id is None: 62 | if new_parent is None: 63 | return 64 | elif new_parent is not None and node.ltree.parent() == new_parent.ltree: 65 | return 66 | 67 | old_parent_ltree = node.ltree.parent() 68 | old_parent_level = old_parent_ltree.level() if old_parent_ltree else -1 69 | 70 | # An expression which refers to the part of a given node's path which is 71 | # not in the current node's old parent path. 72 | # i.e. when node is 'a.b.c', the old parent will be 'a.b', 73 | # so for a descendant called 'a.b.c.d.e' we want to find 'c.d.e'. 74 | ltree_tail_expr = Subpath(models.F("ltree"), old_parent_level + 1) 75 | 76 | # Update the ltree on all descendant nodes to match new_parent 77 | qs = self.filter(ltree__descendant_or_equal=node.ltree) 78 | if new_parent is None: 79 | new_ltree_expr = ltree_tail_expr 80 | else: 81 | # TODO: how to do this without raw sql? django needs a cast expression 82 | # here otherwise the concat() fails because the ltree is interpreted 83 | # as text. Additionally, there's no available concatenation operator for ltrees 84 | # exposable in django. 85 | new_ltree_expr = CombinedExpression( 86 | lhs=RawSQL("%s::ltree", [new_parent.ltree]), 87 | connector="||", 88 | rhs=ltree_tail_expr, 89 | ) 90 | qs.update( 91 | ltree=new_ltree_expr, 92 | # Update parent at the same time as ltree, otherwise the check constraint fails 93 | parent=models.Case( 94 | models.When( 95 | pk=node.pk, 96 | then=models.Value(new_parent.ltree if new_parent else None), 97 | ), 98 | default=Subpath(new_ltree_expr, 0, -1), 99 | output_field=node.__class__._meta.get_field("parent"), 100 | ), 101 | ) 102 | 103 | # Update node in memory 104 | node.parent = new_parent 105 | node._set_ltree() 106 | node.save(update_fields=["parent"]) 107 | 108 | 109 | class MPathNode(models.Model): 110 | ltree = LTreeField(null=False, unique=True) 111 | label = models.CharField(null=False, blank=False, max_length=255) 112 | 113 | # Duplicating the whole path is annoying, but we need this field so that the 114 | # database can ensure consistency when we create a node. 115 | # Otherwise, we could create 'a.b' without first creating 'a'. 116 | parent = models.ForeignKey( 117 | "self", 118 | related_name="children", 119 | null=True, 120 | to_field="ltree", 121 | db_index=False, 122 | on_delete=models.CASCADE, 123 | ) 124 | 125 | objects = MPathManager() 126 | 127 | class Meta: 128 | abstract = True 129 | indexes = [ 130 | GistIndex(fields=["ltree"]), 131 | GistIndex(fields=["parent"]), 132 | ] 133 | 134 | def _set_ltree(self): 135 | if self.parent_id: 136 | ltree = "%s.%s" % (self.parent_id, self.label) 137 | else: 138 | ltree = self.label 139 | self.ltree = LTree(ltree) 140 | 141 | def save(self, **kwargs): 142 | # If no label, let the db throw an error. Otherwise, ensure path is consistent with 143 | # parent and label. 144 | if not self.label: 145 | raise ValueError( 146 | "%s objects must have a label. Got: label=%r" 147 | % (self.__class__.__name__, self.label) 148 | ) 149 | 150 | # If this is a new node or parent has changed, re-calculate ltree. 151 | # NOTE: This only works for leaf nodes, so shouldnt be relied on. 152 | # It's here as a convenience so you can create nodes with a pre-set parent. 153 | # For *changing* parent of an existing node, use MPathManager.move_subtree() 154 | self._set_ltree() 155 | 156 | return super(MPathNode, self).save(**kwargs) 157 | 158 | def is_ancestor_of(self, other, include_self=False): 159 | if other is None: 160 | return False 161 | return self.ltree.is_ancestor_of(other.ltree, include_self=include_self) 162 | 163 | def is_descendant_of(self, other, include_self=False): 164 | if other is None: 165 | return True 166 | return self.ltree.is_descendant_of(other.ltree, include_self=include_self) 167 | 168 | def get_siblings(self, include_self=False): 169 | """ 170 | Returns a queryset of this node's siblings, using the default manager. 171 | 172 | If include_self=True is given, the queryset will include this node. 173 | """ 174 | mgr = self.__class__._default_manager 175 | qs = mgr.filter(parent__ltree=self.parent_id) 176 | 177 | if not include_self: 178 | qs = qs.exclude(ltree=self.ltree) 179 | return qs 180 | 181 | def get_children(self): 182 | """ 183 | Returns a queryset of children for this node, using the default manager. 184 | """ 185 | try: 186 | # Shortcut the database if this node has been fetched using 187 | # qs.get_cached_trees() 188 | return self._cached_children 189 | except AttributeError: 190 | mgr = self.__class__._default_manager 191 | return mgr.filter(ltree__lquery=self.ltree.children_lquery()) 192 | 193 | def get_descendants(self, include_self=False): 194 | """ 195 | Returns a queryset of descendants for this node, using the default manager. 196 | 197 | If include_self=True is given, the queryset will include this node. 198 | """ 199 | mgr = self.__class__._default_manager 200 | qs = mgr.filter(ltree__descendant_or_equal=self.ltree) 201 | 202 | if not include_self: 203 | qs = qs.exclude(ltree=self.ltree) 204 | return qs 205 | 206 | def get_ancestors(self, include_self=False): 207 | """ 208 | Returns a queryset of ancestors for this node, using the default manager. 209 | 210 | If include_self=True is given, the queryset will include this node. 211 | """ 212 | mgr = self.__class__._default_manager 213 | qs = mgr.filter(ltree__ancestor_or_equal=self.ltree) 214 | 215 | if not include_self: 216 | qs = qs.exclude(ltree=self.ltree) 217 | return qs 218 | -------------------------------------------------------------------------------- /mpathy/operations.py: -------------------------------------------------------------------------------- 1 | from psycopg2.extensions import quote_ident 2 | 3 | from django.apps import apps as global_apps 4 | from django.contrib.postgres.operations import CreateExtension 5 | from django.core.exceptions import FieldDoesNotExist 6 | from django.db import connection, DEFAULT_DB_ALIAS, migrations 7 | 8 | from .fields import LTreeField 9 | from .models import MPathNode 10 | 11 | 12 | class LTreeExtension(CreateExtension): 13 | 14 | def __init__(self): 15 | self.name = 'ltree' 16 | 17 | 18 | def inject_pre_migration_operations(plan=None, apps=global_apps, using=DEFAULT_DB_ALIAS, **kwargs): 19 | """ 20 | Insert a `LTreeExtension` operation before every planned `CreateModel` operation. 21 | """ 22 | if plan is None: 23 | return 24 | 25 | for migration, backward in plan: 26 | for index, operation in enumerate(migration.operations): 27 | if isinstance(operation, migrations.CreateModel): 28 | for name, field in operation.fields: 29 | if isinstance(field, LTreeField): 30 | migration.operations.insert(index, LTreeExtension()) 31 | return 32 | 33 | 34 | def post_migrate_mpathnode(model): 35 | # Note: model *isn't* a subclass of MPathNode, because django migrations are Weird. 36 | # if not issubclass(model, MPathNode): 37 | # Hence the following workaround: 38 | try: 39 | ltree_field = model._meta.get_field('ltree') 40 | if not isinstance(ltree_field, LTreeField): 41 | return 42 | except FieldDoesNotExist: 43 | return 44 | 45 | names = { 46 | "table": quote_ident(model._meta.db_table, connection.connection), 47 | "check_constraint": quote_ident('%s__check_ltree' % model._meta.db_table, connection.connection), 48 | } 49 | 50 | cur = connection.cursor() 51 | # Check that the ltree is always consistent with being a child of _parent 52 | cur.execute(''' 53 | ALTER TABLE %(table)s ADD CONSTRAINT %(check_constraint)s CHECK ( 54 | (parent_id IS NOT NULL AND ltree ~ (parent_id::text || '.*{1}')::lquery) 55 | OR (parent_id IS NULL AND ltree ~ '*{1}'::lquery) 56 | ) 57 | ''' % names) 58 | 59 | 60 | def inject_post_migration_operations(plan=None, apps=global_apps, using=DEFAULT_DB_ALIAS, **kwargs): 61 | if plan is None: 62 | return 63 | 64 | for migration, backward in plan: 65 | for index, operation in reversed(list(enumerate(migration.operations))): 66 | if isinstance(operation, migrations.CreateModel): 67 | model = apps.get_model(migration.app_label, operation.name) 68 | post_migrate_mpathnode(model) 69 | -------------------------------------------------------------------------------- /mpathy/templatetags/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/craigds/django-mpathy/a5d423914c2920c6a144744774b1d3856a467b73/mpathy/templatetags/__init__.py -------------------------------------------------------------------------------- /mpathy/templatetags/mpathy.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | 4 | 5 | from django import template 6 | 7 | from django.utils.encoding import force_str 8 | from django.utils.translation import gettext as _ 9 | 10 | register = template.Library() 11 | 12 | 13 | @register.filter 14 | def tree_path(items, separator=' > '): 15 | """ 16 | Creates a tree path represented by a list of ``items`` by joining 17 | the items with a ``separator``. 18 | 19 | Each path item will be coerced to unicode, so a list of model 20 | instances may be given if required. 21 | 22 | Example:: 23 | 24 | {{ some_list|tree_path }} 25 | {{ some_node.get_ancestors|tree_path:" >> " }} 26 | 27 | """ 28 | return separator.join(force_str(i) for i in items) 29 | 30 | 31 | NOTSET = object() 32 | 33 | 34 | class RecurseTreeNode(template.Node): 35 | def __init__(self, nodelist, parent_queryset_var): 36 | self.nodelist = nodelist 37 | self.parent_queryset_var = template.Variable(parent_queryset_var) 38 | 39 | def render(self, context, qs=NOTSET): 40 | with context.push(): 41 | if qs is NOTSET: 42 | # At the top level, turn the given queryset into a list 43 | # of top-level nodes. 44 | qs = self.parent_queryset_var.resolve(context) 45 | root_nodes = qs.get_cached_trees() 46 | context[self.parent_queryset_var.var] = root_nodes 47 | else: 48 | # At lower levels, we've been passed in a list of child nodes. 49 | context[self.parent_queryset_var.var] = qs 50 | context['_mpathy_recursetree_parent'] = self 51 | return self.nodelist.render(context) 52 | 53 | 54 | @register.tag 55 | def recursetree(parser, token): 56 | """ 57 | Iterates over the nodes in the tree, and renders the contained block for each node. 58 | This tag will recursively render children into the template variable {{ children }}. 59 | Only one database query is required (children are cached for the whole tree) 60 | 61 | Usage: 62 | 76 | """ 77 | bits = token.split_contents() 78 | if len(bits) != 2: 79 | raise template.TemplateSyntaxError(_('%s tag takes one argument') % bits[0]) 80 | 81 | parent_queryset_var = bits[1] 82 | 83 | nodelist = parser.parse(['endrecursetree']) 84 | parser.delete_first_token() 85 | 86 | return RecurseTreeNode(nodelist, parent_queryset_var) 87 | 88 | 89 | class RecurseNode(template.Node): 90 | def __init__(self, child_queryset_var): 91 | self.child_queryset_var = template.Variable(child_queryset_var) 92 | 93 | def render(self, context): 94 | if '_mpathy_recursetree_parent' not in context: 95 | raise template.TemplateSyntaxError( 96 | 'Invalid syntax: {% recurse %} must be inside {% recursetree %}' 97 | ) 98 | 99 | parent = context['_mpathy_recursetree_parent'] 100 | qs = self.child_queryset_var.resolve(context) 101 | with context.push(): 102 | return parent.render(context, qs) 103 | 104 | 105 | @register.tag 106 | def recurse(parser, token): 107 | bits = token.split_contents() 108 | if len(bits) != 2: 109 | raise template.TemplateSyntaxError(_('%s tag takes one argument') % bits[0]) 110 | 111 | return RecurseNode(bits[1]) 112 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [bdist_wheel] 2 | universal = 1 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from setuptools import setup 3 | import os 4 | import re 5 | 6 | 7 | def get_version(): 8 | """ 9 | Return package version as listed in `__version__` in `init.py`. 10 | """ 11 | init_py = open(os.path.join("mpathy", "__init__.py")).read() 12 | return re.search("__version__ = ['\"]([^'\"]+)['\"]", init_py).group(1) 13 | 14 | 15 | setup( 16 | name="django-mpathy", 17 | description="""Simple hierarchical data using materialised path, in a postgres database.""", 18 | version=get_version(), 19 | author="Craig de Stigter", 20 | author_email="craig.ds@gmail.com", 21 | url="http://github.com/craigds/django-mpathy", 22 | packages=["mpathy"], 23 | install_requires=[ 24 | "psycopg2>=2.7", 25 | "Django>=2.2", 26 | ], 27 | classifiers=[ 28 | "Development Status :: 3 - Alpha", 29 | "Topic :: Database", 30 | "Framework :: Django", 31 | "Intended Audience :: Developers", 32 | "License :: OSI Approved :: BSD License", 33 | "Operating System :: OS Independent", 34 | "Programming Language :: Python", 35 | "Programming Language :: Python :: 3", 36 | "Programming Language :: Python :: 3.7", 37 | "Programming Language :: Python :: 3.8", 38 | "Programming Language :: Python :: 3.9", 39 | "Programming Language :: Python :: 3.10", 40 | "Topic :: Utilities", 41 | ], 42 | ) 43 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/craigds/django-mpathy/a5d423914c2920c6a144744774b1d3856a467b73/tests/__init__.py -------------------------------------------------------------------------------- /tests/conftest.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function, unicode_literals 2 | import os 3 | 4 | import django 5 | 6 | 7 | def pytest_configure(): 8 | from django.conf import settings 9 | 10 | # Best-guess the db user name 11 | db_user = os.environ.get('PGUSER') 12 | if not db_user: 13 | # On MacOS, brew installs postgres with the superuser account called $USER 14 | db_user = os.environ.get('USER') 15 | if os.environ.get('CI'): 16 | # On travis superuser is called postgres 17 | db_user = 'postgres' 18 | 19 | if not db_user: 20 | db_user = 'postgres' 21 | 22 | print(f"pytest_configure - using db user: {db_user}") 23 | 24 | settings.configure( 25 | DEBUG_PROPAGATE_EXCEPTIONS=True, 26 | DATABASES={ 27 | 'default': { 28 | 'ENGINE': 'django.db.backends.postgresql_psycopg2', 29 | 'NAME': 'mpathy_tests', 30 | 'USER': db_user, 31 | 'PASSWORD': '', 32 | 'HOST': 'localhost', 33 | 'TEST': {'NAME': 'mpathy_tests',}, 34 | } 35 | }, 36 | SECRET_KEY='hunter2', 37 | TEMPLATES=[ 38 | { 39 | 'BACKEND': 'django.template.backends.django.DjangoTemplates', 40 | 'APP_DIRS': True, 41 | }, 42 | ], 43 | MIDDLEWARE=( 44 | 'django.middleware.common.CommonMiddleware', 45 | 'django.contrib.sessions.middleware.SessionMiddleware', 46 | 'django.contrib.auth.middleware.AuthenticationMiddleware', 47 | 'django.contrib.messages.middleware.MessageMiddleware', 48 | ), 49 | INSTALLED_APPS=('mpathy', 'tests',), 50 | ) 51 | django.setup() 52 | -------------------------------------------------------------------------------- /tests/migrations/0001_initial.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from __future__ import print_function, unicode_literals 3 | 4 | from django.db import migrations, models 5 | import django.contrib.postgres.indexes 6 | import django.db.models.deletion 7 | 8 | import mpathy.fields 9 | 10 | 11 | def forwards(apps, schema_editor): 12 | pass 13 | 14 | 15 | class Migration(migrations.Migration): 16 | 17 | dependencies = [] 18 | initial = True 19 | 20 | operations = [ 21 | migrations.CreateModel( 22 | name="MyTree", 23 | fields=[ 24 | ("ltree", mpathy.fields.LTreeField(unique=True)), 25 | ("label", models.CharField(max_length=255)), 26 | ( 27 | "parent", 28 | models.ForeignKey( 29 | db_index=False, 30 | null=True, 31 | on_delete=django.db.models.deletion.CASCADE, 32 | to="tests.MyTree", 33 | to_field="ltree", 34 | ), 35 | ), 36 | ], 37 | options={ 38 | "abstract": False, 39 | }, 40 | ), 41 | migrations.AddIndex( 42 | model_name="mytree", 43 | index=django.contrib.postgres.indexes.GistIndex( 44 | fields=["ltree"], name="tests_mytre_ltree_600965_gist" 45 | ), 46 | ), 47 | migrations.AddIndex( 48 | model_name="mytree", 49 | index=django.contrib.postgres.indexes.GistIndex( 50 | fields=["parent"], name="tests_mytre_parent_05b59f_gist" 51 | ), 52 | ), 53 | ] 54 | -------------------------------------------------------------------------------- /tests/migrations/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/craigds/django-mpathy/a5d423914c2920c6a144744774b1d3856a467b73/tests/migrations/__init__.py -------------------------------------------------------------------------------- /tests/models.py: -------------------------------------------------------------------------------- 1 | from mpathy.models import MPathNode 2 | 3 | 4 | class MyTree(MPathNode): 5 | def __str__(self): 6 | return self.ltree 7 | -------------------------------------------------------------------------------- /tests/test_db_consistency.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | from django.db import connection, IntegrityError 4 | 5 | from .models import MyTree 6 | 7 | 8 | def flush_constraints(): 9 | # the default db setup is to have constraints DEFERRED. 10 | # So IntegrityErrors only happen when the transaction commits. 11 | # Django's testcase thing does eventually flush the constraints but to 12 | # actually test it *within* a testcase we have to flush it manually. 13 | connection.cursor().execute("SET CONSTRAINTS ALL IMMEDIATE") 14 | 15 | 16 | def test_node_creation_simple(db): 17 | MyTree.objects.create(label='root1') 18 | MyTree.objects.create(label='root2') 19 | 20 | 21 | def test_node_creation_with_no_label(db): 22 | # You need a label 23 | with pytest.raises(ValueError): 24 | MyTree.objects.create(label='') 25 | with pytest.raises(ValueError): 26 | MyTree.objects.create(label=None) 27 | with pytest.raises(ValueError): 28 | MyTree.objects.create() 29 | 30 | 31 | def test_root_node_already_exists(db): 32 | MyTree.objects.create(label='root1') 33 | 34 | with pytest.raises(IntegrityError): 35 | MyTree.objects.create(label='root1') 36 | 37 | 38 | def test_same_label_but_different_parent(db): 39 | root1 = MyTree.objects.create(label='root1') 40 | MyTree.objects.create(label='root1', parent=root1) 41 | 42 | 43 | def test_same_label_as_sibling(db): 44 | root1 = MyTree.objects.create(label='root1') 45 | MyTree.objects.create(label='child', parent=root1) 46 | with pytest.raises(IntegrityError): 47 | MyTree.objects.create(label='child', parent=root1) 48 | 49 | 50 | def test_parent_is_self_errors(db): 51 | root1 = MyTree.objects.create(label='root1') 52 | root1.parent = root1 53 | with pytest.raises(IntegrityError): 54 | root1.save() 55 | flush_constraints() 56 | 57 | 58 | def test_parent_is_remote_ancestor_errors(db): 59 | root1 = MyTree.objects.create(label='root1') 60 | child2 = MyTree.objects.create(label='child2', parent=root1) 61 | desc3 = MyTree.objects.create(label='desc3', parent=child2) 62 | with pytest.raises(IntegrityError): 63 | # To test this integrity error, have to update table without calling save() 64 | # (because save() changes `ltree` to match `parent_id`) 65 | MyTree.objects.filter(pk=desc3.pk).update(parent=root1) 66 | flush_constraints() 67 | 68 | 69 | def test_parent_is_descendant_errors(db): 70 | root1 = MyTree.objects.create(label='root1') 71 | child2 = MyTree.objects.create(label='child2', parent=root1) 72 | desc3 = MyTree.objects.create(label='desc3', parent=child2) 73 | child2.parent = desc3 74 | with pytest.raises(IntegrityError): 75 | child2.save() 76 | flush_constraints() 77 | -------------------------------------------------------------------------------- /tests/test_models.py: -------------------------------------------------------------------------------- 1 | from mpathy.fields import LTree 2 | from .models import MyTree 3 | 4 | 5 | def test_ltree_field(db): 6 | a = MyTree.objects.create(label='a') 7 | assert isinstance(a.ltree, LTree) 8 | 9 | a = MyTree.objects.get() 10 | assert isinstance(a.ltree, LTree) 11 | 12 | 13 | def test_get_descendants_of_root(db): 14 | a = MyTree.objects.create(label='a') 15 | desc = { 16 | MyTree.objects.create(label='a', parent_id='a'), 17 | MyTree.objects.create(label='a', parent_id='a.a'), 18 | MyTree.objects.create(label='a', parent_id='a.a.a'), 19 | } 20 | MyTree.objects.create(label='b') 21 | MyTree.objects.create(label='b', parent_id='b') 22 | 23 | assert set(a.get_descendants()) == desc 24 | 25 | 26 | def test_get_descendants_of_leaf(db): 27 | a = MyTree.objects.create(label='a') 28 | MyTree.objects.create(label='b') 29 | MyTree.objects.create(label='b', parent_id='b') 30 | 31 | assert set(a.get_descendants()) == set() 32 | 33 | 34 | def test_get_ancestors_of_root(db): 35 | a = MyTree.objects.create(label='a') 36 | MyTree.objects.create(label='b') 37 | MyTree.objects.create(label='b', parent_id='b') 38 | 39 | assert set(a.get_ancestors()) == set() 40 | 41 | 42 | def test_get_ancestors_of_leaf(db): 43 | ancestors = { 44 | MyTree.objects.create(label='a'), 45 | MyTree.objects.create(label='a', parent_id='a'), 46 | MyTree.objects.create(label='a', parent_id='a.a'), 47 | } 48 | aaaa = MyTree.objects.create(label='a', parent_id='a.a.a') 49 | 50 | MyTree.objects.create(label='b') 51 | MyTree.objects.create(label='b', parent_id='b') 52 | 53 | assert set(aaaa.get_ancestors()) == ancestors 54 | 55 | 56 | def test_is_ancestor_of(db): 57 | a = MyTree.objects.create(label='a') 58 | aa = MyTree.objects.create(label='a', parent_id='a') 59 | b = MyTree.objects.create(label='b') 60 | 61 | assert a.is_ancestor_of(aa) 62 | assert not aa.is_ancestor_of(a) 63 | assert not b.is_ancestor_of(a) 64 | assert not a.is_ancestor_of(b) 65 | assert not b.is_ancestor_of(aa) 66 | assert not aa.is_ancestor_of(b) 67 | 68 | assert not a.is_ancestor_of(a) 69 | assert a.is_ancestor_of(a, include_self=True) 70 | assert not a.is_ancestor_of(b, include_self=True) 71 | 72 | # we allow None as it's like an actual single root node. 73 | assert not a.is_ancestor_of(None) 74 | assert not a.is_ancestor_of(None, include_self=True) 75 | 76 | 77 | def test_is_descendant_of(db): 78 | a = MyTree.objects.create(label='a') 79 | aa = MyTree.objects.create(label='a', parent_id='a') 80 | b = MyTree.objects.create(label='b') 81 | 82 | assert not a.is_descendant_of(aa) 83 | assert aa.is_descendant_of(a) 84 | assert not b.is_descendant_of(a) 85 | assert not a.is_descendant_of(b) 86 | assert not b.is_descendant_of(aa) 87 | assert not aa.is_descendant_of(b) 88 | 89 | assert not a.is_descendant_of(a) 90 | assert a.is_descendant_of(a, include_self=True) 91 | assert not a.is_descendant_of(b, include_self=True) 92 | 93 | # we allow None as it's like an actual single root node. 94 | assert a.is_descendant_of(None) 95 | assert a.is_descendant_of(None, include_self=True) 96 | -------------------------------------------------------------------------------- /tests/test_moving.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | from django.db import IntegrityError 4 | 5 | from mpathy.models import BadMove 6 | 7 | from .models import MyTree 8 | from .test_db_consistency import flush_constraints 9 | 10 | 11 | def test_move_to_self(db): 12 | a = MyTree.objects.create(label='a') 13 | with pytest.raises(BadMove): 14 | MyTree.objects.move_subtree(a, a) 15 | 16 | 17 | def test_move_to_child(db): 18 | a = MyTree.objects.create(label='a') 19 | b = MyTree.objects.create(label='b', parent=a) 20 | with pytest.raises(BadMove): 21 | MyTree.objects.move_subtree(a, b) 22 | 23 | 24 | def test_root_to_root(db): 25 | a = MyTree.objects.create(label='a') 26 | MyTree.objects.move_subtree(a, None) 27 | assert a.parent is None 28 | assert a.ltree == 'a' 29 | 30 | 31 | def test_move_nonroot_to_nonroot(db): 32 | a = MyTree.objects.create(label='a') 33 | b = MyTree.objects.create(label='b', parent=a) 34 | c = MyTree.objects.create(label='c') 35 | MyTree.objects.move_subtree(b, c) 36 | assert b.parent == c 37 | assert b.ltree == 'c.b' 38 | assert set(a.get_children()) == set() 39 | assert set(c.get_children()) == {b} 40 | 41 | 42 | def test_move_nonroot_to_root(db): 43 | a = MyTree.objects.create(label='a') 44 | b = MyTree.objects.create(label='b', parent=a) 45 | MyTree.objects.move_subtree(b, None) 46 | assert b.parent is None 47 | assert b.ltree == 'b' 48 | assert set(a.get_children()) == set() 49 | 50 | 51 | def test_move_root_to_nonroot(db): 52 | a = MyTree.objects.create(label='a') 53 | b = MyTree.objects.create(label='b') 54 | MyTree.objects.move_subtree(b, a) 55 | assert b.parent.label == 'a' 56 | assert b.ltree == 'a.b' 57 | assert set(a.get_children()) == {b} 58 | 59 | 60 | def test_move_nonroot_with_children_to_root(db): 61 | a = MyTree.objects.create(label='a') 62 | b = MyTree.objects.create(label='b', parent=a) 63 | c = MyTree.objects.create(label='c', parent=b) 64 | MyTree.objects.move_subtree(b, None) 65 | assert b.parent is None 66 | assert b.ltree == 'b' 67 | assert set(a.get_children()) == set() 68 | assert set(b.get_children()) == {c} 69 | 70 | flush_constraints() 71 | with pytest.raises(IntegrityError): 72 | # This node no longer has a valid ltree path, it was re-written in 73 | # the database when we moved b. 74 | # So we check that saving it with the old path correctly raises an error. 75 | c.save() 76 | flush_constraints() 77 | 78 | 79 | def test_move_root_with_children_to_nonroot(db): 80 | a = MyTree.objects.create(label='a') 81 | b = MyTree.objects.create(label='b') 82 | c = MyTree.objects.create(label='c', parent=b) 83 | MyTree.objects.move_subtree(b, a) 84 | assert b.parent.label == 'a' 85 | assert b.ltree == 'a.b' 86 | assert set(a.get_children()) == {b} 87 | assert set(b.get_children()) == {c} 88 | 89 | flush_constraints() 90 | with pytest.raises(IntegrityError): 91 | # This node no longer has a valid ltree path, it was re-written in 92 | # the database when we moved b. 93 | # So we check that saving it with the old path correctly raises an error. 94 | c.save() 95 | flush_constraints() 96 | -------------------------------------------------------------------------------- /tests/test_queryset.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | from __future__ import absolute_import, division, print_function, unicode_literals 3 | 4 | 5 | from .models import MyTree 6 | 7 | 8 | def test_get_cached_trees_empty(db): 9 | assert MyTree.objects.get_cached_trees() == [] 10 | 11 | 12 | def test_get_cached_trees(db, django_assert_num_queries): 13 | a = MyTree.objects.create(label='a') 14 | aa = MyTree.objects.create(label='aa', parent=a) 15 | aaa = MyTree.objects.create(label='aaa', parent=aa) 16 | aaz = MyTree.objects.create(label='aaz', parent=aa) 17 | b = MyTree.objects.create(label='b') 18 | bb = MyTree.objects.create(label='bb', parent=b) 19 | 20 | # just ordering this to make the asserts a bit simpler. not required. 21 | qs = MyTree.objects.order_by('label') 22 | 23 | with django_assert_num_queries(1): 24 | cached = qs.get_cached_trees() 25 | 26 | with django_assert_num_queries(0): 27 | assert cached == [a, b] 28 | assert cached[0].get_children() == [aa] 29 | assert cached[0].get_children()[0].get_children() == [aaa, aaz] 30 | assert cached[1].get_children() == [bb] 31 | assert cached[1].get_children()[0].get_children() == [] 32 | -------------------------------------------------------------------------------- /tests/test_templatetags.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | from __future__ import absolute_import, division, print_function, unicode_literals 3 | 4 | 5 | from django.template import Template, Context 6 | 7 | from .models import MyTree 8 | 9 | 10 | def test_render_recursetree(db): 11 | a = MyTree.objects.create(label='a') 12 | MyTree.objects.create(label='ab', parent=a) 13 | 14 | t = Template( 15 | "{% load mpathy %}{% recursetree nodes %}" 16 | "{% for node in nodes %}\n" 17 | "
  • {{ node.label }}
    • " 18 | "{% endfor %}" 19 | "{% endrecursetree %}" 20 | ) 21 | 22 | context = Context({ 23 | 'nodes': MyTree.objects.all(), 24 | }) 25 | rendered = t.render(context) 26 | 27 | assert rendered == ( 28 | '\n' 29 | '
  • a
    • ' 31 | ) 32 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [pytest] 2 | addopts=--tb=short 3 | 4 | [tox] 5 | envlist = 6 | {py38}-{django32,django40} 7 | {py310}-{django40} 8 | 9 | [testenv] 10 | passenv = 11 | USER 12 | PGUSER 13 | CI 14 | commands = pytest {posargs} --cov=mpathy 15 | deps = 16 | django32: Django>=3.2,<4 17 | django40: Django>=4.0,<4.1 18 | pytest 19 | pytest-django 20 | pytest-cov 21 | pytest-sugar 22 | psycopg2-binary 23 | ipdb 24 | --------------------------------------------------------------------------------