├── tests ├── __init__.py ├── helpers.py ├── django_settings.py ├── test_paginator.py ├── models.py ├── test_management_command.py ├── test_signals.py ├── conftest.py ├── test_mixins.py ├── test_integration.py ├── test_query.py ├── test_registry.py └── test_documents.py ├── example ├── __init__.py ├── core │ ├── __init__.py │ ├── migrations │ │ ├── __init__.py │ │ └── 0001_initial.py │ ├── templatetags │ │ ├── __init__.py │ │ └── build_query_string.py │ ├── urls.py │ ├── apps.py │ ├── admin.py │ ├── models.py │ ├── documents.py │ ├── views.py │ └── templates │ │ └── core │ │ └── search.html ├── example │ ├── __init__.py │ ├── urls.py │ ├── asgi.py │ ├── wsgi.py │ └── settings.py ├── requirements.txt ├── README.md └── manage.py ├── redis_search_django ├── py.typed ├── __init__.py ├── management │ ├── __init__.py │ └── commands │ │ ├── __init__.py │ │ └── index.py ├── apps.py ├── paginator.py ├── signals.py ├── mixins.py ├── registry.py ├── config.py ├── query.py └── documents.py ├── requirements_test.txt ├── MANIFEST.in ├── CHANGELOG.md ├── docker-compose.yaml ├── .github ├── workflows │ ├── changelog-ci.yaml │ ├── pypi_publish.yaml │ └── test.yml └── dependabot.yml ├── tox.ini ├── pyproject.toml ├── LICENSE ├── .pre-commit-config.yaml ├── setup.cfg ├── .gitignore └── README.md /tests/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /example/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /example/core/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /example/example/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /redis_search_django/py.typed: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /redis_search_django/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /example/core/migrations/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /example/core/templatetags/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /redis_search_django/management/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /redis_search_django/management/commands/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /requirements_test.txt: -------------------------------------------------------------------------------- 1 | coverage 2 | pytest 3 | pytest-cov 4 | pytest-django 5 | -------------------------------------------------------------------------------- /example/requirements.txt: -------------------------------------------------------------------------------- 1 | # Install redis-search-django 2 | -e .. 3 | Django==4.2.2 4 | Pillow==9.5.0 5 | -------------------------------------------------------------------------------- /example/core/urls.py: -------------------------------------------------------------------------------- 1 | from django.urls import path 2 | 3 | from .views import SearchView 4 | 5 | urlpatterns = [ 6 | path("", SearchView.as_view(), name="search"), 7 | ] 8 | -------------------------------------------------------------------------------- /example/core/apps.py: -------------------------------------------------------------------------------- 1 | from django.apps import AppConfig 2 | 3 | 4 | class CoreConfig(AppConfig): 5 | default_auto_field = "django.db.models.BigAutoField" 6 | name = "core" 7 | -------------------------------------------------------------------------------- /example/core/admin.py: -------------------------------------------------------------------------------- 1 | from django.contrib import admin 2 | 3 | from .models import Category, Product, Tag, Vendor 4 | 5 | admin.site.register(Tag) 6 | admin.site.register(Category) 7 | admin.site.register(Product) 8 | admin.site.register(Vendor) 9 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include CHANGELOG.md 2 | include README.md 3 | include LICENSE 4 | include pyproject.toml 5 | include redis_search_django/py.typed 6 | 7 | recursive-include redis_search_django * 8 | 9 | global-exclude __pycache__ 10 | global-exclude *.py[co] 11 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Version: 0.1.0 2 | 3 | * [#2](https://github.com/saadmk11/redis-search-django/pull/2): Bump actions/checkout from 2 to 3 4 | * [#5](https://github.com/saadmk11/redis-search-django/pull/5): Create LICENSE 5 | * [#4](https://github.com/saadmk11/redis-search-django/pull/4): Add Tests and Improvements 6 | -------------------------------------------------------------------------------- /docker-compose.yaml: -------------------------------------------------------------------------------- 1 | version: "3.9" 2 | 3 | services: 4 | redis: 5 | image: redis/redis-stack:latest 6 | ports: 7 | - "6379:6379" 8 | - "8001:8001" 9 | volumes: 10 | - redis_data:/data 11 | environment: 12 | - REDIS_PASSWORD=redis 13 | 14 | volumes: 15 | redis_data: 16 | -------------------------------------------------------------------------------- /.github/workflows/changelog-ci.yaml: -------------------------------------------------------------------------------- 1 | name: Changelog CI 2 | 3 | on: 4 | pull_request: 5 | types: [opened] 6 | 7 | jobs: 8 | build: 9 | runs-on: ubuntu-latest 10 | 11 | steps: 12 | - uses: actions/checkout@v3 13 | 14 | - name: Run Changelog CI 15 | uses: saadmk11/changelog-ci@v1.1.1 16 | -------------------------------------------------------------------------------- /tests/helpers.py: -------------------------------------------------------------------------------- 1 | from functools import lru_cache 2 | 3 | import redis 4 | from redis_om import has_redisearch 5 | 6 | 7 | @lru_cache(maxsize=1) 8 | def is_redis_running(): 9 | """Check if redis is running.""" 10 | try: 11 | return has_redisearch() 12 | except redis.exceptions.ConnectionError: 13 | return False 14 | -------------------------------------------------------------------------------- /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | version: 2 2 | updates: 3 | - package-ecosystem: pip 4 | directory: "/" 5 | schedule: 6 | interval: weekly 7 | labels: 8 | - skip-changelog 9 | - dependencies 10 | - package-ecosystem: github-actions 11 | directory: "/" 12 | schedule: 13 | interval: monthly 14 | labels: 15 | - skip-changelog 16 | - dependencies 17 | -------------------------------------------------------------------------------- /example/core/templatetags/build_query_string.py: -------------------------------------------------------------------------------- 1 | from django import template 2 | 3 | register = template.Library() 4 | 5 | 6 | @register.simple_tag 7 | def query_string(request, **kwargs): 8 | """Build URL query string from request.GET.""" 9 | params = request.GET.copy() 10 | params.pop("page", None) 11 | 12 | if params: 13 | return "&" + params.urlencode() 14 | return "" 15 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | isolated_build = True 3 | envlist = py{37,38,39,310}-django32,py{38,39,310}-django40,py{38,39,310}-django41 4 | 5 | [testenv] 6 | commands = 7 | python -m pytest {posargs:tests} 8 | deps = 9 | -r {toxinidir}/requirements_test.txt 10 | django32: Django >=3.2, <3.3 11 | django40: Django >=4.0, <4.1 12 | django41: Django >=4.1, <4.2 13 | setenv = 14 | PYTHONDEVMODE=1 15 | -------------------------------------------------------------------------------- /example/example/urls.py: -------------------------------------------------------------------------------- 1 | from django.conf import settings 2 | from django.conf.urls.static import static 3 | from django.contrib import admin 4 | from django.urls import include, path 5 | 6 | urlpatterns = [ 7 | path("admin/", admin.site.urls), 8 | path("", include("core.urls")), 9 | ] 10 | 11 | if settings.DEBUG: 12 | # Serve media files in development 13 | urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) 14 | -------------------------------------------------------------------------------- /example/example/asgi.py: -------------------------------------------------------------------------------- 1 | """ 2 | ASGI config for example project. 3 | 4 | It exposes the ASGI callable as a module-level variable named ``application``. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/3.2/howto/deployment/asgi/ 8 | """ 9 | 10 | 11 | import os 12 | 13 | from django.core.asgi import get_asgi_application 14 | 15 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "example.settings") 16 | 17 | application = get_asgi_application() 18 | -------------------------------------------------------------------------------- /example/example/wsgi.py: -------------------------------------------------------------------------------- 1 | """ 2 | WSGI config for example project. 3 | 4 | It exposes the WSGI callable as a module-level variable named ``application``. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/3.2/howto/deployment/wsgi/ 8 | """ 9 | 10 | 11 | import os 12 | 13 | from django.core.wsgi import get_wsgi_application 14 | 15 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "example.settings") 16 | 17 | application = get_wsgi_application() 18 | -------------------------------------------------------------------------------- /tests/django_settings.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | 3 | BASE_DIR = Path(__file__).resolve().parent.parent 4 | 5 | SECRET_KEY = "django-insecure-secret" 6 | 7 | INSTALLED_APPS = [ 8 | "django.contrib.contenttypes", 9 | "redis_search_django", 10 | "tests", 11 | ] 12 | 13 | DATABASES = { 14 | "default": { 15 | "ENGINE": "django.db.backends.sqlite3", 16 | "NAME": BASE_DIR / "db.sqlite3", 17 | } 18 | } 19 | 20 | REDIS_SEARCH_AUTO_INDEX = True 21 | 22 | USE_TZ = True 23 | -------------------------------------------------------------------------------- /redis_search_django/apps.py: -------------------------------------------------------------------------------- 1 | from django.apps import AppConfig 2 | from django.utils.module_loading import autodiscover_modules 3 | 4 | 5 | class DjangoRedisSearchConfig(AppConfig): 6 | default_auto_field = "django.db.models.BigAutoField" 7 | name = "redis_search_django" 8 | 9 | def ready(self) -> None: 10 | # Auto Discover Document modules 11 | # Required for Document classes to be registered 12 | autodiscover_modules("documents") 13 | import redis_search_django.signals # noqa: F401 14 | -------------------------------------------------------------------------------- /example/README.md: -------------------------------------------------------------------------------- 1 | # Setup Example Project 2 | 3 | #### Clone the repository: 4 | 5 | ```bash 6 | git clone git@github.com:saadmk11/redis-search-django.git 7 | ``` 8 | 9 | #### Create a virtual environment and activate it: 10 | 11 | ```bash 12 | virtualenv -p python3 venv 13 | source venv/bin/activate 14 | ``` 15 | 16 | #### Change to project directory: 17 | 18 | ```bash 19 | cd redis-search-django/example 20 | ``` 21 | 22 | #### Install requirements: 23 | 24 | ```bash 25 | pip install -r requirements.txt 26 | ``` 27 | 28 | #### run migrations and runserver: 29 | 30 | ```bash 31 | python manage.py migrate 32 | python manage.py runserver # http://127.0.0.1:8000/ 33 | ``` 34 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["setuptools"] 3 | build-backend = "setuptools.build_meta" 4 | 5 | [tool.pytest.ini_options] 6 | addopts = "--ds=tests.django_settings --cov=redis_search_django --cov-report xml" 7 | 8 | [tool.mypy] 9 | mypy_path = "redis_search_django/" 10 | check_untyped_defs = true 11 | warn_unreachable = true 12 | warn_unused_ignores = true 13 | show_error_codes = true 14 | disallow_any_generics = true 15 | disallow_incomplete_defs = true 16 | disallow_untyped_defs = true 17 | no_implicit_optional = true 18 | 19 | [[tool.mypy.overrides]] 20 | module = "tests.*" 21 | allow_untyped_defs = true 22 | 23 | [tool.black] 24 | target-version = ['py310'] 25 | 26 | [tool.isort] 27 | profile = "black" 28 | -------------------------------------------------------------------------------- /example/manage.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | """Django's command-line utility for administrative tasks.""" 3 | 4 | 5 | import os 6 | import sys 7 | 8 | 9 | def main() -> None: 10 | """Run administrative tasks.""" 11 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "example.settings") 12 | try: 13 | from django.core.management import execute_from_command_line 14 | except ImportError as exc: 15 | raise ImportError( 16 | "Couldn't import Django. Are you sure it's installed and " 17 | "available on your PYTHONPATH environment variable? Did you " 18 | "forget to activate a virtual environment?" 19 | ) from exc 20 | execute_from_command_line(sys.argv) 21 | 22 | 23 | if __name__ == "__main__": 24 | main() 25 | -------------------------------------------------------------------------------- /.github/workflows/pypi_publish.yaml: -------------------------------------------------------------------------------- 1 | name: Publish to PyPI 2 | 3 | on: 4 | workflow_dispatch: 5 | release: 6 | types: [published] 7 | 8 | permissions: 9 | contents: read 10 | 11 | jobs: 12 | deploy: 13 | 14 | runs-on: ubuntu-latest 15 | 16 | steps: 17 | - uses: actions/checkout@v3 18 | 19 | - name: Set up Python 20 | uses: actions/setup-python@v4 21 | with: 22 | python-version: '3.10' 23 | 24 | - name: Install dependencies 25 | run: | 26 | python -m pip install --upgrade pip 27 | pip install build 28 | 29 | - name: Build package 30 | run: python -m build 31 | 32 | - name: Publish package 33 | uses: pypa/gh-action-pypi-publish@release/v1 34 | with: 35 | user: __token__ 36 | password: ${{ secrets.PYPI_API_TOKEN }} 37 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Maksudul Haque 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | default_language_version: 2 | python: python3.10 3 | 4 | repos: 5 | - repo: https://github.com/pre-commit/pre-commit-hooks 6 | rev: v4.4.0 7 | hooks: 8 | - id: end-of-file-fixer 9 | - id: trailing-whitespace 10 | - id: check-toml 11 | - id: check-case-conflict 12 | - id: check-merge-conflict 13 | 14 | - repo: https://github.com/asottile/pyupgrade 15 | rev: v3.9.0 16 | hooks: 17 | - id: pyupgrade 18 | args: [--py37-plus] 19 | 20 | - repo: https://github.com/psf/black 21 | rev: 23.7.0 22 | hooks: 23 | - id: black 24 | 25 | - repo: https://github.com/asottile/blacken-docs 26 | rev: 1.15.0 27 | hooks: 28 | - id: blacken-docs 29 | additional_dependencies: 30 | - black==22.6.0 31 | 32 | - repo: https://github.com/pycqa/isort 33 | rev: 5.12.0 34 | hooks: 35 | - id: isort 36 | 37 | - repo: https://github.com/PyCQA/flake8 38 | rev: 6.0.0 39 | hooks: 40 | - id: flake8 41 | additional_dependencies: 42 | - flake8-bugbear 43 | - flake8-comprehensions 44 | - flake8-tidy-imports 45 | - flake8-typing-imports 46 | 47 | - repo: https://github.com/pre-commit/mirrors-mypy 48 | rev: v1.4.1 49 | hooks: 50 | - id: mypy 51 | files: ^redis_search_django/ 52 | additional_dependencies: [types-redis] 53 | -------------------------------------------------------------------------------- /redis_search_django/management/commands/index.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | from typing import Any 3 | 4 | from django.core.management import BaseCommand 5 | from redis_om import Migrator, get_redis_connection 6 | 7 | from redis_search_django.registry import document_registry 8 | 9 | 10 | class Command(BaseCommand): 11 | help = "Index Documents to Redis Search" 12 | 13 | def add_arguments(self, parser: argparse.ArgumentParser) -> None: 14 | parser.add_argument( 15 | "--models", 16 | type=str, 17 | nargs="*", 18 | help="Django Models to index to Redis. e.g. 'app_name.ModelName'", 19 | ) 20 | parser.add_argument( 21 | "--only-migrate", 22 | action="store_true", 23 | dest="only_migrate", 24 | help="Only update the indices schema.", 25 | ) 26 | 27 | def handle(self, *args: Any, **options: Any) -> None: 28 | models = options["models"] 29 | only_migrate = options["only_migrate"] 30 | 31 | get_redis_connection() 32 | Migrator().run() 33 | 34 | self.stdout.write(self.style.SUCCESS("Successfully migrated indices")) 35 | 36 | if not only_migrate: 37 | document_registry.index_documents(models) 38 | self.stdout.write(self.style.SUCCESS("Successfully indexed documents")) 39 | -------------------------------------------------------------------------------- /tests/test_paginator.py: -------------------------------------------------------------------------------- 1 | from unittest import mock 2 | 3 | import pytest 4 | from django.core.paginator import EmptyPage, PageNotAnInteger 5 | 6 | from redis_search_django.paginator import RediSearchPaginator 7 | from redis_search_django.query import RediSearchQuery, RediSearchResult 8 | 9 | from .models import Category 10 | 11 | 12 | def test_paginator_validate_number(): 13 | paginator = RediSearchPaginator(range(10), 5) 14 | assert paginator.validate_number(1) == 1 15 | assert paginator.validate_number("1") == 1 16 | 17 | with pytest.raises(PageNotAnInteger): 18 | paginator.validate_number("abc") 19 | 20 | with pytest.raises(EmptyPage): 21 | paginator.validate_number(0) 22 | 23 | with pytest.raises(EmptyPage): 24 | paginator.validate_number(-1) 25 | 26 | 27 | @mock.patch("redis_search_django.query.RediSearchQuery.execute") 28 | def test_paginator(execute): 29 | query = RediSearchQuery(mock.MagicMock(), model=mock.MagicMock()) 30 | result = RediSearchResult([mock.MagicMock()], 100, Category) 31 | query._model_cache = result 32 | execute.return_value = result 33 | 34 | paginator = RediSearchPaginator(query, 5) 35 | paginator.page(1) 36 | 37 | assert paginator.count == 100 38 | assert paginator.num_pages == 20 39 | 40 | with pytest.raises(EmptyPage): 41 | paginator.page(50) 42 | -------------------------------------------------------------------------------- /tests/models.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | 3 | 4 | class Tag(models.Model): 5 | name = models.CharField(max_length=30) 6 | 7 | class Meta: 8 | app_label = "tests" 9 | 10 | def __str__(self) -> str: 11 | return self.name 12 | 13 | 14 | class Category(models.Model): 15 | name = models.CharField(max_length=30) 16 | 17 | class Meta: 18 | app_label = "tests" 19 | 20 | def __str__(self) -> str: 21 | return self.name 22 | 23 | 24 | class Vendor(models.Model): 25 | name = models.CharField(max_length=30) 26 | establishment_date = models.DateField() 27 | 28 | class Meta: 29 | app_label = "tests" 30 | 31 | def __str__(self) -> str: 32 | return self.name 33 | 34 | 35 | class Product(models.Model): 36 | name = models.CharField(max_length=256) 37 | description = models.TextField(blank=True) 38 | category = models.ForeignKey( 39 | Category, on_delete=models.SET_NULL, blank=True, null=True 40 | ) 41 | vendor = models.OneToOneField(Vendor, on_delete=models.CASCADE) 42 | tags = models.ManyToManyField(Tag, blank=True) 43 | price = models.DecimalField(max_digits=6, decimal_places=2) 44 | created_at = models.DateTimeField(auto_now_add=True) 45 | 46 | class Meta: 47 | app_label = "tests" 48 | 49 | def __str__(self) -> str: 50 | return self.name 51 | -------------------------------------------------------------------------------- /redis_search_django/paginator.py: -------------------------------------------------------------------------------- 1 | from typing import Union 2 | 3 | from django.core.paginator import EmptyPage, Page, PageNotAnInteger, Paginator 4 | 5 | 6 | class RediSearchPaginator(Paginator): 7 | """Custom Paginator that Allows Pagination of a Redis Search Query""" 8 | 9 | def validate_number(self, number: Union[int, str]) -> int: 10 | try: 11 | number = int(number) 12 | except (TypeError, ValueError): 13 | raise PageNotAnInteger("That page number is not an integer") 14 | if number < 1: 15 | raise EmptyPage("That page number is less than 1") 16 | return number 17 | 18 | def page(self, number: Union[int, str]) -> Page: 19 | number = self.validate_number(number) 20 | bottom = (number - 1) * self.per_page 21 | # Set limit and offset for the redis search query 22 | self.object_list.paginate(offset=bottom, limit=self.per_page) 23 | # Executes the redis search query which returns RediSearchResult object 24 | result = self.object_list.execute(exhaust_results=False) 25 | 26 | page = Page(result, number, self) 27 | 28 | if number > self.num_pages: 29 | if number == 1 and self.allow_empty_first_page: 30 | pass 31 | else: 32 | raise EmptyPage("That page contains no results") 33 | 34 | return page 35 | -------------------------------------------------------------------------------- /example/core/models.py: -------------------------------------------------------------------------------- 1 | import uuid 2 | 3 | from django.db import models 4 | 5 | 6 | class Tag(models.Model): 7 | name = models.CharField(max_length=30) 8 | 9 | def __str__(self) -> str: 10 | return self.name 11 | 12 | 13 | class Category(models.Model): 14 | name = models.CharField(max_length=30) 15 | slug = models.SlugField(max_length=30) 16 | 17 | def __str__(self) -> str: 18 | return self.name 19 | 20 | 21 | class Vendor(models.Model): 22 | logo = models.ImageField(upload_to="core/vendor_logo", blank=True) 23 | identifier = models.UUIDField(default=uuid.uuid4, editable=False) 24 | name = models.CharField(max_length=30) 25 | email = models.EmailField() 26 | establishment_date = models.DateField() 27 | 28 | def __str__(self) -> str: 29 | return self.name 30 | 31 | 32 | class Product(models.Model): 33 | name = models.CharField(max_length=256) 34 | description = models.TextField(blank=True) 35 | category = models.ForeignKey( 36 | Category, on_delete=models.SET_NULL, blank=True, null=True 37 | ) 38 | vendor = models.OneToOneField(Vendor, on_delete=models.CASCADE) 39 | tags = models.ManyToManyField(Tag, blank=True) 40 | price = models.DecimalField(max_digits=6, decimal_places=2) 41 | available = models.BooleanField(default=True) 42 | quantity = models.IntegerField(default=1) 43 | created_at = models.DateTimeField(auto_now_add=True) 44 | 45 | def __str__(self) -> str: 46 | return self.name 47 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | name = redis-search-django 3 | version = 0.1.0 4 | description = Django Integration with Redis Search 5 | long_description = file: README.md 6 | long_description_content_type = text/markdown 7 | license = MIT 8 | url = https://github.com/saadmk11/redis-search-django 9 | author = Maksudul Haque 10 | author_email = saad.mk112@gmail.com 11 | keywords = 12 | Django 13 | Redis 14 | Search 15 | RediSearch 16 | classifiers = 17 | Development Status :: 4 - Beta 18 | Natural Language :: English 19 | Intended Audience :: Developers 20 | License :: OSI Approved :: MIT License 21 | Operating System :: OS Independent 22 | Framework :: Django :: 3.2 23 | Framework :: Django :: 4.0 24 | Framework :: Django :: 4.1 25 | Programming Language :: Python :: 3 :: Only 26 | Programming Language :: Python :: 3 27 | Programming Language :: Python :: 3.7 28 | Programming Language :: Python :: 3.8 29 | Programming Language :: Python :: 3.9 30 | Programming Language :: Python :: 3.10 31 | 32 | [options] 33 | packages = redis_search_django 34 | include_package_data = True 35 | python_requires = >=3.7 36 | install_requires = 37 | django>=3.2 38 | redis-om>=0.0.27 39 | 40 | [options.packages.find] 41 | where = . 42 | 43 | [coverage:run] 44 | branch = True 45 | parallel = True 46 | source = 47 | redis_search_django 48 | tests 49 | 50 | [coverage:report] 51 | exclude_lines = 52 | pragma: no cover 53 | if TYPE_CHECKING: 54 | 55 | [coverage:paths] 56 | source = . 57 | 58 | [flake8] 59 | max-line-length = 88 60 | -------------------------------------------------------------------------------- /.github/workflows/test.yml: -------------------------------------------------------------------------------- 1 | name: Django Tests 2 | 3 | on: 4 | push: 5 | branches: [ main ] 6 | pull_request: 7 | 8 | jobs: 9 | test: 10 | runs-on: ubuntu-latest 11 | 12 | strategy: 13 | fail-fast: false 14 | 15 | matrix: 16 | python-version: ["3.7", "3.8", "3.9", "3.10"] 17 | django-version: ["3.2", "4.0", "4.1"] 18 | 19 | exclude: 20 | - python-version: "3.7" 21 | django-version: "4.0" 22 | 23 | - python-version: "3.7" 24 | django-version: "4.1" 25 | 26 | services: 27 | redis: 28 | image: redis/redis-stack-server:latest 29 | ports: 30 | - 6379:6379 31 | options: >- 32 | --health-cmd "redis-cli ping" 33 | --health-interval 10s 34 | --health-timeout 5s 35 | --health-retries 5 36 | 37 | steps: 38 | - uses: actions/checkout@v3 39 | 40 | - name: Install and Setup Python ${{ matrix.python-version }} 41 | uses: actions/setup-python@v4 42 | with: 43 | python-version: ${{ matrix.python-version }} 44 | cache: pip 45 | cache-dependency-path: requirements_test.txt 46 | 47 | - name: Install Dependencies 48 | run: python -m pip install --upgrade pip tox setuptools wheel 49 | 50 | - name: Run tests with Python ${{ matrix.python-version }} and Django ${{ matrix.django-version }} 51 | run: | 52 | TOX_ENV=$(echo "py${{ matrix.python-version}}-django${{ matrix.django-version}}" | tr -d .) 53 | python -m tox -e $TOX_ENV 54 | 55 | - name: Upload coverage to Codecov 56 | uses: codecov/codecov-action@v3 57 | with: 58 | files: coverage.xml 59 | -------------------------------------------------------------------------------- /tests/test_management_command.py: -------------------------------------------------------------------------------- 1 | from unittest import mock 2 | 3 | from django.core.management import call_command 4 | 5 | 6 | @mock.patch("redis_search_django.management.commands.index.get_redis_connection") 7 | @mock.patch("redis_search_django.management.commands.index.Migrator") 8 | @mock.patch( 9 | "redis_search_django.management.commands.index.document_registry.index_documents" 10 | ) 11 | def test_index_command_only_migrate( 12 | index_documents, migrator, get_redis_connection, document_class 13 | ): 14 | call_command("index", "--only-migrate") 15 | 16 | get_redis_connection.assert_called_once() 17 | migrator().run.assert_called_once() 18 | index_documents.assert_not_called() 19 | 20 | 21 | @mock.patch("redis_search_django.management.commands.index.get_redis_connection") 22 | @mock.patch("redis_search_django.management.commands.index.Migrator") 23 | @mock.patch( 24 | "redis_search_django.management.commands.index.document_registry.index_documents" 25 | ) 26 | def test_index_command(index_documents, migrator, get_redis_connection, document_class): 27 | call_command("index") 28 | 29 | get_redis_connection.assert_called_once() 30 | migrator().run.assert_called_once() 31 | index_documents.assert_called_once() 32 | 33 | 34 | @mock.patch("redis_search_django.management.commands.index.get_redis_connection") 35 | @mock.patch("redis_search_django.management.commands.index.Migrator") 36 | @mock.patch( 37 | "redis_search_django.management.commands.index.document_registry.index_documents" 38 | ) 39 | def test_index_command_with_models_option( 40 | index_documents, migrator, get_redis_connection, document_class 41 | ): 42 | call_command("index", "--models", "tests.Vendor", "tests.Category") 43 | 44 | get_redis_connection.assert_called_once() 45 | migrator().run.assert_called_once() 46 | index_documents.assert_called_once_with(["tests.Vendor", "tests.Category"]) 47 | -------------------------------------------------------------------------------- /tests/test_signals.py: -------------------------------------------------------------------------------- 1 | from unittest import mock 2 | 3 | import pytest 4 | 5 | from tests.models import Category 6 | 7 | 8 | @pytest.mark.django_db 9 | @mock.patch("redis_search_django.signals.document_registry") 10 | def test_add_document_to_redis_index(document_registry): 11 | category = Category.objects.create(name="Test") 12 | document_registry.update_document.assert_called_once_with(category, create=True) 13 | 14 | category.name = "Test2" 15 | category.save() 16 | 17 | document_registry.update_related_documents.assert_called_once_with( 18 | category, exclude=None 19 | ) 20 | 21 | 22 | @pytest.mark.django_db 23 | @mock.patch("redis_search_django.signals.document_registry") 24 | def test_remove_document_from_redis_index(document_registry): 25 | category = Category.objects.create(name="Test") 26 | category.delete() 27 | document_registry.remove_document.assert_called_once_with(category) 28 | 29 | 30 | @pytest.mark.django_db 31 | @mock.patch("redis_search_django.signals.document_registry") 32 | def test_remove_related_documents_from_redis_index(document_registry): 33 | category = Category.objects.create(name="Test") 34 | category.delete() 35 | document_registry.update_related_documents.assert_called_once_with( 36 | category, exclude=category 37 | ) 38 | 39 | 40 | @pytest.mark.django_db 41 | @mock.patch("redis_search_django.signals.document_registry") 42 | def test_update_redis_index_on_m2m_changed_add(document_registry, product_obj, tag_obj): 43 | product_obj.tags.add(tag_obj) 44 | document_registry.update_document.assert_called_once_with(product_obj, create=True) 45 | document_registry.update_related_documents.assert_called_once_with( 46 | product_obj, exclude=None 47 | ) 48 | 49 | 50 | @pytest.mark.django_db 51 | @mock.patch("redis_search_django.signals.document_registry") 52 | def test_update_redis_index_on_m2m_changed_remove(document_registry, product_with_tag): 53 | product, tag = product_with_tag 54 | product.tags.clear() 55 | document_registry.update_related_documents.assert_called_with(product, exclude=None) 56 | -------------------------------------------------------------------------------- /redis_search_django/signals.py: -------------------------------------------------------------------------------- 1 | from typing import Any, Type 2 | 3 | from django.conf import settings 4 | from django.db import models 5 | from django.db.models.signals import m2m_changed, post_delete, post_save, pre_delete 6 | 7 | from .registry import document_registry 8 | 9 | 10 | def add_document_to_redis_index( 11 | sender: Type[models.Model], instance: models.Model, created: bool, **kwargs: Any 12 | ) -> None: 13 | """Signal handler for populating the redis index.""" 14 | document_registry.update_document(instance, create=True) 15 | 16 | if not created: 17 | document_registry.update_related_documents(instance, exclude=None) 18 | 19 | 20 | def remove_document_from_redis_index( 21 | sender: Type[models.Model], instance: models.Model, **kwargs: Any 22 | ) -> None: 23 | """Signal handler for removing data from redis index.""" 24 | document_registry.remove_document(instance) 25 | 26 | 27 | def remove_related_documents_from_redis_index( 28 | sender: Type[models.Model], instance: models.Model, **kwargs: Any 29 | ) -> None: 30 | """Signal handler for removing related model data from redis index.""" 31 | document_registry.update_related_documents(instance, exclude=instance) 32 | 33 | 34 | def update_redis_index_on_m2m_changed( 35 | sender: Type[models.Model], instance: models.Model, action: str, **kwargs: Any 36 | ) -> None: 37 | """Signal handler for updating redis index on m2m changed.""" 38 | if action in ("post_add", "post_remove", "post_clear"): 39 | document_registry.update_document(instance, create=True) 40 | document_registry.update_related_documents(instance, exclude=None) 41 | elif action in ("pre_remove", "pre_clear"): 42 | document_registry.update_related_documents(instance, exclude=instance) 43 | 44 | 45 | # Check if Auto Index is globally turned off using Django settings. 46 | if getattr(settings, "REDIS_SEARCH_AUTO_INDEX", True): 47 | post_save.connect(add_document_to_redis_index) 48 | pre_delete.connect(remove_related_documents_from_redis_index) 49 | post_delete.connect(remove_document_from_redis_index) 50 | m2m_changed.connect(update_redis_index_on_m2m_changed) 51 | -------------------------------------------------------------------------------- /redis_search_django/mixins.py: -------------------------------------------------------------------------------- 1 | from typing import Any, Iterable, List, Type, Union 2 | 3 | from django.core.exceptions import ImproperlyConfigured 4 | from django.http import HttpRequest 5 | from django.template.response import TemplateResponse 6 | from django.utils.functional import cached_property 7 | from django.views.generic.list import ( 8 | MultipleObjectMixin, 9 | MultipleObjectTemplateResponseMixin, 10 | ) 11 | 12 | from .documents import Document 13 | from .paginator import RediSearchPaginator 14 | 15 | 16 | class RediSearchMixin: 17 | paginator_class = RediSearchPaginator 18 | document_class: Type[Document] 19 | 20 | @cached_property 21 | def search_query_expression(self) -> Any: 22 | return None 23 | 24 | @cached_property 25 | def sort_by(self) -> Union[str, None]: 26 | return None 27 | 28 | def facets(self) -> Any: 29 | return None 30 | 31 | def search(self) -> Document: 32 | if not hasattr(self, "document_class") or not self.document_class: 33 | raise ImproperlyConfigured( 34 | "%(cls)s requires a 'document_class' attribute" 35 | % {"cls": self.__class__.__name__} 36 | ) 37 | if self.search_query_expression: 38 | search_query = self.document_class.find(self.search_query_expression) 39 | else: 40 | search_query = self.document_class.find() 41 | 42 | if self.sort_by: 43 | search_query = search_query.sort_by(self.sort_by) 44 | 45 | return search_query 46 | 47 | 48 | class RediSearchTemplateResponseMixin(MultipleObjectTemplateResponseMixin): 49 | def get_template_names(self) -> List[str]: 50 | """ 51 | Return a list of template names to be used for the request. Must return 52 | a list. May not be called if render_to_response() is overridden. 53 | """ 54 | if self.template_name: 55 | return [self.template_name] 56 | 57 | template_names = [] 58 | 59 | if self.document_class: 60 | opts = self.document_class._django.model._meta 61 | template_names.append( 62 | "%s/%s%s.html" 63 | % (opts.app_label, opts.model_name, self.template_name_suffix) 64 | ) 65 | elif not template_names: 66 | raise ImproperlyConfigured( 67 | "%(cls)s requires either a 'template_name' attribute " 68 | "or a get_queryset() method that returns a QuerySet." 69 | % { 70 | "cls": self.__class__.__name__, 71 | } 72 | ) 73 | return template_names 74 | 75 | 76 | class RediSearchMultipleObjectMixin(MultipleObjectMixin): 77 | def get_context_object_name(self, object_list: Iterable[Any]) -> Union[str, None]: 78 | """Get the name of the item to be used in the context.""" 79 | if self.context_object_name: 80 | return self.context_object_name 81 | elif self.document_class: 82 | return "%s_list" % self.document_class._django.model._meta.model_name 83 | else: 84 | return None 85 | 86 | 87 | class RediSearchListViewMixin( 88 | RediSearchMixin, RediSearchTemplateResponseMixin, RediSearchMultipleObjectMixin 89 | ): 90 | def get(self, request: HttpRequest, *args: Any, **kwargs: Any) -> TemplateResponse: 91 | self.object_list = self.search() 92 | context = self.get_context_data(facets=self.facets()) 93 | return self.render_to_response(context) 94 | -------------------------------------------------------------------------------- /example/core/documents.py: -------------------------------------------------------------------------------- 1 | from typing import List, Optional 2 | 3 | from django.db import models 4 | from redis_om import Field 5 | 6 | from redis_search_django.documents import EmbeddedJsonDocument, JsonDocument 7 | 8 | from .models import Category, Product, Tag, Vendor 9 | 10 | 11 | class CategoryDocument(EmbeddedJsonDocument): 12 | # name: str = Field(index=True, full_text_search=True) 13 | # slug: str = Field(index=True) 14 | custom_field: str = Field(index=True, full_text_search=True) 15 | 16 | class Django: 17 | model = Category 18 | fields = ["name", "slug"] 19 | 20 | @classmethod 21 | def prepare_custom_field(cls, obj): 22 | return "CUSTOM FIELD VALUE" 23 | 24 | 25 | class TagDocument(EmbeddedJsonDocument): 26 | # name: str = Field(index=True) 27 | 28 | class Django: 29 | model = Tag 30 | fields = ["name"] 31 | 32 | 33 | class VendorDocument(EmbeddedJsonDocument): 34 | # logo: str = Field(index=True) 35 | # identifier: str = Field(index=True) 36 | # name: str = Field(index=True, full_text_search=True, sortable=True) 37 | # email: str = Field(index=True, full_text_search=True, sortable=True) 38 | # establishment_date: datetime.date = Field( 39 | # index=True, full_text_search=True, sortable=True 40 | # ) 41 | 42 | class Django: 43 | model = Vendor 44 | fields = ["logo", "identifier", "name", "email", "establishment_date"] 45 | 46 | @classmethod 47 | def prepare_logo(cls, obj): 48 | return obj.logo.url if obj.logo else "" 49 | 50 | 51 | class ProductDocument(JsonDocument): 52 | # Fields can be defined manually or 53 | # `Django.fields` can be used to define the Django Model fields automatically. 54 | 55 | # name: str = Field(index=True, full_text_search=True, sortable=True) 56 | # description: str = Field(index=True, full_text_search=True) 57 | # price: Decimal = Field(index=True, sortable=True) 58 | # created_at: Optional[datetime.date] = Field(index=True) 59 | # quantity: Optional[int] = Field(index=True, sortable=True) 60 | # available: int = Field(index=True, sortable=True) 61 | 62 | # OnetoOneField 63 | vendor: VendorDocument 64 | # ForeignKey field 65 | category: Optional[CategoryDocument] 66 | # ManyToManyField 67 | tags: List[TagDocument] 68 | 69 | # class Meta: 70 | # model_key_prefix = "product" 71 | # global_key_prefix = "redis_search" 72 | 73 | class Django: 74 | model = Product 75 | # Django Model fields can be added to the Document automatically. 76 | fields = ["name", "description", "price", "created_at", "quantity", "available"] 77 | prefetch_related_fields = ["tags"] 78 | select_related_fields = ["vendor", "category"] 79 | related_models = { 80 | Vendor: { 81 | "related_name": "product", 82 | "many": False, 83 | }, 84 | Category: { 85 | "related_name": "product_set", 86 | "many": True, 87 | }, 88 | Tag: { 89 | "related_name": "product_set", 90 | "many": True, 91 | }, 92 | } 93 | 94 | @classmethod 95 | def get_queryset(cls) -> models.QuerySet: 96 | return super().get_queryset().filter(available=True) 97 | 98 | @classmethod 99 | def prepare_name(cls, obj): 100 | return obj.name.upper() 101 | -------------------------------------------------------------------------------- /tests/conftest.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | import uuid 3 | from typing import List, Optional 4 | 5 | import pytest 6 | 7 | from redis_search_django.documents import EmbeddedJsonDocument, JsonDocument 8 | 9 | from .models import Category, Product, Tag, Vendor 10 | 11 | 12 | @pytest.fixture(scope="session") 13 | def document_class(): 14 | def build_document_class( 15 | document_type, 16 | model_class, 17 | model_fields, 18 | enable_auto_index=True, 19 | ): 20 | class DocumentClass(document_type): 21 | class Meta: 22 | model_key_prefix = ( 23 | f"{model_class.__name__.lower()}-{uuid.uuid4().hex[:5]}" 24 | ) 25 | global_key_prefix = "test_redis_search" 26 | 27 | class Django: 28 | model = model_class 29 | fields = model_fields 30 | auto_index = enable_auto_index 31 | 32 | return DocumentClass 33 | 34 | return build_document_class 35 | 36 | 37 | @pytest.fixture(scope="session") 38 | def nested_document_class(document_class): 39 | CategoryEmbeddedJsonDocument = document_class( 40 | EmbeddedJsonDocument, Category, ["name"] 41 | ) 42 | TagEmbeddedJsonDocument = document_class(EmbeddedJsonDocument, Tag, ["name"]) 43 | VendorEmbeddedJsonDocument = document_class( 44 | EmbeddedJsonDocument, Vendor, ["name", "establishment_date"] 45 | ) 46 | 47 | class ProductJsonDocument(JsonDocument): 48 | # OnetoOneField with null=True 49 | vendor: VendorEmbeddedJsonDocument 50 | # ForeignKey field 51 | category: Optional[CategoryEmbeddedJsonDocument] 52 | # ManyToManyField 53 | tags: List[TagEmbeddedJsonDocument] 54 | 55 | class Meta: 56 | model_key_prefix = f"{Product.__name__.lower()}-{uuid.uuid4().hex[:5]}" 57 | global_key_prefix = "test_redis_search" 58 | 59 | class Django: 60 | model = Product 61 | fields = ["name", "description", "price", "created_at"] 62 | select_related_fields = ["vendor", "category"] 63 | prefetch_related_fields = ["tags"] 64 | related_models = { 65 | Vendor: { 66 | "related_name": "product", 67 | "many": False, 68 | }, 69 | Category: { 70 | "related_name": "product_set", 71 | "many": True, 72 | }, 73 | Tag: { 74 | "related_name": "product_set", 75 | "many": True, 76 | }, 77 | } 78 | 79 | return ProductJsonDocument, ( 80 | CategoryEmbeddedJsonDocument, 81 | TagEmbeddedJsonDocument, 82 | VendorEmbeddedJsonDocument, 83 | ) 84 | 85 | 86 | @pytest.fixture 87 | def product_obj(): 88 | return Product.objects.create( 89 | name="Test", 90 | price=10.0, 91 | vendor=Vendor.objects.create( 92 | name="Test", establishment_date=datetime.date.today() 93 | ), 94 | ) 95 | 96 | 97 | @pytest.fixture 98 | def tag_obj(): 99 | return Tag.objects.create(name="Test") 100 | 101 | 102 | @pytest.fixture 103 | def category_obj(): 104 | return Category.objects.create(name="Test") 105 | 106 | 107 | @pytest.fixture 108 | def product_with_tag(product_obj, tag_obj): 109 | product_obj.tags.add(tag_obj) 110 | return product_obj, tag_obj 111 | -------------------------------------------------------------------------------- /example/core/views.py: -------------------------------------------------------------------------------- 1 | from django.utils.functional import cached_property 2 | from django.views.generic import ListView 3 | from redis.commands.search import reducers 4 | 5 | from redis_search_django.mixins import RediSearchListViewMixin 6 | 7 | from .documents import ProductDocument 8 | from .models import Product 9 | 10 | 11 | class SearchView(RediSearchListViewMixin, ListView): 12 | paginate_by = 20 13 | model = Product 14 | template_name = "core/search.html" 15 | document_class = ProductDocument 16 | 17 | def get_context_data(self, *, object_list=None, **kwargs): 18 | return super().get_context_data(query_data=dict(self.request.GET), **kwargs) 19 | 20 | @cached_property 21 | def search_query_expression(self): 22 | query = self.request.GET.get("query") 23 | min_price = self.request.GET.get("min_price") 24 | max_price = self.request.GET.get("max_price") 25 | categories = list(filter(None, self.request.GET.getlist("category"))) 26 | tags = list(filter(None, self.request.GET.getlist("tags"))) 27 | query_expression = None 28 | 29 | if query: 30 | query_expression = ( 31 | self.document_class.name % query 32 | | self.document_class.description % query 33 | ) 34 | 35 | if min_price: 36 | min_price_expression = self.document_class.price >= float(min_price) 37 | query_expression = ( 38 | query_expression & min_price_expression 39 | if query_expression 40 | else min_price_expression 41 | ) 42 | 43 | if max_price: 44 | max_price_expression = self.document_class.price <= float(max_price) 45 | query_expression = ( 46 | query_expression & max_price_expression 47 | if query_expression 48 | else max_price_expression 49 | ) 50 | 51 | if categories: 52 | category_expression = self.document_class.category.name << categories 53 | query_expression = ( 54 | query_expression & category_expression 55 | if query_expression 56 | else category_expression 57 | ) 58 | 59 | if tags: 60 | tag_expression = self.document_class.tags.name << tags 61 | query_expression = ( 62 | query_expression & tag_expression 63 | if query_expression 64 | else tag_expression 65 | ) 66 | 67 | return query_expression 68 | 69 | @cached_property 70 | def sort_by(self): 71 | return self.request.GET.get("sort") 72 | 73 | def facets(self): 74 | if self.search_query_expression: 75 | request1 = self.document_class.build_aggregate_request( 76 | self.search_query_expression 77 | ) 78 | request2 = self.document_class.build_aggregate_request( 79 | self.search_query_expression 80 | ) 81 | else: 82 | request1 = self.document_class.build_aggregate_request() 83 | request2 = self.document_class.build_aggregate_request() 84 | result = self.document_class.aggregate( 85 | request1.group_by( 86 | ["@category_name"], 87 | reducers.count().alias("count"), 88 | ) 89 | ) 90 | result2 = self.document_class.aggregate( 91 | request2.group_by( 92 | ["@tags_name"], 93 | reducers.count().alias("count"), 94 | ) 95 | ) 96 | return [result, result2] 97 | -------------------------------------------------------------------------------- /example/core/migrations/0001_initial.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 3.2.15 on 2022-08-12 13:08 2 | 3 | import uuid 4 | 5 | import django.db.models.deletion 6 | from django.db import migrations, models 7 | 8 | 9 | class Migration(migrations.Migration): 10 | initial = True 11 | 12 | dependencies = [] 13 | 14 | operations = [ 15 | migrations.CreateModel( 16 | name="Category", 17 | fields=[ 18 | ( 19 | "id", 20 | models.BigAutoField( 21 | auto_created=True, 22 | primary_key=True, 23 | serialize=False, 24 | verbose_name="ID", 25 | ), 26 | ), 27 | ("name", models.CharField(max_length=30)), 28 | ("slug", models.SlugField(max_length=30)), 29 | ], 30 | ), 31 | migrations.CreateModel( 32 | name="Tag", 33 | fields=[ 34 | ( 35 | "id", 36 | models.BigAutoField( 37 | auto_created=True, 38 | primary_key=True, 39 | serialize=False, 40 | verbose_name="ID", 41 | ), 42 | ), 43 | ("name", models.CharField(max_length=30)), 44 | ], 45 | ), 46 | migrations.CreateModel( 47 | name="Vendor", 48 | fields=[ 49 | ( 50 | "id", 51 | models.BigAutoField( 52 | auto_created=True, 53 | primary_key=True, 54 | serialize=False, 55 | verbose_name="ID", 56 | ), 57 | ), 58 | ("logo", models.ImageField(blank=True, upload_to="core/vendor_logo")), 59 | ("identifier", models.UUIDField(default=uuid.uuid4, editable=False)), 60 | ("name", models.CharField(max_length=30)), 61 | ("email", models.EmailField(max_length=254)), 62 | ("establishment_date", models.DateField()), 63 | ], 64 | ), 65 | migrations.CreateModel( 66 | name="Product", 67 | fields=[ 68 | ( 69 | "id", 70 | models.BigAutoField( 71 | auto_created=True, 72 | primary_key=True, 73 | serialize=False, 74 | verbose_name="ID", 75 | ), 76 | ), 77 | ("name", models.CharField(max_length=256)), 78 | ("description", models.TextField(blank=True)), 79 | ("price", models.DecimalField(decimal_places=2, max_digits=6)), 80 | ("available", models.BooleanField(default=True)), 81 | ("quantity", models.IntegerField(default=1)), 82 | ("created_at", models.DateTimeField(auto_now_add=True)), 83 | ( 84 | "category", 85 | models.ForeignKey( 86 | blank=True, 87 | null=True, 88 | on_delete=django.db.models.deletion.SET_NULL, 89 | to="core.category", 90 | ), 91 | ), 92 | ("tags", models.ManyToManyField(blank=True, to="core.Tag")), 93 | ( 94 | "vendor", 95 | models.OneToOneField( 96 | on_delete=django.db.models.deletion.CASCADE, to="core.vendor" 97 | ), 98 | ), 99 | ], 100 | ), 101 | ] 102 | -------------------------------------------------------------------------------- /redis_search_django/registry.py: -------------------------------------------------------------------------------- 1 | from collections import defaultdict 2 | from dataclasses import dataclass 3 | from typing import TYPE_CHECKING, Dict, List, Set, Type, Union 4 | 5 | from django.conf import settings 6 | from django.db import models 7 | 8 | if TYPE_CHECKING: 9 | from .documents import Document 10 | 11 | 12 | @dataclass 13 | class DocumentRegistry: 14 | """Registry for Document classes.""" 15 | 16 | django_model_map: Dict[Type[models.Model], Set[Type["Document"]]] 17 | related_django_model_map: Dict[Type[models.Model], Set[Type["Document"]]] 18 | 19 | def __init__(self) -> None: 20 | """Initialize the registry.""" 21 | self.django_model_map = defaultdict(set) 22 | self.related_django_model_map = defaultdict(set) 23 | 24 | def register(self, document_class: Type["Document"]) -> None: 25 | """Register a Document class.""" 26 | self.django_model_map[document_class._django.model].add(document_class) 27 | 28 | for related_model in document_class._django.related_models: 29 | self.related_django_model_map[related_model].add(document_class) 30 | 31 | def update_document(self, model_object: models.Model, create: bool = True) -> None: 32 | """Update document of specific model.""" 33 | if not getattr(settings, "REDIS_SEARCH_AUTO_INDEX", True): 34 | return 35 | 36 | document_classes = self.django_model_map.get(model_object.__class__, set()) 37 | 38 | for document_class in document_classes: 39 | # Check if Auto Index is turned off for this specific Document. 40 | if not document_class._django.auto_index: 41 | continue 42 | 43 | # Try to Update the Document if not object is created. 44 | document_class.update_from_model_instance(model_object, create=create) 45 | 46 | def update_related_documents( 47 | self, 48 | model_object: models.Model, 49 | exclude: models.Model = None, 50 | ) -> None: 51 | """Update related documents of a specific model.""" 52 | if not getattr(settings, "REDIS_SEARCH_AUTO_INDEX", True): 53 | return 54 | 55 | document_classes = self.related_django_model_map.get( 56 | model_object.__class__, set() 57 | ) 58 | 59 | # Update the related Documents if any. 60 | for document_class in document_classes: 61 | if not document_class._django.auto_index: 62 | continue 63 | 64 | document_class.update_from_related_model_instance( 65 | model_object, exclude=exclude 66 | ) 67 | 68 | def remove_document(self, model_object: models.Model) -> None: 69 | """Remove document of a specific model.""" 70 | if not getattr(settings, "REDIS_SEARCH_AUTO_INDEX", True): 71 | return 72 | 73 | document_classes = self.django_model_map.get(model_object.__class__, set()) 74 | 75 | for document_class in document_classes: 76 | # Check if Auto Index is turned off for this specific Document. 77 | if not document_class._django.auto_index: 78 | continue 79 | 80 | # Try to Delete the Document from Redis Index. 81 | document_class.delete(model_object.pk) 82 | 83 | def index_documents(self, models: Union[List[str], None] = None) -> None: 84 | """Index documents for all or specific registered Django models.""" 85 | for ( 86 | django_model, 87 | document_classes, 88 | ) in self.django_model_map.items(): 89 | for document_class in document_classes: 90 | if models: 91 | if django_model._meta.label in models: 92 | document_class.index_all() 93 | else: 94 | document_class.index_all() 95 | 96 | 97 | document_registry: DocumentRegistry = DocumentRegistry() 98 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | share/python-wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | MANIFEST 28 | 29 | # PyInstaller 30 | # Usually these files are written by a python script from a template 31 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 32 | *.manifest 33 | *.spec 34 | 35 | # Installer logs 36 | pip-log.txt 37 | pip-delete-this-directory.txt 38 | 39 | # Unit test / coverage reports 40 | htmlcov/ 41 | .tox/ 42 | .nox/ 43 | .coverage 44 | .coverage.* 45 | .cache 46 | nosetests.xml 47 | coverage.xml 48 | *.cover 49 | *.py,cover 50 | .hypothesis/ 51 | .pytest_cache/ 52 | cover/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | .pybuilder/ 76 | target/ 77 | 78 | # Jupyter Notebook 79 | .ipynb_checkpoints 80 | 81 | # IPython 82 | profile_default/ 83 | ipython_config.py 84 | 85 | # pyenv 86 | # For a library or package, you might want to ignore these files since the code is 87 | # intended to run in multiple environments; otherwise, check them in: 88 | # .python-version 89 | 90 | # pipenv 91 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 92 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 93 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 94 | # install all needed dependencies. 95 | #Pipfile.lock 96 | 97 | # poetry 98 | # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. 99 | # This is especially recommended for binary packages to ensure reproducibility, and is more 100 | # commonly ignored for libraries. 101 | # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control 102 | #poetry.lock 103 | 104 | # pdm 105 | # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. 106 | #pdm.lock 107 | # pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it 108 | # in version control. 109 | # https://pdm.fming.dev/#use-with-ide 110 | .pdm.toml 111 | 112 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm 113 | __pypackages__/ 114 | 115 | # Celery stuff 116 | celerybeat-schedule 117 | celerybeat.pid 118 | 119 | # SageMath parsed files 120 | *.sage.py 121 | 122 | # Environments 123 | .env 124 | .venv 125 | env/ 126 | venv/ 127 | ENV/ 128 | env.bak/ 129 | venv.bak/ 130 | 131 | # Spyder project settings 132 | .spyderproject 133 | .spyproject 134 | 135 | # Rope project settings 136 | .ropeproject 137 | 138 | # mkdocs documentation 139 | /site 140 | 141 | # mypy 142 | .mypy_cache/ 143 | .dmypy.json 144 | dmypy.json 145 | 146 | # Pyre type checker 147 | .pyre/ 148 | 149 | # pytype static type analyzer 150 | .pytype/ 151 | 152 | # Cython debug symbols 153 | cython_debug/ 154 | 155 | # PyCharm 156 | # JetBrains specific template is maintained in a separate JetBrains.gitignore that can 157 | # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore 158 | # and can be added to the global gitignore or merged into this file. For a more nuclear 159 | # option (not recommended) you can uncomment the following to ignore the entire idea folder. 160 | .idea/ 161 | 162 | media/ 163 | -------------------------------------------------------------------------------- /example/example/settings.py: -------------------------------------------------------------------------------- 1 | """ 2 | Django settings for example project. 3 | 4 | Generated by 'django-admin startproject' using Django 3.2.15. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/3.2/topics/settings/ 8 | 9 | For the full list of settings and their values, see 10 | https://docs.djangoproject.com/en/3.2/ref/settings/ 11 | """ 12 | 13 | 14 | from pathlib import Path 15 | 16 | # Build paths inside the project like this: BASE_DIR / 'subdir'. 17 | from typing import List 18 | 19 | BASE_DIR = Path(__file__).resolve().parent.parent 20 | 21 | 22 | # Quick-start development settings - unsuitable for production 23 | # See https://docs.djangoproject.com/en/3.2/howto/deployment/checklist/ 24 | 25 | # SECURITY WARNING: keep the secret key used in production secret! 26 | SECRET_KEY = "django-insecure-*^iil*n4=4iv%d*)#9c!b5s=54cyl$@r&a^lm_$f@gb-*%e^q^" 27 | 28 | # SECURITY WARNING: don't run with debug turned on in production! 29 | DEBUG = True 30 | 31 | ALLOWED_HOSTS: List[str] = [] 32 | 33 | 34 | # Application definition 35 | 36 | INSTALLED_APPS = [ 37 | "django.contrib.admin", 38 | "django.contrib.auth", 39 | "django.contrib.contenttypes", 40 | "django.contrib.sessions", 41 | "django.contrib.messages", 42 | "django.contrib.staticfiles", 43 | "redis_search_django", 44 | "core", 45 | ] 46 | 47 | MIDDLEWARE = [ 48 | "django.middleware.security.SecurityMiddleware", 49 | "django.contrib.sessions.middleware.SessionMiddleware", 50 | "django.middleware.common.CommonMiddleware", 51 | "django.middleware.csrf.CsrfViewMiddleware", 52 | "django.contrib.auth.middleware.AuthenticationMiddleware", 53 | "django.contrib.messages.middleware.MessageMiddleware", 54 | "django.middleware.clickjacking.XFrameOptionsMiddleware", 55 | ] 56 | 57 | ROOT_URLCONF = "example.urls" 58 | 59 | TEMPLATES = [ 60 | { 61 | "BACKEND": "django.template.backends.django.DjangoTemplates", 62 | "DIRS": [], 63 | "APP_DIRS": True, 64 | "OPTIONS": { 65 | "context_processors": [ 66 | "django.template.context_processors.debug", 67 | "django.template.context_processors.request", 68 | "django.contrib.auth.context_processors.auth", 69 | "django.contrib.messages.context_processors.messages", 70 | ], 71 | }, 72 | }, 73 | ] 74 | 75 | WSGI_APPLICATION = "example.wsgi.application" 76 | 77 | 78 | # Database 79 | # https://docs.djangoproject.com/en/3.2/ref/settings/#databases 80 | 81 | DATABASES = { 82 | "default": { 83 | "ENGINE": "django.db.backends.sqlite3", 84 | "NAME": BASE_DIR / "db.sqlite3", 85 | } 86 | } 87 | 88 | 89 | # Password validation 90 | # https://docs.djangoproject.com/en/3.2/ref/settings/#auth-password-validators 91 | 92 | AUTH_PASSWORD_VALIDATORS = [ 93 | { 94 | "NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator", # noqa: E501 95 | }, 96 | { 97 | "NAME": "django.contrib.auth.password_validation.MinimumLengthValidator", 98 | }, 99 | { 100 | "NAME": "django.contrib.auth.password_validation.CommonPasswordValidator", 101 | }, 102 | { 103 | "NAME": "django.contrib.auth.password_validation.NumericPasswordValidator", 104 | }, 105 | ] 106 | 107 | 108 | # Internationalization 109 | # https://docs.djangoproject.com/en/3.2/topics/i18n/ 110 | 111 | LANGUAGE_CODE = "en-us" 112 | 113 | TIME_ZONE = "UTC" 114 | 115 | USE_I18N = True 116 | 117 | USE_L10N = True 118 | 119 | USE_TZ = True 120 | 121 | 122 | # Static files (CSS, JavaScript, Images) 123 | # https://docs.djangoproject.com/en/3.2/howto/static-files/ 124 | 125 | STATIC_URL = "/static/" 126 | 127 | # URL to serve media files 128 | MEDIA_URL = "/media/" 129 | 130 | # Path where media is stored 131 | MEDIA_ROOT = BASE_DIR / "media/" 132 | 133 | # Default primary key field type 134 | # https://docs.djangoproject.com/en/3.2/ref/settings/#default-auto-field 135 | 136 | DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField" 137 | 138 | # redis-search-django settings 139 | 140 | REDIS_SEARCH_AUTO_INDEX = True 141 | -------------------------------------------------------------------------------- /redis_search_django/config.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | import uuid 3 | from decimal import Decimal 4 | from typing import Any, Dict, Type 5 | 6 | from django.db import models 7 | 8 | # A mapping for Django model fields and Redis OM fields data 9 | model_field_class_config: Dict[Type[models.Field], Dict[str, Any]] = { 10 | models.AutoField: { 11 | "type": int, 12 | "full_text_search": False, 13 | "index": True, 14 | "sortable": True, 15 | }, 16 | models.BigAutoField: { 17 | "type": int, 18 | "full_text_search": False, 19 | "index": True, 20 | "sortable": True, 21 | }, 22 | models.BigIntegerField: { 23 | "type": int, 24 | "full_text_search": False, 25 | "index": True, 26 | "sortable": True, 27 | }, 28 | # We are using int type for models.BooleanField 29 | # as redis-om creates schema for this field as NUMERIC field. 30 | # see https://github.com/redis/redis-om-python/issues/193 31 | models.BooleanField: { 32 | "type": int, 33 | "full_text_search": False, 34 | "index": True, 35 | "sortable": True, 36 | }, 37 | models.CharField: { 38 | "type": str, 39 | "full_text_search": True, 40 | "index": True, 41 | "sortable": True, 42 | }, 43 | models.DateField: { 44 | "type": datetime.date, 45 | "full_text_search": False, 46 | "index": True, 47 | "sortable": False, 48 | }, 49 | models.DateTimeField: { 50 | "type": datetime.datetime, 51 | "full_text_search": False, 52 | "index": True, 53 | "sortable": False, 54 | }, 55 | models.DecimalField: { 56 | "type": Decimal, 57 | "full_text_search": False, 58 | "index": True, 59 | "sortable": True, 60 | }, 61 | models.EmailField: { 62 | "type": str, 63 | "full_text_search": True, 64 | "index": True, 65 | "sortable": False, 66 | }, 67 | models.FileField: { 68 | "type": str, 69 | "full_text_search": False, 70 | "index": True, 71 | "sortable": False, 72 | }, 73 | models.FilePathField: { 74 | "type": str, 75 | "full_text_search": False, 76 | "index": True, 77 | "sortable": False, 78 | }, 79 | models.FloatField: { 80 | "type": float, 81 | "full_text_search": False, 82 | "index": True, 83 | "sortable": True, 84 | }, 85 | models.ImageField: { 86 | "type": str, 87 | "full_text_search": False, 88 | "index": True, 89 | "sortable": False, 90 | }, 91 | models.IntegerField: { 92 | "type": int, 93 | "full_text_search": False, 94 | "index": True, 95 | "sortable": True, 96 | }, 97 | models.PositiveIntegerField: { 98 | "type": int, 99 | "full_text_search": False, 100 | "index": True, 101 | "sortable": True, 102 | }, 103 | models.PositiveSmallIntegerField: { 104 | "type": int, 105 | "full_text_search": False, 106 | "index": True, 107 | "sortable": True, 108 | }, 109 | models.SlugField: { 110 | "type": str, 111 | "full_text_search": False, 112 | "index": True, 113 | "sortable": False, 114 | }, 115 | models.SmallIntegerField: { 116 | "type": int, 117 | "full_text_search": False, 118 | "index": True, 119 | "sortable": True, 120 | }, 121 | models.TextField: { 122 | "type": str, 123 | "full_text_search": True, 124 | "index": True, 125 | "sortable": False, 126 | }, 127 | models.TimeField: { 128 | "type": datetime.time, 129 | "full_text_search": False, 130 | "index": True, 131 | "sortable": False, 132 | }, 133 | models.URLField: { 134 | "type": str, 135 | "full_text_search": False, 136 | "index": True, 137 | "sortable": False, 138 | }, 139 | models.UUIDField: { 140 | "type": uuid.UUID, 141 | "full_text_search": False, 142 | "index": True, 143 | "sortable": False, 144 | }, 145 | } 146 | -------------------------------------------------------------------------------- /tests/test_mixins.py: -------------------------------------------------------------------------------- 1 | from unittest import mock 2 | 3 | import pytest 4 | from django.core.exceptions import ImproperlyConfigured 5 | from django.test import RequestFactory 6 | from django.utils.functional import cached_property 7 | from django.views import View 8 | from django.views.generic import ListView 9 | 10 | from redis_search_django.documents import JsonDocument 11 | from redis_search_django.mixins import ( 12 | RediSearchListViewMixin, 13 | RediSearchMixin, 14 | RediSearchMultipleObjectMixin, 15 | RediSearchTemplateResponseMixin, 16 | ) 17 | from redis_search_django.paginator import RediSearchPaginator 18 | from redis_search_django.query import RediSearchQuery 19 | 20 | from .helpers import is_redis_running 21 | from .models import Category 22 | 23 | 24 | def test_redis_search_mixin(document_class): 25 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 26 | DocumentClass.db = mock.MagicMock() 27 | mixin = RediSearchMixin() 28 | 29 | with pytest.raises(ImproperlyConfigured): 30 | mixin.search() 31 | 32 | mixin.document_class = DocumentClass 33 | 34 | assert mixin.search_query_expression is None 35 | assert mixin.sort_by is None 36 | assert mixin.facets() is None 37 | assert isinstance(mixin.search(), RediSearchQuery) 38 | 39 | 40 | def test_redis_search_mixin_with_view(document_class): 41 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 42 | DocumentClass.db = mock.MagicMock() 43 | 44 | class SearchView(RediSearchMixin, View): 45 | document_class = DocumentClass 46 | 47 | @cached_property 48 | def search_query_expression(self): 49 | return self.document_class.name % "test" 50 | 51 | @cached_property 52 | def sort_by(self): 53 | return "name" 54 | 55 | view = SearchView() 56 | search = view.search() 57 | 58 | assert isinstance(search, RediSearchQuery) 59 | assert search.sort_fields == ["name"] 60 | assert search.expressions 61 | 62 | 63 | def test_redis_search_template_response_mixin(document_class): 64 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 65 | DocumentClass.db = mock.MagicMock() 66 | 67 | class SearchView(RediSearchMixin, RediSearchTemplateResponseMixin, View): 68 | document_class = DocumentClass 69 | template_name = "search.html" 70 | 71 | view = SearchView() 72 | 73 | assert view.get_template_names() == ["search.html"] 74 | 75 | 76 | def test_redis_search_template_response_mixin_without_template_name(document_class): 77 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 78 | DocumentClass.db = mock.MagicMock() 79 | 80 | class SearchView(RediSearchMixin, RediSearchTemplateResponseMixin, View): 81 | document_class = DocumentClass 82 | 83 | view = SearchView() 84 | 85 | assert view.get_template_names() == ["tests/category_list.html"] 86 | 87 | 88 | def test_redis_search_template_response_mixin_without_document_class(document_class): 89 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 90 | DocumentClass.db = mock.MagicMock() 91 | 92 | class SearchView(RediSearchMixin, RediSearchTemplateResponseMixin, View): 93 | document_class = None 94 | 95 | view = SearchView() 96 | 97 | with pytest.raises(ImproperlyConfigured): 98 | view.get_template_names() 99 | 100 | 101 | def test_redis_search_multiple_object_mixin(document_class): 102 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 103 | DocumentClass.db = mock.MagicMock() 104 | 105 | class SearchView(RediSearchMixin, RediSearchMultipleObjectMixin, View): 106 | document_class = DocumentClass 107 | context_object_name = "categories" 108 | 109 | view = SearchView() 110 | 111 | assert view.get_context_object_name([]) == "categories" 112 | 113 | 114 | def test_redis_search_multiple_object_mixin_without_object_name(document_class): 115 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 116 | DocumentClass.db = mock.MagicMock() 117 | 118 | class SearchView(RediSearchMixin, RediSearchMultipleObjectMixin, View): 119 | document_class = DocumentClass 120 | 121 | view = SearchView() 122 | 123 | assert view.get_context_object_name([]) == "category_list" 124 | 125 | 126 | def test_redis_search_multiple_object_mixin_without_document_class(document_class): 127 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 128 | DocumentClass.db = mock.MagicMock() 129 | 130 | class SearchView(RediSearchMixin, RediSearchMultipleObjectMixin, View): 131 | document_class = None 132 | 133 | view = SearchView() 134 | 135 | assert view.get_context_object_name([]) is None 136 | 137 | 138 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 139 | @mock.patch("redis_search_django.documents.JsonDocument.find") 140 | def test_list_view_mixin(find, document_class): 141 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 142 | 143 | class SearchView(RediSearchListViewMixin, ListView): 144 | paginate_by = 20 145 | model = Category 146 | template_name = "core/search.html" 147 | document_class = DocumentClass 148 | 149 | request = RequestFactory().get("/some_url") 150 | response = SearchView.as_view()(request) 151 | 152 | find.assert_called_once() 153 | assert isinstance(response.context_data["paginator"], RediSearchPaginator) 154 | -------------------------------------------------------------------------------- /example/core/templates/core/search.html: -------------------------------------------------------------------------------- 1 | {% load build_query_string %} 2 | 3 | 4 | 5 | 6 | 28 | 29 | 30 |

Django Redis Search Example

31 |
32 |
33 |

Full Text Search

 34 | Search Query

35 |

36 | 37 | Min Price

38 |

39 | Max Price

40 |

41 | 42 | Sort

43 | 44 |
45 | 46 |

47 | 48 | 49 |
50 | 51 |

52 | 53 | Category

54 | {% for category in facets.0 %} 55 | 56 |
57 | {% endfor %}
58 | 59 | Tags

60 | {% for tag in facets.1 %} 61 | 62 |
63 | {% endfor %}
64 | 65 | 66 |
67 |
68 | 69 |

Search results ({{ object_list.count }})

70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | {% for result in object_list %} 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | {% empty %} 94 | 95 | {% endfor %} 96 |
# IDProduct NameVendorPriceQuantityCategoryTagsAdded At
{{ result.id }}{{ result.name }}{{ result.vendor.name }}{{ result.price }} ${{ result.quantity }}{{ result.category.name }}{% for tag in result.tags %} {{ tag.name }} {% if not forloop.last %}, {% endif %}{% endfor %}{{ result.created_at }}
No results found.
97 | 98 | {% if is_paginated %} 99 | 120 | {% endif %} 121 |
122 | 123 | 124 | -------------------------------------------------------------------------------- /redis_search_django/query.py: -------------------------------------------------------------------------------- 1 | from typing import Any, Generator, List, Type, Union 2 | 3 | from django.db import models 4 | from django.db.models import Case, IntegerField, When 5 | from redis_om import FindQuery, RedisModel 6 | 7 | 8 | class RediSearchResult: 9 | """Class That Stores Redis Search Results""" 10 | 11 | def __init__( 12 | self, 13 | results: List[RedisModel], 14 | hit_count: int, 15 | django_model: Union[Type[models.Model], None], 16 | ): 17 | self.results = results 18 | self.hit_count = hit_count 19 | self.django_model = django_model 20 | 21 | def __str__(self) -> str: 22 | return repr(self) 23 | 24 | def __repr__(self) -> str: 25 | return "<{} {} of {}>".format( 26 | self.__class__.__name__, len(self), self.hit_count 27 | ) 28 | 29 | def __iter__(self, *args: Any, **kwargs: Any) -> Generator[Any, None, None]: 30 | yield from self.results 31 | 32 | def __len__(self) -> int: 33 | return len(self.results) 34 | 35 | def __getitem__(self, index: int) -> RedisModel: 36 | return self.results[index] 37 | 38 | def __setitem__(self, index: int, value: RedisModel) -> None: 39 | self.results[index] = value 40 | 41 | def __contains__(self, item: RedisModel) -> bool: 42 | return item in self.results 43 | 44 | def __bool__(self) -> bool: 45 | return bool(self.results) 46 | 47 | def clear(self) -> None: 48 | """Clears the results""" 49 | self.results = [] 50 | self.hit_count = 0 51 | 52 | def add(self, data: Union[RedisModel, List[RedisModel]]) -> None: 53 | """Adds data to the results""" 54 | if isinstance(data, list): 55 | self.results += data 56 | else: 57 | self.results.append(data) 58 | 59 | def count(self) -> int: 60 | """ 61 | Returns the number of hits. 62 | 63 | This is not same as `len(self.results)` as 64 | `self.results` may contain only paginated results. 65 | """ 66 | return self.hit_count 67 | 68 | def exists(self) -> bool: 69 | """Returns True if there are results""" 70 | return bool(self.results) 71 | 72 | def to_queryset(self) -> models.QuerySet: 73 | """Converts the search results to a Django QuerySet""" 74 | if not self.django_model: 75 | raise ValueError("No Django Model has been set") 76 | 77 | # if no results, return empty queryset 78 | if not self.results: 79 | return self.django_model.objects.none() 80 | 81 | pks = [result.pk for result in self.results] 82 | return self.django_model.objects.filter(pk__in=pks).order_by( 83 | Case( 84 | *[When(pk=pk, then=position) for position, pk in enumerate(pks)], 85 | output_field=IntegerField(), 86 | ) 87 | ) 88 | 89 | 90 | class RediSearchQuery(FindQuery): 91 | def __init__(self, *args: Any, **kwargs: Any) -> None: 92 | self.django_model = kwargs.pop("django_model", None) 93 | super().__init__(*args, **kwargs) 94 | # Initialize the cache with empty RediSearchResult. 95 | self._model_cache: RediSearchResult = RediSearchResult( 96 | results=[], hit_count=0, django_model=self.django_model 97 | ) 98 | 99 | def to_queryset(self) -> models.QuerySet: 100 | """Converts the search results to a Django QuerySet""" 101 | if self._model_cache: 102 | return self._model_cache.to_queryset() 103 | # If no results, then execute the query and return Django QuerySet. 104 | return self.execute().to_queryset() 105 | 106 | def all(self, batch_size: int = 10) -> "RediSearchQuery": 107 | """ 108 | Only Update batch size and return instance. 109 | 110 | Overridden so that we do not run `execute()` if this method is called. 111 | """ 112 | if batch_size != self.page_size: 113 | query = self.copy(page_size=batch_size, limit=batch_size) 114 | return query 115 | return self 116 | 117 | def count(self) -> int: 118 | """Return the number of hits from `_model_cache`""" 119 | return self._model_cache.count() 120 | 121 | def exists(self) -> bool: 122 | """Returns True if there are results from `_model_cache`""" 123 | return self._model_cache.exists() 124 | 125 | def copy(self, **kwargs: Any) -> "RediSearchQuery": 126 | """ "Returns a copy of the class""" 127 | original = self.dict() 128 | original.update(**kwargs) 129 | return self.__class__(**original) 130 | 131 | def paginate(self, offset: int = 0, limit: int = 100) -> None: 132 | """Paginates the results.""" 133 | self.offset = offset 134 | self.limit = limit 135 | 136 | def execute(self, exhaust_results: bool = True) -> RediSearchResult: 137 | """Executes the search query and returns a RediSearchResult""" 138 | args = ["ft.search", self.model.Meta.index_name, self.query, *self.pagination] 139 | 140 | if self.sort_fields: 141 | args += self.resolve_redisearch_sort_fields() 142 | 143 | # Reset the cache if we're executing from offset 0. 144 | if self.offset == 0: 145 | self._model_cache.clear() 146 | 147 | # If the offset is greater than 0, we're paginating through a result set, 148 | # so append the new results to results already in the cache. 149 | raw_result = self.model.db().execute_command(*args) 150 | count = raw_result[0] 151 | results = self.model.from_redis(raw_result) 152 | # Update the cache with the new results. 153 | self._model_cache.add(results) 154 | self._model_cache.hit_count = count 155 | 156 | if not exhaust_results: 157 | return self._model_cache 158 | 159 | # The query returned all results, so we have no more work to do. 160 | if count <= len(results): 161 | return self._model_cache 162 | 163 | # Transparently (to the user) make subsequent requests to paginate 164 | # through the results and finally return them all. 165 | query = self 166 | while True: 167 | # Make a query for each pass of the loop, with a new offset equal to the 168 | # current offset plus `page_size`, until we stop getting results back. 169 | query = query.copy(offset=query.offset + query.page_size) 170 | _results = query.execute(exhaust_results=False) 171 | if not _results: 172 | break 173 | self._model_cache.add(_results) 174 | 175 | return self._model_cache 176 | -------------------------------------------------------------------------------- /tests/test_integration.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | 3 | import pytest 4 | from redis_om import NotFoundError 5 | 6 | from redis_search_django.documents import JsonDocument 7 | 8 | from .helpers import is_redis_running 9 | from .models import Category, Product, Tag, Vendor 10 | 11 | 12 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 13 | @pytest.mark.django_db 14 | def test_object_create(document_class): 15 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 16 | 17 | category = Category.objects.create(name="test") 18 | document_obj = DocumentClass.get(pk=category.pk) 19 | 20 | assert int(document_obj.pk) == category.pk 21 | assert document_obj.name == category.name 22 | 23 | 24 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 25 | @pytest.mark.django_db 26 | def test_object_update(document_class): 27 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 28 | 29 | category = Category.objects.create(name="test") 30 | document_obj = DocumentClass.get(pk=category.pk) 31 | 32 | assert document_obj.name == "test" 33 | 34 | category.name = "test2" 35 | category.save() 36 | 37 | document_obj = DocumentClass.get(pk=category.pk) 38 | 39 | assert document_obj.name == "test2" 40 | 41 | 42 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 43 | @pytest.mark.django_db 44 | def test_object_delete(document_class): 45 | DocumentClass = document_class(JsonDocument, Category, ["name"]) 46 | 47 | category = Category.objects.create(name="test") 48 | document_obj = DocumentClass.get(pk=category.pk) 49 | 50 | assert int(document_obj.pk) == category.pk 51 | 52 | category.delete() 53 | 54 | with pytest.raises(NotFoundError): 55 | DocumentClass.get(pk=category.pk) 56 | 57 | 58 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 59 | @pytest.mark.django_db 60 | def test_related_object_add(nested_document_class): 61 | ProductDocumentCalss = nested_document_class[0] 62 | vendor = Vendor.objects.create( 63 | name="test", establishment_date=datetime.date.today() 64 | ) 65 | product = Product.objects.create( 66 | name="Test", 67 | price=10.0, 68 | vendor=vendor, 69 | ) 70 | 71 | document_obj = ProductDocumentCalss.get(pk=product.pk) 72 | 73 | assert int(document_obj.pk) == product.pk 74 | assert document_obj.name == product.name 75 | assert document_obj.vendor.name == vendor.name 76 | assert document_obj.category is None 77 | assert document_obj.tags == [] 78 | 79 | category = Category.objects.create(name="test") 80 | product.category = category 81 | product.save() 82 | 83 | document_obj = ProductDocumentCalss.get(pk=product.pk) 84 | 85 | assert document_obj.category.name == category.name 86 | assert int(document_obj.category.pk) == category.pk 87 | assert document_obj.tags == [] 88 | 89 | tag = Tag.objects.create(name="test") 90 | tag2 = Tag.objects.create(name="test2") 91 | product.tags.set([tag, tag2]) 92 | 93 | document_obj = ProductDocumentCalss.get(pk=product.pk) 94 | 95 | assert len(document_obj.tags) == 2 96 | assert int(document_obj.tags[0].pk) == tag.pk 97 | assert int(document_obj.tags[1].pk) == tag2.pk 98 | 99 | 100 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 101 | @pytest.mark.django_db 102 | def test_related_object_update(nested_document_class): 103 | ProductDocumentCalss = nested_document_class[0] 104 | vendor = Vendor.objects.create( 105 | name="test", establishment_date=datetime.date.today() 106 | ) 107 | product = Product.objects.create( 108 | name="Test", 109 | price=10.0, 110 | vendor=vendor, 111 | ) 112 | 113 | document_obj = ProductDocumentCalss.get(pk=product.pk) 114 | 115 | assert document_obj.vendor.name == "test" 116 | 117 | vendor.name = "test2" 118 | vendor.save() 119 | 120 | document_obj = ProductDocumentCalss.get(pk=product.pk) 121 | 122 | assert document_obj.vendor.name == "test2" 123 | 124 | category = Category.objects.create(name="test") 125 | product.category = category 126 | product.save() 127 | 128 | document_obj = ProductDocumentCalss.get(pk=product.pk) 129 | 130 | assert document_obj.category.name == "test" 131 | 132 | category.name = "test2" 133 | category.save() 134 | 135 | document_obj = ProductDocumentCalss.get(pk=product.pk) 136 | 137 | assert document_obj.category.name == "test2" 138 | 139 | tag = Tag.objects.create(name="test") 140 | tag2 = Tag.objects.create(name="test2") 141 | product.tags.set([tag, tag2]) 142 | 143 | document_obj = ProductDocumentCalss.get(pk=product.pk) 144 | 145 | assert document_obj.tags[0].name == "test" 146 | assert document_obj.tags[1].name == "test2" 147 | 148 | tag.name = "test3" 149 | tag.save() 150 | tag2.name = "test4" 151 | tag2.save() 152 | 153 | document_obj = ProductDocumentCalss.get(pk=product.pk) 154 | 155 | assert document_obj.tags[0].name == "test3" 156 | assert document_obj.tags[1].name == "test4" 157 | 158 | 159 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 160 | @pytest.mark.django_db 161 | def test_related_object_delete(nested_document_class): 162 | ProductDocumentCalss = nested_document_class[0] 163 | vendor = Vendor.objects.create( 164 | name="test", establishment_date=datetime.date.today() 165 | ) 166 | product = Product.objects.create( 167 | name="Test", 168 | price=10.0, 169 | vendor=vendor, 170 | ) 171 | 172 | category = Category.objects.create(name="test") 173 | product.category = category 174 | product.save() 175 | 176 | document_obj = ProductDocumentCalss.get(pk=product.pk) 177 | 178 | assert int(document_obj.category.pk) == category.pk 179 | 180 | category.delete() 181 | 182 | document_obj = ProductDocumentCalss.get(pk=product.pk) 183 | 184 | assert document_obj.category is None 185 | 186 | tag = Tag.objects.create(name="test") 187 | tag2 = Tag.objects.create(name="test2") 188 | product.tags.set([tag, tag2]) 189 | 190 | document_obj = ProductDocumentCalss.get(pk=product.pk) 191 | 192 | assert int(document_obj.tags[0].pk) == tag.pk 193 | assert int(document_obj.tags[1].pk) == tag2.pk 194 | 195 | tag.delete() 196 | 197 | document_obj = ProductDocumentCalss.get(pk=product.pk) 198 | 199 | assert len(document_obj.tags) == 1 200 | assert int(document_obj.tags[0].pk) == tag2.pk 201 | 202 | tag2.delete() 203 | 204 | document_obj = ProductDocumentCalss.get(pk=product.pk) 205 | 206 | assert len(document_obj.tags) == 0 207 | 208 | vendor.delete() 209 | 210 | with pytest.raises(NotFoundError): 211 | ProductDocumentCalss.get(pk=product.pk) 212 | -------------------------------------------------------------------------------- /tests/test_query.py: -------------------------------------------------------------------------------- 1 | from unittest import mock 2 | 3 | import pytest 4 | from pytest_django.asserts import assertQuerysetEqual 5 | 6 | from redis_search_django.query import RediSearchQuery, RediSearchResult 7 | 8 | from .models import Category 9 | 10 | 11 | def test_search_result_str(): 12 | result = RediSearchResult([], 10, None) 13 | assert str(result) == "" 14 | 15 | 16 | def test_search_result_repr(): 17 | result = RediSearchResult([], 10, None) 18 | assert repr(result) == "" 19 | 20 | 21 | def test_search_result_iter(): 22 | item_1 = mock.MagicMock() 23 | item_2 = mock.MagicMock() 24 | result = RediSearchResult([item_1, item_2], 10, None) 25 | assert list(result) == [item_1, item_2] 26 | 27 | 28 | def test_search_result_len(): 29 | result = RediSearchResult([mock.MagicMock(), mock.MagicMock()], 10, None) 30 | assert len(result) == 2 31 | 32 | 33 | def test_search_result_getitem(): 34 | item_1 = mock.MagicMock() 35 | item_2 = mock.MagicMock() 36 | result = RediSearchResult([item_1, item_2], 10, None) 37 | 38 | assert result[0] == item_1 39 | assert result[1] == item_2 40 | 41 | 42 | def test_search_result_setitem(): 43 | item = mock.MagicMock() 44 | result = RediSearchResult([mock.MagicMock()], 10, None) 45 | result[0] = item 46 | 47 | assert result[0] == item 48 | 49 | 50 | def test_search_result_contains(): 51 | item = mock.MagicMock() 52 | result = RediSearchResult([item], 10, None) 53 | assert item in result 54 | 55 | 56 | def test_search_result_bool(): 57 | result = RediSearchResult([mock.MagicMock()], 10, None) 58 | assert bool(result) 59 | 60 | 61 | def test_search_result_clear(): 62 | item = mock.MagicMock() 63 | result = RediSearchResult([item], 10, None) 64 | assert len(result) == 1 65 | assert result.hit_count == 10 66 | 67 | result.clear() 68 | 69 | assert len(result) == 0 70 | assert result.hit_count == 0 71 | 72 | 73 | def test_search_result_add(): 74 | item = mock.MagicMock() 75 | result = RediSearchResult([], 10, None) 76 | 77 | assert len(result) == 0 78 | 79 | result.add(item) 80 | 81 | assert len(result) == 1 82 | assert result.hit_count == 10 83 | 84 | result.add([item, item]) 85 | 86 | assert len(result) == 3 87 | 88 | 89 | def test_search_result_count(): 90 | result = RediSearchResult([], 10, None) 91 | 92 | assert result.count() == 10 93 | 94 | result.clear() 95 | 96 | assert result.count() == 0 97 | 98 | result.add([mock.MagicMock(), mock.MagicMock()]) 99 | 100 | assert result.count() == 0 101 | 102 | 103 | def test_search_result_exists(): 104 | result = RediSearchResult([mock.MagicMock()], 10, None) 105 | assert result.exists() 106 | result.clear() 107 | assert not result.exists() 108 | 109 | 110 | @pytest.mark.django_db 111 | def test_search_result_to_queryset(category_obj): 112 | item = mock.MagicMock(name="Item", pk=category_obj.pk) 113 | 114 | assertQuerysetEqual( 115 | RediSearchResult([item], 10, Category).to_queryset(), 116 | Category.objects.all(), 117 | ) 118 | assertQuerysetEqual( 119 | RediSearchResult([], 10, Category).to_queryset(), Category.objects.none() 120 | ) 121 | 122 | 123 | @pytest.mark.django_db 124 | def test_search_result_to_queryset_no_model_set(): 125 | with pytest.raises(ValueError): 126 | RediSearchResult([], 10, None).to_queryset() 127 | 128 | 129 | @pytest.mark.django_db 130 | def test_search_query_to_queryset(category_obj): 131 | item = mock.MagicMock(name="Item", pk=category_obj.pk) 132 | 133 | query = RediSearchQuery(mock.MagicMock(), model=mock.MagicMock()) 134 | query._model_cache = RediSearchResult([item], 10, Category) 135 | 136 | assertQuerysetEqual(query.to_queryset(), Category.objects.all()) 137 | 138 | 139 | @mock.patch("redis_search_django.query.RediSearchQuery.execute") 140 | @pytest.mark.django_db 141 | def test_search_query_to_queryset_without_result(execute, category_obj): 142 | query = RediSearchQuery(mock.MagicMock(), model=mock.MagicMock()) 143 | 144 | query.to_queryset() 145 | 146 | execute().to_queryset.assert_called_once() 147 | 148 | 149 | @pytest.mark.django_db 150 | def test_search_query_all(): 151 | query = RediSearchQuery(mock.MagicMock(), model=mock.MagicMock()) 152 | assert isinstance(query.all(), RediSearchQuery) 153 | new_query = query.all(batch_size=1000) 154 | assert isinstance(new_query, RediSearchQuery) 155 | assert new_query.limit == 1000 156 | 157 | 158 | def test_search_query_count(): 159 | query = RediSearchQuery(mock.MagicMock(), model=mock.MagicMock()) 160 | query._model_cache = RediSearchResult([], 10, None) 161 | 162 | assert query.count() == 10 163 | 164 | 165 | def test_search_query_exists(): 166 | query = RediSearchQuery(mock.MagicMock(), model=mock.MagicMock()) 167 | query._model_cache = RediSearchResult([mock.MagicMock()], 10, None) 168 | assert query.exists() 169 | 170 | 171 | def test_search_query_copy(): 172 | query = RediSearchQuery(mock.MagicMock(), model=mock.MagicMock()) 173 | new_query = query.copy() 174 | assert query is not new_query 175 | 176 | 177 | def test_search_query_paginate(): 178 | query = RediSearchQuery(mock.MagicMock(), model=mock.MagicMock()) 179 | offset = 10 180 | limit = 20 181 | query.paginate(offset=offset, limit=limit) 182 | assert query.offset == offset 183 | assert query.limit == limit 184 | 185 | 186 | def test_search_query_execute(): 187 | model = mock.MagicMock() 188 | model.db().execute_command.return_value = [1, [mock.MagicMock()]] 189 | model.from_redis.return_value = [mock.MagicMock()] 190 | query = RediSearchQuery([], model=model) 191 | 192 | result = query.execute() 193 | 194 | assert isinstance(result, RediSearchResult) 195 | assert len(result) == 1 196 | assert result.hit_count == 1 197 | 198 | 199 | def test_search_query_execute_with_exhaust_results(): 200 | model = mock.MagicMock() 201 | model.db().execute_command.return_value = [2, [mock.MagicMock()]] 202 | model.from_redis.side_effect = [[mock.MagicMock()], [mock.MagicMock()], []] 203 | query = RediSearchQuery([], model=model) 204 | 205 | result = query.execute(exhaust_results=True) 206 | 207 | assert isinstance(result, RediSearchResult) 208 | assert len(result) == 2 209 | assert result.hit_count == 2 210 | 211 | 212 | def test_search_query_execute_without_offset(): 213 | model = mock.MagicMock() 214 | model.db().execute_command.return_value = [1, [mock.MagicMock()]] 215 | model.from_redis.return_value = [mock.MagicMock()] 216 | query = RediSearchQuery([], model=model, offset=0) 217 | query._model_cache = RediSearchResult([mock.MagicMock()], 10, None) 218 | 219 | result = query.execute() 220 | 221 | assert isinstance(result, RediSearchResult) 222 | assert len(result) == 1 223 | assert result.hit_count == 1 224 | 225 | 226 | def test_search_query_execute_with_offset(): 227 | model = mock.MagicMock() 228 | model.db().execute_command.return_value = [1, [mock.MagicMock()]] 229 | model.from_redis.return_value = [mock.MagicMock()] 230 | query = RediSearchQuery([], model=model, offset=10) 231 | query._model_cache = RediSearchResult([mock.MagicMock()], 10, None) 232 | 233 | result = query.execute() 234 | 235 | assert isinstance(result, RediSearchResult) 236 | assert len(result) == 2 237 | assert result.hit_count == 1 238 | -------------------------------------------------------------------------------- /tests/test_registry.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | from collections import defaultdict 3 | from typing import List, Optional 4 | from unittest import mock 5 | 6 | from redis_search_django.documents import ( 7 | EmbeddedJsonDocument, 8 | HashDocument, 9 | JsonDocument, 10 | ) 11 | from redis_search_django.registry import DocumentRegistry 12 | from tests.models import Category, Product, Tag, Vendor 13 | 14 | 15 | def test_empty_registry(): 16 | registry = DocumentRegistry() 17 | assert registry.django_model_map == {} 18 | assert registry.related_django_model_map == defaultdict(set) 19 | 20 | 21 | def test_register(document_class): 22 | registry = DocumentRegistry() 23 | VendorDocumentClass = document_class(JsonDocument, Vendor, ["name"]) 24 | CategoryDocumentClass = document_class(HashDocument, Category, ["name"]) 25 | registry.register(VendorDocumentClass) 26 | registry.register(CategoryDocumentClass) 27 | 28 | assert registry.django_model_map == { 29 | Vendor: {VendorDocumentClass}, 30 | Category: {CategoryDocumentClass}, 31 | } 32 | assert registry.related_django_model_map == defaultdict(set) 33 | 34 | 35 | def test_register_with_nested_documents(document_class): 36 | CategoryJsonDocument = document_class(JsonDocument, Category, ["name"]) 37 | CategoryEmbeddedJsonDocument = document_class( 38 | EmbeddedJsonDocument, Category, ["name"] 39 | ) 40 | TagEmbeddedJsonDocument = document_class(EmbeddedJsonDocument, Tag, ["name"]) 41 | VendorEmbeddedJsonDocument = document_class( 42 | EmbeddedJsonDocument, Vendor, ["name", "establishment_date"] 43 | ) 44 | 45 | class ProductJsonDocument(JsonDocument): 46 | # OnetoOneField 47 | vendor: VendorEmbeddedJsonDocument 48 | # ForeignKey field 49 | category: Optional[CategoryEmbeddedJsonDocument] 50 | # ManyToManyField 51 | tags: List[TagEmbeddedJsonDocument] 52 | 53 | class Django: 54 | model = Product 55 | fields = ["name", "description", "price", "created_at"] 56 | related_models = { 57 | Vendor: { 58 | "related_name": "product", 59 | "many": False, 60 | }, 61 | Category: { 62 | "related_name": "product_set", 63 | "many": True, 64 | }, 65 | Tag: { 66 | "related_name": "product_set", 67 | "many": True, 68 | }, 69 | } 70 | 71 | class ProductLightJsonDocument(JsonDocument): 72 | category: Optional[CategoryEmbeddedJsonDocument] 73 | 74 | class Django: 75 | model = Product 76 | fields = ["name", "description", "price", "created_at"] 77 | related_models = { 78 | Category: { 79 | "related_name": "product_set", 80 | "many": True, 81 | }, 82 | } 83 | 84 | registry = DocumentRegistry() 85 | registry.register(ProductJsonDocument) 86 | registry.register(ProductLightJsonDocument) 87 | registry.register(CategoryJsonDocument) 88 | 89 | assert registry.django_model_map == { 90 | Product: {ProductJsonDocument, ProductLightJsonDocument}, 91 | Category: {CategoryJsonDocument}, 92 | } 93 | assert registry.related_django_model_map == { 94 | Category: {ProductJsonDocument, ProductLightJsonDocument}, 95 | Vendor: {ProductJsonDocument}, 96 | Tag: {ProductJsonDocument}, 97 | } 98 | 99 | 100 | @mock.patch("redis_search_django.documents.JsonDocument.update_from_model_instance") 101 | def test_update_document(update_from_model_instance, document_class): 102 | CategoryJsonDocument = document_class(JsonDocument, Category, ["name"]) 103 | registry = DocumentRegistry() 104 | registry.register(CategoryJsonDocument) 105 | 106 | model_obj1 = Category(name="test") 107 | # Tag is not registered, so it should not be updated. 108 | model_obj2 = Tag(name="test") 109 | 110 | registry.update_document(model_obj1) 111 | registry.update_document(model_obj2) 112 | 113 | update_from_model_instance.assert_called_once_with(model_obj1, create=True) 114 | 115 | 116 | @mock.patch("redis_search_django.documents.JsonDocument.update_from_model_instance") 117 | def test_update_document_with_auto_index_disabled( 118 | update_from_model_instance, document_class 119 | ): 120 | CategoryJsonDocument = document_class( 121 | JsonDocument, Category, ["name"], enable_auto_index=False 122 | ) 123 | registry = DocumentRegistry() 124 | registry.register(CategoryJsonDocument) 125 | model_obj1 = Category(name="test") 126 | 127 | registry.update_document(model_obj1) 128 | update_from_model_instance.assert_not_called() 129 | 130 | 131 | @mock.patch("redis_search_django.documents.JsonDocument.update_from_model_instance") 132 | def test_update_document_with_global_auto_index_disabled( 133 | update_from_model_instance, settings, document_class 134 | ): 135 | settings.REDIS_SEARCH_AUTO_INDEX = False 136 | 137 | CategoryJsonDocument = document_class(JsonDocument, Category, ["name"]) 138 | registry = DocumentRegistry() 139 | registry.register(CategoryJsonDocument) 140 | model_obj1 = Category(name="test") 141 | 142 | registry.update_document(model_obj1) 143 | update_from_model_instance.assert_not_called() 144 | 145 | 146 | @mock.patch( 147 | "redis_search_django.documents.JsonDocument.update_from_related_model_instance" 148 | ) 149 | def test_update_related_documents(update_from_related_model_instance, document_class): 150 | CategoryEmbeddedJsonDocument = document_class( 151 | EmbeddedJsonDocument, Category, ["name"] 152 | ) 153 | 154 | class ProductJsonDocument(JsonDocument): 155 | category: Optional[CategoryEmbeddedJsonDocument] 156 | 157 | class Django: 158 | model = Product 159 | fields = ["name", "description", "price", "created_at"] 160 | related_models = { 161 | Category: { 162 | "related_name": "product_set", 163 | "many": True, 164 | }, 165 | } 166 | 167 | registry = DocumentRegistry() 168 | registry.register(ProductJsonDocument) 169 | 170 | model_obj1 = Category(name="test") 171 | # Tag and Vendor are not registered, so it should not be updated. 172 | model_obj2 = Tag(name="test") 173 | model_obj3 = Vendor(name="test", establishment_date=datetime.date.today()) 174 | 175 | registry.update_related_documents(model_obj1) 176 | registry.update_related_documents(model_obj2) 177 | registry.update_related_documents(model_obj3) 178 | 179 | update_from_related_model_instance.assert_called_once_with(model_obj1, exclude=None) 180 | 181 | 182 | @mock.patch( 183 | "redis_search_django.documents.JsonDocument.update_from_related_model_instance" 184 | ) 185 | def test_update_related_documents_with_auto_index_disabled( 186 | update_from_related_model_instance, document_class 187 | ): 188 | CategoryEmbeddedJsonDocument = document_class( 189 | EmbeddedJsonDocument, Category, ["name"], enable_auto_index=False 190 | ) 191 | 192 | class ProductJsonDocument(JsonDocument): 193 | category: Optional[CategoryEmbeddedJsonDocument] 194 | 195 | class Django: 196 | model = Product 197 | fields = ["name", "description", "price", "created_at"] 198 | related_models = { 199 | Category: { 200 | "related_name": "product_set", 201 | "many": True, 202 | }, 203 | } 204 | auto_index = False 205 | 206 | registry = DocumentRegistry() 207 | registry.register(ProductJsonDocument) 208 | 209 | model_obj1 = Category(name="test") 210 | 211 | registry.update_related_documents(model_obj1) 212 | 213 | update_from_related_model_instance.assert_not_called() 214 | 215 | 216 | @mock.patch( 217 | "redis_search_django.documents.JsonDocument.update_from_related_model_instance" 218 | ) 219 | def test_update_related_documents_with_global_auto_index_disabled( 220 | update_from_related_model_instance, settings, document_class 221 | ): 222 | settings.REDIS_SEARCH_AUTO_INDEX = False 223 | CategoryEmbeddedJsonDocument = document_class( 224 | EmbeddedJsonDocument, Category, ["name"] 225 | ) 226 | 227 | class ProductJsonDocument(JsonDocument): 228 | category: Optional[CategoryEmbeddedJsonDocument] 229 | 230 | class Django: 231 | model = Product 232 | fields = ["name", "description", "price", "created_at"] 233 | related_models = { 234 | Category: { 235 | "related_name": "product_set", 236 | "many": True, 237 | }, 238 | } 239 | auto_index = True 240 | 241 | registry = DocumentRegistry() 242 | registry.register(ProductJsonDocument) 243 | 244 | model_obj1 = Category(name="test") 245 | 246 | registry.update_related_documents(model_obj1) 247 | 248 | update_from_related_model_instance.assert_not_called() 249 | 250 | 251 | @mock.patch("redis_search_django.documents.JsonDocument.delete") 252 | def test_remove_document(delete, document_class): 253 | CategoryJsonDocument = document_class(JsonDocument, Category, ["name"]) 254 | registry = DocumentRegistry() 255 | registry.register(CategoryJsonDocument) 256 | 257 | model_obj1 = Category(name="test") 258 | # Tag is not registered, so it should not be deleted. 259 | model_obj2 = Tag(name="test") 260 | 261 | registry.remove_document(model_obj1) 262 | registry.remove_document(model_obj2) 263 | 264 | delete.assert_called_once_with(model_obj1.pk) 265 | 266 | 267 | @mock.patch("redis_search_django.documents.JsonDocument.delete") 268 | def test_remove_document_with_auto_index_disabled(delete, document_class): 269 | CategoryJsonDocument = document_class( 270 | JsonDocument, Category, ["name"], enable_auto_index=False 271 | ) 272 | registry = DocumentRegistry() 273 | registry.register(CategoryJsonDocument) 274 | 275 | model_obj1 = Category(name="test") 276 | 277 | registry.remove_document(model_obj1) 278 | 279 | delete.assert_not_called() 280 | 281 | 282 | @mock.patch("redis_search_django.documents.JsonDocument.delete") 283 | def test_remove_document_with_global_auto_index_disabled( 284 | delete, settings, document_class 285 | ): 286 | settings.REDIS_SEARCH_AUTO_INDEX = False 287 | 288 | CategoryJsonDocument = document_class(JsonDocument, Category, ["name"]) 289 | registry = DocumentRegistry() 290 | registry.register(CategoryJsonDocument) 291 | 292 | model_obj1 = Category(name="test") 293 | 294 | registry.remove_document(model_obj1) 295 | 296 | delete.assert_not_called() 297 | 298 | 299 | @mock.patch("redis_search_django.documents.Document.index_all") 300 | def test_index_documents(index_all, document_class): 301 | registry = DocumentRegistry() 302 | VendorDocumentClass = document_class(JsonDocument, Vendor, ["name"]) 303 | CategoryDocumentClass = document_class(HashDocument, Category, ["name"]) 304 | registry.register(VendorDocumentClass) 305 | registry.register(CategoryDocumentClass) 306 | 307 | registry.index_documents() 308 | 309 | assert index_all.call_count == 2 310 | 311 | 312 | @mock.patch("redis_search_django.documents.Document.index_all") 313 | def test_index_documents_with_specific_model(index_all, document_class): 314 | registry = DocumentRegistry() 315 | VendorDocumentClass = document_class(JsonDocument, Vendor, ["name"]) 316 | CategoryDocumentClass = document_class(HashDocument, Category, ["name"]) 317 | registry.register(VendorDocumentClass) 318 | registry.register(CategoryDocumentClass) 319 | 320 | registry.index_documents(["tests.Category"]) 321 | 322 | index_all.assert_called_once() 323 | -------------------------------------------------------------------------------- /redis_search_django/documents.py: -------------------------------------------------------------------------------- 1 | import operator 2 | from abc import ABC 3 | from dataclasses import dataclass 4 | from functools import reduce 5 | from typing import Any, Dict, List, Type, Union 6 | 7 | from django.core.exceptions import ImproperlyConfigured 8 | from django.db import models 9 | from pydantic.fields import ModelField 10 | from redis.commands.search.aggregation import AggregateRequest 11 | from redis_om import Field, HashModel, JsonModel 12 | from redis_om.model.model import ( 13 | EmbeddedJsonModel, 14 | Expression, 15 | NotFoundError, 16 | RedisModel, 17 | ) 18 | 19 | from .config import model_field_class_config 20 | from .query import RediSearchQuery 21 | from .registry import document_registry 22 | 23 | 24 | @dataclass 25 | class DjangoOptions: 26 | """Settings for a Django model.""" 27 | 28 | model: models.Model 29 | fields: List[str] 30 | select_related_fields: List[str] 31 | prefetch_related_fields: List[str] 32 | related_models: Dict[Type[models.Model], Dict[str, Union[str, bool]]] 33 | auto_index: bool 34 | 35 | def __init__(self, options: Any = None) -> None: 36 | self.model = getattr(options, "model", None) 37 | 38 | if not self.model: 39 | raise ImproperlyConfigured("Django options requires field `model`.") 40 | 41 | # If the attributes do not exist or explicitly set to None, 42 | # use the default value 43 | self.fields = getattr(options, "fields", None) or [] 44 | self.select_related_fields = ( 45 | getattr(options, "select_related_fields", None) or [] 46 | ) 47 | self.prefetch_related_fields = ( 48 | getattr(options, "prefetch_related_fields", None) or [] 49 | ) 50 | self.related_models = getattr(options, "related_models", None) or {} 51 | self.auto_index = getattr(options, "auto_index", True) 52 | 53 | 54 | class Document(RedisModel, ABC): 55 | """Base class for all documents.""" 56 | 57 | _query_class = RediSearchQuery 58 | 59 | class Meta: 60 | global_key_prefix = "redis_search" 61 | 62 | @classmethod 63 | def find( 64 | cls, *expressions: Union[Any, Expression], **kwargs: Any 65 | ) -> RediSearchQuery: 66 | """Query Redis Search Index for documents.""" 67 | return cls._query_class( 68 | expressions=expressions, django_model=cls._django.model, model=cls, **kwargs 69 | ) 70 | 71 | @classmethod 72 | def build_aggregate_request( 73 | cls, *expressions: Union[Any, Expression] 74 | ) -> AggregateRequest: 75 | """Build an aggregation request using the given expressions""" 76 | search_query_expression = ( 77 | cls._query_class.resolve_redisearch_query( 78 | reduce(operator.and_, expressions) 79 | ) 80 | if expressions 81 | else "*" 82 | ) 83 | return AggregateRequest(search_query_expression) 84 | 85 | @classmethod 86 | def aggregate(cls, aggregate_request: AggregateRequest) -> List[Dict[str, Any]]: 87 | """Aggregate data and return a list of dictionaries containing the results""" 88 | results = cls.db().ft(cls._meta.index_name).aggregate(aggregate_request) 89 | 90 | return [ 91 | { 92 | decode_string(result[i]): decode_string(result[i + 1]) 93 | for i in range(0, len(result), 2) 94 | } 95 | for result in results.rows 96 | ] 97 | 98 | @classmethod 99 | def data_from_model_instance( 100 | cls, instance: models.Model, exclude_obj: Union[models.Model, None] = None 101 | ) -> Dict[str, Any]: 102 | """Build a document data dictionary from a Django Model instance""" 103 | data = {} 104 | 105 | for field_name, field in cls.__fields__.items(): 106 | field_type = field.type_ 107 | # Check if Document is embedded in another Document 108 | is_embedded = issubclass(field_type, EmbeddedJsonDocument) 109 | # Get Target Field Value 110 | target = getattr(instance, field_name, None) 111 | # If the target field is a ManyToManyField, 112 | # get the related objects and build data using the objects 113 | if ( 114 | is_embedded 115 | and target 116 | and hasattr(target, "all") 117 | and callable(target.all) 118 | ): 119 | data[field_name] = [ 120 | field_type.data_from_model_instance(obj, exclude_obj=exclude_obj) 121 | for obj in target.all() 122 | if obj != exclude_obj 123 | ] 124 | # If the target field is a ForeignKey or OneToOneField, 125 | # get the related object to build data 126 | elif is_embedded and target: 127 | if target != exclude_obj: 128 | data[field_name] = field_type.data_from_model_instance( 129 | target, exclude_obj=exclude_obj 130 | ) 131 | else: 132 | # Check if the Document class has a `prepare_{field_name}` class method 133 | # and use it to prepare the field value 134 | prepare_func = getattr(cls, f"prepare_{field_name}", None) 135 | value = prepare_func(instance) if prepare_func else target 136 | 137 | # If the field is required and the value is None, raise an error 138 | if value is None and field.required: 139 | raise ValueError( 140 | f"Field '{field_name}' is required, either use a Django " 141 | f"model field or define 'prepare_{field_name}' class " 142 | f"method on the '{cls.__name__}' class " 143 | f"that returns a value of type {field_type}" 144 | ) 145 | 146 | if field_name == "pk": 147 | value = str(value) 148 | 149 | # This is added to convert models.BooleanField value to int 150 | # as redis-om creates schema for the field as NUMERIC field. 151 | # see https://github.com/redis/redis-om-python/issues/193 152 | data[field_name] = int(value) if isinstance(value, bool) else value 153 | return data 154 | 155 | @classmethod 156 | def from_data(cls, data: Dict[str, Any]) -> RedisModel: 157 | """Build a document from a data dictionary""" 158 | return cls(**data) 159 | 160 | @classmethod 161 | def from_model_instance( 162 | cls, 163 | instance: models.Model, 164 | exclude_obj: Union[models.Model, None] = None, 165 | save: bool = True, 166 | ) -> RedisModel: 167 | """Build a document from a Django Model instance""" 168 | assert instance._meta.model == cls._django.model 169 | 170 | obj = cls.from_data( 171 | cls.data_from_model_instance(instance, exclude_obj=exclude_obj) 172 | ) 173 | 174 | if save: 175 | obj.save() 176 | 177 | return obj 178 | 179 | @classmethod 180 | def update_from_model_instance( 181 | cls, instance: models.Model, create: bool = True 182 | ) -> RedisModel: 183 | """Update a document from a Django Model instance""" 184 | try: 185 | obj = cls.get(instance.pk) 186 | obj.update(**cls.data_from_model_instance(instance)) 187 | except NotFoundError: 188 | if not create: 189 | raise 190 | # Create the Document if not found. 191 | obj = cls.from_model_instance(instance, save=True) 192 | return obj 193 | 194 | @classmethod 195 | def update_from_related_model_instance( 196 | cls, instance: models.Model, exclude: models.Model = None 197 | ) -> None: 198 | """Update a document from a Django Related Model instance""" 199 | related_model = instance.__class__ 200 | related_model_config = cls._django.related_models.get(related_model) 201 | 202 | # If the related model is not configured, return 203 | if not related_model_config: 204 | return 205 | 206 | related_name = related_model_config["related_name"] 207 | attribute = getattr(instance, related_name, None) 208 | 209 | # If the related name attribute is not found, return 210 | if not attribute: 211 | return 212 | 213 | if related_model_config["many"]: 214 | cls.index_queryset(attribute.all(), exclude_obj=exclude) 215 | else: 216 | # If the related model instance will delete 217 | # the document's Django model instance 218 | # Then we need to skip re-indexing the document 219 | if ( 220 | exclude 221 | and related_model._meta.get_field(related_name).on_delete 222 | == models.CASCADE 223 | ): 224 | exclude = None 225 | cls.from_model_instance(attribute, exclude_obj=exclude, save=True) 226 | 227 | @classmethod 228 | def index_queryset( 229 | cls, 230 | queryset: models.QuerySet, 231 | exclude_obj: Union[models.Model, None] = None, 232 | ) -> None: 233 | """Index all items in the Django model queryset""" 234 | obj_list = [] 235 | 236 | for instance in queryset.iterator(chunk_size=2000): 237 | obj_list.append( 238 | cls.from_model_instance(instance, exclude_obj=exclude_obj, save=False) 239 | ) 240 | 241 | cls.add(obj_list) 242 | 243 | @classmethod 244 | def index_all(cls) -> None: 245 | """Index all instances of the model""" 246 | cls.index_queryset(cls.get_queryset()) 247 | 248 | @classmethod 249 | def get_queryset(cls) -> models.QuerySet: 250 | """Get Django model queryset, can be overridden to filter queryset""" 251 | queryset = cls._django.model._default_manager.all() 252 | 253 | if cls._django.select_related_fields: 254 | queryset = queryset.select_related(*cls._django.select_related_fields) 255 | 256 | if cls._django.prefetch_related_fields: 257 | queryset = queryset.prefetch_related(*cls._django.prefetch_related_fields) 258 | return queryset 259 | 260 | @classmethod 261 | def add_django_fields(cls, field_names: List[str]) -> None: 262 | """Dynamically add fields to the document""" 263 | fields: Dict[str, ModelField] = {} 264 | type_annotations: Dict[str, type] = {} 265 | is_embedded = issubclass(cls, EmbeddedJsonDocument) 266 | 267 | for field_name in field_names: 268 | if field_name in cls.__fields__ or field_name in ["id", "pk"]: 269 | continue 270 | 271 | field_type = cls._django.model._meta.get_field(field_name) 272 | 273 | field_config = model_field_class_config.get(field_type.__class__) 274 | 275 | if not field_config: 276 | raise ImproperlyConfigured( 277 | f"Either the field '{field_type}' is not a Django model field or " 278 | "is a Related Model Field (OneToOneField, ForeignKey, ManyToMany) " 279 | f"which needs to be explicitly added to the '{cls.__name__}' " 280 | f"document class using 'EmbeddedJsonDocument'" 281 | ) 282 | 283 | field_config = field_config.copy() 284 | annotation = field_config.pop("type") 285 | 286 | if is_embedded: 287 | field_config["full_text_search"] = False 288 | 289 | if annotation == str: 290 | field_config["sortable"] = False 291 | 292 | required = not (field_type.null or field_type.blank) 293 | field_info = Field(**field_config) 294 | type_annotations[field_name] = annotation 295 | 296 | # Create a new Field object with the field_info dict 297 | fields[field_name] = ModelField( 298 | name=field_name, 299 | class_validators={}, 300 | model_config=cls.__config__, 301 | type_=annotation, 302 | required=required, 303 | field_info=field_info, 304 | ) 305 | 306 | cls.__fields__.update(fields) 307 | cls.__annotations__.update(type_annotations) 308 | 309 | @property 310 | def id(self) -> Union[int, str]: 311 | """Alias for the primary key of the document""" 312 | return self.pk 313 | 314 | 315 | def decode_string(value: Union[str, bytes]) -> str: 316 | """Decode a string from bytes to str""" 317 | 318 | if isinstance(value, bytes): 319 | return value.decode("utf-8") 320 | return value 321 | 322 | 323 | class JsonDocument(Document, JsonModel, ABC): 324 | """A Document that uses Redis JSON storage""" 325 | 326 | def __init_subclass__(cls, **kwargs: Any) -> None: 327 | """Initialize the Subclass class with proper Django options and fields""" 328 | cls._django = DjangoOptions(getattr(cls, "Django", None)) 329 | 330 | if cls._django.fields: 331 | cls.add_django_fields(cls._django.fields) 332 | 333 | super().__init_subclass__(**kwargs) 334 | document_registry.register(cls) 335 | 336 | 337 | class EmbeddedJsonDocument(Document, EmbeddedJsonModel, ABC): 338 | """An Embedded Document that uses Redis JSON storage""" 339 | 340 | def __init_subclass__(cls, **kwargs: Any) -> None: 341 | """Initialize the Subclass class with proper Django options and fields""" 342 | cls._django = DjangoOptions(getattr(cls, "Django", None)) 343 | 344 | if cls._django.fields: 345 | cls.add_django_fields(cls._django.fields) 346 | 347 | super().__init_subclass__(**kwargs) 348 | document_registry.register(cls) 349 | 350 | 351 | class HashDocument(Document, HashModel, ABC): 352 | """A Document that uses Redis Hash storage""" 353 | 354 | def __init_subclass__(cls, **kwargs: Any) -> None: 355 | """Initialize the Subclass class with proper Django options and fields""" 356 | cls._django = DjangoOptions(getattr(cls, "Django", None)) 357 | 358 | if cls._django.fields: 359 | cls.add_django_fields(cls._django.fields) 360 | 361 | super().__init_subclass__(**kwargs) 362 | document_registry.register(cls) 363 | -------------------------------------------------------------------------------- /tests/test_documents.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | from unittest import mock 3 | 4 | import pytest 5 | from django.core.exceptions import ImproperlyConfigured 6 | from redis.commands.search import reducers 7 | from redis_om import Migrator, NotFoundError 8 | 9 | from redis_search_django.documents import ( 10 | DjangoOptions, 11 | EmbeddedJsonDocument, 12 | HashDocument, 13 | JsonDocument, 14 | ) 15 | 16 | from .helpers import is_redis_running 17 | from .models import Category, Product, Tag, Vendor 18 | 19 | 20 | def test_django_options(): 21 | class Django: 22 | model = Product 23 | fields = ["name", "description", "price", "created_at"] 24 | select_related_fields = ["vendor", "category"] 25 | prefetch_related_fields = ["tags"] 26 | auto_index = True 27 | related_models = { 28 | Vendor: { 29 | "related_name": "product", 30 | "many": False, 31 | }, 32 | Category: { 33 | "related_name": "product_set", 34 | "many": True, 35 | }, 36 | Tag: { 37 | "related_name": "product_set", 38 | "many": True, 39 | }, 40 | } 41 | 42 | options = DjangoOptions(Django) 43 | 44 | assert options.model == Product 45 | assert options.auto_index is True 46 | assert options.fields == ["name", "description", "price", "created_at"] 47 | assert options.select_related_fields == ["vendor", "category"] 48 | assert options.prefetch_related_fields == ["tags"] 49 | assert options.related_models == { 50 | Vendor: { 51 | "related_name": "product", 52 | "many": False, 53 | }, 54 | Category: { 55 | "related_name": "product_set", 56 | "many": True, 57 | }, 58 | Tag: { 59 | "related_name": "product_set", 60 | "many": True, 61 | }, 62 | } 63 | 64 | 65 | def test_django_options_with_empty_optional_value(): 66 | class Django: 67 | model = Product 68 | fields = None 69 | auto_index = False 70 | related_models = None 71 | select_related_fields = None 72 | prefetch_related_fields = None 73 | 74 | options = DjangoOptions(Django) 75 | 76 | assert options.model == Product 77 | assert options.auto_index is False 78 | assert options.fields == [] 79 | assert options.select_related_fields == [] 80 | assert options.prefetch_related_fields == [] 81 | assert options.related_models == {} 82 | 83 | 84 | def test_django_options_with_empty_required_value(): 85 | class Django: 86 | model = None 87 | 88 | with pytest.raises(ImproperlyConfigured): 89 | DjangoOptions(Django) 90 | 91 | 92 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 93 | @pytest.mark.django_db 94 | def test_document_id(document_class, category_obj): 95 | CategoryJsonDocument = document_class(JsonDocument, Category, ["name"]) 96 | 97 | document = CategoryJsonDocument.from_model_instance(category_obj, save=False) 98 | 99 | assert document.id == str(category_obj.id) 100 | assert document.id == document.pk 101 | 102 | 103 | def test_json_document_with_django_fields_including_related_field(document_class): 104 | with pytest.raises(ImproperlyConfigured): 105 | document_class(JsonDocument, Product, ["name", "vendor"]) 106 | 107 | 108 | def test_json_document_with_django_fields(document_class): 109 | CategoryJsonDocument = document_class(JsonDocument, Category, ["name"]) 110 | 111 | assert list(CategoryJsonDocument.__fields__.keys()) == ["pk", "name"] 112 | 113 | 114 | def test_json_document_with_django_fields_including_id(document_class): 115 | CategoryJsonDocument = document_class(JsonDocument, Category, ["name", "id"]) 116 | 117 | assert list(CategoryJsonDocument.__fields__.keys()) == ["pk", "name"] 118 | 119 | 120 | def test_json_document_without_django_fields(document_class): 121 | CategoryJsonDocument = document_class(JsonDocument, Category, []) 122 | 123 | assert list(CategoryJsonDocument.__fields__.keys()) == ["pk"] 124 | 125 | 126 | def test_embedded_json_document_with_django_fields(document_class): 127 | CategoryJsonDocument = document_class(EmbeddedJsonDocument, Category, ["name"]) 128 | 129 | assert list(CategoryJsonDocument.__fields__.keys()) == ["pk", "name"] 130 | 131 | 132 | def test_embedded_json_document_without_django_fields(document_class): 133 | CategoryJsonDocument = document_class(EmbeddedJsonDocument, Category, []) 134 | 135 | assert list(CategoryJsonDocument.__fields__.keys()) == ["pk"] 136 | 137 | 138 | def test_hash_document_with_django_fields(document_class): 139 | CategoryJsonDocument = document_class(HashDocument, Category, ["name"]) 140 | 141 | assert list(CategoryJsonDocument.__fields__.keys()) == ["pk", "name"] 142 | 143 | 144 | def test_hash_document_without_django_fields(document_class): 145 | CategoryJsonDocument = document_class(HashDocument, Category, []) 146 | 147 | assert list(CategoryJsonDocument.__fields__.keys()) == ["pk"] 148 | 149 | 150 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 151 | @pytest.mark.django_db 152 | def test_get_queryset(document_class, product_obj): 153 | ProductJsonDocument = document_class(JsonDocument, Product, ["name"]) 154 | assert ProductJsonDocument.get_queryset().count() == 1 155 | 156 | 157 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 158 | @pytest.mark.django_db 159 | def test_index_all(nested_document_class): 160 | ProductDocumentCalss = nested_document_class[0] 161 | vendor = Vendor.objects.create( 162 | name="test", establishment_date=datetime.date.today() 163 | ) 164 | product = Product.objects.create( 165 | name="Test", 166 | price=10.0, 167 | vendor=vendor, 168 | category=Category.objects.create(name="test"), 169 | ) 170 | 171 | tag = Tag.objects.create(name="test") 172 | tag2 = Tag.objects.create(name="test2") 173 | product.tags.set([tag, tag2]) 174 | 175 | ProductDocumentCalss.delete(pk=product.pk) 176 | 177 | with pytest.raises(NotFoundError): 178 | ProductDocumentCalss.get(pk=product.pk) 179 | 180 | ProductDocumentCalss.index_all() 181 | 182 | assert ProductDocumentCalss.get(pk=product.pk).pk == str(product.pk) 183 | 184 | 185 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 186 | @pytest.mark.django_db 187 | def test_index_queryset(nested_document_class): 188 | ProductDocumentCalss = nested_document_class[0] 189 | vendor = Vendor.objects.create( 190 | name="test", establishment_date=datetime.date.today() 191 | ) 192 | product = Product.objects.create( 193 | name="Test", 194 | price=10.0, 195 | vendor=vendor, 196 | category=Category.objects.create(name="test"), 197 | ) 198 | 199 | tag = Tag.objects.create(name="test") 200 | tag2 = Tag.objects.create(name="test2") 201 | product.tags.set([tag, tag2]) 202 | 203 | ProductDocumentCalss.delete(pk=product.pk) 204 | 205 | with pytest.raises(NotFoundError): 206 | ProductDocumentCalss.get(pk=product.pk) 207 | 208 | ProductDocumentCalss.index_queryset(Product.objects.all()) 209 | 210 | assert ProductDocumentCalss.get(pk=product.pk).pk == str(product.pk) 211 | 212 | 213 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 214 | @pytest.mark.django_db 215 | def test_update_from_model_instance(document_class, category_obj): 216 | CategoryDocumentClass = document_class(JsonDocument, Category, ["name"]) 217 | 218 | with pytest.raises(NotFoundError): 219 | CategoryDocumentClass.get(pk=category_obj.pk) 220 | 221 | CategoryDocumentClass.update_from_model_instance(category_obj, create=True) 222 | 223 | assert CategoryDocumentClass.get(pk=category_obj.pk) is not None 224 | 225 | 226 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 227 | @pytest.mark.django_db 228 | def test_update_from_model_instance_no_create(document_class, category_obj): 229 | CategoryDocumentClass = document_class(JsonDocument, Category, ["name"]) 230 | 231 | with pytest.raises(NotFoundError): 232 | CategoryDocumentClass.get(pk=category_obj.pk) 233 | 234 | with pytest.raises(NotFoundError): 235 | CategoryDocumentClass.update_from_model_instance(category_obj, create=False) 236 | 237 | 238 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 239 | def test_from_data(document_class): 240 | CategoryDocumentClass = document_class(JsonDocument, Category, ["name"]) 241 | 242 | data = {"name": "test"} 243 | document = CategoryDocumentClass.from_data(data) 244 | 245 | assert document.name == "test" 246 | 247 | 248 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 249 | @pytest.mark.django_db 250 | def test_from_model_instance(document_class, category_obj): 251 | CategoryDocumentClass = document_class(JsonDocument, Category, ["name"]) 252 | 253 | with pytest.raises(NotFoundError): 254 | CategoryDocumentClass.get(pk=category_obj.pk) 255 | 256 | CategoryDocumentClass.from_model_instance(category_obj) 257 | 258 | assert CategoryDocumentClass.get(pk=category_obj.pk) is not None 259 | 260 | 261 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 262 | @pytest.mark.django_db 263 | def test_update_from_related_model_instance(nested_document_class, product_with_tag): 264 | ProductDocumentCalss = nested_document_class[0] 265 | product = Product.objects.create( 266 | name="Test", 267 | price=10.0, 268 | vendor=Vendor.objects.create( 269 | name="test", establishment_date=datetime.date.today() 270 | ), 271 | ) 272 | 273 | tag = Tag.objects.create(name="test") 274 | product.tags.add(tag) 275 | 276 | with mock.patch( 277 | "redis_search_django.registry.DocumentRegistry.update_related_documents" 278 | ): 279 | tag.name = "test2" 280 | tag.save() 281 | 282 | assert ProductDocumentCalss.get(pk=product.pk).tags[0].name == "test" 283 | 284 | ProductDocumentCalss.update_from_related_model_instance(tag) 285 | 286 | assert ProductDocumentCalss.get(pk=product.pk).tags[0].name == "test2" 287 | 288 | 289 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 290 | @pytest.mark.django_db 291 | def test_update_from_related_model_instance_one_to_one( 292 | nested_document_class, product_with_tag 293 | ): 294 | ProductDocumentCalss = nested_document_class[0] 295 | vendor = Vendor.objects.create( 296 | name="test", establishment_date=datetime.date.today() 297 | ) 298 | product = Product.objects.create(name="Test", price=10.0, vendor=vendor) 299 | 300 | with mock.patch( 301 | "redis_search_django.registry.DocumentRegistry.update_related_documents" 302 | ): 303 | vendor.name = "test2" 304 | vendor.save() 305 | 306 | assert ProductDocumentCalss.get(pk=product.pk).vendor.name == "test" 307 | 308 | ProductDocumentCalss.update_from_related_model_instance(vendor) 309 | 310 | assert ProductDocumentCalss.get(pk=product.pk).vendor.name == "test2" 311 | 312 | 313 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 314 | @pytest.mark.django_db 315 | def test_update_from_related_model_instance_exclude_required_field( 316 | nested_document_class, product_with_tag 317 | ): 318 | ProductDocumentCalss = nested_document_class[0] 319 | vendor = Vendor.objects.create( 320 | name="test", establishment_date=datetime.date.today() 321 | ) 322 | product = Product.objects.create(name="Test", price=10.0, vendor=vendor) 323 | 324 | assert ProductDocumentCalss.get(pk=product.pk).vendor.name == "test" 325 | 326 | ProductDocumentCalss.update_from_related_model_instance(vendor, exclude=vendor) 327 | 328 | assert ProductDocumentCalss.get(pk=product.pk).vendor.name == "test" 329 | 330 | 331 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 332 | @pytest.mark.django_db 333 | def test_update_from_related_model_instance_with_no_related_model_config( 334 | document_class, product_with_tag 335 | ): 336 | ProductDocumentCalss = document_class(JsonDocument, Product, ["name"]) 337 | product = Product.objects.create( 338 | name="Test", 339 | price=10.0, 340 | vendor=Vendor.objects.create( 341 | name="test", establishment_date=datetime.date.today() 342 | ), 343 | ) 344 | 345 | tag = Tag.objects.create(name="test") 346 | product.tags.add(tag) 347 | 348 | with mock.patch( 349 | "redis_search_django.registry.DocumentRegistry.update_related_documents" 350 | ): 351 | tag.name = "test2" 352 | tag.save() 353 | 354 | assert not hasattr(ProductDocumentCalss.get(pk=product.pk), "tags") 355 | 356 | ProductDocumentCalss.update_from_related_model_instance(tag) 357 | 358 | assert not hasattr(ProductDocumentCalss.get(pk=product.pk), "tags") 359 | 360 | 361 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 362 | @pytest.mark.django_db 363 | def test_update_from_related_model_instance_with_exclude_optional_field( 364 | nested_document_class, product_with_tag 365 | ): 366 | ProductDocumentCalss = nested_document_class[0] 367 | product = Product.objects.create( 368 | name="Test", 369 | price=10.0, 370 | vendor=Vendor.objects.create( 371 | name="test", establishment_date=datetime.date.today() 372 | ), 373 | ) 374 | 375 | tag = Tag.objects.create(name="test") 376 | product.tags.add(tag) 377 | 378 | assert ProductDocumentCalss.get(pk=product.pk).tags[0].name == "test" 379 | 380 | ProductDocumentCalss.update_from_related_model_instance(tag, exclude=tag) 381 | 382 | assert ProductDocumentCalss.get(pk=product.pk).tags == [] 383 | 384 | 385 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 386 | @pytest.mark.django_db 387 | def test_data_from_model_instance(nested_document_class, product_with_tag): 388 | ProductDocumentCalss = nested_document_class[0] 389 | vendor = Vendor.objects.create(name="test", establishment_date="2022-08-18") 390 | category = Category.objects.create(name="test") 391 | product = Product.objects.create( 392 | name="Test", price=10.0, vendor=vendor, category=category 393 | ) 394 | 395 | tag = Tag.objects.create(name="test") 396 | tag2 = Tag.objects.create(name="test2") 397 | product.tags.set([tag, tag2]) 398 | 399 | data = ProductDocumentCalss.data_from_model_instance(product) 400 | 401 | assert data == { 402 | "pk": str(product.pk), 403 | "vendor": { 404 | "pk": str(vendor.pk), 405 | "name": vendor.name, 406 | "establishment_date": vendor.establishment_date, 407 | }, 408 | "category": {"pk": str(category.pk), "name": category.name}, 409 | "tags": [ 410 | {"pk": str(tag.pk), "name": tag.name}, 411 | {"pk": str(tag2.pk), "name": tag2.name}, 412 | ], 413 | "name": product.name, 414 | "description": product.description, 415 | "price": product.price, 416 | "created_at": product.created_at, 417 | } 418 | 419 | 420 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 421 | @pytest.mark.django_db 422 | def test_data_from_model_instance_exclude_obj(nested_document_class, product_with_tag): 423 | ProductDocumentCalss = nested_document_class[0] 424 | vendor = Vendor.objects.create(name="test", establishment_date="2022-08-18") 425 | category = Category.objects.create(name="test") 426 | product = Product.objects.create( 427 | name="Test", price=10.0, vendor=vendor, category=category 428 | ) 429 | 430 | tag = Tag.objects.create(name="test") 431 | tag2 = Tag.objects.create(name="test2") 432 | product.tags.set([tag, tag2]) 433 | 434 | data = ProductDocumentCalss.data_from_model_instance(product, category) 435 | print(data) 436 | 437 | assert data == { 438 | "pk": str(product.pk), 439 | "vendor": { 440 | "pk": str(vendor.pk), 441 | "name": vendor.name, 442 | "establishment_date": vendor.establishment_date, 443 | }, 444 | "tags": [ 445 | {"pk": str(tag.pk), "name": tag.name}, 446 | {"pk": str(tag2.pk), "name": tag2.name}, 447 | ], 448 | "name": product.name, 449 | "description": product.description, 450 | "price": product.price, 451 | "created_at": product.created_at, 452 | } 453 | 454 | 455 | def test_build_aggregate_request_without_expressions(document_class): 456 | CategoryDocumentCalss = document_class(JsonDocument, Category, ["name"]) 457 | request = CategoryDocumentCalss.build_aggregate_request() 458 | 459 | assert request._query == "*" 460 | 461 | 462 | def test_build_aggregate_request_with_expressions(document_class): 463 | CategoryDocumentCalss = document_class(JsonDocument, Category, ["name"]) 464 | request = CategoryDocumentCalss.build_aggregate_request( 465 | CategoryDocumentCalss.name == "test" 466 | ) 467 | assert request._query == "@name:{test}" 468 | 469 | 470 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 471 | @pytest.mark.django_db 472 | def test_aggregate(document_class): 473 | CategoryDocumentClass = document_class(JsonDocument, Category, ["name"]) 474 | 475 | Migrator().run() 476 | 477 | Category.objects.create(name="test") 478 | Category.objects.create(name="test2") 479 | 480 | request = CategoryDocumentClass.build_aggregate_request() 481 | 482 | result = CategoryDocumentClass.aggregate( 483 | request.group_by( 484 | ["@pk"], 485 | reducers.count().alias("count"), 486 | ) 487 | ) 488 | 489 | assert len(result) == 2 490 | assert result[0]["count"] == "1" 491 | 492 | 493 | @pytest.mark.skipif(not is_redis_running(), reason="Redis is not running") 494 | @pytest.mark.django_db 495 | def test_find(document_class): 496 | CategoryDocumentClass = document_class(JsonDocument, Category, ["name"]) 497 | 498 | Migrator().run() 499 | 500 | category_1 = Category.objects.create(name="shoes") 501 | Category.objects.create(name="toys") 502 | 503 | result = CategoryDocumentClass.find(CategoryDocumentClass.name % "shoes").execute() 504 | 505 | assert len(result) == 1 506 | assert result[0].pk == str(category_1.pk) 507 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # redis-search-django 2 | 3 | [![Pypi Version](https://img.shields.io/pypi/v/redis-search-django.svg?style=flat-square)](https://pypi.org/project/redis-search-django/) 4 | [![Supported Python Versions](https://img.shields.io/pypi/pyversions/redis-search-django?style=flat-square)](https://pypi.org/project/redis-search-django/) 5 | [![Supported Django Versions](https://img.shields.io/pypi/frameworkversions/django/redis-search-django?color=darkgreen&style=flat-square)](https://pypi.org/project/redis-search-django/) 6 | [![License](https://img.shields.io/github/license/saadmk11/redis-search-django?style=flat-square)](https://github.com/saadmk11/redis-search-django/blob/main/LICENSE) 7 | 8 | ![Django Tests](https://img.shields.io/github/actions/workflow/status/saadmk11/redis-search-django/test.yml??label=Test&style=flat-square&branch=main) 9 | ![Codecov](https://img.shields.io/codecov/c/github/saadmk11/redis-search-django?style=flat-square&token=ugjHXbEKib) 10 | ![pre-commit.ci](https://img.shields.io/badge/pre--commit.ci-enabled-brightgreen?logo=pre-commit&logoColor=white&style=flat-square) 11 | ![Changelog-CI](https://img.shields.io/github/actions/workflow/status/saadmk11/redis-search-django/changelog-ci.yaml??label=Changelog-CI&style=flat-square&branch=main) 12 | ![Code Style](https://img.shields.io/badge/Code%20Style-Black-black?style=flat-square) 13 | 14 | # About 15 | 16 | A Django package that provides **auto indexing** and **searching** capabilities for Django model instances using **[RediSearch](https://redis.io/docs/stack/search/)**. 17 | 18 | # Features 19 | 20 | - Management Command to create, update and populate the RediSearch Index. 21 | - Auto Index on Model object Create, Update and Delete. 22 | - Auto Index on Related Model object Add, Update, Remove and Delete. 23 | - Easy to create Document classes (Uses Django Model Form Class like structure). 24 | - Index nested models (e.g: `OneToOneField`, `ForeignKey` and `ManyToManyField`). 25 | - Search documents using `redis-om`. 26 | - Search Result Pagination. 27 | - Search Result Sorting. 28 | - RediSearch Result to Django QuerySet. 29 | - Faceted Search. 30 | 31 | # Requirements 32 | 33 | - Python: 3.7, 3.8, 3.9, 3.10 34 | - Django: 3.2, 4.0, 4.1 35 | - redis-om: >= 0.0.27 36 | 37 | # Redis 38 | 39 | ## Downloading Redis 40 | 41 | The latest version of Redis is available from [Redis.io](https://redis.io/). You can also install Redis with your operating system's package manager. 42 | 43 | ## RediSearch and RedisJSON 44 | 45 | `redis-search-django` relies on the [RediSearch](https://redis.io/docs/stack/search/) and [RedisJSON](https://redis.io/docs/stack/json/) Redis modules to support rich queries and embedded models. 46 | You need these Redis modules to use `redis-search-django`. 47 | 48 | The easiest way to run these Redis modules during local development is to use the [redis-stack](https://hub.docker.com/r/redis/redis-stack) Docker image. 49 | 50 | ## Docker Compose 51 | 52 | There is a `docker-compose.yaml` file provided in the project's root directory. 53 | This file will run Redis with RedisJSON and RediSearch modules during development. 54 | 55 | Run the following command to start the Redis container: 56 | 57 | ```bash 58 | docker compose up -d 59 | ``` 60 | 61 | # Example Project 62 | 63 | There is an example project available at [Example Project](https://github.com/saadmk11/redis-search-django/tree/main/example). 64 | 65 | 66 | # Documentation 67 | 68 | ## Installation 69 | 70 | ```bash 71 | pip install redis-search-django 72 | ``` 73 | 74 | Then add `redis_search_django` to your `INSTALLED_APPS`: 75 | 76 | ```bash 77 | INSTALLED_APPS = [ 78 | ... 79 | 'redis_search_django', 80 | ] 81 | ``` 82 | 83 | ## Usage 84 | 85 | ### Document Types 86 | 87 | There are **3 types** of documents class available: 88 | 89 | - **JsonDocument:** This uses `RedisJSON` to store the document. If you want to use Embedded Documents (Required For `OneToOneField`, `ForeignKey` and `ManyToManyField`) then use `JsonDocument`. 90 | - **EmbeddedJsonDocument:** If the document will be embedded inside another document class then use this. Embedded Json Documents are used for `OneToOneField`, `ForeignKey` and `ManyToManyField` or any types of nested documents. 91 | - **HashDocument:** This uses `RedisHash` to store the documents. It can not be used for nested documents. 92 | 93 | ### Creating Document Classes 94 | 95 | You need to inherit from The Base Document Classes mentioned above to build a document class. 96 | 97 | #### Simple Example 98 | 99 | **1. For Django Model:** 100 | 101 | ```python 102 | # models.py 103 | 104 | from django.db import models 105 | 106 | 107 | class Category(models.Model): 108 | name = models.CharField(max_length=30) 109 | slug = models.SlugField(max_length=30) 110 | 111 | def __str__(self) -> str: 112 | return self.name 113 | ``` 114 | 115 | **2. You can create a document class like this:** 116 | 117 | **Note:** Document classes must be stored in `documents.py` file. 118 | 119 | ```python 120 | # documents.py 121 | 122 | from redis_search_django.documents import JsonDocument 123 | 124 | from .models import Category 125 | 126 | 127 | class CategoryDocument(JsonDocument): 128 | class Django: 129 | model = Category 130 | fields = ["name", "slug"] 131 | ``` 132 | 133 | **3. Run Index Django Management Command to create the index on Redis:** 134 | 135 | ```bash 136 | python manage.py index 137 | ``` 138 | 139 | **Note:** This will also populate the index with existing data from the database 140 | 141 | Now category objects will be indexed on create/update/delete. 142 | 143 | #### More Complex Example 144 | 145 | **1. For Django Models:** 146 | 147 | ```python 148 | # models.py 149 | 150 | from django.db import models 151 | 152 | 153 | class Tag(models.Model): 154 | name = models.CharField(max_length=30) 155 | 156 | def __str__(self) -> str: 157 | return self.name 158 | 159 | 160 | class Vendor(models.Model): 161 | name = models.CharField(max_length=30) 162 | email = models.EmailField() 163 | establishment_date = models.DateField() 164 | 165 | def __str__(self) -> str: 166 | return self.name 167 | 168 | 169 | class Product(models.Model): 170 | name = models.CharField(max_length=256) 171 | description = models.TextField(blank=True) 172 | vendor = models.OneToOneField(Vendor, on_delete=models.CASCADE) 173 | tags = models.ManyToManyField(Tag, blank=True) 174 | price = models.DecimalField(max_digits=6, decimal_places=2) 175 | 176 | def __str__(self) -> str: 177 | return self.name 178 | ``` 179 | 180 | **2. You can create a document classes like this:** 181 | 182 | **Note:** Document classes must be stored in `documents.py` file. 183 | 184 | ```python 185 | # documents.py 186 | 187 | from typing import List 188 | 189 | from django.db import models 190 | from redis_om import Field 191 | 192 | from redis_search_django.documents import EmbeddedJsonDocument, JsonDocument 193 | 194 | from .models import Product, Tag, Vendor 195 | 196 | 197 | class TagDocument(EmbeddedJsonDocument): 198 | custom_field: str = Field(index=True, full_text_search=True) 199 | 200 | class Django: 201 | model = Tag 202 | # Model Fields 203 | fields = ["name"] 204 | 205 | @classmethod 206 | def prepare_custom_field(cls, obj): 207 | return "CUSTOM FIELD VALUE" 208 | 209 | 210 | class VendorDocument(EmbeddedJsonDocument): 211 | class Django: 212 | model = Vendor 213 | # Model Fields 214 | fields = ["name", "establishment_date"] 215 | 216 | 217 | class ProductDocument(JsonDocument): 218 | # OnetoOneField, with null=False 219 | vendor: VendorDocument 220 | # ManyToManyField 221 | tags: List[TagDocument] 222 | 223 | class Django: 224 | model = Product 225 | # Model Fields 226 | fields = ["name", "description", "price"] 227 | # Related Model Options 228 | related_models = { 229 | Vendor: { 230 | "related_name": "product", 231 | "many": False, 232 | }, 233 | Tag: { 234 | "related_name": "product_set", 235 | "many": True, 236 | }, 237 | } 238 | 239 | @classmethod 240 | def get_queryset(cls) -> models.QuerySet: 241 | """Override Queryset to filter out available products.""" 242 | return super().get_queryset().filter(available=True) 243 | 244 | @classmethod 245 | def prepare_name(cls, obj): 246 | """Use this to update field value.""" 247 | return obj.name.upper() 248 | ``` 249 | 250 | **Note:** 251 | 252 | - You can not inherit from `HashDocument` for documents that include nested fields. 253 | - You need to inherit from `EmbeddedJsonDocument` for document classes that will be embedded inside another document class. 254 | - You need to explicitly add `OneToOneField`, `ForeignKey` or `ManyToManyField` (e.g: `tags: List[TagDocument]`) with an embedded document class if you want to index them. 255 | you can not add it in the `Django.fields` option. 256 | - For `related_models` option, you need to specify the fields `related_name` and if it is a `ManyToManyField` or a `ForeignKey` Field then specify `"many": True`. 257 | - `related_models` will be used when a related object is saved that contributes to the document. 258 | - You can define `prepare_{field_name}` method to update the value of a field before indexing. 259 | - If it is a custom field (not a model field) you must define a `prepare_{field_name}` method that returns the value of the field. 260 | - You can override `get_queryset` method to provide more filtering. This will be used while indexing a queryset. 261 | - Field names must match model field names or define a `prepare_{field_name}` method. 262 | 263 | 264 | **3. Run Index Django Management Command to create the index on Redis:** 265 | 266 | ```bash 267 | python manage.py index 268 | ``` 269 | 270 | **Note:** This will also populate the index with existing data from the database 271 | 272 | 273 | ### Management Command 274 | 275 | This package comes with `index` management command that can be used to index all the model instances to Redis index if it has a Document class defined. 276 | 277 | **Note:** Make sure that Redis is running before running the command. 278 | 279 | Run the following command to index **all** models that have Document classes defined: 280 | 281 | ```bash 282 | python manage.py index 283 | ``` 284 | 285 | You can use `--migrate-only` option to **only** update the **index schema**. 286 | 287 | ```bash 288 | python manage.py index --migrate-only 289 | ``` 290 | 291 | You can use `--models` to **specify** which models to index (models must have a Document class defined to be indexed). 292 | 293 | ```bash 294 | python manage.py index --models app_name.ModelName app_name2.ModelName2 295 | ``` 296 | 297 | ### Views 298 | 299 | You can use the `redis_search_django.mixin.RediSearchListViewMixin` with a Django Generic View to search for documents. 300 | `RediSearchPaginator` which helps paginate `ReadiSearch` results is also added to this mixin. 301 | 302 | #### Example 303 | 304 | ```python 305 | # views.py 306 | 307 | from django.utils.functional import cached_property 308 | from django.views.generic import ListView 309 | from redis.commands.search import reducers 310 | 311 | from redis_search_django.mixins import RediSearchListViewMixin 312 | 313 | from .documents import ProductDocument 314 | from .models import Product 315 | 316 | 317 | class SearchView(RediSearchListViewMixin, ListView): 318 | paginate_by = 20 319 | model = Product 320 | template_name = "core/search.html" 321 | document_class = ProductDocument 322 | 323 | @cached_property 324 | def search_query_expression(self): 325 | query = self.request.GET.get("query") 326 | query_expression = None 327 | 328 | if query: 329 | query_expression = ( 330 | self.document_class.name % query 331 | | self.document_class.description % query 332 | ) 333 | 334 | return query_expression 335 | 336 | @cached_property 337 | def sort_by(self): 338 | return self.request.GET.get("sort") 339 | 340 | def facets(self): 341 | if self.search_query_expression: 342 | request = self.document_class.build_aggregate_request( 343 | self.search_query_expression 344 | ) 345 | else: 346 | request = self.document_class.build_aggregate_request() 347 | 348 | result = self.document_class.aggregate( 349 | request.group_by( 350 | ["@tags_name"], 351 | reducers.count().alias("count"), 352 | ) 353 | ) 354 | return result 355 | ``` 356 | 357 | ### Search 358 | 359 | This package uses `redis-om` to search for documents. 360 | 361 | #### Example 362 | 363 | ```python 364 | from .documents import ProductDocument 365 | 366 | 367 | categories = ["category1", "category2"] 368 | tags = ["tag1", "tag2"] 369 | 370 | # Search For Products That Match The Search Query (name or description) 371 | query_expression = ( 372 | ProductDocument.name % "Some search query" 373 | | ProductDocument.description % "Some search query" 374 | ) 375 | 376 | # Search For Products That Match The Price Range 377 | query_expression = ( 378 | ProductDocument.price >= float(10) & ProductDocument.price <= float(100) 379 | ) 380 | 381 | # Search for Products that include following Categories 382 | query_expression = ProductDocument.category.name << ["category1", "category2"] 383 | 384 | # Search for Products that include following Tags 385 | query_expression = ProductDocument.tags.name << ["tag1", "tag2"] 386 | 387 | # Query expression can be passed on the `find` method 388 | result = ProductDocument.find(query_expression).sort_by("-price").execute() 389 | ``` 390 | 391 | For more details checkout [redis-om docs](https://github.com/redis/redis-om-python/blob/main/docs/getting_started.md) 392 | 393 | ### RediSearch Aggregation / Faceted Search 394 | 395 | `redis-om` does not support faceted search (RediSearch Aggregation). So this package uses `redis-py` to do faceted search. 396 | 397 | #### Example 398 | 399 | ```python 400 | from redis.commands.search import reducers 401 | 402 | from .documents import ProductDocument 403 | 404 | 405 | query_expression = ( 406 | ProductDocument.name % "Some search query" 407 | | ProductDocument.description % "Some search query" 408 | ) 409 | 410 | # First we need to build the aggregation request 411 | request1 = ProductDocument.build_aggregate_request(query_expression) 412 | request2 = ProductDocument.build_aggregate_request(query_expression) 413 | 414 | # Get the number of products for each category 415 | ProductDocument.aggregate( 416 | request1.group_by( 417 | ["@category_name"], 418 | reducers.count().alias("count"), 419 | ) 420 | ) 421 | # >> [{"category_name": "Shoes", "count": "112"}, {"category_name": "Cloths", "count": "200"}] 422 | 423 | 424 | # Get the number of products for each tag 425 | ProductDocument.aggregate( 426 | request2.group_by( 427 | ["@tags_name"], 428 | reducers.count().alias("count"), 429 | ) 430 | ) 431 | # >> [{"tags_name": "Blue", "count": "14"}, {"tags_name": "Small", "count": "57"}] 432 | ``` 433 | 434 | For more details checkout [redis-py docs](https://redis.readthedocs.io/en/stable/examples/search_json_examples.html?highlight=aggregate#Aggregation) and 435 | [RediSearch Aggregation docs](https://redis.io/docs/stack/search/reference/aggregations/) 436 | 437 | ### Settings 438 | 439 | #### Environment Variables 440 | 441 | - **`REDIS_OM_URL`** (Default: `redis://localhost:6379`): This environment variable follows the `redis-py` URL format. If you are using external redis server 442 | You need to set this variable with the URL of the redis server following this pattern: `redis://[[username]:[password]]@[host]:[post]/[database number]` 443 | 444 | **Example:** `redis://redis_user:password@some.other.part.cloud.redislabs.com:6379/0` 445 | 446 | For more details checkout [redis-om docs](https://github.com/redis/redis-om-python/blob/main/docs/getting_started.md#setting-the-redis-url-environment-variable) 447 | 448 | 449 | #### Django Document Options 450 | 451 | You can add these options on the `Django` class of each Document class: 452 | 453 | ```python 454 | # documents.py 455 | 456 | from redis_search_django.documents import JsonDocument 457 | 458 | from .models import Category, Product, Tag, Vendor 459 | 460 | 461 | class ProductDocument(JsonDocument): 462 | class Django: 463 | model = Product 464 | fields = ["name", "description", "price", "created_at"] 465 | select_related_fields = ["vendor", "category"] 466 | prefetch_related_fields = ["tags"] 467 | auto_index = True 468 | related_models = { 469 | Vendor: { 470 | "related_name": "product", 471 | "many": False, 472 | }, 473 | Category: { 474 | "related_name": "product_set", 475 | "many": True, 476 | }, 477 | Tag: { 478 | "related_name": "product_set", 479 | "many": True, 480 | }, 481 | } 482 | ``` 483 | 484 | - **`model`** (Required): Django Model class to index. 485 | - **`auto_index`** (Default: `True`, Optional): If True, the model instances will be indexed on create/update/delete. 486 | - **`fields`** (Default: `[]`, Optional): List of model fields to index. (Do not add `OneToOneField`, `ForeignKey` or `ManyToManyField` here. These need to be explicitly added to the Document class using `EmbeddedJsonDocument`.) 487 | - **`select_related_fields`** (Default: `[]`, Optional): List of fields to use on `queryset.select_related()`. 488 | - **`prefetch_related_fields`** (Default: `[]`, Optional): List of fields to use on `queryset.prefetch_related()`. 489 | - **`related_models`** (Default: `{}`, Optional): Dictionary of related models. 490 | You need to specify the fields `related_name` and if it is a `ManyToManyField` or a `ForeignKey` Field then specify `"many": True`. 491 | These are used to update the document data if any of the related model instances are updated. 492 | `related_models` will be used when a related object is saved/added/removed/deleted that contributes to the document. 493 | 494 | For `redis-om` specific options checkout [redis-om docs](https://github.com/redis/redis-om-python/blob/main/docs/models.md) 495 | 496 | #### Global Options 497 | 498 | You can add these options to your Django `settings.py` File: 499 | 500 | - **`REDIS_SEARCH_AUTO_INDEX`** (Default: `True`): Enable or Disable Auto Index when model instance is created/updated/deleted for all document classes. 501 | 502 | 503 | # Example Application Screenshot 504 | 505 | ![RediSearch Django](https://user-images.githubusercontent.com/24854406/185760315-4e12d02b-68a2-499a-a6d6-88d8162b5447.png) 506 | 507 | 508 | # License 509 | 510 | The code in this project is released under the [MIT License](LICENSE). 511 | --------------------------------------------------------------------------------