├── .gitignore ├── LICENSE ├── Makefile ├── app ├── cars │ ├── __init__.py │ ├── admin.py │ ├── apps.py │ ├── documents.py │ ├── fixtures │ │ ├── cars.json │ │ └── manufacturers.json │ ├── management │ │ └── commands │ │ │ └── bootstrap.py │ ├── migrations │ │ ├── 0001_initial.py │ │ ├── 0002_alter_car_id_alter_manufacturer_id.py │ │ └── __init__.py │ ├── models.py │ ├── serializers.py │ ├── urls.py │ └── views.py ├── django_elasticsearch_example │ ├── __init__.py │ ├── exceptions.py │ ├── serializers.py │ ├── settings.py │ ├── urls.py │ ├── utils.py │ ├── views.py │ └── wsgi.py └── manage.py ├── docker-compose.yml ├── docker └── Dockerfile ├── readme.md └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | pip-wheel-metadata/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | .hypothesis/ 51 | .pytest_cache/ 52 | 53 | # Translations 54 | *.mo 55 | *.pot 56 | 57 | # Django stuff: 58 | *.log 59 | local_settings.py 60 | db.sqlite3 61 | 62 | # Flask stuff: 63 | instance/ 64 | .webassets-cache 65 | 66 | # Scrapy stuff: 67 | .scrapy 68 | 69 | # Sphinx documentation 70 | docs/_build/ 71 | 72 | # PyBuilder 73 | target/ 74 | 75 | # Jupyter Notebook 76 | .ipynb_checkpoints 77 | 78 | # IPython 79 | profile_default/ 80 | ipython_config.py 81 | 82 | # pyenv 83 | .python-version 84 | 85 | # pipenv 86 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 87 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 88 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 89 | # install all needed dependencies. 90 | #Pipfile.lock 91 | 92 | # celery beat schedule file 93 | celerybeat-schedule 94 | 95 | # SageMath parsed files 96 | *.sage.py 97 | 98 | # Environments 99 | .env 100 | .venv 101 | env/ 102 | venv/ 103 | ENV/ 104 | env.bak/ 105 | venv.bak/ 106 | 107 | # Spyder project settings 108 | .spyderproject 109 | .spyproject 110 | 111 | # Rope project settings 112 | .ropeproject 113 | 114 | # mkdocs documentation 115 | /site 116 | 117 | # mypy 118 | .mypy_cache/ 119 | .dmypy.json 120 | dmypy.json 121 | 122 | # Pyre type checker 123 | .pyre/ 124 | 125 | .DS_Store 126 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2019 SUNSCRAPERS 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: build up down attach bootstrap bash shell migrate migrations 2 | build: 3 | docker compose build 4 | 5 | up: 6 | make build 7 | docker compose up 8 | 9 | down: 10 | docker compose down 11 | 12 | attach: 13 | docker attach django_app 14 | 15 | bootstrap: 16 | docker exec -it django_app python manage.py bootstrap 17 | 18 | bash: 19 | docker exec -it django_app bash 20 | 21 | shell: 22 | docker exec -it django_app python manage.py shell 23 | 24 | migrate: 25 | docker exec -it django_app python manage.py migrate 26 | 27 | migrations: 28 | docker exec -it django_app python manage.py makemigrations 29 | -------------------------------------------------------------------------------- /app/cars/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunscrapers/Django-elasticsearch-example/c2d9ed20550eff11c6b64a08fb095db527ab4563/app/cars/__init__.py -------------------------------------------------------------------------------- /app/cars/admin.py: -------------------------------------------------------------------------------- 1 | from django.contrib import admin 2 | 3 | from cars.models import Car 4 | from cars.models import Manufacturer 5 | 6 | admin.site.register(Car) 7 | admin.site.register(Manufacturer) 8 | -------------------------------------------------------------------------------- /app/cars/apps.py: -------------------------------------------------------------------------------- 1 | from django.apps import AppConfig 2 | 3 | 4 | class CarsConfig(AppConfig): 5 | default_auto_field = "django.db.models.BigAutoField" 6 | name = "cars" 7 | -------------------------------------------------------------------------------- /app/cars/documents.py: -------------------------------------------------------------------------------- 1 | from django_elasticsearch_dsl import Document 2 | from django_elasticsearch_dsl import fields 3 | from django_elasticsearch_dsl.registries import registry 4 | 5 | from cars.models import Car 6 | from cars.models import Manufacturer 7 | 8 | 9 | @registry.register_document 10 | class CarDocument(Document): 11 | name = fields.TextField( 12 | fields={ 13 | "raw": fields.TextField(analyzer="standard"), 14 | "suggest": fields.CompletionField(), 15 | } 16 | ) 17 | manufacturer = fields.ObjectField( 18 | properties={ 19 | "name": fields.TextField(), 20 | "country_code": fields.TextField(), 21 | } 22 | ) 23 | auction_title = fields.TextField(attr="get_auction_title") 24 | points = fields.IntegerField() 25 | 26 | def prepare_points(self, instance): 27 | if instance.color == "silver": 28 | return 2 29 | return 1 30 | 31 | class Index: 32 | name = "cars" 33 | settings = {"number_of_shards": 1, "number_of_replicas": 0} 34 | 35 | class Django: 36 | model = Car 37 | fields = [ 38 | "id", 39 | "color", 40 | "description", 41 | "type", 42 | ] 43 | 44 | related_models = [Manufacturer] 45 | 46 | def get_queryset(self): 47 | return super().get_queryset().select_related("manufacturer") 48 | 49 | def get_instances_from_related(self, related_instance): 50 | if isinstance(related_instance, Manufacturer): 51 | return related_instance.car_set.all() 52 | -------------------------------------------------------------------------------- /app/cars/fixtures/cars.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "model": "cars.car", 4 | "pk": 2, 5 | "fields": { 6 | "name": "Corolla", 7 | "color": "black", 8 | "description": "Why do we use it?\r\nIt is a long established fact that a reader will be distracted by the readable content of a page when looking at its layout. The point of using Lorem Ipsum is that it has a more-or-less normal distribution of letters, as opposed to using 'Content here, content here', making it look like readable English. Many desktop publishing packages and web page editors now use Lorem Ipsum as their default model text, and a search for 'lorem ipsum' will uncover many web sites still in their infancy. Various versions have evolved over the years, sometimes by accident, sometimes on purpose (injected humour and the like).", 9 | "type": 1, 10 | "manufacturer": 2 11 | } 12 | }, 13 | { 14 | "model": "cars.car", 15 | "pk": 3, 16 | "fields": { 17 | "name": "Toledo", 18 | "color": "blue", 19 | "description": "Where does it come from?\r\nContrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, discovered the undoubtable source. Lorem Ipsum comes from sections 1.10.32 and 1.10.33 of \"de Finibus Bonorum et Malorum\" (The Extremes of Good and Evil) by Cicero, written in 45 BC. This book is a treatise on the theory of ethics, very popular during the Renaissance. The first line of Lorem Ipsum, \"Lorem ipsum dolor sit amet..\", comes from a line in section 1.10.32.\r\n\r\nThe standard chunk of Lorem Ipsum used since the 1500s is reproduced below for those interested. Sections 1.10.32 and 1.10.33 from \"de Finibus Bonorum et Malorum\" by Cicero are also reproduced in their exact original form, accompanied by English versions from the 1914 translation by H. Rackham.", 20 | "type": 1, 21 | "manufacturer": 1 22 | } 23 | }, 24 | { 25 | "model": "cars.car", 26 | "pk": 4, 27 | "fields": { 28 | "name": "Corsa", 29 | "color": "green", 30 | "description": "What is Lorem Ipsum?\r\nLorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.", 31 | "type": 3, 32 | "manufacturer": 2 33 | } 34 | }, 35 | { 36 | "model": "cars.car", 37 | "pk": 5, 38 | "fields": { 39 | "name": "Megan", 40 | "color": "black", 41 | "description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam non aliquam nulla, vitae luctus ligula. Aenean interdum iaculis nisi, volutpat consectetur purus lacinia id. Phasellus accumsan fringilla magna quis facilisis. Aenean dignissim tempus nibh, id porta tortor rhoncus at. Nunc auctor quam et consequat congue. Donec mollis sapien nibh, nec iaculis augue vestibulum vel. Nullam viverra mauris vel posuere tincidunt. Duis vulputate lacus mi, sit amet efficitur lorem posuere quis.", 42 | "type": 2, 43 | "manufacturer": 2 44 | } 45 | }, 46 | { 47 | "model": "cars.car", 48 | "pk": 6, 49 | "fields": { 50 | "name": "Ducato", 51 | "color": "white", 52 | "description": "Nam non metus ac odio tincidunt finibus ut sit amet nisl. Ut in mauris sagittis, mattis nunc a, euismod felis. Aenean viverra nisl vitae dolor dictum vulputate. Maecenas tortor justo, faucibus at convallis id, suscipit ultrices magna. Donec aliquam sapien quis augue convallis, in maximus urna rhoncus. Nullam dapibus, tortor at fermentum faucibus, nibh purus fermentum nulla, sed aliquam leo est id tortor. Nulla semper enim eget lorem consequat, non tristique sem sodales. Integer aliquet felis sit amet magna sollicitudin malesuada. Mauris lacinia diam turpis, a iaculis tortor ultricies sed. Suspendisse cursus erat leo, eu porttitor elit dictum in. Morbi ac dolor sed nisl lacinia imperdiet. Sed ante augue, lobortis vel tristique sed, efficitur ut dolor. Aliquam elementum, urna pellentesque viverra cursus, dui augue consequat quam, sed euismod leo sem ac sem. In mattis arcu tellus, id tincidunt quam dignissim ut. Praesent vitae libero et ante efficitur interdum eu vel eros. Quisque urna arcu, tincidunt ut elit sit amet, malesuada elementum urna.", 53 | "type": 2, 54 | "manufacturer": 1 55 | } 56 | }, 57 | { 58 | "model": "cars.car", 59 | "pk": 7, 60 | "fields": { 61 | "name": "A3", 62 | "color": "silver", 63 | "description": "Quisque nec augue semper, cursus lacus vitae, aliquam urna. Etiam pretium consectetur odio, et lacinia eros interdum sagittis. Donec cursus lorem sed commodo scelerisque. Nullam eu diam ut justo vestibulum luctus id in nulla. Nulla scelerisque ornare dictum. Curabitur ullamcorper dolor in risus imperdiet, non tincidunt nibh euismod. Quisque mauris nulla, sodales auctor erat a, dapibus iaculis lacus. Nulla ligula metus, rutrum ut urna et, imperdiet accumsan sapien. Integer ut malesuada mauris, eget ornare ipsum. Praesent non iaculis nibh, venenatis porta mauris. Morbi eu nisi ipsum. Donec efficitur nunc odio, et imperdiet augue tincidunt sit amet.", 64 | "type": 1, 65 | "manufacturer": 1 66 | } 67 | }, 68 | { 69 | "model": "cars.car", 70 | "pk": 8, 71 | "fields": { 72 | "name": "A6", 73 | "color": "black", 74 | "description": "Ut pellentesque sit amet lacus vitae iaculis. In tincidunt vitae elit non mollis. Morbi varius congue nisl nec pulvinar. Pellentesque vel viverra libero, et fermentum mauris. Pellentesque molestie purus ut enim consequat, sit amet vehicula sem mattis. In hac habitasse platea dictumst. Integer enim mi, rhoncus tempor mauris id, porta interdum urna. Phasellus malesuada id turpis vitae cursus. Sed pellentesque est non elit bibendum, eget auctor metus aliquam. Donec elementum ante sed magna mattis, ut eleifend est tristique. Duis vitae tellus vel lacus luctus ornare. Donec vestibulum a augue eget blandit.", 75 | "type": 3, 76 | "manufacturer": 1 77 | } 78 | }, 79 | { 80 | "model": "cars.car", 81 | "pk": 9, 82 | "fields": { 83 | "name": "Octavia", 84 | "color": "white", 85 | "description": "Aliquam tempor, ex vel pellentesque maximus, nunc risus lacinia est, eget tincidunt nisl nibh eget justo. Nunc porta arcu arcu, at accumsan turpis pharetra et. Suspendisse vel diam et quam dictum consectetur eleifend a lorem. Praesent tempor consequat dapibus. Sed sit amet sem ultrices, pharetra nunc sit amet, accumsan augue. Donec eget ante urna. Mauris neque est, varius sed dui id, rutrum faucibus purus. Mauris vel tellus eget leo tincidunt bibendum ut vel ipsum. Duis sem sem, feugiat eget pulvinar eget, blandit id mi. Nam vel velit id mauris finibus sagittis.", 86 | "type": 1, 87 | "manufacturer": 1 88 | } 89 | }, 90 | { 91 | "model": "cars.car", 92 | "pk": 10, 93 | "fields": { 94 | "name": "Kona", 95 | "color": "blue", 96 | "description": "Integer placerat neque aliquet nisi suscipit tempor. Sed maximus, augue non lobortis cursus, quam purus vestibulum neque, lobortis aliquet est dolor sed ligula. Nullam molestie ex neque, sit amet viverra magna sagittis et. Sed interdum, purus vel pellentesque tincidunt, eros nibh gravida elit, non iaculis nulla elit vitae est. Aliquam elementum tellus vel est pellentesque consectetur. Suspendisse porttitor neque odio, vel viverra purus pellentesque quis. Nunc feugiat eu augue nec rhoncus. Suspendisse tempus iaculis nisl ac iaculis. Suspendisse non iaculis nisl, vel suscipit neque. In efficitur turpis sit amet faucibus vulputate. Aliquam malesuada augue faucibus euismod porta.", 97 | "type": 1, 98 | "manufacturer": 1 99 | } 100 | }, 101 | { 102 | "model": "cars.car", 103 | "pk": 11, 104 | "fields": { 105 | "name": "Rover", 106 | "color": "black", 107 | "description": "Nullam quis ante ac metus finibus laoreet. Sed congue erat ut elit accumsan, in venenatis est fringilla. Etiam quis nisl posuere, scelerisque nulla vitae, eleifend lorem. Ut non gravida ligula. Sed sed egestas urna. Praesent congue laoreet cursus. Vestibulum ac eros vel elit eleifend venenatis in ac eros. Aliquam eget placerat nibh. Pellentesque euismod, quam sed ultricies fringilla, neque leo pretium sapien, non finibus lorem ex ac libero. Vivamus libero velit, commodo eget mauris et, posuere vestibulum est. Etiam suscipit felis non nunc vehicula, quis venenatis diam semper. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras lacinia tellus condimentum, rutrum massa ac, faucibus erat. Donec quam elit, ornare sit amet ullamcorper quis, interdum ac ipsum. Proin iaculis tincidunt lectus a sagittis. Proin commodo tortor sit amet est sagittis venenatis eu id justo.", 108 | "type": 3, 109 | "manufacturer": 1 110 | } 111 | } 112 | ] 113 | -------------------------------------------------------------------------------- /app/cars/fixtures/manufacturers.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "model": "cars.manufacturer", 4 | "pk": 1, 5 | "fields": { 6 | "name": "Opel", 7 | "country_code": "de", 8 | "created": "1862-01-21" 9 | } 10 | }, 11 | { 12 | "model": "cars.manufacturer", 13 | "pk": 2, 14 | "fields": { 15 | "name": "Honda", 16 | "country_code": "jp", 17 | "created": "1948-09-24" 18 | } 19 | } 20 | ] 21 | -------------------------------------------------------------------------------- /app/cars/management/commands/bootstrap.py: -------------------------------------------------------------------------------- 1 | from django.core.management import call_command 2 | from django.core.management.base import BaseCommand 3 | from django.core.management.base import CommandError 4 | 5 | from django_elasticsearch_example.utils import wait_elasticsearch_availability 6 | 7 | 8 | class Command(BaseCommand): 9 | help = """ 10 | Setup project. 11 | - apply migrations 12 | - loaddata 13 | - create and populate index and mapping 14 | """ 15 | 16 | def handle(self, *args, **options): 17 | wait_elasticsearch_availability() 18 | try: 19 | call_command("migrate", "--noinput") 20 | call_command("loaddata", "manufacturers.json") 21 | call_command("loaddata", "cars.json") 22 | call_command("search_index", "--rebuild", "-f") 23 | except Exception as exception: 24 | raise CommandError("Something went wrong during executing commands: {}".format(exception)) 25 | self.stdout.write(self.style.SUCCESS("Successfully ended commands")) 26 | -------------------------------------------------------------------------------- /app/cars/migrations/0001_initial.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.2.1 on 2019-05-16 11:41 2 | 3 | from django.db import migrations, models 4 | import django.db.models.deletion 5 | 6 | 7 | class Migration(migrations.Migration): 8 | initial = True 9 | 10 | dependencies = [] 11 | 12 | operations = [ 13 | migrations.CreateModel( 14 | name="Manufacturer", 15 | fields=[ 16 | ("id", models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")), 17 | ("name", models.CharField(max_length=100, verbose_name="name")), 18 | ("country_code", models.CharField(max_length=2, verbose_name="country code")), 19 | ("created", models.DateField(verbose_name="created")), 20 | ], 21 | ), 22 | migrations.CreateModel( 23 | name="Car", 24 | fields=[ 25 | ("id", models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")), 26 | ("name", models.CharField(max_length=100, verbose_name="name")), 27 | ("color", models.CharField(max_length=30, verbose_name="color")), 28 | ("description", models.TextField(verbose_name="description")), 29 | ("type", models.IntegerField(choices=[(1, "Sedan"), (2, "Truck"), (3, "SUV")], verbose_name="type")), 30 | ( 31 | "manufacturer", 32 | models.ForeignKey( 33 | on_delete=django.db.models.deletion.CASCADE, to="cars.Manufacturer", verbose_name="manufacturer" 34 | ), 35 | ), 36 | ], 37 | options={ 38 | "verbose_name": "Car", 39 | "verbose_name_plural": "Cars", 40 | }, 41 | ), 42 | ] 43 | -------------------------------------------------------------------------------- /app/cars/migrations/0002_alter_car_id_alter_manufacturer_id.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 4.2.5 on 2023-11-07 13:59 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | dependencies = [ 8 | ("cars", "0001_initial"), 9 | ] 10 | 11 | operations = [ 12 | migrations.AlterField( 13 | model_name="car", 14 | name="id", 15 | field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID"), 16 | ), 17 | migrations.AlterField( 18 | model_name="manufacturer", 19 | name="id", 20 | field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID"), 21 | ), 22 | ] 23 | -------------------------------------------------------------------------------- /app/cars/migrations/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunscrapers/Django-elasticsearch-example/c2d9ed20550eff11c6b64a08fb095db527ab4563/app/cars/migrations/__init__.py -------------------------------------------------------------------------------- /app/cars/models.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | from django.utils.translation import gettext_lazy as _ 3 | 4 | 5 | class Manufacturer(models.Model): 6 | name = models.CharField( 7 | _("name"), 8 | max_length=100, 9 | ) 10 | country_code = models.CharField( 11 | _("country code"), 12 | max_length=2, 13 | ) 14 | created = models.DateField( 15 | _("created"), 16 | ) 17 | 18 | def __str__(self): 19 | return self.name 20 | 21 | 22 | class Car(models.Model): 23 | TYPES = [ 24 | (1, "Sedan"), 25 | (2, "Truck"), 26 | (3, "SUV"), 27 | ] 28 | 29 | class Meta: 30 | verbose_name = _("Car") 31 | verbose_name_plural = _("Cars") 32 | 33 | name = models.CharField( 34 | _("name"), 35 | max_length=100, 36 | ) 37 | color = models.CharField( 38 | _("color"), 39 | max_length=30, 40 | ) 41 | description = models.TextField( 42 | _("description"), 43 | ) 44 | type = models.IntegerField( 45 | _("type"), 46 | choices=TYPES, 47 | ) 48 | manufacturer = models.ForeignKey( 49 | Manufacturer, 50 | on_delete=models.CASCADE, 51 | verbose_name=_("manufacturer"), 52 | ) 53 | 54 | def __str__(self): 55 | return self.name 56 | 57 | def get_auction_title(self): 58 | return f"{self.name} - {self.color}" 59 | -------------------------------------------------------------------------------- /app/cars/serializers.py: -------------------------------------------------------------------------------- 1 | from rest_framework import serializers 2 | from rest_framework.serializers import ModelSerializer 3 | 4 | from cars.models import Car 5 | from cars.models import Manufacturer 6 | 7 | 8 | class SearchQuerySerializer(serializers.Serializer): 9 | query = serializers.CharField(max_length=200) 10 | limit = serializers.IntegerField(required=False, default=2) 11 | offset = serializers.IntegerField(required=False, default=0) 12 | 13 | 14 | class ManufacturerSerializer(ModelSerializer): 15 | class Meta: 16 | model = Manufacturer 17 | fields = ( 18 | "name", 19 | "country_code", 20 | ) 21 | 22 | 23 | class CarSerializer(ModelSerializer): 24 | manufacturer = ManufacturerSerializer() 25 | points = serializers.IntegerField( 26 | required=False, 27 | default=1, 28 | ) 29 | auction_title = serializers.CharField() 30 | 31 | class Meta: 32 | model = Car 33 | fields = ( 34 | "id", 35 | "name", 36 | "type", 37 | "description", 38 | "points", 39 | "color", 40 | "auction_title", 41 | "manufacturer", 42 | ) 43 | -------------------------------------------------------------------------------- /app/cars/urls.py: -------------------------------------------------------------------------------- 1 | from django.urls import path 2 | 3 | from cars.views import CarSearchAPIView 4 | 5 | app_name = "cars" 6 | 7 | urlpatterns = [ 8 | path("", CarSearchAPIView.as_view(), name="cars-api"), 9 | ] 10 | -------------------------------------------------------------------------------- /app/cars/views.py: -------------------------------------------------------------------------------- 1 | from elasticsearch_dsl import Q 2 | 3 | from cars.documents import CarDocument 4 | from cars.serializers import CarSerializer 5 | from django_elasticsearch_example.views import ElasticSearchAPIView 6 | 7 | 8 | class CarSearchAPIView(ElasticSearchAPIView): 9 | serializer_class = CarSerializer 10 | document_class = CarDocument 11 | 12 | def elasticsearch_query_expression(self, query): 13 | return Q( 14 | "bool", 15 | should=[ 16 | Q("match", name=query), 17 | Q("match", color=query), 18 | Q("match", description=query), 19 | ], 20 | minimum_should_match=1, 21 | ) 22 | -------------------------------------------------------------------------------- /app/django_elasticsearch_example/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunscrapers/Django-elasticsearch-example/c2d9ed20550eff11c6b64a08fb095db527ab4563/app/django_elasticsearch_example/__init__.py -------------------------------------------------------------------------------- /app/django_elasticsearch_example/exceptions.py: -------------------------------------------------------------------------------- 1 | class APIViewError(Exception): 2 | ... 3 | -------------------------------------------------------------------------------- /app/django_elasticsearch_example/serializers.py: -------------------------------------------------------------------------------- 1 | from rest_framework import serializers 2 | 3 | 4 | class SearchQuerySerializer(serializers.Serializer): 5 | query = serializers.CharField(max_length=200) 6 | limit = serializers.IntegerField(required=False, default=10) 7 | offset = serializers.IntegerField(required=False, default=0) 8 | -------------------------------------------------------------------------------- /app/django_elasticsearch_example/settings.py: -------------------------------------------------------------------------------- 1 | """ 2 | Django settings for django_elasticsearch_example project. 3 | 4 | Generated by 'django-admin startproject' using Django 2.2.1. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/2.2/topics/settings/ 8 | 9 | For the full list of settings and their values, see 10 | https://docs.djangoproject.com/en/2.2/ref/settings/ 11 | """ 12 | 13 | import os 14 | 15 | # Build paths inside the project like this: os.path.join(BASE_DIR, ...) 16 | BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 17 | 18 | 19 | # Quick-start development settings - unsuitable for production 20 | # See https://docs.djangoproject.com/en/2.2/howto/deployment/checklist/ 21 | 22 | # SECURITY WARNING: keep the secret key used in production secret! 23 | SECRET_KEY = "h2%%w^!n#(ce9#p)&jf6_%-fje*b^zi+t=i&4picrkp^$e!z_c" 24 | 25 | # SECURITY WARNING: don't run with debug turned on in production! 26 | DEBUG = True 27 | 28 | ALLOWED_HOSTS = [] 29 | 30 | 31 | # Application definition 32 | 33 | INSTALLED_APPS = [ 34 | "django.contrib.admin", 35 | "django.contrib.auth", 36 | "django.contrib.contenttypes", 37 | "django.contrib.sessions", 38 | "django.contrib.messages", 39 | "django.contrib.staticfiles", 40 | # Third-party apps 41 | "rest_framework", 42 | "django_elasticsearch_dsl", 43 | # Local apps 44 | "cars", 45 | ] 46 | 47 | MIDDLEWARE = [ 48 | "django.middleware.security.SecurityMiddleware", 49 | "django.contrib.sessions.middleware.SessionMiddleware", 50 | "django.middleware.common.CommonMiddleware", 51 | "django.middleware.csrf.CsrfViewMiddleware", 52 | "django.contrib.auth.middleware.AuthenticationMiddleware", 53 | "django.contrib.messages.middleware.MessageMiddleware", 54 | "django.middleware.clickjacking.XFrameOptionsMiddleware", 55 | ] 56 | 57 | ROOT_URLCONF = "django_elasticsearch_example.urls" 58 | 59 | TEMPLATES = [ 60 | { 61 | "BACKEND": "django.template.backends.django.DjangoTemplates", 62 | "DIRS": [], 63 | "APP_DIRS": True, 64 | "OPTIONS": { 65 | "context_processors": [ 66 | "django.template.context_processors.debug", 67 | "django.template.context_processors.request", 68 | "django.contrib.auth.context_processors.auth", 69 | "django.contrib.messages.context_processors.messages", 70 | ], 71 | }, 72 | }, 73 | ] 74 | 75 | WSGI_APPLICATION = "django_elasticsearch_example.wsgi.application" 76 | 77 | 78 | # Database 79 | # https://docs.djangoproject.com/en/2.2/ref/settings/#databases 80 | 81 | DATABASES = { 82 | "default": { 83 | "ENGINE": "django.db.backends.postgresql", 84 | "NAME": "postgres", 85 | "USER": "postgres", 86 | "PASSWORD": "postgres", 87 | "HOST": "db", 88 | "PORT": "5432", 89 | } 90 | } 91 | 92 | 93 | # Password validation 94 | # https://docs.djangoproject.com/en/2.2/ref/settings/#auth-password-validators 95 | 96 | AUTH_PASSWORD_VALIDATORS = [ 97 | { 98 | "NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator", 99 | }, 100 | { 101 | "NAME": "django.contrib.auth.password_validation.MinimumLengthValidator", 102 | }, 103 | { 104 | "NAME": "django.contrib.auth.password_validation.CommonPasswordValidator", 105 | }, 106 | { 107 | "NAME": "django.contrib.auth.password_validation.NumericPasswordValidator", 108 | }, 109 | ] 110 | 111 | 112 | # Internationalization 113 | # https://docs.djangoproject.com/en/2.2/topics/i18n/ 114 | 115 | LANGUAGE_CODE = "en-us" 116 | 117 | TIME_ZONE = "UTC" 118 | 119 | USE_I18N = True 120 | 121 | USE_L10N = True 122 | 123 | USE_TZ = True 124 | 125 | 126 | # Static files (CSS, JavaScript, Images) 127 | # https://docs.djangoproject.com/en/2.2/howto/static-files/ 128 | 129 | STATIC_URL = "/static/" 130 | 131 | # Elasticsearch configuration 132 | ELASTICSEARCH_DSL = { 133 | "default": { 134 | "hosts": ["http://elasticsearch:9200"], 135 | }, 136 | } 137 | -------------------------------------------------------------------------------- /app/django_elasticsearch_example/urls.py: -------------------------------------------------------------------------------- 1 | """django_elasticsearch_example URL Configuration 2 | 3 | The `urlpatterns` list routes URLs to views. For more information please see: 4 | https://docs.djangoproject.com/en/2.2/topics/http/urls/ 5 | Examples: 6 | Function views 7 | 1. Add an import: from my_app import views 8 | 2. Add a URL to urlpatterns: path('', views.home, name='home') 9 | Class-based views 10 | 1. Add an import: from other_app.views import Home 11 | 2. Add a URL to urlpatterns: path('', Home.as_view(), name='home') 12 | Including another URLconf 13 | 1. Import the include() function: from django.urls import include, path 14 | 2. Add a URL to urlpatterns: path('blog/', include('blog.urls')) 15 | """ 16 | from django.contrib import admin 17 | from django.urls import include 18 | from django.urls import path 19 | 20 | from django_elasticsearch_example.views import IndexRedirectView 21 | 22 | urlpatterns = [ 23 | path("", IndexRedirectView.as_view(), name="index"), 24 | path("admin/", admin.site.urls), 25 | path("cars/", include("cars.urls", namespace="cars")), 26 | ] 27 | -------------------------------------------------------------------------------- /app/django_elasticsearch_example/utils.py: -------------------------------------------------------------------------------- 1 | import logging 2 | 3 | from elasticsearch_dsl.connections import connections 4 | 5 | 6 | class DisableLogger: 7 | def __enter__(self): 8 | logging.disable(logging.CRITICAL) 9 | 10 | def __exit__(self, a, b, c): 11 | logging.disable(logging.NOTSET) 12 | 13 | 14 | def wait_elasticsearch_availability(): 15 | good_statuses = ["green", "yellow"] 16 | with DisableLogger(): 17 | while True: 18 | try: 19 | respone = connections.get_connection().cluster.health() 20 | if respone.get("status") in good_statuses: 21 | return 22 | except Exception: 23 | pass 24 | -------------------------------------------------------------------------------- /app/django_elasticsearch_example/views.py: -------------------------------------------------------------------------------- 1 | import abc 2 | from typing import Any 3 | from typing import Dict 4 | 5 | from django.urls import reverse 6 | from django.views.generic import RedirectView 7 | from elasticsearch_dsl.query import Bool 8 | from elasticsearch_dsl.response import Response 9 | from elasticsearch_dsl.search import Search 10 | from rest_framework import status 11 | from rest_framework.response import Response as DRFResponse 12 | from rest_framework.views import APIView 13 | 14 | from django_elasticsearch_example.exceptions import APIViewError 15 | from django_elasticsearch_example.serializers import SearchQuerySerializer 16 | 17 | 18 | class IndexRedirectView(RedirectView): 19 | def get_redirect_url(self, *args, **kwargs): 20 | return str(reverse("cars:cars-api")) 21 | 22 | 23 | class ElasticSearchAPIView(APIView): 24 | """ 25 | An API view for paginating and retrieving search results from Elasticsearch. 26 | 27 | Attributes: 28 | serializer_class (type): The serializer class to use for formatting search results. 29 | document_class (type): The Elasticsearch document class representing the data. 30 | 31 | Subclasses must implement the 'elasticsearch_query_expression' method, which returns a query expression (Q()). 32 | 33 | Pagination Limitation: 34 | By default, you cannot use 'from' and 'size' to page through more than 10,000 hits. 35 | This limit is a safeguard set by the 'index.max_result_window' index setting. 36 | If you need to page through more than 10,000 hits, use more advanced solution. 37 | 38 | Elasticsearch docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html 39 | """ 40 | 41 | serializer_class = None 42 | document_class = None 43 | query_serializer_class = SearchQuerySerializer 44 | 45 | @abc.abstractmethod 46 | def elasticsearch_query_expression(self, query): 47 | """This method should be overridden and return a Q() expression.""" 48 | 49 | def get(self, request): 50 | """ 51 | Handle GET requests for paginated search results. 52 | 53 | This endpoint allows paginated retrieval of search results from Elasticsearch. 54 | Pagination parameters are expected as query parameters. 55 | 56 | Parameters: 57 | - query (str): The search query string. 58 | - offset (int): The starting position of the search results. 59 | - limit (int): The number of results to retrieve in a single page. 60 | 61 | Returns: 62 | A paginated list of search results serialized according to 'serializer_class'. 63 | 64 | Raises: 65 | - HTTP 400 Bad Request: If query parameters are invalid. 66 | - HTTP 500 Internal Server Error: If an error occurs during data retrieval. 67 | """ 68 | search_query: SearchQuerySerializer = self.query_serializer_class(data=request.GET.dict()) 69 | if not search_query.is_valid(): 70 | return DRFResponse(f"Validation error: {search_query.errors}", status=status.HTTP_400_BAD_REQUEST) 71 | 72 | query_data: Dict[str, Any] = search_query.data 73 | try: 74 | search_query: Bool = self.elasticsearch_query_expression(query_data["query"]) 75 | search: Search = self.document_class.search().query(search_query) 76 | 77 | search = search[query_data["offset"] : query_data["limit"]] 78 | response: Response = search.execute() 79 | 80 | serializer = self.serializer_class(list(response.hits), many=True) 81 | return DRFResponse(serializer.data, status=status.HTTP_200_OK) 82 | except APIViewError: 83 | return DRFResponse("Error during fetching data", status=status.HTTP_500_INTERNAL_SERVER_ERROR) 84 | -------------------------------------------------------------------------------- /app/django_elasticsearch_example/wsgi.py: -------------------------------------------------------------------------------- 1 | """ 2 | WSGI config for django_elasticsearch_example project. 3 | 4 | It exposes the WSGI callable as a module-level variable named ``application``. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/2.2/howto/deployment/wsgi/ 8 | """ 9 | 10 | import os 11 | 12 | from django.core.wsgi import get_wsgi_application 13 | 14 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "django_elasticsearch_example.settings") 15 | 16 | application = get_wsgi_application() 17 | -------------------------------------------------------------------------------- /app/manage.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | """Django's command-line utility for administrative tasks.""" 3 | import os 4 | import sys 5 | 6 | 7 | def main(): 8 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "django_elasticsearch_example.settings") 9 | try: 10 | from django.core.management import execute_from_command_line 11 | except ImportError as exc: 12 | raise ImportError( 13 | "Couldn't import Django. Are you sure it's installed and " 14 | "available on your PYTHONPATH environment variable? Did you " 15 | "forget to activate a virtual environment?" 16 | ) from exc 17 | execute_from_command_line(sys.argv) 18 | 19 | 20 | if __name__ == "__main__": 21 | main() 22 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: "3" 2 | 3 | x-base-app-conf: &base_app_conf # env_file: .env 4 | stdin_open: true 5 | tty: true 6 | 7 | services: 8 | django_app: 9 | <<: *base_app_conf 10 | image: django_app:latest 11 | container_name: django_app 12 | restart: always 13 | build: 14 | context: . 15 | dockerfile: docker/Dockerfile 16 | ports: 17 | - "8000:8000" 18 | volumes: 19 | - "./app:/app" 20 | depends_on: 21 | - db 22 | - elasticsearch 23 | - django-migrations 24 | 25 | # Apply Django migrations 26 | django-migrations: 27 | <<: *base_app_conf 28 | image: django_app:latest 29 | container_name: django-migrations 30 | command: python manage.py migrate 31 | restart: no 32 | build: 33 | context: . 34 | dockerfile: docker/Dockerfile 35 | volumes: 36 | - "./app:/app" 37 | depends_on: 38 | - db 39 | 40 | db: 41 | image: postgres:16.0-alpine 42 | command: ["postgres", "-c", "log_statement=all", "-c", "log_destination=stderr"] 43 | container_name: db 44 | restart: always 45 | environment: 46 | - POSTGRES_DB=mydatabase 47 | - POSTGRES_USER=postgres 48 | - POSTGRES_PASSWORD=postgres 49 | ports: 50 | - "5432:5432" 51 | volumes: 52 | - django_app_db:/var/lib/postgresql/data 53 | 54 | elasticsearch: 55 | image: elasticsearch:8.10.3 56 | container_name: elasticsearch 57 | restart: always 58 | environment: 59 | - cluster.name=docker-cluster 60 | - bootstrap.memory_lock=true 61 | - "ES_JAVA_OPTS=-Xms512m -Xmx512m" 62 | - discovery.type=single-node 63 | - xpack.security.enabled=false 64 | ports: 65 | - 9200:9200 66 | volumes: 67 | - elasticsearch_data:/usr/share/elasticsearch/data 68 | 69 | volumes: 70 | django_app_db: 71 | elasticsearch_data: 72 | -------------------------------------------------------------------------------- /docker/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.11.5-slim 2 | 3 | ADD requirements.txt requirements.txt 4 | RUN pip install -r requirements.txt 5 | 6 | ADD /app /app 7 | WORKDIR /app 8 | 9 | CMD python manage.py runserver 0.0.0.0:8000 10 | -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | # Django Elasticsearch Example 2 | 3 | Simple project to test Elasticsearch with Django, build on docker. 4 | 5 | ###### WARNING! This project is only for local testing, it's not prepared for deployment into remote server. 6 | 7 | ## Prerequisites 8 | 9 | - Docker 10 | - Docker-compose 11 | 12 | ## Getting Started 13 | 14 | Steps to build, load data from fixtures and run project: 15 | 16 | 1. Enter into root of the project 17 | 2. `make up` 18 | 3. `make bootstrap` (in another terminal window) 19 | 20 | To test Elasticsearch in shell run these commands: 21 | 22 | 1. `make up` 23 | 2. `make shell` or `make bash` 24 | 25 | ## Examples of usage 26 | 27 | ``` 28 | cars = CarDocument.search().query('match', color='black') 29 | for car in cars: 30 | print(car.color) 31 | 32 | cars = CarDocument.search().extra(size=0) 33 | cars.aggs.bucket('points_count', 'terms', field='points') 34 | result = cars.execute() 35 | for point in result.aggregations.points_count: 36 | print(point) 37 | ``` 38 | 39 | - http://localhost:8000/cars/?query=is - search and display cars, which contain the word ‘is’ in at least one of those fields: `name`, `color`, `description` 40 | 41 | ## Links 42 | 43 | Check out article to this project: 44 | https://sunscrapers.com/blog/how-to-use-elasticsearch-with-django/ 45 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | Django==4.2.7 2 | psycopg2-binary==2.9.7 3 | elasticsearch-dsl==8.9.0 4 | django-elasticsearch-dsl==8.0 5 | 6 | djangorestframework==3.14.0 7 | django-filter==23.3 8 | Markdown==3.5.1 9 | 10 | # Local development 11 | ipython 12 | ipdb 13 | --------------------------------------------------------------------------------