98 |
99 | Tested on:
100 | - Django 3.1+
101 | - Postgres, SQLite, MariaDB
102 | - Python 3.7+
103 |
104 | Tested with the following libraries:
105 | - [Django Rest Framework](https://www.django-rest-framework.org/)
106 | - [django-reversion](https://github.com/etianen/django-reversion) & [django-reversion-compare](https://github.com/jedie/django-reversion-compare)
107 | - [django-filters](https://github.com/carltongibson/django-filter)
108 |
109 | Pros:
110 | * Battletested in production - [Aristotle Metadata](https://www.aristotlemetadata.com) built, support and uses this library for 2 separate products, served to government and enterprise clients!
111 | * Fetching all translations for a model requires a single query
112 | * Translations are stored in a single database field with the model
113 | * All translations act like a regular field, so `Model.field_name = "some string"` and `print(Model.field_name)` work as you would expect
114 | * Includes a configurable middleware that can set the current language context based on users cookies, query string or HTTP headers
115 | * Works nicely with Django Rest Framework - translatable fields can be set as strings or as json dictionaries
116 | * Works nicely with Django `F()` and `Q()` objects within the ORM - and when it doesn't we have a language aware `LangF()` replacement you can use.
117 |
118 | Cons:
119 | * You need to alter the models, so you can't make third-party libraries translatable.
120 | * It doesn't work on `queryset.values_list` or `queryset.values` natively - but we have an easy Queryset Mixin to add language support for both below.
121 |
122 | ## Why write a new Django field translator?
123 |
124 | A few reasons:
125 | * Most existing django field translation libraries are static, and add separate database columns or extra tables per translation.
126 | * Other libraries may not be compatible with common django libraries, like django-rest-framework.
127 | * We had a huge codebase that we wanted to upgrade to be multilingual - so we needed a library that could be added in without requiring a rewriting every access to fields on models, and only required minor tweaks.
128 |
129 | Note: Field language is different to the django display language. Django can be set up to translate your pages based on the users browser and serve them with a user interface in their preferred language.
130 |
131 | Garnett *does not* use the browser language by design - a user with a French browser may want the user interface in French, but want to see content in English or French based on their needs.
132 |
133 |
134 | # How to install
135 |
136 | 1. Add `django-garnett` to your dependencies. eg. `pip install django-garnett`
137 | 2. Convert your chosen field using the `Translated` function
138 |
139 | * For example: `title = fields.Translated(models.CharField(*args))`
140 |
141 | 3. Add `GARNETT_TRANSLATABLE_LANGUAGES` (a callable or list of [language codes][term-language-code]) to your django settings.
142 | > Note: At the moment there is no way to allow "a user to enter any language".
143 | 4. Add `GARNETT_DEFAULT_TRANSLATABLE_LANGUAGE` (a callable or single [language code][term-language-code]) to your settings.
144 | 5. Re-run `django makemigrations`, perform a data migration to make your existing data translatable (See 'Data migrations') & `django migrate` for any apps you've updated.
145 | 6. Thats mostly it.
146 |
147 | You can also add a few optional value adds:
148 |
149 | 7. (Optional) Add a garnett middleware to take care of field language handling:
150 |
151 | * You want to capture the garnett language in a context variable available in views use: `garnett.middleware.TranslationContextMiddleware`
152 |
153 | * You want to capture the garnett language in a context variable available in views, and want to raise a 404 if the user requests an invalid language use: `garnett.middleware.TranslationContextNotFoundMiddleware`
154 |
155 | * (Future addition) You want to capture the garnett language in a context variable available in views, and want to redirect to the default language if the user requests an invalid language use: `garnett.middleware.TranslationContextRedirectDefaultMiddleware`
156 |
157 | * If you want to cache the current language in session storage use `garnett.middleware.TranslationCacheMiddleware` after one of the above middleware (this is useful with the session selector mentioned below)
158 |
159 | 8. (Optional) Add the `garnett` app to your `INSTALLED_APPS` to use garnett's template_tags. If this is installed before `django.contrib.admin` it also include a language switcher in the Django Admin Site.
160 |
161 | 9. (Optional) Add a template processor:
162 |
163 | * Install `garnett.context_processors.languages` this will add `garnett_languages` (a list of available `Language`s) and `garnett_current_language` (the currently selected language).
164 |
165 | 10. (Optional) Add a custom translation fallback:
166 |
167 | By default, if a language isn't available for a field, Garnett will show a mesage like:
168 | > No translation of this field available in English
169 |
170 | You can override this either by creating a custom fallback method:
171 | ```
172 | Translated(CharField(max_length=150), fallback=my_fallback_method))
173 | ```
174 | Where `my_fallback_method` takes a dictionary of language codes and corresponding strings, and returns the necessary text.
175 |
176 | Additionally, you can customise how django outputs text in templates by creating a new
177 | `TranslationStr` class, and overriding the [`__html__` method][dunder-html].
178 |
179 |
180 | ## Data migrations
181 |
182 | If you have lots of existing data (and if you are using this library you probably do) you will need to perform data migrations to make sure all of your existing data is multi-lingual aware. Fortunately, we've added some well tested migration utilities that can take care of this for you.
183 |
184 | Once you have run `django-admin makemigrations` you just need to add the `step_1_safe_encode_content` before and `step_2_safe_prepare_translations` after your schema migrations, like in the following example:
185 |
186 | ```python
187 | # Generated by Django 3.1.13 on 2022-01-11 10:13
188 |
189 | from django.db import migrations, models
190 | import garnett.fields
191 | import library_app.models
192 |
193 | #### Add this line in
194 | from garnett.migrate import step_1_safe_encode_content, step_2_safe_prepare_translations
195 |
196 | #### Define the models and fields you want ot migrate
197 | model_fields = {
198 | "book": ["title", "description"],
199 | }
200 |
201 |
202 | class Migration(migrations.Migration):
203 |
204 | dependencies = [
205 | ("library_app", "0001_initial"),
206 | ]
207 |
208 | operations = [
209 | ## Add this operation at the start
210 | step_1_safe_encode_content("library_app", model_fields),
211 |
212 | ## These are the automatically generated migrations
213 | migrations.AlterField( # ... migrate title to TranslatedField),
214 | migrations.AlterField( # ... migrate description to TranslatedField),
215 |
216 | ## Add this operation at the start
217 | step_2_safe_prepare_translations("library_app", model_fields),
218 | ]
219 |
220 | ```
221 |
222 |
223 | ## `Language` vs language
224 |
225 | Django Garnett uses the python `langcodes` library to determine more information about the languages being used - including the full name and local name of the language being used. This is stored as a `Language` object.
226 |
227 |
228 | ## Django Settings options:
229 |
230 | * `GARNETT_DEFAULT_TRANSLATABLE_LANGUAGE`
231 | * Stores the default language to be used for reading and writing fields if no language is set in a context manager or by a request.
232 | * By default it is 'en-AU' the [language code][term-language-code] for 'Strayan, the native tongue of inhabitants of 'Straya (or more commonly known as Australia).
233 | * This can also be a callable that returns list of language codes. Combined with storing user settings in something like (django-solo)[https://github.com/lazybird/django-solo] users can dynamically add or change their language settings.
234 | * default: `'en-AU'`
235 | * `GARNETT_TRANSLATABLE_LANGUAGES`:
236 | * Stores a list of [language codes][term-language-code] that users can use to save against TranslatableFields.
237 | * This can also be a callable that returns list of language codes. Combined with storing user settings in something like (django-solo)[https://github.com/lazybird/django-solo] users can dynamically add or change their language settings.
238 | * default `[GARNETT_DEFAULT_TRANSLATABLE_LANGUAGE]`
239 | * `GARNETT_REQUEST_LANGUAGE_SELECTORS`:
240 | * A list of string modules that determines the order of options used to determine the language selected by the user. The first selector found is used for the language for the request, if none are found the DEFAULT_LANGUAGE is used. These can any of the following in any order:
241 | * `garnett.selector.query`: Checks the `GARNETT_QUERY_PARAMETER_NAME` for a language to display
242 | * `garnett.selector.cookie`: Checks for a cookie called `GARNETT_LANGUAGE_CODE` for a language to display.
243 | Note: you cannot change this cookie name.
244 | * `garnett.selector.session`: Checks for a session key `GARNETT_LANGUAGE_CODE` for a language to display.
245 | Note: you cannot change this key name.
246 | * `garnett.selector.header`: Checks for a HTTP Header called `X-Garnett-Language-Code` for a language to display.
247 | Note: you cannot change this Header name.
248 | * `garnett.selector.browser`: Uses Django's `get_language` function to get the users browser/UI language [as determined by Django][django-how].
249 | * For example, if you only want to check headers and cookies in that order, set this to `['garnett.selectors.header', 'garnett.selectors.cookie']`.
250 | * default: `['garnett.selectors.header', 'garnett.selectors.query', 'garnett.selectors.cookie']`
251 | * `GARNETT_QUERY_PARAMETER_NAME`:
252 | * The query parameter used to determine the language requested by a user during a HTTP request.
253 | * default: `glang`
254 | * `GARNETT_ALLOW_BLANK_FALLBACK_OVERRIDE`
255 | * If set to true, when allpying the current language middleware, this will check for an extra get URL parameter to override the fallback to return blank strings where a languge has no content. This is useful for APIs
256 | * default: False
257 |
258 | Advanced Settings (you probably don't need to adjust these)
259 | * `GARNETT_TRANSLATABLE_FIELDS_PROPERTY_NAME`:
260 | * Garnett adds a property to all models that returns a list of all TranslatableFields. By default, this is 'translatable_fields', but you can customise it here if you want.
261 | * default: `translatable_fields`
262 | * `GARNETT_TRANSLATIONS_PROPERTY_NAME`:
263 | * Garnett adds a property to all models that returns a dictionary of all translations of all TranslatableFields. By default, this is 'translations', but you can customise it here if you want.
264 | * default: `translations`
265 |
266 | # Using Garnett
267 |
268 | If you did everything above correctly, garnett should for the most part "just work".
269 |
270 | ## Switching the active language
271 |
272 | Garnett comes with a handy context manager that can be used to specify the current language. In any place where you want to manually control the current language, wrap your code in `set_field_language` and garnett will correctly store the language. This can be nested, or you can change the language for a context multiple times before saving.
273 |
274 | ```python
275 | from garnett.context import set_field_language
276 | greeting = Greeting(text="Hello", target="World")
277 |
278 | with set_field_language("en"):
279 | greeting.text = "Hello"
280 | with set_field_language("fr"):
281 | greeting.text = "Bonjour"
282 |
283 | greeting.save()
284 | ```
285 |
286 | ## Using Garnett with `values_list` or `values`
287 |
288 | This is one of the areas that garnett _doesn't_ work immediately, but there is a solution.
289 |
290 | In the places you are using values lists or `values`, wrap any translated field in an L-expression and the values list will return correctly. For example:
291 |
292 | ```python
293 | from garnett.expressions import L
294 | Book.objects.values_list(L("title"))
295 | Book.objects.values(L("title"))
296 | ```
297 |
298 |
299 | ## Using Garnett with Django-Rest-Framework
300 |
301 | As `TranslationField`s are based on JSONField, by default Django-Rest-Framework renders these as a JSONField, which may not be ideal.
302 |
303 | You can get around this by using the `TranslatableSerializerMixin` _as the first mixin_, which adds the necessary hooks to your serializer. This will mean class changes, but you won't need to update or override every field.
304 |
305 | For example:
306 |
307 | ```
308 | from rest_framework import serializers
309 | from library_app import models
310 | from garnett.ext.drf import TranslatableSerializerMixin
311 |
312 |
313 | class BookSerializer(TranslatableSerializerMixin, serializers.ModelSerializer):
314 | class Meta:
315 | model = models.Book
316 | fields = "__all__"
317 | ```
318 |
319 | This will allow you to set the value for a translatable as either a string for the active langauge, or by setting a dictionary that has all languages to be saved (note: this will override the existing language set).
320 | For example:
321 |
322 | To override just the active language:
323 |
324 | curl -X PATCH ... -d "{ \"title\": \"Hello\"}"
325 |
326 | To specifically override a single language (for example, Klingon):
327 |
328 | curl -X PATCH ... -H "X-Garnett-Language-Code: tlh" -d "{ \"title\": \"Hello\"}"
329 |
330 | To override all languages:
331 |
332 | curl -X PATCH ... -d "{ \"title\": {\"en\": \"Hello\", \"fr\": \"Bonjour\"}}"
333 |
334 | ## Using Garnett with django-reversion and django-reversion-compare
335 |
336 | There are a few minor tweaks required to get Garnett to operate properly with
337 | django-reversion and django-reversion-compare based on how they serialise and display data.
338 |
339 | This is because Garnett does not use the same 'field.attname' and 'field.name' which means serialization in Django will not work correctly.
340 |
341 | To get django-reversion to work you will need to use a translation-aware serialiser and apply a patch to ensure that django-reversion-compare can show the right information.
342 |
343 | An example json translation-aware serializer is included with Garnett and this can be applied with the following two settings in `settings.py`:
344 |
345 | ```
346 | # In settings.py
347 |
348 | GARNETT_PATCH_REVERSION_COMPARE = True
349 | SERIALIZATION_MODULES = {"json": "garnett.serializers.json"}
350 | ```
351 |
352 | TranslatedFields will list the history and changes in json, but it does do comparisons correctly.
353 |
354 | ## Why call it Garnett?
355 |
356 | * Libraries need a good name.
357 | * Searching for "Famous Translators" will tell you about [Constance Garnett](https://en.wikipedia.org/wiki/Constance_Garnett).
358 | * Searching for "Django Garnett" showed there was no python library with this name.
359 | * It did however talk about [Garnet Clark](https://en.wikipedia.org/wiki/Garnet_Clark) (also spelled Garnett), a jazz pianist who played with Django Reinhart - the namesake of the Django Web Framework.
360 | * Voila - a nice name
361 |
362 | ## Warnings
363 |
364 | * `contains` acts like `icontains` on SQLite only when doing a contains query
365 | it does a case insensitive search. I don't know why - https://www.youtube.com/watch?v=PgGNWRtceag
366 | * Due to how django sets admin form fields you will not get the admin specific widgets like
367 | `AdminTextAreaWidget` on translated fields in the django admin site by default. They can however
368 | be specified explicitly on the corresponding admin model form.
369 |
370 | ## Want to help maintain this library?
371 |
372 | There is a `/dev/` directory with a docker-compose stack you can ues to bring up a database and clean development environment.
373 |
374 | ## Want other options?
375 |
376 | There are a few good options for adding translatable strings to Django that may meet other use cases. We've included a few other options here, their strengths and why we didn't go with them.
377 |
378 | * [django-modeltranslation](https://github.com/deschler/django-modeltranslation)
379 |
380 | **Pros:** this library lets you apply translations to external apps, without altering their models.
381 |
382 | **Cons:** each translation adds an extra column, which means languages are specified in code, and can't be altered by users later.
383 | * [django-translated-fields](https://github.com/matthiask/django-translated-fields)
384 |
385 | **Pros:** uses a great context processor for switching lanugages (which is where we got the idea for ours).
386 |
387 | **Cons:** Languages are specified in django-settings, and can't be altered by users later.
388 | * [django-parler](https://github.com/django-parler/django-parler)
389 |
390 | **Pros:** Django admin site support.
391 |
392 | **Cons:** Languages are stored in a separate table and can't be altered by users later. Translated fields are specified in model meta, away from the fields definition which makes complex lookups harder.
393 |
394 |
395 |
396 | [term-language-code]: https://docs.djangoproject.com/en/3.1/topics/i18n/#term-language-code
397 | [django-how]: https://docs.djangoproject.com/en/3.1/topics/i18n/translation/#how-django-discovers-language-preference
398 | [dunder-html]: https://code.djangoproject.com/ticket/7261
--------------------------------------------------------------------------------
/dev/Dockerfile:
--------------------------------------------------------------------------------
1 | FROM python:3.12
2 |
3 | # Install python package management tools
4 | RUN pip install --upgrade setuptools pip poetry tox
5 |
6 | COPY ./* /usr/src/app/
7 | WORKDIR /usr/src/app
8 |
9 | RUN poetry config virtualenvs.create false \
10 | && poetry lock \
11 | && poetry install --no-root
12 |
13 | ENV PYTHONPATH=/usr/src/app
14 |
15 |
--------------------------------------------------------------------------------
/dev/README.md:
--------------------------------------------------------------------------------
1 | * Start the dev environment (including Postgres & MariaDB databases) with:
2 |
3 | ```
4 | docker-compose up
5 | ```
6 |
7 | * After you have started the dev stack, start the tests with:
8 |
9 | ```
10 | docker-compose exec dev tox
11 | ```
12 |
13 | * Relock with
14 |
15 | ```
16 | docker-compose run dev poetry relock
17 | ```
18 |
19 | * Testing using script and tox (so you don't have to remember the db urls):
20 |
21 |
22 | -e is the tox environment to run - eg. dj-31, dj-32 or dj-40
23 | -PSM are optional flags to run for Postgres, SQLite, MariaDB
24 | ```
25 | ./dev/local-test.sh -e dj-31 -PSM
26 | ```
27 |
28 | * Reformat the file using black command:
29 |
30 | Run `black .` at the root directory of the project
31 |
32 | * Testing against a specific database:
33 |
34 | DATABASE_URL=postgres://postgres:changeme@postgres_db/postgres tox -e dj-51
--------------------------------------------------------------------------------
/dev/docker-compose.yml:
--------------------------------------------------------------------------------
1 | # This is a very simple docker-compose file
2 | # to make testing different databases easier on a local machine
3 | version: '3'
4 |
5 | services:
6 | maria_db:
7 | image: mariadb:10
8 | environment:
9 | - MYSQL_ROOT_PASSWORD=changeme
10 | postgres_db:
11 | build:
12 | context: ./pg
13 | dockerfile: Dockerfile
14 | environment:
15 | - POSTGRES_PASSWORD=changeme
16 |
17 | dev:
18 | build:
19 | context: ..
20 | dockerfile: ./dev/Dockerfile
21 | environment:
22 | - PYTHONPATH=/usr/src/app:/usr/src/app/tests/
23 | - DJANGO_SETTINGS_MODULE=library_app.settings
24 | # env_file:
25 | # - default.env
26 | # - .env
27 | ports:
28 | - "8001:8001"
29 | depends_on:
30 | # - maria_db
31 | - postgres_db
32 | volumes:
33 | - ..:/usr/src/app
34 | command:
35 | tail -f /dev/null
36 |
--------------------------------------------------------------------------------
/dev/local-test.sh:
--------------------------------------------------------------------------------
1 | #!/bin/bash
2 | # A little script to run on the different dev databases
3 | RUN_PG=0
4 | RUN_SQLITE=0
5 | RUN_MARIA=0
6 |
7 | while getopts e:PSM flag
8 | do
9 | case "${flag}" in
10 | e) TOX_ENV=${OPTARG};;
11 | P) RUN_PG=1;;
12 | S) RUN_SQLITE=1;;
13 | M) RUN_MARIA=1;;
14 | esac
15 | done
16 |
17 | if [ $RUN_SQLITE -eq 1 ]
18 | then
19 | echo "Testing with SQLLite"
20 | tox -e ${TOX_ENV} # Sqlite
21 | fi
22 |
23 | if [ $RUN_MARIA -eq 1 ]
24 | then
25 | echo "Testing with MariaDB"
26 | DATABASE_URL=mysql://root:@maria_db:13306/test tox -e ${TOX_ENV}
27 | fi
28 |
29 | if [ $RUN_PG -eq 1 ]
30 | then
31 | echo "Testing with Postgres"
32 | DATABASE_URL=postgres://postgres:changeme@postgres_db:5432/test tox -e ${TOX_ENV} # Sqlite
33 | fi
34 |
35 |
--------------------------------------------------------------------------------
/dev/pg/Dockerfile:
--------------------------------------------------------------------------------
1 | FROM postgres:13
2 | COPY install-extensions.sql /docker-entrypoint-initdb.d
--------------------------------------------------------------------------------
/dev/pg/install-extensions.sql:
--------------------------------------------------------------------------------
1 | CREATE EXTENSION IF NOT EXISTS pg_trgm;
2 |
--------------------------------------------------------------------------------
/garnett/__init__.py:
--------------------------------------------------------------------------------
1 | default_app_config = "garnett.apps.AppConfig"
2 |
--------------------------------------------------------------------------------
/garnett/apps.py:
--------------------------------------------------------------------------------
1 | from django.apps import AppConfig
2 |
3 |
4 | class AppConfig(AppConfig):
5 | name = "garnett"
6 |
7 | def ready(self):
8 | from django.conf import settings
9 |
10 | if getattr(settings, "GARNETT_PATCH_REVERSION_COMPARE", False):
11 | from garnett.ext.reversion import patch_compare
12 |
13 | patch_compare()
14 |
15 | if getattr(settings, "GARNETT_PATCH_DJANGO_FILTERS", False):
16 | from garnett.ext.filters import patch_filters
17 |
18 | patch_filters()
19 |
--------------------------------------------------------------------------------
/garnett/context.py:
--------------------------------------------------------------------------------
1 | from contextlib import ContextDecorator
2 | import contextvars
3 | from langcodes import Language
4 |
5 | # Internal context var should be set via set_field_language and get via get_current_language
6 | _ctx_language = contextvars.ContextVar("garnett_language")
7 | _ctx_force_blank = contextvars.ContextVar("garnett_language_blank")
8 |
9 |
10 | class set_field_language(ContextDecorator):
11 | def __init__(self, language, force_blank=False):
12 | if isinstance(language, Language):
13 | self.language = language
14 | else:
15 | self.language = Language.get(language)
16 | self.token = None
17 | self.token_blank = None
18 | self.force_blank = force_blank
19 |
20 | def __enter__(self):
21 | self.token = _ctx_language.set(self.language)
22 | self.token_blank = _ctx_force_blank.set(self.force_blank)
23 |
24 | def __exit__(self, exc_type, exc_value, traceback):
25 | _ctx_language.reset(self.token)
26 | _ctx_force_blank.reset(self.token_blank)
27 |
--------------------------------------------------------------------------------
/garnett/context_processors.py:
--------------------------------------------------------------------------------
1 | from .utils import get_languages, get_current_language
2 |
3 |
4 | def languages(request):
5 | return {
6 | "garnett_languages": get_languages(),
7 | "garnett_current_language": get_current_language(),
8 | }
9 |
--------------------------------------------------------------------------------
/garnett/exceptions.py:
--------------------------------------------------------------------------------
1 | class LanguageStructureError(Exception):
2 | """
3 | A language structure isn't a dictionary
4 | """
5 |
6 | pass
7 |
8 |
9 | class LanguageContentError(Exception):
10 | """
11 | A language in a language dictionary has non-string content
12 | """
13 |
14 | pass
15 |
16 |
17 | class LanguageSelectionError(Exception):
18 | """
19 | A language in a language dictionary has non-string content
20 | """
21 |
22 | pass
23 |
--------------------------------------------------------------------------------
/garnett/expressions.py:
--------------------------------------------------------------------------------
1 | from django.db.models import F
2 | from django.db.models.fields.json import KeyTextTransform
3 |
4 | from garnett.utils import get_current_language_code
5 | from garnett.fields import TranslatedField
6 |
7 |
8 | # Based on: https://code.djangoproject.com/ticket/29769#comment:5
9 | # Updated comment here
10 | # https://code.djangoproject.com/ticket/31639
11 | class LangF(F):
12 | def resolve_expression(self, *args, **kwargs):
13 | rhs = super().resolve_expression(*args, **kwargs)
14 | if isinstance(rhs.field, TranslatedField):
15 | field_list = self.name.split("__")
16 | # TODO: should this always lookup lang
17 | if len(field_list) == 1:
18 | # Lookup current lang for one field
19 | field_list.extend([get_current_language_code()])
20 | for name in field_list[1:]:
21 | # Perform key lookups along path
22 | rhs = KeyTextTransform(name, rhs)
23 | return rhs
24 |
25 |
26 | # TODO: should this just inherit from LangF or do we want one without reference lookups
27 | class L(KeyTextTransform):
28 | """Expression to return the current language"""
29 |
30 | def __init__(self, *args, **kwargs):
31 | super().__init__(get_current_language_code(), *args, **kwargs)
32 |
--------------------------------------------------------------------------------
/garnett/ext/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Aristotle-Metadata-Enterprises/django-garnett/70f00f381a74831a5f99e08f01f568fa9fb98ffa/garnett/ext/__init__.py
--------------------------------------------------------------------------------
/garnett/ext/drf.py:
--------------------------------------------------------------------------------
1 | """
2 | Helper methods for working with django-rest-framework
3 | """
4 |
5 | from enum import Enum
6 | from rest_framework.fields import JSONField, empty
7 | from rest_framework import serializers
8 |
9 | from garnett.fields import TranslatedField
10 |
11 |
12 | class LanguageTypes(Enum):
13 | monolingual = 1
14 | multilingual = 2
15 |
16 |
17 | class TranslatableAPIField(JSONField):
18 | def __init__(self, *args, **kwargs):
19 | self.innerfield = kwargs.pop("innerfield", None)
20 | super().__init__(*args, **kwargs)
21 |
22 | @property
23 | def validators(self):
24 | if not hasattr(self, "_validators"):
25 | self._validators = self.get_validators()
26 |
27 | if self.translation_type == LanguageTypes.monolingual:
28 | return self.innerfield.validators
29 | else:
30 | return self._validators
31 |
32 | @validators.setter
33 | def validators(self, validators):
34 | self._validators = validators
35 |
36 | def run_validation(self, data=empty):
37 | (is_empty_value, data) = self.validate_empty_values(data)
38 | if is_empty_value:
39 | return data
40 | value = self.to_internal_value(data)
41 |
42 | if isinstance(data, str):
43 | self.translation_type = LanguageTypes.monolingual
44 | elif isinstance(data, dict):
45 | self.translation_type = LanguageTypes.multilingual
46 | else:
47 | # This branch shouldn't occur, but we'll let it pass.
48 | # This will get caught in the main validators.
49 | self.translation_type = LanguageTypes.multilingual
50 | self.run_validators(value)
51 | return value
52 |
53 |
54 | translatable_serializer_field_mapping = {TranslatedField: TranslatableAPIField}
55 | translatable_serializer_field_mapping.update(
56 | serializers.ModelSerializer.serializer_field_mapping.copy()
57 | )
58 |
59 |
60 | class TranslatableSerializerMixin:
61 | serializer_field_mapping = translatable_serializer_field_mapping
62 |
63 | def build_standard_field(self, field_name, model_field):
64 | field_class, field_kwargs = super().build_standard_field(
65 | field_name, model_field
66 | )
67 | if field_class is TranslatableAPIField:
68 | field_kwargs["innerfield"] = model_field.field
69 | return field_class, field_kwargs
70 |
--------------------------------------------------------------------------------
/garnett/ext/filters.py:
--------------------------------------------------------------------------------
1 | # Patches for django-filters
2 | """
3 | This has been added as django-filters is used by multiple libraries,
4 | including DRF and Djgnao-Graphene.
5 | """
6 |
7 | from django_filters.filterset import CharFilter
8 | from garnett.fields import TranslatedField
9 |
10 |
11 | class TranslatedCharFilter(CharFilter):
12 | pass
13 |
14 |
15 | def patch_filters():
16 | from django_filters.filterset import FILTER_FOR_DBFIELD_DEFAULTS
17 |
18 | FILTER_FOR_DBFIELD_DEFAULTS.update(
19 | {TranslatedField: {"filter_class": TranslatedCharFilter}}
20 | )
21 |
22 | from django_filters.rest_framework.filterset import FILTER_FOR_DBFIELD_DEFAULTS
23 |
24 | FILTER_FOR_DBFIELD_DEFAULTS.update(
25 | {TranslatedField: {"filter_class": TranslatedCharFilter}}
26 | )
27 |
--------------------------------------------------------------------------------
/garnett/ext/reversion.py:
--------------------------------------------------------------------------------
1 | from garnett.fields import TranslatedField
2 |
3 |
4 | def patch_compare():
5 | from reversion_compare.compare import CompareObjects as COBase
6 |
7 | class CompareObjects(COBase):
8 | def __init__(self, field, field_name, obj, version1, version2, is_reversed):
9 | if isinstance(field, TranslatedField):
10 | field_name = field.attname
11 | super().__init__(field, field_name, obj, version1, version2, is_reversed)
12 |
13 | # Import this as late as possible as we are monkey patching over it
14 | import reversion_compare.compare
15 |
16 | reversion_compare.compare.CompareObjects = CompareObjects
17 |
--------------------------------------------------------------------------------
/garnett/fields.py:
--------------------------------------------------------------------------------
1 | from django.conf import settings
2 | from django.core import exceptions
3 | from django.db.models import JSONField
4 | from django.db.models.fields.json import KeyTransform
5 | from dataclasses import make_dataclass
6 | from functools import partial
7 | import logging
8 | from typing import Callable, Dict, Union
9 |
10 | from garnett.translatedstr import TranslatedStr, VerboseTranslatedStr
11 | from garnett.utils import (
12 | get_current_language_code,
13 | get_property_name,
14 | get_languages,
15 | is_valid_language,
16 | normalise_language_codes,
17 | )
18 |
19 | logger = logging.getLogger(__name__)
20 |
21 |
22 | def innerfield_validator_factory(innerfield) -> callable:
23 | def validator(values: dict):
24 | from garnett import exceptions as e
25 |
26 | if not isinstance(values, dict):
27 | raise e.LanguageStructureError(
28 | "Invalid value assigned to translatable field"
29 | )
30 |
31 | # Run validators on sub field
32 | errors = []
33 | for code, value in values.items():
34 | # Check language codes
35 | if not isinstance(value, str):
36 | raise exceptions.ValidationError(f'Invalid value for language "{code}"')
37 | if not is_valid_language(code):
38 | raise exceptions.ValidationError(
39 | f'"{code}" is not a valid language code'
40 | )
41 |
42 | for v in innerfield.validators:
43 | try:
44 | v(value)
45 | except exceptions.ValidationError as e:
46 | errors.extend(e.error_list)
47 | if errors:
48 | raise exceptions.ValidationError(errors)
49 |
50 | return validator
51 |
52 |
53 | def translatable_default(
54 | inner_default: Union[str, Callable[[], str]]
55 | ) -> Dict[str, str]:
56 | """Return default from inner field as dict with current language"""
57 | lang = get_current_language_code()
58 | if callable(inner_default):
59 | return {lang: inner_default()}
60 |
61 | return {lang: inner_default}
62 |
63 |
64 | class TranslatedField(JSONField):
65 | """Translated text field that mirrors the behaviour of another text field
66 |
67 | All arguments except fallback can be provided on the inner field
68 | """
69 |
70 | def __init__(self, field, *args, fallback=None, **kwargs):
71 | self.field = field
72 | self._fallback = fallback
73 |
74 | if type(fallback) is type and issubclass(fallback, TranslatedStr):
75 | self.fallback = fallback
76 | elif callable(fallback):
77 | self.fallback = partial(TranslatedStr, fallback=fallback)
78 | else:
79 | self.fallback = VerboseTranslatedStr
80 |
81 | # Move some args to outer field
82 | outer_args = [
83 | "db_column",
84 | "db_index",
85 | "db_tablespace",
86 | "help_text",
87 | "verbose_name",
88 | ]
89 | inner_kwargs = self.field.deconstruct()[3]
90 | for arg_name in outer_args:
91 | if arg_name in inner_kwargs:
92 | kwargs[arg_name] = inner_kwargs[arg_name]
93 |
94 | # Create default for outer field based on inner field
95 | if "default" not in kwargs and "default" in inner_kwargs:
96 | # Use partial because it is serializable in django migrations
97 | kwargs["default"] = partial(translatable_default, inner_kwargs["default"])
98 |
99 | super().__init__(*args, **kwargs)
100 | self.validators.append(innerfield_validator_factory(self.field))
101 |
102 | def formfield(self, **kwargs):
103 | # We need to bypass the JSONField implementation
104 | return self.field.formfield(**kwargs)
105 |
106 | def get_prep_value(self, value):
107 | if hasattr(value, "items"):
108 | value = {
109 | lang_code: self.field.get_prep_value(text)
110 | for lang_code, text in value.items()
111 | }
112 | elif type(value) == str:
113 | value = self.field.get_prep_value(value)
114 | return value
115 | return super().get_prep_value(value)
116 |
117 | def get_db_prep_save(self, value, connection):
118 | """This ensures that any custom get_db_prep_save() method in self.field can be triggered"""
119 | if hasattr(value, "items"):
120 | value = {
121 | lang_code: self.field.get_db_prep_save(text, connection)
122 | for lang_code, text in value.items()
123 | }
124 | elif type(value) == str:
125 | value = self.field.get_db_prep_save(value, connection)
126 | return value
127 | return super().get_db_prep_save(value, connection)
128 |
129 | def from_db_value(self, value, expression, connection):
130 | value = super().from_db_value(value, expression, connection)
131 | if hasattr(self.field, "from_db_value"):
132 | value = {
133 | k: self.field.from_db_value(v, expression, connection)
134 | for k, v in value.items()
135 | }
136 | return value
137 |
138 | def value_from_object(self, obj):
139 | """Return the value of this field in the given model instance."""
140 | all_ts = getattr(obj, self.ts_name)
141 | if type(all_ts) is not dict:
142 | logger.warning(
143 | "Displaying an untranslatable field - model:{} (pk:{}), field:{}".format(
144 | type(obj), obj.pk, self.name
145 | )
146 | )
147 | return str(all_ts)
148 |
149 | language = get_current_language_code()
150 | return all_ts.get(language, None)
151 |
152 | def translations_from_object(self, obj):
153 | """Return the value of this field in the given model instance."""
154 | return getattr(obj, self.ts_name)
155 |
156 | def get_attname(self):
157 | # Use field with _tsall as the attribute name on the object
158 | # We've overrided this as this is usually the name of the attribute used to populate the database
159 | return self.name + "_tsall"
160 |
161 | def get_attname_column(self):
162 | attname = self.get_attname()
163 | # Use name without _tsall as the column name
164 | column = self.db_column or self.name
165 | return attname, column
166 |
167 | @property
168 | def ts_name(self):
169 | return f"{self.name}_tsall"
170 |
171 | def contribute_to_class(self, cls, name, private_only=False):
172 | super().contribute_to_class(cls, name, private_only)
173 |
174 | # We use `ego` to differentiate scope here as this is the inner self
175 | # Maybe its not necessary, but it is funny.
176 |
177 | @property
178 | def translator(ego):
179 | return self.fallback(getattr(ego, self.ts_name))
180 |
181 | @translator.setter
182 | def translator(ego, value):
183 | """Setter for main field (without _tsall)"""
184 | all_ts = getattr(ego, self.ts_name)
185 | if not all_ts:
186 | # This is probably the first save through
187 | all_ts = {}
188 | elif type(all_ts) is not dict:
189 | logger.warning(
190 | "Saving a broken field - model:{} (pk:{}), field:{}".format(
191 | type(ego), ego.pk, name
192 | )
193 | )
194 | logger.debug("Field data was - {}".format(all_ts))
195 | all_ts = {}
196 |
197 | if isinstance(value, str):
198 | language_code = get_current_language_code()
199 | all_ts[language_code] = value
200 | elif value is None:
201 | language_code = get_current_language_code()
202 | all_ts[language_code] = ""
203 | elif isinstance(value, dict):
204 | # normalise all language codes
205 | all_ts = normalise_language_codes(value)
206 | else:
207 | bad_type = type(value)
208 | raise TypeError(
209 | f"Invalid type assigned to translatable field - {bad_type}"
210 | )
211 |
212 | setattr(ego, self.ts_name, all_ts)
213 |
214 | setattr(cls, f"{name}", translator)
215 |
216 | # This can probably be cached on the class
217 | @property
218 | def translatable_fields(ego):
219 | return [
220 | field
221 | for field in ego._meta.get_fields()
222 | if isinstance(field, TranslatedField)
223 | ]
224 |
225 | propname = getattr(
226 | settings, "GARNETT_TRANSLATABLE_FIELDS_PROPERTY_NAME", "translatable_fields"
227 | )
228 | setattr(cls, propname, translatable_fields)
229 |
230 | @property
231 | def translations(ego):
232 | translatable_fields = ego.translatable_fields
233 | Translations = make_dataclass(
234 | "Translations", [(f.name, dict) for f in translatable_fields]
235 | )
236 | kwargs = {}
237 | for field in translatable_fields:
238 | kwargs[field.name] = getattr(ego, field.ts_name)
239 | return Translations(**kwargs)
240 |
241 | setattr(cls, get_property_name(), translations)
242 |
243 | @property
244 | def available_languages(ego):
245 | """Returns a list of codes available on the whole model"""
246 | langs = set()
247 | for field in ego.translatable_fields:
248 | langs |= getattr(ego, field.ts_name, {}).keys()
249 | return [lang for lang in get_languages() if lang.to_tag() in langs]
250 |
251 | setattr(cls, "available_languages", available_languages)
252 |
253 | def get_transform(self, name):
254 | # Call back to the Field get_transform
255 | transform = super(JSONField, self).get_transform(name)
256 | if transform:
257 | return transform
258 | # Use our new factory
259 | return TranslatedKeyTransformFactory(name)
260 |
261 | def deconstruct(self):
262 | name, path, args, kwargs = super().deconstruct()
263 | args.insert(0, self.field)
264 | kwargs["fallback"] = self._fallback
265 | return name, path, args, kwargs
266 |
267 |
268 | class TranslatedKeyTransform(KeyTransform):
269 | """Key transform for translate fields
270 |
271 | so we can register lookups on this without affecting the regular json field
272 | """
273 |
274 |
275 | class TranslatedKeyTransformFactory:
276 | def __init__(self, key_name):
277 | self.key_name = key_name
278 |
279 | def __call__(self, *args, **kwargs):
280 | return TranslatedKeyTransform(self.key_name, *args, **kwargs)
281 |
282 |
283 | # Shorter name for the class
284 | Translated = TranslatedField
285 |
286 |
287 | # Import lookups here so that they are registered by just importing the field
288 | from garnett import lookups # noqa: F401, E402
289 |
--------------------------------------------------------------------------------
/garnett/lookups.py:
--------------------------------------------------------------------------------
1 | from django.db.models import lookups
2 | from django.db.models.fields import json, CharField
3 | from django.db.models.functions import Cast
4 | from django.db.models.fields.json import KeyTextTransform
5 | from django.contrib.postgres.lookups import SearchLookup, TrigramSimilar
6 | from django.contrib.postgres.search import TrigramSimilarity
7 |
8 | from garnett.fields import TranslatedField, TranslatedKeyTransform
9 | from garnett.utils import get_current_language
10 |
11 |
12 | # We duplicate and process_lhs and process_rhs to be certain these are
13 | # Actually called during tests.
14 | # Otherwise, the blank classes appear to give 100% coverage
15 |
16 |
17 | @TranslatedField.register_lookup
18 | class HasLang(json.HasKey):
19 | lookup_name = "has_lang"
20 |
21 | def process_lhs(self, compiler, connection):
22 | return super().process_lhs(compiler, connection)
23 |
24 | def process_rhs(self, compiler, connection):
25 | return super().process_rhs(compiler, connection)
26 |
27 |
28 | @TranslatedField.register_lookup
29 | class HasLangs(json.HasKeys):
30 | lookup_name = "has_langs"
31 |
32 | def process_lhs(self, compiler, connection):
33 | return super().process_lhs(compiler, connection)
34 |
35 | def process_rhs(self, compiler, connection):
36 | return super().process_rhs(compiler, connection)
37 |
38 |
39 | @TranslatedField.register_lookup
40 | class HasAnyLangs(json.HasAnyKeys):
41 | lookup_name = "has_any_langs"
42 |
43 | def process_lhs(self, compiler, connection):
44 | return super().process_lhs(compiler, connection)
45 |
46 | def process_rhs(self, compiler, connection):
47 | return super().process_rhs(compiler, connection)
48 |
49 |
50 | # Override default lookups on our field to handle language lookups
51 |
52 |
53 | class CurrentLanguageMixin:
54 | """Mixin to perform language lookup on lhs"""
55 |
56 | def __init__(self, lhs, *args, **kwargs):
57 | tlhs = json.KeyTransform(
58 | str(get_current_language()),
59 | lhs,
60 | )
61 | super().__init__(tlhs, *args, **kwargs)
62 |
63 |
64 | @TranslatedField.register_lookup
65 | class BaseLanguageExact(
66 | CurrentLanguageMixin, json.KeyTransformTextLookupMixin, lookups.Exact
67 | ):
68 | # Note: On some database engines lookup_name actually has an effect on the result
69 | # (See lookup_cast in the django postgres backend)
70 | lookup_name = "exact"
71 | prepare_rhs = False
72 |
73 | def process_lhs(self, compiler, connection):
74 | return super().process_lhs(compiler, connection)
75 |
76 | def process_rhs(self, compiler, connection):
77 | return super().process_rhs(compiler, connection)
78 |
79 |
80 | @TranslatedField.register_lookup
81 | class BaseLanguageIExact(CurrentLanguageMixin, json.KeyTransformIExact):
82 | lookup_name = "iexact"
83 |
84 | def process_lhs(self, compiler, connection):
85 | return super().process_lhs(compiler, connection)
86 |
87 | def process_rhs(self, compiler, connection):
88 | return super().process_rhs(compiler, connection)
89 |
90 |
91 | @TranslatedField.register_lookup
92 | class BaseLanguageIContains(CurrentLanguageMixin, json.KeyTransformIContains):
93 | def process_lhs(self, compiler, connection):
94 | return super().process_lhs(compiler, connection)
95 |
96 | def process_rhs(self, compiler, connection):
97 | return super().process_rhs(compiler, connection)
98 |
99 |
100 | @TranslatedField.register_lookup
101 | class BaseLanguageContains(
102 | CurrentLanguageMixin, json.KeyTransformTextLookupMixin, lookups.Contains
103 | ):
104 | # Override the default json field contains which is not a text contains
105 | # https://docs.djangoproject.com/en/3.1/topics/db/queries/#contains
106 | lookup_name = "contains"
107 |
108 | def process_lhs(self, compiler, connection):
109 | return super().process_lhs(compiler, connection)
110 |
111 | def process_rhs(self, compiler, connection):
112 | return super().process_rhs(compiler, connection)
113 |
114 |
115 | # Override contains lookup for after a key lookup i.e. title__en__contains="thing"
116 | @TranslatedKeyTransform.register_lookup
117 | class KeyTransformContains(json.KeyTransformTextLookupMixin, lookups.Contains):
118 | lookup_name = "contains"
119 |
120 | def process_lhs(self, compiler, connection):
121 | return super().process_lhs(compiler, connection)
122 |
123 | def process_rhs(self, compiler, connection):
124 | return super().process_rhs(compiler, connection)
125 |
126 |
127 | @TranslatedKeyTransform.register_lookup
128 | class KeyTransformExact(json.KeyTransformTextLookupMixin, lookups.Exact):
129 | def process_lhs(self, compiler, connection):
130 | self.lhs = Cast(self.lhs, CharField())
131 | return super().process_lhs(compiler, connection)
132 |
133 | def process_rhs(self, compiler, connection):
134 | return super().process_rhs(compiler, connection)
135 |
136 |
137 | @TranslatedKeyTransform.register_lookup
138 | class KeyTransformGreaterThan(json.KeyTransformTextLookupMixin, lookups.GreaterThan):
139 | def process_lhs(self, compiler, connection):
140 | self.lhs = Cast(self.lhs, CharField())
141 | return super().process_lhs(compiler, connection)
142 |
143 | def process_rhs(self, compiler, connection):
144 | return super().process_rhs(compiler, connection)
145 |
146 |
147 | @TranslatedKeyTransform.register_lookup
148 | class KeyTransformGreaterThanOrEqual(
149 | json.KeyTransformTextLookupMixin, lookups.GreaterThanOrEqual
150 | ):
151 | def process_lhs(self, compiler, connection):
152 | self.lhs = Cast(self.lhs, CharField())
153 | return super().process_lhs(compiler, connection)
154 |
155 | def process_rhs(self, compiler, connection):
156 | return super().process_rhs(compiler, connection)
157 |
158 |
159 | @TranslatedKeyTransform.register_lookup
160 | class KeyTransformLessThan(json.KeyTransformTextLookupMixin, lookups.LessThan):
161 | def process_lhs(self, compiler, connection):
162 | self.lhs = Cast(self.lhs, CharField())
163 | return super().process_lhs(compiler, connection)
164 |
165 | def process_rhs(self, compiler, connection):
166 | return super().process_rhs(compiler, connection)
167 |
168 |
169 | @TranslatedKeyTransform.register_lookup
170 | class KeyTransformLesshanOrEqual(
171 | json.KeyTransformTextLookupMixin, lookups.LessThanOrEqual
172 | ):
173 | def process_lhs(self, compiler, connection):
174 | self.lhs = Cast(self.lhs, CharField())
175 | return super().process_lhs(compiler, connection)
176 |
177 | def process_rhs(self, compiler, connection):
178 | return super().process_rhs(compiler, connection)
179 |
180 |
181 | @TranslatedField.register_lookup
182 | class BaseLanguageStartsWith(CurrentLanguageMixin, json.KeyTransformStartsWith):
183 | def process_lhs(self, compiler, connection):
184 | return super().process_lhs(compiler, connection)
185 |
186 | def process_rhs(self, compiler, connection):
187 | return super().process_rhs(compiler, connection)
188 |
189 |
190 | @TranslatedField.register_lookup
191 | class BaseLanguageIStartsWith(CurrentLanguageMixin, json.KeyTransformIStartsWith):
192 | def process_lhs(self, compiler, connection):
193 | return super().process_lhs(compiler, connection)
194 |
195 | def process_rhs(self, compiler, connection):
196 | return super().process_rhs(compiler, connection)
197 |
198 |
199 | @TranslatedField.register_lookup
200 | class BaseLanguageEndsWith(CurrentLanguageMixin, json.KeyTransformEndsWith):
201 | def process_lhs(self, compiler, connection):
202 | return super().process_lhs(compiler, connection)
203 |
204 | def process_rhs(self, compiler, connection):
205 | return super().process_rhs(compiler, connection)
206 |
207 |
208 | @TranslatedField.register_lookup
209 | class BaseLanguageIEndsWith(CurrentLanguageMixin, json.KeyTransformIEndsWith):
210 | def process_lhs(self, compiler, connection):
211 | return super().process_lhs(compiler, connection)
212 |
213 | def process_rhs(self, compiler, connection):
214 | return super().process_rhs(compiler, connection)
215 |
216 |
217 | @TranslatedField.register_lookup
218 | class BaseLanguageRegex(CurrentLanguageMixin, json.KeyTransformRegex):
219 | def process_lhs(self, compiler, connection):
220 | return super().process_lhs(compiler, connection)
221 |
222 | def process_rhs(self, compiler, connection):
223 | return super().process_rhs(compiler, connection)
224 |
225 |
226 | @TranslatedField.register_lookup
227 | class BaseLanguageIRegex(CurrentLanguageMixin, json.KeyTransformIRegex):
228 | def process_lhs(self, compiler, connection):
229 | return super().process_lhs(compiler, connection)
230 |
231 | def process_rhs(self, compiler, connection):
232 | return super().process_rhs(compiler, connection)
233 |
234 |
235 | @TranslatedField.register_lookup
236 | class BaseLanguageGreaterThan(
237 | CurrentLanguageMixin, json.KeyTransformTextLookupMixin, lookups.GreaterThan
238 | ):
239 | def process_lhs(self, compiler, connection):
240 | self.lhs = Cast(self.lhs, CharField())
241 | return super().process_lhs(compiler, connection)
242 |
243 | def process_rhs(self, compiler, connection):
244 | return super().process_rhs(compiler, connection)
245 |
246 |
247 | @TranslatedField.register_lookup
248 | class BaseLanguageGreaterThanOrEqual(
249 | CurrentLanguageMixin, json.KeyTransformTextLookupMixin, lookups.GreaterThanOrEqual
250 | ):
251 | def process_lhs(self, compiler, connection):
252 | self.lhs = Cast(self.lhs, CharField())
253 | return super().process_lhs(compiler, connection)
254 |
255 | def process_rhs(self, compiler, connection):
256 | return super().process_rhs(compiler, connection)
257 |
258 |
259 | @TranslatedField.register_lookup
260 | class BaseLanguageLessThan(
261 | CurrentLanguageMixin, json.KeyTransformTextLookupMixin, lookups.LessThan
262 | ):
263 | def process_lhs(self, compiler, connection):
264 | self.lhs = Cast(self.lhs, CharField())
265 | return super().process_lhs(compiler, connection)
266 |
267 | def process_rhs(self, compiler, connection):
268 | return super().process_rhs(compiler, connection)
269 |
270 |
271 | @TranslatedField.register_lookup
272 | class BaseLanguageLessThanOrEqual(
273 | CurrentLanguageMixin, json.KeyTransformTextLookupMixin, lookups.LessThanOrEqual
274 | ):
275 | def process_lhs(self, compiler, connection):
276 | self.lhs = Cast(self.lhs, CharField())
277 | return super().process_lhs(compiler, connection)
278 |
279 | def process_rhs(self, compiler, connection):
280 | return super().process_rhs(compiler, connection)
281 |
282 |
283 | @TranslatedField.register_lookup
284 | class BaseLanguageSearch(
285 | CurrentLanguageMixin, json.KeyTransformTextLookupMixin, SearchLookup
286 | ):
287 | def process_lhs(self, compiler, connection):
288 | return super().process_lhs(compiler, connection)
289 |
290 | def process_rhs(self, compiler, connection):
291 | return super().process_rhs(compiler, connection)
292 |
293 |
294 | # --- Postgres only functions ---
295 |
296 |
297 | @TranslatedField.register_lookup
298 | class BaseLanguageTrigramSimilar(
299 | CurrentLanguageMixin, json.KeyTransformTextLookupMixin, TrigramSimilar
300 | ):
301 | def process_lhs(self, compiler, connection):
302 | return super().process_lhs(compiler, connection)
303 |
304 | def process_rhs(self, compiler, connection):
305 | return super().process_rhs(compiler, connection)
306 |
307 |
308 | class LangTrigramSimilarity(TrigramSimilarity):
309 | # There is no way we can calculate if a field is a language or not
310 | # So we have to write our own function here
311 | # We also need to cast the JSONB field to a string
312 | def __init__(self, expression, string, **extra):
313 | lang = str(get_current_language())
314 | expression = KeyTextTransform(lang, expression)
315 | super().__init__(expression, string, **extra)
316 |
--------------------------------------------------------------------------------
/garnett/middleware.py:
--------------------------------------------------------------------------------
1 | from django.conf import settings
2 | from django.http import Http404
3 | from django.utils.module_loading import import_string
4 | from django.utils.translation import gettext as _
5 |
6 | import logging
7 | import langcodes
8 | from langcodes import Language
9 |
10 | from .utils import is_valid_language, get_default_language
11 | from .context import set_field_language
12 |
13 | logger = logging.getLogger(__name__)
14 |
15 |
16 | class TranslationContextMiddleware:
17 | """
18 | This middleware catches the requested "garnett language" and:
19 | * sets a garnett language attribute on the request
20 | * defines a context variable that is used when reading or altering fields
21 | """
22 |
23 | def __init__(self, get_response):
24 | self.get_response = get_response
25 |
26 | def validate(self, language):
27 | """Validate the language raising http errors if invalid"""
28 | return None
29 |
30 | def __call__(self, request):
31 | request.garnett_language = get_language_from_request(request)
32 | request.garnett_fallback_blank = False
33 | if getattr(settings, "GARNETT_ALLOW_BLANK_FALLBACK_OVERRIDE", False):
34 | request.garnett_fallback_blank = bool(request.GET.get("gblank", False))
35 | elif paths := getattr(
36 | settings, "GARNETT_FORCE_BLANK_FALLBACK_OVERRIDE_PATHS", []
37 | ):
38 | for path in paths:
39 | if request.path.startswith(path):
40 | request.garnett_fallback_blank = True
41 | break
42 |
43 | self.validate(request.garnett_language)
44 | with set_field_language(
45 | request.garnett_language, force_blank=request.garnett_fallback_blank
46 | ):
47 | response = self.get_response(request)
48 | return response
49 |
50 |
51 | class TranslationContextNotFoundMiddleware(TranslationContextMiddleware):
52 | """
53 | This middleware catches the requested "garnett language" and:
54 | * sets a garnett language attribute on the request
55 | * defines a context variable that is used when reading or altering fields
56 | * will raise a 404 if the request language is not in languages list
57 | """
58 |
59 | def validate(self, language):
60 | if not is_valid_language(language):
61 | lang_obj = language
62 | lang_name = lang_obj.display_name(language)
63 | lang_en_name = lang_obj.display_name()
64 | raise Http404(
65 | _("This server does not support %(lang_name)s" " [%(lang_en_name)s].")
66 | % {
67 | "lang_name": lang_name,
68 | "lang_en_name": lang_en_name,
69 | }
70 | )
71 |
72 |
73 | class TranslationCacheMiddleware:
74 | """Middleware to cache the garnett language in the users session storage
75 |
76 | This must be after one of the above middlewares and after the session middleware
77 | """
78 |
79 | def __init__(self, get_response):
80 | self.get_response = get_response
81 |
82 | def __call__(self, request):
83 | if hasattr(request, "garnett_language") and hasattr(request, "session"):
84 | request.session["GARNETT_LANGUAGE_CODE"] = request.garnett_language
85 | else:
86 | logger.error(
87 | "TranslationCacheMiddleware must come after main garnett middleware "
88 | "and the session middleware."
89 | )
90 |
91 | return self.get_response(request)
92 |
93 |
94 | def get_language_from_request(request) -> Language:
95 | opt_order = getattr(
96 | settings,
97 | "GARNETT_REQUEST_LANGUAGE_SELECTORS",
98 | [
99 | "garnett.selectors.header",
100 | "garnett.selectors.query",
101 | "garnett.selectors.cookie",
102 | ],
103 | )
104 | for opt in opt_order:
105 | func = import_string(opt)
106 | if lang := func(request):
107 | try:
108 | return Language.get(lang)
109 | except langcodes.tag_parser.LanguageTagError:
110 | raise Http404(
111 | _(
112 | "The provided language %(lang_code)s is not a valid language code"
113 | )
114 | % {
115 | "lang_code": lang,
116 | }
117 | )
118 |
119 | return get_default_language()
120 |
--------------------------------------------------------------------------------
/garnett/migrate.py:
--------------------------------------------------------------------------------
1 | import base64
2 | import json
3 | import pickle
4 | from django.apps.registry import Apps
5 | from django.db.migrations import RunPython
6 | from django.db.backends.base.schema import BaseDatabaseSchemaEditor
7 | from garnett.utils import get_current_language_code
8 | from typing import Callable, Dict, List
9 |
10 |
11 | """
12 | These methods convert a string to and from a safe base64 encoded version of text.
13 | This helps with database encoding
14 | """
15 |
16 |
17 | def safe_encode(value: str) -> str:
18 | return base64.b64encode(pickle.dumps(value)).decode("ascii")
19 |
20 |
21 | def safe_decode(value: str) -> str:
22 | return pickle.loads(base64.urlsafe_b64decode(value))
23 |
24 |
25 | def _get_migrate_function(
26 | app_label: str,
27 | model_fields: Dict[str, List[str]],
28 | update: Callable[[str, str], str],
29 | ts_all: bool = False,
30 | ) -> Callable[[Apps, BaseDatabaseSchemaEditor], None]:
31 | """Generate a migration function given an update function for each value
32 |
33 | update is a function taking current language and old value and returning a new value
34 | """
35 |
36 | def migrate(apps: Apps, schema_editor: BaseDatabaseSchemaEditor) -> None:
37 | current_lang = get_current_language_code()
38 |
39 | for model_name, fields in model_fields.items():
40 | updated = []
41 | model = apps.get_model(app_label, model_name)
42 | for item in model.objects.all():
43 | for field_name in fields:
44 | # Set new value retrieved from update function
45 | if ts_all:
46 | field_name = f"{field_name}_tsall"
47 | value = getattr(item, field_name)
48 | setattr(item, field_name, update(current_lang, value))
49 | updated.append(item)
50 |
51 | # Bulk update only the required fields
52 | model.objects.bulk_update(updated, fields)
53 |
54 | return migrate
55 |
56 |
57 | # Part 1
58 |
59 |
60 | def update_safe_encode_content_forwards(current_lang: str, value: str) -> str:
61 | value = safe_encode(value)
62 | return json.dumps({current_lang: value})
63 |
64 |
65 | def update_safe_encode_content_backwards(current_lang: str, value: str) -> str:
66 | value = json.loads(value)
67 | return safe_decode(value.get(current_lang, safe_encode("")))
68 |
69 |
70 | def step_1_safe_encode_content(
71 | app_label: str, model_fields: Dict[str, List[str]]
72 | ) -> RunPython:
73 | """Generate a migration operation to safely store text content prior to field migration
74 | This is needed as standard json is not escaped according to Postgres (and maybe other database) requirements.
75 | For example, Postgres requires single quotes (') to be double escaped: ('')
76 | For more info see here: https://stackoverflow.com/questions/35677204/psql-insert-json-with-double-quotes-inside-strings
77 |
78 | Args:
79 | app_label: label for django app the migration is in
80 | model_fields: mapping of model name to list of text fields to migrate
81 |
82 | Returns:
83 | RunPython migration operation
84 | """
85 |
86 | return RunPython(
87 | code=_get_migrate_function(
88 | app_label, model_fields, update_safe_encode_content_forwards
89 | ),
90 | reverse_code=_get_migrate_function(
91 | app_label, model_fields, update_safe_encode_content_backwards, ts_all=True
92 | ),
93 | )
94 |
95 |
96 | # Part 2
97 |
98 |
99 | def update_safe_prepare_translations_forwards(current_lang: str, value: dict) -> dict:
100 | value = safe_decode(value.get(current_lang, safe_encode("")))
101 | return {current_lang: value}
102 |
103 |
104 | def update_safe_prepare_translations_backwards(current_lang: str, value: dict) -> dict:
105 | value = safe_encode(value.get(current_lang, ""))
106 | return {current_lang: value}
107 |
108 |
109 | def step_2_safe_prepare_translations(
110 | app_label: str, model_fields: Dict[str, List[str]]
111 | ) -> RunPython:
112 | """Generate a migration operation to prepare text fields for being translated
113 |
114 | Args:
115 | app_label: label for django app the migration is in
116 | model_fields: mapping of model name to list of text fields to migrate
117 |
118 | Returns:
119 | RunPython migration operation
120 | """
121 |
122 | return RunPython(
123 | code=_get_migrate_function(
124 | app_label,
125 | model_fields,
126 | update_safe_prepare_translations_forwards,
127 | ts_all=True,
128 | ),
129 | reverse_code=_get_migrate_function(
130 | app_label,
131 | model_fields,
132 | update_safe_prepare_translations_backwards,
133 | ts_all=True,
134 | ),
135 | )
136 |
--------------------------------------------------------------------------------
/garnett/mixins.py:
--------------------------------------------------------------------------------
1 | from django.db.models.query import BaseIterable
2 | from garnett.expressions import L
3 |
4 | PREFIX = "L_garnett__"
5 |
6 |
7 | class TranslatableValuesIterable(BaseIterable):
8 | """
9 | Unwind the modifications that are made before calling .values()
10 | Iterable returned by QuerySet.values() that yields a dict for each row.
11 | """
12 |
13 | def clean_garnett_field(self, field_name) -> str:
14 | """Return the field name minus the prefix"""
15 | return field_name.replace(PREFIX, "")
16 |
17 | def __iter__(self):
18 | queryset = self.queryset
19 | query = queryset.query
20 | compiler = query.get_compiler(queryset.db)
21 |
22 | # extra(select=...) cols are always at the start of the row.
23 | names = [
24 | *query.extra_select,
25 | *query.values_select,
26 | *query.annotation_select,
27 | ]
28 | indexes = range(len(names))
29 | for row in compiler.results_iter(
30 | chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size
31 | ):
32 | yield {self.clean_garnett_field(names[i]): row[i] for i in indexes}
33 |
34 |
35 | class TranslatedQuerySetMixin:
36 | """
37 | A translated QuerySet mixin to add extra functionality to translated fields
38 | Must be mixedin to a QuerySet
39 | """
40 |
41 | def values(self, *fields, **expressions):
42 | """
43 | .values() for translatable fields
44 | Still expects values to be passed with L()
45 | """
46 |
47 | # Convert anything that is an L from a field to an expression - so it treats it as an expression
48 | # rather than a field.
49 | # We will clean the field prefix in our custom iterable class "TranslatableQuerySetMixin"
50 | cleaned_fields = []
51 | for field in fields:
52 | if isinstance(field, L):
53 | expressions.update(
54 | {f"{PREFIX}{field.source_expressions[0].name}": field}
55 | )
56 | else:
57 | cleaned_fields.append(field)
58 |
59 | clone = super().values(*cleaned_fields, **expressions)
60 | clone._iterable_class = TranslatableValuesIterable
61 |
62 | return clone
63 |
--------------------------------------------------------------------------------
/garnett/patch.py:
--------------------------------------------------------------------------------
1 | from contextlib import ContextDecorator
2 | from django.core.exceptions import FieldError
3 | from django.db.models.sql import query
4 | from dataclasses import dataclass
5 | from typing import Any
6 | import functools
7 |
8 | from garnett.fields import TranslatedField
9 | from garnett.utils import get_current_language
10 |
11 |
12 | # Save current join info
13 | _JoinInfo = query.JoinInfo
14 |
15 |
16 | @dataclass
17 | class JoinInfo:
18 | final_field: Any
19 | targets: Any
20 | opts: Any
21 | joins: Any
22 | path: Any
23 | transform_function_func: Any
24 |
25 | @property
26 | def transform_function(self):
27 | if isinstance(self.final_field, TranslatedField):
28 | # If it's a partial, it must have had a transformer applied - leave it alone!
29 | if isinstance(self.transform_function_func, functools.partial):
30 | return self.transform_function_func
31 |
32 | name = get_current_language()
33 |
34 | # Cloned in from django
35 | def transform(field, alias, *, name, previous):
36 | try:
37 | wrapped = previous(field, alias)
38 | return self.try_transform(wrapped, name)
39 | except FieldError:
40 | # TODO: figure out how to handle this case as we don't have
41 | # final_field or last_field_exception
42 |
43 | # FieldError is raised if the transform doesn't exist.
44 | # if isinstance(final_field, Field) and last_field_exception:
45 | # raise last_field_exception
46 | # else:
47 | # raise
48 | raise
49 |
50 | # -------------------
51 |
52 | return functools.partial(
53 | transform, name=name, previous=self.transform_function_func
54 | )
55 |
56 | return self.transform_function_func
57 |
58 | def try_transform(self, lhs, name):
59 | # Cloned in from django
60 | import difflib
61 |
62 | """
63 | Helper method for build_lookup(). Try to fetch and initialize
64 | a transform for name parameter from lhs.
65 | """
66 | transform_class = lhs.get_transform(name)
67 | if transform_class:
68 | return transform_class(lhs)
69 | else:
70 | output_field = lhs.output_field.__class__
71 | suggested_lookups = difflib.get_close_matches(
72 | name, output_field.get_lookups()
73 | )
74 | if suggested_lookups:
75 | suggestion = ", perhaps you meant %s?" % " or ".join(suggested_lookups)
76 | else:
77 | suggestion = "."
78 | raise FieldError(
79 | "Unsupported lookup '%s' for %s or join on the field not "
80 | "permitted%s" % (name, output_field.__name__, suggestion)
81 | )
82 |
83 | def __iter__(self):
84 | # Necessary to mimic a tuple
85 | for x in [
86 | self.final_field,
87 | self.targets,
88 | self.opts,
89 | self.joins,
90 | self.path,
91 | self.transform_function,
92 | ]:
93 | yield x
94 |
95 |
96 | def apply_patches():
97 | """Apply monkey patches to django"""
98 | # This is needed to allow values/values_list and F lookups to work
99 | # The most dangerous monkey patch of all time
100 | query.JoinInfo = JoinInfo
101 |
102 |
103 | def revert_patches():
104 | """Revert monkey patches to django"""
105 | query.JoinInfo = _JoinInfo
106 |
107 |
108 | class patch_lookups(ContextDecorator):
109 | def __enter__(self):
110 | apply_patches()
111 |
112 | def __exit__(self, exc_type, exc_value, traceback):
113 | revert_patches()
114 |
--------------------------------------------------------------------------------
/garnett/selectors.py:
--------------------------------------------------------------------------------
1 | from django.utils.translation import get_language
2 |
3 | from garnett.utils import lang_param
4 |
5 |
6 | def query(request):
7 | return request.GET.get(lang_param(), None)
8 |
9 |
10 | def cookie(request):
11 | return request.COOKIES.get("GARNETT_LANGUAGE_CODE", None)
12 |
13 |
14 | def session(request):
15 | return request.session.get("GARNETT_LANGUAGE_CODE", None)
16 |
17 |
18 | def header(request):
19 | return request.META.get("HTTP_X_GARNETT_LANGUAGE_CODE", None)
20 |
21 |
22 | def browser(request):
23 | return get_language()
24 |
--------------------------------------------------------------------------------
/garnett/serializers/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Aristotle-Metadata-Enterprises/django-garnett/70f00f381a74831a5f99e08f01f568fa9fb98ffa/garnett/serializers/__init__.py
--------------------------------------------------------------------------------
/garnett/serializers/base.py:
--------------------------------------------------------------------------------
1 | from django.core.serializers.base import Serializer as BaseSerializer
2 | from django.utils.encoding import is_protected_type
3 | from garnett.fields import TranslatedField
4 |
5 |
6 | def fetch_item_and_convert_to_generator(queryset):
7 | # This method exists so we can get the first item of an iterable
8 | # (which can include a generator), and then still iterate through the entire
9 | # set later.
10 | # This ensures that if we have a queryset we only fetch it once,
11 | # or run through the generator once.
12 | inner_iterable = iter(queryset)
13 |
14 | # Get the first item so we can do introspection
15 | item = next(inner_iterable)
16 |
17 | # Make a generator that returns the first item, then the rest of the iterable
18 | def inner_generator():
19 | yield item
20 | yield from inner_iterable
21 |
22 | return item, inner_generator()
23 |
24 |
25 | class TranslatableSerializer(BaseSerializer):
26 | def serialize(
27 | self,
28 | queryset,
29 | *,
30 | stream=None,
31 | fields=None,
32 | use_natural_foreign_keys=False,
33 | use_natural_primary_keys=False,
34 | progress_output=None,
35 | object_count=0,
36 | **options
37 | ):
38 | if fields is not None:
39 | item, queryset = fetch_item_and_convert_to_generator(queryset)
40 | selected_fields = []
41 | for f in item._meta.fields:
42 | if f.name in fields:
43 | if isinstance(f, TranslatedField):
44 | selected_fields.append(f.attname)
45 | else:
46 | selected_fields.append(f.name)
47 | fields = selected_fields
48 |
49 | return super().serialize(
50 | queryset,
51 | stream=stream,
52 | fields=fields,
53 | use_natural_foreign_keys=use_natural_foreign_keys,
54 | use_natural_primary_keys=use_natural_primary_keys,
55 | progress_output=progress_output,
56 | object_count=object_count,
57 | **options
58 | )
59 |
60 | def _value_from_field(self, obj, field):
61 | value = field.value_from_object(obj)
62 | # Protected types (i.e., primitives like None, numbers, dates,
63 | # and Decimals) are passed through as is. All other values are
64 | # converted to string first.
65 | if isinstance(field, TranslatedField):
66 | return field.translations_from_object(obj)
67 | return value if is_protected_type(value) else field.value_to_string(obj)
68 |
--------------------------------------------------------------------------------
/garnett/serializers/json.py:
--------------------------------------------------------------------------------
1 | from django.core.serializers import json
2 | from garnett.serializers.base import TranslatableSerializer
3 |
4 |
5 | class Serializer(TranslatableSerializer, json.Serializer):
6 | pass
7 |
8 |
9 | def Deserializer(stream_or_string, **options):
10 | yield from json.Deserializer(stream_or_string, **options)
11 |
--------------------------------------------------------------------------------
/garnett/templates/admin/change_form_object_tools.html:
--------------------------------------------------------------------------------
1 | {% load i18n admin_urls %}
2 | {% load garnett %}
3 |
4 | {% block object-tools-items %}
5 |
6 | {% for lang in garnett_languages %}
7 | {{ lang.to_tag }}
14 | {% endfor %}
15 |
23 | {% for lang, val in book.title.translations.items %}
24 |
{{ lang.display_name }}: {{val}}
25 | {% endfor %}
26 |
27 |
28 |
29 |
30 |
Pages:
31 | (book.number_of_pages)
32 |
33 |
{{ book.number_of_pages }}
34 |
35 |
36 |
37 |
Extra details:
38 |
39 |
40 | Note: This is a plain json field to demonstrate how garnett
41 | overrides of certain json behaviours don't impact other non-garnett fields.
42 |
43 |
44 | {% for k, v in book.category.items %}
45 |
46 |
{{k}}:
47 |
{{v}}
48 |
49 | {% endfor %}
50 |
51 |
52 |
53 |
54 |
This book available in:
55 | (book.available_languages)
56 |
57 |
58 | Note: This method returns a list of languages that have
59 | values for all fields on this model.
60 |
61 |
62 | {% for lang in book.available_languages %}
63 |
64 | {{ lang.display_name }}
65 |
66 | {% endfor %}
67 |
68 |
69 |
70 |
71 |
72 |
73 | Regardless of which language is selected the properties shown as code
74 | above show the properties used to populate this django template.
75 |
76 | Eg. book.title will return the
77 | name of the book in your selected langauge (if it is available).
78 |