├── .github └── workflows │ ├── lockThreads.yml │ └── main.yml ├── .gitignore ├── CHANGELOG.md ├── LICENSE ├── MANIFEST.in ├── README.rst ├── pyproject.toml ├── src └── django_cleanup │ ├── __init__.py │ ├── apps.py │ ├── cache.py │ ├── cleanup.py │ ├── handlers.py │ └── signals.py ├── test ├── __init__.py ├── conftest.py ├── media │ └── pic.jpg ├── models │ ├── __init__.py │ ├── app.py │ └── integration.py ├── requirements.txt ├── settings.py ├── storage.py ├── test_all.py ├── test_integration.py └── testing_helpers.py └── tox.ini /.github/workflows/lockThreads.yml: -------------------------------------------------------------------------------- 1 | name: 'Lock Threads' 2 | 3 | on: 4 | schedule: 5 | - cron: '25 4 * * *' 6 | workflow_dispatch: 7 | 8 | permissions: 9 | issues: write 10 | pull-requests: write 11 | 12 | concurrency: 13 | group: lock 14 | 15 | jobs: 16 | action: 17 | runs-on: ubuntu-latest 18 | steps: 19 | - uses: dessant/lock-threads@v5 20 | with: 21 | issue-inactive-days: '30' 22 | issue-comment: > 23 | This issue has been automatically locked since there 24 | has not been any recent activity after it was closed. 25 | Please open a new issue for related bugs. 26 | pr-inactive-days: '30' 27 | pr-comment: > 28 | This pull request has been automatically locked since there 29 | has not been any recent activity after it was closed. 30 | Please open a new issue for related bugs. 31 | -------------------------------------------------------------------------------- /.github/workflows/main.yml: -------------------------------------------------------------------------------- 1 | name: CI 2 | on: 3 | push: 4 | branches: [ master ] 5 | pull_request: 6 | branches: [ master ] 7 | workflow_dispatch: 8 | 9 | jobs: 10 | test: 11 | runs-on: ubuntu-latest 12 | steps: 13 | - uses: actions/checkout@v4 14 | - name: Setup python 15 | uses: actions/setup-python@v5 16 | with: 17 | python-version: | 18 | 3.9 19 | 3.10 20 | pypy-3.10 21 | 3.11 22 | 3.12 23 | 3.13 24 | - name: Install test framework 25 | run: pip install tox 26 | - name: Run tests 27 | run: tox r 28 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *.egg-info 3 | .tox 4 | .coverage 5 | .cache 6 | .pypirc 7 | .DS_Store 8 | .python-version 9 | build/* 10 | dist/* 11 | env/* 12 | .vscode/* 13 | README.html 14 | *.txt 15 | .venv/* -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | All notable changes to this project will be documented in this file. 3 | 4 | The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), 5 | and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). 6 | 7 | ## [Unreleased] 8 | ## [9.0.0] - 2024-09-18 9 | ## Added 10 | - pyproject.toml 11 | - Documentation on how to use transaction test case when using pytest. PR [#108] from [@pavel-kalmykov](https://github.com/pavel-kalmykov). 12 | 13 | ### Changed 14 | - Update to remove specific version references, since there haven't been significant changes the approach on versioning will change. The version will no longer update when only tests or supported versions are updated. 15 | - Updated lock thread version and update job to not run at contested times to avoid github rate limiting errors. 16 | - Updated ci build action versions. 17 | - Move isort and pytest settings to toml file. 18 | - Simplify tox.ini and github actions CI job. 19 | - Update a getattr call to remove unnecessary default of None so it will fail on an attribute error. 20 | - Change from .format() to f-strings. 21 | 22 | ### Removed 23 | - Removed setup.py/setup.cfg 24 | 25 | ## [8.1.0] - 2024-01-28 26 | ### Added 27 | - Run tests for django 5.0 and python 3.12. 28 | 29 | ## [8.0.0] - 2023-06-14 30 | ### Added 31 | - Run tests for django 4.2. PR [#100] from [@johnthagen](https://github.com/johnthagen). 32 | 33 | ### Removed 34 | - Dropped support for django 4.0. 35 | 36 | ## [7.0.0] - 2023-02-11 37 | ### Added 38 | - Run tests for django 4.1. 39 | - Run tests on python 3.11 with django 4.1. 40 | - Select mode, with the ability to only cleanup selected models using a `select` decorator. Resolves issue [#75] for [@daviddavis](https://github.com/daviddavis). 41 | - Documentation on the known limitations of referencing a file by multiple model instances. Resolves issue [#98] for [@Grosskopf](https://github.com/Grosskopf) 42 | 43 | ## Changed 44 | - Pass more data to the cleanup_pre_delete and cleanup_post_delete signals. Resolves issue [#96] for [@NadavK](https://github.com/NadavK). 45 | 46 | ### Removed 47 | - Dropped support for django 2.2 and python 3.5. 48 | 49 | ## [6.0.0] - 2022-01-24 50 | ### Added 51 | - Update to run tests for python 3.10. PR [#88] from [@johnthagen](https://github.com/johnthagen). 52 | - GitHub Actions. Resolves issue [#89] for [@johnthagen](https://github.com/johnthagen). 53 | 54 | ### Changed 55 | - Fix default_app_config deprecation. PR [#86] from [@nikolaik](https://github.com/nikolaik). 56 | 57 | ### Removed 58 | - Dropped support for django 3.0 and 3.1. 59 | - Travis configuration. 60 | 61 | ## [5.2.0] - 2021-04-18 62 | ### Added 63 | - New test to ensure cache is reset on create. PR [#81] from [@Flauschbaellchen](https://github.com/Flauschbaellchen). 64 | 65 | ### Changed 66 | - Update to run tests for django 3.2. 67 | - Update to document support for django 3.2. 68 | - Update to run tests for python 3.9. PR [#80] from [@D3X](https://github.com/D3X). 69 | - Reset cache for created instances in the post_save handler. PR [#81] from [@Flauschbaellchen](https://github.com/Flauschbaellchen). 70 | 71 | ## [5.1.0] - 2020-09-15 72 | ### Added 73 | - This change log. Resolves issue [#73] for [@DmytroLitvinov](https://github.com/DmytroLitvinov). 74 | 75 | ### Changed 76 | - Update to run tests for django 3.1. PR [#76] from [@johnthagen](https://github.com/johnthagen). 77 | - Update to document support for django 3.1. PR [#76] from [@johnthagen](https://github.com/johnthagen). 78 | 79 | ### Removed 80 | - Removed providing_args kwarg from Signal construction. PR [#74] from [@coredumperror](https://github.com/coredumperror). 81 | 82 | ## [5.0.0] - 2020-06-07 83 | ## [4.0.1] - 2020-06-06 84 | ## [4.0.0] - 2019-07-13 85 | ## [3.2.0] - 2019-02-17 86 | ## [3.1.0] - 2019-02-05 87 | ## [3.0.1] - 2018-11-18 88 | ## [3.0.0] - 2018-11-18 89 | ## [2.1.0] - 2017-12-30 90 | ## [2.0.0] - 2017-12-27 91 | ## [1.1.0] - 2017-12-27 92 | ## [1.0.1] - 2017-07-14 93 | ## [1.0.0] - 2017-06-30 94 | ## [0.4.2] - 2015-12-17 95 | ## [0.4.1] - 2015-12-02 96 | ## [0.4.0] - 2015-10-06 97 | ## [0.3.1] - 2015-06-25 98 | ## [0.3.0] - 2015-05-12 99 | ## [0.2.1] - 2015-03-07 100 | ## [0.2.0] - 2015-03-06 101 | ## [0.1.13] - 2015-02-21 102 | ## [0.1.12] - 2015-02-08 103 | ## [0.1.11] - 2015-02-01 104 | ## [0.1.10] - 2014-04-29 105 | ## [0.1.9] - 2014-04-29 106 | ## [0.1.8] - 2013-04-07 107 | ## [0.1.7] - 2013-04-03 108 | ## [0.1.6] - 2013-02-12 109 | ## [0.1.5] - 2012-08-17 110 | ## [0.1.4] - 2012-08-16 111 | ## [0.1.0] - 2012-08-14 112 | 113 | [Unreleased]: https://github.com/un1t/django-cleanup/compare/9.0.0...HEAD 114 | [9.0.0]: https://github.com/un1t/django-cleanup/compare/8.1.0...9.0.0 115 | [8.1.0]: https://github.com/un1t/django-cleanup/compare/8.0.0...8.1.0 116 | [8.0.0]: https://github.com/un1t/django-cleanup/compare/7.0.0...8.0.0 117 | [7.0.0]: https://github.com/un1t/django-cleanup/compare/6.0.0...7.0.0 118 | [6.0.0]: https://github.com/un1t/django-cleanup/compare/5.2.0...6.0.0 119 | [5.2.0]: https://github.com/un1t/django-cleanup/compare/5.1.0...5.2.0 120 | [5.1.0]: https://github.com/un1t/django-cleanup/compare/5.0.0...5.1.0 121 | [5.0.0]: https://github.com/un1t/django-cleanup/compare/4.0.1...5.0.0 122 | [4.0.1]: https://github.com/un1t/django-cleanup/compare/4.0.0...4.0.1 123 | [4.0.0]: https://github.com/un1t/django-cleanup/compare/3.2.0...4.0.0 124 | [3.2.0]: https://github.com/un1t/django-cleanup/compare/3.1.0...3.2.0 125 | [3.1.0]: https://github.com/un1t/django-cleanup/compare/3.0.1...3.1.0 126 | [3.0.1]: https://github.com/un1t/django-cleanup/compare/3.0.0...3.0.1 127 | [3.0.0]: https://github.com/un1t/django-cleanup/compare/2.1.0...3.0.0 128 | [2.1.0]: https://github.com/un1t/django-cleanup/compare/2.0.0...2.1.0 129 | [2.0.0]: https://github.com/un1t/django-cleanup/compare/1.1.0...2.0.0 130 | [1.1.0]: https://github.com/un1t/django-cleanup/compare/1.0.1...1.1.0 131 | [1.0.1]: https://github.com/un1t/django-cleanup/compare/1.0.0...1.0.1 132 | [1.0.0]: https://github.com/un1t/django-cleanup/compare/0.4.2...1.0.0 133 | [0.4.2]: https://github.com/un1t/django-cleanup/compare/0.4.1...0.4.2 134 | [0.4.1]: https://github.com/un1t/django-cleanup/compare/0.4.0...0.4.1 135 | [0.4.0]: https://github.com/un1t/django-cleanup/compare/0.3.1...0.4.0 136 | [0.3.1]: https://github.com/un1t/django-cleanup/compare/0.3.0...0.3.1 137 | [0.3.0]: https://github.com/un1t/django-cleanup/compare/0.2.1...0.3.0 138 | [0.2.1]: https://github.com/un1t/django-cleanup/compare/0.2.0...0.2.1 139 | [0.2.0]: https://github.com/un1t/django-cleanup/compare/0.1.13...0.2.0 140 | [0.1.13]: https://github.com/un1t/django-cleanup/compare/0.1.12...0.1.13 141 | [0.1.12]: https://github.com/un1t/django-cleanup/compare/0.1.11...0.1.12 142 | [0.1.11]: https://github.com/un1t/django-cleanup/compare/0.1.10...0.1.11 143 | [0.1.10]: https://github.com/un1t/django-cleanup/compare/0.1.9...0.1.10 144 | [0.1.9]: https://github.com/un1t/django-cleanup/compare/0.1.8...0.1.9 145 | [0.1.8]: https://github.com/un1t/django-cleanup/compare/0.1.7...0.1.8 146 | [0.1.7]: https://github.com/un1t/django-cleanup/compare/0.1.6...0.1.7 147 | [0.1.6]: https://github.com/un1t/django-cleanup/compare/0.1.5...0.1.6 148 | [0.1.5]: https://github.com/un1t/django-cleanup/compare/0.1.4...0.1.5 149 | [0.1.4]: https://github.com/un1t/django-cleanup/compare/0.1.0...0.1.4 150 | [0.1.0]: https://github.com/un1t/django-cleanup/releases/tag/0.1.0 151 | 152 | [#108]: https://github.com/un1t/django-cleanup/pull/108 153 | [#100]: https://github.com/un1t/django-cleanup/pull/100 154 | [#98]: https://github.com/un1t/django-cleanup/issues/98 155 | [#96]: https://github.com/un1t/django-cleanup/issues/96 156 | [#89]: https://github.com/un1t/django-cleanup/issues/89 157 | [#88]: https://github.com/un1t/django-cleanup/pull/88 158 | [#86]: https://github.com/un1t/django-cleanup/pull/86 159 | [#81]: https://github.com/un1t/django-cleanup/pull/81 160 | [#80]: https://github.com/un1t/django-cleanup/pull/80 161 | [#76]: https://github.com/un1t/django-cleanup/pull/76 162 | [#75]: https://github.com/un1t/django-cleanup/issues/75 163 | [#74]: https://github.com/un1t/django-cleanup/pull/74 164 | [#73]: https://github.com/un1t/django-cleanup/issues/73 165 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (C) 2012 by Ilya Shalyapin, ishalyapin@gmail.com 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include CHANGELOG.md 2 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | Django Cleanup 2 | ************** 3 | |Version| |Status| |License| 4 | 5 | Features 6 | ======== 7 | The django-cleanup app automatically deletes files for :code:`FileField`, :code:`ImageField` and 8 | subclasses. When a :code:`FileField`'s value is changed and the model is saved, the old file is 9 | deleted. When a model that has a :code:`FileField` is deleted, the file is also deleted. A file that 10 | is set as the :code:`FileField`'s default value will not be deleted. 11 | 12 | Compatibility 13 | ------------- 14 | - This app follows Django's `supported versions`_ and `Python version support`_. 15 | - Compatible with `sorl-thumbnail `_ 16 | - Compatible with `easy-thumbnail `_ 17 | 18 | How does it work? 19 | ================= 20 | In order to track changes of a :code:`FileField` and facilitate file deletions, django-cleanup 21 | connects :code:`post_init`, :code:`pre_save`, :code:`post_save` and :code:`post_delete` signals to 22 | signal handlers for each :code:`INSTALLED_APPS` model that has a :code:`FileField`. In order to tell 23 | whether or not a :code:`FileField`'s value has changed a local cache of original values is kept on 24 | the model instance. If a condition is detected that should result in a file deletion, a function to 25 | delete the file is setup and inserted into the commit phase of the current transaction. 26 | 27 | **Warning! Please be aware of the known limitations documented below!** 28 | 29 | Installation 30 | ============ 31 | :: 32 | 33 | pip install django-cleanup 34 | 35 | 36 | Configuration 37 | ============= 38 | Add ``django_cleanup`` to the bottom of ``INSTALLED_APPS`` in ``settings.py`` 39 | 40 | .. code-block:: py 41 | 42 | INSTALLED_APPS = ( 43 | ..., 44 | 'django_cleanup.apps.CleanupConfig', 45 | ) 46 | 47 | That is all, no other configuration is necessary. 48 | 49 | Note: Order of ``INSTALLED_APPS`` is important. To ensure that exceptions inside other apps' signal 50 | handlers do not affect the integrity of file deletions within transactions, ``django_cleanup`` 51 | should be placed last in ``INSTALLED_APPS``. 52 | 53 | Troubleshooting 54 | =============== 55 | If you notice that ``django-cleanup`` is not removing files when expected, check that your models 56 | are being properly 57 | `loaded `_: 58 | 59 | You must define or import all models in your application's models.py or models/__init__.py. 60 | Otherwise, the application registry may not be fully populated at this point, which could cause 61 | the ORM to malfunction. 62 | 63 | If your models are not loaded, ``django-cleanup`` will not be able to discover their 64 | ``FileField``'s. 65 | 66 | You can check if your ``Model`` is loaded by using 67 | 68 | .. code-block:: py 69 | 70 | from django.apps import apps 71 | apps.get_models() 72 | 73 | Known limitations 74 | ================= 75 | 76 | Database should support transactions 77 | ------------------------------------ 78 | If you are using a database that does not support transactions you may lose files if a 79 | transaction will rollback at the right instance. This outcome is mitigated by our use of 80 | post_save and post_delete signals, and by following the recommended configuration in this README. 81 | This outcome will still occur if there are signals registered after app initialization and there are 82 | exceptions when those signals are handled. In this case, the old file will be lost and the new file 83 | will not be referenced in a model, though the new file will likely still exist on disk. If you are 84 | concerned about this behavior you will need another solution for old file deletion in your project. 85 | 86 | File referenced by multiple model instances 87 | ------------------------------------------- 88 | This app is designed with the assumption that each file is referenced only once. If you are sharing 89 | a file over two or more model instances you will not have the desired functionality. Be cautious of 90 | copying model instances, as this will cause a file to be shared by more than one instance. If you 91 | want to reference a file from multiple models add a level of indirection. That is, use a separate 92 | file model that is referenced from other models through a foreign key. There are many file 93 | management apps already available in the django ecosystem that fulfill this behavior. 94 | 95 | Advanced 96 | ======== 97 | This section contains additional functionality that can be used to interact with django-cleanup for 98 | special cases. 99 | 100 | Signals 101 | ------- 102 | To facilitate interactions with other django apps django-cleanup sends the following signals which 103 | can be imported from :code:`django_cleanup.signals`: 104 | 105 | - :code:`cleanup_pre_delete`: just before a file is deleted. Passes a :code:`file` keyword argument. 106 | - :code:`cleanup_post_delete`: just after a file is deleted. Passes a :code:`file` keyword argument. 107 | 108 | Signals example for sorl.thumbnail: 109 | 110 | .. code-block:: py 111 | 112 | from django_cleanup.signals import cleanup_pre_delete 113 | from sorl.thumbnail import delete 114 | 115 | def sorl_delete(**kwargs): 116 | delete(kwargs['file']) 117 | 118 | cleanup_pre_delete.connect(sorl_delete) 119 | 120 | Refresh the cache 121 | ----------------- 122 | There have been rare cases where the cache would need to be refreshed. To do so the 123 | :code:`django_cleanup.cleanup.refresh` method can be used: 124 | 125 | .. code-block:: py 126 | 127 | from django_cleanup import cleanup 128 | 129 | cleanup.refresh(model_instance) 130 | 131 | Ignore cleanup for a specific model 132 | ----------------------------------- 133 | To ignore a model and not have cleanup performed when the model is deleted or its files change, use 134 | the :code:`ignore` decorator to mark that model: 135 | 136 | .. code-block:: py 137 | 138 | from django_cleanup import cleanup 139 | 140 | @cleanup.ignore 141 | class MyModel(models.Model): 142 | image = models.FileField() 143 | 144 | Only cleanup selected models 145 | ---------------------------- 146 | If you have many models to ignore, or if you prefer to be explicit about what models are selected, 147 | you can change the mode of django-cleanup to "select mode" by using the select mode app config. In 148 | your ``INSTALLED_APPS`` setting you will replace ':code:`django_cleanup.apps.CleanupConfig`' 149 | with ':code:`django_cleanup.apps.CleanupSelectedConfig`'. Then use the :code:`select` decorator to 150 | mark a model for cleanup: 151 | 152 | .. code-block:: py 153 | 154 | from django_cleanup import cleanup 155 | 156 | @cleanup.select 157 | class MyModel(models.Model): 158 | image = models.FileField() 159 | 160 | How to run tests 161 | ================ 162 | Install, setup and use pyenv_ to install all the required versions of cPython 163 | (see the `tox.ini `_). 164 | 165 | Setup pyenv_ to have all versions of python activated within your local django-cleanup repository. 166 | Ensuring that the latest supported python version that was installed is first priority. 167 | 168 | Install tox_ on the latest supported python version and run the :code:`tox` command from your local 169 | django-cleanup repository. 170 | 171 | How to write tests 172 | ================== 173 | This app requires the use of django.test.TransactionTestCase_ when writing tests. 174 | 175 | For details on why this is required see `here 176 | `_: 177 | 178 | Django's :code:`TestCase` class wraps each test in a transaction and rolls back that transaction 179 | after each test, in order to provide test isolation. This means that no transaction is ever 180 | actually committed, thus your :code:`on_commit()` callbacks will never be run. If you need to 181 | test the results of an :code:`on_commit()` callback, use a :code:`TransactionTestCase` instead. 182 | 183 | pytest 184 | ------ 185 | When writing tests with pytest_ use `@pytest.mark.django_db(transaction=True)`_ with the 186 | :code:`transaction` argument set to :code:`True` to ensure that the behavior will be the same as 187 | using a transaction test case. 188 | 189 | License 190 | ======= 191 | django-cleanup is free software under terms of the: 192 | 193 | MIT License 194 | 195 | Copyright (C) 2012 by Ilya Shalyapin, ishalyapin@gmail.com 196 | 197 | Permission is hereby granted, free of charge, to any person obtaining a copy 198 | of this software and associated documentation files (the "Software"), to deal 199 | in the Software without restriction, including without limitation the rights 200 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 201 | copies of the Software, and to permit persons to whom the Software is 202 | furnished to do so, subject to the following conditions: 203 | 204 | The above copyright notice and this permission notice shall be included in all 205 | copies or substantial portions of the Software. 206 | 207 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 208 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 209 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 210 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 211 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 212 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 213 | SOFTWARE. 214 | 215 | 216 | .. _django.test.TransactionTestCase: https://docs.djangoproject.com/en/stable/topics/testing/tools/#django.test.TransactionTestCase 217 | .. _pytest: https://docs.pytest.org 218 | .. _pyenv: https://github.com/pyenv/pyenv 219 | .. _tox: https://tox.readthedocs.io/en/latest/ 220 | .. _supported versions: https://www.djangoproject.com/download/#supported-versions 221 | .. _Python version support: https://docs.djangoproject.com/en/dev/faq/install/#what-python-version-can-i-use-with-django 222 | .. _@pytest.mark.django_db(transaction=True): https://pytest-django.readthedocs.io/en/latest/helpers.html#pytest.mark.django_db 223 | 224 | .. |Version| image:: https://img.shields.io/pypi/v/django-cleanup.svg 225 | :target: https://pypi.python.org/pypi/django-cleanup/ 226 | :alt: PyPI Package 227 | .. |Status| image:: https://github.com/un1t/django-cleanup/actions/workflows/main.yml/badge.svg 228 | :target: https://github.com/un1t/django-cleanup/actions/workflows/main.yml 229 | :alt: Build Status 230 | .. |License| image:: https://img.shields.io/badge/license-MIT-maroon 231 | :target: https://github.com/un1t/django-cleanup/blob/master/LICENSE 232 | :alt: MIT License 233 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["setuptools"] 3 | build-backend = "setuptools.build_meta" 4 | 5 | [project] 6 | name='django-cleanup' 7 | authors = [ 8 | {name = "Ilya Shalyapin", email = "ishalyapin@gmail.com"} 9 | ] 10 | description = "Deletes old files." 11 | readme = "README.rst" 12 | keywords = ["django"] 13 | license = "MIT" 14 | license-files = [ "LICENSE" ] 15 | classifiers = [ 16 | 'Development Status :: 5 - Production/Stable', 17 | 'Environment :: Web Environment', 18 | 'Framework :: Django', 19 | 'Intended Audience :: Developers', 20 | 'Operating System :: OS Independent', 21 | 'Programming Language :: Python', 22 | 'Programming Language :: Python :: Implementation :: CPython', 23 | 'Programming Language :: Python :: Implementation :: PyPy', 24 | 'Topic :: Utilities' 25 | ] 26 | requires-python = ">=3.9" 27 | dynamic = ["version"] 28 | 29 | [project.urls] 30 | Homepage = "https://github.com/un1t/django-cleanup" 31 | Changelog = "https://github.com/un1t/django-cleanup/blob/master/CHANGELOG.md" 32 | 33 | [tool.setuptools.dynamic] 34 | version = {attr = "django_cleanup.__version__"} 35 | 36 | [tool.isort] 37 | sections = ["FUTURE", "STDLIB", "DJANGO", "THIRDPARTY", "FIRSTPARTY", "LOCALFOLDER"] 38 | line_length = 100 39 | default_section = "THIRDPARTY" 40 | known_first_party = ["django_cleanup"] 41 | known_third_party = ["easy_thumbnails", "sorl", "pytest"] 42 | known_django = "django" 43 | multi_line_output = 4 44 | lines_after_imports = 2 45 | combine_as_imports = true 46 | 47 | [tool.pytest.ini_options] 48 | DJANGO_SETTINGS_MODULE = "test.settings" 49 | pythonpath = [".", "src"] 50 | addopts = ["-v", "--cov-report=term-missing", "--cov=django_cleanup"] 51 | markers = [ 52 | "cleanup_selected_config: marks test as using the CleanupSelectedConfig app config", 53 | "django_storage: change django storage backends" 54 | ] 55 | -------------------------------------------------------------------------------- /src/django_cleanup/__init__.py: -------------------------------------------------------------------------------- 1 | ''' 2 | django-cleanup automatically deletes files for FileField, ImageField, and 3 | subclasses. It will delete old files when a new file is being save and it 4 | will delete files on model instance deletion. 5 | ''' 6 | 7 | __version__ = '9.0.0' 8 | -------------------------------------------------------------------------------- /src/django_cleanup/apps.py: -------------------------------------------------------------------------------- 1 | ''' 2 | AppConfig for django-cleanup, prepare the cache and connect signal handlers 3 | ''' 4 | from django.apps import AppConfig 5 | 6 | from . import cache, handlers 7 | 8 | 9 | class CleanupConfig(AppConfig): 10 | name = 'django_cleanup' 11 | verbose_name = 'Django Cleanup' 12 | default = True 13 | 14 | def ready(self): 15 | cache.prepare(False) 16 | handlers.connect() 17 | 18 | class CleanupSelectedConfig(AppConfig): 19 | name = 'django_cleanup' 20 | verbose_name = 'Django Cleanup' 21 | 22 | def ready(self): 23 | cache.prepare(True) 24 | handlers.connect() 25 | -------------------------------------------------------------------------------- /src/django_cleanup/cache.py: -------------------------------------------------------------------------------- 1 | ''' Our local cache of filefields, everything is private to this package.''' 2 | from collections import defaultdict 3 | 4 | from django.apps import apps 5 | from django.db import models 6 | from django.utils.module_loading import import_string 7 | 8 | 9 | CACHE_NAME = '_django_cleanup_original_cache' 10 | 11 | 12 | def fields_default(): 13 | return set() 14 | FIELDS = defaultdict(fields_default) 15 | 16 | 17 | def fields_dict_default(): 18 | return {} 19 | FIELDS_FIELDS = defaultdict(fields_dict_default) 20 | FIELDS_STORAGE = defaultdict(fields_dict_default) 21 | 22 | 23 | # cache init ## 24 | 25 | 26 | def prepare(select_mode): 27 | '''Prepare the cache for all models, non-reentrant''' 28 | if FIELDS: # pragma: no cover 29 | return 30 | 31 | for model in apps.get_models(): 32 | if ignore_model(model, select_mode): 33 | continue 34 | name = get_model_name(model) 35 | if model_has_filefields(name): # pragma: no cover 36 | continue 37 | opts = model._meta 38 | for field in opts.get_fields(): 39 | if isinstance(field, models.FileField): 40 | add_field_for_model(name, field.name, field) 41 | 42 | 43 | def add_field_for_model(model_name, field_name, field): 44 | '''Centralized function to make all our local caches.''' 45 | # store models that have filefields and the field names 46 | FIELDS[model_name].add(field_name) 47 | # store the dotted path of the field class for each field 48 | # in case we need to restore it later on 49 | FIELDS_FIELDS[model_name][field_name] = get_dotted_path(field) 50 | # also store the dotted path of the storage for the same reason 51 | FIELDS_STORAGE[model_name][field_name] = get_dotted_path(field.storage) 52 | 53 | 54 | # generators ## 55 | 56 | 57 | def get_fields_for_model(model_name, exclude=None): 58 | '''Get the filefields for a model if it has them''' 59 | if model_has_filefields(model_name): 60 | fields = FIELDS[model_name] 61 | 62 | if exclude is not None: 63 | assert isinstance(exclude, set) 64 | fields = fields.difference(exclude) 65 | 66 | for field_name in fields: 67 | yield field_name 68 | 69 | 70 | def fields_for_model_instance(instance, using=None): 71 | ''' 72 | Yields (name, descriptor) for each file field given an instance 73 | 74 | Can use the `using` kwarg to change the instance that the `FieldFile` 75 | will receive. 76 | ''' 77 | if using is None: 78 | using = instance 79 | model_name = get_model_name(instance) 80 | 81 | deferred_fields = instance.get_deferred_fields() 82 | 83 | for field_name in get_fields_for_model(model_name, exclude=deferred_fields): 84 | fieldfile = getattr(instance, field_name) 85 | yield field_name, fieldfile.__class__(using, fieldfile.field, fieldfile.name) 86 | 87 | 88 | # restore ## 89 | 90 | 91 | def get_field(model_name, field_name): 92 | '''Restore a field from its dotted path''' 93 | return import_string(FIELDS_FIELDS[model_name][field_name]) 94 | 95 | 96 | def get_field_storage(model_name, field_name): 97 | '''Restore a storage from its dotted path''' 98 | return import_string(FIELDS_STORAGE[model_name][field_name]) 99 | 100 | 101 | # utilities ## 102 | 103 | 104 | def get_dotted_path(object_): 105 | '''get the dotted path for an object''' 106 | klass = object_.__class__ 107 | return f'{klass.__module__}.{klass.__qualname__}' 108 | 109 | 110 | def get_model_name(model): 111 | '''returns a unique model name''' 112 | opt = model._meta 113 | return f'{opt.app_label}.{opt.model_name}' 114 | 115 | 116 | def get_mangled_ignore(model): 117 | '''returns a mangled attribute name specific to the model for ignore functionality''' 118 | opt = model._meta 119 | return f'_{opt.model_name}__{opt.app_label}_cleanup_ignore' 120 | 121 | 122 | def get_mangled_select(model): 123 | '''returns a mangled attribute name specific to the model for select functionality''' 124 | opt = model._meta 125 | return f'_{opt.model_name}__{opt.app_label}_cleanup_select' 126 | 127 | 128 | # booleans ## 129 | 130 | 131 | def model_has_filefields(model_name): 132 | '''Check if a model has filefields''' 133 | return model_name in FIELDS 134 | 135 | 136 | def ignore_model(model, select_mode): 137 | '''Check if a model should be ignored''' 138 | return ((not hasattr(model, get_mangled_select(model))) 139 | if select_mode else hasattr(model, get_mangled_ignore(model))) 140 | 141 | 142 | # instance functions ## 143 | 144 | 145 | def remove_instance_cache(instance): 146 | '''Remove the cache from an instance''' 147 | if has_cache(instance): 148 | delattr(instance, CACHE_NAME) 149 | 150 | 151 | def make_cleanup_cache(instance, source=None): 152 | ''' 153 | Make the cleanup cache for an instance. 154 | 155 | Can also change the source of the data with the `source` kwarg. 156 | ''' 157 | 158 | if source is None: 159 | source = instance 160 | setattr(instance, CACHE_NAME, dict( 161 | fields_for_model_instance(source, using=instance))) 162 | 163 | 164 | def has_cache(instance): 165 | '''Check if an instance has a cache on it''' 166 | return hasattr(instance, CACHE_NAME) 167 | 168 | 169 | def get_field_attr(instance, field_name): 170 | '''Get a value from the cache on an instance''' 171 | return getattr(instance, CACHE_NAME)[field_name] 172 | 173 | 174 | # data sharing ## 175 | 176 | 177 | def cleanup_models(): 178 | '''Get all the models we have in the FIELDS cache''' 179 | for model_name in FIELDS: 180 | yield apps.get_model(model_name) 181 | 182 | 183 | def cleanup_fields(): 184 | '''Get a copy of the FIELDS cache''' 185 | return FIELDS.copy() 186 | -------------------------------------------------------------------------------- /src/django_cleanup/cleanup.py: -------------------------------------------------------------------------------- 1 | '''Public utilities''' 2 | from .cache import ( 3 | get_mangled_ignore as _get_mangled_ignore, get_mangled_select as _get_mangled_select, 4 | make_cleanup_cache as _make_cleanup_cache) 5 | 6 | 7 | __all__ = ['refresh', 'cleanup_ignore', 'cleanup_select'] 8 | 9 | 10 | def refresh(instance): 11 | '''Refresh the cache for an instance''' 12 | return _make_cleanup_cache(instance) 13 | 14 | 15 | def ignore(cls): 16 | '''Mark a model to ignore for cleanup''' 17 | setattr(cls, _get_mangled_ignore(cls), None) 18 | return cls 19 | cleanup_ignore = ignore 20 | 21 | 22 | def select(cls): 23 | '''Mark a model to select for cleanup''' 24 | setattr(cls, _get_mangled_select(cls), None) 25 | return cls 26 | cleanup_select = select 27 | -------------------------------------------------------------------------------- /src/django_cleanup/handlers.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Signal handlers to manage FileField files. 3 | ''' 4 | import logging 5 | 6 | from django.db.models.signals import post_delete, post_init, post_save, pre_save 7 | from django.db.transaction import on_commit 8 | 9 | from . import cache 10 | from .signals import cleanup_post_delete, cleanup_pre_delete 11 | 12 | 13 | logger = logging.getLogger(__name__) 14 | 15 | 16 | class FakeInstance: 17 | '''A Fake model instance to ensure an instance is not modified''' 18 | 19 | 20 | def cache_original_post_init(sender, instance, **kwargs): 21 | '''Post_init on all models with file fields, saves original values''' 22 | cache.make_cleanup_cache(instance) 23 | 24 | 25 | def fallback_pre_save(sender, instance, raw, update_fields, using, **kwargs): 26 | '''Fallback to the database to remake the cleanup cache if there is none''' 27 | if raw: # pragma: no cover 28 | return 29 | 30 | if instance.pk and not cache.has_cache(instance): 31 | try: 32 | db_instance = sender.objects.get(pk=instance.pk) 33 | except sender.DoesNotExist: # pragma: no cover 34 | return 35 | cache.make_cleanup_cache(instance, source=db_instance) 36 | 37 | 38 | def delete_old_post_save(sender, instance, raw, created, update_fields, using, 39 | **kwargs): 40 | '''Post_save on all models with file fields, deletes old files''' 41 | if raw: 42 | return 43 | 44 | if not created: 45 | for field_name, new_file in cache.fields_for_model_instance(instance): 46 | if update_fields is None or field_name in update_fields: 47 | old_file = cache.get_field_attr(instance, field_name) 48 | if old_file != new_file: 49 | delete_file(sender, instance, field_name, old_file, using, 'updated') 50 | 51 | # reset cache 52 | cache.make_cleanup_cache(instance) 53 | 54 | 55 | def delete_all_post_delete(sender, instance, using, **kwargs): 56 | '''Post_delete on all models with file fields, deletes all files''' 57 | for field_name, file_ in cache.fields_for_model_instance(instance): 58 | delete_file(sender, instance, field_name, file_, using, 'deleted') 59 | 60 | 61 | def delete_file(sender, instance, field_name, file_, using, reason): 62 | '''Deletes a file''' 63 | 64 | if not file_.name: 65 | return 66 | 67 | # add a fake instance to the file being deleted to avoid 68 | # any changes to the real instance. 69 | file_.instance = FakeInstance() 70 | 71 | # pickled filefields lose lots of data, and contrary to how it is 72 | # documented, the file descriptor does not recover them 73 | 74 | model_name = cache.get_model_name(instance) 75 | 76 | # recover the 'field' if necessary 77 | if not hasattr(file_, 'field'): 78 | file_.field = cache.get_field(model_name, field_name)() 79 | file_.field.name = field_name 80 | 81 | # if our file name is default don't delete 82 | default = file_.field.default if not callable(file_.field.default) else file_.field.default() 83 | 84 | if file_.name == default: 85 | return 86 | 87 | # recover the 'storage' if necessary 88 | if not hasattr(file_, 'storage'): 89 | file_.storage = cache.get_field_storage(model_name, field_name)() 90 | 91 | event = { 92 | 'deleted': reason == 'deleted', 93 | 'model_name': model_name, 94 | 'field_name': field_name, 95 | 'file_name': file_.name, 96 | 'default_file_name': default, 97 | 'file': file_, 98 | 'instance': instance, 99 | 'updated': reason == 'updated' 100 | } 101 | 102 | # this will run after a successful commit 103 | # assuming you are in a transaction and on a database that supports 104 | # transactions, otherwise it will run immediately 105 | def run_on_commit(): 106 | cleanup_pre_delete.send(sender=sender, **event) 107 | success = False 108 | error = None 109 | try: 110 | file_.delete(save=False) 111 | success = True 112 | except Exception as ex: 113 | error = ex 114 | opts = instance._meta 115 | logger.exception( 116 | 'There was an exception deleting the file `%s` on field `%s.%s.%s`', 117 | file_, opts.app_label, opts.model_name, field_name) 118 | cleanup_post_delete.send(sender=sender, error=error, success=success, **event) 119 | 120 | on_commit(run_on_commit, using) 121 | 122 | 123 | def connect(): 124 | '''Connect signals to the cleanup models''' 125 | for model in cache.cleanup_models(): 126 | suffix = f'_django_cleanup_{cache.get_model_name(model)}' 127 | post_init.connect(cache_original_post_init, sender=model, 128 | dispatch_uid=f'post_init{suffix}') 129 | pre_save.connect(fallback_pre_save, sender=model, 130 | dispatch_uid=f'pre_save{suffix}') 131 | post_save.connect(delete_old_post_save, sender=model, 132 | dispatch_uid=f'post_save{suffix}') 133 | post_delete.connect(delete_all_post_delete, sender=model, 134 | dispatch_uid=f'post_delete{suffix}') 135 | -------------------------------------------------------------------------------- /src/django_cleanup/signals.py: -------------------------------------------------------------------------------- 1 | ''' 2 | django-cleanup sends the following signals 3 | ''' 4 | from django.dispatch import Signal 5 | 6 | 7 | __all__ = ['cleanup_pre_delete', 'cleanup_post_delete'] 8 | 9 | 10 | cleanup_pre_delete = Signal() 11 | '''Called just before a file is deleted. Passes a `file` keyword argument.''' 12 | 13 | 14 | cleanup_post_delete = Signal() 15 | '''Called just after a file is deleted. Passes a `file` keyword argument.''' 16 | -------------------------------------------------------------------------------- /test/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/un1t/django-cleanup/380b4bb4c7d277fc125ffa7edce2d3911483912e/test/__init__.py -------------------------------------------------------------------------------- /test/conftest.py: -------------------------------------------------------------------------------- 1 | import copy 2 | import os 3 | import shutil 4 | 5 | from django.conf import settings as django_settings 6 | from django.db.models.signals import post_delete, post_init, post_save, pre_save 7 | 8 | import pytest 9 | 10 | from django_cleanup import cache, handlers 11 | 12 | from .testing_helpers import get_random_pic_name 13 | 14 | 15 | pytest_plugins = () 16 | 17 | def pytest_collection_modifyitems(items): 18 | for item in items: 19 | item.add_marker(pytest.mark.django_db(transaction=True)) 20 | 21 | 22 | @pytest.fixture(autouse=True) 23 | def setup_django_cleanup_state(request, settings): 24 | for model in cache.cleanup_models(): 25 | suffix = f'_django_cleanup_{cache.get_model_name(model)}' 26 | post_init.disconnect(None, sender=model, 27 | dispatch_uid=f'post_init{suffix}') 28 | pre_save.disconnect(None, sender=model, 29 | dispatch_uid=f'pre_save{suffix}') 30 | post_save.disconnect(None, sender=model, 31 | dispatch_uid=f'post_save{suffix}') 32 | post_delete.disconnect(None, sender=model, 33 | dispatch_uid=f'post_delete{suffix}') 34 | cache.FIELDS.clear() 35 | cache.prepare(request.node.get_closest_marker('cleanup_selected_config') is not None) 36 | handlers.connect() 37 | 38 | stroage_marker = request.node.get_closest_marker('django_storage') 39 | if stroage_marker is not None: 40 | storages = copy.deepcopy(settings.STORAGES) 41 | for key, value in stroage_marker.kwargs.items(): 42 | storages[key]['BACKEND'] = value 43 | settings.STORAGES = storages 44 | 45 | 46 | @pytest.fixture(params=[django_settings.MEDIA_ROOT]) 47 | def picture(request): 48 | src = os.path.join(request.param, 'pic.jpg') 49 | dst = os.path.join(request.param, get_random_pic_name()) 50 | shutil.copyfile(src, dst) 51 | try: 52 | yield { 53 | 'path': dst, 54 | 'filename': os.path.split(dst)[-1], 55 | 'srcpath': src 56 | } 57 | finally: 58 | if os.path.exists(dst): 59 | os.remove(dst) 60 | -------------------------------------------------------------------------------- /test/media/pic.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/un1t/django-cleanup/380b4bb4c7d277fc125ffa7edce2d3911483912e/test/media/pic.jpg -------------------------------------------------------------------------------- /test/models/__init__.py: -------------------------------------------------------------------------------- 1 | from .app import * 2 | 3 | 4 | try: 5 | from .integration import * 6 | except ImportError as e: 7 | pass 8 | -------------------------------------------------------------------------------- /test/models/app.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | 3 | from django_cleanup import cleanup 4 | from django_cleanup.cleanup import cleanup_ignore 5 | 6 | 7 | def default_image(): 8 | return 'pic.jpg' 9 | 10 | 11 | # ignore this cleanup_ignore decorator 12 | # it is only here to test that name mangling is in place 13 | # so that ignores on parent classes don't impact subclasses 14 | # and is not a demonstration on how to use this decorator 15 | # see the ProductIgnore model below for how to use the decorator 16 | @cleanup_ignore 17 | class ProductAbstract(models.Model): 18 | image = models.FileField(upload_to='test', blank=True, null=True) 19 | image_default = models.FileField( 20 | upload_to='test', blank=True, null=True, 21 | default='pic.jpg') 22 | image_default_callable = models.FileField( 23 | upload_to='test', blank=True, null=True, default=default_image) 24 | 25 | class Meta: 26 | abstract = True 27 | 28 | 29 | @cleanup.select 30 | class Product(ProductAbstract): 31 | pass 32 | 33 | 34 | @cleanup.ignore 35 | class ProductIgnore(ProductAbstract): 36 | pass 37 | 38 | 39 | class ProductProxy(Product): 40 | class Meta: 41 | proxy = True 42 | 43 | 44 | class ProductUnmanaged(ProductAbstract): 45 | id = models.AutoField(primary_key=True) 46 | 47 | class Meta: 48 | managed = False 49 | db_table = 'test_product' 50 | 51 | 52 | class RootProduct(models.Model): 53 | pass 54 | 55 | 56 | class BranchProduct(models.Model): 57 | root = models.ForeignKey(RootProduct, on_delete=models.CASCADE) 58 | image = models.FileField(upload_to='test', blank=True, null=True) 59 | -------------------------------------------------------------------------------- /test/models/integration.py: -------------------------------------------------------------------------------- 1 | from easy_thumbnails.fields import ThumbnailerImageField 2 | from sorl.thumbnail import ImageField 3 | 4 | from .app import ProductAbstract 5 | 6 | 7 | class ProductIntegrationAbstract(ProductAbstract): 8 | sorl_image = ImageField(upload_to='test', blank=True) 9 | easy_image = ThumbnailerImageField(upload_to='test', blank=True) 10 | 11 | class Meta: 12 | abstract = True 13 | 14 | 15 | class ProductIntegration(ProductIntegrationAbstract): 16 | pass 17 | 18 | 19 | def sorl_delete(**kwargs): 20 | from sorl.thumbnail import delete 21 | delete(kwargs['file']) 22 | -------------------------------------------------------------------------------- /test/requirements.txt: -------------------------------------------------------------------------------- 1 | sorl-thumbnail 2 | easy-thumbnails 3 | pillow 4 | pytest 5 | pytest-django 6 | pytest-cov 7 | -------------------------------------------------------------------------------- /test/settings.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | import django 4 | 5 | 6 | BASE_DIR = os.path.dirname(__file__) 7 | 8 | DATABASES = { 9 | 'default': { 10 | 'ENGINE': 'django.db.backends.sqlite3', 11 | 'NAME': ':memory:', 12 | }, 13 | } 14 | 15 | INSTALLED_APPS = ( 16 | 'test', 17 | 'django_cleanup', 18 | ) 19 | 20 | INSTALLED_APPS_INTEGRATION = ( 21 | 'sorl.thumbnail', 22 | 'easy_thumbnails', 23 | ) 24 | 25 | try: 26 | import easy_thumbnails.fields 27 | import sorl.thumbnail 28 | except ImportError: 29 | pass 30 | except (django.core.exceptions.AppRegistryNotReady, django.core.exceptions.ImproperlyConfigured): 31 | INSTALLED_APPS = INSTALLED_APPS + INSTALLED_APPS_INTEGRATION 32 | 33 | MIDDLEWARE_CLASSES = [] 34 | SECRET_KEY = '123' 35 | MEDIA_ROOT = os.path.join(BASE_DIR, 'media') 36 | 37 | ## workaround: https://github.com/SmileyChris/easy-thumbnails/issues/641#issuecomment-2291098096 38 | THUMBNAIL_DEFAULT_STORAGE_ALIAS = 'default' 39 | -------------------------------------------------------------------------------- /test/storage.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | from django.core.files.storage import FileSystemStorage 4 | 5 | 6 | class DeleteErrorStorage(FileSystemStorage): 7 | def delete(self, name): 8 | ''' delete modified to not catch FileNotFoundError 9 | does not support deleting directories 10 | ''' 11 | name = self.path(name) 12 | # If the file or directory exists, delete it from the filesystem. 13 | os.remove(name) 14 | -------------------------------------------------------------------------------- /test/test_all.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import os 3 | import pickle 4 | import re 5 | import sys 6 | import tempfile 7 | 8 | from django.conf import settings as django_settings 9 | from django.core.files import File 10 | from django.core.files.base import ContentFile 11 | from django.db import transaction 12 | from django.db.models.fields import NOT_PROVIDED, files 13 | 14 | import pytest 15 | 16 | from django_cleanup import cache, handlers 17 | from django_cleanup.signals import cleanup_post_delete, cleanup_pre_delete 18 | 19 | from . import storage 20 | from .models.app import ( 21 | BranchProduct, Product, ProductIgnore, ProductProxy, ProductUnmanaged, RootProduct) 22 | from .testing_helpers import get_random_pic_name, get_using 23 | 24 | 25 | LINE = re.compile(r'line \d{1,3}') 26 | 27 | 28 | 29 | def get_traceback(picture): 30 | fileabspath = os.path.abspath 31 | error = 'FileNotFoundError' 32 | if sys.version_info < (3, 13): 33 | return f'''Traceback (most recent call last): 34 | File "{fileabspath(handlers.__file__)}", line xxx, in run_on_commit 35 | file_.delete(save=False) 36 | File "{fileabspath(files.__file__)}", line xxx, in delete 37 | self.storage.delete(self.name) 38 | File "{fileabspath(storage.__file__)}", line xxx, in delete 39 | os.remove(name) 40 | {error}: [Errno 2] No such file or directory: '{picture}\'''' 41 | else: 42 | return f'''Traceback (most recent call last): 43 | File "{fileabspath(handlers.__file__)}", line xxx, in run_on_commit 44 | file_.delete(save=False) 45 | ~~~~~~~~~~~~^^^^^^^^^^^^ 46 | File "{fileabspath(files.__file__)}", line xxx, in delete 47 | self.storage.delete(self.name) 48 | ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^ 49 | File "{fileabspath(storage.__file__)}", line xxx, in delete 50 | os.remove(name) 51 | ~~~~~~~~~^^^^^^ 52 | {error}: [Errno 2] No such file or directory: '{picture}\'''' 53 | 54 | 55 | def _raise(message): 56 | def _func(x): # pragma: no cover 57 | raise Exception(message) 58 | return _func 59 | 60 | 61 | def test_refresh_from_db_without_refresh(picture): 62 | product = Product.objects.create(image=picture['filename']) 63 | assert os.path.exists(picture['path']) 64 | product.refresh_from_db() 65 | assert id(product.image.instance) == id(product) 66 | product.image = get_random_pic_name() 67 | with transaction.atomic(get_using(product)): 68 | product.save() 69 | assert not os.path.exists(picture['path']) 70 | 71 | 72 | def test_cache_gone(picture): 73 | product = Product.objects.create(image=picture['filename']) 74 | assert os.path.exists(picture['path']) 75 | product.image = get_random_pic_name() 76 | cache.remove_instance_cache(product) 77 | with transaction.atomic(get_using(product)): 78 | product.save() 79 | assert not os.path.exists(picture['path']) 80 | 81 | 82 | def test_storage_gone(picture): 83 | product = Product.objects.create(image=picture['filename']) 84 | assert os.path.exists(picture['path']) 85 | product.image = get_random_pic_name() 86 | product = pickle.loads(pickle.dumps(product)) 87 | assert hasattr(product.image, 'storage') 88 | with transaction.atomic(get_using(product)): 89 | product.save() 90 | assert not os.path.exists(picture['path']) 91 | 92 | 93 | def test_replace_file_with_file(picture): 94 | product = Product.objects.create(image=picture['filename']) 95 | assert os.path.exists(picture['path']) 96 | random_pic_name = get_random_pic_name() 97 | product.image = random_pic_name 98 | with transaction.atomic(get_using(product)): 99 | product.save() 100 | assert not os.path.exists(picture['path']) 101 | assert product.image 102 | new_image_path = os.path.join(django_settings.MEDIA_ROOT, random_pic_name) 103 | assert product.image.path == new_image_path 104 | 105 | 106 | def test_replace_file_with_blank(picture): 107 | product = Product.objects.create(image=picture['filename']) 108 | assert os.path.exists(picture['path']) 109 | product.image = '' 110 | with transaction.atomic(get_using(product)): 111 | product.save() 112 | assert not os.path.exists(picture['path']) 113 | assert not product.image 114 | assert product.image.name == '' 115 | 116 | 117 | def test_replace_file_with_none(picture): 118 | product = Product.objects.create(image=picture['filename']) 119 | assert os.path.exists(picture['path']) 120 | product.image = None 121 | with transaction.atomic(get_using(product)): 122 | product.save() 123 | assert not os.path.exists(picture['path']) 124 | assert not product.image 125 | assert product.image.name is None 126 | 127 | 128 | def test_replace_file_proxy(picture): 129 | product = ProductProxy.objects.create(image=picture['filename']) 130 | assert os.path.exists(picture['path']) 131 | product.image = get_random_pic_name() 132 | with transaction.atomic(get_using(product)): 133 | product.save() 134 | assert not os.path.exists(picture['path']) 135 | 136 | 137 | def test_replace_file_unmanaged(picture): 138 | product = ProductUnmanaged.objects.create(image=picture['filename']) 139 | assert os.path.exists(picture['path']) 140 | product.image = get_random_pic_name() 141 | with transaction.atomic(get_using(product)): 142 | product.save() 143 | assert not os.path.exists(picture['path']) 144 | 145 | 146 | def test_replace_file_deferred(picture): 147 | '''probably shouldn't save from a deferred model but someone might do it''' 148 | product = Product.objects.create(image=picture['filename']) 149 | assert os.path.exists(picture['path']) 150 | product_deferred = Product.objects.defer('image_default').get(id=product.id) 151 | product_deferred.image = get_random_pic_name() 152 | with transaction.atomic(get_using(product)): 153 | product_deferred.save() 154 | assert not os.path.exists(picture['path']) 155 | 156 | 157 | def test_remove_model_instance(picture): 158 | product = Product.objects.create(image=picture['filename']) 159 | assert os.path.exists(picture['path']) 160 | with transaction.atomic(get_using(product)): 161 | product.delete() 162 | assert not os.path.exists(picture['path']) 163 | 164 | 165 | def test_remove_model_instance_default(picture): 166 | product = Product.objects.create() 167 | assert product.image_default.path == picture['srcpath'] 168 | assert product.image_default_callable.path == picture['srcpath'] 169 | assert os.path.exists(picture['srcpath']) 170 | with transaction.atomic(get_using(product)): 171 | product.delete() 172 | assert os.path.exists(picture['srcpath']) 173 | 174 | 175 | def test_replace_file_with_file_default(picture): 176 | product = Product.objects.create() 177 | assert os.path.exists(picture['srcpath']) 178 | random_pic_name1 = get_random_pic_name() 179 | random_pic_name2 = get_random_pic_name() 180 | product.image_default = random_pic_name1 181 | product.image_default_callable = random_pic_name2 182 | with transaction.atomic(get_using(product)): 183 | product.save() 184 | assert os.path.exists(picture['srcpath']) 185 | 186 | 187 | def test_remove_model_instance_ignore(picture): 188 | product = ProductIgnore.objects.create(image=picture['filename']) 189 | assert os.path.exists(picture['path']) 190 | with transaction.atomic(get_using(product)): 191 | product.delete() 192 | assert os.path.exists(picture['path']) 193 | 194 | 195 | def test_replace_file_with_file_ignore(picture): 196 | product = ProductIgnore.objects.create(image=picture['filename']) 197 | assert os.path.exists(picture['path']) 198 | random_pic_name = get_random_pic_name() 199 | product.image = random_pic_name 200 | with transaction.atomic(get_using(product)): 201 | product.save() 202 | assert os.path.exists(picture['path']) 203 | assert product.image 204 | new_image_path = os.path.join(django_settings.MEDIA_ROOT, random_pic_name) 205 | assert product.image.path == new_image_path 206 | 207 | 208 | def test_remove_model_instance_proxy(picture): 209 | product = ProductProxy.objects.create(image=picture['filename']) 210 | assert os.path.exists(picture['path']) 211 | with transaction.atomic(get_using(product)): 212 | product.delete() 213 | assert not os.path.exists(picture['path']) 214 | 215 | 216 | def test_remove_model_instance_unmanaged(picture): 217 | product = ProductUnmanaged.objects.create(image=picture['filename']) 218 | assert os.path.exists(picture['path']) 219 | with transaction.atomic(get_using(product)): 220 | product.delete() 221 | assert not os.path.exists(picture['path']) 222 | 223 | 224 | def test_remove_model_instance_deferred(picture): 225 | product = Product.objects.create(image=picture['filename']) 226 | assert os.path.exists(picture['path']) 227 | product_deferred = Product.objects.defer('image_default').get(id=product.id) 228 | with transaction.atomic(get_using(product)): 229 | product_deferred.delete() 230 | assert not os.path.exists(picture['path']) 231 | 232 | 233 | def test_remove_blank_file(monkeypatch): 234 | product = Product.objects.create(image='') 235 | monkeypatch.setattr( 236 | product.image.storage, 'exists', _raise('should not call exists')) 237 | monkeypatch.setattr( 238 | product.image.storage, 'delete', _raise('should not call delete')) 239 | with transaction.atomic(get_using(product)): 240 | product.delete() 241 | 242 | 243 | def test_remove_not_exists(): 244 | product = Product.objects.create(image='no-such-file') 245 | with transaction.atomic(get_using(product)): 246 | product.delete() 247 | 248 | 249 | def test_remove_none(monkeypatch): 250 | product = Product.objects.create(image=None) 251 | monkeypatch.setattr( 252 | product.image.storage, 'exists', _raise('should not call exists')) 253 | monkeypatch.setattr( 254 | product.image.storage, 'delete', _raise('should not call delete')) 255 | with transaction.atomic(get_using(product)): 256 | product.delete() 257 | 258 | 259 | @pytest.mark.django_storage(default='test.storage.DeleteErrorStorage') 260 | def test_exception_on_save(picture, caplog): 261 | filename = picture['filename'] 262 | product = Product.objects.create(image=filename) 263 | # simulate a fieldfile that has a storage that raises a filenotfounderror on delete 264 | assert os.path.exists(picture['path']) 265 | product.image.delete(save=False) 266 | product.image = None 267 | assert not os.path.exists(picture['path']) 268 | with transaction.atomic(get_using(product)): 269 | product.save() 270 | assert not os.path.exists(picture['path']) 271 | 272 | for record in caplog.records: 273 | assert LINE.sub('line xxx', record.exc_text) == get_traceback(picture['path']) 274 | assert caplog.record_tuples == [ 275 | ( 276 | 'django_cleanup.handlers', 277 | logging.ERROR, 278 | f'There was an exception deleting the file `{filename}` on field `test.product.image`' 279 | ) 280 | ] 281 | 282 | 283 | def test_cascade_delete(picture): 284 | root = RootProduct.objects.create() 285 | branch = BranchProduct.objects.create(root=root, image=picture['filename']) 286 | assert os.path.exists(picture['path']) 287 | root = RootProduct.objects.get() 288 | with transaction.atomic(get_using(root)): 289 | root.delete() 290 | assert not os.path.exists(picture['path']) 291 | 292 | 293 | def test_file_exists_on_create_and_update(): 294 | # If a filepath is specified which already exists, 295 | # the FileField generates a random suffix to choose a different location. 296 | # We need to make sure, that we fetch this change and would delete the correct one 297 | # on further edits or the final deletion. 298 | # In this test case it is simulated by using a temporary file located 299 | # directly within the same directory as the image would be uploaded to. 300 | 301 | upload_to = Product._meta.get_field("image").upload_to 302 | dst_directory = os.path.join(django_settings.MEDIA_ROOT, upload_to) 303 | if not os.path.isdir(dst_directory): 304 | os.makedirs(dst_directory) 305 | 306 | # create the new product with a file to simulate an "upload" 307 | # a file aleady exists so the new file is renamed then saved 308 | with tempfile.NamedTemporaryFile(prefix="f1__", dir=dst_directory) as f1: 309 | with transaction.atomic(): 310 | product = Product.objects.create( 311 | image=File(f1, name=os.path.join(upload_to, os.path.basename(f1.name)))) 312 | 313 | assert f1.name != product.image.path 314 | assert os.path.exists(f1.name) 315 | assert os.path.exists(product.image.path) 316 | 317 | path_prior_to_edit = product.image.path 318 | 319 | # edit the product to change the product file to a different file 320 | # check that it deletes the renamed file, not the original existing file 321 | with tempfile.NamedTemporaryFile(prefix="f2__", dir=dst_directory) as f2: 322 | with transaction.atomic(get_using(product)): 323 | product.image = File(f2, name=os.path.join(upload_to, os.path.basename(f2.name))) 324 | assert f2.name == product.image.path 325 | product.save() 326 | 327 | assert f1.name != product.image.path 328 | assert os.path.exists(f1.name) 329 | assert f2.name != product.image.path 330 | assert os.path.isfile(f2.name) 331 | assert os.path.isfile(product.image.path) 332 | assert not os.path.isfile(path_prior_to_edit) 333 | 334 | with transaction.atomic(get_using(product)): 335 | product.delete() 336 | 337 | assert os.path.isfile(f1.name) 338 | assert os.path.isfile(f2.name) 339 | assert not os.path.isfile(product.image.path) 340 | 341 | 342 | def test_signals(picture): 343 | prekwargs = {} 344 | postkwargs = {} 345 | def assn_prekwargs(**kwargs): 346 | nonlocal prekwargs 347 | prekwargs = kwargs 348 | 349 | def assn_postkwargs(**kwargs): 350 | nonlocal postkwargs 351 | postkwargs = kwargs 352 | 353 | cleanup_pre_delete.connect( 354 | assn_prekwargs, dispatch_uid='pre_test_replace_file_with_file_signals') 355 | cleanup_post_delete.connect( 356 | assn_postkwargs, dispatch_uid='post_test_replace_file_with_file_signals') 357 | product = Product.objects.create(image=picture['filename']) 358 | random_pic_name = get_random_pic_name() 359 | product.image = random_pic_name 360 | with transaction.atomic(get_using(product)): 361 | product.save() 362 | 363 | assert prekwargs['deleted'] is False 364 | assert prekwargs['updated'] is True 365 | assert prekwargs['instance'] == product 366 | assert prekwargs['file'] is not None 367 | assert prekwargs['file_name'] == picture['filename'] 368 | assert isinstance(prekwargs['default_file_name'], NOT_PROVIDED) 369 | assert prekwargs['model_name'] == 'test.product' 370 | assert prekwargs['field_name'] == 'image' 371 | 372 | assert postkwargs['deleted'] is False 373 | assert postkwargs['updated'] is True 374 | assert postkwargs['instance'] == product 375 | assert postkwargs['file'] is not None 376 | assert postkwargs['file_name'] == picture['filename'] 377 | assert isinstance(postkwargs['default_file_name'], NOT_PROVIDED) 378 | assert postkwargs['model_name'] == 'test.product' 379 | assert postkwargs['field_name'] == 'image' 380 | assert postkwargs['success'] is True 381 | assert postkwargs['error'] is None 382 | 383 | with transaction.atomic(get_using(product)): 384 | product.delete() 385 | 386 | assert prekwargs['deleted'] is True 387 | assert prekwargs['updated'] is False 388 | assert prekwargs['instance'] == product 389 | assert prekwargs['file'] is not None 390 | assert prekwargs['file_name'] == random_pic_name 391 | assert isinstance(prekwargs['default_file_name'], NOT_PROVIDED) 392 | assert prekwargs['model_name'] == 'test.product' 393 | assert prekwargs['field_name'] == 'image' 394 | 395 | assert postkwargs['deleted'] is True 396 | assert postkwargs['updated'] is False 397 | assert postkwargs['instance'] == product 398 | assert postkwargs['file'] is not None 399 | assert postkwargs['file_name'] == random_pic_name 400 | assert isinstance(postkwargs['default_file_name'], NOT_PROVIDED) 401 | assert postkwargs['model_name'] == 'test.product' 402 | assert postkwargs['field_name'] == 'image' 403 | print(postkwargs['error']) 404 | assert postkwargs['success'] is True 405 | assert postkwargs['error'] is None 406 | 407 | cleanup_pre_delete.disconnect(None, dispatch_uid='pre_test_replace_file_with_file_signals') 408 | cleanup_post_delete.disconnect(None, dispatch_uid='post_test_replace_file_with_file_signals') 409 | 410 | 411 | #region select config 412 | @pytest.mark.cleanup_selected_config 413 | def test__select_config__replace_file_with_file(picture): 414 | product = Product.objects.create(image=picture['filename']) 415 | assert os.path.exists(picture['path']) 416 | random_pic_name = get_random_pic_name() 417 | product.image = random_pic_name 418 | with transaction.atomic(get_using(product)): 419 | product.save() 420 | assert not os.path.exists(picture['path']) 421 | assert product.image 422 | new_image_path = os.path.join(django_settings.MEDIA_ROOT, random_pic_name) 423 | assert product.image.path == new_image_path 424 | 425 | 426 | @pytest.mark.cleanup_selected_config 427 | def test__select_config__replace_file_with_file_ignore(picture): 428 | product = ProductIgnore.objects.create(image=picture['filename']) 429 | assert os.path.exists(picture['path']) 430 | random_pic_name = get_random_pic_name() 431 | product.image = random_pic_name 432 | with transaction.atomic(get_using(product)): 433 | product.save() 434 | assert os.path.exists(picture['path']) 435 | assert product.image 436 | new_image_path = os.path.join(django_settings.MEDIA_ROOT, random_pic_name) 437 | assert product.image.path == new_image_path 438 | #endregion 439 | -------------------------------------------------------------------------------- /test/test_integration.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | from django.conf import settings as django_settings 4 | from django.db import transaction 5 | 6 | import pytest 7 | 8 | from django_cleanup.signals import cleanup_pre_delete 9 | 10 | from .testing_helpers import get_using 11 | 12 | 13 | def test_sorlthumbnail_replace(picture): 14 | # https://github.com/mariocesar/sorl-thumbnail 15 | get_thumbnail = pytest.importorskip('sorl.thumbnail').get_thumbnail 16 | models = pytest.importorskip('test.models.integration') 17 | product_integration = models.ProductIntegration 18 | sorl_delete = models.sorl_delete 19 | cleanup_pre_delete.connect(sorl_delete) 20 | product = product_integration.objects.create(sorl_image=picture['filename']) 21 | assert os.path.exists(picture['path']) 22 | im = get_thumbnail( 23 | product.sorl_image, '100x100', crop='center', quality=50) 24 | thumbnail_path = os.path.join(django_settings.MEDIA_ROOT, im.name) 25 | assert os.path.exists(thumbnail_path) 26 | product.sorl_image = 'new.png' 27 | with transaction.atomic(get_using(product)): 28 | product.save() 29 | assert not os.path.exists(picture['path']) 30 | assert not os.path.exists(thumbnail_path) 31 | cleanup_pre_delete.disconnect(sorl_delete) 32 | 33 | 34 | def test_sorlthumbnail_delete(picture): 35 | # https://github.com/mariocesar/sorl-thumbnail 36 | get_thumbnail = pytest.importorskip('sorl.thumbnail').get_thumbnail 37 | models = pytest.importorskip( 'test.models.integration') 38 | product_integration = models.ProductIntegration 39 | sorl_delete = models.sorl_delete 40 | cleanup_pre_delete.connect(sorl_delete) 41 | product = product_integration.objects.create(sorl_image=picture['filename']) 42 | assert os.path.exists(picture['path']) 43 | im = get_thumbnail( 44 | product.sorl_image, '100x100', crop='center', quality=50) 45 | thumbnail_path = os.path.join(django_settings.MEDIA_ROOT, im.name) 46 | assert os.path.exists(thumbnail_path) 47 | with transaction.atomic(get_using(product)): 48 | product.delete() 49 | assert not os.path.exists(picture['path']) 50 | assert not os.path.exists(thumbnail_path) 51 | cleanup_pre_delete.disconnect(sorl_delete) 52 | 53 | 54 | def test_easythumbnails_replace(picture): 55 | # https://github.com/SmileyChris/easy-thumbnails 56 | get_thumbnailer = pytest.importorskip('easy_thumbnails.files').get_thumbnailer 57 | models = pytest.importorskip( 'test.models.integration') 58 | product_integration = models.ProductIntegration 59 | product = product_integration.objects.create(easy_image=picture['filename']) 60 | assert os.path.exists(picture['path']) 61 | im = get_thumbnailer(product.easy_image).get_thumbnail( 62 | {'size': (100, 100)}) 63 | thumbnail_path = os.path.join(django_settings.MEDIA_ROOT, im.name) 64 | assert os.path.exists(thumbnail_path) 65 | product.easy_image = 'new.png' 66 | with transaction.atomic(get_using(product)): 67 | product.save() 68 | assert not os.path.exists(picture['path']) 69 | assert not os.path.exists(thumbnail_path) 70 | 71 | 72 | def test_easythumbnails_delete(picture): 73 | # https://github.com/SmileyChris/easy-thumbnails 74 | get_thumbnailer = pytest.importorskip('easy_thumbnails.files').get_thumbnailer 75 | models = pytest.importorskip( 'test.models.integration') 76 | product_integration = models.ProductIntegration 77 | product = product_integration.objects.create(easy_image=picture['filename']) 78 | assert os.path.exists(picture['path']) 79 | im = get_thumbnailer(product.easy_image).get_thumbnail( 80 | {'size': (100, 100)}) 81 | thumbnail_path = os.path.join(django_settings.MEDIA_ROOT, im.name) 82 | assert os.path.exists(thumbnail_path) 83 | with transaction.atomic(get_using(product)): 84 | product.delete() 85 | assert not os.path.exists(picture['path']) 86 | assert not os.path.exists(thumbnail_path) 87 | -------------------------------------------------------------------------------- /test/testing_helpers.py: -------------------------------------------------------------------------------- 1 | import random 2 | import string 3 | 4 | from django.db import router 5 | 6 | 7 | def get_using(instance): 8 | return router.db_for_write(instance.__class__, instance=instance) 9 | 10 | 11 | def get_random_pic_name(length=20): 12 | random_str = ''.join(random.choice(string.ascii_letters) for m in range(length)) 13 | return f'pic{random_str}.jpg' 14 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = 3 | py{39,310,311,312}-django{42} 4 | py{310,311,312,313,py3}-django{51,52} 5 | [testenv] 6 | deps = 7 | # LTS April 2025 - April 2028 8 | django52: django<5.3 9 | # August 2024 - December 2025 10 | django51: django<5.2 11 | # LTS April 2023 - April 2026 12 | django42: django<4.3 13 | -rtest/requirements.txt 14 | commands = pytest test 15 | --------------------------------------------------------------------------------