├── .gitignore ├── CHANGELOG.md ├── LICENSE.md ├── MANIFEST.in ├── README.md ├── django_tm ├── __init__.py ├── admin.py ├── apps.py ├── celery.py ├── celery_logs │ └── .gitignore ├── celery_scripts │ ├── check-db-health.py │ ├── check-disk-free.py │ └── clean-database.py ├── config.py ├── migrations │ └── __init__.py ├── models.py ├── signals.py ├── tasks.py ├── templates │ ├── static │ │ └── .gitkeep │ └── tasks │ │ ├── base.html │ │ └── index.html ├── templatetags │ ├── __init__.py │ └── formats.py ├── tests.py ├── urls.py └── views.py ├── docs └── blank.txt ├── publish.txt └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *.DS_Store 3 | *.egg* 4 | /dist/ 5 | /.idea 6 | /docs/_build/ 7 | /node_modules/ 8 | build/ 9 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Change Log 2 | 3 | ## [1.0.1] 2022-11-01 4 | ### Improvements 5 | 6 | - Update `DOCS` 7 | 8 | ## [1.0.0] 2022-10-30 9 | ### Improvements 10 | 11 | - STABLE_VERSION 12 | 13 | ## [0.0.6] 2022-10-30 14 | ### Improvements 15 | 16 | - Update `DOCS` 17 | 18 | ## [0.0.5] 2022-10-14 19 | ### Improvements 20 | 21 | - Update `DOCS` 22 | - Added sample project (step-by-step info) 23 | - [Django Tasks Manager Sample](https://github.com/app-generator/sample-django-tasks-manager) 24 | 25 | ## [0.0.4] 2022-10-13 26 | ### Improvements 27 | 28 | - Update `DOCS` 29 | - Added SShots 30 | 31 | ## [0.0.3] 2022-10-13 32 | ### Improvements 33 | 34 | - Update `README` 35 | 36 | ## [0.0.2] 2022-10-13 37 | ### Improvements 38 | 39 | - Update `README` 40 | 41 | ## [0.0.1] 2022-10-12 42 | ### Initial Release 43 | 44 | - Minimal Version 45 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 [App Generator](https://appseed.us) 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include LICENSE.md 2 | include README.md 3 | recursive-include django_tm/static * 4 | recursive-include django_tm/templates * 5 | recursive-include django_tm/celery_logs * 6 | recursive-include django_tm/celery_scripts * 7 | recursive-include docs * 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # [Django Tasks Manager](https://appseed.us/developer-tools/django-tasks-manager/) 2 | 3 | A super simple **Django & Celery** integration - This library is actively supported by [AppSeed](https://appseed.us/). 4 | 5 |
6 | 7 | > Features: 8 | 9 | - Create/Revoke `Celery Tasks` 10 | - `View LOGS` & Output 11 | - `Minimal Configuration` 12 | - Available [TASKS](https://github.com/app-generator/django-tasks-manager/blob/main/django_tm/tasks.py) (provided as starting samples) 13 | - `users_in_db()` - List all registered users 14 | - `execute_script()` - let users execute the [scripts](https://github.com/app-generator/django-tasks-manager/tree/main/django_tm/celery_scripts) defined in `CELERY_SCRIPTS_DIR` (CFG parameter) 15 | - [check-db-health.py](https://github.com/app-generator/django-tasks-manager/blob/main/django_tm/celery_scripts/check-db-health.py) (sample) 16 | 17 |
18 | 19 | ![Django Tasks Manager - View Running Tasks.](https://user-images.githubusercontent.com/51070104/195670211-a24f7d72-37c1-48fc-a842-ab45b4559ca0.jpg) 20 | 21 |
22 | 23 | ## How to use it 24 | 25 |
26 | 27 | > **Install the package** via `PIP` 28 | 29 | ```bash 30 | $ pip install django-tasks-manager 31 | // OR 32 | $ pip install git+https://github.com/app-generator/django-tasks-manager.git 33 | ``` 34 | 35 |
36 | 37 | > **Include the new routing** 38 | 39 | ```python 40 | # core/urls.py 41 | 42 | from django.urls import path, include # <-- UPDATE: Add 'include' HELPER 43 | 44 | urlpatterns = [ 45 | ... 46 | 47 | path("", include("django_tm.urls")), # <-- New Routes 48 | 49 | ... 50 | ] 51 | ``` 52 | 53 |
54 | 55 | > Create **Scrips & LOGS** directories - The Recomended place is in the root of the project: 56 | 57 | ```bash 58 | $ mkdir celery_scripts # Used in Settings -> CELERY_SCRIPTS_DIR 59 | $ mkdir celery_logs # Used in Settings -> CELERY_SCRIPTS_DIR 60 | ``` 61 | 62 | - Make sure the user that executes the app has write permission. 63 | - Copy the [sample scripts](./django_tm/celery_scripts) in the scripts directory. 64 | - All scripts will be available in the UI, ready to be executed by the manager. 65 | 66 |
67 | 68 | > **Update Configuration** Add `os` object import 69 | 70 | ```python 71 | import os # <-- NEW 72 | ``` 73 | 74 |
75 | 76 | > **Update Configuration**: Include the new APPS 77 | 78 | ```python 79 | INSTALLED_APPS = [ 80 | ... 81 | 'django_tm', # Django Tasks Manager # <-- NEW 82 | 'django_celery_results', # Django Celery Results # <-- NEW 83 | ] 84 | ``` 85 | 86 |
87 | 88 | > **Update Configuration**: Include the new templates 89 | 90 | ```python 91 | TEMPLATE_DIR_TASKS = os.path.join(BASE_DIR, "django_tm/templates") # <-- NEW 92 | 93 | TEMPLATES = [ 94 | { 95 | 'BACKEND': 'django.template.backends.django.DjangoTemplates', 96 | 'DIRS': [TEMPLATE_DIR_TASKS], # <-- Updated 97 | 'APP_DIRS': True, 98 | }, 99 | ] 100 | ``` 101 | 102 |
103 | 104 | > **Update Configuration**: New **CELERY_** Section 105 | 106 | ```python 107 | 108 | ############################################################# 109 | # Celery configurations 110 | # https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html 111 | 112 | # !!! 113 | # BASE_DIR points to the ROOT of the project 114 | # Note: make sure you have 'os' object imported 115 | # !!! 116 | 117 | # !!! 118 | # BASE_DIR = os.path.dirname(os.path.dirname(__file__)) 119 | # !!! 120 | 121 | # Working Directories required write permission 122 | CELERY_SCRIPTS_DIR = os.path.join(BASE_DIR, "celery_scripts" ) 123 | CELERY_LOGS_DIR = os.path.join(BASE_DIR, "celery_logs" ) 124 | 125 | CELERY_BROKER_URL = os.environ.get("CELERY_BROKER", "redis://localhost:6379") 126 | CELERY_RESULT_BACKEND = os.environ.get("CELERY_BROKER", "redis://localhost:6379") 127 | 128 | CELERY_TASK_TRACK_STARTED = True 129 | CELERY_TASK_TIME_LIMIT = 30 * 60 130 | CELERY_CACHE_BACKEND = "django-cache" 131 | CELERY_RESULT_BACKEND = "django-db" 132 | CELERY_RESULT_EXTENDED = True 133 | CELERY_RESULT_EXPIRES = 60*60*24*30 # Results expire after 1 month 134 | CELERY_ACCEPT_CONTENT = ["json"] 135 | CELERY_TASK_SERIALIZER = 'json' 136 | CELERY_RESULT_SERIALIZER = 'json' 137 | 138 | ############################################################# 139 | ############################################################# 140 | 141 | ``` 142 | 143 |
144 | 145 | 146 | > **Start the App** 147 | 148 | ```bash 149 | $ # Set up the database 150 | $ python manage.py makemigrations 151 | $ python manage.py migrate 152 | $ 153 | $ # Create the superuser 154 | $ python manage.py createsuperuser 155 | $ 156 | $ # Start the application (development mode) 157 | $ python manage.py runserver # default port 8000 158 | ``` 159 | 160 | - **Authenticate** as `superuser` 161 | - Access the `Tasks` page: `http://127.0.0.1:8000/tasks` 162 | 163 |
164 | 165 | > **Start the Celery Manager** (another terminal) & **Update Environment** 166 | 167 | Export `DJANGO_SETTINGS_MODULE` using the value provided in `manage.py` 168 | 169 | ```bash 170 | $ export DJANGO_SETTINGS_MODULE=cfg.settings 171 | ``` 172 | 173 | The value used export should be taken from `manage.py`: 174 | 175 | ```python 176 | def main(): 177 | 178 | """Run administrative tasks.""" 179 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cfg.settings") # <-- VALUE to be exported 180 | 181 | try: 182 | from django.core.management import execute_from_command_line 183 | except ImportError as exc: 184 | ... 185 | ``` 186 | 187 |
188 | 189 | **Note**: `Redis` server is expected on port `6379` (default). In case Redis runs on other `PORT`, please update the configuration: `CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND`. 190 | 191 | ```bash 192 | $ celery --app=django_tm.celery.app worker --loglevel=info 193 | ``` 194 | 195 |
196 | 197 | ## Screens 198 | 199 | > `View all tasks` 200 | 201 | ![Django Tasks Manager - View All Tasks.](https://user-images.githubusercontent.com/51070104/195669853-677e887e-f8b2-4b56-bcf3-f81d98b175b0.jpg) 202 | 203 |
204 | 205 | > `View Task LOG` 206 | 207 | ![Django Tasks Manager - View Task LOG.](https://user-images.githubusercontent.com/51070104/195669981-c64e3d13-1d83-496a-b527-cade9cda2cd2.jpg) 208 | 209 |
210 | 211 | ## Links & Resources 212 | 213 | - Free [Support](https://appseed.us/support/) via Email and Discord 214 | - [Django Tasks Manager](https://github.com/app-generator/sample-django-tasks-manager) - `free sample` that explains: 215 | - `Project Creation` (minimal files added) 216 | - `Install` & Generate `Django` Core 217 | - Install `Django-TM` (this package) 218 | - `Update Configuration` 219 | - `Start the app` 220 | - Use the `Tasks Manager` 221 | 222 |
223 | 224 | --- 225 | [Django Tasks Manager](https://appseed.us/developer-tools/django-tasks-manager/) - Open-source library provided by **[AppSeed](https://appseed.us/)** 226 | -------------------------------------------------------------------------------- /django_tm/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | -------------------------------------------------------------------------------- /django_tm/admin.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | from django.contrib import admin 7 | 8 | # Register your models here. 9 | -------------------------------------------------------------------------------- /django_tm/apps.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | from django.apps import AppConfig 7 | 8 | class TasksConfig(AppConfig): 9 | 10 | """ 11 | See https://docs.djangoproject.com/en/2.1/ref/applications/#django.apps.AppConfig 12 | """ 13 | 14 | name = "django_tm" 15 | label = "django_tm" 16 | verbose_name = "Django Tasks Manager" 17 | 18 | def ready(self): 19 | from . import signals 20 | -------------------------------------------------------------------------------- /django_tm/celery.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | import os 7 | 8 | from celery import Celery 9 | 10 | if os.environ.get('DJANGO_SETTINGS_MODULE'): 11 | 12 | app = Celery('core') 13 | 14 | # Using a string here means the worker doesn't have to serialize 15 | # the configuration object to child processes. 16 | # - namespace='CELERY' means all celery-related configuration keys 17 | # should have a `CELERY_` prefix. 18 | app.config_from_object('django.conf:settings', namespace='CELERY') 19 | 20 | # Load task modules from all registered Django apps. 21 | app.autodiscover_tasks() 22 | 23 | else: 24 | print(' ') 25 | print('Celery Configuration ERROR: ') 26 | print(' > "DJANGO_SETTINGS_MODULE" not set in environment (value in manage.py)') 27 | print(' Hint: export DJANGO_SETTINGS_MODULE=project.settings ') 28 | print(' ') -------------------------------------------------------------------------------- /django_tm/celery_logs/.gitignore: -------------------------------------------------------------------------------- 1 | *.log 2 | -------------------------------------------------------------------------------- /django_tm/celery_scripts/check-db-health.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | import os 7 | import sys 8 | 9 | def main(): 10 | 11 | try: 12 | 13 | print(' EXEC -> ' + os.path.basename(__file__)) 14 | 15 | # Unix ErrCode 16 | exit(0) 17 | 18 | except Exception as e: 19 | 20 | print( 'Err: ' + str( e ) ) 21 | exit(1) 22 | 23 | if __name__ == '__main__': 24 | main() 25 | -------------------------------------------------------------------------------- /django_tm/celery_scripts/check-disk-free.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | import os 7 | import sys 8 | 9 | def main(): 10 | 11 | try: 12 | 13 | print(' EXEC -> ' + os.path.basename(__file__)) 14 | 15 | # Unix ErrCode 16 | exit(0) 17 | 18 | except Exception as e: 19 | 20 | print( 'Err: ' + str( e ) ) 21 | exit(1) 22 | 23 | if __name__ == '__main__': 24 | main() 25 | -------------------------------------------------------------------------------- /django_tm/celery_scripts/clean-database.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | import os 7 | import sys 8 | 9 | def main(): 10 | 11 | try: 12 | 13 | print(' EXEC -> ' + os.path.basename(__file__)) 14 | 15 | # Unix ErrCode 16 | exit(0) 17 | 18 | except Exception as e: 19 | 20 | print( 'Err: ' + str( e ) ) 21 | exit(1) 22 | 23 | if __name__ == '__main__': 24 | main() 25 | -------------------------------------------------------------------------------- /django_tm/config.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | from django.apps import AppConfig 7 | 8 | class MyConfig(AppConfig): 9 | name = 'django_tm' 10 | label = 'django_tm' 11 | -------------------------------------------------------------------------------- /django_tm/migrations/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | -------------------------------------------------------------------------------- /django_tm/models.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | from django.db import models 7 | 8 | # Create your models here. 9 | 10 | -------------------------------------------------------------------------------- /django_tm/signals.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | import os, json 7 | 8 | from django_celery_results.models import TaskResult 9 | from django.db.models.signals import post_save, pre_save 10 | from django.dispatch import receiver 11 | from django.conf import settings 12 | 13 | 14 | @receiver(pre_save, sender=TaskResult) 15 | def pre_result_updated(sender, **kwargs): 16 | ''' 17 | Before a task result is saved, the following checks and updates are carried out if task is complete: 18 | - Check if error key & value exist in result.result, if yes upate instance.status=`FAILURE` and instance.traceback=`logs produced during task execution` 19 | ''' 20 | if 'instance' not in kwargs: 21 | return 22 | instance = kwargs.pop("instance") 23 | if instance.status != "STARTED": 24 | result = json.loads(instance.result) 25 | if result.get("error"): 26 | instance.status = "FAILURE" 27 | instance.traceback = result.get("logs") 28 | 29 | 30 | @receiver(post_save, sender=TaskResult) 31 | def post_result_updated(sender, **kwargs): 32 | """ 33 | After a task is completed, we store its logs in a file. 34 | """ 35 | if 'instance' not in kwargs: 36 | return 37 | instance = kwargs.pop("instance") 38 | if instance.status != "STARTED": 39 | add_log(instance) 40 | 41 | 42 | def add_log(result: TaskResult): 43 | """ 44 | Adds log file to celery logs with formatted date as name. 45 | :param result TaskResult: Result of executed celery task 46 | :rtype: None 47 | """ 48 | if not result.task_name: 49 | return 50 | 51 | try: 52 | 53 | log_file_path = os.path.join( 54 | settings.CELERY_LOGS_DIR, f"{result.date_created.strftime(r'%Y%m%d-%H_%M')}-{result.task_id}-{result.status}.log") 55 | 56 | with open(log_file_path, "w+") as f: 57 | if result.status == "FAILURE": 58 | f.writelines([result.result, result.traceback]) 59 | else: 60 | f.write(result.result) 61 | 62 | f.close() 63 | 64 | except Exception as e: 65 | print( 'Logging ERR: ' + str( e) ) 66 | return 67 | -------------------------------------------------------------------------------- /django_tm/tasks.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | import os, time, subprocess 7 | from os import listdir 8 | from os.path import isfile, join 9 | 10 | from .celery import app 11 | from celery.contrib.abortable import AbortableTask 12 | 13 | from django.contrib.auth.models import User 14 | from django.conf import settings 15 | 16 | def get_scripts(): 17 | """ 18 | Returns all scripts from 'ROOT_DIR/celery_scripts' 19 | """ 20 | try: 21 | scripts = [f for f in listdir(settings.CELERY_SCRIPTS_DIR) if isfile(join(settings.CELERY_SCRIPTS_DIR, f))] 22 | return scripts, None 23 | except Exception as e: 24 | return None, 'Error CELERY_SCRIPTS_DIR: ' + str( e ) 25 | 26 | @app.task(bind=True, base=AbortableTask) 27 | def users_in_db(self, data: dict): 28 | """ 29 | Outputs all users in DB 30 | :param data dict: contains data needed for task execution. 31 | :rtype: None 32 | """ 33 | users = User.objects.all() 34 | return {"output": "\n".join([u.email for u in users])} 35 | 36 | @app.task(bind=True, base=AbortableTask) 37 | def execute_script(self, data: dict): 38 | """ 39 | This task executes scripts found in settings.CELERY_SCRIPTS_DIR and logs are later generated and stored in settings.CELERY_LOGS_DIR 40 | :param data dict: contains data needed for task execution. Example `input` which is the script to be executed. 41 | :rtype: None 42 | """ 43 | script = data.get("input") 44 | 45 | scripts, ErrInfo = get_scripts() 46 | 47 | if script and script in scripts: 48 | # Executing related script 49 | script_path = os.path.join(settings.CELERY_SCRIPTS_DIR, script) 50 | process = subprocess.Popen( 51 | f"python {script_path}", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) 52 | time.sleep(8) 53 | error = False 54 | if process.wait() == 0: # If script execution successfull 55 | logs = process.stdout.read().decode() 56 | else: 57 | logs = process.stderr.read().decode() 58 | error = True 59 | 60 | return {"logs": logs, "input": script, "error": error, "output": ""} 61 | -------------------------------------------------------------------------------- /django_tm/templates/static/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/app-generator/django-tasks-manager/923dd1c0e61d460ea13070ce8cc8671177c1bcbb/django_tm/templates/static/.gitkeep -------------------------------------------------------------------------------- /django_tm/templates/tasks/base.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Django Tasks Manager {% block title %} {% endblock title %} | AppSeed 10 | 11 | 12 | 13 | 14 | 15 | 16 | {% block stylesheets %}{% endblock stylesheets %} 17 | 18 | 19 | 20 | 21 | {% block content %} {% endblock content %} 22 | 23 | 24 | 25 | 26 | 27 | 28 | {% block javascripts %}{% endblock javascripts %} 29 | 30 | 31 | -------------------------------------------------------------------------------- /django_tm/templates/tasks/index.html: -------------------------------------------------------------------------------- 1 | {% extends 'tasks/base.html' %} 2 | {% load formats %} 3 | 4 | {% block title %} Tables {% endblock title %} 5 | 6 | {% block stylesheets %} 7 | 8 | {% endblock stylesheets %} 9 | 10 | 11 | {% block content %} 12 | 13 |
14 | 15 | 16 |
17 |
18 |
19 |
20 |
21 | Async Tasks Manager 22 |
23 |
24 |
25 | 26 | {% if scripts %} 27 |

28 | Superusers are able to create/cancel tasks. 29 |
30 | Ordinary users can only view execution logs and running tasks (no other interactions allowed). 31 |

32 | {% else %} 33 |

34 | No scripts detected - Please update the configuration (CELERY_SCRIPTS_DIR, CELERY_LOGS_DIR) 35 |

36 | {% endif %} 37 | 38 |
39 |
40 |
41 |
42 | 43 | 44 |
45 |
46 |
47 |
48 |
Tasks List
49 |
50 |
51 |
52 | 53 | 54 | 55 | 56 | 57 | 58 | 61 | 64 | {% if request.user.is_superuser %} 65 | 66 | {% endif %} 67 | 68 | 69 | 70 | 71 | 72 | {% for task in tasks %} 73 | 74 | {% if task.status == "STARTED" %} 75 | 76 | {% else %} 77 | 78 | {% endif %} 79 | 80 | 81 | {% csrf_token %} 82 | 83 | 98 | 106 | 119 | 140 | 145 | 146 | {% if request.user.is_superuser %} 147 | 172 | {% endif %} 173 | 174 | 175 | 176 | {% endfor %} 177 | 178 | 179 |
NameScriptSTATE 59 | Input 60 | 62 | Latest EXEC 63 | Action
84 |
85 |
86 | 87 |
88 |
89 |
90 | {{task.name}} 91 |
92 |

93 | Celery Task 94 |

95 |
96 |
97 | 		99 |

100 | {{ task.script }} 101 |

102 |
103 | Latest status: {{ task.status }} 104 |
105 | 		107 | 108 | {% if task.status == "STARTED" %} 109 | RUNNING 110 | {% elif task.status == "FAILURE" %} 111 | FINISHED 112 | {% elif task.status == "REVOKED" %} 113 | CANCELLED 114 | {% else %} 115 | FINISHED 116 | {% endif %} 117 | 118 | 120 | 121 | 122 | {% if task.name == 'execute_script' %} 123 | 124 | 133 | 134 | {% else %} 135 | NA 136 | {% endif %} 137 | 138 | 139 | 141 | 142 | {{task.date_created|date_format}} 143 | 144 | 148 | 149 | {% if task.status == "STARTED" %} 150 | 151 | 154 | 155 | {% elif task.status == "FAILURE" or task.status == "REVOKED" %} 156 | 157 | 161 | 162 | {% else %} 163 | 164 | 168 | 169 | {% endif %} 170 | 171 |
180 |
181 |
182 |
183 |
184 |
185 | 186 | 187 |
188 |
189 |
190 |
191 |
192 | LOGS 193 |
194 |
195 |
196 |
197 | 198 | 199 | 200 | 201 | 202 | 203 | 204 | 205 | 206 | 207 | 208 | 209 | 210 | {% for result in task_results %} 211 | 212 | 213 | 225 | 226 | 229 | 230 | 239 | 240 | 243 | 244 | 247 | 248 | 251 | 256 | 257 | 258 | {% endfor %} 259 | 260 | 261 |
TaskInputStatusStart TSEnd TSOutputLogs
214 |
215 |
216 |
217 | {{result.id}} - {{result.task_name}} 218 |
219 |

220 | {{result.task_id}} 221 |

222 |
223 |
224 | 		227 |

{{result|get_result_field:"input"}}

228 | 		231 |

236 | {{result.status}} 237 |

238 | 		241 |

{{result.date_created|date_format}}

242 | 		245 |

{{result.date_done|date_format}}

246 | 		249 |

{{result|get_result_field:"output"}}

250 | 		252 |

253 | View LOG 254 |

255 |
262 |
263 |
264 |
265 |
266 |
267 | 268 |
269 | 270 | 289 | 290 | {% endblock content %} 291 | 292 | 293 | 294 | {% block javascripts %} 295 | 296 | 315 | 316 | {% endblock javascripts %} 317 | -------------------------------------------------------------------------------- /django_tm/templatetags/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | -------------------------------------------------------------------------------- /django_tm/templatetags/formats.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | import json 7 | 8 | from django import template 9 | 10 | register = template.Library() 11 | 12 | def date_format(date): 13 | """ 14 | Returns a formatted date string 15 | Format: `Year-Month-Day-Hour-Minute-Second` 16 | Example: `2022-10-10-00-20-33` 17 | :param date datetime: Date object to be formatted 18 | :rtype: str 19 | """ 20 | try: 21 | return date.strftime(r'%Y-%m-%d-%H-%M-%S') 22 | except: 23 | return date 24 | 25 | register.filter("date_format", date_format) 26 | 27 | def get_result_field(result, field: str): 28 | """ 29 | Returns a field from the content of the result attibute in result 30 | Example: `result.result['field']` 31 | :param result AbortableAsyncResult: Result object to get field from 32 | :param field str: Field to return from result object 33 | :rtype: str 34 | """ 35 | result = json.loads(result.result) 36 | if result: 37 | return result.get(field) 38 | 39 | register.filter("get_result_field", get_result_field) 40 | -------------------------------------------------------------------------------- /django_tm/tests.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | from django.test import TestCase 7 | 8 | # Create your tests here. 9 | -------------------------------------------------------------------------------- /django_tm/urls.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | from django.urls import path, re_path 7 | from . import views 8 | 9 | urlpatterns = [ 10 | path('tasks', views.tasks, name="tasks"), 11 | path('tasks/run/' , views.run_task, name="run-task" ), 12 | path('tasks/cancel/' , views.cancel_task, name="cancel-task" ), 13 | path('tasks/output/' , views.task_output, name="task-output" ), 14 | path('tasks/log/' , views.task_log, name="task-log" ), 15 | ] -------------------------------------------------------------------------------- /django_tm/views.py: -------------------------------------------------------------------------------- 1 | # -*- encoding: utf-8 -*- 2 | """ 3 | Copyright (c) 2019 - present AppSeed.us 4 | """ 5 | 6 | import os, time, json 7 | from os import listdir 8 | from os.path import isfile, join 9 | 10 | from django.contrib.auth.decorators import login_required 11 | from django.http import HttpResponse 12 | from django.shortcuts import redirect 13 | from django.template import loader 14 | from django.conf import settings 15 | 16 | from celery import current_app 17 | from .tasks import execute_script, users_in_db, get_scripts 18 | from django_celery_results.models import TaskResult 19 | from celery.contrib.abortable import AbortableAsyncResult 20 | from .celery import app 21 | 22 | @login_required(login_url="/login/") 23 | def tasks(request): 24 | 25 | scripts, ErrInfo = get_scripts() 26 | 27 | context = { 28 | 'cfgError' : ErrInfo, 29 | 'tasks' : get_celery_all_tasks(), 30 | 'scripts' : scripts} 31 | 32 | # django_celery_results_task_result 33 | task_results = TaskResult.objects.all() 34 | context["task_results"] = task_results 35 | 36 | html_template = loader.get_template('tasks/index.html') 37 | return HttpResponse(html_template.render(context, request)) 38 | 39 | def run_task(request, task_name): 40 | ''' 41 | Runs a celery task 42 | :param request HttpRequest: Request 43 | :param task_name str: Name of task to execute 44 | :rtype: (HttpResponseRedirect | HttpResponsePermanentRedirect) 45 | ''' 46 | tasks = [execute_script, users_in_db] 47 | _input = request.POST.get("input") 48 | for task in tasks: 49 | if task.__name__ == task_name: 50 | task.delay({"input": _input}) 51 | time.sleep(1) # Waiting for task status to update in db 52 | 53 | return redirect("tasks") 54 | 55 | def cancel_task(request, task_id): 56 | ''' 57 | Cancels a celery task using its task id 58 | :param request HttpRequest: Request 59 | :param task_id str: task_id of result to cancel execution 60 | :rtype: (HttpResponseRedirect | HttpResponsePermanentRedirect) 61 | ''' 62 | result = TaskResult.objects.get(task_id=task_id) 63 | abortable_result = AbortableAsyncResult( 64 | result.task_id, task_name=result.task_name, app=app) 65 | if not abortable_result.is_aborted(): 66 | abortable_result.revoke(terminate=True) 67 | time.sleep(1) 68 | return redirect("tasks") 69 | 70 | def get_celery_all_tasks(): 71 | current_app.loader.import_default_modules() 72 | tasks = list(sorted(name for name in current_app.tasks 73 | if not name.startswith('celery.'))) 74 | tasks = [{"name": name.split(".")[-1], "script":name} for name in tasks] 75 | for task in tasks: 76 | last_task = TaskResult.objects.filter( 77 | task_name=task["script"]).order_by("date_created").last() 78 | if last_task: 79 | task["id"] = last_task.task_id 80 | task["has_result"] = True 81 | task["status"] = last_task.status 82 | task["successfull"] = last_task.status == "SUCCESS" or last_task.status == "STARTED" 83 | task["date_created"] = last_task.date_created 84 | task["date_done"] = last_task.date_done 85 | task["result"] = last_task.result 86 | if last_task.result: 87 | task["input"] = json.loads(last_task.result).get("input") 88 | 89 | return tasks 90 | 91 | def task_output(request): 92 | ''' 93 | Returns a task output 94 | ''' 95 | 96 | task_id = request.GET.get('task_id') 97 | task = TaskResult.objects.get(id=task_id) 98 | 99 | if not task: 100 | return '' 101 | 102 | # task.result -> JSON Format 103 | return HttpResponse( task.result ) 104 | 105 | def task_log(request): 106 | ''' 107 | Returns a task LOG file (if located on disk) 108 | ''' 109 | 110 | task_id = request.GET.get('task_id') 111 | task = TaskResult.objects.get(id=task_id) 112 | task_log = 'NOT FOUND' 113 | 114 | if not task: 115 | return '' 116 | 117 | try: 118 | 119 | # Get logs file 120 | all_logs = [f for f in listdir(settings.CELERY_LOGS_DIR) if isfile(join(settings.CELERY_LOGS_DIR, f))] 121 | 122 | for log in all_logs: 123 | 124 | # Task HASH name is saved in the log name 125 | if task.task_id in log: 126 | 127 | with open( os.path.join( settings.CELERY_LOGS_DIR, log) ) as f: 128 | 129 | # task_log -> JSON Format 130 | task_log = f.readlines() 131 | 132 | break 133 | 134 | except Exception as e: 135 | 136 | task_log = json.dumps( { 'Error CELERY_LOGS_DIR: ' : str( e) } ) 137 | 138 | return HttpResponse(task_log) 139 | -------------------------------------------------------------------------------- /docs/blank.txt: -------------------------------------------------------------------------------- 1 | "coming soon" -------------------------------------------------------------------------------- /publish.txt: -------------------------------------------------------------------------------- 1 | python setup.py sdist 2 | 3 | twine check dist/* 4 | 5 | twine upload .\dist\THE_GENERATED_PACKAGE 6 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import os 2 | from setuptools import find_packages, setup 3 | 4 | with open(os.path.join(os.path.dirname(__file__), 'README.md')) as readme: 5 | README = readme.read() 6 | 7 | os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir))) 8 | 9 | setup( 10 | name='django-tasks-manager', 11 | version='1.0.1', 12 | zip_safe=False, 13 | packages=find_packages(), 14 | include_package_data=True, 15 | description='Simple Django & celery Integration', 16 | long_description=README, 17 | long_description_content_type="text/markdown", 18 | url='https://github.com/app-generator/django-tasks-manager', 19 | author='AppSeed.us', 20 | author_email='support@appseed.us', 21 | license='MIT License', 22 | install_requires=[ 23 | 'celery', 24 | 'redis', 25 | 'hiredis', 26 | 'django-celery-results', 27 | ], 28 | classifiers=[ 29 | 'Intended Audience :: Developers', 30 | 'Intended Audience :: System Administrators', 31 | 'License :: OSI Approved :: MIT License', 32 | 'Operating System :: OS Independent', 33 | 'Programming Language :: Python', 34 | 'Programming Language :: Python :: 2.6', 35 | 'Programming Language :: Python :: 2.7', 36 | 'Programming Language :: Python :: 3.2', 37 | 'Programming Language :: Python :: 3.3', 38 | 'Programming Language :: Python :: 3.4', 39 | 'Programming Language :: Python :: 3.5', 40 | 'Programming Language :: Python :: 3.6', 41 | 'Environment :: Web Environment', 42 | 'Topic :: Software Development', 43 | 'Topic :: Software Development :: User Interfaces', 44 | ], 45 | ) 46 | --------------------------------------------------------------------------------