├── README.md ├── checklist-en.md ├── checklist-es.md ├── checklist-ko-kr.md └── checklist-pt-br.md /README.md: -------------------------------------------------------------------------------- 1 | [![alt text](https://s3-sa-east-1.amazonaws.com/www.vinta.com.br/images/dont_delete/devchecklist-logo.png "Devchecklists Logo")](https://devchecklists.com) 2 | 3 | # Celery Tasks Checklist 4 | 5 | ## Adding a New Language: 6 | * [ ] To add a new language you will need to create another markdown file containing the new [acronym language:](https://www.loc.gov/standards/iso639-2/php/code_list.php) 7 | * E.g. `checklist-pt.md`. 8 | * [ ] No further approval will be necessary; 9 | * [ ] The new checklist language should be visible on Devckecklists after you saved the file on your repository. 10 | 11 | ## Important Notes: 12 | * [ ] Markdown is the only way to write checklists on Devchecklists; 13 | * [ ] For tips about how to use markdown to write checklists, please take a look at these raw files: 14 | * [ ] [`checklist-en.md`](https://raw.githubusercontent.com/vintasoftware/devchecklists-template/master/checklist-en.md) sample markdown; 15 | * [ ] [`README.md`](https://raw.githubusercontent.com/vintasoftware/devchecklists-template/master/README.md) contained on this repository; 16 | * [ ] Another checklists markdowns from [Devchecklists](https://devchecklists.com). 17 | * [ ] Changes on the checklist repository will reflect on the current checklist of Devchecklists. 18 | 19 | ## Commercial Support 20 | [![alt text](https://avatars2.githubusercontent.com/u/5529080?s=200&v=4 "Vinta Logo")](https://vintasoftware.com) 21 | 22 | This project, as other Vinta open-source projects, is used in products of Vinta clients. We are always looking for exciting work, so if you need any commercial support, feel free to get in touch: contact@vinta.com.br 23 | 24 | Copyright (c) 2018 Vinta Serviços e Soluções Tecnológicas Ltda. 25 | [MIT License](LICENSE.txt) 26 | -------------------------------------------------------------------------------- /checklist-en.md: -------------------------------------------------------------------------------- 1 | ## 1. Best Practices 2 | * [ ] Prefer [RabbitMQ](https://www.rabbitmq.com/) or [Redis](https://redis.io/) as broker (never use a relational database as production broker). 3 | * [ ] Do not use complex objects in task as parameters. E.g.: Avoid Django model objects: 4 | ``` 5 | # Good 6 | @app.task 7 | def my_task(user_id): 8 | user = User.objects.get(id=user_id) 9 | print(user.name) 10 | # ... 11 | ``` 12 | 13 | ``` 14 | # Bad 15 | @app.task 16 | def my_task(user): 17 | print(user.name) 18 | # ... 19 | ``` 20 | * [ ] Do not wait for other tasks inside a task. 21 | * [ ] Prefer idempotent tasks: 22 | * "Idempotence is the property of certain operations in mathematics and computer science, that can be applied multiple times without changing the result beyond the initial application." - Wikipedia. 23 | * [ ] Prefer atomic tasks: 24 | * "An operation (or set of operations) is atomic ... if it appears to the rest of the system to occur instantaneously. Atomicity is a guarantee of isolation from concurrent processes. Additionally, atomic operations commonly have a succeed-or-fail definition—they either successfully change the state of the system, or have no apparent effect." - Wikipedia. 25 | * [ ] Retry when possible. But make sure tasks are idempotent and atomic before doing so. [(Retrying)](http://docs.celeryproject.org/en/latest/userguide/tasks.html#retrying) 26 | * [ ] Set `retry_limit` to avoid broken tasks to keep retrying forever. 27 | * [ ] Exponentially backoff if things look like they are not going to get fixed soon. Throw in a random factor to avoid cluttering services: 28 | ``` 29 | def exponential_backoff(task_self): 30 | minutes = task_self.default_retry_delay / 60 31 | rand = random.uniform(minutes, minutes * 1.3) 32 | return int(rand ** task_self.request.retries) * 60 33 | 34 | # in the task 35 | raise self.retry(exc=e, countdown=exponential_backoff(self)) 36 | ``` 37 | * [ ] Use [`autoretry_for`](http://docs.celeryproject.org/en/master/userguide/tasks.html#automatic-retry-for-known-exceptions) to reduce the boilerplate code for retrying tasks. 38 | * [ ] Use [`retry_backoff`](http://docs.celeryproject.org/en/master/userguide/tasks.html#Task.retry_backoff) to reduce the boilerplate code when doing exponention backoff. 39 | * [ ] For tasks that require high level of reliability, use `acks_late` in combination with `retry`. Again, make sure tasks are idempotent and atomic. [(Should I use retry or acks_late?)](http://docs.celeryproject.org/en/latest/faq.html#faq-acks-late-vs-retry) 40 | * [ ] Set hard and soft time limits. Recover gracefully if things take longer than expected: 41 | ``` 42 | from celery.exceptions import SoftTimeLimitExceeded 43 | 44 | @app.task(task_time_limit=60, task_soft_time_limit=45) 45 | def my_task(): 46 | try: 47 | something_possibly_long() 48 | except SoftTimeLimitExceeded: 49 | recover() 50 | ``` 51 | * [ ] Use multiple queues to have more control over throughput and make things more scalable. [(Routing Tasks)](http://docs.celeryproject.org/en/latest/userguide/routing.html) 52 | * [ ] Extend the base task class to define default behaviour. [(Custom Task Classes)](http://docs.celeryproject.org/en/latest/userguide/tasks.html#custom-task-classes) 53 | * [ ] Use canvas features to control task flows and deal with concurrency. [(Canvas: Designing Work-flows)](http://docs.celeryproject.org/en/latest/userguide/canvas.html) 54 | 55 | ## 2. Monitoring & Tests 56 | * [ ] Log as much as possible. Use `get_task_logger` to automatically get the task name and unique id as part of the logs. 57 | * [ ] In case of failure, make sure stack traces get logged and people get notified (services like [Sentry](https://sentry.io) are a good idea). 58 | * [ ] Monitor activity using Flower. [(Flower: Real-time Celery web-monitor)](http://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor) 59 | * [ ] Use `task_always_eager` to test your tasks are geting called. 60 | 61 | ## 3. Resources to check 62 | * [ ] [Celery: an overview of the architecture and how it works](https://www.vinta.com.br/blog/2017/celery-overview-archtecture-and-how-it-works/) by [Vinta.](https://www.vinta.com.br/) 63 | * [ ] [Celery in the wild: tips and tricks to run async tasks in the real world](https://www.vinta.com.br/blog/2018/celery-wild-tips-and-tricks-run-async-tasks-real-world/) by [Vinta.](https://www.vinta.com.br/) 64 | * [ ] [Celery Best Practices](https://blog.balthazar-rouberol.com/celery-best-practices) by Balthazar Rouberol. 65 | * [ ] [Dealing with resource-consuming tasks on Celery](https://www.vinta.com.br/blog/2018/dealing-resource-consuming-tasks-celery/) by [Vinta.](https://www.vinta.com.br/) 66 | * [ ] [Tips and Best Practices](http://celery.readthedocs.io/en/latest/userguide/tasks.html#tips-and-best-practices) from the official documentation. 67 | * [ ] [Task Queues](https://www.fullstackpython.com/task-queues.html) by Full Stack Python 68 | [Flower: Real-time Celery web-monitor](http://celery.readthedocs.io/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor) from the official documentation. 69 | * [ ] [Celery Best Practices: practical approach](https://khashtamov.com/en/celery-best-practices-practical-approach/) by Adil. 70 | * [ ] [3 GOTCHAS FOR CELERY](https://wiredcraft.com/blog/3-gotchas-for-celery/) from Wiredcraft. 71 | * [ ] [CELERY - BEST PRACTICES](https://denibertovic.com/posts/celery-best-practices/) by Deni Bertovic. 72 | * [ ] [Hacker News thread on the above post.](https://news.ycombinator.com/item?id=7909201) 73 | * [ ] [[video] Painting on a Distributed Canvas: An Advanced Guide to Celery Workflows](https://www.youtube.com/watch?v=XoMu8vhdc-A) by David Gouldin. 74 | * [ ] [Celery in Production](https://www.caktusgroup.com/blog/2014/09/29/celery-production/) by Dan Poirier from Caktus Group. 75 | * [ ] [[video] Implementing Celery, Lessons Learned](https://www.youtube.com/watch?v=hmtSe0yPi6I) by Michael Robellard. 76 | * [ ] [[video] Advanced Celery](https://www.youtube.com/watch?v=gpKMwPoldak&t=1416s) by Ask Solem Hoel. 77 | -------------------------------------------------------------------------------- /checklist-es.md: -------------------------------------------------------------------------------- 1 | ## 1. Buenas Prácticas 2 | 3 | * [ ] Prefiera [RabbitMQ](https://www.rabbitmq.com/) o [Redis](https://redis.io/) como broker (nunca use una base de datos relacional como broker en producción. 4 | * [ ] No use objetos complejos como parámetros de las tareas. Por ejemplo: evite usar instancias de modelos de Django. 5 | ``` 6 | # Correcto 7 | @app.task 8 | def my_task(user_id): 9 | user = User.objects.get(id=user_id) 10 | print(user.name) 11 | # ... 12 | ``` 13 | 14 | ``` 15 | # Incorrecto 16 | @app.task 17 | def my_task(user): 18 | print(user.name) 19 | # ... 20 | ``` 21 | * [ ] No espere por otras tareas dentro de una tarea. 22 | * [ ] Prefiera tareas idempotentes. 23 | * "En matemática y lógica, la idempotencia es la propiedad para realizar una acción determinada varias veces y aun así conseguir el mismo resultado que se obtendría si se realizase una sola vez." - Wikipedia 24 | * [ ] Prefiera tareas atómicas. 25 | * "Transación Atómica, en Ciencias de la Computación, es una operación, o conjunto de operaciones, en una base de datos, o en cualquier otro sistema computacional, que debe ser ejecutada completamente en caso de éxito o ser abortada completamente en caso de error." - Wikipedia 26 | * [ ] Vuelva a procesar mientras sea posible; pero asegurese de que las tareas sea atómicas antes de hacerlo. [(Ver documentación: Retrying)](http://docs.celeryproject.org/en/latest/userguide/tasks.html#retrying) 27 | * [ ] Defina `retry_limit` para evitar que tareas dañadas se mantengan intentando procesar infinitamente. 28 | * [ ] Use retroceso exponencial si las cosas parecen no arreglarse pronto. Use un factor aleatorio para evitar desordenar los servicios. 29 | ``` 30 | def exponential_backoff(task_self): 31 | minutes = task_self.default_retry_delay / 60 32 | rand = random.uniform(minutes, minutes * 1.3) 33 | return int(rand ** task_self.request.retries) * 60 34 | 35 | # in the task 36 | raise self.retry(exc=e, countdown=exponential_backoff(self)) 37 | ``` 38 | * [ ] Para tareas que requieren un alto nivel de confiabilidad, use `acks_late` en combinación con `retry`. De nuevo, asegúrese que las tareas son edempotentes y atómicas. [(Ver documentación: Should I use retry or acks_late?)](http://docs.celeryproject.org/en/latest/faq.html#faq-acks-late-vs-retry) 39 | * [ ] Establezca límites de tiempo duros y suaves. Recupérese elegantemente si las cosas toman más de lo esperado. 40 | ``` 41 | from celery.exceptions import SoftTimeLimitExceeded 42 | 43 | @app.task(task_time_limit=60, task_soft_time_limit=45) 44 | def my_task(): 45 | try: 46 | something_possibly_long() 47 | except SoftTimeLimitExceeded: 48 | recover() 49 | ``` 50 | * [ ] Use múltiples colas para tener más control sobre el rendimiento y hacer las cosas más escalables. [(Ver documentación: Routing Tasks)](http://docs.celeryproject.org/en/latest/userguide/routing.html) 51 | * [ ] Extienda la clase de tarea básica para definir su propia lógica por defecto.[(Ver documentación: Custom Task Classes)](http://docs.celeryproject.org/en/latest/userguide/tasks.html#custom-task-classes) 52 | * [ ] Use el lienzo para controlar flujos de tareas y manejar la concurrencia. [(Ver documentación: Canvas Designing Work-flows)](http://docs.celeryproject.org/en/latest/userguide/canvas.html) 53 | 54 | ## 2. Monitoreo y Pruebas 55 | * [ ] Genere registros lo más que pueda. Use `get_task_logger` para automáticamente obtener el nombre de la tarea y su identificador único como parte del registro. 56 | * [ ] En caso de fallo, asegúrese que los seguimientos de pila sean registrados y que las personas sean notificadas. (servicios como [Sentry](https://sentry.io) are a good idea). 57 | * [ ] Use Flower [(Flower: Real-time Celery web-monitor)](http://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor) para monitorear la actividad de las tareas. 58 | * [ ] Use `task_always_eager` para probar que sus tareas están siendo ejecutadas. 59 | 60 | ## 3. Recursos 61 | * [ ] [Celery: an overview of the architecture and how it works](https://www.vinta.com.br/blog/2017/celery-overview-archtecture-and-how-it-works/) por [Vinta](https://www.vinta.com.br/) 62 | * [ ] [Celery in the wild: tips and tricks to run async tasks in the real world](https://www.vinta.com.br/blog/2018/celery-wild-tips-and-tricks-run-async-tasks-real-world/) por [Vinta](https://www.vinta.com.br/) 63 | * [ ] [Dealing with resource-consuming tasks on Celery](https://www.vinta.com.br/blog/2018/dealing-resource-consuming-tasks-celery/) por [Vinta](https://www.vinta.com.br/) 64 | * [ ] [Trucos y Mejores Prácticas](http://celery.readthedocs.io/en/latest/userguide/tasks.html#tips-and-best-practices) de la documentación oficial. 65 | * [ ] [Colas de Tareas](https://www.fullstackpython.com/task-queues.html) por Full Stack Python 66 | * [ ] [Flower: Monitor de Celery en tiempo real](http://celery.readthedocs.io/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor) de la documentación oficial. 67 | * [ ] [Celery - Mejores prácticas: enfoque práctico](https://khashtamov.com/en/celery-best-practices-practical-approach/) por Adil 68 | * [ ] [3 pilladas para Celery](https://wiredcraft.com/blog/3-gotchas-for-celery/) por Wiredcraft 69 | * [ ] [CELERY - MEJORES PRÁCTICAS](https://denibertovic.com/posts/celery-best-practices/) por Deni Bertovic 70 | * [ ] [Hilo en Hacker News sobre el post anterior](https://news.ycombinator.com/item?id=7909201) 71 | * [ ] [[video] Pintando en un lienzo distribuido: Una guía avanzada a los flujos de trabajo de Celery](https://www.youtube.com/watch?v=XoMu8vhdc-A) por David Gouldin 72 | * [ ] [Celery en Producción](https://www.caktusgroup.com/blog/2014/09/29/celery-production/) por Dan Poirier de Caktus Group 73 | * [ ] [[video] Implementando Celery. Lecciones aprendidas](https://www.youtube.com/watch?v=hmtSe0yPi6I) por Michael Robellard 74 | * [ ] [[video] Celery Avanzado](https://www.youtube.com/watch?v=gpKMwPoldak&t=1416s) por Ask Solem Hoel 75 | * [ ] [Mejores pácticas para Celery](https://blog.balthazar-rouberol.com/celery-best-practices) por Balthazar Rouberol 76 | -------------------------------------------------------------------------------- /checklist-ko-kr.md: -------------------------------------------------------------------------------- 1 | ## 1. Best Practices 2 | * [ ] [RabbitMQ](https://www.rabbitmq.com/) 또는 [Redis](https://redis.io/)를 브로커(broker)로 쓰세요. (상용 환경에서 RDB를 브로커로 쓰지 마세요.) 3 | * [ ] 복잡한 객체(object)를 태스크(task)에 파라미터(parameter)로 사용하지 마세요. (예를 들면 Django의 모델 객체) 4 | ``` 5 | # 좋은 예시 6 | @app.task 7 | def my_task(user_id): 8 | user = User.objects.get(id=user_id) 9 | print(user.name) 10 | # ... 11 | ``` 12 | 13 | ``` 14 | # 나쁜 예시 15 | @app.task 16 | def my_task(user): 17 | print(user.name) 18 | # ... 19 | ``` 20 | * [ ] 태스크 안에서 태스크를 기다리게 하지 마세요. 21 | * [ ] 태스크가 멱등성을 가지도록 하세요. 22 | > "멱등성(冪等性, 영어: idempotence)은 수학이나 전산학에서 연산의 한 성질을 나타내는 것으로, 연산을 여러 번 적용하더라도 결과가 달라지지 않는 성질을 의미한다" - Wikipedia 23 | * [ ] 태스크가 원자성을 가지도록 하세요. 24 | > "어떠한 작업이 실행될때 언제나 완전하게 진행되어 종료되거나, 그럴 수 없는 경우 실행을 하지 않는 경우를 말한다. 원자성을 가지는 작업은 실행되어 진행되다가 종료하지 않고 중간에서 멈추는 경우는 있을 수 없다." - Wikipedia 25 | * [ ] 가능하면 재시도하세요. 단, 실행전에 멱등성과 원자성을 가지는 것을 확실히 하세요. 26 | [(Retrying)](http://docs.celeryproject.org/en/latest/userguide/tasks.html#retrying) 27 | * [ ] `retry_limit`를 설정해서 문제 있는 태스크가 계속 재시도하지 않도록 하세요. 28 | * [ ] 태스크가 금방 고쳐질 것 같지 않다면 재시도 횟수를 지수적으로 감소시키세요.([exponentially backoff](https://en.wikipedia.org/wiki/Exponential_backoff)) 혼란스러운 서비스를 피하려면 임의의 요소를 사용하세요. 29 | 30 | ``` 31 | def exponential_backoff(task_self): 32 | minutes = task_self.default_retry_delay / 60 33 | rand = random.uniform(minutes, minutes * 1.3) 34 | return int(rand ** task_self.request.retries) * 60 35 | 36 | # in the task 37 | raise self.retry(exc=e, countdown=exponential_backoff(self)) 38 | ``` 39 | * [ ] 재시도 태스크에서 boilerplate code를 줄이기 위해서 [`autoretry_for`](http://docs.celeryproject.org/en/master/userguide/tasks.html#automatic-retry-for-known-exceptions) 를 사용하세요. 40 | * [ ] 재시도 횟수를 지수적으로 감소시킬때 boilerplate code를 줄이기 위해서 [`retry_backoff`](http://docs.celeryproject.org/en/master/userguide/tasks.html#Task.retry_backoff) 를 사용하세요. 41 | * [ ] 높은 신뢰도가 필요한 태스크에서는 `acks_late`와 `retry`를 함께 사용하세요. 물론 멱등성과 원자성을 가지도록 해야합니다. [(Should I use retry or acks_late?)](http://docs.celeryproject.org/en/latest/faq.html#faq-acks-late-vs-retry) 42 | * [ ] 최대 제한([hard limit]())과 경고 제한([soft limit]()) 시간을 설정하세요. 태스크가 예상보다 오래 걸린다면 완만하게 복구하세요. 43 | ``` 44 | from celery.exceptions import SoftTimeLimitExceeded 45 | 46 | @app.task(task_time_limit=60, task_soft_time_limit=45) 47 | def my_task(): 48 | try: 49 | something_possibly_long() 50 | except SoftTimeLimitExceeded: 51 | recover() 52 | ``` 53 | * [ ] 처리량과 확장성을 제어하기 위해서 큐를 여러개 사용하세요. [(Routing Tasks)](http://docs.celeryproject.org/en/latest/userguide/routing.html) 54 | * [ ] 기본적인 작업을 정의하려면 기본 태스크 클래스를 만들어 상속하세요. [(Custom Task Classes)](http://docs.celeryproject.org/en/latest/userguide/tasks.html#custom-task-classes) 55 | * [ ] 태스크 흐름과 동시성을 제어하기 위해서 Canvas 기능을 사용하세요. [(Canvas: Designing Work-flows)](http://docs.celeryproject.org/en/latest/userguide/canvas.html) 56 | 57 | ## 2. Monitoring & Tests 58 | * [ ] 가능한 한 더 많은 로그를 남기세요. `get_task_logger`를 이용해서 태스크 이름과 고유 id를 자동으로 가져오세요. 59 | * [ ] 실패했을 때 stack trace를 남기고 사람이 알 수 있도록 하세요. ([Sentry](https://sentry.io) 같은 서비스를 쓰는게 좋아요) 60 | * [ ] [Flower: Real-time Celery web-monitor](http://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor)를 사용해서 샐러리 모니터링을 하세요. 61 | * [ ] 태스크 호출을 테스트할 때 `task_always_eager`를 사용하세요. 62 | 63 | ## 3. Resources 64 | * [ ] [Tips and Best Practices](http://celery.readthedocs.io/en/latest/userguide/tasks.html#tips-and-best-practices) from the official documentation. 65 | * [ ] [Task Queues](https://www.fullstackpython.com/task-queues.html) by Full Stack Python 66 | * [ ] [Flower: Real-time Celery web-monitor](http://celery.readthedocs.io/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor) from the official documentation. 67 | * [ ] [Celery Best Practices: practical approach](https://khashtamov.com/en/celery-best-practices-practical-approach/) by Adil 68 | * [ ] [3 GOTCHAS FOR CELERY](https://wiredcraft.com/blog/3-gotchas-for-celery/) from Wiredcraft 69 | * [ ] [CELERY - BEST PRACTICES](https://denibertovic.com/posts/celery-best-practices/) by Deni Bertovic 70 | * [ ] [Hacker News thread on the above post](https://news.ycombinator.com/item?id=7909201) 71 | * [ ] [[video] Painting on a Distributed Canvas: An Advanced Guide to Celery Workflows](https://www.youtube.com/watch?v=XoMu8vhdc-A) by David Gouldin 72 | * [ ] [Celery in Production](https://www.caktusgroup.com/blog/2014/09/29/celery-production/) by Dan Poirier from Caktus Group 73 | * [ ] [[video] Implementing Celery, Lessons Learned](https://www.youtube.com/watch?v=hmtSe0yPi6I) by Michael Robellard 74 | * [ ] [[video] Advanced Celery](https://www.youtube.com/watch?v=gpKMwPoldak&t=1416s) by Ask Solem Hoel 75 | * [ ] [Celery Best Practices](https://blog.balthazar-rouberol.com/celery-best-practices) by Balthazar Rouberol 76 | -------------------------------------------------------------------------------- /checklist-pt-br.md: -------------------------------------------------------------------------------- 1 | ## 1. Boas Práticas 2 | 3 | * [ ] Prefira [RabbitMQ](https://www.rabbitmq.com/) ou [Redis](https://redis.io/) como broker (nunca use banco de dados relacional como broker de produção) 4 | * [ ] Não use objetos complexos como parametros em tarefas. Exemplo: Evite objetos Django model: 5 | ``` 6 | # Bom 7 | @app.task 8 | def my_task(user_id): 9 | user = User.objects.get(id=user_id) 10 | print(user.name) 11 | # ... 12 | ``` 13 | 14 | ``` 15 | # Ruim 16 | @app.task 17 | def my_task(user): 18 | print(user.name) 19 | # ... 20 | ``` 21 | * [ ] Não espere por outras tarefas dentro de uma tarefa. 22 | * [ ] Prefira tarefas indempotentes. 23 | * "Em matemática e ciência da computação, a idempotência é a propriedade que algumas operações têm de poderem ser aplicadas várias vezes sem que o valor do resultado se altere após a aplicação inicial." - Wikipedia 24 | * [ ] Prefira tarefas atômicas. 25 | * "Transação Atômica, em ciência da computação, é uma operação, ou conjunto de operações, em uma base de dados, ou em qualquer outro sistema computacional, que deve ser executada completamente em caso de sucesso, ou ser abortada completamente em caso de erro." - Wikipedia 26 | * [ ] Tente novamente quando possível. Mas tenha certeza que as tarefas são indempotentes e atômicos antes de fazê-lo. [(Retrying)](http://docs.celeryproject.org/en/latest/userguide/tasks.html#retrying) 27 | * [ ] Defina `retry_limit` para evitar tarefas quebradas de fazer tentativas infinitas. 28 | * [ ] Exponenciamente caia fora se coisas parecerem que não vão ser corrigidos logo. Jogue um fator randômico para evitar confusão de serviços. 29 | ``` 30 | def exponencial_backoff(task_self): 31 | minutes = task_self.default_retry_delay / 60 32 | rand = random.uniform(minutes, minutes * 1.3) 33 | return int(rand ** task_self.request.retries) * 60 34 | 35 | # na tarefa 36 | raise self.retry(exc=e, countdown=exponencial_backoff(self)) 37 | ``` 38 | * [ ] Para tarefas que requere alto nível de confiabilidade, use `acks_late` em combinação com `retry`. Novamente, tenha certeza de que são tarefas idempotentes e atômicas. [(Should I use retry or acks_late?)](http://docs.celeryproject.org/en/latest/faq.html#faq-acks-late-vs-retry) 39 | * [ ] Defina tempos de limite duros e suaves. Recupere graciosamente se as coisas demorarem mais tempo que o esperado. 40 | ``` 41 | from celery.exceptions import SoftTimeLimitExceeded 42 | 43 | @app.task(task_time_limit=60, task_soft_time_limit=45) 44 | def my_task(): 45 | try: 46 | something_possibly_long() 47 | except SoftTimeLimitExceeded: 48 | recover() 49 | ``` 50 | * [ ] Use múltiplas filas para ter mais controle sobre a taxa de transferência e gerar mais escalabilidade. [(Routing Tasks)](http://docs.celeryproject.org/en/latest/userguide/routing.html) 51 | * [ ] Estenda a classe de tarefa base para definir comportamentos padrão. [(Custom Task Classes)](http://docs.celeryproject.org/en/latest/userguide/tasks.html#custom-task-classes) 52 | * [ ] Use recursos de `canvas` para controlar fluxos de tarefas e lidar com concorrência. [(Canvas: Designing Work-flows)](http://docs.celeryproject.org/en/latest/userguide/canvas.html) 53 | 54 | ## 2. Monitoração & Testes 55 | 56 | * [ ] Faça logging na medida do possível. Use `get_task_logger` para automaticamente adicionar o nome e o id único da sua task aos logs. 57 | * [ ] No caso de falha, tenha certeza de que `stack traces` sejam logados e que pessoas sejam notificadas (serviços como [Sentry](https://sentry.io) é uma boa idéia). 58 | * [ ] Monitore atividades usando Flower. [(Flower: Real-time Celery web-monitor)](http://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor) 59 | * [ ] Use `task_aways_eager` para testar suas tarefas que estão sendo chamadas. 60 | 61 | ## 3. Artigos interessantes: 62 | * [ ] [Celery: an overview of the architecture and how it works](https://www.vinta.com.br/blog/2017/celery-overview-archtecture-and-how-it-works/) por [Vinta](https://www.vinta.com.br/) 63 | * [ ] [Celery in the wild: tips and tricks to run async tasks in the real world](https://www.vinta.com.br/blog/2018/celery-wild-tips-and-tricks-run-async-tasks-real-world/) por [Vinta](https://www.vinta.com.br/) 64 | * [ ] [Dealing with resource-consuming tasks on Celery](https://www.vinta.com.br/blog/2018/dealing-resource-consuming-tasks-celery/) por [Vinta](https://www.vinta.com.br/) 65 | * [ ] [Dicas e Boas Práticas](http://celery.readthedocs.io/en/latest/userguide/tasks.html#tips-and-best-practices) da documentação oficial. 66 | * [ ] [Task Queues](https://www.fullstackpython.com/task-queues.html) por Full Stack Python 67 | * [ ] [Flower: Real-time Celery web-monitor](http://celery.readthedocs.io/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor) da documentação oficial 68 | * [ ] [Celery Best Practices: practical approach](https://khashtamov.com/en/celery-best-practices-practical-approach/) por Adil 69 | * [ ] [3 GOTCHAS FOR CELERY](https://wiredcraft.com/blog/3-gotchas-for-celery/) do Wiredcraft 70 | * [ ] [CELERY * [ ] BEST PRACTICES](https://denibertovic.com/posts/celery-best-practices/) por Deni Bertovic 71 | * [ ] [Hacker News thread on the above post](https://news.ycombinator.com/item?id=7909201) 72 | * [ ] [[video] Painting on a Distributed Canvas: An Advanced Guide to Celery Workflows](https://www.youtube.com/watch?v=XoMu8vhdc-A) por David Gouldin 73 | * [ ] [Celery in Production](https://www.caktusgroup.com/blog/2014/09/29/celery-production/) por Dan Poirier do Caktus Group 74 | * [ ] [[video] Implementing Celery, Lessons Learned](https://www.youtube.com/watch?v=hmtSe0yPi6I) por Michael Robellard 75 | * [ ] [[video] Advanced Celery](https://www.youtube.com/watch?v=gpKMwPoldak&t=1416s) por Ask Solem Hoel 76 | * [ ] [Celery Best Practices](https://blog.balthazar-rouberol.com/celery-best-practices) por Balthazar Rouberol 77 | --------------------------------------------------------------------------------