├── .gitignore ├── Procfile ├── README.md ├── app.json ├── djangoproject ├── brokenapp │ ├── __init__.py │ ├── admin.py │ ├── apps.py │ ├── migrations │ │ └── __init__.py │ ├── models.py │ ├── tests.py │ └── views.py ├── djangoproject │ ├── __init__.py │ ├── logger.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py └── manage.py ├── requirements.txt └── runtime.txt /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | \#*\# 3 | venv 4 | *.pyc 5 | staticfiles 6 | .env 7 | *.sqlite3 8 | *.log 9 | -------------------------------------------------------------------------------- /Procfile: -------------------------------------------------------------------------------- 1 | web: gunicorn --pythonpath="$PWD/djangoproject" djangoproject.wsgi --log-file - -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | fail-nicely-django 2 | ================== 3 | 4 | An example Django project with nice logging settings for debugging failures. 5 | 6 | ![Bugs falling down](https://media.giphy.com/media/6WMXVZxdSIONW/giphy.gif) 7 | 8 | Features 9 | -------- 10 | 11 | - [x] stdout & rotating file logging 12 | - [x] timestamps in the log format 13 | - [ ] logs visible from runserver, gunicorn, uwsgi, systemd, honcho, Docker 14 | - [x] deploy to Heroku button which just works with `heroku logs` 15 | - [ ] show how to upgrade to Sentry 16 | 17 | Tested in Django 1.9, but should work since 1.3, though 1.9 made 18 | [some changes](https://docs.djangoproject.com/en/1.9/releases/1.9/#default-logging-changes-19) 19 | to the default Django loggers. 20 | 21 | Sample log file output: 22 | 23 | ``` 24 | 2016-04-05 22:12:32,984 [Thread-1 ] [INFO ] [djangoproject.logger] This is a manually logged INFO string. 25 | 2016-04-05 22:12:32,984 [Thread-1 ] [DEBUG] [djangoproject.logger] This is a manually logged DEBUG string. 26 | 2016-04-05 22:12:32,984 [Thread-1 ] [ERROR] [django.request ] Internal Server Error: / 27 | Traceback (most recent call last): 28 | File "/Users/kermit/.virtualenvs/fail-nicely-django/lib/python3.5/site-packages/django/core/handlers/base.py", line 149, in get_response 29 | response = self.process_exception_by_middleware(e, request) 30 | File "/Users/kermit/.virtualenvs/fail-nicely-django/lib/python3.5/site-packages/django/core/handlers/base.py", line 147, in get_response 31 | response = wrapped_callback(request, *callback_args, **callback_kwargs) 32 | File "/Users/kermit/projekti/git/fail-nicely-django/djangoproject/brokenapp/views.py", line 12, in brokenview 33 | raise Exception('This is an exception raised in a view.') 34 | Exception: This is an exception raised in a view. 35 | ``` 36 | 37 | 38 | Why? 39 | ---- 40 | 41 | Django by default seems to prefer not to write the things you log to it 42 | anywhere. It instead lets its loggers shortly meditate on the messages sent 43 | their way, after which they silently discard them. 44 | The messages are simply disintegrated... 45 | They evaporate into the circuitry's chasm of nothingness... They cease to be. 46 | 47 | 48 | It somehow seems odd to have “the default” and 49 | “it's probably not what you want” describe the same thing :smiley: ... 50 | 51 | > “If the disable_existing_loggers key in the LOGGING dictConfig is set to True 52 | (which is the default) then all loggers from the default configuration will be 53 | disabled. Disabled loggers are not the same as removed; the logger will still 54 | exist, but will silently discard anything logged to it, not even propagating 55 | entries to a parent logger. Thus you should be very careful using 56 | 'disable_existing_loggers': True; it’s probably not what you want. 57 | Instead, you can set disable_existing_loggers to False and redefine 58 | some or all of the default loggers [...].” 59 | > 60 | > — Excerpt From: The Official Django Documentation 61 | 62 | Second, if you do use loggers, the `runserver`/`gunicorn` process doesn't output 63 | log messages in your console. Things like Docker kind of religiously depend on 64 | the executable outputting its own log files to stdout, 65 | so why not do it this way? 66 | 67 | [Others](http://stackoverflow.com/questions/5739830/simple-log-to-file-example-for-django-1-3/7045981) 68 | seem to be missing "complete working example code" for effectively using 69 | Django's logging capabilities too. 70 | 71 | For these reason and after having 72 | [some problems of my own](https://github.com/benoitc/gunicorn/issues/1124#issuecomment-161990634) 73 | getting Django/Flask, `gunicorn` and `supervisord` 74 | to behave nicely with the log output, I started assembling 75 | the best options I found into a nice example configuration. This is still 76 | work in progress, so suggestions and patches are welcome :smiley:. 77 | 78 | 79 | Can I try it? 80 | ------------- 81 | 82 | To test the setup locally: 83 | 84 | python3 -m venv venv 85 | source ./venv/bin/activate 86 | pip install -r requirements.txt 87 | python djangoproject/manage.py migrate 88 | python djangoproject/manage.py runserver 89 | 90 | And in another tab admire your logs: 91 | 92 | tail -f djangoproject/djangoproject.log 93 | 94 | To trigger some errors and log messages 95 | just open/refresh . 96 | 97 | To see gunicorn output, issue `honcho start` 98 | and open/refresh 99 | 100 | You can also try it on Heroku: 101 | 102 | [![Deploy](https://www.herokucdn.com/deploy/button.png)](https://heroku.com/deploy) 103 | 104 | And you should be able to examine the logs after you refresh the page. 105 | 106 | heroku logs -t --app myherokuapp 107 | 108 | Open in your browser 109 | to generate some exceptions (substitute your real app name). 110 | 111 | 112 | Nice, I want this! 113 | ------------------ 114 | 115 | Cool, then just copy the 116 | [*djangoproject/djangoproject/logger.py*](https://github.com/metakermit/fail-nicely-django/blob/master/djangoproject/djangoproject/logger.py) 117 | file to your project (into the same folder where your *settings.py* is located) 118 | and add the following line to the bottom of your *settings.py* file: 119 | 120 | from .logger import LOGGING 121 | 122 | Then in every part of your project where you wanna log something manually, 123 | either import the logger like in 124 | [*djangoproject/brokenapp/views.py*](https://github.com/metakermit/fail-nicely-django/blob/master/djangoproject/brokenapp/views.py). 125 | Alternatively, if you want the logger name to be something other 126 | than `djangoproject.logger`, add to the top of every module 127 | where you want to log something: 128 | 129 | import logging 130 | log = logging.getLogger(__name__) 131 | 132 | Profit: 133 | 134 | log.info('this is very important to log') 135 | 136 | That's it, rock on! 137 | 138 | References 139 | ---------- 140 | 141 | - https://docs.djangoproject.com/en/1.9/topics/logging/ 142 | - https://docs.python.org/3.5/library/logging.config.html 143 | - http://stackoverflow.com/questions/5739830/simple-log-to-file-example-for-django-1-3 144 | - http://stackoverflow.com/questions/12942195/django-verbose-request-logging 145 | -------------------------------------------------------------------------------- /app.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "fail-nicely-django", 3 | "description": "An example Django project with nice logging settings for debugging failures.", 4 | "image": "heroku/python", 5 | "repository": "https://github.com/metakermit/fail-nicely-django", 6 | "keywords": ["python", "django" ], 7 | "addons": [] 8 | } 9 | -------------------------------------------------------------------------------- /djangoproject/brokenapp/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/metakermit/fail-nicely-django/471b9a2de54fb4629ff8cc98248169f68fe3db8a/djangoproject/brokenapp/__init__.py -------------------------------------------------------------------------------- /djangoproject/brokenapp/admin.py: -------------------------------------------------------------------------------- 1 | from django.contrib import admin 2 | 3 | # Register your models here. 4 | -------------------------------------------------------------------------------- /djangoproject/brokenapp/apps.py: -------------------------------------------------------------------------------- 1 | from django.apps import AppConfig 2 | 3 | 4 | class BrokenappConfig(AppConfig): 5 | name = 'brokenapp' 6 | -------------------------------------------------------------------------------- /djangoproject/brokenapp/migrations/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/metakermit/fail-nicely-django/471b9a2de54fb4629ff8cc98248169f68fe3db8a/djangoproject/brokenapp/migrations/__init__.py -------------------------------------------------------------------------------- /djangoproject/brokenapp/models.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | 3 | # Create your models here. 4 | -------------------------------------------------------------------------------- /djangoproject/brokenapp/tests.py: -------------------------------------------------------------------------------- 1 | from django.test import TestCase 2 | 3 | # Create your tests here. 4 | -------------------------------------------------------------------------------- /djangoproject/brokenapp/views.py: -------------------------------------------------------------------------------- 1 | from django.shortcuts import render 2 | from django.http import HttpResponse 3 | 4 | from djangoproject.logger import log 5 | 6 | # Create your views here. 7 | def brokenview(request): 8 | # first, intentionally log something 9 | log.info('This is a manually logged INFO string.') 10 | log.debug('This is a manually logged DEBUG string.') 11 | # then have the view raise an exception (e.g. something went wrong) 12 | raise Exception('This is an exception raised in a view.') 13 | #return HttpResponse('hello') 14 | -------------------------------------------------------------------------------- /djangoproject/djangoproject/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/metakermit/fail-nicely-django/471b9a2de54fb4629ff8cc98248169f68fe3db8a/djangoproject/djangoproject/__init__.py -------------------------------------------------------------------------------- /djangoproject/djangoproject/logger.py: -------------------------------------------------------------------------------- 1 | import os 2 | import logging 3 | 4 | from .settings import BASE_DIR 5 | from .settings import DEBUG 6 | 7 | # Usage in other modules: 8 | # 9 | # from djangoproject.logger import log 10 | # log.info('some output') 11 | # 12 | # Note, doing this manually in other modules results in nicer output: 13 | # 14 | # import logging 15 | # log = logging.getLogger(__name__) 16 | # log.info('some output') 17 | 18 | # the basic logger other apps can import 19 | log = logging.getLogger(__name__) 20 | 21 | # the minimum reported level 22 | if DEBUG: 23 | min_level = 'DEBUG' 24 | else: 25 | min_level = 'INFO' 26 | 27 | # the minimum reported level for Django's modules 28 | # optionally set to DEBUG to see database queries etc. 29 | # or set to min_level to control it using the DEBUG flag 30 | min_django_level = 'INFO' 31 | 32 | # logging dictConfig configuration 33 | LOGGING = { 34 | 'version': 1, 35 | 'disable_existing_loggers': False, # keep Django's default loggers 36 | 'formatters': { 37 | # see full list of attributes here: 38 | # https://docs.python.org/3/library/logging.html#logrecord-attributes 39 | 'verbose': { 40 | 'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s' 41 | }, 42 | 'simple': { 43 | 'format': '%(levelname)s %(message)s' 44 | }, 45 | 'timestampthread': { 46 | 'format': "%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s] [%(name)-20.20s] %(message)s", 47 | }, 48 | }, 49 | 'handlers': { 50 | 'logfile': { 51 | # optionally raise to INFO to not fill the log file too quickly 52 | 'level': min_level, # this level or higher goes to the log file 53 | 'class': 'logging.handlers.RotatingFileHandler', 54 | # IMPORTANT: replace with your desired logfile name! 55 | 'filename': os.path.join(BASE_DIR, 'djangoproject.log'), 56 | 'maxBytes': 50 * 10**6, # will 50 MB do? 57 | 'backupCount': 3, # keep this many extra historical files 58 | 'formatter': 'timestampthread' 59 | }, 60 | 'console': { 61 | 'level': min_level, # this level or higher goes to the console 62 | 'class': 'logging.StreamHandler', 63 | }, 64 | }, 65 | 'loggers': { 66 | 'django': { # configure all of Django's loggers 67 | 'handlers': ['logfile', 'console'], 68 | 'level': min_django_level, # this level or higher goes to the console 69 | 'propagate': False, # don't propagate further, to avoid duplication 70 | }, 71 | # root configuration – for all of our own apps 72 | # (feel free to do separate treatment for e.g. brokenapp vs. sth else) 73 | '': { 74 | 'handlers': ['logfile', 'console'], 75 | 'level': min_level, # this level or higher goes to the console, 76 | }, 77 | }, 78 | } 79 | -------------------------------------------------------------------------------- /djangoproject/djangoproject/settings.py: -------------------------------------------------------------------------------- 1 | """ 2 | Django settings for djangoproject project. 3 | 4 | Generated by 'django-admin startproject' using Django 1.9.4. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/1.9/topics/settings/ 8 | 9 | For the full list of settings and their values, see 10 | https://docs.djangoproject.com/en/1.9/ref/settings/ 11 | """ 12 | 13 | import os 14 | 15 | # Build paths inside the project like this: os.path.join(BASE_DIR, ...) 16 | BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 17 | PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__)) 18 | 19 | 20 | # Quick-start development settings - unsuitable for production 21 | # See https://docs.djangoproject.com/en/1.9/howto/deployment/checklist/ 22 | 23 | # SECURITY WARNING: keep the secret key used in production secret! 24 | SECRET_KEY = '&r&6%q+mmczm_#yad6z6&9y)!7%+1cjn-#t)zn%=#39&^#cum)' 25 | 26 | # SECURITY WARNING: don't run with debug turned on in production! 27 | DEBUG = True 28 | 29 | ALLOWED_HOSTS = [] 30 | 31 | 32 | # Application definition 33 | 34 | INSTALLED_APPS = [ 35 | 'django.contrib.admin', 36 | 'django.contrib.auth', 37 | 'django.contrib.contenttypes', 38 | 'django.contrib.sessions', 39 | 'django.contrib.messages', 40 | 'django.contrib.staticfiles', 41 | ] 42 | 43 | MIDDLEWARE_CLASSES = [ 44 | 'django.middleware.security.SecurityMiddleware', 45 | 'django.contrib.sessions.middleware.SessionMiddleware', 46 | 'django.middleware.common.CommonMiddleware', 47 | 'django.middleware.csrf.CsrfViewMiddleware', 48 | 'django.contrib.auth.middleware.AuthenticationMiddleware', 49 | 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', 50 | 'django.contrib.messages.middleware.MessageMiddleware', 51 | 'django.middleware.clickjacking.XFrameOptionsMiddleware', 52 | ] 53 | 54 | ROOT_URLCONF = 'djangoproject.urls' 55 | 56 | TEMPLATES = [ 57 | { 58 | 'BACKEND': 'django.template.backends.django.DjangoTemplates', 59 | 'DIRS': [], 60 | 'APP_DIRS': True, 61 | 'OPTIONS': { 62 | 'context_processors': [ 63 | 'django.template.context_processors.debug', 64 | 'django.template.context_processors.request', 65 | 'django.contrib.auth.context_processors.auth', 66 | 'django.contrib.messages.context_processors.messages', 67 | ], 68 | }, 69 | }, 70 | ] 71 | 72 | WSGI_APPLICATION = 'djangoproject.wsgi.application' 73 | 74 | 75 | # Database 76 | # https://docs.djangoproject.com/en/1.9/ref/settings/#databases 77 | 78 | DATABASES = { 79 | 'default': { 80 | 'ENGINE': 'django.db.backends.sqlite3', 81 | 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), 82 | } 83 | } 84 | 85 | 86 | # Password validation 87 | # https://docs.djangoproject.com/en/1.9/ref/settings/#auth-password-validators 88 | 89 | AUTH_PASSWORD_VALIDATORS = [ 90 | { 91 | 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', 92 | }, 93 | { 94 | 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', 95 | }, 96 | { 97 | 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', 98 | }, 99 | { 100 | 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', 101 | }, 102 | ] 103 | 104 | 105 | # Internationalization 106 | # https://docs.djangoproject.com/en/1.9/topics/i18n/ 107 | 108 | LANGUAGE_CODE = 'en-us' 109 | 110 | TIME_ZONE = 'UTC' 111 | 112 | USE_I18N = True 113 | 114 | USE_L10N = True 115 | 116 | USE_TZ = True 117 | 118 | 119 | # Static files (CSS, JavaScript, Images) 120 | # https://docs.djangoproject.com/en/1.9/howto/static-files/ 121 | 122 | STATIC_ROOT = os.path.join(PROJECT_ROOT, 'staticfiles') 123 | STATIC_URL = '/static/' 124 | 125 | from .logger import LOGGING 126 | -------------------------------------------------------------------------------- /djangoproject/djangoproject/urls.py: -------------------------------------------------------------------------------- 1 | """djangoproject URL Configuration 2 | 3 | The `urlpatterns` list routes URLs to views. For more information please see: 4 | https://docs.djangoproject.com/en/1.9/topics/http/urls/ 5 | Examples: 6 | Function views 7 | 1. Add an import: from my_app import views 8 | 2. Add a URL to urlpatterns: url(r'^$', views.home, name='home') 9 | Class-based views 10 | 1. Add an import: from other_app.views import Home 11 | 2. Add a URL to urlpatterns: url(r'^$', Home.as_view(), name='home') 12 | Including another URLconf 13 | 1. Import the include() function: from django.conf.urls import url, include 14 | 2. Add a URL to urlpatterns: url(r'^blog/', include('blog.urls')) 15 | """ 16 | from django.conf.urls import url 17 | from django.contrib import admin 18 | 19 | from brokenapp.views import brokenview 20 | 21 | urlpatterns = [ 22 | url(r'^admin/', admin.site.urls), 23 | url(r'^$', brokenview, name='brokenview') 24 | ] 25 | -------------------------------------------------------------------------------- /djangoproject/djangoproject/wsgi.py: -------------------------------------------------------------------------------- 1 | """ 2 | WSGI config for djangoproject project. 3 | 4 | It exposes the WSGI callable as a module-level variable named ``application``. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/1.9/howto/deployment/wsgi/ 8 | """ 9 | 10 | import os 11 | 12 | from django.core.wsgi import get_wsgi_application 13 | 14 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "djangoproject.settings") 15 | 16 | application = get_wsgi_application() 17 | -------------------------------------------------------------------------------- /djangoproject/manage.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import os 3 | import sys 4 | 5 | if __name__ == "__main__": 6 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "djangoproject.settings") 7 | 8 | from django.core.management import execute_from_command_line 9 | 10 | execute_from_command_line(sys.argv) 11 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | Django==1.11.28 2 | gunicorn==19.6.0 3 | honcho==0.7.1 4 | -------------------------------------------------------------------------------- /runtime.txt: -------------------------------------------------------------------------------- 1 | python-3.7.3 2 | --------------------------------------------------------------------------------