├── .github └── workflows │ └── ci.yml ├── .gitignore ├── LICENSE ├── MANIFEST.in ├── Makefile ├── README.rst ├── json_log_formatter └── __init__.py ├── pyproject.toml ├── requirements.txt ├── setup.py ├── tests.py └── tox.ini /.github/workflows/ci.yml: -------------------------------------------------------------------------------- 1 | on: 2 | push: 3 | branches: 4 | - master 5 | pull_request: 6 | branches: 7 | - master 8 | 9 | jobs: 10 | test: 11 | runs-on: ubuntu-latest 12 | strategy: 13 | matrix: 14 | python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"] 15 | steps: 16 | - uses: actions/checkout@v4 17 | - name: Set up Python ${{ matrix.python-version }} 18 | uses: actions/setup-python@v5 19 | with: 20 | python-version: ${{ matrix.python-version }} 21 | - name: Install dependencies 22 | run: | 23 | python -m pip install --upgrade pip 24 | pip install flake8 pytest ujson simplejson django 25 | - name: Lint with flake8 26 | run: | 27 | flake8 . --count --show-source --max-complexity=10 --max-line-length=127 --statistics 28 | - name: Test with pytest 29 | run: | 30 | pytest -s tests.py 31 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | .DS_Store 3 | __pycache__ 4 | /MANIFEST 5 | /dist 6 | /.tox 7 | /.cache 8 | /JSON_log_formatter.egg-info 9 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2025 Marsel Mavletkulov 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.rst 2 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | test: 2 | tox 3 | 4 | pypi: 5 | python -m build 6 | python -m twine upload dist/* 7 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ===================== 2 | JSON log formatter 🪵 3 | ===================== 4 | 5 | The library helps you to store logs in JSON format. Why is it important? 6 | Well, it facilitates integration with **Logstash**. 7 | 8 | Usage example: 9 | 10 | .. code-block:: python 11 | 12 | import logging 13 | 14 | import json_log_formatter 15 | 16 | formatter = json_log_formatter.JSONFormatter() 17 | 18 | json_handler = logging.FileHandler(filename='/var/log/my-log.json') 19 | json_handler.setFormatter(formatter) 20 | 21 | logger = logging.getLogger('my_json') 22 | logger.addHandler(json_handler) 23 | logger.setLevel(logging.INFO) 24 | 25 | logger.info('Sign up', extra={'referral_code': '52d6ce'}) 26 | 27 | try: 28 | raise ValueError('something wrong') 29 | except ValueError: 30 | logger.error('Request failed', exc_info=True) 31 | 32 | The log file will contain the following log record (inline). 33 | 34 | .. code-block:: json 35 | 36 | { 37 | "message": "Sign up", 38 | "time": "2015-09-01T06:06:26.524448", 39 | "referral_code": "52d6ce" 40 | } 41 | { 42 | "message": "Request failed", 43 | "time": "2015-09-01T06:06:26.524449", 44 | "exc_info": "Traceback (most recent call last): ..." 45 | } 46 | 47 | If you use a log collection and analysis system, 48 | you might need to include the built-in 49 | `log record attributes `_ 50 | with ``VerboseJSONFormatter``. 51 | 52 | .. code-block:: python 53 | 54 | json_handler.setFormatter(json_log_formatter.VerboseJSONFormatter()) 55 | logger.error('An error has occured') 56 | 57 | .. code-block:: json 58 | 59 | { 60 | "filename": "tests.py", 61 | "funcName": "test_file_name_is_testspy", 62 | "levelname": "ERROR", 63 | "lineno": 276, 64 | "module": "tests", 65 | "name": "my_json", 66 | "pathname": "/Users/bob/json-log-formatter/tests.py", 67 | "process": 3081, 68 | "processName": "MainProcess", 69 | "stack_info": null, 70 | "thread": 4664270272, 71 | "threadName": "MainThread", 72 | "message": "An error has occured", 73 | "time": "2021-07-04T21:05:42.767726" 74 | } 75 | 76 | If you need to flatten complex objects as strings, use ``FlatJSONFormatter``. 77 | 78 | .. code-block:: python 79 | 80 | json_handler.setFormatter(json_log_formatter.FlatJSONFormatter()) 81 | logger.error('An error has occured') 82 | 83 | logger.info('Sign up', extra={'request': WSGIRequest({ 84 | 'PATH_INFO': 'bogus', 85 | 'REQUEST_METHOD': 'bogus', 86 | 'CONTENT_TYPE': 'text/html; charset=utf8', 87 | 'wsgi.input': BytesIO(b''), 88 | })}) 89 | 90 | .. code-block:: json 91 | 92 | { 93 | "message": "Sign up", 94 | "time": "2024-10-01T00:59:29.332888+00:00", 95 | "request": "" 96 | } 97 | 98 | JSON libraries 99 | -------------- 100 | 101 | You can use **ujson** or **simplejson** instead of built-in **json** library. 102 | 103 | .. code-block:: python 104 | 105 | import json_log_formatter 106 | import ujson 107 | 108 | formatter = json_log_formatter.JSONFormatter() 109 | formatter.json_lib = ujson 110 | 111 | Note, **ujson** doesn't support ``dumps(default=f)`` argument: 112 | if it can't serialize an attribute, it might fail with ``TypeError`` or skip an attribute. 113 | 114 | Django integration 115 | ------------------ 116 | 117 | Here is an example of how the JSON formatter can be used with Django. 118 | 119 | .. code-block:: python 120 | 121 | LOGGING['formatters']['json'] = { 122 | '()': 'json_log_formatter.JSONFormatter', 123 | } 124 | LOGGING['handlers']['json_file'] = { 125 | 'level': 'INFO', 126 | 'class': 'logging.FileHandler', 127 | 'filename': '/var/log/my-log.json', 128 | 'formatter': 'json', 129 | } 130 | LOGGING['loggers']['my_json'] = { 131 | 'handlers': ['json_file'], 132 | 'level': 'INFO', 133 | } 134 | 135 | Let's try to log something. 136 | 137 | .. code-block:: python 138 | 139 | import logging 140 | 141 | logger = logging.getLogger('my_json') 142 | 143 | logger.info('Sign up', extra={'referral_code': '52d6ce'}) 144 | 145 | Custom formatter 146 | ---------------- 147 | 148 | You will likely need a custom log formatter. For instance, you want to log 149 | a user ID, an IP address and ``time`` as ``django.utils.timezone.now()``. 150 | To do so you should override ``JSONFormatter.json_record()``. 151 | 152 | .. code-block:: python 153 | 154 | class CustomisedJSONFormatter(json_log_formatter.JSONFormatter): 155 | def json_record(self, message: str, extra: dict, record: logging.LogRecord) -> dict: 156 | extra['message'] = message 157 | extra['user_id'] = current_user_id() 158 | extra['ip'] = current_ip() 159 | 160 | # Include builtins 161 | extra['level'] = record.levelname 162 | extra['name'] = record.name 163 | 164 | if 'time' not in extra: 165 | extra['time'] = django.utils.timezone.now() 166 | 167 | if record.exc_info: 168 | extra['exc_info'] = self.formatException(record.exc_info) 169 | 170 | return extra 171 | 172 | Let's say you want ``datetime`` to be serialized as timestamp. 173 | You can use **ujson** (which does it by default) and disable 174 | ISO8601 date mutation. 175 | 176 | .. code-block:: python 177 | 178 | class CustomisedJSONFormatter(json_log_formatter.JSONFormatter): 179 | json_lib = ujson 180 | 181 | def mutate_json_record(self, json_record): 182 | return json_record 183 | 184 | Tests 185 | ----- 186 | 187 | .. code-block:: console 188 | 189 | $ pip install -r requirements.txt 190 | $ tox 191 | -------------------------------------------------------------------------------- /json_log_formatter/__init__.py: -------------------------------------------------------------------------------- 1 | import logging 2 | from decimal import Decimal 3 | from datetime import datetime, timezone 4 | 5 | import json 6 | 7 | BUILTIN_ATTRS = { 8 | 'args', 9 | 'asctime', 10 | 'created', 11 | 'exc_info', 12 | 'exc_text', 13 | 'filename', 14 | 'funcName', 15 | 'levelname', 16 | 'levelno', 17 | 'lineno', 18 | 'module', 19 | 'msecs', 20 | 'message', 21 | 'msg', 22 | 'name', 23 | 'pathname', 24 | 'process', 25 | 'processName', 26 | 'relativeCreated', 27 | 'stack_info', 28 | 'taskName', 29 | 'thread', 30 | 'threadName', 31 | } 32 | 33 | 34 | class JSONFormatter(logging.Formatter): 35 | """JSON log formatter. 36 | 37 | Usage example:: 38 | 39 | import logging 40 | 41 | import json_log_formatter 42 | 43 | json_handler = logging.FileHandler(filename='/var/log/my-log.json') 44 | json_handler.setFormatter(json_log_formatter.JSONFormatter()) 45 | 46 | logger = logging.getLogger('my_json') 47 | logger.addHandler(json_handler) 48 | 49 | logger.info('Sign up', extra={'referral_code': '52d6ce'}) 50 | 51 | The log file will contain the following log record (inline):: 52 | 53 | { 54 | "message": "Sign up", 55 | "time": "2015-09-01T06:06:26.524448", 56 | "referral_code": "52d6ce" 57 | } 58 | 59 | """ 60 | 61 | json_lib = json 62 | 63 | def format(self, record): 64 | message = record.getMessage() 65 | extra = self.extra_from_record(record) 66 | json_record = self.json_record(message, extra, record) 67 | mutated_record = self.mutate_json_record(json_record) 68 | # Backwards compatibility: Functions that overwrite this but don't 69 | # return a new value will return None because they modified the 70 | # argument passed in. 71 | if mutated_record is None: 72 | mutated_record = json_record 73 | return self.to_json(mutated_record) 74 | 75 | def to_json(self, record): 76 | """Converts record dict to a JSON string. 77 | 78 | It makes best effort to serialize a record (represents an object as a string) 79 | instead of raising TypeError if json library supports default argument. 80 | Note, ujson doesn't support it. 81 | ValueError and OverflowError are also caught to avoid crashing an app, 82 | e.g., due to circular reference. 83 | 84 | Override this method to change the way dict is converted to JSON. 85 | 86 | """ 87 | try: 88 | return self.json_lib.dumps(record, default=_json_serializable) 89 | # ujson doesn't support default argument and raises TypeError. 90 | # "ValueError: Circular reference detected" is raised 91 | # when there is a reference to object inside the object itself. 92 | except (TypeError, ValueError, OverflowError): 93 | try: 94 | return self.json_lib.dumps(record) 95 | except (TypeError, ValueError, OverflowError): 96 | return '{}' 97 | 98 | def extra_from_record(self, record): 99 | """Returns `extra` dict you passed to logger. 100 | 101 | The `extra` keyword argument is used to populate the `__dict__` of 102 | the `LogRecord`. 103 | 104 | """ 105 | return { 106 | attr_name: record.__dict__[attr_name] 107 | for attr_name in record.__dict__ 108 | if attr_name not in BUILTIN_ATTRS 109 | } 110 | 111 | def json_record(self, message, extra, record): 112 | """Prepares a JSON payload which will be logged. 113 | 114 | Override this method to change JSON log format. 115 | 116 | :param message: Log message, e.g., `logger.info(msg='Sign up')`. 117 | :param extra: Dictionary that was passed as `extra` param 118 | `logger.info('Sign up', extra={'referral_code': '52d6ce'})`. 119 | :param record: `LogRecord` we got from `JSONFormatter.format()`. 120 | :return: Dictionary which will be passed to JSON lib. 121 | 122 | """ 123 | extra['message'] = message 124 | if 'time' not in extra: 125 | extra['time'] = datetime.now(timezone.utc) 126 | 127 | if record.exc_info: 128 | extra['exc_info'] = self.formatException(record.exc_info) 129 | 130 | return extra 131 | 132 | def mutate_json_record(self, json_record): 133 | """Override it to convert fields of `json_record` to needed types. 134 | 135 | Default implementation converts `datetime` to string in ISO8601 format. 136 | 137 | """ 138 | for attr_name in json_record: 139 | attr = json_record[attr_name] 140 | if isinstance(attr, datetime): 141 | json_record[attr_name] = attr.isoformat() 142 | return json_record 143 | 144 | 145 | def _json_serializable(obj): 146 | try: 147 | return obj.__dict__ 148 | except AttributeError: 149 | return str(obj) 150 | 151 | 152 | class VerboseJSONFormatter(JSONFormatter): 153 | """JSON log formatter with built-in log record attributes such as log level. 154 | 155 | Usage example:: 156 | 157 | import logging 158 | 159 | import json_log_formatter 160 | 161 | json_handler = logging.FileHandler(filename='/var/log/my-log.json') 162 | json_handler.setFormatter(json_log_formatter.VerboseJSONFormatter()) 163 | 164 | logger = logging.getLogger('my_verbose_json') 165 | logger.addHandler(json_handler) 166 | 167 | logger.error('An error has occured') 168 | 169 | The log file will contain the following log record (inline):: 170 | 171 | { 172 | "filename": "tests.py", 173 | "funcName": "test_file_name_is_testspy", 174 | "levelname": "ERROR", 175 | "lineno": 276, 176 | "module": "tests", 177 | "name": "my_verbose_json", 178 | "pathname": "/Users/bob/json-log-formatter/tests.py", 179 | "process": 3081, 180 | "processName": "MainProcess", 181 | "stack_info": null, 182 | "thread": 4664270272, 183 | "threadName": "MainThread", 184 | "message": "An error has occured", 185 | "time": "2021-07-04T21:05:42.767726" 186 | } 187 | 188 | Read more about the built-in log record attributes 189 | https://docs.python.org/3/library/logging.html#logrecord-attributes. 190 | 191 | """ 192 | def json_record(self, message, extra, record): 193 | extra['filename'] = record.filename 194 | extra['funcName'] = record.funcName 195 | extra['levelname'] = record.levelname 196 | extra['lineno'] = record.lineno 197 | extra['module'] = record.module 198 | extra['name'] = record.name 199 | extra['pathname'] = record.pathname 200 | extra['process'] = record.process 201 | extra['processName'] = record.processName 202 | if hasattr(record, 'stack_info'): 203 | extra['stack_info'] = record.stack_info 204 | else: 205 | extra['stack_info'] = None 206 | extra['thread'] = record.thread 207 | extra['threadName'] = record.threadName 208 | return super(VerboseJSONFormatter, self).json_record(message, extra, record) 209 | 210 | 211 | class FlatJSONFormatter(JSONFormatter): 212 | """Flat JSON log formatter ensures that complex objects are stored as strings. 213 | 214 | Usage example:: 215 | 216 | logger.info('Sign up', extra={'request': WSGIRequest({ 217 | 'PATH_INFO': 'bogus', 218 | 'REQUEST_METHOD': 'bogus', 219 | 'CONTENT_TYPE': 'text/html; charset=utf8', 220 | 'wsgi.input': BytesIO(b''), 221 | })}) 222 | 223 | The log file will contain the following log record (inline):: 224 | 225 | { 226 | "message": "Sign up", 227 | "time": "2024-10-01T00:59:29.332888+00:00", 228 | "request": "" 229 | } 230 | 231 | """ 232 | 233 | keep = (bool, int, float, Decimal, complex, str, datetime) 234 | 235 | def json_record(self, message, extra, record): 236 | extra = super(FlatJSONFormatter, self).json_record(message, extra, record) 237 | return { 238 | k: v if v is None or isinstance(v, self.keep) else str(v) 239 | for k, v in extra.items() 240 | } 241 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["setuptools"] 3 | build-backend = "setuptools.build_meta" 4 | 5 | [tool.setuptools] 6 | packages = ["json_log_formatter"] 7 | 8 | [project] 9 | name = "JSON-log-formatter" 10 | version = "1.2" 11 | description = "JSON log formatter" 12 | readme = "README.rst" 13 | requires-python = ">=3.9" 14 | license = {text = "MIT"} 15 | authors = [ 16 | {name = "Marsel Mavletkulov"}, 17 | ] 18 | classifiers=[ 19 | "License :: OSI Approved :: MIT License", 20 | "Intended Audience :: Developers", 21 | "Operating System :: OS Independent", 22 | "Programming Language :: Python", 23 | "Programming Language :: Python :: 3", 24 | "Topic :: Software Development :: Libraries :: Python Modules" 25 | ] 26 | 27 | [project.urls] 28 | repository = "https://github.com/marselester/json-log-formatter" 29 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | -e . 2 | 3 | tox==4.11.4 4 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | setup( 4 | name='JSON-log-formatter', 5 | version='1.2', 6 | license='MIT', 7 | packages=['json_log_formatter'], 8 | author='Marsel Mavletkulov', 9 | url='https://github.com/marselester/json-log-formatter', 10 | description='JSON log formatter', 11 | long_description=open('README.rst').read(), 12 | classifiers=[ 13 | 'License :: OSI Approved :: MIT License', 14 | 'Intended Audience :: Developers', 15 | 'Operating System :: OS Independent', 16 | 'Programming Language :: Python', 17 | 'Programming Language :: Python :: 3', 18 | 'Topic :: Software Development :: Libraries :: Python Modules' 19 | ], 20 | ) 21 | -------------------------------------------------------------------------------- /tests.py: -------------------------------------------------------------------------------- 1 | from datetime import datetime 2 | from decimal import Decimal 3 | from io import BytesIO 4 | import unittest 5 | import logging 6 | import json 7 | import os.path 8 | 9 | 10 | from django.core.handlers.wsgi import WSGIRequest 11 | from django.conf import settings 12 | import ujson 13 | import simplejson 14 | 15 | try: 16 | from cStringIO import StringIO 17 | except ImportError: 18 | from io import StringIO 19 | 20 | from json_log_formatter import JSONFormatter, VerboseJSONFormatter, FlatJSONFormatter 21 | 22 | log_buffer = StringIO() 23 | json_handler = logging.StreamHandler(log_buffer) 24 | 25 | logger = logging.getLogger('test') 26 | logger.addHandler(json_handler) 27 | logger.setLevel(logging.DEBUG) 28 | logging.propagate = False 29 | 30 | DATETIME = datetime(2015, 9, 1, 6, 9, 42, 797203) 31 | DATETIME_ISO = u'2015-09-01T06:09:42.797203' 32 | 33 | settings.configure(DEBUG=True) 34 | 35 | 36 | class TestCase(unittest.TestCase): 37 | def tearDown(self): 38 | log_buffer.seek(0) 39 | log_buffer.truncate() 40 | 41 | 42 | class JSONFormatterTest(TestCase): 43 | def setUp(self): 44 | json_handler.setFormatter(JSONFormatter()) 45 | 46 | def test_given_time_is_used_in_log_record(self): 47 | logger.info('Sign up', extra={'time': DATETIME}) 48 | expected_time = '"time": "2015-09-01T06:09:42.797203"' 49 | self.assertIn(expected_time, log_buffer.getvalue()) 50 | 51 | def test_current_time_is_used_by_default_in_log_record(self): 52 | logger.info('Sign up', extra={'fizz': 'bazz'}) 53 | self.assertNotIn(DATETIME_ISO, log_buffer.getvalue()) 54 | 55 | def test_message_and_time_are_in_json_record_when_extra_is_blank(self): 56 | logger.info('Sign up') 57 | json_record = json.loads(log_buffer.getvalue()) 58 | expected_fields = set([ 59 | 'message', 60 | 'time', 61 | ]) 62 | self.assertTrue(expected_fields.issubset(json_record)) 63 | 64 | def test_message_and_time_and_extra_are_in_json_record_when_extra_is_provided(self): 65 | logger.info('Sign up', extra={'fizz': 'bazz'}) 66 | json_record = json.loads(log_buffer.getvalue()) 67 | expected_fields = set([ 68 | 'message', 69 | 'time', 70 | 'fizz', 71 | ]) 72 | self.assertTrue(expected_fields.issubset(json_record)) 73 | 74 | def test_exc_info_is_logged(self): 75 | try: 76 | raise ValueError('something wrong') 77 | except ValueError: 78 | logger.error('Request failed', exc_info=True) 79 | json_record = json.loads(log_buffer.getvalue()) 80 | self.assertIn( 81 | 'Traceback (most recent call last)', 82 | json_record['exc_info'] 83 | ) 84 | 85 | 86 | class MutatingFormatter(JSONFormatter): 87 | def mutate_json_record(self, json_record): 88 | new_record = {} 89 | for k, v in json_record.items(): 90 | if isinstance(v, datetime): 91 | v = v.isoformat() 92 | new_record[k] = v 93 | return new_record 94 | 95 | 96 | class MutatingFormatterTest(TestCase): 97 | def setUp(self): 98 | json_handler.setFormatter(MutatingFormatter()) 99 | 100 | def test_new_record_accepted(self): 101 | logger.info('Sign up', extra={'fizz': DATETIME}) 102 | json_record = json.loads(log_buffer.getvalue()) 103 | self.assertEqual(json_record['fizz'], DATETIME_ISO) 104 | 105 | 106 | class JsonLibTest(TestCase): 107 | def setUp(self): 108 | json_handler.setFormatter(JSONFormatter()) 109 | 110 | def test_builtin_types_are_serialized(self): 111 | logger.log(level=logging.ERROR, msg='Payment was sent', extra={ 112 | 'first_name': 'bob', 113 | 'amount': 0.00497265, 114 | 'context': { 115 | 'tags': ['fizz', 'bazz'], 116 | }, 117 | 'things': ('a', 'b'), 118 | 'ok': True, 119 | 'none': None, 120 | }) 121 | 122 | json_record = json.loads(log_buffer.getvalue()) 123 | self.assertEqual(json_record['first_name'], 'bob') 124 | self.assertEqual(json_record['amount'], 0.00497265) 125 | self.assertEqual(json_record['context'], {'tags': ['fizz', 'bazz']}) 126 | self.assertEqual(json_record['things'], ['a', 'b']) 127 | self.assertEqual(json_record['ok'], True) 128 | self.assertEqual(json_record['none'], None) 129 | 130 | def test_decimal_is_serialized_as_string(self): 131 | logger.log(level=logging.ERROR, msg='Payment was sent', extra={ 132 | 'amount': Decimal('0.00497265') 133 | }) 134 | expected_amount = '"amount": "0.00497265"' 135 | self.assertIn(expected_amount, log_buffer.getvalue()) 136 | 137 | def test_django_wsgi_request_is_serialized_as_dict(self): 138 | request = WSGIRequest({ 139 | 'PATH_INFO': 'bogus', 140 | 'REQUEST_METHOD': 'bogus', 141 | 'CONTENT_TYPE': 'text/html; charset=utf8', 142 | 'wsgi.input': BytesIO(b''), 143 | }) 144 | 145 | logger.log(level=logging.ERROR, msg='Django response error', extra={ 146 | 'status_code': 500, 147 | 'request': request 148 | }) 149 | json_record = json.loads(log_buffer.getvalue()) 150 | self.assertEqual(json_record['status_code'], 500) 151 | self.assertEqual(json_record['request']['path'], '/bogus') 152 | self.assertEqual(json_record['request']['method'], 'BOGUS') 153 | 154 | def test_json_circular_reference_is_handled(self): 155 | d = {} 156 | d['circle'] = d 157 | logger.info('Referer checking', extra=d) 158 | self.assertEqual('{}\n', log_buffer.getvalue()) 159 | 160 | 161 | class UjsonLibTest(TestCase): 162 | def setUp(self): 163 | formatter = JSONFormatter() 164 | formatter.json_lib = ujson 165 | json_handler.setFormatter(formatter) 166 | 167 | def test_builtin_types_are_serialized(self): 168 | logger.log(level=logging.ERROR, msg='Payment was sent', extra={ 169 | 'first_name': 'bob', 170 | 'amount': 0.00497265, 171 | 'context': { 172 | 'tags': ['fizz', 'bazz'], 173 | }, 174 | 'things': ('a', 'b'), 175 | 'ok': True, 176 | 'none': None, 177 | }) 178 | 179 | json_record = json.loads(log_buffer.getvalue()) 180 | self.assertEqual(json_record['first_name'], 'bob') 181 | self.assertEqual(json_record['amount'], 0.00497265) 182 | self.assertEqual(json_record['context'], {'tags': ['fizz', 'bazz']}) 183 | self.assertEqual(json_record['things'], ['a', 'b']) 184 | self.assertEqual(json_record['ok'], True) 185 | self.assertEqual(json_record['none'], None) 186 | 187 | def test_decimal_is_serialized_as_number(self): 188 | logger.info('Payment was sent', extra={ 189 | 'amount': Decimal('0.00497265') 190 | }) 191 | expected_amount = '"amount":0.00497265' 192 | self.assertIn(expected_amount, log_buffer.getvalue()) 193 | 194 | def test_zero_expected_when_decimal_is_in_scientific_notation(self): 195 | logger.info('Payment was sent', extra={ 196 | 'amount': Decimal('0E-8') 197 | }) 198 | expected_amount = '"amount":0.0' 199 | self.assertIn(expected_amount, log_buffer.getvalue()) 200 | 201 | def test_django_wsgi_request_is_serialized_as_empty_list(self): 202 | request = WSGIRequest({ 203 | 'PATH_INFO': 'bogus', 204 | 'REQUEST_METHOD': 'bogus', 205 | 'CONTENT_TYPE': 'text/html; charset=utf8', 206 | 'wsgi.input': BytesIO(b''), 207 | }) 208 | 209 | logger.log(level=logging.ERROR, msg='Django response error', extra={ 210 | 'status_code': 500, 211 | 'request': request 212 | }) 213 | json_record = json.loads(log_buffer.getvalue()) 214 | if 'status_code' in json_record: 215 | self.assertEqual(json_record['status_code'], 500) 216 | if 'request' in json_record: 217 | self.assertEqual(json_record['request']['path'], '/bogus') 218 | self.assertEqual(json_record['request']['method'], 'BOGUS') 219 | 220 | def test_json_circular_reference_is_handled(self): 221 | d = {} 222 | d['circle'] = d 223 | logger.info('Referer checking', extra=d) 224 | self.assertEqual('{}\n', log_buffer.getvalue()) 225 | 226 | 227 | class SimplejsonLibTest(TestCase): 228 | def setUp(self): 229 | formatter = JSONFormatter() 230 | formatter.json_lib = simplejson 231 | json_handler.setFormatter(formatter) 232 | 233 | def test_builtin_types_are_serialized(self): 234 | logger.log(level=logging.ERROR, msg='Payment was sent', extra={ 235 | 'first_name': 'bob', 236 | 'amount': 0.00497265, 237 | 'context': { 238 | 'tags': ['fizz', 'bazz'], 239 | }, 240 | 'things': ('a', 'b'), 241 | 'ok': True, 242 | 'none': None, 243 | }) 244 | 245 | json_record = json.loads(log_buffer.getvalue()) 246 | self.assertEqual(json_record['first_name'], 'bob') 247 | self.assertEqual(json_record['amount'], 0.00497265) 248 | self.assertEqual(json_record['context'], {'tags': ['fizz', 'bazz']}) 249 | self.assertEqual(json_record['things'], ['a', 'b']) 250 | self.assertEqual(json_record['ok'], True) 251 | self.assertEqual(json_record['none'], None) 252 | 253 | def test_decimal_is_serialized_as_number(self): 254 | logger.info('Payment was sent', extra={ 255 | 'amount': Decimal('0.00497265') 256 | }) 257 | expected_amount = '"amount": 0.00497265' 258 | self.assertIn(expected_amount, log_buffer.getvalue()) 259 | 260 | def test_decimal_is_serialized_as_it_is_when_it_is_in_scientific_notation(self): 261 | logger.info('Payment was sent', extra={ 262 | 'amount': Decimal('0E-8') 263 | }) 264 | expected_amount = '"amount": 0E-8' 265 | self.assertIn(expected_amount, log_buffer.getvalue()) 266 | 267 | def test_django_wsgi_request_is_serialized_as_dict(self): 268 | request = WSGIRequest({ 269 | 'PATH_INFO': 'bogus', 270 | 'REQUEST_METHOD': 'bogus', 271 | 'CONTENT_TYPE': 'text/html; charset=utf8', 272 | 'wsgi.input': BytesIO(b''), 273 | }) 274 | 275 | logger.log(level=logging.ERROR, msg='Django response error', extra={ 276 | 'status_code': 500, 277 | 'request': request 278 | }) 279 | json_record = json.loads(log_buffer.getvalue()) 280 | self.assertEqual(json_record['status_code'], 500) 281 | self.assertEqual(json_record['request']['path'], '/bogus') 282 | self.assertEqual(json_record['request']['method'], 'BOGUS') 283 | 284 | def test_json_circular_reference_is_handled(self): 285 | d = {} 286 | d['circle'] = d 287 | logger.info('Referer checking', extra=d) 288 | self.assertEqual('{}\n', log_buffer.getvalue()) 289 | 290 | 291 | class VerboseJSONFormatterTest(TestCase): 292 | def setUp(self): 293 | json_handler.setFormatter(VerboseJSONFormatter()) 294 | 295 | def test_file_name_is_testspy(self): 296 | logger.error('An error has occured') 297 | json_record = json.loads(log_buffer.getvalue()) 298 | self.assertEqual(json_record['filename'], 'tests.py') 299 | 300 | def test_function_name(self): 301 | logger.error('An error has occured') 302 | json_record = json.loads(log_buffer.getvalue()) 303 | self.assertEqual(json_record['funcName'], 'test_function_name') 304 | 305 | def test_level_name_is_error(self): 306 | logger.error('An error has occured') 307 | json_record = json.loads(log_buffer.getvalue()) 308 | self.assertEqual(json_record['levelname'], 'ERROR') 309 | 310 | def test_module_name_is_tests(self): 311 | logger.error('An error has occured') 312 | json_record = json.loads(log_buffer.getvalue()) 313 | self.assertEqual(json_record['module'], 'tests') 314 | 315 | def test_logger_name_is_test(self): 316 | logger.error('An error has occured') 317 | json_record = json.loads(log_buffer.getvalue()) 318 | self.assertEqual(json_record['name'], 'test') 319 | 320 | def test_path_name_is_test(self): 321 | logger.error('An error has occured') 322 | json_record = json.loads(log_buffer.getvalue()) 323 | self.assertIn(os.path.basename(os.path.abspath('.')) + '/tests.py', json_record['pathname']) 324 | 325 | def test_process_name_is_MainProcess(self): 326 | logger.error('An error has occured') 327 | json_record = json.loads(log_buffer.getvalue()) 328 | self.assertEqual(json_record['processName'], 'MainProcess') 329 | 330 | def test_thread_name_is_MainThread(self): 331 | logger.error('An error has occured') 332 | json_record = json.loads(log_buffer.getvalue()) 333 | self.assertEqual(json_record['threadName'], 'MainThread') 334 | 335 | def test_stack_info_is_none(self): 336 | logger.error('An error has occured') 337 | json_record = json.loads(log_buffer.getvalue()) 338 | self.assertIsNone(json_record['stack_info']) 339 | 340 | 341 | class FlatJSONFormatterTest(TestCase): 342 | def setUp(self): 343 | json_handler.setFormatter(FlatJSONFormatter()) 344 | 345 | def test_given_time_is_used_in_log_record(self): 346 | logger.info('Sign up', extra={'time': DATETIME}) 347 | expected_time = '"time": "2015-09-01T06:09:42.797203"' 348 | self.assertIn(expected_time, log_buffer.getvalue()) 349 | 350 | def test_current_time_is_used_by_default_in_log_record(self): 351 | logger.info('Sign up', extra={'fizz': 'bazz'}) 352 | self.assertNotIn(DATETIME_ISO, log_buffer.getvalue()) 353 | 354 | def test_message_and_time_are_in_json_record_when_extra_is_blank(self): 355 | logger.info('Sign up') 356 | json_record = json.loads(log_buffer.getvalue()) 357 | expected_fields = set([ 358 | 'message', 359 | 'time', 360 | ]) 361 | self.assertTrue(expected_fields.issubset(json_record)) 362 | 363 | def test_message_and_time_and_extra_are_in_json_record_when_extra_is_provided(self): 364 | logger.info('Sign up', extra={'fizz': 'bazz'}) 365 | json_record = json.loads(log_buffer.getvalue()) 366 | expected_fields = set([ 367 | 'message', 368 | 'time', 369 | 'fizz', 370 | ]) 371 | self.assertTrue(expected_fields.issubset(json_record)) 372 | 373 | def test_exc_info_is_logged(self): 374 | try: 375 | raise ValueError('something wrong') 376 | except ValueError: 377 | logger.error('Request failed', exc_info=True) 378 | json_record = json.loads(log_buffer.getvalue()) 379 | self.assertIn( 380 | 'Traceback (most recent call last)', 381 | json_record['exc_info'] 382 | ) 383 | 384 | def test_builtin_types_are_serialized(self): 385 | logger.log(level=logging.ERROR, msg='Payment was sent', extra={ 386 | 'first_name': 'bob', 387 | 'amount': 0.00497265, 388 | 'context': { 389 | 'tags': ['fizz', 'bazz'], 390 | }, 391 | 'things': ('a', 'b'), 392 | 'ok': True, 393 | 'none': None, 394 | }) 395 | 396 | json_record = json.loads(log_buffer.getvalue()) 397 | self.assertEqual(json_record['first_name'], 'bob') 398 | self.assertEqual(json_record['amount'], 0.00497265) 399 | self.assertEqual(json_record['context'], "{'tags': ['fizz', 'bazz']}") 400 | self.assertEqual(json_record['things'], "('a', 'b')") 401 | self.assertEqual(json_record['ok'], True) 402 | self.assertEqual(json_record['none'], None) 403 | 404 | def test_decimal_is_serialized_as_string(self): 405 | logger.log(level=logging.ERROR, msg='Payment was sent', extra={ 406 | 'amount': Decimal('0.00497265') 407 | }) 408 | expected_amount = '"amount": "0.00497265"' 409 | self.assertIn(expected_amount, log_buffer.getvalue()) 410 | 411 | def test_django_wsgi_request_is_serialized_as_dict(self): 412 | request = WSGIRequest({ 413 | 'PATH_INFO': 'bogus', 414 | 'REQUEST_METHOD': 'bogus', 415 | 'CONTENT_TYPE': 'text/html; charset=utf8', 416 | 'wsgi.input': BytesIO(b''), 417 | }) 418 | 419 | logger.log(level=logging.ERROR, msg='Django response error', extra={ 420 | 'status_code': 500, 421 | 'request': request, 422 | 'dict': { 423 | 'request': request, 424 | }, 425 | 'list': [request], 426 | }) 427 | json_record = json.loads(log_buffer.getvalue()) 428 | self.assertEqual(json_record['status_code'], 500) 429 | self.assertEqual(json_record['request'], "") 430 | self.assertEqual(json_record['dict'], "{'request': }") 431 | self.assertEqual(json_record['list'], "[]") 432 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist=py39,py310,py311,py312,py313 3 | 4 | [testenv] 5 | deps= 6 | pytest 7 | ujson 8 | simplejson 9 | django 10 | commands= 11 | pytest -s tests.py 12 | --------------------------------------------------------------------------------