└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Schema Comparison between Structured Logging Libraries 2 | 3 | The goal of this document is to summarize the differences between various 4 | structured logging libraries, in order to encourage a discussion so that a 5 | consensus can be reached for a common format. 6 | 7 | Please use the GitHub issues for discussion, and submit GitHub Pull requests 8 | to add/fix the information in the tables. 9 | 10 | 11 | ## Message and Payload 12 | 13 | Behold the following logging command that you might find in an application. 14 | There is a "message" string, and a "payload" object: 15 | 16 | logger.info("Hello World", {"animal": "cat", "numLegs": 4}) 17 | ^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 18 | Message Payload object 19 | 20 | Some libraries merge the payload object with the root object, so the event 21 | will look like this: 22 | 23 | ```json 24 | { 25 | "timestamp": "2018-06-18T23:16:45.000Z", 26 | "message": "Hello World", 27 | "animal": "cat", 28 | "numLegs": 4 29 | } 30 | ``` 31 | 32 | While other libraries nest the payload object beneath some key, with a result like this: 33 | 34 | ```json 35 | { 36 | "timestamp": "2018-06-18T23:16:45.000Z", 37 | "message": "Hello World", 38 | "data": { 39 | "animal": "cat", 40 | "numLegs": 4 41 | } 42 | } 43 | ``` 44 | 45 | | Library | Message JSON Key | Payload Object JSON Key | 46 | |---------|--------------------|------------------| 47 | | [bunyan](https://github.com/trentm/node-bunyan) (JavaScript) | `msg` | *Merged with root* | 48 | | [katip](https://github.com/Soostone/katip) (Haskell) | `msg` | `data` | 49 | | [logrus](https://github.com/sirupsen/logrus) (Go) | `msg` | *Merged with root* | 50 | | [logstash](https://github.com/elastic/logstash) | `message` | *Merged with root* | 51 | | [ougai](https://github.com/tilfin/ougai) (Ruby) | `msg` | *Merged with root* | 52 | | [pygogo](https://github.com/reubano/pygogo) (Python) | `message` | *Merged with root* | 53 | | [roarr](https://github.com/gajus/roarr) (JavaScript) | `message` | `context` | 54 | | [semantic logger](https://github.com/rocketjob/semantic_logger) (Ruby) | `message` | payload | 55 | | [serilog](https://github.com/serilog/serilog) (C#) | `@m` / `@mt` | *Merged with root* | 56 | | [slog](https://github.com/slog-rs/slog) (Rust) | `msg` | *Merged with root* | 57 | | [structlog](https://github.com/hynek/structlog) (Python) | `msg` | **HELP NEEDED** | 58 | 59 | 60 | ## Timestamp JSON Key and format 61 | 62 | All libraries automatically add a timestamp to all events. 63 | 64 | | Library | Timestamp JSON Key | Timestamp format | 65 | |---------|--------------------|------------------| 66 | | [bunyan](https://github.com/trentm/node-bunyan) (JavaScript) | `time` | ISO 8601 | 67 | | [katip](https://github.com/Soostone/katip) (Haskell) | `at` | ISO 8601 | 68 | | [logrus](https://github.com/sirupsen/logrus) (Go) | `time` | ISO 8601 | 69 | | [logstash](https://github.com/elastic/logstash) | `timestamp` / `@timestamp` | ISO 8601 / varies | 70 | | [ougai](https://github.com/tilfin/ougai) (Ruby) | `time` | ISO 8601 | 71 | | [pygogo](https://github.com/reubano/pygogo) (Python) | `time` | ISO 8601 | 72 | | [roarr](https://github.com/gajus/roarr) (JavaScript) | `time` | Epoch milliseconds | 73 | | [semantic logger](https://github.com/rocketjob/semantic_logger) (Ruby) | `timestamp` | ISO 8601 | 74 | | [serilog](https://github.com/serilog/serilog) (C#) | `Timestamp` / `@t` | ISO 8601 | 75 | | [slog](https://github.com/slog-rs/slog) (Rust) | `ts` | ISO 8601 | 76 | | [structlog](https://github.com/hynek/structlog) (Python) | `timestamp` | ISO 8601 | 77 | 78 | 79 | ## Log Level 80 | 81 | All libraries require that that all events are tagged with a level, chosen 82 | from a pre-defined list of available levels. 83 | 84 | ### Log Level JSON Key 85 | 86 | | Library | Log Level JSON Key | 87 | |---------|--------------------| 88 | | [bunyan](https://github.com/trentm/node-bunyan) (JavaScript) | `level` | 89 | | [katip](https://github.com/Soostone/katip) (Haskell) | `sev` | 90 | | [logrus](https://github.com/sirupsen/logrus) (Go) | `level` | 91 | | [logstash](https://github.com/elastic/logstash) | **HELP NEEDED** | 92 | | [ougai](https://github.com/tilfin/ougai) (Ruby) | `level` | 93 | | [pygogo](https://github.com/reubano/pygogo) (Python) | `level` | 94 | | [roarr](https://github.com/gajus/roarr) (JavaScript) | `context.logLevel` | 95 | | [semantic logger](https://github.com/rocketjob/semantic_logger) (Ruby) | `level` | 96 | | [serilog](https://github.com/serilog/serilog) (C#) | `Level` / `@l` | 97 | | [slog](https://github.com/slog-rs/slog) (Rust) | `level` | 98 | | [structlog](https://github.com/hynek/structlog) (Python) | `level` | 99 | 100 | ### Available Log Levels 101 | 102 | | Library | VERBOSE | TRACE | DEBUG | INFO | NOTICE | WARNING | ERROR | CRITICAL | ALERT | FATAL | PANIC | EMERGENCY | 103 | |---------|-----------|---------|----|----|----|----|----|----|----|----|----|----| 104 | | [bunyan](https://github.com/trentm/node-bunyan) (JavaScript) | *N/A* | `10` | `20` | `30` | *N/A* | `40` | `50` | *N/A* | *N/A* | `60` | *N/A* | *N/A* | 105 | | [katip](https://github.com/Soostone/katip) (Haskell) | *N/A* | *N/A* | `Debug` | `Info` | `Notice` | `Warning` | `Error` | `Critical` | `Alert` | *N/A* | *N/A* | `Emergency` | 106 | | [logrus](https://github.com/sirupsen/logrus) (Go) | *N/A* | *N/A* | `debug` | `info` | *N/A* | `warning` | `error` | *N/A* | *N/A* | `fatal` | `panic` | *N/A* | 107 | | [logstash](https://github.com/elastic/logstash) | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | 108 | | [ougai](https://github.com/tilfin/ougai) (Ruby) | *N/A* | `10` | `20` | `30` | *N/A* | `40` | `50` | *N/A* | *N/A* | `60` | *N/A* | *N/A* | 109 | | [pygogo](https://github.com/reubano/pygogo) (Python) | *N/A* | *N/A* | `DEBUG` | `INFO` | *N/A* | `WARNING` | `ERROR` | `CRITICAL` | *N/A* | *N/A* | *N/A* | *N/A* | 110 | | [roarr](https://github.com/gajus/roarr) (JavaScript) | *N/A* | `10` | `20` | `30` | *N/A* | `40` | `50` | *N/A* | *N/A* | `60` | *N/A* | *N/A* | 111 | | [semantic logger](https://github.com/rocketjob/semantic_logger) (Ruby) | *N/A* | `trace` | `debug` | `info` | *N/A* | `warn` | `error` | *N/A* | *N/A* | `fatal` | *N/A* | *N/A* | 112 | | [serilog](https://github.com/serilog/serilog) (C#) | `Verbose` | *N/A* | `Debug` | `Information` | *N/A* | `Warning` | `Error` | *N/A* | *N/A* | `Fatal` | *N/A* | *N/A* | 113 | | [slog](https://github.com/slog-rs/slog) (Rust) | *N/A* | `TRACE` | `DEBUG` | `INFO` | *N/A* | `WARN` | `ERROR` | `CRITICAL` | *N/A* | *N/A* | *N/A* | *N/A* | 114 | | [structlog](https://github.com/hynek/structlog) (Python) | *N/A* | *N/A* | `debug` | `info` | *N/A* | `warning` | `error` | `critical` | *N/A* | *N/A* | *N/A* | *N/A* | 115 | 116 | 117 | ## System Context Data 118 | 119 | Some libraries automatically add additional fields to all events (such as 120 | process ID and thread ID) and/or allow you to configure a few pre-determined 121 | global fields suchs the current environment and application. 122 | 123 | | Library | Environment | Application | Namespace | Package | Machine Hostname | Process ID | Thread ID | 124 | |---------|-----------------|----|----|----|----|----|----| 125 | | [bunyan](https://github.com/trentm/node-bunyan) (JavaScript) | *N/A* | *N/A* | *N/A* | *N/A* | `hostname` | `pid` | *N/A* | 126 | | [katip](https://github.com/Soostone/katip) (Haskell) | `env` | `app` (Array of strings) | `ns` (Array of strings) | *N/A* | `host` | `pid` |`thread` | 127 | | [logrus](https://github.com/sirupsen/logrus) (Go) | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | 128 | | [logstash](https://github.com/elastic/logstash) | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | **HELP NEEDED** | 129 | | [ougai](https://github.com/tilfin/ougai) (Ruby) | *N/A* | `app` | *N/A* | *N/A* | `hostname` | `pid` | *N/A* | 130 | | [pygogo](https://github.com/reubano/pygogo) (Python) | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | 131 | | [roarr](https://github.com/gajus/roarr) (JavaScript) | *N/A* | `context.application` | `context.namespace` | `context.package` | `context.hostname` | *N/A* | *N/A* | 132 | | [semantic logger](https://github.com/rocketjob/semantic_logger) (Ruby) | `environment` | `application` | *N/A* | *N/A* | `host` | `pid` | `thread` | 133 | | [serilog](https://github.com/serilog/serilog) (C#) | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | 134 | | [slog](https://github.com/slog-rs/slog) (Rust) | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | 135 | | [structlog](https://github.com/hynek/structlog) (Python) | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | *N/A* | 136 | 137 | 138 | ## HTTP Request Context Data 139 | 140 | Logging of HTTP requests is very common. Let's figure out a common format for 141 | HTTP method, host, url, headers, remote IP, remote IP geo-lookup, response 142 | status code, response size, etc... 143 | 144 | !!! TODO !!! 145 | --------------------------------------------------------------------------------