├── .tool-versions ├── Gemfile ├── test ├── test_helper.rb └── simple_structured_logger_test.rb ├── bin ├── setup └── console ├── .gitignore ├── Rakefile ├── .github ├── dependabot.yml └── workflows │ ├── ruby.yml │ └── codeql-analysis.yml ├── simple_structured_logger.gemspec ├── LICENSE ├── lib └── simple_structured_logger.rb └── README.md /.tool-versions: -------------------------------------------------------------------------------- 1 | ruby 3.1.1 -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | gemspec 3 | -------------------------------------------------------------------------------- /test/test_helper.rb: -------------------------------------------------------------------------------- 1 | $LOAD_PATH.unshift File.expand_path('../../lib', __FILE__) 2 | require 'simple_structured_logger' 3 | 4 | require 'minitest/autorun' 5 | -------------------------------------------------------------------------------- /bin/setup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -euo pipefail 3 | IFS=$'\n\t' 4 | set -vx 5 | 6 | bundle install 7 | 8 | # Do any other automated setup that you need to do here 9 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /.bundle/ 2 | /.yardoc 3 | /Gemfile.lock 4 | /_yardoc/ 5 | /coverage/ 6 | /doc/ 7 | /pkg/ 8 | /spec/reports/ 9 | /tmp/ 10 | /vendor/bundle 11 | /*.gem 12 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require "bundler/gem_tasks" 2 | require "rake/testtask" 3 | 4 | Rake::TestTask.new(:test) do |t| 5 | t.libs << "test" 6 | t.libs << "lib" 7 | t.test_files = FileList['test/**/*_test.rb'] 8 | end 9 | 10 | task :default => :test 11 | -------------------------------------------------------------------------------- /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | # https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates 2 | version: 2 3 | updates: 4 | - package-ecosystem: "bundler" 5 | directory: "/" # Location of package manifests 6 | schedule: 7 | interval: "monthly" 8 | 9 | - package-ecosystem: "github-actions" 10 | directory: "/" 11 | schedule: 12 | interval: "monthly" -------------------------------------------------------------------------------- /bin/console: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | require "bundler/setup" 4 | require "simple_structured_logger" 5 | 6 | # You can add fixtures and/or initialization code here to make experimenting 7 | # with your gem easier. You can also use a different console, if you like. 8 | 9 | # (If you use this, don't forget to add pry to your Gemfile!) 10 | # require "pry" 11 | # Pry.start 12 | 13 | require "irb" 14 | IRB.start 15 | -------------------------------------------------------------------------------- /.github/workflows/ruby.yml: -------------------------------------------------------------------------------- 1 | # This workflow uses actions that are not certified by GitHub. 2 | # They are provided by a third-party and are governed by 3 | # separate terms of service, privacy policy, and support 4 | # documentation. 5 | # This workflow will download a prebuilt Ruby version, install dependencies and run tests with Rake 6 | # For more information see: https://github.com/marketplace/actions/setup-ruby-jruby-and-truffleruby 7 | 8 | name: Ruby 9 | 10 | on: 11 | push: 12 | branches: [ master ] 13 | pull_request: 14 | branches: [ master ] 15 | 16 | jobs: 17 | test: 18 | 19 | runs-on: ubuntu-latest 20 | strategy: 21 | matrix: 22 | ruby-version: ['2.6', '2.7', '3.0', '3.1'] 23 | 24 | steps: 25 | - uses: actions/checkout@v4 26 | - name: Set up Ruby 27 | uses: ruby/setup-ruby@v1 28 | with: 29 | ruby-version: ${{ matrix.ruby-version }} 30 | bundler-cache: true 31 | - name: Run tests 32 | run: bundle exec rake 33 | 34 | -------------------------------------------------------------------------------- /simple_structured_logger.gemspec: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | lib = File.expand_path('../lib', __FILE__) 3 | $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) 4 | 5 | Gem::Specification.new do |spec| 6 | spec.name = "simple_structured_logger" 7 | spec.version = '1.0.2' 8 | spec.authors = ["Michael Bianco"] 9 | spec.email = ["mike@mikebian.co"] 10 | spec.licenses = ['MIT'] 11 | 12 | spec.summary = "Simple structured logging with a simple codebase" 13 | spec.homepage = "https://github.com/iloveitaly/simple_structured_logger" 14 | 15 | spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) } 16 | spec.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) } 17 | spec.require_paths = ["lib"] 18 | 19 | spec.metadata['rubygems_mfa_required'] = 'true' 20 | 21 | spec.add_development_dependency "bundler", "~> 2.3.10" 22 | spec.add_development_dependency "rake", "~> 13.1.0" 23 | spec.add_development_dependency "minitest", "~> 5.16" 24 | end 25 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Michael Bianco 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /.github/workflows/codeql-analysis.yml: -------------------------------------------------------------------------------- 1 | name: "CodeQL" 2 | 3 | on: 4 | push: 5 | branches: [ master ] 6 | pull_request: 7 | # The branches below must be a subset of the branches above 8 | branches: [ master ] 9 | schedule: 10 | - cron: '24 16 * * 1' 11 | 12 | jobs: 13 | analyze: 14 | name: Analyze 15 | runs-on: ubuntu-latest 16 | permissions: 17 | actions: read 18 | contents: read 19 | security-events: write 20 | 21 | strategy: 22 | fail-fast: false 23 | matrix: 24 | language: [ 'ruby' ] 25 | # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ] 26 | # Learn more about CodeQL language support at https://git.io/codeql-language-support 27 | 28 | steps: 29 | - name: Checkout repository 30 | uses: actions/checkout@v4 31 | 32 | # Initializes the CodeQL tools for scanning. 33 | - name: Initialize CodeQL 34 | uses: github/codeql-action/init@v2 35 | with: 36 | languages: ${{ matrix.language }} 37 | # If you wish to specify custom queries, you can do so here or in a config file. 38 | # By default, queries listed here will override any specified in a config file. 39 | # Prefix the list here with "+" to use these queries and those in the config file. 40 | # queries: ./path/to/local/query, your-org/your-repo/queries@main 41 | 42 | # Autobuild attempts to build any compiled languages (C/C++, C#, or Java). 43 | # If this step fails, then you should remove it and run the build manually (see below) 44 | - name: Autobuild 45 | uses: github/codeql-action/autobuild@v2 46 | 47 | # ℹ️ Command-line programs to run using the OS shell. 48 | # 📚 https://git.io/JvXDl 49 | 50 | # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines 51 | # and modify them (or add more) to build your code if your project 52 | # uses a compiled language 53 | 54 | #- run: | 55 | # make bootstrap 56 | # make release 57 | 58 | - name: Perform CodeQL Analysis 59 | uses: github/codeql-action/analyze@v2 60 | -------------------------------------------------------------------------------- /test/simple_structured_logger_test.rb: -------------------------------------------------------------------------------- 1 | require_relative 'test_helper' 2 | require 'ostruct' 3 | 4 | class SimpleStructuredLoggerTest < Minitest::Test 5 | include SimpleStructuredLogger 6 | 7 | def teardown 8 | SimpleStructuredLogger.configure do 9 | @expand_context = nil 10 | @expand_log = nil 11 | end 12 | 13 | ENV.delete('LOG_LEVEL') 14 | log.set_log_level_from_environment 15 | 16 | log.reset_context! 17 | end 18 | 19 | def capture_logs(&block) 20 | out = StringIO.new 21 | log.logger.reopen(out) 22 | 23 | yield 24 | 25 | log.logger.reopen(STDOUT) 26 | out.string 27 | end 28 | 29 | def test_logging_stdout 30 | log.info("hey", foo: "bar") 31 | log.error("hey", foo: "bar") 32 | log.debug("hey", foo: "bar") 33 | log.warn("hey", foo: "bar") 34 | end 35 | 36 | def test_expand_context 37 | SimpleStructuredLogger.configure do 38 | expand_context do |context| 39 | # you can pass in a object and use `expand_context` to extract the relevant keys 40 | if context[:user] 41 | context[:user_id] = context[:user].id 42 | context[:user_name] = context[:user].name 43 | context.delete(:user) 44 | end 45 | 46 | context 47 | end 48 | end 49 | 50 | user = OpenStruct.new(id: 1, name: "mike") 51 | log.set_context(user: user, other: 'argument') 52 | 53 | out = capture_logs do 54 | log.error "core" 55 | end 56 | 57 | assert_match("user_id=1", out) 58 | assert_match("user_name=mike", out) 59 | assert_match("other=argument", out) 60 | end 61 | 62 | def test_expand_log 63 | SimpleStructuredLogger.configure do 64 | expand_log do |tags, default_tags| 65 | if tags[:stripe_resource] && tags[:stripe_resource].respond_to?(:id) 66 | stripe_resource = tags.delete(:stripe_resource) 67 | tags[:stripe_resource_id] = stripe_resource.id 68 | tags[:stripe_resource_type] = stripe_resource.class.to_s 69 | end 70 | 71 | tags 72 | end 73 | end 74 | 75 | stripe_resource = OpenStruct.new(id: 'cus_123') 76 | 77 | out = capture_logs do 78 | log.error "core", stripe_resource: stripe_resource 79 | end 80 | 81 | assert_match("core: stripe_resource_id=cus_123 stripe_resource_type=OpenStruct\n", out) 82 | end 83 | 84 | def test_environment_level 85 | log.logger.level = Logger::ERROR 86 | 87 | out = capture_logs do 88 | log.debug 'should be empty' 89 | end 90 | 91 | assert_empty(out) 92 | 93 | ENV['LOG_LEVEL'] = 'DEBUG' 94 | 95 | log.set_log_level_from_environment 96 | 97 | out = capture_logs do 98 | log.debug 'should exist' 99 | end 100 | 101 | assert_match('should exist', 'should exist') 102 | 103 | # we don't care about case 104 | ENV['LOG_LEVEL'] = 'error' 105 | log.set_log_level_from_environment 106 | assert_equal(Logger::ERROR, log.logger.level) 107 | end 108 | end 109 | -------------------------------------------------------------------------------- /lib/simple_structured_logger.rb: -------------------------------------------------------------------------------- 1 | require 'logger' 2 | require 'singleton' 3 | 4 | module SimpleStructuredLogger 5 | def self.configure(&block) 6 | SimpleStructuredLogger::Configuration.instance_eval(&block) 7 | end 8 | 9 | def self.logger 10 | SimpleStructuredLogger::Writer.instance.logger 11 | end 12 | 13 | def log 14 | SimpleStructuredLogger::Writer.instance 15 | end 16 | 17 | def self.included(klass) 18 | # TODO there's got to be a cleaner way to add a class method from `include` 19 | klass.class_eval do 20 | def self.log 21 | SimpleStructuredLogger::Writer.instance 22 | end 23 | end 24 | end 25 | 26 | module Configuration 27 | extend self 28 | 29 | @expand_context = nil 30 | @expand_log = nil 31 | 32 | def expand_context(&block) 33 | if block.nil? 34 | @expand_context 35 | else 36 | @expand_context = block 37 | end 38 | end 39 | 40 | def expand_log(&block) 41 | if block.nil? 42 | @expand_log 43 | else 44 | @expand_log = block 45 | end 46 | end 47 | end 48 | 49 | class Writer 50 | include Singleton 51 | 52 | attr_accessor :default_tags, :logger 53 | 54 | def initialize 55 | @logger = ::Logger.new(STDOUT) 56 | @default_tags = {} 57 | 58 | set_log_level_from_environment 59 | end 60 | 61 | # returns true if log level is set from env 62 | def set_log_level_from_environment 63 | env_log_level = ENV['LOG_LEVEL'] 64 | 65 | if !env_log_level.nil? && Logger::Severity.const_defined?(env_log_level.upcase) 66 | @logger.level = Logger::Severity.const_get(env_log_level.upcase) 67 | true 68 | else 69 | false 70 | end 71 | end 72 | 73 | def reset_context! 74 | @default_tags = {} 75 | end 76 | 77 | def set_context(context) 78 | reset_context! 79 | 80 | if SimpleStructuredLogger::Configuration.expand_context 81 | context = SimpleStructuredLogger::Configuration.expand_context.call(context) 82 | end 83 | 84 | @default_tags.merge!(context) 85 | end 86 | 87 | def error(msg, opts={}) 88 | @logger.error("#{msg}: #{stringify_tags(opts)}") 89 | end 90 | 91 | def info(msg, opts={}) 92 | @logger.info("#{msg}: #{stringify_tags(opts)}") 93 | end 94 | 95 | def debug(msg, opts={}) 96 | @logger.debug("#{msg}: #{stringify_tags(opts)}") 97 | end 98 | 99 | def warn(msg, opts={}) 100 | @logger.warn("#{msg}: #{stringify_tags(opts)}") 101 | end 102 | 103 | private def stringify_tags(additional_tags) 104 | additional_tags = additional_tags.dup 105 | 106 | if SimpleStructuredLogger::Configuration.expand_log 107 | additional_tags = SimpleStructuredLogger::Configuration.expand_log.call(additional_tags, self.default_tags) 108 | end 109 | 110 | additional_tags.merge(@default_tags).map {|k, v| "#{k}=#{v}" }.join(' ') 111 | end 112 | 113 | end 114 | end 115 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Ruby](https://github.com/iloveitaly/simple_structured_logger/actions/workflows/ruby.yml/badge.svg)](https://github.com/iloveitaly/simple_structured_logger/actions/workflows/ruby.yml) 2 | [![gem](https://img.shields.io/gem/v/simple_structured_logger.svg)](https://rubygems.org/gems/simple_structured_logger) 3 | 4 | # SimpleStructuredLogger 5 | 6 | Dead-simple structured logging in ruby with a dead-simple codebase. No dependencies, everything logs to stdout, and simple hooks to customize. That's it. 7 | 8 | ```ruby 9 | gem 'simple_structured_logger' 10 | ``` 11 | 12 | ## Usage 13 | 14 | You can use this logger anywhere. Class methods, instance methods, use it as a logger for libraries, etc. 15 | 16 | Some examples: 17 | 18 | ```ruby 19 | # in a console or simple script 20 | include SimpleStructuredLogger 21 | log.info 'core message', key: Time.now.to_i 22 | 23 | # in class & instance methods 24 | class LoggingInModule 25 | include SimpleStructuredLogger 26 | 27 | def self.log_something 28 | log.info 'including the module enables a class and instance method', key: Time.now.to_i 29 | end 30 | 31 | def log_something_else 32 | log.info 'the class and instance method share the same logging context', key: Time.now.to_i 33 | end 34 | end 35 | 36 | # So, how do I set the context? How can I customize how it's set? 37 | SimpleStructuredLogger.configure do 38 | expand_context do |context| 39 | # you can pass in a object and use `expand_context` to extract the relevant keys 40 | if context[:user] 41 | context[:user_id] = context[:user].id 42 | context[:user_name] = context[:user].name 43 | end 44 | 45 | context 46 | end 47 | end 48 | 49 | class ExampleJob 50 | def perform(user_id) 51 | user = get_user(user_id, job_argument) 52 | log.set_context(user: user, job: self.class, job_argument: job_argument) 53 | log.info 'the log will contain the user_id, job_argument, and job class' 54 | 55 | # you can also add additional default pairs without resetting context 56 | log.default_tags[:something] = 'else' 57 | end 58 | end 59 | 60 | # Can you pass object arguments as values and automatically expand them? Well, yes, you can! 61 | SimpleStructuredLogger.configure do 62 | expand_log do |tags, default_tags| 63 | if tags[:stripe_resource] && tags[:stripe_resource].respond_to?(:id) 64 | stripe_resource = tags.delete(:stripe_resource) 65 | tags[:stripe_resource_id] = stripe_resource.id 66 | tags[:stripe_resource_type] = stripe_resource.class.to_s 67 | end 68 | 69 | # this is a really nice pattern I like to use. The `metric` key can trigger a call out to your observability tooling 70 | if tags[:metric] 71 | dimensions = default_tags.slice(:stripe_user_id, :other_default_tag) 72 | metrics.track_counter(tags[:metric], dimensions: dimensions) 73 | end 74 | 75 | tags 76 | end 77 | end 78 | 79 | # want simple formatting? You got it! 80 | SimpleStructuredLogger.logger.formatter = proc do |severity, _datetime, _progname, msg| 81 | "#{severity}: #{msg}\n" 82 | end 83 | 84 | # Configure the logger directly if you need to 85 | SimpleStructuredLogger.logger.level(Logger::INFO) 86 | ``` 87 | 88 | Want to change the log level quickly? Without modifying source? 89 | 90 | ```shell 91 | LOG_LEVEL=DEBUG ruby your_script.rb 92 | 93 | # case does not matter 94 | LOG_LEVEL=info ruby your_script.rb 95 | ``` 96 | 97 | ## Design Goals 98 | 99 | * Extremely simple codebase that's easy to read and override 100 | * Structured logging that reads nicely and is easy to filter using grep or something like Papertrail 101 | * Ability to easily set context, and expand context with user-configurable hook 102 | * Ability to easily add structured log pre-processing. I want to be able to pass 103 | in an object specific to my application and for the relevant important keys to 104 | be expanded automatically. 105 | * `Rails.logger = SimpleStructuredLogger.new(STDOUT)` 106 | * Not designed around massive systems or scale: no JSON logging, multiple log destinations, and other fanciness. 107 | * Don't build in fancy pre-processing for errors or other common ruby objects 108 | 109 | ### Opinionated Devops Setup 110 | 111 | * Errors are tracked using Rollbar, Airbrake, Sentry, etc. 112 | * Log to STDOUT 113 | * Pipe STDOUT to PaperTrail, Loggly, etc 114 | * Great for Heroku or [dokku/docker](http://mikebian.co/sending-dokku-container-logs-to-papertrail/) hosted system 115 | 116 | ## Alternatives 117 | 118 | * https://github.com/jordansissel/ruby-cabin 119 | * https://github.com/asenchi/scrolls 120 | * https://github.com/stripe/chalk-log 121 | * https://github.com/nishidayuya/structured_logger 122 | * https://github.com/rocketjob/semantic_logger 123 | 124 | ## Related 125 | 126 | * https://github.com/roidrage/lograge 127 | 128 | ## Why is structured logging important? 129 | 130 | * https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying 131 | * http://juliusdavies.ca/logging.html 132 | 133 | ## What about Rail's tagged logging? 134 | 135 | Tagged logging is not structured logging. I want to be able to search through 136 | PaperTrail/Splunk/etc and easily grab an audit trail for a specific context, i.e. `the_job=FailingJob the_user=1`. 137 | 138 | ## Testing 139 | 140 | ``` 141 | bundle exec rake 142 | ``` 143 | 144 | ## TODO 145 | 146 | - [ ] Support logs as blocks? --------------------------------------------------------------------------------