├── .gitignore ├── CHANGELOG.md ├── CONTRIBUTORS ├── Gemfile ├── LICENSE ├── README.md ├── Rakefile ├── lib └── logstash │ └── outputs │ └── sentry.rb ├── logstash-output-sentry.gemspec └── spec └── outputs └── sentry_spec.rb /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | *.gem 3 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## 0.4.2 4 | - Fix string templating for key, url, secret and project_id fields 5 | 6 | ## 0.4.1 7 | - Fixing versions for development dependencies. 8 | 9 | ## 0.4.0 10 | - Implements the full sentry interface. 11 | 12 | ## 0.2.0 13 | - Improving sentry.rb script based on antho31/logstash-output-sentry and bigpandaio/logstash-output-sentry projects. 14 | - Creating documentation (starting from bigpandaio/logstash-output-sentry with some fixes for the differences that we have introduced to the code). 15 | 16 | ## 0.1.0 17 | - Starting project from clarkdave/logstash-sentry.rb. 18 | - Packing script in a gem so it can be easily installed. 19 | - Adding parameters so that we can use the plugin to send data to our own sentry server. 20 | -------------------------------------------------------------------------------- /CONTRIBUTORS: -------------------------------------------------------------------------------- 1 | The following is a list of people who have contributed ideas, code, bug 2 | reports, or in general have helped this plugin along its way. 3 | 4 | Contributors: 5 | * Javier Matos Odut (javiermatos) 6 | # Julian Rüth (saraedum) 7 | 8 | Note: If you've sent us patches, bug reports, or otherwise contributed to this 9 | plugin, and you aren't on the list above and want to be, please let us know and 10 | we'll make sure you're here. Contributions from folks like you are what make 11 | open source awesome. 12 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | gemspec 3 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Licensed under the Apache License, Version 2.0 (the "License"); 2 | you may not use this file except in compliance with the License. 3 | You may obtain a copy of the License at 4 | 5 | http://www.apache.org/licenses/LICENSE-2.0 6 | 7 | Unless required by applicable law or agreed to in writing, software 8 | distributed under the License is distributed on an "AS IS" BASIS, 9 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 10 | See the License for the specific language governing permissions and 11 | limitations under the License. 12 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Logstash-Output-Sentry Plugin 2 | 3 | This is a plugin for [Logstash](https://github.com/elasticsearch/logstash). 4 | 5 | This plugin gives you the possibility to send your output parsed with Logstash to a Sentry host. 6 | 7 | This plugin is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way. 8 | 9 | But keep in mind that this is not an official plugin, and this plugin is not supported by the Logstash community. 10 | 11 | 12 | ## Documentation 13 | 14 | ### Installation 15 | 16 | You must have [Logstash](https://github.com/elasticsearch/logstash) installed for using this plugin. You can find instructions on how to install it on the [Logstash website](https://www.elastic.co/downloads/logstash). Maybe the easiest way to install is using their [repositories](https://www.elastic.co/guide/en/logstash/current/package-repositories.html). 17 | 18 | As this plugin has been shared on [RubyGems](https://rubygems.org) with the name [logstash-output-sentry](https://rubygems.org/gems/logstash-output-sentry) you can install it using the following command from your Logstash installation path: 19 | 20 | ```sh 21 | bin/logstash-plugin install logstash-output-sentry 22 | ``` 23 | 24 | When installing from official repository as suggested below, the installation path is `/opt/logstash`. 25 | 26 | ### Usage 27 | 28 | [Sentry](https://getsentry.com/) is a modern error logging and aggregation platform. 29 | It's important to note that Sentry should not be thought of as a log stream, but as an aggregator. 30 | It fits somewhere in-between a simple metrics solution (such as Graphite) and a full-on log stream aggregator (like Logstash). 31 | 32 | * In Sentry, generate and get your client key (Settings -> Client key). The client key has this form: 33 | ``` 34 | [http|https]://[key]:[secret]@[host]/[project_id] 35 | ``` 36 | 37 | * Setup logstash to write to sentry: 38 | ```ruby 39 | output { 40 | sentry { 41 | 'key' => "yourkey" 42 | 'secret' => "yoursecret" 43 | 'project_id' => "yourprojectid" 44 | } 45 | } 46 | ``` 47 | 48 | * By default, the plugin connects to https://app.getsentry.com/api. Set the `url` if you have installed Sentry on your own machine: 49 | ```ruby 50 | output { 51 | sentry { 52 | 'url' => "http://local.sentry:9000/api" 53 | 'key' => "yourkey" 54 | 'secret' => "yoursecret" 55 | 'project_id' => "yourprojectid" 56 | } 57 | } 58 | ``` 59 | 60 | * If you don't configure anything else, the necessary fields will be set automatically, i.e., `event_id`, `timestamp` (set to `@timestamp`), `logger` (set to `"logstash"`) and `platform` (set to `"other"`). All the other fields from logstash are going to be put into the `extra` field in sentry. Additionally, the `level` is set to `"error"` and the `server_name` to the value of `host`. 61 | 62 | * The plugin can write to all the fields that the sentry interface currently supports, i.e., `timestamp`, `message`, `logger`, `platform`, `sdk`, `level`, `culprit`, `server_name`, `release`, `tags`, `environment`, `modules`, `extra`, `fingerprint`, `exception`, `sentry.interface.Message`, `stacktrace`, `template`, `breadcrumbs`, `contexts`, `request`, `threads`, `user`, `debug_meta`, `repos`, `sdk`. To set a field, you can either read the value from another field or set it to a constant value by setting the corresponding `_value`: 63 | ```ruby 64 | output { 65 | sentry { 66 | 'message' => "message" # sets message to the contents of the message field 67 | 'environment' => "[tag][Environment]" # sets message to the contents of the field Environment in tag 68 | 'exception' => "[@metadata][sentry][exception]" # sets exception to the metadata field, see below for a complete example 69 | 'user_value' => "nobody" # sets the user to the constant "nobody" 70 | 71 | 'key' => "yourkey" 72 | 'secret' => "yoursecret" 73 | 'project_id' => "yourprojectid" 74 | } 75 | } 76 | ``` 77 | 78 | * You can also prepare the settings in a filter to create a cleaner config: 79 | ```ruby 80 | input { 81 | syslog { 82 | port => 514 83 | type => "syslog" 84 | } 85 | 86 | tcp { 87 | port => 1514 88 | type => "cisco-ios" 89 | } 90 | 91 | tcp { 92 | port => 2514 93 | type => "application" 94 | } 95 | } 96 | filter { 97 | if [type] == "syslog" { 98 | mutate { 99 | add_field => { 100 | "[@metadata][sentry][msg]" => "%{host}" 101 | "[@metadata][sentry][severity]" => "%{severity}" 102 | "[@metadata][sentry][host]" => "192.168.1.101" 103 | "[@metadata][sentry][pid]" => "2" 104 | "[@metadata][sentry][key]" => "d3921923d34a4344878f7b83e2061229" 105 | "[@metadata][sentry][secret]" => "d0163ef306c04148aee49fe4ce7621b1" 106 | } 107 | } 108 | } 109 | else if [type] == "cisco-ios" { 110 | mutate { 111 | add_field => { 112 | "[@metadata][sentry][msg]" => "%{host}" 113 | "[@metadata][sentry][severity]" => "%{severity}" 114 | "[@metadata][sentry][host]" => "192.168.1.101" 115 | "[@metadata][sentry][pid]" => "3" 116 | "[@metadata][sentry][key]" => "d398098q2349883e206178098" 117 | "[@metadata][sentry][secret]" => "da098d890f098d09809f6098c87e0" 118 | } 119 | } 120 | } 121 | else if [type] == "application" { 122 | mutate { 123 | add_field => { 124 | "[@metadata][sentry][msg]" => "%{host}" 125 | "[@metadata][sentry][severity]" => "%{severity}" 126 | "[@metadata][sentry][host]" => "192.168.1.150" 127 | "[@metadata][sentry][pid]" => "4" 128 | "[@metadata][sentry][key]" => "d39dc435326d987d5678e98d76cf78098" 129 | "[@metadata][sentry][secret]" => "07d09876d543d2a345e43c4e567d" 130 | } 131 | } 132 | } 133 | } 134 | output { 135 | sentry { 136 | server_name => "[@metadata][sentry][host]" 137 | level => "[@metadata][sentry][severity]" 138 | message => "[@metadata][sentry][msg]" 139 | 140 | project_id => "%{[@metadata][sentry][pid]}" 141 | key => "%{[@metadata][sentry][key]}" 142 | secret => "%{[@metadata][sentry][secret]}" 143 | } 144 | } 145 | ``` 146 | 147 | ## Contributing 148 | 149 | All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin. 150 | 151 | Note that this plugin has been written from [this Gist](https://gist.github.com/clarkdave/edaab9be9eaa9bf1ee5f). 152 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require 'logstash/devutils/rake' 2 | -------------------------------------------------------------------------------- /lib/logstash/outputs/sentry.rb: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | require 'logstash/outputs/base' 3 | require 'logstash/namespace' 4 | require 'json' 5 | 6 | class LogStash::Outputs::Sentry < LogStash::Outputs::Base 7 | config_name 'sentry' 8 | concurrency :shared 9 | 10 | # Sentry API URL 11 | config :url, :validate => :uri, :required => false, :default => 'https://app.getsentry.com/api' 12 | 13 | # Project id, key and secret 14 | config :project_id, :validate => :string, :required => true 15 | config :key, :validate => :string, :required => true 16 | config :secret, :validate => :string, :required => true 17 | 18 | def self.sentry_key(name, field_default=nil, value_default=nil) 19 | name = name.to_s if name.is_a?(Symbol) 20 | 21 | @sentry_keys ||= [] 22 | @sentry_keys << name 23 | 24 | opts = { 25 | :validate => :string, 26 | :required => false, 27 | } 28 | 29 | config name, opts.merge(if field_default then {:default => field_default} else {} end) 30 | config "#{name}_value", opts.merge(if value_default then {:default => value_default} else {} end) 31 | end 32 | class << self; attr_accessor :sentry_keys end 33 | # https://docs.sentry.io/clientdev/attributes/ 34 | sentry_key :timestamp, field_default='@timestamp' 35 | sentry_key :message 36 | sentry_key :_logger 37 | sentry_key :platform 38 | sentry_key :sdk 39 | sentry_key :level, value_default='error' 40 | sentry_key :culprit 41 | sentry_key :server_name, field_default='host' 42 | sentry_key :release 43 | sentry_key :tags 44 | sentry_key :environment 45 | sentry_key :modules 46 | sentry_key :extra, field_default='' # puts all fields into extra 47 | sentry_key :fingerprint 48 | # https://docs.sentry.io/clientdev/interfaces/exception/ 49 | sentry_key :exception 50 | # https://docs.sentry.io/clientdev/interfaces/message/ 51 | sentry_key :"sentry.interfaces.Message" 52 | # https://docs.sentry.io/clientdev/interfaces/stacktrace/ 53 | sentry_key :stacktrace 54 | # https://docs.sentry.io/clientdev/interfaces/template/ 55 | sentry_key :template 56 | # https://docs.sentry.io/clientdev/interfaces/breadcrumbs/ 57 | sentry_key :breadcrumbs 58 | # https://docs.sentry.io/clientdev/interfaces/contexts/ 59 | sentry_key :contexts 60 | # https://docs.sentry.io/clientdev/interfaces/http/ 61 | sentry_key :request 62 | # https://docs.sentry.io/clientdev/interfaces/threads/ 63 | sentry_key :threads 64 | # https://docs.sentry.io/clientdev/interfaces/user/ 65 | sentry_key :user 66 | # https://docs.sentry.io/clientdev/interfaces/debug/ 67 | sentry_key :debug_meta 68 | # https://docs.sentry.io/clientdev/interfaces/repos/ 69 | sentry_key :repos 70 | # https://docs.sentry.io/clientdev/interfaces/sdk/ 71 | sentry_key :sdk 72 | 73 | public 74 | def register 75 | end 76 | 77 | def get(event, key) 78 | key = key.to_s if key.is_a?(Symbol) 79 | 80 | instance_variable_name = key.gsub(/\./, '') 81 | 82 | field = instance_variable_get("@#{instance_variable_name}") 83 | if field == '' 84 | ret = event.to_hash 85 | ret.delete('tags') 86 | return ret 87 | elsif field 88 | return event.get(field) if event.get(field) 89 | end 90 | 91 | value = instance_variable_get("@#{instance_variable_name}_value") 92 | return value # can be nil 93 | end 94 | 95 | def multi_receive(events) 96 | for event in events 97 | receive(event) 98 | end 99 | end 100 | 101 | def create_packet(event, timestamp) 102 | require 'securerandom' 103 | event_id = SecureRandom.uuid.gsub('-', '') 104 | 105 | packet = { 106 | # parameters required by sentry 107 | :event_id => event_id, 108 | :timestamp => timestamp.to_s, 109 | :logger => get(event, :_logger) || "logstash", 110 | :platform => get(event, :platform) || "other", 111 | } 112 | 113 | for key in LogStash::Outputs::Sentry.sentry_keys 114 | sentry_key = key.gsub(/^_/,'') 115 | next if packet[sentry_key]; 116 | value = get(event, key) 117 | packet[sentry_key] = value if value 118 | end 119 | 120 | return packet 121 | end 122 | 123 | def send_packet(event, packet, timestamp) 124 | auth_header = "Sentry sentry_version=5," + 125 | "sentry_client=raven_logstash/0.4.0," + 126 | "sentry_timestamp=#{timestamp.to_i}," + 127 | "sentry_key=#{event.sprintf(@key)}," + 128 | "sentry_secret=#{event.sprintf(@secret)}" 129 | 130 | url = "#{event.sprintf(@url)}/#{event.sprintf(@project_id)}/store/" 131 | 132 | require 'http' 133 | response = HTTP.post(url, :body => packet.to_json, :headers => {:"X-Sentry-Auth" => auth_header}) 134 | raise "Sentry answered with #{response} and code #{response.code} to our request #{packet}" unless response.code == 200 135 | end 136 | 137 | def receive(event) 138 | begin 139 | require 'time' 140 | timestamp = get(event, :timestamp) || Time.now 141 | 142 | sentry_packet = create_packet(event, timestamp) 143 | @logger.debug('Sentry packet', :sentry_packet => sentry_packet) 144 | 145 | send_packet(event, sentry_packet, timestamp) 146 | rescue Exception => e 147 | @logger.warn('Unhandled exception', :exception => e) 148 | end 149 | end 150 | end 151 | -------------------------------------------------------------------------------- /logstash-output-sentry.gemspec: -------------------------------------------------------------------------------- 1 | Gem::Specification.new do |s| 2 | s.name = 'logstash-output-sentry' 3 | s.version = '0.4.2' 4 | s.licenses = ['Apache-2.0'] 5 | s.summary = 'This output plugin sends messages to any sentry server.' 6 | s.description = 'This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install logstash-output-sentry. This gem is not a stand-alone program.' 7 | s.authors = ['Javier Matos Odut', 'Julian Rüth'] 8 | s.email = 'iam@javiermatos.com' 9 | s.homepage = 'https://github.com/javiermatos/logstash-output-sentry' 10 | s.require_paths = ['lib'] 11 | 12 | # Files 13 | s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT'] 14 | # Tests 15 | s.test_files = s.files.grep(%r{^(test|spec|features)/}) 16 | 17 | # Special flag to let us know this is actually a logstash plugin 18 | s.metadata = { 'logstash_plugin' => 'true', 'logstash_group' => 'output' } 19 | 20 | # Gem dependencies 21 | s.add_runtime_dependency 'logstash-core-plugin-api', '>= 1.60', '< 3.0' 22 | s.add_runtime_dependency 'json', '>= 1.8.0', '< 2.0.0' 23 | s.add_development_dependency 'logstash-devutils', '>= 1.0.0', '< 2.0.0' 24 | s.add_development_dependency 'logstash-codec-plain', '>= 3.0.0', '< 4.0.0' 25 | end 26 | -------------------------------------------------------------------------------- /spec/outputs/sentry_spec.rb: -------------------------------------------------------------------------------- 1 | require 'logstash/devutils/rspec/spec_helper' 2 | require 'logstash/outputs/sentry' 3 | 4 | describe LogStash::Outputs::Sentry do 5 | describe "Registration" do 6 | config <<-CONFIG 7 | output { 8 | sentry { 9 | url => "http://web:9000/api" 10 | project_id => "2" 11 | key => "Sheez5ohZ8Ohdiquei2E" 12 | secret => "vie4eituy2aYoongeege" 13 | } 14 | } 15 | CONFIG 16 | 17 | sample("...") do 18 | ; 19 | end 20 | end 21 | end 22 | --------------------------------------------------------------------------------