├── .gitignore ├── .ruby-gemset ├── .ruby-version ├── Gemfile ├── Gemfile.lock ├── MIT-LICENSE ├── README.md └── rds-pgbadger.rb /.gitignore: -------------------------------------------------------------------------------- 1 | out/* 2 | *.swp 3 | -------------------------------------------------------------------------------- /.ruby-gemset: -------------------------------------------------------------------------------- 1 | rds-pgbadger 2 | -------------------------------------------------------------------------------- /.ruby-version: -------------------------------------------------------------------------------- 1 | ruby-2.1.2 2 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | gem 'ox' 4 | gem 'aws-sdk-core' 5 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: https://rubygems.org/ 3 | specs: 4 | aws-sdk-core (2.0.17) 5 | builder (~> 3.0) 6 | jmespath (~> 1.0) 7 | multi_json (~> 1.0) 8 | multi_xml (~> 0.5) 9 | builder (3.2.2) 10 | jmespath (1.0.2) 11 | multi_json (~> 1.0) 12 | multi_json (1.10.1) 13 | multi_xml (0.5.5) 14 | ox (2.1.6) 15 | 16 | PLATFORMS 17 | ruby 18 | 19 | DEPENDENCIES 20 | aws-sdk-core 21 | ox 22 | -------------------------------------------------------------------------------- /MIT-LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016 Sport Ngin 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # RDS PgBadger 2 | 3 | A tool to easily download logs from an AWS RDS instance and generate [pgBadger](http://dalibo.github.io/pgbadger/) reports. 4 | 5 | ## Dependencies 6 | 7 | pgBadger must be installed. Download and install from the [site](http://dalibo.github.io/pgbadger/) or on OSX run `brew install pgbadger`. 8 | 9 | Setup your AWS credentials in your `~/.fog` file. Example: 10 | 11 | ``` 12 | env1: 13 | aws_access_key_id: tacos 14 | aws_secret_access_key: foobar 15 | env1: 16 | aws_access_key_id: cats 17 | aws_secret_access_key: foobaz 18 | ``` 19 | 20 | If you do not have a fog file, the AWS SDK will try to use its [regular](https://github.com/aws/aws-sdk-core-ruby#credentials) credential resolving methods. 21 | 22 | ## Usage 23 | 24 | Execute the Ruby script proving the target environment and RDS instance identifier. 25 | 26 | ``` 27 | ./rds-pgbadger.rb -e env1 -i my-rds-instance 28 | ``` 29 | 30 | It will download the log files and generate a report in the out folder named after the instance id and a timestamp of when the script executed. 31 | 32 | You can additionally filter the retrieved log files by date: 33 | 34 | ``` 35 | ./rds-pgbadger.rb -e env1 -i my-rds-instance -d 2015-01-13 36 | ``` 37 | 38 | 39 | -------------------------------------------------------------------------------- /rds-pgbadger.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | require 'optparse' 4 | require 'yaml' 5 | require 'ox' 6 | require 'aws-sdk-core' 7 | 8 | options = {} 9 | OptionParser.new do |opts| 10 | opts.banner = "Usage: rds-pgbadger.rb [options]" 11 | 12 | opts.on('-e', '--env NAME', 'Environement name') { |v| options[:env] = v } 13 | opts.on('-i', '--instance-id NAME', 'RDS instance identifier') { |v| options[:instance_id] = v } 14 | opts.on('-d', '--date DATE', 'Filter logs to given date in format YYYY-MM-DD.') { |v| options[:date] = v } 15 | 16 | end.parse! 17 | 18 | raise OptionParser::MissingArgument.new(:env) if options[:env].nil? 19 | raise OptionParser::MissingArgument.new(:instance_id) if options[:instance_id].nil? 20 | 21 | creds = YAML.load(File.read(File.expand_path('~/.fog'))) 22 | 23 | puts "Instantiating RDS client for #{options[:env]} environment." 24 | rds = Aws::RDS::Client.new( 25 | region: 'us-east-1', 26 | access_key_id: creds[options[:env]]['aws_access_key_id'], 27 | secret_access_key: creds[options[:env]]['aws_secret_access_key'] 28 | ) 29 | log_files = rds.describe_db_log_files(db_instance_identifier: options[:instance_id], filename_contains: "postgresql.log.#{options[:date]}")[:describe_db_log_files].map(&:log_file_name) 30 | 31 | dir_name = "#{options[:instance_id]}-#{Time.now.to_i}" 32 | 33 | Dir.mkdir("out/#{dir_name}") 34 | Dir.mkdir("out/#{dir_name}/error") 35 | log_files.each do |log_file| 36 | puts "Downloading log file: #{log_file}" 37 | open("out/#{dir_name}/#{log_file}", 'w') do |f| 38 | rds.download_db_log_file_portion(db_instance_identifier: options[:instance_id], log_file_name: log_file).each do |r| 39 | print "." 40 | f.puts r[:log_file_data] 41 | end 42 | puts "." 43 | end 44 | puts "Saved log to out/#{dir_name}/#{log_file}." 45 | end 46 | puts "Generating PG Badger report." 47 | `pgbadger --prefix "%t:%r:%u@%d:[%p]:" --outfile out/#{dir_name}/#{dir_name}.html out/#{dir_name}/error/*.log.*` 48 | puts "Opening report out/#{dir_name}/#{dir_name}.html." 49 | `open out/#{dir_name}/#{dir_name}.html` 50 | 51 | --------------------------------------------------------------------------------