├── .gitignore ├── .ruby-gemset ├── .ruby-version ├── .travis.yml ├── Gemfile ├── LICENSE.txt ├── README.md ├── Rakefile ├── jynx.jpg ├── lib ├── logger_jynx.rb ├── service_jynx.rb └── service_jynx │ └── version.rb ├── run_test ├── service_jynx.gemspec └── spec ├── service_jynx_spec.rb └── spec_helper.rb /.gitignore: -------------------------------------------------------------------------------- 1 | *.gem 2 | *.rbc 3 | .bundle 4 | .config 5 | .yardoc 6 | Gemfile.lock 7 | InstalledFiles 8 | _yardoc 9 | coverage 10 | doc/ 11 | lib/bundler/man 12 | pkg 13 | rdoc 14 | spec/reports 15 | test/tmp 16 | test/version_tmp 17 | tmp 18 | -------------------------------------------------------------------------------- /.ruby-gemset: -------------------------------------------------------------------------------- 1 | jynx_gemset 2 | -------------------------------------------------------------------------------- /.ruby-version: -------------------------------------------------------------------------------- 1 | 2.0.0-p247 2 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: ruby 2 | rvm: 3 | - 1.9.3 4 | - 2.0.0 5 | gemfile: 6 | - Gemfile 7 | branches: 8 | only: 9 | - master 10 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | gemspec 4 | 5 | gem 'rake' 6 | 7 | group :test do 8 | gem 'rspec' 9 | gem 'pry' 10 | gem 'pry-debugger' 11 | end -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2013 Avner Cohen 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining 4 | a copy of this software and associated documentation files (the 5 | "Software"), to deal in the Software without restriction, including 6 | without limitation the rights to use, copy, modify, merge, publish, 7 | distribute, sublicense, and/or sell copies of the Software, and to 8 | permit persons to whom the Software is furnished to do so, subject to 9 | the following conditions: 10 | 11 | The above copyright notice and this permission notice shall be 12 | included in all copies or substantial portions of the Software. 13 | 14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 15 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 16 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 17 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 18 | LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 19 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 20 | WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Service-Jynx 2 | [![Build Status](https://secure.travis-ci.org/AvnerCohen/service-jynx.png)](http://travis-ci.org/AvnerCohen/service-jynx) 3 | 4 | Eurasian Wryneck - ***Jynx torquilla*** 5 | 6 | ![Eurasian Wryneck](jynx.jpg) 7 | 8 | **Jinx** - ***A condition or period of bad luck that appears to have been caused by a specific person or thing.*** 9 | 10 | A simple solution, to allow a Ruby application to manage automatic failover and block calls to an external service and return a stubbed data when service is reported as down. 11 | 12 | The code is MRI depended and is not thread safe(!), is is also designed specifically to run on a single VM and manage in memory hashes of data, though it can very well be executed with an external shared persistance counters such as, say, Redis. 13 | 14 | 15 | ````ruby 16 | 17 | def index 18 | begin 19 | response = {country: "Israel"} 20 | if ServiceJynx.alive?("inbox") 21 | response = {country: HTTParty.get("https://api.github.com/users/AvnerCohen", :timeout => 20)["location"]} 22 | else 23 | response.merge!({error: "service down ! #{__method__}"}) 24 | end 25 | rescue Exception => e 26 | ServiceJynx.failure!("inbox") 27 | response.merge!({error: "exception occured, #{__method__}"}) 28 | ensure 29 | render json: response and return 30 | end 31 | end 32 | 33 | ```` 34 | 35 | 36 | ## Complete Use case extracted 37 | 38 | [1] Register the service for Jynx monitoring at application start time: 39 | 40 | ```` 41 | opts = { 42 | time_window_in_seconds: 20, 43 | max_errors: 10, 44 | grace_period: 60 45 | } 46 | ServiceJynx.register!("github_api", opts) 47 | ```` 48 | 49 | [2] Define a module that wraps your HTTP calls to have a generic safe api 50 | 51 | ```` 52 | module HttpInternalWrapper 53 | extend self 54 | def get_api(service_name, url, &on_error_block) 55 | if ServiceJynx.alive?(service_name) 56 | HTTParty.get(url, :timeout => 20) 57 | else 58 | on_error_block.call("#{service_name}_service set as down.") 59 | end 60 | rescue Exception => e 61 | ServiceJynx.failure!(service_name) 62 | on_error_block.call("Exception in #{service_name}_service exceution - #{e.message}") 63 | end 64 | end 65 | ```` 66 | 67 | [3] Execute with a stubbed on_error_block that gets executed on failure or service down 68 | 69 | ```` 70 | HttpInternalWrapper.get_api("github_api", "https://api.github.com/users/AvnerCohen") do |msg| 71 | @logger.error "#{msg} -- #{path} failed at #{__method__} !!" 72 | {error: "Github api is currently unavailable"} # stub an empty hash with error messages 73 | end 74 | ```` 75 | 76 | 77 | ## Defaults 78 | 79 | Defined when registering a service: 80 | 81 | ***time_window_in_seconds***: **10** 82 | 83 | ***max_errors***: **40** 84 | 85 | ***grace_period***: **360** 86 | 87 | 88 | Defaults means that *40 errors* during *10 seconds* would turn the service automatically off, for 5 minutes. 89 | 90 | ## Methods 91 | 92 | ````ruby 93 | 94 | ServiceJynx.register!(:name, {time_window_in_seconds: 360, max_errors: 40}) 95 | ServiceJynx.alive?(:name) 96 | ServiceJynx.failure!(:name) 97 | ServiceJynx.down!(:name) 98 | ServiceJynx.up!(:name) 99 | 100 | ```` 101 | 102 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require 'rake' 2 | require 'rspec/core/rake_task' 3 | 4 | desc 'Run specs' 5 | RSpec::Core::RakeTask.new do |t| 6 | t.pattern = './spec/**/*_spec.rb' 7 | end 8 | 9 | task :default => :spec 10 | task :test => :spec -------------------------------------------------------------------------------- /jynx.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AvnerCohen/service-jynx/01f21aee273d12021d20e5d7b97e13ec6b9bc291/jynx.jpg -------------------------------------------------------------------------------- /lib/logger_jynx.rb: -------------------------------------------------------------------------------- 1 | module LoggerJynx 2 | def self.logger 3 | @logger ||= (rails_logger || default_logger) 4 | end 5 | 6 | def self.rails_logger 7 | (defined?(Rails) && Rails.respond_to?(:logger) && Rails.logger) || 8 | (defined?(RAILS_DEFAULT_LOGGER) && RAILS_DEFAULT_LOGGER.respond_to?(:debug) && RAILS_DEFAULT_LOGGER) 9 | end 10 | 11 | def self.default_logger 12 | require 'logger' 13 | l = Logger.new(STDOUT) 14 | l.level = Logger::INFO 15 | l 16 | end 17 | 18 | def self.logger=(logger) 19 | @logger = logger 20 | end 21 | end -------------------------------------------------------------------------------- /lib/service_jynx.rb: -------------------------------------------------------------------------------- 1 | require "service_jynx/version" 2 | require "logger_jynx" 3 | 4 | module ServiceJynx 5 | 6 | @counters = {} 7 | def self.counters 8 | @counters 9 | end 10 | 11 | def self.register!(name, options = {}) 12 | @counters[name] = Jynx.new(name, options) 13 | end 14 | 15 | def self.flush! 16 | @counters = {} 17 | end 18 | 19 | def self.alive?(name) 20 | @counters[name].alive? == true 21 | end 22 | 23 | def self.down!(name, reason) 24 | @counters[name].down!(reason) 25 | end 26 | 27 | def self.up!(name) 28 | @counters[name].up! 29 | end 30 | 31 | def self.failure!(name) 32 | jynx = @counters[name] 33 | now = Time.now.to_i 34 | jynx.errors << now 35 | jynx.clean_aged(now) 36 | if jynx.errors.count > jynx.max_errors 37 | down!(name, "Max error count (#{jynx.max_errors}) reached at #{Time.now}.") 38 | :WENT_DOWN 39 | else 40 | :FAIL_MARKED 41 | end 42 | end 43 | 44 | 45 | class Jynx 46 | attr_accessor :errors, :name, :time_window_in_seconds, :max_errors, :alive, :down_at, :grace_period 47 | def initialize(name, options) 48 | @name = name 49 | @down_at = 0 50 | @alive = true 51 | @errors = [] 52 | opts = { 53 | time_window_in_seconds: 10, 54 | max_errors: 40, 55 | grace_period: 360 56 | }.merge!(options) 57 | @time_window_in_seconds = opts[:time_window_in_seconds] 58 | @max_errors = opts[:max_errors] 59 | @grace_period = opts[:grace_period] 60 | end 61 | 62 | ## clean up errors that are older than time_window_in_secons 63 | def clean_aged(time_now) 64 | near_past = time_now - @time_window_in_seconds 65 | @errors = @errors.reverse.select{|time_stamp| time_stamp > near_past }.reverse.to_a 66 | end 67 | 68 | def down!(reason) 69 | @alive = false 70 | @down_at = Time.now.to_i 71 | LoggerJynx.logger.error "Shutting down [#{@name}] #{reason} at #{@down_at}." 72 | end 73 | 74 | def up! 75 | LoggerJynx.logger.error "Upping [#{@name}]." 76 | @alive = true 77 | @down_at = 0 78 | end 79 | 80 | def alive? 81 | return true if @alive 82 | near_past = Time.now.to_i - @grace_period 83 | up! if (@down_at < near_past) and return true 84 | false 85 | end 86 | 87 | end 88 | 89 | 90 | 91 | end 92 | -------------------------------------------------------------------------------- /lib/service_jynx/version.rb: -------------------------------------------------------------------------------- 1 | module ServiceJynx 2 | VERSION = "0.0.3" 3 | end 4 | -------------------------------------------------------------------------------- /run_test: -------------------------------------------------------------------------------- 1 | rspec ./spec/**.* 2 | -------------------------------------------------------------------------------- /service_jynx.gemspec: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | lib = File.expand_path('../lib', __FILE__) 3 | $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) 4 | require 'service_jynx/version' 5 | 6 | Gem::Specification.new do |spec| 7 | spec.name = "service_jynx" 8 | spec.version = ServiceJynx::VERSION 9 | spec.authors = ["Avner Cohen"] 10 | spec.email = ["israbirding@gmail.com"] 11 | spec.summary = %q{Use errors count over sliding windows to block calls to an external service or method, or whatever.} 12 | spec.description = %q{Use errors count over sliding windows to block calls to an external service or method, or whatever.} 13 | spec.homepage = "https://github.com/AvnerCohen/service-jynx" 14 | spec.license = "MIT" 15 | 16 | spec.files = `git ls-files`.split($/) 17 | spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) } 18 | spec.test_files = spec.files.grep(%r{^(test|spec|features)/}) 19 | spec.require_paths = ["lib"] 20 | 21 | end 22 | -------------------------------------------------------------------------------- /spec/service_jynx_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | # Add when debugging 4 | # require 'pry' 5 | # require 'pry-debugger' 6 | 7 | describe ServiceJynx do 8 | before(:all) do 9 | ServiceJynx.flush! 10 | end 11 | 12 | 13 | it "should add new services to counter when registerd" do 14 | jynx = ServiceJynx.counters.fetch(:dummy_service, :ServiceNotFound) 15 | jynx.should eq(:ServiceNotFound) 16 | ServiceJynx.register!(:dummy_service) 17 | jynx = ServiceJynx.counters.fetch(:dummy_service) 18 | jynx.errors == 0 19 | end 20 | 21 | it "should allow registering with options" do 22 | ServiceJynx.register!(:dummy_service, {max_errors: 5}) 23 | jynx = ServiceJynx.counters.fetch(:dummy_service, :ServiceNotFound) 24 | jynx.max_errors.should eq(5) 25 | end 26 | 27 | it "should allow checking if service is alive" do 28 | ServiceJynx.register!(:dummy_service) 29 | ServiceJynx.alive?(:dummy_service).should eq(true) 30 | end 31 | 32 | it "should allow shutting down a service" do 33 | ServiceJynx.register!(:dummy_service) 34 | ServiceJynx.alive?(:dummy_service).should eq(true) 35 | ServiceJynx.down!(:dummy_service, "Because of testing") 36 | ServiceJynx.alive?(:dummy_service).should eq(false) 37 | end 38 | 39 | it "should allow upping a service" do 40 | ServiceJynx.register!(:dummy_service) 41 | ServiceJynx.alive?(:dummy_service).should eq(true) 42 | ServiceJynx.down!(:dummy_service, "Because of testing") 43 | ServiceJynx.alive?(:dummy_service).should eq(false) 44 | ServiceJynx.up!(:dummy_service) 45 | ServiceJynx.alive?(:dummy_service).should eq(true) 46 | end 47 | 48 | it "should allow marking a failure" do 49 | ServiceJynx.register!(:dummy_service) 50 | jynx = ServiceJynx.counters.fetch(:dummy_service) 51 | jynx.errors.length.should eq(0) 52 | ServiceJynx.failure!(:dummy_service) 53 | jynx.errors.length.should eq(1) 54 | end 55 | 56 | it "should allow marking multiple failures" do 57 | ServiceJynx.register!(:dummy_service) 58 | jynx = ServiceJynx.counters.fetch(:dummy_service) 59 | jynx.errors.length.should eq(0) 60 | 10.times {ServiceJynx.failure!(:dummy_service)} 61 | jynx.errors.length.should eq(10) 62 | end 63 | 64 | it "should allow overrding defaults" do 65 | ServiceJynx.register!(:dummy_service, {time_window_in_seconds: 999}) 66 | jynx = ServiceJynx.counters.fetch(:dummy_service) 67 | jynx.time_window_in_seconds.should eq(999) 68 | end 69 | 70 | it "should clean old errors" do 71 | ServiceJynx.register!(:dummy_service, {time_window_in_seconds: 2}) 72 | jynx = ServiceJynx.counters.fetch(:dummy_service) 73 | jynx.errors.length.should eq(0) 74 | 10.times {ServiceJynx.failure!(:dummy_service)} 75 | jynx.errors.length.should eq(10) 76 | sleep 5 ## make sure aged errors are cleaned 77 | 10.times {ServiceJynx.failure!(:dummy_service)} 78 | jynx.errors.length.should eq(10) 79 | end 80 | 81 | it "should auto disable when errors limit reached old errors" do 82 | ServiceJynx.register!(:dummy_service, {time_window_in_seconds: 2, max_errors: 20}) 83 | jynx = ServiceJynx.counters.fetch(:dummy_service) 84 | ServiceJynx.alive?(:dummy_service).should eq(true) 85 | 10.times {ServiceJynx.failure!(:dummy_service)} 86 | ServiceJynx.alive?(:dummy_service).should eq(true) 87 | 11.times {ServiceJynx.failure!(:dummy_service)} 88 | ServiceJynx.alive?(:dummy_service).should eq(false) 89 | end 90 | 91 | it "should report result for failure" do 92 | ServiceJynx.register!(:dummy_service, {time_window_in_seconds: 2, max_errors: 3}) 93 | jynx = ServiceJynx.counters.fetch(:dummy_service) 94 | ServiceJynx.alive?(:dummy_service).should eq(true) 95 | ServiceJynx.failure!(:dummy_service).should eq(:FAIL_MARKED) 96 | ServiceJynx.failure!(:dummy_service).should eq(:FAIL_MARKED) 97 | ServiceJynx.failure!(:dummy_service).should eq(:FAIL_MARKED) 98 | 99 | ## After 3 errors, report as down 100 | ServiceJynx.failure!(:dummy_service).should eq(:WENT_DOWN) 101 | ServiceJynx.alive?(:dummy_service).should eq(false) 102 | end 103 | 104 | 105 | it "should auto disable when errors limit reached old errors and restart again when grace period passes" do 106 | ServiceJynx.register!(:dummy_service, {time_window_in_seconds: 2, max_errors: 20, grace_period: 5}) 107 | jynx = ServiceJynx.counters.fetch(:dummy_service) 108 | ServiceJynx.alive?(:dummy_service).should eq(true) 109 | 10.times {ServiceJynx.failure!(:dummy_service)} 110 | ServiceJynx.alive?(:dummy_service).should eq(true) 111 | 11.times {ServiceJynx.failure!(:dummy_service)} 112 | ServiceJynx.alive?(:dummy_service).should eq(false) 113 | sleep 7 114 | ServiceJynx.alive?(:dummy_service).should eq(true) 115 | end 116 | 117 | 118 | end -------------------------------------------------------------------------------- /spec/spec_helper.rb: -------------------------------------------------------------------------------- 1 | require 'rspec' 2 | 3 | Dir["#{File.dirname(__FILE__)}/../lib/**/*.rb"].each { |f| load(f) } 4 | --------------------------------------------------------------------------------