├── .gitignore ├── .travis.yml ├── CHANGELOG.md ├── Gemfile ├── LICENSE ├── README.md ├── Rakefile ├── lib ├── redis-semaphore.rb └── redis │ └── semaphore.rb ├── redis-semaphore.gemspec └── spec ├── semaphore_spec.rb └── spec_helper.rb /.gitignore: -------------------------------------------------------------------------------- 1 | .project 2 | *.gem 3 | .rspec 4 | .bundle 5 | .rbenv-version 6 | Gemfile.lock 7 | 8 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: ruby 2 | rvm: 3 | - "1.9.2" 4 | - "1.9.3" 5 | - "2.0.0" 6 | - "2.1.1" 7 | - jruby-19mode 8 | - rbx 9 | services: 10 | - redis-server 11 | before_install: 12 | - gem install bundler 13 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | Changelog 2 | --------- 3 | 4 | ### 0.3.1 April 17, 2016 5 | - Fix `sem.lock(0)` bug (thanks eugenk!). 6 | 7 | ### 0.3.0 January 24, 2016 8 | - Change API to include non-blocking option for `#lock` (thanks tomclose!). 9 | - Fix unwanted persisting of `available_key` (thanks dany1468!). 10 | - Fix `available_count` returning 0 for nonexisting semaphores (thanks mikeryz!). 11 | 12 | ### 0.2.4 January 11, 2015 13 | - Fix bug with TIME and redis-namespace (thanks sos4nt!). 14 | - Add expiration option (thanks jcalvert!). 15 | - Update API version logic. 16 | 17 | ### 0.2.3 September 7, 2014 18 | - Block-based locking return the value of the block (thanks frobcode!). 19 | 20 | ### 0.2.2 June 16, 2014 21 | - Fixed bug in `all_tokens` (thanks presskey!). 22 | - Fixed bug in error message (thanks Dmitriy!). 23 | 24 | ### 0.2.1 August 6, 2013 25 | - Remove dependency on Redis 2.6+ using fallback for TIME command (thanks dubdromic!). 26 | - Add ```:use_local_time``` option 27 | 28 | ### 0.2.0 June 2, 2013 29 | - Use Redis TIME command for lock timeouts (thanks dubdromic!). 30 | - Version increase because of new dependency on Redis 2.6+ 31 | 32 | ### 0.1.7 April 18, 2013 33 | - Fix bug where ```release_stale_locks!``` was not public (thanks scomma!). 34 | 35 | ### 0.1.6 March 31, 2013 36 | - Add non-ownership of tokens 37 | - Add stale client timeout (thanks timgaleckas!). 38 | 39 | ### 0.1.5 October 1, 2012 40 | - Add detection of Redis::Namespace definition to avoid potential bug (thanks ruud!). 41 | 42 | ### 0.1.4 October 1, 2012 43 | - Fixed empty namespaces (thanks ruurd!). 44 | 45 | ### 0.1.3 July 9, 2012 46 | - Tokens are now identifiable (thanks timgaleckas!). 47 | 48 | ### 0.1.2 June 1, 2012 49 | - Add redis-namespace support (thanks neovintage!). 50 | 51 | ### 0.1.1 September 17, 2011 52 | - When an exception is raised during locked period, ensure it unlocks. 53 | 54 | ### 0.1.0 August 4, 2011 55 | - Initial release. 56 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | gemspec 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (C) 2011 by David Verhasselt (david@crowdway.com) 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | THE SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Code Climate](https://codeclimate.com/github/dv/redis-semaphore.svg?branch=master)](https://codeclimate.com/github/dv/redis-semaphore) 2 | [![Build Status](https://travis-ci.org/dv/redis-semaphore.svg?branch=master)](https://travis-ci.org/dv/redis-semaphore) 3 | 4 | redis-semaphore 5 | =============== 6 | 7 | Implements a mutex and semaphore using Redis and the neat BLPOP command. 8 | 9 | The mutex and semaphore is blocking, not polling, and has a fair queue serving processes on a first-come, first-serve basis. It can also have an optional timeout after which a lock is unlocked automatically, to protect against dead clients. 10 | 11 | For more info see [Wikipedia](http://en.wikipedia.org/wiki/Semaphore_(programming)). 12 | 13 | Important change in v0.3.0 14 | =========================== 15 | 16 | If you've been using `redis-semaphore` before version `0.3.0` you should be aware that the interface for `lock` has changed slightly. Before `0.3` calling `semaphore.lock(0)` (with `0` as the timeout) would block the semaphore indefinitely, just like a redis `blpop` command would. 17 | 18 | This has changed in `0.3` to mean *do not block at all*. You can still omit the argument entirely, or pass in `nil` to get the old functionality back. Examples: 19 | 20 | ```ruby 21 | # These block indefinitely until a resource becomes available: 22 | semaphore.lock 23 | semaphore.lock(nil) 24 | 25 | # This does not block at all and rather returns immediately if there's no 26 | # resource available: 27 | semaphore.lock(0) 28 | ``` 29 | 30 | Usage 31 | ----- 32 | 33 | Create a mutex: 34 | 35 | ```ruby 36 | s = Redis::Semaphore.new(:semaphore_name, :host => "localhost") 37 | s.lock do 38 | # We're now in a mutex protected area 39 | # No matter how many processes are running this program, 40 | # there will be only one running this code block at a time. 41 | work 42 | end 43 | ``` 44 | 45 | While our application is inside the code block given to ```s.lock```, other calls to use the mutex with the same name will block until our code block is finished. Once our mutex unlocks, the next process will unblock and be able to execute the code block. The blocking processes get unblocked in order of arrival, creating a fair queue. 46 | 47 | You can also allow a set number of processes inside the semaphore-protected block, in case you have a well-defined number of resources available: 48 | 49 | ```ruby 50 | s = Redis::Semaphore.new(:semaphore_name, :resources => 5, :host => "localhost") 51 | s.lock do 52 | # Up to five processes at a time will be able to get inside this code 53 | # block simultaneously. 54 | work 55 | end 56 | ``` 57 | 58 | You're not obligated to use code blocks, linear calls work just fine: 59 | 60 | ```ruby 61 | s = Redis::Semaphore.new(:semaphore_name, :host => "localhost") 62 | s.lock 63 | work 64 | s.unlock # Don't forget this, or the mutex will stay locked! 65 | ``` 66 | 67 | If you don't want to wait forever until the semaphore releases, you can pass in a timeout of seconds you want to wait: 68 | 69 | ```ruby 70 | if s.lock(5) # This will only block for at most 5 seconds if the semaphore stays locked. 71 | work 72 | s.unlock 73 | else 74 | puts "Aborted." 75 | end 76 | ``` 77 | 78 | You can check if the mutex or semaphore already exists, or how many resources are left in the semaphore: 79 | 80 | ```ruby 81 | puts "This semaphore does exist." if s.exists? 82 | puts "There are #{s.available_count} resources available right now." 83 | ``` 84 | 85 | When calling ```unlock```, the new number of available resources is returned: 86 | 87 | ```ruby 88 | sem.lock 89 | sem.unlock # returns 1 90 | sem.available_count # also returns 1 91 | ``` 92 | 93 | In the constructor you can pass in any arguments that you would pass to a regular Redis constructor. You can even pass in your custom Redis client: 94 | 95 | ```ruby 96 | r = Redis.new(:host => "localhost", :db => 222) 97 | s = Redis::Semaphore.new(:another_name, :redis => r) 98 | #... 99 | ``` 100 | 101 | Note that it's [a bad idea to reuse the same redis client across threads](https://github.com/dv/redis-semaphore/issues/18), due to the blocking nature of the `blpop` command. We might add support for this in a future version. 102 | 103 | If an exception happens during a lock, the lock will automatically be released: 104 | 105 | ```ruby 106 | begin 107 | s.lock do 108 | raise Exception 109 | end 110 | rescue 111 | s.locked? # false 112 | end 113 | ``` 114 | 115 | 116 | Staleness 117 | --------- 118 | 119 | To allow for clients to die, and the token returned to the list, a stale-check was added. As soon as a lock is started, the time of the lock is set. If another process detects that the timeout has passed since the lock was set, it can force unlock the lock itself. 120 | 121 | There are two ways to take advantage of this. You can either define a :stale\_client\_timeout upon initialization. This will check for stale locks everytime your program wants to lock the semaphore: 122 | 123 | ```ruby 124 | s = Redis::Semaphore.new(:stale_semaphore, :redis = r, :stale_client_timeout => 5) # in seconds 125 | ``` 126 | 127 | Or you could start a different thread or program that frequently checks for stale locks. This has the advantage of unblocking blocking calls to Semaphore#lock as well: 128 | 129 | ```ruby 130 | normal_sem = Redis::Semaphore.new(:semaphore, :host => "localhost") 131 | 132 | Thread.new do 133 | watchdog = Redis::Semaphore.new(:semaphore, :host => "localhost", :stale_client_timeout => 5) 134 | 135 | while(true) do 136 | watchdog.release_stale_locks! 137 | sleep 1 138 | end 139 | end 140 | 141 | normal_sem.lock 142 | sleep 5 143 | normal_sem.locked? # returns false 144 | 145 | normal_sem.lock 146 | normal_sem.lock(5) # will block until the watchdog releases the previous lock after 1 second 147 | ``` 148 | 149 | 150 | Advanced 151 | -------- 152 | 153 | ### Wait and Signal 154 | 155 | The methods ```wait``` and ```signal```, the traditional method names of a Semaphore, are also implemented. ```wait``` is aliased to lock, while ```signal``` puts the specified token back on the semaphore, or generates a unique new token and puts that back if none is passed: 156 | 157 | ```ruby 158 | # Retrieve 2 resources 159 | token1 = sem.wait 160 | token2 = sem.wait 161 | 162 | work 163 | 164 | # Put 3 resources back 165 | sem.signal(token1) 166 | sem.signal(token2) 167 | sem.signal 168 | 169 | sem.available_count # returns 3 170 | ``` 171 | 172 | This can be used to create a semaphore where the process that consumes resources, and the process that generates resources, are not the same. An example is a dynamic queue system with a consumer process and a producer process: 173 | 174 | ```ruby 175 | # Consumer process 176 | job = semaphore.wait 177 | 178 | # Producer process 179 | semaphore.signal(new_job) # Job can be any string, it will be passed unmodified to the consumer process 180 | ``` 181 | 182 | Used in this fashion, a timeout does not make sense. Using the :stale\_client\_timeout here is not recommended. 183 | 184 | 185 | ### Use local time 186 | 187 | When calculating the timeouts, redis-semaphore uses the Redis TIME command by default, which fetches the time on the Redis server. This is good if you're running distributed semaphores to keep all clients on the same clock, but does incur an extra round-trip for every action that requires the time. 188 | 189 | You can add the option ```:use_local_time => true``` during initialization to use the local time of the client instead of the Redis server time, which saves one extra roundtrip. This is good if e.g. you're only running one client. 190 | 191 | ```ruby 192 | s = Redis::Semaphore.new(:local_semaphore, :redis = r, :stale_client_timeout => 5, :use_local_time => true) 193 | ``` 194 | 195 | Redis servers earlier than version 2.6 don't support the TIME command. In that case we fall back to using the local time automatically. 196 | 197 | 198 | ### Expiration 199 | 200 | ```redis-semaphore``` supports an expiration option, which will call the **EXPIRE** Redis command on all related keys (except for `grabbed_keys`), to make sure that after a while all evidence of the semaphore will disappear and your Redis server will not be cluttered with unused keys. Pass in the expiration timeout in seconds: 201 | 202 | ```ruby 203 | s = Redis::Semaphore.new(:local_semaphore, :redis = r, :expiration => 100) 204 | ``` 205 | 206 | This option should only be used if you know what you're doing. If you chose a wrong expiration timeout then the semaphore might disappear in the middle of a critical section. For most situations just using the `delete!` command should suffice to remove all semaphore keys from the server after you're done using the semaphore. 207 | 208 | Installation 209 | ------------ 210 | 211 | $ gem install redis-semaphore 212 | 213 | Testing 214 | ------- 215 | 216 | $ bundle install 217 | $ rake 218 | 219 | Changelog 220 | --------- 221 | 222 | ###0.3.1 April 17, 2016 223 | - Fix `sem.lock(0)` bug (thanks eugenk!). 224 | - Fix `release_stale_locks!` deadlock bug (thanks mfischer-zd for the bug-report!). 225 | 226 | ###0.3.0 January 24, 2016 227 | - Change API to include non-blocking option for `#lock` (thanks tomclose!). 228 | - Fix unwanted persisting of `available_key` (thanks dany1468!). 229 | - Fix `available_count` returning 0 for nonexisting semaphores (thanks mikeryz!). 230 | 231 | ###0.2.4 January 11, 2015 232 | - Fix bug with TIME and redis-namespace (thanks sos4nt!). 233 | - Add expiration option (thanks jcalvert!). 234 | - Update API version logic. 235 | 236 | More in [CHANGELOG](CHANGELOG.md). 237 | 238 | Contributors 239 | ------------ 240 | 241 | Thanks to these awesome people for their contributions: 242 | 243 | - [Rimas Silkaitis](https://github.com/neovintage) 244 | - [Tim Galeckas](https://github.com/timgaleckas) 245 | - [Ruurd Pels](https://github.com/ruurd) 246 | - [Prathan Thananart](https://github.com/scomma) 247 | - [dubdromic](https://github.com/dubdromic) 248 | - [Dmitriy Kiriyenko](https://github.com/dmitriy-kiriyenko) 249 | - [presskey](https://github.com/presskey) 250 | - [Stephen Bussey](https://github.com/sb8244) 251 | - [frobcode](https://github.com/frobcode) 252 | - [Petteri Räty](https://github.com/betelgeuse) 253 | - [Stefan Schüßler](https://github.com/sos4nt) 254 | - [Jonathan Calvert](https://github.com/jcalvert) 255 | - [mikeryz](https://github.com/mikeryz) 256 | - [tomclose](https://github.com/tomclose) 257 | - [Eugen Kuksa](https://github.com/eugenk) 258 | - [Eugene Kenny](https://github.com/eugeneius) 259 | 260 | ### "Merge"-button clicker 261 | 262 | [David Verhasselt](http://davidverhasselt.com) - david@crowdway.com 263 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | task :default => :spec 2 | task :test => :spec 3 | 4 | desc "Run specs" 5 | task :spec do 6 | exec "rspec spec/semaphore_spec.rb" 7 | end 8 | -------------------------------------------------------------------------------- /lib/redis-semaphore.rb: -------------------------------------------------------------------------------- 1 | require 'redis/semaphore' -------------------------------------------------------------------------------- /lib/redis/semaphore.rb: -------------------------------------------------------------------------------- 1 | require 'redis' 2 | 3 | class Redis 4 | class Semaphore 5 | EXISTS_TOKEN = "1" 6 | API_VERSION = "1" 7 | 8 | # stale_client_timeout is the threshold of time before we assume 9 | # that something has gone terribly wrong with a client and we 10 | # invalidate it's lock. 11 | # Default is nil for which we don't check for stale clients 12 | # Redis::Semaphore.new(:my_semaphore, :stale_client_timeout => 30, :redis => myRedis) 13 | # Redis::Semaphore.new(:my_semaphore, :redis => myRedis) 14 | # Redis::Semaphore.new(:my_semaphore, :resources => 1, :redis => myRedis) 15 | # Redis::Semaphore.new(:my_semaphore, :host => "", :port => "") 16 | # Redis::Semaphore.new(:my_semaphore, :path => "bla") 17 | def initialize(name, opts = {}) 18 | @name = name 19 | @expiration = opts.delete(:expiration) 20 | @resource_count = opts.delete(:resources) || 1 21 | @stale_client_timeout = opts.delete(:stale_client_timeout) 22 | @redis = opts.delete(:redis) || Redis.new(opts) 23 | @use_local_time = opts.delete(:use_local_time) 24 | @tokens = [] 25 | end 26 | 27 | def exists_or_create! 28 | token = @redis.getset(exists_key, EXISTS_TOKEN) 29 | 30 | if token.nil? 31 | create! 32 | else 33 | # Previous versions of redis-semaphore did not set `version_key`. 34 | # Make sure it's set now, so we can use it in future versions. 35 | 36 | if token == API_VERSION && @redis.get(version_key).nil? 37 | @redis.set(version_key, API_VERSION) 38 | end 39 | 40 | true 41 | end 42 | end 43 | 44 | def available_count 45 | if exists? 46 | @redis.llen(available_key) 47 | else 48 | @resource_count 49 | end 50 | end 51 | 52 | def delete! 53 | @redis.del(available_key) 54 | @redis.del(grabbed_key) 55 | @redis.del(exists_key) 56 | @redis.del(version_key) 57 | end 58 | 59 | def lock(timeout = nil) 60 | exists_or_create! 61 | release_stale_locks! if check_staleness? 62 | 63 | if timeout.nil? || timeout > 0 64 | # passing timeout 0 to blpop causes it to block 65 | _key, current_token = @redis.blpop(available_key, timeout || 0) 66 | else 67 | current_token = @redis.lpop(available_key) 68 | end 69 | 70 | return false if current_token.nil? 71 | 72 | @tokens.push(current_token) 73 | @redis.hset(grabbed_key, current_token, current_time.to_f) 74 | return_value = current_token 75 | 76 | if block_given? 77 | begin 78 | return_value = yield current_token 79 | ensure 80 | signal(current_token) 81 | end 82 | end 83 | 84 | return_value 85 | end 86 | alias_method :wait, :lock 87 | 88 | def unlock 89 | return false unless locked? 90 | signal(@tokens.pop)[1] 91 | end 92 | 93 | def locked?(token = nil) 94 | if token 95 | @redis.hexists(grabbed_key, token) 96 | else 97 | @tokens.each do |token| 98 | return true if locked?(token) 99 | end 100 | 101 | false 102 | end 103 | end 104 | 105 | def signal(token = 1) 106 | token ||= generate_unique_token 107 | 108 | @redis.multi do 109 | @redis.hdel grabbed_key, token 110 | @redis.lpush available_key, token 111 | 112 | set_expiration_if_necessary 113 | end 114 | end 115 | 116 | def exists? 117 | @redis.exists(exists_key) 118 | end 119 | 120 | def all_tokens 121 | @redis.multi do 122 | @redis.lrange(available_key, 0, -1) 123 | @redis.hkeys(grabbed_key) 124 | end.flatten 125 | end 126 | 127 | def generate_unique_token 128 | tokens = all_tokens 129 | token = Random.rand.to_s 130 | 131 | while(tokens.include? token) 132 | token = Random.rand.to_s 133 | end 134 | end 135 | 136 | def release_stale_locks! 137 | simple_expiring_mutex(:release_locks, 10) do 138 | @redis.hgetall(grabbed_key).each do |token, locked_at| 139 | timed_out_at = locked_at.to_f + @stale_client_timeout 140 | 141 | if timed_out_at < current_time.to_f 142 | signal(token) 143 | end 144 | end 145 | end 146 | end 147 | 148 | private 149 | 150 | def simple_expiring_mutex(key_name, expires_in) 151 | # Using the locking mechanism as described in 152 | # http://redis.io/commands/setnx 153 | 154 | key_name = namespaced_key(key_name) 155 | cached_current_time = current_time.to_f 156 | my_lock_expires_at = cached_current_time + expires_in + 1 157 | 158 | got_lock = @redis.setnx(key_name, my_lock_expires_at) 159 | 160 | if !got_lock 161 | # Check if expired 162 | other_lock_expires_at = @redis.get(key_name).to_f 163 | 164 | if other_lock_expires_at < cached_current_time 165 | old_expires_at = @redis.getset(key_name, my_lock_expires_at).to_f 166 | 167 | # Check if another client started cleanup yet. If not, 168 | # then we now have the lock. 169 | got_lock = (old_expires_at == other_lock_expires_at) 170 | end 171 | end 172 | 173 | return false if !got_lock 174 | 175 | begin 176 | yield 177 | ensure 178 | # Make sure not to delete the lock in case someone else already expired 179 | # our lock, with one second in between to account for some lag. 180 | @redis.del(key_name) if my_lock_expires_at > (current_time.to_f - 1) 181 | end 182 | end 183 | 184 | def create! 185 | @redis.expire(exists_key, 10) 186 | 187 | @redis.multi do 188 | @redis.del(grabbed_key) 189 | @redis.del(available_key) 190 | @resource_count.times do |index| 191 | @redis.rpush(available_key, index) 192 | end 193 | @redis.set(version_key, API_VERSION) 194 | @redis.persist(exists_key) 195 | 196 | set_expiration_if_necessary 197 | end 198 | end 199 | 200 | def set_expiration_if_necessary 201 | if @expiration 202 | [available_key, exists_key, version_key].each do |key| 203 | @redis.expire(key, @expiration) 204 | end 205 | end 206 | end 207 | 208 | def check_staleness? 209 | !@stale_client_timeout.nil? 210 | end 211 | 212 | def redis_namespace? 213 | (defined?(Redis::Namespace) && @redis.is_a?(Redis::Namespace)) 214 | end 215 | 216 | def namespaced_key(variable) 217 | if redis_namespace? 218 | "#{@name}:#{variable}" 219 | else 220 | "SEMAPHORE:#{@name}:#{variable}" 221 | end 222 | end 223 | 224 | def available_key 225 | @available_key ||= namespaced_key('AVAILABLE') 226 | end 227 | 228 | def exists_key 229 | @exists_key ||= namespaced_key('EXISTS') 230 | end 231 | 232 | def grabbed_key 233 | @grabbed_key ||= namespaced_key('GRABBED') 234 | end 235 | 236 | def version_key 237 | @version_key ||= namespaced_key('VERSION') 238 | end 239 | 240 | def current_time 241 | if @use_local_time 242 | Time.now 243 | else 244 | begin 245 | instant = redis_namespace? ? @redis.redis.time : @redis.time 246 | Time.at(instant[0], instant[1]) 247 | rescue 248 | @use_local_time = true 249 | current_time 250 | end 251 | end 252 | end 253 | end 254 | end 255 | -------------------------------------------------------------------------------- /redis-semaphore.gemspec: -------------------------------------------------------------------------------- 1 | Gem::Specification.new do |s| 2 | s.name = 'redis-semaphore' 3 | s.version = '0.3.1' 4 | s.summary = 'Implements a distributed semaphore or mutex using Redis.' 5 | s.authors = ['David Verhasselt'] 6 | s.email = 'david@crowdway.com' 7 | s.homepage = 'http://github.com/dv/redis-semaphore' 8 | s.license = 'MIT' 9 | 10 | files = %w(README.md Rakefile LICENSE) 11 | files += Dir.glob("lib/**/*") 12 | files += Dir.glob("spec/**/*") 13 | s.files = files 14 | 15 | s.add_dependency 'redis' 16 | s.add_development_dependency 'rake', '< 11' 17 | s.add_development_dependency 'rspec', '>= 2.14' 18 | s.add_development_dependency 'timecop' 19 | s.add_development_dependency 'pry' 20 | 21 | s.description = < 15 7 | end 8 | 9 | before(:each) do 10 | @redis.flushdb 11 | end 12 | 13 | after(:all) do 14 | @redis.quit 15 | end 16 | 17 | shared_examples_for "a semaphore" do 18 | 19 | it "has the correct amount of available resources" do 20 | semaphore.lock 21 | expect(semaphore.unlock).to eq(1) 22 | expect(semaphore.available_count).to eq(1) 23 | end 24 | 25 | it "has the correct amount of available resources before locking" do 26 | expect(semaphore.available_count).to eq(1) 27 | end 28 | 29 | it "should not exist from the start" do 30 | expect(semaphore.exists?).to eq(false) 31 | semaphore.lock 32 | expect(semaphore.exists?).to eq(true) 33 | end 34 | 35 | it "should be unlocked from the start" do 36 | expect(semaphore.locked?).to eq(false) 37 | end 38 | 39 | it "should lock and unlock" do 40 | semaphore.lock(1) 41 | expect(semaphore.locked?).to eq(true) 42 | semaphore.unlock 43 | expect(semaphore.locked?).to eq(false) 44 | end 45 | 46 | it "should not lock twice as a mutex" do 47 | expect(semaphore.lock(1)).not_to eq(false) 48 | expect(semaphore.lock(1)).to eq(false) 49 | end 50 | 51 | it "should not lock three times when only two available" do 52 | expect(multisem.lock(1)).not_to eq(false) 53 | expect(multisem.lock(1)).not_to eq(false) 54 | expect(multisem.lock(1)).to eq(false) 55 | end 56 | 57 | it "should always have the correct lock-status" do 58 | multisem.lock(1) 59 | multisem.lock(1) 60 | 61 | expect(multisem.locked?).to eq(true) 62 | multisem.unlock 63 | expect(multisem.locked?).to eq(true) 64 | multisem.unlock 65 | expect(multisem.locked?).to eq(false) 66 | end 67 | 68 | it "should get all different tokens when saturating" do 69 | ids = [] 70 | 2.times do 71 | ids << multisem.lock(1) 72 | end 73 | 74 | expect(ids).to eq(%w(0 1)) 75 | end 76 | 77 | it "should execute the given code block" do 78 | code_executed = false 79 | semaphore.lock(1) do 80 | code_executed = true 81 | end 82 | expect(code_executed).to eq(true) 83 | end 84 | 85 | it "should pass an exception right through" do 86 | expect { 87 | semaphore.lock(1) do 88 | raise Exception, "redis semaphore exception" 89 | end 90 | }.to raise_error(Exception, "redis semaphore exception") 91 | end 92 | 93 | it "should not leave the semaphore locked after raising an exception" do 94 | expect { 95 | semaphore.lock(1) do 96 | raise Exception, "redis semaphore exception" 97 | end 98 | }.to raise_error(Exception, "redis semaphore exception") 99 | 100 | expect(semaphore.locked?).to eq(false) 101 | end 102 | 103 | it "should return the value of the block if block-style locking is used" do 104 | block_value = semaphore.lock(1) do 105 | 42 106 | end 107 | expect(block_value).to eq(42) 108 | end 109 | 110 | it "can return the passed in token to replicate old behaviour" do 111 | lock_token = semaphore.lock(1) 112 | semaphore.unlock() 113 | 114 | block_value = semaphore.lock(1) do |token| 115 | token 116 | end 117 | expect(block_value).to eq(lock_token) 118 | end 119 | 120 | it "should disappear without a trace when calling `delete!`" do 121 | original_key_size = @redis.keys.count 122 | 123 | semaphore.exists_or_create! 124 | semaphore.delete! 125 | 126 | expect(@redis.keys.count).to eq(original_key_size) 127 | end 128 | 129 | it "should not block when the timeout is zero" do 130 | did_we_get_in = false 131 | 132 | semaphore.lock do 133 | semaphore.lock(0) do 134 | did_we_get_in = true 135 | end 136 | end 137 | 138 | expect(did_we_get_in).to be false 139 | end 140 | 141 | it "should be locked when the timeout is zero" do 142 | semaphore.lock(0) do 143 | expect(semaphore.locked?).to be true 144 | end 145 | end 146 | end 147 | 148 | describe "semaphore with expiration" do 149 | let(:semaphore) { Redis::Semaphore.new(:my_semaphore, :redis => @redis, :expiration => 2) } 150 | let(:multisem) { Redis::Semaphore.new(:my_semaphore_2, :resources => 2, :redis => @redis, :expiration => 2) } 151 | 152 | it_behaves_like "a semaphore" 153 | 154 | it "expires keys" do 155 | original_key_size = @redis.keys.count 156 | semaphore.exists_or_create! 157 | sleep 3.0 158 | expect(@redis.keys.count).to eq(original_key_size) 159 | end 160 | 161 | it "expires keys after unlocking" do 162 | original_key_size = @redis.keys.count 163 | semaphore.lock do 164 | # noop 165 | end 166 | sleep 3.0 167 | expect(@redis.keys.count).to eq(original_key_size) 168 | end 169 | end 170 | 171 | describe "semaphore without staleness checking" do 172 | let(:semaphore) { Redis::Semaphore.new(:my_semaphore, :redis => @redis) } 173 | let(:multisem) { Redis::Semaphore.new(:my_semaphore_2, :resources => 2, :redis => @redis) } 174 | 175 | it_behaves_like "a semaphore" 176 | 177 | it "can dynamically add resources" do 178 | semaphore.exists_or_create! 179 | 180 | 3.times do 181 | semaphore.signal 182 | end 183 | 184 | expect(semaphore.available_count).to eq(4) 185 | 186 | semaphore.wait(1) 187 | semaphore.wait(1) 188 | semaphore.wait(1) 189 | 190 | expect(semaphore.available_count).to eq(1) 191 | end 192 | 193 | it "can have stale locks released by a third process" do 194 | watchdog = Redis::Semaphore.new(:my_semaphore, :redis => @redis, :stale_client_timeout => 1) 195 | semaphore.lock 196 | 197 | sleep 0.5 198 | 199 | watchdog.release_stale_locks! 200 | expect(semaphore.locked?).to eq(true) 201 | 202 | sleep 0.6 203 | 204 | watchdog.release_stale_locks! 205 | expect(semaphore.locked?).to eq(false) 206 | end 207 | end 208 | 209 | describe "semaphore with staleness checking" do 210 | let(:semaphore) { Redis::Semaphore.new(:my_semaphore, :redis => @redis, :stale_client_timeout => 5) } 211 | let(:multisem) { Redis::Semaphore.new(:my_semaphore_2, :resources => 2, :redis => @redis, :stale_client_timeout => 5) } 212 | 213 | it_behaves_like "a semaphore" 214 | 215 | it "should restore resources of stale clients" do 216 | hyper_aggressive_sem = Redis::Semaphore.new(:hyper_aggressive_sem, :resources => 1, :redis => @redis, :stale_client_timeout => 1) 217 | 218 | expect(hyper_aggressive_sem.lock(1)).not_to eq(false) 219 | expect(hyper_aggressive_sem.lock(1)).to eq(false) 220 | expect(hyper_aggressive_sem.lock(1)).not_to eq(false) 221 | end 222 | end 223 | 224 | describe "redis time" do 225 | let(:semaphore) { Redis::Semaphore.new(:my_semaphore, :redis => @redis, :stale_client_timeout => 5) } 226 | 227 | before(:all) do 228 | Timecop.freeze(Time.local(1990)) 229 | end 230 | 231 | it "with time support should return a different time than frozen time" do 232 | expect(semaphore.send(:current_time)).not_to eq(Time.now) 233 | end 234 | 235 | it "with use_local_time should return the same time as frozen time" do 236 | semaphore = Redis::Semaphore.new(:my_semaphore, :redis => @redis, :stale_client_timeout => 5, :use_local_time => true) 237 | expect(semaphore.send(:current_time)).to eq(Time.now) 238 | end 239 | 240 | it "without time support should return the same time as frozen time" do 241 | expect(@redis).to receive(:time).and_raise(Redis::CommandError) 242 | expect(semaphore.send(:current_time)).to eq(Time.now) 243 | end 244 | end 245 | 246 | describe "all_tokens" do 247 | let(:semaphore) { Redis::Semaphore.new(:my_semaphore, :redis => @redis, :stale_client_timeout => 5) } 248 | 249 | it "includes tokens from available and grabbed keys" do 250 | semaphore.exists_or_create! 251 | available_keys = semaphore.all_tokens 252 | semaphore.lock(1) 253 | grabbed_keys = semaphore.all_tokens 254 | 255 | expect(available_keys).to eq(grabbed_keys) 256 | end 257 | end 258 | 259 | describe "version" do 260 | context "with an existing versionless semaphore" do 261 | let(:old_sem) { Redis::Semaphore.new(:my_semaphore, :redis => @redis) } 262 | let(:semaphore) { Redis::Semaphore.new(:my_semaphore, :redis => @redis) } 263 | let(:version_key) { old_sem.send(:version_key) } 264 | 265 | before do 266 | old_sem.exists_or_create! 267 | @redis.del(version_key) 268 | end 269 | 270 | it "sets the version key" do 271 | semaphore.exists_or_create! 272 | expect(@redis.get(version_key)).not_to be_nil 273 | end 274 | end 275 | end 276 | 277 | # Private method tests, do not use 278 | describe "simple_expiring_mutex" do 279 | let(:semaphore) { Redis::Semaphore.new(:my_semaphore, :redis => @redis) } 280 | 281 | before do 282 | semaphore.class.send(:public, :simple_expiring_mutex) 283 | end 284 | 285 | it "gracefully expires stale lock" do 286 | expiration = 1 287 | 288 | thread = 289 | Thread.new do 290 | semaphore.simple_expiring_mutex(:test, expiration) do 291 | sleep 3 292 | end 293 | end 294 | 295 | sleep 1.5 296 | 297 | expect(semaphore.simple_expiring_mutex(:test, expiration)).to be_falsy 298 | 299 | sleep expiration 300 | 301 | it_worked = false 302 | semaphore.simple_expiring_mutex(:test, expiration) do 303 | it_worked = true 304 | end 305 | 306 | expect(it_worked).to be_truthy 307 | thread.join 308 | end 309 | end 310 | end 311 | -------------------------------------------------------------------------------- /spec/spec_helper.rb: -------------------------------------------------------------------------------- 1 | require 'rubygems' 2 | require 'bundler/setup' 3 | Bundler.require(:development) 4 | 5 | $TESTING=true 6 | $:.unshift File.join(File.dirname(__FILE__), '..', 'lib') 7 | require 'redis/semaphore' 8 | 9 | RSpec.configure do |c| 10 | c.filter_run focus: true 11 | c.run_all_when_everything_filtered = true 12 | end 13 | --------------------------------------------------------------------------------