├── .gitignore ├── .travis.yml ├── CHANGELOG.md ├── MIT-LICENSE ├── README.rdoc ├── Rakefile ├── VERSION ├── lib ├── active_record │ └── connection_adapters │ │ └── seamless_database_pool_adapter.rb ├── seamless_database_pool.rb └── seamless_database_pool │ ├── arel_compiler.rb │ ├── connection_statistics.rb │ ├── controller_filter.rb │ └── railtie.rb ├── seamless_database_pool.gemspec └── spec ├── connection_adapters_spec.rb ├── connection_statistics_spec.rb ├── controller_filter_spec.rb ├── database.yml ├── seamless_database_pool_adapter_spec.rb ├── seamless_database_pool_spec.rb ├── spec_helper.rb ├── test_adapter └── active_record │ └── connection_adapters │ └── read_only_adapter.rb └── test_model.rb /.gitignore: -------------------------------------------------------------------------------- 1 | pkg 2 | tmp -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | rvm: 2 | - 1.8.7 3 | - 1.9.2 4 | - rbx 5 | - jruby 6 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | ## 1.0.20 2 | 3 | * Remove calls to `alias_method_chain` for Rails 5.1 compatibility (thanks yukideluxe & wjordan) 4 | 5 | * Don't check read only connections on calls to `verify!` and `active?` when not necessary (thanks wjordan) 6 | 7 | ## 1.0.19 8 | 9 | * Require ruby 2.0 or greater 10 | 11 | * Eliminate deprecation warning on Rails 5 (thanks wjordan) 12 | 13 | ## 1.0.18 14 | 15 | * ActiveRecord 5.0 compatibility (thanks jkowens) 16 | 17 | * End support for ActiveRecord 3.1 18 | 19 | ## 1.0.17 20 | 21 | * Do not update the HTTP session if there are no changes. 22 | 23 | ## 1.0.16 24 | 25 | * Use shorter to_s output for output on connection.inspect. 26 | 27 | ## 1.0.15 28 | 29 | * Implement less wordy connection to string method so logs don't fill up with long messages on connection errors. 30 | 31 | * Update specs to remove deprecation warnings 32 | 33 | * Fix adapter specs to work with ActiveRecord 4.1 configuration changes 34 | 35 | ## 1.0.14 36 | 37 | * Remove custom connection timeout logic; Use the underlying driver's timeouts instead. 38 | 39 | * Fix to work with query cache. 40 | 41 | * Make driver less aggressive about overriding methods to proxy to the master connection. 42 | 43 | * End support for ActiveRecord 2.x 44 | 45 | * Add support for ActiveRecord 4.0 46 | 47 | ## 1.0.13 48 | 49 | * Fix to work with `rake db:*` Rails tasks by removing the adapter from the configuration when db:* tasks are run. 50 | 51 | * Fix connection pool issues so checkout/checkins don't interact with the underlying connections (thanks afex) 52 | 53 | * Ruby 2.0/Rails 4.0 compatibility (thanks t27duck) 54 | 55 | ## 1.0.12 56 | 57 | * Remove excessively long log messages on reconnect attempts. 58 | 59 | ## 1.0.11 60 | 61 | * Remove debug code that prevented recovering from errors. 62 | 63 | ## 1.0.10 64 | 65 | * Compatibility with ActiveRecord 3.1.0 66 | 67 | ## 1.0.9 68 | 69 | * Compatibility with bind variables. 70 | 71 | ## 1.0.8 72 | 73 | * Compatibility with ActiveRecord 3.1.0rc4 74 | 75 | ## 1.0.7 76 | 77 | * Make compatible with ActionController 3.0 78 | 79 | * Improved handling of down slave instances. 80 | 81 | ## 1.0.6 82 | 83 | * Make compatible with ActiveRecord 3.0. 84 | 85 | * Make compatible with database adapters other than MySQL including PostgrSQL. 86 | 87 | * Better test suite to actually hit three different database adapters. 88 | 89 | ## 1.0.5 90 | 91 | * Update docs. 92 | 93 | * Remove rake dependency on rspec 94 | -------------------------------------------------------------------------------- /MIT-LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2008-2017 Brian Durand 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining 4 | a copy of this software and associated documentation files (the 5 | "Software"), to deal in the Software without restriction, including 6 | without limitation the rights to use, copy, modify, merge, publish, 7 | distribute, sublicense, and/or sell copies of the Software, and to 8 | permit persons to whom the Software is furnished to do so, subject to 9 | the following conditions: 10 | 11 | The above copyright notice and this permission notice shall be 12 | included in all copies or substantial portions of the Software. 13 | 14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 15 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 16 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 17 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 18 | LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 19 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 20 | WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 21 | -------------------------------------------------------------------------------- /README.rdoc: -------------------------------------------------------------------------------- 1 | Seamless Database Pool provides a simple way in which to add support for a master/slave database cluster to ActiveRecord to allow massive scalability and automatic failover. The guiding design principle behind this code is to make it absolutely trivial to add to an existing, complex application. That way when you have a big, nasty application which needs to scale the database you won't have to stop all feature development just to refactor your database connection code. Let's face it, when the database is having scaling problems, you are in for a world of hurt and the faster you can fix the problem the better. 2 | 3 | This code is available as both a Rails plugin and a gem so it will work with any ActiveRecord application. 4 | 5 | = Database Clusters 6 | 7 | In a master/slave cluster you have one master database server which uses replication to feed all changes to one or more slave databases which are set up to only handle reads. Since most applications put most of the load on the server with reads, this setup can scale out an application quite well. You'll need to work with your database of choice to get replication set up. This plugin has an connection adapter which will handle proxying database requests to the right server. 8 | 9 | = Simple Integration 10 | 11 | You can convert a standard Rails application (i.e. one that follows the scaffold conventions) to use a database cluster with three simple steps: 12 | 13 | 1. Set up the database cluster (OK maybe this one isn't simple) 14 | 2. Update database.yml settings to point to the servers in the cluster 15 | 3. Add this code to ApplicationController: 16 | 17 | include SeamlessDatabasePool::ControllerFilter 18 | use_database_pool :all => :persistent, [:create, :update, :destroy] => :master 19 | 20 | If needed you can control how the connection pool is utilized by wrapping your code in some simple blocks. 21 | 22 | = Failover 23 | 24 | One of the other main advantages of using any sort of cluster is that one node can fail without bringing down your application. This plugin automatically handles failing over dead database connections in the read pool. That is if it tries to use a read connection and it is found to be inactive, the connector will try to reconnect. If that fails, it will try another connection in the read pool. After thirty seconds it will try to reconnect the dead connection again. 25 | 26 | One limitation on failover is when database servers are down when the pool is being initialized during startup. In this case, the connections cannot be initialized and are not added to the pool. If this happens, you will need to restart your processes once the database servers are back online. 27 | 28 | = Configuration 29 | 30 | == The pool configuration 31 | 32 | The cluster connections are configured in database.yml using the seamless_database_pool adapter. Any properties you configure for the connection will be inherited by all connections in the pool. In this way, you can configure ports, usernames, etc. once instead of for each connection. One exception is that you can set the pool_adapter property which each connection will inherit as the adapter property. Each connection in the pool uses all the same configuration properties as normal for the adapters. 33 | 34 | == The read pool 35 | 36 | The read pool is specified with a read_pool property in the pool connection definition in database.yml. This should be specified as an array of hashes where each hash is the configuration for each read connection you'd like to use (see below for an example). As noted above, the configuration for the entire pool will be merged in with the options for each connection. 37 | 38 | Each connection can be assigned an additional option of pool_weight. This value should be number which indicates the relative weight that the connection should be given in the pool. If no value is specified, it will default to one. Setting the value to zero will keep the connection out of the pool. 39 | 40 | If possible, you should set the permissions on the database user for the read connections to one that only has select permission. This can be especially useful in development and testing to ensure that the read connection never have writes sent to them. 41 | 42 | == The master connection 43 | 44 | The master connection is specified with a master_connection property in the pool connection definition in database.yml (see below for an example). The master connection will be used for all non-select statements against the database (i.e. insert, update, delete, etc.). It will also be used for all statements inside a transaction or any reload commands. 45 | 46 | By default, the master connection will be included in the read pool. If you would like to dedicate this connection only for write operations, you should set the pool weight to zero. Do not duplicate the master connection in the read pool as this will result in the additional overhead of two connections to the database. 47 | 48 | == Example configuration 49 | 50 | development: 51 | adapter: seamless_database_pool 52 | database: mydb_development 53 | username: read_user 54 | password: abc123 55 | pool_adapter: mysql2 56 | prepared_statements: false # required for ActiveRecord 5 57 | port: 3306 58 | master: 59 | host: master-db.example.com 60 | port: 6000 61 | username: master_user 62 | password: 567pass 63 | read_pool: 64 | - host: read-db-1.example.com 65 | pool_weight: 2 66 | - host: read-db-2.example.com 67 | 68 | In this configuration, the master connection will be a mysql connection to master-db.example.com:6000 using the username master_user and the password 567pass. 69 | 70 | The read pool will use three mysql connections to master-db, read-db-1, and read-db-2. The master connection will use a different port, username, password for the connection. The read connections will use the same values. Further, the connection read-db-1 will get twice as many read requests as each of the other two connections, so presumably it's on a more powerful box. 71 | 72 | You must use compatible database adapters for both the master and the read connections. For example, you cannot use an Oracle server as your master and PostgreSQL servers as you read slaves. 73 | 74 | = Using the read pool 75 | 76 | By default, the master connection will be used for everything. This is not terribly useful, so you should really specify a method of using the read pool for the actions that need it. Read connections will only be used for select statements against the database. 77 | 78 | This is done with static methods on SeamlessDatabasePool. 79 | 80 | = Controller Filters 81 | 82 | To ease integration into a Ruby on Rails application, several controller filters are provided to invoke the above connection methods in a block. These are not implemented as standard controller filters so that the connection methods can be in effect for other filters. 83 | 84 | See SeamlessDatabasePool::ControllerFilter for more details. 85 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require 'rubygems' 2 | require 'rake' 3 | require 'yaml' 4 | require "bundler/gem_tasks" 5 | 6 | desc 'Default: run unit tests.' 7 | task :default => :test 8 | 9 | begin 10 | require 'rspec' 11 | require 'rspec/core/rake_task' 12 | desc 'Run the unit tests' 13 | RSpec::Core::RakeTask.new(:test) 14 | 15 | namespace :test do 16 | desc "Run all tests including for all database adapters" 17 | task :all do 18 | save_val = ENV['TEST_ADAPTERS'] 19 | begin 20 | ENV['TEST_ADAPTERS'] = YAML.load_file(File.expand_path("../spec/database.yml", __FILE__)).keys.join(' ') 21 | Rake::Task["test"].execute 22 | ensure 23 | ENV['TEST_ADAPTERS'] = save_val 24 | end 25 | end 26 | 27 | desc "Test all database adapters defined in database.yml or just the one specified in TEST_ADAPTERS" 28 | task :adapters do 29 | save_val = ENV['TEST_ADAPTERS'] 30 | begin 31 | ENV['TEST_ADAPTERS'] ||= YAML.load_file(File.expand_path("../spec/database.yml", __FILE__)).keys.join(' ') 32 | Rake::Task["test:adapters:specified"].execute 33 | ensure 34 | ENV['TEST_ADAPTERS'] = save_val 35 | end 36 | end 37 | 38 | namespace :adapters do 39 | desc "Internal task to run database adapter tests" 40 | RSpec::Core::RakeTask.new(:specified) do |t| 41 | t.pattern = FileList.new('spec/connection_adapters_spec.rb') 42 | end 43 | 44 | YAML.load_file(File.expand_path("../spec/database.yml", __FILE__)).keys.each do |adapter_name| 45 | desc "Test the #{adapter_name} database adapter" 46 | task adapter_name do 47 | save_val = ENV['TEST_ADAPTERS'] 48 | begin 49 | ENV['TEST_ADAPTERS'] = adapter_name 50 | Rake::Task["test:adapters:specified"].execute 51 | ensure 52 | ENV['TEST_ADAPTERS'] = save_val 53 | end 54 | end 55 | end 56 | end 57 | end 58 | rescue LoadError 59 | task :test do 60 | STDERR.puts "You must have rspec >= 2.0 to run the tests" 61 | end 62 | end 63 | -------------------------------------------------------------------------------- /VERSION: -------------------------------------------------------------------------------- 1 | 1.0.20 -------------------------------------------------------------------------------- /lib/active_record/connection_adapters/seamless_database_pool_adapter.rb: -------------------------------------------------------------------------------- 1 | module ActiveRecord 2 | class Base 3 | class << self 4 | def seamless_database_pool_connection(config) 5 | pool_weights = {} 6 | 7 | config = config.with_indifferent_access 8 | default_config = {:pool_weight => 1}.merge(config.merge(:adapter => config[:pool_adapter])).with_indifferent_access 9 | default_config.delete(:master) 10 | default_config.delete(:read_pool) 11 | default_config.delete(:pool_adapter) 12 | 13 | master_config = default_config.merge(config[:master]).with_indifferent_access 14 | establish_adapter(master_config[:adapter]) 15 | master_connection = send("#{master_config[:adapter]}_connection".to_sym, master_config) 16 | pool_weights[master_connection] = master_config[:pool_weight].to_i if master_config[:pool_weight].to_i > 0 17 | 18 | read_connections = [] 19 | config[:read_pool].each do |read_config| 20 | read_config = default_config.merge(read_config).with_indifferent_access 21 | read_config[:pool_weight] = read_config[:pool_weight].to_i 22 | if read_config[:pool_weight] > 0 23 | begin 24 | establish_adapter(read_config[:adapter]) 25 | conn = send("#{read_config[:adapter]}_connection".to_sym, read_config) 26 | read_connections << conn 27 | pool_weights[conn] = read_config[:pool_weight] 28 | rescue Exception => e 29 | if logger 30 | logger.error("Error connecting to read connection #{read_config.inspect}") 31 | logger.error(e) 32 | end 33 | end 34 | end 35 | end if config[:read_pool] 36 | 37 | klass = ::ActiveRecord::ConnectionAdapters::SeamlessDatabasePoolAdapter.adapter_class(master_connection) 38 | klass.new(nil, logger, master_connection, read_connections, pool_weights, config) 39 | end 40 | 41 | def establish_adapter(adapter) 42 | raise AdapterNotSpecified.new("database configuration does not specify adapter") unless adapter 43 | raise AdapterNotFound.new("database pool must specify adapters") if adapter == 'seamless_database_pool' 44 | 45 | begin 46 | require 'rubygems' 47 | gem "activerecord-#{adapter}-adapter" 48 | require "active_record/connection_adapters/#{adapter}_adapter" 49 | rescue LoadError 50 | begin 51 | require "active_record/connection_adapters/#{adapter}_adapter" 52 | rescue LoadError 53 | raise LoadError.new("Please install the #{adapter} adapter: `gem install activerecord-#{adapter}-adapter` (#{$!})") 54 | end 55 | end 56 | 57 | adapter_method = "#{adapter}_connection" 58 | if !respond_to?(adapter_method) 59 | raise AdapterNotFound, "database configuration specifies nonexistent #{adapter} adapter" 60 | end 61 | end 62 | end 63 | 64 | module SeamlessDatabasePoolBehavior 65 | 66 | # Force reload to use the master connection since it's probably being called for a reason. 67 | def reload(*args) 68 | SeamlessDatabasePool.use_master_connection do 69 | super *args 70 | end 71 | end 72 | end 73 | 74 | prepend SeamlessDatabasePoolBehavior 75 | end 76 | 77 | module ConnectionAdapters 78 | class SeamlessDatabasePoolAdapter < AbstractAdapter 79 | 80 | attr_reader :read_connections, :master_connection 81 | 82 | class << self 83 | # Create an anonymous class that extends this one and proxies methods to the pool connections. 84 | def adapter_class(master_connection) 85 | adapter_class_name = master_connection.adapter_name.classify 86 | return const_get(adapter_class_name) if const_defined?(adapter_class_name, false) 87 | 88 | # Define methods to proxy to the appropriate pool 89 | read_only_methods = [:select, :select_rows, :execute, :tables, :columns] 90 | clear_cache_methods = [:insert, :update, :delete] 91 | 92 | # Get a list of all methods redefined by the underlying adapter. These will be 93 | # proxied to the master connection. 94 | master_methods = [] 95 | override_classes = (master_connection.class.ancestors - AbstractAdapter.ancestors) 96 | override_classes.each do |connection_class| 97 | master_methods.concat(connection_class.public_instance_methods(false)) 98 | master_methods.concat(connection_class.protected_instance_methods(false)) 99 | master_methods.concat(connection_class.private_instance_methods(false)) 100 | end 101 | master_methods = master_methods.collect{|m| m.to_sym}.uniq 102 | master_methods -= public_instance_methods(false) + protected_instance_methods(false) + private_instance_methods(false) 103 | master_methods -= read_only_methods 104 | master_methods -= [:select_all, :select_one, :select_value, :select_values] 105 | master_methods -= clear_cache_methods 106 | 107 | klass = Class.new(self) 108 | master_methods.each do |method_name| 109 | klass.class_eval <<-EOS, __FILE__, __LINE__ + 1 110 | def #{method_name}(*args, &block) 111 | use_master_connection do 112 | return proxy_connection_method(master_connection, :#{method_name}, :master, *args, &block) 113 | end 114 | end 115 | EOS 116 | end 117 | 118 | clear_cache_methods.each do |method_name| 119 | klass.class_eval <<-EOS, __FILE__, __LINE__ + 1 120 | def #{method_name}(*args, &block) 121 | clear_query_cache if query_cache_enabled 122 | use_master_connection do 123 | return proxy_connection_method(master_connection, :#{method_name}, :master, *args, &block) 124 | end 125 | end 126 | EOS 127 | end 128 | 129 | read_only_methods.each do |method_name| 130 | klass.class_eval <<-EOS, __FILE__, __LINE__ + 1 131 | def #{method_name}(*args, &block) 132 | connection = @use_master ? master_connection : current_read_connection 133 | proxy_connection_method(connection, :#{method_name}, :read, *args, &block) 134 | end 135 | EOS 136 | end 137 | klass.send :protected, :select 138 | 139 | const_set(adapter_class_name, klass) 140 | 141 | return klass 142 | end 143 | 144 | # Set the arel visitor on the connections. 145 | def visitor_for(pool) 146 | # This is ugly, but then again, so is the code in ActiveRecord for setting the arel 147 | # visitor. There is a note in the code indicating the method signatures should be updated. 148 | config = pool.spec.config.with_indifferent_access 149 | adapter = config[:master][:adapter] || config[:pool_adapter] 150 | SeamlessDatabasePool.adapter_class_for(adapter).visitor_for(pool) 151 | end 152 | end 153 | 154 | def initialize(connection, logger, master_connection, read_connections, pool_weights, config) 155 | @master_connection = master_connection 156 | @read_connections = read_connections.dup.freeze 157 | 158 | super(connection, logger, config) 159 | 160 | @weighted_read_connections = [] 161 | pool_weights.each_pair do |conn, weight| 162 | weight.times{@weighted_read_connections << conn} 163 | end 164 | @available_read_connections = [AvailableConnections.new(@weighted_read_connections)] 165 | end 166 | 167 | def adapter_name #:nodoc: 168 | 'Seamless_Database_Pool' 169 | end 170 | 171 | # Returns an array of the master connection and the read pool connections 172 | def all_connections 173 | [@master_connection] + @read_connections 174 | end 175 | 176 | # Get the pool weight of a connection 177 | def pool_weight(connection) 178 | return @weighted_read_connections.select{|conn| conn == connection}.size 179 | end 180 | 181 | def requires_reloading? 182 | false 183 | end 184 | 185 | def transaction(options = {}) 186 | use_master_connection do 187 | super 188 | end 189 | end 190 | 191 | def visitor=(visitor) 192 | all_connections.each{|conn| conn.visitor = visitor} 193 | end 194 | 195 | def visitor 196 | master_connection.visitor 197 | end 198 | 199 | def active? 200 | if SeamlessDatabasePool.read_only_connection_type == :master 201 | @master_connection.active? 202 | else 203 | active = true 204 | do_to_connections {|conn| active &= conn.active?} 205 | active 206 | end 207 | end 208 | 209 | def reconnect! 210 | do_to_connections {|conn| conn.reconnect!} 211 | end 212 | 213 | def disconnect! 214 | do_to_connections {|conn| conn.disconnect!} 215 | end 216 | 217 | def reset! 218 | do_to_connections {|conn| conn.reset!} 219 | end 220 | 221 | def verify!(*ignored) 222 | if SeamlessDatabasePool.read_only_connection_type == :master 223 | @master_connection.verify!(*ignored) 224 | else 225 | do_to_connections {|conn| conn.verify!(*ignored)} 226 | end 227 | end 228 | 229 | def reset_runtime 230 | total = 0.0 231 | do_to_connections {|conn| total += conn.reset_runtime} 232 | total 233 | end 234 | 235 | # Get a random read connection from the pool. If the connection is not active, it will attempt to reconnect 236 | # to the database. If that fails, it will be removed from the pool for one minute. 237 | def random_read_connection 238 | weighted_read_connections = available_read_connections 239 | if @use_master || weighted_read_connections.empty? 240 | return master_connection 241 | else 242 | weighted_read_connections[rand(weighted_read_connections.length)] 243 | end 244 | end 245 | 246 | # Get the current read connection 247 | def current_read_connection 248 | return SeamlessDatabasePool.read_only_connection(self) 249 | end 250 | 251 | def using_master_connection? 252 | !!@use_master 253 | end 254 | 255 | # Force using the master connection in a block. 256 | def use_master_connection 257 | save_val = @use_master 258 | begin 259 | @use_master = true 260 | yield if block_given? 261 | ensure 262 | @use_master = save_val 263 | end 264 | end 265 | 266 | def to_s 267 | "#<#{self.class.name}:0x#{object_id.to_s(16)} #{all_connections.size} connections>" 268 | end 269 | 270 | def inspect 271 | to_s 272 | end 273 | 274 | class DatabaseConnectionError < StandardError 275 | end 276 | 277 | # This simple class puts an expire time on an array of connections. It is used so the a connection 278 | # to a down database won't try to reconnect over and over. 279 | class AvailableConnections 280 | attr_reader :connections, :failed_connection 281 | attr_writer :expires 282 | 283 | def initialize(connections, failed_connection = nil, expires = nil) 284 | @connections = connections 285 | @failed_connection = failed_connection 286 | @expires = expires 287 | end 288 | 289 | def expired? 290 | @expires ? @expires <= Time.now : false 291 | end 292 | 293 | def reconnect! 294 | failed_connection.reconnect! 295 | raise DatabaseConnectionError.new unless failed_connection.active? 296 | end 297 | end 298 | 299 | # Get the available weighted connections. When a connection is dead and cannot be reconnected, it will 300 | # be temporarily removed from the read pool so we don't keep trying to reconnect to a database that isn't 301 | # listening. 302 | def available_read_connections 303 | available = @available_read_connections.last 304 | if available.expired? 305 | begin 306 | @logger.info("Adding dead database connection back to the pool") if @logger 307 | available.reconnect! 308 | rescue => e 309 | # Couldn't reconnect so try again in a little bit 310 | if @logger 311 | @logger.warn("Failed to reconnect to database when adding connection back to the pool") 312 | @logger.warn(e) 313 | end 314 | available.expires = 30.seconds.from_now 315 | return available.connections 316 | end 317 | 318 | # If reconnect is successful, the connection will have been re-added to @available_read_connections list, 319 | # so let's pop this old version of the connection 320 | @available_read_connections.pop 321 | 322 | # Now we'll try again after either expiring our bad connection or re-adding our good one 323 | return available_read_connections 324 | else 325 | return available.connections 326 | end 327 | end 328 | 329 | def reset_available_read_connections 330 | @available_read_connections.slice!(1, @available_read_connections.length) 331 | @available_read_connections.first.connections.each do |connection| 332 | unless connection.active? 333 | connection.reconnect! rescue nil 334 | end 335 | end 336 | end 337 | 338 | # Temporarily remove a connection from the read pool. 339 | def suppress_read_connection(conn, expire) 340 | available = available_read_connections 341 | connections = available.reject{|c| c == conn} 342 | 343 | # This wasn't a read connection so don't suppress it 344 | return if connections.length == available.length 345 | 346 | if connections.empty? 347 | @logger.warn("All read connections are marked dead; trying them all again.") if @logger 348 | # No connections available so we might as well try them all again 349 | reset_available_read_connections 350 | else 351 | @logger.warn("Removing #{conn.inspect} from the connection pool for #{expire} seconds") if @logger 352 | # Available connections will now not include the suppressed connection for a while 353 | @available_read_connections.push(AvailableConnections.new(connections, conn, expire.seconds.from_now)) 354 | end 355 | end 356 | 357 | private 358 | 359 | def proxy_connection_method(connection, method, proxy_type, *args, &block) 360 | begin 361 | connection.send(method, *args, &block) 362 | rescue => e 363 | # If the statement was a read statement and it wasn't forced against the master connection 364 | # try to reconnect if the connection is dead and then re-run the statement. 365 | if proxy_type == :read && !using_master_connection? 366 | unless connection.active? 367 | suppress_read_connection(connection, 30) 368 | connection = current_read_connection 369 | SeamlessDatabasePool.set_persistent_read_connection(self, connection) 370 | end 371 | proxy_connection_method(connection, method, :retry, *args, &block) 372 | else 373 | raise e 374 | end 375 | end 376 | end 377 | 378 | # Yield a block to each connection in the pool. If the connection is dead, ignore the error 379 | # unless it is the master connection 380 | def do_to_connections 381 | all_connections.each do |conn| 382 | begin 383 | yield(conn) 384 | rescue => e 385 | raise e if conn == master_connection 386 | end 387 | end 388 | nil 389 | end 390 | end 391 | end 392 | end 393 | -------------------------------------------------------------------------------- /lib/seamless_database_pool.rb: -------------------------------------------------------------------------------- 1 | require File.join(File.dirname(__FILE__), 'seamless_database_pool', 'connection_statistics.rb') 2 | require File.join(File.dirname(__FILE__), 'seamless_database_pool', 'controller_filter.rb') 3 | require File.join(File.dirname(__FILE__), 'active_record', 'connection_adapters', 'seamless_database_pool_adapter.rb') 4 | require File.join(File.dirname(__FILE__), 'seamless_database_pool', 'railtie.rb') if defined?(Rails::Railtie) 5 | 6 | $LOAD_PATH << File.dirname(__FILE__) unless $LOAD_PATH.include?(File.dirname(__FILE__)) 7 | 8 | # This module allows setting the read pool connection type. Generally you will use one of 9 | # 10 | # - use_random_read_connection 11 | # - use_persistent_read_connection 12 | # - use_master_connection 13 | # 14 | # Each of these methods can take an optional block. If they are called with a block, they 15 | # will set the read connection type only within the block. Otherwise they will set the default 16 | # read connection type. If none is ever called, the read connection type will be :master. 17 | 18 | module SeamlessDatabasePool 19 | 20 | # Adapter name to class name map. This exists because there isn't an obvious way to translate things like 21 | # sqlite3 to SQLite3. The adapters that ship with ActiveRecord are defined here. If you use 22 | # an adapter that doesn't translate directly to camel case, then add the mapping here in an initializer. 23 | ADAPTER_TO_CLASS_NAME_MAP = {"sqlite" => "SQLite", "sqlite3" => "SQLite3", "postgresql" => "PostgreSQL"} 24 | 25 | READ_CONNECTION_METHODS = [:master, :persistent, :random] 26 | 27 | class << self 28 | # Call this method to use a random connection from the read pool for every select statement. 29 | # This method is good if your replication is very fast. Otherwise there is a chance you could 30 | # get inconsistent results from one request to the next. This can result in mysterious failures 31 | # if your code selects a value in one statement and then uses in another statement. You can wind 32 | # up trying to use a value from one server that hasn't been replicated to another one yet. 33 | # This method is best if you have few processes which generate a lot of queries and you have 34 | # fast replication. 35 | def use_random_read_connection 36 | if block_given? 37 | set_read_only_connection_type(:random){yield} 38 | else 39 | Thread.current[:read_only_connection] = :random 40 | end 41 | end 42 | 43 | # Call this method to pick a random connection from the read pool and use it for all subsequent 44 | # select statements. This provides consistency from one select statement to the next. This 45 | # method should always be called with a block otherwise you can end up with an imbalanced read 46 | # pool. This method is best if you have lots of processes which have a relatively few select 47 | # statements or a slow replication mechanism. Generally this is the best method to use for web 48 | # applications. 49 | def use_persistent_read_connection 50 | if block_given? 51 | set_read_only_connection_type(:persistent){yield} 52 | else 53 | Thread.current[:read_only_connection] = {} 54 | end 55 | end 56 | 57 | # Call this method to use the master connection for all subsequent select statements. This 58 | # method is most useful when you are doing lots of updates since it guarantees consistency 59 | # if you do a select immediately after an update or insert. 60 | # 61 | # The master connection will also be used for selects inside any transaction blocks. It will 62 | # also be used if you pass :readonly => false to any ActiveRecord.find method. 63 | def use_master_connection 64 | if block_given? 65 | set_read_only_connection_type(:master){yield} 66 | else 67 | Thread.current[:read_only_connection] = :master 68 | end 69 | end 70 | 71 | # Set the read only connection type to either :master, :random, or :persistent. 72 | def set_read_only_connection_type(connection_type) 73 | saved_connection = Thread.current[:read_only_connection] 74 | retval = nil 75 | begin 76 | connection_type = {} if connection_type == :persistent 77 | Thread.current[:read_only_connection] = connection_type 78 | retval = yield if block_given? 79 | ensure 80 | Thread.current[:read_only_connection] = saved_connection 81 | end 82 | return retval 83 | end 84 | 85 | # Get the read only connection type currently in use. Will be one of :master, :random, or :persistent. 86 | def read_only_connection_type(default = :master) 87 | connection_type = Thread.current[:read_only_connection] || default 88 | connection_type = :persistent if connection_type.kind_of?(Hash) 89 | return connection_type 90 | end 91 | 92 | # Get a read only connection from a connection pool. 93 | def read_only_connection(pool_connection) 94 | return pool_connection.master_connection if pool_connection.using_master_connection? 95 | connection_type = Thread.current[:read_only_connection] 96 | 97 | if connection_type.kind_of?(Hash) 98 | connection = connection_type[pool_connection] 99 | unless connection 100 | connection = pool_connection.random_read_connection 101 | connection_type[pool_connection] = connection 102 | end 103 | return connection 104 | elsif connection_type == :random 105 | return pool_connection.random_read_connection 106 | else 107 | return pool_connection.master_connection 108 | end 109 | end 110 | 111 | # This method is provided as a way to change the persistent connection when it fails and a new one is substituted. 112 | def set_persistent_read_connection(pool_connection, read_connection) 113 | connection_type = Thread.current[:read_only_connection] 114 | connection_type[pool_connection] = read_connection if connection_type.kind_of?(Hash) 115 | end 116 | 117 | def clear_read_only_connection 118 | Thread.current[:read_only_connection] = nil 119 | end 120 | 121 | # Get the connection adapter class for an adapter name. The class will be loaded from 122 | # ActiveRecord::ConnectionAdapters::NameAdapter where Name is the camelized version of the name. 123 | # If the adapter class does not fit this pattern (i.e. sqlite3 => SQLite3Adapter), then add 124 | # the mapping to the +ADAPTER_TO_CLASS_NAME_MAP+ Hash. 125 | def adapter_class_for(name) 126 | name = name.to_s 127 | class_name = ADAPTER_TO_CLASS_NAME_MAP[name] || name.camelize 128 | "ActiveRecord::ConnectionAdapters::#{class_name}Adapter".constantize 129 | end 130 | 131 | # Pull out the master configuration for compatibility with such things as the Rails' rake db:* 132 | # tasks which only support known adapters. 133 | def master_database_configuration(database_configs) 134 | configs = {} 135 | database_configs.each do |key, values| 136 | if values['adapter'] == 'seamless_database_pool' 137 | values['adapter'] = values.delete('pool_adapter') 138 | values = values.merge(values['master']) if values['master'].is_a?(Hash) 139 | values.delete('pool_weight') 140 | values.delete('master') 141 | values.delete('read_pool') 142 | end 143 | configs[key] = values 144 | end 145 | configs 146 | end 147 | end 148 | end 149 | -------------------------------------------------------------------------------- /lib/seamless_database_pool/arel_compiler.rb: -------------------------------------------------------------------------------- 1 | module Arel 2 | module SqlCompiler 3 | # Hook into arel to use the compiler used by the master connection. 4 | class Seamless_Database_PoolCompiler < GenericCompiler 5 | def self.new(relation) 6 | @compiler_classes ||= {} 7 | master_adapter = relation.engine.connection.master_connection.adapter_name 8 | compiler_class = @compiler_classes[master_adapter] 9 | unless compiler_class 10 | begin 11 | require "arel/engines/sql/compilers/#{master_adapter.downcase}_compiler" 12 | rescue LoadError 13 | begin 14 | # try to load an externally defined compiler, in case this adapter has defined the compiler on its own. 15 | require "#{master_adapter.downcase}/arel_compiler" 16 | rescue LoadError 17 | raise LoadError.new("#{master_adapter} is not supported by Arel.") 18 | end 19 | end 20 | compiler_class = Arel::SqlCompiler.const_get("#{master_adapter}Compiler") 21 | @compiler_classes[master_adapter] = compiler_class 22 | end 23 | compiler_class.new(relation) 24 | end 25 | end 26 | end 27 | end 28 | -------------------------------------------------------------------------------- /lib/seamless_database_pool/connection_statistics.rb: -------------------------------------------------------------------------------- 1 | module SeamlessDatabasePool 2 | # This module is included for testing. Mix it into each of your database pool connections 3 | # and it will keep track of how often each connection calls update, insert, execute, 4 | # or select. 5 | module ConnectionStatistics 6 | # Get the connection statistics 7 | def connection_statistics 8 | @connection_statistics ||= {} 9 | end 10 | 11 | def reset_connection_statistics 12 | @connection_statistics = {} 13 | end 14 | 15 | def update(sql, name = nil) 16 | increment_connection_statistic(:update) do 17 | super(sql, name) 18 | end 19 | end 20 | 21 | def insert(sql, name = nil) 22 | increment_connection_statistic(:insert) do 23 | super(sql, name) 24 | end 25 | end 26 | 27 | def execute(sql, name = nil) 28 | increment_connection_statistic(:execute) do 29 | super(sql, name) 30 | end 31 | end 32 | 33 | protected 34 | 35 | def select(sql, name = nil, *args) 36 | increment_connection_statistic(:select) do 37 | super(sql, name, *args) 38 | end 39 | end 40 | 41 | def increment_connection_statistic(method) 42 | if @counting_pool_statistics 43 | yield 44 | else 45 | begin 46 | @counting_pool_statistics = true 47 | stat = connection_statistics[method] || 0 48 | @connection_statistics[method] = stat + 1 49 | yield 50 | ensure 51 | @counting_pool_statistics = false 52 | end 53 | end 54 | end 55 | end 56 | end 57 | -------------------------------------------------------------------------------- /lib/seamless_database_pool/controller_filter.rb: -------------------------------------------------------------------------------- 1 | module SeamlessDatabasePool 2 | # This module provides a simple method of declaring which read pool connection type should 3 | # be used for various ActionController actions. To use it, you must first mix it into 4 | # you controller and then call use_database_pool to configure the connection types. Generally 5 | # you should just do this in ApplicationController and call use_database_pool in your controllers 6 | # when you need different connection types. 7 | # 8 | # Example: 9 | # 10 | # ApplicationController < ActionController::Base 11 | # include SeamlessDatabasePool::ControllerFilter 12 | # use_database_pool :all => :persistent, [:save, :delete] => :master 13 | # ... 14 | module ControllerFilter 15 | def self.included(base) 16 | unless base.respond_to?(:use_database_pool) 17 | base.extend(ClassMethods) 18 | base.class_eval do 19 | send(:prepend, ControllerFilterHooks) 20 | end 21 | end 22 | end 23 | 24 | module ClassMethods 25 | def seamless_database_pool_options 26 | return @seamless_database_pool_options if @seamless_database_pool_options 27 | @seamless_database_pool_options = superclass.seamless_database_pool_options.dup if superclass.respond_to?(:seamless_database_pool_options) 28 | @seamless_database_pool_options ||= {} 29 | end 30 | 31 | # Call this method to set up the connection types that will be used for your actions. 32 | # The configuration is given as a hash where the key is the action name and the value is 33 | # the connection type (:master, :persistent, or :random). You can specify :all as the action 34 | # to define a default connection type. You can also specify the action names in an array 35 | # to easily map multiple actions to one connection type. 36 | # 37 | # The configuration is inherited from parent controller classes, so if you have default 38 | # behavior, you should simply specify it in ApplicationController to have it available 39 | # globally. 40 | def use_database_pool(options) 41 | remapped_options = seamless_database_pool_options 42 | options.each_pair do |actions, connection_method| 43 | unless SeamlessDatabasePool::READ_CONNECTION_METHODS.include?(connection_method) 44 | raise "Invalid read pool method: #{connection_method}; should be one of #{SeamlessDatabasePool::READ_CONNECTION_METHODS.inspect}" 45 | end 46 | actions = [actions] unless actions.kind_of?(Array) 47 | actions.each do |action| 48 | remapped_options[action.to_sym] = connection_method 49 | end 50 | end 51 | @seamless_database_pool_options = remapped_options 52 | end 53 | end 54 | 55 | # Force the master connection to be used on the next request. This is very useful for the Post-Redirect pattern 56 | # where you post a request to your save action and then redirect the user back to the edit action. By calling 57 | # this method, you won't have to worry if the replication engine is slower than the redirect. Normally you 58 | # won't need to call this method yourself as it is automatically called when you perform a redirect from within 59 | # a master connection block. It is made available just in case you have special needs that don't quite fit 60 | # into this module's default logic. 61 | def use_master_db_connection_on_next_request 62 | session[:next_request_db_connection] = :master if session 63 | end 64 | 65 | def seamless_database_pool_options 66 | self.class.seamless_database_pool_options 67 | end 68 | 69 | private 70 | 71 | # Set the read only connection for a block. Used to set the connection for a controller action. 72 | def set_read_only_connection_for_block(action) 73 | read_pool_method = nil 74 | if session 75 | read_pool_method = session[:next_request_db_connection] 76 | session.delete(:next_request_db_connection) if session[:next_request_db_connection] 77 | end 78 | 79 | read_pool_method ||= seamless_database_pool_options[action.to_sym] || seamless_database_pool_options[:all] 80 | if read_pool_method 81 | SeamlessDatabasePool.set_read_only_connection_type(read_pool_method) do 82 | yield 83 | end 84 | else 85 | yield 86 | end 87 | end 88 | end 89 | 90 | module ControllerFilterHooks 91 | # Rails 3.x hook for setting the read connection for the request. 92 | def process(action, *args) 93 | set_read_only_connection_for_block(action) do 94 | super(action, *args) 95 | end 96 | end 97 | 98 | # Rails 2.x hook for setting the read connection for the request. 99 | def perform_action(*args) 100 | set_read_only_connection_for_block(action_name) do 101 | super 102 | end 103 | end 104 | 105 | def redirect_to(options = {}, response_status = {}) 106 | if SeamlessDatabasePool.read_only_connection_type(nil) == :master 107 | use_master_db_connection_on_next_request 108 | end 109 | 110 | super(options, response_status) 111 | end 112 | end 113 | end 114 | -------------------------------------------------------------------------------- /lib/seamless_database_pool/railtie.rb: -------------------------------------------------------------------------------- 1 | module SeamlessDatabasePool 2 | class Railtie < ::Rails::Railtie 3 | rake_tasks do 4 | namespace :db do 5 | task :load_config do 6 | # Override seamless_database_pool configuration so db:* rake tasks work as expected. 7 | module DatabaseConfiguration 8 | def configurations 9 | SeamlessDatabasePool.master_database_configuration(super.deep_dup) 10 | end 11 | end 12 | ActiveRecord::Base.singleton_class.prepend(DatabaseConfiguration) 13 | end 14 | end 15 | end 16 | end 17 | end 18 | -------------------------------------------------------------------------------- /seamless_database_pool.gemspec: -------------------------------------------------------------------------------- 1 | Gem::Specification.new do |spec| 2 | spec.name = "seamless_database_pool" 3 | spec.version = File.read(File.expand_path("../VERSION", __FILE__)).chomp 4 | spec.authors = ["Brian Durand"] 5 | spec.email = ["bbdurand@gmail.com"] 6 | spec.description = %q{Add support for master/slave database database clusters in ActiveRecord to improve performance.} 7 | spec.summary = %q{Add support for master/slave database clusters in ActiveRecord to improve performance.} 8 | spec.homepage = "https://github.com/bdurand/seamless_database_pool" 9 | spec.license = "MIT" 10 | 11 | spec.files = `git ls-files`.split($/) 12 | spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) } 13 | spec.test_files = spec.files.grep(%r{^(test|spec|features)/}) 14 | spec.require_paths = ["lib"] 15 | spec.required_ruby_version = '~> 2.0' 16 | 17 | spec.add_runtime_dependency(%q, [">= 3.2.0"]) 18 | spec.add_development_dependency(%q, [">= 2.0"]) 19 | spec.add_development_dependency(%q, [">= 0"]) 20 | spec.add_development_dependency(%q, [">= 0"]) 21 | spec.add_development_dependency(%q, [">= 0"]) 22 | end 23 | -------------------------------------------------------------------------------- /spec/connection_adapters_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | require 'active_record/connection_adapters/read_only_adapter' 3 | 4 | describe "Test connection adapters" do 5 | if SeamlessDatabasePool::TestModel.database_configs.empty? 6 | puts "No adapters specified for testing. Specify the adapters with TEST_ADAPTERS variable" 7 | else 8 | SeamlessDatabasePool::TestModel.database_configs.keys.each do |adapter| 9 | context adapter do 10 | let(:model){ SeamlessDatabasePool::TestModel.db_model(adapter) } 11 | let(:connection){ model.connection } 12 | let(:read_connection){ connection.available_read_connections.first } 13 | let(:master_connection){ connection.master_connection } 14 | 15 | before(:all) do 16 | if ActiveRecord::VERSION::MAJOR < 4 || (ActiveRecord::VERSION::MAJOR == 4 && ActiveRecord::VERSION::MINOR == 0) 17 | ActiveRecord::Base.configurations = {'adapter' => "sqlite3", 'database' => ":memory:"} 18 | else 19 | ActiveRecord::Base.configurations = {"test" => {'adapter' => "sqlite3", 'database' => ":memory:"}} 20 | end 21 | ActiveRecord::Base.establish_connection('adapter' => "sqlite3", 'database' => ":memory:") 22 | ActiveRecord::Base.connection 23 | SeamlessDatabasePool::TestModel.db_model(adapter).create_tables 24 | end 25 | 26 | after(:all) do 27 | SeamlessDatabasePool::TestModel.db_model(adapter).drop_tables 28 | SeamlessDatabasePool::TestModel.db_model(adapter).cleanup_database! 29 | end 30 | 31 | before(:each) do 32 | model.create!(:name => 'test', :value => 1) 33 | SeamlessDatabasePool.use_persistent_read_connection 34 | end 35 | 36 | after(:each) do 37 | model.delete_all 38 | SeamlessDatabasePool.use_master_connection 39 | end 40 | 41 | it "should force the master connection on reload" do 42 | record = model.first 43 | SeamlessDatabasePool.should_not_receive(:current_read_connection) 44 | record.reload 45 | end 46 | 47 | it "should quote table names properly" do 48 | connection.quote_table_name("foo").should == master_connection.quote_table_name("foo") 49 | end 50 | 51 | it "should quote column names properly" do 52 | connection.quote_column_name("foo").should == master_connection.quote_column_name("foo") 53 | end 54 | 55 | it "should quote string properly" do 56 | connection.quote_string("foo").should == master_connection.quote_string("foo") 57 | end 58 | 59 | it "should quote booleans properly" do 60 | connection.quoted_true.should == master_connection.quoted_true 61 | connection.quoted_false.should == master_connection.quoted_false 62 | end 63 | 64 | it "should quote dates properly" do 65 | date = Date.today 66 | time = Time.now 67 | connection.quoted_date(date).should == master_connection.quoted_date(date) 68 | connection.quoted_date(time).should == master_connection.quoted_date(time) 69 | end 70 | 71 | it "should query for records" do 72 | record = model.find_by_name("test") 73 | record.name.should == "test" 74 | end 75 | 76 | it "should work with query caching" do 77 | record_id = model.first.id 78 | model.cache do 79 | found = model.find(record_id) 80 | found.value.should == 1 81 | connection.master_connection.update("UPDATE #{model.table_name} SET value = 0 WHERE id = #{record_id}") 82 | model.find(record_id).value.should == 1 83 | end 84 | end 85 | 86 | it "should work bust the query cache on update" do 87 | record_id = model.first.id 88 | model.cache do 89 | found = model.find(record_id) 90 | found.name = "new value" 91 | found.save! 92 | model.find(record_id).name.should == "new value" 93 | end 94 | end 95 | 96 | context "read connection" do 97 | let(:sample_sql){"SELECT #{connection.quote_column_name('name')} FROM #{connection.quote_table_name(model.table_name)}"} 98 | 99 | it "should not include the master connection in the read pool for these tests" do 100 | connection.available_read_connections.should_not include(master_connection) 101 | connection.current_read_connection.should_not == master_connection 102 | end 103 | 104 | it "should send select to the read connection" do 105 | results = connection.send(:select, sample_sql) 106 | results.to_a.should == [{"name" => "test"}] 107 | results.to_a.should == master_connection.send(:select, sample_sql).to_a 108 | results.should be_read_only 109 | end 110 | 111 | it "should send select_rows to the read connection" do 112 | results = connection.select_rows(sample_sql) 113 | results.should == [["test"]] 114 | results.should == master_connection.select_rows(sample_sql) 115 | results.should be_read_only 116 | end 117 | 118 | it "should send execute to the read connection" do 119 | results = connection.execute(sample_sql) 120 | results.should be_read_only 121 | end 122 | 123 | it "should send columns to the read connection" do 124 | results = connection.columns(model.table_name) 125 | columns = results.collect{|c| c.name}.sort.should 126 | columns.should == ["id", "name", "value"] 127 | columns.should == master_connection.columns(model.table_name).collect{|c| c.name}.sort 128 | results.should be_read_only 129 | end 130 | 131 | it "should send tables to the read connection" do 132 | results = connection.tables 133 | results.should == [model.table_name] 134 | results.should == master_connection.tables 135 | results.should be_read_only 136 | end 137 | 138 | it "should reconnect dead connections in the read pool" do 139 | read_connection.disconnect! 140 | read_connection.should_not be_active 141 | results = connection.select_all(sample_sql) 142 | results.should be_read_only 143 | read_connection.should be_active 144 | end 145 | end 146 | 147 | context "methods not overridden" do 148 | let(:sample_sql){"SELECT #{connection.quote_column_name('name')} FROM #{connection.quote_table_name(model.table_name)}"} 149 | 150 | it "should use select_all" do 151 | results = connection.select_all(sample_sql) 152 | results.to_a.should == [{"name" => "test"}].to_a 153 | results.to_a.should == master_connection.select_all(sample_sql).to_a 154 | end 155 | 156 | it "should use select_one" do 157 | results = connection.select_one(sample_sql) 158 | results.should == {"name" => "test"} 159 | results.should == master_connection.select_one(sample_sql) 160 | end 161 | 162 | it "should use select_values" do 163 | results = connection.select_values(sample_sql) 164 | results.should == ["test"] 165 | results.should == master_connection.select_values(sample_sql) 166 | end 167 | 168 | it "should use select_value" do 169 | results = connection.select_value(sample_sql) 170 | results.should == "test" 171 | results.should == master_connection.select_value(sample_sql) 172 | end 173 | end 174 | 175 | context "master connection" do 176 | let(:insert_sql){ "INSERT INTO #{connection.quote_table_name(model.table_name)} (#{connection.quote_column_name('name')}) VALUES ('new')" } 177 | let(:update_sql){ "UPDATE #{connection.quote_table_name(model.table_name)} SET #{connection.quote_column_name('value')} = 2" } 178 | let(:delete_sql){ "DELETE FROM #{connection.quote_table_name(model.table_name)}" } 179 | 180 | it "should blow up if a master connection method is sent to the read only connection" do 181 | lambda{read_connection.update(update_sql)}.should raise_error(NotImplementedError) 182 | lambda{read_connection.update(insert_sql)}.should raise_error(NotImplementedError) 183 | lambda{read_connection.update(delete_sql)}.should raise_error(NotImplementedError) 184 | lambda{read_connection.transaction{}}.should raise_error(NotImplementedError) 185 | lambda{read_connection.create_table(:test)}.should raise_error(NotImplementedError) 186 | end 187 | 188 | it "should send update to the master connection" do 189 | connection.update(update_sql) 190 | model.first.value.should == 2 191 | end 192 | 193 | it "should send insert to the master connection" do 194 | connection.update(insert_sql) 195 | model.find_by_name("new").should_not == nil 196 | end 197 | 198 | it "should send delete to the master connection" do 199 | connection.update(delete_sql) 200 | model.first.should == nil 201 | end 202 | 203 | it "should send transaction to the master connection" do 204 | connection.transaction do 205 | connection.update(update_sql) 206 | end 207 | model.first.value.should == 2 208 | end 209 | 210 | it "should send schema altering statements to the master connection" do 211 | SeamlessDatabasePool.use_master_connection do 212 | begin 213 | connection.create_table(:foo) do |t| 214 | t.string :name 215 | end 216 | connection.add_index(:foo, :name) 217 | ensure 218 | connection.remove_index(:foo, :name) 219 | connection.drop_table(:foo) 220 | end 221 | end 222 | end 223 | 224 | it "should properly dump the schema" do 225 | with_driver = StringIO.new 226 | ActiveRecord::SchemaDumper.dump(connection, with_driver) 227 | 228 | without_driver = StringIO.new 229 | ActiveRecord::SchemaDumper.dump(master_connection, without_driver) 230 | 231 | with_driver.string.should == without_driver.string 232 | end 233 | 234 | it "should allow for database specific types" do 235 | if adapter == "postgresql" 236 | SeamlessDatabasePool.use_master_connection do 237 | connection.enable_extension "hstore" 238 | connection.create_table(:pg) do |t| 239 | t.hstore :my_hash 240 | end 241 | end 242 | connection.drop_table(:pg) 243 | end 244 | end 245 | 246 | end 247 | end 248 | end 249 | end 250 | end 251 | -------------------------------------------------------------------------------- /spec/connection_statistics_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | describe SeamlessDatabasePool::ConnectionStatistics do 4 | module SeamlessDatabasePool 5 | class ConnectionStatisticsTester 6 | def insert (sql, name = nil) 7 | "INSERT #{sql}/#{name}" 8 | end 9 | 10 | def update (sql, name = nil) 11 | execute(sql) 12 | "UPDATE #{sql}/#{name}" 13 | end 14 | 15 | def execute (sql, name = nil) 16 | "EXECUTE #{sql}/#{name}" 17 | end 18 | 19 | protected 20 | 21 | def select (sql, name = nil, binds = []) 22 | "SELECT #{sql}/#{name}" 23 | end 24 | 25 | prepend ::SeamlessDatabasePool::ConnectionStatistics 26 | end 27 | end 28 | 29 | it "should increment statistics on update" do 30 | connection = SeamlessDatabasePool::ConnectionStatisticsTester.new 31 | connection.update('SQL', 'name').should == "UPDATE SQL/name" 32 | connection.connection_statistics.should == {:update => 1} 33 | connection.update('SQL 2').should == "UPDATE SQL 2/" 34 | connection.connection_statistics.should == {:update => 2} 35 | end 36 | 37 | it "should increment statistics on insert" do 38 | connection = SeamlessDatabasePool::ConnectionStatisticsTester.new 39 | connection.insert('SQL', 'name').should == "INSERT SQL/name" 40 | connection.connection_statistics.should == {:insert => 1} 41 | connection.insert('SQL 2').should == "INSERT SQL 2/" 42 | connection.connection_statistics.should == {:insert => 2} 43 | end 44 | 45 | it "should increment statistics on execute" do 46 | connection = SeamlessDatabasePool::ConnectionStatisticsTester.new 47 | connection.execute('SQL', 'name').should == "EXECUTE SQL/name" 48 | connection.connection_statistics.should == {:execute => 1} 49 | connection.execute('SQL 2').should == "EXECUTE SQL 2/" 50 | connection.connection_statistics.should == {:execute => 2} 51 | end 52 | 53 | it "should increment statistics on select" do 54 | connection = SeamlessDatabasePool::ConnectionStatisticsTester.new 55 | connection.send(:select, 'SQL', 'name').should == "SELECT SQL/name" 56 | connection.connection_statistics.should == {:select => 1} 57 | connection.send(:select, 'SQL 2').should == "SELECT SQL 2/" 58 | connection.connection_statistics.should == {:select => 2} 59 | end 60 | 61 | it "should increment counts only once within a block" do 62 | connection = SeamlessDatabasePool::ConnectionStatisticsTester.new 63 | expect(connection).to receive(:execute).with('SQL') 64 | connection.update('SQL') 65 | connection.connection_statistics.should == {:update => 1} 66 | end 67 | 68 | it "should be able to clear the statistics" do 69 | connection = SeamlessDatabasePool::ConnectionStatisticsTester.new 70 | connection.update('SQL') 71 | connection.connection_statistics.should == {:update => 1} 72 | connection.reset_connection_statistics 73 | connection.connection_statistics.should == {} 74 | end 75 | 76 | end 77 | -------------------------------------------------------------------------------- /spec/controller_filter_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | describe "SeamlessDatabasePool::ControllerFilter" do 4 | 5 | module SeamlessDatabasePool 6 | class TestApplicationController 7 | attr_reader :session 8 | 9 | def initialize(session) 10 | @session = session 11 | end 12 | 13 | def process(action, *args) 14 | send action 15 | end 16 | 17 | def redirect_to (options = {}, response_status = {}) 18 | options 19 | end 20 | 21 | def base_action 22 | ::SeamlessDatabasePool.read_only_connection_type 23 | end 24 | end 25 | 26 | class TestBaseController < TestApplicationController 27 | include ::SeamlessDatabasePool::ControllerFilter 28 | 29 | use_database_pool :read => :persistent 30 | 31 | def read 32 | ::SeamlessDatabasePool.read_only_connection_type 33 | end 34 | 35 | def other 36 | ::SeamlessDatabasePool.read_only_connection_type 37 | end 38 | end 39 | 40 | class TestOtherController < TestBaseController 41 | use_database_pool :all => :random, [:edit, :save, :redirect_master_action] => :master 42 | 43 | def edit 44 | ::SeamlessDatabasePool.read_only_connection_type 45 | end 46 | 47 | def save 48 | ::SeamlessDatabasePool.read_only_connection_type 49 | end 50 | 51 | def redirect_master_action 52 | redirect_to(:action => :read) 53 | end 54 | 55 | def redirect_read_action 56 | redirect_to(:action => :read) 57 | end 58 | end 59 | 60 | class TestRails2ApplicationController < TestApplicationController 61 | attr_reader :action_name 62 | 63 | def process(action, *args) 64 | @action_name = action 65 | perform_action 66 | end 67 | 68 | private 69 | 70 | def perform_action 71 | send action_name 72 | end 73 | end 74 | 75 | class TestRails2BaseController < TestRails2ApplicationController 76 | include ::SeamlessDatabasePool::ControllerFilter 77 | 78 | use_database_pool :read => :persistent 79 | 80 | def read 81 | ::SeamlessDatabasePool.read_only_connection_type 82 | end 83 | end 84 | end 85 | 86 | let(:session){Hash.new} 87 | let(:controller){SeamlessDatabasePool::TestOtherController.new(session)} 88 | 89 | it "should work with nothing set" do 90 | controller = SeamlessDatabasePool::TestApplicationController.new(session) 91 | controller.process('base_action').should == :master 92 | end 93 | 94 | it "should allow setting a connection type for a single action" do 95 | controller = SeamlessDatabasePool::TestBaseController.new(session) 96 | controller.process('read').should == :persistent 97 | end 98 | 99 | it "should allow setting a connection type for actions" do 100 | controller.process('edit').should == :master 101 | controller.process('save').should == :master 102 | end 103 | 104 | it "should allow setting a connection type for all actions" do 105 | controller.process('other').should == :random 106 | end 107 | 108 | it "should inherit the superclass' options" do 109 | controller.process('read').should == :persistent 110 | end 111 | 112 | it "should be able to force using the master connection on the next request" do 113 | # First request 114 | controller.process('read').should == :persistent 115 | controller.use_master_db_connection_on_next_request 116 | 117 | # Second request 118 | controller.process('read').should == :master 119 | 120 | # Third request 121 | controller.process('read').should == :persistent 122 | end 123 | 124 | it "should not break trying to force the master connection if sessions are not enabled" do 125 | controller.process('read').should == :persistent 126 | controller.use_master_db_connection_on_next_request 127 | 128 | # Second request 129 | session.clear 130 | controller.process('read').should == :persistent 131 | end 132 | 133 | it "should force the master connection on the next request for a redirect in master connection block" do 134 | controller = SeamlessDatabasePool::TestOtherController.new(session) 135 | controller.process('redirect_master_action').should == {:action => :read} 136 | 137 | controller.process('read').should == :master 138 | end 139 | 140 | it "should not force the master connection on the next request for a redirect not in master connection block" do 141 | controller.process('redirect_read_action').should == {:action => :read} 142 | 143 | controller.process('read').should == :persistent 144 | end 145 | 146 | it "should work with a Rails 2 controller" do 147 | controller = SeamlessDatabasePool::TestRails2BaseController.new(session) 148 | controller.process('read').should == :persistent 149 | end 150 | end 151 | -------------------------------------------------------------------------------- /spec/database.yml: -------------------------------------------------------------------------------- 1 | # This file contains the databases that the test suite will be run against if you run rake:test:adapters or set the 2 | # environment variable ADAPTER (use commas to test multiple adapters). If you want, you can add your own adapter below 3 | # and it will be added to the test suite. 4 | 5 | sqlite3: 6 | adapter: seamless_database_pool 7 | database: test.sqlite3 8 | prepared_statements: false 9 | master: 10 | adapter: sqlite3 11 | pool_weight: 0 12 | read_pool: 13 | - adapter: read_only 14 | real_adapter: sqlite3 15 | 16 | postgresql: 17 | adapter: seamless_database_pool 18 | database: seamless_database_pool_test 19 | prepared_statements: false 20 | username: postgres 21 | password: postgres 22 | master: 23 | adapter: postgresql 24 | pool_weight: 0 25 | read_pool: 26 | - adapter: read_only 27 | real_adapter: postgresql 28 | 29 | mysql: 30 | adapter: seamless_database_pool 31 | database: seamless_database_pool_test 32 | prepared_statements: false 33 | username: root 34 | password: 35 | master: 36 | adapter: mysql2 37 | pool_weight: 0 38 | read_pool: 39 | - adapter: read_only 40 | real_adapter: mysql2 41 | -------------------------------------------------------------------------------- /spec/seamless_database_pool_adapter_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | module SeamlessDatabasePool 4 | class MockConnection < ActiveRecord::ConnectionAdapters::AbstractAdapter 5 | def initialize (name) 6 | @name = name 7 | end 8 | 9 | def inspect 10 | "#{@name} connection" 11 | end 12 | 13 | def reconnect! 14 | sleep(0.1) 15 | end 16 | 17 | def active? 18 | true 19 | end 20 | 21 | def begin_db_transaction 22 | end 23 | 24 | def commit_db_transaction 25 | end 26 | end 27 | 28 | class MockMasterConnection < MockConnection 29 | def insert (sql, name = nil); end 30 | def update (sql, name = nil); end 31 | def execute (sql, name = nil); end 32 | def columns (table_name, name = nil); end 33 | end 34 | end 35 | 36 | describe "SeamlessDatabasePoolAdapter ActiveRecord::Base extension" do 37 | 38 | it "should establish the connections in the pool merging global options into the connection options" do 39 | options = { 40 | :adapter => 'seamless_database_pool', 41 | :pool_adapter => 'reader', 42 | :username => 'user', 43 | :master => { 44 | 'adapter' => 'writer', 45 | 'host' => 'master_host' 46 | }, 47 | :read_pool => [ 48 | {'host' => 'read_host_1'}, 49 | {'host' => 'read_host_2', 'pool_weight' => '2'}, 50 | {'host' => 'read_host_3', 'pool_weight' => '0'} 51 | ] 52 | } 53 | 54 | pool_connection = double(:connection) 55 | master_connection = SeamlessDatabasePool::MockConnection.new("master") 56 | read_connection_1 = SeamlessDatabasePool::MockConnection.new("read_1") 57 | read_connection_2 = SeamlessDatabasePool::MockConnection.new("read_2") 58 | logger = ActiveRecord::Base.logger 59 | weights = {master_connection => 1, read_connection_1 => 1, read_connection_2 => 2} 60 | 61 | expect(ActiveRecord::Base).to receive(:writer_connection).with('adapter' => 'writer', 'host' => 'master_host', 'username' => 'user', 'pool_weight' => 1).and_return(master_connection) 62 | expect(ActiveRecord::Base).to receive(:reader_connection).with('adapter' => 'reader', 'host' => 'read_host_1', 'username' => 'user', 'pool_weight' => 1).and_return(read_connection_1) 63 | expect(ActiveRecord::Base).to receive(:reader_connection).with('adapter' => 'reader', 'host' => 'read_host_2', 'username' => 'user', 'pool_weight' => 2).and_return(read_connection_2) 64 | 65 | klass = double(:class) 66 | expect(ActiveRecord::ConnectionAdapters::SeamlessDatabasePoolAdapter).to receive(:adapter_class).with(master_connection).and_return(klass) 67 | expect(klass).to receive(:new).with(nil, logger, master_connection, [read_connection_1, read_connection_2], weights, options).and_return(pool_connection) 68 | 69 | expect(ActiveRecord::Base).to receive(:establish_adapter).with('writer') 70 | expect(ActiveRecord::Base).to receive(:establish_adapter).with('reader').twice 71 | 72 | ActiveRecord::Base.seamless_database_pool_connection(options).should == pool_connection 73 | end 74 | 75 | it "should raise an error if the adapter would be recursive" do 76 | lambda{ActiveRecord::Base.seamless_database_pool_connection('seamless_database_pool').should_raise(ActiveRecord::AdapterNotFound)} 77 | end 78 | end 79 | 80 | describe "SeamlessDatabasePoolAdapter" do 81 | 82 | let(:master_connection){ SeamlessDatabasePool::MockMasterConnection.new("master") } 83 | let(:read_connection_1){ SeamlessDatabasePool::MockConnection.new("read_1") } 84 | let(:read_connection_2){ SeamlessDatabasePool::MockConnection.new("read_2") } 85 | let(:config){ {} } 86 | let(:pool_connection) do 87 | weights = {master_connection => 1, read_connection_1 => 1, read_connection_2 => 2} 88 | connection_class = ActiveRecord::ConnectionAdapters::SeamlessDatabasePoolAdapter.adapter_class(master_connection) 89 | connection_class.new(nil, nil, master_connection, [read_connection_1, read_connection_2], weights, config) 90 | end 91 | 92 | it "should be able to be converted to a string" do 93 | pool_connection.to_s.should =~ /\A#\z/ 94 | pool_connection.inspect.should == pool_connection.to_s 95 | end 96 | 97 | context "selecting a connection from the pool" do 98 | it "should initialize the connection pool" do 99 | pool_connection.master_connection.should == master_connection 100 | pool_connection.read_connections.should == [read_connection_1, read_connection_2] 101 | pool_connection.all_connections.should == [master_connection, read_connection_1, read_connection_2] 102 | pool_connection.pool_weight(master_connection).should == 1 103 | pool_connection.pool_weight(read_connection_1).should == 1 104 | pool_connection.pool_weight(read_connection_2).should == 2 105 | end 106 | 107 | it "should return the current read connection" do 108 | expect(SeamlessDatabasePool).to receive(:read_only_connection).with(pool_connection).and_return(:current) 109 | pool_connection.current_read_connection.should == :current 110 | end 111 | 112 | it "should select a random read connection" do 113 | mock_connection = double(:connection) 114 | mock_connection.stub(:active? => true) 115 | expect(pool_connection).to receive(:available_read_connections).and_return([:fake1, :fake2, mock_connection]) 116 | expect(pool_connection).to receive(:rand).with(3).and_return(2) 117 | pool_connection.random_read_connection.should == mock_connection 118 | end 119 | 120 | it "should select the master connection if the read pool is empty" do 121 | expect(pool_connection).to receive(:available_read_connections).and_return([]) 122 | pool_connection.random_read_connection.should == master_connection 123 | end 124 | 125 | it "should use the master connection in a block" do 126 | connection_class = ActiveRecord::ConnectionAdapters::SeamlessDatabasePoolAdapter.adapter_class(master_connection) 127 | connection = connection_class.new(nil, double(:logger), master_connection, [read_connection_1], {read_connection_1 => 1}, config) 128 | connection.random_read_connection.should == read_connection_1 129 | connection.use_master_connection do 130 | connection.random_read_connection.should == master_connection 131 | end 132 | connection.random_read_connection.should == read_connection_1 133 | end 134 | 135 | it "should use the master connection inside a transaction" do 136 | connection_class = ActiveRecord::ConnectionAdapters::SeamlessDatabasePoolAdapter.adapter_class(master_connection) 137 | connection = connection_class.new(nil, double(:logger), master_connection, [read_connection_1], {read_connection_1 => 1}, config) 138 | expect(master_connection).to receive(:begin_db_transaction) 139 | expect(master_connection).to receive(:commit_db_transaction) 140 | expect(master_connection).to receive(:select).with('Transaction SQL', nil) 141 | expect(read_connection_1).to receive(:select).with('SQL 1', nil) 142 | expect(read_connection_1).to receive(:select).with('SQL 2', nil) 143 | 144 | SeamlessDatabasePool.use_persistent_read_connection do 145 | connection.send(:select, 'SQL 1', nil) 146 | connection.transaction do 147 | connection.send(:select, 'Transaction SQL', nil) 148 | end 149 | connection.send(:select, 'SQL 2', nil) 150 | end 151 | end 152 | end 153 | 154 | context "read connection methods" do 155 | it "should proxy select methods to a read connection" do 156 | expect(pool_connection).to receive(:current_read_connection).and_return(read_connection_1) 157 | expect(read_connection_1).to receive(:select).with('SQL').and_return(:retval) 158 | pool_connection.send(:select, 'SQL').should == :retval 159 | end 160 | 161 | it "should proxy execute methods to a read connection" do 162 | expect(pool_connection).to receive(:current_read_connection).and_return(read_connection_1) 163 | expect(read_connection_1).to receive(:execute).with('SQL').and_return(:retval) 164 | pool_connection.execute('SQL').should == :retval 165 | end 166 | 167 | it "should proxy select_rows methods to a read connection" do 168 | expect(pool_connection).to receive(:current_read_connection).and_return(read_connection_1) 169 | expect(read_connection_1).to receive(:select_rows).with('SQL').and_return(:retval) 170 | pool_connection.select_rows('SQL').should == :retval 171 | end 172 | end 173 | 174 | context "master connection methods" do 175 | it "should proxy insert method to the master connection" do 176 | expect(master_connection).to receive(:insert).with('SQL').and_return(:retval) 177 | pool_connection.insert('SQL').should == :retval 178 | end 179 | 180 | it "should proxy update method to the master connection" do 181 | expect(master_connection).to receive(:update).with('SQL').and_return(:retval) 182 | pool_connection.update('SQL').should == :retval 183 | end 184 | 185 | it "should proxy columns method to the master connection" do 186 | expect(master_connection).to receive(:columns).with(:table).and_return(:retval) 187 | pool_connection.columns(:table).should == :retval 188 | end 189 | end 190 | 191 | context "fork to all connections" do 192 | context "when read-only connection type is master" do 193 | it "should fork active? to master connection only" do 194 | expect(master_connection).to receive(:active?).and_return(true) 195 | expect(read_connection_1).not_to receive(:active?) 196 | expect(read_connection_2).not_to receive(:active?) 197 | pool_connection.active?.should == true 198 | end 199 | 200 | it "should fork verify! to master connection only" do 201 | expect(master_connection).to receive(:verify!).with(5) 202 | expect(read_connection_1).not_to receive(:verify!) 203 | expect(read_connection_2).not_to receive(:verify!) 204 | pool_connection.verify!(5) 205 | end 206 | end 207 | 208 | context "When read-only connection type is persistent or random" do 209 | around do |example| 210 | SeamlessDatabasePool.set_read_only_connection_type(:persistent) do 211 | example.run 212 | end 213 | SeamlessDatabasePool.set_read_only_connection_type(:random) do 214 | example.run 215 | end 216 | end 217 | 218 | it "should fork active? to all connections and return true if all are up" do 219 | expect(master_connection).to receive(:active?).and_return(true) 220 | expect(read_connection_1).to receive(:active?).and_return(true) 221 | expect(read_connection_2).to receive(:active?).and_return(true) 222 | pool_connection.active?.should == true 223 | end 224 | 225 | it "should fork active? to all connections and return false if one is down" do 226 | expect(master_connection).to receive(:active?).and_return(true) 227 | expect(read_connection_1).to receive(:active?).and_return(true) 228 | expect(read_connection_2).to receive(:active?).and_return(false) 229 | pool_connection.active?.should == false 230 | end 231 | 232 | it "should fork verify! to all connections" do 233 | expect(master_connection).to receive(:verify!).with(5) 234 | expect(read_connection_1).to receive(:verify!).with(5) 235 | expect(read_connection_2).to receive(:verify!).with(5) 236 | pool_connection.verify!(5) 237 | end 238 | end 239 | 240 | it "should fork disconnect! to all connections" do 241 | expect(master_connection).to receive(:disconnect!) 242 | expect(read_connection_1).to receive(:disconnect!) 243 | expect(read_connection_2).to receive(:disconnect!) 244 | pool_connection.disconnect! 245 | end 246 | 247 | it "should fork reconnect! to all connections" do 248 | expect(master_connection).to receive(:reconnect!) 249 | expect(read_connection_1).to receive(:reconnect!) 250 | expect(read_connection_2).to receive(:reconnect!) 251 | pool_connection.reconnect! 252 | end 253 | 254 | it "should fork reset_runtime to all connections" do 255 | expect(master_connection).to receive(:reset_runtime).and_return(1) 256 | expect(read_connection_1).to receive(:reset_runtime).and_return(2) 257 | expect(read_connection_2).to receive(:reset_runtime).and_return(3) 258 | pool_connection.reset_runtime.should == 6 259 | end 260 | end 261 | 262 | context "reconnection" do 263 | it "should proxy requests to a connection" do 264 | args = [:arg1, :arg2] 265 | block = Proc.new{} 266 | expect(master_connection).to receive(:select_value).with(*args, &block) 267 | master_connection.should_not_receive(:active?) 268 | master_connection.should_not_receive(:reconnect!) 269 | pool_connection.send(:proxy_connection_method, master_connection, :select_value, :master, *args, &block) 270 | end 271 | 272 | it "should try to reconnect dead connections when they become available again" do 273 | master_connection.stub(:select).and_raise("SQL ERROR") # Rails 3, 4 274 | master_connection.stub(:select_rows).and_raise("SQL ERROR") # Rails 5 275 | expect(master_connection).to receive(:active?).and_return(false, false, true) 276 | expect(master_connection).to receive(:reconnect!) 277 | now = Time.now 278 | lambda{pool_connection.select_value("SQL")}.should raise_error("SQL ERROR") 279 | Time.stub(:now => now + 31) 280 | lambda{pool_connection.select_value("SQL")}.should raise_error("SQL ERROR") 281 | end 282 | 283 | it "should not try to reconnect live connections" do 284 | args = [:arg1, :arg2] 285 | block = Proc.new{} 286 | expect(master_connection).to receive(:select).with(*args, &block).twice.and_raise("SQL ERROR") 287 | expect(master_connection).to receive(:active?).and_return(true) 288 | master_connection.should_not_receive(:reconnect!) 289 | lambda{pool_connection.send(:proxy_connection_method, master_connection, :select, :read, *args, &block)}.should raise_error("SQL ERROR") 290 | end 291 | 292 | it "should not try to reconnect a connection during a retry" do 293 | args = [:arg1, :arg2] 294 | block = Proc.new{} 295 | expect(master_connection).to receive(:select).with(*args, &block).and_raise("SQL ERROR") 296 | master_connection.should_not_receive(:active?) 297 | master_connection.should_not_receive(:reconnect!) 298 | lambda{pool_connection.send(:proxy_connection_method, master_connection, :select, :retry, *args, &block)}.should raise_error("SQL ERROR") 299 | end 300 | 301 | it "should try to execute a read statement again after a connection error" do 302 | connection_error = ActiveRecord::ConnectionAdapters::SeamlessDatabasePoolAdapter::DatabaseConnectionError.new 303 | expect(pool_connection).to receive(:current_read_connection).and_return(read_connection_1) 304 | expect(read_connection_1).to receive(:select).with('SQL').and_raise(connection_error) 305 | expect(read_connection_1).to receive(:active?).and_return(true) 306 | pool_connection.should_not_receive(:suppress_read_connection) 307 | SeamlessDatabasePool.should_not_receive(:set_persistent_read_connection) 308 | expect(read_connection_1).to receive(:select).with('SQL').and_return(:results) 309 | pool_connection.send(:select, 'SQL').should == :results 310 | end 311 | 312 | it "should not try to execute a read statement again after a connection error if the master connection must be used" do 313 | expect(master_connection).to receive(:select).with('SQL').and_raise("Fail") 314 | pool_connection.use_master_connection do 315 | lambda{pool_connection.send(:select, 'SQL')}.should raise_error("Fail") 316 | end 317 | end 318 | 319 | it "should not try to execute a read statement again after a non-connection error" do 320 | expect(pool_connection).to receive(:current_read_connection).and_return(read_connection_1) 321 | expect(pool_connection).to receive(:proxy_connection_method).with(read_connection_1, :select, :read, 'SQL').and_raise("SQL Error") 322 | lambda{pool_connection.send(:select, 'SQL')}.should raise_error("SQL Error") 323 | end 324 | 325 | it "should use a different connection on a retry if the original connection could not be reconnected" do 326 | expect(pool_connection).to receive(:current_read_connection).and_return(read_connection_1, read_connection_2) 327 | expect(read_connection_1).to receive(:select).with('SQL').and_raise("Fail") 328 | expect(read_connection_1).to receive(:active?).and_return(false) 329 | expect(pool_connection).to receive(:suppress_read_connection).with(read_connection_1, 30) 330 | expect(SeamlessDatabasePool).to receive(:set_persistent_read_connection).with(pool_connection, read_connection_2) 331 | expect(read_connection_2).to receive(:select).with('SQL').and_return(:results) 332 | pool_connection.send(:select, 'SQL').should == :results 333 | end 334 | 335 | it "should keep track of read connections that can't be reconnected for a set period" do 336 | pool_connection.available_read_connections.should include(read_connection_1) 337 | pool_connection.suppress_read_connection(read_connection_1, 30) 338 | pool_connection.available_read_connections.should_not include(read_connection_1) 339 | end 340 | 341 | it "should return dead connections to the pool after the timeout has expired" do 342 | pool_connection.available_read_connections.should include(read_connection_1) 343 | pool_connection.suppress_read_connection(read_connection_1, 0.2) 344 | pool_connection.available_read_connections.should_not include(read_connection_1) 345 | sleep(0.3) 346 | pool_connection.available_read_connections.should include(read_connection_1) 347 | end 348 | 349 | it "should not return a connection to the pool until it can be reconnected" do 350 | pool_connection.available_read_connections.should include(read_connection_1) 351 | pool_connection.suppress_read_connection(read_connection_1, 0.2) 352 | pool_connection.available_read_connections.should_not include(read_connection_1) 353 | sleep(0.3) 354 | expect(read_connection_1).to receive(:reconnect!) 355 | expect(read_connection_1).to receive(:active?).and_return(false) 356 | pool_connection.available_read_connections.should_not include(read_connection_1) 357 | end 358 | 359 | it "should try all connections again if none of them can be reconnected" do 360 | stack = pool_connection.instance_variable_get(:@available_read_connections) 361 | 362 | available = pool_connection.available_read_connections 363 | available.should include(read_connection_1) 364 | available.should include(read_connection_2) 365 | available.should include(master_connection) 366 | stack.size.should == 1 367 | 368 | pool_connection.suppress_read_connection(read_connection_1, 30) 369 | available = pool_connection.available_read_connections 370 | available.should_not include(read_connection_1) 371 | available.should include(read_connection_2) 372 | available.should include(master_connection) 373 | stack.size.should == 2 374 | 375 | pool_connection.suppress_read_connection(master_connection, 30) 376 | available = pool_connection.available_read_connections 377 | available.should_not include(read_connection_1) 378 | available.should include(read_connection_2) 379 | available.should_not include(master_connection) 380 | stack.size.should == 3 381 | 382 | pool_connection.suppress_read_connection(read_connection_2, 30) 383 | available = pool_connection.available_read_connections 384 | available.should include(read_connection_1) 385 | available.should include(read_connection_2) 386 | available.should include(master_connection) 387 | stack.size.should == 1 388 | end 389 | 390 | it "should not try to suppress a read connection that wasn't available in the read pool" do 391 | stack = pool_connection.instance_variable_get(:@available_read_connections) 392 | stack.size.should == 1 393 | pool_connection.suppress_read_connection(read_connection_1, 30) 394 | stack.size.should == 2 395 | pool_connection.suppress_read_connection(read_connection_1, 30) 396 | stack.size.should == 2 397 | end 398 | end 399 | end 400 | -------------------------------------------------------------------------------- /spec/seamless_database_pool_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | describe "SeamlessDatabasePool" do 4 | 5 | before(:each) do 6 | SeamlessDatabasePool.clear_read_only_connection 7 | end 8 | 9 | after(:each) do 10 | SeamlessDatabasePool.clear_read_only_connection 11 | end 12 | 13 | it "should use the master connection by default" do 14 | connection = double(:connection, :master_connection => :master_db_connection, :using_master_connection? => false) 15 | SeamlessDatabasePool.read_only_connection_type.should == :master 16 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 17 | end 18 | 19 | it "should be able to set using persistent read connections" do 20 | connection = double(:connection) 21 | connection.should_receive(:random_read_connection).once.and_return(:read_db_connection) 22 | connection.stub(:using_master_connection? => false) 23 | SeamlessDatabasePool.use_persistent_read_connection 24 | SeamlessDatabasePool.read_only_connection_type.should == :persistent 25 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 26 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 27 | end 28 | 29 | it "should be able to set using random read connections" do 30 | connection = double(:connection) 31 | connection.should_receive(:random_read_connection).and_return(:read_db_connection_1, :read_db_connection_2) 32 | connection.stub(:using_master_connection? => false) 33 | SeamlessDatabasePool.use_random_read_connection 34 | SeamlessDatabasePool.read_only_connection_type.should == :random 35 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection_1 36 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection_2 37 | end 38 | 39 | it "should use the master connection if the connection is forcing it" do 40 | connection = double(:connection, :master_connection => :master_db_connection) 41 | connection.should_receive(:using_master_connection?).and_return(true) 42 | SeamlessDatabasePool.use_persistent_read_connection 43 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 44 | end 45 | 46 | it "should be able to set using the master connection" do 47 | connection = double(:connection, :master_connection => :master_db_connection) 48 | connection.stub(:using_master_connection? => false) 49 | SeamlessDatabasePool.use_master_connection 50 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 51 | end 52 | 53 | it "should be able to use persistent read connections within a block" do 54 | connection = double(:connection, :master_connection => :master_db_connection) 55 | connection.should_receive(:random_read_connection).once.and_return(:read_db_connection) 56 | connection.stub(:using_master_connection? => false) 57 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 58 | SeamlessDatabasePool.use_persistent_read_connection do 59 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 60 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 61 | :test_val 62 | end.should == :test_val 63 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 64 | end 65 | 66 | it "should be able to use random read connections within a block" do 67 | connection = double(:connection, :master_connection => :master_db_connection) 68 | connection.should_receive(:random_read_connection).and_return(:read_db_connection_1, :read_db_connection_2) 69 | connection.stub(:using_master_connection? => false) 70 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 71 | SeamlessDatabasePool.use_random_read_connection do 72 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection_1 73 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection_2 74 | :test_val 75 | end.should == :test_val 76 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 77 | end 78 | 79 | it "should be able to use the master connection within a block" do 80 | connection = double(:connection, :master_connection => :master_db_connection) 81 | connection.should_receive(:random_read_connection).once.and_return(:read_db_connection) 82 | connection.stub(:using_master_connection? => false) 83 | SeamlessDatabasePool.use_persistent_read_connection 84 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 85 | SeamlessDatabasePool.use_master_connection do 86 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 87 | :test_val 88 | end.should == :test_val 89 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 90 | SeamlessDatabasePool.clear_read_only_connection 91 | end 92 | 93 | it "should be able to use connection blocks within connection blocks" do 94 | connection = double(:connection, :master_connection => :master_db_connection) 95 | connection.stub(:random_read_connection => :read_db_connection) 96 | connection.stub(:using_master_connection? => false) 97 | SeamlessDatabasePool.use_persistent_read_connection do 98 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 99 | SeamlessDatabasePool.use_master_connection do 100 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 101 | SeamlessDatabasePool.use_random_read_connection do 102 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 103 | end 104 | SeamlessDatabasePool.read_only_connection(connection).should == :master_db_connection 105 | end 106 | end 107 | SeamlessDatabasePool.clear_read_only_connection 108 | end 109 | 110 | it "should be able to change the persistent connection" do 111 | connection = double(:connection) 112 | connection.stub(:random_read_connection => :read_db_connection, :using_master_connection? => false) 113 | 114 | SeamlessDatabasePool.use_persistent_read_connection 115 | SeamlessDatabasePool.read_only_connection_type.should == :persistent 116 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 117 | SeamlessDatabasePool.set_persistent_read_connection(connection, :another_db_connection) 118 | SeamlessDatabasePool.read_only_connection(connection).should == :another_db_connection 119 | 120 | SeamlessDatabasePool.use_random_read_connection 121 | SeamlessDatabasePool.read_only_connection_type.should == :random 122 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 123 | SeamlessDatabasePool.set_persistent_read_connection(connection, :another_db_connection) 124 | SeamlessDatabasePool.read_only_connection(connection).should == :read_db_connection 125 | end 126 | 127 | it "should be able to specify a default read connection type instead of :master" do 128 | SeamlessDatabasePool.read_only_connection_type.should == :master 129 | SeamlessDatabasePool.read_only_connection_type(nil).should == nil 130 | end 131 | 132 | it "should pull out the master configurations for compatibility with rake db:* tasks" do 133 | config = { 134 | 'development' => { 135 | 'adapter' => 'seamless_database_pool', 136 | 'pool_adapter' => 'mysql2', 137 | 'database' => 'development', 138 | 'username' => 'root', 139 | 'master' => { 140 | 'host' => 'localhost', 141 | 'pool_weight' => 2 142 | }, 143 | 'read_pool' => { 144 | 'host' => 'slavehost', 145 | 'pool_weight' => 5 146 | } 147 | }, 148 | 'test' => { 149 | 'adapter' => 'mysql2', 150 | 'database' => 'test' 151 | } 152 | } 153 | SeamlessDatabasePool.master_database_configuration(config).should == { 154 | 'development' => { 155 | 'adapter' => 'mysql2', 156 | 'database' => 'development', 157 | 'username' => 'root', 158 | 'host' => 'localhost' 159 | }, 160 | 'test' => { 161 | 'adapter' => 'mysql2', 162 | 'database' => 'test' 163 | } 164 | } 165 | end 166 | end 167 | -------------------------------------------------------------------------------- /spec/spec_helper.rb: -------------------------------------------------------------------------------- 1 | require 'rubygems' 2 | 3 | active_record_version = ENV["ACTIVE_RECORD_VERSION"] || [">= 2.2.2"] 4 | active_record_version = [active_record_version] unless active_record_version.is_a?(Array) 5 | gem 'activerecord', *active_record_version 6 | 7 | require 'active_record' 8 | puts "Testing Against ActiveRecord #{ActiveRecord::VERSION::STRING}" if defined?(ActiveRecord::VERSION) 9 | 10 | require File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib', 'seamless_database_pool')) 11 | require File.expand_path(File.join(File.dirname(__FILE__), 'test_model')) 12 | 13 | $LOAD_PATH << File.expand_path("../test_adapter", __FILE__) 14 | 15 | RSpec.configure do |config| 16 | config.run_all_when_everything_filtered = true 17 | config.filter_run :focus 18 | 19 | # Run specs in random order to surface order dependencies. If you find an 20 | # order dependency and want to debug it, you can fix the order by providing 21 | # the seed, which is printed after each run. 22 | # --seed 1234 23 | config.order = 'random' 24 | config.expect_with(:rspec) { |c| c.syntax = [:should, :expect] } 25 | config.mock_with(:rspec) { |c| c.syntax = [:should, :expect] } 26 | end 27 | -------------------------------------------------------------------------------- /spec/test_adapter/active_record/connection_adapters/read_only_adapter.rb: -------------------------------------------------------------------------------- 1 | module ActiveRecord 2 | class Base 3 | def self.read_only_connection(config) 4 | real_adapter = config.delete("real_adapter") 5 | connection = send("#{real_adapter}_connection", config.merge("adapter" => real_adapter)) 6 | ConnectionAdapters::ReadOnlyAdapter.new(connection) 7 | end 8 | end 9 | 10 | module ConnectionAdapters 11 | class ReadOnlyAdapter < AbstractAdapter 12 | %w(select select_rows execute tables columns).each do |read_method| 13 | class_eval <<-EOS, __FILE__, __LINE__ + 1 14 | def #{read_method} (*args, &block) 15 | raise "Not Connected" unless @connected 16 | result = @connection.send(:#{read_method}, *args, &block) 17 | def result.read_only? 18 | true 19 | end 20 | result 21 | end 22 | EOS 23 | 24 | %w(update insert delete reload create_table drop_table add_index remove_index transaction).each do |write_method| 25 | class_eval <<-EOS, __FILE__, __LINE__ + 1 26 | def #{write_method} (*args, &block) 27 | raise NotImplementedError.new("Master method '#{write_method}' called on read only connection") 28 | end 29 | EOS 30 | end 31 | end 32 | 33 | def initialize(connection) 34 | @connection = connection 35 | @connected = true 36 | super 37 | end 38 | 39 | def test_select 40 | @connection.select_all('SELECT "test_models".* FROM "test_models" LIMIT 1') 41 | end 42 | 43 | def visitor 44 | @connection.visitor 45 | end 46 | 47 | def visitor=(v) 48 | @connection.visitor = v 49 | end 50 | 51 | def reconnect! 52 | @connected = true 53 | end 54 | 55 | def disconnect! 56 | @connected = false 57 | end 58 | 59 | def active? 60 | @connected 61 | end 62 | end 63 | end 64 | end 65 | -------------------------------------------------------------------------------- /spec/test_model.rb: -------------------------------------------------------------------------------- 1 | module SeamlessDatabasePool 2 | class TestModel < ActiveRecord::Base 3 | self.abstract_class = true 4 | 5 | class << self 6 | def database_configs 7 | adapters = ENV['TEST_ADAPTERS'].blank? ? [] : ENV['TEST_ADAPTERS'].split(/\s+/) 8 | configs = {} 9 | YAML.load_file(File.expand_path("../database.yml", __FILE__)).each do |adapter_name, adapter_config| 10 | configs[adapter_name] = adapter_config if adapters.include?(adapter_name.downcase) 11 | end 12 | configs 13 | end 14 | 15 | def use_database_connection(db_name) 16 | establish_connection(database_configs[db_name.to_s]) 17 | end 18 | 19 | def db_model(db_name) 20 | model_class_name = "#{db_name.classify}TestModel" 21 | unless const_defined?(model_class_name) 22 | klass = Class.new(self) 23 | const_set(model_class_name, klass) 24 | klass = const_get(model_class_name) 25 | klass.use_database_connection(db_name) 26 | end 27 | const_get(model_class_name) 28 | end 29 | 30 | def create_tables 31 | connection.create_table(table_name) do |t| 32 | t.column :name, :string 33 | t.column :value, :integer 34 | end unless table_exists? 35 | connection.clear_cache! if connection.respond_to?(:clear_cache!) 36 | undefine_attribute_methods if respond_to?(:undefine_attribute_methods) 37 | end 38 | 39 | def drop_tables 40 | connection.drop_table(table_name) 41 | connection.clear_cache! if connection.respond_to?(:clear_cache!) 42 | undefine_attribute_methods if respond_to?(:undefine_attribute_methods) 43 | end 44 | 45 | def cleanup_database! 46 | connection.disconnect! 47 | sqlite3_config = database_configs['sqlite3'] 48 | if sqlite3_config && File.exist?(sqlite3_config['database']) 49 | File.delete(sqlite3_config['database']) 50 | end 51 | end 52 | end 53 | end 54 | end 55 | --------------------------------------------------------------------------------