├── .gitignore ├── Gemfile ├── LICENSE.txt ├── README.md ├── Rakefile ├── bin ├── console └── setup ├── lib ├── super_fast_rails.rb └── super_fast_rails │ └── version.rb ├── rorvswild_logo.jpg └── super_fast_rails.gemspec /.gitignore: -------------------------------------------------------------------------------- 1 | /.bundle/ 2 | /.yardoc 3 | /_yardoc/ 4 | /coverage/ 5 | /doc/ 6 | /pkg/ 7 | /spec/reports/ 8 | /tmp/ 9 | 10 | *.gem 11 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | source "https://rubygems.org" 4 | 5 | # Specify your gem's dependencies in super_fast_rails.gemspec 6 | gemspec 7 | 8 | gem "rake", "~> 13.0" 9 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2024 Alexis Bernard 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # SuperFastRails 2 | 3 | Most of the time, optimizing a Rails application requires repeating the same techniques. 4 | For example, at the database layer, it's about creating the proper indexes, preventing 1+N queries, etc. 5 | Could we do that automatically? 6 | 7 | --- 8 | 9 | RorVsWild logoMade by RorVsWild, performances & exceptions monitoring for Ruby on Rails applications. 10 | 11 | --- 12 | 13 | ## Introducing SuperFastRails! 14 | 15 | We are releasing a gem to help developers write ultra-optimized code. 16 | The goal is to allow developers to write code as fast as possible without caring about performance. 17 | Rails scales; it's just a matter of writing the correct code. 18 | 19 | *SuperFastRails* automatically improves the requests in your Rails application. 20 | Thus, we focus only on the business logic and don't have to think about indexes, 1+n queries, dangerous migrations, etc. 21 | 22 | For the first version, *SuperFastRails* takes good care of the database layer. 23 | We want to keep adding more automatic optimizations in the future. 24 | Here is the list of the current automatic optimizations. 25 | 26 | 27 | ## Create automatically missing indexes 28 | 29 | Let's see the following query: 30 | 31 | ```sql 32 | SELECT * FROM projects WHERE projects.user_id = ? 33 | ``` 34 | 35 | Without an index on user_id, the query planner scans the entire table, which is slow because it must go through all rows. 36 | By enabling the following option: 37 | 38 | ```ruby 39 | SuperFastRails.create_missing_indexes = 500.in_milliseconds 40 | ``` 41 | 42 | If the query takes longer than 500ms, SuperFastRails analyzes the query plan with `EXPLAIN` to detect if an index is missing. 43 | In that case, it creates it to match the condition in the `where` clause. 44 | Thus, when creating a new table in a migration, we don't have to think about which indexes to make because it does so automatically when the application is running. 45 | 46 | 47 | ## Remove unused indexes 48 | 49 | Indexes speed up reads, but they slow down writes. 50 | So, unused indexes are not good and should be removed. 51 | By enabling the following options, it will automatically get rid of unused indexes: 52 | 53 | ```ruby 54 | SuperFastRails.remove_unused_indexes = true 55 | ``` 56 | 57 | The combination of the two options `create_missing_indexes` and `remove_unused_indexes` ensures that the database always has relevant indexes, even if the where clauses change. 58 | 59 | 60 | ## Optimise automatically SQL 61 | 62 | Indexes are essential, but SQL also needs to be written correctly. 63 | For example, the following query: 64 | 65 | ```sql 66 | SELECT * 67 | FROM users 68 | WHERE (SELECT count(*) FROM projects WHERE projects.user_id = users.id) > 1 69 | ``` 70 | 71 | is slower than: 72 | 73 | ```sql 74 | SELECT * 75 | FROM users 76 | WHERE EXISTS (SELECT 1 FROM projects WHERE projects.user_id = users.id) 77 | ``` 78 | 79 | Because it stops once it finds a row instead of keep counting. 80 | There are a bunch of tricks to know about SQL. 81 | Learning them requires time, and we need to remember them. 82 | The option `SuperFastRails.auto_optimise_queries = true` parses the SQL, and rewrites it before sending it to the database. 83 | Thanks to it, we don't have to care about all these tricks. 84 | 85 | You can also use it with a block if you prefer to enable it on a smaller scope: 86 | 87 | ```ruby 88 | SuperFastRails.optimise_queries do 89 | User.where("(SELECT count(*) FROM projects WHERE user_id = users.id) > 1") 90 | # SELECT * 91 | # FROM users 92 | # WHERE EXISTS (SELECT 1 FROM projects WHERE user_id = users.id) 93 | end 94 | 95 | ``` 96 | 97 | 98 | ## Get rid of 1+N queries 99 | 100 | The 1+N queries problem happens when iterating through a collection where the same query is repeated. 101 | For the following example: 102 | 103 | ```ruby 104 | Project.all.each do |project| 105 | puts "#{project.name} by #{project.user.name}" 106 | end 107 | ``` 108 | 109 | It triggers one extra query for each project: 110 | 111 | ```sql 112 | SELECT * FROM projects 113 | SELECT * FROM users WHERE users.id = ? 114 | SELECT * FROM users WHERE users.id = ? 115 | SELECT * FROM users WHERE users.id = ? 116 | SELECT * FROM users WHERE users.id = ? 117 | SELECT * FROM users WHERE users.id = ? 118 | SELECT * FROM users WHERE users.id = ? 119 | -- And so on 120 | ``` 121 | 122 | To solve this, SuperFastRails adds a method `each_without_1_plus_n_queries` to ActiveRecord relations. 123 | It detects when two identical queries are triggered to load all the missing data in a single query. 124 | 125 | ```ruby 126 | Project.all.each_without_1_plus_n_queries do |project| 127 | puts "#{project.name} by #{project.user.name}" 128 | end 129 | ``` 130 | 131 | ```sql 132 | SELECT * FROM projects 133 | SELECT * FROM users WHERE users.id = ? 134 | -- Before the 2nd repetition, superFastRails detects the 1+N pattern. 135 | -- So it loads all relevant users in a single query. 136 | SELECT * FROM users WHERE users.id IN (?) 137 | -- No more queries 138 | ``` 139 | 140 | 141 | 142 | ## Protect against dangerous migrations 143 | 144 | There are already many gems that help to detect dangerous migrations. 145 | They are great, but we still have to do the work manually. 146 | For example, renaming a column must be achieved in many steps: 147 | 148 | 1. Create a new column 149 | 2. Backfill values 150 | 3. Synchronizing both columns 151 | 4. Switching the code to the new column 152 | 5. Ignoring the old column 153 | 6. Removing the old column 154 | 155 | All this is tiring and wasting time. 156 | Thanks to SuperFastRails, it can be achieved in a single step with only one line: 157 | 158 | ```ruby 159 | SuperFastRails.rename_column :table, :old_name, :new_name 160 | ``` 161 | 162 | It takes care of creating the new columns, backfilling, and switching them. 163 | 164 | 165 | ## Tune database settings 166 | 167 | A database must be tuned to efficiently use the hardware. 168 | PostgreSQL should be told how much RAM it can use for the cache, the wall size, and the working memory. 169 | It's the same for SQLite, where the journal mode, page size, and so on can be changed. 170 | 171 | Currently, there is no other way to do that manually. 172 | That's why we added a method that tunes perfectly the settings: 173 | 174 | ```ruby 175 | SuperFastRails.tune_database! 176 | ``` 177 | 178 | It modifies the settings according to many parameters such as the database hardware, the ratio of reads and writes, the number of concurrent connections, etc. 179 | 180 | 181 | ## Install 182 | 183 | Install the gem right today and speed up your app automatically: 184 | 185 | ```ruby 186 | gem "super_fast_rails" 187 | ``` 188 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "bundler/gem_tasks" 4 | task default: %i[] 5 | -------------------------------------------------------------------------------- /bin/console: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | require "bundler/setup" 5 | require "super_fast_rails" 6 | 7 | # You can add fixtures and/or initialization code here to make experimenting 8 | # with your gem easier. You can also use a different console, if you like. 9 | 10 | # (If you use this, don't forget to add pry to your Gemfile!) 11 | # require "pry" 12 | # Pry.start 13 | 14 | require "irb" 15 | IRB.start(__FILE__) 16 | -------------------------------------------------------------------------------- /bin/setup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -euo pipefail 3 | IFS=$'\n\t' 4 | set -vx 5 | 6 | bundle install 7 | 8 | # Do any other automated setup that you need to do here 9 | -------------------------------------------------------------------------------- /lib/super_fast_rails.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require_relative "super_fast_rails/version" 4 | 5 | module SuperFastRails 6 | class Error < StandardError; end 7 | def self.create_missing_indexes=(value) 8 | end 9 | end 10 | -------------------------------------------------------------------------------- /lib/super_fast_rails/version.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module SuperFastRails 4 | VERSION = "0.1.0" 5 | end 6 | -------------------------------------------------------------------------------- /rorvswild_logo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BaseSecrete/super_fast_rails/bbb90bc49ac91e395da1a6b60cd1449ca11f95ae/rorvswild_logo.jpg -------------------------------------------------------------------------------- /super_fast_rails.gemspec: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require_relative "lib/super_fast_rails/version" 4 | 5 | Gem::Specification.new do |spec| 6 | spec.name = "super_fast_rails" 7 | spec.version = SuperFastRails::VERSION 8 | spec.authors = ["Alexis Bernard"] 9 | spec.email = ["alexis@basesecrete.com"] 10 | 11 | spec.summary = "Optimise automatically Rails apps" 12 | spec.description = "Create automatically missing index, remove unused, get rid of 1+N queries, ..." 13 | spec.homepage = "https://www.rorvswild.com" 14 | spec.license = "MIT" 15 | spec.required_ruby_version = ">= 2.4.0" 16 | 17 | 18 | spec.metadata["homepage_uri"] = spec.homepage 19 | spec.metadata["source_code_uri"] = "https://github.com/BaseSecrete/super_fast_rails" 20 | 21 | # Specify which files should be added to the gem when it is released. 22 | # The `git ls-files -z` loads the files in the RubyGem that have been added into git. 23 | spec.files = Dir.chdir(File.expand_path(__dir__)) do 24 | `git ls-files -z`.split("\x0").reject { |f| f.match(%r{\A(?:test|spec|features)/}) } 25 | end 26 | spec.bindir = "exe" 27 | spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) } 28 | spec.require_paths = ["lib"] 29 | 30 | spec.post_install_message = "MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM 31 | MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNNNXXXNNNWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM 32 | MMMMMMMMMMMMMMMMMMMMMMWWNNWWMMMMWNXK00OOOOOOO0KKXWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM 33 | MMMMMMMMMMMMMMMMMMWK0kkxxxxkO0KXK0OOOkkxddddddxkO0KNMMMMMMMMMMMMMMMMMMMMMMMMMMMM 34 | MMMMMMMMMMMMMMMMWKkdddddooooooxOOkkxdol:::clllccoxO0XWMMMMMMMMMMMMMMMMMMMMMMMMMM 35 | MMMMMMMMMMMMMMMW0dddollllllllloxxxdlc::oxOKXNXKOxllxOXWMMMMMMMMMMMMMMMMMMMMMMMMM 36 | MMMMMMMMMMMMMMMXxddllc::;;;::cldxdc:;cOWMMMMMMMMWXkloOXWMMMMMMMMMMMMMMMMMMMMMMMM 37 | MMMMMMMMMMMMMWWKxdoc:;,,,,,,,;cooc:;;kWMMMMMMMMMMMWXxxKNWMMMMMMMMMMMMMMMMMMMMMMM 38 | MMMMMMMMWNXKKKKOxdl:,,'''.''',,cc;;;;kWMMMMMMMMMMMMMNKKXXXWMMMMMMMMMMMMMMMMMMMMM 39 | MMMMMMNXK000OOOkddl;,''',,''''',;;;;;oXMMMMMMMMMMMMWXK0OKNWMMMMMMMMMMMMMMMMMMMMM 40 | MMMMWX00OOxdolllloo:'.';;;;;,'.';:c;;;dKKXWMMMMMMMMMWWNKNWMMMMMMMMMMMMMWWWMMMMMM 41 | MMMWK0Okdlc::::::coc,',;;;;;;'',,cxkdclxxOXWMMMMMMMMMMMMMMMMMMMMMWWWMNkddONMMMMM 42 | MMWK0kdlccldddoc;;ll;',;;:ll:;cdlco0XOkkkkONMMMMMMMMMMMMMMMMMMMNOxdxXKoccxNMMMMM 43 | MMN0kdccd0NWMMWXkc:lc,';;l0KOddxkkOKK000OOOXWMMMMMMMMMMMMMMMMMMXdccl0KoccxNMMMMM 44 | MWXOxl:xNMMMMMMMMXd:l:,:clOXXX0kkxkNXKNXKKKXWWXXXNMNOxxk0OxxxOXWN00XWKoccxNMMMMM 45 | MMXOd:lKMMMMMMMMMMNxokO0OxOKOxolccl0MMNxclloxdllloOKdccclcccccdXXKKKNKlccxNMMMMM 46 | MMN0d:lKMMMMMMMMMMWX0KKKKK0occcccccxNMXd:cccoxdlcclkdcccokkoccoKxcclKKlccxNMMMMM 47 | MMMXkcc0MMMMMMMMMMWNKKKOKNkcccllcccoKMXd:cclOWNxcclxdccl0MWkccoKxccl0KlccxNMMMMM 48 | MMMWXd:xNMMMMMMMMMMMW0okNKocccx0dccckWNd:ccl0MWOcclxdccdXMWOlcdKxccl0KlccxNMMMMM 49 | MMMMWXO0XNWMMMMMMMMMMN0XWkcccl0WOcccoKNd:ccl0MM0cclkdccdXMMNXXXXxccl0KlccxNMMMMM 50 | MMMMWXKKKKNWMMMMMMMMMMMMXocccdXWKoccckXd:ccl0MM0lclkdccdXMMMMMMNxccl0KlccxNMMMMM 51 | MMMMWXXKOXWMMMMMMMMMMMMWOccccldxdlccco0d:ccl0MMOcclkdccoXMMMMMMNxccl0KoccxNMMMMM 52 | MMMMMMWWWWMMMMMMMMMMMMMXdccccllllccccldo:ccl0MWkcclkdccoKMMMMMMNxccl0KoccxNMMMMM 53 | MMMMMMMMMMMMMMMMMMMMMMWOlccckKXXXOocccccccccxXKdccoOdcclKMMMMMMNxccl0KoccxNMMMMM 54 | MMMMMMMMMMMMMMMMMMMMMMXxclcdXMMMMWkccccccccccllcccdKdccl0MMMMMMNxccl0KoccxNMMMMM 55 | MMMMMMMMMMMMMMMMMMMMMMWXKKKXWMMMMMXxodolccclddlloxKNkoodKMMMMMMWOoodKXxookNMMMMM 56 | MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWNNKd:ccoKWXXNWMMWN0xxx0WMMMMWNNWWWWWWWMMMMMM 57 | MMMMMMMMMMMMMMMMMMMWNXXXXXXXXNWMMMMMMMXd:ccoKMMMMMMMMMOc::xXOxx0WMMMMMMMMMMMMMMM 58 | MMMMMMMMMMMMMMMMMMMXdclllllllxNMMMMWWWNkloodXMMWWWMMMMOc::xXd::xNMMMWWWWMMMMMMMM 59 | MMMMMMMMMMMMMMMMMMMXo:cccccc:dNWKkxdddkKXNNXKkxdddxKWWOc::xXd::xNWKkdoodkXWMMMMM 60 | MMMMMMMMMMMMMMMMMMMXo:c:oO000XNOc:lddc:okXWOl:cddl:l0WOc::xXd::xNOc:cc:::l0WMMMM 61 | MMMMMMMMMMMMMMMMMMMXo:c:dNMMMMXo:ckWNx:ccONd::dNWkc:xNOcc:xXd:cx0o:clddlc:lKMMMM 62 | MMMMMMMMMMMMMMMMMMMXo:c:dNMMMMKl:c0MWkc::xKd::xNMOc:dXOcc:xNX0KXOc:ckWWkc:ckWMMM 63 | MMMMMMMMMMMMMMMMMMMXo:c:lxxkXMKo:c0MWkc::xKd::xNMOc:dXOcc:xNMMMWkc:cOMWOccckWMMM 64 | MMMMMMMMMMMMMMMMMMMXo:cccc:l0MXo:c0MWkc::xXd::xNMOc:dXOc::xNMMMWOc:cxNW0dddKMMMM 65 | MMMMMMMMMMMMMMMMMMMXo:c:o0KKNMXo:c0MWkc::xKd::xNMOc:oXOc::xNMMMMKo:ccoOKNWWMMMMM 66 | MMMMMMMMMMMMMMMMMMMXo:c:dNMMMMKo:l0MWOc::xKd:ckWM0c:oXOcc:xNMMMMWXkdoc:cd0WMMMMM 67 | MMMMMMMMMMMMMMMMMMMXo:c:dNMMMMKo:cOMWkc::xKo:cxNWOc:oXOcc:xNMMMMNOOO0Kxc:cOWMMMM 68 | MMMMMMMMMMMMMMMMMMMXo:c:dNMMMMKo:cokxl:::xKd:clxxlc:oXOc::xNMMMM0l:cxKOl::dNMMMM 69 | MMMMMMMMMMMMMMMMMMMXo:c:dNMMMMNkc::c:::ld0WOc:c::c:cOWOcc:xNMMMMXd:cclcc:cOWMMMM 70 | MMMMMMMMMMMMMMMMMMMXdclcxNMMMMMN0xdddxkXWWMWKkddddkKWM0oolkWMMMMWXxlcccld0WMMMMM 71 | MMMMMMMMMMMMMMMMMMMWNXXXNWW0xxxxddoodkXWMMMMMMWWWWMMMMWNNNWMMMMMMMWXK0KXWMMMMMMM 72 | MMMMMMMMMMMMMMMMMMMMMMMMMMNo,;;;;;;;;;lKMMMMWNXKKXWMMMMMMMMMMMMMMMMMMMMMMMMMMMMM 73 | MMMMMMMMMMMMMMMMMMMMMMMMMMXo;;;:ooc;;;;kWWWKdl:::clkNMMWXKKNMMMXkxxONMMMMMMMMMMM 74 | MMMMMMMMMMMMMMMMMMMMMMMMMMXo;;;oNWKl;;;xXN0c;;;;;;;;dNMXo::xWMWk;;;xWMMMMMMMMMMM 75 | MMMMMMMMMMMMMMMMMMMMMMMMMMXl;;;dNMNo;;;xNXo;;;col:;;:OMWk:;lXMXo;;c0MMMMMMMMMMMM 76 | MMMMMMMMMMMMMMMMMMMMMMMMMMXl;;;dWMNd;;;kW0c;;c0WNx;;;dNM0c;:OW0c;;oXMMMMMMMMMMMM 77 | MMMMMMMMMMMMMMMMMMMMMMMMMMXl;;;xWMWd;;;kW0c;;lKMM0c;;dNMNd;;oXx;;;kWMMMMMMMMMMMM 78 | MMMMMMMMMMMMMMMMMMMMMMMMMMXl;;;dNMWd;;;kM0c;;cKMM0:;;oNMMO:,cdl;;cKMMMMMMMMMMMMM 79 | MMMMMMMMMMMMMMMMMMMMMMMMMMXo;;;dNMNd;;;kW0l::lKWKo;;;dNMMXo;;:;;;dNMMMMMMMMMMMMM 80 | MMMMMMMMMMMMMMMMMMMMMMMMMMXo;;;dNMNd;;;xNNX0Oxdo:;;;;dNMMWO:;;;;:OMMMMMMMMMMMMMM 81 | MMMMMMMMMMMMMMMMMMMMMMMMMMXo;;;dNMXo;;;xWN0ocokOxc;;;xWMMMXl;;;;lKMMMMMMMMMMMMMM 82 | MMMMMMMMMMMMMMMMMMMMMMMMMMXo;;;lOOd:;;:kWO:,:OMMWk;;;kWMMMWk:;;;xNMMMMMMMMMMMMMM 83 | MMMMMMMMMMMMMMMMMMMMMMMMMMXo;;;;;;;;;;lKWO:;;cxOxl:;;kWMMMMNkc,c0MMMMMMMMMMMMMMM 84 | MMMMMMMMMMMMMMMMMMMMMMMMMMNkoddooooddx0WMNkl:;;co0Oc;kWNNNNXx:;oNMMMMMMMMMMMMMMM 85 | MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMNKKKXWMWX0XKocclc;;:OMMMMMMMMMMMMMMMM 86 | MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM0c,;;;;;oXMMMMMMMMMMMMMMMM 87 | MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMNOkkkkkOXWMMMMMMMMMMMMMMMM 88 | MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM 89 | MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM" 90 | 91 | # Uncomment to register a new dependency of your gem 92 | # spec.add_dependency "example-gem", "~> 1.0" 93 | 94 | # For more information and examples about making a new gem, checkout our 95 | # guide at: https://bundler.io/guides/creating_gem.html 96 | end 97 | --------------------------------------------------------------------------------